2024-05-08T23:22:48.898 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T23:22:48.898 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --verbose --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181 -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181/orig.config.yaml 2024-05-08T23:22:48.962 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T23:22:49.059 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698181' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIgYMpA+K7Xo2hfyxkoCe3MuebWycbrN3u6nXiCzFENLrl7khsFlvraZMHQD7zFkslGw+iLvb7cW8hukWoRl1ls= smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD+oE330yNl8F0ekvyf1zqWsMviGPhAVi9Iy586zVFTw3o4qdOeEM4r8E1ZgnjuA/un7wT7KjZI9ZrkZyoiJxPM= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPr6ZECMg47ucc05ArBTB2bSnRYwF/BjXs/yXUAlNPIwV5lftwJruqL/h9JO5AFn5guEbXalWn7dYQGep5eRoNM= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T23:22:49.060 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T23:22:49.060 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T23:22:49.061 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T23:22:49.062 INFO:teuthology.task.internal:Checking packages... 2024-05-08T23:22:49.094 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T23:22:49.094 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T23:22:49.095 INFO:teuthology.packaging:ref: None 2024-05-08T23:22:49.095 INFO:teuthology.packaging:tag: None 2024-05-08T23:22:49.095 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:22:49.095 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:22:49.095 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:22:49.326 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322.gd21e4f96 2024-05-08T23:22:49.327 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T23:22:49.334 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T23:22:49.335 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T23:22:49.372 INFO:teuthology.task.internal:Saving configuration 2024-05-08T23:22:49.397 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T23:22:49.414 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T23:22:49.435 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 23:17:16.414902', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIgYMpA+K7Xo2hfyxkoCe3MuebWycbrN3u6nXiCzFENLrl7khsFlvraZMHQD7zFkslGw+iLvb7cW8hukWoRl1ls='} 2024-05-08T23:22:49.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 23:17:16.413271', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD+oE330yNl8F0ekvyf1zqWsMviGPhAVi9Iy586zVFTw3o4qdOeEM4r8E1ZgnjuA/un7wT7KjZI9ZrkZyoiJxPM='} 2024-05-08T23:22:49.479 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 23:17:16.411332', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPr6ZECMg47ucc05ArBTB2bSnRYwF/BjXs/yXUAlNPIwV5lftwJruqL/h9JO5AFn5guEbXalWn7dYQGep5eRoNM='} 2024-05-08T23:22:49.479 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T23:22:49.486 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T23:22:49.486 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T23:22:49.486 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T23:22:49.486 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T23:22:49.602 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4511206160>, signals=[15]) 2024-05-08T23:22:49.603 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T23:22:49.610 INFO:teuthology.task.internal:Opening connections... 2024-05-08T23:22:49.611 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-05-08T23:22:49.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:22:49.690 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2024-05-08T23:22:49.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:22:49.765 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-05-08T23:22:49.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:22:49.841 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T23:22:49.848 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-05-08T23:22:49.871 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-05-08T23:22:49.872 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-05-08T23:22:49.934 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T23:22:49.934 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T23:22:49.934 INFO:teuthology.orchestra.run.smithi012.stdout:ID="rhel" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="fedora" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="8.6" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el8" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T23:22:49.935 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T23:22:49.936 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-05-08T23:22:49.959 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-05-08T23:22:49.980 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-05-08T23:22:49.980 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-05-08T23:22:50.040 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T23:22:50.040 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T23:22:50.040 INFO:teuthology.orchestra.run.smithi160.stdout:ID="rhel" 2024-05-08T23:22:50.040 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="fedora" 2024-05-08T23:22:50.040 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8.6" 2024-05-08T23:22:50.040 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2024-05-08T23:22:50.040 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T23:22:50.040 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-05-08T23:22:50.040 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T23:22:50.041 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T23:22:50.041 INFO:teuthology.orchestra.run.smithi160.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T23:22:50.041 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T23:22:50.041 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:22:50.041 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T23:22:50.041 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T23:22:50.041 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T23:22:50.041 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T23:22:50.041 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-05-08T23:22:50.093 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-05-08T23:22:50.114 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-05-08T23:22:50.114 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-05-08T23:22:50.172 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:ID="rhel" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="fedora" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="8.6" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el8" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T23:22:50.173 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T23:22:50.174 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:22:50.174 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T23:22:50.174 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T23:22:50.174 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T23:22:50.174 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T23:22:50.174 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-05-08T23:22:50.200 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T23:22:50.211 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T23:22:50.217 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T23:22:50.217 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:22:50.220 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:22:50.222 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:22:50.241 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T23:22:50.249 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T23:22:50.249 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:22:50.282 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:22:50.284 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:22:50.320 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T23:22:50.395 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T23:22:50.409 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-08T23:22:50.409 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-08T23:22:50.409 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-08T23:22:50.410 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:22:50.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:22:50.431 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-05-08T23:22:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:22:50.493 INFO:teuthology.task.kernel:Running kernel on smithi012: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:22:50.493 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum install -y kernel 2024-05-08T23:22:52.094 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2024-05-08T23:22:52.094 INFO:teuthology.orchestra.run.smithi012.stdout:Unable to read consumer identity 2024-05-08T23:22:52.360 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:22:52.360 INFO:teuthology.orchestra.run.smithi012.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:22:52.360 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:22:54.116 INFO:teuthology.orchestra.run.smithi012.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-05-08T23:22:55.808 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 16 MB 00:01 2024-05-08T23:22:59.883 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 444 kB/s | 24 kB 00:00 2024-05-08T23:23:00.677 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T23:23:00.677 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T23:23:00.754 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-05-08T23:23:00.755 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-05-08T23:23:00.755 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-05-08T23:23:00.917 DEBUG:teuthology.orchestra.run.smithi012:> echo no | sudo yum reinstall kernel || true 2024-05-08T23:23:01.305 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2024-05-08T23:23:01.306 INFO:teuthology.orchestra.run.smithi012.stdout:Unable to read consumer identity 2024-05-08T23:23:01.509 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:23:01.509 INFO:teuthology.orchestra.run.smithi012.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:23:01.510 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:23:01.871 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 May 2024 11:22:59 PM UTC. 2024-05-08T23:23:01.956 INFO:teuthology.orchestra.run.smithi012.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T23:23:01.962 INFO:teuthology.orchestra.run.smithi012.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T23:23:02.010 INFO:teuthology.orchestra.run.smithi012.stderr:Error: No packages marked for reinstall. 2024-05-08T23:23:02.066 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum reinstall -y kernel || true 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi012.stdout:Unable to read consumer identity 2024-05-08T23:23:02.822 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:23:02.822 INFO:teuthology.orchestra.run.smithi012.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:23:02.823 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:23:03.182 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:00:04 ago on Wed 08 May 2024 11:22:59 PM UTC. 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi012.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi012.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T23:23:03.289 INFO:teuthology.orchestra.run.smithi012.stderr:Error: No packages marked for reinstall. 2024-05-08T23:23:03.350 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T23:23:03.518 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:23:03.519 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:23:03.519 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T23:23:03.519 DEBUG:teuthology.parallel:result is None 2024-05-08T23:23:03.519 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T23:23:03.526 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T23:23:03.526 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:23:03.528 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:23:03.531 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:23:03.551 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T23:23:03.558 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T23:23:03.566 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T23:23:03.566 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:23:03.587 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:23:03.592 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:23:03.630 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T23:23:03.639 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T23:23:03.639 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:23:03.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:23:03.683 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:23:03.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:23:03.702 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:23:03.705 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:23:03.725 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:23:03.746 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:23:03.760 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:23:03.770 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:23:03.783 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:23:03.789 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:23:03.800 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:23:03.801 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T23:23:03.809 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T23:23:03.809 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:23:03.812 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:23:03.832 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:23:03.871 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T23:23:03.880 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T23:23:03.880 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:23:03.887 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:23:03.901 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:23:03.929 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:23:03.996 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:23:04.053 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:23:04.142 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:23:04.192 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:23:04.192 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:23:04.267 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:23:04.309 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:23:04.364 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:23:04.450 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:23:04.494 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:23:04.494 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:23:04.562 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:23:04.611 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:23:04.664 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:23:04.752 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:23:04.796 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:23:04.796 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:23:04.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-05-08T23:23:04.867 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2024-05-08T23:23:04.869 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-05-08T23:23:04.919 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:23:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:23:04.988 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:23:05.284 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T23:23:05.292 INFO:teuthology.task.internal:Starting timer... 2024-05-08T23:23:05.292 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T23:23:05.304 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T23:23:05.313 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2024-05-08T23:23:05.314 DEBUG:teuthology.orchestra.run.smithi012:> sudo service auditd rotate 2024-05-08T23:23:05.386 INFO:teuthology.orchestra.run.smithi012.stdout:Rotating logs: 2024-05-08T23:23:05.388 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2024-05-08T23:23:05.476 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2024-05-08T23:23:05.479 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-05-08T23:23:05.532 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-05-08T23:23:05.533 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T23:23:05.534 DEBUG:teuthology.orchestra.run.smithi012:> /usr/sbin/getenforce 2024-05-08T23:23:05.571 INFO:teuthology.orchestra.run.smithi012.stdout:Permissive 2024-05-08T23:23:05.572 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2024-05-08T23:23:05.593 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2024-05-08T23:23:05.593 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-05-08T23:23:05.617 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-05-08T23:23:05.618 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi012.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi160.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-05-08T23:23:05.618 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:23:05.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:23:05.652 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:23:05.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:23:05.682 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:23:05.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:23:05.710 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T23:23:05.710 DEBUG:teuthology.orchestra.run.smithi012:> sudo /usr/sbin/setenforce permissive 2024-05-08T23:23:05.741 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2024-05-08T23:23:05.770 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-05-08T23:23:05.798 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T23:23:05.809 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T23:23:05.815 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T23:23:05.946 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T23:23:05.969 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T23:23:05.971 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi160.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T23:33:55.009 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi160.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-05-08T23:33:55.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-08T23:33:55.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:33:55.097 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-05-08T23:33:55.172 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-08T23:33:55.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-08T23:33:55.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:33:55.254 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-05-08T23:33:55.325 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-08T23:33:55.326 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-08T23:33:55.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:33:55.401 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-05-08T23:33:55.468 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-08T23:33:55.468 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T23:33:55.481 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T23:33:55.481 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:33:55.482 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:33:55.484 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:33:55.484 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:33:55.486 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:33:55.486 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:33:55.520 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:33:55.520 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:33:55.535 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:33:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:33:55.542 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:33:55.557 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:33:55.614 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: ntpd: command not found 2024-05-08T23:33:55.618 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2024-05-08T23:33:55.627 INFO:teuthology.orchestra.run.smithi012.stdout:506 Cannot talk to daemon 2024-05-08T23:33:55.630 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2024-05-08T23:33:55.644 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-05-08T23:33:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:33:55.647 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:33:55.656 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-05-08T23:33:55.661 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:33:55.662 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:33:55.671 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:33:55.685 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:33:55.737 INFO:teuthology.orchestra.run.smithi160.stderr:bash: ntpq: command not found 2024-05-08T23:33:55.740 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:33:55.740 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-08T23:33:55.740 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.740 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.741 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.741 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.758 INFO:teuthology.orchestra.run.smithi204.stderr:bash: ntpq: command not found 2024-05-08T23:33:55.758 INFO:teuthology.orchestra.run.smithi012.stderr:bash: ntpq: command not found 2024-05-08T23:33:55.762 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:33:55.762 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-05-08T23:33:55.763 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.763 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.763 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.763 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.764 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:33:55.764 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-05-08T23:33:55.764 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.764 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.764 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.764 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:33:55.764 INFO:teuthology.run_tasks:Running task pexec... 2024-05-08T23:33:55.774 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-08T23:33:55.774 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi012.front.sepia.ceph.com 2024-05-08T23:33:55.775 DEBUG:teuthology.orchestra.run.smithi012:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T23:33:55.775 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi160.front.sepia.ceph.com 2024-05-08T23:33:55.775 DEBUG:teuthology.orchestra.run.smithi160:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T23:33:55.776 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi204.front.sepia.ceph.com 2024-05-08T23:33:55.776 DEBUG:teuthology.orchestra.run.smithi204:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T23:33:56.159 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2024-05-08T23:33:56.182 INFO:teuthology.orchestra.run.smithi204.stdout:Updating Subscription Management repositories. 2024-05-08T23:33:56.237 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2024-05-08T23:33:57.953 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-05-08T23:33:58.050 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-05-08T23:33:58.119 INFO:teuthology.orchestra.run.smithi012.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 kB/s | 4.1 kB 00:00 2024-05-08T23:33:58.520 INFO:teuthology.orchestra.run.smithi012.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-05-08T23:33:58.606 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-05-08T23:33:58.649 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-05-08T23:34:01.570 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-08T23:34:01.571 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-08T23:34:01.571 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-05-08T23:34:01.572 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-08T23:34:01.572 INFO:teuthology.orchestra.run.smithi160.stdout:Resetting modules: 2024-05-08T23:34:01.572 INFO:teuthology.orchestra.run.smithi160.stdout: container-tools 2024-05-08T23:34:01.572 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:34:01.572 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-08T23:34:01.572 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-08T23:34:01.572 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:34:01.748 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-05-08T23:34:01.749 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-05-08T23:34:01.749 INFO:teuthology.orchestra.run.smithi204.stdout: Package Architecture Version Repository Size 2024-05-08T23:34:01.749 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-05-08T23:34:01.749 INFO:teuthology.orchestra.run.smithi204.stdout:Resetting modules: 2024-05-08T23:34:01.749 INFO:teuthology.orchestra.run.smithi204.stdout: container-tools 2024-05-08T23:34:01.749 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:34:01.749 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-05-08T23:34:01.749 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-05-08T23:34:01.749 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:34:01.957 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-05-08T23:34:01.959 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-05-08T23:34:01.959 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-05-08T23:34:01.959 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-05-08T23:34:01.959 INFO:teuthology.orchestra.run.smithi012.stdout:Resetting modules: 2024-05-08T23:34:01.959 INFO:teuthology.orchestra.run.smithi012.stdout: container-tools 2024-05-08T23:34:01.959 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:34:01.959 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-05-08T23:34:01.959 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-05-08T23:34:01.959 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:34:12.297 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-05-08T23:34:12.798 INFO:teuthology.orchestra.run.smithi204.stdout:Updating Subscription Management repositories. 2024-05-08T23:34:14.331 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-08T23:34:14.366 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-05-08T23:34:14.672 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 28 kB/s | 4.5 kB 00:00 2024-05-08T23:34:14.810 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2024-05-08T23:34:14.932 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2024-05-08T23:34:15.300 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-05-08T23:34:16.507 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 kB/s | 4.5 kB 00:00 2024-05-08T23:34:16.698 INFO:teuthology.orchestra.run.smithi012.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-05-08T23:34:17.098 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 kB/s | 4.1 kB 00:00 2024-05-08T23:34:17.170 INFO:teuthology.orchestra.run.smithi012.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 19 kB/s | 4.5 kB 00:00 2024-05-08T23:34:18.156 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-05-08T23:34:18.321 INFO:teuthology.orchestra.run.smithi204.stdout:========================================================================================================== 2024-05-08T23:34:18.321 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-05-08T23:34:18.321 INFO:teuthology.orchestra.run.smithi204.stdout:========================================================================================================== 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout:Installing group/module packages: 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T23:34:18.322 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout:Installing module profiles: 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout: container-tools/common 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout:Enabling module streams: 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout: container-tools rhel8 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout:========================================================================================================== 2024-05-08T23:34:18.323 INFO:teuthology.orchestra.run.smithi204.stdout:Install 13 Packages 2024-05-08T23:34:18.324 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:34:18.324 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 26 M 2024-05-08T23:34:18.324 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 87 M 2024-05-08T23:34:18.324 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-05-08T23:34:18.492 INFO:teuthology.orchestra.run.smithi204.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 148 kB/s | 25 kB 00:00 2024-05-08T23:34:18.517 INFO:teuthology.orchestra.run.smithi204.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 204 kB/s | 39 kB 00:00 2024-05-08T23:34:18.542 INFO:teuthology.orchestra.run.smithi204.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 431 kB/s | 94 kB 00:00 2024-05-08T23:34:18.609 INFO:teuthology.orchestra.run.smithi204.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 349 kB/s | 41 kB 00:00 2024-05-08T23:34:18.676 INFO:teuthology.orchestra.run.smithi204.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.4 MB/s | 1.0 MB 00:00 2024-05-08T23:34:18.751 INFO:teuthology.orchestra.run.smithi204.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 5.1 MB/s | 739 kB 00:00 2024-05-08T23:34:18.785 INFO:teuthology.orchestra.run.smithi204.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 2.2 MB/s | 239 kB 00:00 2024-05-08T23:34:18.911 INFO:teuthology.orchestra.run.smithi204.stdout:(8/13): buildah-1.31.5-1.module+el8.9.0+21697+6 24 MB/s | 8.8 MB 00:00 2024-05-08T23:34:18.936 INFO:teuthology.orchestra.run.smithi204.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.0 MB/s | 154 kB 00:00 2024-05-08T23:34:19.028 INFO:teuthology.orchestra.run.smithi204.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 13 MB/s | 3.7 MB 00:00 2024-05-08T23:34:19.170 INFO:teuthology.orchestra.run.smithi204.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 11 MB/s | 2.5 MB 00:00 2024-05-08T23:34:19.194 INFO:teuthology.orchestra.run.smithi204.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 297 kB/s | 49 kB 00:00 2024-05-08T23:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 20 MB/s | 8.1 MB 00:00 2024-05-08T23:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:Total 26 MB/s | 26 MB 00:00 2024-05-08T23:34:19.604 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-05-08T23:34:19.625 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-05-08T23:34:19.625 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-05-08T23:34:19.856 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-05-08T23:34:19.865 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-05-08T23:34:19.967 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-08T23:34:20.132 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================================== 2024-05-08T23:34:20.132 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-08T23:34:20.132 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================================== 2024-05-08T23:34:20.132 INFO:teuthology.orchestra.run.smithi160.stdout:Installing group/module packages: 2024-05-08T23:34:20.132 INFO:teuthology.orchestra.run.smithi160.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T23:34:20.133 INFO:teuthology.orchestra.run.smithi160.stdout:Installing weak dependencies: 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout:Installing module profiles: 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout: container-tools/common 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling module streams: 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout: container-tools rhel8 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================================== 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout:Install 13 Packages 2024-05-08T23:34:20.134 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:34:20.135 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 26 M 2024-05-08T23:34:20.135 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 87 M 2024-05-08T23:34:20.135 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-08T23:34:20.277 INFO:teuthology.orchestra.run.smithi160.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 277 kB/s | 39 kB 00:00 2024-05-08T23:34:20.302 INFO:teuthology.orchestra.run.smithi160.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 148 kB/s | 25 kB 00:00 2024-05-08T23:34:20.328 INFO:teuthology.orchestra.run.smithi160.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 488 kB/s | 94 kB 00:00 2024-05-08T23:34:20.403 INFO:teuthology.orchestra.run.smithi160.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 326 kB/s | 41 kB 00:00 2024-05-08T23:34:20.445 INFO:teuthology.orchestra.run.smithi160.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 7.1 MB/s | 1.0 MB 00:00 2024-05-08T23:34:20.554 INFO:teuthology.orchestra.run.smithi160.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 4.8 MB/s | 739 kB 00:00 2024-05-08T23:34:20.636 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-05-08T23:34:20.696 INFO:teuthology.orchestra.run.smithi160.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 24 MB/s | 8.8 MB 00:00 2024-05-08T23:34:20.721 INFO:teuthology.orchestra.run.smithi160.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 868 kB/s | 239 kB 00:00 2024-05-08T23:34:20.737 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-05-08T23:34:20.823 INFO:teuthology.orchestra.run.smithi012.stdout:========================================================================================================== 2024-05-08T23:34:20.823 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-05-08T23:34:20.823 INFO:teuthology.orchestra.run.smithi012.stdout:========================================================================================================== 2024-05-08T23:34:20.823 INFO:teuthology.orchestra.run.smithi012.stdout:Installing group/module packages: 2024-05-08T23:34:20.823 INFO:teuthology.orchestra.run.smithi012.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T23:34:20.823 INFO:teuthology.orchestra.run.smithi012.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T23:34:20.823 INFO:teuthology.orchestra.run.smithi012.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout:Installing weak dependencies: 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout:Installing module profiles: 2024-05-08T23:34:20.824 INFO:teuthology.orchestra.run.smithi012.stdout: container-tools/common 2024-05-08T23:34:20.825 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:34:20.825 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling module streams: 2024-05-08T23:34:20.825 INFO:teuthology.orchestra.run.smithi012.stdout: container-tools rhel8 2024-05-08T23:34:20.825 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:34:20.825 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-05-08T23:34:20.825 INFO:teuthology.orchestra.run.smithi012.stdout:========================================================================================================== 2024-05-08T23:34:20.825 INFO:teuthology.orchestra.run.smithi012.stdout:Install 13 Packages 2024-05-08T23:34:20.825 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:34:20.826 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 26 M 2024-05-08T23:34:20.826 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 87 M 2024-05-08T23:34:20.826 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-05-08T23:34:20.871 INFO:teuthology.orchestra.run.smithi160.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 12 MB/s | 3.7 MB 00:00 2024-05-08T23:34:20.897 INFO:teuthology.orchestra.run.smithi160.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 766 kB/s | 154 kB 00:00 2024-05-08T23:34:20.984 INFO:teuthology.orchestra.run.smithi012.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 158 kB/s | 25 kB 00:00 2024-05-08T23:34:21.009 INFO:teuthology.orchestra.run.smithi012.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 216 kB/s | 39 kB 00:00 2024-05-08T23:34:21.034 INFO:teuthology.orchestra.run.smithi012.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 452 kB/s | 94 kB 00:00 2024-05-08T23:34:21.085 INFO:teuthology.orchestra.run.smithi012.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 408 kB/s | 41 kB 00:00 2024-05-08T23:34:21.122 INFO:teuthology.orchestra.run.smithi160.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 20 MB/s | 8.1 MB 00:00 2024-05-08T23:34:21.139 INFO:teuthology.orchestra.run.smithi160.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 204 kB/s | 49 kB 00:00 2024-05-08T23:34:21.185 INFO:teuthology.orchestra.run.smithi012.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 5.8 MB/s | 1.0 MB 00:00 2024-05-08T23:34:21.215 INFO:teuthology.orchestra.run.smithi160.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 7.4 MB/s | 2.5 MB 00:00 2024-05-08T23:34:21.215 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:34:21.215 INFO:teuthology.orchestra.run.smithi160.stdout:Total 24 MB/s | 26 MB 00:01 2024-05-08T23:34:21.245 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T23:34:21.260 INFO:teuthology.orchestra.run.smithi012.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 4.1 MB/s | 739 kB 00:00 2024-05-08T23:34:21.402 INFO:teuthology.orchestra.run.smithi012.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 24 MB/s | 8.8 MB 00:00 2024-05-08T23:34:21.408 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T23:34:21.428 INFO:teuthology.orchestra.run.smithi012.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 989 kB/s | 239 kB 00:00 2024-05-08T23:34:21.486 INFO:teuthology.orchestra.run.smithi012.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.8 MB/s | 154 kB 00:00 2024-05-08T23:34:21.497 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-08T23:34:21.518 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-08T23:34:21.518 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-08T23:34:21.545 INFO:teuthology.orchestra.run.smithi012.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 13 MB/s | 3.7 MB 00:00 2024-05-08T23:34:21.569 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T23:34:21.678 INFO:teuthology.orchestra.run.smithi012.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 13 MB/s | 2.5 MB 00:00 2024-05-08T23:34:21.704 INFO:teuthology.orchestra.run.smithi012.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 312 kB/s | 49 kB 00:00 2024-05-08T23:34:21.737 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T23:34:21.746 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-08T23:34:21.755 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-08T23:34:21.804 INFO:teuthology.orchestra.run.smithi012.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 22 MB/s | 8.1 MB 00:00 2024-05-08T23:34:21.804 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:34:21.805 INFO:teuthology.orchestra.run.smithi012.stdout:Total 26 MB/s | 26 MB 00:00 2024-05-08T23:34:21.916 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T23:34:22.108 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-05-08T23:34:22.125 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T23:34:22.132 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-05-08T23:34:22.132 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-05-08T23:34:22.344 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T23:34:22.389 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-05-08T23:34:22.399 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-05-08T23:34:22.531 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-08T23:34:23.024 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T23:34:23.194 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T23:34:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-05-08T23:34:23.199 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:34:23.381 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T23:34:23.413 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:34:23.575 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T23:34:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T23:34:23.785 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T23:34:23.786 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T23:34:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T23:34:24.022 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T23:34:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T23:34:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T23:34:24.290 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T23:34:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T23:34:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T23:34:24.685 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T23:34:24.873 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T23:34:24.883 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T23:34:25.039 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:34:25.201 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:34:25.465 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:34:25.465 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T23:34:25.465 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T23:34:25.465 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T23:34:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T23:34:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T23:34:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T23:34:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T23:34:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:34:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:34:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T23:34:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T23:34:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T23:34:25.466 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:34:25.836 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:34:25.890 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T23:34:26.050 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:34:26.472 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T23:34:26.786 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T23:34:27.011 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T23:34:27.210 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:34:27.433 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T23:34:27.622 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T23:34:27.666 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:34:27.666 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T23:34:27.666 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T23:34:27.666 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T23:34:27.666 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T23:34:27.666 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T23:34:27.666 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T23:34:27.666 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T23:34:27.667 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:34:27.667 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:34:27.667 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T23:34:27.667 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T23:34:27.667 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T23:34:27.827 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:34:28.275 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:34:28.275 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T23:34:28.275 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T23:34:28.275 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T23:34:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T23:34:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T23:34:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T23:34:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T23:34:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:34:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:34:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T23:34:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T23:34:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T23:34:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T23:34:34.882 INFO:teuthology.orchestra.run.smithi204.stdout:Installed products updated. 2024-05-08T23:34:35.451 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:34:35.451 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-05-08T23:34:35.451 INFO:teuthology.orchestra.run.smithi204.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:35.451 INFO:teuthology.orchestra.run.smithi204.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:35.451 INFO:teuthology.orchestra.run.smithi204.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:34:35.451 INFO:teuthology.orchestra.run.smithi204.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T23:34:35.451 INFO:teuthology.orchestra.run.smithi204.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:35.451 INFO:teuthology.orchestra.run.smithi204.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:35.452 INFO:teuthology.orchestra.run.smithi204.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:34:35.452 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T23:34:35.452 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T23:34:35.452 INFO:teuthology.orchestra.run.smithi204.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:35.452 INFO:teuthology.orchestra.run.smithi204.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:35.452 INFO:teuthology.orchestra.run.smithi204.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:34:35.452 INFO:teuthology.orchestra.run.smithi204.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T23:34:35.452 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:34:35.452 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-05-08T23:34:35.624 DEBUG:teuthology.parallel:result is None 2024-05-08T23:34:39.749 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T23:34:39.749 INFO:teuthology.orchestra.run.smithi160.stdout:Installed products updated. 2024-05-08T23:34:40.316 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:34:40.316 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-08T23:34:40.316 INFO:teuthology.orchestra.run.smithi160.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:40.316 INFO:teuthology.orchestra.run.smithi160.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:40.316 INFO:teuthology.orchestra.run.smithi160.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:34:40.316 INFO:teuthology.orchestra.run.smithi160.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T23:34:40.316 INFO:teuthology.orchestra.run.smithi160.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:40.317 INFO:teuthology.orchestra.run.smithi160.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:40.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:34:40.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T23:34:40.317 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T23:34:40.317 INFO:teuthology.orchestra.run.smithi160.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:40.317 INFO:teuthology.orchestra.run.smithi160.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:40.317 INFO:teuthology.orchestra.run.smithi160.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:34:40.317 INFO:teuthology.orchestra.run.smithi160.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T23:34:40.317 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:34:40.317 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-08T23:34:40.494 DEBUG:teuthology.parallel:result is None 2024-05-08T23:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T23:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout:Installed products updated. 2024-05-08T23:34:41.119 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:34:41.119 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-05-08T23:34:41.119 INFO:teuthology.orchestra.run.smithi012.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:41.119 INFO:teuthology.orchestra.run.smithi012.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:41.119 INFO:teuthology.orchestra.run.smithi012.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:34:41.120 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-05-08T23:34:41.309 DEBUG:teuthology.parallel:result is None 2024-05-08T23:34:41.310 INFO:teuthology.run_tasks:Running task install... 2024-05-08T23:34:41.318 DEBUG:teuthology.task.install:project ceph 2024-05-08T23:34:41.318 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T23:34:41.318 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T23:34:41.319 INFO:teuthology.task.install:Using flavor: default 2024-05-08T23:34:41.324 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T23:34:41.324 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T23:34:41.325 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T23:34:41.325 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:34:41.327 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T23:34:41.327 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:34:41.328 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T23:34:41.329 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:34:41.474 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T23:34:41.474 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T23:34:41.478 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T23:34:41.479 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T23:34:41.553 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T23:34:41.554 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T23:34:41.590 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:34:41.590 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:34:41.590 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:34:41.623 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:34:41.624 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:34:41.646 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:34:41.646 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:34:41.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:34:41.681 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:34:41.682 DEBUG:teuthology.orchestra.run.smithi012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:34:41.695 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:34:41.718 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:34:41.718 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:34:41.718 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:34:41.752 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:34:41.753 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:34:41.757 DEBUG:teuthology.orchestra.run.smithi012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:34:41.777 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:34:41.823 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:34:41.845 DEBUG:teuthology.orchestra.run.smithi012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:34:41.845 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2024-05-08T23:34:41.847 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2024-05-08T23:34:41.877 INFO:teuthology.orchestra.run.smithi012.stdout:check_obsoletes = 1 2024-05-08T23:34:41.878 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum clean all 2024-05-08T23:34:41.904 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:34:41.972 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-05-08T23:34:41.974 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-05-08T23:34:42.226 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2024-05-08T23:34:42.316 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2024-05-08T23:34:42.352 INFO:teuthology.orchestra.run.smithi204.stdout:Updating Subscription Management repositories. 2024-05-08T23:34:43.656 INFO:teuthology.orchestra.run.smithi160.stdout:63 files removed 2024-05-08T23:34:43.707 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:34:43.785 INFO:teuthology.orchestra.run.smithi204.stdout:63 files removed 2024-05-08T23:34:43.814 INFO:teuthology.orchestra.run.smithi012.stdout:63 files removed 2024-05-08T23:34:43.837 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:34:43.869 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:34:44.044 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2024-05-08T23:34:44.176 INFO:teuthology.orchestra.run.smithi204.stdout:Updating Subscription Management repositories. 2024-05-08T23:34:44.257 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2024-05-08T23:34:46.019 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 237 kB/s | 80 kB 00:00 2024-05-08T23:34:46.127 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 187 kB/s | 80 kB 00:00 2024-05-08T23:34:46.278 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 51 kB/s | 11 kB 00:00 2024-05-08T23:34:46.316 INFO:teuthology.orchestra.run.smithi012.stdout:ceph packages for x86_64 221 kB/s | 80 kB 00:00 2024-05-08T23:34:46.519 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-05-08T23:34:46.621 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 4.7 kB/s | 1.5 kB 00:00 2024-05-08T23:34:46.683 INFO:teuthology.orchestra.run.smithi012.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-05-08T23:34:46.771 INFO:teuthology.orchestra.run.smithi204.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-05-08T23:34:46.878 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2024-05-08T23:34:46.977 INFO:teuthology.orchestra.run.smithi012.stdout:ceph source packages 5.7 kB/s | 1.5 kB 00:00 2024-05-08T23:34:47.071 INFO:teuthology.orchestra.run.smithi160.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-05-08T23:34:47.135 INFO:teuthology.orchestra.run.smithi012.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-05-08T23:34:47.788 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2024-05-08T23:34:47.856 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2024-05-08T23:34:48.371 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2024-05-08T23:34:51.363 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 383 kB/s | 24 kB 00:00 2024-05-08T23:34:51.431 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 396 kB/s | 24 kB 00:00 2024-05-08T23:34:52.435 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 309 kB/s | 24 kB 00:00 2024-05-08T23:34:52.466 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 9.1 MB/s | 8.8 MB 00:00 2024-05-08T23:34:52.657 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 MB/s | 61 MB 00:01 2024-05-08T23:34:53.137 INFO:teuthology.orchestra.run.smithi012.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-08T23:34:55.334 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 61 MB 00:01 2024-05-08T23:34:56.388 INFO:teuthology.orchestra.run.smithi012.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 MB/s | 67 MB 00:01 2024-05-08T23:35:04.208 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 67 MB 00:01 2024-05-08T23:35:06.802 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 67 MB 00:01 2024-05-08T23:35:08.529 INFO:teuthology.orchestra.run.smithi012.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 61 MB 00:01 2024-05-08T23:35:14.633 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 16 MB/s | 8.8 MB 00:00 2024-05-08T23:35:21.655 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:35:21.657 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:35:21.675 INFO:teuthology.orchestra.run.smithi160.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T23:35:21.676 INFO:teuthology.orchestra.run.smithi160.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T23:35:21.678 INFO:teuthology.orchestra.run.smithi160.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T23:35:21.679 INFO:teuthology.orchestra.run.smithi160.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T23:35:21.682 INFO:teuthology.orchestra.run.smithi160.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T23:35:21.686 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T23:35:21.687 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T23:35:21.823 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:35:21.824 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:35:21.842 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T23:35:21.843 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T23:35:21.844 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T23:35:21.846 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T23:35:21.848 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T23:35:21.852 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T23:35:21.853 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T23:35:21.906 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-08T23:35:21.920 INFO:teuthology.orchestra.run.smithi160.stdout:=================================================================================================================================== 2024-05-08T23:35:21.920 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-08T23:35:21.920 INFO:teuthology.orchestra.run.smithi160.stdout:=================================================================================================================================== 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T23:35:21.921 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T23:35:21.922 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T23:35:21.923 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T23:35:21.924 INFO:teuthology.orchestra.run.smithi160.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T23:35:21.925 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T23:35:21.926 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T23:35:21.926 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T23:35:21.926 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T23:35:21.926 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T23:35:21.926 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T23:35:21.926 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T23:35:21.926 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T23:35:21.926 INFO:teuthology.orchestra.run.smithi160.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T23:35:21.926 INFO:teuthology.orchestra.run.smithi160.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T23:35:21.927 INFO:teuthology.orchestra.run.smithi160.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T23:35:21.928 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T23:35:21.929 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T23:35:21.930 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T23:35:21.930 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T23:35:21.930 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T23:35:21.930 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T23:35:21.930 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T23:35:21.930 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T23:35:21.930 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:35:21.930 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T23:35:21.930 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T23:35:21.930 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T23:35:21.931 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T23:35:21.932 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T23:35:21.933 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T23:35:21.934 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling module streams: 2024-05-08T23:35:21.934 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql 10 2024-05-08T23:35:21.934 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:35:21.934 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-08T23:35:21.934 INFO:teuthology.orchestra.run.smithi160.stdout:=================================================================================================================================== 2024-05-08T23:35:21.934 INFO:teuthology.orchestra.run.smithi160.stdout:Install 130 Packages 2024-05-08T23:35:21.934 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 16 Packages 2024-05-08T23:35:21.934 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:35:21.935 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 215 M 2024-05-08T23:35:21.935 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-08T23:35:22.070 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-05-08T23:35:22.084 INFO:teuthology.orchestra.run.smithi204.stdout:=================================================================================================================================== 2024-05-08T23:35:22.084 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-05-08T23:35:22.084 INFO:teuthology.orchestra.run.smithi204.stdout:=================================================================================================================================== 2024-05-08T23:35:22.084 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-05-08T23:35:22.084 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T23:35:22.084 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T23:35:22.084 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T23:35:22.084 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T23:35:22.084 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T23:35:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T23:35:22.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T23:35:22.087 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-05-08T23:35:22.088 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T23:35:22.089 INFO:teuthology.orchestra.run.smithi204.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T23:35:22.090 INFO:teuthology.orchestra.run.smithi204.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T23:35:22.091 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T23:35:22.092 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T23:35:22.093 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:35:22.094 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T23:35:22.095 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout:Enabling module streams: 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql 10 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout:=================================================================================================================================== 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout:Install 130 Packages 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 16 Packages 2024-05-08T23:35:22.096 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:35:22.097 INFO:teuthology.orchestra.run.smithi160.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 52 kB/s | 6.4 kB 00:00 2024-05-08T23:35:22.097 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 215 M 2024-05-08T23:35:22.097 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-05-08T23:35:22.275 INFO:teuthology.orchestra.run.smithi160.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 4.6 MB/s | 860 kB 00:00 2024-05-08T23:35:22.342 INFO:teuthology.orchestra.run.smithi160.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 2.4 MB/s | 166 kB 00:00 2024-05-08T23:35:22.377 INFO:teuthology.orchestra.run.smithi204.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2024-05-08T23:35:22.751 INFO:teuthology.orchestra.run.smithi160.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 5.6 MB/s | 2.3 MB 00:00 2024-05-08T23:35:23.226 INFO:teuthology.orchestra.run.smithi160.stdout:(5/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 4.4 MB/s | 5.5 MB 00:01 2024-05-08T23:35:23.259 INFO:teuthology.orchestra.run.smithi160.stdout:(6/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 3.1 MB/s | 1.6 MB 00:00 2024-05-08T23:35:23.852 INFO:teuthology.orchestra.run.smithi204.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 583 kB/s | 860 kB 00:01 2024-05-08T23:35:24.186 INFO:teuthology.orchestra.run.smithi204.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 498 kB/s | 166 kB 00:00 2024-05-08T23:35:24.243 INFO:teuthology.orchestra.run.smithi160.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 4.8 MB/s | 4.8 MB 00:01 2024-05-08T23:35:24.286 INFO:teuthology.orchestra.run.smithi204.stdout:(4/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 2.6 MB/s | 5.5 MB 00:02 2024-05-08T23:35:24.485 INFO:teuthology.orchestra.run.smithi160.stdout:(8/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 8.6 MB/s | 22 MB 00:02 2024-05-08T23:35:24.543 INFO:teuthology.orchestra.run.smithi160.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 419 kB/s | 24 kB 00:00 2024-05-08T23:35:24.770 INFO:teuthology.orchestra.run.smithi204.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 3.2 MB/s | 1.6 MB 00:00 2024-05-08T23:35:25.828 INFO:teuthology.orchestra.run.smithi204.stdout:(6/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 1.4 MB/s | 2.3 MB 00:01 2024-05-08T23:35:25.980 INFO:teuthology.orchestra.run.smithi012.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:35:25.981 INFO:teuthology.orchestra.run.smithi012.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:35:26.002 INFO:teuthology.orchestra.run.smithi012.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T23:35:26.004 INFO:teuthology.orchestra.run.smithi012.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T23:35:26.005 INFO:teuthology.orchestra.run.smithi012.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T23:35:26.007 INFO:teuthology.orchestra.run.smithi012.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T23:35:26.010 INFO:teuthology.orchestra.run.smithi012.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T23:35:26.015 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T23:35:26.016 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T23:35:26.119 INFO:teuthology.orchestra.run.smithi160.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 6.4 MB/s | 18 MB 00:02 2024-05-08T23:35:26.275 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-05-08T23:35:26.291 INFO:teuthology.orchestra.run.smithi012.stdout:=================================================================================================================================== 2024-05-08T23:35:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout:=================================================================================================================================== 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T23:35:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T23:35:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T23:35:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrading: 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T23:35:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T23:35:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T23:35:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T23:35:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T23:35:26.299 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T23:35:26.299 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T23:35:26.299 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T23:35:26.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T23:35:26.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T23:35:26.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T23:35:26.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T23:35:26.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T23:35:26.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T23:35:26.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T23:35:26.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T23:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T23:35:26.303 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling module streams: 2024-05-08T23:35:26.304 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql 10 2024-05-08T23:35:26.304 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:35:26.304 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-05-08T23:35:26.304 INFO:teuthology.orchestra.run.smithi012.stdout:=================================================================================================================================== 2024-05-08T23:35:26.304 INFO:teuthology.orchestra.run.smithi012.stdout:Install 130 Packages 2024-05-08T23:35:26.304 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrade 16 Packages 2024-05-08T23:35:26.304 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:35:26.305 INFO:teuthology.orchestra.run.smithi160.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 1.4 MB/s | 261 kB 00:00 2024-05-08T23:35:26.305 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 215 M 2024-05-08T23:35:26.305 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-05-08T23:35:26.378 INFO:teuthology.orchestra.run.smithi160.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 416 kB/s | 31 kB 00:00 2024-05-08T23:35:26.503 INFO:teuthology.orchestra.run.smithi204.stdout:(7/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 5.0 MB/s | 22 MB 00:04 2024-05-08T23:35:26.645 INFO:teuthology.orchestra.run.smithi160.stdout:(13/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 5.5 MB/s | 13 MB 00:02 2024-05-08T23:35:26.678 INFO:teuthology.orchestra.run.smithi160.stdout:(14/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.5 MB/s | 782 kB 00:00 2024-05-08T23:35:26.745 INFO:teuthology.orchestra.run.smithi160.stdout:(15/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.4 MB/s | 142 kB 00:00 2024-05-08T23:35:26.795 INFO:teuthology.orchestra.run.smithi160.stdout:(16/146): librados-devel-18.2.2-1322.gd21e4f96. 1.2 MB/s | 139 kB 00:00 2024-05-08T23:35:26.845 INFO:teuthology.orchestra.run.smithi204.stdout:(8/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 2.3 MB/s | 4.8 MB 00:02 2024-05-08T23:35:26.870 INFO:teuthology.orchestra.run.smithi160.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 3.7 MB/s | 474 kB 00:00 2024-05-08T23:35:26.945 INFO:teuthology.orchestra.run.smithi204.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 243 kB/s | 24 kB 00:00 2024-05-08T23:35:26.960 INFO:teuthology.orchestra.run.smithi012.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2024-05-08T23:35:26.979 INFO:teuthology.orchestra.run.smithi160.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 412 kB/s | 45 kB 00:00 2024-05-08T23:35:27.454 INFO:teuthology.orchestra.run.smithi160.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 271 kB/s | 129 kB 00:00 2024-05-08T23:35:27.596 INFO:teuthology.orchestra.run.smithi160.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 1.4 MB/s | 197 kB 00:00 2024-05-08T23:35:27.797 INFO:teuthology.orchestra.run.smithi160.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 1.8 MB/s | 370 kB 00:00 2024-05-08T23:35:27.980 INFO:teuthology.orchestra.run.smithi160.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 1.9 MB/s | 365 kB 00:00 2024-05-08T23:35:28.044 INFO:teuthology.orchestra.run.smithi012.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 794 kB/s | 860 kB 00:01 2024-05-08T23:35:28.064 INFO:teuthology.orchestra.run.smithi160.stdout:(23/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 5.2 MB/s | 6.5 MB 00:01 2024-05-08T23:35:28.080 INFO:teuthology.orchestra.run.smithi160.stdout:(24/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.0 MB/s | 107 kB 00:00 2024-05-08T23:35:28.148 INFO:teuthology.orchestra.run.smithi160.stdout:(25/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.1 MB/s | 95 kB 00:00 2024-05-08T23:35:28.256 INFO:teuthology.orchestra.run.smithi160.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.6 MB/s | 181 kB 00:00 2024-05-08T23:35:28.311 INFO:teuthology.orchestra.run.smithi012.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 622 kB/s | 166 kB 00:00 2024-05-08T23:35:28.348 INFO:teuthology.orchestra.run.smithi160.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 265 kB/s | 24 kB 00:00 2024-05-08T23:35:28.448 INFO:teuthology.orchestra.run.smithi160.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.4 MB/s | 138 kB 00:00 2024-05-08T23:35:29.015 INFO:teuthology.orchestra.run.smithi160.stdout:(29/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 3.7 MB/s | 3.5 MB 00:00 2024-05-08T23:35:29.441 INFO:teuthology.orchestra.run.smithi160.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 3.6 MB/s | 3.5 MB 00:00 2024-05-08T23:35:29.445 INFO:teuthology.orchestra.run.smithi204.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 5.1 MB/s | 18 MB 00:03 2024-05-08T23:35:29.554 INFO:teuthology.orchestra.run.smithi204.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 2.4 MB/s | 261 kB 00:00 2024-05-08T23:35:29.636 INFO:teuthology.orchestra.run.smithi160.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 1.2 MB/s | 245 kB 00:00 2024-05-08T23:35:29.663 INFO:teuthology.orchestra.run.smithi204.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 285 kB/s | 31 kB 00:00 2024-05-08T23:35:29.724 INFO:teuthology.orchestra.run.smithi160.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 557 kB/s | 49 kB 00:00 2024-05-08T23:35:29.813 INFO:teuthology.orchestra.run.smithi204.stdout:(13/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 4.0 MB/s | 13 MB 00:03 2024-05-08T23:35:29.833 INFO:teuthology.orchestra.run.smithi160.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 151 kB/s | 16 kB 00:00 2024-05-08T23:35:29.905 INFO:teuthology.orchestra.run.smithi204.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.5 MB/s | 142 kB 00:00 2024-05-08T23:35:29.941 INFO:teuthology.orchestra.run.smithi160.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 2.4 MB/s | 263 kB 00:00 2024-05-08T23:35:29.963 INFO:teuthology.orchestra.run.smithi204.stdout:(15/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.5 MB/s | 782 kB 00:00 2024-05-08T23:35:29.997 INFO:teuthology.orchestra.run.smithi204.stdout:(16/146): librados-devel-18.2.2-1322.gd21e4f96. 1.5 MB/s | 139 kB 00:00 2024-05-08T23:35:30.042 INFO:teuthology.orchestra.run.smithi160.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.7 MB/s | 170 kB 00:00 2024-05-08T23:35:30.134 INFO:teuthology.orchestra.run.smithi160.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 197 kB/s | 18 kB 00:00 2024-05-08T23:35:30.147 INFO:teuthology.orchestra.run.smithi204.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.5 MB/s | 474 kB 00:00 2024-05-08T23:35:30.248 INFO:teuthology.orchestra.run.smithi204.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 446 kB/s | 45 kB 00:00 2024-05-08T23:35:30.340 INFO:teuthology.orchestra.run.smithi204.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.4 MB/s | 129 kB 00:00 2024-05-08T23:35:30.448 INFO:teuthology.orchestra.run.smithi204.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 1.8 MB/s | 197 kB 00:00 2024-05-08T23:35:30.451 INFO:teuthology.orchestra.run.smithi160.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 968 kB/s | 306 kB 00:00 2024-05-08T23:35:30.522 INFO:teuthology.orchestra.run.smithi160.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 883 kB/s | 59 kB 00:00 2024-05-08T23:35:30.550 INFO:teuthology.orchestra.run.smithi204.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 3.6 MB/s | 370 kB 00:00 2024-05-08T23:35:30.585 INFO:teuthology.orchestra.run.smithi160.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2024-05-08T23:35:30.594 INFO:teuthology.orchestra.run.smithi012.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 1.0 MB/s | 2.3 MB 00:02 2024-05-08T23:35:30.649 INFO:teuthology.orchestra.run.smithi204.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 3.6 MB/s | 365 kB 00:00 2024-05-08T23:35:30.669 INFO:teuthology.orchestra.run.smithi160.stdout:(40/146): ceph-mgr-diskprediction-local-18.2.2- 4.5 MB/s | 7.4 MB 00:01 2024-05-08T23:35:30.702 INFO:teuthology.orchestra.run.smithi160.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2024-05-08T23:35:30.733 INFO:teuthology.orchestra.run.smithi204.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.3 MB/s | 107 kB 00:00 2024-05-08T23:35:30.802 INFO:teuthology.orchestra.run.smithi160.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2024-05-08T23:35:30.824 INFO:teuthology.orchestra.run.smithi204.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.0 MB/s | 95 kB 00:00 2024-05-08T23:35:30.886 INFO:teuthology.orchestra.run.smithi160.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 398 kB/s | 33 kB 00:00 2024-05-08T23:35:30.953 INFO:teuthology.orchestra.run.smithi160.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 245 kB/s | 16 kB 00:00 2024-05-08T23:35:30.978 INFO:teuthology.orchestra.run.smithi160.stdout:(45/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 141 kB/s | 44 kB 00:00 2024-05-08T23:35:31.052 INFO:teuthology.orchestra.run.smithi160.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2024-05-08T23:35:31.121 INFO:teuthology.orchestra.run.smithi160.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2024-05-08T23:35:31.154 INFO:teuthology.orchestra.run.smithi160.stdout:(48/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2024-05-08T23:35:31.179 INFO:teuthology.orchestra.run.smithi160.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2024-05-08T23:35:31.221 INFO:teuthology.orchestra.run.smithi160.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2024-05-08T23:35:31.313 INFO:teuthology.orchestra.run.smithi160.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 532 kB/s | 49 kB 00:00 2024-05-08T23:35:31.347 INFO:teuthology.orchestra.run.smithi160.stdout:(52/146): python3-kubernetes-11.0.0-6.el8.noarc 7.0 MB/s | 1.2 MB 00:00 2024-05-08T23:35:31.381 INFO:teuthology.orchestra.run.smithi160.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 878 kB/s | 59 kB 00:00 2024-05-08T23:35:31.414 INFO:teuthology.orchestra.run.smithi160.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 900 kB/s | 60 kB 00:00 2024-05-08T23:35:31.448 INFO:teuthology.orchestra.run.smithi160.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2024-05-08T23:35:31.473 INFO:teuthology.orchestra.run.smithi160.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2024-05-08T23:35:31.540 INFO:teuthology.orchestra.run.smithi160.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 366 kB/s | 34 kB 00:00 2024-05-08T23:35:31.542 INFO:teuthology.orchestra.run.smithi204.stdout:(25/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 4.8 MB/s | 3.5 MB 00:00 2024-05-08T23:35:31.625 INFO:teuthology.orchestra.run.smithi204.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 2.1 MB/s | 181 kB 00:00 2024-05-08T23:35:31.750 INFO:teuthology.orchestra.run.smithi204.stdout:(27/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 3.7 MB/s | 6.5 MB 00:01 2024-05-08T23:35:31.809 INFO:teuthology.orchestra.run.smithi204.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1322.g 132 kB/s | 24 kB 00:00 2024-05-08T23:35:31.834 INFO:teuthology.orchestra.run.smithi204.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.6 MB/s | 138 kB 00:00 2024-05-08T23:35:32.007 INFO:teuthology.orchestra.run.smithi160.stdout:(58/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 6.3 MB/s | 47 MB 00:07 2024-05-08T23:35:32.027 INFO:teuthology.orchestra.run.smithi012.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 1.1 MB/s | 1.6 MB 00:01 2024-05-08T23:35:32.069 INFO:teuthology.orchestra.run.smithi160.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 330 kB/s | 196 kB 00:00 2024-05-08T23:35:32.099 INFO:teuthology.orchestra.run.smithi160.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 115 kB/s | 64 kB 00:00 2024-05-08T23:35:32.141 INFO:teuthology.orchestra.run.smithi160.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 263 kB/s | 19 kB 00:00 2024-05-08T23:35:32.166 INFO:teuthology.orchestra.run.smithi160.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 437 kB/s | 29 kB 00:00 2024-05-08T23:35:32.208 INFO:teuthology.orchestra.run.smithi160.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 413 kB/s | 27 kB 00:00 2024-05-08T23:35:32.267 INFO:teuthology.orchestra.run.smithi160.stdout:(64/146): python3-singledispatch-3.4.0.3-18.el8 121 kB/s | 24 kB 00:00 2024-05-08T23:35:32.325 INFO:teuthology.orchestra.run.smithi160.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 1.6 MB/s | 255 kB 00:00 2024-05-08T23:35:32.342 INFO:teuthology.orchestra.run.smithi160.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2024-05-08T23:35:32.367 INFO:teuthology.orchestra.run.smithi160.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 610 kB/s | 61 kB 00:00 2024-05-08T23:35:32.384 INFO:teuthology.orchestra.run.smithi160.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2024-05-08T23:35:32.401 INFO:teuthology.orchestra.run.smithi160.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 390 kB/s | 23 kB 00:00 2024-05-08T23:35:32.418 INFO:teuthology.orchestra.run.smithi160.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.0 MB/s | 70 kB 00:00 2024-05-08T23:35:32.427 INFO:teuthology.orchestra.run.smithi012.stdout:(6/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 977 kB/s | 5.5 MB 00:05 2024-05-08T23:35:32.468 INFO:teuthology.orchestra.run.smithi204.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 5.4 MB/s | 3.5 MB 00:00 2024-05-08T23:35:32.560 INFO:teuthology.orchestra.run.smithi160.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 126 kB/s | 20 kB 00:00 2024-05-08T23:35:32.585 INFO:teuthology.orchestra.run.smithi204.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 2.1 MB/s | 245 kB 00:00 2024-05-08T23:35:32.586 INFO:teuthology.orchestra.run.smithi160.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 263 kB/s | 44 kB 00:00 2024-05-08T23:35:32.636 INFO:teuthology.orchestra.run.smithi160.stdout:(73/146): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2024-05-08T23:35:32.653 INFO:teuthology.orchestra.run.smithi160.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 398 kB/s | 27 kB 00:00 2024-05-08T23:35:32.678 INFO:teuthology.orchestra.run.smithi160.stdout:(75/146): python3-pyasn1-modules-0.3.7-6.el8.no 937 kB/s | 110 kB 00:00 2024-05-08T23:35:32.693 INFO:teuthology.orchestra.run.smithi204.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 454 kB/s | 49 kB 00:00 2024-05-08T23:35:32.794 INFO:teuthology.orchestra.run.smithi204.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 163 kB/s | 16 kB 00:00 2024-05-08T23:35:32.803 INFO:teuthology.orchestra.run.smithi160.stdout:(76/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.0 MB/s | 457 kB 00:00 2024-05-08T23:35:32.854 INFO:teuthology.orchestra.run.smithi160.stdout:(77/146): bison-3.0.4-10.el8.x86_64.rpm 4.0 MB/s | 688 kB 00:00 2024-05-08T23:35:32.879 INFO:teuthology.orchestra.run.smithi160.stdout:(78/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 521 kB/s | 126 kB 00:00 2024-05-08T23:35:32.902 INFO:teuthology.orchestra.run.smithi204.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 2.4 MB/s | 263 kB 00:00 2024-05-08T23:35:32.929 INFO:teuthology.orchestra.run.smithi160.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 2.5 MB/s | 320 kB 00:00 2024-05-08T23:35:33.011 INFO:teuthology.orchestra.run.smithi204.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.5 MB/s | 170 kB 00:00 2024-05-08T23:35:33.021 INFO:teuthology.orchestra.run.smithi160.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 273 kB/s | 39 kB 00:00 2024-05-08T23:35:33.055 INFO:teuthology.orchestra.run.smithi160.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 3.4 MB/s | 709 kB 00:00 2024-05-08T23:35:33.097 INFO:teuthology.orchestra.run.smithi160.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.5 MB/s | 259 kB 00:00 2024-05-08T23:35:33.119 INFO:teuthology.orchestra.run.smithi204.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 167 kB/s | 18 kB 00:00 2024-05-08T23:35:33.172 INFO:teuthology.orchestra.run.smithi160.stdout:(83/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 886 kB/s | 103 kB 00:00 2024-05-08T23:35:33.197 INFO:teuthology.orchestra.run.smithi160.stdout:(84/146): uuid-1.6.2-43.el8.x86_64.rpm 364 kB/s | 64 kB 00:00 2024-05-08T23:35:33.211 INFO:teuthology.orchestra.run.smithi204.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 3.3 MB/s | 306 kB 00:00 2024-05-08T23:35:33.245 INFO:teuthology.orchestra.run.smithi204.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2024-05-08T23:35:33.279 INFO:teuthology.orchestra.run.smithi204.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 2.2 MB/s | 75 kB 00:00 2024-05-08T23:35:33.331 INFO:teuthology.orchestra.run.smithi160.stdout:(85/146): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2024-05-08T23:35:33.337 INFO:teuthology.orchestra.run.smithi204.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 8.0 MB/s | 476 kB 00:00 2024-05-08T23:35:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.7 MB/s | 538 kB 00:00 2024-05-08T23:35:33.371 INFO:teuthology.orchestra.run.smithi204.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2024-05-08T23:35:33.404 INFO:teuthology.orchestra.run.smithi204.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 5.5 MB/s | 185 kB 00:00 2024-05-08T23:35:33.430 INFO:teuthology.orchestra.run.smithi204.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 1.3 MB/s | 33 kB 00:00 2024-05-08T23:35:33.464 INFO:teuthology.orchestra.run.smithi204.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 488 kB/s | 16 kB 00:00 2024-05-08T23:35:33.471 INFO:teuthology.orchestra.run.smithi160.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2024-05-08T23:35:33.497 INFO:teuthology.orchestra.run.smithi204.stdout:(45/146): python3-cheroot-8.5.2-1.el8.noarch.rp 5.1 MB/s | 173 kB 00:00 2024-05-08T23:35:33.506 INFO:teuthology.orchestra.run.smithi160.stdout:(88/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2024-05-08T23:35:33.532 INFO:teuthology.orchestra.run.smithi160.stdout:(89/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2024-05-08T23:35:33.539 INFO:teuthology.orchestra.run.smithi204.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 9.1 MB/s | 384 kB 00:00 2024-05-08T23:35:33.573 INFO:teuthology.orchestra.run.smithi204.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 2.4 MB/s | 81 kB 00:00 2024-05-08T23:35:33.590 INFO:teuthology.orchestra.run.smithi160.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.9 MB/s | 354 kB 00:00 2024-05-08T23:35:33.607 INFO:teuthology.orchestra.run.smithi204.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 332 kB/s | 11 kB 00:00 2024-05-08T23:35:33.641 INFO:teuthology.orchestra.run.smithi160.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2024-05-08T23:35:33.641 INFO:teuthology.orchestra.run.smithi204.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 529 kB/s | 18 kB 00:00 2024-05-08T23:35:33.721 INFO:teuthology.orchestra.run.smithi204.stdout:(50/146): python3-kubernetes-11.0.0-6.el8.noarc 15 MB/s | 1.2 MB 00:00 2024-05-08T23:35:33.757 INFO:teuthology.orchestra.run.smithi160.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 21 MB/s | 5.2 MB 00:00 2024-05-08T23:35:33.758 INFO:teuthology.orchestra.run.smithi204.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 1.3 MB/s | 49 kB 00:00 2024-05-08T23:35:33.791 INFO:teuthology.orchestra.run.smithi204.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 1.7 MB/s | 59 kB 00:00 2024-05-08T23:35:33.799 INFO:teuthology.orchestra.run.smithi160.stdout:(93/146): automake-1.16.1-8.el8.noarch.rpm 4.4 MB/s | 714 kB 00:00 2024-05-08T23:35:33.825 INFO:teuthology.orchestra.run.smithi204.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2024-05-08T23:35:33.859 INFO:teuthology.orchestra.run.smithi204.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 8.3 MB/s | 283 kB 00:00 2024-05-08T23:35:33.908 INFO:teuthology.orchestra.run.smithi160.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 16 MB/s | 5.0 MB 00:00 2024-05-08T23:35:33.925 INFO:teuthology.orchestra.run.smithi160.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2024-05-08T23:35:33.942 INFO:teuthology.orchestra.run.smithi204.stdout:(55/146): ceph-mgr-diskprediction-local-18.2.2- 3.5 MB/s | 7.4 MB 00:02 2024-05-08T23:35:33.968 INFO:teuthology.orchestra.run.smithi204.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2024-05-08T23:35:33.987 INFO:teuthology.orchestra.run.smithi160.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2024-05-08T23:35:33.993 INFO:teuthology.orchestra.run.smithi204.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 672 kB/s | 34 kB 00:00 2024-05-08T23:35:34.018 INFO:teuthology.orchestra.run.smithi204.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 2.5 MB/s | 64 kB 00:00 2024-05-08T23:35:34.052 INFO:teuthology.orchestra.run.smithi204.stdout:(59/146): python3-simplegeneric-0.8.1-17.el8.no 560 kB/s | 19 kB 00:00 2024-05-08T23:35:34.059 INFO:teuthology.orchestra.run.smithi160.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 361 kB/s | 54 kB 00:00 2024-05-08T23:35:34.077 INFO:teuthology.orchestra.run.smithi204.stdout:(60/146): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2024-05-08T23:35:34.102 INFO:teuthology.orchestra.run.smithi204.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 478 kB/s | 24 kB 00:00 2024-05-08T23:35:34.117 INFO:teuthology.orchestra.run.smithi160.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-05-08T23:35:34.128 INFO:teuthology.orchestra.run.smithi204.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2024-05-08T23:35:34.153 INFO:teuthology.orchestra.run.smithi204.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 550 kB/s | 27 kB 00:00 2024-05-08T23:35:34.178 INFO:teuthology.orchestra.run.smithi204.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2024-05-08T23:35:34.195 INFO:teuthology.orchestra.run.smithi204.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 5.9 MB/s | 251 kB 00:00 2024-05-08T23:35:34.221 INFO:teuthology.orchestra.run.smithi204.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2024-05-08T23:35:34.246 INFO:teuthology.orchestra.run.smithi204.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2024-05-08T23:35:34.318 INFO:teuthology.orchestra.run.smithi160.stdout:(99/146): python3-scipy-1.0.0-21.module+el8.9.0 35 MB/s | 14 MB 00:00 2024-05-08T23:35:34.338 INFO:teuthology.orchestra.run.smithi204.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2024-05-08T23:35:34.360 INFO:teuthology.orchestra.run.smithi160.stdout:(100/146): postgresql-contrib-10.23-4.module+el 3.3 MB/s | 812 kB 00:00 2024-05-08T23:35:34.401 INFO:teuthology.orchestra.run.smithi160.stdout:(101/146): postgresql-10.23-4.module+el8.9.0+21 4.4 MB/s | 1.5 MB 00:00 2024-05-08T23:35:34.568 INFO:teuthology.orchestra.run.smithi160.stdout:(102/146): postgresql-server-10.23-4.module+el8 20 MB/s | 5.1 MB 00:00 2024-05-08T23:35:34.605 INFO:teuthology.orchestra.run.smithi204.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 261 kB/s | 70 kB 00:00 2024-05-08T23:35:34.647 INFO:teuthology.orchestra.run.smithi204.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 4.3 MB/s | 1.7 MB 00:00 2024-05-08T23:35:34.789 INFO:teuthology.orchestra.run.smithi204.stdout:(71/146): python3-packaging-16.8-9.el8.noarch.r 262 kB/s | 48 kB 00:00 2024-05-08T23:35:34.844 INFO:teuthology.orchestra.run.smithi160.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 37 kB/s | 18 kB 00:00 2024-05-08T23:35:34.848 INFO:teuthology.orchestra.run.smithi204.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 102 kB/s | 20 kB 00:00 2024-05-08T23:35:34.881 INFO:teuthology.orchestra.run.smithi204.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 479 kB/s | 44 kB 00:00 2024-05-08T23:35:35.003 INFO:teuthology.orchestra.run.smithi160.stdout:(104/146): python3-jwt-1.6.1-2.el8.noarch.rpm 71 kB/s | 43 kB 00:00 2024-05-08T23:35:35.032 INFO:teuthology.orchestra.run.smithi204.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 177 kB/s | 27 kB 00:00 2024-05-08T23:35:35.036 INFO:teuthology.orchestra.run.smithi160.stdout:(105/146): python3-oauthlib-2.1.0-1.el8.noarch. 331 kB/s | 155 kB 00:00 2024-05-08T23:35:35.107 INFO:teuthology.orchestra.run.smithi204.stdout:(75/146): python3-pyasn1-modules-0.3.7-6.el8.no 425 kB/s | 110 kB 00:00 2024-05-08T23:35:35.141 INFO:teuthology.orchestra.run.smithi204.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-05-08T23:35:35.203 INFO:teuthology.orchestra.run.smithi160.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 304 kB/s | 109 kB 00:00 2024-05-08T23:35:35.216 INFO:teuthology.orchestra.run.smithi204.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 4.1 MB/s | 457 kB 00:00 2024-05-08T23:35:35.253 INFO:teuthology.orchestra.run.smithi160.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 170 kB/s | 43 kB 00:00 2024-05-08T23:35:35.304 INFO:teuthology.orchestra.run.smithi160.stdout:(108/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 438 kB/s | 44 kB 00:00 2024-05-08T23:35:35.329 INFO:teuthology.orchestra.run.smithi160.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 64 kB/s | 19 kB 00:00 2024-05-08T23:35:35.333 INFO:teuthology.orchestra.run.smithi204.stdout:(78/146): bison-3.0.4-10.el8.x86_64.rpm 3.5 MB/s | 688 kB 00:00 2024-05-08T23:35:35.366 INFO:teuthology.orchestra.run.smithi204.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2024-05-08T23:35:35.492 INFO:teuthology.orchestra.run.smithi204.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 311 kB/s | 39 kB 00:00 2024-05-08T23:35:35.526 INFO:teuthology.orchestra.run.smithi204.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 3.6 MB/s | 709 kB 00:00 2024-05-08T23:35:35.530 INFO:teuthology.orchestra.run.smithi160.stdout:(110/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.1 MB/s | 605 kB 00:00 2024-05-08T23:35:35.563 INFO:teuthology.orchestra.run.smithi160.stdout:(111/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 884 kB/s | 229 kB 00:00 2024-05-08T23:35:35.597 INFO:teuthology.orchestra.run.smithi160.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 825 kB/s | 220 kB 00:00 2024-05-08T23:35:35.600 INFO:teuthology.orchestra.run.smithi204.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.3 MB/s | 259 kB 00:00 2024-05-08T23:35:35.651 INFO:teuthology.orchestra.run.smithi204.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 510 kB/s | 64 kB 00:00 2024-05-08T23:35:35.668 INFO:teuthology.orchestra.run.smithi204.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.5 MB/s | 103 kB 00:00 2024-05-08T23:35:35.755 INFO:teuthology.orchestra.run.smithi160.stdout:(113/146): libselinux-devel-2.9-5.el8.x86_64.rp 886 kB/s | 200 kB 00:00 2024-05-08T23:35:35.772 INFO:teuthology.orchestra.run.smithi160.stdout:(114/146): libsepol-devel-2.9-3.el8.x86_64.rpm 496 kB/s | 87 kB 00:00 2024-05-08T23:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 1.3 MB/s | 4.8 MB 00:03 2024-05-08T23:35:35.785 INFO:teuthology.orchestra.run.smithi204.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 4.5 MB/s | 538 kB 00:00 2024-05-08T23:35:35.814 INFO:teuthology.orchestra.run.smithi160.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 192 kB/s | 48 kB 00:00 2024-05-08T23:35:35.877 INFO:teuthology.orchestra.run.smithi204.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2024-05-08T23:35:36.044 INFO:teuthology.orchestra.run.smithi204.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2024-05-08T23:35:36.086 INFO:teuthology.orchestra.run.smithi204.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2024-05-08T23:35:36.114 INFO:teuthology.orchestra.run.smithi160.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 161 kB/s | 58 kB 00:00 2024-05-08T23:35:36.169 INFO:teuthology.orchestra.run.smithi204.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.8 MB/s | 225 kB 00:00 2024-05-08T23:35:36.189 INFO:teuthology.orchestra.run.smithi160.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 92 kB/s | 39 kB 00:00 2024-05-08T23:35:36.223 INFO:teuthology.orchestra.run.smithi160.stdout:(118/146): fuse-2.9.7-17.el8.x86_64.rpm 203 kB/s | 83 kB 00:00 2024-05-08T23:35:36.286 INFO:teuthology.orchestra.run.smithi204.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.7 MB/s | 354 kB 00:00 2024-05-08T23:35:36.412 INFO:teuthology.orchestra.run.smithi204.stdout:(91/146): openblas-threads-0.3.15-6.el8.x86_64. 22 MB/s | 5.2 MB 00:00 2024-05-08T23:35:36.437 INFO:teuthology.orchestra.run.smithi204.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2024-05-08T23:35:36.457 INFO:teuthology.orchestra.run.smithi160.stdout:(119/146): libgfortran-8.5.0-20.el8.x86_64.rpm 1.8 MB/s | 645 kB 00:00 2024-05-08T23:35:36.482 INFO:teuthology.orchestra.run.smithi160.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 590 kB/s | 172 kB 00:00 2024-05-08T23:35:36.507 INFO:teuthology.orchestra.run.smithi160.stdout:(121/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 838 kB/s | 238 kB 00:00 2024-05-08T23:35:36.621 INFO:teuthology.orchestra.run.smithi204.stdout:(93/146): openblas-0.3.15-6.el8.x86_64.rpm 24 MB/s | 5.0 MB 00:00 2024-05-08T23:35:36.654 INFO:teuthology.orchestra.run.smithi204.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 3.2 MB/s | 714 kB 00:00 2024-05-08T23:35:36.749 INFO:teuthology.orchestra.run.smithi160.stdout:(122/146): elfutils-libelf-devel-0.189-3.el8.x8 231 kB/s | 62 kB 00:00 2024-05-08T23:35:36.754 INFO:teuthology.orchestra.run.smithi204.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2024-05-08T23:35:36.780 INFO:teuthology.orchestra.run.smithi204.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2024-05-08T23:35:36.783 INFO:teuthology.orchestra.run.smithi160.stdout:(123/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 2.0 MB/s | 562 kB 00:00 2024-05-08T23:35:36.800 INFO:teuthology.orchestra.run.smithi160.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 212 kB/s | 73 kB 00:00 2024-05-08T23:35:36.939 INFO:teuthology.orchestra.run.smithi204.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 294 kB/s | 54 kB 00:00 2024-05-08T23:35:37.008 INFO:teuthology.orchestra.run.smithi160.stdout:(125/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 209 kB/s | 47 kB 00:00 2024-05-08T23:35:37.050 INFO:teuthology.orchestra.run.smithi160.stdout:(126/146): python3-cryptography-3.2.1-7.el8_9.x 2.2 MB/s | 559 kB 00:00 2024-05-08T23:35:37.133 INFO:teuthology.orchestra.run.smithi160.stdout:(127/146): openssl-devel-1.1.1k-12.el8_9.x86_64 6.1 MB/s | 2.3 MB 00:00 2024-05-08T23:35:37.214 INFO:teuthology.orchestra.run.smithi204.stdout:(98/146): python3-scipy-1.0.0-21.module+el8.9.0 32 MB/s | 14 MB 00:00 2024-05-08T23:35:37.247 INFO:teuthology.orchestra.run.smithi204.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 611 kB/s | 188 kB 00:00 2024-05-08T23:35:37.284 INFO:teuthology.orchestra.run.smithi160.stdout:(128/146): python3-packaging-16.8-9.el8.noarch. 320 kB/s | 48 kB 00:00 2024-05-08T23:35:37.326 INFO:teuthology.orchestra.run.smithi160.stdout:(129/146): NetworkManager-initscripts-updown-1. 454 kB/s | 144 kB 00:00 2024-05-08T23:35:37.387 INFO:teuthology.orchestra.run.smithi160.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 297 kB/s | 100 kB 00:00 2024-05-08T23:35:37.398 INFO:teuthology.orchestra.run.smithi204.stdout:(100/146): postgresql-contrib-10.23-4.module+el 5.3 MB/s | 812 kB 00:00 2024-05-08T23:35:37.440 INFO:teuthology.orchestra.run.smithi204.stdout:(101/146): postgresql-10.23-4.module+el8.9.0+21 6.7 MB/s | 1.5 MB 00:00 2024-05-08T23:35:37.615 INFO:teuthology.orchestra.run.smithi204.stdout:(102/146): postgresql-server-10.23-4.module+el8 23 MB/s | 5.1 MB 00:00 2024-05-08T23:35:37.640 INFO:teuthology.orchestra.run.smithi204.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 89 kB/s | 18 kB 00:00 2024-05-08T23:35:37.643 INFO:teuthology.orchestra.run.smithi160.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 907 kB/s | 232 kB 00:00 2024-05-08T23:35:37.766 INFO:teuthology.orchestra.run.smithi204.stdout:(104/146): python3-jwt-1.6.1-2.el8.noarch.rpm 284 kB/s | 43 kB 00:00 2024-05-08T23:35:37.808 INFO:teuthology.orchestra.run.smithi204.stdout:(105/146): python3-oauthlib-2.1.0-1.el8.noarch. 927 kB/s | 155 kB 00:00 2024-05-08T23:35:37.875 INFO:teuthology.orchestra.run.smithi204.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 1.0 MB/s | 109 kB 00:00 2024-05-08T23:35:37.925 INFO:teuthology.orchestra.run.smithi204.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 366 kB/s | 43 kB 00:00 2024-05-08T23:35:37.968 INFO:teuthology.orchestra.run.smithi160.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 932 kB/s | 303 kB 00:00 2024-05-08T23:35:37.992 INFO:teuthology.orchestra.run.smithi204.stdout:(108/146): libaio-devel-0.3.112-1.el8.x86_64.rp 160 kB/s | 19 kB 00:00 2024-05-08T23:35:38.050 INFO:teuthology.orchestra.run.smithi204.stdout:(109/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 350 kB/s | 44 kB 00:00 2024-05-08T23:35:38.167 INFO:teuthology.orchestra.run.smithi204.stdout:(110/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.4 MB/s | 605 kB 00:00 2024-05-08T23:35:38.318 INFO:teuthology.orchestra.run.smithi204.stdout:(111/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 857 kB/s | 229 kB 00:00 2024-05-08T23:35:38.343 INFO:teuthology.orchestra.run.smithi204.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2024-05-08T23:35:38.427 INFO:teuthology.orchestra.run.smithi160.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 1.8 MB/s | 854 kB 00:00 2024-05-08T23:35:38.468 INFO:teuthology.orchestra.run.smithi204.stdout:(113/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.3 MB/s | 200 kB 00:00 2024-05-08T23:35:38.485 INFO:teuthology.orchestra.run.smithi204.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 339 kB/s | 48 kB 00:00 2024-05-08T23:35:38.594 INFO:teuthology.orchestra.run.smithi204.stdout:(115/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 532 kB/s | 58 kB 00:00 2024-05-08T23:35:38.711 INFO:teuthology.orchestra.run.smithi204.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 359 kB/s | 87 kB 00:00 2024-05-08T23:35:38.794 INFO:teuthology.orchestra.run.smithi160.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 1.9 MB/s | 711 kB 00:00 2024-05-08T23:35:38.844 INFO:teuthology.orchestra.run.smithi204.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 154 kB/s | 39 kB 00:00 2024-05-08T23:35:38.852 INFO:teuthology.orchestra.run.smithi160.stdout:(135/146): librados2-18.2.2-1322.gd21e4f96.el8. 2.4 MB/s | 3.7 MB 00:01 2024-05-08T23:35:38.870 INFO:teuthology.orchestra.run.smithi204.stdout:(118/146): fuse-2.9.7-17.el8.x86_64.rpm 524 kB/s | 83 kB 00:00 2024-05-08T23:35:39.028 INFO:teuthology.orchestra.run.smithi204.stdout:(119/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 645 kB 00:00 2024-05-08T23:35:39.031 INFO:teuthology.orchestra.run.smithi160.stdout:(136/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 2.1 MB/s | 3.6 MB 00:01 2024-05-08T23:35:39.053 INFO:teuthology.orchestra.run.smithi204.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 939 kB/s | 172 kB 00:00 2024-05-08T23:35:39.170 INFO:teuthology.orchestra.run.smithi160.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 3.9 MB/s | 1.5 MB 00:00 2024-05-08T23:35:39.212 INFO:teuthology.orchestra.run.smithi204.stdout:(121/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 459 kB/s | 73 kB 00:00 2024-05-08T23:35:39.246 INFO:teuthology.orchestra.run.smithi204.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.1 MB/s | 238 kB 00:00 2024-05-08T23:35:39.295 INFO:teuthology.orchestra.run.smithi160.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 754 kB/s | 333 kB 00:00 2024-05-08T23:35:39.337 INFO:teuthology.orchestra.run.smithi160.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 3.5 MB/s | 1.1 MB 00:00 2024-05-08T23:35:39.354 INFO:teuthology.orchestra.run.smithi204.stdout:(123/146): elfutils-libelf-devel-0.189-3.el8.x8 434 kB/s | 62 kB 00:00 2024-05-08T23:35:39.388 INFO:teuthology.orchestra.run.smithi204.stdout:(124/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.9 MB/s | 562 kB 00:00 2024-05-08T23:35:39.395 INFO:teuthology.orchestra.run.smithi160.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 979 kB/s | 221 kB 00:00 2024-05-08T23:35:39.529 INFO:teuthology.orchestra.run.smithi160.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.2 MB/s | 237 kB 00:00 2024-05-08T23:35:39.538 INFO:teuthology.orchestra.run.smithi204.stdout:(125/146): openssl-devel-1.1.1k-12.el8_9.x86_64 13 MB/s | 2.3 MB 00:00 2024-05-08T23:35:39.554 INFO:teuthology.orchestra.run.smithi160.stdout:(142/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 981 kB/s | 254 kB 00:00 2024-05-08T23:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout:(8/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x86 2.6 MB/s | 18 MB 00:07 2024-05-08T23:35:39.564 INFO:teuthology.orchestra.run.smithi204.stdout:(126/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 268 kB/s | 47 kB 00:00 2024-05-08T23:35:39.618 INFO:teuthology.orchestra.run.smithi012.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 419 kB/s | 24 kB 00:00 2024-05-08T23:35:39.638 INFO:teuthology.orchestra.run.smithi160.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 742 kB/s | 180 kB 00:00 2024-05-08T23:35:39.681 INFO:teuthology.orchestra.run.smithi204.stdout:(127/146): NetworkManager-initscripts-updown-1. 1.2 MB/s | 144 kB 00:00 2024-05-08T23:35:39.739 INFO:teuthology.orchestra.run.smithi204.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 2.7 MB/s | 559 kB 00:00 2024-05-08T23:35:39.822 INFO:teuthology.orchestra.run.smithi160.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 339 kB/s | 99 kB 00:00 2024-05-08T23:35:39.872 INFO:teuthology.orchestra.run.smithi160.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 7.8 MB/s | 2.5 MB 00:00 2024-05-08T23:35:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:(129/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 323 kB/s | 100 kB 00:00 2024-05-08T23:35:40.055 INFO:teuthology.orchestra.run.smithi160.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 246 kB/s | 103 kB 00:00 2024-05-08T23:35:40.060 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:35:40.060 INFO:teuthology.orchestra.run.smithi160.stdout:Total 12 MB/s | 215 MB 00:18 2024-05-08T23:35:40.774 INFO:teuthology.orchestra.run.smithi204.stdout:(130/146): librados2-18.2.2-1322.gd21e4f96.el8. 3.6 MB/s | 3.7 MB 00:01 2024-05-08T23:35:40.777 INFO:teuthology.orchestra.run.smithi012.stdout:(10/146): ceph-common-18.2.2-1322.gd21e4f96.el8 1.5 MB/s | 22 MB 00:14 2024-05-08T23:35:40.916 INFO:teuthology.orchestra.run.smithi204.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.6 MB/s | 232 kB 00:00 2024-05-08T23:35:40.977 INFO:teuthology.orchestra.run.smithi012.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 1.3 MB/s | 261 kB 00:00 2024-05-08T23:35:41.102 INFO:teuthology.orchestra.run.smithi012.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 248 kB/s | 31 kB 00:00 2024-05-08T23:35:41.107 INFO:teuthology.orchestra.run.smithi204.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.5 MB/s | 303 kB 00:00 2024-05-08T23:35:41.119 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-08T23:35:41.183 INFO:teuthology.orchestra.run.smithi204.stdout:(133/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 3.0 MB/s | 3.6 MB 00:01 2024-05-08T23:35:41.258 INFO:teuthology.orchestra.run.smithi204.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.5 MB/s | 854 kB 00:00 2024-05-08T23:35:41.291 INFO:teuthology.orchestra.run.smithi204.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 6.4 MB/s | 711 kB 00:00 2024-05-08T23:35:41.417 INFO:teuthology.orchestra.run.smithi204.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.3 MB/s | 1.5 MB 00:00 2024-05-08T23:35:41.450 INFO:teuthology.orchestra.run.smithi204.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.1 MB/s | 333 kB 00:00 2024-05-08T23:35:41.477 INFO:teuthology.orchestra.run.smithi012.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.0 MB/s | 782 kB 00:00 2024-05-08T23:35:41.518 INFO:teuthology.orchestra.run.smithi204.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 11 MB/s | 1.1 MB 00:00 2024-05-08T23:35:41.518 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-08T23:35:41.518 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-08T23:35:41.593 INFO:teuthology.orchestra.run.smithi204.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 221 kB 00:00 2024-05-08T23:35:41.603 INFO:teuthology.orchestra.run.smithi012.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.1 MB/s | 142 kB 00:00 2024-05-08T23:35:41.635 INFO:teuthology.orchestra.run.smithi204.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 254 kB 00:00 2024-05-08T23:35:41.712 INFO:teuthology.orchestra.run.smithi012.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 1.3 MB/s | 139 kB 00:00 2024-05-08T23:35:41.735 INFO:teuthology.orchestra.run.smithi204.stdout:(141/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.8 MB/s | 180 kB 00:00 2024-05-08T23:35:41.760 INFO:teuthology.orchestra.run.smithi204.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.4 MB/s | 237 kB 00:00 2024-05-08T23:35:41.844 INFO:teuthology.orchestra.run.smithi204.stdout:(143/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 913 kB/s | 99 kB 00:00 2024-05-08T23:35:41.954 INFO:teuthology.orchestra.run.smithi012.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 1.9 MB/s | 474 kB 00:00 2024-05-08T23:35:41.969 INFO:teuthology.orchestra.run.smithi204.stdout:(144/146): util-linux-2.32.1-44.el8_9.1.x86_64. 12 MB/s | 2.5 MB 00:00 2024-05-08T23:35:41.986 INFO:teuthology.orchestra.run.smithi204.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 725 kB/s | 103 kB 00:00 2024-05-08T23:35:42.736 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-08T23:35:42.747 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-08T23:35:43.346 INFO:teuthology.orchestra.run.smithi012.stdout:(17/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 1.7 MB/s | 13 MB 00:07 2024-05-08T23:35:43.437 INFO:teuthology.orchestra.run.smithi012.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 489 kB/s | 45 kB 00:00 2024-05-08T23:35:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.1 MB/s | 129 kB 00:00 2024-05-08T23:35:43.746 INFO:teuthology.orchestra.run.smithi012.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 1.0 MB/s | 197 kB 00:00 2024-05-08T23:35:44.046 INFO:teuthology.orchestra.run.smithi012.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 1.2 MB/s | 370 kB 00:00 2024-05-08T23:35:44.255 INFO:teuthology.orchestra.run.smithi012.stdout:(22/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 2.8 MB/s | 6.5 MB 00:02 2024-05-08T23:35:44.347 INFO:teuthology.orchestra.run.smithi012.stdout:(23/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 1.2 MB/s | 365 kB 00:00 2024-05-08T23:35:44.372 INFO:teuthology.orchestra.run.smithi012.stdout:(24/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 915 kB/s | 107 kB 00:00 2024-05-08T23:35:44.447 INFO:teuthology.orchestra.run.smithi012.stdout:(25/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 949 kB/s | 95 kB 00:00 2024-05-08T23:35:44.614 INFO:teuthology.orchestra.run.smithi012.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.1 MB/s | 181 kB 00:00 2024-05-08T23:35:44.731 INFO:teuthology.orchestra.run.smithi012.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 208 kB/s | 24 kB 00:00 2024-05-08T23:35:44.840 INFO:teuthology.orchestra.run.smithi012.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.2 MB/s | 138 kB 00:00 2024-05-08T23:35:45.848 INFO:teuthology.orchestra.run.smithi012.stdout:(29/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 2.3 MB/s | 3.5 MB 00:01 2024-05-08T23:35:46.690 INFO:teuthology.orchestra.run.smithi012.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 1.9 MB/s | 3.5 MB 00:01 2024-05-08T23:35:46.848 INFO:teuthology.orchestra.run.smithi012.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 1.5 MB/s | 245 kB 00:00 2024-05-08T23:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 492 kB/s | 49 kB 00:00 2024-05-08T23:35:47.074 INFO:teuthology.orchestra.run.smithi012.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 130 kB/s | 16 kB 00:00 2024-05-08T23:35:47.283 INFO:teuthology.orchestra.run.smithi012.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.2 MB/s | 263 kB 00:00 2024-05-08T23:35:47.400 INFO:teuthology.orchestra.run.smithi012.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.4 MB/s | 170 kB 00:00 2024-05-08T23:35:48.100 INFO:teuthology.orchestra.run.smithi012.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 26 kB/s | 18 kB 00:00 2024-05-08T23:35:48.550 INFO:teuthology.orchestra.run.smithi012.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 681 kB/s | 306 kB 00:00 2024-05-08T23:35:48.683 INFO:teuthology.orchestra.run.smithi012.stdout:(38/146): ceph-mgr-diskprediction-local-18.2.2- 2.6 MB/s | 7.4 MB 00:02 2024-05-08T23:35:48.709 INFO:teuthology.orchestra.run.smithi012.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 371 kB/s | 59 kB 00:00 2024-05-08T23:35:48.884 INFO:teuthology.orchestra.run.smithi012.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2024-05-08T23:35:48.973 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-08T23:35:48.976 INFO:teuthology.orchestra.run.smithi012.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 476 kB/s | 44 kB 00:00 2024-05-08T23:35:49.001 INFO:teuthology.orchestra.run.smithi012.stdout:(42/146): libunwind-1.3.1-3.el8.x86_64.rpm 237 kB/s | 75 kB 00:00 2024-05-08T23:35:49.077 INFO:teuthology.orchestra.run.smithi012.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2024-05-08T23:35:49.108 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T23:35:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 246 kB/s | 33 kB 00:00 2024-05-08T23:35:49.169 INFO:teuthology.orchestra.run.smithi012.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 178 kB/s | 16 kB 00:00 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi012.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 3.8 MB/s | 384 kB 00:00 2024-05-08T23:35:49.303 INFO:teuthology.orchestra.run.smithi012.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.0 MB/s | 173 kB 00:00 2024-05-08T23:35:49.361 INFO:teuthology.orchestra.run.smithi012.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 885 kB/s | 81 kB 00:00 2024-05-08T23:35:49.373 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:35:49.395 INFO:teuthology.orchestra.run.smithi012.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 120 kB/s | 11 kB 00:00 2024-05-08T23:35:49.454 INFO:teuthology.orchestra.run.smithi012.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2024-05-08T23:35:49.554 INFO:teuthology.orchestra.run.smithi012.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 487 kB/s | 49 kB 00:00 2024-05-08T23:35:49.571 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:35:49.654 INFO:teuthology.orchestra.run.smithi012.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 588 kB/s | 59 kB 00:00 2024-05-08T23:35:49.688 INFO:teuthology.orchestra.run.smithi012.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 4.0 MB/s | 1.2 MB 00:00 2024-05-08T23:35:49.763 INFO:teuthology.orchestra.run.smithi012.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 555 kB/s | 60 kB 00:00 2024-05-08T23:35:49.777 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T23:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2024-05-08T23:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2024-05-08T23:35:49.889 INFO:teuthology.orchestra.run.smithi012.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 366 kB/s | 34 kB 00:00 2024-05-08T23:35:49.956 INFO:teuthology.orchestra.run.smithi012.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2024-05-08T23:35:49.973 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:35:49.981 INFO:teuthology.orchestra.run.smithi012.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 698 kB/s | 64 kB 00:00 2024-05-08T23:35:50.048 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:35:50.048 INFO:teuthology.orchestra.run.smithi012.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 204 kB/s | 19 kB 00:00 2024-05-08T23:35:50.073 INFO:teuthology.orchestra.run.smithi012.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 260 kB/s | 24 kB 00:00 2024-05-08T23:35:50.165 INFO:teuthology.orchestra.run.smithi012.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2024-05-08T23:35:50.191 INFO:teuthology.orchestra.run.smithi012.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2024-05-08T23:35:50.231 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:35:50.301 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:35:50.554 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T23:35:50.685 INFO:teuthology.orchestra.run.smithi204.stdout:(146/146): ceph-test-18.2.2-1322.gd21e4f96.el8. 2.0 MB/s | 47 MB 00:23 2024-05-08T23:35:50.689 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:35:50.689 INFO:teuthology.orchestra.run.smithi204.stdout:Total 7.5 MB/s | 215 MB 00:28 2024-05-08T23:35:50.773 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:35:50.852 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:35:50.908 INFO:teuthology.orchestra.run.smithi012.stdout:(64/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 4.2 MB/s | 47 MB 00:11 2024-05-08T23:35:51.033 INFO:teuthology.orchestra.run.smithi012.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 294 kB/s | 255 kB 00:00 2024-05-08T23:35:51.050 INFO:teuthology.orchestra.run.smithi012.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 292 kB/s | 251 kB 00:00 2024-05-08T23:35:51.127 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T23:35:51.150 INFO:teuthology.orchestra.run.smithi012.stdout:(67/146): python3-zc-lockfile-2.0-2.el8.noarch. 230 kB/s | 23 kB 00:00 2024-05-08T23:35:51.175 INFO:teuthology.orchestra.run.smithi012.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 683 kB/s | 85 kB 00:00 2024-05-08T23:35:51.234 INFO:teuthology.orchestra.run.smithi012.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2024-05-08T23:35:51.284 INFO:teuthology.orchestra.run.smithi012.stdout:(70/146): python3-websocket-client-0.56.0-5.el8 163 kB/s | 61 kB 00:00 2024-05-08T23:35:51.331 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:35:51.384 INFO:teuthology.orchestra.run.smithi012.stdout:(71/146): python3-packaging-16.8-9.el8.noarch.r 321 kB/s | 48 kB 00:00 2024-05-08T23:35:51.460 INFO:teuthology.orchestra.run.smithi012.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 102 kB/s | 18 kB 00:00 2024-05-08T23:35:51.510 INFO:teuthology.orchestra.run.smithi012.stdout:(73/146): thrift-0.13.0-2.el8.x86_64.rpm 4.8 MB/s | 1.7 MB 00:00 2024-05-08T23:35:51.577 INFO:teuthology.orchestra.run.smithi012.stdout:(74/146): python3-jwt-1.6.1-2.el8.noarch.rpm 221 kB/s | 43 kB 00:00 2024-05-08T23:35:51.635 INFO:teuthology.orchestra.run.smithi012.stdout:(75/146): python3-pycparser-2.14-14.el8.noarch. 872 kB/s | 109 kB 00:00 2024-05-08T23:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout:(76/146): python3-oauthlib-2.1.0-1.el8.noarch.r 639 kB/s | 155 kB 00:00 2024-05-08T23:35:51.718 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-05-08T23:35:51.727 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout:(77/146): python3-requests-oauthlib-1.0.0-1.el8 232 kB/s | 43 kB 00:00 2024-05-08T23:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout:(78/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 131 kB/s | 19 kB 00:00 2024-05-08T23:35:51.812 INFO:teuthology.orchestra.run.smithi012.stdout:(79/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 403 kB/s | 44 kB 00:00 2024-05-08T23:35:51.878 INFO:teuthology.orchestra.run.smithi012.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.2 MB/s | 229 kB 00:00 2024-05-08T23:35:51.937 INFO:teuthology.orchestra.run.smithi012.stdout:(81/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.4 MB/s | 605 kB 00:00 2024-05-08T23:35:51.973 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:35:52.020 INFO:teuthology.orchestra.run.smithi012.stdout:(82/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.0 MB/s | 220 kB 00:00 2024-05-08T23:35:52.046 INFO:teuthology.orchestra.run.smithi012.stdout:(83/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.2 MB/s | 200 kB 00:00 2024-05-08T23:35:52.112 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-05-08T23:35:52.112 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-05-08T23:35:52.163 INFO:teuthology.orchestra.run.smithi012.stdout:(84/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 213 kB/s | 48 kB 00:00 2024-05-08T23:35:52.259 INFO:teuthology.orchestra.run.smithi012.stdout:(85/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 271 kB/s | 58 kB 00:00 2024-05-08T23:35:52.330 INFO:teuthology.orchestra.run.smithi012.stdout:(86/146): fuse-2.9.7-17.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2024-05-08T23:35:52.347 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:35:52.464 INFO:teuthology.orchestra.run.smithi012.stdout:(87/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.7 MB/s | 645 kB 00:00 2024-05-08T23:35:52.480 INFO:teuthology.orchestra.run.smithi012.stdout:(88/146): libsepol-devel-2.9-3.el8.x86_64.rpm 189 kB/s | 87 kB 00:00 2024-05-08T23:35:52.539 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:35:52.547 INFO:teuthology.orchestra.run.smithi012.stdout:(89/146): libcom_err-devel-1.45.6-4.el8.x86_64. 100 kB/s | 39 kB 00:00 2024-05-08T23:35:52.598 INFO:teuthology.orchestra.run.smithi012.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.0 MB/s | 238 kB 00:00 2024-05-08T23:35:52.623 INFO:teuthology.orchestra.run.smithi012.stdout:(91/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2024-05-08T23:35:52.650 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:35:52.673 INFO:teuthology.orchestra.run.smithi012.stdout:(92/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 579 kB/s | 73 kB 00:00 2024-05-08T23:35:52.732 INFO:teuthology.orchestra.run.smithi012.stdout:(93/146): elfutils-libelf-devel-0.189-3.el8.x86 462 kB/s | 62 kB 00:00 2024-05-08T23:35:52.790 INFO:teuthology.orchestra.run.smithi012.stdout:(94/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.3 MB/s | 562 kB 00:00 2024-05-08T23:35:52.859 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:35:52.891 INFO:teuthology.orchestra.run.smithi012.stdout:(95/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-08T23:35:52.933 INFO:teuthology.orchestra.run.smithi012.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 3.9 MB/s | 559 kB 00:00 2024-05-08T23:35:52.963 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:35:52.966 INFO:teuthology.orchestra.run.smithi012.stdout:(97/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 201 kB/s | 47 kB 00:00 2024-05-08T23:35:53.000 INFO:teuthology.orchestra.run.smithi012.stdout:(98/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 1.5 MB/s | 100 kB 00:00 2024-05-08T23:35:53.083 INFO:teuthology.orchestra.run.smithi012.stdout:(99/146): python3-jsonpointer-1.10-11.el8.noarc 171 kB/s | 20 kB 00:00 2024-05-08T23:35:53.109 INFO:teuthology.orchestra.run.smithi012.stdout:(100/146): NetworkManager-initscripts-updown-1. 663 kB/s | 144 kB 00:00 2024-05-08T23:35:53.142 INFO:teuthology.orchestra.run.smithi012.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 309 kB/s | 44 kB 00:00 2024-05-08T23:35:53.201 INFO:teuthology.orchestra.run.smithi012.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 941 kB/s | 110 kB 00:00 2024-05-08T23:35:53.226 INFO:teuthology.orchestra.run.smithi012.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 226 kB/s | 27 kB 00:00 2024-05-08T23:35:53.281 INFO:teuthology.orchestra.run.smithi012.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 912 kB/s | 126 kB 00:00 2024-05-08T23:35:53.304 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:35:53.320 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-05-08T23:35:53.330 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-05-08T23:35:53.352 INFO:teuthology.orchestra.run.smithi012.stdout:(105/146): bison-3.0.4-10.el8.x86_64.rpm 5.4 MB/s | 688 kB 00:00 2024-05-08T23:35:53.394 INFO:teuthology.orchestra.run.smithi012.stdout:(106/146): python3-werkzeug-0.12.2-4.el8.noarch 2.3 MB/s | 457 kB 00:00 2024-05-08T23:35:53.427 INFO:teuthology.orchestra.run.smithi012.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2024-05-08T23:35:53.486 INFO:teuthology.orchestra.run.smithi012.stdout:(108/146): python3-markupsafe-0.23-19.el8.x86_6 424 kB/s | 39 kB 00:00 2024-05-08T23:35:53.520 INFO:teuthology.orchestra.run.smithi012.stdout:(109/146): libtool-2.4.6-25.el8.x86_64.rpm 4.1 MB/s | 709 kB 00:00 2024-05-08T23:35:53.561 INFO:teuthology.orchestra.run.smithi012.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.9 MB/s | 259 kB 00:00 2024-05-08T23:35:53.588 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:35:53.628 INFO:teuthology.orchestra.run.smithi012.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 450 kB/s | 64 kB 00:00 2024-05-08T23:35:53.663 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:35:53.754 INFO:teuthology.orchestra.run.smithi012.stdout:(112/146): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2024-05-08T23:35:53.801 INFO:teuthology.orchestra.run.smithi012.stdout:(113/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 368 kB/s | 103 kB 00:00 2024-05-08T23:35:53.846 INFO:teuthology.orchestra.run.smithi012.stdout:(114/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.4 MB/s | 538 kB 00:00 2024-05-08T23:35:53.879 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T23:35:53.954 INFO:teuthology.orchestra.run.smithi012.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.1 MB/s | 323 kB 00:00 2024-05-08T23:35:53.979 INFO:teuthology.orchestra.run.smithi012.stdout:(116/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.6 MB/s | 225 kB 00:00 2024-05-08T23:35:54.055 INFO:teuthology.orchestra.run.smithi012.stdout:(117/146): python3-numpy-1.14.3-10.el8.x86_64.r 12 MB/s | 3.7 MB 00:00 2024-05-08T23:35:54.122 INFO:teuthology.orchestra.run.smithi012.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.1 MB/s | 354 kB 00:00 2024-05-08T23:35:54.176 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T23:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout:(119/146): openblas-threads-0.3.15-6.el8.x86_64 19 MB/s | 5.2 MB 00:00 2024-05-08T23:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout:(120/146): python3-mako-1.0.6-14.el8.noarch.rpm 696 kB/s | 157 kB 00:00 2024-05-08T23:35:54.392 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T23:35:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-05-08T23:35:54.448 INFO:teuthology.orchestra.run.smithi012.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2024-05-08T23:35:54.490 INFO:teuthology.orchestra.run.smithi012.stdout:(123/146): automake-1.16.1-8.el8.noarch.rpm 3.0 MB/s | 714 kB 00:00 2024-05-08T23:35:54.582 INFO:teuthology.orchestra.run.smithi012.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-05-08T23:35:54.607 INFO:teuthology.orchestra.run.smithi012.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 342 kB/s | 54 kB 00:00 2024-05-08T23:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-08T23:35:54.794 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T23:35:54.799 INFO:teuthology.orchestra.run.smithi012.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 7.8 MB/s | 1.5 MB 00:00 2024-05-08T23:35:54.900 INFO:teuthology.orchestra.run.smithi012.stdout:(128/146): postgresql-contrib-10.23-4.module+el 4.1 MB/s | 812 kB 00:00 2024-05-08T23:35:55.059 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T23:35:55.075 INFO:teuthology.orchestra.run.smithi012.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 23 MB/s | 14 MB 00:00 2024-05-08T23:35:55.183 INFO:teuthology.orchestra.run.smithi012.stdout:(130/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-05-08T23:35:55.205 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:35:55.300 INFO:teuthology.orchestra.run.smithi012.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.9 MB/s | 232 kB 00:00 2024-05-08T23:35:55.348 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:35:55.442 INFO:teuthology.orchestra.run.smithi012.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.1 MB/s | 303 kB 00:00 2024-05-08T23:35:55.530 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:35:55.593 INFO:teuthology.orchestra.run.smithi012.stdout:(133/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 6.9 MB/s | 3.6 MB 00:00 2024-05-08T23:35:55.611 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:35:55.643 INFO:teuthology.orchestra.run.smithi012.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.2 MB/s | 854 kB 00:00 2024-05-08T23:35:55.768 INFO:teuthology.orchestra.run.smithi012.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.0 MB/s | 711 kB 00:00 2024-05-08T23:35:55.797 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:35:55.877 INFO:teuthology.orchestra.run.smithi012.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 6.3 MB/s | 1.5 MB 00:00 2024-05-08T23:35:55.910 INFO:teuthology.orchestra.run.smithi012.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.3 MB/s | 333 kB 00:00 2024-05-08T23:35:55.929 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:35:55.994 INFO:teuthology.orchestra.run.smithi012.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 9.1 MB/s | 1.1 MB 00:00 2024-05-08T23:35:56.019 INFO:teuthology.orchestra.run.smithi012.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 2.0 MB/s | 221 kB 00:00 2024-05-08T23:35:56.136 INFO:teuthology.orchestra.run.smithi012.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.8 MB/s | 254 kB 00:00 2024-05-08T23:35:56.138 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:35:56.203 INFO:teuthology.orchestra.run.smithi012.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.3 MB/s | 237 kB 00:00 2024-05-08T23:35:56.261 INFO:teuthology.orchestra.run.smithi012.stdout:(142/146): librados2-18.2.2-1322.gd21e4f96.el8. 2.7 MB/s | 3.7 MB 00:01 2024-05-08T23:35:56.346 INFO:teuthology.orchestra.run.smithi012.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 861 kB/s | 180 kB 00:00 2024-05-08T23:35:56.479 INFO:teuthology.orchestra.run.smithi012.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 359 kB/s | 99 kB 00:00 2024-05-08T23:35:56.537 INFO:teuthology.orchestra.run.smithi012.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 9.0 MB/s | 2.5 MB 00:00 2024-05-08T23:35:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 474 kB/s | 103 kB 00:00 2024-05-08T23:35:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:35:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:Total 7.1 MB/s | 215 MB 00:30 2024-05-08T23:35:56.804 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:35:57.005 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:35:57.596 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:35:57.677 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-05-08T23:35:57.780 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:35:58.127 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-05-08T23:35:58.127 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-05-08T23:35:58.499 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:35:58.819 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T23:35:58.874 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-05-08T23:35:58.960 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T23:35:59.007 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T23:35:59.158 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T23:35:59.246 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:35:59.439 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T23:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-05-08T23:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-05-08T23:35:59.483 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:35:59.664 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T23:35:59.727 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T23:35:59.827 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:35:59.903 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:35:59.990 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T23:36:00.069 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:36:00.140 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:36:00.169 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T23:36:00.332 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T23:36:00.358 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T23:36:00.511 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:36:00.583 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T23:36:00.592 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:36:00.769 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T23:36:00.944 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:36:01.329 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:36:01.535 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:36:01.543 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T23:36:01.919 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T23:36:01.927 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:36:02.085 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:36:02.215 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:36:02.233 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T23:36:02.399 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:36:02.471 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:36:02.494 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:36:02.802 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:36:03.052 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:36:03.135 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:36:03.350 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T23:36:03.630 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T23:36:03.830 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T23:36:04.198 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T23:36:04.216 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:36:04.464 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T23:36:04.579 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T23:36:04.602 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:36:04.704 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:36:04.840 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T23:36:04.877 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:36:04.951 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:36:05.031 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T23:36:05.062 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-05-08T23:36:05.126 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:36:05.201 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T23:36:05.210 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T23:36:05.260 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:36:05.378 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T23:36:05.418 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:36:05.445 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:36:05.587 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:36:05.669 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:36:05.678 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:36:05.859 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T23:36:05.912 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:36:05.963 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:36:06.029 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:36:06.035 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:36:06.107 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:36:06.200 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T23:36:06.226 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:36:06.296 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:36:06.359 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:36:06.431 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T23:36:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T23:36:06.662 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:36:06.765 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-08T23:36:06.833 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:36:06.954 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T23:36:07.043 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:36:07.061 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T23:36:07.122 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:36:07.279 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:36:07.496 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:36:07.522 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:36:07.712 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:36:07.734 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:36:07.737 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-packaging-16.8-9.el8.noarch 49/162 2024-05-08T23:36:07.887 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:36:07.899 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.7-17.el8.x86_64 50/162 2024-05-08T23:36:07.991 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T23:36:08.126 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 51/162 2024-05-08T23:36:08.131 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T23:36:08.313 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T23:36:08.315 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:36:08.325 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 52/162 2024-05-08T23:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:36:08.548 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 53/162 2024-05-08T23:36:08.593 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T23:36:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:36:08.749 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 54/162 2024-05-08T23:36:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:36:08.865 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T23:36:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:36:09.058 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 55/162 2024-05-08T23:36:09.119 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T23:36:09.274 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T23:36:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:36:09.368 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/162 2024-05-08T23:36:09.387 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T23:36:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:36:09.583 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/162 2024-05-08T23:36:09.636 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T23:36:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:36:09.774 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 58/162 2024-05-08T23:36:09.872 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T23:36:10.023 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/162 2024-05-08T23:36:10.144 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T23:36:10.342 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T23:36:10.420 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/162 2024-05-08T23:36:10.577 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T23:36:10.739 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 61/162 2024-05-08T23:36:10.741 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T23:36:10.905 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T23:36:11.009 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T23:36:11.036 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/162 2024-05-08T23:36:11.153 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:36:11.173 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T23:36:11.261 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 63/162 2024-05-08T23:36:11.270 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:36:11.372 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:36:11.461 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:36:11.521 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:36:11.525 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:36:11.685 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-08T23:36:11.766 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 65/162 2024-05-08T23:36:11.994 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-08T23:36:12.024 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 66/162 2024-05-08T23:36:12.218 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-08T23:36:12.262 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 67/162 2024-05-08T23:36:12.394 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T23:36:12.472 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 68/162 2024-05-08T23:36:12.526 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T23:36:12.661 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T23:36:12.684 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 69/162 2024-05-08T23:36:12.948 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 70/162 2024-05-08T23:36:13.105 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:36:13.169 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T23:36:13.254 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T23:36:13.353 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T23:36:13.433 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T23:36:13.512 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T23:36:13.549 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-05-08T23:36:13.645 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T23:36:13.828 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-05-08T23:36:13.852 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T23:36:13.995 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T23:36:14.064 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T23:36:14.098 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T23:36:14.183 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T23:36:14.216 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T23:36:14.259 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-05-08T23:36:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T23:36:14.392 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:36:14.453 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:36:14.467 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-05-08T23:36:14.687 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-05-08T23:36:14.717 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:36:14.780 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:36:14.869 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T23:36:14.987 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T23:36:15.088 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T23:36:15.153 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T23:36:15.160 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T23:36:15.350 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:36:15.376 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-08T23:36:15.434 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T23:36:15.634 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-08T23:36:15.768 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T23:36:15.802 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:36:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T23:36:15.957 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T23:36:16.092 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:36:16.093 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T23:36:16.109 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:36:16.147 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T23:36:16.224 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T23:36:16.296 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T23:36:16.380 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T23:36:16.479 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T23:36:16.480 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T23:36:16.615 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T23:36:16.661 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T23:36:16.695 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T23:36:16.877 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T23:36:16.883 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T23:36:17.054 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T23:36:17.072 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T23:36:17.258 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T23:36:17.377 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T23:36:17.490 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T23:36:17.631 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T23:36:17.739 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T23:36:17.880 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T23:36:17.920 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T23:36:18.037 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T23:36:18.189 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T23:36:18.269 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T23:36:18.484 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T23:36:18.624 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T23:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T23:36:18.701 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T23:36:18.919 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T23:36:18.953 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-08T23:36:18.959 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T23:36:19.198 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T23:36:19.432 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T23:36:19.601 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T23:36:19.811 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T23:36:20.031 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-08T23:36:20.066 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T23:36:20.086 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T23:36:20.317 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T23:36:20.384 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-08T23:36:20.398 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T23:36:20.527 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T23:36:20.634 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T23:36:20.667 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-08T23:36:20.721 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T23:36:20.912 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T23:36:20.954 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T23:36:21.010 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T23:36:21.158 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T23:36:21.210 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T23:36:21.224 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T23:36:21.308 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T23:36:21.342 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T23:36:21.426 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T23:36:21.510 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T23:36:21.621 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-05-08T23:36:21.726 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:36:21.808 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:36:21.850 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-05-08T23:36:22.001 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:36:22.016 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T23:36:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:36:22.085 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T23:36:22.256 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T23:36:22.319 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T23:36:22.466 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T23:36:22.473 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-05-08T23:36:22.504 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T23:36:22.659 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T23:36:22.659 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:36:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:36:22.734 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:36:22.859 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T23:36:22.927 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T23:36:23.066 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T23:36:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:36:23.243 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T23:36:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:36:23.522 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T23:36:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:36:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T23:36:23.765 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T23:36:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T23:36:23.929 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T23:36:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-08T23:36:24.094 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T23:36:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-08T23:36:24.334 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T23:36:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T23:36:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T23:36:24.545 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T23:36:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-08T23:36:24.755 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T23:36:24.901 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-08T23:36:25.011 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T23:36:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-08T23:36:25.172 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T23:36:25.272 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/162 2024-05-08T23:36:25.387 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T23:36:25.460 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/162 2024-05-08T23:36:25.628 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T23:36:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/162 2024-05-08T23:36:25.779 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T23:36:25.855 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/162 2024-05-08T23:36:26.020 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T23:36:26.047 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/162 2024-05-08T23:36:26.281 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T23:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/162 2024-05-08T23:36:26.456 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T23:36:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/162 2024-05-08T23:36:26.720 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:36:27.089 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T23:36:27.258 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/162 2024-05-08T23:36:27.443 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/162 2024-05-08T23:36:27.644 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/162 2024-05-08T23:36:28.001 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/162 2024-05-08T23:36:28.092 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:36:28.165 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T23:36:28.257 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 67/162 2024-05-08T23:36:28.452 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T23:36:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/162 2024-05-08T23:36:28.664 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T23:36:28.693 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:36:28.793 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 69/162 2024-05-08T23:36:28.983 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T23:36:28.997 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 70/162 2024-05-08T23:36:29.042 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:36:29.122 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:36:29.224 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T23:36:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 71/162 2024-05-08T23:36:29.283 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:36:29.440 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T23:36:29.508 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/162 2024-05-08T23:36:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/162 2024-05-08T23:36:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/162 2024-05-08T23:36:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/162 2024-05-08T23:36:30.205 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T23:36:30.381 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:36:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/162 2024-05-08T23:36:30.456 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:36:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T23:36:30.623 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T23:36:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T23:36:30.945 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T23:36:31.261 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T23:36:31.523 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T23:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T23:36:31.861 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T23:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T23:36:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T23:36:32.699 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T23:36:32.852 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T23:36:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T23:36:33.249 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T23:36:33.428 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T23:36:33.596 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T23:36:33.851 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T23:36:34.115 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T23:36:34.291 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T23:36:34.518 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T23:36:34.703 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:36:34.956 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:36:35.550 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:36:35.718 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T23:36:35.858 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:36:35.963 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:36:36.092 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T23:36:36.106 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:36:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T23:36:36.611 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T23:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T23:36:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T23:36:37.862 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T23:36:38.005 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:36:38.084 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:36:38.256 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T23:36:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:36:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:36:44.155 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:36:44.429 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:36:44.524 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:36:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:36:48.592 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:36:48.592 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2024-05-08T23:36:48.592 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2024-05-08T23:36:48.592 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /dev 2024-05-08T23:36:48.592 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /run 2024-05-08T23:36:48.592 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2024-05-08T23:36:48.593 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2024-05-08T23:36:48.593 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2024-05-08T23:36:48.593 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2024-05-08T23:36:48.593 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /dev 2024-05-08T23:36:48.593 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:36:48.981 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:36:50.619 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:36:50.830 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:36:51.050 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:36:51.205 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:36:51.681 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:36:51.855 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:36:51.873 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:36:53.771 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:36:53.949 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:36:54.061 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T23:36:54.313 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T23:36:54.624 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:36:55.646 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:36:55.789 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:36:55.789 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-05-08T23:36:55.789 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-05-08T23:36:55.789 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /dev 2024-05-08T23:36:55.789 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /run 2024-05-08T23:36:55.789 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-05-08T23:36:55.789 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-05-08T23:36:55.790 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-05-08T23:36:55.790 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-05-08T23:36:55.790 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /dev 2024-05-08T23:36:55.790 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:36:55.904 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:36:56.162 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:36:56.872 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:36:57.062 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:36:57.156 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:36:57.525 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T23:36:57.729 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:36:57.768 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:36:57.797 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:36:57.962 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:36:57.963 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:36:57.974 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:36:58.181 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:36:58.319 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:36:58.793 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:36:58.961 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:36:58.979 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:36:59.228 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T23:36:59.446 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:36:59.481 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:36:59.695 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:36:59.739 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:36:59.937 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:37:00.286 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:37:00.478 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:37:00.953 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:37:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:37:01.275 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T23:37:01.512 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T23:37:01.740 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:37:02.770 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:37:02.995 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:37:03.958 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:37:04.126 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:37:04.220 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:37:04.558 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T23:37:04.737 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:37:04.776 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:37:04.930 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:37:04.951 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:37:04.963 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:37:05.415 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T23:37:05.705 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T23:37:05.941 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:37:06.259 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T23:37:06.479 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:37:06.514 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:37:06.641 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:37:06.693 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:37:06.736 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:37:06.810 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:37:06.811 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /sys 2024-05-08T23:37:06.811 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /proc 2024-05-08T23:37:06.811 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /dev 2024-05-08T23:37:06.811 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /run 2024-05-08T23:37:06.811 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /mnt 2024-05-08T23:37:06.811 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /var/tmp 2024-05-08T23:37:06.811 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /home 2024-05-08T23:37:06.811 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /tmp 2024-05-08T23:37:06.811 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /dev 2024-05-08T23:37:06.811 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:37:06.893 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:37:06.965 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T23:37:07.174 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:37:07.240 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:37:07.245 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T23:37:07.402 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:37:07.475 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:37:07.730 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T23:37:07.949 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T23:37:08.162 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T23:37:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:37:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:37:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:37:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:37:10.076 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:37:10.215 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:37:10.236 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:37:10.489 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:37:10.827 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:37:10.865 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:37:11.348 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T23:37:11.708 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T23:37:11.857 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:37:11.980 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T23:37:12.143 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T23:37:12.282 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:37:12.285 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T23:37:12.351 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T23:37:12.425 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:37:12.589 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T23:37:12.627 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T23:37:12.645 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T23:37:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T23:37:12.878 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:37:12.953 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T23:37:13.064 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:37:13.101 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:37:13.110 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:37:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T23:37:13.543 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:37:13.570 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:37:13.621 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:37:13.855 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T23:37:13.876 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:37:13.889 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:37:14.056 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T23:37:14.167 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T23:37:14.203 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:37:14.228 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T23:37:14.365 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:37:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:37:14.438 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T23:37:14.438 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:37:14.569 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T23:37:14.721 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:37:14.721 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:37:14.779 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T23:37:14.951 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T23:37:14.963 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:37:14.963 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:37:15.249 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:37:15.494 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T23:37:15.504 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:37:15.686 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:37:15.696 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:37:15.806 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:37:15.944 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T23:37:15.944 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:37:16.175 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T23:37:16.228 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:37:16.229 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:37:16.338 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:37:16.385 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:37:16.495 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:37:16.495 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:37:16.555 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:37:16.570 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:37:16.737 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:37:16.737 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:37:16.995 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:37:16.995 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:37:17.253 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:37:17.254 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:37:17.453 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T23:37:17.453 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:37:17.559 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:37:17.597 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:37:17.972 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T23:37:18.000 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T23:37:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:37:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:37:18.332 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T23:37:18.373 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:37:18.373 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T23:37:18.373 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T23:37:18.373 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T23:37:18.373 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T23:37:18.374 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T23:37:18.375 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T23:37:18.375 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T23:37:18.375 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:37:18.375 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T23:37:18.375 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T23:37:18.375 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T23:37:18.375 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T23:37:18.375 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T23:37:18.375 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T23:37:18.375 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T23:37:18.376 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T23:37:18.376 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T23:37:18.376 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T23:37:18.376 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T23:37:18.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T23:37:18.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T23:37:18.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T23:37:18.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T23:37:18.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T23:37:18.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T23:37:18.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T23:37:18.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T23:37:18.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T23:37:18.389 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T23:37:18.390 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:37:18.391 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-05-08T23:37:18.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-05-08T23:37:18.393 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-05-08T23:37:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 103/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 104/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 105/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 106/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 107/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 108/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 109/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 110/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 112/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 113/162 2024-05-08T23:37:18.395 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 114/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 115/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 117/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 118/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 119/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 120/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 121/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 122/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 123/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 124/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 125/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 126/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 127/162 2024-05-08T23:37:18.396 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 128/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 129/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 130/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T23:37:18.397 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T23:37:18.398 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T23:37:18.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T23:37:18.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:37:18.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T23:37:18.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:37:18.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T23:37:18.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:37:18.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T23:37:18.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:37:18.399 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T23:37:18.455 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:37:18.506 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:37:18.596 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T23:37:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:37:18.783 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T23:37:18.974 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T23:37:19.049 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:37:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:37:19.233 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T23:37:19.501 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T23:37:19.675 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:37:19.684 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:37:19.919 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T23:37:20.157 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:37:20.234 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:37:20.491 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:37:20.504 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:37:20.672 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T23:37:20.843 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T23:37:21.011 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T23:37:21.011 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:37:21.319 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:37:21.319 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:37:21.536 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:37:21.537 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:37:21.806 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:37:22.018 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T23:37:22.203 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:37:22.451 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T23:37:22.451 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:37:22.677 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:37:22.677 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:37:22.902 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:37:22.902 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:37:23.160 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:37:23.160 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:37:23.410 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:37:23.410 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:37:23.618 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:37:23.802 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T23:37:23.802 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:37:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:37:24.703 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:37:24.703 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T23:37:24.703 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T23:37:24.703 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T23:37:24.703 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T23:37:24.703 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T23:37:24.704 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T23:37:24.704 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T23:37:24.704 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T23:37:24.704 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:37:24.704 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:37:24.704 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:37:24.704 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:37:24.704 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T23:37:24.704 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T23:37:24.704 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T23:37:24.705 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T23:37:24.706 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T23:37:24.706 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T23:37:24.706 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T23:37:24.720 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T23:37:24.720 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T23:37:24.721 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T23:37:24.722 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T23:37:24.723 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-05-08T23:37:24.724 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-05-08T23:37:24.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T23:37:24.726 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T23:37:24.727 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T23:37:24.728 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T23:37:24.729 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T23:37:24.730 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:37:24.730 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T23:37:24.730 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:37:24.730 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T23:37:24.730 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:37:24.730 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T23:37:24.730 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:37:24.730 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T23:37:25.284 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T23:37:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-08T23:37:25.878 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-08T23:37:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 129/162 2024-05-08T23:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-08T23:37:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T23:37:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T23:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T23:37:27.258 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T23:37:27.420 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T23:37:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:37:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:37:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:37:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T23:37:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T23:37:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T23:37:31.586 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T23:37:31.787 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T23:37:31.895 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T23:37:31.895 INFO:teuthology.orchestra.run.smithi160.stdout:Installed products updated. 2024-05-08T23:37:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T23:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T23:37:32.557 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:37:32.557 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2024-05-08T23:37:32.557 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T23:37:32.557 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T23:37:32.557 INFO:teuthology.orchestra.run.smithi160.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T23:37:32.557 INFO:teuthology.orchestra.run.smithi160.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:32.557 INFO:teuthology.orchestra.run.smithi160.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T23:37:32.558 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.559 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: jq-1.6-7.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.561 INFO:teuthology.orchestra.run.smithi160.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T23:37:32.562 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:37:32.563 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:37:32.564 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:37:32.564 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T23:37:32.564 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T23:37:32.564 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T23:37:32.564 INFO:teuthology.orchestra.run.smithi160.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T23:37:32.564 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T23:37:32.564 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.564 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T23:37:32.565 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T23:37:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T23:37:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T23:37:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T23:37:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T23:37:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T23:37:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T23:37:32.567 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T23:37:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T23:37:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T23:37:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T23:37:32.568 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.569 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.570 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:32.570 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T23:37:32.570 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T23:37:32.570 INFO:teuthology.orchestra.run.smithi160.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T23:37:32.570 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T23:37:32.570 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T23:37:32.570 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T23:37:32.570 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-08T23:37:32.570 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-08T23:37:32.600 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T23:37:32.688 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T23:37:32.802 DEBUG:teuthology.parallel:result is None 2024-05-08T23:37:32.877 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T23:37:32.893 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T23:37:33.051 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T23:37:33.062 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T23:37:33.246 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-08T23:37:33.397 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T23:37:33.553 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T23:37:33.755 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T23:37:33.755 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:37:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:37:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:37:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:37:34.255 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:37:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:37:34.703 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T23:37:34.872 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:37:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T23:37:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:37:35.354 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:37:35.354 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:37:35.561 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:37:35.562 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:37:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:37:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:37:35.995 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:37:35.995 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:37:36.195 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:37:36.353 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T23:37:36.353 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:37:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:37:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T23:37:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T23:37:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T23:37:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T23:37:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T23:37:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T23:37:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T23:37:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T23:37:37.497 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T23:37:37.498 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T23:37:37.499 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/162 2024-05-08T23:37:37.500 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 93/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 94/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 95/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 96/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 97/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 98/162 2024-05-08T23:37:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 99/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-08T23:37:37.502 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-08T23:37:37.503 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T23:37:37.504 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T23:37:37.505 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T23:37:37.506 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T23:37:37.506 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:37:37.506 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T23:37:37.506 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:37:37.506 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T23:37:37.506 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:37:37.506 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T23:37:37.506 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:37:37.506 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T23:37:40.240 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T23:37:40.240 INFO:teuthology.orchestra.run.smithi204.stdout:Installed products updated. 2024-05-08T23:37:41.553 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:37:41.553 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-05-08T23:37:41.554 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T23:37:41.554 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T23:37:41.554 INFO:teuthology.orchestra.run.smithi204.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T23:37:41.554 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:41.554 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:41.554 INFO:teuthology.orchestra.run.smithi204.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:41.555 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.555 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.555 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:41.555 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:41.555 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:37:41.555 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:37:41.555 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:41.556 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:41.556 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T23:37:41.556 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T23:37:41.556 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-05-08T23:37:41.556 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T23:37:41.556 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T23:37:41.556 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T23:37:41.556 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.557 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.557 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.557 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.557 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:41.557 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.557 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.557 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.557 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:41.558 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:41.558 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:41.558 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:41.558 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:41.558 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.558 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.558 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: jq-1.6-7.el8.x86_64 2024-05-08T23:37:41.559 INFO:teuthology.orchestra.run.smithi204.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.560 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T23:37:41.561 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T23:37:41.563 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T23:37:41.564 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T23:37:41.565 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T23:37:41.565 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T23:37:41.565 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T23:37:41.565 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T23:37:41.565 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T23:37:41.565 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.565 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.565 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T23:37:41.565 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T23:37:41.566 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T23:37:41.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T23:37:41.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T23:37:41.567 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.567 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.567 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:41.567 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T23:37:41.567 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T23:37:41.567 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T23:37:41.567 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T23:37:41.567 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T23:37:41.568 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T23:37:41.568 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:37:41.568 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-05-08T23:37:41.776 DEBUG:teuthology.parallel:result is None 2024-05-08T23:37:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T23:37:55.517 INFO:teuthology.orchestra.run.smithi012.stdout:Installed products updated. 2024-05-08T23:37:56.211 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:37:56.211 INFO:teuthology.orchestra.run.smithi012.stdout:Upgraded: 2024-05-08T23:37:56.211 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T23:37:56.211 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T23:37:56.211 INFO:teuthology.orchestra.run.smithi012.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T23:37:56.211 INFO:teuthology.orchestra.run.smithi012.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:56.211 INFO:teuthology.orchestra.run.smithi012.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-05-08T23:37:56.212 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:56.213 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.214 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: jq-1.6-7.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T23:37:56.215 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:37:56.216 INFO:teuthology.orchestra.run.smithi012.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:37:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T23:37:56.218 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T23:37:56.219 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T23:37:56.220 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T23:37:56.221 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T23:37:56.222 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T23:37:56.223 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T23:37:56.223 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:37:56.223 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-05-08T23:37:57.692 DEBUG:teuthology.parallel:result is None 2024-05-08T23:37:57.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:37:57.836 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:37:57.864 INFO:teuthology.orchestra.run.smithi012.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T23:37:57.864 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T23:37:57.865 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T23:37:57.867 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:37:58.016 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:37:58.046 INFO:teuthology.orchestra.run.smithi160.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T23:37:58.046 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T23:37:58.046 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T23:37:58.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:37:58.189 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:37:58.218 INFO:teuthology.orchestra.run.smithi204.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T23:37:58.219 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T23:37:58.219 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T23:37:58.221 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T23:37:58.221 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:37:58.221 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:37:58.275 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:37:58.276 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:37:58.325 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:37:58.325 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:37:58.373 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T23:37:58.374 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:37:58.374 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:37:58.410 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:37:58.485 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:37:58.485 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:37:58.518 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:37:58.590 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:37:58.590 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:37:58.623 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:37:58.695 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T23:37:58.695 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:37:58.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:37:58.731 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:37:58.806 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:37:58.807 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:37:58.839 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:37:58.913 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:37:58.914 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:37:58.946 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:37:59.019 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T23:37:59.019 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:37:59.019 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:37:59.056 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:37:59.131 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:37:59.131 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:37:59.164 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:37:59.235 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:37:59.235 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:37:59.268 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:37:59.339 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T23:37:59.440 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-08T23:37:59.440 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:37:59.441 INFO:tasks.cephadm:Cluster fsid is 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:37:59.441 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T23:37:59.441 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.12', 'mon.b': '172.21.15.160', 'mon.c': '172.21.15.204'} 2024-05-08T23:37:59.441 INFO:tasks.cephadm:First mon is mon.a on smithi012 2024-05-08T23:37:59.441 INFO:tasks.cephadm:First mgr is x 2024-05-08T23:37:59.442 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T23:37:59.442 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname $(hostname -s) 2024-05-08T23:37:59.478 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname $(hostname -s) 2024-05-08T23:37:59.511 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-05-08T23:37:59.544 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T23:37:59.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:37:59.760 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79222/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:11:17.581213', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['arm64', 'source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79220/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 18:26:58.979379', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T23:37:59.979 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240508.150423-reef, sha1 d21e4f961318b56f0d915caa2490c3711bf13c34 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:37:59.981 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm 2024-05-08T23:37:59.981 DEBUG:teuthology.orchestra.run.smithi012:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T23:38:00.508 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 23:38 /home/ubuntu/cephtest/cephadm 2024-05-08T23:38:00.508 DEBUG:teuthology.orchestra.run.smithi160:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T23:38:01.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 23:38 /home/ubuntu/cephtest/cephadm 2024-05-08T23:38:01.067 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T23:38:01.611 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 23:38 /home/ubuntu/cephtest/cephadm 2024-05-08T23:38:01.611 DEBUG:teuthology.orchestra.run.smithi012:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T23:38:01.635 DEBUG:teuthology.orchestra.run.smithi160:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T23:38:01.656 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T23:38:01.693 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 on all hosts... 2024-05-08T23:38:01.693 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T23:38:01.696 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T23:38:01.698 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T23:38:02.601 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T23:38:02.678 INFO:teuthology.orchestra.run.smithi160.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T23:38:03.356 INFO:teuthology.orchestra.run.smithi012.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T23:38:46.181 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-08T23:38:46.181 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:38:46.182 INFO:teuthology.orchestra.run.smithi160.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T23:38:46.182 INFO:teuthology.orchestra.run.smithi160.stdout: "repo_digests": [ 2024-05-08T23:38:46.182 INFO:teuthology.orchestra.run.smithi160.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T23:38:46.182 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-05-08T23:38:46.182 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-08T23:38:50.916 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-08T23:38:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:38:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T23:38:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-05-08T23:38:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T23:38:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-05-08T23:38:50.917 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-08T23:38:52.043 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:38:52.043 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:38:52.044 INFO:teuthology.orchestra.run.smithi012.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T23:38:52.044 INFO:teuthology.orchestra.run.smithi012.stdout: "repo_digests": [ 2024-05-08T23:38:52.044 INFO:teuthology.orchestra.run.smithi012.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T23:38:52.044 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-08T23:38:52.044 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:38:52.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph 2024-05-08T23:38:52.110 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph 2024-05-08T23:38:52.147 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-05-08T23:38:52.184 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /etc/ceph 2024-05-08T23:38:52.220 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /etc/ceph 2024-05-08T23:38:52.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-05-08T23:38:52.282 INFO:tasks.cephadm:Writing seed config... 2024-05-08T23:38:52.283 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T23:38:52.283 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T23:38:52.283 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T23:38:52.283 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T23:38:52.284 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T23:38:52.284 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-08T23:38:52.284 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T23:38:52.284 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T23:38:52.284 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T23:38:52.284 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T23:38:52.284 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T23:38:52.284 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T23:38:52.284 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T23:38:52.284 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T23:38:52.285 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T23:38:52.286 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T23:38:52.287 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T23:38:52.287 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T23:38:52.287 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T23:38:52.287 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T23:38:52.287 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-08T23:38:52.288 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:38:52.288 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T23:38:52.310 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 00f8fb28-0d94-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T23:38:52.310 DEBUG:teuthology.orchestra.run.smithi012:mon.a> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T23:38:52.353 DEBUG:teuthology.orchestra.run.smithi012:mgr.x> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T23:38:52.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:-- Logs begin at Wed 2024-05-08 23:21:09 UTC. -- 2024-05-08T23:38:52.389 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T23:38:52.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 -v bootstrap --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.12 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:38:52.421 INFO:journalctl@ceph.mgr.x.smithi012.stdout:-- Logs begin at Wed 2024-05-08 23:21:09 UTC. -- 2024-05-08T23:38:52.578 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:38:52.579 INFO:teuthology.orchestra.run.smithi012.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34', '-v', 'bootstrap', '--fsid', '00f8fb28-0d94-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.12', '--skip-admin-label'] 2024-05-08T23:38:52.616 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.6.1 2024-05-08T23:38:52.617 INFO:teuthology.orchestra.run.smithi012.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T23:38:52.617 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying podman|docker is present... 2024-05-08T23:38:52.652 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.6.1 2024-05-08T23:38:52.652 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying lvm2 is present... 2024-05-08T23:38:52.652 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying time synchronization is in place... 2024-05-08T23:38:52.660 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T23:38:52.660 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T23:38:52.666 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T23:38:52.667 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-08T23:38:52.673 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-05-08T23:38:52.680 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-05-08T23:38:52.680 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-05-08T23:38:52.680 INFO:teuthology.orchestra.run.smithi012.stdout:Repeating the final host check... 2024-05-08T23:38:52.712 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.6.1 2024-05-08T23:38:52.712 INFO:teuthology.orchestra.run.smithi012.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-08T23:38:52.712 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl is present 2024-05-08T23:38:52.712 INFO:teuthology.orchestra.run.smithi012.stdout:lvcreate is present 2024-05-08T23:38:52.719 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T23:38:52.719 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T23:38:52.725 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T23:38:52.725 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-08T23:38:52.732 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-05-08T23:38:52.739 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-05-08T23:38:52.739 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-05-08T23:38:52.739 INFO:teuthology.orchestra.run.smithi012.stdout:Host looks OK 2024-05-08T23:38:52.739 INFO:teuthology.orchestra.run.smithi012.stdout:Cluster fsid: 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:38:52.739 INFO:teuthology.orchestra.run.smithi012.stdout:Acquiring lock 139810720647656 on /run/cephadm/00f8fb28-0d94-11ef-bc97-c7b262605968.lock 2024-05-08T23:38:52.739 INFO:teuthology.orchestra.run.smithi012.stdout:Lock 139810720647656 acquired on /run/cephadm/00f8fb28-0d94-11ef-bc97-c7b262605968.lock 2024-05-08T23:38:52.740 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 3300 ... 2024-05-08T23:38:52.740 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 6789 ... 2024-05-08T23:38:52.740 INFO:teuthology.orchestra.run.smithi012.stdout:Base mon IP(s) is [172.21.15.12:3300, 172.21.15.12:6789], mon addrv is [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-05-08T23:38:52.745 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2024-05-08T23:38:52.745 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.12 metric 100 2024-05-08T23:38:52.747 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T23:38:52.747 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-05-08T23:38:52.747 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2024-05-08T23:38:52.750 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T23:38:52.751 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T23:38:52.751 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T23:38:52.751 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-08T23:38:52.751 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2024-05-08T23:38:52.751 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T23:38:52.751 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-05-08T23:38:52.752 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-05-08T23:38:52.752 INFO:teuthology.orchestra.run.smithi012.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T23:38:52.752 INFO:teuthology.orchestra.run.smithi012.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T23:38:52.753 INFO:teuthology.orchestra.run.smithi012.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T23:38:55.716 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T23:38:55.717 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T23:38:55.717 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T23:38:55.717 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T23:38:55.717 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:7850648fa4707607dddc5eec7674b8c9296086cf87f20a8b1acf1b014b952e9f 2024-05-08T23:38:55.717 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying config sha256:af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T23:38:55.717 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T23:38:56.465 INFO:teuthology.orchestra.run.smithi012.stdout:ceph: stdout ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T23:38:56.465 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph version: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T23:38:56.465 INFO:teuthology.orchestra.run.smithi012.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T23:38:57.080 INFO:teuthology.orchestra.run.smithi012.stdout:stat: stdout 167 167 2024-05-08T23:38:57.081 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial keys... 2024-05-08T23:38:57.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQARDTxmM4qlFhAAcrppEhlNi+QZtQHKya7Sjw== 2024-05-08T23:38:58.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQASDTxmrJfCABAALPUMUNrBpJpFvpEo/DqMgQ== 2024-05-08T23:38:59.059 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQASDTxm1MSEKRAAaXGEPRpUYOfWMcYGgaoTjQ== 2024-05-08T23:38:59.060 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial monmap... 2024-05-08T23:38:59.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T23:38:59.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-08T23:38:59.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:38:59.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T23:38:59.747 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool for a [v2:172.21.15.12:3300,v1:172.21.15.12:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T23:38:59.747 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = pacific 2024-05-08T23:38:59.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: set fsid to 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:38:59.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T23:38:59.748 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:38:59.748 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mon... 2024-05-08T23:39:00.480 INFO:teuthology.orchestra.run.smithi012.stdout:create mon.a on 2024-05-08T23:39:00.647 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-08T23:39:00.753 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T23:39:00.889 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-00f8fb28-0d94-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-00f8fb28-0d94-11ef-bc97-c7b262605968.target. 2024-05-08T23:39:00.889 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-00f8fb28-0d94-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-00f8fb28-0d94-11ef-bc97-c7b262605968.target. 2024-05-08T23:39:01.098 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.a 2024-05-08T23:39:01.098 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T23:39:01.200 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-00f8fb28-0d94-11ef-bc97-c7b262605968.target.wants/ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@.service. 2024-05-08T23:39:01.584 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T23:39:01.584 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-08T23:39:01.593 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T23:39:01.593 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-08T23:39:01.593 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-08T23:39:01.593 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T23:39:01.594 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon to start... 2024-05-08T23:39:01.594 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon... 2024-05-08T23:39:01.821 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:01 smithi012 ceph-mon[42019]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T23:39:02.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T23:39:02.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout id: 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:39:02.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T23:39:02.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:02.509 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout services: 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.51982s) 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout data: 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.050+0000 7f91ae0c0700 1 Processor -- start 2024-05-08T23:39:02.510 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.050+0000 7f91ae0c0700 1 -- start start 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.050+0000 7f91ae0c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8105b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.050+0000 7f91ae0c0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81060d0 con 0x7f91a81056e0 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.051+0000 7f91a77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8105b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.051+0000 7f91a77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8105b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56250/0 (socket says 172.21.15.12:56250) 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.051+0000 7f91a77fe700 1 -- 172.21.15.12:0/1413175819 learned_addr learned my addr 172.21.15.12:0/1413175819 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.051+0000 7f91a77fe700 1 -- 172.21.15.12:0/1413175819 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a81068e0 con 0x7f91a81056e0 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.051+0000 7f91a77fe700 1 --2- 172.21.15.12:0/1413175819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8105b00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f91980098a0 tx=0x7f9198009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4219ed105fcf0106 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.051+0000 7f91a67fc700 1 -- 172.21.15.12:0/1413175819 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9198004160 con 0x7f91a81056e0 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.051+0000 7f91a67fc700 1 -- 172.21.15.12:0/1413175819 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91980042c0 con 0x7f91a81056e0 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.051+0000 7f91a67fc700 1 -- 172.21.15.12:0/1413175819 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91980045a0 con 0x7f91a81056e0 2024-05-08T23:39:02.511 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.052+0000 7f91ae0c0700 1 -- 172.21.15.12:0/1413175819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 msgr2=0x7f91a8105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.052+0000 7f91ae0c0700 1 --2- 172.21.15.12:0/1413175819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8105b00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f91980098a0 tx=0x7f9198009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.052+0000 7f91ae0c0700 1 -- 172.21.15.12:0/1413175819 shutdown_connections 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.052+0000 7f91ae0c0700 1 --2- 172.21.15.12:0/1413175819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8105b00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.052+0000 7f91ae0c0700 1 -- 172.21.15.12:0/1413175819 >> 172.21.15.12:0/1413175819 conn(0x7f91a8100c60 msgr2=0x7f91a81030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.052+0000 7f91ae0c0700 1 -- 172.21.15.12:0/1413175819 shutdown_connections 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.052+0000 7f91ae0c0700 1 -- 172.21.15.12:0/1413175819 wait complete. 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.053+0000 7f91ae0c0700 1 Processor -- start 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.053+0000 7f91ae0c0700 1 -- start start 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.053+0000 7f91ae0c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.053+0000 7f91ae0c0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a819a270 con 0x7f91a81056e0 2024-05-08T23:39:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.053+0000 7f91a77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.053+0000 7f91a77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56264/0 (socket says 172.21.15.12:56264) 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.053+0000 7f91a77fe700 1 -- 172.21.15.12:0/3125126553 learned_addr learned my addr 172.21.15.12:0/3125126553 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.054+0000 7f91a77fe700 1 -- 172.21.15.12:0/3125126553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9198009550 con 0x7f91a81056e0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.054+0000 7f91a77fe700 1 --2- 172.21.15.12:0/3125126553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8199d30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9198004000 tx=0x7f9198004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.054+0000 7f91a4ff9700 1 -- 172.21.15.12:0/3125126553 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9198011630 con 0x7f91a81056e0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.054+0000 7f91a4ff9700 1 -- 172.21.15.12:0/3125126553 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9198011790 con 0x7f91a81056e0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.054+0000 7f91ae0c0700 1 -- 172.21.15.12:0/3125126553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a819a470 con 0x7f91a81056e0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.054+0000 7f91a4ff9700 1 -- 172.21.15.12:0/3125126553 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9198011a30 con 0x7f91a81056e0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.054+0000 7f91ae0c0700 1 -- 172.21.15.12:0/3125126553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a819a910 con 0x7f91a81056e0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.054+0000 7f91a4ff9700 1 -- 172.21.15.12:0/3125126553 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9198021070 con 0x7f91a81056e0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.054+0000 7f91a4ff9700 1 -- 172.21.15.12:0/3125126553 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f919801b780 con 0x7f91a81056e0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.055+0000 7f91ae0c0700 1 -- 172.21.15.12:0/3125126553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a81939c0 con 0x7f91a81056e0 2024-05-08T23:39:02.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.058+0000 7f91a4ff9700 1 -- 172.21.15.12:0/3125126553 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f919801acc0 con 0x7f91a81056e0 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.134+0000 7f91ae0c0700 1 -- 172.21.15.12:0/3125126553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f91a8106f00 con 0x7f91a81056e0 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.134+0000 7f91a4ff9700 1 -- 172.21.15.12:0/3125126553 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7f919801ae60 con 0x7f91a81056e0 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.135+0000 7f91ae0c0700 1 -- 172.21.15.12:0/3125126553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 msgr2=0x7f91a8199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.135+0000 7f91ae0c0700 1 --2- 172.21.15.12:0/3125126553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8199d30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9198004000 tx=0x7f9198004750 comp rx=0 tx=0).stop 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.135+0000 7f91ae0c0700 1 -- 172.21.15.12:0/3125126553 shutdown_connections 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.135+0000 7f91ae0c0700 1 --2- 172.21.15.12:0/3125126553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a81056e0 0x7f91a8199d30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.135+0000 7f91ae0c0700 1 -- 172.21.15.12:0/3125126553 >> 172.21.15.12:0/3125126553 conn(0x7f91a8100c60 msgr2=0x7f91a8190b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.135+0000 7f91ae0c0700 1 -- 172.21.15.12:0/3125126553 shutdown_connections 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.135+0000 7f91ae0c0700 1 -- 172.21.15.12:0/3125126553 wait complete. 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:mon is available 2024-05-08T23:39:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T23:39:02.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:02 smithi012 ceph-mon[42019]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T23:39:02.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:02 smithi012 ceph-mon[42019]: monmap e1: 1 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:39:02.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:02 smithi012 ceph-mon[42019]: fsmap 2024-05-08T23:39:02.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:02 smithi012 ceph-mon[42019]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T23:39:02.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:02 smithi012 ceph-mon[42019]: mgrmap e1: no daemons active 2024-05-08T23:39:02.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:02 smithi012 ceph-mon[42019]: from='client.? 172.21.15.12:0/3125126553' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T23:39:03.369 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T23:39:03.370 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.938+0000 7f1db02f5700 1 Processor -- start 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.939+0000 7f1db02f5700 1 -- start start 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.939+0000 7f1db02f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.939+0000 7f1db02f5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da8107170 con 0x7f1da8106780 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.939+0000 7f1dae091700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.939+0000 7f1dae091700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56270/0 (socket says 172.21.15.12:56270) 2024-05-08T23:39:03.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.939+0000 7f1dae091700 1 -- 172.21.15.12:0/218792436 learned_addr learned my addr 172.21.15.12:0/218792436 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:03.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.940+0000 7f1dae091700 1 -- 172.21.15.12:0/218792436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da8107980 con 0x7f1da8106780 2024-05-08T23:39:03.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.940+0000 7f1dae091700 1 --2- 172.21.15.12:0/218792436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8106ba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1da40098a0 tx=0x7f1da4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8abe695d8cd211 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:03.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.940+0000 7f1dad08f700 1 -- 172.21.15.12:0/218792436 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1da4004160 con 0x7f1da8106780 2024-05-08T23:39:03.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.940+0000 7f1dad08f700 1 -- 172.21.15.12:0/218792436 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da40042c0 con 0x7f1da8106780 2024-05-08T23:39:03.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.940+0000 7f1dad08f700 1 -- 172.21.15.12:0/218792436 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1da40045a0 con 0x7f1da8106780 2024-05-08T23:39:03.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.940+0000 7f1db02f5700 1 -- 172.21.15.12:0/218792436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 msgr2=0x7f1da8106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:03.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.940+0000 7f1db02f5700 1 --2- 172.21.15.12:0/218792436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8106ba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1da40098a0 tx=0x7f1da4009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:03.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.941+0000 7f1db02f5700 1 -- 172.21.15.12:0/218792436 shutdown_connections 2024-05-08T23:39:03.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.941+0000 7f1db02f5700 1 --2- 172.21.15.12:0/218792436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8106ba0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:03.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.941+0000 7f1db02f5700 1 -- 172.21.15.12:0/218792436 >> 172.21.15.12:0/218792436 conn(0x7f1da8101d00 msgr2=0x7f1da8104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:03.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.941+0000 7f1db02f5700 1 -- 172.21.15.12:0/218792436 shutdown_connections 2024-05-08T23:39:03.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.941+0000 7f1db02f5700 1 -- 172.21.15.12:0/218792436 wait complete. 2024-05-08T23:39:03.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.941+0000 7f1db02f5700 1 Processor -- start 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.941+0000 7f1db02f5700 1 -- start start 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1db02f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1db02f5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da819a1f0 con 0x7f1da8106780 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1dae091700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1dae091700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56280/0 (socket says 172.21.15.12:56280) 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1dae091700 1 -- 172.21.15.12:0/686313398 learned_addr learned my addr 172.21.15.12:0/686313398 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1dae091700 1 -- 172.21.15.12:0/686313398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da4009550 con 0x7f1da8106780 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1dae091700 1 --2- 172.21.15.12:0/686313398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8199cb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1da4004000 tx=0x7f1da4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1d9f7fe700 1 -- 172.21.15.12:0/686313398 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1da4011630 con 0x7f1da8106780 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1d9f7fe700 1 -- 172.21.15.12:0/686313398 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da4011790 con 0x7f1da8106780 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1db02f5700 1 -- 172.21.15.12:0/686313398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da819a3f0 con 0x7f1da8106780 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1d9f7fe700 1 -- 172.21.15.12:0/686313398 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1da4011a30 con 0x7f1da8106780 2024-05-08T23:39:03.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.942+0000 7f1db02f5700 1 -- 172.21.15.12:0/686313398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da819a890 con 0x7f1da8106780 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.943+0000 7f1d9f7fe700 1 -- 172.21.15.12:0/686313398 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1da4022070 con 0x7f1da8106780 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.943+0000 7f1d9f7fe700 1 -- 172.21.15.12:0/686313398 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1da401c450 con 0x7f1da8106780 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.944+0000 7f1db02f5700 1 -- 172.21.15.12:0/686313398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da8193a40 con 0x7f1da8106780 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:02.946+0000 7f1d9f7fe700 1 -- 172.21.15.12:0/686313398 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f1da401b450 con 0x7f1da8106780 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.019+0000 7f1db02f5700 1 -- 172.21.15.12:0/686313398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f1da8107cf0 con 0x7f1da8106780 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.024+0000 7f1d9f7fe700 1 -- 172.21.15.12:0/686313398 <== mon.0 v2:172.21.15.12:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1da401b5f0 con 0x7f1da8106780 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.024+0000 7f1d9f7fe700 1 -- 172.21.15.12:0/686313398 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f1da4024630 con 0x7f1da8106780 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.025+0000 7f1db02f5700 1 -- 172.21.15.12:0/686313398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 msgr2=0x7f1da8199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.025+0000 7f1db02f5700 1 --2- 172.21.15.12:0/686313398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8199cb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1da4004000 tx=0x7f1da4004750 comp rx=0 tx=0).stop 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.025+0000 7f1db02f5700 1 -- 172.21.15.12:0/686313398 shutdown_connections 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.025+0000 7f1db02f5700 1 --2- 172.21.15.12:0/686313398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1da8106780 0x7f1da8199cb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.025+0000 7f1db02f5700 1 -- 172.21.15.12:0/686313398 >> 172.21.15.12:0/686313398 conn(0x7f1da8101d00 msgr2=0x7f1da8190b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.025+0000 7f1db02f5700 1 -- 172.21.15.12:0/686313398 shutdown_connections 2024-05-08T23:39:03.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.025+0000 7f1db02f5700 1 -- 172.21.15.12:0/686313398 wait complete. 2024-05-08T23:39:03.376 INFO:teuthology.orchestra.run.smithi012.stdout:Generating new minimal ceph.conf... 2024-05-08T23:39:03.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:03 smithi012 ceph-mon[42019]: from='client.? 172.21.15.12:0/686313398' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T23:39:03.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:03 smithi012 ceph-mon[42019]: from='client.? 172.21.15.12:0/686313398' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T23:39:04.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.737+0000 7f57cf818700 1 Processor -- start 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.737+0000 7f57cf818700 1 -- start start 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.738+0000 7f57cf818700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c8105490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.738+0000 7f57cf818700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8079e40 con 0x7f57c8105070 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.738+0000 7f57cd5b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c8105490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.738+0000 7f57cd5b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c8105490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56290/0 (socket says 172.21.15.12:56290) 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.738+0000 7f57cd5b4700 1 -- 172.21.15.12:0/2638824164 learned_addr learned my addr 172.21.15.12:0/2638824164 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.738+0000 7f57cd5b4700 1 -- 172.21.15.12:0/2638824164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c81059d0 con 0x7f57c8105070 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.738+0000 7f57cd5b4700 1 --2- 172.21.15.12:0/2638824164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c8105490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57c40098a0 tx=0x7f57c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1515cb69bf6461ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.739+0000 7f57bbfff700 1 -- 172.21.15.12:0/2638824164 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57c4004160 con 0x7f57c8105070 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.739+0000 7f57bbfff700 1 -- 172.21.15.12:0/2638824164 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57c40042c0 con 0x7f57c8105070 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.739+0000 7f57bbfff700 1 -- 172.21.15.12:0/2638824164 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57c4011a00 con 0x7f57c8105070 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.739+0000 7f57cf818700 1 -- 172.21.15.12:0/2638824164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 msgr2=0x7f57c8105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:04.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.739+0000 7f57cf818700 1 --2- 172.21.15.12:0/2638824164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c8105490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57c40098a0 tx=0x7f57c4009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.739+0000 7f57cf818700 1 -- 172.21.15.12:0/2638824164 shutdown_connections 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.739+0000 7f57cf818700 1 --2- 172.21.15.12:0/2638824164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c8105490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.739+0000 7f57cf818700 1 -- 172.21.15.12:0/2638824164 >> 172.21.15.12:0/2638824164 conn(0x7f57c8100c70 msgr2=0x7f57c8103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.739+0000 7f57cf818700 1 -- 172.21.15.12:0/2638824164 shutdown_connections 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.739+0000 7f57cf818700 1 -- 172.21.15.12:0/2638824164 wait complete. 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.740+0000 7f57cf818700 1 Processor -- start 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.740+0000 7f57cf818700 1 -- start start 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.740+0000 7f57cf818700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.740+0000 7f57cf818700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c819cc20 con 0x7f57c8105070 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.740+0000 7f57cd5b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.740+0000 7f57cd5b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c819c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56302/0 (socket says 172.21.15.12:56302) 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.740+0000 7f57cd5b4700 1 -- 172.21.15.12:0/4105469791 learned_addr learned my addr 172.21.15.12:0/4105469791 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:04.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.741+0000 7f57cd5b4700 1 -- 172.21.15.12:0/4105469791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4009550 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.741+0000 7f57cd5b4700 1 --2- 172.21.15.12:0/4105469791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c819c6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009fb0 tx=0x7f57c4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.741+0000 7f57ba7fc700 1 -- 172.21.15.12:0/4105469791 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57c4003a40 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.741+0000 7f57ba7fc700 1 -- 172.21.15.12:0/4105469791 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57c4003ba0 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.741+0000 7f57cf818700 1 -- 172.21.15.12:0/4105469791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c819ce20 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.741+0000 7f57ba7fc700 1 -- 172.21.15.12:0/4105469791 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57c401a750 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.741+0000 7f57cf818700 1 -- 172.21.15.12:0/4105469791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c819d2c0 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.741+0000 7f57ba7fc700 1 -- 172.21.15.12:0/4105469791 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f57c401ac80 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.741+0000 7f57ba7fc700 1 -- 172.21.15.12:0/4105469791 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f57c40193c0 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.742+0000 7f57cf818700 1 -- 172.21.15.12:0/4105469791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c81963f0 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.745+0000 7f57ba7fc700 1 -- 172.21.15.12:0/4105469791 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f57c4024b60 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.818+0000 7f57cf818700 1 -- 172.21.15.12:0/4105469791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f57c8106840 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.819+0000 7f57ba7fc700 1 -- 172.21.15.12:0/4105469791 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f57c4024d00 con 0x7f57c8105070 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.819+0000 7f57cf818700 1 -- 172.21.15.12:0/4105469791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 msgr2=0x7f57c819c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:04.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.819+0000 7f57cf818700 1 --2- 172.21.15.12:0/4105469791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c819c6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009fb0 tx=0x7f57c4011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:04.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.819+0000 7f57cf818700 1 -- 172.21.15.12:0/4105469791 shutdown_connections 2024-05-08T23:39:04.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.819+0000 7f57cf818700 1 --2- 172.21.15.12:0/4105469791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105070 0x7f57c819c6e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:04.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.819+0000 7f57cf818700 1 -- 172.21.15.12:0/4105469791 >> 172.21.15.12:0/4105469791 conn(0x7f57c8100c70 msgr2=0x7f57c8193530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:04.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.819+0000 7f57cf818700 1 -- 172.21.15.12:0/4105469791 shutdown_connections 2024-05-08T23:39:04.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:03.820+0000 7f57cf818700 1 -- 172.21.15.12:0/4105469791 wait complete. 2024-05-08T23:39:04.170 INFO:teuthology.orchestra.run.smithi012.stdout:Restarting the monitor... 2024-05-08T23:39:04.390 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:04 smithi012 systemd[1]: Stopping Ceph mon.a for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:39:04.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:04 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-a[41996]: 2024-05-08T23:39:04.392+0000 7f90fd5e7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:39:04.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:04 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-a[41996]: 2024-05-08T23:39:04.392+0000 7f90fd5e7700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T23:39:05.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:04 smithi012 bash[42345]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-a 2024-05-08T23:39:05.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 systemd[1]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.a.service: Succeeded. 2024-05-08T23:39:05.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 systemd[1]: Stopped Ceph mon.a for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:39:05.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 systemd[1]: Starting Ceph mon.a for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:39:05.552 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 bash[42465]: 1067817c4ecf1ce296e219924ca4e0ba028c6acda4108b7d986c0f0f3e6327af 2024-05-08T23:39:05.553 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T23:39:05.555 INFO:teuthology.orchestra.run.smithi012.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T23:39:05.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable), process ceph-mon, pid 7 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: pidfile_write: ignore empty --pid-file 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 systemd[1]: Started Ceph mon.a for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: load: jerasure load: lrc 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: RocksDB version: 7.9.2 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Git sha 0 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: DB SUMMARY 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: DB Session ID: P8UGHDD4QWZS80LRPFV0 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: CURRENT file: CURRENT 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T23:39:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98258 ; 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.error_if_exists: 0 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.create_if_missing: 0 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.paranoid_checks: 1 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.env: 0x5598841d9820 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.info_log: 0x559885f2f520 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.statistics: (nil) 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.use_fsync: 0 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_log_file_size: 0 2024-05-08T23:39:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.allow_fallocate: 1 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.use_direct_reads: 0 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.db_log_dir: 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.wal_dir: 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T23:39:05.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.write_buffer_manager: 0x5598851c4320 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T23:39:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.unordered_write: 0 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.row_cache: None 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.wal_filter: None 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.two_write_queues: 0 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.wal_compression: 0 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.atomic_flush: 0 2024-05-08T23:39:05.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.log_readahead_size: 0 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_background_jobs: 2 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_background_compactions: -1 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_subcompactions: 1 2024-05-08T23:39:05.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_open_files: -1 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_background_flushes: -1 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Compression algorithms supported: 2024-05-08T23:39:05.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: kZSTD supported: 0 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: kXpressCompression supported: 0 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: kZlibCompression supported: 1 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: kSnappyCompression supported: 1 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: kLZ4Compression supported: 1 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: kBZip2Compression supported: 0 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.merge_operator: 2024-05-08T23:39:05.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_filter: None 2024-05-08T23:39:05.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T23:39:05.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T23:39:05.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T23:39:05.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T23:39:05.926 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559885f2f620) 2024-05-08T23:39:05.926 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-05-08T23:39:05.926 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T23:39:05.927 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T23:39:05.927 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-05-08T23:39:05.927 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_type: 0 2024-05-08T23:39:05.927 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_index_type: 0 2024-05-08T23:39:05.927 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_shortening: 1 2024-05-08T23:39:05.927 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T23:39:05.927 INFO:journalctl@ceph.mon.a.smithi012.stdout: checksum: 4 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: no_block_cache: 0 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache: 0x5598852471f0 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_options: 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: capacity : 536870912 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_shard_bits : 4 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: strict_capacity_limit : 0 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_compressed: (nil) 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: persistent_cache: (nil) 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size: 4096 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size_deviation: 10 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_restart_interval: 16 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_block_restart_interval: 1 2024-05-08T23:39:05.928 INFO:journalctl@ceph.mon.a.smithi012.stdout: metadata_block_size: 4096 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: partition_filters: 0 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: use_delta_encoding: 1 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: filter_policy: bloomfilter 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: whole_key_filtering: 1 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: verify_compression: 0 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: format_version: 5 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_index_compression: 1 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_align: 0 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: max_auto_readahead_size: 262144 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: prepopulate_block_cache: 0 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: initial_auto_readahead_size: 8192 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T23:39:05.929 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compression: NoCompression 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.num_levels: 7 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T23:39:05.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T23:39:05.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T23:39:05.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T23:39:05.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.table_properties_collectors: 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.inplace_update_support: 0 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.bloom_locality: 0 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.max_successive_merges: 0 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T23:39:05.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.ttl: 2592000 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.enable_blob_files: false 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.min_blob_size: 0 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T23:39:05.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 11856923-d0b9-459c-824b-24bf28c78881 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715211545563707, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715211545564899, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93585, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 311, "table_properties": {"data_size": 91576, "index_size": 291, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13802, "raw_average_key_size": 50, "raw_value_size": 84074, "raw_average_value_size": 306, "num_data_blocks": 11, "num_entries": 274, "num_filter_entries": 274, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715211545, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "11856923-d0b9-459c-824b-24bf28c78881", "db_session_id": "P8UGHDD4QWZS80LRPFV0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715211545565014, "job": 1, "event": "recovery_finished"} 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5598852e4000 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: DB pointer 0x5598852d2000 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** DB Stats ** 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:39:05.936 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: L0 2/0 93.23 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: Sum 2/0 93.23 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-08T23:39:05.937 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 120.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative compaction: 0.00 GB write, 23.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval compaction: 0.00 GB write, 23.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:39:05.938 INFO:journalctl@ceph.mon.a.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache BinnedLRUCache@0x5598852471f0#7 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: starting mon.a rank 0 at public addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] at bind addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: mon.a@-1(???) e1 preinit fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: mon.a@-1(???).mds e1 new map 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: mon.a@-1(???).mds e1 print_map 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout: e1 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout: legacy client fscid: -1 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-08T23:39:05.939 INFO:journalctl@ceph.mon.a.smithi012.stdout: No filesystems configured 2024-05-08T23:39:05.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T23:39:05.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:39:05.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:39:05.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:39:05.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T23:39:05.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T23:39:05.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: monmap e1: 1 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:39:05.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: fsmap 2024-05-08T23:39:05.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T23:39:05.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:05 smithi012 ceph-mon[42503]: mgrmap e1: no daemons active 2024-05-08T23:39:06.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.094+0000 7efdb6826700 1 Processor -- start 2024-05-08T23:39:06.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.094+0000 7efdb6826700 1 -- start start 2024-05-08T23:39:06.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.095+0000 7efdb6826700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb0105aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:06.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.095+0000 7efdb6826700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb0106070 con 0x7efdb0105680 2024-05-08T23:39:06.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.095+0000 7efdaffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb0105aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:06.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.095+0000 7efdaffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb0105aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56308/0 (socket says 172.21.15.12:56308) 2024-05-08T23:39:06.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.095+0000 7efdaffff700 1 -- 172.21.15.12:0/4005660810 learned_addr learned my addr 172.21.15.12:0/4005660810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:06.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.095+0000 7efdaffff700 1 -- 172.21.15.12:0/4005660810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb0106880 con 0x7efdb0105680 2024-05-08T23:39:06.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.096+0000 7efdaffff700 1 --2- 172.21.15.12:0/4005660810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb0105aa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efda00098a0 tx=0x7efda0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c69367aad63ffb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:06.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.096+0000 7efdaeffd700 1 -- 172.21.15.12:0/4005660810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efda0004160 con 0x7efdb0105680 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.096+0000 7efdaeffd700 1 -- 172.21.15.12:0/4005660810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efda00042c0 con 0x7efdb0105680 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.096+0000 7efdaeffd700 1 -- 172.21.15.12:0/4005660810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efda0011a00 con 0x7efdb0105680 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.096+0000 7efdb6826700 1 -- 172.21.15.12:0/4005660810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 msgr2=0x7efdb0105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.096+0000 7efdb6826700 1 --2- 172.21.15.12:0/4005660810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb0105aa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efda00098a0 tx=0x7efda0009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.096+0000 7efdb6826700 1 -- 172.21.15.12:0/4005660810 shutdown_connections 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.096+0000 7efdb6826700 1 --2- 172.21.15.12:0/4005660810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb0105aa0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.096+0000 7efdb6826700 1 -- 172.21.15.12:0/4005660810 >> 172.21.15.12:0/4005660810 conn(0x7efdb0100cc0 msgr2=0x7efdb01030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.096+0000 7efdb6826700 1 -- 172.21.15.12:0/4005660810 shutdown_connections 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.097+0000 7efdb6826700 1 -- 172.21.15.12:0/4005660810 wait complete. 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.097+0000 7efdb6826700 1 Processor -- start 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.097+0000 7efdb6826700 1 -- start start 2024-05-08T23:39:06.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdb6826700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdb6826700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb019cc90 con 0x7efdb0105680 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdaffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdaffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb019c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56312/0 (socket says 172.21.15.12:56312) 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdaffff700 1 -- 172.21.15.12:0/3982256355 learned_addr learned my addr 172.21.15.12:0/3982256355 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdaffff700 1 -- 172.21.15.12:0/3982256355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda0009550 con 0x7efdb0105680 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdaffff700 1 --2- 172.21.15.12:0/3982256355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb019c750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efda0009130 tx=0x7efda0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdad7fa700 1 -- 172.21.15.12:0/3982256355 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efda0003ab0 con 0x7efdb0105680 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdad7fa700 1 -- 172.21.15.12:0/3982256355 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efda0003c10 con 0x7efdb0105680 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdad7fa700 1 -- 172.21.15.12:0/3982256355 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efda00247d0 con 0x7efdb0105680 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.098+0000 7efdb6826700 1 -- 172.21.15.12:0/3982256355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb019ce90 con 0x7efdb0105680 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.099+0000 7efdb6826700 1 -- 172.21.15.12:0/3982256355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb019d330 con 0x7efdb0105680 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.099+0000 7efdad7fa700 1 -- 172.21.15.12:0/3982256355 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7efda0024d00 con 0x7efdb0105680 2024-05-08T23:39:06.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.099+0000 7efdad7fa700 1 -- 172.21.15.12:0/3982256355 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efda00193c0 con 0x7efdb0105680 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.099+0000 7efdb6826700 1 -- 172.21.15.12:0/3982256355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb00625d0 con 0x7efdb0105680 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.102+0000 7efdad7fa700 1 -- 172.21.15.12:0/3982256355 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7efda001abb0 con 0x7efdb0105680 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.176+0000 7efdb6826700 1 -- 172.21.15.12:0/3982256355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7efdb019fbc0 con 0x7efdb0105680 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.179+0000 7efdad7fa700 1 -- 172.21.15.12:0/3982256355 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7efda001ad50 con 0x7efdb0105680 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.180+0000 7efdb6826700 1 -- 172.21.15.12:0/3982256355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 msgr2=0x7efdb019c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.180+0000 7efdb6826700 1 --2- 172.21.15.12:0/3982256355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb019c750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efda0009130 tx=0x7efda0011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.180+0000 7efdb6826700 1 -- 172.21.15.12:0/3982256355 shutdown_connections 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.180+0000 7efdb6826700 1 --2- 172.21.15.12:0/3982256355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdb0105680 0x7efdb019c750 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.180+0000 7efdb6826700 1 -- 172.21.15.12:0/3982256355 >> 172.21.15.12:0/3982256355 conn(0x7efdb0100cc0 msgr2=0x7efdb0101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.180+0000 7efdb6826700 1 -- 172.21.15.12:0/3982256355 shutdown_connections 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:06.180+0000 7efdb6826700 1 -- 172.21.15.12:0/3982256355 wait complete. 2024-05-08T23:39:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T23:39:06.663 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:39:06.664 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mgr... 2024-05-08T23:39:06.664 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T23:39:06.665 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T23:39:06.887 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.x 2024-05-08T23:39:06.888 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T23:39:06.990 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-00f8fb28-0d94-11ef-bc97-c7b262605968.target.wants/ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@.service. 2024-05-08T23:39:06.999 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:07 smithi012 systemd[1]: Starting Ceph mgr.x for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:39:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T23:39:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-08T23:39:07.442 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T23:39:07.442 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-08T23:39:07.442 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-08T23:39:07.443 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T23:39:07.452 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T23:39:07.452 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-08T23:39:07.461 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T23:39:07.461 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-08T23:39:07.461 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-08T23:39:07.461 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T23:39:07.461 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr to start... 2024-05-08T23:39:07.461 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr... 2024-05-08T23:39:07.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:07 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3982256355' entity='client.admin' 2024-05-08T23:39:07.588 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:07 smithi012 bash[42737]: 572a774ebb79c95a31444953962ff709ac6372c5e3eb235b6fb2b7274c88a34b 2024-05-08T23:39:07.588 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:07 smithi012 systemd[1]: Started Ceph mgr.x for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:39:08.167 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:07 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:07.890+0000 7ff339a1d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:39:08.167 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:07 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:07.997+0000 7ff339a1d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:39:08.167 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:08.099+0000 7ff339a1d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:39:08.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:08.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:39:08.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "00f8fb28-0d94-11ef-bc97-c7b262605968", 2024-05-08T23:39:08.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T23:39:08.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T23:39:08.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T23:39:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:08.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T23:39:08.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T23:39:08.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:08.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T23:39:08.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:08.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T23:39:08.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T23:39:08.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T23:39:08.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T23:39:08.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T23:39:08.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T23:39:08.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:08.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T23:39:08.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T23:39:08.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T23:39:08.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:39:08.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T23:39:01.617832+0000", 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.837+0000 7f09f20aa700 1 Processor -- start 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.837+0000 7f09f20aa700 1 -- start start 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.837+0000 7f09f20aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec105b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.837+0000 7f09f20aa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec1060d0 con 0x7f09ec1056e0 2024-05-08T23:39:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.838+0000 7f09eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec105b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.838+0000 7f09eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec105b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56352/0 (socket says 172.21.15.12:56352) 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.838+0000 7f09eb7fe700 1 -- 172.21.15.12:0/2198760185 learned_addr learned my addr 172.21.15.12:0/2198760185 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.838+0000 7f09eb7fe700 1 -- 172.21.15.12:0/2198760185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec1068e0 con 0x7f09ec1056e0 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.838+0000 7f09eb7fe700 1 --2- 172.21.15.12:0/2198760185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec105b00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0098a0 tx=0x7f09dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1b1c0d8932ccf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.838+0000 7f09ea7fc700 1 -- 172.21.15.12:0/2198760185 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09dc004160 con 0x7f09ec1056e0 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.838+0000 7f09ea7fc700 1 -- 172.21.15.12:0/2198760185 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f09dc0042c0 con 0x7f09ec1056e0 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.838+0000 7f09ea7fc700 1 -- 172.21.15.12:0/2198760185 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09dc011a00 con 0x7f09ec1056e0 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.839+0000 7f09f20aa700 1 -- 172.21.15.12:0/2198760185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 msgr2=0x7f09ec105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.839+0000 7f09f20aa700 1 --2- 172.21.15.12:0/2198760185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec105b00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0098a0 tx=0x7f09dc009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.839+0000 7f09f20aa700 1 -- 172.21.15.12:0/2198760185 shutdown_connections 2024-05-08T23:39:08.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.839+0000 7f09f20aa700 1 --2- 172.21.15.12:0/2198760185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec105b00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.839+0000 7f09f20aa700 1 -- 172.21.15.12:0/2198760185 >> 172.21.15.12:0/2198760185 conn(0x7f09ec100c60 msgr2=0x7f09ec1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.839+0000 7f09f20aa700 1 -- 172.21.15.12:0/2198760185 shutdown_connections 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.839+0000 7f09f20aa700 1 -- 172.21.15.12:0/2198760185 wait complete. 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.840+0000 7f09f20aa700 1 Processor -- start 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.841+0000 7f09f20aa700 1 -- start start 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.841+0000 7f09f20aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec074450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.841+0000 7f09f20aa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec074990 con 0x7f09ec1056e0 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.841+0000 7f09eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec074450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.841+0000 7f09eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec074450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56366/0 (socket says 172.21.15.12:56366) 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.841+0000 7f09eb7fe700 1 -- 172.21.15.12:0/1642716448 learned_addr learned my addr 172.21.15.12:0/1642716448 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:08.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.841+0000 7f09eb7fe700 1 -- 172.21.15.12:0/1642716448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc009550 con 0x7f09ec1056e0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.841+0000 7f09eb7fe700 1 --2- 172.21.15.12:0/1642716448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec074450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009130 tx=0x7f09dc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.842+0000 7f09e8ff9700 1 -- 172.21.15.12:0/1642716448 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09dc0038b0 con 0x7f09ec1056e0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.842+0000 7f09e8ff9700 1 -- 172.21.15.12:0/1642716448 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f09dc003a10 con 0x7f09ec1056e0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.842+0000 7f09f20aa700 1 -- 172.21.15.12:0/1642716448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ec19d2f0 con 0x7f09ec1056e0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.842+0000 7f09e8ff9700 1 -- 172.21.15.12:0/1642716448 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09dc019550 con 0x7f09ec1056e0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.842+0000 7f09f20aa700 1 -- 172.21.15.12:0/1642716448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec19d790 con 0x7f09ec1056e0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.842+0000 7f09e8ff9700 1 -- 172.21.15.12:0/1642716448 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f09dc019b30 con 0x7f09ec1056e0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.842+0000 7f09e8ff9700 1 -- 172.21.15.12:0/1642716448 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f09dc02de30 con 0x7f09ec1056e0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.843+0000 7f09f20aa700 1 -- 172.21.15.12:0/1642716448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ec109b10 con 0x7f09ec1056e0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.846+0000 7f09e8ff9700 1 -- 172.21.15.12:0/1642716448 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f09dc003b80 con 0x7f09ec1056e0 2024-05-08T23:39:08.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.923+0000 7f09f20aa700 1 -- 172.21.15.12:0/1642716448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f09ec19d9d0 con 0x7f09ec1056e0 2024-05-08T23:39:08.323 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.924+0000 7f09e8ff9700 1 -- 172.21.15.12:0/1642716448 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f09dc01a960 con 0x7f09ec1056e0 2024-05-08T23:39:08.323 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.925+0000 7f09f20aa700 1 -- 172.21.15.12:0/1642716448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 msgr2=0x7f09ec074450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:08.323 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.925+0000 7f09f20aa700 1 --2- 172.21.15.12:0/1642716448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec074450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009130 tx=0x7f09dc011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:08.323 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.925+0000 7f09f20aa700 1 -- 172.21.15.12:0/1642716448 shutdown_connections 2024-05-08T23:39:08.323 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.925+0000 7f09f20aa700 1 --2- 172.21.15.12:0/1642716448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ec1056e0 0x7f09ec074450 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:08.323 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.925+0000 7f09f20aa700 1 -- 172.21.15.12:0/1642716448 >> 172.21.15.12:0/1642716448 conn(0x7f09ec100c60 msgr2=0x7f09ec1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:08.323 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.925+0000 7f09f20aa700 1 -- 172.21.15.12:0/1642716448 shutdown_connections 2024-05-08T23:39:08.323 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:07.925+0000 7f09f20aa700 1 -- 172.21.15.12:0/1642716448 wait complete. 2024-05-08T23:39:08.323 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (1/15)... 2024-05-08T23:39:08.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:08 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1642716448' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T23:39:08.667 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:08.196+0000 7ff339a1d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:39:09.218 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:08.936+0000 7ff339a1d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:39:09.219 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:09.221+0000 7ff339a1d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:39:09.621 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:09.318+0000 7ff339a1d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:39:09.621 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:09.443+0000 7ff339a1d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:39:09.621 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:09.624+0000 7ff339a1d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:39:09.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:09.769+0000 7ff339a1d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:39:09.917 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:09.865+0000 7ff339a1d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:39:10.666 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:10 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:10.289+0000 7ff339a1d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:39:11.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:10 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/872218183' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T23:39:11.029 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:10 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:10.747+0000 7ff339a1d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:39:11.029 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:10 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:10.970+0000 7ff339a1d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:39:11.104 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:11.104 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:39:11.104 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "00f8fb28-0d94-11ef-bc97-c7b262605968", 2024-05-08T23:39:11.104 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T23:39:11.104 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T23:39:11.104 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T23:39:11.104 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T23:39:11.104 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:11.104 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:11.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T23:39:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T23:39:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T23:39:01.617832+0000", 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T23:39:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.669+0000 7f616ea4d700 1 Processor -- start 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.669+0000 7f616ea4d700 1 -- start start 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.669+0000 7f616ea4d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f6168106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.669+0000 7f616ea4d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6168107170 con 0x7f6168106780 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.669+0000 7f6167fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f6168106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.669+0000 7f6167fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f6168106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47754/0 (socket says 172.21.15.12:47754) 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.669+0000 7f6167fff700 1 -- 172.21.15.12:0/60593047 learned_addr learned my addr 172.21.15.12:0/60593047 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.670+0000 7f6167fff700 1 -- 172.21.15.12:0/60593047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168107980 con 0x7f6168106780 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.670+0000 7f6167fff700 1 --2- 172.21.15.12:0/60593047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f6168106ba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61580098a0 tx=0x7f6158009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10e4a33de1d865f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.670+0000 7f6166ffd700 1 -- 172.21.15.12:0/60593047 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6158004160 con 0x7f6168106780 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.670+0000 7f6166ffd700 1 -- 172.21.15.12:0/60593047 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61580042c0 con 0x7f6168106780 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.670+0000 7f6166ffd700 1 -- 172.21.15.12:0/60593047 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6158011a00 con 0x7f6168106780 2024-05-08T23:39:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.671+0000 7f616ea4d700 1 -- 172.21.15.12:0/60593047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 msgr2=0x7f6168106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.671+0000 7f616ea4d700 1 --2- 172.21.15.12:0/60593047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f6168106ba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61580098a0 tx=0x7f6158009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.671+0000 7f616ea4d700 1 -- 172.21.15.12:0/60593047 shutdown_connections 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.671+0000 7f616ea4d700 1 --2- 172.21.15.12:0/60593047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f6168106ba0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.671+0000 7f616ea4d700 1 -- 172.21.15.12:0/60593047 >> 172.21.15.12:0/60593047 conn(0x7f6168101d00 msgr2=0x7f6168104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.671+0000 7f616ea4d700 1 -- 172.21.15.12:0/60593047 shutdown_connections 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.671+0000 7f616ea4d700 1 -- 172.21.15.12:0/60593047 wait complete. 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.672+0000 7f616ea4d700 1 Processor -- start 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.672+0000 7f616ea4d700 1 -- start start 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.672+0000 7f616ea4d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f616807bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.672+0000 7f616ea4d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616807c050 con 0x7f6168106780 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.672+0000 7f6167fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f616807bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.672+0000 7f6167fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f616807bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47756/0 (socket says 172.21.15.12:47756) 2024-05-08T23:39:11.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.672+0000 7f6167fff700 1 -- 172.21.15.12:0/872218183 learned_addr learned my addr 172.21.15.12:0/872218183 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.673+0000 7f6167fff700 1 -- 172.21.15.12:0/872218183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6158009550 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.673+0000 7f6167fff700 1 --2- 172.21.15.12:0/872218183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f616807bb10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6158009fb0 tx=0x7f6158011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.673+0000 7f61657fa700 1 -- 172.21.15.12:0/872218183 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6158003a40 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.673+0000 7f61657fa700 1 -- 172.21.15.12:0/872218183 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6158003ba0 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.673+0000 7f616ea4d700 1 -- 172.21.15.12:0/872218183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616807a290 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.673+0000 7f61657fa700 1 -- 172.21.15.12:0/872218183 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f615801a750 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.673+0000 7f616ea4d700 1 -- 172.21.15.12:0/872218183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616807a730 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.673+0000 7f61657fa700 1 -- 172.21.15.12:0/872218183 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f615801acd0 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.673+0000 7f61657fa700 1 -- 172.21.15.12:0/872218183 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f61580193c0 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.674+0000 7f616ea4d700 1 -- 172.21.15.12:0/872218183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61680625d0 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.677+0000 7f61657fa700 1 -- 172.21.15.12:0/872218183 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f615802de40 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.754+0000 7f616ea4d700 1 -- 172.21.15.12:0/872218183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f616810ac00 con 0x7f6168106780 2024-05-08T23:39:11.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.754+0000 7f61657fa700 1 -- 172.21.15.12:0/872218183 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f6158024b60 con 0x7f6168106780 2024-05-08T23:39:11.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.755+0000 7f616ea4d700 1 -- 172.21.15.12:0/872218183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 msgr2=0x7f616807bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:11.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.755+0000 7f616ea4d700 1 --2- 172.21.15.12:0/872218183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f616807bb10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6158009fb0 tx=0x7f6158011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:11.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.756+0000 7f616ea4d700 1 -- 172.21.15.12:0/872218183 shutdown_connections 2024-05-08T23:39:11.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.756+0000 7f616ea4d700 1 --2- 172.21.15.12:0/872218183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6168106780 0x7f616807bb10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:11.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.756+0000 7f616ea4d700 1 -- 172.21.15.12:0/872218183 >> 172.21.15.12:0/872218183 conn(0x7f6168101d00 msgr2=0x7f6168102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:11.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.756+0000 7f616ea4d700 1 -- 172.21.15.12:0/872218183 shutdown_connections 2024-05-08T23:39:11.112 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:10.756+0000 7f616ea4d700 1 -- 172.21.15.12:0/872218183 wait complete. 2024-05-08T23:39:11.112 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (2/15)... 2024-05-08T23:39:11.416 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:11.075+0000 7ff339a1d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:39:12.230 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:11.955+0000 7ff339a1d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:39:12.231 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:12 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:12.065+0000 7ff339a1d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:39:12.666 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:12 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:12.233+0000 7ff339a1d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:39:13.216 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:12 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:12.958+0000 7ff339a1d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:39:13.216 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:13 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:13.172+0000 7ff339a1d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:39:13.595 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:13 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:13.396+0000 7ff339a1d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:39:13.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:13 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1241705246' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T23:39:13.953 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:13.954 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:39:13.954 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "00f8fb28-0d94-11ef-bc97-c7b262605968", 2024-05-08T23:39:13.954 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T23:39:13.954 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T23:39:13.954 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T23:39:13.954 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T23:39:13.954 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:13.954 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T23:39:13.954 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T23:39:13.954 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T23:39:13.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T23:39:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T23:39:13.957 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T23:39:13.957 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T23:39:13.957 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:13.957 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T23:39:13.957 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:13.957 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T23:39:13.957 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T23:39:13.957 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:13.957 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T23:39:13.957 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T23:39:01.617832+0000", 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:39:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.487+0000 7f55ab4d4700 1 Processor -- start 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.487+0000 7f55ab4d4700 1 -- start start 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.488+0000 7f55ab4d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a4105b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.488+0000 7f55ab4d4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41060d0 con 0x7f55a41056e0 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.488+0000 7f55a9270700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a4105b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.488+0000 7f55a9270700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a4105b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47770/0 (socket says 172.21.15.12:47770) 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.488+0000 7f55a9270700 1 -- 172.21.15.12:0/165397296 learned_addr learned my addr 172.21.15.12:0/165397296 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.488+0000 7f55a9270700 1 -- 172.21.15.12:0/165397296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a41068e0 con 0x7f55a41056e0 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.488+0000 7f55a9270700 1 --2- 172.21.15.12:0/165397296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a4105b00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55a00098a0 tx=0x7f55a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad287319c0a1808 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.488+0000 7f559bfff700 1 -- 172.21.15.12:0/165397296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55a0004160 con 0x7f55a41056e0 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.489+0000 7f559bfff700 1 -- 172.21.15.12:0/165397296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f55a00042c0 con 0x7f55a41056e0 2024-05-08T23:39:13.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.489+0000 7f559bfff700 1 -- 172.21.15.12:0/165397296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55a0011a00 con 0x7f55a41056e0 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.489+0000 7f55ab4d4700 1 -- 172.21.15.12:0/165397296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 msgr2=0x7f55a4105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.489+0000 7f55ab4d4700 1 --2- 172.21.15.12:0/165397296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a4105b00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55a00098a0 tx=0x7f55a0009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.489+0000 7f55ab4d4700 1 -- 172.21.15.12:0/165397296 shutdown_connections 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.489+0000 7f55ab4d4700 1 --2- 172.21.15.12:0/165397296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a4105b00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.489+0000 7f55ab4d4700 1 -- 172.21.15.12:0/165397296 >> 172.21.15.12:0/165397296 conn(0x7f55a4100c60 msgr2=0x7f55a41030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.489+0000 7f55ab4d4700 1 -- 172.21.15.12:0/165397296 shutdown_connections 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.489+0000 7f55ab4d4700 1 -- 172.21.15.12:0/165397296 wait complete. 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.490+0000 7f55ab4d4700 1 Processor -- start 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.490+0000 7f55ab4d4700 1 -- start start 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.490+0000 7f55ab4d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.490+0000 7f55ab4d4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a419ab30 con 0x7f55a41056e0 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.490+0000 7f55a9270700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.490+0000 7f55a9270700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47776/0 (socket says 172.21.15.12:47776) 2024-05-08T23:39:13.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.490+0000 7f55a9270700 1 -- 172.21.15.12:0/1241705246 learned_addr learned my addr 172.21.15.12:0/1241705246 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.491+0000 7f55a9270700 1 -- 172.21.15.12:0/1241705246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a0009550 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.491+0000 7f55a9270700 1 --2- 172.21.15.12:0/1241705246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a419a5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009130 tx=0x7f55a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.491+0000 7f559a7fc700 1 -- 172.21.15.12:0/1241705246 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55a00038b0 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.491+0000 7f559a7fc700 1 -- 172.21.15.12:0/1241705246 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f55a0003a10 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.491+0000 7f55ab4d4700 1 -- 172.21.15.12:0/1241705246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a419ad30 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.491+0000 7f559a7fc700 1 -- 172.21.15.12:0/1241705246 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55a0019550 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.491+0000 7f55ab4d4700 1 -- 172.21.15.12:0/1241705246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a419b1d0 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.491+0000 7f559a7fc700 1 -- 172.21.15.12:0/1241705246 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f55a0019b30 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.491+0000 7f559a7fc700 1 -- 172.21.15.12:0/1241705246 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f55a002de30 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.492+0000 7f55ab4d4700 1 -- 172.21.15.12:0/1241705246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a41946c0 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.495+0000 7f559a7fc700 1 -- 172.21.15.12:0/1241705246 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f55a0003b80 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.571+0000 7f55ab4d4700 1 -- 172.21.15.12:0/1241705246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f55a4109b10 con 0x7f55a41056e0 2024-05-08T23:39:13.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.572+0000 7f559a7fc700 1 -- 172.21.15.12:0/1241705246 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f55a001a960 con 0x7f55a41056e0 2024-05-08T23:39:13.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.573+0000 7f55ab4d4700 1 -- 172.21.15.12:0/1241705246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 msgr2=0x7f55a419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:13.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.573+0000 7f55ab4d4700 1 --2- 172.21.15.12:0/1241705246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a419a5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009130 tx=0x7f55a0011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:13.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.573+0000 7f55ab4d4700 1 -- 172.21.15.12:0/1241705246 shutdown_connections 2024-05-08T23:39:13.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.573+0000 7f55ab4d4700 1 --2- 172.21.15.12:0/1241705246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55a41056e0 0x7f55a419a5f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:13.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.573+0000 7f55ab4d4700 1 -- 172.21.15.12:0/1241705246 >> 172.21.15.12:0/1241705246 conn(0x7f55a4100c60 msgr2=0x7f55a4101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:13.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.573+0000 7f55ab4d4700 1 -- 172.21.15.12:0/1241705246 shutdown_connections 2024-05-08T23:39:13.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:13.573+0000 7f55ab4d4700 1 -- 172.21.15.12:0/1241705246 wait complete. 2024-05-08T23:39:13.962 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (3/15)... 2024-05-08T23:39:14.793 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:14 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:14.472+0000 7ff339a1d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:39:15.110 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:14 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:14.796+0000 7ff339a1d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:39:15.110 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:14 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:14.895+0000 7ff339a1d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:39:15.416 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:15 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:15.112+0000 7ff339a1d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:39:15.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: Activating manager daemon x 2024-05-08T23:39:15.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: mgrmap e2: x(active, starting, since 0.00326903s) 2024-05-08T23:39:15.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: from='mgr.14100 172.21.15.12:0/1406644389' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:39:15.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: from='mgr.14100 172.21.15.12:0/1406644389' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:39:15.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: from='mgr.14100 172.21.15.12:0/1406644389' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:39:15.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: from='mgr.14100 172.21.15.12:0/1406644389' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:39:15.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: from='mgr.14100 172.21.15.12:0/1406644389' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:39:15.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: Manager daemon x is now available 2024-05-08T23:39:15.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: from='mgr.14100 172.21.15.12:0/1406644389' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:39:15.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: from='mgr.14100 172.21.15.12:0/1406644389' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:39:15.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: from='mgr.14100 172.21.15.12:0/1406644389' entity='mgr.x' 2024-05-08T23:39:15.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: from='mgr.14100 172.21.15.12:0/1406644389' entity='mgr.x' 2024-05-08T23:39:15.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:15 smithi012 ceph-mon[42503]: from='mgr.14100 172.21.15.12:0/1406644389' entity='mgr.x' 2024-05-08T23:39:17.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:17 smithi012 ceph-mon[42503]: mgrmap e3: x(active, since 1.00706s) 2024-05-08T23:39:18.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:18 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2555835238' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T23:39:19.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:19 smithi012 ceph-mon[42503]: mgrmap e4: x(active, since 3s) 2024-05-08T23:39:20.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:20.036 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:39:20.036 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "00f8fb28-0d94-11ef-bc97-c7b262605968", 2024-05-08T23:39:20.036 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-08T23:39:20.037 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T23:39:20.038 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T23:39:20.039 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:39:20.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T23:39:01.617832+0000", 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.141+0000 7efc8977c700 1 Processor -- start 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.141+0000 7efc8977c700 1 -- start start 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.141+0000 7efc8977c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84109670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.141+0000 7efc8977c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc84079e60 con 0x7efc84107280 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.142+0000 7efc82ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.142+0000 7efc82ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84109670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47834/0 (socket says 172.21.15.12:47834) 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.142+0000 7efc82ffd700 1 -- 172.21.15.12:0/391893302 learned_addr learned my addr 172.21.15.12:0/391893302 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:20.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.142+0000 7efc82ffd700 1 -- 172.21.15.12:0/391893302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc84109bb0 con 0x7efc84107280 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.142+0000 7efc82ffd700 1 --2- 172.21.15.12:0/391893302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84109670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efc740098a0 tx=0x7efc74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d64b76e68fbe76d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.142+0000 7efc827fc700 1 -- 172.21.15.12:0/391893302 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc74004160 con 0x7efc84107280 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.142+0000 7efc827fc700 1 -- 172.21.15.12:0/391893302 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efc740042c0 con 0x7efc84107280 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.142+0000 7efc827fc700 1 -- 172.21.15.12:0/391893302 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc74011a00 con 0x7efc84107280 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.143+0000 7efc8977c700 1 -- 172.21.15.12:0/391893302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 msgr2=0x7efc84109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.143+0000 7efc8977c700 1 --2- 172.21.15.12:0/391893302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84109670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efc740098a0 tx=0x7efc74009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.143+0000 7efc8977c700 1 -- 172.21.15.12:0/391893302 shutdown_connections 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.143+0000 7efc8977c700 1 --2- 172.21.15.12:0/391893302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84109670 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.143+0000 7efc8977c700 1 -- 172.21.15.12:0/391893302 >> 172.21.15.12:0/391893302 conn(0x7efc84100cd0 msgr2=0x7efc841030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.143+0000 7efc8977c700 1 -- 172.21.15.12:0/391893302 shutdown_connections 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.143+0000 7efc8977c700 1 -- 172.21.15.12:0/391893302 wait complete. 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.144+0000 7efc8977c700 1 Processor -- start 2024-05-08T23:39:20.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.144+0000 7efc8977c700 1 -- start start 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.144+0000 7efc8977c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.144+0000 7efc8977c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc84106ff0 con 0x7efc84107280 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.144+0000 7efc82ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.144+0000 7efc82ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47840/0 (socket says 172.21.15.12:47840) 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.144+0000 7efc82ffd700 1 -- 172.21.15.12:0/2555835238 learned_addr learned my addr 172.21.15.12:0/2555835238 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.145+0000 7efc82ffd700 1 -- 172.21.15.12:0/2555835238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc74009550 con 0x7efc84107280 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.145+0000 7efc82ffd700 1 --2- 172.21.15.12:0/2555835238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84106ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc74009130 tx=0x7efc74011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.145+0000 7efc80ff9700 1 -- 172.21.15.12:0/2555835238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc740038b0 con 0x7efc84107280 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.145+0000 7efc8977c700 1 -- 172.21.15.12:0/2555835238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc84105230 con 0x7efc84107280 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.145+0000 7efc80ff9700 1 -- 172.21.15.12:0/2555835238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efc74003a10 con 0x7efc84107280 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.145+0000 7efc80ff9700 1 -- 172.21.15.12:0/2555835238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc74019550 con 0x7efc84107280 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.145+0000 7efc8977c700 1 -- 172.21.15.12:0/2555835238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc841056d0 con 0x7efc84107280 2024-05-08T23:39:20.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.146+0000 7efc80ff9700 1 -- 172.21.15.12:0/2555835238 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7efc7401a430 con 0x7efc84107280 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.146+0000 7efc8977c700 1 -- 172.21.15.12:0/2555835238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8410a950 con 0x7efc84107280 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.146+0000 7efc80ff9700 1 --2- 172.21.15.12:0/2555835238 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7efc70038570 0x7efc7003aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.146+0000 7efc80ff9700 1 -- 172.21.15.12:0/2555835238 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efc7404c1b0 con 0x7efc84107280 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.147+0000 7efc7a7fc700 1 --2- 172.21.15.12:0/2555835238 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7efc70038570 0x7efc7003aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.147+0000 7efc7a7fc700 1 --2- 172.21.15.12:0/2555835238 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7efc70038570 0x7efc7003aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc6c006fd0 tx=0x7efc6c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.152+0000 7efc80ff9700 1 -- 172.21.15.12:0/2555835238 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efc74003b80 con 0x7efc84107280 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.450+0000 7efc8977c700 1 -- 172.21.15.12:0/2555835238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efc8410abc0 con 0x7efc84107280 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.450+0000 7efc80ff9700 1 -- 172.21.15.12:0/2555835238 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7efc74023b80 con 0x7efc84107280 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.452+0000 7efc8977c700 1 -- 172.21.15.12:0/2555835238 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7efc70038570 msgr2=0x7efc7003aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.452+0000 7efc8977c700 1 --2- 172.21.15.12:0/2555835238 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7efc70038570 0x7efc7003aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc6c006fd0 tx=0x7efc6c006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.452+0000 7efc8977c700 1 -- 172.21.15.12:0/2555835238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 msgr2=0x7efc84106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.452+0000 7efc8977c700 1 --2- 172.21.15.12:0/2555835238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84106ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc74009130 tx=0x7efc74011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:20.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.453+0000 7efc8977c700 1 -- 172.21.15.12:0/2555835238 shutdown_connections 2024-05-08T23:39:20.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.453+0000 7efc8977c700 1 --2- 172.21.15.12:0/2555835238 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7efc70038570 0x7efc7003aa30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:20.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.453+0000 7efc8977c700 1 --2- 172.21.15.12:0/2555835238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84107280 0x7efc84106ab0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:20.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.453+0000 7efc8977c700 1 -- 172.21.15.12:0/2555835238 >> 172.21.15.12:0/2555835238 conn(0x7efc84100cd0 msgr2=0x7efc841018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:20.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.453+0000 7efc8977c700 1 -- 172.21.15.12:0/2555835238 shutdown_connections 2024-05-08T23:39:20.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:17.453+0000 7efc8977c700 1 -- 172.21.15.12:0/2555835238 wait complete. 2024-05-08T23:39:20.045 INFO:teuthology.orchestra.run.smithi012.stdout:mgr is available 2024-05-08T23:39:21.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:21 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/437889267' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T23:39:23.465 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:23.465 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T23:39:23.465 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:39:23.465 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T23:39:23.465 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T23:39:23.465 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T23:39:23.465 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.177+0000 7f468c764700 1 Processor -- start 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.177+0000 7f468c764700 1 -- start start 2024-05-08T23:39:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.177+0000 7f468c764700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f4684105ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.177+0000 7f468c764700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46841060a0 con 0x7f46841056b0 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.177+0000 7f468a500700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f4684105ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.177+0000 7f468a500700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f4684105ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56334/0 (socket says 172.21.15.12:56334) 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.177+0000 7f468a500700 1 -- 172.21.15.12:0/620058400 learned_addr learned my addr 172.21.15.12:0/620058400 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.178+0000 7f468a500700 1 -- 172.21.15.12:0/620058400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46841068b0 con 0x7f46841056b0 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.178+0000 7f468a500700 1 --2- 172.21.15.12:0/620058400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f4684105ad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f46800098a0 tx=0x7f4680009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3bbbf53a7667dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.178+0000 7f46894fe700 1 -- 172.21.15.12:0/620058400 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4680004160 con 0x7f46841056b0 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.178+0000 7f46894fe700 1 -- 172.21.15.12:0/620058400 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46800042c0 con 0x7f46841056b0 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.178+0000 7f46894fe700 1 -- 172.21.15.12:0/620058400 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4680011a00 con 0x7f46841056b0 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.179+0000 7f468c764700 1 -- 172.21.15.12:0/620058400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 msgr2=0x7f4684105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.179+0000 7f468c764700 1 --2- 172.21.15.12:0/620058400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f4684105ad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f46800098a0 tx=0x7f4680009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.179+0000 7f468c764700 1 -- 172.21.15.12:0/620058400 shutdown_connections 2024-05-08T23:39:23.467 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.179+0000 7f468c764700 1 --2- 172.21.15.12:0/620058400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f4684105ad0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.179+0000 7f468c764700 1 -- 172.21.15.12:0/620058400 >> 172.21.15.12:0/620058400 conn(0x7f4684100c70 msgr2=0x7f4684103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.179+0000 7f468c764700 1 -- 172.21.15.12:0/620058400 shutdown_connections 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.179+0000 7f468c764700 1 -- 172.21.15.12:0/620058400 wait complete. 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.179+0000 7f468c764700 1 Processor -- start 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.179+0000 7f468c764700 1 -- start start 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f468c764700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f468419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f468c764700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468419cca0 con 0x7f46841056b0 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f468a500700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f468419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f468a500700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f468419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56342/0 (socket says 172.21.15.12:56342) 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f468a500700 1 -- 172.21.15.12:0/437889267 learned_addr learned my addr 172.21.15.12:0/437889267 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f468a500700 1 -- 172.21.15.12:0/437889267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4680009550 con 0x7f46841056b0 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f468a500700 1 --2- 172.21.15.12:0/437889267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f468419c760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4680009fb0 tx=0x7f4680011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f467b7fe700 1 -- 172.21.15.12:0/437889267 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4680003a40 con 0x7f46841056b0 2024-05-08T23:39:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f468c764700 1 -- 172.21.15.12:0/437889267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468419cea0 con 0x7f46841056b0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f467b7fe700 1 -- 172.21.15.12:0/437889267 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4680003ba0 con 0x7f46841056b0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f467b7fe700 1 -- 172.21.15.12:0/437889267 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f468001a750 con 0x7f46841056b0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.180+0000 7f468c764700 1 -- 172.21.15.12:0/437889267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468419d340 con 0x7f46841056b0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.181+0000 7f467b7fe700 1 -- 172.21.15.12:0/437889267 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f468001a9d0 con 0x7f46841056b0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.181+0000 7f467b7fe700 1 --2- 172.21.15.12:0/437889267 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f4670038660 0x7f467003ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.181+0000 7f468c764700 1 -- 172.21.15.12:0/437889267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4684196a90 con 0x7f46841056b0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.182+0000 7f467b7fe700 1 -- 172.21.15.12:0/437889267 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f468004c350 con 0x7f46841056b0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.182+0000 7f4689cff700 1 --2- 172.21.15.12:0/437889267 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f4670038660 0x7f467003ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.182+0000 7f4689cff700 1 --2- 172.21.15.12:0/437889267 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f4670038660 0x7f467003ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4674006fd0 tx=0x7f4674006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.187+0000 7f467b7fe700 1 -- 172.21.15.12:0/437889267 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4680018b80 con 0x7f46841056b0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.397+0000 7f468c764700 1 -- 172.21.15.12:0/437889267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4684109ae0 con 0x7f46841056b0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.398+0000 7f467b7fe700 1 -- 172.21.15.12:0/437889267 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f4680018350 con 0x7f46841056b0 2024-05-08T23:39:23.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.401+0000 7f468c764700 1 -- 172.21.15.12:0/437889267 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f4670038660 msgr2=0x7f467003ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:23.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.401+0000 7f468c764700 1 --2- 172.21.15.12:0/437889267 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f4670038660 0x7f467003ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4674006fd0 tx=0x7f4674006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:23.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.401+0000 7f468c764700 1 -- 172.21.15.12:0/437889267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 msgr2=0x7f468419c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:23.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.401+0000 7f468c764700 1 --2- 172.21.15.12:0/437889267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f468419c760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4680009fb0 tx=0x7f4680011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:23.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.401+0000 7f468c764700 1 -- 172.21.15.12:0/437889267 shutdown_connections 2024-05-08T23:39:23.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.401+0000 7f468c764700 1 --2- 172.21.15.12:0/437889267 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f4670038660 0x7f467003ab20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:23.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.401+0000 7f468c764700 1 --2- 172.21.15.12:0/437889267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46841056b0 0x7f468419c760 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:23.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.401+0000 7f468c764700 1 -- 172.21.15.12:0/437889267 >> 172.21.15.12:0/437889267 conn(0x7f4684100c70 msgr2=0x7f4684101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:23.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.401+0000 7f468c764700 1 -- 172.21.15.12:0/437889267 shutdown_connections 2024-05-08T23:39:23.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:21.401+0000 7f468c764700 1 -- 172.21.15.12:0/437889267 wait complete. 2024-05-08T23:39:23.470 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling cephadm module... 2024-05-08T23:39:25.009 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:24 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/4023372159' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T23:39:25.009 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:24 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: ignoring --setuser ceph since I am not root 2024-05-08T23:39:25.009 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:24 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: ignoring --setgroup ceph since I am not root 2024-05-08T23:39:25.020 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.360+0000 7f150cf09700 1 Processor -- start 2024-05-08T23:39:25.020 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.360+0000 7f150cf09700 1 -- start start 2024-05-08T23:39:25.020 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.360+0000 7f150cf09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f1508105b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:25.020 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.360+0000 7f150cf09700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081060e0 con 0x7f15081056f0 2024-05-08T23:39:25.020 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.360+0000 7f150659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f1508105b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:25.020 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.361+0000 7f150659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f1508105b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56354/0 (socket says 172.21.15.12:56354) 2024-05-08T23:39:25.020 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.361+0000 7f150659c700 1 -- 172.21.15.12:0/399713887 learned_addr learned my addr 172.21.15.12:0/399713887 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.361+0000 7f150659c700 1 -- 172.21.15.12:0/399713887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15081068f0 con 0x7f15081056f0 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.361+0000 7f150659c700 1 --2- 172.21.15.12:0/399713887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f1508105b10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f14f800bb40 tx=0x7f14f800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2c6305a783cff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.361+0000 7f150559a700 1 -- 172.21.15.12:0/399713887 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14f800dbe0 con 0x7f15081056f0 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.361+0000 7f150559a700 1 -- 172.21.15.12:0/399713887 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14f80044d0 con 0x7f15081056f0 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.361+0000 7f150559a700 1 -- 172.21.15.12:0/399713887 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14f8010470 con 0x7f15081056f0 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.362+0000 7f150cf09700 1 -- 172.21.15.12:0/399713887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 msgr2=0x7f1508105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.362+0000 7f150cf09700 1 --2- 172.21.15.12:0/399713887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f1508105b10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f14f800bb40 tx=0x7f14f800be50 comp rx=0 tx=0).stop 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.362+0000 7f150cf09700 1 -- 172.21.15.12:0/399713887 shutdown_connections 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.362+0000 7f150cf09700 1 --2- 172.21.15.12:0/399713887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f1508105b10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.362+0000 7f150cf09700 1 -- 172.21.15.12:0/399713887 >> 172.21.15.12:0/399713887 conn(0x7f1508100c90 msgr2=0x7f15081030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.362+0000 7f150cf09700 1 -- 172.21.15.12:0/399713887 shutdown_connections 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.362+0000 7f150cf09700 1 -- 172.21.15.12:0/399713887 wait complete. 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.363+0000 7f150cf09700 1 Processor -- start 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.363+0000 7f150cf09700 1 -- start start 2024-05-08T23:39:25.021 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.363+0000 7f150cf09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f15081983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.363+0000 7f150cf09700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081988f0 con 0x7f15081056f0 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.363+0000 7f150659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f15081983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.363+0000 7f150659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f15081983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56368/0 (socket says 172.21.15.12:56368) 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.363+0000 7f150659c700 1 -- 172.21.15.12:0/4023372159 learned_addr learned my addr 172.21.15.12:0/4023372159 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.364+0000 7f150659c700 1 -- 172.21.15.12:0/4023372159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f800b7f0 con 0x7f15081056f0 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.364+0000 7f150659c700 1 --2- 172.21.15.12:0/4023372159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f15081983b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f14f800bfd0 tx=0x7f14f8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.364+0000 7f14f77fe700 1 -- 172.21.15.12:0/4023372159 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14f801e070 con 0x7f15081056f0 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.364+0000 7f14f77fe700 1 -- 172.21.15.12:0/4023372159 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14f80042a0 con 0x7f15081056f0 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.364+0000 7f150cf09700 1 -- 172.21.15.12:0/4023372159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1508198af0 con 0x7f15081056f0 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.364+0000 7f14f77fe700 1 -- 172.21.15.12:0/4023372159 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14f8022ac0 con 0x7f15081056f0 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.364+0000 7f150cf09700 1 -- 172.21.15.12:0/4023372159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1508198f90 con 0x7f15081056f0 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.365+0000 7f14f77fe700 1 -- 172.21.15.12:0/4023372159 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f14f802c440 con 0x7f15081056f0 2024-05-08T23:39:25.022 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.365+0000 7f150cf09700 1 -- 172.21.15.12:0/4023372159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1508191f80 con 0x7f15081056f0 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.365+0000 7f14f77fe700 1 --2- 172.21.15.12:0/4023372159 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f14f00386b0 0x7f14f003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.365+0000 7f14f77fe700 1 -- 172.21.15.12:0/4023372159 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f14f804ecf0 con 0x7f15081056f0 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.365+0000 7f1505d9b700 1 --2- 172.21.15.12:0/4023372159 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f14f00386b0 0x7f14f003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.366+0000 7f1505d9b700 1 --2- 172.21.15.12:0/4023372159 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f14f00386b0 0x7f14f003ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006fd0 tx=0x7f14fc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.371+0000 7f14f77fe700 1 -- 172.21.15.12:0/4023372159 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f14f80193f0 con 0x7f15081056f0 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.643+0000 7f150cf09700 1 -- 172.21.15.12:0/4023372159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f1508073a00 con 0x7f15081056f0 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.696+0000 7f14f77fe700 1 -- 172.21.15.12:0/4023372159 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f14f8016d40 con 0x7f15081056f0 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.698+0000 7f14f77fe700 1 -- 172.21.15.12:0/4023372159 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f14f804ea50 con 0x7f15081056f0 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.701+0000 7f150cf09700 1 -- 172.21.15.12:0/4023372159 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f14f00386b0 msgr2=0x7f14f003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.701+0000 7f150cf09700 1 --2- 172.21.15.12:0/4023372159 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f14f00386b0 0x7f14f003ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006fd0 tx=0x7f14fc006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.701+0000 7f150cf09700 1 -- 172.21.15.12:0/4023372159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 msgr2=0x7f15081983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.701+0000 7f150cf09700 1 --2- 172.21.15.12:0/4023372159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f15081983b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f14f800bfd0 tx=0x7f14f8003d40 comp rx=0 tx=0).stop 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.701+0000 7f150cf09700 1 -- 172.21.15.12:0/4023372159 shutdown_connections 2024-05-08T23:39:25.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.701+0000 7f150cf09700 1 --2- 172.21.15.12:0/4023372159 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f14f00386b0 0x7f14f003ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:25.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.701+0000 7f150cf09700 1 --2- 172.21.15.12:0/4023372159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15081056f0 0x7f15081983b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:25.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.701+0000 7f150cf09700 1 -- 172.21.15.12:0/4023372159 >> 172.21.15.12:0/4023372159 conn(0x7f1508100c90 msgr2=0x7f150818f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:25.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.701+0000 7f150cf09700 1 -- 172.21.15.12:0/4023372159 shutdown_connections 2024-05-08T23:39:25.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:24.701+0000 7f150cf09700 1 -- 172.21.15.12:0/4023372159 wait complete. 2024-05-08T23:39:25.298 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:25.194+0000 7fbb71edc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:39:25.298 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:25.301+0000 7fbb71edc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:39:25.666 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:25.403+0000 7fbb71edc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:39:25.666 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:25.498+0000 7fbb71edc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:39:25.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:39:25.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-08T23:39:25.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T23:39:25.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T23:39:25.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T23:39:25.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:39:25.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.362+0000 7f419b2fa700 1 Processor -- start 2024-05-08T23:39:25.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.362+0000 7f419b2fa700 1 -- start start 2024-05-08T23:39:25.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.362+0000 7f419b2fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f4194105b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:25.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.362+0000 7f419b2fa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941060d0 con 0x7f41941056e0 2024-05-08T23:39:25.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.363+0000 7f4199096700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f4194105b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:25.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.363+0000 7f4199096700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f4194105b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56398/0 (socket says 172.21.15.12:56398) 2024-05-08T23:39:25.967 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.363+0000 7f4199096700 1 -- 172.21.15.12:0/2885080210 learned_addr learned my addr 172.21.15.12:0/2885080210 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:25.967 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.363+0000 7f4199096700 1 -- 172.21.15.12:0/2885080210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41941068e0 con 0x7f41941056e0 2024-05-08T23:39:25.967 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.363+0000 7f4199096700 1 --2- 172.21.15.12:0/2885080210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f4194105b00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f41900098a0 tx=0x7f4190009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd429ca44b92386 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:25.967 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.363+0000 7f418bfff700 1 -- 172.21.15.12:0/2885080210 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4190004160 con 0x7f41941056e0 2024-05-08T23:39:25.967 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.363+0000 7f418bfff700 1 -- 172.21.15.12:0/2885080210 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f41900042c0 con 0x7f41941056e0 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.363+0000 7f418bfff700 1 -- 172.21.15.12:0/2885080210 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4190011a00 con 0x7f41941056e0 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.364+0000 7f419b2fa700 1 -- 172.21.15.12:0/2885080210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 msgr2=0x7f4194105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.364+0000 7f419b2fa700 1 --2- 172.21.15.12:0/2885080210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f4194105b00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f41900098a0 tx=0x7f4190009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.364+0000 7f419b2fa700 1 -- 172.21.15.12:0/2885080210 shutdown_connections 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.364+0000 7f419b2fa700 1 --2- 172.21.15.12:0/2885080210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f4194105b00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.364+0000 7f419b2fa700 1 -- 172.21.15.12:0/2885080210 >> 172.21.15.12:0/2885080210 conn(0x7f4194100c60 msgr2=0x7f41941030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.364+0000 7f419b2fa700 1 -- 172.21.15.12:0/2885080210 shutdown_connections 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.364+0000 7f419b2fa700 1 -- 172.21.15.12:0/2885080210 wait complete. 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.365+0000 7f419b2fa700 1 Processor -- start 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.365+0000 7f419b2fa700 1 -- start start 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.365+0000 7f419b2fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f41941906f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.365+0000 7f419b2fa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4194190c30 con 0x7f41941056e0 2024-05-08T23:39:25.968 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.366+0000 7f4199096700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f41941906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.366+0000 7f4199096700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f41941906f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56402/0 (socket says 172.21.15.12:56402) 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.366+0000 7f4199096700 1 -- 172.21.15.12:0/743731301 learned_addr learned my addr 172.21.15.12:0/743731301 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.366+0000 7f4199096700 1 -- 172.21.15.12:0/743731301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190009550 con 0x7f41941056e0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.366+0000 7f4199096700 1 --2- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f41941906f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4190011710 tx=0x7f4190011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.366+0000 7f418a7fc700 1 -- 172.21.15.12:0/743731301 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4190003aa0 con 0x7f41941056e0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.366+0000 7f418a7fc700 1 -- 172.21.15.12:0/743731301 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4190003c00 con 0x7f41941056e0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.366+0000 7f419b2fa700 1 -- 172.21.15.12:0/743731301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419418eec0 con 0x7f41941056e0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.366+0000 7f418a7fc700 1 -- 172.21.15.12:0/743731301 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4190024800 con 0x7f41941056e0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.366+0000 7f419b2fa700 1 -- 172.21.15.12:0/743731301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419418f360 con 0x7f41941056e0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.367+0000 7f418a7fc700 1 -- 172.21.15.12:0/743731301 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f4190024960 con 0x7f41941056e0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.367+0000 7f419b2fa700 1 -- 172.21.15.12:0/743731301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419418fd70 con 0x7f41941056e0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.367+0000 7f418a7fc700 1 --2- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f41800387a0 0x7f418003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.368+0000 7f418a7fc700 1 -- 172.21.15.12:0/743731301 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f419004d2e0 con 0x7f41941056e0 2024-05-08T23:39:25.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.368+0000 7f4198895700 1 -- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f41800387a0 msgr2=0x7f418003ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4231005605 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.368+0000 7f4198895700 1 --2- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f41800387a0 0x7f418003ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.373+0000 7f418a7fc700 1 -- 172.21.15.12:0/743731301 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4190024c10 con 0x7f41941056e0 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.569+0000 7f4198895700 1 -- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f41800387a0 msgr2=0x7f418003ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4231005605 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.569+0000 7f4198895700 1 --2- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f41800387a0 0x7f418003ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.648+0000 7f419b2fa700 1 -- 172.21.15.12:0/743731301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f41941941e0 con 0x7f41941056e0 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.648+0000 7f418a7fc700 1 -- 172.21.15.12:0/743731301 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f4190018350 con 0x7f41941056e0 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.651+0000 7f419b2fa700 1 -- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f41800387a0 msgr2=0x7f418003ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.651+0000 7f419b2fa700 1 --2- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f41800387a0 0x7f418003ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.651+0000 7f419b2fa700 1 -- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 msgr2=0x7f41941906f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.651+0000 7f419b2fa700 1 --2- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f41941906f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4190011710 tx=0x7f4190011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.651+0000 7f419b2fa700 1 -- 172.21.15.12:0/743731301 shutdown_connections 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.651+0000 7f419b2fa700 1 --2- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7f41800387a0 0x7f418003ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:25.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.651+0000 7f419b2fa700 1 --2- 172.21.15.12:0/743731301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41941056e0 0x7f41941906f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:25.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.651+0000 7f419b2fa700 1 -- 172.21.15.12:0/743731301 >> 172.21.15.12:0/743731301 conn(0x7f4194100c60 msgr2=0x7f4194101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:25.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.651+0000 7f419b2fa700 1 -- 172.21.15.12:0/743731301 shutdown_connections 2024-05-08T23:39:25.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:25.651+0000 7f419b2fa700 1 -- 172.21.15.12:0/743731301 wait complete. 2024-05-08T23:39:25.971 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-05-08T23:39:25.971 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 5... 2024-05-08T23:39:26.103 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:25 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/4023372159' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T23:39:26.103 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:25 smithi012 ceph-mon[42503]: mgrmap e5: x(active, since 9s) 2024-05-08T23:39:26.103 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:25 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/743731301' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T23:39:26.416 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:26 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:26.243+0000 7fbb71edc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:39:26.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:26 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:26.533+0000 7fbb71edc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:39:26.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:26 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:26.629+0000 7fbb71edc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:39:26.917 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:26 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:26.752+0000 7fbb71edc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:39:27.416 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:26 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:26.933+0000 7fbb71edc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:39:27.416 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:27 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:27.075+0000 7fbb71edc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:39:27.417 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:27 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:27.171+0000 7fbb71edc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:39:27.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:27 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:27.593+0000 7fbb71edc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:39:28.351 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:28 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:28.033+0000 7fbb71edc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:39:28.352 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:28 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:28.249+0000 7fbb71edc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:39:28.666 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:28 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:28.353+0000 7fbb71edc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:39:29.493 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:29 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:29.219+0000 7fbb71edc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:39:29.493 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:29 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:29.327+0000 7fbb71edc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:39:29.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:29 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:29.495+0000 7fbb71edc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:39:30.632 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:30 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:30.214+0000 7fbb71edc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:39:30.633 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:30 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:30.418+0000 7fbb71edc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:39:30.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:30 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:30.634+0000 7fbb71edc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:39:32.007 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:31 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:31.690+0000 7fbb71edc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:39:32.319 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:32 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:32.009+0000 7fbb71edc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:39:32.320 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:32 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:32.108+0000 7fbb71edc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:39:32.666 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:32 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:32.321+0000 7fbb71edc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:39:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: Active manager daemon x restarted 2024-05-08T23:39:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: Activating manager daemon x 2024-05-08T23:39:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T23:39:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: mgrmap e6: x(active, starting, since 0.00373375s) 2024-05-08T23:39:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:39:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:39:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:39:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:39:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:39:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: Manager daemon x is now available 2024-05-08T23:39:32.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:32.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:32.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:32 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:39:33.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:33 smithi012 ceph-mon[42503]: Found migration_current of "None". Setting to last migration. 2024-05-08T23:39:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:33 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:39:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:33 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:39:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:33 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:39:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:33 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:33 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:33 smithi012 ceph-mon[42503]: mgrmap e7: x(active, since 1.00557s) 2024-05-08T23:39:33.694 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:39:33.694 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.355+0000 7fae382d5700 1 Processor -- start 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.355+0000 7fae382d5700 1 -- start start 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.355+0000 7fae382d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae30105b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.356+0000 7fae382d5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae301060e0 con 0x7fae301056f0 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.356+0000 7fae36071700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae30105b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.356+0000 7fae36071700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae30105b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56404/0 (socket says 172.21.15.12:56404) 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.356+0000 7fae36071700 1 -- 172.21.15.12:0/1209453532 learned_addr learned my addr 172.21.15.12:0/1209453532 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.356+0000 7fae36071700 1 -- 172.21.15.12:0/1209453532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae301068f0 con 0x7fae301056f0 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.356+0000 7fae36071700 1 --2- 172.21.15.12:0/1209453532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae30105b10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae240098a0 tx=0x7fae24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c0a117d221a621 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.356+0000 7fae3506f700 1 -- 172.21.15.12:0/1209453532 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae24004160 con 0x7fae301056f0 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.356+0000 7fae3506f700 1 -- 172.21.15.12:0/1209453532 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae240042c0 con 0x7fae301056f0 2024-05-08T23:39:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.356+0000 7fae3506f700 1 -- 172.21.15.12:0/1209453532 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae24011a00 con 0x7fae301056f0 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.357+0000 7fae382d5700 1 -- 172.21.15.12:0/1209453532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 msgr2=0x7fae30105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.357+0000 7fae382d5700 1 --2- 172.21.15.12:0/1209453532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae30105b10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae240098a0 tx=0x7fae24009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.357+0000 7fae382d5700 1 -- 172.21.15.12:0/1209453532 shutdown_connections 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.357+0000 7fae382d5700 1 --2- 172.21.15.12:0/1209453532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae30105b10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.357+0000 7fae382d5700 1 -- 172.21.15.12:0/1209453532 >> 172.21.15.12:0/1209453532 conn(0x7fae30100c90 msgr2=0x7fae301030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.357+0000 7fae382d5700 1 -- 172.21.15.12:0/1209453532 shutdown_connections 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.357+0000 7fae382d5700 1 -- 172.21.15.12:0/1209453532 wait complete. 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.358+0000 7fae382d5700 1 Processor -- start 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.358+0000 7fae382d5700 1 -- start start 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.358+0000 7fae382d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae3019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.358+0000 7fae382d5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3019ab60 con 0x7fae301056f0 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.358+0000 7fae36071700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae3019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.358+0000 7fae36071700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae3019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56416/0 (socket says 172.21.15.12:56416) 2024-05-08T23:39:33.696 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.358+0000 7fae36071700 1 -- 172.21.15.12:0/3087101546 learned_addr learned my addr 172.21.15.12:0/3087101546 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.359+0000 7fae36071700 1 -- 172.21.15.12:0/3087101546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae24009550 con 0x7fae301056f0 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.359+0000 7fae36071700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae3019a620 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fae24011710 tx=0x7fae24011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.359+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae24003aa0 con 0x7fae301056f0 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.359+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae24003c00 con 0x7fae301056f0 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.359+0000 7fae382d5700 1 -- 172.21.15.12:0/3087101546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3019ad60 con 0x7fae301056f0 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.359+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae24024800 con 0x7fae301056f0 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.359+0000 7fae382d5700 1 -- 172.21.15.12:0/3087101546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3019b200 con 0x7fae301056f0 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.360+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fae2401abd0 con 0x7fae301056f0 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.360+0000 7fae237fe700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 0x7fae1c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.360+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 --> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae1c03b2d0 con 0x7fae1c038700 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.360+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fae2404c310 con 0x7fae301056f0 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.360+0000 7fae35870700 1 -- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 msgr2=0x7fae1c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4231005605 2024-05-08T23:39:33.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.360+0000 7fae35870700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 0x7fae1c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.561+0000 7fae35870700 1 -- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 msgr2=0x7fae1c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4231005605 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.561+0000 7fae35870700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 0x7fae1c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.962+0000 7fae35870700 1 -- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 msgr2=0x7fae1c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4231005605 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:26.962+0000 7fae35870700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 0x7fae1c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:27.763+0000 7fae35870700 1 -- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 msgr2=0x7fae1c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4231005605 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:27.763+0000 7fae35870700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 0x7fae1c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:29.363+0000 7fae35870700 1 -- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 msgr2=0x7fae1c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4231005605 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:29.363+0000 7fae35870700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 0x7fae1c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:32.337+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7fae24024c10 con 0x7fae301056f0 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:32.337+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 msgr2=0x7fae1c03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:32.337+0000 7fae237fe700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 0x7fae1c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.339+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fae2404e240 con 0x7fae301056f0 2024-05-08T23:39:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.339+0000 7fae237fe700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fae1c03bfc0 0x7fae1c03e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.339+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 --> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae1c03b2d0 con 0x7fae1c03bfc0 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.340+0000 7fae35870700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fae1c03bfc0 0x7fae1c03e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.340+0000 7fae35870700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fae1c03bfc0 0x7fae1c03e3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fae2c003a10 tx=0x7fae2c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.341+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 <== mgr.14120 v2:172.21.15.12:6800/31089560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fae1c03b2d0 con 0x7fae1c03bfc0 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.348+0000 7fae382d5700 1 -- 172.21.15.12:0/3087101546 --> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fae300625d0 con 0x7fae1c03bfc0 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.348+0000 7fae237fe700 1 -- 172.21.15.12:0/3087101546 <== mgr.14120 v2:172.21.15.12:6800/31089560 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fae300625d0 con 0x7fae1c03bfc0 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 -- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fae1c03bfc0 msgr2=0x7fae1c03e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fae1c03bfc0 0x7fae1c03e3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fae2c003a10 tx=0x7fae2c009290 comp rx=0 tx=0).stop 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 -- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 msgr2=0x7fae3019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae3019a620 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fae24011710 tx=0x7fae24011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 -- 172.21.15.12:0/3087101546 shutdown_connections 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fae1c03bfc0 0x7fae1c03e3b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae301056f0 0x7fae3019a620 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:33.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 --2- 172.21.15.12:0/3087101546 >> [v2:172.21.15.12:6800/4231005605,v1:172.21.15.12:6801/4231005605] conn(0x7fae1c038700 0x7fae1c03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:33.700 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 -- 172.21.15.12:0/3087101546 >> 172.21.15.12:0/3087101546 conn(0x7fae30100c90 msgr2=0x7fae30101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:33.700 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 -- 172.21.15.12:0/3087101546 shutdown_connections 2024-05-08T23:39:33.700 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:33.349+0000 7fae382d5700 1 -- 172.21.15.12:0/3087101546 wait complete. 2024-05-08T23:39:33.700 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 5 is available 2024-05-08T23:39:33.700 INFO:teuthology.orchestra.run.smithi012.stdout:Setting orchestrator backend to cephadm... 2024-05-08T23:39:34.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.065+0000 7fcd19559700 1 Processor -- start 2024-05-08T23:39:34.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.065+0000 7fcd19559700 1 -- start start 2024-05-08T23:39:34.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.066+0000 7fcd19559700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd14106c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:34.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.066+0000 7fcd19559700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd141071e0 con 0x7fcd141067f0 2024-05-08T23:39:34.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.066+0000 7fcd12ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd14106c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:34.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.066+0000 7fcd12ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd14106c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59602/0 (socket says 172.21.15.12:59602) 2024-05-08T23:39:34.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.066+0000 7fcd12ffd700 1 -- 172.21.15.12:0/101137161 learned_addr learned my addr 172.21.15.12:0/101137161 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:34.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.066+0000 7fcd12ffd700 1 -- 172.21.15.12:0/101137161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd141079a0 con 0x7fcd141067f0 2024-05-08T23:39:34.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.066+0000 7fcd12ffd700 1 --2- 172.21.15.12:0/101137161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd14106c10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0098a0 tx=0x7fccfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf7b4b87715573b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:34.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.066+0000 7fcd11ffb700 1 -- 172.21.15.12:0/101137161 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccfc004160 con 0x7fcd141067f0 2024-05-08T23:39:34.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.066+0000 7fcd11ffb700 1 -- 172.21.15.12:0/101137161 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fccfc0042c0 con 0x7fcd141067f0 2024-05-08T23:39:34.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.067+0000 7fcd11ffb700 1 -- 172.21.15.12:0/101137161 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccfc011a00 con 0x7fcd141067f0 2024-05-08T23:39:34.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.067+0000 7fcd19559700 1 -- 172.21.15.12:0/101137161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 msgr2=0x7fcd14106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:34.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.067+0000 7fcd19559700 1 --2- 172.21.15.12:0/101137161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd14106c10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0098a0 tx=0x7fccfc009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.067+0000 7fcd19559700 1 -- 172.21.15.12:0/101137161 shutdown_connections 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.067+0000 7fcd19559700 1 --2- 172.21.15.12:0/101137161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd14106c10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.067+0000 7fcd19559700 1 -- 172.21.15.12:0/101137161 >> 172.21.15.12:0/101137161 conn(0x7fcd14101d70 msgr2=0x7fcd141041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.067+0000 7fcd19559700 1 -- 172.21.15.12:0/101137161 shutdown_connections 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.067+0000 7fcd19559700 1 -- 172.21.15.12:0/101137161 wait complete. 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.068+0000 7fcd19559700 1 Processor -- start 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.068+0000 7fcd19559700 1 -- start start 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.068+0000 7fcd19559700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd1407bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.068+0000 7fcd19559700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1407c050 con 0x7fcd141067f0 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.068+0000 7fcd12ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd1407bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.068+0000 7fcd12ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd1407bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59616/0 (socket says 172.21.15.12:59616) 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.068+0000 7fcd12ffd700 1 -- 172.21.15.12:0/2840794563 learned_addr learned my addr 172.21.15.12:0/2840794563 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:34.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.069+0000 7fcd12ffd700 1 -- 172.21.15.12:0/2840794563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc009550 con 0x7fcd141067f0 2024-05-08T23:39:34.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.069+0000 7fcd12ffd700 1 --2- 172.21.15.12:0/2840794563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd1407bb10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fccfc011710 tx=0x7fccfc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:34.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.069+0000 7fcd0bfff700 1 -- 172.21.15.12:0/2840794563 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccfc003aa0 con 0x7fcd141067f0 2024-05-08T23:39:34.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.069+0000 7fcd0bfff700 1 -- 172.21.15.12:0/2840794563 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fccfc003c00 con 0x7fcd141067f0 2024-05-08T23:39:34.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.069+0000 7fcd19559700 1 -- 172.21.15.12:0/2840794563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1407a290 con 0x7fcd141067f0 2024-05-08T23:39:34.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.069+0000 7fcd0bfff700 1 -- 172.21.15.12:0/2840794563 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccfc024800 con 0x7fcd141067f0 2024-05-08T23:39:34.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.069+0000 7fcd19559700 1 -- 172.21.15.12:0/2840794563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1407a730 con 0x7fcd141067f0 2024-05-08T23:39:34.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.070+0000 7fcd0bfff700 1 -- 172.21.15.12:0/2840794563 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fccfc01a430 con 0x7fcd141067f0 2024-05-08T23:39:34.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.070+0000 7fcd19559700 1 -- 172.21.15.12:0/2840794563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd140625d0 con 0x7fcd141067f0 2024-05-08T23:39:34.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.070+0000 7fcd0bfff700 1 --2- 172.21.15.12:0/2840794563 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fcd00038620 0x7fcd0003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:34.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.070+0000 7fcd0bfff700 1 -- 172.21.15.12:0/2840794563 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fccfc04c460 con 0x7fcd141067f0 2024-05-08T23:39:34.658 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.070+0000 7fcd127fc700 1 --2- 172.21.15.12:0/2840794563 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fcd00038620 0x7fcd0003aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:34.658 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.071+0000 7fcd127fc700 1 --2- 172.21.15.12:0/2840794563 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fcd00038620 0x7fcd0003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd04006fd0 tx=0x7fcd04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:34.658 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.076+0000 7fcd0bfff700 1 -- 172.21.15.12:0/2840794563 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fccfc02cb40 con 0x7fcd141067f0 2024-05-08T23:39:34.658 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.304+0000 7fcd19559700 1 -- 172.21.15.12:0/2840794563 --> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd1407ad90 con 0x7fcd00038620 2024-05-08T23:39:34.658 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.313+0000 7fcd0bfff700 1 -- 172.21.15.12:0/2840794563 <== mgr.14120 v2:172.21.15.12:6800/31089560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcd1407ad90 con 0x7fcd00038620 2024-05-08T23:39:34.658 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.315+0000 7fcd19559700 1 -- 172.21.15.12:0/2840794563 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fcd00038620 msgr2=0x7fcd0003aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:34.658 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.315+0000 7fcd19559700 1 --2- 172.21.15.12:0/2840794563 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fcd00038620 0x7fcd0003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd04006fd0 tx=0x7fcd04006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:34.658 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.315+0000 7fcd19559700 1 -- 172.21.15.12:0/2840794563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 msgr2=0x7fcd1407bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:34.658 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.315+0000 7fcd19559700 1 --2- 172.21.15.12:0/2840794563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd1407bb10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fccfc011710 tx=0x7fccfc011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:34.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.315+0000 7fcd19559700 1 -- 172.21.15.12:0/2840794563 shutdown_connections 2024-05-08T23:39:34.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.316+0000 7fcd19559700 1 --2- 172.21.15.12:0/2840794563 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fcd00038620 0x7fcd0003aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:34.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.316+0000 7fcd19559700 1 --2- 172.21.15.12:0/2840794563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd141067f0 0x7fcd1407bb10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:34.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.316+0000 7fcd19559700 1 -- 172.21.15.12:0/2840794563 >> 172.21.15.12:0/2840794563 conn(0x7fcd14101d70 msgr2=0x7fcd141029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:34.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.316+0000 7fcd19559700 1 -- 172.21.15.12:0/2840794563 shutdown_connections 2024-05-08T23:39:34.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:34.316+0000 7fcd19559700 1 -- 172.21.15.12:0/2840794563 wait complete. 2024-05-08T23:39:34.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:34 smithi012 ceph-mon[42503]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T23:39:34.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:34 smithi012 ceph-mon[42503]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T23:39:34.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:34 smithi012 ceph-mon[42503]: [08/May/2024:23:39:33] ENGINE Bus STARTING 2024-05-08T23:39:34.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:34 smithi012 ceph-mon[42503]: [08/May/2024:23:39:33] ENGINE Serving on http://172.21.15.12:8765 2024-05-08T23:39:34.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:34 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:39:34.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:34 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:34.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:34 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:39:35.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:35 smithi012 ceph-mon[42503]: [08/May/2024:23:39:33] ENGINE Serving on https://172.21.15.12:7150 2024-05-08T23:39:35.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:35 smithi012 ceph-mon[42503]: [08/May/2024:23:39:33] ENGINE Bus STARTED 2024-05-08T23:39:35.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:35 smithi012 ceph-mon[42503]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:39:35.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:35 smithi012 ceph-mon[42503]: mgrmap e8: x(active, since 2s) 2024-05-08T23:39:35.636 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.045+0000 7f9443553700 1 Processor -- start 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.045+0000 7f9443553700 1 -- start start 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.045+0000 7f9443553700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.045+0000 7f9443553700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c07a8c0 con 0x7f943c07be90 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.046+0000 7f94412ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.046+0000 7f94412ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59618/0 (socket says 172.21.15.12:59618) 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.046+0000 7f94412ef700 1 -- 172.21.15.12:0/3173788235 learned_addr learned my addr 172.21.15.12:0/3173788235 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.046+0000 7f94412ef700 1 -- 172.21.15.12:0/3173788235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c07aa00 con 0x7f943c07be90 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.046+0000 7f94412ef700 1 --2- 172.21.15.12:0/3173788235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c07a2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f942c0098a0 tx=0x7f942c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b92522b3859158b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.046+0000 7f9433fff700 1 -- 172.21.15.12:0/3173788235 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f942c004160 con 0x7f943c07be90 2024-05-08T23:39:35.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.046+0000 7f9433fff700 1 -- 172.21.15.12:0/3173788235 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f942c0042c0 con 0x7f943c07be90 2024-05-08T23:39:35.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.046+0000 7f9433fff700 1 -- 172.21.15.12:0/3173788235 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f942c011a00 con 0x7f943c07be90 2024-05-08T23:39:35.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.047+0000 7f9443553700 1 -- 172.21.15.12:0/3173788235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 msgr2=0x7f943c07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:35.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.047+0000 7f9443553700 1 --2- 172.21.15.12:0/3173788235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c07a2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f942c0098a0 tx=0x7f942c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:35.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.047+0000 7f9443553700 1 -- 172.21.15.12:0/3173788235 shutdown_connections 2024-05-08T23:39:35.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.047+0000 7f9443553700 1 --2- 172.21.15.12:0/3173788235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c07a2f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:35.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.047+0000 7f9443553700 1 -- 172.21.15.12:0/3173788235 >> 172.21.15.12:0/3173788235 conn(0x7f943c101d00 msgr2=0x7f943c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:35.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.047+0000 7f9443553700 1 -- 172.21.15.12:0/3173788235 shutdown_connections 2024-05-08T23:39:35.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.047+0000 7f9443553700 1 -- 172.21.15.12:0/3173788235 wait complete. 2024-05-08T23:39:35.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.048+0000 7f9443553700 1 Processor -- start 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.048+0000 7f9443553700 1 -- start start 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.048+0000 7f9443553700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.048+0000 7f9443553700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c19cc60 con 0x7f943c07be90 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.048+0000 7f94412ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.048+0000 7f94412ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59622/0 (socket says 172.21.15.12:59622) 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.048+0000 7f94412ef700 1 -- 172.21.15.12:0/1501064599 learned_addr learned my addr 172.21.15.12:0/1501064599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.048+0000 7f94412ef700 1 -- 172.21.15.12:0/1501064599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c009550 con 0x7f943c07be90 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.049+0000 7f94412ef700 1 --2- 172.21.15.12:0/1501064599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c19c720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f942c009fb0 tx=0x7f942c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.049+0000 7f94327fc700 1 -- 172.21.15.12:0/1501064599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f942c003a40 con 0x7f943c07be90 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.049+0000 7f94327fc700 1 -- 172.21.15.12:0/1501064599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f942c003ba0 con 0x7f943c07be90 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.049+0000 7f94327fc700 1 -- 172.21.15.12:0/1501064599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f942c01a750 con 0x7f943c07be90 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.049+0000 7f9443553700 1 -- 172.21.15.12:0/1501064599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c19ce60 con 0x7f943c07be90 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.049+0000 7f9443553700 1 -- 172.21.15.12:0/1501064599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c19d300 con 0x7f943c07be90 2024-05-08T23:39:35.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.050+0000 7f94327fc700 1 -- 172.21.15.12:0/1501064599 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f942c024430 con 0x7f943c07be90 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.050+0000 7f94327fc700 1 --2- 172.21.15.12:0/1501064599 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f94280385e0 0x7f942803aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.050+0000 7f94327fc700 1 -- 172.21.15.12:0/1501064599 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f942c04c4e0 con 0x7f943c07be90 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.050+0000 7f9440aee700 1 --2- 172.21.15.12:0/1501064599 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f94280385e0 0x7f942803aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.050+0000 7f9443553700 1 -- 172.21.15.12:0/1501064599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c10cb50 con 0x7f943c07be90 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.051+0000 7f9440aee700 1 --2- 172.21.15.12:0/1501064599 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f94280385e0 0x7f942803aaa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9438006fd0 tx=0x7f9438006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.056+0000 7f94327fc700 1 -- 172.21.15.12:0/1501064599 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f942c0246e0 con 0x7f943c07be90 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.271+0000 7f9443553700 1 -- 172.21.15.12:0/1501064599 --> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f943c103050 con 0x7f94280385e0 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.272+0000 7f94327fc700 1 -- 172.21.15.12:0/1501064599 <== mgr.14120 v2:172.21.15.12:6800/31089560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f943c103050 con 0x7f94280385e0 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.274+0000 7f9443553700 1 -- 172.21.15.12:0/1501064599 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f94280385e0 msgr2=0x7f942803aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.274+0000 7f9443553700 1 --2- 172.21.15.12:0/1501064599 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f94280385e0 0x7f942803aaa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9438006fd0 tx=0x7f9438006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.274+0000 7f9443553700 1 -- 172.21.15.12:0/1501064599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 msgr2=0x7f943c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:35.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.274+0000 7f9443553700 1 --2- 172.21.15.12:0/1501064599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c19c720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f942c009fb0 tx=0x7f942c011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:35.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.274+0000 7f9443553700 1 -- 172.21.15.12:0/1501064599 shutdown_connections 2024-05-08T23:39:35.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.274+0000 7f9443553700 1 --2- 172.21.15.12:0/1501064599 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f94280385e0 0x7f942803aaa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:35.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.274+0000 7f9443553700 1 --2- 172.21.15.12:0/1501064599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943c07be90 0x7f943c19c720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:35.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.274+0000 7f9443553700 1 -- 172.21.15.12:0/1501064599 >> 172.21.15.12:0/1501064599 conn(0x7f943c101d00 msgr2=0x7f943c102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:35.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.274+0000 7f9443553700 1 -- 172.21.15.12:0/1501064599 shutdown_connections 2024-05-08T23:39:35.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:35.274+0000 7f9443553700 1 -- 172.21.15.12:0/1501064599 wait complete. 2024-05-08T23:39:35.641 INFO:teuthology.orchestra.run.smithi012.stdout:Generating ssh key... 2024-05-08T23:39:36.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:36 smithi012 ceph-mon[42503]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:39:37.016 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: Generating public/private rsa key pair. 2024-05-08T23:39:37.016 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: Your identification has been saved in /tmp/tmp7jmvl3mc/key. 2024-05-08T23:39:37.016 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: Your public key has been saved in /tmp/tmp7jmvl3mc/key.pub. 2024-05-08T23:39:37.016 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: The key fingerprint is: 2024-05-08T23:39:37.016 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: SHA256:lXpANzFVrEm/Y5E+BVUyb8x+ZZorkd+cO8F5MBeS8ao ceph-00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:39:37.017 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: The key's randomart image is: 2024-05-08T23:39:37.017 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: +---[RSA 3072]----+ 2024-05-08T23:39:37.017 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: | . =o.+*.+| 2024-05-08T23:39:37.017 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: | . . +.o+B | 2024-05-08T23:39:37.017 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: | . o. +.+O| 2024-05-08T23:39:37.017 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: | + o.B*+| 2024-05-08T23:39:37.017 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: | S . oo+B+| 2024-05-08T23:39:37.018 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: | . .o*B+| 2024-05-08T23:39:37.018 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: | E..o+=| 2024-05-08T23:39:37.018 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: | . ..| 2024-05-08T23:39:37.018 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: | ..| 2024-05-08T23:39:37.018 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: +----[SHA256]-----+ 2024-05-08T23:39:37.094 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.010+0000 7f0d9111f700 1 Processor -- start 2024-05-08T23:39:37.094 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.010+0000 7f0d9111f700 1 -- start start 2024-05-08T23:39:37.094 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.010+0000 7f0d9111f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.094 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.010+0000 7f0d9111f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c107170 con 0x7f0d8c106780 2024-05-08T23:39:37.094 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.011+0000 7f0d8ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:37.094 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.011+0000 7f0d8ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59628/0 (socket says 172.21.15.12:59628) 2024-05-08T23:39:37.094 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.011+0000 7f0d8ad9d700 1 -- 172.21.15.12:0/4134246365 learned_addr learned my addr 172.21.15.12:0/4134246365 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:37.094 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.011+0000 7f0d8ad9d700 1 -- 172.21.15.12:0/4134246365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c107980 con 0x7f0d8c106780 2024-05-08T23:39:37.094 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.011+0000 7f0d8ad9d700 1 --2- 172.21.15.12:0/4134246365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c106ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0d800098a0 tx=0x7f0d80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6b65d9cc2b74d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.011+0000 7f0d89d9b700 1 -- 172.21.15.12:0/4134246365 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d80004160 con 0x7f0d8c106780 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.011+0000 7f0d89d9b700 1 -- 172.21.15.12:0/4134246365 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0d800042c0 con 0x7f0d8c106780 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.011+0000 7f0d89d9b700 1 -- 172.21.15.12:0/4134246365 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d80011a00 con 0x7f0d8c106780 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.012+0000 7f0d9111f700 1 -- 172.21.15.12:0/4134246365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 msgr2=0x7f0d8c106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.012+0000 7f0d9111f700 1 --2- 172.21.15.12:0/4134246365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c106ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0d800098a0 tx=0x7f0d80009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.012+0000 7f0d9111f700 1 -- 172.21.15.12:0/4134246365 shutdown_connections 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.012+0000 7f0d9111f700 1 --2- 172.21.15.12:0/4134246365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c106ba0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.012+0000 7f0d9111f700 1 -- 172.21.15.12:0/4134246365 >> 172.21.15.12:0/4134246365 conn(0x7f0d8c101d00 msgr2=0x7f0d8c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.012+0000 7f0d9111f700 1 -- 172.21.15.12:0/4134246365 shutdown_connections 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.012+0000 7f0d9111f700 1 -- 172.21.15.12:0/4134246365 wait complete. 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.013+0000 7f0d9111f700 1 Processor -- start 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.013+0000 7f0d9111f700 1 -- start start 2024-05-08T23:39:37.095 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.013+0000 7f0d9111f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.013+0000 7f0d9111f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c19aa60 con 0x7f0d8c106780 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.013+0000 7f0d8ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.013+0000 7f0d8ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59642/0 (socket says 172.21.15.12:59642) 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.013+0000 7f0d8ad9d700 1 -- 172.21.15.12:0/843217990 learned_addr learned my addr 172.21.15.12:0/843217990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.014+0000 7f0d8ad9d700 1 -- 172.21.15.12:0/843217990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80009550 con 0x7f0d8c106780 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.014+0000 7f0d8ad9d700 1 --2- 172.21.15.12:0/843217990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c19a520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009130 tx=0x7f0d80011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.014+0000 7f0d73fff700 1 -- 172.21.15.12:0/843217990 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d80003ab0 con 0x7f0d8c106780 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.014+0000 7f0d73fff700 1 -- 172.21.15.12:0/843217990 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0d80003c10 con 0x7f0d8c106780 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.014+0000 7f0d9111f700 1 -- 172.21.15.12:0/843217990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8c19ac60 con 0x7f0d8c106780 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.014+0000 7f0d73fff700 1 -- 172.21.15.12:0/843217990 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d800247d0 con 0x7f0d8c106780 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.014+0000 7f0d9111f700 1 -- 172.21.15.12:0/843217990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8c19b100 con 0x7f0d8c106780 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.015+0000 7f0d73fff700 1 -- 172.21.15.12:0/843217990 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f0d8002d3f0 con 0x7f0d8c106780 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.015+0000 7f0d73fff700 1 --2- 172.21.15.12:0/843217990 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0d74038700 0x7f0d7403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.015+0000 7f0d9111f700 1 -- 172.21.15.12:0/843217990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8c10abb0 con 0x7f0d8c106780 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.015+0000 7f0d73fff700 1 -- 172.21.15.12:0/843217990 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0d8004c8f0 con 0x7f0d8c106780 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.015+0000 7f0d8a59c700 1 --2- 172.21.15.12:0/843217990 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0d74038700 0x7f0d7403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.016+0000 7f0d8a59c700 1 --2- 172.21.15.12:0/843217990 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0d74038700 0x7f0d7403abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c006fd0 tx=0x7f0d7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.021+0000 7f0d73fff700 1 -- 172.21.15.12:0/843217990 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0d80010740 con 0x7f0d8c106780 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.233+0000 7f0d9111f700 1 -- 172.21.15.12:0/843217990 --> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d8c103060 con 0x7f0d74038700 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.740+0000 7f0d73fff700 1 -- 172.21.15.12:0/843217990 <== mgr.14120 v2:172.21.15.12:6800/31089560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0d8c103060 con 0x7f0d74038700 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.743+0000 7f0d9111f700 1 -- 172.21.15.12:0/843217990 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0d74038700 msgr2=0x7f0d7403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.743+0000 7f0d9111f700 1 --2- 172.21.15.12:0/843217990 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0d74038700 0x7f0d7403abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c006fd0 tx=0x7f0d7c006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.743+0000 7f0d9111f700 1 -- 172.21.15.12:0/843217990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 msgr2=0x7f0d8c19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.743+0000 7f0d9111f700 1 --2- 172.21.15.12:0/843217990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c19a520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009130 tx=0x7f0d80011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.743+0000 7f0d9111f700 1 -- 172.21.15.12:0/843217990 shutdown_connections 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.743+0000 7f0d9111f700 1 --2- 172.21.15.12:0/843217990 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0d74038700 0x7f0d7403abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.743+0000 7f0d9111f700 1 --2- 172.21.15.12:0/843217990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8c106780 0x7f0d8c19a520 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.097 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.743+0000 7f0d9111f700 1 -- 172.21.15.12:0/843217990 >> 172.21.15.12:0/843217990 conn(0x7f0d8c101d00 msgr2=0x7f0d8c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:37.098 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.743+0000 7f0d9111f700 1 -- 172.21.15.12:0/843217990 shutdown_connections 2024-05-08T23:39:37.098 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:36.743+0000 7f0d9111f700 1 -- 172.21.15.12:0/843217990 wait complete. 2024-05-08T23:39:37.990 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:37 smithi012 ceph-mon[42503]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:39:37.990 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:37 smithi012 ceph-mon[42503]: Generating ssh key... 2024-05-08T23:39:37.990 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:37 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:37.990 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:37 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:38.067 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZFLLVFDooOlZg9S73lL6B0lKuCfF7O7KoAhZsFHPAXiFlBpuoYBdKqz9w/Rdc4/4kq8XYN1PH61pVxh32sVRr+YduHGmTTZ7D8YWBlWGrWpMf4tGrudx+Fyrgc9ni4yyCyCK+CUBjV0mx8GyMky7muDKTQveuCXyzwFw+HMgxkvo4xfUzxXllz2nuO6fWXGh7s6m7Q9q0T6/he1huVkkcpCT/B30hTLVYsI1M7C29G0wLdbp7uxTOd+cirBNAoMfZ6HweZyKXxQfBOxonfna9LalSjsWPtfTA5SNeLwB9Uu9Wn48mgacbvoSCg0WhINT4/FMx92OuIpglisNnUeiqSCG9c4xoqTVFAF8UMEdfeKNcY7i1eg2YCR1v8Gd47xurTf7FpBf0BTS5Azc37gFKhPmKdZH75e1sGw1bKHL8jJChgdreqk1kzgbldbIc2bmARYzeSkKBnct3lBPBAcoHmQmGC4ZBLlO7prtWWGuaNIuKQv30qKFcWd0mQBb+Z9E= ceph-00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.457+0000 7fdf78f09700 1 Processor -- start 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.457+0000 7fdf78f09700 1 -- start start 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.457+0000 7fdf78f09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf74105b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.457+0000 7fdf78f09700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741060d0 con 0x7fdf741056e0 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.457+0000 7fdf7259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf74105b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.457+0000 7fdf7259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf74105b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59654/0 (socket says 172.21.15.12:59654) 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.457+0000 7fdf7259c700 1 -- 172.21.15.12:0/3515491203 learned_addr learned my addr 172.21.15.12:0/3515491203 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.458+0000 7fdf7259c700 1 -- 172.21.15.12:0/3515491203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf741068e0 con 0x7fdf741056e0 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.458+0000 7fdf7259c700 1 --2- 172.21.15.12:0/3515491203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf74105b00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf640098a0 tx=0x7fdf64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2a631b48f90f3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.458+0000 7fdf7159a700 1 -- 172.21.15.12:0/3515491203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf64004160 con 0x7fdf741056e0 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.458+0000 7fdf7159a700 1 -- 172.21.15.12:0/3515491203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf640042c0 con 0x7fdf741056e0 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.458+0000 7fdf7159a700 1 -- 172.21.15.12:0/3515491203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf64011a00 con 0x7fdf741056e0 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.459+0000 7fdf78f09700 1 -- 172.21.15.12:0/3515491203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 msgr2=0x7fdf74105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.459+0000 7fdf78f09700 1 --2- 172.21.15.12:0/3515491203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf74105b00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf640098a0 tx=0x7fdf64009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.459+0000 7fdf78f09700 1 -- 172.21.15.12:0/3515491203 shutdown_connections 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.459+0000 7fdf78f09700 1 --2- 172.21.15.12:0/3515491203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf74105b00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.459+0000 7fdf78f09700 1 -- 172.21.15.12:0/3515491203 >> 172.21.15.12:0/3515491203 conn(0x7fdf74100c60 msgr2=0x7fdf741030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.459+0000 7fdf78f09700 1 -- 172.21.15.12:0/3515491203 shutdown_connections 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.459+0000 7fdf78f09700 1 -- 172.21.15.12:0/3515491203 wait complete. 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.460+0000 7fdf78f09700 1 Processor -- start 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.460+0000 7fdf78f09700 1 -- start start 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.460+0000 7fdf78f09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf741906f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.460+0000 7fdf78f09700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74190c30 con 0x7fdf741056e0 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.460+0000 7fdf7259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf741906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.460+0000 7fdf7259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf741906f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59668/0 (socket says 172.21.15.12:59668) 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.460+0000 7fdf7259c700 1 -- 172.21.15.12:0/3972423831 learned_addr learned my addr 172.21.15.12:0/3972423831 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:38.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.460+0000 7fdf7259c700 1 -- 172.21.15.12:0/3972423831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64009550 con 0x7fdf741056e0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.460+0000 7fdf7259c700 1 --2- 172.21.15.12:0/3972423831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf741906f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009130 tx=0x7fdf64011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.461+0000 7fdf637fe700 1 -- 172.21.15.12:0/3972423831 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf640038b0 con 0x7fdf741056e0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.461+0000 7fdf637fe700 1 -- 172.21.15.12:0/3972423831 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf64003a10 con 0x7fdf741056e0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.461+0000 7fdf637fe700 1 -- 172.21.15.12:0/3972423831 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf64019550 con 0x7fdf741056e0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.461+0000 7fdf78f09700 1 -- 172.21.15.12:0/3972423831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7418eec0 con 0x7fdf741056e0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.461+0000 7fdf78f09700 1 -- 172.21.15.12:0/3972423831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7418f360 con 0x7fdf741056e0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.462+0000 7fdf637fe700 1 -- 172.21.15.12:0/3972423831 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fdf6401a430 con 0x7fdf741056e0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.462+0000 7fdf637fe700 1 --2- 172.21.15.12:0/3972423831 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdf5c038690 0x7fdf5c03ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.462+0000 7fdf78f09700 1 -- 172.21.15.12:0/3972423831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7418fd70 con 0x7fdf741056e0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.462+0000 7fdf637fe700 1 -- 172.21.15.12:0/3972423831 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdf6404c7a0 con 0x7fdf741056e0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.462+0000 7fdf71d9b700 1 --2- 172.21.15.12:0/3972423831 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdf5c038690 0x7fdf5c03ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.462+0000 7fdf71d9b700 1 --2- 172.21.15.12:0/3972423831 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdf5c038690 0x7fdf5c03ab50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf68006fd0 tx=0x7fdf68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:38.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.467+0000 7fdf637fe700 1 -- 172.21.15.12:0/3972423831 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdf640239e0 con 0x7fdf741056e0 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.679+0000 7fdf78f09700 1 -- 172.21.15.12:0/3972423831 --> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf74194250 con 0x7fdf5c038690 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.680+0000 7fdf637fe700 1 -- 172.21.15.12:0/3972423831 <== mgr.14120 v2:172.21.15.12:6800/31089560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fdf74194250 con 0x7fdf5c038690 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.682+0000 7fdf78f09700 1 -- 172.21.15.12:0/3972423831 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdf5c038690 msgr2=0x7fdf5c03ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.682+0000 7fdf78f09700 1 --2- 172.21.15.12:0/3972423831 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdf5c038690 0x7fdf5c03ab50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf68006fd0 tx=0x7fdf68006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.682+0000 7fdf78f09700 1 -- 172.21.15.12:0/3972423831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 msgr2=0x7fdf741906f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.682+0000 7fdf78f09700 1 --2- 172.21.15.12:0/3972423831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf741906f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009130 tx=0x7fdf64011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.682+0000 7fdf78f09700 1 -- 172.21.15.12:0/3972423831 shutdown_connections 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.682+0000 7fdf78f09700 1 --2- 172.21.15.12:0/3972423831 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdf5c038690 0x7fdf5c03ab50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.682+0000 7fdf78f09700 1 --2- 172.21.15.12:0/3972423831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf741056e0 0x7fdf741906f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.682+0000 7fdf78f09700 1 -- 172.21.15.12:0/3972423831 >> 172.21.15.12:0/3972423831 conn(0x7fdf74100c60 msgr2=0x7fdf74101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.682+0000 7fdf78f09700 1 -- 172.21.15.12:0/3972423831 shutdown_connections 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:37.682+0000 7fdf78f09700 1 -- 172.21.15.12:0/3972423831 wait complete. 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T23:39:38.071 INFO:teuthology.orchestra.run.smithi012.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T23:39:38.072 INFO:teuthology.orchestra.run.smithi012.stdout:Adding host smithi012... 2024-05-08T23:39:38.945 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:38 smithi012 ceph-mon[42503]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:39:39.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:39 smithi012 ceph-mon[42503]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:39:40.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:40 smithi012 ceph-mon[42503]: Deploying cephadm binary to smithi012 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Added host 'smithi012' with addr '172.21.15.12' 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.427+0000 7effe2f54700 1 Processor -- start 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.427+0000 7effe2f54700 1 -- start start 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.427+0000 7effe2f54700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc105aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.427+0000 7effe2f54700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc106070 con 0x7effdc105680 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.427+0000 7effe0cf0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc105aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.427+0000 7effe0cf0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc105aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33666/0 (socket says 172.21.15.12:33666) 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.427+0000 7effe0cf0700 1 -- 172.21.15.12:0/2875536168 learned_addr learned my addr 172.21.15.12:0/2875536168 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.428+0000 7effe0cf0700 1 -- 172.21.15.12:0/2875536168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc106880 con 0x7effdc105680 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.428+0000 7effe0cf0700 1 --2- 172.21.15.12:0/2875536168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc105aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7effd80098a0 tx=0x7effd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ab74f3c86c2ab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.428+0000 7effd37fe700 1 -- 172.21.15.12:0/2875536168 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effd8004160 con 0x7effdc105680 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.428+0000 7effd37fe700 1 -- 172.21.15.12:0/2875536168 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7effd80042c0 con 0x7effdc105680 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.428+0000 7effd37fe700 1 -- 172.21.15.12:0/2875536168 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effd8011a00 con 0x7effdc105680 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.429+0000 7effe2f54700 1 -- 172.21.15.12:0/2875536168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 msgr2=0x7effdc105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.429+0000 7effe2f54700 1 --2- 172.21.15.12:0/2875536168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc105aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7effd80098a0 tx=0x7effd8009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.429+0000 7effe2f54700 1 -- 172.21.15.12:0/2875536168 shutdown_connections 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.429+0000 7effe2f54700 1 --2- 172.21.15.12:0/2875536168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc105aa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.429+0000 7effe2f54700 1 -- 172.21.15.12:0/2875536168 >> 172.21.15.12:0/2875536168 conn(0x7effdc100cc0 msgr2=0x7effdc1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.429+0000 7effe2f54700 1 -- 172.21.15.12:0/2875536168 shutdown_connections 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.429+0000 7effe2f54700 1 -- 172.21.15.12:0/2875536168 wait complete. 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.429+0000 7effe2f54700 1 Processor -- start 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.429+0000 7effe2f54700 1 -- start start 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effe2f54700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc1906e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effe2f54700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc190c20 con 0x7effdc105680 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effe0cf0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc1906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effe0cf0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc1906e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33674/0 (socket says 172.21.15.12:33674) 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effe0cf0700 1 -- 172.21.15.12:0/844933134 learned_addr learned my addr 172.21.15.12:0/844933134 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effe0cf0700 1 -- 172.21.15.12:0/844933134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8009550 con 0x7effdc105680 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effe0cf0700 1 --2- 172.21.15.12:0/844933134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc1906e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7effd8009130 tx=0x7effd8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effd1ffb700 1 -- 172.21.15.12:0/844933134 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effd8003ab0 con 0x7effdc105680 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effd1ffb700 1 -- 172.21.15.12:0/844933134 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7effd8003c10 con 0x7effdc105680 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effe2f54700 1 -- 172.21.15.12:0/844933134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effdc18ee60 con 0x7effdc105680 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.430+0000 7effd1ffb700 1 -- 172.21.15.12:0/844933134 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effd80247d0 con 0x7effdc105680 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.431+0000 7effe2f54700 1 -- 172.21.15.12:0/844933134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc18f300 con 0x7effdc105680 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.431+0000 7effd1ffb700 1 -- 172.21.15.12:0/844933134 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7effd802d3f0 con 0x7effdc105680 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.432+0000 7effd1ffb700 1 --2- 172.21.15.12:0/844933134 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7effc4038750 0x7effc403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.432+0000 7effd1ffb700 1 -- 172.21.15.12:0/844933134 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7effd804c7d0 con 0x7effdc105680 2024-05-08T23:39:41.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.432+0000 7effd3fff700 1 --2- 172.21.15.12:0/844933134 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7effc4038750 0x7effc403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.432+0000 7effe2f54700 1 -- 172.21.15.12:0/844933134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effdc18fd10 con 0x7effdc105680 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.432+0000 7effd3fff700 1 --2- 172.21.15.12:0/844933134 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7effc4038750 0x7effc403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effcc006fd0 tx=0x7effcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.437+0000 7effd1ffb700 1 -- 172.21.15.12:0/844933134 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7effd802d6f0 con 0x7effdc105680 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:38.653+0000 7effe2f54700 1 -- 172.21.15.12:0/844933134 --> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}) v1 -- 0x7effdc1941b0 con 0x7effc4038750 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.012+0000 7effd1ffb700 1 -- 172.21.15.12:0/844933134 <== mgr.14120 v2:172.21.15.12:6800/31089560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7effdc1941b0 con 0x7effc4038750 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.015+0000 7effe2f54700 1 -- 172.21.15.12:0/844933134 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7effc4038750 msgr2=0x7effc403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.015+0000 7effe2f54700 1 --2- 172.21.15.12:0/844933134 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7effc4038750 0x7effc403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effcc006fd0 tx=0x7effcc006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.015+0000 7effe2f54700 1 -- 172.21.15.12:0/844933134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 msgr2=0x7effdc1906e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.015+0000 7effe2f54700 1 --2- 172.21.15.12:0/844933134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc1906e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7effd8009130 tx=0x7effd8011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.015+0000 7effe2f54700 1 -- 172.21.15.12:0/844933134 shutdown_connections 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.015+0000 7effe2f54700 1 --2- 172.21.15.12:0/844933134 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7effc4038750 0x7effc403ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.015+0000 7effe2f54700 1 --2- 172.21.15.12:0/844933134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effdc105680 0x7effdc1906e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.015+0000 7effe2f54700 1 -- 172.21.15.12:0/844933134 >> 172.21.15.12:0/844933134 conn(0x7effdc100cc0 msgr2=0x7effdc101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:41.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.015+0000 7effe2f54700 1 -- 172.21.15.12:0/844933134 shutdown_connections 2024-05-08T23:39:41.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.015+0000 7effe2f54700 1 -- 172.21.15.12:0/844933134 wait complete. 2024-05-08T23:39:41.321 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mon service... 2024-05-08T23:39:42.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:42 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:42.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:42 smithi012 ceph-mon[42503]: Added host smithi012 2024-05-08T23:39:42.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:42 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:39:42.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:42 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:42.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T23:39:42.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.681+0000 7fdd61ca7700 1 Processor -- start 2024-05-08T23:39:42.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.681+0000 7fdd61ca7700 1 -- start start 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.681+0000 7fdd61ca7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.681+0000 7fdd61ca7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c108840 con 0x7fdd5c107e50 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.682+0000 7fdd5b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.682+0000 7fdd5b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33688/0 (socket says 172.21.15.12:33688) 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.682+0000 7fdd5b7fe700 1 -- 172.21.15.12:0/1592034597 learned_addr learned my addr 172.21.15.12:0/1592034597 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.682+0000 7fdd5b7fe700 1 -- 172.21.15.12:0/1592034597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c1063e0 con 0x7fdd5c107e50 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.682+0000 7fdd5b7fe700 1 --2- 172.21.15.12:0/1592034597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c108270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0098a0 tx=0x7fdd4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9729284bc186f091 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.682+0000 7fdd5a7fc700 1 -- 172.21.15.12:0/1592034597 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd4c004160 con 0x7fdd5c107e50 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.683+0000 7fdd5a7fc700 1 -- 172.21.15.12:0/1592034597 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd4c0042c0 con 0x7fdd5c107e50 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.683+0000 7fdd5a7fc700 1 -- 172.21.15.12:0/1592034597 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd4c011a00 con 0x7fdd5c107e50 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.683+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1592034597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 msgr2=0x7fdd5c108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:42.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.683+0000 7fdd61ca7700 1 --2- 172.21.15.12:0/1592034597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c108270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0098a0 tx=0x7fdd4c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.683+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1592034597 shutdown_connections 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.683+0000 7fdd61ca7700 1 --2- 172.21.15.12:0/1592034597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c108270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.683+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1592034597 >> 172.21.15.12:0/1592034597 conn(0x7fdd5c101d40 msgr2=0x7fdd5c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.683+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1592034597 shutdown_connections 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.683+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1592034597 wait complete. 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.684+0000 7fdd61ca7700 1 Processor -- start 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.684+0000 7fdd61ca7700 1 -- start start 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.685+0000 7fdd61ca7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.685+0000 7fdd61ca7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c19ab50 con 0x7fdd5c107e50 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.685+0000 7fdd5b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.685+0000 7fdd5b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33702/0 (socket says 172.21.15.12:33702) 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.685+0000 7fdd5b7fe700 1 -- 172.21.15.12:0/1957823750 learned_addr learned my addr 172.21.15.12:0/1957823750 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:42.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.685+0000 7fdd5b7fe700 1 -- 172.21.15.12:0/1957823750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c009550 con 0x7fdd5c107e50 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.685+0000 7fdd5b7fe700 1 --2- 172.21.15.12:0/1957823750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c19a610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009fb0 tx=0x7fdd4c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.686+0000 7fdd58ff9700 1 -- 172.21.15.12:0/1957823750 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd4c003a40 con 0x7fdd5c107e50 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.686+0000 7fdd58ff9700 1 -- 172.21.15.12:0/1957823750 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd4c003ba0 con 0x7fdd5c107e50 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.686+0000 7fdd58ff9700 1 -- 172.21.15.12:0/1957823750 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd4c01a750 con 0x7fdd5c107e50 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.686+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1957823750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c19ad50 con 0x7fdd5c107e50 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.686+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1957823750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c19b1f0 con 0x7fdd5c107e50 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.688+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1957823750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c0625d0 con 0x7fdd5c107e50 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.688+0000 7fdd58ff9700 1 -- 172.21.15.12:0/1957823750 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fdd4c024b60 con 0x7fdd5c107e50 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.689+0000 7fdd58ff9700 1 --2- 172.21.15.12:0/1957823750 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdd48038700 0x7fdd4803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.689+0000 7fdd58ff9700 1 -- 172.21.15.12:0/1957823750 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdd4c04cb70 con 0x7fdd5c107e50 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.689+0000 7fdd5affd700 1 --2- 172.21.15.12:0/1957823750 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdd48038700 0x7fdd4803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.689+0000 7fdd5affd700 1 --2- 172.21.15.12:0/1957823750 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdd48038700 0x7fdd4803abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdd44006fd0 tx=0x7fdd44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:42.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.694+0000 7fdd58ff9700 1 -- 172.21.15.12:0/1957823750 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdd4c02c7a0 con 0x7fdd5c107e50 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.925+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1957823750 --> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdd5c10ab70 con 0x7fdd48038700 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.930+0000 7fdd58ff9700 1 -- 172.21.15.12:0/1957823750 <== mgr.14120 v2:172.21.15.12:6800/31089560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdd5c10ab70 con 0x7fdd48038700 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.932+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1957823750 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdd48038700 msgr2=0x7fdd4803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.932+0000 7fdd61ca7700 1 --2- 172.21.15.12:0/1957823750 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdd48038700 0x7fdd4803abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdd44006fd0 tx=0x7fdd44006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.932+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1957823750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 msgr2=0x7fdd5c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.932+0000 7fdd61ca7700 1 --2- 172.21.15.12:0/1957823750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c19a610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009fb0 tx=0x7fdd4c011ad0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.932+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1957823750 shutdown_connections 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.932+0000 7fdd61ca7700 1 --2- 172.21.15.12:0/1957823750 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fdd48038700 0x7fdd4803abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.932+0000 7fdd61ca7700 1 --2- 172.21.15.12:0/1957823750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5c107e50 0x7fdd5c19a610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.932+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1957823750 >> 172.21.15.12:0/1957823750 conn(0x7fdd5c101d40 msgr2=0x7fdd5c102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.933+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1957823750 shutdown_connections 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:41.933+0000 7fdd61ca7700 1 -- 172.21.15.12:0/1957823750 wait complete. 2024-05-08T23:39:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mgr service... 2024-05-08T23:39:43.381 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:43 smithi012 ceph-mon[42503]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:39:43.381 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:43 smithi012 ceph-mon[42503]: Saving service mon spec with placement count:5 2024-05-08T23:39:43.381 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:43 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:43.381 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:43 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:43.381 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:43 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:43.684 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T23:39:43.685 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.828+0000 7fb6b855f700 1 Processor -- start 2024-05-08T23:39:43.685 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.828+0000 7fb6b855f700 1 -- start start 2024-05-08T23:39:43.685 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.829+0000 7fb6b855f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b0105b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:43.685 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.829+0000 7fb6b855f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b01060e0 con 0x7fb6b01056f0 2024-05-08T23:39:43.685 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.829+0000 7fb6b62fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b0105b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:43.685 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.829+0000 7fb6b62fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b0105b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33714/0 (socket says 172.21.15.12:33714) 2024-05-08T23:39:43.685 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.829+0000 7fb6b62fb700 1 -- 172.21.15.12:0/1100163133 learned_addr learned my addr 172.21.15.12:0/1100163133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:43.685 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.829+0000 7fb6b62fb700 1 -- 172.21.15.12:0/1100163133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b01068f0 con 0x7fb6b01056f0 2024-05-08T23:39:43.685 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.829+0000 7fb6b62fb700 1 --2- 172.21.15.12:0/1100163133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b0105b10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00bb40 tx=0x7fb6ac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae82a0ef4a535f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:43.685 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.829+0000 7fb6b52f9700 1 -- 172.21.15.12:0/1100163133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6ac00dbe0 con 0x7fb6b01056f0 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.830+0000 7fb6b52f9700 1 -- 172.21.15.12:0/1100163133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ac0044f0 con 0x7fb6b01056f0 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.830+0000 7fb6b52f9700 1 -- 172.21.15.12:0/1100163133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6ac003e60 con 0x7fb6b01056f0 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.830+0000 7fb6b855f700 1 -- 172.21.15.12:0/1100163133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 msgr2=0x7fb6b0105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.830+0000 7fb6b855f700 1 --2- 172.21.15.12:0/1100163133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b0105b10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00bb40 tx=0x7fb6ac00be50 comp rx=0 tx=0).stop 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.830+0000 7fb6b855f700 1 -- 172.21.15.12:0/1100163133 shutdown_connections 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.830+0000 7fb6b855f700 1 --2- 172.21.15.12:0/1100163133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b0105b10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.830+0000 7fb6b855f700 1 -- 172.21.15.12:0/1100163133 >> 172.21.15.12:0/1100163133 conn(0x7fb6b0100c90 msgr2=0x7fb6b01030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.830+0000 7fb6b855f700 1 -- 172.21.15.12:0/1100163133 shutdown_connections 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.830+0000 7fb6b855f700 1 -- 172.21.15.12:0/1100163133 wait complete. 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.831+0000 7fb6b855f700 1 Processor -- start 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.831+0000 7fb6b855f700 1 -- start start 2024-05-08T23:39:43.686 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.831+0000 7fb6b855f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.831+0000 7fb6b855f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b019cd20 con 0x7fb6b01056f0 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.831+0000 7fb6b62fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.831+0000 7fb6b62fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b019c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33724/0 (socket says 172.21.15.12:33724) 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.832+0000 7fb6b62fb700 1 -- 172.21.15.12:0/2459196947 learned_addr learned my addr 172.21.15.12:0/2459196947 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.832+0000 7fb6b62fb700 1 -- 172.21.15.12:0/2459196947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac00b7f0 con 0x7fb6b01056f0 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.832+0000 7fb6b62fb700 1 --2- 172.21.15.12:0/2459196947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b019c7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac003bf0 tx=0x7fb6ac003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.832+0000 7fb6a77fe700 1 -- 172.21.15.12:0/2459196947 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6ac010500 con 0x7fb6b01056f0 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.832+0000 7fb6a77fe700 1 -- 172.21.15.12:0/2459196947 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ac010b20 con 0x7fb6b01056f0 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.832+0000 7fb6b855f700 1 -- 172.21.15.12:0/2459196947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b019cf20 con 0x7fb6b01056f0 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.832+0000 7fb6a77fe700 1 -- 172.21.15.12:0/2459196947 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6ac019ab0 con 0x7fb6b01056f0 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.832+0000 7fb6b855f700 1 -- 172.21.15.12:0/2459196947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b019d360 con 0x7fb6b01056f0 2024-05-08T23:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.834+0000 7fb6a77fe700 1 -- 172.21.15.12:0/2459196947 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fb6ac019d30 con 0x7fb6b01056f0 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.834+0000 7fb6b855f700 1 -- 172.21.15.12:0/2459196947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b0073a00 con 0x7fb6b01056f0 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.834+0000 7fb6a77fe700 1 --2- 172.21.15.12:0/2459196947 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fb69c038700 0x7fb69c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.834+0000 7fb6a77fe700 1 -- 172.21.15.12:0/2459196947 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb6ac04c7e0 con 0x7fb6b01056f0 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.834+0000 7fb6b5afa700 1 --2- 172.21.15.12:0/2459196947 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fb69c038700 0x7fb69c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.834+0000 7fb6b5afa700 1 --2- 172.21.15.12:0/2459196947 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fb69c038700 0x7fb69c03abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0006fd0 tx=0x7fb6a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:42.839+0000 7fb6a77fe700 1 -- 172.21.15.12:0/2459196947 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb6ac00f5b0 con 0x7fb6b01056f0 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.087+0000 7fb6b855f700 1 -- 172.21.15.12:0/2459196947 --> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb6b00613e0 con 0x7fb69c038700 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.092+0000 7fb6a77fe700 1 -- 172.21.15.12:0/2459196947 <== mgr.14120 v2:172.21.15.12:6800/31089560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb6b00613e0 con 0x7fb69c038700 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.094+0000 7fb6b855f700 1 -- 172.21.15.12:0/2459196947 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fb69c038700 msgr2=0x7fb69c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.094+0000 7fb6b855f700 1 --2- 172.21.15.12:0/2459196947 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fb69c038700 0x7fb69c03abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0006fd0 tx=0x7fb6a0006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.094+0000 7fb6b855f700 1 -- 172.21.15.12:0/2459196947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 msgr2=0x7fb6b019c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:43.688 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.094+0000 7fb6b855f700 1 --2- 172.21.15.12:0/2459196947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b019c7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac003bf0 tx=0x7fb6ac003fb0 comp rx=0 tx=0).stop 2024-05-08T23:39:43.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.094+0000 7fb6b855f700 1 -- 172.21.15.12:0/2459196947 shutdown_connections 2024-05-08T23:39:43.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.094+0000 7fb6b855f700 1 --2- 172.21.15.12:0/2459196947 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fb69c038700 0x7fb69c03abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:43.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.094+0000 7fb6b855f700 1 --2- 172.21.15.12:0/2459196947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01056f0 0x7fb6b019c7e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:43.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.094+0000 7fb6b855f700 1 -- 172.21.15.12:0/2459196947 >> 172.21.15.12:0/2459196947 conn(0x7fb6b0100c90 msgr2=0x7fb6b018ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:43.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.094+0000 7fb6b855f700 1 -- 172.21.15.12:0/2459196947 shutdown_connections 2024-05-08T23:39:43.689 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:43.094+0000 7fb6b855f700 1 -- 172.21.15.12:0/2459196947 wait complete. 2024-05-08T23:39:44.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.061+0000 7f5b422f4700 1 Processor -- start 2024-05-08T23:39:44.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.062+0000 7f5b422f4700 1 -- start start 2024-05-08T23:39:44.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.062+0000 7f5b422f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:44.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.062+0000 7f5b422f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c107170 con 0x7f5b3c106780 2024-05-08T23:39:44.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.062+0000 7f5b3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:44.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.062+0000 7f5b3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33738/0 (socket says 172.21.15.12:33738) 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.062+0000 7f5b3bfff700 1 -- 172.21.15.12:0/3884025007 learned_addr learned my addr 172.21.15.12:0/3884025007 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.063+0000 7f5b3bfff700 1 -- 172.21.15.12:0/3884025007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c107980 con 0x7f5b3c106780 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.063+0000 7f5b3bfff700 1 --2- 172.21.15.12:0/3884025007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c106ba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0098a0 tx=0x7f5b2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03bc64927c201c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.063+0000 7f5b3affd700 1 -- 172.21.15.12:0/3884025007 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b2c004160 con 0x7f5b3c106780 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.063+0000 7f5b3affd700 1 -- 172.21.15.12:0/3884025007 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b2c0042c0 con 0x7f5b3c106780 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.063+0000 7f5b3affd700 1 -- 172.21.15.12:0/3884025007 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b2c0119b0 con 0x7f5b3c106780 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.064+0000 7f5b422f4700 1 -- 172.21.15.12:0/3884025007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 msgr2=0x7f5b3c106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.064+0000 7f5b422f4700 1 --2- 172.21.15.12:0/3884025007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c106ba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0098a0 tx=0x7f5b2c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.064+0000 7f5b422f4700 1 -- 172.21.15.12:0/3884025007 shutdown_connections 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.064+0000 7f5b422f4700 1 --2- 172.21.15.12:0/3884025007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c106ba0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:44.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.064+0000 7f5b422f4700 1 -- 172.21.15.12:0/3884025007 >> 172.21.15.12:0/3884025007 conn(0x7f5b3c101da0 msgr2=0x7f5b3c1041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.064+0000 7f5b422f4700 1 -- 172.21.15.12:0/3884025007 shutdown_connections 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.064+0000 7f5b422f4700 1 -- 172.21.15.12:0/3884025007 wait complete. 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.065+0000 7f5b422f4700 1 Processor -- start 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.065+0000 7f5b422f4700 1 -- start start 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.065+0000 7f5b422f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c07bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.065+0000 7f5b422f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c07c050 con 0x7f5b3c106780 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.065+0000 7f5b3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c07bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.065+0000 7f5b3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c07bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33754/0 (socket says 172.21.15.12:33754) 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.065+0000 7f5b3bfff700 1 -- 172.21.15.12:0/3828874458 learned_addr learned my addr 172.21.15.12:0/3828874458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.066+0000 7f5b3bfff700 1 -- 172.21.15.12:0/3828874458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c009550 con 0x7f5b3c106780 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.066+0000 7f5b3bfff700 1 --2- 172.21.15.12:0/3828874458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c07bb10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c003e60 tx=0x7f5b2c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.066+0000 7f5b397fa700 1 -- 172.21.15.12:0/3828874458 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b2c003a60 con 0x7f5b3c106780 2024-05-08T23:39:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.066+0000 7f5b422f4700 1 -- 172.21.15.12:0/3828874458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c07a290 con 0x7f5b3c106780 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.066+0000 7f5b397fa700 1 -- 172.21.15.12:0/3828874458 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b2c003bc0 con 0x7f5b3c106780 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.066+0000 7f5b397fa700 1 -- 172.21.15.12:0/3828874458 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b2c0196c0 con 0x7f5b3c106780 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.066+0000 7f5b422f4700 1 -- 172.21.15.12:0/3828874458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c07a730 con 0x7f5b3c106780 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.068+0000 7f5b422f4700 1 -- 172.21.15.12:0/3828874458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3c10abb0 con 0x7f5b3c106780 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.068+0000 7f5b397fa700 1 -- 172.21.15.12:0/3828874458 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f5b2c01ab60 con 0x7f5b3c106780 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.068+0000 7f5b397fa700 1 --2- 172.21.15.12:0/3828874458 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f5b240386e0 0x7f5b2403aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.068+0000 7f5b397fa700 1 -- 172.21.15.12:0/3828874458 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5b2c04cae0 con 0x7f5b3c106780 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.069+0000 7f5b3b7fe700 1 --2- 172.21.15.12:0/3828874458 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f5b240386e0 0x7f5b2403aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.069+0000 7f5b3b7fe700 1 --2- 172.21.15.12:0/3828874458 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f5b240386e0 0x7f5b2403aba0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5b30006fd0 tx=0x7f5b30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.074+0000 7f5b397fa700 1 -- 172.21.15.12:0/3828874458 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5b2c019980 con 0x7f5b3c106780 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.297+0000 7f5b422f4700 1 -- 172.21.15.12:0/3828874458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f5b3c10adf0 con 0x7f5b3c106780 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.302+0000 7f5b397fa700 1 -- 172.21.15.12:0/3828874458 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f5b2c00bb90 con 0x7f5b3c106780 2024-05-08T23:39:44.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.305+0000 7f5b422f4700 1 -- 172.21.15.12:0/3828874458 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f5b240386e0 msgr2=0x7f5b2403aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.305+0000 7f5b422f4700 1 --2- 172.21.15.12:0/3828874458 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f5b240386e0 0x7f5b2403aba0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5b30006fd0 tx=0x7f5b30006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.305+0000 7f5b422f4700 1 -- 172.21.15.12:0/3828874458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 msgr2=0x7f5b3c07bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.305+0000 7f5b422f4700 1 --2- 172.21.15.12:0/3828874458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c07bb10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c003e60 tx=0x7f5b2c011a30 comp rx=0 tx=0).stop 2024-05-08T23:39:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.305+0000 7f5b422f4700 1 -- 172.21.15.12:0/3828874458 shutdown_connections 2024-05-08T23:39:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.305+0000 7f5b422f4700 1 --2- 172.21.15.12:0/3828874458 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f5b240386e0 0x7f5b2403aba0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.305+0000 7f5b422f4700 1 --2- 172.21.15.12:0/3828874458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c106780 0x7f5b3c07bb10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.305+0000 7f5b422f4700 1 -- 172.21.15.12:0/3828874458 >> 172.21.15.12:0/3828874458 conn(0x7f5b3c101da0 msgr2=0x7f5b3c1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.306+0000 7f5b422f4700 1 -- 172.21.15.12:0/3828874458 shutdown_connections 2024-05-08T23:39:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:44.306+0000 7f5b422f4700 1 -- 172.21.15.12:0/3828874458 wait complete. 2024-05-08T23:39:44.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:44 smithi012 ceph-mon[42503]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:39:44.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:44 smithi012 ceph-mon[42503]: Saving service mgr spec with placement count:2 2024-05-08T23:39:44.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:44 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:44.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:44 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3828874458' entity='client.admin' 2024-05-08T23:39:44.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:44 smithi012 ceph-mon[42503]: from='mgr.14120 172.21.15.12:0/3326786642' entity='mgr.x' 2024-05-08T23:39:46.086 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.191+0000 7f21c0e2f700 1 Processor -- start 2024-05-08T23:39:46.086 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.191+0000 7f21c0e2f700 1 -- start start 2024-05-08T23:39:46.086 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.191+0000 7f21c0e2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc1054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:46.086 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.191+0000 7f21c0e2f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc074880 con 0x7f21bc1050b0 2024-05-08T23:39:46.086 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.191+0000 7f21ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc1054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:46.086 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.191+0000 7f21ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc1054d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33756/0 (socket says 172.21.15.12:33756) 2024-05-08T23:39:46.086 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.191+0000 7f21ba59c700 1 -- 172.21.15.12:0/1443540497 learned_addr learned my addr 172.21.15.12:0/1443540497 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:46.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.192+0000 7f21ba59c700 1 -- 172.21.15.12:0/1443540497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc105a10 con 0x7f21bc1050b0 2024-05-08T23:39:46.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.192+0000 7f21ba59c700 1 --2- 172.21.15.12:0/1443540497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc1054d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0098a0 tx=0x7f21ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faf60b4272f164d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:46.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.192+0000 7f21b959a700 1 -- 172.21.15.12:0/1443540497 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21ac004160 con 0x7f21bc1050b0 2024-05-08T23:39:46.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.192+0000 7f21b959a700 1 -- 172.21.15.12:0/1443540497 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21ac0042c0 con 0x7f21bc1050b0 2024-05-08T23:39:46.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.192+0000 7f21b959a700 1 -- 172.21.15.12:0/1443540497 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21ac0119b0 con 0x7f21bc1050b0 2024-05-08T23:39:46.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.192+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1443540497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 msgr2=0x7f21bc1054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:46.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.192+0000 7f21c0e2f700 1 --2- 172.21.15.12:0/1443540497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc1054d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0098a0 tx=0x7f21ac009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:46.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.193+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1443540497 shutdown_connections 2024-05-08T23:39:46.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.193+0000 7f21c0e2f700 1 --2- 172.21.15.12:0/1443540497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc1054d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:46.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.193+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1443540497 >> 172.21.15.12:0/1443540497 conn(0x7f21bc100c90 msgr2=0x7f21bc1030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:46.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.193+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1443540497 shutdown_connections 2024-05-08T23:39:46.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.193+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1443540497 wait complete. 2024-05-08T23:39:46.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.193+0000 7f21c0e2f700 1 Processor -- start 2024-05-08T23:39:46.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.194+0000 7f21c0e2f700 1 -- start start 2024-05-08T23:39:46.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.194+0000 7f21c0e2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:46.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.194+0000 7f21c0e2f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc19cce0 con 0x7f21bc1050b0 2024-05-08T23:39:46.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.194+0000 7f21ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.194+0000 7f21ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33770/0 (socket says 172.21.15.12:33770) 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.194+0000 7f21ba59c700 1 -- 172.21.15.12:0/1837665529 learned_addr learned my addr 172.21.15.12:0/1837665529 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.194+0000 7f21ba59c700 1 -- 172.21.15.12:0/1837665529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac009550 con 0x7f21bc1050b0 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.194+0000 7f21ba59c700 1 --2- 172.21.15.12:0/1837665529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc19c7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21ac011980 tx=0x7f21ac011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.194+0000 7f21b37fe700 1 -- 172.21.15.12:0/1837665529 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21ac003af0 con 0x7f21bc1050b0 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.195+0000 7f21b37fe700 1 -- 172.21.15.12:0/1837665529 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21ac003c50 con 0x7f21bc1050b0 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.195+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1837665529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc19cee0 con 0x7f21bc1050b0 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.195+0000 7f21b37fe700 1 -- 172.21.15.12:0/1837665529 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21ac019760 con 0x7f21bc1050b0 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.195+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1837665529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc19d380 con 0x7f21bc1050b0 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.195+0000 7f21b37fe700 1 -- 172.21.15.12:0/1837665529 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f21ac01a430 con 0x7f21bc1050b0 2024-05-08T23:39:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.196+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1837665529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc196470 con 0x7f21bc1050b0 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.196+0000 7f21b37fe700 1 --2- 172.21.15.12:0/1837665529 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f21a8038700 0x7f21a803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.196+0000 7f21b37fe700 1 -- 172.21.15.12:0/1837665529 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f21ac04c930 con 0x7f21bc1050b0 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.196+0000 7f21b9d9b700 1 --2- 172.21.15.12:0/1837665529 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f21a8038700 0x7f21a803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.196+0000 7f21b9d9b700 1 --2- 172.21.15.12:0/1837665529 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f21a8038700 0x7f21a803abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f21a4006fd0 tx=0x7f21a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.201+0000 7f21b37fe700 1 -- 172.21.15.12:0/1837665529 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f21ac02d950 con 0x7f21bc1050b0 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.423+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1837665529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f21bc106a90 con 0x7f21bc1050b0 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.427+0000 7f21b37fe700 1 -- 172.21.15.12:0/1837665529 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f21ac02d350 con 0x7f21bc1050b0 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.430+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1837665529 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f21a8038700 msgr2=0x7f21a803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.430+0000 7f21c0e2f700 1 --2- 172.21.15.12:0/1837665529 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f21a8038700 0x7f21a803abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f21a4006fd0 tx=0x7f21a4006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.430+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1837665529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 msgr2=0x7f21bc19c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.430+0000 7f21c0e2f700 1 --2- 172.21.15.12:0/1837665529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc19c7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21ac011980 tx=0x7f21ac011a60 comp rx=0 tx=0).stop 2024-05-08T23:39:46.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.430+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1837665529 shutdown_connections 2024-05-08T23:39:46.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.430+0000 7f21c0e2f700 1 --2- 172.21.15.12:0/1837665529 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f21a8038700 0x7f21a803abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:46.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.430+0000 7f21c0e2f700 1 --2- 172.21.15.12:0/1837665529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21bc1050b0 0x7f21bc19c7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:46.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.430+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1837665529 >> 172.21.15.12:0/1837665529 conn(0x7f21bc100c90 msgr2=0x7f21bc1935b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:46.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.430+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1837665529 shutdown_connections 2024-05-08T23:39:46.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:45.430+0000 7f21c0e2f700 1 -- 172.21.15.12:0/1837665529 wait complete. 2024-05-08T23:39:46.091 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling the dashboard module... 2024-05-08T23:39:46.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:46 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1837665529' entity='client.admin' 2024-05-08T23:39:47.763 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:47 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3965952161' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T23:39:47.763 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:47 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: ignoring --setuser ceph since I am not root 2024-05-08T23:39:47.763 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:47 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: ignoring --setgroup ceph since I am not root 2024-05-08T23:39:47.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.471+0000 7f933b4de700 1 Processor -- start 2024-05-08T23:39:47.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.472+0000 7f933b4de700 1 -- start start 2024-05-08T23:39:47.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.472+0000 7f933b4de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f93341096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:47.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.472+0000 7f933b4de700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9334079e20 con 0x7f93341072d0 2024-05-08T23:39:47.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.472+0000 7f933927a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f93341096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:47.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.472+0000 7f933927a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f93341096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33784/0 (socket says 172.21.15.12:33784) 2024-05-08T23:39:47.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.472+0000 7f933927a700 1 -- 172.21.15.12:0/207266966 learned_addr learned my addr 172.21.15.12:0/207266966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:47.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.473+0000 7f933927a700 1 -- 172.21.15.12:0/207266966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334109c00 con 0x7f93341072d0 2024-05-08T23:39:47.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.473+0000 7f933927a700 1 --2- 172.21.15.12:0/207266966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f93341096c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f93280098a0 tx=0x7f9328009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2fc60b84298b86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.473+0000 7f9327fff700 1 -- 172.21.15.12:0/207266966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9328004160 con 0x7f93341072d0 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.473+0000 7f9327fff700 1 -- 172.21.15.12:0/207266966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93280042c0 con 0x7f93341072d0 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.473+0000 7f9327fff700 1 -- 172.21.15.12:0/207266966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93280119b0 con 0x7f93341072d0 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.473+0000 7f933b4de700 1 -- 172.21.15.12:0/207266966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 msgr2=0x7f93341096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.473+0000 7f933b4de700 1 --2- 172.21.15.12:0/207266966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f93341096c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f93280098a0 tx=0x7f9328009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.474+0000 7f933b4de700 1 -- 172.21.15.12:0/207266966 shutdown_connections 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.474+0000 7f933b4de700 1 --2- 172.21.15.12:0/207266966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f93341096c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.474+0000 7f933b4de700 1 -- 172.21.15.12:0/207266966 >> 172.21.15.12:0/207266966 conn(0x7f9334100ca0 msgr2=0x7f93341030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.474+0000 7f933b4de700 1 -- 172.21.15.12:0/207266966 shutdown_connections 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.474+0000 7f933b4de700 1 -- 172.21.15.12:0/207266966 wait complete. 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.474+0000 7f933b4de700 1 Processor -- start 2024-05-08T23:39:47.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.474+0000 7f933b4de700 1 -- start start 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f933b4de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f933419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f933b4de700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933419a9f0 con 0x7f93341072d0 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f933927a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f933419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f933927a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f933419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33800/0 (socket says 172.21.15.12:33800) 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f933927a700 1 -- 172.21.15.12:0/3965952161 learned_addr learned my addr 172.21.15.12:0/3965952161 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f933927a700 1 -- 172.21.15.12:0/3965952161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9328009550 con 0x7f93341072d0 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f933927a700 1 --2- 172.21.15.12:0/3965952161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f933419a4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9328006af0 tx=0x7f9328011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f93267fc700 1 -- 172.21.15.12:0/3965952161 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9328003a40 con 0x7f93341072d0 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f93267fc700 1 -- 172.21.15.12:0/3965952161 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9328003ba0 con 0x7f93341072d0 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f933b4de700 1 -- 172.21.15.12:0/3965952161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933419abf0 con 0x7f93341072d0 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f93267fc700 1 -- 172.21.15.12:0/3965952161 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93280196c0 con 0x7f93341072d0 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.475+0000 7f933b4de700 1 -- 172.21.15.12:0/3965952161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933419b090 con 0x7f93341072d0 2024-05-08T23:39:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.476+0000 7f93267fc700 1 -- 172.21.15.12:0/3965952161 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f9328019930 con 0x7f93341072d0 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.476+0000 7f93267fc700 1 --2- 172.21.15.12:0/3965952161 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f9320038700 0x7f932003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.476+0000 7f93267fc700 1 -- 172.21.15.12:0/3965952161 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f932804c820 con 0x7f93341072d0 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.476+0000 7f933b4de700 1 -- 172.21.15.12:0/3965952161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933410a990 con 0x7f93341072d0 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.477+0000 7f9338a79700 1 --2- 172.21.15.12:0/3965952161 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f9320038700 0x7f932003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.477+0000 7f9338a79700 1 --2- 172.21.15.12:0/3965952161 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f9320038700 0x7f932003abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9330006fd0 tx=0x7f9330006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.482+0000 7f93267fc700 1 -- 172.21.15.12:0/3965952161 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f932802b340 con 0x7f93341072d0 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:46.752+0000 7f933b4de700 1 -- 172.21.15.12:0/3965952161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f933419db80 con 0x7f93341072d0 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.435+0000 7f93267fc700 1 -- 172.21.15.12:0/3965952161 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f932802d960 con 0x7f93341072d0 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.436+0000 7f93267fc700 1 -- 172.21.15.12:0/3965952161 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f932800f2e0 con 0x7f93341072d0 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.440+0000 7f933b4de700 1 -- 172.21.15.12:0/3965952161 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f9320038700 msgr2=0x7f932003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.440+0000 7f933b4de700 1 --2- 172.21.15.12:0/3965952161 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f9320038700 0x7f932003abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9330006fd0 tx=0x7f9330006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.440+0000 7f933b4de700 1 -- 172.21.15.12:0/3965952161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 msgr2=0x7f933419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.440+0000 7f933b4de700 1 --2- 172.21.15.12:0/3965952161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f933419a4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9328006af0 tx=0x7f9328011a10 comp rx=0 tx=0).stop 2024-05-08T23:39:47.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.441+0000 7f933b4de700 1 -- 172.21.15.12:0/3965952161 shutdown_connections 2024-05-08T23:39:47.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.441+0000 7f933b4de700 1 --2- 172.21.15.12:0/3965952161 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f9320038700 0x7f932003abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:47.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.441+0000 7f933b4de700 1 --2- 172.21.15.12:0/3965952161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93341072d0 0x7f933419a4b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:47.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.441+0000 7f933b4de700 1 -- 172.21.15.12:0/3965952161 >> 172.21.15.12:0/3965952161 conn(0x7f9334100ca0 msgr2=0x7f9334101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:47.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.441+0000 7f933b4de700 1 -- 172.21.15.12:0/3965952161 shutdown_connections 2024-05-08T23:39:47.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:47.441+0000 7f933b4de700 1 -- 172.21.15.12:0/3965952161 wait complete. 2024-05-08T23:39:48.054 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:47 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:47.949+0000 7f34eec0a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:39:48.416 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:48 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:48.056+0000 7f34eec0a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:39:48.416 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:48 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:48.158+0000 7f34eec0a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:39:48.416 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:48 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:48.295+0000 7f34eec0a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:39:48.724 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:48 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3965952161' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T23:39:48.724 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:48 smithi012 ceph-mon[42503]: mgrmap e9: x(active, since 15s) 2024-05-08T23:39:48.893 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:39:48.893 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-08T23:39:48.893 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T23:39:48.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T23:39:48.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T23:39:48.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:39:48.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.189+0000 7f0a833d0700 1 Processor -- start 2024-05-08T23:39:48.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.190+0000 7f0a833d0700 1 -- start start 2024-05-08T23:39:48.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.190+0000 7f0a833d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c1054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:48.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.190+0000 7f0a833d0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c074880 con 0x7f0a7c1050a0 2024-05-08T23:39:48.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.190+0000 7f0a8116c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c1054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:48.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.190+0000 7f0a8116c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c1054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33834/0 (socket says 172.21.15.12:33834) 2024-05-08T23:39:48.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.190+0000 7f0a8116c700 1 -- 172.21.15.12:0/218383369 learned_addr learned my addr 172.21.15.12:0/218383369 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.191+0000 7f0a8116c700 1 -- 172.21.15.12:0/218383369 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c105a00 con 0x7f0a7c1050a0 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.191+0000 7f0a8116c700 1 --2- 172.21.15.12:0/218383369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c1054c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a780098a0 tx=0x7f0a78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800053ddab04e55e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.191+0000 7f0a73fff700 1 -- 172.21.15.12:0/218383369 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a78004160 con 0x7f0a7c1050a0 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.191+0000 7f0a73fff700 1 -- 172.21.15.12:0/218383369 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a780042c0 con 0x7f0a7c1050a0 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.191+0000 7f0a73fff700 1 -- 172.21.15.12:0/218383369 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a780119b0 con 0x7f0a7c1050a0 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.192+0000 7f0a833d0700 1 -- 172.21.15.12:0/218383369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 msgr2=0x7f0a7c1054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.192+0000 7f0a833d0700 1 --2- 172.21.15.12:0/218383369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c1054c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a780098a0 tx=0x7f0a78009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.192+0000 7f0a833d0700 1 -- 172.21.15.12:0/218383369 shutdown_connections 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.192+0000 7f0a833d0700 1 --2- 172.21.15.12:0/218383369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c1054c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.192+0000 7f0a833d0700 1 -- 172.21.15.12:0/218383369 >> 172.21.15.12:0/218383369 conn(0x7f0a7c100c60 msgr2=0x7f0a7c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.192+0000 7f0a833d0700 1 -- 172.21.15.12:0/218383369 shutdown_connections 2024-05-08T23:39:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.192+0000 7f0a833d0700 1 -- 172.21.15.12:0/218383369 wait complete. 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.192+0000 7f0a833d0700 1 Processor -- start 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.193+0000 7f0a833d0700 1 -- start start 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.193+0000 7f0a833d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.193+0000 7f0a833d0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c19ccf0 con 0x7f0a7c1050a0 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.193+0000 7f0a8116c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.193+0000 7f0a8116c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c19c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33846/0 (socket says 172.21.15.12:33846) 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.193+0000 7f0a8116c700 1 -- 172.21.15.12:0/1149024374 learned_addr learned my addr 172.21.15.12:0/1149024374 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.193+0000 7f0a8116c700 1 -- 172.21.15.12:0/1149024374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a78009550 con 0x7f0a7c1050a0 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.193+0000 7f0a8116c700 1 --2- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c19c7b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a78011980 tx=0x7f0a78011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.194+0000 7f0a727fc700 1 -- 172.21.15.12:0/1149024374 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a78003af0 con 0x7f0a7c1050a0 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.194+0000 7f0a727fc700 1 -- 172.21.15.12:0/1149024374 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a78003c50 con 0x7f0a7c1050a0 2024-05-08T23:39:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.194+0000 7f0a833d0700 1 -- 172.21.15.12:0/1149024374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c19cef0 con 0x7f0a7c1050a0 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.194+0000 7f0a727fc700 1 -- 172.21.15.12:0/1149024374 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a78019760 con 0x7f0a7c1050a0 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.194+0000 7f0a833d0700 1 -- 172.21.15.12:0/1149024374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c19d390 con 0x7f0a7c1050a0 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.195+0000 7f0a727fc700 1 -- 172.21.15.12:0/1149024374 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f0a78019950 con 0x7f0a7c1050a0 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.195+0000 7f0a727fc700 1 --2- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0a680387a0 0x7f0a6803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.195+0000 7f0a727fc700 1 -- 172.21.15.12:0/1149024374 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0a7804c9c0 con 0x7f0a7c1050a0 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.195+0000 7f0a8096b700 1 -- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0a680387a0 msgr2=0x7f0a6803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/31089560 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.195+0000 7f0a8096b700 1 --2- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0a680387a0 0x7f0a6803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.195+0000 7f0a833d0700 1 -- 172.21.15.12:0/1149024374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7c1964c0 con 0x7f0a7c1050a0 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.205+0000 7f0a727fc700 1 -- 172.21.15.12:0/1149024374 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0a78018db0 con 0x7f0a7c1050a0 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.396+0000 7f0a8096b700 1 -- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0a680387a0 msgr2=0x7f0a6803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/31089560 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.396+0000 7f0a8096b700 1 --2- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0a680387a0 0x7f0a6803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.527+0000 7f0a833d0700 1 -- 172.21.15.12:0/1149024374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0a7c02ce40 con 0x7f0a7c1050a0 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.528+0000 7f0a727fc700 1 -- 172.21.15.12:0/1149024374 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0a78018350 con 0x7f0a7c1050a0 2024-05-08T23:39:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.530+0000 7f0a833d0700 1 -- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0a680387a0 msgr2=0x7f0a6803ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.530+0000 7f0a833d0700 1 --2- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0a680387a0 0x7f0a6803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.530+0000 7f0a833d0700 1 -- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 msgr2=0x7f0a7c19c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.530+0000 7f0a833d0700 1 --2- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c19c7b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a78011980 tx=0x7f0a78011a60 comp rx=0 tx=0).stop 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.531+0000 7f0a833d0700 1 -- 172.21.15.12:0/1149024374 shutdown_connections 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.531+0000 7f0a833d0700 1 --2- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7f0a680387a0 0x7f0a6803ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.531+0000 7f0a833d0700 1 --2- 172.21.15.12:0/1149024374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a7c1050a0 0x7f0a7c19c7b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.531+0000 7f0a833d0700 1 -- 172.21.15.12:0/1149024374 >> 172.21.15.12:0/1149024374 conn(0x7f0a7c100c60 msgr2=0x7f0a7c193600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.531+0000 7f0a833d0700 1 -- 172.21.15.12:0/1149024374 shutdown_connections 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:48.531+0000 7f0a833d0700 1 -- 172.21.15.12:0/1149024374 wait complete. 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-05-08T23:39:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 9... 2024-05-08T23:39:49.336 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:49.048+0000 7f34eec0a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:39:49.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:49 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1149024374' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T23:39:49.667 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:49.339+0000 7f34eec0a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:39:49.667 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:49.436+0000 7f34eec0a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:39:49.667 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:49.560+0000 7f34eec0a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:39:50.167 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:49.740+0000 7f34eec0a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:39:50.167 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:49.885+0000 7f34eec0a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:39:50.167 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:49.982+0000 7f34eec0a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:39:50.666 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:50.406+0000 7f34eec0a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:39:51.166 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:50.849+0000 7f34eec0a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:39:51.166 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:51 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:51.068+0000 7f34eec0a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:39:51.666 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:51 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:51.174+0000 7f34eec0a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:39:52.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:52.047+0000 7f34eec0a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:39:52.327 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:52.156+0000 7f34eec0a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:39:52.666 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:52.325+0000 7f34eec0a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:39:53.416 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:53 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:53.046+0000 7f34eec0a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:39:53.416 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:53 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:53.252+0000 7f34eec0a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:39:53.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:53 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:53.472+0000 7f34eec0a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:39:54.855 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:54 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:54.534+0000 7f34eec0a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:39:55.166 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:54 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:54.858+0000 7f34eec0a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:39:55.166 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:54 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:54.958+0000 7f34eec0a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:39:55.667 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:39:55 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:39:55.175+0000 7f34eec0a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:39:55.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: Active manager daemon x restarted 2024-05-08T23:39:55.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: Activating manager daemon x 2024-05-08T23:39:55.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T23:39:55.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: mgrmap e10: x(active, starting, since 0.00350779s) 2024-05-08T23:39:55.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:39:55.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:39:55.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:39:55.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:39:55.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:39:55.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: Manager daemon x is now available 2024-05-08T23:39:55.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:39:56.502 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:39:56.502 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:39:56.502 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:56 smithi012 ceph-mon[42503]: mgrmap e11: x(active, since 1.00616s) 2024-05-08T23:39:56.502 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:39:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:39:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-08T23:39:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T23:39:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:39:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.266+0000 7fa720e57700 1 Processor -- start 2024-05-08T23:39:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.266+0000 7fa720e57700 1 -- start start 2024-05-08T23:39:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.267+0000 7fa720e57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c109670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.267+0000 7fa720e57700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c079e40 con 0x7fa71c107280 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.267+0000 7fa71a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.267+0000 7fa71a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c109670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56386/0 (socket says 172.21.15.12:56386) 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.267+0000 7fa71a59c700 1 -- 172.21.15.12:0/382929779 learned_addr learned my addr 172.21.15.12:0/382929779 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.267+0000 7fa71a59c700 1 -- 172.21.15.12:0/382929779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c109bb0 con 0x7fa71c107280 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.267+0000 7fa71a59c700 1 --2- 172.21.15.12:0/382929779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c109670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0098a0 tx=0x7fa70c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97221c2024432977 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.267+0000 7fa71959a700 1 -- 172.21.15.12:0/382929779 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa70c004160 con 0x7fa71c107280 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.268+0000 7fa71959a700 1 -- 172.21.15.12:0/382929779 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70c0042c0 con 0x7fa71c107280 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.268+0000 7fa71959a700 1 -- 172.21.15.12:0/382929779 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa70c0119b0 con 0x7fa71c107280 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.268+0000 7fa720e57700 1 -- 172.21.15.12:0/382929779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 msgr2=0x7fa71c109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.268+0000 7fa720e57700 1 --2- 172.21.15.12:0/382929779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c109670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0098a0 tx=0x7fa70c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.268+0000 7fa720e57700 1 -- 172.21.15.12:0/382929779 shutdown_connections 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.269+0000 7fa720e57700 1 --2- 172.21.15.12:0/382929779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c109670 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.269+0000 7fa720e57700 1 -- 172.21.15.12:0/382929779 >> 172.21.15.12:0/382929779 conn(0x7fa71c100c70 msgr2=0x7fa71c103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.269+0000 7fa720e57700 1 -- 172.21.15.12:0/382929779 shutdown_connections 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.269+0000 7fa720e57700 1 -- 172.21.15.12:0/382929779 wait complete. 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.269+0000 7fa720e57700 1 Processor -- start 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.270+0000 7fa720e57700 1 -- start start 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.270+0000 7fa720e57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.270+0000 7fa720e57700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c19aa40 con 0x7fa71c107280 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.270+0000 7fa71a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.270+0000 7fa71a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56406/0 (socket says 172.21.15.12:56406) 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.270+0000 7fa71a59c700 1 -- 172.21.15.12:0/3233543306 learned_addr learned my addr 172.21.15.12:0/3233543306 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.270+0000 7fa71a59c700 1 -- 172.21.15.12:0/3233543306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c009550 con 0x7fa71c107280 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.270+0000 7fa71a59c700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c19a500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa70c006af0 tx=0x7fa70c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.271+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa70c003a40 con 0x7fa71c107280 2024-05-08T23:39:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.271+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70c003ba0 con 0x7fa71c107280 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.271+0000 7fa720e57700 1 -- 172.21.15.12:0/3233543306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71c19ac40 con 0x7fa71c107280 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.271+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa70c0196c0 con 0x7fa71c107280 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.271+0000 7fa720e57700 1 -- 172.21.15.12:0/3233543306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71c19b0e0 con 0x7fa71c107280 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.272+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fa70c019980 con 0x7fa71c107280 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.272+0000 7fa70b7fe700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 0x7fa70403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.272+0000 7fa720e57700 1 -- 172.21.15.12:0/3233543306 --> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa71c10a990 con 0x7fa704038700 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.272+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa70c04c7a0 con 0x7fa71c107280 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.272+0000 7fa719d9b700 1 -- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 msgr2=0x7fa70403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/31089560 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.272+0000 7fa719d9b700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 0x7fa70403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.473+0000 7fa719d9b700 1 -- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 msgr2=0x7fa70403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/31089560 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.473+0000 7fa719d9b700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 0x7fa70403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.874+0000 7fa719d9b700 1 -- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 msgr2=0x7fa70403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/31089560 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:49.874+0000 7fa719d9b700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 0x7fa70403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:39:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:50.675+0000 7fa719d9b700 1 -- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 msgr2=0x7fa70403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/31089560 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:50.675+0000 7fa719d9b700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 0x7fa70403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:52.276+0000 7fa719d9b700 1 -- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 msgr2=0x7fa70403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/31089560 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:52.276+0000 7fa719d9b700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 0x7fa70403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:55.190+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7fa70c02dcd0 con 0x7fa71c107280 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:55.190+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 msgr2=0x7fa70403abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:55.190+0000 7fa70b7fe700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 0x7fa70403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.192+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fa70c04c0f0 con 0x7fa71c107280 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.192+0000 7fa70b7fe700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa70403bfc0 0x7fa70403e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.192+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa71c10a990 con 0x7fa70403bfc0 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.192+0000 7fa719d9b700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa70403bfc0 0x7fa70403e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.193+0000 7fa719d9b700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa70403bfc0 0x7fa70403e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa710003a10 tx=0x7fa7100092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.194+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fa71c10a990 con 0x7fa70403bfc0 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 -- 172.21.15.12:0/3233543306 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa71c10ab00 con 0x7fa70403bfc0 2024-05-08T23:39:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa70b7fe700 1 -- 172.21.15.12:0/3233543306 <== mgr.14152 v2:172.21.15.12:6800/1305168755 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa71c10ab00 con 0x7fa70403bfc0 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 -- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa70403bfc0 msgr2=0x7fa70403e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa70403bfc0 0x7fa70403e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa710003a10 tx=0x7fa7100092b0 comp rx=0 tx=0).stop 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 -- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 msgr2=0x7fa71c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c19a500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa70c006af0 tx=0x7fa70c011a10 comp rx=0 tx=0).stop 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 -- 172.21.15.12:0/3233543306 shutdown_connections 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa70403bfc0 0x7fa70403e3b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa71c107280 0x7fa71c19a500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 --2- 172.21.15.12:0/3233543306 >> [v2:172.21.15.12:6800/31089560,v1:172.21.15.12:6801/31089560] conn(0x7fa704038700 0x7fa70403abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 -- 172.21.15.12:0/3233543306 >> 172.21.15.12:0/3233543306 conn(0x7fa71c100c70 msgr2=0x7fa71c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 -- 172.21.15.12:0/3233543306 shutdown_connections 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.201+0000 7fa720e57700 1 -- 172.21.15.12:0/3233543306 wait complete. 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 9 is available 2024-05-08T23:39:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:Generating a dashboard self-signed certificate... 2024-05-08T23:39:57.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:57 smithi012 ceph-mon[42503]: [08/May/2024:23:39:55] ENGINE Bus STARTING 2024-05-08T23:39:57.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:57 smithi012 ceph-mon[42503]: [08/May/2024:23:39:55] ENGINE Serving on http://172.21.15.12:8765 2024-05-08T23:39:57.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:57 smithi012 ceph-mon[42503]: [08/May/2024:23:39:56] ENGINE Serving on https://172.21.15.12:7150 2024-05-08T23:39:57.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:57 smithi012 ceph-mon[42503]: [08/May/2024:23:39:56] ENGINE Bus STARTED 2024-05-08T23:39:57.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:57 smithi012 ceph-mon[42503]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T23:39:57.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:57 smithi012 ceph-mon[42503]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T23:39:57.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:57 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:39:57.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T23:39:57.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.962+0000 7fbf77eca700 1 Processor -- start 2024-05-08T23:39:57.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.963+0000 7fbf77eca700 1 -- start start 2024-05-08T23:39:57.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.963+0000 7fbf77eca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf70106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:57.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.963+0000 7fbf77eca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70107180 con 0x7fbf70106790 2024-05-08T23:39:57.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.963+0000 7fbf75c66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf70106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:57.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.963+0000 7fbf75c66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf70106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56476/0 (socket says 172.21.15.12:56476) 2024-05-08T23:39:57.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.963+0000 7fbf75c66700 1 -- 172.21.15.12:0/817149103 learned_addr learned my addr 172.21.15.12:0/817149103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:57.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.963+0000 7fbf75c66700 1 -- 172.21.15.12:0/817149103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf70107990 con 0x7fbf70106790 2024-05-08T23:39:57.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.964+0000 7fbf75c66700 1 --2- 172.21.15.12:0/817149103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf70106bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0098a0 tx=0x7fbf6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed175b6b7c91d93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.964+0000 7fbf74c64700 1 -- 172.21.15.12:0/817149103 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf6c004160 con 0x7fbf70106790 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.964+0000 7fbf74c64700 1 -- 172.21.15.12:0/817149103 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf6c0042c0 con 0x7fbf70106790 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.964+0000 7fbf74c64700 1 -- 172.21.15.12:0/817149103 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf6c0119b0 con 0x7fbf70106790 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.965+0000 7fbf77eca700 1 -- 172.21.15.12:0/817149103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 msgr2=0x7fbf70106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.965+0000 7fbf77eca700 1 --2- 172.21.15.12:0/817149103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf70106bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0098a0 tx=0x7fbf6c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.965+0000 7fbf77eca700 1 -- 172.21.15.12:0/817149103 shutdown_connections 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.965+0000 7fbf77eca700 1 --2- 172.21.15.12:0/817149103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf70106bb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.965+0000 7fbf77eca700 1 -- 172.21.15.12:0/817149103 >> 172.21.15.12:0/817149103 conn(0x7fbf70101df0 msgr2=0x7fbf70104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.965+0000 7fbf77eca700 1 -- 172.21.15.12:0/817149103 shutdown_connections 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.965+0000 7fbf77eca700 1 -- 172.21.15.12:0/817149103 wait complete. 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.966+0000 7fbf77eca700 1 Processor -- start 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.966+0000 7fbf77eca700 1 -- start start 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.966+0000 7fbf77eca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf7019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:57.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.966+0000 7fbf77eca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7019aa60 con 0x7fbf70106790 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.966+0000 7fbf75c66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf7019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.966+0000 7fbf75c66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf7019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56478/0 (socket says 172.21.15.12:56478) 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.966+0000 7fbf75c66700 1 -- 172.21.15.12:0/925196536 learned_addr learned my addr 172.21.15.12:0/925196536 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.967+0000 7fbf75c66700 1 -- 172.21.15.12:0/925196536 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c009550 con 0x7fbf70106790 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.967+0000 7fbf75c66700 1 --2- 172.21.15.12:0/925196536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf7019a520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c006af0 tx=0x7fbf6c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.967+0000 7fbf66ffd700 1 -- 172.21.15.12:0/925196536 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf6c003a40 con 0x7fbf70106790 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.967+0000 7fbf77eca700 1 -- 172.21.15.12:0/925196536 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7019ac60 con 0x7fbf70106790 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.967+0000 7fbf66ffd700 1 -- 172.21.15.12:0/925196536 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf6c003ba0 con 0x7fbf70106790 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.967+0000 7fbf66ffd700 1 -- 172.21.15.12:0/925196536 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf6c019690 con 0x7fbf70106790 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.967+0000 7fbf77eca700 1 -- 172.21.15.12:0/925196536 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7019b100 con 0x7fbf70106790 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.968+0000 7fbf66ffd700 1 -- 172.21.15.12:0/925196536 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fbf6c019950 con 0x7fbf70106790 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.968+0000 7fbf66ffd700 1 --2- 172.21.15.12:0/925196536 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbf5c0385e0 0x7fbf5c03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:57.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.968+0000 7fbf66ffd700 1 -- 172.21.15.12:0/925196536 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbf6c04c980 con 0x7fbf70106790 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.968+0000 7fbf77eca700 1 -- 172.21.15.12:0/925196536 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7010abc0 con 0x7fbf70106790 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.969+0000 7fbf75465700 1 --2- 172.21.15.12:0/925196536 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbf5c0385e0 0x7fbf5c03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.969+0000 7fbf75465700 1 --2- 172.21.15.12:0/925196536 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbf5c0385e0 0x7fbf5c03aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf60006fd0 tx=0x7fbf60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:56.975+0000 7fbf66ffd700 1 -- 172.21.15.12:0/925196536 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbf6c0243f0 con 0x7fbf70106790 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.216+0000 7fbf77eca700 1 -- 172.21.15.12:0/925196536 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf70103120 con 0x7fbf5c0385e0 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.290+0000 7fbf66ffd700 1 -- 172.21.15.12:0/925196536 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fbf70103120 con 0x7fbf5c0385e0 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.293+0000 7fbf77eca700 1 -- 172.21.15.12:0/925196536 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbf5c0385e0 msgr2=0x7fbf5c03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.293+0000 7fbf77eca700 1 --2- 172.21.15.12:0/925196536 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbf5c0385e0 0x7fbf5c03aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf60006fd0 tx=0x7fbf60006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.293+0000 7fbf77eca700 1 -- 172.21.15.12:0/925196536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 msgr2=0x7fbf7019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.293+0000 7fbf77eca700 1 --2- 172.21.15.12:0/925196536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf7019a520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c006af0 tx=0x7fbf6c011a10 comp rx=0 tx=0).stop 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.293+0000 7fbf77eca700 1 -- 172.21.15.12:0/925196536 shutdown_connections 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.293+0000 7fbf77eca700 1 --2- 172.21.15.12:0/925196536 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbf5c0385e0 0x7fbf5c03aaa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:57.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.293+0000 7fbf77eca700 1 --2- 172.21.15.12:0/925196536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf70106790 0x7fbf7019a520 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c006af0 tx=0x7fbf6c011a10 comp rx=0 tx=0).stop 2024-05-08T23:39:57.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.293+0000 7fbf77eca700 1 -- 172.21.15.12:0/925196536 >> 172.21.15.12:0/925196536 conn(0x7fbf70101df0 msgr2=0x7fbf70102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:57.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.293+0000 7fbf77eca700 1 -- 172.21.15.12:0/925196536 shutdown_connections 2024-05-08T23:39:57.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:57.293+0000 7fbf77eca700 1 -- 172.21.15.12:0/925196536 wait complete. 2024-05-08T23:39:57.676 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial admin user... 2024-05-08T23:39:58.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:58 smithi012 ceph-mon[42503]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:39:58.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:39:58.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:39:58.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:58 smithi012 ceph-mon[42503]: mgrmap e12: x(active, since 2s) 2024-05-08T23:39:59.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$BC5bMqf5srfNBjo.Eb44BuRuSa9OWoe8ovgYBLL6iOaAnXAa/O4fi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715211598, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T23:39:59.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.240+0000 7f3987a59700 1 Processor -- start 2024-05-08T23:39:59.105 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.240+0000 7f3987a59700 1 -- start start 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.240+0000 7f3987a59700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f3980109720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.241+0000 7f3987a59700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3980074800 con 0x7f3980107330 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.241+0000 7f39857f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f3980109720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.241+0000 7f39857f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f3980109720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56488/0 (socket says 172.21.15.12:56488) 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.241+0000 7f39857f5700 1 -- 172.21.15.12:0/798183304 learned_addr learned my addr 172.21.15.12:0/798183304 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.241+0000 7f39857f5700 1 -- 172.21.15.12:0/798183304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3980074940 con 0x7f3980107330 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.241+0000 7f39857f5700 1 --2- 172.21.15.12:0/798183304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f3980109720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f397c0098a0 tx=0x7f397c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41335a102023d27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.241+0000 7f3973fff700 1 -- 172.21.15.12:0/798183304 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f397c004160 con 0x7f3980107330 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.241+0000 7f3973fff700 1 -- 172.21.15.12:0/798183304 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f397c0042c0 con 0x7f3980107330 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.241+0000 7f3973fff700 1 -- 172.21.15.12:0/798183304 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f397c0119b0 con 0x7f3980107330 2024-05-08T23:39:59.106 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.242+0000 7f3987a59700 1 -- 172.21.15.12:0/798183304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 msgr2=0x7f3980109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.242+0000 7f3987a59700 1 --2- 172.21.15.12:0/798183304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f3980109720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f397c0098a0 tx=0x7f397c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.242+0000 7f3987a59700 1 -- 172.21.15.12:0/798183304 shutdown_connections 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.242+0000 7f3987a59700 1 --2- 172.21.15.12:0/798183304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f3980109720 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.242+0000 7f3987a59700 1 -- 172.21.15.12:0/798183304 >> 172.21.15.12:0/798183304 conn(0x7f3980100c60 msgr2=0x7f39801030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.242+0000 7f3987a59700 1 -- 172.21.15.12:0/798183304 shutdown_connections 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.242+0000 7f3987a59700 1 -- 172.21.15.12:0/798183304 wait complete. 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.243+0000 7f3987a59700 1 Processor -- start 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.243+0000 7f3987a59700 1 -- start start 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.243+0000 7f3987a59700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f398019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.243+0000 7f3987a59700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398019ccb0 con 0x7f3980107330 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.243+0000 7f39857f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f398019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.243+0000 7f39857f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f398019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56494/0 (socket says 172.21.15.12:56494) 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.243+0000 7f39857f5700 1 -- 172.21.15.12:0/3890358925 learned_addr learned my addr 172.21.15.12:0/3890358925 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:39:59.107 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.244+0000 7f39857f5700 1 -- 172.21.15.12:0/3890358925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c009550 con 0x7f3980107330 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.244+0000 7f39857f5700 1 --2- 172.21.15.12:0/3890358925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f398019c770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f397c011980 tx=0x7f397c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.244+0000 7f39727fc700 1 -- 172.21.15.12:0/3890358925 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f397c003af0 con 0x7f3980107330 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.244+0000 7f39727fc700 1 -- 172.21.15.12:0/3890358925 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f397c003c50 con 0x7f3980107330 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.244+0000 7f3987a59700 1 -- 172.21.15.12:0/3890358925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398019ceb0 con 0x7f3980107330 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.244+0000 7f39727fc700 1 -- 172.21.15.12:0/3890358925 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f397c019760 con 0x7f3980107330 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.244+0000 7f3987a59700 1 -- 172.21.15.12:0/3890358925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398019d350 con 0x7f3980107330 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.245+0000 7f39727fc700 1 -- 172.21.15.12:0/3890358925 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f397c01a430 con 0x7f3980107330 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.245+0000 7f39727fc700 1 --2- 172.21.15.12:0/3890358925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f396c0387a0 0x7f396c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.245+0000 7f39727fc700 1 -- 172.21.15.12:0/3890358925 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f397c04ca20 con 0x7f3980107330 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.245+0000 7f3987a59700 1 -- 172.21.15.12:0/3890358925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39800625d0 con 0x7f3980107330 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.246+0000 7f3984ff4700 1 --2- 172.21.15.12:0/3890358925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f396c0387a0 0x7f396c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.246+0000 7f3984ff4700 1 --2- 172.21.15.12:0/3890358925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f396c0387a0 0x7f396c03ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3974006fd0 tx=0x7f3974006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:59.108 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.251+0000 7f39727fc700 1 -- 172.21.15.12:0/3890358925 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f397c02d950 con 0x7f3980107330 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.478+0000 7f3987a59700 1 -- 172.21.15.12:0/3890358925 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f398010a9d0 con 0x7f396c0387a0 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.731+0000 7f39727fc700 1 -- 172.21.15.12:0/3890358925 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f398010a9d0 con 0x7f396c0387a0 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.734+0000 7f3987a59700 1 -- 172.21.15.12:0/3890358925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f396c0387a0 msgr2=0x7f396c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.734+0000 7f3987a59700 1 --2- 172.21.15.12:0/3890358925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f396c0387a0 0x7f396c03ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3974006fd0 tx=0x7f3974006e40 comp rx=0 tx=0).stop 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.734+0000 7f3987a59700 1 -- 172.21.15.12:0/3890358925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 msgr2=0x7f398019c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.734+0000 7f3987a59700 1 --2- 172.21.15.12:0/3890358925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f398019c770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f397c011980 tx=0x7f397c011a60 comp rx=0 tx=0).stop 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.734+0000 7f3987a59700 1 -- 172.21.15.12:0/3890358925 shutdown_connections 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.734+0000 7f3987a59700 1 --2- 172.21.15.12:0/3890358925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f396c0387a0 0x7f396c03ac60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.734+0000 7f3987a59700 1 --2- 172.21.15.12:0/3890358925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3980107330 0x7f398019c770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.734+0000 7f3987a59700 1 -- 172.21.15.12:0/3890358925 >> 172.21.15.12:0/3890358925 conn(0x7f3980100c60 msgr2=0x7f39801017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.734+0000 7f3987a59700 1 -- 172.21.15.12:0/3890358925 shutdown_connections 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:58.734+0000 7f3987a59700 1 -- 172.21.15.12:0/3890358925 wait complete. 2024-05-08T23:39:59.109 INFO:teuthology.orchestra.run.smithi012.stdout:Fetching dashboard port number... 2024-05-08T23:39:59.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:59 smithi012 ceph-mon[42503]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:39:59.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:39:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:00.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T23:40:00.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.486+0000 7fc68d4b3700 1 Processor -- start 2024-05-08T23:40:00.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.486+0000 7fc68d4b3700 1 -- start start 2024-05-08T23:40:00.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.486+0000 7fc68d4b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc6881054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:00.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.486+0000 7fc68d4b3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc688079e40 con 0x7fc6881050a0 2024-05-08T23:40:00.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.486+0000 7fc686ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc6881054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:00.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.486+0000 7fc686ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc6881054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48448/0 (socket says 172.21.15.12:48448) 2024-05-08T23:40:00.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.486+0000 7fc686ffd700 1 -- 172.21.15.12:0/622929941 learned_addr learned my addr 172.21.15.12:0/622929941 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:00.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.487+0000 7fc686ffd700 1 -- 172.21.15.12:0/622929941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc688105a00 con 0x7fc6881050a0 2024-05-08T23:40:00.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.487+0000 7fc686ffd700 1 --2- 172.21.15.12:0/622929941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc6881054c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc6780098a0 tx=0x7fc678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228ebca107cca42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.487+0000 7fc685ffb700 1 -- 172.21.15.12:0/622929941 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc678004160 con 0x7fc6881050a0 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.487+0000 7fc685ffb700 1 -- 172.21.15.12:0/622929941 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6780042c0 con 0x7fc6881050a0 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.487+0000 7fc685ffb700 1 -- 172.21.15.12:0/622929941 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6780119b0 con 0x7fc6881050a0 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.488+0000 7fc68d4b3700 1 -- 172.21.15.12:0/622929941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 msgr2=0x7fc6881054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.488+0000 7fc68d4b3700 1 --2- 172.21.15.12:0/622929941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc6881054c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc6780098a0 tx=0x7fc678009bb0 comp rx=0 tx=0).stop 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.488+0000 7fc68d4b3700 1 -- 172.21.15.12:0/622929941 shutdown_connections 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.488+0000 7fc68d4b3700 1 --2- 172.21.15.12:0/622929941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc6881054c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.488+0000 7fc68d4b3700 1 -- 172.21.15.12:0/622929941 >> 172.21.15.12:0/622929941 conn(0x7fc688100ca0 msgr2=0x7fc6881030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.488+0000 7fc68d4b3700 1 -- 172.21.15.12:0/622929941 shutdown_connections 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.488+0000 7fc68d4b3700 1 -- 172.21.15.12:0/622929941 wait complete. 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.489+0000 7fc68d4b3700 1 Processor -- start 2024-05-08T23:40:00.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.489+0000 7fc68d4b3700 1 -- start start 2024-05-08T23:40:00.083 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.489+0000 7fc68d4b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc68819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:00.083 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.489+0000 7fc68d4b3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68819cc10 con 0x7fc6881050a0 2024-05-08T23:40:00.083 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.489+0000 7fc686ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc68819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:00.083 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.489+0000 7fc686ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc68819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48452/0 (socket says 172.21.15.12:48452) 2024-05-08T23:40:00.083 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.489+0000 7fc686ffd700 1 -- 172.21.15.12:0/2567882624 learned_addr learned my addr 172.21.15.12:0/2567882624 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:00.083 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.489+0000 7fc686ffd700 1 -- 172.21.15.12:0/2567882624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc678009550 con 0x7fc6881050a0 2024-05-08T23:40:00.083 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.490+0000 7fc686ffd700 1 --2- 172.21.15.12:0/2567882624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc68819c6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc678006af0 tx=0x7fc678011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:00.083 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.490+0000 7fc66ffff700 1 -- 172.21.15.12:0/2567882624 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc678003a40 con 0x7fc6881050a0 2024-05-08T23:40:00.083 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.490+0000 7fc66ffff700 1 -- 172.21.15.12:0/2567882624 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc678003ba0 con 0x7fc6881050a0 2024-05-08T23:40:00.083 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.490+0000 7fc68d4b3700 1 -- 172.21.15.12:0/2567882624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68819ce10 con 0x7fc6881050a0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.490+0000 7fc66ffff700 1 -- 172.21.15.12:0/2567882624 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6780196c0 con 0x7fc6881050a0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.490+0000 7fc68d4b3700 1 -- 172.21.15.12:0/2567882624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68819d2b0 con 0x7fc6881050a0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.491+0000 7fc66ffff700 1 -- 172.21.15.12:0/2567882624 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fc67801a430 con 0x7fc6881050a0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.491+0000 7fc68d4b3700 1 -- 172.21.15.12:0/2567882624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc688196420 con 0x7fc6881050a0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.491+0000 7fc66ffff700 1 --2- 172.21.15.12:0/2567882624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc670038700 0x7fc67003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.491+0000 7fc66ffff700 1 -- 172.21.15.12:0/2567882624 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc67804cb40 con 0x7fc6881050a0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.491+0000 7fc6867fc700 1 --2- 172.21.15.12:0/2567882624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc670038700 0x7fc67003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.492+0000 7fc6867fc700 1 --2- 172.21.15.12:0/2567882624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc670038700 0x7fc67003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc67c006fd0 tx=0x7fc67c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.497+0000 7fc66ffff700 1 -- 172.21.15.12:0/2567882624 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc67801aa80 con 0x7fc6881050a0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.707+0000 7fc68d4b3700 1 -- 172.21.15.12:0/2567882624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fc688106a80 con 0x7fc6881050a0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.708+0000 7fc66ffff700 1 -- 172.21.15.12:0/2567882624 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fc678023390 con 0x7fc6881050a0 2024-05-08T23:40:00.084 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.710+0000 7fc68d4b3700 1 -- 172.21.15.12:0/2567882624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc670038700 msgr2=0x7fc67003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:00.085 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.710+0000 7fc68d4b3700 1 --2- 172.21.15.12:0/2567882624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc670038700 0x7fc67003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc67c006fd0 tx=0x7fc67c006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:00.085 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.710+0000 7fc68d4b3700 1 -- 172.21.15.12:0/2567882624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 msgr2=0x7fc68819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:00.085 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.710+0000 7fc68d4b3700 1 --2- 172.21.15.12:0/2567882624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc68819c6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc678006af0 tx=0x7fc678011a10 comp rx=0 tx=0).stop 2024-05-08T23:40:00.085 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.710+0000 7fc68d4b3700 1 -- 172.21.15.12:0/2567882624 shutdown_connections 2024-05-08T23:40:00.085 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.710+0000 7fc68d4b3700 1 --2- 172.21.15.12:0/2567882624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc670038700 0x7fc67003abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:00.085 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.710+0000 7fc68d4b3700 1 --2- 172.21.15.12:0/2567882624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6881050a0 0x7fc68819c6d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:00.085 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.710+0000 7fc68d4b3700 1 -- 172.21.15.12:0/2567882624 >> 172.21.15.12:0/2567882624 conn(0x7fc688100ca0 msgr2=0x7fc688193560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:00.085 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.710+0000 7fc68d4b3700 1 -- 172.21.15.12:0/2567882624 shutdown_connections 2024-05-08T23:40:00.085 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:39:59.710+0000 7fc68d4b3700 1 -- 172.21.15.12:0/2567882624 wait complete. 2024-05-08T23:40:00.091 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T23:40:00.091 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-08T23:40:00.102 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T23:40:00.102 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-08T23:40:00.103 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-08T23:40:00.103 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T23:40:00.103 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph Dashboard is now available at: 2024-05-08T23:40:00.103 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:00.103 INFO:teuthology.orchestra.run.smithi012.stdout: URL: https://smithi012:8443/ 2024-05-08T23:40:00.103 INFO:teuthology.orchestra.run.smithi012.stdout: User: admin 2024-05-08T23:40:00.103 INFO:teuthology.orchestra.run.smithi012.stdout: Password: 445zt9ijdx 2024-05-08T23:40:00.103 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:00.103 INFO:teuthology.orchestra.run.smithi012.stdout:Saving cluster configuration to /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config directory 2024-05-08T23:40:00.103 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling autotune for osd_memory_target 2024-05-08T23:40:00.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:00 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2567882624' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T23:40:01.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.491+0000 7f0f6bbf3700 1 Processor -- start 2024-05-08T23:40:01.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.491+0000 7f0f6bbf3700 1 -- start start 2024-05-08T23:40:01.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.491+0000 7f0f6bbf3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:01.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.491+0000 7f0f6bbf3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f64079ea0 con 0x7f0f641071f0 2024-05-08T23:40:01.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.492+0000 7f0f6998f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:01.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.492+0000 7f0f6998f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48468/0 (socket says 172.21.15.12:48468) 2024-05-08T23:40:01.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.492+0000 7f0f6998f700 1 -- 172.21.15.12:0/8353774 learned_addr learned my addr 172.21.15.12:0/8353774 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:01.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.492+0000 7f0f6998f700 1 -- 172.21.15.12:0/8353774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f64109b20 con 0x7f0f641071f0 2024-05-08T23:40:01.087 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.492+0000 7f0f6998f700 1 --2- 172.21.15.12:0/8353774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641095e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0f600098a0 tx=0x7f0f60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c0a736f7454503 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.492+0000 7f0f6898d700 1 -- 172.21.15.12:0/8353774 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f60004160 con 0x7f0f641071f0 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.492+0000 7f0f6898d700 1 -- 172.21.15.12:0/8353774 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f600042c0 con 0x7f0f641071f0 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.492+0000 7f0f6898d700 1 -- 172.21.15.12:0/8353774 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f600119b0 con 0x7f0f641071f0 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.493+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/8353774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 msgr2=0x7f0f641095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.493+0000 7f0f6bbf3700 1 --2- 172.21.15.12:0/8353774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641095e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0f600098a0 tx=0x7f0f60009bb0 comp rx=0 tx=0).stop 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.493+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/8353774 shutdown_connections 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.493+0000 7f0f6bbf3700 1 --2- 172.21.15.12:0/8353774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641095e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.493+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/8353774 >> 172.21.15.12:0/8353774 conn(0x7f0f64100cc0 msgr2=0x7f0f641030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.493+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/8353774 shutdown_connections 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.493+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/8353774 wait complete. 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.494+0000 7f0f6bbf3700 1 Processor -- start 2024-05-08T23:40:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.494+0000 7f0f6bbf3700 1 -- start start 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.494+0000 7f0f6bbf3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641069d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.494+0000 7f0f6bbf3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f64106f10 con 0x7f0f641071f0 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.494+0000 7f0f6998f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.494+0000 7f0f6998f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48482/0 (socket says 172.21.15.12:48482) 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.494+0000 7f0f6998f700 1 -- 172.21.15.12:0/3515869427 learned_addr learned my addr 172.21.15.12:0/3515869427 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.495+0000 7f0f6998f700 1 -- 172.21.15.12:0/3515869427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f60009550 con 0x7f0f641071f0 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.495+0000 7f0f6998f700 1 --2- 172.21.15.12:0/3515869427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641069d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0f60011670 tx=0x7f0f60011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.495+0000 7f0f5affd700 1 -- 172.21.15.12:0/3515869427 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f60003b20 con 0x7f0f641071f0 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.495+0000 7f0f5affd700 1 -- 172.21.15.12:0/3515869427 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f60003c80 con 0x7f0f641071f0 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.495+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/3515869427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f641051a0 con 0x7f0f641071f0 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.495+0000 7f0f5affd700 1 -- 172.21.15.12:0/3515869427 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f60019770 con 0x7f0f641071f0 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.495+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/3515869427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f64105640 con 0x7f0f641071f0 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.496+0000 7f0f5affd700 1 -- 172.21.15.12:0/3515869427 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f0f6001a430 con 0x7f0f641071f0 2024-05-08T23:40:01.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.496+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/3515869427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f640625d0 con 0x7f0f641071f0 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.496+0000 7f0f5affd700 1 --2- 172.21.15.12:0/3515869427 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f50038750 0x7f0f5003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.496+0000 7f0f5affd700 1 -- 172.21.15.12:0/3515869427 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0f6004cb80 con 0x7f0f641071f0 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.496+0000 7f0f6918e700 1 --2- 172.21.15.12:0/3515869427 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f50038750 0x7f0f5003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.497+0000 7f0f6918e700 1 --2- 172.21.15.12:0/3515869427 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f50038750 0x7f0f5003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f54006fd0 tx=0x7f0f54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.502+0000 7f0f5affd700 1 -- 172.21.15.12:0/3515869427 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0f6002d9a0 con 0x7f0f641071f0 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.711+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/3515869427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f0f64105880 con 0x7f0f641071f0 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.711+0000 7f0f5affd700 1 -- 172.21.15.12:0/3515869427 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0f6004ce60 con 0x7f0f641071f0 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.713+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/3515869427 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f50038750 msgr2=0x7f0f5003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.713+0000 7f0f6bbf3700 1 --2- 172.21.15.12:0/3515869427 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f50038750 0x7f0f5003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f54006fd0 tx=0x7f0f54006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.713+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/3515869427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 msgr2=0x7f0f641069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.713+0000 7f0f6bbf3700 1 --2- 172.21.15.12:0/3515869427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641069d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0f60011670 tx=0x7f0f60011a30 comp rx=0 tx=0).stop 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.713+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/3515869427 shutdown_connections 2024-05-08T23:40:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.713+0000 7f0f6bbf3700 1 --2- 172.21.15.12:0/3515869427 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f50038750 0x7f0f5003ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:01.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.713+0000 7f0f6bbf3700 1 --2- 172.21.15.12:0/3515869427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f641071f0 0x7f0f641069d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:01.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.713+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/3515869427 >> 172.21.15.12:0/3515869427 conn(0x7f0f64100cc0 msgr2=0x7f0f64101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:01.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.714+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/3515869427 shutdown_connections 2024-05-08T23:40:01.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:00.714+0000 7f0f6bbf3700 1 -- 172.21.15.12:0/3515869427 wait complete. 2024-05-08T23:40:02.195 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.476+0000 7f475b4dd700 1 Processor -- start 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.476+0000 7f475b4dd700 1 -- start start 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.477+0000 7f475b4dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f47541096b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.477+0000 7f475b4dd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4754074880 con 0x7f47541072c0 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.477+0000 7f4759279700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f47541096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.477+0000 7f4759279700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f47541096b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48498/0 (socket says 172.21.15.12:48498) 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.477+0000 7f4759279700 1 -- 172.21.15.12:0/2008732347 learned_addr learned my addr 172.21.15.12:0/2008732347 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.477+0000 7f4759279700 1 -- 172.21.15.12:0/2008732347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4754109bf0 con 0x7f47541072c0 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.477+0000 7f4759279700 1 --2- 172.21.15.12:0/2008732347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f47541096b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47480098a0 tx=0x7f4748009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d47a26fded50688 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.477+0000 7f4747fff700 1 -- 172.21.15.12:0/2008732347 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4748004160 con 0x7f47541072c0 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.478+0000 7f4747fff700 1 -- 172.21.15.12:0/2008732347 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47480042c0 con 0x7f47541072c0 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.478+0000 7f4747fff700 1 -- 172.21.15.12:0/2008732347 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47480119b0 con 0x7f47541072c0 2024-05-08T23:40:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.478+0000 7f475b4dd700 1 -- 172.21.15.12:0/2008732347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 msgr2=0x7f47541096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.478+0000 7f475b4dd700 1 --2- 172.21.15.12:0/2008732347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f47541096b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47480098a0 tx=0x7f4748009bb0 comp rx=0 tx=0).stop 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.478+0000 7f475b4dd700 1 -- 172.21.15.12:0/2008732347 shutdown_connections 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.478+0000 7f475b4dd700 1 --2- 172.21.15.12:0/2008732347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f47541096b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.478+0000 7f475b4dd700 1 -- 172.21.15.12:0/2008732347 >> 172.21.15.12:0/2008732347 conn(0x7f4754100c90 msgr2=0x7f47541030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.478+0000 7f475b4dd700 1 -- 172.21.15.12:0/2008732347 shutdown_connections 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.478+0000 7f475b4dd700 1 -- 172.21.15.12:0/2008732347 wait complete. 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.479+0000 7f475b4dd700 1 Processor -- start 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.479+0000 7f475b4dd700 1 -- start start 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.479+0000 7f475b4dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f4754106af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.479+0000 7f475b4dd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4754107030 con 0x7f47541072c0 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.479+0000 7f4759279700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f4754106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.479+0000 7f4759279700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f4754106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48502/0 (socket says 172.21.15.12:48502) 2024-05-08T23:40:02.197 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.479+0000 7f4759279700 1 -- 172.21.15.12:0/3183176845 learned_addr learned my addr 172.21.15.12:0/3183176845 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.480+0000 7f4759279700 1 -- 172.21.15.12:0/3183176845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4748009550 con 0x7f47541072c0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.480+0000 7f4759279700 1 --2- 172.21.15.12:0/3183176845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f4754106af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4748011980 tx=0x7f4748011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.480+0000 7f47467fc700 1 -- 172.21.15.12:0/3183176845 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4748003af0 con 0x7f47541072c0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.480+0000 7f47467fc700 1 -- 172.21.15.12:0/3183176845 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4748003c50 con 0x7f47541072c0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.480+0000 7f475b4dd700 1 -- 172.21.15.12:0/3183176845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4754105270 con 0x7f47541072c0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.480+0000 7f47467fc700 1 -- 172.21.15.12:0/3183176845 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4748019760 con 0x7f47541072c0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.480+0000 7f475b4dd700 1 -- 172.21.15.12:0/3183176845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4754105710 con 0x7f47541072c0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.481+0000 7f47467fc700 1 -- 172.21.15.12:0/3183176845 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f474801a430 con 0x7f47541072c0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.481+0000 7f475b4dd700 1 -- 172.21.15.12:0/3183176845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47540625d0 con 0x7f47541072c0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.481+0000 7f47467fc700 1 --2- 172.21.15.12:0/3183176845 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47400387f0 0x7f474003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.481+0000 7f47467fc700 1 -- 172.21.15.12:0/3183176845 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f474804cb30 con 0x7f47541072c0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.482+0000 7f4758a78700 1 --2- 172.21.15.12:0/3183176845 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47400387f0 0x7f474003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:02.198 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.482+0000 7f4758a78700 1 --2- 172.21.15.12:0/3183176845 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47400387f0 0x7f474003acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4750006fd0 tx=0x7f4750006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.487+0000 7f47467fc700 1 -- 172.21.15.12:0/3183176845 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f474802d9a0 con 0x7f47541072c0 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.817+0000 7f475b4dd700 1 -- 172.21.15.12:0/3183176845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f475410a9d0 con 0x7f47541072c0 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.820+0000 7f47467fc700 1 -- 172.21.15.12:0/3183176845 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f474802d350 con 0x7f47541072c0 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.822+0000 7f475b4dd700 1 -- 172.21.15.12:0/3183176845 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47400387f0 msgr2=0x7f474003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.822+0000 7f475b4dd700 1 --2- 172.21.15.12:0/3183176845 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47400387f0 0x7f474003acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4750006fd0 tx=0x7f4750006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.822+0000 7f475b4dd700 1 -- 172.21.15.12:0/3183176845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 msgr2=0x7f4754106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.822+0000 7f475b4dd700 1 --2- 172.21.15.12:0/3183176845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f4754106af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4748011980 tx=0x7f4748011a60 comp rx=0 tx=0).stop 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.822+0000 7f475b4dd700 1 -- 172.21.15.12:0/3183176845 shutdown_connections 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.822+0000 7f475b4dd700 1 --2- 172.21.15.12:0/3183176845 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47400387f0 0x7f474003acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.822+0000 7f475b4dd700 1 --2- 172.21.15.12:0/3183176845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47541072c0 0x7f4754106af0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.822+0000 7f475b4dd700 1 -- 172.21.15.12:0/3183176845 >> 172.21.15.12:0/3183176845 conn(0x7f4754100c90 msgr2=0x7f4754101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.822+0000 7f475b4dd700 1 -- 172.21.15.12:0/3183176845 shutdown_connections 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-08T23:40:01.822+0000 7f475b4dd700 1 -- 172.21.15.12:0/3183176845 wait complete. 2024-05-08T23:40:02.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: ceph telemetry on 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout:For more information see: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:02.200 INFO:teuthology.orchestra.run.smithi012.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T23:40:02.201 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:02.201 INFO:teuthology.orchestra.run.smithi012.stdout:Bootstrap complete. 2024-05-08T23:40:02.240 INFO:tasks.cephadm:Fetching config... 2024-05-08T23:40:02.241 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:40:02.241 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T23:40:02.261 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T23:40:02.261 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:40:02.261 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T23:40:02.323 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T23:40:02.323 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:40:02.323 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T23:40:02.392 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:02 smithi012 ceph-mon[42503]: mgrmap e13: x(active, since 6s) 2024-05-08T23:40:02.392 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:02 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3183176845' entity='client.admin' 2024-05-08T23:40:02.402 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T23:40:02.402 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:40:02.402 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T23:40:02.462 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T23:40:02.463 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZFLLVFDooOlZg9S73lL6B0lKuCfF7O7KoAhZsFHPAXiFlBpuoYBdKqz9w/Rdc4/4kq8XYN1PH61pVxh32sVRr+YduHGmTTZ7D8YWBlWGrWpMf4tGrudx+Fyrgc9ni4yyCyCK+CUBjV0mx8GyMky7muDKTQveuCXyzwFw+HMgxkvo4xfUzxXllz2nuO6fWXGh7s6m7Q9q0T6/he1huVkkcpCT/B30hTLVYsI1M7C29G0wLdbp7uxTOd+cirBNAoMfZ6HweZyKXxQfBOxonfna9LalSjsWPtfTA5SNeLwB9Uu9Wn48mgacbvoSCg0WhINT4/FMx92OuIpglisNnUeiqSCG9c4xoqTVFAF8UMEdfeKNcY7i1eg2YCR1v8Gd47xurTf7FpBf0BTS5Azc37gFKhPmKdZH75e1sGw1bKHL8jJChgdreqk1kzgbldbIc2bmARYzeSkKBnct3lBPBAcoHmQmGC4ZBLlO7prtWWGuaNIuKQv30qKFcWd0mQBb+Z9E= ceph-00f8fb28-0d94-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T23:40:02.548 INFO:teuthology.orchestra.run.smithi012.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZFLLVFDooOlZg9S73lL6B0lKuCfF7O7KoAhZsFHPAXiFlBpuoYBdKqz9w/Rdc4/4kq8XYN1PH61pVxh32sVRr+YduHGmTTZ7D8YWBlWGrWpMf4tGrudx+Fyrgc9ni4yyCyCK+CUBjV0mx8GyMky7muDKTQveuCXyzwFw+HMgxkvo4xfUzxXllz2nuO6fWXGh7s6m7Q9q0T6/he1huVkkcpCT/B30hTLVYsI1M7C29G0wLdbp7uxTOd+cirBNAoMfZ6HweZyKXxQfBOxonfna9LalSjsWPtfTA5SNeLwB9Uu9Wn48mgacbvoSCg0WhINT4/FMx92OuIpglisNnUeiqSCG9c4xoqTVFAF8UMEdfeKNcY7i1eg2YCR1v8Gd47xurTf7FpBf0BTS5Azc37gFKhPmKdZH75e1sGw1bKHL8jJChgdreqk1kzgbldbIc2bmARYzeSkKBnct3lBPBAcoHmQmGC4ZBLlO7prtWWGuaNIuKQv30qKFcWd0mQBb+Z9E= ceph-00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:40:02.562 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZFLLVFDooOlZg9S73lL6B0lKuCfF7O7KoAhZsFHPAXiFlBpuoYBdKqz9w/Rdc4/4kq8XYN1PH61pVxh32sVRr+YduHGmTTZ7D8YWBlWGrWpMf4tGrudx+Fyrgc9ni4yyCyCK+CUBjV0mx8GyMky7muDKTQveuCXyzwFw+HMgxkvo4xfUzxXllz2nuO6fWXGh7s6m7Q9q0T6/he1huVkkcpCT/B30hTLVYsI1M7C29G0wLdbp7uxTOd+cirBNAoMfZ6HweZyKXxQfBOxonfna9LalSjsWPtfTA5SNeLwB9Uu9Wn48mgacbvoSCg0WhINT4/FMx92OuIpglisNnUeiqSCG9c4xoqTVFAF8UMEdfeKNcY7i1eg2YCR1v8Gd47xurTf7FpBf0BTS5Azc37gFKhPmKdZH75e1sGw1bKHL8jJChgdreqk1kzgbldbIc2bmARYzeSkKBnct3lBPBAcoHmQmGC4ZBLlO7prtWWGuaNIuKQv30qKFcWd0mQBb+Z9E= ceph-00f8fb28-0d94-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T23:40:02.610 INFO:teuthology.orchestra.run.smithi160.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZFLLVFDooOlZg9S73lL6B0lKuCfF7O7KoAhZsFHPAXiFlBpuoYBdKqz9w/Rdc4/4kq8XYN1PH61pVxh32sVRr+YduHGmTTZ7D8YWBlWGrWpMf4tGrudx+Fyrgc9ni4yyCyCK+CUBjV0mx8GyMky7muDKTQveuCXyzwFw+HMgxkvo4xfUzxXllz2nuO6fWXGh7s6m7Q9q0T6/he1huVkkcpCT/B30hTLVYsI1M7C29G0wLdbp7uxTOd+cirBNAoMfZ6HweZyKXxQfBOxonfna9LalSjsWPtfTA5SNeLwB9Uu9Wn48mgacbvoSCg0WhINT4/FMx92OuIpglisNnUeiqSCG9c4xoqTVFAF8UMEdfeKNcY7i1eg2YCR1v8Gd47xurTf7FpBf0BTS5Azc37gFKhPmKdZH75e1sGw1bKHL8jJChgdreqk1kzgbldbIc2bmARYzeSkKBnct3lBPBAcoHmQmGC4ZBLlO7prtWWGuaNIuKQv30qKFcWd0mQBb+Z9E= ceph-00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:40:02.623 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZFLLVFDooOlZg9S73lL6B0lKuCfF7O7KoAhZsFHPAXiFlBpuoYBdKqz9w/Rdc4/4kq8XYN1PH61pVxh32sVRr+YduHGmTTZ7D8YWBlWGrWpMf4tGrudx+Fyrgc9ni4yyCyCK+CUBjV0mx8GyMky7muDKTQveuCXyzwFw+HMgxkvo4xfUzxXllz2nuO6fWXGh7s6m7Q9q0T6/he1huVkkcpCT/B30hTLVYsI1M7C29G0wLdbp7uxTOd+cirBNAoMfZ6HweZyKXxQfBOxonfna9LalSjsWPtfTA5SNeLwB9Uu9Wn48mgacbvoSCg0WhINT4/FMx92OuIpglisNnUeiqSCG9c4xoqTVFAF8UMEdfeKNcY7i1eg2YCR1v8Gd47xurTf7FpBf0BTS5Azc37gFKhPmKdZH75e1sGw1bKHL8jJChgdreqk1kzgbldbIc2bmARYzeSkKBnct3lBPBAcoHmQmGC4ZBLlO7prtWWGuaNIuKQv30qKFcWd0mQBb+Z9E= ceph-00f8fb28-0d94-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T23:40:02.672 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZFLLVFDooOlZg9S73lL6B0lKuCfF7O7KoAhZsFHPAXiFlBpuoYBdKqz9w/Rdc4/4kq8XYN1PH61pVxh32sVRr+YduHGmTTZ7D8YWBlWGrWpMf4tGrudx+Fyrgc9ni4yyCyCK+CUBjV0mx8GyMky7muDKTQveuCXyzwFw+HMgxkvo4xfUzxXllz2nuO6fWXGh7s6m7Q9q0T6/he1huVkkcpCT/B30hTLVYsI1M7C29G0wLdbp7uxTOd+cirBNAoMfZ6HweZyKXxQfBOxonfna9LalSjsWPtfTA5SNeLwB9Uu9Wn48mgacbvoSCg0WhINT4/FMx92OuIpglisNnUeiqSCG9c4xoqTVFAF8UMEdfeKNcY7i1eg2YCR1v8Gd47xurTf7FpBf0BTS5Azc37gFKhPmKdZH75e1sGw1bKHL8jJChgdreqk1kzgbldbIc2bmARYzeSkKBnct3lBPBAcoHmQmGC4ZBLlO7prtWWGuaNIuKQv30qKFcWd0mQBb+Z9E= ceph-00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:40:02.685 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T23:40:02.931 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:40:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.169+0000 7feae657e700 1 -- 172.21.15.12:0/3985024969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae00ff820 msgr2=0x7feae00ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.169+0000 7feae657e700 1 --2- 172.21.15.12:0/3985024969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae00ff820 0x7feae00ffc40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fead0009910 tx=0x7fead0009c20 comp rx=0 tx=0).stop 2024-05-08T23:40:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.169+0000 7feae657e700 1 -- 172.21.15.12:0/3985024969 shutdown_connections 2024-05-08T23:40:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.169+0000 7feae657e700 1 --2- 172.21.15.12:0/3985024969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae00ff820 0x7feae00ffc40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.169+0000 7feae657e700 1 -- 172.21.15.12:0/3985024969 >> 172.21.15.12:0/3985024969 conn(0x7feae00fadc0 msgr2=0x7feae00fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.169+0000 7feae657e700 1 -- 172.21.15.12:0/3985024969 shutdown_connections 2024-05-08T23:40:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.169+0000 7feae657e700 1 -- 172.21.15.12:0/3985024969 wait complete. 2024-05-08T23:40:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.169+0000 7feae657e700 1 Processor -- start 2024-05-08T23:40:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.169+0000 7feae657e700 1 -- start start 2024-05-08T23:40:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.170+0000 7feae657e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae00ff820 0x7feae00fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.170+0000 7feae657e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae00fd170 con 0x7feae00ff820 2024-05-08T23:40:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.170+0000 7feadffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae00ff820 0x7feae00fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.170+0000 7feadffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae00ff820 0x7feae00fcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48524/0 (socket says 172.21.15.12:48524) 2024-05-08T23:40:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.170+0000 7feadffff700 1 -- 172.21.15.12:0/692386240 learned_addr learned my addr 172.21.15.12:0/692386240 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.170+0000 7feadffff700 1 -- 172.21.15.12:0/692386240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead00095f0 con 0x7feae00ff820 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.170+0000 7feadffff700 1 --2- 172.21.15.12:0/692386240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae00ff820 0x7feae00fcc30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fead0004f70 tx=0x7fead0004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.171+0000 7feadd7fa700 1 -- 172.21.15.12:0/692386240 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fead001d070 con 0x7feae00ff820 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.171+0000 7feae657e700 1 -- 172.21.15.12:0/692386240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae00fb3b0 con 0x7feae00ff820 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.171+0000 7feadd7fa700 1 -- 172.21.15.12:0/692386240 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fead000be20 con 0x7feae00ff820 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.171+0000 7feadd7fa700 1 -- 172.21.15.12:0/692386240 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fead0022df0 con 0x7feae00ff820 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.171+0000 7feae657e700 1 -- 172.21.15.12:0/692386240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae00fb850 con 0x7feae00ff820 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.172+0000 7feadd7fa700 1 -- 172.21.15.12:0/692386240 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fead000f4b0 con 0x7feae00ff820 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.172+0000 7feae657e700 1 -- 172.21.15.12:0/692386240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae0105e70 con 0x7feae00ff820 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.172+0000 7feadd7fa700 1 --2- 172.21.15.12:0/692386240 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feacc038780 0x7feacc03ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.172+0000 7feadd7fa700 1 -- 172.21.15.12:0/692386240 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fead004c8d0 con 0x7feae00ff820 2024-05-08T23:40:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.172+0000 7feadf7fe700 1 --2- 172.21.15.12:0/692386240 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feacc038780 0x7feacc03ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:04.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.173+0000 7feadf7fe700 1 --2- 172.21.15.12:0/692386240 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feacc038780 0x7feacc03ac40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feac8006fd0 tx=0x7feac8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:04.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.178+0000 7feadd7fa700 1 -- 172.21.15.12:0/692386240 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fead0020960 con 0x7feae00ff820 2024-05-08T23:40:04.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.395+0000 7feae657e700 1 -- 172.21.15.12:0/692386240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7feae00625d0 con 0x7feae00ff820 2024-05-08T23:40:04.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.399+0000 7feadd7fa700 1 -- 172.21.15.12:0/692386240 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fead0027030 con 0x7feae00ff820 2024-05-08T23:40:04.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.401+0000 7feae657e700 1 -- 172.21.15.12:0/692386240 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feacc038780 msgr2=0x7feacc03ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:04.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.401+0000 7feae657e700 1 --2- 172.21.15.12:0/692386240 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feacc038780 0x7feacc03ac40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feac8006fd0 tx=0x7feac8006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:04.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.401+0000 7feae657e700 1 -- 172.21.15.12:0/692386240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae00ff820 msgr2=0x7feae00fcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:04.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.401+0000 7feae657e700 1 --2- 172.21.15.12:0/692386240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae00ff820 0x7feae00fcc30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fead0004f70 tx=0x7fead0004fa0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.402+0000 7feae657e700 1 -- 172.21.15.12:0/692386240 shutdown_connections 2024-05-08T23:40:04.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.402+0000 7feae657e700 1 --2- 172.21.15.12:0/692386240 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feacc038780 0x7feacc03ac40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.402+0000 7feae657e700 1 --2- 172.21.15.12:0/692386240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae00ff820 0x7feae00fcc30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.402+0000 7feae657e700 1 -- 172.21.15.12:0/692386240 >> 172.21.15.12:0/692386240 conn(0x7feae00fadc0 msgr2=0x7feae0069540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:04.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.402+0000 7feae657e700 1 -- 172.21.15.12:0/692386240 shutdown_connections 2024-05-08T23:40:04.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:04.402+0000 7feae657e700 1 -- 172.21.15.12:0/692386240 wait complete. 2024-05-08T23:40:04.787 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T23:40:04.787 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T23:40:05.060 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:40:05.629 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:05 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/692386240' entity='client.admin' 2024-05-08T23:40:06.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.337+0000 7fa80362d700 1 -- 172.21.15.12:0/526910095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7fc0ff830 msgr2=0x7fa7fc0ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:06.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.337+0000 7fa80362d700 1 --2- 172.21.15.12:0/526910095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7fc0ff830 0x7fa7fc0ffc50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009910 tx=0x7fa7f8009c20 comp rx=0 tx=0).stop 2024-05-08T23:40:06.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.337+0000 7fa80362d700 1 -- 172.21.15.12:0/526910095 shutdown_connections 2024-05-08T23:40:06.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.337+0000 7fa80362d700 1 --2- 172.21.15.12:0/526910095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7fc0ff830 0x7fa7fc0ffc50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.337+0000 7fa80362d700 1 -- 172.21.15.12:0/526910095 >> 172.21.15.12:0/526910095 conn(0x7fa7fc0fadd0 msgr2=0x7fa7fc0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:06.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.337+0000 7fa80362d700 1 -- 172.21.15.12:0/526910095 shutdown_connections 2024-05-08T23:40:06.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.337+0000 7fa80362d700 1 -- 172.21.15.12:0/526910095 wait complete. 2024-05-08T23:40:06.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.337+0000 7fa80362d700 1 Processor -- start 2024-05-08T23:40:06.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.337+0000 7fa80362d700 1 -- start start 2024-05-08T23:40:06.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.338+0000 7fa80362d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7fc0ff830 0x7fa7fc195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.338+0000 7fa80362d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc1962f0 con 0x7fa7fc0ff830 2024-05-08T23:40:06.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.338+0000 7fa8013c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7fc0ff830 0x7fa7fc195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:06.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.338+0000 7fa8013c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7fc0ff830 0x7fa7fc195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48546/0 (socket says 172.21.15.12:48546) 2024-05-08T23:40:06.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.338+0000 7fa8013c9700 1 -- 172.21.15.12:0/3412523156 learned_addr learned my addr 172.21.15.12:0/3412523156 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:06.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.338+0000 7fa8013c9700 1 -- 172.21.15.12:0/3412523156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f80095f0 con 0x7fa7fc0ff830 2024-05-08T23:40:06.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.338+0000 7fa8013c9700 1 --2- 172.21.15.12:0/3412523156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7fc0ff830 0x7fa7fc195db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8005950 tx=0x7fa7f80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:06.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.338+0000 7fa7f27fc700 1 -- 172.21.15.12:0/3412523156 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7f801d070 con 0x7fa7fc0ff830 2024-05-08T23:40:06.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.338+0000 7fa7f27fc700 1 -- 172.21.15.12:0/3412523156 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7f800bc10 con 0x7fa7fc0ff830 2024-05-08T23:40:06.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.339+0000 7fa80362d700 1 -- 172.21.15.12:0/3412523156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc1964f0 con 0x7fa7fc0ff830 2024-05-08T23:40:06.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.339+0000 7fa7f27fc700 1 -- 172.21.15.12:0/3412523156 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7f8003a10 con 0x7fa7fc0ff830 2024-05-08T23:40:06.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.339+0000 7fa80362d700 1 -- 172.21.15.12:0/3412523156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc196990 con 0x7fa7fc0ff830 2024-05-08T23:40:06.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.339+0000 7fa7f27fc700 1 -- 172.21.15.12:0/3412523156 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fa7f8003c80 con 0x7fa7fc0ff830 2024-05-08T23:40:06.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.340+0000 7fa7f27fc700 1 --2- 172.21.15.12:0/3412523156 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7e80387f0 0x7fa7e803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.340+0000 7fa7f27fc700 1 -- 172.21.15.12:0/3412523156 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa7f804c520 con 0x7fa7fc0ff830 2024-05-08T23:40:06.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.340+0000 7fa80362d700 1 -- 172.21.15.12:0/3412523156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7fc18f560 con 0x7fa7fc0ff830 2024-05-08T23:40:06.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.340+0000 7fa800bc8700 1 --2- 172.21.15.12:0/3412523156 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7e80387f0 0x7fa7e803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:06.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.340+0000 7fa800bc8700 1 --2- 172.21.15.12:0/3412523156 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7e80387f0 0x7fa7e803acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec006fd0 tx=0x7fa7ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:06.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.346+0000 7fa7f27fc700 1 -- 172.21.15.12:0/3412523156 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa7f8026020 con 0x7fa7fc0ff830 2024-05-08T23:40:06.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.558+0000 7fa80362d700 1 -- 172.21.15.12:0/3412523156 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7fc0613e0 con 0x7fa7e80387f0 2024-05-08T23:40:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.562+0000 7fa7f27fc700 1 -- 172.21.15.12:0/3412523156 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa7fc0613e0 con 0x7fa7e80387f0 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.564+0000 7fa80362d700 1 -- 172.21.15.12:0/3412523156 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7e80387f0 msgr2=0x7fa7e803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.564+0000 7fa80362d700 1 --2- 172.21.15.12:0/3412523156 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7e80387f0 0x7fa7e803acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec006fd0 tx=0x7fa7ec006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.564+0000 7fa80362d700 1 -- 172.21.15.12:0/3412523156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7fc0ff830 msgr2=0x7fa7fc195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.564+0000 7fa80362d700 1 --2- 172.21.15.12:0/3412523156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7fc0ff830 0x7fa7fc195db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8005950 tx=0x7fa7f80057c0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.564+0000 7fa80362d700 1 -- 172.21.15.12:0/3412523156 shutdown_connections 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.564+0000 7fa80362d700 1 --2- 172.21.15.12:0/3412523156 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7e80387f0 0x7fa7e803acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.564+0000 7fa80362d700 1 --2- 172.21.15.12:0/3412523156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7fc0ff830 0x7fa7fc195db0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.564+0000 7fa80362d700 1 -- 172.21.15.12:0/3412523156 >> 172.21.15.12:0/3412523156 conn(0x7fa7fc0fadd0 msgr2=0x7fa7fc0691a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:06.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.565+0000 7fa80362d700 1 -- 172.21.15.12:0/3412523156 shutdown_connections 2024-05-08T23:40:06.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:06.565+0000 7fa80362d700 1 -- 172.21.15.12:0/3412523156 wait complete. 2024-05-08T23:40:06.942 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi160 2024-05-08T23:40:06.942 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:40:06.942 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.conf 2024-05-08T23:40:06.963 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:40:06.963 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:07.022 INFO:tasks.cephadm:Adding host smithi160 to orchestrator... 2024-05-08T23:40:07.023 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch host add smithi160 2024-05-08T23:40:07.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:07.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:07.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi012", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:40:07.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:07.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:07.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:07.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:07.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:07.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:07.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:07.301 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:40:08.107 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:08 smithi012 ceph-mon[42503]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:08.107 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:08 smithi012 ceph-mon[42503]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-08T23:40:08.107 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:08 smithi012 ceph-mon[42503]: Updating smithi012:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:40:08.107 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:08 smithi012 ceph-mon[42503]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:08.108 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:08 smithi012 ceph-mon[42503]: Updating smithi012:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:40:08.108 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:08.108 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:08.108 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.299+0000 7f7f8bd25700 1 -- 172.21.15.12:0/2266582216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84106900 msgr2=0x7f7f84106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.299+0000 7f7f8bd25700 1 --2- 172.21.15.12:0/2266582216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84106900 0x7f7f84106ce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009910 tx=0x7f7f78009c20 comp rx=0 tx=0).stop 2024-05-08T23:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.299+0000 7f7f8bd25700 1 -- 172.21.15.12:0/2266582216 shutdown_connections 2024-05-08T23:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.299+0000 7f7f8bd25700 1 --2- 172.21.15.12:0/2266582216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84106900 0x7f7f84106ce0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.299+0000 7f7f8bd25700 1 -- 172.21.15.12:0/2266582216 >> 172.21.15.12:0/2266582216 conn(0x7f7f840fc410 msgr2=0x7f7f840fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.299+0000 7f7f8bd25700 1 -- 172.21.15.12:0/2266582216 shutdown_connections 2024-05-08T23:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.299+0000 7f7f8bd25700 1 -- 172.21.15.12:0/2266582216 wait complete. 2024-05-08T23:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.300+0000 7f7f8bd25700 1 Processor -- start 2024-05-08T23:40:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.300+0000 7f7f8bd25700 1 -- start start 2024-05-08T23:40:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.300+0000 7f7f8bd25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84106900 0x7f7f840732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.300+0000 7f7f8bd25700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f84075160 con 0x7f7f84106900 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.300+0000 7f7f89ac1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84106900 0x7f7f840732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.300+0000 7f7f89ac1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84106900 0x7f7f840732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48562/0 (socket says 172.21.15.12:48562) 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.300+0000 7f7f89ac1700 1 -- 172.21.15.12:0/835949236 learned_addr learned my addr 172.21.15.12:0/835949236 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.301+0000 7f7f89ac1700 1 -- 172.21.15.12:0/835949236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f780095f0 con 0x7f7f84106900 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.301+0000 7f7f89ac1700 1 --2- 172.21.15.12:0/835949236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84106900 0x7f7f840732a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f78004f40 tx=0x7f7f78005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.301+0000 7f7f76ffd700 1 -- 172.21.15.12:0/835949236 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f7801d070 con 0x7f7f84106900 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.301+0000 7f7f76ffd700 1 -- 172.21.15.12:0/835949236 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f7800bbb0 con 0x7f7f84106900 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.301+0000 7f7f8bd25700 1 -- 172.21.15.12:0/835949236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f840737e0 con 0x7f7f84106900 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.301+0000 7f7f76ffd700 1 -- 172.21.15.12:0/835949236 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f78003810 con 0x7f7f84106900 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.301+0000 7f7f8bd25700 1 -- 172.21.15.12:0/835949236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f84073c60 con 0x7f7f84106900 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.302+0000 7f7f76ffd700 1 -- 172.21.15.12:0/835949236 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f7f78003a40 con 0x7f7f84106900 2024-05-08T23:40:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.302+0000 7f7f76ffd700 1 --2- 172.21.15.12:0/835949236 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7f700387f0 0x7f7f7003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.302+0000 7f7f76ffd700 1 -- 172.21.15.12:0/835949236 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7f7804c3f0 con 0x7f7f84106900 2024-05-08T23:40:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.302+0000 7f7f892c0700 1 --2- 172.21.15.12:0/835949236 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7f700387f0 0x7f7f7003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.302+0000 7f7f8bd25700 1 -- 172.21.15.12:0/835949236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f84104820 con 0x7f7f84106900 2024-05-08T23:40:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.303+0000 7f7f892c0700 1 --2- 172.21.15.12:0/835949236 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7f700387f0 0x7f7f7003acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f80006fd0 tx=0x7f7f80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.308+0000 7f7f76ffd700 1 -- 172.21.15.12:0/835949236 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7f78026070 con 0x7f7f84106900 2024-05-08T23:40:08.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:08.524+0000 7f7f8bd25700 1 -- 172.21.15.12:0/835949236 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f84074890 con 0x7f7f700387f0 2024-05-08T23:40:09.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:09 smithi012 ceph-mon[42503]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:09.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:09 smithi012 ceph-mon[42503]: Deploying cephadm binary to smithi160 2024-05-08T23:40:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.843+0000 7f7f76ffd700 1 -- 172.21.15.12:0/835949236 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7f84074890 con 0x7f7f700387f0 2024-05-08T23:40:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi160' with addr '172.21.15.160' 2024-05-08T23:40:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.846+0000 7f7f8bd25700 1 -- 172.21.15.12:0/835949236 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7f700387f0 msgr2=0x7f7f7003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.846+0000 7f7f8bd25700 1 --2- 172.21.15.12:0/835949236 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7f700387f0 0x7f7f7003acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f80006fd0 tx=0x7f7f80006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.846+0000 7f7f8bd25700 1 -- 172.21.15.12:0/835949236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84106900 msgr2=0x7f7f840732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.846+0000 7f7f8bd25700 1 --2- 172.21.15.12:0/835949236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84106900 0x7f7f840732a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f78004f40 tx=0x7f7f78005020 comp rx=0 tx=0).stop 2024-05-08T23:40:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.846+0000 7f7f8bd25700 1 -- 172.21.15.12:0/835949236 shutdown_connections 2024-05-08T23:40:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.846+0000 7f7f8bd25700 1 --2- 172.21.15.12:0/835949236 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7f700387f0 0x7f7f7003acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.846+0000 7f7f8bd25700 1 --2- 172.21.15.12:0/835949236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84106900 0x7f7f840732a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.846+0000 7f7f8bd25700 1 -- 172.21.15.12:0/835949236 >> 172.21.15.12:0/835949236 conn(0x7f7f840fc410 msgr2=0x7f7f84108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.846+0000 7f7f8bd25700 1 -- 172.21.15.12:0/835949236 shutdown_connections 2024-05-08T23:40:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:10.846+0000 7f7f8bd25700 1 -- 172.21.15.12:0/835949236 wait complete. 2024-05-08T23:40:11.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T23:40:11.478 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:40:12.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:12.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:11 smithi012 ceph-mon[42503]: Added host smithi160 2024-05-08T23:40:12.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:12.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:12.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:12.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.362+0000 7f71cd842700 1 -- 172.21.15.12:0/2534999348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c81057c0 msgr2=0x7f71c8105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:12.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.362+0000 7f71cd842700 1 --2- 172.21.15.12:0/2534999348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c81057c0 0x7f71c8105ba0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009910 tx=0x7f71b8009c20 comp rx=0 tx=0).stop 2024-05-08T23:40:12.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.362+0000 7f71cd842700 1 -- 172.21.15.12:0/2534999348 shutdown_connections 2024-05-08T23:40:12.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.362+0000 7f71cd842700 1 --2- 172.21.15.12:0/2534999348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c81057c0 0x7f71c8105ba0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:12.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.362+0000 7f71cd842700 1 -- 172.21.15.12:0/2534999348 >> 172.21.15.12:0/2534999348 conn(0x7f71c80fb2b0 msgr2=0x7f71c80fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:12.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.362+0000 7f71cd842700 1 -- 172.21.15.12:0/2534999348 shutdown_connections 2024-05-08T23:40:12.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.362+0000 7f71cd842700 1 -- 172.21.15.12:0/2534999348 wait complete. 2024-05-08T23:40:12.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.363+0000 7f71cd842700 1 Processor -- start 2024-05-08T23:40:12.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.363+0000 7f71cd842700 1 -- start start 2024-05-08T23:40:12.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71cd842700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c81057c0 0x7f71c8196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:12.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71cd842700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8196670 con 0x7f71c81057c0 2024-05-08T23:40:12.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c81057c0 0x7f71c8196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:12.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c81057c0 0x7f71c8196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38760/0 (socket says 172.21.15.12:38760) 2024-05-08T23:40:12.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71c6ffd700 1 -- 172.21.15.12:0/3957647432 learned_addr learned my addr 172.21.15.12:0/3957647432 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:12.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71c6ffd700 1 -- 172.21.15.12:0/3957647432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80095f0 con 0x7f71c81057c0 2024-05-08T23:40:12.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71c6ffd700 1 --2- 172.21.15.12:0/3957647432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c81057c0 0x7f71c8196130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f71b8004f40 tx=0x7f71b8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:12.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71cc840700 1 -- 172.21.15.12:0/3957647432 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71b801d070 con 0x7f71c81057c0 2024-05-08T23:40:12.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71cc840700 1 -- 172.21.15.12:0/3957647432 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71b800bbb0 con 0x7f71c81057c0 2024-05-08T23:40:12.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71cd842700 1 -- 172.21.15.12:0/3957647432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c81968d0 con 0x7f71c81057c0 2024-05-08T23:40:12.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71cc840700 1 -- 172.21.15.12:0/3957647432 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71b8003810 con 0x7f71c81057c0 2024-05-08T23:40:12.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.364+0000 7f71cd842700 1 -- 172.21.15.12:0/3957647432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c8196d50 con 0x7f71c81057c0 2024-05-08T23:40:12.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.365+0000 7f71cc840700 1 -- 172.21.15.12:0/3957647432 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f71b8003a90 con 0x7f71c81057c0 2024-05-08T23:40:12.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.365+0000 7f71cc840700 1 --2- 172.21.15.12:0/3957647432 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f71b40387a0 0x7f71b403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:12.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.365+0000 7f71cc840700 1 -- 172.21.15.12:0/3957647432 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f71b804d3b0 con 0x7f71c81057c0 2024-05-08T23:40:12.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.366+0000 7f71be7fc700 1 --2- 172.21.15.12:0/3957647432 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f71b40387a0 0x7f71b403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:12.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.366+0000 7f71cd842700 1 -- 172.21.15.12:0/3957647432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c8100f00 con 0x7f71c81057c0 2024-05-08T23:40:12.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.366+0000 7f71be7fc700 1 --2- 172.21.15.12:0/3957647432 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f71b40387a0 0x7f71b403ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f71b0006fd0 tx=0x7f71b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:12.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.371+0000 7f71cc840700 1 -- 172.21.15.12:0/3957647432 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f71b8026020 con 0x7f71c81057c0 2024-05-08T23:40:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.587+0000 7f71cd842700 1 -- 172.21.15.12:0/3957647432 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71c81974b0 con 0x7f71b40387a0 2024-05-08T23:40:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.589+0000 7f71cc840700 1 -- 172.21.15.12:0/3957647432 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f71c81974b0 con 0x7f71b40387a0 2024-05-08T23:40:12.586 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:12.586 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}] 2024-05-08T23:40:12.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.591+0000 7f71cd842700 1 -- 172.21.15.12:0/3957647432 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f71b40387a0 msgr2=0x7f71b403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:12.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.591+0000 7f71cd842700 1 --2- 172.21.15.12:0/3957647432 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f71b40387a0 0x7f71b403ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f71b0006fd0 tx=0x7f71b0006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:12.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.591+0000 7f71cd842700 1 -- 172.21.15.12:0/3957647432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c81057c0 msgr2=0x7f71c8196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:12.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.591+0000 7f71cd842700 1 --2- 172.21.15.12:0/3957647432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c81057c0 0x7f71c8196130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f71b8004f40 tx=0x7f71b8005020 comp rx=0 tx=0).stop 2024-05-08T23:40:12.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.591+0000 7f71cd842700 1 -- 172.21.15.12:0/3957647432 shutdown_connections 2024-05-08T23:40:12.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.591+0000 7f71cd842700 1 --2- 172.21.15.12:0/3957647432 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f71b40387a0 0x7f71b403ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:12.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.591+0000 7f71cd842700 1 --2- 172.21.15.12:0/3957647432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c81057c0 0x7f71c8196130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:12.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.591+0000 7f71cd842700 1 -- 172.21.15.12:0/3957647432 >> 172.21.15.12:0/3957647432 conn(0x7f71c80fb2b0 msgr2=0x7f71c8068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:12.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.591+0000 7f71cd842700 1 -- 172.21.15.12:0/3957647432 shutdown_connections 2024-05-08T23:40:12.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:12.591+0000 7f71cd842700 1 -- 172.21.15.12:0/3957647432 wait complete. 2024-05-08T23:40:12.966 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-05-08T23:40:12.966 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:40:12.966 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-05-08T23:40:12.989 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:40:12.989 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:13.060 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-05-08T23:40:13.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch host add smithi204 2024-05-08T23:40:13.314 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:40:13.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:13 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:14.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.358+0000 7fb477cab700 1 -- 172.21.15.12:0/1109657734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb470068120 msgr2=0x7fb470068500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:14.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.358+0000 7fb477cab700 1 --2- 172.21.15.12:0/1109657734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb470068120 0x7fb470068500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009910 tx=0x7fb46c009c20 comp rx=0 tx=0).stop 2024-05-08T23:40:14.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.358+0000 7fb477cab700 1 -- 172.21.15.12:0/1109657734 shutdown_connections 2024-05-08T23:40:14.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.358+0000 7fb477cab700 1 --2- 172.21.15.12:0/1109657734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb470068120 0x7fb470068500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:14.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.358+0000 7fb477cab700 1 -- 172.21.15.12:0/1109657734 >> 172.21.15.12:0/1109657734 conn(0x7fb470075350 msgr2=0x7fb470075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:14.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.358+0000 7fb477cab700 1 -- 172.21.15.12:0/1109657734 shutdown_connections 2024-05-08T23:40:14.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.358+0000 7fb477cab700 1 -- 172.21.15.12:0/1109657734 wait complete. 2024-05-08T23:40:14.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.359+0000 7fb477cab700 1 Processor -- start 2024-05-08T23:40:14.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.359+0000 7fb477cab700 1 -- start start 2024-05-08T23:40:14.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb477cab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb470068120 0x7fb4701983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:14.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb477cab700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4701988e0 con 0x7fb470068120 2024-05-08T23:40:14.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb475a47700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb470068120 0x7fb4701983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:14.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb475a47700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb470068120 0x7fb4701983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38772/0 (socket says 172.21.15.12:38772) 2024-05-08T23:40:14.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb475a47700 1 -- 172.21.15.12:0/2869130972 learned_addr learned my addr 172.21.15.12:0/2869130972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:14.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb475a47700 1 -- 172.21.15.12:0/2869130972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0095f0 con 0x7fb470068120 2024-05-08T23:40:14.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb475a47700 1 --2- 172.21.15.12:0/2869130972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb470068120 0x7fb4701983a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb46c004f40 tx=0x7fb46c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:14.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb466ffd700 1 -- 172.21.15.12:0/2869130972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb46c01d070 con 0x7fb470068120 2024-05-08T23:40:14.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb477cab700 1 -- 172.21.15.12:0/2869130972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb470198ae0 con 0x7fb470068120 2024-05-08T23:40:14.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb466ffd700 1 -- 172.21.15.12:0/2869130972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb46c00bc10 con 0x7fb470068120 2024-05-08T23:40:14.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.360+0000 7fb466ffd700 1 -- 172.21.15.12:0/2869130972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb46c0037d0 con 0x7fb470068120 2024-05-08T23:40:14.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.361+0000 7fb477cab700 1 -- 172.21.15.12:0/2869130972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb470198f00 con 0x7fb470068120 2024-05-08T23:40:14.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.361+0000 7fb466ffd700 1 -- 172.21.15.12:0/2869130972 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fb46c00b6f0 con 0x7fb470068120 2024-05-08T23:40:14.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.362+0000 7fb477cab700 1 -- 172.21.15.12:0/2869130972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4700ffdf0 con 0x7fb470068120 2024-05-08T23:40:14.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.362+0000 7fb466ffd700 1 --2- 172.21.15.12:0/2869130972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb45c0387a0 0x7fb45c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:14.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.362+0000 7fb466ffd700 1 -- 172.21.15.12:0/2869130972 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb46c04c3b0 con 0x7fb470068120 2024-05-08T23:40:14.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.362+0000 7fb475246700 1 --2- 172.21.15.12:0/2869130972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb45c0387a0 0x7fb45c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:14.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.362+0000 7fb475246700 1 --2- 172.21.15.12:0/2869130972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb45c0387a0 0x7fb45c03ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb460006fd0 tx=0x7fb460006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:14.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.367+0000 7fb466ffd700 1 -- 172.21.15.12:0/2869130972 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb46c00bd80 con 0x7fb470068120 2024-05-08T23:40:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:14.585+0000 7fb477cab700 1 -- 172.21.15.12:0/2869130972 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) v1 -- 0x7fb47010cdf0 con 0x7fb45c0387a0 2024-05-08T23:40:14.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:14 smithi012 ceph-mon[42503]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:40:15.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:15.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:15.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:15.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:15.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:40:15.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:15.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:15.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:15 smithi012 ceph-mon[42503]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-08T23:40:16.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:16 smithi012 ceph-mon[42503]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:16.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:16 smithi012 ceph-mon[42503]: Updating smithi160:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:40:16.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:16 smithi012 ceph-mon[42503]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:16.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:16 smithi012 ceph-mon[42503]: Deploying cephadm binary to smithi204 2024-05-08T23:40:16.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:16 smithi012 ceph-mon[42503]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:16.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:16 smithi012 ceph-mon[42503]: Updating smithi160:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:40:16.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:16.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:16.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:16.946 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-05-08T23:40:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.948+0000 7fb466ffd700 1 -- 172.21.15.12:0/2869130972 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb47010cdf0 con 0x7fb45c0387a0 2024-05-08T23:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.951+0000 7fb477cab700 1 -- 172.21.15.12:0/2869130972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb45c0387a0 msgr2=0x7fb45c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.951+0000 7fb477cab700 1 --2- 172.21.15.12:0/2869130972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb45c0387a0 0x7fb45c03ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb460006fd0 tx=0x7fb460006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.951+0000 7fb477cab700 1 -- 172.21.15.12:0/2869130972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb470068120 msgr2=0x7fb4701983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.951+0000 7fb477cab700 1 --2- 172.21.15.12:0/2869130972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb470068120 0x7fb4701983a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb46c004f40 tx=0x7fb46c005020 comp rx=0 tx=0).stop 2024-05-08T23:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.951+0000 7fb477cab700 1 -- 172.21.15.12:0/2869130972 shutdown_connections 2024-05-08T23:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.951+0000 7fb477cab700 1 --2- 172.21.15.12:0/2869130972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb45c0387a0 0x7fb45c03ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.951+0000 7fb477cab700 1 --2- 172.21.15.12:0/2869130972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb470068120 0x7fb4701983a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:16.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.951+0000 7fb477cab700 1 -- 172.21.15.12:0/2869130972 >> 172.21.15.12:0/2869130972 conn(0x7fb470075350 msgr2=0x7fb470107e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:16.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.951+0000 7fb477cab700 1 -- 172.21.15.12:0/2869130972 shutdown_connections 2024-05-08T23:40:16.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:16.951+0000 7fb477cab700 1 -- 172.21.15.12:0/2869130972 wait complete. 2024-05-08T23:40:17.296 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T23:40:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:40:18.363 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:17 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:18.363 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:17 smithi012 ceph-mon[42503]: Added host smithi204 2024-05-08T23:40:18.363 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:17 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:18.363 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:17 smithi012 ceph-mon[42503]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:18.363 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:17 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:18.363 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:17 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.555+0000 7ffbbb076700 1 -- 172.21.15.12:0/3649216306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb41068c0 msgr2=0x7ffbb4106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.555+0000 7ffbbb076700 1 --2- 172.21.15.12:0/3649216306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb41068c0 0x7ffbb4106ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009910 tx=0x7ffbb0009c20 comp rx=0 tx=0).stop 2024-05-08T23:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.555+0000 7ffbbb076700 1 -- 172.21.15.12:0/3649216306 shutdown_connections 2024-05-08T23:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.555+0000 7ffbbb076700 1 --2- 172.21.15.12:0/3649216306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb41068c0 0x7ffbb4106ca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.555+0000 7ffbbb076700 1 -- 172.21.15.12:0/3649216306 >> 172.21.15.12:0/3649216306 conn(0x7ffbb4075350 msgr2=0x7ffbb4075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.555+0000 7ffbbb076700 1 -- 172.21.15.12:0/3649216306 shutdown_connections 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.555+0000 7ffbbb076700 1 -- 172.21.15.12:0/3649216306 wait complete. 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.555+0000 7ffbbb076700 1 Processor -- start 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.555+0000 7ffbbb076700 1 -- start start 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffbbb076700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb41068c0 0x7ffbb4196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffbbb076700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb4196670 con 0x7ffbb41068c0 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffbb8e12700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb41068c0 0x7ffbb4196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffbb8e12700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb41068c0 0x7ffbb4196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51434/0 (socket says 172.21.15.12:51434) 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffbb8e12700 1 -- 172.21.15.12:0/3408561433 learned_addr learned my addr 172.21.15.12:0/3408561433 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffbb8e12700 1 -- 172.21.15.12:0/3408561433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb00095f0 con 0x7ffbb41068c0 2024-05-08T23:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffbb8e12700 1 --2- 172.21.15.12:0/3408561433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb41068c0 0x7ffbb4196130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0004f40 tx=0x7ffbb0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffba9ffb700 1 -- 172.21.15.12:0/3408561433 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbb001d070 con 0x7ffbb41068c0 2024-05-08T23:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffba9ffb700 1 -- 172.21.15.12:0/3408561433 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbb000bbb0 con 0x7ffbb41068c0 2024-05-08T23:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffbbb076700 1 -- 172.21.15.12:0/3408561433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb41968d0 con 0x7ffbb41068c0 2024-05-08T23:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.556+0000 7ffba9ffb700 1 -- 172.21.15.12:0/3408561433 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbb0003810 con 0x7ffbb41068c0 2024-05-08T23:40:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.557+0000 7ffbbb076700 1 -- 172.21.15.12:0/3408561433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb4196d50 con 0x7ffbb41068c0 2024-05-08T23:40:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.558+0000 7ffba9ffb700 1 -- 172.21.15.12:0/3408561433 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7ffbb0003a90 con 0x7ffbb41068c0 2024-05-08T23:40:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.558+0000 7ffbbb076700 1 -- 172.21.15.12:0/3408561433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb4102000 con 0x7ffbb41068c0 2024-05-08T23:40:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.558+0000 7ffba9ffb700 1 --2- 172.21.15.12:0/3408561433 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ffb9c0387a0 0x7ffb9c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.558+0000 7ffba9ffb700 1 -- 172.21.15.12:0/3408561433 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffbb004c7b0 con 0x7ffbb41068c0 2024-05-08T23:40:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.558+0000 7ffbabfff700 1 --2- 172.21.15.12:0/3408561433 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ffb9c0387a0 0x7ffb9c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.558+0000 7ffbabfff700 1 --2- 172.21.15.12:0/3408561433 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ffb9c0387a0 0x7ffb9c03ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffba4006fd0 tx=0x7ffba4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:18.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.564+0000 7ffba9ffb700 1 -- 172.21.15.12:0/3408561433 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffbb0029770 con 0x7ffbb41068c0 2024-05-08T23:40:18.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.781+0000 7ffbbb076700 1 -- 172.21.15.12:0/3408561433 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbb410abb0 con 0x7ffb9c0387a0 2024-05-08T23:40:18.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.782+0000 7ffba9ffb700 1 -- 172.21.15.12:0/3408561433 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7ffbb410abb0 con 0x7ffb9c0387a0 2024-05-08T23:40:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:40:18.779 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-05-08T23:40:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.784+0000 7ffbbb076700 1 -- 172.21.15.12:0/3408561433 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ffb9c0387a0 msgr2=0x7ffb9c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.784+0000 7ffbbb076700 1 --2- 172.21.15.12:0/3408561433 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ffb9c0387a0 0x7ffb9c03ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffba4006fd0 tx=0x7ffba4006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.784+0000 7ffbbb076700 1 -- 172.21.15.12:0/3408561433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb41068c0 msgr2=0x7ffbb4196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.784+0000 7ffbbb076700 1 --2- 172.21.15.12:0/3408561433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb41068c0 0x7ffbb4196130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0004f40 tx=0x7ffbb0005020 comp rx=0 tx=0).stop 2024-05-08T23:40:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.784+0000 7ffbbb076700 1 -- 172.21.15.12:0/3408561433 shutdown_connections 2024-05-08T23:40:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.784+0000 7ffbbb076700 1 --2- 172.21.15.12:0/3408561433 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ffb9c0387a0 0x7ffb9c03ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.784+0000 7ffbbb076700 1 --2- 172.21.15.12:0/3408561433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb41068c0 0x7ffbb4196130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.784+0000 7ffbbb076700 1 -- 172.21.15.12:0/3408561433 >> 172.21.15.12:0/3408561433 conn(0x7ffbb4075350 msgr2=0x7ffbb40fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.785+0000 7ffbbb076700 1 -- 172.21.15.12:0/3408561433 shutdown_connections 2024-05-08T23:40:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:18.785+0000 7ffbbb076700 1 -- 172.21.15.12:0/3408561433 wait complete. 2024-05-08T23:40:19.173 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T23:40:19.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T23:40:19.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:19 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:19.425 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:40:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.354+0000 7ff6749a1700 1 -- 172.21.15.12:0/2949932429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6701068c0 msgr2=0x7ff670106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.354+0000 7ff6749a1700 1 --2- 172.21.15.12:0/2949932429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6701068c0 0x7ff670106ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff660009910 tx=0x7ff660009c20 comp rx=0 tx=0).stop 2024-05-08T23:40:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.354+0000 7ff6749a1700 1 -- 172.21.15.12:0/2949932429 shutdown_connections 2024-05-08T23:40:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.354+0000 7ff6749a1700 1 --2- 172.21.15.12:0/2949932429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6701068c0 0x7ff670106ca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.354+0000 7ff6749a1700 1 -- 172.21.15.12:0/2949932429 >> 172.21.15.12:0/2949932429 conn(0x7ff670075350 msgr2=0x7ff670075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.354+0000 7ff6749a1700 1 -- 172.21.15.12:0/2949932429 shutdown_connections 2024-05-08T23:40:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.354+0000 7ff6749a1700 1 -- 172.21.15.12:0/2949932429 wait complete. 2024-05-08T23:40:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.355+0000 7ff6749a1700 1 Processor -- start 2024-05-08T23:40:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.355+0000 7ff6749a1700 1 -- start start 2024-05-08T23:40:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.355+0000 7ff6749a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6701068c0 0x7ff670196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.355+0000 7ff6749a1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff670196670 con 0x7ff6701068c0 2024-05-08T23:40:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.356+0000 7ff66e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6701068c0 0x7ff670196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.356+0000 7ff66e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6701068c0 0x7ff670196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51454/0 (socket says 172.21.15.12:51454) 2024-05-08T23:40:20.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.356+0000 7ff66e59c700 1 -- 172.21.15.12:0/336002683 learned_addr learned my addr 172.21.15.12:0/336002683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:20.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.356+0000 7ff66e59c700 1 -- 172.21.15.12:0/336002683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600095f0 con 0x7ff6701068c0 2024-05-08T23:40:20.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.356+0000 7ff66e59c700 1 --2- 172.21.15.12:0/336002683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6701068c0 0x7ff670196130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff660006010 tx=0x7ff660004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:20.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.356+0000 7ff6677fe700 1 -- 172.21.15.12:0/336002683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff66001d070 con 0x7ff6701068c0 2024-05-08T23:40:20.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.356+0000 7ff6677fe700 1 -- 172.21.15.12:0/336002683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff66000bbb0 con 0x7ff6701068c0 2024-05-08T23:40:20.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.356+0000 7ff6749a1700 1 -- 172.21.15.12:0/336002683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6701968d0 con 0x7ff6701068c0 2024-05-08T23:40:20.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.356+0000 7ff6677fe700 1 -- 172.21.15.12:0/336002683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff660003810 con 0x7ff6701068c0 2024-05-08T23:40:20.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.356+0000 7ff6749a1700 1 -- 172.21.15.12:0/336002683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff670196d50 con 0x7ff6701068c0 2024-05-08T23:40:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.357+0000 7ff6749a1700 1 -- 172.21.15.12:0/336002683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff670102000 con 0x7ff6701068c0 2024-05-08T23:40:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.358+0000 7ff6677fe700 1 -- 172.21.15.12:0/336002683 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7ff66000bd20 con 0x7ff6701068c0 2024-05-08T23:40:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.358+0000 7ff6677fe700 1 --2- 172.21.15.12:0/336002683 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff65c0387f0 0x7ff65c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.358+0000 7ff6677fe700 1 -- 172.21.15.12:0/336002683 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff66004c7b0 con 0x7ff6701068c0 2024-05-08T23:40:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.358+0000 7ff66dd9b700 1 --2- 172.21.15.12:0/336002683 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff65c0387f0 0x7ff65c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.358+0000 7ff66dd9b700 1 --2- 172.21.15.12:0/336002683 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff65c0387f0 0x7ff65c03acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff658006fd0 tx=0x7ff658006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.363+0000 7ff6677fe700 1 -- 172.21.15.12:0/336002683 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff66000e3e0 con 0x7ff6701068c0 2024-05-08T23:40:20.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:20.580+0000 7ff6749a1700 1 -- 172.21.15.12:0/336002683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff67010abb0 con 0x7ff6701068c0 2024-05-08T23:40:20.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:20 smithi012 ceph-mon[42503]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:40:20.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:20 smithi012 ceph-mon[42503]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:21.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.317+0000 7ff6677fe700 1 -- 172.21.15.12:0/336002683 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff660026070 con 0x7ff6701068c0 2024-05-08T23:40:21.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.320+0000 7ff6749a1700 1 -- 172.21.15.12:0/336002683 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff65c0387f0 msgr2=0x7ff65c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:21.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.320+0000 7ff6749a1700 1 --2- 172.21.15.12:0/336002683 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff65c0387f0 0x7ff65c03acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff658006fd0 tx=0x7ff658006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:21.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.320+0000 7ff6749a1700 1 -- 172.21.15.12:0/336002683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6701068c0 msgr2=0x7ff670196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:21.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.320+0000 7ff6749a1700 1 --2- 172.21.15.12:0/336002683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6701068c0 0x7ff670196130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff660006010 tx=0x7ff660004ff0 comp rx=0 tx=0).stop 2024-05-08T23:40:21.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.320+0000 7ff6749a1700 1 -- 172.21.15.12:0/336002683 shutdown_connections 2024-05-08T23:40:21.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.320+0000 7ff6749a1700 1 --2- 172.21.15.12:0/336002683 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff65c0387f0 0x7ff65c03acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:21.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.320+0000 7ff6749a1700 1 --2- 172.21.15.12:0/336002683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6701068c0 0x7ff670196130 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:21.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.320+0000 7ff6749a1700 1 -- 172.21.15.12:0/336002683 >> 172.21.15.12:0/336002683 conn(0x7ff670075350 msgr2=0x7ff6700fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:21.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.320+0000 7ff6749a1700 1 -- 172.21.15.12:0/336002683 shutdown_connections 2024-05-08T23:40:21.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:21.320+0000 7ff6749a1700 1 -- 172.21.15.12:0/336002683 wait complete. 2024-05-08T23:40:21.318 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-05-08T23:40:21.639 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:21 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/336002683' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T23:40:21.639 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:21.639 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:21.639 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:21.639 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:21.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:40:21.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:21.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:21.696 INFO:tasks.cephadm:Adding mon.a on smithi012 2024-05-08T23:40:21.697 INFO:tasks.cephadm:Adding mon.b on smithi160 2024-05-08T23:40:21.697 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-05-08T23:40:21.697 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi012:172.21.15.12=a;smithi160:172.21.15.160=b;smithi204:172.21.15.204=c' 2024-05-08T23:40:21.932 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:40:22.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:22 smithi012 ceph-mon[42503]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-08T23:40:22.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:22 smithi012 ceph-mon[42503]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:40:22.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:22 smithi012 ceph-mon[42503]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:22.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:22 smithi012 ceph-mon[42503]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:22.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:22 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/336002683' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T23:40:22.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:22 smithi012 ceph-mon[42503]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:22.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:22 smithi012 ceph-mon[42503]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:40:22.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:22.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:22.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:22.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.856+0000 7f8e970bd700 1 -- 172.21.15.204:0/2377740566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e90105820 msgr2=0x7f8e90105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:22.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.856+0000 7f8e970bd700 1 --2- 172.21.15.204:0/2377740566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e90105820 0x7f8e90105c00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009910 tx=0x7f8e84009c20 comp rx=0 tx=0).stop 2024-05-08T23:40:22.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.856+0000 7f8e970bd700 1 -- 172.21.15.204:0/2377740566 shutdown_connections 2024-05-08T23:40:22.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.856+0000 7f8e970bd700 1 --2- 172.21.15.204:0/2377740566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e90105820 0x7f8e90105c00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:22.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.856+0000 7f8e970bd700 1 -- 172.21.15.204:0/2377740566 >> 172.21.15.204:0/2377740566 conn(0x7f8e90075350 msgr2=0x7f8e90075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:22.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.856+0000 7f8e970bd700 1 -- 172.21.15.204:0/2377740566 shutdown_connections 2024-05-08T23:40:22.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.856+0000 7f8e970bd700 1 -- 172.21.15.204:0/2377740566 wait complete. 2024-05-08T23:40:22.855 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.857+0000 7f8e970bd700 1 Processor -- start 2024-05-08T23:40:22.855 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.857+0000 7f8e970bd700 1 -- start start 2024-05-08T23:40:22.855 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.857+0000 7f8e970bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e90105820 0x7f8e901961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.857+0000 7f8e970bd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e901966f0 con 0x7f8e90105820 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.857+0000 7f8e94e59700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e90105820 0x7f8e901961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.857+0000 7f8e94e59700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e90105820 0x7f8e901961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.204:44432/0 (socket says 172.21.15.204:44432) 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.857+0000 7f8e94e59700 1 -- 172.21.15.204:0/2484578731 learned_addr learned my addr 172.21.15.204:0/2484578731 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.858+0000 7f8e94e59700 1 -- 172.21.15.204:0/2484578731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e840095f0 con 0x7f8e90105820 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.858+0000 7f8e94e59700 1 --2- 172.21.15.204:0/2484578731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e90105820 0x7f8e901961b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e84004f40 tx=0x7f8e84005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.858+0000 7f8e8dffb700 1 -- 172.21.15.204:0/2484578731 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e8401d070 con 0x7f8e90105820 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.858+0000 7f8e970bd700 1 -- 172.21.15.204:0/2484578731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e901968f0 con 0x7f8e90105820 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.858+0000 7f8e8dffb700 1 -- 172.21.15.204:0/2484578731 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e84004370 con 0x7f8e90105820 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.858+0000 7f8e8dffb700 1 -- 172.21.15.204:0/2484578731 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e84003970 con 0x7f8e90105820 2024-05-08T23:40:22.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.858+0000 7f8e970bd700 1 -- 172.21.15.204:0/2484578731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e90196d10 con 0x7f8e90105820 2024-05-08T23:40:22.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.859+0000 7f8e970bd700 1 -- 172.21.15.204:0/2484578731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e900ff330 con 0x7f8e90105820 2024-05-08T23:40:22.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.859+0000 7f8e8dffb700 1 -- 172.21.15.204:0/2484578731 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f8e84003c30 con 0x7f8e90105820 2024-05-08T23:40:22.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.860+0000 7f8e8dffb700 1 --2- 172.21.15.204:0/2484578731 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8e800387a0 0x7f8e8003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:22.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.860+0000 7f8e8dffb700 1 -- 172.21.15.204:0/2484578731 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e8404c470 con 0x7f8e90105820 2024-05-08T23:40:22.858 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.860+0000 7f8e8ffff700 1 --2- 172.21.15.204:0/2484578731 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8e800387a0 0x7f8e8003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:22.858 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.860+0000 7f8e8ffff700 1 --2- 172.21.15.204:0/2484578731 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8e800387a0 0x7f8e8003ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c006fd0 tx=0x7f8e7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:22.862 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:22.866+0000 7f8e8dffb700 1 -- 172.21.15.204:0/2484578731 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8e8402a3b0 con 0x7f8e90105820 2024-05-08T23:40:23.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.069+0000 7f8e970bd700 1 -- 172.21.15.204:0/2484578731 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi160:172.21.15.160=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e90109b30 con 0x7f8e800387a0 2024-05-08T23:40:23.073 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.077+0000 7f8e8dffb700 1 -- 172.21.15.204:0/2484578731 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8e90109b30 con 0x7f8e800387a0 2024-05-08T23:40:23.073 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-05-08T23:40:23.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.079+0000 7f8e970bd700 1 -- 172.21.15.204:0/2484578731 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8e800387a0 msgr2=0x7f8e8003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:23.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.079+0000 7f8e970bd700 1 --2- 172.21.15.204:0/2484578731 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8e800387a0 0x7f8e8003ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c006fd0 tx=0x7f8e7c006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:23.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.079+0000 7f8e970bd700 1 -- 172.21.15.204:0/2484578731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e90105820 msgr2=0x7f8e901961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:23.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.079+0000 7f8e970bd700 1 --2- 172.21.15.204:0/2484578731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e90105820 0x7f8e901961b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e84004f40 tx=0x7f8e84005020 comp rx=0 tx=0).stop 2024-05-08T23:40:23.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.079+0000 7f8e970bd700 1 -- 172.21.15.204:0/2484578731 shutdown_connections 2024-05-08T23:40:23.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.079+0000 7f8e970bd700 1 --2- 172.21.15.204:0/2484578731 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8e800387a0 0x7f8e8003ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.079+0000 7f8e970bd700 1 --2- 172.21.15.204:0/2484578731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e90105820 0x7f8e901961b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.077 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.079+0000 7f8e970bd700 1 -- 172.21.15.204:0/2484578731 >> 172.21.15.204:0/2484578731 conn(0x7f8e90075350 msgr2=0x7f8e900680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:23.077 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.079+0000 7f8e970bd700 1 -- 172.21.15.204:0/2484578731 shutdown_connections 2024-05-08T23:40:23.077 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:23.079+0000 7f8e970bd700 1 -- 172.21.15.204:0/2484578731 wait complete. 2024-05-08T23:40:23.473 DEBUG:teuthology.orchestra.run.smithi160:mon.b> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T23:40:23.478 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T23:40:23.480 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T23:40:23.481 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T23:40:23.511 INFO:journalctl@ceph.mon.c.smithi204.stdout:-- Logs begin at Wed 2024-05-08 23:21:56 UTC. -- 2024-05-08T23:40:23.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:-- Logs begin at Wed 2024-05-08 23:21:47 UTC. -- 2024-05-08T23:40:23.826 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:40:24.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi160:172.21.15.160=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:24.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi160:172.21.15.160=b;smithi204:172.21.15.204=c;count:3 2024-05-08T23:40:24.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:24.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:24.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:24.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:24.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:24.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:24.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:24.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: Deploying daemon mon.c on smithi204 2024-05-08T23:40:24.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:24 smithi012 ceph-mon[42503]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:25.410 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.412+0000 7f6ea5336700 1 -- 172.21.15.204:0/3667295765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea00ff020 msgr2=0x7f6ea00ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:25.410 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.412+0000 7f6ea5336700 1 --2- 172.21.15.204:0/3667295765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea00ff020 0x7f6ea00ff400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009910 tx=0x7f6e90009c20 comp rx=0 tx=0).stop 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.412+0000 7f6ea5336700 1 -- 172.21.15.204:0/3667295765 shutdown_connections 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.412+0000 7f6ea5336700 1 --2- 172.21.15.204:0/3667295765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea00ff020 0x7f6ea00ff400 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.412+0000 7f6ea5336700 1 -- 172.21.15.204:0/3667295765 >> 172.21.15.204:0/3667295765 conn(0x7f6ea00faa00 msgr2=0x7f6ea00fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.412+0000 7f6ea5336700 1 -- 172.21.15.204:0/3667295765 shutdown_connections 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.412+0000 7f6ea5336700 1 -- 172.21.15.204:0/3667295765 wait complete. 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.412+0000 7f6ea5336700 1 Processor -- start 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.413+0000 7f6ea5336700 1 -- start start 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.413+0000 7f6ea5336700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea00ff020 0x7f6ea01961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.413+0000 7f6ea5336700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01966f0 con 0x7f6ea00ff020 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.413+0000 7f6e9effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea00ff020 0x7f6ea01961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.413+0000 7f6e9effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea00ff020 0x7f6ea01961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.204:56610/0 (socket says 172.21.15.204:56610) 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.413+0000 7f6e9effd700 1 -- 172.21.15.204:0/1920056614 learned_addr learned my addr 172.21.15.204:0/1920056614 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.414+0000 7f6e9effd700 1 -- 172.21.15.204:0/1920056614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900095f0 con 0x7f6ea00ff020 2024-05-08T23:40:25.411 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.414+0000 7f6e9effd700 1 --2- 172.21.15.204:0/1920056614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea00ff020 0x7f6ea01961b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e90004f40 tx=0x7f6e90005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.414+0000 7f6e97fff700 1 -- 172.21.15.204:0/1920056614 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e9001d070 con 0x7f6ea00ff020 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.414+0000 7f6ea5336700 1 -- 172.21.15.204:0/1920056614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea01968f0 con 0x7f6ea00ff020 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.414+0000 7f6e97fff700 1 -- 172.21.15.204:0/1920056614 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e9000bc10 con 0x7f6ea00ff020 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.414+0000 7f6e97fff700 1 -- 172.21.15.204:0/1920056614 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e900037d0 con 0x7f6ea00ff020 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.414+0000 7f6ea5336700 1 -- 172.21.15.204:0/1920056614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea0196d10 con 0x7f6ea00ff020 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.415+0000 7f6ea5336700 1 -- 172.21.15.204:0/1920056614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea0108a90 con 0x7f6ea00ff020 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.415+0000 7f6e97fff700 1 -- 172.21.15.204:0/1920056614 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f6e9000b6f0 con 0x7f6ea00ff020 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.415+0000 7f6e97fff700 1 --2- 172.21.15.204:0/1920056614 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e8c0387a0 0x7f6e8c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.415+0000 7f6e97fff700 1 -- 172.21.15.204:0/1920056614 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e9004c390 con 0x7f6ea00ff020 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.415+0000 7f6e9e7fc700 1 --2- 172.21.15.204:0/1920056614 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e8c0387a0 0x7f6e8c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:25.413 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.416+0000 7f6e9e7fc700 1 --2- 172.21.15.204:0/1920056614 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e8c0387a0 0x7f6e8c03ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e88006fd0 tx=0x7f6e88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:25.418 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.421+0000 7f6e97fff700 1 -- 172.21.15.204:0/1920056614 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6e90004440 con 0x7f6ea00ff020 2024-05-08T23:40:25.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.728+0000 7f6ea5336700 1 -- 172.21.15.204:0/1920056614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6ea01020e0 con 0x7f6ea00ff020 2024-05-08T23:40:25.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.729+0000 7f6e97fff700 1 -- 172.21.15.204:0/1920056614 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f6e90026070 con 0x7f6ea00ff020 2024-05-08T23:40:25.726 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:40:25.726 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","modified":"2024-05-08T23:38:59.411324Z","created":"2024-05-08T23:38:59.411324Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T23:40:25.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.731+0000 7f6ea5336700 1 -- 172.21.15.204:0/1920056614 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e8c0387a0 msgr2=0x7f6e8c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:25.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.731+0000 7f6ea5336700 1 --2- 172.21.15.204:0/1920056614 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e8c0387a0 0x7f6e8c03ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e88006fd0 tx=0x7f6e88006e40 comp rx=0 tx=0).stop 2024-05-08T23:40:25.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.731+0000 7f6ea5336700 1 -- 172.21.15.204:0/1920056614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea00ff020 msgr2=0x7f6ea01961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:25.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.731+0000 7f6ea5336700 1 --2- 172.21.15.204:0/1920056614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea00ff020 0x7f6ea01961b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e90004f40 tx=0x7f6e90005020 comp rx=0 tx=0).stop 2024-05-08T23:40:25.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.731+0000 7f6ea5336700 1 -- 172.21.15.204:0/1920056614 shutdown_connections 2024-05-08T23:40:25.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.731+0000 7f6ea5336700 1 --2- 172.21.15.204:0/1920056614 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e8c0387a0 0x7f6e8c03ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:25.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.731+0000 7f6ea5336700 1 --2- 172.21.15.204:0/1920056614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea00ff020 0x7f6ea01961b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:25.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.731+0000 7f6ea5336700 1 -- 172.21.15.204:0/1920056614 >> 172.21.15.204:0/1920056614 conn(0x7f6ea00faa00 msgr2=0x7f6ea01024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:25.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.731+0000 7f6ea5336700 1 -- 172.21.15.204:0/1920056614 shutdown_connections 2024-05-08T23:40:25.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:25.731+0000 7f6ea5336700 1 -- 172.21.15.204:0/1920056614 wait complete. 2024-05-08T23:40:25.729 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-05-08T23:40:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 systemd[1]: Started Ceph mon.c for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:40:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T23:40:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T23:40:26.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T23:40:26.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.table_properties_collectors: 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.inplace_update_support: 0 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T23:40:26.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.bloom_locality: 0 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.max_successive_merges: 0 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.ttl: 2592000 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.enable_blob_files: false 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.min_blob_size: 0 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T23:40:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4872a7fb-94fe-4d5d-9671-78f214887a95 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715211626222623, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715211626223085, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715211626, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4872a7fb-94fe-4d5d-9671-78f214887a95", "db_session_id": "XJHYGSOCVE2OWVORX3IS", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715211626223162, "job": 1, "event": "recovery_finished"} 2024-05-08T23:40:26.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557b33d40000 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: DB pointer 0x557b33d2e000 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** DB Stats ** 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:40:26.271 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-08T23:40:26.272 INFO:journalctl@ceph.mon.c.smithi204.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache BinnedLRUCache@0x557b33ca31f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T23:40:26.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: using public_addr v2:172.21.15.204:0/0 -> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: starting mon.c rank -1 at public addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] at bind addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(???) e0 preinit fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).mds e1 new map 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout: e1 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout: legacy client fscid: -1 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-08T23:40:26.274 INFO:journalctl@ceph.mon.c.smithi204.stdout: No filesystems configured 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/336002683' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:26.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/336002683' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:26.276 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:40:26.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:26.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:26.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:26.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi160:172.21.15.160=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:26.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi160:172.21.15.160=b;smithi204:172.21.15.204=c;count:3 2024-05-08T23:40:26.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:26.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:26.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:26.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:26.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:26.278 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:26.278 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:26.278 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: Deploying daemon mon.c on smithi204 2024-05-08T23:40:26.278 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:26.278 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:26 smithi204 ceph-mon[47373]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T23:40:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:26 smithi012 ceph-mon[42503]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:26 smithi012 ceph-mon[42503]: from='client.? 172.21.15.204:0/1920056614' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:40:27.060 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T23:40:27.060 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T23:40:27.292 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:40:31.445 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.447+0000 7f4312640700 1 -- 172.21.15.204:0/3725014097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f430c101c20 msgr2=0x7f42f4005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:31.445 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.447+0000 7f4312640700 1 --2- 172.21.15.204:0/3725014097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f430c101c20 0x7f42f4005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f42fc005fa0 tx=0x7f42fc00f4a0 comp rx=0 tx=0).stop 2024-05-08T23:40:31.445 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.447+0000 7f4312640700 1 -- 172.21.15.204:0/3725014097 shutdown_connections 2024-05-08T23:40:31.445 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.447+0000 7f4312640700 1 --2- 172.21.15.204:0/3725014097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f430c101c20 0x7f42f4005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:31.445 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.447+0000 7f4312640700 1 -- 172.21.15.204:0/3725014097 >> 172.21.15.204:0/3725014097 conn(0x7f430c076570 msgr2=0x7f430c0fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:31.445 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.447+0000 7f4312640700 1 -- 172.21.15.204:0/3725014097 shutdown_connections 2024-05-08T23:40:31.446 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.447+0000 7f4312640700 1 -- 172.21.15.204:0/3725014097 wait complete. 2024-05-08T23:40:31.446 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.448+0000 7f4312640700 1 Processor -- start 2024-05-08T23:40:31.446 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.448+0000 7f4312640700 1 -- start start 2024-05-08T23:40:31.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f4312640700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f430c101c20 0x7f430c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:31.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f4312640700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c1987e0 0x7f430c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f4312640700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c198d70 con 0x7f430c101c20 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f4312640700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c198ee0 con 0x7f430c1987e0 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c1987e0 0x7f430c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c1987e0 0x7f430c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:35244/0 (socket says 172.21.15.204:35244) 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430b7fe700 1 -- 172.21.15.204:0/882817266 learned_addr learned my addr 172.21.15.204:0/882817266 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430bfff700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f430c101c20 0x7f430c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430b7fe700 1 -- 172.21.15.204:0/882817266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c1987e0 msgr2=0x7f430c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430b7fe700 1 -- 172.21.15.204:0/882817266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c1987e0 msgr2=0x7f430c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430b7fe700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c1987e0 0x7f430c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430b7fe700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c1987e0 0x7f430c19cb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430bfff700 1 -- 172.21.15.204:0/882817266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c1987e0 msgr2=0x7f430c19cb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:31.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430bfff700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c1987e0 0x7f430c19cb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:31.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.449+0000 7f430bfff700 1 -- 172.21.15.204:0/882817266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0139a0 con 0x7f430c101c20 2024-05-08T23:40:31.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.450+0000 7f430bfff700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f430c101c20 0x7f430c1982a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f42fc01e040 tx=0x7f42fc004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:31.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.450+0000 7f43097fa700 1 -- 172.21.15.204:0/882817266 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f42fc018070 con 0x7f430c101c20 2024-05-08T23:40:31.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.450+0000 7f43097fa700 1 -- 172.21.15.204:0/882817266 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42fc005650 con 0x7f430c101c20 2024-05-08T23:40:31.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.450+0000 7f4312640700 1 -- 172.21.15.204:0/882817266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430c19d130 con 0x7f430c101c20 2024-05-08T23:40:31.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.450+0000 7f43097fa700 1 -- 172.21.15.204:0/882817266 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f42fc01cba0 con 0x7f430c101c20 2024-05-08T23:40:31.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.450+0000 7f4312640700 1 -- 172.21.15.204:0/882817266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430c19d680 con 0x7f430c101c20 2024-05-08T23:40:31.449 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.451+0000 7f43097fa700 1 -- 172.21.15.204:0/882817266 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f42fc01cd20 con 0x7f430c101c20 2024-05-08T23:40:31.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.451+0000 7f43097fa700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f42f80385e0 0x7f42f803aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:31.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.451+0000 7f43097fa700 1 -- 172.21.15.204:0/882817266 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f42fc04e160 con 0x7f430c101c20 2024-05-08T23:40:31.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.451+0000 7f4312640700 1 -- 172.21.15.204:0/882817266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430c105790 con 0x7f430c101c20 2024-05-08T23:40:31.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.451+0000 7f430b7fe700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f42f80385e0 0x7f42f803aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:31.450 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.452+0000 7f430b7fe700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f42f80385e0 0x7f42f803aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4300009180 tx=0x7f4300008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:31.454 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.457+0000 7f43097fa700 1 -- 172.21.15.204:0/882817266 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f42fc025e40 con 0x7f430c101c20 2024-05-08T23:40:31.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:40:31.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: mon.a calling monitor election 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: mon.c calling monitor election 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: monmap e2: 2 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: fsmap 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:31.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: mgrmap e13: x(active, since 36s) 2024-05-08T23:40:31.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: overall HEALTH_OK 2024-05-08T23:40:31.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:31.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:31.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.729+0000 7f4312640700 1 -- 172.21.15.204:0/882817266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f430c19d950 con 0x7f430c101c20 2024-05-08T23:40:31.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.730+0000 7f43097fa700 1 -- 172.21.15.204:0/882817266 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f42fc03c020 con 0x7f430c101c20 2024-05-08T23:40:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:40:31.727 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","modified":"2024-05-08T23:40:26.258755Z","created":"2024-05-08T23:38:59.411324Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T23:40:31.729 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.732+0000 7f4312640700 1 -- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f42f80385e0 msgr2=0x7f42f803aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.732+0000 7f4312640700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f42f80385e0 0x7f42f803aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4300009180 tx=0x7f4300008040 comp rx=0 tx=0).stop 2024-05-08T23:40:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.732+0000 7f4312640700 1 -- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f430c101c20 msgr2=0x7f430c1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.732+0000 7f4312640700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f430c101c20 0x7f430c1982a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f42fc01e040 tx=0x7f42fc004d20 comp rx=0 tx=0).stop 2024-05-08T23:40:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.733+0000 7f4312640700 1 -- 172.21.15.204:0/882817266 shutdown_connections 2024-05-08T23:40:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.733+0000 7f4312640700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f42f80385e0 0x7f42f803aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.733+0000 7f4312640700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f430c101c20 0x7f430c1982a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.733+0000 7f4312640700 1 --2- 172.21.15.204:0/882817266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c1987e0 0x7f430c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.733+0000 7f4312640700 1 -- 172.21.15.204:0/882817266 >> 172.21.15.204:0/882817266 conn(0x7f430c076570 msgr2=0x7f430c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.733+0000 7f4312640700 1 -- 172.21.15.204:0/882817266 shutdown_connections 2024-05-08T23:40:31.730 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:31.733+0000 7f4312640700 1 -- 172.21.15.204:0/882817266 wait complete. 2024-05-08T23:40:31.731 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-05-08T23:40:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: from='client.? 172.21.15.204:0/1920056614' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:40:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:40:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: mon.a calling monitor election 2024-05-08T23:40:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: mon.c calling monitor election 2024-05-08T23:40:31.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: monmap e2: 2 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: fsmap 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: mgrmap e13: x(active, since 36s) 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: overall HEALTH_OK 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:31.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:32.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:32 smithi012 ceph-mon[42503]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:32.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:32 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:32.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:32 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:32.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:32 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:32 smithi012 ceph-mon[42503]: Deploying daemon mon.b on smithi160 2024-05-08T23:40:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:32 smithi012 ceph-mon[42503]: from='client.? 172.21.15.204:0/882817266' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:40:32.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:32 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:32.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:32 smithi204 ceph-mon[47373]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:32.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:32 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:32.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:32 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:32.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:32 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:32 smithi204 ceph-mon[47373]: Deploying daemon mon.b on smithi160 2024-05-08T23:40:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:32 smithi204 ceph-mon[47373]: from='client.? 172.21.15.204:0/882817266' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:40:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:32 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:33.104 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T23:40:33.104 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T23:40:33.336 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:40:34.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 systemd[1]: Started Ceph mon.b for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:40:34.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T23:40:34.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T23:40:34.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8a508e98-d895-4e5f-9c9e-c76d1badc926 2024-05-08T23:40:34.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715211633960828, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T23:40:34.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T23:40:34.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715211633961427, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715211633, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8a508e98-d895-4e5f-9c9e-c76d1badc926", "db_session_id": "E28E5Y6W0UN9RPMZD5YF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T23:40:34.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715211633961517, "job": 1, "event": "recovery_finished"} 2024-05-08T23:40:34.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T23:40:34.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T23:40:34.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5611d28c2000 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: DB pointer 0x5611d28b0000 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** DB Stats ** 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:40:34.269 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T23:40:34.270 INFO:journalctl@ceph.mon.b.smithi160.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T23:40:34.271 INFO:journalctl@ceph.mon.b.smithi160.stdout: Block cache BinnedLRUCache@0x5611d28251f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-08T23:40:34.272 INFO:journalctl@ceph.mon.b.smithi160.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T23:40:34.272 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T23:40:34.272 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T23:40:34.272 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-08T23:40:34.272 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: using public_addr v2:172.21.15.160:0/0 -> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-05-08T23:40:34.272 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: starting mon.b rank -1 at public addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] at bind addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:40:34.272 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(???) e0 preinit fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:40:34.272 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).mds e1 new map 2024-05-08T23:40:34.272 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-08T23:40:34.272 INFO:journalctl@ceph.mon.b.smithi160.stdout: e1 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout: legacy client fscid: -1 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout: No filesystems configured 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T23:40:34.273 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/336002683' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-08T23:40:34.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:40:34.275 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:34.275 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:34.275 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/336002683' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T23:40:34.275 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:34.275 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:40:34.275 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.275 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi160:172.21.15.160=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi160:172.21.15.160=b;smithi204:172.21.15.204=c;count:3 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: Deploying daemon mon.c on smithi204 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:34.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='client.? 172.21.15.204:0/1920056614' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.a calling monitor election 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.c calling monitor election 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:40:34.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: monmap e2: 2 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: fsmap 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mgrmap e13: x(active, since 36s) 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: overall HEALTH_OK 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: Deploying daemon mon.b on smithi160 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='client.? 172.21.15.204:0/882817266' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:34.278 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:33 smithi160 ceph-mon[46152]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T23:40:39.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:40:39.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: mon.a calling monitor election 2024-05-08T23:40:39.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: mon.c calling monitor election 2024-05-08T23:40:39.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:39.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:39.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:39.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:40:39.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:40:39.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: fsmap 2024-05-08T23:40:39.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:39.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: mgrmap e13: x(active, since 43s) 2024-05-08T23:40:39.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T23:40:39.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T23:40:39.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T23:40:39.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: mon.b (rank 2) addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] is down (out of quorum) 2024-05-08T23:40:39.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:39.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:39.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:39.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:39.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:40:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: mon.a calling monitor election 2024-05-08T23:40:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: mon.c calling monitor election 2024-05-08T23:40:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: fsmap 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: mgrmap e13: x(active, since 43s) 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: mon.b (rank 2) addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] is down (out of quorum) 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:39.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:39.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:39.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:40.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.220+0000 7fa75a1c4700 1 -- 172.21.15.204:0/2939593395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa740005c30 msgr2=0x7fa740003640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:40.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.220+0000 7fa75a1c4700 1 --2- 172.21.15.204:0/2939593395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa740005c30 0x7fa740003640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7540680d0 tx=0x7fa73c00ff20 comp rx=0 tx=0).stop 2024-05-08T23:40:40.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.220+0000 7fa75a1c4700 1 -- 172.21.15.204:0/2939593395 shutdown_connections 2024-05-08T23:40:40.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.220+0000 7fa75a1c4700 1 --2- 172.21.15.204:0/2939593395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa740005c30 0x7fa740003640 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:40.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.220+0000 7fa75a1c4700 1 -- 172.21.15.204:0/2939593395 >> 172.21.15.204:0/2939593395 conn(0x7fa754076ea0 msgr2=0x7fa7540772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:40.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.220+0000 7fa75a1c4700 1 -- 172.21.15.204:0/2939593395 shutdown_connections 2024-05-08T23:40:40.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.220+0000 7fa75a1c4700 1 -- 172.21.15.204:0/2939593395 wait complete. 2024-05-08T23:40:40.220 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa75a1c4700 1 Processor -- start 2024-05-08T23:40:40.220 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa75a1c4700 1 -- start start 2024-05-08T23:40:40.220 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa75a1c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa740005c30 0x7fa754100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:40.220 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa75a1c4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541007b0 0x7fa754100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa75a1c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7541a2590 0x7fa7541a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa75a1c4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7541a4f50 con 0x7fa740005c30 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa75a1c4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7541a5090 con 0x7fa7541a2590 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa75a1c4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7541a5250 con 0x7fa7541007b0 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa752ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541007b0 0x7fa754100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa7537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa740005c30 0x7fa754100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa752ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541007b0 0x7fa754100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43642/0 (socket says 172.21.15.204:43642) 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa752ffd700 1 -- 172.21.15.204:0/3855406080 learned_addr learned my addr 172.21.15.204:0/3855406080 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa752ffd700 1 -- 172.21.15.204:0/3855406080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541007b0 msgr2=0x7fa754100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa752ffd700 1 -- 172.21.15.204:0/3855406080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541007b0 msgr2=0x7fa754100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa752ffd700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541007b0 0x7fa754100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:40.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa752ffd700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541007b0 0x7fa754100c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa753fff700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7541a2590 0x7fa7541a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa753fff700 1 -- 172.21.15.204:0/3855406080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7541a2590 msgr2=0x7fa7541a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa753fff700 1 -- 172.21.15.204:0/3855406080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7541a2590 msgr2=0x7fa7541a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.221+0000 7fa753fff700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7541a2590 0x7fa7541a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa753fff700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7541a2590 0x7fa7541a4980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa7537fe700 1 -- 172.21.15.204:0/3855406080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541007b0 msgr2=0x7fa754100c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa7537fe700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541007b0 0x7fa754100c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa7537fe700 1 -- 172.21.15.204:0/3855406080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7541a2590 msgr2=0x7fa7541a4980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa7537fe700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7541a2590 0x7fa7541a4980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa7537fe700 1 -- 172.21.15.204:0/3855406080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c00f970 con 0x7fa740005c30 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa7537fe700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa740005c30 0x7fa754100270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7540680d0 tx=0x7fa73c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:40.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa750ff9700 1 -- 172.21.15.204:0/3855406080 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c01dd40 con 0x7fa740005c30 2024-05-08T23:40:40.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa750ff9700 1 -- 172.21.15.204:0/3855406080 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa73c02b070 con 0x7fa740005c30 2024-05-08T23:40:40.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa75a1c4700 1 -- 172.21.15.204:0/3855406080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7541a54d0 con 0x7fa740005c30 2024-05-08T23:40:40.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa750ff9700 1 -- 172.21.15.204:0/3855406080 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c01da00 con 0x7fa740005c30 2024-05-08T23:40:40.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.222+0000 7fa75a1c4700 1 -- 172.21.15.204:0/3855406080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7541a5a20 con 0x7fa740005c30 2024-05-08T23:40:40.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.223+0000 7fa750ff9700 1 -- 172.21.15.204:0/3855406080 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fa73c003680 con 0x7fa740005c30 2024-05-08T23:40:40.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.223+0000 7fa750ff9700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa734038c70 0x7fa73403b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:40.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.223+0000 7fa750ff9700 1 -- 172.21.15.204:0/3855406080 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa73c04e4d0 con 0x7fa740005c30 2024-05-08T23:40:40.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.223+0000 7fa75a1c4700 1 -- 172.21.15.204:0/3855406080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa754068340 con 0x7fa740005c30 2024-05-08T23:40:40.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.223+0000 7fa752ffd700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa734038c70 0x7fa73403b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:40.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.224+0000 7fa752ffd700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa734038c70 0x7fa73403b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa744009180 tx=0x7fa744008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:40.227 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.230+0000 7fa750ff9700 1 -- 172.21.15.204:0/3855406080 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa73c025560 con 0x7fa740005c30 2024-05-08T23:40:40.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:40 smithi012 ceph-mon[42503]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:40.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:40.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:40.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:40.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:40.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:40.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:40.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:40.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:40.513 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.516+0000 7fa75a1c4700 1 -- 172.21.15.204:0/3855406080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa754101750 con 0x7fa740005c30 2024-05-08T23:40:40.513 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.516+0000 7fa750ff9700 1 -- 172.21.15.204:0/3855406080 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7fa73c0263e0 con 0x7fa740005c30 2024-05-08T23:40:40.514 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-08T23:40:40.514 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","modified":"2024-05-08T23:40:33.993249Z","created":"2024-05-08T23:38:59.411324Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T23:40:40.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:40 smithi204 ceph-mon[47373]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:40.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:40.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:40.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:40.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:40.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:40.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:40.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:40.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:40.516 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 -- 172.21.15.204:0/3855406080 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa734038c70 msgr2=0x7fa73403b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:40.516 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa734038c70 0x7fa73403b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa744009180 tx=0x7fa744008040 comp rx=0 tx=0).stop 2024-05-08T23:40:40.516 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 -- 172.21.15.204:0/3855406080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa740005c30 msgr2=0x7fa754100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:40.516 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa740005c30 0x7fa754100270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7540680d0 tx=0x7fa73c005710 comp rx=0 tx=0).stop 2024-05-08T23:40:40.516 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 -- 172.21.15.204:0/3855406080 shutdown_connections 2024-05-08T23:40:40.516 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa734038c70 0x7fa73403b130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:40.517 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7541a2590 0x7fa7541a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:40.517 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa740005c30 0x7fa754100270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:40.517 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 --2- 172.21.15.204:0/3855406080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7541007b0 0x7fa754100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:40.517 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 -- 172.21.15.204:0/3855406080 >> 172.21.15.204:0/3855406080 conn(0x7fa754076ea0 msgr2=0x7fa7540fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:40.517 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 -- 172.21.15.204:0/3855406080 shutdown_connections 2024-05-08T23:40:40.517 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:40.519+0000 7fa75a1c4700 1 -- 172.21.15.204:0/3855406080 wait complete. 2024-05-08T23:40:40.517 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-05-08T23:40:40.885 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T23:40:40.886 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T23:40:41.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:40:41.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mon.a calling monitor election 2024-05-08T23:40:41.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mon.c calling monitor election 2024-05-08T23:40:41.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:41.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:40:41.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:41.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: fsmap 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mgrmap e13: x(active, since 43s) 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T23:40:41.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mon.b (rank 2) addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] is down (out of quorum) 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:41.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:41.423 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:40:41.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: mon.b calling monitor election 2024-05-08T23:40:41.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T23:40:41.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: Reconfiguring daemon mon.a on smithi012 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: mon.b calling monitor election 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: mon.c calling monitor election 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: mon.a calling monitor election 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: fsmap 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: mgrmap e13: x(active, since 45s) 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: Cluster is now healthy 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: overall HEALTH_OK 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:42.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: mon.b calling monitor election 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: Reconfiguring daemon mon.a on smithi012 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: mon.b calling monitor election 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: mon.c calling monitor election 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: mon.a calling monitor election 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: fsmap 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: mgrmap e13: x(active, since 45s) 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T23:40:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: Cluster is now healthy 2024-05-08T23:40:42.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: overall HEALTH_OK 2024-05-08T23:40:42.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:42.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:42.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:42.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mon.b calling monitor election 2024-05-08T23:40:42.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: Reconfiguring daemon mon.a on smithi012 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mon.b calling monitor election 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mon.c calling monitor election 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mon.a calling monitor election 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: fsmap 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: mgrmap e13: x(active, since 45s) 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: Cluster is now healthy 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: overall HEALTH_OK 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:42.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.557+0000 7fd3031b1700 1 -- 172.21.15.12:0/1891671886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2fc100890 msgr2=0x7fd2fc10cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.557+0000 7fd3031b1700 1 --2- 172.21.15.12:0/1891671886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2fc100890 0x7fd2fc10cd40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b0a0 tx=0x7fd2f400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.557+0000 7fd3031b1700 1 -- 172.21.15.12:0/1891671886 shutdown_connections 2024-05-08T23:40:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.557+0000 7fd3031b1700 1 --2- 172.21.15.12:0/1891671886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2fc100890 0x7fd2fc10cd40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.557+0000 7fd3031b1700 1 --2- 172.21.15.12:0/1891671886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2fc105f00 0x7fd2fc1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.557+0000 7fd3031b1700 1 --2- 172.21.15.12:0/1891671886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc0ffed0 0x7fd2fc100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.557+0000 7fd3031b1700 1 -- 172.21.15.12:0/1891671886 >> 172.21.15.12:0/1891671886 conn(0x7fd2fc075f80 msgr2=0x7fd2fc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.557+0000 7fd3031b1700 1 -- 172.21.15.12:0/1891671886 shutdown_connections 2024-05-08T23:40:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.557+0000 7fd3031b1700 1 -- 172.21.15.12:0/1891671886 wait complete. 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd3031b1700 1 Processor -- start 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd3031b1700 1 -- start start 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd3031b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2fc0ffed0 0x7fd2fc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd3031b1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2fc100890 0x7fd2fc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd3031b1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc105f00 0x7fd2fc19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd3031b1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc19c4f0 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd3031b1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc19c660 con 0x7fd2fc105f00 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd3031b1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc19c820 con 0x7fd2fc100890 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd300f4d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2fc0ffed0 0x7fd2fc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd300f4d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2fc0ffed0 0x7fd2fc1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43144/0 (socket says 172.21.15.12:43144) 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd300f4d700 1 -- 172.21.15.12:0/1377611190 learned_addr learned my addr 172.21.15.12:0/1377611190 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd30174e700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc105f00 0x7fd2fc19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd2fbfff700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2fc100890 0x7fd2fc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd300f4d700 1 -- 172.21.15.12:0/1377611190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2fc100890 msgr2=0x7fd2fc197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd300f4d700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2fc100890 0x7fd2fc197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd300f4d700 1 -- 172.21.15.12:0/1377611190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc105f00 msgr2=0x7fd2fc19bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd300f4d700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc105f00 0x7fd2fc19bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd300f4d700 1 -- 172.21.15.12:0/1377611190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4009d00 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.558+0000 7fd30174e700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc105f00 0x7fd2fc19bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.559+0000 7fd300f4d700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2fc0ffed0 0x7fd2fc1975f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b530 tx=0x7fd2ec00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.559+0000 7fd2f9ffb700 1 -- 172.21.15.12:0/1377611190 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec011830 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.559+0000 7fd2f9ffb700 1 -- 172.21.15.12:0/1377611190 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2ec011e50 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.559+0000 7fd3031b1700 1 -- 172.21.15.12:0/1377611190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc19cb00 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.559+0000 7fd2f9ffb700 1 -- 172.21.15.12:0/1377611190 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00f2e0 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.559+0000 7fd3031b1700 1 -- 172.21.15.12:0/1377611190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc1a4360 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.560+0000 7fd2f9ffb700 1 -- 172.21.15.12:0/1377611190 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fd2ec00f440 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.560+0000 7fd2f9ffb700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2e80389d0 0x7fd2e803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:42.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.560+0000 7fd2f9ffb700 1 -- 172.21.15.12:0/1377611190 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd2ec04ad60 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.560+0000 7fd2fbfff700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2e80389d0 0x7fd2e803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:42.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.560+0000 7fd3031b1700 1 -- 172.21.15.12:0/1377611190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc108420 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.561+0000 7fd2fbfff700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2e80389d0 0x7fd2e803ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc198b50 tx=0x7fd2e4005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:42.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.566+0000 7fd2f9ffb700 1 -- 172.21.15.12:0/1377611190 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd2ec0178b0 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.776+0000 7fd3031b1700 1 -- 172.21.15.12:0/1377611190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd2fc1015b0 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.776+0000 7fd2f9ffb700 1 -- 172.21.15.12:0/1377611190 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fd2fc1015b0 con 0x7fd2fc0ffed0 2024-05-08T23:40:42.773 INFO:teuthology.orchestra.run.smithi012.stdout:# minimal ceph.conf for 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:40:42.773 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-05-08T23:40:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:40:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: mon_host = [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-08T23:40:42.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 -- 172.21.15.12:0/1377611190 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2e80389d0 msgr2=0x7fd2e803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:42.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2e80389d0 0x7fd2e803ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc198b50 tx=0x7fd2e4005d50 comp rx=0 tx=0).stop 2024-05-08T23:40:42.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 -- 172.21.15.12:0/1377611190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2fc0ffed0 msgr2=0x7fd2fc1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:42.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2fc0ffed0 0x7fd2fc1975f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b530 tx=0x7fd2ec00b8f0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 -- 172.21.15.12:0/1377611190 shutdown_connections 2024-05-08T23:40:42.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2e80389d0 0x7fd2e803ae90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc105f00 0x7fd2fc19bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2fc0ffed0 0x7fd2fc1975f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 --2- 172.21.15.12:0/1377611190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2fc100890 0x7fd2fc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:42.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 -- 172.21.15.12:0/1377611190 >> 172.21.15.12:0/1377611190 conn(0x7fd2fc075f80 msgr2=0x7fd2fc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:42.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 -- 172.21.15.12:0/1377611190 shutdown_connections 2024-05-08T23:40:42.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:42.779+0000 7fd3031b1700 1 -- 172.21.15.12:0/1377611190 wait complete. 2024-05-08T23:40:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:42 smithi160 ceph-mon[46152]: Reconfiguring mon.b (monmap changed)... 2024-05-08T23:40:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:40:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:42 smithi160 ceph-mon[46152]: Reconfiguring daemon mon.b on smithi160 2024-05-08T23:40:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:42 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1377611190' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:43.050 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:40:42 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:40:42.993+0000 7f34abbec700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T23:40:43.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:42 smithi012 ceph-mon[42503]: Reconfiguring mon.b (monmap changed)... 2024-05-08T23:40:43.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:43.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:40:43.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:43.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:42 smithi012 ceph-mon[42503]: Reconfiguring daemon mon.b on smithi160 2024-05-08T23:40:43.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:43.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:42 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1377611190' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:43.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:43.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:43.151 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T23:40:43.151 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:40:43.151 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T23:40:43.191 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:40:43.191 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:43.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:42 smithi204 ceph-mon[47373]: Reconfiguring mon.b (monmap changed)... 2024-05-08T23:40:43.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:43.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:40:43.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:43.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:42 smithi204 ceph-mon[47373]: Reconfiguring daemon mon.b on smithi160 2024-05-08T23:40:43.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:40:43.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:42 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1377611190' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:43.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:43.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:43.266 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:40:43.266 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T23:40:43.299 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:40:43.300 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:43.367 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:40:43.367 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T23:40:43.397 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:40:43.397 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:40:43.468 INFO:tasks.cephadm:Adding mgr.x on smithi012 2024-05-08T23:40:43.468 INFO:tasks.cephadm:Adding mgr.y on smithi160 2024-05-08T23:40:43.468 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-05-08T23:40:43.468 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi012=x;smithi160=y;smithi204=z' 2024-05-08T23:40:43.722 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:40:44.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: Reconfiguring mon.c (monmap changed)... 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: Reconfiguring daemon mon.c on smithi204 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:44.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: Reconfiguring mon.c (monmap changed)... 2024-05-08T23:40:44.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:44.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:40:44.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:44.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: Reconfiguring daemon mon.c on smithi204 2024-05-08T23:40:44.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:44.180 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:44.180 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:44.180 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:44.180 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:44.180 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:44.180 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:44.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: Reconfiguring mon.c (monmap changed)... 2024-05-08T23:40:44.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:40:44.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:40:44.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:44.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: Reconfiguring daemon mon.c on smithi204 2024-05-08T23:40:44.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:44.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:44.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:44.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:44.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:44.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:44.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:44.668 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.661+0000 7fd839418700 1 -- 172.21.15.204:0/2200417065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8341021e0 msgr2=0x7fd834102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:44.668 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.661+0000 7fd839418700 1 --2- 172.21.15.204:0/2200417065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8341021e0 0x7fd834102640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd828009990 tx=0x7fd828009ca0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.661+0000 7fd839418700 1 -- 172.21.15.204:0/2200417065 shutdown_connections 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.661+0000 7fd839418700 1 --2- 172.21.15.204:0/2200417065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd834102b80 0x7fd834109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.661+0000 7fd839418700 1 --2- 172.21.15.204:0/2200417065 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834100fe0 0x7fd834101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.661+0000 7fd839418700 1 --2- 172.21.15.204:0/2200417065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8341021e0 0x7fd834102640 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 -- 172.21.15.204:0/2200417065 >> 172.21.15.204:0/2200417065 conn(0x7fd8340fc5a0 msgr2=0x7fd8340fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 -- 172.21.15.204:0/2200417065 shutdown_connections 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 -- 172.21.15.204:0/2200417065 wait complete. 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 Processor -- start 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 -- start start 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd834100fe0 0x7fd834196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8341021e0 0x7fd8341974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834102b80 0x7fd83419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:44.669 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83419ead0 con 0x7fd834100fe0 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83419ec10 con 0x7fd834102b80 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.662+0000 7fd839418700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83419eda0 con 0x7fd8341021e0 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd8327fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8341021e0 0x7fd8341974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd8327fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8341021e0 0x7fd8341974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43672/0 (socket says 172.21.15.204:43672) 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd8327fc700 1 -- 172.21.15.204:0/311971251 learned_addr learned my addr 172.21.15.204:0/311971251 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd832ffd700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd834100fe0 0x7fd834196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd8337fe700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834102b80 0x7fd83419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd8327fc700 1 -- 172.21.15.204:0/311971251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834102b80 msgr2=0x7fd83419e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd8327fc700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834102b80 0x7fd83419e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd8327fc700 1 -- 172.21.15.204:0/311971251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd834100fe0 msgr2=0x7fd834196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd8327fc700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd834100fe0 0x7fd834196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd8327fc700 1 -- 172.21.15.204:0/311971251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280095f0 con 0x7fd8341021e0 2024-05-08T23:40:44.670 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd8327fc700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8341021e0 0x7fd8341974d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd828003790 tx=0x7fd82800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:44.671 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd81bfff700 1 -- 172.21.15.204:0/311971251 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82801d070 con 0x7fd8341021e0 2024-05-08T23:40:44.671 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd839418700 1 -- 172.21.15.204:0/311971251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83419f020 con 0x7fd8341021e0 2024-05-08T23:40:44.671 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd81bfff700 1 -- 172.21.15.204:0/311971251 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd828022470 con 0x7fd8341021e0 2024-05-08T23:40:44.671 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd81bfff700 1 -- 172.21.15.204:0/311971251 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828017650 con 0x7fd8341021e0 2024-05-08T23:40:44.671 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.663+0000 7fd839418700 1 -- 172.21.15.204:0/311971251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83419f430 con 0x7fd8341021e0 2024-05-08T23:40:44.671 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.664+0000 7fd81bfff700 1 -- 172.21.15.204:0/311971251 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fd8280177b0 con 0x7fd8341021e0 2024-05-08T23:40:44.671 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.664+0000 7fd839418700 1 -- 172.21.15.204:0/311971251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8341084c0 con 0x7fd8341021e0 2024-05-08T23:40:44.671 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.664+0000 7fd81bfff700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd81c0389d0 0x7fd81c03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:44.671 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.664+0000 7fd81bfff700 1 -- 172.21.15.204:0/311971251 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd82804c720 con 0x7fd8341021e0 2024-05-08T23:40:44.672 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.665+0000 7fd832ffd700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd81c0389d0 0x7fd81c03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:44.672 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.665+0000 7fd832ffd700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd81c0389d0 0x7fd81c03ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd834102040 tx=0x7fd82400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:44.672 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.669+0000 7fd81bfff700 1 -- 172.21.15.204:0/311971251 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd82802a360 con 0x7fd8341021e0 2024-05-08T23:40:44.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.870+0000 7fd839418700 1 -- 172.21.15.204:0/311971251 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi160=y;smithi204=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fd83402ce60 con 0x7fd81c0389d0 2024-05-08T23:40:44.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.877+0000 7fd81bfff700 1 -- 172.21.15.204:0/311971251 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd83402ce60 con 0x7fd81c0389d0 2024-05-08T23:40:44.875 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-05-08T23:40:44.878 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.879+0000 7fd839418700 1 -- 172.21.15.204:0/311971251 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd81c0389d0 msgr2=0x7fd81c03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:44.878 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.879+0000 7fd839418700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd81c0389d0 0x7fd81c03ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd834102040 tx=0x7fd82400b410 comp rx=0 tx=0).stop 2024-05-08T23:40:44.878 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.879+0000 7fd839418700 1 -- 172.21.15.204:0/311971251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8341021e0 msgr2=0x7fd8341974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:44.878 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.879+0000 7fd839418700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8341021e0 0x7fd8341974d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd828003790 tx=0x7fd82800f690 comp rx=0 tx=0).stop 2024-05-08T23:40:44.878 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.880+0000 7fd839418700 1 -- 172.21.15.204:0/311971251 shutdown_connections 2024-05-08T23:40:44.878 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.880+0000 7fd839418700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd81c0389d0 0x7fd81c03ae90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.880+0000 7fd839418700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd834102b80 0x7fd83419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.880+0000 7fd839418700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd834100fe0 0x7fd834196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.880+0000 7fd839418700 1 --2- 172.21.15.204:0/311971251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8341021e0 0x7fd8341974d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.880+0000 7fd839418700 1 -- 172.21.15.204:0/311971251 >> 172.21.15.204:0/311971251 conn(0x7fd8340fc5a0 msgr2=0x7fd834105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:44.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.880+0000 7fd839418700 1 -- 172.21.15.204:0/311971251 shutdown_connections 2024-05-08T23:40:44.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:40:44.880+0000 7fd839418700 1 -- 172.21.15.204:0/311971251 wait complete. 2024-05-08T23:40:45.250 DEBUG:teuthology.orchestra.run.smithi160:mgr.y> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T23:40:45.253 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T23:40:45.255 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T23:40:45.255 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:40:45.255 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:40:45.277 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:40:45.278 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-05-08T23:40:45.284 INFO:journalctl@ceph.mgr.y.smithi160.stdout:-- Logs begin at Wed 2024-05-08 23:21:47 UTC. -- 2024-05-08T23:40:45.285 INFO:journalctl@ceph.mgr.z.smithi204.stdout:-- Logs begin at Wed 2024-05-08 23:21:56 UTC. -- 2024-05-08T23:40:45.340 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:40:45.340 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:45.340 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 183653 Links: 1 2024-05-08T23:40:45.340 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:45.340 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:45.340 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-08 23:40:04.824698739 +0000 2024-05-08T23:40:45.340 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-08 23:33:04.706825018 +0000 2024-05-08T23:40:45.340 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-08 23:33:04.706825018 +0000 2024-05-08T23:40:45.341 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-05-08T23:40:45.341 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:40:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-08T23:40:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-08T23:40:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000358087 s, 1.4 MB/s 2024-05-08T23:40:45.416 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:40:45.479 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-05-08T23:40:45.542 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:40:45.543 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 177863 Links: 1 2024-05-08T23:40:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-08 23:40:04.887698225 +0000 2024-05-08T23:40:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-08 23:33:05.093821985 +0000 2024-05-08T23:40:45.543 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-08 23:33:05.093821985 +0000 2024-05-08T23:40:45.543 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-05-08T23:40:45.543 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:40:45.615 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-08T23:40:45.616 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-08T23:40:45.616 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000229889 s, 2.2 MB/s 2024-05-08T23:40:45.617 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:40:45.680 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-05-08T23:40:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:40:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:45.744 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 178917 Links: 1 2024-05-08T23:40:45.744 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:45.744 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:45.744 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-08 23:40:04.948697728 +0000 2024-05-08T23:40:45.744 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-08 23:33:05.465819069 +0000 2024-05-08T23:40:45.744 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-08 23:33:05.465819069 +0000 2024-05-08T23:40:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-05-08T23:40:45.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:40:45.817 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-08T23:40:45.817 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-08T23:40:45.817 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000228268 s, 2.2 MB/s 2024-05-08T23:40:45.818 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:40:45.882 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-05-08T23:40:45.945 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:40:45.946 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:45.946 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 181821 Links: 1 2024-05-08T23:40:45.946 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:45.946 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:45.946 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-08 23:40:05.013697197 +0000 2024-05-08T23:40:45.946 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-08 23:33:05.837816152 +0000 2024-05-08T23:40:45.946 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-08 23:33:05.837816152 +0000 2024-05-08T23:40:45.946 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-05-08T23:40:45.946 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:40:46.020 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-08T23:40:46.020 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-08T23:40:46.020 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000232968 s, 2.2 MB/s 2024-05-08T23:40:46.021 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:40:46.084 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-08T23:40:46.084 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:40:46.105 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:40:46.105 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2024-05-08T23:40:46.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi160=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:46.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: Saving service mgr spec with placement smithi012=x;smithi160=y;smithi204=z;count:3 2024-05-08T23:40:46.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:46.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:46.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:46.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:46.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:46.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:40:46.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:40:46.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:40:46.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:46.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: Deploying daemon mgr.z on smithi204 2024-05-08T23:40:46.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:45 smithi204 ceph-mon[47373]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:46.164 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:40:46.165 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:46.165 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 182577 Links: 1 2024-05-08T23:40:46.165 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:46.165 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:46.165 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-08 23:40:33.565421262 +0000 2024-05-08T23:40:46.165 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-08 23:32:37.525173940 +0000 2024-05-08T23:40:46.165 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-08 23:32:37.525173940 +0000 2024-05-08T23:40:46.165 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-05-08T23:40:46.165 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:40:46.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi160=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:46.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: Saving service mgr spec with placement smithi012=x;smithi160=y;smithi204=z;count:3 2024-05-08T23:40:46.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:46.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:46.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:46.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:46.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:46.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:40:46.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:40:46.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:40:46.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:46.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: Deploying daemon mgr.z on smithi204 2024-05-08T23:40:46.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:45 smithi012 ceph-mon[42503]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi160=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: Saving service mgr spec with placement smithi012=x;smithi160=y;smithi204=z;count:3 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:40:46.231 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:46.232 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: Deploying daemon mgr.z on smithi204 2024-05-08T23:40:46.232 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:45 smithi160 ceph-mon[46152]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:46.233 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-08T23:40:46.233 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-08T23:40:46.233 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000237681 s, 2.2 MB/s 2024-05-08T23:40:46.234 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:40:46.294 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2024-05-08T23:40:46.356 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:40:46.356 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:46.357 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 184365 Links: 1 2024-05-08T23:40:46.357 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:46.357 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:46.357 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-08 23:40:33.565421262 +0000 2024-05-08T23:40:46.357 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-08 23:32:37.872168585 +0000 2024-05-08T23:40:46.357 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-08 23:32:37.872168585 +0000 2024-05-08T23:40:46.357 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-05-08T23:40:46.357 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:40:46.423 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-08T23:40:46.423 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-08T23:40:46.423 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000217713 s, 2.4 MB/s 2024-05-08T23:40:46.424 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:40:46.483 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2024-05-08T23:40:46.514 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:46 smithi204 systemd[1]: Starting Ceph mgr.z for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:40:46.542 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:40:46.542 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:46.542 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 179190 Links: 1 2024-05-08T23:40:46.542 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:46.542 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:46.542 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-08 23:40:33.565421262 +0000 2024-05-08T23:40:46.542 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-08 23:32:38.241162892 +0000 2024-05-08T23:40:46.542 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-08 23:32:38.241162892 +0000 2024-05-08T23:40:46.542 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-05-08T23:40:46.543 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:40:46.608 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-08T23:40:46.608 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-08T23:40:46.608 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000193524 s, 2.6 MB/s 2024-05-08T23:40:46.609 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:40:46.668 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2024-05-08T23:40:46.725 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:40:46.725 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:46.725 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 180954 Links: 1 2024-05-08T23:40:46.725 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:46.725 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:46.726 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-08 23:40:33.565421262 +0000 2024-05-08T23:40:46.726 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-08 23:32:38.602157321 +0000 2024-05-08T23:40:46.726 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-08 23:32:38.602157321 +0000 2024-05-08T23:40:46.726 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-05-08T23:40:46.726 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:40:46.792 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-08T23:40:46.792 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-08T23:40:46.792 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000235928 s, 2.2 MB/s 2024-05-08T23:40:46.793 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:40:46.852 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-08T23:40:46.853 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:40:46.871 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:40:46.871 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-05-08T23:40:46.929 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:40:46.929 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:46.929 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 184566 Links: 1 2024-05-08T23:40:46.929 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:46.930 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:46.930 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-08 23:40:46.334616299 +0000 2024-05-08T23:40:46.930 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-08 23:32:35.377818902 +0000 2024-05-08T23:40:46.930 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-08 23:32:35.377818902 +0000 2024-05-08T23:40:46.930 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-05-08T23:40:46.930 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:40:46.996 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:46 smithi204 ceph-mgr[49254]: -- 172.21.15.204:0/4038310085 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5611d56221a0 con 0x5611d55f8c00 2024-05-08T23:40:46.996 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:46 smithi204 bash[49216]: 6d491f20c8cfeb22a6c8a9c0d136c61af3806e1c3c3828573fd108a6b1953264 2024-05-08T23:40:46.996 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:46 smithi204 systemd[1]: Started Ceph mgr.z for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:40:46.999 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-08T23:40:46.999 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-08T23:40:46.999 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000253146 s, 2.0 MB/s 2024-05-08T23:40:47.001 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:40:47.060 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-05-08T23:40:47.117 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:40:47.117 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:47.117 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 177793 Links: 1 2024-05-08T23:40:47.117 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:47.117 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:47.118 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-08 23:40:46.334616299 +0000 2024-05-08T23:40:47.118 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-08 23:32:35.732814092 +0000 2024-05-08T23:40:47.118 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-08 23:32:35.732814092 +0000 2024-05-08T23:40:47.118 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-05-08T23:40:47.118 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:40:47.183 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:47.132+0000 7fb38e6f3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:40:47.186 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-08T23:40:47.186 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-08T23:40:47.186 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000204239 s, 2.5 MB/s 2024-05-08T23:40:47.187 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:40:47.244 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-05-08T23:40:47.264 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:47.231+0000 7fb38e6f3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:40:47.281 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:40:47.281 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:47.281 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 177886 Links: 1 2024-05-08T23:40:47.281 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:47.281 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:47.281 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-08 23:40:46.334616299 +0000 2024-05-08T23:40:47.281 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-08 23:32:36.066809566 +0000 2024-05-08T23:40:47.281 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-08 23:32:36.066809566 +0000 2024-05-08T23:40:47.282 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-05-08T23:40:47.282 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:40:47.348 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-08T23:40:47.348 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-08T23:40:47.348 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000204235 s, 2.5 MB/s 2024-05-08T23:40:47.350 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:40:47.407 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-05-08T23:40:47.465 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:40:47.465 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:40:47.465 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 177969 Links: 1 2024-05-08T23:40:47.465 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:40:47.465 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:40:47.465 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-08 23:40:46.334616299 +0000 2024-05-08T23:40:47.465 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-08 23:32:36.410804905 +0000 2024-05-08T23:40:47.465 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-08 23:32:36.410804905 +0000 2024-05-08T23:40:47.465 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-05-08T23:40:47.466 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:40:47.514 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:47.323+0000 7fb38e6f3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:40:47.514 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:47.413+0000 7fb38e6f3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:40:47.530 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-08T23:40:47.530 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-08T23:40:47.530 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000228162 s, 2.2 MB/s 2024-05-08T23:40:47.532 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:40:47.589 INFO:tasks.cephadm:Deploying osd.0 on smithi012 with /dev/vg_nvme/lv_4... 2024-05-08T23:40:47.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T23:40:47.843 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:40:47.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:47.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:47.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:47.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:40:47.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:40:47.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:40:47.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:47.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:47 smithi160 ceph-mon[46152]: Deploying daemon mgr.y on smithi160 2024-05-08T23:40:47.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:47 smithi160 ceph-mon[46152]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:47.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:47.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:47.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:47.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:40:47.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:40:47.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:40:47.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:47.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:47 smithi012 ceph-mon[42503]: Deploying daemon mgr.y on smithi160 2024-05-08T23:40:47.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:47 smithi012 ceph-mon[42503]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:48.065 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:48.065 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:48.065 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:48.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:40:48.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:40:48.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:40:48.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:48.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:47 smithi204 ceph-mon[47373]: Deploying daemon mgr.y on smithi160 2024-05-08T23:40:48.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:47 smithi204 ceph-mon[47373]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:48.267 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:48 smithi160 systemd[1]: Starting Ceph mgr.y for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:40:48.318 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:48 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:48.068+0000 7fb38e6f3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:40:48.675 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:48 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:48.322+0000 7fb38e6f3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:40:48.675 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:48 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:48.408+0000 7fb38e6f3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:40:48.675 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:48 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:48.517+0000 7fb38e6f3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:40:48.953 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:48 smithi160 ceph-mgr[47653]: -- 172.21.15.160:0/1745934748 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579a6bda1a0 con 0x5579a6bb0c00 2024-05-08T23:40:48.954 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:48 smithi160 bash[47615]: c45bed0283f30c6e4076e3302b6a11fd42f208dd34e900fbe2a07e71a980d2ee 2024-05-08T23:40:48.954 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:48 smithi160 systemd[1]: Started Ceph mgr.y for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:40:49.014 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:48 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:48.678+0000 7fb38e6f3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:40:49.014 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:48 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:48.811+0000 7fb38e6f3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:40:49.014 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:48 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:48.902+0000 7fb38e6f3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:40:49.061 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:48 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:48.957+0000 7eff9049d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:40:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T23:40:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T23:40:49.206 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-08T23:40:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-08T23:40:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012311 s, 852 MB/s 2024-05-08T23:40:49.206 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-08T23:40:49.247 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:49 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:49.064+0000 7eff9049d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:40:49.247 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:49 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:49.162+0000 7eff9049d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:40:49.517 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:49 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:49.251+0000 7eff9049d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:40:49.609 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:49 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:49.310+0000 7fb38e6f3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:40:49.626 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_4 2024-05-08T23:40:49.886 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:40:49.913 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:49 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:49.720+0000 7fb38e6f3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:40:49.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:49.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:49.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:49.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:49.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:49.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:49.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:49.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:49.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.920 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.920 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.920 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.920 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.920 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:49.920 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:49.920 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.920 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.920 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.921 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:49.921 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:49.921 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:49.921 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:50.171 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:49 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:49.923+0000 7eff9049d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:40:50.264 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:49 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:49.916+0000 7fb38e6f3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:40:50.264 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:50 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:50.011+0000 7fb38e6f3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:40:50.517 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:50.175+0000 7eff9049d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:40:50.517 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:50.260+0000 7eff9049d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:40:50.517 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:50.369+0000 7eff9049d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:40:50.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:50 smithi012 ceph-mon[42503]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T23:40:50.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:40:50.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:40:50.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:50.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:50 smithi012 ceph-mon[42503]: Reconfiguring daemon mgr.x on smithi012 2024-05-08T23:40:50.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:51.014 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:50 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:50.780+0000 7fb38e6f3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:40:51.014 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:50 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:50.878+0000 7fb38e6f3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:40:51.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:50 smithi204 ceph-mon[47373]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T23:40:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:40:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:40:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:50 smithi204 ceph-mon[47373]: Reconfiguring daemon mgr.x on smithi012 2024-05-08T23:40:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:51.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:50 smithi160 ceph-mon[46152]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T23:40:51.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:40:51.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:40:51.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:51.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:50 smithi160 ceph-mon[46152]: Reconfiguring daemon mgr.x on smithi012 2024-05-08T23:40:51.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:51.018 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:50.526+0000 7eff9049d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:40:51.018 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:50.654+0000 7eff9049d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:40:51.018 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:50.739+0000 7eff9049d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:40:51.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.232+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3459946819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee4101000 msgr2=0x7f1ee4101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:51.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.232+0000 7f1eeaa1d700 1 --2- 172.21.15.12:0/3459946819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee4101000 0x7f1ee4101420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40099c0 tx=0x7f1ed4009cd0 comp rx=0 tx=0).stop 2024-05-08T23:40:51.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.232+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3459946819 shutdown_connections 2024-05-08T23:40:51.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.232+0000 7f1eeaa1d700 1 --2- 172.21.15.12:0/3459946819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ee4102ba0 0x7f1ee4109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:51.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.233+0000 7f1eeaa1d700 1 --2- 172.21.15.12:0/3459946819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee4101000 0x7f1ee4101420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:51.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.233+0000 7f1eeaa1d700 1 --2- 172.21.15.12:0/3459946819 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ee4102200 0x7f1ee4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:51.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.233+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3459946819 >> 172.21.15.12:0/3459946819 conn(0x7f1ee40fc5a0 msgr2=0x7f1ee40fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:51.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.233+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3459946819 shutdown_connections 2024-05-08T23:40:51.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.233+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3459946819 wait complete. 2024-05-08T23:40:51.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.233+0000 7f1eeaa1d700 1 Processor -- start 2024-05-08T23:40:51.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.233+0000 7f1eeaa1d700 1 -- start start 2024-05-08T23:40:51.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1eeaa1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee4101000 0x7f1ee4074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:51.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1eeaa1d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ee4102200 0x7f1ee4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:51.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1eeaa1d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ee4102ba0 0x7f1ee40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:51.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1eeaa1d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee4074070 con 0x7f1ee4101000 2024-05-08T23:40:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1eeaa1d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40741e0 con 0x7f1ee4102200 2024-05-08T23:40:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1eeaa1d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee41a0fb0 con 0x7f1ee4102ba0 2024-05-08T23:40:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee4101000 0x7f1ee4074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee4101000 0x7f1ee4074cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43570/0 (socket says 172.21.15.12:43570) 2024-05-08T23:40:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee3fff700 1 -- 172.21.15.12:0/3562542355 learned_addr learned my addr 172.21.15.12:0/3562542355 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:40:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee37fe700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ee4102200 0x7f1ee4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee8fba700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ee4102ba0 0x7f1ee40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee3fff700 1 -- 172.21.15.12:0/3562542355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ee4102ba0 msgr2=0x7f1ee40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee3fff700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ee4102ba0 0x7f1ee40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee3fff700 1 -- 172.21.15.12:0/3562542355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ee4102200 msgr2=0x7f1ee4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee3fff700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ee4102200 0x7f1ee4073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee3fff700 1 -- 172.21.15.12:0/3562542355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed40095f0 con 0x7f1ee4101000 2024-05-08T23:40:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee37fe700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ee4102200 0x7f1ee4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:40:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.234+0000 7f1ee3fff700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee4101000 0x7f1ee4074cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4006010 tx=0x7f1ed400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.235+0000 7f1ee17fa700 1 -- 172.21.15.12:0/3562542355 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed401d070 con 0x7f1ee4101000 2024-05-08T23:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.235+0000 7f1ee17fa700 1 -- 172.21.15.12:0/3562542355 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ed4005020 con 0x7f1ee4101000 2024-05-08T23:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.235+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3562542355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee41a11e0 con 0x7f1ee4101000 2024-05-08T23:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.235+0000 7f1ee17fa700 1 -- 172.21.15.12:0/3562542355 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4004b10 con 0x7f1ee4101000 2024-05-08T23:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.235+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3562542355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee41a15f0 con 0x7f1ee4101000 2024-05-08T23:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.236+0000 7f1ee17fa700 1 -- 172.21.15.12:0/3562542355 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f1ed4004c70 con 0x7f1ee4101000 2024-05-08T23:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.236+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3562542355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee41084e0 con 0x7f1ee4101000 2024-05-08T23:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.236+0000 7f1ee17fa700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f1ecc0389d0 0x7f1ecc03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.236+0000 7f1ee17fa700 1 -- 172.21.15.12:0/3562542355 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1ed404bc50 con 0x7f1ee4101000 2024-05-08T23:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.236+0000 7f1ee37fe700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f1ecc0389d0 0x7f1ecc03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:51.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.236+0000 7f1ee37fe700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f1ecc0389d0 0x7f1ecc03ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8011490 tx=0x7f1ed8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:51.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.242+0000 7f1ee17fa700 1 -- 172.21.15.12:0/3562542355 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ed4016990 con 0x7f1ee4101000 2024-05-08T23:40:51.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:51.465+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3562542355 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ee402cdf0 con 0x7f1ecc0389d0 2024-05-08T23:40:51.506 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:51 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:51.116+0000 7eff9049d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:40:51.514 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:51 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:51.028+0000 7fb38e6f3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:40:51.766 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:51 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:51.510+0000 7eff9049d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:40:51.767 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:51 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:51.703+0000 7eff9049d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:40:52.007 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:51 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:51.670+0000 7fb38e6f3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:40:52.007 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:51 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:51.855+0000 7fb38e6f3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:40:52.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:40:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:40:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:52.264 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:52 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:52.051+0000 7fb38e6f3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:40:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:40:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:40:52.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:52.266 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:51 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:51.795+0000 7eff9049d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:40:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:40:53.017 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:52 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:52.555+0000 7eff9049d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:40:53.017 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:52 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:52.651+0000 7eff9049d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:40:53.017 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:52 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:52.797+0000 7eff9049d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:40:53.264 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:52 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:52.988+0000 7fb38e6f3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:40:53.556 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:53 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:53.275+0000 7fb38e6f3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:40:53.556 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:53 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:53.364+0000 7fb38e6f3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:40:53.766 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:53 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:53.426+0000 7eff9049d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:40:53.767 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:53 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:53.607+0000 7eff9049d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:40:54.014 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:40:53 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:40:53.559+0000 7fb38e6f3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:40:54.266 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:53 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:53.804+0000 7eff9049d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:40:54.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:54.459+0000 7f1ee17fa700 1 -- 172.21.15.12:0/3562542355 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 14) v1 ==== 90707+0+0 (secure 0 0 0) 0x7f1ed4005190 con 0x7f1ee4101000 2024-05-08T23:40:54.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:54.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/513623125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6051d38-71b4-4946-b055-5ccf985ceed4"}]: dispatch 2024-05-08T23:40:54.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/513623125' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6051d38-71b4-4946-b055-5ccf985ceed4"}]': finished 2024-05-08T23:40:54.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T23:40:54.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:40:54.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: Standby manager daemon z started 2024-05-08T23:40:54.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:40:54.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:40:54.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:40:54.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:40:54.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:54 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1771933225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:40:54.734 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:54.734 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/513623125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6051d38-71b4-4946-b055-5ccf985ceed4"}]: dispatch 2024-05-08T23:40:54.734 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/513623125' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6051d38-71b4-4946-b055-5ccf985ceed4"}]': finished 2024-05-08T23:40:54.734 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T23:40:54.734 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:40:54.734 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: Standby manager daemon z started 2024-05-08T23:40:54.734 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:40:54.735 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:40:54.735 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:40:54.735 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:40:54.735 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:54 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1771933225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:40:54.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:54.805 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/513623125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6051d38-71b4-4946-b055-5ccf985ceed4"}]: dispatch 2024-05-08T23:40:54.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/513623125' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6051d38-71b4-4946-b055-5ccf985ceed4"}]': finished 2024-05-08T23:40:54.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T23:40:54.807 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:40:54.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: Standby manager daemon z started 2024-05-08T23:40:54.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:40:54.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:40:54.809 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:40:54.809 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.204:0/1788023131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:40:54.809 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:54 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1771933225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:40:55.016 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:54 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:54.737+0000 7eff9049d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:40:55.300 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:55 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:55.024+0000 7eff9049d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:40:55.301 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:55 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:55.112+0000 7eff9049d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:40:55.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:40:55.468+0000 7f1ee17fa700 1 -- 172.21.15.12:0/3562542355 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f1ed4016bc0 con 0x7f1ee4101000 2024-05-08T23:40:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:55 smithi204 ceph-mon[47373]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-08T23:40:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:40:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:55 smithi204 ceph-mon[47373]: Standby manager daemon y started 2024-05-08T23:40:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:55 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:40:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:55 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:40:55.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:55 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:40:55.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:55 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:40:55.767 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:40:55 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:40:55.304+0000 7eff9049d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:40:55.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:55 smithi160 ceph-mon[46152]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-08T23:40:55.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:40:55.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:55 smithi160 ceph-mon[46152]: Standby manager daemon y started 2024-05-08T23:40:55.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:55 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:40:55.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:55 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:40:55.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:55 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:40:55.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:55 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:40:55.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:55 smithi012 ceph-mon[42503]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-08T23:40:55.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:40:55.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:55 smithi012 ceph-mon[42503]: Standby manager daemon y started 2024-05-08T23:40:55.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:55 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:40:55.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:55 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:40:55.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:55 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:40:55.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:55 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.160:0/3293020638' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:40:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:56 smithi204 ceph-mon[47373]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:56 smithi204 ceph-mon[47373]: mgrmap e15: x(active, since 60s), standbys: z, y 2024-05-08T23:40:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:56 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:40:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:56 smithi160 ceph-mon[46152]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:56 smithi160 ceph-mon[46152]: mgrmap e15: x(active, since 60s), standbys: z, y 2024-05-08T23:40:56.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:56 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:40:56.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:56 smithi012 ceph-mon[42503]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:56.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:56 smithi012 ceph-mon[42503]: mgrmap e15: x(active, since 60s), standbys: z, y 2024-05-08T23:40:56.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:40:58.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:40:58 smithi204 ceph-mon[47373]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:58.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:40:58 smithi160 ceph-mon[46152]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:40:58.878 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:40:58 smithi012 ceph-mon[42503]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:00.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:00 smithi012 ceph-mon[42503]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:00.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T23:41:00.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:00 smithi204 ceph-mon[47373]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T23:41:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:00 smithi160 ceph-mon[46152]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T23:41:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:01.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:01 smithi204 ceph-mon[47373]: Deploying daemon osd.0 on smithi012 2024-05-08T23:41:01.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:01 smithi160 ceph-mon[46152]: Deploying daemon osd.0 on smithi012 2024-05-08T23:41:01.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:01 smithi012 ceph-mon[42503]: Deploying daemon osd.0 on smithi012 2024-05-08T23:41:02.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:02 smithi012 ceph-mon[42503]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:02.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:02 smithi204 ceph-mon[47373]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:02.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:02 smithi160 ceph-mon[46152]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:04.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:04 smithi012 ceph-mon[42503]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:04.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:04.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:04.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:04.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:04 smithi204 ceph-mon[47373]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:04.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:04.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:04.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:04.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:04 smithi160 ceph-mon[46152]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:04.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:04.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:04.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:05.860 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 0 on host 'smithi012' 2024-05-08T23:41:05.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.863+0000 7f1ee17fa700 1 -- 172.21.15.12:0/3562542355 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1ee402cdf0 con 0x7f1ecc0389d0 2024-05-08T23:41:05.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.866+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3562542355 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f1ecc0389d0 msgr2=0x7f1ecc03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:05.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.866+0000 7f1eeaa1d700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f1ecc0389d0 0x7f1ecc03ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8011490 tx=0x7f1ed8009260 comp rx=0 tx=0).stop 2024-05-08T23:41:05.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.866+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3562542355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee4101000 msgr2=0x7f1ee4074cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:05.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.866+0000 7f1eeaa1d700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee4101000 0x7f1ee4074cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4006010 tx=0x7f1ed400f690 comp rx=0 tx=0).stop 2024-05-08T23:41:05.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.867+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3562542355 shutdown_connections 2024-05-08T23:41:05.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.867+0000 7f1eeaa1d700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ee4102ba0 0x7f1ee40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:05.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.867+0000 7f1eeaa1d700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee4101000 0x7f1ee4074cf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:05.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.867+0000 7f1eeaa1d700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f1ecc0389d0 0x7f1ecc03ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:05.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.867+0000 7f1eeaa1d700 1 --2- 172.21.15.12:0/3562542355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ee4102200 0x7f1ee4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:05.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.867+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3562542355 >> 172.21.15.12:0/3562542355 conn(0x7f1ee40fc5a0 msgr2=0x7f1ee4105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:05.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.867+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3562542355 shutdown_connections 2024-05-08T23:41:05.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:05.867+0000 7f1eeaa1d700 1 -- 172.21.15.12:0/3562542355 wait complete. 2024-05-08T23:41:06.198 DEBUG:teuthology.orchestra.run.smithi012:osd.0> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T23:41:06.200 INFO:tasks.cephadm:Deploying osd.1 on smithi160 with /dev/vg_nvme/lv_4... 2024-05-08T23:41:06.200 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T23:41:06.239 INFO:journalctl@ceph.osd.0.smithi012.stdout:-- Logs begin at Wed 2024-05-08 23:21:09 UTC. -- 2024-05-08T23:41:06.383 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.383 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.383 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:06 smithi012 ceph-mon[42503]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:06.383 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:06.383 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:06.383 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.384 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.384 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.384 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:06.431 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:41:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:06 smithi204 ceph-mon[47373]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:06.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:06 smithi160 ceph-mon[46152]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:07.721 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T23:41:07.721 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T23:41:07.722 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-08T23:41:07.722 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-08T23:41:07.722 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112253 s, 934 MB/s 2024-05-08T23:41:07.722 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-08T23:41:07.916 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:07 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-0[60992]: 2024-05-08T23:41:07.568+0000 7fbac11287c0 -1 osd.0 0 log_to_monitors true 2024-05-08T23:41:08.163 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_4 2024-05-08T23:41:08.380 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:41:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:08 smithi012 ceph-mon[42503]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:08 smithi012 ceph-mon[42503]: from='osd.0 [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:41:08.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:08 smithi012 ceph-mon[42503]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:41:08.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:08 smithi204 ceph-mon[47373]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:08.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:08 smithi204 ceph-mon[47373]: from='osd.0 [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:41:08.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:08 smithi204 ceph-mon[47373]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:41:08.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:08 smithi160 ceph-mon[46152]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:08.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:08 smithi160 ceph-mon[46152]: from='osd.0 [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:41:08.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:08 smithi160 ceph-mon[46152]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:41:09.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.465+0000 7fdf21a93700 1 -- 172.21.15.160:0/1530967321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c102200 msgr2=0x7fdf1c102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.465+0000 7fdf21a93700 1 --2- 172.21.15.160:0/1530967321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c102200 0x7fdf1c102660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0099c0 tx=0x7fdf0c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.465+0000 7fdf21a93700 1 -- 172.21.15.160:0/1530967321 shutdown_connections 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.465+0000 7fdf21a93700 1 --2- 172.21.15.160:0/1530967321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf1c102ba0 0x7fdf1c109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.465+0000 7fdf21a93700 1 --2- 172.21.15.160:0/1530967321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf1c101000 0x7fdf1c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.465+0000 7fdf21a93700 1 --2- 172.21.15.160:0/1530967321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c102200 0x7fdf1c102660 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.465+0000 7fdf21a93700 1 -- 172.21.15.160:0/1530967321 >> 172.21.15.160:0/1530967321 conn(0x7fdf1c0fc5c0 msgr2=0x7fdf1c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.465+0000 7fdf21a93700 1 -- 172.21.15.160:0/1530967321 shutdown_connections 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.465+0000 7fdf21a93700 1 -- 172.21.15.160:0/1530967321 wait complete. 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf21a93700 1 Processor -- start 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf21a93700 1 -- start start 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf21a93700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf1c101000 0x7fdf1c074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf21a93700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf1c102200 0x7fdf1c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf21a93700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c102ba0 0x7fdf1c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf21a93700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c074070 con 0x7fdf1c101000 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf21a93700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0741e0 con 0x7fdf1c102ba0 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf21a93700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c1a0fb0 con 0x7fdf1c102200 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c102ba0 0x7fdf1c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c102ba0 0x7fdf1c0738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52948/0 (socket says 172.21.15.160:52948) 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1bfff700 1 -- 172.21.15.160:0/2739753958 learned_addr learned my addr 172.21.15.160:0/2739753958 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1affd700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf1c102200 0x7fdf1c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1b7fe700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf1c101000 0x7fdf1c074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1bfff700 1 -- 172.21.15.160:0/2739753958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf1c102200 msgr2=0x7fdf1c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1bfff700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf1c102200 0x7fdf1c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1bfff700 1 -- 172.21.15.160:0/2739753958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf1c101000 msgr2=0x7fdf1c074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1bfff700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf1c101000 0x7fdf1c074cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1bfff700 1 -- 172.21.15.160:0/2739753958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0095f0 con 0x7fdf1c102ba0 2024-05-08T23:41:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.466+0000 7fdf1bfff700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c102ba0 0x7fdf1c0738d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf14007ee0 tx=0x7fdf1400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.467+0000 7fdf18ff9700 1 -- 172.21.15.160:0/2739753958 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14017070 con 0x7fdf1c102ba0 2024-05-08T23:41:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.467+0000 7fdf18ff9700 1 -- 172.21.15.160:0/2739753958 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf1400f040 con 0x7fdf1c102ba0 2024-05-08T23:41:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.467+0000 7fdf18ff9700 1 -- 172.21.15.160:0/2739753958 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14005740 con 0x7fdf1c102ba0 2024-05-08T23:41:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.467+0000 7fdf21a93700 1 -- 172.21.15.160:0/2739753958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c1a1240 con 0x7fdf1c102ba0 2024-05-08T23:41:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.467+0000 7fdf21a93700 1 -- 172.21.15.160:0/2739753958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c1a1740 con 0x7fdf1c102ba0 2024-05-08T23:41:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.468+0000 7fdf21a93700 1 -- 172.21.15.160:0/2739753958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c1084e0 con 0x7fdf1c102ba0 2024-05-08T23:41:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.469+0000 7fdf18ff9700 1 -- 172.21.15.160:0/2739753958 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7fdf1400dc60 con 0x7fdf1c102ba0 2024-05-08T23:41:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.469+0000 7fdf18ff9700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf040a1110 0x7fdf040a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.469+0000 7fdf18ff9700 1 -- 172.21.15.160:0/2739753958 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7fdf140ca470 con 0x7fdf1c102ba0 2024-05-08T23:41:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.469+0000 7fdf1b7fe700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf040a1110 0x7fdf040a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.470+0000 7fdf1b7fe700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf040a1110 0x7fdf040a35d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005950 tx=0x7fdf100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:09.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.474+0000 7fdf18ff9700 1 -- 172.21.15.160:0/2739753958 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdf14095790 con 0x7fdf1c102ba0 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='osd.0 [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:09.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:09.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:09 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.666 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-0[60992]: 2024-05-08T23:41:09.272+0000 7fbab5f8b700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T23:41:09.666 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-0[60992]: 2024-05-08T23:41:09.276+0000 7fbab2581700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='osd.0 [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:09.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:09 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:09.676+0000 7fdf21a93700 1 -- 172.21.15.160:0/2739753958 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf1c02ce10 con 0x7fdf040a1110 2024-05-08T23:41:09.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T23:41:09.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T23:41:09.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:09.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='osd.0 [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:41:09.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:41:09.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:41:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:09 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:10.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:10 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi012 2024-05-08T23:41:10.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:10 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi012 to 17118M 2024-05-08T23:41:10.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:10 smithi160 ceph-mon[46152]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:10.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:10 smithi160 ceph-mon[46152]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:41:10.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:10 smithi160 ceph-mon[46152]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T23:41:10.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:10 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:10.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:10 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:10.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:10 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:41:10.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:10 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:41:10.342 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:10 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:10.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:10 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi012 2024-05-08T23:41:10.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:10 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi012 to 17118M 2024-05-08T23:41:10.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:10 smithi012 ceph-mon[42503]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:10.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:10 smithi012 ceph-mon[42503]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:41:10.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:10 smithi012 ceph-mon[42503]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T23:41:10.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:10 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:10.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:10 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:10.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:10 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:41:10.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:10 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:41:10.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:10 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:10.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:10 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi012 2024-05-08T23:41:10.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:10 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi012 to 17118M 2024-05-08T23:41:10.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:10 smithi204 ceph-mon[47373]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:10.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:10 smithi204 ceph-mon[47373]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:41:10.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:10 smithi204 ceph-mon[47373]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T23:41:10.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:10 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:10.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:10 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:10.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:10 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:41:10.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:10 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:41:10.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:10 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:11.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:11 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:41:11.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:11 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:41:11.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:11 smithi012 ceph-mon[42503]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:41:11.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:11 smithi012 ceph-mon[42503]: osd.0 [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] boot 2024-05-08T23:41:11.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:11 smithi012 ceph-mon[42503]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T23:41:11.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:11 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:41:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:11 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:41:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:11 smithi204 ceph-mon[47373]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:41:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:11 smithi204 ceph-mon[47373]: osd.0 [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] boot 2024-05-08T23:41:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:11 smithi204 ceph-mon[47373]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T23:41:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:11 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:41:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:11 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:41:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:11 smithi160 ceph-mon[46152]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:41:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:11 smithi160 ceph-mon[46152]: osd.0 [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] boot 2024-05-08T23:41:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:11 smithi160 ceph-mon[46152]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T23:41:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:41:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:12 smithi012 ceph-mon[42503]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:12 smithi012 ceph-mon[42503]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T23:41:12.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:12 smithi012 ceph-mon[42503]: from='client.? 172.21.15.160:0/2296049530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83b3995f-c70f-49b5-a4eb-0969ce93f22b"}]: dispatch 2024-05-08T23:41:12.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:12 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83b3995f-c70f-49b5-a4eb-0969ce93f22b"}]: dispatch 2024-05-08T23:41:12.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:12 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83b3995f-c70f-49b5-a4eb-0969ce93f22b"}]': finished 2024-05-08T23:41:12.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:12 smithi012 ceph-mon[42503]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T23:41:12.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:12.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:12 smithi012 ceph-mon[42503]: from='client.? 172.21.15.160:0/4200572530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:41:12.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:12 smithi204 ceph-mon[47373]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:12.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:12 smithi204 ceph-mon[47373]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T23:41:12.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:12 smithi204 ceph-mon[47373]: from='client.? 172.21.15.160:0/2296049530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83b3995f-c70f-49b5-a4eb-0969ce93f22b"}]: dispatch 2024-05-08T23:41:12.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:12 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83b3995f-c70f-49b5-a4eb-0969ce93f22b"}]: dispatch 2024-05-08T23:41:12.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:12 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83b3995f-c70f-49b5-a4eb-0969ce93f22b"}]': finished 2024-05-08T23:41:12.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:12 smithi204 ceph-mon[47373]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T23:41:12.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:12.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:12 smithi204 ceph-mon[47373]: from='client.? 172.21.15.160:0/4200572530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:41:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:12 smithi160 ceph-mon[46152]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:41:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:12 smithi160 ceph-mon[46152]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T23:41:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:12 smithi160 ceph-mon[46152]: from='client.? 172.21.15.160:0/2296049530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83b3995f-c70f-49b5-a4eb-0969ce93f22b"}]: dispatch 2024-05-08T23:41:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:12 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83b3995f-c70f-49b5-a4eb-0969ce93f22b"}]: dispatch 2024-05-08T23:41:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:12 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83b3995f-c70f-49b5-a4eb-0969ce93f22b"}]': finished 2024-05-08T23:41:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:12 smithi160 ceph-mon[46152]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T23:41:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:12 smithi160 ceph-mon[46152]: from='client.? 172.21.15.160:0/4200572530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:41:14.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:14 smithi012 ceph-mon[42503]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:14.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:14 smithi204 ceph-mon[47373]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:14.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:14 smithi160 ceph-mon[46152]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:16.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:16 smithi160 ceph-mon[46152]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:16.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:16 smithi012 ceph-mon[42503]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:16 smithi204 ceph-mon[47373]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:18.386 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:18 smithi160 ceph-mon[46152]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:18.386 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T23:41:18.386 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:18 smithi012 ceph-mon[42503]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T23:41:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:18.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:18 smithi204 ceph-mon[47373]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:18.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T23:41:18.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:19.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:19 smithi160 ceph-mon[46152]: Deploying daemon osd.1 on smithi160 2024-05-08T23:41:19.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:19 smithi012 ceph-mon[42503]: Deploying daemon osd.1 on smithi160 2024-05-08T23:41:19.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:19 smithi204 ceph-mon[47373]: Deploying daemon osd.1 on smithi160 2024-05-08T23:41:20.566 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:20 smithi160 ceph-mon[46152]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:20.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:20 smithi012 ceph-mon[42503]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:20.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:20 smithi204 ceph-mon[47373]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:22 smithi204 ceph-mon[47373]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:22 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:22 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:22 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:22.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:22 smithi160 ceph-mon[46152]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:22 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:22 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:22 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:22.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:22 smithi012 ceph-mon[42503]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:22.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:22.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:22.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:23.589 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 1 on host 'smithi160' 2024-05-08T23:41:23.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.592+0000 7fdf18ff9700 1 -- 172.21.15.160:0/2739753958 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdf1c02ce10 con 0x7fdf040a1110 2024-05-08T23:41:23.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.594+0000 7fdf21a93700 1 -- 172.21.15.160:0/2739753958 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf040a1110 msgr2=0x7fdf040a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:23.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.594+0000 7fdf21a93700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf040a1110 0x7fdf040a35d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005950 tx=0x7fdf100058e0 comp rx=0 tx=0).stop 2024-05-08T23:41:23.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.594+0000 7fdf21a93700 1 -- 172.21.15.160:0/2739753958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c102ba0 msgr2=0x7fdf1c0738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:23.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.594+0000 7fdf21a93700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c102ba0 0x7fdf1c0738d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf14007ee0 tx=0x7fdf1400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:41:23.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.594+0000 7fdf21a93700 1 -- 172.21.15.160:0/2739753958 shutdown_connections 2024-05-08T23:41:23.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.594+0000 7fdf21a93700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf040a1110 0x7fdf040a35d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:23.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.594+0000 7fdf21a93700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c102ba0 0x7fdf1c0738d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:23.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.594+0000 7fdf21a93700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf1c101000 0x7fdf1c074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:23.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.594+0000 7fdf21a93700 1 --2- 172.21.15.160:0/2739753958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf1c102200 0x7fdf1c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:23.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.594+0000 7fdf21a93700 1 -- 172.21.15.160:0/2739753958 >> 172.21.15.160:0/2739753958 conn(0x7fdf1c0fc5c0 msgr2=0x7fdf1c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:23.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.595+0000 7fdf21a93700 1 -- 172.21.15.160:0/2739753958 shutdown_connections 2024-05-08T23:41:23.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:41:23.595+0000 7fdf21a93700 1 -- 172.21.15.160:0/2739753958 wait complete. 2024-05-08T23:41:23.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:23 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:23 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:23 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:23 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:23 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:23 smithi160 ceph-mon[46152]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:23 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:23 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:23 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:23.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:23 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:23.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:23 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:23.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:23 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:23.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:23 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:23.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:23 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:23.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:23 smithi012 ceph-mon[42503]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:23.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:23 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:23.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:23 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:23.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:23 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:23.988 DEBUG:teuthology.orchestra.run.smithi160:osd.1> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T23:41:23.990 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-05-08T23:41:23.990 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T23:41:24.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:23 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:24.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:23 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:24.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:23 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:24.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:23 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:24.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:23 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:24.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:23 smithi204 ceph-mon[47373]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:24.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:23 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:24.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:23 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:24.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:23 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:24.022 INFO:journalctl@ceph.osd.1.smithi160.stdout:-- Logs begin at Wed 2024-05-08 23:21:47 UTC. -- 2024-05-08T23:41:24.218 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:41:25.523 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T23:41:25.524 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T23:41:25.524 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-08T23:41:25.524 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-08T23:41:25.524 INFO:teuthology.orchestra.run.smithi204.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110342 s, 950 MB/s 2024-05-08T23:41:25.524 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 2024-05-08T23:41:25.524 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-08T23:41:25.767 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:25 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-1[51153]: 2024-05-08T23:41:25.271+0000 7fac206757c0 -1 osd.1 0 log_to_monitors true 2024-05-08T23:41:25.905 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-05-08T23:41:26.162 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:41:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:26 smithi204 ceph-mon[47373]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:26 smithi204 ceph-mon[47373]: from='osd.1 [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T23:41:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:26 smithi204 ceph-mon[47373]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T23:41:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:41:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:26 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:26 smithi160 ceph-mon[46152]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:26.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:26 smithi160 ceph-mon[46152]: from='osd.1 [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T23:41:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:26 smithi160 ceph-mon[46152]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T23:41:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:26 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:26 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:26 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:41:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:26 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:26 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:26 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:26 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:26 smithi012 ceph-mon[42503]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:26 smithi012 ceph-mon[42503]: from='osd.1 [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T23:41:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:26 smithi012 ceph-mon[42503]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T23:41:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:26 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:26 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:26 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:41:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:26 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:26 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:26 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:26 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 -- 172.21.15.204:0/1134732082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6490101060 msgr2=0x7f64901014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 --2- 172.21.15.204:0/1134732082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6490101060 0x7f64901014e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f647c009990 tx=0x7f647c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 -- 172.21.15.204:0/1134732082 shutdown_connections 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 --2- 172.21.15.204:0/1134732082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6490101a20 0x7f6490108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 --2- 172.21.15.204:0/1134732082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64900fff00 0x7f6490100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 --2- 172.21.15.204:0/1134732082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6490101060 0x7f64901014e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 -- 172.21.15.204:0/1134732082 >> 172.21.15.204:0/1134732082 conn(0x7f64900fb4c0 msgr2=0x7f64900fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 -- 172.21.15.204:0/1134732082 shutdown_connections 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 -- 172.21.15.204:0/1134732082 wait complete. 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 Processor -- start 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.119+0000 7f64971f3700 1 -- start start 2024-05-08T23:41:27.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f64971f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64900fff00 0x7f6490196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f64971f3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6490101060 0x7f64901974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f64971f3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6490101a20 0x7f649019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f64971f3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649019ea60 con 0x7f64900fff00 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f64971f3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649019eba0 con 0x7f6490101060 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f64971f3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649019ed30 con 0x7f6490101a20 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6495790700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6490101a20 0x7f649019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6495790700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6490101a20 0x7f649019e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:51398/0 (socket says 172.21.15.204:51398) 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6495790700 1 -- 172.21.15.204:0/1817149896 learned_addr learned my addr 172.21.15.204:0/1817149896 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f648ffff700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6490101060 0x7f64901974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6494f8f700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64900fff00 0x7f6490196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6495790700 1 -- 172.21.15.204:0/1817149896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6490101060 msgr2=0x7f64901974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6495790700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6490101060 0x7f64901974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:27.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6495790700 1 -- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64900fff00 msgr2=0x7f6490196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:27.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6495790700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64900fff00 0x7f6490196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:27.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6495790700 1 -- 172.21.15.204:0/1817149896 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c0095f0 con 0x7f6490101a20 2024-05-08T23:41:27.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6494f8f700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64900fff00 0x7f6490196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:41:27.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f6495790700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6490101a20 0x7f649019e490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f648800b9a0 tx=0x7f648800bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:27.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f648dffb700 1 -- 172.21.15.204:0/1817149896 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648800c930 con 0x7f6490101a20 2024-05-08T23:41:27.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f648dffb700 1 -- 172.21.15.204:0/1817149896 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6488014440 con 0x7f6490101a20 2024-05-08T23:41:27.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f64971f3700 1 -- 172.21.15.204:0/1817149896 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649019efb0 con 0x7f6490101a20 2024-05-08T23:41:27.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.120+0000 7f648dffb700 1 -- 172.21.15.204:0/1817149896 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64880075d0 con 0x7f6490101a20 2024-05-08T23:41:27.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.121+0000 7f64971f3700 1 -- 172.21.15.204:0/1817149896 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649019f480 con 0x7f6490101a20 2024-05-08T23:41:27.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.122+0000 7f64971f3700 1 -- 172.21.15.204:0/1817149896 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649004ec60 con 0x7f6490101a20 2024-05-08T23:41:27.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.122+0000 7f648dffb700 1 -- 172.21.15.204:0/1817149896 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f64880145b0 con 0x7f6490101a20 2024-05-08T23:41:27.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.123+0000 7f648dffb700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f64800a1110 0x7f64800a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:27.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.123+0000 7f648dffb700 1 -- 172.21.15.204:0/1817149896 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1903+0+0 (secure 0 0 0) 0x7f6488006e40 con 0x7f6490101a20 2024-05-08T23:41:27.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.123+0000 7f6494f8f700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f64800a1110 0x7f64800a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:27.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.124+0000 7f6494f8f700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f64800a1110 0x7f64800a35d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6484005fd0 tx=0x7f6484005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:27.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.128+0000 7f648dffb700 1 -- 172.21.15.204:0/1817149896 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6488095620 con 0x7f6490101a20 2024-05-08T23:41:27.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:27.331+0000 7f64971f3700 1 -- 172.21.15.204:0/1817149896 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f64900fd690 con 0x7f64800a1110 2024-05-08T23:41:27.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:27 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi160 2024-05-08T23:41:27.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:27 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi160 to 17114M 2024-05-08T23:41:27.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:27 smithi012 ceph-mon[42503]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T23:41:27.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:27 smithi012 ceph-mon[42503]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T23:41:27.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:27 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:27.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:27 smithi012 ceph-mon[42503]: from='osd.1 [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:41:27.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:27 smithi012 ceph-mon[42503]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:41:27.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:27 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi160 2024-05-08T23:41:27.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:27 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi160 to 17114M 2024-05-08T23:41:27.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:27 smithi204 ceph-mon[47373]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T23:41:27.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:27 smithi204 ceph-mon[47373]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T23:41:27.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:27 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:27.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:27 smithi204 ceph-mon[47373]: from='osd.1 [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:41:27.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:27 smithi204 ceph-mon[47373]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:41:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:27 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi160 2024-05-08T23:41:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:27 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi160 to 17114M 2024-05-08T23:41:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:27 smithi160 ceph-mon[46152]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T23:41:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:27 smithi160 ceph-mon[46152]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T23:41:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:27 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:27 smithi160 ceph-mon[46152]: from='osd.1 [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:41:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:27 smithi160 ceph-mon[46152]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:41:28.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:41:28.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:41:28.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:28.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:41:28.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T23:41:28.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:28.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:28.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:41:28.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:41:28.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:41:28.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:28.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:28 smithi204 ceph-mon[47373]: from='osd.1 ' entity='osd.1' 2024-05-08T23:41:28.517 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:28 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-1[51153]: 2024-05-08T23:41:28.193+0000 7fac154d8700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T23:41:28.517 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:28 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-1[51153]: 2024-05-08T23:41:28.203+0000 7fac0dac6700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:41:28.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:41:28.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:41:28.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:28.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:41:28.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T23:41:28.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:28.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:28.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:41:28.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:41:28.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:41:28.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:28.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:28 smithi160 ceph-mon[46152]: from='osd.1 ' entity='osd.1' 2024-05-08T23:41:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:41:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:41:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:41:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T23:41:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:41:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:41:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:41:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:28 smithi012 ceph-mon[42503]: from='osd.1 ' entity='osd.1' 2024-05-08T23:41:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:29 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:29 smithi012 ceph-mon[42503]: from='client.? 172.21.15.204:0/1194258012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a297a7-f16c-4e12-ab9f-f197e06ff07e"}]: dispatch 2024-05-08T23:41:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:29 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a297a7-f16c-4e12-ab9f-f197e06ff07e"}]: dispatch 2024-05-08T23:41:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:29 smithi012 ceph-mon[42503]: osd.1 [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] boot 2024-05-08T23:41:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:29 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71a297a7-f16c-4e12-ab9f-f197e06ff07e"}]': finished 2024-05-08T23:41:29.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:29 smithi012 ceph-mon[42503]: osdmap e13: 3 total, 2 up, 3 in 2024-05-08T23:41:29.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:29 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:29.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:29 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:29 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:29 smithi204 ceph-mon[47373]: from='client.? 172.21.15.204:0/1194258012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a297a7-f16c-4e12-ab9f-f197e06ff07e"}]: dispatch 2024-05-08T23:41:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:29 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a297a7-f16c-4e12-ab9f-f197e06ff07e"}]: dispatch 2024-05-08T23:41:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:29 smithi204 ceph-mon[47373]: osd.1 [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] boot 2024-05-08T23:41:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:29 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71a297a7-f16c-4e12-ab9f-f197e06ff07e"}]': finished 2024-05-08T23:41:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:29 smithi204 ceph-mon[47373]: osdmap e13: 3 total, 2 up, 3 in 2024-05-08T23:41:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:29 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:29 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:29 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:29 smithi160 ceph-mon[46152]: from='client.? 172.21.15.204:0/1194258012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a297a7-f16c-4e12-ab9f-f197e06ff07e"}]: dispatch 2024-05-08T23:41:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:29 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a297a7-f16c-4e12-ab9f-f197e06ff07e"}]: dispatch 2024-05-08T23:41:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:29 smithi160 ceph-mon[46152]: osd.1 [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] boot 2024-05-08T23:41:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:29 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71a297a7-f16c-4e12-ab9f-f197e06ff07e"}]': finished 2024-05-08T23:41:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:29 smithi160 ceph-mon[46152]: osdmap e13: 3 total, 2 up, 3 in 2024-05-08T23:41:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:29 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:41:29.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:29 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:30 smithi012 ceph-mon[42503]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:30 smithi012 ceph-mon[42503]: from='client.? 172.21.15.204:0/1093901393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:41:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:30 smithi204 ceph-mon[47373]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:30 smithi204 ceph-mon[47373]: from='client.? 172.21.15.204:0/1093901393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:41:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:30 smithi160 ceph-mon[46152]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:41:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:30 smithi160 ceph-mon[46152]: from='client.? 172.21.15.204:0/1093901393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:41:31.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:31 smithi012 ceph-mon[42503]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T23:41:31.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:31.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:31 smithi204 ceph-mon[47373]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T23:41:31.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:31.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:31 smithi160 ceph-mon[46152]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T23:41:31.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:31 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:32.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:32 smithi012 ceph-mon[42503]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:32.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:32 smithi204 ceph-mon[47373]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:32 smithi160 ceph-mon[46152]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:34.457 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:34 smithi204 ceph-mon[47373]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:34.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:34 smithi012 ceph-mon[42503]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:34 smithi160 ceph-mon[46152]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:35.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T23:41:35.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:35.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T23:41:35.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:35.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T23:41:35.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:36.624 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:36 smithi204 ceph-mon[47373]: Deploying daemon osd.2 on smithi204 2024-05-08T23:41:36.625 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:36 smithi204 ceph-mon[47373]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:36.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:36 smithi012 ceph-mon[42503]: Deploying daemon osd.2 on smithi204 2024-05-08T23:41:36.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:36 smithi012 ceph-mon[42503]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:36.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:36 smithi160 ceph-mon[46152]: Deploying daemon osd.2 on smithi204 2024-05-08T23:41:36.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:36 smithi160 ceph-mon[46152]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:38.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:38 smithi204 ceph-mon[47373]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:38 smithi012 ceph-mon[42503]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:38 smithi160 ceph-mon[46152]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:40.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:40.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:40.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:39 smithi204 ceph-mon[47373]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:40.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:40.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:40.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:40.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:39 smithi012 ceph-mon[42503]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:40.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:39 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:40.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:39 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:40.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:39 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:40.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:39 smithi160 ceph-mon[46152]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:41.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.184+0000 7f648dffb700 1 -- 172.21.15.204:0/1817149896 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f64900fd690 con 0x7f64800a1110 2024-05-08T23:41:41.181 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-05-08T23:41:41.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.186+0000 7f64971f3700 1 -- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f64800a1110 msgr2=0x7f64800a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.186+0000 7f64971f3700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f64800a1110 0x7f64800a35d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6484005fd0 tx=0x7f6484005e30 comp rx=0 tx=0).stop 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.187+0000 7f64971f3700 1 -- 172.21.15.204:0/1817149896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6490101a20 msgr2=0x7f649019e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.187+0000 7f64971f3700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6490101a20 0x7f649019e490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f648800b9a0 tx=0x7f648800bcb0 comp rx=0 tx=0).stop 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.187+0000 7f64971f3700 1 -- 172.21.15.204:0/1817149896 shutdown_connections 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.187+0000 7f64971f3700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6490101a20 0x7f649019e490 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.187+0000 7f64971f3700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64900fff00 0x7f6490196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.187+0000 7f64971f3700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f64800a1110 0x7f64800a35d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.187+0000 7f64971f3700 1 --2- 172.21.15.204:0/1817149896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6490101060 0x7f64901974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.187+0000 7f64971f3700 1 -- 172.21.15.204:0/1817149896 >> 172.21.15.204:0/1817149896 conn(0x7f64900fb4c0 msgr2=0x7f64900fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.187+0000 7f64971f3700 1 -- 172.21.15.204:0/1817149896 shutdown_connections 2024-05-08T23:41:41.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:41:41.187+0000 7f64971f3700 1 -- 172.21.15.204:0/1817149896 wait complete. 2024-05-08T23:41:41.561 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T23:41:41.564 INFO:tasks.cephadm:Deploying osd.3 on smithi012 with /dev/vg_nvme/lv_3... 2024-05-08T23:41:41.564 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T23:41:41.597 INFO:journalctl@ceph.osd.2.smithi204.stdout:-- Logs begin at Wed 2024-05-08 23:21:56 UTC. -- 2024-05-08T23:41:41.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:41.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:41.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:41.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:41.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:41.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:41.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:41.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:41.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:41.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:41.824 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:41:42.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:42 smithi012 ceph-mon[42503]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:42.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:42 smithi204 ceph-mon[47373]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:42.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:42 smithi160 ceph-mon[46152]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T23:41:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T23:41:43.163 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-08T23:41:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-08T23:41:43.164 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123325 s, 850 MB/s 2024-05-08T23:41:43.164 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 2024-05-08T23:41:43.164 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-08T23:41:43.264 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:42 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-2[53134]: 2024-05-08T23:41:42.891+0000 7f019a6277c0 -1 osd.2 0 log_to_monitors true 2024-05-08T23:41:43.470 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:43 smithi012 ceph-mon[42503]: from='osd.2 [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:41:43.470 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:43 smithi012 ceph-mon[42503]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:41:43.587 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_3 2024-05-08T23:41:43.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:43 smithi204 ceph-mon[47373]: from='osd.2 [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:41:43.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:43 smithi204 ceph-mon[47373]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:41:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:43 smithi160 ceph-mon[46152]: from='osd.2 [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:41:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:43 smithi160 ceph-mon[46152]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:41:43.850 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:41:44.660 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:44.660 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='osd.2 [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:44.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T23:41:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T23:41:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='osd.2 [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:41:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:41:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:41:44.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:44.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:44.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='osd.2 [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:41:44.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:44.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.846+0000 7fa79417c700 1 -- 172.21.15.12:0/3521710904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78c074fd0 msgr2=0x7fa78c073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:44.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.846+0000 7fa79417c700 1 --2- 172.21.15.12:0/3521710904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78c074fd0 0x7fa78c073430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa788009990 tx=0x7fa788009ca0 comp rx=0 tx=0).stop 2024-05-08T23:41:44.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.846+0000 7fa79417c700 1 -- 172.21.15.12:0/3521710904 shutdown_connections 2024-05-08T23:41:44.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.846+0000 7fa79417c700 1 --2- 172.21.15.12:0/3521710904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c10d5e0 0x7fa78c10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:44.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.846+0000 7fa79417c700 1 --2- 172.21.15.12:0/3521710904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78c074fd0 0x7fa78c073430 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:44.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.846+0000 7fa79417c700 1 --2- 172.21.15.12:0/3521710904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa78c073a00 0x7fa78c073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:44.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.846+0000 7fa79417c700 1 -- 172.21.15.12:0/3521710904 >> 172.21.15.12:0/3521710904 conn(0x7fa78c0fc5c0 msgr2=0x7fa78c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:44.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.846+0000 7fa79417c700 1 -- 172.21.15.12:0/3521710904 shutdown_connections 2024-05-08T23:41:44.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.846+0000 7fa79417c700 1 -- 172.21.15.12:0/3521710904 wait complete. 2024-05-08T23:41:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa79417c700 1 Processor -- start 2024-05-08T23:41:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa79417c700 1 -- start start 2024-05-08T23:41:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa79417c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa78c073a00 0x7fa78c19d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa79417c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c074fd0 0x7fa78c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa79417c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78c10d5e0 0x7fa78c1a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa79417c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1a5050 con 0x7fa78c10d5e0 2024-05-08T23:41:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa79417c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1a5190 con 0x7fa78c073a00 2024-05-08T23:41:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa79417c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1a5320 con 0x7fa78c074fd0 2024-05-08T23:41:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa791717700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c074fd0 0x7fa78c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa791717700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c074fd0 0x7fa78c19da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:55844/0 (socket says 172.21.15.12:55844) 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa792719700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78c10d5e0 0x7fa78c1a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa791717700 1 -- 172.21.15.12:0/11691927 learned_addr learned my addr 172.21.15.12:0/11691927 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.847+0000 7fa791f18700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa78c073a00 0x7fa78c19d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa791717700 1 -- 172.21.15.12:0/11691927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa78c073a00 msgr2=0x7fa78c19d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa791717700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa78c073a00 0x7fa78c19d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa791717700 1 -- 172.21.15.12:0/11691927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78c10d5e0 msgr2=0x7fa78c1a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa791717700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78c10d5e0 0x7fa78c1a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa791717700 1 -- 172.21.15.12:0/11691927 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7880095f0 con 0x7fa78c074fd0 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa792719700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78c10d5e0 0x7fa78c1a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa791717700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c074fd0 0x7fa78c19da50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00c9a0 tx=0x7fa77c00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa782ffd700 1 -- 172.21.15.12:0/11691927 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c004220 con 0x7fa78c074fd0 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa79417c700 1 -- 172.21.15.12:0/11691927 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c1a5600 con 0x7fa78c074fd0 2024-05-08T23:41:44.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa782ffd700 1 -- 172.21.15.12:0/11691927 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77c004380 con 0x7fa78c074fd0 2024-05-08T23:41:44.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa782ffd700 1 -- 172.21.15.12:0/11691927 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c003b40 con 0x7fa78c074fd0 2024-05-08T23:41:44.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.848+0000 7fa79417c700 1 -- 172.21.15.12:0/11691927 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c1a5b00 con 0x7fa78c074fd0 2024-05-08T23:41:44.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.849+0000 7fa79417c700 1 -- 172.21.15.12:0/11691927 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78c103840 con 0x7fa78c074fd0 2024-05-08T23:41:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.851+0000 7fa782ffd700 1 -- 172.21.15.12:0/11691927 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7fa77c003cc0 con 0x7fa78c074fd0 2024-05-08T23:41:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.851+0000 7fa782ffd700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7780a1090 0x7fa7780a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.851+0000 7fa791f18700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7780a1090 0x7fa7780a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.851+0000 7fa782ffd700 1 -- 172.21.15.12:0/11691927 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2571+0+0 (secure 0 0 0) 0x7fa77c0cb0f0 con 0x7fa78c074fd0 2024-05-08T23:41:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.852+0000 7fa791f18700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7780a1090 0x7fa7780a3550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa78800b680 tx=0x7fa788005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:44.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:44.855+0000 7fa782ffd700 1 -- 172.21.15.12:0/11691927 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa77c095f30 con 0x7fa78c074fd0 2024-05-08T23:41:45.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:45.077+0000 7fa79417c700 1 -- 172.21.15.12:0/11691927 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa78c02ce10 con 0x7fa7780a1090 2024-05-08T23:41:45.514 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:45 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-2[53134]: 2024-05-08T23:41:45.191+0000 7f018f48a700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T23:41:45.514 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:45 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-2[53134]: 2024-05-08T23:41:45.198+0000 7f0187a78700 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:41:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:45 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi204 2024-05-08T23:41:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:45 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi204 to 17113M 2024-05-08T23:41:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:45 smithi204 ceph-mon[47373]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:41:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:45 smithi204 ceph-mon[47373]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T23:41:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:41:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:41:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:45 smithi204 ceph-mon[47373]: from='osd.2 ' entity='osd.2' 2024-05-08T23:41:45.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:45 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi204 2024-05-08T23:41:45.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:45 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi204 to 17113M 2024-05-08T23:41:45.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:45 smithi012 ceph-mon[42503]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:41:45.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:45 smithi012 ceph-mon[42503]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T23:41:45.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:45.644 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:45.644 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:41:45.644 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:41:45.644 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:45.644 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:45 smithi012 ceph-mon[42503]: from='osd.2 ' entity='osd.2' 2024-05-08T23:41:45.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:45 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi204 2024-05-08T23:41:45.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:45 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi204 to 17113M 2024-05-08T23:41:45.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:45 smithi160 ceph-mon[46152]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:41:45.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:45 smithi160 ceph-mon[46152]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T23:41:45.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:45.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:45.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:41:45.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:41:45.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:45.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:45 smithi160 ceph-mon[46152]: from='osd.2 ' entity='osd.2' 2024-05-08T23:41:46.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:46 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:41:46.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:46 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:41:46.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:46 smithi012 ceph-mon[42503]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:41:46.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:46 smithi012 ceph-mon[42503]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:46.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:46 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:46.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:46 smithi012 ceph-mon[42503]: osd.2 [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] boot 2024-05-08T23:41:46.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:46 smithi012 ceph-mon[42503]: osdmap e17: 3 total, 3 up, 3 in 2024-05-08T23:41:46.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:46 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:46.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:46 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:41:46.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:46 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:41:46.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:46 smithi204 ceph-mon[47373]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:41:46.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:46 smithi204 ceph-mon[47373]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:46.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:46 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:46.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:46 smithi204 ceph-mon[47373]: osd.2 [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] boot 2024-05-08T23:41:46.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:46 smithi204 ceph-mon[47373]: osdmap e17: 3 total, 3 up, 3 in 2024-05-08T23:41:46.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:46 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:46.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:46 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:41:46.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:46 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:41:46.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:46 smithi160 ceph-mon[46152]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:41:46.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:46 smithi160 ceph-mon[46152]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:41:46.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:46 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:46.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:46 smithi160 ceph-mon[46152]: osd.2 [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] boot 2024-05-08T23:41:46.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:46 smithi160 ceph-mon[46152]: osdmap e17: 3 total, 3 up, 3 in 2024-05-08T23:41:46.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:46 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:41:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:47 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/259064612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ebe819d-3d5e-4128-8c6a-23921df4320b"}]: dispatch 2024-05-08T23:41:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:47 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/259064612' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ebe819d-3d5e-4128-8c6a-23921df4320b"}]': finished 2024-05-08T23:41:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:47 smithi204 ceph-mon[47373]: osdmap e18: 4 total, 3 up, 4 in 2024-05-08T23:41:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:48.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:47 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/259064612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ebe819d-3d5e-4128-8c6a-23921df4320b"}]: dispatch 2024-05-08T23:41:48.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:47 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/259064612' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ebe819d-3d5e-4128-8c6a-23921df4320b"}]': finished 2024-05-08T23:41:48.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:47 smithi160 ceph-mon[46152]: osdmap e18: 4 total, 3 up, 4 in 2024-05-08T23:41:48.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:48.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:47 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/259064612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ebe819d-3d5e-4128-8c6a-23921df4320b"}]: dispatch 2024-05-08T23:41:48.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:47 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/259064612' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ebe819d-3d5e-4128-8c6a-23921df4320b"}]': finished 2024-05-08T23:41:48.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:47 smithi012 ceph-mon[42503]: osdmap e18: 4 total, 3 up, 4 in 2024-05-08T23:41:48.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:49.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:48 smithi204 ceph-mon[47373]: pgmap v64: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:49.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:41:49.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:48 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2511925384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:41:49.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T23:41:49.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:48 smithi204 ceph-mon[47373]: osdmap e19: 4 total, 3 up, 4 in 2024-05-08T23:41:49.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:49.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:41:49.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:48 smithi160 ceph-mon[46152]: pgmap v64: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:49.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:41:49.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:48 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2511925384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:41:49.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T23:41:49.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:48 smithi160 ceph-mon[46152]: osdmap e19: 4 total, 3 up, 4 in 2024-05-08T23:41:49.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:49.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:41:49.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:48 smithi012 ceph-mon[42503]: pgmap v64: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:49.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:41:49.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:48 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2511925384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:41:49.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T23:41:49.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:48 smithi012 ceph-mon[42503]: osdmap e19: 4 total, 3 up, 4 in 2024-05-08T23:41:49.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:49.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:41:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:41:50.184+0000 7fa782ffd700 1 -- 172.21.15.12:0/11691927 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa77c096300 con 0x7fa78c074fd0 2024-05-08T23:41:50.416 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:50 smithi012 sudo[67528]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:41:50.416 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:50 smithi012 sudo[67528]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:50.416 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:50 smithi012 sudo[67528]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:50.416 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:50 smithi012 sudo[67528]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:50.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:41:50.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:50 smithi012 ceph-mon[42503]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T23:41:50.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:50.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:50 smithi012 ceph-mon[42503]: pgmap v67: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:41:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:50 smithi204 ceph-mon[47373]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T23:41:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:50 smithi204 ceph-mon[47373]: pgmap v67: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:50.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:41:50.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:50 smithi160 ceph-mon[46152]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T23:41:50.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:50.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:50 smithi160 ceph-mon[46152]: pgmap v67: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:50.916 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:50 smithi012 sudo[67908]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:41:50.916 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:50 smithi012 sudo[67908]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:50.916 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:50 smithi012 sudo[67908]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:50.916 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 08 23:41:50 smithi012 sudo[67908]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:51.105 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:50 smithi160 sudo[55981]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:41:51.105 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:50 smithi160 sudo[55981]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:51.106 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:50 smithi160 sudo[55981]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:51.106 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:50 smithi160 sudo[55981]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:51.106 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:51 smithi160 sudo[55992]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:41:51.106 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:51 smithi160 sudo[55992]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:51.106 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:51 smithi160 sudo[55992]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:51.106 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 08 23:41:51 smithi160 sudo[55992]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:51.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:51 smithi012 ceph-mon[42503]: mgrmap e16: x(active, since 114s), standbys: z, y 2024-05-08T23:41:51.352 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:51 smithi012 ceph-mon[42503]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T23:41:51.352 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:51 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:51.352 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:51 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:51.352 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:51 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:51.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:51 smithi204 ceph-mon[47373]: mgrmap e16: x(active, since 114s), standbys: z, y 2024-05-08T23:41:51.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:51 smithi204 ceph-mon[47373]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T23:41:51.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:51 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:51.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:51 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:51.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:51 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:51.514 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:51 smithi204 sudo[58571]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:41:51.514 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:51 smithi204 sudo[58571]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:51.515 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:51 smithi204 sudo[58571]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:51.515 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:51 smithi204 sudo[58571]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:51.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:51 smithi160 ceph-mon[46152]: mgrmap e16: x(active, since 114s), standbys: z, y 2024-05-08T23:41:51.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:51 smithi160 ceph-mon[46152]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T23:41:51.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:51 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:41:51.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:51 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:51.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:51 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:52.014 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:51 smithi204 sudo[58574]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:41:52.014 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:51 smithi204 sudo[58574]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:52.014 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:51 smithi204 sudo[58574]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:52.015 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 08 23:41:51 smithi204 sudo[58574]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:52.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 sudo[68220]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:41:52.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 sudo[68220]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:52.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 sudo[68220]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:52.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 sudo[68220]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:52.705 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 sudo[68265]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:41:52.706 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 sudo[68265]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:52.706 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 sudo[68265]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:52.706 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 sudo[68265]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:52.706 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 ceph-mon[42503]: pgmap v69: 1 pgs: 1 creating+activating; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:52.706 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:52.706 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:52 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:41:53.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:52 smithi204 ceph-mon[47373]: pgmap v69: 1 pgs: 1 creating+activating; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:53.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:53.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:52 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:41:53.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:52 smithi204 sudo[58577]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:41:53.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:52 smithi204 sudo[58577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:53.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:52 smithi204 sudo[58577]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:53.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:52 smithi204 sudo[58577]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:53.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:52 smithi160 ceph-mon[46152]: pgmap v69: 1 pgs: 1 creating+activating; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:53.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:53.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:52 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:41:53.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 sudo[58580]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:41:53.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 sudo[58580]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:53.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 sudo[58580]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:53.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 sudo[58580]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:53.600 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 sudo[55995]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:41:53.601 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 sudo[55995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:53.601 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 sudo[55995]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:53.601 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 sudo[55995]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:53.601 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 sudo[55998]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:41:53.601 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 sudo[55998]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:41:53.601 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 sudo[55998]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:41:53.601 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 sudo[55998]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:41:53.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:41:53.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:41:53.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:41:53.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:41:53.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:41:53.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:41:53.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:41:53.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:41:53.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:41:53.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:41:53.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:41:53.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:41:53.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:41:53.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: pgmap v70: 1 pgs: 1 creating+activating; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:53.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:53 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:41:54.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:41:54.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:41:54.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:41:54.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:41:54.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:41:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:41:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:41:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:41:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:41:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:41:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:41:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:41:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:41:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: pgmap v70: 1 pgs: 1 creating+activating; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:53 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:41:54.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:41:54.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:41:54.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: pgmap v70: 1 pgs: 1 creating+activating; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:53 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:41:54.848 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T23:41:54.848 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:54.849 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:54 smithi012 ceph-mon[42503]: Deploying daemon osd.3 on smithi012 2024-05-08T23:41:55.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T23:41:55.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:55.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:54 smithi204 ceph-mon[47373]: Deploying daemon osd.3 on smithi012 2024-05-08T23:41:55.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T23:41:55.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:41:55.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:54 smithi160 ceph-mon[46152]: Deploying daemon osd.3 on smithi012 2024-05-08T23:41:55.749 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:55 smithi012 ceph-mon[42503]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:56.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:55 smithi204 ceph-mon[47373]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:56.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:55 smithi160 ceph-mon[46152]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:58.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:58 smithi012 ceph-mon[42503]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:58.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:58.388 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:58.389 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:41:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:58.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:58 smithi160 ceph-mon[46152]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:58.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:58.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:58.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:41:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:41:58.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:58 smithi204 ceph-mon[47373]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:41:58.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:58.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:41:58.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:41:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:00.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.224+0000 7fa782ffd700 1 -- 172.21.15.12:0/11691927 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa78c02ce10 con 0x7fa7780a1090 2024-05-08T23:42:00.220 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 3 on host 'smithi012' 2024-05-08T23:42:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.226+0000 7fa79417c700 1 -- 172.21.15.12:0/11691927 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7780a1090 msgr2=0x7fa7780a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.226+0000 7fa79417c700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7780a1090 0x7fa7780a3550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa78800b680 tx=0x7fa788005fd0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.226+0000 7fa79417c700 1 -- 172.21.15.12:0/11691927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c074fd0 msgr2=0x7fa78c19da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.226+0000 7fa79417c700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c074fd0 0x7fa78c19da50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00c9a0 tx=0x7fa77c00ccb0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.227+0000 7fa79417c700 1 -- 172.21.15.12:0/11691927 shutdown_connections 2024-05-08T23:42:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.227+0000 7fa79417c700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78c10d5e0 0x7fa78c1a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.227+0000 7fa79417c700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa7780a1090 0x7fa7780a3550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.227+0000 7fa79417c700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa78c073a00 0x7fa78c19d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.227+0000 7fa79417c700 1 --2- 172.21.15.12:0/11691927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c074fd0 0x7fa78c19da50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.227+0000 7fa79417c700 1 -- 172.21.15.12:0/11691927 >> 172.21.15.12:0/11691927 conn(0x7fa78c0fc5c0 msgr2=0x7fa78c100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.227+0000 7fa79417c700 1 -- 172.21.15.12:0/11691927 shutdown_connections 2024-05-08T23:42:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:00.227+0000 7fa79417c700 1 -- 172.21.15.12:0/11691927 wait complete. 2024-05-08T23:42:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:00 smithi012 ceph-mon[42503]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:42:00.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:00.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:00.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:00.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:00.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:00.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:00 smithi204 ceph-mon[47373]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:42:00.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:00.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:00.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:00.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:00.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:00.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:00 smithi160 ceph-mon[46152]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:42:00.650 DEBUG:teuthology.orchestra.run.smithi012:osd.3> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T23:42:00.653 INFO:tasks.cephadm:Deploying osd.4 on smithi160 with /dev/vg_nvme/lv_3... 2024-05-08T23:42:00.653 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T23:42:00.689 INFO:journalctl@ceph.osd.3.smithi012.stdout:-- Logs begin at Wed 2024-05-08 23:21:09 UTC. -- 2024-05-08T23:42:00.886 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:42:01.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:01 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:01.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:01 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:01.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:01 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:01.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:01 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:01.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:01 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:01.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:01 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:01.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:01 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:01.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:01 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:01.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:01 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:01.917 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 08 23:42:01 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-3[68974]: 2024-05-08T23:42:01.562+0000 7fecb3b167c0 -1 osd.3 0 log_to_monitors true 2024-05-08T23:42:02.219 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T23:42:02.219 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T23:42:02.219 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-08T23:42:02.219 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-08T23:42:02.220 INFO:teuthology.orchestra.run.smithi160.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123606 s, 848 MB/s 2024-05-08T23:42:02.220 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-08T23:42:02.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:02 smithi012 ceph-mon[42503]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:42:02.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:02 smithi012 ceph-mon[42503]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T23:42:02.449 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:02 smithi160 ceph-mon[46152]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:42:02.449 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:02 smithi160 ceph-mon[46152]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T23:42:02.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:02 smithi204 ceph-mon[47373]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:42:02.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:02 smithi204 ceph-mon[47373]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T23:42:02.631 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_3 2024-05-08T23:42:02.850 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:42:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T23:42:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T23:42:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:42:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:03.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:03.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:03.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T23:42:03.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T23:42:03.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:03.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:42:03.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:03.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:03.250 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.250 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:03.250 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:03.250 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T23:42:03.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T23:42:03.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:03.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:42:03.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:03.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.775+0000 7f269d88d700 1 -- 172.21.15.160:0/2731299650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2698106cc0 msgr2=0x7f2698109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:03.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.775+0000 7f269d88d700 1 --2- 172.21.15.160:0/2731299650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2698106cc0 0x7f2698109130 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f269000b0a0 tx=0x7f269000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:42:03.771 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.775+0000 7f269d88d700 1 -- 172.21.15.160:0/2731299650 shutdown_connections 2024-05-08T23:42:03.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.775+0000 7f269d88d700 1 --2- 172.21.15.160:0/2731299650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2698106cc0 0x7f2698109130 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:03.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.775+0000 7f269d88d700 1 --2- 172.21.15.160:0/2731299650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2698101a60 0x7f2698103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:03.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.775+0000 7f269d88d700 1 --2- 172.21.15.160:0/2731299650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2698104390 0x7f2698106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:03.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.775+0000 7f269d88d700 1 -- 172.21.15.160:0/2731299650 >> 172.21.15.160:0/2731299650 conn(0x7f26980fb470 msgr2=0x7f26980fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:03.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.775+0000 7f269d88d700 1 -- 172.21.15.160:0/2731299650 shutdown_connections 2024-05-08T23:42:03.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.775+0000 7f269d88d700 1 -- 172.21.15.160:0/2731299650 wait complete. 2024-05-08T23:42:03.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f269d88d700 1 Processor -- start 2024-05-08T23:42:03.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f269d88d700 1 -- start start 2024-05-08T23:42:03.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f269d88d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2698101a60 0x7f2698197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:03.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f269d88d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2698104390 0x7f2698197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:03.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f269d88d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2698106cc0 0x7f269819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:03.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f269d88d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269819eb80 con 0x7f2698106cc0 2024-05-08T23:42:03.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f269d88d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269819ecc0 con 0x7f2698104390 2024-05-08T23:42:03.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f269d88d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269819ee50 con 0x7f2698101a60 2024-05-08T23:42:03.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f26967fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2698104390 0x7f2698197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:03.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f26967fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2698104390 0x7f2698197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38202/0 (socket says 172.21.15.160:38202) 2024-05-08T23:42:03.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f26967fc700 1 -- 172.21.15.160:0/3207005415 learned_addr learned my addr 172.21.15.160:0/3207005415 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f26977fe700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2698106cc0 0x7f269819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.776+0000 7f2696ffd700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2698101a60 0x7f2698197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f26967fc700 1 -- 172.21.15.160:0/3207005415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2698101a60 msgr2=0x7f2698197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f26967fc700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2698101a60 0x7f2698197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f26967fc700 1 -- 172.21.15.160:0/3207005415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2698106cc0 msgr2=0x7f269819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f26967fc700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2698106cc0 0x7f269819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f26967fc700 1 -- 172.21.15.160:0/3207005415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2690009d00 con 0x7f2698104390 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f2696ffd700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2698101a60 0x7f2698197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f26967fc700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2698104390 0x7f2698197580 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f268800d980 tx=0x7f268800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f269c88b700 1 -- 172.21.15.160:0/3207005415 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688004220 con 0x7f2698104390 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f269c88b700 1 -- 172.21.15.160:0/3207005415 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2688007640 con 0x7f2698104390 2024-05-08T23:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f269c88b700 1 -- 172.21.15.160:0/3207005415 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688005680 con 0x7f2698104390 2024-05-08T23:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f269d88d700 1 -- 172.21.15.160:0/3207005415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269819f130 con 0x7f2698104390 2024-05-08T23:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.777+0000 7f269d88d700 1 -- 172.21.15.160:0/3207005415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269819f630 con 0x7f2698104390 2024-05-08T23:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.778+0000 7f269d88d700 1 -- 172.21.15.160:0/3207005415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26980fcff0 con 0x7f2698104390 2024-05-08T23:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.779+0000 7f269c88b700 1 -- 172.21.15.160:0/3207005415 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2688004380 con 0x7f2698104390 2024-05-08T23:42:03.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.779+0000 7f269c88b700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f26800a11a0 0x7f26800a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:03.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.779+0000 7f269c88b700 1 -- 172.21.15.160:0/3207005415 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3389+0+0 (secure 0 0 0) 0x7f26880cb820 con 0x7f2698104390 2024-05-08T23:42:03.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.780+0000 7f2696ffd700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f26800a11a0 0x7f26800a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:03.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.780+0000 7f2696ffd700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f26800a11a0 0x7f26800a3660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f268c007900 tx=0x7f268c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:03.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.783+0000 7f269c88b700 1 -- 172.21.15.160:0/3207005415 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f26880962f0 con 0x7f2698104390 2024-05-08T23:42:03.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:03.986+0000 7f269d88d700 1 -- 172.21.15.160:0/3207005415 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f269802ce10 con 0x7f26800a11a0 2024-05-08T23:42:04.346 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi012 2024-05-08T23:42:04.347 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi012 to 8559M 2024-05-08T23:42:04.347 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:42:04.347 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T23:42:04.347 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:04.347 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:04.348 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:42:04.348 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:04.348 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:04.348 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:04.348 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:04 smithi160 ceph-mon[46152]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' 2024-05-08T23:42:04.416 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 08 23:42:04 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-3[68974]: 2024-05-08T23:42:04.032+0000 7feca8979700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T23:42:04.416 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 08 23:42:04 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-3[68974]: 2024-05-08T23:42:04.043+0000 7feca0f67700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:42:04.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi012 2024-05-08T23:42:04.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi012 to 8559M 2024-05-08T23:42:04.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:42:04.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T23:42:04.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:04.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:04.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:42:04.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:04.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:04.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:04.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:04 smithi012 ceph-mon[42503]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' 2024-05-08T23:42:04.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi012 2024-05-08T23:42:04.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi012 to 8559M 2024-05-08T23:42:04.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:42:04.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T23:42:04.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:04.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:04.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:42:04.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:04.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:04.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:04.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:04 smithi204 ceph-mon[47373]: from='osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336]' entity='osd.3' 2024-05-08T23:42:05.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:05 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:42:05.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:05 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:42:05.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:05 smithi012 ceph-mon[42503]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:42:05.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:05 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:05.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:05 smithi012 ceph-mon[42503]: osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] boot 2024-05-08T23:42:05.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:05 smithi012 ceph-mon[42503]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T23:42:05.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:05 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:05.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:05 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:42:05.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:05 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:42:05.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:05 smithi204 ceph-mon[47373]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:42:05.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:05 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:05.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:05 smithi204 ceph-mon[47373]: osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] boot 2024-05-08T23:42:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:05 smithi204 ceph-mon[47373]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T23:42:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:05 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:05.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:05 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:42:05.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:05 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:42:05.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:05 smithi160 ceph-mon[46152]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:42:05.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:05 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:05.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:05 smithi160 ceph-mon[46152]: osd.3 [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] boot 2024-05-08T23:42:05.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:05 smithi160 ceph-mon[46152]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T23:42:05.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:05 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:42:06.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:06 smithi012 ceph-mon[42503]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T23:42:06.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:06 smithi012 ceph-mon[42503]: pgmap v80: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:06.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:06 smithi012 ceph-mon[42503]: from='client.? 172.21.15.160:0/3251313570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e77c894-013b-40d3-9249-85a0c63076ea"}]: dispatch 2024-05-08T23:42:06.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:06 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e77c894-013b-40d3-9249-85a0c63076ea"}]: dispatch 2024-05-08T23:42:06.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:06 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e77c894-013b-40d3-9249-85a0c63076ea"}]': finished 2024-05-08T23:42:06.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:06 smithi012 ceph-mon[42503]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T23:42:06.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:06 smithi204 ceph-mon[47373]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T23:42:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:06 smithi204 ceph-mon[47373]: pgmap v80: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:06 smithi204 ceph-mon[47373]: from='client.? 172.21.15.160:0/3251313570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e77c894-013b-40d3-9249-85a0c63076ea"}]: dispatch 2024-05-08T23:42:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:06 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e77c894-013b-40d3-9249-85a0c63076ea"}]: dispatch 2024-05-08T23:42:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:06 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e77c894-013b-40d3-9249-85a0c63076ea"}]': finished 2024-05-08T23:42:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:06 smithi204 ceph-mon[47373]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T23:42:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:06.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:06 smithi160 ceph-mon[46152]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T23:42:06.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:06 smithi160 ceph-mon[46152]: pgmap v80: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:06.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:06 smithi160 ceph-mon[46152]: from='client.? 172.21.15.160:0/3251313570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e77c894-013b-40d3-9249-85a0c63076ea"}]: dispatch 2024-05-08T23:42:06.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:06 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e77c894-013b-40d3-9249-85a0c63076ea"}]: dispatch 2024-05-08T23:42:06.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:06 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e77c894-013b-40d3-9249-85a0c63076ea"}]': finished 2024-05-08T23:42:06.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:06 smithi160 ceph-mon[46152]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T23:42:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:07.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:07 smithi012 ceph-mon[42503]: from='client.? 172.21.15.160:0/3918146576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:42:07.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:07 smithi012 ceph-mon[42503]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T23:42:07.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:07.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:07 smithi204 ceph-mon[47373]: from='client.? 172.21.15.160:0/3918146576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:42:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:07 smithi204 ceph-mon[47373]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T23:42:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:07 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:07.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:07 smithi160 ceph-mon[46152]: from='client.? 172.21.15.160:0/3918146576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:42:07.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:07 smithi160 ceph-mon[46152]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T23:42:07.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:07 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:08 smithi012 ceph-mon[42503]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:08 smithi012 ceph-mon[42503]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T23:42:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:08.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:08 smithi204 ceph-mon[47373]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:08.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:08 smithi204 ceph-mon[47373]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T23:42:08.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:08 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:08.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:08 smithi160 ceph-mon[46152]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:08.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:08 smithi160 ceph-mon[46152]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T23:42:08.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:08 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:10.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:10 smithi160 ceph-mon[46152]: pgmap v85: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:10.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:10 smithi012 ceph-mon[42503]: pgmap v85: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:10.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:10 smithi204 ceph-mon[47373]: pgmap v85: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:12.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:12 smithi160 ceph-mon[46152]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:42:12.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T23:42:12.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:12 smithi012 ceph-mon[42503]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:42:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T23:42:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:12.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:12 smithi204 ceph-mon[47373]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:42:12.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T23:42:12.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:13.473 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:13 smithi160 ceph-mon[46152]: Deploying daemon osd.4 on smithi160 2024-05-08T23:42:13.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:13 smithi012 ceph-mon[42503]: Deploying daemon osd.4 on smithi160 2024-05-08T23:42:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:13 smithi204 ceph-mon[47373]: Deploying daemon osd.4 on smithi160 2024-05-08T23:42:14.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:14 smithi160 ceph-mon[46152]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-08T23:42:14.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:14 smithi012 ceph-mon[42503]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-08T23:42:14.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:14 smithi204 ceph-mon[47373]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-08T23:42:16.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:16 smithi012 ceph-mon[42503]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-08T23:42:16.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:16.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:16.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:16.673 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:16 smithi160 ceph-mon[46152]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-08T23:42:16.674 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:16 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:16.674 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:16 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:16.674 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:16 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:16 smithi204 ceph-mon[47373]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-08T23:42:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:16 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:16 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:16 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:18.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.242+0000 7f269c88b700 1 -- 172.21.15.160:0/3207005415 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f269802ce10 con 0x7f26800a11a0 2024-05-08T23:42:18.239 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 4 on host 'smithi160' 2024-05-08T23:42:18.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 -- 172.21.15.160:0/3207005415 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f26800a11a0 msgr2=0x7f26800a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:18.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f26800a11a0 0x7f26800a3660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f268c007900 tx=0x7f268c008040 comp rx=0 tx=0).stop 2024-05-08T23:42:18.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 -- 172.21.15.160:0/3207005415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2698104390 msgr2=0x7f2698197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:18.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2698104390 0x7f2698197580 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f268800d980 tx=0x7f268800dc90 comp rx=0 tx=0).stop 2024-05-08T23:42:18.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 -- 172.21.15.160:0/3207005415 shutdown_connections 2024-05-08T23:42:18.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2698106cc0 0x7f269819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:18.242 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2698101a60 0x7f2698197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:18.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f26800a11a0 0x7f26800a3660 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:18.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 --2- 172.21.15.160:0/3207005415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2698104390 0x7f2698197580 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:18.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 -- 172.21.15.160:0/3207005415 >> 172.21.15.160:0/3207005415 conn(0x7f26980fb470 msgr2=0x7f2698102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:18.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 -- 172.21.15.160:0/3207005415 shutdown_connections 2024-05-08T23:42:18.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:18.245+0000 7f269d88d700 1 -- 172.21.15.160:0/3207005415 wait complete. 2024-05-08T23:42:18.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:18 smithi204 ceph-mon[47373]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:42:18.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:18.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:18.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:18.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:18 smithi160 ceph-mon[46152]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:42:18.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:18.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:18.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:18.627 DEBUG:teuthology.orchestra.run.smithi160:osd.4> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T23:42:18.629 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-05-08T23:42:18.629 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T23:42:18.663 INFO:journalctl@ceph.osd.4.smithi160.stdout:-- Logs begin at Wed 2024-05-08 23:21:47 UTC. -- 2024-05-08T23:42:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:18 smithi012 ceph-mon[42503]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:42:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:18.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:18.863 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:42:20.142 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T23:42:20.143 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T23:42:20.143 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-08T23:42:20.143 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-08T23:42:20.143 INFO:teuthology.orchestra.run.smithi204.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111881 s, 937 MB/s 2024-05-08T23:42:20.143 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 2024-05-08T23:42:20.143 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-08T23:42:20.242 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 23:42:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-4[59112]: 2024-05-08T23:42:19.873+0000 7f3f157807c0 -1 osd.4 0 log_to_monitors true 2024-05-08T23:42:20.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:20 smithi204 ceph-mon[47373]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-08T23:42:20.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:20 smithi204 ceph-mon[47373]: from='osd.4 [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:42:20.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:20 smithi204 ceph-mon[47373]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:42:20.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:20 smithi160 ceph-mon[46152]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-08T23:42:20.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:20 smithi160 ceph-mon[46152]: from='osd.4 [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:42:20.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:20 smithi160 ceph-mon[46152]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:42:20.522 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-05-08T23:42:20.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:20 smithi012 ceph-mon[42503]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-08T23:42:20.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:20 smithi012 ceph-mon[42503]: from='osd.4 [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:42:20.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:20 smithi012 ceph-mon[42503]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:42:20.739 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:42:21.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T23:42:21.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T23:42:21.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:21.509 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='osd.4 [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:42:21.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:42:21.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:21.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:21.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:21.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:21.510 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:21 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.516 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 23:42:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-4[59112]: 2024-05-08T23:42:21.258+0000 7f3f0a5e3700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T23:42:21.517 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 08 23:42:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-4[59112]: 2024-05-08T23:42:21.267+0000 7f3f06bd9700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='osd.4 [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:21.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:21.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:21 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T23:42:21.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T23:42:21.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:21.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='osd.4 [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:42:21.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:42:21.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:21.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:21.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:21.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:21.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:21.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 -- 172.21.15.204:0/1882919032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc4c101a20 msgr2=0x7efc4c108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 --2- 172.21.15.204:0/1882919032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc4c101a20 0x7efc4c108340 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b0a0 tx=0x7efc4400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 -- 172.21.15.204:0/1882919032 shutdown_connections 2024-05-08T23:42:21.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 --2- 172.21.15.204:0/1882919032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc4c101a20 0x7efc4c108340 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 --2- 172.21.15.204:0/1882919032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc4c0fff00 0x7efc4c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 --2- 172.21.15.204:0/1882919032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c101060 0x7efc4c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 -- 172.21.15.204:0/1882919032 >> 172.21.15.204:0/1882919032 conn(0x7efc4c0fb4a0 msgr2=0x7efc4c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 -- 172.21.15.204:0/1882919032 shutdown_connections 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 -- 172.21.15.204:0/1882919032 wait complete. 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 Processor -- start 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.684+0000 7efc541cc700 1 -- start start 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc541cc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc4c0fff00 0x7efc4c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc541cc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc4c101060 0x7efc4c19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc541cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c101a20 0x7efc4c19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc541cc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c19e870 con 0x7efc4c101a20 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc541cc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c19e9e0 con 0x7efc4c0fff00 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc541cc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c19eba0 con 0x7efc4c101060 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc51767700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc4c101060 0x7efc4c19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc51767700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc4c101060 0x7efc4c19bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33426/0 (socket says 172.21.15.204:33426) 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc51767700 1 -- 172.21.15.204:0/472586402 learned_addr learned my addr 172.21.15.204:0/472586402 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc51f68700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc4c0fff00 0x7efc4c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc52769700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c101a20 0x7efc4c19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc51767700 1 -- 172.21.15.204:0/472586402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc4c0fff00 msgr2=0x7efc4c069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc51767700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc4c0fff00 0x7efc4c069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc51767700 1 -- 172.21.15.204:0/472586402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c101a20 msgr2=0x7efc4c19e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc51767700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c101a20 0x7efc4c19e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc51767700 1 -- 172.21.15.204:0/472586402 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc44009d00 con 0x7efc4c101060 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc51767700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc4c101060 0x7efc4c19bb00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00d950 tx=0x7efc3c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc42ffd700 1 -- 172.21.15.204:0/472586402 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c00f890 con 0x7efc4c101060 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc42ffd700 1 -- 172.21.15.204:0/472586402 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc3c010460 con 0x7efc4c101060 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc541cc700 1 -- 172.21.15.204:0/472586402 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c19ee80 con 0x7efc4c101060 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc42ffd700 1 -- 172.21.15.204:0/472586402 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c00e610 con 0x7efc4c101060 2024-05-08T23:42:21.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.685+0000 7efc541cc700 1 -- 172.21.15.204:0/472586402 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c19f2f0 con 0x7efc4c101060 2024-05-08T23:42:21.685 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.686+0000 7efc541cc700 1 -- 172.21.15.204:0/472586402 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c107300 con 0x7efc4c101060 2024-05-08T23:42:21.685 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.687+0000 7efc42ffd700 1 -- 172.21.15.204:0/472586402 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efc3c0108d0 con 0x7efc4c101060 2024-05-08T23:42:21.686 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.688+0000 7efc42ffd700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efc380a11a0 0x7efc380a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.686 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.688+0000 7efc42ffd700 1 -- 172.21.15.204:0/472586402 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3821+0+0 (secure 0 0 0) 0x7efc3c0cb5a0 con 0x7efc4c101060 2024-05-08T23:42:21.686 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.688+0000 7efc51f68700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efc380a11a0 0x7efc380a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.686 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.689+0000 7efc51f68700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efc380a11a0 0x7efc380a3660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efc48005950 tx=0x7efc4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:21.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.692+0000 7efc42ffd700 1 -- 172.21.15.204:0/472586402 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efc3c095e20 con 0x7efc4c101060 2024-05-08T23:42:21.888 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:21.891+0000 7efc541cc700 1 -- 172.21.15.204:0/472586402 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7efc4c02ce10 con 0x7efc380a11a0 2024-05-08T23:42:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:22 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi160 2024-05-08T23:42:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:22 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi160 to 8557M 2024-05-08T23:42:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:22 smithi204 ceph-mon[47373]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:22 smithi204 ceph-mon[47373]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:42:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:22 smithi204 ceph-mon[47373]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T23:42:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:22 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:22 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:22 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:22 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:22 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:22.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:22 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi160 2024-05-08T23:42:22.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:22 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi160 to 8557M 2024-05-08T23:42:22.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:22 smithi160 ceph-mon[46152]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:22 smithi160 ceph-mon[46152]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:42:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:22 smithi160 ceph-mon[46152]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T23:42:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:22 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:22 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:22 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:22 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:22 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:22.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:22 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi160 2024-05-08T23:42:22.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:22 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi160 to 8557M 2024-05-08T23:42:22.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:22 smithi012 ceph-mon[42503]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:22.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:22 smithi012 ceph-mon[42503]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:42:22.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:22 smithi012 ceph-mon[42503]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T23:42:22.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:22.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:22.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:22.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:22.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:22 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:23.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:23 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:42:23.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:23 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:42:23.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:23 smithi012 ceph-mon[42503]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:42:23.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:23 smithi012 ceph-mon[42503]: osd.4 [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] boot 2024-05-08T23:42:23.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:23 smithi012 ceph-mon[42503]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T23:42:23.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:23 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:23.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:23 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:42:23.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:23 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:42:23.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:23 smithi204 ceph-mon[47373]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:42:23.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:23 smithi204 ceph-mon[47373]: osd.4 [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] boot 2024-05-08T23:42:23.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:23 smithi204 ceph-mon[47373]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T23:42:23.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:23 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:23.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:23 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:42:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:23 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:42:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:23 smithi160 ceph-mon[46152]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:42:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:23 smithi160 ceph-mon[46152]: osd.4 [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] boot 2024-05-08T23:42:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:23 smithi160 ceph-mon[46152]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T23:42:23.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:23 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:42:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:24 smithi160 ceph-mon[46152]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:24 smithi160 ceph-mon[46152]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T23:42:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:24 smithi160 ceph-mon[46152]: from='client.? 172.21.15.204:0/4064534633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f45217cd-6ced-413f-8be8-66c2740e7fee"}]: dispatch 2024-05-08T23:42:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:24 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f45217cd-6ced-413f-8be8-66c2740e7fee"}]: dispatch 2024-05-08T23:42:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:24 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f45217cd-6ced-413f-8be8-66c2740e7fee"}]': finished 2024-05-08T23:42:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:24 smithi160 ceph-mon[46152]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T23:42:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:24 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:24.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:24 smithi012 ceph-mon[42503]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:24.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:24 smithi012 ceph-mon[42503]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T23:42:24.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:24 smithi012 ceph-mon[42503]: from='client.? 172.21.15.204:0/4064534633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f45217cd-6ced-413f-8be8-66c2740e7fee"}]: dispatch 2024-05-08T23:42:24.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:24 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f45217cd-6ced-413f-8be8-66c2740e7fee"}]: dispatch 2024-05-08T23:42:24.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:24 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f45217cd-6ced-413f-8be8-66c2740e7fee"}]': finished 2024-05-08T23:42:24.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:24 smithi012 ceph-mon[42503]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T23:42:24.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:24.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:24 smithi204 ceph-mon[47373]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:42:24.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:24 smithi204 ceph-mon[47373]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T23:42:24.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:24 smithi204 ceph-mon[47373]: from='client.? 172.21.15.204:0/4064534633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f45217cd-6ced-413f-8be8-66c2740e7fee"}]: dispatch 2024-05-08T23:42:24.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:24 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f45217cd-6ced-413f-8be8-66c2740e7fee"}]: dispatch 2024-05-08T23:42:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:24 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f45217cd-6ced-413f-8be8-66c2740e7fee"}]': finished 2024-05-08T23:42:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:24 smithi204 ceph-mon[47373]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T23:42:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:24 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:25.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:25 smithi012 ceph-mon[42503]: from='client.? 172.21.15.204:0/628778657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:42:25.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:25 smithi204 ceph-mon[47373]: from='client.? 172.21.15.204:0/628778657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:42:25.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:25 smithi160 ceph-mon[46152]: from='client.? 172.21.15.204:0/628778657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:42:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:26 smithi012 ceph-mon[42503]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:26.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:26 smithi204 ceph-mon[47373]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:26.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:26 smithi160 ceph-mon[46152]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:28.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:28 smithi204 ceph-mon[47373]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:28 smithi012 ceph-mon[42503]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:28 smithi160 ceph-mon[46152]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:30.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:30 smithi204 ceph-mon[47373]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:30.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:30 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T23:42:30.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:30 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:30 smithi012 ceph-mon[42503]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:30 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T23:42:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:30 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:30 smithi160 ceph-mon[46152]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:30 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T23:42:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:30 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:31.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:31 smithi012 ceph-mon[42503]: Deploying daemon osd.5 on smithi204 2024-05-08T23:42:31.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:31 smithi204 ceph-mon[47373]: Deploying daemon osd.5 on smithi204 2024-05-08T23:42:31.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:31 smithi160 ceph-mon[46152]: Deploying daemon osd.5 on smithi204 2024-05-08T23:42:32.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:32 smithi204 ceph-mon[47373]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:32.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:32 smithi012 ceph-mon[42503]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:32 smithi160 ceph-mon[46152]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:34.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:34 smithi204 ceph-mon[47373]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:34.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:34 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:34.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:34 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:34.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:34 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:34 smithi160 ceph-mon[46152]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:34 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:34 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:34.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:34 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:34 smithi012 ceph-mon[42503]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:34 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:34 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:34 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:35.648 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-05-08T23:42:35.648 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.652+0000 7efc42ffd700 1 -- 172.21.15.204:0/472586402 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efc4c02ce10 con 0x7efc380a11a0 2024-05-08T23:42:35.650 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.654+0000 7efc541cc700 1 -- 172.21.15.204:0/472586402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efc380a11a0 msgr2=0x7efc380a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:35.650 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.654+0000 7efc541cc700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efc380a11a0 0x7efc380a3660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efc48005950 tx=0x7efc4800b410 comp rx=0 tx=0).stop 2024-05-08T23:42:35.650 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.654+0000 7efc541cc700 1 -- 172.21.15.204:0/472586402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc4c101060 msgr2=0x7efc4c19bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:35.650 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.654+0000 7efc541cc700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc4c101060 0x7efc4c19bb00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00d950 tx=0x7efc3c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:42:35.650 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.654+0000 7efc541cc700 1 -- 172.21.15.204:0/472586402 shutdown_connections 2024-05-08T23:42:35.651 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.654+0000 7efc541cc700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c101a20 0x7efc4c19e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:35.651 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.654+0000 7efc541cc700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efc380a11a0 0x7efc380a3660 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:35.651 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.655+0000 7efc541cc700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc4c0fff00 0x7efc4c069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:35.651 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.655+0000 7efc541cc700 1 --2- 172.21.15.204:0/472586402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc4c101060 0x7efc4c19bb00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:35.651 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.655+0000 7efc541cc700 1 -- 172.21.15.204:0/472586402 >> 172.21.15.204:0/472586402 conn(0x7efc4c0fb4a0 msgr2=0x7efc4c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:35.651 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.655+0000 7efc541cc700 1 -- 172.21.15.204:0/472586402 shutdown_connections 2024-05-08T23:42:35.651 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:42:35.655+0000 7efc541cc700 1 -- 172.21.15.204:0/472586402 wait complete. 2024-05-08T23:42:35.807 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:35.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:35.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:35.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:35.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:35.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:35.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:35.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:35.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:35.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:36.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:36.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:36.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:36.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:36.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:36.034 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T23:42:36.036 INFO:tasks.cephadm:Deploying osd.6 on smithi012 with /dev/vg_nvme/lv_2... 2024-05-08T23:42:36.036 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T23:42:36.071 INFO:journalctl@ceph.osd.5.smithi204.stdout:-- Logs begin at Wed 2024-05-08 23:21:56 UTC. -- 2024-05-08T23:42:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:42:36.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:36 smithi012 ceph-mon[42503]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:36.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:36 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:36.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:36 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:36.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:36 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:36.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:36 smithi204 ceph-mon[47373]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:36.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:36 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:36.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:36 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:36.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:36 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:37.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:36 smithi160 ceph-mon[46152]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:37.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:36 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:37.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:36 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:37.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:36 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:37.708 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T23:42:37.708 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T23:42:37.708 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-08T23:42:37.708 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-08T23:42:37.708 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0137475 s, 763 MB/s 2024-05-08T23:42:37.709 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-08T23:42:37.735 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 08 23:42:37 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-08T23:42:37.406+0000 7fe5b9d627c0 -1 osd.5 0 log_to_monitors true 2024-05-08T23:42:37.735 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:37 smithi204 ceph-mon[47373]: from='osd.5 [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:42:37.735 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:37 smithi204 ceph-mon[47373]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:42:37.886 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:37 smithi012 ceph-mon[42503]: from='osd.5 [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:42:37.886 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:37 smithi012 ceph-mon[42503]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:42:38.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:37 smithi160 ceph-mon[46152]: from='osd.5 [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:42:38.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:37 smithi160 ceph-mon[46152]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:42:38.102 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_2 2024-05-08T23:42:38.359 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:42:38.861 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:38.861 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T23:42:38.861 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T23:42:38.861 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:38.861 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='osd.5 [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi204 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi204 to 8556M 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:38.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:39.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:39.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T23:42:39.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T23:42:39.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:39.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='osd.5 [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:42:39.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:42:39.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi204 2024-05-08T23:42:39.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:39.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:39.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:39.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:39.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi204 to 8556M 2024-05-08T23:42:39.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:39.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:39.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:39.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:39.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:39.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T23:42:39.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T23:42:39.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='osd.5 [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi204 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi204 to 8556M 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:39.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:39.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.401+0000 7fc86c3a6700 1 -- 172.21.15.12:0/1483145314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8640fff00 msgr2=0x7fc864100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.401+0000 7fc86c3a6700 1 --2- 172.21.15.12:0/1483145314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8640fff00 0x7fc864100320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc860009990 tx=0x7fc860009ca0 comp rx=0 tx=0).stop 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.401+0000 7fc86c3a6700 1 -- 172.21.15.12:0/1483145314 shutdown_connections 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.401+0000 7fc86c3a6700 1 --2- 172.21.15.12:0/1483145314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc864101a20 0x7fc864108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.401+0000 7fc86c3a6700 1 --2- 172.21.15.12:0/1483145314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8640fff00 0x7fc864100320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.401+0000 7fc86c3a6700 1 --2- 172.21.15.12:0/1483145314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc864101060 0x7fc8641014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.401+0000 7fc86c3a6700 1 -- 172.21.15.12:0/1483145314 >> 172.21.15.12:0/1483145314 conn(0x7fc8640fb4c0 msgr2=0x7fc8640fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.401+0000 7fc86c3a6700 1 -- 172.21.15.12:0/1483145314 shutdown_connections 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.401+0000 7fc86c3a6700 1 -- 172.21.15.12:0/1483145314 wait complete. 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.402+0000 7fc86c3a6700 1 Processor -- start 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.402+0000 7fc86c3a6700 1 -- start start 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86c3a6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8640fff00 0x7fc864196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86c3a6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc864101060 0x7fc864197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86c3a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc864101a20 0x7fc86419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86c3a6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86419ea60 con 0x7fc864101a20 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86c3a6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86419eba0 con 0x7fc864101060 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86c3a6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86419ed30 con 0x7fc8640fff00 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a943700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc864101a20 0x7fc86419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a142700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8640fff00 0x7fc864196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a943700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc864101a20 0x7fc86419e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51556/0 (socket says 172.21.15.12:51556) 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a142700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8640fff00 0x7fc864196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44188/0 (socket says 172.21.15.12:44188) 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a943700 1 -- 172.21.15.12:0/624854402 learned_addr learned my addr 172.21.15.12:0/624854402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc869941700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc864101060 0x7fc864197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a943700 1 -- 172.21.15.12:0/624854402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8640fff00 msgr2=0x7fc864196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a943700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8640fff00 0x7fc864196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a943700 1 -- 172.21.15.12:0/624854402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc864101060 msgr2=0x7fc864197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a943700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc864101060 0x7fc864197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:39.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a943700 1 -- 172.21.15.12:0/624854402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8600095f0 con 0x7fc864101a20 2024-05-08T23:42:39.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc869941700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc864101060 0x7fc864197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:42:39.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a142700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8640fff00 0x7fc864196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:42:39.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.403+0000 7fc86a943700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc864101a20 0x7fc86419e490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00ef60 tx=0x7fc85c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:39.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.404+0000 7fc85b7fe700 1 -- 172.21.15.12:0/624854402 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c018070 con 0x7fc864101a20 2024-05-08T23:42:39.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.404+0000 7fc85b7fe700 1 -- 172.21.15.12:0/624854402 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc85c00f040 con 0x7fc864101a20 2024-05-08T23:42:39.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.404+0000 7fc86c3a6700 1 -- 172.21.15.12:0/624854402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86419efb0 con 0x7fc864101a20 2024-05-08T23:42:39.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.404+0000 7fc85b7fe700 1 -- 172.21.15.12:0/624854402 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00ccc0 con 0x7fc864101a20 2024-05-08T23:42:39.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.404+0000 7fc86c3a6700 1 -- 172.21.15.12:0/624854402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86419f420 con 0x7fc864101a20 2024-05-08T23:42:39.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.405+0000 7fc86c3a6700 1 -- 172.21.15.12:0/624854402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86404ec60 con 0x7fc864101a20 2024-05-08T23:42:39.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.406+0000 7fc85b7fe700 1 -- 172.21.15.12:0/624854402 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc85c003a00 con 0x7fc864101a20 2024-05-08T23:42:39.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.407+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc8500a1120 0x7fc8500a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:39.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.407+0000 7fc86a142700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc8500a1120 0x7fc8500a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:39.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.407+0000 7fc85b7fe700 1 -- 172.21.15.12:0/624854402 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4253+0+0 (secure 0 0 0) 0x7fc85c0cb370 con 0x7fc864101a20 2024-05-08T23:42:39.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.408+0000 7fc86a142700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc8500a1120 0x7fc8500a35e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc86000b2e0 tx=0x7fc86000b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:39.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.413+0000 7fc85b7fe700 1 -- 172.21.15.12:0/624854402 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc85c095b70 con 0x7fc864101a20 2024-05-08T23:42:39.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:39.636+0000 7fc86c3a6700 1 -- 172.21.15.12:0/624854402 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc864069920 con 0x7fc8500a1120 2024-05-08T23:42:39.764 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 08 23:42:39 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-08T23:42:39.335+0000 7fe5aebc5700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T23:42:39.764 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 08 23:42:39 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-08T23:42:39.347+0000 7fe5a71b3700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:42:39.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:39 smithi204 ceph-mon[47373]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:42:39.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:39 smithi204 ceph-mon[47373]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T23:42:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:39 smithi204 ceph-mon[47373]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:39 smithi204 ceph-mon[47373]: from='osd.5 ' entity='osd.5' 2024-05-08T23:42:39.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:39 smithi012 ceph-mon[42503]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:42:39.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:39 smithi012 ceph-mon[42503]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T23:42:39.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:39.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:39.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:39 smithi012 ceph-mon[42503]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:39.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:39 smithi012 ceph-mon[42503]: from='osd.5 ' entity='osd.5' 2024-05-08T23:42:40.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:39 smithi160 ceph-mon[46152]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:42:40.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:39 smithi160 ceph-mon[46152]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T23:42:40.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:39 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:40.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:39 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:40.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:39 smithi160 ceph-mon[46152]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:42:40.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:39 smithi160 ceph-mon[46152]: from='osd.5 ' entity='osd.5' 2024-05-08T23:42:40.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:40 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:42:40.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:40 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:42:40.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:40.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:40 smithi012 ceph-mon[42503]: osd.5 [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] boot 2024-05-08T23:42:40.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:40 smithi012 ceph-mon[42503]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T23:42:40.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:40.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:40 smithi012 ceph-mon[42503]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:42:40.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:40.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:40.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:41.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:40 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:42:41.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:40 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:42:41.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:41.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:40 smithi204 ceph-mon[47373]: osd.5 [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] boot 2024-05-08T23:42:41.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:40 smithi204 ceph-mon[47373]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T23:42:41.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:41.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:40 smithi204 ceph-mon[47373]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:42:41.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:41.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:41.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:41.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:40 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:42:41.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:40 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:42:41.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:41.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:40 smithi160 ceph-mon[46152]: osd.5 [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] boot 2024-05-08T23:42:41.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:40 smithi160 ceph-mon[46152]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T23:42:41.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:42:41.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:40 smithi160 ceph-mon[46152]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:42:41.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:41.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:41.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:41.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:41 smithi012 ceph-mon[42503]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T23:42:41.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:41 smithi012 ceph-mon[42503]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:42:42.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:41 smithi204 ceph-mon[47373]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T23:42:42.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:41 smithi204 ceph-mon[47373]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:42:42.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:41 smithi160 ceph-mon[46152]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T23:42:42.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:41 smithi160 ceph-mon[46152]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:42:42.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:42 smithi012 ceph-mon[42503]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T23:42:42.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:42 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/241063096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c76b479-95f8-4516-bf24-3677fa408097"}]: dispatch 2024-05-08T23:42:42.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:42 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/241063096' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c76b479-95f8-4516-bf24-3677fa408097"}]': finished 2024-05-08T23:42:42.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:42 smithi012 ceph-mon[42503]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T23:42:42.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:42.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:42 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3149424678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:42:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:42 smithi204 ceph-mon[47373]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T23:42:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:42 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/241063096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c76b479-95f8-4516-bf24-3677fa408097"}]: dispatch 2024-05-08T23:42:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:42 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/241063096' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c76b479-95f8-4516-bf24-3677fa408097"}]': finished 2024-05-08T23:42:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:42 smithi204 ceph-mon[47373]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T23:42:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:42 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3149424678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:42:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:42 smithi160 ceph-mon[46152]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T23:42:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:42 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/241063096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c76b479-95f8-4516-bf24-3677fa408097"}]: dispatch 2024-05-08T23:42:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:42 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/241063096' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c76b479-95f8-4516-bf24-3677fa408097"}]': finished 2024-05-08T23:42:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:42 smithi160 ceph-mon[46152]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T23:42:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:42 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3149424678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:42:44.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:43 smithi204 ceph-mon[47373]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T23:42:44.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:44.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:43 smithi204 ceph-mon[47373]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:42:44.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:43 smithi160 ceph-mon[46152]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T23:42:44.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:44.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:43 smithi160 ceph-mon[46152]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:42:44.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:43 smithi012 ceph-mon[42503]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T23:42:44.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:44.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:43 smithi012 ceph-mon[42503]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:42:45.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:44 smithi204 ceph-mon[47373]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T23:42:45.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:45.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:44 smithi160 ceph-mon[46152]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T23:42:45.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:45.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:44 smithi012 ceph-mon[42503]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T23:42:45.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:46.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:45 smithi204 ceph-mon[47373]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:42:46.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:45 smithi160 ceph-mon[46152]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:42:46.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:45 smithi012 ceph-mon[42503]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:42:48.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:48 smithi012 ceph-mon[42503]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-08T23:42:48.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:48 smithi204 ceph-mon[47373]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-08T23:42:48.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:48 smithi160 ceph-mon[46152]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-08T23:42:49.635 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T23:42:49.635 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:49.635 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:49 smithi012 ceph-mon[42503]: Deploying daemon osd.6 on smithi012 2024-05-08T23:42:49.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T23:42:49.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:49.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:49 smithi204 ceph-mon[47373]: Deploying daemon osd.6 on smithi012 2024-05-08T23:42:49.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T23:42:49.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:49.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:49 smithi160 ceph-mon[46152]: Deploying daemon osd.6 on smithi012 2024-05-08T23:42:50.609 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:50 smithi012 ceph-mon[42503]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T23:42:50.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:50 smithi204 ceph-mon[47373]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T23:42:50.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:50 smithi160 ceph-mon[46152]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T23:42:52.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:52 smithi012 ceph-mon[42503]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T23:42:52.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:52 smithi204 ceph-mon[47373]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T23:42:52.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:52 smithi160 ceph-mon[46152]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T23:42:53.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:53.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:53.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:53.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:53.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:53.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:53.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:53.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:53.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:54.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:54 smithi204 ceph-mon[47373]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:42:54.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:54.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:54.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:54.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:54.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:54.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:54 smithi160 ceph-mon[46152]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:42:54.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:54.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:54.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:54.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:54.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:54.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:54 smithi012 ceph-mon[42503]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:42:54.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:54.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:54.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:54.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:54.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:55.030 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 6 on host 'smithi012' 2024-05-08T23:42:55.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.033+0000 7fc85b7fe700 1 -- 172.21.15.12:0/624854402 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc864069920 con 0x7fc8500a1120 2024-05-08T23:42:55.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 -- 172.21.15.12:0/624854402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc8500a1120 msgr2=0x7fc8500a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:55.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc8500a1120 0x7fc8500a35e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc86000b2e0 tx=0x7fc86000b6f0 comp rx=0 tx=0).stop 2024-05-08T23:42:55.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 -- 172.21.15.12:0/624854402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc864101a20 msgr2=0x7fc86419e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:55.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc864101a20 0x7fc86419e490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00ef60 tx=0x7fc85c00ef90 comp rx=0 tx=0).stop 2024-05-08T23:42:55.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 -- 172.21.15.12:0/624854402 shutdown_connections 2024-05-08T23:42:55.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc864101a20 0x7fc86419e490 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:55.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8640fff00 0x7fc864196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:55.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc8500a1120 0x7fc8500a35e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:55.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 --2- 172.21.15.12:0/624854402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc864101060 0x7fc864197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:55.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 -- 172.21.15.12:0/624854402 >> 172.21.15.12:0/624854402 conn(0x7fc8640fb4c0 msgr2=0x7fc864069210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:55.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 -- 172.21.15.12:0/624854402 shutdown_connections 2024-05-08T23:42:55.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:42:55.036+0000 7fc86c3a6700 1 -- 172.21.15.12:0/624854402 wait complete. 2024-05-08T23:42:55.414 DEBUG:teuthology.orchestra.run.smithi012:osd.6> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T23:42:55.417 INFO:tasks.cephadm:Deploying osd.7 on smithi160 with /dev/vg_nvme/lv_2... 2024-05-08T23:42:55.417 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T23:42:55.453 INFO:journalctl@ceph.osd.6.smithi012.stdout:-- Logs begin at Wed 2024-05-08 23:21:09 UTC. -- 2024-05-08T23:42:55.649 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:42:56.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:56.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:56.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:56.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:55 smithi160 ceph-mon[46152]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T23:42:56.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:56.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:56.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:56.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:55 smithi012 ceph-mon[42503]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T23:42:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:42:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:55 smithi204 ceph-mon[47373]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T23:42:56.666 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 08 23:42:56 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-08T23:42:56.451+0000 7fd3da5d37c0 -1 osd.6 0 log_to_monitors true 2024-05-08T23:42:56.997 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T23:42:56.997 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T23:42:56.997 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-08T23:42:56.997 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-08T23:42:56.997 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137993 s, 760 MB/s 2024-05-08T23:42:56.997 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-08T23:42:57.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:56 smithi204 ceph-mon[47373]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T23:42:57.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:56 smithi160 ceph-mon[46152]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T23:42:57.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:56 smithi012 ceph-mon[42503]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T23:42:57.423 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_2 2024-05-08T23:42:57.641 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:42:58.166 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 08 23:42:58 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-08T23:42:58.010+0000 7fd3cf436700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T23:42:58.166 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 08 23:42:58 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-08T23:42:58.024+0000 7fd3cba2c700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:42:58.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T23:42:58.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:58 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T23:42:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T23:42:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:42:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:42:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:58 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T23:42:58.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T23:42:58.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:58.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:42:58.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:42:58.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:58.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:58.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:42:58.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:58.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:42:58.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:58 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:42:58.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.599+0000 7f9878a52700 1 -- 172.21.15.160:0/4084932111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98740fff00 msgr2=0x7f9874100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:58.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.599+0000 7f9878a52700 1 --2- 172.21.15.160:0/4084932111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98740fff00 0x7f9874100320 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f985c009990 tx=0x7f985c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:42:58.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.599+0000 7f9878a52700 1 -- 172.21.15.160:0/4084932111 shutdown_connections 2024-05-08T23:42:58.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.599+0000 7f9878a52700 1 --2- 172.21.15.160:0/4084932111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9874101a20 0x7f9874108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:58.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.599+0000 7f9878a52700 1 --2- 172.21.15.160:0/4084932111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98740fff00 0x7f9874100320 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:58.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.599+0000 7f9878a52700 1 --2- 172.21.15.160:0/4084932111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9874101060 0x7f98741014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.599+0000 7f9878a52700 1 -- 172.21.15.160:0/4084932111 >> 172.21.15.160:0/4084932111 conn(0x7f98740fb4a0 msgr2=0x7f98740fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.599+0000 7f9878a52700 1 -- 172.21.15.160:0/4084932111 shutdown_connections 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.599+0000 7f9878a52700 1 -- 172.21.15.160:0/4084932111 wait complete. 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.600+0000 7f9878a52700 1 Processor -- start 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.600+0000 7f9878a52700 1 -- start start 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.600+0000 7f9878a52700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98740fff00 0x7f9874196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.600+0000 7f9878a52700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9874101060 0x7f9874197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.600+0000 7f9878a52700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9874101a20 0x7f987419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9878a52700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987419eb00 con 0x7f9874101a20 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9878a52700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987419ec40 con 0x7f9874101060 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9878a52700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987419edd0 con 0x7f98740fff00 2024-05-08T23:42:58.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9871d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9874101060 0x7f9874197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9871d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9874101060 0x7f9874197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55678/0 (socket says 172.21.15.160:55678) 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9871d9b700 1 -- 172.21.15.160:0/3313379165 learned_addr learned my addr 172.21.15.160:0/3313379165 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f987259c700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98740fff00 0x7f9874196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9872d9d700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9874101a20 0x7f987419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9871d9b700 1 -- 172.21.15.160:0/3313379165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98740fff00 msgr2=0x7f9874196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9871d9b700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98740fff00 0x7f9874196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9871d9b700 1 -- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9874101a20 msgr2=0x7f987419e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9871d9b700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9874101a20 0x7f987419e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9871d9b700 1 -- 172.21.15.160:0/3313379165 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0095f0 con 0x7f9874101060 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9872d9d700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9874101a20 0x7f987419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9871d9b700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9874101060 0x7f9874197500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f986400b530 tx=0x7f986400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f986b7fe700 1 -- 172.21.15.160:0/3313379165 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864011830 con 0x7f9874101060 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9878a52700 1 -- 172.21.15.160:0/3313379165 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987419f0b0 con 0x7f9874101060 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f986b7fe700 1 -- 172.21.15.160:0/3313379165 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9864011e50 con 0x7f9874101060 2024-05-08T23:42:58.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f986b7fe700 1 -- 172.21.15.160:0/3313379165 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400f2e0 con 0x7f9874101060 2024-05-08T23:42:58.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.601+0000 7f9878a52700 1 -- 172.21.15.160:0/3313379165 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987419f600 con 0x7f9874101060 2024-05-08T23:42:58.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.602+0000 7f9878a52700 1 -- 172.21.15.160:0/3313379165 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987404ec60 con 0x7f9874101060 2024-05-08T23:42:58.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.603+0000 7f986b7fe700 1 -- 172.21.15.160:0/3313379165 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9864010380 con 0x7f9874101060 2024-05-08T23:42:58.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.604+0000 7f986b7fe700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98600a1120 0x7f98600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:58.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.604+0000 7f986b7fe700 1 -- 172.21.15.160:0/3313379165 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4685+0+0 (secure 0 0 0) 0x7f98640cbc00 con 0x7f9874101060 2024-05-08T23:42:58.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.604+0000 7f987259c700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98600a1120 0x7f98600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:58.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.605+0000 7f987259c700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98600a1120 0x7f98600a35e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f985c00b5c0 tx=0x7f985c003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:58.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.608+0000 7f986b7fe700 1 -- 172.21.15.160:0/3313379165 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f98640961a0 con 0x7f9874101060 2024-05-08T23:42:58.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:42:58.810+0000 7f9878a52700 1 -- 172.21.15.160:0/3313379165 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f987410b790 con 0x7f98600a1120 2024-05-08T23:42:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:59 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi012 2024-05-08T23:42:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:59 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi012 to 5706M 2024-05-08T23:42:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:59 smithi204 ceph-mon[47373]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:42:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:59 smithi204 ceph-mon[47373]: osdmap e43: 7 total, 6 up, 7 in 2024-05-08T23:42:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:42:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:59.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:59 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi012 2024-05-08T23:42:59.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:59 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi012 to 5706M 2024-05-08T23:42:59.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:59 smithi160 ceph-mon[46152]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:42:59.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:59 smithi160 ceph-mon[46152]: osdmap e43: 7 total, 6 up, 7 in 2024-05-08T23:42:59.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:59.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:59.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:59.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:59.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:42:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:42:59.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:59 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi012 2024-05-08T23:42:59.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:59 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi012 to 5706M 2024-05-08T23:42:59.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:59 smithi012 ceph-mon[42503]: from='osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:42:59.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:59 smithi012 ceph-mon[42503]: osdmap e43: 7 total, 6 up, 7 in 2024-05-08T23:42:59.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:59.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:42:59.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:42:59.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:42:59.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:42:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:00 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:43:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:00 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:43:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:00 smithi204 ceph-mon[47373]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:43:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:00 smithi204 ceph-mon[47373]: osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] boot 2024-05-08T23:43:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:00 smithi204 ceph-mon[47373]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T23:43:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:43:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:00 smithi204 ceph-mon[47373]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:43:00.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:00 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:43:00.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:00 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:43:00.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:00 smithi160 ceph-mon[46152]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:00.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:43:00.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:00 smithi160 ceph-mon[46152]: osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] boot 2024-05-08T23:43:00.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:00 smithi160 ceph-mon[46152]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T23:43:00.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:43:00.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:00 smithi160 ceph-mon[46152]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:43:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:00 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:43:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:00 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:43:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:00 smithi012 ceph-mon[42503]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:43:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:00 smithi012 ceph-mon[42503]: osd.6 [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] boot 2024-05-08T23:43:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:00 smithi012 ceph-mon[42503]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T23:43:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:43:00.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:00 smithi012 ceph-mon[42503]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:43:01.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:01 smithi012 ceph-mon[42503]: osdmap e45: 7 total, 7 up, 7 in 2024-05-08T23:43:01.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:01 smithi012 ceph-mon[42503]: from='client.? 172.21.15.160:0/3483144037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3df920-f628-41de-a0eb-fd1d6b1bf06a"}]: dispatch 2024-05-08T23:43:01.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:01 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3df920-f628-41de-a0eb-fd1d6b1bf06a"}]: dispatch 2024-05-08T23:43:01.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:01 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc3df920-f628-41de-a0eb-fd1d6b1bf06a"}]': finished 2024-05-08T23:43:01.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:01 smithi012 ceph-mon[42503]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T23:43:01.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:01 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:01.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:01 smithi204 ceph-mon[47373]: osdmap e45: 7 total, 7 up, 7 in 2024-05-08T23:43:01.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:01 smithi204 ceph-mon[47373]: from='client.? 172.21.15.160:0/3483144037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3df920-f628-41de-a0eb-fd1d6b1bf06a"}]: dispatch 2024-05-08T23:43:01.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:01 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3df920-f628-41de-a0eb-fd1d6b1bf06a"}]: dispatch 2024-05-08T23:43:01.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:01 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc3df920-f628-41de-a0eb-fd1d6b1bf06a"}]': finished 2024-05-08T23:43:01.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:01 smithi204 ceph-mon[47373]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T23:43:01.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:01 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:01.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:01 smithi160 ceph-mon[46152]: osdmap e45: 7 total, 7 up, 7 in 2024-05-08T23:43:01.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:01 smithi160 ceph-mon[46152]: from='client.? 172.21.15.160:0/3483144037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3df920-f628-41de-a0eb-fd1d6b1bf06a"}]: dispatch 2024-05-08T23:43:01.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:01 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3df920-f628-41de-a0eb-fd1d6b1bf06a"}]: dispatch 2024-05-08T23:43:01.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:01 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc3df920-f628-41de-a0eb-fd1d6b1bf06a"}]': finished 2024-05-08T23:43:01.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:01 smithi160 ceph-mon[46152]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T23:43:01.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:01 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:02.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:02 smithi012 ceph-mon[42503]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:02.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:02 smithi012 ceph-mon[42503]: from='client.? 172.21.15.160:0/1091848263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:02.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:02 smithi204 ceph-mon[47373]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:02.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:02 smithi204 ceph-mon[47373]: from='client.? 172.21.15.160:0/1091848263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:02.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:02 smithi160 ceph-mon[46152]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:02.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:02 smithi160 ceph-mon[46152]: from='client.? 172.21.15.160:0/1091848263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:04.621 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:04 smithi160 ceph-mon[46152]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:04.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:04 smithi012 ceph-mon[42503]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:04.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:04 smithi204 ceph-mon[47373]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:06.589 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:06 smithi160 ceph-mon[46152]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:06.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:06 smithi012 ceph-mon[42503]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:06.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:06 smithi204 ceph-mon[47373]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:07.456 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:07 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T23:43:07.456 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:07 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:07.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T23:43:07.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:07 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:07.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:07 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T23:43:07.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:07 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:08.483 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:08 smithi160 ceph-mon[46152]: Deploying daemon osd.7 on smithi160 2024-05-08T23:43:08.484 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:08 smithi160 ceph-mon[46152]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:08.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:08 smithi012 ceph-mon[42503]: Deploying daemon osd.7 on smithi160 2024-05-08T23:43:08.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:08 smithi012 ceph-mon[42503]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:08.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:08 smithi204 ceph-mon[47373]: Deploying daemon osd.7 on smithi160 2024-05-08T23:43:08.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:08 smithi204 ceph-mon[47373]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:10.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:10 smithi012 ceph-mon[42503]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:10.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:10 smithi204 ceph-mon[47373]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:10.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:10 smithi160 ceph-mon[46152]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:11.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:11.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:11.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:11.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:11.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:12.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:12.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:12.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:12.700 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:12 smithi160 ceph-mon[46152]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:12.700 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:12.700 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:12.700 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:12.700 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:12.700 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:12.864 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 7 on host 'smithi160' 2024-05-08T23:43:12.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.867+0000 7f986b7fe700 1 -- 172.21.15.160:0/3313379165 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f987410b790 con 0x7f98600a1120 2024-05-08T23:43:12.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.869+0000 7f9878a52700 1 -- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98600a1120 msgr2=0x7f98600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:12.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.869+0000 7f9878a52700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98600a1120 0x7f98600a35e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f985c00b5c0 tx=0x7f985c003790 comp rx=0 tx=0).stop 2024-05-08T23:43:12.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.869+0000 7f9878a52700 1 -- 172.21.15.160:0/3313379165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9874101060 msgr2=0x7f9874197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:12.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.869+0000 7f9878a52700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9874101060 0x7f9874197500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f986400b530 tx=0x7f986400b840 comp rx=0 tx=0).stop 2024-05-08T23:43:12.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.870+0000 7f9878a52700 1 -- 172.21.15.160:0/3313379165 shutdown_connections 2024-05-08T23:43:12.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.870+0000 7f9878a52700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9874101a20 0x7f987419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:12.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.870+0000 7f9878a52700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98740fff00 0x7f9874196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:12.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.870+0000 7f9878a52700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98600a1120 0x7f98600a35e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:12.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.870+0000 7f9878a52700 1 --2- 172.21.15.160:0/3313379165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9874101060 0x7f9874197500 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:12.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.870+0000 7f9878a52700 1 -- 172.21.15.160:0/3313379165 >> 172.21.15.160:0/3313379165 conn(0x7f98740fb4a0 msgr2=0x7f9874068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:43:12.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.870+0000 7f9878a52700 1 -- 172.21.15.160:0/3313379165 shutdown_connections 2024-05-08T23:43:12.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:12.870+0000 7f9878a52700 1 -- 172.21.15.160:0/3313379165 wait complete. 2024-05-08T23:43:12.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:12 smithi012 ceph-mon[42503]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:12.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:12.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:12.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:12.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:12.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:13.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:12 smithi204 ceph-mon[47373]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:13.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:13.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:13.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:13.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:13.301 DEBUG:teuthology.orchestra.run.smithi160:osd.7> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T23:43:13.303 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-05-08T23:43:13.303 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T23:43:13.347 INFO:journalctl@ceph.osd.7.smithi160.stdout:-- Logs begin at Wed 2024-05-08 23:21:47 UTC. -- 2024-05-08T23:43:13.548 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:43:13.968 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:13 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:13.968 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:13 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:13.968 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:13 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:13.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:13 smithi204 ceph-mon[47373]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:14.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:13 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:14.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:13 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:14.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:13 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:14.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:13 smithi160 ceph-mon[46152]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:14.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:13 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:14.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:13 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:14.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:13 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:14.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:13 smithi012 ceph-mon[42503]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:14.847 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 23:43:14 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-08T23:43:14.530+0000 7f70a7dc17c0 -1 osd.7 0 log_to_monitors true 2024-05-08T23:43:14.939 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T23:43:14.939 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T23:43:14.939 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-08T23:43:14.939 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-08T23:43:14.939 INFO:teuthology.orchestra.run.smithi204.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0152062 s, 690 MB/s 2024-05-08T23:43:14.939 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-08T23:43:15.118 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:14 smithi204 ceph-mon[47373]: from='osd.7 [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:43:15.119 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:14 smithi204 ceph-mon[47373]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:43:15.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:14 smithi012 ceph-mon[42503]: from='osd.7 [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:43:15.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:14 smithi012 ceph-mon[42503]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:43:15.188 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:14 smithi160 ceph-mon[46152]: from='osd.7 [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:43:15.188 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:14 smithi160 ceph-mon[46152]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:43:15.291 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-05-08T23:43:15.512 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:43:15.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T23:43:15.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T23:43:15.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:15.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='osd.7 [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:43:15.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:43:15.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:15.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi160 2024-05-08T23:43:15.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:15.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:15.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:15.970 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:15.970 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:15.970 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi160 to 5704M 2024-05-08T23:43:15.970 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:15.970 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:15.970 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:15.970 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:15 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T23:43:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T23:43:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='osd.7 [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:43:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:43:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi160 2024-05-08T23:43:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:16.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:16.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:16.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:16.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi160 to 5704M 2024-05-08T23:43:16.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:16.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:16.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:16.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:15 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:16.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T23:43:16.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='osd.7 [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi160 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi160 to 5704M 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:16.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:15 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:16.508 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.509+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2183509830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc598100b90 msgr2=0x7fc598108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:16.508 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.509+0000 7fc59d8f8700 1 --2- 172.21.15.204:0/2183509830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc598100b90 0x7fc598108450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc59000b0a0 tx=0x7fc59000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:43:16.508 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.509+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2183509830 shutdown_connections 2024-05-08T23:43:16.508 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.509+0000 7fc59d8f8700 1 --2- 172.21.15.204:0/2183509830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc598100b90 0x7fc598108450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:16.508 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.509+0000 7fc59d8f8700 1 --2- 172.21.15.204:0/2183509830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5981014b0 0x7fc5981018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:16.508 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.509+0000 7fc59d8f8700 1 --2- 172.21.15.204:0/2183509830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5981001f0 0x7fc598100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:16.508 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.509+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2183509830 >> 172.21.15.204:0/2183509830 conn(0x7fc5980fac40 msgr2=0x7fc5980fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:43:16.508 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.509+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2183509830 shutdown_connections 2024-05-08T23:43:16.508 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.509+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2183509830 wait complete. 2024-05-08T23:43:16.508 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.510+0000 7fc59d8f8700 1 Processor -- start 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.510+0000 7fc59d8f8700 1 -- start start 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.510+0000 7fc59d8f8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5981001f0 0x7fc598196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.510+0000 7fc59d8f8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc598100b90 0x7fc598197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.510+0000 7fc59d8f8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5981014b0 0x7fc59819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.510+0000 7fc59d8f8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59819eb20 con 0x7fc5981014b0 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.510+0000 7fc59d8f8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59819ec60 con 0x7fc5981001f0 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.510+0000 7fc59d8f8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59819edf0 con 0x7fc598100b90 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc5967fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc598100b90 0x7fc598197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc5967fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc598100b90 0x7fc598197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41492/0 (socket says 172.21.15.204:41492) 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc5967fc700 1 -- 172.21.15.204:0/2710220133 learned_addr learned my addr 172.21.15.204:0/2710220133 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc596ffd700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5981001f0 0x7fc598196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc5977fe700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5981014b0 0x7fc59819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:16.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc5967fc700 1 -- 172.21.15.204:0/2710220133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5981001f0 msgr2=0x7fc598196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:16.510 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc5967fc700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5981001f0 0x7fc598196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:16.510 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc5967fc700 1 -- 172.21.15.204:0/2710220133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5981014b0 msgr2=0x7fc59819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:16.510 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc5967fc700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5981014b0 0x7fc59819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:16.510 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc5967fc700 1 -- 172.21.15.204:0/2710220133 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc590009d00 con 0x7fc598100b90 2024-05-08T23:43:16.510 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc5967fc700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc598100b90 0x7fc598197520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc58800e9d0 tx=0x7fc58800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:43:16.510 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc59c8f6700 1 -- 172.21.15.204:0/2710220133 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588009790 con 0x7fc598100b90 2024-05-08T23:43:16.510 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc59c8f6700 1 -- 172.21.15.204:0/2710220133 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc58800ee70 con 0x7fc598100b90 2024-05-08T23:43:16.510 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2710220133 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59819f0d0 con 0x7fc598100b90 2024-05-08T23:43:16.510 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc59c8f6700 1 -- 172.21.15.204:0/2710220133 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588010ea0 con 0x7fc598100b90 2024-05-08T23:43:16.511 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.511+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2710220133 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59819f5d0 con 0x7fc598100b90 2024-05-08T23:43:16.511 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.512+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2710220133 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59804ec60 con 0x7fc598100b90 2024-05-08T23:43:16.511 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.513+0000 7fc59c8f6700 1 -- 172.21.15.204:0/2710220133 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc58800cbd0 con 0x7fc598100b90 2024-05-08T23:43:16.511 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.514+0000 7fc59c8f6700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc5800a1120 0x7fc5800a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:16.511 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.514+0000 7fc59c8f6700 1 -- 172.21.15.204:0/2710220133 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5117+0+0 (secure 0 0 0) 0x7fc588014070 con 0x7fc598100b90 2024-05-08T23:43:16.512 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.514+0000 7fc596ffd700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc5800a1120 0x7fc5800a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:16.512 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.515+0000 7fc596ffd700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc5800a1120 0x7fc5800a35e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc598069400 tx=0x7fc58c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:43:16.514 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.518+0000 7fc59c8f6700 1 -- 172.21.15.204:0/2710220133 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc588052d60 con 0x7fc598100b90 2024-05-08T23:43:16.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:16.723+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2710220133 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5980fce20 con 0x7fc5800a1120 2024-05-08T23:43:17.016 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 23:43:16 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-08T23:43:16.670+0000 7f709e427700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T23:43:17.016 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 08 23:43:16 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-08T23:43:16.682+0000 7f7095212700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:43:17.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:16 smithi160 ceph-mon[46152]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:43:17.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:16 smithi160 ceph-mon[46152]: osdmap e48: 8 total, 7 up, 8 in 2024-05-08T23:43:17.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:16 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:17.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:16 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:17.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:16 smithi160 ceph-mon[46152]: from='osd.7 ' entity='osd.7' 2024-05-08T23:43:17.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:16 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:43:17.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:16 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:43:17.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:16 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:17.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:16 smithi160 ceph-mon[46152]: osd.7 [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] boot 2024-05-08T23:43:17.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:16 smithi204 ceph-mon[47373]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:43:17.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:16 smithi204 ceph-mon[47373]: osdmap e48: 8 total, 7 up, 8 in 2024-05-08T23:43:17.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:16 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:17.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:16 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:17.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:16 smithi204 ceph-mon[47373]: from='osd.7 ' entity='osd.7' 2024-05-08T23:43:17.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:16 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:43:17.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:16 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:43:17.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:16 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:17.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:16 smithi204 ceph-mon[47373]: osd.7 [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] boot 2024-05-08T23:43:17.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:16 smithi012 ceph-mon[42503]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:43:17.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:16 smithi012 ceph-mon[42503]: osdmap e48: 8 total, 7 up, 8 in 2024-05-08T23:43:17.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:17.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:17.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:16 smithi012 ceph-mon[42503]: from='osd.7 ' entity='osd.7' 2024-05-08T23:43:17.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:43:17.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:43:17.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:17.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:16 smithi012 ceph-mon[42503]: osd.7 [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] boot 2024-05-08T23:43:18.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:17 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:43:18.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:17 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:43:18.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:17 smithi012 ceph-mon[42503]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:18.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:17 smithi012 ceph-mon[42503]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T23:43:18.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:17 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:18.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:17 smithi012 ceph-mon[42503]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:18.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:17 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:43:18.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:17 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:43:18.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:17 smithi204 ceph-mon[47373]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:17 smithi204 ceph-mon[47373]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T23:43:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:17 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:17 smithi204 ceph-mon[47373]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:18.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:17 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:43:18.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:17 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:43:18.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:17 smithi160 ceph-mon[46152]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:18.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:17 smithi160 ceph-mon[46152]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T23:43:18.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:17 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:43:18.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:17 smithi160 ceph-mon[46152]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:19.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:18 smithi012 ceph-mon[42503]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T23:43:19.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:18 smithi012 ceph-mon[42503]: from='client.? 172.21.15.204:0/3803967129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91051f6f-5519-48e0-93a6-e897d0cb3929"}]: dispatch 2024-05-08T23:43:19.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:18 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91051f6f-5519-48e0-93a6-e897d0cb3929"}]: dispatch 2024-05-08T23:43:19.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:18 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91051f6f-5519-48e0-93a6-e897d0cb3929"}]': finished 2024-05-08T23:43:19.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:18 smithi012 ceph-mon[42503]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T23:43:19.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:18 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:19.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:18 smithi204 ceph-mon[47373]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T23:43:19.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:18 smithi204 ceph-mon[47373]: from='client.? 172.21.15.204:0/3803967129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91051f6f-5519-48e0-93a6-e897d0cb3929"}]: dispatch 2024-05-08T23:43:19.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:18 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91051f6f-5519-48e0-93a6-e897d0cb3929"}]: dispatch 2024-05-08T23:43:19.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:18 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91051f6f-5519-48e0-93a6-e897d0cb3929"}]': finished 2024-05-08T23:43:19.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:18 smithi204 ceph-mon[47373]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T23:43:19.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:18 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:19.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:18 smithi160 ceph-mon[46152]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T23:43:19.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:18 smithi160 ceph-mon[46152]: from='client.? 172.21.15.204:0/3803967129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91051f6f-5519-48e0-93a6-e897d0cb3929"}]: dispatch 2024-05-08T23:43:19.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:18 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91051f6f-5519-48e0-93a6-e897d0cb3929"}]: dispatch 2024-05-08T23:43:19.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:18 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91051f6f-5519-48e0-93a6-e897d0cb3929"}]': finished 2024-05-08T23:43:19.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:18 smithi160 ceph-mon[46152]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T23:43:19.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:18 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:20.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:19 smithi012 ceph-mon[42503]: from='client.? 172.21.15.204:0/2625697896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:20.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:19 smithi012 ceph-mon[42503]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:20.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:19 smithi012 ceph-mon[42503]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T23:43:20.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:19 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:20.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:19 smithi204 ceph-mon[47373]: from='client.? 172.21.15.204:0/2625697896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:20.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:19 smithi204 ceph-mon[47373]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:20.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:19 smithi204 ceph-mon[47373]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T23:43:20.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:19 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:20.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:19 smithi160 ceph-mon[46152]: from='client.? 172.21.15.204:0/2625697896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:20.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:19 smithi160 ceph-mon[46152]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:20.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:19 smithi160 ceph-mon[46152]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T23:43:20.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:19 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:21.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:21 smithi012 ceph-mon[42503]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T23:43:21.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:21 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:21.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:21 smithi012 ceph-mon[42503]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:22.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:21 smithi204 ceph-mon[47373]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T23:43:22.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:21 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:22.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:21 smithi204 ceph-mon[47373]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:22.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:21 smithi160 ceph-mon[46152]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T23:43:22.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:21 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:22.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:21 smithi160 ceph-mon[46152]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:24.554 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:24 smithi204 ceph-mon[47373]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:24.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:24 smithi012 ceph-mon[42503]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:24.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:24 smithi160 ceph-mon[46152]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:25.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:25 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T23:43:25.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:25 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:25.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:25 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T23:43:25.726 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:25 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:25.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:25 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T23:43:25.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:25 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:26.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:26 smithi204 ceph-mon[47373]: Deploying daemon osd.8 on smithi204 2024-05-08T23:43:26.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:26 smithi204 ceph-mon[47373]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:26 smithi012 ceph-mon[42503]: Deploying daemon osd.8 on smithi204 2024-05-08T23:43:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:26 smithi012 ceph-mon[42503]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:26.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:26 smithi160 ceph-mon[46152]: Deploying daemon osd.8 on smithi204 2024-05-08T23:43:26.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:26 smithi160 ceph-mon[46152]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:28.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:28 smithi204 ceph-mon[47373]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:28.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:28.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:28.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:28 smithi012 ceph-mon[42503]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:28 smithi160 ceph-mon[46152]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:30.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:30 smithi204 ceph-mon[47373]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:30.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:30 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:30.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:30 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:30.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:30 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:30.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:30 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:30.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:30 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:30.526 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-05-08T23:43:30.527 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.529+0000 7fc59c8f6700 1 -- 172.21.15.204:0/2710220133 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc5980fce20 con 0x7fc5800a1120 2024-05-08T23:43:30.528 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2710220133 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc5800a1120 msgr2=0x7fc5800a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:30.528 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc5800a1120 0x7fc5800a35e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc598069400 tx=0x7fc58c00b410 comp rx=0 tx=0).stop 2024-05-08T23:43:30.528 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2710220133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc598100b90 msgr2=0x7fc598197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:30.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc598100b90 0x7fc598197520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc58800e9d0 tx=0x7fc58800ece0 comp rx=0 tx=0).stop 2024-05-08T23:43:30.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2710220133 shutdown_connections 2024-05-08T23:43:30.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5981014b0 0x7fc59819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:30.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc5800a1120 0x7fc5800a35e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:30.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5981001f0 0x7fc598196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:30.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 --2- 172.21.15.204:0/2710220133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc598100b90 0x7fc598197520 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:30.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2710220133 >> 172.21.15.204:0/2710220133 conn(0x7fc5980fac40 msgr2=0x7fc5980fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:43:30.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2710220133 shutdown_connections 2024-05-08T23:43:30.529 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:43:30.532+0000 7fc59d8f8700 1 -- 172.21.15.204:0/2710220133 wait complete. 2024-05-08T23:43:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:30 smithi012 ceph-mon[42503]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:30 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:30 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:30 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:30 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:30.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:30 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:30 smithi160 ceph-mon[46152]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:30 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:30 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:30 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:30 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:30 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:30.947 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T23:43:30.949 INFO:tasks.cephadm:Deploying osd.9 on smithi012 with /dev/vg_nvme/lv_1... 2024-05-08T23:43:30.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T23:43:30.982 INFO:journalctl@ceph.osd.8.smithi204.stdout:-- Logs begin at Wed 2024-05-08 23:21:56 UTC. -- 2024-05-08T23:43:31.220 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:43:31.520 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:31.520 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:31.520 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:31 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:31.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:31.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:31.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:31 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:31.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:31 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:31.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:31 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:31.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:31 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:32.650 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T23:43:32.650 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T23:43:32.650 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-08T23:43:32.650 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-08T23:43:32.650 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135971 s, 771 MB/s 2024-05-08T23:43:32.650 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 2024-05-08T23:43:32.650 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-08T23:43:32.736 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 08 23:43:32 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-08T23:43:32.280+0000 7f2ef9feb7c0 -1 osd.8 0 log_to_monitors true 2024-05-08T23:43:32.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:32 smithi204 ceph-mon[47373]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:32.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:32 smithi204 ceph-mon[47373]: from='osd.8 [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:43:32.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:32 smithi204 ceph-mon[47373]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:43:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:32 smithi160 ceph-mon[46152]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:32 smithi160 ceph-mon[46152]: from='osd.8 [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:43:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:32 smithi160 ceph-mon[46152]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:43:32.832 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:32 smithi012 ceph-mon[42503]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:32.832 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:32 smithi012 ceph-mon[42503]: from='osd.8 [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:43:32.832 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:32 smithi012 ceph-mon[42503]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:43:33.037 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_1 2024-05-08T23:43:33.301 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:43:33.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T23:43:33.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T23:43:33.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:33.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='osd.8 [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:43:33.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:43:33.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:33.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:33 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T23:43:33.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T23:43:33.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:33.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='osd.8 [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:43:33.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:43:33.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:33.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:33 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T23:43:33.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T23:43:33.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:33.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='osd.8 [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:43:33.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:43:33.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:33.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:33.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:33.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:33.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:33.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:33.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:33 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.391+0000 7f8feaf83700 1 -- 172.21.15.12:0/4200504303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fe4101000 msgr2=0x7f8fe4101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.391+0000 7f8feaf83700 1 --2- 172.21.15.12:0/4200504303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fe4101000 0x7f8fe4101420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80099c0 tx=0x7f8fd8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:43:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.391+0000 7f8feaf83700 1 -- 172.21.15.12:0/4200504303 shutdown_connections 2024-05-08T23:43:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.391+0000 7f8feaf83700 1 --2- 172.21.15.12:0/4200504303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fe4102ba0 0x7f8fe4109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.391+0000 7f8feaf83700 1 --2- 172.21.15.12:0/4200504303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fe4101000 0x7f8fe4101420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.391+0000 7f8feaf83700 1 --2- 172.21.15.12:0/4200504303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fe4102200 0x7f8fe4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.391+0000 7f8feaf83700 1 -- 172.21.15.12:0/4200504303 >> 172.21.15.12:0/4200504303 conn(0x7f8fe40fc5c0 msgr2=0x7f8fe40fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:43:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.391+0000 7f8feaf83700 1 -- 172.21.15.12:0/4200504303 shutdown_connections 2024-05-08T23:43:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.391+0000 7f8feaf83700 1 -- 172.21.15.12:0/4200504303 wait complete. 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8feaf83700 1 Processor -- start 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8feaf83700 1 -- start start 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8feaf83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fe4101000 0x7f8fe4074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8feaf83700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fe4102200 0x7f8fe4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8feaf83700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fe4102ba0 0x7f8fe40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8feaf83700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe4074070 con 0x7f8fe4101000 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8feaf83700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe40741e0 con 0x7f8fe4102ba0 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8feaf83700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe41a0f90 con 0x7f8fe4102200 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8fe8d1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fe4101000 0x7f8fe4074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8fe8d1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fe4101000 0x7f8fe4074cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49744/0 (socket says 172.21.15.12:49744) 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8fe8d1f700 1 -- 172.21.15.12:0/4258723162 learned_addr learned my addr 172.21.15.12:0/4258723162 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8fe3fff700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fe4102200 0x7f8fe4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:34.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.392+0000 7f8fe9520700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fe4102ba0 0x7f8fe40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8fe8d1f700 1 -- 172.21.15.12:0/4258723162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fe4102200 msgr2=0x7f8fe4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8fe8d1f700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fe4102200 0x7f8fe4073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8fe8d1f700 1 -- 172.21.15.12:0/4258723162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fe4102ba0 msgr2=0x7f8fe40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8fe8d1f700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fe4102ba0 0x7f8fe40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8fe8d1f700 1 -- 172.21.15.12:0/4258723162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd80095f0 con 0x7f8fe4101000 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8fe3fff700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fe4102200 0x7f8fe4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8fe8d1f700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fe4101000 0x7f8fe4074cf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009fd0 tx=0x7f8fd8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8fe1ffb700 1 -- 172.21.15.12:0/4258723162 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd801d070 con 0x7f8fe4101000 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8fe1ffb700 1 -- 172.21.15.12:0/4258723162 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd80059a0 con 0x7f8fe4101000 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8feaf83700 1 -- 172.21.15.12:0/4258723162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe41a11c0 con 0x7f8fe4101000 2024-05-08T23:43:34.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8fe1ffb700 1 -- 172.21.15.12:0/4258723162 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80173f0 con 0x7f8fe4101000 2024-05-08T23:43:34.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.393+0000 7f8feaf83700 1 -- 172.21.15.12:0/4258723162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe41a16b0 con 0x7f8fe4101000 2024-05-08T23:43:34.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.395+0000 7f8feaf83700 1 -- 172.21.15.12:0/4258723162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe4108530 con 0x7f8fe4101000 2024-05-08T23:43:34.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.396+0000 7f8fe1ffb700 1 -- 172.21.15.12:0/4258723162 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8fd8022600 con 0x7f8fe4101000 2024-05-08T23:43:34.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.396+0000 7f8fe1ffb700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8fd40a11a0 0x7f8fd40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:34.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.396+0000 7f8fe1ffb700 1 -- 172.21.15.12:0/4258723162 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5549+0+0 (secure 0 0 0) 0x7f8fd80cd6a0 con 0x7f8fe4101000 2024-05-08T23:43:34.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.396+0000 7f8fe3fff700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8fd40a11a0 0x7f8fd40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:34.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.397+0000 7f8fe3fff700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8fd40a11a0 0x7f8fd40a3660 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0011490 tx=0x7f8fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:43:34.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.401+0000 7f8fe1ffb700 1 -- 172.21.15.12:0/4258723162 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8fd8097990 con 0x7f8fe4101000 2024-05-08T23:43:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:34.629+0000 7f8feaf83700 1 -- 172.21.15.12:0/4258723162 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fe40613e0 con 0x7f8fd40a11a0 2024-05-08T23:43:34.764 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 08 23:43:34 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-08T23:43:34.299+0000 7f2eeee4e700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T23:43:34.764 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 08 23:43:34 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-08T23:43:34.316+0000 7f2ee743c700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:43:34.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:34 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi204 2024-05-08T23:43:34.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:34 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi204 to 5704M 2024-05-08T23:43:34.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:34 smithi204 ceph-mon[47373]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:34.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:34 smithi204 ceph-mon[47373]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:43:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:34 smithi204 ceph-mon[47373]: osdmap e55: 9 total, 8 up, 9 in 2024-05-08T23:43:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:34 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:34 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:34 smithi204 ceph-mon[47373]: from='osd.8 ' entity='osd.8' 2024-05-08T23:43:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:34 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi204 2024-05-08T23:43:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:34 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi204 to 5704M 2024-05-08T23:43:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:34 smithi160 ceph-mon[46152]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:34 smithi160 ceph-mon[46152]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:43:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:34 smithi160 ceph-mon[46152]: osdmap e55: 9 total, 8 up, 9 in 2024-05-08T23:43:34.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:34 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:34.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:34 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:34.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:34 smithi160 ceph-mon[46152]: from='osd.8 ' entity='osd.8' 2024-05-08T23:43:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:34 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi204 2024-05-08T23:43:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:34 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi204 to 5704M 2024-05-08T23:43:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:34 smithi012 ceph-mon[42503]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:34 smithi012 ceph-mon[42503]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:43:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:34 smithi012 ceph-mon[42503]: osdmap e55: 9 total, 8 up, 9 in 2024-05-08T23:43:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:34 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:34 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:34.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:34 smithi012 ceph-mon[42503]: from='osd.8 ' entity='osd.8' 2024-05-08T23:43:35.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:35 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:43:35.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:35 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:43:35.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:35 smithi012 ceph-mon[42503]: osd.8 [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] boot 2024-05-08T23:43:35.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:35 smithi012 ceph-mon[42503]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T23:43:35.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:35.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:43:35.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:43:35.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:35 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:35.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:35 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:43:35.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:35 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:43:35.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:35 smithi160 ceph-mon[46152]: osd.8 [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] boot 2024-05-08T23:43:35.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:35 smithi160 ceph-mon[46152]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T23:43:35.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:35.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:43:35.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:43:35.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:35 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:36.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:35 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:43:36.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:35 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:43:36.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:35 smithi204 ceph-mon[47373]: osd.8 [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] boot 2024-05-08T23:43:36.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:35 smithi204 ceph-mon[47373]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T23:43:36.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:43:36.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:43:36.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:43:36.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:35 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:36.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:36 smithi012 ceph-mon[42503]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:36.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:36 smithi012 ceph-mon[42503]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:36.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:36 smithi012 ceph-mon[42503]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T23:43:37.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:36 smithi204 ceph-mon[47373]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:37.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:36 smithi204 ceph-mon[47373]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:37.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:36 smithi204 ceph-mon[47373]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T23:43:37.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:36 smithi160 ceph-mon[46152]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:37.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:36 smithi160 ceph-mon[46152]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:43:37.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:36 smithi160 ceph-mon[46152]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T23:43:37.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:37 smithi012 ceph-mon[42503]: osdmap e58: 9 total, 9 up, 9 in 2024-05-08T23:43:37.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:37 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2210037964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d7852c8-2f31-4385-84a1-6a4c07279741"}]: dispatch 2024-05-08T23:43:37.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:37 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2210037964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d7852c8-2f31-4385-84a1-6a4c07279741"}]': finished 2024-05-08T23:43:37.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:37 smithi012 ceph-mon[42503]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T23:43:37.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:37 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:37.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:37 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2286831456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:38.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:37 smithi204 ceph-mon[47373]: osdmap e58: 9 total, 9 up, 9 in 2024-05-08T23:43:38.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:37 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2210037964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d7852c8-2f31-4385-84a1-6a4c07279741"}]: dispatch 2024-05-08T23:43:38.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:37 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2210037964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d7852c8-2f31-4385-84a1-6a4c07279741"}]': finished 2024-05-08T23:43:38.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:37 smithi204 ceph-mon[47373]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T23:43:38.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:37 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:38.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:37 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2286831456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:38.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:37 smithi160 ceph-mon[46152]: osdmap e58: 9 total, 9 up, 9 in 2024-05-08T23:43:38.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:37 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2210037964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d7852c8-2f31-4385-84a1-6a4c07279741"}]: dispatch 2024-05-08T23:43:38.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:37 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2210037964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d7852c8-2f31-4385-84a1-6a4c07279741"}]': finished 2024-05-08T23:43:38.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:37 smithi160 ceph-mon[46152]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T23:43:38.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:37 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:38.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:37 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2286831456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:38.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:38 smithi012 ceph-mon[42503]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-05-08T23:43:38.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:38 smithi012 ceph-mon[42503]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T23:43:38.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:39.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:38 smithi204 ceph-mon[47373]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-05-08T23:43:39.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:38 smithi204 ceph-mon[47373]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T23:43:39.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:39.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:38 smithi160 ceph-mon[46152]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-05-08T23:43:39.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:38 smithi160 ceph-mon[46152]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T23:43:39.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:40.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:39 smithi204 ceph-mon[47373]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T23:43:40.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:39 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:40.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:39 smithi204 ceph-mon[47373]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-05-08T23:43:40.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:39 smithi160 ceph-mon[46152]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T23:43:40.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:39 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:40.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:39 smithi160 ceph-mon[46152]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-05-08T23:43:40.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:39 smithi012 ceph-mon[42503]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T23:43:40.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:39 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:40.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:39 smithi012 ceph-mon[42503]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-05-08T23:43:42.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:42 smithi012 ceph-mon[42503]: pgmap v164: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-08T23:43:42.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:42 smithi204 ceph-mon[47373]: pgmap v164: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-08T23:43:42.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:42 smithi160 ceph-mon[46152]: pgmap v164: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-08T23:43:43.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T23:43:43.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:43.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T23:43:43.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T23:43:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:44.555 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:44 smithi012 ceph-mon[42503]: Deploying daemon osd.9 on smithi012 2024-05-08T23:43:44.555 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:44 smithi012 ceph-mon[42503]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-08T23:43:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:44 smithi204 ceph-mon[47373]: Deploying daemon osd.9 on smithi012 2024-05-08T23:43:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:44 smithi204 ceph-mon[47373]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-08T23:43:44.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:44 smithi160 ceph-mon[46152]: Deploying daemon osd.9 on smithi012 2024-05-08T23:43:44.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:44 smithi160 ceph-mon[46152]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-08T23:43:46.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:46 smithi012 ceph-mon[42503]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:46.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:46 smithi204 ceph-mon[47373]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:46.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:46 smithi160 ceph-mon[46152]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:48.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:48.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:48.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:48.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:47 smithi012 ceph-mon[42503]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:48.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:48.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:48.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:48.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:47 smithi204 ceph-mon[47373]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:48.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:48.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:48.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:48.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:47 smithi160 ceph-mon[46152]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:49.531 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 9 on host 'smithi012' 2024-05-08T23:43:49.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.534+0000 7f8fe1ffb700 1 -- 172.21.15.12:0/4258723162 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8fe40613e0 con 0x7f8fd40a11a0 2024-05-08T23:43:49.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 -- 172.21.15.12:0/4258723162 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8fd40a11a0 msgr2=0x7f8fd40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:49.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8fd40a11a0 0x7f8fd40a3660 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0011490 tx=0x7f8fd0009260 comp rx=0 tx=0).stop 2024-05-08T23:43:49.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 -- 172.21.15.12:0/4258723162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fe4101000 msgr2=0x7f8fe4074cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:49.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fe4101000 0x7f8fe4074cf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009fd0 tx=0x7f8fd8005f10 comp rx=0 tx=0).stop 2024-05-08T23:43:49.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 -- 172.21.15.12:0/4258723162 shutdown_connections 2024-05-08T23:43:49.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8fd40a11a0 0x7f8fd40a3660 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:49.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fe4102ba0 0x7f8fe40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:49.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fe4101000 0x7f8fe4074cf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:49.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 --2- 172.21.15.12:0/4258723162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fe4102200 0x7f8fe4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:49.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 -- 172.21.15.12:0/4258723162 >> 172.21.15.12:0/4258723162 conn(0x7f8fe40fc5c0 msgr2=0x7f8fe4105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:43:49.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 -- 172.21.15.12:0/4258723162 shutdown_connections 2024-05-08T23:43:49.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:43:49.537+0000 7f8feaf83700 1 -- 172.21.15.12:0/4258723162 wait complete. 2024-05-08T23:43:49.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:49.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:49.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:49.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:49.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:49.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:49 smithi012 ceph-mon[42503]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:49.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:49.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:49.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:49 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:49.943 DEBUG:teuthology.orchestra.run.smithi012:osd.9> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T23:43:49.945 INFO:tasks.cephadm:Deploying osd.10 on smithi160 with /dev/vg_nvme/lv_1... 2024-05-08T23:43:49.945 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T23:43:49.978 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:49.978 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:49.978 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:49.978 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:49.979 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:49.979 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:49 smithi160 ceph-mon[46152]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:49.979 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:49.979 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:49.979 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:49 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:49.981 INFO:journalctl@ceph.osd.9.smithi012.stdout:-- Logs begin at Wed 2024-05-08 23:21:09 UTC. -- 2024-05-08T23:43:50.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:50.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:50.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:50.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:50.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:50.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:49 smithi204 ceph-mon[47373]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:50.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:50.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:50.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:49 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:43:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:43:51.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:50 smithi012 ceph-mon[42503]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:43:51.167 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 08 23:43:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-08T23:43:50.933+0000 7fe1e08397c0 -1 osd.9 0 log_to_monitors true 2024-05-08T23:43:51.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:50 smithi204 ceph-mon[47373]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:43:51.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:50 smithi160 ceph-mon[46152]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:43:51.560 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T23:43:51.561 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T23:43:51.561 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-08T23:43:51.561 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-08T23:43:51.561 INFO:teuthology.orchestra.run.smithi160.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112352 s, 933 MB/s 2024-05-08T23:43:51.561 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 2024-05-08T23:43:51.561 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-08T23:43:52.006 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_1 2024-05-08T23:43:52.165 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 08 23:43:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-08T23:43:51.999+0000 7fe1d569c700 -1 osd.9 0 waiting for initial osdmap 2024-05-08T23:43:52.166 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 08 23:43:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-08T23:43:52.019+0000 7fe1d1c92700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:43:52.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:52 smithi012 ceph-mon[42503]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T23:43:52.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:52 smithi012 ceph-mon[42503]: osdmap e62: 10 total, 9 up, 10 in 2024-05-08T23:43:52.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:52.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:52 smithi012 ceph-mon[42503]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:43:52.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:52 smithi012 ceph-mon[42503]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:52.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:52 smithi012 ceph-mon[42503]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:43:52.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:52 smithi012 ceph-mon[42503]: osdmap e63: 10 total, 9 up, 10 in 2024-05-08T23:43:52.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:52 smithi204 ceph-mon[47373]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T23:43:52.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:52 smithi204 ceph-mon[47373]: osdmap e62: 10 total, 9 up, 10 in 2024-05-08T23:43:52.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:52.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:52 smithi204 ceph-mon[47373]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:43:52.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:52 smithi204 ceph-mon[47373]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:52.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:52 smithi204 ceph-mon[47373]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:43:52.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:52 smithi204 ceph-mon[47373]: osdmap e63: 10 total, 9 up, 10 in 2024-05-08T23:43:52.265 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:43:52.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:52 smithi160 ceph-mon[46152]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T23:43:52.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:52 smithi160 ceph-mon[46152]: osdmap e62: 10 total, 9 up, 10 in 2024-05-08T23:43:52.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:52.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:52 smithi160 ceph-mon[46152]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-08T23:43:52.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:52 smithi160 ceph-mon[46152]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:52.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:52 smithi160 ceph-mon[46152]: from='osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-08T23:43:52.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:52 smithi160 ceph-mon[46152]: osdmap e63: 10 total, 9 up, 10 in 2024-05-08T23:43:53.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.264+0000 7f80013b1700 1 -- 172.21.15.160:0/3322184172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc101b20 msgr2=0x7f7ffc1083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:53.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.264+0000 7f80013b1700 1 --2- 172.21.15.160:0/3322184172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc101b20 0x7f7ffc1083b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400b0a0 tx=0x7f7ff400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:43:53.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.264+0000 7f80013b1700 1 -- 172.21.15.160:0/3322184172 shutdown_connections 2024-05-08T23:43:53.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.264+0000 7f80013b1700 1 --2- 172.21.15.160:0/3322184172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc101b20 0x7f7ffc1083b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:53.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.264+0000 7f80013b1700 1 --2- 172.21.15.160:0/3322184172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc100440 0x7f7ffc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:53.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.264+0000 7f80013b1700 1 --2- 172.21.15.160:0/3322184172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc101160 0x7f7ffc1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.265+0000 7f80013b1700 1 -- 172.21.15.160:0/3322184172 >> 172.21.15.160:0/3322184172 conn(0x7f7ffc0fac40 msgr2=0x7f7ffc0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.265+0000 7f80013b1700 1 -- 172.21.15.160:0/3322184172 shutdown_connections 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.265+0000 7f80013b1700 1 -- 172.21.15.160:0/3322184172 wait complete. 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.265+0000 7f80013b1700 1 Processor -- start 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.265+0000 7f80013b1700 1 -- start start 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f80013b1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc100440 0x7f7ffc197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f80013b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc101160 0x7f7ffc1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f80013b1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc101b20 0x7f7ffc19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f80013b1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc19eba0 con 0x7f7ffc101160 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f80013b1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc19ece0 con 0x7f7ffc100440 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f80013b1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc19ee70 con 0x7f7ffc101b20 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffaffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc100440 0x7f7ffc197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:53.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffaffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc100440 0x7f7ffc197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43500/0 (socket says 172.21.15.160:43500) 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffaffd700 1 -- 172.21.15.160:0/2880247712 learned_addr learned my addr 172.21.15.160:0/2880247712 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffb7fe700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc101b20 0x7f7ffc19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffaffd700 1 -- 172.21.15.160:0/2880247712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc101b20 msgr2=0x7f7ffc19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffaffd700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc101b20 0x7f7ffc19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffaffd700 1 -- 172.21.15.160:0/2880247712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc101160 msgr2=0x7f7ffc1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffaffd700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc101160 0x7f7ffc1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffaffd700 1 -- 172.21.15.160:0/2880247712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4009d00 con 0x7f7ffc100440 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffb7fe700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc101b20 0x7f7ffc19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7ffaffd700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc100440 0x7f7ffc197060 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00b500 tx=0x7f7fec00b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7fe3fff700 1 -- 172.21.15.160:0/2880247712 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec011830 con 0x7f7ffc100440 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f80013b1700 1 -- 172.21.15.160:0/2880247712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ffc19f150 con 0x7f7ffc100440 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7fe3fff700 1 -- 172.21.15.160:0/2880247712 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fec011e50 con 0x7f7ffc100440 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f7fe3fff700 1 -- 172.21.15.160:0/2880247712 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec00f2e0 con 0x7f7ffc100440 2024-05-08T23:43:53.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.266+0000 7f80013b1700 1 -- 172.21.15.160:0/2880247712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ffc19f650 con 0x7f7ffc100440 2024-05-08T23:43:53.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:53.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:53.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi012 2024-05-08T23:43:53.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi012 to 4279M 2024-05-08T23:43:53.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:53.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:53.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:53.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.267+0000 7f80013b1700 1 -- 172.21.15.160:0/2880247712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ffc107340 con 0x7f7ffc100440 2024-05-08T23:43:53.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.268+0000 7f7fe3fff700 1 -- 172.21.15.160:0/2880247712 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7fec010580 con 0x7f7ffc100440 2024-05-08T23:43:53.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.269+0000 7f7fe3fff700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7fe40a1120 0x7f7fe40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:43:53.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.269+0000 7f7fe3fff700 1 -- 172.21.15.160:0/2880247712 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f7fec0cc130 con 0x7f7ffc100440 2024-05-08T23:43:53.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.269+0000 7f7ffa7fc700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7fe40a1120 0x7f7fe40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:43:53.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.270+0000 7f7ffa7fc700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7fe40a1120 0x7f7fe40a35e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0005fd0 tx=0x7f7ff0005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:43:53.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi012 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi012 to 4279M 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:53.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:53.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:53.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.273+0000 7f7fe3fff700 1 -- 172.21.15.160:0/2880247712 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7fec0960c0 con 0x7f7ffc100440 2024-05-08T23:43:53.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:53.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:53.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi012 2024-05-08T23:43:53.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:43:53.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi012 to 4279M 2024-05-08T23:43:53.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:53.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:43:53.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:43:53.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:53.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:43:53.473+0000 7f80013b1700 1 -- 172.21.15.160:0/2880247712 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ffc0613e0 con 0x7f7fe40a1120 2024-05-08T23:43:54.248 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:54 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:43:54.248 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:54 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:43:54.248 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:54 smithi160 ceph-mon[46152]: osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] boot 2024-05-08T23:43:54.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:54 smithi160 ceph-mon[46152]: osdmap e64: 10 total, 10 up, 10 in 2024-05-08T23:43:54.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:54.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:54 smithi160 ceph-mon[46152]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:54.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:54 smithi160 ceph-mon[46152]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:54.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:43:54.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:43:54.249 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:54 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:43:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:54 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:43:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:54 smithi204 ceph-mon[47373]: osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] boot 2024-05-08T23:43:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:54 smithi204 ceph-mon[47373]: osdmap e64: 10 total, 10 up, 10 in 2024-05-08T23:43:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:54 smithi204 ceph-mon[47373]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:54 smithi204 ceph-mon[47373]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:43:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:43:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:54.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:54 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:43:54.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:54 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:43:54.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:54 smithi012 ceph-mon[42503]: osd.9 [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] boot 2024-05-08T23:43:54.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:54 smithi012 ceph-mon[42503]: osdmap e64: 10 total, 10 up, 10 in 2024-05-08T23:43:54.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:43:54.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:54 smithi012 ceph-mon[42503]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:43:54.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:54 smithi012 ceph-mon[42503]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:43:54.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:43:54.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:43:54.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:43:55.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:55 smithi204 ceph-mon[47373]: osdmap e65: 10 total, 10 up, 10 in 2024-05-08T23:43:55.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:55 smithi160 ceph-mon[46152]: osdmap e65: 10 total, 10 up, 10 in 2024-05-08T23:43:55.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:55 smithi012 ceph-mon[42503]: osdmap e65: 10 total, 10 up, 10 in 2024-05-08T23:43:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:56 smithi160 ceph-mon[46152]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:43:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:56 smithi160 ceph-mon[46152]: from='client.? 172.21.15.160:0/1441307966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8a47be8-7650-4178-86b0-0b529d02ae21"}]: dispatch 2024-05-08T23:43:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:56 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8a47be8-7650-4178-86b0-0b529d02ae21"}]: dispatch 2024-05-08T23:43:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:56 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8a47be8-7650-4178-86b0-0b529d02ae21"}]': finished 2024-05-08T23:43:56.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:56 smithi160 ceph-mon[46152]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T23:43:56.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:56 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:43:56.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:56 smithi012 ceph-mon[42503]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:43:56.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:56 smithi012 ceph-mon[42503]: from='client.? 172.21.15.160:0/1441307966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8a47be8-7650-4178-86b0-0b529d02ae21"}]: dispatch 2024-05-08T23:43:56.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:56 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8a47be8-7650-4178-86b0-0b529d02ae21"}]: dispatch 2024-05-08T23:43:56.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:56 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8a47be8-7650-4178-86b0-0b529d02ae21"}]': finished 2024-05-08T23:43:56.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:56 smithi012 ceph-mon[42503]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T23:43:56.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:43:56.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:56 smithi204 ceph-mon[47373]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:43:56.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:56 smithi204 ceph-mon[47373]: from='client.? 172.21.15.160:0/1441307966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8a47be8-7650-4178-86b0-0b529d02ae21"}]: dispatch 2024-05-08T23:43:56.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:56 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8a47be8-7650-4178-86b0-0b529d02ae21"}]: dispatch 2024-05-08T23:43:56.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:56 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8a47be8-7650-4178-86b0-0b529d02ae21"}]': finished 2024-05-08T23:43:56.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:56 smithi204 ceph-mon[47373]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T23:43:56.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:56 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:43:57.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:57 smithi012 ceph-mon[42503]: from='client.? 172.21.15.160:0/621049485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:57.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:57 smithi204 ceph-mon[47373]: from='client.? 172.21.15.160:0/621049485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:57.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:57 smithi160 ceph-mon[46152]: from='client.? 172.21.15.160:0/621049485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:43:58.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:43:58 smithi012 ceph-mon[42503]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:43:58.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:43:58 smithi204 ceph-mon[47373]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:43:58.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:43:58 smithi160 ceph-mon[46152]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:00.456 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:00 smithi160 ceph-mon[46152]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:00 smithi012 ceph-mon[42503]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:00 smithi204 ceph-mon[47373]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:02.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:02 smithi160 ceph-mon[46152]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:02.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:02 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T23:44:02.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:02 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:02.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:02 smithi012 ceph-mon[42503]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:02.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:02 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T23:44:02.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:02 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:02.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:02 smithi204 ceph-mon[47373]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:02.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:02 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T23:44:02.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:02 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:03.333 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:03 smithi160 ceph-mon[46152]: Deploying daemon osd.10 on smithi160 2024-05-08T23:44:03.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:03 smithi012 ceph-mon[42503]: Deploying daemon osd.10 on smithi160 2024-05-08T23:44:03.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:03 smithi204 ceph-mon[47373]: Deploying daemon osd.10 on smithi160 2024-05-08T23:44:04.614 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:04 smithi160 ceph-mon[46152]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:04.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:04 smithi012 ceph-mon[42503]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:04 smithi204 ceph-mon[47373]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:06.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:06 smithi160 ceph-mon[46152]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:06.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:06 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:06.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:06 smithi012 ceph-mon[42503]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:06.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:06.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:06.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:06 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:06 smithi204 ceph-mon[47373]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:06.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:06.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:06.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:06 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:07.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.631+0000 7f7fe3fff700 1 -- 172.21.15.160:0/2880247712 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f7ffc0613e0 con 0x7f7fe40a1120 2024-05-08T23:44:07.627 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 10 on host 'smithi160' 2024-05-08T23:44:07.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 -- 172.21.15.160:0/2880247712 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7fe40a1120 msgr2=0x7f7fe40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:07.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7fe40a1120 0x7f7fe40a35e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0005fd0 tx=0x7f7ff0005d00 comp rx=0 tx=0).stop 2024-05-08T23:44:07.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 -- 172.21.15.160:0/2880247712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc100440 msgr2=0x7f7ffc197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:07.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc100440 0x7f7ffc197060 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00b500 tx=0x7f7fec00b8c0 comp rx=0 tx=0).stop 2024-05-08T23:44:07.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 -- 172.21.15.160:0/2880247712 shutdown_connections 2024-05-08T23:44:07.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc101b20 0x7f7ffc19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:07.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7fe40a1120 0x7f7fe40a35e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:07.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc100440 0x7f7ffc197060 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:07.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 --2- 172.21.15.160:0/2880247712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc101160 0x7f7ffc1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:07.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 -- 172.21.15.160:0/2880247712 >> 172.21.15.160:0/2880247712 conn(0x7f7ffc0fac40 msgr2=0x7f7ffc104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:07.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 -- 172.21.15.160:0/2880247712 shutdown_connections 2024-05-08T23:44:07.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-08T23:44:07.634+0000 7f80013b1700 1 -- 172.21.15.160:0/2880247712 wait complete. 2024-05-08T23:44:08.167 DEBUG:teuthology.orchestra.run.smithi160:osd.10> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T23:44:08.169 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-05-08T23:44:08.169 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T23:44:08.193 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:08 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.193 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:08 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.193 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:08 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:08.193 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:08 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:08.194 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:08 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.194 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:08 smithi160 ceph-mon[46152]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:08.194 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:08 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.194 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:08 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.194 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:08 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:08.199 INFO:journalctl@ceph.osd.10.smithi160.stdout:-- Logs begin at Wed 2024-05-08 23:21:47 UTC. -- 2024-05-08T23:44:08.403 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:44:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:08 smithi012 ceph-mon[42503]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:08 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:08.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:08 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:08 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:08 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:08.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:08 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:08.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:08 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:08 smithi204 ceph-mon[47373]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:08.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:08 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:08 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:08.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:08 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:09.702 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T23:44:09.702 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T23:44:09.702 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-08T23:44:09.702 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-08T23:44:09.702 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136103 s, 770 MB/s 2024-05-08T23:44:09.702 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-08T23:44:09.766 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 23:44:09 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-08T23:44:09.370+0000 7fcb375c67c0 -1 osd.10 0 log_to_monitors true 2024-05-08T23:44:10.098 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-05-08T23:44:10.319 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:44:10.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:10 smithi204 ceph-mon[47373]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:10.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:10 smithi204 ceph-mon[47373]: from='osd.10 [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:44:10.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:10 smithi204 ceph-mon[47373]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:44:10.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:10 smithi012 ceph-mon[42503]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:10.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:10 smithi012 ceph-mon[42503]: from='osd.10 [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:44:10.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:10 smithi012 ceph-mon[42503]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:44:10.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:10 smithi160 ceph-mon[46152]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:10.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:10 smithi160 ceph-mon[46152]: from='osd.10 [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:44:10.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:10 smithi160 ceph-mon[46152]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:44:11.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.303+0000 7fd904da8700 1 -- 172.21.15.204:0/2647095155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9001021e0 msgr2=0x7fd900102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:11.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.303+0000 7fd904da8700 1 --2- 172.21.15.204:0/2647095155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9001021e0 0x7fd900102640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009990 tx=0x7fd8f4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:44:11.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.303+0000 7fd904da8700 1 -- 172.21.15.204:0/2647095155 shutdown_connections 2024-05-08T23:44:11.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.303+0000 7fd904da8700 1 --2- 172.21.15.204:0/2647095155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd900102b80 0x7fd900109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:11.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.303+0000 7fd904da8700 1 --2- 172.21.15.204:0/2647095155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd900100fe0 0x7fd900101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:11.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.303+0000 7fd904da8700 1 --2- 172.21.15.204:0/2647095155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9001021e0 0x7fd900102640 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:11.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.303+0000 7fd904da8700 1 -- 172.21.15.204:0/2647095155 >> 172.21.15.204:0/2647095155 conn(0x7fd9000fc580 msgr2=0x7fd9000fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:11.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.303+0000 7fd904da8700 1 -- 172.21.15.204:0/2647095155 shutdown_connections 2024-05-08T23:44:11.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.303+0000 7fd904da8700 1 -- 172.21.15.204:0/2647095155 wait complete. 2024-05-08T23:44:11.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd904da8700 1 Processor -- start 2024-05-08T23:44:11.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd904da8700 1 -- start start 2024-05-08T23:44:11.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd904da8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd900100fe0 0x7fd900074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:11.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd904da8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9001021e0 0x7fd900073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:11.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd904da8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd900102b80 0x7fd9000738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:11.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd904da8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd900074070 con 0x7fd900100fe0 2024-05-08T23:44:11.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd904da8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000741e0 con 0x7fd900102b80 2024-05-08T23:44:11.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd904da8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001a0f00 con 0x7fd9001021e0 2024-05-08T23:44:11.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd8fdd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9001021e0 0x7fd900073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:11.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd8fdd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9001021e0 0x7fd900073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55520/0 (socket says 172.21.15.204:55520) 2024-05-08T23:44:11.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd8fdd9b700 1 -- 172.21.15.204:0/2545732708 learned_addr learned my addr 172.21.15.204:0/2545732708 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-08T23:44:11.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd8fed9d700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd900102b80 0x7fd9000738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:11.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd8fe59c700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd900100fe0 0x7fd900074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:11.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd8fdd9b700 1 -- 172.21.15.204:0/2545732708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd900102b80 msgr2=0x7fd9000738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:11.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd8fdd9b700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd900102b80 0x7fd9000738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:11.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd8fdd9b700 1 -- 172.21.15.204:0/2545732708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd900100fe0 msgr2=0x7fd900074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:11.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd8fdd9b700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd900100fe0 0x7fd900074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:11.304 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.304+0000 7fd8fdd9b700 1 -- 172.21.15.204:0/2545732708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f40095f0 con 0x7fd9001021e0 2024-05-08T23:44:11.304 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.305+0000 7fd8fdd9b700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9001021e0 0x7fd900073390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400b5c0 tx=0x7fd8f400f700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:11.305 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.309+0000 7fd8ef7fe700 1 -- 172.21.15.204:0/2545732708 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f401d070 con 0x7fd9001021e0 2024-05-08T23:44:11.305 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.309+0000 7fd8ef7fe700 1 -- 172.21.15.204:0/2545732708 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8f4003e20 con 0x7fd9001021e0 2024-05-08T23:44:11.305 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.309+0000 7fd904da8700 1 -- 172.21.15.204:0/2545732708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9001a1130 con 0x7fd9001021e0 2024-05-08T23:44:11.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.309+0000 7fd8ef7fe700 1 -- 172.21.15.204:0/2545732708 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40173f0 con 0x7fd9001021e0 2024-05-08T23:44:11.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.309+0000 7fd904da8700 1 -- 172.21.15.204:0/2545732708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9001a15a0 con 0x7fd9001021e0 2024-05-08T23:44:11.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.310+0000 7fd904da8700 1 -- 172.21.15.204:0/2545732708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9001083c0 con 0x7fd9001021e0 2024-05-08T23:44:11.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.311+0000 7fd8ef7fe700 1 -- 172.21.15.204:0/2545732708 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd8f4003960 con 0x7fd9001021e0 2024-05-08T23:44:11.308 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.312+0000 7fd8ef7fe700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd8e80a11a0 0x7fd8e80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:11.308 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.312+0000 7fd8ef7fe700 1 -- 172.21.15.204:0/2545732708 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd8f40cd360 con 0x7fd9001021e0 2024-05-08T23:44:11.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.312+0000 7fd8fe59c700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd8e80a11a0 0x7fd8e80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:11.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.313+0000 7fd8fe59c700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd8e80a11a0 0x7fd8e80a3660 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd900102040 tx=0x7fd8f000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:11.312 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.317+0000 7fd8ef7fe700 1 -- 172.21.15.204:0/2545732708 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd8f40972f0 con 0x7fd9001021e0 2024-05-08T23:44:11.511 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:11.515+0000 7fd904da8700 1 -- 172.21.15.204:0/2545732708 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9000613e0 con 0x7fd8e80a11a0 2024-05-08T23:44:11.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T23:44:11.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: osdmap e67: 11 total, 10 up, 11 in 2024-05-08T23:44:11.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:11.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='osd.10 [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:44:11.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:44:11.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi160 2024-05-08T23:44:11.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi160 to 4278M 2024-05-08T23:44:11.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:11.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:11.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:11 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T23:44:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: osdmap e67: 11 total, 10 up, 11 in 2024-05-08T23:44:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='osd.10 [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:44:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:44:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi160 2024-05-08T23:44:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi160 to 4278M 2024-05-08T23:44:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:11 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.766 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 23:44:11 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-08T23:44:11.318+0000 7fcb2c429700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T23:44:11.766 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 08 23:44:11 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-08T23:44:11.334+0000 7fcb28a1f700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:44:11.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T23:44:11.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: osdmap e67: 11 total, 10 up, 11 in 2024-05-08T23:44:11.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:11.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='osd.10 [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:44:11.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi160 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi160 to 4278M 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:11.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:11 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:12.494 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:12 smithi204 ceph-mon[47373]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:12.495 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:12 smithi204 ceph-mon[47373]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:44:12.495 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:12 smithi204 ceph-mon[47373]: osdmap e68: 11 total, 10 up, 11 in 2024-05-08T23:44:12.495 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:12.495 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:12.495 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:12 smithi204 ceph-mon[47373]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:44:12.495 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:44:12.495 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:44:12.495 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:12 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:12 smithi012 ceph-mon[42503]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:12 smithi012 ceph-mon[42503]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:44:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:12 smithi012 ceph-mon[42503]: osdmap e68: 11 total, 10 up, 11 in 2024-05-08T23:44:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:12 smithi012 ceph-mon[42503]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:44:12.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:44:12.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:44:12.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:12 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:12.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:12 smithi160 ceph-mon[46152]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:12.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:12 smithi160 ceph-mon[46152]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-08T23:44:12.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:12 smithi160 ceph-mon[46152]: osdmap e68: 11 total, 10 up, 11 in 2024-05-08T23:44:12.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:12 smithi160 ceph-mon[46152]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:44:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:44:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:44:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:12 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:13.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:13 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:44:13.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:13 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:44:13.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:13 smithi012 ceph-mon[42503]: osd.10 [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] boot 2024-05-08T23:44:13.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:13 smithi012 ceph-mon[42503]: osdmap e69: 11 total, 11 up, 11 in 2024-05-08T23:44:13.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:13 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:13.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:13 smithi012 ceph-mon[42503]: from='client.? 172.21.15.204:0/2500862973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9361b5a-5dd2-4920-b89f-d040b487b6d7"}]: dispatch 2024-05-08T23:44:13.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:13 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9361b5a-5dd2-4920-b89f-d040b487b6d7"}]: dispatch 2024-05-08T23:44:13.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:13 smithi012 ceph-mon[42503]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9361b5a-5dd2-4920-b89f-d040b487b6d7"}]': finished 2024-05-08T23:44:13.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:13 smithi012 ceph-mon[42503]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T23:44:13.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:13 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:13 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:44:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:13 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:44:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:13 smithi204 ceph-mon[47373]: osd.10 [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] boot 2024-05-08T23:44:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:13 smithi204 ceph-mon[47373]: osdmap e69: 11 total, 11 up, 11 in 2024-05-08T23:44:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:13 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:13 smithi204 ceph-mon[47373]: from='client.? 172.21.15.204:0/2500862973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9361b5a-5dd2-4920-b89f-d040b487b6d7"}]: dispatch 2024-05-08T23:44:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:13 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9361b5a-5dd2-4920-b89f-d040b487b6d7"}]: dispatch 2024-05-08T23:44:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:13 smithi204 ceph-mon[47373]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9361b5a-5dd2-4920-b89f-d040b487b6d7"}]': finished 2024-05-08T23:44:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:13 smithi204 ceph-mon[47373]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T23:44:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:13 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:13.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:13 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:44:13.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:13 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:44:13.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:13 smithi160 ceph-mon[46152]: osd.10 [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] boot 2024-05-08T23:44:13.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:13 smithi160 ceph-mon[46152]: osdmap e69: 11 total, 11 up, 11 in 2024-05-08T23:44:13.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:13 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:44:13.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:13 smithi160 ceph-mon[46152]: from='client.? 172.21.15.204:0/2500862973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9361b5a-5dd2-4920-b89f-d040b487b6d7"}]: dispatch 2024-05-08T23:44:13.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:13 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9361b5a-5dd2-4920-b89f-d040b487b6d7"}]: dispatch 2024-05-08T23:44:13.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:13 smithi160 ceph-mon[46152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9361b5a-5dd2-4920-b89f-d040b487b6d7"}]': finished 2024-05-08T23:44:13.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:13 smithi160 ceph-mon[46152]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T23:44:13.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:13 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:14.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:14 smithi012 ceph-mon[42503]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:14.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:14 smithi012 ceph-mon[42503]: from='client.? 172.21.15.204:0/620381045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:44:14.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:14 smithi012 ceph-mon[42503]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T23:44:14.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:14 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:14.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:14 smithi204 ceph-mon[47373]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:14.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:14 smithi204 ceph-mon[47373]: from='client.? 172.21.15.204:0/620381045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:44:14.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:14 smithi204 ceph-mon[47373]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T23:44:14.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:14 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:14.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:14 smithi160 ceph-mon[46152]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:44:14.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:14 smithi160 ceph-mon[46152]: from='client.? 172.21.15.204:0/620381045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:44:14.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:14 smithi160 ceph-mon[46152]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T23:44:14.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:14 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:16.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:16 smithi012 ceph-mon[42503]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:44:16.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:16 smithi012 ceph-mon[42503]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T23:44:16.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:16 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:16 smithi204 ceph-mon[47373]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:44:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:16 smithi204 ceph-mon[47373]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T23:44:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:16 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:16.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:16 smithi160 ceph-mon[46152]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:44:16.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:16 smithi160 ceph-mon[46152]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T23:44:16.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:16 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:17.555 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:17 smithi204 ceph-mon[47373]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T23:44:17.555 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:17 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:17.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:17 smithi012 ceph-mon[42503]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T23:44:17.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:17 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:17.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:17 smithi160 ceph-mon[46152]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T23:44:17.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:17 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:18.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:18 smithi012 ceph-mon[42503]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:44:18.719 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:18 smithi204 ceph-mon[47373]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:44:18.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:18 smithi160 ceph-mon[46152]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:44:20.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:20 smithi204 ceph-mon[47373]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T23:44:20.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:20 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T23:44:20.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:20 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:20.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:20 smithi204 ceph-mon[47373]: Deploying daemon osd.11 on smithi204 2024-05-08T23:44:20.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:20 smithi012 ceph-mon[42503]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T23:44:20.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:20 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T23:44:20.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:20 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:20.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:20 smithi012 ceph-mon[42503]: Deploying daemon osd.11 on smithi204 2024-05-08T23:44:20.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:20 smithi160 ceph-mon[46152]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T23:44:20.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:20 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T23:44:20.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:20 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:20.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:20 smithi160 ceph-mon[46152]: Deploying daemon osd.11 on smithi204 2024-05-08T23:44:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:22 smithi012 ceph-mon[42503]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-08T23:44:22.668 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:22 smithi204 ceph-mon[47373]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-08T23:44:22.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:22 smithi160 ceph-mon[46152]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-08T23:44:24.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:24 smithi204 ceph-mon[47373]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T23:44:24.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:24 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:24.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:24 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:24.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:24 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:24 smithi160 ceph-mon[46152]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T23:44:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:24 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:24 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:24 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:24.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:24 smithi012 ceph-mon[42503]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T23:44:24.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:24.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:24.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:24 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:25.517 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-05-08T23:44:25.517 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.520+0000 7fd8ef7fe700 1 -- 172.21.15.204:0/2545732708 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd9000613e0 con 0x7fd8e80a11a0 2024-05-08T23:44:25.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.522+0000 7fd904da8700 1 -- 172.21.15.204:0/2545732708 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd8e80a11a0 msgr2=0x7fd8e80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:25.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd8e80a11a0 0x7fd8e80a3660 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd900102040 tx=0x7fd8f000a3b0 comp rx=0 tx=0).stop 2024-05-08T23:44:25.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 -- 172.21.15.204:0/2545732708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9001021e0 msgr2=0x7fd900073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:25.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9001021e0 0x7fd900073390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400b5c0 tx=0x7fd8f400f700 comp rx=0 tx=0).stop 2024-05-08T23:44:25.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 -- 172.21.15.204:0/2545732708 shutdown_connections 2024-05-08T23:44:25.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd8e80a11a0 0x7fd8e80a3660 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:25.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd900102b80 0x7fd9000738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:25.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd900100fe0 0x7fd900074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:25.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 --2- 172.21.15.204:0/2545732708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9001021e0 0x7fd900073390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:25.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 -- 172.21.15.204:0/2545732708 >> 172.21.15.204:0/2545732708 conn(0x7fd9000fc580 msgr2=0x7fd900105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:25.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 -- 172.21.15.204:0/2545732708 shutdown_connections 2024-05-08T23:44:25.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:25.523+0000 7fd904da8700 1 -- 172.21.15.204:0/2545732708 wait complete. 2024-05-08T23:44:26.099 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T23:44:26.102 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T23:44:26.102 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T23:44:26.132 INFO:journalctl@ceph.osd.11.smithi204.stdout:-- Logs begin at Wed 2024-05-08 23:21:56 UTC. -- 2024-05-08T23:44:26.224 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:25 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.224 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:25 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.224 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:25 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:26.224 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:25 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:26.224 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:25 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.224 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:25 smithi204 ceph-mon[47373]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:44:26.224 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:25 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.224 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:25 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:26.225 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:25 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:25 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:25 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:25 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:26.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:25 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:26.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:25 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:25 smithi160 ceph-mon[46152]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:44:26.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:25 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:25 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:26.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:25 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.360 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:44:26.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:25 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:25 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:25 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:26.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:25 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:26.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:25 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:25 smithi012 ceph-mon[42503]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:44:26.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:25 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:26.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:25 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:26.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:25 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.374+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/131426692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b581001e0 msgr2=0x7f9b58100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.374+0000 7f9b5fdc1700 1 --2- 172.21.15.12:0/131426692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b581001e0 0x7f9b58100660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009990 tx=0x7f9b48009ca0 comp rx=0 tx=0).stop 2024-05-08T23:44:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.374+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/131426692 shutdown_connections 2024-05-08T23:44:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.374+0000 7f9b5fdc1700 1 --2- 172.21.15.12:0/131426692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b58100ba0 0x7f9b58101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.374+0000 7f9b5fdc1700 1 --2- 172.21.15.12:0/131426692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b580ff880 0x7f9b580ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.374+0000 7f9b5fdc1700 1 --2- 172.21.15.12:0/131426692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b581001e0 0x7f9b58100660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.374+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/131426692 >> 172.21.15.12:0/131426692 conn(0x7f9b580fb4a0 msgr2=0x7f9b580fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.375+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/131426692 shutdown_connections 2024-05-08T23:44:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.375+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/131426692 wait complete. 2024-05-08T23:44:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.375+0000 7f9b5fdc1700 1 Processor -- start 2024-05-08T23:44:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.375+0000 7f9b5fdc1700 1 -- start start 2024-05-08T23:44:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5fdc1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b580ff880 0x7f9b58199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5fdc1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b581001e0 0x7f9b58199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5fdc1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b58100ba0 0x7f9b581a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5fdc1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b581a0bc0 con 0x7f9b580ff880 2024-05-08T23:44:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5fdc1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b581a0d00 con 0x7f9b581001e0 2024-05-08T23:44:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5fdc1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b581a0e90 con 0x7f9b58100ba0 2024-05-08T23:44:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5db5d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b580ff880 0x7f9b58199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5db5d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b580ff880 0x7f9b58199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34208/0 (socket says 172.21.15.12:34208) 2024-05-08T23:44:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5db5d700 1 -- 172.21.15.12:0/805763537 learned_addr learned my addr 172.21.15.12:0/805763537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:27.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5d35c700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b581001e0 0x7f9b58199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:27.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5e35e700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b58100ba0 0x7f9b581a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:27.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5db5d700 1 -- 172.21.15.12:0/805763537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b58100ba0 msgr2=0x7f9b581a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:27.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5db5d700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b58100ba0 0x7f9b581a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:27.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5db5d700 1 -- 172.21.15.12:0/805763537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b581001e0 msgr2=0x7f9b58199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:27.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5db5d700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b581001e0 0x7f9b58199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:27.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5db5d700 1 -- 172.21.15.12:0/805763537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b480095f0 con 0x7f9b580ff880 2024-05-08T23:44:27.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5d35c700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b581001e0 0x7f9b58199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:44:27.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b5db5d700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b580ff880 0x7f9b58199120 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400e9d0 tx=0x7f9b5400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:27.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.376+0000 7f9b4effd700 1 -- 172.21.15.12:0/805763537 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54009790 con 0x7f9b580ff880 2024-05-08T23:44:27.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.377+0000 7f9b4effd700 1 -- 172.21.15.12:0/805763537 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b5400cbd0 con 0x7f9b580ff880 2024-05-08T23:44:27.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.377+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/805763537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b581a1170 con 0x7f9b580ff880 2024-05-08T23:44:27.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.377+0000 7f9b4effd700 1 -- 172.21.15.12:0/805763537 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54018680 con 0x7f9b580ff880 2024-05-08T23:44:27.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.377+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/805763537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b581a16c0 con 0x7f9b580ff880 2024-05-08T23:44:27.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.378+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/805763537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5810a660 con 0x7f9b580ff880 2024-05-08T23:44:27.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.379+0000 7f9b4effd700 1 -- 172.21.15.12:0/805763537 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9b54010430 con 0x7f9b580ff880 2024-05-08T23:44:27.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.380+0000 7f9b4effd700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b440a11a0 0x7f9b440a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:27.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.380+0000 7f9b5d35c700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b440a11a0 0x7f9b440a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:27.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.380+0000 7f9b4effd700 1 -- 172.21.15.12:0/805763537 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f9b54014070 con 0x7f9b580ff880 2024-05-08T23:44:27.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.380+0000 7f9b5d35c700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b440a11a0 0x7f9b440a3660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800b2e0 tx=0x7f9b4800b770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:27.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.384+0000 7f9b4effd700 1 -- 172.21.15.12:0/805763537 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9b54096c40 con 0x7f9b580ff880 2024-05-08T23:44:27.514 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 08 23:44:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-08T23:44:27.245+0000 7f2313c0d7c0 -1 osd.11 0 log_to_monitors true 2024-05-08T23:44:27.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:27 smithi204 ceph-mon[47373]: from='osd.11 [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:44:27.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:27 smithi204 ceph-mon[47373]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:44:27.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.596+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/805763537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f9b5810eb70 con 0x7f9b580ff880 2024-05-08T23:44:27.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.596+0000 7f9b4effd700 1 -- 172.21.15.12:0/805763537 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) v1 ==== 74+0+133 (secure 0 0 0) 0x7f9b5409a260 con 0x7f9b580ff880 2024-05-08T23:44:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/805763537 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b440a11a0 msgr2=0x7f9b440a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b440a11a0 0x7f9b440a3660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800b2e0 tx=0x7f9b4800b770 comp rx=0 tx=0).stop 2024-05-08T23:44:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/805763537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b580ff880 msgr2=0x7f9b58199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b580ff880 0x7f9b58199120 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400e9d0 tx=0x7f9b5400ed90 comp rx=0 tx=0).stop 2024-05-08T23:44:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/805763537 shutdown_connections 2024-05-08T23:44:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b58100ba0 0x7f9b581a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b580ff880 0x7f9b58199120 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b440a11a0 0x7f9b440a3660 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 --2- 172.21.15.12:0/805763537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b581001e0 0x7f9b58199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/805763537 >> 172.21.15.12:0/805763537 conn(0x7f9b580fb4a0 msgr2=0x7f9b580fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:27.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/805763537 shutdown_connections 2024-05-08T23:44:27.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:27.599+0000 7f9b5fdc1700 1 -- 172.21.15.12:0/805763537 wait complete. 2024-05-08T23:44:27.622 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:27 smithi012 ceph-mon[42503]: from='osd.11 [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:44:27.622 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:27 smithi012 ceph-mon[42503]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:44:27.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:27 smithi160 ceph-mon[46152]: from='osd.11 [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:44:27.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:27 smithi160 ceph-mon[46152]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:44:27.973 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1715211852,"num_in_osds":12,"osd_in_since":1715211853,"num_remapped_pgs":0} 2024-05-08T23:44:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-08T23:44:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T23:44:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: osdmap e74: 12 total, 11 up, 12 in 2024-05-08T23:44:28.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='osd.11 [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:44:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:44:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/805763537' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:44:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:28.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:28.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:28 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: osdmap e74: 12 total, 11 up, 12 in 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='osd.11 [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/805763537' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:28 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-08T23:44:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T23:44:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: osdmap e74: 12 total, 11 up, 12 in 2024-05-08T23:44:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='osd.11 [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:44:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-08T23:44:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/805763537' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:44:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:44:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:28 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:28.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T23:44:29.232 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:44:29.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:29 smithi204 ceph-mon[47373]: Detected new or changed devices on smithi204 2024-05-08T23:44:29.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:29 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi204 to 4278M 2024-05-08T23:44:29.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:29 smithi204 ceph-mon[47373]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:44:29.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:29 smithi204 ceph-mon[47373]: osdmap e75: 12 total, 11 up, 12 in 2024-05-08T23:44:29.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:29 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:29.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:29 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:29.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:29 smithi204 ceph-mon[47373]: from='osd.11 ' entity='osd.11' 2024-05-08T23:44:29.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 08 23:44:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-08T23:44:29.119+0000 7f2308a70700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T23:44:29.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 08 23:44:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-08T23:44:29.141+0000 7f230105e700 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:44:29.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:29 smithi012 ceph-mon[42503]: Detected new or changed devices on smithi204 2024-05-08T23:44:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:29 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi204 to 4278M 2024-05-08T23:44:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:29 smithi012 ceph-mon[42503]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:44:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:29 smithi012 ceph-mon[42503]: osdmap e75: 12 total, 11 up, 12 in 2024-05-08T23:44:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:29 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:29 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:29 smithi012 ceph-mon[42503]: from='osd.11 ' entity='osd.11' 2024-05-08T23:44:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:29 smithi160 ceph-mon[46152]: Detected new or changed devices on smithi204 2024-05-08T23:44:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:29 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi204 to 4278M 2024-05-08T23:44:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:29 smithi160 ceph-mon[46152]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-08T23:44:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:29 smithi160 ceph-mon[46152]: osdmap e75: 12 total, 11 up, 12 in 2024-05-08T23:44:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:29 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:29 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:29 smithi160 ceph-mon[46152]: from='osd.11 ' entity='osd.11' 2024-05-08T23:44:30.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.304+0000 7fb3c7c58700 1 -- 172.21.15.12:0/1594677109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c0100fe0 msgr2=0x7fb3c0101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:30.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.304+0000 7fb3c7c58700 1 --2- 172.21.15.12:0/1594677109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c0100fe0 0x7fb3c0101400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40099c0 tx=0x7fb3b4009cd0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.304+0000 7fb3c7c58700 1 -- 172.21.15.12:0/1594677109 shutdown_connections 2024-05-08T23:44:30.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.304+0000 7fb3c7c58700 1 --2- 172.21.15.12:0/1594677109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3c0102b80 0x7fb3c0109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.304+0000 7fb3c7c58700 1 --2- 172.21.15.12:0/1594677109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c0100fe0 0x7fb3c0101400 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.304+0000 7fb3c7c58700 1 --2- 172.21.15.12:0/1594677109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3c01021e0 0x7fb3c0102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.304+0000 7fb3c7c58700 1 -- 172.21.15.12:0/1594677109 >> 172.21.15.12:0/1594677109 conn(0x7fb3c00fc580 msgr2=0x7fb3c00fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.304+0000 7fb3c7c58700 1 -- 172.21.15.12:0/1594677109 shutdown_connections 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.304+0000 7fb3c7c58700 1 -- 172.21.15.12:0/1594677109 wait complete. 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c7c58700 1 Processor -- start 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c7c58700 1 -- start start 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c7c58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c0100fe0 0x7fb3c0074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c7c58700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3c01021e0 0x7fb3c0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c7c58700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3c0102b80 0x7fb3c00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c7c58700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c0074070 con 0x7fb3c0100fe0 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c7c58700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c00741e0 con 0x7fb3c01021e0 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c7c58700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c01a0f70 con 0x7fb3c0102b80 2024-05-08T23:44:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c59f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c0100fe0 0x7fb3c0074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c59f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c0100fe0 0x7fb3c0074cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39262/0 (socket says 172.21.15.12:39262) 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.305+0000 7fb3c59f4700 1 -- 172.21.15.12:0/2289035129 learned_addr learned my addr 172.21.15.12:0/2289035129 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c51f3700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3c01021e0 0x7fb3c0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c61f5700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3c0102b80 0x7fb3c00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c59f4700 1 -- 172.21.15.12:0/2289035129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3c0102b80 msgr2=0x7fb3c00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c59f4700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3c0102b80 0x7fb3c00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c59f4700 1 -- 172.21.15.12:0/2289035129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3c01021e0 msgr2=0x7fb3c0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c59f4700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3c01021e0 0x7fb3c0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c59f4700 1 -- 172.21.15.12:0/2289035129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b40095f0 con 0x7fb3c0100fe0 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c61f5700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3c0102b80 0x7fb3c00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c59f4700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c0100fe0 0x7fb3c0074cf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4003820 tx=0x7fb3b4005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3b2ffd700 1 -- 172.21.15.12:0/2289035129 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b401d070 con 0x7fb3c0100fe0 2024-05-08T23:44:30.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3b2ffd700 1 -- 172.21.15.12:0/2289035129 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3b4004110 con 0x7fb3c0100fe0 2024-05-08T23:44:30.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3b2ffd700 1 -- 172.21.15.12:0/2289035129 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4022e40 con 0x7fb3c0100fe0 2024-05-08T23:44:30.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c7c58700 1 -- 172.21.15.12:0/2289035129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c01a11a0 con 0x7fb3c0100fe0 2024-05-08T23:44:30.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.306+0000 7fb3c7c58700 1 -- 172.21.15.12:0/2289035129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c01a1640 con 0x7fb3c0100fe0 2024-05-08T23:44:30.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.308+0000 7fb3c7c58700 1 -- 172.21.15.12:0/2289035129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c0108560 con 0x7fb3c0100fe0 2024-05-08T23:44:30.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.309+0000 7fb3b2ffd700 1 -- 172.21.15.12:0/2289035129 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb3b4022570 con 0x7fb3c0100fe0 2024-05-08T23:44:30.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.309+0000 7fb3b2ffd700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb3ac0a11a0 0x7fb3ac0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:30.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.309+0000 7fb3c51f3700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb3ac0a11a0 0x7fb3ac0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:30.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.309+0000 7fb3b2ffd700 1 -- 172.21.15.12:0/2289035129 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb3b40cd790 con 0x7fb3c0100fe0 2024-05-08T23:44:30.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.310+0000 7fb3c51f3700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb3ac0a11a0 0x7fb3ac0a3660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005950 tx=0x7fb3bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:30.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.314+0000 7fb3b2ffd700 1 -- 172.21.15.12:0/2289035129 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb3b4097450 con 0x7fb3c0100fe0 2024-05-08T23:44:30.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:30 smithi012 ceph-mon[42503]: purged_snaps scrub starts 2024-05-08T23:44:30.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:30 smithi012 ceph-mon[42503]: purged_snaps scrub ok 2024-05-08T23:44:30.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:30 smithi012 ceph-mon[42503]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:44:30.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:30 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:30.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:30 smithi012 ceph-mon[42503]: osd.11 [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] boot 2024-05-08T23:44:30.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:30 smithi012 ceph-mon[42503]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T23:44:30.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:30 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:30.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.527+0000 7fb3c7c58700 1 -- 172.21.15.12:0/2289035129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fb3c010c7e0 con 0x7fb3c0100fe0 2024-05-08T23:44:30.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.527+0000 7fb3b2ffd700 1 -- 172.21.15.12:0/2289035129 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7fb3b409fe60 con 0x7fb3c0100fe0 2024-05-08T23:44:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:30.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 -- 172.21.15.12:0/2289035129 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb3ac0a11a0 msgr2=0x7fb3ac0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:30.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb3ac0a11a0 0x7fb3ac0a3660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005950 tx=0x7fb3bc0058e0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 -- 172.21.15.12:0/2289035129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c0100fe0 msgr2=0x7fb3c0074cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:30.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c0100fe0 0x7fb3c0074cf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4003820 tx=0x7fb3b4005ad0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 -- 172.21.15.12:0/2289035129 shutdown_connections 2024-05-08T23:44:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3c0102b80 0x7fb3c00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c0100fe0 0x7fb3c0074cf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb3ac0a11a0 0x7fb3ac0a3660 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 --2- 172.21.15.12:0/2289035129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3c01021e0 0x7fb3c0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 -- 172.21.15.12:0/2289035129 >> 172.21.15.12:0/2289035129 conn(0x7fb3c00fc580 msgr2=0x7fb3c01056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 -- 172.21.15.12:0/2289035129 shutdown_connections 2024-05-08T23:44:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:30.529+0000 7fb3c7c58700 1 -- 172.21.15.12:0/2289035129 wait complete. 2024-05-08T23:44:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:30 smithi204 ceph-mon[47373]: purged_snaps scrub starts 2024-05-08T23:44:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:30 smithi204 ceph-mon[47373]: purged_snaps scrub ok 2024-05-08T23:44:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:30 smithi204 ceph-mon[47373]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:44:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:30 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:30 smithi204 ceph-mon[47373]: osd.11 [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] boot 2024-05-08T23:44:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:30 smithi204 ceph-mon[47373]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T23:44:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:30 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:30 smithi160 ceph-mon[46152]: purged_snaps scrub starts 2024-05-08T23:44:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:30 smithi160 ceph-mon[46152]: purged_snaps scrub ok 2024-05-08T23:44:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:30 smithi160 ceph-mon[46152]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:44:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:30 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:30 smithi160 ceph-mon[46152]: osd.11 [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] boot 2024-05-08T23:44:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:30 smithi160 ceph-mon[46152]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T23:44:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:30 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:44:30.907 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1715211870,"num_in_osds":12,"osd_in_since":1715211853,"num_remapped_pgs":0} 2024-05-08T23:44:30.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T23:44:31.162 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:44:31.560 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:31 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2289035129' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:44:31.560 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:31 smithi012 ceph-mon[42503]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T23:44:31.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:31 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2289035129' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:44:31.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:31 smithi204 ceph-mon[47373]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T23:44:31.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:31 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2289035129' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:44:31.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:31 smithi160 ceph-mon[46152]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T23:44:32.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.080+0000 7fe3f72e5700 1 -- 172.21.15.12:0/306195608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0100ff0 msgr2=0x7fe3f0101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:32.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.080+0000 7fe3f72e5700 1 --2- 172.21.15.12:0/306195608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0100ff0 0x7fe3f0101450 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009990 tx=0x7fe3ec009ca0 comp rx=0 tx=0).stop 2024-05-08T23:44:32.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.080+0000 7fe3f72e5700 1 -- 172.21.15.12:0/306195608 shutdown_connections 2024-05-08T23:44:32.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.080+0000 7fe3f72e5700 1 --2- 172.21.15.12:0/306195608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f0101990 0x7fe3f010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:32.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.080+0000 7fe3f72e5700 1 --2- 172.21.15.12:0/306195608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f0107030 0x7fe3f0107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:32.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.080+0000 7fe3f72e5700 1 --2- 172.21.15.12:0/306195608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0100ff0 0x7fe3f0101450 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:32.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.080+0000 7fe3f72e5700 1 -- 172.21.15.12:0/306195608 >> 172.21.15.12:0/306195608 conn(0x7fe3f0078190 msgr2=0x7fe3f00785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:32.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.080+0000 7fe3f72e5700 1 -- 172.21.15.12:0/306195608 shutdown_connections 2024-05-08T23:44:32.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.080+0000 7fe3f72e5700 1 -- 172.21.15.12:0/306195608 wait complete. 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f72e5700 1 Processor -- start 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f72e5700 1 -- start start 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f72e5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f0100ff0 0x7fe3f0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f72e5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0101990 0x7fe3f0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f72e5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f0107030 0x7fe3f019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f72e5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019c330 con 0x7fe3f0101990 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f72e5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019c4a0 con 0x7fe3f0107030 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f72e5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019c660 con 0x7fe3f0100ff0 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f4880700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0101990 0x7fe3f0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f4880700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0101990 0x7fe3f0197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39284/0 (socket says 172.21.15.12:39284) 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f4880700 1 -- 172.21.15.12:0/3700454022 learned_addr learned my addr 172.21.15.12:0/3700454022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:32.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f5081700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f0100ff0 0x7fe3f0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f5882700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f0107030 0x7fe3f019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f4880700 1 -- 172.21.15.12:0/3700454022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f0100ff0 msgr2=0x7fe3f0197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f4880700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f0100ff0 0x7fe3f0197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f4880700 1 -- 172.21.15.12:0/3700454022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f0107030 msgr2=0x7fe3f019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f4880700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f0107030 0x7fe3f019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f4880700 1 -- 172.21.15.12:0/3700454022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0095f0 con 0x7fe3f0101990 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.081+0000 7fe3f5882700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f0107030 0x7fe3f019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.082+0000 7fe3f4880700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0101990 0x7fe3f0197970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec005910 tx=0x7fe3ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.082+0000 7fe3e67fc700 1 -- 172.21.15.12:0/3700454022 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec01d070 con 0x7fe3f0101990 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.082+0000 7fe3e67fc700 1 -- 172.21.15.12:0/3700454022 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3ec022470 con 0x7fe3f0101990 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.082+0000 7fe3e67fc700 1 -- 172.21.15.12:0/3700454022 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec017650 con 0x7fe3f0101990 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.082+0000 7fe3f72e5700 1 -- 172.21.15.12:0/3700454022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f019c8e0 con 0x7fe3f0101990 2024-05-08T23:44:32.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.082+0000 7fe3f72e5700 1 -- 172.21.15.12:0/3700454022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f01a3fd0 con 0x7fe3f0101990 2024-05-08T23:44:32.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.083+0000 7fe3f72e5700 1 -- 172.21.15.12:0/3700454022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f0109550 con 0x7fe3f0101990 2024-05-08T23:44:32.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.085+0000 7fe3e67fc700 1 -- 172.21.15.12:0/3700454022 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe3ec0225e0 con 0x7fe3f0101990 2024-05-08T23:44:32.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.085+0000 7fe3e67fc700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe3dc0a11a0 0x7fe3dc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:32.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.085+0000 7fe3e67fc700 1 -- 172.21.15.12:0/3700454022 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe3ec0cd440 con 0x7fe3f0101990 2024-05-08T23:44:32.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.085+0000 7fe3f5081700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe3dc0a11a0 0x7fe3dc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:32.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.086+0000 7fe3f5081700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe3dc0a11a0 0x7fe3dc0a3660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0005950 tx=0x7fe3e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.089+0000 7fe3e67fc700 1 -- 172.21.15.12:0/3700454022 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe3ec05ed60 con 0x7fe3f0101990 2024-05-08T23:44:32.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.301+0000 7fe3f72e5700 1 -- 172.21.15.12:0/3700454022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3f01026e0 con 0x7fe3f0101990 2024-05-08T23:44:32.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.302+0000 7fe3e67fc700 1 -- 172.21.15.12:0/3700454022 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18408 (secure 0 0 0) 0x7fe3ec097100 con 0x7fe3f0101990 2024-05-08T23:44:32.299 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:32.299 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":77,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:44:31.131279+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:44:32.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 -- 172.21.15.12:0/3700454022 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe3dc0a11a0 msgr2=0x7fe3dc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:32.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe3dc0a11a0 0x7fe3dc0a3660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0005950 tx=0x7fe3e000b410 comp rx=0 tx=0).stop 2024-05-08T23:44:32.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 -- 172.21.15.12:0/3700454022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0101990 msgr2=0x7fe3f0197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:32.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0101990 0x7fe3f0197970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec005910 tx=0x7fe3ec004730 comp rx=0 tx=0).stop 2024-05-08T23:44:32.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 -- 172.21.15.12:0/3700454022 shutdown_connections 2024-05-08T23:44:32.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe3dc0a11a0 0x7fe3dc0a3660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:32.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f0107030 0x7fe3f019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:32.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f0100ff0 0x7fe3f0197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:32.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 --2- 172.21.15.12:0/3700454022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0101990 0x7fe3f0197970 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:32.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 -- 172.21.15.12:0/3700454022 >> 172.21.15.12:0/3700454022 conn(0x7fe3f0078190 msgr2=0x7fe3f00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:32.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 -- 172.21.15.12:0/3700454022 shutdown_connections 2024-05-08T23:44:32.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:32.305+0000 7fe3f72e5700 1 -- 172.21.15.12:0/3700454022 wait complete. 2024-05-08T23:44:32.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:32 smithi012 ceph-mon[42503]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:32.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:32 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3700454022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:44:32.690 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:44:32.690 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T23:44:32.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:32 smithi204 ceph-mon[47373]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:32.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:32 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3700454022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:44:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:32 smithi160 ceph-mon[46152]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:32 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3700454022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:44:32.944 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:44:34.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.043+0000 7f835b373700 1 -- 172.21.15.12:0/992901807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354100850 msgr2=0x7f835410ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:34.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.043+0000 7f835b373700 1 --2- 172.21.15.12:0/992901807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354100850 0x7f835410ccf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f834c00b0a0 tx=0x7f834c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:44:34.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.043+0000 7f835b373700 1 -- 172.21.15.12:0/992901807 shutdown_connections 2024-05-08T23:44:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.043+0000 7f835b373700 1 --2- 172.21.15.12:0/992901807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354100850 0x7f835410ccf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.043+0000 7f835b373700 1 --2- 172.21.15.12:0/992901807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8354105f40 0x7f8354106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.043+0000 7f835b373700 1 --2- 172.21.15.12:0/992901807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83540ffe90 0x7f8354100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.043+0000 7f835b373700 1 -- 172.21.15.12:0/992901807 >> 172.21.15.12:0/992901807 conn(0x7f8354076e70 msgr2=0x7f8354077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.043+0000 7f835b373700 1 -- 172.21.15.12:0/992901807 shutdown_connections 2024-05-08T23:44:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.043+0000 7f835b373700 1 -- 172.21.15.12:0/992901807 wait complete. 2024-05-08T23:44:34.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835b373700 1 Processor -- start 2024-05-08T23:44:34.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835b373700 1 -- start start 2024-05-08T23:44:34.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835b373700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83540ffe90 0x7f8354197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835b373700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354100850 0x7f83541978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835b373700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354105f40 0x7f835419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835b373700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835419c270 con 0x7f8354100850 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835b373700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835419c3e0 con 0x7f83540ffe90 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835b373700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835419c5a0 con 0x7f8354105f40 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835890e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354100850 0x7f83541978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835890e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354100850 0x7f83541978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39314/0 (socket says 172.21.15.12:39314) 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835890e700 1 -- 172.21.15.12:0/4170489882 learned_addr learned my addr 172.21.15.12:0/4170489882 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f835910f700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83540ffe90 0x7f8354197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.044+0000 7f8359910700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354105f40 0x7f835419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f835890e700 1 -- 172.21.15.12:0/4170489882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354105f40 msgr2=0x7f835419bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f835890e700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354105f40 0x7f835419bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f835890e700 1 -- 172.21.15.12:0/4170489882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83540ffe90 msgr2=0x7f8354197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f835890e700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83540ffe90 0x7f8354197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f835890e700 1 -- 172.21.15.12:0/4170489882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c009d00 con 0x7f8354100850 2024-05-08T23:44:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f835890e700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354100850 0x7f83541978b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f834400d950 tx=0x7f834400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f834a7fc700 1 -- 172.21.15.12:0/4170489882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834400f890 con 0x7f8354100850 2024-05-08T23:44:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f835b373700 1 -- 172.21.15.12:0/4170489882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835419c880 con 0x7f8354100850 2024-05-08T23:44:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f834a7fc700 1 -- 172.21.15.12:0/4170489882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8344010460 con 0x7f8354100850 2024-05-08T23:44:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f834a7fc700 1 -- 172.21.15.12:0/4170489882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834400e610 con 0x7f8354100850 2024-05-08T23:44:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.045+0000 7f835b373700 1 -- 172.21.15.12:0/4170489882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83541a40e0 con 0x7f8354100850 2024-05-08T23:44:34.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.046+0000 7f835b373700 1 -- 172.21.15.12:0/4170489882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8354101580 con 0x7f8354100850 2024-05-08T23:44:34.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.047+0000 7f834a7fc700 1 -- 172.21.15.12:0/4170489882 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f83440108d0 con 0x7f8354100850 2024-05-08T23:44:34.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.048+0000 7f834a7fc700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f83400a1120 0x7f83400a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:34.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.048+0000 7f835910f700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f83400a1120 0x7f83400a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:34.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.048+0000 7f834a7fc700 1 -- 172.21.15.12:0/4170489882 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f83440cc200 con 0x7f8354100850 2024-05-08T23:44:34.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.049+0000 7f835910f700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f83400a1120 0x7f83400a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f835000ac50 tx=0x7f835000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:34.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.053+0000 7f834a7fc700 1 -- 172.21.15.12:0/4170489882 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8344095e30 con 0x7f8354100850 2024-05-08T23:44:34.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.266+0000 7f835b373700 1 -- 172.21.15.12:0/4170489882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f835404ec60 con 0x7f8354100850 2024-05-08T23:44:34.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.266+0000 7f834a7fc700 1 -- 172.21.15.12:0/4170489882 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8344099450 con 0x7f8354100850 2024-05-08T23:44:34.263 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-08T23:44:34.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 -- 172.21.15.12:0/4170489882 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f83400a1120 msgr2=0x7f83400a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:34.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f83400a1120 0x7f83400a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f835000ac50 tx=0x7f835000a380 comp rx=0 tx=0).stop 2024-05-08T23:44:34.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 -- 172.21.15.12:0/4170489882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354100850 msgr2=0x7f83541978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354100850 0x7f83541978b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f834400d950 tx=0x7f834400dc60 comp rx=0 tx=0).stop 2024-05-08T23:44:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 -- 172.21.15.12:0/4170489882 shutdown_connections 2024-05-08T23:44:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354105f40 0x7f835419bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f83400a1120 0x7f83400a35e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83540ffe90 0x7f8354197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 --2- 172.21.15.12:0/4170489882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354100850 0x7f83541978b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 -- 172.21.15.12:0/4170489882 >> 172.21.15.12:0/4170489882 conn(0x7f8354076e70 msgr2=0x7f83540ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 -- 172.21.15.12:0/4170489882 shutdown_connections 2024-05-08T23:44:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:34.269+0000 7f835b373700 1 -- 172.21.15.12:0/4170489882 wait complete. 2024-05-08T23:44:34.605 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:34 smithi012 ceph-mon[42503]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:34.606 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:34 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/4170489882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:44:34.664 INFO:tasks.cephadm:Adding mds.a on smithi012 2024-05-08T23:44:34.664 INFO:tasks.cephadm:Adding mds.d on smithi012 2024-05-08T23:44:34.665 INFO:tasks.cephadm:Adding mds.g on smithi012 2024-05-08T23:44:34.665 INFO:tasks.cephadm:Adding mds.j on smithi012 2024-05-08T23:44:34.665 INFO:tasks.cephadm:Adding mds.b on smithi160 2024-05-08T23:44:34.665 INFO:tasks.cephadm:Adding mds.e on smithi160 2024-05-08T23:44:34.665 INFO:tasks.cephadm:Adding mds.h on smithi160 2024-05-08T23:44:34.665 INFO:tasks.cephadm:Adding mds.k on smithi160 2024-05-08T23:44:34.665 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-05-08T23:44:34.665 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-05-08T23:44:34.665 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-05-08T23:44:34.665 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-05-08T23:44:34.665 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-05-08T23:44:34.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:34 smithi204 ceph-mon[47373]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:34.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:34 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/4170489882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:44:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:34 smithi160 ceph-mon[46152]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:34 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/4170489882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:44:34.902 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:44:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.872+0000 7f6bf9488700 1 -- 172.21.15.204:0/2649190448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf41021e0 msgr2=0x7f6bf4102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.872+0000 7f6bf9488700 1 --2- 172.21.15.204:0/2649190448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf41021e0 0x7f6bf4102640 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009990 tx=0x7f6be4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:44:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.872+0000 7f6bf9488700 1 -- 172.21.15.204:0/2649190448 shutdown_connections 2024-05-08T23:44:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.872+0000 7f6bf9488700 1 --2- 172.21.15.204:0/2649190448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bf4102b80 0x7f6bf4109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.872+0000 7f6bf9488700 1 --2- 172.21.15.204:0/2649190448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bf4100fe0 0x7f6bf4101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.872+0000 7f6bf9488700 1 --2- 172.21.15.204:0/2649190448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf41021e0 0x7f6bf4102640 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.872+0000 7f6bf9488700 1 -- 172.21.15.204:0/2649190448 >> 172.21.15.204:0/2649190448 conn(0x7f6bf40fc5a0 msgr2=0x7f6bf40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.872+0000 7f6bf9488700 1 -- 172.21.15.204:0/2649190448 shutdown_connections 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.872+0000 7f6bf9488700 1 -- 172.21.15.204:0/2649190448 wait complete. 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf9488700 1 Processor -- start 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf9488700 1 -- start start 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf9488700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf4100fe0 0x7f6bf4196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf9488700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bf41021e0 0x7f6bf4197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf9488700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bf4102b80 0x7f6bf419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf9488700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf419ea40 con 0x7f6bf4102b80 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf9488700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf419eb80 con 0x7f6bf41021e0 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf9488700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf419ed10 con 0x7f6bf4100fe0 2024-05-08T23:44:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf4100fe0 0x7f6bf4196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf4100fe0 0x7f6bf4196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43866/0 (socket says 172.21.15.204:43866) 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.873+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3710437639 learned_addr learned my addr 172.21.15.204:0/3710437639 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf37fe700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bf4102b80 0x7f6bf419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf27fc700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bf41021e0 0x7f6bf4197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3710437639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bf41021e0 msgr2=0x7f6bf4197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf2ffd700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bf41021e0 0x7f6bf4197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3710437639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bf4102b80 msgr2=0x7f6bf419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf2ffd700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bf4102b80 0x7f6bf419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf2ffd700 1 -- 172.21.15.204:0/3710437639 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40095f0 con 0x7f6bf4100fe0 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf27fc700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bf41021e0 0x7f6bf4197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf2ffd700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf4100fe0 0x7f6bf4196f00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00d9b0 tx=0x7f6bdc00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bebfff700 1 -- 172.21.15.204:0/3710437639 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc004020 con 0x7f6bf4100fe0 2024-05-08T23:44:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bebfff700 1 -- 172.21.15.204:0/3710437639 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bdc009730 con 0x7f6bf4100fe0 2024-05-08T23:44:35.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf9488700 1 -- 172.21.15.204:0/3710437639 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf419eff0 con 0x7f6bf4100fe0 2024-05-08T23:44:35.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bebfff700 1 -- 172.21.15.204:0/3710437639 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc003a30 con 0x7f6bf4100fe0 2024-05-08T23:44:35.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.874+0000 7f6bf9488700 1 -- 172.21.15.204:0/3710437639 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf419f410 con 0x7f6bf4100fe0 2024-05-08T23:44:35.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.875+0000 7f6bf9488700 1 -- 172.21.15.204:0/3710437639 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf4108410 con 0x7f6bf4100fe0 2024-05-08T23:44:35.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.876+0000 7f6bebfff700 1 -- 172.21.15.204:0/3710437639 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6bdc0098a0 con 0x7f6bf4100fe0 2024-05-08T23:44:35.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.876+0000 7f6bebfff700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6be00a11a0 0x7f6be00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:35.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.877+0000 7f6bebfff700 1 -- 172.21.15.204:0/3710437639 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6bdc0cc130 con 0x7f6bf4100fe0 2024-05-08T23:44:35.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.877+0000 7f6bf27fc700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6be00a11a0 0x7f6be00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:35.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.877+0000 7f6bf27fc700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6be00a11a0 0x7f6be00a3660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6be400b5c0 tx=0x7f6be4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:35.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:35.881+0000 7f6bebfff700 1 -- 172.21.15.204:0/3710437639 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6bdc095cc0 con 0x7f6bf4100fe0 2024-05-08T23:44:36.077 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.081+0000 7f6bf9488700 1 -- 172.21.15.204:0/3710437639 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bf402cfb0 con 0x7f6be00a11a0 2024-05-08T23:44:36.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.090+0000 7f6bebfff700 1 -- 172.21.15.204:0/3710437639 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f6bf402cfb0 con 0x7f6be00a11a0 2024-05-08T23:44:36.086 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-05-08T23:44:36.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 -- 172.21.15.204:0/3710437639 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6be00a11a0 msgr2=0x7f6be00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:36.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6be00a11a0 0x7f6be00a3660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6be400b5c0 tx=0x7f6be4005fb0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 -- 172.21.15.204:0/3710437639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf4100fe0 msgr2=0x7f6bf4196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:36.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf4100fe0 0x7f6bf4196f00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00d9b0 tx=0x7f6bdc00dcc0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 -- 172.21.15.204:0/3710437639 shutdown_connections 2024-05-08T23:44:36.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bf4102b80 0x7f6bf419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bf4100fe0 0x7f6bf4196f00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6be00a11a0 0x7f6be00a3660 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 --2- 172.21.15.204:0/3710437639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bf41021e0 0x7f6bf4197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 -- 172.21.15.204:0/3710437639 >> 172.21.15.204:0/3710437639 conn(0x7f6bf40fc5a0 msgr2=0x7f6bf4105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:36.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 -- 172.21.15.204:0/3710437639 shutdown_connections 2024-05-08T23:44:36.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-08T23:44:36.092+0000 7f6bf9488700 1 -- 172.21.15.204:0/3710437639 wait complete. 2024-05-08T23:44:36.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:36 smithi204 ceph-mon[47373]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:36.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:36 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:36.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:36 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:36.453 DEBUG:teuthology.orchestra.run.smithi012:mds.a> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T23:44:36.456 DEBUG:teuthology.orchestra.run.smithi012:mds.d> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T23:44:36.490 INFO:journalctl@ceph.mds.a.smithi012.stdout:-- Logs begin at Wed 2024-05-08 23:21:09 UTC. -- 2024-05-08T23:44:36.491 DEBUG:teuthology.orchestra.run.smithi012:mds.g> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T23:44:36.522 INFO:journalctl@ceph.mds.d.smithi012.stdout:-- Logs begin at Wed 2024-05-08 23:21:09 UTC. -- 2024-05-08T23:44:36.523 DEBUG:teuthology.orchestra.run.smithi012:mds.j> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T23:44:36.553 INFO:journalctl@ceph.mds.g.smithi012.stdout:-- Logs begin at Wed 2024-05-08 23:21:09 UTC. -- 2024-05-08T23:44:36.554 DEBUG:teuthology.orchestra.run.smithi160:mds.b> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T23:44:36.557 DEBUG:teuthology.orchestra.run.smithi160:mds.e> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T23:44:36.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:36 smithi012 ceph-mon[42503]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:36.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:36 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:36.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:36 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:36.585 INFO:journalctl@ceph.mds.j.smithi012.stdout:-- Logs begin at Wed 2024-05-08 23:21:09 UTC. -- 2024-05-08T23:44:36.588 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:36 smithi160 ceph-mon[46152]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:36.588 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:36 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:36.588 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:36 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:36.589 DEBUG:teuthology.orchestra.run.smithi160:mds.h> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T23:44:36.596 INFO:journalctl@ceph.mds.b.smithi160.stdout:-- Logs begin at Wed 2024-05-08 23:21:47 UTC. -- 2024-05-08T23:44:36.597 DEBUG:teuthology.orchestra.run.smithi160:mds.k> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T23:44:36.621 INFO:journalctl@ceph.mds.e.smithi160.stdout:-- Logs begin at Wed 2024-05-08 23:21:47 UTC. -- 2024-05-08T23:44:36.622 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T23:44:36.624 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T23:44:36.625 INFO:journalctl@ceph.mds.h.smithi160.stdout:-- Logs begin at Wed 2024-05-08 23:21:47 UTC. -- 2024-05-08T23:44:36.650 INFO:journalctl@ceph.mds.k.smithi160.stdout:-- Logs begin at Wed 2024-05-08 23:21:47 UTC. -- 2024-05-08T23:44:36.654 INFO:journalctl@ceph.mds.c.smithi204.stdout:-- Logs begin at Wed 2024-05-08 23:21:56 UTC. -- 2024-05-08T23:44:36.655 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T23:44:36.682 INFO:journalctl@ceph.mds.f.smithi204.stdout:-- Logs begin at Wed 2024-05-08 23:21:56 UTC. -- 2024-05-08T23:44:36.683 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T23:44:36.709 INFO:journalctl@ceph.mds.i.smithi204.stdout:-- Logs begin at Wed 2024-05-08 23:21:56 UTC. -- 2024-05-08T23:44:36.710 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T23:44:36.710 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:44:36.737 INFO:journalctl@ceph.mds.l.smithi204.stdout:-- Logs begin at Wed 2024-05-08 23:21:56 UTC. -- 2024-05-08T23:44:36.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.881+0000 7f2e6064d700 1 -- 172.21.15.12:0/3703828905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e581018f0 msgr2=0x7f2e581081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.881+0000 7f2e6064d700 1 --2- 172.21.15.12:0/3703828905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e581018f0 0x7f2e581081a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2e500099c0 tx=0x7f2e50009cd0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.881+0000 7f2e6064d700 1 -- 172.21.15.12:0/3703828905 shutdown_connections 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.881+0000 7f2e6064d700 1 --2- 172.21.15.12:0/3703828905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e581018f0 0x7f2e581081a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.881+0000 7f2e6064d700 1 --2- 172.21.15.12:0/3703828905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e580ffd60 0x7f2e58100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.881+0000 7f2e6064d700 1 --2- 172.21.15.12:0/3703828905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e58100f50 0x7f2e581013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.881+0000 7f2e6064d700 1 -- 172.21.15.12:0/3703828905 >> 172.21.15.12:0/3703828905 conn(0x7f2e580fb2f0 msgr2=0x7f2e580fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.881+0000 7f2e6064d700 1 -- 172.21.15.12:0/3703828905 shutdown_connections 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.881+0000 7f2e6064d700 1 -- 172.21.15.12:0/3703828905 wait complete. 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e6064d700 1 Processor -- start 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e6064d700 1 -- start start 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e6064d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e580ffd60 0x7f2e58103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:36.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e6064d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e58100f50 0x7f2e58102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e6064d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e581018f0 0x7f2e58102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e6064d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58102f70 con 0x7f2e58100f50 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e6064d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581030e0 con 0x7f2e581018f0 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e6064d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581a0e00 con 0x7f2e580ffd60 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e5dbe8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e58100f50 0x7f2e58102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e5dbe8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e58100f50 0x7f2e58102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39328/0 (socket says 172.21.15.12:39328) 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e5dbe8700 1 -- 172.21.15.12:0/367183003 learned_addr learned my addr 172.21.15.12:0/367183003 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e5e3e9700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e580ffd60 0x7f2e58103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.882+0000 7f2e5ebea700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e581018f0 0x7f2e58102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e5dbe8700 1 -- 172.21.15.12:0/367183003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e580ffd60 msgr2=0x7f2e58103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e5dbe8700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e580ffd60 0x7f2e58103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e5dbe8700 1 -- 172.21.15.12:0/367183003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e581018f0 msgr2=0x7f2e58102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e5dbe8700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e581018f0 0x7f2e58102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e5dbe8700 1 -- 172.21.15.12:0/367183003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e500095f0 con 0x7f2e58100f50 2024-05-08T23:44:36.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e5ebea700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e581018f0 0x7f2e58102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:44:36.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e5dbe8700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e58100f50 0x7f2e58102330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00c920 tx=0x7f2e4c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:36.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e4b7fe700 1 -- 172.21.15.12:0/367183003 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c007b00 con 0x7f2e58100f50 2024-05-08T23:44:36.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e4b7fe700 1 -- 172.21.15.12:0/367183003 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e4c00ce70 con 0x7f2e58100f50 2024-05-08T23:44:36.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e6064d700 1 -- 172.21.15.12:0/367183003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e581a1060 con 0x7f2e58100f50 2024-05-08T23:44:36.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e4b7fe700 1 -- 172.21.15.12:0/367183003 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0054e0 con 0x7f2e58100f50 2024-05-08T23:44:36.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.883+0000 7f2e6064d700 1 -- 172.21.15.12:0/367183003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e581a1570 con 0x7f2e58100f50 2024-05-08T23:44:36.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.884+0000 7f2e6064d700 1 -- 172.21.15.12:0/367183003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e58067c10 con 0x7f2e58100f50 2024-05-08T23:44:36.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.885+0000 7f2e4b7fe700 1 -- 172.21.15.12:0/367183003 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2e4c004750 con 0x7f2e58100f50 2024-05-08T23:44:36.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.886+0000 7f2e4b7fe700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2e440a1120 0x7f2e440a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:36.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.886+0000 7f2e5e3e9700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2e440a1120 0x7f2e440a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:36.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.886+0000 7f2e4b7fe700 1 -- 172.21.15.12:0/367183003 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2e4c0cc060 con 0x7f2e58100f50 2024-05-08T23:44:36.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.886+0000 7f2e5e3e9700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2e440a1120 0x7f2e440a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2e58100db0 tx=0x7f2e5400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:36.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:36.891+0000 7f2e4b7fe700 1 -- 172.21.15.12:0/367183003 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2e4c095bf0 con 0x7f2e58100f50 2024-05-08T23:44:37.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.110+0000 7f2e6064d700 1 -- 172.21.15.12:0/367183003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e5810b750 con 0x7f2e58100f50 2024-05-08T23:44:37.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.111+0000 7f2e4b7fe700 1 -- 172.21.15.12:0/367183003 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18408 (secure 0 0 0) 0x7f2e4c099210 con 0x7f2e58100f50 2024-05-08T23:44:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:37.108 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":77,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:44:31.131279+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:44:37.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 -- 172.21.15.12:0/367183003 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2e440a1120 msgr2=0x7f2e440a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2e440a1120 0x7f2e440a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2e58100db0 tx=0x7f2e5400b040 comp rx=0 tx=0).stop 2024-05-08T23:44:37.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 -- 172.21.15.12:0/367183003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e58100f50 msgr2=0x7f2e58102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e58100f50 0x7f2e58102330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00c920 tx=0x7f2e4c00cc30 comp rx=0 tx=0).stop 2024-05-08T23:44:37.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 -- 172.21.15.12:0/367183003 shutdown_connections 2024-05-08T23:44:37.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2e440a1120 0x7f2e440a35e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e581018f0 0x7f2e58102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e580ffd60 0x7f2e58103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 --2- 172.21.15.12:0/367183003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e58100f50 0x7f2e58102330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 -- 172.21.15.12:0/367183003 >> 172.21.15.12:0/367183003 conn(0x7f2e580fb2f0 msgr2=0x7f2e580fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:37.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 -- 172.21.15.12:0/367183003 shutdown_connections 2024-05-08T23:44:37.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.114+0000 7f2e6064d700 1 -- 172.21.15.12:0/367183003 wait complete. 2024-05-08T23:44:37.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:44:37.125 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:44:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.270+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/1128933656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9c8104330 msgr2=0x7fd9c8106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.270+0000 7fd9cd3b1700 1 --2- 172.21.15.12:0/1128933656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9c8104330 0x7fd9c8106720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009990 tx=0x7fd9bc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.270+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/1128933656 shutdown_connections 2024-05-08T23:44:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.270+0000 7fd9cd3b1700 1 --2- 172.21.15.12:0/1128933656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8106c60 0x7fd9c81090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.270+0000 7fd9cd3b1700 1 --2- 172.21.15.12:0/1128933656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9c8101a00 0x7fd9c8103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.270+0000 7fd9cd3b1700 1 --2- 172.21.15.12:0/1128933656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9c8104330 0x7fd9c8106720 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.270+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/1128933656 >> 172.21.15.12:0/1128933656 conn(0x7fd9c80fb3f0 msgr2=0x7fd9c80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.270+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/1128933656 shutdown_connections 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.270+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/1128933656 wait complete. 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.270+0000 7fd9cd3b1700 1 Processor -- start 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9cd3b1700 1 -- start start 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9cd3b1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9c8101a00 0x7fd9c8196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9cd3b1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8104330 0x7fd9c8197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9cd3b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9c8106c60 0x7fd9c819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9cd3b1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c819ea20 con 0x7fd9c8106c60 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9cd3b1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c819eb60 con 0x7fd9c8101a00 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9cd3b1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c819ecf0 con 0x7fd9c8104330 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9c8106c60 0x7fd9c819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9c8106c60 0x7fd9c819e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39336/0 (socket says 172.21.15.12:39336) 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c77fe700 1 -- 172.21.15.12:0/741021179 learned_addr learned my addr 172.21.15.12:0/741021179 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c6ffd700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9c8101a00 0x7fd9c8196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c67fc700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8104330 0x7fd9c8197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c77fe700 1 -- 172.21.15.12:0/741021179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8104330 msgr2=0x7fd9c8197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c77fe700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8104330 0x7fd9c8197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c77fe700 1 -- 172.21.15.12:0/741021179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9c8101a00 msgr2=0x7fd9c8196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c77fe700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9c8101a00 0x7fd9c8196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c77fe700 1 -- 172.21.15.12:0/741021179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc0095f0 con 0x7fd9c8106c60 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.271+0000 7fd9c67fc700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8104330 0x7fd9c8197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.272+0000 7fd9c77fe700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9c8106c60 0x7fd9c819e450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0007ee0 tx=0x7fd9c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.272+0000 7fd9affff700 1 -- 172.21.15.12:0/741021179 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0017070 con 0x7fd9c8106c60 2024-05-08T23:44:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.272+0000 7fd9affff700 1 -- 172.21.15.12:0/741021179 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9c000f040 con 0x7fd9c8106c60 2024-05-08T23:44:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.272+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/741021179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c819efd0 con 0x7fd9c8106c60 2024-05-08T23:44:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.272+0000 7fd9affff700 1 -- 172.21.15.12:0/741021179 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0005740 con 0x7fd9c8106c60 2024-05-08T23:44:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.272+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/741021179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c819f400 con 0x7fd9c8106c60 2024-05-08T23:44:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.273+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/741021179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c80fcf70 con 0x7fd9c8106c60 2024-05-08T23:44:37.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.274+0000 7fd9affff700 1 -- 172.21.15.12:0/741021179 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd9c000dc60 con 0x7fd9c8106c60 2024-05-08T23:44:37.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.275+0000 7fd9affff700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9b00a1120 0x7fd9b00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:37.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.275+0000 7fd9c6ffd700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9b00a1120 0x7fd9b00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:37.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.275+0000 7fd9affff700 1 -- 172.21.15.12:0/741021179 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd9c00cc240 con 0x7fd9c8106c60 2024-05-08T23:44:37.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.275+0000 7fd9c6ffd700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9b00a1120 0x7fd9b00a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8005950 tx=0x7fd9b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:37.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.279+0000 7fd9affff700 1 -- 172.21.15.12:0/741021179 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd9c0095e70 con 0x7fd9c8106c60 2024-05-08T23:44:37.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:37 smithi204 ceph-mon[47373]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:44:37.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:37 smithi204 ceph-mon[47373]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-08T23:44:37.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:37 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:37.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:37 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:37.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:37 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:37.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:37 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:37.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:37 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:37.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:37 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:37.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:37 smithi204 ceph-mon[47373]: Deploying daemon mds.l on smithi204 2024-05-08T23:44:37.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:37 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/367183003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:44:37.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:37 smithi012 ceph-mon[42503]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:44:37.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:37 smithi012 ceph-mon[42503]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-08T23:44:37.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:37 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:37.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:37 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:37.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:37 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:37.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:37 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:37.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:37 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:37.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:37 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:37.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:37 smithi012 ceph-mon[42503]: Deploying daemon mds.l on smithi204 2024-05-08T23:44:37.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:37 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/367183003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:44:37.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.498+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/741021179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9c810c770 con 0x7fd9c8106c60 2024-05-08T23:44:37.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.499+0000 7fd9affff700 1 -- 172.21.15.12:0/741021179 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9c0099490 con 0x7fd9c8106c60 2024-05-08T23:44:37.496 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-08T23:44:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.501+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/741021179 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9b00a1120 msgr2=0x7fd9b00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.501+0000 7fd9cd3b1700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9b00a1120 0x7fd9b00a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8005950 tx=0x7fd9b800b410 comp rx=0 tx=0).stop 2024-05-08T23:44:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.501+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/741021179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9c8106c60 msgr2=0x7fd9c819e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.501+0000 7fd9cd3b1700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9c8106c60 0x7fd9c819e450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0007ee0 tx=0x7fd9c000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.501+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/741021179 shutdown_connections 2024-05-08T23:44:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.501+0000 7fd9cd3b1700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9c8106c60 0x7fd9c819e450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.501+0000 7fd9cd3b1700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9b00a1120 0x7fd9b00a35e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.501+0000 7fd9cd3b1700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9c8101a00 0x7fd9c8196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.501+0000 7fd9cd3b1700 1 --2- 172.21.15.12:0/741021179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c8104330 0x7fd9c8197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.501+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/741021179 >> 172.21.15.12:0/741021179 conn(0x7fd9c80fb3f0 msgr2=0x7fd9c81038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:37.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.502+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/741021179 shutdown_connections 2024-05-08T23:44:37.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.502+0000 7fd9cd3b1700 1 -- 172.21.15.12:0/741021179 wait complete. 2024-05-08T23:44:37.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:44:37.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.652+0000 7feb63db2700 1 -- 172.21.15.12:0/3085516514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb5c100f50 msgr2=0x7feb5c1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.652+0000 7feb63db2700 1 --2- 172.21.15.12:0/3085516514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb5c100f50 0x7feb5c1013b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0099c0 tx=0x7feb4c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.652+0000 7feb63db2700 1 -- 172.21.15.12:0/3085516514 shutdown_connections 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.652+0000 7feb63db2700 1 --2- 172.21.15.12:0/3085516514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb5c1018f0 0x7feb5c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.652+0000 7feb63db2700 1 --2- 172.21.15.12:0/3085516514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb5c0ffd60 0x7feb5c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.652+0000 7feb63db2700 1 --2- 172.21.15.12:0/3085516514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb5c100f50 0x7feb5c1013b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.652+0000 7feb63db2700 1 -- 172.21.15.12:0/3085516514 >> 172.21.15.12:0/3085516514 conn(0x7feb5c0fb2f0 msgr2=0x7feb5c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.652+0000 7feb63db2700 1 -- 172.21.15.12:0/3085516514 shutdown_connections 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.652+0000 7feb63db2700 1 -- 172.21.15.12:0/3085516514 wait complete. 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb63db2700 1 Processor -- start 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb63db2700 1 -- start start 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb63db2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb5c0ffd60 0x7feb5c103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:37.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb63db2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb5c100f50 0x7feb5c102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb63db2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb5c1018f0 0x7feb5c102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb63db2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c103010 con 0x7feb5c1018f0 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb63db2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c103180 con 0x7feb5c0ffd60 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb63db2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1a0e90 con 0x7feb5c100f50 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb6234f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb5c1018f0 0x7feb5c102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb6234f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb5c1018f0 0x7feb5c102870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39364/0 (socket says 172.21.15.12:39364) 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb6234f700 1 -- 172.21.15.12:0/2406451370 learned_addr learned my addr 172.21.15.12:0/2406451370 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.653+0000 7feb6134d700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb5c100f50 0x7feb5c102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb61b4e700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb5c0ffd60 0x7feb5c103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb6234f700 1 -- 172.21.15.12:0/2406451370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb5c100f50 msgr2=0x7feb5c102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb6234f700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb5c100f50 0x7feb5c102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb6234f700 1 -- 172.21.15.12:0/2406451370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb5c0ffd60 msgr2=0x7feb5c103ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb6234f700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb5c0ffd60 0x7feb5c103ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb6234f700 1 -- 172.21.15.12:0/2406451370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c0095f0 con 0x7feb5c1018f0 2024-05-08T23:44:37.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb6234f700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb5c1018f0 0x7feb5c102870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feb54007ee0 tx=0x7feb5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:37.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb52ffd700 1 -- 172.21.15.12:0/2406451370 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54017070 con 0x7feb5c1018f0 2024-05-08T23:44:37.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb52ffd700 1 -- 172.21.15.12:0/2406451370 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb5400f040 con 0x7feb5c1018f0 2024-05-08T23:44:37.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb63db2700 1 -- 172.21.15.12:0/2406451370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c1a1120 con 0x7feb5c1018f0 2024-05-08T23:44:37.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb52ffd700 1 -- 172.21.15.12:0/2406451370 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb540054b0 con 0x7feb5c1018f0 2024-05-08T23:44:37.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.654+0000 7feb63db2700 1 -- 172.21.15.12:0/2406451370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c1a1630 con 0x7feb5c1018f0 2024-05-08T23:44:37.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.655+0000 7feb63db2700 1 -- 172.21.15.12:0/2406451370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5c067c10 con 0x7feb5c1018f0 2024-05-08T23:44:37.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.657+0000 7feb52ffd700 1 -- 172.21.15.12:0/2406451370 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7feb54004ad0 con 0x7feb5c1018f0 2024-05-08T23:44:37.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.657+0000 7feb52ffd700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feb480a11a0 0x7feb480a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:37.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.658+0000 7feb61b4e700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feb480a11a0 0x7feb480a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:37.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.658+0000 7feb52ffd700 1 -- 172.21.15.12:0/2406451370 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feb540cc1e0 con 0x7feb5c1018f0 2024-05-08T23:44:37.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.658+0000 7feb61b4e700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feb480a11a0 0x7feb480a3660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb5c100db0 tx=0x7feb58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:37.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.662+0000 7feb52ffd700 1 -- 172.21.15.12:0/2406451370 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feb54095e10 con 0x7feb5c1018f0 2024-05-08T23:44:37.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:37 smithi160 ceph-mon[46152]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:44:37.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:37 smithi160 ceph-mon[46152]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-08T23:44:37.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:37 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:37.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:37 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:44:37.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:37 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:37.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:37 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:37.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:37 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:37.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:37 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:37.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:37 smithi160 ceph-mon[46152]: Deploying daemon mds.l on smithi204 2024-05-08T23:44:37.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:37 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/367183003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:44:37.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.927+0000 7feb63db2700 1 -- 172.21.15.12:0/2406451370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb5c10b5c0 con 0x7feb5c1018f0 2024-05-08T23:44:37.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.928+0000 7feb52ffd700 1 -- 172.21.15.12:0/2406451370 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7feb54099430 con 0x7feb5c1018f0 2024-05-08T23:44:37.924 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:37.924 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-08T23:44:37.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 -- 172.21.15.12:0/2406451370 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feb480a11a0 msgr2=0x7feb480a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feb480a11a0 0x7feb480a3660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb5c100db0 tx=0x7feb58009260 comp rx=0 tx=0).stop 2024-05-08T23:44:37.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 -- 172.21.15.12:0/2406451370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb5c1018f0 msgr2=0x7feb5c102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:37.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb5c1018f0 0x7feb5c102870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feb54007ee0 tx=0x7feb5400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 -- 172.21.15.12:0/2406451370 shutdown_connections 2024-05-08T23:44:37.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb5c1018f0 0x7feb5c102870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7feb480a11a0 0x7feb480a3660 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb5c0ffd60 0x7feb5c103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 --2- 172.21.15.12:0/2406451370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb5c100f50 0x7feb5c102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:37.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 -- 172.21.15.12:0/2406451370 >> 172.21.15.12:0/2406451370 conn(0x7feb5c0fb2f0 msgr2=0x7feb5c0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:37.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 -- 172.21.15.12:0/2406451370 shutdown_connections 2024-05-08T23:44:37.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:37.930+0000 7feb63db2700 1 -- 172.21.15.12:0/2406451370 wait complete. 2024-05-08T23:44:37.927 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 1 2024-05-08T23:44:38.010 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 08 23:44:37 smithi204 systemd[1]: Starting Ceph mds.l for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:44:38.514 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mds[83951]: -- 172.21.15.204:0/3955532886 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ed536501a0 con 0x55ed53627000 2024-05-08T23:44:38.514 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 08 23:44:38 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: starting mds.l at 2024-05-08T23:44:38.514 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 08 23:44:38 smithi204 bash[83913]: a3b3ac03cee18ff1ca0e18b2bbf20e467905c73de9e2e1d18802892fd64f6289 2024-05-08T23:44:38.514 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 08 23:44:38 smithi204 systemd[1]: Started Ceph mds.l for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:38.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:38.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/741021179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:44:38.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2406451370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:38.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6832/4091100213,v1:172.21.15.204:6833/4091100213] up:boot 2024-05-08T23:44:38.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: fsmap 1 up:standby 2024-05-08T23:44:38.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:44:38.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:38.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:38.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:38.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:38.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:38.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:38 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/741021179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2406451370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6832/4091100213,v1:172.21.15.204:6833/4091100213] up:boot 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: fsmap 1 up:standby 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:38.641 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:38 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/741021179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:44:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2406451370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6832/4091100213,v1:172.21.15.204:6833/4091100213] up:boot 2024-05-08T23:44:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: fsmap 1 up:standby 2024-05-08T23:44:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:44:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:38 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:39.504 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:39 smithi160 ceph-mon[46152]: Deploying daemon mds.k on smithi160 2024-05-08T23:44:39.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:39 smithi012 ceph-mon[42503]: Deploying daemon mds.k on smithi160 2024-05-08T23:44:39.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:39 smithi204 ceph-mon[47373]: Deploying daemon mds.k on smithi160 2024-05-08T23:44:39.940 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:44:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.099+0000 7fa18c988700 1 -- 172.21.15.12:0/1906665915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa188106b40 msgr2=0x7fa188108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.099+0000 7fa18c988700 1 --2- 172.21.15.12:0/1906665915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa188106b40 0x7fa188108fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa18000b0a0 tx=0x7fa18000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.099+0000 7fa18c988700 1 -- 172.21.15.12:0/1906665915 shutdown_connections 2024-05-08T23:44:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.099+0000 7fa18c988700 1 --2- 172.21.15.12:0/1906665915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa188106b40 0x7fa188108fb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.099+0000 7fa18c988700 1 --2- 172.21.15.12:0/1906665915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1881018e0 0x7fa188103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.099+0000 7fa18c988700 1 --2- 172.21.15.12:0/1906665915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa188104210 0x7fa188106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.099+0000 7fa18c988700 1 -- 172.21.15.12:0/1906665915 >> 172.21.15.12:0/1906665915 conn(0x7fa1880fb2f0 msgr2=0x7fa1880fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.099+0000 7fa18c988700 1 -- 172.21.15.12:0/1906665915 shutdown_connections 2024-05-08T23:44:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.099+0000 7fa18c988700 1 -- 172.21.15.12:0/1906665915 wait complete. 2024-05-08T23:44:40.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.100+0000 7fa18c988700 1 Processor -- start 2024-05-08T23:44:40.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.100+0000 7fa18c988700 1 -- start start 2024-05-08T23:44:40.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.100+0000 7fa18c988700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1881018e0 0x7fa188194ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:40.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.100+0000 7fa18c988700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa188104210 0x7fa188195220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:40.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.100+0000 7fa18c988700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa188106b40 0x7fa18819c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.100+0000 7fa18c988700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18819c800 con 0x7fa188104210 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.100+0000 7fa18c988700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18819c970 con 0x7fa1881018e0 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.100+0000 7fa18c988700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18819cb30 con 0x7fa188106b40 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa18659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1881018e0 0x7fa188194ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa185d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa188104210 0x7fa188195220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa18659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1881018e0 0x7fa188194ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:40674/0 (socket says 172.21.15.12:40674) 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa18659c700 1 -- 172.21.15.12:0/854078813 learned_addr learned my addr 172.21.15.12:0/854078813 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa186d9d700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa188106b40 0x7fa18819c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa18659c700 1 -- 172.21.15.12:0/854078813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa188106b40 msgr2=0x7fa18819c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa18659c700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa188106b40 0x7fa18819c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa18659c700 1 -- 172.21.15.12:0/854078813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa188104210 msgr2=0x7fa188195220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa18659c700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa188104210 0x7fa188195220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa18659c700 1 -- 172.21.15.12:0/854078813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180009d00 con 0x7fa1881018e0 2024-05-08T23:44:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa185d9b700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa188104210 0x7fa188195220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:44:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa18659c700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1881018e0 0x7fa188194ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa17800e9a0 tx=0x7fa17800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa1777fe700 1 -- 172.21.15.12:0/854078813 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178009790 con 0x7fa1881018e0 2024-05-08T23:44:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa1777fe700 1 -- 172.21.15.12:0/854078813 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa178010430 con 0x7fa1881018e0 2024-05-08T23:44:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.101+0000 7fa1777fe700 1 -- 172.21.15.12:0/854078813 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178018680 con 0x7fa1881018e0 2024-05-08T23:44:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.102+0000 7fa18c988700 1 -- 172.21.15.12:0/854078813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18819ce10 con 0x7fa1881018e0 2024-05-08T23:44:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.102+0000 7fa18c988700 1 -- 172.21.15.12:0/854078813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18819d230 con 0x7fa1881018e0 2024-05-08T23:44:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.103+0000 7fa18c988700 1 -- 172.21.15.12:0/854078813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1880fce70 con 0x7fa1881018e0 2024-05-08T23:44:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.104+0000 7fa1777fe700 1 -- 172.21.15.12:0/854078813 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa1780105a0 con 0x7fa1881018e0 2024-05-08T23:44:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.104+0000 7fa1777fe700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1700a11a0 0x7fa1700a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.104+0000 7fa1777fe700 1 -- 172.21.15.12:0/854078813 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa178014070 con 0x7fa1881018e0 2024-05-08T23:44:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.104+0000 7fa185d9b700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1700a11a0 0x7fa1700a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.105+0000 7fa185d9b700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1700a11a0 0x7fa1700a3660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa188196190 tx=0x7fa17c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:40.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.109+0000 7fa1777fe700 1 -- 172.21.15.12:0/854078813 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa178052d60 con 0x7fa1881018e0 2024-05-08T23:44:40.266 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 23:44:39 smithi160 bash[81694]: d6bec3a8ee1f405901ab3a42b69ffc8bc89dd91f71be58ec3cd9584018b1054f 2024-05-08T23:44:40.266 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 08 23:44:39 smithi160 systemd[1]: Started Ceph mds.k for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:40.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:40 smithi012 ceph-mon[42503]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:40.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:40.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.389+0000 7fa18c988700 1 -- 172.21.15.12:0/854078813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa188067c10 con 0x7fa1881018e0 2024-05-08T23:44:40.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.389+0000 7fa1777fe700 1 -- 172.21.15.12:0/854078813 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7fa1780961b0 con 0x7fa1881018e0 2024-05-08T23:44:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:40.386 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-08T23:44:40.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 -- 172.21.15.12:0/854078813 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1700a11a0 msgr2=0x7fa1700a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:40.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1700a11a0 0x7fa1700a3660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa188196190 tx=0x7fa17c009260 comp rx=0 tx=0).stop 2024-05-08T23:44:40.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 -- 172.21.15.12:0/854078813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1881018e0 msgr2=0x7fa188194ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:40.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1881018e0 0x7fa188194ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa17800e9a0 tx=0x7fa17800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 -- 172.21.15.12:0/854078813 shutdown_connections 2024-05-08T23:44:40.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa188106b40 0x7fa18819c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1700a11a0 0x7fa1700a3660 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1881018e0 0x7fa188194ce0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 --2- 172.21.15.12:0/854078813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa188104210 0x7fa188195220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:40.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 -- 172.21.15.12:0/854078813 >> 172.21.15.12:0/854078813 conn(0x7fa1880fb2f0 msgr2=0x7fa188102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:40.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 -- 172.21.15.12:0/854078813 shutdown_connections 2024-05-08T23:44:40.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:40.392+0000 7fa18c988700 1 -- 172.21.15.12:0/854078813 wait complete. 2024-05-08T23:44:40.389 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2024-05-08T23:44:40.656 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:40.656 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:40.656 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:40.656 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:40.656 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:40 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:40.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:40 smithi204 ceph-mon[47373]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:40.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:40.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:40.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:40.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:40.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:40.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:40 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:40.819 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:40 smithi160 ceph-mon[46152]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:40.819 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:40.820 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:40.820 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:40.820 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:40.820 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:40.820 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:40 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:41.352 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:41 smithi012 ceph-mon[42503]: Deploying daemon mds.j on smithi012 2024-05-08T23:44:41.352 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:41 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6832/1870735591,v1:172.21.15.160:6833/1870735591] up:boot 2024-05-08T23:44:41.352 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:41 smithi012 ceph-mon[42503]: fsmap 2 up:standby 2024-05-08T23:44:41.352 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:41 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:44:41.352 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:41 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/854078813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:41.626 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 08 23:44:41 smithi012 systemd[1]: Starting Ceph mds.j for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:44:41.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:41 smithi204 ceph-mon[47373]: Deploying daemon mds.j on smithi012 2024-05-08T23:44:41.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:41 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6832/1870735591,v1:172.21.15.160:6833/1870735591] up:boot 2024-05-08T23:44:41.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:41 smithi204 ceph-mon[47373]: fsmap 2 up:standby 2024-05-08T23:44:41.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:41 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:44:41.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:41 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/854078813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:41.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:41 smithi160 ceph-mon[46152]: Deploying daemon mds.j on smithi012 2024-05-08T23:44:41.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:41 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6832/1870735591,v1:172.21.15.160:6833/1870735591] up:boot 2024-05-08T23:44:41.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:41 smithi160 ceph-mon[46152]: fsmap 2 up:standby 2024-05-08T23:44:41.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:41 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:44:41.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:41 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/854078813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:42.166 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 08 23:44:41 smithi012 ceph-mds[93130]: -- 172.21.15.12:0/597280151 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0618741a0 con 0x55d06184b000 2024-05-08T23:44:42.166 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 08 23:44:41 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-j[93107]: starting mds.j at 2024-05-08T23:44:42.166 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 08 23:44:41 smithi012 bash[93092]: 4770fff46bb97341abafabb8d4934a848ffa3760781efeeeb1941041f6e3d287 2024-05-08T23:44:42.166 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 08 23:44:41 smithi012 systemd[1]: Started Ceph mds.j for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:42.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:44:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.556+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/3506659375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a380ffd60 msgr2=0x7f2a38100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.556+0000 7f2a3d9b2700 1 --2- 172.21.15.12:0/3506659375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a380ffd60 0x7f2a38100180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009920 tx=0x7f2a28009c30 comp rx=0 tx=0).stop 2024-05-08T23:44:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.556+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/3506659375 shutdown_connections 2024-05-08T23:44:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.556+0000 7f2a3d9b2700 1 --2- 172.21.15.12:0/3506659375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a381018f0 0x7f2a381081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.556+0000 7f2a3d9b2700 1 --2- 172.21.15.12:0/3506659375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a380ffd60 0x7f2a38100180 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.556+0000 7f2a3d9b2700 1 --2- 172.21.15.12:0/3506659375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a38100f50 0x7f2a381013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.556+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/3506659375 >> 172.21.15.12:0/3506659375 conn(0x7f2a380fb2f0 msgr2=0x7f2a380fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.556+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/3506659375 shutdown_connections 2024-05-08T23:44:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.556+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/3506659375 wait complete. 2024-05-08T23:44:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.556+0000 7f2a3d9b2700 1 Processor -- start 2024-05-08T23:44:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a3d9b2700 1 -- start start 2024-05-08T23:44:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a3d9b2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a380ffd60 0x7f2a38196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a3d9b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a38100f50 0x7f2a381973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a3d9b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a381018f0 0x7f2a3819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a3d9b2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819e9c0 con 0x7f2a381018f0 2024-05-08T23:44:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a3d9b2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819eb00 con 0x7f2a38100f50 2024-05-08T23:44:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a3d9b2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819ec90 con 0x7f2a380ffd60 2024-05-08T23:44:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a377fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a381018f0 0x7f2a3819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:42.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a377fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a381018f0 0x7f2a3819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57536/0 (socket says 172.21.15.12:57536) 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a377fe700 1 -- 172.21.15.12:0/1734833894 learned_addr learned my addr 172.21.15.12:0/1734833894 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a367fc700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a38100f50 0x7f2a381973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.557+0000 7f2a36ffd700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a380ffd60 0x7f2a38196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a377fe700 1 -- 172.21.15.12:0/1734833894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a380ffd60 msgr2=0x7f2a38196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a377fe700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a380ffd60 0x7f2a38196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a377fe700 1 -- 172.21.15.12:0/1734833894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a38100f50 msgr2=0x7f2a381973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a377fe700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a38100f50 0x7f2a381973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a377fe700 1 -- 172.21.15.12:0/1734833894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a28009520 con 0x7f2a381018f0 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a377fe700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a381018f0 0x7f2a3819e3f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c110 tx=0x7f2a3000c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a3c9b0700 1 -- 172.21.15.12:0/1734833894 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30017070 con 0x7f2a381018f0 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a3c9b0700 1 -- 172.21.15.12:0/1734833894 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a3000e050 con 0x7f2a381018f0 2024-05-08T23:44:42.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/1734833894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3819ef10 con 0x7f2a381018f0 2024-05-08T23:44:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a3c9b0700 1 -- 172.21.15.12:0/1734833894 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30007940 con 0x7f2a381018f0 2024-05-08T23:44:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.558+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/1734833894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3819f340 con 0x7f2a381018f0 2024-05-08T23:44:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.559+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/1734833894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a38067c10 con 0x7f2a381018f0 2024-05-08T23:44:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.560+0000 7f2a3c9b0700 1 -- 172.21.15.12:0/1734833894 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2a30004ad0 con 0x7f2a381018f0 2024-05-08T23:44:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.561+0000 7f2a3c9b0700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2a200a1120 0x7f2a200a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.561+0000 7f2a3c9b0700 1 -- 172.21.15.12:0/1734833894 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a300cbf20 con 0x7f2a381018f0 2024-05-08T23:44:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.561+0000 7f2a36ffd700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2a200a1120 0x7f2a200a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:42.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.562+0000 7f2a36ffd700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2a200a1120 0x7f2a200a35e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800fa40 tx=0x7f2a28019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:42.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.567+0000 7f2a3c9b0700 1 -- 172.21.15.12:0/1734833894 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2a30095b50 con 0x7f2a381018f0 2024-05-08T23:44:42.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:42 smithi012 ceph-mon[42503]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:42.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:42.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:42.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:42.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:42.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:42.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:42 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:42.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:42 smithi204 ceph-mon[47373]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:42.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:42.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:42.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:42.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:42.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:42.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:42 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:42.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:42 smithi160 ceph-mon[46152]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:42.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:42.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:42.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:42.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:42.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:42.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:42 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:42.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.834+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/1734833894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a3810b720 con 0x7f2a381018f0 2024-05-08T23:44:42.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.835+0000 7f2a3c9b0700 1 -- 172.21.15.12:0/1734833894 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7f2a30099170 con 0x7f2a381018f0 2024-05-08T23:44:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:42.831 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-08T23:44:42.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/1734833894 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2a200a1120 msgr2=0x7f2a200a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:42.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2a200a1120 0x7f2a200a35e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800fa40 tx=0x7f2a28019040 comp rx=0 tx=0).stop 2024-05-08T23:44:42.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/1734833894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a381018f0 msgr2=0x7f2a3819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:42.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a381018f0 0x7f2a3819e3f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c110 tx=0x7f2a3000c420 comp rx=0 tx=0).stop 2024-05-08T23:44:42.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/1734833894 shutdown_connections 2024-05-08T23:44:42.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a381018f0 0x7f2a3819e3f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:42.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a380ffd60 0x7f2a38196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:42.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2a200a1120 0x7f2a200a35e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:42.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 --2- 172.21.15.12:0/1734833894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a38100f50 0x7f2a381973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:42.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/1734833894 >> 172.21.15.12:0/1734833894 conn(0x7f2a380fb2f0 msgr2=0x7f2a38067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:42.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/1734833894 shutdown_connections 2024-05-08T23:44:42.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:42.837+0000 7f2a3d9b2700 1 -- 172.21.15.12:0/1734833894 wait complete. 2024-05-08T23:44:42.834 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 4 2024-05-08T23:44:43.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:43 smithi012 ceph-mon[42503]: Deploying daemon mds.d on smithi012 2024-05-08T23:44:43.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:43 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.12:6834/2488990436,v1:172.21.15.12:6835/2488990436] up:boot 2024-05-08T23:44:43.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:43 smithi012 ceph-mon[42503]: fsmap 3 up:standby 2024-05-08T23:44:43.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:43 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:44:43.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:43 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1734833894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:43.667 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 08 23:44:43 smithi012 systemd[1]: Starting Ceph mds.d for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:44:43.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:43 smithi204 ceph-mon[47373]: Deploying daemon mds.d on smithi012 2024-05-08T23:44:43.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:43 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.12:6834/2488990436,v1:172.21.15.12:6835/2488990436] up:boot 2024-05-08T23:44:43.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:43 smithi204 ceph-mon[47373]: fsmap 3 up:standby 2024-05-08T23:44:43.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:43 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:44:43.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:43 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1734833894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:43 smithi160 ceph-mon[46152]: Deploying daemon mds.d on smithi012 2024-05-08T23:44:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:43 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.12:6834/2488990436,v1:172.21.15.12:6835/2488990436] up:boot 2024-05-08T23:44:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:43 smithi160 ceph-mon[46152]: fsmap 3 up:standby 2024-05-08T23:44:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:43 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:44:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:43 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1734833894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:44.373 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 08 23:44:43 smithi012 ceph-mds[93518]: -- 172.21.15.12:0/2024755208 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5569480c61a0 con 0x55694809cc00 2024-05-08T23:44:44.373 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 08 23:44:43 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-d[93495]: starting mds.d at 2024-05-08T23:44:44.373 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 08 23:44:43 smithi012 bash[93480]: db2833e70de370c809c5bc630020bbde3175e26cd20edfac29a8a2635598f30d 2024-05-08T23:44:44.373 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 08 23:44:43 smithi012 systemd[1]: Started Ceph mds.d for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:44.634 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:44 smithi160 ceph-mon[46152]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:44.635 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:44.635 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:44.635 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:44.635 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:44.635 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:44.635 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:44 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:44.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:44 smithi012 ceph-mon[42503]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:44.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:44.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:44.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:44.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:44.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:44.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:44 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:44.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:44 smithi204 ceph-mon[47373]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:44.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:44 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:44.848 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:44:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.005+0000 7f6be1734700 1 -- 172.21.15.12:0/646284305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bdc0ff180 msgr2=0x7f6bdc0ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.005+0000 7f6be1734700 1 --2- 172.21.15.12:0/646284305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bdc0ff180 0x7f6bdc0ff5a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009990 tx=0x7f6bcc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:44:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.005+0000 7f6be1734700 1 -- 172.21.15.12:0/646284305 shutdown_connections 2024-05-08T23:44:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.005+0000 7f6be1734700 1 --2- 172.21.15.12:0/646284305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bdc102b90 0x7f6bdc109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.005+0000 7f6be1734700 1 --2- 172.21.15.12:0/646284305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bdc0ff180 0x7f6bdc0ff5a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.005+0000 7f6be1734700 1 --2- 172.21.15.12:0/646284305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bdc1021f0 0x7f6bdc102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.005+0000 7f6be1734700 1 -- 172.21.15.12:0/646284305 >> 172.21.15.12:0/646284305 conn(0x7f6bdc0faab0 msgr2=0x7f6bdc0fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.005+0000 7f6be1734700 1 -- 172.21.15.12:0/646284305 shutdown_connections 2024-05-08T23:44:45.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.005+0000 7f6be1734700 1 -- 172.21.15.12:0/646284305 wait complete. 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.006+0000 7f6be1734700 1 Processor -- start 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6be1734700 1 -- start start 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6be1734700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bdc0ff180 0x7f6bdc101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6be1734700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bdc1021f0 0x7f6bdc1001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6be1734700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bdc102b90 0x7f6bdc1006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6be1734700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc100e80 con 0x7f6bdc1021f0 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6be1734700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc100ff0 con 0x7f6bdc0ff180 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6be1734700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1a0e30 con 0x7f6bdc102b90 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bda7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bdc1021f0 0x7f6bdc1001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bda7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bdc1021f0 0x7f6bdc1001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57558/0 (socket says 172.21.15.12:57558) 2024-05-08T23:44:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bda7fc700 1 -- 172.21.15.12:0/201557666 learned_addr learned my addr 172.21.15.12:0/201557666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bdaffd700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bdc0ff180 0x7f6bdc101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bdb7fe700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bdc102b90 0x7f6bdc1006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bda7fc700 1 -- 172.21.15.12:0/201557666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bdc102b90 msgr2=0x7f6bdc1006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bda7fc700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bdc102b90 0x7f6bdc1006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bda7fc700 1 -- 172.21.15.12:0/201557666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bdc0ff180 msgr2=0x7f6bdc101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bda7fc700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bdc0ff180 0x7f6bdc101b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bda7fc700 1 -- 172.21.15.12:0/201557666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc0095f0 con 0x7f6bdc1021f0 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bdaffd700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bdc0ff180 0x7f6bdc101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.007+0000 7f6bda7fc700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bdc1021f0 0x7f6bdc1001a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400e9a0 tx=0x7f6bc400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.008+0000 7f6bd3fff700 1 -- 172.21.15.12:0/201557666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4009790 con 0x7f6bdc1021f0 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.008+0000 7f6be1734700 1 -- 172.21.15.12:0/201557666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc1a10c0 con 0x7f6bdc1021f0 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.008+0000 7f6be1734700 1 -- 172.21.15.12:0/201557666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc1a1620 con 0x7f6bdc1021f0 2024-05-08T23:44:45.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.008+0000 7f6bd3fff700 1 -- 172.21.15.12:0/201557666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bc400cb70 con 0x7f6bdc1021f0 2024-05-08T23:44:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.008+0000 7f6bd3fff700 1 -- 172.21.15.12:0/201557666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4018680 con 0x7f6bdc1021f0 2024-05-08T23:44:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.009+0000 7f6be1734700 1 -- 172.21.15.12:0/201557666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc005320 con 0x7f6bdc1021f0 2024-05-08T23:44:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.010+0000 7f6bd3fff700 1 -- 172.21.15.12:0/201557666 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6bc400cce0 con 0x7f6bdc1021f0 2024-05-08T23:44:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.010+0000 7f6bd3fff700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bc80a1120 0x7f6bc80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.011+0000 7f6bdaffd700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bc80a1120 0x7f6bc80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:45.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.011+0000 7f6bd3fff700 1 -- 172.21.15.12:0/201557666 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6bc4014070 con 0x7f6bdc1021f0 2024-05-08T23:44:45.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.011+0000 7f6bdaffd700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bc80a1120 0x7f6bc80a35e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc006010 tx=0x7f6bcc00b770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:45.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.015+0000 7f6bd3fff700 1 -- 172.21.15.12:0/201557666 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6bc4052d60 con 0x7f6bdc1021f0 2024-05-08T23:44:45.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.286+0000 7f6be1734700 1 -- 172.21.15.12:0/201557666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bbc005190 con 0x7f6bdc1021f0 2024-05-08T23:44:45.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.286+0000 7f6bd3fff700 1 -- 172.21.15.12:0/201557666 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3534 (secure 0 0 0) 0x7f6bc4099b80 con 0x7f6bdc1021f0 2024-05-08T23:44:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:45.283 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 -- 172.21.15.12:0/201557666 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bc80a1120 msgr2=0x7f6bc80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bc80a1120 0x7f6bc80a35e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc006010 tx=0x7f6bcc00b770 comp rx=0 tx=0).stop 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 -- 172.21.15.12:0/201557666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bdc1021f0 msgr2=0x7f6bdc1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bdc1021f0 0x7f6bdc1001a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400e9a0 tx=0x7f6bc400ed60 comp rx=0 tx=0).stop 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 -- 172.21.15.12:0/201557666 shutdown_connections 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bdc102b90 0x7f6bdc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bc80a1120 0x7f6bc80a35e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bdc0ff180 0x7f6bdc101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 --2- 172.21.15.12:0/201557666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bdc1021f0 0x7f6bdc1001a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 -- 172.21.15.12:0/201557666 >> 172.21.15.12:0/201557666 conn(0x7f6bdc0faab0 msgr2=0x7f6bdc105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 -- 172.21.15.12:0/201557666 shutdown_connections 2024-05-08T23:44:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:45.289+0000 7f6be1734700 1 -- 172.21.15.12:0/201557666 wait complete. 2024-05-08T23:44:45.286 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 5 2024-05-08T23:44:45.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:45 smithi160 ceph-mon[46152]: Deploying daemon mds.e on smithi160 2024-05-08T23:44:45.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:45 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.12:6836/4284773137,v1:172.21.15.12:6837/4284773137] up:boot 2024-05-08T23:44:45.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:45 smithi160 ceph-mon[46152]: fsmap 4 up:standby 2024-05-08T23:44:45.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:45 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:44:45.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:45 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/201557666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:45.517 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 23:44:45 smithi160 systemd[1]: Starting Ceph mds.e for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:44:45.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:45 smithi012 ceph-mon[42503]: Deploying daemon mds.e on smithi160 2024-05-08T23:44:45.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:45 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.12:6836/4284773137,v1:172.21.15.12:6837/4284773137] up:boot 2024-05-08T23:44:45.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:45 smithi012 ceph-mon[42503]: fsmap 4 up:standby 2024-05-08T23:44:45.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:45 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:44:45.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:45 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/201557666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:45.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:45 smithi204 ceph-mon[47373]: Deploying daemon mds.e on smithi160 2024-05-08T23:44:45.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:45 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.12:6836/4284773137,v1:172.21.15.12:6837/4284773137] up:boot 2024-05-08T23:44:45.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:45 smithi204 ceph-mon[47373]: fsmap 4 up:standby 2024-05-08T23:44:45.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:45 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:44:45.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:45 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/201557666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:46.016 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 23:44:45 smithi160 bash[82036]: 7580a00f604c7e4fbe6e2eba5dcbd4d0e0eb8a00d4f9ec10288ce88e024a90c5 2024-05-08T23:44:46.016 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 23:44:45 smithi160 systemd[1]: Started Ceph mds.e for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:46.016 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 23:44:45 smithi160 ceph-mds[82074]: -- 172.21.15.160:0/3762735009 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562f09fa81a0 con 0x562f09f7ec00 2024-05-08T23:44:46.016 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 08 23:44:45 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-e[82051]: starting mds.e at 2024-05-08T23:44:46.474 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:46 smithi204 ceph-mon[47373]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:46.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:46 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:46.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:46 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:46.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:46 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:46.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:46 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:46.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:46 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:46.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:46 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:46.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:46 smithi012 ceph-mon[42503]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:46.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:46 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:46.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:46 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:46.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:46 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:46.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:46 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:46.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:46 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:46.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:46 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:46.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:46 smithi160 ceph-mon[46152]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:46.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:46 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:46.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:46 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:46.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:46 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:46.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:46 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:46.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:46 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:46.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:46 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:47.057 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 08 23:44:47 smithi204 systemd[1]: Starting Ceph mds.f for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:44:47.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:44:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.452+0000 7fb220153700 1 -- 172.21.15.12:0/3516875938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb218074fe0 msgr2=0x7fb218073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.452+0000 7fb220153700 1 --2- 172.21.15.12:0/3516875938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb218074fe0 0x7fb218073430 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb214009990 tx=0x7fb214009ca0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.452+0000 7fb220153700 1 -- 172.21.15.12:0/3516875938 shutdown_connections 2024-05-08T23:44:47.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.452+0000 7fb220153700 1 --2- 172.21.15.12:0/3516875938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb21810d450 0x7fb21810f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.452+0000 7fb220153700 1 --2- 172.21.15.12:0/3516875938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb218074fe0 0x7fb218073430 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.452+0000 7fb220153700 1 --2- 172.21.15.12:0/3516875938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb218073a00 0x7fb218073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.452+0000 7fb220153700 1 -- 172.21.15.12:0/3516875938 >> 172.21.15.12:0/3516875938 conn(0x7fb2180fc410 msgr2=0x7fb2180fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:47.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.452+0000 7fb220153700 1 -- 172.21.15.12:0/3516875938 shutdown_connections 2024-05-08T23:44:47.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.452+0000 7fb220153700 1 -- 172.21.15.12:0/3516875938 wait complete. 2024-05-08T23:44:47.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.453+0000 7fb220153700 1 Processor -- start 2024-05-08T23:44:47.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.453+0000 7fb220153700 1 -- start start 2024-05-08T23:44:47.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.453+0000 7fb220153700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb218073a00 0x7fb21819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.453+0000 7fb220153700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb218074fe0 0x7fb21819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.453+0000 7fb220153700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb21810d450 0x7fb2181a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.453+0000 7fb220153700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181a4f40 con 0x7fb218074fe0 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.453+0000 7fb220153700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181a5080 con 0x7fb21810d450 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.453+0000 7fb220153700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181a5210 con 0x7fb218073a00 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.453+0000 7fb21d6ee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb218074fe0 0x7fb21819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.453+0000 7fb21d6ee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb218074fe0 0x7fb21819d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57578/0 (socket says 172.21.15.12:57578) 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb21d6ee700 1 -- 172.21.15.12:0/397683564 learned_addr learned my addr 172.21.15.12:0/397683564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb21deef700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb218073a00 0x7fb21819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb21e6f0700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb21810d450 0x7fb2181a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb21d6ee700 1 -- 172.21.15.12:0/397683564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb218073a00 msgr2=0x7fb21819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb21d6ee700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb218073a00 0x7fb21819d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb21d6ee700 1 -- 172.21.15.12:0/397683564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb21810d450 msgr2=0x7fb2181a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb21d6ee700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb21810d450 0x7fb2181a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb21d6ee700 1 -- 172.21.15.12:0/397683564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2140095f0 con 0x7fb218074fe0 2024-05-08T23:44:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb21e6f0700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb21810d450 0x7fb2181a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:44:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb21d6ee700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb218074fe0 0x7fb21819d950 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb20800e970 tx=0x7fb20800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb20effd700 1 -- 172.21.15.12:0/397683564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208009790 con 0x7fb218074fe0 2024-05-08T23:44:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb20effd700 1 -- 172.21.15.12:0/397683564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb20800eec0 con 0x7fb218074fe0 2024-05-08T23:44:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb220153700 1 -- 172.21.15.12:0/397683564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2181a54f0 con 0x7fb218074fe0 2024-05-08T23:44:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb20effd700 1 -- 172.21.15.12:0/397683564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208003cf0 con 0x7fb218074fe0 2024-05-08T23:44:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.454+0000 7fb220153700 1 -- 172.21.15.12:0/397683564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2181a5a50 con 0x7fb218074fe0 2024-05-08T23:44:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.455+0000 7fb220153700 1 -- 172.21.15.12:0/397683564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2181036b0 con 0x7fb218074fe0 2024-05-08T23:44:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.457+0000 7fb20effd700 1 -- 172.21.15.12:0/397683564 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb20800ccd0 con 0x7fb218074fe0 2024-05-08T23:44:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.457+0000 7fb20effd700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb2040a11a0 0x7fb2040a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.457+0000 7fb21deef700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb2040a11a0 0x7fb2040a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.457+0000 7fb20effd700 1 -- 172.21.15.12:0/397683564 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb208014070 con 0x7fb218074fe0 2024-05-08T23:44:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.458+0000 7fb21deef700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb2040a11a0 0x7fb2040a3660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb21400b2e0 tx=0x7fb214005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:47.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.461+0000 7fb20effd700 1 -- 172.21.15.12:0/397683564 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb2080963b0 con 0x7fb218074fe0 2024-05-08T23:44:47.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:47 smithi012 ceph-mon[42503]: Deploying daemon mds.f on smithi204 2024-05-08T23:44:47.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:47 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6834/3313911488,v1:172.21.15.160:6835/3313911488] up:boot 2024-05-08T23:44:47.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:47 smithi012 ceph-mon[42503]: fsmap 5 up:standby 2024-05-08T23:44:47.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:47 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:44:47.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.730+0000 7fb220153700 1 -- 172.21.15.12:0/397683564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb218067c10 con 0x7fb218074fe0 2024-05-08T23:44:47.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.731+0000 7fb20effd700 1 -- 172.21.15.12:0/397683564 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4294 (secure 0 0 0) 0x7fb2080999d0 con 0x7fb218074fe0 2024-05-08T23:44:47.727 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:47.727 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-05-08T23:44:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 -- 172.21.15.12:0/397683564 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb2040a11a0 msgr2=0x7fb2040a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb2040a11a0 0x7fb2040a3660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb21400b2e0 tx=0x7fb214005fb0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 -- 172.21.15.12:0/397683564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb218074fe0 msgr2=0x7fb21819d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb218074fe0 0x7fb21819d950 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb20800e970 tx=0x7fb20800ec80 comp rx=0 tx=0).stop 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 -- 172.21.15.12:0/397683564 shutdown_connections 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb2040a11a0 0x7fb2040a3660 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb21810d450 0x7fb2181a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb218073a00 0x7fb21819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 --2- 172.21.15.12:0/397683564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb218074fe0 0x7fb21819d950 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 -- 172.21.15.12:0/397683564 >> 172.21.15.12:0/397683564 conn(0x7fb2180fc410 msgr2=0x7fb2181007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 -- 172.21.15.12:0/397683564 shutdown_connections 2024-05-08T23:44:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:47.733+0000 7fb220153700 1 -- 172.21.15.12:0/397683564 wait complete. 2024-05-08T23:44:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 6 2024-05-08T23:44:47.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:47 smithi204 ceph-mon[47373]: Deploying daemon mds.f on smithi204 2024-05-08T23:44:47.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:47 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6834/3313911488,v1:172.21.15.160:6835/3313911488] up:boot 2024-05-08T23:44:47.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:47 smithi204 ceph-mon[47373]: fsmap 5 up:standby 2024-05-08T23:44:47.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:47 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:44:47.764 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 08 23:44:47 smithi204 ceph-mds[84296]: -- 172.21.15.204:0/146711976 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555596d121a0 con 0x555596ce8c00 2024-05-08T23:44:47.764 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 08 23:44:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-f[84273]: starting mds.f at 2024-05-08T23:44:47.764 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 08 23:44:47 smithi204 bash[84250]: 21dc30b5d5a05044da565597a4ba3164c6812d8dec5c838788bfc951ec24760e 2024-05-08T23:44:47.764 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 08 23:44:47 smithi204 systemd[1]: Started Ceph mds.f for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:47.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:47 smithi160 ceph-mon[46152]: Deploying daemon mds.f on smithi204 2024-05-08T23:44:47.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:47 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6834/3313911488,v1:172.21.15.160:6835/3313911488] up:boot 2024-05-08T23:44:47.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:47 smithi160 ceph-mon[46152]: fsmap 5 up:standby 2024-05-08T23:44:47.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:47 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:44:48.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: Deploying daemon mds.c on smithi204 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/397683564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6834/1481890752,v1:172.21.15.204:6835/1481890752] up:boot 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: fsmap 6 up:standby 2024-05-08T23:44:48.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:48 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:44:48.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:48.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:48.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:48.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:48.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:48.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:48.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:48.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: Deploying daemon mds.c on smithi204 2024-05-08T23:44:48.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/397683564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:48.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6834/1481890752,v1:172.21.15.204:6835/1481890752] up:boot 2024-05-08T23:44:48.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: fsmap 6 up:standby 2024-05-08T23:44:48.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:48 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: Deploying daemon mds.c on smithi204 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/397683564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6834/1481890752,v1:172.21.15.204:6835/1481890752] up:boot 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: fsmap 6 up:standby 2024-05-08T23:44:48.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:48 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:44:49.002 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:44:48 smithi204 systemd[1]: Starting Ceph mds.c for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:44:49.513 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:44:49 smithi204 ceph-mds[84635]: -- 172.21.15.204:0/3080808076 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556a845581a0 con 0x556a8452e800 2024-05-08T23:44:49.513 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:44:49 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: starting mds.c at 2024-05-08T23:44:49.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:44:49 smithi204 bash[84597]: 5cd71f56cfaef572f630ec5b2d26053e7c758aec279206d224eb7232016d77af 2024-05-08T23:44:49.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:44:49 smithi204 systemd[1]: Started Ceph mds.c for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:49.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:44:49.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.894+0000 7ff25f8dd700 1 -- 172.21.15.12:0/2916887607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff258100f50 msgr2=0x7ff2581013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.894+0000 7ff25f8dd700 1 --2- 172.21.15.12:0/2916887607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff258100f50 0x7ff2581013b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff2480099c0 tx=0x7ff248009cd0 comp rx=0 tx=0).stop 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.894+0000 7ff25f8dd700 1 -- 172.21.15.12:0/2916887607 shutdown_connections 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.894+0000 7ff25f8dd700 1 --2- 172.21.15.12:0/2916887607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2581018f0 0x7ff2581081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.894+0000 7ff25f8dd700 1 --2- 172.21.15.12:0/2916887607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2580ffd60 0x7ff258100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.894+0000 7ff25f8dd700 1 --2- 172.21.15.12:0/2916887607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff258100f50 0x7ff2581013b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.894+0000 7ff25f8dd700 1 -- 172.21.15.12:0/2916887607 >> 172.21.15.12:0/2916887607 conn(0x7ff2580fb2f0 msgr2=0x7ff2580fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.894+0000 7ff25f8dd700 1 -- 172.21.15.12:0/2916887607 shutdown_connections 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.894+0000 7ff25f8dd700 1 -- 172.21.15.12:0/2916887607 wait complete. 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25f8dd700 1 Processor -- start 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25f8dd700 1 -- start start 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25f8dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2580ffd60 0x7ff258198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:49.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25f8dd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff258100f50 0x7ff258199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25f8dd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2581018f0 0x7ff2581a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25f8dd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2581a0b00 con 0x7ff2580ffd60 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25f8dd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2581a0c40 con 0x7ff258100f50 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25f8dd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2581a0dd0 con 0x7ff2581018f0 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25d679700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2580ffd60 0x7ff258198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25d679700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2580ffd60 0x7ff258198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37098/0 (socket says 172.21.15.12:37098) 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.895+0000 7ff25d679700 1 -- 172.21.15.12:0/1631775382 learned_addr learned my addr 172.21.15.12:0/1631775382 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff25ce78700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff258100f50 0x7ff258199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff25de7a700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2581018f0 0x7ff2581a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff25d679700 1 -- 172.21.15.12:0/1631775382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2581018f0 msgr2=0x7ff2581a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff25d679700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2581018f0 0x7ff2581a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff25d679700 1 -- 172.21.15.12:0/1631775382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff258100f50 msgr2=0x7ff258199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:49.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff25d679700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff258100f50 0x7ff258199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:49.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff25d679700 1 -- 172.21.15.12:0/1631775382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2480095f0 con 0x7ff2580ffd60 2024-05-08T23:44:49.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff25d679700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2580ffd60 0x7ff258198fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff25400e970 tx=0x7ff25400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:49.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff24e7fc700 1 -- 172.21.15.12:0/1631775382 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254009790 con 0x7ff2580ffd60 2024-05-08T23:44:49.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff24e7fc700 1 -- 172.21.15.12:0/1631775382 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff25400eec0 con 0x7ff2580ffd60 2024-05-08T23:44:49.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff25f8dd700 1 -- 172.21.15.12:0/1631775382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2581a10b0 con 0x7ff2580ffd60 2024-05-08T23:44:49.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff24e7fc700 1 -- 172.21.15.12:0/1631775382 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254003cf0 con 0x7ff2580ffd60 2024-05-08T23:44:49.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.896+0000 7ff25f8dd700 1 -- 172.21.15.12:0/1631775382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2581a1610 con 0x7ff2580ffd60 2024-05-08T23:44:49.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.897+0000 7ff25f8dd700 1 -- 172.21.15.12:0/1631775382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff258067c10 con 0x7ff2580ffd60 2024-05-08T23:44:49.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.898+0000 7ff24e7fc700 1 -- 172.21.15.12:0/1631775382 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff25400ccd0 con 0x7ff2580ffd60 2024-05-08T23:44:49.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.899+0000 7ff24e7fc700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2440a11a0 0x7ff2440a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:49.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.899+0000 7ff25ce78700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2440a11a0 0x7ff2440a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:49.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.899+0000 7ff24e7fc700 1 -- 172.21.15.12:0/1631775382 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff254014070 con 0x7ff2580ffd60 2024-05-08T23:44:49.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.900+0000 7ff25ce78700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2440a11a0 0x7ff2440a3660 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff248003730 tx=0x7ff24800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:49.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:49.904+0000 7ff24e7fc700 1 -- 172.21.15.12:0/1631775382 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff254096260 con 0x7ff2580ffd60 2024-05-08T23:44:50.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.173+0000 7ff25f8dd700 1 -- 172.21.15.12:0/1631775382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff25810b5c0 con 0x7ff2580ffd60 2024-05-08T23:44:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.174+0000 7ff24e7fc700 1 -- 172.21.15.12:0/1631775382 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5814 (secure 0 0 0) 0x7ff254099880 con 0x7ff2580ffd60 2024-05-08T23:44:50.170 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:50.170 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-08T23:44:50.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 -- 172.21.15.12:0/1631775382 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2440a11a0 msgr2=0x7ff2440a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:50.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2440a11a0 0x7ff2440a3660 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff248003730 tx=0x7ff24800b540 comp rx=0 tx=0).stop 2024-05-08T23:44:50.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 -- 172.21.15.12:0/1631775382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2580ffd60 msgr2=0x7ff258198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:50.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2580ffd60 0x7ff258198fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff25400e970 tx=0x7ff25400ec80 comp rx=0 tx=0).stop 2024-05-08T23:44:50.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 -- 172.21.15.12:0/1631775382 shutdown_connections 2024-05-08T23:44:50.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2581018f0 0x7ff2581a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:50.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2580ffd60 0x7ff258198fd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:50.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2440a11a0 0x7ff2440a3660 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:50.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 --2- 172.21.15.12:0/1631775382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff258100f50 0x7ff258199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:50.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 -- 172.21.15.12:0/1631775382 >> 172.21.15.12:0/1631775382 conn(0x7ff2580fb2f0 msgr2=0x7ff2580fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:50.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 -- 172.21.15.12:0/1631775382 shutdown_connections 2024-05-08T23:44:50.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:50.177+0000 7ff25f8dd700 1 -- 172.21.15.12:0/1631775382 wait complete. 2024-05-08T23:44:50.174 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-05-08T23:44:50.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:50.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:50.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:50.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:50.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:50.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:50.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: Deploying daemon mds.h on smithi160 2024-05-08T23:44:50.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:50.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/2858273239,v1:172.21.15.204:6837/2858273239] up:boot 2024-05-08T23:44:50.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: fsmap 7 up:standby 2024-05-08T23:44:50.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:44:50.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1631775382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:50.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: Deploying daemon mds.h on smithi160 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/2858273239,v1:172.21.15.204:6837/2858273239] up:boot 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: fsmap 7 up:standby 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:44:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:50 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1631775382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:50.525 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 23:44:50 smithi160 systemd[1]: Starting Ceph mds.h for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:44:50.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: Deploying daemon mds.h on smithi160 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/2858273239,v1:172.21.15.204:6837/2858273239] up:boot 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: fsmap 7 up:standby 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:44:50.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:50 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1631775382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:51.266 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 23:44:50 smithi160 bash[82376]: 469e43844c720649412c85a463e63782ca0430b6237858800cb13888f1079496 2024-05-08T23:44:51.266 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 23:44:50 smithi160 systemd[1]: Started Ceph mds.h for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:51.266 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 23:44:50 smithi160 ceph-mds[82414]: -- 172.21.15.160:0/3234452284 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55afe6f561a0 con 0x55afe6f2cc00 2024-05-08T23:44:51.266 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 08 23:44:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-h[82391]: starting mds.h at 2024-05-08T23:44:52.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:52.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:52.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:52.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:52.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:52.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:52.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: Deploying daemon mds.i on smithi204 2024-05-08T23:44:52.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] up:boot 2024-05-08T23:44:52.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: fsmap 8 up:standby 2024-05-08T23:44:52.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:44:52.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:52 smithi204 ceph-mon[47373]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:52.188 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:44:52.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.340+0000 7fc77beb9700 1 -- 172.21.15.12:0/2434742417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7741018a0 msgr2=0x7fc774108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:52.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.340+0000 7fc77beb9700 1 --2- 172.21.15.12:0/2434742417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7741018a0 0x7fc774108150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00b0d0 tx=0x7fc76c00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.340+0000 7fc77beb9700 1 -- 172.21.15.12:0/2434742417 shutdown_connections 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.340+0000 7fc77beb9700 1 --2- 172.21.15.12:0/2434742417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7741018a0 0x7fc774108150 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.340+0000 7fc77beb9700 1 --2- 172.21.15.12:0/2434742417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7740ffd10 0x7fc774100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.340+0000 7fc77beb9700 1 --2- 172.21.15.12:0/2434742417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc774100f00 0x7fc774101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.340+0000 7fc77beb9700 1 -- 172.21.15.12:0/2434742417 >> 172.21.15.12:0/2434742417 conn(0x7fc7740fb260 msgr2=0x7fc7740fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.340+0000 7fc77beb9700 1 -- 172.21.15.12:0/2434742417 shutdown_connections 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.340+0000 7fc77beb9700 1 -- 172.21.15.12:0/2434742417 wait complete. 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc77beb9700 1 Processor -- start 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc77beb9700 1 -- start start 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc77beb9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7740ffd10 0x7fc774068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc77beb9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc774100f00 0x7fc77419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc77beb9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7741018a0 0x7fc77419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc77beb9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77419e7f0 con 0x7fc7740ffd10 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc77beb9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77419e960 con 0x7fc7741018a0 2024-05-08T23:44:52.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc77beb9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77419eb20 con 0x7fc774100f00 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc779c55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7740ffd10 0x7fc774068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc779c55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7740ffd10 0x7fc774068b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37124/0 (socket says 172.21.15.12:37124) 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc779c55700 1 -- 172.21.15.12:0/2416174896 learned_addr learned my addr 172.21.15.12:0/2416174896 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc779454700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc774100f00 0x7fc77419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.341+0000 7fc77a456700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7741018a0 0x7fc77419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc779c55700 1 -- 172.21.15.12:0/2416174896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc774100f00 msgr2=0x7fc77419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc779c55700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc774100f00 0x7fc77419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc779c55700 1 -- 172.21.15.12:0/2416174896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7741018a0 msgr2=0x7fc77419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc779c55700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7741018a0 0x7fc77419e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc779c55700 1 -- 172.21.15.12:0/2416174896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c009d00 con 0x7fc7740ffd10 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc779c55700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7740ffd10 0x7fc774068b10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc77000c990 tx=0x7fc77000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc76affd700 1 -- 172.21.15.12:0/2416174896 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770007b00 con 0x7fc7740ffd10 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc76affd700 1 -- 172.21.15.12:0/2416174896 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc770007c60 con 0x7fc7740ffd10 2024-05-08T23:44:52.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc76affd700 1 -- 172.21.15.12:0/2416174896 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770018660 con 0x7fc7740ffd10 2024-05-08T23:44:52.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc77beb9700 1 -- 172.21.15.12:0/2416174896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77419ee00 con 0x7fc7740ffd10 2024-05-08T23:44:52.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.342+0000 7fc77beb9700 1 -- 172.21.15.12:0/2416174896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77419f230 con 0x7fc7740ffd10 2024-05-08T23:44:52.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.343+0000 7fc77beb9700 1 -- 172.21.15.12:0/2416174896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc774067c10 con 0x7fc7740ffd10 2024-05-08T23:44:52.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.345+0000 7fc76affd700 1 -- 172.21.15.12:0/2416174896 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc77000f450 con 0x7fc7740ffd10 2024-05-08T23:44:52.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.345+0000 7fc76affd700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc7600a11a0 0x7fc7600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:52.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.346+0000 7fc779454700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc7600a11a0 0x7fc7600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:52.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.346+0000 7fc76affd700 1 -- 172.21.15.12:0/2416174896 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc7700cc1d0 con 0x7fc7740ffd10 2024-05-08T23:44:52.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.346+0000 7fc779454700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc7600a11a0 0x7fc7600a3660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc7740690d0 tx=0x7fc76400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:52.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.350+0000 7fc76affd700 1 -- 172.21.15.12:0/2416174896 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc770095f10 con 0x7fc7740ffd10 2024-05-08T23:44:52.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:52.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:52.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:52.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:52.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:52.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:52.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: Deploying daemon mds.i on smithi204 2024-05-08T23:44:52.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] up:boot 2024-05-08T23:44:52.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: fsmap 8 up:standby 2024-05-08T23:44:52.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:44:52.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:52 smithi012 ceph-mon[42503]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:52.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:52.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:52.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:52.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:52.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:52.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:52.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: Deploying daemon mds.i on smithi204 2024-05-08T23:44:52.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] up:boot 2024-05-08T23:44:52.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: fsmap 8 up:standby 2024-05-08T23:44:52.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:44:52.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:52 smithi160 ceph-mon[46152]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:52.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.621+0000 7fc77beb9700 1 -- 172.21.15.12:0/2416174896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc774107060 con 0x7fc7740ffd10 2024-05-08T23:44:52.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.622+0000 7fc76affd700 1 -- 172.21.15.12:0/2416174896 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6574 (secure 0 0 0) 0x7fc770099530 con 0x7fc7740ffd10 2024-05-08T23:44:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:52.618 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-05-08T23:44:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 -- 172.21.15.12:0/2416174896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc7600a11a0 msgr2=0x7fc7600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc7600a11a0 0x7fc7600a3660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc7740690d0 tx=0x7fc76400a380 comp rx=0 tx=0).stop 2024-05-08T23:44:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 -- 172.21.15.12:0/2416174896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7740ffd10 msgr2=0x7fc774068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7740ffd10 0x7fc774068b10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc77000c990 tx=0x7fc77000cca0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 -- 172.21.15.12:0/2416174896 shutdown_connections 2024-05-08T23:44:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc7600a11a0 0x7fc7600a3660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7741018a0 0x7fc77419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7740ffd10 0x7fc774068b10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 --2- 172.21.15.12:0/2416174896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc774100f00 0x7fc77419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 -- 172.21.15.12:0/2416174896 >> 172.21.15.12:0/2416174896 conn(0x7fc7740fb260 msgr2=0x7fc7741041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 -- 172.21.15.12:0/2416174896 shutdown_connections 2024-05-08T23:44:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:52.624+0000 7fc77beb9700 1 -- 172.21.15.12:0/2416174896 wait complete. 2024-05-08T23:44:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2024-05-08T23:44:53.013 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 08 23:44:52 smithi204 bash[84943]: ed6d96846bcb1fdda9f5652222e36135a9758d851d1fbf84decae13dfc43ab0d 2024-05-08T23:44:53.013 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 08 23:44:52 smithi204 systemd[1]: Started Ceph mds.i for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:53.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:53 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2416174896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:53.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:53.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:53.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:53.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:53.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:53.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:53 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:53.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:53 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2416174896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:53.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:53.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:53.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:53.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:53.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:53.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:53 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:53.545 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:53 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2416174896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:53.545 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:53.545 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:53.545 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:53.545 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:53.545 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:53.545 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:53 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:54.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:54 smithi204 ceph-mon[47373]: Deploying daemon mds.g on smithi012 2024-05-08T23:44:54.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:54 smithi204 ceph-mon[47373]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:54.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:54 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] up:boot 2024-05-08T23:44:54.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:54 smithi204 ceph-mon[47373]: fsmap 9 up:standby 2024-05-08T23:44:54.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:54 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:44:54.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:54 smithi160 ceph-mon[46152]: Deploying daemon mds.g on smithi012 2024-05-08T23:44:54.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:54 smithi160 ceph-mon[46152]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:54.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:54 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] up:boot 2024-05-08T23:44:54.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:54 smithi160 ceph-mon[46152]: fsmap 9 up:standby 2024-05-08T23:44:54.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:54 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:44:54.634 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:44:54.650 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 08 23:44:54 smithi012 systemd[1]: Starting Ceph mds.g for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:44:54.650 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:54 smithi012 ceph-mon[42503]: Deploying daemon mds.g on smithi012 2024-05-08T23:44:54.650 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:54 smithi012 ceph-mon[42503]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:54.650 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:54 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] up:boot 2024-05-08T23:44:54.650 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:54 smithi012 ceph-mon[42503]: fsmap 9 up:standby 2024-05-08T23:44:54.650 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:54 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:44:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.788+0000 7f4d07a48700 1 -- 172.21.15.12:0/232835791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d000fffe0 msgr2=0x7f4d00100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.788+0000 7f4d07a48700 1 --2- 172.21.15.12:0/232835791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d000fffe0 0x7f4d00100440 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009990 tx=0x7f4cf0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:44:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.788+0000 7f4d07a48700 1 -- 172.21.15.12:0/232835791 shutdown_connections 2024-05-08T23:44:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.788+0000 7f4d07a48700 1 --2- 172.21.15.12:0/232835791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00100980 0x7f4d00108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.788+0000 7f4d07a48700 1 --2- 172.21.15.12:0/232835791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d00101340 0x7f4d00101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.788+0000 7f4d07a48700 1 --2- 172.21.15.12:0/232835791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d000fffe0 0x7f4d00100440 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.788+0000 7f4d07a48700 1 -- 172.21.15.12:0/232835791 >> 172.21.15.12:0/232835791 conn(0x7f4d000faa70 msgr2=0x7f4d000fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.788+0000 7f4d07a48700 1 -- 172.21.15.12:0/232835791 shutdown_connections 2024-05-08T23:44:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.788+0000 7f4d07a48700 1 -- 172.21.15.12:0/232835791 wait complete. 2024-05-08T23:44:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.789+0000 7f4d07a48700 1 Processor -- start 2024-05-08T23:44:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.789+0000 7f4d07a48700 1 -- start start 2024-05-08T23:44:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.789+0000 7f4d07a48700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fffe0 0x7f4d000ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.789+0000 7f4d07a48700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d00100980 0x7f4d0019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.789+0000 7f4d07a48700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00101340 0x7f4d0019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.789+0000 7f4d07a48700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0019e890 con 0x7f4d00100980 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.789+0000 7f4d07a48700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0019ea00 con 0x7f4d000fffe0 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.789+0000 7f4d07a48700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0019ebc0 con 0x7f4d00101340 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.789+0000 7f4d04fe3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d00100980 0x7f4d0019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.789+0000 7f4d04fe3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d00100980 0x7f4d0019bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37158/0 (socket says 172.21.15.12:37158) 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d04fe3700 1 -- 172.21.15.12:0/3902002047 learned_addr learned my addr 172.21.15.12:0/3902002047 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d05fe5700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00101340 0x7f4d0019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d057e4700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fffe0 0x7f4d000ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d04fe3700 1 -- 172.21.15.12:0/3902002047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00101340 msgr2=0x7f4d0019e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d04fe3700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00101340 0x7f4d0019e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d04fe3700 1 -- 172.21.15.12:0/3902002047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fffe0 msgr2=0x7f4d000ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d04fe3700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fffe0 0x7f4d000ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d04fe3700 1 -- 172.21.15.12:0/3902002047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf00095f0 con 0x7f4d00100980 2024-05-08T23:44:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d04fe3700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d00100980 0x7f4d0019bb00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00098e0 tx=0x7f4cf0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:54.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4cf67fc700 1 -- 172.21.15.12:0/3902002047 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf001d070 con 0x7f4d00100980 2024-05-08T23:44:54.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4cf67fc700 1 -- 172.21.15.12:0/3902002047 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cf0022470 con 0x7f4d00100980 2024-05-08T23:44:54.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d07a48700 1 -- 172.21.15.12:0/3902002047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0019ee40 con 0x7f4d00100980 2024-05-08T23:44:54.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4cf67fc700 1 -- 172.21.15.12:0/3902002047 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0017650 con 0x7f4d00100980 2024-05-08T23:44:54.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.790+0000 7f4d07a48700 1 -- 172.21.15.12:0/3902002047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0019f200 con 0x7f4d00100980 2024-05-08T23:44:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.792+0000 7f4d07a48700 1 -- 172.21.15.12:0/3902002047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d001072d0 con 0x7f4d00100980 2024-05-08T23:44:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.793+0000 7f4cf67fc700 1 -- 172.21.15.12:0/3902002047 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4cf0022850 con 0x7f4d00100980 2024-05-08T23:44:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.793+0000 7f4cf67fc700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4cec0a1120 0x7f4cec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.794+0000 7f4d057e4700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4cec0a1120 0x7f4cec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.794+0000 7f4cf67fc700 1 -- 172.21.15.12:0/3902002047 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4cf00cda60 con 0x7f4d00100980 2024-05-08T23:44:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.794+0000 7f4d057e4700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4cec0a1120 0x7f4cec0a35e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005950 tx=0x7f4cfc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:54.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:54.799+0000 7f4cf67fc700 1 -- 172.21.15.12:0/3902002047 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4cf0097840 con 0x7f4d00100980 2024-05-08T23:44:55.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.073+0000 7f4d07a48700 1 -- 172.21.15.12:0/3902002047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d0010b560 con 0x7f4d00100980 2024-05-08T23:44:55.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.073+0000 7f4cf67fc700 1 -- 172.21.15.12:0/3902002047 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7336 (secure 0 0 0) 0x7f4cf0027090 con 0x7f4d00100980 2024-05-08T23:44:55.069 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:55.069 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-05-08T23:44:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 -- 172.21.15.12:0/3902002047 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4cec0a1120 msgr2=0x7f4cec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4cec0a1120 0x7f4cec0a35e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005950 tx=0x7f4cfc0058e0 comp rx=0 tx=0).stop 2024-05-08T23:44:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 -- 172.21.15.12:0/3902002047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d00100980 msgr2=0x7f4d0019bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d00100980 0x7f4d0019bb00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00098e0 tx=0x7f4cf0005720 comp rx=0 tx=0).stop 2024-05-08T23:44:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 -- 172.21.15.12:0/3902002047 shutdown_connections 2024-05-08T23:44:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00101340 0x7f4d0019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4cec0a1120 0x7f4cec0a35e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d000fffe0 0x7f4d000ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 --2- 172.21.15.12:0/3902002047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d00100980 0x7f4d0019bb00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 -- 172.21.15.12:0/3902002047 >> 172.21.15.12:0/3902002047 conn(0x7f4d000faa70 msgr2=0x7f4d00104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 -- 172.21.15.12:0/3902002047 shutdown_connections 2024-05-08T23:44:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:55.076+0000 7f4d07a48700 1 -- 172.21.15.12:0/3902002047 wait complete. 2024-05-08T23:44:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 10 2024-05-08T23:44:55.086 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 08 23:44:54 smithi012 ceph-mds[94064]: -- 172.21.15.12:0/3333115653 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5629dcabc1a0 con 0x5629dca92c00 2024-05-08T23:44:55.086 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 08 23:44:54 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-g[94018]: starting mds.g at 2024-05-08T23:44:55.087 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 08 23:44:54 smithi012 bash[94003]: 95af58eef1929037ad056d412225f6fe1f007118f786b3ccfd345286ddb67054 2024-05-08T23:44:55.087 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 08 23:44:54 smithi012 systemd[1]: Started Ceph mds.g for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:56.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:56.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:56.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:56.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:56.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:56.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:56.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: Deploying daemon mds.a on smithi012 2024-05-08T23:44:56.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3902002047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:56.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:56.116 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.12:6838/3503743277,v1:172.21.15.12:6839/3503743277] up:boot 2024-05-08T23:44:56.116 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: fsmap 10 up:standby 2024-05-08T23:44:56.116 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:44:56.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: Deploying daemon mds.a on smithi012 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3902002047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.12:6838/3503743277,v1:172.21.15.12:6839/3503743277] up:boot 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: fsmap 10 up:standby 2024-05-08T23:44:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:44:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: Deploying daemon mds.a on smithi012 2024-05-08T23:44:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3902002047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.12:6838/3503743277,v1:172.21.15.12:6839/3503743277] up:boot 2024-05-08T23:44:56.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: fsmap 10 up:standby 2024-05-08T23:44:56.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:44:56.415 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 08 23:44:56 smithi012 systemd[1]: Starting Ceph mds.a for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:44:57.088 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:44:57.125 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 08 23:44:56 smithi012 bash[94391]: 15146ce08466fcacb880a78fd1f3e44efb8c90eebc873b0351e6ec335e813bb7 2024-05-08T23:44:57.125 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 08 23:44:56 smithi012 systemd[1]: Started Ceph mds.a for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:57.125 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 08 23:44:56 smithi012 ceph-mds[94429]: -- 172.21.15.12:0/990374558 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579f6df01a0 con 0x5579f6dc6800 2024-05-08T23:44:57.125 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 08 23:44:56 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-a[94406]: starting mds.a at 2024-05-08T23:44:57.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.245+0000 7f3496571700 1 -- 172.21.15.12:0/3958366894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3490101900 msgr2=0x7f34901081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:57.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.245+0000 7f3496571700 1 --2- 172.21.15.12:0/3958366894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3490101900 0x7f34901081b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f348800b0d0 tx=0x7f348800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:44:57.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.245+0000 7f3496571700 1 -- 172.21.15.12:0/3958366894 shutdown_connections 2024-05-08T23:44:57.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.245+0000 7f3496571700 1 --2- 172.21.15.12:0/3958366894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3490101900 0x7f34901081b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:57.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.245+0000 7f3496571700 1 --2- 172.21.15.12:0/3958366894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34900ffd70 0x7f3490100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:57.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.245+0000 7f3496571700 1 --2- 172.21.15.12:0/3958366894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3490100f60 0x7f34901013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:57.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.245+0000 7f3496571700 1 -- 172.21.15.12:0/3958366894 >> 172.21.15.12:0/3958366894 conn(0x7f34900fb2e0 msgr2=0x7f34900fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:57.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.245+0000 7f3496571700 1 -- 172.21.15.12:0/3958366894 shutdown_connections 2024-05-08T23:44:57.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.245+0000 7f3496571700 1 -- 172.21.15.12:0/3958366894 wait complete. 2024-05-08T23:44:57.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f3496571700 1 Processor -- start 2024-05-08T23:44:57.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f3496571700 1 -- start start 2024-05-08T23:44:57.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f3496571700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34900ffd70 0x7f3490103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:57.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f3496571700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3490100f60 0x7f3490102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:57.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f3496571700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3490101900 0x7f3490102880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:57.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f3496571700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3490103020 con 0x7f3490100f60 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f3496571700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3490103190 con 0x7f34900ffd70 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f3496571700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34901a0fd0 con 0x7f3490101900 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f348f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3490100f60 0x7f3490102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f348f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3490100f60 0x7f3490102340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37192/0 (socket says 172.21.15.12:37192) 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f348f7fe700 1 -- 172.21.15.12:0/4236863057 learned_addr learned my addr 172.21.15.12:0/4236863057 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f3494b0e700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3490101900 0x7f3490102880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.246+0000 7f348ffff700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34900ffd70 0x7f3490103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f348f7fe700 1 -- 172.21.15.12:0/4236863057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3490101900 msgr2=0x7f3490102880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f348f7fe700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3490101900 0x7f3490102880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f348f7fe700 1 -- 172.21.15.12:0/4236863057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34900ffd70 msgr2=0x7f3490103cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f348f7fe700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34900ffd70 0x7f3490103cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f348f7fe700 1 -- 172.21.15.12:0/4236863057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488009d00 con 0x7f3490100f60 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f348f7fe700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3490100f60 0x7f3490102340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f348400b530 tx=0x7f348400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:57.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f348d7fa700 1 -- 172.21.15.12:0/4236863057 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484011830 con 0x7f3490100f60 2024-05-08T23:44:57.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f348d7fa700 1 -- 172.21.15.12:0/4236863057 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3484011e50 con 0x7f3490100f60 2024-05-08T23:44:57.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f348d7fa700 1 -- 172.21.15.12:0/4236863057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484010de0 con 0x7f3490100f60 2024-05-08T23:44:57.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f3496571700 1 -- 172.21.15.12:0/4236863057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34901a1260 con 0x7f3490100f60 2024-05-08T23:44:57.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.247+0000 7f3496571700 1 -- 172.21.15.12:0/4236863057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34901a17f0 con 0x7f3490100f60 2024-05-08T23:44:57.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.248+0000 7f3496571700 1 -- 172.21.15.12:0/4236863057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3490067380 con 0x7f3490100f60 2024-05-08T23:44:57.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.249+0000 7f348d7fa700 1 -- 172.21.15.12:0/4236863057 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3484010480 con 0x7f3490100f60 2024-05-08T23:44:57.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.250+0000 7f348d7fa700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f34780a11a0 0x7f34780a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:57.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.250+0000 7f348d7fa700 1 -- 172.21.15.12:0/4236863057 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f34840cc330 con 0x7f3490100f60 2024-05-08T23:44:57.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.250+0000 7f348ffff700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f34780a11a0 0x7f34780a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:57.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.250+0000 7f348ffff700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f34780a11a0 0x7f34780a3660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3490100dc0 tx=0x7f3480009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:57.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.254+0000 7f348d7fa700 1 -- 172.21.15.12:0/4236863057 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3484095f60 con 0x7f3490100f60 2024-05-08T23:44:57.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.525+0000 7f3496571700 1 -- 172.21.15.12:0/4236863057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f349010b850 con 0x7f3490100f60 2024-05-08T23:44:57.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.526+0000 7f348d7fa700 1 -- 172.21.15.12:0/4236863057 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8852 (secure 0 0 0) 0x7f3484099580 con 0x7f3490100f60 2024-05-08T23:44:57.522 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:57.522 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-05-08T23:44:57.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.528+0000 7f3496571700 1 -- 172.21.15.12:0/4236863057 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f34780a11a0 msgr2=0x7f34780a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.528+0000 7f3496571700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f34780a11a0 0x7f34780a3660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3490100dc0 tx=0x7f3480009260 comp rx=0 tx=0).stop 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.528+0000 7f3496571700 1 -- 172.21.15.12:0/4236863057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3490100f60 msgr2=0x7f3490102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.528+0000 7f3496571700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3490100f60 0x7f3490102340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f348400b530 tx=0x7f348400b840 comp rx=0 tx=0).stop 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.529+0000 7f3496571700 1 -- 172.21.15.12:0/4236863057 shutdown_connections 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.529+0000 7f3496571700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3490101900 0x7f3490102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.529+0000 7f3496571700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f34780a11a0 0x7f34780a3660 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.529+0000 7f3496571700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34900ffd70 0x7f3490103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.529+0000 7f3496571700 1 --2- 172.21.15.12:0/4236863057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3490100f60 0x7f3490102340 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.529+0000 7f3496571700 1 -- 172.21.15.12:0/4236863057 >> 172.21.15.12:0/4236863057 conn(0x7f34900fb2e0 msgr2=0x7f34900fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.529+0000 7f3496571700 1 -- 172.21.15.12:0/4236863057 shutdown_connections 2024-05-08T23:44:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:57.529+0000 7f3496571700 1 -- 172.21.15.12:0/4236863057 wait complete. 2024-05-08T23:44:57.526 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 12 2024-05-08T23:44:57.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: Deploying daemon mds.b on smithi160 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.12:6840/2743302125,v1:172.21.15.12:6841/2743302125] up:boot 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: fsmap 11 up:standby 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:57.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:57 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/4236863057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:57.951 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: Deploying daemon mds.b on smithi160 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.12:6840/2743302125,v1:172.21.15.12:6841/2743302125] up:boot 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: fsmap 11 up:standby 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:57.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:57 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/4236863057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:58.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:58.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:44:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:44:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:44:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: Deploying daemon mds.b on smithi160 2024-05-08T23:44:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.12:6840/2743302125,v1:172.21.15.12:6841/2743302125] up:boot 2024-05-08T23:44:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: fsmap 11 up:standby 2024-05-08T23:44:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:44:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:44:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:57 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/4236863057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:44:58.766 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 23:44:58 smithi160 bash[82726]: 55b25d727a85789ca3950196914a50a862ae4ab36e38b4707568d6055cad8081 2024-05-08T23:44:58.766 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 23:44:58 smithi160 systemd[1]: Started Ceph mds.b for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:44:59.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:44:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:59.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:59 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] up:boot 2024-05-08T23:44:59.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:59 smithi012 ceph-mon[42503]: fsmap 12 up:standby 2024-05-08T23:44:59.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:44:59 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:44:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.706+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/2184063881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f8104210 msgr2=0x7fe5f8106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.706+0000 7fe5fe9ab700 1 --2- 172.21.15.12:0/2184063881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f8104210 0x7fe5f8106600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009990 tx=0x7fe5ec009ca0 comp rx=0 tx=0).stop 2024-05-08T23:44:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.707+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/2184063881 shutdown_connections 2024-05-08T23:44:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.707+0000 7fe5fe9ab700 1 --2- 172.21.15.12:0/2184063881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f8106b40 0x7fe5f8108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.707+0000 7fe5fe9ab700 1 --2- 172.21.15.12:0/2184063881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f81018e0 0x7fe5f8103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.707+0000 7fe5fe9ab700 1 --2- 172.21.15.12:0/2184063881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f8104210 0x7fe5f8106600 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.707+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/2184063881 >> 172.21.15.12:0/2184063881 conn(0x7fe5f80fb2f0 msgr2=0x7fe5f80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.707+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/2184063881 shutdown_connections 2024-05-08T23:44:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.707+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/2184063881 wait complete. 2024-05-08T23:44:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.707+0000 7fe5fe9ab700 1 Processor -- start 2024-05-08T23:44:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5fe9ab700 1 -- start start 2024-05-08T23:44:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5fe9ab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f81018e0 0x7fe5f8101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5fe9ab700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f8104210 0x7fe5f80ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5fe9ab700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f8106b40 0x7fe5f80ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5fe9ab700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f8100570 con 0x7fe5f81018e0 2024-05-08T23:44:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5fe9ab700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f81006e0 con 0x7fe5f8104210 2024-05-08T23:44:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5fe9ab700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f81a0e60 con 0x7fe5f8106b40 2024-05-08T23:44:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f81018e0 0x7fe5f8101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5f77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f8104210 0x7fe5f80ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f81018e0 0x7fe5f8101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53160/0 (socket says 172.21.15.12:53160) 2024-05-08T23:44:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5f7fff700 1 -- 172.21.15.12:0/1973677491 learned_addr learned my addr 172.21.15.12:0/1973677491 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:44:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.708+0000 7fe5fcf48700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f8106b40 0x7fe5f80ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5f77fe700 1 -- 172.21.15.12:0/1973677491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f8106b40 msgr2=0x7fe5f80ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5f77fe700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f8106b40 0x7fe5f80ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5f77fe700 1 -- 172.21.15.12:0/1973677491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f81018e0 msgr2=0x7fe5f8101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5f77fe700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f81018e0 0x7fe5f8101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5f77fe700 1 -- 172.21.15.12:0/1973677491 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec0095f0 con 0x7fe5f8104210 2024-05-08T23:44:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5fcf48700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f8106b40 0x7fe5f80ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:44:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5f77fe700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f8104210 0x7fe5f80ff890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec000c00 tx=0x7fe5ec009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5f57fa700 1 -- 172.21.15.12:0/1973677491 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec01d070 con 0x7fe5f8104210 2024-05-08T23:44:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5f57fa700 1 -- 172.21.15.12:0/1973677491 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5ec021410 con 0x7fe5f8104210 2024-05-08T23:44:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5f57fa700 1 -- 172.21.15.12:0/1973677491 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec017650 con 0x7fe5f8104210 2024-05-08T23:44:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/1973677491 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f81a1090 con 0x7fe5f8104210 2024-05-08T23:44:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.709+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/1973677491 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f81a1450 con 0x7fe5f8104210 2024-05-08T23:44:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.711+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/1973677491 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f80fce70 con 0x7fe5f8104210 2024-05-08T23:44:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.712+0000 7fe5f57fa700 1 -- 172.21.15.12:0/1973677491 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe5ec00fb10 con 0x7fe5f8104210 2024-05-08T23:44:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.713+0000 7fe5f57fa700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5e00a11a0 0x7fe5e00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:44:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.713+0000 7fe5f57fa700 1 -- 172.21.15.12:0/1973677491 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe5ec0cd710 con 0x7fe5f8104210 2024-05-08T23:44:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.713+0000 7fe5f7fff700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5e00a11a0 0x7fe5e00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:44:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.714+0000 7fe5f7fff700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5e00a11a0 0x7fe5e00a3660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009d20 tx=0x7fe5e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:44:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.718+0000 7fe5f57fa700 1 -- 172.21.15.12:0/1973677491 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe5ec0973d0 con 0x7fe5f8104210 2024-05-08T23:44:59.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:59 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] up:boot 2024-05-08T23:44:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:59 smithi204 ceph-mon[47373]: fsmap 12 up:standby 2024-05-08T23:44:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:44:59 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:44:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:44:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:44:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:59 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] up:boot 2024-05-08T23:44:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:59 smithi160 ceph-mon[46152]: fsmap 12 up:standby 2024-05-08T23:44:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:44:59 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:44:59.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.994+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/1973677491 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5f8067c10 con 0x7fe5f8104210 2024-05-08T23:44:59.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.995+0000 7fe5f57fa700 1 -- 172.21.15.12:0/1973677491 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9610 (secure 0 0 0) 0x7fe5ec09a9f0 con 0x7fe5f8104210 2024-05-08T23:44:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:44:59.993 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24548,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-08T23:44:59.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/1973677491 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5e00a11a0 msgr2=0x7fe5e00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:59.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5e00a11a0 0x7fe5e00a3660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009d20 tx=0x7fe5e8009260 comp rx=0 tx=0).stop 2024-05-08T23:44:59.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/1973677491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f8104210 msgr2=0x7fe5f80ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:44:59.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f8104210 0x7fe5f80ff890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec000c00 tx=0x7fe5ec009580 comp rx=0 tx=0).stop 2024-05-08T23:44:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/1973677491 shutdown_connections 2024-05-08T23:44:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f8106b40 0x7fe5f80ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f81018e0 0x7fe5f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5e00a11a0 0x7fe5e00a3660 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 --2- 172.21.15.12:0/1973677491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f8104210 0x7fe5f80ff890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:44:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/1973677491 >> 172.21.15.12:0/1973677491 conn(0x7fe5f80fb2f0 msgr2=0x7fe5f81037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:44:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/1973677491 shutdown_connections 2024-05-08T23:44:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:44:59.998+0000 7fe5fe9ab700 1 -- 172.21.15.12:0/1973677491 wait complete. 2024-05-08T23:44:59.996 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2024-05-08T23:45:00.008 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:45:00.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.151+0000 7f5eca1d0700 1 -- 172.21.15.12:0/3534487321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ec4100a80 msgr2=0x7f5ec4100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:00.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.151+0000 7f5eca1d0700 1 --2- 172.21.15.12:0/3534487321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ec4100a80 0x7f5ec4100f00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00b0d0 tx=0x7f5ebc00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.151+0000 7f5eca1d0700 1 -- 172.21.15.12:0/3534487321 shutdown_connections 2024-05-08T23:45:00.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.151+0000 7f5eca1d0700 1 --2- 172.21.15.12:0/3534487321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ec4100a80 0x7f5ec4100f00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.151+0000 7f5eca1d0700 1 --2- 172.21.15.12:0/3534487321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec40ff760 0x7f5ec40ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.151+0000 7f5eca1d0700 1 --2- 172.21.15.12:0/3534487321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ec41000c0 0x7f5ec4100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.151+0000 7f5eca1d0700 1 -- 172.21.15.12:0/3534487321 >> 172.21.15.12:0/3534487321 conn(0x7f5ec40fb300 msgr2=0x7f5ec40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:00.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.151+0000 7f5eca1d0700 1 -- 172.21.15.12:0/3534487321 shutdown_connections 2024-05-08T23:45:00.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.151+0000 7f5eca1d0700 1 -- 172.21.15.12:0/3534487321 wait complete. 2024-05-08T23:45:00.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5eca1d0700 1 Processor -- start 2024-05-08T23:45:00.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5eca1d0700 1 -- start start 2024-05-08T23:45:00.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5eca1d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ec40ff760 0x7f5ec41990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:00.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5eca1d0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41000c0 0x7f5ec4199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:00.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5eca1d0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ec4100a80 0x7f5ec41a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5eca1d0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec41a0c10 con 0x7f5ec40ff760 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5eca1d0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec41a0d50 con 0x7f5ec4100a80 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5eca1d0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec41a0ee0 con 0x7f5ec41000c0 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5ec37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ec40ff760 0x7f5ec41990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5ec37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ec40ff760 0x7f5ec41990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53184/0 (socket says 172.21.15.12:53184) 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5ec37fe700 1 -- 172.21.15.12:0/1528254300 learned_addr learned my addr 172.21.15.12:0/1528254300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5ec2ffd700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41000c0 0x7f5ec4199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.152+0000 7f5ec3fff700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ec4100a80 0x7f5ec41a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5ec3fff700 1 -- 172.21.15.12:0/1528254300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41000c0 msgr2=0x7f5ec4199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5ec3fff700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41000c0 0x7f5ec4199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5ec3fff700 1 -- 172.21.15.12:0/1528254300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ec40ff760 msgr2=0x7f5ec41990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5ec3fff700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ec40ff760 0x7f5ec41990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5ec3fff700 1 -- 172.21.15.12:0/1528254300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc009d00 con 0x7f5ec4100a80 2024-05-08T23:45:00.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5ec37fe700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ec40ff760 0x7f5ec41990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5ec3fff700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ec4100a80 0x7f5ec41a0640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc007b60 tx=0x7f5ebc008e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5ec0ff9700 1 -- 172.21.15.12:0/1528254300 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00e040 con 0x7f5ec4100a80 2024-05-08T23:45:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5ec0ff9700 1 -- 172.21.15.12:0/1528254300 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ebc005070 con 0x7f5ec4100a80 2024-05-08T23:45:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5eca1d0700 1 -- 172.21.15.12:0/1528254300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec41a1160 con 0x7f5ec4100a80 2024-05-08T23:45:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5ec0ff9700 1 -- 172.21.15.12:0/1528254300 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc01d6d0 con 0x7f5ec4100a80 2024-05-08T23:45:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.153+0000 7f5eca1d0700 1 -- 172.21.15.12:0/1528254300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec41a15e0 con 0x7f5ec4100a80 2024-05-08T23:45:00.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.154+0000 7f5eca1d0700 1 -- 172.21.15.12:0/1528254300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec410a6d0 con 0x7f5ec4100a80 2024-05-08T23:45:00.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.155+0000 7f5ec0ff9700 1 -- 172.21.15.12:0/1528254300 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5ebc003a00 con 0x7f5ec4100a80 2024-05-08T23:45:00.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.156+0000 7f5ec0ff9700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5eac0a11a0 0x7f5eac0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:00.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.156+0000 7f5ec37fe700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5eac0a11a0 0x7f5eac0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:00.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.156+0000 7f5ec0ff9700 1 -- 172.21.15.12:0/1528254300 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5ebc0cd2e0 con 0x7f5ec4100a80 2024-05-08T23:45:00.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.156+0000 7f5ec37fe700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5eac0a11a0 0x7f5eac0a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40104e0 tx=0x7f5eb4007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:00.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.160+0000 7f5ec0ff9700 1 -- 172.21.15.12:0/1528254300 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5ebc097030 con 0x7f5ec4100a80 2024-05-08T23:45:00.376 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:45:00.376 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.376 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.376 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.376 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1973677491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:00.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:45:00.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:45:00.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.377 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:00 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:45:00.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.402+0000 7f5eca1d0700 1 -- 172.21.15.12:0/1528254300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ec410e8e0 con 0x7f5ec4100a80 2024-05-08T23:45:00.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.403+0000 7f5ec0ff9700 1 -- 172.21.15.12:0/1528254300 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18408 (secure 0 0 0) 0x7f5ebc003cb0 con 0x7f5ec4100a80 2024-05-08T23:45:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:00.399 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":77,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:44:31.131279+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:45:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.405+0000 7f5eca1d0700 1 -- 172.21.15.12:0/1528254300 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5eac0a11a0 msgr2=0x7f5eac0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.405+0000 7f5eca1d0700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5eac0a11a0 0x7f5eac0a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40104e0 tx=0x7f5eb4007480 comp rx=0 tx=0).stop 2024-05-08T23:45:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.405+0000 7f5eca1d0700 1 -- 172.21.15.12:0/1528254300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ec4100a80 msgr2=0x7f5ec41a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.405+0000 7f5eca1d0700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ec4100a80 0x7f5ec41a0640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc007b60 tx=0x7f5ebc008e90 comp rx=0 tx=0).stop 2024-05-08T23:45:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.405+0000 7f5eca1d0700 1 -- 172.21.15.12:0/1528254300 shutdown_connections 2024-05-08T23:45:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.405+0000 7f5eca1d0700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5eac0a11a0 0x7f5eac0a3660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.405+0000 7f5eca1d0700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ec4100a80 0x7f5ec41a0640 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.405+0000 7f5eca1d0700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ec40ff760 0x7f5ec41990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.405+0000 7f5eca1d0700 1 --2- 172.21.15.12:0/1528254300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41000c0 0x7f5ec4199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.405+0000 7f5eca1d0700 1 -- 172.21.15.12:0/1528254300 >> 172.21.15.12:0/1528254300 conn(0x7f5ec40fb300 msgr2=0x7f5ec40fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.406+0000 7f5eca1d0700 1 -- 172.21.15.12:0/1528254300 shutdown_connections 2024-05-08T23:45:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.406+0000 7f5eca1d0700 1 -- 172.21.15.12:0/1528254300 wait complete. 2024-05-08T23:45:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:45:00.417 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:45:00.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.588+0000 7fbbf8226700 1 -- 172.21.15.12:0/3918778920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbf0104320 msgr2=0x7fbbf0106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:00.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.588+0000 7fbbf8226700 1 --2- 172.21.15.12:0/3918778920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbf0104320 0x7fbbf0106710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009990 tx=0x7fbbe0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.588+0000 7fbbf8226700 1 -- 172.21.15.12:0/3918778920 shutdown_connections 2024-05-08T23:45:00.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.588+0000 7fbbf8226700 1 --2- 172.21.15.12:0/3918778920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbf0106c50 0x7fbbf01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.588+0000 7fbbf8226700 1 --2- 172.21.15.12:0/3918778920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf01019f0 0x7fbbf0103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.588+0000 7fbbf8226700 1 --2- 172.21.15.12:0/3918778920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbf0104320 0x7fbbf0106710 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.588+0000 7fbbf8226700 1 -- 172.21.15.12:0/3918778920 >> 172.21.15.12:0/3918778920 conn(0x7fbbf00fb3e0 msgr2=0x7fbbf00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:00.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.589+0000 7fbbf8226700 1 -- 172.21.15.12:0/3918778920 shutdown_connections 2024-05-08T23:45:00.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.589+0000 7fbbf8226700 1 -- 172.21.15.12:0/3918778920 wait complete. 2024-05-08T23:45:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.589+0000 7fbbf8226700 1 Processor -- start 2024-05-08T23:45:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf8226700 1 -- start start 2024-05-08T23:45:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf8226700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbf01019f0 0x7fbbf0196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf8226700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbf0104320 0x7fbbf0197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf8226700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0106c50 0x7fbbf019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf8226700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf019e9f0 con 0x7fbbf01019f0 2024-05-08T23:45:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf8226700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf019eb30 con 0x7fbbf0104320 2024-05-08T23:45:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf8226700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf019ecc0 con 0x7fbbf0106c50 2024-05-08T23:45:00.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf5fc2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbf01019f0 0x7fbbf0196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:00.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf57c1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbf0104320 0x7fbbf0197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:00.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf5fc2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbf01019f0 0x7fbbf0196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53210/0 (socket says 172.21.15.12:53210) 2024-05-08T23:45:00.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf5fc2700 1 -- 172.21.15.12:0/2985948180 learned_addr learned my addr 172.21.15.12:0/2985948180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:00.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf67c3700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0106c50 0x7fbbf019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:00.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf57c1700 1 -- 172.21.15.12:0/2985948180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0106c50 msgr2=0x7fbbf019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:00.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf57c1700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0106c50 0x7fbbf019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf57c1700 1 -- 172.21.15.12:0/2985948180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbf01019f0 msgr2=0x7fbbf0196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf57c1700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbf01019f0 0x7fbbf0196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.590+0000 7fbbf57c1700 1 -- 172.21.15.12:0/2985948180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00095f0 con 0x7fbbf0104320 2024-05-08T23:45:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.591+0000 7fbbf57c1700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbf0104320 0x7fbbf0197400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00098e0 tx=0x7fbbe0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.591+0000 7fbbe6ffd700 1 -- 172.21.15.12:0/2985948180 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe001d070 con 0x7fbbf0104320 2024-05-08T23:45:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.591+0000 7fbbe6ffd700 1 -- 172.21.15.12:0/2985948180 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe0021410 con 0x7fbbf0104320 2024-05-08T23:45:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.591+0000 7fbbf8226700 1 -- 172.21.15.12:0/2985948180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf019ef40 con 0x7fbbf0104320 2024-05-08T23:45:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.591+0000 7fbbe6ffd700 1 -- 172.21.15.12:0/2985948180 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0017650 con 0x7fbbf0104320 2024-05-08T23:45:00.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.591+0000 7fbbf8226700 1 -- 172.21.15.12:0/2985948180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf019f360 con 0x7fbbf0104320 2024-05-08T23:45:00.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.592+0000 7fbbf8226700 1 -- 172.21.15.12:0/2985948180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf00fcf60 con 0x7fbbf0104320 2024-05-08T23:45:00.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.593+0000 7fbbe6ffd700 1 -- 172.21.15.12:0/2985948180 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbbe000fb10 con 0x7fbbf0104320 2024-05-08T23:45:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.594+0000 7fbbe6ffd700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbdc0a11a0 0x7fbbdc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.594+0000 7fbbf5fc2700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbdc0a11a0 0x7fbbdc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.594+0000 7fbbe6ffd700 1 -- 172.21.15.12:0/2985948180 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbbe00cd600 con 0x7fbbf0104320 2024-05-08T23:45:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.595+0000 7fbbf5fc2700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbdc0a11a0 0x7fbbdc0a3660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbbec007d10 tx=0x7fbbec007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:00.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.600+0000 7fbbe6ffd700 1 -- 172.21.15.12:0/2985948180 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbbe00972c0 con 0x7fbbf0104320 2024-05-08T23:45:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:45:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1973677491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:45:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:45:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:00 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:45:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:45:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1973677491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:45:00.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:45:00.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:00.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:00 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:45:00.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.847+0000 7fbbf8226700 1 -- 172.21.15.12:0/2985948180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbf010c960 con 0x7fbbf0104320 2024-05-08T23:45:00.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.847+0000 7fbbe6ffd700 1 -- 172.21.15.12:0/2985948180 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbe009a8e0 con 0x7fbbf0104320 2024-05-08T23:45:00.843 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-08T23:45:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 -- 172.21.15.12:0/2985948180 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbdc0a11a0 msgr2=0x7fbbdc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbdc0a11a0 0x7fbbdc0a3660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbbec007d10 tx=0x7fbbec007480 comp rx=0 tx=0).stop 2024-05-08T23:45:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 -- 172.21.15.12:0/2985948180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbf0104320 msgr2=0x7fbbf0197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbf0104320 0x7fbbf0197400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00098e0 tx=0x7fbbe0005740 comp rx=0 tx=0).stop 2024-05-08T23:45:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 -- 172.21.15.12:0/2985948180 shutdown_connections 2024-05-08T23:45:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0106c50 0x7fbbf019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbf01019f0 0x7fbbf0196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbdc0a11a0 0x7fbbdc0a3660 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 --2- 172.21.15.12:0/2985948180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbf0104320 0x7fbbf0197400 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 -- 172.21.15.12:0/2985948180 >> 172.21.15.12:0/2985948180 conn(0x7fbbf00fb3e0 msgr2=0x7fbbf01038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 -- 172.21.15.12:0/2985948180 shutdown_connections 2024-05-08T23:45:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:00.850+0000 7fbbf8226700 1 -- 172.21.15.12:0/2985948180 wait complete. 2024-05-08T23:45:00.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:45:01.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.010+0000 7fdf55847700 1 -- 172.21.15.12:0/3050213142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf50104240 msgr2=0x7fdf50106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:01.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.010+0000 7fdf55847700 1 --2- 172.21.15.12:0/3050213142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf50104240 0x7fdf50106630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009990 tx=0x7fdf38009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.010+0000 7fdf55847700 1 -- 172.21.15.12:0/3050213142 shutdown_connections 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.010+0000 7fdf55847700 1 --2- 172.21.15.12:0/3050213142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf50106b70 0x7fdf50108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.010+0000 7fdf55847700 1 --2- 172.21.15.12:0/3050213142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf50101910 0x7fdf50103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.010+0000 7fdf55847700 1 --2- 172.21.15.12:0/3050213142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf50104240 0x7fdf50106630 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.010+0000 7fdf55847700 1 -- 172.21.15.12:0/3050213142 >> 172.21.15.12:0/3050213142 conn(0x7fdf500fb320 msgr2=0x7fdf500fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.010+0000 7fdf55847700 1 -- 172.21.15.12:0/3050213142 shutdown_connections 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.010+0000 7fdf55847700 1 -- 172.21.15.12:0/3050213142 wait complete. 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf55847700 1 Processor -- start 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf55847700 1 -- start start 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf55847700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf50101910 0x7fdf50101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf55847700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf50104240 0x7fdf500ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf55847700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf50106b70 0x7fdf500ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf55847700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501005a0 con 0x7fdf50104240 2024-05-08T23:45:01.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf55847700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50100710 con 0x7fdf50101910 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf55847700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501a0e60 con 0x7fdf50106b70 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf4e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf50104240 0x7fdf500ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf4e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf50104240 0x7fdf500ff8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53220/0 (socket says 172.21.15.12:53220) 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf4e7fc700 1 -- 172.21.15.12:0/1888909574 learned_addr learned my addr 172.21.15.12:0/1888909574 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf4f7fe700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf50106b70 0x7fdf500ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.011+0000 7fdf4effd700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf50101910 0x7fdf50101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf4e7fc700 1 -- 172.21.15.12:0/1888909574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf50106b70 msgr2=0x7fdf500ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf4e7fc700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf50106b70 0x7fdf500ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf4e7fc700 1 -- 172.21.15.12:0/1888909574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf50101910 msgr2=0x7fdf50101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf4e7fc700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf50101910 0x7fdf50101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf4e7fc700 1 -- 172.21.15.12:0/1888909574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380095f0 con 0x7fdf50104240 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf4e7fc700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf50104240 0x7fdf500ff8c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf38005910 tx=0x7fdf38009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:01.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf54845700 1 -- 172.21.15.12:0/1888909574 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3801d070 con 0x7fdf50104240 2024-05-08T23:45:01.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf54845700 1 -- 172.21.15.12:0/1888909574 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf38021410 con 0x7fdf50104240 2024-05-08T23:45:01.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf55847700 1 -- 172.21.15.12:0/1888909574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf501a1090 con 0x7fdf50104240 2024-05-08T23:45:01.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf54845700 1 -- 172.21.15.12:0/1888909574 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38005720 con 0x7fdf50104240 2024-05-08T23:45:01.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.012+0000 7fdf55847700 1 -- 172.21.15.12:0/1888909574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf501a1450 con 0x7fdf50104240 2024-05-08T23:45:01.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.013+0000 7fdf55847700 1 -- 172.21.15.12:0/1888909574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf500fcea0 con 0x7fdf50104240 2024-05-08T23:45:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.014+0000 7fdf54845700 1 -- 172.21.15.12:0/1888909574 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdf38021580 con 0x7fdf50104240 2024-05-08T23:45:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.015+0000 7fdf54845700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf3c0a1120 0x7fdf3c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.015+0000 7fdf4effd700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf3c0a1120 0x7fdf3c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.015+0000 7fdf54845700 1 -- 172.21.15.12:0/1888909574 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdf380cdfa0 con 0x7fdf50104240 2024-05-08T23:45:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.016+0000 7fdf4effd700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf3c0a1120 0x7fdf3c0a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009c60 tx=0x7fdf40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:01.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.019+0000 7fdf54845700 1 -- 172.21.15.12:0/1888909574 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdf38097bc0 con 0x7fdf50104240 2024-05-08T23:45:01.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.275+0000 7fdf55847700 1 -- 172.21.15.12:0/1888909574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdf5010c6d0 con 0x7fdf50104240 2024-05-08T23:45:01.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.276+0000 7fdf54845700 1 -- 172.21.15.12:0/1888909574 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdf3809b1e0 con 0x7fdf50104240 2024-05-08T23:45:01.272 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:01.272 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2024-05-08T23:45:01.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-08T23:45:01.272 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2024-05-08T23:45:01.273 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2024-05-08T23:45:01.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:45:01.273 INFO:teuthology.orchestra.run.smithi012.stdout:] 2024-05-08T23:45:01.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 -- 172.21.15.12:0/1888909574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf3c0a1120 msgr2=0x7fdf3c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:01.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf3c0a1120 0x7fdf3c0a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009c60 tx=0x7fdf40009260 comp rx=0 tx=0).stop 2024-05-08T23:45:01.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 -- 172.21.15.12:0/1888909574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf50104240 msgr2=0x7fdf500ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:01.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf50104240 0x7fdf500ff8c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf38005910 tx=0x7fdf38009580 comp rx=0 tx=0).stop 2024-05-08T23:45:01.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 -- 172.21.15.12:0/1888909574 shutdown_connections 2024-05-08T23:45:01.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf50106b70 0x7fdf500ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fdf3c0a1120 0x7fdf3c0a35e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf50101910 0x7fdf50101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 --2- 172.21.15.12:0/1888909574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf50104240 0x7fdf500ff8c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 -- 172.21.15.12:0/1888909574 >> 172.21.15.12:0/1888909574 conn(0x7fdf500fb320 msgr2=0x7fdf501037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 -- 172.21.15.12:0/1888909574 shutdown_connections 2024-05-08T23:45:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.279+0000 7fdf55847700 1 -- 172.21.15.12:0/1888909574 wait complete. 2024-05-08T23:45:01.291 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:45:01.292 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:45:01.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.454+0000 7f72f642e700 1 -- 172.21.15.12:0/3303309195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f00ffe60 msgr2=0x7f72f0100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:01.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.454+0000 7f72f642e700 1 --2- 172.21.15.12:0/3303309195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f00ffe60 0x7f72f0100280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009990 tx=0x7f72e0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.455+0000 7f72f642e700 1 -- 172.21.15.12:0/3303309195 shutdown_connections 2024-05-08T23:45:01.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.455+0000 7f72f642e700 1 --2- 172.21.15.12:0/3303309195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72f0101970 0x7f72f0108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.455+0000 7f72f642e700 1 --2- 172.21.15.12:0/3303309195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f00ffe60 0x7f72f0100280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.455+0000 7f72f642e700 1 --2- 172.21.15.12:0/3303309195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72f0100fb0 0x7f72f0101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.455+0000 7f72f642e700 1 -- 172.21.15.12:0/3303309195 >> 172.21.15.12:0/3303309195 conn(0x7f72f00fb3f0 msgr2=0x7f72f00fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:01.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.455+0000 7f72f642e700 1 -- 172.21.15.12:0/3303309195 shutdown_connections 2024-05-08T23:45:01.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.455+0000 7f72f642e700 1 -- 172.21.15.12:0/3303309195 wait complete. 2024-05-08T23:45:01.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.455+0000 7f72f642e700 1 Processor -- start 2024-05-08T23:45:01.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.455+0000 7f72f642e700 1 -- start start 2024-05-08T23:45:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f642e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72f00ffe60 0x7f72f0196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f642e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72f0100fb0 0x7f72f0197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f642e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f0101970 0x7f72f019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f642e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f019ea70 con 0x7f72f0101970 2024-05-08T23:45:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f642e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f019ebb0 con 0x7f72f00ffe60 2024-05-08T23:45:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f642e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f019ed40 con 0x7f72f0100fb0 2024-05-08T23:45:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f49cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f0101970 0x7f72f019e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f49cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f0101970 0x7f72f019e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53234/0 (socket says 172.21.15.12:53234) 2024-05-08T23:45:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f49cb700 1 -- 172.21.15.12:0/1187447364 learned_addr learned my addr 172.21.15.12:0/1187447364 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72ef7fe700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72f0100fb0 0x7f72f0197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72effff700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72f00ffe60 0x7f72f0196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f49cb700 1 -- 172.21.15.12:0/1187447364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72f0100fb0 msgr2=0x7f72f0197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f49cb700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72f0100fb0 0x7f72f0197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f49cb700 1 -- 172.21.15.12:0/1187447364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72f00ffe60 msgr2=0x7f72f0196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f49cb700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72f00ffe60 0x7f72f0196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f49cb700 1 -- 172.21.15.12:0/1187447364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e8009c40 con 0x7f72f0101970 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72ef7fe700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72f0100fb0 0x7f72f0197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.456+0000 7f72f49cb700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f0101970 0x7f72f019e4a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f72e800c370 tx=0x7f72e800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.457+0000 7f72ed7fa700 1 -- 172.21.15.12:0/1187447364 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8017070 con 0x7f72f0101970 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.457+0000 7f72ed7fa700 1 -- 172.21.15.12:0/1187447364 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e800f040 con 0x7f72f0101970 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.457+0000 7f72f642e700 1 -- 172.21.15.12:0/1187447364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e00095f0 con 0x7f72f0101970 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.457+0000 7f72ed7fa700 1 -- 172.21.15.12:0/1187447364 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e800e050 con 0x7f72f0101970 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.457+0000 7f72f642e700 1 -- 172.21.15.12:0/1187447364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f019f300 con 0x7f72f0101970 2024-05-08T23:45:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.458+0000 7f72f642e700 1 -- 172.21.15.12:0/1187447364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f0067cf0 con 0x7f72f0101970 2024-05-08T23:45:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.460+0000 7f72ed7fa700 1 -- 172.21.15.12:0/1187447364 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f72e800cb60 con 0x7f72f0101970 2024-05-08T23:45:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.460+0000 7f72ed7fa700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f72d80a11a0 0x7f72d80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.460+0000 7f72ed7fa700 1 -- 172.21.15.12:0/1187447364 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f72e80cbd00 con 0x7f72f0101970 2024-05-08T23:45:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.460+0000 7f72effff700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f72d80a11a0 0x7f72d80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:01.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.461+0000 7f72effff700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f72d80a11a0 0x7f72d80a3660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f72e000b2e0 tx=0x7f72e000b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:01.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.466+0000 7f72ed7fa700 1 -- 172.21.15.12:0/1187447364 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f72e8095930 con 0x7f72f0101970 2024-05-08T23:45:01.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:01 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1528254300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:01.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:01 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2985948180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:45:01.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:01 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1888909574' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T23:45:01.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:01 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:01 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:01 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:01 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:01.713+0000 7f72f642e700 1 -- 172.21.15.12:0/1187447364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f72f010b8d0 con 0x7f72f0101970 2024-05-08T23:45:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:01 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1528254300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:01 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2985948180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:45:01.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:01 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1888909574' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T23:45:01.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:01 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:01 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:01 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:01 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:01 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1528254300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:01.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:01 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2985948180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:45:01.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:01 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1888909574' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T23:45:01.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:01 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:01 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:01 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:01.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:01 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:02.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.553+0000 7f72ed7fa700 1 -- 172.21.15.12:0/1187447364 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7f72e8098f50 con 0x7f72f0101970 2024-05-08T23:45:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.557+0000 7f72f642e700 1 -- 172.21.15.12:0/1187447364 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f72d80a11a0 msgr2=0x7f72d80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.557+0000 7f72f642e700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f72d80a11a0 0x7f72d80a3660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f72e000b2e0 tx=0x7f72e000b6f0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.558+0000 7f72f642e700 1 -- 172.21.15.12:0/1187447364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f0101970 msgr2=0x7f72f019e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.558+0000 7f72f642e700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f0101970 0x7f72f019e4a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f72e800c370 tx=0x7f72e800c680 comp rx=0 tx=0).stop 2024-05-08T23:45:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.558+0000 7f72f642e700 1 -- 172.21.15.12:0/1187447364 shutdown_connections 2024-05-08T23:45:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.558+0000 7f72f642e700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72f0101970 0x7f72f019e4a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.558+0000 7f72f642e700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f72d80a11a0 0x7f72d80a3660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.558+0000 7f72f642e700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72f00ffe60 0x7f72f0196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.558+0000 7f72f642e700 1 --2- 172.21.15.12:0/1187447364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72f0100fb0 0x7f72f0197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.558+0000 7f72f642e700 1 -- 172.21.15.12:0/1187447364 >> 172.21.15.12:0/1187447364 conn(0x7f72f00fb3f0 msgr2=0x7f72f00680e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:02.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.558+0000 7f72f642e700 1 -- 172.21.15.12:0/1187447364 shutdown_connections 2024-05-08T23:45:02.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.558+0000 7f72f642e700 1 -- 172.21.15.12:0/1187447364 wait complete. 2024-05-08T23:45:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2024-05-08T23:45:02.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:45:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.724+0000 7f9756deb700 1 -- 172.21.15.12:0/2784468851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97501020f0 msgr2=0x7f9750102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.724+0000 7f9756deb700 1 --2- 172.21.15.12:0/2784468851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97501020f0 0x7f9750102570 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9738009990 tx=0x7f9738009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.724+0000 7f9756deb700 1 -- 172.21.15.12:0/2784468851 shutdown_connections 2024-05-08T23:45:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.724+0000 7f9756deb700 1 --2- 172.21.15.12:0/2784468851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9750102ab0 0x7f97501093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.724+0000 7f9756deb700 1 --2- 172.21.15.12:0/2784468851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9750100f20 0x7f9750101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.724+0000 7f9756deb700 1 --2- 172.21.15.12:0/2784468851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97501020f0 0x7f9750102570 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.724+0000 7f9756deb700 1 -- 172.21.15.12:0/2784468851 >> 172.21.15.12:0/2784468851 conn(0x7f97500fc4e0 msgr2=0x7f97500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.724+0000 7f9756deb700 1 -- 172.21.15.12:0/2784468851 shutdown_connections 2024-05-08T23:45:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.724+0000 7f9756deb700 1 -- 172.21.15.12:0/2784468851 wait complete. 2024-05-08T23:45:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f9756deb700 1 Processor -- start 2024-05-08T23:45:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f9756deb700 1 -- start start 2024-05-08T23:45:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f9756deb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9750100f20 0x7f9750196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f9756deb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97501020f0 0x7f9750197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f9756deb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9750102ab0 0x7f975019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f9756deb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975019ea30 con 0x7f97501020f0 2024-05-08T23:45:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f9756deb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975019eb70 con 0x7f9750102ab0 2024-05-08T23:45:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f9756deb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975019ed00 con 0x7f9750100f20 2024-05-08T23:45:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f974ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97501020f0 0x7f9750197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f974ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97501020f0 0x7f9750197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53258/0 (socket says 172.21.15.12:53258) 2024-05-08T23:45:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.725+0000 7f974ffff700 1 -- 172.21.15.12:0/1697968775 learned_addr learned my addr 172.21.15.12:0/1697968775 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f9754b87700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9750100f20 0x7f9750196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f9755388700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9750102ab0 0x7f975019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f974ffff700 1 -- 172.21.15.12:0/1697968775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9750100f20 msgr2=0x7f9750196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f974ffff700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9750100f20 0x7f9750196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f974ffff700 1 -- 172.21.15.12:0/1697968775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9750102ab0 msgr2=0x7f975019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f974ffff700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9750102ab0 0x7f975019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f974ffff700 1 -- 172.21.15.12:0/1697968775 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380095f0 con 0x7f97501020f0 2024-05-08T23:45:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f9755388700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9750102ab0 0x7f975019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f974ffff700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97501020f0 0x7f9750197440 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f973800c010 tx=0x7f9738004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f974dffb700 1 -- 172.21.15.12:0/1697968775 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973801d070 con 0x7f97501020f0 2024-05-08T23:45:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f974dffb700 1 -- 172.21.15.12:0/1697968775 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97380054f0 con 0x7f97501020f0 2024-05-08T23:45:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f9756deb700 1 -- 172.21.15.12:0/1697968775 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975019ef80 con 0x7f97501020f0 2024-05-08T23:45:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f974dffb700 1 -- 172.21.15.12:0/1697968775 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738004d10 con 0x7f97501020f0 2024-05-08T23:45:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.726+0000 7f9756deb700 1 -- 172.21.15.12:0/1697968775 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975019f2f0 con 0x7f97501020f0 2024-05-08T23:45:02.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.727+0000 7f9756deb700 1 -- 172.21.15.12:0/1697968775 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97501084b0 con 0x7f97501020f0 2024-05-08T23:45:02.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.729+0000 7f974dffb700 1 -- 172.21.15.12:0/1697968775 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f973800fc50 con 0x7f97501020f0 2024-05-08T23:45:02.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.729+0000 7f974dffb700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f973c0a1120 0x7f973c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:02.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.729+0000 7f9754b87700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f973c0a1120 0x7f973c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:02.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.729+0000 7f974dffb700 1 -- 172.21.15.12:0/1697968775 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f97380cd9d0 con 0x7f97501020f0 2024-05-08T23:45:02.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.730+0000 7f9754b87700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f973c0a1120 0x7f973c0a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9750100c40 tx=0x7f9740009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:02.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.734+0000 7f974dffb700 1 -- 172.21.15.12:0/1697968775 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9738097520 con 0x7f97501020f0 2024-05-08T23:45:02.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:02 smithi012 ceph-mon[42503]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:45:02.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:02 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:02.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:02 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:02.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:02 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:45:02.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:02 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:45:02.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:02 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:02.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:02 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1187447364' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T23:45:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:02.953+0000 7f9756deb700 1 -- 172.21.15.12:0/1697968775 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9750067460 con 0x7f97501020f0 2024-05-08T23:45:03.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:02 smithi204 ceph-mon[47373]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:45:03.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:02 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:03.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:02 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:03.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:02 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:45:03.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:02 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:45:03.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:02 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:03.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:02 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1187447364' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T23:45:03.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:02 smithi160 ceph-mon[46152]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:45:03.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:02 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:03.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:02 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:03.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:02 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:45:03.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:02 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:45:03.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:02 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:45:03.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:02 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1187447364' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T23:45:03.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.568+0000 7f974dffb700 1 -- 172.21.15.12:0/1697968775 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) v1 ==== 170+0+0 (secure 0 0 0) 0x7f973809ab40 con 0x7f97501020f0 2024-05-08T23:45:03.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.574+0000 7f9756deb700 1 -- 172.21.15.12:0/1697968775 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f973c0a1120 msgr2=0x7f973c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:03.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.574+0000 7f9756deb700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f973c0a1120 0x7f973c0a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9750100c40 tx=0x7f9740009260 comp rx=0 tx=0).stop 2024-05-08T23:45:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.574+0000 7f9756deb700 1 -- 172.21.15.12:0/1697968775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97501020f0 msgr2=0x7f9750197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.574+0000 7f9756deb700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97501020f0 0x7f9750197440 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f973800c010 tx=0x7f9738004730 comp rx=0 tx=0).stop 2024-05-08T23:45:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.578+0000 7f9756deb700 1 -- 172.21.15.12:0/1697968775 shutdown_connections 2024-05-08T23:45:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.578+0000 7f9756deb700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f973c0a1120 0x7f973c0a35e0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9750100c40 tx=0x7f9740009260 comp rx=0 tx=0).stop 2024-05-08T23:45:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.578+0000 7f9756deb700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9750102ab0 0x7f975019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.578+0000 7f9756deb700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9750100f20 0x7f9750196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.578+0000 7f9756deb700 1 --2- 172.21.15.12:0/1697968775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97501020f0 0x7f9750197440 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.578+0000 7f9756deb700 1 -- 172.21.15.12:0/1697968775 >> 172.21.15.12:0/1697968775 conn(0x7f97500fc4e0 msgr2=0x7f97501055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:03.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.586+0000 7f9756deb700 1 -- 172.21.15.12:0/1697968775 shutdown_connections 2024-05-08T23:45:03.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.586+0000 7f9756deb700 1 -- 172.21.15.12:0/1697968775 wait complete. 2024-05-08T23:45:03.584 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2024-05-08T23:45:03.600 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:45:03.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.748+0000 7f367b0bd700 1 -- 172.21.15.12:0/1223126678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36740ffe80 msgr2=0x7f36741002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:03.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.748+0000 7f367b0bd700 1 --2- 172.21.15.12:0/1223126678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36740ffe80 0x7f36741002a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3668009990 tx=0x7f3668009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:03.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.748+0000 7f367b0bd700 1 -- 172.21.15.12:0/1223126678 shutdown_connections 2024-05-08T23:45:03.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.748+0000 7f367b0bd700 1 --2- 172.21.15.12:0/1223126678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674101990 0x7f36741082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:03.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.748+0000 7f367b0bd700 1 --2- 172.21.15.12:0/1223126678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36740ffe80 0x7f36741002a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:03.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.748+0000 7f367b0bd700 1 --2- 172.21.15.12:0/1223126678 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3674100fd0 0x7f3674101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:03.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.748+0000 7f367b0bd700 1 -- 172.21.15.12:0/1223126678 >> 172.21.15.12:0/1223126678 conn(0x7f36740fb3f0 msgr2=0x7f36740fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:03.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.748+0000 7f367b0bd700 1 -- 172.21.15.12:0/1223126678 shutdown_connections 2024-05-08T23:45:03.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.748+0000 7f367b0bd700 1 -- 172.21.15.12:0/1223126678 wait complete. 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.749+0000 7f367b0bd700 1 Processor -- start 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.749+0000 7f367b0bd700 1 -- start start 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.749+0000 7f367b0bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36740ffe80 0x7f3674195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.749+0000 7f367b0bd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674100fd0 0x7f3674196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.749+0000 7f367b0bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3674101990 0x7f367419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.749+0000 7f367b0bd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367419d8f0 con 0x7f3674101990 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.749+0000 7f367b0bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367419da60 con 0x7f36740ffe80 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.749+0000 7f367b0bd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367419dc20 con 0x7f3674100fd0 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f367965a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3674101990 0x7f367419d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f367965a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3674101990 0x7f367419d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53278/0 (socket says 172.21.15.12:53278) 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f367965a700 1 -- 172.21.15.12:0/79577486 learned_addr learned my addr 172.21.15.12:0/79577486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:03.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f3678e59700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36740ffe80 0x7f3674195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f3673fff700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674100fd0 0x7f3674196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f367965a700 1 -- 172.21.15.12:0/79577486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674100fd0 msgr2=0x7f3674196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f367965a700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674100fd0 0x7f3674196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f367965a700 1 -- 172.21.15.12:0/79577486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36740ffe80 msgr2=0x7f3674195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f367965a700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36740ffe80 0x7f3674195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f367965a700 1 -- 172.21.15.12:0/79577486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3660009520 con 0x7f3674101990 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f367965a700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3674101990 0x7f367419d320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f366c00c370 tx=0x7f366c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f3671ffb700 1 -- 172.21.15.12:0/79577486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c017070 con 0x7f3674101990 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f3671ffb700 1 -- 172.21.15.12:0/79577486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f366c00f040 con 0x7f3674101990 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f367b0bd700 1 -- 172.21.15.12:0/79577486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366c009c40 con 0x7f3674101990 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.750+0000 7f3671ffb700 1 -- 172.21.15.12:0/79577486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c00e050 con 0x7f3674101990 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.751+0000 7f367b0bd700 1 -- 172.21.15.12:0/79577486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36680095f0 con 0x7f3674101990 2024-05-08T23:45:03.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.752+0000 7f367b0bd700 1 -- 172.21.15.12:0/79577486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3674107340 con 0x7f3674101990 2024-05-08T23:45:03.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.753+0000 7f3671ffb700 1 -- 172.21.15.12:0/79577486 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f366c00cb60 con 0x7f3674101990 2024-05-08T23:45:03.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.754+0000 7f3671ffb700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f36640a1120 0x7f36640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:03.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.754+0000 7f3678e59700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f36640a1120 0x7f36640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:03.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.754+0000 7f3671ffb700 1 -- 172.21.15.12:0/79577486 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f366c0cc7d0 con 0x7f3674101990 2024-05-08T23:45:03.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.754+0000 7f3678e59700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f36640a1120 0x7f36640a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f366800b2e0 tx=0x7f366800b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:03.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:03.758+0000 7f3671ffb700 1 -- 172.21.15.12:0/79577486 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f366c096070 con 0x7f3674101990 2024-05-08T23:45:03.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:03 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1187447364' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T23:45:03.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:03 smithi012 ceph-mon[42503]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T23:45:03.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:03 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1697968775' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:45:04.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:03 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1187447364' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T23:45:04.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:03 smithi204 ceph-mon[47373]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T23:45:04.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:03 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1697968775' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:45:04.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:03 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1187447364' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T23:45:04.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:03 smithi160 ceph-mon[46152]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T23:45:04.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:03 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1697968775' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:45:04.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.026+0000 7f367b0bd700 1 -- 172.21.15.12:0/79577486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3674067460 con 0x7f3674101990 2024-05-08T23:45:04.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.033+0000 7f3671ffb700 1 -- 172.21.15.12:0/79577486 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f366c099690 con 0x7f3674101990 2024-05-08T23:45:04.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 -- 172.21.15.12:0/79577486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f36640a1120 msgr2=0x7f36640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f36640a1120 0x7f36640a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f366800b2e0 tx=0x7f366800b6f0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 -- 172.21.15.12:0/79577486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3674101990 msgr2=0x7f367419d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3674101990 0x7f367419d320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f366c00c370 tx=0x7f366c00c680 comp rx=0 tx=0).stop 2024-05-08T23:45:04.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 -- 172.21.15.12:0/79577486 shutdown_connections 2024-05-08T23:45:04.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3674101990 0x7f367419d320 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f36640a1120 0x7f36640a35e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36740ffe80 0x7f3674195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 --2- 172.21.15.12:0/79577486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3674100fd0 0x7f3674196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 -- 172.21.15.12:0/79577486 >> 172.21.15.12:0/79577486 conn(0x7f36740fb3f0 msgr2=0x7f3674104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:04.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.039+0000 7f367b0bd700 1 -- 172.21.15.12:0/79577486 shutdown_connections 2024-05-08T23:45:04.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.040+0000 7f367b0bd700 1 -- 172.21.15.12:0/79577486 wait complete. 2024-05-08T23:45:04.042 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T23:45:04.063 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:45:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 -- 172.21.15.12:0/1545748122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30100f60 msgr2=0x7f5a301013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 --2- 172.21.15.12:0/1545748122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30100f60 0x7f5a301013c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0099c0 tx=0x7f5a1c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 -- 172.21.15.12:0/1545748122 shutdown_connections 2024-05-08T23:45:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 --2- 172.21.15.12:0/1545748122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30101900 0x7f5a301081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 --2- 172.21.15.12:0/1545748122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a300ffd70 0x7f5a30100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 --2- 172.21.15.12:0/1545748122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30100f60 0x7f5a301013c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 -- 172.21.15.12:0/1545748122 >> 172.21.15.12:0/1545748122 conn(0x7f5a300fb2e0 msgr2=0x7f5a300fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 -- 172.21.15.12:0/1545748122 shutdown_connections 2024-05-08T23:45:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 -- 172.21.15.12:0/1545748122 wait complete. 2024-05-08T23:45:04.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 Processor -- start 2024-05-08T23:45:04.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.205+0000 7f5a36b65700 1 -- start start 2024-05-08T23:45:04.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a36b65700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a300ffd70 0x7f5a30196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a36b65700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100f60 0x7f5a30197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a36b65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30101900 0x7f5a3019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a36b65700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3019e900 con 0x7f5a30101900 2024-05-08T23:45:04.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a36b65700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3019ea70 con 0x7f5a300ffd70 2024-05-08T23:45:04.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a36b65700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3019ec30 con 0x7f5a30100f60 2024-05-08T23:45:04.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a35102700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30101900 0x7f5a3019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a35102700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30101900 0x7f5a3019e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53302/0 (socket says 172.21.15.12:53302) 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a35102700 1 -- 172.21.15.12:0/2350822447 learned_addr learned my addr 172.21.15.12:0/2350822447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a2ffff700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100f60 0x7f5a30197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a34901700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a300ffd70 0x7f5a30196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a35102700 1 -- 172.21.15.12:0/2350822447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100f60 msgr2=0x7f5a30197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a35102700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100f60 0x7f5a30197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a35102700 1 -- 172.21.15.12:0/2350822447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a300ffd70 msgr2=0x7f5a30196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a35102700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a300ffd70 0x7f5a30196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a35102700 1 -- 172.21.15.12:0/2350822447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c0095f0 con 0x7f5a30101900 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.206+0000 7f5a35102700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30101900 0x7f5a3019e330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5a28007ee0 tx=0x7f5a2800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.207+0000 7f5a2dffb700 1 -- 172.21.15.12:0/2350822447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28017070 con 0x7f5a30101900 2024-05-08T23:45:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.207+0000 7f5a2dffb700 1 -- 172.21.15.12:0/2350822447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a2800f040 con 0x7f5a30101900 2024-05-08T23:45:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.207+0000 7f5a36b65700 1 -- 172.21.15.12:0/2350822447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3019ef10 con 0x7f5a30101900 2024-05-08T23:45:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.207+0000 7f5a2dffb700 1 -- 172.21.15.12:0/2350822447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28005740 con 0x7f5a30101900 2024-05-08T23:45:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.207+0000 7f5a36b65700 1 -- 172.21.15.12:0/2350822447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3019f2e0 con 0x7f5a30101900 2024-05-08T23:45:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.208+0000 7f5a36b65700 1 -- 172.21.15.12:0/2350822447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a30067380 con 0x7f5a30101900 2024-05-08T23:45:04.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.209+0000 7f5a2dffb700 1 -- 172.21.15.12:0/2350822447 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5a2800dc60 con 0x7f5a30101900 2024-05-08T23:45:04.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.209+0000 7f5a2dffb700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5a200a1120 0x7f5a200a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.210+0000 7f5a34901700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5a200a1120 0x7f5a200a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.210+0000 7f5a2dffb700 1 -- 172.21.15.12:0/2350822447 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5a280cc030 con 0x7f5a30101900 2024-05-08T23:45:04.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.210+0000 7f5a34901700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5a200a1120 0x7f5a200a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a24005950 tx=0x7f5a240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:04.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.215+0000 7f5a2dffb700 1 -- 172.21.15.12:0/2350822447 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5a280ffd60 con 0x7f5a30101900 2024-05-08T23:45:04.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.438+0000 7f5a36b65700 1 -- 172.21.15.12:0/2350822447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a3019f680 con 0x7f5a30101900 2024-05-08T23:45:04.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.440+0000 7f5a2dffb700 1 -- 172.21.15.12:0/2350822447 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22118 (secure 0 0 0) 0x7f5a28095870 con 0x7f5a30101900 2024-05-08T23:45:04.436 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:04.436 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:04.027446+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:45:04.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 -- 172.21.15.12:0/2350822447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5a200a1120 msgr2=0x7f5a200a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5a200a1120 0x7f5a200a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a24005950 tx=0x7f5a240058e0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 -- 172.21.15.12:0/2350822447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30101900 msgr2=0x7f5a3019e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30101900 0x7f5a3019e330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5a28007ee0 tx=0x7f5a2800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 -- 172.21.15.12:0/2350822447 shutdown_connections 2024-05-08T23:45:04.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30101900 0x7f5a3019e330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5a200a1120 0x7f5a200a35e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a300ffd70 0x7f5a30196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 --2- 172.21.15.12:0/2350822447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a30100f60 0x7f5a30197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 -- 172.21.15.12:0/2350822447 >> 172.21.15.12:0/2350822447 conn(0x7f5a300fb2e0 msgr2=0x7f5a30069300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:04.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 -- 172.21.15.12:0/2350822447 shutdown_connections 2024-05-08T23:45:04.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.442+0000 7f5a36b65700 1 -- 172.21.15.12:0/2350822447 wait complete. 2024-05-08T23:45:04.452 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:45:04.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 -- 172.21.15.12:0/767725539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a4101fc0 msgr2=0x7f05a4102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 --2- 172.21.15.12:0/767725539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a4101fc0 0x7f05a4102440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0594009990 tx=0x7f0594009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 -- 172.21.15.12:0/767725539 shutdown_connections 2024-05-08T23:45:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 --2- 172.21.15.12:0/767725539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a4102980 0x7f05a41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 --2- 172.21.15.12:0/767725539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a4100e70 0x7f05a4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 --2- 172.21.15.12:0/767725539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a4101fc0 0x7f05a4102440 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 -- 172.21.15.12:0/767725539 >> 172.21.15.12:0/767725539 conn(0x7f05a40fc400 msgr2=0x7f05a40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 -- 172.21.15.12:0/767725539 shutdown_connections 2024-05-08T23:45:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 -- 172.21.15.12:0/767725539 wait complete. 2024-05-08T23:45:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 Processor -- start 2024-05-08T23:45:04.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.595+0000 7f05aa9e6700 1 -- start start 2024-05-08T23:45:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05aa9e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a4100e70 0x7f05a4196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05aa9e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a4101fc0 0x7f05a4197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05aa9e6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a4102980 0x7f05a419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05aa9e6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a419e8f0 con 0x7f05a4102980 2024-05-08T23:45:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05aa9e6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a419ea60 con 0x7f05a4100e70 2024-05-08T23:45:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05aa9e6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a419ec20 con 0x7f05a4101fc0 2024-05-08T23:45:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a8f83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a4102980 0x7f05a419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a8f83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a4102980 0x7f05a419e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53310/0 (socket says 172.21.15.12:53310) 2024-05-08T23:45:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a8f83700 1 -- 172.21.15.12:0/3596983652 learned_addr learned my addr 172.21.15.12:0/3596983652 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a37fe700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a4101fc0 0x7f05a4197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a3fff700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a4100e70 0x7f05a4196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a8f83700 1 -- 172.21.15.12:0/3596983652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a4101fc0 msgr2=0x7f05a4197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a8f83700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a4101fc0 0x7f05a4197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a8f83700 1 -- 172.21.15.12:0/3596983652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a4100e70 msgr2=0x7f05a4196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a8f83700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a4100e70 0x7f05a4196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a8f83700 1 -- 172.21.15.12:0/3596983652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05940095f0 con 0x7f05a4102980 2024-05-08T23:45:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.596+0000 7f05a8f83700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a4102980 0x7f05a419e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f059c007ee0 tx=0x7f059c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.597+0000 7f05a17fa700 1 -- 172.21.15.12:0/3596983652 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c017070 con 0x7f05a4102980 2024-05-08T23:45:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.597+0000 7f05a17fa700 1 -- 172.21.15.12:0/3596983652 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f059c00f040 con 0x7f05a4102980 2024-05-08T23:45:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.597+0000 7f05a17fa700 1 -- 172.21.15.12:0/3596983652 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c005580 con 0x7f05a4102980 2024-05-08T23:45:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.597+0000 7f05aa9e6700 1 -- 172.21.15.12:0/3596983652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a419ef00 con 0x7f05a4102980 2024-05-08T23:45:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.597+0000 7f05aa9e6700 1 -- 172.21.15.12:0/3596983652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a419f2d0 con 0x7f05a4102980 2024-05-08T23:45:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.598+0000 7f05aa9e6700 1 -- 172.21.15.12:0/3596983652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a4108260 con 0x7f05a4102980 2024-05-08T23:45:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.599+0000 7f05a17fa700 1 -- 172.21.15.12:0/3596983652 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f059c00db20 con 0x7f05a4102980 2024-05-08T23:45:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.599+0000 7f05a17fa700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f05900a11a0 0x7f05900a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.600+0000 7f05a17fa700 1 -- 172.21.15.12:0/3596983652 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f059c0cc470 con 0x7f05a4102980 2024-05-08T23:45:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.600+0000 7f05a3fff700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f05900a11a0 0x7f05900a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.600+0000 7f05a3fff700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f05900a11a0 0x7f05900a3660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f05a4198280 tx=0x7f058c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.604+0000 7f05a17fa700 1 -- 172.21.15.12:0/3596983652 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f059c095d50 con 0x7f05a4102980 2024-05-08T23:45:04.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.823+0000 7f05aa9e6700 1 -- 172.21.15.12:0/3596983652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05a410c740 con 0x7f05a4102980 2024-05-08T23:45:04.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.824+0000 7f05a17fa700 1 -- 172.21.15.12:0/3596983652 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22118 (secure 0 0 0) 0x7f059c099370 con 0x7f05a4102980 2024-05-08T23:45:04.820 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:04.821 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:04.027446+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:45:04.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 -- 172.21.15.12:0/3596983652 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f05900a11a0 msgr2=0x7f05900a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f05900a11a0 0x7f05900a3660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f05a4198280 tx=0x7f058c00b410 comp rx=0 tx=0).stop 2024-05-08T23:45:04.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 -- 172.21.15.12:0/3596983652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a4102980 msgr2=0x7f05a419e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a4102980 0x7f05a419e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f059c007ee0 tx=0x7f059c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 -- 172.21.15.12:0/3596983652 shutdown_connections 2024-05-08T23:45:04.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a4102980 0x7f05a419e320 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f05900a11a0 0x7f05900a3660 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05a4100e70 0x7f05a4196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 --2- 172.21.15.12:0/3596983652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a4101fc0 0x7f05a4197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 -- 172.21.15.12:0/3596983652 >> 172.21.15.12:0/3596983652 conn(0x7f05a40fc400 msgr2=0x7f05a41053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 -- 172.21.15.12:0/3596983652 shutdown_connections 2024-05-08T23:45:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.827+0000 7f05aa9e6700 1 -- 172.21.15.12:0/3596983652 wait complete. 2024-05-08T23:45:04.835 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: pgmap v222: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:45:04.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T23:45:04.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1697968775' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T23:45:04.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T23:45:04.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/79577486' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:45:04.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T23:45:04.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T23:45:04.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/79577486' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T23:45:04.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T23:45:04.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T23:45:04.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:04 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2350822447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:04.838 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:45:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.981+0000 7f6d880a5700 1 -- 172.21.15.12:0/1197481414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d80106c60 msgr2=0x7f6d801090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.981+0000 7f6d880a5700 1 --2- 172.21.15.12:0/1197481414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d80106c60 0x7f6d801090d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800b0a0 tx=0x7f6d7800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.981+0000 7f6d880a5700 1 -- 172.21.15.12:0/1197481414 shutdown_connections 2024-05-08T23:45:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 --2- 172.21.15.12:0/1197481414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d80106c60 0x7f6d801090d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 --2- 172.21.15.12:0/1197481414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d80101a00 0x7f6d80103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 --2- 172.21.15.12:0/1197481414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80104330 0x7f6d80106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 -- 172.21.15.12:0/1197481414 >> 172.21.15.12:0/1197481414 conn(0x7f6d800fb3f0 msgr2=0x7f6d800fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 -- 172.21.15.12:0/1197481414 shutdown_connections 2024-05-08T23:45:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 -- 172.21.15.12:0/1197481414 wait complete. 2024-05-08T23:45:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 Processor -- start 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 -- start start 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d80101a00 0x7f6d80196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80104330 0x7f6d80197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d80106c60 0x7f6d8019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8019eaf0 con 0x7f6d80106c60 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8019ec30 con 0x7f6d80101a00 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.982+0000 7f6d880a5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8019edc0 con 0x7f6d80104330 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d86642700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d80106c60 0x7f6d8019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d86642700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d80106c60 0x7f6d8019e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53330/0 (socket says 172.21.15.12:53330) 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d86642700 1 -- 172.21.15.12:0/4153298163 learned_addr learned my addr 172.21.15.12:0/4153298163 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d85640700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80104330 0x7f6d80197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d85e41700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d80101a00 0x7f6d80196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d86642700 1 -- 172.21.15.12:0/4153298163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80104330 msgr2=0x7f6d80197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d86642700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80104330 0x7f6d80197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d86642700 1 -- 172.21.15.12:0/4153298163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d80101a00 msgr2=0x7f6d80196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d86642700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d80101a00 0x7f6d80196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d86642700 1 -- 172.21.15.12:0/4153298163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78009d00 con 0x7f6d80106c60 2024-05-08T23:45:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d85e41700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d80101a00 0x7f6d80196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d86642700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d80106c60 0x7f6d8019e520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6d78007ab0 tx=0x7f6d78004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d76ffd700 1 -- 172.21.15.12:0/4153298163 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800e040 con 0x7f6d80106c60 2024-05-08T23:45:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d76ffd700 1 -- 172.21.15.12:0/4153298163 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d7801cd90 con 0x7f6d80106c60 2024-05-08T23:45:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d880a5700 1 -- 172.21.15.12:0/4153298163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d800ff7f0 con 0x7f6d80106c60 2024-05-08T23:45:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d76ffd700 1 -- 172.21.15.12:0/4153298163 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d780059d0 con 0x7f6d80106c60 2024-05-08T23:45:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.983+0000 7f6d880a5700 1 -- 172.21.15.12:0/4153298163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d800ffd00 con 0x7f6d80106c60 2024-05-08T23:45:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.984+0000 7f6d880a5700 1 -- 172.21.15.12:0/4153298163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d800fcf70 con 0x7f6d80106c60 2024-05-08T23:45:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.986+0000 7f6d76ffd700 1 -- 172.21.15.12:0/4153298163 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6d780050d0 con 0x7f6d80106c60 2024-05-08T23:45:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.987+0000 7f6d76ffd700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6d6c0a1120 0x7f6d6c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.987+0000 7f6d85e41700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6d6c0a1120 0x7f6d6c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.987+0000 7f6d76ffd700 1 -- 172.21.15.12:0/4153298163 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d780d2310 con 0x7f6d80106c60 2024-05-08T23:45:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.987+0000 7f6d85e41700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6d6c0a1120 0x7f6d6c0a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:04.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:04.991+0000 7f6d76ffd700 1 -- 172.21.15.12:0/4153298163 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6d7809bc80 con 0x7f6d80106c60 2024-05-08T23:45:05.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: pgmap v222: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:45:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T23:45:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1697968775' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T23:45:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T23:45:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/79577486' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:45:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T23:45:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T23:45:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/79577486' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T23:45:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T23:45:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T23:45:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:04 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2350822447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:05.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: pgmap v222: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:45:05.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T23:45:05.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1697968775' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T23:45:05.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T23:45:05.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/79577486' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:45:05.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T23:45:05.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T23:45:05.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/79577486' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T23:45:05.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T23:45:05.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T23:45:05.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:04 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2350822447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:05.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:05.264+0000 7f6d880a5700 1 -- 172.21.15.12:0/4153298163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6d80067460 con 0x7f6d80106c60 2024-05-08T23:45:05.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:05 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3596983652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:05 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] up:active 2024-05-08T23:45:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:05 smithi012 ceph-mon[42503]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:05 smithi012 ceph-mon[42503]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T23:45:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:05 smithi012 ceph-mon[42503]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:05 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/4153298163' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:45:06.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:05 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3596983652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:06.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:05 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] up:active 2024-05-08T23:45:06.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:05 smithi204 ceph-mon[47373]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:06.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:05 smithi204 ceph-mon[47373]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T23:45:06.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:05 smithi204 ceph-mon[47373]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:06.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:05 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/4153298163' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:45:06.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:05 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3596983652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:06.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:05 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] up:active 2024-05-08T23:45:06.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:05 smithi160 ceph-mon[46152]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:06.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:05 smithi160 ceph-mon[46152]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T23:45:06.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:05 smithi160 ceph-mon[46152]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:06.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:05 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/4153298163' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:45:06.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.045+0000 7f6d76ffd700 1 -- 172.21.15.12:0/4153298163 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6d7809f2a0 con 0x7f6d80106c60 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 -- 172.21.15.12:0/4153298163 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6d6c0a1120 msgr2=0x7f6d6c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6d6c0a1120 0x7f6d6c0a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c005ee0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 -- 172.21.15.12:0/4153298163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d80106c60 msgr2=0x7f6d8019e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d80106c60 0x7f6d8019e520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6d78007ab0 tx=0x7f6d78004c60 comp rx=0 tx=0).stop 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 -- 172.21.15.12:0/4153298163 shutdown_connections 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d80106c60 0x7f6d8019e520 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6d6c0a1120 0x7f6d6c0a35e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d80101a00 0x7f6d80196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 --2- 172.21.15.12:0/4153298163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d80104330 0x7f6d80197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 -- 172.21.15.12:0/4153298163 >> 172.21.15.12:0/4153298163 conn(0x7f6d800fb3f0 msgr2=0x7f6d80102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 -- 172.21.15.12:0/4153298163 shutdown_connections 2024-05-08T23:45:06.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.048+0000 7f6d880a5700 1 -- 172.21.15.12:0/4153298163 wait complete. 2024-05-08T23:45:06.060 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-08T23:45:06.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-08T23:45:06.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.208+0000 7f19ef7d0700 1 -- 172.21.15.12:0/1755217664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e8106cb0 msgr2=0x7f19e81090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.208+0000 7f19ef7d0700 1 --2- 172.21.15.12:0/1755217664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e8106cb0 0x7f19e81090c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f19e000b0a0 tx=0x7f19e000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.208+0000 7f19ef7d0700 1 -- 172.21.15.12:0/1755217664 shutdown_connections 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.208+0000 7f19ef7d0700 1 --2- 172.21.15.12:0/1755217664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e8106cb0 0x7f19e81090c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.208+0000 7f19ef7d0700 1 --2- 172.21.15.12:0/1755217664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19e8101a50 0x7f19e8103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.208+0000 7f19ef7d0700 1 --2- 172.21.15.12:0/1755217664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19e8104380 0x7f19e8106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.208+0000 7f19ef7d0700 1 -- 172.21.15.12:0/1755217664 >> 172.21.15.12:0/1755217664 conn(0x7f19e80fb400 msgr2=0x7f19e80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.208+0000 7f19ef7d0700 1 -- 172.21.15.12:0/1755217664 shutdown_connections 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.208+0000 7f19ef7d0700 1 -- 172.21.15.12:0/1755217664 wait complete. 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ef7d0700 1 Processor -- start 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ef7d0700 1 -- start start 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ef7d0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19e8101a50 0x7f19e81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ef7d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e8104380 0x7f19e8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:06.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ef7d0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19e8106cb0 0x7f19e81a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ef7d0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81a0d10 con 0x7f19e8104380 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ef7d0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81a0e50 con 0x7f19e8106cb0 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ef7d0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81a0fe0 con 0x7f19e8101a50 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ecd6b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e8104380 0x7f19e8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ecd6b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e8104380 0x7f19e8199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53352/0 (socket says 172.21.15.12:53352) 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ecd6b700 1 -- 172.21.15.12:0/2185972855 learned_addr learned my addr 172.21.15.12:0/2185972855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ed56c700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19e8101a50 0x7f19e81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19edd6d700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19e8106cb0 0x7f19e81a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ecd6b700 1 -- 172.21.15.12:0/2185972855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19e8101a50 msgr2=0x7f19e81991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ecd6b700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19e8101a50 0x7f19e81991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ecd6b700 1 -- 172.21.15.12:0/2185972855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19e8106cb0 msgr2=0x7f19e81a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:06.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ecd6b700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19e8106cb0 0x7f19e81a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:06.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19ecd6b700 1 -- 172.21.15.12:0/2185972855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0009d00 con 0x7f19e8104380 2024-05-08T23:45:06.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.209+0000 7f19edd6d700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19e8106cb0 0x7f19e81a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:06.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.210+0000 7f19ecd6b700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e8104380 0x7f19e8199720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f19d800d950 tx=0x7f19d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:06.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.210+0000 7f19de7fc700 1 -- 172.21.15.12:0/2185972855 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8009940 con 0x7f19e8104380 2024-05-08T23:45:06.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.210+0000 7f19ef7d0700 1 -- 172.21.15.12:0/2185972855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e81a12c0 con 0x7f19e8104380 2024-05-08T23:45:06.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.210+0000 7f19de7fc700 1 -- 172.21.15.12:0/2185972855 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19d8010460 con 0x7f19e8104380 2024-05-08T23:45:06.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.210+0000 7f19de7fc700 1 -- 172.21.15.12:0/2185972855 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d800f620 con 0x7f19e8104380 2024-05-08T23:45:06.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.210+0000 7f19ef7d0700 1 -- 172.21.15.12:0/2185972855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e81a1820 con 0x7f19e8104380 2024-05-08T23:45:06.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.211+0000 7f19ef7d0700 1 -- 172.21.15.12:0/2185972855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e8108830 con 0x7f19e8104380 2024-05-08T23:45:06.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.212+0000 7f19de7fc700 1 -- 172.21.15.12:0/2185972855 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f19d80108d0 con 0x7f19e8104380 2024-05-08T23:45:06.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.213+0000 7f19de7fc700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f19d40a1120 0x7f19d40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:06.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.213+0000 7f19ed56c700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f19d40a1120 0x7f19d40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:06.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.213+0000 7f19de7fc700 1 -- 172.21.15.12:0/2185972855 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f19d80cc6d0 con 0x7f19e8104380 2024-05-08T23:45:06.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.213+0000 7f19ed56c700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f19d40a1120 0x7f19d40a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009c60 tx=0x7f19e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:06.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.218+0000 7f19de7fc700 1 -- 172.21.15.12:0/2185972855 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f19d8095f10 con 0x7f19e8104380 2024-05-08T23:45:06.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:06.488+0000 7f19ef7d0700 1 -- 172.21.15.12:0/2185972855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f19e8068140 con 0x7f19e8104380 2024-05-08T23:45:06.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:06 smithi012 ceph-mon[42503]: pgmap v226: 97 pgs: 8 creating+peering, 30 active+clean, 59 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T23:45:06.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:06 smithi012 ceph-mon[42503]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T23:45:06.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:06 smithi012 ceph-mon[42503]: Cluster is now healthy 2024-05-08T23:45:06.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:06 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/4153298163' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T23:45:06.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:06 smithi012 ceph-mon[42503]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:06.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:06 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2185972855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T23:45:07.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:06 smithi204 ceph-mon[47373]: pgmap v226: 97 pgs: 8 creating+peering, 30 active+clean, 59 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T23:45:07.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:06 smithi204 ceph-mon[47373]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T23:45:07.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:06 smithi204 ceph-mon[47373]: Cluster is now healthy 2024-05-08T23:45:07.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:06 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/4153298163' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T23:45:07.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:06 smithi204 ceph-mon[47373]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:07.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:06 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2185972855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T23:45:07.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:06 smithi160 ceph-mon[46152]: pgmap v226: 97 pgs: 8 creating+peering, 30 active+clean, 59 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T23:45:07.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:06 smithi160 ceph-mon[46152]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T23:45:07.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:06 smithi160 ceph-mon[46152]: Cluster is now healthy 2024-05-08T23:45:07.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:06 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/4153298163' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T23:45:07.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:06 smithi160 ceph-mon[46152]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:07.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:06 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2185972855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T23:45:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.047+0000 7f19de7fc700 1 -- 172.21.15.12:0/2185972855 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f19e8068140 con 0x7f19e8104380 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 -- 172.21.15.12:0/2185972855 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f19d40a1120 msgr2=0x7f19d40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f19d40a1120 0x7f19d40a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009c60 tx=0x7f19e4009260 comp rx=0 tx=0).stop 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 -- 172.21.15.12:0/2185972855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e8104380 msgr2=0x7f19e8199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e8104380 0x7f19e8199720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f19d800d950 tx=0x7f19d800dc60 comp rx=0 tx=0).stop 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 -- 172.21.15.12:0/2185972855 shutdown_connections 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f19d40a1120 0x7f19d40a35e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19e8106cb0 0x7f19e81a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19e8101a50 0x7f19e81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 --2- 172.21.15.12:0/2185972855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e8104380 0x7f19e8199720 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 -- 172.21.15.12:0/2185972855 >> 172.21.15.12:0/2185972855 conn(0x7f19e80fb400 msgr2=0x7f19e8101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 -- 172.21.15.12:0/2185972855 shutdown_connections 2024-05-08T23:45:07.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.051+0000 7f19ef7d0700 1 -- 172.21.15.12:0/2185972855 wait complete. 2024-05-08T23:45:07.065 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-08T23:45:07.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.214+0000 7fd2e07de700 1 -- 172.21.15.12:0/1395846528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2d81020f0 msgr2=0x7fd2d8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:07.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.214+0000 7fd2e07de700 1 --2- 172.21.15.12:0/1395846528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2d81020f0 0x7fd2d8102570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009990 tx=0x7fd2c8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:07.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 -- 172.21.15.12:0/1395846528 shutdown_connections 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 --2- 172.21.15.12:0/1395846528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8102ab0 0x7fd2d81093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 --2- 172.21.15.12:0/1395846528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2d8100f20 0x7fd2d8101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 --2- 172.21.15.12:0/1395846528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2d81020f0 0x7fd2d8102570 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 -- 172.21.15.12:0/1395846528 >> 172.21.15.12:0/1395846528 conn(0x7fd2d80fc4e0 msgr2=0x7fd2d80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 -- 172.21.15.12:0/1395846528 shutdown_connections 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 -- 172.21.15.12:0/1395846528 wait complete. 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 Processor -- start 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 -- start start 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8100f20 0x7fd2d8196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2d81020f0 0x7fd2d8197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.215+0000 7fd2e07de700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2d8102ab0 0x7fd2d819e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2e07de700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d819e980 con 0x7fd2d81020f0 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2e07de700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d819eac0 con 0x7fd2d8102ab0 2024-05-08T23:45:07.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2e07de700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d819ec50 con 0x7fd2d8100f20 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ddd79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2d81020f0 0x7fd2d8197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ddd79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2d81020f0 0x7fd2d8197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53376/0 (socket says 172.21.15.12:53376) 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ddd79700 1 -- 172.21.15.12:0/456554867 learned_addr learned my addr 172.21.15.12:0/456554867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2de57a700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8100f20 0x7fd2d8196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ddd79700 1 -- 172.21.15.12:0/456554867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8100f20 msgr2=0x7fd2d8196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ded7b700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2d8102ab0 0x7fd2d819e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ddd79700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8100f20 0x7fd2d8196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ddd79700 1 -- 172.21.15.12:0/456554867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2d8102ab0 msgr2=0x7fd2d819e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ddd79700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2d8102ab0 0x7fd2d819e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ddd79700 1 -- 172.21.15.12:0/456554867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80095f0 con 0x7fd2d81020f0 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2de57a700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8100f20 0x7fd2d8196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ded7b700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2d8102ab0 0x7fd2d819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2ddd79700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2d81020f0 0x7fd2d8197390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80059d0 tx=0x7fd2c8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:07.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2cf7fe700 1 -- 172.21.15.12:0/456554867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c801d070 con 0x7fd2d81020f0 2024-05-08T23:45:07.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2cf7fe700 1 -- 172.21.15.12:0/456554867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c8004120 con 0x7fd2d81020f0 2024-05-08T23:45:07.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2e07de700 1 -- 172.21.15.12:0/456554867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d819eed0 con 0x7fd2d81020f0 2024-05-08T23:45:07.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2cf7fe700 1 -- 172.21.15.12:0/456554867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8022d40 con 0x7fd2d81020f0 2024-05-08T23:45:07.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.216+0000 7fd2e07de700 1 -- 172.21.15.12:0/456554867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d819f2f0 con 0x7fd2d81020f0 2024-05-08T23:45:07.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.218+0000 7fd2e07de700 1 -- 172.21.15.12:0/456554867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d81083c0 con 0x7fd2d81020f0 2024-05-08T23:45:07.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.219+0000 7fd2cf7fe700 1 -- 172.21.15.12:0/456554867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd2c8022470 con 0x7fd2d81020f0 2024-05-08T23:45:07.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.219+0000 7fd2cf7fe700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2c40a11a0 0x7fd2c40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:07.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.219+0000 7fd2cf7fe700 1 -- 172.21.15.12:0/456554867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd2c80ce400 con 0x7fd2d81020f0 2024-05-08T23:45:07.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.219+0000 7fd2de57a700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2c40a11a0 0x7fd2c40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:07.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.220+0000 7fd2de57a700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2c40a11a0 0x7fd2c40a3660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8100c40 tx=0x7fd2d4009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:07.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.223+0000 7fd2cf7fe700 1 -- 172.21.15.12:0/456554867 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd2c8097d70 con 0x7fd2d81020f0 2024-05-08T23:45:07.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:07.492+0000 7fd2e07de700 1 -- 172.21.15.12:0/456554867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fd2d810c790 con 0x7fd2d81020f0 2024-05-08T23:45:07.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:07 smithi012 ceph-mon[42503]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T23:45:07.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:07 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2185972855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T23:45:07.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:07 smithi012 ceph-mon[42503]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T23:45:07.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:07 smithi012 ceph-mon[42503]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:07.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:07 smithi012 ceph-mon[42503]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-08T23:45:07.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:07 smithi012 ceph-mon[42503]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-08T23:45:07.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:07 smithi012 ceph-mon[42503]: pgmap v227: 97 pgs: 8 creating+peering, 83 active+clean, 6 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-08T23:45:07.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:07 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/456554867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:45:08.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:07 smithi204 ceph-mon[47373]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T23:45:08.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:07 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2185972855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T23:45:08.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:07 smithi204 ceph-mon[47373]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T23:45:08.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:07 smithi204 ceph-mon[47373]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:08.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:07 smithi204 ceph-mon[47373]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-08T23:45:08.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:07 smithi204 ceph-mon[47373]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-08T23:45:08.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:07 smithi204 ceph-mon[47373]: pgmap v227: 97 pgs: 8 creating+peering, 83 active+clean, 6 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-08T23:45:08.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:07 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/456554867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:45:08.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:07 smithi160 ceph-mon[46152]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T23:45:08.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:07 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2185972855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T23:45:08.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:07 smithi160 ceph-mon[46152]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T23:45:08.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:07 smithi160 ceph-mon[46152]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:45:08.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:07 smithi160 ceph-mon[46152]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-08T23:45:08.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:07 smithi160 ceph-mon[46152]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-08T23:45:08.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:07 smithi160 ceph-mon[46152]: pgmap v227: 97 pgs: 8 creating+peering, 83 active+clean, 6 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-08T23:45:08.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:07 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/456554867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:45:08.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.058+0000 7fd2cf7fe700 1 -- 172.21.15.12:0/456554867 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fd2c809b390 con 0x7fd2d81020f0 2024-05-08T23:45:08.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 -- 172.21.15.12:0/456554867 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2c40a11a0 msgr2=0x7fd2c40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:08.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2c40a11a0 0x7fd2c40a3660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8100c40 tx=0x7fd2d4009670 comp rx=0 tx=0).stop 2024-05-08T23:45:08.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 -- 172.21.15.12:0/456554867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2d81020f0 msgr2=0x7fd2d8197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:08.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2d81020f0 0x7fd2d8197390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80059d0 tx=0x7fd2c8005ef0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 -- 172.21.15.12:0/456554867 shutdown_connections 2024-05-08T23:45:08.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd2c40a11a0 0x7fd2c40a3660 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2d8102ab0 0x7fd2d819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8100f20 0x7fd2d8196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 --2- 172.21.15.12:0/456554867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2d81020f0 0x7fd2d8197390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 -- 172.21.15.12:0/456554867 >> 172.21.15.12:0/456554867 conn(0x7fd2d80fc4e0 msgr2=0x7fd2d8105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:08.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 -- 172.21.15.12:0/456554867 shutdown_connections 2024-05-08T23:45:08.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.061+0000 7fd2e07de700 1 -- 172.21.15.12:0/456554867 wait complete. 2024-05-08T23:45:08.077 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-08T23:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.225+0000 7f0630d0a700 1 -- 172.21.15.12:0/2075667084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f062c100fb0 msgr2=0x7f062c101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.225+0000 7f0630d0a700 1 --2- 172.21.15.12:0/2075667084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f062c100fb0 0x7f062c101430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0614009990 tx=0x7f0614009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.225+0000 7f0630d0a700 1 -- 172.21.15.12:0/2075667084 shutdown_connections 2024-05-08T23:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.225+0000 7f0630d0a700 1 --2- 172.21.15.12:0/2075667084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f062c101970 0x7f062c108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.225+0000 7f0630d0a700 1 --2- 172.21.15.12:0/2075667084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062c0ffe60 0x7f062c100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.225+0000 7f0630d0a700 1 --2- 172.21.15.12:0/2075667084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f062c100fb0 0x7f062c101430 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.225+0000 7f0630d0a700 1 -- 172.21.15.12:0/2075667084 >> 172.21.15.12:0/2075667084 conn(0x7f062c0fb3f0 msgr2=0x7f062c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.225+0000 7f0630d0a700 1 -- 172.21.15.12:0/2075667084 shutdown_connections 2024-05-08T23:45:08.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.225+0000 7f0630d0a700 1 -- 172.21.15.12:0/2075667084 wait complete. 2024-05-08T23:45:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.226+0000 7f0630d0a700 1 Processor -- start 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.226+0000 7f0630d0a700 1 -- start start 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.226+0000 7f0630d0a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062c0ffe60 0x7f062c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0630d0a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f062c100fb0 0x7f062c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0630d0a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f062c101970 0x7f062c19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0630d0a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c19ea50 con 0x7f062c100fb0 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0630d0a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c19eb90 con 0x7f062c101970 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0630d0a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c19ed20 con 0x7f062c0ffe60 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0629d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f062c100fb0 0x7f062c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0629d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f062c100fb0 0x7f062c197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53396/0 (socket says 172.21.15.12:53396) 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0629d9b700 1 -- 172.21.15.12:0/1044123405 learned_addr learned my addr 172.21.15.12:0/1044123405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f062a59c700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062c0ffe60 0x7f062c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f062ad9d700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f062c101970 0x7f062c19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0629d9b700 1 -- 172.21.15.12:0/1044123405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062c0ffe60 msgr2=0x7f062c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0629d9b700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062c0ffe60 0x7f062c196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0629d9b700 1 -- 172.21.15.12:0/1044123405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f062c101970 msgr2=0x7f062c19e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0629d9b700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f062c101970 0x7f062c19e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0629d9b700 1 -- 172.21.15.12:0/1044123405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06140095f0 con 0x7f062c100fb0 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0629d9b700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f062c100fb0 0x7f062c197460 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f061400b620 tx=0x7f0614004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f06237fe700 1 -- 172.21.15.12:0/1044123405 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061401d070 con 0x7f062c100fb0 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f06237fe700 1 -- 172.21.15.12:0/1044123405 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0614022470 con 0x7f062c100fb0 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0630d0a700 1 -- 172.21.15.12:0/1044123405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c19efa0 con 0x7f062c100fb0 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f06237fe700 1 -- 172.21.15.12:0/1044123405 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614017650 con 0x7f062c100fb0 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.227+0000 7f0630d0a700 1 -- 172.21.15.12:0/1044123405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c19f360 con 0x7f062c100fb0 2024-05-08T23:45:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.228+0000 7f0630d0a700 1 -- 172.21.15.12:0/1044123405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062c067cf0 con 0x7f062c100fb0 2024-05-08T23:45:08.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.230+0000 7f06237fe700 1 -- 172.21.15.12:0/1044123405 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f06140225e0 con 0x7f062c100fb0 2024-05-08T23:45:08.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.230+0000 7f06237fe700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06180a1120 0x7f06180a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:08.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.230+0000 7f062a59c700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06180a1120 0x7f06180a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:08.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.230+0000 7f06237fe700 1 -- 172.21.15.12:0/1044123405 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f06140cda10 con 0x7f062c100fb0 2024-05-08T23:45:08.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.231+0000 7f062a59c700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06180a1120 0x7f06180a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f061c009c60 tx=0x7f061c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.235+0000 7f06237fe700 1 -- 172.21.15.12:0/1044123405 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0614097380 con 0x7f062c100fb0 2024-05-08T23:45:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:08.504+0000 7f0630d0a700 1 -- 172.21.15.12:0/1044123405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f062c10b6b0 con 0x7f062c100fb0 2024-05-08T23:45:09.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.066+0000 7f06237fe700 1 -- 172.21.15.12:0/1044123405 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f061409a9a0 con 0x7f062c100fb0 2024-05-08T23:45:09.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.069+0000 7f0630d0a700 1 -- 172.21.15.12:0/1044123405 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06180a1120 msgr2=0x7f06180a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.069+0000 7f0630d0a700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06180a1120 0x7f06180a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f061c009c60 tx=0x7f061c009260 comp rx=0 tx=0).stop 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.069+0000 7f0630d0a700 1 -- 172.21.15.12:0/1044123405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f062c100fb0 msgr2=0x7f062c197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.069+0000 7f0630d0a700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f062c100fb0 0x7f062c197460 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f061400b620 tx=0x7f0614004730 comp rx=0 tx=0).stop 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.069+0000 7f0630d0a700 1 -- 172.21.15.12:0/1044123405 shutdown_connections 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.070+0000 7f0630d0a700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06180a1120 0x7f06180a35e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.070+0000 7f0630d0a700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f062c101970 0x7f062c19e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.070+0000 7f0630d0a700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062c0ffe60 0x7f062c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.070+0000 7f0630d0a700 1 --2- 172.21.15.12:0/1044123405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f062c100fb0 0x7f062c197460 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.070+0000 7f0630d0a700 1 -- 172.21.15.12:0/1044123405 >> 172.21.15.12:0/1044123405 conn(0x7f062c0fb3f0 msgr2=0x7f062c0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.070+0000 7f0630d0a700 1 -- 172.21.15.12:0/1044123405 shutdown_connections 2024-05-08T23:45:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.070+0000 7f0630d0a700 1 -- 172.21.15.12:0/1044123405 wait complete. 2024-05-08T23:45:09.082 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-08T23:45:09.082 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-08T23:45:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.232+0000 7f210b8ae700 1 -- 172.21.15.12:0/295887609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2104100f20 msgr2=0x7f2104101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.232+0000 7f210b8ae700 1 --2- 172.21.15.12:0/295887609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2104100f20 0x7f2104101360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009990 tx=0x7f20f4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.232+0000 7f210b8ae700 1 -- 172.21.15.12:0/295887609 shutdown_connections 2024-05-08T23:45:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.232+0000 7f210b8ae700 1 --2- 172.21.15.12:0/295887609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2104102ab0 0x7f21041093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.232+0000 7f210b8ae700 1 --2- 172.21.15.12:0/295887609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2104100f20 0x7f2104101360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.232+0000 7f210b8ae700 1 --2- 172.21.15.12:0/295887609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21041020f0 0x7f2104102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.232+0000 7f210b8ae700 1 -- 172.21.15.12:0/295887609 >> 172.21.15.12:0/295887609 conn(0x7f21040fc4e0 msgr2=0x7f21040fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.232+0000 7f210b8ae700 1 -- 172.21.15.12:0/295887609 shutdown_connections 2024-05-08T23:45:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.232+0000 7f210b8ae700 1 -- 172.21.15.12:0/295887609 wait complete. 2024-05-08T23:45:09.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210b8ae700 1 Processor -- start 2024-05-08T23:45:09.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210b8ae700 1 -- start start 2024-05-08T23:45:09.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210b8ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2104100f20 0x7f2104074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:09.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210b8ae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21041020f0 0x7f2104073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:09.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210b8ae700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2104102ab0 0x7f21040738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210b8ae700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2104074070 con 0x7f2104100f20 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210b8ae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21040741e0 con 0x7f21041020f0 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210b8ae700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21041a0f40 con 0x7f2104102ab0 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210964a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2104100f20 0x7f2104074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210964a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2104100f20 0x7f2104074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37522/0 (socket says 172.21.15.12:37522) 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210964a700 1 -- 172.21.15.12:0/3301929559 learned_addr learned my addr 172.21.15.12:0/3301929559 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f2108e49700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21041020f0 0x7f2104073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210964a700 1 -- 172.21.15.12:0/3301929559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2104102ab0 msgr2=0x7f21040738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f2109e4b700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2104102ab0 0x7f21040738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210964a700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2104102ab0 0x7f21040738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210964a700 1 -- 172.21.15.12:0/3301929559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21041020f0 msgr2=0x7f2104073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210964a700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21041020f0 0x7f2104073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f210964a700 1 -- 172.21.15.12:0/3301929559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f40095f0 con 0x7f2104100f20 2024-05-08T23:45:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.233+0000 7f2108e49700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21041020f0 0x7f2104073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.234+0000 7f210964a700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2104100f20 0x7f2104074d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f20f4000c00 tx=0x7f20f4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.234+0000 7f20fa7fc700 1 -- 172.21.15.12:0/3301929559 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f401d070 con 0x7f2104100f20 2024-05-08T23:45:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.234+0000 7f210b8ae700 1 -- 172.21.15.12:0/3301929559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21041a1170 con 0x7f2104100f20 2024-05-08T23:45:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.234+0000 7f20fa7fc700 1 -- 172.21.15.12:0/3301929559 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f4004ca0 con 0x7f2104100f20 2024-05-08T23:45:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.234+0000 7f20fa7fc700 1 -- 172.21.15.12:0/3301929559 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4005590 con 0x7f2104100f20 2024-05-08T23:45:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.234+0000 7f210b8ae700 1 -- 172.21.15.12:0/3301929559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21041a14e0 con 0x7f2104100f20 2024-05-08T23:45:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.235+0000 7f210b8ae700 1 -- 172.21.15.12:0/3301929559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2104108460 con 0x7f2104100f20 2024-05-08T23:45:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.236+0000 7f20fa7fc700 1 -- 172.21.15.12:0/3301929559 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f20f400fc40 con 0x7f2104100f20 2024-05-08T23:45:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.237+0000 7f20fa7fc700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f20f00a1120 0x7f20f00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.237+0000 7f20fa7fc700 1 -- 172.21.15.12:0/3301929559 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f20f40cdca0 con 0x7f2104100f20 2024-05-08T23:45:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.237+0000 7f2108e49700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f20f00a1120 0x7f20f00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.237+0000 7f2108e49700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f20f00a1120 0x7f20f00a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2100009be0 tx=0x7f2100009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:09.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.241+0000 7f20fa7fc700 1 -- 172.21.15.12:0/3301929559 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f20f4097610 con 0x7f2104100f20 2024-05-08T23:45:09.266 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 23:45:08 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-b[82741]: 2024-05-08T23:45:08.894+0000 7fd00da00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:45:09.266 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 23:45:08 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-b[82741]: 2024-05-08T23:45:08.894+0000 7fd00da00700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T23:45:09.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:09 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] up:active 2024-05-08T23:45:09.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:09 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/456554867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T23:45:09.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:09 smithi160 ceph-mon[46152]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T23:45:09.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:09 smithi160 ceph-mon[46152]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T23:45:09.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:09 smithi160 ceph-mon[46152]: Cluster is now healthy 2024-05-08T23:45:09.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:09 smithi160 ceph-mon[46152]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-08T23:45:09.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:09 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-08T23:45:09.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:09 smithi160 ceph-mon[46152]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T23:45:09.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:09 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1044123405' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T23:45:09.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:09 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] up:active 2024-05-08T23:45:09.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:09 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/456554867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T23:45:09.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:09 smithi012 ceph-mon[42503]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T23:45:09.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:09 smithi012 ceph-mon[42503]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T23:45:09.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:09 smithi012 ceph-mon[42503]: Cluster is now healthy 2024-05-08T23:45:09.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:09 smithi012 ceph-mon[42503]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-08T23:45:09.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:09 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-08T23:45:09.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:09 smithi012 ceph-mon[42503]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T23:45:09.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:09 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1044123405' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T23:45:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.491+0000 7f210b8ae700 1 -- 172.21.15.12:0/3301929559 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f21040613e0 con 0x7f20f00a1120 2024-05-08T23:45:09.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.512+0000 7f20fa7fc700 1 -- 172.21.15.12:0/3301929559 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f21040613e0 con 0x7f20f00a1120 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.514+0000 7f210b8ae700 1 -- 172.21.15.12:0/3301929559 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f20f00a1120 msgr2=0x7f20f00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f20f00a1120 0x7f20f00a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2100009be0 tx=0x7f2100009260 comp rx=0 tx=0).stop 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 -- 172.21.15.12:0/3301929559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2104100f20 msgr2=0x7f2104074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2104100f20 0x7f2104074d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f20f4000c00 tx=0x7f20f4005e80 comp rx=0 tx=0).stop 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 -- 172.21.15.12:0/3301929559 shutdown_connections 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2104102ab0 0x7f21040738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2104100f20 0x7f2104074d00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f20f00a1120 0x7f20f00a35e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 --2- 172.21.15.12:0/3301929559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21041020f0 0x7f2104073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 -- 172.21.15.12:0/3301929559 >> 172.21.15.12:0/3301929559 conn(0x7f21040fc4e0 msgr2=0x7f21041055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 -- 172.21.15.12:0/3301929559 shutdown_connections 2024-05-08T23:45:09.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.515+0000 7f210b8ae700 1 -- 172.21.15.12:0/3301929559 wait complete. 2024-05-08T23:45:09.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:09 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] up:active 2024-05-08T23:45:09.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:09 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/456554867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T23:45:09.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:09 smithi204 ceph-mon[47373]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T23:45:09.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:09 smithi204 ceph-mon[47373]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T23:45:09.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:09 smithi204 ceph-mon[47373]: Cluster is now healthy 2024-05-08T23:45:09.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:09 smithi204 ceph-mon[47373]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-08T23:45:09.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:09 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-08T23:45:09.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:09 smithi204 ceph-mon[47373]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T23:45:09.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:09 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1044123405' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T23:45:09.525 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T23:45:09.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.672+0000 7ffa14bbd700 1 -- 172.21.15.12:0/2469432445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa10106d00 msgr2=0x7ffa101090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.672+0000 7ffa14bbd700 1 --2- 172.21.15.12:0/2469432445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa10106d00 0x7ffa101090f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b0a0 tx=0x7ffa0800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.672+0000 7ffa14bbd700 1 -- 172.21.15.12:0/2469432445 shutdown_connections 2024-05-08T23:45:09.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.672+0000 7ffa14bbd700 1 --2- 172.21.15.12:0/2469432445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa10106d00 0x7ffa101090f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.672+0000 7ffa14bbd700 1 --2- 172.21.15.12:0/2469432445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa10101aa0 0x7ffa10103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.672+0000 7ffa14bbd700 1 --2- 172.21.15.12:0/2469432445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa101043d0 0x7ffa101067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.672+0000 7ffa14bbd700 1 -- 172.21.15.12:0/2469432445 >> 172.21.15.12:0/2469432445 conn(0x7ffa100fb3f0 msgr2=0x7ffa100fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:09.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.673+0000 7ffa14bbd700 1 -- 172.21.15.12:0/2469432445 shutdown_connections 2024-05-08T23:45:09.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.673+0000 7ffa14bbd700 1 -- 172.21.15.12:0/2469432445 wait complete. 2024-05-08T23:45:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.673+0000 7ffa14bbd700 1 Processor -- start 2024-05-08T23:45:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.673+0000 7ffa14bbd700 1 -- start start 2024-05-08T23:45:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa14bbd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa10101aa0 0x7ffa10197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa14bbd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa101043d0 0x7ffa10197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa14bbd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa10106d00 0x7ffa1019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa14bbd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1019eb70 con 0x7ffa10101aa0 2024-05-08T23:45:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa14bbd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1019ecb0 con 0x7ffa101043d0 2024-05-08T23:45:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa14bbd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1019ee40 con 0x7ffa10106d00 2024-05-08T23:45:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa10101aa0 0x7ffa10197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa10101aa0 0x7ffa10197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37542/0 (socket says 172.21.15.12:37542) 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0e59c700 1 -- 172.21.15.12:0/3224427963 learned_addr learned my addr 172.21.15.12:0/3224427963 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0dd9b700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa101043d0 0x7ffa10197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0ed9d700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa10106d00 0x7ffa1019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0e59c700 1 -- 172.21.15.12:0/3224427963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa10106d00 msgr2=0x7ffa1019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0e59c700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa10106d00 0x7ffa1019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0e59c700 1 -- 172.21.15.12:0/3224427963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa101043d0 msgr2=0x7ffa10197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0e59c700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa101043d0 0x7ffa10197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0e59c700 1 -- 172.21.15.12:0/3224427963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa08009d00 con 0x7ffa10101aa0 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0ed9d700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa10106d00 0x7ffa1019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0dd9b700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa101043d0 0x7ffa10197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ffa0e59c700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa10101aa0 0x7ffa10197040 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400e9a0 tx=0x7ffa0400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3224427963 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04009790 con 0x7ffa10101aa0 2024-05-08T23:45:09.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.674+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3224427963 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0400cbd0 con 0x7ffa10101aa0 2024-05-08T23:45:09.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.675+0000 7ffa14bbd700 1 -- 172.21.15.12:0/3224427963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1019f120 con 0x7ffa10101aa0 2024-05-08T23:45:09.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.675+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3224427963 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04018680 con 0x7ffa10101aa0 2024-05-08T23:45:09.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.675+0000 7ffa14bbd700 1 -- 172.21.15.12:0/3224427963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1019f680 con 0x7ffa10101aa0 2024-05-08T23:45:09.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.675+0000 7ffa14bbd700 1 -- 172.21.15.12:0/3224427963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa100fcf70 con 0x7ffa10101aa0 2024-05-08T23:45:09.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.677+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3224427963 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ffa0400cd40 con 0x7ffa10101aa0 2024-05-08T23:45:09.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.677+0000 7ff9ff7fe700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff9f80a11a0 0x7ff9f80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:09.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.678+0000 7ffa0dd9b700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff9f80a11a0 0x7ff9f80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:09.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.678+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3224427963 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffa04014070 con 0x7ffa10101aa0 2024-05-08T23:45:09.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.678+0000 7ffa0dd9b700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff9f80a11a0 0x7ff9f80a3660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009cb0 tx=0x7ffa00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:09.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.682+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3224427963 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffa04096830 con 0x7ffa10101aa0 2024-05-08T23:45:09.915 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:45:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:45:09.497+0000 7f345125e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:45:09.915 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:45:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:45:09.497+0000 7f345125e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:45:09.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:45:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:45:09.497+0000 7f345125e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:45:09.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:45:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:45:09.497+0000 7f345125e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:45:09.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:45:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:45:09.497+0000 7f345125e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:45:09.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.935+0000 7ffa14bbd700 1 -- 172.21.15.12:0/3224427963 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa100613e0 con 0x7ff9f80a11a0 2024-05-08T23:45:09.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.958+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3224427963 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ffa100613e0 con 0x7ff9f80a11a0 2024-05-08T23:45:09.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.960+0000 7ffa14bbd700 1 -- 172.21.15.12:0/3224427963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff9f80a11a0 msgr2=0x7ff9f80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.960+0000 7ffa14bbd700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff9f80a11a0 0x7ff9f80a3660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009cb0 tx=0x7ffa00009260 comp rx=0 tx=0).stop 2024-05-08T23:45:09.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.960+0000 7ffa14bbd700 1 -- 172.21.15.12:0/3224427963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa10101aa0 msgr2=0x7ffa10197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.960+0000 7ffa14bbd700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa10101aa0 0x7ffa10197040 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400e9a0 tx=0x7ffa0400ed60 comp rx=0 tx=0).stop 2024-05-08T23:45:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.960+0000 7ffa14bbd700 1 -- 172.21.15.12:0/3224427963 shutdown_connections 2024-05-08T23:45:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.960+0000 7ffa14bbd700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa10106d00 0x7ffa1019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.960+0000 7ffa14bbd700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa10101aa0 0x7ffa10197040 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.960+0000 7ffa14bbd700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff9f80a11a0 0x7ff9f80a3660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.960+0000 7ffa14bbd700 1 --2- 172.21.15.12:0/3224427963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa101043d0 0x7ffa10197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.960+0000 7ffa14bbd700 1 -- 172.21.15.12:0/3224427963 >> 172.21.15.12:0/3224427963 conn(0x7ffa100fb3f0 msgr2=0x7ffa10102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.961+0000 7ffa14bbd700 1 -- 172.21.15.12:0/3224427963 shutdown_connections 2024-05-08T23:45:09.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:09.961+0000 7ffa14bbd700 1 -- 172.21.15.12:0/3224427963 wait complete. 2024-05-08T23:45:09.971 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T23:45:10.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.120+0000 7f6c5ac66700 1 -- 172.21.15.12:0/1487626749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c541001a0 msgr2=0x7f6c54100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.120+0000 7f6c5ac66700 1 --2- 172.21.15.12:0/1487626749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c541001a0 0x7f6c54100620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009990 tx=0x7f6c40009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.120+0000 7f6c5ac66700 1 -- 172.21.15.12:0/1487626749 shutdown_connections 2024-05-08T23:45:10.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.120+0000 7f6c5ac66700 1 --2- 172.21.15.12:0/1487626749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54100b60 0x7f6c54100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.120+0000 7f6c5ac66700 1 --2- 172.21.15.12:0/1487626749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c540ff840 0x7f6c540ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.120+0000 7f6c5ac66700 1 --2- 172.21.15.12:0/1487626749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c541001a0 0x7f6c54100620 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.120+0000 7f6c5ac66700 1 -- 172.21.15.12:0/1487626749 >> 172.21.15.12:0/1487626749 conn(0x7f6c540fb400 msgr2=0x7f6c540fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:10.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.120+0000 7f6c5ac66700 1 -- 172.21.15.12:0/1487626749 shutdown_connections 2024-05-08T23:45:10.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.120+0000 7f6c5ac66700 1 -- 172.21.15.12:0/1487626749 wait complete. 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.121+0000 7f6c5ac66700 1 Processor -- start 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.121+0000 7f6c5ac66700 1 -- start start 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.121+0000 7f6c5ac66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c540ff840 0x7f6c54199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.121+0000 7f6c5ac66700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c541001a0 0x7f6c54199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.121+0000 7f6c5ac66700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54100b60 0x7f6c541a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c5ac66700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541a0c60 con 0x7f6c540ff840 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c5ac66700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541a0da0 con 0x7f6c54100b60 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c5ac66700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541a0f30 con 0x7f6c541001a0 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c58a02700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c540ff840 0x7f6c54199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c58a02700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c540ff840 0x7f6c54199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37550/0 (socket says 172.21.15.12:37550) 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c58a02700 1 -- 172.21.15.12:0/2590011770 learned_addr learned my addr 172.21.15.12:0/2590011770 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:10.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c53fff700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c541001a0 0x7f6c54199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c59203700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54100b60 0x7f6c541a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c58a02700 1 -- 172.21.15.12:0/2590011770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c541001a0 msgr2=0x7f6c54199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c58a02700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c541001a0 0x7f6c54199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c58a02700 1 -- 172.21.15.12:0/2590011770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54100b60 msgr2=0x7f6c541a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c58a02700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54100b60 0x7f6c541a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c58a02700 1 -- 172.21.15.12:0/2590011770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400095f0 con 0x7f6c540ff840 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c58a02700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c540ff840 0x7f6c54199130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e9a0 tx=0x7f6c4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c51ffb700 1 -- 172.21.15.12:0/2590011770 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48009790 con 0x7f6c540ff840 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c5ac66700 1 -- 172.21.15.12:0/2590011770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c541a1210 con 0x7f6c540ff840 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c51ffb700 1 -- 172.21.15.12:0/2590011770 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c48010430 con 0x7f6c540ff840 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.122+0000 7f6c51ffb700 1 -- 172.21.15.12:0/2590011770 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48018680 con 0x7f6c540ff840 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.123+0000 7f6c5ac66700 1 -- 172.21.15.12:0/2590011770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c541a1770 con 0x7f6c540ff840 2024-05-08T23:45:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.124+0000 7f6c5ac66700 1 -- 172.21.15.12:0/2590011770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5410a890 con 0x7f6c540ff840 2024-05-08T23:45:10.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.125+0000 7f6c51ffb700 1 -- 172.21.15.12:0/2590011770 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6c480105a0 con 0x7f6c540ff840 2024-05-08T23:45:10.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.126+0000 7f6c51ffb700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6c440a1120 0x7f6c440a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.126+0000 7f6c53fff700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6c440a1120 0x7f6c440a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:10.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.126+0000 7f6c51ffb700 1 -- 172.21.15.12:0/2590011770 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6c48014070 con 0x7f6c540ff840 2024-05-08T23:45:10.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.126+0000 7f6c53fff700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6c440a1120 0x7f6c440a35e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b2e0 tx=0x7f6c40005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:10.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.130+0000 7f6c51ffb700 1 -- 172.21.15.12:0/2590011770 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6c48096400 con 0x7f6c540ff840 2024-05-08T23:45:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.385+0000 7f6c5ac66700 1 -- 172.21.15.12:0/2590011770 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c540613e0 con 0x7f6c440a1120 2024-05-08T23:45:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.390+0000 7f6c51ffb700 1 -- 172.21.15.12:0/2590011770 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f6c540613e0 con 0x7f6c440a1120 2024-05-08T23:45:10.386 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_0/8b6966b3-1d1f-41d1-aa20-be8b0621a83d 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.392+0000 7f6c5ac66700 1 -- 172.21.15.12:0/2590011770 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6c440a1120 msgr2=0x7f6c440a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.392+0000 7f6c5ac66700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6c440a1120 0x7f6c440a35e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b2e0 tx=0x7f6c40005fd0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.392+0000 7f6c5ac66700 1 -- 172.21.15.12:0/2590011770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c540ff840 msgr2=0x7f6c54199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.392+0000 7f6c5ac66700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c540ff840 0x7f6c54199130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e9a0 tx=0x7f6c4800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.392+0000 7f6c5ac66700 1 -- 172.21.15.12:0/2590011770 shutdown_connections 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.392+0000 7f6c5ac66700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6c440a1120 0x7f6c440a35e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.392+0000 7f6c5ac66700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54100b60 0x7f6c541a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.392+0000 7f6c5ac66700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c540ff840 0x7f6c54199130 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.392+0000 7f6c5ac66700 1 --2- 172.21.15.12:0/2590011770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c541001a0 0x7f6c54199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.392+0000 7f6c5ac66700 1 -- 172.21.15.12:0/2590011770 >> 172.21.15.12:0/2590011770 conn(0x7f6c540fb400 msgr2=0x7f6c540fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.393+0000 7f6c5ac66700 1 -- 172.21.15.12:0/2590011770 shutdown_connections 2024-05-08T23:45:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.393+0000 7f6c5ac66700 1 -- 172.21.15.12:0/2590011770 wait complete. 2024-05-08T23:45:10.401 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:10 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] up:active 2024-05-08T23:45:10.401 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:10 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1044123405' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T23:45:10.401 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:10 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-08T23:45:10.401 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:10 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:45:10.401 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:10 smithi012 ceph-mon[42503]: pgmap v228: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 12 op/s 2024-05-08T23:45:10.401 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:10 smithi012 ceph-mon[42503]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:10.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T23:45:10.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:10 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] up:active 2024-05-08T23:45:10.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:10 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1044123405' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T23:45:10.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:10 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-08T23:45:10.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:10 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:45:10.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:10 smithi204 ceph-mon[47373]: pgmap v228: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 12 op/s 2024-05-08T23:45:10.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:10 smithi204 ceph-mon[47373]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:10.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:10 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] up:active 2024-05-08T23:45:10.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:10 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1044123405' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T23:45:10.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:10 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-08T23:45:10.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:10 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:45:10.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:10 smithi160 ceph-mon[46152]: pgmap v228: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 12 op/s 2024-05-08T23:45:10.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:10 smithi160 ceph-mon[46152]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.550+0000 7fafda6da700 1 -- 172.21.15.12:0/1725668414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafd4100fb0 msgr2=0x7fafd4101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.550+0000 7fafda6da700 1 --2- 172.21.15.12:0/1725668414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafd4100fb0 0x7fafd4101430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009990 tx=0x7fafc8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.550+0000 7fafda6da700 1 -- 172.21.15.12:0/1725668414 shutdown_connections 2024-05-08T23:45:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.550+0000 7fafda6da700 1 --2- 172.21.15.12:0/1725668414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafd4101970 0x7fafd4108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.550+0000 7fafda6da700 1 --2- 172.21.15.12:0/1725668414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd40ffe60 0x7fafd4100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.550+0000 7fafda6da700 1 --2- 172.21.15.12:0/1725668414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafd4100fb0 0x7fafd4101430 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.550+0000 7fafda6da700 1 -- 172.21.15.12:0/1725668414 >> 172.21.15.12:0/1725668414 conn(0x7fafd40fb3f0 msgr2=0x7fafd40fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.550+0000 7fafda6da700 1 -- 172.21.15.12:0/1725668414 shutdown_connections 2024-05-08T23:45:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.550+0000 7fafda6da700 1 -- 172.21.15.12:0/1725668414 wait complete. 2024-05-08T23:45:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.550+0000 7fafda6da700 1 Processor -- start 2024-05-08T23:45:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafda6da700 1 -- start start 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafda6da700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafd40ffe60 0x7fafd4103d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafda6da700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd4100fb0 0x7fafd4102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafda6da700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafd4101970 0x7fafd4102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafda6da700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd4103100 con 0x7fafd40ffe60 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafda6da700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd4103270 con 0x7fafd4101970 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafda6da700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd41a0f80 con 0x7fafd4100fb0 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafd3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafd40ffe60 0x7fafd4103d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafd3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafd40ffe60 0x7fafd4103d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37568/0 (socket says 172.21.15.12:37568) 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafd3fff700 1 -- 172.21.15.12:0/2971660989 learned_addr learned my addr 172.21.15.12:0/2971660989 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafd8c77700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafd4101970 0x7fafd4102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafd37fe700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd4100fb0 0x7fafd4102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafd3fff700 1 -- 172.21.15.12:0/2971660989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd4100fb0 msgr2=0x7fafd4102420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafd3fff700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd4100fb0 0x7fafd4102420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafd3fff700 1 -- 172.21.15.12:0/2971660989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafd4101970 msgr2=0x7fafd4102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafd3fff700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafd4101970 0x7fafd4102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.551+0000 7fafd3fff700 1 -- 172.21.15.12:0/2971660989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc80095f0 con 0x7fafd40ffe60 2024-05-08T23:45:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.552+0000 7fafd3fff700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafd40ffe60 0x7fafd4103d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fafc400d950 tx=0x7fafc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.552+0000 7fafd17fa700 1 -- 172.21.15.12:0/2971660989 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc40098e0 con 0x7fafd40ffe60 2024-05-08T23:45:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.552+0000 7fafda6da700 1 -- 172.21.15.12:0/2971660989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd41a1210 con 0x7fafd40ffe60 2024-05-08T23:45:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.552+0000 7fafd17fa700 1 -- 172.21.15.12:0/2971660989 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafc4010460 con 0x7fafd40ffe60 2024-05-08T23:45:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.552+0000 7fafd17fa700 1 -- 172.21.15.12:0/2971660989 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc400f620 con 0x7fafd40ffe60 2024-05-08T23:45:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.552+0000 7fafda6da700 1 -- 172.21.15.12:0/2971660989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd41a1720 con 0x7fafd40ffe60 2024-05-08T23:45:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.553+0000 7fafda6da700 1 -- 172.21.15.12:0/2971660989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd4067cf0 con 0x7fafd40ffe60 2024-05-08T23:45:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.554+0000 7fafd17fa700 1 -- 172.21.15.12:0/2971660989 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fafc40108d0 con 0x7fafd40ffe60 2024-05-08T23:45:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.555+0000 7fafd17fa700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fafbc0a1120 0x7fafbc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.555+0000 7fafd37fe700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fafbc0a1120 0x7fafbc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.555+0000 7fafd17fa700 1 -- 172.21.15.12:0/2971660989 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fafc40cc680 con 0x7fafd40ffe60 2024-05-08T23:45:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.555+0000 7fafd37fe700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fafbc0a1120 0x7fafbc0a35e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fafc8006010 tx=0x7fafc80037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:10.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.559+0000 7fafd17fa700 1 -- 172.21.15.12:0/2971660989 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fafc4095ec0 con 0x7fafd40ffe60 2024-05-08T23:45:10.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.815+0000 7fafda6da700 1 -- 172.21.15.12:0/2971660989 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fafd40fd620 con 0x7fafbc0a1120 2024-05-08T23:45:10.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.838+0000 7fafd17fa700 1 -- 172.21.15.12:0/2971660989 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fafd40fd620 con 0x7fafbc0a1120 2024-05-08T23:45:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 -- 172.21.15.12:0/2971660989 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fafbc0a1120 msgr2=0x7fafbc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fafbc0a1120 0x7fafbc0a35e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fafc8006010 tx=0x7fafc80037b0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 -- 172.21.15.12:0/2971660989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafd40ffe60 msgr2=0x7fafd4103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafd40ffe60 0x7fafd4103d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fafc400d950 tx=0x7fafc400dc60 comp rx=0 tx=0).stop 2024-05-08T23:45:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 -- 172.21.15.12:0/2971660989 shutdown_connections 2024-05-08T23:45:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fafbc0a1120 0x7fafbc0a35e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafd4101970 0x7fafd4102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafd40ffe60 0x7fafd4103d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 --2- 172.21.15.12:0/2971660989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd4100fb0 0x7fafd4102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 -- 172.21.15.12:0/2971660989 >> 172.21.15.12:0/2971660989 conn(0x7fafd40fb3f0 msgr2=0x7fafd40fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 -- 172.21.15.12:0/2971660989 shutdown_connections 2024-05-08T23:45:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.841+0000 7fafda6da700 1 -- 172.21.15.12:0/2971660989 wait complete. 2024-05-08T23:45:10.851 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T23:45:10.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.998+0000 7f73385f0700 1 -- 172.21.15.12:0/2737353365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7330101980 msgr2=0x7f73301082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.998+0000 7f73385f0700 1 --2- 172.21.15.12:0/2737353365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7330101980 0x7f73301082a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f732800b0a0 tx=0x7f732800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.998+0000 7f73385f0700 1 -- 172.21.15.12:0/2737353365 shutdown_connections 2024-05-08T23:45:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.998+0000 7f73385f0700 1 --2- 172.21.15.12:0/2737353365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7330101980 0x7f73301082a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.998+0000 7f73385f0700 1 --2- 172.21.15.12:0/2737353365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73300ffe70 0x7f7330100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.998+0000 7f73385f0700 1 --2- 172.21.15.12:0/2737353365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7330100fc0 0x7f7330101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.998+0000 7f73385f0700 1 -- 172.21.15.12:0/2737353365 >> 172.21.15.12:0/2737353365 conn(0x7f73300fb400 msgr2=0x7f73300fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f73385f0700 1 -- 172.21.15.12:0/2737353365 shutdown_connections 2024-05-08T23:45:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f73385f0700 1 -- 172.21.15.12:0/2737353365 wait complete. 2024-05-08T23:45:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f73385f0700 1 Processor -- start 2024-05-08T23:45:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f73385f0700 1 -- start start 2024-05-08T23:45:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f73385f0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300ffe70 0x7f7330103da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f73385f0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7330100fc0 0x7f7330102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f73385f0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7330101980 0x7f7330102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f73385f0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7330103110 con 0x7f73300ffe70 2024-05-08T23:45:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f73385f0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7330103280 con 0x7f7330101980 2024-05-08T23:45:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f73385f0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73301a1080 con 0x7f7330100fc0 2024-05-08T23:45:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:10.999+0000 7f733638c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300ffe70 0x7f7330103da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f733638c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300ffe70 0x7f7330103da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37592/0 (socket says 172.21.15.12:37592) 2024-05-08T23:45:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f733638c700 1 -- 172.21.15.12:0/2122580487 learned_addr learned my addr 172.21.15.12:0/2122580487 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f733638c700 1 -- 172.21.15.12:0/2122580487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7330100fc0 msgr2=0x7f7330102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f7335b8b700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7330100fc0 0x7f7330102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f733638c700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7330100fc0 0x7f7330102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f733638c700 1 -- 172.21.15.12:0/2122580487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7330101980 msgr2=0x7f7330102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f733638c700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7330101980 0x7f7330102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f733638c700 1 -- 172.21.15.12:0/2122580487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7328009d00 con 0x7f73300ffe70 2024-05-08T23:45:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f733638c700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300ffe70 0x7f7330103da0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f732400d950 tx=0x7f732400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f73237fe700 1 -- 172.21.15.12:0/2122580487 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324009ae0 con 0x7f73300ffe70 2024-05-08T23:45:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f73237fe700 1 -- 172.21.15.12:0/2122580487 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7324009c40 con 0x7f73300ffe70 2024-05-08T23:45:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f73385f0700 1 -- 172.21.15.12:0/2122580487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73301a1310 con 0x7f73300ffe70 2024-05-08T23:45:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f73237fe700 1 -- 172.21.15.12:0/2122580487 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732400f620 con 0x7f73300ffe70 2024-05-08T23:45:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.000+0000 7f73385f0700 1 -- 172.21.15.12:0/2122580487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73301a1820 con 0x7f73300ffe70 2024-05-08T23:45:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.001+0000 7f73385f0700 1 -- 172.21.15.12:0/2122580487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7330068140 con 0x7f73300ffe70 2024-05-08T23:45:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.003+0000 7f73237fe700 1 -- 172.21.15.12:0/2122580487 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7324010460 con 0x7f73300ffe70 2024-05-08T23:45:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.004+0000 7f73237fe700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f731c0a1120 0x7f731c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.004+0000 7f7335b8b700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f731c0a1120 0x7f731c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.004+0000 7f73237fe700 1 -- 172.21.15.12:0/2122580487 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f73240cc740 con 0x7f73300ffe70 2024-05-08T23:45:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.004+0000 7f7335b8b700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f731c0a1120 0x7f731c0a35e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f732c009b30 tx=0x7f732c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:11.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.008+0000 7f73237fe700 1 -- 172.21.15.12:0/2122580487 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7324096020 con 0x7f73300ffe70 2024-05-08T23:45:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.087+0000 7f73237fe700 1 -- 172.21.15.12:0/2122580487 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f73240963f0 con 0x7f73300ffe70 2024-05-08T23:45:11.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:11 smithi012 ceph-mon[42503]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:11.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:11 smithi012 ceph-mon[42503]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:11.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.273+0000 7f73385f0700 1 -- 172.21.15.12:0/2122580487 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f73300fd5a0 con 0x7f731c0a1120 2024-05-08T23:45:11.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.277+0000 7f73237fe700 1 -- 172.21.15.12:0/2122580487 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f73300fd5a0 con 0x7f731c0a1120 2024-05-08T23:45:11.273 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685 2024-05-08T23:45:11.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.279+0000 7f73385f0700 1 -- 172.21.15.12:0/2122580487 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f731c0a1120 msgr2=0x7f731c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.279+0000 7f73385f0700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f731c0a1120 0x7f731c0a35e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f732c009b30 tx=0x7f732c009260 comp rx=0 tx=0).stop 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.279+0000 7f73385f0700 1 -- 172.21.15.12:0/2122580487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300ffe70 msgr2=0x7f7330103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.279+0000 7f73385f0700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300ffe70 0x7f7330103da0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f732400d950 tx=0x7f732400dd10 comp rx=0 tx=0).stop 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.280+0000 7f73385f0700 1 -- 172.21.15.12:0/2122580487 shutdown_connections 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.280+0000 7f73385f0700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f731c0a1120 0x7f731c0a35e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.280+0000 7f73385f0700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7330101980 0x7f7330102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.280+0000 7f73385f0700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300ffe70 0x7f7330103da0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.280+0000 7f73385f0700 1 --2- 172.21.15.12:0/2122580487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7330100fc0 0x7f7330102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.280+0000 7f73385f0700 1 -- 172.21.15.12:0/2122580487 >> 172.21.15.12:0/2122580487 conn(0x7f73300fb400 msgr2=0x7f73300fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.280+0000 7f73385f0700 1 -- 172.21.15.12:0/2122580487 shutdown_connections 2024-05-08T23:45:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.280+0000 7f73385f0700 1 -- 172.21.15.12:0/2122580487 wait complete. 2024-05-08T23:45:11.290 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 -- 172.21.15.12:0/1099077691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7400ff6d0 msgr2=0x7ff7400ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 --2- 172.21.15.12:0/1099077691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7400ff6d0 0x7ff7400ffaf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009990 tx=0x7ff73c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 -- 172.21.15.12:0/1099077691 shutdown_connections 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 --2- 172.21.15.12:0/1099077691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7401009f0 0x7ff740100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 --2- 172.21.15.12:0/1099077691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7400ff6d0 0x7ff7400ffaf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 --2- 172.21.15.12:0/1099077691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff740100030 0x7ff7401004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 -- 172.21.15.12:0/1099077691 >> 172.21.15.12:0/1099077691 conn(0x7ff7400fb2f0 msgr2=0x7ff7400fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 -- 172.21.15.12:0/1099077691 shutdown_connections 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 -- 172.21.15.12:0/1099077691 wait complete. 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 Processor -- start 2024-05-08T23:45:11.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.432+0000 7ff7481e9700 1 -- start start 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff7481e9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7400ff6d0 0x7ff740198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff7481e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740100030 0x7ff7401994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff7481e9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7401009f0 0x7ff7401a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff7481e9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7401a0ab0 con 0x7ff740100030 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff7481e9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7401a0bf0 con 0x7ff7400ff6d0 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff7481e9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7401a0d80 con 0x7ff7401009f0 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff745784700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740100030 0x7ff7401994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff745784700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740100030 0x7ff7401994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37608/0 (socket says 172.21.15.12:37608) 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff745784700 1 -- 172.21.15.12:0/530328957 learned_addr learned my addr 172.21.15.12:0/530328957 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff745f85700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7400ff6d0 0x7ff740198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff746786700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7401009f0 0x7ff7401a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff745784700 1 -- 172.21.15.12:0/530328957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7401009f0 msgr2=0x7ff7401a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff745784700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7401009f0 0x7ff7401a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff745784700 1 -- 172.21.15.12:0/530328957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7400ff6d0 msgr2=0x7ff740198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:11.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff745784700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7400ff6d0 0x7ff740198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff745784700 1 -- 172.21.15.12:0/530328957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c0095f0 con 0x7ff740100030 2024-05-08T23:45:11.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.433+0000 7ff745784700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740100030 0x7ff7401994c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff73000e9a0 tx=0x7ff73000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:11.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.434+0000 7ff736ffd700 1 -- 172.21.15.12:0/530328957 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730009790 con 0x7ff740100030 2024-05-08T23:45:11.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.434+0000 7ff736ffd700 1 -- 172.21.15.12:0/530328957 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff730010430 con 0x7ff740100030 2024-05-08T23:45:11.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.434+0000 7ff7481e9700 1 -- 172.21.15.12:0/530328957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7401a1060 con 0x7ff740100030 2024-05-08T23:45:11.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.434+0000 7ff736ffd700 1 -- 172.21.15.12:0/530328957 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730018680 con 0x7ff740100030 2024-05-08T23:45:11.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.434+0000 7ff7481e9700 1 -- 172.21.15.12:0/530328957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7401a1570 con 0x7ff740100030 2024-05-08T23:45:11.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.435+0000 7ff7481e9700 1 -- 172.21.15.12:0/530328957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74010a700 con 0x7ff740100030 2024-05-08T23:45:11.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.436+0000 7ff736ffd700 1 -- 172.21.15.12:0/530328957 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff7300105a0 con 0x7ff740100030 2024-05-08T23:45:11.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.437+0000 7ff736ffd700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff72c0a11b0 0x7ff72c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:11.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.437+0000 7ff736ffd700 1 -- 172.21.15.12:0/530328957 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff730014070 con 0x7ff740100030 2024-05-08T23:45:11.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.437+0000 7ff745f85700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff72c0a11b0 0x7ff72c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:11.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.437+0000 7ff745f85700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff72c0a11b0 0x7ff72c0a3670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00b2e0 tx=0x7ff73c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:11.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.441+0000 7ff736ffd700 1 -- 172.21.15.12:0/530328957 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff730096430 con 0x7ff740100030 2024-05-08T23:45:11.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:11 smithi204 ceph-mon[47373]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:11.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:11 smithi204 ceph-mon[47373]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:11.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:11 smithi160 ceph-mon[46152]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:11.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:11 smithi160 ceph-mon[46152]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.715+0000 7ff7481e9700 1 -- 172.21.15.12:0/530328957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff740067c10 con 0x7ff740100030 2024-05-08T23:45:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.716+0000 7ff736ffd700 1 -- 172.21.15.12:0/530328957 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10876 (secure 0 0 0) 0x7ff730026090 con 0x7ff740100030 2024-05-08T23:45:11.712 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:11.712 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:45:09.067610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:45:11.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 -- 172.21.15.12:0/530328957 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff72c0a11b0 msgr2=0x7ff72c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:11.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff72c0a11b0 0x7ff72c0a3670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00b2e0 tx=0x7ff73c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 -- 172.21.15.12:0/530328957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740100030 msgr2=0x7ff7401994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:11.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740100030 0x7ff7401994c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff73000e9a0 tx=0x7ff73000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 -- 172.21.15.12:0/530328957 shutdown_connections 2024-05-08T23:45:11.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7401009f0 0x7ff7401a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff72c0a11b0 0x7ff72c0a3670 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7400ff6d0 0x7ff740198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 --2- 172.21.15.12:0/530328957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740100030 0x7ff7401994c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 -- 172.21.15.12:0/530328957 >> 172.21.15.12:0/530328957 conn(0x7ff7400fb2f0 msgr2=0x7ff7400fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:11.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 -- 172.21.15.12:0/530328957 shutdown_connections 2024-05-08T23:45:11.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.718+0000 7ff7481e9700 1 -- 172.21.15.12:0/530328957 wait complete. 2024-05-08T23:45:11.715 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 23 2024-05-08T23:45:11.728 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:45:11.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.869+0000 7f02572f6700 1 -- 172.21.15.12:0/3564605917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02500ffd60 msgr2=0x7f0250100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:11.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.869+0000 7f02572f6700 1 --2- 172.21.15.12:0/3564605917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02500ffd60 0x7f0250100180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f024c0099c0 tx=0x7f024c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.869+0000 7f02572f6700 1 -- 172.21.15.12:0/3564605917 shutdown_connections 2024-05-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.869+0000 7f02572f6700 1 --2- 172.21.15.12:0/3564605917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02501018f0 0x7f02501081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.869+0000 7f02572f6700 1 --2- 172.21.15.12:0/3564605917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02500ffd60 0x7f0250100180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.869+0000 7f02572f6700 1 --2- 172.21.15.12:0/3564605917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0250100f50 0x7f02501013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.869+0000 7f02572f6700 1 -- 172.21.15.12:0/3564605917 >> 172.21.15.12:0/3564605917 conn(0x7f02500fb2f0 msgr2=0x7f02500fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.869+0000 7f02572f6700 1 -- 172.21.15.12:0/3564605917 shutdown_connections 2024-05-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.869+0000 7f02572f6700 1 -- 172.21.15.12:0/3564605917 wait complete. 2024-05-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f02572f6700 1 Processor -- start 2024-05-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f02572f6700 1 -- start start 2024-05-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f02572f6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02500ffd60 0x7f0250103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f02572f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0250100f50 0x7f0250102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f02572f6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02501018f0 0x7f0250102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f02572f6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0250103010 con 0x7f0250100f50 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f02572f6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0250103180 con 0x7f02501018f0 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f02572f6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02501a0e80 con 0x7f02500ffd60 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f0254891700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0250100f50 0x7f0250102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f0254891700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0250100f50 0x7f0250102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37624/0 (socket says 172.21.15.12:37624) 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f0254891700 1 -- 172.21.15.12:0/266007972 learned_addr learned my addr 172.21.15.12:0/266007972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.870+0000 7f0255893700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02501018f0 0x7f0250102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f0255092700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02500ffd60 0x7f0250103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f0254891700 1 -- 172.21.15.12:0/266007972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02500ffd60 msgr2=0x7f0250103ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f0254891700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02500ffd60 0x7f0250103ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f0254891700 1 -- 172.21.15.12:0/266007972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02501018f0 msgr2=0x7f0250102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f0254891700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02501018f0 0x7f0250102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f0254891700 1 -- 172.21.15.12:0/266007972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0095f0 con 0x7f0250100f50 2024-05-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f0255092700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02500ffd60 0x7f0250103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f0254891700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0250100f50 0x7f0250102330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f024000d980 tx=0x7f024000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f02467fc700 1 -- 172.21.15.12:0/266007972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240004220 con 0x7f0250100f50 2024-05-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f02467fc700 1 -- 172.21.15.12:0/266007972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0240007640 con 0x7f0250100f50 2024-05-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f02572f6700 1 -- 172.21.15.12:0/266007972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02501a1110 con 0x7f0250100f50 2024-05-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f02467fc700 1 -- 172.21.15.12:0/266007972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240005610 con 0x7f0250100f50 2024-05-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.871+0000 7f02572f6700 1 -- 172.21.15.12:0/266007972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02501a1620 con 0x7f0250100f50 2024-05-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.872+0000 7f02572f6700 1 -- 172.21.15.12:0/266007972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0250067c10 con 0x7f0250100f50 2024-05-08T23:45:11.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.873+0000 7f02467fc700 1 -- 172.21.15.12:0/266007972 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0240004750 con 0x7f0250100f50 2024-05-08T23:45:11.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.874+0000 7f02467fc700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f023c0a11b0 0x7f023c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:11.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.874+0000 7f0255092700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f023c0a11b0 0x7f023c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:11.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.874+0000 7f02467fc700 1 -- 172.21.15.12:0/266007972 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f02400ccc80 con 0x7f0250100f50 2024-05-08T23:45:11.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.874+0000 7f0255092700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f023c0a11b0 0x7f023c0a3670 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f024c0036f0 tx=0x7f024c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:11.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:11.878+0000 7f02467fc700 1 -- 172.21.15.12:0/266007972 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0240096460 con 0x7f0250100f50 2024-05-08T23:45:12.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.102+0000 7f02572f6700 1 -- 172.21.15.12:0/266007972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f025010b6f0 con 0x7f0250100f50 2024-05-08T23:45:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.104+0000 7f02467fc700 1 -- 172.21.15.12:0/266007972 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22105 (secure 0 0 0) 0x7f0240016020 con 0x7f0250100f50 2024-05-08T23:45:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:12.103 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:05.029999+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:45:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 -- 172.21.15.12:0/266007972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f023c0a11b0 msgr2=0x7f023c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f023c0a11b0 0x7f023c0a3670 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f024c0036f0 tx=0x7f024c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 -- 172.21.15.12:0/266007972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0250100f50 msgr2=0x7f0250102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0250100f50 0x7f0250102330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f024000d980 tx=0x7f024000dc90 comp rx=0 tx=0).stop 2024-05-08T23:45:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 -- 172.21.15.12:0/266007972 shutdown_connections 2024-05-08T23:45:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f023c0a11b0 0x7f023c0a3670 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02501018f0 0x7f0250102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02500ffd60 0x7f0250103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 --2- 172.21.15.12:0/266007972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0250100f50 0x7f0250102330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 -- 172.21.15.12:0/266007972 >> 172.21.15.12:0/266007972 conn(0x7f02500fb2f0 msgr2=0x7f0250068040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 -- 172.21.15.12:0/266007972 shutdown_connections 2024-05-08T23:45:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.106+0000 7f02572f6700 1 -- 172.21.15.12:0/266007972 wait complete. 2024-05-08T23:45:12.116 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:45:12.116 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:45:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.257+0000 7ff354022700 1 -- 172.21.15.12:0/2265333412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c101fe0 msgr2=0x7ff34c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.257+0000 7ff354022700 1 --2- 172.21.15.12:0/2265333412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c101fe0 0x7ff34c102460 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009990 tx=0x7ff33c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.257+0000 7ff354022700 1 -- 172.21.15.12:0/2265333412 shutdown_connections 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.257+0000 7ff354022700 1 --2- 172.21.15.12:0/2265333412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34c1029a0 0x7ff34c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.257+0000 7ff354022700 1 --2- 172.21.15.12:0/2265333412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34c100e90 0x7ff34c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.257+0000 7ff354022700 1 --2- 172.21.15.12:0/2265333412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c101fe0 0x7ff34c102460 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.257+0000 7ff354022700 1 -- 172.21.15.12:0/2265333412 >> 172.21.15.12:0/2265333412 conn(0x7ff34c0fc400 msgr2=0x7ff34c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.258+0000 7ff354022700 1 -- 172.21.15.12:0/2265333412 shutdown_connections 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.258+0000 7ff354022700 1 -- 172.21.15.12:0/2265333412 wait complete. 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.258+0000 7ff354022700 1 Processor -- start 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.258+0000 7ff354022700 1 -- start start 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.258+0000 7ff354022700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c100e90 0x7ff34c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.258+0000 7ff354022700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34c101fe0 0x7ff34c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.258+0000 7ff354022700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34c1029a0 0x7ff34c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff354022700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c074070 con 0x7ff34c100e90 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff354022700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c0741e0 con 0x7ff34c101fe0 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff354022700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c1a0e20 con 0x7ff34c1029a0 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff351dbe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c100e90 0x7ff34c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff351dbe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c100e90 0x7ff34c074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37634/0 (socket says 172.21.15.12:37634) 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff351dbe700 1 -- 172.21.15.12:0/257272885 learned_addr learned my addr 172.21.15.12:0/257272885 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff3525bf700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34c1029a0 0x7ff34c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff3515bd700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34c101fe0 0x7ff34c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff351dbe700 1 -- 172.21.15.12:0/257272885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34c1029a0 msgr2=0x7ff34c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff351dbe700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34c1029a0 0x7ff34c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff351dbe700 1 -- 172.21.15.12:0/257272885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34c101fe0 msgr2=0x7ff34c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff351dbe700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34c101fe0 0x7ff34c073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff351dbe700 1 -- 172.21.15.12:0/257272885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c0095f0 con 0x7ff34c100e90 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff351dbe700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c100e90 0x7ff34c074d00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff34800d950 tx=0x7ff34800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff342ffd700 1 -- 172.21.15.12:0/257272885 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3480098e0 con 0x7ff34c100e90 2024-05-08T23:45:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff354022700 1 -- 172.21.15.12:0/257272885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34c1a10b0 con 0x7ff34c100e90 2024-05-08T23:45:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff342ffd700 1 -- 172.21.15.12:0/257272885 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff348010460 con 0x7ff34c100e90 2024-05-08T23:45:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff342ffd700 1 -- 172.21.15.12:0/257272885 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34800f620 con 0x7ff34c100e90 2024-05-08T23:45:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.259+0000 7ff354022700 1 -- 172.21.15.12:0/257272885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34c1a15c0 con 0x7ff34c100e90 2024-05-08T23:45:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.261+0000 7ff354022700 1 -- 172.21.15.12:0/257272885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34c108230 con 0x7ff34c100e90 2024-05-08T23:45:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.262+0000 7ff342ffd700 1 -- 172.21.15.12:0/257272885 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff3480108d0 con 0x7ff34c100e90 2024-05-08T23:45:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.262+0000 7ff342ffd700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff3380a11b0 0x7ff3380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.262+0000 7ff342ffd700 1 -- 172.21.15.12:0/257272885 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3480cc810 con 0x7ff34c100e90 2024-05-08T23:45:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.263+0000 7ff3515bd700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff3380a11b0 0x7ff3380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.263+0000 7ff3515bd700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff3380a11b0 0x7ff3380a3670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0095c0 tx=0x7ff33c009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:12.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.266+0000 7ff342ffd700 1 -- 172.21.15.12:0/257272885 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff348095ef0 con 0x7ff34c100e90 2024-05-08T23:45:12.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:12 smithi012 ceph-mon[42503]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:12.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:12 smithi012 ceph-mon[42503]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T23:45:12.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:12 smithi012 ceph-mon[42503]: pgmap v229: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.8 KiB/s wr, 15 op/s 2024-05-08T23:45:12.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:12 smithi012 ceph-mon[42503]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:12.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:12 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/530328957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.481+0000 7ff354022700 1 -- 172.21.15.12:0/257272885 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff34c02ce10 con 0x7ff3380a11b0 2024-05-08T23:45:12.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.493+0000 7ff342ffd700 1 -- 172.21.15.12:0/257272885 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+328108 (secure 0 0 0) 0x7ff34c02ce10 con 0x7ff3380a11b0 2024-05-08T23:45:12.493 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:12.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 -- 172.21.15.12:0/257272885 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff3380a11b0 msgr2=0x7ff3380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff3380a11b0 0x7ff3380a3670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0095c0 tx=0x7ff33c009550 comp rx=0 tx=0).stop 2024-05-08T23:45:12.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 -- 172.21.15.12:0/257272885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c100e90 msgr2=0x7ff34c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c100e90 0x7ff34c074d00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff34800d950 tx=0x7ff34800dc60 comp rx=0 tx=0).stop 2024-05-08T23:45:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 -- 172.21.15.12:0/257272885 shutdown_connections 2024-05-08T23:45:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34c1029a0 0x7ff34c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c100e90 0x7ff34c074d00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff3380a11b0 0x7ff3380a3670 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 --2- 172.21.15.12:0/257272885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34c101fe0 0x7ff34c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 -- 172.21.15.12:0/257272885 >> 172.21.15.12:0/257272885 conn(0x7ff34c0fc400 msgr2=0x7ff34c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 -- 172.21.15.12:0/257272885 shutdown_connections 2024-05-08T23:45:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.497+0000 7ff354022700 1 -- 172.21.15.12:0/257272885 wait complete. 2024-05-08T23:45:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-08T23:45:12.506 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-05-08T23:45:11.253291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594864,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":148,"num_read_kb":129,"num_write":288,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":266,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373160,"kb_used_data":12064,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124568664,"statfs":{"total":1151940427776,"available":1151558311936,"internally_reserved":0,"allocated":12353536,"data_stored":6803012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4496,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":107,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.692496"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040868+0000","last_change":"2024-05-08T23:45:04.044088+0000","last_active":"2024-05-08T23:45:05.040868+0000","last_peered":"2024-05-08T23:45:05.040868+0000","last_clean":"2024-05-08T23:45:05.040868+0000","last_became_active":"2024-05-08T23:45:04.043535+0000","last_became_peered":"2024-05-08T23:45:04.043535+0000","last_unstale":"2024-05-08T23:45:05.040868+0000","last_undegraded":"2024-05-08T23:45:05.040868+0000","last_fullsized":"2024-05-08T23:45:05.040868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:52.363402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282575+0000","last_change":"2024-05-08T23:45:04.049201+0000","last_active":"2024-05-08T23:45:06.282575+0000","last_peered":"2024-05-08T23:45:06.282575+0000","last_clean":"2024-05-08T23:45:06.282575+0000","last_became_active":"2024-05-08T23:45:04.048743+0000","last_became_peered":"2024-05-08T23:45:04.048743+0000","last_unstale":"2024-05-08T23:45:06.282575+0000","last_undegraded":"2024-05-08T23:45:06.282575+0000","last_fullsized":"2024-05-08T23:45:06.282575+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:26.191549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282032+0000","last_change":"2024-05-08T23:45:04.047356+0000","last_active":"2024-05-08T23:45:06.282032+0000","last_peered":"2024-05-08T23:45:06.282032+0000","last_clean":"2024-05-08T23:45:06.282032+0000","last_became_active":"2024-05-08T23:45:04.046996+0000","last_became_peered":"2024-05-08T23:45:04.046996+0000","last_unstale":"2024-05-08T23:45:06.282032+0000","last_undegraded":"2024-05-08T23:45:06.282032+0000","last_fullsized":"2024-05-08T23:45:06.282032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:58.722522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282798+0000","last_change":"2024-05-08T23:45:04.054754+0000","last_active":"2024-05-08T23:45:06.282798+0000","last_peered":"2024-05-08T23:45:06.282798+0000","last_clean":"2024-05-08T23:45:06.282798+0000","last_became_active":"2024-05-08T23:45:04.054302+0000","last_became_peered":"2024-05-08T23:45:04.054302+0000","last_unstale":"2024-05-08T23:45:06.282798+0000","last_undegraded":"2024-05-08T23:45:06.282798+0000","last_fullsized":"2024-05-08T23:45:06.282798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:28.882982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517234+0000","last_change":"2024-05-08T23:45:04.047189+0000","last_active":"2024-05-08T23:45:05.517234+0000","last_peered":"2024-05-08T23:45:05.517234+0000","last_clean":"2024-05-08T23:45:05.517234+0000","last_became_active":"2024-05-08T23:45:04.046756+0000","last_became_peered":"2024-05-08T23:45:04.046756+0000","last_unstale":"2024-05-08T23:45:05.517234+0000","last_undegraded":"2024-05-08T23:45:05.517234+0000","last_fullsized":"2024-05-08T23:45:05.517234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.780169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045028+0000","last_change":"2024-05-08T23:45:04.060897+0000","last_active":"2024-05-08T23:45:05.045028+0000","last_peered":"2024-05-08T23:45:05.045028+0000","last_clean":"2024-05-08T23:45:05.045028+0000","last_became_active":"2024-05-08T23:45:04.060550+0000","last_became_peered":"2024-05-08T23:45:04.060550+0000","last_unstale":"2024-05-08T23:45:05.045028+0000","last_undegraded":"2024-05-08T23:45:05.045028+0000","last_fullsized":"2024-05-08T23:45:05.045028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:59.356057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781043+0000","last_change":"2024-05-08T23:45:04.059308+0000","last_active":"2024-05-08T23:45:06.781043+0000","last_peered":"2024-05-08T23:45:06.781043+0000","last_clean":"2024-05-08T23:45:06.781043+0000","last_became_active":"2024-05-08T23:45:04.058920+0000","last_became_peered":"2024-05-08T23:45:04.058920+0000","last_unstale":"2024-05-08T23:45:06.781043+0000","last_undegraded":"2024-05-08T23:45:06.781043+0000","last_fullsized":"2024-05-08T23:45:06.781043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:11.857062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039706+0000","last_change":"2024-05-08T23:45:04.061542+0000","last_active":"2024-05-08T23:45:05.039706+0000","last_peered":"2024-05-08T23:45:05.039706+0000","last_clean":"2024-05-08T23:45:05.039706+0000","last_became_active":"2024-05-08T23:45:04.061145+0000","last_became_peered":"2024-05-08T23:45:04.061145+0000","last_unstale":"2024-05-08T23:45:05.039706+0000","last_undegraded":"2024-05-08T23:45:05.039706+0000","last_fullsized":"2024-05-08T23:45:05.039706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:53.159292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300493+0000","last_change":"2024-05-08T23:45:04.052292+0000","last_active":"2024-05-08T23:45:04.300493+0000","last_peered":"2024-05-08T23:45:04.300493+0000","last_clean":"2024-05-08T23:45:04.300493+0000","last_became_active":"2024-05-08T23:45:04.051164+0000","last_became_peered":"2024-05-08T23:45:04.051164+0000","last_unstale":"2024-05-08T23:45:04.300493+0000","last_undegraded":"2024-05-08T23:45:04.300493+0000","last_fullsized":"2024-05-08T23:45:04.300493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:09.218130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045014+0000","last_change":"2024-05-08T23:45:04.054566+0000","last_active":"2024-05-08T23:45:05.045014+0000","last_peered":"2024-05-08T23:45:05.045014+0000","last_clean":"2024-05-08T23:45:05.045014+0000","last_became_active":"2024-05-08T23:45:04.053878+0000","last_became_peered":"2024-05-08T23:45:04.053878+0000","last_unstale":"2024-05-08T23:45:05.045014+0000","last_undegraded":"2024-05-08T23:45:05.045014+0000","last_fullsized":"2024-05-08T23:45:05.045014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:34.760038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781480+0000","last_change":"2024-05-08T23:45:04.049696+0000","last_active":"2024-05-08T23:45:06.781480+0000","last_peered":"2024-05-08T23:45:06.781480+0000","last_clean":"2024-05-08T23:45:06.781480+0000","last_became_active":"2024-05-08T23:45:04.049351+0000","last_became_peered":"2024-05-08T23:45:04.049351+0000","last_unstale":"2024-05-08T23:45:06.781480+0000","last_undegraded":"2024-05-08T23:45:06.781480+0000","last_fullsized":"2024-05-08T23:45:06.781480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:53.557738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282779+0000","last_change":"2024-05-08T23:45:04.051010+0000","last_active":"2024-05-08T23:45:06.282779+0000","last_peered":"2024-05-08T23:45:06.282779+0000","last_clean":"2024-05-08T23:45:06.282779+0000","last_became_active":"2024-05-08T23:45:04.050715+0000","last_became_peered":"2024-05-08T23:45:04.050715+0000","last_unstale":"2024-05-08T23:45:06.282779+0000","last_undegraded":"2024-05-08T23:45:06.282779+0000","last_fullsized":"2024-05-08T23:45:06.282779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:09.305974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022192+0000","last_change":"2024-05-08T23:45:04.051005+0000","last_active":"2024-05-08T23:45:05.022192+0000","last_peered":"2024-05-08T23:45:05.022192+0000","last_clean":"2024-05-08T23:45:05.022192+0000","last_became_active":"2024-05-08T23:45:04.050123+0000","last_became_peered":"2024-05-08T23:45:04.050123+0000","last_unstale":"2024-05-08T23:45:05.022192+0000","last_undegraded":"2024-05-08T23:45:05.022192+0000","last_fullsized":"2024-05-08T23:45:05.022192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:52.206389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282033+0000","last_change":"2024-05-08T23:45:04.049942+0000","last_active":"2024-05-08T23:45:06.282033+0000","last_peered":"2024-05-08T23:45:06.282033+0000","last_clean":"2024-05-08T23:45:06.282033+0000","last_became_active":"2024-05-08T23:45:04.049596+0000","last_became_peered":"2024-05-08T23:45:04.049596+0000","last_unstale":"2024-05-08T23:45:06.282033+0000","last_undegraded":"2024-05-08T23:45:06.282033+0000","last_fullsized":"2024-05-08T23:45:06.282033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:30.907023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282203+0000","last_change":"2024-05-08T23:45:04.048985+0000","last_active":"2024-05-08T23:45:06.282203+0000","last_peered":"2024-05-08T23:45:06.282203+0000","last_clean":"2024-05-08T23:45:06.282203+0000","last_became_active":"2024-05-08T23:45:04.048658+0000","last_became_peered":"2024-05-08T23:45:04.048658+0000","last_unstale":"2024-05-08T23:45:06.282203+0000","last_undegraded":"2024-05-08T23:45:06.282203+0000","last_fullsized":"2024-05-08T23:45:06.282203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:17.030930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300415+0000","last_change":"2024-05-08T23:45:04.051903+0000","last_active":"2024-05-08T23:45:04.300415+0000","last_peered":"2024-05-08T23:45:04.300415+0000","last_clean":"2024-05-08T23:45:04.300415+0000","last_became_active":"2024-05-08T23:45:04.051221+0000","last_became_peered":"2024-05-08T23:45:04.051221+0000","last_unstale":"2024-05-08T23:45:04.300415+0000","last_undegraded":"2024-05-08T23:45:04.300415+0000","last_fullsized":"2024-05-08T23:45:04.300415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:01.554463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040970+0000","last_change":"2024-05-08T23:45:04.043854+0000","last_active":"2024-05-08T23:45:05.040970+0000","last_peered":"2024-05-08T23:45:05.040970+0000","last_clean":"2024-05-08T23:45:05.040970+0000","last_became_active":"2024-05-08T23:45:04.043056+0000","last_became_peered":"2024-05-08T23:45:04.043056+0000","last_unstale":"2024-05-08T23:45:05.040970+0000","last_undegraded":"2024-05-08T23:45:05.040970+0000","last_fullsized":"2024-05-08T23:45:05.040970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:18.175575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.058883+0000","last_change":"2024-05-08T23:45:04.049332+0000","last_active":"2024-05-08T23:45:04.058883+0000","last_peered":"2024-05-08T23:45:04.058883+0000","last_clean":"2024-05-08T23:45:04.058883+0000","last_became_active":"2024-05-08T23:45:04.048987+0000","last_became_peered":"2024-05-08T23:45:04.048987+0000","last_unstale":"2024-05-08T23:45:04.058883+0000","last_undegraded":"2024-05-08T23:45:04.058883+0000","last_fullsized":"2024-05-08T23:45:04.058883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:02.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225413+0000","last_change":"2024-05-08T23:45:04.048536+0000","last_active":"2024-05-08T23:45:06.225413+0000","last_peered":"2024-05-08T23:45:06.225413+0000","last_clean":"2024-05-08T23:45:06.225413+0000","last_became_active":"2024-05-08T23:45:04.048102+0000","last_became_peered":"2024-05-08T23:45:04.048102+0000","last_unstale":"2024-05-08T23:45:06.225413+0000","last_undegraded":"2024-05-08T23:45:06.225413+0000","last_fullsized":"2024-05-08T23:45:06.225413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:11.794372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.070828+0000","last_change":"2024-05-08T23:45:03.576833+0000","last_active":"2024-05-08T23:45:08.070828+0000","last_peered":"2024-05-08T23:45:08.070828+0000","last_clean":"2024-05-08T23:45:08.070828+0000","last_became_active":"2024-05-08T23:45:03.576495+0000","last_became_peered":"2024-05-08T23:45:03.576495+0000","last_unstale":"2024-05-08T23:45:08.070828+0000","last_undegraded":"2024-05-08T23:45:08.070828+0000","last_fullsized":"2024-05-08T23:45:08.070828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:50.266658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040105+0000","last_change":"2024-05-08T23:45:04.055444+0000","last_active":"2024-05-08T23:45:05.040105+0000","last_peered":"2024-05-08T23:45:05.040105+0000","last_clean":"2024-05-08T23:45:05.040105+0000","last_became_active":"2024-05-08T23:45:04.054998+0000","last_became_peered":"2024-05-08T23:45:04.054998+0000","last_unstale":"2024-05-08T23:45:05.040105+0000","last_undegraded":"2024-05-08T23:45:05.040105+0000","last_fullsized":"2024-05-08T23:45:05.040105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:03.470555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.100638+0000","last_change":"2024-05-08T23:45:03.578362+0000","last_active":"2024-05-08T23:45:05.100638+0000","last_peered":"2024-05-08T23:45:05.100638+0000","last_clean":"2024-05-08T23:45:05.100638+0000","last_became_active":"2024-05-08T23:45:03.577834+0000","last_became_peered":"2024-05-08T23:45:03.577834+0000","last_unstale":"2024-05-08T23:45:05.100638+0000","last_undegraded":"2024-05-08T23:45:05.100638+0000","last_fullsized":"2024-05-08T23:45:05.100638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:01.943490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282763+0000","last_change":"2024-05-08T23:45:04.047205+0000","last_active":"2024-05-08T23:45:06.282763+0000","last_peered":"2024-05-08T23:45:06.282763+0000","last_clean":"2024-05-08T23:45:06.282763+0000","last_became_active":"2024-05-08T23:45:04.046860+0000","last_became_peered":"2024-05-08T23:45:04.046860+0000","last_unstale":"2024-05-08T23:45:06.282763+0000","last_undegraded":"2024-05-08T23:45:06.282763+0000","last_fullsized":"2024-05-08T23:45:06.282763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:36.837101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.069768+0000","last_change":"2024-05-08T23:45:03.582026+0000","last_active":"2024-05-08T23:45:08.069768+0000","last_peered":"2024-05-08T23:45:08.069768+0000","last_clean":"2024-05-08T23:45:08.069768+0000","last_became_active":"2024-05-08T23:45:03.581672+0000","last_became_peered":"2024-05-08T23:45:03.581672+0000","last_unstale":"2024-05-08T23:45:08.069768+0000","last_undegraded":"2024-05-08T23:45:08.069768+0000","last_fullsized":"2024-05-08T23:45:08.069768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:50.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300470+0000","last_change":"2024-05-08T23:45:04.052639+0000","last_active":"2024-05-08T23:45:04.300470+0000","last_peered":"2024-05-08T23:45:04.300470+0000","last_clean":"2024-05-08T23:45:04.300470+0000","last_became_active":"2024-05-08T23:45:04.052158+0000","last_became_peered":"2024-05-08T23:45:04.052158+0000","last_unstale":"2024-05-08T23:45:04.300470+0000","last_undegraded":"2024-05-08T23:45:04.300470+0000","last_fullsized":"2024-05-08T23:45:04.300470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:25.638555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517264+0000","last_change":"2024-05-08T23:45:03.579221+0000","last_active":"2024-05-08T23:45:05.517264+0000","last_peered":"2024-05-08T23:45:05.517264+0000","last_clean":"2024-05-08T23:45:05.517264+0000","last_became_active":"2024-05-08T23:45:03.578795+0000","last_became_peered":"2024-05-08T23:45:03.578795+0000","last_unstale":"2024-05-08T23:45:05.517264+0000","last_undegraded":"2024-05-08T23:45:05.517264+0000","last_fullsized":"2024-05-08T23:45:05.517264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:14.064108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563925+0000","last_change":"2024-05-08T23:45:04.057573+0000","last_active":"2024-05-08T23:45:05.563925+0000","last_peered":"2024-05-08T23:45:05.563925+0000","last_clean":"2024-05-08T23:45:05.563925+0000","last_became_active":"2024-05-08T23:45:04.053201+0000","last_became_peered":"2024-05-08T23:45:04.053201+0000","last_unstale":"2024-05-08T23:45:05.563925+0000","last_undegraded":"2024-05-08T23:45:05.563925+0000","last_fullsized":"2024-05-08T23:45:05.563925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:23.612609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:09.076983+0000","last_change":"2024-05-08T23:45:03.578110+0000","last_active":"2024-05-08T23:45:09.076983+0000","last_peered":"2024-05-08T23:45:09.076983+0000","last_clean":"2024-05-08T23:45:09.076983+0000","last_became_active":"2024-05-08T23:45:03.577749+0000","last_became_peered":"2024-05-08T23:45:03.577749+0000","last_unstale":"2024-05-08T23:45:09.076983+0000","last_undegraded":"2024-05-08T23:45:09.076983+0000","last_fullsized":"2024-05-08T23:45:09.076983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:50.526535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517216+0000","last_change":"2024-05-08T23:45:04.048131+0000","last_active":"2024-05-08T23:45:05.517216+0000","last_peered":"2024-05-08T23:45:05.517216+0000","last_clean":"2024-05-08T23:45:05.517216+0000","last_became_active":"2024-05-08T23:45:04.047773+0000","last_became_peered":"2024-05-08T23:45:04.047773+0000","last_unstale":"2024-05-08T23:45:05.517216+0000","last_undegraded":"2024-05-08T23:45:05.517216+0000","last_fullsized":"2024-05-08T23:45:05.517216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:47.944900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.070447+0000","last_change":"2024-05-08T23:45:03.586710+0000","last_active":"2024-05-08T23:45:08.070447+0000","last_peered":"2024-05-08T23:45:08.070447+0000","last_clean":"2024-05-08T23:45:08.070447+0000","last_became_active":"2024-05-08T23:45:03.586411+0000","last_became_peered":"2024-05-08T23:45:03.586411+0000","last_unstale":"2024-05-08T23:45:08.070447+0000","last_undegraded":"2024-05-08T23:45:08.070447+0000","last_fullsized":"2024-05-08T23:45:08.070447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:25.938846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563896+0000","last_change":"2024-05-08T23:45:04.053885+0000","last_active":"2024-05-08T23:45:05.563896+0000","last_peered":"2024-05-08T23:45:05.563896+0000","last_clean":"2024-05-08T23:45:05.563896+0000","last_became_active":"2024-05-08T23:45:04.053385+0000","last_became_peered":"2024-05-08T23:45:04.053385+0000","last_unstale":"2024-05-08T23:45:05.563896+0000","last_undegraded":"2024-05-08T23:45:05.563896+0000","last_fullsized":"2024-05-08T23:45:05.563896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:10.805481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.071417+0000","last_change":"2024-05-08T23:45:03.574988+0000","last_active":"2024-05-08T23:45:08.071417+0000","last_peered":"2024-05-08T23:45:08.071417+0000","last_clean":"2024-05-08T23:45:08.071417+0000","last_became_active":"2024-05-08T23:45:03.574586+0000","last_became_peered":"2024-05-08T23:45:03.574586+0000","last_unstale":"2024-05-08T23:45:08.071417+0000","last_undegraded":"2024-05-08T23:45:08.071417+0000","last_fullsized":"2024-05-08T23:45:08.071417+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:18.005107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039815+0000","last_change":"2024-05-08T23:45:04.046817+0000","last_active":"2024-05-08T23:45:05.039815+0000","last_peered":"2024-05-08T23:45:05.039815+0000","last_clean":"2024-05-08T23:45:05.039815+0000","last_became_active":"2024-05-08T23:45:04.046515+0000","last_became_peered":"2024-05-08T23:45:04.046515+0000","last_unstale":"2024-05-08T23:45:05.039815+0000","last_undegraded":"2024-05-08T23:45:05.039815+0000","last_fullsized":"2024-05-08T23:45:05.039815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:29.439670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282372+0000","last_change":"2024-05-08T23:45:03.579002+0000","last_active":"2024-05-08T23:45:06.282372+0000","last_peered":"2024-05-08T23:45:06.282372+0000","last_clean":"2024-05-08T23:45:06.282372+0000","last_became_active":"2024-05-08T23:45:03.577891+0000","last_became_peered":"2024-05-08T23:45:03.577891+0000","last_unstale":"2024-05-08T23:45:06.282372+0000","last_undegraded":"2024-05-08T23:45:06.282372+0000","last_fullsized":"2024-05-08T23:45:06.282372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:01.027574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.061375+0000","last_change":"2024-05-08T23:45:04.060715+0000","last_active":"2024-05-08T23:45:04.061375+0000","last_peered":"2024-05-08T23:45:04.061375+0000","last_clean":"2024-05-08T23:45:04.061375+0000","last_became_active":"2024-05-08T23:45:04.060375+0000","last_became_peered":"2024-05-08T23:45:04.060375+0000","last_unstale":"2024-05-08T23:45:04.061375+0000","last_undegraded":"2024-05-08T23:45:04.061375+0000","last_fullsized":"2024-05-08T23:45:04.061375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:42.133323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061410+0000","last_change":"2024-05-08T23:45:03.587784+0000","last_active":"2024-05-08T23:45:07.061410+0000","last_peered":"2024-05-08T23:45:07.061410+0000","last_clean":"2024-05-08T23:45:07.061410+0000","last_became_active":"2024-05-08T23:45:03.587529+0000","last_became_peered":"2024-05-08T23:45:03.587529+0000","last_unstale":"2024-05-08T23:45:07.061410+0000","last_undegraded":"2024-05-08T23:45:07.061410+0000","last_fullsized":"2024-05-08T23:45:07.061410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:00.323599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300481+0000","last_change":"2024-05-08T23:45:04.062672+0000","last_active":"2024-05-08T23:45:04.300481+0000","last_peered":"2024-05-08T23:45:04.300481+0000","last_clean":"2024-05-08T23:45:04.300481+0000","last_became_active":"2024-05-08T23:45:04.062351+0000","last_became_peered":"2024-05-08T23:45:04.062351+0000","last_unstale":"2024-05-08T23:45:04.300481+0000","last_undegraded":"2024-05-08T23:45:04.300481+0000","last_fullsized":"2024-05-08T23:45:04.300481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:13.740239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781350+0000","last_change":"2024-05-08T23:45:03.583501+0000","last_active":"2024-05-08T23:45:06.781350+0000","last_peered":"2024-05-08T23:45:06.781350+0000","last_clean":"2024-05-08T23:45:06.781350+0000","last_became_active":"2024-05-08T23:45:03.583116+0000","last_became_peered":"2024-05-08T23:45:03.583116+0000","last_unstale":"2024-05-08T23:45:06.781350+0000","last_undegraded":"2024-05-08T23:45:06.781350+0000","last_fullsized":"2024-05-08T23:45:06.781350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:03.797524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282765+0000","last_change":"2024-05-08T23:45:04.048336+0000","last_active":"2024-05-08T23:45:06.282765+0000","last_peered":"2024-05-08T23:45:06.282765+0000","last_clean":"2024-05-08T23:45:06.282765+0000","last_became_active":"2024-05-08T23:45:04.048000+0000","last_became_peered":"2024-05-08T23:45:04.048000+0000","last_unstale":"2024-05-08T23:45:06.282765+0000","last_undegraded":"2024-05-08T23:45:06.282765+0000","last_fullsized":"2024-05-08T23:45:06.282765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:41.806600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062204+0000","last_change":"2024-05-08T23:45:03.581215+0000","last_active":"2024-05-08T23:45:07.062204+0000","last_peered":"2024-05-08T23:45:07.062204+0000","last_clean":"2024-05-08T23:45:07.062204+0000","last_became_active":"2024-05-08T23:45:03.580496+0000","last_became_peered":"2024-05-08T23:45:03.580496+0000","last_unstale":"2024-05-08T23:45:07.062204+0000","last_undegraded":"2024-05-08T23:45:07.062204+0000","last_fullsized":"2024-05-08T23:45:07.062204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:25.602401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300459+0000","last_change":"2024-05-08T23:45:04.058857+0000","last_active":"2024-05-08T23:45:04.300459+0000","last_peered":"2024-05-08T23:45:04.300459+0000","last_clean":"2024-05-08T23:45:04.300459+0000","last_became_active":"2024-05-08T23:45:04.057988+0000","last_became_peered":"2024-05-08T23:45:04.057988+0000","last_unstale":"2024-05-08T23:45:04.300459+0000","last_undegraded":"2024-05-08T23:45:04.300459+0000","last_fullsized":"2024-05-08T23:45:04.300459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:46.725976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'8","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061189+0000","last_change":"2024-05-08T23:45:03.582121+0000","last_active":"2024-05-08T23:45:07.061189+0000","last_peered":"2024-05-08T23:45:07.061189+0000","last_clean":"2024-05-08T23:45:07.061189+0000","last_became_active":"2024-05-08T23:45:03.581796+0000","last_became_peered":"2024-05-08T23:45:03.581796+0000","last_unstale":"2024-05-08T23:45:07.061189+0000","last_undegraded":"2024-05-08T23:45:07.061189+0000","last_fullsized":"2024-05-08T23:45:07.061189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:55.417404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517284+0000","last_change":"2024-05-08T23:45:04.052993+0000","last_active":"2024-05-08T23:45:05.517284+0000","last_peered":"2024-05-08T23:45:05.517284+0000","last_clean":"2024-05-08T23:45:05.517284+0000","last_became_active":"2024-05-08T23:45:04.052088+0000","last_became_peered":"2024-05-08T23:45:04.052088+0000","last_unstale":"2024-05-08T23:45:05.517284+0000","last_undegraded":"2024-05-08T23:45:05.517284+0000","last_fullsized":"2024-05-08T23:45:05.517284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:44.531649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225314+0000","last_change":"2024-05-08T23:45:03.575311+0000","last_active":"2024-05-08T23:45:06.225314+0000","last_peered":"2024-05-08T23:45:06.225314+0000","last_clean":"2024-05-08T23:45:06.225314+0000","last_became_active":"2024-05-08T23:45:03.574949+0000","last_became_peered":"2024-05-08T23:45:03.574949+0000","last_unstale":"2024-05-08T23:45:06.225314+0000","last_undegraded":"2024-05-08T23:45:06.225314+0000","last_fullsized":"2024-05-08T23:45:06.225314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:18.129488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022182+0000","last_change":"2024-05-08T23:45:04.050294+0000","last_active":"2024-05-08T23:45:05.022182+0000","last_peered":"2024-05-08T23:45:05.022182+0000","last_clean":"2024-05-08T23:45:05.022182+0000","last_became_active":"2024-05-08T23:45:04.049949+0000","last_became_peered":"2024-05-08T23:45:04.049949+0000","last_unstale":"2024-05-08T23:45:05.022182+0000","last_undegraded":"2024-05-08T23:45:05.022182+0000","last_fullsized":"2024-05-08T23:45:05.022182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:17.253201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061055+0000","last_change":"2024-05-08T23:45:03.579821+0000","last_active":"2024-05-08T23:45:07.061055+0000","last_peered":"2024-05-08T23:45:07.061055+0000","last_clean":"2024-05-08T23:45:07.061055+0000","last_became_active":"2024-05-08T23:45:03.578635+0000","last_became_peered":"2024-05-08T23:45:03.578635+0000","last_unstale":"2024-05-08T23:45:07.061055+0000","last_undegraded":"2024-05-08T23:45:07.061055+0000","last_fullsized":"2024-05-08T23:45:07.061055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:46.179505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781264+0000","last_change":"2024-05-08T23:45:04.059357+0000","last_active":"2024-05-08T23:45:06.781264+0000","last_peered":"2024-05-08T23:45:06.781264+0000","last_clean":"2024-05-08T23:45:06.781264+0000","last_became_active":"2024-05-08T23:45:04.059070+0000","last_became_peered":"2024-05-08T23:45:04.059070+0000","last_unstale":"2024-05-08T23:45:06.781264+0000","last_undegraded":"2024-05-08T23:45:06.781264+0000","last_fullsized":"2024-05-08T23:45:06.781264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:50.626787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.069255+0000","last_change":"2024-05-08T23:45:03.577199+0000","last_active":"2024-05-08T23:45:08.069255+0000","last_peered":"2024-05-08T23:45:08.069255+0000","last_clean":"2024-05-08T23:45:08.069255+0000","last_became_active":"2024-05-08T23:45:03.576844+0000","last_became_peered":"2024-05-08T23:45:03.576844+0000","last_unstale":"2024-05-08T23:45:08.069255+0000","last_undegraded":"2024-05-08T23:45:08.069255+0000","last_fullsized":"2024-05-08T23:45:08.069255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:27.139493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.780964+0000","last_change":"2024-05-08T23:45:04.064515+0000","last_active":"2024-05-08T23:45:06.780964+0000","last_peered":"2024-05-08T23:45:06.780964+0000","last_clean":"2024-05-08T23:45:06.780964+0000","last_became_active":"2024-05-08T23:45:04.064184+0000","last_became_peered":"2024-05-08T23:45:04.064184+0000","last_unstale":"2024-05-08T23:45:06.780964+0000","last_undegraded":"2024-05-08T23:45:06.780964+0000","last_fullsized":"2024-05-08T23:45:06.780964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:46.423121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.072701+0000","last_change":"2024-05-08T23:45:03.576766+0000","last_active":"2024-05-08T23:45:08.072701+0000","last_peered":"2024-05-08T23:45:08.072701+0000","last_clean":"2024-05-08T23:45:08.072701+0000","last_became_active":"2024-05-08T23:45:03.576410+0000","last_became_peered":"2024-05-08T23:45:03.576410+0000","last_unstale":"2024-05-08T23:45:08.072701+0000","last_undegraded":"2024-05-08T23:45:08.072701+0000","last_fullsized":"2024-05-08T23:45:08.072701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:29.444589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563841+0000","last_change":"2024-05-08T23:45:04.056563+0000","last_active":"2024-05-08T23:45:05.563841+0000","last_peered":"2024-05-08T23:45:05.563841+0000","last_clean":"2024-05-08T23:45:05.563841+0000","last_became_active":"2024-05-08T23:45:04.053132+0000","last_became_peered":"2024-05-08T23:45:04.053132+0000","last_unstale":"2024-05-08T23:45:05.563841+0000","last_undegraded":"2024-05-08T23:45:05.563841+0000","last_fullsized":"2024-05-08T23:45:05.563841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:41.625804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.283481+0000","last_change":"2024-05-08T23:45:03.580343+0000","last_active":"2024-05-08T23:45:06.283481+0000","last_peered":"2024-05-08T23:45:06.283481+0000","last_clean":"2024-05-08T23:45:06.283481+0000","last_became_active":"2024-05-08T23:45:03.578719+0000","last_became_peered":"2024-05-08T23:45:03.578719+0000","last_unstale":"2024-05-08T23:45:06.283481+0000","last_undegraded":"2024-05-08T23:45:06.283481+0000","last_fullsized":"2024-05-08T23:45:06.283481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:54.831252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040662+0000","last_change":"2024-05-08T23:45:04.050643+0000","last_active":"2024-05-08T23:45:05.040662+0000","last_peered":"2024-05-08T23:45:05.040662+0000","last_clean":"2024-05-08T23:45:05.040662+0000","last_became_active":"2024-05-08T23:45:04.050309+0000","last_became_peered":"2024-05-08T23:45:04.050309+0000","last_unstale":"2024-05-08T23:45:05.040662+0000","last_undegraded":"2024-05-08T23:45:05.040662+0000","last_fullsized":"2024-05-08T23:45:05.040662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:30.259530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.036233+0000","last_change":"2024-05-08T23:45:03.579590+0000","last_active":"2024-05-08T23:45:04.036233+0000","last_peered":"2024-05-08T23:45:04.036233+0000","last_clean":"2024-05-08T23:45:04.036233+0000","last_became_active":"2024-05-08T23:45:03.579213+0000","last_became_peered":"2024-05-08T23:45:03.579213+0000","last_unstale":"2024-05-08T23:45:04.036233+0000","last_undegraded":"2024-05-08T23:45:04.036233+0000","last_fullsized":"2024-05-08T23:45:04.036233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:55.666543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.044845+0000","last_change":"2024-05-08T23:45:04.049082+0000","last_active":"2024-05-08T23:45:05.044845+0000","last_peered":"2024-05-08T23:45:05.044845+0000","last_clean":"2024-05-08T23:45:05.044845+0000","last_became_active":"2024-05-08T23:45:04.048709+0000","last_became_peered":"2024-05-08T23:45:04.048709+0000","last_unstale":"2024-05-08T23:45:05.044845+0000","last_undegraded":"2024-05-08T23:45:05.044845+0000","last_fullsized":"2024-05-08T23:45:05.044845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:56.365817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781150+0000","last_change":"2024-05-08T23:45:03.581363+0000","last_active":"2024-05-08T23:45:06.781150+0000","last_peered":"2024-05-08T23:45:06.781150+0000","last_clean":"2024-05-08T23:45:06.781150+0000","last_became_active":"2024-05-08T23:45:03.580840+0000","last_became_peered":"2024-05-08T23:45:03.580840+0000","last_unstale":"2024-05-08T23:45:06.781150+0000","last_undegraded":"2024-05-08T23:45:06.781150+0000","last_fullsized":"2024-05-08T23:45:06.781150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:27.281228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045138+0000","last_change":"2024-05-08T23:45:04.053524+0000","last_active":"2024-05-08T23:45:05.045138+0000","last_peered":"2024-05-08T23:45:05.045138+0000","last_clean":"2024-05-08T23:45:05.045138+0000","last_became_active":"2024-05-08T23:45:04.053118+0000","last_became_peered":"2024-05-08T23:45:04.053118+0000","last_unstale":"2024-05-08T23:45:05.045138+0000","last_undegraded":"2024-05-08T23:45:05.045138+0000","last_fullsized":"2024-05-08T23:45:05.045138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:56.542262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563822+0000","last_change":"2024-05-08T23:45:04.061189+0000","last_active":"2024-05-08T23:45:05.563822+0000","last_peered":"2024-05-08T23:45:05.563822+0000","last_clean":"2024-05-08T23:45:05.563822+0000","last_became_active":"2024-05-08T23:45:04.060838+0000","last_became_peered":"2024-05-08T23:45:04.060838+0000","last_unstale":"2024-05-08T23:45:05.563822+0000","last_undegraded":"2024-05-08T23:45:05.563822+0000","last_fullsized":"2024-05-08T23:45:05.563822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:43.794987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.073509+0000","last_change":"2024-05-08T23:45:03.583841+0000","last_active":"2024-05-08T23:45:08.073509+0000","last_peered":"2024-05-08T23:45:08.073509+0000","last_clean":"2024-05-08T23:45:08.073509+0000","last_became_active":"2024-05-08T23:45:03.583275+0000","last_became_peered":"2024-05-08T23:45:03.583275+0000","last_unstale":"2024-05-08T23:45:08.073509+0000","last_undegraded":"2024-05-08T23:45:08.073509+0000","last_fullsized":"2024-05-08T23:45:08.073509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:14.560505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039949+0000","last_change":"2024-05-08T23:45:04.052667+0000","last_active":"2024-05-08T23:45:05.039949+0000","last_peered":"2024-05-08T23:45:05.039949+0000","last_clean":"2024-05-08T23:45:05.039949+0000","last_became_active":"2024-05-08T23:45:04.052008+0000","last_became_peered":"2024-05-08T23:45:04.052008+0000","last_unstale":"2024-05-08T23:45:05.039949+0000","last_undegraded":"2024-05-08T23:45:05.039949+0000","last_fullsized":"2024-05-08T23:45:05.039949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:35.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062503+0000","last_change":"2024-05-08T23:45:03.573665+0000","last_active":"2024-05-08T23:45:07.062503+0000","last_peered":"2024-05-08T23:45:07.062503+0000","last_clean":"2024-05-08T23:45:07.062503+0000","last_became_active":"2024-05-08T23:45:03.573265+0000","last_became_peered":"2024-05-08T23:45:03.573265+0000","last_unstale":"2024-05-08T23:45:07.062503+0000","last_undegraded":"2024-05-08T23:45:07.062503+0000","last_fullsized":"2024-05-08T23:45:07.062503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:13.410954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":329,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.038848+0000","last_change":"2024-05-08T23:44:16.373962+0000","last_active":"2024-05-08T23:45:04.038848+0000","last_peered":"2024-05-08T23:45:04.038848+0000","last_clean":"2024-05-08T23:45:04.038848+0000","last_became_active":"2024-05-08T23:44:16.373358+0000","last_became_peered":"2024-05-08T23:44:16.373358+0000","last_unstale":"2024-05-08T23:45:04.038848+0000","last_undegraded":"2024-05-08T23:45:04.038848+0000","last_fullsized":"2024-05-08T23:45:04.038848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.067168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:24.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022109+0000","last_change":"2024-05-08T23:45:04.048468+0000","last_active":"2024-05-08T23:45:05.022109+0000","last_peered":"2024-05-08T23:45:05.022109+0000","last_clean":"2024-05-08T23:45:05.022109+0000","last_became_active":"2024-05-08T23:45:04.048085+0000","last_became_peered":"2024-05-08T23:45:04.048085+0000","last_unstale":"2024-05-08T23:45:05.022109+0000","last_undegraded":"2024-05-08T23:45:05.022109+0000","last_fullsized":"2024-05-08T23:45:05.022109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:18.039780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781331+0000","last_change":"2024-05-08T23:45:03.575821+0000","last_active":"2024-05-08T23:45:06.781331+0000","last_peered":"2024-05-08T23:45:06.781331+0000","last_clean":"2024-05-08T23:45:06.781331+0000","last_became_active":"2024-05-08T23:45:03.575407+0000","last_became_peered":"2024-05-08T23:45:03.575407+0000","last_unstale":"2024-05-08T23:45:06.781331+0000","last_undegraded":"2024-05-08T23:45:06.781331+0000","last_fullsized":"2024-05-08T23:45:06.781331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:35.718942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.060787+0000","last_change":"2024-05-08T23:45:03.580100+0000","last_active":"2024-05-08T23:45:07.060787+0000","last_peered":"2024-05-08T23:45:07.060787+0000","last_clean":"2024-05-08T23:45:07.060787+0000","last_became_active":"2024-05-08T23:45:03.579791+0000","last_became_peered":"2024-05-08T23:45:03.579791+0000","last_unstale":"2024-05-08T23:45:07.060787+0000","last_undegraded":"2024-05-08T23:45:07.060787+0000","last_fullsized":"2024-05-08T23:45:07.060787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:41.086291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022221+0000","last_change":"2024-05-08T23:45:04.047687+0000","last_active":"2024-05-08T23:45:05.022221+0000","last_peered":"2024-05-08T23:45:05.022221+0000","last_clean":"2024-05-08T23:45:05.022221+0000","last_became_active":"2024-05-08T23:45:04.047343+0000","last_became_peered":"2024-05-08T23:45:04.047343+0000","last_unstale":"2024-05-08T23:45:05.022221+0000","last_undegraded":"2024-05-08T23:45:05.022221+0000","last_fullsized":"2024-05-08T23:45:05.022221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:58.763421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.070036+0000","last_change":"2024-05-08T23:45:03.573344+0000","last_active":"2024-05-08T23:45:08.070036+0000","last_peered":"2024-05-08T23:45:08.070036+0000","last_clean":"2024-05-08T23:45:08.070036+0000","last_became_active":"2024-05-08T23:45:03.572951+0000","last_became_peered":"2024-05-08T23:45:03.572951+0000","last_unstale":"2024-05-08T23:45:08.070036+0000","last_undegraded":"2024-05-08T23:45:08.070036+0000","last_fullsized":"2024-05-08T23:45:08.070036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.422908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040278+0000","last_change":"2024-05-08T23:45:04.055111+0000","last_active":"2024-05-08T23:45:05.040278+0000","last_peered":"2024-05-08T23:45:05.040278+0000","last_clean":"2024-05-08T23:45:05.040278+0000","last_became_active":"2024-05-08T23:45:04.054665+0000","last_became_peered":"2024-05-08T23:45:04.054665+0000","last_unstale":"2024-05-08T23:45:05.040278+0000","last_undegraded":"2024-05-08T23:45:05.040278+0000","last_fullsized":"2024-05-08T23:45:05.040278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:23.292630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062624+0000","last_change":"2024-05-08T23:45:03.578168+0000","last_active":"2024-05-08T23:45:07.062624+0000","last_peered":"2024-05-08T23:45:07.062624+0000","last_clean":"2024-05-08T23:45:07.062624+0000","last_became_active":"2024-05-08T23:45:03.577833+0000","last_became_peered":"2024-05-08T23:45:03.577833+0000","last_unstale":"2024-05-08T23:45:07.062624+0000","last_undegraded":"2024-05-08T23:45:07.062624+0000","last_fullsized":"2024-05-08T23:45:07.062624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:13:25.743151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282635+0000","last_change":"2024-05-08T23:45:04.049103+0000","last_active":"2024-05-08T23:45:06.282635+0000","last_peered":"2024-05-08T23:45:06.282635+0000","last_clean":"2024-05-08T23:45:06.282635+0000","last_became_active":"2024-05-08T23:45:04.048302+0000","last_became_peered":"2024-05-08T23:45:04.048302+0000","last_unstale":"2024-05-08T23:45:06.282635+0000","last_undegraded":"2024-05-08T23:45:06.282635+0000","last_fullsized":"2024-05-08T23:45:06.282635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:02.404522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061597+0000","last_change":"2024-05-08T23:45:03.577765+0000","last_active":"2024-05-08T23:45:07.061597+0000","last_peered":"2024-05-08T23:45:07.061597+0000","last_clean":"2024-05-08T23:45:07.061597+0000","last_became_active":"2024-05-08T23:45:03.577452+0000","last_became_peered":"2024-05-08T23:45:03.577452+0000","last_unstale":"2024-05-08T23:45:07.061597+0000","last_undegraded":"2024-05-08T23:45:07.061597+0000","last_fullsized":"2024-05-08T23:45:07.061597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:02.107878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517201+0000","last_change":"2024-05-08T23:45:04.061360+0000","last_active":"2024-05-08T23:45:05.517201+0000","last_peered":"2024-05-08T23:45:05.517201+0000","last_clean":"2024-05-08T23:45:05.517201+0000","last_became_active":"2024-05-08T23:45:04.061000+0000","last_became_peered":"2024-05-08T23:45:04.061000+0000","last_unstale":"2024-05-08T23:45:05.517201+0000","last_undegraded":"2024-05-08T23:45:05.517201+0000","last_fullsized":"2024-05-08T23:45:05.517201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:35.757571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061500+0000","last_change":"2024-05-08T23:45:03.578769+0000","last_active":"2024-05-08T23:45:07.061500+0000","last_peered":"2024-05-08T23:45:07.061500+0000","last_clean":"2024-05-08T23:45:07.061500+0000","last_became_active":"2024-05-08T23:45:03.578359+0000","last_became_peered":"2024-05-08T23:45:03.578359+0000","last_unstale":"2024-05-08T23:45:07.061500+0000","last_undegraded":"2024-05-08T23:45:07.061500+0000","last_fullsized":"2024-05-08T23:45:07.061500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:13.213382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040679+0000","last_change":"2024-05-08T23:45:04.049237+0000","last_active":"2024-05-08T23:45:05.040679+0000","last_peered":"2024-05-08T23:45:05.040679+0000","last_clean":"2024-05-08T23:45:05.040679+0000","last_became_active":"2024-05-08T23:45:04.048922+0000","last_became_peered":"2024-05-08T23:45:04.048922+0000","last_unstale":"2024-05-08T23:45:05.040679+0000","last_undegraded":"2024-05-08T23:45:05.040679+0000","last_fullsized":"2024-05-08T23:45:05.040679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:29.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225187+0000","last_change":"2024-05-08T23:45:03.574990+0000","last_active":"2024-05-08T23:45:06.225187+0000","last_peered":"2024-05-08T23:45:06.225187+0000","last_clean":"2024-05-08T23:45:06.225187+0000","last_became_active":"2024-05-08T23:45:03.574617+0000","last_became_peered":"2024-05-08T23:45:03.574617+0000","last_unstale":"2024-05-08T23:45:06.225187+0000","last_undegraded":"2024-05-08T23:45:06.225187+0000","last_fullsized":"2024-05-08T23:45:06.225187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:51.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039948+0000","last_change":"2024-05-08T23:45:04.054371+0000","last_active":"2024-05-08T23:45:05.039948+0000","last_peered":"2024-05-08T23:45:05.039948+0000","last_clean":"2024-05-08T23:45:05.039948+0000","last_became_active":"2024-05-08T23:45:04.054070+0000","last_became_peered":"2024-05-08T23:45:04.054070+0000","last_unstale":"2024-05-08T23:45:05.039948+0000","last_undegraded":"2024-05-08T23:45:05.039948+0000","last_fullsized":"2024-05-08T23:45:05.039948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:27.926522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062373+0000","last_change":"2024-05-08T23:45:03.579413+0000","last_active":"2024-05-08T23:45:07.062373+0000","last_peered":"2024-05-08T23:45:07.062373+0000","last_clean":"2024-05-08T23:45:07.062373+0000","last_became_active":"2024-05-08T23:45:03.577705+0000","last_became_peered":"2024-05-08T23:45:03.577705+0000","last_unstale":"2024-05-08T23:45:07.062373+0000","last_undegraded":"2024-05-08T23:45:07.062373+0000","last_fullsized":"2024-05-08T23:45:07.062373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:29.615247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781372+0000","last_change":"2024-05-08T23:45:04.050492+0000","last_active":"2024-05-08T23:45:06.781372+0000","last_peered":"2024-05-08T23:45:06.781372+0000","last_clean":"2024-05-08T23:45:06.781372+0000","last_became_active":"2024-05-08T23:45:04.050176+0000","last_became_peered":"2024-05-08T23:45:04.050176+0000","last_unstale":"2024-05-08T23:45:06.781372+0000","last_undegraded":"2024-05-08T23:45:06.781372+0000","last_fullsized":"2024-05-08T23:45:06.781372+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:01.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225570+0000","last_change":"2024-05-08T23:45:03.577651+0000","last_active":"2024-05-08T23:45:06.225570+0000","last_peered":"2024-05-08T23:45:06.225570+0000","last_clean":"2024-05-08T23:45:06.225570+0000","last_became_active":"2024-05-08T23:45:03.576887+0000","last_became_peered":"2024-05-08T23:45:03.576887+0000","last_unstale":"2024-05-08T23:45:06.225570+0000","last_undegraded":"2024-05-08T23:45:06.225570+0000","last_fullsized":"2024-05-08T23:45:06.225570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:52.822959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781487+0000","last_change":"2024-05-08T23:45:04.049793+0000","last_active":"2024-05-08T23:45:06.781487+0000","last_peered":"2024-05-08T23:45:06.781487+0000","last_clean":"2024-05-08T23:45:06.781487+0000","last_became_active":"2024-05-08T23:45:04.049456+0000","last_became_peered":"2024-05-08T23:45:04.049456+0000","last_unstale":"2024-05-08T23:45:06.781487+0000","last_undegraded":"2024-05-08T23:45:06.781487+0000","last_fullsized":"2024-05-08T23:45:06.781487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:16.030406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282639+0000","last_change":"2024-05-08T23:45:03.580371+0000","last_active":"2024-05-08T23:45:06.282639+0000","last_peered":"2024-05-08T23:45:06.282639+0000","last_clean":"2024-05-08T23:45:06.282639+0000","last_became_active":"2024-05-08T23:45:03.579394+0000","last_became_peered":"2024-05-08T23:45:03.579394+0000","last_unstale":"2024-05-08T23:45:06.282639+0000","last_undegraded":"2024-05-08T23:45:06.282639+0000","last_fullsized":"2024-05-08T23:45:06.282639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.123632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781264+0000","last_change":"2024-05-08T23:45:04.063138+0000","last_active":"2024-05-08T23:45:06.781264+0000","last_peered":"2024-05-08T23:45:06.781264+0000","last_clean":"2024-05-08T23:45:06.781264+0000","last_became_active":"2024-05-08T23:45:04.062835+0000","last_became_peered":"2024-05-08T23:45:04.062835+0000","last_unstale":"2024-05-08T23:45:06.781264+0000","last_undegraded":"2024-05-08T23:45:06.781264+0000","last_fullsized":"2024-05-08T23:45:06.781264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:49.538896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022233+0000","last_change":"2024-05-08T23:45:03.584351+0000","last_active":"2024-05-08T23:45:05.022233+0000","last_peered":"2024-05-08T23:45:05.022233+0000","last_clean":"2024-05-08T23:45:05.022233+0000","last_became_active":"2024-05-08T23:45:03.583996+0000","last_became_peered":"2024-05-08T23:45:03.583996+0000","last_unstale":"2024-05-08T23:45:05.022233+0000","last_undegraded":"2024-05-08T23:45:05.022233+0000","last_fullsized":"2024-05-08T23:45:05.022233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:40.530921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781432+0000","last_change":"2024-05-08T23:45:04.048918+0000","last_active":"2024-05-08T23:45:06.781432+0000","last_peered":"2024-05-08T23:45:06.781432+0000","last_clean":"2024-05-08T23:45:06.781432+0000","last_became_active":"2024-05-08T23:45:04.048550+0000","last_became_peered":"2024-05-08T23:45:04.048550+0000","last_unstale":"2024-05-08T23:45:06.781432+0000","last_undegraded":"2024-05-08T23:45:06.781432+0000","last_fullsized":"2024-05-08T23:45:06.781432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:48.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040646+0000","last_change":"2024-05-08T23:45:04.050790+0000","last_active":"2024-05-08T23:45:05.040646+0000","last_peered":"2024-05-08T23:45:05.040646+0000","last_clean":"2024-05-08T23:45:05.040646+0000","last_became_active":"2024-05-08T23:45:04.050429+0000","last_became_peered":"2024-05-08T23:45:04.050429+0000","last_unstale":"2024-05-08T23:45:05.040646+0000","last_undegraded":"2024-05-08T23:45:05.040646+0000","last_fullsized":"2024-05-08T23:45:05.040646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:43.224351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225201+0000","last_change":"2024-05-08T23:45:04.062063+0000","last_active":"2024-05-08T23:45:06.225201+0000","last_peered":"2024-05-08T23:45:06.225201+0000","last_clean":"2024-05-08T23:45:06.225201+0000","last_became_active":"2024-05-08T23:45:04.061206+0000","last_became_peered":"2024-05-08T23:45:04.061206+0000","last_unstale":"2024-05-08T23:45:06.225201+0000","last_undegraded":"2024-05-08T23:45:06.225201+0000","last_fullsized":"2024-05-08T23:45:06.225201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:23.137397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225238+0000","last_change":"2024-05-08T23:45:04.049707+0000","last_active":"2024-05-08T23:45:06.225238+0000","last_peered":"2024-05-08T23:45:06.225238+0000","last_clean":"2024-05-08T23:45:06.225238+0000","last_became_active":"2024-05-08T23:45:04.049385+0000","last_became_peered":"2024-05-08T23:45:04.049385+0000","last_unstale":"2024-05-08T23:45:06.225238+0000","last_undegraded":"2024-05-08T23:45:06.225238+0000","last_fullsized":"2024-05-08T23:45:06.225238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282351+0000","last_change":"2024-05-08T23:45:04.050721+0000","last_active":"2024-05-08T23:45:06.282351+0000","last_peered":"2024-05-08T23:45:06.282351+0000","last_clean":"2024-05-08T23:45:06.282351+0000","last_became_active":"2024-05-08T23:45:04.050368+0000","last_became_peered":"2024-05-08T23:45:04.050368+0000","last_unstale":"2024-05-08T23:45:06.282351+0000","last_undegraded":"2024-05-08T23:45:06.282351+0000","last_fullsized":"2024-05-08T23:45:06.282351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:20.815754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022211+0000","last_change":"2024-05-08T23:45:04.048431+0000","last_active":"2024-05-08T23:45:05.022211+0000","last_peered":"2024-05-08T23:45:05.022211+0000","last_clean":"2024-05-08T23:45:05.022211+0000","last_became_active":"2024-05-08T23:45:04.048071+0000","last_became_peered":"2024-05-08T23:45:04.048071+0000","last_unstale":"2024-05-08T23:45:05.022211+0000","last_undegraded":"2024-05-08T23:45:05.022211+0000","last_fullsized":"2024-05-08T23:45:05.022211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:51.547256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040715+0000","last_change":"2024-05-08T23:45:04.045256+0000","last_active":"2024-05-08T23:45:05.040715+0000","last_peered":"2024-05-08T23:45:05.040715+0000","last_clean":"2024-05-08T23:45:05.040715+0000","last_became_active":"2024-05-08T23:45:04.044973+0000","last_became_peered":"2024-05-08T23:45:04.044973+0000","last_unstale":"2024-05-08T23:45:05.040715+0000","last_undegraded":"2024-05-08T23:45:05.040715+0000","last_fullsized":"2024-05-08T23:45:05.040715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:06.021001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225634+0000","last_change":"2024-05-08T23:45:04.058287+0000","last_active":"2024-05-08T23:45:06.225634+0000","last_peered":"2024-05-08T23:45:06.225634+0000","last_clean":"2024-05-08T23:45:06.225634+0000","last_became_active":"2024-05-08T23:45:04.057977+0000","last_became_peered":"2024-05-08T23:45:04.057977+0000","last_unstale":"2024-05-08T23:45:06.225634+0000","last_undegraded":"2024-05-08T23:45:06.225634+0000","last_fullsized":"2024-05-08T23:45:06.225634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:01.146838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040216+0000","last_change":"2024-05-08T23:45:04.055780+0000","last_active":"2024-05-08T23:45:05.040216+0000","last_peered":"2024-05-08T23:45:05.040216+0000","last_clean":"2024-05-08T23:45:05.040216+0000","last_became_active":"2024-05-08T23:45:04.055272+0000","last_became_peered":"2024-05-08T23:45:04.055272+0000","last_unstale":"2024-05-08T23:45:05.040216+0000","last_undegraded":"2024-05-08T23:45:05.040216+0000","last_fullsized":"2024-05-08T23:45:05.040216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:28.587333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563811+0000","last_change":"2024-05-08T23:45:04.050648+0000","last_active":"2024-05-08T23:45:05.563811+0000","last_peered":"2024-05-08T23:45:05.563811+0000","last_clean":"2024-05-08T23:45:05.563811+0000","last_became_active":"2024-05-08T23:45:04.050353+0000","last_became_peered":"2024-05-08T23:45:04.050353+0000","last_unstale":"2024-05-08T23:45:05.563811+0000","last_undegraded":"2024-05-08T23:45:05.563811+0000","last_fullsized":"2024-05-08T23:45:05.563811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:58.154006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039875+0000","last_change":"2024-05-08T23:45:04.056951+0000","last_active":"2024-05-08T23:45:05.039875+0000","last_peered":"2024-05-08T23:45:05.039875+0000","last_clean":"2024-05-08T23:45:05.039875+0000","last_became_active":"2024-05-08T23:45:04.056585+0000","last_became_peered":"2024-05-08T23:45:04.056585+0000","last_unstale":"2024-05-08T23:45:05.039875+0000","last_undegraded":"2024-05-08T23:45:05.039875+0000","last_fullsized":"2024-05-08T23:45:05.039875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:22.658629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300447+0000","last_change":"2024-05-08T23:45:04.052396+0000","last_active":"2024-05-08T23:45:04.300447+0000","last_peered":"2024-05-08T23:45:04.300447+0000","last_clean":"2024-05-08T23:45:04.300447+0000","last_became_active":"2024-05-08T23:45:04.051604+0000","last_became_peered":"2024-05-08T23:45:04.051604+0000","last_unstale":"2024-05-08T23:45:04.300447+0000","last_undegraded":"2024-05-08T23:45:04.300447+0000","last_fullsized":"2024-05-08T23:45:04.300447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:52.602830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022201+0000","last_change":"2024-05-08T23:45:04.051578+0000","last_active":"2024-05-08T23:45:05.022201+0000","last_peered":"2024-05-08T23:45:05.022201+0000","last_clean":"2024-05-08T23:45:05.022201+0000","last_became_active":"2024-05-08T23:45:04.051298+0000","last_became_peered":"2024-05-08T23:45:04.051298+0000","last_unstale":"2024-05-08T23:45:05.022201+0000","last_undegraded":"2024-05-08T23:45:05.022201+0000","last_fullsized":"2024-05-08T23:45:05.022201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:47.204849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040483+0000","last_change":"2024-05-08T23:45:04.051264+0000","last_active":"2024-05-08T23:45:05.040483+0000","last_peered":"2024-05-08T23:45:05.040483+0000","last_clean":"2024-05-08T23:45:05.040483+0000","last_became_active":"2024-05-08T23:45:04.050870+0000","last_became_peered":"2024-05-08T23:45:04.050870+0000","last_unstale":"2024-05-08T23:45:05.040483+0000","last_undegraded":"2024-05-08T23:45:05.040483+0000","last_fullsized":"2024-05-08T23:45:05.040483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4496,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":107,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":11464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":876544,"data_stored":421419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":10,"up_from":69,"seq":296352743438,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1013932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":9,"up_from":64,"seq":274877906962,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":421655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54300000000000004}]},{"osd":2,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.61099999999999999}]},{"osd":3,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.754}]},{"osd":5,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.82899999999999996}]},{"osd":7,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79100000000000004}]},{"osd":8,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.877},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89500000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":888832,"data_stored":421244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.53500000000000003}]},{"osd":1,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58099999999999996}]},{"osd":2,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.65300000000000002}]},{"osd":4,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.627},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.69799999999999995}]},{"osd":5,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.46000000000000002}]},{"osd":6,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.746}]},{"osd":7,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68899999999999995}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.75800000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":7,"up_from":49,"seq":210453397528,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32544,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712608,"statfs":{"total":95995035648,"available":95961710592,"internally_reserved":0,"allocated":1454080,"data_stored":1002886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.628},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59099999999999997}]},{"osd":4,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.42699999999999999}]},{"osd":5,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56299999999999994}]},{"osd":8,"last update":"Wed May 8 23:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.66300000000000003}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.67200000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561052,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":876544,"data_stored":420927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.46500000000000002,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.442}]},{"osd":1,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.622},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.64800000000000002,"15min":0.66200000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79400000000000004,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.56299999999999994,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92400000000000004,"5min":1.288,"15min":1.288},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.505},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.44800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89000000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.39300000000000002}]},{"osd":4,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":0.97799999999999998,"5min":1.113,"15min":1.113},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.496,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59299999999999997}]},{"osd":7,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.73899999999999999}]},{"osd":8,"last update":"Wed May 8 23:44:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.68500000000000005}]},{"osd":9,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.66600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36068,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709084,"statfs":{"total":95995035648,"available":95958102016,"internally_reserved":0,"allocated":876544,"data_stored":421761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.505,"15min":0.48899999999999999},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.51300000000000001,"15min":0.496},"min":{"1min":0.37,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.48099999999999998}]},{"osd":2,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.54400000000000004}]},{"osd":3,"last update":"Wed May 8 23:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79000000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.38700000000000001,"15min":0.378},"min":{"1min":0.29299999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.41199999999999998,"15min":0.40300000000000002},"min":{"1min":0.32200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.42299999999999999}]},{"osd":5,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.41599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63800000000000001}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88400000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.61399999999999999}]},{"osd":7,"last update":"Wed May 8 23:44:17 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed May 8 23:44:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69699999999999995}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.72799999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738418,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":411990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:44:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.51600000000000001,"15min":0.495},"min":{"1min":0.34000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.38600000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.36299999999999999}]},{"osd":2,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.51800000000000002},"min":{"1min":0.40400000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.47099999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed May 8 23:44:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.41399999999999998,"15min":0.39400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.42099999999999999,"15min":0.40600000000000003},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.751}]},{"osd":4,"last update":"Wed May 8 23:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56100000000000005,"15min":0.55200000000000005},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.495,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.502}]},{"osd":5,"last update":"Wed May 8 23:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.52700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.46200000000000002}]},{"osd":6,"last update":"Wed May 8 23:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.39500000000000002}]},{"osd":7,"last update":"Wed May 8 23:44:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997}]},{"osd":8,"last update":"Wed May 8 23:44:37 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57499999999999996}]},{"osd":9,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68600000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":2,"up_from":17,"seq":73014444075,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":868352,"data_stored":413225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53700000000000003,"15min":0.52000000000000002},"min":{"1min":0.42999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53500000000000003,"15min":0.51300000000000001},"min":{"1min":0.41999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.501,"15min":0.48099999999999998},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.49199999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.79600000000000004}]},{"osd":3,"last update":"Wed May 8 23:44:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.753},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.72499999999999998}]},{"osd":4,"last update":"Wed May 8 23:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.91400000000000003}]},{"osd":5,"last update":"Wed May 8 23:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.38600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed May 8 23:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.5,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.42199999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.98699999999999999}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":1.014}]},{"osd":9,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.94999999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215144,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36076,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709076,"statfs":{"total":95995035648,"available":95958093824,"internally_reserved":0,"allocated":888832,"data_stored":421121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.373,"15min":0.36899999999999999},"min":{"1min":0.28999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58299999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.308,"5min":0.248,"15min":0.248},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed May 8 23:44:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed May 8 23:44:10 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.49399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Wed May 8 23:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.504,"5min":0.434,"15min":0.434},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003}]},{"osd":5,"last update":"Wed May 8 23:44:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.47699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.88100000000000001}]},{"osd":8,"last update":"Wed May 8 23:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed May 8 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.747}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986210,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":868352,"data_stored":420219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.28199999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.432},{"interface":"front","average":{"1min":0.437,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.58599999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81000000000000005,"5min":1.222,"15min":1.222},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.57099999999999995,"15min":0.55400000000000005},"min":{"1min":0.48399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.75900000000000001}]},{"osd":3,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.379,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.62},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed May 8 23:44:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89700000000000002,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.91900000000000004,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed May 8 23:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.50900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.504}]},{"osd":7,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.47899999999999998}]},{"osd":8,"last update":"Wed May 8 23:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86099999999999999}]},{"osd":9,"last update":"Wed May 8 23:44:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.79300000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":5,"up_from":36,"seq":154618822689,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32572,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712580,"statfs":{"total":95995035648,"available":95961681920,"internally_reserved":0,"allocated":1495040,"data_stored":1012633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.36399999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51300000000000001,"15min":0.505},"min":{"1min":0.38600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.502},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.625},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.86199999999999999,"5min":0.873,"15min":0.873},"last":0.68100000000000005}]},{"osd":2,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47599999999999998}]},{"osd":3,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.49299999999999999,"15min":0.47899999999999998},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.434,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56799999999999995}]},{"osd":4,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.40999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58399999999999996}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.73699999999999999}]},{"osd":8,"last update":"Wed May 8 23:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.66000000000000003}]},{"osd":9,"last update":"Wed May 8 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.751},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:45:12.513 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:45:12.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:12 smithi204 ceph-mon[47373]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:12.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:12 smithi204 ceph-mon[47373]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T23:45:12.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:12 smithi204 ceph-mon[47373]: pgmap v229: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.8 KiB/s wr, 15 op/s 2024-05-08T23:45:12.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:12 smithi204 ceph-mon[47373]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:12.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:12 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/530328957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:12.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:12 smithi160 ceph-mon[46152]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:12.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:12 smithi160 ceph-mon[46152]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T23:45:12.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:12 smithi160 ceph-mon[46152]: pgmap v229: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.8 KiB/s wr, 15 op/s 2024-05-08T23:45:12.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:12 smithi160 ceph-mon[46152]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:45:12.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:12 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/530328957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:12.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.691+0000 7fee35e82700 1 -- 172.21.15.12:0/102982160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee30106b40 msgr2=0x7fee30108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.691+0000 7fee35e82700 1 --2- 172.21.15.12:0/102982160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee30106b40 0x7fee30108fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fee2800b0a0 tx=0x7fee2800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.691+0000 7fee35e82700 1 -- 172.21.15.12:0/102982160 shutdown_connections 2024-05-08T23:45:12.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.691+0000 7fee35e82700 1 --2- 172.21.15.12:0/102982160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee30106b40 0x7fee30108fb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.691+0000 7fee35e82700 1 --2- 172.21.15.12:0/102982160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee301018e0 0x7fee30103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.691+0000 7fee35e82700 1 --2- 172.21.15.12:0/102982160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30104210 0x7fee30106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.691+0000 7fee35e82700 1 -- 172.21.15.12:0/102982160 >> 172.21.15.12:0/102982160 conn(0x7fee300fb2f0 msgr2=0x7fee300fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:12.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.692+0000 7fee35e82700 1 -- 172.21.15.12:0/102982160 shutdown_connections 2024-05-08T23:45:12.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.692+0000 7fee35e82700 1 -- 172.21.15.12:0/102982160 wait complete. 2024-05-08T23:45:12.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.692+0000 7fee35e82700 1 Processor -- start 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.692+0000 7fee35e82700 1 -- start start 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee35e82700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee301018e0 0x7fee30196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee35e82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee30104210 0x7fee30197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee35e82700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30106b40 0x7fee3019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee35e82700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3019ea00 con 0x7fee30104210 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee35e82700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3019eb40 con 0x7fee301018e0 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee35e82700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3019ecd0 con 0x7fee30106b40 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee30104210 0x7fee30197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee30104210 0x7fee30197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37664/0 (socket says 172.21.15.12:37664) 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2effd700 1 -- 172.21.15.12:0/1403398895 learned_addr learned my addr 172.21.15.12:0/1403398895 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2f7fe700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee301018e0 0x7fee30196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2ffff700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30106b40 0x7fee3019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2effd700 1 -- 172.21.15.12:0/1403398895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30106b40 msgr2=0x7fee3019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2effd700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30106b40 0x7fee3019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2effd700 1 -- 172.21.15.12:0/1403398895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee301018e0 msgr2=0x7fee30196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2effd700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee301018e0 0x7fee30196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2effd700 1 -- 172.21.15.12:0/1403398895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee28009d00 con 0x7fee30104210 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2effd700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee30104210 0x7fee30197410 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fee2400e9a0 tx=0x7fee2400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.693+0000 7fee2cff9700 1 -- 172.21.15.12:0/1403398895 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24009790 con 0x7fee30104210 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.694+0000 7fee2cff9700 1 -- 172.21.15.12:0/1403398895 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee2400cbd0 con 0x7fee30104210 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.694+0000 7fee2cff9700 1 -- 172.21.15.12:0/1403398895 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24018680 con 0x7fee30104210 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.694+0000 7fee35e82700 1 -- 172.21.15.12:0/1403398895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3019efb0 con 0x7fee30104210 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.694+0000 7fee35e82700 1 -- 172.21.15.12:0/1403398895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3019f3d0 con 0x7fee30104210 2024-05-08T23:45:12.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.695+0000 7fee35e82700 1 -- 172.21.15.12:0/1403398895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee30067c80 con 0x7fee30104210 2024-05-08T23:45:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.696+0000 7fee2cff9700 1 -- 172.21.15.12:0/1403398895 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fee2400cd40 con 0x7fee30104210 2024-05-08T23:45:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.697+0000 7fee2cff9700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fee180a11b0 0x7fee180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.697+0000 7fee2f7fe700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fee180a11b0 0x7fee180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.697+0000 7fee2cff9700 1 -- 172.21.15.12:0/1403398895 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fee24014070 con 0x7fee30104210 2024-05-08T23:45:12.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.697+0000 7fee2f7fe700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fee180a11b0 0x7fee180a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee20005950 tx=0x7fee200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.702+0000 7fee2cff9700 1 -- 172.21.15.12:0/1403398895 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fee24052d60 con 0x7fee30104210 2024-05-08T23:45:12.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.917+0000 7fee35e82700 1 -- 172.21.15.12:0/1403398895 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee3002cf50 con 0x7fee180a11b0 2024-05-08T23:45:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.927+0000 7fee2cff9700 1 -- 172.21.15.12:0/1403398895 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+328108 (secure 0 0 0) 0x7fee3002cf50 con 0x7fee180a11b0 2024-05-08T23:45:12.928 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:12.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.931+0000 7fee35e82700 1 -- 172.21.15.12:0/1403398895 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fee180a11b0 msgr2=0x7fee180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.931+0000 7fee35e82700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fee180a11b0 0x7fee180a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee20005950 tx=0x7fee200058e0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.931+0000 7fee35e82700 1 -- 172.21.15.12:0/1403398895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee30104210 msgr2=0x7fee30197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:12.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.931+0000 7fee35e82700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee30104210 0x7fee30197410 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fee2400e9a0 tx=0x7fee2400ed60 comp rx=0 tx=0).stop 2024-05-08T23:45:12.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.931+0000 7fee35e82700 1 -- 172.21.15.12:0/1403398895 shutdown_connections 2024-05-08T23:45:12.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.931+0000 7fee35e82700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30106b40 0x7fee3019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.931+0000 7fee35e82700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fee180a11b0 0x7fee180a3670 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.931+0000 7fee35e82700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee301018e0 0x7fee30196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.931+0000 7fee35e82700 1 --2- 172.21.15.12:0/1403398895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee30104210 0x7fee30197410 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:12.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.932+0000 7fee35e82700 1 -- 172.21.15.12:0/1403398895 >> 172.21.15.12:0/1403398895 conn(0x7fee300fb2f0 msgr2=0x7fee30102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:12.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.932+0000 7fee35e82700 1 -- 172.21.15.12:0/1403398895 shutdown_connections 2024-05-08T23:45:12.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:12.932+0000 7fee35e82700 1 -- 172.21.15.12:0/1403398895 wait complete. 2024-05-08T23:45:12.932 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-08T23:45:12.941 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-05-08T23:45:11.253291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594864,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":148,"num_read_kb":129,"num_write":288,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":266,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373160,"kb_used_data":12064,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124568664,"statfs":{"total":1151940427776,"available":1151558311936,"internally_reserved":0,"allocated":12353536,"data_stored":6803012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4496,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":107,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.692496"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040868+0000","last_change":"2024-05-08T23:45:04.044088+0000","last_active":"2024-05-08T23:45:05.040868+0000","last_peered":"2024-05-08T23:45:05.040868+0000","last_clean":"2024-05-08T23:45:05.040868+0000","last_became_active":"2024-05-08T23:45:04.043535+0000","last_became_peered":"2024-05-08T23:45:04.043535+0000","last_unstale":"2024-05-08T23:45:05.040868+0000","last_undegraded":"2024-05-08T23:45:05.040868+0000","last_fullsized":"2024-05-08T23:45:05.040868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:52.363402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282575+0000","last_change":"2024-05-08T23:45:04.049201+0000","last_active":"2024-05-08T23:45:06.282575+0000","last_peered":"2024-05-08T23:45:06.282575+0000","last_clean":"2024-05-08T23:45:06.282575+0000","last_became_active":"2024-05-08T23:45:04.048743+0000","last_became_peered":"2024-05-08T23:45:04.048743+0000","last_unstale":"2024-05-08T23:45:06.282575+0000","last_undegraded":"2024-05-08T23:45:06.282575+0000","last_fullsized":"2024-05-08T23:45:06.282575+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:26.191549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282032+0000","last_change":"2024-05-08T23:45:04.047356+0000","last_active":"2024-05-08T23:45:06.282032+0000","last_peered":"2024-05-08T23:45:06.282032+0000","last_clean":"2024-05-08T23:45:06.282032+0000","last_became_active":"2024-05-08T23:45:04.046996+0000","last_became_peered":"2024-05-08T23:45:04.046996+0000","last_unstale":"2024-05-08T23:45:06.282032+0000","last_undegraded":"2024-05-08T23:45:06.282032+0000","last_fullsized":"2024-05-08T23:45:06.282032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:58.722522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282798+0000","last_change":"2024-05-08T23:45:04.054754+0000","last_active":"2024-05-08T23:45:06.282798+0000","last_peered":"2024-05-08T23:45:06.282798+0000","last_clean":"2024-05-08T23:45:06.282798+0000","last_became_active":"2024-05-08T23:45:04.054302+0000","last_became_peered":"2024-05-08T23:45:04.054302+0000","last_unstale":"2024-05-08T23:45:06.282798+0000","last_undegraded":"2024-05-08T23:45:06.282798+0000","last_fullsized":"2024-05-08T23:45:06.282798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:28.882982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517234+0000","last_change":"2024-05-08T23:45:04.047189+0000","last_active":"2024-05-08T23:45:05.517234+0000","last_peered":"2024-05-08T23:45:05.517234+0000","last_clean":"2024-05-08T23:45:05.517234+0000","last_became_active":"2024-05-08T23:45:04.046756+0000","last_became_peered":"2024-05-08T23:45:04.046756+0000","last_unstale":"2024-05-08T23:45:05.517234+0000","last_undegraded":"2024-05-08T23:45:05.517234+0000","last_fullsized":"2024-05-08T23:45:05.517234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.780169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045028+0000","last_change":"2024-05-08T23:45:04.060897+0000","last_active":"2024-05-08T23:45:05.045028+0000","last_peered":"2024-05-08T23:45:05.045028+0000","last_clean":"2024-05-08T23:45:05.045028+0000","last_became_active":"2024-05-08T23:45:04.060550+0000","last_became_peered":"2024-05-08T23:45:04.060550+0000","last_unstale":"2024-05-08T23:45:05.045028+0000","last_undegraded":"2024-05-08T23:45:05.045028+0000","last_fullsized":"2024-05-08T23:45:05.045028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:59.356057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781043+0000","last_change":"2024-05-08T23:45:04.059308+0000","last_active":"2024-05-08T23:45:06.781043+0000","last_peered":"2024-05-08T23:45:06.781043+0000","last_clean":"2024-05-08T23:45:06.781043+0000","last_became_active":"2024-05-08T23:45:04.058920+0000","last_became_peered":"2024-05-08T23:45:04.058920+0000","last_unstale":"2024-05-08T23:45:06.781043+0000","last_undegraded":"2024-05-08T23:45:06.781043+0000","last_fullsized":"2024-05-08T23:45:06.781043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:11.857062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039706+0000","last_change":"2024-05-08T23:45:04.061542+0000","last_active":"2024-05-08T23:45:05.039706+0000","last_peered":"2024-05-08T23:45:05.039706+0000","last_clean":"2024-05-08T23:45:05.039706+0000","last_became_active":"2024-05-08T23:45:04.061145+0000","last_became_peered":"2024-05-08T23:45:04.061145+0000","last_unstale":"2024-05-08T23:45:05.039706+0000","last_undegraded":"2024-05-08T23:45:05.039706+0000","last_fullsized":"2024-05-08T23:45:05.039706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:53.159292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300493+0000","last_change":"2024-05-08T23:45:04.052292+0000","last_active":"2024-05-08T23:45:04.300493+0000","last_peered":"2024-05-08T23:45:04.300493+0000","last_clean":"2024-05-08T23:45:04.300493+0000","last_became_active":"2024-05-08T23:45:04.051164+0000","last_became_peered":"2024-05-08T23:45:04.051164+0000","last_unstale":"2024-05-08T23:45:04.300493+0000","last_undegraded":"2024-05-08T23:45:04.300493+0000","last_fullsized":"2024-05-08T23:45:04.300493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:09.218130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045014+0000","last_change":"2024-05-08T23:45:04.054566+0000","last_active":"2024-05-08T23:45:05.045014+0000","last_peered":"2024-05-08T23:45:05.045014+0000","last_clean":"2024-05-08T23:45:05.045014+0000","last_became_active":"2024-05-08T23:45:04.053878+0000","last_became_peered":"2024-05-08T23:45:04.053878+0000","last_unstale":"2024-05-08T23:45:05.045014+0000","last_undegraded":"2024-05-08T23:45:05.045014+0000","last_fullsized":"2024-05-08T23:45:05.045014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:34.760038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781480+0000","last_change":"2024-05-08T23:45:04.049696+0000","last_active":"2024-05-08T23:45:06.781480+0000","last_peered":"2024-05-08T23:45:06.781480+0000","last_clean":"2024-05-08T23:45:06.781480+0000","last_became_active":"2024-05-08T23:45:04.049351+0000","last_became_peered":"2024-05-08T23:45:04.049351+0000","last_unstale":"2024-05-08T23:45:06.781480+0000","last_undegraded":"2024-05-08T23:45:06.781480+0000","last_fullsized":"2024-05-08T23:45:06.781480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:53.557738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282779+0000","last_change":"2024-05-08T23:45:04.051010+0000","last_active":"2024-05-08T23:45:06.282779+0000","last_peered":"2024-05-08T23:45:06.282779+0000","last_clean":"2024-05-08T23:45:06.282779+0000","last_became_active":"2024-05-08T23:45:04.050715+0000","last_became_peered":"2024-05-08T23:45:04.050715+0000","last_unstale":"2024-05-08T23:45:06.282779+0000","last_undegraded":"2024-05-08T23:45:06.282779+0000","last_fullsized":"2024-05-08T23:45:06.282779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:09.305974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022192+0000","last_change":"2024-05-08T23:45:04.051005+0000","last_active":"2024-05-08T23:45:05.022192+0000","last_peered":"2024-05-08T23:45:05.022192+0000","last_clean":"2024-05-08T23:45:05.022192+0000","last_became_active":"2024-05-08T23:45:04.050123+0000","last_became_peered":"2024-05-08T23:45:04.050123+0000","last_unstale":"2024-05-08T23:45:05.022192+0000","last_undegraded":"2024-05-08T23:45:05.022192+0000","last_fullsized":"2024-05-08T23:45:05.022192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:52.206389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282033+0000","last_change":"2024-05-08T23:45:04.049942+0000","last_active":"2024-05-08T23:45:06.282033+0000","last_peered":"2024-05-08T23:45:06.282033+0000","last_clean":"2024-05-08T23:45:06.282033+0000","last_became_active":"2024-05-08T23:45:04.049596+0000","last_became_peered":"2024-05-08T23:45:04.049596+0000","last_unstale":"2024-05-08T23:45:06.282033+0000","last_undegraded":"2024-05-08T23:45:06.282033+0000","last_fullsized":"2024-05-08T23:45:06.282033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:30.907023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282203+0000","last_change":"2024-05-08T23:45:04.048985+0000","last_active":"2024-05-08T23:45:06.282203+0000","last_peered":"2024-05-08T23:45:06.282203+0000","last_clean":"2024-05-08T23:45:06.282203+0000","last_became_active":"2024-05-08T23:45:04.048658+0000","last_became_peered":"2024-05-08T23:45:04.048658+0000","last_unstale":"2024-05-08T23:45:06.282203+0000","last_undegraded":"2024-05-08T23:45:06.282203+0000","last_fullsized":"2024-05-08T23:45:06.282203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:17.030930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300415+0000","last_change":"2024-05-08T23:45:04.051903+0000","last_active":"2024-05-08T23:45:04.300415+0000","last_peered":"2024-05-08T23:45:04.300415+0000","last_clean":"2024-05-08T23:45:04.300415+0000","last_became_active":"2024-05-08T23:45:04.051221+0000","last_became_peered":"2024-05-08T23:45:04.051221+0000","last_unstale":"2024-05-08T23:45:04.300415+0000","last_undegraded":"2024-05-08T23:45:04.300415+0000","last_fullsized":"2024-05-08T23:45:04.300415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:01.554463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040970+0000","last_change":"2024-05-08T23:45:04.043854+0000","last_active":"2024-05-08T23:45:05.040970+0000","last_peered":"2024-05-08T23:45:05.040970+0000","last_clean":"2024-05-08T23:45:05.040970+0000","last_became_active":"2024-05-08T23:45:04.043056+0000","last_became_peered":"2024-05-08T23:45:04.043056+0000","last_unstale":"2024-05-08T23:45:05.040970+0000","last_undegraded":"2024-05-08T23:45:05.040970+0000","last_fullsized":"2024-05-08T23:45:05.040970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:18.175575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.058883+0000","last_change":"2024-05-08T23:45:04.049332+0000","last_active":"2024-05-08T23:45:04.058883+0000","last_peered":"2024-05-08T23:45:04.058883+0000","last_clean":"2024-05-08T23:45:04.058883+0000","last_became_active":"2024-05-08T23:45:04.048987+0000","last_became_peered":"2024-05-08T23:45:04.048987+0000","last_unstale":"2024-05-08T23:45:04.058883+0000","last_undegraded":"2024-05-08T23:45:04.058883+0000","last_fullsized":"2024-05-08T23:45:04.058883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:02.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225413+0000","last_change":"2024-05-08T23:45:04.048536+0000","last_active":"2024-05-08T23:45:06.225413+0000","last_peered":"2024-05-08T23:45:06.225413+0000","last_clean":"2024-05-08T23:45:06.225413+0000","last_became_active":"2024-05-08T23:45:04.048102+0000","last_became_peered":"2024-05-08T23:45:04.048102+0000","last_unstale":"2024-05-08T23:45:06.225413+0000","last_undegraded":"2024-05-08T23:45:06.225413+0000","last_fullsized":"2024-05-08T23:45:06.225413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:11.794372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.070828+0000","last_change":"2024-05-08T23:45:03.576833+0000","last_active":"2024-05-08T23:45:08.070828+0000","last_peered":"2024-05-08T23:45:08.070828+0000","last_clean":"2024-05-08T23:45:08.070828+0000","last_became_active":"2024-05-08T23:45:03.576495+0000","last_became_peered":"2024-05-08T23:45:03.576495+0000","last_unstale":"2024-05-08T23:45:08.070828+0000","last_undegraded":"2024-05-08T23:45:08.070828+0000","last_fullsized":"2024-05-08T23:45:08.070828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:50.266658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040105+0000","last_change":"2024-05-08T23:45:04.055444+0000","last_active":"2024-05-08T23:45:05.040105+0000","last_peered":"2024-05-08T23:45:05.040105+0000","last_clean":"2024-05-08T23:45:05.040105+0000","last_became_active":"2024-05-08T23:45:04.054998+0000","last_became_peered":"2024-05-08T23:45:04.054998+0000","last_unstale":"2024-05-08T23:45:05.040105+0000","last_undegraded":"2024-05-08T23:45:05.040105+0000","last_fullsized":"2024-05-08T23:45:05.040105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:03.470555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.100638+0000","last_change":"2024-05-08T23:45:03.578362+0000","last_active":"2024-05-08T23:45:05.100638+0000","last_peered":"2024-05-08T23:45:05.100638+0000","last_clean":"2024-05-08T23:45:05.100638+0000","last_became_active":"2024-05-08T23:45:03.577834+0000","last_became_peered":"2024-05-08T23:45:03.577834+0000","last_unstale":"2024-05-08T23:45:05.100638+0000","last_undegraded":"2024-05-08T23:45:05.100638+0000","last_fullsized":"2024-05-08T23:45:05.100638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:01.943490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282763+0000","last_change":"2024-05-08T23:45:04.047205+0000","last_active":"2024-05-08T23:45:06.282763+0000","last_peered":"2024-05-08T23:45:06.282763+0000","last_clean":"2024-05-08T23:45:06.282763+0000","last_became_active":"2024-05-08T23:45:04.046860+0000","last_became_peered":"2024-05-08T23:45:04.046860+0000","last_unstale":"2024-05-08T23:45:06.282763+0000","last_undegraded":"2024-05-08T23:45:06.282763+0000","last_fullsized":"2024-05-08T23:45:06.282763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:36.837101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.069768+0000","last_change":"2024-05-08T23:45:03.582026+0000","last_active":"2024-05-08T23:45:08.069768+0000","last_peered":"2024-05-08T23:45:08.069768+0000","last_clean":"2024-05-08T23:45:08.069768+0000","last_became_active":"2024-05-08T23:45:03.581672+0000","last_became_peered":"2024-05-08T23:45:03.581672+0000","last_unstale":"2024-05-08T23:45:08.069768+0000","last_undegraded":"2024-05-08T23:45:08.069768+0000","last_fullsized":"2024-05-08T23:45:08.069768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:50.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300470+0000","last_change":"2024-05-08T23:45:04.052639+0000","last_active":"2024-05-08T23:45:04.300470+0000","last_peered":"2024-05-08T23:45:04.300470+0000","last_clean":"2024-05-08T23:45:04.300470+0000","last_became_active":"2024-05-08T23:45:04.052158+0000","last_became_peered":"2024-05-08T23:45:04.052158+0000","last_unstale":"2024-05-08T23:45:04.300470+0000","last_undegraded":"2024-05-08T23:45:04.300470+0000","last_fullsized":"2024-05-08T23:45:04.300470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:25.638555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517264+0000","last_change":"2024-05-08T23:45:03.579221+0000","last_active":"2024-05-08T23:45:05.517264+0000","last_peered":"2024-05-08T23:45:05.517264+0000","last_clean":"2024-05-08T23:45:05.517264+0000","last_became_active":"2024-05-08T23:45:03.578795+0000","last_became_peered":"2024-05-08T23:45:03.578795+0000","last_unstale":"2024-05-08T23:45:05.517264+0000","last_undegraded":"2024-05-08T23:45:05.517264+0000","last_fullsized":"2024-05-08T23:45:05.517264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:14.064108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563925+0000","last_change":"2024-05-08T23:45:04.057573+0000","last_active":"2024-05-08T23:45:05.563925+0000","last_peered":"2024-05-08T23:45:05.563925+0000","last_clean":"2024-05-08T23:45:05.563925+0000","last_became_active":"2024-05-08T23:45:04.053201+0000","last_became_peered":"2024-05-08T23:45:04.053201+0000","last_unstale":"2024-05-08T23:45:05.563925+0000","last_undegraded":"2024-05-08T23:45:05.563925+0000","last_fullsized":"2024-05-08T23:45:05.563925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:23.612609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:09.076983+0000","last_change":"2024-05-08T23:45:03.578110+0000","last_active":"2024-05-08T23:45:09.076983+0000","last_peered":"2024-05-08T23:45:09.076983+0000","last_clean":"2024-05-08T23:45:09.076983+0000","last_became_active":"2024-05-08T23:45:03.577749+0000","last_became_peered":"2024-05-08T23:45:03.577749+0000","last_unstale":"2024-05-08T23:45:09.076983+0000","last_undegraded":"2024-05-08T23:45:09.076983+0000","last_fullsized":"2024-05-08T23:45:09.076983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:50.526535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517216+0000","last_change":"2024-05-08T23:45:04.048131+0000","last_active":"2024-05-08T23:45:05.517216+0000","last_peered":"2024-05-08T23:45:05.517216+0000","last_clean":"2024-05-08T23:45:05.517216+0000","last_became_active":"2024-05-08T23:45:04.047773+0000","last_became_peered":"2024-05-08T23:45:04.047773+0000","last_unstale":"2024-05-08T23:45:05.517216+0000","last_undegraded":"2024-05-08T23:45:05.517216+0000","last_fullsized":"2024-05-08T23:45:05.517216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:47.944900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.070447+0000","last_change":"2024-05-08T23:45:03.586710+0000","last_active":"2024-05-08T23:45:08.070447+0000","last_peered":"2024-05-08T23:45:08.070447+0000","last_clean":"2024-05-08T23:45:08.070447+0000","last_became_active":"2024-05-08T23:45:03.586411+0000","last_became_peered":"2024-05-08T23:45:03.586411+0000","last_unstale":"2024-05-08T23:45:08.070447+0000","last_undegraded":"2024-05-08T23:45:08.070447+0000","last_fullsized":"2024-05-08T23:45:08.070447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:25.938846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563896+0000","last_change":"2024-05-08T23:45:04.053885+0000","last_active":"2024-05-08T23:45:05.563896+0000","last_peered":"2024-05-08T23:45:05.563896+0000","last_clean":"2024-05-08T23:45:05.563896+0000","last_became_active":"2024-05-08T23:45:04.053385+0000","last_became_peered":"2024-05-08T23:45:04.053385+0000","last_unstale":"2024-05-08T23:45:05.563896+0000","last_undegraded":"2024-05-08T23:45:05.563896+0000","last_fullsized":"2024-05-08T23:45:05.563896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:10.805481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.071417+0000","last_change":"2024-05-08T23:45:03.574988+0000","last_active":"2024-05-08T23:45:08.071417+0000","last_peered":"2024-05-08T23:45:08.071417+0000","last_clean":"2024-05-08T23:45:08.071417+0000","last_became_active":"2024-05-08T23:45:03.574586+0000","last_became_peered":"2024-05-08T23:45:03.574586+0000","last_unstale":"2024-05-08T23:45:08.071417+0000","last_undegraded":"2024-05-08T23:45:08.071417+0000","last_fullsized":"2024-05-08T23:45:08.071417+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:18.005107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039815+0000","last_change":"2024-05-08T23:45:04.046817+0000","last_active":"2024-05-08T23:45:05.039815+0000","last_peered":"2024-05-08T23:45:05.039815+0000","last_clean":"2024-05-08T23:45:05.039815+0000","last_became_active":"2024-05-08T23:45:04.046515+0000","last_became_peered":"2024-05-08T23:45:04.046515+0000","last_unstale":"2024-05-08T23:45:05.039815+0000","last_undegraded":"2024-05-08T23:45:05.039815+0000","last_fullsized":"2024-05-08T23:45:05.039815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:29.439670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282372+0000","last_change":"2024-05-08T23:45:03.579002+0000","last_active":"2024-05-08T23:45:06.282372+0000","last_peered":"2024-05-08T23:45:06.282372+0000","last_clean":"2024-05-08T23:45:06.282372+0000","last_became_active":"2024-05-08T23:45:03.577891+0000","last_became_peered":"2024-05-08T23:45:03.577891+0000","last_unstale":"2024-05-08T23:45:06.282372+0000","last_undegraded":"2024-05-08T23:45:06.282372+0000","last_fullsized":"2024-05-08T23:45:06.282372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:01.027574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.061375+0000","last_change":"2024-05-08T23:45:04.060715+0000","last_active":"2024-05-08T23:45:04.061375+0000","last_peered":"2024-05-08T23:45:04.061375+0000","last_clean":"2024-05-08T23:45:04.061375+0000","last_became_active":"2024-05-08T23:45:04.060375+0000","last_became_peered":"2024-05-08T23:45:04.060375+0000","last_unstale":"2024-05-08T23:45:04.061375+0000","last_undegraded":"2024-05-08T23:45:04.061375+0000","last_fullsized":"2024-05-08T23:45:04.061375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:42.133323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061410+0000","last_change":"2024-05-08T23:45:03.587784+0000","last_active":"2024-05-08T23:45:07.061410+0000","last_peered":"2024-05-08T23:45:07.061410+0000","last_clean":"2024-05-08T23:45:07.061410+0000","last_became_active":"2024-05-08T23:45:03.587529+0000","last_became_peered":"2024-05-08T23:45:03.587529+0000","last_unstale":"2024-05-08T23:45:07.061410+0000","last_undegraded":"2024-05-08T23:45:07.061410+0000","last_fullsized":"2024-05-08T23:45:07.061410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:00.323599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300481+0000","last_change":"2024-05-08T23:45:04.062672+0000","last_active":"2024-05-08T23:45:04.300481+0000","last_peered":"2024-05-08T23:45:04.300481+0000","last_clean":"2024-05-08T23:45:04.300481+0000","last_became_active":"2024-05-08T23:45:04.062351+0000","last_became_peered":"2024-05-08T23:45:04.062351+0000","last_unstale":"2024-05-08T23:45:04.300481+0000","last_undegraded":"2024-05-08T23:45:04.300481+0000","last_fullsized":"2024-05-08T23:45:04.300481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:13.740239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781350+0000","last_change":"2024-05-08T23:45:03.583501+0000","last_active":"2024-05-08T23:45:06.781350+0000","last_peered":"2024-05-08T23:45:06.781350+0000","last_clean":"2024-05-08T23:45:06.781350+0000","last_became_active":"2024-05-08T23:45:03.583116+0000","last_became_peered":"2024-05-08T23:45:03.583116+0000","last_unstale":"2024-05-08T23:45:06.781350+0000","last_undegraded":"2024-05-08T23:45:06.781350+0000","last_fullsized":"2024-05-08T23:45:06.781350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:03.797524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282765+0000","last_change":"2024-05-08T23:45:04.048336+0000","last_active":"2024-05-08T23:45:06.282765+0000","last_peered":"2024-05-08T23:45:06.282765+0000","last_clean":"2024-05-08T23:45:06.282765+0000","last_became_active":"2024-05-08T23:45:04.048000+0000","last_became_peered":"2024-05-08T23:45:04.048000+0000","last_unstale":"2024-05-08T23:45:06.282765+0000","last_undegraded":"2024-05-08T23:45:06.282765+0000","last_fullsized":"2024-05-08T23:45:06.282765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:41.806600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062204+0000","last_change":"2024-05-08T23:45:03.581215+0000","last_active":"2024-05-08T23:45:07.062204+0000","last_peered":"2024-05-08T23:45:07.062204+0000","last_clean":"2024-05-08T23:45:07.062204+0000","last_became_active":"2024-05-08T23:45:03.580496+0000","last_became_peered":"2024-05-08T23:45:03.580496+0000","last_unstale":"2024-05-08T23:45:07.062204+0000","last_undegraded":"2024-05-08T23:45:07.062204+0000","last_fullsized":"2024-05-08T23:45:07.062204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:25.602401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300459+0000","last_change":"2024-05-08T23:45:04.058857+0000","last_active":"2024-05-08T23:45:04.300459+0000","last_peered":"2024-05-08T23:45:04.300459+0000","last_clean":"2024-05-08T23:45:04.300459+0000","last_became_active":"2024-05-08T23:45:04.057988+0000","last_became_peered":"2024-05-08T23:45:04.057988+0000","last_unstale":"2024-05-08T23:45:04.300459+0000","last_undegraded":"2024-05-08T23:45:04.300459+0000","last_fullsized":"2024-05-08T23:45:04.300459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:46.725976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'8","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061189+0000","last_change":"2024-05-08T23:45:03.582121+0000","last_active":"2024-05-08T23:45:07.061189+0000","last_peered":"2024-05-08T23:45:07.061189+0000","last_clean":"2024-05-08T23:45:07.061189+0000","last_became_active":"2024-05-08T23:45:03.581796+0000","last_became_peered":"2024-05-08T23:45:03.581796+0000","last_unstale":"2024-05-08T23:45:07.061189+0000","last_undegraded":"2024-05-08T23:45:07.061189+0000","last_fullsized":"2024-05-08T23:45:07.061189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:55.417404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517284+0000","last_change":"2024-05-08T23:45:04.052993+0000","last_active":"2024-05-08T23:45:05.517284+0000","last_peered":"2024-05-08T23:45:05.517284+0000","last_clean":"2024-05-08T23:45:05.517284+0000","last_became_active":"2024-05-08T23:45:04.052088+0000","last_became_peered":"2024-05-08T23:45:04.052088+0000","last_unstale":"2024-05-08T23:45:05.517284+0000","last_undegraded":"2024-05-08T23:45:05.517284+0000","last_fullsized":"2024-05-08T23:45:05.517284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:44.531649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225314+0000","last_change":"2024-05-08T23:45:03.575311+0000","last_active":"2024-05-08T23:45:06.225314+0000","last_peered":"2024-05-08T23:45:06.225314+0000","last_clean":"2024-05-08T23:45:06.225314+0000","last_became_active":"2024-05-08T23:45:03.574949+0000","last_became_peered":"2024-05-08T23:45:03.574949+0000","last_unstale":"2024-05-08T23:45:06.225314+0000","last_undegraded":"2024-05-08T23:45:06.225314+0000","last_fullsized":"2024-05-08T23:45:06.225314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:18.129488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022182+0000","last_change":"2024-05-08T23:45:04.050294+0000","last_active":"2024-05-08T23:45:05.022182+0000","last_peered":"2024-05-08T23:45:05.022182+0000","last_clean":"2024-05-08T23:45:05.022182+0000","last_became_active":"2024-05-08T23:45:04.049949+0000","last_became_peered":"2024-05-08T23:45:04.049949+0000","last_unstale":"2024-05-08T23:45:05.022182+0000","last_undegraded":"2024-05-08T23:45:05.022182+0000","last_fullsized":"2024-05-08T23:45:05.022182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:17.253201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061055+0000","last_change":"2024-05-08T23:45:03.579821+0000","last_active":"2024-05-08T23:45:07.061055+0000","last_peered":"2024-05-08T23:45:07.061055+0000","last_clean":"2024-05-08T23:45:07.061055+0000","last_became_active":"2024-05-08T23:45:03.578635+0000","last_became_peered":"2024-05-08T23:45:03.578635+0000","last_unstale":"2024-05-08T23:45:07.061055+0000","last_undegraded":"2024-05-08T23:45:07.061055+0000","last_fullsized":"2024-05-08T23:45:07.061055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:46.179505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781264+0000","last_change":"2024-05-08T23:45:04.059357+0000","last_active":"2024-05-08T23:45:06.781264+0000","last_peered":"2024-05-08T23:45:06.781264+0000","last_clean":"2024-05-08T23:45:06.781264+0000","last_became_active":"2024-05-08T23:45:04.059070+0000","last_became_peered":"2024-05-08T23:45:04.059070+0000","last_unstale":"2024-05-08T23:45:06.781264+0000","last_undegraded":"2024-05-08T23:45:06.781264+0000","last_fullsized":"2024-05-08T23:45:06.781264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:50.626787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.069255+0000","last_change":"2024-05-08T23:45:03.577199+0000","last_active":"2024-05-08T23:45:08.069255+0000","last_peered":"2024-05-08T23:45:08.069255+0000","last_clean":"2024-05-08T23:45:08.069255+0000","last_became_active":"2024-05-08T23:45:03.576844+0000","last_became_peered":"2024-05-08T23:45:03.576844+0000","last_unstale":"2024-05-08T23:45:08.069255+0000","last_undegraded":"2024-05-08T23:45:08.069255+0000","last_fullsized":"2024-05-08T23:45:08.069255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:27.139493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.780964+0000","last_change":"2024-05-08T23:45:04.064515+0000","last_active":"2024-05-08T23:45:06.780964+0000","last_peered":"2024-05-08T23:45:06.780964+0000","last_clean":"2024-05-08T23:45:06.780964+0000","last_became_active":"2024-05-08T23:45:04.064184+0000","last_became_peered":"2024-05-08T23:45:04.064184+0000","last_unstale":"2024-05-08T23:45:06.780964+0000","last_undegraded":"2024-05-08T23:45:06.780964+0000","last_fullsized":"2024-05-08T23:45:06.780964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:46.423121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.072701+0000","last_change":"2024-05-08T23:45:03.576766+0000","last_active":"2024-05-08T23:45:08.072701+0000","last_peered":"2024-05-08T23:45:08.072701+0000","last_clean":"2024-05-08T23:45:08.072701+0000","last_became_active":"2024-05-08T23:45:03.576410+0000","last_became_peered":"2024-05-08T23:45:03.576410+0000","last_unstale":"2024-05-08T23:45:08.072701+0000","last_undegraded":"2024-05-08T23:45:08.072701+0000","last_fullsized":"2024-05-08T23:45:08.072701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:29.444589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563841+0000","last_change":"2024-05-08T23:45:04.056563+0000","last_active":"2024-05-08T23:45:05.563841+0000","last_peered":"2024-05-08T23:45:05.563841+0000","last_clean":"2024-05-08T23:45:05.563841+0000","last_became_active":"2024-05-08T23:45:04.053132+0000","last_became_peered":"2024-05-08T23:45:04.053132+0000","last_unstale":"2024-05-08T23:45:05.563841+0000","last_undegraded":"2024-05-08T23:45:05.563841+0000","last_fullsized":"2024-05-08T23:45:05.563841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:41.625804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.283481+0000","last_change":"2024-05-08T23:45:03.580343+0000","last_active":"2024-05-08T23:45:06.283481+0000","last_peered":"2024-05-08T23:45:06.283481+0000","last_clean":"2024-05-08T23:45:06.283481+0000","last_became_active":"2024-05-08T23:45:03.578719+0000","last_became_peered":"2024-05-08T23:45:03.578719+0000","last_unstale":"2024-05-08T23:45:06.283481+0000","last_undegraded":"2024-05-08T23:45:06.283481+0000","last_fullsized":"2024-05-08T23:45:06.283481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:54.831252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040662+0000","last_change":"2024-05-08T23:45:04.050643+0000","last_active":"2024-05-08T23:45:05.040662+0000","last_peered":"2024-05-08T23:45:05.040662+0000","last_clean":"2024-05-08T23:45:05.040662+0000","last_became_active":"2024-05-08T23:45:04.050309+0000","last_became_peered":"2024-05-08T23:45:04.050309+0000","last_unstale":"2024-05-08T23:45:05.040662+0000","last_undegraded":"2024-05-08T23:45:05.040662+0000","last_fullsized":"2024-05-08T23:45:05.040662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:30.259530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.036233+0000","last_change":"2024-05-08T23:45:03.579590+0000","last_active":"2024-05-08T23:45:04.036233+0000","last_peered":"2024-05-08T23:45:04.036233+0000","last_clean":"2024-05-08T23:45:04.036233+0000","last_became_active":"2024-05-08T23:45:03.579213+0000","last_became_peered":"2024-05-08T23:45:03.579213+0000","last_unstale":"2024-05-08T23:45:04.036233+0000","last_undegraded":"2024-05-08T23:45:04.036233+0000","last_fullsized":"2024-05-08T23:45:04.036233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:55.666543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.044845+0000","last_change":"2024-05-08T23:45:04.049082+0000","last_active":"2024-05-08T23:45:05.044845+0000","last_peered":"2024-05-08T23:45:05.044845+0000","last_clean":"2024-05-08T23:45:05.044845+0000","last_became_active":"2024-05-08T23:45:04.048709+0000","last_became_peered":"2024-05-08T23:45:04.048709+0000","last_unstale":"2024-05-08T23:45:05.044845+0000","last_undegraded":"2024-05-08T23:45:05.044845+0000","last_fullsized":"2024-05-08T23:45:05.044845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:56.365817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781150+0000","last_change":"2024-05-08T23:45:03.581363+0000","last_active":"2024-05-08T23:45:06.781150+0000","last_peered":"2024-05-08T23:45:06.781150+0000","last_clean":"2024-05-08T23:45:06.781150+0000","last_became_active":"2024-05-08T23:45:03.580840+0000","last_became_peered":"2024-05-08T23:45:03.580840+0000","last_unstale":"2024-05-08T23:45:06.781150+0000","last_undegraded":"2024-05-08T23:45:06.781150+0000","last_fullsized":"2024-05-08T23:45:06.781150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:27.281228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045138+0000","last_change":"2024-05-08T23:45:04.053524+0000","last_active":"2024-05-08T23:45:05.045138+0000","last_peered":"2024-05-08T23:45:05.045138+0000","last_clean":"2024-05-08T23:45:05.045138+0000","last_became_active":"2024-05-08T23:45:04.053118+0000","last_became_peered":"2024-05-08T23:45:04.053118+0000","last_unstale":"2024-05-08T23:45:05.045138+0000","last_undegraded":"2024-05-08T23:45:05.045138+0000","last_fullsized":"2024-05-08T23:45:05.045138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:56.542262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563822+0000","last_change":"2024-05-08T23:45:04.061189+0000","last_active":"2024-05-08T23:45:05.563822+0000","last_peered":"2024-05-08T23:45:05.563822+0000","last_clean":"2024-05-08T23:45:05.563822+0000","last_became_active":"2024-05-08T23:45:04.060838+0000","last_became_peered":"2024-05-08T23:45:04.060838+0000","last_unstale":"2024-05-08T23:45:05.563822+0000","last_undegraded":"2024-05-08T23:45:05.563822+0000","last_fullsized":"2024-05-08T23:45:05.563822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:43.794987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.073509+0000","last_change":"2024-05-08T23:45:03.583841+0000","last_active":"2024-05-08T23:45:08.073509+0000","last_peered":"2024-05-08T23:45:08.073509+0000","last_clean":"2024-05-08T23:45:08.073509+0000","last_became_active":"2024-05-08T23:45:03.583275+0000","last_became_peered":"2024-05-08T23:45:03.583275+0000","last_unstale":"2024-05-08T23:45:08.073509+0000","last_undegraded":"2024-05-08T23:45:08.073509+0000","last_fullsized":"2024-05-08T23:45:08.073509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:14.560505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039949+0000","last_change":"2024-05-08T23:45:04.052667+0000","last_active":"2024-05-08T23:45:05.039949+0000","last_peered":"2024-05-08T23:45:05.039949+0000","last_clean":"2024-05-08T23:45:05.039949+0000","last_became_active":"2024-05-08T23:45:04.052008+0000","last_became_peered":"2024-05-08T23:45:04.052008+0000","last_unstale":"2024-05-08T23:45:05.039949+0000","last_undegraded":"2024-05-08T23:45:05.039949+0000","last_fullsized":"2024-05-08T23:45:05.039949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:35.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062503+0000","last_change":"2024-05-08T23:45:03.573665+0000","last_active":"2024-05-08T23:45:07.062503+0000","last_peered":"2024-05-08T23:45:07.062503+0000","last_clean":"2024-05-08T23:45:07.062503+0000","last_became_active":"2024-05-08T23:45:03.573265+0000","last_became_peered":"2024-05-08T23:45:03.573265+0000","last_unstale":"2024-05-08T23:45:07.062503+0000","last_undegraded":"2024-05-08T23:45:07.062503+0000","last_fullsized":"2024-05-08T23:45:07.062503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:13.410954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":329,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.038848+0000","last_change":"2024-05-08T23:44:16.373962+0000","last_active":"2024-05-08T23:45:04.038848+0000","last_peered":"2024-05-08T23:45:04.038848+0000","last_clean":"2024-05-08T23:45:04.038848+0000","last_became_active":"2024-05-08T23:44:16.373358+0000","last_became_peered":"2024-05-08T23:44:16.373358+0000","last_unstale":"2024-05-08T23:45:04.038848+0000","last_undegraded":"2024-05-08T23:45:04.038848+0000","last_fullsized":"2024-05-08T23:45:04.038848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.067168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:24.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022109+0000","last_change":"2024-05-08T23:45:04.048468+0000","last_active":"2024-05-08T23:45:05.022109+0000","last_peered":"2024-05-08T23:45:05.022109+0000","last_clean":"2024-05-08T23:45:05.022109+0000","last_became_active":"2024-05-08T23:45:04.048085+0000","last_became_peered":"2024-05-08T23:45:04.048085+0000","last_unstale":"2024-05-08T23:45:05.022109+0000","last_undegraded":"2024-05-08T23:45:05.022109+0000","last_fullsized":"2024-05-08T23:45:05.022109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:18.039780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781331+0000","last_change":"2024-05-08T23:45:03.575821+0000","last_active":"2024-05-08T23:45:06.781331+0000","last_peered":"2024-05-08T23:45:06.781331+0000","last_clean":"2024-05-08T23:45:06.781331+0000","last_became_active":"2024-05-08T23:45:03.575407+0000","last_became_peered":"2024-05-08T23:45:03.575407+0000","last_unstale":"2024-05-08T23:45:06.781331+0000","last_undegraded":"2024-05-08T23:45:06.781331+0000","last_fullsized":"2024-05-08T23:45:06.781331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:35.718942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.060787+0000","last_change":"2024-05-08T23:45:03.580100+0000","last_active":"2024-05-08T23:45:07.060787+0000","last_peered":"2024-05-08T23:45:07.060787+0000","last_clean":"2024-05-08T23:45:07.060787+0000","last_became_active":"2024-05-08T23:45:03.579791+0000","last_became_peered":"2024-05-08T23:45:03.579791+0000","last_unstale":"2024-05-08T23:45:07.060787+0000","last_undegraded":"2024-05-08T23:45:07.060787+0000","last_fullsized":"2024-05-08T23:45:07.060787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:41.086291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022221+0000","last_change":"2024-05-08T23:45:04.047687+0000","last_active":"2024-05-08T23:45:05.022221+0000","last_peered":"2024-05-08T23:45:05.022221+0000","last_clean":"2024-05-08T23:45:05.022221+0000","last_became_active":"2024-05-08T23:45:04.047343+0000","last_became_peered":"2024-05-08T23:45:04.047343+0000","last_unstale":"2024-05-08T23:45:05.022221+0000","last_undegraded":"2024-05-08T23:45:05.022221+0000","last_fullsized":"2024-05-08T23:45:05.022221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:58.763421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.070036+0000","last_change":"2024-05-08T23:45:03.573344+0000","last_active":"2024-05-08T23:45:08.070036+0000","last_peered":"2024-05-08T23:45:08.070036+0000","last_clean":"2024-05-08T23:45:08.070036+0000","last_became_active":"2024-05-08T23:45:03.572951+0000","last_became_peered":"2024-05-08T23:45:03.572951+0000","last_unstale":"2024-05-08T23:45:08.070036+0000","last_undegraded":"2024-05-08T23:45:08.070036+0000","last_fullsized":"2024-05-08T23:45:08.070036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.422908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040278+0000","last_change":"2024-05-08T23:45:04.055111+0000","last_active":"2024-05-08T23:45:05.040278+0000","last_peered":"2024-05-08T23:45:05.040278+0000","last_clean":"2024-05-08T23:45:05.040278+0000","last_became_active":"2024-05-08T23:45:04.054665+0000","last_became_peered":"2024-05-08T23:45:04.054665+0000","last_unstale":"2024-05-08T23:45:05.040278+0000","last_undegraded":"2024-05-08T23:45:05.040278+0000","last_fullsized":"2024-05-08T23:45:05.040278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:23.292630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062624+0000","last_change":"2024-05-08T23:45:03.578168+0000","last_active":"2024-05-08T23:45:07.062624+0000","last_peered":"2024-05-08T23:45:07.062624+0000","last_clean":"2024-05-08T23:45:07.062624+0000","last_became_active":"2024-05-08T23:45:03.577833+0000","last_became_peered":"2024-05-08T23:45:03.577833+0000","last_unstale":"2024-05-08T23:45:07.062624+0000","last_undegraded":"2024-05-08T23:45:07.062624+0000","last_fullsized":"2024-05-08T23:45:07.062624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:13:25.743151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282635+0000","last_change":"2024-05-08T23:45:04.049103+0000","last_active":"2024-05-08T23:45:06.282635+0000","last_peered":"2024-05-08T23:45:06.282635+0000","last_clean":"2024-05-08T23:45:06.282635+0000","last_became_active":"2024-05-08T23:45:04.048302+0000","last_became_peered":"2024-05-08T23:45:04.048302+0000","last_unstale":"2024-05-08T23:45:06.282635+0000","last_undegraded":"2024-05-08T23:45:06.282635+0000","last_fullsized":"2024-05-08T23:45:06.282635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:02.404522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061597+0000","last_change":"2024-05-08T23:45:03.577765+0000","last_active":"2024-05-08T23:45:07.061597+0000","last_peered":"2024-05-08T23:45:07.061597+0000","last_clean":"2024-05-08T23:45:07.061597+0000","last_became_active":"2024-05-08T23:45:03.577452+0000","last_became_peered":"2024-05-08T23:45:03.577452+0000","last_unstale":"2024-05-08T23:45:07.061597+0000","last_undegraded":"2024-05-08T23:45:07.061597+0000","last_fullsized":"2024-05-08T23:45:07.061597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:02.107878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517201+0000","last_change":"2024-05-08T23:45:04.061360+0000","last_active":"2024-05-08T23:45:05.517201+0000","last_peered":"2024-05-08T23:45:05.517201+0000","last_clean":"2024-05-08T23:45:05.517201+0000","last_became_active":"2024-05-08T23:45:04.061000+0000","last_became_peered":"2024-05-08T23:45:04.061000+0000","last_unstale":"2024-05-08T23:45:05.517201+0000","last_undegraded":"2024-05-08T23:45:05.517201+0000","last_fullsized":"2024-05-08T23:45:05.517201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:35.757571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061500+0000","last_change":"2024-05-08T23:45:03.578769+0000","last_active":"2024-05-08T23:45:07.061500+0000","last_peered":"2024-05-08T23:45:07.061500+0000","last_clean":"2024-05-08T23:45:07.061500+0000","last_became_active":"2024-05-08T23:45:03.578359+0000","last_became_peered":"2024-05-08T23:45:03.578359+0000","last_unstale":"2024-05-08T23:45:07.061500+0000","last_undegraded":"2024-05-08T23:45:07.061500+0000","last_fullsized":"2024-05-08T23:45:07.061500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:13.213382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040679+0000","last_change":"2024-05-08T23:45:04.049237+0000","last_active":"2024-05-08T23:45:05.040679+0000","last_peered":"2024-05-08T23:45:05.040679+0000","last_clean":"2024-05-08T23:45:05.040679+0000","last_became_active":"2024-05-08T23:45:04.048922+0000","last_became_peered":"2024-05-08T23:45:04.048922+0000","last_unstale":"2024-05-08T23:45:05.040679+0000","last_undegraded":"2024-05-08T23:45:05.040679+0000","last_fullsized":"2024-05-08T23:45:05.040679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:29.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225187+0000","last_change":"2024-05-08T23:45:03.574990+0000","last_active":"2024-05-08T23:45:06.225187+0000","last_peered":"2024-05-08T23:45:06.225187+0000","last_clean":"2024-05-08T23:45:06.225187+0000","last_became_active":"2024-05-08T23:45:03.574617+0000","last_became_peered":"2024-05-08T23:45:03.574617+0000","last_unstale":"2024-05-08T23:45:06.225187+0000","last_undegraded":"2024-05-08T23:45:06.225187+0000","last_fullsized":"2024-05-08T23:45:06.225187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:51.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039948+0000","last_change":"2024-05-08T23:45:04.054371+0000","last_active":"2024-05-08T23:45:05.039948+0000","last_peered":"2024-05-08T23:45:05.039948+0000","last_clean":"2024-05-08T23:45:05.039948+0000","last_became_active":"2024-05-08T23:45:04.054070+0000","last_became_peered":"2024-05-08T23:45:04.054070+0000","last_unstale":"2024-05-08T23:45:05.039948+0000","last_undegraded":"2024-05-08T23:45:05.039948+0000","last_fullsized":"2024-05-08T23:45:05.039948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:27.926522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062373+0000","last_change":"2024-05-08T23:45:03.579413+0000","last_active":"2024-05-08T23:45:07.062373+0000","last_peered":"2024-05-08T23:45:07.062373+0000","last_clean":"2024-05-08T23:45:07.062373+0000","last_became_active":"2024-05-08T23:45:03.577705+0000","last_became_peered":"2024-05-08T23:45:03.577705+0000","last_unstale":"2024-05-08T23:45:07.062373+0000","last_undegraded":"2024-05-08T23:45:07.062373+0000","last_fullsized":"2024-05-08T23:45:07.062373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:29.615247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781372+0000","last_change":"2024-05-08T23:45:04.050492+0000","last_active":"2024-05-08T23:45:06.781372+0000","last_peered":"2024-05-08T23:45:06.781372+0000","last_clean":"2024-05-08T23:45:06.781372+0000","last_became_active":"2024-05-08T23:45:04.050176+0000","last_became_peered":"2024-05-08T23:45:04.050176+0000","last_unstale":"2024-05-08T23:45:06.781372+0000","last_undegraded":"2024-05-08T23:45:06.781372+0000","last_fullsized":"2024-05-08T23:45:06.781372+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:01.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225570+0000","last_change":"2024-05-08T23:45:03.577651+0000","last_active":"2024-05-08T23:45:06.225570+0000","last_peered":"2024-05-08T23:45:06.225570+0000","last_clean":"2024-05-08T23:45:06.225570+0000","last_became_active":"2024-05-08T23:45:03.576887+0000","last_became_peered":"2024-05-08T23:45:03.576887+0000","last_unstale":"2024-05-08T23:45:06.225570+0000","last_undegraded":"2024-05-08T23:45:06.225570+0000","last_fullsized":"2024-05-08T23:45:06.225570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:52.822959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781487+0000","last_change":"2024-05-08T23:45:04.049793+0000","last_active":"2024-05-08T23:45:06.781487+0000","last_peered":"2024-05-08T23:45:06.781487+0000","last_clean":"2024-05-08T23:45:06.781487+0000","last_became_active":"2024-05-08T23:45:04.049456+0000","last_became_peered":"2024-05-08T23:45:04.049456+0000","last_unstale":"2024-05-08T23:45:06.781487+0000","last_undegraded":"2024-05-08T23:45:06.781487+0000","last_fullsized":"2024-05-08T23:45:06.781487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:16.030406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282639+0000","last_change":"2024-05-08T23:45:03.580371+0000","last_active":"2024-05-08T23:45:06.282639+0000","last_peered":"2024-05-08T23:45:06.282639+0000","last_clean":"2024-05-08T23:45:06.282639+0000","last_became_active":"2024-05-08T23:45:03.579394+0000","last_became_peered":"2024-05-08T23:45:03.579394+0000","last_unstale":"2024-05-08T23:45:06.282639+0000","last_undegraded":"2024-05-08T23:45:06.282639+0000","last_fullsized":"2024-05-08T23:45:06.282639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.123632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781264+0000","last_change":"2024-05-08T23:45:04.063138+0000","last_active":"2024-05-08T23:45:06.781264+0000","last_peered":"2024-05-08T23:45:06.781264+0000","last_clean":"2024-05-08T23:45:06.781264+0000","last_became_active":"2024-05-08T23:45:04.062835+0000","last_became_peered":"2024-05-08T23:45:04.062835+0000","last_unstale":"2024-05-08T23:45:06.781264+0000","last_undegraded":"2024-05-08T23:45:06.781264+0000","last_fullsized":"2024-05-08T23:45:06.781264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:49.538896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022233+0000","last_change":"2024-05-08T23:45:03.584351+0000","last_active":"2024-05-08T23:45:05.022233+0000","last_peered":"2024-05-08T23:45:05.022233+0000","last_clean":"2024-05-08T23:45:05.022233+0000","last_became_active":"2024-05-08T23:45:03.583996+0000","last_became_peered":"2024-05-08T23:45:03.583996+0000","last_unstale":"2024-05-08T23:45:05.022233+0000","last_undegraded":"2024-05-08T23:45:05.022233+0000","last_fullsized":"2024-05-08T23:45:05.022233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:40.530921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781432+0000","last_change":"2024-05-08T23:45:04.048918+0000","last_active":"2024-05-08T23:45:06.781432+0000","last_peered":"2024-05-08T23:45:06.781432+0000","last_clean":"2024-05-08T23:45:06.781432+0000","last_became_active":"2024-05-08T23:45:04.048550+0000","last_became_peered":"2024-05-08T23:45:04.048550+0000","last_unstale":"2024-05-08T23:45:06.781432+0000","last_undegraded":"2024-05-08T23:45:06.781432+0000","last_fullsized":"2024-05-08T23:45:06.781432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:48.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040646+0000","last_change":"2024-05-08T23:45:04.050790+0000","last_active":"2024-05-08T23:45:05.040646+0000","last_peered":"2024-05-08T23:45:05.040646+0000","last_clean":"2024-05-08T23:45:05.040646+0000","last_became_active":"2024-05-08T23:45:04.050429+0000","last_became_peered":"2024-05-08T23:45:04.050429+0000","last_unstale":"2024-05-08T23:45:05.040646+0000","last_undegraded":"2024-05-08T23:45:05.040646+0000","last_fullsized":"2024-05-08T23:45:05.040646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:43.224351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225201+0000","last_change":"2024-05-08T23:45:04.062063+0000","last_active":"2024-05-08T23:45:06.225201+0000","last_peered":"2024-05-08T23:45:06.225201+0000","last_clean":"2024-05-08T23:45:06.225201+0000","last_became_active":"2024-05-08T23:45:04.061206+0000","last_became_peered":"2024-05-08T23:45:04.061206+0000","last_unstale":"2024-05-08T23:45:06.225201+0000","last_undegraded":"2024-05-08T23:45:06.225201+0000","last_fullsized":"2024-05-08T23:45:06.225201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:23.137397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225238+0000","last_change":"2024-05-08T23:45:04.049707+0000","last_active":"2024-05-08T23:45:06.225238+0000","last_peered":"2024-05-08T23:45:06.225238+0000","last_clean":"2024-05-08T23:45:06.225238+0000","last_became_active":"2024-05-08T23:45:04.049385+0000","last_became_peered":"2024-05-08T23:45:04.049385+0000","last_unstale":"2024-05-08T23:45:06.225238+0000","last_undegraded":"2024-05-08T23:45:06.225238+0000","last_fullsized":"2024-05-08T23:45:06.225238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282351+0000","last_change":"2024-05-08T23:45:04.050721+0000","last_active":"2024-05-08T23:45:06.282351+0000","last_peered":"2024-05-08T23:45:06.282351+0000","last_clean":"2024-05-08T23:45:06.282351+0000","last_became_active":"2024-05-08T23:45:04.050368+0000","last_became_peered":"2024-05-08T23:45:04.050368+0000","last_unstale":"2024-05-08T23:45:06.282351+0000","last_undegraded":"2024-05-08T23:45:06.282351+0000","last_fullsized":"2024-05-08T23:45:06.282351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:20.815754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022211+0000","last_change":"2024-05-08T23:45:04.048431+0000","last_active":"2024-05-08T23:45:05.022211+0000","last_peered":"2024-05-08T23:45:05.022211+0000","last_clean":"2024-05-08T23:45:05.022211+0000","last_became_active":"2024-05-08T23:45:04.048071+0000","last_became_peered":"2024-05-08T23:45:04.048071+0000","last_unstale":"2024-05-08T23:45:05.022211+0000","last_undegraded":"2024-05-08T23:45:05.022211+0000","last_fullsized":"2024-05-08T23:45:05.022211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:51.547256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040715+0000","last_change":"2024-05-08T23:45:04.045256+0000","last_active":"2024-05-08T23:45:05.040715+0000","last_peered":"2024-05-08T23:45:05.040715+0000","last_clean":"2024-05-08T23:45:05.040715+0000","last_became_active":"2024-05-08T23:45:04.044973+0000","last_became_peered":"2024-05-08T23:45:04.044973+0000","last_unstale":"2024-05-08T23:45:05.040715+0000","last_undegraded":"2024-05-08T23:45:05.040715+0000","last_fullsized":"2024-05-08T23:45:05.040715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:06.021001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225634+0000","last_change":"2024-05-08T23:45:04.058287+0000","last_active":"2024-05-08T23:45:06.225634+0000","last_peered":"2024-05-08T23:45:06.225634+0000","last_clean":"2024-05-08T23:45:06.225634+0000","last_became_active":"2024-05-08T23:45:04.057977+0000","last_became_peered":"2024-05-08T23:45:04.057977+0000","last_unstale":"2024-05-08T23:45:06.225634+0000","last_undegraded":"2024-05-08T23:45:06.225634+0000","last_fullsized":"2024-05-08T23:45:06.225634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:01.146838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040216+0000","last_change":"2024-05-08T23:45:04.055780+0000","last_active":"2024-05-08T23:45:05.040216+0000","last_peered":"2024-05-08T23:45:05.040216+0000","last_clean":"2024-05-08T23:45:05.040216+0000","last_became_active":"2024-05-08T23:45:04.055272+0000","last_became_peered":"2024-05-08T23:45:04.055272+0000","last_unstale":"2024-05-08T23:45:05.040216+0000","last_undegraded":"2024-05-08T23:45:05.040216+0000","last_fullsized":"2024-05-08T23:45:05.040216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:28.587333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563811+0000","last_change":"2024-05-08T23:45:04.050648+0000","last_active":"2024-05-08T23:45:05.563811+0000","last_peered":"2024-05-08T23:45:05.563811+0000","last_clean":"2024-05-08T23:45:05.563811+0000","last_became_active":"2024-05-08T23:45:04.050353+0000","last_became_peered":"2024-05-08T23:45:04.050353+0000","last_unstale":"2024-05-08T23:45:05.563811+0000","last_undegraded":"2024-05-08T23:45:05.563811+0000","last_fullsized":"2024-05-08T23:45:05.563811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:58.154006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039875+0000","last_change":"2024-05-08T23:45:04.056951+0000","last_active":"2024-05-08T23:45:05.039875+0000","last_peered":"2024-05-08T23:45:05.039875+0000","last_clean":"2024-05-08T23:45:05.039875+0000","last_became_active":"2024-05-08T23:45:04.056585+0000","last_became_peered":"2024-05-08T23:45:04.056585+0000","last_unstale":"2024-05-08T23:45:05.039875+0000","last_undegraded":"2024-05-08T23:45:05.039875+0000","last_fullsized":"2024-05-08T23:45:05.039875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:22.658629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300447+0000","last_change":"2024-05-08T23:45:04.052396+0000","last_active":"2024-05-08T23:45:04.300447+0000","last_peered":"2024-05-08T23:45:04.300447+0000","last_clean":"2024-05-08T23:45:04.300447+0000","last_became_active":"2024-05-08T23:45:04.051604+0000","last_became_peered":"2024-05-08T23:45:04.051604+0000","last_unstale":"2024-05-08T23:45:04.300447+0000","last_undegraded":"2024-05-08T23:45:04.300447+0000","last_fullsized":"2024-05-08T23:45:04.300447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:52.602830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022201+0000","last_change":"2024-05-08T23:45:04.051578+0000","last_active":"2024-05-08T23:45:05.022201+0000","last_peered":"2024-05-08T23:45:05.022201+0000","last_clean":"2024-05-08T23:45:05.022201+0000","last_became_active":"2024-05-08T23:45:04.051298+0000","last_became_peered":"2024-05-08T23:45:04.051298+0000","last_unstale":"2024-05-08T23:45:05.022201+0000","last_undegraded":"2024-05-08T23:45:05.022201+0000","last_fullsized":"2024-05-08T23:45:05.022201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:47.204849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040483+0000","last_change":"2024-05-08T23:45:04.051264+0000","last_active":"2024-05-08T23:45:05.040483+0000","last_peered":"2024-05-08T23:45:05.040483+0000","last_clean":"2024-05-08T23:45:05.040483+0000","last_became_active":"2024-05-08T23:45:04.050870+0000","last_became_peered":"2024-05-08T23:45:04.050870+0000","last_unstale":"2024-05-08T23:45:05.040483+0000","last_undegraded":"2024-05-08T23:45:05.040483+0000","last_fullsized":"2024-05-08T23:45:05.040483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4496,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":107,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":11464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":876544,"data_stored":421419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":10,"up_from":69,"seq":296352743438,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1013932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":9,"up_from":64,"seq":274877906962,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":421655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54300000000000004}]},{"osd":2,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.61099999999999999}]},{"osd":3,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.754}]},{"osd":5,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.82899999999999996}]},{"osd":7,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79100000000000004}]},{"osd":8,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.877},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89500000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":888832,"data_stored":421244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.53500000000000003}]},{"osd":1,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58099999999999996}]},{"osd":2,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.65300000000000002}]},{"osd":4,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.627},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.69799999999999995}]},{"osd":5,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.46000000000000002}]},{"osd":6,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.746}]},{"osd":7,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68899999999999995}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.75800000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":7,"up_from":49,"seq":210453397528,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32544,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712608,"statfs":{"total":95995035648,"available":95961710592,"internally_reserved":0,"allocated":1454080,"data_stored":1002886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.628},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59099999999999997}]},{"osd":4,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.42699999999999999}]},{"osd":5,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56299999999999994}]},{"osd":8,"last update":"Wed May 8 23:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.66300000000000003}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.67200000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561052,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":876544,"data_stored":420927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.46500000000000002,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.442}]},{"osd":1,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.622},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.64800000000000002,"15min":0.66200000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79400000000000004,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.56299999999999994,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92400000000000004,"5min":1.288,"15min":1.288},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.505},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.44800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89000000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.39300000000000002}]},{"osd":4,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":0.97799999999999998,"5min":1.113,"15min":1.113},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.496,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59299999999999997}]},{"osd":7,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.73899999999999999}]},{"osd":8,"last update":"Wed May 8 23:44:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.68500000000000005}]},{"osd":9,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.66600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36068,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709084,"statfs":{"total":95995035648,"available":95958102016,"internally_reserved":0,"allocated":876544,"data_stored":421761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.505,"15min":0.48899999999999999},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.51300000000000001,"15min":0.496},"min":{"1min":0.37,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.48099999999999998}]},{"osd":2,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.54400000000000004}]},{"osd":3,"last update":"Wed May 8 23:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79000000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.38700000000000001,"15min":0.378},"min":{"1min":0.29299999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.41199999999999998,"15min":0.40300000000000002},"min":{"1min":0.32200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.42299999999999999}]},{"osd":5,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.41599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63800000000000001}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88400000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.61399999999999999}]},{"osd":7,"last update":"Wed May 8 23:44:17 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed May 8 23:44:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69699999999999995}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.72799999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738418,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":411990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:44:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.51600000000000001,"15min":0.495},"min":{"1min":0.34000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.38600000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.36299999999999999}]},{"osd":2,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.51800000000000002},"min":{"1min":0.40400000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.47099999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed May 8 23:44:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.41399999999999998,"15min":0.39400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.42099999999999999,"15min":0.40600000000000003},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.751}]},{"osd":4,"last update":"Wed May 8 23:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56100000000000005,"15min":0.55200000000000005},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.495,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.502}]},{"osd":5,"last update":"Wed May 8 23:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.52700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.46200000000000002}]},{"osd":6,"last update":"Wed May 8 23:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.39500000000000002}]},{"osd":7,"last update":"Wed May 8 23:44:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997}]},{"osd":8,"last update":"Wed May 8 23:44:37 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57499999999999996}]},{"osd":9,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68600000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":2,"up_from":17,"seq":73014444075,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":868352,"data_stored":413225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53700000000000003,"15min":0.52000000000000002},"min":{"1min":0.42999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53500000000000003,"15min":0.51300000000000001},"min":{"1min":0.41999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.501,"15min":0.48099999999999998},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.49199999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.79600000000000004}]},{"osd":3,"last update":"Wed May 8 23:44:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.753},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.72499999999999998}]},{"osd":4,"last update":"Wed May 8 23:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.91400000000000003}]},{"osd":5,"last update":"Wed May 8 23:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.38600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed May 8 23:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.5,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.42199999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.98699999999999999}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed May 8 23:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":1.014}]},{"osd":9,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.94999999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215144,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36076,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709076,"statfs":{"total":95995035648,"available":95958093824,"internally_reserved":0,"allocated":888832,"data_stored":421121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.373,"15min":0.36899999999999999},"min":{"1min":0.28999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58299999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.308,"5min":0.248,"15min":0.248},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed May 8 23:44:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed May 8 23:44:10 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.49399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Wed May 8 23:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.504,"5min":0.434,"15min":0.434},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003}]},{"osd":5,"last update":"Wed May 8 23:44:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.47699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.88100000000000001}]},{"osd":8,"last update":"Wed May 8 23:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed May 8 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.747}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986210,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":868352,"data_stored":420219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.28199999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.432},{"interface":"front","average":{"1min":0.437,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.58599999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81000000000000005,"5min":1.222,"15min":1.222},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.57099999999999995,"15min":0.55400000000000005},"min":{"1min":0.48399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.75900000000000001}]},{"osd":3,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54100000000000004,"15min":0.53200000000000003},"min":{"1min":0.379,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.62},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed May 8 23:44:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89700000000000002,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.91900000000000004,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed May 8 23:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.50900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.504}]},{"osd":7,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.47899999999999998}]},{"osd":8,"last update":"Wed May 8 23:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86099999999999999}]},{"osd":9,"last update":"Wed May 8 23:44:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.79300000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":5,"up_from":36,"seq":154618822689,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32572,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712580,"statfs":{"total":95995035648,"available":95961681920,"internally_reserved":0,"allocated":1495040,"data_stored":1012633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.36399999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51300000000000001,"15min":0.505},"min":{"1min":0.38600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.502},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.625},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.86199999999999999,"5min":0.873,"15min":0.873},"last":0.68100000000000005}]},{"osd":2,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47599999999999998}]},{"osd":3,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.49299999999999999,"15min":0.47899999999999998},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.434,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56799999999999995}]},{"osd":4,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.40999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58399999999999996}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.73699999999999999}]},{"osd":8,"last update":"Wed May 8 23:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.66000000000000003}]},{"osd":9,"last update":"Wed May 8 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.751},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:45:12.948 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:45:12.948 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T23:45:12.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T23:45:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:13.392 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:13 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/266007972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:13.392 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:13 smithi012 ceph-mon[42503]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:13.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:13 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/266007972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:13.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:13 smithi204 ceph-mon[47373]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:13.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:13 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/266007972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:13.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:13 smithi160 ceph-mon[46152]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:14.254 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:14 smithi012 ceph-mon[42503]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:14.254 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:14 smithi012 ceph-mon[42503]: pgmap v230: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2024-05-08T23:45:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.464+0000 7fb12e38c700 1 -- 172.21.15.12:0/3378857830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b80 msgr2=0x7fb128109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.464+0000 7fb12e38c700 1 --2- 172.21.15.12:0/3378857830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b80 0x7fb128109430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb12000b0d0 tx=0x7fb12000b3e0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.464+0000 7fb12e38c700 1 -- 172.21.15.12:0/3378857830 shutdown_connections 2024-05-08T23:45:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.464+0000 7fb12e38c700 1 --2- 172.21.15.12:0/3378857830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b80 0x7fb128109430 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.464+0000 7fb12e38c700 1 --2- 172.21.15.12:0/3378857830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb128100fe0 0x7fb128101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.464+0000 7fb12e38c700 1 --2- 172.21.15.12:0/3378857830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1281021e0 0x7fb128102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.464+0000 7fb12e38c700 1 -- 172.21.15.12:0/3378857830 >> 172.21.15.12:0/3378857830 conn(0x7fb1280fc5a0 msgr2=0x7fb1280fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.464+0000 7fb12e38c700 1 -- 172.21.15.12:0/3378857830 shutdown_connections 2024-05-08T23:45:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.464+0000 7fb12e38c700 1 -- 172.21.15.12:0/3378857830 wait complete. 2024-05-08T23:45:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.465+0000 7fb12e38c700 1 Processor -- start 2024-05-08T23:45:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.465+0000 7fb12e38c700 1 -- start start 2024-05-08T23:45:14.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.465+0000 7fb12e38c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb128100fe0 0x7fb128197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.465+0000 7fb12e38c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1281021e0 0x7fb128197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.465+0000 7fb12e38c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b80 0x7fb12819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.465+0000 7fb12e38c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12819eb70 con 0x7fb128102b80 2024-05-08T23:45:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.465+0000 7fb12e38c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12819ecb0 con 0x7fb1281021e0 2024-05-08T23:45:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.465+0000 7fb12e38c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12819ee40 con 0x7fb128100fe0 2024-05-08T23:45:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12c929700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b80 0x7fb12819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12c929700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b80 0x7fb12819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37682/0 (socket says 172.21.15.12:37682) 2024-05-08T23:45:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12c929700 1 -- 172.21.15.12:0/794363939 learned_addr learned my addr 172.21.15.12:0/794363939 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb127fff700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb128100fe0 0x7fb128197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb1277fe700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1281021e0 0x7fb128197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12c929700 1 -- 172.21.15.12:0/794363939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb128100fe0 msgr2=0x7fb128197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12c929700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb128100fe0 0x7fb128197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12c929700 1 -- 172.21.15.12:0/794363939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1281021e0 msgr2=0x7fb128197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12c929700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1281021e0 0x7fb128197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12c929700 1 -- 172.21.15.12:0/794363939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120009d00 con 0x7fb128102b80 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12c929700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b80 0x7fb12819e5a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb120008e90 tx=0x7fb120008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb1257fa700 1 -- 172.21.15.12:0/794363939 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12000e040 con 0x7fb128102b80 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb1257fa700 1 -- 172.21.15.12:0/794363939 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb12001d930 con 0x7fb128102b80 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12e38c700 1 -- 172.21.15.12:0/794363939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12819f0c0 con 0x7fb128102b80 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb1257fa700 1 -- 172.21.15.12:0/794363939 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12001b960 con 0x7fb128102b80 2024-05-08T23:45:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.466+0000 7fb12e38c700 1 -- 172.21.15.12:0/794363939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12819f530 con 0x7fb128102b80 2024-05-08T23:45:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.467+0000 7fb12e38c700 1 -- 172.21.15.12:0/794363939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb128108420 con 0x7fb128102b80 2024-05-08T23:45:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.469+0000 7fb1257fa700 1 -- 172.21.15.12:0/794363939 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb12001daa0 con 0x7fb128102b80 2024-05-08T23:45:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.469+0000 7fb1257fa700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb1100a11b0 0x7fb1100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.469+0000 7fb127fff700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb1100a11b0 0x7fb1100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.469+0000 7fb1257fa700 1 -- 172.21.15.12:0/794363939 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb1200cdce0 con 0x7fb128102b80 2024-05-08T23:45:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.470+0000 7fb127fff700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb1100a11b0 0x7fb1100a3670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb128102040 tx=0x7fb11800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.474+0000 7fb1257fa700 1 -- 172.21.15.12:0/794363939 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb120097550 con 0x7fb128102b80 2024-05-08T23:45:14.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:14 smithi204 ceph-mon[47373]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:14.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:14 smithi204 ceph-mon[47373]: pgmap v230: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2024-05-08T23:45:14.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:14 smithi160 ceph-mon[46152]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:14.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:14 smithi160 ceph-mon[46152]: pgmap v230: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2024-05-08T23:45:14.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.772+0000 7fb12e38c700 1 -- 172.21.15.12:0/794363939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb12810c9e0 con 0x7fb128102b80 2024-05-08T23:45:14.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.776+0000 7fb1257fa700 1 -- 172.21.15.12:0/794363939 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb12009ab70 con 0x7fb128102b80 2024-05-08T23:45:14.772 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-05-08T23:45:14.772 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCKDjxmcssbLhAA5m/OW/Yhan7MtHe7w8GI1Q== 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 -- 172.21.15.12:0/794363939 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb1100a11b0 msgr2=0x7fb1100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb1100a11b0 0x7fb1100a3670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb128102040 tx=0x7fb11800b410 comp rx=0 tx=0).stop 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 -- 172.21.15.12:0/794363939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b80 msgr2=0x7fb12819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b80 0x7fb12819e5a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb120008e90 tx=0x7fb120008ec0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 -- 172.21.15.12:0/794363939 shutdown_connections 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b80 0x7fb12819e5a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb128100fe0 0x7fb128197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb1100a11b0 0x7fb1100a3670 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 --2- 172.21.15.12:0/794363939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1281021e0 0x7fb128197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 -- 172.21.15.12:0/794363939 >> 172.21.15.12:0/794363939 conn(0x7fb1280fc5a0 msgr2=0x7fb128105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 -- 172.21.15.12:0/794363939 shutdown_connections 2024-05-08T23:45:14.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:14.779+0000 7fb12e38c700 1 -- 172.21.15.12:0/794363939 wait complete. 2024-05-08T23:45:15.138 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:45:15.138 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T23:45:15.138 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T23:45:15.191 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T23:45:15.191 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T23:45:15.191 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T23:45:15.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:15 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/794363939' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:45:15.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:15 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/794363939' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T23:45:15.488 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:15.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:15 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/794363939' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:45:15.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:15 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/794363939' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T23:45:15.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:15 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/794363939' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:45:15.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:15 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/794363939' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T23:45:16.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:16 smithi012 ceph-mon[42503]: pgmap v231: 97 pgs: 97 active+clean; 617 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2024-05-08T23:45:16.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:16 smithi204 ceph-mon[47373]: pgmap v231: 97 pgs: 97 active+clean; 617 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2024-05-08T23:45:16.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:16 smithi160 ceph-mon[46152]: pgmap v231: 97 pgs: 97 active+clean; 617 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2024-05-08T23:45:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.692+0000 7f435e7d4700 1 -- 172.21.15.12:0/525385694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4358106250 msgr2=0x7f435810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.692+0000 7f435e7d4700 1 --2- 172.21.15.12:0/525385694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4358106250 0x7f435810de20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f435000b0a0 tx=0x7f435000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.692+0000 7f435e7d4700 1 -- 172.21.15.12:0/525385694 shutdown_connections 2024-05-08T23:45:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.692+0000 7f435e7d4700 1 --2- 172.21.15.12:0/525385694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4358106250 0x7f435810de20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.692+0000 7f435e7d4700 1 --2- 172.21.15.12:0/525385694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4358101170 0x7f4358101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.692+0000 7f435e7d4700 1 --2- 172.21.15.12:0/525385694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4358101b20 0x7f4358105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.692+0000 7f435e7d4700 1 -- 172.21.15.12:0/525385694 >> 172.21.15.12:0/525385694 conn(0x7f43580fc9c0 msgr2=0x7f43580fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.692+0000 7f435e7d4700 1 -- 172.21.15.12:0/525385694 shutdown_connections 2024-05-08T23:45:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.692+0000 7f435e7d4700 1 -- 172.21.15.12:0/525385694 wait complete. 2024-05-08T23:45:16.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.693+0000 7f435e7d4700 1 Processor -- start 2024-05-08T23:45:16.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.693+0000 7f435e7d4700 1 -- start start 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f435e7d4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4358101170 0x7f43580732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f435e7d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4358101b20 0x7f43580737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f435e7d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4358106250 0x7f4358073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f435e7d4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43581a57e0 con 0x7f4358101b20 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f435e7d4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43581a5950 con 0x7f4358106250 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f435e7d4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43581a5b10 con 0x7f4358101170 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4358101b20 0x7f43580737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4358101b20 0x7f43580737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37702/0 (socket says 172.21.15.12:37702) 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43577fe700 1 -- 172.21.15.12:0/1795137025 learned_addr learned my addr 172.21.15.12:0/1795137025 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f4357fff700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4358101170 0x7f43580732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f435cd71700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4358106250 0x7f4358073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43577fe700 1 -- 172.21.15.12:0/1795137025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4358101170 msgr2=0x7f43580732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43577fe700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4358101170 0x7f43580732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43577fe700 1 -- 172.21.15.12:0/1795137025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4358106250 msgr2=0x7f4358073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43577fe700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4358106250 0x7f4358073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43577fe700 1 -- 172.21.15.12:0/1795137025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4350009d00 con 0x7f4358101b20 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43577fe700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4358101b20 0x7f43580737e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e970 tx=0x7f434c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43557fa700 1 -- 172.21.15.12:0/1795137025 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c009790 con 0x7f4358101b20 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.694+0000 7f43557fa700 1 -- 172.21.15.12:0/1795137025 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f434c00c7e0 con 0x7f4358101b20 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.695+0000 7f435e7d4700 1 -- 172.21.15.12:0/1795137025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43581a5df0 con 0x7f4358101b20 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.695+0000 7f43557fa700 1 -- 172.21.15.12:0/1795137025 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0054e0 con 0x7f4358101b20 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.695+0000 7f435e7d4700 1 -- 172.21.15.12:0/1795137025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43581a6270 con 0x7f4358101b20 2024-05-08T23:45:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.696+0000 7f435e7d4700 1 -- 172.21.15.12:0/1795137025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4358109510 con 0x7f4358101b20 2024-05-08T23:45:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.697+0000 7f43557fa700 1 -- 172.21.15.12:0/1795137025 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f434c00c950 con 0x7f4358101b20 2024-05-08T23:45:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.697+0000 7f43557fa700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f43400a1230 0x7f43400a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.698+0000 7f4357fff700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f43400a1230 0x7f43400a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.698+0000 7f43557fa700 1 -- 172.21.15.12:0/1795137025 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f434c014070 con 0x7f4358101b20 2024-05-08T23:45:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.698+0000 7f4357fff700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f43400a1230 0x7f43400a36f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4348007be0 tx=0x7f43480073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:16.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.702+0000 7f43557fa700 1 -- 172.21.15.12:0/1795137025 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f434c0963c0 con 0x7f4358101b20 2024-05-08T23:45:16.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.977+0000 7f435e7d4700 1 -- 172.21.15.12:0/1795137025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4358102650 con 0x7f4358101b20 2024-05-08T23:45:16.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.983+0000 7f43557fa700 1 -- 172.21.15.12:0/1795137025 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260285 (secure 0 0 0) 0x7f434c0999e0 con 0x7f4358101b20 2024-05-08T23:45:16.983 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 -- 172.21.15.12:0/1795137025 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f43400a1230 msgr2=0x7f43400a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f43400a1230 0x7f43400a36f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4348007be0 tx=0x7f43480073d0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 -- 172.21.15.12:0/1795137025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4358101b20 msgr2=0x7f43580737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4358101b20 0x7f43580737e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e970 tx=0x7f434c00ec80 comp rx=0 tx=0).stop 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 -- 172.21.15.12:0/1795137025 shutdown_connections 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f43400a1230 0x7f43400a36f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4358106250 0x7f4358073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4358101170 0x7f43580732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 --2- 172.21.15.12:0/1795137025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4358101b20 0x7f43580737e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 -- 172.21.15.12:0/1795137025 >> 172.21.15.12:0/1795137025 conn(0x7f43580fc9c0 msgr2=0x7f43580fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 -- 172.21.15.12:0/1795137025 shutdown_connections 2024-05-08T23:45:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:16.987+0000 7f435e7d4700 1 -- 172.21.15.12:0/1795137025 wait complete. 2024-05-08T23:45:17.380 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:17 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1795137025' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T23:45:17.421 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":1305168755},{"type":"v1","addr":"172.21.15.12:6801","nonce":1305168755}]},"active_addr":"172.21.15.12:6801/1305168755","active_change":"2024-05-08T23:39:55.189183+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.12:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3622028746}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3062429456}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2381834553}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2695290437}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":716473668}]}]} 2024-05-08T23:45:17.426 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T23:45:17.426 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T23:45:17.426 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T23:45:17.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:17 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1795137025' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T23:45:17.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:17 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1795137025' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T23:45:17.693 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:18.372 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:18 smithi012 ceph-mon[42503]: pgmap v232: 97 pgs: 97 active+clean; 617 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-05-08T23:45:18.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:18 smithi204 ceph-mon[47373]: pgmap v232: 97 pgs: 97 active+clean; 617 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-05-08T23:45:18.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:18 smithi160 ceph-mon[46152]: pgmap v232: 97 pgs: 97 active+clean; 617 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-05-08T23:45:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.888+0000 7f2751922700 1 -- 172.21.15.12:0/790786556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274c10d9d0 msgr2=0x7f274c10fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.888+0000 7f2751922700 1 --2- 172.21.15.12:0/790786556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274c10d9d0 0x7f274c10fdc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f274400b0a0 tx=0x7f274400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.888+0000 7f2751922700 1 -- 172.21.15.12:0/790786556 shutdown_connections 2024-05-08T23:45:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.888+0000 7f2751922700 1 --2- 172.21.15.12:0/790786556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274c10d9d0 0x7f274c10fdc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.888+0000 7f2751922700 1 --2- 172.21.15.12:0/790786556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c0731d0 0x7f274c0735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.888+0000 7f2751922700 1 --2- 172.21.15.12:0/790786556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f274c073af0 0x7f274c10d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.889+0000 7f2751922700 1 -- 172.21.15.12:0/790786556 >> 172.21.15.12:0/790786556 conn(0x7f274c078100 msgr2=0x7f274c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.889+0000 7f2751922700 1 -- 172.21.15.12:0/790786556 shutdown_connections 2024-05-08T23:45:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.889+0000 7f2751922700 1 -- 172.21.15.12:0/790786556 wait complete. 2024-05-08T23:45:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.889+0000 7f2751922700 1 Processor -- start 2024-05-08T23:45:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f2751922700 1 -- start start 2024-05-08T23:45:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f2751922700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c0731d0 0x7f274c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f2751922700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274c073af0 0x7f274c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f2751922700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f274c10d9d0 0x7f274c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f2751922700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c19d5f0 con 0x7f274c073af0 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f2751922700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c19d760 con 0x7f274c10d9d0 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f2751922700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c19d920 con 0x7f274c0731d0 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274c073af0 0x7f274c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274c073af0 0x7f274c199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54378/0 (socket says 172.21.15.12:54378) 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274a7fc700 1 -- 172.21.15.12:0/2954874404 learned_addr learned my addr 172.21.15.12:0/2954874404 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274affd700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c0731d0 0x7f274c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274b7fe700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f274c10d9d0 0x7f274c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274a7fc700 1 -- 172.21.15.12:0/2954874404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c0731d0 msgr2=0x7f274c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274a7fc700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c0731d0 0x7f274c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274a7fc700 1 -- 172.21.15.12:0/2954874404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f274c10d9d0 msgr2=0x7f274c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274a7fc700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f274c10d9d0 0x7f274c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274a7fc700 1 -- 172.21.15.12:0/2954874404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744009d00 con 0x7f274c073af0 2024-05-08T23:45:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.890+0000 7f274a7fc700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274c073af0 0x7f274c199cf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f274000d950 tx=0x7f274000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.891+0000 7f2750920700 1 -- 172.21.15.12:0/2954874404 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274000f890 con 0x7f274c073af0 2024-05-08T23:45:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.891+0000 7f2750920700 1 -- 172.21.15.12:0/2954874404 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2740010460 con 0x7f274c073af0 2024-05-08T23:45:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.891+0000 7f2751922700 1 -- 172.21.15.12:0/2954874404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c19dc00 con 0x7f274c073af0 2024-05-08T23:45:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.891+0000 7f2750920700 1 -- 172.21.15.12:0/2954874404 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274000e610 con 0x7f274c073af0 2024-05-08T23:45:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.891+0000 7f2751922700 1 -- 172.21.15.12:0/2954874404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c1a6550 con 0x7f274c073af0 2024-05-08T23:45:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.892+0000 7f2751922700 1 -- 172.21.15.12:0/2954874404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c108be0 con 0x7f274c073af0 2024-05-08T23:45:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.893+0000 7f2750920700 1 -- 172.21.15.12:0/2954874404 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f27400108d0 con 0x7f274c073af0 2024-05-08T23:45:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.893+0000 7f2750920700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f27340a11b0 0x7f27340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.894+0000 7f274affd700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f27340a11b0 0x7f27340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.894+0000 7f2750920700 1 -- 172.21.15.12:0/2954874404 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f27400cc650 con 0x7f274c073af0 2024-05-08T23:45:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.894+0000 7f274affd700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f27340a11b0 0x7f27340a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f273c005950 tx=0x7f273c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:18.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:18.898+0000 7f2750920700 1 -- 172.21.15.12:0/2954874404 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2740095e30 con 0x7f274c073af0 2024-05-08T23:45:19.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.114+0000 7f2751922700 1 -- 172.21.15.12:0/2954874404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f274c074860 con 0x7f274c073af0 2024-05-08T23:45:19.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.116+0000 7f2750920700 1 -- 172.21.15.12:0/2954874404 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22105 (secure 0 0 0) 0x7f2740016090 con 0x7f274c073af0 2024-05-08T23:45:19.112 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:19.112 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:05.029999+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:45:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.118+0000 7f2751922700 1 -- 172.21.15.12:0/2954874404 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f27340a11b0 msgr2=0x7f27340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.118+0000 7f2751922700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f27340a11b0 0x7f27340a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f273c005950 tx=0x7f273c00b410 comp rx=0 tx=0).stop 2024-05-08T23:45:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.118+0000 7f2751922700 1 -- 172.21.15.12:0/2954874404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274c073af0 msgr2=0x7f274c199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.118+0000 7f2751922700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274c073af0 0x7f274c199cf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f274000d950 tx=0x7f274000dc60 comp rx=0 tx=0).stop 2024-05-08T23:45:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.118+0000 7f2751922700 1 -- 172.21.15.12:0/2954874404 shutdown_connections 2024-05-08T23:45:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.118+0000 7f2751922700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f27340a11b0 0x7f27340a3670 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.118+0000 7f2751922700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f274c10d9d0 0x7f274c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.118+0000 7f2751922700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f274c0731d0 0x7f274c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.118+0000 7f2751922700 1 --2- 172.21.15.12:0/2954874404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274c073af0 0x7f274c199cf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.118+0000 7f2751922700 1 -- 172.21.15.12:0/2954874404 >> 172.21.15.12:0/2954874404 conn(0x7f274c078100 msgr2=0x7f274c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.119+0000 7f2751922700 1 -- 172.21.15.12:0/2954874404 shutdown_connections 2024-05-08T23:45:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:19.119+0000 7f2751922700 1 -- 172.21.15.12:0/2954874404 wait complete. 2024-05-08T23:45:19.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:19 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2954874404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:19.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:19 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2954874404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:19.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:19 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2954874404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:19.646 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T23:45:19.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T23:45:19.902 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:20.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:20 smithi012 ceph-mon[42503]: pgmap v233: 97 pgs: 97 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-05-08T23:45:20.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:20 smithi204 ceph-mon[47373]: pgmap v233: 97 pgs: 97 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-05-08T23:45:20.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:20 smithi160 ceph-mon[46152]: pgmap v233: 97 pgs: 97 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-05-08T23:45:20.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.974+0000 7f3f8afcf700 1 -- 172.21.15.12:0/1367092714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f84069210 msgr2=0x7f3f84069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:20.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.974+0000 7f3f8afcf700 1 --2- 172.21.15.12:0/1367092714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f84069210 0x7f3f84069690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0099c0 tx=0x7f3f6c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:45:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.974+0000 7f3f8afcf700 1 -- 172.21.15.12:0/1367092714 shutdown_connections 2024-05-08T23:45:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.974+0000 7f3f8afcf700 1 --2- 172.21.15.12:0/1367092714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f84069bd0 0x7f3f8410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.974+0000 7f3f8afcf700 1 --2- 172.21.15.12:0/1367092714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f84106fe0 0x7f3f841073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.974+0000 7f3f8afcf700 1 --2- 172.21.15.12:0/1367092714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f84069210 0x7f3f84069690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.974+0000 7f3f8afcf700 1 -- 172.21.15.12:0/1367092714 >> 172.21.15.12:0/1367092714 conn(0x7f3f84076e70 msgr2=0x7f3f84077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.974+0000 7f3f8afcf700 1 -- 172.21.15.12:0/1367092714 shutdown_connections 2024-05-08T23:45:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.974+0000 7f3f8afcf700 1 -- 172.21.15.12:0/1367092714 wait complete. 2024-05-08T23:45:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.975+0000 7f3f8afcf700 1 Processor -- start 2024-05-08T23:45:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.975+0000 7f3f8afcf700 1 -- start start 2024-05-08T23:45:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8afcf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f84069210 0x7f3f84197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8afcf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f84069bd0 0x7f3f841978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8afcf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f84106fe0 0x7f3f8419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8afcf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8419c290 con 0x7f3f84069210 2024-05-08T23:45:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8afcf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8419c400 con 0x7f3f84106fe0 2024-05-08T23:45:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8afcf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8419c5c0 con 0x7f3f84069bd0 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f83fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f84069bd0 0x7f3f841978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8956c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f84106fe0 0x7f3f8419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8956c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f84106fe0 0x7f3f8419bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47284/0 (socket says 172.21.15.12:47284) 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8956c700 1 -- 172.21.15.12:0/2359699019 learned_addr learned my addr 172.21.15.12:0/2359699019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f88d6b700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f84069210 0x7f3f84197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8956c700 1 -- 172.21.15.12:0/2359699019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f84069bd0 msgr2=0x7f3f841978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8956c700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f84069bd0 0x7f3f841978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8956c700 1 -- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f84069210 msgr2=0x7f3f84197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8956c700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f84069210 0x7f3f84197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f8956c700 1 -- 172.21.15.12:0/2359699019 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c0095f0 con 0x7f3f84106fe0 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f88d6b700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f84069210 0x7f3f84197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.976+0000 7f3f83fff700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f84069bd0 0x7f3f841978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:20.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.977+0000 7f3f8956c700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f84106fe0 0x7f3f8419bc80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c007ee0 tx=0x7f3f7c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.977+0000 7f3f81ffb700 1 -- 172.21.15.12:0/2359699019 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c017070 con 0x7f3f84106fe0 2024-05-08T23:45:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.977+0000 7f3f81ffb700 1 -- 172.21.15.12:0/2359699019 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7c00f040 con 0x7f3f84106fe0 2024-05-08T23:45:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.977+0000 7f3f8afcf700 1 -- 172.21.15.12:0/2359699019 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8419c8a0 con 0x7f3f84106fe0 2024-05-08T23:45:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.977+0000 7f3f81ffb700 1 -- 172.21.15.12:0/2359699019 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c005580 con 0x7f3f84106fe0 2024-05-08T23:45:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.977+0000 7f3f8afcf700 1 -- 172.21.15.12:0/2359699019 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f841a4100 con 0x7f3f84106fe0 2024-05-08T23:45:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.978+0000 7f3f8afcf700 1 -- 172.21.15.12:0/2359699019 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f841094e0 con 0x7f3f84106fe0 2024-05-08T23:45:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.979+0000 7f3f81ffb700 1 -- 172.21.15.12:0/2359699019 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3f7c00db20 con 0x7f3f84106fe0 2024-05-08T23:45:20.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.980+0000 7f3f81ffb700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f3f700a11b0 0x7f3f700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:20.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.980+0000 7f3f88d6b700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f3f700a11b0 0x7f3f700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:20.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.980+0000 7f3f81ffb700 1 -- 172.21.15.12:0/2359699019 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3f7c0cc110 con 0x7f3f84106fe0 2024-05-08T23:45:20.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.981+0000 7f3f88d6b700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f3f700a11b0 0x7f3f700a3670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f74005950 tx=0x7f3f7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:20.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:20.984+0000 7f3f81ffb700 1 -- 172.21.15.12:0/2359699019 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3f7c0957f0 con 0x7f3f84106fe0 2024-05-08T23:45:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.198+0000 7f3f8afcf700 1 -- 172.21.15.12:0/2359699019 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f8404ecc0 con 0x7f3f84106fe0 2024-05-08T23:45:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.199+0000 7f3f81ffb700 1 -- 172.21.15.12:0/2359699019 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22105 (secure 0 0 0) 0x7f3f7c098e10 con 0x7f3f84106fe0 2024-05-08T23:45:21.196 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:21.196 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:05.029999+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:45:21.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 -- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f3f700a11b0 msgr2=0x7f3f700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f3f700a11b0 0x7f3f700a3670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f74005950 tx=0x7f3f7400b410 comp rx=0 tx=0).stop 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 -- 172.21.15.12:0/2359699019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f84106fe0 msgr2=0x7f3f8419bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f84106fe0 0x7f3f8419bc80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c007ee0 tx=0x7f3f7c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 -- 172.21.15.12:0/2359699019 shutdown_connections 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f3f700a11b0 0x7f3f700a3670 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f84106fe0 0x7f3f8419bc80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f84069210 0x7f3f84197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 --2- 172.21.15.12:0/2359699019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f84069bd0 0x7f3f841978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 -- 172.21.15.12:0/2359699019 >> 172.21.15.12:0/2359699019 conn(0x7f3f84076e70 msgr2=0x7f3f840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 -- 172.21.15.12:0/2359699019 shutdown_connections 2024-05-08T23:45:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:21.202+0000 7f3f8afcf700 1 -- 172.21.15.12:0/2359699019 wait complete. 2024-05-08T23:45:21.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:21 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2359699019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:21.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:21 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2359699019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:21.561 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:21 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2359699019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:21.587 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T23:45:21.588 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T23:45:21.588 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T23:45:21.589 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T23:45:21.589 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T23:45:21.589 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T23:45:21.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T23:45:21.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T23:45:21.591 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T23:45:21.591 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T23:45:21.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T23:45:21.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T23:45:22.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:22 smithi204 ceph-mon[47373]: pgmap v234: 97 pgs: 97 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-05-08T23:45:22.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:22 smithi160 ceph-mon[46152]: pgmap v234: 97 pgs: 97 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-05-08T23:45:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:22 smithi012 ceph-mon[42503]: pgmap v234: 97 pgs: 97 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-05-08T23:45:22.830 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:23.538 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:24 smithi160 ceph-mon[46152]: pgmap v235: 97 pgs: 97 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-05-08T23:45:24.595 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:24 smithi012 ceph-mon[42503]: pgmap v235: 97 pgs: 97 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-05-08T23:45:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:24 smithi204 ceph-mon[47373]: pgmap v235: 97 pgs: 97 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-05-08T23:45:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.696+0000 7f58fe554700 1 -- 172.21.15.12:0/768599363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8107030 msgr2=0x7f58f8107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.696+0000 7f58fe554700 1 --2- 172.21.15.12:0/768599363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8107030 0x7f58f8107410 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009990 tx=0x7f58e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.696+0000 7f58fe554700 1 -- 172.21.15.12:0/768599363 shutdown_connections 2024-05-08T23:45:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.696+0000 7f58fe554700 1 --2- 172.21.15.12:0/768599363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8101990 0x7f58f810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.696+0000 7f58fe554700 1 --2- 172.21.15.12:0/768599363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8107030 0x7f58f8107410 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.696+0000 7f58fe554700 1 --2- 172.21.15.12:0/768599363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58f8100ff0 0x7f58f8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:25.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.696+0000 7f58fe554700 1 -- 172.21.15.12:0/768599363 >> 172.21.15.12:0/768599363 conn(0x7f58f8078190 msgr2=0x7f58f80785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:25.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.696+0000 7f58fe554700 1 -- 172.21.15.12:0/768599363 shutdown_connections 2024-05-08T23:45:25.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.696+0000 7f58fe554700 1 -- 172.21.15.12:0/768599363 wait complete. 2024-05-08T23:45:25.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.697+0000 7f58fe554700 1 Processor -- start 2024-05-08T23:45:25.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.697+0000 7f58fe554700 1 -- start start 2024-05-08T23:45:25.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fe554700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58f8100ff0 0x7f58f80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:25.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fe554700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8101990 0x7f58f80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:25.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fe554700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8107030 0x7f58f8073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fe554700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81a5840 con 0x7f58f8107030 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fe554700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81a59b0 con 0x7f58f8101990 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fe554700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81a5b70 con 0x7f58f8100ff0 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fcaf1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8107030 0x7f58f8073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fcaf1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8107030 0x7f58f8073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54420/0 (socket says 172.21.15.12:54420) 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fcaf1700 1 -- 172.21.15.12:0/3096292224 learned_addr learned my addr 172.21.15.12:0/3096292224 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58f7fff700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58f8100ff0 0x7f58f80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58f77fe700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8101990 0x7f58f80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fcaf1700 1 -- 172.21.15.12:0/3096292224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58f8100ff0 msgr2=0x7f58f80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fcaf1700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58f8100ff0 0x7f58f80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fcaf1700 1 -- 172.21.15.12:0/3096292224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8101990 msgr2=0x7f58f80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fcaf1700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8101990 0x7f58f80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fcaf1700 1 -- 172.21.15.12:0/3096292224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80095f0 con 0x7f58f8107030 2024-05-08T23:45:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.698+0000 7f58fcaf1700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8107030 0x7f58f8073d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b9e0 tx=0x7f58f000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:25.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.699+0000 7f58f57fa700 1 -- 172.21.15.12:0/3096292224 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000d820 con 0x7f58f8107030 2024-05-08T23:45:25.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.699+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f81a5e50 con 0x7f58f8107030 2024-05-08T23:45:25.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.699+0000 7f58f57fa700 1 -- 172.21.15.12:0/3096292224 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f000de40 con 0x7f58f8107030 2024-05-08T23:45:25.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.699+0000 7f58f57fa700 1 -- 172.21.15.12:0/3096292224 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f00143d0 con 0x7f58f8107030 2024-05-08T23:45:25.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.699+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81a6320 con 0x7f58f8107030 2024-05-08T23:45:25.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.700+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f58f81111b0 con 0x7f58f8107030 2024-05-08T23:45:25.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.701+0000 7f58f57fa700 1 -- 172.21.15.12:0/3096292224 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f58f0015620 con 0x7f58f8107030 2024-05-08T23:45:25.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.701+0000 7f58f57fa700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58e00a11b0 0x7f58e00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:25.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.702+0000 7f58f7fff700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58e00a11b0 0x7f58e00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:25.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.702+0000 7f58f57fa700 1 -- 172.21.15.12:0/3096292224 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f58f0011070 con 0x7f58f8107030 2024-05-08T23:45:25.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.702+0000 7f58f57fa700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] conn(0x7f58e00aa990 0x7f58e00acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:25.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.702+0000 7f58f57fa700 1 -- 172.21.15.12:0/3096292224 --> [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58e00ad460 con 0x7f58e00aa990 2024-05-08T23:45:25.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.702+0000 7f58f57fa700 1 -- 172.21.15.12:0/3096292224 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58f00cceb0 con 0x7f58f8107030 2024-05-08T23:45:25.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.702+0000 7f58f7fff700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58e00a11b0 0x7f58e00a3670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b5c0 tx=0x7f58e8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:25.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.702+0000 7f58f77fe700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] conn(0x7f58e00aa990 0x7f58e00acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:25.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.703+0000 7f58f77fe700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] conn(0x7f58e00aa990 0x7f58e00acdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:25.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.703+0000 7f58f57fa700 1 -- 172.21.15.12:0/3096292224 <== osd.4 v2:172.21.15.160:6808/2838790327 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f58e00ad460 con 0x7f58e00aa990 2024-05-08T23:45:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.730+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 --> [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f58f81026e0 con 0x7f58e00aa990 2024-05-08T23:45:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.731+0000 7f58f57fa700 1 -- 172.21.15.12:0/3096292224 <== osd.4 v2:172.21.15.160:6808/2838790327 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f58f81026e0 con 0x7f58e00aa990 2024-05-08T23:45:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.731+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 >> [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] conn(0x7f58e00aa990 msgr2=0x7f58e00acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.731+0000 7f58fe554700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] conn(0x7f58e00aa990 0x7f58e00acdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.732+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58e00a11b0 msgr2=0x7f58e00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.732+0000 7f58fe554700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58e00a11b0 0x7f58e00a3670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b5c0 tx=0x7f58e8005960 comp rx=0 tx=0).stop 2024-05-08T23:45:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.732+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8107030 msgr2=0x7f58f8073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.732+0000 7f58fe554700 1 --2- 172.21.15.12:0/3096292224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8107030 0x7f58f8073d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b9e0 tx=0x7f58f000bcf0 comp rx=0 tx=0).stop 2024-05-08T23:45:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.732+0000 7f58fcaf1700 1 -- 172.21.15.12:0/3096292224 reap_dead start 2024-05-08T23:45:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.732+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 shutdown_connections 2024-05-08T23:45:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.732+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 >> 172.21.15.12:0/3096292224 conn(0x7f58f8078190 msgr2=0x7f58f80ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.732+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 shutdown_connections 2024-05-08T23:45:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:25.732+0000 7f58fe554700 1 -- 172.21.15.12:0/3096292224 wait complete. 2024-05-08T23:45:26.342 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:26.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:26 smithi012 ceph-mon[42503]: pgmap v236: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-08T23:45:26.693 INFO:teuthology.orchestra.run.smithi012.stdout:133143986214 2024-05-08T23:45:26.693 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T23:45:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:26.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:26 smithi204 ceph-mon[47373]: pgmap v236: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-08T23:45:26.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:26 smithi160 ceph-mon[46152]: pgmap v236: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-08T23:45:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.244+0000 7fea53581700 1 -- 172.21.15.12:0/1932262984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea4c107020 msgr2=0x7fea4c107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.244+0000 7fea53581700 1 --2- 172.21.15.12:0/1932262984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea4c107020 0x7fea4c107400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea48009990 tx=0x7fea48009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.244+0000 7fea53581700 1 -- 172.21.15.12:0/1932262984 shutdown_connections 2024-05-08T23:45:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.244+0000 7fea53581700 1 --2- 172.21.15.12:0/1932262984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea4c0692f0 0x7fea4c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.244+0000 7fea53581700 1 --2- 172.21.15.12:0/1932262984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea4c107020 0x7fea4c107400 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.244+0000 7fea53581700 1 --2- 172.21.15.12:0/1932262984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4c068950 0x7fea4c068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.244+0000 7fea53581700 1 -- 172.21.15.12:0/1932262984 >> 172.21.15.12:0/1932262984 conn(0x7fea4c075f80 msgr2=0x7fea4c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.244+0000 7fea53581700 1 -- 172.21.15.12:0/1932262984 shutdown_connections 2024-05-08T23:45:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.245+0000 7fea53581700 1 -- 172.21.15.12:0/1932262984 wait complete. 2024-05-08T23:45:27.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.245+0000 7fea53581700 1 Processor -- start 2024-05-08T23:45:27.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea53581700 1 -- start start 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea53581700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea4c068950 0x7fea4c0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea53581700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4c0692f0 0x7fea4c1004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea53581700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea4c107020 0x7fea4c100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea53581700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c1a57a0 con 0x7fea4c0692f0 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea53581700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c1a5910 con 0x7fea4c068950 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea53581700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c1a5ad0 con 0x7fea4c107020 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea50b1c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4c0692f0 0x7fea4c1004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea50b1c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4c0692f0 0x7fea4c1004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54448/0 (socket says 172.21.15.12:54448) 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea50b1c700 1 -- 172.21.15.12:0/2698906998 learned_addr learned my addr 172.21.15.12:0/2698906998 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea5131d700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea4c068950 0x7fea4c0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.246+0000 7fea51b1e700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea4c107020 0x7fea4c100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea50b1c700 1 -- 172.21.15.12:0/2698906998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea4c107020 msgr2=0x7fea4c100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea50b1c700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea4c107020 0x7fea4c100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea50b1c700 1 -- 172.21.15.12:0/2698906998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea4c068950 msgr2=0x7fea4c0fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea50b1c700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea4c068950 0x7fea4c0fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea50b1c700 1 -- 172.21.15.12:0/2698906998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea480095f0 con 0x7fea4c0692f0 2024-05-08T23:45:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea50b1c700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4c0692f0 0x7fea4c1004e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e9a0 tx=0x7fea3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea427fc700 1 -- 172.21.15.12:0/2698906998 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c009790 con 0x7fea4c0692f0 2024-05-08T23:45:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c1a5db0 con 0x7fea4c0692f0 2024-05-08T23:45:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea427fc700 1 -- 172.21.15.12:0/2698906998 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea3c010430 con 0x7fea4c0692f0 2024-05-08T23:45:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea427fc700 1 -- 172.21.15.12:0/2698906998 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c018680 con 0x7fea4c0692f0 2024-05-08T23:45:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.247+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c1a6220 con 0x7fea4c0692f0 2024-05-08T23:45:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.248+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea4c1111b0 con 0x7fea4c0692f0 2024-05-08T23:45:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.250+0000 7fea427fc700 1 -- 172.21.15.12:0/2698906998 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fea3c0105a0 con 0x7fea4c0692f0 2024-05-08T23:45:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.251+0000 7fea427fc700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea380a11b0 0x7fea380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.251+0000 7fea427fc700 1 -- 172.21.15.12:0/2698906998 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fea3c014070 con 0x7fea4c0692f0 2024-05-08T23:45:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.251+0000 7fea5131d700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea380a11b0 0x7fea380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.251+0000 7fea427fc700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] conn(0x7fea380aa990 0x7fea380acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.251+0000 7fea427fc700 1 -- 172.21.15.12:0/2698906998 --> [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea380ad460 con 0x7fea380aa990 2024-05-08T23:45:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.251+0000 7fea427fc700 1 -- 172.21.15.12:0/2698906998 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea3c0cd220 con 0x7fea4c0692f0 2024-05-08T23:45:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.252+0000 7fea5131d700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea380a11b0 0x7fea380a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fea48005b00 tx=0x7fea48005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.252+0000 7fea51b1e700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] conn(0x7fea380aa990 0x7fea380acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.252+0000 7fea51b1e700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] conn(0x7fea380aa990 0x7fea380acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.253+0000 7fea427fc700 1 -- 172.21.15.12:0/2698906998 <== osd.2 v2:172.21.15.204:6800/474619962 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fea380ad460 con 0x7fea380aa990 2024-05-08T23:45:27.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.272+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 --> [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea4c0616a0 con 0x7fea380aa990 2024-05-08T23:45:27.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.273+0000 7fea427fc700 1 -- 172.21.15.12:0/2698906998 <== osd.2 v2:172.21.15.204:6800/474619962 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fea4c0616a0 con 0x7fea380aa990 2024-05-08T23:45:27.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 >> [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] conn(0x7fea380aa990 msgr2=0x7fea380acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea53581700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] conn(0x7fea380aa990 0x7fea380acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea380a11b0 msgr2=0x7fea380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea53581700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea380a11b0 0x7fea380a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fea48005b00 tx=0x7fea48005a90 comp rx=0 tx=0).stop 2024-05-08T23:45:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4c0692f0 msgr2=0x7fea4c1004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea53581700 1 --2- 172.21.15.12:0/2698906998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4c0692f0 0x7fea4c1004e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e9a0 tx=0x7fea3c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea51b1e700 1 -- 172.21.15.12:0/2698906998 reap_dead start 2024-05-08T23:45:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 shutdown_connections 2024-05-08T23:45:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 >> 172.21.15.12:0/2698906998 conn(0x7fea4c075f80 msgr2=0x7fea4c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 shutdown_connections 2024-05-08T23:45:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.274+0000 7fea53581700 1 -- 172.21.15.12:0/2698906998 wait complete. 2024-05-08T23:45:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.543+0000 7f2d12896700 1 -- 172.21.15.12:0/3077980246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d0c100850 msgr2=0x7f2d0c10ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.543+0000 7f2d12896700 1 --2- 172.21.15.12:0/3077980246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d0c100850 0x7f2d0c10ccf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b0a0 tx=0x7f2d0400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.544+0000 7f2d12896700 1 -- 172.21.15.12:0/3077980246 shutdown_connections 2024-05-08T23:45:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.544+0000 7f2d12896700 1 --2- 172.21.15.12:0/3077980246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d0c100850 0x7f2d0c10ccf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.544+0000 7f2d12896700 1 --2- 172.21.15.12:0/3077980246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0c105f40 0x7f2d0c106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.544+0000 7f2d12896700 1 --2- 172.21.15.12:0/3077980246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d0c0ffe90 0x7f2d0c100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.544+0000 7f2d12896700 1 -- 172.21.15.12:0/3077980246 >> 172.21.15.12:0/3077980246 conn(0x7f2d0c076e70 msgr2=0x7f2d0c077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.544+0000 7f2d12896700 1 -- 172.21.15.12:0/3077980246 shutdown_connections 2024-05-08T23:45:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.544+0000 7f2d12896700 1 -- 172.21.15.12:0/3077980246 wait complete. 2024-05-08T23:45:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.544+0000 7f2d12896700 1 Processor -- start 2024-05-08T23:45:27.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.544+0000 7f2d12896700 1 -- start start 2024-05-08T23:45:27.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d12896700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d0c0ffe90 0x7f2d0c196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d12896700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0c100850 0x7f2d0c196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d12896700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d0c105f40 0x7f2d0c196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d12896700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1a5870 con 0x7f2d0c105f40 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d12896700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1a59e0 con 0x7f2d0c0ffe90 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d12896700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1a5ba0 con 0x7f2d0c100850 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d10e33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d0c105f40 0x7f2d0c196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d10e33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d0c105f40 0x7f2d0c196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54476/0 (socket says 172.21.15.12:54476) 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d10e33700 1 -- 172.21.15.12:0/3148976279 learned_addr learned my addr 172.21.15.12:0/3148976279 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d0b7fe700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0c100850 0x7f2d0c196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d0bfff700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d0c0ffe90 0x7f2d0c196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d10e33700 1 -- 172.21.15.12:0/3148976279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0c100850 msgr2=0x7f2d0c196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d10e33700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0c100850 0x7f2d0c196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d10e33700 1 -- 172.21.15.12:0/3148976279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d0c0ffe90 msgr2=0x7f2d0c196120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d10e33700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d0c0ffe90 0x7f2d0c196120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.545+0000 7f2d10e33700 1 -- 172.21.15.12:0/3148976279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d04009d00 con 0x7f2d0c105f40 2024-05-08T23:45:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.546+0000 7f2d10e33700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d0c105f40 0x7f2d0c196ba0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b070 tx=0x7f2d04007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.546+0000 7f2d097fa700 1 -- 172.21.15.12:0/3148976279 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400e040 con 0x7f2d0c105f40 2024-05-08T23:45:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.546+0000 7f2d097fa700 1 -- 172.21.15.12:0/3148976279 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d04009440 con 0x7f2d0c105f40 2024-05-08T23:45:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.546+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c1a5e20 con 0x7f2d0c105f40 2024-05-08T23:45:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.546+0000 7f2d097fa700 1 -- 172.21.15.12:0/3148976279 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0401b450 con 0x7f2d0c105f40 2024-05-08T23:45:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.546+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c1a62f0 con 0x7f2d0c105f40 2024-05-08T23:45:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.547+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d0c110050 con 0x7f2d0c105f40 2024-05-08T23:45:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.548+0000 7f2d097fa700 1 -- 172.21.15.12:0/3148976279 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2d0401d480 con 0x7f2d0c105f40 2024-05-08T23:45:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.549+0000 7f2d097fa700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2cf40a11b0 0x7f2cf40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.549+0000 7f2d0bfff700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2cf40a11b0 0x7f2cf40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.549+0000 7f2d097fa700 1 -- 172.21.15.12:0/3148976279 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2d040ce7b0 con 0x7f2d0c105f40 2024-05-08T23:45:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.549+0000 7f2d097fa700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] conn(0x7f2cf40aa990 0x7f2cf40acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.549+0000 7f2d097fa700 1 -- 172.21.15.12:0/3148976279 --> [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2cf40ad460 con 0x7f2cf40aa990 2024-05-08T23:45:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.549+0000 7f2d097fa700 1 -- 172.21.15.12:0/3148976279 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d040cec50 con 0x7f2d0c105f40 2024-05-08T23:45:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.549+0000 7f2d0bfff700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2cf40a11b0 0x7f2cf40a3670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc005950 tx=0x7f2cfc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.549+0000 7f2d0b7fe700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] conn(0x7f2cf40aa990 0x7f2cf40acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.550+0000 7f2d0b7fe700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] conn(0x7f2cf40aa990 0x7f2cf40acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.551+0000 7f2d097fa700 1 -- 172.21.15.12:0/3148976279 <== osd.5 v2:172.21.15.204:6808/3032887482 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2cf40ad460 con 0x7f2cf40aa990 2024-05-08T23:45:27.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.569+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 --> [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2d0c061620 con 0x7f2cf40aa990 2024-05-08T23:45:27.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.569+0000 7f2d097fa700 1 -- 172.21.15.12:0/3148976279 <== osd.5 v2:172.21.15.204:6808/3032887482 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2d0c061620 con 0x7f2cf40aa990 2024-05-08T23:45:27.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 >> [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] conn(0x7f2cf40aa990 msgr2=0x7f2cf40acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d12896700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] conn(0x7f2cf40aa990 0x7f2cf40acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2cf40a11b0 msgr2=0x7f2cf40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d12896700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f2cf40a11b0 0x7f2cf40a3670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc005950 tx=0x7f2cfc00b410 comp rx=0 tx=0).stop 2024-05-08T23:45:27.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d0c105f40 msgr2=0x7f2d0c196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d12896700 1 --2- 172.21.15.12:0/3148976279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d0c105f40 0x7f2d0c196ba0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b070 tx=0x7f2d04007870 comp rx=0 tx=0).stop 2024-05-08T23:45:27.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d10e33700 1 -- 172.21.15.12:0/3148976279 reap_dead start 2024-05-08T23:45:27.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 shutdown_connections 2024-05-08T23:45:27.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 >> 172.21.15.12:0/3148976279 conn(0x7f2d0c076e70 msgr2=0x7f2d0c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:27.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 shutdown_connections 2024-05-08T23:45:27.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.570+0000 7f2d12896700 1 -- 172.21.15.12:0/3148976279 wait complete. 2024-05-08T23:45:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.792+0000 7fcb2194f700 1 -- 172.21.15.12:0/1505680372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb1c107020 msgr2=0x7fcb1c107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.792+0000 7fcb2194f700 1 --2- 172.21.15.12:0/1505680372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb1c107020 0x7fcb1c107400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009990 tx=0x7fcb0c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.793+0000 7fcb2194f700 1 -- 172.21.15.12:0/1505680372 shutdown_connections 2024-05-08T23:45:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.793+0000 7fcb2194f700 1 --2- 172.21.15.12:0/1505680372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb1c0692f0 0x7fcb1c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.793+0000 7fcb2194f700 1 --2- 172.21.15.12:0/1505680372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb1c107020 0x7fcb1c107400 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.793+0000 7fcb2194f700 1 --2- 172.21.15.12:0/1505680372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb1c068950 0x7fcb1c068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.793+0000 7fcb2194f700 1 -- 172.21.15.12:0/1505680372 >> 172.21.15.12:0/1505680372 conn(0x7fcb1c075f80 msgr2=0x7fcb1c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.793+0000 7fcb2194f700 1 -- 172.21.15.12:0/1505680372 shutdown_connections 2024-05-08T23:45:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.793+0000 7fcb2194f700 1 -- 172.21.15.12:0/1505680372 wait complete. 2024-05-08T23:45:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.793+0000 7fcb2194f700 1 Processor -- start 2024-05-08T23:45:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.793+0000 7fcb2194f700 1 -- start start 2024-05-08T23:45:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb2194f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb1c068950 0x7fcb1c197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb2194f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb1c0692f0 0x7fcb1c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb2194f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb1c107020 0x7fcb1c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb2194f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c19c300 con 0x7fcb1c0692f0 2024-05-08T23:45:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb2194f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c19c470 con 0x7fcb1c107020 2024-05-08T23:45:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb2194f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c19c630 con 0x7fcb1c068950 2024-05-08T23:45:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb127fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb1c0692f0 0x7fcb1c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb127fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb1c0692f0 0x7fcb1c197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54494/0 (socket says 172.21.15.12:54494) 2024-05-08T23:45:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb127fc700 1 -- 172.21.15.12:0/2770668772 learned_addr learned my addr 172.21.15.12:0/2770668772 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb1affd700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb1c068950 0x7fcb1c197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.794+0000 7fcb1b7fe700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb1c107020 0x7fcb1c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb127fc700 1 -- 172.21.15.12:0/2770668772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb1c068950 msgr2=0x7fcb1c197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb127fc700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb1c068950 0x7fcb1c197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb127fc700 1 -- 172.21.15.12:0/2770668772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb1c107020 msgr2=0x7fcb1c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb127fc700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb1c107020 0x7fcb1c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb127fc700 1 -- 172.21.15.12:0/2770668772 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c0095f0 con 0x7fcb1c0692f0 2024-05-08T23:45:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb1affd700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb1c068950 0x7fcb1c197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb127fc700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb1c0692f0 0x7fcb1c197940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400e9a0 tx=0x7fcb0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb2094d700 1 -- 172.21.15.12:0/2770668772 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04009790 con 0x7fcb1c0692f0 2024-05-08T23:45:27.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c19c910 con 0x7fcb1c0692f0 2024-05-08T23:45:27.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c1a4170 con 0x7fcb1c0692f0 2024-05-08T23:45:27.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb2094d700 1 -- 172.21.15.12:0/2770668772 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb04010430 con 0x7fcb1c0692f0 2024-05-08T23:45:27.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.795+0000 7fcb2094d700 1 -- 172.21.15.12:0/2770668772 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04018680 con 0x7fcb1c0692f0 2024-05-08T23:45:27.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.796+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcafc000ff0 con 0x7fcb1c0692f0 2024-05-08T23:45:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.798+0000 7fcb2094d700 1 -- 172.21.15.12:0/2770668772 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcb0400ca10 con 0x7fcb1c0692f0 2024-05-08T23:45:27.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.799+0000 7fcb2094d700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcb080a1230 0x7fcb080a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.799+0000 7fcb1affd700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcb080a1230 0x7fcb080a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.799+0000 7fcb2094d700 1 -- 172.21.15.12:0/2770668772 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcb04014070 con 0x7fcb1c0692f0 2024-05-08T23:45:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.799+0000 7fcb2094d700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] conn(0x7fcb080aaa10 0x7fcb080ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.799+0000 7fcb2094d700 1 -- 172.21.15.12:0/2770668772 --> [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb080ad4e0 con 0x7fcb080aaa10 2024-05-08T23:45:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.799+0000 7fcb2094d700 1 -- 172.21.15.12:0/2770668772 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb040cd6c0 con 0x7fcb1c0692f0 2024-05-08T23:45:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.800+0000 7fcb1affd700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcb080a1230 0x7fcb080a36f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00b2e0 tx=0x7fcb0c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.800+0000 7fcb1b7fe700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] conn(0x7fcb080aaa10 0x7fcb080ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.800+0000 7fcb1b7fe700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] conn(0x7fcb080aaa10 0x7fcb080ace30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.801+0000 7fcb2094d700 1 -- 172.21.15.12:0/2770668772 <== osd.8 v2:172.21.15.204:6816/4190277185 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fcb080ad4e0 con 0x7fcb080aaa10 2024-05-08T23:45:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.818+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 --> [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcafc002d70 con 0x7fcb080aaa10 2024-05-08T23:45:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.819+0000 7fcb2094d700 1 -- 172.21.15.12:0/2770668772 <== osd.8 v2:172.21.15.204:6816/4190277185 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcafc002d70 con 0x7fcb080aaa10 2024-05-08T23:45:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.819+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 >> [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] conn(0x7fcb080aaa10 msgr2=0x7fcb080ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.819+0000 7fcb2194f700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] conn(0x7fcb080aaa10 0x7fcb080ace30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.819+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcb080a1230 msgr2=0x7fcb080a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.819+0000 7fcb2194f700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcb080a1230 0x7fcb080a36f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00b2e0 tx=0x7fcb0c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.819+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb1c0692f0 msgr2=0x7fcb1c197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.819+0000 7fcb2194f700 1 --2- 172.21.15.12:0/2770668772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb1c0692f0 0x7fcb1c197940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400e9a0 tx=0x7fcb0400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.819+0000 7fcb1b7fe700 1 -- 172.21.15.12:0/2770668772 reap_dead start 2024-05-08T23:45:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.819+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 shutdown_connections 2024-05-08T23:45:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.819+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 >> 172.21.15.12:0/2770668772 conn(0x7fcb1c075f80 msgr2=0x7fcb1c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.820+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 shutdown_connections 2024-05-08T23:45:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.820+0000 7fcb2194f700 1 -- 172.21.15.12:0/2770668772 wait complete. 2024-05-08T23:45:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.854+0000 7fb696183700 1 -- 172.21.15.12:0/1024822896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6900ffe90 msgr2=0x7fb690100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.854+0000 7fb696183700 1 --2- 172.21.15.12:0/1024822896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6900ffe90 0x7fb690100310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb678009990 tx=0x7fb678009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.854+0000 7fb696183700 1 -- 172.21.15.12:0/1024822896 shutdown_connections 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.854+0000 7fb696183700 1 --2- 172.21.15.12:0/1024822896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb690100850 0x7fb69010ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.854+0000 7fb696183700 1 --2- 172.21.15.12:0/1024822896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb690105f40 0x7fb690106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.854+0000 7fb696183700 1 --2- 172.21.15.12:0/1024822896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6900ffe90 0x7fb690100310 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.854+0000 7fb696183700 1 -- 172.21.15.12:0/1024822896 >> 172.21.15.12:0/1024822896 conn(0x7fb690076e70 msgr2=0x7fb690077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.854+0000 7fb696183700 1 -- 172.21.15.12:0/1024822896 shutdown_connections 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.854+0000 7fb696183700 1 -- 172.21.15.12:0/1024822896 wait complete. 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb696183700 1 Processor -- start 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb696183700 1 -- start start 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb696183700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6900ffe90 0x7fb690195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb696183700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb690100850 0x7fb6901964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb696183700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb690105f40 0x7fb690196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb696183700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901a5690 con 0x7fb690105f40 2024-05-08T23:45:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb696183700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901a57d0 con 0x7fb690100850 2024-05-08T23:45:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb696183700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901a5990 con 0x7fb6900ffe90 2024-05-08T23:45:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb68f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6900ffe90 0x7fb690195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb690105f40 0x7fb690196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb68f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6900ffe90 0x7fb690195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53186/0 (socket says 172.21.15.12:53186) 2024-05-08T23:45:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb68f7fe700 1 -- 172.21.15.12:0/1652105797 learned_addr learned my addr 172.21.15.12:0/1652105797 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.855+0000 7fb687fff700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb690100850 0x7fb6901964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb68ffff700 1 -- 172.21.15.12:0/1652105797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6900ffe90 msgr2=0x7fb690195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb68ffff700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6900ffe90 0x7fb690195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb68ffff700 1 -- 172.21.15.12:0/1652105797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb690100850 msgr2=0x7fb6901964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb68ffff700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb690100850 0x7fb6901964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb68ffff700 1 -- 172.21.15.12:0/1652105797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780095f0 con 0x7fb690105f40 2024-05-08T23:45:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb68ffff700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb690105f40 0x7fb690196a00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb688007f10 tx=0x7fb68800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb68d7fa700 1 -- 172.21.15.12:0/1652105797 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688004db0 con 0x7fb690105f40 2024-05-08T23:45:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb68d7fa700 1 -- 172.21.15.12:0/1652105797 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb68800f040 con 0x7fb690105f40 2024-05-08T23:45:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6901a5c70 con 0x7fb690105f40 2024-05-08T23:45:27.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6901a6140 con 0x7fb690105f40 2024-05-08T23:45:27.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.856+0000 7fb68d7fa700 1 -- 172.21.15.12:0/1652105797 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688005990 con 0x7fb690105f40 2024-05-08T23:45:27.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.858+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb670000fc0 con 0x7fb690105f40 2024-05-08T23:45:27.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.860+0000 7fb68d7fa700 1 -- 172.21.15.12:0/1652105797 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb688005000 con 0x7fb690105f40 2024-05-08T23:45:27.856 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.861+0000 7fb68d7fa700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb67c0a1230 0x7fb67c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.861+0000 7fb68f7fe700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb67c0a1230 0x7fb67c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.861+0000 7fb68d7fa700 1 -- 172.21.15.12:0/1652105797 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb6880ccb70 con 0x7fb690105f40 2024-05-08T23:45:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.861+0000 7fb68d7fa700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] conn(0x7fb67c0aaa10 0x7fb67c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.861+0000 7fb68d7fa700 1 -- 172.21.15.12:0/1652105797 --> [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb67c0ad4e0 con 0x7fb67c0aaa10 2024-05-08T23:45:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.861+0000 7fb68d7fa700 1 -- 172.21.15.12:0/1652105797 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6880cd010 con 0x7fb690105f40 2024-05-08T23:45:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.862+0000 7fb687fff700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] conn(0x7fb67c0aaa10 0x7fb67c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.862+0000 7fb68f7fe700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb67c0a1230 0x7fb67c0a36f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb680009d20 tx=0x7fb680009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.862+0000 7fb687fff700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] conn(0x7fb67c0aaa10 0x7fb67c0ace30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.863+0000 7fb68d7fa700 1 -- 172.21.15.12:0/1652105797 <== osd.0 v2:172.21.15.12:6802/2326888222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb67c0ad4e0 con 0x7fb67c0aaa10 2024-05-08T23:45:27.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.881+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 --> [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb670002d40 con 0x7fb67c0aaa10 2024-05-08T23:45:27.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.882+0000 7fb68d7fa700 1 -- 172.21.15.12:0/1652105797 <== osd.0 v2:172.21.15.12:6802/2326888222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb670002d40 con 0x7fb67c0aaa10 2024-05-08T23:45:27.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] conn(0x7fb67c0aaa10 msgr2=0x7fb67c0ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb696183700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] conn(0x7fb67c0aaa10 0x7fb67c0ace30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb67c0a1230 msgr2=0x7fb67c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb696183700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb67c0a1230 0x7fb67c0a36f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb680009d20 tx=0x7fb680009260 comp rx=0 tx=0).stop 2024-05-08T23:45:27.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb690105f40 msgr2=0x7fb690196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:27.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb696183700 1 --2- 172.21.15.12:0/1652105797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb690105f40 0x7fb690196a00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb688007f10 tx=0x7fb68800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:27.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb68ffff700 1 -- 172.21.15.12:0/1652105797 reap_dead start 2024-05-08T23:45:27.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 shutdown_connections 2024-05-08T23:45:27.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 >> 172.21.15.12:0/1652105797 conn(0x7fb690076e70 msgr2=0x7fb6900feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:27.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 shutdown_connections 2024-05-08T23:45:27.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:27.884+0000 7fb696183700 1 -- 172.21.15.12:0/1652105797 wait complete. 2024-05-08T23:45:28.403 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:28 smithi012 ceph-mon[42503]: pgmap v237: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-05-08T23:45:28.427 INFO:teuthology.orchestra.run.smithi012.stdout:73014444080 2024-05-08T23:45:28.427 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T23:45:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.587+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/3817474860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c6410da80 msgr2=0x7f0c6410fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.587+0000 7f0c6c0a0700 1 --2- 172.21.15.12:0/3817474860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c6410da80 0x7f0c6410fe70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c00b0a0 tx=0x7f0c5c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.587+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/3817474860 shutdown_connections 2024-05-08T23:45:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.587+0000 7f0c6c0a0700 1 --2- 172.21.15.12:0/3817474860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c6410da80 0x7f0c6410fe70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.587+0000 7f0c6c0a0700 1 --2- 172.21.15.12:0/3817474860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c64068950 0x7f0c64068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.587+0000 7f0c6c0a0700 1 --2- 172.21.15.12:0/3817474860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c64069270 0x7f0c6410d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.587+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/3817474860 >> 172.21.15.12:0/3817474860 conn(0x7f0c64075f80 msgr2=0x7f0c64076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.587+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/3817474860 shutdown_connections 2024-05-08T23:45:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.587+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/3817474860 wait complete. 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6c0a0700 1 Processor -- start 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6c0a0700 1 -- start start 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6c0a0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c64068950 0x7f0c64199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6c0a0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c64069270 0x7f0c64199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6c0a0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6410da80 0x7f0c6419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6c0a0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6419e5d0 con 0x7f0c64069270 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6c0a0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6419e740 con 0x7f0c64068950 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6c0a0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6419e900 con 0x7f0c6410da80 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6963b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c64069270 0x7f0c64199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6963b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c64069270 0x7f0c64199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45072/0 (socket says 172.21.15.12:45072) 2024-05-08T23:45:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.588+0000 7f0c6963b700 1 -- 172.21.15.12:0/2780826817 learned_addr learned my addr 172.21.15.12:0/2780826817 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c69e3c700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c64068950 0x7f0c64199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c6a63d700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6410da80 0x7f0c6419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c6963b700 1 -- 172.21.15.12:0/2780826817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6410da80 msgr2=0x7f0c6419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c6963b700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6410da80 0x7f0c6419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c6963b700 1 -- 172.21.15.12:0/2780826817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c64068950 msgr2=0x7f0c64199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c6963b700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c64068950 0x7f0c64199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c6963b700 1 -- 172.21.15.12:0/2780826817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c009d00 con 0x7f0c64069270 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c6a63d700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6410da80 0x7f0c6419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c69e3c700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c64068950 0x7f0c64199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c6963b700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c64069270 0x7f0c64199c60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400e9a0 tx=0x7f0c5400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c5affd700 1 -- 172.21.15.12:0/2780826817 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54009790 con 0x7f0c64069270 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c5affd700 1 -- 172.21.15.12:0/2780826817 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c5400cbd0 con 0x7f0c64069270 2024-05-08T23:45:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6419ebe0 con 0x7f0c64069270 2024-05-08T23:45:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c5affd700 1 -- 172.21.15.12:0/2780826817 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54018680 con 0x7f0c64069270 2024-05-08T23:45:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.589+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c641a6440 con 0x7f0c64069270 2024-05-08T23:45:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.590+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c64113350 con 0x7f0c64069270 2024-05-08T23:45:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.592+0000 7f0c5affd700 1 -- 172.21.15.12:0/2780826817 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0c5400cd40 con 0x7f0c64069270 2024-05-08T23:45:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.592+0000 7f0c5affd700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0c500a1230 0x7f0c500a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.593+0000 7f0c69e3c700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0c500a1230 0x7f0c500a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.593+0000 7f0c5affd700 1 -- 172.21.15.12:0/2780826817 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0c54014070 con 0x7f0c64069270 2024-05-08T23:45:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.593+0000 7f0c5affd700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] conn(0x7f0c500aaa10 0x7f0c500ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.593+0000 7f0c5affd700 1 -- 172.21.15.12:0/2780826817 --> [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c500ad4e0 con 0x7f0c500aaa10 2024-05-08T23:45:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.593+0000 7f0c5affd700 1 -- 172.21.15.12:0/2780826817 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c540cd660 con 0x7f0c64069270 2024-05-08T23:45:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.593+0000 7f0c6a63d700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] conn(0x7f0c500aaa10 0x7f0c500ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.593+0000 7f0c69e3c700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0c500a1230 0x7f0c500a36f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009cb0 tx=0x7f0c60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:28.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.593+0000 7f0c6a63d700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] conn(0x7f0c500aaa10 0x7f0c500ace30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:28.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.594+0000 7f0c5affd700 1 -- 172.21.15.12:0/2780826817 <== osd.6 v2:172.21.15.12:6818/3301040337 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0c500ad4e0 con 0x7f0c500aaa10 2024-05-08T23:45:28.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.613+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 --> [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c64061620 con 0x7f0c500aaa10 2024-05-08T23:45:28.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.614+0000 7f0c5affd700 1 -- 172.21.15.12:0/2780826817 <== osd.6 v2:172.21.15.12:6818/3301040337 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0c64061620 con 0x7f0c500aaa10 2024-05-08T23:45:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.614+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] conn(0x7f0c500aaa10 msgr2=0x7f0c500ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.614+0000 7f0c6c0a0700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] conn(0x7f0c500aaa10 0x7f0c500ace30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.615+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0c500a1230 msgr2=0x7f0c500a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.615+0000 7f0c6c0a0700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0c500a1230 0x7f0c500a36f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009cb0 tx=0x7f0c60009260 comp rx=0 tx=0).stop 2024-05-08T23:45:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.615+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c64069270 msgr2=0x7f0c64199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.615+0000 7f0c6c0a0700 1 --2- 172.21.15.12:0/2780826817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c64069270 0x7f0c64199c60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400e9a0 tx=0x7f0c5400ed60 comp rx=0 tx=0).stop 2024-05-08T23:45:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.615+0000 7f0c6a63d700 1 -- 172.21.15.12:0/2780826817 reap_dead start 2024-05-08T23:45:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.615+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 shutdown_connections 2024-05-08T23:45:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.615+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 >> 172.21.15.12:0/2780826817 conn(0x7f0c64075f80 msgr2=0x7f0c6410ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.615+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 shutdown_connections 2024-05-08T23:45:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.615+0000 7f0c6c0a0700 1 -- 172.21.15.12:0/2780826817 wait complete. 2024-05-08T23:45:28.700 INFO:teuthology.orchestra.run.smithi012.stdout:154618822693 2024-05-08T23:45:28.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T23:45:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:28 smithi204 ceph-mon[47373]: pgmap v237: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-05-08T23:45:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:28 smithi160 ceph-mon[46152]: pgmap v237: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-05-08T23:45:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.999+0000 7f48671a7700 1 -- 172.21.15.12:0/2380490799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4860105f40 msgr2=0x7f4860106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.999+0000 7f48671a7700 1 --2- 172.21.15.12:0/2380490799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4860105f40 0x7f4860106320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4850009990 tx=0x7f4850009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.999+0000 7f48671a7700 1 -- 172.21.15.12:0/2380490799 shutdown_connections 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.999+0000 7f48671a7700 1 --2- 172.21.15.12:0/2380490799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4860100850 0x7f486010ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.999+0000 7f48671a7700 1 --2- 172.21.15.12:0/2380490799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4860105f40 0x7f4860106320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.999+0000 7f48671a7700 1 --2- 172.21.15.12:0/2380490799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48600ffe90 0x7f4860100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.999+0000 7f48671a7700 1 -- 172.21.15.12:0/2380490799 >> 172.21.15.12:0/2380490799 conn(0x7f4860076e70 msgr2=0x7f4860077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.999+0000 7f48671a7700 1 -- 172.21.15.12:0/2380490799 shutdown_connections 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.999+0000 7f48671a7700 1 -- 172.21.15.12:0/2380490799 wait complete. 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:28.999+0000 7f48671a7700 1 Processor -- start 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f48671a7700 1 -- start start 2024-05-08T23:45:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f48671a7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48600ffe90 0x7f4860101b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f48671a7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4860100850 0x7f4860102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f48671a7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4860105f40 0x7f48601025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f48671a7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601a5690 con 0x7f4860105f40 2024-05-08T23:45:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f48671a7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601a5800 con 0x7f4860100850 2024-05-08T23:45:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f48671a7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601a59c0 con 0x7f48600ffe90 2024-05-08T23:45:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f4865744700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4860105f40 0x7f48601025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f4865744700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4860105f40 0x7f48601025c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45096/0 (socket says 172.21.15.12:45096) 2024-05-08T23:45:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f4865744700 1 -- 172.21.15.12:0/3445967929 learned_addr learned my addr 172.21.15.12:0/3445967929 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f485ffff700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4860100850 0x7f4860102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f4864f43700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48600ffe90 0x7f4860101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f4865744700 1 -- 172.21.15.12:0/3445967929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48600ffe90 msgr2=0x7f4860101b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f4865744700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48600ffe90 0x7f4860101b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f4865744700 1 -- 172.21.15.12:0/3445967929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4860100850 msgr2=0x7f4860102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f4865744700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4860100850 0x7f4860102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f4865744700 1 -- 172.21.15.12:0/3445967929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48500095f0 con 0x7f4860105f40 2024-05-08T23:45:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.000+0000 7f4865744700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4860105f40 0x7f48601025c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f485800ef30 tx=0x7f485800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.001+0000 7f485dffb700 1 -- 172.21.15.12:0/3445967929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858018070 con 0x7f4860105f40 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.001+0000 7f485dffb700 1 -- 172.21.15.12:0/3445967929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f485800f040 con 0x7f4860105f40 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.001+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48601a5c40 con 0x7f4860105f40 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.001+0000 7f485dffb700 1 -- 172.21.15.12:0/3445967929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485800cd00 con 0x7f4860105f40 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.001+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48601a60f0 con 0x7f4860105f40 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.002+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4860101580 con 0x7f4860105f40 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.003+0000 7f485dffb700 1 -- 172.21.15.12:0/3445967929 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4858003a00 con 0x7f4860105f40 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.004+0000 7f485dffb700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f484c0a1230 0x7f484c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.004+0000 7f4864f43700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f484c0a1230 0x7f484c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.004+0000 7f485dffb700 1 -- 172.21.15.12:0/3445967929 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f48580cce30 con 0x7f4860105f40 2024-05-08T23:45:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.004+0000 7f485dffb700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] conn(0x7f484c0aaa10 0x7f484c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:29.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.004+0000 7f485dffb700 1 -- 172.21.15.12:0/3445967929 --> [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f484c0ad4e0 con 0x7f484c0aaa10 2024-05-08T23:45:29.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.004+0000 7f485dffb700 1 -- 172.21.15.12:0/3445967929 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48580cd1e0 con 0x7f4860105f40 2024-05-08T23:45:29.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.004+0000 7f485ffff700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] conn(0x7f484c0aaa10 0x7f484c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:29.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.004+0000 7f4864f43700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f484c0a1230 0x7f484c0a36f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f485000b2e0 tx=0x7f4850005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.004+0000 7f485ffff700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] conn(0x7f484c0aaa10 0x7f484c0ace30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.005+0000 7f485dffb700 1 -- 172.21.15.12:0/3445967929 <== osd.9 v2:172.21.15.12:6826/609239172 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f484c0ad4e0 con 0x7f484c0aaa10 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.022+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 --> [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4860110090 con 0x7f484c0aaa10 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.022+0000 7f485dffb700 1 -- 172.21.15.12:0/3445967929 <== osd.9 v2:172.21.15.12:6826/609239172 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4848007cf0 con 0x7f484c0aaa10 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] conn(0x7f484c0aaa10 msgr2=0x7f484c0ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f48671a7700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] conn(0x7f484c0aaa10 0x7f484c0ace30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f484c0a1230 msgr2=0x7f484c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f48671a7700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f484c0a1230 0x7f484c0a36f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f485000b2e0 tx=0x7f4850005960 comp rx=0 tx=0).stop 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4860105f40 msgr2=0x7f48601025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f48671a7700 1 --2- 172.21.15.12:0/3445967929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4860105f40 0x7f48601025c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f485800ef30 tx=0x7f485800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f4865744700 1 -- 172.21.15.12:0/3445967929 reap_dead start 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 shutdown_connections 2024-05-08T23:45:29.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 >> 172.21.15.12:0/3445967929 conn(0x7f4860076e70 msgr2=0x7f48600ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:29.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 shutdown_connections 2024-05-08T23:45:29.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:29.023+0000 7f48671a7700 1 -- 172.21.15.12:0/3445967929 wait complete. 2024-05-08T23:45:29.532 INFO:teuthology.orchestra.run.smithi012.stdout:240518168602 2024-05-08T23:45:29.533 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T23:45:29.656 INFO:teuthology.orchestra.run.smithi012.stdout:34359738423 2024-05-08T23:45:29.657 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T23:45:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:30.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:30 smithi012 ceph-mon[42503]: pgmap v238: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-08T23:45:30.510 INFO:teuthology.orchestra.run.smithi012.stdout:188978561056 2024-05-08T23:45:30.510 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T23:45:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:30 smithi204 ceph-mon[47373]: pgmap v238: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-08T23:45:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:30 smithi160 ceph-mon[46152]: pgmap v238: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-08T23:45:31.443 INFO:teuthology.orchestra.run.smithi012.stdout:274877906967 2024-05-08T23:45:31.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T23:45:31.814 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.839+0000 7f6283acc700 1 -- 172.21.15.12:0/2077905178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f627c0ffe90 msgr2=0x7f627c100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.839+0000 7f6283acc700 1 --2- 172.21.15.12:0/2077905178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f627c0ffe90 0x7f627c100310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6270009990 tx=0x7f6270009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.839+0000 7f6283acc700 1 -- 172.21.15.12:0/2077905178 shutdown_connections 2024-05-08T23:45:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.839+0000 7f6283acc700 1 --2- 172.21.15.12:0/2077905178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c100850 0x7f627c10ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.839+0000 7f6283acc700 1 --2- 172.21.15.12:0/2077905178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f627c105f40 0x7f627c106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.839+0000 7f6283acc700 1 --2- 172.21.15.12:0/2077905178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f627c0ffe90 0x7f627c100310 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.839+0000 7f6283acc700 1 -- 172.21.15.12:0/2077905178 >> 172.21.15.12:0/2077905178 conn(0x7f627c076e70 msgr2=0x7f627c077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.839+0000 7f6283acc700 1 -- 172.21.15.12:0/2077905178 shutdown_connections 2024-05-08T23:45:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.839+0000 7f6283acc700 1 -- 172.21.15.12:0/2077905178 wait complete. 2024-05-08T23:45:31.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6283acc700 1 Processor -- start 2024-05-08T23:45:31.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6283acc700 1 -- start start 2024-05-08T23:45:31.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6283acc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f627c0ffe90 0x7f627c101b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:31.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6283acc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f627c100850 0x7f627c102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:31.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6283acc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c105f40 0x7f627c1025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:31.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6283acc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1a5750 con 0x7f627c0ffe90 2024-05-08T23:45:31.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6283acc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1a58c0 con 0x7f627c105f40 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6283acc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1a5a80 con 0x7f627c100850 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6281868700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f627c0ffe90 0x7f627c101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6281868700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f627c0ffe90 0x7f627c101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45110/0 (socket says 172.21.15.12:45110) 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6281868700 1 -- 172.21.15.12:0/4251557925 learned_addr learned my addr 172.21.15.12:0/4251557925 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.840+0000 7f6282069700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c105f40 0x7f627c1025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f6281868700 1 -- 172.21.15.12:0/4251557925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f627c100850 msgr2=0x7f627c102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f6281868700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f627c100850 0x7f627c102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f6281868700 1 -- 172.21.15.12:0/4251557925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c105f40 msgr2=0x7f627c1025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f6281868700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f627c105f40 0x7f627c1025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f6281868700 1 -- 172.21.15.12:0/4251557925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62700095f0 con 0x7f627c0ffe90 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f6281868700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f627c0ffe90 0x7f627c101b40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f626c00e9a0 tx=0x7f626c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f627affd700 1 -- 172.21.15.12:0/4251557925 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c009790 con 0x7f627c0ffe90 2024-05-08T23:45:31.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c1a5d60 con 0x7f627c0ffe90 2024-05-08T23:45:31.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c1a6250 con 0x7f627c0ffe90 2024-05-08T23:45:31.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f627affd700 1 -- 172.21.15.12:0/4251557925 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626c00cbd0 con 0x7f627c0ffe90 2024-05-08T23:45:31.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.841+0000 7f627affd700 1 -- 172.21.15.12:0/4251557925 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c018680 con 0x7f627c0ffe90 2024-05-08T23:45:31.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.842+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f627c110010 con 0x7f627c0ffe90 2024-05-08T23:45:31.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.844+0000 7f627affd700 1 -- 172.21.15.12:0/4251557925 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f626c00cd40 con 0x7f627c0ffe90 2024-05-08T23:45:31.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.844+0000 7f627affd700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f62640a1230 0x7f62640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:31.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.844+0000 7f627affd700 1 -- 172.21.15.12:0/4251557925 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f626c014070 con 0x7f627c0ffe90 2024-05-08T23:45:31.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.844+0000 7f6281067700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f62640a1230 0x7f62640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:31.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.844+0000 7f627affd700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] conn(0x7f62640aaa10 0x7f62640ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:31.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.844+0000 7f627affd700 1 -- 172.21.15.12:0/4251557925 --> [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62640ad4e0 con 0x7f62640aaa10 2024-05-08T23:45:31.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.845+0000 7f627affd700 1 -- 172.21.15.12:0/4251557925 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f626c0cd880 con 0x7f627c0ffe90 2024-05-08T23:45:31.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.845+0000 7f6281067700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f62640a1230 0x7f62640a36f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f627000b2e0 tx=0x7f6270005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:31.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.845+0000 7f6282069700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] conn(0x7f62640aaa10 0x7f62640ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:31.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.845+0000 7f6282069700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] conn(0x7f62640aaa10 0x7f62640ace30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:31.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.846+0000 7f627affd700 1 -- 172.21.15.12:0/4251557925 <== osd.10 v2:172.21.15.160:6824/2916593770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f62640ad4e0 con 0x7f62640aaa10 2024-05-08T23:45:31.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.865+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 --> [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f627c103ec0 con 0x7f62640aaa10 2024-05-08T23:45:31.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.866+0000 7f627affd700 1 -- 172.21.15.12:0/4251557925 <== osd.10 v2:172.21.15.160:6824/2916593770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f627c103ec0 con 0x7f62640aaa10 2024-05-08T23:45:31.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 >> [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] conn(0x7f62640aaa10 msgr2=0x7f62640ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:31.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6283acc700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] conn(0x7f62640aaa10 0x7f62640ace30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:31.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f62640a1230 msgr2=0x7f62640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:31.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6283acc700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f62640a1230 0x7f62640a36f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f627000b2e0 tx=0x7f6270005fb0 comp rx=0 tx=0).stop 2024-05-08T23:45:31.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f627c0ffe90 msgr2=0x7f627c101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:31.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6283acc700 1 --2- 172.21.15.12:0/4251557925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f627c0ffe90 0x7f627c101b40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f626c00e9a0 tx=0x7f626c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:45:31.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6282069700 1 -- 172.21.15.12:0/4251557925 reap_dead start 2024-05-08T23:45:31.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 shutdown_connections 2024-05-08T23:45:31.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 >> 172.21.15.12:0/4251557925 conn(0x7f627c076e70 msgr2=0x7f627c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:31.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 shutdown_connections 2024-05-08T23:45:31.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:31.867+0000 7f6283acc700 1 -- 172.21.15.12:0/4251557925 wait complete. 2024-05-08T23:45:32.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.155+0000 7f8a36cca700 1 -- 172.21.15.12:0/1236189290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a300691e0 msgr2=0x7f8a300695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.155+0000 7f8a36cca700 1 --2- 172.21.15.12:0/1236189290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a300691e0 0x7f8a300695c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8a240099c0 tx=0x7f8a24009cd0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.155+0000 7f8a36cca700 1 -- 172.21.15.12:0/1236189290 shutdown_connections 2024-05-08T23:45:32.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.155+0000 7f8a36cca700 1 --2- 172.21.15.12:0/1236189290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a3010da40 0x7f8a3010fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.155+0000 7f8a36cca700 1 --2- 172.21.15.12:0/1236189290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a300691e0 0x7f8a300695c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.155+0000 7f8a36cca700 1 --2- 172.21.15.12:0/1236189290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a30069b00 0x7f8a3010d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.155+0000 7f8a36cca700 1 -- 172.21.15.12:0/1236189290 >> 172.21.15.12:0/1236189290 conn(0x7f8a30076e70 msgr2=0x7f8a30077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:32.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.155+0000 7f8a36cca700 1 -- 172.21.15.12:0/1236189290 shutdown_connections 2024-05-08T23:45:32.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.155+0000 7f8a36cca700 1 -- 172.21.15.12:0/1236189290 wait complete. 2024-05-08T23:45:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.156+0000 7f8a36cca700 1 Processor -- start 2024-05-08T23:45:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.156+0000 7f8a36cca700 1 -- start start 2024-05-08T23:45:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.156+0000 7f8a36cca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a300691e0 0x7f8a301995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.156+0000 7f8a36cca700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30069b00 0x7f8a30199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.156+0000 7f8a36cca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3010da40 0x7f8a3019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.156+0000 7f8a36cca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3019e4a0 con 0x7f8a3010da40 2024-05-08T23:45:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.156+0000 7f8a36cca700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3019e610 con 0x7f8a300691e0 2024-05-08T23:45:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.156+0000 7f8a36cca700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3019e7d0 con 0x7f8a30069b00 2024-05-08T23:45:32.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.156+0000 7f8a35267700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3010da40 0x7f8a3019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.156+0000 7f8a2ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30069b00 0x7f8a30199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a35267700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3010da40 0x7f8a3019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45126/0 (socket says 172.21.15.12:45126) 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a35267700 1 -- 172.21.15.12:0/2691417040 learned_addr learned my addr 172.21.15.12:0/2691417040 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a2ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30069b00 0x7f8a30199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53394/0 (socket says 172.21.15.12:53394) 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a35267700 1 -- 172.21.15.12:0/2691417040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30069b00 msgr2=0x7f8a30199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a35267700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30069b00 0x7f8a30199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a35267700 1 -- 172.21.15.12:0/2691417040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a300691e0 msgr2=0x7f8a301995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a35267700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a300691e0 0x7f8a301995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a35267700 1 -- 172.21.15.12:0/2691417040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a240095f0 con 0x7f8a3010da40 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a2ffff700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30069b00 0x7f8a30199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a35267700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3010da40 0x7f8a3019de90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800ef90 tx=0x7f8a2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a2dffb700 1 -- 172.21.15.12:0/2691417040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28018070 con 0x7f8a3010da40 2024-05-08T23:45:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a2dffb700 1 -- 172.21.15.12:0/2691417040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a2800f040 con 0x7f8a3010da40 2024-05-08T23:45:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.157+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3019ea50 con 0x7f8a3010da40 2024-05-08T23:45:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.158+0000 7f8a2dffb700 1 -- 172.21.15.12:0/2691417040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28003e00 con 0x7f8a3010da40 2024-05-08T23:45:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.158+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a301a6310 con 0x7f8a3010da40 2024-05-08T23:45:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.159+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a30113390 con 0x7f8a3010da40 2024-05-08T23:45:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.160+0000 7f8a2dffb700 1 -- 172.21.15.12:0/2691417040 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8a2800ca50 con 0x7f8a3010da40 2024-05-08T23:45:32.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.161+0000 7f8a2dffb700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a200a1230 0x7f8a200a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.161+0000 7f8a2dffb700 1 -- 172.21.15.12:0/2691417040 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8a280ccf90 con 0x7f8a3010da40 2024-05-08T23:45:32.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.161+0000 7f8a34a66700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a200a1230 0x7f8a200a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.161+0000 7f8a2dffb700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] conn(0x7f8a200aaa10 0x7f8a200ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.161+0000 7f8a2dffb700 1 -- 172.21.15.12:0/2691417040 --> [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a200ad4e0 con 0x7f8a200aaa10 2024-05-08T23:45:32.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.161+0000 7f8a2dffb700 1 -- 172.21.15.12:0/2691417040 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a280cd340 con 0x7f8a3010da40 2024-05-08T23:45:32.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.162+0000 7f8a2ffff700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] conn(0x7f8a200aaa10 0x7f8a200ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.162+0000 7f8a34a66700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a200a1230 0x7f8a200a36f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a240058e0 tx=0x7f8a24000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:32.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.162+0000 7f8a2ffff700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] conn(0x7f8a200aaa10 0x7f8a200ace30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:32.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.163+0000 7f8a2dffb700 1 -- 172.21.15.12:0/2691417040 <== osd.11 v2:172.21.15.204:6824/3148638065 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8a200ad4e0 con 0x7f8a200aaa10 2024-05-08T23:45:32.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.182+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 --> [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8a30100470 con 0x7f8a200aaa10 2024-05-08T23:45:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.182+0000 7f8a2dffb700 1 -- 172.21.15.12:0/2691417040 <== osd.11 v2:172.21.15.204:6824/3148638065 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8a30100470 con 0x7f8a200aaa10 2024-05-08T23:45:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 >> [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] conn(0x7f8a200aaa10 msgr2=0x7f8a200ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a36cca700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] conn(0x7f8a200aaa10 0x7f8a200ace30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a200a1230 msgr2=0x7f8a200a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a36cca700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a200a1230 0x7f8a200a36f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a240058e0 tx=0x7f8a24000bc0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3010da40 msgr2=0x7f8a3019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a36cca700 1 --2- 172.21.15.12:0/2691417040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3010da40 0x7f8a3019de90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800ef90 tx=0x7f8a2800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a35267700 1 -- 172.21.15.12:0/2691417040 reap_dead start 2024-05-08T23:45:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 shutdown_connections 2024-05-08T23:45:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 >> 172.21.15.12:0/2691417040 conn(0x7f8a30076e70 msgr2=0x7f8a3010f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 shutdown_connections 2024-05-08T23:45:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.183+0000 7f8a36cca700 1 -- 172.21.15.12:0/2691417040 wait complete. 2024-05-08T23:45:32.402 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:32 smithi012 ceph-mon[42503]: pgmap v239: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-08T23:45:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.485+0000 7fabe69d0700 1 -- 172.21.15.12:0/1865938616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe01019e0 msgr2=0x7fabe010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.485+0000 7fabe69d0700 1 --2- 172.21.15.12:0/1865938616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe01019e0 0x7fabe010ddf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fabd800b0a0 tx=0x7fabd800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.485+0000 7fabe69d0700 1 -- 172.21.15.12:0/1865938616 shutdown_connections 2024-05-08T23:45:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.485+0000 7fabe69d0700 1 --2- 172.21.15.12:0/1865938616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe01019e0 0x7fabe010ddf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.485+0000 7fabe69d0700 1 --2- 172.21.15.12:0/1865938616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe0107040 0x7fabe0107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.485+0000 7fabe69d0700 1 --2- 172.21.15.12:0/1865938616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabe0101020 0x7fabe01014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.485+0000 7fabe69d0700 1 -- 172.21.15.12:0/1865938616 >> 172.21.15.12:0/1865938616 conn(0x7fabe0078100 msgr2=0x7fabe0078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.485+0000 7fabe69d0700 1 -- 172.21.15.12:0/1865938616 shutdown_connections 2024-05-08T23:45:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.485+0000 7fabe69d0700 1 -- 172.21.15.12:0/1865938616 wait complete. 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.486+0000 7fabe69d0700 1 Processor -- start 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.486+0000 7fabe69d0700 1 -- start start 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.486+0000 7fabe69d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe0101020 0x7fabe00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.486+0000 7fabe69d0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabe01019e0 0x7fabe00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.486+0000 7fabe69d0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe0107040 0x7fabe0073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.486+0000 7fabe69d0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01a5780 con 0x7fabe0101020 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.486+0000 7fabe69d0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01a58f0 con 0x7fabe0107040 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.486+0000 7fabe69d0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01a5ab0 con 0x7fabe01019e0 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.486+0000 7fabdffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe0101020 0x7fabe00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe0101020 0x7fabe00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45180/0 (socket says 172.21.15.12:45180) 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdffff700 1 -- 172.21.15.12:0/3447146012 learned_addr learned my addr 172.21.15.12:0/3447146012 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:32.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdf7fe700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabe01019e0 0x7fabe00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabe4f6d700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe0107040 0x7fabe0073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdffff700 1 -- 172.21.15.12:0/3447146012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabe01019e0 msgr2=0x7fabe00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdffff700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabe01019e0 0x7fabe00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdffff700 1 -- 172.21.15.12:0/3447146012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe0107040 msgr2=0x7fabe0073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdffff700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe0107040 0x7fabe0073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdffff700 1 -- 172.21.15.12:0/3447146012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8009d00 con 0x7fabe0101020 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdffff700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe0101020 0x7fabe00732a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fabd000d950 tx=0x7fabd000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdd7fa700 1 -- 172.21.15.12:0/3447146012 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000f890 con 0x7fabe0101020 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe01a5d90 con 0x7fabe0101020 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdd7fa700 1 -- 172.21.15.12:0/3447146012 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabd0010460 con 0x7fabe0101020 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabdd7fa700 1 -- 172.21.15.12:0/3447146012 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000e610 con 0x7fabe0101020 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.487+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe01a6200 con 0x7fabe0101020 2024-05-08T23:45:32.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.488+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fabe0111110 con 0x7fabe0101020 2024-05-08T23:45:32.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.490+0000 7fabdd7fa700 1 -- 172.21.15.12:0/3447146012 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fabd00108d0 con 0x7fabe0101020 2024-05-08T23:45:32.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.490+0000 7fabdd7fa700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fabc80a11b0 0x7fabc80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.490+0000 7fabdf7fe700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fabc80a11b0 0x7fabc80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.491+0000 7fabdd7fa700 1 -- 172.21.15.12:0/3447146012 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fabd00ccb50 con 0x7fabe0101020 2024-05-08T23:45:32.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.491+0000 7fabdd7fa700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] conn(0x7fabc80aa990 0x7fabc80acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.491+0000 7fabdd7fa700 1 -- 172.21.15.12:0/3447146012 --> [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabc80ad460 con 0x7fabc80aa990 2024-05-08T23:45:32.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.491+0000 7fabdd7fa700 1 -- 172.21.15.12:0/3447146012 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabd00ccff0 con 0x7fabe0101020 2024-05-08T23:45:32.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.491+0000 7fabdf7fe700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fabc80a11b0 0x7fabc80a3670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fabe00749d0 tx=0x7fabd400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:32.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.491+0000 7fabe4f6d700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] conn(0x7fabc80aa990 0x7fabc80acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.491+0000 7fabe4f6d700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] conn(0x7fabc80aa990 0x7fabc80acdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:32.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.492+0000 7fabdd7fa700 1 -- 172.21.15.12:0/3447146012 <== osd.3 v2:172.21.15.12:6810/2160100336 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fabc80ad460 con 0x7fabc80aa990 2024-05-08T23:45:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.509+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 --> [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fabe0102680 con 0x7fabc80aa990 2024-05-08T23:45:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.509+0000 7fabdd7fa700 1 -- 172.21.15.12:0/3447146012 <== osd.3 v2:172.21.15.12:6810/2160100336 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fabe0102680 con 0x7fabc80aa990 2024-05-08T23:45:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.509+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] conn(0x7fabc80aa990 msgr2=0x7fabc80acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.509+0000 7fabe69d0700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] conn(0x7fabc80aa990 0x7fabc80acdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.509+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fabc80a11b0 msgr2=0x7fabc80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.509+0000 7fabe69d0700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fabc80a11b0 0x7fabc80a3670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fabe00749d0 tx=0x7fabd400b410 comp rx=0 tx=0).stop 2024-05-08T23:45:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.509+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe0101020 msgr2=0x7fabe00732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.509+0000 7fabe69d0700 1 --2- 172.21.15.12:0/3447146012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe0101020 0x7fabe00732a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fabd000d950 tx=0x7fabd000dc60 comp rx=0 tx=0).stop 2024-05-08T23:45:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.510+0000 7fabe4f6d700 1 -- 172.21.15.12:0/3447146012 reap_dead start 2024-05-08T23:45:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.510+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 shutdown_connections 2024-05-08T23:45:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.510+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 >> 172.21.15.12:0/3447146012 conn(0x7fabe0078100 msgr2=0x7fabe00ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.510+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 shutdown_connections 2024-05-08T23:45:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.510+0000 7fabe69d0700 1 -- 172.21.15.12:0/3447146012 wait complete. 2024-05-08T23:45:32.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:32 smithi204 ceph-mon[47373]: pgmap v239: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-08T23:45:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:32 smithi160 ceph-mon[46152]: pgmap v239: 97 pgs: 97 active+clean; 637 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-08T23:45:32.932 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:32.932 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.353+0000 7fc6446a1700 1 -- 172.21.15.12:0/1357742653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc63c068e90 msgr2=0x7fc63c069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.353+0000 7fc6446a1700 1 --2- 172.21.15.12:0/1357742653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc63c068e90 0x7fc63c069310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009990 tx=0x7fc62c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.353+0000 7fc6446a1700 1 -- 172.21.15.12:0/1357742653 shutdown_connections 2024-05-08T23:45:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.353+0000 7fc6446a1700 1 --2- 172.21.15.12:0/1357742653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc63c1039d0 0x7fc63c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.353+0000 7fc6446a1700 1 --2- 172.21.15.12:0/1357742653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc63c105ec0 0x7fc63c068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.353+0000 7fc6446a1700 1 --2- 172.21.15.12:0/1357742653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc63c068e90 0x7fc63c069310 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.353+0000 7fc6446a1700 1 -- 172.21.15.12:0/1357742653 >> 172.21.15.12:0/1357742653 conn(0x7fc63c075f80 msgr2=0x7fc63c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.354+0000 7fc6446a1700 1 -- 172.21.15.12:0/1357742653 shutdown_connections 2024-05-08T23:45:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.354+0000 7fc6446a1700 1 -- 172.21.15.12:0/1357742653 wait complete. 2024-05-08T23:45:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.354+0000 7fc6446a1700 1 Processor -- start 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.354+0000 7fc6446a1700 1 -- start start 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc6446a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc63c068e90 0x7fc63c1a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc6446a1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc63c1039d0 0x7fc63c1a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc6446a1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc63c105ec0 0x7fc63c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc6446a1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c19de80 con 0x7fc63c068e90 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc6446a1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c19dff0 con 0x7fc63c1039d0 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc6446a1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c19e1b0 con 0x7fc63c105ec0 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc641c3c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc63c1039d0 0x7fc63c1a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc641c3c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc63c1039d0 0x7fc63c1a3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:40352/0 (socket says 172.21.15.12:40352) 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc642c3e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc63c105ec0 0x7fc63c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc641c3c700 1 -- 172.21.15.12:0/3148953326 learned_addr learned my addr 172.21.15.12:0/3148953326 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc641c3c700 1 -- 172.21.15.12:0/3148953326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc63c105ec0 msgr2=0x7fc63c19d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc641c3c700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc63c105ec0 0x7fc63c19d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc641c3c700 1 -- 172.21.15.12:0/3148953326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc63c068e90 msgr2=0x7fc63c1a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc641c3c700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc63c068e90 0x7fc63c1a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.355+0000 7fc641c3c700 1 -- 172.21.15.12:0/3148953326 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c0095f0 con 0x7fc63c1039d0 2024-05-08T23:45:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.356+0000 7fc641c3c700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc63c1039d0 0x7fc63c1a3c10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00b5c0 tx=0x7fc62c00f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.356+0000 7fc6337fe700 1 -- 172.21.15.12:0/3148953326 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c01d070 con 0x7fc63c1039d0 2024-05-08T23:45:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.356+0000 7fc6337fe700 1 -- 172.21.15.12:0/3148953326 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc62c00fb60 con 0x7fc63c1039d0 2024-05-08T23:45:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.356+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c19e430 con 0x7fc63c1039d0 2024-05-08T23:45:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.356+0000 7fc6337fe700 1 -- 172.21.15.12:0/3148953326 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c017650 con 0x7fc63c1039d0 2024-05-08T23:45:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.356+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c1066c0 con 0x7fc63c1039d0 2024-05-08T23:45:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.357+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc63c1177d0 con 0x7fc63c1039d0 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.358+0000 7fc6337fe700 1 -- 172.21.15.12:0/3148953326 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc62c00fcd0 con 0x7fc63c1039d0 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.359+0000 7fc6337fe700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc6280a11b0 0x7fc6280a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.359+0000 7fc6337fe700 1 -- 172.21.15.12:0/3148953326 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc62c0ce600 con 0x7fc63c1039d0 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.359+0000 7fc64243d700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc6280a11b0 0x7fc6280a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.359+0000 7fc6337fe700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] conn(0x7fc6280aa990 0x7fc6280acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.359+0000 7fc6337fe700 1 -- 172.21.15.12:0/3148953326 --> [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6280ad460 con 0x7fc6280aa990 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.359+0000 7fc6337fe700 1 -- 172.21.15.12:0/3148953326 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc62c0ceaa0 con 0x7fc63c1039d0 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.359+0000 7fc64243d700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc6280a11b0 0x7fc6280a3670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc638005fd0 tx=0x7fc638005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.359+0000 7fc642c3e700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] conn(0x7fc6280aa990 0x7fc6280acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.360+0000 7fc642c3e700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] conn(0x7fc6280aa990 0x7fc6280acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:32.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.361+0000 7fc6337fe700 1 -- 172.21.15.12:0/3148953326 <== osd.1 v2:172.21.15.160:6800/72539183 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc6280ad460 con 0x7fc6280aa990 2024-05-08T23:45:32.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.380+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 --> [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc63c103380 con 0x7fc6280aa990 2024-05-08T23:45:32.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.380+0000 7fc6337fe700 1 -- 172.21.15.12:0/3148953326 <== osd.1 v2:172.21.15.160:6800/72539183 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc63c103380 con 0x7fc6280aa990 2024-05-08T23:45:32.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 >> [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] conn(0x7fc6280aa990 msgr2=0x7fc6280acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc6446a1700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] conn(0x7fc6280aa990 0x7fc6280acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc6280a11b0 msgr2=0x7fc6280a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc6446a1700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc6280a11b0 0x7fc6280a3670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc638005fd0 tx=0x7fc638005ee0 comp rx=0 tx=0).stop 2024-05-08T23:45:32.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc63c1039d0 msgr2=0x7fc63c1a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:32.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc6446a1700 1 --2- 172.21.15.12:0/3148953326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc63c1039d0 0x7fc63c1a3c10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00b5c0 tx=0x7fc62c00f670 comp rx=0 tx=0).stop 2024-05-08T23:45:32.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc642c3e700 1 -- 172.21.15.12:0/3148953326 reap_dead start 2024-05-08T23:45:32.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 shutdown_connections 2024-05-08T23:45:32.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 >> 172.21.15.12:0/3148953326 conn(0x7fc63c075f80 msgr2=0x7fc63c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:32.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 shutdown_connections 2024-05-08T23:45:32.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:32.381+0000 7fc6446a1700 1 -- 172.21.15.12:0/3148953326 wait complete. 2024-05-08T23:45:33.040 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:33.495 INFO:teuthology.orchestra.run.smithi012.stdout:296352743443 2024-05-08T23:45:33.495 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T23:45:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.731+0000 7f7dd93d7700 1 -- 172.21.15.12:0/3332174580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd4101020 msgr2=0x7f7dd41014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.731+0000 7f7dd93d7700 1 --2- 172.21.15.12:0/3332174580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd4101020 0x7f7dd41014a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009990 tx=0x7f7dbc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.732+0000 7f7dd93d7700 1 -- 172.21.15.12:0/3332174580 shutdown_connections 2024-05-08T23:45:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.732+0000 7f7dd93d7700 1 --2- 172.21.15.12:0/3332174580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd41019e0 0x7f7dd410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.732+0000 7f7dd93d7700 1 --2- 172.21.15.12:0/3332174580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd4107040 0x7f7dd4107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.732+0000 7f7dd93d7700 1 --2- 172.21.15.12:0/3332174580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd4101020 0x7f7dd41014a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.732+0000 7f7dd93d7700 1 -- 172.21.15.12:0/3332174580 >> 172.21.15.12:0/3332174580 conn(0x7f7dd4078100 msgr2=0x7f7dd4078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.732+0000 7f7dd93d7700 1 -- 172.21.15.12:0/3332174580 shutdown_connections 2024-05-08T23:45:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.732+0000 7f7dd93d7700 1 -- 172.21.15.12:0/3332174580 wait complete. 2024-05-08T23:45:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.732+0000 7f7dd93d7700 1 Processor -- start 2024-05-08T23:45:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.733+0000 7f7dd93d7700 1 -- start start 2024-05-08T23:45:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.733+0000 7f7dd93d7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd4101020 0x7f7dd40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.733+0000 7f7dd93d7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd41019e0 0x7f7dd40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.733+0000 7f7dd93d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd4107040 0x7f7dd4073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.733+0000 7f7dd93d7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd41a5720 con 0x7f7dd4107040 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.733+0000 7f7dd93d7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd41a5890 con 0x7f7dd4101020 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.733+0000 7f7dd93d7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd41a5a50 con 0x7f7dd41019e0 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.733+0000 7f7dd37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd4107040 0x7f7dd4073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.733+0000 7f7dd37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd4107040 0x7f7dd4073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45212/0 (socket says 172.21.15.12:45212) 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.733+0000 7f7dd37fe700 1 -- 172.21.15.12:0/390148963 learned_addr learned my addr 172.21.15.12:0/390148963 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd2ffd700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd4101020 0x7f7dd40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd27fc700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd41019e0 0x7f7dd40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd37fe700 1 -- 172.21.15.12:0/390148963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd41019e0 msgr2=0x7f7dd40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd37fe700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd41019e0 0x7f7dd40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd37fe700 1 -- 172.21.15.12:0/390148963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd4101020 msgr2=0x7f7dd40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd37fe700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd4101020 0x7f7dd40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd37fe700 1 -- 172.21.15.12:0/390148963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0095f0 con 0x7f7dd4107040 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd2ffd700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd4101020 0x7f7dd40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd37fe700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd4107040 0x7f7dd4073d20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400d980 tx=0x7f7dc400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dcbfff700 1 -- 172.21.15.12:0/390148963 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4009940 con 0x7f7dd4107040 2024-05-08T23:45:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dcbfff700 1 -- 172.21.15.12:0/390148963 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc4004830 con 0x7f7dd4107040 2024-05-08T23:45:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd41a5d30 con 0x7f7dd4107040 2024-05-08T23:45:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dcbfff700 1 -- 172.21.15.12:0/390148963 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40054e0 con 0x7f7dd4107040 2024-05-08T23:45:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.734+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd41a61e0 con 0x7f7dd4107040 2024-05-08T23:45:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.735+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7dd4102680 con 0x7f7dd4107040 2024-05-08T23:45:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.736+0000 7f7dcbfff700 1 -- 172.21.15.12:0/390148963 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7dc4004ca0 con 0x7f7dd4107040 2024-05-08T23:45:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.737+0000 7f7dcbfff700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7dc00a11b0 0x7f7dc00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.737+0000 7f7dd2ffd700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7dc00a11b0 0x7f7dc00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.737+0000 7f7dcbfff700 1 -- 172.21.15.12:0/390148963 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7dc40ccde0 con 0x7f7dd4107040 2024-05-08T23:45:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.737+0000 7f7dcbfff700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] conn(0x7f7dc00aa990 0x7f7dc00acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.737+0000 7f7dcbfff700 1 -- 172.21.15.12:0/390148963 --> [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7dc00ad460 con 0x7f7dc00aa990 2024-05-08T23:45:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.737+0000 7f7dcbfff700 1 -- 172.21.15.12:0/390148963 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc40cd280 con 0x7f7dd4107040 2024-05-08T23:45:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.738+0000 7f7dd2ffd700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7dc00a11b0 0x7f7dc00a3670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009d50 tx=0x7f7dcc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.738+0000 7f7dd27fc700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] conn(0x7f7dc00aa990 0x7f7dc00acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.738+0000 7f7dd27fc700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] conn(0x7f7dc00aa990 0x7f7dc00acdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:33.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.739+0000 7f7dcbfff700 1 -- 172.21.15.12:0/390148963 <== osd.7 v2:172.21.15.160:6816/1443429618 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f7dc00ad460 con 0x7f7dc00aa990 2024-05-08T23:45:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.757+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 --> [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7dd4111110 con 0x7f7dc00aa990 2024-05-08T23:45:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.758+0000 7f7dcbfff700 1 -- 172.21.15.12:0/390148963 <== osd.7 v2:172.21.15.160:6816/1443429618 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7dd4111110 con 0x7f7dc00aa990 2024-05-08T23:45:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.758+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 >> [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] conn(0x7f7dc00aa990 msgr2=0x7f7dc00acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.758+0000 7f7dd93d7700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] conn(0x7f7dc00aa990 0x7f7dc00acdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.759+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7dc00a11b0 msgr2=0x7f7dc00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.759+0000 7f7dd93d7700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7dc00a11b0 0x7f7dc00a3670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009d50 tx=0x7f7dcc009040 comp rx=0 tx=0).stop 2024-05-08T23:45:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.759+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd4107040 msgr2=0x7f7dd4073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.759+0000 7f7dd93d7700 1 --2- 172.21.15.12:0/390148963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd4107040 0x7f7dd4073d20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400d980 tx=0x7f7dc400dd40 comp rx=0 tx=0).stop 2024-05-08T23:45:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.759+0000 7f7dd37fe700 1 -- 172.21.15.12:0/390148963 reap_dead start 2024-05-08T23:45:33.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.759+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 shutdown_connections 2024-05-08T23:45:33.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.759+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 >> 172.21.15.12:0/390148963 conn(0x7f7dd4078100 msgr2=0x7f7dd40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:33.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.759+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 shutdown_connections 2024-05-08T23:45:33.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:33.759+0000 7f7dd93d7700 1 -- 172.21.15.12:0/390148963 wait complete. 2024-05-08T23:45:34.423 INFO:teuthology.orchestra.run.smithi012.stdout:326417514512 2024-05-08T23:45:34.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T23:45:34.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:34 smithi012 ceph-mon[42503]: pgmap v240: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-08T23:45:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:34 smithi204 ceph-mon[47373]: pgmap v240: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-08T23:45:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:34 smithi160 ceph-mon[46152]: pgmap v240: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-08T23:45:34.826 INFO:teuthology.orchestra.run.smithi012.stdout:210453397534 2024-05-08T23:45:34.826 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T23:45:35.159 INFO:teuthology.orchestra.run.smithi012.stdout:103079215149 2024-05-08T23:45:35.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T23:45:35.258 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:35.291 INFO:teuthology.orchestra.run.smithi012.stdout:55834574900 2024-05-08T23:45:35.291 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T23:45:35.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.668+0000 7f6a71b41700 1 -- 172.21.15.12:0/2091278122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0692f0 msgr2=0x7f6a6c10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:35.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.668+0000 7f6a71b41700 1 --2- 172.21.15.12:0/2091278122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0692f0 0x7f6a6c10de60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400b0a0 tx=0x7f6a6400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.668+0000 7f6a71b41700 1 -- 172.21.15.12:0/2091278122 shutdown_connections 2024-05-08T23:45:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.668+0000 7f6a71b41700 1 --2- 172.21.15.12:0/2091278122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0692f0 0x7f6a6c10de60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.668+0000 7f6a71b41700 1 --2- 172.21.15.12:0/2091278122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c107020 0x7f6a6c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.668+0000 7f6a71b41700 1 --2- 172.21.15.12:0/2091278122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c068950 0x7f6a6c068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.668+0000 7f6a71b41700 1 -- 172.21.15.12:0/2091278122 >> 172.21.15.12:0/2091278122 conn(0x7f6a6c075f80 msgr2=0x7f6a6c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.668+0000 7f6a71b41700 1 -- 172.21.15.12:0/2091278122 shutdown_connections 2024-05-08T23:45:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.668+0000 7f6a71b41700 1 -- 172.21.15.12:0/2091278122 wait complete. 2024-05-08T23:45:35.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.669+0000 7f6a71b41700 1 Processor -- start 2024-05-08T23:45:35.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.669+0000 7f6a71b41700 1 -- start start 2024-05-08T23:45:35.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.669+0000 7f6a71b41700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c068950 0x7f6a6c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:35.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.669+0000 7f6a71b41700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c0692f0 0x7f6a6c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.669+0000 7f6a71b41700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c107020 0x7f6a6c19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.669+0000 7f6a71b41700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c19c2e0 con 0x7f6a6c068950 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.669+0000 7f6a71b41700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c19c450 con 0x7f6a6c0692f0 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.669+0000 7f6a71b41700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c19c610 con 0x7f6a6c107020 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c068950 0x7f6a6c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c068950 0x7f6a6c197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45230/0 (socket says 172.21.15.12:45230) 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6b7fe700 1 -- 172.21.15.12:0/3472869292 learned_addr learned my addr 172.21.15.12:0/3472869292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6affd700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c0692f0 0x7f6a6c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6bfff700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c107020 0x7f6a6c19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6b7fe700 1 -- 172.21.15.12:0/3472869292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c107020 msgr2=0x7f6a6c19bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6b7fe700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c107020 0x7f6a6c19bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6b7fe700 1 -- 172.21.15.12:0/3472869292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c0692f0 msgr2=0x7f6a6c197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6b7fe700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c0692f0 0x7f6a6c197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6b7fe700 1 -- 172.21.15.12:0/3472869292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64009d00 con 0x7f6a6c068950 2024-05-08T23:45:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a6b7fe700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c068950 0x7f6a6c197500 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00d950 tx=0x7f6a5c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a68ff9700 1 -- 172.21.15.12:0/3472869292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00f890 con 0x7f6a6c068950 2024-05-08T23:45:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a68ff9700 1 -- 172.21.15.12:0/3472869292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a5c010460 con 0x7f6a6c068950 2024-05-08T23:45:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a71b41700 1 -- 172.21.15.12:0/3472869292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c19c810 con 0x7f6a6c068950 2024-05-08T23:45:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.670+0000 7f6a68ff9700 1 -- 172.21.15.12:0/3472869292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00e610 con 0x7f6a6c068950 2024-05-08T23:45:35.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.671+0000 7f6a71b41700 1 -- 172.21.15.12:0/3472869292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c1a2040 con 0x7f6a6c068950 2024-05-08T23:45:35.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.672+0000 7f6a71b41700 1 -- 172.21.15.12:0/3472869292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c109540 con 0x7f6a6c068950 2024-05-08T23:45:35.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.673+0000 7f6a68ff9700 1 -- 172.21.15.12:0/3472869292 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6a5c0108d0 con 0x7f6a6c068950 2024-05-08T23:45:35.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.673+0000 7f6a68ff9700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6a540a1300 0x7f6a540a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:35.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.674+0000 7f6a68ff9700 1 -- 172.21.15.12:0/3472869292 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6a5c0cc500 con 0x7f6a6c068950 2024-05-08T23:45:35.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.674+0000 7f6a6affd700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6a540a1300 0x7f6a540a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:35.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.674+0000 7f6a6affd700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6a540a1300 0x7f6a540a37c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c198a10 tx=0x7f6a600073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:35.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.679+0000 7f6a68ff9700 1 -- 172.21.15.12:0/3472869292 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6a5c095ce0 con 0x7f6a6c068950 2024-05-08T23:45:35.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.907+0000 7f6a71b41700 1 -- 172.21.15.12:0/3472869292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6a6c04ec60 con 0x7f6a6c068950 2024-05-08T23:45:35.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.907+0000 7f6a68ff9700 1 -- 172.21.15.12:0/3472869292 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a5c099300 con 0x7f6a6c068950 2024-05-08T23:45:35.903 INFO:teuthology.orchestra.run.smithi012.stdout:133143986216 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 -- 172.21.15.12:0/3472869292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6a540a1300 msgr2=0x7f6a540a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6a540a1300 0x7f6a540a37c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c198a10 tx=0x7f6a600073d0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 -- 172.21.15.12:0/3472869292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c068950 msgr2=0x7f6a6c197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c068950 0x7f6a6c197500 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00d950 tx=0x7f6a5c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 -- 172.21.15.12:0/3472869292 shutdown_connections 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c107020 0x7f6a6c19bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c068950 0x7f6a6c197500 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6a540a1300 0x7f6a540a37c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 --2- 172.21.15.12:0/3472869292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c0692f0 0x7f6a6c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 -- 172.21.15.12:0/3472869292 >> 172.21.15.12:0/3472869292 conn(0x7f6a6c075f80 msgr2=0x7f6a6c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 -- 172.21.15.12:0/3472869292 shutdown_connections 2024-05-08T23:45:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:35.910+0000 7f6a71b41700 1 -- 172.21.15.12:0/3472869292 wait complete. 2024-05-08T23:45:35.978 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:36.200 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:36.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:36 smithi012 ceph-mon[42503]: pgmap v241: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-08T23:45:36.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:36 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3472869292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T23:45:36.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:36 smithi204 ceph-mon[47373]: pgmap v241: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-08T23:45:36.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:36 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3472869292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T23:45:36.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:36 smithi160 ceph-mon[46152]: pgmap v241: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-08T23:45:36.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:36 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3472869292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T23:45:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.270+0000 7fbc04552700 1 -- 172.21.15.12:0/1538219593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbfc0692f0 msgr2=0x7fbbfc10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.270+0000 7fbc04552700 1 --2- 172.21.15.12:0/1538219593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbfc0692f0 0x7fbbfc10de60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009990 tx=0x7fbbf4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.270+0000 7fbc04552700 1 -- 172.21.15.12:0/1538219593 shutdown_connections 2024-05-08T23:45:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.270+0000 7fbc04552700 1 --2- 172.21.15.12:0/1538219593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbfc0692f0 0x7fbbfc10de60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.270+0000 7fbc04552700 1 --2- 172.21.15.12:0/1538219593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbfc107020 0x7fbbfc107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.270+0000 7fbc04552700 1 --2- 172.21.15.12:0/1538219593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbfc068950 0x7fbbfc068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.270+0000 7fbc04552700 1 -- 172.21.15.12:0/1538219593 >> 172.21.15.12:0/1538219593 conn(0x7fbbfc075f80 msgr2=0x7fbbfc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.270+0000 7fbc04552700 1 -- 172.21.15.12:0/1538219593 shutdown_connections 2024-05-08T23:45:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.270+0000 7fbc04552700 1 -- 172.21.15.12:0/1538219593 wait complete. 2024-05-08T23:45:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc04552700 1 Processor -- start 2024-05-08T23:45:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc04552700 1 -- start start 2024-05-08T23:45:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc04552700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbfc068950 0x7fbbfc1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc04552700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbfc0692f0 0x7fbbfc199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc04552700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbfc107020 0x7fbbfc19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc04552700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc19e4d0 con 0x7fbbfc068950 2024-05-08T23:45:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc04552700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc19e640 con 0x7fbbfc107020 2024-05-08T23:45:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc04552700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc19e800 con 0x7fbbfc0692f0 2024-05-08T23:45:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc022ee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbfc068950 0x7fbbfc1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc01aed700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbfc0692f0 0x7fbbfc199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc022ee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbfc068950 0x7fbbfc1995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45248/0 (socket says 172.21.15.12:45248) 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc01aed700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbfc0692f0 0x7fbbfc199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53456/0 (socket says 172.21.15.12:53456) 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.271+0000 7fbc022ee700 1 -- 172.21.15.12:0/1378353676 learned_addr learned my addr 172.21.15.12:0/1378353676 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbc02aef700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbfc107020 0x7fbbfc19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbc022ee700 1 -- 172.21.15.12:0/1378353676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbfc0692f0 msgr2=0x7fbbfc199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbc022ee700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbfc0692f0 0x7fbbfc199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbc022ee700 1 -- 172.21.15.12:0/1378353676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbfc107020 msgr2=0x7fbbfc19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbc022ee700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbfc107020 0x7fbbfc19dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbc022ee700 1 -- 172.21.15.12:0/1378353676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf40095f0 con 0x7fbbfc068950 2024-05-08T23:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbc022ee700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbfc068950 0x7fbbfc1995d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80060b0 tx=0x7fbbf800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbbf37fe700 1 -- 172.21.15.12:0/1378353676 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800dd20 con 0x7fbbfc068950 2024-05-08T23:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbbf37fe700 1 -- 172.21.15.12:0/1378353676 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf800f040 con 0x7fbbfc068950 2024-05-08T23:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbc04552700 1 -- 172.21.15.12:0/1378353676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc19eae0 con 0x7fbbfc068950 2024-05-08T23:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbbf37fe700 1 -- 172.21.15.12:0/1378353676 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8014600 con 0x7fbbfc068950 2024-05-08T23:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.272+0000 7fbc04552700 1 -- 172.21.15.12:0/1378353676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc1a6340 con 0x7fbbfc068950 2024-05-08T23:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.273+0000 7fbc04552700 1 -- 172.21.15.12:0/1378353676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc1026d0 con 0x7fbbfc068950 2024-05-08T23:45:37.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.275+0000 7fbbf37fe700 1 -- 172.21.15.12:0/1378353676 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbbf8004ad0 con 0x7fbbfc068950 2024-05-08T23:45:37.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.276+0000 7fbbf37fe700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbe80a1230 0x7fbbe80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:37.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.276+0000 7fbc01aed700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbe80a1230 0x7fbbe80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:37.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.276+0000 7fbbf37fe700 1 -- 172.21.15.12:0/1378353676 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbbf80cc380 con 0x7fbbfc068950 2024-05-08T23:45:37.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.276+0000 7fbc01aed700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbe80a1230 0x7fbbe80a36f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc19ab30 tx=0x7fbbec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.280+0000 7fbbf37fe700 1 -- 172.21.15.12:0/1378353676 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbbf8095b60 con 0x7fbbfc068950 2024-05-08T23:45:37.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.503+0000 7fbc04552700 1 -- 172.21.15.12:0/1378353676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbbfc19a980 con 0x7fbbfc068950 2024-05-08T23:45:37.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.504+0000 7fbbf37fe700 1 -- 172.21.15.12:0/1378353676 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbbf8024090 con 0x7fbbfc068950 2024-05-08T23:45:37.501 INFO:teuthology.orchestra.run.smithi012.stdout:34359738425 2024-05-08T23:45:37.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 -- 172.21.15.12:0/1378353676 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbe80a1230 msgr2=0x7fbbe80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:37.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbe80a1230 0x7fbbe80a36f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc19ab30 tx=0x7fbbec00b410 comp rx=0 tx=0).stop 2024-05-08T23:45:37.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 -- 172.21.15.12:0/1378353676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbfc068950 msgr2=0x7fbbfc1995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:37.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbfc068950 0x7fbbfc1995d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80060b0 tx=0x7fbbf800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 -- 172.21.15.12:0/1378353676 shutdown_connections 2024-05-08T23:45:37.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbbe80a1230 0x7fbbe80a36f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbfc107020 0x7fbbfc19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbfc068950 0x7fbbfc1995d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 --2- 172.21.15.12:0/1378353676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbfc0692f0 0x7fbbfc199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:37.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 -- 172.21.15.12:0/1378353676 >> 172.21.15.12:0/1378353676 conn(0x7fbbfc075f80 msgr2=0x7fbbfc10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:37.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.506+0000 7fbc04552700 1 -- 172.21.15.12:0/1378353676 shutdown_connections 2024-05-08T23:45:37.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.507+0000 7fbc04552700 1 -- 172.21.15.12:0/1378353676 wait complete. 2024-05-08T23:45:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.895+0000 7f8a1f69b700 1 -- 172.21.15.12:0/2821243974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a18105f00 msgr2=0x7f8a181062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.895+0000 7f8a1f69b700 1 --2- 172.21.15.12:0/2821243974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a18105f00 0x7f8a181062e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009990 tx=0x7f8a14009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.895+0000 7f8a1f69b700 1 -- 172.21.15.12:0/2821243974 shutdown_connections 2024-05-08T23:45:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.895+0000 7f8a1f69b700 1 --2- 172.21.15.12:0/2821243974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a18100890 0x7f8a1810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.895+0000 7f8a1f69b700 1 --2- 172.21.15.12:0/2821243974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a18105f00 0x7f8a181062e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.895+0000 7f8a1f69b700 1 --2- 172.21.15.12:0/2821243974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a180ffed0 0x7f8a18100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.895+0000 7f8a1f69b700 1 -- 172.21.15.12:0/2821243974 >> 172.21.15.12:0/2821243974 conn(0x7f8a18075f80 msgr2=0x7f8a18076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.895+0000 7f8a1f69b700 1 -- 172.21.15.12:0/2821243974 shutdown_connections 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.895+0000 7f8a1f69b700 1 -- 172.21.15.12:0/2821243974 wait complete. 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1f69b700 1 Processor -- start 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1f69b700 1 -- start start 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1f69b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a180ffed0 0x7f8a181982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1f69b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a18100890 0x7f8a18198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1f69b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a18105f00 0x7f8a181a3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1f69b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18199050 con 0x7f8a18105f00 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1f69b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181a3880 con 0x7f8a180ffed0 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1f69b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181a39c0 con 0x7f8a18100890 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1dc38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a18105f00 0x7f8a181a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1dc38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a18105f00 0x7f8a181a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45274/0 (socket says 172.21.15.12:45274) 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1dc38700 1 -- 172.21.15.12:0/1451331061 learned_addr learned my addr 172.21.15.12:0/1451331061 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:38.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.896+0000 7f8a1d437700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a180ffed0 0x7f8a181982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a1cc36700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a18100890 0x7f8a18198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a1dc38700 1 -- 172.21.15.12:0/1451331061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a18100890 msgr2=0x7f8a18198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a1dc38700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a18100890 0x7f8a18198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a1dc38700 1 -- 172.21.15.12:0/1451331061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a180ffed0 msgr2=0x7f8a181982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a1dc38700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a180ffed0 0x7f8a181982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a1dc38700 1 -- 172.21.15.12:0/1451331061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140095f0 con 0x7f8a18105f00 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a1dc38700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a18105f00 0x7f8a181a3340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a10007f10 tx=0x7f8a1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a0e7fc700 1 -- 172.21.15.12:0/1451331061 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000dd20 con 0x7f8a18105f00 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a0e7fc700 1 -- 172.21.15.12:0/1451331061 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1000f040 con 0x7f8a18105f00 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a0e7fc700 1 -- 172.21.15.12:0/1451331061 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100048a0 con 0x7f8a18105f00 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a1f69b700 1 -- 172.21.15.12:0/1451331061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a181a3c20 con 0x7f8a18105f00 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.897+0000 7f8a1f69b700 1 -- 172.21.15.12:0/1451331061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a181a40d0 con 0x7f8a18105f00 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.898+0000 7f8a1f69b700 1 -- 172.21.15.12:0/1451331061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a18108420 con 0x7f8a18105f00 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.900+0000 7f8a0e7fc700 1 -- 172.21.15.12:0/1451331061 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8a10003a00 con 0x7f8a18105f00 2024-05-08T23:45:38.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.901+0000 7f8a0e7fc700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a040a1300 0x7f8a040a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.901+0000 7f8a1d437700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a040a1300 0x7f8a040a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.901+0000 7f8a0e7fc700 1 -- 172.21.15.12:0/1451331061 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8a100cc7c0 con 0x7f8a18105f00 2024-05-08T23:45:38.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.901+0000 7f8a1d437700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a040a1300 0x7f8a040a37c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a140098e0 tx=0x7f8a14005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:38.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:37.905+0000 7f8a0e7fc700 1 -- 172.21.15.12:0/1451331061 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8a10096020 con 0x7f8a18105f00 2024-05-08T23:45:38.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.125+0000 7f8a1f69b700 1 -- 172.21.15.12:0/1451331061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f8a18068ef0 con 0x7f8a18105f00 2024-05-08T23:45:38.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.125+0000 7f8a0e7fc700 1 -- 172.21.15.12:0/1451331061 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a10023070 con 0x7f8a18105f00 2024-05-08T23:45:38.123 INFO:teuthology.orchestra.run.smithi012.stdout:240518168604 2024-05-08T23:45:38.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 -- 172.21.15.12:0/1451331061 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a040a1300 msgr2=0x7f8a040a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:38.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a040a1300 0x7f8a040a37c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a140098e0 tx=0x7f8a14005960 comp rx=0 tx=0).stop 2024-05-08T23:45:38.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 -- 172.21.15.12:0/1451331061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a18105f00 msgr2=0x7f8a181a3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:38.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a18105f00 0x7f8a181a3340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a10007f10 tx=0x7f8a1000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 -- 172.21.15.12:0/1451331061 shutdown_connections 2024-05-08T23:45:38.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a18105f00 0x7f8a181a3340 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f8a040a1300 0x7f8a040a37c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a180ffed0 0x7f8a181982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 --2- 172.21.15.12:0/1451331061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a18100890 0x7f8a18198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 -- 172.21.15.12:0/1451331061 >> 172.21.15.12:0/1451331061 conn(0x7f8a18075f80 msgr2=0x7f8a180feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 -- 172.21.15.12:0/1451331061 shutdown_connections 2024-05-08T23:45:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.128+0000 7f8a1f69b700 1 -- 172.21.15.12:0/1451331061 wait complete. 2024-05-08T23:45:38.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738423 got 34359738425 for osd.0 2024-05-08T23:45:38.394 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:38.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:38 smithi012 ceph-mon[42503]: pgmap v242: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:45:38.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:38 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1378353676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T23:45:38.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:38 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1451331061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T23:45:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:38 smithi204 ceph-mon[47373]: pgmap v242: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:45:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:38 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1378353676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T23:45:38.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:38 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1451331061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T23:45:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:38 smithi160 ceph-mon[46152]: pgmap v242: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:45:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:38 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1378353676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T23:45:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:38 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1451331061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T23:45:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.785+0000 7fe328f14700 1 -- 172.21.15.12:0/208690031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe324105f00 msgr2=0x7fe3241062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.785+0000 7fe328f14700 1 --2- 172.21.15.12:0/208690031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe324105f00 0x7fe3241062e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe314009990 tx=0x7fe314009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.785+0000 7fe328f14700 1 -- 172.21.15.12:0/208690031 shutdown_connections 2024-05-08T23:45:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.785+0000 7fe328f14700 1 --2- 172.21.15.12:0/208690031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe324100890 0x7fe32410cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.785+0000 7fe328f14700 1 --2- 172.21.15.12:0/208690031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe324105f00 0x7fe3241062e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.785+0000 7fe328f14700 1 --2- 172.21.15.12:0/208690031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240ffed0 0x7fe324100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.785+0000 7fe328f14700 1 -- 172.21.15.12:0/208690031 >> 172.21.15.12:0/208690031 conn(0x7fe324075f80 msgr2=0x7fe324076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.785+0000 7fe328f14700 1 -- 172.21.15.12:0/208690031 shutdown_connections 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.785+0000 7fe328f14700 1 -- 172.21.15.12:0/208690031 wait complete. 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.786+0000 7fe328f14700 1 Processor -- start 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.786+0000 7fe328f14700 1 -- start start 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe328f14700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240ffed0 0x7fe324199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe328f14700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe324100890 0x7fe324199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe328f14700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe324105f00 0x7fe32419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe328f14700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32419e530 con 0x7fe324105f00 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe328f14700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32419e6a0 con 0x7fe324100890 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe328f14700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32419e860 con 0x7fe3240ffed0 2024-05-08T23:45:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe322d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe324105f00 0x7fe32419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe322d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe324105f00 0x7fe32419df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47860/0 (socket says 172.21.15.12:47860) 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe322d9d700 1 -- 172.21.15.12:0/3342758557 learned_addr learned my addr 172.21.15.12:0/3342758557 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe32259c700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240ffed0 0x7fe324199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe321d9b700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe324100890 0x7fe324199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe322d9d700 1 -- 172.21.15.12:0/3342758557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240ffed0 msgr2=0x7fe324199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe322d9d700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240ffed0 0x7fe324199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe322d9d700 1 -- 172.21.15.12:0/3342758557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe324100890 msgr2=0x7fe324199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe322d9d700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe324100890 0x7fe324199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe322d9d700 1 -- 172.21.15.12:0/3342758557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3140095f0 con 0x7fe324105f00 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.787+0000 7fe322d9d700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe324105f00 0x7fe32419df20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe31c007ee0 tx=0x7fe31c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.788+0000 7fe3137fe700 1 -- 172.21.15.12:0/3342758557 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c017070 con 0x7fe324105f00 2024-05-08T23:45:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.788+0000 7fe328f14700 1 -- 172.21.15.12:0/3342758557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32419eb40 con 0x7fe324105f00 2024-05-08T23:45:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.788+0000 7fe328f14700 1 -- 172.21.15.12:0/3342758557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3241a63a0 con 0x7fe324105f00 2024-05-08T23:45:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.788+0000 7fe3137fe700 1 -- 172.21.15.12:0/3342758557 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe31c00f040 con 0x7fe324105f00 2024-05-08T23:45:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.788+0000 7fe3137fe700 1 -- 172.21.15.12:0/3342758557 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c005530 con 0x7fe324105f00 2024-05-08T23:45:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.789+0000 7fe328f14700 1 -- 172.21.15.12:0/3342758557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe304005320 con 0x7fe324105f00 2024-05-08T23:45:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.790+0000 7fe3137fe700 1 -- 172.21.15.12:0/3342758557 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe31c00db20 con 0x7fe324105f00 2024-05-08T23:45:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.791+0000 7fe3137fe700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe30c0a11b0 0x7fe30c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.791+0000 7fe32259c700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe30c0a11b0 0x7fe30c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.791+0000 7fe3137fe700 1 -- 172.21.15.12:0/3342758557 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe31c0cbe90 con 0x7fe324105f00 2024-05-08T23:45:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.791+0000 7fe32259c700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe30c0a11b0 0x7fe30c0a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe31400b5c0 tx=0x7fe314005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:38.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.799+0000 7fe3137fe700 1 -- 172.21.15.12:0/3342758557 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe31c095700 con 0x7fe324105f00 2024-05-08T23:45:38.797 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:38.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.816+0000 7f0f2387d700 1 -- 172.21.15.12:0/1110079708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f1c069250 msgr2=0x7f0f1c0696d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:38.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.816+0000 7f0f2387d700 1 --2- 172.21.15.12:0/1110079708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f1c069250 0x7f0f1c0696d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0099c0 tx=0x7f0f0c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.816+0000 7f0f2387d700 1 -- 172.21.15.12:0/1110079708 shutdown_connections 2024-05-08T23:45:38.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.816+0000 7f0f2387d700 1 --2- 172.21.15.12:0/1110079708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f1c069c10 0x7f0f1c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.816+0000 7f0f2387d700 1 --2- 172.21.15.12:0/1110079708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f1c106fe0 0x7f0f1c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.816+0000 7f0f2387d700 1 --2- 172.21.15.12:0/1110079708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f1c069250 0x7f0f1c0696d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.816+0000 7f0f2387d700 1 -- 172.21.15.12:0/1110079708 >> 172.21.15.12:0/1110079708 conn(0x7f0f1c076e70 msgr2=0x7f0f1c077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.816+0000 7f0f2387d700 1 -- 172.21.15.12:0/1110079708 shutdown_connections 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.816+0000 7f0f2387d700 1 -- 172.21.15.12:0/1110079708 wait complete. 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f2387d700 1 Processor -- start 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f2387d700 1 -- start start 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f2387d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f1c069250 0x7f0f1c197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f2387d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f1c069c10 0x7f0f1c1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f2387d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f1c106fe0 0x7f0f1c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f2387d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c19c290 con 0x7f0f1c106fe0 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f2387d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c19c400 con 0x7f0f1c069250 2024-05-08T23:45:38.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f2387d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c19c5c0 con 0x7f0f1c069c10 2024-05-08T23:45:38.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f21e1a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f1c106fe0 0x7f0f1c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f21e1a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f1c106fe0 0x7f0f1c19bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47866/0 (socket says 172.21.15.12:47866) 2024-05-08T23:45:38.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.817+0000 7f0f21e1a700 1 -- 172.21.15.12:0/2391974366 learned_addr learned my addr 172.21.15.12:0/2391974366 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:38.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.818+0000 7f0f20e18700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f1c069c10 0x7f0f1c1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.818+0000 7f0f21e1a700 1 -- 172.21.15.12:0/2391974366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f1c069c10 msgr2=0x7f0f1c1978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:38.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.818+0000 7f0f21e1a700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f1c069c10 0x7f0f1c1978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.818+0000 7f0f21e1a700 1 -- 172.21.15.12:0/2391974366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f1c069250 msgr2=0x7f0f1c197390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:45:38.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.818+0000 7f0f21e1a700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f1c069250 0x7f0f1c197390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:38.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.818+0000 7f0f21e1a700 1 -- 172.21.15.12:0/2391974366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0095f0 con 0x7f0f1c106fe0 2024-05-08T23:45:38.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.818+0000 7f0f21e1a700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f1c106fe0 0x7f0f1c19bc80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f14007ee0 tx=0x7f0f1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:38.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.818+0000 7f0f127fc700 1 -- 172.21.15.12:0/2391974366 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14004db0 con 0x7f0f1c106fe0 2024-05-08T23:45:38.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.818+0000 7f0f127fc700 1 -- 172.21.15.12:0/2391974366 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f14016070 con 0x7f0f1c106fe0 2024-05-08T23:45:38.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.818+0000 7f0f127fc700 1 -- 172.21.15.12:0/2391974366 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14012d10 con 0x7f0f1c106fe0 2024-05-08T23:45:38.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.819+0000 7f0f2387d700 1 -- 172.21.15.12:0/2391974366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1c19c8a0 con 0x7f0f1c106fe0 2024-05-08T23:45:38.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.819+0000 7f0f2387d700 1 -- 172.21.15.12:0/2391974366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1c100060 con 0x7f0f1c106fe0 2024-05-08T23:45:38.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.820+0000 7f0f2387d700 1 -- 172.21.15.12:0/2391974366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1c1094e0 con 0x7f0f1c106fe0 2024-05-08T23:45:38.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.821+0000 7f0f127fc700 1 -- 172.21.15.12:0/2391974366 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0f14018030 con 0x7f0f1c106fe0 2024-05-08T23:45:38.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.822+0000 7f0f127fc700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f080a1230 0x7f0f080a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:38.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.823+0000 7f0f21619700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f080a1230 0x7f0f080a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:38.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.823+0000 7f0f127fc700 1 -- 172.21.15.12:0/2391974366 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0f140cfff0 con 0x7f0f1c106fe0 2024-05-08T23:45:38.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.823+0000 7f0f21619700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f080a1230 0x7f0f080a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009a70 tx=0x7f0f18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:38.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:38.828+0000 7f0f127fc700 1 -- 172.21.15.12:0/2391974366 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0f14099840 con 0x7f0f1c106fe0 2024-05-08T23:45:38.943 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168602 got 240518168604 for osd.8 2024-05-08T23:45:38.944 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:39.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.043+0000 7fe328f14700 1 -- 172.21.15.12:0/3342758557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe304005190 con 0x7fe324105f00 2024-05-08T23:45:39.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.043+0000 7fe3137fe700 1 -- 172.21.15.12:0/3342758557 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe31c009390 con 0x7fe324105f00 2024-05-08T23:45:39.040 INFO:teuthology.orchestra.run.smithi012.stdout:73014444082 2024-05-08T23:45:39.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 -- 172.21.15.12:0/3342758557 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe30c0a11b0 msgr2=0x7fe30c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe30c0a11b0 0x7fe30c0a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe31400b5c0 tx=0x7fe314005fd0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 -- 172.21.15.12:0/3342758557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe324105f00 msgr2=0x7fe32419df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe324105f00 0x7fe32419df20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe31c007ee0 tx=0x7fe31c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 -- 172.21.15.12:0/3342758557 shutdown_connections 2024-05-08T23:45:39.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe324105f00 0x7fe32419df20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240ffed0 0x7fe324199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe30c0a11b0 0x7fe30c0a3670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 --2- 172.21.15.12:0/3342758557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe324100890 0x7fe324199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 -- 172.21.15.12:0/3342758557 >> 172.21.15.12:0/3342758557 conn(0x7fe324075f80 msgr2=0x7fe3240689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:39.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.046+0000 7fe328f14700 1 -- 172.21.15.12:0/3342758557 shutdown_connections 2024-05-08T23:45:39.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.047+0000 7fe328f14700 1 -- 172.21.15.12:0/3342758557 wait complete. 2024-05-08T23:45:39.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.060+0000 7f0f2387d700 1 -- 172.21.15.12:0/2391974366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0f1c102690 con 0x7f0f1c106fe0 2024-05-08T23:45:39.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.060+0000 7f0f127fc700 1 -- 172.21.15.12:0/2391974366 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f1409ce60 con 0x7f0f1c106fe0 2024-05-08T23:45:39.058 INFO:teuthology.orchestra.run.smithi012.stdout:154618822695 2024-05-08T23:45:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 -- 172.21.15.12:0/2391974366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f080a1230 msgr2=0x7f0f080a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f080a1230 0x7f0f080a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009a70 tx=0x7f0f18009260 comp rx=0 tx=0).stop 2024-05-08T23:45:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 -- 172.21.15.12:0/2391974366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f1c106fe0 msgr2=0x7f0f1c19bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f1c106fe0 0x7f0f1c19bc80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f14007ee0 tx=0x7f0f1400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 -- 172.21.15.12:0/2391974366 shutdown_connections 2024-05-08T23:45:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f1c106fe0 0x7f0f1c19bc80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f0f080a1230 0x7f0f080a36f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f1c069250 0x7f0f1c197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 --2- 172.21.15.12:0/2391974366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f1c069c10 0x7f0f1c1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 -- 172.21.15.12:0/2391974366 >> 172.21.15.12:0/2391974366 conn(0x7f0f1c076e70 msgr2=0x7f0f1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 -- 172.21.15.12:0/2391974366 shutdown_connections 2024-05-08T23:45:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.063+0000 7f0f2387d700 1 -- 172.21.15.12:0/2391974366 wait complete. 2024-05-08T23:45:39.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.250+0000 7f58a5203700 1 -- 172.21.15.12:0/698523621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58a0068950 msgr2=0x7f58a0068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.250+0000 7f58a5203700 1 --2- 172.21.15.12:0/698523621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58a0068950 0x7f58a0068db0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5894009990 tx=0x7f5894009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.250+0000 7f58a5203700 1 -- 172.21.15.12:0/698523621 shutdown_connections 2024-05-08T23:45:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.250+0000 7f58a5203700 1 --2- 172.21.15.12:0/698523621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00692f0 0x7f58a010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.250+0000 7f58a5203700 1 --2- 172.21.15.12:0/698523621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a0107020 0x7f58a0107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.250+0000 7f58a5203700 1 --2- 172.21.15.12:0/698523621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58a0068950 0x7f58a0068db0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.250+0000 7f58a5203700 1 -- 172.21.15.12:0/698523621 >> 172.21.15.12:0/698523621 conn(0x7f58a0075f80 msgr2=0x7f58a0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.250+0000 7f58a5203700 1 -- 172.21.15.12:0/698523621 shutdown_connections 2024-05-08T23:45:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.250+0000 7f58a5203700 1 -- 172.21.15.12:0/698523621 wait complete. 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.251+0000 7f58a5203700 1 Processor -- start 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.251+0000 7f58a5203700 1 -- start start 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f58a5203700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58a0068950 0x7f58a01973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f58a5203700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a00692f0 0x7f58a0197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f58a5203700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a0107020 0x7f58a019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f58a5203700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a019c230 con 0x7f58a0068950 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f58a5203700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a019c3a0 con 0x7f58a0107020 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f58a5203700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a019c560 con 0x7f58a00692f0 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f589ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58a0068950 0x7f58a01973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f589ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58a0068950 0x7f58a01973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47880/0 (socket says 172.21.15.12:47880) 2024-05-08T23:45:39.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f589ed9d700 1 -- 172.21.15.12:0/1635911470 learned_addr learned my addr 172.21.15.12:0/1635911470 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f589f59e700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a0107020 0x7f58a019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f589e59c700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a00692f0 0x7f58a0197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f589ed9d700 1 -- 172.21.15.12:0/1635911470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a00692f0 msgr2=0x7f58a0197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f589ed9d700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a00692f0 0x7f58a0197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f589ed9d700 1 -- 172.21.15.12:0/1635911470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a0107020 msgr2=0x7f58a019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f589ed9d700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a0107020 0x7f58a019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.252+0000 7f589ed9d700 1 -- 172.21.15.12:0/1635911470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58940095f0 con 0x7f58a0068950 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.253+0000 7f589ed9d700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58a0068950 0x7f58a01973f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f589000d950 tx=0x7f589000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.253+0000 7f5887fff700 1 -- 172.21.15.12:0/1635911470 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589000f890 con 0x7f58a0068950 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.253+0000 7f5887fff700 1 -- 172.21.15.12:0/1635911470 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5890010460 con 0x7f58a0068950 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.253+0000 7f58a5203700 1 -- 172.21.15.12:0/1635911470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a019c840 con 0x7f58a0068950 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.253+0000 7f5887fff700 1 -- 172.21.15.12:0/1635911470 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589000e610 con 0x7f58a0068950 2024-05-08T23:45:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.253+0000 7f58a5203700 1 -- 172.21.15.12:0/1635911470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a01a40a0 con 0x7f58a0068950 2024-05-08T23:45:39.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.254+0000 7f58a5203700 1 -- 172.21.15.12:0/1635911470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a0109540 con 0x7f58a0068950 2024-05-08T23:45:39.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.255+0000 7f5887fff700 1 -- 172.21.15.12:0/1635911470 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f58900108d0 con 0x7f58a0068950 2024-05-08T23:45:39.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.256+0000 7f5887fff700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58880a11b0 0x7f58880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:39.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.256+0000 7f589e59c700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58880a11b0 0x7f58880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:39.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.256+0000 7f5887fff700 1 -- 172.21.15.12:0/1635911470 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f58900cc5f0 con 0x7f58a0068950 2024-05-08T23:45:39.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.257+0000 7f589e59c700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58880a11b0 0x7f58880a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f589400b5c0 tx=0x7f5894005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:39.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.261+0000 7f5887fff700 1 -- 172.21.15.12:0/1635911470 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5890095dd0 con 0x7f58a0068950 2024-05-08T23:45:39.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.471+0000 7f4bd4553700 1 -- 172.21.15.12:0/1911451445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bcc10da80 msgr2=0x7f4bcc10fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.471+0000 7f4bd4553700 1 --2- 172.21.15.12:0/1911451445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bcc10da80 0x7f4bcc10fe70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400b0a0 tx=0x7f4bc400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.471+0000 7f4bd4553700 1 -- 172.21.15.12:0/1911451445 shutdown_connections 2024-05-08T23:45:39.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.471+0000 7f4bd4553700 1 --2- 172.21.15.12:0/1911451445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bcc10da80 0x7f4bcc10fe70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.471+0000 7f4bd4553700 1 --2- 172.21.15.12:0/1911451445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc068950 0x7f4bcc068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.471+0000 7f4bd4553700 1 --2- 172.21.15.12:0/1911451445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bcc069270 0x7f4bcc10d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.471+0000 7f4bd4553700 1 -- 172.21.15.12:0/1911451445 >> 172.21.15.12:0/1911451445 conn(0x7f4bcc075f80 msgr2=0x7f4bcc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:39.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.471+0000 7f4bd4553700 1 -- 172.21.15.12:0/1911451445 shutdown_connections 2024-05-08T23:45:39.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.471+0000 7f4bd4553700 1 -- 172.21.15.12:0/1911451445 wait complete. 2024-05-08T23:45:39.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd4553700 1 Processor -- start 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd4553700 1 -- start start 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd4553700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc068950 0x7f4bcc199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd4553700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bcc069270 0x7f4bcc199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd4553700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bcc10da80 0x7f4bcc19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd4553700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc19d5b0 con 0x7f4bcc10da80 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd4553700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc19d720 con 0x7f4bcc069270 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd4553700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc19d8e0 con 0x7f4bcc068950 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd2af0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bcc10da80 0x7f4bcc19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd2af0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bcc10da80 0x7f4bcc19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47900/0 (socket says 172.21.15.12:47900) 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd2af0700 1 -- 172.21.15.12:0/1342836566 learned_addr learned my addr 172.21.15.12:0/1342836566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd22ef700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc068950 0x7f4bcc199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd1aee700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bcc069270 0x7f4bcc199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd2af0700 1 -- 172.21.15.12:0/1342836566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc068950 msgr2=0x7f4bcc199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd2af0700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc068950 0x7f4bcc199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd2af0700 1 -- 172.21.15.12:0/1342836566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bcc069270 msgr2=0x7f4bcc199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.472+0000 7f4bd2af0700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bcc069270 0x7f4bcc199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.473+0000 7f4bd2af0700 1 -- 172.21.15.12:0/1342836566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc4009d00 con 0x7f4bcc10da80 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.473+0000 7f4bd1aee700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bcc069270 0x7f4bcc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.473+0000 7f4bd2af0700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bcc10da80 0x7f4bcc19cfb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006070 tx=0x7f4bc40077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.473+0000 7f4bcb7fe700 1 -- 172.21.15.12:0/1342836566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400e040 con 0x7f4bcc10da80 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.473+0000 7f4bd4553700 1 -- 172.21.15.12:0/1342836566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc19db60 con 0x7f4bcc10da80 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.473+0000 7f4bcb7fe700 1 -- 172.21.15.12:0/1342836566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bc4003c10 con 0x7f4bcc10da80 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.473+0000 7f4bcb7fe700 1 -- 172.21.15.12:0/1342836566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc401c660 con 0x7f4bcc10da80 2024-05-08T23:45:39.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.473+0000 7f4bd4553700 1 -- 172.21.15.12:0/1342836566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc1a6510 con 0x7f4bcc10da80 2024-05-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.474+0000 7f4bd4553700 1 -- 172.21.15.12:0/1342836566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc108ca0 con 0x7f4bcc10da80 2024-05-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.475+0000 7f4bcb7fe700 1 -- 172.21.15.12:0/1342836566 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4bc4005040 con 0x7f4bcc10da80 2024-05-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.476+0000 7f4bcb7fe700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4bb40a1230 0x7f4bb40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.476+0000 7f4bd22ef700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4bb40a1230 0x7f4bb40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.476+0000 7f4bcb7fe700 1 -- 172.21.15.12:0/1342836566 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4bc40cd7b0 con 0x7f4bcc10da80 2024-05-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.477+0000 7f4bd22ef700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4bb40a1230 0x7f4bb40a36f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc005950 tx=0x7f4bbc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.482+0000 7f4bcb7fe700 1 -- 172.21.15.12:0/1342836566 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4bc4096f20 con 0x7f4bcc10da80 2024-05-08T23:45:39.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.487+0000 7f58a5203700 1 -- 172.21.15.12:0/1635911470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f58a01026d0 con 0x7f58a0068950 2024-05-08T23:45:39.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.487+0000 7f5887fff700 1 -- 172.21.15.12:0/1635911470 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5890016090 con 0x7f58a0068950 2024-05-08T23:45:39.484 INFO:teuthology.orchestra.run.smithi012.stdout:188978561058 2024-05-08T23:45:39.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 -- 172.21.15.12:0/1635911470 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58880a11b0 msgr2=0x7f58880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58880a11b0 0x7f58880a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f589400b5c0 tx=0x7f5894005960 comp rx=0 tx=0).stop 2024-05-08T23:45:39.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 -- 172.21.15.12:0/1635911470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58a0068950 msgr2=0x7f58a01973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58a0068950 0x7f58a01973f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f589000d950 tx=0x7f589000dc60 comp rx=0 tx=0).stop 2024-05-08T23:45:39.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 -- 172.21.15.12:0/1635911470 shutdown_connections 2024-05-08T23:45:39.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f58880a11b0 0x7f58880a3670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a0107020 0x7f58a019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58a0068950 0x7f58a01973f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 --2- 172.21.15.12:0/1635911470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a00692f0 0x7f58a0197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 -- 172.21.15.12:0/1635911470 >> 172.21.15.12:0/1635911470 conn(0x7f58a0075f80 msgr2=0x7f58a00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:39.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 -- 172.21.15.12:0/1635911470 shutdown_connections 2024-05-08T23:45:39.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.490+0000 7f58a5203700 1 -- 172.21.15.12:0/1635911470 wait complete. 2024-05-08T23:45:39.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:39 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3342758557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T23:45:39.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:39 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2391974366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T23:45:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.704+0000 7f4bd4553700 1 -- 172.21.15.12:0/1342836566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f4bcc04ec60 con 0x7f4bcc10da80 2024-05-08T23:45:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.704+0000 7f4bcb7fe700 1 -- 172.21.15.12:0/1342836566 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bc409a540 con 0x7f4bcc10da80 2024-05-08T23:45:39.701 INFO:teuthology.orchestra.run.smithi012.stdout:274877906969 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 -- 172.21.15.12:0/1342836566 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4bb40a1230 msgr2=0x7f4bb40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4bb40a1230 0x7f4bb40a36f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc005950 tx=0x7f4bbc00b410 comp rx=0 tx=0).stop 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 -- 172.21.15.12:0/1342836566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bcc10da80 msgr2=0x7f4bcc19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bcc10da80 0x7f4bcc19cfb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006070 tx=0x7f4bc40077f0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 -- 172.21.15.12:0/1342836566 shutdown_connections 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bcc10da80 0x7f4bcc19cfb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bcc068950 0x7f4bcc199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f4bb40a1230 0x7f4bb40a36f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 --2- 172.21.15.12:0/1342836566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bcc069270 0x7f4bcc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 -- 172.21.15.12:0/1342836566 >> 172.21.15.12:0/1342836566 conn(0x7f4bcc075f80 msgr2=0x7f4bcc10eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 -- 172.21.15.12:0/1342836566 shutdown_connections 2024-05-08T23:45:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:39.707+0000 7f4bd4553700 1 -- 172.21.15.12:0/1342836566 wait complete. 2024-05-08T23:45:39.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:39 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3342758557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T23:45:39.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:39 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2391974366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T23:45:39.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:39 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3342758557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T23:45:39.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:39 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2391974366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T23:45:39.984 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444080 got 73014444082 for osd.2 2024-05-08T23:45:39.985 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:40.399 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:40 smithi012 ceph-mon[42503]: pgmap v243: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:45:40.399 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:40 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1635911470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T23:45:40.399 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:40 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1342836566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T23:45:40.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561056 got 188978561058 for osd.6 2024-05-08T23:45:40.424 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:40.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:40 smithi204 ceph-mon[47373]: pgmap v243: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:45:40.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:40 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1635911470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T23:45:40.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:40 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1342836566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T23:45:40.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:40 smithi160 ceph-mon[46152]: pgmap v243: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:45:40.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:40 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1635911470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T23:45:40.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:40 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1342836566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T23:45:40.859 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906967 got 274877906969 for osd.9 2024-05-08T23:45:40.859 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:40.874 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822693 got 154618822695 for osd.5 2024-05-08T23:45:40.874 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:40.948 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:40.968 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986214 got 133143986216 for osd.4 2024-05-08T23:45:40.968 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:41.098 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:42.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.246+0000 7fbe79526700 1 -- 172.21.15.12:0/1452804414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74073af0 msgr2=0x7fbe7410d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:42.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.246+0000 7fbe79526700 1 --2- 172.21.15.12:0/1452804414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74073af0 0x7fbe7410d460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009990 tx=0x7fbe68009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.246+0000 7fbe79526700 1 -- 172.21.15.12:0/1452804414 shutdown_connections 2024-05-08T23:45:42.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.246+0000 7fbe79526700 1 --2- 172.21.15.12:0/1452804414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7410d9a0 0x7fbe7410fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.246+0000 7fbe79526700 1 --2- 172.21.15.12:0/1452804414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe740731d0 0x7fbe740735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.246+0000 7fbe79526700 1 --2- 172.21.15.12:0/1452804414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74073af0 0x7fbe7410d460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.246+0000 7fbe79526700 1 -- 172.21.15.12:0/1452804414 >> 172.21.15.12:0/1452804414 conn(0x7fbe74078100 msgr2=0x7fbe74078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:42.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.246+0000 7fbe79526700 1 -- 172.21.15.12:0/1452804414 shutdown_connections 2024-05-08T23:45:42.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.246+0000 7fbe79526700 1 -- 172.21.15.12:0/1452804414 wait complete. 2024-05-08T23:45:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.247+0000 7fbe79526700 1 Processor -- start 2024-05-08T23:45:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.247+0000 7fbe79526700 1 -- start start 2024-05-08T23:45:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.247+0000 7fbe79526700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe740731d0 0x7fbe741995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.247+0000 7fbe79526700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74073af0 0x7fbe74199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.247+0000 7fbe79526700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe7410d9a0 0x7fbe7419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.247+0000 7fbe79526700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7419e4f0 con 0x7fbe74073af0 2024-05-08T23:45:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.247+0000 7fbe79526700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7419e660 con 0x7fbe7410d9a0 2024-05-08T23:45:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.247+0000 7fbe79526700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7419e820 con 0x7fbe740731d0 2024-05-08T23:45:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe727fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74073af0 0x7fbe74199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe727fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74073af0 0x7fbe74199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47922/0 (socket says 172.21.15.12:47922) 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe727fc700 1 -- 172.21.15.12:0/3899026715 learned_addr learned my addr 172.21.15.12:0/3899026715 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe72ffd700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe740731d0 0x7fbe741995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe737fe700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe7410d9a0 0x7fbe7419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe737fe700 1 -- 172.21.15.12:0/3899026715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe740731d0 msgr2=0x7fbe741995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe737fe700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe740731d0 0x7fbe741995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe737fe700 1 -- 172.21.15.12:0/3899026715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74073af0 msgr2=0x7fbe74199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe737fe700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74073af0 0x7fbe74199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe737fe700 1 -- 172.21.15.12:0/3899026715 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe680095f0 con 0x7fbe7410d9a0 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe727fc700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74073af0 0x7fbe74199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe737fe700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe7410d9a0 0x7fbe7419dee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c007ee0 tx=0x7fbe6c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe5bfff700 1 -- 172.21.15.12:0/3899026715 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c004db0 con 0x7fbe7410d9a0 2024-05-08T23:45:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe79526700 1 -- 172.21.15.12:0/3899026715 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7419eb00 con 0x7fbe7410d9a0 2024-05-08T23:45:42.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe5bfff700 1 -- 172.21.15.12:0/3899026715 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe6c00f040 con 0x7fbe7410d9a0 2024-05-08T23:45:42.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.248+0000 7fbe5bfff700 1 -- 172.21.15.12:0/3899026715 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c005900 con 0x7fbe7410d9a0 2024-05-08T23:45:42.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.249+0000 7fbe79526700 1 -- 172.21.15.12:0/3899026715 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe741a6360 con 0x7fbe7410d9a0 2024-05-08T23:45:42.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.249+0000 7fbe79526700 1 -- 172.21.15.12:0/3899026715 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe74108bb0 con 0x7fbe7410d9a0 2024-05-08T23:45:42.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.251+0000 7fbe5bfff700 1 -- 172.21.15.12:0/3899026715 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbe6c00db20 con 0x7fbe7410d9a0 2024-05-08T23:45:42.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.252+0000 7fbe5bfff700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbe5c0a11b0 0x7fbe5c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:42.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.252+0000 7fbe72ffd700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbe5c0a11b0 0x7fbe5c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:42.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.252+0000 7fbe5bfff700 1 -- 172.21.15.12:0/3899026715 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbe6c0cca60 con 0x7fbe7410d9a0 2024-05-08T23:45:42.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.252+0000 7fbe72ffd700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbe5c0a11b0 0x7fbe5c0a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009c60 tx=0x7fbe64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:42.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.255+0000 7fbe5bfff700 1 -- 172.21.15.12:0/3899026715 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbe6c096140 con 0x7fbe7410d9a0 2024-05-08T23:45:42.405 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:42.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.476+0000 7fbe79526700 1 -- 172.21.15.12:0/3899026715 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fbe74074890 con 0x7fbe7410d9a0 2024-05-08T23:45:42.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.477+0000 7fbe5bfff700 1 -- 172.21.15.12:0/3899026715 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fbe6c023020 con 0x7fbe7410d9a0 2024-05-08T23:45:42.474 INFO:teuthology.orchestra.run.smithi012.stdout:326417514514 2024-05-08T23:45:42.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.479+0000 7fbe79526700 1 -- 172.21.15.12:0/3899026715 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbe5c0a11b0 msgr2=0x7fbe5c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:42.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.479+0000 7fbe79526700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbe5c0a11b0 0x7fbe5c0a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009c60 tx=0x7fbe64009260 comp rx=0 tx=0).stop 2024-05-08T23:45:42.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.479+0000 7fbe79526700 1 -- 172.21.15.12:0/3899026715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe7410d9a0 msgr2=0x7fbe7419dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.479+0000 7fbe79526700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe7410d9a0 0x7fbe7419dee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c007ee0 tx=0x7fbe6c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.479+0000 7fbe79526700 1 -- 172.21.15.12:0/3899026715 shutdown_connections 2024-05-08T23:45:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.479+0000 7fbe79526700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbe5c0a11b0 0x7fbe5c0a3670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.479+0000 7fbe79526700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe7410d9a0 0x7fbe7419dee0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.479+0000 7fbe79526700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe740731d0 0x7fbe741995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.479+0000 7fbe79526700 1 --2- 172.21.15.12:0/3899026715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74073af0 0x7fbe74199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.479+0000 7fbe79526700 1 -- 172.21.15.12:0/3899026715 >> 172.21.15.12:0/3899026715 conn(0x7fbe74078100 msgr2=0x7fbe740fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.480+0000 7fbe79526700 1 -- 172.21.15.12:0/3899026715 shutdown_connections 2024-05-08T23:45:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.480+0000 7fbe79526700 1 -- 172.21.15.12:0/3899026715 wait complete. 2024-05-08T23:45:42.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:42 smithi012 ceph-mon[42503]: pgmap v244: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:45:42.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.742+0000 7f9f2e216700 1 -- 172.21.15.12:0/630874521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f28069c10 msgr2=0x7f9f2810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:42.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.742+0000 7f9f2e216700 1 --2- 172.21.15.12:0/630874521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f28069c10 0x7f9f2810de00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000b0d0 tx=0x7f9f2000b3e0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.742+0000 7f9f2e216700 1 -- 172.21.15.12:0/630874521 shutdown_connections 2024-05-08T23:45:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.742+0000 7f9f2e216700 1 --2- 172.21.15.12:0/630874521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f28069c10 0x7f9f2810de00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.742+0000 7f9f2e216700 1 --2- 172.21.15.12:0/630874521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f28106fe0 0x7f9f281073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.742+0000 7f9f2e216700 1 --2- 172.21.15.12:0/630874521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f28069250 0x7f9f280696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.742+0000 7f9f2e216700 1 -- 172.21.15.12:0/630874521 >> 172.21.15.12:0/630874521 conn(0x7f9f28076e70 msgr2=0x7f9f28077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.742+0000 7f9f2e216700 1 -- 172.21.15.12:0/630874521 shutdown_connections 2024-05-08T23:45:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.742+0000 7f9f2e216700 1 -- 172.21.15.12:0/630874521 wait complete. 2024-05-08T23:45:42.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f2e216700 1 Processor -- start 2024-05-08T23:45:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f2e216700 1 -- start start 2024-05-08T23:45:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f2e216700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f28069250 0x7f9f280fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f2e216700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f28069c10 0x7f9f281004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f2e216700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f28106fe0 0x7f9f281009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f2e216700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f281a5770 con 0x7f9f28069c10 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f2e216700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f281a58e0 con 0x7f9f28069250 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f2e216700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f281a5aa0 con 0x7f9f28106fe0 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f26ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f28069c10 0x7f9f281004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f26ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f28069c10 0x7f9f281004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47950/0 (socket says 172.21.15.12:47950) 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.743+0000 7f9f26ffd700 1 -- 172.21.15.12:0/2467005678 learned_addr learned my addr 172.21.15.12:0/2467005678 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f27fff700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f28106fe0 0x7f9f281009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f277fe700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f28069250 0x7f9f280fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f26ffd700 1 -- 172.21.15.12:0/2467005678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f28106fe0 msgr2=0x7f9f281009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f26ffd700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f28106fe0 0x7f9f281009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f26ffd700 1 -- 172.21.15.12:0/2467005678 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f28069250 msgr2=0x7f9f280fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f26ffd700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f28069250 0x7f9f280fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f26ffd700 1 -- 172.21.15.12:0/2467005678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f20009d00 con 0x7f9f28069c10 2024-05-08T23:45:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f277fe700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f28069250 0x7f9f280fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f26ffd700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f28069c10 0x7f9f281004a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00d950 tx=0x7f9f1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f24ff9700 1 -- 172.21.15.12:0/2467005678 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c009940 con 0x7f9f28069c10 2024-05-08T23:45:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f24ff9700 1 -- 172.21.15.12:0/2467005678 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f1c010460 con 0x7f9f28069c10 2024-05-08T23:45:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f2e216700 1 -- 172.21.15.12:0/2467005678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f281a5d80 con 0x7f9f28069c10 2024-05-08T23:45:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f24ff9700 1 -- 172.21.15.12:0/2467005678 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00f620 con 0x7f9f28069c10 2024-05-08T23:45:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.744+0000 7f9f2e216700 1 -- 172.21.15.12:0/2467005678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f281a6220 con 0x7f9f28069c10 2024-05-08T23:45:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.745+0000 7f9f2e216700 1 -- 172.21.15.12:0/2467005678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f281094e0 con 0x7f9f28069c10 2024-05-08T23:45:42.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.747+0000 7f9f24ff9700 1 -- 172.21.15.12:0/2467005678 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9f1c0108d0 con 0x7f9f28069c10 2024-05-08T23:45:42.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.748+0000 7f9f24ff9700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9f100a11b0 0x7f9f100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:42.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.748+0000 7f9f277fe700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9f100a11b0 0x7f9f100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:42.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.748+0000 7f9f24ff9700 1 -- 172.21.15.12:0/2467005678 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9f1c0cc810 con 0x7f9f28069c10 2024-05-08T23:45:42.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.748+0000 7f9f277fe700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9f100a11b0 0x7f9f100a3670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800bc10 tx=0x7f9f1800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:42.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.751+0000 7f9f24ff9700 1 -- 172.21.15.12:0/2467005678 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9f1c095ef0 con 0x7f9f28069c10 2024-05-08T23:45:42.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:42 smithi204 ceph-mon[47373]: pgmap v244: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:45:42.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:42 smithi160 ceph-mon[46152]: pgmap v244: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:45:42.847 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:42.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.988+0000 7f9f2e216700 1 -- 172.21.15.12:0/2467005678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f9f28101440 con 0x7f9f28069c10 2024-05-08T23:45:42.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.988+0000 7f9f24ff9700 1 -- 172.21.15.12:0/2467005678 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9f1c020020 con 0x7f9f28069c10 2024-05-08T23:45:42.985 INFO:teuthology.orchestra.run.smithi012.stdout:296352743445 2024-05-08T23:45:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.991+0000 7f9f2e216700 1 -- 172.21.15.12:0/2467005678 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9f100a11b0 msgr2=0x7f9f100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.991+0000 7f9f2e216700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9f100a11b0 0x7f9f100a3670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800bc10 tx=0x7f9f1800b5f0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.991+0000 7f9f2e216700 1 -- 172.21.15.12:0/2467005678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f28069c10 msgr2=0x7f9f281004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.991+0000 7f9f2e216700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f28069c10 0x7f9f281004a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00d950 tx=0x7f9f1c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:45:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.991+0000 7f9f2e216700 1 -- 172.21.15.12:0/2467005678 shutdown_connections 2024-05-08T23:45:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.991+0000 7f9f2e216700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f28106fe0 0x7f9f281009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.991+0000 7f9f2e216700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9f100a11b0 0x7f9f100a3670 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.991+0000 7f9f2e216700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f28069250 0x7f9f280fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.991+0000 7f9f2e216700 1 --2- 172.21.15.12:0/2467005678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f28069c10 0x7f9f281004a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.991+0000 7f9f2e216700 1 -- 172.21.15.12:0/2467005678 >> 172.21.15.12:0/2467005678 conn(0x7f9f28076e70 msgr2=0x7f9f280fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.992+0000 7f9f2e216700 1 -- 172.21.15.12:0/2467005678 shutdown_connections 2024-05-08T23:45:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:42.992+0000 7f9f2e216700 1 -- 172.21.15.12:0/2467005678 wait complete. 2024-05-08T23:45:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.112+0000 7fc35b780700 1 -- 172.21.15.12:0/2472314542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3541062f0 msgr2=0x7fc35410ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.112+0000 7fc35b780700 1 --2- 172.21.15.12:0/2472314542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3541062f0 0x7fc35410ded0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00b0a0 tx=0x7fc34c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.112+0000 7fc35b780700 1 -- 172.21.15.12:0/2472314542 shutdown_connections 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.112+0000 7fc35b780700 1 --2- 172.21.15.12:0/2472314542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3541062f0 0x7fc35410ded0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.112+0000 7fc35b780700 1 --2- 172.21.15.12:0/2472314542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3540698a0 0x7fc354069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.112+0000 7fc35b780700 1 --2- 172.21.15.12:0/2472314542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc35406a250 0x7fc354105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.112+0000 7fc35b780700 1 -- 172.21.15.12:0/2472314542 >> 172.21.15.12:0/2472314542 conn(0x7fc354076cd0 msgr2=0x7fc3540770e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.112+0000 7fc35b780700 1 -- 172.21.15.12:0/2472314542 shutdown_connections 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.112+0000 7fc35b780700 1 -- 172.21.15.12:0/2472314542 wait complete. 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc35b780700 1 Processor -- start 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc35b780700 1 -- start start 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc35b780700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3540698a0 0x7fc354197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc35b780700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc35406a250 0x7fc354197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc35b780700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3541062f0 0x7fc35419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc35b780700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35419c330 con 0x7fc35406a250 2024-05-08T23:45:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc35b780700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35419c4a0 con 0x7fc3540698a0 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc35b780700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35419c660 con 0x7fc3541062f0 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc358d1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc35406a250 0x7fc354197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc35951c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3540698a0 0x7fc354197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc358d1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc35406a250 0x7fc354197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47964/0 (socket says 172.21.15.12:47964) 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc358d1b700 1 -- 172.21.15.12:0/3262720844 learned_addr learned my addr 172.21.15.12:0/3262720844 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.113+0000 7fc359d1d700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3541062f0 0x7fc35419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc358d1b700 1 -- 172.21.15.12:0/3262720844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3541062f0 msgr2=0x7fc35419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc358d1b700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3541062f0 0x7fc35419bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc358d1b700 1 -- 172.21.15.12:0/3262720844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3540698a0 msgr2=0x7fc354197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc358d1b700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3540698a0 0x7fc354197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc358d1b700 1 -- 172.21.15.12:0/3262720844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c009d00 con 0x7fc35406a250 2024-05-08T23:45:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc358d1b700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc35406a250 0x7fc354197970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc34400d950 tx=0x7fc34400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:43.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc34a7fc700 1 -- 172.21.15.12:0/3262720844 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344004220 con 0x7fc35406a250 2024-05-08T23:45:43.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc35b780700 1 -- 172.21.15.12:0/3262720844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35419c940 con 0x7fc35406a250 2024-05-08T23:45:43.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc34a7fc700 1 -- 172.21.15.12:0/3262720844 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc344007620 con 0x7fc35406a250 2024-05-08T23:45:43.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc34a7fc700 1 -- 172.21.15.12:0/3262720844 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344005670 con 0x7fc35406a250 2024-05-08T23:45:43.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.114+0000 7fc35b780700 1 -- 172.21.15.12:0/3262720844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc354100390 con 0x7fc35406a250 2024-05-08T23:45:43.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.115+0000 7fc35b780700 1 -- 172.21.15.12:0/3262720844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3541095b0 con 0x7fc35406a250 2024-05-08T23:45:43.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.116+0000 7fc34a7fc700 1 -- 172.21.15.12:0/3262720844 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc344003680 con 0x7fc35406a250 2024-05-08T23:45:43.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.117+0000 7fc34a7fc700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc3400a11b0 0x7fc3400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:43.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.117+0000 7fc35951c700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc3400a11b0 0x7fc3400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:43.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.118+0000 7fc34a7fc700 1 -- 172.21.15.12:0/3262720844 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3440cca90 con 0x7fc35406a250 2024-05-08T23:45:43.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.118+0000 7fc35951c700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc3400a11b0 0x7fc3400a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc350005950 tx=0x7fc35000f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.122+0000 7fc34a7fc700 1 -- 172.21.15.12:0/3262720844 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc344096270 con 0x7fc35406a250 2024-05-08T23:45:43.304 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:43 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3899026715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T23:45:43.304 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:43 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2467005678' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T23:45:43.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.348+0000 7fc35b780700 1 -- 172.21.15.12:0/3262720844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc3541026f0 con 0x7fc35406a250 2024-05-08T23:45:43.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.348+0000 7fc34a7fc700 1 -- 172.21.15.12:0/3262720844 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc344099890 con 0x7fc35406a250 2024-05-08T23:45:43.344 INFO:teuthology.orchestra.run.smithi012.stdout:103079215151 2024-05-08T23:45:43.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 -- 172.21.15.12:0/3262720844 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc3400a11b0 msgr2=0x7fc3400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:43.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc3400a11b0 0x7fc3400a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc350005950 tx=0x7fc35000f820 comp rx=0 tx=0).stop 2024-05-08T23:45:43.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 -- 172.21.15.12:0/3262720844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc35406a250 msgr2=0x7fc354197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:43.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc35406a250 0x7fc354197970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc34400d950 tx=0x7fc34400dc60 comp rx=0 tx=0).stop 2024-05-08T23:45:43.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 -- 172.21.15.12:0/3262720844 shutdown_connections 2024-05-08T23:45:43.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3541062f0 0x7fc35419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:43.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc3400a11b0 0x7fc3400a3670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:43.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3540698a0 0x7fc354197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:43.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 --2- 172.21.15.12:0/3262720844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc35406a250 0x7fc354197970 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:43.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 -- 172.21.15.12:0/3262720844 >> 172.21.15.12:0/3262720844 conn(0x7fc354076cd0 msgr2=0x7fc3540fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:43.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 -- 172.21.15.12:0/3262720844 shutdown_connections 2024-05-08T23:45:43.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:43.351+0000 7fc35b780700 1 -- 172.21.15.12:0/3262720844 wait complete. 2024-05-08T23:45:43.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:43 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3899026715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T23:45:43.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:43 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2467005678' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T23:45:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:43 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3899026715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T23:45:43.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:43 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2467005678' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T23:45:44.141 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514512 got 326417514514 for osd.11 2024-05-08T23:45:44.141 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:44.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:44 smithi012 ceph-mon[42503]: pgmap v245: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:45:44.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:44 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3262720844' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T23:45:44.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.672+0000 7f333f26a700 1 -- 172.21.15.12:0/3376621241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3338100890 msgr2=0x7f333810cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:44.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.672+0000 7f333f26a700 1 --2- 172.21.15.12:0/3376621241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3338100890 0x7f333810cd40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f333000b0a0 tx=0x7f333000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.672+0000 7f333f26a700 1 -- 172.21.15.12:0/3376621241 shutdown_connections 2024-05-08T23:45:44.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.672+0000 7f333f26a700 1 --2- 172.21.15.12:0/3376621241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3338100890 0x7f333810cd40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.672+0000 7f333f26a700 1 --2- 172.21.15.12:0/3376621241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3338105f00 0x7f33381062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.672+0000 7f333f26a700 1 --2- 172.21.15.12:0/3376621241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33380ffed0 0x7f3338100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.672+0000 7f333f26a700 1 -- 172.21.15.12:0/3376621241 >> 172.21.15.12:0/3376621241 conn(0x7f3338075f80 msgr2=0x7f3338076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:44.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.672+0000 7f333f26a700 1 -- 172.21.15.12:0/3376621241 shutdown_connections 2024-05-08T23:45:44.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.672+0000 7f333f26a700 1 -- 172.21.15.12:0/3376621241 wait complete. 2024-05-08T23:45:44.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333f26a700 1 Processor -- start 2024-05-08T23:45:44.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333f26a700 1 -- start start 2024-05-08T23:45:44.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333f26a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33380ffed0 0x7f3338101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333f26a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3338100890 0x7f33381020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333f26a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3338105f00 0x7f33381025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333f26a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33381a5890 con 0x7f33380ffed0 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333f26a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33381a5a00 con 0x7f3338100890 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333f26a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33381a5bc0 con 0x7f3338105f00 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333d006700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33380ffed0 0x7f3338101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333d006700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33380ffed0 0x7f3338101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47980/0 (socket says 172.21.15.12:47980) 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333d006700 1 -- 172.21.15.12:0/863971565 learned_addr learned my addr 172.21.15.12:0/863971565 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333c805700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3338100890 0x7f33381020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.673+0000 7f333d807700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3338105f00 0x7f33381025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f333d006700 1 -- 172.21.15.12:0/863971565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3338105f00 msgr2=0x7f33381025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f333d006700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3338105f00 0x7f33381025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f333d006700 1 -- 172.21.15.12:0/863971565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3338100890 msgr2=0x7f33381020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f333d006700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3338100890 0x7f33381020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f333d006700 1 -- 172.21.15.12:0/863971565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330009d00 con 0x7f33380ffed0 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f333d807700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3338105f00 0x7f33381025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f333d006700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33380ffed0 0x7f3338101b70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f333400e9a0 tx=0x7f333400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f332e7fc700 1 -- 172.21.15.12:0/863971565 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334009790 con 0x7f33380ffed0 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f333f26a700 1 -- 172.21.15.12:0/863971565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33381a5ea0 con 0x7f33380ffed0 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f332e7fc700 1 -- 172.21.15.12:0/863971565 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f333400cbd0 con 0x7f33380ffed0 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f332e7fc700 1 -- 172.21.15.12:0/863971565 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334018680 con 0x7f33380ffed0 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.674+0000 7f333f26a700 1 -- 172.21.15.12:0/863971565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33381a6360 con 0x7f33380ffed0 2024-05-08T23:45:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.675+0000 7f333f26a700 1 -- 172.21.15.12:0/863971565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33381015b0 con 0x7f33380ffed0 2024-05-08T23:45:44.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.677+0000 7f332e7fc700 1 -- 172.21.15.12:0/863971565 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f333400cd40 con 0x7f33380ffed0 2024-05-08T23:45:44.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.678+0000 7f332e7fc700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f33240a11b0 0x7f33240a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:44.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.678+0000 7f333c805700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f33240a11b0 0x7f33240a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:44.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.678+0000 7f332e7fc700 1 -- 172.21.15.12:0/863971565 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3334014070 con 0x7f33380ffed0 2024-05-08T23:45:44.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.678+0000 7f333c805700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f33240a11b0 0x7f33240a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3338103250 tx=0x7f332800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:44.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.682+0000 7f332e7fc700 1 -- 172.21.15.12:0/863971565 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3334096550 con 0x7f33380ffed0 2024-05-08T23:45:44.739 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215149 got 103079215151 for osd.3 2024-05-08T23:45:44.739 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:44.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:44 smithi204 ceph-mon[47373]: pgmap v245: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:45:44.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:44 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3262720844' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T23:45:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:44 smithi160 ceph-mon[46152]: pgmap v245: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:45:44.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:44 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3262720844' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T23:45:44.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.900+0000 7f333f26a700 1 -- 172.21.15.12:0/863971565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f333810fdb0 con 0x7f33380ffed0 2024-05-08T23:45:44.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.900+0000 7f332e7fc700 1 -- 172.21.15.12:0/863971565 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3334099b70 con 0x7f33380ffed0 2024-05-08T23:45:44.896 INFO:teuthology.orchestra.run.smithi012.stdout:210453397536 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 -- 172.21.15.12:0/863971565 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f33240a11b0 msgr2=0x7f33240a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f33240a11b0 0x7f33240a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3338103250 tx=0x7f332800b410 comp rx=0 tx=0).stop 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 -- 172.21.15.12:0/863971565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33380ffed0 msgr2=0x7f3338101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33380ffed0 0x7f3338101b70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f333400e9a0 tx=0x7f333400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 -- 172.21.15.12:0/863971565 shutdown_connections 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3338105f00 0x7f33381025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33380ffed0 0x7f3338101b70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f33240a11b0 0x7f33240a3670 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 --2- 172.21.15.12:0/863971565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3338100890 0x7f33381020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 -- 172.21.15.12:0/863971565 >> 172.21.15.12:0/863971565 conn(0x7f3338075f80 msgr2=0x7f33380ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:44.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 -- 172.21.15.12:0/863971565 shutdown_connections 2024-05-08T23:45:44.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:44.903+0000 7f333f26a700 1 -- 172.21.15.12:0/863971565 wait complete. 2024-05-08T23:45:45.033 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743443 got 296352743445 for osd.10 2024-05-08T23:45:45.033 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:45.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:45 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/863971565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T23:45:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.161+0000 7fea97bb7700 1 -- 172.21.15.12:0/4060062849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea900ffbc0 msgr2=0x7fea900fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.161+0000 7fea97bb7700 1 --2- 172.21.15.12:0/4060062849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea900ffbc0 0x7fea900fffa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009990 tx=0x7fea8c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.161+0000 7fea97bb7700 1 -- 172.21.15.12:0/4060062849 shutdown_connections 2024-05-08T23:45:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.161+0000 7fea97bb7700 1 --2- 172.21.15.12:0/4060062849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea901063e0 0x7fea9010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.161+0000 7fea97bb7700 1 --2- 172.21.15.12:0/4060062849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea900ffbc0 0x7fea900fffa0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.161+0000 7fea97bb7700 1 --2- 172.21.15.12:0/4060062849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea90100570 0x7fea90105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.161+0000 7fea97bb7700 1 -- 172.21.15.12:0/4060062849 >> 172.21.15.12:0/4060062849 conn(0x7fea90075190 msgr2=0x7fea900755a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.161+0000 7fea97bb7700 1 -- 172.21.15.12:0/4060062849 shutdown_connections 2024-05-08T23:45:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.161+0000 7fea97bb7700 1 -- 172.21.15.12:0/4060062849 wait complete. 2024-05-08T23:45:45.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea97bb7700 1 Processor -- start 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea97bb7700 1 -- start start 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea97bb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea900ffbc0 0x7fea90100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea97bb7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea90100570 0x7fea90101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea97bb7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea901063e0 0x7fea901015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea97bb7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901a5710 con 0x7fea900ffbc0 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea97bb7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901a5880 con 0x7fea901063e0 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea97bb7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901a5a40 con 0x7fea90100570 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea95953700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea900ffbc0 0x7fea90100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea95953700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea900ffbc0 0x7fea90100b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47992/0 (socket says 172.21.15.12:47992) 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.162+0000 7fea95953700 1 -- 172.21.15.12:0/2423023517 learned_addr learned my addr 172.21.15.12:0/2423023517 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea95152700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea90100570 0x7fea90101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea96154700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea901063e0 0x7fea901015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:45.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea95953700 1 -- 172.21.15.12:0/2423023517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea90100570 msgr2=0x7fea90101080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea95953700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea90100570 0x7fea90101080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea95953700 1 -- 172.21.15.12:0/2423023517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea901063e0 msgr2=0x7fea901015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea95953700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea901063e0 0x7fea901015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea95953700 1 -- 172.21.15.12:0/2423023517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c0095f0 con 0x7fea900ffbc0 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea95953700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea900ffbc0 0x7fea90100b40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005770 tx=0x7fea8c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea86ffd700 1 -- 172.21.15.12:0/2423023517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c01d070 con 0x7fea900ffbc0 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea86ffd700 1 -- 172.21.15.12:0/2423023517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea8c022470 con 0x7fea900ffbc0 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea86ffd700 1 -- 172.21.15.12:0/2423023517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c017650 con 0x7fea900ffbc0 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea97bb7700 1 -- 172.21.15.12:0/2423023517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea901a5cc0 con 0x7fea900ffbc0 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.163+0000 7fea97bb7700 1 -- 172.21.15.12:0/2423023517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea901a60d0 con 0x7fea900ffbc0 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.164+0000 7fea97bb7700 1 -- 172.21.15.12:0/2423023517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea901095c0 con 0x7fea900ffbc0 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.165+0000 7fea86ffd700 1 -- 172.21.15.12:0/2423023517 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fea8c00fb00 con 0x7fea900ffbc0 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.166+0000 7fea86ffd700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea7c0a1230 0x7fea7c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:45.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.166+0000 7fea95152700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea7c0a1230 0x7fea7c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:45.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.166+0000 7fea86ffd700 1 -- 172.21.15.12:0/2423023517 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fea8c0ce1a0 con 0x7fea900ffbc0 2024-05-08T23:45:45.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.167+0000 7fea95152700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea7c0a1230 0x7fea7c0a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea90102270 tx=0x7fea80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:45.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.170+0000 7fea86ffd700 1 -- 172.21.15.12:0/2423023517 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fea8c097a10 con 0x7fea900ffbc0 2024-05-08T23:45:45.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.387+0000 7fea97bb7700 1 -- 172.21.15.12:0/2423023517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fea90102020 con 0x7fea900ffbc0 2024-05-08T23:45:45.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.387+0000 7fea86ffd700 1 -- 172.21.15.12:0/2423023517 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fea8c09b030 con 0x7fea900ffbc0 2024-05-08T23:45:45.383 INFO:teuthology.orchestra.run.smithi012.stdout:55834574903 2024-05-08T23:45:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 -- 172.21.15.12:0/2423023517 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea7c0a1230 msgr2=0x7fea7c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:45.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea7c0a1230 0x7fea7c0a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea90102270 tx=0x7fea80009260 comp rx=0 tx=0).stop 2024-05-08T23:45:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 -- 172.21.15.12:0/2423023517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea900ffbc0 msgr2=0x7fea90100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea900ffbc0 0x7fea90100b40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005770 tx=0x7fea8c0057a0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 -- 172.21.15.12:0/2423023517 shutdown_connections 2024-05-08T23:45:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fea7c0a1230 0x7fea7c0a36f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea901063e0 0x7fea901015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea900ffbc0 0x7fea90100b40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 --2- 172.21.15.12:0/2423023517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea90100570 0x7fea90101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 -- 172.21.15.12:0/2423023517 >> 172.21.15.12:0/2423023517 conn(0x7fea90075190 msgr2=0x7fea901074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 -- 172.21.15.12:0/2423023517 shutdown_connections 2024-05-08T23:45:45.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:45.390+0000 7fea97bb7700 1 -- 172.21.15.12:0/2423023517 wait complete. 2024-05-08T23:45:45.426 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397534 got 210453397536 for osd.7 2024-05-08T23:45:45.426 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:45.755 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574900 got 55834574903 for osd.1 2024-05-08T23:45:45.755 DEBUG:teuthology.parallel:result is None 2024-05-08T23:45:45.755 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T23:45:45.756 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T23:45:45.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:45 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/863971565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T23:45:45.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:45 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/863971565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T23:45:46.016 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:46.453 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:46 smithi012 ceph-mon[42503]: pgmap v246: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T23:45:46.453 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:46 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2423023517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T23:45:46.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:46 smithi204 ceph-mon[47373]: pgmap v246: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T23:45:46.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:46 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2423023517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T23:45:46.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:46 smithi160 ceph-mon[46152]: pgmap v246: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T23:45:46.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:46 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2423023517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T23:45:47.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.228+0000 7f2727626700 1 -- 172.21.15.12:0/284174929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2720101410 msgr2=0x7f272010cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:47.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.228+0000 7f2727626700 1 --2- 172.21.15.12:0/284174929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2720101410 0x7f272010cd70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f271800b0d0 tx=0x7f271800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.228+0000 7f2727626700 1 -- 172.21.15.12:0/284174929 shutdown_connections 2024-05-08T23:45:47.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.228+0000 7f2727626700 1 --2- 172.21.15.12:0/284174929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2720101410 0x7f272010cd70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.228+0000 7f2727626700 1 --2- 172.21.15.12:0/284174929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2720104ed0 0x7f27201052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.228+0000 7f2727626700 1 --2- 172.21.15.12:0/284174929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2720100a50 0x7f2720100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.228+0000 7f2727626700 1 -- 172.21.15.12:0/284174929 >> 172.21.15.12:0/284174929 conn(0x7f2720075260 msgr2=0x7f2720075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:47.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.228+0000 7f2727626700 1 -- 172.21.15.12:0/284174929 shutdown_connections 2024-05-08T23:45:47.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.228+0000 7f2727626700 1 -- 172.21.15.12:0/284174929 wait complete. 2024-05-08T23:45:47.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2727626700 1 Processor -- start 2024-05-08T23:45:47.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2727626700 1 -- start start 2024-05-08T23:45:47.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2727626700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2720100a50 0x7f2720197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:47.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2727626700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2720101410 0x7f27201979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2727626700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2720104ed0 0x7f272019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2727626700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272019c360 con 0x7f2720101410 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2727626700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272019c4d0 con 0x7f2720104ed0 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2727626700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272019c690 con 0x7f2720100a50 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2724bc1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2720101410 0x7f27201979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2724bc1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2720101410 0x7f27201979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48014/0 (socket says 172.21.15.12:48014) 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2724bc1700 1 -- 172.21.15.12:0/495515681 learned_addr learned my addr 172.21.15.12:0/495515681 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f27253c2700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2720100a50 0x7f2720197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.229+0000 7f2725bc3700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2720104ed0 0x7f272019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f2724bc1700 1 -- 172.21.15.12:0/495515681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2720100a50 msgr2=0x7f2720197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f2724bc1700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2720100a50 0x7f2720197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f2724bc1700 1 -- 172.21.15.12:0/495515681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2720104ed0 msgr2=0x7f272019bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f2724bc1700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2720104ed0 0x7f272019bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f2724bc1700 1 -- 172.21.15.12:0/495515681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2718009d00 con 0x7f2720101410 2024-05-08T23:45:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f2724bc1700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2720101410 0x7f27201979a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f271000e9c0 tx=0x7f271000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f27167fc700 1 -- 172.21.15.12:0/495515681 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27100097b0 con 0x7f2720101410 2024-05-08T23:45:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f2727626700 1 -- 172.21.15.12:0/495515681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272019c970 con 0x7f2720101410 2024-05-08T23:45:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f27167fc700 1 -- 172.21.15.12:0/495515681 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f271000ee60 con 0x7f2720101410 2024-05-08T23:45:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f27167fc700 1 -- 172.21.15.12:0/495515681 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27100054f0 con 0x7f2720101410 2024-05-08T23:45:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.230+0000 7f2727626700 1 -- 172.21.15.12:0/495515681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27201a41d0 con 0x7f2720101410 2024-05-08T23:45:47.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.231+0000 7f2727626700 1 -- 172.21.15.12:0/495515681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27201094d0 con 0x7f2720101410 2024-05-08T23:45:47.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.232+0000 7f27167fc700 1 -- 172.21.15.12:0/495515681 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2710004750 con 0x7f2720101410 2024-05-08T23:45:47.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.233+0000 7f27167fc700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f270c0a11b0 0x7f270c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:47.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.233+0000 7f27253c2700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f270c0a11b0 0x7f270c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:47.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.233+0000 7f27167fc700 1 -- 172.21.15.12:0/495515681 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2710014070 con 0x7f2720101410 2024-05-08T23:45:47.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.234+0000 7f27253c2700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f270c0a11b0 0x7f270c0a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f271c005950 tx=0x7f271c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:47.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.237+0000 7f27167fc700 1 -- 172.21.15.12:0/495515681 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f27100964a0 con 0x7f2720101410 2024-05-08T23:45:47.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.448+0000 7f2727626700 1 -- 172.21.15.12:0/495515681 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2720198770 con 0x7f270c0a11b0 2024-05-08T23:45:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.459+0000 7f27167fc700 1 -- 172.21.15.12:0/495515681 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341517 (secure 0 0 0) 0x7f2720198770 con 0x7f270c0a11b0 2024-05-08T23:45:47.459 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.462+0000 7f2727626700 1 -- 172.21.15.12:0/495515681 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f270c0a11b0 msgr2=0x7f270c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.462+0000 7f2727626700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f270c0a11b0 0x7f270c0a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f271c005950 tx=0x7f271c00b410 comp rx=0 tx=0).stop 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.462+0000 7f2727626700 1 -- 172.21.15.12:0/495515681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2720101410 msgr2=0x7f27201979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.462+0000 7f2727626700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2720101410 0x7f27201979a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f271000e9c0 tx=0x7f271000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.462+0000 7f2727626700 1 -- 172.21.15.12:0/495515681 shutdown_connections 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.462+0000 7f2727626700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f270c0a11b0 0x7f270c0a3670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.462+0000 7f2727626700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2720104ed0 0x7f272019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.462+0000 7f2727626700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2720100a50 0x7f2720197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.462+0000 7f2727626700 1 --2- 172.21.15.12:0/495515681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2720101410 0x7f27201979a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.462+0000 7f2727626700 1 -- 172.21.15.12:0/495515681 >> 172.21.15.12:0/495515681 conn(0x7f2720075260 msgr2=0x7f27200ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.463+0000 7f2727626700 1 -- 172.21.15.12:0/495515681 shutdown_connections 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:47.463+0000 7f2727626700 1 -- 172.21.15.12:0/495515681 wait complete. 2024-05-08T23:45:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-08T23:45:47.847 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-05-08T23:45:47.263408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653535,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":374,"num_read_kb":304,"num_write":331,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373420,"kb_used_data":12356,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124568404,"statfs":{"total":1151940427776,"available":1151558045696,"internally_reserved":0,"allocated":12652544,"data_stored":7007333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003120"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040868+0000","last_change":"2024-05-08T23:45:04.044088+0000","last_active":"2024-05-08T23:45:05.040868+0000","last_peered":"2024-05-08T23:45:05.040868+0000","last_clean":"2024-05-08T23:45:05.040868+0000","last_became_active":"2024-05-08T23:45:04.043535+0000","last_became_peered":"2024-05-08T23:45:04.043535+0000","last_unstale":"2024-05-08T23:45:05.040868+0000","last_undegraded":"2024-05-08T23:45:05.040868+0000","last_fullsized":"2024-05-08T23:45:05.040868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:52.363402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282575+0000","last_change":"2024-05-08T23:45:04.049201+0000","last_active":"2024-05-08T23:45:06.282575+0000","last_peered":"2024-05-08T23:45:06.282575+0000","last_clean":"2024-05-08T23:45:06.282575+0000","last_became_active":"2024-05-08T23:45:04.048743+0000","last_became_peered":"2024-05-08T23:45:04.048743+0000","last_unstale":"2024-05-08T23:45:06.282575+0000","last_undegraded":"2024-05-08T23:45:06.282575+0000","last_fullsized":"2024-05-08T23:45:06.282575+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:26.191549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282032+0000","last_change":"2024-05-08T23:45:04.047356+0000","last_active":"2024-05-08T23:45:06.282032+0000","last_peered":"2024-05-08T23:45:06.282032+0000","last_clean":"2024-05-08T23:45:06.282032+0000","last_became_active":"2024-05-08T23:45:04.046996+0000","last_became_peered":"2024-05-08T23:45:04.046996+0000","last_unstale":"2024-05-08T23:45:06.282032+0000","last_undegraded":"2024-05-08T23:45:06.282032+0000","last_fullsized":"2024-05-08T23:45:06.282032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:58.722522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282798+0000","last_change":"2024-05-08T23:45:04.054754+0000","last_active":"2024-05-08T23:45:06.282798+0000","last_peered":"2024-05-08T23:45:06.282798+0000","last_clean":"2024-05-08T23:45:06.282798+0000","last_became_active":"2024-05-08T23:45:04.054302+0000","last_became_peered":"2024-05-08T23:45:04.054302+0000","last_unstale":"2024-05-08T23:45:06.282798+0000","last_undegraded":"2024-05-08T23:45:06.282798+0000","last_fullsized":"2024-05-08T23:45:06.282798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:28.882982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517234+0000","last_change":"2024-05-08T23:45:04.047189+0000","last_active":"2024-05-08T23:45:05.517234+0000","last_peered":"2024-05-08T23:45:05.517234+0000","last_clean":"2024-05-08T23:45:05.517234+0000","last_became_active":"2024-05-08T23:45:04.046756+0000","last_became_peered":"2024-05-08T23:45:04.046756+0000","last_unstale":"2024-05-08T23:45:05.517234+0000","last_undegraded":"2024-05-08T23:45:05.517234+0000","last_fullsized":"2024-05-08T23:45:05.517234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.780169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045028+0000","last_change":"2024-05-08T23:45:04.060897+0000","last_active":"2024-05-08T23:45:05.045028+0000","last_peered":"2024-05-08T23:45:05.045028+0000","last_clean":"2024-05-08T23:45:05.045028+0000","last_became_active":"2024-05-08T23:45:04.060550+0000","last_became_peered":"2024-05-08T23:45:04.060550+0000","last_unstale":"2024-05-08T23:45:05.045028+0000","last_undegraded":"2024-05-08T23:45:05.045028+0000","last_fullsized":"2024-05-08T23:45:05.045028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:59.356057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781043+0000","last_change":"2024-05-08T23:45:04.059308+0000","last_active":"2024-05-08T23:45:06.781043+0000","last_peered":"2024-05-08T23:45:06.781043+0000","last_clean":"2024-05-08T23:45:06.781043+0000","last_became_active":"2024-05-08T23:45:04.058920+0000","last_became_peered":"2024-05-08T23:45:04.058920+0000","last_unstale":"2024-05-08T23:45:06.781043+0000","last_undegraded":"2024-05-08T23:45:06.781043+0000","last_fullsized":"2024-05-08T23:45:06.781043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:11.857062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039706+0000","last_change":"2024-05-08T23:45:04.061542+0000","last_active":"2024-05-08T23:45:05.039706+0000","last_peered":"2024-05-08T23:45:05.039706+0000","last_clean":"2024-05-08T23:45:05.039706+0000","last_became_active":"2024-05-08T23:45:04.061145+0000","last_became_peered":"2024-05-08T23:45:04.061145+0000","last_unstale":"2024-05-08T23:45:05.039706+0000","last_undegraded":"2024-05-08T23:45:05.039706+0000","last_fullsized":"2024-05-08T23:45:05.039706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:53.159292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300493+0000","last_change":"2024-05-08T23:45:04.052292+0000","last_active":"2024-05-08T23:45:04.300493+0000","last_peered":"2024-05-08T23:45:04.300493+0000","last_clean":"2024-05-08T23:45:04.300493+0000","last_became_active":"2024-05-08T23:45:04.051164+0000","last_became_peered":"2024-05-08T23:45:04.051164+0000","last_unstale":"2024-05-08T23:45:04.300493+0000","last_undegraded":"2024-05-08T23:45:04.300493+0000","last_fullsized":"2024-05-08T23:45:04.300493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:09.218130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045014+0000","last_change":"2024-05-08T23:45:04.054566+0000","last_active":"2024-05-08T23:45:05.045014+0000","last_peered":"2024-05-08T23:45:05.045014+0000","last_clean":"2024-05-08T23:45:05.045014+0000","last_became_active":"2024-05-08T23:45:04.053878+0000","last_became_peered":"2024-05-08T23:45:04.053878+0000","last_unstale":"2024-05-08T23:45:05.045014+0000","last_undegraded":"2024-05-08T23:45:05.045014+0000","last_fullsized":"2024-05-08T23:45:05.045014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:34.760038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781480+0000","last_change":"2024-05-08T23:45:04.049696+0000","last_active":"2024-05-08T23:45:06.781480+0000","last_peered":"2024-05-08T23:45:06.781480+0000","last_clean":"2024-05-08T23:45:06.781480+0000","last_became_active":"2024-05-08T23:45:04.049351+0000","last_became_peered":"2024-05-08T23:45:04.049351+0000","last_unstale":"2024-05-08T23:45:06.781480+0000","last_undegraded":"2024-05-08T23:45:06.781480+0000","last_fullsized":"2024-05-08T23:45:06.781480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:53.557738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282779+0000","last_change":"2024-05-08T23:45:04.051010+0000","last_active":"2024-05-08T23:45:06.282779+0000","last_peered":"2024-05-08T23:45:06.282779+0000","last_clean":"2024-05-08T23:45:06.282779+0000","last_became_active":"2024-05-08T23:45:04.050715+0000","last_became_peered":"2024-05-08T23:45:04.050715+0000","last_unstale":"2024-05-08T23:45:06.282779+0000","last_undegraded":"2024-05-08T23:45:06.282779+0000","last_fullsized":"2024-05-08T23:45:06.282779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:09.305974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022192+0000","last_change":"2024-05-08T23:45:04.051005+0000","last_active":"2024-05-08T23:45:05.022192+0000","last_peered":"2024-05-08T23:45:05.022192+0000","last_clean":"2024-05-08T23:45:05.022192+0000","last_became_active":"2024-05-08T23:45:04.050123+0000","last_became_peered":"2024-05-08T23:45:04.050123+0000","last_unstale":"2024-05-08T23:45:05.022192+0000","last_undegraded":"2024-05-08T23:45:05.022192+0000","last_fullsized":"2024-05-08T23:45:05.022192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:52.206389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282033+0000","last_change":"2024-05-08T23:45:04.049942+0000","last_active":"2024-05-08T23:45:06.282033+0000","last_peered":"2024-05-08T23:45:06.282033+0000","last_clean":"2024-05-08T23:45:06.282033+0000","last_became_active":"2024-05-08T23:45:04.049596+0000","last_became_peered":"2024-05-08T23:45:04.049596+0000","last_unstale":"2024-05-08T23:45:06.282033+0000","last_undegraded":"2024-05-08T23:45:06.282033+0000","last_fullsized":"2024-05-08T23:45:06.282033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:30.907023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282203+0000","last_change":"2024-05-08T23:45:04.048985+0000","last_active":"2024-05-08T23:45:06.282203+0000","last_peered":"2024-05-08T23:45:06.282203+0000","last_clean":"2024-05-08T23:45:06.282203+0000","last_became_active":"2024-05-08T23:45:04.048658+0000","last_became_peered":"2024-05-08T23:45:04.048658+0000","last_unstale":"2024-05-08T23:45:06.282203+0000","last_undegraded":"2024-05-08T23:45:06.282203+0000","last_fullsized":"2024-05-08T23:45:06.282203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:17.030930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300415+0000","last_change":"2024-05-08T23:45:04.051903+0000","last_active":"2024-05-08T23:45:04.300415+0000","last_peered":"2024-05-08T23:45:04.300415+0000","last_clean":"2024-05-08T23:45:04.300415+0000","last_became_active":"2024-05-08T23:45:04.051221+0000","last_became_peered":"2024-05-08T23:45:04.051221+0000","last_unstale":"2024-05-08T23:45:04.300415+0000","last_undegraded":"2024-05-08T23:45:04.300415+0000","last_fullsized":"2024-05-08T23:45:04.300415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:01.554463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040970+0000","last_change":"2024-05-08T23:45:04.043854+0000","last_active":"2024-05-08T23:45:05.040970+0000","last_peered":"2024-05-08T23:45:05.040970+0000","last_clean":"2024-05-08T23:45:05.040970+0000","last_became_active":"2024-05-08T23:45:04.043056+0000","last_became_peered":"2024-05-08T23:45:04.043056+0000","last_unstale":"2024-05-08T23:45:05.040970+0000","last_undegraded":"2024-05-08T23:45:05.040970+0000","last_fullsized":"2024-05-08T23:45:05.040970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:18.175575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.058883+0000","last_change":"2024-05-08T23:45:04.049332+0000","last_active":"2024-05-08T23:45:04.058883+0000","last_peered":"2024-05-08T23:45:04.058883+0000","last_clean":"2024-05-08T23:45:04.058883+0000","last_became_active":"2024-05-08T23:45:04.048987+0000","last_became_peered":"2024-05-08T23:45:04.048987+0000","last_unstale":"2024-05-08T23:45:04.058883+0000","last_undegraded":"2024-05-08T23:45:04.058883+0000","last_fullsized":"2024-05-08T23:45:04.058883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:02.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225413+0000","last_change":"2024-05-08T23:45:04.048536+0000","last_active":"2024-05-08T23:45:06.225413+0000","last_peered":"2024-05-08T23:45:06.225413+0000","last_clean":"2024-05-08T23:45:06.225413+0000","last_became_active":"2024-05-08T23:45:04.048102+0000","last_became_peered":"2024-05-08T23:45:04.048102+0000","last_unstale":"2024-05-08T23:45:06.225413+0000","last_undegraded":"2024-05-08T23:45:06.225413+0000","last_fullsized":"2024-05-08T23:45:06.225413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:11.794372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.071241+0000","last_change":"2024-05-08T23:45:03.576833+0000","last_active":"2024-05-08T23:45:08.071241+0000","last_peered":"2024-05-08T23:45:08.071241+0000","last_clean":"2024-05-08T23:45:08.071241+0000","last_became_active":"2024-05-08T23:45:03.576495+0000","last_became_peered":"2024-05-08T23:45:03.576495+0000","last_unstale":"2024-05-08T23:45:08.071241+0000","last_undegraded":"2024-05-08T23:45:08.071241+0000","last_fullsized":"2024-05-08T23:45:08.071241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:50.266658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040105+0000","last_change":"2024-05-08T23:45:04.055444+0000","last_active":"2024-05-08T23:45:05.040105+0000","last_peered":"2024-05-08T23:45:05.040105+0000","last_clean":"2024-05-08T23:45:05.040105+0000","last_became_active":"2024-05-08T23:45:04.054998+0000","last_became_peered":"2024-05-08T23:45:04.054998+0000","last_unstale":"2024-05-08T23:45:05.040105+0000","last_undegraded":"2024-05-08T23:45:05.040105+0000","last_fullsized":"2024-05-08T23:45:05.040105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:03.470555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.100638+0000","last_change":"2024-05-08T23:45:03.578362+0000","last_active":"2024-05-08T23:45:05.100638+0000","last_peered":"2024-05-08T23:45:05.100638+0000","last_clean":"2024-05-08T23:45:05.100638+0000","last_became_active":"2024-05-08T23:45:03.577834+0000","last_became_peered":"2024-05-08T23:45:03.577834+0000","last_unstale":"2024-05-08T23:45:05.100638+0000","last_undegraded":"2024-05-08T23:45:05.100638+0000","last_fullsized":"2024-05-08T23:45:05.100638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:01.943490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282763+0000","last_change":"2024-05-08T23:45:04.047205+0000","last_active":"2024-05-08T23:45:06.282763+0000","last_peered":"2024-05-08T23:45:06.282763+0000","last_clean":"2024-05-08T23:45:06.282763+0000","last_became_active":"2024-05-08T23:45:04.046860+0000","last_became_peered":"2024-05-08T23:45:04.046860+0000","last_unstale":"2024-05-08T23:45:06.282763+0000","last_undegraded":"2024-05-08T23:45:06.282763+0000","last_fullsized":"2024-05-08T23:45:06.282763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:36.837101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.069768+0000","last_change":"2024-05-08T23:45:03.582026+0000","last_active":"2024-05-08T23:45:08.069768+0000","last_peered":"2024-05-08T23:45:08.069768+0000","last_clean":"2024-05-08T23:45:08.069768+0000","last_became_active":"2024-05-08T23:45:03.581672+0000","last_became_peered":"2024-05-08T23:45:03.581672+0000","last_unstale":"2024-05-08T23:45:08.069768+0000","last_undegraded":"2024-05-08T23:45:08.069768+0000","last_fullsized":"2024-05-08T23:45:08.069768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:50.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300470+0000","last_change":"2024-05-08T23:45:04.052639+0000","last_active":"2024-05-08T23:45:04.300470+0000","last_peered":"2024-05-08T23:45:04.300470+0000","last_clean":"2024-05-08T23:45:04.300470+0000","last_became_active":"2024-05-08T23:45:04.052158+0000","last_became_peered":"2024-05-08T23:45:04.052158+0000","last_unstale":"2024-05-08T23:45:04.300470+0000","last_undegraded":"2024-05-08T23:45:04.300470+0000","last_fullsized":"2024-05-08T23:45:04.300470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:25.638555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517264+0000","last_change":"2024-05-08T23:45:03.579221+0000","last_active":"2024-05-08T23:45:05.517264+0000","last_peered":"2024-05-08T23:45:05.517264+0000","last_clean":"2024-05-08T23:45:05.517264+0000","last_became_active":"2024-05-08T23:45:03.578795+0000","last_became_peered":"2024-05-08T23:45:03.578795+0000","last_unstale":"2024-05-08T23:45:05.517264+0000","last_undegraded":"2024-05-08T23:45:05.517264+0000","last_fullsized":"2024-05-08T23:45:05.517264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:14.064108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563925+0000","last_change":"2024-05-08T23:45:04.057573+0000","last_active":"2024-05-08T23:45:05.563925+0000","last_peered":"2024-05-08T23:45:05.563925+0000","last_clean":"2024-05-08T23:45:05.563925+0000","last_became_active":"2024-05-08T23:45:04.053201+0000","last_became_peered":"2024-05-08T23:45:04.053201+0000","last_unstale":"2024-05-08T23:45:05.563925+0000","last_undegraded":"2024-05-08T23:45:05.563925+0000","last_fullsized":"2024-05-08T23:45:05.563925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:23.612609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":56,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:44.129534+0000","last_change":"2024-05-08T23:45:03.578110+0000","last_active":"2024-05-08T23:45:44.129534+0000","last_peered":"2024-05-08T23:45:44.129534+0000","last_clean":"2024-05-08T23:45:44.129534+0000","last_became_active":"2024-05-08T23:45:03.577749+0000","last_became_peered":"2024-05-08T23:45:03.577749+0000","last_unstale":"2024-05-08T23:45:44.129534+0000","last_undegraded":"2024-05-08T23:45:44.129534+0000","last_fullsized":"2024-05-08T23:45:44.129534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:50.526535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517216+0000","last_change":"2024-05-08T23:45:04.048131+0000","last_active":"2024-05-08T23:45:05.517216+0000","last_peered":"2024-05-08T23:45:05.517216+0000","last_clean":"2024-05-08T23:45:05.517216+0000","last_became_active":"2024-05-08T23:45:04.047773+0000","last_became_peered":"2024-05-08T23:45:04.047773+0000","last_unstale":"2024-05-08T23:45:05.517216+0000","last_undegraded":"2024-05-08T23:45:05.517216+0000","last_fullsized":"2024-05-08T23:45:05.517216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:47.944900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.070447+0000","last_change":"2024-05-08T23:45:03.586710+0000","last_active":"2024-05-08T23:45:08.070447+0000","last_peered":"2024-05-08T23:45:08.070447+0000","last_clean":"2024-05-08T23:45:08.070447+0000","last_became_active":"2024-05-08T23:45:03.586411+0000","last_became_peered":"2024-05-08T23:45:03.586411+0000","last_unstale":"2024-05-08T23:45:08.070447+0000","last_undegraded":"2024-05-08T23:45:08.070447+0000","last_fullsized":"2024-05-08T23:45:08.070447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:25.938846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563896+0000","last_change":"2024-05-08T23:45:04.053885+0000","last_active":"2024-05-08T23:45:05.563896+0000","last_peered":"2024-05-08T23:45:05.563896+0000","last_clean":"2024-05-08T23:45:05.563896+0000","last_became_active":"2024-05-08T23:45:04.053385+0000","last_became_peered":"2024-05-08T23:45:04.053385+0000","last_unstale":"2024-05-08T23:45:05.563896+0000","last_undegraded":"2024-05-08T23:45:05.563896+0000","last_fullsized":"2024-05-08T23:45:05.563896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:10.805481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.071417+0000","last_change":"2024-05-08T23:45:03.574988+0000","last_active":"2024-05-08T23:45:08.071417+0000","last_peered":"2024-05-08T23:45:08.071417+0000","last_clean":"2024-05-08T23:45:08.071417+0000","last_became_active":"2024-05-08T23:45:03.574586+0000","last_became_peered":"2024-05-08T23:45:03.574586+0000","last_unstale":"2024-05-08T23:45:08.071417+0000","last_undegraded":"2024-05-08T23:45:08.071417+0000","last_fullsized":"2024-05-08T23:45:08.071417+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:18.005107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039815+0000","last_change":"2024-05-08T23:45:04.046817+0000","last_active":"2024-05-08T23:45:05.039815+0000","last_peered":"2024-05-08T23:45:05.039815+0000","last_clean":"2024-05-08T23:45:05.039815+0000","last_became_active":"2024-05-08T23:45:04.046515+0000","last_became_peered":"2024-05-08T23:45:04.046515+0000","last_unstale":"2024-05-08T23:45:05.039815+0000","last_undegraded":"2024-05-08T23:45:05.039815+0000","last_fullsized":"2024-05-08T23:45:05.039815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:29.439670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282372+0000","last_change":"2024-05-08T23:45:03.579002+0000","last_active":"2024-05-08T23:45:06.282372+0000","last_peered":"2024-05-08T23:45:06.282372+0000","last_clean":"2024-05-08T23:45:06.282372+0000","last_became_active":"2024-05-08T23:45:03.577891+0000","last_became_peered":"2024-05-08T23:45:03.577891+0000","last_unstale":"2024-05-08T23:45:06.282372+0000","last_undegraded":"2024-05-08T23:45:06.282372+0000","last_fullsized":"2024-05-08T23:45:06.282372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:01.027574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.061375+0000","last_change":"2024-05-08T23:45:04.060715+0000","last_active":"2024-05-08T23:45:04.061375+0000","last_peered":"2024-05-08T23:45:04.061375+0000","last_clean":"2024-05-08T23:45:04.061375+0000","last_became_active":"2024-05-08T23:45:04.060375+0000","last_became_peered":"2024-05-08T23:45:04.060375+0000","last_unstale":"2024-05-08T23:45:04.061375+0000","last_undegraded":"2024-05-08T23:45:04.061375+0000","last_fullsized":"2024-05-08T23:45:04.061375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:42.133323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:42.125525+0000","last_change":"2024-05-08T23:45:03.587784+0000","last_active":"2024-05-08T23:45:42.125525+0000","last_peered":"2024-05-08T23:45:42.125525+0000","last_clean":"2024-05-08T23:45:42.125525+0000","last_became_active":"2024-05-08T23:45:03.587529+0000","last_became_peered":"2024-05-08T23:45:03.587529+0000","last_unstale":"2024-05-08T23:45:42.125525+0000","last_undegraded":"2024-05-08T23:45:42.125525+0000","last_fullsized":"2024-05-08T23:45:42.125525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:00.323599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300481+0000","last_change":"2024-05-08T23:45:04.062672+0000","last_active":"2024-05-08T23:45:04.300481+0000","last_peered":"2024-05-08T23:45:04.300481+0000","last_clean":"2024-05-08T23:45:04.300481+0000","last_became_active":"2024-05-08T23:45:04.062351+0000","last_became_peered":"2024-05-08T23:45:04.062351+0000","last_unstale":"2024-05-08T23:45:04.300481+0000","last_undegraded":"2024-05-08T23:45:04.300481+0000","last_fullsized":"2024-05-08T23:45:04.300481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:13.740239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.060380+0000","last_change":"2024-05-08T23:45:03.583501+0000","last_active":"2024-05-08T23:45:07.060380+0000","last_peered":"2024-05-08T23:45:07.060380+0000","last_clean":"2024-05-08T23:45:07.060380+0000","last_became_active":"2024-05-08T23:45:03.583116+0000","last_became_peered":"2024-05-08T23:45:03.583116+0000","last_unstale":"2024-05-08T23:45:07.060380+0000","last_undegraded":"2024-05-08T23:45:07.060380+0000","last_fullsized":"2024-05-08T23:45:07.060380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:03.797524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282765+0000","last_change":"2024-05-08T23:45:04.048336+0000","last_active":"2024-05-08T23:45:06.282765+0000","last_peered":"2024-05-08T23:45:06.282765+0000","last_clean":"2024-05-08T23:45:06.282765+0000","last_became_active":"2024-05-08T23:45:04.048000+0000","last_became_peered":"2024-05-08T23:45:04.048000+0000","last_unstale":"2024-05-08T23:45:06.282765+0000","last_undegraded":"2024-05-08T23:45:06.282765+0000","last_fullsized":"2024-05-08T23:45:06.282765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:41.806600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062204+0000","last_change":"2024-05-08T23:45:03.581215+0000","last_active":"2024-05-08T23:45:07.062204+0000","last_peered":"2024-05-08T23:45:07.062204+0000","last_clean":"2024-05-08T23:45:07.062204+0000","last_became_active":"2024-05-08T23:45:03.580496+0000","last_became_peered":"2024-05-08T23:45:03.580496+0000","last_unstale":"2024-05-08T23:45:07.062204+0000","last_undegraded":"2024-05-08T23:45:07.062204+0000","last_fullsized":"2024-05-08T23:45:07.062204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:25.602401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300459+0000","last_change":"2024-05-08T23:45:04.058857+0000","last_active":"2024-05-08T23:45:04.300459+0000","last_peered":"2024-05-08T23:45:04.300459+0000","last_clean":"2024-05-08T23:45:04.300459+0000","last_became_active":"2024-05-08T23:45:04.057988+0000","last_became_peered":"2024-05-08T23:45:04.057988+0000","last_unstale":"2024-05-08T23:45:04.300459+0000","last_undegraded":"2024-05-08T23:45:04.300459+0000","last_fullsized":"2024-05-08T23:45:04.300459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:46.725976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'27","reported_seq":116,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:43.137581+0000","last_change":"2024-05-08T23:45:03.582121+0000","last_active":"2024-05-08T23:45:43.137581+0000","last_peered":"2024-05-08T23:45:43.137581+0000","last_clean":"2024-05-08T23:45:43.137581+0000","last_became_active":"2024-05-08T23:45:03.581796+0000","last_became_peered":"2024-05-08T23:45:03.581796+0000","last_unstale":"2024-05-08T23:45:43.137581+0000","last_undegraded":"2024-05-08T23:45:43.137581+0000","last_fullsized":"2024-05-08T23:45:43.137581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:55.417404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59419,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":77,"num_read_kb":96,"num_write":28,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517284+0000","last_change":"2024-05-08T23:45:04.052993+0000","last_active":"2024-05-08T23:45:05.517284+0000","last_peered":"2024-05-08T23:45:05.517284+0000","last_clean":"2024-05-08T23:45:05.517284+0000","last_became_active":"2024-05-08T23:45:04.052088+0000","last_became_peered":"2024-05-08T23:45:04.052088+0000","last_unstale":"2024-05-08T23:45:05.517284+0000","last_undegraded":"2024-05-08T23:45:05.517284+0000","last_fullsized":"2024-05-08T23:45:05.517284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:44.531649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.070552+0000","last_change":"2024-05-08T23:45:03.575311+0000","last_active":"2024-05-08T23:45:08.070552+0000","last_peered":"2024-05-08T23:45:08.070552+0000","last_clean":"2024-05-08T23:45:08.070552+0000","last_became_active":"2024-05-08T23:45:03.574949+0000","last_became_peered":"2024-05-08T23:45:03.574949+0000","last_unstale":"2024-05-08T23:45:08.070552+0000","last_undegraded":"2024-05-08T23:45:08.070552+0000","last_fullsized":"2024-05-08T23:45:08.070552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:18.129488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022182+0000","last_change":"2024-05-08T23:45:04.050294+0000","last_active":"2024-05-08T23:45:05.022182+0000","last_peered":"2024-05-08T23:45:05.022182+0000","last_clean":"2024-05-08T23:45:05.022182+0000","last_became_active":"2024-05-08T23:45:04.049949+0000","last_became_peered":"2024-05-08T23:45:04.049949+0000","last_unstale":"2024-05-08T23:45:05.022182+0000","last_undegraded":"2024-05-08T23:45:05.022182+0000","last_fullsized":"2024-05-08T23:45:05.022182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:17.253201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061055+0000","last_change":"2024-05-08T23:45:03.579821+0000","last_active":"2024-05-08T23:45:07.061055+0000","last_peered":"2024-05-08T23:45:07.061055+0000","last_clean":"2024-05-08T23:45:07.061055+0000","last_became_active":"2024-05-08T23:45:03.578635+0000","last_became_peered":"2024-05-08T23:45:03.578635+0000","last_unstale":"2024-05-08T23:45:07.061055+0000","last_undegraded":"2024-05-08T23:45:07.061055+0000","last_fullsized":"2024-05-08T23:45:07.061055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:46.179505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781264+0000","last_change":"2024-05-08T23:45:04.059357+0000","last_active":"2024-05-08T23:45:06.781264+0000","last_peered":"2024-05-08T23:45:06.781264+0000","last_clean":"2024-05-08T23:45:06.781264+0000","last_became_active":"2024-05-08T23:45:04.059070+0000","last_became_peered":"2024-05-08T23:45:04.059070+0000","last_unstale":"2024-05-08T23:45:06.781264+0000","last_undegraded":"2024-05-08T23:45:06.781264+0000","last_fullsized":"2024-05-08T23:45:06.781264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:50.626787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:42.130903+0000","last_change":"2024-05-08T23:45:03.577199+0000","last_active":"2024-05-08T23:45:42.130903+0000","last_peered":"2024-05-08T23:45:42.130903+0000","last_clean":"2024-05-08T23:45:42.130903+0000","last_became_active":"2024-05-08T23:45:03.576844+0000","last_became_peered":"2024-05-08T23:45:03.576844+0000","last_unstale":"2024-05-08T23:45:42.130903+0000","last_undegraded":"2024-05-08T23:45:42.130903+0000","last_fullsized":"2024-05-08T23:45:42.130903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:27.139493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.780964+0000","last_change":"2024-05-08T23:45:04.064515+0000","last_active":"2024-05-08T23:45:06.780964+0000","last_peered":"2024-05-08T23:45:06.780964+0000","last_clean":"2024-05-08T23:45:06.780964+0000","last_became_active":"2024-05-08T23:45:04.064184+0000","last_became_peered":"2024-05-08T23:45:04.064184+0000","last_unstale":"2024-05-08T23:45:06.780964+0000","last_undegraded":"2024-05-08T23:45:06.780964+0000","last_fullsized":"2024-05-08T23:45:06.780964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:46.423121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.072701+0000","last_change":"2024-05-08T23:45:03.576766+0000","last_active":"2024-05-08T23:45:08.072701+0000","last_peered":"2024-05-08T23:45:08.072701+0000","last_clean":"2024-05-08T23:45:08.072701+0000","last_became_active":"2024-05-08T23:45:03.576410+0000","last_became_peered":"2024-05-08T23:45:03.576410+0000","last_unstale":"2024-05-08T23:45:08.072701+0000","last_undegraded":"2024-05-08T23:45:08.072701+0000","last_fullsized":"2024-05-08T23:45:08.072701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:29.444589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563841+0000","last_change":"2024-05-08T23:45:04.056563+0000","last_active":"2024-05-08T23:45:05.563841+0000","last_peered":"2024-05-08T23:45:05.563841+0000","last_clean":"2024-05-08T23:45:05.563841+0000","last_became_active":"2024-05-08T23:45:04.053132+0000","last_became_peered":"2024-05-08T23:45:04.053132+0000","last_unstale":"2024-05-08T23:45:05.563841+0000","last_undegraded":"2024-05-08T23:45:05.563841+0000","last_fullsized":"2024-05-08T23:45:05.563841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:41.625804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:09.074643+0000","last_change":"2024-05-08T23:45:03.580343+0000","last_active":"2024-05-08T23:45:09.074643+0000","last_peered":"2024-05-08T23:45:09.074643+0000","last_clean":"2024-05-08T23:45:09.074643+0000","last_became_active":"2024-05-08T23:45:03.578719+0000","last_became_peered":"2024-05-08T23:45:03.578719+0000","last_unstale":"2024-05-08T23:45:09.074643+0000","last_undegraded":"2024-05-08T23:45:09.074643+0000","last_fullsized":"2024-05-08T23:45:09.074643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:54.831252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040662+0000","last_change":"2024-05-08T23:45:04.050643+0000","last_active":"2024-05-08T23:45:05.040662+0000","last_peered":"2024-05-08T23:45:05.040662+0000","last_clean":"2024-05-08T23:45:05.040662+0000","last_became_active":"2024-05-08T23:45:04.050309+0000","last_became_peered":"2024-05-08T23:45:04.050309+0000","last_unstale":"2024-05-08T23:45:05.040662+0000","last_undegraded":"2024-05-08T23:45:05.040662+0000","last_fullsized":"2024-05-08T23:45:05.040662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:30.259530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.036233+0000","last_change":"2024-05-08T23:45:03.579590+0000","last_active":"2024-05-08T23:45:04.036233+0000","last_peered":"2024-05-08T23:45:04.036233+0000","last_clean":"2024-05-08T23:45:04.036233+0000","last_became_active":"2024-05-08T23:45:03.579213+0000","last_became_peered":"2024-05-08T23:45:03.579213+0000","last_unstale":"2024-05-08T23:45:04.036233+0000","last_undegraded":"2024-05-08T23:45:04.036233+0000","last_fullsized":"2024-05-08T23:45:04.036233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:55.666543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.320406+0000","last_change":"2024-05-08T23:45:04.049082+0000","last_active":"2024-05-08T23:45:08.320406+0000","last_peered":"2024-05-08T23:45:08.320406+0000","last_clean":"2024-05-08T23:45:08.320406+0000","last_became_active":"2024-05-08T23:45:04.048709+0000","last_became_peered":"2024-05-08T23:45:04.048709+0000","last_unstale":"2024-05-08T23:45:08.320406+0000","last_undegraded":"2024-05-08T23:45:08.320406+0000","last_fullsized":"2024-05-08T23:45:08.320406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:56.365817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.879739+0000","last_change":"2024-05-08T23:45:03.581363+0000","last_active":"2024-05-08T23:45:06.879739+0000","last_peered":"2024-05-08T23:45:06.879739+0000","last_clean":"2024-05-08T23:45:06.879739+0000","last_became_active":"2024-05-08T23:45:03.580840+0000","last_became_peered":"2024-05-08T23:45:03.580840+0000","last_unstale":"2024-05-08T23:45:06.879739+0000","last_undegraded":"2024-05-08T23:45:06.879739+0000","last_fullsized":"2024-05-08T23:45:06.879739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:27.281228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045138+0000","last_change":"2024-05-08T23:45:04.053524+0000","last_active":"2024-05-08T23:45:05.045138+0000","last_peered":"2024-05-08T23:45:05.045138+0000","last_clean":"2024-05-08T23:45:05.045138+0000","last_became_active":"2024-05-08T23:45:04.053118+0000","last_became_peered":"2024-05-08T23:45:04.053118+0000","last_unstale":"2024-05-08T23:45:05.045138+0000","last_undegraded":"2024-05-08T23:45:05.045138+0000","last_fullsized":"2024-05-08T23:45:05.045138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:56.542262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563822+0000","last_change":"2024-05-08T23:45:04.061189+0000","last_active":"2024-05-08T23:45:05.563822+0000","last_peered":"2024-05-08T23:45:05.563822+0000","last_clean":"2024-05-08T23:45:05.563822+0000","last_became_active":"2024-05-08T23:45:04.060838+0000","last_became_peered":"2024-05-08T23:45:04.060838+0000","last_unstale":"2024-05-08T23:45:05.563822+0000","last_undegraded":"2024-05-08T23:45:05.563822+0000","last_fullsized":"2024-05-08T23:45:05.563822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:43.794987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:43.131741+0000","last_change":"2024-05-08T23:45:03.583841+0000","last_active":"2024-05-08T23:45:43.131741+0000","last_peered":"2024-05-08T23:45:43.131741+0000","last_clean":"2024-05-08T23:45:43.131741+0000","last_became_active":"2024-05-08T23:45:03.583275+0000","last_became_peered":"2024-05-08T23:45:03.583275+0000","last_unstale":"2024-05-08T23:45:43.131741+0000","last_undegraded":"2024-05-08T23:45:43.131741+0000","last_fullsized":"2024-05-08T23:45:43.131741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:14.560505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039949+0000","last_change":"2024-05-08T23:45:04.052667+0000","last_active":"2024-05-08T23:45:05.039949+0000","last_peered":"2024-05-08T23:45:05.039949+0000","last_clean":"2024-05-08T23:45:05.039949+0000","last_became_active":"2024-05-08T23:45:04.052008+0000","last_became_peered":"2024-05-08T23:45:04.052008+0000","last_unstale":"2024-05-08T23:45:05.039949+0000","last_undegraded":"2024-05-08T23:45:05.039949+0000","last_fullsized":"2024-05-08T23:45:05.039949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:35.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:09.073198+0000","last_change":"2024-05-08T23:45:03.573665+0000","last_active":"2024-05-08T23:45:09.073198+0000","last_peered":"2024-05-08T23:45:09.073198+0000","last_clean":"2024-05-08T23:45:09.073198+0000","last_became_active":"2024-05-08T23:45:03.573265+0000","last_became_peered":"2024-05-08T23:45:03.573265+0000","last_unstale":"2024-05-08T23:45:09.073198+0000","last_undegraded":"2024-05-08T23:45:09.073198+0000","last_fullsized":"2024-05-08T23:45:09.073198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:13.410954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781044+0000","last_change":"2024-05-08T23:44:16.373962+0000","last_active":"2024-05-08T23:45:06.781044+0000","last_peered":"2024-05-08T23:45:06.781044+0000","last_clean":"2024-05-08T23:45:06.781044+0000","last_became_active":"2024-05-08T23:44:16.373358+0000","last_became_peered":"2024-05-08T23:44:16.373358+0000","last_unstale":"2024-05-08T23:45:06.781044+0000","last_undegraded":"2024-05-08T23:45:06.781044+0000","last_fullsized":"2024-05-08T23:45:06.781044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.067168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:24.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022109+0000","last_change":"2024-05-08T23:45:04.048468+0000","last_active":"2024-05-08T23:45:05.022109+0000","last_peered":"2024-05-08T23:45:05.022109+0000","last_clean":"2024-05-08T23:45:05.022109+0000","last_became_active":"2024-05-08T23:45:04.048085+0000","last_became_peered":"2024-05-08T23:45:04.048085+0000","last_unstale":"2024-05-08T23:45:05.022109+0000","last_undegraded":"2024-05-08T23:45:05.022109+0000","last_fullsized":"2024-05-08T23:45:05.022109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:18.039780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.069513+0000","last_change":"2024-05-08T23:45:03.575821+0000","last_active":"2024-05-08T23:45:08.069513+0000","last_peered":"2024-05-08T23:45:08.069513+0000","last_clean":"2024-05-08T23:45:08.069513+0000","last_became_active":"2024-05-08T23:45:03.575407+0000","last_became_peered":"2024-05-08T23:45:03.575407+0000","last_unstale":"2024-05-08T23:45:08.069513+0000","last_undegraded":"2024-05-08T23:45:08.069513+0000","last_fullsized":"2024-05-08T23:45:08.069513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:35.718942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.060787+0000","last_change":"2024-05-08T23:45:03.580100+0000","last_active":"2024-05-08T23:45:07.060787+0000","last_peered":"2024-05-08T23:45:07.060787+0000","last_clean":"2024-05-08T23:45:07.060787+0000","last_became_active":"2024-05-08T23:45:03.579791+0000","last_became_peered":"2024-05-08T23:45:03.579791+0000","last_unstale":"2024-05-08T23:45:07.060787+0000","last_undegraded":"2024-05-08T23:45:07.060787+0000","last_fullsized":"2024-05-08T23:45:07.060787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:41.086291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022221+0000","last_change":"2024-05-08T23:45:04.047687+0000","last_active":"2024-05-08T23:45:05.022221+0000","last_peered":"2024-05-08T23:45:05.022221+0000","last_clean":"2024-05-08T23:45:05.022221+0000","last_became_active":"2024-05-08T23:45:04.047343+0000","last_became_peered":"2024-05-08T23:45:04.047343+0000","last_unstale":"2024-05-08T23:45:05.022221+0000","last_undegraded":"2024-05-08T23:45:05.022221+0000","last_fullsized":"2024-05-08T23:45:05.022221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:58.763421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'5","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:13.336311+0000","last_change":"2024-05-08T23:45:03.573344+0000","last_active":"2024-05-08T23:45:13.336311+0000","last_peered":"2024-05-08T23:45:13.336311+0000","last_clean":"2024-05-08T23:45:13.336311+0000","last_became_active":"2024-05-08T23:45:03.572951+0000","last_became_peered":"2024-05-08T23:45:03.572951+0000","last_unstale":"2024-05-08T23:45:13.336311+0000","last_undegraded":"2024-05-08T23:45:13.336311+0000","last_fullsized":"2024-05-08T23:45:13.336311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.422908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040278+0000","last_change":"2024-05-08T23:45:04.055111+0000","last_active":"2024-05-08T23:45:05.040278+0000","last_peered":"2024-05-08T23:45:05.040278+0000","last_clean":"2024-05-08T23:45:05.040278+0000","last_became_active":"2024-05-08T23:45:04.054665+0000","last_became_peered":"2024-05-08T23:45:04.054665+0000","last_unstale":"2024-05-08T23:45:05.040278+0000","last_undegraded":"2024-05-08T23:45:05.040278+0000","last_fullsized":"2024-05-08T23:45:05.040278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:23.292630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062624+0000","last_change":"2024-05-08T23:45:03.578168+0000","last_active":"2024-05-08T23:45:07.062624+0000","last_peered":"2024-05-08T23:45:07.062624+0000","last_clean":"2024-05-08T23:45:07.062624+0000","last_became_active":"2024-05-08T23:45:03.577833+0000","last_became_peered":"2024-05-08T23:45:03.577833+0000","last_unstale":"2024-05-08T23:45:07.062624+0000","last_undegraded":"2024-05-08T23:45:07.062624+0000","last_fullsized":"2024-05-08T23:45:07.062624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:13:25.743151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282635+0000","last_change":"2024-05-08T23:45:04.049103+0000","last_active":"2024-05-08T23:45:06.282635+0000","last_peered":"2024-05-08T23:45:06.282635+0000","last_clean":"2024-05-08T23:45:06.282635+0000","last_became_active":"2024-05-08T23:45:04.048302+0000","last_became_peered":"2024-05-08T23:45:04.048302+0000","last_unstale":"2024-05-08T23:45:06.282635+0000","last_undegraded":"2024-05-08T23:45:06.282635+0000","last_fullsized":"2024-05-08T23:45:06.282635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:02.404522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061597+0000","last_change":"2024-05-08T23:45:03.577765+0000","last_active":"2024-05-08T23:45:07.061597+0000","last_peered":"2024-05-08T23:45:07.061597+0000","last_clean":"2024-05-08T23:45:07.061597+0000","last_became_active":"2024-05-08T23:45:03.577452+0000","last_became_peered":"2024-05-08T23:45:03.577452+0000","last_unstale":"2024-05-08T23:45:07.061597+0000","last_undegraded":"2024-05-08T23:45:07.061597+0000","last_fullsized":"2024-05-08T23:45:07.061597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:02.107878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517201+0000","last_change":"2024-05-08T23:45:04.061360+0000","last_active":"2024-05-08T23:45:05.517201+0000","last_peered":"2024-05-08T23:45:05.517201+0000","last_clean":"2024-05-08T23:45:05.517201+0000","last_became_active":"2024-05-08T23:45:04.061000+0000","last_became_peered":"2024-05-08T23:45:04.061000+0000","last_unstale":"2024-05-08T23:45:05.517201+0000","last_undegraded":"2024-05-08T23:45:05.517201+0000","last_fullsized":"2024-05-08T23:45:05.517201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:35.757571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061500+0000","last_change":"2024-05-08T23:45:03.578769+0000","last_active":"2024-05-08T23:45:07.061500+0000","last_peered":"2024-05-08T23:45:07.061500+0000","last_clean":"2024-05-08T23:45:07.061500+0000","last_became_active":"2024-05-08T23:45:03.578359+0000","last_became_peered":"2024-05-08T23:45:03.578359+0000","last_unstale":"2024-05-08T23:45:07.061500+0000","last_undegraded":"2024-05-08T23:45:07.061500+0000","last_fullsized":"2024-05-08T23:45:07.061500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:13.213382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040679+0000","last_change":"2024-05-08T23:45:04.049237+0000","last_active":"2024-05-08T23:45:05.040679+0000","last_peered":"2024-05-08T23:45:05.040679+0000","last_clean":"2024-05-08T23:45:05.040679+0000","last_became_active":"2024-05-08T23:45:04.048922+0000","last_became_peered":"2024-05-08T23:45:04.048922+0000","last_unstale":"2024-05-08T23:45:05.040679+0000","last_undegraded":"2024-05-08T23:45:05.040679+0000","last_fullsized":"2024-05-08T23:45:05.040679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:29.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.414817+0000","last_change":"2024-05-08T23:45:03.574990+0000","last_active":"2024-05-08T23:45:06.414817+0000","last_peered":"2024-05-08T23:45:06.414817+0000","last_clean":"2024-05-08T23:45:06.414817+0000","last_became_active":"2024-05-08T23:45:03.574617+0000","last_became_peered":"2024-05-08T23:45:03.574617+0000","last_unstale":"2024-05-08T23:45:06.414817+0000","last_undegraded":"2024-05-08T23:45:06.414817+0000","last_fullsized":"2024-05-08T23:45:06.414817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:51.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039948+0000","last_change":"2024-05-08T23:45:04.054371+0000","last_active":"2024-05-08T23:45:05.039948+0000","last_peered":"2024-05-08T23:45:05.039948+0000","last_clean":"2024-05-08T23:45:05.039948+0000","last_became_active":"2024-05-08T23:45:04.054070+0000","last_became_peered":"2024-05-08T23:45:04.054070+0000","last_unstale":"2024-05-08T23:45:05.039948+0000","last_undegraded":"2024-05-08T23:45:05.039948+0000","last_fullsized":"2024-05-08T23:45:05.039948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:27.926522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062373+0000","last_change":"2024-05-08T23:45:03.579413+0000","last_active":"2024-05-08T23:45:07.062373+0000","last_peered":"2024-05-08T23:45:07.062373+0000","last_clean":"2024-05-08T23:45:07.062373+0000","last_became_active":"2024-05-08T23:45:03.577705+0000","last_became_peered":"2024-05-08T23:45:03.577705+0000","last_unstale":"2024-05-08T23:45:07.062373+0000","last_undegraded":"2024-05-08T23:45:07.062373+0000","last_fullsized":"2024-05-08T23:45:07.062373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:29.615247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781372+0000","last_change":"2024-05-08T23:45:04.050492+0000","last_active":"2024-05-08T23:45:06.781372+0000","last_peered":"2024-05-08T23:45:06.781372+0000","last_clean":"2024-05-08T23:45:06.781372+0000","last_became_active":"2024-05-08T23:45:04.050176+0000","last_became_peered":"2024-05-08T23:45:04.050176+0000","last_unstale":"2024-05-08T23:45:06.781372+0000","last_undegraded":"2024-05-08T23:45:06.781372+0000","last_fullsized":"2024-05-08T23:45:06.781372+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:01.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.073575+0000","last_change":"2024-05-08T23:45:03.577651+0000","last_active":"2024-05-08T23:45:08.073575+0000","last_peered":"2024-05-08T23:45:08.073575+0000","last_clean":"2024-05-08T23:45:08.073575+0000","last_became_active":"2024-05-08T23:45:03.576887+0000","last_became_peered":"2024-05-08T23:45:03.576887+0000","last_unstale":"2024-05-08T23:45:08.073575+0000","last_undegraded":"2024-05-08T23:45:08.073575+0000","last_fullsized":"2024-05-08T23:45:08.073575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:52.822959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781487+0000","last_change":"2024-05-08T23:45:04.049793+0000","last_active":"2024-05-08T23:45:06.781487+0000","last_peered":"2024-05-08T23:45:06.781487+0000","last_clean":"2024-05-08T23:45:06.781487+0000","last_became_active":"2024-05-08T23:45:04.049456+0000","last_became_peered":"2024-05-08T23:45:04.049456+0000","last_unstale":"2024-05-08T23:45:06.781487+0000","last_undegraded":"2024-05-08T23:45:06.781487+0000","last_fullsized":"2024-05-08T23:45:06.781487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:16.030406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282639+0000","last_change":"2024-05-08T23:45:03.580371+0000","last_active":"2024-05-08T23:45:06.282639+0000","last_peered":"2024-05-08T23:45:06.282639+0000","last_clean":"2024-05-08T23:45:06.282639+0000","last_became_active":"2024-05-08T23:45:03.579394+0000","last_became_peered":"2024-05-08T23:45:03.579394+0000","last_unstale":"2024-05-08T23:45:06.282639+0000","last_undegraded":"2024-05-08T23:45:06.282639+0000","last_fullsized":"2024-05-08T23:45:06.282639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.123632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.879767+0000","last_change":"2024-05-08T23:45:04.063138+0000","last_active":"2024-05-08T23:45:06.879767+0000","last_peered":"2024-05-08T23:45:06.879767+0000","last_clean":"2024-05-08T23:45:06.879767+0000","last_became_active":"2024-05-08T23:45:04.062835+0000","last_became_peered":"2024-05-08T23:45:04.062835+0000","last_unstale":"2024-05-08T23:45:06.879767+0000","last_undegraded":"2024-05-08T23:45:06.879767+0000","last_fullsized":"2024-05-08T23:45:06.879767+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:49.538896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022233+0000","last_change":"2024-05-08T23:45:03.584351+0000","last_active":"2024-05-08T23:45:05.022233+0000","last_peered":"2024-05-08T23:45:05.022233+0000","last_clean":"2024-05-08T23:45:05.022233+0000","last_became_active":"2024-05-08T23:45:03.583996+0000","last_became_peered":"2024-05-08T23:45:03.583996+0000","last_unstale":"2024-05-08T23:45:05.022233+0000","last_undegraded":"2024-05-08T23:45:05.022233+0000","last_fullsized":"2024-05-08T23:45:05.022233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:40.530921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781432+0000","last_change":"2024-05-08T23:45:04.048918+0000","last_active":"2024-05-08T23:45:06.781432+0000","last_peered":"2024-05-08T23:45:06.781432+0000","last_clean":"2024-05-08T23:45:06.781432+0000","last_became_active":"2024-05-08T23:45:04.048550+0000","last_became_peered":"2024-05-08T23:45:04.048550+0000","last_unstale":"2024-05-08T23:45:06.781432+0000","last_undegraded":"2024-05-08T23:45:06.781432+0000","last_fullsized":"2024-05-08T23:45:06.781432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:48.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040646+0000","last_change":"2024-05-08T23:45:04.050790+0000","last_active":"2024-05-08T23:45:05.040646+0000","last_peered":"2024-05-08T23:45:05.040646+0000","last_clean":"2024-05-08T23:45:05.040646+0000","last_became_active":"2024-05-08T23:45:04.050429+0000","last_became_peered":"2024-05-08T23:45:04.050429+0000","last_unstale":"2024-05-08T23:45:05.040646+0000","last_undegraded":"2024-05-08T23:45:05.040646+0000","last_fullsized":"2024-05-08T23:45:05.040646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:43.224351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225201+0000","last_change":"2024-05-08T23:45:04.062063+0000","last_active":"2024-05-08T23:45:06.225201+0000","last_peered":"2024-05-08T23:45:06.225201+0000","last_clean":"2024-05-08T23:45:06.225201+0000","last_became_active":"2024-05-08T23:45:04.061206+0000","last_became_peered":"2024-05-08T23:45:04.061206+0000","last_unstale":"2024-05-08T23:45:06.225201+0000","last_undegraded":"2024-05-08T23:45:06.225201+0000","last_fullsized":"2024-05-08T23:45:06.225201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:23.137397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225238+0000","last_change":"2024-05-08T23:45:04.049707+0000","last_active":"2024-05-08T23:45:06.225238+0000","last_peered":"2024-05-08T23:45:06.225238+0000","last_clean":"2024-05-08T23:45:06.225238+0000","last_became_active":"2024-05-08T23:45:04.049385+0000","last_became_peered":"2024-05-08T23:45:04.049385+0000","last_unstale":"2024-05-08T23:45:06.225238+0000","last_undegraded":"2024-05-08T23:45:06.225238+0000","last_fullsized":"2024-05-08T23:45:06.225238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282351+0000","last_change":"2024-05-08T23:45:04.050721+0000","last_active":"2024-05-08T23:45:06.282351+0000","last_peered":"2024-05-08T23:45:06.282351+0000","last_clean":"2024-05-08T23:45:06.282351+0000","last_became_active":"2024-05-08T23:45:04.050368+0000","last_became_peered":"2024-05-08T23:45:04.050368+0000","last_unstale":"2024-05-08T23:45:06.282351+0000","last_undegraded":"2024-05-08T23:45:06.282351+0000","last_fullsized":"2024-05-08T23:45:06.282351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:20.815754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022211+0000","last_change":"2024-05-08T23:45:04.048431+0000","last_active":"2024-05-08T23:45:05.022211+0000","last_peered":"2024-05-08T23:45:05.022211+0000","last_clean":"2024-05-08T23:45:05.022211+0000","last_became_active":"2024-05-08T23:45:04.048071+0000","last_became_peered":"2024-05-08T23:45:04.048071+0000","last_unstale":"2024-05-08T23:45:05.022211+0000","last_undegraded":"2024-05-08T23:45:05.022211+0000","last_fullsized":"2024-05-08T23:45:05.022211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:51.547256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040715+0000","last_change":"2024-05-08T23:45:04.045256+0000","last_active":"2024-05-08T23:45:05.040715+0000","last_peered":"2024-05-08T23:45:05.040715+0000","last_clean":"2024-05-08T23:45:05.040715+0000","last_became_active":"2024-05-08T23:45:04.044973+0000","last_became_peered":"2024-05-08T23:45:04.044973+0000","last_unstale":"2024-05-08T23:45:05.040715+0000","last_undegraded":"2024-05-08T23:45:05.040715+0000","last_fullsized":"2024-05-08T23:45:05.040715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:06.021001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225634+0000","last_change":"2024-05-08T23:45:04.058287+0000","last_active":"2024-05-08T23:45:06.225634+0000","last_peered":"2024-05-08T23:45:06.225634+0000","last_clean":"2024-05-08T23:45:06.225634+0000","last_became_active":"2024-05-08T23:45:04.057977+0000","last_became_peered":"2024-05-08T23:45:04.057977+0000","last_unstale":"2024-05-08T23:45:06.225634+0000","last_undegraded":"2024-05-08T23:45:06.225634+0000","last_fullsized":"2024-05-08T23:45:06.225634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:01.146838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040216+0000","last_change":"2024-05-08T23:45:04.055780+0000","last_active":"2024-05-08T23:45:05.040216+0000","last_peered":"2024-05-08T23:45:05.040216+0000","last_clean":"2024-05-08T23:45:05.040216+0000","last_became_active":"2024-05-08T23:45:04.055272+0000","last_became_peered":"2024-05-08T23:45:04.055272+0000","last_unstale":"2024-05-08T23:45:05.040216+0000","last_undegraded":"2024-05-08T23:45:05.040216+0000","last_fullsized":"2024-05-08T23:45:05.040216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:28.587333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563811+0000","last_change":"2024-05-08T23:45:04.050648+0000","last_active":"2024-05-08T23:45:05.563811+0000","last_peered":"2024-05-08T23:45:05.563811+0000","last_clean":"2024-05-08T23:45:05.563811+0000","last_became_active":"2024-05-08T23:45:04.050353+0000","last_became_peered":"2024-05-08T23:45:04.050353+0000","last_unstale":"2024-05-08T23:45:05.563811+0000","last_undegraded":"2024-05-08T23:45:05.563811+0000","last_fullsized":"2024-05-08T23:45:05.563811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:58.154006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039875+0000","last_change":"2024-05-08T23:45:04.056951+0000","last_active":"2024-05-08T23:45:05.039875+0000","last_peered":"2024-05-08T23:45:05.039875+0000","last_clean":"2024-05-08T23:45:05.039875+0000","last_became_active":"2024-05-08T23:45:04.056585+0000","last_became_peered":"2024-05-08T23:45:04.056585+0000","last_unstale":"2024-05-08T23:45:05.039875+0000","last_undegraded":"2024-05-08T23:45:05.039875+0000","last_fullsized":"2024-05-08T23:45:05.039875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:22.658629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300447+0000","last_change":"2024-05-08T23:45:04.052396+0000","last_active":"2024-05-08T23:45:04.300447+0000","last_peered":"2024-05-08T23:45:04.300447+0000","last_clean":"2024-05-08T23:45:04.300447+0000","last_became_active":"2024-05-08T23:45:04.051604+0000","last_became_peered":"2024-05-08T23:45:04.051604+0000","last_unstale":"2024-05-08T23:45:04.300447+0000","last_undegraded":"2024-05-08T23:45:04.300447+0000","last_fullsized":"2024-05-08T23:45:04.300447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:52.602830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022201+0000","last_change":"2024-05-08T23:45:04.051578+0000","last_active":"2024-05-08T23:45:05.022201+0000","last_peered":"2024-05-08T23:45:05.022201+0000","last_clean":"2024-05-08T23:45:05.022201+0000","last_became_active":"2024-05-08T23:45:04.051298+0000","last_became_peered":"2024-05-08T23:45:04.051298+0000","last_unstale":"2024-05-08T23:45:05.022201+0000","last_undegraded":"2024-05-08T23:45:05.022201+0000","last_fullsized":"2024-05-08T23:45:05.022201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:47.204849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040483+0000","last_change":"2024-05-08T23:45:04.051264+0000","last_active":"2024-05-08T23:45:05.040483+0000","last_peered":"2024-05-08T23:45:05.040483+0000","last_clean":"2024-05-08T23:45:05.040483+0000","last_became_active":"2024-05-08T23:45:04.050870+0000","last_became_peered":"2024-05-08T23:45:04.050870+0000","last_unstale":"2024-05-08T23:45:05.040483+0000","last_undegraded":"2024-05-08T23:45:05.040483+0000","last_fullsized":"2024-05-08T23:45:05.040483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62933,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":238,"num_read_kb":185,"num_write":148,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514515,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":876544,"data_stored":421419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56299999999999994}]},{"osd":5,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.82199999999999995}]},{"osd":7,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.66900000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":9,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.80900000000000005}]},{"osd":10,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.753}]}]},{"osd":10,"up_from":69,"seq":296352743446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1071904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66400000000000003}]},{"osd":2,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.628},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68300000000000005}]},{"osd":3,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.60999999999999999}]},{"osd":4,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.75}]},{"osd":6,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.77400000000000002}]},{"osd":9,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.73699999999999999}]},{"osd":11,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.82899999999999996}]}]},{"osd":9,"up_from":64,"seq":274877906970,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":421655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53700000000000003}]},{"osd":1,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.86799999999999999}]},{"osd":2,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.78500000000000003}]},{"osd":3,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47899999999999998}]},{"osd":4,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.84999999999999998}]},{"osd":5,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.88900000000000001}]},{"osd":6,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.95699999999999996}]},{"osd":8,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.71599999999999997}]},{"osd":10,"last update":"Wed May 8 23:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81899999999999995}]}]},{"osd":8,"up_from":56,"seq":240518168606,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":888832,"data_stored":421244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.63100000000000001,"15min":0.59599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.2489999999999997,"5min":6.2489999999999997,"15min":6.2489999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.3869999999999996,"5min":6.3869999999999996,"15min":6.3869999999999996},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.64600000000000002}]},{"osd":2,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.45200000000000001,"15min":0.44500000000000001},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.65300000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38600000000000001}]},{"osd":3,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.593,"5min":6.593,"15min":6.593},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.63900000000000001,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.5039999999999996,"5min":6.5039999999999996,"15min":6.5039999999999996},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.626,"15min":0.622},"min":{"1min":0.45700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53900000000000003}]},{"osd":5,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72399999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.46700000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.52200000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.68100000000000005}]},{"osd":7,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93700000000000006,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.751},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63500000000000001}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68899999999999995}]},{"osd":10,"last update":"Wed May 8 23:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]},{"osd":7,"up_from":49,"seq":210453397537,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1482752,"data_stored":1012687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.30099999999999999}]},{"osd":2,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86299999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.76700000000000002}]},{"osd":3,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86299999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.66800000000000004}]},{"osd":4,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81100000000000005,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.36099999999999999}]},{"osd":5,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84499999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.71499999999999997}]},{"osd":6,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93400000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58899999999999997}]},{"osd":8,"last update":"Wed May 8 23:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.89600000000000002,"5min":1.016,"15min":1.016},"last":0.70299999999999996}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.627}]},{"osd":10,"last update":"Wed May 8 23:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.45500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":6,"up_from":44,"seq":188978561060,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":421044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.46500000000000002,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.378}]},{"osd":1,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.498}]},{"osd":2,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.64800000000000002,"15min":0.66200000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79400000000000004,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.56299999999999994,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92400000000000004,"5min":1.288,"15min":1.288},"last":0.63}]},{"osd":3,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.44800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89000000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56200000000000006}]},{"osd":4,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":0.97799999999999998,"5min":1.113,"15min":1.113},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.496,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67500000000000004}]},{"osd":7,"last update":"Wed May 8 23:45:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72099999999999997,"15min":0.72799999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94599999999999995,"5min":1.321,"15min":1.321},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80200000000000005,"15min":0.79700000000000004},"min":{"1min":0.53700000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.0580000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.52000000000000002}]},{"osd":8,"last update":"Wed May 8 23:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.111,"15min":1.111},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.02,"5min":1.05,"15min":1.05},"last":0.66100000000000003}]},{"osd":9,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.57499999999999996}]},{"osd":10,"last update":"Wed May 8 23:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36120,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709032,"statfs":{"total":95995035648,"available":95958048768,"internally_reserved":0,"allocated":933888,"data_stored":479616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.55600000000000005,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.96,"5min":3.96,"15min":3.96},"last":0.75},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.58099999999999996,"15min":0.51900000000000002},"min":{"1min":0.44400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53400000000000003}]},{"osd":3,"last update":"Wed May 8 23:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79000000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed May 8 23:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41799999999999998,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.84399999999999997,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.40699999999999997},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68400000000000005,"5min":0.88,"15min":0.88},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.41599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68000000000000005}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88400000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.88400000000000001}]},{"osd":7,"last update":"Wed May 8 23:45:20 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77000000000000002}]},{"osd":8,"last update":"Wed May 8 23:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.93500000000000005}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.83699999999999997}]},{"osd":10,"last update":"Wed May 8 23:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":897024,"data_stored":421948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.54800000000000004,"15min":0.50600000000000001},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.36299999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":1.032,"15min":1.032},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.51800000000000002},"min":{"1min":0.40400000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.47099999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.86599999999999999}]},{"osd":3,"last update":"Wed May 8 23:44:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.41399999999999998,"15min":0.39400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.498},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.42099999999999999,"15min":0.40600000000000003},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed May 8 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59099999999999997,"15min":0.56200000000000006},"min":{"1min":0.51100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.65500000000000003,"15min":0.621},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.92000000000000004}]},{"osd":5,"last update":"Wed May 8 23:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.52700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.755}]},{"osd":6,"last update":"Wed May 8 23:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:44:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80400000000000005,"15min":0.80100000000000005},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.68600000000000005}]},{"osd":9,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.95299999999999996}]},{"osd":10,"last update":"Wed May 8 23:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.93799999999999994}]},{"osd":11,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.89600000000000002}]}]},{"osd":2,"up_from":17,"seq":73014444084,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":958464,"data_stored":480918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53700000000000003,"15min":0.52000000000000002},"min":{"1min":0.42999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53500000000000003,"15min":0.51300000000000001},"min":{"1min":0.41999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.501,"15min":0.48099999999999998},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.49199999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.627}]},{"osd":3,"last update":"Wed May 8 23:45:11 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54700000000000004,"15min":0.52400000000000002},"min":{"1min":0.41299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53600000000000003,"15min":0.52100000000000002},"min":{"1min":0.42799999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72499999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed May 8 23:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.755},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.78900000000000003}]},{"osd":5,"last update":"Wed May 8 23:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.45500000000000002},"min":{"1min":0.34499999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.53200000000000003,"15min":0.51100000000000001},"min":{"1min":0.45400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.46200000000000002}]},{"osd":6,"last update":"Wed May 8 23:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.5,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.42199999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.53400000000000003}]},{"osd":9,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76600000000000001}]},{"osd":10,"last update":"Wed May 8 23:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.8879999999999999,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.8600000000000003,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.67000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":3,"up_from":24,"seq":103079215152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36084,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709068,"statfs":{"total":95995035648,"available":95958085632,"internally_reserved":0,"allocated":897024,"data_stored":421355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.39000000000000001,"15min":0.375},"min":{"1min":0.35699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57999999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40699999999999997,"15min":0.38900000000000001},"min":{"1min":0.33300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.60499999999999998}]},{"osd":1,"last update":"Wed May 8 23:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54100000000000004,"15min":0.52000000000000002},"min":{"1min":0.47299999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.503},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.72099999999999997}]},{"osd":2,"last update":"Wed May 8 23:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.52400000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78000000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83099999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed May 8 23:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.60799999999999998,"15min":0.56999999999999995},"min":{"1min":0.51100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed May 8 23:44:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.47699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.91600000000000004}]},{"osd":7,"last update":"Wed May 8 23:45:21 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.68600000000000005,"15min":0.65400000000000003},"min":{"1min":0.57399999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":3.5129999999999999,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.72599999999999998,"15min":0.71099999999999997},"min":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.97199999999999998}]},{"osd":8,"last update":"Wed May 8 23:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed May 8 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.84899999999999998}]},{"osd":10,"last update":"Wed May 8 23:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.4179999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":876544,"data_stored":420793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61799999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.67700000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.45300000000000001}]},{"osd":2,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.40699999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81999999999999995,"5min":1.222,"15min":1.222},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61599999999999999}]},{"osd":3,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84899999999999998,"5min":0.874,"15min":0.874},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed May 8 23:44:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89700000000000002,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.91900000000000004,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Wed May 8 23:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.50900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63700000000000001}]},{"osd":7,"last update":"Wed May 8 23:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55200000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51000000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85499999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91100000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76700000000000002}]},{"osd":9,"last update":"Wed May 8 23:44:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.85599999999999998}]},{"osd":10,"last update":"Wed May 8 23:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822697,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32576,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712576,"statfs":{"total":95995035648,"available":95961677824,"internally_reserved":0,"allocated":1499136,"data_stored":1012750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.36399999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51300000000000001,"15min":0.505},"min":{"1min":0.38600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.502},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.621},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.86199999999999999,"5min":0.873,"15min":0.873},"last":0.52400000000000002}]},{"osd":2,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.317}]},{"osd":3,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.49299999999999999,"15min":0.47899999999999998},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.434,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.433},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.40999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59299999999999997}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed May 8 23:45:18 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55700000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85399999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46600000000000003}]},{"osd":9,"last update":"Wed May 8 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69899999999999995}]},{"osd":10,"last update":"Wed May 8 23:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:45:47.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T23:45:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:48.651 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:48 smithi012 ceph-mon[42503]: pgmap v247: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:45:48.651 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:48 smithi012 ceph-mon[42503]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:48.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:48 smithi204 ceph-mon[47373]: pgmap v247: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:45:48.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:48 smithi204 ceph-mon[47373]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:48 smithi160 ceph-mon[46152]: pgmap v247: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:45:48.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:48 smithi160 ceph-mon[46152]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:49.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.268+0000 7fcf5d452700 1 -- 172.21.15.12:0/1206174973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf581000e0 msgr2=0x7fcf581004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:49.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.268+0000 7fcf5d452700 1 --2- 172.21.15.12:0/1206174973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf581000e0 0x7fcf581004c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009990 tx=0x7fcf48009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:49.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.269+0000 7fcf5d452700 1 -- 172.21.15.12:0/1206174973 shutdown_connections 2024-05-08T23:45:49.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.269+0000 7fcf5d452700 1 --2- 172.21.15.12:0/1206174973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf581051c0 0x7fcf5810cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:49.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.269+0000 7fcf5d452700 1 --2- 172.21.15.12:0/1206174973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf581000e0 0x7fcf581004c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.269+0000 7fcf5d452700 1 --2- 172.21.15.12:0/1206174973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf58100a90 0x7fcf58104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.269+0000 7fcf5d452700 1 -- 172.21.15.12:0/1206174973 >> 172.21.15.12:0/1206174973 conn(0x7fcf58076f40 msgr2=0x7fcf58077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.269+0000 7fcf5d452700 1 -- 172.21.15.12:0/1206174973 shutdown_connections 2024-05-08T23:45:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.269+0000 7fcf5d452700 1 -- 172.21.15.12:0/1206174973 wait complete. 2024-05-08T23:45:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.269+0000 7fcf5d452700 1 Processor -- start 2024-05-08T23:45:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf5d452700 1 -- start start 2024-05-08T23:45:49.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf5d452700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf581000e0 0x7fcf581982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf5d452700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf58100a90 0x7fcf581987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf5d452700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf581051c0 0x7fcf581a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf5d452700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf581990a0 con 0x7fcf58100a90 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf5d452700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf581a3860 con 0x7fcf581000e0 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf5d452700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf581a39a0 con 0x7fcf581051c0 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf4e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf58100a90 0x7fcf581987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf4e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf58100a90 0x7fcf581987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55506/0 (socket says 172.21.15.12:55506) 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf4e7fc700 1 -- 172.21.15.12:0/2784862392 learned_addr learned my addr 172.21.15.12:0/2784862392 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf577fe700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf581051c0 0x7fcf581a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf56ffd700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf581000e0 0x7fcf581982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf4e7fc700 1 -- 172.21.15.12:0/2784862392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf581051c0 msgr2=0x7fcf581a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf4e7fc700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf581051c0 0x7fcf581a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf4e7fc700 1 -- 172.21.15.12:0/2784862392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf581000e0 msgr2=0x7fcf581982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf4e7fc700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf581000e0 0x7fcf581982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf4e7fc700 1 -- 172.21.15.12:0/2784862392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf480095f0 con 0x7fcf58100a90 2024-05-08T23:45:49.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf56ffd700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf581000e0 0x7fcf581982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:49.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.270+0000 7fcf4e7fc700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf58100a90 0x7fcf581987e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000d950 tx=0x7fcf4000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:49.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.271+0000 7fcf54ff9700 1 -- 172.21.15.12:0/2784862392 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40004a30 con 0x7fcf58100a90 2024-05-08T23:45:49.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.271+0000 7fcf54ff9700 1 -- 172.21.15.12:0/2784862392 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf400054e0 con 0x7fcf58100a90 2024-05-08T23:45:49.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.271+0000 7fcf54ff9700 1 -- 172.21.15.12:0/2784862392 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40009bf0 con 0x7fcf58100a90 2024-05-08T23:45:49.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.271+0000 7fcf5d452700 1 -- 172.21.15.12:0/2784862392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf581a3c20 con 0x7fcf58100a90 2024-05-08T23:45:49.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.271+0000 7fcf5d452700 1 -- 172.21.15.12:0/2784862392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf581a40d0 con 0x7fcf58100a90 2024-05-08T23:45:49.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.272+0000 7fcf5d452700 1 -- 172.21.15.12:0/2784862392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf58108480 con 0x7fcf58100a90 2024-05-08T23:45:49.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.273+0000 7fcf54ff9700 1 -- 172.21.15.12:0/2784862392 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcf40009d50 con 0x7fcf58100a90 2024-05-08T23:45:49.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.274+0000 7fcf54ff9700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf440a1230 0x7fcf440a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:49.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.274+0000 7fcf56ffd700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf440a1230 0x7fcf440a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:49.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.274+0000 7fcf54ff9700 1 -- 172.21.15.12:0/2784862392 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcf400cc5a0 con 0x7fcf58100a90 2024-05-08T23:45:49.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.274+0000 7fcf56ffd700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf440a1230 0x7fcf440a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800b5c0 tx=0x7fcf4801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:49.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.278+0000 7fcf54ff9700 1 -- 172.21.15.12:0/2784862392 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcf40095d00 con 0x7fcf58100a90 2024-05-08T23:45:49.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.486+0000 7fcf5d452700 1 -- 172.21.15.12:0/2784862392 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf58069cb0 con 0x7fcf440a1230 2024-05-08T23:45:49.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.497+0000 7fcf54ff9700 1 -- 172.21.15.12:0/2784862392 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341448 (secure 0 0 0) 0x7fcf58069cb0 con 0x7fcf440a1230 2024-05-08T23:45:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:49.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 -- 172.21.15.12:0/2784862392 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf440a1230 msgr2=0x7fcf440a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:49.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf440a1230 0x7fcf440a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800b5c0 tx=0x7fcf4801a040 comp rx=0 tx=0).stop 2024-05-08T23:45:49.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 -- 172.21.15.12:0/2784862392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf58100a90 msgr2=0x7fcf581987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:49.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf58100a90 0x7fcf581987e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000d950 tx=0x7fcf4000dc60 comp rx=0 tx=0).stop 2024-05-08T23:45:49.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 -- 172.21.15.12:0/2784862392 shutdown_connections 2024-05-08T23:45:49.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf581051c0 0x7fcf581a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:49.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf440a1230 0x7fcf440a36f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:49.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf581000e0 0x7fcf581982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:49.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 --2- 172.21.15.12:0/2784862392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf58100a90 0x7fcf581987e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:49.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 -- 172.21.15.12:0/2784862392 >> 172.21.15.12:0/2784862392 conn(0x7fcf58076f40 msgr2=0x7fcf58106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:49.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 -- 172.21.15.12:0/2784862392 shutdown_connections 2024-05-08T23:45:49.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:49.501+0000 7fcf5d452700 1 -- 172.21.15.12:0/2784862392 wait complete. 2024-05-08T23:45:49.500 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-08T23:45:49.897 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-05-08T23:45:49.263936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653535,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":394,"num_read_kb":314,"num_write":331,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373420,"kb_used_data":12356,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124568404,"statfs":{"total":1151940427776,"available":1151558045696,"internally_reserved":0,"allocated":12652544,"data_stored":7007333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003050"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040868+0000","last_change":"2024-05-08T23:45:04.044088+0000","last_active":"2024-05-08T23:45:05.040868+0000","last_peered":"2024-05-08T23:45:05.040868+0000","last_clean":"2024-05-08T23:45:05.040868+0000","last_became_active":"2024-05-08T23:45:04.043535+0000","last_became_peered":"2024-05-08T23:45:04.043535+0000","last_unstale":"2024-05-08T23:45:05.040868+0000","last_undegraded":"2024-05-08T23:45:05.040868+0000","last_fullsized":"2024-05-08T23:45:05.040868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:52.363402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282575+0000","last_change":"2024-05-08T23:45:04.049201+0000","last_active":"2024-05-08T23:45:06.282575+0000","last_peered":"2024-05-08T23:45:06.282575+0000","last_clean":"2024-05-08T23:45:06.282575+0000","last_became_active":"2024-05-08T23:45:04.048743+0000","last_became_peered":"2024-05-08T23:45:04.048743+0000","last_unstale":"2024-05-08T23:45:06.282575+0000","last_undegraded":"2024-05-08T23:45:06.282575+0000","last_fullsized":"2024-05-08T23:45:06.282575+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:26.191549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282032+0000","last_change":"2024-05-08T23:45:04.047356+0000","last_active":"2024-05-08T23:45:06.282032+0000","last_peered":"2024-05-08T23:45:06.282032+0000","last_clean":"2024-05-08T23:45:06.282032+0000","last_became_active":"2024-05-08T23:45:04.046996+0000","last_became_peered":"2024-05-08T23:45:04.046996+0000","last_unstale":"2024-05-08T23:45:06.282032+0000","last_undegraded":"2024-05-08T23:45:06.282032+0000","last_fullsized":"2024-05-08T23:45:06.282032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:58.722522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282798+0000","last_change":"2024-05-08T23:45:04.054754+0000","last_active":"2024-05-08T23:45:06.282798+0000","last_peered":"2024-05-08T23:45:06.282798+0000","last_clean":"2024-05-08T23:45:06.282798+0000","last_became_active":"2024-05-08T23:45:04.054302+0000","last_became_peered":"2024-05-08T23:45:04.054302+0000","last_unstale":"2024-05-08T23:45:06.282798+0000","last_undegraded":"2024-05-08T23:45:06.282798+0000","last_fullsized":"2024-05-08T23:45:06.282798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:28.882982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517234+0000","last_change":"2024-05-08T23:45:04.047189+0000","last_active":"2024-05-08T23:45:05.517234+0000","last_peered":"2024-05-08T23:45:05.517234+0000","last_clean":"2024-05-08T23:45:05.517234+0000","last_became_active":"2024-05-08T23:45:04.046756+0000","last_became_peered":"2024-05-08T23:45:04.046756+0000","last_unstale":"2024-05-08T23:45:05.517234+0000","last_undegraded":"2024-05-08T23:45:05.517234+0000","last_fullsized":"2024-05-08T23:45:05.517234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.780169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045028+0000","last_change":"2024-05-08T23:45:04.060897+0000","last_active":"2024-05-08T23:45:05.045028+0000","last_peered":"2024-05-08T23:45:05.045028+0000","last_clean":"2024-05-08T23:45:05.045028+0000","last_became_active":"2024-05-08T23:45:04.060550+0000","last_became_peered":"2024-05-08T23:45:04.060550+0000","last_unstale":"2024-05-08T23:45:05.045028+0000","last_undegraded":"2024-05-08T23:45:05.045028+0000","last_fullsized":"2024-05-08T23:45:05.045028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:59.356057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781043+0000","last_change":"2024-05-08T23:45:04.059308+0000","last_active":"2024-05-08T23:45:06.781043+0000","last_peered":"2024-05-08T23:45:06.781043+0000","last_clean":"2024-05-08T23:45:06.781043+0000","last_became_active":"2024-05-08T23:45:04.058920+0000","last_became_peered":"2024-05-08T23:45:04.058920+0000","last_unstale":"2024-05-08T23:45:06.781043+0000","last_undegraded":"2024-05-08T23:45:06.781043+0000","last_fullsized":"2024-05-08T23:45:06.781043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:11.857062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039706+0000","last_change":"2024-05-08T23:45:04.061542+0000","last_active":"2024-05-08T23:45:05.039706+0000","last_peered":"2024-05-08T23:45:05.039706+0000","last_clean":"2024-05-08T23:45:05.039706+0000","last_became_active":"2024-05-08T23:45:04.061145+0000","last_became_peered":"2024-05-08T23:45:04.061145+0000","last_unstale":"2024-05-08T23:45:05.039706+0000","last_undegraded":"2024-05-08T23:45:05.039706+0000","last_fullsized":"2024-05-08T23:45:05.039706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:53.159292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300493+0000","last_change":"2024-05-08T23:45:04.052292+0000","last_active":"2024-05-08T23:45:04.300493+0000","last_peered":"2024-05-08T23:45:04.300493+0000","last_clean":"2024-05-08T23:45:04.300493+0000","last_became_active":"2024-05-08T23:45:04.051164+0000","last_became_peered":"2024-05-08T23:45:04.051164+0000","last_unstale":"2024-05-08T23:45:04.300493+0000","last_undegraded":"2024-05-08T23:45:04.300493+0000","last_fullsized":"2024-05-08T23:45:04.300493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:09.218130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045014+0000","last_change":"2024-05-08T23:45:04.054566+0000","last_active":"2024-05-08T23:45:05.045014+0000","last_peered":"2024-05-08T23:45:05.045014+0000","last_clean":"2024-05-08T23:45:05.045014+0000","last_became_active":"2024-05-08T23:45:04.053878+0000","last_became_peered":"2024-05-08T23:45:04.053878+0000","last_unstale":"2024-05-08T23:45:05.045014+0000","last_undegraded":"2024-05-08T23:45:05.045014+0000","last_fullsized":"2024-05-08T23:45:05.045014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:34.760038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781480+0000","last_change":"2024-05-08T23:45:04.049696+0000","last_active":"2024-05-08T23:45:06.781480+0000","last_peered":"2024-05-08T23:45:06.781480+0000","last_clean":"2024-05-08T23:45:06.781480+0000","last_became_active":"2024-05-08T23:45:04.049351+0000","last_became_peered":"2024-05-08T23:45:04.049351+0000","last_unstale":"2024-05-08T23:45:06.781480+0000","last_undegraded":"2024-05-08T23:45:06.781480+0000","last_fullsized":"2024-05-08T23:45:06.781480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:53.557738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282779+0000","last_change":"2024-05-08T23:45:04.051010+0000","last_active":"2024-05-08T23:45:06.282779+0000","last_peered":"2024-05-08T23:45:06.282779+0000","last_clean":"2024-05-08T23:45:06.282779+0000","last_became_active":"2024-05-08T23:45:04.050715+0000","last_became_peered":"2024-05-08T23:45:04.050715+0000","last_unstale":"2024-05-08T23:45:06.282779+0000","last_undegraded":"2024-05-08T23:45:06.282779+0000","last_fullsized":"2024-05-08T23:45:06.282779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:09.305974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022192+0000","last_change":"2024-05-08T23:45:04.051005+0000","last_active":"2024-05-08T23:45:05.022192+0000","last_peered":"2024-05-08T23:45:05.022192+0000","last_clean":"2024-05-08T23:45:05.022192+0000","last_became_active":"2024-05-08T23:45:04.050123+0000","last_became_peered":"2024-05-08T23:45:04.050123+0000","last_unstale":"2024-05-08T23:45:05.022192+0000","last_undegraded":"2024-05-08T23:45:05.022192+0000","last_fullsized":"2024-05-08T23:45:05.022192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:52.206389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282033+0000","last_change":"2024-05-08T23:45:04.049942+0000","last_active":"2024-05-08T23:45:06.282033+0000","last_peered":"2024-05-08T23:45:06.282033+0000","last_clean":"2024-05-08T23:45:06.282033+0000","last_became_active":"2024-05-08T23:45:04.049596+0000","last_became_peered":"2024-05-08T23:45:04.049596+0000","last_unstale":"2024-05-08T23:45:06.282033+0000","last_undegraded":"2024-05-08T23:45:06.282033+0000","last_fullsized":"2024-05-08T23:45:06.282033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:30.907023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282203+0000","last_change":"2024-05-08T23:45:04.048985+0000","last_active":"2024-05-08T23:45:06.282203+0000","last_peered":"2024-05-08T23:45:06.282203+0000","last_clean":"2024-05-08T23:45:06.282203+0000","last_became_active":"2024-05-08T23:45:04.048658+0000","last_became_peered":"2024-05-08T23:45:04.048658+0000","last_unstale":"2024-05-08T23:45:06.282203+0000","last_undegraded":"2024-05-08T23:45:06.282203+0000","last_fullsized":"2024-05-08T23:45:06.282203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:17.030930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300415+0000","last_change":"2024-05-08T23:45:04.051903+0000","last_active":"2024-05-08T23:45:04.300415+0000","last_peered":"2024-05-08T23:45:04.300415+0000","last_clean":"2024-05-08T23:45:04.300415+0000","last_became_active":"2024-05-08T23:45:04.051221+0000","last_became_peered":"2024-05-08T23:45:04.051221+0000","last_unstale":"2024-05-08T23:45:04.300415+0000","last_undegraded":"2024-05-08T23:45:04.300415+0000","last_fullsized":"2024-05-08T23:45:04.300415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:01.554463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040970+0000","last_change":"2024-05-08T23:45:04.043854+0000","last_active":"2024-05-08T23:45:05.040970+0000","last_peered":"2024-05-08T23:45:05.040970+0000","last_clean":"2024-05-08T23:45:05.040970+0000","last_became_active":"2024-05-08T23:45:04.043056+0000","last_became_peered":"2024-05-08T23:45:04.043056+0000","last_unstale":"2024-05-08T23:45:05.040970+0000","last_undegraded":"2024-05-08T23:45:05.040970+0000","last_fullsized":"2024-05-08T23:45:05.040970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:18.175575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.058883+0000","last_change":"2024-05-08T23:45:04.049332+0000","last_active":"2024-05-08T23:45:04.058883+0000","last_peered":"2024-05-08T23:45:04.058883+0000","last_clean":"2024-05-08T23:45:04.058883+0000","last_became_active":"2024-05-08T23:45:04.048987+0000","last_became_peered":"2024-05-08T23:45:04.048987+0000","last_unstale":"2024-05-08T23:45:04.058883+0000","last_undegraded":"2024-05-08T23:45:04.058883+0000","last_fullsized":"2024-05-08T23:45:04.058883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:02.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225413+0000","last_change":"2024-05-08T23:45:04.048536+0000","last_active":"2024-05-08T23:45:06.225413+0000","last_peered":"2024-05-08T23:45:06.225413+0000","last_clean":"2024-05-08T23:45:06.225413+0000","last_became_active":"2024-05-08T23:45:04.048102+0000","last_became_peered":"2024-05-08T23:45:04.048102+0000","last_unstale":"2024-05-08T23:45:06.225413+0000","last_undegraded":"2024-05-08T23:45:06.225413+0000","last_fullsized":"2024-05-08T23:45:06.225413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:11.794372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.071241+0000","last_change":"2024-05-08T23:45:03.576833+0000","last_active":"2024-05-08T23:45:08.071241+0000","last_peered":"2024-05-08T23:45:08.071241+0000","last_clean":"2024-05-08T23:45:08.071241+0000","last_became_active":"2024-05-08T23:45:03.576495+0000","last_became_peered":"2024-05-08T23:45:03.576495+0000","last_unstale":"2024-05-08T23:45:08.071241+0000","last_undegraded":"2024-05-08T23:45:08.071241+0000","last_fullsized":"2024-05-08T23:45:08.071241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:50.266658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040105+0000","last_change":"2024-05-08T23:45:04.055444+0000","last_active":"2024-05-08T23:45:05.040105+0000","last_peered":"2024-05-08T23:45:05.040105+0000","last_clean":"2024-05-08T23:45:05.040105+0000","last_became_active":"2024-05-08T23:45:04.054998+0000","last_became_peered":"2024-05-08T23:45:04.054998+0000","last_unstale":"2024-05-08T23:45:05.040105+0000","last_undegraded":"2024-05-08T23:45:05.040105+0000","last_fullsized":"2024-05-08T23:45:05.040105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:03.470555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.100638+0000","last_change":"2024-05-08T23:45:03.578362+0000","last_active":"2024-05-08T23:45:05.100638+0000","last_peered":"2024-05-08T23:45:05.100638+0000","last_clean":"2024-05-08T23:45:05.100638+0000","last_became_active":"2024-05-08T23:45:03.577834+0000","last_became_peered":"2024-05-08T23:45:03.577834+0000","last_unstale":"2024-05-08T23:45:05.100638+0000","last_undegraded":"2024-05-08T23:45:05.100638+0000","last_fullsized":"2024-05-08T23:45:05.100638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:01.943490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282763+0000","last_change":"2024-05-08T23:45:04.047205+0000","last_active":"2024-05-08T23:45:06.282763+0000","last_peered":"2024-05-08T23:45:06.282763+0000","last_clean":"2024-05-08T23:45:06.282763+0000","last_became_active":"2024-05-08T23:45:04.046860+0000","last_became_peered":"2024-05-08T23:45:04.046860+0000","last_unstale":"2024-05-08T23:45:06.282763+0000","last_undegraded":"2024-05-08T23:45:06.282763+0000","last_fullsized":"2024-05-08T23:45:06.282763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:36.837101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.069768+0000","last_change":"2024-05-08T23:45:03.582026+0000","last_active":"2024-05-08T23:45:08.069768+0000","last_peered":"2024-05-08T23:45:08.069768+0000","last_clean":"2024-05-08T23:45:08.069768+0000","last_became_active":"2024-05-08T23:45:03.581672+0000","last_became_peered":"2024-05-08T23:45:03.581672+0000","last_unstale":"2024-05-08T23:45:08.069768+0000","last_undegraded":"2024-05-08T23:45:08.069768+0000","last_fullsized":"2024-05-08T23:45:08.069768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:50.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300470+0000","last_change":"2024-05-08T23:45:04.052639+0000","last_active":"2024-05-08T23:45:04.300470+0000","last_peered":"2024-05-08T23:45:04.300470+0000","last_clean":"2024-05-08T23:45:04.300470+0000","last_became_active":"2024-05-08T23:45:04.052158+0000","last_became_peered":"2024-05-08T23:45:04.052158+0000","last_unstale":"2024-05-08T23:45:04.300470+0000","last_undegraded":"2024-05-08T23:45:04.300470+0000","last_fullsized":"2024-05-08T23:45:04.300470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:25.638555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517264+0000","last_change":"2024-05-08T23:45:03.579221+0000","last_active":"2024-05-08T23:45:05.517264+0000","last_peered":"2024-05-08T23:45:05.517264+0000","last_clean":"2024-05-08T23:45:05.517264+0000","last_became_active":"2024-05-08T23:45:03.578795+0000","last_became_peered":"2024-05-08T23:45:03.578795+0000","last_unstale":"2024-05-08T23:45:05.517264+0000","last_undegraded":"2024-05-08T23:45:05.517264+0000","last_fullsized":"2024-05-08T23:45:05.517264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:14.064108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563925+0000","last_change":"2024-05-08T23:45:04.057573+0000","last_active":"2024-05-08T23:45:05.563925+0000","last_peered":"2024-05-08T23:45:05.563925+0000","last_clean":"2024-05-08T23:45:05.563925+0000","last_became_active":"2024-05-08T23:45:04.053201+0000","last_became_peered":"2024-05-08T23:45:04.053201+0000","last_unstale":"2024-05-08T23:45:05.563925+0000","last_undegraded":"2024-05-08T23:45:05.563925+0000","last_fullsized":"2024-05-08T23:45:05.563925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:23.612609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":56,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:44.129534+0000","last_change":"2024-05-08T23:45:03.578110+0000","last_active":"2024-05-08T23:45:44.129534+0000","last_peered":"2024-05-08T23:45:44.129534+0000","last_clean":"2024-05-08T23:45:44.129534+0000","last_became_active":"2024-05-08T23:45:03.577749+0000","last_became_peered":"2024-05-08T23:45:03.577749+0000","last_unstale":"2024-05-08T23:45:44.129534+0000","last_undegraded":"2024-05-08T23:45:44.129534+0000","last_fullsized":"2024-05-08T23:45:44.129534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:50.526535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517216+0000","last_change":"2024-05-08T23:45:04.048131+0000","last_active":"2024-05-08T23:45:05.517216+0000","last_peered":"2024-05-08T23:45:05.517216+0000","last_clean":"2024-05-08T23:45:05.517216+0000","last_became_active":"2024-05-08T23:45:04.047773+0000","last_became_peered":"2024-05-08T23:45:04.047773+0000","last_unstale":"2024-05-08T23:45:05.517216+0000","last_undegraded":"2024-05-08T23:45:05.517216+0000","last_fullsized":"2024-05-08T23:45:05.517216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:47.944900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.070447+0000","last_change":"2024-05-08T23:45:03.586710+0000","last_active":"2024-05-08T23:45:08.070447+0000","last_peered":"2024-05-08T23:45:08.070447+0000","last_clean":"2024-05-08T23:45:08.070447+0000","last_became_active":"2024-05-08T23:45:03.586411+0000","last_became_peered":"2024-05-08T23:45:03.586411+0000","last_unstale":"2024-05-08T23:45:08.070447+0000","last_undegraded":"2024-05-08T23:45:08.070447+0000","last_fullsized":"2024-05-08T23:45:08.070447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:25.938846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563896+0000","last_change":"2024-05-08T23:45:04.053885+0000","last_active":"2024-05-08T23:45:05.563896+0000","last_peered":"2024-05-08T23:45:05.563896+0000","last_clean":"2024-05-08T23:45:05.563896+0000","last_became_active":"2024-05-08T23:45:04.053385+0000","last_became_peered":"2024-05-08T23:45:04.053385+0000","last_unstale":"2024-05-08T23:45:05.563896+0000","last_undegraded":"2024-05-08T23:45:05.563896+0000","last_fullsized":"2024-05-08T23:45:05.563896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:10.805481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.071417+0000","last_change":"2024-05-08T23:45:03.574988+0000","last_active":"2024-05-08T23:45:08.071417+0000","last_peered":"2024-05-08T23:45:08.071417+0000","last_clean":"2024-05-08T23:45:08.071417+0000","last_became_active":"2024-05-08T23:45:03.574586+0000","last_became_peered":"2024-05-08T23:45:03.574586+0000","last_unstale":"2024-05-08T23:45:08.071417+0000","last_undegraded":"2024-05-08T23:45:08.071417+0000","last_fullsized":"2024-05-08T23:45:08.071417+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:18.005107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039815+0000","last_change":"2024-05-08T23:45:04.046817+0000","last_active":"2024-05-08T23:45:05.039815+0000","last_peered":"2024-05-08T23:45:05.039815+0000","last_clean":"2024-05-08T23:45:05.039815+0000","last_became_active":"2024-05-08T23:45:04.046515+0000","last_became_peered":"2024-05-08T23:45:04.046515+0000","last_unstale":"2024-05-08T23:45:05.039815+0000","last_undegraded":"2024-05-08T23:45:05.039815+0000","last_fullsized":"2024-05-08T23:45:05.039815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:29.439670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282372+0000","last_change":"2024-05-08T23:45:03.579002+0000","last_active":"2024-05-08T23:45:06.282372+0000","last_peered":"2024-05-08T23:45:06.282372+0000","last_clean":"2024-05-08T23:45:06.282372+0000","last_became_active":"2024-05-08T23:45:03.577891+0000","last_became_peered":"2024-05-08T23:45:03.577891+0000","last_unstale":"2024-05-08T23:45:06.282372+0000","last_undegraded":"2024-05-08T23:45:06.282372+0000","last_fullsized":"2024-05-08T23:45:06.282372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:01.027574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.061375+0000","last_change":"2024-05-08T23:45:04.060715+0000","last_active":"2024-05-08T23:45:04.061375+0000","last_peered":"2024-05-08T23:45:04.061375+0000","last_clean":"2024-05-08T23:45:04.061375+0000","last_became_active":"2024-05-08T23:45:04.060375+0000","last_became_peered":"2024-05-08T23:45:04.060375+0000","last_unstale":"2024-05-08T23:45:04.061375+0000","last_undegraded":"2024-05-08T23:45:04.061375+0000","last_fullsized":"2024-05-08T23:45:04.061375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:42.133323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:47.133665+0000","last_change":"2024-05-08T23:45:03.587784+0000","last_active":"2024-05-08T23:45:47.133665+0000","last_peered":"2024-05-08T23:45:47.133665+0000","last_clean":"2024-05-08T23:45:47.133665+0000","last_became_active":"2024-05-08T23:45:03.587529+0000","last_became_peered":"2024-05-08T23:45:03.587529+0000","last_unstale":"2024-05-08T23:45:47.133665+0000","last_undegraded":"2024-05-08T23:45:47.133665+0000","last_fullsized":"2024-05-08T23:45:47.133665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:00.323599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300481+0000","last_change":"2024-05-08T23:45:04.062672+0000","last_active":"2024-05-08T23:45:04.300481+0000","last_peered":"2024-05-08T23:45:04.300481+0000","last_clean":"2024-05-08T23:45:04.300481+0000","last_became_active":"2024-05-08T23:45:04.062351+0000","last_became_peered":"2024-05-08T23:45:04.062351+0000","last_unstale":"2024-05-08T23:45:04.300481+0000","last_undegraded":"2024-05-08T23:45:04.300481+0000","last_fullsized":"2024-05-08T23:45:04.300481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:13.740239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.060380+0000","last_change":"2024-05-08T23:45:03.583501+0000","last_active":"2024-05-08T23:45:07.060380+0000","last_peered":"2024-05-08T23:45:07.060380+0000","last_clean":"2024-05-08T23:45:07.060380+0000","last_became_active":"2024-05-08T23:45:03.583116+0000","last_became_peered":"2024-05-08T23:45:03.583116+0000","last_unstale":"2024-05-08T23:45:07.060380+0000","last_undegraded":"2024-05-08T23:45:07.060380+0000","last_fullsized":"2024-05-08T23:45:07.060380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:03.797524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282765+0000","last_change":"2024-05-08T23:45:04.048336+0000","last_active":"2024-05-08T23:45:06.282765+0000","last_peered":"2024-05-08T23:45:06.282765+0000","last_clean":"2024-05-08T23:45:06.282765+0000","last_became_active":"2024-05-08T23:45:04.048000+0000","last_became_peered":"2024-05-08T23:45:04.048000+0000","last_unstale":"2024-05-08T23:45:06.282765+0000","last_undegraded":"2024-05-08T23:45:06.282765+0000","last_fullsized":"2024-05-08T23:45:06.282765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:41.806600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062204+0000","last_change":"2024-05-08T23:45:03.581215+0000","last_active":"2024-05-08T23:45:07.062204+0000","last_peered":"2024-05-08T23:45:07.062204+0000","last_clean":"2024-05-08T23:45:07.062204+0000","last_became_active":"2024-05-08T23:45:03.580496+0000","last_became_peered":"2024-05-08T23:45:03.580496+0000","last_unstale":"2024-05-08T23:45:07.062204+0000","last_undegraded":"2024-05-08T23:45:07.062204+0000","last_fullsized":"2024-05-08T23:45:07.062204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:25.602401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300459+0000","last_change":"2024-05-08T23:45:04.058857+0000","last_active":"2024-05-08T23:45:04.300459+0000","last_peered":"2024-05-08T23:45:04.300459+0000","last_clean":"2024-05-08T23:45:04.300459+0000","last_became_active":"2024-05-08T23:45:04.057988+0000","last_became_peered":"2024-05-08T23:45:04.057988+0000","last_unstale":"2024-05-08T23:45:04.300459+0000","last_undegraded":"2024-05-08T23:45:04.300459+0000","last_fullsized":"2024-05-08T23:45:04.300459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:46.725976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'27","reported_seq":126,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:48.145167+0000","last_change":"2024-05-08T23:45:03.582121+0000","last_active":"2024-05-08T23:45:48.145167+0000","last_peered":"2024-05-08T23:45:48.145167+0000","last_clean":"2024-05-08T23:45:48.145167+0000","last_became_active":"2024-05-08T23:45:03.581796+0000","last_became_peered":"2024-05-08T23:45:03.581796+0000","last_unstale":"2024-05-08T23:45:48.145167+0000","last_undegraded":"2024-05-08T23:45:48.145167+0000","last_fullsized":"2024-05-08T23:45:48.145167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:55.417404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59419,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":87,"num_read_kb":101,"num_write":28,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517284+0000","last_change":"2024-05-08T23:45:04.052993+0000","last_active":"2024-05-08T23:45:05.517284+0000","last_peered":"2024-05-08T23:45:05.517284+0000","last_clean":"2024-05-08T23:45:05.517284+0000","last_became_active":"2024-05-08T23:45:04.052088+0000","last_became_peered":"2024-05-08T23:45:04.052088+0000","last_unstale":"2024-05-08T23:45:05.517284+0000","last_undegraded":"2024-05-08T23:45:05.517284+0000","last_fullsized":"2024-05-08T23:45:05.517284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:44.531649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.070552+0000","last_change":"2024-05-08T23:45:03.575311+0000","last_active":"2024-05-08T23:45:08.070552+0000","last_peered":"2024-05-08T23:45:08.070552+0000","last_clean":"2024-05-08T23:45:08.070552+0000","last_became_active":"2024-05-08T23:45:03.574949+0000","last_became_peered":"2024-05-08T23:45:03.574949+0000","last_unstale":"2024-05-08T23:45:08.070552+0000","last_undegraded":"2024-05-08T23:45:08.070552+0000","last_fullsized":"2024-05-08T23:45:08.070552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:18.129488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022182+0000","last_change":"2024-05-08T23:45:04.050294+0000","last_active":"2024-05-08T23:45:05.022182+0000","last_peered":"2024-05-08T23:45:05.022182+0000","last_clean":"2024-05-08T23:45:05.022182+0000","last_became_active":"2024-05-08T23:45:04.049949+0000","last_became_peered":"2024-05-08T23:45:04.049949+0000","last_unstale":"2024-05-08T23:45:05.022182+0000","last_undegraded":"2024-05-08T23:45:05.022182+0000","last_fullsized":"2024-05-08T23:45:05.022182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:17.253201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061055+0000","last_change":"2024-05-08T23:45:03.579821+0000","last_active":"2024-05-08T23:45:07.061055+0000","last_peered":"2024-05-08T23:45:07.061055+0000","last_clean":"2024-05-08T23:45:07.061055+0000","last_became_active":"2024-05-08T23:45:03.578635+0000","last_became_peered":"2024-05-08T23:45:03.578635+0000","last_unstale":"2024-05-08T23:45:07.061055+0000","last_undegraded":"2024-05-08T23:45:07.061055+0000","last_fullsized":"2024-05-08T23:45:07.061055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:46.179505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781264+0000","last_change":"2024-05-08T23:45:04.059357+0000","last_active":"2024-05-08T23:45:06.781264+0000","last_peered":"2024-05-08T23:45:06.781264+0000","last_clean":"2024-05-08T23:45:06.781264+0000","last_became_active":"2024-05-08T23:45:04.059070+0000","last_became_peered":"2024-05-08T23:45:04.059070+0000","last_unstale":"2024-05-08T23:45:06.781264+0000","last_undegraded":"2024-05-08T23:45:06.781264+0000","last_fullsized":"2024-05-08T23:45:06.781264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:50.626787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:47.139952+0000","last_change":"2024-05-08T23:45:03.577199+0000","last_active":"2024-05-08T23:45:47.139952+0000","last_peered":"2024-05-08T23:45:47.139952+0000","last_clean":"2024-05-08T23:45:47.139952+0000","last_became_active":"2024-05-08T23:45:03.576844+0000","last_became_peered":"2024-05-08T23:45:03.576844+0000","last_unstale":"2024-05-08T23:45:47.139952+0000","last_undegraded":"2024-05-08T23:45:47.139952+0000","last_fullsized":"2024-05-08T23:45:47.139952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:27.139493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.780964+0000","last_change":"2024-05-08T23:45:04.064515+0000","last_active":"2024-05-08T23:45:06.780964+0000","last_peered":"2024-05-08T23:45:06.780964+0000","last_clean":"2024-05-08T23:45:06.780964+0000","last_became_active":"2024-05-08T23:45:04.064184+0000","last_became_peered":"2024-05-08T23:45:04.064184+0000","last_unstale":"2024-05-08T23:45:06.780964+0000","last_undegraded":"2024-05-08T23:45:06.780964+0000","last_fullsized":"2024-05-08T23:45:06.780964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:46.423121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.072701+0000","last_change":"2024-05-08T23:45:03.576766+0000","last_active":"2024-05-08T23:45:08.072701+0000","last_peered":"2024-05-08T23:45:08.072701+0000","last_clean":"2024-05-08T23:45:08.072701+0000","last_became_active":"2024-05-08T23:45:03.576410+0000","last_became_peered":"2024-05-08T23:45:03.576410+0000","last_unstale":"2024-05-08T23:45:08.072701+0000","last_undegraded":"2024-05-08T23:45:08.072701+0000","last_fullsized":"2024-05-08T23:45:08.072701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:29.444589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563841+0000","last_change":"2024-05-08T23:45:04.056563+0000","last_active":"2024-05-08T23:45:05.563841+0000","last_peered":"2024-05-08T23:45:05.563841+0000","last_clean":"2024-05-08T23:45:05.563841+0000","last_became_active":"2024-05-08T23:45:04.053132+0000","last_became_peered":"2024-05-08T23:45:04.053132+0000","last_unstale":"2024-05-08T23:45:05.563841+0000","last_undegraded":"2024-05-08T23:45:05.563841+0000","last_fullsized":"2024-05-08T23:45:05.563841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:41.625804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:09.074643+0000","last_change":"2024-05-08T23:45:03.580343+0000","last_active":"2024-05-08T23:45:09.074643+0000","last_peered":"2024-05-08T23:45:09.074643+0000","last_clean":"2024-05-08T23:45:09.074643+0000","last_became_active":"2024-05-08T23:45:03.578719+0000","last_became_peered":"2024-05-08T23:45:03.578719+0000","last_unstale":"2024-05-08T23:45:09.074643+0000","last_undegraded":"2024-05-08T23:45:09.074643+0000","last_fullsized":"2024-05-08T23:45:09.074643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:54.831252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040662+0000","last_change":"2024-05-08T23:45:04.050643+0000","last_active":"2024-05-08T23:45:05.040662+0000","last_peered":"2024-05-08T23:45:05.040662+0000","last_clean":"2024-05-08T23:45:05.040662+0000","last_became_active":"2024-05-08T23:45:04.050309+0000","last_became_peered":"2024-05-08T23:45:04.050309+0000","last_unstale":"2024-05-08T23:45:05.040662+0000","last_undegraded":"2024-05-08T23:45:05.040662+0000","last_fullsized":"2024-05-08T23:45:05.040662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:30.259530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.036233+0000","last_change":"2024-05-08T23:45:03.579590+0000","last_active":"2024-05-08T23:45:04.036233+0000","last_peered":"2024-05-08T23:45:04.036233+0000","last_clean":"2024-05-08T23:45:04.036233+0000","last_became_active":"2024-05-08T23:45:03.579213+0000","last_became_peered":"2024-05-08T23:45:03.579213+0000","last_unstale":"2024-05-08T23:45:04.036233+0000","last_undegraded":"2024-05-08T23:45:04.036233+0000","last_fullsized":"2024-05-08T23:45:04.036233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:55.666543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.320406+0000","last_change":"2024-05-08T23:45:04.049082+0000","last_active":"2024-05-08T23:45:08.320406+0000","last_peered":"2024-05-08T23:45:08.320406+0000","last_clean":"2024-05-08T23:45:08.320406+0000","last_became_active":"2024-05-08T23:45:04.048709+0000","last_became_peered":"2024-05-08T23:45:04.048709+0000","last_unstale":"2024-05-08T23:45:08.320406+0000","last_undegraded":"2024-05-08T23:45:08.320406+0000","last_fullsized":"2024-05-08T23:45:08.320406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:56.365817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.879739+0000","last_change":"2024-05-08T23:45:03.581363+0000","last_active":"2024-05-08T23:45:06.879739+0000","last_peered":"2024-05-08T23:45:06.879739+0000","last_clean":"2024-05-08T23:45:06.879739+0000","last_became_active":"2024-05-08T23:45:03.580840+0000","last_became_peered":"2024-05-08T23:45:03.580840+0000","last_unstale":"2024-05-08T23:45:06.879739+0000","last_undegraded":"2024-05-08T23:45:06.879739+0000","last_fullsized":"2024-05-08T23:45:06.879739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:27.281228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.045138+0000","last_change":"2024-05-08T23:45:04.053524+0000","last_active":"2024-05-08T23:45:05.045138+0000","last_peered":"2024-05-08T23:45:05.045138+0000","last_clean":"2024-05-08T23:45:05.045138+0000","last_became_active":"2024-05-08T23:45:04.053118+0000","last_became_peered":"2024-05-08T23:45:04.053118+0000","last_unstale":"2024-05-08T23:45:05.045138+0000","last_undegraded":"2024-05-08T23:45:05.045138+0000","last_fullsized":"2024-05-08T23:45:05.045138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:56.542262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563822+0000","last_change":"2024-05-08T23:45:04.061189+0000","last_active":"2024-05-08T23:45:05.563822+0000","last_peered":"2024-05-08T23:45:05.563822+0000","last_clean":"2024-05-08T23:45:05.563822+0000","last_became_active":"2024-05-08T23:45:04.060838+0000","last_became_peered":"2024-05-08T23:45:04.060838+0000","last_unstale":"2024-05-08T23:45:05.563822+0000","last_undegraded":"2024-05-08T23:45:05.563822+0000","last_fullsized":"2024-05-08T23:45:05.563822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:43.794987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:43.131741+0000","last_change":"2024-05-08T23:45:03.583841+0000","last_active":"2024-05-08T23:45:43.131741+0000","last_peered":"2024-05-08T23:45:43.131741+0000","last_clean":"2024-05-08T23:45:43.131741+0000","last_became_active":"2024-05-08T23:45:03.583275+0000","last_became_peered":"2024-05-08T23:45:03.583275+0000","last_unstale":"2024-05-08T23:45:43.131741+0000","last_undegraded":"2024-05-08T23:45:43.131741+0000","last_fullsized":"2024-05-08T23:45:43.131741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:14.560505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039949+0000","last_change":"2024-05-08T23:45:04.052667+0000","last_active":"2024-05-08T23:45:05.039949+0000","last_peered":"2024-05-08T23:45:05.039949+0000","last_clean":"2024-05-08T23:45:05.039949+0000","last_became_active":"2024-05-08T23:45:04.052008+0000","last_became_peered":"2024-05-08T23:45:04.052008+0000","last_unstale":"2024-05-08T23:45:05.039949+0000","last_undegraded":"2024-05-08T23:45:05.039949+0000","last_fullsized":"2024-05-08T23:45:05.039949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:35.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:09.073198+0000","last_change":"2024-05-08T23:45:03.573665+0000","last_active":"2024-05-08T23:45:09.073198+0000","last_peered":"2024-05-08T23:45:09.073198+0000","last_clean":"2024-05-08T23:45:09.073198+0000","last_became_active":"2024-05-08T23:45:03.573265+0000","last_became_peered":"2024-05-08T23:45:03.573265+0000","last_unstale":"2024-05-08T23:45:09.073198+0000","last_undegraded":"2024-05-08T23:45:09.073198+0000","last_fullsized":"2024-05-08T23:45:09.073198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:13.410954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781044+0000","last_change":"2024-05-08T23:44:16.373962+0000","last_active":"2024-05-08T23:45:06.781044+0000","last_peered":"2024-05-08T23:45:06.781044+0000","last_clean":"2024-05-08T23:45:06.781044+0000","last_became_active":"2024-05-08T23:44:16.373358+0000","last_became_peered":"2024-05-08T23:44:16.373358+0000","last_unstale":"2024-05-08T23:45:06.781044+0000","last_undegraded":"2024-05-08T23:45:06.781044+0000","last_fullsized":"2024-05-08T23:45:06.781044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.067168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:24.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022109+0000","last_change":"2024-05-08T23:45:04.048468+0000","last_active":"2024-05-08T23:45:05.022109+0000","last_peered":"2024-05-08T23:45:05.022109+0000","last_clean":"2024-05-08T23:45:05.022109+0000","last_became_active":"2024-05-08T23:45:04.048085+0000","last_became_peered":"2024-05-08T23:45:04.048085+0000","last_unstale":"2024-05-08T23:45:05.022109+0000","last_undegraded":"2024-05-08T23:45:05.022109+0000","last_fullsized":"2024-05-08T23:45:05.022109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:18.039780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.069513+0000","last_change":"2024-05-08T23:45:03.575821+0000","last_active":"2024-05-08T23:45:08.069513+0000","last_peered":"2024-05-08T23:45:08.069513+0000","last_clean":"2024-05-08T23:45:08.069513+0000","last_became_active":"2024-05-08T23:45:03.575407+0000","last_became_peered":"2024-05-08T23:45:03.575407+0000","last_unstale":"2024-05-08T23:45:08.069513+0000","last_undegraded":"2024-05-08T23:45:08.069513+0000","last_fullsized":"2024-05-08T23:45:08.069513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:35.718942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.060787+0000","last_change":"2024-05-08T23:45:03.580100+0000","last_active":"2024-05-08T23:45:07.060787+0000","last_peered":"2024-05-08T23:45:07.060787+0000","last_clean":"2024-05-08T23:45:07.060787+0000","last_became_active":"2024-05-08T23:45:03.579791+0000","last_became_peered":"2024-05-08T23:45:03.579791+0000","last_unstale":"2024-05-08T23:45:07.060787+0000","last_undegraded":"2024-05-08T23:45:07.060787+0000","last_fullsized":"2024-05-08T23:45:07.060787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:41.086291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022221+0000","last_change":"2024-05-08T23:45:04.047687+0000","last_active":"2024-05-08T23:45:05.022221+0000","last_peered":"2024-05-08T23:45:05.022221+0000","last_clean":"2024-05-08T23:45:05.022221+0000","last_became_active":"2024-05-08T23:45:04.047343+0000","last_became_peered":"2024-05-08T23:45:04.047343+0000","last_unstale":"2024-05-08T23:45:05.022221+0000","last_undegraded":"2024-05-08T23:45:05.022221+0000","last_fullsized":"2024-05-08T23:45:05.022221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:58.763421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'5","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:13.336311+0000","last_change":"2024-05-08T23:45:03.573344+0000","last_active":"2024-05-08T23:45:13.336311+0000","last_peered":"2024-05-08T23:45:13.336311+0000","last_clean":"2024-05-08T23:45:13.336311+0000","last_became_active":"2024-05-08T23:45:03.572951+0000","last_became_peered":"2024-05-08T23:45:03.572951+0000","last_unstale":"2024-05-08T23:45:13.336311+0000","last_undegraded":"2024-05-08T23:45:13.336311+0000","last_fullsized":"2024-05-08T23:45:13.336311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.422908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040278+0000","last_change":"2024-05-08T23:45:04.055111+0000","last_active":"2024-05-08T23:45:05.040278+0000","last_peered":"2024-05-08T23:45:05.040278+0000","last_clean":"2024-05-08T23:45:05.040278+0000","last_became_active":"2024-05-08T23:45:04.054665+0000","last_became_peered":"2024-05-08T23:45:04.054665+0000","last_unstale":"2024-05-08T23:45:05.040278+0000","last_undegraded":"2024-05-08T23:45:05.040278+0000","last_fullsized":"2024-05-08T23:45:05.040278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:23.292630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062624+0000","last_change":"2024-05-08T23:45:03.578168+0000","last_active":"2024-05-08T23:45:07.062624+0000","last_peered":"2024-05-08T23:45:07.062624+0000","last_clean":"2024-05-08T23:45:07.062624+0000","last_became_active":"2024-05-08T23:45:03.577833+0000","last_became_peered":"2024-05-08T23:45:03.577833+0000","last_unstale":"2024-05-08T23:45:07.062624+0000","last_undegraded":"2024-05-08T23:45:07.062624+0000","last_fullsized":"2024-05-08T23:45:07.062624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:13:25.743151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282635+0000","last_change":"2024-05-08T23:45:04.049103+0000","last_active":"2024-05-08T23:45:06.282635+0000","last_peered":"2024-05-08T23:45:06.282635+0000","last_clean":"2024-05-08T23:45:06.282635+0000","last_became_active":"2024-05-08T23:45:04.048302+0000","last_became_peered":"2024-05-08T23:45:04.048302+0000","last_unstale":"2024-05-08T23:45:06.282635+0000","last_undegraded":"2024-05-08T23:45:06.282635+0000","last_fullsized":"2024-05-08T23:45:06.282635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:02.404522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061597+0000","last_change":"2024-05-08T23:45:03.577765+0000","last_active":"2024-05-08T23:45:07.061597+0000","last_peered":"2024-05-08T23:45:07.061597+0000","last_clean":"2024-05-08T23:45:07.061597+0000","last_became_active":"2024-05-08T23:45:03.577452+0000","last_became_peered":"2024-05-08T23:45:03.577452+0000","last_unstale":"2024-05-08T23:45:07.061597+0000","last_undegraded":"2024-05-08T23:45:07.061597+0000","last_fullsized":"2024-05-08T23:45:07.061597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:02.107878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.517201+0000","last_change":"2024-05-08T23:45:04.061360+0000","last_active":"2024-05-08T23:45:05.517201+0000","last_peered":"2024-05-08T23:45:05.517201+0000","last_clean":"2024-05-08T23:45:05.517201+0000","last_became_active":"2024-05-08T23:45:04.061000+0000","last_became_peered":"2024-05-08T23:45:04.061000+0000","last_unstale":"2024-05-08T23:45:05.517201+0000","last_undegraded":"2024-05-08T23:45:05.517201+0000","last_fullsized":"2024-05-08T23:45:05.517201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:35.757571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.061500+0000","last_change":"2024-05-08T23:45:03.578769+0000","last_active":"2024-05-08T23:45:07.061500+0000","last_peered":"2024-05-08T23:45:07.061500+0000","last_clean":"2024-05-08T23:45:07.061500+0000","last_became_active":"2024-05-08T23:45:03.578359+0000","last_became_peered":"2024-05-08T23:45:03.578359+0000","last_unstale":"2024-05-08T23:45:07.061500+0000","last_undegraded":"2024-05-08T23:45:07.061500+0000","last_fullsized":"2024-05-08T23:45:07.061500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:13.213382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040679+0000","last_change":"2024-05-08T23:45:04.049237+0000","last_active":"2024-05-08T23:45:05.040679+0000","last_peered":"2024-05-08T23:45:05.040679+0000","last_clean":"2024-05-08T23:45:05.040679+0000","last_became_active":"2024-05-08T23:45:04.048922+0000","last_became_peered":"2024-05-08T23:45:04.048922+0000","last_unstale":"2024-05-08T23:45:05.040679+0000","last_undegraded":"2024-05-08T23:45:05.040679+0000","last_fullsized":"2024-05-08T23:45:05.040679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:29.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.414817+0000","last_change":"2024-05-08T23:45:03.574990+0000","last_active":"2024-05-08T23:45:06.414817+0000","last_peered":"2024-05-08T23:45:06.414817+0000","last_clean":"2024-05-08T23:45:06.414817+0000","last_became_active":"2024-05-08T23:45:03.574617+0000","last_became_peered":"2024-05-08T23:45:03.574617+0000","last_unstale":"2024-05-08T23:45:06.414817+0000","last_undegraded":"2024-05-08T23:45:06.414817+0000","last_fullsized":"2024-05-08T23:45:06.414817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:51.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039948+0000","last_change":"2024-05-08T23:45:04.054371+0000","last_active":"2024-05-08T23:45:05.039948+0000","last_peered":"2024-05-08T23:45:05.039948+0000","last_clean":"2024-05-08T23:45:05.039948+0000","last_became_active":"2024-05-08T23:45:04.054070+0000","last_became_peered":"2024-05-08T23:45:04.054070+0000","last_unstale":"2024-05-08T23:45:05.039948+0000","last_undegraded":"2024-05-08T23:45:05.039948+0000","last_fullsized":"2024-05-08T23:45:05.039948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:27.926522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:07.062373+0000","last_change":"2024-05-08T23:45:03.579413+0000","last_active":"2024-05-08T23:45:07.062373+0000","last_peered":"2024-05-08T23:45:07.062373+0000","last_clean":"2024-05-08T23:45:07.062373+0000","last_became_active":"2024-05-08T23:45:03.577705+0000","last_became_peered":"2024-05-08T23:45:03.577705+0000","last_unstale":"2024-05-08T23:45:07.062373+0000","last_undegraded":"2024-05-08T23:45:07.062373+0000","last_fullsized":"2024-05-08T23:45:07.062373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:29.615247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781372+0000","last_change":"2024-05-08T23:45:04.050492+0000","last_active":"2024-05-08T23:45:06.781372+0000","last_peered":"2024-05-08T23:45:06.781372+0000","last_clean":"2024-05-08T23:45:06.781372+0000","last_became_active":"2024-05-08T23:45:04.050176+0000","last_became_peered":"2024-05-08T23:45:04.050176+0000","last_unstale":"2024-05-08T23:45:06.781372+0000","last_undegraded":"2024-05-08T23:45:06.781372+0000","last_fullsized":"2024-05-08T23:45:06.781372+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:01.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:08.073575+0000","last_change":"2024-05-08T23:45:03.577651+0000","last_active":"2024-05-08T23:45:08.073575+0000","last_peered":"2024-05-08T23:45:08.073575+0000","last_clean":"2024-05-08T23:45:08.073575+0000","last_became_active":"2024-05-08T23:45:03.576887+0000","last_became_peered":"2024-05-08T23:45:03.576887+0000","last_unstale":"2024-05-08T23:45:08.073575+0000","last_undegraded":"2024-05-08T23:45:08.073575+0000","last_fullsized":"2024-05-08T23:45:08.073575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:52.822959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781487+0000","last_change":"2024-05-08T23:45:04.049793+0000","last_active":"2024-05-08T23:45:06.781487+0000","last_peered":"2024-05-08T23:45:06.781487+0000","last_clean":"2024-05-08T23:45:06.781487+0000","last_became_active":"2024-05-08T23:45:04.049456+0000","last_became_peered":"2024-05-08T23:45:04.049456+0000","last_unstale":"2024-05-08T23:45:06.781487+0000","last_undegraded":"2024-05-08T23:45:06.781487+0000","last_fullsized":"2024-05-08T23:45:06.781487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:16.030406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282639+0000","last_change":"2024-05-08T23:45:03.580371+0000","last_active":"2024-05-08T23:45:06.282639+0000","last_peered":"2024-05-08T23:45:06.282639+0000","last_clean":"2024-05-08T23:45:06.282639+0000","last_became_active":"2024-05-08T23:45:03.579394+0000","last_became_peered":"2024-05-08T23:45:03.579394+0000","last_unstale":"2024-05-08T23:45:06.282639+0000","last_undegraded":"2024-05-08T23:45:06.282639+0000","last_fullsized":"2024-05-08T23:45:06.282639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.123632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.879767+0000","last_change":"2024-05-08T23:45:04.063138+0000","last_active":"2024-05-08T23:45:06.879767+0000","last_peered":"2024-05-08T23:45:06.879767+0000","last_clean":"2024-05-08T23:45:06.879767+0000","last_became_active":"2024-05-08T23:45:04.062835+0000","last_became_peered":"2024-05-08T23:45:04.062835+0000","last_unstale":"2024-05-08T23:45:06.879767+0000","last_undegraded":"2024-05-08T23:45:06.879767+0000","last_fullsized":"2024-05-08T23:45:06.879767+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:49.538896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022233+0000","last_change":"2024-05-08T23:45:03.584351+0000","last_active":"2024-05-08T23:45:05.022233+0000","last_peered":"2024-05-08T23:45:05.022233+0000","last_clean":"2024-05-08T23:45:05.022233+0000","last_became_active":"2024-05-08T23:45:03.583996+0000","last_became_peered":"2024-05-08T23:45:03.583996+0000","last_unstale":"2024-05-08T23:45:05.022233+0000","last_undegraded":"2024-05-08T23:45:05.022233+0000","last_fullsized":"2024-05-08T23:45:05.022233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:40.530921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.781432+0000","last_change":"2024-05-08T23:45:04.048918+0000","last_active":"2024-05-08T23:45:06.781432+0000","last_peered":"2024-05-08T23:45:06.781432+0000","last_clean":"2024-05-08T23:45:06.781432+0000","last_became_active":"2024-05-08T23:45:04.048550+0000","last_became_peered":"2024-05-08T23:45:04.048550+0000","last_unstale":"2024-05-08T23:45:06.781432+0000","last_undegraded":"2024-05-08T23:45:06.781432+0000","last_fullsized":"2024-05-08T23:45:06.781432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:48.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040646+0000","last_change":"2024-05-08T23:45:04.050790+0000","last_active":"2024-05-08T23:45:05.040646+0000","last_peered":"2024-05-08T23:45:05.040646+0000","last_clean":"2024-05-08T23:45:05.040646+0000","last_became_active":"2024-05-08T23:45:04.050429+0000","last_became_peered":"2024-05-08T23:45:04.050429+0000","last_unstale":"2024-05-08T23:45:05.040646+0000","last_undegraded":"2024-05-08T23:45:05.040646+0000","last_fullsized":"2024-05-08T23:45:05.040646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:43.224351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225201+0000","last_change":"2024-05-08T23:45:04.062063+0000","last_active":"2024-05-08T23:45:06.225201+0000","last_peered":"2024-05-08T23:45:06.225201+0000","last_clean":"2024-05-08T23:45:06.225201+0000","last_became_active":"2024-05-08T23:45:04.061206+0000","last_became_peered":"2024-05-08T23:45:04.061206+0000","last_unstale":"2024-05-08T23:45:06.225201+0000","last_undegraded":"2024-05-08T23:45:06.225201+0000","last_fullsized":"2024-05-08T23:45:06.225201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:23.137397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225238+0000","last_change":"2024-05-08T23:45:04.049707+0000","last_active":"2024-05-08T23:45:06.225238+0000","last_peered":"2024-05-08T23:45:06.225238+0000","last_clean":"2024-05-08T23:45:06.225238+0000","last_became_active":"2024-05-08T23:45:04.049385+0000","last_became_peered":"2024-05-08T23:45:04.049385+0000","last_unstale":"2024-05-08T23:45:06.225238+0000","last_undegraded":"2024-05-08T23:45:06.225238+0000","last_fullsized":"2024-05-08T23:45:06.225238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.282351+0000","last_change":"2024-05-08T23:45:04.050721+0000","last_active":"2024-05-08T23:45:06.282351+0000","last_peered":"2024-05-08T23:45:06.282351+0000","last_clean":"2024-05-08T23:45:06.282351+0000","last_became_active":"2024-05-08T23:45:04.050368+0000","last_became_peered":"2024-05-08T23:45:04.050368+0000","last_unstale":"2024-05-08T23:45:06.282351+0000","last_undegraded":"2024-05-08T23:45:06.282351+0000","last_fullsized":"2024-05-08T23:45:06.282351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:20.815754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022211+0000","last_change":"2024-05-08T23:45:04.048431+0000","last_active":"2024-05-08T23:45:05.022211+0000","last_peered":"2024-05-08T23:45:05.022211+0000","last_clean":"2024-05-08T23:45:05.022211+0000","last_became_active":"2024-05-08T23:45:04.048071+0000","last_became_peered":"2024-05-08T23:45:04.048071+0000","last_unstale":"2024-05-08T23:45:05.022211+0000","last_undegraded":"2024-05-08T23:45:05.022211+0000","last_fullsized":"2024-05-08T23:45:05.022211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:51.547256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040715+0000","last_change":"2024-05-08T23:45:04.045256+0000","last_active":"2024-05-08T23:45:05.040715+0000","last_peered":"2024-05-08T23:45:05.040715+0000","last_clean":"2024-05-08T23:45:05.040715+0000","last_became_active":"2024-05-08T23:45:04.044973+0000","last_became_peered":"2024-05-08T23:45:04.044973+0000","last_unstale":"2024-05-08T23:45:05.040715+0000","last_undegraded":"2024-05-08T23:45:05.040715+0000","last_fullsized":"2024-05-08T23:45:05.040715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:06.021001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:06.225634+0000","last_change":"2024-05-08T23:45:04.058287+0000","last_active":"2024-05-08T23:45:06.225634+0000","last_peered":"2024-05-08T23:45:06.225634+0000","last_clean":"2024-05-08T23:45:06.225634+0000","last_became_active":"2024-05-08T23:45:04.057977+0000","last_became_peered":"2024-05-08T23:45:04.057977+0000","last_unstale":"2024-05-08T23:45:06.225634+0000","last_undegraded":"2024-05-08T23:45:06.225634+0000","last_fullsized":"2024-05-08T23:45:06.225634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:01.146838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040216+0000","last_change":"2024-05-08T23:45:04.055780+0000","last_active":"2024-05-08T23:45:05.040216+0000","last_peered":"2024-05-08T23:45:05.040216+0000","last_clean":"2024-05-08T23:45:05.040216+0000","last_became_active":"2024-05-08T23:45:04.055272+0000","last_became_peered":"2024-05-08T23:45:04.055272+0000","last_unstale":"2024-05-08T23:45:05.040216+0000","last_undegraded":"2024-05-08T23:45:05.040216+0000","last_fullsized":"2024-05-08T23:45:05.040216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:28.587333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.563811+0000","last_change":"2024-05-08T23:45:04.050648+0000","last_active":"2024-05-08T23:45:05.563811+0000","last_peered":"2024-05-08T23:45:05.563811+0000","last_clean":"2024-05-08T23:45:05.563811+0000","last_became_active":"2024-05-08T23:45:04.050353+0000","last_became_peered":"2024-05-08T23:45:04.050353+0000","last_unstale":"2024-05-08T23:45:05.563811+0000","last_undegraded":"2024-05-08T23:45:05.563811+0000","last_fullsized":"2024-05-08T23:45:05.563811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:58.154006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.039875+0000","last_change":"2024-05-08T23:45:04.056951+0000","last_active":"2024-05-08T23:45:05.039875+0000","last_peered":"2024-05-08T23:45:05.039875+0000","last_clean":"2024-05-08T23:45:05.039875+0000","last_became_active":"2024-05-08T23:45:04.056585+0000","last_became_peered":"2024-05-08T23:45:04.056585+0000","last_unstale":"2024-05-08T23:45:05.039875+0000","last_undegraded":"2024-05-08T23:45:05.039875+0000","last_fullsized":"2024-05-08T23:45:05.039875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:22.658629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:04.300447+0000","last_change":"2024-05-08T23:45:04.052396+0000","last_active":"2024-05-08T23:45:04.300447+0000","last_peered":"2024-05-08T23:45:04.300447+0000","last_clean":"2024-05-08T23:45:04.300447+0000","last_became_active":"2024-05-08T23:45:04.051604+0000","last_became_peered":"2024-05-08T23:45:04.051604+0000","last_unstale":"2024-05-08T23:45:04.300447+0000","last_undegraded":"2024-05-08T23:45:04.300447+0000","last_fullsized":"2024-05-08T23:45:04.300447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:52.602830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.022201+0000","last_change":"2024-05-08T23:45:04.051578+0000","last_active":"2024-05-08T23:45:05.022201+0000","last_peered":"2024-05-08T23:45:05.022201+0000","last_clean":"2024-05-08T23:45:05.022201+0000","last_became_active":"2024-05-08T23:45:04.051298+0000","last_became_peered":"2024-05-08T23:45:04.051298+0000","last_unstale":"2024-05-08T23:45:05.022201+0000","last_undegraded":"2024-05-08T23:45:05.022201+0000","last_fullsized":"2024-05-08T23:45:05.022201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:47.204849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:45:05.040483+0000","last_change":"2024-05-08T23:45:04.051264+0000","last_active":"2024-05-08T23:45:05.040483+0000","last_peered":"2024-05-08T23:45:05.040483+0000","last_clean":"2024-05-08T23:45:05.040483+0000","last_became_active":"2024-05-08T23:45:04.050870+0000","last_became_peered":"2024-05-08T23:45:04.050870+0000","last_unstale":"2024-05-08T23:45:05.040483+0000","last_undegraded":"2024-05-08T23:45:05.040483+0000","last_fullsized":"2024-05-08T23:45:05.040483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62933,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":258,"num_read_kb":195,"num_write":148,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514515,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":876544,"data_stored":421419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56299999999999994}]},{"osd":5,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.82199999999999995}]},{"osd":7,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.66900000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":9,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.80900000000000005}]},{"osd":10,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.753}]}]},{"osd":10,"up_from":69,"seq":296352743447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1071904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.32600000000000001}]},{"osd":2,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.38}]},{"osd":5,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.624}]},{"osd":6,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.498},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.63500000000000001}]},{"osd":9,"last update":"Wed May 8 23:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.48199999999999998}]},{"osd":11,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.68500000000000005}]}]},{"osd":9,"up_from":64,"seq":274877906971,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":421655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.374}]},{"osd":1,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.57399999999999995}]},{"osd":3,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.44600000000000001}]},{"osd":4,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.626},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.69299999999999995}]},{"osd":6,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.497}]},{"osd":8,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.67400000000000004}]},{"osd":10,"last update":"Wed May 8 23:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65400000000000003}]}]},{"osd":8,"up_from":56,"seq":240518168606,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":888832,"data_stored":421244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.63100000000000001,"15min":0.59599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.2489999999999997,"5min":6.2489999999999997,"15min":6.2489999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.3869999999999996,"5min":6.3869999999999996,"15min":6.3869999999999996},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.64600000000000002}]},{"osd":2,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.45200000000000001,"15min":0.44500000000000001},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.65300000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38600000000000001}]},{"osd":3,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.593,"5min":6.593,"15min":6.593},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.63900000000000001,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.5039999999999996,"5min":6.5039999999999996,"15min":6.5039999999999996},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.626,"15min":0.622},"min":{"1min":0.45700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53900000000000003}]},{"osd":5,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72399999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.46700000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.52200000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.68100000000000005}]},{"osd":7,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93700000000000006,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.751},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63500000000000001}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68899999999999995}]},{"osd":10,"last update":"Wed May 8 23:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]},{"osd":7,"up_from":49,"seq":210453397537,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1482752,"data_stored":1012687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.30099999999999999}]},{"osd":2,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86299999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.76700000000000002}]},{"osd":3,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86299999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.66800000000000004}]},{"osd":4,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81100000000000005,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.36099999999999999}]},{"osd":5,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84499999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.71499999999999997}]},{"osd":6,"last update":"Wed May 8 23:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93400000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58899999999999997}]},{"osd":8,"last update":"Wed May 8 23:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.89600000000000002,"5min":1.016,"15min":1.016},"last":0.70299999999999996}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.627}]},{"osd":10,"last update":"Wed May 8 23:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.45500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":6,"up_from":44,"seq":188978561060,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":421044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.46500000000000002,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.378}]},{"osd":1,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.498}]},{"osd":2,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.64800000000000002,"15min":0.66200000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79400000000000004,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.56299999999999994,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92400000000000004,"5min":1.288,"15min":1.288},"last":0.63}]},{"osd":3,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.44800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89000000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56200000000000006}]},{"osd":4,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":0.97799999999999998,"5min":1.113,"15min":1.113},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.496,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67500000000000004}]},{"osd":7,"last update":"Wed May 8 23:45:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72099999999999997,"15min":0.72799999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94599999999999995,"5min":1.321,"15min":1.321},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80200000000000005,"15min":0.79700000000000004},"min":{"1min":0.53700000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.0580000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.52000000000000002}]},{"osd":8,"last update":"Wed May 8 23:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.111,"15min":1.111},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.02,"5min":1.05,"15min":1.05},"last":0.66100000000000003}]},{"osd":9,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.57499999999999996}]},{"osd":10,"last update":"Wed May 8 23:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36120,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709032,"statfs":{"total":95995035648,"available":95958048768,"internally_reserved":0,"allocated":933888,"data_stored":479616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.55600000000000005,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.96,"5min":3.96,"15min":3.96},"last":0.75},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.58099999999999996,"15min":0.51900000000000002},"min":{"1min":0.44400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53400000000000003}]},{"osd":3,"last update":"Wed May 8 23:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79000000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed May 8 23:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41799999999999998,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.84399999999999997,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.40699999999999997},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68400000000000005,"5min":0.88,"15min":0.88},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.41599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68000000000000005}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88400000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.88400000000000001}]},{"osd":7,"last update":"Wed May 8 23:45:20 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77000000000000002}]},{"osd":8,"last update":"Wed May 8 23:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.93500000000000005}]},{"osd":9,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.83699999999999997}]},{"osd":10,"last update":"Wed May 8 23:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":897024,"data_stored":421948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.54800000000000004,"15min":0.50600000000000001},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.36299999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":1.032,"15min":1.032},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed May 8 23:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.51800000000000002},"min":{"1min":0.40400000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.47099999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.86599999999999999}]},{"osd":3,"last update":"Wed May 8 23:44:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.41399999999999998,"15min":0.39400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.498},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.42099999999999999,"15min":0.40600000000000003},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed May 8 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59099999999999997,"15min":0.56200000000000006},"min":{"1min":0.51100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.65500000000000003,"15min":0.621},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.92000000000000004}]},{"osd":5,"last update":"Wed May 8 23:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.52700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.755}]},{"osd":6,"last update":"Wed May 8 23:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:44:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80400000000000005,"15min":0.80100000000000005},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.68600000000000005}]},{"osd":9,"last update":"Wed May 8 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.95299999999999996}]},{"osd":10,"last update":"Wed May 8 23:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.93799999999999994}]},{"osd":11,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.89600000000000002}]}]},{"osd":2,"up_from":17,"seq":73014444084,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":958464,"data_stored":480918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53700000000000003,"15min":0.52000000000000002},"min":{"1min":0.42999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.53500000000000003,"15min":0.51300000000000001},"min":{"1min":0.41999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.501,"15min":0.48099999999999998},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.49199999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.627}]},{"osd":3,"last update":"Wed May 8 23:45:11 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54700000000000004,"15min":0.52400000000000002},"min":{"1min":0.41299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53600000000000003,"15min":0.52100000000000002},"min":{"1min":0.42799999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72499999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed May 8 23:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.755},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.78900000000000003}]},{"osd":5,"last update":"Wed May 8 23:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.45500000000000002},"min":{"1min":0.34499999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.53200000000000003,"15min":0.51100000000000001},"min":{"1min":0.45400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.46200000000000002}]},{"osd":6,"last update":"Wed May 8 23:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.5,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.42199999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.53400000000000003}]},{"osd":9,"last update":"Wed May 8 23:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76600000000000001}]},{"osd":10,"last update":"Wed May 8 23:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.8879999999999999,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.8600000000000003,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.67000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":3,"up_from":24,"seq":103079215152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36084,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709068,"statfs":{"total":95995035648,"available":95958085632,"internally_reserved":0,"allocated":897024,"data_stored":421355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.39000000000000001,"15min":0.375},"min":{"1min":0.35699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57999999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40699999999999997,"15min":0.38900000000000001},"min":{"1min":0.33300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.60499999999999998}]},{"osd":1,"last update":"Wed May 8 23:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54100000000000004,"15min":0.52000000000000002},"min":{"1min":0.47299999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.503},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.72099999999999997}]},{"osd":2,"last update":"Wed May 8 23:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.52400000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78000000000000003,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83099999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed May 8 23:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.60799999999999998,"15min":0.56999999999999995},"min":{"1min":0.51100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed May 8 23:44:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.47699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.91600000000000004}]},{"osd":7,"last update":"Wed May 8 23:45:21 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.68600000000000005,"15min":0.65400000000000003},"min":{"1min":0.57399999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":3.5129999999999999,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.72599999999999998,"15min":0.71099999999999997},"min":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.97199999999999998}]},{"osd":8,"last update":"Wed May 8 23:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed May 8 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.84899999999999998}]},{"osd":10,"last update":"Wed May 8 23:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.4179999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986219,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":876544,"data_stored":420793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61799999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.67700000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.51600000000000001}]},{"osd":2,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.40699999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81999999999999995,"5min":1.222,"15min":1.222},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84899999999999998,"5min":0.874,"15min":0.874},"last":0.63300000000000001}]},{"osd":5,"last update":"Wed May 8 23:44:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89700000000000002,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.91900000000000004,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Wed May 8 23:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.50900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69399999999999995}]},{"osd":7,"last update":"Wed May 8 23:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.503},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60399999999999998}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51000000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85499999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.747},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91100000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.80300000000000005}]},{"osd":9,"last update":"Wed May 8 23:44:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.83299999999999996}]},{"osd":10,"last update":"Wed May 8 23:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822697,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32576,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712576,"statfs":{"total":95995035648,"available":95961677824,"internally_reserved":0,"allocated":1499136,"data_stored":1012750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.36399999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51300000000000001,"15min":0.505},"min":{"1min":0.38600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.502},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.621},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.86199999999999999,"5min":0.873,"15min":0.873},"last":0.52400000000000002}]},{"osd":2,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.317}]},{"osd":3,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.49299999999999999,"15min":0.47899999999999998},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.434,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed May 8 23:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.433},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.40999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59299999999999997}]},{"osd":6,"last update":"Wed May 8 23:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed May 8 23:45:18 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55700000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85399999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46600000000000003}]},{"osd":9,"last update":"Wed May 8 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69899999999999995}]},{"osd":10,"last update":"Wed May 8 23:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:45:49.903 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T23:45:49.903 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T23:45:49.903 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T23:45:49.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T23:45:50.158 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:50.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:50 smithi012 ceph-mon[42503]: pgmap v248: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:45:50.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:50 smithi012 ceph-mon[42503]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:50.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:50 smithi204 ceph-mon[47373]: pgmap v248: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:45:50.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:50 smithi204 ceph-mon[47373]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:50.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:50 smithi160 ceph-mon[46152]: pgmap v248: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:45:50.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:50 smithi160 ceph-mon[46152]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:45:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.334+0000 7fd08b642700 1 -- 172.21.15.12:0/1938575053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd084105f00 msgr2=0x7fd0841062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.334+0000 7fd08b642700 1 --2- 172.21.15.12:0/1938575053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd084105f00 0x7fd0841062e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd080009990 tx=0x7fd080009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.334+0000 7fd08b642700 1 -- 172.21.15.12:0/1938575053 shutdown_connections 2024-05-08T23:45:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.334+0000 7fd08b642700 1 --2- 172.21.15.12:0/1938575053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd084100890 0x7fd08410cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.334+0000 7fd08b642700 1 --2- 172.21.15.12:0/1938575053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd084105f00 0x7fd0841062e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.334+0000 7fd08b642700 1 --2- 172.21.15.12:0/1938575053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0840ffed0 0x7fd084100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.334+0000 7fd08b642700 1 -- 172.21.15.12:0/1938575053 >> 172.21.15.12:0/1938575053 conn(0x7fd084075f80 msgr2=0x7fd084076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.334+0000 7fd08b642700 1 -- 172.21.15.12:0/1938575053 shutdown_connections 2024-05-08T23:45:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.334+0000 7fd08b642700 1 -- 172.21.15.12:0/1938575053 wait complete. 2024-05-08T23:45:51.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.335+0000 7fd08b642700 1 Processor -- start 2024-05-08T23:45:51.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.335+0000 7fd08b642700 1 -- start start 2024-05-08T23:45:51.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.335+0000 7fd08b642700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0840ffed0 0x7fd084196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:51.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.335+0000 7fd08b642700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd084100890 0x7fd0841967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:51.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.335+0000 7fd08b642700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd084105f00 0x7fd08419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:51.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd08b642700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08419b190 con 0x7fd084105f00 2024-05-08T23:45:51.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd08b642700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08419b300 con 0x7fd084100890 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd08b642700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08419b4c0 con 0x7fd0840ffed0 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd089bdf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd084105f00 0x7fd08419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd0893de700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0840ffed0 0x7fd084196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd089bdf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd084105f00 0x7fd08419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55524/0 (socket says 172.21.15.12:55524) 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd0893de700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0840ffed0 0x7fd084196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59774/0 (socket says 172.21.15.12:59774) 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd089bdf700 1 -- 172.21.15.12:0/450241574 learned_addr learned my addr 172.21.15.12:0/450241574 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd088bdd700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd084100890 0x7fd0841967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd089bdf700 1 -- 172.21.15.12:0/450241574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0840ffed0 msgr2=0x7fd084196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd089bdf700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0840ffed0 0x7fd084196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd089bdf700 1 -- 172.21.15.12:0/450241574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd084100890 msgr2=0x7fd0841967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd089bdf700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd084100890 0x7fd0841967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd089bdf700 1 -- 172.21.15.12:0/450241574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0800095f0 con 0x7fd084105f00 2024-05-08T23:45:51.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd088bdd700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd084100890 0x7fd0841967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:51.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd0893de700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0840ffed0 0x7fd084196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:51.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd089bdf700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd084105f00 0x7fd08419ab80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd07c007ee0 tx=0x7fd07c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:51.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd07a7fc700 1 -- 172.21.15.12:0/450241574 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c017070 con 0x7fd084105f00 2024-05-08T23:45:51.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd08b642700 1 -- 172.21.15.12:0/450241574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08419b7a0 con 0x7fd084105f00 2024-05-08T23:45:51.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd07a7fc700 1 -- 172.21.15.12:0/450241574 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd07c00f040 con 0x7fd084105f00 2024-05-08T23:45:51.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.336+0000 7fd07a7fc700 1 -- 172.21.15.12:0/450241574 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c005740 con 0x7fd084105f00 2024-05-08T23:45:51.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.337+0000 7fd08b642700 1 -- 172.21.15.12:0/450241574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0841a3000 con 0x7fd084105f00 2024-05-08T23:45:51.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.337+0000 7fd08b642700 1 -- 172.21.15.12:0/450241574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd084108420 con 0x7fd084105f00 2024-05-08T23:45:51.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.339+0000 7fd07a7fc700 1 -- 172.21.15.12:0/450241574 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd07c00dc60 con 0x7fd084105f00 2024-05-08T23:45:51.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.339+0000 7fd07a7fc700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd0700a11b0 0x7fd0700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:51.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.339+0000 7fd0893de700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd0700a11b0 0x7fd0700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:51.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.339+0000 7fd07a7fc700 1 -- 172.21.15.12:0/450241574 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd07c0cc680 con 0x7fd084105f00 2024-05-08T23:45:51.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.340+0000 7fd0893de700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd0700a11b0 0x7fd0700a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b5c0 tx=0x7fd08001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:51.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.344+0000 7fd07a7fc700 1 -- 172.21.15.12:0/450241574 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd07c095e60 con 0x7fd084105f00 2024-05-08T23:45:51.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.606+0000 7fd08b642700 1 -- 172.21.15.12:0/450241574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd0841015b0 con 0x7fd084105f00 2024-05-08T23:45:51.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.606+0000 7fd07a7fc700 1 -- 172.21.15.12:0/450241574 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd07c099480 con 0x7fd084105f00 2024-05-08T23:45:51.602 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:51.602 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T23:45:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 -- 172.21.15.12:0/450241574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd0700a11b0 msgr2=0x7fd0700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd0700a11b0 0x7fd0700a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b5c0 tx=0x7fd08001a040 comp rx=0 tx=0).stop 2024-05-08T23:45:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 -- 172.21.15.12:0/450241574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd084105f00 msgr2=0x7fd08419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd084105f00 0x7fd08419ab80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd07c007ee0 tx=0x7fd07c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 -- 172.21.15.12:0/450241574 shutdown_connections 2024-05-08T23:45:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd084105f00 0x7fd08419ab80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0840ffed0 0x7fd084196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd0700a11b0 0x7fd0700a3670 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 --2- 172.21.15.12:0/450241574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd084100890 0x7fd0841967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 -- 172.21.15.12:0/450241574 >> 172.21.15.12:0/450241574 conn(0x7fd084075f80 msgr2=0x7fd0840fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 -- 172.21.15.12:0/450241574 shutdown_connections 2024-05-08T23:45:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:51.609+0000 7fd08b642700 1 -- 172.21.15.12:0/450241574 wait complete. 2024-05-08T23:45:52.004 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T23:45:52.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:45:52.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.160+0000 7fa12ede7700 1 -- 172.21.15.12:0/3510391725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281042a0 msgr2=0x7fa128106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.160+0000 7fa12ede7700 1 --2- 172.21.15.12:0/3510391725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281042a0 0x7fa128106690 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa118009990 tx=0x7fa118009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.160+0000 7fa12ede7700 1 -- 172.21.15.12:0/3510391725 shutdown_connections 2024-05-08T23:45:52.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.160+0000 7fa12ede7700 1 --2- 172.21.15.12:0/3510391725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa128106bd0 0x7fa128108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.160+0000 7fa12ede7700 1 --2- 172.21.15.12:0/3510391725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa128101970 0x7fa128103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.160+0000 7fa12ede7700 1 --2- 172.21.15.12:0/3510391725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281042a0 0x7fa128106690 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.160+0000 7fa12ede7700 1 -- 172.21.15.12:0/3510391725 >> 172.21.15.12:0/3510391725 conn(0x7fa1280fb300 msgr2=0x7fa1280fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:52.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.160+0000 7fa12ede7700 1 -- 172.21.15.12:0/3510391725 shutdown_connections 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.160+0000 7fa12ede7700 1 -- 172.21.15.12:0/3510391725 wait complete. 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12ede7700 1 Processor -- start 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12ede7700 1 -- start start 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12ede7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa128101970 0x7fa128101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12ede7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281042a0 0x7fa1280ff920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12ede7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa128106bd0 0x7fa1280ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12ede7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa128100600 con 0x7fa1281042a0 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12ede7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa128100770 con 0x7fa128106bd0 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12ede7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1281a0db0 con 0x7fa128101970 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa11ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281042a0 0x7fa1280ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa11ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281042a0 0x7fa1280ff920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55544/0 (socket says 172.21.15.12:55544) 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa11ffff700 1 -- 172.21.15.12:0/713264397 learned_addr learned my addr 172.21.15.12:0/713264397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12cb83700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa128101970 0x7fa128101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12d384700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa128106bd0 0x7fa1280ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa11ffff700 1 -- 172.21.15.12:0/713264397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa128101970 msgr2=0x7fa128101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa11ffff700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa128101970 0x7fa128101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa11ffff700 1 -- 172.21.15.12:0/713264397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa128106bd0 msgr2=0x7fa1280ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa11ffff700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa128106bd0 0x7fa1280ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa11ffff700 1 -- 172.21.15.12:0/713264397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180095f0 con 0x7fa1281042a0 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.161+0000 7fa12d384700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa128106bd0 0x7fa1280ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.162+0000 7fa11ffff700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281042a0 0x7fa1280ff920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa1180095c0 tx=0x7fa11800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.162+0000 7fa11dffb700 1 -- 172.21.15.12:0/713264397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11801d070 con 0x7fa1281042a0 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.162+0000 7fa12ede7700 1 -- 172.21.15.12:0/713264397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1281a0fe0 con 0x7fa1281042a0 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.162+0000 7fa11dffb700 1 -- 172.21.15.12:0/713264397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa118022470 con 0x7fa1281042a0 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.162+0000 7fa11dffb700 1 -- 172.21.15.12:0/713264397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118017650 con 0x7fa1281042a0 2024-05-08T23:45:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.162+0000 7fa12ede7700 1 -- 172.21.15.12:0/713264397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1281a13a0 con 0x7fa1281042a0 2024-05-08T23:45:52.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.163+0000 7fa12ede7700 1 -- 172.21.15.12:0/713264397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1280fce80 con 0x7fa1281042a0 2024-05-08T23:45:52.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.164+0000 7fa11dffb700 1 -- 172.21.15.12:0/713264397 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa118022850 con 0x7fa1281042a0 2024-05-08T23:45:52.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.165+0000 7fa11dffb700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1100a11b0 0x7fa1100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.165+0000 7fa12cb83700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1100a11b0 0x7fa1100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.165+0000 7fa11dffb700 1 -- 172.21.15.12:0/713264397 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa1180cdfe0 con 0x7fa1281042a0 2024-05-08T23:45:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.165+0000 7fa12cb83700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1100a11b0 0x7fa1100a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa124005950 tx=0x7fa12400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.169+0000 7fa11dffb700 1 -- 172.21.15.12:0/713264397 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa118097960 con 0x7fa1281042a0 2024-05-08T23:45:52.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.389+0000 7fa12ede7700 1 -- 172.21.15.12:0/713264397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa12810c650 con 0x7fa1281042a0 2024-05-08T23:45:52.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.390+0000 7fa11dffb700 1 -- 172.21.15.12:0/713264397 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22105 (secure 0 0 0) 0x7fa11809af80 con 0x7fa1281042a0 2024-05-08T23:45:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:52.387 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:05.029999+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 -- 172.21.15.12:0/713264397 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1100a11b0 msgr2=0x7fa1100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1100a11b0 0x7fa1100a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa124005950 tx=0x7fa12400b410 comp rx=0 tx=0).stop 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 -- 172.21.15.12:0/713264397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281042a0 msgr2=0x7fa1280ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281042a0 0x7fa1280ff920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa1180095c0 tx=0x7fa11800f690 comp rx=0 tx=0).stop 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 -- 172.21.15.12:0/713264397 shutdown_connections 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa1100a11b0 0x7fa1100a3670 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa128106bd0 0x7fa1280ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa128101970 0x7fa128101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 --2- 172.21.15.12:0/713264397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1281042a0 0x7fa1280ff920 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 -- 172.21.15.12:0/713264397 >> 172.21.15.12:0/713264397 conn(0x7fa1280fb300 msgr2=0x7fa1281023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 -- 172.21.15.12:0/713264397 shutdown_connections 2024-05-08T23:45:52.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.393+0000 7fa12ede7700 1 -- 172.21.15.12:0/713264397 wait complete. 2024-05-08T23:45:52.403 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:45:02.165326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:45:03.407498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:45:52.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:45:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.549+0000 7f6bf3191700 1 -- 172.21.15.12:0/3536786516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bec100190 msgr2=0x7f6bec100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.549+0000 7f6bf3191700 1 --2- 172.21.15.12:0/3536786516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bec100190 0x7f6bec100610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80099c0 tx=0x7f6bd8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.549+0000 7f6bf3191700 1 -- 172.21.15.12:0/3536786516 shutdown_connections 2024-05-08T23:45:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.549+0000 7f6bf3191700 1 --2- 172.21.15.12:0/3536786516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bec100b50 0x7f6bec100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.549+0000 7f6bf3191700 1 --2- 172.21.15.12:0/3536786516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bec0ff830 0x7f6bec0ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.549+0000 7f6bf3191700 1 --2- 172.21.15.12:0/3536786516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bec100190 0x7f6bec100610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.549+0000 7f6bf3191700 1 -- 172.21.15.12:0/3536786516 >> 172.21.15.12:0/3536786516 conn(0x7f6bec0fb3f0 msgr2=0x7f6bec0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.549+0000 7f6bf3191700 1 -- 172.21.15.12:0/3536786516 shutdown_connections 2024-05-08T23:45:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.549+0000 7f6bf3191700 1 -- 172.21.15.12:0/3536786516 wait complete. 2024-05-08T23:45:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf3191700 1 Processor -- start 2024-05-08T23:45:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf3191700 1 -- start start 2024-05-08T23:45:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf3191700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bec0ff830 0x7f6bec199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf3191700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bec100190 0x7f6bec1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf3191700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bec100b50 0x7f6bec1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf3191700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1a0ba0 con 0x7f6bec0ff830 2024-05-08T23:45:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf3191700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1a0ce0 con 0x7f6bec100b50 2024-05-08T23:45:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf3191700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1a0e70 con 0x7f6bec100190 2024-05-08T23:45:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf0f2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bec0ff830 0x7f6bec199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf0f2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bec0ff830 0x7f6bec199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55564/0 (socket says 172.21.15.12:55564) 2024-05-08T23:45:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf0f2d700 1 -- 172.21.15.12:0/2007484366 learned_addr learned my addr 172.21.15.12:0/2007484366 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf172e700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bec100b50 0x7f6bec1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bebfff700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bec100190 0x7f6bec1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf0f2d700 1 -- 172.21.15.12:0/2007484366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bec100190 msgr2=0x7f6bec1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf0f2d700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bec100190 0x7f6bec1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf0f2d700 1 -- 172.21.15.12:0/2007484366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bec100b50 msgr2=0x7f6bec1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf0f2d700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bec100b50 0x7f6bec1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf0f2d700 1 -- 172.21.15.12:0/2007484366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd80095f0 con 0x7f6bec0ff830 2024-05-08T23:45:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.550+0000 7f6bf0f2d700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bec0ff830 0x7f6bec199070 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6be000e9a0 tx=0x7f6be000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.551+0000 7f6be9ffb700 1 -- 172.21.15.12:0/2007484366 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be0009790 con 0x7f6bec0ff830 2024-05-08T23:45:52.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.551+0000 7f6be9ffb700 1 -- 172.21.15.12:0/2007484366 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6be0010430 con 0x7f6bec0ff830 2024-05-08T23:45:52.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.551+0000 7f6bf3191700 1 -- 172.21.15.12:0/2007484366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec1a1150 con 0x7f6bec0ff830 2024-05-08T23:45:52.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.551+0000 7f6be9ffb700 1 -- 172.21.15.12:0/2007484366 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be0018680 con 0x7f6bec0ff830 2024-05-08T23:45:52.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.551+0000 7f6bf3191700 1 -- 172.21.15.12:0/2007484366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec1a1660 con 0x7f6bec0ff830 2024-05-08T23:45:52.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.552+0000 7f6bf3191700 1 -- 172.21.15.12:0/2007484366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec10a840 con 0x7f6bec0ff830 2024-05-08T23:45:52.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.553+0000 7f6be9ffb700 1 -- 172.21.15.12:0/2007484366 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6be00105a0 con 0x7f6bec0ff830 2024-05-08T23:45:52.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.554+0000 7f6be9ffb700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bdc0a11b0 0x7f6bdc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.554+0000 7f6be9ffb700 1 -- 172.21.15.12:0/2007484366 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6be0014070 con 0x7f6bec0ff830 2024-05-08T23:45:52.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.554+0000 7f6bebfff700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bdc0a11b0 0x7f6bdc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.555+0000 7f6bebfff700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bdc0a11b0 0x7f6bdc0a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009990 tx=0x7f6bd8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:52.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.558+0000 7f6be9ffb700 1 -- 172.21.15.12:0/2007484366 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6be0096430 con 0x7f6bec0ff830 2024-05-08T23:45:52.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:52 smithi012 ceph-mon[42503]: pgmap v249: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:45:52.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:52 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/450241574' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T23:45:52.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:52 smithi204 ceph-mon[47373]: pgmap v249: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:45:52.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:52 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/450241574' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T23:45:52.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:52 smithi160 ceph-mon[46152]: pgmap v249: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:45:52.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:52 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/450241574' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T23:45:52.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.774+0000 7f6bf3191700 1 -- 172.21.15.12:0/2007484366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6bec10e950 con 0x7f6bec0ff830 2024-05-08T23:45:52.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.774+0000 7f6be9ffb700 1 -- 172.21.15.12:0/2007484366 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6be00260a0 con 0x7f6bec0ff830 2024-05-08T23:45:52.770 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 -- 172.21.15.12:0/2007484366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bdc0a11b0 msgr2=0x7f6bdc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bdc0a11b0 0x7f6bdc0a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009990 tx=0x7f6bd8000bc0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 -- 172.21.15.12:0/2007484366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bec0ff830 msgr2=0x7f6bec199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bec0ff830 0x7f6bec199070 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6be000e9a0 tx=0x7f6be000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 -- 172.21.15.12:0/2007484366 shutdown_connections 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6bdc0a11b0 0x7f6bdc0a3670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bec100b50 0x7f6bec1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bec0ff830 0x7f6bec199070 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 --2- 172.21.15.12:0/2007484366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bec100190 0x7f6bec1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 -- 172.21.15.12:0/2007484366 >> 172.21.15.12:0/2007484366 conn(0x7f6bec0fb3f0 msgr2=0x7f6bec0fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:52.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 -- 172.21.15.12:0/2007484366 shutdown_connections 2024-05-08T23:45:52.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.777+0000 7f6bf3191700 1 -- 172.21.15.12:0/2007484366 wait complete. 2024-05-08T23:45:52.786 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:45:52.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.930+0000 7f881961f700 1 -- 172.21.15.12:0/522486057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8814100f80 msgr2=0x7f88141013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.930+0000 7f881961f700 1 --2- 172.21.15.12:0/522486057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8814100f80 0x7f88141013a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009920 tx=0x7f87fc009c30 comp rx=0 tx=0).stop 2024-05-08T23:45:52.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.931+0000 7f881961f700 1 -- 172.21.15.12:0/522486057 shutdown_connections 2024-05-08T23:45:52.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.931+0000 7f881961f700 1 --2- 172.21.15.12:0/522486057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8814102b10 0x7f88141093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.931+0000 7f881961f700 1 --2- 172.21.15.12:0/522486057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8814100f80 0x7f88141013a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.931+0000 7f881961f700 1 --2- 172.21.15.12:0/522486057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8814102170 0x7f88141025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.931+0000 7f881961f700 1 -- 172.21.15.12:0/522486057 >> 172.21.15.12:0/522486057 conn(0x7f88140fc4f0 msgr2=0x7f88140fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:52.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.931+0000 7f881961f700 1 -- 172.21.15.12:0/522486057 shutdown_connections 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.931+0000 7f881961f700 1 -- 172.21.15.12:0/522486057 wait complete. 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.931+0000 7f881961f700 1 Processor -- start 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.931+0000 7f881961f700 1 -- start start 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f881961f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8814100f80 0x7f8814197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f881961f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8814102170 0x7f8814197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f881961f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8814102b10 0x7f881419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f881961f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881419eae0 con 0x7f8814102b10 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f881961f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881419ec20 con 0x7f8814102170 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f881961f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881419edb0 con 0x7f8814100f80 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8814102b10 0x7f881419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88127fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8814102170 0x7f8814197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f8812ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8814100f80 0x7f8814197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8814102b10 0x7f881419e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55578/0 (socket says 172.21.15.12:55578) 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88137fe700 1 -- 172.21.15.12:0/1377610641 learned_addr learned my addr 172.21.15.12:0/1377610641 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88127fc700 1 -- 172.21.15.12:0/1377610641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8814100f80 msgr2=0x7f8814197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88127fc700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8814100f80 0x7f8814197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88127fc700 1 -- 172.21.15.12:0/1377610641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8814102b10 msgr2=0x7f881419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88127fc700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8814102b10 0x7f881419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88127fc700 1 -- 172.21.15.12:0/1377610641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8804009520 con 0x7f8814102170 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f8812ffd700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8814100f80 0x7f8814197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88137fe700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8814102b10 0x7f881419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.932+0000 7f88127fc700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8814102170 0x7f8814197540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f880400e9f0 tx=0x7f880400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.933+0000 7f880bfff700 1 -- 172.21.15.12:0/1377610641 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880400cb70 con 0x7f8814102170 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.933+0000 7f881961f700 1 -- 172.21.15.12:0/1377610641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880c009d00 con 0x7f8814102170 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.933+0000 7f880bfff700 1 -- 172.21.15.12:0/1377610641 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f880400ee90 con 0x7f8814102170 2024-05-08T23:45:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.933+0000 7f880bfff700 1 -- 172.21.15.12:0/1377610641 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804003cf0 con 0x7f8814102170 2024-05-08T23:45:52.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.933+0000 7f881961f700 1 -- 172.21.15.12:0/1377610641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc009520 con 0x7f8814102170 2024-05-08T23:45:52.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.934+0000 7f881961f700 1 -- 172.21.15.12:0/1377610641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8814108440 con 0x7f8814102170 2024-05-08T23:45:52.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.935+0000 7f880bfff700 1 -- 172.21.15.12:0/1377610641 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f880400ccd0 con 0x7f8814102170 2024-05-08T23:45:52.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.936+0000 7f880bfff700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f88000a11b0 0x7f88000a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:52.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.936+0000 7f8812ffd700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f88000a11b0 0x7f88000a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:52.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.936+0000 7f880bfff700 1 -- 172.21.15.12:0/1377610641 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8804014070 con 0x7f8814102170 2024-05-08T23:45:52.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.936+0000 7f8812ffd700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f88000a11b0 0x7f88000a3670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0098f0 tx=0x7f87fc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:52.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:52.939+0000 7f880bfff700 1 -- 172.21.15.12:0/1377610641 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f88040964a0 con 0x7f8814102170 2024-05-08T23:45:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.161+0000 7f881961f700 1 -- 172.21.15.12:0/1377610641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f881410c8a0 con 0x7f8814102170 2024-05-08T23:45:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.161+0000 7f880bfff700 1 -- 172.21.15.12:0/1377610641 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f880400fe60 con 0x7f8814102170 2024-05-08T23:45:53.157 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-08T23:45:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 -- 172.21.15.12:0/1377610641 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f88000a11b0 msgr2=0x7f88000a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f88000a11b0 0x7f88000a3670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0098f0 tx=0x7f87fc019040 comp rx=0 tx=0).stop 2024-05-08T23:45:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 -- 172.21.15.12:0/1377610641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8814102170 msgr2=0x7f8814197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8814102170 0x7f8814197540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f880400e9f0 tx=0x7f880400ed00 comp rx=0 tx=0).stop 2024-05-08T23:45:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 -- 172.21.15.12:0/1377610641 shutdown_connections 2024-05-08T23:45:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8814102b10 0x7f881419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8814100f80 0x7f8814197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f88000a11b0 0x7f88000a3670 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 --2- 172.21.15.12:0/1377610641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8814102170 0x7f8814197540 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 -- 172.21.15.12:0/1377610641 >> 172.21.15.12:0/1377610641 conn(0x7f88140fc4f0 msgr2=0x7f8814105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:53.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 -- 172.21.15.12:0/1377610641 shutdown_connections 2024-05-08T23:45:53.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.164+0000 7f881961f700 1 -- 172.21.15.12:0/1377610641 wait complete. 2024-05-08T23:45:53.174 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:45:53.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.319+0000 7faccee97700 1 -- 172.21.15.12:0/4125070095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc8106c50 msgr2=0x7facc81090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.319+0000 7faccee97700 1 --2- 172.21.15.12:0/4125070095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc8106c50 0x7facc81090c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7facc000b0a0 tx=0x7facc000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.319+0000 7faccee97700 1 -- 172.21.15.12:0/4125070095 shutdown_connections 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.319+0000 7faccee97700 1 --2- 172.21.15.12:0/4125070095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc8106c50 0x7facc81090c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.319+0000 7faccee97700 1 --2- 172.21.15.12:0/4125070095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc81019f0 0x7facc8103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.319+0000 7faccee97700 1 --2- 172.21.15.12:0/4125070095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc8104320 0x7facc8106710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.319+0000 7faccee97700 1 -- 172.21.15.12:0/4125070095 >> 172.21.15.12:0/4125070095 conn(0x7facc80fb400 msgr2=0x7facc80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.320+0000 7faccee97700 1 -- 172.21.15.12:0/4125070095 shutdown_connections 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.320+0000 7faccee97700 1 -- 172.21.15.12:0/4125070095 wait complete. 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.320+0000 7faccee97700 1 Processor -- start 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.320+0000 7faccee97700 1 -- start start 2024-05-08T23:45:53.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.320+0000 7faccee97700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc81019f0 0x7facc8101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.320+0000 7faccee97700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8104320 0x7facc80ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.320+0000 7faccee97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc8106c50 0x7facc80ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.320+0000 7faccee97700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc8100680 con 0x7facc81019f0 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.320+0000 7faccee97700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc81007f0 con 0x7facc8106c50 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.320+0000 7faccee97700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc81a0ff0 con 0x7facc8104320 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facccc33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc81019f0 0x7facc8101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facccc33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc81019f0 0x7facc8101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55606/0 (socket says 172.21.15.12:55606) 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facccc33700 1 -- 172.21.15.12:0/257453599 learned_addr learned my addr 172.21.15.12:0/257453599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7faccd434700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc8106c50 0x7facc80ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facc7fff700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8104320 0x7facc80ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facccc33700 1 -- 172.21.15.12:0/257453599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8104320 msgr2=0x7facc80ff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facccc33700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8104320 0x7facc80ff9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facccc33700 1 -- 172.21.15.12:0/257453599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc8106c50 msgr2=0x7facc80ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facccc33700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc8106c50 0x7facc80ffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facccc33700 1 -- 172.21.15.12:0/257453599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc0009d00 con 0x7facc81019f0 2024-05-08T23:45:53.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facccc33700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc81019f0 0x7facc8101310 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7facbc00d980 tx=0x7facbc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:53.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facc5ffb700 1 -- 172.21.15.12:0/257453599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc004220 con 0x7facc81019f0 2024-05-08T23:45:53.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facc5ffb700 1 -- 172.21.15.12:0/257453599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facbc007640 con 0x7facc81019f0 2024-05-08T23:45:53.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7faccee97700 1 -- 172.21.15.12:0/257453599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc81a1280 con 0x7facc81019f0 2024-05-08T23:45:53.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.321+0000 7facc5ffb700 1 -- 172.21.15.12:0/257453599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc005610 con 0x7facc81019f0 2024-05-08T23:45:53.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.322+0000 7faccee97700 1 -- 172.21.15.12:0/257453599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc81a1790 con 0x7facc81019f0 2024-05-08T23:45:53.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.323+0000 7faccee97700 1 -- 172.21.15.12:0/257453599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc80fcf80 con 0x7facc81019f0 2024-05-08T23:45:53.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.324+0000 7facc5ffb700 1 -- 172.21.15.12:0/257453599 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7facbc004750 con 0x7facc81019f0 2024-05-08T23:45:53.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.324+0000 7facc5ffb700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7facb80a11b0 0x7facb80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:53.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.324+0000 7facc7fff700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7facb80a11b0 0x7facb80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:53.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.324+0000 7facc5ffb700 1 -- 172.21.15.12:0/257453599 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7facbc0cc770 con 0x7facc81019f0 2024-05-08T23:45:53.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.325+0000 7facc7fff700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7facb80a11b0 0x7facb80a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7facb4005950 tx=0x7facb40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:53.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.329+0000 7facc5ffb700 1 -- 172.21.15.12:0/257453599 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7facbc095f50 con 0x7facc81019f0 2024-05-08T23:45:53.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.548+0000 7faccee97700 1 -- 172.21.15.12:0/257453599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facc810c900 con 0x7facc81019f0 2024-05-08T23:45:53.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.548+0000 7facc5ffb700 1 -- 172.21.15.12:0/257453599 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7facbc0160a0 con 0x7facc81019f0 2024-05-08T23:45:53.544 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-08T23:45:53.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 -- 172.21.15.12:0/257453599 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7facb80a11b0 msgr2=0x7facb80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7facb80a11b0 0x7facb80a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7facb4005950 tx=0x7facb40058e0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 -- 172.21.15.12:0/257453599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc81019f0 msgr2=0x7facc8101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc81019f0 0x7facc8101310 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7facbc00d980 tx=0x7facbc00dc90 comp rx=0 tx=0).stop 2024-05-08T23:45:53.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 -- 172.21.15.12:0/257453599 shutdown_connections 2024-05-08T23:45:53.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7facb80a11b0 0x7facb80a3670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facc8106c50 0x7facc80ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc81019f0 0x7facc8101310 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 --2- 172.21.15.12:0/257453599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8104320 0x7facc80ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 -- 172.21.15.12:0/257453599 >> 172.21.15.12:0/257453599 conn(0x7facc80fb400 msgr2=0x7facc8102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:53.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 -- 172.21.15.12:0/257453599 shutdown_connections 2024-05-08T23:45:53.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.551+0000 7faccee97700 1 -- 172.21.15.12:0/257453599 wait complete. 2024-05-08T23:45:53.560 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:53 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/713264397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:53.560 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:53 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2007484366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:45:53.560 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:53 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1377610641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:45:53.561 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:45:53.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.703+0000 7f7586903700 1 -- 172.21.15.12:0/1278378626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7580100f50 msgr2=0x7f75801013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.703+0000 7f7586903700 1 --2- 172.21.15.12:0/1278378626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7580100f50 0x7f75801013b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f75740099c0 tx=0x7f7574009cd0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.703+0000 7f7586903700 1 -- 172.21.15.12:0/1278378626 shutdown_connections 2024-05-08T23:45:53.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.703+0000 7f7586903700 1 --2- 172.21.15.12:0/1278378626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75801018f0 0x7f75801081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.703+0000 7f7586903700 1 --2- 172.21.15.12:0/1278378626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75800ffd60 0x7f7580100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.703+0000 7f7586903700 1 --2- 172.21.15.12:0/1278378626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7580100f50 0x7f75801013b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.703+0000 7f7586903700 1 -- 172.21.15.12:0/1278378626 >> 172.21.15.12:0/1278378626 conn(0x7f75800fb2f0 msgr2=0x7f75800fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:53.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.703+0000 7f7586903700 1 -- 172.21.15.12:0/1278378626 shutdown_connections 2024-05-08T23:45:53.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.703+0000 7f7586903700 1 -- 172.21.15.12:0/1278378626 wait complete. 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.704+0000 7f7586903700 1 Processor -- start 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.704+0000 7f7586903700 1 -- start start 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.704+0000 7f7586903700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75800ffd60 0x7f7580103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.704+0000 7f7586903700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7580100f50 0x7f7580102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7586903700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75801018f0 0x7f7580102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7586903700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7580103010 con 0x7f75801018f0 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7586903700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7580103180 con 0x7f7580100f50 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7586903700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75801a0e40 con 0x7f75800ffd60 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7584ea0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75801018f0 0x7f7580102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7584ea0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75801018f0 0x7f7580102870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55632/0 (socket says 172.21.15.12:55632) 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7584ea0700 1 -- 172.21.15.12:0/1264050983 learned_addr learned my addr 172.21.15.12:0/1264050983 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:53.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f757ffff700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75800ffd60 0x7f7580103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f757f7fe700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7580100f50 0x7f7580102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7584ea0700 1 -- 172.21.15.12:0/1264050983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75800ffd60 msgr2=0x7f7580103ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7584ea0700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75800ffd60 0x7f7580103ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7584ea0700 1 -- 172.21.15.12:0/1264050983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7580100f50 msgr2=0x7f7580102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7584ea0700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7580100f50 0x7f7580102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7584ea0700 1 -- 172.21.15.12:0/1264050983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75740095f0 con 0x7f75801018f0 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f757ffff700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75800ffd60 0x7f7580103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7584ea0700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75801018f0 0x7f7580102870 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7578007ee0 tx=0x7f757800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f757d7fa700 1 -- 172.21.15.12:0/1264050983 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578017070 con 0x7f75801018f0 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f757d7fa700 1 -- 172.21.15.12:0/1264050983 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f757800f040 con 0x7f75801018f0 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7586903700 1 -- 172.21.15.12:0/1264050983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75801a10d0 con 0x7f75801018f0 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f757d7fa700 1 -- 172.21.15.12:0/1264050983 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780054b0 con 0x7f75801018f0 2024-05-08T23:45:53.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.705+0000 7f7586903700 1 -- 172.21.15.12:0/1264050983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75801a1590 con 0x7f75801018f0 2024-05-08T23:45:53.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.706+0000 7f7586903700 1 -- 172.21.15.12:0/1264050983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7580067c10 con 0x7f75801018f0 2024-05-08T23:45:53.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.708+0000 7f757d7fa700 1 -- 172.21.15.12:0/1264050983 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7578004ad0 con 0x7f75801018f0 2024-05-08T23:45:53.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.708+0000 7f757d7fa700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f75680a11b0 0x7f75680a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:53.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.709+0000 7f757ffff700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f75680a11b0 0x7f75680a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:53.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.709+0000 7f757d7fa700 1 -- 172.21.15.12:0/1264050983 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f75780cc730 con 0x7f75801018f0 2024-05-08T23:45:53.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.709+0000 7f757ffff700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f75680a11b0 0x7f75680a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7580100db0 tx=0x7f7570009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:53.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.712+0000 7f757d7fa700 1 -- 172.21.15.12:0/1264050983 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7578095f10 con 0x7f75801018f0 2024-05-08T23:45:53.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:53 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/713264397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:53.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:53 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2007484366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:45:53.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:53 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1377610641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:45:53.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:53 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/713264397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:53.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:53 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2007484366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:45:53.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:53 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1377610641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:45:53.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.978+0000 7f7586903700 1 -- 172.21.15.12:0/1264050983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f758010b570 con 0x7f75801018f0 2024-05-08T23:45:53.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.979+0000 7f757d7fa700 1 -- 172.21.15.12:0/1264050983 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10876 (secure 0 0 0) 0x7f7578099530 con 0x7f75801018f0 2024-05-08T23:45:53.975 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:53.975 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:45:09.067610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:45:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 -- 172.21.15.12:0/1264050983 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f75680a11b0 msgr2=0x7f75680a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f75680a11b0 0x7f75680a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7580100db0 tx=0x7f7570009260 comp rx=0 tx=0).stop 2024-05-08T23:45:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 -- 172.21.15.12:0/1264050983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75801018f0 msgr2=0x7f7580102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75801018f0 0x7f7580102870 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7578007ee0 tx=0x7f757800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 -- 172.21.15.12:0/1264050983 shutdown_connections 2024-05-08T23:45:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75801018f0 0x7f7580102870 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75800ffd60 0x7f7580103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f75680a11b0 0x7f75680a3670 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 --2- 172.21.15.12:0/1264050983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7580100f50 0x7f7580102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:53.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 -- 172.21.15.12:0/1264050983 >> 172.21.15.12:0/1264050983 conn(0x7f75800fb2f0 msgr2=0x7f7580068040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:53.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 -- 172.21.15.12:0/1264050983 shutdown_connections 2024-05-08T23:45:53.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:53.982+0000 7f7586903700 1 -- 172.21.15.12:0/1264050983 wait complete. 2024-05-08T23:45:53.979 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 23 2024-05-08T23:45:53.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:45:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.133+0000 7fbde1d07700 1 -- 172.21.15.12:0/1075735241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbddc106b70 msgr2=0x7fbddc108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.133+0000 7fbde1d07700 1 --2- 172.21.15.12:0/1075735241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbddc106b70 0x7fbddc108fe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400b0a0 tx=0x7fbdd400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.134+0000 7fbde1d07700 1 -- 172.21.15.12:0/1075735241 shutdown_connections 2024-05-08T23:45:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.134+0000 7fbde1d07700 1 --2- 172.21.15.12:0/1075735241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbddc106b70 0x7fbddc108fe0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.134+0000 7fbde1d07700 1 --2- 172.21.15.12:0/1075735241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc101910 0x7fbddc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.134+0000 7fbde1d07700 1 --2- 172.21.15.12:0/1075735241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc104240 0x7fbddc106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.134+0000 7fbde1d07700 1 -- 172.21.15.12:0/1075735241 >> 172.21.15.12:0/1075735241 conn(0x7fbddc0fb300 msgr2=0x7fbddc0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.134+0000 7fbde1d07700 1 -- 172.21.15.12:0/1075735241 shutdown_connections 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.134+0000 7fbde1d07700 1 -- 172.21.15.12:0/1075735241 wait complete. 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.134+0000 7fbde1d07700 1 Processor -- start 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbde1d07700 1 -- start start 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbde1d07700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc101910 0x7fbddc101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbde1d07700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc104240 0x7fbddc0ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbde1d07700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbddc106b70 0x7fbddc0ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbde1d07700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc1005a0 con 0x7fbddc106b70 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbde1d07700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc100710 con 0x7fbddc101910 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbde1d07700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc1a0f60 con 0x7fbddc104240 2024-05-08T23:45:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbddc106b70 0x7fbddc0ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbddc106b70 0x7fbddc0ffe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55654/0 (socket says 172.21.15.12:55654) 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddbfff700 1 -- 172.21.15.12:0/1704849067 learned_addr learned my addr 172.21.15.12:0/1704849067 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddb7fe700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc101910 0x7fbddc101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddaffd700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc104240 0x7fbddc0ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddbfff700 1 -- 172.21.15.12:0/1704849067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc104240 msgr2=0x7fbddc0ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddbfff700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc104240 0x7fbddc0ff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddbfff700 1 -- 172.21.15.12:0/1704849067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc101910 msgr2=0x7fbddc101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddbfff700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc101910 0x7fbddc101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddbfff700 1 -- 172.21.15.12:0/1704849067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd4009d00 con 0x7fbddc106b70 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddaffd700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc104240 0x7fbddc0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddb7fe700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc101910 0x7fbddc101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.135+0000 7fbddbfff700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbddc106b70 0x7fbddc0ffe00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4007a60 tx=0x7fbdd400bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.136+0000 7fbdd8ff9700 1 -- 172.21.15.12:0/1704849067 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400e040 con 0x7fbddc106b70 2024-05-08T23:45:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.136+0000 7fbde1d07700 1 -- 172.21.15.12:0/1704849067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc1a1190 con 0x7fbddc106b70 2024-05-08T23:45:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.136+0000 7fbdd8ff9700 1 -- 172.21.15.12:0/1704849067 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdd4004e60 con 0x7fbddc106b70 2024-05-08T23:45:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.136+0000 7fbdd8ff9700 1 -- 172.21.15.12:0/1704849067 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd40052a0 con 0x7fbddc106b70 2024-05-08T23:45:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.136+0000 7fbde1d07700 1 -- 172.21.15.12:0/1704849067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc1a15b0 con 0x7fbddc106b70 2024-05-08T23:45:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.137+0000 7fbde1d07700 1 -- 172.21.15.12:0/1704849067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc0fce80 con 0x7fbddc106b70 2024-05-08T23:45:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.138+0000 7fbdd8ff9700 1 -- 172.21.15.12:0/1704849067 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbdd401d9c0 con 0x7fbddc106b70 2024-05-08T23:45:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.139+0000 7fbdd8ff9700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbdc40a11b0 0x7fbdc40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.139+0000 7fbddb7fe700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbdc40a11b0 0x7fbdc40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.139+0000 7fbdd8ff9700 1 -- 172.21.15.12:0/1704849067 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbdd40ce810 con 0x7fbddc106b70 2024-05-08T23:45:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.139+0000 7fbddb7fe700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbdc40a11b0 0x7fbdc40a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc005fd0 tx=0x7fbdcc005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:54.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.143+0000 7fbdd8ff9700 1 -- 172.21.15.12:0/1704849067 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbdd4098000 con 0x7fbddc106b70 2024-05-08T23:45:54.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.367+0000 7fbde1d07700 1 -- 172.21.15.12:0/1704849067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbddc067380 con 0x7fbddc106b70 2024-05-08T23:45:54.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.368+0000 7fbdd8ff9700 1 -- 172.21.15.12:0/1704849067 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22105 (secure 0 0 0) 0x7fbdd40253b0 con 0x7fbddc106b70 2024-05-08T23:45:54.364 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:54.365 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:05.029999+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:45:54.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 -- 172.21.15.12:0/1704849067 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbdc40a11b0 msgr2=0x7fbdc40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbdc40a11b0 0x7fbdc40a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc005fd0 tx=0x7fbdcc005f60 comp rx=0 tx=0).stop 2024-05-08T23:45:54.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 -- 172.21.15.12:0/1704849067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbddc106b70 msgr2=0x7fbddc0ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbddc106b70 0x7fbddc0ffe00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4007a60 tx=0x7fbdd400bd90 comp rx=0 tx=0).stop 2024-05-08T23:45:54.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 -- 172.21.15.12:0/1704849067 shutdown_connections 2024-05-08T23:45:54.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbddc106b70 0x7fbddc0ffe00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fbdc40a11b0 0x7fbdc40a3670 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbddc101910 0x7fbddc101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 --2- 172.21.15.12:0/1704849067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc104240 0x7fbddc0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 -- 172.21.15.12:0/1704849067 >> 172.21.15.12:0/1704849067 conn(0x7fbddc0fb300 msgr2=0x7fbddc102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:54.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 -- 172.21.15.12:0/1704849067 shutdown_connections 2024-05-08T23:45:54.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.371+0000 7fbde1d07700 1 -- 172.21.15.12:0/1704849067 wait complete. 2024-05-08T23:45:54.381 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:45:02.165326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:45:03.407498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:45:54.382 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:45:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.526+0000 7f94cf020700 1 -- 172.21.15.12:0/2096038477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c81043d0 msgr2=0x7f94c81067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.526+0000 7f94cf020700 1 --2- 172.21.15.12:0/2096038477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c81043d0 0x7f94c81067c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009990 tx=0x7f94b4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.526+0000 7f94cf020700 1 -- 172.21.15.12:0/2096038477 shutdown_connections 2024-05-08T23:45:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.526+0000 7f94cf020700 1 --2- 172.21.15.12:0/2096038477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c8106d00 0x7f94c81090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.526+0000 7f94cf020700 1 --2- 172.21.15.12:0/2096038477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c8101aa0 0x7f94c8103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.526+0000 7f94cf020700 1 --2- 172.21.15.12:0/2096038477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c81043d0 0x7f94c81067c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.526+0000 7f94cf020700 1 -- 172.21.15.12:0/2096038477 >> 172.21.15.12:0/2096038477 conn(0x7f94c80fb3f0 msgr2=0x7f94c80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.526+0000 7f94cf020700 1 -- 172.21.15.12:0/2096038477 shutdown_connections 2024-05-08T23:45:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.526+0000 7f94cf020700 1 -- 172.21.15.12:0/2096038477 wait complete. 2024-05-08T23:45:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.527+0000 7f94cf020700 1 Processor -- start 2024-05-08T23:45:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.527+0000 7f94cf020700 1 -- start start 2024-05-08T23:45:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.527+0000 7f94cf020700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c8101aa0 0x7f94c81013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.527+0000 7f94cf020700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c81043d0 0x7f94c80ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.527+0000 7f94cf020700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c8106d00 0x7f94c80fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.527+0000 7f94cf020700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c8100730 con 0x7f94c8101aa0 2024-05-08T23:45:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.527+0000 7f94cf020700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c81008a0 con 0x7f94c81043d0 2024-05-08T23:45:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94cf020700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c81a0f00 con 0x7f94c8106d00 2024-05-08T23:45:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94ccdbc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c8101aa0 0x7f94c81013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94ccdbc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c8101aa0 0x7f94c81013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55660/0 (socket says 172.21.15.12:55660) 2024-05-08T23:45:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94ccdbc700 1 -- 172.21.15.12:0/934398488 learned_addr learned my addr 172.21.15.12:0/934398488 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94cd5bd700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c8106d00 0x7f94c80fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94c7fff700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c81043d0 0x7f94c80ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94ccdbc700 1 -- 172.21.15.12:0/934398488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c8106d00 msgr2=0x7f94c80fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94ccdbc700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c8106d00 0x7f94c80fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94ccdbc700 1 -- 172.21.15.12:0/934398488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c81043d0 msgr2=0x7f94c80ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94ccdbc700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c81043d0 0x7f94c80ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94ccdbc700 1 -- 172.21.15.12:0/934398488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b40095f0 con 0x7f94c8101aa0 2024-05-08T23:45:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94cd5bd700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c8106d00 0x7f94c80fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94c7fff700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c81043d0 0x7f94c80ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94ccdbc700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c8101aa0 0x7f94c81013c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d950 tx=0x7f94bc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94c5ffb700 1 -- 172.21.15.12:0/934398488 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc009ae0 con 0x7f94c8101aa0 2024-05-08T23:45:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94c5ffb700 1 -- 172.21.15.12:0/934398488 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94bc009c40 con 0x7f94c8101aa0 2024-05-08T23:45:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94cf020700 1 -- 172.21.15.12:0/934398488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c81a1190 con 0x7f94c8101aa0 2024-05-08T23:45:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.528+0000 7f94c5ffb700 1 -- 172.21.15.12:0/934398488 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00f620 con 0x7f94c8101aa0 2024-05-08T23:45:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.529+0000 7f94cf020700 1 -- 172.21.15.12:0/934398488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c81a16a0 con 0x7f94c8101aa0 2024-05-08T23:45:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.530+0000 7f94cf020700 1 -- 172.21.15.12:0/934398488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c80fcf70 con 0x7f94c8101aa0 2024-05-08T23:45:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.531+0000 7f94c5ffb700 1 -- 172.21.15.12:0/934398488 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f94bc010460 con 0x7f94c8101aa0 2024-05-08T23:45:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.531+0000 7f94c5ffb700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f94b80a11b0 0x7f94b80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.531+0000 7f94c7fff700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f94b80a11b0 0x7f94b80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.531+0000 7f94c5ffb700 1 -- 172.21.15.12:0/934398488 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f94bc0cc950 con 0x7f94c8101aa0 2024-05-08T23:45:54.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.532+0000 7f94c7fff700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f94b80a11b0 0x7f94b80a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f94b40037f0 tx=0x7f94b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:54.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.536+0000 7f94c5ffb700 1 -- 172.21.15.12:0/934398488 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f94bc096030 con 0x7f94c8101aa0 2024-05-08T23:45:54.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:54 smithi012 ceph-mon[42503]: pgmap v250: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:45:54.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:54 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/257453599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:45:54.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:54 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1264050983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.754+0000 7f94cf020700 1 -- 172.21.15.12:0/934398488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94c810c910 con 0x7f94c8101aa0 2024-05-08T23:45:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.755+0000 7f94c5ffb700 1 -- 172.21.15.12:0/934398488 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94bc0160a0 con 0x7f94c8101aa0 2024-05-08T23:45:54.751 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-08T23:45:54.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 -- 172.21.15.12:0/934398488 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f94b80a11b0 msgr2=0x7f94b80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f94b80a11b0 0x7f94b80a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f94b40037f0 tx=0x7f94b4005fb0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 -- 172.21.15.12:0/934398488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c8101aa0 msgr2=0x7f94c81013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c8101aa0 0x7f94c81013c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d950 tx=0x7f94bc00dd10 comp rx=0 tx=0).stop 2024-05-08T23:45:54.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 -- 172.21.15.12:0/934398488 shutdown_connections 2024-05-08T23:45:54.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c8106d00 0x7f94c80fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c8101aa0 0x7f94c81013c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f94b80a11b0 0x7f94b80a3670 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 --2- 172.21.15.12:0/934398488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c81043d0 0x7f94c80ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 -- 172.21.15.12:0/934398488 >> 172.21.15.12:0/934398488 conn(0x7f94c80fb3f0 msgr2=0x7f94c8102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:54.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 -- 172.21.15.12:0/934398488 shutdown_connections 2024-05-08T23:45:54.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.757+0000 7f94cf020700 1 -- 172.21.15.12:0/934398488 wait complete. 2024-05-08T23:45:54.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:54 smithi204 ceph-mon[47373]: pgmap v250: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:45:54.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:54 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/257453599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:45:54.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:54 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1264050983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:54.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:54 smithi160 ceph-mon[46152]: pgmap v250: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:45:54.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:54 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/257453599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:45:54.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:54 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1264050983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:54.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:45:54.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.911+0000 7ff2b8df0700 1 -- 172.21.15.12:0/4155763811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b4101a00 msgr2=0x7ff2b4103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.911+0000 7ff2b8df0700 1 --2- 172.21.15.12:0/4155763811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b4101a00 0x7ff2b4103df0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009990 tx=0x7ff2a4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.911+0000 7ff2b8df0700 1 -- 172.21.15.12:0/4155763811 shutdown_connections 2024-05-08T23:45:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.911+0000 7ff2b8df0700 1 --2- 172.21.15.12:0/4155763811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2b4106c60 0x7ff2b41090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.911+0000 7ff2b8df0700 1 --2- 172.21.15.12:0/4155763811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b4101a00 0x7ff2b4103df0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.911+0000 7ff2b8df0700 1 --2- 172.21.15.12:0/4155763811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b4104330 0x7ff2b4106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.911+0000 7ff2b8df0700 1 -- 172.21.15.12:0/4155763811 >> 172.21.15.12:0/4155763811 conn(0x7ff2b40fb3f0 msgr2=0x7ff2b40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.911+0000 7ff2b8df0700 1 -- 172.21.15.12:0/4155763811 shutdown_connections 2024-05-08T23:45:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.911+0000 7ff2b8df0700 1 -- 172.21.15.12:0/4155763811 wait complete. 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.912+0000 7ff2b8df0700 1 Processor -- start 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.912+0000 7ff2b8df0700 1 -- start start 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.912+0000 7ff2b8df0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2b4101a00 0x7ff2b4196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.912+0000 7ff2b8df0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b4104330 0x7ff2b4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.912+0000 7ff2b8df0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b4106c60 0x7ff2b419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.912+0000 7ff2b8df0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419ea00 con 0x7ff2b4106c60 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.912+0000 7ff2b8df0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419eb40 con 0x7ff2b4101a00 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.912+0000 7ff2b8df0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419ecd0 con 0x7ff2b4104330 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b4106c60 0x7ff2b419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b4106c60 0x7ff2b419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55688/0 (socket says 172.21.15.12:55688) 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b2d9d700 1 -- 172.21.15.12:0/665398469 learned_addr learned my addr 172.21.15.12:0/665398469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b259c700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2b4101a00 0x7ff2b4196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2a9d9b700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b4104330 0x7ff2b4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b2d9d700 1 -- 172.21.15.12:0/665398469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b4104330 msgr2=0x7ff2b4197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b2d9d700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b4104330 0x7ff2b4197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b2d9d700 1 -- 172.21.15.12:0/665398469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2b4101a00 msgr2=0x7ff2b4196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b2d9d700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2b4101a00 0x7ff2b4196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b2d9d700 1 -- 172.21.15.12:0/665398469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40095f0 con 0x7ff2b4106c60 2024-05-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2a9d9b700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b4104330 0x7ff2b4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b2d9d700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b4106c60 0x7ff2b419e430 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac007ee0 tx=0x7ff2ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2abfff700 1 -- 172.21.15.12:0/665398469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac00ddd0 con 0x7ff2b4106c60 2024-05-08T23:45:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b8df0700 1 -- 172.21.15.12:0/665398469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b419efb0 con 0x7ff2b4106c60 2024-05-08T23:45:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2abfff700 1 -- 172.21.15.12:0/665398469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2ac00f040 con 0x7ff2b4106c60 2024-05-08T23:45:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2abfff700 1 -- 172.21.15.12:0/665398469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac0048e0 con 0x7ff2b4106c60 2024-05-08T23:45:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.913+0000 7ff2b8df0700 1 -- 172.21.15.12:0/665398469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b419f430 con 0x7ff2b4106c60 2024-05-08T23:45:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.915+0000 7ff2b8df0700 1 -- 172.21.15.12:0/665398469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b40fcf70 con 0x7ff2b4106c60 2024-05-08T23:45:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.916+0000 7ff2abfff700 1 -- 172.21.15.12:0/665398469 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff2ac003a00 con 0x7ff2b4106c60 2024-05-08T23:45:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.916+0000 7ff2abfff700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2a00a11b0 0x7ff2a00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.917+0000 7ff2b259c700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2a00a11b0 0x7ff2a00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.917+0000 7ff2abfff700 1 -- 172.21.15.12:0/665398469 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff2ac0cc1a0 con 0x7ff2b4106c60 2024-05-08T23:45:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.917+0000 7ff2b259c700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2a00a11b0 0x7ff2a00a3670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400b5c0 tx=0x7ff2a4003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:54.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:54.922+0000 7ff2abfff700 1 -- 172.21.15.12:0/665398469 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff2ac095980 con 0x7ff2b4106c60 2024-05-08T23:45:55.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.144+0000 7ff2b8df0700 1 -- 172.21.15.12:0/665398469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2b410c870 con 0x7ff2b4106c60 2024-05-08T23:45:55.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.145+0000 7ff2abfff700 1 -- 172.21.15.12:0/665398469 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff2ac098fa0 con 0x7ff2b4106c60 2024-05-08T23:45:55.140 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-08T23:45:55.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 -- 172.21.15.12:0/665398469 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2a00a11b0 msgr2=0x7ff2a00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2a00a11b0 0x7ff2a00a3670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400b5c0 tx=0x7ff2a4003790 comp rx=0 tx=0).stop 2024-05-08T23:45:55.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 -- 172.21.15.12:0/665398469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b4106c60 msgr2=0x7ff2b419e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b4106c60 0x7ff2b419e430 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac007ee0 tx=0x7ff2ac00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 -- 172.21.15.12:0/665398469 shutdown_connections 2024-05-08T23:45:55.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b4106c60 0x7ff2b419e430 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff2a00a11b0 0x7ff2a00a3670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2b4101a00 0x7ff2b4196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 --2- 172.21.15.12:0/665398469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b4104330 0x7ff2b4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 -- 172.21.15.12:0/665398469 >> 172.21.15.12:0/665398469 conn(0x7ff2b40fb3f0 msgr2=0x7ff2b4102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:55.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 -- 172.21.15.12:0/665398469 shutdown_connections 2024-05-08T23:45:55.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.147+0000 7ff2b8df0700 1 -- 172.21.15.12:0/665398469 wait complete. 2024-05-08T23:45:55.158 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:45:55.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.305+0000 7ff90f488700 1 -- 172.21.15.12:0/1404758939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9081019e0 msgr2=0x7ff908103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.305+0000 7ff90f488700 1 --2- 172.21.15.12:0/1404758939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9081019e0 0x7ff908103dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff904009990 tx=0x7ff904009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.305+0000 7ff90f488700 1 -- 172.21.15.12:0/1404758939 shutdown_connections 2024-05-08T23:45:55.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.305+0000 7ff90f488700 1 --2- 172.21.15.12:0/1404758939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff908106c40 0x7ff9081090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.306+0000 7ff90f488700 1 --2- 172.21.15.12:0/1404758939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9081019e0 0x7ff908103dd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.306+0000 7ff90f488700 1 --2- 172.21.15.12:0/1404758939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff908104310 0x7ff908106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.306+0000 7ff90f488700 1 -- 172.21.15.12:0/1404758939 >> 172.21.15.12:0/1404758939 conn(0x7ff9080fb3f0 msgr2=0x7ff9080fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:55.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.306+0000 7ff90f488700 1 -- 172.21.15.12:0/1404758939 shutdown_connections 2024-05-08T23:45:55.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.306+0000 7ff90f488700 1 -- 172.21.15.12:0/1404758939 wait complete. 2024-05-08T23:45:55.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.306+0000 7ff90f488700 1 Processor -- start 2024-05-08T23:45:55.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90f488700 1 -- start start 2024-05-08T23:45:55.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90f488700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9081019e0 0x7ff908196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:55.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90f488700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff908104310 0x7ff908197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:55.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90f488700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff908106c40 0x7ff90819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:55.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90f488700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90819e940 con 0x7ff9081019e0 2024-05-08T23:45:55.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90f488700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90819ea80 con 0x7ff908106c40 2024-05-08T23:45:55.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90f488700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90819ec10 con 0x7ff908104310 2024-05-08T23:45:55.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90d224700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9081019e0 0x7ff908196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:55.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90d224700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9081019e0 0x7ff908196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55706/0 (socket says 172.21.15.12:55706) 2024-05-08T23:45:55.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90d224700 1 -- 172.21.15.12:0/2476896486 learned_addr learned my addr 172.21.15.12:0/2476896486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:55.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90ca23700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff908104310 0x7ff908197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:55.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90da25700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff908106c40 0x7ff90819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:55.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90d224700 1 -- 172.21.15.12:0/2476896486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff908104310 msgr2=0x7ff908197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90d224700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff908104310 0x7ff908197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90d224700 1 -- 172.21.15.12:0/2476896486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff908106c40 msgr2=0x7ff90819e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90d224700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff908106c40 0x7ff90819e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90d224700 1 -- 172.21.15.12:0/2476896486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9040095f0 con 0x7ff9081019e0 2024-05-08T23:45:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90ca23700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff908104310 0x7ff908197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.307+0000 7ff90d224700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9081019e0 0x7ff908196e10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff9040095c0 tx=0x7ff90400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.308+0000 7ff8fe7fc700 1 -- 172.21.15.12:0/2476896486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90401d070 con 0x7ff9081019e0 2024-05-08T23:45:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.308+0000 7ff8fe7fc700 1 -- 172.21.15.12:0/2476896486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff904005510 con 0x7ff9081019e0 2024-05-08T23:45:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.308+0000 7ff90f488700 1 -- 172.21.15.12:0/2476896486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90819ee90 con 0x7ff9081019e0 2024-05-08T23:45:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.308+0000 7ff8fe7fc700 1 -- 172.21.15.12:0/2476896486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904004cd0 con 0x7ff9081019e0 2024-05-08T23:45:55.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.308+0000 7ff90f488700 1 -- 172.21.15.12:0/2476896486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90819f250 con 0x7ff9081019e0 2024-05-08T23:45:55.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.309+0000 7ff90f488700 1 -- 172.21.15.12:0/2476896486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9080fcf70 con 0x7ff9081019e0 2024-05-08T23:45:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.311+0000 7ff8fe7fc700 1 -- 172.21.15.12:0/2476896486 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff90400fc40 con 0x7ff9081019e0 2024-05-08T23:45:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.311+0000 7ff8fe7fc700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff8f40a1230 0x7ff8f40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.311+0000 7ff90ca23700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff8f40a1230 0x7ff8f40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.311+0000 7ff8fe7fc700 1 -- 172.21.15.12:0/2476896486 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff9040cdbf0 con 0x7ff9081019e0 2024-05-08T23:45:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.312+0000 7ff90ca23700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff8f40a1230 0x7ff8f40a36f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8005fd0 tx=0x7ff8f8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:55.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.315+0000 7ff8fe7fc700 1 -- 172.21.15.12:0/2476896486 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff904097580 con 0x7ff9081019e0 2024-05-08T23:45:55.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:55 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1704849067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:55.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:55 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/934398488' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:45:55.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:55 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/665398469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:45:55.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-08T23:45:55.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-08T23:45:55.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T23:45:55.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-08T23:45:55.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-08T23:45:55.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:55 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-08T23:45:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.539+0000 7ff90f488700 1 -- 172.21.15.12:0/2476896486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff90810c740 con 0x7ff9081019e0 2024-05-08T23:45:55.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.540+0000 7ff8fe7fc700 1 -- 172.21.15.12:0/2476896486 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff904026020 con 0x7ff9081019e0 2024-05-08T23:45:55.536 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-08T23:45:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 -- 172.21.15.12:0/2476896486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff8f40a1230 msgr2=0x7ff8f40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff8f40a1230 0x7ff8f40a36f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8005fd0 tx=0x7ff8f8005e30 comp rx=0 tx=0).stop 2024-05-08T23:45:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 -- 172.21.15.12:0/2476896486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9081019e0 msgr2=0x7ff908196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9081019e0 0x7ff908196e10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff9040095c0 tx=0x7ff90400f690 comp rx=0 tx=0).stop 2024-05-08T23:45:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 -- 172.21.15.12:0/2476896486 shutdown_connections 2024-05-08T23:45:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff8f40a1230 0x7ff8f40a36f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff908106c40 0x7ff90819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9081019e0 0x7ff908196e10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 --2- 172.21.15.12:0/2476896486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff908104310 0x7ff908197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 -- 172.21.15.12:0/2476896486 >> 172.21.15.12:0/2476896486 conn(0x7ff9080fb3f0 msgr2=0x7ff908102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 -- 172.21.15.12:0/2476896486 shutdown_connections 2024-05-08T23:45:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.542+0000 7ff90f488700 1 -- 172.21.15.12:0/2476896486 wait complete. 2024-05-08T23:45:55.552 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:45:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.694+0000 7f47240a6700 1 -- 172.21.15.12:0/3192761863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f471c0feee0 msgr2=0x7f471c0ff340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.694+0000 7f47240a6700 1 --2- 172.21.15.12:0/3192761863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f471c0feee0 0x7f471c0ff340 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f470c0099c0 tx=0x7f470c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.694+0000 7f47240a6700 1 -- 172.21.15.12:0/3192761863 shutdown_connections 2024-05-08T23:45:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.694+0000 7f47240a6700 1 --2- 172.21.15.12:0/3192761863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471c0ff880 0x7f471c109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.694+0000 7f47240a6700 1 --2- 172.21.15.12:0/3192761863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c102470 0x7f471c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.694+0000 7f47240a6700 1 --2- 172.21.15.12:0/3192761863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f471c0feee0 0x7f471c0ff340 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.694+0000 7f47240a6700 1 -- 172.21.15.12:0/3192761863 >> 172.21.15.12:0/3192761863 conn(0x7f471c0faa80 msgr2=0x7f471c0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.694+0000 7f47240a6700 1 -- 172.21.15.12:0/3192761863 shutdown_connections 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.694+0000 7f47240a6700 1 -- 172.21.15.12:0/3192761863 wait complete. 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.694+0000 7f47240a6700 1 Processor -- start 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f47240a6700 1 -- start start 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f47240a6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471c0feee0 0x7f471c101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f47240a6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c0ff880 0x7f471c1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f47240a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f471c102470 0x7f471c1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f47240a6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c100e90 con 0x7f471c102470 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f47240a6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c101000 con 0x7f471c0feee0 2024-05-08T23:45:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f47240a6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c1a0e20 con 0x7f471c0ff880 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4722643700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f471c102470 0x7f471c1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4722643700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f471c102470 0x7f471c1006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55714/0 (socket says 172.21.15.12:55714) 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4722643700 1 -- 172.21.15.12:0/3972951499 learned_addr learned my addr 172.21.15.12:0/3972951499 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4721641700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c0ff880 0x7f471c1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4721e42700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471c0feee0 0x7f471c101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4722643700 1 -- 172.21.15.12:0/3972951499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c0ff880 msgr2=0x7f471c1001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4722643700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c0ff880 0x7f471c1001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4722643700 1 -- 172.21.15.12:0/3972951499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471c0feee0 msgr2=0x7f471c101b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4722643700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471c0feee0 0x7f471c101b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4722643700 1 -- 172.21.15.12:0/3972951499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c0095f0 con 0x7f471c102470 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.695+0000 7f4722643700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f471c102470 0x7f471c1006f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4714007ee0 tx=0x7f471400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:55.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.696+0000 7f4712ffd700 1 -- 172.21.15.12:0/3972951499 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714017070 con 0x7f471c102470 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.696+0000 7f4712ffd700 1 -- 172.21.15.12:0/3972951499 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f471400f040 con 0x7f471c102470 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.696+0000 7f47240a6700 1 -- 172.21.15.12:0/3972951499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471c1a10b0 con 0x7f471c102470 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.696+0000 7f4712ffd700 1 -- 172.21.15.12:0/3972951499 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140054b0 con 0x7f471c102470 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.696+0000 7f47240a6700 1 -- 172.21.15.12:0/3972951499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c1a1570 con 0x7f471c102470 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.697+0000 7f47240a6700 1 -- 172.21.15.12:0/3972951499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471c1083c0 con 0x7f471c102470 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.698+0000 7f4712ffd700 1 -- 172.21.15.12:0/3972951499 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4714004ad0 con 0x7f471c102470 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.699+0000 7f4712ffd700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47080a11b0 0x7f47080a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.699+0000 7f4721e42700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47080a11b0 0x7f47080a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.699+0000 7f4712ffd700 1 -- 172.21.15.12:0/3972951499 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f47140cc7e0 con 0x7f471c102470 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.699+0000 7f4721e42700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47080a11b0 0x7f47080a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f471c069130 tx=0x7f471800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.703+0000 7f4712ffd700 1 -- 172.21.15.12:0/3972951499 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4714095f10 con 0x7f471c102470 2024-05-08T23:45:55.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:55 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1704849067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:55.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:55 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/934398488' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:45:55.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:55 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/665398469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:45:55.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-08T23:45:55.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-08T23:45:55.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T23:45:55.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-08T23:45:55.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-08T23:45:55.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:55 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-08T23:45:55.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:55 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1704849067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:55.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:55 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/934398488' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:45:55.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:55 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/665398469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:45:55.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-08T23:45:55.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-08T23:45:55.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T23:45:55.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-08T23:45:55.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-08T23:45:55.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:55 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-08T23:45:55.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.968+0000 7f47240a6700 1 -- 172.21.15.12:0/3972951499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f471c068060 con 0x7f471c102470 2024-05-08T23:45:55.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.969+0000 7f4712ffd700 1 -- 172.21.15.12:0/3972951499 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10876 (secure 0 0 0) 0x7f4714099530 con 0x7f471c102470 2024-05-08T23:45:55.965 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:55.965 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:45:09.067610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:45:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 -- 172.21.15.12:0/3972951499 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47080a11b0 msgr2=0x7f47080a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47080a11b0 0x7f47080a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f471c069130 tx=0x7f471800b410 comp rx=0 tx=0).stop 2024-05-08T23:45:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 -- 172.21.15.12:0/3972951499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f471c102470 msgr2=0x7f471c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f471c102470 0x7f471c1006f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4714007ee0 tx=0x7f471400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 -- 172.21.15.12:0/3972951499 shutdown_connections 2024-05-08T23:45:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f471c102470 0x7f471c1006f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f47080a11b0 0x7f47080a3670 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471c0feee0 0x7f471c101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 --2- 172.21.15.12:0/3972951499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f471c0ff880 0x7f471c1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 -- 172.21.15.12:0/3972951499 >> 172.21.15.12:0/3972951499 conn(0x7f471c0faa80 msgr2=0x7f471c105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 -- 172.21.15.12:0/3972951499 shutdown_connections 2024-05-08T23:45:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:55.971+0000 7f47240a6700 1 -- 172.21.15.12:0/3972951499 wait complete. 2024-05-08T23:45:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 23 2024-05-08T23:45:55.981 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:45:56.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.122+0000 7f042789d700 1 -- 172.21.15.12:0/3833375546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0420100f50 msgr2=0x7f04201013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.122+0000 7f042789d700 1 --2- 172.21.15.12:0/3833375546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0420100f50 0x7f04201013b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f04100099c0 tx=0x7f0410009cd0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.122+0000 7f042789d700 1 -- 172.21.15.12:0/3833375546 shutdown_connections 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.122+0000 7f042789d700 1 --2- 172.21.15.12:0/3833375546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04201018f0 0x7f04201081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.122+0000 7f042789d700 1 --2- 172.21.15.12:0/3833375546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04200ffd60 0x7f0420100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.122+0000 7f042789d700 1 --2- 172.21.15.12:0/3833375546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0420100f50 0x7f04201013b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.122+0000 7f042789d700 1 -- 172.21.15.12:0/3833375546 >> 172.21.15.12:0/3833375546 conn(0x7f04200fb2f0 msgr2=0x7f04200fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.122+0000 7f042789d700 1 -- 172.21.15.12:0/3833375546 shutdown_connections 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.122+0000 7f042789d700 1 -- 172.21.15.12:0/3833375546 wait complete. 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.123+0000 7f042789d700 1 Processor -- start 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.123+0000 7f042789d700 1 -- start start 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.123+0000 7f042789d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04200ffd60 0x7f0420103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.123+0000 7f042789d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0420100f50 0x7f0420102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.123+0000 7f042789d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04201018f0 0x7f0420102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.123+0000 7f042789d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0420103010 con 0x7f04200ffd60 2024-05-08T23:45:56.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.123+0000 7f042789d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0420103180 con 0x7f04201018f0 2024-05-08T23:45:56.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.123+0000 7f042789d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04201a0e20 con 0x7f0420100f50 2024-05-08T23:45:56.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.123+0000 7f0425639700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04200ffd60 0x7f0420103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0425639700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04200ffd60 0x7f0420103ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55726/0 (socket says 172.21.15.12:55726) 2024-05-08T23:45:56.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0425639700 1 -- 172.21.15.12:0/516800876 learned_addr learned my addr 172.21.15.12:0/516800876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:56.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0424e38700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0420100f50 0x7f0420102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0425e3a700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04201018f0 0x7f0420102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0425639700 1 -- 172.21.15.12:0/516800876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0420100f50 msgr2=0x7f0420102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0425639700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0420100f50 0x7f0420102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0425639700 1 -- 172.21.15.12:0/516800876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04201018f0 msgr2=0x7f0420102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0425639700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04201018f0 0x7f0420102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0425639700 1 -- 172.21.15.12:0/516800876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04100095f0 con 0x7f04200ffd60 2024-05-08T23:45:56.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0425e3a700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04201018f0 0x7f0420102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:56.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f0425639700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04200ffd60 0x7f0420103ca0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f041c00e970 tx=0x7f041c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:56.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f04167fc700 1 -- 172.21.15.12:0/516800876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c009790 con 0x7f04200ffd60 2024-05-08T23:45:56.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f042789d700 1 -- 172.21.15.12:0/516800876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04201a10b0 con 0x7f04200ffd60 2024-05-08T23:45:56.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f04167fc700 1 -- 172.21.15.12:0/516800876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f041c00eec0 con 0x7f04200ffd60 2024-05-08T23:45:56.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f04167fc700 1 -- 172.21.15.12:0/516800876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c003c90 con 0x7f04200ffd60 2024-05-08T23:45:56.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.124+0000 7f042789d700 1 -- 172.21.15.12:0/516800876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04201a15c0 con 0x7f04200ffd60 2024-05-08T23:45:56.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.125+0000 7f042789d700 1 -- 172.21.15.12:0/516800876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0420067c10 con 0x7f04200ffd60 2024-05-08T23:45:56.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.126+0000 7f04167fc700 1 -- 172.21.15.12:0/516800876 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f041c00cc70 con 0x7f04200ffd60 2024-05-08T23:45:56.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.127+0000 7f04167fc700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f040c0a11b0 0x7f040c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.127+0000 7f0424e38700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f040c0a11b0 0x7f040c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.127+0000 7f04167fc700 1 -- 172.21.15.12:0/516800876 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f041c014070 con 0x7f04200ffd60 2024-05-08T23:45:56.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.128+0000 7f0424e38700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f040c0a11b0 0x7f040c0a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0410009990 tx=0x7f0410000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:56.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.131+0000 7f04167fc700 1 -- 172.21.15.12:0/516800876 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f041c096450 con 0x7f04200ffd60 2024-05-08T23:45:56.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.351+0000 7f042789d700 1 -- 172.21.15.12:0/516800876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f042010b570 con 0x7f04200ffd60 2024-05-08T23:45:56.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.352+0000 7f04167fc700 1 -- 172.21.15.12:0/516800876 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22445 (secure 0 0 0) 0x7f041c09ee60 con 0x7f04200ffd60 2024-05-08T23:45:56.349 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:56.349 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:56.347006+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:45:56.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 -- 172.21.15.12:0/516800876 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f040c0a11b0 msgr2=0x7f040c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f040c0a11b0 0x7f040c0a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0410009990 tx=0x7f0410000bc0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 -- 172.21.15.12:0/516800876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04200ffd60 msgr2=0x7f0420103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04200ffd60 0x7f0420103ca0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f041c00e970 tx=0x7f041c00ec80 comp rx=0 tx=0).stop 2024-05-08T23:45:56.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 -- 172.21.15.12:0/516800876 shutdown_connections 2024-05-08T23:45:56.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f040c0a11b0 0x7f040c0a3670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04201018f0 0x7f0420102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04200ffd60 0x7f0420103ca0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 --2- 172.21.15.12:0/516800876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0420100f50 0x7f0420102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 -- 172.21.15.12:0/516800876 >> 172.21.15.12:0/516800876 conn(0x7f04200fb2f0 msgr2=0x7f04200fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:56.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 -- 172.21.15.12:0/516800876 shutdown_connections 2024-05-08T23:45:56.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.356+0000 7f042789d700 1 -- 172.21.15.12:0/516800876 wait complete. 2024-05-08T23:45:56.370 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:45:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.512+0000 7f6fbfb98700 1 -- 172.21.15.12:0/793793475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb81018f0 msgr2=0x7f6fb81081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.512+0000 7f6fbfb98700 1 --2- 172.21.15.12:0/793793475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb81018f0 0x7f6fb81081a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000b0d0 tx=0x7f6fb000b3e0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.512+0000 7f6fbfb98700 1 -- 172.21.15.12:0/793793475 shutdown_connections 2024-05-08T23:45:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.512+0000 7f6fbfb98700 1 --2- 172.21.15.12:0/793793475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb81018f0 0x7f6fb81081a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.512+0000 7f6fbfb98700 1 --2- 172.21.15.12:0/793793475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb80ffd60 0x7f6fb8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.512+0000 7f6fbfb98700 1 --2- 172.21.15.12:0/793793475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fb8100f50 0x7f6fb81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.512+0000 7f6fbfb98700 1 -- 172.21.15.12:0/793793475 >> 172.21.15.12:0/793793475 conn(0x7f6fb80fb2f0 msgr2=0x7f6fb80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.512+0000 7f6fbfb98700 1 -- 172.21.15.12:0/793793475 shutdown_connections 2024-05-08T23:45:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.512+0000 7f6fbfb98700 1 -- 172.21.15.12:0/793793475 wait complete. 2024-05-08T23:45:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.512+0000 7f6fbfb98700 1 Processor -- start 2024-05-08T23:45:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbfb98700 1 -- start start 2024-05-08T23:45:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbfb98700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fb80ffd60 0x7f6fb8103d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbfb98700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb8100f50 0x7f6fb8102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbfb98700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb81018f0 0x7f6fb8102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbfb98700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb81030e0 con 0x7f6fb8100f50 2024-05-08T23:45:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbfb98700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb8103250 con 0x7f6fb80ffd60 2024-05-08T23:45:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbfb98700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb81a0f40 con 0x7f6fb81018f0 2024-05-08T23:45:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbd133700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb8100f50 0x7f6fb8102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbd133700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb8100f50 0x7f6fb8102400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55748/0 (socket says 172.21.15.12:55748) 2024-05-08T23:45:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbd133700 1 -- 172.21.15.12:0/316909820 learned_addr learned my addr 172.21.15.12:0/316909820 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbe135700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb81018f0 0x7f6fb8102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbd934700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fb80ffd60 0x7f6fb8103d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbd133700 1 -- 172.21.15.12:0/316909820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb81018f0 msgr2=0x7f6fb8102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbd133700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb81018f0 0x7f6fb8102940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbd133700 1 -- 172.21.15.12:0/316909820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fb80ffd60 msgr2=0x7f6fb8103d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbd133700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fb80ffd60 0x7f6fb8103d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbd133700 1 -- 172.21.15.12:0/316909820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb0009d00 con 0x7f6fb8100f50 2024-05-08T23:45:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.513+0000 7f6fbd133700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb8100f50 0x7f6fb8102400 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800d980 tx=0x7f6fa800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.514+0000 7f6faeffd700 1 -- 172.21.15.12:0/316909820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8004220 con 0x7f6fb8100f50 2024-05-08T23:45:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.514+0000 7f6faeffd700 1 -- 172.21.15.12:0/316909820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa8007640 con 0x7f6fb8100f50 2024-05-08T23:45:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.514+0000 7f6faeffd700 1 -- 172.21.15.12:0/316909820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8005690 con 0x7f6fb8100f50 2024-05-08T23:45:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.514+0000 7f6fbfb98700 1 -- 172.21.15.12:0/316909820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb81a11d0 con 0x7f6fb8100f50 2024-05-08T23:45:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.514+0000 7f6fbfb98700 1 -- 172.21.15.12:0/316909820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb81a1690 con 0x7f6fb8100f50 2024-05-08T23:45:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.515+0000 7f6fbfb98700 1 -- 172.21.15.12:0/316909820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb8067c10 con 0x7f6fb8100f50 2024-05-08T23:45:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.516+0000 7f6faeffd700 1 -- 172.21.15.12:0/316909820 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6fa8004750 con 0x7f6fb8100f50 2024-05-08T23:45:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.517+0000 7f6faeffd700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6fa40a11b0 0x7f6fa40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.517+0000 7f6fbd934700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6fa40a11b0 0x7f6fa40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.517+0000 7f6faeffd700 1 -- 172.21.15.12:0/316909820 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6fa80cccf0 con 0x7f6fb8100f50 2024-05-08T23:45:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.517+0000 7f6fbd934700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6fa40a11b0 0x7f6fa40a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8100db0 tx=0x7f6fb400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:56.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.521+0000 7f6faeffd700 1 -- 172.21.15.12:0/316909820 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6fa8096320 con 0x7f6fb8100f50 2024-05-08T23:45:56.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:56 smithi012 ceph-mon[42503]: pgmap v251: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:45:56.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-08T23:45:56.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-08T23:45:56.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T23:45:56.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-08T23:45:56.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-08T23:45:56.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:56 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-08T23:45:56.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:56 smithi012 ceph-mon[42503]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T23:45:56.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:56 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2476896486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:45:56.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:56 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3972951499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:56 smithi204 ceph-mon[47373]: pgmap v251: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:45:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:56 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-08T23:45:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:56 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-08T23:45:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:56 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T23:45:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:56 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-08T23:45:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:56 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-08T23:45:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:56 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-08T23:45:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:56 smithi204 ceph-mon[47373]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T23:45:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:56 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2476896486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:45:56.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:56 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3972951499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:56 smithi160 ceph-mon[46152]: pgmap v251: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:45:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:56 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-08T23:45:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:56 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-08T23:45:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:56 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T23:45:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:56 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-08T23:45:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:56 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-08T23:45:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:56 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-08T23:45:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:56 smithi160 ceph-mon[46152]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T23:45:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:56 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2476896486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:45:56.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:56 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3972951499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:56.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.789+0000 7f6fbfb98700 1 -- 172.21.15.12:0/316909820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fb810b580 con 0x7f6fb8100f50 2024-05-08T23:45:56.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.790+0000 7f6faeffd700 1 -- 172.21.15.12:0/316909820 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10876 (secure 0 0 0) 0x7f6fa8016020 con 0x7f6fb8100f50 2024-05-08T23:45:56.785 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:45:56.786 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:45:09.067610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:45:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 -- 172.21.15.12:0/316909820 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6fa40a11b0 msgr2=0x7f6fa40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6fa40a11b0 0x7f6fa40a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8100db0 tx=0x7f6fb400b410 comp rx=0 tx=0).stop 2024-05-08T23:45:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 -- 172.21.15.12:0/316909820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb8100f50 msgr2=0x7f6fb8102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb8100f50 0x7f6fb8102400 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800d980 tx=0x7f6fa800dc90 comp rx=0 tx=0).stop 2024-05-08T23:45:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 -- 172.21.15.12:0/316909820 shutdown_connections 2024-05-08T23:45:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb81018f0 0x7f6fb8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6fa40a11b0 0x7f6fa40a3670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fb80ffd60 0x7f6fb8103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 --2- 172.21.15.12:0/316909820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb8100f50 0x7f6fb8102400 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 -- 172.21.15.12:0/316909820 >> 172.21.15.12:0/316909820 conn(0x7f6fb80fb2f0 msgr2=0x7f6fb8067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 -- 172.21.15.12:0/316909820 shutdown_connections 2024-05-08T23:45:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.792+0000 7f6fbfb98700 1 -- 172.21.15.12:0/316909820 wait complete. 2024-05-08T23:45:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 23 2024-05-08T23:45:56.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T23:45:04.027837+0000', 'modified': '2024-05-08T23:45:09.067610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24548, 'mds_1': 24515, 'mds_2': 24511}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/1481890752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 1481890752}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1481890752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/3313911488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 3313911488}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 3313911488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/2858273239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 2858273239}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 2858273239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.204:6839/2873951026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 2873951026}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 2873951026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.160:6837/1904912186', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 1904912186}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 1904912186}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.160:6839/924146489', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 924146489}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 924146489}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:45:56.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T23:45:56.802 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-08T23:45:56.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.945+0000 7f13669e0700 1 -- 172.21.15.12:0/2423721195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1360100fd0 msgr2=0x7f1360101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.945+0000 7f13669e0700 1 --2- 172.21.15.12:0/2423721195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1360100fd0 0x7f1360101450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1354009990 tx=0x7f1354009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.945+0000 7f13669e0700 1 -- 172.21.15.12:0/2423721195 shutdown_connections 2024-05-08T23:45:56.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.945+0000 7f13669e0700 1 --2- 172.21.15.12:0/2423721195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1360101990 0x7f13601082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.945+0000 7f13669e0700 1 --2- 172.21.15.12:0/2423721195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13600ffe80 0x7f13601002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.945+0000 7f13669e0700 1 --2- 172.21.15.12:0/2423721195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1360100fd0 0x7f1360101450 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.945+0000 7f13669e0700 1 -- 172.21.15.12:0/2423721195 >> 172.21.15.12:0/2423721195 conn(0x7f13600fb3f0 msgr2=0x7f13600fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:56.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.945+0000 7f13669e0700 1 -- 172.21.15.12:0/2423721195 shutdown_connections 2024-05-08T23:45:56.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.945+0000 7f13669e0700 1 -- 172.21.15.12:0/2423721195 wait complete. 2024-05-08T23:45:56.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.946+0000 7f13669e0700 1 Processor -- start 2024-05-08T23:45:56.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.946+0000 7f13669e0700 1 -- start start 2024-05-08T23:45:56.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.946+0000 7f13669e0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13600ffe80 0x7f1360103db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.946+0000 7f13669e0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1360100fd0 0x7f1360102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.946+0000 7f13669e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1360101990 0x7f1360102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.946+0000 7f13669e0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1360103120 con 0x7f1360101990 2024-05-08T23:45:56.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.946+0000 7f13669e0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1360103290 con 0x7f13600ffe80 2024-05-08T23:45:56.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.946+0000 7f13669e0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13601a0fa0 con 0x7f1360100fd0 2024-05-08T23:45:56.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f1364f7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1360101990 0x7f1360102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f1364f7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1360101990 0x7f1360102980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55770/0 (socket says 172.21.15.12:55770) 2024-05-08T23:45:56.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f1364f7d700 1 -- 172.21.15.12:0/412510127 learned_addr learned my addr 172.21.15.12:0/412510127 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:56.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f135ffff700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13600ffe80 0x7f1360103db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f135f7fe700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1360100fd0 0x7f1360102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f1364f7d700 1 -- 172.21.15.12:0/412510127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1360100fd0 msgr2=0x7f1360102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f1364f7d700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1360100fd0 0x7f1360102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f1364f7d700 1 -- 172.21.15.12:0/412510127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13600ffe80 msgr2=0x7f1360103db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f1364f7d700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13600ffe80 0x7f1360103db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f1364f7d700 1 -- 172.21.15.12:0/412510127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13540095f0 con 0x7f1360101990 2024-05-08T23:45:56.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f135f7fe700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1360100fd0 0x7f1360102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:56.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f1364f7d700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1360101990 0x7f1360102980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f135800ef30 tx=0x7f135800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:56.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f135d7fa700 1 -- 172.21.15.12:0/412510127 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358018070 con 0x7f1360101990 2024-05-08T23:45:56.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f135d7fa700 1 -- 172.21.15.12:0/412510127 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f135800f040 con 0x7f1360101990 2024-05-08T23:45:56.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f13669e0700 1 -- 172.21.15.12:0/412510127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13601a11d0 con 0x7f1360101990 2024-05-08T23:45:56.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f135d7fa700 1 -- 172.21.15.12:0/412510127 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800ce00 con 0x7f1360101990 2024-05-08T23:45:56.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.947+0000 7f13669e0700 1 -- 172.21.15.12:0/412510127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13601a16e0 con 0x7f1360101990 2024-05-08T23:45:56.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.949+0000 7f135d7fa700 1 -- 172.21.15.12:0/412510127 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1358003a00 con 0x7f1360101990 2024-05-08T23:45:56.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.950+0000 7f135d7fa700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f13480a1230 0x7f13480a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.950+0000 7f135ffff700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f13480a1230 0x7f13480a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.950+0000 7f135d7fa700 1 -- 172.21.15.12:0/412510127 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f13580cc580 con 0x7f1360101990 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f135ffff700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f13480a1230 0x7f13480a36f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1350005950 tx=0x7f13500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f1372b9ab80 1 Processor -- start 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f1372b9ab80 1 -- start start 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e60424f0 0x55e0e6042910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6042e50 0x55e0e60432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6043810 0x55e0e6043c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e5f99100 con 0x55e0e6042e50 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e5f96ac0 con 0x55e0e60424f0 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e5f923b0 con 0x55e0e6043810 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f135f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6042e50 0x55e0e60432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f135f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6042e50 0x55e0e60432d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55772/0 (socket says 172.21.15.12:55772) 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f135f7fe700 1 -- 172.21.15.12:0/41202862 learned_addr learned my addr 172.21.15.12:0/41202862 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.951+0000 7f135ffff700 1 --2- 172.21.15.12:0/41202862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e60424f0 0x55e0e6042910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.952+0000 7f1364f7d700 1 --2- 172.21.15.12:0/41202862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6043810 0x55e0e6043c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.952+0000 7f135f7fe700 1 -- 172.21.15.12:0/41202862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6043810 msgr2=0x55e0e6043c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.952+0000 7f135f7fe700 1 --2- 172.21.15.12:0/41202862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6043810 0x55e0e6043c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.952+0000 7f135f7fe700 1 -- 172.21.15.12:0/41202862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e60424f0 msgr2=0x55e0e6042910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.952+0000 7f135f7fe700 1 --2- 172.21.15.12:0/41202862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e60424f0 0x55e0e6042910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.952+0000 7f135f7fe700 1 -- 172.21.15.12:0/41202862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0e5e13360 con 0x55e0e6042e50 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.952+0000 7f135f7fe700 1 --2- 172.21.15.12:0/41202862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6042e50 0x55e0e60432d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1354005910 tx=0x7f1354005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78af4f8c98642b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.952+0000 7f13467fc700 1 -- 172.21.15.12:0/41202862 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135401e070 con 0x55e0e6042e50 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.952+0000 7f13467fc700 1 -- 172.21.15.12:0/41202862 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1354004cf0 con 0x55e0e6042e50 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.952+0000 7f13467fc700 1 -- 172.21.15.12:0/41202862 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354005280 con 0x55e0e6042e50 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.953+0000 7f1372b9ab80 1 -- 172.21.15.12:0/41202862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6042e50 msgr2=0x55e0e60432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.953+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/41202862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6042e50 0x55e0e60432d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1354005910 tx=0x7f1354005e80 comp rx=0 tx=0).stop 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.953+0000 7f1372b9ab80 1 -- 172.21.15.12:0/41202862 shutdown_connections 2024-05-08T23:45:56.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.953+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/41202862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6043810 0x55e0e6043c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.953+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/41202862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e60424f0 0x55e0e6042910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.953+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/41202862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6042e50 0x55e0e60432d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.953+0000 7f1372b9ab80 1 -- 172.21.15.12:0/41202862 >> 172.21.15.12:0/41202862 conn(0x55e0e5fa01c0 msgr2=0x55e0e603cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.953+0000 7f1372b9ab80 1 -- 172.21.15.12:0/41202862 shutdown_connections 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.953+0000 7f1372b9ab80 1 -- 172.21.15.12:0/41202862 wait complete. 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.953+0000 7f1372b9ab80 1 -- start start 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1372b9ab80 1 Processor -- start 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e60424f0 0x55e0e604bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 0x55e0e604c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6053210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e5f2e510 con 0x55e0e6043810 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e5f2e1b0 con 0x55e0e60424f0 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e5e15650 con 0x55e0e6042e50 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1372b9ab80 10 client.0 _ll_register_callbacks cb 0x55e0e5f37ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:45:56.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1364f7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6053210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1364f7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6053210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55786/0 (socket says 172.21.15.12:55786) 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1364f7d700 1 -- 172.21.15.12:0/2182744458 learned_addr learned my addr 172.21.15.12:0/2182744458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f135ffff700 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e60424f0 0x55e0e604bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f135f7fe700 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 0x55e0e604c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1364f7d700 1 -- 172.21.15.12:0/2182744458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 msgr2=0x55e0e604c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1364f7d700 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 0x55e0e604c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1364f7d700 1 -- 172.21.15.12:0/2182744458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e60424f0 msgr2=0x55e0e604bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1364f7d700 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e60424f0 0x55e0e604bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.954+0000 7f1364f7d700 1 -- 172.21.15.12:0/2182744458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1360108be0 con 0x55e0e6043810 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f1364f7d700 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6053210 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f13580ca060 tx=0x7f1358004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8696b32c0541a451 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f133f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f133f7fe700 1 -- 172.21.15.12:0/2182744458 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358099a00 con 0x55e0e6043810 2024-05-08T23:45:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f133f7fe700 1 -- 172.21.15.12:0/2182744458 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f135809d830 con 0x55e0e6043810 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f1372b9ab80 1 -- 172.21.15.12:0/2182744458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0e5eeb220 con 0x55e0e6043810 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f133f7fe700 1 -- 172.21.15.12:0/2182744458 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13580a1df0 con 0x55e0e6043810 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f1372b9ab80 10 client.15027 fetch_fsmap learned FSMap version 23 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f1372b9ab80 1 -- 172.21.15.12:0/2182744458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e0e5dbfb10 con 0x55e0e6043810 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f133f7fe700 1 -- 172.21.15.12:0/2182744458 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0e5eeb220 con 0x55e0e6043810 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f133f7fe700 1 -- 172.21.15.12:0/2182744458 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f13580a1960 con 0x55e0e6043810 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f1372b9ab80 10 client.15027 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f1372b9ab80 10 client.15027 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f1372b9ab80 20 client.15027 populate_metadata read hostname 'smithi012' 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e0e6056ed0 0x55e0e605c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f1372b9ab80 4 client.15027 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.955+0000 7f1372b9ab80 1 -- 172.21.15.12:0/2182744458 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0e60117c0 con 0x55e0e6056ed0 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.956+0000 7f135ffff700 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e0e6056ed0 0x55e0e605c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.956+0000 7f135ffff700 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e0e6056ed0 0x55e0e605c1f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ce303db0499942e server_cookie=5eff8ae739f40576 in_seq=0 out_seq=0 2024-05-08T23:45:56.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.956+0000 7f133f7fe700 10 client.15027 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f133f7fe700 1 -- 172.21.15.12:0/2182744458 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0e60117c0 con 0x55e0e6056ed0 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f133f7fe700 10 client.15027 handle_command_reply: tid=0 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f1372b9ab80 1 client.15027 shutdown 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f1372b9ab80 1 -- 172.21.15.12:0/2182744458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 msgr2=0x55e0e6053210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6053210 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f13580ca060 tx=0x7f1358004b50 comp rx=0 tx=0).stop 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f1372b9ab80 1 -- 172.21.15.12:0/2182744458 shutdown_connections 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6053210 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e60424f0 0x55e0e604bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e0e6056ed0 0x55e0e605c1f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/2182744458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 0x55e0e604c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f133f7fe700 0 client.15027 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.957+0000 7f1372b9ab80 1 -- 172.21.15.12:0/2182744458 >> 172.21.15.12:0/2182744458 conn(0x55e0e5fa01c0 msgr2=0x55e0e603c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:56.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.958+0000 7f1372b9ab80 1 -- 172.21.15.12:0/2182744458 shutdown_connections 2024-05-08T23:45:56.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.958+0000 7f1372b9ab80 1 -- 172.21.15.12:0/2182744458 wait complete. 2024-05-08T23:45:56.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.958+0000 7f1372b9ab80 20 client.15027 trim_cache size 0 max 16384 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f1372b9ab80 1 Processor -- start 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f1372b9ab80 1 -- start start 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e60424f0 0x55e0e60538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 0x55e0e60531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6043810 0x55e0e60466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e60117c0 con 0x55e0e60424f0 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e5eeb220 con 0x55e0e6043810 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e5f502b0 con 0x55e0e6042e50 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f135ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e60424f0 0x55e0e60538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f135ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e60424f0 0x55e0e60538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55790/0 (socket says 172.21.15.12:55790) 2024-05-08T23:45:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f135ffff700 1 -- 172.21.15.12:0/359222304 learned_addr learned my addr 172.21.15.12:0/359222304 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f135f7fe700 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 0x55e0e60531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f135ffff700 1 -- 172.21.15.12:0/359222304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 msgr2=0x55e0e60531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f135ffff700 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 0x55e0e60531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f135ffff700 1 -- 172.21.15.12:0/359222304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6043810 msgr2=0x55e0e60466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f1364f7d700 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6043810 0x55e0e60466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f135ffff700 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6043810 0x55e0e60466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f135ffff700 1 -- 172.21.15.12:0/359222304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0e5f58510 con 0x55e0e60424f0 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f135f7fe700 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 0x55e0e60531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.977+0000 7f1364f7d700 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6043810 0x55e0e60466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f135ffff700 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e60424f0 0x55e0e60538c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55e0e604bc60 tx=0x7f1350012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56205641717dd260 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1346ffd700 1 -- 172.21.15.12:0/359222304 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1350024040 con 0x55e0e60424f0 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1346ffd700 1 -- 172.21.15.12:0/359222304 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f135002a8d0 con 0x55e0e60424f0 2024-05-08T23:45:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1346ffd700 1 -- 172.21.15.12:0/359222304 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135001fb50 con 0x55e0e60424f0 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1372b9ab80 1 -- 172.21.15.12:0/359222304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e60424f0 msgr2=0x55e0e60538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e60424f0 0x55e0e60538c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55e0e604bc60 tx=0x7f1350012000 comp rx=0 tx=0).stop 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1372b9ab80 1 -- 172.21.15.12:0/359222304 shutdown_connections 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6043810 0x55e0e60466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e60424f0 0x55e0e60538c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/359222304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e6042e50 0x55e0e60531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1372b9ab80 1 -- 172.21.15.12:0/359222304 >> 172.21.15.12:0/359222304 conn(0x55e0e5fa01c0 msgr2=0x55e0e60391c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1372b9ab80 1 -- 172.21.15.12:0/359222304 shutdown_connections 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.978+0000 7f1372b9ab80 1 -- 172.21.15.12:0/359222304 wait complete. 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1372b9ab80 1 -- start start 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1372b9ab80 1 Processor -- start 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e60424f0 0x55e0e604d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6042e50 0x55e0e604d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1372b9ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6052780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e6052cc0 con 0x55e0e6043810 2024-05-08T23:45:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e6052e30 con 0x55e0e6042e50 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1372b9ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e6052ff0 con 0x55e0e60424f0 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1372b9ab80 10 client.0 _ll_register_callbacks cb 0x55e0e5f37ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1364f7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6052780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1364f7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6052780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55792/0 (socket says 172.21.15.12:55792) 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.979+0000 7f1364f7d700 1 -- 172.21.15.12:0/4194280966 learned_addr learned my addr 172.21.15.12:0/4194280966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f135ffff700 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e60424f0 0x55e0e604d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f135f7fe700 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6042e50 0x55e0e604d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f1364f7d700 1 -- 172.21.15.12:0/4194280966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e60424f0 msgr2=0x55e0e604d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f1364f7d700 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e60424f0 0x55e0e604d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f1364f7d700 1 -- 172.21.15.12:0/4194280966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6042e50 msgr2=0x55e0e604d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f1364f7d700 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6042e50 0x55e0e604d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f1364f7d700 1 -- 172.21.15.12:0/4194280966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135000f9a0 con 0x55e0e6043810 2024-05-08T23:45:56.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f135ffff700 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e60424f0 0x55e0e604d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f135f7fe700 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6042e50 0x55e0e604d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f1364f7d700 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6052780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1358095eb0 tx=0x7f135801b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce79c681ffd717b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f133f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f133f7fe700 1 -- 172.21.15.12:0/4194280966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358099a00 con 0x55e0e6043810 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f133f7fe700 1 -- 172.21.15.12:0/4194280966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f13580a1df0 con 0x55e0e6043810 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f1372b9ab80 1 -- 172.21.15.12:0/4194280966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1354004cf0 con 0x55e0e6043810 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f133f7fe700 1 -- 172.21.15.12:0/4194280966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1358098ef0 con 0x55e0e6043810 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f1372b9ab80 10 client.15033 fetch_fsmap learned FSMap version 23 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f1372b9ab80 1 -- 172.21.15.12:0/4194280966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e0e5e17c80 con 0x55e0e6043810 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.980+0000 7f133f7fe700 1 -- 172.21.15.12:0/4194280966 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1354004cf0 con 0x55e0e6043810 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.981+0000 7f133f7fe700 1 -- 172.21.15.12:0/4194280966 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1358081020 con 0x55e0e6043810 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.981+0000 7f1372b9ab80 10 client.15033 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.981+0000 7f1372b9ab80 10 client.15033 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:45:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.981+0000 7f1372b9ab80 20 client.15033 populate_metadata read hostname 'smithi012' 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.981+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e0e60b1fb0 0x55e0e60ae980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.981+0000 7f1372b9ab80 4 client.15033 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.981+0000 7f1372b9ab80 1 -- 172.21.15.12:0/4194280966 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e0e5f8ae70 con 0x55e0e60b1fb0 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.981+0000 7f135ffff700 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e0e60b1fb0 0x55e0e60ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.981+0000 7f133f7fe700 10 client.15033 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.981+0000 7f135ffff700 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e0e60b1fb0 0x55e0e60ae980 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36b2d071d05b525c server_cookie=5ca41bbb41a25c53 in_seq=0 out_seq=0 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f133f7fe700 1 -- 172.21.15.12:0/4194280966 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55e0e5f8ae70 con 0x55e0e60b1fb0 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f133f7fe700 10 client.15033 handle_command_reply: tid=0 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 client.15033 shutdown 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 -- 172.21.15.12:0/4194280966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 msgr2=0x55e0e6052780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6052780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1358095eb0 tx=0x7f135801b600 comp rx=0 tx=0).stop 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 -- 172.21.15.12:0/4194280966 shutdown_connections 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0e6043810 0x55e0e6052780 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e0e60b1fb0 0x55e0e60ae980 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0e60424f0 0x55e0e604d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 --2- 172.21.15.12:0/4194280966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0e6042e50 0x55e0e604d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 -- 172.21.15.12:0/4194280966 >> 172.21.15.12:0/4194280966 conn(0x55e0e5fa01c0 msgr2=0x55e0e604ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f133f7fe700 0 client.15033 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:45:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 -- 172.21.15.12:0/4194280966 shutdown_connections 2024-05-08T23:45:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 1 -- 172.21.15.12:0/4194280966 wait complete. 2024-05-08T23:45:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.982+0000 7f1372b9ab80 20 client.15033 trim_cache size 0 max 16384 2024-05-08T23:45:56.985 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:45:56.985 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "00f8fb28-0d94-11ef-bc97-c7b262605968", 2024-05-08T23:45:56.985 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-05-08T23:45:56.986 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24511, 2024-05-08T23:45:56.986 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-08T23:45:56.986 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-08T23:45:56.986 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-08T23:45:56.986 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 48.921215003, 2024-05-08T23:45:56.986 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 23, 2024-05-08T23:45:56.986 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-05-08T23:45:56.986 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:45:56.986 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 64.242432382000004 2024-05-08T23:45:56.986 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:45:56.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 -- 172.21.15.12:0/412510127 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f13480a1230 msgr2=0x7f13480a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f13480a1230 0x7f13480a36f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1350005950 tx=0x7f13500058e0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 -- 172.21.15.12:0/412510127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1360101990 msgr2=0x7f1360102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:56.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1360101990 0x7f1360102980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f135800ef30 tx=0x7f135800c450 comp rx=0 tx=0).stop 2024-05-08T23:45:56.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 -- 172.21.15.12:0/412510127 shutdown_connections 2024-05-08T23:45:56.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1360101990 0x7f1360102980 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f13480a1230 0x7f13480a36f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13600ffe80 0x7f1360103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 --2- 172.21.15.12:0/412510127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1360100fd0 0x7f1360102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 -- 172.21.15.12:0/412510127 >> 172.21.15.12:0/412510127 conn(0x7f13600fb3f0 msgr2=0x7f13600fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 -- 172.21.15.12:0/412510127 shutdown_connections 2024-05-08T23:45:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:56.983+0000 7f13669e0700 1 -- 172.21.15.12:0/412510127 wait complete. 2024-05-08T23:45:56.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-08T23:45:57.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.136+0000 7f050d8dc700 1 -- 172.21.15.12:0/1883600523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05081020f0 msgr2=0x7f0508102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.136+0000 7f050d8dc700 1 --2- 172.21.15.12:0/1883600523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05081020f0 0x7f0508102570 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0098f0 tx=0x7f04fc009c00 comp rx=0 tx=0).stop 2024-05-08T23:45:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.136+0000 7f050d8dc700 1 -- 172.21.15.12:0/1883600523 shutdown_connections 2024-05-08T23:45:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.136+0000 7f050d8dc700 1 --2- 172.21.15.12:0/1883600523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0508102ab0 0x7f05081093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.136+0000 7f050d8dc700 1 --2- 172.21.15.12:0/1883600523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0508100f20 0x7f0508101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.136+0000 7f050d8dc700 1 --2- 172.21.15.12:0/1883600523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05081020f0 0x7f0508102570 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.136+0000 7f050d8dc700 1 -- 172.21.15.12:0/1883600523 >> 172.21.15.12:0/1883600523 conn(0x7f05080fc4e0 msgr2=0x7f05080fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.136+0000 7f050d8dc700 1 -- 172.21.15.12:0/1883600523 shutdown_connections 2024-05-08T23:45:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.136+0000 7f050d8dc700 1 -- 172.21.15.12:0/1883600523 wait complete. 2024-05-08T23:45:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f050d8dc700 1 Processor -- start 2024-05-08T23:45:57.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f050d8dc700 1 -- start start 2024-05-08T23:45:57.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f050d8dc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0508100f20 0x7f0508074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f050d8dc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05081020f0 0x7f0508073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f050d8dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0508102ab0 0x7f05080738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f050d8dc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0508074070 con 0x7f0508102ab0 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f050d8dc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05080741e0 con 0x7f05081020f0 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f050d8dc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05081a0fe0 con 0x7f0508100f20 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f05077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0508102ab0 0x7f05080738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f05077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0508102ab0 0x7f05080738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55810/0 (socket says 172.21.15.12:55810) 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f05077fe700 1 -- 172.21.15.12:0/575469212 learned_addr learned my addr 172.21.15.12:0/575469212 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.137+0000 7f0506ffd700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0508100f20 0x7f0508074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f05067fc700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05081020f0 0x7f0508073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f05077fe700 1 -- 172.21.15.12:0/575469212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0508100f20 msgr2=0x7f0508074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f05077fe700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0508100f20 0x7f0508074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f05077fe700 1 -- 172.21.15.12:0/575469212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05081020f0 msgr2=0x7f0508073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f05077fe700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05081020f0 0x7f0508073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f05077fe700 1 -- 172.21.15.12:0/575469212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0500009d00 con 0x7f0508102ab0 2024-05-08T23:45:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f05077fe700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0508102ab0 0x7f05080738d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f050000e3c0 tx=0x7f050000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f050c8da700 1 -- 172.21.15.12:0/575469212 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05000090d0 con 0x7f0508102ab0 2024-05-08T23:45:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f050d8dc700 1 -- 172.21.15.12:0/575469212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc009520 con 0x7f0508102ab0 2024-05-08T23:45:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f050c8da700 1 -- 172.21.15.12:0/575469212 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f050000f040 con 0x7f0508102ab0 2024-05-08T23:45:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f050c8da700 1 -- 172.21.15.12:0/575469212 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0500007c20 con 0x7f0508102ab0 2024-05-08T23:45:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.138+0000 7f050d8dc700 1 -- 172.21.15.12:0/575469212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05081a15f0 con 0x7f0508102ab0 2024-05-08T23:45:57.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.140+0000 7f050c8da700 1 -- 172.21.15.12:0/575469212 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0500003a00 con 0x7f0508102ab0 2024-05-08T23:45:57.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.141+0000 7f050c8da700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f04f00a11b0 0x7f04f00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.141+0000 7f050c8da700 1 -- 172.21.15.12:0/575469212 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f05000cc790 con 0x7f0508102ab0 2024-05-08T23:45:57.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.141+0000 7f0506ffd700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f04f00a11b0 0x7f04f00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.141+0000 7f0506ffd700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f04f00a11b0 0x7f04f00a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0508100c40 tx=0x7f04f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0519a96b80 1 Processor -- start 2024-05-08T23:45:57.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0519a96b80 1 -- start start 2024-05-08T23:45:57.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c104f0 0x563686c10910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c10e50 0x563686c112d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c11810 0x563686c11c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0519a96b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686b67100 con 0x563686c104f0 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0519a96b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686b64ac0 con 0x563686c11810 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0519a96b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686b603b0 con 0x563686c10e50 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0506ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c104f0 0x563686c10910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0506ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c104f0 0x563686c10910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55822/0 (socket says 172.21.15.12:55822) 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f0506ffd700 1 -- 172.21.15.12:0/3217183387 learned_addr learned my addr 172.21.15.12:0/3217183387 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f05067fc700 1 --2- 172.21.15.12:0/3217183387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c10e50 0x563686c112d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.142+0000 7f05077fe700 1 --2- 172.21.15.12:0/3217183387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c11810 0x563686c11c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.143+0000 7f0506ffd700 1 -- 172.21.15.12:0/3217183387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c10e50 msgr2=0x563686c112d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.143+0000 7f0506ffd700 1 --2- 172.21.15.12:0/3217183387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c10e50 0x563686c112d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.143+0000 7f0506ffd700 1 -- 172.21.15.12:0/3217183387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c11810 msgr2=0x563686c11c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.143+0000 7f0506ffd700 1 --2- 172.21.15.12:0/3217183387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c11810 0x563686c11c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.143+0000 7f0506ffd700 1 -- 172.21.15.12:0/3217183387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636869e1360 con 0x563686c104f0 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.143+0000 7f05077fe700 1 --2- 172.21.15.12:0/3217183387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c11810 0x563686c11c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.143+0000 7f0506ffd700 1 --2- 172.21.15.12:0/3217183387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c104f0 0x563686c10910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f04f8011d50 tx=0x7f04f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcbb18e44bea373 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.143+0000 7f04edffb700 1 -- 172.21.15.12:0/3217183387 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f800ed20 con 0x563686c104f0 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.143+0000 7f04edffb700 1 -- 172.21.15.12:0/3217183387 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04f8021070 con 0x563686c104f0 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.143+0000 7f04edffb700 1 -- 172.21.15.12:0/3217183387 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f801d610 con 0x563686c104f0 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.144+0000 7f0519a96b80 1 -- 172.21.15.12:0/3217183387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c104f0 msgr2=0x563686c10910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.144+0000 7f0519a96b80 1 --2- 172.21.15.12:0/3217183387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c104f0 0x563686c10910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f04f8011d50 tx=0x7f04f8014000 comp rx=0 tx=0).stop 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.144+0000 7f0519a96b80 1 -- 172.21.15.12:0/3217183387 shutdown_connections 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.144+0000 7f0519a96b80 1 --2- 172.21.15.12:0/3217183387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c11810 0x563686c11c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.144+0000 7f0519a96b80 1 --2- 172.21.15.12:0/3217183387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c104f0 0x563686c10910 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.144+0000 7f0519a96b80 1 --2- 172.21.15.12:0/3217183387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c10e50 0x563686c112d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.144+0000 7f0519a96b80 1 -- 172.21.15.12:0/3217183387 >> 172.21.15.12:0/3217183387 conn(0x563686b6e1c0 msgr2=0x563686c0ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.144+0000 7f0519a96b80 1 -- 172.21.15.12:0/3217183387 shutdown_connections 2024-05-08T23:45:57.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.144+0000 7f0519a96b80 1 -- 172.21.15.12:0/3217183387 wait complete. 2024-05-08T23:45:57.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.144+0000 7f0519a96b80 1 -- start start 2024-05-08T23:45:57.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f0519a96b80 1 Processor -- start 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c104f0 0x563686c19d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c10e50 0x563686c1a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c212a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f0519a96b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686afc510 con 0x563686c11810 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f0519a96b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686afc1b0 con 0x563686c10e50 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f0519a96b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636869e3650 con 0x563686c104f0 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f0519a96b80 10 client.0 _ll_register_callbacks cb 0x563686b05ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f05077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c212a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f05077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c212a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55824/0 (socket says 172.21.15.12:55824) 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f05077fe700 1 -- 172.21.15.12:0/297619869 learned_addr learned my addr 172.21.15.12:0/297619869 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f0506ffd700 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c104f0 0x563686c19d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f05067fc700 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c10e50 0x563686c1a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f05077fe700 1 -- 172.21.15.12:0/297619869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c104f0 msgr2=0x563686c19d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f05077fe700 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c104f0 0x563686c19d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f05077fe700 1 -- 172.21.15.12:0/297619869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c10e50 msgr2=0x563686c1a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f05077fe700 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c10e50 0x563686c1a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f05077fe700 1 -- 172.21.15.12:0/297619869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f8017070 con 0x563686c11810 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.145+0000 7f05067fc700 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c10e50 0x563686c1a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f05077fe700 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c212a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0500096110 tx=0x7f050009a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd5a8ac75146e36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f04e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f04e6ffd700 1 -- 172.21.15.12:0/297619869 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05000b1070 con 0x563686c11810 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f04e6ffd700 1 -- 172.21.15.12:0/297619869 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f05000a2000 con 0x563686c11810 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f0519a96b80 1 -- 172.21.15.12:0/297619869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563686ab9220 con 0x563686c11810 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f04e6ffd700 1 -- 172.21.15.12:0/297619869 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0500012e20 con 0x563686c11810 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f0519a96b80 10 client.15045 fetch_fsmap learned FSMap version 23 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f0519a96b80 1 -- 172.21.15.12:0/297619869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56368698db10 con 0x563686c11810 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f04e6ffd700 1 -- 172.21.15.12:0/297619869 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0500063020 con 0x563686c11810 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f04e6ffd700 1 -- 172.21.15.12:0/297619869 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0500099100 con 0x563686c11810 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f0519a96b80 10 client.15045 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f0519a96b80 10 client.15045 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T23:45:57.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f0519a96b80 20 client.15045 populate_metadata read hostname 'smithi012' 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f0519a96b80 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563686c24e80 0x563686c2a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f0519a96b80 4 client.15045 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.146+0000 7f0519a96b80 1 -- 172.21.15.12:0/297619869 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563686bdf7c0 con 0x563686c24e80 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.147+0000 7f0506ffd700 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563686c24e80 0x563686c2a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.147+0000 7f0506ffd700 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563686c24e80 0x563686c2a180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e9e97d3e18ab686 server_cookie=c9ea5e08cbe0e78f in_seq=0 out_seq=0 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.147+0000 7f04e6ffd700 10 client.15045 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.148+0000 7f04e6ffd700 1 -- 172.21.15.12:0/297619869 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563686bdf7c0 con 0x563686c24e80 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.148+0000 7f04e6ffd700 10 client.15045 handle_command_reply: tid=0 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.148+0000 7f0519a96b80 1 client.15045 shutdown 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 1 -- 172.21.15.12:0/297619869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 msgr2=0x563686c212a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c212a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0500096110 tx=0x7f050009a3c0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 1 -- 172.21.15.12:0/297619869 shutdown_connections 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563686c24e80 0x563686c2a180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c212a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c104f0 0x563686c19d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f04e6ffd700 0 client.15045 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:45:57.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 1 --2- 172.21.15.12:0/297619869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c10e50 0x563686c1a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 1 -- 172.21.15.12:0/297619869 >> 172.21.15.12:0/297619869 conn(0x563686b6e1c0 msgr2=0x563686c0a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 1 -- 172.21.15.12:0/297619869 shutdown_connections 2024-05-08T23:45:57.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 1 -- 172.21.15.12:0/297619869 wait complete. 2024-05-08T23:45:57.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.149+0000 7f0519a96b80 20 client.15045 trim_cache size 0 max 16384 2024-05-08T23:45:57.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.168+0000 7f0519a96b80 1 Processor -- start 2024-05-08T23:45:57.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.168+0000 7f0519a96b80 1 -- start start 2024-05-08T23:45:57.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c104f0 0x563686c108d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c10e50 0x563686c21950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c21260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f0519a96b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686c18d60 con 0x563686c11810 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f0519a96b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686b59150 con 0x563686c104f0 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f0519a96b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686aff5a0 con 0x563686c10e50 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f05077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c21260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f05077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c21260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55826/0 (socket says 172.21.15.12:55826) 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f05077fe700 1 -- 172.21.15.12:0/3213017665 learned_addr learned my addr 172.21.15.12:0/3213017665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f0506ffd700 1 --2- 172.21.15.12:0/3213017665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c104f0 0x563686c108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f05067fc700 1 --2- 172.21.15.12:0/3213017665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c10e50 0x563686c21950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f05077fe700 1 -- 172.21.15.12:0/3213017665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c10e50 msgr2=0x563686c21950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f05077fe700 1 --2- 172.21.15.12:0/3213017665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c10e50 0x563686c21950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f05077fe700 1 -- 172.21.15.12:0/3213017665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c104f0 msgr2=0x563686c108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f05077fe700 1 --2- 172.21.15.12:0/3213017665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c104f0 0x563686c108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f05077fe700 1 -- 172.21.15.12:0/3213017665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563686b26510 con 0x563686c11810 2024-05-08T23:45:57.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f05077fe700 1 --2- 172.21.15.12:0/3213017665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c21260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0500099ba0 tx=0x7f050009dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb774888bab3362 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f04ee7fc700 1 -- 172.21.15.12:0/3213017665 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05000b1070 con 0x563686c11810 2024-05-08T23:45:57.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f04ee7fc700 1 -- 172.21.15.12:0/3213017665 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0500063070 con 0x563686c11810 2024-05-08T23:45:57.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.169+0000 7f04ee7fc700 1 -- 172.21.15.12:0/3213017665 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0500099e70 con 0x563686c11810 2024-05-08T23:45:57.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.170+0000 7f0519a96b80 1 -- 172.21.15.12:0/3213017665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 msgr2=0x563686c21260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.170+0000 7f0519a96b80 1 --2- 172.21.15.12:0/3213017665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c21260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0500099ba0 tx=0x7f050009dbc0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.170+0000 7f0519a96b80 1 -- 172.21.15.12:0/3213017665 shutdown_connections 2024-05-08T23:45:57.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.170+0000 7f0519a96b80 1 --2- 172.21.15.12:0/3213017665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c11810 0x563686c21260 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.170+0000 7f0519a96b80 1 --2- 172.21.15.12:0/3213017665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c104f0 0x563686c108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.170+0000 7f0519a96b80 1 --2- 172.21.15.12:0/3213017665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c10e50 0x563686c21950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.170+0000 7f0519a96b80 1 -- 172.21.15.12:0/3213017665 >> 172.21.15.12:0/3213017665 conn(0x563686b6e1c0 msgr2=0x563686c11bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.170+0000 7f0519a96b80 1 -- 172.21.15.12:0/3213017665 shutdown_connections 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.170+0000 7f0519a96b80 1 -- 172.21.15.12:0/3213017665 wait complete. 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f0519a96b80 1 -- start start 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f0519a96b80 1 Processor -- start 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c104f0 0x563686c09f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c10e50 0x563686c0a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f0519a96b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c11810 0x563686c20470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f0519a96b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686c209b0 con 0x563686c10e50 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f0519a96b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686c20b20 con 0x563686c104f0 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f0519a96b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563686c20ce0 con 0x563686c11810 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f0519a96b80 10 client.0 _ll_register_callbacks cb 0x563686b05ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:45:57.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f05067fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c10e50 0x563686c0a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f05067fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c10e50 0x563686c0a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55834/0 (socket says 172.21.15.12:55834) 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f05067fc700 1 -- 172.21.15.12:0/1961499698 learned_addr learned my addr 172.21.15.12:0/1961499698 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.171+0000 7f0506ffd700 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c104f0 0x563686c09f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f05077fe700 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c11810 0x563686c20470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f05067fc700 1 -- 172.21.15.12:0/1961499698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c11810 msgr2=0x563686c20470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f05067fc700 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c11810 0x563686c20470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f05067fc700 1 -- 172.21.15.12:0/1961499698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c104f0 msgr2=0x563686c09f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f05067fc700 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c104f0 0x563686c09f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f05067fc700 1 -- 172.21.15.12:0/1961499698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56368698db10 con 0x563686c10e50 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f05077fe700 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c11810 0x563686c20470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f05067fc700 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c10e50 0x563686c0a450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0098f0 tx=0x7f04fc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f24e02bdf6d49b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f04e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f04e6ffd700 1 -- 172.21.15.12:0/1961499698 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc01c040 con 0x563686c10e50 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f04e6ffd700 1 -- 172.21.15.12:0/1961499698 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f04fc0058e0 con 0x563686c10e50 2024-05-08T23:45:57.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f0519a96b80 1 -- 172.21.15.12:0/1961499698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04f8021070 con 0x563686c10e50 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f04e6ffd700 1 -- 172.21.15.12:0/1961499698 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04fc00b910 con 0x563686c10e50 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f0519a96b80 10 client.15051 fetch_fsmap learned FSMap version 23 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.172+0000 7f0519a96b80 1 -- 172.21.15.12:0/1961499698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5636869e5c80 con 0x563686c10e50 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f04e6ffd700 1 -- 172.21.15.12:0/1961499698 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04f8021070 con 0x563686c10e50 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f04e6ffd700 1 -- 172.21.15.12:0/1961499698 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f04fc015020 con 0x563686c10e50 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f0519a96b80 10 client.15051 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f0519a96b80 10 client.15051 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f0519a96b80 20 client.15051 populate_metadata read hostname 'smithi012' 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f0519a96b80 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563686c800e0 0x563686c7ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f0519a96b80 4 client.15051 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f0519a96b80 1 -- 172.21.15.12:0/1961499698 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563686b58e70 con 0x563686c800e0 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f0506ffd700 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563686c800e0 0x563686c7ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f04e6ffd700 10 client.15051 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:45:57.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.173+0000 7f0506ffd700 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563686c800e0 0x563686c7ca30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=816dc232941c4a67 server_cookie=312cfb803bb77fe in_seq=0 out_seq=0 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f04e6ffd700 1 -- 172.21.15.12:0/1961499698 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563686b58e70 con 0x563686c800e0 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f04e6ffd700 10 client.15051 handle_command_reply: tid=0 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 client.15051 shutdown 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 -- 172.21.15.12:0/1961499698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c10e50 msgr2=0x563686c0a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c10e50 0x563686c0a450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0098f0 tx=0x7f04fc00f690 comp rx=0 tx=0).stop 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 -- 172.21.15.12:0/1961499698 shutdown_connections 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563686c800e0 0x563686c7ca30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563686c11810 0x563686c20470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563686c104f0 0x563686c09f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 --2- 172.21.15.12:0/1961499698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563686c10e50 0x563686c0a450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f04e6ffd700 0 client.15051 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 -- 172.21.15.12:0/1961499698 >> 172.21.15.12:0/1961499698 conn(0x563686b6e1c0 msgr2=0x563686c146a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 -- 172.21.15.12:0/1961499698 shutdown_connections 2024-05-08T23:45:57.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 1 -- 172.21.15.12:0/1961499698 wait complete. 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.174+0000 7f0519a96b80 20 client.15051 trim_cache size 0 max 16384 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "00f8fb28-0d94-11ef-bc97-c7b262605968", 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24515, 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 50.123070116999997, 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 23, 2024-05-08T23:45:57.176 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-05-08T23:45:57.177 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:45:57.177 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 66.173904379999996 2024-05-08T23:45:57.177 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:45:57.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 -- 172.21.15.12:0/575469212 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f04f00a11b0 msgr2=0x7f04f00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f04f00a11b0 0x7f04f00a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0508100c40 tx=0x7f04f8009260 comp rx=0 tx=0).stop 2024-05-08T23:45:57.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 -- 172.21.15.12:0/575469212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0508102ab0 msgr2=0x7f05080738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0508102ab0 0x7f05080738d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f050000e3c0 tx=0x7f050000e6d0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 -- 172.21.15.12:0/575469212 shutdown_connections 2024-05-08T23:45:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0508102ab0 0x7f05080738d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0508100f20 0x7f0508074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f04f00a11b0 0x7f04f00a3670 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 --2- 172.21.15.12:0/575469212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05081020f0 0x7f0508073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 -- 172.21.15.12:0/575469212 >> 172.21.15.12:0/575469212 conn(0x7f05080fc4e0 msgr2=0x7f05081055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 -- 172.21.15.12:0/575469212 shutdown_connections 2024-05-08T23:45:57.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.175+0000 7f050d8dc700 1 -- 172.21.15.12:0/575469212 wait complete. 2024-05-08T23:45:57.184 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:45:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.326+0000 7f60df72a700 1 -- 172.21.15.12:0/1771275682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60d81019e0 msgr2=0x7f60d8103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.326+0000 7f60df72a700 1 --2- 172.21.15.12:0/1771275682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60d81019e0 0x7f60d8103dd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009990 tx=0x7f60d4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.326+0000 7f60df72a700 1 -- 172.21.15.12:0/1771275682 shutdown_connections 2024-05-08T23:45:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.326+0000 7f60df72a700 1 --2- 172.21.15.12:0/1771275682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d8106c40 0x7f60d81090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.326+0000 7f60df72a700 1 --2- 172.21.15.12:0/1771275682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60d81019e0 0x7f60d8103dd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.326+0000 7f60df72a700 1 --2- 172.21.15.12:0/1771275682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60d8104310 0x7f60d8106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.326+0000 7f60df72a700 1 -- 172.21.15.12:0/1771275682 >> 172.21.15.12:0/1771275682 conn(0x7f60d80fb3f0 msgr2=0x7f60d80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.326+0000 7f60df72a700 1 -- 172.21.15.12:0/1771275682 shutdown_connections 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.326+0000 7f60df72a700 1 -- 172.21.15.12:0/1771275682 wait complete. 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60df72a700 1 Processor -- start 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60df72a700 1 -- start start 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60df72a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60d81019e0 0x7f60d8101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60df72a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d8104310 0x7f60d80ff990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60df72a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60d8106c40 0x7f60d80ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60df72a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8100670 con 0x7f60d81019e0 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60df72a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81007e0 con 0x7f60d8106c40 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60df72a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81a0ed0 con 0x7f60d8104310 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60dd4c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60d81019e0 0x7f60d8101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60dd4c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60d81019e0 0x7f60d8101300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55856/0 (socket says 172.21.15.12:55856) 2024-05-08T23:45:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60dd4c6700 1 -- 172.21.15.12:0/2568807579 learned_addr learned my addr 172.21.15.12:0/2568807579 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60dccc5700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d8104310 0x7f60d80ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60ddcc7700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60d8106c40 0x7f60d80ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60dd4c6700 1 -- 172.21.15.12:0/2568807579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d8104310 msgr2=0x7f60d80ff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d8104310 0x7f60d80ff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60dd4c6700 1 -- 172.21.15.12:0/2568807579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60d8106c40 msgr2=0x7f60d80ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60d8106c40 0x7f60d80ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.327+0000 7f60dd4c6700 1 -- 172.21.15.12:0/2568807579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d40095f0 con 0x7f60d81019e0 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.328+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60d81019e0 0x7f60d8101300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f60d4000c00 tx=0x7f60d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.328+0000 7f60ce7fc700 1 -- 172.21.15.12:0/2568807579 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d401d070 con 0x7f60d81019e0 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.328+0000 7f60df72a700 1 -- 172.21.15.12:0/2568807579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d81a1100 con 0x7f60d81019e0 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.328+0000 7f60ce7fc700 1 -- 172.21.15.12:0/2568807579 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60d40054f0 con 0x7f60d81019e0 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.328+0000 7f60ce7fc700 1 -- 172.21.15.12:0/2568807579 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4004bd0 con 0x7f60d81019e0 2024-05-08T23:45:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.328+0000 7f60df72a700 1 -- 172.21.15.12:0/2568807579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d81a1470 con 0x7f60d81019e0 2024-05-08T23:45:57.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.330+0000 7f60ce7fc700 1 -- 172.21.15.12:0/2568807579 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f60d400fb90 con 0x7f60d81019e0 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.331+0000 7f60ce7fc700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f60c40a11b0 0x7f60c40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.331+0000 7f60dccc5700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f60c40a11b0 0x7f60c40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.331+0000 7f60ce7fc700 1 -- 172.21.15.12:0/2568807579 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f60d40cdbb0 con 0x7f60d81019e0 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.331+0000 7f60dccc5700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f60c40a11b0 0x7f60c40a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009c60 tx=0x7f60c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60eb8e4b80 1 Processor -- start 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60eb8e4b80 1 -- start start 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2f4f0 0x564a8cb2f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2fe50 0x564a8cb302d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 0x564a8cb30c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ca86100 con 0x564a8cb2fe50 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ca83ac0 con 0x564a8cb30810 2024-05-08T23:45:57.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ca7f3b0 con 0x564a8cb2f4f0 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60dccc5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2fe50 0x564a8cb302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60dccc5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2fe50 0x564a8cb302d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55864/0 (socket says 172.21.15.12:55864) 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60dccc5700 1 -- 172.21.15.12:0/1751948941 learned_addr learned my addr 172.21.15.12:0/1751948941 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/1751948941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2f4f0 0x564a8cb2f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60ddcc7700 1 --2- 172.21.15.12:0/1751948941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 0x564a8cb30c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60dccc5700 1 -- 172.21.15.12:0/1751948941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2f4f0 msgr2=0x564a8cb2f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60dccc5700 1 --2- 172.21.15.12:0/1751948941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2f4f0 0x564a8cb2f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60dccc5700 1 -- 172.21.15.12:0/1751948941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 msgr2=0x564a8cb30c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60dccc5700 1 --2- 172.21.15.12:0/1751948941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 0x564a8cb30c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.332+0000 7f60dccc5700 1 -- 172.21.15.12:0/1751948941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a8c900360 con 0x564a8cb2fe50 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.333+0000 7f60dccc5700 1 --2- 172.21.15.12:0/1751948941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2fe50 0x564a8cb302d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f60c800e4e0 tx=0x7f60c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde5691e491dce4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.333+0000 7f60c37fe700 1 -- 172.21.15.12:0/1751948941 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800ee00 con 0x564a8cb2fe50 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.333+0000 7f60c37fe700 1 -- 172.21.15.12:0/1751948941 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60c8021070 con 0x564a8cb2fe50 2024-05-08T23:45:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.333+0000 7f60c37fe700 1 -- 172.21.15.12:0/1751948941 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c801d830 con 0x564a8cb2fe50 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/1751948941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2fe50 msgr2=0x564a8cb302d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/1751948941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2fe50 0x564a8cb302d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f60c800e4e0 tx=0x7f60c8014000 comp rx=0 tx=0).stop 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/1751948941 shutdown_connections 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/1751948941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 0x564a8cb30c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/1751948941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2f4f0 0x564a8cb2f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/1751948941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2fe50 0x564a8cb302d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/1751948941 >> 172.21.15.12:0/1751948941 conn(0x564a8ca8d1c0 msgr2=0x564a8cb29d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/1751948941 shutdown_connections 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/1751948941 wait complete. 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 -- start start 2024-05-08T23:45:57.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.334+0000 7f60eb8e4b80 1 Processor -- start 2024-05-08T23:45:57.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2f4f0 0x564a8cb38de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 0x564a8cb39320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 0x564a8cb40360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ca1b510 con 0x564a8cb2f4f0 2024-05-08T23:45:57.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ca1b1b0 con 0x564a8cb30810 2024-05-08T23:45:57.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8c902650 con 0x564a8cb2fe50 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60eb8e4b80 10 client.0 _ll_register_callbacks cb 0x564a8ca24ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60dd4c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2f4f0 0x564a8cb38de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60dd4c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2f4f0 0x564a8cb38de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55868/0 (socket says 172.21.15.12:55868) 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60dd4c6700 1 -- 172.21.15.12:0/2309584485 learned_addr learned my addr 172.21.15.12:0/2309584485 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60dccc5700 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 0x564a8cb39320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60ddcc7700 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 0x564a8cb40360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60dd4c6700 1 -- 172.21.15.12:0/2309584485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 msgr2=0x564a8cb39320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 0x564a8cb39320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60dd4c6700 1 -- 172.21.15.12:0/2309584485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 msgr2=0x564a8cb40360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 0x564a8cb40360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60dd4c6700 1 -- 172.21.15.12:0/2309584485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c8017070 con 0x564a8cb2f4f0 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60ddcc7700 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 0x564a8cb40360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:45:57.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2f4f0 0x564a8cb38de0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f60d4005cb0 tx=0x7f60d409b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ae718c4888727a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:45:57.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60c0ff9700 1 -- 172.21.15.12:0/2309584485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d40a3190 con 0x564a8cb2f4f0 2024-05-08T23:45:57.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60c0ff9700 1 -- 172.21.15.12:0/2309584485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f60d409b030 con 0x564a8cb2f4f0 2024-05-08T23:45:57.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.335+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/2309584485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a8c9d8220 con 0x564a8cb2f4f0 2024-05-08T23:45:57.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60c0ff9700 1 -- 172.21.15.12:0/2309584485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60d409ee60 con 0x564a8cb2f4f0 2024-05-08T23:45:57.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60eb8e4b80 10 client.15063 fetch_fsmap learned FSMap version 23 2024-05-08T23:45:57.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/2309584485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564a8c8acb10 con 0x564a8cb2f4f0 2024-05-08T23:45:57.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60c0ff9700 1 -- 172.21.15.12:0/2309584485 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60d40b9030 con 0x564a8cb2f4f0 2024-05-08T23:45:57.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60c0ff9700 1 -- 172.21.15.12:0/2309584485 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f60d409a190 con 0x564a8cb2f4f0 2024-05-08T23:45:57.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60eb8e4b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:45:57.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60eb8e4b80 10 client.15063 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:45:57.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60eb8e4b80 20 client.15063 populate_metadata read hostname 'smithi012' 2024-05-08T23:45:57.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8cb43c50 0x564a8cb48f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60eb8e4b80 4 client.15063 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:45:57.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.336+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/2309584485 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a8cafe7c0 con 0x564a8cb43c50 2024-05-08T23:45:57.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.337+0000 7f60dccc5700 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8cb43c50 0x564a8cb48f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.337+0000 7f60dccc5700 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8cb43c50 0x564a8cb48f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d24de434a6736bf server_cookie=66c103d178c100b4 in_seq=0 out_seq=0 2024-05-08T23:45:57.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.337+0000 7f60c0ff9700 10 client.15063 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:45:57.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60c0ff9700 1 -- 172.21.15.12:0/2309584485 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a8cafe7c0 con 0x564a8cb43c50 2024-05-08T23:45:57.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60c0ff9700 10 client.15063 handle_command_reply: tid=0 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 client.15063 shutdown 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/2309584485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2f4f0 msgr2=0x564a8cb38de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2f4f0 0x564a8cb38de0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f60d4005cb0 tx=0x7f60d409b570 comp rx=0 tx=0).stop 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/2309584485 shutdown_connections 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8cb43c50 0x564a8cb48f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb30810 0x564a8cb40360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb2f4f0 0x564a8cb38de0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60c0ff9700 0 client.15063 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/2309584485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 0x564a8cb39320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/2309584485 >> 172.21.15.12:0/2309584485 conn(0x564a8ca8d1c0 msgr2=0x564a8cb29550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/2309584485 shutdown_connections 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/2309584485 wait complete. 2024-05-08T23:45:57.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.338+0000 7f60eb8e4b80 20 client.15063 trim_cache size 0 max 16384 2024-05-08T23:45:57.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.357+0000 7f60eb8e4b80 1 Processor -- start 2024-05-08T23:45:57.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.357+0000 7f60eb8e4b80 1 -- start start 2024-05-08T23:45:57.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2f4f0 0x564a8cb36dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2fe50 0x564a8cb40320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 0x564a8cb382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8cb39ce0 con 0x564a8cb30810 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8cafe7c0 con 0x564a8cb2fe50 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ca78150 con 0x564a8cb2f4f0 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60ddcc7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 0x564a8cb382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60ddcc7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 0x564a8cb382f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55874/0 (socket says 172.21.15.12:55874) 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60ddcc7700 1 -- 172.21.15.12:0/744318646 learned_addr learned my addr 172.21.15.12:0/744318646 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/744318646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2f4f0 0x564a8cb36dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60dccc5700 1 --2- 172.21.15.12:0/744318646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2fe50 0x564a8cb40320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60ddcc7700 1 -- 172.21.15.12:0/744318646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2f4f0 msgr2=0x564a8cb36dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60ddcc7700 1 --2- 172.21.15.12:0/744318646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2f4f0 0x564a8cb36dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60ddcc7700 1 -- 172.21.15.12:0/744318646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2fe50 msgr2=0x564a8cb40320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60ddcc7700 1 --2- 172.21.15.12:0/744318646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2fe50 0x564a8cb40320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60ddcc7700 1 -- 172.21.15.12:0/744318646 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a8ca45510 con 0x564a8cb30810 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60ddcc7700 1 --2- 172.21.15.12:0/744318646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 0x564a8cb382f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f60d00099f0 tx=0x7f60d0009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272212cebf79c6a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60c3fff700 1 -- 172.21.15.12:0/744318646 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d00048c0 con 0x564a8cb30810 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60c3fff700 1 -- 172.21.15.12:0/744318646 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60d000f040 con 0x564a8cb30810 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.358+0000 7f60c3fff700 1 -- 172.21.15.12:0/744318646 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d00167e0 con 0x564a8cb30810 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.359+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/744318646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 msgr2=0x564a8cb382f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.359+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/744318646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 0x564a8cb382f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f60d00099f0 tx=0x7f60d0009d00 comp rx=0 tx=0).stop 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.359+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/744318646 shutdown_connections 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.359+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/744318646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 0x564a8cb382f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.359+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/744318646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2f4f0 0x564a8cb36dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.359+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/744318646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2fe50 0x564a8cb40320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.359+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/744318646 >> 172.21.15.12:0/744318646 conn(0x564a8ca8d1c0 msgr2=0x564a8cb30bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.359+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/744318646 shutdown_connections 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.359+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/744318646 wait complete. 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.359+0000 7f60eb8e4b80 1 -- start start 2024-05-08T23:45:57.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60eb8e4b80 1 Processor -- start 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2f4f0 0x564a8cb3a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 0x564a8cb3ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60eb8e4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 0x564a8cb3fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8cb40330 con 0x564a8cb30810 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8cb404a0 con 0x564a8cb2f4f0 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60eb8e4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8cb40660 con 0x564a8cb2fe50 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60eb8e4b80 10 client.0 _ll_register_callbacks cb 0x564a8ca24ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60dd4c6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2f4f0 0x564a8cb3a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60dd4c6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2f4f0 0x564a8cb3a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39832/0 (socket says 172.21.15.12:39832) 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60dd4c6700 1 -- 172.21.15.12:0/3277388569 learned_addr learned my addr 172.21.15.12:0/3277388569 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:57.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60ddcc7700 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 0x564a8cb3fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.360+0000 7f60dccc5700 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 0x564a8cb3ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.361+0000 7f60dd4c6700 1 -- 172.21.15.12:0/3277388569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 msgr2=0x564a8cb3ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.361+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 0x564a8cb3ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.361+0000 7f60dd4c6700 1 -- 172.21.15.12:0/3277388569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 msgr2=0x564a8cb3fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:45:57.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.361+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 0x564a8cb3fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.361+0000 7f60dd4c6700 1 -- 172.21.15.12:0/3277388569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d0003da0 con 0x564a8cb2f4f0 2024-05-08T23:45:57.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.361+0000 7f60dccc5700 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 0x564a8cb3ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:57.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.361+0000 7f60dd4c6700 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2f4f0 0x564a8cb3a8d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60d4006010 tx=0x7f60d40b9460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4863cd040119a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:57.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.361+0000 7f60c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:45:57.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.364+0000 7f60c0ff9700 1 -- 172.21.15.12:0/3277388569 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4004e30 con 0x564a8cb2f4f0 2024-05-08T23:45:57.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.364+0000 7f60c0ff9700 1 -- 172.21.15.12:0/3277388569 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f60d409b030 con 0x564a8cb2f4f0 2024-05-08T23:45:57.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.364+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/3277388569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60c8021070 con 0x564a8cb2f4f0 2024-05-08T23:45:57.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60c0ff9700 1 -- 172.21.15.12:0/3277388569 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60d409ee60 con 0x564a8cb2f4f0 2024-05-08T23:45:57.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60eb8e4b80 10 client.24959 fetch_fsmap learned FSMap version 23 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/3277388569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564a8c904c80 con 0x564a8cb2f4f0 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60c0ff9700 1 -- 172.21.15.12:0/3277388569 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60c8021070 con 0x564a8cb2f4f0 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60c0ff9700 1 -- 172.21.15.12:0/3277388569 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f60d409a0b0 con 0x564a8cb2f4f0 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60eb8e4b80 10 client.24959 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60eb8e4b80 10 client.24959 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60eb8e4b80 20 client.24959 populate_metadata read hostname 'smithi012' 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8cb9f1e0 0x564a8cb9bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60eb8e4b80 4 client.24959 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.370+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/3277388569 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564a8ca77e70 con 0x564a8cb9f1e0 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60dccc5700 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8cb9f1e0 0x564a8cb9bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60c0ff9700 10 client.24959 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60dccc5700 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8cb9f1e0 0x564a8cb9bbf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862898660dba94a4 server_cookie=6a12f0e15abebabc in_seq=0 out_seq=0 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60c0ff9700 1 -- 172.21.15.12:0/3277388569 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564a8ca77e70 con 0x564a8cb9f1e0 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60c0ff9700 10 client.24959 handle_command_reply: tid=0 2024-05-08T23:45:57.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60eb8e4b80 1 client.24959 shutdown 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/3277388569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2f4f0 msgr2=0x564a8cb3a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2f4f0 0x564a8cb3a8d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60d4006010 tx=0x7f60d40b9460 comp rx=0 tx=0).stop 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/3277388569 shutdown_connections 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8cb30810 0x564a8cb3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8cb9f1e0 0x564a8cb9bbf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8cb2f4f0 0x564a8cb3a8d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60eb8e4b80 1 --2- 172.21.15.12:0/3277388569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8cb2fe50 0x564a8cb3ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60c0ff9700 0 client.24959 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.371+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/3277388569 >> 172.21.15.12:0/3277388569 conn(0x564a8ca8d1c0 msgr2=0x564a8cb382f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/3277388569 shutdown_connections 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60eb8e4b80 1 -- 172.21.15.12:0/3277388569 wait complete. 2024-05-08T23:45:57.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60eb8e4b80 20 client.24959 trim_cache size 0 max 16384 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "00f8fb28-0d94-11ef-bc97-c7b262605968", 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24548, 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 53.341589057999997, 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 23, 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-05-08T23:45:57.370 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:45:57.371 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 59.039938364000001 2024-05-08T23:45:57.371 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:45:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 -- 172.21.15.12:0/2568807579 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f60c40a11b0 msgr2=0x7f60c40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f60c40a11b0 0x7f60c40a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009c60 tx=0x7f60c8009260 comp rx=0 tx=0).stop 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 -- 172.21.15.12:0/2568807579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60d81019e0 msgr2=0x7f60d8101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60d81019e0 0x7f60d8101300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f60d4000c00 tx=0x7f60d4004730 comp rx=0 tx=0).stop 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 -- 172.21.15.12:0/2568807579 shutdown_connections 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f60c40a11b0 0x7f60c40a3670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60d8106c40 0x7f60d80ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60d81019e0 0x7f60d8101300 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 --2- 172.21.15.12:0/2568807579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d8104310 0x7f60d80ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 -- 172.21.15.12:0/2568807579 >> 172.21.15.12:0/2568807579 conn(0x7f60d80fb3f0 msgr2=0x7f60d8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 -- 172.21.15.12:0/2568807579 shutdown_connections 2024-05-08T23:45:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:57.372+0000 7f60df72a700 1 -- 172.21.15.12:0/2568807579 wait complete. 2024-05-08T23:45:57.382 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T23:45:57.382 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T23:45:57.392 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-08T23:45:57.392 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T23:45:57.640 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:45:57.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:57 smithi012 ceph-mon[42503]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T23:45:57.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:57 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/516800876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:57.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:57 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/316909820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:57.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:57 smithi204 ceph-mon[47373]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T23:45:57.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:57 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/516800876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:57.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:57 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/316909820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:57.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:57 smithi160 ceph-mon[46152]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T23:45:57.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:57 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/516800876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:45:57.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:57 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/316909820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:45:58.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:45:58 smithi012 ceph-mon[42503]: pgmap v254: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:45:58.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.731+0000 7f065ff49700 1 -- 172.21.15.12:0/334473519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06580ff880 msgr2=0x7f06580ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:58.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.731+0000 7f065ff49700 1 --2- 172.21.15.12:0/334473519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06580ff880 0x7f06580ffca0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f06540098f0 tx=0x7f0654009c00 comp rx=0 tx=0).stop 2024-05-08T23:45:58.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.731+0000 7f065ff49700 1 -- 172.21.15.12:0/334473519 shutdown_connections 2024-05-08T23:45:58.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.731+0000 7f065ff49700 1 --2- 172.21.15.12:0/334473519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0658100ba0 0x7f0658101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:58.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.731+0000 7f065ff49700 1 --2- 172.21.15.12:0/334473519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06580ff880 0x7f06580ffca0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:58.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.731+0000 7f065ff49700 1 --2- 172.21.15.12:0/334473519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06581001e0 0x7f0658100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.731+0000 7f065ff49700 1 -- 172.21.15.12:0/334473519 >> 172.21.15.12:0/334473519 conn(0x7f06580fb4a0 msgr2=0x7f06580fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.731+0000 7f065ff49700 1 -- 172.21.15.12:0/334473519 shutdown_connections 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.731+0000 7f065ff49700 1 -- 172.21.15.12:0/334473519 wait complete. 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065ff49700 1 Processor -- start 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065ff49700 1 -- start start 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065ff49700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06580ff880 0x7f0658199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065ff49700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06581001e0 0x7f06581996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065ff49700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0658100ba0 0x7f06581a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065ff49700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06581a0ca0 con 0x7f06580ff880 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065ff49700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06581a0de0 con 0x7f0658100ba0 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065ff49700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06581a0f70 con 0x7f06581001e0 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065d4e4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06581001e0 0x7f06581996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:58.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065e4e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0658100ba0 0x7f06581a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:58.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065e4e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0658100ba0 0x7f06581a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:35932/0 (socket says 172.21.15.12:35932) 2024-05-08T23:45:58.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065d4e4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06581001e0 0x7f06581996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48012/0 (socket says 172.21.15.12:48012) 2024-05-08T23:45:58.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.732+0000 7f065e4e6700 1 -- 172.21.15.12:0/573018185 learned_addr learned my addr 172.21.15.12:0/573018185 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:45:58.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f065dce5700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06580ff880 0x7f0658199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:58.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f065e4e6700 1 -- 172.21.15.12:0/573018185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06581001e0 msgr2=0x7f06581996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:58.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f065e4e6700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06581001e0 0x7f06581996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:58.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f065e4e6700 1 -- 172.21.15.12:0/573018185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06580ff880 msgr2=0x7f0658199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:58.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f065e4e6700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06580ff880 0x7f0658199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:58.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f065e4e6700 1 -- 172.21.15.12:0/573018185 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0654009520 con 0x7f0658100ba0 2024-05-08T23:45:58.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f065d4e4700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06581001e0 0x7f06581996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:45:58.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f065e4e6700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0658100ba0 0x7f06581a06d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f065000ef30 tx=0x7f065000c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:58.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f064effd700 1 -- 172.21.15.12:0/573018185 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650018070 con 0x7f0658100ba0 2024-05-08T23:45:58.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f064effd700 1 -- 172.21.15.12:0/573018185 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f065000f040 con 0x7f0658100ba0 2024-05-08T23:45:58.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f065ff49700 1 -- 172.21.15.12:0/573018185 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06581a11f0 con 0x7f0658100ba0 2024-05-08T23:45:58.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f064effd700 1 -- 172.21.15.12:0/573018185 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065000cda0 con 0x7f0658100ba0 2024-05-08T23:45:58.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.733+0000 7f065ff49700 1 -- 172.21.15.12:0/573018185 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06581a16f0 con 0x7f0658100ba0 2024-05-08T23:45:58.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.734+0000 7f065ff49700 1 -- 172.21.15.12:0/573018185 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065810a660 con 0x7f0658100ba0 2024-05-08T23:45:58.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.735+0000 7f064effd700 1 -- 172.21.15.12:0/573018185 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0650003a00 con 0x7f0658100ba0 2024-05-08T23:45:58.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.736+0000 7f064effd700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06440a1230 0x7f06440a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:45:58.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.736+0000 7f065dce5700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06440a1230 0x7f06440a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:45:58.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.736+0000 7f064effd700 1 -- 172.21.15.12:0/573018185 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f06500cc790 con 0x7f0658100ba0 2024-05-08T23:45:58.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.737+0000 7f065dce5700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06440a1230 0x7f06440a36f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f065400fa40 tx=0x7f0654019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:45:58.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.740+0000 7f064effd700 1 -- 172.21.15.12:0/573018185 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0650095e50 con 0x7f0658100ba0 2024-05-08T23:45:58.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:45:58 smithi204 ceph-mon[47373]: pgmap v254: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:45:58.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:45:58 smithi160 ceph-mon[46152]: pgmap v254: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:45:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.969+0000 7f065ff49700 1 -- 172.21.15.12:0/573018185 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f065802ce10 con 0x7f06440a1230 2024-05-08T23:45:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.970+0000 7f064effd700 1 -- 172.21.15.12:0/573018185 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f065802ce10 con 0x7f06440a1230 2024-05-08T23:45:58.966 INFO:teuthology.orchestra.run.smithi012.stdout:Backend: cephadm 2024-05-08T23:45:58.966 INFO:teuthology.orchestra.run.smithi012.stdout:Available: Yes 2024-05-08T23:45:58.966 INFO:teuthology.orchestra.run.smithi012.stdout:Paused: No 2024-05-08T23:45:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.972+0000 7f065ff49700 1 -- 172.21.15.12:0/573018185 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06440a1230 msgr2=0x7f06440a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.972+0000 7f065ff49700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06440a1230 0x7f06440a36f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f065400fa40 tx=0x7f0654019040 comp rx=0 tx=0).stop 2024-05-08T23:45:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.972+0000 7f065ff49700 1 -- 172.21.15.12:0/573018185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0658100ba0 msgr2=0x7f06581a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:45:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.972+0000 7f065ff49700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0658100ba0 0x7f06581a06d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f065000ef30 tx=0x7f065000c450 comp rx=0 tx=0).stop 2024-05-08T23:45:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.972+0000 7f065ff49700 1 -- 172.21.15.12:0/573018185 shutdown_connections 2024-05-08T23:45:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.972+0000 7f065ff49700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f06440a1230 0x7f06440a36f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.972+0000 7f065ff49700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0658100ba0 0x7f06581a06d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.972+0000 7f065ff49700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06580ff880 0x7f0658199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.972+0000 7f065ff49700 1 --2- 172.21.15.12:0/573018185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06581001e0 0x7f06581996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:45:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.972+0000 7f065ff49700 1 -- 172.21.15.12:0/573018185 >> 172.21.15.12:0/573018185 conn(0x7f06580fb4a0 msgr2=0x7f06580fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:45:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.973+0000 7f065ff49700 1 -- 172.21.15.12:0/573018185 shutdown_connections 2024-05-08T23:45:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:45:58.973+0000 7f065ff49700 1 -- 172.21.15.12:0/573018185 wait complete. 2024-05-08T23:45:59.376 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T23:45:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:46:00.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.663+0000 7fa08f76e700 1 -- 172.21.15.12:0/15139243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa088102d00 msgr2=0x7fa0881095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:00.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.663+0000 7fa08f76e700 1 --2- 172.21.15.12:0/15139243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa088102d00 0x7fa0881095b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa08000b0a0 tx=0x7fa08000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.663+0000 7fa08f76e700 1 -- 172.21.15.12:0/15139243 shutdown_connections 2024-05-08T23:46:00.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.663+0000 7fa08f76e700 1 --2- 172.21.15.12:0/15139243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa088102d00 0x7fa0881095b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.663+0000 7fa08f76e700 1 --2- 172.21.15.12:0/15139243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0880ff2e0 0x7fa0880ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.663+0000 7fa08f76e700 1 --2- 172.21.15.12:0/15139243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa088102360 0x7fa0881027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.663+0000 7fa08f76e700 1 -- 172.21.15.12:0/15139243 >> 172.21.15.12:0/15139243 conn(0x7fa0880fac00 msgr2=0x7fa0880fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:00.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.663+0000 7fa08f76e700 1 -- 172.21.15.12:0/15139243 shutdown_connections 2024-05-08T23:46:00.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.663+0000 7fa08f76e700 1 -- 172.21.15.12:0/15139243 wait complete. 2024-05-08T23:46:00.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08f76e700 1 Processor -- start 2024-05-08T23:46:00.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08f76e700 1 -- start start 2024-05-08T23:46:00.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08f76e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0880ff2e0 0x7fa088101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:00.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08f76e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa088102360 0x7fa088100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:00.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08f76e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa088102d00 0x7fa088100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:00.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08f76e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa088100ff0 con 0x7fa088102d00 2024-05-08T23:46:00.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08f76e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa088101160 con 0x7fa088102360 2024-05-08T23:46:00.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08f76e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0881a1030 con 0x7fa0880ff2e0 2024-05-08T23:46:00.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08dd0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa088102d00 0x7fa088100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:00.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08dd0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa088102d00 0x7fa088100850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43668/0 (socket says 172.21.15.12:43668) 2024-05-08T23:46:00.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08dd0b700 1 -- 172.21.15.12:0/109890970 learned_addr learned my addr 172.21.15.12:0/109890970 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:00.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08cd09700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa088102360 0x7fa088100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:00.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08d50a700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0880ff2e0 0x7fa088101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:00.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08dd0b700 1 -- 172.21.15.12:0/109890970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0880ff2e0 msgr2=0x7fa088101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:00.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08dd0b700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0880ff2e0 0x7fa088101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08dd0b700 1 -- 172.21.15.12:0/109890970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa088102360 msgr2=0x7fa088100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:00.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08dd0b700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa088102360 0x7fa088100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08dd0b700 1 -- 172.21.15.12:0/109890970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa080009d00 con 0x7fa088102d00 2024-05-08T23:46:00.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.664+0000 7fa08d50a700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0880ff2e0 0x7fa088101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:00.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.665+0000 7fa08dd0b700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa088102d00 0x7fa088100850 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa080004d00 tx=0x7fa080004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:00.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.665+0000 7fa07e7fc700 1 -- 172.21.15.12:0/109890970 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08000e040 con 0x7fa088102d00 2024-05-08T23:46:00.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.665+0000 7fa07e7fc700 1 -- 172.21.15.12:0/109890970 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa08000cec0 con 0x7fa088102d00 2024-05-08T23:46:00.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.665+0000 7fa08f76e700 1 -- 172.21.15.12:0/109890970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0881a1260 con 0x7fa088102d00 2024-05-08T23:46:00.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.665+0000 7fa07e7fc700 1 -- 172.21.15.12:0/109890970 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08001ba80 con 0x7fa088102d00 2024-05-08T23:46:00.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.665+0000 7fa08f76e700 1 -- 172.21.15.12:0/109890970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0881a16d0 con 0x7fa088102d00 2024-05-08T23:46:00.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.666+0000 7fa08f76e700 1 -- 172.21.15.12:0/109890970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0881085a0 con 0x7fa088102d00 2024-05-08T23:46:00.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.667+0000 7fa07e7fc700 1 -- 172.21.15.12:0/109890970 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa08001bbe0 con 0x7fa088102d00 2024-05-08T23:46:00.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.668+0000 7fa07e7fc700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa0740a11b0 0x7fa0740a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:00.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.668+0000 7fa08d50a700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa0740a11b0 0x7fa0740a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:00.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.668+0000 7fa07e7fc700 1 -- 172.21.15.12:0/109890970 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa0800ce710 con 0x7fa088102d00 2024-05-08T23:46:00.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.669+0000 7fa08d50a700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa0740a11b0 0x7fa0740a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa084005950 tx=0x7fa0840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:00.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:00 smithi012 ceph-mon[42503]: from='client.24965 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:00.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:00 smithi012 ceph-mon[42503]: pgmap v255: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T23:46:00.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.674+0000 7fa07e7fc700 1 -- 172.21.15.12:0/109890970 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa080097ed0 con 0x7fa088102d00 2024-05-08T23:46:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:00 smithi204 ceph-mon[47373]: from='client.24965 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:00 smithi204 ceph-mon[47373]: pgmap v255: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T23:46:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:00 smithi160 ceph-mon[46152]: from='client.24965 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:00 smithi160 ceph-mon[46152]: pgmap v255: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T23:46:00.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.896+0000 7fa08f76e700 1 -- 172.21.15.12:0/109890970 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0880613e0 con 0x7fa0740a11b0 2024-05-08T23:46:00.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.911+0000 7fa07e7fc700 1 -- 172.21.15.12:0/109890970 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fa0880613e0 con 0x7fa0740a11b0 2024-05-08T23:46:00.907 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T23:46:00.907 INFO:teuthology.orchestra.run.smithi012.stdout:mds.a smithi012 running (64s) 59s ago 64s 16.6M - 18.2.2-1322-gd21e4f96 af23193ff691 15146ce08466 2024-05-08T23:46:00.907 INFO:teuthology.orchestra.run.smithi012.stdout:mds.b smithi160 running (62s) 59s ago 62s 15.7M - 18.2.2-1322-gd21e4f96 af23193ff691 55b25d727a85 2024-05-08T23:46:00.907 INFO:teuthology.orchestra.run.smithi012.stdout:mds.c smithi204 running (72s) 59s ago 71s 14.3M - 18.2.2-1322-gd21e4f96 af23193ff691 5cd71f56cfae 2024-05-08T23:46:00.907 INFO:teuthology.orchestra.run.smithi012.stdout:mds.d smithi012 running (77s) 59s ago 76s 15.7M - 18.2.2-1322-gd21e4f96 af23193ff691 db2833e70de3 2024-05-08T23:46:00.907 INFO:teuthology.orchestra.run.smithi012.stdout:mds.e smithi160 running (75s) 59s ago 75s 14.3M - 18.2.2-1322-gd21e4f96 af23193ff691 7580a00f604c 2024-05-08T23:46:00.907 INFO:teuthology.orchestra.run.smithi012.stdout:mds.f smithi204 running (73s) 59s ago 73s 13.3M - 18.2.2-1322-gd21e4f96 af23193ff691 21dc30b5d5a0 2024-05-08T23:46:00.907 INFO:teuthology.orchestra.run.smithi012.stdout:mds.g smithi012 running (66s) 59s ago 66s 15.7M - 18.2.2-1322-gd21e4f96 af23193ff691 95af58eef192 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mds.h smithi160 running (70s) 59s ago 69s 14.4M - 18.2.2-1322-gd21e4f96 af23193ff691 469e43844c72 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mds.i smithi204 running (68s) 59s ago 68s 14.1M - 18.2.2-1322-gd21e4f96 af23193ff691 ed6d96846bcb 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mds.j smithi012 running (79s) 59s ago 79s 14.0M - 18.2.2-1322-gd21e4f96 af23193ff691 4770fff46bb9 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mds.k smithi160 running (81s) 59s ago 80s 14.4M - 18.2.2-1322-gd21e4f96 af23193ff691 d6bec3a8ee1f 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mds.l smithi204 running (82s) 59s ago 82s 14.2M - 18.2.2-1322-gd21e4f96 af23193ff691 a3b3ac03cee1 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.x smithi012 *:9283,8765 running (6m) 59s ago 6m 524M - 18.2.2-1322-gd21e4f96 af23193ff691 572a774ebb79 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.y smithi160 *:8443,8765 running (5m) 59s ago 5m 460M - 18.2.2-1322-gd21e4f96 af23193ff691 c45bed0283f3 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.z smithi204 *:8443,8765 running (5m) 59s ago 5m 460M - 18.2.2-1322-gd21e4f96 af23193ff691 6d491f20c8cf 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a smithi012 running (6m) 59s ago 6m 80.4M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 1067817c4ecf 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b smithi160 running (5m) 59s ago 5m 55.4M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 f2a6c0818020 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c smithi204 running (5m) 59s ago 5m 57.7M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 da0fcfb9a540 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (4m) 59s ago 4m 55.9M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 2536ad6d3f49 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi160 running (4m) 59s ago 4m 71.2M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 5ff584120ae9 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi204 running (4m) 59s ago 4m 64.1M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 c4dde3bf2a50 2024-05-08T23:46:00.908 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi012 running (4m) 59s ago 4m 60.0M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 39f252d2e378 2024-05-08T23:46:00.909 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi160 running (3m) 59s ago 3m 47.1M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 4f1a12366076 2024-05-08T23:46:00.909 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi204 running (3m) 59s ago 3m 64.0M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 b57e0626d4a1 2024-05-08T23:46:00.909 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 smithi012 running (3m) 59s ago 3m 47.4M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 09b9cfbdb8f8 2024-05-08T23:46:00.909 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 smithi160 running (2m) 59s ago 2m 60.6M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 b3c952cb582d 2024-05-08T23:46:00.909 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 smithi204 running (2m) 59s ago 2m 60.6M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 b4a78b1011fb 2024-05-08T23:46:00.909 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 smithi012 running (2m) 59s ago 2m 45.6M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 661599c57676 2024-05-08T23:46:00.909 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 smithi160 running (115s) 59s ago 115s 50.0M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 df4a19c3332a 2024-05-08T23:46:00.909 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 smithi204 running (97s) 59s ago 97s 57.6M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 4ad2d646f1e3 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.913+0000 7fa08f76e700 1 -- 172.21.15.12:0/109890970 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa0740a11b0 msgr2=0x7fa0740a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.913+0000 7fa08f76e700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa0740a11b0 0x7fa0740a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa084005950 tx=0x7fa0840058e0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.913+0000 7fa08f76e700 1 -- 172.21.15.12:0/109890970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa088102d00 msgr2=0x7fa088100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.913+0000 7fa08f76e700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa088102d00 0x7fa088100850 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa080004d00 tx=0x7fa080004d30 comp rx=0 tx=0).stop 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.913+0000 7fa08f76e700 1 -- 172.21.15.12:0/109890970 shutdown_connections 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.913+0000 7fa08f76e700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa088102d00 0x7fa088100850 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.913+0000 7fa08f76e700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0880ff2e0 0x7fa088101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.913+0000 7fa08f76e700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa0740a11b0 0x7fa0740a3670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.913+0000 7fa08f76e700 1 --2- 172.21.15.12:0/109890970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa088102360 0x7fa088100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.913+0000 7fa08f76e700 1 -- 172.21.15.12:0/109890970 >> 172.21.15.12:0/109890970 conn(0x7fa0880fac00 msgr2=0x7fa0881056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.914+0000 7fa08f76e700 1 -- 172.21.15.12:0/109890970 shutdown_connections 2024-05-08T23:46:00.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:00.914+0000 7fa08f76e700 1 -- 172.21.15.12:0/109890970 wait complete. 2024-05-08T23:46:01.295 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T23:46:01.551 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:46:02.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:02 smithi012 ceph-mon[42503]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:02.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:02 smithi012 ceph-mon[42503]: pgmap v256: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:46:02.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:02 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:46:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:02 smithi204 ceph-mon[47373]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:02 smithi204 ceph-mon[47373]: pgmap v256: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:46:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:02 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:46:02.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:02 smithi160 ceph-mon[46152]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:02.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:02 smithi160 ceph-mon[46152]: pgmap v256: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:46:02.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:02 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:46:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.826+0000 7f6eaf985700 1 -- 172.21.15.12:0/1515015568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea8101000 msgr2=0x7f6ea8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.826+0000 7f6eaf985700 1 --2- 172.21.15.12:0/1515015568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea8101000 0x7f6ea8101420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40099c0 tx=0x7f6ea4009cd0 comp rx=0 tx=0).stop 2024-05-08T23:46:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.826+0000 7f6eaf985700 1 -- 172.21.15.12:0/1515015568 shutdown_connections 2024-05-08T23:46:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.826+0000 7f6eaf985700 1 --2- 172.21.15.12:0/1515015568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea8102ba0 0x7f6ea8109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.826+0000 7f6eaf985700 1 --2- 172.21.15.12:0/1515015568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea8101000 0x7f6ea8101420 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.826+0000 7f6eaf985700 1 --2- 172.21.15.12:0/1515015568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea8102200 0x7f6ea8102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.826+0000 7f6eaf985700 1 -- 172.21.15.12:0/1515015568 >> 172.21.15.12:0/1515015568 conn(0x7f6ea80fc5a0 msgr2=0x7f6ea80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.826+0000 7f6eaf985700 1 -- 172.21.15.12:0/1515015568 shutdown_connections 2024-05-08T23:46:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.826+0000 7f6eaf985700 1 -- 172.21.15.12:0/1515015568 wait complete. 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eaf985700 1 Processor -- start 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eaf985700 1 -- start start 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eaf985700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea8101000 0x7f6ea8196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eaf985700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea8102200 0x7f6ea8197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eaf985700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea8102ba0 0x7f6ea819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eaf985700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea819ea60 con 0x7f6ea8102200 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eaf985700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea819eba0 con 0x7f6ea8101000 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eaf985700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea819ed30 con 0x7f6ea8102ba0 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eacf20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea8102200 0x7f6ea8197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eacf20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea8102200 0x7f6ea8197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43682/0 (socket says 172.21.15.12:43682) 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eacf20700 1 -- 172.21.15.12:0/4032803841 learned_addr learned my addr 172.21.15.12:0/4032803841 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6ead721700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea8101000 0x7f6ea8196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.827+0000 7f6eadf22700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea8102ba0 0x7f6ea819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.828+0000 7f6eacf20700 1 -- 172.21.15.12:0/4032803841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea8102ba0 msgr2=0x7f6ea819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.828+0000 7f6eacf20700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea8102ba0 0x7f6ea819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.828+0000 7f6eacf20700 1 -- 172.21.15.12:0/4032803841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea8101000 msgr2=0x7f6ea8196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.828+0000 7f6eacf20700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea8101000 0x7f6ea8196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.828+0000 7f6eacf20700 1 -- 172.21.15.12:0/4032803841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40095f0 con 0x7f6ea8102200 2024-05-08T23:46:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.828+0000 7f6ead721700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea8101000 0x7f6ea8196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.828+0000 7f6eacf20700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea8102200 0x7f6ea8197460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800e9d0 tx=0x7f6e9800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.830+0000 7f6e9e7fc700 1 -- 172.21.15.12:0/4032803841 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98009790 con 0x7f6ea8102200 2024-05-08T23:46:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.830+0000 7f6eaf985700 1 -- 172.21.15.12:0/4032803841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea819efb0 con 0x7f6ea8102200 2024-05-08T23:46:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.830+0000 7f6eaf985700 1 -- 172.21.15.12:0/4032803841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea819f4d0 con 0x7f6ea8102200 2024-05-08T23:46:02.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.831+0000 7f6e9e7fc700 1 -- 172.21.15.12:0/4032803841 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e9800ee70 con 0x7f6ea8102200 2024-05-08T23:46:02.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.831+0000 7f6e9e7fc700 1 -- 172.21.15.12:0/4032803841 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98018420 con 0x7f6ea8102200 2024-05-08T23:46:02.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.832+0000 7f6eaf985700 1 -- 172.21.15.12:0/4032803841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8c005320 con 0x7f6ea8102200 2024-05-08T23:46:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.833+0000 7f6e9e7fc700 1 -- 172.21.15.12:0/4032803841 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6e9800cbd0 con 0x7f6ea8102200 2024-05-08T23:46:02.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.833+0000 7f6e9e7fc700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e940a11b0 0x7f6e940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:02.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.834+0000 7f6ead721700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e940a11b0 0x7f6e940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:02.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.834+0000 7f6ead721700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e940a11b0 0x7f6e940a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009990 tx=0x7f6ea4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:02.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.835+0000 7f6e9e7fc700 1 -- 172.21.15.12:0/4032803841 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6e98014070 con 0x7f6ea8102200 2024-05-08T23:46:02.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:02.841+0000 7f6e9e7fc700 1 -- 172.21.15.12:0/4032803841 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6e98099d50 con 0x7f6ea8102200 2024-05-08T23:46:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.079+0000 7f6eaf985700 1 -- 172.21.15.12:0/4032803841 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e8c000bf0 con 0x7f6e940a11b0 2024-05-08T23:46:03.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.085+0000 7f6e9e7fc700 1 -- 172.21.15.12:0/4032803841 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f6e8c000bf0 con 0x7f6e940a11b0 2024-05-08T23:46:03.081 INFO:teuthology.orchestra.run.smithi012.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T23:46:03.081 INFO:teuthology.orchestra.run.smithi012.stdout:mds.all 12/12 61s ago 87s smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-08T23:46:03.081 INFO:teuthology.orchestra.run.smithi012.stdout:mgr 3/3 61s ago 5m smithi012=x;smithi160=y;smithi204=z;count:3 2024-05-08T23:46:03.081 INFO:teuthology.orchestra.run.smithi012.stdout:mon 3/3 61s ago 5m smithi012:172.21.15.12=a;smithi160:172.21.15.160=b;smithi204:172.21.15.204=c;count:3 2024-05-08T23:46:03.081 INFO:teuthology.orchestra.run.smithi012.stdout:osd 12 61s ago - 2024-05-08T23:46:03.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 -- 172.21.15.12:0/4032803841 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e940a11b0 msgr2=0x7f6e940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:03.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e940a11b0 0x7f6e940a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009990 tx=0x7f6ea4000bc0 comp rx=0 tx=0).stop 2024-05-08T23:46:03.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 -- 172.21.15.12:0/4032803841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea8102200 msgr2=0x7f6ea8197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:03.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea8102200 0x7f6ea8197460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800e9d0 tx=0x7f6e9800ece0 comp rx=0 tx=0).stop 2024-05-08T23:46:03.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 -- 172.21.15.12:0/4032803841 shutdown_connections 2024-05-08T23:46:03.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea8102ba0 0x7f6ea819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:03.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f6e940a11b0 0x7f6e940a3670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:03.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ea8101000 0x7f6ea8196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:03.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 --2- 172.21.15.12:0/4032803841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ea8102200 0x7f6ea8197460 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:03.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 -- 172.21.15.12:0/4032803841 >> 172.21.15.12:0/4032803841 conn(0x7f6ea80fc5a0 msgr2=0x7f6ea8105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:03.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 -- 172.21.15.12:0/4032803841 shutdown_connections 2024-05-08T23:46:03.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:03.088+0000 7f6eaf985700 1 -- 172.21.15.12:0/4032803841 wait complete. 2024-05-08T23:46:03.489 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T23:46:03.764 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:46:04.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:03 smithi012 ceph-mon[42503]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:04.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:03 smithi012 ceph-mon[42503]: pgmap v257: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:46:04.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:03 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:03 smithi204 ceph-mon[47373]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:04.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:03 smithi204 ceph-mon[47373]: pgmap v257: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:46:04.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:03 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:03 smithi160 ceph-mon[46152]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:04.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:03 smithi160 ceph-mon[46152]: pgmap v257: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:46:04.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:03 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:04.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.838+0000 7fc47dcd8700 1 -- 172.21.15.12:0/1755194878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc478102ba0 msgr2=0x7fc478109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:04.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.838+0000 7fc47dcd8700 1 --2- 172.21.15.12:0/1755194878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc478102ba0 0x7fc478109450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc47000b0d0 tx=0x7fc47000b3e0 comp rx=0 tx=0).stop 2024-05-08T23:46:04.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.838+0000 7fc47dcd8700 1 -- 172.21.15.12:0/1755194878 shutdown_connections 2024-05-08T23:46:04.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.838+0000 7fc47dcd8700 1 --2- 172.21.15.12:0/1755194878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc478102ba0 0x7fc478109450 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.838+0000 7fc47dcd8700 1 --2- 172.21.15.12:0/1755194878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc478101000 0x7fc478101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.838+0000 7fc47dcd8700 1 --2- 172.21.15.12:0/1755194878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc478102200 0x7fc478102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.838+0000 7fc47dcd8700 1 -- 172.21.15.12:0/1755194878 >> 172.21.15.12:0/1755194878 conn(0x7fc4780fc5c0 msgr2=0x7fc4780fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.838+0000 7fc47dcd8700 1 -- 172.21.15.12:0/1755194878 shutdown_connections 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.838+0000 7fc47dcd8700 1 -- 172.21.15.12:0/1755194878 wait complete. 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.839+0000 7fc47dcd8700 1 Processor -- start 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.839+0000 7fc47dcd8700 1 -- start start 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.839+0000 7fc47dcd8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc478101000 0x7fc478197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.839+0000 7fc47dcd8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc478102200 0x7fc478197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.839+0000 7fc47dcd8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc478102ba0 0x7fc47819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.839+0000 7fc47dcd8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47819eaf0 con 0x7fc478102200 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.839+0000 7fc47dcd8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47819ec30 con 0x7fc478101000 2024-05-08T23:46:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.839+0000 7fc47dcd8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47819edc0 con 0x7fc478102ba0 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.839+0000 7fc476ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc478102200 0x7fc478197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc476ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc478102200 0x7fc478197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43702/0 (socket says 172.21.15.12:43702) 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc476ffd700 1 -- 172.21.15.12:0/3854843527 learned_addr learned my addr 172.21.15.12:0/3854843527 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc4777fe700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc478101000 0x7fc478197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc477fff700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc478102ba0 0x7fc47819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc476ffd700 1 -- 172.21.15.12:0/3854843527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc478102ba0 msgr2=0x7fc47819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc476ffd700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc478102ba0 0x7fc47819e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc476ffd700 1 -- 172.21.15.12:0/3854843527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc478101000 msgr2=0x7fc478197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc476ffd700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc478101000 0x7fc478197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc476ffd700 1 -- 172.21.15.12:0/3854843527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc470009d00 con 0x7fc478102200 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc477fff700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc478102ba0 0x7fc47819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc476ffd700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc478102200 0x7fc478197540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc46800d9b0 tx=0x7fc46800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc474ff9700 1 -- 172.21.15.12:0/3854843527 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468004220 con 0x7fc478102200 2024-05-08T23:46:04.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc474ff9700 1 -- 172.21.15.12:0/3854843527 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc468007640 con 0x7fc478102200 2024-05-08T23:46:04.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc47dcd8700 1 -- 172.21.15.12:0/3854843527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47819f0a0 con 0x7fc478102200 2024-05-08T23:46:04.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc474ff9700 1 -- 172.21.15.12:0/3854843527 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468005650 con 0x7fc478102200 2024-05-08T23:46:04.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.840+0000 7fc47dcd8700 1 -- 172.21.15.12:0/3854843527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47819f5f0 con 0x7fc478102200 2024-05-08T23:46:04.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.841+0000 7fc47dcd8700 1 -- 172.21.15.12:0/3854843527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4781083c0 con 0x7fc478102200 2024-05-08T23:46:04.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.842+0000 7fc474ff9700 1 -- 172.21.15.12:0/3854843527 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc468004750 con 0x7fc478102200 2024-05-08T23:46:04.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.843+0000 7fc474ff9700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc4640a1230 0x7fc4640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:04.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.844+0000 7fc4777fe700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc4640a1230 0x7fc4640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:04.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.844+0000 7fc474ff9700 1 -- 172.21.15.12:0/3854843527 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc4680ccd40 con 0x7fc478102200 2024-05-08T23:46:04.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.844+0000 7fc4777fe700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc4640a1230 0x7fc4640a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc478102060 tx=0x7fc46000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:04.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:04.848+0000 7fc474ff9700 1 -- 172.21.15.12:0/3854843527 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc468096370 con 0x7fc478102200 2024-05-08T23:46:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.068+0000 7fc47dcd8700 1 -- 172.21.15.12:0/3854843527 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc47802ce10 con 0x7fc4640a1230 2024-05-08T23:46:05.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.069+0000 7fc474ff9700 1 -- 172.21.15.12:0/3854843527 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fc47802ce10 con 0x7fc4640a1230 2024-05-08T23:46:05.065 INFO:teuthology.orchestra.run.smithi012.stdout:HOST ADDR LABELS STATUS 2024-05-08T23:46:05.065 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 172.21.15.12 2024-05-08T23:46:05.065 INFO:teuthology.orchestra.run.smithi012.stdout:smithi160 172.21.15.160 2024-05-08T23:46:05.065 INFO:teuthology.orchestra.run.smithi012.stdout:smithi204 172.21.15.204 2024-05-08T23:46:05.065 INFO:teuthology.orchestra.run.smithi012.stdout:3 hosts in cluster 2024-05-08T23:46:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.071+0000 7fc47dcd8700 1 -- 172.21.15.12:0/3854843527 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc4640a1230 msgr2=0x7fc4640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.071+0000 7fc47dcd8700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc4640a1230 0x7fc4640a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc478102060 tx=0x7fc46000b410 comp rx=0 tx=0).stop 2024-05-08T23:46:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.071+0000 7fc47dcd8700 1 -- 172.21.15.12:0/3854843527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc478102200 msgr2=0x7fc478197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.071+0000 7fc47dcd8700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc478102200 0x7fc478197540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc46800d9b0 tx=0x7fc46800dcc0 comp rx=0 tx=0).stop 2024-05-08T23:46:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.072+0000 7fc47dcd8700 1 -- 172.21.15.12:0/3854843527 shutdown_connections 2024-05-08T23:46:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.072+0000 7fc47dcd8700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc478102ba0 0x7fc47819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.072+0000 7fc47dcd8700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fc4640a1230 0x7fc4640a36f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.072+0000 7fc47dcd8700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc478101000 0x7fc478197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.072+0000 7fc47dcd8700 1 --2- 172.21.15.12:0/3854843527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc478102200 0x7fc478197540 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.072+0000 7fc47dcd8700 1 -- 172.21.15.12:0/3854843527 >> 172.21.15.12:0/3854843527 conn(0x7fc4780fc5c0 msgr2=0x7fc478105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.072+0000 7fc47dcd8700 1 -- 172.21.15.12:0/3854843527 shutdown_connections 2024-05-08T23:46:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:05.072+0000 7fc47dcd8700 1 -- 172.21.15.12:0/3854843527 wait complete. 2024-05-08T23:46:05.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:46:05.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:46:05.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:04 smithi012 ceph-mon[42503]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:05.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:46:05.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:46:05.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:04 smithi204 ceph-mon[47373]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:05.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:46:05.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:46:05.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:04 smithi160 ceph-mon[46152]: from='mgr.14152 172.21.15.12:0/1234176845' entity='mgr.x' 2024-05-08T23:46:05.461 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T23:46:05.718 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:46:06.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:05 smithi012 ceph-mon[42503]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:06.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:05 smithi012 ceph-mon[42503]: pgmap v258: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:46:06.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:05 smithi204 ceph-mon[47373]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:06.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:05 smithi204 ceph-mon[47373]: pgmap v258: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:46:06.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:05 smithi160 ceph-mon[46152]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:06.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:05 smithi160 ceph-mon[46152]: pgmap v258: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:46:06.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.810+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3098778982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf68102200 msgr2=0x7fcf68102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:06.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.810+0000 7fcf6ca16700 1 --2- 172.21.15.12:0/3098778982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf68102200 0x7fcf68102660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcf500099c0 tx=0x7fcf50009cd0 comp rx=0 tx=0).stop 2024-05-08T23:46:06.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.810+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3098778982 shutdown_connections 2024-05-08T23:46:06.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.810+0000 7fcf6ca16700 1 --2- 172.21.15.12:0/3098778982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf68102ba0 0x7fcf68109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:06.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.810+0000 7fcf6ca16700 1 --2- 172.21.15.12:0/3098778982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf68101000 0x7fcf68101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:06.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.810+0000 7fcf6ca16700 1 --2- 172.21.15.12:0/3098778982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf68102200 0x7fcf68102660 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:06.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.810+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3098778982 >> 172.21.15.12:0/3098778982 conn(0x7fcf680fc5c0 msgr2=0x7fcf680fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:06.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.811+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3098778982 shutdown_connections 2024-05-08T23:46:06.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.811+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3098778982 wait complete. 2024-05-08T23:46:06.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6ca16700 1 Processor -- start 2024-05-08T23:46:06.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6ca16700 1 -- start start 2024-05-08T23:46:06.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6ca16700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf68101000 0x7fcf68196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:06.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6ca16700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf68102200 0x7fcf68197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:06.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6ca16700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf68102ba0 0x7fcf6819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:06.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6ca16700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6819ea50 con 0x7fcf68101000 2024-05-08T23:46:06.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6ca16700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6819eb90 con 0x7fcf68102200 2024-05-08T23:46:06.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6ca16700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6819ed20 con 0x7fcf68102ba0 2024-05-08T23:46:06.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf68101000 0x7fcf68196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf68101000 0x7fcf68196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43718/0 (socket says 172.21.15.12:43718) 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf6659c700 1 -- 172.21.15.12:0/3423296011 learned_addr learned my addr 172.21.15.12:0/3423296011 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.812+0000 7fcf66d9d700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf68102ba0 0x7fcf6819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf5dd9b700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf68102200 0x7fcf68197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf6659c700 1 -- 172.21.15.12:0/3423296011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf68102ba0 msgr2=0x7fcf6819e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf6659c700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf68102ba0 0x7fcf6819e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf6659c700 1 -- 172.21.15.12:0/3423296011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf68102200 msgr2=0x7fcf68197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf6659c700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf68102200 0x7fcf68197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf6659c700 1 -- 172.21.15.12:0/3423296011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf500095f0 con 0x7fcf68101000 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf6659c700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf68101000 0x7fcf68196f10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800d9b0 tx=0x7fcf5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf5ffff700 1 -- 172.21.15.12:0/3423296011 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58009930 con 0x7fcf68101000 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf5ffff700 1 -- 172.21.15.12:0/3423296011 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf58010460 con 0x7fcf68101000 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3423296011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6819efa0 con 0x7fcf68101000 2024-05-08T23:46:06.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf5ffff700 1 -- 172.21.15.12:0/3423296011 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800f620 con 0x7fcf68101000 2024-05-08T23:46:06.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.813+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3423296011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6819f410 con 0x7fcf68101000 2024-05-08T23:46:06.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.815+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3423296011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf681084e0 con 0x7fcf68101000 2024-05-08T23:46:06.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.816+0000 7fcf5ffff700 1 -- 172.21.15.12:0/3423296011 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcf58010860 con 0x7fcf68101000 2024-05-08T23:46:06.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.817+0000 7fcf5ffff700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf540a1230 0x7fcf540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:06.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.817+0000 7fcf5dd9b700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf540a1230 0x7fcf540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:06.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.817+0000 7fcf5ffff700 1 -- 172.21.15.12:0/3423296011 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcf580cc310 con 0x7fcf68101000 2024-05-08T23:46:06.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.817+0000 7fcf5dd9b700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf540a1230 0x7fcf540a36f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009990 tx=0x7fcf50005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:06.822+0000 7fcf5ffff700 1 -- 172.21.15.12:0/3423296011 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcf58095a40 con 0x7fcf68101000 2024-05-08T23:46:07.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.042+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3423296011 --> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf6802ceb0 con 0x7fcf540a1230 2024-05-08T23:46:07.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.044+0000 7fcf5ffff700 1 -- 172.21.15.12:0/3423296011 <== mgr.14152 v2:172.21.15.12:6800/1305168755 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fcf6802ceb0 con 0x7fcf540a1230 2024-05-08T23:46:07.040 INFO:teuthology.orchestra.run.smithi012.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T23:46:07.040 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005J400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T23:46:07.040 INFO:teuthology.orchestra.run.smithi012.stdout:smithi160 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JY400BGN 372G No 116s ago Has a FileSystem, LVM detected 2024-05-08T23:46:07.040 INFO:teuthology.orchestra.run.smithi012.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 98s ago Has a FileSystem, LVM detected 2024-05-08T23:46:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.046+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3423296011 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf540a1230 msgr2=0x7fcf540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.046+0000 7fcf6ca16700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf540a1230 0x7fcf540a36f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009990 tx=0x7fcf50005ad0 comp rx=0 tx=0).stop 2024-05-08T23:46:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.046+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3423296011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf68101000 msgr2=0x7fcf68196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.046+0000 7fcf6ca16700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf68101000 0x7fcf68196f10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800d9b0 tx=0x7fcf5800dcc0 comp rx=0 tx=0).stop 2024-05-08T23:46:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.047+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3423296011 shutdown_connections 2024-05-08T23:46:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.047+0000 7fcf6ca16700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf68102ba0 0x7fcf6819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.047+0000 7fcf6ca16700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf68101000 0x7fcf68196f10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.047+0000 7fcf6ca16700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fcf540a1230 0x7fcf540a36f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.047+0000 7fcf6ca16700 1 --2- 172.21.15.12:0/3423296011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf68102200 0x7fcf68197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.047+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3423296011 >> 172.21.15.12:0/3423296011 conn(0x7fcf680fc5c0 msgr2=0x7fcf68105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.047+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3423296011 shutdown_connections 2024-05-08T23:46:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:07.047+0000 7fcf6ca16700 1 -- 172.21.15.12:0/3423296011 wait complete. 2024-05-08T23:46:07.435 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T23:46:07.447 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-08T23:46:07.447 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T23:46:07.708 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:46:08.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:08 smithi012 ceph-mon[42503]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:08.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:08 smithi012 ceph-mon[42503]: pgmap v259: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:46:08.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:08 smithi204 ceph-mon[47373]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:08.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:08 smithi204 ceph-mon[47373]: pgmap v259: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:46:08.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:08 smithi160 ceph-mon[46152]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:08.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:08 smithi160 ceph-mon[46152]: pgmap v259: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:46:08.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.879+0000 7efda7b19700 1 -- 172.21.15.12:0/2663010987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efda0101060 msgr2=0x7efda01014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.879+0000 7efda7b19700 1 --2- 172.21.15.12:0/2663010987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efda0101060 0x7efda01014e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efd90009990 tx=0x7efd90009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.880+0000 7efda7b19700 1 -- 172.21.15.12:0/2663010987 shutdown_connections 2024-05-08T23:46:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.880+0000 7efda7b19700 1 --2- 172.21.15.12:0/2663010987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efda0101a20 0x7efda0108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.880+0000 7efda7b19700 1 --2- 172.21.15.12:0/2663010987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda00fff00 0x7efda0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.880+0000 7efda7b19700 1 --2- 172.21.15.12:0/2663010987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efda0101060 0x7efda01014e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.880+0000 7efda7b19700 1 -- 172.21.15.12:0/2663010987 >> 172.21.15.12:0/2663010987 conn(0x7efda00fb4a0 msgr2=0x7efda00fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.880+0000 7efda7b19700 1 -- 172.21.15.12:0/2663010987 shutdown_connections 2024-05-08T23:46:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.880+0000 7efda7b19700 1 -- 172.21.15.12:0/2663010987 wait complete. 2024-05-08T23:46:08.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.880+0000 7efda7b19700 1 Processor -- start 2024-05-08T23:46:08.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.880+0000 7efda7b19700 1 -- start start 2024-05-08T23:46:08.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda7b19700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda00fff00 0x7efda0103e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:08.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda7b19700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efda0101060 0x7efda01024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:08.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda7b19700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efda0101a20 0x7efda0102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda7b19700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda01031b0 con 0x7efda0101060 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda7b19700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda0103320 con 0x7efda00fff00 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda7b19700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda01a1000 con 0x7efda0101a20 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda50b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efda0101060 0x7efda01024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda50b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efda0101060 0x7efda01024d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33524/0 (socket says 172.21.15.12:33524) 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda50b4700 1 -- 172.21.15.12:0/3236190445 learned_addr learned my addr 172.21.15.12:0/3236190445 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda60b6700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efda0101a20 0x7efda0102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda58b5700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda00fff00 0x7efda0103e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda50b4700 1 -- 172.21.15.12:0/3236190445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efda0101a20 msgr2=0x7efda0102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda50b4700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efda0101a20 0x7efda0102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda50b4700 1 -- 172.21.15.12:0/3236190445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda00fff00 msgr2=0x7efda0103e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda50b4700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda00fff00 0x7efda0103e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda50b4700 1 -- 172.21.15.12:0/3236190445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900095f0 con 0x7efda0101060 2024-05-08T23:46:08.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efda50b4700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efda0101060 0x7efda01024d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efd90006010 tx=0x7efd90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:08.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.881+0000 7efd96ffd700 1 -- 172.21.15.12:0/3236190445 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9001d070 con 0x7efda0101060 2024-05-08T23:46:08.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.882+0000 7efd96ffd700 1 -- 172.21.15.12:0/3236190445 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd90005030 con 0x7efda0101060 2024-05-08T23:46:08.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.882+0000 7efda7b19700 1 -- 172.21.15.12:0/3236190445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda01a1230 con 0x7efda0101060 2024-05-08T23:46:08.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.882+0000 7efd96ffd700 1 -- 172.21.15.12:0/3236190445 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90004ba0 con 0x7efda0101060 2024-05-08T23:46:08.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.882+0000 7efda7b19700 1 -- 172.21.15.12:0/3236190445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda01a1640 con 0x7efda0101060 2024-05-08T23:46:08.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.883+0000 7efda7b19700 1 -- 172.21.15.12:0/3236190445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda004ec60 con 0x7efda0101060 2024-05-08T23:46:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.884+0000 7efd96ffd700 1 -- 172.21.15.12:0/3236190445 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7efd90004d00 con 0x7efda0101060 2024-05-08T23:46:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.885+0000 7efd96ffd700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efd8c0a11b0 0x7efd8c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.885+0000 7efda58b5700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efd8c0a11b0 0x7efd8c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.885+0000 7efd96ffd700 1 -- 172.21.15.12:0/3236190445 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7efd900cdf20 con 0x7efda0101060 2024-05-08T23:46:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.886+0000 7efda58b5700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efd8c0a11b0 0x7efd8c0a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009c60 tx=0x7efd9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:08.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:08.889+0000 7efd96ffd700 1 -- 172.21.15.12:0/3236190445 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efd900975e0 con 0x7efda0101060 2024-05-08T23:46:09.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.146+0000 7efda7b19700 1 -- 172.21.15.12:0/3236190445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efda010b9b0 con 0x7efda0101060 2024-05-08T23:46:09.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.147+0000 7efd96ffd700 1 -- 172.21.15.12:0/3236190445 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2903 (secure 0 0 0) 0x7efd9009ac00 con 0x7efda0101060 2024-05-08T23:46:09.143 INFO:teuthology.orchestra.run.smithi012.stdout:e23 2024-05-08T23:46:09.143 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T23:46:09.143 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:46:09.143 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 23 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-05-08T23:45:04.027837+0000 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-05-08T23:45:09.067610+0000 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 300 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-05-08T23:46:09.144 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 3 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:in 0,1,2 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=24548,1=24515,2=24511} 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-05-08T23:46:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 0 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b{0:24548} state up:active seq 3 addr [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c{0:24490} state up:standby-replay seq 1 addr [v2:172.21.15.204:6836/2858273239,v1:172.21.15.204:6837/2858273239] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.h{1:24515} state up:active seq 6 addr [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e{1:24488} state up:standby-replay seq 1 addr [v2:172.21.15.160:6834/3313911488,v1:172.21.15.160:6835/3313911488] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.i{2:24511} state up:active seq 5 addr [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f{2:24484} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/1481890752,v1:172.21.15.204:6835/1481890752] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.12:6834/2488990436,v1:172.21.15.12:6835/2488990436] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.147 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.g{-1:14604} state up:standby seq 1 addr [v2:172.21.15.12:6838/3503743277,v1:172.21.15.12:6839/3503743277] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.147 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a{-1:14613} state up:standby seq 1 addr [v2:172.21.15.12:6840/2743302125,v1:172.21.15.12:6841/2743302125] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.147 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.l{-1:24439} state up:standby seq 1 addr [v2:172.21.15.204:6832/4091100213,v1:172.21.15.204:6833/4091100213] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.147 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.k{-1:24452} state up:standby seq 1 addr [v2:172.21.15.160:6832/1870735591,v1:172.21.15.160:6833/1870735591] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.147 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d{-1:24463} state up:standby seq 1 addr [v2:172.21.15.12:6836/4284773137,v1:172.21.15.12:6837/4284773137] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:46:09.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 -- 172.21.15.12:0/3236190445 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efd8c0a11b0 msgr2=0x7efd8c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efd8c0a11b0 0x7efd8c0a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009c60 tx=0x7efd9c009260 comp rx=0 tx=0).stop 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 -- 172.21.15.12:0/3236190445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efda0101060 msgr2=0x7efda01024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efda0101060 0x7efda01024d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efd90006010 tx=0x7efd90004730 comp rx=0 tx=0).stop 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 -- 172.21.15.12:0/3236190445 shutdown_connections 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efda0101a20 0x7efda0102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7efd8c0a11b0 0x7efd8c0a3670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda00fff00 0x7efda0103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 --2- 172.21.15.12:0/3236190445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efda0101060 0x7efda01024d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 -- 172.21.15.12:0/3236190445 >> 172.21.15.12:0/3236190445 conn(0x7efda00fb4a0 msgr2=0x7efda0068a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 -- 172.21.15.12:0/3236190445 shutdown_connections 2024-05-08T23:46:09.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:09.150+0000 7efda7b19700 1 -- 172.21.15.12:0/3236190445 wait complete. 2024-05-08T23:46:09.149 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 23 2024-05-08T23:46:09.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:09 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3236190445' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T23:46:09.543 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T23:46:09.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:09 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3236190445' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T23:46:09.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:09 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3236190445' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T23:46:09.803 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:46:10.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:10 smithi012 ceph-mon[42503]: pgmap v260: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:46:10.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:10 smithi204 ceph-mon[47373]: pgmap v260: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:46:10.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:10 smithi160 ceph-mon[46152]: pgmap v260: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:46:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.935+0000 7f74d8721700 1 -- 172.21.15.12:0/3874880324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d0101a40 msgr2=0x7f74d0108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.935+0000 7f74d8721700 1 --2- 172.21.15.12:0/3874880324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d0101a40 0x7f74d0108360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f74c800b0a0 tx=0x7f74c800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.935+0000 7f74d8721700 1 -- 172.21.15.12:0/3874880324 shutdown_connections 2024-05-08T23:46:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.935+0000 7f74d8721700 1 --2- 172.21.15.12:0/3874880324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d0101a40 0x7f74d0108360 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.935+0000 7f74d8721700 1 --2- 172.21.15.12:0/3874880324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d00ffe90 0x7f74d01002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.935+0000 7f74d8721700 1 --2- 172.21.15.12:0/3874880324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74d0101080 0x7f74d0101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.935+0000 7f74d8721700 1 -- 172.21.15.12:0/3874880324 >> 172.21.15.12:0/3874880324 conn(0x7f74d00fb4a0 msgr2=0x7f74d00fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.935+0000 7f74d8721700 1 -- 172.21.15.12:0/3874880324 shutdown_connections 2024-05-08T23:46:10.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.935+0000 7f74d8721700 1 -- 172.21.15.12:0/3874880324 wait complete. 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d8721700 1 Processor -- start 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d8721700 1 -- start start 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d8721700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d00ffe90 0x7f74d0197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d8721700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d0101080 0x7f74d0197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d8721700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74d0101a40 0x7f74d019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d8721700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d019eb40 con 0x7f74d00ffe90 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d8721700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d019ec80 con 0x7f74d0101a40 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d8721700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d019ee10 con 0x7f74d0101080 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d64bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d00ffe90 0x7f74d0197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d64bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d00ffe90 0x7f74d0197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33530/0 (socket says 172.21.15.12:33530) 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d64bd700 1 -- 172.21.15.12:0/3518377447 learned_addr learned my addr 172.21.15.12:0/3518377447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d6cbe700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74d0101a40 0x7f74d019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.936+0000 7f74d5cbc700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d0101080 0x7f74d0197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74d64bd700 1 -- 172.21.15.12:0/3518377447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d0101080 msgr2=0x7f74d0197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74d64bd700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d0101080 0x7f74d0197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74d64bd700 1 -- 172.21.15.12:0/3518377447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74d0101a40 msgr2=0x7f74d019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74d64bd700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74d0101a40 0x7f74d019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74d64bd700 1 -- 172.21.15.12:0/3518377447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c8009d00 con 0x7f74d00ffe90 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74d6cbe700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74d0101a40 0x7f74d019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74d5cbc700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d0101080 0x7f74d0197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74d64bd700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d00ffe90 0x7f74d0197000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00d9b0 tx=0x7f74cc00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74c77fe700 1 -- 172.21.15.12:0/3518377447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc00bb60 con 0x7f74d00ffe90 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74c77fe700 1 -- 172.21.15.12:0/3518377447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74cc007620 con 0x7f74d00ffe90 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74d8721700 1 -- 172.21.15.12:0/3518377447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d019f0f0 con 0x7f74d00ffe90 2024-05-08T23:46:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74c77fe700 1 -- 172.21.15.12:0/3518377447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc00f620 con 0x7f74d00ffe90 2024-05-08T23:46:10.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.937+0000 7f74d8721700 1 -- 172.21.15.12:0/3518377447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d019f5f0 con 0x7f74d00ffe90 2024-05-08T23:46:10.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.938+0000 7f74d8721700 1 -- 172.21.15.12:0/3518377447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d0107350 con 0x7f74d00ffe90 2024-05-08T23:46:10.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.939+0000 7f74c77fe700 1 -- 172.21.15.12:0/3518377447 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f74cc010460 con 0x7f74d00ffe90 2024-05-08T23:46:10.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.940+0000 7f74c77fe700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f74bc0a11b0 0x7f74bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:10.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.940+0000 7f74d5cbc700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f74bc0a11b0 0x7f74bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:10.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.940+0000 7f74c77fe700 1 -- 172.21.15.12:0/3518377447 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f74cc0cd0d0 con 0x7f74d00ffe90 2024-05-08T23:46:10.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.941+0000 7f74d5cbc700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f74bc0a11b0 0x7f74bc0a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f74c0005e10 tx=0x7f74c0005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:10.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:10.945+0000 7f74c77fe700 1 -- 172.21.15.12:0/3518377447 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f74cc096800 con 0x7f74d00ffe90 2024-05-08T23:46:11.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.158+0000 7f74d8721700 1 -- 172.21.15.12:0/3518377447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f74d002cdd0 con 0x7f74d00ffe90 2024-05-08T23:46:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.159+0000 7f74c77fe700 1 -- 172.21.15.12:0/3518377447 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) v1 ==== 56+0+5225 (secure 0 0 0) 0x7f74cc007e60 con 0x7f74d00ffe90 2024-05-08T23:46:11.155 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 83 2024-05-08T23:46:11.155 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-08T23:46:11.155 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-05-08T23:39:01.616862+0000 2024-05-08T23:46:11.155 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-05-08T23:45:56.347006+0000 2024-05-08T23:46:11.155 INFO:teuthology.orchestra.run.smithi012.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T23:46:11.155 INFO:teuthology.orchestra.run.smithi012.stdout:crush_version 26 2024-05-08T23:46:11.155 INFO:teuthology.orchestra.run.smithi012.stdout:full_ratio 0.95 2024-05-08T23:46:11.155 INFO:teuthology.orchestra.run.smithi012.stdout:backfillfull_ratio 0.9 2024-05-08T23:46:11.155 INFO:teuthology.orchestra.run.smithi012.stdout:nearfull_ratio 0.85 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:require_min_compat_client luminous 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:min_compat_client luminous 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:require_osd_release reef 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:stretch_mode_enabled false 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:max_osd 12 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6802/2326888222,v1:172.21.15.12:6803/2326888222] [v2:172.21.15.12:6804/2326888222,v1:172.21.15.12:6805/2326888222] exists,up e6051d38-71b4-4946-b055-5ccf985ceed4 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6800/72539183,v1:172.21.15.160:6801/72539183] [v2:172.21.15.160:6802/72539183,v1:172.21.15.160:6803/72539183] exists,up 83b3995f-c70f-49b5-a4eb-0969ce93f22b 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 up in weight 1 up_from 17 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/474619962,v1:172.21.15.204:6801/474619962] [v2:172.21.15.204:6802/474619962,v1:172.21.15.204:6803/474619962] exists,up 71a297a7-f16c-4e12-ab9f-f197e06ff07e 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 up in weight 1 up_from 24 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6810/2160100336,v1:172.21.15.12:6811/2160100336] [v2:172.21.15.12:6812/2160100336,v1:172.21.15.12:6813/2160100336] exists,up 1ebe819d-3d5e-4128-8c6a-23921df4320b 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 up in weight 1 up_from 31 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6808/2838790327,v1:172.21.15.160:6809/2838790327] [v2:172.21.15.160:6810/2838790327,v1:172.21.15.160:6811/2838790327] exists,up 0e77c894-013b-40d3-9249-85a0c63076ea 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 up in weight 1 up_from 36 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/3032887482,v1:172.21.15.204:6809/3032887482] [v2:172.21.15.204:6810/3032887482,v1:172.21.15.204:6811/3032887482] exists,up f45217cd-6ced-413f-8be8-66c2740e7fee 2024-05-08T23:46:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 up in weight 1 up_from 44 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6818/3301040337,v1:172.21.15.12:6819/3301040337] [v2:172.21.15.12:6820/3301040337,v1:172.21.15.12:6821/3301040337] exists,up 7c76b479-95f8-4516-bf24-3677fa408097 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 up in weight 1 up_from 49 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6816/1443429618,v1:172.21.15.160:6817/1443429618] [v2:172.21.15.160:6818/1443429618,v1:172.21.15.160:6819/1443429618] exists,up fc3df920-f628-41de-a0eb-fd1d6b1bf06a 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 up in weight 1 up_from 56 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/4190277185,v1:172.21.15.204:6817/4190277185] [v2:172.21.15.204:6818/4190277185,v1:172.21.15.204:6819/4190277185] exists,up 91051f6f-5519-48e0-93a6-e897d0cb3929 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 up in weight 1 up_from 64 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6826/609239172,v1:172.21.15.12:6827/609239172] [v2:172.21.15.12:6828/609239172,v1:172.21.15.12:6829/609239172] exists,up 3d7852c8-2f31-4385-84a1-6a4c07279741 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 up in weight 1 up_from 69 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6824/2916593770,v1:172.21.15.160:6825/2916593770] [v2:172.21.15.160:6826/2916593770,v1:172.21.15.160:6827/2916593770] exists,up f8a47be8-7650-4178-86b0-0b529d02ae21 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 up in weight 1 up_from 76 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/3148638065,v1:172.21.15.204:6825/3148638065] [v2:172.21.15.204:6826/3148638065,v1:172.21.15.204:6827/3148638065] exists,up e9361b5a-5dd2-4920-b89f-d040b487b6d7 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.5 [8,11] 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.13 [2,11] 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.3 [4,9] 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.10 [5,9] 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.21 [4,8] 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.38 [3,9] 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/4176796914 expires 2024-05-09T23:39:55.189061+0000 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2607913710 expires 2024-05-09T23:39:55.189061+0000 2024-05-08T23:46:11.157 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/3737654591 expires 2024-05-09T23:39:32.336894+0000 2024-05-08T23:46:11.158 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2069126425 expires 2024-05-09T23:39:32.336894+0000 2024-05-08T23:46:11.158 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/4231005605 expires 2024-05-09T23:39:32.336894+0000 2024-05-08T23:46:11.158 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2114420865 expires 2024-05-09T23:39:32.336894+0000 2024-05-08T23:46:11.158 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/4231005605 expires 2024-05-09T23:39:32.336894+0000 2024-05-08T23:46:11.158 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/31089560 expires 2024-05-09T23:39:55.189061+0000 2024-05-08T23:46:11.158 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/31089560 expires 2024-05-09T23:39:55.189061+0000 2024-05-08T23:46:11.158 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/1163805495 expires 2024-05-09T23:39:55.189061+0000 2024-05-08T23:46:11.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.161+0000 7f74d8721700 1 -- 172.21.15.12:0/3518377447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f74bc0a11b0 msgr2=0x7f74bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:11.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.161+0000 7f74d8721700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f74bc0a11b0 0x7f74bc0a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f74c0005e10 tx=0x7f74c0005d20 comp rx=0 tx=0).stop 2024-05-08T23:46:11.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.161+0000 7f74d8721700 1 -- 172.21.15.12:0/3518377447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d00ffe90 msgr2=0x7f74d0197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:11.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.161+0000 7f74d8721700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d00ffe90 0x7f74d0197000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00d9b0 tx=0x7f74cc00dd70 comp rx=0 tx=0).stop 2024-05-08T23:46:11.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.162+0000 7f74d8721700 1 -- 172.21.15.12:0/3518377447 shutdown_connections 2024-05-08T23:46:11.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.162+0000 7f74d8721700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f74bc0a11b0 0x7f74bc0a3670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.162+0000 7f74d8721700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74d0101a40 0x7f74d019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.162+0000 7f74d8721700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74d00ffe90 0x7f74d0197000 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.162+0000 7f74d8721700 1 --2- 172.21.15.12:0/3518377447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d0101080 0x7f74d0197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.162+0000 7f74d8721700 1 -- 172.21.15.12:0/3518377447 >> 172.21.15.12:0/3518377447 conn(0x7f74d00fb4a0 msgr2=0x7f74d0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:11.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.162+0000 7f74d8721700 1 -- 172.21.15.12:0/3518377447 shutdown_connections 2024-05-08T23:46:11.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.162+0000 7f74d8721700 1 -- 172.21.15.12:0/3518377447 wait complete. 2024-05-08T23:46:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:11 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3518377447' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T23:46:11.559 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T23:46:11.574 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:46:11.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.729+0000 7f82158f9700 1 -- 172.21.15.12:0/3494825788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8210104220 msgr2=0x7f8210106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:11.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.729+0000 7f82158f9700 1 --2- 172.21.15.12:0/3494825788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8210104220 0x7f8210106610 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009990 tx=0x7f81f8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.729+0000 7f82158f9700 1 -- 172.21.15.12:0/3494825788 shutdown_connections 2024-05-08T23:46:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.729+0000 7f82158f9700 1 --2- 172.21.15.12:0/3494825788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8210106b50 0x7f8210108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.729+0000 7f82158f9700 1 --2- 172.21.15.12:0/3494825788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82101018f0 0x7f8210103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.729+0000 7f82158f9700 1 --2- 172.21.15.12:0/3494825788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8210104220 0x7f8210106610 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.729+0000 7f82158f9700 1 -- 172.21.15.12:0/3494825788 >> 172.21.15.12:0/3494825788 conn(0x7f82100fb2e0 msgr2=0x7f82100fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.729+0000 7f82158f9700 1 -- 172.21.15.12:0/3494825788 shutdown_connections 2024-05-08T23:46:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.729+0000 7f82158f9700 1 -- 172.21.15.12:0/3494825788 wait complete. 2024-05-08T23:46:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f82158f9700 1 Processor -- start 2024-05-08T23:46:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f82158f9700 1 -- start start 2024-05-08T23:46:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f82158f9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82101018f0 0x7f8210101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f82158f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8210104220 0x7f82100ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f82158f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8210106b50 0x7f82100ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f82158f9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8210100580 con 0x7f8210106b50 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f82158f9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82101006f0 con 0x7f8210104220 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f82158f9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82101a0e60 con 0x7f82101018f0 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f820f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8210106b50 0x7f82100ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f820f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8210106b50 0x7f82100ffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33558/0 (socket says 172.21.15.12:33558) 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.730+0000 7f820f7fe700 1 -- 172.21.15.12:0/2723038542 learned_addr learned my addr 172.21.15.12:0/2723038542 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f820effd700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82101018f0 0x7f8210101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f820e7fc700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8210104220 0x7f82100ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f820f7fe700 1 -- 172.21.15.12:0/2723038542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82101018f0 msgr2=0x7f8210101210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f820f7fe700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82101018f0 0x7f8210101210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f820f7fe700 1 -- 172.21.15.12:0/2723038542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8210104220 msgr2=0x7f82100ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f820f7fe700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8210104220 0x7f82100ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f820f7fe700 1 -- 172.21.15.12:0/2723038542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f80095f0 con 0x7f8210106b50 2024-05-08T23:46:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f820e7fc700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8210104220 0x7f82100ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f820f7fe700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8210106b50 0x7f82100ffde0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f820800ef30 tx=0x7f820800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f82148f7700 1 -- 172.21.15.12:0/2723038542 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208018070 con 0x7f8210106b50 2024-05-08T23:46:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f82148f7700 1 -- 172.21.15.12:0/2723038542 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f820800f040 con 0x7f8210106b50 2024-05-08T23:46:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f82158f9700 1 -- 172.21.15.12:0/2723038542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82101a1090 con 0x7f8210106b50 2024-05-08T23:46:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f82148f7700 1 -- 172.21.15.12:0/2723038542 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820800ce00 con 0x7f8210106b50 2024-05-08T23:46:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.731+0000 7f82158f9700 1 -- 172.21.15.12:0/2723038542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82101a15a0 con 0x7f8210106b50 2024-05-08T23:46:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.732+0000 7f82158f9700 1 -- 172.21.15.12:0/2723038542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82100fce60 con 0x7f8210106b50 2024-05-08T23:46:11.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.733+0000 7f82148f7700 1 -- 172.21.15.12:0/2723038542 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8208003a00 con 0x7f8210106b50 2024-05-08T23:46:11.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.734+0000 7f82148f7700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f81fc0a11b0 0x7f81fc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:11.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.734+0000 7f820effd700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f81fc0a11b0 0x7f81fc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:11.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.734+0000 7f82148f7700 1 -- 172.21.15.12:0/2723038542 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f82080cd080 con 0x7f8210106b50 2024-05-08T23:46:11.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.734+0000 7f820effd700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f81fc0a11b0 0x7f81fc0a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8200005950 tx=0x7f82000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:11.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.738+0000 7f82148f7700 1 -- 172.21.15.12:0/2723038542 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8208096740 con 0x7f8210106b50 2024-05-08T23:46:11.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:11 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3518377447' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T23:46:11.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:11 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3518377447' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T23:46:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.960+0000 7f82158f9700 1 -- 172.21.15.12:0/2723038542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f821010c6b0 con 0x7f8210106b50 2024-05-08T23:46:11.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.961+0000 7f82148f7700 1 -- 172.21.15.12:0/2723038542 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22445 (secure 0 0 0) 0x7f8208099d60 con 0x7f8210106b50 2024-05-08T23:46:11.957 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:11.958 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:56.347006+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:46:11.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 -- 172.21.15.12:0/2723038542 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f81fc0a11b0 msgr2=0x7f81fc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:11.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f81fc0a11b0 0x7f81fc0a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8200005950 tx=0x7f82000058e0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 -- 172.21.15.12:0/2723038542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8210106b50 msgr2=0x7f82100ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:11.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8210106b50 0x7f82100ffde0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f820800ef30 tx=0x7f820800c450 comp rx=0 tx=0).stop 2024-05-08T23:46:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 -- 172.21.15.12:0/2723038542 shutdown_connections 2024-05-08T23:46:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8210106b50 0x7f82100ffde0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82101018f0 0x7f8210101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f81fc0a11b0 0x7f81fc0a3670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 --2- 172.21.15.12:0/2723038542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8210104220 0x7f82100ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 -- 172.21.15.12:0/2723038542 >> 172.21.15.12:0/2723038542 conn(0x7f82100fb2e0 msgr2=0x7f82101037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 -- 172.21.15.12:0/2723038542 shutdown_connections 2024-05-08T23:46:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:11.964+0000 7f82158f9700 1 -- 172.21.15.12:0/2723038542 wait complete. 2024-05-08T23:46:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:45:02.165326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:45:03.407498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:46:11.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:46:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.120+0000 7f91328df700 1 -- 172.21.15.12:0/1866518076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f912c1020f0 msgr2=0x7f912c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.120+0000 7f91328df700 1 --2- 172.21.15.12:0/1866518076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f912c1020f0 0x7f912c102570 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9120009990 tx=0x7f9120009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.120+0000 7f91328df700 1 -- 172.21.15.12:0/1866518076 shutdown_connections 2024-05-08T23:46:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.120+0000 7f91328df700 1 --2- 172.21.15.12:0/1866518076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c102ab0 0x7f912c1093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.120+0000 7f91328df700 1 --2- 172.21.15.12:0/1866518076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f912c100f20 0x7f912c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.120+0000 7f91328df700 1 --2- 172.21.15.12:0/1866518076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f912c1020f0 0x7f912c102570 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.120+0000 7f91328df700 1 -- 172.21.15.12:0/1866518076 >> 172.21.15.12:0/1866518076 conn(0x7f912c0fc4e0 msgr2=0x7f912c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.120+0000 7f91328df700 1 -- 172.21.15.12:0/1866518076 shutdown_connections 2024-05-08T23:46:12.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.120+0000 7f91328df700 1 -- 172.21.15.12:0/1866518076 wait complete. 2024-05-08T23:46:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f91328df700 1 Processor -- start 2024-05-08T23:46:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f91328df700 1 -- start start 2024-05-08T23:46:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f91328df700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f912c100f20 0x7f912c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f91328df700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f912c1020f0 0x7f912c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f91328df700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c102ab0 0x7f912c19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f91328df700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19ea60 con 0x7f912c1020f0 2024-05-08T23:46:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f91328df700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19eba0 con 0x7f912c100f20 2024-05-08T23:46:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f91328df700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19ed30 con 0x7f912c102ab0 2024-05-08T23:46:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f912c1020f0 0x7f912c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f912c1020f0 0x7f912c197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33580/0 (socket says 172.21.15.12:33580) 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f912b7fe700 1 -- 172.21.15.12:0/1953343098 learned_addr learned my addr 172.21.15.12:0/1953343098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f9130e7c700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c102ab0 0x7f912c19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.121+0000 7f912bfff700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f912c100f20 0x7f912c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f912b7fe700 1 -- 172.21.15.12:0/1953343098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c102ab0 msgr2=0x7f912c19e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f912b7fe700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c102ab0 0x7f912c19e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f912b7fe700 1 -- 172.21.15.12:0/1953343098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f912c100f20 msgr2=0x7f912c196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f912b7fe700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f912c100f20 0x7f912c196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f912b7fe700 1 -- 172.21.15.12:0/1953343098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91200095f0 con 0x7f912c1020f0 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f912b7fe700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f912c1020f0 0x7f912c197470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f912000b620 tx=0x7f9120004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f91297fa700 1 -- 172.21.15.12:0/1953343098 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912001d070 con 0x7f912c1020f0 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f91297fa700 1 -- 172.21.15.12:0/1953343098 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9120022470 con 0x7f912c1020f0 2024-05-08T23:46:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f91297fa700 1 -- 172.21.15.12:0/1953343098 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120017650 con 0x7f912c1020f0 2024-05-08T23:46:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f91328df700 1 -- 172.21.15.12:0/1953343098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c19efb0 con 0x7f912c1020f0 2024-05-08T23:46:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.122+0000 7f91328df700 1 -- 172.21.15.12:0/1953343098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c19f370 con 0x7f912c1020f0 2024-05-08T23:46:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.123+0000 7f91328df700 1 -- 172.21.15.12:0/1953343098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c108430 con 0x7f912c1020f0 2024-05-08T23:46:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.124+0000 7f91297fa700 1 -- 172.21.15.12:0/1953343098 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f91200225e0 con 0x7f912c1020f0 2024-05-08T23:46:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.125+0000 7f91297fa700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f91140a1230 0x7f91140a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.125+0000 7f912bfff700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f91140a1230 0x7f91140a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.125+0000 7f91297fa700 1 -- 172.21.15.12:0/1953343098 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f91200ce0e0 con 0x7f912c1020f0 2024-05-08T23:46:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.125+0000 7f912bfff700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f91140a1230 0x7f91140a36f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f912c100c40 tx=0x7f911c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:12.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.129+0000 7f91297fa700 1 -- 172.21.15.12:0/1953343098 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f91200977a0 con 0x7f912c1020f0 2024-05-08T23:46:12.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.347+0000 7f91328df700 1 -- 172.21.15.12:0/1953343098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f912c10c7e0 con 0x7f912c1020f0 2024-05-08T23:46:12.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.347+0000 7f91297fa700 1 -- 172.21.15.12:0/1953343098 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91200228e0 con 0x7f912c1020f0 2024-05-08T23:46:12.343 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-08T23:46:12.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.349+0000 7f91328df700 1 -- 172.21.15.12:0/1953343098 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f91140a1230 msgr2=0x7f91140a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.349+0000 7f91328df700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f91140a1230 0x7f91140a36f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f912c100c40 tx=0x7f911c009260 comp rx=0 tx=0).stop 2024-05-08T23:46:12.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.349+0000 7f91328df700 1 -- 172.21.15.12:0/1953343098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f912c1020f0 msgr2=0x7f912c197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.349+0000 7f91328df700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f912c1020f0 0x7f912c197470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f912000b620 tx=0x7f9120004730 comp rx=0 tx=0).stop 2024-05-08T23:46:12.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.350+0000 7f91328df700 1 -- 172.21.15.12:0/1953343098 shutdown_connections 2024-05-08T23:46:12.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.350+0000 7f91328df700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c102ab0 0x7f912c19e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.350+0000 7f91328df700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f91140a1230 0x7f91140a36f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.350+0000 7f91328df700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f912c100f20 0x7f912c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.350+0000 7f91328df700 1 --2- 172.21.15.12:0/1953343098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f912c1020f0 0x7f912c197470 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.350+0000 7f91328df700 1 -- 172.21.15.12:0/1953343098 >> 172.21.15.12:0/1953343098 conn(0x7f912c0fc4e0 msgr2=0x7f912c105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:12.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.350+0000 7f91328df700 1 -- 172.21.15.12:0/1953343098 shutdown_connections 2024-05-08T23:46:12.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.350+0000 7f91328df700 1 -- 172.21.15.12:0/1953343098 wait complete. 2024-05-08T23:46:12.360 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:46:12.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 -- 172.21.15.12:0/2579211858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8106ce0 msgr2=0x7ff5c81090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 --2- 172.21.15.12:0/2579211858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8106ce0 0x7ff5c81090d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000b0a0 tx=0x7ff5c000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 -- 172.21.15.12:0/2579211858 shutdown_connections 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 --2- 172.21.15.12:0/2579211858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8106ce0 0x7ff5c81090d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 --2- 172.21.15.12:0/2579211858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c8101a80 0x7ff5c8103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 --2- 172.21.15.12:0/2579211858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5c81043b0 0x7ff5c81067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 -- 172.21.15.12:0/2579211858 >> 172.21.15.12:0/2579211858 conn(0x7ff5c80fb3f0 msgr2=0x7ff5c80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 -- 172.21.15.12:0/2579211858 shutdown_connections 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 -- 172.21.15.12:0/2579211858 wait complete. 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 Processor -- start 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 -- start start 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c8101a80 0x7ff5c8196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5c81043b0 0x7ff5c8197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8106ce0 0x7ff5c819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c819eab0 con 0x7ff5c8106ce0 2024-05-08T23:46:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c819ebf0 con 0x7ff5c81043b0 2024-05-08T23:46:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.504+0000 7ff5cebc5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c819ed80 con 0x7ff5c8101a80 2024-05-08T23:46:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cd162700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8106ce0 0x7ff5c819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cd162700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8106ce0 0x7ff5c819e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33596/0 (socket says 172.21.15.12:33596) 2024-05-08T23:46:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cd162700 1 -- 172.21.15.12:0/896117157 learned_addr learned my addr 172.21.15.12:0/896117157 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cc961700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c8101a80 0x7ff5c8196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5bffff700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5c81043b0 0x7ff5c8197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cd162700 1 -- 172.21.15.12:0/896117157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c8101a80 msgr2=0x7ff5c8196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cd162700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c8101a80 0x7ff5c8196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cd162700 1 -- 172.21.15.12:0/896117157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5c81043b0 msgr2=0x7ff5c8197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cd162700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5c81043b0 0x7ff5c8197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cd162700 1 -- 172.21.15.12:0/896117157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0009d00 con 0x7ff5c8106ce0 2024-05-08T23:46:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5bffff700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5c81043b0 0x7ff5c8197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cd162700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8106ce0 0x7ff5c819e4e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0004d00 tx=0x7ff5c0004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5bdffb700 1 -- 172.21.15.12:0/896117157 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000e040 con 0x7ff5c8106ce0 2024-05-08T23:46:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5bdffb700 1 -- 172.21.15.12:0/896117157 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5c000cec0 con 0x7ff5c8106ce0 2024-05-08T23:46:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5cebc5700 1 -- 172.21.15.12:0/896117157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c819f000 con 0x7ff5c8106ce0 2024-05-08T23:46:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.505+0000 7ff5bdffb700 1 -- 172.21.15.12:0/896117157 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c001ba80 con 0x7ff5c8106ce0 2024-05-08T23:46:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.506+0000 7ff5cebc5700 1 -- 172.21.15.12:0/896117157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c819f480 con 0x7ff5c8106ce0 2024-05-08T23:46:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.507+0000 7ff5cebc5700 1 -- 172.21.15.12:0/896117157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c80fcf70 con 0x7ff5c8106ce0 2024-05-08T23:46:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.509+0000 7ff5bdffb700 1 -- 172.21.15.12:0/896117157 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff5c001bbe0 con 0x7ff5c8106ce0 2024-05-08T23:46:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.509+0000 7ff5bdffb700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff5b00a11b0 0x7ff5b00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.509+0000 7ff5cc961700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff5b00a11b0 0x7ff5b00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.509+0000 7ff5bdffb700 1 -- 172.21.15.12:0/896117157 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff5c00ceaa0 con 0x7ff5c8106ce0 2024-05-08T23:46:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.510+0000 7ff5cc961700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff5b00a11b0 0x7ff5b00a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005950 tx=0x7ff5c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.513+0000 7ff5bdffb700 1 -- 172.21.15.12:0/896117157 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff5c0098260 con 0x7ff5c8106ce0 2024-05-08T23:46:12.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:12 smithi012 ceph-mon[42503]: pgmap v261: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:46:12.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:12 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2723038542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:12.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.732+0000 7ff5cebc5700 1 -- 172.21.15.12:0/896117157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5c810c850 con 0x7ff5c8106ce0 2024-05-08T23:46:12.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.733+0000 7ff5bdffb700 1 -- 172.21.15.12:0/896117157 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5c009b880 con 0x7ff5c8106ce0 2024-05-08T23:46:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-08T23:46:12.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 -- 172.21.15.12:0/896117157 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff5b00a11b0 msgr2=0x7ff5b00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff5b00a11b0 0x7ff5b00a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005950 tx=0x7ff5c400b410 comp rx=0 tx=0).stop 2024-05-08T23:46:12.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 -- 172.21.15.12:0/896117157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8106ce0 msgr2=0x7ff5c819e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8106ce0 0x7ff5c819e4e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0004d00 tx=0x7ff5c0004d30 comp rx=0 tx=0).stop 2024-05-08T23:46:12.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 -- 172.21.15.12:0/896117157 shutdown_connections 2024-05-08T23:46:12.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5c8106ce0 0x7ff5c819e4e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5c8101a80 0x7ff5c8196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7ff5b00a11b0 0x7ff5b00a3670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 --2- 172.21.15.12:0/896117157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5c81043b0 0x7ff5c8197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 -- 172.21.15.12:0/896117157 >> 172.21.15.12:0/896117157 conn(0x7ff5c80fb3f0 msgr2=0x7ff5c8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:12.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 -- 172.21.15.12:0/896117157 shutdown_connections 2024-05-08T23:46:12.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.735+0000 7ff5cebc5700 1 -- 172.21.15.12:0/896117157 wait complete. 2024-05-08T23:46:12.745 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:46:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:12 smithi204 ceph-mon[47373]: pgmap v261: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:46:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:12 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2723038542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:12 smithi160 ceph-mon[46152]: pgmap v261: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:46:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:12 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2723038542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:12.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 -- 172.21.15.12:0/61359331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f81043d0 msgr2=0x7f22f81067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 --2- 172.21.15.12:0/61359331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f81043d0 0x7f22f81067c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009990 tx=0x7f22e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 -- 172.21.15.12:0/61359331 shutdown_connections 2024-05-08T23:46:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 --2- 172.21.15.12:0/61359331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22f8106d00 0x7f22f81090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 --2- 172.21.15.12:0/61359331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22f8101aa0 0x7f22f8103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 --2- 172.21.15.12:0/61359331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f81043d0 0x7f22f81067c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 -- 172.21.15.12:0/61359331 >> 172.21.15.12:0/61359331 conn(0x7f22f80fb3f0 msgr2=0x7f22f80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 -- 172.21.15.12:0/61359331 shutdown_connections 2024-05-08T23:46:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 -- 172.21.15.12:0/61359331 wait complete. 2024-05-08T23:46:12.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 Processor -- start 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 -- start start 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.890+0000 7f230055d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f8101aa0 0x7f22f8196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f230055d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22f81043d0 0x7f22f81973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f230055d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22f8106d00 0x7f22f819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f230055d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f819e9d0 con 0x7f22f8101aa0 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f230055d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f819eb10 con 0x7f22f8106d00 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f230055d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f819eca0 con 0x7f22f81043d0 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22fe2f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f8101aa0 0x7f22f8196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22fe2f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f8101aa0 0x7f22f8196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33614/0 (socket says 172.21.15.12:33614) 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22fe2f9700 1 -- 172.21.15.12:0/2822483659 learned_addr learned my addr 172.21.15.12:0/2822483659 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22fdaf8700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22f81043d0 0x7f22f81973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22feafa700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22f8106d00 0x7f22f819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22fe2f9700 1 -- 172.21.15.12:0/2822483659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22f81043d0 msgr2=0x7f22f81973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22fe2f9700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22f81043d0 0x7f22f81973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22fe2f9700 1 -- 172.21.15.12:0/2822483659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22f8106d00 msgr2=0x7f22f819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22fe2f9700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22f8106d00 0x7f22f819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22fe2f9700 1 -- 172.21.15.12:0/2822483659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e80095f0 con 0x7f22f8101aa0 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22fe2f9700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f8101aa0 0x7f22f8196ea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f22f400e9a0 tx=0x7f22f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22ef7fe700 1 -- 172.21.15.12:0/2822483659 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f4009790 con 0x7f22f8101aa0 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f230055d700 1 -- 172.21.15.12:0/2822483659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f819ef80 con 0x7f22f8101aa0 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22ef7fe700 1 -- 172.21.15.12:0/2822483659 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22f4010430 con 0x7f22f8101aa0 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f22ef7fe700 1 -- 172.21.15.12:0/2822483659 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f4018680 con 0x7f22f8101aa0 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.891+0000 7f230055d700 1 -- 172.21.15.12:0/2822483659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f819f3a0 con 0x7f22f8101aa0 2024-05-08T23:46:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.892+0000 7f230055d700 1 -- 172.21.15.12:0/2822483659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f80fcf70 con 0x7f22f8101aa0 2024-05-08T23:46:12.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.894+0000 7f22ef7fe700 1 -- 172.21.15.12:0/2822483659 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f22f40105a0 con 0x7f22f8101aa0 2024-05-08T23:46:12.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.895+0000 7f22ef7fe700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f22e40a11b0 0x7f22e40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:12.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.895+0000 7f22fdaf8700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f22e40a11b0 0x7f22e40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:12.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.895+0000 7f22ef7fe700 1 -- 172.21.15.12:0/2822483659 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f22f4014070 con 0x7f22f8101aa0 2024-05-08T23:46:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.895+0000 7f22fdaf8700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f22e40a11b0 0x7f22e40a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f22e800b2e0 tx=0x7f22e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:12.898+0000 7f22ef7fe700 1 -- 172.21.15.12:0/2822483659 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f22f4096420 con 0x7f22f8101aa0 2024-05-08T23:46:13.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.115+0000 7f230055d700 1 -- 172.21.15.12:0/2822483659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f22f810c990 con 0x7f22f8101aa0 2024-05-08T23:46:13.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.116+0000 7f22ef7fe700 1 -- 172.21.15.12:0/2822483659 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f22f40260a0 con 0x7f22f8101aa0 2024-05-08T23:46:13.112 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-08T23:46:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.118+0000 7f230055d700 1 -- 172.21.15.12:0/2822483659 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f22e40a11b0 msgr2=0x7f22e40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.118+0000 7f230055d700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f22e40a11b0 0x7f22e40a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f22e800b2e0 tx=0x7f22e8005fd0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.118+0000 7f230055d700 1 -- 172.21.15.12:0/2822483659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f8101aa0 msgr2=0x7f22f8196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.118+0000 7f230055d700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f8101aa0 0x7f22f8196ea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f22f400e9a0 tx=0x7f22f400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.119+0000 7f230055d700 1 -- 172.21.15.12:0/2822483659 shutdown_connections 2024-05-08T23:46:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.119+0000 7f230055d700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f22e40a11b0 0x7f22e40a3670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.119+0000 7f230055d700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22f8106d00 0x7f22f819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.119+0000 7f230055d700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f8101aa0 0x7f22f8196ea0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.119+0000 7f230055d700 1 --2- 172.21.15.12:0/2822483659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22f81043d0 0x7f22f81973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.119+0000 7f230055d700 1 -- 172.21.15.12:0/2822483659 >> 172.21.15.12:0/2822483659 conn(0x7f22f80fb3f0 msgr2=0x7f22f8102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.119+0000 7f230055d700 1 -- 172.21.15.12:0/2822483659 shutdown_connections 2024-05-08T23:46:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.119+0000 7f230055d700 1 -- 172.21.15.12:0/2822483659 wait complete. 2024-05-08T23:46:13.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:46:13.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.270+0000 7f5299a22700 1 -- 172.21.15.12:0/4201230967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5294101900 msgr2=0x7f52941081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.270+0000 7f5299a22700 1 --2- 172.21.15.12:0/4201230967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5294101900 0x7f52941081b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f528c00b0d0 tx=0x7f528c00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.270+0000 7f5299a22700 1 -- 172.21.15.12:0/4201230967 shutdown_connections 2024-05-08T23:46:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.270+0000 7f5299a22700 1 --2- 172.21.15.12:0/4201230967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5294101900 0x7f52941081b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.270+0000 7f5299a22700 1 --2- 172.21.15.12:0/4201230967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52940ffd70 0x7f5294100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.270+0000 7f5299a22700 1 --2- 172.21.15.12:0/4201230967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5294100f60 0x7f52941013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.270+0000 7f5299a22700 1 -- 172.21.15.12:0/4201230967 >> 172.21.15.12:0/4201230967 conn(0x7f52940fb2e0 msgr2=0x7f52940fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.270+0000 7f5299a22700 1 -- 172.21.15.12:0/4201230967 shutdown_connections 2024-05-08T23:46:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.270+0000 7f5299a22700 1 -- 172.21.15.12:0/4201230967 wait complete. 2024-05-08T23:46:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5299a22700 1 Processor -- start 2024-05-08T23:46:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5299a22700 1 -- start start 2024-05-08T23:46:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5299a22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52940ffd70 0x7f5294196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5299a22700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5294100f60 0x7f5294197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5299a22700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5294101900 0x7f529419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5299a22700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529419e9f0 con 0x7f52940ffd70 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5299a22700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529419eb30 con 0x7f5294100f60 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5299a22700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529419ecc0 con 0x7f5294101900 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5292ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52940ffd70 0x7f5294196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5292ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52940ffd70 0x7f5294196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33638/0 (socket says 172.21.15.12:33638) 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f5292ffd700 1 -- 172.21.15.12:0/3035177624 learned_addr learned my addr 172.21.15.12:0/3035177624 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f52937fe700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5294101900 0x7f529419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.271+0000 7f52927fc700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5294100f60 0x7f5294197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5292ffd700 1 -- 172.21.15.12:0/3035177624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5294101900 msgr2=0x7f529419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5292ffd700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5294101900 0x7f529419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5292ffd700 1 -- 172.21.15.12:0/3035177624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5294100f60 msgr2=0x7f5294197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5292ffd700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5294100f60 0x7f5294197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5292ffd700 1 -- 172.21.15.12:0/3035177624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5284009520 con 0x7f52940ffd70 2024-05-08T23:46:13.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5292ffd700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52940ffd70 0x7f5294196f10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f528400ea50 tx=0x7f528400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:13.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5298a20700 1 -- 172.21.15.12:0/3035177624 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528400cbd0 con 0x7f52940ffd70 2024-05-08T23:46:13.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5299a22700 1 -- 172.21.15.12:0/3035177624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c009d00 con 0x7f52940ffd70 2024-05-08T23:46:13.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5298a20700 1 -- 172.21.15.12:0/3035177624 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f528400cd30 con 0x7f52940ffd70 2024-05-08T23:46:13.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5298a20700 1 -- 172.21.15.12:0/3035177624 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284018680 con 0x7f52940ffd70 2024-05-08T23:46:13.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.272+0000 7f5299a22700 1 -- 172.21.15.12:0/3035177624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529419f290 con 0x7f52940ffd70 2024-05-08T23:46:13.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.273+0000 7f5299a22700 1 -- 172.21.15.12:0/3035177624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5294068060 con 0x7f52940ffd70 2024-05-08T23:46:13.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.274+0000 7f5298a20700 1 -- 172.21.15.12:0/3035177624 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5284010430 con 0x7f52940ffd70 2024-05-08T23:46:13.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.275+0000 7f5298a20700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f527c0a1230 0x7f527c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:13.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.275+0000 7f52927fc700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f527c0a1230 0x7f527c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:13.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.275+0000 7f5298a20700 1 -- 172.21.15.12:0/3035177624 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5284014070 con 0x7f52940ffd70 2024-05-08T23:46:13.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.276+0000 7f52927fc700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f527c0a1230 0x7f527c0a36f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5288005950 tx=0x7f528800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:13.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.280+0000 7f5298a20700 1 -- 172.21.15.12:0/3035177624 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5284096610 con 0x7f52940ffd70 2024-05-08T23:46:13.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:13 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1953343098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:13.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:13 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/896117157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:13.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:13 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2822483659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:13.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.549+0000 7f5299a22700 1 -- 172.21.15.12:0/3035177624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f529410b590 con 0x7f52940ffd70 2024-05-08T23:46:13.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.550+0000 7f5298a20700 1 -- 172.21.15.12:0/3035177624 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10876 (secure 0 0 0) 0x7f5284099c30 con 0x7f52940ffd70 2024-05-08T23:46:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:13.546 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:45:09.067610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 -- 172.21.15.12:0/3035177624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f527c0a1230 msgr2=0x7f527c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f527c0a1230 0x7f527c0a36f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5288005950 tx=0x7f528800b410 comp rx=0 tx=0).stop 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 -- 172.21.15.12:0/3035177624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52940ffd70 msgr2=0x7f5294196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52940ffd70 0x7f5294196f10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f528400ea50 tx=0x7f528400ed60 comp rx=0 tx=0).stop 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 -- 172.21.15.12:0/3035177624 shutdown_connections 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5294101900 0x7f529419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52940ffd70 0x7f5294196f10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f527c0a1230 0x7f527c0a36f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 --2- 172.21.15.12:0/3035177624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5294100f60 0x7f5294197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 -- 172.21.15.12:0/3035177624 >> 172.21.15.12:0/3035177624 conn(0x7f52940fb2e0 msgr2=0x7f52940fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:13.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 -- 172.21.15.12:0/3035177624 shutdown_connections 2024-05-08T23:46:13.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.553+0000 7f5299a22700 1 -- 172.21.15.12:0/3035177624 wait complete. 2024-05-08T23:46:13.550 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 23 2024-05-08T23:46:13.563 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.705+0000 7f792a3ac700 1 -- 172.21.15.12:0/4288953610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7924101fc0 msgr2=0x7f7924102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.705+0000 7f792a3ac700 1 --2- 172.21.15.12:0/4288953610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7924101fc0 0x7f7924102440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7918009990 tx=0x7f7918009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.705+0000 7f792a3ac700 1 -- 172.21.15.12:0/4288953610 shutdown_connections 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.705+0000 7f792a3ac700 1 --2- 172.21.15.12:0/4288953610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7924102980 0x7f79241092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.705+0000 7f792a3ac700 1 --2- 172.21.15.12:0/4288953610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7924100e70 0x7f7924101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.705+0000 7f792a3ac700 1 --2- 172.21.15.12:0/4288953610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7924101fc0 0x7f7924102440 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.705+0000 7f792a3ac700 1 -- 172.21.15.12:0/4288953610 >> 172.21.15.12:0/4288953610 conn(0x7f79240fc400 msgr2=0x7f79240fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.705+0000 7f792a3ac700 1 -- 172.21.15.12:0/4288953610 shutdown_connections 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.706+0000 7f792a3ac700 1 -- 172.21.15.12:0/4288953610 wait complete. 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.706+0000 7f792a3ac700 1 Processor -- start 2024-05-08T23:46:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.706+0000 7f792a3ac700 1 -- start start 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.706+0000 7f792a3ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7924100e70 0x7f7924196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f792a3ac700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7924101fc0 0x7f7924197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f792a3ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7924102980 0x7f792419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f792a3ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792419e8f0 con 0x7f7924100e70 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f792a3ac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792419ea60 con 0x7f7924102980 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f792a3ac700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792419ec20 con 0x7f7924101fc0 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7923fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7924100e70 0x7f7924196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7923fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7924100e70 0x7f7924196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33642/0 (socket says 172.21.15.12:33642) 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7923fff700 1 -- 172.21.15.12:0/2735713198 learned_addr learned my addr 172.21.15.12:0/2735713198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f79237fe700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7924101fc0 0x7f7924197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7928949700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7924102980 0x7f792419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7923fff700 1 -- 172.21.15.12:0/2735713198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7924101fc0 msgr2=0x7f7924197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7923fff700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7924101fc0 0x7f7924197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7923fff700 1 -- 172.21.15.12:0/2735713198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7924102980 msgr2=0x7f792419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7923fff700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7924102980 0x7f792419e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7923fff700 1 -- 172.21.15.12:0/2735713198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79180095f0 con 0x7f7924100e70 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f79237fe700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7924101fc0 0x7f7924197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7928949700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7924102980 0x7f792419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f7923fff700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7924100e70 0x7f7924196e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f791400c920 tx=0x7f791400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f79217fa700 1 -- 172.21.15.12:0/2735713198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914007b00 con 0x7f7924100e70 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f79217fa700 1 -- 172.21.15.12:0/2735713198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7914004cf0 con 0x7f7924100e70 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f792a3ac700 1 -- 172.21.15.12:0/2735713198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792419ef00 con 0x7f7924100e70 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.707+0000 7f79217fa700 1 -- 172.21.15.12:0/2735713198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914005610 con 0x7f7924100e70 2024-05-08T23:46:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.708+0000 7f792a3ac700 1 -- 172.21.15.12:0/2735713198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792419f330 con 0x7f7924100e70 2024-05-08T23:46:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.709+0000 7f792a3ac700 1 -- 172.21.15.12:0/2735713198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79241082e0 con 0x7f7924100e70 2024-05-08T23:46:13.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.710+0000 7f79217fa700 1 -- 172.21.15.12:0/2735713198 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7914004750 con 0x7f7924100e70 2024-05-08T23:46:13.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.710+0000 7f79217fa700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f790c0a1230 0x7f790c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:13.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.711+0000 7f79217fa700 1 -- 172.21.15.12:0/2735713198 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f79140cc860 con 0x7f7924100e70 2024-05-08T23:46:13.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.711+0000 7f79237fe700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f790c0a1230 0x7f790c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:13.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.711+0000 7f79237fe700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f790c0a1230 0x7f790c0a36f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7918006010 tx=0x7f79180037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:13.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.714+0000 7f79217fa700 1 -- 172.21.15.12:0/2735713198 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7914096010 con 0x7f7924100e70 2024-05-08T23:46:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:13 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1953343098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:13 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/896117157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:13.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:13 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2822483659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:13.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:13 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1953343098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:13.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:13 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/896117157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:13.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:13 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2822483659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:13.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.936+0000 7f792a3ac700 1 -- 172.21.15.12:0/2735713198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7924067380 con 0x7f7924100e70 2024-05-08T23:46:13.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.937+0000 7f79217fa700 1 -- 172.21.15.12:0/2735713198 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22445 (secure 0 0 0) 0x7f791401d070 con 0x7f7924100e70 2024-05-08T23:46:13.933 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:13.933 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:56.347006+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:46:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.939+0000 7f792a3ac700 1 -- 172.21.15.12:0/2735713198 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f790c0a1230 msgr2=0x7f790c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f790c0a1230 0x7f790c0a36f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7918006010 tx=0x7f79180037b0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 -- 172.21.15.12:0/2735713198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7924100e70 msgr2=0x7f7924196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7924100e70 0x7f7924196e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f791400c920 tx=0x7f791400cce0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 -- 172.21.15.12:0/2735713198 shutdown_connections 2024-05-08T23:46:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f790c0a1230 0x7f790c0a36f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7924102980 0x7f792419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7924100e70 0x7f7924196e00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 --2- 172.21.15.12:0/2735713198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7924101fc0 0x7f7924197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 -- 172.21.15.12:0/2735713198 >> 172.21.15.12:0/2735713198 conn(0x7f79240fc400 msgr2=0x7f7924105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 -- 172.21.15.12:0/2735713198 shutdown_connections 2024-05-08T23:46:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:13.940+0000 7f792a3ac700 1 -- 172.21.15.12:0/2735713198 wait complete. 2024-05-08T23:46:13.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:45:02.165326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:45:03.407498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:46:13.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:46:14.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.095+0000 7fb9c4150700 1 -- 172.21.15.12:0/2239671288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9bc101980 msgr2=0x7fb9bc1082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.095+0000 7fb9c4150700 1 --2- 172.21.15.12:0/2239671288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9bc101980 0x7fb9bc1082a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b0a0 tx=0x7fb9b400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.095+0000 7fb9c4150700 1 -- 172.21.15.12:0/2239671288 shutdown_connections 2024-05-08T23:46:14.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.095+0000 7fb9c4150700 1 --2- 172.21.15.12:0/2239671288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9bc101980 0x7fb9bc1082a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.095+0000 7fb9c4150700 1 --2- 172.21.15.12:0/2239671288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9bc0ffe70 0x7fb9bc100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.095+0000 7fb9c4150700 1 --2- 172.21.15.12:0/2239671288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc100fc0 0x7fb9bc101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.095+0000 7fb9c4150700 1 -- 172.21.15.12:0/2239671288 >> 172.21.15.12:0/2239671288 conn(0x7fb9bc0fb400 msgr2=0x7fb9bc0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.095+0000 7fb9c4150700 1 -- 172.21.15.12:0/2239671288 shutdown_connections 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.095+0000 7fb9c4150700 1 -- 172.21.15.12:0/2239671288 wait complete. 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c4150700 1 Processor -- start 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c4150700 1 -- start start 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c4150700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9bc0ffe70 0x7fb9bc196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c4150700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc100fc0 0x7fb9bc1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c4150700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9bc101980 0x7fb9bc19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c4150700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc19eae0 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c4150700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc19ec20 con 0x7fb9bc101980 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c4150700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc19edb0 con 0x7fb9bc100fc0 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c1eec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9bc0ffe70 0x7fb9bc196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c1eec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9bc0ffe70 0x7fb9bc196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33648/0 (socket says 172.21.15.12:33648) 2024-05-08T23:46:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c1eec700 1 -- 172.21.15.12:0/3535988857 learned_addr learned my addr 172.21.15.12:0/3535988857 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c16eb700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc100fc0 0x7fb9bc1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.096+0000 7fb9c26ed700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9bc101980 0x7fb9bc19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9c1eec700 1 -- 172.21.15.12:0/3535988857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc100fc0 msgr2=0x7fb9bc1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9c1eec700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc100fc0 0x7fb9bc1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9c1eec700 1 -- 172.21.15.12:0/3535988857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9bc101980 msgr2=0x7fb9bc19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9c1eec700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9bc101980 0x7fb9bc19e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9c1eec700 1 -- 172.21.15.12:0/3535988857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b4009d00 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9c1eec700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9bc0ffe70 0x7fb9bc196fb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800e9a0 tx=0x7fb9b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9b2ffd700 1 -- 172.21.15.12:0/3535988857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8009790 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9b2ffd700 1 -- 172.21.15.12:0/3535988857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b8010430 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9c4150700 1 -- 172.21.15.12:0/3535988857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc19f090 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9b2ffd700 1 -- 172.21.15.12:0/3535988857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8018680 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.097+0000 7fb9c4150700 1 -- 172.21.15.12:0/3535988857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc19f5a0 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.098+0000 7fb9c4150700 1 -- 172.21.15.12:0/3535988857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9bc068140 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.099+0000 7fb9b2ffd700 1 -- 172.21.15.12:0/3535988857 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb9b80105a0 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.100+0000 7fb9b2ffd700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb9a80a11b0 0x7fb9a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.100+0000 7fb9c16eb700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb9a80a11b0 0x7fb9a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.100+0000 7fb9b2ffd700 1 -- 172.21.15.12:0/3535988857 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb9b8014070 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.100+0000 7fb9c16eb700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb9a80a11b0 0x7fb9a80a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009c60 tx=0x7fb9ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.104+0000 7fb9b2ffd700 1 -- 172.21.15.12:0/3535988857 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb9b8096430 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.320+0000 7fb9c4150700 1 -- 172.21.15.12:0/3535988857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9bc10b820 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.321+0000 7fb9b2ffd700 1 -- 172.21.15.12:0/3535988857 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9b80260a0 con 0x7fb9bc0ffe70 2024-05-08T23:46:14.317 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-08T23:46:14.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.323+0000 7fb9c4150700 1 -- 172.21.15.12:0/3535988857 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb9a80a11b0 msgr2=0x7fb9a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.323+0000 7fb9c4150700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb9a80a11b0 0x7fb9a80a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009c60 tx=0x7fb9ac009260 comp rx=0 tx=0).stop 2024-05-08T23:46:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.323+0000 7fb9c4150700 1 -- 172.21.15.12:0/3535988857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9bc0ffe70 msgr2=0x7fb9bc196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.323+0000 7fb9c4150700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9bc0ffe70 0x7fb9bc196fb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800e9a0 tx=0x7fb9b800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.323+0000 7fb9c4150700 1 -- 172.21.15.12:0/3535988857 shutdown_connections 2024-05-08T23:46:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.323+0000 7fb9c4150700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fb9a80a11b0 0x7fb9a80a3670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.323+0000 7fb9c4150700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9bc101980 0x7fb9bc19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.323+0000 7fb9c4150700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9bc0ffe70 0x7fb9bc196fb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.323+0000 7fb9c4150700 1 --2- 172.21.15.12:0/3535988857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc100fc0 0x7fb9bc1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.323+0000 7fb9c4150700 1 -- 172.21.15.12:0/3535988857 >> 172.21.15.12:0/3535988857 conn(0x7fb9bc0fb400 msgr2=0x7fb9bc0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.324+0000 7fb9c4150700 1 -- 172.21.15.12:0/3535988857 shutdown_connections 2024-05-08T23:46:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.324+0000 7fb9c4150700 1 -- 172.21.15.12:0/3535988857 wait complete. 2024-05-08T23:46:14.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:46:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.478+0000 7f5d8e040700 1 -- 172.21.15.12:0/1551898350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d881020f0 msgr2=0x7f5d88102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.478+0000 7f5d8e040700 1 --2- 172.21.15.12:0/1551898350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d881020f0 0x7f5d88102570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009990 tx=0x7f5d78009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.478+0000 7f5d8e040700 1 -- 172.21.15.12:0/1551898350 shutdown_connections 2024-05-08T23:46:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.478+0000 7f5d8e040700 1 --2- 172.21.15.12:0/1551898350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d88102ab0 0x7f5d881093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.478+0000 7f5d8e040700 1 --2- 172.21.15.12:0/1551898350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88100f20 0x7f5d88101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.478+0000 7f5d8e040700 1 --2- 172.21.15.12:0/1551898350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d881020f0 0x7f5d88102570 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.478+0000 7f5d8e040700 1 -- 172.21.15.12:0/1551898350 >> 172.21.15.12:0/1551898350 conn(0x7f5d880fc4e0 msgr2=0x7f5d880fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.479+0000 7f5d8e040700 1 -- 172.21.15.12:0/1551898350 shutdown_connections 2024-05-08T23:46:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.479+0000 7f5d8e040700 1 -- 172.21.15.12:0/1551898350 wait complete. 2024-05-08T23:46:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.479+0000 7f5d8e040700 1 Processor -- start 2024-05-08T23:46:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d8e040700 1 -- start start 2024-05-08T23:46:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d8e040700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d88100f20 0x7f5d88196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d8e040700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d881020f0 0x7f5d88197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d8e040700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88102ab0 0x7f5d8819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d8e040700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8819ea10 con 0x7f5d88100f20 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d8e040700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8819eb50 con 0x7f5d881020f0 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d8e040700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8819ece0 con 0x7f5d88102ab0 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d877fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d88100f20 0x7f5d88196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d877fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d88100f20 0x7f5d88196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33672/0 (socket says 172.21.15.12:33672) 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d877fe700 1 -- 172.21.15.12:0/3122509728 learned_addr learned my addr 172.21.15.12:0/3122509728 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d87fff700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88102ab0 0x7f5d8819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d86ffd700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d881020f0 0x7f5d88197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d877fe700 1 -- 172.21.15.12:0/3122509728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88102ab0 msgr2=0x7f5d8819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d877fe700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88102ab0 0x7f5d8819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d877fe700 1 -- 172.21.15.12:0/3122509728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d881020f0 msgr2=0x7f5d88197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d877fe700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d881020f0 0x7f5d88197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d877fe700 1 -- 172.21.15.12:0/3122509728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d780095f0 con 0x7f5d88100f20 2024-05-08T23:46:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.480+0000 7f5d86ffd700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d881020f0 0x7f5d88197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.481+0000 7f5d877fe700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d88100f20 0x7f5d88196ee0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000e970 tx=0x7f5d7000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.481+0000 7f5d84ff9700 1 -- 172.21.15.12:0/3122509728 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70009790 con 0x7f5d88100f20 2024-05-08T23:46:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.481+0000 7f5d84ff9700 1 -- 172.21.15.12:0/3122509728 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d7000eec0 con 0x7f5d88100f20 2024-05-08T23:46:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.481+0000 7f5d8e040700 1 -- 172.21.15.12:0/3122509728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8819efc0 con 0x7f5d88100f20 2024-05-08T23:46:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.481+0000 7f5d84ff9700 1 -- 172.21.15.12:0/3122509728 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70003cf0 con 0x7f5d88100f20 2024-05-08T23:46:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.481+0000 7f5d8e040700 1 -- 172.21.15.12:0/3122509728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8819f3e0 con 0x7f5d88100f20 2024-05-08T23:46:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.482+0000 7f5d8e040700 1 -- 172.21.15.12:0/3122509728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d88108430 con 0x7f5d88100f20 2024-05-08T23:46:14.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.483+0000 7f5d84ff9700 1 -- 172.21.15.12:0/3122509728 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5d7000ccd0 con 0x7f5d88100f20 2024-05-08T23:46:14.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.484+0000 7f5d84ff9700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5d740a11b0 0x7f5d740a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.484+0000 7f5d84ff9700 1 -- 172.21.15.12:0/3122509728 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5d70014070 con 0x7f5d88100f20 2024-05-08T23:46:14.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.484+0000 7f5d86ffd700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5d740a11b0 0x7f5d740a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.484+0000 7f5d86ffd700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5d740a11b0 0x7f5d740a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800b750 tx=0x7f5d78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.488+0000 7f5d84ff9700 1 -- 172.21.15.12:0/3122509728 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5d70096460 con 0x7f5d88100f20 2024-05-08T23:46:14.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:14 smithi012 ceph-mon[42503]: pgmap v262: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:46:14.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:14 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3035177624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:14.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:14 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2735713198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:14.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:14 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3535988857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:14.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.712+0000 7f5d8e040700 1 -- 172.21.15.12:0/3122509728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d8810c790 con 0x7f5d88100f20 2024-05-08T23:46:14.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.713+0000 7f5d84ff9700 1 -- 172.21.15.12:0/3122509728 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d70099a80 con 0x7f5d88100f20 2024-05-08T23:46:14.709 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-08T23:46:14.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 -- 172.21.15.12:0/3122509728 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5d740a11b0 msgr2=0x7f5d740a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5d740a11b0 0x7f5d740a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800b750 tx=0x7f5d78005fb0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 -- 172.21.15.12:0/3122509728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d88100f20 msgr2=0x7f5d88196ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d88100f20 0x7f5d88196ee0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000e970 tx=0x7f5d7000ec80 comp rx=0 tx=0).stop 2024-05-08T23:46:14.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 -- 172.21.15.12:0/3122509728 shutdown_connections 2024-05-08T23:46:14.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88102ab0 0x7f5d8819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d88100f20 0x7f5d88196ee0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f5d740a11b0 0x7f5d740a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 --2- 172.21.15.12:0/3122509728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d881020f0 0x7f5d88197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 -- 172.21.15.12:0/3122509728 >> 172.21.15.12:0/3122509728 conn(0x7f5d880fc4e0 msgr2=0x7f5d88105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:14.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.715+0000 7f5d8e040700 1 -- 172.21.15.12:0/3122509728 shutdown_connections 2024-05-08T23:46:14.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.716+0000 7f5d8e040700 1 -- 172.21.15.12:0/3122509728 wait complete. 2024-05-08T23:46:14.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:46:14.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:14 smithi204 ceph-mon[47373]: pgmap v262: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:46:14.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:14 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3035177624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:14.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:14 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2735713198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:14.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:14 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3535988857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:14 smithi160 ceph-mon[46152]: pgmap v262: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:46:14.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:14 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3035177624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:14.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:14 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2735713198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:14.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:14 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3535988857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.871+0000 7f4648383700 1 -- 172.21.15.12:0/3870213287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4640101970 msgr2=0x7f4640108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.871+0000 7f4648383700 1 --2- 172.21.15.12:0/3870213287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4640101970 0x7f4640108290 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f463800b0a0 tx=0x7f463800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.871+0000 7f4648383700 1 -- 172.21.15.12:0/3870213287 shutdown_connections 2024-05-08T23:46:14.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.871+0000 7f4648383700 1 --2- 172.21.15.12:0/3870213287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4640101970 0x7f4640108290 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.871+0000 7f4648383700 1 --2- 172.21.15.12:0/3870213287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46400ffe60 0x7f4640100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.871+0000 7f4648383700 1 --2- 172.21.15.12:0/3870213287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4640100fb0 0x7f4640101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.871+0000 7f4648383700 1 -- 172.21.15.12:0/3870213287 >> 172.21.15.12:0/3870213287 conn(0x7f46400fb3f0 msgr2=0x7f46400fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:14.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.871+0000 7f4648383700 1 -- 172.21.15.12:0/3870213287 shutdown_connections 2024-05-08T23:46:14.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.871+0000 7f4648383700 1 -- 172.21.15.12:0/3870213287 wait complete. 2024-05-08T23:46:14.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4648383700 1 Processor -- start 2024-05-08T23:46:14.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4648383700 1 -- start start 2024-05-08T23:46:14.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4648383700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46400ffe60 0x7f4640103d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4648383700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4640100fb0 0x7f4640102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4648383700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4640101970 0x7f4640102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4648383700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4640103100 con 0x7f4640101970 2024-05-08T23:46:14.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4648383700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4640103270 con 0x7f4640100fb0 2024-05-08T23:46:14.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4648383700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46401a1090 con 0x7f46400ffe60 2024-05-08T23:46:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4646920700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4640101970 0x7f4640102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4646920700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4640101970 0x7f4640102960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33690/0 (socket says 172.21.15.12:33690) 2024-05-08T23:46:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4646920700 1 -- 172.21.15.12:0/758177204 learned_addr learned my addr 172.21.15.12:0/758177204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f464611f700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46400ffe60 0x7f4640103d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f464591e700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4640100fb0 0x7f4640102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4646920700 1 -- 172.21.15.12:0/758177204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46400ffe60 msgr2=0x7f4640103d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4646920700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46400ffe60 0x7f4640103d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4646920700 1 -- 172.21.15.12:0/758177204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4640100fb0 msgr2=0x7f4640102420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4646920700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4640100fb0 0x7f4640102420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4646920700 1 -- 172.21.15.12:0/758177204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4638009d00 con 0x7f4640101970 2024-05-08T23:46:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.872+0000 7f4646920700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4640101970 0x7f4640102960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4638006070 tx=0x7f463800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.873+0000 7f46377fe700 1 -- 172.21.15.12:0/758177204 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463800e040 con 0x7f4640101970 2024-05-08T23:46:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.873+0000 7f46377fe700 1 -- 172.21.15.12:0/758177204 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4638003c00 con 0x7f4640101970 2024-05-08T23:46:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.873+0000 7f4648383700 1 -- 172.21.15.12:0/758177204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46401a12c0 con 0x7f4640101970 2024-05-08T23:46:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.873+0000 7f46377fe700 1 -- 172.21.15.12:0/758177204 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463801b6b0 con 0x7f4640101970 2024-05-08T23:46:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.873+0000 7f4648383700 1 -- 172.21.15.12:0/758177204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46401a16e0 con 0x7f4640101970 2024-05-08T23:46:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.874+0000 7f4648383700 1 -- 172.21.15.12:0/758177204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4640067cf0 con 0x7f4640101970 2024-05-08T23:46:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.875+0000 7f46377fe700 1 -- 172.21.15.12:0/758177204 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f463801d480 con 0x7f4640101970 2024-05-08T23:46:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.875+0000 7f46377fe700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f462c0a11b0 0x7f462c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.876+0000 7f46377fe700 1 -- 172.21.15.12:0/758177204 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f46380ceb60 con 0x7f4640101970 2024-05-08T23:46:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.876+0000 7f464611f700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f462c0a11b0 0x7f462c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.876+0000 7f464611f700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f462c0a11b0 0x7f462c0a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f463c009c60 tx=0x7f463c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:14.879+0000 7f46377fe700 1 -- 172.21.15.12:0/758177204 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4638098220 con 0x7f4640101970 2024-05-08T23:46:15.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.099+0000 7f4648383700 1 -- 172.21.15.12:0/758177204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46401a1900 con 0x7f4640101970 2024-05-08T23:46:15.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.100+0000 7f46377fe700 1 -- 172.21.15.12:0/758177204 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f463809b840 con 0x7f4640101970 2024-05-08T23:46:15.095 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-08T23:46:15.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 -- 172.21.15.12:0/758177204 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f462c0a11b0 msgr2=0x7f462c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f462c0a11b0 0x7f462c0a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f463c009c60 tx=0x7f463c009260 comp rx=0 tx=0).stop 2024-05-08T23:46:15.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 -- 172.21.15.12:0/758177204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4640101970 msgr2=0x7f4640102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4640101970 0x7f4640102960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4638006070 tx=0x7f463800ba50 comp rx=0 tx=0).stop 2024-05-08T23:46:15.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 -- 172.21.15.12:0/758177204 shutdown_connections 2024-05-08T23:46:15.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4640101970 0x7f4640102960 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46400ffe60 0x7f4640103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f462c0a11b0 0x7f462c0a3670 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 --2- 172.21.15.12:0/758177204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4640100fb0 0x7f4640102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 -- 172.21.15.12:0/758177204 >> 172.21.15.12:0/758177204 conn(0x7f46400fb3f0 msgr2=0x7f46400fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:15.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 -- 172.21.15.12:0/758177204 shutdown_connections 2024-05-08T23:46:15.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.102+0000 7f4648383700 1 -- 172.21.15.12:0/758177204 wait complete. 2024-05-08T23:46:15.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:46:15.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.252+0000 7f39a6a01700 1 -- 172.21.15.12:0/100908298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101fc0 msgr2=0x7f39a0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.252+0000 7f39a6a01700 1 --2- 172.21.15.12:0/100908298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101fc0 0x7f39a0102440 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3994009990 tx=0x7f3994009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.252+0000 7f39a6a01700 1 -- 172.21.15.12:0/100908298 shutdown_connections 2024-05-08T23:46:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.252+0000 7f39a6a01700 1 --2- 172.21.15.12:0/100908298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39a0102980 0x7f39a01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.252+0000 7f39a6a01700 1 --2- 172.21.15.12:0/100908298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39a0100e70 0x7f39a0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.252+0000 7f39a6a01700 1 --2- 172.21.15.12:0/100908298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101fc0 0x7f39a0102440 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.252+0000 7f39a6a01700 1 -- 172.21.15.12:0/100908298 >> 172.21.15.12:0/100908298 conn(0x7f39a00fc400 msgr2=0x7f39a00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.252+0000 7f39a6a01700 1 -- 172.21.15.12:0/100908298 shutdown_connections 2024-05-08T23:46:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.252+0000 7f39a6a01700 1 -- 172.21.15.12:0/100908298 wait complete. 2024-05-08T23:46:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.253+0000 7f39a6a01700 1 Processor -- start 2024-05-08T23:46:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.253+0000 7f39a6a01700 1 -- start start 2024-05-08T23:46:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f39a6a01700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39a0100e70 0x7f39a0199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f39a6a01700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101fc0 0x7f39a0199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f39a6a01700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39a0102980 0x7f39a01a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f39a6a01700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a01a0b40 con 0x7f39a0101fc0 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f39a6a01700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a01a0c80 con 0x7f39a0100e70 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f39a6a01700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a01a0e10 con 0x7f39a0102980 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f399f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101fc0 0x7f39a0199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f399f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101fc0 0x7f39a0199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33714/0 (socket says 172.21.15.12:33714) 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f399f7fe700 1 -- 172.21.15.12:0/2576063916 learned_addr learned my addr 172.21.15.12:0/2576063916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f399ffff700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39a0100e70 0x7f39a0199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f39a4f9e700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39a0102980 0x7f39a01a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f399f7fe700 1 -- 172.21.15.12:0/2576063916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39a0102980 msgr2=0x7f39a01a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f399f7fe700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39a0102980 0x7f39a01a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f399f7fe700 1 -- 172.21.15.12:0/2576063916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39a0100e70 msgr2=0x7f39a0199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f399f7fe700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39a0100e70 0x7f39a0199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f399f7fe700 1 -- 172.21.15.12:0/2576063916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39940095f0 con 0x7f39a0101fc0 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.254+0000 7f399f7fe700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101fc0 0x7f39a0199550 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f399400b5c0 tx=0x7f3994004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.255+0000 7f399d7fa700 1 -- 172.21.15.12:0/2576063916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399401d070 con 0x7f39a0101fc0 2024-05-08T23:46:15.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.255+0000 7f39a6a01700 1 -- 172.21.15.12:0/2576063916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a01a1090 con 0x7f39a0101fc0 2024-05-08T23:46:15.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.255+0000 7f399d7fa700 1 -- 172.21.15.12:0/2576063916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3994003f60 con 0x7f39a0101fc0 2024-05-08T23:46:15.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.255+0000 7f399d7fa700 1 -- 172.21.15.12:0/2576063916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994022d10 con 0x7f39a0101fc0 2024-05-08T23:46:15.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.255+0000 7f39a6a01700 1 -- 172.21.15.12:0/2576063916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a01a1510 con 0x7f39a0101fc0 2024-05-08T23:46:15.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.256+0000 7f39a6a01700 1 -- 172.21.15.12:0/2576063916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a0067380 con 0x7f39a0101fc0 2024-05-08T23:46:15.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.257+0000 7f399d7fa700 1 -- 172.21.15.12:0/2576063916 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3994022470 con 0x7f39a0101fc0 2024-05-08T23:46:15.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.258+0000 7f399d7fa700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f39880a11b0 0x7f39880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:15.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.258+0000 7f399ffff700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f39880a11b0 0x7f39880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:15.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.258+0000 7f399d7fa700 1 -- 172.21.15.12:0/2576063916 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f39940cdb60 con 0x7f39a0101fc0 2024-05-08T23:46:15.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.259+0000 7f399ffff700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f39880a11b0 0x7f39880a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3990005950 tx=0x7f399000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:15.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.262+0000 7f399d7fa700 1 -- 172.21.15.12:0/2576063916 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3994097320 con 0x7f39a0101fc0 2024-05-08T23:46:15.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:15 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3122509728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:15.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:15 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/758177204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:15.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.537+0000 7f39a6a01700 1 -- 172.21.15.12:0/2576063916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39a010c820 con 0x7f39a0101fc0 2024-05-08T23:46:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.538+0000 7f399d7fa700 1 -- 172.21.15.12:0/2576063916 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10876 (secure 0 0 0) 0x7f399409a940 con 0x7f39a0101fc0 2024-05-08T23:46:15.533 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:45:09.067610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:46:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 -- 172.21.15.12:0/2576063916 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f39880a11b0 msgr2=0x7f39880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f39880a11b0 0x7f39880a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3990005950 tx=0x7f399000b410 comp rx=0 tx=0).stop 2024-05-08T23:46:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 -- 172.21.15.12:0/2576063916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101fc0 msgr2=0x7f39a0199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101fc0 0x7f39a0199550 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f399400b5c0 tx=0x7f3994004730 comp rx=0 tx=0).stop 2024-05-08T23:46:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 -- 172.21.15.12:0/2576063916 shutdown_connections 2024-05-08T23:46:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39a0102980 0x7f39a01a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f39880a11b0 0x7f39880a3670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39a0100e70 0x7f39a0199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 --2- 172.21.15.12:0/2576063916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39a0101fc0 0x7f39a0199550 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 -- 172.21.15.12:0/2576063916 >> 172.21.15.12:0/2576063916 conn(0x7f39a00fc400 msgr2=0x7f39a00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 -- 172.21.15.12:0/2576063916 shutdown_connections 2024-05-08T23:46:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.540+0000 7f39a6a01700 1 -- 172.21.15.12:0/2576063916 wait complete. 2024-05-08T23:46:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 23 2024-05-08T23:46:15.550 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:46:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.693+0000 7fa8e307b700 1 -- 172.21.15.12:0/1920395551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8dc106b40 msgr2=0x7fa8dc108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.693+0000 7fa8e307b700 1 --2- 172.21.15.12:0/1920395551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8dc106b40 0x7fa8dc108fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400b0a0 tx=0x7fa8d400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.693+0000 7fa8e307b700 1 -- 172.21.15.12:0/1920395551 shutdown_connections 2024-05-08T23:46:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.693+0000 7fa8e307b700 1 --2- 172.21.15.12:0/1920395551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8dc106b40 0x7fa8dc108fb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.693+0000 7fa8e307b700 1 --2- 172.21.15.12:0/1920395551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8dc1018e0 0x7fa8dc103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.693+0000 7fa8e307b700 1 --2- 172.21.15.12:0/1920395551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc104210 0x7fa8dc106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.693+0000 7fa8e307b700 1 -- 172.21.15.12:0/1920395551 >> 172.21.15.12:0/1920395551 conn(0x7fa8dc0fb2f0 msgr2=0x7fa8dc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.693+0000 7fa8e307b700 1 -- 172.21.15.12:0/1920395551 shutdown_connections 2024-05-08T23:46:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.693+0000 7fa8e307b700 1 -- 172.21.15.12:0/1920395551 wait complete. 2024-05-08T23:46:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.694+0000 7fa8e307b700 1 Processor -- start 2024-05-08T23:46:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.694+0000 7fa8e307b700 1 -- start start 2024-05-08T23:46:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.694+0000 7fa8e307b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8dc1018e0 0x7fa8dc196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.694+0000 7fa8e307b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc104210 0x7fa8dc197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.694+0000 7fa8e307b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8dc106b40 0x7fa8dc19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.694+0000 7fa8e307b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc19ea00 con 0x7fa8dc106b40 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.694+0000 7fa8e307b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc19eb40 con 0x7fa8dc1018e0 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.694+0000 7fa8e307b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc19ecd0 con 0x7fa8dc104210 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.694+0000 7fa8e1618700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8dc106b40 0x7fa8dc19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8e1618700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8dc106b40 0x7fa8dc19e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33746/0 (socket says 172.21.15.12:33746) 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8e1618700 1 -- 172.21.15.12:0/1168344324 learned_addr learned my addr 172.21.15.12:0/1168344324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8e0e17700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8dc1018e0 0x7fa8dc196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8d3fff700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc104210 0x7fa8dc197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8e1618700 1 -- 172.21.15.12:0/1168344324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc104210 msgr2=0x7fa8dc197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8e1618700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc104210 0x7fa8dc197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8e1618700 1 -- 172.21.15.12:0/1168344324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8dc1018e0 msgr2=0x7fa8dc196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8e1618700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8dc1018e0 0x7fa8dc196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8e1618700 1 -- 172.21.15.12:0/1168344324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d4009d00 con 0x7fa8dc106b40 2024-05-08T23:46:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8e1618700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8dc106b40 0x7fa8dc19e430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4007a60 tx=0x7fa8d4007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8d1ffb700 1 -- 172.21.15.12:0/1168344324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400e040 con 0x7fa8dc106b40 2024-05-08T23:46:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8d1ffb700 1 -- 172.21.15.12:0/1168344324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8d4004c50 con 0x7fa8dc106b40 2024-05-08T23:46:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8e307b700 1 -- 172.21.15.12:0/1168344324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc19ef50 con 0x7fa8dc106b40 2024-05-08T23:46:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.695+0000 7fa8d1ffb700 1 -- 172.21.15.12:0/1168344324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40052a0 con 0x7fa8dc106b40 2024-05-08T23:46:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.696+0000 7fa8e307b700 1 -- 172.21.15.12:0/1168344324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc19f370 con 0x7fa8dc106b40 2024-05-08T23:46:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.696+0000 7fa8e307b700 1 -- 172.21.15.12:0/1168344324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc0fce70 con 0x7fa8dc106b40 2024-05-08T23:46:15.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.698+0000 7fa8d1ffb700 1 -- 172.21.15.12:0/1168344324 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa8d4005610 con 0x7fa8dc106b40 2024-05-08T23:46:15.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.699+0000 7fa8d1ffb700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa8c40a11b0 0x7fa8c40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:15.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.699+0000 7fa8e0e17700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa8c40a11b0 0x7fa8c40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:15.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.699+0000 7fa8d1ffb700 1 -- 172.21.15.12:0/1168344324 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa8d40ce450 con 0x7fa8dc106b40 2024-05-08T23:46:15.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.699+0000 7fa8e0e17700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa8c40a11b0 0x7fa8c40a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009c60 tx=0x7fa8d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:15.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.703+0000 7fa8d1ffb700 1 -- 172.21.15.12:0/1168344324 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa8d4097c10 con 0x7fa8dc106b40 2024-05-08T23:46:15.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:15 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3122509728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:15.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:15 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/758177204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:15.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:15 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3122509728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:15.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:15 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/758177204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:15.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.924+0000 7fa8e307b700 1 -- 172.21.15.12:0/1168344324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8dc10c690 con 0x7fa8dc106b40 2024-05-08T23:46:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.926+0000 7fa8d1ffb700 1 -- 172.21.15.12:0/1168344324 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22445 (secure 0 0 0) 0x7fa8d409b230 con 0x7fa8dc106b40 2024-05-08T23:46:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:15.922 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:45:56.347006+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:46:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 -- 172.21.15.12:0/1168344324 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa8c40a11b0 msgr2=0x7fa8c40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa8c40a11b0 0x7fa8c40a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009c60 tx=0x7fa8d8009260 comp rx=0 tx=0).stop 2024-05-08T23:46:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 -- 172.21.15.12:0/1168344324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8dc106b40 msgr2=0x7fa8dc19e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8dc106b40 0x7fa8dc19e430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4007a60 tx=0x7fa8d4007930 comp rx=0 tx=0).stop 2024-05-08T23:46:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 -- 172.21.15.12:0/1168344324 shutdown_connections 2024-05-08T23:46:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8dc106b40 0x7fa8dc19e430 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fa8c40a11b0 0x7fa8c40a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8dc1018e0 0x7fa8dc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 --2- 172.21.15.12:0/1168344324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc104210 0x7fa8dc197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 -- 172.21.15.12:0/1168344324 >> 172.21.15.12:0/1168344324 conn(0x7fa8dc0fb2f0 msgr2=0x7fa8dc102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:15.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 -- 172.21.15.12:0/1168344324 shutdown_connections 2024-05-08T23:46:15.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:15.929+0000 7fa8e307b700 1 -- 172.21.15.12:0/1168344324 wait complete. 2024-05-08T23:46:15.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T23:45:04.027837+0000', 'modified': '2024-05-08T23:45:09.067610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24548, 'mds_1': 24515, 'mds_2': 24511}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/1481890752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 1481890752}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1481890752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/3313911488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 3313911488}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 3313911488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/2858273239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 2858273239}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 2858273239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.204:6839/2873951026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 2873951026}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 2873951026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.160:6837/1904912186', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 1904912186}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 1904912186}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.160:6839/924146489', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 924146489}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 924146489}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:46:15.940 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T23:46:15.940 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-08T23:46:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 -- 172.21.15.12:0/4001928228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6568104330 msgr2=0x7f6568106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 --2- 172.21.15.12:0/4001928228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6568104330 0x7f6568106720 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6558009990 tx=0x7f6558009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 -- 172.21.15.12:0/4001928228 shutdown_connections 2024-05-08T23:46:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 --2- 172.21.15.12:0/4001928228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6568106c60 0x7f65681090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 --2- 172.21.15.12:0/4001928228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6568101a00 0x7f6568103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 --2- 172.21.15.12:0/4001928228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6568104330 0x7f6568106720 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 -- 172.21.15.12:0/4001928228 >> 172.21.15.12:0/4001928228 conn(0x7f65680fb3f0 msgr2=0x7f65680fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 -- 172.21.15.12:0/4001928228 shutdown_connections 2024-05-08T23:46:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 -- 172.21.15.12:0/4001928228 wait complete. 2024-05-08T23:46:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 Processor -- start 2024-05-08T23:46:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.086+0000 7f656e571700 1 -- start start 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f656e571700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6568101a00 0x7f6568101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f656e571700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6568104330 0x7f65680ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f656e571700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6568106c60 0x7f65680ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f656e571700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6568100690 con 0x7f6568101a00 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f656e571700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6568100800 con 0x7f6568106c60 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f656e571700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65681a0fa0 con 0x7f6568104330 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f6567fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6568101a00 0x7f6568101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f6567fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6568101a00 0x7f6568101320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33772/0 (socket says 172.21.15.12:33772) 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f6567fff700 1 -- 172.21.15.12:0/2211163498 learned_addr learned my addr 172.21.15.12:0/2211163498 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f656cb0e700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6568106c60 0x7f65680ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f65677fe700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6568104330 0x7f65680ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f6567fff700 1 -- 172.21.15.12:0/2211163498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6568104330 msgr2=0x7f65680ff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f6567fff700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6568104330 0x7f65680ff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f6567fff700 1 -- 172.21.15.12:0/2211163498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6568106c60 msgr2=0x7f65680ffef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f6567fff700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6568106c60 0x7f65680ffef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f6567fff700 1 -- 172.21.15.12:0/2211163498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65580095f0 con 0x7f6568101a00 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f656cb0e700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6568106c60 0x7f65680ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f65677fe700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6568104330 0x7f65680ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.087+0000 7f6567fff700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6568101a00 0x7f6568101320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f655000d950 tx=0x7f655000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.088+0000 7f65657fa700 1 -- 172.21.15.12:0/2211163498 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550009ae0 con 0x7f6568101a00 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.088+0000 7f65657fa700 1 -- 172.21.15.12:0/2211163498 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6550009c40 con 0x7f6568101a00 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.088+0000 7f656e571700 1 -- 172.21.15.12:0/2211163498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65681a1230 con 0x7f6568101a00 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.088+0000 7f65657fa700 1 -- 172.21.15.12:0/2211163498 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655000f620 con 0x7f6568101a00 2024-05-08T23:46:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.088+0000 7f656e571700 1 -- 172.21.15.12:0/2211163498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65681a1740 con 0x7f6568101a00 2024-05-08T23:46:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.090+0000 7f65657fa700 1 -- 172.21.15.12:0/2211163498 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6550010460 con 0x7f6568101a00 2024-05-08T23:46:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.091+0000 7f65657fa700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f65540a11b0 0x7f65540a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.091+0000 7f65677fe700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f65540a11b0 0x7f65540a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.091+0000 7f65657fa700 1 -- 172.21.15.12:0/2211163498 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f65500cca00 con 0x7f6568101a00 2024-05-08T23:46:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.091+0000 7f65677fe700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f65540a11b0 0x7f65540a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6558006010 tx=0x7f65580037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.091+0000 7f657a72bb80 1 Processor -- start 2024-05-08T23:46:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f657a72bb80 1 -- start start 2024-05-08T23:46:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 0x55d98ecd2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd2e50 0x55d98ecd32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ecd3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f657a72bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ec29100 con 0x55d98ecd3810 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f657a72bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ec26ac0 con 0x55d98ecd2e50 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f657a72bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ec223b0 con 0x55d98ecd24f0 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f656cb0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ecd3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f6567fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 0x55d98ecd2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f656cb0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ecd3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33774/0 (socket says 172.21.15.12:33774) 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f6567fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 0x55d98ecd2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39056/0 (socket says 172.21.15.12:39056) 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f656cb0e700 1 -- 172.21.15.12:0/3602165365 learned_addr learned my addr 172.21.15.12:0/3602165365 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f65677fe700 1 --2- 172.21.15.12:0/3602165365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd2e50 0x55d98ecd32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f656cb0e700 1 -- 172.21.15.12:0/3602165365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 msgr2=0x55d98ecd2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f656cb0e700 1 --2- 172.21.15.12:0/3602165365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 0x55d98ecd2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f656cb0e700 1 -- 172.21.15.12:0/3602165365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd2e50 msgr2=0x55d98ecd32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f656cb0e700 1 --2- 172.21.15.12:0/3602165365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd2e50 0x55d98ecd32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f656cb0e700 1 -- 172.21.15.12:0/3602165365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d98eaa3360 con 0x55d98ecd3810 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.092+0000 7f656cb0e700 1 --2- 172.21.15.12:0/3602165365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ecd3c90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e50 tx=0x7f65600077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f96eb15dc1edb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.093+0000 7f655e7fc700 1 -- 172.21.15.12:0/3602165365 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560014610 con 0x55d98ecd3810 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.093+0000 7f655e7fc700 1 -- 172.21.15.12:0/3602165365 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f656000f050 con 0x55d98ecd3810 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.093+0000 7f655e7fc700 1 -- 172.21.15.12:0/3602165365 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560013ac0 con 0x55d98ecd3810 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.093+0000 7f657a72bb80 1 -- 172.21.15.12:0/3602165365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 msgr2=0x55d98ecd3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.093+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3602165365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ecd3c90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e50 tx=0x7f65600077e0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.093+0000 7f657a72bb80 1 -- 172.21.15.12:0/3602165365 shutdown_connections 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.093+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3602165365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ecd3c90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.093+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3602165365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 0x55d98ecd2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.093+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3602165365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd2e50 0x55d98ecd32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.093+0000 7f657a72bb80 1 -- 172.21.15.12:0/3602165365 >> 172.21.15.12:0/3602165365 conn(0x55d98ec301c0 msgr2=0x55d98ecccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.094+0000 7f657a72bb80 1 -- 172.21.15.12:0/3602165365 shutdown_connections 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.094+0000 7f657a72bb80 1 -- 172.21.15.12:0/3602165365 wait complete. 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.094+0000 7f657a72bb80 1 -- start start 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.094+0000 7f657a72bb80 1 Processor -- start 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.094+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd24f0 0x55d98ecdbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.094+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 0x55d98ecdc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.094+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ece3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.094+0000 7f657a72bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ebbe510 con 0x55d98ecd3810 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.094+0000 7f657a72bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ebbe1b0 con 0x55d98ecd24f0 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.094+0000 7f657a72bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98eaa5650 con 0x55d98ecd2e50 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f657a72bb80 10 client.0 _ll_register_callbacks cb 0x55d98ebc7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f656cb0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ece3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f656cb0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ece3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33786/0 (socket says 172.21.15.12:33786) 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f656cb0e700 1 -- 172.21.15.12:0/3984165516 learned_addr learned my addr 172.21.15.12:0/3984165516 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f6567fff700 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd24f0 0x55d98ecdbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f65677fe700 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 0x55d98ecdc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f656cb0e700 1 -- 172.21.15.12:0/3984165516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 msgr2=0x55d98ecdc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f656cb0e700 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 0x55d98ecdc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f656cb0e700 1 -- 172.21.15.12:0/3984165516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd24f0 msgr2=0x55d98ecdbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f656cb0e700 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd24f0 0x55d98ecdbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f656cb0e700 1 -- 172.21.15.12:0/3984165516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6560009a80 con 0x55d98ecd3810 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f656cb0e700 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ece3320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6560000f80 tx=0x7f6560013a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec2a479be6ebcf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f65477fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f65477fe700 1 -- 172.21.15.12:0/3984165516 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560003c00 con 0x55d98ecd3810 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f65477fe700 1 -- 172.21.15.12:0/3984165516 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6560003d60 con 0x55d98ecd3810 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.095+0000 7f657a72bb80 1 -- 172.21.15.12:0/3984165516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d98eb7b220 con 0x55d98ecd3810 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f65477fe700 1 -- 172.21.15.12:0/3984165516 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6560004020 con 0x55d98ecd3810 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f657a72bb80 10 client.15189 fetch_fsmap learned FSMap version 23 2024-05-08T23:46:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f657a72bb80 1 -- 172.21.15.12:0/3984165516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d98ea4fb10 con 0x55d98ecd3810 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f65477fe700 1 -- 172.21.15.12:0/3984165516 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d98eb7b220 con 0x55d98ecd3810 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f65477fe700 1 -- 172.21.15.12:0/3984165516 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6560004760 con 0x55d98ecd3810 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f657a72bb80 10 client.15189 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f657a72bb80 10 client.15189 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f657a72bb80 20 client.15189 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d98ece6ea0 0x55d98ecec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f657a72bb80 4 client.15189 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f657a72bb80 1 -- 172.21.15.12:0/3984165516 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d98eca17c0 con 0x55d98ece6ea0 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.096+0000 7f6567fff700 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d98ece6ea0 0x55d98ecec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.097+0000 7f6567fff700 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d98ece6ea0 0x55d98ecec1a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be83f35297aa9f9b server_cookie=b1cb3715e1f0f1ec in_seq=0 out_seq=0 2024-05-08T23:46:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.097+0000 7f65477fe700 10 client.15189 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:46:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f65477fe700 1 -- 172.21.15.12:0/3984165516 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d98eca17c0 con 0x55d98ece6ea0 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f65477fe700 10 client.15189 handle_command_reply: tid=0 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 client.15189 shutdown 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 -- 172.21.15.12:0/3984165516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 msgr2=0x55d98ece3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ece3320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6560000f80 tx=0x7f6560013a70 comp rx=0 tx=0).stop 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 -- 172.21.15.12:0/3984165516 shutdown_connections 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd3810 0x55d98ece3320 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd24f0 0x55d98ecdbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d98ece6ea0 0x55d98ecec1a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3984165516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 0x55d98ecdc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f65477fe700 0 client.15189 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 -- 172.21.15.12:0/3984165516 >> 172.21.15.12:0/3984165516 conn(0x55d98ec301c0 msgr2=0x55d98ecd12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 -- 172.21.15.12:0/3984165516 shutdown_connections 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 1 -- 172.21.15.12:0/3984165516 wait complete. 2024-05-08T23:46:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.098+0000 7f657a72bb80 20 client.15189 trim_cache size 0 max 16384 2024-05-08T23:46:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.117+0000 7f657a72bb80 1 Processor -- start 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.117+0000 7f657a72bb80 1 -- start start 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd24f0 0x55d98ecd28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 0x55d98ecdb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 0x55d98ece3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f657a72bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ec1b150 con 0x55d98ecd24f0 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f657a72bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ec1f010 con 0x55d98ecd3810 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f657a72bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ecdb910 con 0x55d98ecd2e50 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f6567fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd24f0 0x55d98ecd28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f6567fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd24f0 0x55d98ecd28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33802/0 (socket says 172.21.15.12:33802) 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f6567fff700 1 -- 172.21.15.12:0/2279427155 learned_addr learned my addr 172.21.15.12:0/2279427155 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f65677fe700 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 0x55d98ecdb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f656cb0e700 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 0x55d98ece3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f6567fff700 1 -- 172.21.15.12:0/2279427155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 msgr2=0x55d98ecdb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f6567fff700 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 0x55d98ecdb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f6567fff700 1 -- 172.21.15.12:0/2279427155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 msgr2=0x55d98ece3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f6567fff700 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 0x55d98ece3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f6567fff700 1 -- 172.21.15.12:0/2279427155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d98ebe8510 con 0x55d98ecd24f0 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f65677fe700 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 0x55d98ecdb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f656cb0e700 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 0x55d98ece3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f6567fff700 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd24f0 0x55d98ecd28d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6550099c80 tx=0x7f65500cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60f42fe2fb50521 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f655effd700 1 -- 172.21.15.12:0/2279427155 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550014070 con 0x55d98ecd24f0 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f655effd700 1 -- 172.21.15.12:0/2279427155 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65500a2170 con 0x55d98ecd24f0 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.118+0000 7f655effd700 1 -- 172.21.15.12:0/2279427155 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550099440 con 0x55d98ecd24f0 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.119+0000 7f657a72bb80 1 -- 172.21.15.12:0/2279427155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd24f0 msgr2=0x55d98ecd28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.119+0000 7f657a72bb80 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd24f0 0x55d98ecd28d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6550099c80 tx=0x7f65500cc040 comp rx=0 tx=0).stop 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.119+0000 7f657a72bb80 1 -- 172.21.15.12:0/2279427155 shutdown_connections 2024-05-08T23:46:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.119+0000 7f657a72bb80 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 0x55d98ece3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.119+0000 7f657a72bb80 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd24f0 0x55d98ecd28d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.119+0000 7f657a72bb80 1 --2- 172.21.15.12:0/2279427155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd2e50 0x55d98ecdb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.119+0000 7f657a72bb80 1 -- 172.21.15.12:0/2279427155 >> 172.21.15.12:0/2279427155 conn(0x55d98ec301c0 msgr2=0x55d98ecc91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.119+0000 7f657a72bb80 1 -- 172.21.15.12:0/2279427155 shutdown_connections 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.119+0000 7f657a72bb80 1 -- 172.21.15.12:0/2279427155 wait complete. 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.119+0000 7f657a72bb80 1 -- start start 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f657a72bb80 1 Processor -- start 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 0x55d98ece0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd2e50 0x55d98ece27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f657a72bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 0x55d98ece2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f657a72bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ece33e0 con 0x55d98ecd2e50 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f657a72bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ece3550 con 0x55d98ecd3810 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f657a72bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ece3710 con 0x55d98ecd24f0 2024-05-08T23:46:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f657a72bb80 10 client.0 _ll_register_callbacks cb 0x55d98ebc7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f65677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd2e50 0x55d98ece27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f65677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd2e50 0x55d98ece27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33804/0 (socket says 172.21.15.12:33804) 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f6567fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 0x55d98ece0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f65677fe700 1 -- 172.21.15.12:0/3850200084 learned_addr learned my addr 172.21.15.12:0/3850200084 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f656cb0e700 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 0x55d98ece2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f6567fff700 1 -- 172.21.15.12:0/3850200084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 msgr2=0x55d98ece2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f6567fff700 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 0x55d98ece2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f6567fff700 1 -- 172.21.15.12:0/3850200084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd2e50 msgr2=0x55d98ece27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f6567fff700 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd2e50 0x55d98ece27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f6567fff700 1 -- 172.21.15.12:0/3850200084 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65681a1740 con 0x55d98ecd24f0 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.120+0000 7f65677fe700 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd2e50 0x55d98ece27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.121+0000 7f6567fff700 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 0x55d98ece0270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6550099c80 tx=0x7f6550099f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df122f98c9b713a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.121+0000 7f65477fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.121+0000 7f65477fe700 1 -- 172.21.15.12:0/3850200084 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550014040 con 0x55d98ecd24f0 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.121+0000 7f65477fe700 1 -- 172.21.15.12:0/3850200084 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6550009e30 con 0x55d98ecd24f0 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.121+0000 7f657a72bb80 1 -- 172.21.15.12:0/3850200084 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f656000f050 con 0x55d98ecd24f0 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.121+0000 7f65477fe700 1 -- 172.21.15.12:0/3850200084 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65500993e0 con 0x55d98ecd24f0 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.121+0000 7f657a72bb80 10 client.25075 fetch_fsmap learned FSMap version 23 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.121+0000 7f657a72bb80 1 -- 172.21.15.12:0/3850200084 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d98eaa7c80 con 0x55d98ecd24f0 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.121+0000 7f65477fe700 1 -- 172.21.15.12:0/3850200084 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f656000f050 con 0x55d98ecd24f0 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.121+0000 7f65477fe700 1 -- 172.21.15.12:0/3850200084 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6550010710 con 0x55d98ecd24f0 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f657a72bb80 10 client.25075 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f657a72bb80 10 client.25075 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f657a72bb80 20 client.25075 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d98ed42560 0x55d98ed3f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f657a72bb80 4 client.25075 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:46:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f657a72bb80 1 -- 172.21.15.12:0/3850200084 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d98ec1ae70 con 0x55d98ed42560 2024-05-08T23:46:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f65677fe700 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d98ed42560 0x55d98ed3f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f65477fe700 10 client.25075 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:46:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f65677fe700 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d98ed42560 0x55d98ed3f050 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=710d442bf7b9f4ed server_cookie=e834b6f698e0c713 in_seq=0 out_seq=0 2024-05-08T23:46:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f65477fe700 1 -- 172.21.15.12:0/3850200084 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d98ec1ae70 con 0x55d98ed42560 2024-05-08T23:46:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f65477fe700 10 client.25075 handle_command_reply: tid=0 2024-05-08T23:46:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.122+0000 7f657a72bb80 1 client.25075 shutdown 2024-05-08T23:46:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 1 -- 172.21.15.12:0/3850200084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 msgr2=0x55d98ece0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 0x55d98ece0270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6550099c80 tx=0x7f6550099f50 comp rx=0 tx=0).stop 2024-05-08T23:46:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 1 -- 172.21.15.12:0/3850200084 shutdown_connections 2024-05-08T23:46:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d98ecd3810 0x55d98ece2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d98ed42560 0x55d98ed3f050 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d98ecd24f0 0x55d98ece0270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 1 --2- 172.21.15.12:0/3850200084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d98ecd2e50 0x55d98ece27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f65477fe700 0 client.25075 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 1 -- 172.21.15.12:0/3850200084 >> 172.21.15.12:0/3850200084 conn(0x55d98ec301c0 msgr2=0x55d98ecd6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 1 -- 172.21.15.12:0/3850200084 shutdown_connections 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 1 -- 172.21.15.12:0/3850200084 wait complete. 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f657a72bb80 20 client.25075 trim_cache size 0 max 16384 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "00f8fb28-0d94-11ef-bc97-c7b262605968", 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24511, 2024-05-08T23:46:16.125 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-08T23:46:16.126 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-08T23:46:16.126 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-08T23:46:16.126 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 68.061957637999996, 2024-05-08T23:46:16.126 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 23, 2024-05-08T23:46:16.126 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-05-08T23:46:16.126 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:46:16.126 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 83.383175022000003 2024-05-08T23:46:16.126 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:46:16.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f656e571700 1 -- 172.21.15.12:0/2211163498 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f65540a11b0 msgr2=0x7f65540a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f656e571700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f65540a11b0 0x7f65540a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6558006010 tx=0x7f65580037b0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f656e571700 1 -- 172.21.15.12:0/2211163498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6568101a00 msgr2=0x7f6568101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f656e571700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6568101a00 0x7f6568101320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f655000d950 tx=0x7f655000dd10 comp rx=0 tx=0).stop 2024-05-08T23:46:16.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f656e571700 1 -- 172.21.15.12:0/2211163498 shutdown_connections 2024-05-08T23:46:16.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f656e571700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f65540a11b0 0x7f65540a3670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f656e571700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6568106c60 0x7f65680ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f656e571700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6568101a00 0x7f6568101320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f656e571700 1 --2- 172.21.15.12:0/2211163498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6568104330 0x7f65680ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.123+0000 7f656e571700 1 -- 172.21.15.12:0/2211163498 >> 172.21.15.12:0/2211163498 conn(0x7f65680fb3f0 msgr2=0x7f6568102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.124+0000 7f656e571700 1 -- 172.21.15.12:0/2211163498 shutdown_connections 2024-05-08T23:46:16.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.124+0000 7f656e571700 1 -- 172.21.15.12:0/2211163498 wait complete. 2024-05-08T23:46:16.133 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-08T23:46:16.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.275+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1652868800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f0100190 msgr2=0x7fe5f0100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.275+0000 7fe5f5fbb700 1 --2- 172.21.15.12:0/1652868800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f0100190 0x7fe5f0100610 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40099c0 tx=0x7fe5e4009cd0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.275+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1652868800 shutdown_connections 2024-05-08T23:46:16.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.275+0000 7fe5f5fbb700 1 --2- 172.21.15.12:0/1652868800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f0100b50 0x7fe5f0100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.275+0000 7fe5f5fbb700 1 --2- 172.21.15.12:0/1652868800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f00ff830 0x7fe5f00ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.275+0000 7fe5f5fbb700 1 --2- 172.21.15.12:0/1652868800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f0100190 0x7fe5f0100610 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.275+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1652868800 >> 172.21.15.12:0/1652868800 conn(0x7fe5f00fb3f0 msgr2=0x7fe5f00fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.275+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1652868800 shutdown_connections 2024-05-08T23:46:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.275+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1652868800 wait complete. 2024-05-08T23:46:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.276+0000 7fe5f5fbb700 1 Processor -- start 2024-05-08T23:46:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.276+0000 7fe5f5fbb700 1 -- start start 2024-05-08T23:46:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5f5fbb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f00ff830 0x7fe5f01990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5f5fbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f0100190 0x7fe5f0199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5f5fbb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f0100b50 0x7fe5f01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5f5fbb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f01a0bf0 con 0x7fe5f0100190 2024-05-08T23:46:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5f5fbb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f01a0d30 con 0x7fe5f0100b50 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5f5fbb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f01a0ec0 con 0x7fe5f00ff830 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f0100190 0x7fe5f0199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f00ff830 0x7fe5f01990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f0100190 0x7fe5f0199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33824/0 (socket says 172.21.15.12:33824) 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f00ff830 0x7fe5f01990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39104/0 (socket says 172.21.15.12:39104) 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5eeffd700 1 -- 172.21.15.12:0/1414992640 learned_addr learned my addr 172.21.15.12:0/1414992640 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5effff700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f0100b50 0x7fe5f01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5ef7fe700 1 -- 172.21.15.12:0/1414992640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f0100b50 msgr2=0x7fe5f01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f0100b50 0x7fe5f01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5ef7fe700 1 -- 172.21.15.12:0/1414992640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f0100190 msgr2=0x7fe5f0199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f0100190 0x7fe5f0199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5ef7fe700 1 -- 172.21.15.12:0/1414992640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40095f0 con 0x7fe5f00ff830 2024-05-08T23:46:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.277+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f00ff830 0x7fe5f01990c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000ef10 tx=0x7fe5e000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.278+0000 7fe5ecff9700 1 -- 172.21.15.12:0/1414992640 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e000c9c0 con 0x7fe5f00ff830 2024-05-08T23:46:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.278+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1414992640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f01a1140 con 0x7fe5f00ff830 2024-05-08T23:46:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.278+0000 7fe5ecff9700 1 -- 172.21.15.12:0/1414992640 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e000f040 con 0x7fe5f00ff830 2024-05-08T23:46:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.278+0000 7fe5ecff9700 1 -- 172.21.15.12:0/1414992640 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0014480 con 0x7fe5f00ff830 2024-05-08T23:46:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.278+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1414992640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f01a16a0 con 0x7fe5f00ff830 2024-05-08T23:46:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.281+0000 7fe5ecff9700 1 -- 172.21.15.12:0/1414992640 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe5e000cb20 con 0x7fe5f00ff830 2024-05-08T23:46:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.282+0000 7fe5ecff9700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5d80a11b0 0x7fe5d80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.282+0000 7fe5ecff9700 1 -- 172.21.15.12:0/1414992640 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe5e00ccc30 con 0x7fe5f00ff830 2024-05-08T23:46:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.282+0000 7fe5eeffd700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5d80a11b0 0x7fe5d80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.282+0000 7fe602175b80 1 Processor -- start 2024-05-08T23:46:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.282+0000 7fe602175b80 1 -- start start 2024-05-08T23:46:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.282+0000 7fe5eeffd700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5d80a11b0 0x7fe5d80a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4005b00 tx=0x7fe5e4005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe602175b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 0x5629157b2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe602175b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe602175b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe602175b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915709100 con 0x5629157b3810 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe602175b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915706ac0 con 0x5629157b2e50 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe602175b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629157023b0 con 0x5629157b24f0 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33840/0 (socket says 172.21.15.12:33840) 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5effff700 1 -- 172.21.15.12:0/2109619805 learned_addr learned my addr 172.21.15.12:0/2109619805 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 0x5629157b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5eeffd700 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5effff700 1 -- 172.21.15.12:0/2109619805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 msgr2=0x5629157b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5effff700 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 0x5629157b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5effff700 1 -- 172.21.15.12:0/2109619805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 msgr2=0x5629157b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5effff700 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5effff700 1 -- 172.21.15.12:0/2109619805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562915583360 con 0x5629157b3810 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5eeffd700 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 0x5629157b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.283+0000 7fe5effff700 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157b3c90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b6c0 tx=0x7fe5e800b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f8bed00b8d5d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.284+0000 7fe5d5ffb700 1 -- 172.21.15.12:0/2109619805 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8004a30 con 0x5629157b3810 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.284+0000 7fe5d5ffb700 1 -- 172.21.15.12:0/2109619805 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e80054e0 con 0x5629157b3810 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.284+0000 7fe5d5ffb700 1 -- 172.21.15.12:0/2109619805 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8003ba0 con 0x5629157b3810 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.284+0000 7fe602175b80 1 -- 172.21.15.12:0/2109619805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 msgr2=0x5629157b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.284+0000 7fe602175b80 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157b3c90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b6c0 tx=0x7fe5e800b9d0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.285+0000 7fe602175b80 1 -- 172.21.15.12:0/2109619805 shutdown_connections 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.285+0000 7fe602175b80 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157b3c90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.285+0000 7fe602175b80 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 0x5629157b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.285+0000 7fe602175b80 1 --2- 172.21.15.12:0/2109619805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.285+0000 7fe602175b80 1 -- 172.21.15.12:0/2109619805 >> 172.21.15.12:0/2109619805 conn(0x5629157101c0 msgr2=0x5629157acd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.286+0000 7fe602175b80 1 -- 172.21.15.12:0/2109619805 shutdown_connections 2024-05-08T23:46:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.286+0000 7fe602175b80 1 -- 172.21.15.12:0/2109619805 wait complete. 2024-05-08T23:46:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.286+0000 7fe602175b80 1 -- start start 2024-05-08T23:46:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe602175b80 1 Processor -- start 2024-05-08T23:46:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe602175b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 0x5629157bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe602175b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157bc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe602175b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe602175b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291569e510 con 0x5629157b3810 2024-05-08T23:46:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe602175b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291569e1b0 con 0x5629157b2e50 2024-05-08T23:46:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe602175b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915585650 con 0x5629157b24f0 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe602175b80 10 client.0 _ll_register_callbacks cb 0x5629156a7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe5effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe5effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33852/0 (socket says 172.21.15.12:33852) 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.287+0000 7fe5effff700 1 -- 172.21.15.12:0/2479032844 learned_addr learned my addr 172.21.15.12:0/2479032844 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 0x5629157bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5eeffd700 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157bc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5effff700 1 -- 172.21.15.12:0/2479032844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 msgr2=0x5629157bbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5effff700 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 0x5629157bbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5effff700 1 -- 172.21.15.12:0/2479032844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 msgr2=0x5629157bc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5effff700 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157bc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5effff700 1 -- 172.21.15.12:0/2479032844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f01a16a0 con 0x5629157b3810 2024-05-08T23:46:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5effff700 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c3360 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b690 tx=0x7fe5e8004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f678474dcb455580 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5ceffd700 1 -- 172.21.15.12:0/2479032844 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e801f070 con 0x5629157b3810 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5ceffd700 1 -- 172.21.15.12:0/2479032844 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe5e8004a80 con 0x5629157b3810 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe602175b80 1 -- 172.21.15.12:0/2479032844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56291565b220 con 0x5629157b3810 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe5ceffd700 1 -- 172.21.15.12:0/2479032844 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5e80051d0 con 0x5629157b3810 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe602175b80 10 client.15207 fetch_fsmap learned FSMap version 23 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.288+0000 7fe602175b80 1 -- 172.21.15.12:0/2479032844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fe5e8004350 con 0x5629157b3810 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.289+0000 7fe5ceffd700 1 -- 172.21.15.12:0/2479032844 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56291565b220 con 0x5629157b3810 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.289+0000 7fe5ceffd700 1 -- 172.21.15.12:0/2479032844 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe5e8011770 con 0x5629157b3810 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.289+0000 7fe602175b80 10 client.15207 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.289+0000 7fe602175b80 10 client.15207 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.289+0000 7fe602175b80 20 client.15207 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.289+0000 7fe602175b80 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5629157c6e60 0x5629157cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.289+0000 7fe602175b80 4 client.15207 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:46:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.289+0000 7fe602175b80 1 -- 172.21.15.12:0/2479032844 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629157817c0 con 0x5629157c6e60 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.289+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5629157c6e60 0x5629157cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.290+0000 7fe5ceffd700 10 client.15207 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.290+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5629157c6e60 0x5629157cc160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=827af64ea397af93 server_cookie=6db80e36e67bf9b0 in_seq=0 out_seq=0 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.290+0000 7fe5ceffd700 1 -- 172.21.15.12:0/2479032844 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629157817c0 con 0x5629157c6e60 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.290+0000 7fe5ceffd700 10 client.15207 handle_command_reply: tid=0 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.290+0000 7fe602175b80 1 client.15207 shutdown 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.290+0000 7fe602175b80 1 -- 172.21.15.12:0/2479032844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 msgr2=0x5629157c3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.290+0000 7fe602175b80 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c3360 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b690 tx=0x7fe5e8004e20 comp rx=0 tx=0).stop 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.291+0000 7fe602175b80 1 -- 172.21.15.12:0/2479032844 shutdown_connections 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.291+0000 7fe602175b80 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5629157c6e60 0x5629157cc160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.291+0000 7fe602175b80 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c3360 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.291+0000 7fe602175b80 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b24f0 0x5629157bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.291+0000 7fe602175b80 1 --2- 172.21.15.12:0/2479032844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157bc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.291+0000 7fe5ceffd700 0 client.15207 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:46:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.291+0000 7fe602175b80 1 -- 172.21.15.12:0/2479032844 >> 172.21.15.12:0/2479032844 conn(0x5629157101c0 msgr2=0x5629157ac550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.291+0000 7fe602175b80 1 -- 172.21.15.12:0/2479032844 shutdown_connections 2024-05-08T23:46:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.291+0000 7fe602175b80 1 -- 172.21.15.12:0/2479032844 wait complete. 2024-05-08T23:46:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.291+0000 7fe602175b80 20 client.15207 trim_cache size 0 max 16384 2024-05-08T23:46:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe602175b80 1 Processor -- start 2024-05-08T23:46:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe602175b80 1 -- start start 2024-05-08T23:46:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe602175b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b24f0 0x5629157b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe602175b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe602175b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b3810 0x5629157c3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe602175b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629157817c0 con 0x5629157b24f0 2024-05-08T23:46:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe602175b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629156c02b0 con 0x5629157b2e50 2024-05-08T23:46:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe602175b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629156fb150 con 0x5629157b3810 2024-05-08T23:46:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b24f0 0x5629157b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b24f0 0x5629157b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33860/0 (socket says 172.21.15.12:33860) 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.310+0000 7fe5ef7fe700 1 -- 172.21.15.12:0/2477361514 learned_addr learned my addr 172.21.15.12:0/2477361514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5eeffd700 1 --2- 172.21.15.12:0/2477361514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5effff700 1 --2- 172.21.15.12:0/2477361514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b3810 0x5629157c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5ef7fe700 1 -- 172.21.15.12:0/2477361514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b3810 msgr2=0x5629157c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/2477361514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b3810 0x5629157c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5ef7fe700 1 -- 172.21.15.12:0/2477361514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 msgr2=0x5629157c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/2477361514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5ef7fe700 1 -- 172.21.15.12:0/2477361514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56291552fb10 con 0x5629157b24f0 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5effff700 1 --2- 172.21.15.12:0/2477361514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b3810 0x5629157c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/2477361514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b24f0 0x5629157b28d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00ca710 tx=0x7fe5e009a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805026b69afce36a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5d67fc700 1 -- 172.21.15.12:0/2477361514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e009de50 con 0x5629157b24f0 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5d67fc700 1 -- 172.21.15.12:0/2477361514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e00a24a0 con 0x5629157b24f0 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.311+0000 7fe5d67fc700 1 -- 172.21.15.12:0/2477361514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0007690 con 0x5629157b24f0 2024-05-08T23:46:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 -- 172.21.15.12:0/2477361514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b24f0 msgr2=0x5629157b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 --2- 172.21.15.12:0/2477361514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b24f0 0x5629157b28d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00ca710 tx=0x7fe5e009a720 comp rx=0 tx=0).stop 2024-05-08T23:46:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 -- 172.21.15.12:0/2477361514 shutdown_connections 2024-05-08T23:46:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 --2- 172.21.15.12:0/2477361514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b3810 0x5629157c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 --2- 172.21.15.12:0/2477361514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b24f0 0x5629157b28d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 --2- 172.21.15.12:0/2477361514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b2e50 0x5629157c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 -- 172.21.15.12:0/2477361514 >> 172.21.15.12:0/2477361514 conn(0x5629157101c0 msgr2=0x5629157a91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 -- 172.21.15.12:0/2477361514 shutdown_connections 2024-05-08T23:46:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 -- 172.21.15.12:0/2477361514 wait complete. 2024-05-08T23:46:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 -- start start 2024-05-08T23:46:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.312+0000 7fe602175b80 1 Processor -- start 2024-05-08T23:46:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe602175b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b24f0 0x5629157abf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe602175b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b2e50 0x5629157ac450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe602175b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe602175b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629157c2d60 con 0x5629157b3810 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe602175b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629157c2ed0 con 0x5629157b24f0 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe602175b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629157c3090 con 0x5629157b2e50 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe602175b80 10 client.0 _ll_register_callbacks cb 0x5629156a7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33864/0 (socket says 172.21.15.12:33864) 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5effff700 1 -- 172.21.15.12:0/615699184 learned_addr learned my addr 172.21.15.12:0/615699184 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b24f0 0x5629157abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5eeffd700 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b2e50 0x5629157ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5effff700 1 -- 172.21.15.12:0/615699184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b2e50 msgr2=0x5629157ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5effff700 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b2e50 0x5629157ac450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5effff700 1 -- 172.21.15.12:0/615699184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b24f0 msgr2=0x5629157abf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5effff700 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b24f0 0x5629157abf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5effff700 1 -- 172.21.15.12:0/615699184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e0023070 con 0x5629157b3810 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5eeffd700 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b2e50 0x5629157ac450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5effff700 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c2820 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800ba10 tx=0x7fe5e8004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47dc0ab15f14fa78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.313+0000 7fe5ceffd700 1 -- 172.21.15.12:0/615699184 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e801f040 con 0x5629157b3810 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe5ceffd700 1 -- 172.21.15.12:0/615699184 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe5e8011df0 con 0x5629157b3810 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe602175b80 1 -- 172.21.15.12:0/615699184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5e80054e0 con 0x5629157b3810 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe5ceffd700 1 -- 172.21.15.12:0/615699184 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5e80051d0 con 0x5629157b3810 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe602175b80 10 client.15213 fetch_fsmap learned FSMap version 23 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe602175b80 1 -- 172.21.15.12:0/615699184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5629156c8510 con 0x5629157b3810 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe5ceffd700 1 -- 172.21.15.12:0/615699184 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e803e020 con 0x5629157b3810 2024-05-08T23:46:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe5ceffd700 1 -- 172.21.15.12:0/615699184 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe5e802f6d0 con 0x5629157b3810 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe602175b80 10 client.15213 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe602175b80 10 client.15213 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe602175b80 20 client.15213 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe602175b80 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5629158220d0 0x56291581ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe602175b80 4 client.15213 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe602175b80 1 -- 172.21.15.12:0/615699184 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5629156fae70 con 0x5629158220d0 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.314+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5629158220d0 0x56291581ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe5ef7fe700 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5629158220d0 0x56291581ea60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c0d0cdac95a26bb server_cookie=3894acb56f5b8a4d in_seq=0 out_seq=0 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe5ceffd700 10 client.15213 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe5ceffd700 1 -- 172.21.15.12:0/615699184 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5629156fae70 con 0x5629158220d0 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe5ceffd700 10 client.15213 handle_command_reply: tid=0 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 client.15213 shutdown 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 -- 172.21.15.12:0/615699184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 msgr2=0x5629157c2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c2820 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800ba10 tx=0x7fe5e8004d50 comp rx=0 tx=0).stop 2024-05-08T23:46:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 -- 172.21.15.12:0/615699184 shutdown_connections 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5629158220d0 0x56291581ea60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629157b3810 0x5629157c2820 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe5ceffd700 0 client.15213 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629157b24f0 0x5629157abf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 --2- 172.21.15.12:0/615699184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629157b2e50 0x5629157ac450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 -- 172.21.15.12:0/615699184 >> 172.21.15.12:0/615699184 conn(0x5629157101c0 msgr2=0x5629157b9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 -- 172.21.15.12:0/615699184 shutdown_connections 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.315+0000 7fe602175b80 1 -- 172.21.15.12:0/615699184 wait complete. 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe602175b80 20 client.15213 trim_cache size 0 max 16384 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:46:16.317 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "00f8fb28-0d94-11ef-bc97-c7b262605968", 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24515, 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 69.264545451000004, 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 23, 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 85.315379485999998 2024-05-08T23:46:16.318 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:46:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1414992640 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5d80a11b0 msgr2=0x7fe5d80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5d80a11b0 0x7fe5d80a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4005b00 tx=0x7fe5e4005a90 comp rx=0 tx=0).stop 2024-05-08T23:46:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1414992640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f00ff830 msgr2=0x7fe5f01990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f00ff830 0x7fe5f01990c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000ef10 tx=0x7fe5e000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1414992640 shutdown_connections 2024-05-08T23:46:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fe5d80a11b0 0x7fe5d80a3670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5f0100b50 0x7fe5f01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5f00ff830 0x7fe5f01990c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 --2- 172.21.15.12:0/1414992640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5f0100190 0x7fe5f0199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1414992640 >> 172.21.15.12:0/1414992640 conn(0x7fe5f00fb3f0 msgr2=0x7fe5f00fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1414992640 shutdown_connections 2024-05-08T23:46:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.316+0000 7fe5f5fbb700 1 -- 172.21.15.12:0/1414992640 wait complete. 2024-05-08T23:46:16.326 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:46:16.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.470+0000 7f38472ce700 1 -- 172.21.15.12:0/2857320571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3840104310 msgr2=0x7f3840106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.470+0000 7f38472ce700 1 --2- 172.21.15.12:0/2857320571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3840104310 0x7f3840106700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3834009990 tx=0x7f3834009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.470+0000 7f38472ce700 1 -- 172.21.15.12:0/2857320571 shutdown_connections 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.470+0000 7f38472ce700 1 --2- 172.21.15.12:0/2857320571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3840106c40 0x7f38401090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.470+0000 7f38472ce700 1 --2- 172.21.15.12:0/2857320571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38401019e0 0x7f3840103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.470+0000 7f38472ce700 1 --2- 172.21.15.12:0/2857320571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3840104310 0x7f3840106700 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.470+0000 7f38472ce700 1 -- 172.21.15.12:0/2857320571 >> 172.21.15.12:0/2857320571 conn(0x7f38400fb3f0 msgr2=0x7f38400fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.470+0000 7f38472ce700 1 -- 172.21.15.12:0/2857320571 shutdown_connections 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.470+0000 7f38472ce700 1 -- 172.21.15.12:0/2857320571 wait complete. 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f38472ce700 1 Processor -- start 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f38472ce700 1 -- start start 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f38472ce700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38401019e0 0x7f3840196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f38472ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3840104310 0x7f38401973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f38472ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3840106c40 0x7f384019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f38472ce700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384019e990 con 0x7f3840104310 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f38472ce700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384019ead0 con 0x7f3840106c40 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f38472ce700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384019ec60 con 0x7f38401019e0 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f3844869700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3840104310 0x7f38401973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f3844869700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3840104310 0x7f38401973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33880/0 (socket says 172.21.15.12:33880) 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.471+0000 7f3844869700 1 -- 172.21.15.12:0/3479157033 learned_addr learned my addr 172.21.15.12:0/3479157033 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f384506a700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38401019e0 0x7f3840196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f384586b700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3840106c40 0x7f384019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f3844869700 1 -- 172.21.15.12:0/3479157033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38401019e0 msgr2=0x7f3840196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f3844869700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38401019e0 0x7f3840196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f3844869700 1 -- 172.21.15.12:0/3479157033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3840106c40 msgr2=0x7f384019e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f3844869700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3840106c40 0x7f384019e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f3844869700 1 -- 172.21.15.12:0/3479157033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38340095f0 con 0x7f3840104310 2024-05-08T23:46:16.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f384586b700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3840106c40 0x7f384019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:16.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f3844869700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3840104310 0x7f38401973a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3834000c00 tx=0x7f3834005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f38327fc700 1 -- 172.21.15.12:0/3479157033 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383401d070 con 0x7f3840104310 2024-05-08T23:46:16.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f38327fc700 1 -- 172.21.15.12:0/3479157033 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3834004ca0 con 0x7f3840104310 2024-05-08T23:46:16.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f38472ce700 1 -- 172.21.15.12:0/3479157033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384019eee0 con 0x7f3840104310 2024-05-08T23:46:16.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f38327fc700 1 -- 172.21.15.12:0/3479157033 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38340055f0 con 0x7f3840104310 2024-05-08T23:46:16.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.472+0000 7f38472ce700 1 -- 172.21.15.12:0/3479157033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384019f2a0 con 0x7f3840104310 2024-05-08T23:46:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f3853488b80 1 Processor -- start 2024-05-08T23:46:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f3853488b80 1 -- start start 2024-05-08T23:46:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f3853488b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f3853488b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15ce50 0x56362c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f3853488b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15d810 0x56362c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f3853488b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c0b3100 con 0x56362c15c4f0 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f3853488b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c0b0ac0 con 0x56362c15d810 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f3853488b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c0ac3b0 con 0x56362c15ce50 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f384586b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f384586b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33896/0 (socket says 172.21.15.12:33896) 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f384586b700 1 -- 172.21.15.12:0/1790394885 learned_addr learned my addr 172.21.15.12:0/1790394885 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.476+0000 7f384506a700 1 --2- 172.21.15.12:0/1790394885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15ce50 0x56362c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.477+0000 7f384586b700 1 -- 172.21.15.12:0/1790394885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15ce50 msgr2=0x56362c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.477+0000 7f384586b700 1 --2- 172.21.15.12:0/1790394885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15ce50 0x56362c15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.477+0000 7f384586b700 1 -- 172.21.15.12:0/1790394885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15d810 msgr2=0x56362c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.477+0000 7f384586b700 1 --2- 172.21.15.12:0/1790394885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15d810 0x56362c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.477+0000 7f384586b700 1 -- 172.21.15.12:0/1790394885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56362bf2d360 con 0x56362c15c4f0 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.477+0000 7f384586b700 1 --2- 172.21.15.12:0/1790394885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c15c910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3838009d50 tx=0x7f38380077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420e6f0afb000489 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.477+0000 7f382b7fe700 1 -- 172.21.15.12:0/1790394885 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838009ea0 con 0x56362c15c4f0 2024-05-08T23:46:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.477+0000 7f382b7fe700 1 -- 172.21.15.12:0/1790394885 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3838014ac0 con 0x56362c15c4f0 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.477+0000 7f382b7fe700 1 -- 172.21.15.12:0/1790394885 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838013b10 con 0x56362c15c4f0 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f38327fc700 1 -- 172.21.15.12:0/3479157033 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f383400fc40 con 0x7f3840104310 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 -- 172.21.15.12:0/1790394885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 msgr2=0x56362c15c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 --2- 172.21.15.12:0/1790394885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c15c910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3838009d50 tx=0x7f38380077e0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 -- 172.21.15.12:0/1790394885 shutdown_connections 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 --2- 172.21.15.12:0/1790394885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15d810 0x56362c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 --2- 172.21.15.12:0/1790394885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c15c910 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 --2- 172.21.15.12:0/1790394885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15ce50 0x56362c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 -- 172.21.15.12:0/1790394885 >> 172.21.15.12:0/1790394885 conn(0x56362c0ba1c0 msgr2=0x56362c156d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 -- 172.21.15.12:0/1790394885 shutdown_connections 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 -- 172.21.15.12:0/1790394885 wait complete. 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 -- start start 2024-05-08T23:46:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f38327fc700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f382c0a1140 0x7f382c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f3853488b80 1 Processor -- start 2024-05-08T23:46:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.478+0000 7f38327fc700 1 -- 172.21.15.12:0/3479157033 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f38340cfd80 con 0x7f3840104310 2024-05-08T23:46:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f384586b700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f382c0a1140 0x7f382c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f384586b700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f382c0a1140 0x7f382c0a3600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f38380091f0 tx=0x7f3838006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f3853488b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c165e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f3853488b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15ce50 0x56362c166340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f3853488b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15d810 0x56362c16d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f3853488b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c048510 con 0x56362c15c4f0 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f3853488b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c0481b0 con 0x56362c15d810 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f3853488b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362bf2f650 con 0x56362c15ce50 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f3853488b80 10 client.0 _ll_register_callbacks cb 0x56362c051ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f384506a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c165e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f384506a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c165e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33902/0 (socket says 172.21.15.12:33902) 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.479+0000 7f384506a700 1 -- 172.21.15.12:0/1646733091 learned_addr learned my addr 172.21.15.12:0/1646733091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f384586b700 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15d810 0x56362c16d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f3844869700 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15ce50 0x56362c166340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f384506a700 1 -- 172.21.15.12:0/1646733091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15ce50 msgr2=0x56362c166340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f384506a700 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15ce50 0x56362c166340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f384506a700 1 -- 172.21.15.12:0/1646733091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15d810 msgr2=0x56362c16d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f384506a700 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15d810 0x56362c16d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f384506a700 1 -- 172.21.15.12:0/1646733091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3838009980 con 0x56362c15c4f0 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f384506a700 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c165e00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f383c012810 tx=0x7f383c012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39535dffbb33ea1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f3828ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f3828ff9700 1 -- 172.21.15.12:0/1646733091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c017070 con 0x56362c15c4f0 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f3853488b80 1 -- 172.21.15.12:0/1646733091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56362c005220 con 0x56362c15c4f0 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f3828ff9700 1 -- 172.21.15.12:0/1646733091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f383c011870 con 0x56362c15c4f0 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f3828ff9700 1 -- 172.21.15.12:0/1646733091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f383c011a30 con 0x56362c15c4f0 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f3853488b80 10 client.15225 fetch_fsmap learned FSMap version 23 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f3853488b80 1 -- 172.21.15.12:0/1646733091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56362bed9b10 con 0x56362c15c4f0 2024-05-08T23:46:16.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.480+0000 7f3828ff9700 1 -- 172.21.15.12:0/1646733091 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56362c005220 con 0x56362c15c4f0 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.481+0000 7f3828ff9700 1 -- 172.21.15.12:0/1646733091 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f383c01c420 con 0x56362c15c4f0 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.481+0000 7f3853488b80 10 client.15225 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.481+0000 7f3853488b80 10 client.15225 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.481+0000 7f3853488b80 20 client.15225 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.481+0000 7f3853488b80 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56362c170530 0x56362c176370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.481+0000 7f3853488b80 4 client.15225 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.481+0000 7f3853488b80 1 -- 172.21.15.12:0/1646733091 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56362c12b7c0 con 0x56362c170530 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.481+0000 7f3844869700 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56362c170530 0x56362c176370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.481+0000 7f3844869700 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56362c170530 0x56362c176370 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4732d70198e38a9 server_cookie=6709cc3e4c5a75db in_seq=0 out_seq=0 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.481+0000 7f3828ff9700 10 client.15225 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3828ff9700 1 -- 172.21.15.12:0/1646733091 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56362c12b7c0 con 0x56362c170530 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3828ff9700 10 client.15225 handle_command_reply: tid=0 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3853488b80 1 client.15225 shutdown 2024-05-08T23:46:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3853488b80 1 -- 172.21.15.12:0/1646733091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 msgr2=0x56362c165e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3853488b80 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c165e00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f383c012810 tx=0x7f383c012bd0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3853488b80 1 -- 172.21.15.12:0/1646733091 shutdown_connections 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3853488b80 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56362c170530 0x56362c176370 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3853488b80 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15d810 0x56362c16d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3853488b80 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15c4f0 0x56362c165e00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3828ff9700 0 client.15225 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3853488b80 1 --2- 172.21.15.12:0/1646733091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15ce50 0x56362c166340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.482+0000 7f3853488b80 1 -- 172.21.15.12:0/1646733091 >> 172.21.15.12:0/1646733091 conn(0x56362c0ba1c0 msgr2=0x56362c156550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.483+0000 7f3853488b80 1 -- 172.21.15.12:0/1646733091 shutdown_connections 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.483+0000 7f3853488b80 1 -- 172.21.15.12:0/1646733091 wait complete. 2024-05-08T23:46:16.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.483+0000 7f3853488b80 20 client.15225 trim_cache size 0 max 16384 2024-05-08T23:46:16.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3853488b80 1 Processor -- start 2024-05-08T23:46:16.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3853488b80 1 -- start start 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3853488b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15c4f0 0x56362c164ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3853488b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c16d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3853488b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 0x56362c16da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3853488b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c06a2b0 con 0x56362c15ce50 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3853488b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c0a5150 con 0x56362c15c4f0 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3853488b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c04b5a0 con 0x56362c15d810 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3844869700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c16d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3844869700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c16d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33914/0 (socket says 172.21.15.12:33914) 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3844869700 1 -- 172.21.15.12:0/1084566688 learned_addr learned my addr 172.21.15.12:0/1084566688 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f384506a700 1 --2- 172.21.15.12:0/1084566688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15c4f0 0x56362c164ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f384586b700 1 --2- 172.21.15.12:0/1084566688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 0x56362c16da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3844869700 1 -- 172.21.15.12:0/1084566688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 msgr2=0x56362c16da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3844869700 1 --2- 172.21.15.12:0/1084566688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 0x56362c16da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3844869700 1 -- 172.21.15.12:0/1084566688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15c4f0 msgr2=0x56362c164ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3844869700 1 --2- 172.21.15.12:0/1084566688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15c4f0 0x56362c164ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.502+0000 7f3844869700 1 -- 172.21.15.12:0/1084566688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56362c072510 con 0x56362c15ce50 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f384586b700 1 --2- 172.21.15.12:0/1084566688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 0x56362c16da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f3844869700 1 --2- 172.21.15.12:0/1084566688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c16d340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f383400b680 tx=0x7f3834088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6eb2660349f4237 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f382bfff700 1 -- 172.21.15.12:0/1084566688 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383409cb60 con 0x56362c15ce50 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f382bfff700 1 -- 172.21.15.12:0/1084566688 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f383409d900 con 0x56362c15ce50 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f382bfff700 1 -- 172.21.15.12:0/1084566688 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383409c950 con 0x56362c15ce50 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f3853488b80 1 -- 172.21.15.12:0/1084566688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 msgr2=0x56362c16d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f3853488b80 1 --2- 172.21.15.12:0/1084566688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c16d340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f383400b680 tx=0x7f3834088040 comp rx=0 tx=0).stop 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f3853488b80 1 -- 172.21.15.12:0/1084566688 shutdown_connections 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f3853488b80 1 --2- 172.21.15.12:0/1084566688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 0x56362c16da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f3853488b80 1 --2- 172.21.15.12:0/1084566688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15c4f0 0x56362c164ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f3853488b80 1 --2- 172.21.15.12:0/1084566688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c16d340 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.503+0000 7f3853488b80 1 -- 172.21.15.12:0/1084566688 >> 172.21.15.12:0/1084566688 conn(0x56362c0ba1c0 msgr2=0x56362c1531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 1 -- 172.21.15.12:0/1084566688 shutdown_connections 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 1 -- 172.21.15.12:0/1084566688 wait complete. 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 1 -- start start 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 1 Processor -- start 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15c4f0 0x56362c155f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c156470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 0x56362c16ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c16cf80 con 0x56362c15ce50 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c16d0f0 con 0x56362c15c4f0 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362c16d2b0 con 0x56362c15d810 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.504+0000 7f3853488b80 10 client.0 _ll_register_callbacks cb 0x56362c051ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3844869700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3844869700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33922/0 (socket says 172.21.15.12:33922) 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3844869700 1 -- 172.21.15.12:0/3924212174 learned_addr learned my addr 172.21.15.12:0/3924212174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f384506a700 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15c4f0 0x56362c155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f384586b700 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 0x56362c16ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3844869700 1 -- 172.21.15.12:0/3924212174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 msgr2=0x56362c16ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3844869700 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 0x56362c16ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3844869700 1 -- 172.21.15.12:0/3924212174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15c4f0 msgr2=0x56362c155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3844869700 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15c4f0 0x56362c155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3844869700 1 -- 172.21.15.12:0/3924212174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384019f2a0 con 0x56362c15ce50 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3844869700 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c156470 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f38340cd8b0 tx=0x7f3834005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2cfcf8f074bf789 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3828ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3828ff9700 1 -- 172.21.15.12:0/3924212174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38340a32c0 con 0x56362c15ce50 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3828ff9700 1 -- 172.21.15.12:0/3924212174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f383409d200 con 0x56362c15ce50 2024-05-08T23:46:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3853488b80 1 -- 172.21.15.12:0/3924212174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3838014ac0 con 0x56362c15ce50 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3828ff9700 1 -- 172.21.15.12:0/3924212174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f383409cb60 con 0x56362c15ce50 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3853488b80 10 client.15231 fetch_fsmap learned FSMap version 23 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.505+0000 7f3853488b80 1 -- 172.21.15.12:0/3924212174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56362bf31c80 con 0x56362c15ce50 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.506+0000 7f3828ff9700 1 -- 172.21.15.12:0/3924212174 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3838014ac0 con 0x56362c15ce50 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.506+0000 7f3828ff9700 1 -- 172.21.15.12:0/3924212174 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38340b5d20 con 0x56362c15ce50 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.506+0000 7f3853488b80 10 client.15231 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.506+0000 7f3853488b80 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.506+0000 7f3853488b80 20 client.15231 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.506+0000 7f3853488b80 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56362c1cc2a0 0x56362c1c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.506+0000 7f3853488b80 4 client.15231 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.506+0000 7f3853488b80 1 -- 172.21.15.12:0/3924212174 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56362c0a4e70 con 0x56362c1cc2a0 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.506+0000 7f384506a700 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56362c1cc2a0 0x56362c1c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3828ff9700 10 client.15231 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f384506a700 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56362c1cc2a0 0x56362c1c8d30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f7b5d6c5a9a97d server_cookie=2c802d01f415b986 in_seq=0 out_seq=0 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3828ff9700 1 -- 172.21.15.12:0/3924212174 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56362c0a4e70 con 0x56362c1cc2a0 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3828ff9700 10 client.15231 handle_command_reply: tid=0 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 client.15231 shutdown 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 -- 172.21.15.12:0/3924212174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 msgr2=0x56362c156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c156470 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f38340cd8b0 tx=0x7f3834005c90 comp rx=0 tx=0).stop 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 -- 172.21.15.12:0/3924212174 shutdown_connections 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56362c15d810 0x56362c16ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56362c1cc2a0 0x56362c1c8d30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56362c15c4f0 0x56362c155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 --2- 172.21.15.12:0/3924212174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56362c15ce50 0x56362c156470 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3828ff9700 0 client.15231 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 -- 172.21.15.12:0/3924212174 >> 172.21.15.12:0/3924212174 conn(0x56362c0ba1c0 msgr2=0x56362c16da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 -- 172.21.15.12:0/3924212174 shutdown_connections 2024-05-08T23:46:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 1 -- 172.21.15.12:0/3924212174 wait complete. 2024-05-08T23:46:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.507+0000 7f3853488b80 20 client.15231 trim_cache size 0 max 16384 2024-05-08T23:46:16.510 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:46:16.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "00f8fb28-0d94-11ef-bc97-c7b262605968", 2024-05-08T23:46:16.510 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-05-08T23:46:16.510 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24548, 2024-05-08T23:46:16.510 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-08T23:46:16.510 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-08T23:46:16.510 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-08T23:46:16.510 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 72.477140517999999, 2024-05-08T23:46:16.511 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 23, 2024-05-08T23:46:16.511 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-05-08T23:46:16.511 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:46:16.511 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 78.175492073000001 2024-05-08T23:46:16.511 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 -- 172.21.15.12:0/3479157033 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f382c0a1140 msgr2=0x7f382c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f382c0a1140 0x7f382c0a3600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f38380091f0 tx=0x7f3838006000 comp rx=0 tx=0).stop 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 -- 172.21.15.12:0/3479157033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3840104310 msgr2=0x7f38401973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3840104310 0x7f38401973a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3834000c00 tx=0x7f3834005ef0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 -- 172.21.15.12:0/3479157033 shutdown_connections 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f382c0a1140 0x7f382c0a3600 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3840106c40 0x7f384019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38401019e0 0x7f3840196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 --2- 172.21.15.12:0/3479157033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3840104310 0x7f38401973a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 -- 172.21.15.12:0/3479157033 >> 172.21.15.12:0/3479157033 conn(0x7f38400fb3f0 msgr2=0x7f3840102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 -- 172.21.15.12:0/3479157033 shutdown_connections 2024-05-08T23:46:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.508+0000 7f38472ce700 1 -- 172.21.15.12:0/3479157033 wait complete. 2024-05-08T23:46:16.517 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T23:46:16.530 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T23:46:16.530 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T23:46:16.530 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T23:46:16.530 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-08T23:46:16.531 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:46:16.531 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-08T23:46:16.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-08T23:46:16.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:16 smithi012 ceph-mon[42503]: pgmap v263: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:46:16.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:16 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2576063916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:16.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:16 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1168344324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.672+0000 7fd943e03700 1 -- 172.21.15.12:0/2504128647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd93c0ffe00 msgr2=0x7fd93c100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.672+0000 7fd943e03700 1 --2- 172.21.15.12:0/2504128647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd93c0ffe00 0x7fd93c100220 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd9380099c0 tx=0x7fd938009cd0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.672+0000 7fd943e03700 1 -- 172.21.15.12:0/2504128647 shutdown_connections 2024-05-08T23:46:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.672+0000 7fd943e03700 1 --2- 172.21.15.12:0/2504128647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd93c101990 0x7fd93c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.672+0000 7fd943e03700 1 --2- 172.21.15.12:0/2504128647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd93c0ffe00 0x7fd93c100220 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.672+0000 7fd943e03700 1 --2- 172.21.15.12:0/2504128647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd93c100ff0 0x7fd93c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 -- 172.21.15.12:0/2504128647 >> 172.21.15.12:0/2504128647 conn(0x7fd93c0fb390 msgr2=0x7fd93c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:16.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 -- 172.21.15.12:0/2504128647 shutdown_connections 2024-05-08T23:46:16.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 -- 172.21.15.12:0/2504128647 wait complete. 2024-05-08T23:46:16.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 Processor -- start 2024-05-08T23:46:16.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 -- start start 2024-05-08T23:46:16.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd93c0ffe00 0x7fd93c068da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd93c100ff0 0x7fd93c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd93c101990 0x7fd93c19e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c19e980 con 0x7fd93c0ffe00 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c19eaf0 con 0x7fd93c100ff0 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.673+0000 7fd943e03700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c19ecb0 con 0x7fd93c101990 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd941b9f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd93c0ffe00 0x7fd93c068da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd941b9f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd93c0ffe00 0x7fd93c068da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33946/0 (socket says 172.21.15.12:33946) 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd941b9f700 1 -- 172.21.15.12:0/3962129605 learned_addr learned my addr 172.21.15.12:0/3962129605 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd9423a0700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd93c101990 0x7fd93c19e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd94139e700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd93c100ff0 0x7fd93c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd941b9f700 1 -- 172.21.15.12:0/3962129605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd93c101990 msgr2=0x7fd93c19e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd941b9f700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd93c101990 0x7fd93c19e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd941b9f700 1 -- 172.21.15.12:0/3962129605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd93c100ff0 msgr2=0x7fd93c19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd941b9f700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd93c100ff0 0x7fd93c19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd941b9f700 1 -- 172.21.15.12:0/3962129605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380095f0 con 0x7fd93c0ffe00 2024-05-08T23:46:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd941b9f700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd93c0ffe00 0x7fd93c068da0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd938006010 tx=0x7fd93800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd932ffd700 1 -- 172.21.15.12:0/3962129605 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93801d070 con 0x7fd93c0ffe00 2024-05-08T23:46:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd932ffd700 1 -- 172.21.15.12:0/3962129605 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd938005020 con 0x7fd93c0ffe00 2024-05-08T23:46:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd943e03700 1 -- 172.21.15.12:0/3962129605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93c19ef30 con 0x7fd93c0ffe00 2024-05-08T23:46:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd932ffd700 1 -- 172.21.15.12:0/3962129605 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd938004bb0 con 0x7fd93c0ffe00 2024-05-08T23:46:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.674+0000 7fd943e03700 1 -- 172.21.15.12:0/3962129605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93c19f2a0 con 0x7fd93c0ffe00 2024-05-08T23:46:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.675+0000 7fd943e03700 1 -- 172.21.15.12:0/3962129605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93c107220 con 0x7fd93c0ffe00 2024-05-08T23:46:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.676+0000 7fd932ffd700 1 -- 172.21.15.12:0/3962129605 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd938005190 con 0x7fd93c0ffe00 2024-05-08T23:46:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.677+0000 7fd932ffd700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9280a11b0 0x7fd9280a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.677+0000 7fd932ffd700 1 -- 172.21.15.12:0/3962129605 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd9380cd6d0 con 0x7fd93c0ffe00 2024-05-08T23:46:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.677+0000 7fd94139e700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9280a11b0 0x7fd9280a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.678+0000 7fd94139e700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9280a11b0 0x7fd9280a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd93c069360 tx=0x7fd92c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.681+0000 7fd932ffd700 1 -- 172.21.15.12:0/3962129605 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd938096d90 con 0x7fd93c0ffe00 2024-05-08T23:46:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:16 smithi204 ceph-mon[47373]: pgmap v263: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:46:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:16 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2576063916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:16 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1168344324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:16 smithi160 ceph-mon[46152]: pgmap v263: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:46:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:16 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2576063916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:16 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1168344324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:16.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:16.968+0000 7fd943e03700 1 -- 172.21.15.12:0/3962129605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fd93c10b7f0 con 0x7fd93c0ffe00 2024-05-08T23:46:17.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.371+0000 7fd932ffd700 1 -- 172.21.15.12:0/3962129605 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7fd938004d10 con 0x7fd93c0ffe00 2024-05-08T23:46:17.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.372+0000 7fd932ffd700 1 -- 172.21.15.12:0/3962129605 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7fd93805b5b0 con 0x7fd93c0ffe00 2024-05-08T23:46:17.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 -- 172.21.15.12:0/3962129605 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9280a11b0 msgr2=0x7fd9280a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:17.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9280a11b0 0x7fd9280a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd93c069360 tx=0x7fd92c009260 comp rx=0 tx=0).stop 2024-05-08T23:46:17.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 -- 172.21.15.12:0/3962129605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd93c0ffe00 msgr2=0x7fd93c068da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd93c0ffe00 0x7fd93c068da0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd938006010 tx=0x7fd93800f690 comp rx=0 tx=0).stop 2024-05-08T23:46:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 -- 172.21.15.12:0/3962129605 shutdown_connections 2024-05-08T23:46:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd93c101990 0x7fd93c19e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd93c0ffe00 0x7fd93c068da0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7fd9280a11b0 0x7fd9280a3670 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 --2- 172.21.15.12:0/3962129605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd93c100ff0 0x7fd93c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 -- 172.21.15.12:0/3962129605 >> 172.21.15.12:0/3962129605 conn(0x7fd93c0fb390 msgr2=0x7fd93c104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 -- 172.21.15.12:0/3962129605 shutdown_connections 2024-05-08T23:46:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.375+0000 7fd943e03700 1 -- 172.21.15.12:0/3962129605 wait complete. 2024-05-08T23:46:17.386 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-08T23:46:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.535+0000 7f9ba1082700 1 -- 172.21.15.12:0/3505521802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b9c102070 msgr2=0x7f9b9c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.535+0000 7f9ba1082700 1 --2- 172.21.15.12:0/3505521802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b9c102070 0x7f9b9c1024f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009990 tx=0x7f9b90009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.535+0000 7f9ba1082700 1 -- 172.21.15.12:0/3505521802 shutdown_connections 2024-05-08T23:46:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.535+0000 7f9ba1082700 1 --2- 172.21.15.12:0/3505521802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c102a30 0x7f9b9c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.535+0000 7f9ba1082700 1 --2- 172.21.15.12:0/3505521802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b9c100f20 0x7f9b9c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.535+0000 7f9ba1082700 1 --2- 172.21.15.12:0/3505521802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b9c102070 0x7f9b9c1024f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.535+0000 7f9ba1082700 1 -- 172.21.15.12:0/3505521802 >> 172.21.15.12:0/3505521802 conn(0x7f9b9c0fc490 msgr2=0x7f9b9c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.535+0000 7f9ba1082700 1 -- 172.21.15.12:0/3505521802 shutdown_connections 2024-05-08T23:46:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.535+0000 7f9ba1082700 1 -- 172.21.15.12:0/3505521802 wait complete. 2024-05-08T23:46:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.536+0000 7f9ba1082700 1 Processor -- start 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.536+0000 7f9ba1082700 1 -- start start 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.536+0000 7f9ba1082700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c100f20 0x7f9b9c196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.536+0000 7f9ba1082700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b9c102070 0x7f9b9c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.536+0000 7f9ba1082700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b9c102a30 0x7f9b9c19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9ba1082700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c19e990 con 0x7f9b9c102070 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9ba1082700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c19ead0 con 0x7f9b9c102a30 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9ba1082700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c19ec60 con 0x7f9b9c100f20 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b9c102070 0x7f9b9c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b9c102070 0x7f9b9c1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33974/0 (socket says 172.21.15.12:33974) 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9a59c700 1 -- 172.21.15.12:0/179205290 learned_addr learned my addr 172.21.15.12:0/179205290 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9b59e700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b9c102a30 0x7f9b9c19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9ad9d700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c100f20 0x7f9b9c196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9b59e700 1 -- 172.21.15.12:0/179205290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c100f20 msgr2=0x7f9b9c196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9b59e700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c100f20 0x7f9b9c196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9b59e700 1 -- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b9c102070 msgr2=0x7f9b9c1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9b59e700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b9c102070 0x7f9b9c1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9b59e700 1 -- 172.21.15.12:0/179205290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b900095f0 con 0x7f9b9c102a30 2024-05-08T23:46:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9a59c700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b9c102070 0x7f9b9c1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9ad9d700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c100f20 0x7f9b9c196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b9b59e700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b9c102a30 0x7f9b9c19e3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b94007f10 tx=0x7f9b9400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.537+0000 7f9b83fff700 1 -- 172.21.15.12:0/179205290 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94009ea0 con 0x7f9b9c102a30 2024-05-08T23:46:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.538+0000 7f9b83fff700 1 -- 172.21.15.12:0/179205290 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b9400f040 con 0x7f9b9c102a30 2024-05-08T23:46:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.538+0000 7f9ba1082700 1 -- 172.21.15.12:0/179205290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c19ef40 con 0x7f9b9c102a30 2024-05-08T23:46:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.538+0000 7f9b83fff700 1 -- 172.21.15.12:0/179205290 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94004400 con 0x7f9b9c102a30 2024-05-08T23:46:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.538+0000 7f9ba1082700 1 -- 172.21.15.12:0/179205290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c19f370 con 0x7f9b9c102a30 2024-05-08T23:46:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.539+0000 7f9ba1082700 1 -- 172.21.15.12:0/179205290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c108390 con 0x7f9b9c102a30 2024-05-08T23:46:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.540+0000 7f9b83fff700 1 -- 172.21.15.12:0/179205290 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f9b94004ad0 con 0x7f9b9c102a30 2024-05-08T23:46:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.540+0000 7f9b83fff700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b840a1200 0x7f9b840a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.540+0000 7f9b9ad9d700 1 -- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b840a1200 msgr2=0x7f9b840a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1305168755 2024-05-08T23:46:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.541+0000 7f9b83fff700 1 -- 172.21.15.12:0/179205290 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9b940cc590 con 0x7f9b9c102a30 2024-05-08T23:46:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.541+0000 7f9b9ad9d700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b840a1200 0x7f9b840a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:46:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.544+0000 7f9b83fff700 1 -- 172.21.15.12:0/179205290 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9b94095bc0 con 0x7f9b9c102a30 2024-05-08T23:46:17.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:17 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3962129605' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:46:17.665 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:17 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: ignoring --setuser ceph since I am not root 2024-05-08T23:46:17.665 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:17 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: ignoring --setgroup ceph since I am not root 2024-05-08T23:46:17.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.742+0000 7f9b9ad9d700 1 -- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b840a1200 msgr2=0x7f9b840a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1305168755 2024-05-08T23:46:17.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.742+0000 7f9b9ad9d700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b840a1200 0x7f9b840a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:46:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.765+0000 7f9ba1082700 1 -- 172.21.15.12:0/179205290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f9b9c067e30 con 0x7f9b9c102a30 2024-05-08T23:46:17.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:17 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3962129605' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:46:17.764 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: ignoring --setuser ceph since I am not root 2024-05-08T23:46:17.764 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: ignoring --setgroup ceph since I am not root 2024-05-08T23:46:17.764 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:17 smithi204 ceph-mgr[49254]: -- 172.21.15.204:0/54750490 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d3203e21a0 con 0x55d3203b8c00 2024-05-08T23:46:17.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:17 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3962129605' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:46:17.766 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: ignoring --setuser ceph since I am not root 2024-05-08T23:46:17.766 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: ignoring --setgroup ceph since I am not root 2024-05-08T23:46:17.766 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:17 smithi160 ceph-mgr[47653]: -- 172.21.15.160:0/1637595072 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cb13c341a0 con 0x55cb13c0a800 2024-05-08T23:46:17.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.771+0000 7f9b83fff700 1 -- 172.21.15.12:0/179205290 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9b940190d0 con 0x7f9b9c102a30 2024-05-08T23:46:17.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.773+0000 7f9ba1082700 1 -- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b840a1200 msgr2=0x7f9b840a36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.773+0000 7f9ba1082700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b840a1200 0x7f9b840a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.773+0000 7f9ba1082700 1 -- 172.21.15.12:0/179205290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b9c102a30 msgr2=0x7f9b9c19e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.773+0000 7f9ba1082700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b9c102a30 0x7f9b9c19e3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b94007f10 tx=0x7f9b9400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.774+0000 7f9ba1082700 1 -- 172.21.15.12:0/179205290 shutdown_connections 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.774+0000 7f9ba1082700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f9b840a1200 0x7f9b840a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.774+0000 7f9ba1082700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b9c102a30 0x7f9b9c19e3c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.774+0000 7f9ba1082700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b9c100f20 0x7f9b9c196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.774+0000 7f9ba1082700 1 --2- 172.21.15.12:0/179205290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b9c102070 0x7f9b9c1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.774+0000 7f9ba1082700 1 -- 172.21.15.12:0/179205290 >> 172.21.15.12:0/179205290 conn(0x7f9b9c0fc490 msgr2=0x7f9b9c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.774+0000 7f9ba1082700 1 -- 172.21.15.12:0/179205290 shutdown_connections 2024-05-08T23:46:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.774+0000 7f9ba1082700 1 -- 172.21.15.12:0/179205290 wait complete. 2024-05-08T23:46:17.785 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-08T23:46:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.928+0000 7f9873721700 1 -- 172.21.15.12:0/2223121944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f986c101890 msgr2=0x7f986c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.928+0000 7f9873721700 1 --2- 172.21.15.12:0/2223121944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f986c101890 0x7f986c103c80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9868009990 tx=0x7f9868009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.928+0000 7f9873721700 1 -- 172.21.15.12:0/2223121944 shutdown_connections 2024-05-08T23:46:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.928+0000 7f9873721700 1 --2- 172.21.15.12:0/2223121944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f986c106af0 0x7f986c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.928+0000 7f9873721700 1 --2- 172.21.15.12:0/2223121944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f986c101890 0x7f986c103c80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.928+0000 7f9873721700 1 --2- 172.21.15.12:0/2223121944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f986c1041c0 0x7f986c1065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.928+0000 7f9873721700 1 -- 172.21.15.12:0/2223121944 >> 172.21.15.12:0/2223121944 conn(0x7f986c0fb2a0 msgr2=0x7f986c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.928+0000 7f9873721700 1 -- 172.21.15.12:0/2223121944 shutdown_connections 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.928+0000 7f9873721700 1 -- 172.21.15.12:0/2223121944 wait complete. 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9873721700 1 Processor -- start 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9873721700 1 -- start start 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9873721700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f986c101890 0x7f986c196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9873721700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f986c1041c0 0x7f986c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9873721700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f986c106af0 0x7f986c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9873721700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c19e830 con 0x7f986c1041c0 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9873721700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c19e9a0 con 0x7f986c101890 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9873721700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c19eb60 con 0x7f986c106af0 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9870cbc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f986c1041c0 0x7f986c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9870cbc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f986c1041c0 0x7f986c197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33996/0 (socket says 172.21.15.12:33996) 2024-05-08T23:46:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9870cbc700 1 -- 172.21.15.12:0/566415943 learned_addr learned my addr 172.21.15.12:0/566415943 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f9871cbe700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f986c106af0 0x7f986c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.929+0000 7f98714bd700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f986c101890 0x7f986c196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f9871cbe700 1 -- 172.21.15.12:0/566415943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f986c101890 msgr2=0x7f986c196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f9871cbe700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f986c101890 0x7f986c196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f9871cbe700 1 -- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f986c1041c0 msgr2=0x7f986c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f9871cbe700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f986c1041c0 0x7f986c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f9871cbe700 1 -- 172.21.15.12:0/566415943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680095f0 con 0x7f986c106af0 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f9871cbe700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f986c106af0 0x7f986c19e260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f986400efc0 tx=0x7f986400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f98627fc700 1 -- 172.21.15.12:0/566415943 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864018070 con 0x7f986c106af0 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f98627fc700 1 -- 172.21.15.12:0/566415943 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f986400f040 con 0x7f986c106af0 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f9873721700 1 -- 172.21.15.12:0/566415943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c19ede0 con 0x7f986c106af0 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f98627fc700 1 -- 172.21.15.12:0/566415943 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400cd40 con 0x7f986c106af0 2024-05-08T23:46:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.930+0000 7f9873721700 1 -- 172.21.15.12:0/566415943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c19f210 con 0x7f986c106af0 2024-05-08T23:46:17.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.931+0000 7f9873721700 1 -- 172.21.15.12:0/566415943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c0fce20 con 0x7f986c106af0 2024-05-08T23:46:17.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.932+0000 7f98627fc700 1 -- 172.21.15.12:0/566415943 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f9864003a00 con 0x7f986c106af0 2024-05-08T23:46:17.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.933+0000 7f98627fc700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98580a11f0 0x7f98580a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:17.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.933+0000 7f98714bd700 1 -- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98580a11f0 msgr2=0x7f98580a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1305168755 2024-05-08T23:46:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.933+0000 7f98627fc700 1 -- 172.21.15.12:0/566415943 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f98640cc6b0 con 0x7f986c106af0 2024-05-08T23:46:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.933+0000 7f98714bd700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98580a11f0 0x7f98580a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:46:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:17.938+0000 7f98627fc700 1 -- 172.21.15.12:0/566415943 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9864095d70 con 0x7f986c106af0 2024-05-08T23:46:18.067 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:17.805+0000 7f09e22eb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:46:18.068 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:17.897+0000 7f09e22eb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:46:18.068 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:17.986+0000 7f09e22eb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:46:18.068 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:18 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:18.071+0000 7f09e22eb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:46:18.074 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:17.811+0000 7f736eb10200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:46:18.074 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:17.904+0000 7f736eb10200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:46:18.074 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:17.993+0000 7f736eb10200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:46:18.076 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:17 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:17.870+0000 7f7d00932200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:46:18.076 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:17 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:17.979+0000 7f7d00932200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:46:18.076 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:18 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:18.080+0000 7f7d00932200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:46:18.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.134+0000 7f98714bd700 1 -- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98580a11f0 msgr2=0x7f98580a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1305168755 2024-05-08T23:46:18.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.134+0000 7f98714bd700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98580a11f0 0x7f98580a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:46:18.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.158+0000 7f9873721700 1 -- 172.21.15.12:0/566415943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f986c10c5f0 con 0x7f986c106af0 2024-05-08T23:46:18.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.164+0000 7f98627fc700 1 -- 172.21.15.12:0/566415943 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9864099390 con 0x7f986c106af0 2024-05-08T23:46:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 -- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98580a11f0 msgr2=0x7f98580a36b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:46:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98580a11f0 0x7f98580a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 -- 172.21.15.12:0/566415943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f986c106af0 msgr2=0x7f986c19e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f986c106af0 0x7f986c19e260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f986400efc0 tx=0x7f986400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:46:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 -- 172.21.15.12:0/566415943 shutdown_connections 2024-05-08T23:46:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f986c106af0 0x7f986c19e260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f98580a11f0 0x7f98580a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f986c101890 0x7f986c196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 --2- 172.21.15.12:0/566415943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f986c1041c0 0x7f986c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 -- 172.21.15.12:0/566415943 >> 172.21.15.12:0/566415943 conn(0x7f986c0fb2a0 msgr2=0x7f986c103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 -- 172.21.15.12:0/566415943 shutdown_connections 2024-05-08T23:46:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:18.167+0000 7f9873721700 1 -- 172.21.15.12:0/566415943 wait complete. 2024-05-08T23:46:18.177 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T23:46:18.203 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T23:46:18.203 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-08T23:46:18.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:18.204 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-08T23:46:18.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:18.225 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-05-08T23:46:18.299 INFO:teuthology.orchestra.run.smithi012.stderr:Cannot find device "ceph-brx" 2024-05-08T23:46:18.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:46:18.301 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-08T23:46:18.301 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:46:18.301 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:46:18.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T23:46:18.357 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:18 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:18.178+0000 7f7d00932200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:46:18.357 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:18.079+0000 7f736eb10200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:46:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:46:18.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:18 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3962129605' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T23:46:18.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:18 smithi012 ceph-mon[42503]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-08T23:46:18.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:18 smithi012 ceph-mon[42503]: from='client.? ' entity='client.admin' 2024-05-08T23:46:18.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:18 smithi012 ceph-mon[42503]: from='client.? ' entity='client.admin' 2024-05-08T23:46:18.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:18 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3962129605' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T23:46:18.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:18 smithi204 ceph-mon[47373]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-08T23:46:18.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:18 smithi204 ceph-mon[47373]: from='client.? ' entity='client.admin' 2024-05-08T23:46:18.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:18 smithi204 ceph-mon[47373]: from='client.? ' entity='client.admin' 2024-05-08T23:46:18.706 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:18 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3962129605' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T23:46:18.706 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:18 smithi160 ceph-mon[46152]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-08T23:46:18.706 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:18 smithi160 ceph-mon[46152]: from='client.? ' entity='client.admin' 2024-05-08T23:46:18.706 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:18 smithi160 ceph-mon[46152]: from='client.? ' entity='client.admin' 2024-05-08T23:46:18.953 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:18 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:18.703+0000 7f09e22eb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:46:18.960 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:18.711+0000 7f736eb10200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:46:19.094 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:18 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:18.946+0000 7f7d00932200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:46:19.263 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:18 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:18.957+0000 7f09e22eb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:46:19.263 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:19.044+0000 7f09e22eb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:46:19.263 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:19.155+0000 7f09e22eb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:46:19.265 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:18.965+0000 7f736eb10200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:46:19.266 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:19.053+0000 7f736eb10200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:46:19.266 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:19.165+0000 7f736eb10200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:46:19.419 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:19 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:19.240+0000 7f7d00932200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:46:19.420 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:19 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:19.339+0000 7f7d00932200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:46:19.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.729+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/605987377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24107020 msgr2=0x7f7e24107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:19.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.729+0000 7f7e2b4a9700 1 --2- 172.21.15.12:0/605987377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24107020 0x7f7e24107400 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009990 tx=0x7f7e20009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.729+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/605987377 shutdown_connections 2024-05-08T23:46:19.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.729+0000 7f7e2b4a9700 1 --2- 172.21.15.12:0/605987377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e240692f0 0x7f7e2410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.729+0000 7f7e2b4a9700 1 --2- 172.21.15.12:0/605987377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24107020 0x7f7e24107400 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.729+0000 7f7e2b4a9700 1 --2- 172.21.15.12:0/605987377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e24068950 0x7f7e24068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.729+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/605987377 >> 172.21.15.12:0/605987377 conn(0x7f7e24075f80 msgr2=0x7f7e24076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:19.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.729+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/605987377 shutdown_connections 2024-05-08T23:46:19.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.729+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/605987377 wait complete. 2024-05-08T23:46:19.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e2b4a9700 1 Processor -- start 2024-05-08T23:46:19.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e2b4a9700 1 -- start start 2024-05-08T23:46:19.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e2b4a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24068950 0x7f7e24197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:19.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e2b4a9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e240692f0 0x7f7e241978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:19.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e2b4a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e24107020 0x7f7e2419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e2b4a9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419c290 con 0x7f7e24068950 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e2b4a9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419c400 con 0x7f7e24107020 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e2b4a9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419c5c0 con 0x7f7e240692f0 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e28a44700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e240692f0 0x7f7e241978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e28a44700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e240692f0 0x7f7e241978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:37102/0 (socket says 172.21.15.12:37102) 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e28a44700 1 -- 172.21.15.12:0/1477801292 learned_addr learned my addr 172.21.15.12:0/1477801292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.730+0000 7f7e29a46700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e24107020 0x7f7e2419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e29245700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24068950 0x7f7e24197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e28a44700 1 -- 172.21.15.12:0/1477801292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e24107020 msgr2=0x7f7e2419bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e28a44700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e24107020 0x7f7e2419bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e28a44700 1 -- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24068950 msgr2=0x7f7e24197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e28a44700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24068950 0x7f7e24197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e28a44700 1 -- 172.21.15.12:0/1477801292 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e200095f0 con 0x7f7e240692f0 2024-05-08T23:46:19.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e29a46700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e24107020 0x7f7e2419bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:19.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e28a44700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e240692f0 0x7f7e241978d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400e9a0 tx=0x7f7e1400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:19.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e1a7fc700 1 -- 172.21.15.12:0/1477801292 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14009790 con 0x7f7e240692f0 2024-05-08T23:46:19.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e1a7fc700 1 -- 172.21.15.12:0/1477801292 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e1400cbd0 con 0x7f7e240692f0 2024-05-08T23:46:19.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e1a7fc700 1 -- 172.21.15.12:0/1477801292 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14018680 con 0x7f7e240692f0 2024-05-08T23:46:19.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/1477801292 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2419c8a0 con 0x7f7e240692f0 2024-05-08T23:46:19.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.731+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/1477801292 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e241a4100 con 0x7f7e240692f0 2024-05-08T23:46:19.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.732+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/1477801292 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e24109540 con 0x7f7e240692f0 2024-05-08T23:46:19.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.733+0000 7f7e1a7fc700 1 -- 172.21.15.12:0/1477801292 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f7e1400cd40 con 0x7f7e240692f0 2024-05-08T23:46:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.734+0000 7f7e1a7fc700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7e100a1280 0x7f7e100a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.734+0000 7f7e29245700 1 -- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7e100a1280 msgr2=0x7f7e100a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1305168755 2024-05-08T23:46:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.734+0000 7f7e1a7fc700 1 -- 172.21.15.12:0/1477801292 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7e14014070 con 0x7f7e240692f0 2024-05-08T23:46:19.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.735+0000 7f7e29245700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7e100a1280 0x7f7e100a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:46:19.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.737+0000 7f7e1a7fc700 1 -- 172.21.15.12:0/1477801292 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7e14096370 con 0x7f7e240692f0 2024-05-08T23:46:19.764 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:19.314+0000 7f09e22eb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:46:19.764 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:19.441+0000 7f09e22eb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:46:19.765 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:19.528+0000 7f09e22eb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:46:19.766 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:19.326+0000 7f736eb10200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:46:19.766 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:19.454+0000 7f736eb10200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:46:19.767 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:19.541+0000 7f736eb10200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:46:19.811 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:19 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:19.468+0000 7f7d00932200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:46:19.811 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:19 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:19.662+0000 7f7d00932200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:46:19.812 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:19 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:19.815+0000 7f7d00932200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:46:19.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.936+0000 7f7e29245700 1 -- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7e100a1280 msgr2=0x7f7e100a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1305168755 2024-05-08T23:46:19.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.936+0000 7f7e29245700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7e100a1280 0x7f7e100a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:46:19.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.996+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/1477801292 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7e2419ca90 con 0x7f7e240692f0 2024-05-08T23:46:19.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.996+0000 7f7e1a7fc700 1 -- 172.21.15.12:0/1477801292 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7e14099990 con 0x7f7e240692f0 2024-05-08T23:46:19.992 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7e100a1280 msgr2=0x7f7e100a3740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7e100a1280 0x7f7e100a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/1477801292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e240692f0 msgr2=0x7f7e241978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e240692f0 0x7f7e241978d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400e9a0 tx=0x7f7e1400ed60 comp rx=0 tx=0).stop 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/1477801292 shutdown_connections 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:6800/1305168755,v1:172.21.15.12:6801/1305168755] conn(0x7f7e100a1280 0x7f7e100a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e24107020 0x7f7e2419bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24068950 0x7f7e24197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 --2- 172.21.15.12:0/1477801292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e240692f0 0x7f7e241978d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/1477801292 >> 172.21.15.12:0/1477801292 conn(0x7f7e24075f80 msgr2=0x7f7e240fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/1477801292 shutdown_connections 2024-05-08T23:46:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:19.999+0000 7f7e2b4a9700 1 -- 172.21.15.12:0/1477801292 wait complete. 2024-05-08T23:46:20.165 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:19 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:19.913+0000 7f7d00932200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:46:20.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:20 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1477801292' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:46:20.263 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:19.900+0000 7f09e22eb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:46:20.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:20 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1477801292' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:46:20.265 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:19.912+0000 7f736eb10200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:46:20.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:20 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1477801292' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:46:20.353 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:46:20.353 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:46:20.353 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-05-08T23:46:20.353 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:46:20.353 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:46:20.353 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:46:20.353 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:46:20.353 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:46:20.353 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:46:20.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:20.354 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-05-08T23:46:20.415 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:46:20.415 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:46:20.415 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:46:20.415 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:46:20.415 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-05-08T23:46:20.415 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:46:20.415 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:46:20.415 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 41708sec preferred_lft 41708sec 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-05-08T23:46:20.416 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-05-08T23:46:20.417 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T23:46:20.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:20.417 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2024-05-08T23:46:20.417 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-08T23:46:20.417 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-05-08T23:46:20.417 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-05-08T23:46:20.417 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-05-08T23:46:20.417 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T23:46:20.418 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-08T23:46:20.501 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:20 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:20.357+0000 7f7d00932200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:46:20.578 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:20.292+0000 7f09e22eb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:46:20.578 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:20.489+0000 7f09e22eb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:46:20.593 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:20 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:20.307+0000 7f736eb10200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:46:20.594 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:20 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:20.506+0000 7f736eb10200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:46:20.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:20.608 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:46:20.642 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-05-08T23:46:20.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:20.643 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-05-08T23:46:20.720 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-05-08T23:46:20.720 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:46:20.720 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:46:20.720 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:46:20.720 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:46:20.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:20.721 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2024-05-08T23:46:20.721 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-08T23:46:20.721 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:46:20.721 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:46:20.721 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:46:20.721 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-08T23:46:20.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:20.952 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-08T23:46:20.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:20.971 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-05-08T23:46:21.013 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:20.582+0000 7f09e22eb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:46:21.015 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:20 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:20.599+0000 7f736eb10200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:46:21.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:21.031 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2024-05-08T23:46:21.031 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-08T23:46:21.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:46:21.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T23:46:21.031 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-08T23:46:21.056 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:20 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:20.828+0000 7f7d00932200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:46:21.119 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T23:46:21.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:21.120 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2024-05-08T23:46:21.120 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-08T23:46:21.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T23:46:21.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T23:46:21.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T23:46:21.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T23:46:21.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T23:46:21.121 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-08T23:46:21.316 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:21 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:21.060+0000 7f7d00932200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:46:21.316 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:21 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:21.174+0000 7f7d00932200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:46:21.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:21.353 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2024-05-08T23:46:21.353 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-08T23:46:21.353 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-05-08T23:46:21.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T23:46:21.354 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-08T23:46:21.459 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:46:21.459 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:46:21.478 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:46:21.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:46:21.479 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:46:21.537 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-08T23:46:21.538 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-08T23:46:21.538 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685 2024-05-08T23:46:21.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:21.538 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-08T23:46:21.630 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-08T23:46:21.630 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-08T23:46:21.732 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-08T23:46:21.733 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-08T23:46:21.733 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.204:3300" 2024-05-08T23:46:21.733 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with new device syntax: 0@00f8fb28-0d94-11ef-bc97-c7b262605968.cephfs=/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685 2024-05-08T23:46:21.733 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.12:3300/172.21.15.160:3300/172.21.15.204:3300" will pass to kernel 2024-05-08T23:46:21.733 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:46:21.733 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:46:21.733 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:3300,172.21.15.160:3300,172.21.15.204:3300:/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685 2024-05-08T23:46:21.733 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mds_namespace=cephfs,fsid=00f8fb28-0d94-11ef-bc97-c7b262605968" will pass to kernel 2024-05-08T23:46:21.736 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:46:21.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:21.736 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:46:21.763 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:21 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:21.351+0000 7f09e22eb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:46:21.763 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:21 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:21.447+0000 7f09e22eb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:46:21.763 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:21 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:21.595+0000 7f09e22eb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:46:21.765 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:21.366+0000 7f736eb10200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:46:21.766 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:21.463+0000 7f736eb10200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:46:21.766 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:21.611+0000 7f736eb10200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:46:22.361 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:22 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:22.081+0000 7f7d00932200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:46:22.361 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:22 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:22.193+0000 7f7d00932200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:46:22.513 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:22.228+0000 7f09e22eb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:46:22.513 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:22.410+0000 7f09e22eb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:46:22.515 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:22 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:22.245+0000 7f736eb10200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:46:22.515 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:22 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:22.427+0000 7f736eb10200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:46:22.637 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:22 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:22.365+0000 7f7d00932200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:46:23.013 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:22.605+0000 7f09e22eb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:46:23.015 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:22 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:22.623+0000 7f736eb10200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:46:23.406 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:23 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:23.094+0000 7f7d00932200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:46:23.407 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:23 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:23.306+0000 7f7d00932200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:46:23.665 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:23 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:23.534+0000 7f7d00932200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:46:23.826 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:23 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:23.540+0000 7f09e22eb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:46:23.843 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:23 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:23.559+0000 7f736eb10200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:46:24.110 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:23 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:23.830+0000 7f09e22eb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:46:24.111 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:23 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:23.920+0000 7f09e22eb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:46:24.126 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:23 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:23.847+0000 7f736eb10200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:46:24.126 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:23 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:23.936+0000 7f736eb10200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:46:24.513 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 08 23:46:24 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z[49231]: 2024-05-08T23:46:24.114+0000 7f09e22eb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:46:24.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: Standby manager daemon z restarted 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: Standby manager daemon z started 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: Standby manager daemon y restarted 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: Standby manager daemon y started 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:46:24.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:24 smithi204 ceph-mon[47373]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 08 23:46:24 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y[47630]: 2024-05-08T23:46:24.130+0000 7f736eb10200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: Standby manager daemon z restarted 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: Standby manager daemon z started 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: Standby manager daemon y restarted 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: Standby manager daemon y started 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:46:24.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:46:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:46:24.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:24 smithi160 ceph-mon[46152]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:46:24.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: Standby manager daemon z restarted 2024-05-08T23:46:24.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: Standby manager daemon z started 2024-05-08T23:46:24.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:46:24.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:46:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:46:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.204:0/1844171637' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:46:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: Standby manager daemon y restarted 2024-05-08T23:46:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: Standby manager daemon y started 2024-05-08T23:46:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:46:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:46:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:46:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:24 smithi012 ceph-mon[42503]: from='mgr.? 172.21.15.160:0/1485844139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:46:24.915 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:24 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:24.615+0000 7f7d00932200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:46:25.272 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:25 smithi012 ceph-mon[42503]: mgrmap e19: x(active, since 6m), standbys: y, z 2024-05-08T23:46:25.273 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:24 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:24.949+0000 7f7d00932200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:46:25.273 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.051+0000 7f7d00932200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:46:25.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:25.345 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:46:25.384 INFO:teuthology.orchestra.run.smithi012.stdout:client15273 2024-05-08T23:46:25.385 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:46:25.385 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:46:25.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:25.448 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:46:25.493 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:25 smithi204 ceph-mon[47373]: mgrmap e19: x(active, since 6m), standbys: y, z 2024-05-08T23:46:25.493 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:25 smithi160 ceph-mon[46152]: mgrmap e19: x(active, since 6m), standbys: y, z 2024-05-08T23:46:25.536 INFO:teuthology.orchestra.run.smithi012.stdout:client15273 2024-05-08T23:46:25.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:25.537 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:46:25.580 INFO:teuthology.orchestra.run.smithi012.stdout:client15273 2024-05-08T23:46:25.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:46:25.580 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968.client15273/status') 2024-05-08T23:46:25.641 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.276+0000 7f7d00932200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:46:25.677 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.15273 (3)192.168.0.1:0/492196832 2024-05-08T23:46:25.677 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2024-05-08T23:46:25.677 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2024-05-08T23:46:25.677 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2024-05-08T23:46:25.677 INFO:teuthology.orchestra.run.smithi012.stderr:69 bytes copied, 2.9488e-05 s, 2.3 MB/s 2024-05-08T23:46:25.679 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T23:46:25.688 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:46:25.689 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-08T23:46:25.689 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.875+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/2781787988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c8101980 msgr2=0x7ff6c8103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.875+0000 7ff6ce8d1700 1 --2- 172.21.15.12:0/2781787988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c8101980 0x7ff6c8103d70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009990 tx=0x7ff6b8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.875+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/2781787988 shutdown_connections 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.875+0000 7ff6ce8d1700 1 --2- 172.21.15.12:0/2781787988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6c8106be0 0x7ff6c8109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.875+0000 7ff6ce8d1700 1 --2- 172.21.15.12:0/2781787988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c8101980 0x7ff6c8103d70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.875+0000 7ff6ce8d1700 1 --2- 172.21.15.12:0/2781787988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6c81042b0 0x7ff6c81066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.875+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/2781787988 >> 172.21.15.12:0/2781787988 conn(0x7ff6c80fb390 msgr2=0x7ff6c80fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.875+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/2781787988 shutdown_connections 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.876+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/2781787988 wait complete. 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.876+0000 7ff6ce8d1700 1 Processor -- start 2024-05-08T23:46:25.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.876+0000 7ff6ce8d1700 1 -- start start 2024-05-08T23:46:25.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6ce8d1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6c8101980 0x7ff6c81012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:25.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6ce8d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6c81042b0 0x7ff6c80ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:25.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6ce8d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c8106be0 0x7ff6c80ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6ce8d1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8100610 con 0x7ff6c8106be0 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6ce8d1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8100780 con 0x7ff6c81042b0 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6ce8d1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c81a0e70 con 0x7ff6c8101980 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6cce6e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c8106be0 0x7ff6c80ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6cce6e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c8106be0 0x7ff6c80ffe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55850/0 (socket says 172.21.15.12:55850) 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6c8101980 0x7ff6c81012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6cce6e700 1 -- 172.21.15.12:0/1373003732 learned_addr learned my addr 172.21.15.12:0/1373003732 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6c77fe700 1 --2- 172.21.15.12:0/1373003732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6c81042b0 0x7ff6c80ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6cce6e700 1 -- 172.21.15.12:0/1373003732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6c8101980 msgr2=0x7ff6c81012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6cce6e700 1 --2- 172.21.15.12:0/1373003732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6c8101980 0x7ff6c81012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6cce6e700 1 -- 172.21.15.12:0/1373003732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6c81042b0 msgr2=0x7ff6c80ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6cce6e700 1 --2- 172.21.15.12:0/1373003732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6c81042b0 0x7ff6c80ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:25.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6cce6e700 1 -- 172.21.15.12:0/1373003732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b80095f0 con 0x7ff6c8106be0 2024-05-08T23:46:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6c7fff700 1 --2- 172.21.15.12:0/1373003732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6c8101980 0x7ff6c81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6c77fe700 1 --2- 172.21.15.12:0/1373003732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6c81042b0 0x7ff6c80ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.877+0000 7ff6cce6e700 1 --2- 172.21.15.12:0/1373003732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c8106be0 0x7ff6c80ffe70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000ef30 tx=0x7ff6c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.878+0000 7ff6c57fa700 1 -- 172.21.15.12:0/1373003732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0018070 con 0x7ff6c8106be0 2024-05-08T23:46:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.878+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/1373003732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c81a10a0 con 0x7ff6c8106be0 2024-05-08T23:46:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.878+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/1373003732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c81a15b0 con 0x7ff6c8106be0 2024-05-08T23:46:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.878+0000 7ff6c57fa700 1 -- 172.21.15.12:0/1373003732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c000f040 con 0x7ff6c8106be0 2024-05-08T23:46:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.878+0000 7ff6c57fa700 1 -- 172.21.15.12:0/1373003732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000cca0 con 0x7ff6c8106be0 2024-05-08T23:46:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.880+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/1373003732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b4005320 con 0x7ff6c8106be0 2024-05-08T23:46:25.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.883+0000 7ff6c57fa700 1 -- 172.21.15.12:0/1373003732 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 20) v1 ==== 135558+0+0 (secure 0 0 0) 0x7ff6c0004ad0 con 0x7ff6c8106be0 2024-05-08T23:46:25.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.884+0000 7ff6c57fa700 1 -- 172.21.15.12:0/1373003732 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff6c00cc1e0 con 0x7ff6c8106be0 2024-05-08T23:46:25.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:25.888+0000 7ff6c57fa700 1 -- 172.21.15.12:0/1373003732 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff6c0095d20 con 0x7ff6c8106be0 2024-05-08T23:46:25.915 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.645+0000 7f7c5ebbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:46:25.915 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.645+0000 7f7c5ebbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:46:25.915 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.645+0000 7f7c5ebbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:46:25.915 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.645+0000 7f7c5ebbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:46:25.915 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.645+0000 7f7c5ebbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:46:25.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.645+0000 7f7c61c04700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:46:25.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.645+0000 7f7c61c04700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:46:25.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.645+0000 7f7c61c04700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:46:25.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.645+0000 7f7c61c04700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:46:25.916 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:46:25 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:46:25.645+0000 7f7c61c04700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:46:26.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.168+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/1373003732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7ff6b4005190 con 0x7ff6c8106be0 2024-05-08T23:46:26.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.299+0000 7ff6c57fa700 1 -- 172.21.15.12:0/1373003732 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v24) v1 ==== 117+0+0 (secure 0 0 0) 0x7ff6c0099340 con 0x7ff6c8106be0 2024-05-08T23:46:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.302+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/1373003732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c8106be0 msgr2=0x7ff6c80ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.302+0000 7ff6ce8d1700 1 --2- 172.21.15.12:0/1373003732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c8106be0 0x7ff6c80ffe70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000ef30 tx=0x7ff6c000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.302+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/1373003732 shutdown_connections 2024-05-08T23:46:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.302+0000 7ff6ce8d1700 1 --2- 172.21.15.12:0/1373003732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c8106be0 0x7ff6c80ffe70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.302+0000 7ff6ce8d1700 1 --2- 172.21.15.12:0/1373003732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6c8101980 0x7ff6c81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.302+0000 7ff6ce8d1700 1 --2- 172.21.15.12:0/1373003732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6c81042b0 0x7ff6c80ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.302+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/1373003732 >> 172.21.15.12:0/1373003732 conn(0x7ff6c80fb390 msgr2=0x7ff6c8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.302+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/1373003732 shutdown_connections 2024-05-08T23:46:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.302+0000 7ff6ce8d1700 1 -- 172.21.15.12:0/1373003732 wait complete. 2024-05-08T23:46:26.326 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T23:46:26.340 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-08T23:46:26.357 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:46:26.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.505+0000 7ffb489f0700 1 -- 172.21.15.12:0/234440606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb44102980 msgr2=0x7ffb441092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.505+0000 7ffb489f0700 1 --2- 172.21.15.12:0/234440606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb44102980 0x7ffb441092a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b0a0 tx=0x7ffb3c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.505+0000 7ffb489f0700 1 -- 172.21.15.12:0/234440606 shutdown_connections 2024-05-08T23:46:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.505+0000 7ffb489f0700 1 --2- 172.21.15.12:0/234440606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb44102980 0x7ffb441092a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.505+0000 7ffb489f0700 1 --2- 172.21.15.12:0/234440606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb44100e70 0x7ffb44101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.505+0000 7ffb489f0700 1 --2- 172.21.15.12:0/234440606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb44101fc0 0x7ffb44102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.505+0000 7ffb489f0700 1 -- 172.21.15.12:0/234440606 >> 172.21.15.12:0/234440606 conn(0x7ffb440fc400 msgr2=0x7ffb440fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.505+0000 7ffb489f0700 1 -- 172.21.15.12:0/234440606 shutdown_connections 2024-05-08T23:46:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.505+0000 7ffb489f0700 1 -- 172.21.15.12:0/234440606 wait complete. 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb489f0700 1 Processor -- start 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb489f0700 1 -- start start 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb489f0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb44100e70 0x7ffb44196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb489f0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb44101fc0 0x7ffb44197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb489f0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb44102980 0x7ffb4419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb489f0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4419ea20 con 0x7ffb44102980 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb489f0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4419eb60 con 0x7ffb44100e70 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb489f0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4419ecf0 con 0x7ffb44101fc0 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb42d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb44102980 0x7ffb4419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb42d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb44102980 0x7ffb4419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55872/0 (socket says 172.21.15.12:55872) 2024-05-08T23:46:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb42d9d700 1 -- 172.21.15.12:0/3863862968 learned_addr learned my addr 172.21.15.12:0/3863862968 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb4259c700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb44100e70 0x7ffb44196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb41d9b700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb44101fc0 0x7ffb44197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb42d9d700 1 -- 172.21.15.12:0/3863862968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb44101fc0 msgr2=0x7ffb44197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb42d9d700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb44101fc0 0x7ffb44197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb42d9d700 1 -- 172.21.15.12:0/3863862968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb44100e70 msgr2=0x7ffb44196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb42d9d700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb44100e70 0x7ffb44196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.507+0000 7ffb42d9d700 1 -- 172.21.15.12:0/3863862968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb34009520 con 0x7ffb44102980 2024-05-08T23:46:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.508+0000 7ffb42d9d700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb44102980 0x7ffb4419e450 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b780 tx=0x7ffb3c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.508+0000 7ffb3b7fe700 1 -- 172.21.15.12:0/3863862968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00e040 con 0x7ffb44102980 2024-05-08T23:46:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.508+0000 7ffb3b7fe700 1 -- 172.21.15.12:0/3863862968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb3c009480 con 0x7ffb44102980 2024-05-08T23:46:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.508+0000 7ffb489f0700 1 -- 172.21.15.12:0/3863862968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c009d00 con 0x7ffb44102980 2024-05-08T23:46:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.508+0000 7ffb3b7fe700 1 -- 172.21.15.12:0/3863862968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c0052a0 con 0x7ffb44102980 2024-05-08T23:46:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.508+0000 7ffb489f0700 1 -- 172.21.15.12:0/3863862968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4419f1b0 con 0x7ffb44102980 2024-05-08T23:46:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.509+0000 7ffb489f0700 1 -- 172.21.15.12:0/3863862968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb440673f0 con 0x7ffb44102980 2024-05-08T23:46:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.510+0000 7ffb3b7fe700 1 -- 172.21.15.12:0/3863862968 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7ffb3c0054c0 con 0x7ffb44102980 2024-05-08T23:46:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.511+0000 7ffb3b7fe700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffb300a0f20 0x7ffb300a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.511+0000 7ffb4259c700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffb300a0f20 0x7ffb300a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.511+0000 7ffb3b7fe700 1 -- 172.21.15.12:0/3863862968 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ffb3c0cd4f0 con 0x7ffb44102980 2024-05-08T23:46:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.512+0000 7ffb4259c700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffb300a0f20 0x7ffb300a33e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009c60 tx=0x7ffb2c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:26.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.515+0000 7ffb3b7fe700 1 -- 172.21.15.12:0/3863862968 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffb3c096ee0 con 0x7ffb44102980 2024-05-08T23:46:26.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: Active manager daemon x restarted 2024-05-08T23:46:26.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: Activating manager daemon x 2024-05-08T23:46:26.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: mgrmap e20: x(active, starting, since 0.0122661s), standbys: y, z 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:46:26.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:46:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: Manager daemon x is now available 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:46:26.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:26 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1373003732' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T23:46:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: Active manager daemon x restarted 2024-05-08T23:46:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: Activating manager daemon x 2024-05-08T23:46:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T23:46:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: mgrmap e20: x(active, starting, since 0.0122661s), standbys: y, z 2024-05-08T23:46:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:46:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:46:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:46:26.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:46:26.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:46:26.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:46:26.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:46:26.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:46:26.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:46:26.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: Manager daemon x is now available 2024-05-08T23:46:26.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:46:26.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:46:26.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:46:26.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:26 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1373003732' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: Active manager daemon x restarted 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: Activating manager daemon x 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: mgrmap e20: x(active, starting, since 0.0122661s), standbys: y, z 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:46:26.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:46:26.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: Manager daemon x is now available 2024-05-08T23:46:26.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:46:26.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:46:26.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:46:26.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:26 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1373003732' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T23:46:26.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.745+0000 7ffb489f0700 1 -- 172.21.15.12:0/3863862968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb4410c6c0 con 0x7ffb44102980 2024-05-08T23:46:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.747+0000 7ffb3b7fe700 1 -- 172.21.15.12:0/3863862968 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22884 (secure 0 0 0) 0x7ffb3c09a500 con 0x7ffb44102980 2024-05-08T23:46:26.743 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:26.743 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":84,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:46:25.291136+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3622028746":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3062429456":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2381834553":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/716473668":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2695290437":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/1305168755":"2024-05-09T23:46:25.291064+0000","172.21.15.12:6801/1305168755":"2024-05-09T23:46:25.291064+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 -- 172.21.15.12:0/3863862968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffb300a0f20 msgr2=0x7ffb300a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffb300a0f20 0x7ffb300a33e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009c60 tx=0x7ffb2c009260 comp rx=0 tx=0).stop 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 -- 172.21.15.12:0/3863862968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb44102980 msgr2=0x7ffb4419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb44102980 0x7ffb4419e450 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b780 tx=0x7ffb3c00bc80 comp rx=0 tx=0).stop 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 -- 172.21.15.12:0/3863862968 shutdown_connections 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffb300a0f20 0x7ffb300a33e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb44102980 0x7ffb4419e450 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb44100e70 0x7ffb44196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 --2- 172.21.15.12:0/3863862968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb44101fc0 0x7ffb44197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 -- 172.21.15.12:0/3863862968 >> 172.21.15.12:0/3863862968 conn(0x7ffb440fc400 msgr2=0x7ffb44105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 -- 172.21.15.12:0/3863862968 shutdown_connections 2024-05-08T23:46:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.750+0000 7ffb489f0700 1 -- 172.21.15.12:0/3863862968 wait complete. 2024-05-08T23:46:26.761 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:45:02.165326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:45:03.407498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:46:26.762 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:46:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.933+0000 7f464b14d700 1 -- 172.21.15.12:0/2178501131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644106ce0 msgr2=0x7f46441090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.933+0000 7f464b14d700 1 --2- 172.21.15.12:0/2178501131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644106ce0 0x7f46441090d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f463c00b0a0 tx=0x7f463c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.933+0000 7f464b14d700 1 -- 172.21.15.12:0/2178501131 shutdown_connections 2024-05-08T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.933+0000 7f464b14d700 1 --2- 172.21.15.12:0/2178501131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644106ce0 0x7f46441090d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.933+0000 7f464b14d700 1 --2- 172.21.15.12:0/2178501131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4644101a80 0x7f4644103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.933+0000 7f464b14d700 1 --2- 172.21.15.12:0/2178501131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46441043b0 0x7f46441067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.933+0000 7f464b14d700 1 -- 172.21.15.12:0/2178501131 >> 172.21.15.12:0/2178501131 conn(0x7f46440fb3f0 msgr2=0x7f46440fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.933+0000 7f464b14d700 1 -- 172.21.15.12:0/2178501131 shutdown_connections 2024-05-08T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.933+0000 7f464b14d700 1 -- 172.21.15.12:0/2178501131 wait complete. 2024-05-08T23:46:26.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.934+0000 7f464b14d700 1 Processor -- start 2024-05-08T23:46:26.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.934+0000 7f464b14d700 1 -- start start 2024-05-08T23:46:26.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.934+0000 7f464b14d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644101a80 0x7f4644101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:26.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.934+0000 7f464b14d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46441043b0 0x7f46440ffa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:26.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.934+0000 7f464b14d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4644106ce0 0x7f46440fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:26.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.934+0000 7f464b14d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4644100670 con 0x7f4644101a80 2024-05-08T23:46:26.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.934+0000 7f464b14d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46441007e0 con 0x7f46441043b0 2024-05-08T23:46:26.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.934+0000 7f464b14d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46441a1060 con 0x7f4644106ce0 2024-05-08T23:46:26.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.934+0000 7f4648ee9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644101a80 0x7f4644101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4648ee9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644101a80 0x7f4644101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55878/0 (socket says 172.21.15.12:55878) 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4648ee9700 1 -- 172.21.15.12:0/983262672 learned_addr learned my addr 172.21.15.12:0/983262672 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f46496ea700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4644106ce0 0x7f46440fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4643fff700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46441043b0 0x7f46440ffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4648ee9700 1 -- 172.21.15.12:0/983262672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4644106ce0 msgr2=0x7f46440fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4648ee9700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4644106ce0 0x7f46440fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4648ee9700 1 -- 172.21.15.12:0/983262672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46441043b0 msgr2=0x7f46440ffa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4648ee9700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46441043b0 0x7f46440ffa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4648ee9700 1 -- 172.21.15.12:0/983262672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c009d00 con 0x7f4644101a80 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4643fff700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46441043b0 0x7f46440ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4648ee9700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644101a80 0x7f4644101350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f463800d950 tx=0x7f463800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4641ffb700 1 -- 172.21.15.12:0/983262672 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638009940 con 0x7f4644101a80 2024-05-08T23:46:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4641ffb700 1 -- 172.21.15.12:0/983262672 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4638010460 con 0x7f4644101a80 2024-05-08T23:46:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f464b14d700 1 -- 172.21.15.12:0/983262672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46441a12c0 con 0x7f4644101a80 2024-05-08T23:46:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.935+0000 7f4641ffb700 1 -- 172.21.15.12:0/983262672 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463800f620 con 0x7f4644101a80 2024-05-08T23:46:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.936+0000 7f464b14d700 1 -- 172.21.15.12:0/983262672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46441a1820 con 0x7f4644101a80 2024-05-08T23:46:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.937+0000 7f464b14d700 1 -- 172.21.15.12:0/983262672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46440fcf70 con 0x7f4644101a80 2024-05-08T23:46:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.938+0000 7f4641ffb700 1 -- 172.21.15.12:0/983262672 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f46380108d0 con 0x7f4644101a80 2024-05-08T23:46:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.938+0000 7f4641ffb700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46340a0fa0 0x7f46340a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.938+0000 7f4641ffb700 1 -- 172.21.15.12:0/983262672 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f46380cc3b0 con 0x7f4644101a80 2024-05-08T23:46:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.938+0000 7f4643fff700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46340a0fa0 0x7f46340a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.939+0000 7f4643fff700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46340a0fa0 0x7f46340a3460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4630005fd0 tx=0x7f4630005ee0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:26.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:26.943+0000 7f4641ffb700 1 -- 172.21.15.12:0/983262672 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4638095d50 con 0x7f4644101a80 2024-05-08T23:46:27.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.178+0000 7f464b14d700 1 -- 172.21.15.12:0/983262672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f464410c9c0 con 0x7f4644101a80 2024-05-08T23:46:27.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.179+0000 7f4641ffb700 1 -- 172.21.15.12:0/983262672 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46380160a0 con 0x7f4644101a80 2024-05-08T23:46:27.174 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-08T23:46:27.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 -- 172.21.15.12:0/983262672 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46340a0fa0 msgr2=0x7f46340a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46340a0fa0 0x7f46340a3460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4630005fd0 tx=0x7f4630005ee0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 -- 172.21.15.12:0/983262672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644101a80 msgr2=0x7f4644101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644101a80 0x7f4644101350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f463800d950 tx=0x7f463800dc60 comp rx=0 tx=0).stop 2024-05-08T23:46:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 -- 172.21.15.12:0/983262672 shutdown_connections 2024-05-08T23:46:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46340a0fa0 0x7f46340a3460 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4644106ce0 0x7f46440fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644101a80 0x7f4644101350 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 --2- 172.21.15.12:0/983262672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46441043b0 0x7f46440ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 -- 172.21.15.12:0/983262672 >> 172.21.15.12:0/983262672 conn(0x7f46440fb3f0 msgr2=0x7f46441023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.181+0000 7f464b14d700 1 -- 172.21.15.12:0/983262672 shutdown_connections 2024-05-08T23:46:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.182+0000 7f464b14d700 1 -- 172.21.15.12:0/983262672 wait complete. 2024-05-08T23:46:27.191 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:46:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.347+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/246521863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101970 msgr2=0x7fdf58108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.347+0000 7fdf5ecb2700 1 --2- 172.21.15.12:0/246521863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101970 0x7fdf58108290 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000b0a0 tx=0x7fdf5000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.347+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/246521863 shutdown_connections 2024-05-08T23:46:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 --2- 172.21.15.12:0/246521863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101970 0x7fdf58108290 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 --2- 172.21.15.12:0/246521863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf580ffe60 0x7fdf58100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 --2- 172.21.15.12:0/246521863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf58100fb0 0x7fdf58101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/246521863 >> 172.21.15.12:0/246521863 conn(0x7fdf580fb3f0 msgr2=0x7fdf580fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/246521863 shutdown_connections 2024-05-08T23:46:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/246521863 wait complete. 2024-05-08T23:46:27.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 Processor -- start 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 -- start start 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf580ffe60 0x7fdf58197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58100fb0 0x7fdf58197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101970 0x7fdf5819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5819eb50 con 0x7fdf58100fb0 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5819ec90 con 0x7fdf58101970 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.348+0000 7fdf5ecb2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5819ee20 con 0x7fdf580ffe60 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf57fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58100fb0 0x7fdf58197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf5ca4e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf580ffe60 0x7fdf58197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf5d24f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101970 0x7fdf5819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf57fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58100fb0 0x7fdf58197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55910/0 (socket says 172.21.15.12:55910) 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf57fff700 1 -- 172.21.15.12:0/2548210346 learned_addr learned my addr 172.21.15.12:0/2548210346 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf57fff700 1 -- 172.21.15.12:0/2548210346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf580ffe60 msgr2=0x7fdf58197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf57fff700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf580ffe60 0x7fdf58197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf57fff700 1 -- 172.21.15.12:0/2548210346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101970 msgr2=0x7fdf5819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf57fff700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101970 0x7fdf5819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf57fff700 1 -- 172.21.15.12:0/2548210346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c009520 con 0x7fdf58100fb0 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf5d24f700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101970 0x7fdf5819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf5ca4e700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf580ffe60 0x7fdf58197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf57fff700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58100fb0 0x7fdf58197560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400e9a0 tx=0x7fdf4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf55ffb700 1 -- 172.21.15.12:0/2548210346 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44009790 con 0x7fdf58100fb0 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf55ffb700 1 -- 172.21.15.12:0/2548210346 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf44010430 con 0x7fdf58100fb0 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.349+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/2548210346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf50009d00 con 0x7fdf58100fb0 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.350+0000 7fdf55ffb700 1 -- 172.21.15.12:0/2548210346 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44018680 con 0x7fdf58100fb0 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.350+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/2548210346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5819f480 con 0x7fdf58100fb0 2024-05-08T23:46:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.351+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/2548210346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf58107220 con 0x7fdf58100fb0 2024-05-08T23:46:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.353+0000 7fdf55ffb700 1 -- 172.21.15.12:0/2548210346 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7fdf440105a0 con 0x7fdf58100fb0 2024-05-08T23:46:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.354+0000 7fdf55ffb700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf480a0f20 0x7fdf480a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:27.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.354+0000 7fdf5ca4e700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf480a0f20 0x7fdf480a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:27.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.354+0000 7fdf55ffb700 1 -- 172.21.15.12:0/2548210346 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fdf44014070 con 0x7fdf58100fb0 2024-05-08T23:46:27.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.354+0000 7fdf5ca4e700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf480a0f20 0x7fdf480a33e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009fd0 tx=0x7fdf4c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.359+0000 7fdf55ffb700 1 -- 172.21.15.12:0/2548210346 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdf44052d60 con 0x7fdf58100fb0 2024-05-08T23:46:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.591+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/2548210346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf5810ca20 con 0x7fdf58100fb0 2024-05-08T23:46:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.592+0000 7fdf55ffb700 1 -- 172.21.15.12:0/2548210346 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf44096120 con 0x7fdf58100fb0 2024-05-08T23:46:27.588 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-08T23:46:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/2548210346 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf480a0f20 msgr2=0x7fdf480a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf480a0f20 0x7fdf480a33e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009fd0 tx=0x7fdf4c009260 comp rx=0 tx=0).stop 2024-05-08T23:46:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/2548210346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58100fb0 msgr2=0x7fdf58197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58100fb0 0x7fdf58197560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400e9a0 tx=0x7fdf4400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/2548210346 shutdown_connections 2024-05-08T23:46:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf480a0f20 0x7fdf480a33e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101970 0x7fdf5819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf580ffe60 0x7fdf58197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 --2- 172.21.15.12:0/2548210346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58100fb0 0x7fdf58197560 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/2548210346 >> 172.21.15.12:0/2548210346 conn(0x7fdf580fb3f0 msgr2=0x7fdf58104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/2548210346 shutdown_connections 2024-05-08T23:46:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.594+0000 7fdf5ecb2700 1 -- 172.21.15.12:0/2548210346 wait complete. 2024-05-08T23:46:27.606 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: [08/May/2024:23:46:25] ENGINE Bus STARTING 2024-05-08T23:46:27.606 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: [08/May/2024:23:46:25] ENGINE Serving on https://172.21.15.12:7150 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: [08/May/2024:23:46:26] ENGINE Serving on http://172.21.15.12:8765 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: [08/May/2024:23:46:26] ENGINE Bus STARTED 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1373003732' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: mgrmap e21: x(active, since 1.02539s), standbys: y, z 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3863862968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/983262672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:27 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.608 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:46:27.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.762+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2578653345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6f01020f0 msgr2=0x7fe6f0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.762+0000 7fe6f7af2700 1 --2- 172.21.15.12:0/2578653345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6f01020f0 0x7fe6f0102570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009990 tx=0x7fe6e0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.762+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2578653345 shutdown_connections 2024-05-08T23:46:27.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.762+0000 7fe6f7af2700 1 --2- 172.21.15.12:0/2578653345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6f0102ab0 0x7fe6f01093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.762+0000 7fe6f7af2700 1 --2- 172.21.15.12:0/2578653345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f0100f20 0x7fe6f0101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.762+0000 7fe6f7af2700 1 --2- 172.21.15.12:0/2578653345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6f01020f0 0x7fe6f0102570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.762+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2578653345 >> 172.21.15.12:0/2578653345 conn(0x7fe6f00fc500 msgr2=0x7fe6f00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:27.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.762+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2578653345 shutdown_connections 2024-05-08T23:46:27.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.762+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2578653345 wait complete. 2024-05-08T23:46:27.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.763+0000 7fe6f7af2700 1 Processor -- start 2024-05-08T23:46:27.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.763+0000 7fe6f7af2700 1 -- start start 2024-05-08T23:46:27.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.763+0000 7fe6f7af2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6f0100f20 0x7fe6f0196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:27.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.763+0000 7fe6f7af2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f01020f0 0x7fe6f0197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.763+0000 7fe6f7af2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6f0102ab0 0x7fe6f019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.763+0000 7fe6f7af2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f019ea50 con 0x7fe6f01020f0 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.763+0000 7fe6f7af2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f019eb90 con 0x7fe6f0100f20 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f7af2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f019ed20 con 0x7fe6f0102ab0 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f508d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f01020f0 0x7fe6f0197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f508d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f01020f0 0x7fe6f0197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55958/0 (socket says 172.21.15.12:55958) 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f508d700 1 -- 172.21.15.12:0/2671352280 learned_addr learned my addr 172.21.15.12:0/2671352280 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f608f700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6f0102ab0 0x7fe6f019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f588e700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6f0100f20 0x7fe6f0196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f508d700 1 -- 172.21.15.12:0/2671352280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6f0102ab0 msgr2=0x7fe6f019e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f508d700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6f0102ab0 0x7fe6f019e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f508d700 1 -- 172.21.15.12:0/2671352280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6f0100f20 msgr2=0x7fe6f0196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f508d700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6f0100f20 0x7fe6f0196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f508d700 1 -- 172.21.15.12:0/2671352280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec009520 con 0x7fe6f01020f0 2024-05-08T23:46:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f608f700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6f0102ab0 0x7fe6f019e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6f508d700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f01020f0 0x7fe6f0197460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000b620 tx=0x7fe6e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.764+0000 7fe6e6ffd700 1 -- 172.21.15.12:0/2671352280 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e001d070 con 0x7fe6f01020f0 2024-05-08T23:46:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.765+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2671352280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e00095f0 con 0x7fe6f01020f0 2024-05-08T23:46:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.765+0000 7fe6e6ffd700 1 -- 172.21.15.12:0/2671352280 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6e0022470 con 0x7fe6f01020f0 2024-05-08T23:46:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.765+0000 7fe6e6ffd700 1 -- 172.21.15.12:0/2671352280 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0017650 con 0x7fe6f01020f0 2024-05-08T23:46:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.765+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2671352280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f019f1e0 con 0x7fe6f01020f0 2024-05-08T23:46:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.766+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2671352280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f0108410 con 0x7fe6f01020f0 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: [08/May/2024:23:46:25] ENGINE Bus STARTING 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: [08/May/2024:23:46:25] ENGINE Serving on https://172.21.15.12:7150 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: [08/May/2024:23:46:26] ENGINE Serving on http://172.21.15.12:8765 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: [08/May/2024:23:46:26] ENGINE Bus STARTED 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1373003732' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: mgrmap e21: x(active, since 1.02539s), standbys: y, z 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3863862968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/983262672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:27 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.767+0000 7fe6e6ffd700 1 -- 172.21.15.12:0/2671352280 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7fe6e00225e0 con 0x7fe6f01020f0 2024-05-08T23:46:27.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.768+0000 7fe6e6ffd700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6dc0a0f20 0x7fe6dc0a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:27.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.768+0000 7fe6f588e700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6dc0a0f20 0x7fe6dc0a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:27.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.768+0000 7fe6e6ffd700 1 -- 172.21.15.12:0/2671352280 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe6e00cdd60 con 0x7fe6f01020f0 2024-05-08T23:46:27.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.768+0000 7fe6f588e700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6dc0a0f20 0x7fe6dc0a33e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0100c40 tx=0x7fe6ec009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:27.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: [08/May/2024:23:46:25] ENGINE Bus STARTING 2024-05-08T23:46:27.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: [08/May/2024:23:46:25] ENGINE Serving on https://172.21.15.12:7150 2024-05-08T23:46:27.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: [08/May/2024:23:46:26] ENGINE Serving on http://172.21.15.12:8765 2024-05-08T23:46:27.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: [08/May/2024:23:46:26] ENGINE Bus STARTED 2024-05-08T23:46:27.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1373003732' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T23:46:27.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:46:27.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: mgrmap e21: x(active, since 1.02539s), standbys: y, z 2024-05-08T23:46:27.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3863862968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/983262672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:27 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:27.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.772+0000 7fe6e6ffd700 1 -- 172.21.15.12:0/2671352280 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe6e0097790 con 0x7fe6f01020f0 2024-05-08T23:46:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.998+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2671352280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6f010c7e0 con 0x7fe6f01020f0 2024-05-08T23:46:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:27.999+0000 7fe6e6ffd700 1 -- 172.21.15.12:0/2671352280 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6e009adb0 con 0x7fe6f01020f0 2024-05-08T23:46:27.995 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-08T23:46:27.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2671352280 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6dc0a0f20 msgr2=0x7fe6dc0a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6dc0a0f20 0x7fe6dc0a33e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0100c40 tx=0x7fe6ec009260 comp rx=0 tx=0).stop 2024-05-08T23:46:27.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2671352280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f01020f0 msgr2=0x7fe6f0197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:27.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f01020f0 0x7fe6f0197460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000b620 tx=0x7fe6e0004730 comp rx=0 tx=0).stop 2024-05-08T23:46:27.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2671352280 shutdown_connections 2024-05-08T23:46:27.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6dc0a0f20 0x7fe6dc0a33e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6f0102ab0 0x7fe6f019e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6f0100f20 0x7fe6f0196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 --2- 172.21.15.12:0/2671352280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f01020f0 0x7fe6f0197460 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:27.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2671352280 >> 172.21.15.12:0/2671352280 conn(0x7fe6f00fc500 msgr2=0x7fe6f0105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:27.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2671352280 shutdown_connections 2024-05-08T23:46:27.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.001+0000 7fe6f7af2700 1 -- 172.21.15.12:0/2671352280 wait complete. 2024-05-08T23:46:28.015 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:46:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.166+0000 7f1fb2220700 1 -- 172.21.15.12:0/1051759766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac100a80 msgr2=0x7f1fac100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.166+0000 7f1fb2220700 1 --2- 172.21.15.12:0/1051759766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac100a80 0x7f1fac100f00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400b0d0 tx=0x7f1fa400b3e0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.166+0000 7f1fb2220700 1 -- 172.21.15.12:0/1051759766 shutdown_connections 2024-05-08T23:46:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.166+0000 7f1fb2220700 1 --2- 172.21.15.12:0/1051759766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac100a80 0x7f1fac100f00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.166+0000 7f1fb2220700 1 --2- 172.21.15.12:0/1051759766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fac0ff760 0x7f1fac0ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.166+0000 7f1fb2220700 1 --2- 172.21.15.12:0/1051759766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac1000c0 0x7f1fac100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.166+0000 7f1fb2220700 1 -- 172.21.15.12:0/1051759766 >> 172.21.15.12:0/1051759766 conn(0x7f1fac0fb300 msgr2=0x7f1fac0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.167+0000 7f1fb2220700 1 -- 172.21.15.12:0/1051759766 shutdown_connections 2024-05-08T23:46:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.167+0000 7f1fb2220700 1 -- 172.21.15.12:0/1051759766 wait complete. 2024-05-08T23:46:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.167+0000 7f1fb2220700 1 Processor -- start 2024-05-08T23:46:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fb2220700 1 -- start start 2024-05-08T23:46:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fb2220700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fac0ff760 0x7f1fac1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fb2220700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac1000c0 0x7f1fac1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fb2220700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac100a80 0x7f1fac1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fb2220700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac1a0be0 con 0x7f1fac100a80 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fb2220700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac1a0d20 con 0x7f1fac0ff760 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fb2220700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac1a0eb0 con 0x7f1fac1000c0 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fabfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac100a80 0x7f1fac1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fabfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac100a80 0x7f1fac1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55974/0 (socket says 172.21.15.12:55974) 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fabfff700 1 -- 172.21.15.12:0/3325473373 learned_addr learned my addr 172.21.15.12:0/3325473373 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1faaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac1000c0 0x7f1fac1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.168+0000 7f1fab7fe700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fac0ff760 0x7f1fac1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1faaffd700 1 -- 172.21.15.12:0/3325473373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fac0ff760 msgr2=0x7f1fac1990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1faaffd700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fac0ff760 0x7f1fac1990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1faaffd700 1 -- 172.21.15.12:0/3325473373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac100a80 msgr2=0x7f1fac1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1faaffd700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac100a80 0x7f1fac1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1faaffd700 1 -- 172.21.15.12:0/3325473373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4009d00 con 0x7f1fac1000c0 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1fabfff700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac100a80 0x7f1fac1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1faaffd700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac1000c0 0x7f1fac1995f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c3d0 tx=0x7f1fa000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1fa8ff9700 1 -- 172.21.15.12:0/3325473373 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000ce10 con 0x7f1fac1000c0 2024-05-08T23:46:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1fa8ff9700 1 -- 172.21.15.12:0/3325473373 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fa001c9e0 con 0x7f1fac1000c0 2024-05-08T23:46:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1fb2220700 1 -- 172.21.15.12:0/3325473373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac1019d0 con 0x7f1fac1000c0 2024-05-08T23:46:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1fa8ff9700 1 -- 172.21.15.12:0/3325473373 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa001eb30 con 0x7f1fac1000c0 2024-05-08T23:46:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.169+0000 7f1fb2220700 1 -- 172.21.15.12:0/3325473373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac101ee0 con 0x7f1fac1000c0 2024-05-08T23:46:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.171+0000 7f1fb2220700 1 -- 172.21.15.12:0/3325473373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac10a700 con 0x7f1fac1000c0 2024-05-08T23:46:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.172+0000 7f1fa8ff9700 1 -- 172.21.15.12:0/3325473373 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f1fa001cb50 con 0x7f1fac1000c0 2024-05-08T23:46:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.172+0000 7f1fa8ff9700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f940a0f20 0x7f1f940a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.172+0000 7f1fa8ff9700 1 -- 172.21.15.12:0/3325473373 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1fa00d4660 con 0x7f1fac1000c0 2024-05-08T23:46:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.172+0000 7f1fab7fe700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f940a0f20 0x7f1f940a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.173+0000 7f1fab7fe700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f940a0f20 0x7f1f940a33e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00bbd0 tx=0x7f1f9c00b5b0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:28.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.177+0000 7f1fa8ff9700 1 -- 172.21.15.12:0/3325473373 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1fa009dfc0 con 0x7f1fac1000c0 2024-05-08T23:46:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.312+0000 7f1fa8ff9700 1 -- 172.21.15.12:0/3325473373 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 22) v1 ==== 135993+0+0 (secure 0 0 0) 0x7f1fa009e390 con 0x7f1fac1000c0 2024-05-08T23:46:28.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:28.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2548210346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:28.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:46:28.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:28 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2671352280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.420+0000 7f1fb2220700 1 -- 172.21.15.12:0/3325473373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fac10a0e0 con 0x7f1fac1000c0 2024-05-08T23:46:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.421+0000 7f1fa8ff9700 1 -- 172.21.15.12:0/3325473373 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22884 (secure 0 0 0) 0x7f1fa005a5f0 con 0x7f1fac1000c0 2024-05-08T23:46:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:28.417 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":84,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:46:25.291136+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3622028746":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3062429456":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2381834553":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/716473668":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2695290437":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/1305168755":"2024-05-09T23:46:25.291064+0000","172.21.15.12:6801/1305168755":"2024-05-09T23:46:25.291064+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:46:28.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 -- 172.21.15.12:0/3325473373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f940a0f20 msgr2=0x7f1f940a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:28.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f940a0f20 0x7f1f940a33e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00bbd0 tx=0x7f1f9c00b5b0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 -- 172.21.15.12:0/3325473373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac1000c0 msgr2=0x7f1fac1995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:28.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac1000c0 0x7f1fac1995f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c3d0 tx=0x7f1fa000c6e0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 -- 172.21.15.12:0/3325473373 shutdown_connections 2024-05-08T23:46:28.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f940a0f20 0x7f1f940a33e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fac100a80 0x7f1fac1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fac0ff760 0x7f1fac1990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 --2- 172.21.15.12:0/3325473373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac1000c0 0x7f1fac1995f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 -- 172.21.15.12:0/3325473373 >> 172.21.15.12:0/3325473373 conn(0x7f1fac0fb300 msgr2=0x7f1fac0fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:28.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 -- 172.21.15.12:0/3325473373 shutdown_connections 2024-05-08T23:46:28.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.424+0000 7f1fb2220700 1 -- 172.21.15.12:0/3325473373 wait complete. 2024-05-08T23:46:28.435 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:45:02.165326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:45:03.407498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:46:28.436 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:46:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.585+0000 7f9d26bc9700 1 -- 172.21.15.12:0/1278064905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d200ffe60 msgr2=0x7f9d20100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.585+0000 7f9d26bc9700 1 --2- 172.21.15.12:0/1278064905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d200ffe60 0x7f9d20100280 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009990 tx=0x7f9d14009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.585+0000 7f9d26bc9700 1 -- 172.21.15.12:0/1278064905 shutdown_connections 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.585+0000 7f9d26bc9700 1 --2- 172.21.15.12:0/1278064905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d20101970 0x7f9d20108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.585+0000 7f9d26bc9700 1 --2- 172.21.15.12:0/1278064905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d200ffe60 0x7f9d20100280 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.585+0000 7f9d26bc9700 1 --2- 172.21.15.12:0/1278064905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d20100fb0 0x7f9d20101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.585+0000 7f9d26bc9700 1 -- 172.21.15.12:0/1278064905 >> 172.21.15.12:0/1278064905 conn(0x7f9d200fb3f0 msgr2=0x7f9d200fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.585+0000 7f9d26bc9700 1 -- 172.21.15.12:0/1278064905 shutdown_connections 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.585+0000 7f9d26bc9700 1 -- 172.21.15.12:0/1278064905 wait complete. 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.586+0000 7f9d26bc9700 1 Processor -- start 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.586+0000 7f9d26bc9700 1 -- start start 2024-05-08T23:46:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.586+0000 7f9d26bc9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d200ffe60 0x7f9d20195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.586+0000 7f9d26bc9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d20100fb0 0x7f9d20196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.586+0000 7f9d26bc9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d20101970 0x7f9d2019d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.586+0000 7f9d26bc9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019d950 con 0x7f9d20101970 2024-05-08T23:46:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.586+0000 7f9d26bc9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019da90 con 0x7f9d20100fb0 2024-05-08T23:46:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.586+0000 7f9d26bc9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019dc20 con 0x7f9d200ffe60 2024-05-08T23:46:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.586+0000 7f9d25166700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d20101970 0x7f9d2019d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d24965700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d200ffe60 0x7f9d20195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d25166700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d20101970 0x7f9d2019d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48130/0 (socket says 172.21.15.12:48130) 2024-05-08T23:46:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d25166700 1 -- 172.21.15.12:0/84723131 learned_addr learned my addr 172.21.15.12:0/84723131 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d1ffff700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d20100fb0 0x7f9d20196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d24965700 1 -- 172.21.15.12:0/84723131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d20100fb0 msgr2=0x7f9d20196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d24965700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d20100fb0 0x7f9d20196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d24965700 1 -- 172.21.15.12:0/84723131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d20101970 msgr2=0x7f9d2019d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d24965700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d20101970 0x7f9d2019d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d24965700 1 -- 172.21.15.12:0/84723131 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d140095f0 con 0x7f9d200ffe60 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d25166700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d20101970 0x7f9d2019d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d24965700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d200ffe60 0x7f9d20195e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400b620 tx=0x7f9d14004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d1dffb700 1 -- 172.21.15.12:0/84723131 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1401d070 con 0x7f9d200ffe60 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d1dffb700 1 -- 172.21.15.12:0/84723131 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d14021410 con 0x7f9d200ffe60 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d26bc9700 1 -- 172.21.15.12:0/84723131 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2019dea0 con 0x7f9d200ffe60 2024-05-08T23:46:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.587+0000 7f9d1dffb700 1 -- 172.21.15.12:0/84723131 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14017650 con 0x7f9d200ffe60 2024-05-08T23:46:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.588+0000 7f9d26bc9700 1 -- 172.21.15.12:0/84723131 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2019e210 con 0x7f9d200ffe60 2024-05-08T23:46:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.589+0000 7f9d26bc9700 1 -- 172.21.15.12:0/84723131 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d201072d0 con 0x7f9d200ffe60 2024-05-08T23:46:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.590+0000 7f9d1dffb700 1 -- 172.21.15.12:0/84723131 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 22) v1 ==== 135993+0+0 (secure 0 0 0) 0x7f9d1400fb10 con 0x7f9d200ffe60 2024-05-08T23:46:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.590+0000 7f9d1dffb700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d100a1310 0x7f9d100a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.591+0000 7f9d1ffff700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d100a1310 0x7f9d100a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.591+0000 7f9d1dffb700 1 -- 172.21.15.12:0/84723131 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9d140cdf10 con 0x7f9d200ffe60 2024-05-08T23:46:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.591+0000 7f9d1ffff700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d100a1310 0x7f9d100a37d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009c60 tx=0x7f9d0c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.594+0000 7f9d1dffb700 1 -- 172.21.15.12:0/84723131 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d14097430 con 0x7f9d200ffe60 2024-05-08T23:46:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2548210346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:46:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:46:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:28 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2671352280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2548210346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:28.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:46:28.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:46:28.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:46:28.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:28 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2671352280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:28.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.843+0000 7f9d26bc9700 1 -- 172.21.15.12:0/84723131 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d2010b7c0 con 0x7f9d200ffe60 2024-05-08T23:46:28.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.843+0000 7f9d1dffb700 1 -- 172.21.15.12:0/84723131 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d14021e60 con 0x7f9d200ffe60 2024-05-08T23:46:28.839 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-08T23:46:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 -- 172.21.15.12:0/84723131 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d100a1310 msgr2=0x7f9d100a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d100a1310 0x7f9d100a37d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009c60 tx=0x7f9d0c009260 comp rx=0 tx=0).stop 2024-05-08T23:46:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 -- 172.21.15.12:0/84723131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d200ffe60 msgr2=0x7f9d20195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d200ffe60 0x7f9d20195e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400b620 tx=0x7f9d14004730 comp rx=0 tx=0).stop 2024-05-08T23:46:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 -- 172.21.15.12:0/84723131 shutdown_connections 2024-05-08T23:46:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d100a1310 0x7f9d100a37d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d20101970 0x7f9d2019d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d200ffe60 0x7f9d20195e20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 --2- 172.21.15.12:0/84723131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d20100fb0 0x7f9d20196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 -- 172.21.15.12:0/84723131 >> 172.21.15.12:0/84723131 conn(0x7f9d200fb3f0 msgr2=0x7f9d20104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 -- 172.21.15.12:0/84723131 shutdown_connections 2024-05-08T23:46:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:28.846+0000 7f9d26bc9700 1 -- 172.21.15.12:0/84723131 wait complete. 2024-05-08T23:46:28.856 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:46:29.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.013+0000 7f4667d52700 1 -- 172.21.15.12:0/4035988003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4660068590 msgr2=0x7f4660068a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.013+0000 7f4667d52700 1 --2- 172.21.15.12:0/4035988003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4660068590 0x7f4660068a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4650009990 tx=0x7f4650009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.013+0000 7f4667d52700 1 -- 172.21.15.12:0/4035988003 shutdown_connections 2024-05-08T23:46:29.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.013+0000 7f4667d52700 1 --2- 172.21.15.12:0/4035988003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4660103930 0x7f4660103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.013+0000 7f4667d52700 1 --2- 172.21.15.12:0/4035988003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4660105e30 0x7f4660068050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.013+0000 7f4667d52700 1 --2- 172.21.15.12:0/4035988003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4660068590 0x7f4660068a10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.013+0000 7f4667d52700 1 -- 172.21.15.12:0/4035988003 >> 172.21.15.12:0/4035988003 conn(0x7f4660076f40 msgr2=0x7f4660077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:29.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.013+0000 7f4667d52700 1 -- 172.21.15.12:0/4035988003 shutdown_connections 2024-05-08T23:46:29.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.013+0000 7f4667d52700 1 -- 172.21.15.12:0/4035988003 wait complete. 2024-05-08T23:46:29.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4667d52700 1 Processor -- start 2024-05-08T23:46:29.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4667d52700 1 -- start start 2024-05-08T23:46:29.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4667d52700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4660068590 0x7f466019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4667d52700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4660103930 0x7f466019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4667d52700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4660105e30 0x7f46601a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4667d52700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46601a2900 con 0x7f4660105e30 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4667d52700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46601a2a70 con 0x7f4660103930 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4667d52700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46601a2c30 con 0x7f4660068590 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4665aee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4660068590 0x7f466019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f46662ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4660105e30 0x7f46601a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4665aee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4660068590 0x7f466019da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:55826/0 (socket says 172.21.15.12:55826) 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f4665aee700 1 -- 172.21.15.12:0/724262813 learned_addr learned my addr 172.21.15.12:0/724262813 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.014+0000 7f46652ed700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4660103930 0x7f466019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4665aee700 1 -- 172.21.15.12:0/724262813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4660103930 msgr2=0x7f466019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4665aee700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4660103930 0x7f466019df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4665aee700 1 -- 172.21.15.12:0/724262813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4660105e30 msgr2=0x7f46601a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4665aee700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4660105e30 0x7f46601a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4665aee700 1 -- 172.21.15.12:0/724262813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c009520 con 0x7f4660068590 2024-05-08T23:46:29.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f46652ed700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4660103930 0x7f466019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:29.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f46662ef700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4660105e30 0x7f46601a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:29.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4665aee700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4660068590 0x7f466019da20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f465c00eb30 tx=0x7f465c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:29.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4656ffd700 1 -- 172.21.15.12:0/724262813 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c009880 con 0x7f4660068590 2024-05-08T23:46:29.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4656ffd700 1 -- 172.21.15.12:0/724262813 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f465c0044e0 con 0x7f4660068590 2024-05-08T23:46:29.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4656ffd700 1 -- 172.21.15.12:0/724262813 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c003d80 con 0x7f4660068590 2024-05-08T23:46:29.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4667d52700 1 -- 172.21.15.12:0/724262813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46500095f0 con 0x7f4660068590 2024-05-08T23:46:29.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.015+0000 7f4667d52700 1 -- 172.21.15.12:0/724262813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46601aa5e0 con 0x7f4660068590 2024-05-08T23:46:29.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.017+0000 7f4667d52700 1 -- 172.21.15.12:0/724262813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4660100410 con 0x7f4660068590 2024-05-08T23:46:29.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.018+0000 7f4656ffd700 1 -- 172.21.15.12:0/724262813 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 22) v1 ==== 135993+0+0 (secure 0 0 0) 0x7f465c003f20 con 0x7f4660068590 2024-05-08T23:46:29.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.019+0000 7f4656ffd700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f464c0a1310 0x7f464c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.019+0000 7f4656ffd700 1 -- 172.21.15.12:0/724262813 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f465c014070 con 0x7f4660068590 2024-05-08T23:46:29.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.019+0000 7f46652ed700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f464c0a1310 0x7f464c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.020+0000 7f46652ed700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f464c0a1310 0x7f464c0a37d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f466019ef70 tx=0x7f465000b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:29.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.024+0000 7f4656ffd700 1 -- 172.21.15.12:0/724262813 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f465c096580 con 0x7f4660068590 2024-05-08T23:46:29.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.254+0000 7f4667d52700 1 -- 172.21.15.12:0/724262813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4660067cf0 con 0x7f4660068590 2024-05-08T23:46:29.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.255+0000 7f4656ffd700 1 -- 172.21.15.12:0/724262813 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f465c0041d0 con 0x7f4660068590 2024-05-08T23:46:29.251 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-08T23:46:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 -- 172.21.15.12:0/724262813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f464c0a1310 msgr2=0x7f464c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f464c0a1310 0x7f464c0a37d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f466019ef70 tx=0x7f465000b260 comp rx=0 tx=0).stop 2024-05-08T23:46:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 -- 172.21.15.12:0/724262813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4660068590 msgr2=0x7f466019da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4660068590 0x7f466019da20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f465c00eb30 tx=0x7f465c00ee40 comp rx=0 tx=0).stop 2024-05-08T23:46:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 -- 172.21.15.12:0/724262813 shutdown_connections 2024-05-08T23:46:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f464c0a1310 0x7f464c0a37d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4660105e30 0x7f46601a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4660068590 0x7f466019da20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 --2- 172.21.15.12:0/724262813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4660103930 0x7f466019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 -- 172.21.15.12:0/724262813 >> 172.21.15.12:0/724262813 conn(0x7f4660076f40 msgr2=0x7f4660102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:29.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 -- 172.21.15.12:0/724262813 shutdown_connections 2024-05-08T23:46:29.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.258+0000 7f4667d52700 1 -- 172.21.15.12:0/724262813 wait complete. 2024-05-08T23:46:29.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:46:29.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.425+0000 7fcb5fe24700 1 -- 172.21.15.12:0/3297533528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb58106280 msgr2=0x7fcb5810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.425+0000 7fcb5fe24700 1 --2- 172.21.15.12:0/3297533528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb58106280 0x7fcb5810de00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b0a0 tx=0x7fcb5000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.425+0000 7fcb5fe24700 1 -- 172.21.15.12:0/3297533528 shutdown_connections 2024-05-08T23:46:29.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.425+0000 7fcb5fe24700 1 --2- 172.21.15.12:0/3297533528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb58106280 0x7fcb5810de00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.425+0000 7fcb5fe24700 1 --2- 172.21.15.12:0/3297533528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb58069770 0x7fcb58069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.425+0000 7fcb5fe24700 1 --2- 172.21.15.12:0/3297533528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb5806a120 0x7fcb58105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.425+0000 7fcb5fe24700 1 -- 172.21.15.12:0/3297533528 >> 172.21.15.12:0/3297533528 conn(0x7fcb58076fa0 msgr2=0x7fcb580773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:29.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.425+0000 7fcb5fe24700 1 -- 172.21.15.12:0/3297533528 shutdown_connections 2024-05-08T23:46:29.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.425+0000 7fcb5fe24700 1 -- 172.21.15.12:0/3297533528 wait complete. 2024-05-08T23:46:29.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5fe24700 1 Processor -- start 2024-05-08T23:46:29.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5fe24700 1 -- start start 2024-05-08T23:46:29.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5fe24700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58069770 0x7fcb580ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5fe24700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb58106280 0x7fcb58100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5fe24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb58100890 0x7fcb581a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5fe24700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581a5cd0 con 0x7fcb58100890 2024-05-08T23:46:29.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5fe24700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581a5e40 con 0x7fcb58106280 2024-05-08T23:46:29.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5fe24700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581a6000 con 0x7fcb58069770 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5e3c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb58100890 0x7fcb581a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5d3bf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb58106280 0x7fcb58100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5e3c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb58100890 0x7fcb581a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48188/0 (socket says 172.21.15.12:48188) 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5e3c1700 1 -- 172.21.15.12:0/1158105275 learned_addr learned my addr 172.21.15.12:0/1158105275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.426+0000 7fcb5dbc0700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58069770 0x7fcb580ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb5e3c1700 1 -- 172.21.15.12:0/1158105275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58069770 msgr2=0x7fcb580ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb5e3c1700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58069770 0x7fcb580ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb5e3c1700 1 -- 172.21.15.12:0/1158105275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb58106280 msgr2=0x7fcb58100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb5e3c1700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb58106280 0x7fcb58100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb5e3c1700 1 -- 172.21.15.12:0/1158105275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50009d00 con 0x7fcb58100890 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb5e3c1700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb58100890 0x7fcb581a5700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b070 tx=0x7fcb50007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb4effd700 1 -- 172.21.15.12:0/1158105275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000e040 con 0x7fcb58100890 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb5fe24700 1 -- 172.21.15.12:0/1158105275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb581a6280 con 0x7fcb58100890 2024-05-08T23:46:29.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb4effd700 1 -- 172.21.15.12:0/1158105275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb5000bdc0 con 0x7fcb58100890 2024-05-08T23:46:29.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb4effd700 1 -- 172.21.15.12:0/1158105275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500052a0 con 0x7fcb58100890 2024-05-08T23:46:29.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.427+0000 7fcb5fe24700 1 -- 172.21.15.12:0/1158105275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb581a66a0 con 0x7fcb58100890 2024-05-08T23:46:29.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.428+0000 7fcb5fe24700 1 -- 172.21.15.12:0/1158105275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb58109550 con 0x7fcb58100890 2024-05-08T23:46:29.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.429+0000 7fcb4effd700 1 -- 172.21.15.12:0/1158105275 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 22) v1 ==== 135993+0+0 (secure 0 0 0) 0x7fcb5001d480 con 0x7fcb58100890 2024-05-08T23:46:29.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.430+0000 7fcb4effd700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcb440a1290 0x7fcb440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.430+0000 7fcb5dbc0700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcb440a1290 0x7fcb440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.430+0000 7fcb4effd700 1 -- 172.21.15.12:0/1158105275 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcb500cdaf0 con 0x7fcb58100890 2024-05-08T23:46:29.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.431+0000 7fcb5dbc0700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcb440a1290 0x7fcb440a3750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009c60 tx=0x7fcb54009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:29.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.435+0000 7fcb4effd700 1 -- 172.21.15.12:0/1158105275 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcb50097010 con 0x7fcb58100890 2024-05-08T23:46:29.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.657+0000 7fcb5fe24700 1 -- 172.21.15.12:0/1158105275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb581012f0 con 0x7fcb58100890 2024-05-08T23:46:29.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.657+0000 7fcb4effd700 1 -- 172.21.15.12:0/1158105275 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb5009a630 con 0x7fcb58100890 2024-05-08T23:46:29.653 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 -- 172.21.15.12:0/1158105275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcb440a1290 msgr2=0x7fcb440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcb440a1290 0x7fcb440a3750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009c60 tx=0x7fcb54009260 comp rx=0 tx=0).stop 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 -- 172.21.15.12:0/1158105275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb58100890 msgr2=0x7fcb581a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb58100890 0x7fcb581a5700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b070 tx=0x7fcb50007cf0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 -- 172.21.15.12:0/1158105275 shutdown_connections 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcb440a1290 0x7fcb440a3750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb58100890 0x7fcb581a5700 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58069770 0x7fcb580ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 --2- 172.21.15.12:0/1158105275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb58106280 0x7fcb58100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 -- 172.21.15.12:0/1158105275 >> 172.21.15.12:0/1158105275 conn(0x7fcb58076fa0 msgr2=0x7fcb580fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 -- 172.21.15.12:0/1158105275 shutdown_connections 2024-05-08T23:46:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.660+0000 7fcb5fe24700 1 -- 172.21.15.12:0/1158105275 wait complete. 2024-05-08T23:46:29.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi204 to 182.4M 2024-05-08T23:46:29.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Unable to set osd_memory_target on smithi204 to 191345459: error parsing value: Value '191345459' is below minimum 939524096 2024-05-08T23:46:29.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-08T23:46:29.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Unable to set osd_memory_target on smithi160 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:46:29.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi012 to 183.5M 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Unable to set osd_memory_target on smithi012 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Updating smithi160:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Updating smithi012:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: mgrmap e22: x(active, since 3s), standbys: y, z 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3325473373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/84723131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:29 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/724262813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:29.669 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-08T23:46:29.670 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:46:29.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi204 to 182.4M 2024-05-08T23:46:29.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Unable to set osd_memory_target on smithi204 to 191345459: error parsing value: Value '191345459' is below minimum 939524096 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Unable to set osd_memory_target on smithi160 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi012 to 183.5M 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Unable to set osd_memory_target on smithi012 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Updating smithi160:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Updating smithi012:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: mgrmap e22: x(active, since 3s), standbys: y, z 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3325473373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/84723131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:29 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/724262813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi204 to 182.4M 2024-05-08T23:46:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Unable to set osd_memory_target on smithi204 to 191345459: error parsing value: Value '191345459' is below minimum 939524096 2024-05-08T23:46:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-08T23:46:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Unable to set osd_memory_target on smithi160 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:46:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi012 to 183.5M 2024-05-08T23:46:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Unable to set osd_memory_target on smithi012 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-08T23:46:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-08T23:46:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-08T23:46:29.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Updating smithi160:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Updating smithi012:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: mgrmap e22: x(active, since 3s), standbys: y, z 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3325473373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/84723131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:29.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:29 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/724262813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:29.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.811+0000 7f55fd29d700 1 -- 172.21.15.12:0/3717868850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55f81028f0 msgr2=0x7f55f810add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.811+0000 7f55fd29d700 1 --2- 172.21.15.12:0/3717868850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55f81028f0 0x7f55f810add0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009990 tx=0x7f55e0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.811+0000 7f55fd29d700 1 -- 172.21.15.12:0/3717868850 shutdown_connections 2024-05-08T23:46:29.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.811+0000 7f55fd29d700 1 --2- 172.21.15.12:0/3717868850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f810b3f0 0x7f55f810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.811+0000 7f55fd29d700 1 --2- 172.21.15.12:0/3717868850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f8101fd0 0x7f55f81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.811+0000 7f55fd29d700 1 --2- 172.21.15.12:0/3717868850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55f81028f0 0x7f55f810add0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.811+0000 7f55fd29d700 1 -- 172.21.15.12:0/3717868850 >> 172.21.15.12:0/3717868850 conn(0x7f55f80fb810 msgr2=0x7f55f80fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.811+0000 7f55fd29d700 1 -- 172.21.15.12:0/3717868850 shutdown_connections 2024-05-08T23:46:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.811+0000 7f55fd29d700 1 -- 172.21.15.12:0/3717868850 wait complete. 2024-05-08T23:46:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55fd29d700 1 Processor -- start 2024-05-08T23:46:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55fd29d700 1 -- start start 2024-05-08T23:46:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55fd29d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55f8101fd0 0x7f55f81972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55fd29d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f81028f0 0x7f55f8197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55fd29d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f810b3f0 0x7f55f819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55fd29d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f819c200 con 0x7f55f8101fd0 2024-05-08T23:46:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55fd29d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f819c370 con 0x7f55f810b3f0 2024-05-08T23:46:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55fd29d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f819c530 con 0x7f55f81028f0 2024-05-08T23:46:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55f8101fd0 0x7f55f81972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55f8101fd0 0x7f55f81972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48194/0 (socket says 172.21.15.12:48194) 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55f6ffd700 1 -- 172.21.15.12:0/35029108 learned_addr learned my addr 172.21.15.12:0/35029108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55f67fc700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f81028f0 0x7f55f8197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.812+0000 7f55f77fe700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f810b3f0 0x7f55f819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55f6ffd700 1 -- 172.21.15.12:0/35029108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f81028f0 msgr2=0x7f55f8197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55f6ffd700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f81028f0 0x7f55f8197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55f6ffd700 1 -- 172.21.15.12:0/35029108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f810b3f0 msgr2=0x7f55f819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55f6ffd700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f810b3f0 0x7f55f819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55f6ffd700 1 -- 172.21.15.12:0/35029108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00095f0 con 0x7f55f8101fd0 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55f77fe700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f810b3f0 0x7f55f819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55f6ffd700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55f8101fd0 0x7f55f81972e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e9a0 tx=0x7f55e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55effff700 1 -- 172.21.15.12:0/35029108 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8009790 con 0x7f55f8101fd0 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55fd29d700 1 -- 172.21.15.12:0/35029108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f819c810 con 0x7f55f8101fd0 2024-05-08T23:46:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55effff700 1 -- 172.21.15.12:0/35029108 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55e800cbd0 con 0x7f55f8101fd0 2024-05-08T23:46:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55effff700 1 -- 172.21.15.12:0/35029108 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8018680 con 0x7f55f8101fd0 2024-05-08T23:46:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.813+0000 7f55fd29d700 1 -- 172.21.15.12:0/35029108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f81a4050 con 0x7f55f8101fd0 2024-05-08T23:46:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.814+0000 7f55fd29d700 1 -- 172.21.15.12:0/35029108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f81064c0 con 0x7f55f8101fd0 2024-05-08T23:46:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.815+0000 7f55effff700 1 -- 172.21.15.12:0/35029108 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 22) v1 ==== 135993+0+0 (secure 0 0 0) 0x7f55e800cd40 con 0x7f55f8101fd0 2024-05-08T23:46:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.816+0000 7f55effff700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55e40a1290 0x7f55e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.816+0000 7f55effff700 1 -- 172.21.15.12:0/35029108 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f55e8014070 con 0x7f55f8101fd0 2024-05-08T23:46:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.816+0000 7f55f67fc700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55e40a1290 0x7f55e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.817+0000 7f55f67fc700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55e40a1290 0x7f55e40a3750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f55f8198830 tx=0x7f55e0005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:29.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:29.821+0000 7f55effff700 1 -- 172.21.15.12:0/35029108 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f55e8052d60 con 0x7f55f8101fd0 2024-05-08T23:46:30.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.088+0000 7f55fd29d700 1 -- 172.21.15.12:0/35029108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55f8069d10 con 0x7f55f8101fd0 2024-05-08T23:46:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.088+0000 7f55effff700 1 -- 172.21.15.12:0/35029108 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10876 (secure 0 0 0) 0x7f55e8096390 con 0x7f55f8101fd0 2024-05-08T23:46:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:30.084 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:46:26.294680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:46:30.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 -- 172.21.15.12:0/35029108 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55e40a1290 msgr2=0x7f55e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55e40a1290 0x7f55e40a3750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f55f8198830 tx=0x7f55e0005fd0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 -- 172.21.15.12:0/35029108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55f8101fd0 msgr2=0x7f55f81972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55f8101fd0 0x7f55f81972e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e9a0 tx=0x7f55e800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 -- 172.21.15.12:0/35029108 shutdown_connections 2024-05-08T23:46:30.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55e40a1290 0x7f55e40a3750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f810b3f0 0x7f55f819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55f8101fd0 0x7f55f81972e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 --2- 172.21.15.12:0/35029108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f81028f0 0x7f55f8197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 -- 172.21.15.12:0/35029108 >> 172.21.15.12:0/35029108 conn(0x7f55f80fb810 msgr2=0x7f55f80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:30.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 -- 172.21.15.12:0/35029108 shutdown_connections 2024-05-08T23:46:30.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.091+0000 7f55fd29d700 1 -- 172.21.15.12:0/35029108 wait complete. 2024-05-08T23:46:30.088 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 24 2024-05-08T23:46:30.101 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-08T23:46:30.101 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-08T23:46:30.101 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:46:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.248+0000 7fc9f2b46700 1 -- 172.21.15.12:0/4006748665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9ec101a00 msgr2=0x7fc9ec105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.248+0000 7fc9f2b46700 1 --2- 172.21.15.12:0/4006748665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9ec101a00 0x7fc9ec105a40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009990 tx=0x7fc9d8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.248+0000 7fc9f2b46700 1 -- 172.21.15.12:0/4006748665 shutdown_connections 2024-05-08T23:46:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.248+0000 7fc9f2b46700 1 --2- 172.21.15.12:0/4006748665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9ec106170 0x7fc9ec10dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.248+0000 7fc9f2b46700 1 --2- 172.21.15.12:0/4006748665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec101050 0x7fc9ec101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.248+0000 7fc9f2b46700 1 --2- 172.21.15.12:0/4006748665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9ec101a00 0x7fc9ec105a40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.248+0000 7fc9f2b46700 1 -- 172.21.15.12:0/4006748665 >> 172.21.15.12:0/4006748665 conn(0x7fc9ec0fc910 msgr2=0x7fc9ec0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.248+0000 7fc9f2b46700 1 -- 172.21.15.12:0/4006748665 shutdown_connections 2024-05-08T23:46:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.248+0000 7fc9f2b46700 1 -- 172.21.15.12:0/4006748665 wait complete. 2024-05-08T23:46:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.249+0000 7fc9f2b46700 1 Processor -- start 2024-05-08T23:46:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.249+0000 7fc9f2b46700 1 -- start start 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f2b46700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9ec101050 0x7fc9ec1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f2b46700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9ec106170 0x7fc9ec197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f2b46700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec19bb50 0x7fc9ec19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f2b46700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec19c5d0 con 0x7fc9ec101050 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f2b46700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec19c740 con 0x7fc9ec19bb50 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f2b46700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec19c900 con 0x7fc9ec106170 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f08e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9ec101050 0x7fc9ec1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f08e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9ec101050 0x7fc9ec1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48218/0 (socket says 172.21.15.12:48218) 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f08e2700 1 -- 172.21.15.12:0/1990720949 learned_addr learned my addr 172.21.15.12:0/1990720949 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f10e3700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec19bb50 0x7fc9ec19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9ebfff700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9ec106170 0x7fc9ec197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f08e2700 1 -- 172.21.15.12:0/1990720949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9ec106170 msgr2=0x7fc9ec197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f08e2700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9ec106170 0x7fc9ec197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f08e2700 1 -- 172.21.15.12:0/1990720949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec19bb50 msgr2=0x7fc9ec19bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f08e2700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec19bb50 0x7fc9ec19bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f08e2700 1 -- 172.21.15.12:0/1990720949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d80095f0 con 0x7fc9ec101050 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9f08e2700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9ec101050 0x7fc9ec1972c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000d980 tx=0x7fc9e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.250+0000 7fc9e9ffb700 1 -- 172.21.15.12:0/1990720949 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0004220 con 0x7fc9ec101050 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.251+0000 7fc9f2b46700 1 -- 172.21.15.12:0/1990720949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec1a3e60 con 0x7fc9ec101050 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.251+0000 7fc9e9ffb700 1 -- 172.21.15.12:0/1990720949 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e0007640 con 0x7fc9ec101050 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.251+0000 7fc9e9ffb700 1 -- 172.21.15.12:0/1990720949 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0005610 con 0x7fc9ec101050 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.251+0000 7fc9f2b46700 1 -- 172.21.15.12:0/1990720949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec1a42e0 con 0x7fc9ec101050 2024-05-08T23:46:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.252+0000 7fc9f2b46700 1 -- 172.21.15.12:0/1990720949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ec109440 con 0x7fc9ec101050 2024-05-08T23:46:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.253+0000 7fc9e9ffb700 1 -- 172.21.15.12:0/1990720949 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7fc9e0004750 con 0x7fc9ec101050 2024-05-08T23:46:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.253+0000 7fc9e9ffb700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1320 0x7fc9dc0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.253+0000 7fc9e9ffb700 1 -- 172.21.15.12:0/1990720949 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc9e00cccd0 con 0x7fc9ec101050 2024-05-08T23:46:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.253+0000 7fc9ebfff700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1320 0x7fc9dc0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.254+0000 7fc9ebfff700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1320 0x7fc9dc0a37e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8006010 tx=0x7fc9d800b540 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:30.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.258+0000 7fc9e9ffb700 1 -- 172.21.15.12:0/1990720949 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc9e0095ff0 con 0x7fc9ec101050 2024-05-08T23:46:30.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:46:30.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: Updating smithi160:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: Updating smithi012:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1158105275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/35029108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:30 smithi012 ceph-mon[42503]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-05-08T23:46:30.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.474+0000 7fc9f2b46700 1 -- 172.21.15.12:0/1990720949 --> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9ec198570 con 0x7fc9dc0a1320 2024-05-08T23:46:30.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.486+0000 7fc9e9ffb700 1 -- 172.21.15.12:0/1990720949 <== mgr.15252 v2:172.21.15.12:6800/2525408937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343590 (secure 0 0 0) 0x7fc9ec198570 con 0x7fc9dc0a1320 2024-05-08T23:46:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 -- 172.21.15.12:0/1990720949 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1320 msgr2=0x7fc9dc0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1320 0x7fc9dc0a37e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8006010 tx=0x7fc9d800b540 comp rx=0 tx=0).stop 2024-05-08T23:46:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 -- 172.21.15.12:0/1990720949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9ec101050 msgr2=0x7fc9ec1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9ec101050 0x7fc9ec1972c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000d980 tx=0x7fc9e000dc90 comp rx=0 tx=0).stop 2024-05-08T23:46:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 -- 172.21.15.12:0/1990720949 shutdown_connections 2024-05-08T23:46:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1320 0x7fc9dc0a37e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec19bb50 0x7fc9ec19bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9ec101050 0x7fc9ec1972c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 --2- 172.21.15.12:0/1990720949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9ec106170 0x7fc9ec197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 -- 172.21.15.12:0/1990720949 >> 172.21.15.12:0/1990720949 conn(0x7fc9ec0fc910 msgr2=0x7fc9ec0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:30.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 -- 172.21.15.12:0/1990720949 shutdown_connections 2024-05-08T23:46:30.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.490+0000 7fc9f2b46700 1 -- 172.21.15.12:0/1990720949 wait complete. 2024-05-08T23:46:30.490 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-08T23:46:30.500 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-08T23:46:29.310190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1184721,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":723,"num_read_kb":518,"num_write":450,"num_write_kb":4364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":851,"ondisk_log_size":851,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374900,"kb_used_data":13736,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124566924,"statfs":{"total":1151940427776,"available":1151556530176,"internally_reserved":0,"allocated":14065664,"data_stored":8245746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.979738"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340326+0000","last_change":"2024-05-08T23:45:04.044088+0000","last_active":"2024-05-08T23:46:25.340326+0000","last_peered":"2024-05-08T23:46:25.340326+0000","last_clean":"2024-05-08T23:46:25.340326+0000","last_became_active":"2024-05-08T23:45:04.043535+0000","last_became_peered":"2024-05-08T23:45:04.043535+0000","last_unstale":"2024-05-08T23:46:25.340326+0000","last_undegraded":"2024-05-08T23:46:25.340326+0000","last_fullsized":"2024-05-08T23:46:25.340326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:52.363402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344567+0000","last_change":"2024-05-08T23:45:04.049201+0000","last_active":"2024-05-08T23:46:25.344567+0000","last_peered":"2024-05-08T23:46:25.344567+0000","last_clean":"2024-05-08T23:46:25.344567+0000","last_became_active":"2024-05-08T23:45:04.048743+0000","last_became_peered":"2024-05-08T23:45:04.048743+0000","last_unstale":"2024-05-08T23:46:25.344567+0000","last_undegraded":"2024-05-08T23:46:25.344567+0000","last_fullsized":"2024-05-08T23:46:25.344567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:26.191549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348268+0000","last_change":"2024-05-08T23:45:04.047356+0000","last_active":"2024-05-08T23:46:25.348268+0000","last_peered":"2024-05-08T23:46:25.348268+0000","last_clean":"2024-05-08T23:46:25.348268+0000","last_became_active":"2024-05-08T23:45:04.046996+0000","last_became_peered":"2024-05-08T23:45:04.046996+0000","last_unstale":"2024-05-08T23:46:25.348268+0000","last_undegraded":"2024-05-08T23:46:25.348268+0000","last_fullsized":"2024-05-08T23:46:25.348268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:58.722522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349109+0000","last_change":"2024-05-08T23:45:04.054754+0000","last_active":"2024-05-08T23:46:25.349109+0000","last_peered":"2024-05-08T23:46:25.349109+0000","last_clean":"2024-05-08T23:46:25.349109+0000","last_became_active":"2024-05-08T23:45:04.054302+0000","last_became_peered":"2024-05-08T23:45:04.054302+0000","last_unstale":"2024-05-08T23:46:25.349109+0000","last_undegraded":"2024-05-08T23:46:25.349109+0000","last_fullsized":"2024-05-08T23:46:25.349109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:28.882982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354073+0000","last_change":"2024-05-08T23:45:04.047189+0000","last_active":"2024-05-08T23:46:25.354073+0000","last_peered":"2024-05-08T23:46:25.354073+0000","last_clean":"2024-05-08T23:46:25.354073+0000","last_became_active":"2024-05-08T23:45:04.046756+0000","last_became_peered":"2024-05-08T23:45:04.046756+0000","last_unstale":"2024-05-08T23:46:25.354073+0000","last_undegraded":"2024-05-08T23:46:25.354073+0000","last_fullsized":"2024-05-08T23:46:25.354073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.780169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.338455+0000","last_change":"2024-05-08T23:45:04.060897+0000","last_active":"2024-05-08T23:46:25.338455+0000","last_peered":"2024-05-08T23:46:25.338455+0000","last_clean":"2024-05-08T23:46:25.338455+0000","last_became_active":"2024-05-08T23:45:04.060550+0000","last_became_peered":"2024-05-08T23:45:04.060550+0000","last_unstale":"2024-05-08T23:46:25.338455+0000","last_undegraded":"2024-05-08T23:46:25.338455+0000","last_fullsized":"2024-05-08T23:46:25.338455+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:59.356057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343816+0000","last_change":"2024-05-08T23:45:04.059308+0000","last_active":"2024-05-08T23:46:25.343816+0000","last_peered":"2024-05-08T23:46:25.343816+0000","last_clean":"2024-05-08T23:46:25.343816+0000","last_became_active":"2024-05-08T23:45:04.058920+0000","last_became_peered":"2024-05-08T23:45:04.058920+0000","last_unstale":"2024-05-08T23:46:25.343816+0000","last_undegraded":"2024-05-08T23:46:25.343816+0000","last_fullsized":"2024-05-08T23:46:25.343816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:11.857062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342864+0000","last_change":"2024-05-08T23:45:56.359452+0000","last_active":"2024-05-08T23:46:25.342864+0000","last_peered":"2024-05-08T23:46:25.342864+0000","last_clean":"2024-05-08T23:46:25.342864+0000","last_became_active":"2024-05-08T23:45:56.359066+0000","last_became_peered":"2024-05-08T23:45:56.359066+0000","last_unstale":"2024-05-08T23:46:25.342864+0000","last_undegraded":"2024-05-08T23:46:25.342864+0000","last_fullsized":"2024-05-08T23:46:25.342864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:13.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344525+0000","last_change":"2024-05-08T23:45:04.052292+0000","last_active":"2024-05-08T23:46:25.344525+0000","last_peered":"2024-05-08T23:46:25.344525+0000","last_clean":"2024-05-08T23:46:25.344525+0000","last_became_active":"2024-05-08T23:45:04.051164+0000","last_became_peered":"2024-05-08T23:45:04.051164+0000","last_unstale":"2024-05-08T23:46:25.344525+0000","last_undegraded":"2024-05-08T23:46:25.344525+0000","last_fullsized":"2024-05-08T23:46:25.344525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:09.218130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348445+0000","last_change":"2024-05-08T23:45:04.054566+0000","last_active":"2024-05-08T23:46:25.348445+0000","last_peered":"2024-05-08T23:46:25.348445+0000","last_clean":"2024-05-08T23:46:25.348445+0000","last_became_active":"2024-05-08T23:45:04.053878+0000","last_became_peered":"2024-05-08T23:45:04.053878+0000","last_unstale":"2024-05-08T23:46:25.348445+0000","last_undegraded":"2024-05-08T23:46:25.348445+0000","last_fullsized":"2024-05-08T23:46:25.348445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:34.760038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342587+0000","last_change":"2024-05-08T23:45:04.049696+0000","last_active":"2024-05-08T23:46:25.342587+0000","last_peered":"2024-05-08T23:46:25.342587+0000","last_clean":"2024-05-08T23:46:25.342587+0000","last_became_active":"2024-05-08T23:45:04.049351+0000","last_became_peered":"2024-05-08T23:45:04.049351+0000","last_unstale":"2024-05-08T23:46:25.342587+0000","last_undegraded":"2024-05-08T23:46:25.342587+0000","last_fullsized":"2024-05-08T23:46:25.342587+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:53.557738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348957+0000","last_change":"2024-05-08T23:45:04.051010+0000","last_active":"2024-05-08T23:46:25.348957+0000","last_peered":"2024-05-08T23:46:25.348957+0000","last_clean":"2024-05-08T23:46:25.348957+0000","last_became_active":"2024-05-08T23:45:04.050715+0000","last_became_peered":"2024-05-08T23:45:04.050715+0000","last_unstale":"2024-05-08T23:46:25.348957+0000","last_undegraded":"2024-05-08T23:46:25.348957+0000","last_fullsized":"2024-05-08T23:46:25.348957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:09.305974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.353741+0000","last_change":"2024-05-08T23:45:04.051005+0000","last_active":"2024-05-08T23:46:25.353741+0000","last_peered":"2024-05-08T23:46:25.353741+0000","last_clean":"2024-05-08T23:46:25.353741+0000","last_became_active":"2024-05-08T23:45:04.050123+0000","last_became_peered":"2024-05-08T23:45:04.050123+0000","last_unstale":"2024-05-08T23:46:25.353741+0000","last_undegraded":"2024-05-08T23:46:25.353741+0000","last_fullsized":"2024-05-08T23:46:25.353741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:52.206389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348724+0000","last_change":"2024-05-08T23:45:04.049942+0000","last_active":"2024-05-08T23:46:25.348724+0000","last_peered":"2024-05-08T23:46:25.348724+0000","last_clean":"2024-05-08T23:46:25.348724+0000","last_became_active":"2024-05-08T23:45:04.049596+0000","last_became_peered":"2024-05-08T23:45:04.049596+0000","last_unstale":"2024-05-08T23:46:25.348724+0000","last_undegraded":"2024-05-08T23:46:25.348724+0000","last_fullsized":"2024-05-08T23:46:25.348724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:30.907023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350562+0000","last_change":"2024-05-08T23:45:04.048985+0000","last_active":"2024-05-08T23:46:25.350562+0000","last_peered":"2024-05-08T23:46:25.350562+0000","last_clean":"2024-05-08T23:46:25.350562+0000","last_became_active":"2024-05-08T23:45:04.048658+0000","last_became_peered":"2024-05-08T23:45:04.048658+0000","last_unstale":"2024-05-08T23:46:25.350562+0000","last_undegraded":"2024-05-08T23:46:25.350562+0000","last_fullsized":"2024-05-08T23:46:25.350562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:17.030930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.338683+0000","last_change":"2024-05-08T23:45:04.051903+0000","last_active":"2024-05-08T23:46:25.338683+0000","last_peered":"2024-05-08T23:46:25.338683+0000","last_clean":"2024-05-08T23:46:25.338683+0000","last_became_active":"2024-05-08T23:45:04.051221+0000","last_became_peered":"2024-05-08T23:45:04.051221+0000","last_unstale":"2024-05-08T23:46:25.338683+0000","last_undegraded":"2024-05-08T23:46:25.338683+0000","last_fullsized":"2024-05-08T23:46:25.338683+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:01.554463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340534+0000","last_change":"2024-05-08T23:45:04.043854+0000","last_active":"2024-05-08T23:46:25.340534+0000","last_peered":"2024-05-08T23:46:25.340534+0000","last_clean":"2024-05-08T23:46:25.340534+0000","last_became_active":"2024-05-08T23:45:04.043056+0000","last_became_peered":"2024-05-08T23:45:04.043056+0000","last_unstale":"2024-05-08T23:46:25.340534+0000","last_undegraded":"2024-05-08T23:46:25.340534+0000","last_fullsized":"2024-05-08T23:46:25.340534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:18.175575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.338204+0000","last_change":"2024-05-08T23:45:04.049332+0000","last_active":"2024-05-08T23:46:25.338204+0000","last_peered":"2024-05-08T23:46:25.338204+0000","last_clean":"2024-05-08T23:46:25.338204+0000","last_became_active":"2024-05-08T23:45:04.048987+0000","last_became_peered":"2024-05-08T23:45:04.048987+0000","last_unstale":"2024-05-08T23:46:25.338204+0000","last_undegraded":"2024-05-08T23:46:25.338204+0000","last_fullsized":"2024-05-08T23:46:25.338204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:02.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333688+0000","last_change":"2024-05-08T23:45:04.048536+0000","last_active":"2024-05-08T23:46:25.333688+0000","last_peered":"2024-05-08T23:46:25.333688+0000","last_clean":"2024-05-08T23:46:25.333688+0000","last_became_active":"2024-05-08T23:45:04.048102+0000","last_became_peered":"2024-05-08T23:45:04.048102+0000","last_unstale":"2024-05-08T23:46:25.333688+0000","last_undegraded":"2024-05-08T23:46:25.333688+0000","last_fullsized":"2024-05-08T23:46:25.333688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:11.794372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.353062+0000","last_change":"2024-05-08T23:45:03.576833+0000","last_active":"2024-05-08T23:45:56.353062+0000","last_peered":"2024-05-08T23:45:56.353062+0000","last_clean":"2024-05-08T23:45:56.353062+0000","last_became_active":"2024-05-08T23:45:03.576495+0000","last_became_peered":"2024-05-08T23:45:03.576495+0000","last_unstale":"2024-05-08T23:45:56.353062+0000","last_undegraded":"2024-05-08T23:45:56.353062+0000","last_fullsized":"2024-05-08T23:45:56.353062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:50.266658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340176+0000","last_change":"2024-05-08T23:45:04.055444+0000","last_active":"2024-05-08T23:46:25.340176+0000","last_peered":"2024-05-08T23:46:25.340176+0000","last_clean":"2024-05-08T23:46:25.340176+0000","last_became_active":"2024-05-08T23:45:04.054998+0000","last_became_peered":"2024-05-08T23:45:04.054998+0000","last_unstale":"2024-05-08T23:46:25.340176+0000","last_undegraded":"2024-05-08T23:46:25.340176+0000","last_fullsized":"2024-05-08T23:46:25.340176+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:03.470555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.359775+0000","last_change":"2024-05-08T23:45:03.578362+0000","last_active":"2024-05-08T23:45:56.359775+0000","last_peered":"2024-05-08T23:45:56.359775+0000","last_clean":"2024-05-08T23:45:56.359775+0000","last_became_active":"2024-05-08T23:45:03.577834+0000","last_became_peered":"2024-05-08T23:45:03.577834+0000","last_unstale":"2024-05-08T23:45:56.359775+0000","last_undegraded":"2024-05-08T23:45:56.359775+0000","last_fullsized":"2024-05-08T23:45:56.359775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:01.943490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.346351+0000","last_change":"2024-05-08T23:45:04.047205+0000","last_active":"2024-05-08T23:46:25.346351+0000","last_peered":"2024-05-08T23:46:25.346351+0000","last_clean":"2024-05-08T23:46:25.346351+0000","last_became_active":"2024-05-08T23:45:04.046860+0000","last_became_peered":"2024-05-08T23:45:04.046860+0000","last_unstale":"2024-05-08T23:46:25.346351+0000","last_undegraded":"2024-05-08T23:46:25.346351+0000","last_fullsized":"2024-05-08T23:46:25.346351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:36.837101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.361012+0000","last_change":"2024-05-08T23:45:03.582026+0000","last_active":"2024-05-08T23:45:56.361012+0000","last_peered":"2024-05-08T23:45:56.361012+0000","last_clean":"2024-05-08T23:45:56.361012+0000","last_became_active":"2024-05-08T23:45:03.581672+0000","last_became_peered":"2024-05-08T23:45:03.581672+0000","last_unstale":"2024-05-08T23:45:56.361012+0000","last_undegraded":"2024-05-08T23:45:56.361012+0000","last_fullsized":"2024-05-08T23:45:56.361012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:50.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349108+0000","last_change":"2024-05-08T23:45:04.052639+0000","last_active":"2024-05-08T23:46:25.349108+0000","last_peered":"2024-05-08T23:46:25.349108+0000","last_clean":"2024-05-08T23:46:25.349108+0000","last_became_active":"2024-05-08T23:45:04.052158+0000","last_became_peered":"2024-05-08T23:45:04.052158+0000","last_unstale":"2024-05-08T23:46:25.349108+0000","last_undegraded":"2024-05-08T23:46:25.349108+0000","last_fullsized":"2024-05-08T23:46:25.349108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:25.638555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360530+0000","last_change":"2024-05-08T23:45:03.579221+0000","last_active":"2024-05-08T23:45:56.360530+0000","last_peered":"2024-05-08T23:45:56.360530+0000","last_clean":"2024-05-08T23:45:56.360530+0000","last_became_active":"2024-05-08T23:45:03.578795+0000","last_became_peered":"2024-05-08T23:45:03.578795+0000","last_unstale":"2024-05-08T23:45:56.360530+0000","last_undegraded":"2024-05-08T23:45:56.360530+0000","last_fullsized":"2024-05-08T23:45:56.360530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:14.064108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350032+0000","last_change":"2024-05-08T23:45:04.057573+0000","last_active":"2024-05-08T23:46:25.350032+0000","last_peered":"2024-05-08T23:46:25.350032+0000","last_clean":"2024-05-08T23:46:25.350032+0000","last_became_active":"2024-05-08T23:45:04.053201+0000","last_became_peered":"2024-05-08T23:45:04.053201+0000","last_unstale":"2024-05-08T23:46:25.350032+0000","last_undegraded":"2024-05-08T23:46:25.350032+0000","last_fullsized":"2024-05-08T23:46:25.350032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:23.612609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.313218+0000","last_change":"2024-05-08T23:45:03.578110+0000","last_active":"2024-05-08T23:46:25.313218+0000","last_peered":"2024-05-08T23:46:25.313218+0000","last_clean":"2024-05-08T23:46:25.313218+0000","last_became_active":"2024-05-08T23:45:03.577749+0000","last_became_peered":"2024-05-08T23:45:03.577749+0000","last_unstale":"2024-05-08T23:46:25.313218+0000","last_undegraded":"2024-05-08T23:46:25.313218+0000","last_fullsized":"2024-05-08T23:46:25.313218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:50.526535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.337443+0000","last_change":"2024-05-08T23:45:04.048131+0000","last_active":"2024-05-08T23:46:25.337443+0000","last_peered":"2024-05-08T23:46:25.337443+0000","last_clean":"2024-05-08T23:46:25.337443+0000","last_became_active":"2024-05-08T23:45:04.047773+0000","last_became_peered":"2024-05-08T23:45:04.047773+0000","last_unstale":"2024-05-08T23:46:25.337443+0000","last_undegraded":"2024-05-08T23:46:25.337443+0000","last_fullsized":"2024-05-08T23:46:25.337443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:47.944900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.353596+0000","last_change":"2024-05-08T23:45:03.586710+0000","last_active":"2024-05-08T23:45:56.353596+0000","last_peered":"2024-05-08T23:45:56.353596+0000","last_clean":"2024-05-08T23:45:56.353596+0000","last_became_active":"2024-05-08T23:45:03.586411+0000","last_became_peered":"2024-05-08T23:45:03.586411+0000","last_unstale":"2024-05-08T23:45:56.353596+0000","last_undegraded":"2024-05-08T23:45:56.353596+0000","last_fullsized":"2024-05-08T23:45:56.353596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:25.938846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349408+0000","last_change":"2024-05-08T23:45:04.053885+0000","last_active":"2024-05-08T23:46:25.349408+0000","last_peered":"2024-05-08T23:46:25.349408+0000","last_clean":"2024-05-08T23:46:25.349408+0000","last_became_active":"2024-05-08T23:45:04.053385+0000","last_became_peered":"2024-05-08T23:45:04.053385+0000","last_unstale":"2024-05-08T23:46:25.349408+0000","last_undegraded":"2024-05-08T23:46:25.349408+0000","last_fullsized":"2024-05-08T23:46:25.349408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:10.805481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.350475+0000","last_change":"2024-05-08T23:45:03.574988+0000","last_active":"2024-05-08T23:45:56.350475+0000","last_peered":"2024-05-08T23:45:56.350475+0000","last_clean":"2024-05-08T23:45:56.350475+0000","last_became_active":"2024-05-08T23:45:03.574586+0000","last_became_peered":"2024-05-08T23:45:03.574586+0000","last_unstale":"2024-05-08T23:45:56.350475+0000","last_undegraded":"2024-05-08T23:45:56.350475+0000","last_fullsized":"2024-05-08T23:45:56.350475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:18.005107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354197+0000","last_change":"2024-05-08T23:45:04.046817+0000","last_active":"2024-05-08T23:46:25.354197+0000","last_peered":"2024-05-08T23:46:25.354197+0000","last_clean":"2024-05-08T23:46:25.354197+0000","last_became_active":"2024-05-08T23:45:04.046515+0000","last_became_peered":"2024-05-08T23:45:04.046515+0000","last_unstale":"2024-05-08T23:46:25.354197+0000","last_undegraded":"2024-05-08T23:46:25.354197+0000","last_fullsized":"2024-05-08T23:46:25.354197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:29.439670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355956+0000","last_change":"2024-05-08T23:45:03.579002+0000","last_active":"2024-05-08T23:45:56.355956+0000","last_peered":"2024-05-08T23:45:56.355956+0000","last_clean":"2024-05-08T23:45:56.355956+0000","last_became_active":"2024-05-08T23:45:03.577891+0000","last_became_peered":"2024-05-08T23:45:03.577891+0000","last_unstale":"2024-05-08T23:45:56.355956+0000","last_undegraded":"2024-05-08T23:45:56.355956+0000","last_fullsized":"2024-05-08T23:45:56.355956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:01.027574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344908+0000","last_change":"2024-05-08T23:45:04.060715+0000","last_active":"2024-05-08T23:46:25.344908+0000","last_peered":"2024-05-08T23:46:25.344908+0000","last_clean":"2024-05-08T23:46:25.344908+0000","last_became_active":"2024-05-08T23:45:04.060375+0000","last_became_peered":"2024-05-08T23:45:04.060375+0000","last_unstale":"2024-05-08T23:46:25.344908+0000","last_undegraded":"2024-05-08T23:46:25.344908+0000","last_fullsized":"2024-05-08T23:46:25.344908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:42.133323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.307895+0000","last_change":"2024-05-08T23:45:03.587784+0000","last_active":"2024-05-08T23:46:25.307895+0000","last_peered":"2024-05-08T23:46:25.307895+0000","last_clean":"2024-05-08T23:46:25.307895+0000","last_became_active":"2024-05-08T23:45:03.587529+0000","last_became_peered":"2024-05-08T23:45:03.587529+0000","last_unstale":"2024-05-08T23:46:25.307895+0000","last_undegraded":"2024-05-08T23:46:25.307895+0000","last_fullsized":"2024-05-08T23:46:25.307895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:00.323599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344334+0000","last_change":"2024-05-08T23:45:04.062672+0000","last_active":"2024-05-08T23:46:25.344334+0000","last_peered":"2024-05-08T23:46:25.344334+0000","last_clean":"2024-05-08T23:46:25.344334+0000","last_became_active":"2024-05-08T23:45:04.062351+0000","last_became_peered":"2024-05-08T23:45:04.062351+0000","last_unstale":"2024-05-08T23:46:25.344334+0000","last_undegraded":"2024-05-08T23:46:25.344334+0000","last_fullsized":"2024-05-08T23:46:25.344334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:13.740239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355455+0000","last_change":"2024-05-08T23:45:03.583501+0000","last_active":"2024-05-08T23:45:56.355455+0000","last_peered":"2024-05-08T23:45:56.355455+0000","last_clean":"2024-05-08T23:45:56.355455+0000","last_became_active":"2024-05-08T23:45:03.583116+0000","last_became_peered":"2024-05-08T23:45:03.583116+0000","last_unstale":"2024-05-08T23:45:56.355455+0000","last_undegraded":"2024-05-08T23:45:56.355455+0000","last_fullsized":"2024-05-08T23:45:56.355455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:03.797524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350404+0000","last_change":"2024-05-08T23:45:04.048336+0000","last_active":"2024-05-08T23:46:25.350404+0000","last_peered":"2024-05-08T23:46:25.350404+0000","last_clean":"2024-05-08T23:46:25.350404+0000","last_became_active":"2024-05-08T23:45:04.048000+0000","last_became_peered":"2024-05-08T23:45:04.048000+0000","last_unstale":"2024-05-08T23:46:25.350404+0000","last_undegraded":"2024-05-08T23:46:25.350404+0000","last_fullsized":"2024-05-08T23:46:25.350404+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:41.806600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355898+0000","last_change":"2024-05-08T23:45:03.581215+0000","last_active":"2024-05-08T23:45:56.355898+0000","last_peered":"2024-05-08T23:45:56.355898+0000","last_clean":"2024-05-08T23:45:56.355898+0000","last_became_active":"2024-05-08T23:45:03.580496+0000","last_became_peered":"2024-05-08T23:45:03.580496+0000","last_unstale":"2024-05-08T23:45:56.355898+0000","last_undegraded":"2024-05-08T23:45:56.355898+0000","last_fullsized":"2024-05-08T23:45:56.355898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:25.602401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.346299+0000","last_change":"2024-05-08T23:45:04.058857+0000","last_active":"2024-05-08T23:46:25.346299+0000","last_peered":"2024-05-08T23:46:25.346299+0000","last_clean":"2024-05-08T23:46:25.346299+0000","last_became_active":"2024-05-08T23:45:04.057988+0000","last_became_peered":"2024-05-08T23:45:04.057988+0000","last_unstale":"2024-05-08T23:46:25.346299+0000","last_undegraded":"2024-05-08T23:46:25.346299+0000","last_fullsized":"2024-05-08T23:46:25.346299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:46.725976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:26.195804+0000","last_change":"2024-05-08T23:45:03.582121+0000","last_active":"2024-05-08T23:46:26.195804+0000","last_peered":"2024-05-08T23:46:26.195804+0000","last_clean":"2024-05-08T23:46:26.195804+0000","last_became_active":"2024-05-08T23:45:03.581796+0000","last_became_peered":"2024-05-08T23:45:03.581796+0000","last_unstale":"2024-05-08T23:46:26.195804+0000","last_undegraded":"2024-05-08T23:46:26.195804+0000","last_fullsized":"2024-05-08T23:46:26.195804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:55.417404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65082,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":145,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.330922+0000","last_change":"2024-05-08T23:45:04.052993+0000","last_active":"2024-05-08T23:46:25.330922+0000","last_peered":"2024-05-08T23:46:25.330922+0000","last_clean":"2024-05-08T23:46:25.330922+0000","last_became_active":"2024-05-08T23:45:04.052088+0000","last_became_peered":"2024-05-08T23:45:04.052088+0000","last_unstale":"2024-05-08T23:46:25.330922+0000","last_undegraded":"2024-05-08T23:46:25.330922+0000","last_fullsized":"2024-05-08T23:46:25.330922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:44.531649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:46:00.332257+0000","last_change":"2024-05-08T23:45:56.372913+0000","last_active":"2024-05-08T23:46:00.332257+0000","last_peered":"2024-05-08T23:46:00.332257+0000","last_clean":"2024-05-08T23:46:00.332257+0000","last_became_active":"2024-05-08T23:45:56.361402+0000","last_became_peered":"2024-05-08T23:45:56.361402+0000","last_unstale":"2024-05-08T23:46:00.332257+0000","last_undegraded":"2024-05-08T23:46:00.332257+0000","last_fullsized":"2024-05-08T23:46:00.332257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:34.652216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.353482+0000","last_change":"2024-05-08T23:45:04.050294+0000","last_active":"2024-05-08T23:46:25.353482+0000","last_peered":"2024-05-08T23:46:25.353482+0000","last_clean":"2024-05-08T23:46:25.353482+0000","last_became_active":"2024-05-08T23:45:04.049949+0000","last_became_peered":"2024-05-08T23:45:04.049949+0000","last_unstale":"2024-05-08T23:46:25.353482+0000","last_undegraded":"2024-05-08T23:46:25.353482+0000","last_fullsized":"2024-05-08T23:46:25.353482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:17.253201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355432+0000","last_change":"2024-05-08T23:45:03.579821+0000","last_active":"2024-05-08T23:45:56.355432+0000","last_peered":"2024-05-08T23:45:56.355432+0000","last_clean":"2024-05-08T23:45:56.355432+0000","last_became_active":"2024-05-08T23:45:03.578635+0000","last_became_peered":"2024-05-08T23:45:03.578635+0000","last_unstale":"2024-05-08T23:45:56.355432+0000","last_undegraded":"2024-05-08T23:45:56.355432+0000","last_fullsized":"2024-05-08T23:45:56.355432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:46.179505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354613+0000","last_change":"2024-05-08T23:45:04.059357+0000","last_active":"2024-05-08T23:46:25.354613+0000","last_peered":"2024-05-08T23:46:25.354613+0000","last_clean":"2024-05-08T23:46:25.354613+0000","last_became_active":"2024-05-08T23:45:04.059070+0000","last_became_peered":"2024-05-08T23:45:04.059070+0000","last_unstale":"2024-05-08T23:46:25.354613+0000","last_undegraded":"2024-05-08T23:46:25.354613+0000","last_fullsized":"2024-05-08T23:46:25.354613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:50.626787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:26.197796+0000","last_change":"2024-05-08T23:45:03.577199+0000","last_active":"2024-05-08T23:46:26.197796+0000","last_peered":"2024-05-08T23:46:26.197796+0000","last_clean":"2024-05-08T23:46:26.197796+0000","last_became_active":"2024-05-08T23:45:03.576844+0000","last_became_peered":"2024-05-08T23:45:03.576844+0000","last_unstale":"2024-05-08T23:46:26.197796+0000","last_undegraded":"2024-05-08T23:46:26.197796+0000","last_fullsized":"2024-05-08T23:46:26.197796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:27.139493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340110+0000","last_change":"2024-05-08T23:45:56.356625+0000","last_active":"2024-05-08T23:46:25.340110+0000","last_peered":"2024-05-08T23:46:25.340110+0000","last_clean":"2024-05-08T23:46:25.340110+0000","last_became_active":"2024-05-08T23:45:56.356240+0000","last_became_peered":"2024-05-08T23:45:56.356240+0000","last_unstale":"2024-05-08T23:46:25.340110+0000","last_undegraded":"2024-05-08T23:46:25.340110+0000","last_fullsized":"2024-05-08T23:46:25.340110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:50.251606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360317+0000","last_change":"2024-05-08T23:45:03.576766+0000","last_active":"2024-05-08T23:45:56.360317+0000","last_peered":"2024-05-08T23:45:56.360317+0000","last_clean":"2024-05-08T23:45:56.360317+0000","last_became_active":"2024-05-08T23:45:03.576410+0000","last_became_peered":"2024-05-08T23:45:03.576410+0000","last_unstale":"2024-05-08T23:45:56.360317+0000","last_undegraded":"2024-05-08T23:45:56.360317+0000","last_fullsized":"2024-05-08T23:45:56.360317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:29.444589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349612+0000","last_change":"2024-05-08T23:45:04.056563+0000","last_active":"2024-05-08T23:46:25.349612+0000","last_peered":"2024-05-08T23:46:25.349612+0000","last_clean":"2024-05-08T23:46:25.349612+0000","last_became_active":"2024-05-08T23:45:04.053132+0000","last_became_peered":"2024-05-08T23:45:04.053132+0000","last_unstale":"2024-05-08T23:46:25.349612+0000","last_undegraded":"2024-05-08T23:46:25.349612+0000","last_fullsized":"2024-05-08T23:46:25.349612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:41.625804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:46:22.745519+0000","last_change":"2024-05-08T23:45:03.580343+0000","last_active":"2024-05-08T23:46:22.745519+0000","last_peered":"2024-05-08T23:46:22.745519+0000","last_clean":"2024-05-08T23:46:22.745519+0000","last_became_active":"2024-05-08T23:45:03.578719+0000","last_became_peered":"2024-05-08T23:45:03.578719+0000","last_unstale":"2024-05-08T23:46:22.745519+0000","last_undegraded":"2024-05-08T23:46:22.745519+0000","last_fullsized":"2024-05-08T23:46:22.745519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:54.831252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343273+0000","last_change":"2024-05-08T23:45:04.050643+0000","last_active":"2024-05-08T23:46:25.343273+0000","last_peered":"2024-05-08T23:46:25.343273+0000","last_clean":"2024-05-08T23:46:25.343273+0000","last_became_active":"2024-05-08T23:45:04.050309+0000","last_became_peered":"2024-05-08T23:45:04.050309+0000","last_unstale":"2024-05-08T23:46:25.343273+0000","last_undegraded":"2024-05-08T23:46:25.343273+0000","last_fullsized":"2024-05-08T23:46:25.343273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:30.259530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.352954+0000","last_change":"2024-05-08T23:45:03.579590+0000","last_active":"2024-05-08T23:45:56.352954+0000","last_peered":"2024-05-08T23:45:56.352954+0000","last_clean":"2024-05-08T23:45:56.352954+0000","last_became_active":"2024-05-08T23:45:03.579213+0000","last_became_peered":"2024-05-08T23:45:03.579213+0000","last_unstale":"2024-05-08T23:45:56.352954+0000","last_undegraded":"2024-05-08T23:45:56.352954+0000","last_fullsized":"2024-05-08T23:45:56.352954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:55.666543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348872+0000","last_change":"2024-05-08T23:45:04.049082+0000","last_active":"2024-05-08T23:46:25.348872+0000","last_peered":"2024-05-08T23:46:25.348872+0000","last_clean":"2024-05-08T23:46:25.348872+0000","last_became_active":"2024-05-08T23:45:04.048709+0000","last_became_peered":"2024-05-08T23:45:04.048709+0000","last_unstale":"2024-05-08T23:46:25.348872+0000","last_undegraded":"2024-05-08T23:46:25.348872+0000","last_fullsized":"2024-05-08T23:46:25.348872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:56.365817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.354871+0000","last_change":"2024-05-08T23:45:03.581363+0000","last_active":"2024-05-08T23:45:56.354871+0000","last_peered":"2024-05-08T23:45:56.354871+0000","last_clean":"2024-05-08T23:45:56.354871+0000","last_became_active":"2024-05-08T23:45:03.580840+0000","last_became_peered":"2024-05-08T23:45:03.580840+0000","last_unstale":"2024-05-08T23:45:56.354871+0000","last_undegraded":"2024-05-08T23:45:56.354871+0000","last_fullsized":"2024-05-08T23:45:56.354871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:27.281228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.345492+0000","last_change":"2024-05-08T23:45:56.361444+0000","last_active":"2024-05-08T23:46:25.345492+0000","last_peered":"2024-05-08T23:46:25.345492+0000","last_clean":"2024-05-08T23:46:25.345492+0000","last_became_active":"2024-05-08T23:45:56.361099+0000","last_became_peered":"2024-05-08T23:45:56.361099+0000","last_unstale":"2024-05-08T23:46:25.345492+0000","last_undegraded":"2024-05-08T23:46:25.345492+0000","last_fullsized":"2024-05-08T23:46:25.345492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:25.342212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349895+0000","last_change":"2024-05-08T23:45:04.061189+0000","last_active":"2024-05-08T23:46:25.349895+0000","last_peered":"2024-05-08T23:46:25.349895+0000","last_clean":"2024-05-08T23:46:25.349895+0000","last_became_active":"2024-05-08T23:45:04.060838+0000","last_became_peered":"2024-05-08T23:45:04.060838+0000","last_unstale":"2024-05-08T23:46:25.349895+0000","last_undegraded":"2024-05-08T23:46:25.349895+0000","last_fullsized":"2024-05-08T23:46:25.349895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:43.794987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":98,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.296864+0000","last_change":"2024-05-08T23:45:03.583841+0000","last_active":"2024-05-08T23:46:25.296864+0000","last_peered":"2024-05-08T23:46:25.296864+0000","last_clean":"2024-05-08T23:46:25.296864+0000","last_became_active":"2024-05-08T23:45:03.583275+0000","last_became_peered":"2024-05-08T23:45:03.583275+0000","last_unstale":"2024-05-08T23:46:25.296864+0000","last_undegraded":"2024-05-08T23:46:25.296864+0000","last_fullsized":"2024-05-08T23:46:25.296864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:14.560505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.339715+0000","last_change":"2024-05-08T23:45:04.052667+0000","last_active":"2024-05-08T23:46:25.339715+0000","last_peered":"2024-05-08T23:46:25.339715+0000","last_clean":"2024-05-08T23:46:25.339715+0000","last_became_active":"2024-05-08T23:45:04.052008+0000","last_became_peered":"2024-05-08T23:45:04.052008+0000","last_unstale":"2024-05-08T23:46:25.339715+0000","last_undegraded":"2024-05-08T23:46:25.339715+0000","last_fullsized":"2024-05-08T23:46:25.339715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:35.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.351515+0000","last_change":"2024-05-08T23:45:03.573665+0000","last_active":"2024-05-08T23:45:56.351515+0000","last_peered":"2024-05-08T23:45:56.351515+0000","last_clean":"2024-05-08T23:45:56.351515+0000","last_became_active":"2024-05-08T23:45:03.573265+0000","last_became_peered":"2024-05-08T23:45:03.573265+0000","last_unstale":"2024-05-08T23:45:56.351515+0000","last_undegraded":"2024-05-08T23:45:56.351515+0000","last_fullsized":"2024-05-08T23:45:56.351515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:13.410954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":333,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355192+0000","last_change":"2024-05-08T23:44:16.373962+0000","last_active":"2024-05-08T23:45:56.355192+0000","last_peered":"2024-05-08T23:45:56.355192+0000","last_clean":"2024-05-08T23:45:56.355192+0000","last_became_active":"2024-05-08T23:44:16.373358+0000","last_became_peered":"2024-05-08T23:44:16.373358+0000","last_unstale":"2024-05-08T23:45:56.355192+0000","last_undegraded":"2024-05-08T23:45:56.355192+0000","last_fullsized":"2024-05-08T23:45:56.355192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.067168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:24.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349501+0000","last_change":"2024-05-08T23:45:04.048468+0000","last_active":"2024-05-08T23:46:25.349501+0000","last_peered":"2024-05-08T23:46:25.349501+0000","last_clean":"2024-05-08T23:46:25.349501+0000","last_became_active":"2024-05-08T23:45:04.048085+0000","last_became_peered":"2024-05-08T23:45:04.048085+0000","last_unstale":"2024-05-08T23:46:25.349501+0000","last_undegraded":"2024-05-08T23:46:25.349501+0000","last_fullsized":"2024-05-08T23:46:25.349501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:18.039780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355333+0000","last_change":"2024-05-08T23:45:03.575821+0000","last_active":"2024-05-08T23:45:56.355333+0000","last_peered":"2024-05-08T23:45:56.355333+0000","last_clean":"2024-05-08T23:45:56.355333+0000","last_became_active":"2024-05-08T23:45:03.575407+0000","last_became_peered":"2024-05-08T23:45:03.575407+0000","last_unstale":"2024-05-08T23:45:56.355333+0000","last_undegraded":"2024-05-08T23:45:56.355333+0000","last_fullsized":"2024-05-08T23:45:56.355333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:35.718942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:46:00.332281+0000","last_change":"2024-05-08T23:45:56.365483+0000","last_active":"2024-05-08T23:46:00.332281+0000","last_peered":"2024-05-08T23:46:00.332281+0000","last_clean":"2024-05-08T23:46:00.332281+0000","last_became_active":"2024-05-08T23:45:56.359626+0000","last_became_peered":"2024-05-08T23:45:56.359626+0000","last_unstale":"2024-05-08T23:46:00.332281+0000","last_undegraded":"2024-05-08T23:46:00.332281+0000","last_fullsized":"2024-05-08T23:46:00.332281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:38.284296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349154+0000","last_change":"2024-05-08T23:45:04.047687+0000","last_active":"2024-05-08T23:46:25.349154+0000","last_peered":"2024-05-08T23:46:25.349154+0000","last_clean":"2024-05-08T23:46:25.349154+0000","last_became_active":"2024-05-08T23:45:04.047343+0000","last_became_peered":"2024-05-08T23:45:04.047343+0000","last_unstale":"2024-05-08T23:46:25.349154+0000","last_undegraded":"2024-05-08T23:46:25.349154+0000","last_fullsized":"2024-05-08T23:46:25.349154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:58.763421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'39","reported_seq":77,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.567780+0000","last_change":"2024-05-08T23:45:03.573344+0000","last_active":"2024-05-08T23:46:25.567780+0000","last_peered":"2024-05-08T23:46:25.567780+0000","last_clean":"2024-05-08T23:46:25.567780+0000","last_became_active":"2024-05-08T23:45:03.572951+0000","last_became_peered":"2024-05-08T23:45:03.572951+0000","last_unstale":"2024-05-08T23:46:25.567780+0000","last_undegraded":"2024-05-08T23:46:25.567780+0000","last_fullsized":"2024-05-08T23:46:25.567780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.422908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":68,"num_write_kb":833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.335456+0000","last_change":"2024-05-08T23:45:04.055111+0000","last_active":"2024-05-08T23:46:25.335456+0000","last_peered":"2024-05-08T23:46:25.335456+0000","last_clean":"2024-05-08T23:46:25.335456+0000","last_became_active":"2024-05-08T23:45:04.054665+0000","last_became_peered":"2024-05-08T23:45:04.054665+0000","last_unstale":"2024-05-08T23:46:25.335456+0000","last_undegraded":"2024-05-08T23:46:25.335456+0000","last_fullsized":"2024-05-08T23:46:25.335456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:23.292630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.351606+0000","last_change":"2024-05-08T23:45:03.578168+0000","last_active":"2024-05-08T23:45:56.351606+0000","last_peered":"2024-05-08T23:45:56.351606+0000","last_clean":"2024-05-08T23:45:56.351606+0000","last_became_active":"2024-05-08T23:45:03.577833+0000","last_became_peered":"2024-05-08T23:45:03.577833+0000","last_unstale":"2024-05-08T23:45:56.351606+0000","last_undegraded":"2024-05-08T23:45:56.351606+0000","last_fullsized":"2024-05-08T23:45:56.351606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:13:25.743151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354570+0000","last_change":"2024-05-08T23:45:04.049103+0000","last_active":"2024-05-08T23:46:25.354570+0000","last_peered":"2024-05-08T23:46:25.354570+0000","last_clean":"2024-05-08T23:46:25.354570+0000","last_became_active":"2024-05-08T23:45:04.048302+0000","last_became_peered":"2024-05-08T23:45:04.048302+0000","last_unstale":"2024-05-08T23:46:25.354570+0000","last_undegraded":"2024-05-08T23:46:25.354570+0000","last_fullsized":"2024-05-08T23:46:25.354570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:02.404522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.614524+0000","last_change":"2024-05-08T23:45:03.577765+0000","last_active":"2024-05-08T23:45:56.614524+0000","last_peered":"2024-05-08T23:45:56.614524+0000","last_clean":"2024-05-08T23:45:56.614524+0000","last_became_active":"2024-05-08T23:45:03.577452+0000","last_became_peered":"2024-05-08T23:45:03.577452+0000","last_unstale":"2024-05-08T23:45:56.614524+0000","last_undegraded":"2024-05-08T23:45:56.614524+0000","last_fullsized":"2024-05-08T23:45:56.614524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:02.107878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.315831+0000","last_change":"2024-05-08T23:45:04.061360+0000","last_active":"2024-05-08T23:46:25.315831+0000","last_peered":"2024-05-08T23:46:25.315831+0000","last_clean":"2024-05-08T23:46:25.315831+0000","last_became_active":"2024-05-08T23:45:04.061000+0000","last_became_peered":"2024-05-08T23:45:04.061000+0000","last_unstale":"2024-05-08T23:46:25.315831+0000","last_undegraded":"2024-05-08T23:46:25.315831+0000","last_fullsized":"2024-05-08T23:46:25.315831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:35.757571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360024+0000","last_change":"2024-05-08T23:45:03.578769+0000","last_active":"2024-05-08T23:45:56.360024+0000","last_peered":"2024-05-08T23:45:56.360024+0000","last_clean":"2024-05-08T23:45:56.360024+0000","last_became_active":"2024-05-08T23:45:03.578359+0000","last_became_peered":"2024-05-08T23:45:03.578359+0000","last_unstale":"2024-05-08T23:45:56.360024+0000","last_undegraded":"2024-05-08T23:45:56.360024+0000","last_fullsized":"2024-05-08T23:45:56.360024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:13.213382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333169+0000","last_change":"2024-05-08T23:45:04.049237+0000","last_active":"2024-05-08T23:46:25.333169+0000","last_peered":"2024-05-08T23:46:25.333169+0000","last_clean":"2024-05-08T23:46:25.333169+0000","last_became_active":"2024-05-08T23:45:04.048922+0000","last_became_peered":"2024-05-08T23:45:04.048922+0000","last_unstale":"2024-05-08T23:46:25.333169+0000","last_undegraded":"2024-05-08T23:46:25.333169+0000","last_fullsized":"2024-05-08T23:46:25.333169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:29.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.354233+0000","last_change":"2024-05-08T23:45:03.574990+0000","last_active":"2024-05-08T23:45:56.354233+0000","last_peered":"2024-05-08T23:45:56.354233+0000","last_clean":"2024-05-08T23:45:56.354233+0000","last_became_active":"2024-05-08T23:45:03.574617+0000","last_became_peered":"2024-05-08T23:45:03.574617+0000","last_unstale":"2024-05-08T23:45:56.354233+0000","last_undegraded":"2024-05-08T23:45:56.354233+0000","last_fullsized":"2024-05-08T23:45:56.354233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:51.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.337429+0000","last_change":"2024-05-08T23:45:04.054371+0000","last_active":"2024-05-08T23:46:25.337429+0000","last_peered":"2024-05-08T23:46:25.337429+0000","last_clean":"2024-05-08T23:46:25.337429+0000","last_became_active":"2024-05-08T23:45:04.054070+0000","last_became_peered":"2024-05-08T23:45:04.054070+0000","last_unstale":"2024-05-08T23:46:25.337429+0000","last_undegraded":"2024-05-08T23:46:25.337429+0000","last_fullsized":"2024-05-08T23:46:25.337429+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:27.926522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360465+0000","last_change":"2024-05-08T23:45:03.579413+0000","last_active":"2024-05-08T23:45:56.360465+0000","last_peered":"2024-05-08T23:45:56.360465+0000","last_clean":"2024-05-08T23:45:56.360465+0000","last_became_active":"2024-05-08T23:45:03.577705+0000","last_became_peered":"2024-05-08T23:45:03.577705+0000","last_unstale":"2024-05-08T23:45:56.360465+0000","last_undegraded":"2024-05-08T23:45:56.360465+0000","last_fullsized":"2024-05-08T23:45:56.360465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:29.615247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.353876+0000","last_change":"2024-05-08T23:45:04.050492+0000","last_active":"2024-05-08T23:46:25.353876+0000","last_peered":"2024-05-08T23:46:25.353876+0000","last_clean":"2024-05-08T23:46:25.353876+0000","last_became_active":"2024-05-08T23:45:04.050176+0000","last_became_peered":"2024-05-08T23:45:04.050176+0000","last_unstale":"2024-05-08T23:46:25.353876+0000","last_undegraded":"2024-05-08T23:46:25.353876+0000","last_fullsized":"2024-05-08T23:46:25.353876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:01.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.354571+0000","last_change":"2024-05-08T23:45:03.577651+0000","last_active":"2024-05-08T23:45:56.354571+0000","last_peered":"2024-05-08T23:45:56.354571+0000","last_clean":"2024-05-08T23:45:56.354571+0000","last_became_active":"2024-05-08T23:45:03.576887+0000","last_became_peered":"2024-05-08T23:45:03.576887+0000","last_unstale":"2024-05-08T23:45:56.354571+0000","last_undegraded":"2024-05-08T23:45:56.354571+0000","last_fullsized":"2024-05-08T23:45:56.354571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:52.822959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342689+0000","last_change":"2024-05-08T23:45:04.049793+0000","last_active":"2024-05-08T23:46:25.342689+0000","last_peered":"2024-05-08T23:46:25.342689+0000","last_clean":"2024-05-08T23:46:25.342689+0000","last_became_active":"2024-05-08T23:45:04.049456+0000","last_became_peered":"2024-05-08T23:45:04.049456+0000","last_unstale":"2024-05-08T23:46:25.342689+0000","last_undegraded":"2024-05-08T23:46:25.342689+0000","last_fullsized":"2024-05-08T23:46:25.342689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:16.030406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":62,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.564007+0000","last_change":"2024-05-08T23:45:03.580371+0000","last_active":"2024-05-08T23:46:25.564007+0000","last_peered":"2024-05-08T23:46:25.564007+0000","last_clean":"2024-05-08T23:46:25.564007+0000","last_became_active":"2024-05-08T23:45:03.579394+0000","last_became_peered":"2024-05-08T23:45:03.579394+0000","last_unstale":"2024-05-08T23:46:25.564007+0000","last_undegraded":"2024-05-08T23:46:25.564007+0000","last_fullsized":"2024-05-08T23:46:25.564007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.123632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343270+0000","last_change":"2024-05-08T23:45:04.063138+0000","last_active":"2024-05-08T23:46:25.343270+0000","last_peered":"2024-05-08T23:46:25.343270+0000","last_clean":"2024-05-08T23:46:25.343270+0000","last_became_active":"2024-05-08T23:45:04.062835+0000","last_became_peered":"2024-05-08T23:45:04.062835+0000","last_unstale":"2024-05-08T23:46:25.343270+0000","last_undegraded":"2024-05-08T23:46:25.343270+0000","last_fullsized":"2024-05-08T23:46:25.343270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:49.538896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.351022+0000","last_change":"2024-05-08T23:45:03.584351+0000","last_active":"2024-05-08T23:45:56.351022+0000","last_peered":"2024-05-08T23:45:56.351022+0000","last_clean":"2024-05-08T23:45:56.351022+0000","last_became_active":"2024-05-08T23:45:03.583996+0000","last_became_peered":"2024-05-08T23:45:03.583996+0000","last_unstale":"2024-05-08T23:45:56.351022+0000","last_undegraded":"2024-05-08T23:45:56.351022+0000","last_fullsized":"2024-05-08T23:45:56.351022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:40.530921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342375+0000","last_change":"2024-05-08T23:45:04.048918+0000","last_active":"2024-05-08T23:46:25.342375+0000","last_peered":"2024-05-08T23:46:25.342375+0000","last_clean":"2024-05-08T23:46:25.342375+0000","last_became_active":"2024-05-08T23:45:04.048550+0000","last_became_peered":"2024-05-08T23:45:04.048550+0000","last_unstale":"2024-05-08T23:46:25.342375+0000","last_undegraded":"2024-05-08T23:46:25.342375+0000","last_fullsized":"2024-05-08T23:46:25.342375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:48.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342987+0000","last_change":"2024-05-08T23:45:04.050790+0000","last_active":"2024-05-08T23:46:25.342987+0000","last_peered":"2024-05-08T23:46:25.342987+0000","last_clean":"2024-05-08T23:46:25.342987+0000","last_became_active":"2024-05-08T23:45:04.050429+0000","last_became_peered":"2024-05-08T23:45:04.050429+0000","last_unstale":"2024-05-08T23:46:25.342987+0000","last_undegraded":"2024-05-08T23:46:25.342987+0000","last_fullsized":"2024-05-08T23:46:25.342987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:43.224351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343805+0000","last_change":"2024-05-08T23:45:56.361737+0000","last_active":"2024-05-08T23:46:25.343805+0000","last_peered":"2024-05-08T23:46:25.343805+0000","last_clean":"2024-05-08T23:46:25.343805+0000","last_became_active":"2024-05-08T23:45:56.361305+0000","last_became_peered":"2024-05-08T23:45:56.361305+0000","last_unstale":"2024-05-08T23:46:25.343805+0000","last_undegraded":"2024-05-08T23:46:25.343805+0000","last_fullsized":"2024-05-08T23:46:25.343805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:55.163729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333222+0000","last_change":"2024-05-08T23:45:04.049707+0000","last_active":"2024-05-08T23:46:25.333222+0000","last_peered":"2024-05-08T23:46:25.333222+0000","last_clean":"2024-05-08T23:46:25.333222+0000","last_became_active":"2024-05-08T23:45:04.049385+0000","last_became_peered":"2024-05-08T23:45:04.049385+0000","last_unstale":"2024-05-08T23:46:25.333222+0000","last_undegraded":"2024-05-08T23:46:25.333222+0000","last_fullsized":"2024-05-08T23:46:25.333222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350792+0000","last_change":"2024-05-08T23:45:04.050721+0000","last_active":"2024-05-08T23:46:25.350792+0000","last_peered":"2024-05-08T23:46:25.350792+0000","last_clean":"2024-05-08T23:46:25.350792+0000","last_became_active":"2024-05-08T23:45:04.050368+0000","last_became_peered":"2024-05-08T23:45:04.050368+0000","last_unstale":"2024-05-08T23:46:25.350792+0000","last_undegraded":"2024-05-08T23:46:25.350792+0000","last_fullsized":"2024-05-08T23:46:25.350792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:20.815754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.352647+0000","last_change":"2024-05-08T23:45:04.048431+0000","last_active":"2024-05-08T23:46:25.352647+0000","last_peered":"2024-05-08T23:46:25.352647+0000","last_clean":"2024-05-08T23:46:25.352647+0000","last_became_active":"2024-05-08T23:45:04.048071+0000","last_became_peered":"2024-05-08T23:45:04.048071+0000","last_unstale":"2024-05-08T23:46:25.352647+0000","last_undegraded":"2024-05-08T23:46:25.352647+0000","last_fullsized":"2024-05-08T23:46:25.352647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:51.547256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342668+0000","last_change":"2024-05-08T23:45:04.045256+0000","last_active":"2024-05-08T23:46:25.342668+0000","last_peered":"2024-05-08T23:46:25.342668+0000","last_clean":"2024-05-08T23:46:25.342668+0000","last_became_active":"2024-05-08T23:45:04.044973+0000","last_became_peered":"2024-05-08T23:45:04.044973+0000","last_unstale":"2024-05-08T23:46:25.342668+0000","last_undegraded":"2024-05-08T23:46:25.342668+0000","last_fullsized":"2024-05-08T23:46:25.342668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:06.021001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342519+0000","last_change":"2024-05-08T23:45:04.058287+0000","last_active":"2024-05-08T23:46:25.342519+0000","last_peered":"2024-05-08T23:46:25.342519+0000","last_clean":"2024-05-08T23:46:25.342519+0000","last_became_active":"2024-05-08T23:45:04.057977+0000","last_became_peered":"2024-05-08T23:45:04.057977+0000","last_unstale":"2024-05-08T23:46:25.342519+0000","last_undegraded":"2024-05-08T23:46:25.342519+0000","last_fullsized":"2024-05-08T23:46:25.342519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:01.146838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333669+0000","last_change":"2024-05-08T23:45:04.055780+0000","last_active":"2024-05-08T23:46:25.333669+0000","last_peered":"2024-05-08T23:46:25.333669+0000","last_clean":"2024-05-08T23:46:25.333669+0000","last_became_active":"2024-05-08T23:45:04.055272+0000","last_became_peered":"2024-05-08T23:45:04.055272+0000","last_unstale":"2024-05-08T23:46:25.333669+0000","last_undegraded":"2024-05-08T23:46:25.333669+0000","last_fullsized":"2024-05-08T23:46:25.333669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:28.587333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343971+0000","last_change":"2024-05-08T23:45:04.050648+0000","last_active":"2024-05-08T23:46:25.343971+0000","last_peered":"2024-05-08T23:46:25.343971+0000","last_clean":"2024-05-08T23:46:25.343971+0000","last_became_active":"2024-05-08T23:45:04.050353+0000","last_became_peered":"2024-05-08T23:45:04.050353+0000","last_unstale":"2024-05-08T23:46:25.343971+0000","last_undegraded":"2024-05-08T23:46:25.343971+0000","last_fullsized":"2024-05-08T23:46:25.343971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:58.154006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.339378+0000","last_change":"2024-05-08T23:45:04.056951+0000","last_active":"2024-05-08T23:46:25.339378+0000","last_peered":"2024-05-08T23:46:25.339378+0000","last_clean":"2024-05-08T23:46:25.339378+0000","last_became_active":"2024-05-08T23:45:04.056585+0000","last_became_peered":"2024-05-08T23:45:04.056585+0000","last_unstale":"2024-05-08T23:46:25.339378+0000","last_undegraded":"2024-05-08T23:46:25.339378+0000","last_fullsized":"2024-05-08T23:46:25.339378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:22.658629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340138+0000","last_change":"2024-05-08T23:45:04.052396+0000","last_active":"2024-05-08T23:46:25.340138+0000","last_peered":"2024-05-08T23:46:25.340138+0000","last_clean":"2024-05-08T23:46:25.340138+0000","last_became_active":"2024-05-08T23:45:04.051604+0000","last_became_peered":"2024-05-08T23:45:04.051604+0000","last_unstale":"2024-05-08T23:46:25.340138+0000","last_undegraded":"2024-05-08T23:46:25.340138+0000","last_fullsized":"2024-05-08T23:46:25.340138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:52.602830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.352889+0000","last_change":"2024-05-08T23:45:04.051578+0000","last_active":"2024-05-08T23:46:25.352889+0000","last_peered":"2024-05-08T23:46:25.352889+0000","last_clean":"2024-05-08T23:46:25.352889+0000","last_became_active":"2024-05-08T23:45:04.051298+0000","last_became_peered":"2024-05-08T23:45:04.051298+0000","last_unstale":"2024-05-08T23:46:25.352889+0000","last_undegraded":"2024-05-08T23:46:25.352889+0000","last_fullsized":"2024-05-08T23:46:25.352889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:47.204849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.346483+0000","last_change":"2024-05-08T23:45:04.051264+0000","last_active":"2024-05-08T23:46:25.346483+0000","last_peered":"2024-05-08T23:46:25.346483+0000","last_clean":"2024-05-08T23:46:25.346483+0000","last_became_active":"2024-05-08T23:45:04.050870+0000","last_became_peered":"2024-05-08T23:45:04.050870+0000","last_unstale":"2024-05-08T23:46:25.346483+0000","last_undegraded":"2024-05-08T23:46:25.346483+0000","last_fullsized":"2024-05-08T23:46:25.346483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":594119,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":587,"num_read_kb":399,"num_write":267,"num_write_kb":1424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2015232,"data_stored":1782357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514522,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":925696,"data_stored":447350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.497},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48099999999999998}]},{"osd":1,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.44},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.63400000000000001}]},{"osd":7,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.628},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.61699999999999999}]},{"osd":8,"last update":"Wed May 8 23:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66700000000000004}]},{"osd":9,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.73999999999999999}]},{"osd":10,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.64100000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743453,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1748992,"data_stored":1235749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.49099999999999999}]},{"osd":1,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.439,"15min":0.442},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66400000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.33000000000000002}]},{"osd":2,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.80600000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78500000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.45900000000000002}]},{"osd":3,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.46999999999999997}]},{"osd":4,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.439},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.40100000000000002}]},{"osd":5,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80900000000000005,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.505}]},{"osd":6,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.69499999999999995,"15min":0.71199999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79500000000000004,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81200000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88800000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92800000000000005,"5min":1.236,"15min":1.236},"last":0.61099999999999999}]},{"osd":9,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.78100000000000003,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88200000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89200000000000002,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.63900000000000001}]},{"osd":11,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.629}]}]},{"osd":9,"up_from":64,"seq":274877906977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28316,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716836,"statfs":{"total":95995035648,"available":95966040064,"internally_reserved":0,"allocated":1327104,"data_stored":840746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63700000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.70899999999999996}]},{"osd":3,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55800000000000005,"15min":0.56599999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.66400000000000003}]},{"osd":4,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93700000000000006,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84999999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.91800000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.748},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.502,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":1.212,"15min":1.212},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.74399999999999999,"15min":0.751},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.012,"5min":1.03,"15min":1.03},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76900000000000002,"15min":0.77600000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.999,"5min":1.117,"15min":1.117},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0089999999999999,"5min":1.139,"15min":1.139},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.624,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95199999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.878}]},{"osd":10,"last update":"Wed May 8 23:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77100000000000002,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.63,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.98699999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.92900000000000005}]}]},{"osd":8,"up_from":56,"seq":240518168613,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32404,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712748,"statfs":{"total":95995035648,"available":95961853952,"internally_reserved":0,"allocated":1318912,"data_stored":840313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.63100000000000001,"15min":0.59599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.2489999999999997,"5min":6.2489999999999997,"15min":6.2489999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.3869999999999996,"5min":6.3869999999999996,"15min":6.3869999999999996},"last":0.39900000000000002}]},{"osd":1,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52400000000000002}]},{"osd":2,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.45200000000000001,"15min":0.44500000000000001},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.65300000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38100000000000001}]},{"osd":3,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.593,"5min":6.593,"15min":6.593},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.63900000000000001,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.5039999999999996,"5min":6.5039999999999996,"15min":6.5039999999999996},"last":0.65600000000000003}]},{"osd":4,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.626,"15min":0.622},"min":{"1min":0.45700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.495}]},{"osd":5,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72399999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.46700000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.52200000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.55200000000000005}]},{"osd":7,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93700000000000006,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60199999999999998}]},{"osd":9,"last update":"Wed May 8 23:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.70199999999999996,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.73099999999999998,"15min":0.70999999999999996},"min":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.59299999999999997}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.72799999999999998,"15min":0.71899999999999997},"min":{"1min":0.52300000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61599999999999999}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.627}]}]},{"osd":7,"up_from":49,"seq":210453397544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1523712,"data_stored":1039269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.437,"15min":0.44500000000000001},"min":{"1min":0.27300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.51600000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.30099999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.54400000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.753,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.432},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76700000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45400000000000001}]},{"osd":3,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.59899999999999998},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71599999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83299999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.754,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.501,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.64500000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.34300000000000003}]},{"osd":5,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.624,"15min":0.628},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76000000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80000000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.69399999999999995,"15min":0.70799999999999996},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.83599999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77100000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.48599999999999999}]},{"osd":8,"last update":"Wed May 8 23:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.502},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.89600000000000002,"5min":1.016,"15min":1.016},"last":0.58399999999999996}]},{"osd":9,"last update":"Wed May 8 23:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.58799999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.91700000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.48599999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.89500000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.57599999999999996}]},{"osd":10,"last update":"Wed May 8 23:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.501,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.78400000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.70199999999999996,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81899999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.52500000000000002}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63}]}]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":446919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65000000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.438,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68999999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41299999999999998}]},{"osd":1,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.65600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88200000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90000000000000002,"5min":1.288,"15min":1.288},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":1.018,"15min":1.018},"last":0.498},{"interface":"front","average":{"1min":0.503,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81899999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.376}]},{"osd":4,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":1.113,"15min":1.113},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98399999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.49299999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96499999999999997,"5min":1.002,"15min":1.002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82099999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed May 8 23:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.88800000000000001,"5min":1.321,"15min":1.321},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.52000000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.74099999999999999}]},{"osd":8,"last update":"Wed May 8 23:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.111,"15min":1.111},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.02,"5min":1.05,"15min":1.05},"last":0.60599999999999998}]},{"osd":9,"last update":"Wed May 8 23:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.85499999999999998,"15min":0.872},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0940000000000001,"5min":1.095,"15min":1.095},"last":0.67300000000000004}]},{"osd":10,"last update":"Wed May 8 23:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.78600000000000003},"min":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.75},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.107,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.78500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":978944,"data_stored":510368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.55600000000000005,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.96,"5min":3.96,"15min":3.96},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.58099999999999996,"15min":0.51900000000000002},"min":{"1min":0.44400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed May 8 23:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed May 8 23:46:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.60099999999999998,"15min":0.54900000000000004},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.101,"5min":4.101,"15min":4.101},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.63900000000000001,"15min":0.55700000000000005},"min":{"1min":0.51400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.1820000000000004,"5min":4.1820000000000004,"15min":4.1820000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed May 8 23:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41799999999999998,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.84399999999999997,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.37},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.40699999999999997},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68400000000000005,"5min":0.88,"15min":0.88},"last":0.32200000000000001}]},{"osd":5,"last update":"Wed May 8 23:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.57299999999999995,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.68300000000000005,"15min":0.65700000000000003},"min":{"1min":0.49199999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.7549999999999999,"5min":4.7549999999999999,"15min":4.7549999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.175,"5min":0.747,"15min":0.66700000000000004},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Wed May 8 23:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55300000000000005,"15min":0.51800000000000002},"min":{"1min":0.39800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92900000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60299999999999998,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54500000000000004}]},{"osd":8,"last update":"Wed May 8 23:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52900000000000003}]},{"osd":9,"last update":"Wed May 8 23:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.9240000000000004,"5min":4.9240000000000004,"15min":4.9240000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.81999999999999995,"15min":0.78000000000000003},"min":{"1min":0.57499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":6.3769999999999998,"5min":6.3769999999999998,"15min":6.3769999999999998},"last":0.64900000000000002}]},{"osd":10,"last update":"Wed May 8 23:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0589999999999999,"5min":1.095,"15min":1.095},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.46800000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96499999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59699999999999998}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.626},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":447823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.54800000000000004,"15min":0.50600000000000001},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.36299999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":1.032,"15min":1.032},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed May 8 23:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56699999999999995,"15min":0.52700000000000002},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.60599999999999998,"15min":0.54800000000000004},"min":{"1min":0.44700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.625},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":3.1099999999999999,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed May 8 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59099999999999997,"15min":0.56200000000000006},"min":{"1min":0.51100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.65500000000000003,"15min":0.621},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed May 8 23:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.66300000000000003,"15min":0.627},"min":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Wed May 8 23:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.64200000000000002,"15min":0.58999999999999997},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":6.0949999999999998,"5min":6.0949999999999998,"15min":6.0949999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.70599999999999996,"15min":0.65100000000000002},"min":{"1min":0.39500000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":6.1989999999999998,"5min":6.1989999999999998,"15min":6.1989999999999998},"last":0.746}]},{"osd":7,"last update":"Wed May 8 23:44:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80400000000000005,"15min":0.80100000000000005},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.47299999999999998}]},{"osd":9,"last update":"Wed May 8 23:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.46700000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98099999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83099999999999996,"15min":0.83299999999999996},"min":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.78900000000000003}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.82799999999999996},"min":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.96699999999999997,"5min":1.127,"15min":1.127},"last":0.90300000000000002}]},{"osd":11,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.64100000000000001}]}]},{"osd":2,"up_from":17,"seq":73014444091,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":999424,"data_stored":512422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82799999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54400000000000004,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.70099999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.52600000000000002,"15min":0.48999999999999999},"min":{"1min":0.40799999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83199999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.5},"min":{"1min":0.38300000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.80600000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64400000000000002}]},{"osd":3,"last update":"Wed May 8 23:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55300000000000005,"15min":0.52600000000000002},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54600000000000004,"15min":0.52400000000000002},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68799999999999994}]},{"osd":5,"last update":"Wed May 8 23:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.45500000000000002},"min":{"1min":0.34499999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.53200000000000003,"15min":0.51100000000000001},"min":{"1min":0.45400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed May 8 23:46:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64100000000000001,"15min":0.629},"min":{"1min":0.51200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.77900000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.46899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90400000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.42899999999999999}]},{"osd":9,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.627},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.63100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63600000000000001}]},{"osd":10,"last update":"Wed May 8 23:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.51500000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85699999999999998,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.92200000000000004,"15min":0.95799999999999996},"min":{"1min":0.58599999999999997,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93899999999999995,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.69699999999999995}]},{"osd":11,"last update":"Wed May 8 23:46:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.66400000000000003}]}]},{"osd":3,"up_from":24,"seq":103079215159,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":921600,"data_stored":438275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.443,"15min":0.39300000000000002},"min":{"1min":0.312,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.4790000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.434,"15min":0.39800000000000002},"min":{"1min":0.309,"5min":0.248,"15min":0.248},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.497}]},{"osd":1,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.57899999999999996,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.48299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.5459999999999998,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":0.72399999999999998}]},{"osd":2,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.65100000000000002,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.64800000000000002,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed May 8 23:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.60799999999999998,"15min":0.56999999999999995},"min":{"1min":0.51100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69899999999999995}]},{"osd":5,"last update":"Wed May 8 23:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.372},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.64500000000000002,"15min":0.61099999999999999},"min":{"1min":0.53000000000000003,"5min":0.371,"15min":0.371},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.441}]},{"osd":6,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.59999999999999998,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.0950000000000002,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":0.41899999999999998}]},{"osd":7,"last update":"Wed May 8 23:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70599999999999996,"15min":0.66000000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9119999999999999,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.745,"15min":0.71699999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.63500000000000001}]},{"osd":8,"last update":"Wed May 8 23:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed May 8 23:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.76500000000000001,"15min":0.748},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.431,"5min":3.431,"15min":3.431},"last":0.56299999999999994}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.84699999999999998,"15min":0.81899999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.874,"15min":0.86399999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.67600000000000005}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.617,"5min":3.617,"15min":3.617},"last":0.624}]}]},{"osd":4,"up_from":31,"seq":133143986225,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":447375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61799999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.377},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.67700000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.307}]},{"osd":2,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.40699999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81999999999999995,"5min":1.222,"15min":1.222},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.62}]},{"osd":3,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.438},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84899999999999998,"5min":0.874,"15min":0.874},"last":0.41699999999999998}]},{"osd":5,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80900000000000005,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78900000000000003,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48099999999999998}]},{"osd":7,"last update":"Wed May 8 23:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.82499999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53200000000000003,"15min":0.51700000000000002},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76700000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.495}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51000000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85499999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91100000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.53000000000000003}]},{"osd":9,"last update":"Wed May 8 23:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.71099999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90600000000000003,"5min":0.997,"15min":0.997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92100000000000004,"5min":1.115,"15min":1.115},"last":0.64600000000000002}]},{"osd":10,"last update":"Wed May 8 23:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89400000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.48699999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76400000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55800000000000005}]},{"osd":11,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60399999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1544192,"data_stored":1039137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.51200000000000001,"15min":0.47899999999999998},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80100000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.436},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.34699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.50600000000000001},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70699999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.68100000000000005,"5min":0.873,"15min":0.873},"last":0.57699999999999996}]},{"osd":2,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.372,"15min":0.35899999999999999},"min":{"1min":0.27100000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.40500000000000003},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.63400000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.34399999999999997}]},{"osd":3,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.48599999999999999},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74299999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.501},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70699999999999996}]},{"osd":4,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.55500000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.373},"max":{"1min":0.872,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89100000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.49299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82099999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.45800000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.61699999999999999}]},{"osd":7,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59899999999999998,"15min":0.58499999999999996},"min":{"1min":0.46100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63200000000000001,"15min":0.62},"min":{"1min":0.5,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91000000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67700000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85399999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46600000000000003}]},{"osd":9,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92600000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.878,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63700000000000001}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.76900000000000002},"min":{"1min":0.58999999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.88600000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.76400000000000001},"min":{"1min":0.60699999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93700000000000006,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":198919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:46:30.506 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.684+0000 7f9ac8750700 1 -- 172.21.15.12:0/4174633268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ac0101a10 msgr2=0x7f9ac0105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.684+0000 7f9ac8750700 1 --2- 172.21.15.12:0/4174633268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ac0101a10 0x7f9ac0105a50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009990 tx=0x7f9ab0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 -- 172.21.15.12:0/4174633268 shutdown_connections 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 --2- 172.21.15.12:0/4174633268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ac0106180 0x7f9ac010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 --2- 172.21.15.12:0/4174633268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ac0101060 0x7f9ac0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 --2- 172.21.15.12:0/4174633268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ac0101a10 0x7f9ac0105a50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 -- 172.21.15.12:0/4174633268 >> 172.21.15.12:0/4174633268 conn(0x7f9ac0078360 msgr2=0x7f9ac0078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 -- 172.21.15.12:0/4174633268 shutdown_connections 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 -- 172.21.15.12:0/4174633268 wait complete. 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 Processor -- start 2024-05-08T23:46:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 -- start start 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ac0101060 0x7f9ac00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.685+0000 7f9ac8750700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ac0106180 0x7f9ac00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac8750700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ac0073d20 0x7f9ac01a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac8750700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac01a5ac0 con 0x7f9ac0101060 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac8750700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac01a5c30 con 0x7f9ac0106180 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac8750700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac01a5df0 con 0x7f9ac0073d20 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac64ec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ac0101060 0x7f9ac00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac64ec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ac0101060 0x7f9ac00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48238/0 (socket says 172.21.15.12:48238) 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac64ec700 1 -- 172.21.15.12:0/2532236844 learned_addr learned my addr 172.21.15.12:0/2532236844 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac6ced700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ac0073d20 0x7f9ac01a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac5ceb700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ac0106180 0x7f9ac00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac64ec700 1 -- 172.21.15.12:0/2532236844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ac0073d20 msgr2=0x7f9ac01a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac64ec700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ac0073d20 0x7f9ac01a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac64ec700 1 -- 172.21.15.12:0/2532236844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ac0106180 msgr2=0x7f9ac00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac64ec700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ac0106180 0x7f9ac00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac64ec700 1 -- 172.21.15.12:0/2532236844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab00095f0 con 0x7f9ac0101060 2024-05-08T23:46:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac64ec700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ac0101060 0x7f9ac00732a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00d950 tx=0x7f9abc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ab77fe700 1 -- 172.21.15.12:0/2532236844 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc009b40 con 0x7f9ac0101060 2024-05-08T23:46:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ab77fe700 1 -- 172.21.15.12:0/2532236844 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9abc009ca0 con 0x7f9ac0101060 2024-05-08T23:46:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac8750700 1 -- 172.21.15.12:0/2532236844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac01a60d0 con 0x7f9ac0101060 2024-05-08T23:46:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ab77fe700 1 -- 172.21.15.12:0/2532236844 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00f620 con 0x7f9ac0101060 2024-05-08T23:46:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.686+0000 7f9ac8750700 1 -- 172.21.15.12:0/2532236844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac01a6630 con 0x7f9ac0101060 2024-05-08T23:46:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.687+0000 7f9ac8750700 1 -- 172.21.15.12:0/2532236844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac0109450 con 0x7f9ac0101060 2024-05-08T23:46:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.689+0000 7f9ab77fe700 1 -- 172.21.15.12:0/2532236844 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f9abc010460 con 0x7f9ac0101060 2024-05-08T23:46:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.689+0000 7f9ab77fe700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9aac0a13a0 0x7f9aac0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.690+0000 7f9ac5ceb700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9aac0a13a0 0x7f9aac0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.690+0000 7f9ab77fe700 1 -- 172.21.15.12:0/2532236844 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9abc0cd190 con 0x7f9ac0101060 2024-05-08T23:46:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.690+0000 7f9ac5ceb700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9aac0a13a0 0x7f9aac0a3860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b5c0 tx=0x7f9ab0005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:30.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.695+0000 7f9ab77fe700 1 -- 172.21.15.12:0/2532236844 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9abc0964b0 con 0x7f9ac0101060 2024-05-08T23:46:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:46:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:46:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: Updating smithi160:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: Updating smithi012:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1158105275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/35029108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:30.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:30 smithi204 ceph-mon[47373]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-05-08T23:46:30.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: Updating smithi160:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: Updating smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: Updating smithi012:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1158105275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/35029108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:30 smithi160 ceph-mon[46152]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-05-08T23:46:30.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.908+0000 7f9ac8750700 1 -- 172.21.15.12:0/2532236844 --> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ac0074840 con 0x7f9aac0a13a0 2024-05-08T23:46:30.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.918+0000 7f9ab77fe700 1 -- 172.21.15.12:0/2532236844 <== mgr.15252 v2:172.21.15.12:6800/2525408937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343590 (secure 0 0 0) 0x7f9ac0074840 con 0x7f9aac0a13a0 2024-05-08T23:46:30.918 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:30.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.922+0000 7f9ac8750700 1 -- 172.21.15.12:0/2532236844 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9aac0a13a0 msgr2=0x7f9aac0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.922+0000 7f9ac8750700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9aac0a13a0 0x7f9aac0a3860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b5c0 tx=0x7f9ab0005fd0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.922+0000 7f9ac8750700 1 -- 172.21.15.12:0/2532236844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ac0101060 msgr2=0x7f9ac00732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.922+0000 7f9ac8750700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ac0101060 0x7f9ac00732a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00d950 tx=0x7f9abc00dd10 comp rx=0 tx=0).stop 2024-05-08T23:46:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.922+0000 7f9ac8750700 1 -- 172.21.15.12:0/2532236844 shutdown_connections 2024-05-08T23:46:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.922+0000 7f9ac8750700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9aac0a13a0 0x7f9aac0a3860 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.922+0000 7f9ac8750700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ac0073d20 0x7f9ac01a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.922+0000 7f9ac8750700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ac0101060 0x7f9ac00732a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.922+0000 7f9ac8750700 1 --2- 172.21.15.12:0/2532236844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ac0106180 0x7f9ac00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.922+0000 7f9ac8750700 1 -- 172.21.15.12:0/2532236844 >> 172.21.15.12:0/2532236844 conn(0x7f9ac0078360 msgr2=0x7f9ac00ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.923+0000 7f9ac8750700 1 -- 172.21.15.12:0/2532236844 shutdown_connections 2024-05-08T23:46:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:30.923+0000 7f9ac8750700 1 -- 172.21.15.12:0/2532236844 wait complete. 2024-05-08T23:46:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-08T23:46:30.932 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-08T23:46:29.310190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1184721,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":723,"num_read_kb":518,"num_write":450,"num_write_kb":4364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":851,"ondisk_log_size":851,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374900,"kb_used_data":13736,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124566924,"statfs":{"total":1151940427776,"available":1151556530176,"internally_reserved":0,"allocated":14065664,"data_stored":8245746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.979738"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340326+0000","last_change":"2024-05-08T23:45:04.044088+0000","last_active":"2024-05-08T23:46:25.340326+0000","last_peered":"2024-05-08T23:46:25.340326+0000","last_clean":"2024-05-08T23:46:25.340326+0000","last_became_active":"2024-05-08T23:45:04.043535+0000","last_became_peered":"2024-05-08T23:45:04.043535+0000","last_unstale":"2024-05-08T23:46:25.340326+0000","last_undegraded":"2024-05-08T23:46:25.340326+0000","last_fullsized":"2024-05-08T23:46:25.340326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:52.363402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344567+0000","last_change":"2024-05-08T23:45:04.049201+0000","last_active":"2024-05-08T23:46:25.344567+0000","last_peered":"2024-05-08T23:46:25.344567+0000","last_clean":"2024-05-08T23:46:25.344567+0000","last_became_active":"2024-05-08T23:45:04.048743+0000","last_became_peered":"2024-05-08T23:45:04.048743+0000","last_unstale":"2024-05-08T23:46:25.344567+0000","last_undegraded":"2024-05-08T23:46:25.344567+0000","last_fullsized":"2024-05-08T23:46:25.344567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:26.191549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348268+0000","last_change":"2024-05-08T23:45:04.047356+0000","last_active":"2024-05-08T23:46:25.348268+0000","last_peered":"2024-05-08T23:46:25.348268+0000","last_clean":"2024-05-08T23:46:25.348268+0000","last_became_active":"2024-05-08T23:45:04.046996+0000","last_became_peered":"2024-05-08T23:45:04.046996+0000","last_unstale":"2024-05-08T23:46:25.348268+0000","last_undegraded":"2024-05-08T23:46:25.348268+0000","last_fullsized":"2024-05-08T23:46:25.348268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:58.722522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349109+0000","last_change":"2024-05-08T23:45:04.054754+0000","last_active":"2024-05-08T23:46:25.349109+0000","last_peered":"2024-05-08T23:46:25.349109+0000","last_clean":"2024-05-08T23:46:25.349109+0000","last_became_active":"2024-05-08T23:45:04.054302+0000","last_became_peered":"2024-05-08T23:45:04.054302+0000","last_unstale":"2024-05-08T23:46:25.349109+0000","last_undegraded":"2024-05-08T23:46:25.349109+0000","last_fullsized":"2024-05-08T23:46:25.349109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:28.882982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354073+0000","last_change":"2024-05-08T23:45:04.047189+0000","last_active":"2024-05-08T23:46:25.354073+0000","last_peered":"2024-05-08T23:46:25.354073+0000","last_clean":"2024-05-08T23:46:25.354073+0000","last_became_active":"2024-05-08T23:45:04.046756+0000","last_became_peered":"2024-05-08T23:45:04.046756+0000","last_unstale":"2024-05-08T23:46:25.354073+0000","last_undegraded":"2024-05-08T23:46:25.354073+0000","last_fullsized":"2024-05-08T23:46:25.354073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.780169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.338455+0000","last_change":"2024-05-08T23:45:04.060897+0000","last_active":"2024-05-08T23:46:25.338455+0000","last_peered":"2024-05-08T23:46:25.338455+0000","last_clean":"2024-05-08T23:46:25.338455+0000","last_became_active":"2024-05-08T23:45:04.060550+0000","last_became_peered":"2024-05-08T23:45:04.060550+0000","last_unstale":"2024-05-08T23:46:25.338455+0000","last_undegraded":"2024-05-08T23:46:25.338455+0000","last_fullsized":"2024-05-08T23:46:25.338455+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:59.356057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343816+0000","last_change":"2024-05-08T23:45:04.059308+0000","last_active":"2024-05-08T23:46:25.343816+0000","last_peered":"2024-05-08T23:46:25.343816+0000","last_clean":"2024-05-08T23:46:25.343816+0000","last_became_active":"2024-05-08T23:45:04.058920+0000","last_became_peered":"2024-05-08T23:45:04.058920+0000","last_unstale":"2024-05-08T23:46:25.343816+0000","last_undegraded":"2024-05-08T23:46:25.343816+0000","last_fullsized":"2024-05-08T23:46:25.343816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:11.857062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342864+0000","last_change":"2024-05-08T23:45:56.359452+0000","last_active":"2024-05-08T23:46:25.342864+0000","last_peered":"2024-05-08T23:46:25.342864+0000","last_clean":"2024-05-08T23:46:25.342864+0000","last_became_active":"2024-05-08T23:45:56.359066+0000","last_became_peered":"2024-05-08T23:45:56.359066+0000","last_unstale":"2024-05-08T23:46:25.342864+0000","last_undegraded":"2024-05-08T23:46:25.342864+0000","last_fullsized":"2024-05-08T23:46:25.342864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:13.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344525+0000","last_change":"2024-05-08T23:45:04.052292+0000","last_active":"2024-05-08T23:46:25.344525+0000","last_peered":"2024-05-08T23:46:25.344525+0000","last_clean":"2024-05-08T23:46:25.344525+0000","last_became_active":"2024-05-08T23:45:04.051164+0000","last_became_peered":"2024-05-08T23:45:04.051164+0000","last_unstale":"2024-05-08T23:46:25.344525+0000","last_undegraded":"2024-05-08T23:46:25.344525+0000","last_fullsized":"2024-05-08T23:46:25.344525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:09.218130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348445+0000","last_change":"2024-05-08T23:45:04.054566+0000","last_active":"2024-05-08T23:46:25.348445+0000","last_peered":"2024-05-08T23:46:25.348445+0000","last_clean":"2024-05-08T23:46:25.348445+0000","last_became_active":"2024-05-08T23:45:04.053878+0000","last_became_peered":"2024-05-08T23:45:04.053878+0000","last_unstale":"2024-05-08T23:46:25.348445+0000","last_undegraded":"2024-05-08T23:46:25.348445+0000","last_fullsized":"2024-05-08T23:46:25.348445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:34.760038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342587+0000","last_change":"2024-05-08T23:45:04.049696+0000","last_active":"2024-05-08T23:46:25.342587+0000","last_peered":"2024-05-08T23:46:25.342587+0000","last_clean":"2024-05-08T23:46:25.342587+0000","last_became_active":"2024-05-08T23:45:04.049351+0000","last_became_peered":"2024-05-08T23:45:04.049351+0000","last_unstale":"2024-05-08T23:46:25.342587+0000","last_undegraded":"2024-05-08T23:46:25.342587+0000","last_fullsized":"2024-05-08T23:46:25.342587+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:53.557738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348957+0000","last_change":"2024-05-08T23:45:04.051010+0000","last_active":"2024-05-08T23:46:25.348957+0000","last_peered":"2024-05-08T23:46:25.348957+0000","last_clean":"2024-05-08T23:46:25.348957+0000","last_became_active":"2024-05-08T23:45:04.050715+0000","last_became_peered":"2024-05-08T23:45:04.050715+0000","last_unstale":"2024-05-08T23:46:25.348957+0000","last_undegraded":"2024-05-08T23:46:25.348957+0000","last_fullsized":"2024-05-08T23:46:25.348957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:09.305974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.353741+0000","last_change":"2024-05-08T23:45:04.051005+0000","last_active":"2024-05-08T23:46:25.353741+0000","last_peered":"2024-05-08T23:46:25.353741+0000","last_clean":"2024-05-08T23:46:25.353741+0000","last_became_active":"2024-05-08T23:45:04.050123+0000","last_became_peered":"2024-05-08T23:45:04.050123+0000","last_unstale":"2024-05-08T23:46:25.353741+0000","last_undegraded":"2024-05-08T23:46:25.353741+0000","last_fullsized":"2024-05-08T23:46:25.353741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:52.206389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348724+0000","last_change":"2024-05-08T23:45:04.049942+0000","last_active":"2024-05-08T23:46:25.348724+0000","last_peered":"2024-05-08T23:46:25.348724+0000","last_clean":"2024-05-08T23:46:25.348724+0000","last_became_active":"2024-05-08T23:45:04.049596+0000","last_became_peered":"2024-05-08T23:45:04.049596+0000","last_unstale":"2024-05-08T23:46:25.348724+0000","last_undegraded":"2024-05-08T23:46:25.348724+0000","last_fullsized":"2024-05-08T23:46:25.348724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:30.907023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350562+0000","last_change":"2024-05-08T23:45:04.048985+0000","last_active":"2024-05-08T23:46:25.350562+0000","last_peered":"2024-05-08T23:46:25.350562+0000","last_clean":"2024-05-08T23:46:25.350562+0000","last_became_active":"2024-05-08T23:45:04.048658+0000","last_became_peered":"2024-05-08T23:45:04.048658+0000","last_unstale":"2024-05-08T23:46:25.350562+0000","last_undegraded":"2024-05-08T23:46:25.350562+0000","last_fullsized":"2024-05-08T23:46:25.350562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:17.030930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.338683+0000","last_change":"2024-05-08T23:45:04.051903+0000","last_active":"2024-05-08T23:46:25.338683+0000","last_peered":"2024-05-08T23:46:25.338683+0000","last_clean":"2024-05-08T23:46:25.338683+0000","last_became_active":"2024-05-08T23:45:04.051221+0000","last_became_peered":"2024-05-08T23:45:04.051221+0000","last_unstale":"2024-05-08T23:46:25.338683+0000","last_undegraded":"2024-05-08T23:46:25.338683+0000","last_fullsized":"2024-05-08T23:46:25.338683+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:01.554463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340534+0000","last_change":"2024-05-08T23:45:04.043854+0000","last_active":"2024-05-08T23:46:25.340534+0000","last_peered":"2024-05-08T23:46:25.340534+0000","last_clean":"2024-05-08T23:46:25.340534+0000","last_became_active":"2024-05-08T23:45:04.043056+0000","last_became_peered":"2024-05-08T23:45:04.043056+0000","last_unstale":"2024-05-08T23:46:25.340534+0000","last_undegraded":"2024-05-08T23:46:25.340534+0000","last_fullsized":"2024-05-08T23:46:25.340534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:18.175575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.338204+0000","last_change":"2024-05-08T23:45:04.049332+0000","last_active":"2024-05-08T23:46:25.338204+0000","last_peered":"2024-05-08T23:46:25.338204+0000","last_clean":"2024-05-08T23:46:25.338204+0000","last_became_active":"2024-05-08T23:45:04.048987+0000","last_became_peered":"2024-05-08T23:45:04.048987+0000","last_unstale":"2024-05-08T23:46:25.338204+0000","last_undegraded":"2024-05-08T23:46:25.338204+0000","last_fullsized":"2024-05-08T23:46:25.338204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:02.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333688+0000","last_change":"2024-05-08T23:45:04.048536+0000","last_active":"2024-05-08T23:46:25.333688+0000","last_peered":"2024-05-08T23:46:25.333688+0000","last_clean":"2024-05-08T23:46:25.333688+0000","last_became_active":"2024-05-08T23:45:04.048102+0000","last_became_peered":"2024-05-08T23:45:04.048102+0000","last_unstale":"2024-05-08T23:46:25.333688+0000","last_undegraded":"2024-05-08T23:46:25.333688+0000","last_fullsized":"2024-05-08T23:46:25.333688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:11.794372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.353062+0000","last_change":"2024-05-08T23:45:03.576833+0000","last_active":"2024-05-08T23:45:56.353062+0000","last_peered":"2024-05-08T23:45:56.353062+0000","last_clean":"2024-05-08T23:45:56.353062+0000","last_became_active":"2024-05-08T23:45:03.576495+0000","last_became_peered":"2024-05-08T23:45:03.576495+0000","last_unstale":"2024-05-08T23:45:56.353062+0000","last_undegraded":"2024-05-08T23:45:56.353062+0000","last_fullsized":"2024-05-08T23:45:56.353062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:50.266658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340176+0000","last_change":"2024-05-08T23:45:04.055444+0000","last_active":"2024-05-08T23:46:25.340176+0000","last_peered":"2024-05-08T23:46:25.340176+0000","last_clean":"2024-05-08T23:46:25.340176+0000","last_became_active":"2024-05-08T23:45:04.054998+0000","last_became_peered":"2024-05-08T23:45:04.054998+0000","last_unstale":"2024-05-08T23:46:25.340176+0000","last_undegraded":"2024-05-08T23:46:25.340176+0000","last_fullsized":"2024-05-08T23:46:25.340176+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:03.470555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.359775+0000","last_change":"2024-05-08T23:45:03.578362+0000","last_active":"2024-05-08T23:45:56.359775+0000","last_peered":"2024-05-08T23:45:56.359775+0000","last_clean":"2024-05-08T23:45:56.359775+0000","last_became_active":"2024-05-08T23:45:03.577834+0000","last_became_peered":"2024-05-08T23:45:03.577834+0000","last_unstale":"2024-05-08T23:45:56.359775+0000","last_undegraded":"2024-05-08T23:45:56.359775+0000","last_fullsized":"2024-05-08T23:45:56.359775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:01.943490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.346351+0000","last_change":"2024-05-08T23:45:04.047205+0000","last_active":"2024-05-08T23:46:25.346351+0000","last_peered":"2024-05-08T23:46:25.346351+0000","last_clean":"2024-05-08T23:46:25.346351+0000","last_became_active":"2024-05-08T23:45:04.046860+0000","last_became_peered":"2024-05-08T23:45:04.046860+0000","last_unstale":"2024-05-08T23:46:25.346351+0000","last_undegraded":"2024-05-08T23:46:25.346351+0000","last_fullsized":"2024-05-08T23:46:25.346351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:36.837101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.361012+0000","last_change":"2024-05-08T23:45:03.582026+0000","last_active":"2024-05-08T23:45:56.361012+0000","last_peered":"2024-05-08T23:45:56.361012+0000","last_clean":"2024-05-08T23:45:56.361012+0000","last_became_active":"2024-05-08T23:45:03.581672+0000","last_became_peered":"2024-05-08T23:45:03.581672+0000","last_unstale":"2024-05-08T23:45:56.361012+0000","last_undegraded":"2024-05-08T23:45:56.361012+0000","last_fullsized":"2024-05-08T23:45:56.361012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:50.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349108+0000","last_change":"2024-05-08T23:45:04.052639+0000","last_active":"2024-05-08T23:46:25.349108+0000","last_peered":"2024-05-08T23:46:25.349108+0000","last_clean":"2024-05-08T23:46:25.349108+0000","last_became_active":"2024-05-08T23:45:04.052158+0000","last_became_peered":"2024-05-08T23:45:04.052158+0000","last_unstale":"2024-05-08T23:46:25.349108+0000","last_undegraded":"2024-05-08T23:46:25.349108+0000","last_fullsized":"2024-05-08T23:46:25.349108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:25.638555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360530+0000","last_change":"2024-05-08T23:45:03.579221+0000","last_active":"2024-05-08T23:45:56.360530+0000","last_peered":"2024-05-08T23:45:56.360530+0000","last_clean":"2024-05-08T23:45:56.360530+0000","last_became_active":"2024-05-08T23:45:03.578795+0000","last_became_peered":"2024-05-08T23:45:03.578795+0000","last_unstale":"2024-05-08T23:45:56.360530+0000","last_undegraded":"2024-05-08T23:45:56.360530+0000","last_fullsized":"2024-05-08T23:45:56.360530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:14.064108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350032+0000","last_change":"2024-05-08T23:45:04.057573+0000","last_active":"2024-05-08T23:46:25.350032+0000","last_peered":"2024-05-08T23:46:25.350032+0000","last_clean":"2024-05-08T23:46:25.350032+0000","last_became_active":"2024-05-08T23:45:04.053201+0000","last_became_peered":"2024-05-08T23:45:04.053201+0000","last_unstale":"2024-05-08T23:46:25.350032+0000","last_undegraded":"2024-05-08T23:46:25.350032+0000","last_fullsized":"2024-05-08T23:46:25.350032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:23.612609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.313218+0000","last_change":"2024-05-08T23:45:03.578110+0000","last_active":"2024-05-08T23:46:25.313218+0000","last_peered":"2024-05-08T23:46:25.313218+0000","last_clean":"2024-05-08T23:46:25.313218+0000","last_became_active":"2024-05-08T23:45:03.577749+0000","last_became_peered":"2024-05-08T23:45:03.577749+0000","last_unstale":"2024-05-08T23:46:25.313218+0000","last_undegraded":"2024-05-08T23:46:25.313218+0000","last_fullsized":"2024-05-08T23:46:25.313218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:50.526535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.337443+0000","last_change":"2024-05-08T23:45:04.048131+0000","last_active":"2024-05-08T23:46:25.337443+0000","last_peered":"2024-05-08T23:46:25.337443+0000","last_clean":"2024-05-08T23:46:25.337443+0000","last_became_active":"2024-05-08T23:45:04.047773+0000","last_became_peered":"2024-05-08T23:45:04.047773+0000","last_unstale":"2024-05-08T23:46:25.337443+0000","last_undegraded":"2024-05-08T23:46:25.337443+0000","last_fullsized":"2024-05-08T23:46:25.337443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:47.944900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.353596+0000","last_change":"2024-05-08T23:45:03.586710+0000","last_active":"2024-05-08T23:45:56.353596+0000","last_peered":"2024-05-08T23:45:56.353596+0000","last_clean":"2024-05-08T23:45:56.353596+0000","last_became_active":"2024-05-08T23:45:03.586411+0000","last_became_peered":"2024-05-08T23:45:03.586411+0000","last_unstale":"2024-05-08T23:45:56.353596+0000","last_undegraded":"2024-05-08T23:45:56.353596+0000","last_fullsized":"2024-05-08T23:45:56.353596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:25.938846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349408+0000","last_change":"2024-05-08T23:45:04.053885+0000","last_active":"2024-05-08T23:46:25.349408+0000","last_peered":"2024-05-08T23:46:25.349408+0000","last_clean":"2024-05-08T23:46:25.349408+0000","last_became_active":"2024-05-08T23:45:04.053385+0000","last_became_peered":"2024-05-08T23:45:04.053385+0000","last_unstale":"2024-05-08T23:46:25.349408+0000","last_undegraded":"2024-05-08T23:46:25.349408+0000","last_fullsized":"2024-05-08T23:46:25.349408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:10.805481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.350475+0000","last_change":"2024-05-08T23:45:03.574988+0000","last_active":"2024-05-08T23:45:56.350475+0000","last_peered":"2024-05-08T23:45:56.350475+0000","last_clean":"2024-05-08T23:45:56.350475+0000","last_became_active":"2024-05-08T23:45:03.574586+0000","last_became_peered":"2024-05-08T23:45:03.574586+0000","last_unstale":"2024-05-08T23:45:56.350475+0000","last_undegraded":"2024-05-08T23:45:56.350475+0000","last_fullsized":"2024-05-08T23:45:56.350475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:18.005107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354197+0000","last_change":"2024-05-08T23:45:04.046817+0000","last_active":"2024-05-08T23:46:25.354197+0000","last_peered":"2024-05-08T23:46:25.354197+0000","last_clean":"2024-05-08T23:46:25.354197+0000","last_became_active":"2024-05-08T23:45:04.046515+0000","last_became_peered":"2024-05-08T23:45:04.046515+0000","last_unstale":"2024-05-08T23:46:25.354197+0000","last_undegraded":"2024-05-08T23:46:25.354197+0000","last_fullsized":"2024-05-08T23:46:25.354197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:29.439670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355956+0000","last_change":"2024-05-08T23:45:03.579002+0000","last_active":"2024-05-08T23:45:56.355956+0000","last_peered":"2024-05-08T23:45:56.355956+0000","last_clean":"2024-05-08T23:45:56.355956+0000","last_became_active":"2024-05-08T23:45:03.577891+0000","last_became_peered":"2024-05-08T23:45:03.577891+0000","last_unstale":"2024-05-08T23:45:56.355956+0000","last_undegraded":"2024-05-08T23:45:56.355956+0000","last_fullsized":"2024-05-08T23:45:56.355956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:01.027574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344908+0000","last_change":"2024-05-08T23:45:04.060715+0000","last_active":"2024-05-08T23:46:25.344908+0000","last_peered":"2024-05-08T23:46:25.344908+0000","last_clean":"2024-05-08T23:46:25.344908+0000","last_became_active":"2024-05-08T23:45:04.060375+0000","last_became_peered":"2024-05-08T23:45:04.060375+0000","last_unstale":"2024-05-08T23:46:25.344908+0000","last_undegraded":"2024-05-08T23:46:25.344908+0000","last_fullsized":"2024-05-08T23:46:25.344908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:42.133323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.307895+0000","last_change":"2024-05-08T23:45:03.587784+0000","last_active":"2024-05-08T23:46:25.307895+0000","last_peered":"2024-05-08T23:46:25.307895+0000","last_clean":"2024-05-08T23:46:25.307895+0000","last_became_active":"2024-05-08T23:45:03.587529+0000","last_became_peered":"2024-05-08T23:45:03.587529+0000","last_unstale":"2024-05-08T23:46:25.307895+0000","last_undegraded":"2024-05-08T23:46:25.307895+0000","last_fullsized":"2024-05-08T23:46:25.307895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:00.323599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344334+0000","last_change":"2024-05-08T23:45:04.062672+0000","last_active":"2024-05-08T23:46:25.344334+0000","last_peered":"2024-05-08T23:46:25.344334+0000","last_clean":"2024-05-08T23:46:25.344334+0000","last_became_active":"2024-05-08T23:45:04.062351+0000","last_became_peered":"2024-05-08T23:45:04.062351+0000","last_unstale":"2024-05-08T23:46:25.344334+0000","last_undegraded":"2024-05-08T23:46:25.344334+0000","last_fullsized":"2024-05-08T23:46:25.344334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:13.740239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355455+0000","last_change":"2024-05-08T23:45:03.583501+0000","last_active":"2024-05-08T23:45:56.355455+0000","last_peered":"2024-05-08T23:45:56.355455+0000","last_clean":"2024-05-08T23:45:56.355455+0000","last_became_active":"2024-05-08T23:45:03.583116+0000","last_became_peered":"2024-05-08T23:45:03.583116+0000","last_unstale":"2024-05-08T23:45:56.355455+0000","last_undegraded":"2024-05-08T23:45:56.355455+0000","last_fullsized":"2024-05-08T23:45:56.355455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:03.797524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350404+0000","last_change":"2024-05-08T23:45:04.048336+0000","last_active":"2024-05-08T23:46:25.350404+0000","last_peered":"2024-05-08T23:46:25.350404+0000","last_clean":"2024-05-08T23:46:25.350404+0000","last_became_active":"2024-05-08T23:45:04.048000+0000","last_became_peered":"2024-05-08T23:45:04.048000+0000","last_unstale":"2024-05-08T23:46:25.350404+0000","last_undegraded":"2024-05-08T23:46:25.350404+0000","last_fullsized":"2024-05-08T23:46:25.350404+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:41.806600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355898+0000","last_change":"2024-05-08T23:45:03.581215+0000","last_active":"2024-05-08T23:45:56.355898+0000","last_peered":"2024-05-08T23:45:56.355898+0000","last_clean":"2024-05-08T23:45:56.355898+0000","last_became_active":"2024-05-08T23:45:03.580496+0000","last_became_peered":"2024-05-08T23:45:03.580496+0000","last_unstale":"2024-05-08T23:45:56.355898+0000","last_undegraded":"2024-05-08T23:45:56.355898+0000","last_fullsized":"2024-05-08T23:45:56.355898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:25.602401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.346299+0000","last_change":"2024-05-08T23:45:04.058857+0000","last_active":"2024-05-08T23:46:25.346299+0000","last_peered":"2024-05-08T23:46:25.346299+0000","last_clean":"2024-05-08T23:46:25.346299+0000","last_became_active":"2024-05-08T23:45:04.057988+0000","last_became_peered":"2024-05-08T23:45:04.057988+0000","last_unstale":"2024-05-08T23:46:25.346299+0000","last_undegraded":"2024-05-08T23:46:25.346299+0000","last_fullsized":"2024-05-08T23:46:25.346299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:46.725976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:26.195804+0000","last_change":"2024-05-08T23:45:03.582121+0000","last_active":"2024-05-08T23:46:26.195804+0000","last_peered":"2024-05-08T23:46:26.195804+0000","last_clean":"2024-05-08T23:46:26.195804+0000","last_became_active":"2024-05-08T23:45:03.581796+0000","last_became_peered":"2024-05-08T23:45:03.581796+0000","last_unstale":"2024-05-08T23:46:26.195804+0000","last_undegraded":"2024-05-08T23:46:26.195804+0000","last_fullsized":"2024-05-08T23:46:26.195804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:55.417404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65082,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":145,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.330922+0000","last_change":"2024-05-08T23:45:04.052993+0000","last_active":"2024-05-08T23:46:25.330922+0000","last_peered":"2024-05-08T23:46:25.330922+0000","last_clean":"2024-05-08T23:46:25.330922+0000","last_became_active":"2024-05-08T23:45:04.052088+0000","last_became_peered":"2024-05-08T23:45:04.052088+0000","last_unstale":"2024-05-08T23:46:25.330922+0000","last_undegraded":"2024-05-08T23:46:25.330922+0000","last_fullsized":"2024-05-08T23:46:25.330922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:44.531649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:46:00.332257+0000","last_change":"2024-05-08T23:45:56.372913+0000","last_active":"2024-05-08T23:46:00.332257+0000","last_peered":"2024-05-08T23:46:00.332257+0000","last_clean":"2024-05-08T23:46:00.332257+0000","last_became_active":"2024-05-08T23:45:56.361402+0000","last_became_peered":"2024-05-08T23:45:56.361402+0000","last_unstale":"2024-05-08T23:46:00.332257+0000","last_undegraded":"2024-05-08T23:46:00.332257+0000","last_fullsized":"2024-05-08T23:46:00.332257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:34.652216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.353482+0000","last_change":"2024-05-08T23:45:04.050294+0000","last_active":"2024-05-08T23:46:25.353482+0000","last_peered":"2024-05-08T23:46:25.353482+0000","last_clean":"2024-05-08T23:46:25.353482+0000","last_became_active":"2024-05-08T23:45:04.049949+0000","last_became_peered":"2024-05-08T23:45:04.049949+0000","last_unstale":"2024-05-08T23:46:25.353482+0000","last_undegraded":"2024-05-08T23:46:25.353482+0000","last_fullsized":"2024-05-08T23:46:25.353482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:17.253201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355432+0000","last_change":"2024-05-08T23:45:03.579821+0000","last_active":"2024-05-08T23:45:56.355432+0000","last_peered":"2024-05-08T23:45:56.355432+0000","last_clean":"2024-05-08T23:45:56.355432+0000","last_became_active":"2024-05-08T23:45:03.578635+0000","last_became_peered":"2024-05-08T23:45:03.578635+0000","last_unstale":"2024-05-08T23:45:56.355432+0000","last_undegraded":"2024-05-08T23:45:56.355432+0000","last_fullsized":"2024-05-08T23:45:56.355432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:46.179505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354613+0000","last_change":"2024-05-08T23:45:04.059357+0000","last_active":"2024-05-08T23:46:25.354613+0000","last_peered":"2024-05-08T23:46:25.354613+0000","last_clean":"2024-05-08T23:46:25.354613+0000","last_became_active":"2024-05-08T23:45:04.059070+0000","last_became_peered":"2024-05-08T23:45:04.059070+0000","last_unstale":"2024-05-08T23:46:25.354613+0000","last_undegraded":"2024-05-08T23:46:25.354613+0000","last_fullsized":"2024-05-08T23:46:25.354613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:50.626787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:26.197796+0000","last_change":"2024-05-08T23:45:03.577199+0000","last_active":"2024-05-08T23:46:26.197796+0000","last_peered":"2024-05-08T23:46:26.197796+0000","last_clean":"2024-05-08T23:46:26.197796+0000","last_became_active":"2024-05-08T23:45:03.576844+0000","last_became_peered":"2024-05-08T23:45:03.576844+0000","last_unstale":"2024-05-08T23:46:26.197796+0000","last_undegraded":"2024-05-08T23:46:26.197796+0000","last_fullsized":"2024-05-08T23:46:26.197796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:27.139493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340110+0000","last_change":"2024-05-08T23:45:56.356625+0000","last_active":"2024-05-08T23:46:25.340110+0000","last_peered":"2024-05-08T23:46:25.340110+0000","last_clean":"2024-05-08T23:46:25.340110+0000","last_became_active":"2024-05-08T23:45:56.356240+0000","last_became_peered":"2024-05-08T23:45:56.356240+0000","last_unstale":"2024-05-08T23:46:25.340110+0000","last_undegraded":"2024-05-08T23:46:25.340110+0000","last_fullsized":"2024-05-08T23:46:25.340110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:50.251606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360317+0000","last_change":"2024-05-08T23:45:03.576766+0000","last_active":"2024-05-08T23:45:56.360317+0000","last_peered":"2024-05-08T23:45:56.360317+0000","last_clean":"2024-05-08T23:45:56.360317+0000","last_became_active":"2024-05-08T23:45:03.576410+0000","last_became_peered":"2024-05-08T23:45:03.576410+0000","last_unstale":"2024-05-08T23:45:56.360317+0000","last_undegraded":"2024-05-08T23:45:56.360317+0000","last_fullsized":"2024-05-08T23:45:56.360317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:29.444589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349612+0000","last_change":"2024-05-08T23:45:04.056563+0000","last_active":"2024-05-08T23:46:25.349612+0000","last_peered":"2024-05-08T23:46:25.349612+0000","last_clean":"2024-05-08T23:46:25.349612+0000","last_became_active":"2024-05-08T23:45:04.053132+0000","last_became_peered":"2024-05-08T23:45:04.053132+0000","last_unstale":"2024-05-08T23:46:25.349612+0000","last_undegraded":"2024-05-08T23:46:25.349612+0000","last_fullsized":"2024-05-08T23:46:25.349612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:41.625804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:46:22.745519+0000","last_change":"2024-05-08T23:45:03.580343+0000","last_active":"2024-05-08T23:46:22.745519+0000","last_peered":"2024-05-08T23:46:22.745519+0000","last_clean":"2024-05-08T23:46:22.745519+0000","last_became_active":"2024-05-08T23:45:03.578719+0000","last_became_peered":"2024-05-08T23:45:03.578719+0000","last_unstale":"2024-05-08T23:46:22.745519+0000","last_undegraded":"2024-05-08T23:46:22.745519+0000","last_fullsized":"2024-05-08T23:46:22.745519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:54.831252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343273+0000","last_change":"2024-05-08T23:45:04.050643+0000","last_active":"2024-05-08T23:46:25.343273+0000","last_peered":"2024-05-08T23:46:25.343273+0000","last_clean":"2024-05-08T23:46:25.343273+0000","last_became_active":"2024-05-08T23:45:04.050309+0000","last_became_peered":"2024-05-08T23:45:04.050309+0000","last_unstale":"2024-05-08T23:46:25.343273+0000","last_undegraded":"2024-05-08T23:46:25.343273+0000","last_fullsized":"2024-05-08T23:46:25.343273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:30.259530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.352954+0000","last_change":"2024-05-08T23:45:03.579590+0000","last_active":"2024-05-08T23:45:56.352954+0000","last_peered":"2024-05-08T23:45:56.352954+0000","last_clean":"2024-05-08T23:45:56.352954+0000","last_became_active":"2024-05-08T23:45:03.579213+0000","last_became_peered":"2024-05-08T23:45:03.579213+0000","last_unstale":"2024-05-08T23:45:56.352954+0000","last_undegraded":"2024-05-08T23:45:56.352954+0000","last_fullsized":"2024-05-08T23:45:56.352954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:55.666543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348872+0000","last_change":"2024-05-08T23:45:04.049082+0000","last_active":"2024-05-08T23:46:25.348872+0000","last_peered":"2024-05-08T23:46:25.348872+0000","last_clean":"2024-05-08T23:46:25.348872+0000","last_became_active":"2024-05-08T23:45:04.048709+0000","last_became_peered":"2024-05-08T23:45:04.048709+0000","last_unstale":"2024-05-08T23:46:25.348872+0000","last_undegraded":"2024-05-08T23:46:25.348872+0000","last_fullsized":"2024-05-08T23:46:25.348872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:56.365817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.354871+0000","last_change":"2024-05-08T23:45:03.581363+0000","last_active":"2024-05-08T23:45:56.354871+0000","last_peered":"2024-05-08T23:45:56.354871+0000","last_clean":"2024-05-08T23:45:56.354871+0000","last_became_active":"2024-05-08T23:45:03.580840+0000","last_became_peered":"2024-05-08T23:45:03.580840+0000","last_unstale":"2024-05-08T23:45:56.354871+0000","last_undegraded":"2024-05-08T23:45:56.354871+0000","last_fullsized":"2024-05-08T23:45:56.354871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:27.281228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.345492+0000","last_change":"2024-05-08T23:45:56.361444+0000","last_active":"2024-05-08T23:46:25.345492+0000","last_peered":"2024-05-08T23:46:25.345492+0000","last_clean":"2024-05-08T23:46:25.345492+0000","last_became_active":"2024-05-08T23:45:56.361099+0000","last_became_peered":"2024-05-08T23:45:56.361099+0000","last_unstale":"2024-05-08T23:46:25.345492+0000","last_undegraded":"2024-05-08T23:46:25.345492+0000","last_fullsized":"2024-05-08T23:46:25.345492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:25.342212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349895+0000","last_change":"2024-05-08T23:45:04.061189+0000","last_active":"2024-05-08T23:46:25.349895+0000","last_peered":"2024-05-08T23:46:25.349895+0000","last_clean":"2024-05-08T23:46:25.349895+0000","last_became_active":"2024-05-08T23:45:04.060838+0000","last_became_peered":"2024-05-08T23:45:04.060838+0000","last_unstale":"2024-05-08T23:46:25.349895+0000","last_undegraded":"2024-05-08T23:46:25.349895+0000","last_fullsized":"2024-05-08T23:46:25.349895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:43.794987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":98,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.296864+0000","last_change":"2024-05-08T23:45:03.583841+0000","last_active":"2024-05-08T23:46:25.296864+0000","last_peered":"2024-05-08T23:46:25.296864+0000","last_clean":"2024-05-08T23:46:25.296864+0000","last_became_active":"2024-05-08T23:45:03.583275+0000","last_became_peered":"2024-05-08T23:45:03.583275+0000","last_unstale":"2024-05-08T23:46:25.296864+0000","last_undegraded":"2024-05-08T23:46:25.296864+0000","last_fullsized":"2024-05-08T23:46:25.296864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:14.560505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.339715+0000","last_change":"2024-05-08T23:45:04.052667+0000","last_active":"2024-05-08T23:46:25.339715+0000","last_peered":"2024-05-08T23:46:25.339715+0000","last_clean":"2024-05-08T23:46:25.339715+0000","last_became_active":"2024-05-08T23:45:04.052008+0000","last_became_peered":"2024-05-08T23:45:04.052008+0000","last_unstale":"2024-05-08T23:46:25.339715+0000","last_undegraded":"2024-05-08T23:46:25.339715+0000","last_fullsized":"2024-05-08T23:46:25.339715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:35.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.351515+0000","last_change":"2024-05-08T23:45:03.573665+0000","last_active":"2024-05-08T23:45:56.351515+0000","last_peered":"2024-05-08T23:45:56.351515+0000","last_clean":"2024-05-08T23:45:56.351515+0000","last_became_active":"2024-05-08T23:45:03.573265+0000","last_became_peered":"2024-05-08T23:45:03.573265+0000","last_unstale":"2024-05-08T23:45:56.351515+0000","last_undegraded":"2024-05-08T23:45:56.351515+0000","last_fullsized":"2024-05-08T23:45:56.351515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:13.410954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":333,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355192+0000","last_change":"2024-05-08T23:44:16.373962+0000","last_active":"2024-05-08T23:45:56.355192+0000","last_peered":"2024-05-08T23:45:56.355192+0000","last_clean":"2024-05-08T23:45:56.355192+0000","last_became_active":"2024-05-08T23:44:16.373358+0000","last_became_peered":"2024-05-08T23:44:16.373358+0000","last_unstale":"2024-05-08T23:45:56.355192+0000","last_undegraded":"2024-05-08T23:45:56.355192+0000","last_fullsized":"2024-05-08T23:45:56.355192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.067168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:24.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349501+0000","last_change":"2024-05-08T23:45:04.048468+0000","last_active":"2024-05-08T23:46:25.349501+0000","last_peered":"2024-05-08T23:46:25.349501+0000","last_clean":"2024-05-08T23:46:25.349501+0000","last_became_active":"2024-05-08T23:45:04.048085+0000","last_became_peered":"2024-05-08T23:45:04.048085+0000","last_unstale":"2024-05-08T23:46:25.349501+0000","last_undegraded":"2024-05-08T23:46:25.349501+0000","last_fullsized":"2024-05-08T23:46:25.349501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:18.039780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355333+0000","last_change":"2024-05-08T23:45:03.575821+0000","last_active":"2024-05-08T23:45:56.355333+0000","last_peered":"2024-05-08T23:45:56.355333+0000","last_clean":"2024-05-08T23:45:56.355333+0000","last_became_active":"2024-05-08T23:45:03.575407+0000","last_became_peered":"2024-05-08T23:45:03.575407+0000","last_unstale":"2024-05-08T23:45:56.355333+0000","last_undegraded":"2024-05-08T23:45:56.355333+0000","last_fullsized":"2024-05-08T23:45:56.355333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:35.718942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:46:00.332281+0000","last_change":"2024-05-08T23:45:56.365483+0000","last_active":"2024-05-08T23:46:00.332281+0000","last_peered":"2024-05-08T23:46:00.332281+0000","last_clean":"2024-05-08T23:46:00.332281+0000","last_became_active":"2024-05-08T23:45:56.359626+0000","last_became_peered":"2024-05-08T23:45:56.359626+0000","last_unstale":"2024-05-08T23:46:00.332281+0000","last_undegraded":"2024-05-08T23:46:00.332281+0000","last_fullsized":"2024-05-08T23:46:00.332281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:38.284296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349154+0000","last_change":"2024-05-08T23:45:04.047687+0000","last_active":"2024-05-08T23:46:25.349154+0000","last_peered":"2024-05-08T23:46:25.349154+0000","last_clean":"2024-05-08T23:46:25.349154+0000","last_became_active":"2024-05-08T23:45:04.047343+0000","last_became_peered":"2024-05-08T23:45:04.047343+0000","last_unstale":"2024-05-08T23:46:25.349154+0000","last_undegraded":"2024-05-08T23:46:25.349154+0000","last_fullsized":"2024-05-08T23:46:25.349154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:58.763421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'39","reported_seq":77,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.567780+0000","last_change":"2024-05-08T23:45:03.573344+0000","last_active":"2024-05-08T23:46:25.567780+0000","last_peered":"2024-05-08T23:46:25.567780+0000","last_clean":"2024-05-08T23:46:25.567780+0000","last_became_active":"2024-05-08T23:45:03.572951+0000","last_became_peered":"2024-05-08T23:45:03.572951+0000","last_unstale":"2024-05-08T23:46:25.567780+0000","last_undegraded":"2024-05-08T23:46:25.567780+0000","last_fullsized":"2024-05-08T23:46:25.567780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.422908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":68,"num_write_kb":833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.335456+0000","last_change":"2024-05-08T23:45:04.055111+0000","last_active":"2024-05-08T23:46:25.335456+0000","last_peered":"2024-05-08T23:46:25.335456+0000","last_clean":"2024-05-08T23:46:25.335456+0000","last_became_active":"2024-05-08T23:45:04.054665+0000","last_became_peered":"2024-05-08T23:45:04.054665+0000","last_unstale":"2024-05-08T23:46:25.335456+0000","last_undegraded":"2024-05-08T23:46:25.335456+0000","last_fullsized":"2024-05-08T23:46:25.335456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:23.292630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.351606+0000","last_change":"2024-05-08T23:45:03.578168+0000","last_active":"2024-05-08T23:45:56.351606+0000","last_peered":"2024-05-08T23:45:56.351606+0000","last_clean":"2024-05-08T23:45:56.351606+0000","last_became_active":"2024-05-08T23:45:03.577833+0000","last_became_peered":"2024-05-08T23:45:03.577833+0000","last_unstale":"2024-05-08T23:45:56.351606+0000","last_undegraded":"2024-05-08T23:45:56.351606+0000","last_fullsized":"2024-05-08T23:45:56.351606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:13:25.743151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354570+0000","last_change":"2024-05-08T23:45:04.049103+0000","last_active":"2024-05-08T23:46:25.354570+0000","last_peered":"2024-05-08T23:46:25.354570+0000","last_clean":"2024-05-08T23:46:25.354570+0000","last_became_active":"2024-05-08T23:45:04.048302+0000","last_became_peered":"2024-05-08T23:45:04.048302+0000","last_unstale":"2024-05-08T23:46:25.354570+0000","last_undegraded":"2024-05-08T23:46:25.354570+0000","last_fullsized":"2024-05-08T23:46:25.354570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:02.404522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.614524+0000","last_change":"2024-05-08T23:45:03.577765+0000","last_active":"2024-05-08T23:45:56.614524+0000","last_peered":"2024-05-08T23:45:56.614524+0000","last_clean":"2024-05-08T23:45:56.614524+0000","last_became_active":"2024-05-08T23:45:03.577452+0000","last_became_peered":"2024-05-08T23:45:03.577452+0000","last_unstale":"2024-05-08T23:45:56.614524+0000","last_undegraded":"2024-05-08T23:45:56.614524+0000","last_fullsized":"2024-05-08T23:45:56.614524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:02.107878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.315831+0000","last_change":"2024-05-08T23:45:04.061360+0000","last_active":"2024-05-08T23:46:25.315831+0000","last_peered":"2024-05-08T23:46:25.315831+0000","last_clean":"2024-05-08T23:46:25.315831+0000","last_became_active":"2024-05-08T23:45:04.061000+0000","last_became_peered":"2024-05-08T23:45:04.061000+0000","last_unstale":"2024-05-08T23:46:25.315831+0000","last_undegraded":"2024-05-08T23:46:25.315831+0000","last_fullsized":"2024-05-08T23:46:25.315831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:35.757571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360024+0000","last_change":"2024-05-08T23:45:03.578769+0000","last_active":"2024-05-08T23:45:56.360024+0000","last_peered":"2024-05-08T23:45:56.360024+0000","last_clean":"2024-05-08T23:45:56.360024+0000","last_became_active":"2024-05-08T23:45:03.578359+0000","last_became_peered":"2024-05-08T23:45:03.578359+0000","last_unstale":"2024-05-08T23:45:56.360024+0000","last_undegraded":"2024-05-08T23:45:56.360024+0000","last_fullsized":"2024-05-08T23:45:56.360024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:13.213382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333169+0000","last_change":"2024-05-08T23:45:04.049237+0000","last_active":"2024-05-08T23:46:25.333169+0000","last_peered":"2024-05-08T23:46:25.333169+0000","last_clean":"2024-05-08T23:46:25.333169+0000","last_became_active":"2024-05-08T23:45:04.048922+0000","last_became_peered":"2024-05-08T23:45:04.048922+0000","last_unstale":"2024-05-08T23:46:25.333169+0000","last_undegraded":"2024-05-08T23:46:25.333169+0000","last_fullsized":"2024-05-08T23:46:25.333169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:29.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.354233+0000","last_change":"2024-05-08T23:45:03.574990+0000","last_active":"2024-05-08T23:45:56.354233+0000","last_peered":"2024-05-08T23:45:56.354233+0000","last_clean":"2024-05-08T23:45:56.354233+0000","last_became_active":"2024-05-08T23:45:03.574617+0000","last_became_peered":"2024-05-08T23:45:03.574617+0000","last_unstale":"2024-05-08T23:45:56.354233+0000","last_undegraded":"2024-05-08T23:45:56.354233+0000","last_fullsized":"2024-05-08T23:45:56.354233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:51.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.337429+0000","last_change":"2024-05-08T23:45:04.054371+0000","last_active":"2024-05-08T23:46:25.337429+0000","last_peered":"2024-05-08T23:46:25.337429+0000","last_clean":"2024-05-08T23:46:25.337429+0000","last_became_active":"2024-05-08T23:45:04.054070+0000","last_became_peered":"2024-05-08T23:45:04.054070+0000","last_unstale":"2024-05-08T23:46:25.337429+0000","last_undegraded":"2024-05-08T23:46:25.337429+0000","last_fullsized":"2024-05-08T23:46:25.337429+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:27.926522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360465+0000","last_change":"2024-05-08T23:45:03.579413+0000","last_active":"2024-05-08T23:45:56.360465+0000","last_peered":"2024-05-08T23:45:56.360465+0000","last_clean":"2024-05-08T23:45:56.360465+0000","last_became_active":"2024-05-08T23:45:03.577705+0000","last_became_peered":"2024-05-08T23:45:03.577705+0000","last_unstale":"2024-05-08T23:45:56.360465+0000","last_undegraded":"2024-05-08T23:45:56.360465+0000","last_fullsized":"2024-05-08T23:45:56.360465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:29.615247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.353876+0000","last_change":"2024-05-08T23:45:04.050492+0000","last_active":"2024-05-08T23:46:25.353876+0000","last_peered":"2024-05-08T23:46:25.353876+0000","last_clean":"2024-05-08T23:46:25.353876+0000","last_became_active":"2024-05-08T23:45:04.050176+0000","last_became_peered":"2024-05-08T23:45:04.050176+0000","last_unstale":"2024-05-08T23:46:25.353876+0000","last_undegraded":"2024-05-08T23:46:25.353876+0000","last_fullsized":"2024-05-08T23:46:25.353876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:01.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.354571+0000","last_change":"2024-05-08T23:45:03.577651+0000","last_active":"2024-05-08T23:45:56.354571+0000","last_peered":"2024-05-08T23:45:56.354571+0000","last_clean":"2024-05-08T23:45:56.354571+0000","last_became_active":"2024-05-08T23:45:03.576887+0000","last_became_peered":"2024-05-08T23:45:03.576887+0000","last_unstale":"2024-05-08T23:45:56.354571+0000","last_undegraded":"2024-05-08T23:45:56.354571+0000","last_fullsized":"2024-05-08T23:45:56.354571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:52.822959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342689+0000","last_change":"2024-05-08T23:45:04.049793+0000","last_active":"2024-05-08T23:46:25.342689+0000","last_peered":"2024-05-08T23:46:25.342689+0000","last_clean":"2024-05-08T23:46:25.342689+0000","last_became_active":"2024-05-08T23:45:04.049456+0000","last_became_peered":"2024-05-08T23:45:04.049456+0000","last_unstale":"2024-05-08T23:46:25.342689+0000","last_undegraded":"2024-05-08T23:46:25.342689+0000","last_fullsized":"2024-05-08T23:46:25.342689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:16.030406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":62,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.564007+0000","last_change":"2024-05-08T23:45:03.580371+0000","last_active":"2024-05-08T23:46:25.564007+0000","last_peered":"2024-05-08T23:46:25.564007+0000","last_clean":"2024-05-08T23:46:25.564007+0000","last_became_active":"2024-05-08T23:45:03.579394+0000","last_became_peered":"2024-05-08T23:45:03.579394+0000","last_unstale":"2024-05-08T23:46:25.564007+0000","last_undegraded":"2024-05-08T23:46:25.564007+0000","last_fullsized":"2024-05-08T23:46:25.564007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.123632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343270+0000","last_change":"2024-05-08T23:45:04.063138+0000","last_active":"2024-05-08T23:46:25.343270+0000","last_peered":"2024-05-08T23:46:25.343270+0000","last_clean":"2024-05-08T23:46:25.343270+0000","last_became_active":"2024-05-08T23:45:04.062835+0000","last_became_peered":"2024-05-08T23:45:04.062835+0000","last_unstale":"2024-05-08T23:46:25.343270+0000","last_undegraded":"2024-05-08T23:46:25.343270+0000","last_fullsized":"2024-05-08T23:46:25.343270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:49.538896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.351022+0000","last_change":"2024-05-08T23:45:03.584351+0000","last_active":"2024-05-08T23:45:56.351022+0000","last_peered":"2024-05-08T23:45:56.351022+0000","last_clean":"2024-05-08T23:45:56.351022+0000","last_became_active":"2024-05-08T23:45:03.583996+0000","last_became_peered":"2024-05-08T23:45:03.583996+0000","last_unstale":"2024-05-08T23:45:56.351022+0000","last_undegraded":"2024-05-08T23:45:56.351022+0000","last_fullsized":"2024-05-08T23:45:56.351022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:40.530921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342375+0000","last_change":"2024-05-08T23:45:04.048918+0000","last_active":"2024-05-08T23:46:25.342375+0000","last_peered":"2024-05-08T23:46:25.342375+0000","last_clean":"2024-05-08T23:46:25.342375+0000","last_became_active":"2024-05-08T23:45:04.048550+0000","last_became_peered":"2024-05-08T23:45:04.048550+0000","last_unstale":"2024-05-08T23:46:25.342375+0000","last_undegraded":"2024-05-08T23:46:25.342375+0000","last_fullsized":"2024-05-08T23:46:25.342375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:48.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342987+0000","last_change":"2024-05-08T23:45:04.050790+0000","last_active":"2024-05-08T23:46:25.342987+0000","last_peered":"2024-05-08T23:46:25.342987+0000","last_clean":"2024-05-08T23:46:25.342987+0000","last_became_active":"2024-05-08T23:45:04.050429+0000","last_became_peered":"2024-05-08T23:45:04.050429+0000","last_unstale":"2024-05-08T23:46:25.342987+0000","last_undegraded":"2024-05-08T23:46:25.342987+0000","last_fullsized":"2024-05-08T23:46:25.342987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:43.224351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343805+0000","last_change":"2024-05-08T23:45:56.361737+0000","last_active":"2024-05-08T23:46:25.343805+0000","last_peered":"2024-05-08T23:46:25.343805+0000","last_clean":"2024-05-08T23:46:25.343805+0000","last_became_active":"2024-05-08T23:45:56.361305+0000","last_became_peered":"2024-05-08T23:45:56.361305+0000","last_unstale":"2024-05-08T23:46:25.343805+0000","last_undegraded":"2024-05-08T23:46:25.343805+0000","last_fullsized":"2024-05-08T23:46:25.343805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:55.163729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333222+0000","last_change":"2024-05-08T23:45:04.049707+0000","last_active":"2024-05-08T23:46:25.333222+0000","last_peered":"2024-05-08T23:46:25.333222+0000","last_clean":"2024-05-08T23:46:25.333222+0000","last_became_active":"2024-05-08T23:45:04.049385+0000","last_became_peered":"2024-05-08T23:45:04.049385+0000","last_unstale":"2024-05-08T23:46:25.333222+0000","last_undegraded":"2024-05-08T23:46:25.333222+0000","last_fullsized":"2024-05-08T23:46:25.333222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350792+0000","last_change":"2024-05-08T23:45:04.050721+0000","last_active":"2024-05-08T23:46:25.350792+0000","last_peered":"2024-05-08T23:46:25.350792+0000","last_clean":"2024-05-08T23:46:25.350792+0000","last_became_active":"2024-05-08T23:45:04.050368+0000","last_became_peered":"2024-05-08T23:45:04.050368+0000","last_unstale":"2024-05-08T23:46:25.350792+0000","last_undegraded":"2024-05-08T23:46:25.350792+0000","last_fullsized":"2024-05-08T23:46:25.350792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:20.815754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.352647+0000","last_change":"2024-05-08T23:45:04.048431+0000","last_active":"2024-05-08T23:46:25.352647+0000","last_peered":"2024-05-08T23:46:25.352647+0000","last_clean":"2024-05-08T23:46:25.352647+0000","last_became_active":"2024-05-08T23:45:04.048071+0000","last_became_peered":"2024-05-08T23:45:04.048071+0000","last_unstale":"2024-05-08T23:46:25.352647+0000","last_undegraded":"2024-05-08T23:46:25.352647+0000","last_fullsized":"2024-05-08T23:46:25.352647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:51.547256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342668+0000","last_change":"2024-05-08T23:45:04.045256+0000","last_active":"2024-05-08T23:46:25.342668+0000","last_peered":"2024-05-08T23:46:25.342668+0000","last_clean":"2024-05-08T23:46:25.342668+0000","last_became_active":"2024-05-08T23:45:04.044973+0000","last_became_peered":"2024-05-08T23:45:04.044973+0000","last_unstale":"2024-05-08T23:46:25.342668+0000","last_undegraded":"2024-05-08T23:46:25.342668+0000","last_fullsized":"2024-05-08T23:46:25.342668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:06.021001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342519+0000","last_change":"2024-05-08T23:45:04.058287+0000","last_active":"2024-05-08T23:46:25.342519+0000","last_peered":"2024-05-08T23:46:25.342519+0000","last_clean":"2024-05-08T23:46:25.342519+0000","last_became_active":"2024-05-08T23:45:04.057977+0000","last_became_peered":"2024-05-08T23:45:04.057977+0000","last_unstale":"2024-05-08T23:46:25.342519+0000","last_undegraded":"2024-05-08T23:46:25.342519+0000","last_fullsized":"2024-05-08T23:46:25.342519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:01.146838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333669+0000","last_change":"2024-05-08T23:45:04.055780+0000","last_active":"2024-05-08T23:46:25.333669+0000","last_peered":"2024-05-08T23:46:25.333669+0000","last_clean":"2024-05-08T23:46:25.333669+0000","last_became_active":"2024-05-08T23:45:04.055272+0000","last_became_peered":"2024-05-08T23:45:04.055272+0000","last_unstale":"2024-05-08T23:46:25.333669+0000","last_undegraded":"2024-05-08T23:46:25.333669+0000","last_fullsized":"2024-05-08T23:46:25.333669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:28.587333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343971+0000","last_change":"2024-05-08T23:45:04.050648+0000","last_active":"2024-05-08T23:46:25.343971+0000","last_peered":"2024-05-08T23:46:25.343971+0000","last_clean":"2024-05-08T23:46:25.343971+0000","last_became_active":"2024-05-08T23:45:04.050353+0000","last_became_peered":"2024-05-08T23:45:04.050353+0000","last_unstale":"2024-05-08T23:46:25.343971+0000","last_undegraded":"2024-05-08T23:46:25.343971+0000","last_fullsized":"2024-05-08T23:46:25.343971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:58.154006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.339378+0000","last_change":"2024-05-08T23:45:04.056951+0000","last_active":"2024-05-08T23:46:25.339378+0000","last_peered":"2024-05-08T23:46:25.339378+0000","last_clean":"2024-05-08T23:46:25.339378+0000","last_became_active":"2024-05-08T23:45:04.056585+0000","last_became_peered":"2024-05-08T23:45:04.056585+0000","last_unstale":"2024-05-08T23:46:25.339378+0000","last_undegraded":"2024-05-08T23:46:25.339378+0000","last_fullsized":"2024-05-08T23:46:25.339378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:22.658629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340138+0000","last_change":"2024-05-08T23:45:04.052396+0000","last_active":"2024-05-08T23:46:25.340138+0000","last_peered":"2024-05-08T23:46:25.340138+0000","last_clean":"2024-05-08T23:46:25.340138+0000","last_became_active":"2024-05-08T23:45:04.051604+0000","last_became_peered":"2024-05-08T23:45:04.051604+0000","last_unstale":"2024-05-08T23:46:25.340138+0000","last_undegraded":"2024-05-08T23:46:25.340138+0000","last_fullsized":"2024-05-08T23:46:25.340138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:52.602830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.352889+0000","last_change":"2024-05-08T23:45:04.051578+0000","last_active":"2024-05-08T23:46:25.352889+0000","last_peered":"2024-05-08T23:46:25.352889+0000","last_clean":"2024-05-08T23:46:25.352889+0000","last_became_active":"2024-05-08T23:45:04.051298+0000","last_became_peered":"2024-05-08T23:45:04.051298+0000","last_unstale":"2024-05-08T23:46:25.352889+0000","last_undegraded":"2024-05-08T23:46:25.352889+0000","last_fullsized":"2024-05-08T23:46:25.352889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:47.204849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.346483+0000","last_change":"2024-05-08T23:45:04.051264+0000","last_active":"2024-05-08T23:46:25.346483+0000","last_peered":"2024-05-08T23:46:25.346483+0000","last_clean":"2024-05-08T23:46:25.346483+0000","last_became_active":"2024-05-08T23:45:04.050870+0000","last_became_peered":"2024-05-08T23:45:04.050870+0000","last_unstale":"2024-05-08T23:46:25.346483+0000","last_undegraded":"2024-05-08T23:46:25.346483+0000","last_fullsized":"2024-05-08T23:46:25.346483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":594119,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":587,"num_read_kb":399,"num_write":267,"num_write_kb":1424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2015232,"data_stored":1782357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514522,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":925696,"data_stored":447350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.497},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48099999999999998}]},{"osd":1,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.44},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.63400000000000001}]},{"osd":7,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.628},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.61699999999999999}]},{"osd":8,"last update":"Wed May 8 23:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66700000000000004}]},{"osd":9,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.73999999999999999}]},{"osd":10,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.64100000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743453,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1748992,"data_stored":1235749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.49099999999999999}]},{"osd":1,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.439,"15min":0.442},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66400000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.33000000000000002}]},{"osd":2,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.80600000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78500000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.45900000000000002}]},{"osd":3,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.46999999999999997}]},{"osd":4,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.439},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.40100000000000002}]},{"osd":5,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80900000000000005,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.505}]},{"osd":6,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.69499999999999995,"15min":0.71199999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79500000000000004,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81200000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88800000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92800000000000005,"5min":1.236,"15min":1.236},"last":0.61099999999999999}]},{"osd":9,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.78100000000000003,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88200000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89200000000000002,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.63900000000000001}]},{"osd":11,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.629}]}]},{"osd":9,"up_from":64,"seq":274877906977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28316,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716836,"statfs":{"total":95995035648,"available":95966040064,"internally_reserved":0,"allocated":1327104,"data_stored":840746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63700000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.70899999999999996}]},{"osd":3,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55800000000000005,"15min":0.56599999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.66400000000000003}]},{"osd":4,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93700000000000006,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84999999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.91800000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.748},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.502,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":1.212,"15min":1.212},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.74399999999999999,"15min":0.751},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.012,"5min":1.03,"15min":1.03},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76900000000000002,"15min":0.77600000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.999,"5min":1.117,"15min":1.117},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0089999999999999,"5min":1.139,"15min":1.139},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.624,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95199999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.878}]},{"osd":10,"last update":"Wed May 8 23:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77100000000000002,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.63,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.98699999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.92900000000000005}]}]},{"osd":8,"up_from":56,"seq":240518168613,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32404,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712748,"statfs":{"total":95995035648,"available":95961853952,"internally_reserved":0,"allocated":1318912,"data_stored":840313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.63100000000000001,"15min":0.59599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.2489999999999997,"5min":6.2489999999999997,"15min":6.2489999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.3869999999999996,"5min":6.3869999999999996,"15min":6.3869999999999996},"last":0.39900000000000002}]},{"osd":1,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52400000000000002}]},{"osd":2,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.45200000000000001,"15min":0.44500000000000001},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.65300000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38100000000000001}]},{"osd":3,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.593,"5min":6.593,"15min":6.593},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.63900000000000001,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.5039999999999996,"5min":6.5039999999999996,"15min":6.5039999999999996},"last":0.65600000000000003}]},{"osd":4,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.626,"15min":0.622},"min":{"1min":0.45700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.495}]},{"osd":5,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72399999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.46700000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.52200000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.55200000000000005}]},{"osd":7,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93700000000000006,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60199999999999998}]},{"osd":9,"last update":"Wed May 8 23:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.70199999999999996,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.73099999999999998,"15min":0.70999999999999996},"min":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.59299999999999997}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.72799999999999998,"15min":0.71899999999999997},"min":{"1min":0.52300000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61599999999999999}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.627}]}]},{"osd":7,"up_from":49,"seq":210453397544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1523712,"data_stored":1039269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.437,"15min":0.44500000000000001},"min":{"1min":0.27300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.51600000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.30099999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.54400000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.753,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.432},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76700000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45400000000000001}]},{"osd":3,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.59899999999999998},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71599999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83299999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.754,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.501,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.64500000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.34300000000000003}]},{"osd":5,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.624,"15min":0.628},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76000000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80000000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.69399999999999995,"15min":0.70799999999999996},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.83599999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77100000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.48599999999999999}]},{"osd":8,"last update":"Wed May 8 23:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.502},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.89600000000000002,"5min":1.016,"15min":1.016},"last":0.58399999999999996}]},{"osd":9,"last update":"Wed May 8 23:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.58799999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.91700000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.48599999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.89500000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.57599999999999996}]},{"osd":10,"last update":"Wed May 8 23:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.501,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.78400000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.70199999999999996,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81899999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.52500000000000002}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63}]}]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":446919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65000000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.438,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68999999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41299999999999998}]},{"osd":1,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.65600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88200000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90000000000000002,"5min":1.288,"15min":1.288},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":1.018,"15min":1.018},"last":0.498},{"interface":"front","average":{"1min":0.503,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81899999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.376}]},{"osd":4,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":1.113,"15min":1.113},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98399999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.49299999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96499999999999997,"5min":1.002,"15min":1.002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82099999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed May 8 23:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.88800000000000001,"5min":1.321,"15min":1.321},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.52000000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.74099999999999999}]},{"osd":8,"last update":"Wed May 8 23:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.111,"15min":1.111},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.02,"5min":1.05,"15min":1.05},"last":0.60599999999999998}]},{"osd":9,"last update":"Wed May 8 23:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.85499999999999998,"15min":0.872},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0940000000000001,"5min":1.095,"15min":1.095},"last":0.67300000000000004}]},{"osd":10,"last update":"Wed May 8 23:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.78600000000000003},"min":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.75},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.107,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.78500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":978944,"data_stored":510368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.55600000000000005,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.96,"5min":3.96,"15min":3.96},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.58099999999999996,"15min":0.51900000000000002},"min":{"1min":0.44400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed May 8 23:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed May 8 23:46:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.60099999999999998,"15min":0.54900000000000004},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.101,"5min":4.101,"15min":4.101},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.63900000000000001,"15min":0.55700000000000005},"min":{"1min":0.51400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.1820000000000004,"5min":4.1820000000000004,"15min":4.1820000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed May 8 23:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41799999999999998,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.84399999999999997,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.37},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.40699999999999997},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68400000000000005,"5min":0.88,"15min":0.88},"last":0.32200000000000001}]},{"osd":5,"last update":"Wed May 8 23:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.57299999999999995,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.68300000000000005,"15min":0.65700000000000003},"min":{"1min":0.49199999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.7549999999999999,"5min":4.7549999999999999,"15min":4.7549999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.175,"5min":0.747,"15min":0.66700000000000004},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Wed May 8 23:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55300000000000005,"15min":0.51800000000000002},"min":{"1min":0.39800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92900000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60299999999999998,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54500000000000004}]},{"osd":8,"last update":"Wed May 8 23:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52900000000000003}]},{"osd":9,"last update":"Wed May 8 23:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.9240000000000004,"5min":4.9240000000000004,"15min":4.9240000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.81999999999999995,"15min":0.78000000000000003},"min":{"1min":0.57499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":6.3769999999999998,"5min":6.3769999999999998,"15min":6.3769999999999998},"last":0.64900000000000002}]},{"osd":10,"last update":"Wed May 8 23:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0589999999999999,"5min":1.095,"15min":1.095},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.46800000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96499999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59699999999999998}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.626},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":447823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.54800000000000004,"15min":0.50600000000000001},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.36299999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":1.032,"15min":1.032},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed May 8 23:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56699999999999995,"15min":0.52700000000000002},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.60599999999999998,"15min":0.54800000000000004},"min":{"1min":0.44700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.625},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":3.1099999999999999,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed May 8 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59099999999999997,"15min":0.56200000000000006},"min":{"1min":0.51100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.65500000000000003,"15min":0.621},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed May 8 23:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.66300000000000003,"15min":0.627},"min":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Wed May 8 23:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.64200000000000002,"15min":0.58999999999999997},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":6.0949999999999998,"5min":6.0949999999999998,"15min":6.0949999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.70599999999999996,"15min":0.65100000000000002},"min":{"1min":0.39500000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":6.1989999999999998,"5min":6.1989999999999998,"15min":6.1989999999999998},"last":0.746}]},{"osd":7,"last update":"Wed May 8 23:44:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80400000000000005,"15min":0.80100000000000005},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.47299999999999998}]},{"osd":9,"last update":"Wed May 8 23:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.46700000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98099999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83099999999999996,"15min":0.83299999999999996},"min":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.78900000000000003}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.82799999999999996},"min":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.96699999999999997,"5min":1.127,"15min":1.127},"last":0.90300000000000002}]},{"osd":11,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.64100000000000001}]}]},{"osd":2,"up_from":17,"seq":73014444091,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":999424,"data_stored":512422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82799999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54400000000000004,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.70099999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.52600000000000002,"15min":0.48999999999999999},"min":{"1min":0.40799999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83199999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.5},"min":{"1min":0.38300000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.80600000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64400000000000002}]},{"osd":3,"last update":"Wed May 8 23:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55300000000000005,"15min":0.52600000000000002},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54600000000000004,"15min":0.52400000000000002},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68799999999999994}]},{"osd":5,"last update":"Wed May 8 23:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.45500000000000002},"min":{"1min":0.34499999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.53200000000000003,"15min":0.51100000000000001},"min":{"1min":0.45400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed May 8 23:46:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64100000000000001,"15min":0.629},"min":{"1min":0.51200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.77900000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.46899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90400000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.42899999999999999}]},{"osd":9,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.627},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.63100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63600000000000001}]},{"osd":10,"last update":"Wed May 8 23:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.51500000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85699999999999998,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.92200000000000004,"15min":0.95799999999999996},"min":{"1min":0.58599999999999997,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93899999999999995,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.69699999999999995}]},{"osd":11,"last update":"Wed May 8 23:46:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.66400000000000003}]}]},{"osd":3,"up_from":24,"seq":103079215159,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":921600,"data_stored":438275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.443,"15min":0.39300000000000002},"min":{"1min":0.312,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.4790000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.434,"15min":0.39800000000000002},"min":{"1min":0.309,"5min":0.248,"15min":0.248},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.497}]},{"osd":1,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.57899999999999996,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.48299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.5459999999999998,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":0.72399999999999998}]},{"osd":2,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.65100000000000002,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.64800000000000002,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed May 8 23:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.60799999999999998,"15min":0.56999999999999995},"min":{"1min":0.51100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69899999999999995}]},{"osd":5,"last update":"Wed May 8 23:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.372},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.64500000000000002,"15min":0.61099999999999999},"min":{"1min":0.53000000000000003,"5min":0.371,"15min":0.371},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.441}]},{"osd":6,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.59999999999999998,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.0950000000000002,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":0.41899999999999998}]},{"osd":7,"last update":"Wed May 8 23:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70599999999999996,"15min":0.66000000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9119999999999999,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.745,"15min":0.71699999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.63500000000000001}]},{"osd":8,"last update":"Wed May 8 23:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed May 8 23:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.76500000000000001,"15min":0.748},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.431,"5min":3.431,"15min":3.431},"last":0.56299999999999994}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.84699999999999998,"15min":0.81899999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.874,"15min":0.86399999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.67600000000000005}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.617,"5min":3.617,"15min":3.617},"last":0.624}]}]},{"osd":4,"up_from":31,"seq":133143986225,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":447375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61799999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.377},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.67700000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.307}]},{"osd":2,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.40699999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81999999999999995,"5min":1.222,"15min":1.222},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.62}]},{"osd":3,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.438},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84899999999999998,"5min":0.874,"15min":0.874},"last":0.41699999999999998}]},{"osd":5,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80900000000000005,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78900000000000003,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48099999999999998}]},{"osd":7,"last update":"Wed May 8 23:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.82499999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53200000000000003,"15min":0.51700000000000002},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76700000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.495}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51000000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85499999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91100000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.53000000000000003}]},{"osd":9,"last update":"Wed May 8 23:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.71099999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90600000000000003,"5min":0.997,"15min":0.997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92100000000000004,"5min":1.115,"15min":1.115},"last":0.64600000000000002}]},{"osd":10,"last update":"Wed May 8 23:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89400000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.48699999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76400000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55800000000000005}]},{"osd":11,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60399999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1544192,"data_stored":1039137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.51200000000000001,"15min":0.47899999999999998},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80100000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.436},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.34699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.50600000000000001},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70699999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.68100000000000005,"5min":0.873,"15min":0.873},"last":0.57699999999999996}]},{"osd":2,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.372,"15min":0.35899999999999999},"min":{"1min":0.27100000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.40500000000000003},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.63400000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.34399999999999997}]},{"osd":3,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.48599999999999999},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74299999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.501},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70699999999999996}]},{"osd":4,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.55500000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.373},"max":{"1min":0.872,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89100000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.49299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82099999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.45800000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.61699999999999999}]},{"osd":7,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59899999999999998,"15min":0.58499999999999996},"min":{"1min":0.46100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63200000000000001,"15min":0.62},"min":{"1min":0.5,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91000000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67700000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85399999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46600000000000003}]},{"osd":9,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92600000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.878,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63700000000000001}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.76900000000000002},"min":{"1min":0.58999999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.88600000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.76400000000000001},"min":{"1min":0.60699999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93700000000000006,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":198919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:46:30.939 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-08T23:46:30.939 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:46:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 -- 172.21.15.12:0/2475533016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde981008f0 msgr2=0x7fde98104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 --2- 172.21.15.12:0/2475533016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde981008f0 0x7fde98104930 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fde88009990 tx=0x7fde88009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 -- 172.21.15.12:0/2475533016 shutdown_connections 2024-05-08T23:46:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 --2- 172.21.15.12:0/2475533016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde98105060 0x7fde9810cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 --2- 172.21.15.12:0/2475533016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde980fff40 0x7fde98100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 --2- 172.21.15.12:0/2475533016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde981008f0 0x7fde98104930 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 -- 172.21.15.12:0/2475533016 >> 172.21.15.12:0/2475533016 conn(0x7fde980fb7e0 msgr2=0x7fde980fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 -- 172.21.15.12:0/2475533016 shutdown_connections 2024-05-08T23:46:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 -- 172.21.15.12:0/2475533016 wait complete. 2024-05-08T23:46:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 Processor -- start 2024-05-08T23:46:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.118+0000 7fde9f589700 1 -- start start 2024-05-08T23:46:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9f589700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde980fff40 0x7fde981995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9f589700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde98105060 0x7fde98199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9f589700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9819de10 0x7fde9819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9f589700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9819e890 con 0x7fde98105060 2024-05-08T23:46:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9f589700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9819ea00 con 0x7fde9819de10 2024-05-08T23:46:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9f589700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9819ebc0 con 0x7fde980fff40 2024-05-08T23:46:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9cb24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde98105060 0x7fde98199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9cb24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde98105060 0x7fde98199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48260/0 (socket says 172.21.15.12:48260) 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9cb24700 1 -- 172.21.15.12:0/1868064011 learned_addr learned my addr 172.21.15.12:0/1868064011 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9d325700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde980fff40 0x7fde981995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9db26700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9819de10 0x7fde9819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9cb24700 1 -- 172.21.15.12:0/1868064011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde980fff40 msgr2=0x7fde981995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9cb24700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde980fff40 0x7fde981995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9cb24700 1 -- 172.21.15.12:0/1868064011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9819de10 msgr2=0x7fde9819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9cb24700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9819de10 0x7fde9819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9cb24700 1 -- 172.21.15.12:0/1868064011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880095f0 con 0x7fde98105060 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde9cb24700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde98105060 0x7fde98199ae0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fde88000c00 tx=0x7fde880036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.119+0000 7fde8e7fc700 1 -- 172.21.15.12:0/1868064011 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8801d070 con 0x7fde98105060 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.120+0000 7fde8e7fc700 1 -- 172.21.15.12:0/1868064011 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde8800fb30 con 0x7fde98105060 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.120+0000 7fde8e7fc700 1 -- 172.21.15.12:0/1868064011 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88022ad0 con 0x7fde98105060 2024-05-08T23:46:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.120+0000 7fde9f589700 1 -- 172.21.15.12:0/1868064011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde981a60c0 con 0x7fde98105060 2024-05-08T23:46:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.120+0000 7fde9f589700 1 -- 172.21.15.12:0/1868064011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde981a64e0 con 0x7fde98105060 2024-05-08T23:46:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.121+0000 7fde9f589700 1 -- 172.21.15.12:0/1868064011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde981030a0 con 0x7fde98105060 2024-05-08T23:46:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.122+0000 7fde8e7fc700 1 -- 172.21.15.12:0/1868064011 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7fde8800fca0 con 0x7fde98105060 2024-05-08T23:46:31.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.122+0000 7fde8e7fc700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde840a13a0 0x7fde840a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.122+0000 7fde9d325700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde840a13a0 0x7fde840a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.122+0000 7fde8e7fc700 1 -- 172.21.15.12:0/1868064011 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fde880ce440 con 0x7fde98105060 2024-05-08T23:46:31.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.123+0000 7fde9d325700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde840a13a0 0x7fde840a3860 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fde94005950 tx=0x7fde9400b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:31.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.127+0000 7fde8e7fc700 1 -- 172.21.15.12:0/1868064011 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fde880977f0 con 0x7fde98105060 2024-05-08T23:46:31.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.347+0000 7fde9f589700 1 -- 172.21.15.12:0/1868064011 --> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde9819a8a0 con 0x7fde840a13a0 2024-05-08T23:46:31.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.358+0000 7fde8e7fc700 1 -- 172.21.15.12:0/1868064011 <== mgr.15252 v2:172.21.15.12:6800/2525408937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343590 (secure 0 0 0) 0x7fde9819a8a0 con 0x7fde840a13a0 2024-05-08T23:46:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:31.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 -- 172.21.15.12:0/1868064011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde840a13a0 msgr2=0x7fde840a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde840a13a0 0x7fde840a3860 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fde94005950 tx=0x7fde9400b410 comp rx=0 tx=0).stop 2024-05-08T23:46:31.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 -- 172.21.15.12:0/1868064011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde98105060 msgr2=0x7fde98199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde98105060 0x7fde98199ae0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fde88000c00 tx=0x7fde880036f0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 -- 172.21.15.12:0/1868064011 shutdown_connections 2024-05-08T23:46:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde840a13a0 0x7fde840a3860 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9819de10 0x7fde9819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde980fff40 0x7fde981995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 --2- 172.21.15.12:0/1868064011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde98105060 0x7fde98199ae0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 -- 172.21.15.12:0/1868064011 >> 172.21.15.12:0/1868064011 conn(0x7fde980fb7e0 msgr2=0x7fde980fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 -- 172.21.15.12:0/1868064011 shutdown_connections 2024-05-08T23:46:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.362+0000 7fde9f589700 1 -- 172.21.15.12:0/1868064011 wait complete. 2024-05-08T23:46:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-08T23:46:31.371 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-08T23:46:31.310541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1184721,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":723,"num_read_kb":518,"num_write":450,"num_write_kb":4364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":851,"ondisk_log_size":851,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374900,"kb_used_data":13736,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124566924,"statfs":{"total":1151940427776,"available":1151556530176,"internally_reserved":0,"allocated":14065664,"data_stored":8245746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":369603909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.980089"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340326+0000","last_change":"2024-05-08T23:45:04.044088+0000","last_active":"2024-05-08T23:46:25.340326+0000","last_peered":"2024-05-08T23:46:25.340326+0000","last_clean":"2024-05-08T23:46:25.340326+0000","last_became_active":"2024-05-08T23:45:04.043535+0000","last_became_peered":"2024-05-08T23:45:04.043535+0000","last_unstale":"2024-05-08T23:46:25.340326+0000","last_undegraded":"2024-05-08T23:46:25.340326+0000","last_fullsized":"2024-05-08T23:46:25.340326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:52.363402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344567+0000","last_change":"2024-05-08T23:45:04.049201+0000","last_active":"2024-05-08T23:46:25.344567+0000","last_peered":"2024-05-08T23:46:25.344567+0000","last_clean":"2024-05-08T23:46:25.344567+0000","last_became_active":"2024-05-08T23:45:04.048743+0000","last_became_peered":"2024-05-08T23:45:04.048743+0000","last_unstale":"2024-05-08T23:46:25.344567+0000","last_undegraded":"2024-05-08T23:46:25.344567+0000","last_fullsized":"2024-05-08T23:46:25.344567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:26.191549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348268+0000","last_change":"2024-05-08T23:45:04.047356+0000","last_active":"2024-05-08T23:46:25.348268+0000","last_peered":"2024-05-08T23:46:25.348268+0000","last_clean":"2024-05-08T23:46:25.348268+0000","last_became_active":"2024-05-08T23:45:04.046996+0000","last_became_peered":"2024-05-08T23:45:04.046996+0000","last_unstale":"2024-05-08T23:46:25.348268+0000","last_undegraded":"2024-05-08T23:46:25.348268+0000","last_fullsized":"2024-05-08T23:46:25.348268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:58.722522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349109+0000","last_change":"2024-05-08T23:45:04.054754+0000","last_active":"2024-05-08T23:46:25.349109+0000","last_peered":"2024-05-08T23:46:25.349109+0000","last_clean":"2024-05-08T23:46:25.349109+0000","last_became_active":"2024-05-08T23:45:04.054302+0000","last_became_peered":"2024-05-08T23:45:04.054302+0000","last_unstale":"2024-05-08T23:46:25.349109+0000","last_undegraded":"2024-05-08T23:46:25.349109+0000","last_fullsized":"2024-05-08T23:46:25.349109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:28.882982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354073+0000","last_change":"2024-05-08T23:45:04.047189+0000","last_active":"2024-05-08T23:46:25.354073+0000","last_peered":"2024-05-08T23:46:25.354073+0000","last_clean":"2024-05-08T23:46:25.354073+0000","last_became_active":"2024-05-08T23:45:04.046756+0000","last_became_peered":"2024-05-08T23:45:04.046756+0000","last_unstale":"2024-05-08T23:46:25.354073+0000","last_undegraded":"2024-05-08T23:46:25.354073+0000","last_fullsized":"2024-05-08T23:46:25.354073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.780169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.338455+0000","last_change":"2024-05-08T23:45:04.060897+0000","last_active":"2024-05-08T23:46:25.338455+0000","last_peered":"2024-05-08T23:46:25.338455+0000","last_clean":"2024-05-08T23:46:25.338455+0000","last_became_active":"2024-05-08T23:45:04.060550+0000","last_became_peered":"2024-05-08T23:45:04.060550+0000","last_unstale":"2024-05-08T23:46:25.338455+0000","last_undegraded":"2024-05-08T23:46:25.338455+0000","last_fullsized":"2024-05-08T23:46:25.338455+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:41:59.356057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343816+0000","last_change":"2024-05-08T23:45:04.059308+0000","last_active":"2024-05-08T23:46:25.343816+0000","last_peered":"2024-05-08T23:46:25.343816+0000","last_clean":"2024-05-08T23:46:25.343816+0000","last_became_active":"2024-05-08T23:45:04.058920+0000","last_became_peered":"2024-05-08T23:45:04.058920+0000","last_unstale":"2024-05-08T23:46:25.343816+0000","last_undegraded":"2024-05-08T23:46:25.343816+0000","last_fullsized":"2024-05-08T23:46:25.343816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:11.857062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342864+0000","last_change":"2024-05-08T23:45:56.359452+0000","last_active":"2024-05-08T23:46:25.342864+0000","last_peered":"2024-05-08T23:46:25.342864+0000","last_clean":"2024-05-08T23:46:25.342864+0000","last_became_active":"2024-05-08T23:45:56.359066+0000","last_became_peered":"2024-05-08T23:45:56.359066+0000","last_unstale":"2024-05-08T23:46:25.342864+0000","last_undegraded":"2024-05-08T23:46:25.342864+0000","last_fullsized":"2024-05-08T23:46:25.342864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:13.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344525+0000","last_change":"2024-05-08T23:45:04.052292+0000","last_active":"2024-05-08T23:46:25.344525+0000","last_peered":"2024-05-08T23:46:25.344525+0000","last_clean":"2024-05-08T23:46:25.344525+0000","last_became_active":"2024-05-08T23:45:04.051164+0000","last_became_peered":"2024-05-08T23:45:04.051164+0000","last_unstale":"2024-05-08T23:46:25.344525+0000","last_undegraded":"2024-05-08T23:46:25.344525+0000","last_fullsized":"2024-05-08T23:46:25.344525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:09.218130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348445+0000","last_change":"2024-05-08T23:45:04.054566+0000","last_active":"2024-05-08T23:46:25.348445+0000","last_peered":"2024-05-08T23:46:25.348445+0000","last_clean":"2024-05-08T23:46:25.348445+0000","last_became_active":"2024-05-08T23:45:04.053878+0000","last_became_peered":"2024-05-08T23:45:04.053878+0000","last_unstale":"2024-05-08T23:46:25.348445+0000","last_undegraded":"2024-05-08T23:46:25.348445+0000","last_fullsized":"2024-05-08T23:46:25.348445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:34.760038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342587+0000","last_change":"2024-05-08T23:45:04.049696+0000","last_active":"2024-05-08T23:46:25.342587+0000","last_peered":"2024-05-08T23:46:25.342587+0000","last_clean":"2024-05-08T23:46:25.342587+0000","last_became_active":"2024-05-08T23:45:04.049351+0000","last_became_peered":"2024-05-08T23:45:04.049351+0000","last_unstale":"2024-05-08T23:46:25.342587+0000","last_undegraded":"2024-05-08T23:46:25.342587+0000","last_fullsized":"2024-05-08T23:46:25.342587+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:38:53.557738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348957+0000","last_change":"2024-05-08T23:45:04.051010+0000","last_active":"2024-05-08T23:46:25.348957+0000","last_peered":"2024-05-08T23:46:25.348957+0000","last_clean":"2024-05-08T23:46:25.348957+0000","last_became_active":"2024-05-08T23:45:04.050715+0000","last_became_peered":"2024-05-08T23:45:04.050715+0000","last_unstale":"2024-05-08T23:46:25.348957+0000","last_undegraded":"2024-05-08T23:46:25.348957+0000","last_fullsized":"2024-05-08T23:46:25.348957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:09.305974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.353741+0000","last_change":"2024-05-08T23:45:04.051005+0000","last_active":"2024-05-08T23:46:25.353741+0000","last_peered":"2024-05-08T23:46:25.353741+0000","last_clean":"2024-05-08T23:46:25.353741+0000","last_became_active":"2024-05-08T23:45:04.050123+0000","last_became_peered":"2024-05-08T23:45:04.050123+0000","last_unstale":"2024-05-08T23:46:25.353741+0000","last_undegraded":"2024-05-08T23:46:25.353741+0000","last_fullsized":"2024-05-08T23:46:25.353741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:52.206389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348724+0000","last_change":"2024-05-08T23:45:04.049942+0000","last_active":"2024-05-08T23:46:25.348724+0000","last_peered":"2024-05-08T23:46:25.348724+0000","last_clean":"2024-05-08T23:46:25.348724+0000","last_became_active":"2024-05-08T23:45:04.049596+0000","last_became_peered":"2024-05-08T23:45:04.049596+0000","last_unstale":"2024-05-08T23:46:25.348724+0000","last_undegraded":"2024-05-08T23:46:25.348724+0000","last_fullsized":"2024-05-08T23:46:25.348724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:30.907023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350562+0000","last_change":"2024-05-08T23:45:04.048985+0000","last_active":"2024-05-08T23:46:25.350562+0000","last_peered":"2024-05-08T23:46:25.350562+0000","last_clean":"2024-05-08T23:46:25.350562+0000","last_became_active":"2024-05-08T23:45:04.048658+0000","last_became_peered":"2024-05-08T23:45:04.048658+0000","last_unstale":"2024-05-08T23:46:25.350562+0000","last_undegraded":"2024-05-08T23:46:25.350562+0000","last_fullsized":"2024-05-08T23:46:25.350562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:17.030930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.338683+0000","last_change":"2024-05-08T23:45:04.051903+0000","last_active":"2024-05-08T23:46:25.338683+0000","last_peered":"2024-05-08T23:46:25.338683+0000","last_clean":"2024-05-08T23:46:25.338683+0000","last_became_active":"2024-05-08T23:45:04.051221+0000","last_became_peered":"2024-05-08T23:45:04.051221+0000","last_unstale":"2024-05-08T23:46:25.338683+0000","last_undegraded":"2024-05-08T23:46:25.338683+0000","last_fullsized":"2024-05-08T23:46:25.338683+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:01.554463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340534+0000","last_change":"2024-05-08T23:45:04.043854+0000","last_active":"2024-05-08T23:46:25.340534+0000","last_peered":"2024-05-08T23:46:25.340534+0000","last_clean":"2024-05-08T23:46:25.340534+0000","last_became_active":"2024-05-08T23:45:04.043056+0000","last_became_peered":"2024-05-08T23:45:04.043056+0000","last_unstale":"2024-05-08T23:46:25.340534+0000","last_undegraded":"2024-05-08T23:46:25.340534+0000","last_fullsized":"2024-05-08T23:46:25.340534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:18.175575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.338204+0000","last_change":"2024-05-08T23:45:04.049332+0000","last_active":"2024-05-08T23:46:25.338204+0000","last_peered":"2024-05-08T23:46:25.338204+0000","last_clean":"2024-05-08T23:46:25.338204+0000","last_became_active":"2024-05-08T23:45:04.048987+0000","last_became_peered":"2024-05-08T23:45:04.048987+0000","last_unstale":"2024-05-08T23:46:25.338204+0000","last_undegraded":"2024-05-08T23:46:25.338204+0000","last_fullsized":"2024-05-08T23:46:25.338204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:02.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333688+0000","last_change":"2024-05-08T23:45:04.048536+0000","last_active":"2024-05-08T23:46:25.333688+0000","last_peered":"2024-05-08T23:46:25.333688+0000","last_clean":"2024-05-08T23:46:25.333688+0000","last_became_active":"2024-05-08T23:45:04.048102+0000","last_became_peered":"2024-05-08T23:45:04.048102+0000","last_unstale":"2024-05-08T23:46:25.333688+0000","last_undegraded":"2024-05-08T23:46:25.333688+0000","last_fullsized":"2024-05-08T23:46:25.333688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:11.794372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.353062+0000","last_change":"2024-05-08T23:45:03.576833+0000","last_active":"2024-05-08T23:45:56.353062+0000","last_peered":"2024-05-08T23:45:56.353062+0000","last_clean":"2024-05-08T23:45:56.353062+0000","last_became_active":"2024-05-08T23:45:03.576495+0000","last_became_peered":"2024-05-08T23:45:03.576495+0000","last_unstale":"2024-05-08T23:45:56.353062+0000","last_undegraded":"2024-05-08T23:45:56.353062+0000","last_fullsized":"2024-05-08T23:45:56.353062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:50.266658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340176+0000","last_change":"2024-05-08T23:45:04.055444+0000","last_active":"2024-05-08T23:46:25.340176+0000","last_peered":"2024-05-08T23:46:25.340176+0000","last_clean":"2024-05-08T23:46:25.340176+0000","last_became_active":"2024-05-08T23:45:04.054998+0000","last_became_peered":"2024-05-08T23:45:04.054998+0000","last_unstale":"2024-05-08T23:46:25.340176+0000","last_undegraded":"2024-05-08T23:46:25.340176+0000","last_fullsized":"2024-05-08T23:46:25.340176+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:03.470555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.359775+0000","last_change":"2024-05-08T23:45:03.578362+0000","last_active":"2024-05-08T23:45:56.359775+0000","last_peered":"2024-05-08T23:45:56.359775+0000","last_clean":"2024-05-08T23:45:56.359775+0000","last_became_active":"2024-05-08T23:45:03.577834+0000","last_became_peered":"2024-05-08T23:45:03.577834+0000","last_unstale":"2024-05-08T23:45:56.359775+0000","last_undegraded":"2024-05-08T23:45:56.359775+0000","last_fullsized":"2024-05-08T23:45:56.359775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:01.943490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.346351+0000","last_change":"2024-05-08T23:45:04.047205+0000","last_active":"2024-05-08T23:46:25.346351+0000","last_peered":"2024-05-08T23:46:25.346351+0000","last_clean":"2024-05-08T23:46:25.346351+0000","last_became_active":"2024-05-08T23:45:04.046860+0000","last_became_peered":"2024-05-08T23:45:04.046860+0000","last_unstale":"2024-05-08T23:46:25.346351+0000","last_undegraded":"2024-05-08T23:46:25.346351+0000","last_fullsized":"2024-05-08T23:46:25.346351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:36.837101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.361012+0000","last_change":"2024-05-08T23:45:03.582026+0000","last_active":"2024-05-08T23:45:56.361012+0000","last_peered":"2024-05-08T23:45:56.361012+0000","last_clean":"2024-05-08T23:45:56.361012+0000","last_became_active":"2024-05-08T23:45:03.581672+0000","last_became_peered":"2024-05-08T23:45:03.581672+0000","last_unstale":"2024-05-08T23:45:56.361012+0000","last_undegraded":"2024-05-08T23:45:56.361012+0000","last_fullsized":"2024-05-08T23:45:56.361012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:50.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349108+0000","last_change":"2024-05-08T23:45:04.052639+0000","last_active":"2024-05-08T23:46:25.349108+0000","last_peered":"2024-05-08T23:46:25.349108+0000","last_clean":"2024-05-08T23:46:25.349108+0000","last_became_active":"2024-05-08T23:45:04.052158+0000","last_became_peered":"2024-05-08T23:45:04.052158+0000","last_unstale":"2024-05-08T23:46:25.349108+0000","last_undegraded":"2024-05-08T23:46:25.349108+0000","last_fullsized":"2024-05-08T23:46:25.349108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:25.638555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360530+0000","last_change":"2024-05-08T23:45:03.579221+0000","last_active":"2024-05-08T23:45:56.360530+0000","last_peered":"2024-05-08T23:45:56.360530+0000","last_clean":"2024-05-08T23:45:56.360530+0000","last_became_active":"2024-05-08T23:45:03.578795+0000","last_became_peered":"2024-05-08T23:45:03.578795+0000","last_unstale":"2024-05-08T23:45:56.360530+0000","last_undegraded":"2024-05-08T23:45:56.360530+0000","last_fullsized":"2024-05-08T23:45:56.360530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:14.064108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350032+0000","last_change":"2024-05-08T23:45:04.057573+0000","last_active":"2024-05-08T23:46:25.350032+0000","last_peered":"2024-05-08T23:46:25.350032+0000","last_clean":"2024-05-08T23:46:25.350032+0000","last_became_active":"2024-05-08T23:45:04.053201+0000","last_became_peered":"2024-05-08T23:45:04.053201+0000","last_unstale":"2024-05-08T23:46:25.350032+0000","last_undegraded":"2024-05-08T23:46:25.350032+0000","last_fullsized":"2024-05-08T23:46:25.350032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:23.612609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.313218+0000","last_change":"2024-05-08T23:45:03.578110+0000","last_active":"2024-05-08T23:46:25.313218+0000","last_peered":"2024-05-08T23:46:25.313218+0000","last_clean":"2024-05-08T23:46:25.313218+0000","last_became_active":"2024-05-08T23:45:03.577749+0000","last_became_peered":"2024-05-08T23:45:03.577749+0000","last_unstale":"2024-05-08T23:46:25.313218+0000","last_undegraded":"2024-05-08T23:46:25.313218+0000","last_fullsized":"2024-05-08T23:46:25.313218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:50.526535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.337443+0000","last_change":"2024-05-08T23:45:04.048131+0000","last_active":"2024-05-08T23:46:25.337443+0000","last_peered":"2024-05-08T23:46:25.337443+0000","last_clean":"2024-05-08T23:46:25.337443+0000","last_became_active":"2024-05-08T23:45:04.047773+0000","last_became_peered":"2024-05-08T23:45:04.047773+0000","last_unstale":"2024-05-08T23:46:25.337443+0000","last_undegraded":"2024-05-08T23:46:25.337443+0000","last_fullsized":"2024-05-08T23:46:25.337443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:47.944900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.353596+0000","last_change":"2024-05-08T23:45:03.586710+0000","last_active":"2024-05-08T23:45:56.353596+0000","last_peered":"2024-05-08T23:45:56.353596+0000","last_clean":"2024-05-08T23:45:56.353596+0000","last_became_active":"2024-05-08T23:45:03.586411+0000","last_became_peered":"2024-05-08T23:45:03.586411+0000","last_unstale":"2024-05-08T23:45:56.353596+0000","last_undegraded":"2024-05-08T23:45:56.353596+0000","last_fullsized":"2024-05-08T23:45:56.353596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:25.938846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349408+0000","last_change":"2024-05-08T23:45:04.053885+0000","last_active":"2024-05-08T23:46:25.349408+0000","last_peered":"2024-05-08T23:46:25.349408+0000","last_clean":"2024-05-08T23:46:25.349408+0000","last_became_active":"2024-05-08T23:45:04.053385+0000","last_became_peered":"2024-05-08T23:45:04.053385+0000","last_unstale":"2024-05-08T23:46:25.349408+0000","last_undegraded":"2024-05-08T23:46:25.349408+0000","last_fullsized":"2024-05-08T23:46:25.349408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:10.805481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.350475+0000","last_change":"2024-05-08T23:45:03.574988+0000","last_active":"2024-05-08T23:45:56.350475+0000","last_peered":"2024-05-08T23:45:56.350475+0000","last_clean":"2024-05-08T23:45:56.350475+0000","last_became_active":"2024-05-08T23:45:03.574586+0000","last_became_peered":"2024-05-08T23:45:03.574586+0000","last_unstale":"2024-05-08T23:45:56.350475+0000","last_undegraded":"2024-05-08T23:45:56.350475+0000","last_fullsized":"2024-05-08T23:45:56.350475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:18.005107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354197+0000","last_change":"2024-05-08T23:45:04.046817+0000","last_active":"2024-05-08T23:46:25.354197+0000","last_peered":"2024-05-08T23:46:25.354197+0000","last_clean":"2024-05-08T23:46:25.354197+0000","last_became_active":"2024-05-08T23:45:04.046515+0000","last_became_peered":"2024-05-08T23:45:04.046515+0000","last_unstale":"2024-05-08T23:46:25.354197+0000","last_undegraded":"2024-05-08T23:46:25.354197+0000","last_fullsized":"2024-05-08T23:46:25.354197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:29.439670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355956+0000","last_change":"2024-05-08T23:45:03.579002+0000","last_active":"2024-05-08T23:45:56.355956+0000","last_peered":"2024-05-08T23:45:56.355956+0000","last_clean":"2024-05-08T23:45:56.355956+0000","last_became_active":"2024-05-08T23:45:03.577891+0000","last_became_peered":"2024-05-08T23:45:03.577891+0000","last_unstale":"2024-05-08T23:45:56.355956+0000","last_undegraded":"2024-05-08T23:45:56.355956+0000","last_fullsized":"2024-05-08T23:45:56.355956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:01.027574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344908+0000","last_change":"2024-05-08T23:45:04.060715+0000","last_active":"2024-05-08T23:46:25.344908+0000","last_peered":"2024-05-08T23:46:25.344908+0000","last_clean":"2024-05-08T23:46:25.344908+0000","last_became_active":"2024-05-08T23:45:04.060375+0000","last_became_peered":"2024-05-08T23:45:04.060375+0000","last_unstale":"2024-05-08T23:46:25.344908+0000","last_undegraded":"2024-05-08T23:46:25.344908+0000","last_fullsized":"2024-05-08T23:46:25.344908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:42.133323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.307895+0000","last_change":"2024-05-08T23:45:03.587784+0000","last_active":"2024-05-08T23:46:25.307895+0000","last_peered":"2024-05-08T23:46:25.307895+0000","last_clean":"2024-05-08T23:46:25.307895+0000","last_became_active":"2024-05-08T23:45:03.587529+0000","last_became_peered":"2024-05-08T23:45:03.587529+0000","last_unstale":"2024-05-08T23:46:25.307895+0000","last_undegraded":"2024-05-08T23:46:25.307895+0000","last_fullsized":"2024-05-08T23:46:25.307895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:00.323599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.344334+0000","last_change":"2024-05-08T23:45:04.062672+0000","last_active":"2024-05-08T23:46:25.344334+0000","last_peered":"2024-05-08T23:46:25.344334+0000","last_clean":"2024-05-08T23:46:25.344334+0000","last_became_active":"2024-05-08T23:45:04.062351+0000","last_became_peered":"2024-05-08T23:45:04.062351+0000","last_unstale":"2024-05-08T23:46:25.344334+0000","last_undegraded":"2024-05-08T23:46:25.344334+0000","last_fullsized":"2024-05-08T23:46:25.344334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:13.740239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355455+0000","last_change":"2024-05-08T23:45:03.583501+0000","last_active":"2024-05-08T23:45:56.355455+0000","last_peered":"2024-05-08T23:45:56.355455+0000","last_clean":"2024-05-08T23:45:56.355455+0000","last_became_active":"2024-05-08T23:45:03.583116+0000","last_became_peered":"2024-05-08T23:45:03.583116+0000","last_unstale":"2024-05-08T23:45:56.355455+0000","last_undegraded":"2024-05-08T23:45:56.355455+0000","last_fullsized":"2024-05-08T23:45:56.355455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:03.797524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350404+0000","last_change":"2024-05-08T23:45:04.048336+0000","last_active":"2024-05-08T23:46:25.350404+0000","last_peered":"2024-05-08T23:46:25.350404+0000","last_clean":"2024-05-08T23:46:25.350404+0000","last_became_active":"2024-05-08T23:45:04.048000+0000","last_became_peered":"2024-05-08T23:45:04.048000+0000","last_unstale":"2024-05-08T23:46:25.350404+0000","last_undegraded":"2024-05-08T23:46:25.350404+0000","last_fullsized":"2024-05-08T23:46:25.350404+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:41.806600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355898+0000","last_change":"2024-05-08T23:45:03.581215+0000","last_active":"2024-05-08T23:45:56.355898+0000","last_peered":"2024-05-08T23:45:56.355898+0000","last_clean":"2024-05-08T23:45:56.355898+0000","last_became_active":"2024-05-08T23:45:03.580496+0000","last_became_peered":"2024-05-08T23:45:03.580496+0000","last_unstale":"2024-05-08T23:45:56.355898+0000","last_undegraded":"2024-05-08T23:45:56.355898+0000","last_fullsized":"2024-05-08T23:45:56.355898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:25.602401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.346299+0000","last_change":"2024-05-08T23:45:04.058857+0000","last_active":"2024-05-08T23:46:25.346299+0000","last_peered":"2024-05-08T23:46:25.346299+0000","last_clean":"2024-05-08T23:46:25.346299+0000","last_became_active":"2024-05-08T23:45:04.057988+0000","last_became_peered":"2024-05-08T23:45:04.057988+0000","last_unstale":"2024-05-08T23:46:25.346299+0000","last_undegraded":"2024-05-08T23:46:25.346299+0000","last_fullsized":"2024-05-08T23:46:25.346299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:46.725976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:26.195804+0000","last_change":"2024-05-08T23:45:03.582121+0000","last_active":"2024-05-08T23:46:26.195804+0000","last_peered":"2024-05-08T23:46:26.195804+0000","last_clean":"2024-05-08T23:46:26.195804+0000","last_became_active":"2024-05-08T23:45:03.581796+0000","last_became_peered":"2024-05-08T23:45:03.581796+0000","last_unstale":"2024-05-08T23:46:26.195804+0000","last_undegraded":"2024-05-08T23:46:26.195804+0000","last_fullsized":"2024-05-08T23:46:26.195804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:55.417404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65082,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":145,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.330922+0000","last_change":"2024-05-08T23:45:04.052993+0000","last_active":"2024-05-08T23:46:25.330922+0000","last_peered":"2024-05-08T23:46:25.330922+0000","last_clean":"2024-05-08T23:46:25.330922+0000","last_became_active":"2024-05-08T23:45:04.052088+0000","last_became_peered":"2024-05-08T23:45:04.052088+0000","last_unstale":"2024-05-08T23:46:25.330922+0000","last_undegraded":"2024-05-08T23:46:25.330922+0000","last_fullsized":"2024-05-08T23:46:25.330922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:44.531649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:46:00.332257+0000","last_change":"2024-05-08T23:45:56.372913+0000","last_active":"2024-05-08T23:46:00.332257+0000","last_peered":"2024-05-08T23:46:00.332257+0000","last_clean":"2024-05-08T23:46:00.332257+0000","last_became_active":"2024-05-08T23:45:56.361402+0000","last_became_peered":"2024-05-08T23:45:56.361402+0000","last_unstale":"2024-05-08T23:46:00.332257+0000","last_undegraded":"2024-05-08T23:46:00.332257+0000","last_fullsized":"2024-05-08T23:46:00.332257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:34.652216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.353482+0000","last_change":"2024-05-08T23:45:04.050294+0000","last_active":"2024-05-08T23:46:25.353482+0000","last_peered":"2024-05-08T23:46:25.353482+0000","last_clean":"2024-05-08T23:46:25.353482+0000","last_became_active":"2024-05-08T23:45:04.049949+0000","last_became_peered":"2024-05-08T23:45:04.049949+0000","last_unstale":"2024-05-08T23:46:25.353482+0000","last_undegraded":"2024-05-08T23:46:25.353482+0000","last_fullsized":"2024-05-08T23:46:25.353482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:17.253201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355432+0000","last_change":"2024-05-08T23:45:03.579821+0000","last_active":"2024-05-08T23:45:56.355432+0000","last_peered":"2024-05-08T23:45:56.355432+0000","last_clean":"2024-05-08T23:45:56.355432+0000","last_became_active":"2024-05-08T23:45:03.578635+0000","last_became_peered":"2024-05-08T23:45:03.578635+0000","last_unstale":"2024-05-08T23:45:56.355432+0000","last_undegraded":"2024-05-08T23:45:56.355432+0000","last_fullsized":"2024-05-08T23:45:56.355432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:21:46.179505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354613+0000","last_change":"2024-05-08T23:45:04.059357+0000","last_active":"2024-05-08T23:46:25.354613+0000","last_peered":"2024-05-08T23:46:25.354613+0000","last_clean":"2024-05-08T23:46:25.354613+0000","last_became_active":"2024-05-08T23:45:04.059070+0000","last_became_peered":"2024-05-08T23:45:04.059070+0000","last_unstale":"2024-05-08T23:46:25.354613+0000","last_undegraded":"2024-05-08T23:46:25.354613+0000","last_fullsized":"2024-05-08T23:46:25.354613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:50.626787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:26.197796+0000","last_change":"2024-05-08T23:45:03.577199+0000","last_active":"2024-05-08T23:46:26.197796+0000","last_peered":"2024-05-08T23:46:26.197796+0000","last_clean":"2024-05-08T23:46:26.197796+0000","last_became_active":"2024-05-08T23:45:03.576844+0000","last_became_peered":"2024-05-08T23:45:03.576844+0000","last_unstale":"2024-05-08T23:46:26.197796+0000","last_undegraded":"2024-05-08T23:46:26.197796+0000","last_fullsized":"2024-05-08T23:46:26.197796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:27.139493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340110+0000","last_change":"2024-05-08T23:45:56.356625+0000","last_active":"2024-05-08T23:46:25.340110+0000","last_peered":"2024-05-08T23:46:25.340110+0000","last_clean":"2024-05-08T23:46:25.340110+0000","last_became_active":"2024-05-08T23:45:56.356240+0000","last_became_peered":"2024-05-08T23:45:56.356240+0000","last_unstale":"2024-05-08T23:46:25.340110+0000","last_undegraded":"2024-05-08T23:46:25.340110+0000","last_fullsized":"2024-05-08T23:46:25.340110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:50.251606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360317+0000","last_change":"2024-05-08T23:45:03.576766+0000","last_active":"2024-05-08T23:45:56.360317+0000","last_peered":"2024-05-08T23:45:56.360317+0000","last_clean":"2024-05-08T23:45:56.360317+0000","last_became_active":"2024-05-08T23:45:03.576410+0000","last_became_peered":"2024-05-08T23:45:03.576410+0000","last_unstale":"2024-05-08T23:45:56.360317+0000","last_undegraded":"2024-05-08T23:45:56.360317+0000","last_fullsized":"2024-05-08T23:45:56.360317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:29.444589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349612+0000","last_change":"2024-05-08T23:45:04.056563+0000","last_active":"2024-05-08T23:46:25.349612+0000","last_peered":"2024-05-08T23:46:25.349612+0000","last_clean":"2024-05-08T23:46:25.349612+0000","last_became_active":"2024-05-08T23:45:04.053132+0000","last_became_peered":"2024-05-08T23:45:04.053132+0000","last_unstale":"2024-05-08T23:46:25.349612+0000","last_undegraded":"2024-05-08T23:46:25.349612+0000","last_fullsized":"2024-05-08T23:46:25.349612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:28:41.625804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:46:22.745519+0000","last_change":"2024-05-08T23:45:03.580343+0000","last_active":"2024-05-08T23:46:22.745519+0000","last_peered":"2024-05-08T23:46:22.745519+0000","last_clean":"2024-05-08T23:46:22.745519+0000","last_became_active":"2024-05-08T23:45:03.578719+0000","last_became_peered":"2024-05-08T23:45:03.578719+0000","last_unstale":"2024-05-08T23:46:22.745519+0000","last_undegraded":"2024-05-08T23:46:22.745519+0000","last_fullsized":"2024-05-08T23:46:22.745519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:54.831252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343273+0000","last_change":"2024-05-08T23:45:04.050643+0000","last_active":"2024-05-08T23:46:25.343273+0000","last_peered":"2024-05-08T23:46:25.343273+0000","last_clean":"2024-05-08T23:46:25.343273+0000","last_became_active":"2024-05-08T23:45:04.050309+0000","last_became_peered":"2024-05-08T23:45:04.050309+0000","last_unstale":"2024-05-08T23:46:25.343273+0000","last_undegraded":"2024-05-08T23:46:25.343273+0000","last_fullsized":"2024-05-08T23:46:25.343273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:30.259530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.352954+0000","last_change":"2024-05-08T23:45:03.579590+0000","last_active":"2024-05-08T23:45:56.352954+0000","last_peered":"2024-05-08T23:45:56.352954+0000","last_clean":"2024-05-08T23:45:56.352954+0000","last_became_active":"2024-05-08T23:45:03.579213+0000","last_became_peered":"2024-05-08T23:45:03.579213+0000","last_unstale":"2024-05-08T23:45:56.352954+0000","last_undegraded":"2024-05-08T23:45:56.352954+0000","last_fullsized":"2024-05-08T23:45:56.352954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:55.666543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.348872+0000","last_change":"2024-05-08T23:45:04.049082+0000","last_active":"2024-05-08T23:46:25.348872+0000","last_peered":"2024-05-08T23:46:25.348872+0000","last_clean":"2024-05-08T23:46:25.348872+0000","last_became_active":"2024-05-08T23:45:04.048709+0000","last_became_peered":"2024-05-08T23:45:04.048709+0000","last_unstale":"2024-05-08T23:46:25.348872+0000","last_undegraded":"2024-05-08T23:46:25.348872+0000","last_fullsized":"2024-05-08T23:46:25.348872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:56.365817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.354871+0000","last_change":"2024-05-08T23:45:03.581363+0000","last_active":"2024-05-08T23:45:56.354871+0000","last_peered":"2024-05-08T23:45:56.354871+0000","last_clean":"2024-05-08T23:45:56.354871+0000","last_became_active":"2024-05-08T23:45:03.580840+0000","last_became_peered":"2024-05-08T23:45:03.580840+0000","last_unstale":"2024-05-08T23:45:56.354871+0000","last_undegraded":"2024-05-08T23:45:56.354871+0000","last_fullsized":"2024-05-08T23:45:56.354871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:27.281228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.345492+0000","last_change":"2024-05-08T23:45:56.361444+0000","last_active":"2024-05-08T23:46:25.345492+0000","last_peered":"2024-05-08T23:46:25.345492+0000","last_clean":"2024-05-08T23:46:25.345492+0000","last_became_active":"2024-05-08T23:45:56.361099+0000","last_became_peered":"2024-05-08T23:45:56.361099+0000","last_unstale":"2024-05-08T23:46:25.345492+0000","last_undegraded":"2024-05-08T23:46:25.345492+0000","last_fullsized":"2024-05-08T23:46:25.345492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:25.342212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349895+0000","last_change":"2024-05-08T23:45:04.061189+0000","last_active":"2024-05-08T23:46:25.349895+0000","last_peered":"2024-05-08T23:46:25.349895+0000","last_clean":"2024-05-08T23:46:25.349895+0000","last_became_active":"2024-05-08T23:45:04.060838+0000","last_became_peered":"2024-05-08T23:45:04.060838+0000","last_unstale":"2024-05-08T23:46:25.349895+0000","last_undegraded":"2024-05-08T23:46:25.349895+0000","last_fullsized":"2024-05-08T23:46:25.349895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:43.794987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":98,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.296864+0000","last_change":"2024-05-08T23:45:03.583841+0000","last_active":"2024-05-08T23:46:25.296864+0000","last_peered":"2024-05-08T23:46:25.296864+0000","last_clean":"2024-05-08T23:46:25.296864+0000","last_became_active":"2024-05-08T23:45:03.583275+0000","last_became_peered":"2024-05-08T23:45:03.583275+0000","last_unstale":"2024-05-08T23:46:25.296864+0000","last_undegraded":"2024-05-08T23:46:25.296864+0000","last_fullsized":"2024-05-08T23:46:25.296864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:14.560505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.339715+0000","last_change":"2024-05-08T23:45:04.052667+0000","last_active":"2024-05-08T23:46:25.339715+0000","last_peered":"2024-05-08T23:46:25.339715+0000","last_clean":"2024-05-08T23:46:25.339715+0000","last_became_active":"2024-05-08T23:45:04.052008+0000","last_became_peered":"2024-05-08T23:45:04.052008+0000","last_unstale":"2024-05-08T23:46:25.339715+0000","last_undegraded":"2024-05-08T23:46:25.339715+0000","last_fullsized":"2024-05-08T23:46:25.339715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:35.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.351515+0000","last_change":"2024-05-08T23:45:03.573665+0000","last_active":"2024-05-08T23:45:56.351515+0000","last_peered":"2024-05-08T23:45:56.351515+0000","last_clean":"2024-05-08T23:45:56.351515+0000","last_became_active":"2024-05-08T23:45:03.573265+0000","last_became_peered":"2024-05-08T23:45:03.573265+0000","last_unstale":"2024-05-08T23:45:56.351515+0000","last_undegraded":"2024-05-08T23:45:56.351515+0000","last_fullsized":"2024-05-08T23:45:56.351515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:13.410954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":333,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355192+0000","last_change":"2024-05-08T23:44:16.373962+0000","last_active":"2024-05-08T23:45:56.355192+0000","last_peered":"2024-05-08T23:45:56.355192+0000","last_clean":"2024-05-08T23:45:56.355192+0000","last_became_active":"2024-05-08T23:44:16.373358+0000","last_became_peered":"2024-05-08T23:44:16.373358+0000","last_unstale":"2024-05-08T23:45:56.355192+0000","last_undegraded":"2024-05-08T23:45:56.355192+0000","last_fullsized":"2024-05-08T23:45:56.355192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:48.067168+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.067168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:24.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349501+0000","last_change":"2024-05-08T23:45:04.048468+0000","last_active":"2024-05-08T23:46:25.349501+0000","last_peered":"2024-05-08T23:46:25.349501+0000","last_clean":"2024-05-08T23:46:25.349501+0000","last_became_active":"2024-05-08T23:45:04.048085+0000","last_became_peered":"2024-05-08T23:45:04.048085+0000","last_unstale":"2024-05-08T23:46:25.349501+0000","last_undegraded":"2024-05-08T23:46:25.349501+0000","last_fullsized":"2024-05-08T23:46:25.349501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:18.039780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.355333+0000","last_change":"2024-05-08T23:45:03.575821+0000","last_active":"2024-05-08T23:45:56.355333+0000","last_peered":"2024-05-08T23:45:56.355333+0000","last_clean":"2024-05-08T23:45:56.355333+0000","last_became_active":"2024-05-08T23:45:03.575407+0000","last_became_peered":"2024-05-08T23:45:03.575407+0000","last_unstale":"2024-05-08T23:45:56.355333+0000","last_undegraded":"2024-05-08T23:45:56.355333+0000","last_fullsized":"2024-05-08T23:45:56.355333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:35.718942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:46:00.332281+0000","last_change":"2024-05-08T23:45:56.365483+0000","last_active":"2024-05-08T23:46:00.332281+0000","last_peered":"2024-05-08T23:46:00.332281+0000","last_clean":"2024-05-08T23:46:00.332281+0000","last_became_active":"2024-05-08T23:45:56.359626+0000","last_became_peered":"2024-05-08T23:45:56.359626+0000","last_unstale":"2024-05-08T23:46:00.332281+0000","last_undegraded":"2024-05-08T23:46:00.332281+0000","last_fullsized":"2024-05-08T23:46:00.332281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:38.284296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.349154+0000","last_change":"2024-05-08T23:45:04.047687+0000","last_active":"2024-05-08T23:46:25.349154+0000","last_peered":"2024-05-08T23:46:25.349154+0000","last_clean":"2024-05-08T23:46:25.349154+0000","last_became_active":"2024-05-08T23:45:04.047343+0000","last_became_peered":"2024-05-08T23:45:04.047343+0000","last_unstale":"2024-05-08T23:46:25.349154+0000","last_undegraded":"2024-05-08T23:46:25.349154+0000","last_fullsized":"2024-05-08T23:46:25.349154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:58.763421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'39","reported_seq":77,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.567780+0000","last_change":"2024-05-08T23:45:03.573344+0000","last_active":"2024-05-08T23:46:25.567780+0000","last_peered":"2024-05-08T23:46:25.567780+0000","last_clean":"2024-05-08T23:46:25.567780+0000","last_became_active":"2024-05-08T23:45:03.572951+0000","last_became_peered":"2024-05-08T23:45:03.572951+0000","last_unstale":"2024-05-08T23:46:25.567780+0000","last_undegraded":"2024-05-08T23:46:25.567780+0000","last_fullsized":"2024-05-08T23:46:25.567780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:04.422908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":68,"num_write_kb":833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.335456+0000","last_change":"2024-05-08T23:45:04.055111+0000","last_active":"2024-05-08T23:46:25.335456+0000","last_peered":"2024-05-08T23:46:25.335456+0000","last_clean":"2024-05-08T23:46:25.335456+0000","last_became_active":"2024-05-08T23:45:04.054665+0000","last_became_peered":"2024-05-08T23:45:04.054665+0000","last_unstale":"2024-05-08T23:46:25.335456+0000","last_undegraded":"2024-05-08T23:46:25.335456+0000","last_fullsized":"2024-05-08T23:46:25.335456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:23.292630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.351606+0000","last_change":"2024-05-08T23:45:03.578168+0000","last_active":"2024-05-08T23:45:56.351606+0000","last_peered":"2024-05-08T23:45:56.351606+0000","last_clean":"2024-05-08T23:45:56.351606+0000","last_became_active":"2024-05-08T23:45:03.577833+0000","last_became_peered":"2024-05-08T23:45:03.577833+0000","last_unstale":"2024-05-08T23:45:56.351606+0000","last_undegraded":"2024-05-08T23:45:56.351606+0000","last_fullsized":"2024-05-08T23:45:56.351606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:13:25.743151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.354570+0000","last_change":"2024-05-08T23:45:04.049103+0000","last_active":"2024-05-08T23:46:25.354570+0000","last_peered":"2024-05-08T23:46:25.354570+0000","last_clean":"2024-05-08T23:46:25.354570+0000","last_became_active":"2024-05-08T23:45:04.048302+0000","last_became_peered":"2024-05-08T23:45:04.048302+0000","last_unstale":"2024-05-08T23:46:25.354570+0000","last_undegraded":"2024-05-08T23:46:25.354570+0000","last_fullsized":"2024-05-08T23:46:25.354570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:14:02.404522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.614524+0000","last_change":"2024-05-08T23:45:03.577765+0000","last_active":"2024-05-08T23:45:56.614524+0000","last_peered":"2024-05-08T23:45:56.614524+0000","last_clean":"2024-05-08T23:45:56.614524+0000","last_became_active":"2024-05-08T23:45:03.577452+0000","last_became_peered":"2024-05-08T23:45:03.577452+0000","last_unstale":"2024-05-08T23:45:56.614524+0000","last_undegraded":"2024-05-08T23:45:56.614524+0000","last_fullsized":"2024-05-08T23:45:56.614524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:28:02.107878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.315831+0000","last_change":"2024-05-08T23:45:04.061360+0000","last_active":"2024-05-08T23:46:25.315831+0000","last_peered":"2024-05-08T23:46:25.315831+0000","last_clean":"2024-05-08T23:46:25.315831+0000","last_became_active":"2024-05-08T23:45:04.061000+0000","last_became_peered":"2024-05-08T23:45:04.061000+0000","last_unstale":"2024-05-08T23:46:25.315831+0000","last_undegraded":"2024-05-08T23:46:25.315831+0000","last_fullsized":"2024-05-08T23:46:25.315831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:35.757571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360024+0000","last_change":"2024-05-08T23:45:03.578769+0000","last_active":"2024-05-08T23:45:56.360024+0000","last_peered":"2024-05-08T23:45:56.360024+0000","last_clean":"2024-05-08T23:45:56.360024+0000","last_became_active":"2024-05-08T23:45:03.578359+0000","last_became_peered":"2024-05-08T23:45:03.578359+0000","last_unstale":"2024-05-08T23:45:56.360024+0000","last_undegraded":"2024-05-08T23:45:56.360024+0000","last_fullsized":"2024-05-08T23:45:56.360024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:13.213382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333169+0000","last_change":"2024-05-08T23:45:04.049237+0000","last_active":"2024-05-08T23:46:25.333169+0000","last_peered":"2024-05-08T23:46:25.333169+0000","last_clean":"2024-05-08T23:46:25.333169+0000","last_became_active":"2024-05-08T23:45:04.048922+0000","last_became_peered":"2024-05-08T23:45:04.048922+0000","last_unstale":"2024-05-08T23:46:25.333169+0000","last_undegraded":"2024-05-08T23:46:25.333169+0000","last_fullsized":"2024-05-08T23:46:25.333169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:29.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.354233+0000","last_change":"2024-05-08T23:45:03.574990+0000","last_active":"2024-05-08T23:45:56.354233+0000","last_peered":"2024-05-08T23:45:56.354233+0000","last_clean":"2024-05-08T23:45:56.354233+0000","last_became_active":"2024-05-08T23:45:03.574617+0000","last_became_peered":"2024-05-08T23:45:03.574617+0000","last_unstale":"2024-05-08T23:45:56.354233+0000","last_undegraded":"2024-05-08T23:45:56.354233+0000","last_fullsized":"2024-05-08T23:45:56.354233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:51.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.337429+0000","last_change":"2024-05-08T23:45:04.054371+0000","last_active":"2024-05-08T23:46:25.337429+0000","last_peered":"2024-05-08T23:46:25.337429+0000","last_clean":"2024-05-08T23:46:25.337429+0000","last_became_active":"2024-05-08T23:45:04.054070+0000","last_became_peered":"2024-05-08T23:45:04.054070+0000","last_unstale":"2024-05-08T23:46:25.337429+0000","last_undegraded":"2024-05-08T23:46:25.337429+0000","last_fullsized":"2024-05-08T23:46:25.337429+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:27.926522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.360465+0000","last_change":"2024-05-08T23:45:03.579413+0000","last_active":"2024-05-08T23:45:56.360465+0000","last_peered":"2024-05-08T23:45:56.360465+0000","last_clean":"2024-05-08T23:45:56.360465+0000","last_became_active":"2024-05-08T23:45:03.577705+0000","last_became_peered":"2024-05-08T23:45:03.577705+0000","last_unstale":"2024-05-08T23:45:56.360465+0000","last_undegraded":"2024-05-08T23:45:56.360465+0000","last_fullsized":"2024-05-08T23:45:56.360465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:29.615247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.353876+0000","last_change":"2024-05-08T23:45:04.050492+0000","last_active":"2024-05-08T23:46:25.353876+0000","last_peered":"2024-05-08T23:46:25.353876+0000","last_clean":"2024-05-08T23:46:25.353876+0000","last_became_active":"2024-05-08T23:45:04.050176+0000","last_became_peered":"2024-05-08T23:45:04.050176+0000","last_unstale":"2024-05-08T23:46:25.353876+0000","last_undegraded":"2024-05-08T23:46:25.353876+0000","last_fullsized":"2024-05-08T23:46:25.353876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:01.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.354571+0000","last_change":"2024-05-08T23:45:03.577651+0000","last_active":"2024-05-08T23:45:56.354571+0000","last_peered":"2024-05-08T23:45:56.354571+0000","last_clean":"2024-05-08T23:45:56.354571+0000","last_became_active":"2024-05-08T23:45:03.576887+0000","last_became_peered":"2024-05-08T23:45:03.576887+0000","last_unstale":"2024-05-08T23:45:56.354571+0000","last_undegraded":"2024-05-08T23:45:56.354571+0000","last_fullsized":"2024-05-08T23:45:56.354571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:52.822959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342689+0000","last_change":"2024-05-08T23:45:04.049793+0000","last_active":"2024-05-08T23:46:25.342689+0000","last_peered":"2024-05-08T23:46:25.342689+0000","last_clean":"2024-05-08T23:46:25.342689+0000","last_became_active":"2024-05-08T23:45:04.049456+0000","last_became_peered":"2024-05-08T23:45:04.049456+0000","last_unstale":"2024-05-08T23:46:25.342689+0000","last_undegraded":"2024-05-08T23:46:25.342689+0000","last_fullsized":"2024-05-08T23:46:25.342689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:16.030406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":62,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.564007+0000","last_change":"2024-05-08T23:45:03.580371+0000","last_active":"2024-05-08T23:46:25.564007+0000","last_peered":"2024-05-08T23:46:25.564007+0000","last_clean":"2024-05-08T23:46:25.564007+0000","last_became_active":"2024-05-08T23:45:03.579394+0000","last_became_peered":"2024-05-08T23:45:03.579394+0000","last_unstale":"2024-05-08T23:46:25.564007+0000","last_undegraded":"2024-05-08T23:46:25.564007+0000","last_fullsized":"2024-05-08T23:46:25.564007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:31.123632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343270+0000","last_change":"2024-05-08T23:45:04.063138+0000","last_active":"2024-05-08T23:46:25.343270+0000","last_peered":"2024-05-08T23:46:25.343270+0000","last_clean":"2024-05-08T23:46:25.343270+0000","last_became_active":"2024-05-08T23:45:04.062835+0000","last_became_peered":"2024-05-08T23:45:04.062835+0000","last_unstale":"2024-05-08T23:46:25.343270+0000","last_undegraded":"2024-05-08T23:46:25.343270+0000","last_fullsized":"2024-05-08T23:46:25.343270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:49.538896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:45:56.351022+0000","last_change":"2024-05-08T23:45:03.584351+0000","last_active":"2024-05-08T23:45:56.351022+0000","last_peered":"2024-05-08T23:45:56.351022+0000","last_clean":"2024-05-08T23:45:56.351022+0000","last_became_active":"2024-05-08T23:45:03.583996+0000","last_became_peered":"2024-05-08T23:45:03.583996+0000","last_unstale":"2024-05-08T23:45:56.351022+0000","last_undegraded":"2024-05-08T23:45:56.351022+0000","last_fullsized":"2024-05-08T23:45:56.351022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:02.548920+0000","last_clean_scrub_stamp":"2024-05-08T23:45:02.548920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:40.530921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342375+0000","last_change":"2024-05-08T23:45:04.048918+0000","last_active":"2024-05-08T23:46:25.342375+0000","last_peered":"2024-05-08T23:46:25.342375+0000","last_clean":"2024-05-08T23:46:25.342375+0000","last_became_active":"2024-05-08T23:45:04.048550+0000","last_became_peered":"2024-05-08T23:45:04.048550+0000","last_unstale":"2024-05-08T23:46:25.342375+0000","last_undegraded":"2024-05-08T23:46:25.342375+0000","last_fullsized":"2024-05-08T23:46:25.342375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:34:48.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342987+0000","last_change":"2024-05-08T23:45:04.050790+0000","last_active":"2024-05-08T23:46:25.342987+0000","last_peered":"2024-05-08T23:46:25.342987+0000","last_clean":"2024-05-08T23:46:25.342987+0000","last_became_active":"2024-05-08T23:45:04.050429+0000","last_became_peered":"2024-05-08T23:45:04.050429+0000","last_unstale":"2024-05-08T23:46:25.342987+0000","last_undegraded":"2024-05-08T23:46:25.342987+0000","last_fullsized":"2024-05-08T23:46:25.342987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:08:43.224351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343805+0000","last_change":"2024-05-08T23:45:56.361737+0000","last_active":"2024-05-08T23:46:25.343805+0000","last_peered":"2024-05-08T23:46:25.343805+0000","last_clean":"2024-05-08T23:46:25.343805+0000","last_became_active":"2024-05-08T23:45:56.361305+0000","last_became_peered":"2024-05-08T23:45:56.361305+0000","last_unstale":"2024-05-08T23:46:25.343805+0000","last_undegraded":"2024-05-08T23:46:25.343805+0000","last_fullsized":"2024-05-08T23:46:25.343805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:55.163729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333222+0000","last_change":"2024-05-08T23:45:04.049707+0000","last_active":"2024-05-08T23:46:25.333222+0000","last_peered":"2024-05-08T23:46:25.333222+0000","last_clean":"2024-05-08T23:46:25.333222+0000","last_became_active":"2024-05-08T23:45:04.049385+0000","last_became_peered":"2024-05-08T23:45:04.049385+0000","last_unstale":"2024-05-08T23:46:25.333222+0000","last_undegraded":"2024-05-08T23:46:25.333222+0000","last_fullsized":"2024-05-08T23:46:25.333222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.350792+0000","last_change":"2024-05-08T23:45:04.050721+0000","last_active":"2024-05-08T23:46:25.350792+0000","last_peered":"2024-05-08T23:46:25.350792+0000","last_clean":"2024-05-08T23:46:25.350792+0000","last_became_active":"2024-05-08T23:45:04.050368+0000","last_became_peered":"2024-05-08T23:45:04.050368+0000","last_unstale":"2024-05-08T23:46:25.350792+0000","last_undegraded":"2024-05-08T23:46:25.350792+0000","last_fullsized":"2024-05-08T23:46:25.350792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:20.815754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.352647+0000","last_change":"2024-05-08T23:45:04.048431+0000","last_active":"2024-05-08T23:46:25.352647+0000","last_peered":"2024-05-08T23:46:25.352647+0000","last_clean":"2024-05-08T23:46:25.352647+0000","last_became_active":"2024-05-08T23:45:04.048071+0000","last_became_peered":"2024-05-08T23:45:04.048071+0000","last_unstale":"2024-05-08T23:46:25.352647+0000","last_undegraded":"2024-05-08T23:46:25.352647+0000","last_fullsized":"2024-05-08T23:46:25.352647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:51.547256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342668+0000","last_change":"2024-05-08T23:45:04.045256+0000","last_active":"2024-05-08T23:46:25.342668+0000","last_peered":"2024-05-08T23:46:25.342668+0000","last_clean":"2024-05-08T23:46:25.342668+0000","last_became_active":"2024-05-08T23:45:04.044973+0000","last_became_peered":"2024-05-08T23:45:04.044973+0000","last_unstale":"2024-05-08T23:46:25.342668+0000","last_undegraded":"2024-05-08T23:46:25.342668+0000","last_fullsized":"2024-05-08T23:46:25.342668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:12:06.021001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.342519+0000","last_change":"2024-05-08T23:45:04.058287+0000","last_active":"2024-05-08T23:46:25.342519+0000","last_peered":"2024-05-08T23:46:25.342519+0000","last_clean":"2024-05-08T23:46:25.342519+0000","last_became_active":"2024-05-08T23:45:04.057977+0000","last_became_peered":"2024-05-08T23:45:04.057977+0000","last_unstale":"2024-05-08T23:46:25.342519+0000","last_undegraded":"2024-05-08T23:46:25.342519+0000","last_fullsized":"2024-05-08T23:46:25.342519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:01.146838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.333669+0000","last_change":"2024-05-08T23:45:04.055780+0000","last_active":"2024-05-08T23:46:25.333669+0000","last_peered":"2024-05-08T23:46:25.333669+0000","last_clean":"2024-05-08T23:46:25.333669+0000","last_became_active":"2024-05-08T23:45:04.055272+0000","last_became_peered":"2024-05-08T23:45:04.055272+0000","last_unstale":"2024-05-08T23:46:25.333669+0000","last_undegraded":"2024-05-08T23:46:25.333669+0000","last_fullsized":"2024-05-08T23:46:25.333669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:28.587333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.343971+0000","last_change":"2024-05-08T23:45:04.050648+0000","last_active":"2024-05-08T23:46:25.343971+0000","last_peered":"2024-05-08T23:46:25.343971+0000","last_clean":"2024-05-08T23:46:25.343971+0000","last_became_active":"2024-05-08T23:45:04.050353+0000","last_became_peered":"2024-05-08T23:45:04.050353+0000","last_unstale":"2024-05-08T23:46:25.343971+0000","last_undegraded":"2024-05-08T23:46:25.343971+0000","last_fullsized":"2024-05-08T23:46:25.343971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:24:58.154006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.339378+0000","last_change":"2024-05-08T23:45:04.056951+0000","last_active":"2024-05-08T23:46:25.339378+0000","last_peered":"2024-05-08T23:46:25.339378+0000","last_clean":"2024-05-08T23:46:25.339378+0000","last_became_active":"2024-05-08T23:45:04.056585+0000","last_became_peered":"2024-05-08T23:45:04.056585+0000","last_unstale":"2024-05-08T23:46:25.339378+0000","last_undegraded":"2024-05-08T23:46:25.339378+0000","last_fullsized":"2024-05-08T23:46:25.339378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:22.658629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.340138+0000","last_change":"2024-05-08T23:45:04.052396+0000","last_active":"2024-05-08T23:46:25.340138+0000","last_peered":"2024-05-08T23:46:25.340138+0000","last_clean":"2024-05-08T23:46:25.340138+0000","last_became_active":"2024-05-08T23:45:04.051604+0000","last_became_peered":"2024-05-08T23:45:04.051604+0000","last_unstale":"2024-05-08T23:46:25.340138+0000","last_undegraded":"2024-05-08T23:46:25.340138+0000","last_fullsized":"2024-05-08T23:46:25.340138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:52.602830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.352889+0000","last_change":"2024-05-08T23:45:04.051578+0000","last_active":"2024-05-08T23:46:25.352889+0000","last_peered":"2024-05-08T23:46:25.352889+0000","last_clean":"2024-05-08T23:46:25.352889+0000","last_became_active":"2024-05-08T23:45:04.051298+0000","last_became_peered":"2024-05-08T23:45:04.051298+0000","last_unstale":"2024-05-08T23:46:25.352889+0000","last_undegraded":"2024-05-08T23:46:25.352889+0000","last_fullsized":"2024-05-08T23:46:25.352889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:47.204849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:46:25.346483+0000","last_change":"2024-05-08T23:45:04.051264+0000","last_active":"2024-05-08T23:46:25.346483+0000","last_peered":"2024-05-08T23:46:25.346483+0000","last_clean":"2024-05-08T23:46:25.346483+0000","last_became_active":"2024-05-08T23:45:04.050870+0000","last_became_peered":"2024-05-08T23:45:04.050870+0000","last_unstale":"2024-05-08T23:46:25.346483+0000","last_undegraded":"2024-05-08T23:46:25.346483+0000","last_fullsized":"2024-05-08T23:46:25.346483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:45:03.551269+0000","last_clean_scrub_stamp":"2024-05-08T23:45:03.551269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:09.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":594119,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":587,"num_read_kb":399,"num_write":267,"num_write_kb":1424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2015232,"data_stored":1782357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514522,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":925696,"data_stored":447350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.497},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48099999999999998}]},{"osd":1,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.44},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.63400000000000001}]},{"osd":7,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.628},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.61699999999999999}]},{"osd":8,"last update":"Wed May 8 23:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66700000000000004}]},{"osd":9,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.73999999999999999}]},{"osd":10,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.64100000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743453,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1748992,"data_stored":1235749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.49099999999999999}]},{"osd":1,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.439,"15min":0.442},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66400000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.33000000000000002}]},{"osd":2,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.80600000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78500000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.45900000000000002}]},{"osd":3,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.46999999999999997}]},{"osd":4,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.439},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.40100000000000002}]},{"osd":5,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80900000000000005,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.505}]},{"osd":6,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.69499999999999995,"15min":0.71199999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79500000000000004,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81200000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88800000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92800000000000005,"5min":1.236,"15min":1.236},"last":0.61099999999999999}]},{"osd":9,"last update":"Wed May 8 23:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.78100000000000003,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88200000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89200000000000002,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.63900000000000001}]},{"osd":11,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.629}]}]},{"osd":9,"up_from":64,"seq":274877906977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28316,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716836,"statfs":{"total":95995035648,"available":95966040064,"internally_reserved":0,"allocated":1327104,"data_stored":840746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63700000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.70899999999999996}]},{"osd":3,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55800000000000005,"15min":0.56599999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.66400000000000003}]},{"osd":4,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93700000000000006,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84999999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.91800000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.748},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.502,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":1.212,"15min":1.212},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.74399999999999999,"15min":0.751},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.012,"5min":1.03,"15min":1.03},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76900000000000002,"15min":0.77600000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.999,"5min":1.117,"15min":1.117},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Wed May 8 23:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0089999999999999,"5min":1.139,"15min":1.139},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.624,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95199999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.878}]},{"osd":10,"last update":"Wed May 8 23:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77100000000000002,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.63,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.98699999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.92900000000000005}]}]},{"osd":8,"up_from":56,"seq":240518168613,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32404,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712748,"statfs":{"total":95995035648,"available":95961853952,"internally_reserved":0,"allocated":1318912,"data_stored":840313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.63100000000000001,"15min":0.59599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.2489999999999997,"5min":6.2489999999999997,"15min":6.2489999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.3869999999999996,"5min":6.3869999999999996,"15min":6.3869999999999996},"last":0.39900000000000002}]},{"osd":1,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52400000000000002}]},{"osd":2,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.33500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.45200000000000001,"15min":0.44500000000000001},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.65300000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38100000000000001}]},{"osd":3,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.65800000000000003,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.593,"5min":6.593,"15min":6.593},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.63900000000000001,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.5039999999999996,"5min":6.5039999999999996,"15min":6.5039999999999996},"last":0.65600000000000003}]},{"osd":4,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.626,"15min":0.622},"min":{"1min":0.45700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.495}]},{"osd":5,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72399999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.46700000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.52200000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.55200000000000005}]},{"osd":7,"last update":"Wed May 8 23:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93700000000000006,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60199999999999998}]},{"osd":9,"last update":"Wed May 8 23:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.70199999999999996,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.73099999999999998,"15min":0.70999999999999996},"min":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.59299999999999997}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.72799999999999998,"15min":0.71899999999999997},"min":{"1min":0.52300000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61599999999999999}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.627}]}]},{"osd":7,"up_from":49,"seq":210453397544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1523712,"data_stored":1039269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.437,"15min":0.44500000000000001},"min":{"1min":0.27300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.51600000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.30099999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.54400000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.753,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.432},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76700000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45400000000000001}]},{"osd":3,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.59899999999999998},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71599999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83299999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.754,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.501,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.64500000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.34300000000000003}]},{"osd":5,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.624,"15min":0.628},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76000000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80000000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.69399999999999995,"15min":0.70799999999999996},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.83599999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77100000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.48599999999999999}]},{"osd":8,"last update":"Wed May 8 23:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.502},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.89600000000000002,"5min":1.016,"15min":1.016},"last":0.58399999999999996}]},{"osd":9,"last update":"Wed May 8 23:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.58799999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.91700000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.48599999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.89500000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.57599999999999996}]},{"osd":10,"last update":"Wed May 8 23:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.501,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.78400000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.70199999999999996,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81899999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.52500000000000002}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63}]}]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":446919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65000000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.438,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68999999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41299999999999998}]},{"osd":1,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.65600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88200000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90000000000000002,"5min":1.288,"15min":1.288},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":1.018,"15min":1.018},"last":0.498},{"interface":"front","average":{"1min":0.503,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81899999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.376}]},{"osd":4,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":1.113,"15min":1.113},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98399999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed May 8 23:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.49299999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96499999999999997,"5min":1.002,"15min":1.002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82099999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed May 8 23:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.88800000000000001,"5min":1.321,"15min":1.321},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.52000000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.74099999999999999}]},{"osd":8,"last update":"Wed May 8 23:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0629999999999999,"5min":1.111,"15min":1.111},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.02,"5min":1.05,"15min":1.05},"last":0.60599999999999998}]},{"osd":9,"last update":"Wed May 8 23:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.85499999999999998,"15min":0.872},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0940000000000001,"5min":1.095,"15min":1.095},"last":0.67300000000000004}]},{"osd":10,"last update":"Wed May 8 23:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.78600000000000003},"min":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.75},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.107,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.78500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":978944,"data_stored":510368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.55600000000000005,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.96,"5min":3.96,"15min":3.96},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.58099999999999996,"15min":0.51900000000000002},"min":{"1min":0.44400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed May 8 23:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed May 8 23:46:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.60099999999999998,"15min":0.54900000000000004},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.101,"5min":4.101,"15min":4.101},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.63900000000000001,"15min":0.55700000000000005},"min":{"1min":0.51400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.1820000000000004,"5min":4.1820000000000004,"15min":4.1820000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed May 8 23:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41799999999999998,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.84399999999999997,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.37},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.40699999999999997},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68400000000000005,"5min":0.88,"15min":0.88},"last":0.32200000000000001}]},{"osd":5,"last update":"Wed May 8 23:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.57299999999999995,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.68300000000000005,"15min":0.65700000000000003},"min":{"1min":0.49199999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.7549999999999999,"5min":4.7549999999999999,"15min":4.7549999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.175,"5min":0.747,"15min":0.66700000000000004},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Wed May 8 23:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55300000000000005,"15min":0.51800000000000002},"min":{"1min":0.39800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92900000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60299999999999998,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54500000000000004}]},{"osd":8,"last update":"Wed May 8 23:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52900000000000003}]},{"osd":9,"last update":"Wed May 8 23:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.9240000000000004,"5min":4.9240000000000004,"15min":4.9240000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.81999999999999995,"15min":0.78000000000000003},"min":{"1min":0.57499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":6.3769999999999998,"5min":6.3769999999999998,"15min":6.3769999999999998},"last":0.64900000000000002}]},{"osd":10,"last update":"Wed May 8 23:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0589999999999999,"5min":1.095,"15min":1.095},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.46800000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96499999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59699999999999998}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.626},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":447823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.54800000000000004,"15min":0.50600000000000001},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.36299999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":1.032,"15min":1.032},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed May 8 23:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56699999999999995,"15min":0.52700000000000002},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.60599999999999998,"15min":0.54800000000000004},"min":{"1min":0.44700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.625},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":3.1099999999999999,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed May 8 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59099999999999997,"15min":0.56200000000000006},"min":{"1min":0.51100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.65500000000000003,"15min":0.621},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed May 8 23:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.66300000000000003,"15min":0.627},"min":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Wed May 8 23:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.64200000000000002,"15min":0.58999999999999997},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":6.0949999999999998,"5min":6.0949999999999998,"15min":6.0949999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.70599999999999996,"15min":0.65100000000000002},"min":{"1min":0.39500000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":6.1989999999999998,"5min":6.1989999999999998,"15min":6.1989999999999998},"last":0.746}]},{"osd":7,"last update":"Wed May 8 23:44:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80400000000000005,"15min":0.80100000000000005},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.47299999999999998}]},{"osd":9,"last update":"Wed May 8 23:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.46700000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98099999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83099999999999996,"15min":0.83299999999999996},"min":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.78900000000000003}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.82799999999999996},"min":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.96699999999999997,"5min":1.127,"15min":1.127},"last":0.90300000000000002}]},{"osd":11,"last update":"Wed May 8 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.64100000000000001}]}]},{"osd":2,"up_from":17,"seq":73014444091,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":999424,"data_stored":512422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82799999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54400000000000004,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.70099999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.52600000000000002,"15min":0.48999999999999999},"min":{"1min":0.40799999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83199999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.5},"min":{"1min":0.38300000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.80600000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64400000000000002}]},{"osd":3,"last update":"Wed May 8 23:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55300000000000005,"15min":0.52600000000000002},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54600000000000004,"15min":0.52400000000000002},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68799999999999994}]},{"osd":5,"last update":"Wed May 8 23:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.45500000000000002},"min":{"1min":0.34499999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.53200000000000003,"15min":0.51100000000000001},"min":{"1min":0.45400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed May 8 23:46:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64100000000000001,"15min":0.629},"min":{"1min":0.51200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.77900000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.46899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90400000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed May 8 23:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.42899999999999999}]},{"osd":9,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.627},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.63100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63600000000000001}]},{"osd":10,"last update":"Wed May 8 23:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.51500000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85699999999999998,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.92200000000000004,"15min":0.95799999999999996},"min":{"1min":0.58599999999999997,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93899999999999995,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.69699999999999995}]},{"osd":11,"last update":"Wed May 8 23:46:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.66400000000000003}]}]},{"osd":3,"up_from":24,"seq":103079215159,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":921600,"data_stored":438275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.443,"15min":0.39300000000000002},"min":{"1min":0.312,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.4790000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.434,"15min":0.39800000000000002},"min":{"1min":0.309,"5min":0.248,"15min":0.248},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.497}]},{"osd":1,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.57899999999999996,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.48299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.5459999999999998,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":0.72399999999999998}]},{"osd":2,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.65100000000000002,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.64800000000000002,"15min":0.60199999999999998},"min":{"1min":0.47099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed May 8 23:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.60799999999999998,"15min":0.56999999999999995},"min":{"1min":0.51100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69899999999999995}]},{"osd":5,"last update":"Wed May 8 23:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.372},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.64500000000000002,"15min":0.61099999999999999},"min":{"1min":0.53000000000000003,"5min":0.371,"15min":0.371},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.441}]},{"osd":6,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.59999999999999998,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.0950000000000002,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":0.41899999999999998}]},{"osd":7,"last update":"Wed May 8 23:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70599999999999996,"15min":0.66000000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9119999999999999,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.745,"15min":0.71699999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.63500000000000001}]},{"osd":8,"last update":"Wed May 8 23:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77600000000000002}]},{"osd":9,"last update":"Wed May 8 23:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.76500000000000001,"15min":0.748},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.431,"5min":3.431,"15min":3.431},"last":0.56299999999999994}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.84699999999999998,"15min":0.81899999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.874,"15min":0.86399999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.67600000000000005}]},{"osd":11,"last update":"Wed May 8 23:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.617,"5min":3.617,"15min":3.617},"last":0.624}]}]},{"osd":4,"up_from":31,"seq":133143986225,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":447375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:44:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61799999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.377},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.67700000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.307}]},{"osd":2,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.40699999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81999999999999995,"5min":1.222,"15min":1.222},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.62}]},{"osd":3,"last update":"Wed May 8 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.438},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84899999999999998,"5min":0.874,"15min":0.874},"last":0.41699999999999998}]},{"osd":5,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80900000000000005,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78900000000000003,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48099999999999998}]},{"osd":7,"last update":"Wed May 8 23:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.82499999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53200000000000003,"15min":0.51700000000000002},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76700000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.495}]},{"osd":8,"last update":"Wed May 8 23:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51000000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85499999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91100000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.53000000000000003}]},{"osd":9,"last update":"Wed May 8 23:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.71099999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90600000000000003,"5min":0.997,"15min":0.997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92100000000000004,"5min":1.115,"15min":1.115},"last":0.64600000000000002}]},{"osd":10,"last update":"Wed May 8 23:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89400000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.48699999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76400000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55800000000000005}]},{"osd":11,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60399999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1544192,"data_stored":1039137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.51200000000000001,"15min":0.47899999999999998},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80100000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.436},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.34699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.50600000000000001},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70699999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.68100000000000005,"5min":0.873,"15min":0.873},"last":0.57699999999999996}]},{"osd":2,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.372,"15min":0.35899999999999999},"min":{"1min":0.27100000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.40500000000000003},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.63400000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.34399999999999997}]},{"osd":3,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.48599999999999999},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74299999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.501},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70699999999999996}]},{"osd":4,"last update":"Wed May 8 23:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.55500000000000005},"min":{"1min":0.433,"5min":0.373,"15min":0.373},"max":{"1min":0.872,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89100000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed May 8 23:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.49299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82099999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.45800000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.61699999999999999}]},{"osd":7,"last update":"Wed May 8 23:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59899999999999998,"15min":0.58499999999999996},"min":{"1min":0.46100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63200000000000001,"15min":0.62},"min":{"1min":0.5,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91000000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67700000000000005}]},{"osd":8,"last update":"Wed May 8 23:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85399999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46600000000000003}]},{"osd":9,"last update":"Wed May 8 23:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92600000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.878,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63700000000000001}]},{"osd":10,"last update":"Wed May 8 23:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.76900000000000002},"min":{"1min":0.58999999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.88600000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.76400000000000001},"min":{"1min":0.60699999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93700000000000006,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":198919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:46:31.377 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:46:31.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.556+0000 7fd92a1c3700 1 -- 172.21.15.12:0/439640121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9241028c0 msgr2=0x7fd92410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.556+0000 7fd92a1c3700 1 --2- 172.21.15.12:0/439640121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9241028c0 0x7fd92410ada0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009990 tx=0x7fd90c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.556+0000 7fd92a1c3700 1 -- 172.21.15.12:0/439640121 shutdown_connections 2024-05-08T23:46:31.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.556+0000 7fd92a1c3700 1 --2- 172.21.15.12:0/439640121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd92410b3c0 0x7fd92410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.556+0000 7fd92a1c3700 1 --2- 172.21.15.12:0/439640121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd924101fa0 0x7fd924102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.556+0000 7fd92a1c3700 1 --2- 172.21.15.12:0/439640121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9241028c0 0x7fd92410ada0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.556+0000 7fd92a1c3700 1 -- 172.21.15.12:0/439640121 >> 172.21.15.12:0/439640121 conn(0x7fd9240fb7e0 msgr2=0x7fd9240fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:31.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.556+0000 7fd92a1c3700 1 -- 172.21.15.12:0/439640121 shutdown_connections 2024-05-08T23:46:31.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.556+0000 7fd92a1c3700 1 -- 172.21.15.12:0/439640121 wait complete. 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.557+0000 7fd92a1c3700 1 Processor -- start 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.557+0000 7fd92a1c3700 1 -- start start 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd92a1c3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd924101fa0 0x7fd924197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd92a1c3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9241028c0 0x7fd924197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd92a1c3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd92410b3c0 0x7fd92419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd92a1c3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92419c1e0 con 0x7fd92410b3c0 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd92a1c3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92419c350 con 0x7fd924101fa0 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd92a1c3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92419c510 con 0x7fd9241028c0 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd923fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd92410b3c0 0x7fd92419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd923fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd92410b3c0 0x7fd92419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48280/0 (socket says 172.21.15.12:48280) 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd9237fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd924101fa0 0x7fd924197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd923fff700 1 -- 172.21.15.12:0/977532856 learned_addr learned my addr 172.21.15.12:0/977532856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:31.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd922ffd700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9241028c0 0x7fd924197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd923fff700 1 -- 172.21.15.12:0/977532856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9241028c0 msgr2=0x7fd924197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd923fff700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9241028c0 0x7fd924197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd923fff700 1 -- 172.21.15.12:0/977532856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd924101fa0 msgr2=0x7fd924197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd923fff700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd924101fa0 0x7fd924197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd923fff700 1 -- 172.21.15.12:0/977532856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c0095f0 con 0x7fd92410b3c0 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd9237fe700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd924101fa0 0x7fd924197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.558+0000 7fd923fff700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd92410b3c0 0x7fd92419bba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00ef30 tx=0x7fd91c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.559+0000 7fd920ff9700 1 -- 172.21.15.12:0/977532856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c018070 con 0x7fd92410b3c0 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.559+0000 7fd920ff9700 1 -- 172.21.15.12:0/977532856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd91c00f040 con 0x7fd92410b3c0 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.559+0000 7fd92a1c3700 1 -- 172.21.15.12:0/977532856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92419c790 con 0x7fd92410b3c0 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.559+0000 7fd920ff9700 1 -- 172.21.15.12:0/977532856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00cd00 con 0x7fd92410b3c0 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.559+0000 7fd92a1c3700 1 -- 172.21.15.12:0/977532856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9241a4030 con 0x7fd92410b3c0 2024-05-08T23:46:31.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.560+0000 7fd92a1c3700 1 -- 172.21.15.12:0/977532856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd924106490 con 0x7fd92410b3c0 2024-05-08T23:46:31.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.561+0000 7fd920ff9700 1 -- 172.21.15.12:0/977532856 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7fd91c003a00 con 0x7fd92410b3c0 2024-05-08T23:46:31.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.562+0000 7fd920ff9700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd9100a13a0 0x7fd9100a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.562+0000 7fd920ff9700 1 -- 172.21.15.12:0/977532856 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd91c0cd2c0 con 0x7fd92410b3c0 2024-05-08T23:46:31.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.562+0000 7fd9237fe700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd9100a13a0 0x7fd9100a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.563+0000 7fd9237fe700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd9100a13a0 0x7fd9100a3860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd914011490 tx=0x7fd914009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:31.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.566+0000 7fd920ff9700 1 -- 172.21.15.12:0/977532856 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd91c096610 con 0x7fd92410b3c0 2024-05-08T23:46:31.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.787+0000 7fd92a1c3700 1 -- 172.21.15.12:0/977532856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9240688d0 con 0x7fd92410b3c0 2024-05-08T23:46:31.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.789+0000 7fd920ff9700 1 -- 172.21.15.12:0/977532856 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22884 (secure 0 0 0) 0x7fd91c099c30 con 0x7fd92410b3c0 2024-05-08T23:46:31.785 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":84,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:46:25.291136+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3622028746":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3062429456":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2381834553":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/716473668":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2695290437":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/1305168755":"2024-05-09T23:46:25.291064+0000","172.21.15.12:6801/1305168755":"2024-05-09T23:46:25.291064+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:46:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.791+0000 7fd92a1c3700 1 -- 172.21.15.12:0/977532856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd9100a13a0 msgr2=0x7fd9100a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.791+0000 7fd92a1c3700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd9100a13a0 0x7fd9100a3860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd914011490 tx=0x7fd914009260 comp rx=0 tx=0).stop 2024-05-08T23:46:31.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.791+0000 7fd92a1c3700 1 -- 172.21.15.12:0/977532856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd92410b3c0 msgr2=0x7fd92419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.791+0000 7fd92a1c3700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd92410b3c0 0x7fd92419bba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00ef30 tx=0x7fd91c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.791+0000 7fd92a1c3700 1 -- 172.21.15.12:0/977532856 shutdown_connections 2024-05-08T23:46:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.791+0000 7fd92a1c3700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd9100a13a0 0x7fd9100a3860 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.791+0000 7fd92a1c3700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd92410b3c0 0x7fd92419bba0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.791+0000 7fd92a1c3700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd924101fa0 0x7fd924197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.791+0000 7fd92a1c3700 1 --2- 172.21.15.12:0/977532856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9241028c0 0x7fd924197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.791+0000 7fd92a1c3700 1 -- 172.21.15.12:0/977532856 >> 172.21.15.12:0/977532856 conn(0x7fd9240fb7e0 msgr2=0x7fd9240fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.792+0000 7fd92a1c3700 1 -- 172.21.15.12:0/977532856 shutdown_connections 2024-05-08T23:46:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.792+0000 7fd92a1c3700 1 -- 172.21.15.12:0/977532856 wait complete. 2024-05-08T23:46:31.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:45:02.165326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:45:03.407498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:46:31.802 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:46:31.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.946+0000 7fb466d6e700 1 -- 172.21.15.12:0/1567024307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460106280 msgr2=0x7fb46010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.946+0000 7fb466d6e700 1 --2- 172.21.15.12:0/1567024307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460106280 0x7fb46010de00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb45800b0a0 tx=0x7fb45800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.946+0000 7fb466d6e700 1 -- 172.21.15.12:0/1567024307 shutdown_connections 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.946+0000 7fb466d6e700 1 --2- 172.21.15.12:0/1567024307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460106280 0x7fb46010de00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.946+0000 7fb466d6e700 1 --2- 172.21.15.12:0/1567024307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460069770 0x7fb460069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.946+0000 7fb466d6e700 1 --2- 172.21.15.12:0/1567024307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46006a120 0x7fb460105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.946+0000 7fb466d6e700 1 -- 172.21.15.12:0/1567024307 >> 172.21.15.12:0/1567024307 conn(0x7fb460076fa0 msgr2=0x7fb4600773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.946+0000 7fb466d6e700 1 -- 172.21.15.12:0/1567024307 shutdown_connections 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.947+0000 7fb466d6e700 1 -- 172.21.15.12:0/1567024307 wait complete. 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.947+0000 7fb466d6e700 1 Processor -- start 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.947+0000 7fb466d6e700 1 -- start start 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.947+0000 7fb466d6e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460069770 0x7fb4600ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.947+0000 7fb466d6e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460106280 0x7fb460100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.947+0000 7fb466d6e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460100890 0x7fb4601a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.947+0000 7fb466d6e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4601a5ce0 con 0x7fb460106280 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.947+0000 7fb466d6e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4601a5e50 con 0x7fb460100890 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.947+0000 7fb466d6e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4601a6010 con 0x7fb460069770 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.947+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460106280 0x7fb460100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460106280 0x7fb460100350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48292/0 (socket says 172.21.15.12:48292) 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45ffff700 1 -- 172.21.15.12:0/1861810379 learned_addr learned my addr 172.21.15.12:0/1861810379 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb464b0a700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460069770 0x7fb4600ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb46530b700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460100890 0x7fb4601a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45ffff700 1 -- 172.21.15.12:0/1861810379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460069770 msgr2=0x7fb4600ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45ffff700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460069770 0x7fb4600ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45ffff700 1 -- 172.21.15.12:0/1861810379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460100890 msgr2=0x7fb4601a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45ffff700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460100890 0x7fb4601a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45ffff700 1 -- 172.21.15.12:0/1861810379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458009d00 con 0x7fb460106280 2024-05-08T23:46:31.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb46530b700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460100890 0x7fb4601a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:31.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45ffff700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460106280 0x7fb460100350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d950 tx=0x7fb44c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:31.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45dffb700 1 -- 172.21.15.12:0/1861810379 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c004a30 con 0x7fb460106280 2024-05-08T23:46:31.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb466d6e700 1 -- 172.21.15.12:0/1861810379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4601a62f0 con 0x7fb460106280 2024-05-08T23:46:31.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45dffb700 1 -- 172.21.15.12:0/1861810379 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb44c0054e0 con 0x7fb460106280 2024-05-08T23:46:31.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.948+0000 7fb45dffb700 1 -- 172.21.15.12:0/1861810379 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c009bf0 con 0x7fb460106280 2024-05-08T23:46:31.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.949+0000 7fb466d6e700 1 -- 172.21.15.12:0/1861810379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4601a67b0 con 0x7fb460106280 2024-05-08T23:46:31.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.950+0000 7fb466d6e700 1 -- 172.21.15.12:0/1861810379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb460109550 con 0x7fb460106280 2024-05-08T23:46:31.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.950+0000 7fb45dffb700 1 -- 172.21.15.12:0/1861810379 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7fb44c009d50 con 0x7fb460106280 2024-05-08T23:46:31.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.951+0000 7fb45dffb700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1320 0x7fb4500a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:31.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.951+0000 7fb464b0a700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1320 0x7fb4500a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:31.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.951+0000 7fb45dffb700 1 -- 172.21.15.12:0/1861810379 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb44c0cc640 con 0x7fb460106280 2024-05-08T23:46:31.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.952+0000 7fb464b0a700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1320 0x7fb4500a37e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb454009c60 tx=0x7fb454009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:31.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:31.956+0000 7fb45dffb700 1 -- 172.21.15.12:0/1861810379 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb44c095960 con 0x7fb460106280 2024-05-08T23:46:32.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.178+0000 7fb466d6e700 1 -- 172.21.15.12:0/1861810379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4601012f0 con 0x7fb460106280 2024-05-08T23:46:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.179+0000 7fb45dffb700 1 -- 172.21.15.12:0/1861810379 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb44c0057e0 con 0x7fb460106280 2024-05-08T23:46:32.175 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-08T23:46:32.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 -- 172.21.15.12:0/1861810379 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1320 msgr2=0x7fb4500a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1320 0x7fb4500a37e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb454009c60 tx=0x7fb454009260 comp rx=0 tx=0).stop 2024-05-08T23:46:32.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 -- 172.21.15.12:0/1861810379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460106280 msgr2=0x7fb460100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460106280 0x7fb460100350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d950 tx=0x7fb44c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:46:32.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 -- 172.21.15.12:0/1861810379 shutdown_connections 2024-05-08T23:46:32.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1320 0x7fb4500a37e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460100890 0x7fb4601a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460069770 0x7fb4600ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 --2- 172.21.15.12:0/1861810379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460106280 0x7fb460100350 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 -- 172.21.15.12:0/1861810379 >> 172.21.15.12:0/1861810379 conn(0x7fb460076fa0 msgr2=0x7fb4600fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 -- 172.21.15.12:0/1861810379 shutdown_connections 2024-05-08T23:46:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.181+0000 7fb466d6e700 1 -- 172.21.15.12:0/1861810379 wait complete. 2024-05-08T23:46:32.191 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:46:32.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.336+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/2623989499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f74069d90 msgr2=0x7f9f74105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.336+0000 7f9f7c4fd700 1 --2- 172.21.15.12:0/2623989499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f74069d90 0x7f9f74105b70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009990 tx=0x7f9f64009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.337+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/2623989499 shutdown_connections 2024-05-08T23:46:32.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.337+0000 7f9f7c4fd700 1 --2- 172.21.15.12:0/2623989499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f741062a0 0x7f9f7410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.337+0000 7f9f7c4fd700 1 --2- 172.21.15.12:0/2623989499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f740693e0 0x7f9f740697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.337+0000 7f9f7c4fd700 1 --2- 172.21.15.12:0/2623989499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f74069d90 0x7f9f74105b70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.337+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/2623989499 >> 172.21.15.12:0/2623989499 conn(0x7f9f74076ba0 msgr2=0x7f9f74076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:32.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.337+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/2623989499 shutdown_connections 2024-05-08T23:46:32.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.337+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/2623989499 wait complete. 2024-05-08T23:46:32.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.337+0000 7f9f7c4fd700 1 Processor -- start 2024-05-08T23:46:32.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.337+0000 7f9f7c4fd700 1 -- start start 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f7c4fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f740693e0 0x7f9f741973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f7c4fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f741062a0 0x7f9f741978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f7c4fd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f7419bc90 0x7f9f7419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f7c4fd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7419c710 con 0x7f9f741062a0 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f7c4fd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7419c880 con 0x7f9f7419bc90 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f7c4fd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7419ca40 con 0x7f9f740693e0 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f79a98700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f741062a0 0x7f9f741978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f79a98700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f741062a0 0x7f9f741978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48300/0 (socket says 172.21.15.12:48300) 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f79a98700 1 -- 172.21.15.12:0/835936818 learned_addr learned my addr 172.21.15.12:0/835936818 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f7a299700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f740693e0 0x7f9f741973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f7aa9a700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f7419bc90 0x7f9f7419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f79a98700 1 -- 172.21.15.12:0/835936818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f740693e0 msgr2=0x7f9f741973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f79a98700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f740693e0 0x7f9f741973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f79a98700 1 -- 172.21.15.12:0/835936818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f7419bc90 msgr2=0x7f9f7419c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f79a98700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f7419bc90 0x7f9f7419c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f79a98700 1 -- 172.21.15.12:0/835936818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640095f0 con 0x7f9f741062a0 2024-05-08T23:46:32.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f7aa9a700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f7419bc90 0x7f9f7419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:32.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f79a98700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f741062a0 0x7f9f741978f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9f640059d0 tx=0x7f9f64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:32.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.338+0000 7f9f6b7fe700 1 -- 172.21.15.12:0/835936818 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6401d070 con 0x7f9f741062a0 2024-05-08T23:46:32.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.339+0000 7f9f6b7fe700 1 -- 172.21.15.12:0/835936818 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f64022470 con 0x7f9f741062a0 2024-05-08T23:46:32.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.339+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/835936818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f741a3f40 con 0x7f9f741062a0 2024-05-08T23:46:32.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.339+0000 7f9f6b7fe700 1 -- 172.21.15.12:0/835936818 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f64017650 con 0x7f9f741062a0 2024-05-08T23:46:32.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.339+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/835936818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f741a4300 con 0x7f9f741062a0 2024-05-08T23:46:32.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.339+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/835936818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f74109570 con 0x7f9f741062a0 2024-05-08T23:46:32.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.341+0000 7f9f6b7fe700 1 -- 172.21.15.12:0/835936818 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f9f640225e0 con 0x7f9f741062a0 2024-05-08T23:46:32.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.341+0000 7f9f6b7fe700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f600a1320 0x7f9f600a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:32.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.342+0000 7f9f7a299700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f600a1320 0x7f9f600a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:32.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.342+0000 7f9f6b7fe700 1 -- 172.21.15.12:0/835936818 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9f640ce980 con 0x7f9f741062a0 2024-05-08T23:46:32.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.342+0000 7f9f7a299700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f600a1320 0x7f9f600a37e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009c60 tx=0x7f9f70009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:32.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.346+0000 7f9f6b7fe700 1 -- 172.21.15.12:0/835936818 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9f64097cd0 con 0x7f9f741062a0 2024-05-08T23:46:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.564+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/835936818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f741986b0 con 0x7f9f741062a0 2024-05-08T23:46:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.564+0000 7f9f6b7fe700 1 -- 172.21.15.12:0/835936818 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f6409b2f0 con 0x7f9f741062a0 2024-05-08T23:46:32.560 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-08T23:46:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.566+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/835936818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f600a1320 msgr2=0x7f9f600a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.566+0000 7f9f7c4fd700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f600a1320 0x7f9f600a37e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009c60 tx=0x7f9f70009260 comp rx=0 tx=0).stop 2024-05-08T23:46:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.566+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/835936818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f741062a0 msgr2=0x7f9f741978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.566+0000 7f9f7c4fd700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f741062a0 0x7f9f741978f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9f640059d0 tx=0x7f9f64004730 comp rx=0 tx=0).stop 2024-05-08T23:46:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.566+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/835936818 shutdown_connections 2024-05-08T23:46:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.566+0000 7f9f7c4fd700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f600a1320 0x7f9f600a37e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.566+0000 7f9f7c4fd700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f7419bc90 0x7f9f7419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.566+0000 7f9f7c4fd700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f740693e0 0x7f9f741973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.566+0000 7f9f7c4fd700 1 --2- 172.21.15.12:0/835936818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f741062a0 0x7f9f741978f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.566+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/835936818 >> 172.21.15.12:0/835936818 conn(0x7f9f74076ba0 msgr2=0x7f9f74107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.567+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/835936818 shutdown_connections 2024-05-08T23:46:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.567+0000 7f9f7c4fd700 1 -- 172.21.15.12:0/835936818 wait complete. 2024-05-08T23:46:32.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:32 smithi012 ceph-mon[42503]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:46:32.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:32 smithi012 ceph-mon[42503]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:46:32.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:32 smithi012 ceph-mon[42503]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:32.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:32 smithi012 ceph-mon[42503]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:46:32.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:32 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/977532856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:32.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:32 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1861810379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:32.576 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:46:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.721+0000 7f1d287fb700 1 -- 172.21.15.12:0/3026630592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d20106280 msgr2=0x7f1d2010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.721+0000 7f1d287fb700 1 --2- 172.21.15.12:0/3026630592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d20106280 0x7f1d2010de00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800b0a0 tx=0x7f1d1800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.721+0000 7f1d287fb700 1 -- 172.21.15.12:0/3026630592 shutdown_connections 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.721+0000 7f1d287fb700 1 --2- 172.21.15.12:0/3026630592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d20106280 0x7f1d2010de00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.721+0000 7f1d287fb700 1 --2- 172.21.15.12:0/3026630592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d20101160 0x7f1d20101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.721+0000 7f1d287fb700 1 --2- 172.21.15.12:0/3026630592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20101b10 0x7f1d20105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.721+0000 7f1d287fb700 1 -- 172.21.15.12:0/3026630592 >> 172.21.15.12:0/3026630592 conn(0x7f1d200fc9e0 msgr2=0x7f1d200fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.721+0000 7f1d287fb700 1 -- 172.21.15.12:0/3026630592 shutdown_connections 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.721+0000 7f1d287fb700 1 -- 172.21.15.12:0/3026630592 wait complete. 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d287fb700 1 Processor -- start 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d287fb700 1 -- start start 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d287fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d20101160 0x7f1d200732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d287fb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20106280 0x7f1d200737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d287fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d20073d20 0x7f1d201a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d287fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d201a5d20 con 0x7f1d20101160 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d287fb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d201a5e90 con 0x7f1d20073d20 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d287fb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d201a6050 con 0x7f1d20106280 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d26597700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d20101160 0x7f1d200732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d26597700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d20101160 0x7f1d200732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48318/0 (socket says 172.21.15.12:48318) 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d26597700 1 -- 172.21.15.12:0/2805813373 learned_addr learned my addr 172.21.15.12:0/2805813373 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d25d96700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20106280 0x7f1d200737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d26d98700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d20073d20 0x7f1d201a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d26597700 1 -- 172.21.15.12:0/2805813373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20106280 msgr2=0x7f1d200737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d26597700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20106280 0x7f1d200737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d26597700 1 -- 172.21.15.12:0/2805813373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d20073d20 msgr2=0x7f1d201a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d26597700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d20073d20 0x7f1d201a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d26597700 1 -- 172.21.15.12:0/2805813373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d18009d00 con 0x7f1d20101160 2024-05-08T23:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.722+0000 7f1d26597700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d20101160 0x7f1d200732a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00d950 tx=0x7f1d1c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.723+0000 7f1d177fe700 1 -- 172.21.15.12:0/2805813373 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c009b40 con 0x7f1d20101160 2024-05-08T23:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.723+0000 7f1d287fb700 1 -- 172.21.15.12:0/2805813373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d201a6330 con 0x7f1d20101160 2024-05-08T23:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.723+0000 7f1d177fe700 1 -- 172.21.15.12:0/2805813373 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d1c009ca0 con 0x7f1d20101160 2024-05-08T23:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.723+0000 7f1d177fe700 1 -- 172.21.15.12:0/2805813373 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c00f620 con 0x7f1d20101160 2024-05-08T23:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.723+0000 7f1d287fb700 1 -- 172.21.15.12:0/2805813373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d201a6890 con 0x7f1d20101160 2024-05-08T23:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.724+0000 7f1d287fb700 1 -- 172.21.15.12:0/2805813373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d20109550 con 0x7f1d20101160 2024-05-08T23:46:32.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.725+0000 7f1d177fe700 1 -- 172.21.15.12:0/2805813373 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f1d1c010460 con 0x7f1d20101160 2024-05-08T23:46:32.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.725+0000 7f1d177fe700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d0c0a1320 0x7f1d0c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:32.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.726+0000 7f1d25d96700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d0c0a1320 0x7f1d0c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:32.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.726+0000 7f1d177fe700 1 -- 172.21.15.12:0/2805813373 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1d1c0cd1f0 con 0x7f1d20101160 2024-05-08T23:46:32.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.726+0000 7f1d25d96700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d0c0a1320 0x7f1d0c0a37e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d10005950 tx=0x7f1d100058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:32.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.731+0000 7f1d177fe700 1 -- 172.21.15.12:0/2805813373 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1d1c096510 con 0x7f1d20101160 2024-05-08T23:46:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:32 smithi204 ceph-mon[47373]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:46:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:32 smithi204 ceph-mon[47373]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:46:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:32 smithi204 ceph-mon[47373]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:32 smithi204 ceph-mon[47373]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:46:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:32 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/977532856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:32.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:32 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1861810379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:32 smithi160 ceph-mon[46152]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:46:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:32 smithi160 ceph-mon[46152]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:46:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:32 smithi160 ceph-mon[46152]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:46:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:32 smithi160 ceph-mon[46152]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:46:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:32 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/977532856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:32.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:32 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1861810379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:46:32.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.948+0000 7f1d287fb700 1 -- 172.21.15.12:0/2805813373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d20074780 con 0x7f1d20101160 2024-05-08T23:46:32.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.948+0000 7f1d177fe700 1 -- 172.21.15.12:0/2805813373 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d1c099b30 con 0x7f1d20101160 2024-05-08T23:46:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-08T23:46:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 -- 172.21.15.12:0/2805813373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d0c0a1320 msgr2=0x7f1d0c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d0c0a1320 0x7f1d0c0a37e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d10005950 tx=0x7f1d100058e0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 -- 172.21.15.12:0/2805813373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d20101160 msgr2=0x7f1d200732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d20101160 0x7f1d200732a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00d950 tx=0x7f1d1c00dd10 comp rx=0 tx=0).stop 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 -- 172.21.15.12:0/2805813373 shutdown_connections 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d0c0a1320 0x7f1d0c0a37e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d20073d20 0x7f1d201a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d20101160 0x7f1d200732a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 --2- 172.21.15.12:0/2805813373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d20106280 0x7f1d200737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 -- 172.21.15.12:0/2805813373 >> 172.21.15.12:0/2805813373 conn(0x7f1d200fc9e0 msgr2=0x7f1d200fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 -- 172.21.15.12:0/2805813373 shutdown_connections 2024-05-08T23:46:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:32.951+0000 7f1d287fb700 1 -- 172.21.15.12:0/2805813373 wait complete. 2024-05-08T23:46:32.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.102+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/541086963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a41050a0 msgr2=0x7ff8a410cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.102+0000 7ff8aa3b7700 1 --2- 172.21.15.12:0/541086963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a41050a0 0x7ff8a410cc20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00b0a0 tx=0x7ff89c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.102+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/541086963 shutdown_connections 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.102+0000 7ff8aa3b7700 1 --2- 172.21.15.12:0/541086963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a41050a0 0x7ff8a410cc20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.102+0000 7ff8aa3b7700 1 --2- 172.21.15.12:0/541086963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8a40fff80 0x7ff8a4100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.102+0000 7ff8aa3b7700 1 --2- 172.21.15.12:0/541086963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8a4100930 0x7ff8a4104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.102+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/541086963 >> 172.21.15.12:0/541086963 conn(0x7ff8a40fb840 msgr2=0x7ff8a40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.102+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/541086963 shutdown_connections 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.103+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/541086963 wait complete. 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.103+0000 7ff8aa3b7700 1 Processor -- start 2024-05-08T23:46:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.103+0000 7ff8aa3b7700 1 -- start start 2024-05-08T23:46:33.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8aa3b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8a40fff80 0x7ff8a4197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8aa3b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8a41050a0 0x7ff8a4197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8aa3b7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a419bd00 0x7ff8a419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8aa3b7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a419c780 con 0x7ff8a419bd00 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8aa3b7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a419c8f0 con 0x7ff8a40fff80 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8aa3b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a419cab0 con 0x7ff8a41050a0 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a8954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a419bd00 0x7ff8a419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8a40fff80 0x7ff8a4197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8a40fff80 0x7ff8a4197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:37782/0 (socket says 172.21.15.12:37782) 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8a41050a0 0x7ff8a4197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a3fff700 1 -- 172.21.15.12:0/3331340997 learned_addr learned my addr 172.21.15.12:0/3331340997 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a8954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a419bd00 0x7ff8a419c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48344/0 (socket says 172.21.15.12:48344) 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a8954700 1 -- 172.21.15.12:0/3331340997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8a41050a0 msgr2=0x7ff8a4197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a8954700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8a41050a0 0x7ff8a4197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a8954700 1 -- 172.21.15.12:0/3331340997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8a40fff80 msgr2=0x7ff8a4197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a8954700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8a40fff80 0x7ff8a4197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a8954700 1 -- 172.21.15.12:0/3331340997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c009d00 con 0x7ff8a419bd00 2024-05-08T23:46:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.104+0000 7ff8a8954700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a419bd00 0x7ff8a419c180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0062a0 tx=0x7ff89c00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.105+0000 7ff8a17fa700 1 -- 172.21.15.12:0/3331340997 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c00e040 con 0x7ff8a419bd00 2024-05-08T23:46:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.105+0000 7ff8a17fa700 1 -- 172.21.15.12:0/3331340997 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff89c01d480 con 0x7ff8a419bd00 2024-05-08T23:46:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.105+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/3331340997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a41a3fb0 con 0x7ff8a419bd00 2024-05-08T23:46:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.105+0000 7ff8a17fa700 1 -- 172.21.15.12:0/3331340997 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c01b6d0 con 0x7ff8a419bd00 2024-05-08T23:46:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.105+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/3331340997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a41a4430 con 0x7ff8a419bd00 2024-05-08T23:46:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.106+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/3331340997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a4109380 con 0x7ff8a419bd00 2024-05-08T23:46:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.107+0000 7ff8a17fa700 1 -- 172.21.15.12:0/3331340997 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7ff89c01d5f0 con 0x7ff8a419bd00 2024-05-08T23:46:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.107+0000 7ff8a17fa700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff88c0a1320 0x7ff88c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.108+0000 7ff8a17fa700 1 -- 172.21.15.12:0/3331340997 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff89c0ce4c0 con 0x7ff8a419bd00 2024-05-08T23:46:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.108+0000 7ff8a3fff700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff88c0a1320 0x7ff88c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.108+0000 7ff8a3fff700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff88c0a1320 0x7ff88c0a37e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff894007c90 tx=0x7ff894007480 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.112+0000 7ff8a17fa700 1 -- 172.21.15.12:0/3331340997 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff89c097810 con 0x7ff8a419bd00 2024-05-08T23:46:33.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.387+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/3331340997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8a4101450 con 0x7ff8a419bd00 2024-05-08T23:46:33.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.388+0000 7ff8a17fa700 1 -- 172.21.15.12:0/3331340997 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10876 (secure 0 0 0) 0x7ff89c09ae30 con 0x7ff8a419bd00 2024-05-08T23:46:33.384 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:33.384 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:46:26.294680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:46:33.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/3331340997 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff88c0a1320 msgr2=0x7ff88c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff88c0a1320 0x7ff88c0a37e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff894007c90 tx=0x7ff894007480 comp rx=0 tx=0).stop 2024-05-08T23:46:33.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/3331340997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a419bd00 msgr2=0x7ff8a419c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a419bd00 0x7ff8a419c180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0062a0 tx=0x7ff89c00be40 comp rx=0 tx=0).stop 2024-05-08T23:46:33.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/3331340997 shutdown_connections 2024-05-08T23:46:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff88c0a1320 0x7ff88c0a37e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a419bd00 0x7ff8a419c180 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8a40fff80 0x7ff8a4197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 --2- 172.21.15.12:0/3331340997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8a41050a0 0x7ff8a4197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/3331340997 >> 172.21.15.12:0/3331340997 conn(0x7ff8a40fb840 msgr2=0x7ff8a4069600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/3331340997 shutdown_connections 2024-05-08T23:46:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.390+0000 7ff8aa3b7700 1 -- 172.21.15.12:0/3331340997 wait complete. 2024-05-08T23:46:33.387 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 24 2024-05-08T23:46:33.400 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:46:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.542+0000 7f3b78f9d700 1 -- 172.21.15.12:0/461338874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b7410b400 msgr2=0x7f3b7410d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.542+0000 7f3b78f9d700 1 --2- 172.21.15.12:0/461338874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b7410b400 0x7f3b7410d8c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00b0a0 tx=0x7f3b6c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.542+0000 7f3b78f9d700 1 -- 172.21.15.12:0/461338874 shutdown_connections 2024-05-08T23:46:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.542+0000 7f3b78f9d700 1 --2- 172.21.15.12:0/461338874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b7410b400 0x7f3b7410d8c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.542+0000 7f3b78f9d700 1 --2- 172.21.15.12:0/461338874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b74101fe0 0x7f3b741023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.542+0000 7f3b78f9d700 1 --2- 172.21.15.12:0/461338874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b74102900 0x7f3b7410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.542+0000 7f3b78f9d700 1 -- 172.21.15.12:0/461338874 >> 172.21.15.12:0/461338874 conn(0x7f3b740fb840 msgr2=0x7f3b740fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.542+0000 7f3b78f9d700 1 -- 172.21.15.12:0/461338874 shutdown_connections 2024-05-08T23:46:33.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.542+0000 7f3b78f9d700 1 -- 172.21.15.12:0/461338874 wait complete. 2024-05-08T23:46:33.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b78f9d700 1 Processor -- start 2024-05-08T23:46:33.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b78f9d700 1 -- start start 2024-05-08T23:46:33.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b78f9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b74101fe0 0x7f3b74197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b78f9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b74102900 0x7f3b74197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b78f9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b7410b400 0x7f3b7419bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b78f9d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7419c320 con 0x7f3b74102900 2024-05-08T23:46:33.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b78f9d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7419c490 con 0x7f3b7410b400 2024-05-08T23:46:33.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b78f9d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7419c650 con 0x7f3b74101fe0 2024-05-08T23:46:33.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b71d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b74102900 0x7f3b74197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b71d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b74102900 0x7f3b74197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48354/0 (socket says 172.21.15.12:48354) 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b71d9b700 1 -- 172.21.15.12:0/3810191149 learned_addr learned my addr 172.21.15.12:0/3810191149 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b7259c700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b74101fe0 0x7f3b74197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b72d9d700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b7410b400 0x7f3b7419bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b71d9b700 1 -- 172.21.15.12:0/3810191149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b74101fe0 msgr2=0x7f3b74197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.543+0000 7f3b71d9b700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b74101fe0 0x7f3b74197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.544+0000 7f3b71d9b700 1 -- 172.21.15.12:0/3810191149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b7410b400 msgr2=0x7f3b7419bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.544+0000 7f3b71d9b700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b7410b400 0x7f3b7419bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.544+0000 7f3b71d9b700 1 -- 172.21.15.12:0/3810191149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c009d00 con 0x7f3b74102900 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.544+0000 7f3b71d9b700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b74102900 0x7f3b74197990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800d950 tx=0x7f3b6800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.544+0000 7f3b637fe700 1 -- 172.21.15.12:0/3810191149 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68009930 con 0x7f3b74102900 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.544+0000 7f3b78f9d700 1 -- 172.21.15.12:0/3810191149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7419c930 con 0x7f3b74102900 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.544+0000 7f3b637fe700 1 -- 172.21.15.12:0/3810191149 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b68010460 con 0x7f3b74102900 2024-05-08T23:46:33.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.544+0000 7f3b637fe700 1 -- 172.21.15.12:0/3810191149 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6800f620 con 0x7f3b74102900 2024-05-08T23:46:33.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.544+0000 7f3b78f9d700 1 -- 172.21.15.12:0/3810191149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b741a4170 con 0x7f3b74102900 2024-05-08T23:46:33.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.545+0000 7f3b78f9d700 1 -- 172.21.15.12:0/3810191149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b74106530 con 0x7f3b74102900 2024-05-08T23:46:33.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.546+0000 7f3b637fe700 1 -- 172.21.15.12:0/3810191149 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f3b68010860 con 0x7f3b74102900 2024-05-08T23:46:33.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.547+0000 7f3b637fe700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b5c0a1320 0x7f3b5c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.547+0000 7f3b637fe700 1 -- 172.21.15.12:0/3810191149 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3b680ccc40 con 0x7f3b74102900 2024-05-08T23:46:33.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.547+0000 7f3b7259c700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b5c0a1320 0x7f3b5c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.548+0000 7f3b7259c700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b5c0a1320 0x7f3b5c0a37e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3b64007be0 tx=0x7f3b640073d0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.551+0000 7f3b637fe700 1 -- 172.21.15.12:0/3810191149 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3b68095f00 con 0x7f3b74102900 2024-05-08T23:46:33.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:33 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/835936818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:33.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:33 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2805813373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:33.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:33 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/835936818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:33.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:33 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2805813373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:33.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:33 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/835936818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:46:33.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:33 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2805813373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:46:33.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.771+0000 7f3b78f9d700 1 -- 172.21.15.12:0/3810191149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b74110db0 con 0x7f3b74102900 2024-05-08T23:46:33.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.772+0000 7f3b637fe700 1 -- 172.21.15.12:0/3810191149 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22884 (secure 0 0 0) 0x7f3b68016090 con 0x7f3b74102900 2024-05-08T23:46:33.768 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:46:33.768 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":84,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-08T23:46:25.291136+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3622028746":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3062429456":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2381834553":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/716473668":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2695290437":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/1305168755":"2024-05-09T23:46:25.291064+0000","172.21.15.12:6801/1305168755":"2024-05-09T23:46:25.291064+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 -- 172.21.15.12:0/3810191149 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b5c0a1320 msgr2=0x7f3b5c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b5c0a1320 0x7f3b5c0a37e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3b64007be0 tx=0x7f3b640073d0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 -- 172.21.15.12:0/3810191149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b74102900 msgr2=0x7f3b74197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b74102900 0x7f3b74197990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800d950 tx=0x7f3b6800dc60 comp rx=0 tx=0).stop 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 -- 172.21.15.12:0/3810191149 shutdown_connections 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b5c0a1320 0x7f3b5c0a37e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b7410b400 0x7f3b7419bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b74101fe0 0x7f3b74197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 --2- 172.21.15.12:0/3810191149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b74102900 0x7f3b74197990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 -- 172.21.15.12:0/3810191149 >> 172.21.15.12:0/3810191149 conn(0x7f3b740fb840 msgr2=0x7f3b740fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 -- 172.21.15.12:0/3810191149 shutdown_connections 2024-05-08T23:46:33.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.775+0000 7f3b78f9d700 1 -- 172.21.15.12:0/3810191149 wait complete. 2024-05-08T23:46:33.785 DEBUG:tasks.fwd_scrub:Yielding 2024-05-08T23:46:33.785 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T23:46:33.801 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:46:33.802 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-08T23:46:33.802 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-08T23:46:33.802 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-08T23:46:33.802 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:46:33.803 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:46:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.951+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1636596094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e900ff3a0 msgr2=0x7f0e900ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.951+0000 7f0e97b2f700 1 --2- 172.21.15.12:0/1636596094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e900ff3a0 0x7f0e900ff820 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009990 tx=0x7f0e80009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.951+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1636596094 shutdown_connections 2024-05-08T23:46:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.951+0000 7f0e97b2f700 1 --2- 172.21.15.12:0/1636596094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e900ffd60 0x7f0e9010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.951+0000 7f0e97b2f700 1 --2- 172.21.15.12:0/1636596094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e90105f00 0x7f0e901062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.951+0000 7f0e97b2f700 1 --2- 172.21.15.12:0/1636596094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e900ff3a0 0x7f0e900ff820 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.951+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1636596094 >> 172.21.15.12:0/1636596094 conn(0x7f0e900773b0 msgr2=0x7f0e900fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.951+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1636596094 shutdown_connections 2024-05-08T23:46:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.951+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1636596094 wait complete. 2024-05-08T23:46:33.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e97b2f700 1 Processor -- start 2024-05-08T23:46:33.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e97b2f700 1 -- start start 2024-05-08T23:46:33.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e97b2f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e900ff3a0 0x7f0e90100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e97b2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e900ffd60 0x7f0e90100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e97b2f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e90105f00 0x7f0e90101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e97b2f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e901a5750 con 0x7f0e900ffd60 2024-05-08T23:46:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e97b2f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e901a58c0 con 0x7f0e900ff3a0 2024-05-08T23:46:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e97b2f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e901a5a80 con 0x7f0e90105f00 2024-05-08T23:46:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f3d23553700 1 -- 172.21.15.12:0/3066423712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1c101b30 msgr2=0x7f3d1c105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f3d23553700 1 --2- 172.21.15.12:0/3066423712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1c101b30 0x7f3d1c105b70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009990 tx=0x7f3d0c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e950ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e900ffd60 0x7f0e90100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e950ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e900ffd60 0x7f0e90100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48382/0 (socket says 172.21.15.12:48382) 2024-05-08T23:46:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e950ca700 1 -- 172.21.15.12:0/1381233405 learned_addr learned my addr 172.21.15.12:0/1381233405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e958cb700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e900ff3a0 0x7f0e90100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f0e960cc700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e90105f00 0x7f0e90101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f3d23553700 1 -- 172.21.15.12:0/3066423712 shutdown_connections 2024-05-08T23:46:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f3d23553700 1 --2- 172.21.15.12:0/3066423712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d1c1062a0 0x7f3d1c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f3d23553700 1 --2- 172.21.15.12:0/3066423712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d1c101180 0x7f3d1c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f3d23553700 1 --2- 172.21.15.12:0/3066423712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1c101b30 0x7f3d1c105b70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f3d23553700 1 -- 172.21.15.12:0/3066423712 >> 172.21.15.12:0/3066423712 conn(0x7f3d1c0fc9e0 msgr2=0x7f3d1c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f3d23553700 1 -- 172.21.15.12:0/3066423712 shutdown_connections 2024-05-08T23:46:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.952+0000 7f3d23553700 1 -- 172.21.15.12:0/3066423712 wait complete. 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e950ca700 1 -- 172.21.15.12:0/1381233405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e90105f00 msgr2=0x7f0e90101000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e950ca700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e90105f00 0x7f0e90101000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e950ca700 1 -- 172.21.15.12:0/1381233405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e900ff3a0 msgr2=0x7f0e90100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e950ca700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e900ff3a0 0x7f0e90100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e950ca700 1 -- 172.21.15.12:0/1381233405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800095f0 con 0x7f0e900ffd60 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e950ca700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e900ffd60 0x7f0e90100ac0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0e80005770 tx=0x7f0e800057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e86ffd700 1 -- 172.21.15.12:0/1381233405 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8001d070 con 0x7f0e900ffd60 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e86ffd700 1 -- 172.21.15.12:0/1381233405 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e80021410 con 0x7f0e900ffd60 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e86ffd700 1 -- 172.21.15.12:0/1381233405 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80017650 con 0x7f0e900ffd60 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1381233405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e901a5d00 con 0x7f0e900ffd60 2024-05-08T23:46:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1381233405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e901a6160 con 0x7f0e900ffd60 2024-05-08T23:46:33.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f3d23553700 1 Processor -- start 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.953+0000 7f3d23553700 1 -- start start 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d23553700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1c101180 0x7f3d1c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d23553700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d1c1062a0 0x7f3d1c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d23553700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d1c073d20 0x7f3d1c1a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d23553700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c1a5b10 con 0x7f3d1c101180 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d23553700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c1a5c80 con 0x7f3d1c073d20 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d23553700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c1a5e40 con 0x7f3d1c1062a0 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d212ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1c101180 0x7f3d1c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d212ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1c101180 0x7f3d1c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48388/0 (socket says 172.21.15.12:48388) 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d212ef700 1 -- 172.21.15.12:0/1246810183 learned_addr learned my addr 172.21.15.12:0/1246810183 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d21af0700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d1c073d20 0x7f3d1c1a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d20aee700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d1c1062a0 0x7f3d1c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d212ef700 1 -- 172.21.15.12:0/1246810183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d1c1062a0 msgr2=0x7f3d1c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d212ef700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d1c1062a0 0x7f3d1c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d212ef700 1 -- 172.21.15.12:0/1246810183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d1c073d20 msgr2=0x7f3d1c1a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d212ef700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d1c073d20 0x7f3d1c1a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d212ef700 1 -- 172.21.15.12:0/1246810183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c0095f0 con 0x7f3d1c101180 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d20aee700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d1c1062a0 0x7f3d1c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d212ef700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1c101180 0x7f3d1c0732a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800c9b0 tx=0x7f3d1800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.954+0000 7f3d127fc700 1 -- 172.21.15.12:0/1246810183 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18007b00 con 0x7f3d1c101180 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.955+0000 7f3d127fc700 1 -- 172.21.15.12:0/1246810183 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d18007c60 con 0x7f3d1c101180 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.955+0000 7f3d23553700 1 -- 172.21.15.12:0/1246810183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c1a6120 con 0x7f3d1c101180 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.955+0000 7f3d127fc700 1 -- 172.21.15.12:0/1246810183 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18018660 con 0x7f3d1c101180 2024-05-08T23:46:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.955+0000 7f3d23553700 1 -- 172.21.15.12:0/1246810183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c1a6630 con 0x7f3d1c101180 2024-05-08T23:46:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.955+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1381233405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e901094e0 con 0x7f0e900ffd60 2024-05-08T23:46:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.956+0000 7f0e86ffd700 1 -- 172.21.15.12:0/1381233405 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f0e800217f0 con 0x7f0e900ffd60 2024-05-08T23:46:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.957+0000 7f0e86ffd700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e7c0a1320 0x7f0e7c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.957+0000 7f0e86ffd700 1 -- 172.21.15.12:0/1381233405 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0e800ce0a0 con 0x7f0e900ffd60 2024-05-08T23:46:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.957+0000 7f0e958cb700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e7c0a1320 0x7f0e7c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.958+0000 7f0e958cb700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e7c0a1320 0x7f0e7c0a37e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009550 tx=0x7f0e8c0091b0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.958+0000 7f3d127fc700 1 -- 172.21.15.12:0/1246810183 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f3d1800f450 con 0x7f3d1c101180 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.958+0000 7f3d127fc700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3d080a1320 0x7f3d080a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.959+0000 7f3d20aee700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3d080a1320 0x7f3d080a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.959+0000 7f3d127fc700 1 -- 172.21.15.12:0/1246810183 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3d180cce00 con 0x7f3d1c101180 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.959+0000 7f3d20aee700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3d080a1320 0x7f3d080a37e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c006010 tx=0x7f3d0c00b540 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d2f70db80 1 Processor -- start 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d2f70db80 1 -- start start 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e81260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e817a0 0x556a26e7b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26dd21d0 con 0x556a26e7b910 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26dcfb90 con 0x556a26e817a0 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26dcb480 con 0x556a26e80e80 2024-05-08T23:46:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d21af0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d21af0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48394/0 (socket says 172.21.15.12:48394) 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d21af0700 1 -- 172.21.15.12:0/3586528052 learned_addr learned my addr 172.21.15.12:0/3586528052 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d20aee700 1 --2- 172.21.15.12:0/3586528052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e817a0 0x556a26e7b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d212ef700 1 --2- 172.21.15.12:0/3586528052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e81260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d21af0700 1 -- 172.21.15.12:0/3586528052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 msgr2=0x556a26e81260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d21af0700 1 --2- 172.21.15.12:0/3586528052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e81260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d21af0700 1 -- 172.21.15.12:0/3586528052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e817a0 msgr2=0x556a26e7b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.960+0000 7f3d21af0700 1 --2- 172.21.15.12:0/3586528052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e817a0 0x556a26e7b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.961+0000 7f3d21af0700 1 -- 172.21.15.12:0/3586528052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a26c4c430 con 0x556a26e7b910 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.961+0000 7f3d21af0700 1 --2- 172.21.15.12:0/3586528052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7bd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d14003f30 tx=0x7f3d14009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d2102ef90279a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.963+0000 7f3d077fe700 1 -- 172.21.15.12:0/3586528052 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14003a00 con 0x556a26e7b910 2024-05-08T23:46:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.963+0000 7f0e86ffd700 1 -- 172.21.15.12:0/1381233405 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0e80097450 con 0x7f0e900ffd60 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.963+0000 7f3d077fe700 1 -- 172.21.15.12:0/3586528052 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d1400f040 con 0x556a26e7b910 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.963+0000 7f3d077fe700 1 -- 172.21.15.12:0/3586528052 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d140156e0 con 0x556a26e7b910 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.964+0000 7f3d2f70db80 1 -- 172.21.15.12:0/3586528052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 msgr2=0x556a26e7bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.964+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/3586528052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7bd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d14003f30 tx=0x7f3d14009c20 comp rx=0 tx=0).stop 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.964+0000 7f3d2f70db80 1 -- 172.21.15.12:0/3586528052 shutdown_connections 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.964+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/3586528052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7bd50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.964+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/3586528052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e81260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.964+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/3586528052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e817a0 0x556a26e7b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.964+0000 7f3d2f70db80 1 -- 172.21.15.12:0/3586528052 >> 172.21.15.12:0/3586528052 conn(0x556a26dd9290 msgr2=0x556a26e75e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.964+0000 7f3d2f70db80 1 -- 172.21.15.12:0/3586528052 shutdown_connections 2024-05-08T23:46:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.964+0000 7f3d2f70db80 1 -- 172.21.15.12:0/3586528052 wait complete. 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.965+0000 7f3d2f70db80 1 -- start start 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.965+0000 7f3d2f70db80 1 Processor -- start 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e84b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e85070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e817a0 0x556a26e855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26d675e0 con 0x556a26e7b910 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26d67280 con 0x556a26e817a0 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26c4e720 con 0x556a26e80e80 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d2f70db80 10 client.0 _ll_register_callbacks cb 0x556a26d70bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d212ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e84b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d212ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e84b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48396/0 (socket says 172.21.15.12:48396) 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d212ef700 1 -- 172.21.15.12:0/436582855 learned_addr learned my addr 172.21.15.12:0/436582855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d20aee700 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e85070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d21af0700 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e817a0 0x556a26e855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d212ef700 1 -- 172.21.15.12:0/436582855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 msgr2=0x556a26e85070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d212ef700 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e85070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d212ef700 1 -- 172.21.15.12:0/436582855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e817a0 msgr2=0x556a26e855b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d212ef700 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e817a0 0x556a26e855b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d212ef700 1 -- 172.21.15.12:0/436582855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d14003b60 con 0x556a26e7b910 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d212ef700 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e84b30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d180ca5b0 tx=0x7f3d1809a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e225e2eec2d41c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.966+0000 7f3d04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d04ff9700 1 -- 172.21.15.12:0/436582855 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180ca980 con 0x556a26e7b910 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d04ff9700 1 -- 172.21.15.12:0/436582855 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3d18099de0 con 0x556a26e7b910 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d2f70db80 1 -- 172.21.15.12:0/436582855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a26d242f0 con 0x556a26e7b910 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d04ff9700 1 -- 172.21.15.12:0/436582855 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d1809dc10 con 0x556a26e7b910 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d2f70db80 10 client.15441 fetch_fsmap learned FSMap version 24 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d2f70db80 1 -- 172.21.15.12:0/436582855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556a26bf8be0 con 0x556a26e7b910 2024-05-08T23:46:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d04ff9700 1 -- 172.21.15.12:0/436582855 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a26d242f0 con 0x556a26e7b910 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d04ff9700 1 -- 172.21.15.12:0/436582855 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3d1809a040 con 0x556a26e7b910 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d2f70db80 10 client.15441 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d2f70db80 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d2f70db80 20 client.15441 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556a26e94fb0 0x556a26e973a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d2f70db80 4 client.15441 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.967+0000 7f3d2f70db80 1 -- 172.21.15.12:0/436582855 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a26e4a890 con 0x556a26e94fb0 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.968+0000 7f3d20aee700 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556a26e94fb0 0x556a26e973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.968+0000 7f3d20aee700 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556a26e94fb0 0x556a26e973a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79f877ad67122602 server_cookie=8ca682fd5d2bf198 in_seq=0 out_seq=0 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.968+0000 7f3d04ff9700 10 client.15441 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d04ff9700 1 -- 172.21.15.12:0/436582855 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a26e4a890 con 0x556a26e94fb0 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d04ff9700 10 client.15441 handle_command_reply: tid=0 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 client.15441 shutdown 2024-05-08T23:46:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 -- 172.21.15.12:0/436582855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 msgr2=0x556a26e84b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e84b30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d180ca5b0 tx=0x7f3d1809a4e0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 -- 172.21.15.12:0/436582855 shutdown_connections 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556a26e94fb0 0x556a26e973a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e817a0 0x556a26e855b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e84b30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d04ff9700 0 client.15441 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/436582855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e85070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 -- 172.21.15.12:0/436582855 >> 172.21.15.12:0/436582855 conn(0x556a26dd9290 msgr2=0x556a26e75620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 -- 172.21.15.12:0/436582855 shutdown_connections 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 1 -- 172.21.15.12:0/436582855 wait complete. 2024-05-08T23:46:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.969+0000 7f3d2f70db80 20 client.15441 trim_cache size 0 max 16384 2024-05-08T23:46:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d2f70db80 1 Processor -- start 2024-05-08T23:46:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d2f70db80 1 -- start start 2024-05-08T23:46:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e80e80 0x556a26e75da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e817a0 0x556a26e853d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26d242f0 con 0x556a26e7b910 2024-05-08T23:46:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26e814b0 con 0x556a26e80e80 2024-05-08T23:46:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26e861a0 con 0x556a26e817a0 2024-05-08T23:46:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d212ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d212ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48406/0 (socket says 172.21.15.12:48406) 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d212ef700 1 -- 172.21.15.12:0/3232916221 learned_addr learned my addr 172.21.15.12:0/3232916221 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.988+0000 7f3d21af0700 1 --2- 172.21.15.12:0/3232916221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e817a0 0x556a26e853d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.989+0000 7f3d20aee700 1 --2- 172.21.15.12:0/3232916221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e80e80 0x556a26e75da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.989+0000 7f3d212ef700 1 -- 172.21.15.12:0/3232916221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e817a0 msgr2=0x556a26e853d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.989+0000 7f3d212ef700 1 --2- 172.21.15.12:0/3232916221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e817a0 0x556a26e853d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.989+0000 7f3d212ef700 1 -- 172.21.15.12:0/3232916221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e80e80 msgr2=0x556a26e75da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.989+0000 7f3d212ef700 1 --2- 172.21.15.12:0/3232916221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e80e80 0x556a26e75da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.989+0000 7f3d212ef700 1 -- 172.21.15.12:0/3232916221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a26e72560 con 0x556a26e7b910 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.989+0000 7f3d212ef700 1 --2- 172.21.15.12:0/3232916221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7b3d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3d18098130 tx=0x7f3d1800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36df3f521a2afa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.989+0000 7f3d07fff700 1 -- 172.21.15.12:0/3232916221 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180ca410 con 0x556a26e7b910 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.989+0000 7f3d07fff700 1 -- 172.21.15.12:0/3232916221 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d18099320 con 0x556a26e7b910 2024-05-08T23:46:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.989+0000 7f3d07fff700 1 -- 172.21.15.12:0/3232916221 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18099510 con 0x556a26e7b910 2024-05-08T23:46:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 -- 172.21.15.12:0/3232916221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 msgr2=0x556a26e7b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/3232916221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7b3d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3d18098130 tx=0x7f3d1800fc30 comp rx=0 tx=0).stop 2024-05-08T23:46:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 -- 172.21.15.12:0/3232916221 shutdown_connections 2024-05-08T23:46:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/3232916221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e817a0 0x556a26e853d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/3232916221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e7b910 0x556a26e7b3d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/3232916221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e80e80 0x556a26e75da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 -- 172.21.15.12:0/3232916221 >> 172.21.15.12:0/3232916221 conn(0x556a26dd9290 msgr2=0x556a26e7fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 -- 172.21.15.12:0/3232916221 shutdown_connections 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 -- 172.21.15.12:0/3232916221 wait complete. 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 -- start start 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.990+0000 7f3d2f70db80 1 Processor -- start 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e7b910 0x556a26e8ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e8b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d2f70db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e817a0 0x556a26e8b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26eeb590 con 0x556a26e817a0 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26eeb6d0 con 0x556a26e7b910 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d2f70db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a26eeb890 con 0x556a26e80e80 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d2f70db80 10 client.0 _ll_register_callbacks cb 0x556a26d70bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d21af0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e817a0 0x556a26e8b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d21af0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e817a0 0x556a26e8b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48420/0 (socket says 172.21.15.12:48420) 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d21af0700 1 -- 172.21.15.12:0/616421443 learned_addr learned my addr 172.21.15.12:0/616421443 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d212ef700 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e7b910 0x556a26e8ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d20aee700 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e8b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d21af0700 1 -- 172.21.15.12:0/616421443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 msgr2=0x556a26e8b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d21af0700 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e8b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d21af0700 1 -- 172.21.15.12:0/616421443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e7b910 msgr2=0x556a26e8ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d21af0700 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e7b910 0x556a26e8ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d21af0700 1 -- 172.21.15.12:0/616421443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a26bf8be0 con 0x556a26e817a0 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.991+0000 7f3d20aee700 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e8b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d21af0700 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e817a0 0x556a26e8b8e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3d14003f30 tx=0x7f3d14004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3e02e2af675e44 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d04ff9700 1 -- 172.21.15.12:0/616421443 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14025070 con 0x556a26e817a0 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d2f70db80 1 -- 172.21.15.12:0/616421443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d1400f040 con 0x556a26e817a0 2024-05-08T23:46:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d04ff9700 1 -- 172.21.15.12:0/616421443 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3d140047b0 con 0x556a26e817a0 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d04ff9700 1 -- 172.21.15.12:0/616421443 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d14003a00 con 0x556a26e817a0 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d2f70db80 10 client.15447 fetch_fsmap learned FSMap version 24 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d2f70db80 1 -- 172.21.15.12:0/616421443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556a26e7bfc0 con 0x556a26e817a0 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d04ff9700 1 -- 172.21.15.12:0/616421443 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d1400f040 con 0x556a26e817a0 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d04ff9700 1 -- 172.21.15.12:0/616421443 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3d14015960 con 0x556a26e817a0 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d2f70db80 10 client.15447 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d2f70db80 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d2f70db80 20 client.15447 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.992+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556a26eed010 0x556a26eef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.993+0000 7f3d2f70db80 4 client.15447 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.993+0000 7f3d2f70db80 1 -- 172.21.15.12:0/616421443 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556a26dc3f40 con 0x556a26eed010 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.993+0000 7f3d212ef700 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556a26eed010 0x556a26eef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.993+0000 7f3d212ef700 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556a26eed010 0x556a26eef430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7001d6ac51640e80 server_cookie=9276e17d2bc2455 in_seq=0 out_seq=0 2024-05-08T23:46:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.993+0000 7f3d04ff9700 10 client.15447 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d04ff9700 1 -- 172.21.15.12:0/616421443 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556a26dc3f40 con 0x556a26eed010 2024-05-08T23:46:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d04ff9700 10 client.15447 handle_command_reply: tid=0 2024-05-08T23:46:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 client.15447 shutdown 2024-05-08T23:46:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 -- 172.21.15.12:0/616421443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e817a0 msgr2=0x556a26e8b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e817a0 0x556a26e8b8e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3d14003f30 tx=0x7f3d14004f70 comp rx=0 tx=0).stop 2024-05-08T23:46:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 -- 172.21.15.12:0/616421443 shutdown_connections 2024-05-08T23:46:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a26e817a0 0x556a26e8b8e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556a26eed010 0x556a26eef430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a26e7b910 0x556a26e8ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 --2- 172.21.15.12:0/616421443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556a26e80e80 0x556a26e8b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 -- 172.21.15.12:0/616421443 >> 172.21.15.12:0/616421443 conn(0x556a26dd9290 msgr2=0x556a26e804b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d04ff9700 0 client.15447 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 -- 172.21.15.12:0/616421443 shutdown_connections 2024-05-08T23:46:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 1 -- 172.21.15.12:0/616421443 wait complete. 2024-05-08T23:46:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.995+0000 7f3d2f70db80 20 client.15447 trim_cache size 0 max 16384 2024-05-08T23:46:33.995 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:46:33.995 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:46:33.996 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "dcc8ec5a-23d3-457b-b352-1f5170a3d139", 2024-05-08T23:46:33.996 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:46:33.996 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:46:33.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 -- 172.21.15.12:0/1246810183 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3d080a1320 msgr2=0x7f3d080a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3d080a1320 0x7f3d080a37e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c006010 tx=0x7f3d0c00b540 comp rx=0 tx=0).stop 2024-05-08T23:46:33.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 -- 172.21.15.12:0/1246810183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1c101180 msgr2=0x7f3d1c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1c101180 0x7f3d1c0732a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800c9b0 tx=0x7f3d1800ccc0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 -- 172.21.15.12:0/1246810183 shutdown_connections 2024-05-08T23:46:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3d080a1320 0x7f3d080a37e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d1c073d20 0x7f3d1c1a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1c101180 0x7f3d1c0732a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 --2- 172.21.15.12:0/1246810183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d1c1062a0 0x7f3d1c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 -- 172.21.15.12:0/1246810183 >> 172.21.15.12:0/1246810183 conn(0x7f3d1c0fc9e0 msgr2=0x7f3d1c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 -- 172.21.15.12:0/1246810183 shutdown_connections 2024-05-08T23:46:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:33.996+0000 7f3d23553700 1 -- 172.21.15.12:0/1246810183 wait complete. 2024-05-08T23:46:34.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:46:34.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.150+0000 7f011affb700 1 -- 172.21.15.12:0/1852277950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01140682e0 msgr2=0x7f01140686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.150+0000 7f011affb700 1 --2- 172.21.15.12:0/1852277950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01140682e0 0x7f01140686c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f01080099c0 tx=0x7f0108009cd0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.150+0000 7f011affb700 1 -- 172.21.15.12:0/1852277950 shutdown_connections 2024-05-08T23:46:34.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.150+0000 7f011affb700 1 --2- 172.21.15.12:0/1852277950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0114106290 0x7f011410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.150+0000 7f011affb700 1 --2- 172.21.15.12:0/1852277950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01140682e0 0x7f01140686c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.150+0000 7f011affb700 1 --2- 172.21.15.12:0/1852277950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0114068c90 0x7f0114105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.150+0000 7f011affb700 1 -- 172.21.15.12:0/1852277950 >> 172.21.15.12:0/1852277950 conn(0x7f0114076f40 msgr2=0x7f0114077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.151+0000 7f011affb700 1 -- 172.21.15.12:0/1852277950 shutdown_connections 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.151+0000 7f011affb700 1 -- 172.21.15.12:0/1852277950 wait complete. 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.151+0000 7f011affb700 1 Processor -- start 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.151+0000 7f011affb700 1 -- start start 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.151+0000 7f011affb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01140682e0 0x7f01140ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.151+0000 7f011affb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0114106290 0x7f01141003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.151+0000 7f011affb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01141008e0 0x7f01141a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.151+0000 7f011affb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01141a5b40 con 0x7f01140682e0 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.151+0000 7f011affb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01141a5cb0 con 0x7f0114106290 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.151+0000 7f011affb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01141a5e70 con 0x7f01141008e0 2024-05-08T23:46:34.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0118d97700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01140682e0 0x7f01140ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0118d97700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01140682e0 0x7f01140ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48436/0 (socket says 172.21.15.12:48436) 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0118d97700 1 -- 172.21.15.12:0/4064858213 learned_addr learned my addr 172.21.15.12:0/4064858213 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0119598700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01141008e0 0x7f01141a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0113fff700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0114106290 0x7f01141003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0118d97700 1 -- 172.21.15.12:0/4064858213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01141008e0 msgr2=0x7f01141a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0118d97700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01141008e0 0x7f01141a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0118d97700 1 -- 172.21.15.12:0/4064858213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0114106290 msgr2=0x7f01141003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0118d97700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0114106290 0x7f01141003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0118d97700 1 -- 172.21.15.12:0/4064858213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080095f0 con 0x7f01140682e0 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0118d97700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01140682e0 0x7f01140ffe60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f010800f690 tx=0x7f010800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0111ffb700 1 -- 172.21.15.12:0/4064858213 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010801c070 con 0x7f01140682e0 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0111ffb700 1 -- 172.21.15.12:0/4064858213 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f010800fda0 con 0x7f01140682e0 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f011affb700 1 -- 172.21.15.12:0/4064858213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01141a60f0 con 0x7f01140682e0 2024-05-08T23:46:34.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.152+0000 7f0111ffb700 1 -- 172.21.15.12:0/4064858213 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108022cd0 con 0x7f01140682e0 2024-05-08T23:46:34.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.153+0000 7f011affb700 1 -- 172.21.15.12:0/4064858213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01141a6570 con 0x7f01140682e0 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.156+0000 7f0111ffb700 1 -- 172.21.15.12:0/4064858213 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f0108020070 con 0x7f01140682e0 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.156+0000 7f01271b5b80 1 Processor -- start 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.156+0000 7f01271b5b80 1 -- start start 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe58e00 0x55605fe591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe59720 0x55605fe53350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe53cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f01271b5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fdaa150 con 0x55605fe53890 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f01271b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fda7b10 con 0x55605fe58e00 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f01271b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fda3400 con 0x55605fe59720 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0111ffb700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01040a13a0 0x7f01040a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0113fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe53cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0119598700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01040a13a0 0x7f01040a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0113fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe53cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48442/0 (socket says 172.21.15.12:48442) 2024-05-08T23:46:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0111ffb700 1 -- 172.21.15.12:0/4064858213 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f01080cdbd0 con 0x7f01140682e0 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0113fff700 1 -- 172.21.15.12:0/3141711524 learned_addr learned my addr 172.21.15.12:0/3141711524 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0119598700 1 --2- 172.21.15.12:0/3141711524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe58e00 0x55605fe591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0118d97700 1 --2- 172.21.15.12:0/3141711524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe59720 0x55605fe53350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0113fff700 1 -- 172.21.15.12:0/3141711524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe59720 msgr2=0x55605fe53350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0113fff700 1 --2- 172.21.15.12:0/3141711524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe59720 0x55605fe53350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0113fff700 1 -- 172.21.15.12:0/3141711524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe58e00 msgr2=0x55605fe591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0113fff700 1 --2- 172.21.15.12:0/3141711524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe58e00 0x55605fe591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0113fff700 1 -- 172.21.15.12:0/3141711524 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55605fc243b0 con 0x55605fe53890 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0119598700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01040a13a0 0x7f01040a3860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f010c008bc0 tx=0x7f010c0092b0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.157+0000 7f0113fff700 1 --2- 172.21.15.12:0/3141711524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe53cd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0100009c80 tx=0x7f0100009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb880d1ab41c867 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.158+0000 7f00feffd700 1 -- 172.21.15.12:0/3141711524 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100009750 con 0x55605fe53890 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.158+0000 7f00feffd700 1 -- 172.21.15.12:0/3141711524 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01000044f0 con 0x55605fe53890 2024-05-08T23:46:34.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.158+0000 7f00feffd700 1 -- 172.21.15.12:0/3141711524 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100003db0 con 0x55605fe53890 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.158+0000 7f01271b5b80 1 -- 172.21.15.12:0/3141711524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 msgr2=0x55605fe53cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.158+0000 7f01271b5b80 1 --2- 172.21.15.12:0/3141711524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe53cd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0100009c80 tx=0x7f0100009f90 comp rx=0 tx=0).stop 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.158+0000 7f01271b5b80 1 -- 172.21.15.12:0/3141711524 shutdown_connections 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.158+0000 7f01271b5b80 1 --2- 172.21.15.12:0/3141711524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe53cd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.158+0000 7f01271b5b80 1 --2- 172.21.15.12:0/3141711524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe58e00 0x55605fe591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.158+0000 7f01271b5b80 1 --2- 172.21.15.12:0/3141711524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe59720 0x55605fe53350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.159+0000 7f01271b5b80 1 -- 172.21.15.12:0/3141711524 >> 172.21.15.12:0/3141711524 conn(0x55605fdb1210 msgr2=0x55605fe4ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.159+0000 7f01271b5b80 1 -- 172.21.15.12:0/3141711524 shutdown_connections 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.159+0000 7f01271b5b80 1 -- 172.21.15.12:0/3141711524 wait complete. 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.159+0000 7f01271b5b80 1 -- start start 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.159+0000 7f01271b5b80 1 Processor -- start 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.159+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe5caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.159+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe58e00 0x55605fe5cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.159+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 0x55605fe5d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f01271b5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fd3f560 con 0x55605fe53890 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f01271b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fd3f200 con 0x55605fe59720 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f01271b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fc266a0 con 0x55605fe58e00 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f01271b5b80 10 client.0 _ll_register_callbacks cb 0x55605fd48b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0118d97700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe5caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0118d97700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe5caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48444/0 (socket says 172.21.15.12:48444) 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0118d97700 1 -- 172.21.15.12:0/1131759035 learned_addr learned my addr 172.21.15.12:0/1131759035 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0113fff700 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe58e00 0x55605fe5cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0119598700 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 0x55605fe5d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0118d97700 1 -- 172.21.15.12:0/1131759035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe58e00 msgr2=0x55605fe5cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0118d97700 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe58e00 0x55605fe5cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0118d97700 1 -- 172.21.15.12:0/1131759035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 msgr2=0x55605fe5d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0118d97700 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 0x55605fe5d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0118d97700 1 -- 172.21.15.12:0/1131759035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01000098b0 con 0x55605fe53890 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0119598700 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 0x55605fe5d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f0118d97700 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe5caa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0108096960 tx=0x7f0108022700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722de604fdfac9cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f00f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f00f3fff700 1 -- 172.21.15.12:0/1131759035 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010809abe0 con 0x55605fe53890 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f00f3fff700 1 -- 172.21.15.12:0/1131759035 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f010809ea10 con 0x55605fe53890 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.160+0000 7f01271b5b80 1 -- 172.21.15.12:0/1131759035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55605fcfc270 con 0x55605fe53890 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f00f3fff700 1 -- 172.21.15.12:0/1131759035 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01080a2fd0 con 0x55605fe53890 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f01271b5b80 10 client.15459 fetch_fsmap learned FSMap version 24 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f01271b5b80 1 -- 172.21.15.12:0/1131759035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55605fbd0b60 con 0x55605fe53890 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f00f3fff700 1 -- 172.21.15.12:0/1131759035 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f010807d070 con 0x55605fe53890 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f00f3fff700 1 -- 172.21.15.12:0/1131759035 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f01080b3220 con 0x55605fe53890 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f01271b5b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f01271b5b80 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:46:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f01271b5b80 20 client.15459 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f01271b5b80 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55605fe6ce70 0x55605fe6f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f01271b5b80 4 client.15459 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f01271b5b80 1 -- 172.21.15.12:0/1131759035 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55605fe22810 con 0x55605fe6ce70 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.161+0000 7f0113fff700 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55605fe6ce70 0x55605fe6f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.162+0000 7f00f3fff700 10 client.15459 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.162+0000 7f0113fff700 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55605fe6ce70 0x55605fe6f260 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960a539927c5daac server_cookie=7a7727a25fbb6256 in_seq=0 out_seq=0 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.162+0000 7f00f3fff700 1 -- 172.21.15.12:0/1131759035 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55605fe22810 con 0x55605fe6ce70 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.162+0000 7f00f3fff700 10 client.15459 handle_command_reply: tid=0 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.162+0000 7f01271b5b80 1 client.15459 shutdown 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 1 -- 172.21.15.12:0/1131759035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 msgr2=0x55605fe5caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe5caa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0108096960 tx=0x7f0108022700 comp rx=0 tx=0).stop 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 1 -- 172.21.15.12:0/1131759035 shutdown_connections 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55605fe6ce70 0x55605fe6f260 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 0x55605fe5d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe53890 0x55605fe5caa0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 1 --2- 172.21.15.12:0/1131759035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe58e00 0x55605fe5cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 1 -- 172.21.15.12:0/1131759035 >> 172.21.15.12:0/1131759035 conn(0x55605fdb1210 msgr2=0x55605fe4d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f00f3fff700 0 client.15459 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 1 -- 172.21.15.12:0/1131759035 shutdown_connections 2024-05-08T23:46:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 1 -- 172.21.15.12:0/1131759035 wait complete. 2024-05-08T23:46:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.163+0000 7f01271b5b80 20 client.15459 trim_cache size 0 max 16384 2024-05-08T23:46:34.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.181+0000 7f01271b5b80 1 Processor -- start 2024-05-08T23:46:34.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.181+0000 7f01271b5b80 1 -- start start 2024-05-08T23:46:34.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.181+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe53890 0x55605fe53350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe58e00 0x55605fe4dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 0x55605fe58280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f01271b5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fe6cce0 con 0x55605fe58e00 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f01271b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fe5e0f0 con 0x55605fe59720 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f01271b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fe59520 con 0x55605fe53890 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0113fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe58e00 0x55605fe4dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0113fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe58e00 0x55605fe4dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48452/0 (socket says 172.21.15.12:48452) 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0113fff700 1 -- 172.21.15.12:0/1620669304 learned_addr learned my addr 172.21.15.12:0/1620669304 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0118d97700 1 --2- 172.21.15.12:0/1620669304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe53890 0x55605fe53350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0119598700 1 --2- 172.21.15.12:0/1620669304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 0x55605fe58280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0113fff700 1 -- 172.21.15.12:0/1620669304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe53890 msgr2=0x55605fe53350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0113fff700 1 --2- 172.21.15.12:0/1620669304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe53890 0x55605fe53350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0113fff700 1 -- 172.21.15.12:0/1620669304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 msgr2=0x55605fe58280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:34.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0113fff700 1 --2- 172.21.15.12:0/1620669304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 0x55605fe58280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0113fff700 1 -- 172.21.15.12:0/1620669304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55605fd69560 con 0x55605fe58e00 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0119598700 1 --2- 172.21.15.12:0/1620669304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 0x55605fe58280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f0113fff700 1 --2- 172.21.15.12:0/1620669304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe58e00 0x55605fe4dd20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0100009c80 tx=0x7f0100009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff09601d04e8b8c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f00ff7fe700 1 -- 172.21.15.12:0/1620669304 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100005ce0 con 0x55605fe58e00 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f00ff7fe700 1 -- 172.21.15.12:0/1620669304 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0100005500 con 0x55605fe58e00 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.182+0000 7f00ff7fe700 1 -- 172.21.15.12:0/1620669304 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100028590 con 0x55605fe58e00 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.183+0000 7f01271b5b80 1 -- 172.21.15.12:0/1620669304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe58e00 msgr2=0x55605fe4dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.183+0000 7f01271b5b80 1 --2- 172.21.15.12:0/1620669304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe58e00 0x55605fe4dd20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0100009c80 tx=0x7f0100009240 comp rx=0 tx=0).stop 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.183+0000 7f01271b5b80 1 -- 172.21.15.12:0/1620669304 shutdown_connections 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.183+0000 7f01271b5b80 1 --2- 172.21.15.12:0/1620669304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe59720 0x55605fe58280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.183+0000 7f01271b5b80 1 --2- 172.21.15.12:0/1620669304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe53890 0x55605fe53350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.183+0000 7f01271b5b80 1 --2- 172.21.15.12:0/1620669304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe58e00 0x55605fe4dd20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.183+0000 7f01271b5b80 1 -- 172.21.15.12:0/1620669304 >> 172.21.15.12:0/1620669304 conn(0x55605fdb1210 msgr2=0x55605fe57b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.183+0000 7f01271b5b80 1 -- 172.21.15.12:0/1620669304 shutdown_connections 2024-05-08T23:46:34.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.183+0000 7f01271b5b80 1 -- 172.21.15.12:0/1620669304 wait complete. 2024-05-08T23:46:34.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f01271b5b80 1 -- start start 2024-05-08T23:46:34.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f01271b5b80 1 Processor -- start 2024-05-08T23:46:34.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe53890 0x55605fe60330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe58e00 0x55605fe60870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f01271b5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe59720 0x55605fec2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f01271b5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fec2840 con 0x55605fe59720 2024-05-08T23:46:34.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f01271b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fec2980 con 0x55605fe53890 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f01271b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605fec2ac0 con 0x55605fe58e00 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f01271b5b80 10 client.0 _ll_register_callbacks cb 0x55605fd48b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f0119598700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe59720 0x55605fec2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f0119598700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe59720 0x55605fec2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48462/0 (socket says 172.21.15.12:48462) 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.184+0000 7f0119598700 1 -- 172.21.15.12:0/234740597 learned_addr learned my addr 172.21.15.12:0/234740597 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f0113fff700 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe58e00 0x55605fe60870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f0118d97700 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe53890 0x55605fe60330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f0119598700 1 -- 172.21.15.12:0/234740597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe58e00 msgr2=0x55605fe60870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f0119598700 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe58e00 0x55605fe60870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f0119598700 1 -- 172.21.15.12:0/234740597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe53890 msgr2=0x55605fe60330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f0119598700 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe53890 0x55605fe60330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f0119598700 1 -- 172.21.15.12:0/234740597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010001e040 con 0x55605fe59720 2024-05-08T23:46:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f0118d97700 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe53890 0x55605fe60330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f0119598700 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe59720 0x55605fec2300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f010c009e00 tx=0x7f010c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e44d97afb312f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f00f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f00f3fff700 1 -- 172.21.15.12:0/234740597 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c00e070 con 0x55605fe59720 2024-05-08T23:46:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f00f3fff700 1 -- 172.21.15.12:0/234740597 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f010c021040 con 0x55605fe59720 2024-05-08T23:46:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f01271b5b80 1 -- 172.21.15.12:0/234740597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01000044f0 con 0x55605fe59720 2024-05-08T23:46:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f00f3fff700 1 -- 172.21.15.12:0/234740597 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f010c003a00 con 0x55605fe59720 2024-05-08T23:46:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f01271b5b80 10 client.15465 fetch_fsmap learned FSMap version 24 2024-05-08T23:46:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.185+0000 7f01271b5b80 1 -- 172.21.15.12:0/234740597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55605fc28cd0 con 0x55605fe59720 2024-05-08T23:46:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.186+0000 7f00f3fff700 1 -- 172.21.15.12:0/234740597 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01000044f0 con 0x55605fe59720 2024-05-08T23:46:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.186+0000 7f00f3fff700 1 -- 172.21.15.12:0/234740597 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f010c004020 con 0x55605fe59720 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.186+0000 7f01271b5b80 10 client.15465 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.186+0000 7f01271b5b80 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.186+0000 7f01271b5b80 20 client.15465 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.186+0000 7f01271b5b80 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55605fec4230 0x55605fec6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.186+0000 7f01271b5b80 4 client.15465 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.186+0000 7f01271b5b80 1 -- 172.21.15.12:0/234740597 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55605fd9bec0 con 0x55605fec4230 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.186+0000 7f0118d97700 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55605fec4230 0x55605fec6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f0118d97700 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55605fec4230 0x55605fec6650 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd0267c07dcce1ec server_cookie=711f905059faa50d in_seq=0 out_seq=0 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f00f3fff700 10 client.15465 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f00f3fff700 1 -- 172.21.15.12:0/234740597 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55605fd9bec0 con 0x55605fec4230 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f00f3fff700 10 client.15465 handle_command_reply: tid=0 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 client.15465 shutdown 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 -- 172.21.15.12:0/234740597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe59720 msgr2=0x55605fec2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe59720 0x55605fec2300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f010c009e00 tx=0x7f010c00b040 comp rx=0 tx=0).stop 2024-05-08T23:46:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 -- 172.21.15.12:0/234740597 shutdown_connections 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55605fe59720 0x55605fec2300 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55605fec4230 0x55605fec6650 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55605fe53890 0x55605fe60330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 --2- 172.21.15.12:0/234740597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55605fe58e00 0x55605fe60870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f00f3fff700 0 client.15465 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 -- 172.21.15.12:0/234740597 >> 172.21.15.12:0/234740597 conn(0x55605fdb1210 msgr2=0x55605fe5a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 -- 172.21.15.12:0/234740597 shutdown_connections 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 1 -- 172.21.15.12:0/234740597 wait complete. 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.187+0000 7f01271b5b80 20 client.15465 trim_cache size 0 max 16384 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:46:34.189 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:46:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:46:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "dcc8ec5a-23d3-457b-b352-1f5170a3d139": { 2024-05-08T23:46:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:46:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "dcc8ec5a-23d3-457b-b352-1f5170a3d139", 2024-05-08T23:46:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:46:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:46:34.190 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:46:34.190 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:46:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 -- 172.21.15.12:0/4064858213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01040a13a0 msgr2=0x7f01040a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01040a13a0 0x7f01040a3860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f010c008bc0 tx=0x7f010c0092b0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 -- 172.21.15.12:0/4064858213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01140682e0 msgr2=0x7f01140ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01140682e0 0x7f01140ffe60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f010800f690 tx=0x7f010800bfd0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 -- 172.21.15.12:0/4064858213 shutdown_connections 2024-05-08T23:46:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01040a13a0 0x7f01040a3860 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01141008e0 0x7f01141a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01140682e0 0x7f01140ffe60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 --2- 172.21.15.12:0/4064858213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0114106290 0x7f01141003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 -- 172.21.15.12:0/4064858213 >> 172.21.15.12:0/4064858213 conn(0x7f0114076f40 msgr2=0x7f01140fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:34.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 -- 172.21.15.12:0/4064858213 shutdown_connections 2024-05-08T23:46:34.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.188+0000 7f011affb700 1 -- 172.21.15.12:0/4064858213 wait complete. 2024-05-08T23:46:34.197 INFO:tasks.cephfs.filesystem:scrub status for tag:dcc8ec5a-23d3-457b-b352-1f5170a3d139 - {'path': '/', 'tag': 'dcc8ec5a-23d3-457b-b352-1f5170a3d139', 'options': 'recursive,force'} 2024-05-08T23:46:34.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.227+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1381233405 --> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e901026a0 con 0x7f0e7c0a1320 2024-05-08T23:46:34.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.300+0000 7f0e86ffd700 1 -- 172.21.15.12:0/1381233405 <== mgr.15252 v2:172.21.15.12:6800/2525408937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0e901026a0 con 0x7f0e7c0a1320 2024-05-08T23:46:34.296 INFO:teuthology.orchestra.run.smithi012.stdout:Schedule set for path / 2024-05-08T23:46:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.302+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1381233405 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e7c0a1320 msgr2=0x7f0e7c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.302+0000 7f0e97b2f700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e7c0a1320 0x7f0e7c0a37e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009550 tx=0x7f0e8c0091b0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.302+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1381233405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e900ffd60 msgr2=0x7f0e90100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.303+0000 7f0e97b2f700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e900ffd60 0x7f0e90100ac0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0e80005770 tx=0x7f0e800057a0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.303+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1381233405 shutdown_connections 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.303+0000 7f0e97b2f700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e7c0a1320 0x7f0e7c0a37e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.303+0000 7f0e97b2f700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e90105f00 0x7f0e90101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.303+0000 7f0e97b2f700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e900ff3a0 0x7f0e90100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.303+0000 7f0e97b2f700 1 --2- 172.21.15.12:0/1381233405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e900ffd60 0x7f0e90100ac0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.303+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1381233405 >> 172.21.15.12:0/1381233405 conn(0x7f0e900773b0 msgr2=0x7f0e90103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.303+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1381233405 shutdown_connections 2024-05-08T23:46:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.303+0000 7f0e97b2f700 1 -- 172.21.15.12:0/1381233405 wait complete. 2024-05-08T23:46:34.313 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-08T23:46:34.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.502+0000 7f573f8c3700 1 -- 172.21.15.12:0/3554442877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5738073c60 msgr2=0x7f5738111950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.502+0000 7f573f8c3700 1 --2- 172.21.15.12:0/3554442877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5738073c60 0x7f5738111950 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5734009990 tx=0x7f5734009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.502+0000 7f573f8c3700 1 -- 172.21.15.12:0/3554442877 shutdown_connections 2024-05-08T23:46:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.502+0000 7f573f8c3700 1 --2- 172.21.15.12:0/3554442877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5738111e90 0x7f5738114280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.502+0000 7f573f8c3700 1 --2- 172.21.15.12:0/3554442877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738073340 0x7f5738073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.502+0000 7f573f8c3700 1 --2- 172.21.15.12:0/3554442877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5738073c60 0x7f5738111950 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.502+0000 7f573f8c3700 1 -- 172.21.15.12:0/3554442877 >> 172.21.15.12:0/3554442877 conn(0x7f57380fc970 msgr2=0x7f57380fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.502+0000 7f573f8c3700 1 -- 172.21.15.12:0/3554442877 shutdown_connections 2024-05-08T23:46:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.502+0000 7f573f8c3700 1 -- 172.21.15.12:0/3554442877 wait complete. 2024-05-08T23:46:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.503+0000 7f573f8c3700 1 Processor -- start 2024-05-08T23:46:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.503+0000 7f573f8c3700 1 -- start start 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573f8c3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5738073340 0x7f57381a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573f8c3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738073c60 0x7f57381a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573f8c3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5738111e90 0x7f573819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573f8c3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573819de20 con 0x7f5738073340 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573f8c3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573819df90 con 0x7f5738111e90 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573f8c3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573819e150 con 0x7f5738073c60 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573d65f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5738073340 0x7f57381a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573d65f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5738073340 0x7f57381a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48470/0 (socket says 172.21.15.12:48470) 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573d65f700 1 -- 172.21.15.12:0/450414733 learned_addr learned my addr 172.21.15.12:0/450414733 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573ce5e700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738073c60 0x7f57381a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573de60700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5738111e90 0x7f573819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573d65f700 1 -- 172.21.15.12:0/450414733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738073c60 msgr2=0x7f57381a3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573d65f700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738073c60 0x7f57381a3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573d65f700 1 -- 172.21.15.12:0/450414733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5738111e90 msgr2=0x7f573819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573d65f700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5738111e90 0x7f573819d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573d65f700 1 -- 172.21.15.12:0/450414733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57340095f0 con 0x7f5738073340 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.504+0000 7f573d65f700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5738073340 0x7f57381a3680 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f572800c920 tx=0x7f572800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.505+0000 7f572e7fc700 1 -- 172.21.15.12:0/450414733 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728007b00 con 0x7f5738073340 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.505+0000 7f573f8c3700 1 -- 172.21.15.12:0/450414733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573819e430 con 0x7f5738073340 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.505+0000 7f572e7fc700 1 -- 172.21.15.12:0/450414733 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f572800ce70 con 0x7f5738073340 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.505+0000 7f572e7fc700 1 -- 172.21.15.12:0/450414733 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728005460 con 0x7f5738073340 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.505+0000 7f573f8c3700 1 -- 172.21.15.12:0/450414733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57381aa670 con 0x7f5738073340 2024-05-08T23:46:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.506+0000 7f573f8c3700 1 -- 172.21.15.12:0/450414733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5738104370 con 0x7f5738073340 2024-05-08T23:46:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.507+0000 7f572e7fc700 1 -- 172.21.15.12:0/450414733 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f57280055c0 con 0x7f5738073340 2024-05-08T23:46:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.507+0000 7f572e7fc700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57240a1320 0x7f57240a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.508+0000 7f572e7fc700 1 -- 172.21.15.12:0/450414733 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f57280cce50 con 0x7f5738073340 2024-05-08T23:46:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.508+0000 7f573ce5e700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57240a1320 0x7f57240a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.508+0000 7f573ce5e700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57240a1320 0x7f57240a37e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f573400b5c0 tx=0x7f5734005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:34.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.512+0000 7f572e7fc700 1 -- 172.21.15.12:0/450414733 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5728096170 con 0x7f5738073340 2024-05-08T23:46:34.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:34 smithi012 ceph-mon[42503]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 146 B/s wr, 22 op/s 2024-05-08T23:46:34.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:34 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3331340997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:34.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:34 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3810191149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.760+0000 7f573f8c3700 1 -- 172.21.15.12:0/450414733 --> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f573819ec70 con 0x7f57240a1320 2024-05-08T23:46:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:34 smithi204 ceph-mon[47373]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 146 B/s wr, 22 op/s 2024-05-08T23:46:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:34 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3331340997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:34 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3810191149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:34 smithi160 ceph-mon[46152]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 146 B/s wr, 22 op/s 2024-05-08T23:46:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:34 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3331340997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:46:34.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:34 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3810191149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:46:34.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.829+0000 7f572e7fc700 1 -- 172.21.15.12:0/450414733 <== mgr.15252 v2:172.21.15.12:6800/2525408937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f573819ec70 con 0x7f57240a1320 2024-05-08T23:46:34.825 INFO:teuthology.orchestra.run.smithi012.stdout:Retention added to path / 2024-05-08T23:46:34.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 -- 172.21.15.12:0/450414733 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57240a1320 msgr2=0x7f57240a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57240a1320 0x7f57240a37e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f573400b5c0 tx=0x7f5734005960 comp rx=0 tx=0).stop 2024-05-08T23:46:34.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 -- 172.21.15.12:0/450414733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5738073340 msgr2=0x7f57381a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5738073340 0x7f57381a3680 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f572800c920 tx=0x7f572800cc30 comp rx=0 tx=0).stop 2024-05-08T23:46:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 -- 172.21.15.12:0/450414733 shutdown_connections 2024-05-08T23:46:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57240a1320 0x7f57240a37e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5738111e90 0x7f573819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5738073340 0x7f57381a3680 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 --2- 172.21.15.12:0/450414733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738073c60 0x7f57381a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 -- 172.21.15.12:0/450414733 >> 172.21.15.12:0/450414733 conn(0x7f57380fc970 msgr2=0x7f57380fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 -- 172.21.15.12:0/450414733 shutdown_connections 2024-05-08T23:46:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.831+0000 7f573f8c3700 1 -- 172.21.15.12:0/450414733 wait complete. 2024-05-08T23:46:34.842 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T23:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.985+0000 7f5777c2a700 1 -- 172.21.15.12:0/941193965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5770105e60 msgr2=0x7f5770106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.985+0000 7f5777c2a700 1 --2- 172.21.15.12:0/941193965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5770105e60 0x7f5770106240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5760009990 tx=0x7f5760009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.985+0000 7f5777c2a700 1 -- 172.21.15.12:0/941193965 shutdown_connections 2024-05-08T23:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.985+0000 7f5777c2a700 1 --2- 172.21.15.12:0/941193965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770100760 0x7f577010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.985+0000 7f5777c2a700 1 --2- 172.21.15.12:0/941193965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5770105e60 0x7f5770106240 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.985+0000 7f5777c2a700 1 --2- 172.21.15.12:0/941193965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57700ffda0 0x7f5770100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.985+0000 7f5777c2a700 1 -- 172.21.15.12:0/941193965 >> 172.21.15.12:0/941193965 conn(0x7f5770076f10 msgr2=0x7f5770077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.985+0000 7f5777c2a700 1 -- 172.21.15.12:0/941193965 shutdown_connections 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.985+0000 7f5777c2a700 1 -- 172.21.15.12:0/941193965 wait complete. 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f5777c2a700 1 Processor -- start 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f5777c2a700 1 -- start start 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f5777c2a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57700ffda0 0x7f5770198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f5777c2a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770100760 0x7f57701985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f5777c2a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5770105e60 0x7f57701a3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f5777c2a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770198e90 con 0x7f5770105e60 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f5777c2a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57701a3650 con 0x7f5770100760 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f5777c2a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57701a3790 con 0x7f57700ffda0 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f57761c7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5770105e60 0x7f57701a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f57761c7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5770105e60 0x7f57701a3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48492/0 (socket says 172.21.15.12:48492) 2024-05-08T23:46:34.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f57761c7700 1 -- 172.21.15.12:0/3469888210 learned_addr learned my addr 172.21.15.12:0/3469888210 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f57751c5700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770100760 0x7f57701985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.986+0000 7f57759c6700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57700ffda0 0x7f5770198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f57761c7700 1 -- 172.21.15.12:0/3469888210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57700ffda0 msgr2=0x7f5770198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f57761c7700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57700ffda0 0x7f5770198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f57761c7700 1 -- 172.21.15.12:0/3469888210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770100760 msgr2=0x7f57701985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f57761c7700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770100760 0x7f57701985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f57761c7700 1 -- 172.21.15.12:0/3469888210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600095f0 con 0x7f5770105e60 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f57761c7700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5770105e60 0x7f57701a3110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5768007ee0 tx=0x7f576800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f5766ffd700 1 -- 172.21.15.12:0/3469888210 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768017070 con 0x7f5770105e60 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f5766ffd700 1 -- 172.21.15.12:0/3469888210 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f576800f040 con 0x7f5770105e60 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f5777c2a700 1 -- 172.21.15.12:0/3469888210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57701a3a70 con 0x7f5770105e60 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f5766ffd700 1 -- 172.21.15.12:0/3469888210 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768005580 con 0x7f5770105e60 2024-05-08T23:46:34.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.987+0000 7f5777c2a700 1 -- 172.21.15.12:0/3469888210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57701a3fd0 con 0x7f5770105e60 2024-05-08T23:46:34.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.988+0000 7f5777c2a700 1 -- 172.21.15.12:0/3469888210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57701082e0 con 0x7f5770105e60 2024-05-08T23:46:34.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.989+0000 7f5766ffd700 1 -- 172.21.15.12:0/3469888210 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f576800db20 con 0x7f5770105e60 2024-05-08T23:46:34.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.990+0000 7f5766ffd700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f575c0a13a0 0x7f575c0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:34.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.990+0000 7f57759c6700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f575c0a13a0 0x7f575c0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:34.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.990+0000 7f5766ffd700 1 -- 172.21.15.12:0/3469888210 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f57680cc590 con 0x7f5770105e60 2024-05-08T23:46:34.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.991+0000 7f57759c6700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f575c0a13a0 0x7f575c0a3860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f57600098e0 tx=0x7f5760005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:34.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:34.995+0000 7f5766ffd700 1 -- 172.21.15.12:0/3469888210 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5768095850 con 0x7f5770105e60 2024-05-08T23:46:35.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.242+0000 7f5777c2a700 1 -- 172.21.15.12:0/3469888210 --> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f577010fcb0 con 0x7f575c0a13a0 2024-05-08T23:46:35.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.251+0000 7f5766ffd700 1 -- 172.21.15.12:0/3469888210 <== mgr.15252 v2:172.21.15.12:6800/2525408937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f577010fcb0 con 0x7f575c0a13a0 2024-05-08T23:46:35.246 INFO:teuthology.orchestra.run.smithi012.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T23:46:34", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-08T23:46:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.253+0000 7f5777c2a700 1 -- 172.21.15.12:0/3469888210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f575c0a13a0 msgr2=0x7f575c0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.253+0000 7f5777c2a700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f575c0a13a0 0x7f575c0a3860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f57600098e0 tx=0x7f5760005960 comp rx=0 tx=0).stop 2024-05-08T23:46:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.253+0000 7f5777c2a700 1 -- 172.21.15.12:0/3469888210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5770105e60 msgr2=0x7f57701a3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.253+0000 7f5777c2a700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5770105e60 0x7f57701a3110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5768007ee0 tx=0x7f576800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.253+0000 7f5777c2a700 1 -- 172.21.15.12:0/3469888210 shutdown_connections 2024-05-08T23:46:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.253+0000 7f5777c2a700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f575c0a13a0 0x7f575c0a3860 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.253+0000 7f5777c2a700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5770105e60 0x7f57701a3110 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.253+0000 7f5777c2a700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57700ffda0 0x7f5770198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.253+0000 7f5777c2a700 1 --2- 172.21.15.12:0/3469888210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770100760 0x7f57701985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.253+0000 7f5777c2a700 1 -- 172.21.15.12:0/3469888210 >> 172.21.15.12:0/3469888210 conn(0x7f5770076f10 msgr2=0x7f57700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.254+0000 7f5777c2a700 1 -- 172.21.15.12:0/3469888210 shutdown_connections 2024-05-08T23:46:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.254+0000 7f5777c2a700 1 -- 172.21.15.12:0/3469888210 wait complete. 2024-05-08T23:46:35.263 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T23:46:35.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.408+0000 7f673159a700 1 -- 172.21.15.12:0/3972676222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c069730 msgr2=0x7f672c069bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:35.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.408+0000 7f673159a700 1 --2- 172.21.15.12:0/3972676222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c069730 0x7f672c069bb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6720009990 tx=0x7f6720009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.408+0000 7f673159a700 1 -- 172.21.15.12:0/3972676222 shutdown_connections 2024-05-08T23:46:35.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.408+0000 7f673159a700 1 --2- 172.21.15.12:0/3972676222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c06a0f0 0x7f672c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.408+0000 7f673159a700 1 --2- 172.21.15.12:0/3972676222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f672c107000 0x7f672c1073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.408+0000 7f673159a700 1 --2- 172.21.15.12:0/3972676222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c069730 0x7f672c069bb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.408+0000 7f673159a700 1 -- 172.21.15.12:0/3972676222 >> 172.21.15.12:0/3972676222 conn(0x7f672c076fd0 msgr2=0x7f672c0773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.408+0000 7f673159a700 1 -- 172.21.15.12:0/3972676222 shutdown_connections 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.408+0000 7f673159a700 1 -- 172.21.15.12:0/3972676222 wait complete. 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f673159a700 1 Processor -- start 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f673159a700 1 -- start start 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f673159a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f672c069730 0x7f672c0ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f673159a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c06a0f0 0x7f672c100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f673159a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c107000 0x7f672c100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f673159a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c1a56a0 con 0x7f672c107000 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f673159a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c1a5810 con 0x7f672c06a0f0 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f673159a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c1a59d0 con 0x7f672c069730 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f672b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c107000 0x7f672c100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f672b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c107000 0x7f672c100890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48510/0 (socket says 172.21.15.12:48510) 2024-05-08T23:46:35.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f672b7fe700 1 -- 172.21.15.12:0/3556264647 learned_addr learned my addr 172.21.15.12:0/3556264647 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f672a7fc700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c06a0f0 0x7f672c100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.409+0000 7f672affd700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f672c069730 0x7f672c0ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f672b7fe700 1 -- 172.21.15.12:0/3556264647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f672c069730 msgr2=0x7f672c0ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f672b7fe700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f672c069730 0x7f672c0ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f672b7fe700 1 -- 172.21.15.12:0/3556264647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c06a0f0 msgr2=0x7f672c100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f672b7fe700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c06a0f0 0x7f672c100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f672b7fe700 1 -- 172.21.15.12:0/3556264647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67200095f0 con 0x7f672c107000 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f672b7fe700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c107000 0x7f672c100890 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6724007ee0 tx=0x7f672400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f6713fff700 1 -- 172.21.15.12:0/3556264647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400dd20 con 0x7f672c107000 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f673159a700 1 -- 172.21.15.12:0/3556264647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c1a5cb0 con 0x7f672c107000 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f6713fff700 1 -- 172.21.15.12:0/3556264647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f672400f040 con 0x7f672c107000 2024-05-08T23:46:35.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f6713fff700 1 -- 172.21.15.12:0/3556264647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67240048a0 con 0x7f672c107000 2024-05-08T23:46:35.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.410+0000 7f673159a700 1 -- 172.21.15.12:0/3556264647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c1a61c0 con 0x7f672c107000 2024-05-08T23:46:35.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.411+0000 7f673159a700 1 -- 172.21.15.12:0/3556264647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672c109380 con 0x7f672c107000 2024-05-08T23:46:35.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.412+0000 7f6713fff700 1 -- 172.21.15.12:0/3556264647 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f6724003a00 con 0x7f672c107000 2024-05-08T23:46:35.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.413+0000 7f6713fff700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67140a1320 0x7f67140a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:35.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.413+0000 7f672affd700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67140a1320 0x7f67140a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:35.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.413+0000 7f6713fff700 1 -- 172.21.15.12:0/3556264647 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f67240cc6f0 con 0x7f672c107000 2024-05-08T23:46:35.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.413+0000 7f672affd700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67140a1320 0x7f67140a37e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f671c009be0 tx=0x7f671c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:35.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.418+0000 7f6713fff700 1 -- 172.21.15.12:0/3556264647 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6724095a10 con 0x7f672c107000 2024-05-08T23:46:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.665+0000 7f673159a700 1 -- 172.21.15.12:0/3556264647 --> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f672c1012a0 con 0x7f67140a1320 2024-05-08T23:46:35.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:35 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:46:35.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:35 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:46:35.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:35 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:46:35.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:35 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:46:35.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:35 smithi012 ceph-mon[42503]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.672+0000 7f6713fff700 1 -- 172.21.15.12:0/3556264647 <== mgr.15252 v2:172.21.15.12:6800/2525408937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f672c1012a0 con 0x7f67140a1320 2024-05-08T23:46:35.668 INFO:teuthology.orchestra.run.smithi012.stdout:/ 1m 6m3h 2024-05-08T23:46:35.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 -- 172.21.15.12:0/3556264647 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67140a1320 msgr2=0x7f67140a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67140a1320 0x7f67140a37e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f671c009be0 tx=0x7f671c009260 comp rx=0 tx=0).stop 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 -- 172.21.15.12:0/3556264647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c107000 msgr2=0x7f672c100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c107000 0x7f672c100890 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6724007ee0 tx=0x7f672400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 -- 172.21.15.12:0/3556264647 shutdown_connections 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67140a1320 0x7f67140a37e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c107000 0x7f672c100890 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f672c069730 0x7f672c0ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 --2- 172.21.15.12:0/3556264647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c06a0f0 0x7f672c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 -- 172.21.15.12:0/3556264647 >> 172.21.15.12:0/3556264647 conn(0x7f672c076fd0 msgr2=0x7f672c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 -- 172.21.15.12:0/3556264647 shutdown_connections 2024-05-08T23:46:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:46:35.675+0000 7f673159a700 1 -- 172.21.15.12:0/3556264647 wait complete. 2024-05-08T23:46:35.684 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-08T23:46:35.716 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-08T23:46:35.730 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-08T23:46:35.741 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-08T23:46:35.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-08T23:46:35.759 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T23:46:35.773 INFO:tasks.workunit:Pulling workunits from ref d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:46:35.773 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T23:46:35.773 INFO:tasks.workunit:timeout=3h 2024-05-08T23:46:35.773 INFO:tasks.workunit:cleanup=True 2024-05-08T23:46:35.774 DEBUG:teuthology.orchestra.run.smithi012:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:46:35.775 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:35 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:46:35.775 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:35 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:46:35.775 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:35 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:46:35.775 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:35 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:46:35.775 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:35 smithi160 ceph-mon[46152]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:35.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:35 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:46:35.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:35 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:46:35.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:35 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:46:35.776 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:35 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:46:35.776 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:35 smithi204 ceph-mon[47373]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T23:46:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-08T23:46:35.796 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2024-05-08T23:46:35.796 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:46:35.796 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:cephfs_t:s0 2024-05-08T23:46:35.796 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-08 23:45:10.820753976 +0000 2024-05-08T23:46:35.796 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-08 23:45:10.820753976 +0000 2024-05-08T23:46:35.797 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-08 23:46:25.318615184 +0000 2024-05-08T23:46:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-05-08 23:45:10.820753976 +0000 2024-05-08T23:46:35.797 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T23:46:35.797 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T23:46:35.875 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:46:35.938 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T23:46:36.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:36 smithi012 ceph-mon[42503]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:36.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:36 smithi012 ceph-mon[42503]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:36.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:36 smithi012 ceph-mon[42503]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 114 B/s wr, 17 op/s 2024-05-08T23:46:36.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:36 smithi204 ceph-mon[47373]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:36.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:36 smithi204 ceph-mon[47373]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:36.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:36 smithi204 ceph-mon[47373]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 114 B/s wr, 17 op/s 2024-05-08T23:46:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:36 smithi160 ceph-mon[46152]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:36.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:36 smithi160 ceph-mon[46152]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:36.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:36 smithi160 ceph-mon[46152]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 114 B/s wr, 17 op/s 2024-05-08T23:46:37.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:37 smithi012 ceph-mon[42503]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:37.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:37 smithi204 ceph-mon[47373]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:37.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:37 smithi160 ceph-mon[46152]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:46:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:38 smithi012 ceph-mon[42503]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-05-08T23:46:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:38 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:46:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:38 smithi204 ceph-mon[47373]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-05-08T23:46:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:38 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:46:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:38 smithi160 ceph-mon[46152]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-05-08T23:46:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:38 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:46:40.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:40 smithi012 ceph-mon[42503]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-05-08T23:46:40.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:40 smithi204 ceph-mon[47373]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-05-08T23:46:40.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:40 smithi160 ceph-mon[46152]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-05-08T23:46:42.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:42 smithi012 ceph-mon[42503]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-05-08T23:46:42.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:42 smithi204 ceph-mon[47373]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-05-08T23:46:42.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:42 smithi160 ceph-mon[46152]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-05-08T23:46:44.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:44 smithi012 ceph-mon[42503]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-05-08T23:46:44.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:44 smithi204 ceph-mon[47373]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-05-08T23:46:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:44 smithi160 ceph-mon[46152]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-05-08T23:46:46.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:46 smithi012 ceph-mon[42503]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T23:46:46.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:46 smithi204 ceph-mon[47373]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T23:46:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:46 smithi160 ceph-mon[46152]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-08T23:46:48.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:48 smithi012 ceph-mon[42503]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T23:46:48.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:48 smithi204 ceph-mon[47373]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T23:46:48.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:48 smithi160 ceph-mon[46152]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T23:46:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 -- 172.21.15.12:0/267603287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d38073c70 msgr2=0x7f5d3810d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 --2- 172.21.15.12:0/267603287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d38073c70 0x7f5d3810d200 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009990 tx=0x7f5d28009ca0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 -- 172.21.15.12:0/267603287 shutdown_connections 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 --2- 172.21.15.12:0/267603287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d3810d740 0x7f5d3810fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 --2- 172.21.15.12:0/267603287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38073350 0x7f5d38073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 --2- 172.21.15.12:0/267603287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d38073c70 0x7f5d3810d200 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 -- 172.21.15.12:0/267603287 >> 172.21.15.12:0/267603287 conn(0x7f5d380fc9d0 msgr2=0x7f5d380fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 -- 172.21.15.12:0/267603287 shutdown_connections 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 -- 172.21.15.12:0/267603287 wait complete. 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 Processor -- start 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.945+0000 7f5d3f33d700 1 -- start start 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3f33d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d38073350 0x7f5d38199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3f33d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38073c70 0x7f5d38199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3f33d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3810d740 0x7f5d3819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3f33d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3819e3d0 con 0x7f5d3810d740 2024-05-08T23:46:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3f33d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3819e540 con 0x7f5d38073350 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3f33d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3819e700 con 0x7f5d38073c70 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3d8da700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3810d740 0x7f5d3819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3d8da700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3810d740 0x7f5d3819dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45620/0 (socket says 172.21.15.12:45620) 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3d8da700 1 -- 172.21.15.12:0/297382735 learned_addr learned my addr 172.21.15.12:0/297382735 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d38073350 0x7f5d38199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3c8d8700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38073c70 0x7f5d38199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3d8da700 1 -- 172.21.15.12:0/297382735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38073c70 msgr2=0x7f5d38199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3d8da700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38073c70 0x7f5d38199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3d8da700 1 -- 172.21.15.12:0/297382735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d38073350 msgr2=0x7f5d38199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3d8da700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d38073350 0x7f5d38199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3d8da700 1 -- 172.21.15.12:0/297382735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d280095f0 con 0x7f5d3810d740 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.946+0000 7f5d3d8da700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3810d740 0x7f5d3819dd90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5d30007ee0 tx=0x7f5d3000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.947+0000 7f5d2e7fc700 1 -- 172.21.15.12:0/297382735 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30017070 con 0x7f5d3810d740 2024-05-08T23:46:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.947+0000 7f5d2e7fc700 1 -- 172.21.15.12:0/297382735 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d3000f040 con 0x7f5d3810d740 2024-05-08T23:46:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.947+0000 7f5d3f33d700 1 -- 172.21.15.12:0/297382735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3819e9e0 con 0x7f5d3810d740 2024-05-08T23:46:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.947+0000 7f5d2e7fc700 1 -- 172.21.15.12:0/297382735 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30005580 con 0x7f5d3810d740 2024-05-08T23:46:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.947+0000 7f5d3f33d700 1 -- 172.21.15.12:0/297382735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d381a6220 con 0x7f5d3810d740 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.949+0000 7f5d2e7fc700 1 -- 172.21.15.12:0/297382735 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f5d3000db20 con 0x7f5d3810d740 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.949+0000 7f5d2e7fc700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d240a13a0 0x7f5d240a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.950+0000 7f5d2e7fc700 1 -- 172.21.15.12:0/297382735 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5d300cc4c0 con 0x7f5d3810d740 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.950+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d240a13a0 0x7f5d240a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.950+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d240a13a0 0x7f5d240a3860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400ac50 tx=0x7f5d3400a380 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d4b4f7b80 1 Processor -- start 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d4b4f7b80 1 -- start start 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b3270 0x55c35e9acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9ad480 0x55c35e9ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e9040f0 con 0x55c35e9b2950 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e901ab0 con 0x55c35e9b3270 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e8fd3a0 con 0x55c35e9ad480 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d3d0d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d3d0d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45624/0 (socket says 172.21.15.12:45624) 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d3d0d9700 1 -- 172.21.15.12:0/129466840 learned_addr learned my addr 172.21.15.12:0/129466840 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d3d8da700 1 --2- 172.21.15.12:0/129466840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9ad480 0x55c35e9ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.951+0000 7f5d3c8d8700 1 --2- 172.21.15.12:0/129466840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b3270 0x55c35e9acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.952+0000 7f5d3d0d9700 1 -- 172.21.15.12:0/129466840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9ad480 msgr2=0x55c35e9ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.952+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/129466840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9ad480 0x55c35e9ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.952+0000 7f5d3d0d9700 1 -- 172.21.15.12:0/129466840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b3270 msgr2=0x55c35e9acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.952+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/129466840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b3270 0x55c35e9acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.952+0000 7f5d3d0d9700 1 -- 172.21.15.12:0/129466840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c35e77e350 con 0x55c35e9b2950 2024-05-08T23:46:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.952+0000 7f5d3c8d8700 1 --2- 172.21.15.12:0/129466840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b3270 0x55c35e9acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.952+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/129466840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9b2d30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400ff90 tx=0x7f5d34012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a815be5bb6d5a74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.952+0000 7f5d237fe700 1 -- 172.21.15.12:0/129466840 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34003a90 con 0x55c35e9b2950 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.952+0000 7f5d237fe700 1 -- 172.21.15.12:0/129466840 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d34003bf0 con 0x55c35e9b2950 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.952+0000 7f5d237fe700 1 -- 172.21.15.12:0/129466840 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3401d600 con 0x55c35e9b2950 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/129466840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 msgr2=0x55c35e9b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/129466840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9b2d30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400ff90 tx=0x7f5d34012000 comp rx=0 tx=0).stop 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/129466840 shutdown_connections 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/129466840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9ad480 0x55c35e9ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/129466840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9b2d30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/129466840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b3270 0x55c35e9acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/129466840 >> 172.21.15.12:0/129466840 conn(0x55c35e90b1b0 msgr2=0x55c35e9a7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/129466840 shutdown_connections 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/129466840 wait complete. 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 -- start start 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.953+0000 7f5d4b4f7b80 1 Processor -- start 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 0x55c35e9b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b2950 0x55c35e9b6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b3270 0x55c35e9b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e899500 con 0x55c35e9b3270 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e8991a0 con 0x55c35e9ad480 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e780640 con 0x55c35e9b2950 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d4b4f7b80 10 client.0 _ll_register_callbacks cb 0x55c35e8a2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d3d8da700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b3270 0x55c35e9b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d3d8da700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b3270 0x55c35e9b6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45636/0 (socket says 172.21.15.12:45636) 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d3d8da700 1 -- 172.21.15.12:0/3331444894 learned_addr learned my addr 172.21.15.12:0/3331444894 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d3c8d8700 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b2950 0x55c35e9b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 0x55c35e9b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d3d8da700 1 -- 172.21.15.12:0/3331444894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b2950 msgr2=0x55c35e9b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d3d8da700 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b2950 0x55c35e9b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d3d8da700 1 -- 172.21.15.12:0/3331444894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 msgr2=0x55c35e9b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d3d8da700 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 0x55c35e9b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.954+0000 7f5d3d8da700 1 -- 172.21.15.12:0/3331444894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3400fc40 con 0x55c35e9b3270 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d3d8da700 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b3270 0x55c35e9b6c60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5d300a1390 tx=0x7f5d30004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31d632d30426d1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3331444894 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30099440 con 0x55c35e9b3270 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3331444894 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5d3009d270 con 0x55c35e9b3270 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3331444894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c35e856210 con 0x55c35e9b3270 2024-05-08T23:46:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3331444894 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d300a18c0 con 0x55c35e9b3270 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d4b4f7b80 10 client.15495 fetch_fsmap learned FSMap version 24 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3331444894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c35e72ab00 con 0x55c35e9b3270 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3331444894 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c35e856210 con 0x55c35e9b3270 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3331444894 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5d30098930 con 0x55c35e9b3270 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d4b4f7b80 10 client.15495 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d4b4f7b80 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d4b4f7b80 20 client.15495 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c35e9c6c10 0x55c35e9c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d4b4f7b80 4 client.15495 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.955+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3331444894 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c35e97c7b0 con 0x55c35e9c6c10 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.956+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c35e9c6c10 0x55c35e9c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.956+0000 7f5d20ff9700 10 client.15495 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.956+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c35e9c6c10 0x55c35e9c9000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1deeed4fec0151 server_cookie=39881a7ac6e556de in_seq=0 out_seq=0 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3331444894 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c35e97c7b0 con 0x55c35e9c6c10 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d20ff9700 10 client.15495 handle_command_reply: tid=0 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 client.15495 shutdown 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3331444894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b3270 msgr2=0x55c35e9b6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b3270 0x55c35e9b6c60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5d300a1390 tx=0x7f5d30004f10 comp rx=0 tx=0).stop 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3331444894 shutdown_connections 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b3270 0x55c35e9b6c60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c35e9c6c10 0x55c35e9c9000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 0x55c35e9b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3331444894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b2950 0x55c35e9b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d20ff9700 0 client.15495 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3331444894 >> 172.21.15.12:0/3331444894 conn(0x55c35e90b1b0 msgr2=0x55c35e9abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3331444894 shutdown_connections 2024-05-08T23:46:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3331444894 wait complete. 2024-05-08T23:46:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.957+0000 7f5d4b4f7b80 20 client.15495 trim_cache size 0 max 16384 2024-05-08T23:46:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d4b4f7b80 1 Processor -- start 2024-05-08T23:46:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d4b4f7b80 1 -- start start 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9ad480 0x55c35e9a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b2950 0x55c35e9a9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 0x55c35e9ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e9a8330 con 0x55c35e9ad480 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e9adda0 con 0x55c35e9b2950 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e9a8a30 con 0x55c35e9b3270 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d3d0d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9ad480 0x55c35e9a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d3d0d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9ad480 0x55c35e9a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45652/0 (socket says 172.21.15.12:45652) 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d3d0d9700 1 -- 172.21.15.12:0/1225469071 learned_addr learned my addr 172.21.15.12:0/1225469071 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.973+0000 7f5d3c8d8700 1 --2- 172.21.15.12:0/1225469071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b2950 0x55c35e9a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d3d0d9700 1 -- 172.21.15.12:0/1225469071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 msgr2=0x55c35e9ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d3d8da700 1 --2- 172.21.15.12:0/1225469071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 0x55c35e9ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/1225469071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 0x55c35e9ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d3d0d9700 1 -- 172.21.15.12:0/1225469071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b2950 msgr2=0x55c35e9a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/1225469071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b2950 0x55c35e9a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d3d0d9700 1 -- 172.21.15.12:0/1225469071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c35e8c3500 con 0x55c35e9ad480 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d3c8d8700 1 --2- 172.21.15.12:0/1225469071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b2950 0x55c35e9a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/1225469071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9ad480 0x55c35e9a7910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400fa30 tx=0x7f5d34005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee94c9dd1a874d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d23fff700 1 -- 172.21.15.12:0/1225469071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3402b050 con 0x55c35e9ad480 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d23fff700 1 -- 172.21.15.12:0/1225469071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d3402c040 con 0x55c35e9ad480 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.974+0000 7f5d23fff700 1 -- 172.21.15.12:0/1225469071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d340054e0 con 0x55c35e9ad480 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/1225469071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9ad480 msgr2=0x55c35e9a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/1225469071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9ad480 0x55c35e9a7910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400fa30 tx=0x7f5d34005df0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/1225469071 shutdown_connections 2024-05-08T23:46:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/1225469071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 0x55c35e9ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/1225469071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9ad480 0x55c35e9a7910 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/1225469071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9b2950 0x55c35e9a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/1225469071 >> 172.21.15.12:0/1225469071 conn(0x55c35e90b1b0 msgr2=0x55c35e9a3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/1225469071 shutdown_connections 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/1225469071 wait complete. 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 -- start start 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 Processor -- start 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 0x55c35e9bde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9bbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.975+0000 7f5d4b4f7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 0x55c35e9bc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e9bc6a0 con 0x55c35e9b2950 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e9bc810 con 0x55c35e9ad480 2024-05-08T23:46:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d4b4f7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e9bc9d0 con 0x55c35e9b3270 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d4b4f7b80 10 client.0 _ll_register_callbacks cb 0x55c35e8a2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3c8d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9bbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3c8d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9bbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45668/0 (socket says 172.21.15.12:45668) 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3c8d8700 1 -- 172.21.15.12:0/3553072996 learned_addr learned my addr 172.21.15.12:0/3553072996 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 0x55c35e9bde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3c8d8700 1 -- 172.21.15.12:0/3553072996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 msgr2=0x55c35e9bc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3d8da700 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 0x55c35e9bc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3c8d8700 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 0x55c35e9bc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3c8d8700 1 -- 172.21.15.12:0/3553072996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 msgr2=0x55c35e9bde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3c8d8700 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 0x55c35e9bde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3c8d8700 1 -- 172.21.15.12:0/3553072996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c35e77e350 con 0x55c35e9b2950 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 0x55c35e9bde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3d8da700 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 0x55c35e9bc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:46:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d3c8d8700 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9bbc20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b740 tx=0x7f5d280050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477036cd127c0e22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3553072996 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d280056e0 con 0x55c35e9b2950 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3553072996 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5d28003790 con 0x55c35e9b2950 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.976+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3553072996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d34003bf0 con 0x55c35e9b2950 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3553072996 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d28003950 con 0x55c35e9b2950 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d4b4f7b80 10 client.15501 fetch_fsmap learned FSMap version 24 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3553072996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c35e782c70 con 0x55c35e9b2950 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3553072996 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d34003bf0 con 0x55c35e9b2950 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3553072996 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5d28024cc0 con 0x55c35e9b2950 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d4b4f7b80 10 client.15501 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d4b4f7b80 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d4b4f7b80 20 client.15501 populate_metadata read hostname 'smithi012' 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c35ea1d560 0x55c35ea1f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:46:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d4b4f7b80 4 client.15501 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:46:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3553072996 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c35e8f5e60 con 0x55c35ea1d560 2024-05-08T23:46:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.977+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c35ea1d560 0x55c35ea1f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:46:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.978+0000 7f5d3d0d9700 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c35ea1d560 0x55c35ea1f980 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44335232f9e7284c server_cookie=d04665393a28d39a in_seq=0 out_seq=0 2024-05-08T23:46:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.978+0000 7f5d20ff9700 10 client.15501 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d20ff9700 1 -- 172.21.15.12:0/3553072996 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c35e8f5e60 con 0x55c35ea1d560 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d20ff9700 10 client.15501 handle_command_reply: tid=0 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 client.15501 shutdown 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3553072996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 msgr2=0x55c35e9bbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9bbc20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b740 tx=0x7f5d280050b0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3553072996 shutdown_connections 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c35e9b3270 0x55c35e9bc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c35ea1d560 0x55c35ea1f980 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c35e9ad480 0x55c35e9bde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d20ff9700 0 client.15501 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:46:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 --2- 172.21.15.12:0/3553072996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c35e9b2950 0x55c35e9bbc20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3553072996 >> 172.21.15.12:0/3553072996 conn(0x55c35e90b1b0 msgr2=0x55c35e9b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3553072996 shutdown_connections 2024-05-08T23:46:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 1 -- 172.21.15.12:0/3553072996 wait complete. 2024-05-08T23:46:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.989+0000 7f5d4b4f7b80 20 client.15501 trim_cache size 0 max 16384 2024-05-08T23:46:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:46:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:46:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:46:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:46:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 -- 172.21.15.12:0/297382735 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d240a13a0 msgr2=0x7f5d240a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d240a13a0 0x7f5d240a3860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400ac50 tx=0x7f5d3400a380 comp rx=0 tx=0).stop 2024-05-08T23:46:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 -- 172.21.15.12:0/297382735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3810d740 msgr2=0x7f5d3819dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:46:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3810d740 0x7f5d3819dd90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5d30007ee0 tx=0x7f5d3000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 -- 172.21.15.12:0/297382735 shutdown_connections 2024-05-08T23:46:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d240a13a0 0x7f5d240a3860 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3810d740 0x7f5d3819dd90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d38073350 0x7f5d38199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 --2- 172.21.15.12:0/297382735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38073c70 0x7f5d38199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:46:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 -- 172.21.15.12:0/297382735 >> 172.21.15.12:0/297382735 conn(0x7f5d380fc9d0 msgr2=0x7f5d380fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:46:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 -- 172.21.15.12:0/297382735 shutdown_connections 2024-05-08T23:46:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:46:48.990+0000 7f5d3f33d700 1 -- 172.21.15.12:0/297382735 wait complete. 2024-05-08T23:46:50.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:50 smithi012 ceph-mon[42503]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:46:50.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:50 smithi204 ceph-mon[47373]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:46:50.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:50 smithi160 ceph-mon[46152]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:46:52.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:52 smithi012 ceph-mon[42503]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:46:52.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:52 smithi204 ceph-mon[47373]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:46:52.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:52 smithi160 ceph-mon[46152]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:46:54.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:54 smithi012 ceph-mon[42503]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-08T23:46:54.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:54 smithi204 ceph-mon[47373]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-08T23:46:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:54 smithi160 ceph-mon[46152]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-08T23:46:56.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:56 smithi012 ceph-mon[42503]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-08T23:46:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:56 smithi204 ceph-mon[47373]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-08T23:46:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:56 smithi160 ceph-mon[46152]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-08T23:46:58.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:46:58 smithi012 ceph-mon[42503]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T23:46:58.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:46:58 smithi204 ceph-mon[47373]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T23:46:58.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:46:58 smithi160 ceph-mon[46152]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T23:47:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:00 smithi012 ceph-mon[42503]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-08T23:47:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:00 smithi204 ceph-mon[47373]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-08T23:47:00.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:00 smithi160 ceph-mon[46152]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-08T23:47:01.165 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:47:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:47:00.840+0000 7f7c3cf3b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:47:01.165 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:47:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:47:00.840+0000 7f7c3cf3b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:47:01.165 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:47:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:47:00.840+0000 7f7c3cf3b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:47:01.165 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:47:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:47:00.841+0000 7f7c3cf3b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:47:01.166 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 08 23:47:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-08T23:47:00.841+0000 7f7c3cf3b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:47:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.117+0000 7ff553c6d700 1 -- 172.21.15.12:0/120622676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c100010 msgr2=0x7ff54c1003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.117+0000 7ff553c6d700 1 --2- 172.21.15.12:0/120622676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c100010 0x7ff54c1003f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff548009990 tx=0x7ff548009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.117+0000 7ff553c6d700 1 -- 172.21.15.12:0/120622676 shutdown_connections 2024-05-08T23:47:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.118+0000 7ff553c6d700 1 --2- 172.21.15.12:0/120622676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c105130 0x7ff54c10ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.118+0000 7ff553c6d700 1 --2- 172.21.15.12:0/120622676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c100010 0x7ff54c1003f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.118+0000 7ff553c6d700 1 --2- 172.21.15.12:0/120622676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff54c1009c0 0x7ff54c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.118+0000 7ff553c6d700 1 -- 172.21.15.12:0/120622676 >> 172.21.15.12:0/120622676 conn(0x7ff54c076f40 msgr2=0x7ff54c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.118+0000 7ff553c6d700 1 -- 172.21.15.12:0/120622676 shutdown_connections 2024-05-08T23:47:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.118+0000 7ff553c6d700 1 -- 172.21.15.12:0/120622676 wait complete. 2024-05-08T23:47:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.118+0000 7ff553c6d700 1 Processor -- start 2024-05-08T23:47:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.118+0000 7ff553c6d700 1 -- start start 2024-05-08T23:47:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.118+0000 7ff553c6d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff54c100010 0x7ff54c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff553c6d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c105130 0x7ff54c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff553c6d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c198c40 0x7ff54c1a3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff553c6d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1a3920 con 0x7ff54c198c40 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff553c6d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1a3a90 con 0x7ff54c100010 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff553c6d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1a3c50 con 0x7ff54c105130 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff55220a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c198c40 0x7ff54c1a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff55220a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c198c40 0x7ff54c1a3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51658/0 (socket says 172.21.15.12:51658) 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff55220a700 1 -- 172.21.15.12:0/300943891 learned_addr learned my addr 172.21.15.12:0/300943891 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff551a09700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff54c100010 0x7ff54c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff551208700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c105130 0x7ff54c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff55220a700 1 -- 172.21.15.12:0/300943891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c105130 msgr2=0x7ff54c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff55220a700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c105130 0x7ff54c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff55220a700 1 -- 172.21.15.12:0/300943891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff54c100010 msgr2=0x7ff54c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff55220a700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff54c100010 0x7ff54c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff55220a700 1 -- 172.21.15.12:0/300943891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5480095f0 con 0x7ff54c198c40 2024-05-08T23:47:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff55220a700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c198c40 0x7ff54c1a3240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff54400ef30 tx=0x7ff54400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff542ffd700 1 -- 172.21.15.12:0/300943891 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544018070 con 0x7ff54c198c40 2024-05-08T23:47:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff542ffd700 1 -- 172.21.15.12:0/300943891 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff54400f040 con 0x7ff54c198c40 2024-05-08T23:47:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.119+0000 7ff553c6d700 1 -- 172.21.15.12:0/300943891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c1a3ed0 con 0x7ff54c198c40 2024-05-08T23:47:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.120+0000 7ff542ffd700 1 -- 172.21.15.12:0/300943891 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400cd00 con 0x7ff54c198c40 2024-05-08T23:47:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.120+0000 7ff553c6d700 1 -- 172.21.15.12:0/300943891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c1a4400 con 0x7ff54c198c40 2024-05-08T23:47:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.122+0000 7ff542ffd700 1 -- 172.21.15.12:0/300943891 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7ff544003a00 con 0x7ff54c198c40 2024-05-08T23:47:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.122+0000 7ff542ffd700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5380a13a0 0x7ff5380a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.123+0000 7ff542ffd700 1 -- 172.21.15.12:0/300943891 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff5440cc880 con 0x7ff54c198c40 2024-05-08T23:47:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.123+0000 7ff551a09700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5380a13a0 0x7ff5380a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.123+0000 7ff55fe27b80 1 Processor -- start 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.123+0000 7ff551a09700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5380a13a0 0x7ff5380a3860 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff54800b2e0 tx=0x7ff548005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.123+0000 7ff55fe27b80 1 -- start start 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1537950 0x55ceb1537d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1538270 0x55ceb1531f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 0x55ceb15328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb14890f0 con 0x55ceb1537950 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb1486ab0 con 0x55ceb1532480 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb14823a0 con 0x55ceb1538270 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff551a09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1537950 0x55ceb1537d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff551a09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1537950 0x55ceb1537d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51662/0 (socket says 172.21.15.12:51662) 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff551a09700 1 -- 172.21.15.12:0/3486535671 learned_addr learned my addr 172.21.15.12:0/3486535671 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff551208700 1 --2- 172.21.15.12:0/3486535671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1538270 0x55ceb1531f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff55220a700 1 --2- 172.21.15.12:0/3486535671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 0x55ceb15328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff551a09700 1 -- 172.21.15.12:0/3486535671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1538270 msgr2=0x55ceb1531f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff551a09700 1 --2- 172.21.15.12:0/3486535671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1538270 0x55ceb1531f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff551a09700 1 -- 172.21.15.12:0/3486535671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 msgr2=0x55ceb15328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff551a09700 1 --2- 172.21.15.12:0/3486535671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 0x55ceb15328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff551a09700 1 -- 172.21.15.12:0/3486535671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceb1303350 con 0x55ceb1537950 2024-05-08T23:47:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.124+0000 7ff55220a700 1 --2- 172.21.15.12:0/3486535671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 0x55ceb15328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.125+0000 7ff551a09700 1 --2- 172.21.15.12:0/3486535671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1537950 0x55ceb1537d30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff548015000 tx=0x7ff548020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78efebaabaf464ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.125+0000 7ff537fff700 1 -- 172.21.15.12:0/3486535671 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548004700 con 0x55ceb1537950 2024-05-08T23:47:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.125+0000 7ff537fff700 1 -- 172.21.15.12:0/3486535671 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff54801e040 con 0x55ceb1537950 2024-05-08T23:47:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.125+0000 7ff537fff700 1 -- 172.21.15.12:0/3486535671 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54800f960 con 0x55ceb1537950 2024-05-08T23:47:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.125+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3486535671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1537950 msgr2=0x55ceb1537d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.125+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3486535671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1537950 0x55ceb1537d30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff548015000 tx=0x7ff548020000 comp rx=0 tx=0).stop 2024-05-08T23:47:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.126+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3486535671 shutdown_connections 2024-05-08T23:47:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.126+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3486535671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 0x55ceb15328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.126+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3486535671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1537950 0x55ceb1537d30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.126+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3486535671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1538270 0x55ceb1531f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.126+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3486535671 >> 172.21.15.12:0/3486535671 conn(0x55ceb14901b0 msgr2=0x55ceb152c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.126+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3486535671 shutdown_connections 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.126+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3486535671 wait complete. 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.126+0000 7ff55fe27b80 1 -- start start 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.126+0000 7ff55fe27b80 1 Processor -- start 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1532480 0x55ceb153e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 0x55ceb153b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1538270 0x55ceb153bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb141e500 con 0x55ceb1532480 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb141e1a0 con 0x55ceb1537950 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb1305640 con 0x55ceb1538270 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff55fe27b80 10 client.0 _ll_register_callbacks cb 0x55ceb1427ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff551a09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1532480 0x55ceb153e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff551a09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1532480 0x55ceb153e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51676/0 (socket says 172.21.15.12:51676) 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff551a09700 1 -- 172.21.15.12:0/3642887187 learned_addr learned my addr 172.21.15.12:0/3642887187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff551208700 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 0x55ceb153b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff55220a700 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1538270 0x55ceb153bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff551a09700 1 -- 172.21.15.12:0/3642887187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1538270 msgr2=0x55ceb153bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff551a09700 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1538270 0x55ceb153bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff551a09700 1 -- 172.21.15.12:0/3642887187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 msgr2=0x55ceb153b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff551a09700 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 0x55ceb153b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff551a09700 1 -- 172.21.15.12:0/3642887187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c10d520 con 0x55ceb1532480 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff551a09700 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1532480 0x55ceb153e210 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff548000c00 tx=0x7ff5480057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4add4098dc2dd5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.127+0000 7ff5357fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff5357fa700 1 -- 172.21.15.12:0/3642887187 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548004700 con 0x55ceb1532480 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3642887187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ceb13db210 con 0x55ceb1532480 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff5357fa700 1 -- 172.21.15.12:0/3642887187 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff54800be60 con 0x55ceb1532480 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff5357fa700 1 -- 172.21.15.12:0/3642887187 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff548004860 con 0x55ceb1532480 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff55fe27b80 10 client.15519 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3642887187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ceb12afb00 con 0x55ceb1532480 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff5357fa700 1 -- 172.21.15.12:0/3642887187 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ceb13db210 con 0x55ceb1532480 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff5357fa700 1 -- 172.21.15.12:0/3642887187 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff54802fa90 con 0x55ceb1532480 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff55fe27b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff55fe27b80 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff55fe27b80 20 client.15519 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ceb154b740 0x55ceb154db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff55fe27b80 4 client.15519 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.128+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3642887187 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ceb15017b0 con 0x55ceb154b740 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.129+0000 7ff551208700 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ceb154b740 0x55ceb154db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.129+0000 7ff551208700 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ceb154b740 0x55ceb154db60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5149661c1074f2 server_cookie=3046eb7ebf02c488 in_seq=0 out_seq=0 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.129+0000 7ff5357fa700 10 client.15519 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff5357fa700 1 -- 172.21.15.12:0/3642887187 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ceb15017b0 con 0x55ceb154b740 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff5357fa700 10 client.15519 handle_command_reply: tid=0 2024-05-08T23:47:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 client.15519 shutdown 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3642887187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1532480 msgr2=0x55ceb153e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1532480 0x55ceb153e210 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff548000c00 tx=0x7ff5480057e0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3642887187 shutdown_connections 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1538270 0x55ceb153bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1532480 0x55ceb153e210 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ceb154b740 0x55ceb154db60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3642887187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 0x55ceb153b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff5357fa700 0 client.15519 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3642887187 >> 172.21.15.12:0/3642887187 conn(0x55ceb14901b0 msgr2=0x55ceb1530f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3642887187 shutdown_connections 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3642887187 wait complete. 2024-05-08T23:47:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.130+0000 7ff55fe27b80 20 client.15519 trim_cache size 0 max 16384 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.146+0000 7ff55fe27b80 1 Processor -- start 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.146+0000 7ff55fe27b80 1 -- start start 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.146+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1532480 0x55ceb152c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.146+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 0x55ceb152e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.146+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb153dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.146+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb15322e0 con 0x55ceb1538270 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.146+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb1552d30 con 0x55ceb1537950 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.146+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb1538080 con 0x55ceb1532480 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff55220a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb153dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff55220a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb153dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51680/0 (socket says 172.21.15.12:51680) 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff55220a700 1 -- 172.21.15.12:0/1127613703 learned_addr learned my addr 172.21.15.12:0/1127613703 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff551a09700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1532480 0x55ceb152c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff551208700 1 --2- 172.21.15.12:0/1127613703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 0x55ceb152e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff55220a700 1 -- 172.21.15.12:0/1127613703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1532480 msgr2=0x55ceb152c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff55220a700 1 --2- 172.21.15.12:0/1127613703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1532480 0x55ceb152c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff55220a700 1 -- 172.21.15.12:0/1127613703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 msgr2=0x55ceb152e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff55220a700 1 --2- 172.21.15.12:0/1127613703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 0x55ceb152e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff55220a700 1 -- 172.21.15.12:0/1127613703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceb1448500 con 0x55ceb1538270 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff551208700 1 --2- 172.21.15.12:0/1127613703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 0x55ceb152e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff551a09700 1 --2- 172.21.15.12:0/1127613703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1532480 0x55ceb152c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff55220a700 1 --2- 172.21.15.12:0/1127613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb153dd40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff544099880 tx=0x7ff544009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf279db08727f96f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff540da4700 1 -- 172.21.15.12:0/1127613703 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54401b320 con 0x55ceb1538270 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff540da4700 1 -- 172.21.15.12:0/1127613703 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff544099ff0 con 0x55ceb1538270 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.147+0000 7ff540da4700 1 -- 172.21.15.12:0/1127613703 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440ca440 con 0x55ceb1538270 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 -- 172.21.15.12:0/1127613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 msgr2=0x55ceb153dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/1127613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb153dd40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff544099880 tx=0x7ff544009430 comp rx=0 tx=0).stop 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 -- 172.21.15.12:0/1127613703 shutdown_connections 2024-05-08T23:47:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/1127613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb153dd40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/1127613703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1532480 0x55ceb152c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/1127613703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1537950 0x55ceb152e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 -- 172.21.15.12:0/1127613703 >> 172.21.15.12:0/1127613703 conn(0x55ceb14901b0 msgr2=0x55ceb1539e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 -- 172.21.15.12:0/1127613703 shutdown_connections 2024-05-08T23:47:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 -- 172.21.15.12:0/1127613703 wait complete. 2024-05-08T23:47:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 -- start start 2024-05-08T23:47:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.148+0000 7ff55fe27b80 1 Processor -- start 2024-05-08T23:47:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 0x55ceb1540bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1537950 0x55ceb1541130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55fe27b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb1541670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb15a0ca0 con 0x55ceb1538270 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb15a0e10 con 0x55ceb1532480 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55fe27b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb15a0fd0 con 0x55ceb1537950 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55fe27b80 10 client.0 _ll_register_callbacks cb 0x55ceb1427ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55220a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb1541670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55220a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb1541670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51690/0 (socket says 172.21.15.12:51690) 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55220a700 1 -- 172.21.15.12:0/3888655418 learned_addr learned my addr 172.21.15.12:0/3888655418 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff551208700 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1537950 0x55ceb1541130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff551a09700 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 0x55ceb1540bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55220a700 1 -- 172.21.15.12:0/3888655418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1537950 msgr2=0x55ceb1541130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55220a700 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1537950 0x55ceb1541130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55220a700 1 -- 172.21.15.12:0/3888655418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 msgr2=0x55ceb1540bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55220a700 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 0x55ceb1540bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55220a700 1 -- 172.21.15.12:0/3888655418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ceb12afb00 con 0x55ceb1538270 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff55220a700 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb1541670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff5440a17b0 tx=0x7ff5440092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b583e79cf7c4408 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.149+0000 7ff5357fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff5357fa700 1 -- 172.21.15.12:0/3888655418 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544099d50 con 0x55ceb1538270 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff5357fa700 1 -- 172.21.15.12:0/3888655418 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff5440b1070 con 0x55ceb1538270 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3888655418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff54801e040 con 0x55ceb1538270 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff5357fa700 1 -- 172.21.15.12:0/3888655418 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff54400ce60 con 0x55ceb1538270 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff55fe27b80 10 client.15525 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3888655418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7ff54c1a4400 con 0x55ceb1538270 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff5357fa700 1 -- 172.21.15.12:0/3888655418 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff54801e040 con 0x55ceb1538270 2024-05-08T23:47:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff5357fa700 1 -- 172.21.15.12:0/3888655418 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff544098de0 con 0x55ceb1538270 2024-05-08T23:47:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff55fe27b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff55fe27b80 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff55fe27b80 20 client.15525 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ceb15a2330 0x55ceb15a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff55fe27b80 4 client.15525 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:47:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3888655418 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ceb147ae60 con 0x55ceb15a2330 2024-05-08T23:47:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.150+0000 7ff551a09700 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ceb15a2330 0x55ceb15a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.151+0000 7ff5357fa700 10 client.15525 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.151+0000 7ff551a09700 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ceb15a2330 0x55ceb15a4750 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51b234adceaadb4 server_cookie=cf5216af6f59c261 in_seq=0 out_seq=0 2024-05-08T23:47:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:02.406+0000 7ff542ffd700 1 -- 172.21.15.12:0/300943891 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7ff544095040 con 0x7ff54c198c40 2024-05-08T23:47:02.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:02 smithi012 ceph-mon[42503]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-08T23:47:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:02 smithi204 ceph-mon[47373]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-08T23:47:02.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:02 smithi160 ceph-mon[46152]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-08T23:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff5357fa700 1 -- 172.21.15.12:0/3888655418 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ceb147ae60 con 0x55ceb15a2330 2024-05-08T23:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff5357fa700 10 client.15525 handle_command_reply: tid=0 2024-05-08T23:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 client.15525 shutdown 2024-05-08T23:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3888655418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 msgr2=0x55ceb1541670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb1541670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff5440a17b0 tx=0x7ff5440092e0 comp rx=0 tx=0).stop 2024-05-08T23:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3888655418 shutdown_connections 2024-05-08T23:47:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ceb1538270 0x55ceb1541670 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ceb15a2330 0x55ceb15a4750 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ceb1532480 0x55ceb1540bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 --2- 172.21.15.12:0/3888655418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceb1537950 0x55ceb1541130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff5357fa700 0 client.15525 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3888655418 >> 172.21.15.12:0/3888655418 conn(0x55ceb14901b0 msgr2=0x55ceb1535380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3888655418 shutdown_connections 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 1 -- 172.21.15.12:0/3888655418 wait complete. 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.343+0000 7ff55fe27b80 20 client.15525 trim_cache size 0 max 16384 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:47:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:47:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:47:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 -- 172.21.15.12:0/300943891 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5380a13a0 msgr2=0x7ff5380a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5380a13a0 0x7ff5380a3860 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff54800b2e0 tx=0x7ff548005960 comp rx=0 tx=0).stop 2024-05-08T23:47:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 -- 172.21.15.12:0/300943891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c198c40 msgr2=0x7ff54c1a3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c198c40 0x7ff54c1a3240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff54400ef30 tx=0x7ff54400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:47:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 -- 172.21.15.12:0/300943891 shutdown_connections 2024-05-08T23:47:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5380a13a0 0x7ff5380a3860 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54c198c40 0x7ff54c1a3240 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff54c100010 0x7ff54c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 --2- 172.21.15.12:0/300943891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c105130 0x7ff54c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 -- 172.21.15.12:0/300943891 >> 172.21.15.12:0/300943891 conn(0x7ff54c076f40 msgr2=0x7ff54c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 -- 172.21.15.12:0/300943891 shutdown_connections 2024-05-08T23:47:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:03.344+0000 7ff553c6d700 1 -- 172.21.15.12:0/300943891 wait complete. 2024-05-08T23:47:03.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:03 smithi012 ceph-mon[42503]: mgrmap e24: x(active, since 37s), standbys: y, z 2024-05-08T23:47:03.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:03 smithi204 ceph-mon[47373]: mgrmap e24: x(active, since 37s), standbys: y, z 2024-05-08T23:47:03.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:03 smithi160 ceph-mon[46152]: mgrmap e24: x(active, since 37s), standbys: y, z 2024-05-08T23:47:04.199 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:47:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.353+0000 7f6523fbe700 1 -- 172.21.15.12:0/2179911862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f651c101180 msgr2=0x7f651c101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.353+0000 7f6523fbe700 1 --2- 172.21.15.12:0/2179911862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f651c101180 0x7f651c101560 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6518009990 tx=0x7f6518009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.353+0000 7f6523fbe700 1 -- 172.21.15.12:0/2179911862 shutdown_connections 2024-05-08T23:47:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.353+0000 7f6523fbe700 1 --2- 172.21.15.12:0/2179911862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c1062a0 0x7f651c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.353+0000 7f6523fbe700 1 --2- 172.21.15.12:0/2179911862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f651c101180 0x7f651c101560 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.353+0000 7f6523fbe700 1 --2- 172.21.15.12:0/2179911862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f651c101b30 0x7f651c105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.353+0000 7f6523fbe700 1 -- 172.21.15.12:0/2179911862 >> 172.21.15.12:0/2179911862 conn(0x7f651c0fc9e0 msgr2=0x7f651c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.353+0000 7f6523fbe700 1 -- 172.21.15.12:0/2179911862 shutdown_connections 2024-05-08T23:47:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.353+0000 7f6523fbe700 1 -- 172.21.15.12:0/2179911862 wait complete. 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f6523fbe700 1 Processor -- start 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f6523fbe700 1 -- start start 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f6523fbe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f651c101180 0x7f651c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f6523fbe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c1062a0 0x7f651c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f6523fbe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f651c073d20 0x7f651c1a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f6523fbe700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c1a5c50 con 0x7f651c073d20 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f6523fbe700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c1a5dc0 con 0x7f651c1062a0 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f6523fbe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c1a5f80 con 0x7f651c101180 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f652255b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f651c073d20 0x7f651c1a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f652255b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f651c073d20 0x7f651c1a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51714/0 (socket says 172.21.15.12:51714) 2024-05-08T23:47:04.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f652255b700 1 -- 172.21.15.12:0/975437309 learned_addr learned my addr 172.21.15.12:0/975437309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f6521559700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c1062a0 0x7f651c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f6521d5a700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f651c101180 0x7f651c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.354+0000 7f652255b700 1 -- 172.21.15.12:0/975437309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f651c101180 msgr2=0x7f651c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.355+0000 7f652255b700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f651c101180 0x7f651c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.355+0000 7f652255b700 1 -- 172.21.15.12:0/975437309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c1062a0 msgr2=0x7f651c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.355+0000 7f652255b700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c1062a0 0x7f651c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.355+0000 7f652255b700 1 -- 172.21.15.12:0/975437309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65180095f0 con 0x7f651c073d20 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.355+0000 7f652255b700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f651c073d20 0x7f651c1a5680 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6514007d70 tx=0x7f651400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.355+0000 7f6512ffd700 1 -- 172.21.15.12:0/975437309 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400ad40 con 0x7f651c073d20 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.355+0000 7f6523fbe700 1 -- 172.21.15.12:0/975437309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c1a6260 con 0x7f651c073d20 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.355+0000 7f6512ffd700 1 -- 172.21.15.12:0/975437309 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65140128e0 con 0x7f651c073d20 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.355+0000 7f6512ffd700 1 -- 172.21.15.12:0/975437309 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140116b0 con 0x7f651c073d20 2024-05-08T23:47:04.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.355+0000 7f6523fbe700 1 -- 172.21.15.12:0/975437309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c1a6770 con 0x7f651c073d20 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.357+0000 7f6512ffd700 1 -- 172.21.15.12:0/975437309 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f6514012420 con 0x7f651c073d20 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.358+0000 7f6512ffd700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f65080a13b0 0x7f65080a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.358+0000 7f6521d5a700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f65080a13b0 0x7f65080a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.358+0000 7f6512ffd700 1 -- 172.21.15.12:0/975437309 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f65140cd130 con 0x7f651c073d20 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.358+0000 7f6521d5a700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f65080a13b0 0x7f65080a3870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6518006010 tx=0x7f651800b540 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.359+0000 7f6530178b80 1 Processor -- start 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.359+0000 7f6530178b80 1 -- start start 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.359+0000 7f6530178b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48be00 0x556acf48c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.359+0000 7f6530178b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 0x556acf486350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.359+0000 7f6530178b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf486890 0x556acf486cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.359+0000 7f6530178b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf3dd150 con 0x556acf486890 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.359+0000 7f6530178b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf3dab10 con 0x556acf48c720 2024-05-08T23:47:04.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.359+0000 7f6530178b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf3d6400 con 0x556acf48be00 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.359+0000 7f652255b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf486890 0x556acf486cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.359+0000 7f652255b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf486890 0x556acf486cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51718/0 (socket says 172.21.15.12:51718) 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f652255b700 1 -- 172.21.15.12:0/3849316533 learned_addr learned my addr 172.21.15.12:0/3849316533 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f6521d5a700 1 --2- 172.21.15.12:0/3849316533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48be00 0x556acf48c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f6521559700 1 --2- 172.21.15.12:0/3849316533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 0x556acf486350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f652255b700 1 -- 172.21.15.12:0/3849316533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48be00 msgr2=0x556acf48c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f652255b700 1 --2- 172.21.15.12:0/3849316533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48be00 0x556acf48c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f652255b700 1 -- 172.21.15.12:0/3849316533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 msgr2=0x556acf486350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f652255b700 1 --2- 172.21.15.12:0/3849316533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 0x556acf486350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f652255b700 1 -- 172.21.15.12:0/3849316533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556acf2573b0 con 0x556acf486890 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f6521559700 1 --2- 172.21.15.12:0/3849316533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 0x556acf486350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f652255b700 1 --2- 172.21.15.12:0/3849316533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf486890 0x556acf486cd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f651400a740 tx=0x7f6514099d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4040a47c26dc4859 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f6507fff700 1 -- 172.21.15.12:0/3849316533 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651409a0e0 con 0x556acf486890 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f6507fff700 1 -- 172.21.15.12:0/3849316533 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f651409df10 con 0x556acf486890 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.360+0000 7f6507fff700 1 -- 172.21.15.12:0/3849316533 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140a20d0 con 0x556acf486890 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.361+0000 7f6530178b80 1 -- 172.21.15.12:0/3849316533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf486890 msgr2=0x556acf486cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.361+0000 7f6530178b80 1 --2- 172.21.15.12:0/3849316533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf486890 0x556acf486cd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f651400a740 tx=0x7f6514099d50 comp rx=0 tx=0).stop 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.361+0000 7f6530178b80 1 -- 172.21.15.12:0/3849316533 shutdown_connections 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.361+0000 7f6530178b80 1 --2- 172.21.15.12:0/3849316533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf486890 0x556acf486cd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.361+0000 7f6530178b80 1 --2- 172.21.15.12:0/3849316533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48be00 0x556acf48c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.361+0000 7f6530178b80 1 --2- 172.21.15.12:0/3849316533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 0x556acf486350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.361+0000 7f6530178b80 1 -- 172.21.15.12:0/3849316533 >> 172.21.15.12:0/3849316533 conn(0x556acf3e4210 msgr2=0x556acf480df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.361+0000 7f6530178b80 1 -- 172.21.15.12:0/3849316533 shutdown_connections 2024-05-08T23:47:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.361+0000 7f6530178b80 1 -- 172.21.15.12:0/3849316533 wait complete. 2024-05-08T23:47:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.361+0000 7f6530178b80 1 -- start start 2024-05-08T23:47:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f6530178b80 1 Processor -- start 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f6530178b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 0x556acf48fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f6530178b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48be00 0x556acf4900a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f6530178b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48c720 0x556acf4905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f6530178b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf372560 con 0x556acf48c720 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f6530178b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf372200 con 0x556acf486890 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f6530178b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf2596a0 con 0x556acf48be00 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f6530178b80 10 client.0 _ll_register_callbacks cb 0x556acf37bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f652255b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48c720 0x556acf4905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f652255b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48c720 0x556acf4905e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51728/0 (socket says 172.21.15.12:51728) 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f652255b700 1 -- 172.21.15.12:0/1844465109 learned_addr learned my addr 172.21.15.12:0/1844465109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f6521d5a700 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 0x556acf48fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.362+0000 7f6521559700 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48be00 0x556acf4900a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f652255b700 1 -- 172.21.15.12:0/1844465109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48be00 msgr2=0x556acf4900a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f652255b700 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48be00 0x556acf4900a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f652255b700 1 -- 172.21.15.12:0/1844465109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 msgr2=0x556acf48fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f652255b700 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 0x556acf48fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f652255b700 1 -- 172.21.15.12:0/1844465109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c1a6770 con 0x556acf48c720 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f6521d5a700 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 0x556acf48fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f652255b700 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48c720 0x556acf4905e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6514007d40 tx=0x7f65140a2680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef99633ec0237378 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f65057fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f65057fa700 1 -- 172.21.15.12:0/1844465109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140b8470 con 0x556acf48c720 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f65057fa700 1 -- 172.21.15.12:0/1844465109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f651409d030 con 0x556acf48c720 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f6530178b80 1 -- 172.21.15.12:0/1844465109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556acf32f270 con 0x556acf48c720 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f65057fa700 1 -- 172.21.15.12:0/1844465109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65140995d0 con 0x556acf48c720 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f6530178b80 10 client.15537 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f6530178b80 1 -- 172.21.15.12:0/1844465109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556acf203b60 con 0x556acf48c720 2024-05-08T23:47:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f65057fa700 1 -- 172.21.15.12:0/1844465109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556acf32f270 con 0x556acf48c720 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.363+0000 7f65057fa700 1 -- 172.21.15.12:0/1844465109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6514061d20 con 0x556acf48c720 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.364+0000 7f6530178b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.364+0000 7f6530178b80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.364+0000 7f6530178b80 20 client.15537 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.364+0000 7f6530178b80 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556acf49feb0 0x556acf4a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.364+0000 7f6530178b80 4 client.15537 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.364+0000 7f6530178b80 1 -- 172.21.15.12:0/1844465109 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556acf455810 con 0x556acf49feb0 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.364+0000 7f6521d5a700 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556acf49feb0 0x556acf4a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.364+0000 7f65057fa700 10 client.15537 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.364+0000 7f6521d5a700 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556acf49feb0 0x556acf4a22d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe7dfc9408e327f server_cookie=ab8515562e65e9fc in_seq=0 out_seq=0 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.365+0000 7f65057fa700 1 -- 172.21.15.12:0/1844465109 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556acf455810 con 0x556acf49feb0 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.365+0000 7f65057fa700 10 client.15537 handle_command_reply: tid=0 2024-05-08T23:47:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.365+0000 7f6530178b80 1 client.15537 shutdown 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.365+0000 7f6530178b80 1 -- 172.21.15.12:0/1844465109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48c720 msgr2=0x556acf4905e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.365+0000 7f6530178b80 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48c720 0x556acf4905e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6514007d40 tx=0x7f65140a2680 comp rx=0 tx=0).stop 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.366+0000 7f6530178b80 1 -- 172.21.15.12:0/1844465109 shutdown_connections 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.366+0000 7f6530178b80 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48c720 0x556acf4905e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.366+0000 7f6530178b80 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556acf49feb0 0x556acf4a22d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.366+0000 7f6530178b80 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 0x556acf48fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.366+0000 7f65057fa700 0 client.15537 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.366+0000 7f6530178b80 1 --2- 172.21.15.12:0/1844465109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48be00 0x556acf4900a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.366+0000 7f6530178b80 1 -- 172.21.15.12:0/1844465109 >> 172.21.15.12:0/1844465109 conn(0x556acf3e4210 msgr2=0x556acf4805a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.366+0000 7f6530178b80 1 -- 172.21.15.12:0/1844465109 shutdown_connections 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.366+0000 7f6530178b80 1 -- 172.21.15.12:0/1844465109 wait complete. 2024-05-08T23:47:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.366+0000 7f6530178b80 20 client.15537 trim_cache size 0 max 16384 2024-05-08T23:47:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.384+0000 7f6530178b80 1 Processor -- start 2024-05-08T23:47:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.384+0000 7f6530178b80 1 -- start start 2024-05-08T23:47:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6530178b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf486890 0x556acf486350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6530178b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf480d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6530178b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 0x556acf490400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6530178b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf491180 con 0x556acf48be00 2024-05-08T23:47:04.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6530178b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf47ddc0 con 0x556acf48c720 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6530178b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf482e50 con 0x556acf486890 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6521559700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf480d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6521559700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf480d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51742/0 (socket says 172.21.15.12:51742) 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6521559700 1 -- 172.21.15.12:0/971159624 learned_addr learned my addr 172.21.15.12:0/971159624 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f652255b700 1 --2- 172.21.15.12:0/971159624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 0x556acf490400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6521d5a700 1 --2- 172.21.15.12:0/971159624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf486890 0x556acf486350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6521559700 1 -- 172.21.15.12:0/971159624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf486890 msgr2=0x556acf486350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6521559700 1 --2- 172.21.15.12:0/971159624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf486890 0x556acf486350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6521559700 1 -- 172.21.15.12:0/971159624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 msgr2=0x556acf490400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6521559700 1 --2- 172.21.15.12:0/971159624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 0x556acf490400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6521559700 1 -- 172.21.15.12:0/971159624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556acf39c560 con 0x556acf48be00 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.385+0000 7f6521559700 1 --2- 172.21.15.12:0/971159624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf480d20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f651c06a5d0 tx=0x7f650c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f559ac67d2482f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6510ff9700 1 -- 172.21.15.12:0/971159624 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c015540 con 0x556acf48be00 2024-05-08T23:47:04.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6510ff9700 1 -- 172.21.15.12:0/971159624 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f650c015b60 con 0x556acf48be00 2024-05-08T23:47:04.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6510ff9700 1 -- 172.21.15.12:0/971159624 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c016ce0 con 0x556acf48be00 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6530178b80 1 -- 172.21.15.12:0/971159624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 msgr2=0x556acf480d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6530178b80 1 --2- 172.21.15.12:0/971159624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf480d20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f651c06a5d0 tx=0x7f650c009190 comp rx=0 tx=0).stop 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6530178b80 1 -- 172.21.15.12:0/971159624 shutdown_connections 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6530178b80 1 --2- 172.21.15.12:0/971159624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf48c720 0x556acf490400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6530178b80 1 --2- 172.21.15.12:0/971159624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf486890 0x556acf486350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6530178b80 1 --2- 172.21.15.12:0/971159624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf480d20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6530178b80 1 -- 172.21.15.12:0/971159624 >> 172.21.15.12:0/971159624 conn(0x556acf3e4210 msgr2=0x556acf48e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6530178b80 1 -- 172.21.15.12:0/971159624 shutdown_connections 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.386+0000 7f6530178b80 1 -- 172.21.15.12:0/971159624 wait complete. 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.387+0000 7f6530178b80 1 -- start start 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.387+0000 7f6530178b80 1 Processor -- start 2024-05-08T23:47:04.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.387+0000 7f6530178b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 0x556acf493330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.387+0000 7f6530178b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf493870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.387+0000 7f6530178b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48c720 0x556acf4f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.387+0000 7f6530178b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf493fe0 con 0x556acf48be00 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.387+0000 7f6530178b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf4f58e0 con 0x556acf486890 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.387+0000 7f6530178b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf4f5aa0 con 0x556acf48c720 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.387+0000 7f6530178b80 10 client.0 _ll_register_callbacks cb 0x556acf37bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.387+0000 7f6521559700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf493870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6521559700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf493870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51744/0 (socket says 172.21.15.12:51744) 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6521559700 1 -- 172.21.15.12:0/84340504 learned_addr learned my addr 172.21.15.12:0/84340504 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f652255b700 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48c720 0x556acf4f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6521d5a700 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 0x556acf493330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6521559700 1 -- 172.21.15.12:0/84340504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48c720 msgr2=0x556acf4f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6521559700 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48c720 0x556acf4f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6521559700 1 -- 172.21.15.12:0/84340504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 msgr2=0x556acf493330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6521559700 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 0x556acf493330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6521559700 1 -- 172.21.15.12:0/84340504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650c009550 con 0x556acf48be00 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6521559700 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf493870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f650c008010 tx=0x7f650c0164d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544e90c492cc886 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f65057fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f65057fa700 1 -- 172.21.15.12:0/84340504 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c005c60 con 0x556acf48be00 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f65057fa700 1 -- 172.21.15.12:0/84340504 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f650c005dc0 con 0x556acf48be00 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6530178b80 1 -- 172.21.15.12:0/84340504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f651409df10 con 0x556acf48be00 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f65057fa700 1 -- 172.21.15.12:0/84340504 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f650c015540 con 0x556acf48be00 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.388+0000 7f6530178b80 10 client.15543 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.389+0000 7f6530178b80 1 -- 172.21.15.12:0/84340504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556acf25bcd0 con 0x556acf48be00 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.389+0000 7f65057fa700 1 -- 172.21.15.12:0/84340504 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f651409df10 con 0x556acf48be00 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.389+0000 7f65057fa700 1 -- 172.21.15.12:0/84340504 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f650c0062a0 con 0x556acf48be00 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.389+0000 7f6530178b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:04.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.389+0000 7f6530178b80 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.389+0000 7f6530178b80 20 client.15543 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.389+0000 7f6530178b80 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556acf4f7220 0x556acf4f9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.389+0000 7f6530178b80 4 client.15543 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.389+0000 7f6530178b80 1 -- 172.21.15.12:0/84340504 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556acf3ceec0 con 0x556acf4f7220 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.389+0000 7f6521d5a700 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556acf4f7220 0x556acf4f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6521d5a700 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556acf4f7220 0x556acf4f9640 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecaa0d376df0876 server_cookie=d317812a0be73d28 in_seq=0 out_seq=0 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f65057fa700 10 client.15543 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f65057fa700 1 -- 172.21.15.12:0/84340504 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556acf3ceec0 con 0x556acf4f7220 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f65057fa700 10 client.15543 handle_command_reply: tid=0 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 client.15543 shutdown 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 -- 172.21.15.12:0/84340504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 msgr2=0x556acf493870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf493870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f650c008010 tx=0x7f650c0164d0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 -- 172.21.15.12:0/84340504 shutdown_connections 2024-05-08T23:47:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556acf48c720 0x556acf4f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556acf4f7220 0x556acf4f9640 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556acf486890 0x556acf493330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 --2- 172.21.15.12:0/84340504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556acf48be00 0x556acf493870 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f65057fa700 0 client.15543 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 -- 172.21.15.12:0/84340504 >> 172.21.15.12:0/84340504 conn(0x556acf3e4210 msgr2=0x556acf48ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 -- 172.21.15.12:0/84340504 shutdown_connections 2024-05-08T23:47:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 1 -- 172.21.15.12:0/84340504 wait complete. 2024-05-08T23:47:04.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.390+0000 7f6530178b80 20 client.15543 trim_cache size 0 max 16384 2024-05-08T23:47:04.391 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:47:04.392 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:47:04.392 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:47:04.392 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:47:04.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 -- 172.21.15.12:0/975437309 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f65080a13b0 msgr2=0x7f65080a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f65080a13b0 0x7f65080a3870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6518006010 tx=0x7f651800b540 comp rx=0 tx=0).stop 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 -- 172.21.15.12:0/975437309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f651c073d20 msgr2=0x7f651c1a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f651c073d20 0x7f651c1a5680 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6514007d70 tx=0x7f651400a340 comp rx=0 tx=0).stop 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 -- 172.21.15.12:0/975437309 shutdown_connections 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f65080a13b0 0x7f65080a3870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f651c073d20 0x7f651c1a5680 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f651c101180 0x7f651c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 --2- 172.21.15.12:0/975437309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c1062a0 0x7f651c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 -- 172.21.15.12:0/975437309 >> 172.21.15.12:0/975437309 conn(0x7f651c0fc9e0 msgr2=0x7f651c0fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 -- 172.21.15.12:0/975437309 shutdown_connections 2024-05-08T23:47:04.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.391+0000 7f6523fbe700 1 -- 172.21.15.12:0/975437309 wait complete. 2024-05-08T23:47:04.400 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:47:04.400 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:47:04.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-mon[47373]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-05-08T23:47:04.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: *** Caught signal (Segmentation fault) ** 2024-05-08T23:47:04.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: in thread 7f5ab2040700 thread_name:md_log_replay 2024-05-08T23:47:04.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T23:47:04.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5ac2c01d20] 2024-05-08T23:47:04.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2: /usr/bin/ceph-mds(+0x4be5a2) [0x556a8242a5a2] 2024-05-08T23:47:04.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556a8242c7f3] 2024-05-08T23:47:04.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 4: (MDLog::_replay_thread()+0x699) [0x556a8248a079] 2024-05-08T23:47:04.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556a821415f1] 2024-05-08T23:47:04.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5ac2bf71ca] 2024-05-08T23:47:04.514 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 7: clone() 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2024-05-08T23:47:04.248+0000 7f5ab2040700 -1 *** Caught signal (Segmentation fault) ** 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: in thread 7f5ab2040700 thread_name:md_log_replay 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5ac2c01d20] 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2: /usr/bin/ceph-mds(+0x4be5a2) [0x556a8242a5a2] 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556a8242c7f3] 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 4: (MDLog::_replay_thread()+0x699) [0x556a8248a079] 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556a821415f1] 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5ac2bf71ca] 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 7: clone() 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2024-05-08T23:47:04.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 0> 2024-05-08T23:47:04.248+0000 7f5ab2040700 -1 *** Caught signal (Segmentation fault) ** 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: in thread 7f5ab2040700 thread_name:md_log_replay 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5ac2c01d20] 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2: /usr/bin/ceph-mds(+0x4be5a2) [0x556a8242a5a2] 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556a8242c7f3] 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 4: (MDLog::_replay_thread()+0x699) [0x556a8248a079] 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556a821415f1] 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5ac2bf71ca] 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 7: clone() 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 0> 2024-05-08T23:47:04.248+0000 7f5ab2040700 -1 *** Caught signal (Segmentation fault) ** 2024-05-08T23:47:04.516 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: in thread 7f5ab2040700 thread_name:md_log_replay 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5ac2c01d20] 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2: /usr/bin/ceph-mds(+0x4be5a2) [0x556a8242a5a2] 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556a8242c7f3] 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 4: (MDLog::_replay_thread()+0x699) [0x556a8248a079] 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556a821415f1] 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5ac2bf71ca] 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 7: clone() 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-08T23:47:04.517 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[84612]: 2024-05-08T23:47:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.542+0000 7f9750e77700 1 -- 172.21.15.12:0/4192355785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f974c100900 msgr2=0x7f974c104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.542+0000 7f9750e77700 1 --2- 172.21.15.12:0/4192355785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f974c100900 0x7f974c104940 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9740009990 tx=0x7f9740009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.542+0000 7f9750e77700 1 -- 172.21.15.12:0/4192355785 shutdown_connections 2024-05-08T23:47:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.542+0000 7f9750e77700 1 --2- 172.21.15.12:0/4192355785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974c105070 0x7f974c10cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.542+0000 7f9750e77700 1 --2- 172.21.15.12:0/4192355785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f974c0fff50 0x7f974c100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.542+0000 7f9750e77700 1 --2- 172.21.15.12:0/4192355785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f974c100900 0x7f974c104940 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.542+0000 7f9750e77700 1 -- 172.21.15.12:0/4192355785 >> 172.21.15.12:0/4192355785 conn(0x7f974c0fb7f0 msgr2=0x7f974c0fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.542+0000 7f9750e77700 1 -- 172.21.15.12:0/4192355785 shutdown_connections 2024-05-08T23:47:04.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.542+0000 7f9750e77700 1 -- 172.21.15.12:0/4192355785 wait complete. 2024-05-08T23:47:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f9750e77700 1 Processor -- start 2024-05-08T23:47:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f9750e77700 1 -- start start 2024-05-08T23:47:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f9750e77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f974c0fff50 0x7f974c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f9750e77700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f974c105070 0x7f974c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f9750e77700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974c19bbe0 0x7f974c19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f9750e77700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c19c660 con 0x7f974c0fff50 2024-05-08T23:47:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f9750e77700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c19c7d0 con 0x7f974c105070 2024-05-08T23:47:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f9750e77700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c19c990 con 0x7f974c19bbe0 2024-05-08T23:47:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f974a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f974c0fff50 0x7f974c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f974a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f974c0fff50 0x7f974c197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51766/0 (socket says 172.21.15.12:51766) 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f974a59c700 1 -- 172.21.15.12:0/2892782063 learned_addr learned my addr 172.21.15.12:0/2892782063 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f9749d9b700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f974c105070 0x7f974c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f974ad9d700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974c19bbe0 0x7f974c19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f974a59c700 1 -- 172.21.15.12:0/2892782063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974c19bbe0 msgr2=0x7f974c19c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f974a59c700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974c19bbe0 0x7f974c19c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f974a59c700 1 -- 172.21.15.12:0/2892782063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f974c105070 msgr2=0x7f974c197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f974a59c700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f974c105070 0x7f974c197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.543+0000 7f974a59c700 1 -- 172.21.15.12:0/2892782063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97400095f0 con 0x7f974c0fff50 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.544+0000 7f974a59c700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f974c0fff50 0x7f974c197350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f973c00d950 tx=0x7f973c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.544+0000 7f973b7fe700 1 -- 172.21.15.12:0/2892782063 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c009940 con 0x7f974c0fff50 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.544+0000 7f973b7fe700 1 -- 172.21.15.12:0/2892782063 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f973c010460 con 0x7f974c0fff50 2024-05-08T23:47:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.544+0000 7f9750e77700 1 -- 172.21.15.12:0/2892782063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c1a3ef0 con 0x7f974c0fff50 2024-05-08T23:47:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.544+0000 7f973b7fe700 1 -- 172.21.15.12:0/2892782063 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c00f620 con 0x7f974c0fff50 2024-05-08T23:47:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.544+0000 7f9750e77700 1 -- 172.21.15.12:0/2892782063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c1a4370 con 0x7f974c0fff50 2024-05-08T23:47:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.545+0000 7f9750e77700 1 -- 172.21.15.12:0/2892782063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c108250 con 0x7f974c0fff50 2024-05-08T23:47:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.546+0000 7f973b7fe700 1 -- 172.21.15.12:0/2892782063 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f973c0108d0 con 0x7f974c0fff50 2024-05-08T23:47:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.547+0000 7f973b7fe700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97340a13b0 0x7f97340a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.547+0000 7f9749d9b700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97340a13b0 0x7f97340a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.547+0000 7f973b7fe700 1 -- 172.21.15.12:0/2892782063 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f973c0cce10 con 0x7f974c0fff50 2024-05-08T23:47:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.548+0000 7f9749d9b700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97340a13b0 0x7f97340a3870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f974c198850 tx=0x7f9740005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.553+0000 7f973b7fe700 1 -- 172.21.15.12:0/2892782063 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f973c0d0030 con 0x7f974c0fff50 2024-05-08T23:47:04.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:04 smithi012 ceph-mon[42503]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-05-08T23:47:04.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:04 smithi160 ceph-mon[46152]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-05-08T23:47:04.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.819+0000 7f9750e77700 1 -- 172.21.15.12:0/2892782063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f974c101420 con 0x7f974c0fff50 2024-05-08T23:47:04.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.820+0000 7f973b7fe700 1 -- 172.21.15.12:0/2892782063 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10876 (secure 0 0 0) 0x7f973c016090 con 0x7f974c0fff50 2024-05-08T23:47:04.816 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:47:04.816 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:46:26.294680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2858273239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2858273239},{"type":"v1","addr":"172.21.15.204:6837","nonce":2858273239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:47:04.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 -- 172.21.15.12:0/2892782063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97340a13b0 msgr2=0x7f97340a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97340a13b0 0x7f97340a3870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f974c198850 tx=0x7f9740005fd0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 -- 172.21.15.12:0/2892782063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f974c0fff50 msgr2=0x7f974c197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f974c0fff50 0x7f974c197350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f973c00d950 tx=0x7f973c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:47:04.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 -- 172.21.15.12:0/2892782063 shutdown_connections 2024-05-08T23:47:04.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97340a13b0 0x7f97340a3870 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974c19bbe0 0x7f974c19c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f974c0fff50 0x7f974c197350 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 --2- 172.21.15.12:0/2892782063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f974c105070 0x7f974c197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 -- 172.21.15.12:0/2892782063 >> 172.21.15.12:0/2892782063 conn(0x7f974c0fb7f0 msgr2=0x7f974c103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 -- 172.21.15.12:0/2892782063 shutdown_connections 2024-05-08T23:47:04.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.822+0000 7f9750e77700 1 -- 172.21.15.12:0/2892782063 wait complete. 2024-05-08T23:47:04.819 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 24 2024-05-08T23:47:04.833 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.977+0000 7fc7a98fb700 1 -- 172.21.15.12:0/2675369344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a4100a10 msgr2=0x7fc7a4104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.977+0000 7fc7a98fb700 1 --2- 172.21.15.12:0/2675369344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a4100a10 0x7fc7a4104a50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009990 tx=0x7fc78c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.977+0000 7fc7a98fb700 1 -- 172.21.15.12:0/2675369344 shutdown_connections 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.977+0000 7fc7a98fb700 1 --2- 172.21.15.12:0/2675369344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a4105180 0x7fc7a410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.977+0000 7fc7a98fb700 1 --2- 172.21.15.12:0/2675369344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7a4100060 0x7fc7a4100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.977+0000 7fc7a98fb700 1 --2- 172.21.15.12:0/2675369344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a4100a10 0x7fc7a4104a50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.977+0000 7fc7a98fb700 1 -- 172.21.15.12:0/2675369344 >> 172.21.15.12:0/2675369344 conn(0x7fc7a4076fa0 msgr2=0x7fc7a40773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.977+0000 7fc7a98fb700 1 -- 172.21.15.12:0/2675369344 shutdown_connections 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.977+0000 7fc7a98fb700 1 -- 172.21.15.12:0/2675369344 wait complete. 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.978+0000 7fc7a98fb700 1 Processor -- start 2024-05-08T23:47:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.978+0000 7fc7a98fb700 1 -- start start 2024-05-08T23:47:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a98fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a4100060 0x7fc7a41962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a98fb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7a4105180 0x7fc7a41967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a98fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a419ab90 0x7fc7a419b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a98fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a419b610 con 0x7fc7a4100060 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a98fb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a419b780 con 0x7fc7a419ab90 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a98fb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a419b940 con 0x7fc7a4105180 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a4100060 0x7fc7a41962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a4100060 0x7fc7a41962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51782/0 (socket says 172.21.15.12:51782) 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/1878183684 learned_addr learned my addr 172.21.15.12:0/1878183684 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a37fe700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a419ab90 0x7fc7a419b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc79a7fc700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7a4105180 0x7fc7a41967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/1878183684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7a4105180 msgr2=0x7fc7a41967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7a4105180 0x7fc7a41967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/1878183684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a419ab90 msgr2=0x7fc7a419b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a419ab90 0x7fc7a419b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/1878183684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c0095f0 con 0x7fc7a4100060 2024-05-08T23:47:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.979+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a4100060 0x7fc7a41962b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc794009970 tx=0x7fc794009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.980+0000 7fc7a88f9700 1 -- 172.21.15.12:0/1878183684 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794012070 con 0x7fc7a4100060 2024-05-08T23:47:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.980+0000 7fc7a98fb700 1 -- 172.21.15.12:0/1878183684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a4069510 con 0x7fc7a4100060 2024-05-08T23:47:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.980+0000 7fc7a88f9700 1 -- 172.21.15.12:0/1878183684 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc79400d260 con 0x7fc7a4100060 2024-05-08T23:47:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.980+0000 7fc7a88f9700 1 -- 172.21.15.12:0/1878183684 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7940179d0 con 0x7fc7a4100060 2024-05-08T23:47:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.980+0000 7fc7a98fb700 1 -- 172.21.15.12:0/1878183684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a4069a20 con 0x7fc7a4100060 2024-05-08T23:47:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.982+0000 7fc7a88f9700 1 -- 172.21.15.12:0/1878183684 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fc794007c00 con 0x7fc7a4100060 2024-05-08T23:47:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.982+0000 7fc7a88f9700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7900a13b0 0x7fc7900a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.983+0000 7fc7a88f9700 1 -- 172.21.15.12:0/1878183684 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc7940ce2d0 con 0x7fc7a4100060 2024-05-08T23:47:04.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.983+0000 7fc79a7fc700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7900a13b0 0x7fc7900a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.983+0000 7fc79a7fc700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7900a13b0 0x7fc7900a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00b2e0 tx=0x7fc78c009510 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.983+0000 7fc7b5ab5b80 1 Processor -- start 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7b5ab5b80 1 -- start start 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 0x55cf9f39c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f39cca0 0x55cf9f39d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f2ef150 con 0x55cf9f3a21f0 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f2ecb10 con 0x55cf9f39cca0 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f2e8400 con 0x55cf9f3a2b10 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51792/0 (socket says 172.21.15.12:51792) 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/800098753 learned_addr learned my addr 172.21.15.12:0/800098753 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7a37fe700 1 --2- 172.21.15.12:0/800098753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f39cca0 0x55cf9f39d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc79a7fc700 1 --2- 172.21.15.12:0/800098753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 0x55cf9f39c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/800098753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 msgr2=0x55cf9f39c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/800098753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 0x55cf9f39c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/800098753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f39cca0 msgr2=0x55cf9f39d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/800098753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f39cca0 0x55cf9f39d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/800098753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf9f1693b0 con 0x55cf9f3a21f0 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.984+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/800098753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a25d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc79409aed0 tx=0x7fc79409ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fca9cda1ed5eb65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc799ffb700 1 -- 172.21.15.12:0/800098753 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79409b8d0 con 0x55cf9f3a21f0 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc799ffb700 1 -- 172.21.15.12:0/800098753 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7940b1020 con 0x55cf9f3a21f0 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc799ffb700 1 -- 172.21.15.12:0/800098753 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7940a31c0 con 0x55cf9f3a21f0 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/800098753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 msgr2=0x55cf9f3a25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/800098753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a25d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc79409aed0 tx=0x7fc79409ad90 comp rx=0 tx=0).stop 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/800098753 shutdown_connections 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/800098753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f39cca0 0x55cf9f39d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/800098753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a25d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/800098753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 0x55cf9f39c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/800098753 >> 172.21.15.12:0/800098753 conn(0x55cf9f2f6210 msgr2=0x55cf9f392df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/800098753 shutdown_connections 2024-05-08T23:47:04.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.985+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/800098753 wait complete. 2024-05-08T23:47:04.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.986+0000 7fc7b5ab5b80 1 -- start start 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.986+0000 7fc7b5ab5b80 1 Processor -- start 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f39cca0 0x55cf9f3a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f284560 con 0x55cf9f3a2b10 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f284200 con 0x55cf9f3a21f0 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f16b6a0 con 0x55cf9f39cca0 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7b5ab5b80 10 client.0 _ll_register_callbacks cb 0x55cf9f28db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7a37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7a37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51798/0 (socket says 172.21.15.12:51798) 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7a37fe700 1 -- 172.21.15.12:0/3595928508 learned_addr learned my addr 172.21.15.12:0/3595928508 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:04.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f39cca0 0x55cf9f3a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc79a7fc700 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7a37fe700 1 -- 172.21.15.12:0/3595928508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f39cca0 msgr2=0x55cf9f3a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7a37fe700 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f39cca0 0x55cf9f3a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7a37fe700 1 -- 172.21.15.12:0/3595928508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f3a21f0 msgr2=0x55cf9f3a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7a37fe700 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7a37fe700 1 -- 172.21.15.12:0/3595928508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a4069a20 con 0x55cf9f3a2b10 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc7a37fe700 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a6940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc79c012e80 tx=0x7fc79c014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876cbf1d4b3d6bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.987+0000 7fc782ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc782ffd700 1 -- 172.21.15.12:0/3595928508 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00e040 con 0x55cf9f3a2b10 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc782ffd700 1 -- 172.21.15.12:0/3595928508 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc79c014b90 con 0x55cf9f3a2b10 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3595928508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf9f241270 con 0x55cf9f3a2b10 2024-05-08T23:47:04.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc782ffd700 1 -- 172.21.15.12:0/3595928508 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc79c014d90 con 0x55cf9f3a2b10 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc7b5ab5b80 10 client.15561 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3595928508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cf9f115b60 con 0x55cf9f3a2b10 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc782ffd700 1 -- 172.21.15.12:0/3595928508 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf9f241270 con 0x55cf9f3a2b10 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc782ffd700 1 -- 172.21.15.12:0/3595928508 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc79c01c020 con 0x55cf9f3a2b10 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc7b5ab5b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc7b5ab5b80 10 client.15561 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc7b5ab5b80 20 client.15561 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55cf9f3b65f0 0x55cf9f3b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc7b5ab5b80 4 client.15561 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.988+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3595928508 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf9f367810 con 0x55cf9f3b65f0 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.989+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55cf9f3b65f0 0x55cf9f3b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.989+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55cf9f3b65f0 0x55cf9f3b8a10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcfa334d6896f00f server_cookie=d58a77e495640114 in_seq=0 out_seq=0 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.989+0000 7fc782ffd700 10 client.15561 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:47:04.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.989+0000 7fc782ffd700 1 -- 172.21.15.12:0/3595928508 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf9f367810 con 0x55cf9f3b65f0 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.989+0000 7fc782ffd700 10 client.15561 handle_command_reply: tid=0 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 client.15561 shutdown 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3595928508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a2b10 msgr2=0x55cf9f3a6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a6940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc79c012e80 tx=0x7fc79c014600 comp rx=0 tx=0).stop 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3595928508 shutdown_connections 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a6940 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55cf9f3b65f0 0x55cf9f3b8a10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f39cca0 0x55cf9f3a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3595928508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc782ffd700 0 client.15561 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3595928508 >> 172.21.15.12:0/3595928508 conn(0x55cf9f2f6210 msgr2=0x55cf9f3925a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3595928508 shutdown_connections 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3595928508 wait complete. 2024-05-08T23:47:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:04.990+0000 7fc7b5ab5b80 20 client.15561 trim_cache size 0 max 16384 2024-05-08T23:47:05.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7b5ab5b80 1 Processor -- start 2024-05-08T23:47:05.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7b5ab5b80 1 -- start start 2024-05-08T23:47:05.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f39cca0 0x55cf9f3b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a21f0 0x55cf9f392d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3ad550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f3a5cb0 con 0x55cf9f39cca0 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f394a40 con 0x55cf9f3a2b10 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f3bdc00 con 0x55cf9f3a21f0 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f39cca0 0x55cf9f3b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f39cca0 0x55cf9f3b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51806/0 (socket says 172.21.15.12:51806) 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/1160761595 learned_addr learned my addr 172.21.15.12:0/1160761595 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.004+0000 7fc79a7fc700 1 --2- 172.21.15.12:0/1160761595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a21f0 0x55cf9f392d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.005+0000 7fc7a37fe700 1 --2- 172.21.15.12:0/1160761595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.005+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/1160761595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a21f0 msgr2=0x55cf9f392d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.005+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/1160761595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a21f0 0x55cf9f392d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.005+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/1160761595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f3a2b10 msgr2=0x55cf9f3ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.005+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/1160761595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.005+0000 7fc7a2ffd700 1 -- 172.21.15.12:0/1160761595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf9f2ae560 con 0x55cf9f39cca0 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.005+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/1160761595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f39cca0 0x55cf9f3b26b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc7940074a0 tx=0x7fc79400f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7dea1b75321e5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.005+0000 7fc79affd700 1 -- 172.21.15.12:0/1160761595 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79409b8d0 con 0x55cf9f39cca0 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.005+0000 7fc79affd700 1 -- 172.21.15.12:0/1160761595 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc79409f080 con 0x55cf9f39cca0 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.005+0000 7fc79affd700 1 -- 172.21.15.12:0/1160761595 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7940a3420 con 0x55cf9f39cca0 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/1160761595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f39cca0 msgr2=0x55cf9f3b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/1160761595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f39cca0 0x55cf9f3b26b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc7940074a0 tx=0x7fc79400f040 comp rx=0 tx=0).stop 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/1160761595 shutdown_connections 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/1160761595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/1160761595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f39cca0 0x55cf9f3b26b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/1160761595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a21f0 0x55cf9f392d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/1160761595 >> 172.21.15.12:0/1160761595 conn(0x55cf9f2f6210 msgr2=0x55cf9f39d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/1160761595 shutdown_connections 2024-05-08T23:47:05.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/1160761595 wait complete. 2024-05-08T23:47:05.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 -- start start 2024-05-08T23:47:05.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.006+0000 7fc7b5ab5b80 1 Processor -- start 2024-05-08T23:47:05.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f39cca0 0x55cf9f3a9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7b5ab5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f3ade40 con 0x55cf9f3a21f0 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f40bae0 con 0x55cf9f39cca0 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7b5ab5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9f40bc70 con 0x55cf9f3a2b10 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7b5ab5b80 10 client.0 _ll_register_callbacks cb 0x55cf9f28db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc79a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc79a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51816/0 (socket says 172.21.15.12:51816) 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc79a7fc700 1 -- 172.21.15.12:0/3061329099 learned_addr learned my addr 172.21.15.12:0/3061329099 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7a37fe700 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f39cca0 0x55cf9f3a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7a37fe700 1 -- 172.21.15.12:0/3061329099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f39cca0 msgr2=0x55cf9f3a9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7a37fe700 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f39cca0 0x55cf9f3a9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7a37fe700 1 -- 172.21.15.12:0/3061329099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 msgr2=0x55cf9f3a9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7a37fe700 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc7a37fe700 1 -- 172.21.15.12:0/3061329099 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf9f1693b0 con 0x55cf9f3a2b10 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.007+0000 7fc79a7fc700 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc7a37fe700 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a9fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc79c012e50 tx=0x7fc79c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af9faa44784b06a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc782ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc782ffd700 1 -- 172.21.15.12:0/3061329099 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00e070 con 0x55cf9f3a2b10 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc782ffd700 1 -- 172.21.15.12:0/3061329099 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc79c0049d0 con 0x55cf9f3a2b10 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3061329099 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7940b1020 con 0x55cf9f3a2b10 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc782ffd700 1 -- 172.21.15.12:0/3061329099 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc79c004bb0 con 0x55cf9f3a2b10 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc7b5ab5b80 10 client.25414 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3061329099 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cf9f16dcd0 con 0x55cf9f3a2b10 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc782ffd700 1 -- 172.21.15.12:0/3061329099 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7940b1020 con 0x55cf9f3a2b10 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc782ffd700 1 -- 172.21.15.12:0/3061329099 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc79c02d020 con 0x55cf9f3a2b10 2024-05-08T23:47:05.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc7b5ab5b80 10 client.25414 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc7b5ab5b80 10 client.25414 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc7b5ab5b80 20 client.25414 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55cf9f40d3e0 0x55cf9f40f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc7b5ab5b80 4 client.25414 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.008+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3061329099 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cf9f2e0ec0 con 0x55cf9f40d3e0 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55cf9f40d3e0 0x55cf9f40f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc782ffd700 10 client.25414 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7a2ffd700 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55cf9f40d3e0 0x55cf9f40f800 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16d7dd6510b10747 server_cookie=74354d2ddde898be in_seq=0 out_seq=0 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc782ffd700 1 -- 172.21.15.12:0/3061329099 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cf9f2e0ec0 con 0x55cf9f40d3e0 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc782ffd700 10 client.25414 handle_command_reply: tid=0 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7b5ab5b80 1 client.25414 shutdown 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3061329099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 msgr2=0x55cf9f3a9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a9fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc79c012e50 tx=0x7fc79c009b20 comp rx=0 tx=0).stop 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3061329099 shutdown_connections 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55cf9f40d3e0 0x55cf9f40f800 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf9f3a2b10 0x55cf9f3a9fb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf9f39cca0 0x55cf9f3a9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7b5ab5b80 1 --2- 172.21.15.12:0/3061329099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf9f3a21f0 0x55cf9f3a9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc782ffd700 0 client.25414 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3061329099 >> 172.21.15.12:0/3061329099 conn(0x55cf9f2f6210 msgr2=0x55cf9f39fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.009+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3061329099 shutdown_connections 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7b5ab5b80 1 -- 172.21.15.12:0/3061329099 wait complete. 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7b5ab5b80 20 client.25414 trim_cache size 0 max 16384 2024-05-08T23:47:05.011 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:47:05.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 -- 172.21.15.12:0/1878183684 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7900a13b0 msgr2=0x7fc7900a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7900a13b0 0x7fc7900a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00b2e0 tx=0x7fc78c009510 comp rx=0 tx=0).stop 2024-05-08T23:47:05.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 -- 172.21.15.12:0/1878183684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a4100060 msgr2=0x7fc7a41962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a4100060 0x7fc7a41962b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc794009970 tx=0x7fc794009c80 comp rx=0 tx=0).stop 2024-05-08T23:47:05.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 -- 172.21.15.12:0/1878183684 shutdown_connections 2024-05-08T23:47:05.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7900a13b0 0x7fc7900a3870 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a419ab90 0x7fc7a419b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a4100060 0x7fc7a41962b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 --2- 172.21.15.12:0/1878183684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7a4105180 0x7fc7a41967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 -- 172.21.15.12:0/1878183684 >> 172.21.15.12:0/1878183684 conn(0x7fc7a4076fa0 msgr2=0x7fc7a40fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 -- 172.21.15.12:0/1878183684 shutdown_connections 2024-05-08T23:47:05.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.010+0000 7fc7a98fb700 1 -- 172.21.15.12:0/1878183684 wait complete. 2024-05-08T23:47:05.013 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 systemd[1]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-05-08T23:47:05.014 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:04 smithi204 systemd[1]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-08T23:47:05.019 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.162+0000 7f748a545700 1 -- 172.21.15.12:0/1890771771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7484101b10 msgr2=0x7f7484105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.162+0000 7f748a545700 1 --2- 172.21.15.12:0/1890771771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7484101b10 0x7f7484105b50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7478009990 tx=0x7f7478009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.162+0000 7f748a545700 1 -- 172.21.15.12:0/1890771771 shutdown_connections 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.162+0000 7f748a545700 1 --2- 172.21.15.12:0/1890771771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7484106280 0x7f748410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.162+0000 7f748a545700 1 --2- 172.21.15.12:0/1890771771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7484101160 0x7f7484101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.162+0000 7f748a545700 1 --2- 172.21.15.12:0/1890771771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7484101b10 0x7f7484105b50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.162+0000 7f748a545700 1 -- 172.21.15.12:0/1890771771 >> 172.21.15.12:0/1890771771 conn(0x7f74840fc9e0 msgr2=0x7f74840fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.163+0000 7f748a545700 1 -- 172.21.15.12:0/1890771771 shutdown_connections 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.163+0000 7f748a545700 1 -- 172.21.15.12:0/1890771771 wait complete. 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.163+0000 7f748a545700 1 Processor -- start 2024-05-08T23:47:05.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.163+0000 7f748a545700 1 -- start start 2024-05-08T23:47:05.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f748a545700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7484101160 0x7f74840732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f748a545700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7484106280 0x7f74840737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f748a545700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7484073d20 0x7f74841a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f748a545700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841a5b70 con 0x7f7484073d20 2024-05-08T23:47:05.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f748a545700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841a5ce0 con 0x7f7484106280 2024-05-08T23:47:05.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f748a545700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841a5ea0 con 0x7f7484101160 2024-05-08T23:47:05.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f7488ae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7484073d20 0x7f74841a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f7488ae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7484073d20 0x7f74841a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51838/0 (socket says 172.21.15.12:51838) 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f7488ae2700 1 -- 172.21.15.12:0/3037149494 learned_addr learned my addr 172.21.15.12:0/3037149494 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f7483fff700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7484101160 0x7f74840732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f74837fe700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7484106280 0x7f74840737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f7488ae2700 1 -- 172.21.15.12:0/3037149494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7484101160 msgr2=0x7f74840732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f7488ae2700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7484101160 0x7f74840732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f7488ae2700 1 -- 172.21.15.12:0/3037149494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7484106280 msgr2=0x7f74840737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f7488ae2700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7484106280 0x7f74840737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f7488ae2700 1 -- 172.21.15.12:0/3037149494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74780095f0 con 0x7f7484073d20 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f7488ae2700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7484073d20 0x7f74841a55a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f747c007ee0 tx=0x7f747c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f74817fa700 1 -- 172.21.15.12:0/3037149494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c017070 con 0x7f7484073d20 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.164+0000 7f748a545700 1 -- 172.21.15.12:0/3037149494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74841a6180 con 0x7f7484073d20 2024-05-08T23:47:05.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.165+0000 7f74817fa700 1 -- 172.21.15.12:0/3037149494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f747c00f040 con 0x7f7484073d20 2024-05-08T23:47:05.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.165+0000 7f74817fa700 1 -- 172.21.15.12:0/3037149494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c005740 con 0x7f7484073d20 2024-05-08T23:47:05.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.165+0000 7f748a545700 1 -- 172.21.15.12:0/3037149494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74841a6690 con 0x7f7484073d20 2024-05-08T23:47:05.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.167+0000 7f74817fa700 1 -- 172.21.15.12:0/3037149494 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f747c0058a0 con 0x7f7484073d20 2024-05-08T23:47:05.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.168+0000 7f74817fa700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f746c0a13b0 0x7f746c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.168+0000 7f74817fa700 1 -- 172.21.15.12:0/3037149494 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f747c0cc7a0 con 0x7f7484073d20 2024-05-08T23:47:05.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.168+0000 7f7483fff700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f746c0a13b0 0x7f746c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.168+0000 7f74966ffb80 1 Processor -- start 2024-05-08T23:47:05.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f74966ffb80 1 -- start start 2024-05-08T23:47:05.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f7483fff700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f746c0a13b0 0x7f746c0a3870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7474005950 tx=0x7f74740058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 0x564cf13dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dd720 0x564cf13d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f74966ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf132e150 con 0x564cf13d7890 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f74966ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf132bb10 con 0x564cf13dd720 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f74966ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf1327400 con 0x564cf13dce00 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f7488ae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f7488ae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51844/0 (socket says 172.21.15.12:51844) 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f7488ae2700 1 -- 172.21.15.12:0/1610527481 learned_addr learned my addr 172.21.15.12:0/1610527481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f7483fff700 1 --2- 172.21.15.12:0/1610527481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 0x564cf13dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f74837fe700 1 --2- 172.21.15.12:0/1610527481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dd720 0x564cf13d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f7488ae2700 1 -- 172.21.15.12:0/1610527481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 msgr2=0x564cf13dd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f7488ae2700 1 --2- 172.21.15.12:0/1610527481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 0x564cf13dd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f7488ae2700 1 -- 172.21.15.12:0/1610527481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dd720 msgr2=0x564cf13d7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f7488ae2700 1 --2- 172.21.15.12:0/1610527481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dd720 0x564cf13d7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.169+0000 7f7488ae2700 1 -- 172.21.15.12:0/1610527481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf11a83b0 con 0x564cf13d7890 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.170+0000 7f7488ae2700 1 --2- 172.21.15.12:0/1610527481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7cd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f747c00d7b0 tx=0x7f747c095b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7b150ad75b744b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.170+0000 7f746a7fc700 1 -- 172.21.15.12:0/1610527481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c082ea0 con 0x564cf13d7890 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.170+0000 7f746a7fc700 1 -- 172.21.15.12:0/1610527481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f747c0996a0 con 0x564cf13d7890 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.170+0000 7f746a7fc700 1 -- 172.21.15.12:0/1610527481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c095db0 con 0x564cf13d7890 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 -- 172.21.15.12:0/1610527481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 msgr2=0x564cf13d7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 --2- 172.21.15.12:0/1610527481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7cd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f747c00d7b0 tx=0x7f747c095b10 comp rx=0 tx=0).stop 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 -- 172.21.15.12:0/1610527481 shutdown_connections 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 --2- 172.21.15.12:0/1610527481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7cd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 --2- 172.21.15.12:0/1610527481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 0x564cf13dd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 --2- 172.21.15.12:0/1610527481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dd720 0x564cf13d7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 -- 172.21.15.12:0/1610527481 >> 172.21.15.12:0/1610527481 conn(0x564cf1335210 msgr2=0x564cf13d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 -- 172.21.15.12:0/1610527481 shutdown_connections 2024-05-08T23:47:05.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 -- 172.21.15.12:0/1610527481 wait complete. 2024-05-08T23:47:05.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 -- start start 2024-05-08T23:47:05.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.171+0000 7f74966ffb80 1 Processor -- start 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13d7890 0x564cf13e0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 0x564cf13e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74966ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf12c3560 con 0x564cf13dd720 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74966ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf12c3200 con 0x564cf13d7890 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74966ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf11aa6a0 con 0x564cf13dce00 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74966ffb80 10 client.0 _ll_register_callbacks cb 0x564cf12ccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f7488ae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74837fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 0x564cf13e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f7488ae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51860/0 (socket says 172.21.15.12:51860) 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f7488ae2700 1 -- 172.21.15.12:0/3546200572 learned_addr learned my addr 172.21.15.12:0/3546200572 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f7483fff700 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13d7890 0x564cf13e0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74837fe700 1 -- 172.21.15.12:0/3546200572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13d7890 msgr2=0x564cf13e0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74837fe700 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13d7890 0x564cf13e0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74837fe700 1 -- 172.21.15.12:0/3546200572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 msgr2=0x564cf13e1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74837fe700 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f74837fe700 1 -- 172.21.15.12:0/3546200572 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74841a6690 con 0x564cf13dce00 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.172+0000 7f7488ae2700 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.173+0000 7f74837fe700 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 0x564cf13e0ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f74840749d0 tx=0x7f7478005790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2284debff8654a35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.173+0000 7f74637fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.173+0000 7f74637fe700 1 -- 172.21.15.12:0/3546200572 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747800bb80 con 0x564cf13dce00 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.173+0000 7f74966ffb80 1 -- 172.21.15.12:0/3546200572 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564cf1280270 con 0x564cf13dce00 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.173+0000 7f74637fe700 1 -- 172.21.15.12:0/3546200572 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f747800bce0 con 0x564cf13dce00 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.173+0000 7f74637fe700 1 -- 172.21.15.12:0/3546200572 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747800bea0 con 0x564cf13dce00 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.173+0000 7f74966ffb80 10 client.25426 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.173+0000 7f74966ffb80 1 -- 172.21.15.12:0/3546200572 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564cf1154b60 con 0x564cf13dce00 2024-05-08T23:47:05.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.173+0000 7f74637fe700 1 -- 172.21.15.12:0/3546200572 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564cf1280270 con 0x564cf13dce00 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.173+0000 7f74637fe700 1 -- 172.21.15.12:0/3546200572 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f747800f580 con 0x564cf13dce00 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.174+0000 7f74966ffb80 10 client.25426 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.174+0000 7f74966ffb80 10 client.25426 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.174+0000 7f74966ffb80 20 client.25426 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.174+0000 7f74966ffb80 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf13f1250 0x564cf13f3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.174+0000 7f74966ffb80 4 client.25426 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.174+0000 7f74966ffb80 1 -- 172.21.15.12:0/3546200572 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cf13a6810 con 0x564cf13f1250 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.174+0000 7f7483fff700 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf13f1250 0x564cf13f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.174+0000 7f74637fe700 10 client.25426 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.174+0000 7f7483fff700 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf13f1250 0x564cf13f3670 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53e754d6a149a62a server_cookie=2b473730f3d4b6db in_seq=0 out_seq=0 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.175+0000 7f74637fe700 1 -- 172.21.15.12:0/3546200572 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cf13a6810 con 0x564cf13f1250 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.175+0000 7f74637fe700 10 client.25426 handle_command_reply: tid=0 2024-05-08T23:47:05.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.175+0000 7f74966ffb80 1 client.25426 shutdown 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 1 -- 172.21.15.12:0/3546200572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 msgr2=0x564cf13e0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 0x564cf13e0ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f74840749d0 tx=0x7f7478005790 comp rx=0 tx=0).stop 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 1 -- 172.21.15.12:0/3546200572 shutdown_connections 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf13f1250 0x564cf13f3670 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13d7890 0x564cf13e0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 1 --2- 172.21.15.12:0/3546200572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dce00 0x564cf13e0ff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74637fe700 0 client.25426 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 1 -- 172.21.15.12:0/3546200572 >> 172.21.15.12:0/3546200572 conn(0x564cf1335210 msgr2=0x564cf13d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 1 -- 172.21.15.12:0/3546200572 shutdown_connections 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 1 -- 172.21.15.12:0/3546200572 wait complete. 2024-05-08T23:47:05.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.176+0000 7f74966ffb80 20 client.25426 trim_cache size 0 max 16384 2024-05-08T23:47:05.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f74966ffb80 1 Processor -- start 2024-05-08T23:47:05.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f74966ffb80 1 -- start start 2024-05-08T23:47:05.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 0x564cf13ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dd720 0x564cf13d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f74966ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf13a6840 con 0x564cf13d7890 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f74966ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf12802a0 con 0x564cf13dce00 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f74966ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf13d2b30 con 0x564cf13dd720 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f7483fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f7483fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51874/0 (socket says 172.21.15.12:51874) 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f7483fff700 1 -- 172.21.15.12:0/3189085958 learned_addr learned my addr 172.21.15.12:0/3189085958 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f74837fe700 1 --2- 172.21.15.12:0/3189085958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 0x564cf13ecbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f7488ae2700 1 --2- 172.21.15.12:0/3189085958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dd720 0x564cf13d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f7483fff700 1 -- 172.21.15.12:0/3189085958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dd720 msgr2=0x564cf13d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f7483fff700 1 --2- 172.21.15.12:0/3189085958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dd720 0x564cf13d1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f7483fff700 1 -- 172.21.15.12:0/3189085958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 msgr2=0x564cf13ecbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f7483fff700 1 --2- 172.21.15.12:0/3189085958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 0x564cf13ecbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.190+0000 7f7483fff700 1 -- 172.21.15.12:0/3189085958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf12ed560 con 0x564cf13d7890 2024-05-08T23:47:05.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f7483fff700 1 --2- 172.21.15.12:0/3189085958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7350 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x564cf13f3f80 tx=0x7f7474012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886cbef0152658db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f746affd700 1 -- 172.21.15.12:0/3189085958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474024040 con 0x564cf13d7890 2024-05-08T23:47:05.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f746affd700 1 -- 172.21.15.12:0/3189085958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f747402a8d0 con 0x564cf13d7890 2024-05-08T23:47:05.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f746affd700 1 -- 172.21.15.12:0/3189085958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747401fd60 con 0x564cf13d7890 2024-05-08T23:47:05.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f74966ffb80 1 -- 172.21.15.12:0/3189085958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 msgr2=0x564cf13d7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f74966ffb80 1 --2- 172.21.15.12:0/3189085958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7350 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x564cf13f3f80 tx=0x7f7474012000 comp rx=0 tx=0).stop 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f74966ffb80 1 -- 172.21.15.12:0/3189085958 shutdown_connections 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f74966ffb80 1 --2- 172.21.15.12:0/3189085958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13dd720 0x564cf13d1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f74966ffb80 1 --2- 172.21.15.12:0/3189085958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13d7890 0x564cf13d7350 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f74966ffb80 1 --2- 172.21.15.12:0/3189085958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 0x564cf13ecbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f74966ffb80 1 -- 172.21.15.12:0/3189085958 >> 172.21.15.12:0/3189085958 conn(0x564cf1335210 msgr2=0x564cf13ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f74966ffb80 1 -- 172.21.15.12:0/3189085958 shutdown_connections 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.191+0000 7f74966ffb80 1 -- 172.21.15.12:0/3189085958 wait complete. 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f74966ffb80 1 -- start start 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f74966ffb80 1 Processor -- start 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13d7890 0x564cf13e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 0x564cf13e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f74966ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f74966ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf1446770 con 0x564cf13dd720 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f74966ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf14468e0 con 0x564cf13dce00 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f74966ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf1446aa0 con 0x564cf13d7890 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f74966ffb80 10 client.0 _ll_register_callbacks cb 0x564cf12ccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f7488ae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f7488ae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51890/0 (socket says 172.21.15.12:51890) 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f7483fff700 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13d7890 0x564cf13e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.192+0000 7f7488ae2700 1 -- 172.21.15.12:0/2724328412 learned_addr learned my addr 172.21.15.12:0/2724328412 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f74837fe700 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 0x564cf13e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f7488ae2700 1 -- 172.21.15.12:0/2724328412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13d7890 msgr2=0x564cf13e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f7488ae2700 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13d7890 0x564cf13e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f7488ae2700 1 -- 172.21.15.12:0/2724328412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 msgr2=0x564cf13e4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f7488ae2700 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 0x564cf13e4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f7488ae2700 1 -- 172.21.15.12:0/2724328412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747400fb60 con 0x564cf13dd720 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f74837fe700 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 0x564cf13e4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f7488ae2700 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e4b60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f747c00dfd0 tx=0x7f747c0991e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a86159aa804eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f74637fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f74637fe700 1 -- 172.21.15.12:0/2724328412 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c082ea0 con 0x564cf13dd720 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f74637fe700 1 -- 172.21.15.12:0/2724328412 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f747c0a1b20 con 0x564cf13dd720 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f74966ffb80 1 -- 172.21.15.12:0/2724328412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f747c0996a0 con 0x564cf13dd720 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f74637fe700 1 -- 172.21.15.12:0/2724328412 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747c099b00 con 0x564cf13dd720 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f74966ffb80 10 client.15585 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.193+0000 7f74966ffb80 1 -- 172.21.15.12:0/2724328412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564cf11accd0 con 0x564cf13dd720 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f74637fe700 1 -- 172.21.15.12:0/2724328412 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f747c0996a0 con 0x564cf13dd720 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f74637fe700 1 -- 172.21.15.12:0/2724328412 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f747c0a1720 con 0x564cf13dd720 2024-05-08T23:47:05.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f74966ffb80 10 client.15585 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f74966ffb80 10 client.15585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f74966ffb80 20 client.15585 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f74966ffb80 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf1447f50 0x564cf144a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f74966ffb80 4 client.15585 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f74966ffb80 1 -- 172.21.15.12:0/2724328412 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564cf131fec0 con 0x564cf1447f50 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f7483fff700 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf1447f50 0x564cf144a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f74637fe700 10 client.15585 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.194+0000 7f7483fff700 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf1447f50 0x564cf144a370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a80ab2940c7c1457 server_cookie=dde4e502c55e2ff7 in_seq=0 out_seq=0 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74637fe700 1 -- 172.21.15.12:0/2724328412 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564cf131fec0 con 0x564cf1447f50 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74637fe700 10 client.15585 handle_command_reply: tid=0 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 client.15585 shutdown 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 -- 172.21.15.12:0/2724328412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 msgr2=0x564cf13e4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e4b60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f747c00dfd0 tx=0x7f747c0991e0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 -- 172.21.15.12:0/2724328412 shutdown_connections 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf1447f50 0x564cf144a370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf13dd720 0x564cf13e4b60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf13d7890 0x564cf13e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74637fe700 0 client.15585 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 --2- 172.21.15.12:0/2724328412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf13dce00 0x564cf13e4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 -- 172.21.15.12:0/2724328412 >> 172.21.15.12:0/2724328412 conn(0x564cf1335210 msgr2=0x564cf13df360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 -- 172.21.15.12:0/2724328412 shutdown_connections 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 1 -- 172.21.15.12:0/2724328412 wait complete. 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.195+0000 7f74966ffb80 20 client.15585 trim_cache size 0 max 16384 2024-05-08T23:47:05.197 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:47:05.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 -- 172.21.15.12:0/3037149494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f746c0a13b0 msgr2=0x7f746c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f746c0a13b0 0x7f746c0a3870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7474005950 tx=0x7f74740058e0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 -- 172.21.15.12:0/3037149494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7484073d20 msgr2=0x7f74841a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7484073d20 0x7f74841a55a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f747c007ee0 tx=0x7f747c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 -- 172.21.15.12:0/3037149494 shutdown_connections 2024-05-08T23:47:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f746c0a13b0 0x7f746c0a3870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7484073d20 0x7f74841a55a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7484101160 0x7f74840732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 --2- 172.21.15.12:0/3037149494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7484106280 0x7f74840737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 -- 172.21.15.12:0/3037149494 >> 172.21.15.12:0/3037149494 conn(0x7f74840fc9e0 msgr2=0x7f74840fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 -- 172.21.15.12:0/3037149494 shutdown_connections 2024-05-08T23:47:05.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.196+0000 7f748a545700 1 -- 172.21.15.12:0/3037149494 wait complete. 2024-05-08T23:47:05.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:47:05.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 -- 172.21.15.12:0/2806466770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4101b90 msgr2=0x7fb5b4105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 --2- 172.21.15.12:0/2806466770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4101b90 0x7fb5b4105bd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009990 tx=0x7fb59c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 -- 172.21.15.12:0/2806466770 shutdown_connections 2024-05-08T23:47:05.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 --2- 172.21.15.12:0/2806466770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b4106300 0x7fb5b410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 --2- 172.21.15.12:0/2806466770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b41011e0 0x7fb5b41015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 --2- 172.21.15.12:0/2806466770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4101b90 0x7fb5b4105bd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 -- 172.21.15.12:0/2806466770 >> 172.21.15.12:0/2806466770 conn(0x7fb5b40fca80 msgr2=0x7fb5b40feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 -- 172.21.15.12:0/2806466770 shutdown_connections 2024-05-08T23:47:05.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 -- 172.21.15.12:0/2806466770 wait complete. 2024-05-08T23:47:05.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 Processor -- start 2024-05-08T23:47:05.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.348+0000 7fb5b9f76700 1 -- start start 2024-05-08T23:47:05.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b9f76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b41011e0 0x7fb5b41974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b9f76700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4106300 0x7fb5b4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b9f76700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b419bd60 0x7fb5b419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b9f76700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b419c7e0 con 0x7fb5b41011e0 2024-05-08T23:47:05.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b9f76700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b419c950 con 0x7fb5b419bd60 2024-05-08T23:47:05.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b9f76700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b419cb10 con 0x7fb5b4106300 2024-05-08T23:47:05.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b41011e0 0x7fb5b41974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b41011e0 0x7fb5b41974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51896/0 (socket says 172.21.15.12:51896) 2024-05-08T23:47:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b37fe700 1 -- 172.21.15.12:0/1902281947 learned_addr learned my addr 172.21.15.12:0/1902281947 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5aaffd700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4106300 0x7fb5b4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b3fff700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b419bd60 0x7fb5b419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b37fe700 1 -- 172.21.15.12:0/1902281947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4106300 msgr2=0x7fb5b4197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4106300 0x7fb5b4197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b37fe700 1 -- 172.21.15.12:0/1902281947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b419bd60 msgr2=0x7fb5b419c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b419bd60 0x7fb5b419c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.349+0000 7fb5b37fe700 1 -- 172.21.15.12:0/1902281947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c0095f0 con 0x7fb5b41011e0 2024-05-08T23:47:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.350+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b41011e0 0x7fb5b41974d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400d980 tx=0x7fb5a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.350+0000 7fb5b17fa700 1 -- 172.21.15.12:0/1902281947 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4004220 con 0x7fb5b41011e0 2024-05-08T23:47:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.350+0000 7fb5b9f76700 1 -- 172.21.15.12:0/1902281947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b41a4070 con 0x7fb5b41011e0 2024-05-08T23:47:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.350+0000 7fb5b17fa700 1 -- 172.21.15.12:0/1902281947 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a4007640 con 0x7fb5b41011e0 2024-05-08T23:47:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.350+0000 7fb5b17fa700 1 -- 172.21.15.12:0/1902281947 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4005610 con 0x7fb5b41011e0 2024-05-08T23:47:05.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.350+0000 7fb5b9f76700 1 -- 172.21.15.12:0/1902281947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b41a45a0 con 0x7fb5b41011e0 2024-05-08T23:47:05.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.352+0000 7fb5b17fa700 1 -- 172.21.15.12:0/1902281947 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fb5a4004750 con 0x7fb5b41011e0 2024-05-08T23:47:05.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.353+0000 7fb5b17fa700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a13b0 0x7fb5a00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.353+0000 7fb5b17fa700 1 -- 172.21.15.12:0/1902281947 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb5a40ccd90 con 0x7fb5b41011e0 2024-05-08T23:47:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.353+0000 7fb5aaffd700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a13b0 0x7fb5a00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.353+0000 7fb5aaffd700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a13b0 0x7fb5a00a3870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5b41989d0 tx=0x7fb59c005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5c6130b80 1 Processor -- start 2024-05-08T23:47:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5c6130b80 1 -- start start 2024-05-08T23:47:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a02944c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944cb30 0x55a029446760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a029446ca0 0x55a0294470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a029399150 con 0x55a02944c210 2024-05-08T23:47:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a029396b10 con 0x55a02944cb30 2024-05-08T23:47:05.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a029392400 con 0x55a029446ca0 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a02944c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a02944c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51898/0 (socket says 172.21.15.12:51898) 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5b37fe700 1 -- 172.21.15.12:0/4281323597 learned_addr learned my addr 172.21.15.12:0/4281323597 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5aaffd700 1 --2- 172.21.15.12:0/4281323597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944cb30 0x55a029446760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5b3fff700 1 --2- 172.21.15.12:0/4281323597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a029446ca0 0x55a0294470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5b37fe700 1 -- 172.21.15.12:0/4281323597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a029446ca0 msgr2=0x55a0294470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/4281323597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a029446ca0 0x55a0294470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5b37fe700 1 -- 172.21.15.12:0/4281323597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944cb30 msgr2=0x55a029446760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/4281323597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944cb30 0x55a029446760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.354+0000 7fb5b37fe700 1 -- 172.21.15.12:0/4281323597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0292133b0 con 0x55a02944c210 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.355+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/4281323597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a02944c5f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4099990 tx=0x7fb5a409a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad22a20144ec1612 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.355+0000 7fb5a9ffb700 1 -- 172.21.15.12:0/4281323597 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a409dac0 con 0x55a02944c210 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.355+0000 7fb5a9ffb700 1 -- 172.21.15.12:0/4281323597 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a40a2110 con 0x55a02944c210 2024-05-08T23:47:05.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.355+0000 7fb5a9ffb700 1 -- 172.21.15.12:0/4281323597 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40a1c80 con 0x55a02944c210 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.355+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4281323597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 msgr2=0x55a02944c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.355+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/4281323597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a02944c5f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4099990 tx=0x7fb5a409a390 comp rx=0 tx=0).stop 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.356+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4281323597 shutdown_connections 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.356+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/4281323597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a029446ca0 0x55a0294470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.356+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/4281323597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a02944c5f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.356+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/4281323597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944cb30 0x55a029446760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.356+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4281323597 >> 172.21.15.12:0/4281323597 conn(0x55a0293a0210 msgr2=0x55a02943cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.356+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4281323597 shutdown_connections 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.356+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4281323597 wait complete. 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.356+0000 7fb5c6130b80 1 -- start start 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.356+0000 7fb5c6130b80 1 Processor -- start 2024-05-08T23:47:05.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a029446ca0 0x55a02944fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944c210 0x55a0294503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944cb30 0x55a029450900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02932e560 con 0x55a02944cb30 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02932e200 con 0x55a029446ca0 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0292156a0 con 0x55a02944c210 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5c6130b80 10 client.0 _ll_register_callbacks cb 0x55a029337b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944cb30 0x55a029450900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5aaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944c210 0x55a0294503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944cb30 0x55a029450900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51900/0 (socket says 172.21.15.12:51900) 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b3fff700 1 -- 172.21.15.12:0/1381146243 learned_addr learned my addr 172.21.15.12:0/1381146243 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a029446ca0 0x55a02944fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b3fff700 1 -- 172.21.15.12:0/1381146243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944c210 msgr2=0x55a0294503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b3fff700 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944c210 0x55a0294503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b3fff700 1 -- 172.21.15.12:0/1381146243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a029446ca0 msgr2=0x55a02944fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b3fff700 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a029446ca0 0x55a02944fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b3fff700 1 -- 172.21.15.12:0/1381146243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b41a45a0 con 0x55a02944cb30 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5aaffd700 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944c210 0x55a0294503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a029446ca0 0x55a02944fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb5b3fff700 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944cb30 0x55a029450900 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac008ed0 tx=0x7fb5ac0114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87b653f987e20fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb592ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.357+0000 7fb592ffd700 1 -- 172.21.15.12:0/1381146243 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac011ec0 con 0x55a02944cb30 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb592ffd700 1 -- 172.21.15.12:0/1381146243 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb5ac007e20 con 0x55a02944cb30 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1381146243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a0292eb270 con 0x55a02944cb30 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb592ffd700 1 -- 172.21.15.12:0/1381146243 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5ac016440 con 0x55a02944cb30 2024-05-08T23:47:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb5c6130b80 10 client.15597 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1381146243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a0291bfb60 con 0x55a02944cb30 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb592ffd700 1 -- 172.21.15.12:0/1381146243 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a0292eb270 con 0x55a02944cb30 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb592ffd700 1 -- 172.21.15.12:0/1381146243 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb5ac01f570 con 0x55a02944cb30 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb5c6130b80 10 client.15597 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb5c6130b80 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb5c6130b80 20 client.15597 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a029460540 0x55a029462960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb5c6130b80 4 client.15597 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1381146243 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a029411810 con 0x55a029460540 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.358+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a029460540 0x55a029462960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.359+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a029460540 0x55a029462960 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9c1d099e00571e server_cookie=51d19e60eaa4c590 in_seq=0 out_seq=0 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.359+0000 7fb592ffd700 10 client.15597 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.359+0000 7fb592ffd700 1 -- 172.21.15.12:0/1381146243 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a029411810 con 0x55a029460540 2024-05-08T23:47:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.359+0000 7fb592ffd700 10 client.15597 handle_command_reply: tid=0 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.359+0000 7fb5c6130b80 1 client.15597 shutdown 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1381146243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944cb30 msgr2=0x55a029450900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944cb30 0x55a029450900 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac008ed0 tx=0x7fb5ac0114a0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1381146243 shutdown_connections 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944cb30 0x55a029450900 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a029460540 0x55a029462960 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a029446ca0 0x55a02944fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/1381146243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944c210 0x55a0294503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb592ffd700 0 client.15597 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1381146243 >> 172.21.15.12:0/1381146243 conn(0x55a0293a0210 msgr2=0x55a02943c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1381146243 shutdown_connections 2024-05-08T23:47:05.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1381146243 wait complete. 2024-05-08T23:47:05.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.360+0000 7fb5c6130b80 20 client.15597 trim_cache size 0 max 16384 2024-05-08T23:47:05.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.374+0000 7fb5c6130b80 1 Processor -- start 2024-05-08T23:47:05.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.374+0000 7fb5c6130b80 1 -- start start 2024-05-08T23:47:05.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.374+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a029446ca0 0x55a029446760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.374+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944c210 0x55a02943cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.374+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944cb30 0x55a029457450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.374+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02944fc70 con 0x55a029446ca0 2024-05-08T23:47:05.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.374+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a029451450 con 0x55a02944c210 2024-05-08T23:47:05.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.374+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a029437800 con 0x55a02944cb30 2024-05-08T23:47:05.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a029446ca0 0x55a029446760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a029446ca0 0x55a029446760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51908/0 (socket says 172.21.15.12:51908) 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5b37fe700 1 -- 172.21.15.12:0/1446671027 learned_addr learned my addr 172.21.15.12:0/1446671027 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5aaffd700 1 --2- 172.21.15.12:0/1446671027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944c210 0x55a02943cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5b3fff700 1 --2- 172.21.15.12:0/1446671027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944cb30 0x55a029457450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5b37fe700 1 -- 172.21.15.12:0/1446671027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944cb30 msgr2=0x55a029457450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/1446671027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944cb30 0x55a029457450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5b37fe700 1 -- 172.21.15.12:0/1446671027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944c210 msgr2=0x55a02943cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/1446671027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944c210 0x55a02943cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5b37fe700 1 -- 172.21.15.12:0/1446671027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a029447350 con 0x55a029446ca0 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/1446671027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a029446ca0 0x55a029446760 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400d950 tx=0x7fb5a4096100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2a86d5534b9d4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5aa7fc700 1 -- 172.21.15.12:0/1446671027 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a409dac0 con 0x55a029446ca0 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5aa7fc700 1 -- 172.21.15.12:0/1446671027 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a4004380 con 0x55a029446ca0 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.375+0000 7fb5aa7fc700 1 -- 172.21.15.12:0/1446671027 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40a1ee0 con 0x55a029446ca0 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.376+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1446671027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a029446ca0 msgr2=0x55a029446760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.376+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/1446671027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a029446ca0 0x55a029446760 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400d950 tx=0x7fb5a4096100 comp rx=0 tx=0).stop 2024-05-08T23:47:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.376+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1446671027 shutdown_connections 2024-05-08T23:47:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.376+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/1446671027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a02944cb30 0x55a029457450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.376+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/1446671027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a029446ca0 0x55a029446760 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.376+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/1446671027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944c210 0x55a02943cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.376+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1446671027 >> 172.21.15.12:0/1446671027 conn(0x55a0293a0210 msgr2=0x55a02944e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.376+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1446671027 shutdown_connections 2024-05-08T23:47:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.376+0000 7fb5c6130b80 1 -- 172.21.15.12:0/1446671027 wait complete. 2024-05-08T23:47:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.376+0000 7fb5c6130b80 1 -- start start 2024-05-08T23:47:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5c6130b80 1 Processor -- start 2024-05-08T23:47:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a029446ca0 0x55a029457930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a029449050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5c6130b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944cb30 0x55a029449790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a029447f10 con 0x55a02944c210 2024-05-08T23:47:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a029448080 con 0x55a02944cb30 2024-05-08T23:47:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5c6130b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a029448240 con 0x55a029446ca0 2024-05-08T23:47:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5c6130b80 10 client.0 _ll_register_callbacks cb 0x55a029337b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5aaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a029449050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5aaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a029449050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51924/0 (socket says 172.21.15.12:51924) 2024-05-08T23:47:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5aaffd700 1 -- 172.21.15.12:0/4102770498 learned_addr learned my addr 172.21.15.12:0/4102770498 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a029446ca0 0x55a029457930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5b3fff700 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944cb30 0x55a029449790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5aaffd700 1 -- 172.21.15.12:0/4102770498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a029446ca0 msgr2=0x55a029457930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5aaffd700 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a029446ca0 0x55a029457930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5aaffd700 1 -- 172.21.15.12:0/4102770498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944cb30 msgr2=0x55a029449790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5aaffd700 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944cb30 0x55a029449790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.377+0000 7fb5aaffd700 1 -- 172.21.15.12:0/4102770498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0292133b0 con 0x55a02944c210 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb5aaffd700 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a029449050 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb5b406a780 tx=0x7fb59c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadbb1d5af855ad8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb592ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb592ffd700 1 -- 172.21.15.12:0/4102770498 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c00f460 con 0x55a02944c210 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb592ffd700 1 -- 172.21.15.12:0/4102770498 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb59c00f5c0 con 0x55a02944c210 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4102770498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5a40a2110 con 0x55a02944c210 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb592ffd700 1 -- 172.21.15.12:0/4102770498 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb59c00f780 con 0x55a02944c210 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb5c6130b80 10 client.15603 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4102770498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a029358560 con 0x55a02944c210 2024-05-08T23:47:05.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb592ffd700 1 -- 172.21.15.12:0/4102770498 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a40a2110 con 0x55a02944c210 2024-05-08T23:47:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb592ffd700 1 -- 172.21.15.12:0/4102770498 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb59c0459d0 con 0x55a02944c210 2024-05-08T23:47:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb5c6130b80 10 client.15603 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb5c6130b80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb5c6130b80 20 client.15603 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a0294b75b0 0x55a0294b99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb5c6130b80 4 client.15603 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:47:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.378+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4102770498 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a02938aec0 con 0x55a0294b75b0 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.379+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a0294b75b0 0x55a0294b99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.379+0000 7fb592ffd700 10 client.15603 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.379+0000 7fb5b37fe700 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a0294b75b0 0x55a0294b99d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d2de5ad4bd94e4 server_cookie=87b164ddffe718e7 in_seq=0 out_seq=0 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.379+0000 7fb592ffd700 1 -- 172.21.15.12:0/4102770498 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a02938aec0 con 0x55a0294b75b0 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.379+0000 7fb592ffd700 10 client.15603 handle_command_reply: tid=0 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.379+0000 7fb5c6130b80 1 client.15603 shutdown 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4102770498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 msgr2=0x55a029449050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a029449050 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb5b406a780 tx=0x7fb59c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4102770498 shutdown_connections 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a0294b75b0 0x55a0294b99d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a02944cb30 0x55a029449790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a029446ca0 0x55a029457930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb592ffd700 0 client.15603 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:05.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 1 --2- 172.21.15.12:0/4102770498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02944c210 0x55a029449050 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4102770498 >> 172.21.15.12:0/4102770498 conn(0x55a0293a0210 msgr2=0x55a02944afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4102770498 shutdown_connections 2024-05-08T23:47:05.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 1 -- 172.21.15.12:0/4102770498 wait complete. 2024-05-08T23:47:05.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5c6130b80 20 client.15603 trim_cache size 0 max 16384 2024-05-08T23:47:05.385 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:47:05.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5b9f76700 1 -- 172.21.15.12:0/1902281947 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a13b0 msgr2=0x7fb5a00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5b9f76700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a13b0 0x7fb5a00a3870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5b41989d0 tx=0x7fb59c005960 comp rx=0 tx=0).stop 2024-05-08T23:47:05.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5b9f76700 1 -- 172.21.15.12:0/1902281947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b41011e0 msgr2=0x7fb5b41974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:05.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.380+0000 7fb5b9f76700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b41011e0 0x7fb5b41974d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400d980 tx=0x7fb5a400dc90 comp rx=0 tx=0).stop 2024-05-08T23:47:05.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.381+0000 7fb5b9f76700 1 -- 172.21.15.12:0/1902281947 shutdown_connections 2024-05-08T23:47:05.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.381+0000 7fb5b9f76700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a13b0 0x7fb5a00a3870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.381+0000 7fb5b9f76700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b419bd60 0x7fb5b419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.381+0000 7fb5b9f76700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b41011e0 0x7fb5b41974d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.381+0000 7fb5b9f76700 1 --2- 172.21.15.12:0/1902281947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4106300 0x7fb5b4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.381+0000 7fb5b9f76700 1 -- 172.21.15.12:0/1902281947 >> 172.21.15.12:0/1902281947 conn(0x7fb5b40fca80 msgr2=0x7fb5b40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.381+0000 7fb5b9f76700 1 -- 172.21.15.12:0/1902281947 shutdown_connections 2024-05-08T23:47:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:05.381+0000 7fb5b9f76700 1 -- 172.21.15.12:0/1902281947 wait complete. 2024-05-08T23:47:05.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:05 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2892782063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:47:05.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:05 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2892782063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:47:05.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:05 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2892782063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:47:06.390 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:47:06.391 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:47:06.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.548+0000 7f8e6d95f700 1 -- 172.21.15.12:0/2706588317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e680697a0 msgr2=0x7f8e68069b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.548+0000 7f8e6d95f700 1 --2- 172.21.15.12:0/2706588317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e680697a0 0x7f8e68069b80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8e580099c0 tx=0x7f8e58009cd0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.548+0000 7f8e6d95f700 1 -- 172.21.15.12:0/2706588317 shutdown_connections 2024-05-08T23:47:06.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.548+0000 7f8e6d95f700 1 --2- 172.21.15.12:0/2706588317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e681062a0 0x7f8e6810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.548+0000 7f8e6d95f700 1 --2- 172.21.15.12:0/2706588317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e680697a0 0x7f8e68069b80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.548+0000 7f8e6d95f700 1 --2- 172.21.15.12:0/2706588317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e6806a150 0x7f8e68105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.548+0000 7f8e6d95f700 1 -- 172.21.15.12:0/2706588317 >> 172.21.15.12:0/2706588317 conn(0x7f8e68076f40 msgr2=0x7f8e68077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.548+0000 7f8e6d95f700 1 -- 172.21.15.12:0/2706588317 shutdown_connections 2024-05-08T23:47:06.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.548+0000 7f8e6d95f700 1 -- 172.21.15.12:0/2706588317 wait complete. 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e6d95f700 1 Processor -- start 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e6d95f700 1 -- start start 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e6d95f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e680697a0 0x7f8e68197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e6d95f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e681062a0 0x7f8e68197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e6d95f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e6819bce0 0x7f8e6819c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e6d95f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6819c760 con 0x7f8e6819bce0 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e6d95f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6819c8d0 con 0x7f8e681062a0 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e6d95f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6819ca90 con 0x7f8e680697a0 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e6819bce0 0x7f8e6819c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e6819bce0 0x7f8e6819c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51940/0 (socket says 172.21.15.12:51940) 2024-05-08T23:47:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e677fe700 1 -- 172.21.15.12:0/582516002 learned_addr learned my addr 172.21.15.12:0/582516002 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.549+0000 7f8e667fc700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e681062a0 0x7f8e68197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e680697a0 0x7f8e68197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e677fe700 1 -- 172.21.15.12:0/582516002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e680697a0 msgr2=0x7f8e68197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e677fe700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e680697a0 0x7f8e68197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e677fe700 1 -- 172.21.15.12:0/582516002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e681062a0 msgr2=0x7f8e68197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e677fe700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e681062a0 0x7f8e68197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e677fe700 1 -- 172.21.15.12:0/582516002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e580095f0 con 0x7f8e6819bce0 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e677fe700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e6819bce0 0x7f8e6819c160 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000ef30 tx=0x7f8e6000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e6c95d700 1 -- 172.21.15.12:0/582516002 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e60018070 con 0x7f8e6819bce0 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e6c95d700 1 -- 172.21.15.12:0/582516002 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e6000f040 con 0x7f8e6819bce0 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e6d95f700 1 -- 172.21.15.12:0/582516002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e681a3f90 con 0x7f8e6819bce0 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e6c95d700 1 -- 172.21.15.12:0/582516002 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6000cdb0 con 0x7f8e6819bce0 2024-05-08T23:47:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.550+0000 7f8e6d95f700 1 -- 172.21.15.12:0/582516002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e681a4420 con 0x7f8e6819bce0 2024-05-08T23:47:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.553+0000 7f8e6c95d700 1 -- 172.21.15.12:0/582516002 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f8e60003a00 con 0x7f8e6819bce0 2024-05-08T23:47:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.553+0000 7f8e6c95d700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8e500a13b0 0x7f8e500a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.554+0000 7f8e6c95d700 1 -- 172.21.15.12:0/582516002 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8e600ccb00 con 0x7f8e6819bce0 2024-05-08T23:47:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.554+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8e500a13b0 0x7f8e500a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.554+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8e500a13b0 0x7f8e500a3870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009990 tx=0x7f8e58005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.554+0000 7f8e79b19b80 1 Processor -- start 2024-05-08T23:47:06.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.554+0000 7f8e79b19b80 1 -- start start 2024-05-08T23:47:06.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 0x55bffa21a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa2147e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa215180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa1671d0 con 0x55bffa21ab90 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa164b90 con 0x55bffa214d20 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa160480 con 0x55bffa21a270 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e667fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa2147e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e667fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa2147e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51956/0 (socket says 172.21.15.12:51956) 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e667fc700 1 -- 172.21.15.12:0/2812787763 learned_addr learned my addr 172.21.15.12:0/2812787763 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/2812787763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 0x55bffa21a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e677fe700 1 --2- 172.21.15.12:0/2812787763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa215180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e667fc700 1 -- 172.21.15.12:0/2812787763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 msgr2=0x55bffa21a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e667fc700 1 --2- 172.21.15.12:0/2812787763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 0x55bffa21a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e667fc700 1 -- 172.21.15.12:0/2812787763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 msgr2=0x55bffa215180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e667fc700 1 --2- 172.21.15.12:0/2812787763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa215180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e667fc700 1 -- 172.21.15.12:0/2812787763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bff9fe1430 con 0x55bffa21ab90 2024-05-08T23:47:06.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e667fc700 1 --2- 172.21.15.12:0/2812787763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa2147e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009cf0 tx=0x7f8e5c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcd9aa1f39151b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.555+0000 7f8e4dffb700 1 -- 172.21.15.12:0/2812787763 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c011d80 con 0x55bffa21ab90 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e4dffb700 1 -- 172.21.15.12:0/2812787763 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e5c004510 con 0x55bffa21ab90 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e4dffb700 1 -- 172.21.15.12:0/2812787763 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c005280 con 0x55bffa21ab90 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2812787763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 msgr2=0x55bffa2147e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2812787763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa2147e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009cf0 tx=0x7f8e5c011440 comp rx=0 tx=0).stop 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2812787763 shutdown_connections 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2812787763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa215180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2812787763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 0x55bffa21a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2812787763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa2147e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2812787763 >> 172.21.15.12:0/2812787763 conn(0x55bffa16e290 msgr2=0x55bffa20ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2812787763 shutdown_connections 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.556+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2812787763 wait complete. 2024-05-08T23:47:06.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.557+0000 7f8e79b19b80 1 -- start start 2024-05-08T23:47:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.557+0000 7f8e79b19b80 1 Processor -- start 2024-05-08T23:47:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.557+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa220b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.557+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 0x55bffa21de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.557+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa21e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.557+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa0fc5e0 con 0x55bffa21ab90 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.557+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa0fc280 con 0x55bffa214d20 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.557+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff9fe3720 con 0x55bffa21a270 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.557+0000 7f8e79b19b80 10 client.0 _ll_register_callbacks cb 0x55bffa105bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa21e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e66ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa220b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa21e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51970/0 (socket says 172.21.15.12:51970) 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e677fe700 1 -- 172.21.15.12:0/1371435684 learned_addr learned my addr 172.21.15.12:0/1371435684 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e667fc700 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 0x55bffa21de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e677fe700 1 -- 172.21.15.12:0/1371435684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 msgr2=0x55bffa21de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e677fe700 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 0x55bffa21de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e677fe700 1 -- 172.21.15.12:0/1371435684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 msgr2=0x55bffa220b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e677fe700 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa220b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e677fe700 1 -- 172.21.15.12:0/1371435684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c009920 con 0x55bffa21ab90 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e667fc700 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 0x55bffa21de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e677fe700 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa21e4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000c7a0 tx=0x7f8e600994a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5146d6c53e627f8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e46ffd700 1 -- 172.21.15.12:0/1371435684 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6001b320 con 0x55bffa21ab90 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e46ffd700 1 -- 172.21.15.12:0/1371435684 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8e6009a190 con 0x55bffa21ab90 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e79b19b80 1 -- 172.21.15.12:0/1371435684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bffa0b92f0 con 0x55bffa21ab90 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e46ffd700 1 -- 172.21.15.12:0/1371435684 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e60099a90 con 0x55bffa21ab90 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e79b19b80 10 client.15615 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.558+0000 7f8e79b19b80 1 -- 172.21.15.12:0/1371435684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bff9f8dbe0 con 0x55bffa21ab90 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.559+0000 7f8e46ffd700 1 -- 172.21.15.12:0/1371435684 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bffa0b92f0 con 0x55bffa21ab90 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.559+0000 7f8e46ffd700 1 -- 172.21.15.12:0/1371435684 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8e600a19f0 con 0x55bffa21ab90 2024-05-08T23:47:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.559+0000 7f8e79b19b80 10 client.15615 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.559+0000 7f8e79b19b80 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.559+0000 7f8e79b19b80 20 client.15615 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.559+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bffa22e610 0x55bffa230a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.559+0000 7f8e79b19b80 4 client.15615 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.559+0000 7f8e79b19b80 1 -- 172.21.15.12:0/1371435684 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bffa1df890 con 0x55bffa22e610 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.559+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bffa22e610 0x55bffa230a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.560+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bffa22e610 0x55bffa230a30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10be78854ebb79eb server_cookie=c8e047a3d580be4c in_seq=0 out_seq=0 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.560+0000 7f8e46ffd700 10 client.15615 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.560+0000 7f8e46ffd700 1 -- 172.21.15.12:0/1371435684 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bffa1df890 con 0x55bffa22e610 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.560+0000 7f8e46ffd700 10 client.15615 handle_command_reply: tid=0 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.560+0000 7f8e79b19b80 1 client.15615 shutdown 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.560+0000 7f8e79b19b80 1 -- 172.21.15.12:0/1371435684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 msgr2=0x55bffa21e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.560+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa21e4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000c7a0 tx=0x7f8e600994a0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.560+0000 7f8e79b19b80 1 -- 172.21.15.12:0/1371435684 shutdown_connections 2024-05-08T23:47:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.560+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21ab90 0x55bffa21e4c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.560+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bffa22e610 0x55bffa230a30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.561+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa220b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.561+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/1371435684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21a270 0x55bffa21de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.561+0000 7f8e46ffd700 0 client.15615 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.561+0000 7f8e79b19b80 1 -- 172.21.15.12:0/1371435684 >> 172.21.15.12:0/1371435684 conn(0x55bffa16e290 msgr2=0x55bffa20a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.561+0000 7f8e79b19b80 1 -- 172.21.15.12:0/1371435684 shutdown_connections 2024-05-08T23:47:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.561+0000 7f8e79b19b80 1 -- 172.21.15.12:0/1371435684 wait complete. 2024-05-08T23:47:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.561+0000 7f8e79b19b80 20 client.15615 trim_cache size 0 max 16384 2024-05-08T23:47:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e79b19b80 1 Processor -- start 2024-05-08T23:47:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e79b19b80 1 -- start start 2024-05-08T23:47:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa20ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21a270 0x55bffa217fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21ab90 0x55bffa2184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa20ced0 con 0x55bffa21a270 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa20bab0 con 0x55bffa214d20 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa235c20 con 0x55bffa21ab90 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e667fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21a270 0x55bffa217fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e667fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21a270 0x55bffa217fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51982/0 (socket says 172.21.15.12:51982) 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e667fc700 1 -- 172.21.15.12:0/2936415748 learned_addr learned my addr 172.21.15.12:0/2936415748 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/2936415748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa20ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e677fe700 1 --2- 172.21.15.12:0/2936415748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21ab90 0x55bffa2184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e667fc700 1 -- 172.21.15.12:0/2936415748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21ab90 msgr2=0x55bffa2184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e667fc700 1 --2- 172.21.15.12:0/2936415748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21ab90 0x55bffa2184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e667fc700 1 -- 172.21.15.12:0/2936415748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 msgr2=0x55bffa20ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e667fc700 1 --2- 172.21.15.12:0/2936415748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa20ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e667fc700 1 -- 172.21.15.12:0/2936415748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bffa207560 con 0x55bffa21a270 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.580+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/2936415748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa20ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e667fc700 1 --2- 172.21.15.12:0/2936415748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21a270 0x55bffa217fb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55bffa21b010 tx=0x7f8e5c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1436c0a85c6a4098 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e4e7fc700 1 -- 172.21.15.12:0/2936415748 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c005710 con 0x55bffa21a270 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e4e7fc700 1 -- 172.21.15.12:0/2936415748 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e5c011b80 con 0x55bffa21a270 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e4e7fc700 1 -- 172.21.15.12:0/2936415748 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c023470 con 0x55bffa21a270 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2936415748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21a270 msgr2=0x55bffa217fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2936415748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21a270 0x55bffa217fb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55bffa21b010 tx=0x7f8e5c005ce0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2936415748 shutdown_connections 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2936415748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21ab90 0x55bffa2184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2936415748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa214d20 0x55bffa20ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2936415748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa21a270 0x55bffa217fb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2936415748 >> 172.21.15.12:0/2936415748 conn(0x55bffa16e290 msgr2=0x55bffa22a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.581+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2936415748 shutdown_connections 2024-05-08T23:47:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.582+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2936415748 wait complete. 2024-05-08T23:47:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.582+0000 7f8e79b19b80 1 -- start start 2024-05-08T23:47:06.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.582+0000 7f8e79b19b80 1 Processor -- start 2024-05-08T23:47:06.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa214d20 0x55bffa225930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa21a270 0x55bffa216350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e79b19b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21ab90 0x55bffa216890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa216fa0 con 0x55bffa214d20 2024-05-08T23:47:06.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa217110 con 0x55bffa21a270 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e79b19b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bffa284b10 con 0x55bffa21ab90 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e79b19b80 10 client.0 _ll_register_callbacks cb 0x55bffa105bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e66ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa214d20 0x55bffa225930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e66ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa214d20 0x55bffa225930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51984/0 (socket says 172.21.15.12:51984) 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e66ffd700 1 -- 172.21.15.12:0/2216266830 learned_addr learned my addr 172.21.15.12:0/2216266830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e667fc700 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa21a270 0x55bffa216350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e677fe700 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21ab90 0x55bffa216890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e66ffd700 1 -- 172.21.15.12:0/2216266830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21ab90 msgr2=0x55bffa216890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21ab90 0x55bffa216890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e66ffd700 1 -- 172.21.15.12:0/2216266830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa21a270 msgr2=0x55bffa216350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa21a270 0x55bffa216350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e66ffd700 1 -- 172.21.15.12:0/2216266830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c01d040 con 0x55bffa214d20 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e66ffd700 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa214d20 0x55bffa225930 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8e580099c0 tx=0x7f8e58009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863ed494f4f40751 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e46ffd700 1 -- 172.21.15.12:0/2216266830 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5802a040 con 0x55bffa214d20 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e46ffd700 1 -- 172.21.15.12:0/2216266830 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8e58003680 con 0x55bffa214d20 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.583+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2216266830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e5c004510 con 0x55bffa214d20 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e46ffd700 1 -- 172.21.15.12:0/2216266830 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e58003840 con 0x55bffa214d20 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e79b19b80 10 client.15621 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2216266830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bffa1265e0 con 0x55bffa214d20 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e46ffd700 1 -- 172.21.15.12:0/2216266830 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e5c004510 con 0x55bffa214d20 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e46ffd700 1 -- 172.21.15.12:0/2216266830 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8e58034770 con 0x55bffa214d20 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e79b19b80 10 client.15621 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e79b19b80 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e79b19b80 20 client.15621 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bffa286140 0x55bffa288560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e79b19b80 4 client.15621 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2216266830 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bffa158f40 con 0x55bffa286140 2024-05-08T23:47:06.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.584+0000 7f8e667fc700 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bffa286140 0x55bffa288560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.585+0000 7f8e667fc700 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bffa286140 0x55bffa288560 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ff0da0ffb71198 server_cookie=f96382f402ca887a in_seq=0 out_seq=0 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.585+0000 7f8e46ffd700 10 client.15621 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e46ffd700 1 -- 172.21.15.12:0/2216266830 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bffa158f40 con 0x55bffa286140 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e46ffd700 10 client.15621 handle_command_reply: tid=0 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 client.15621 shutdown 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2216266830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa214d20 msgr2=0x55bffa225930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa214d20 0x55bffa225930 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8e580099c0 tx=0x7f8e58009cd0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2216266830 shutdown_connections 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bffa21ab90 0x55bffa216890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bffa214d20 0x55bffa225930 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bffa286140 0x55bffa288560 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 --2- 172.21.15.12:0/2216266830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bffa21a270 0x55bffa216350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2216266830 >> 172.21.15.12:0/2216266830 conn(0x55bffa16e290 msgr2=0x55bffa22a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e46ffd700 0 client.15621 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2216266830 shutdown_connections 2024-05-08T23:47:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 1 -- 172.21.15.12:0/2216266830 wait complete. 2024-05-08T23:47:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.586+0000 7f8e79b19b80 20 client.15621 trim_cache size 0 max 16384 2024-05-08T23:47:06.587 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:47:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:47:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "f3398a9b-9ba6-4294-bef0-2bebd500579b", 2024-05-08T23:47:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:47:06.587 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:47:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 -- 172.21.15.12:0/582516002 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8e500a13b0 msgr2=0x7f8e500a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8e500a13b0 0x7f8e500a3870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009990 tx=0x7f8e58005fb0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 -- 172.21.15.12:0/582516002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e6819bce0 msgr2=0x7f8e6819c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e6819bce0 0x7f8e6819c160 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000ef30 tx=0x7f8e6000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 -- 172.21.15.12:0/582516002 shutdown_connections 2024-05-08T23:47:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8e500a13b0 0x7f8e500a3870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e6819bce0 0x7f8e6819c160 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e680697a0 0x7f8e68197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 --2- 172.21.15.12:0/582516002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e681062a0 0x7f8e68197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 -- 172.21.15.12:0/582516002 >> 172.21.15.12:0/582516002 conn(0x7f8e68076f40 msgr2=0x7f8e681073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 -- 172.21.15.12:0/582516002 shutdown_connections 2024-05-08T23:47:06.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.587+0000 7f8e6d95f700 1 -- 172.21.15.12:0/582516002 wait complete. 2024-05-08T23:47:06.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:47:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.740+0000 7f0963a7f700 1 -- 172.21.15.12:0/1469540629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f095c100a00 msgr2=0x7f095c104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.740+0000 7f0963a7f700 1 --2- 172.21.15.12:0/1469540629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f095c100a00 0x7f095c104a40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f094c009990 tx=0x7f094c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.740+0000 7f0963a7f700 1 -- 172.21.15.12:0/1469540629 shutdown_connections 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.740+0000 7f0963a7f700 1 --2- 172.21.15.12:0/1469540629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f095c105170 0x7f095c10ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.740+0000 7f0963a7f700 1 --2- 172.21.15.12:0/1469540629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095c100050 0x7f095c100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.740+0000 7f0963a7f700 1 --2- 172.21.15.12:0/1469540629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f095c100a00 0x7f095c104a40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.740+0000 7f0963a7f700 1 -- 172.21.15.12:0/1469540629 >> 172.21.15.12:0/1469540629 conn(0x7f095c076fa0 msgr2=0x7f095c0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.740+0000 7f0963a7f700 1 -- 172.21.15.12:0/1469540629 shutdown_connections 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.740+0000 7f0963a7f700 1 -- 172.21.15.12:0/1469540629 wait complete. 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f0963a7f700 1 Processor -- start 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f0963a7f700 1 -- start start 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f0963a7f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095c100050 0x7f095c101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f0963a7f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f095c105170 0x7f095c102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f0963a7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f095c102560 0x7f095c1a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f0963a7f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1a5c40 con 0x7f095c105170 2024-05-08T23:47:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f0963a7f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1a5db0 con 0x7f095c102560 2024-05-08T23:47:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f0963a7f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1a5f70 con 0x7f095c100050 2024-05-08T23:47:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f096101a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f095c105170 0x7f095c102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f096101a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f095c105170 0x7f095c102020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52012/0 (socket says 172.21.15.12:52012) 2024-05-08T23:47:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.741+0000 7f096101a700 1 -- 172.21.15.12:0/4172616886 learned_addr learned my addr 172.21.15.12:0/4172616886 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f096181b700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095c100050 0x7f095c101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f096201c700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f095c102560 0x7f095c1a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f096101a700 1 -- 172.21.15.12:0/4172616886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095c100050 msgr2=0x7f095c101ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f096101a700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095c100050 0x7f095c101ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f096101a700 1 -- 172.21.15.12:0/4172616886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f095c102560 msgr2=0x7f095c1a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f096101a700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f095c102560 0x7f095c1a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f096101a700 1 -- 172.21.15.12:0/4172616886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c0095f0 con 0x7f095c105170 2024-05-08T23:47:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f096101a700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f095c105170 0x7f095c102020 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f094c000c00 tx=0x7f094c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f0952ffd700 1 -- 172.21.15.12:0/4172616886 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c01d070 con 0x7f095c105170 2024-05-08T23:47:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f0952ffd700 1 -- 172.21.15.12:0/4172616886 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f094c00fb30 con 0x7f095c105170 2024-05-08T23:47:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f0963a7f700 1 -- 172.21.15.12:0/4172616886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c1a61f0 con 0x7f095c105170 2024-05-08T23:47:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f0952ffd700 1 -- 172.21.15.12:0/4172616886 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c022c00 con 0x7f095c105170 2024-05-08T23:47:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.742+0000 7f0963a7f700 1 -- 172.21.15.12:0/4172616886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c1a6610 con 0x7f095c105170 2024-05-08T23:47:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.744+0000 7f0952ffd700 1 -- 172.21.15.12:0/4172616886 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f094c00fca0 con 0x7f095c105170 2024-05-08T23:47:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.745+0000 7f0952ffd700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09480a13b0 0x7f09480a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.745+0000 7f0952ffd700 1 -- 172.21.15.12:0/4172616886 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f094c0ce450 con 0x7f095c105170 2024-05-08T23:47:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.745+0000 7f096181b700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09480a13b0 0x7f09480a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.746+0000 7f096181b700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09480a13b0 0x7f09480a3870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0958009d10 tx=0x7f0958009040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.746+0000 7f096fc39b80 1 Processor -- start 2024-05-08T23:47:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.746+0000 7f096fc39b80 1 -- start start 2024-05-08T23:47:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce4a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce4a720 0x55accce44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce44cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096fc39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acccd9b150 con 0x55accce44890 2024-05-08T23:47:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096fc39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acccd98b10 con 0x55accce4a720 2024-05-08T23:47:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096fc39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acccd94400 con 0x55accce49e00 2024-05-08T23:47:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096201c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce44cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096201c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce44cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52018/0 (socket says 172.21.15.12:52018) 2024-05-08T23:47:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096201c700 1 -- 172.21.15.12:0/1135524154 learned_addr learned my addr 172.21.15.12:0/1135524154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096181b700 1 --2- 172.21.15.12:0/1135524154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce4a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096101a700 1 --2- 172.21.15.12:0/1135524154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce4a720 0x55accce44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096201c700 1 -- 172.21.15.12:0/1135524154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 msgr2=0x55accce4a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096201c700 1 --2- 172.21.15.12:0/1135524154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce4a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096201c700 1 -- 172.21.15.12:0/1135524154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce4a720 msgr2=0x55accce44350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096201c700 1 --2- 172.21.15.12:0/1135524154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce4a720 0x55accce44350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096201c700 1 -- 172.21.15.12:0/1135524154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acccc153b0 con 0x55accce44890 2024-05-08T23:47:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096181b700 1 --2- 172.21.15.12:0/1135524154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce4a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.747+0000 7f096201c700 1 --2- 172.21.15.12:0/1135524154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce44cd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0954009b70 tx=0x7f0954009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e249941d10269d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f0947fff700 1 -- 172.21.15.12:0/1135524154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095400eaa0 con 0x55accce44890 2024-05-08T23:47:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f0947fff700 1 -- 172.21.15.12:0/1135524154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095400ec00 con 0x55accce44890 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f0947fff700 1 -- 172.21.15.12:0/1135524154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954010630 con 0x55accce44890 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f096fc39b80 1 -- 172.21.15.12:0/1135524154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 msgr2=0x55accce44cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f096fc39b80 1 --2- 172.21.15.12:0/1135524154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce44cd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0954009b70 tx=0x7f0954009e80 comp rx=0 tx=0).stop 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f096fc39b80 1 -- 172.21.15.12:0/1135524154 shutdown_connections 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f096fc39b80 1 --2- 172.21.15.12:0/1135524154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce44cd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f096fc39b80 1 --2- 172.21.15.12:0/1135524154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce4a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f096fc39b80 1 --2- 172.21.15.12:0/1135524154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce4a720 0x55accce44350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f096fc39b80 1 -- 172.21.15.12:0/1135524154 >> 172.21.15.12:0/1135524154 conn(0x55acccda2210 msgr2=0x55accce3edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.748+0000 7f096fc39b80 1 -- 172.21.15.12:0/1135524154 shutdown_connections 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.749+0000 7f096fc39b80 1 -- 172.21.15.12:0/1135524154 wait complete. 2024-05-08T23:47:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.749+0000 7f096fc39b80 1 -- start start 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.749+0000 7f096fc39b80 1 Processor -- start 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.749+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce44890 0x55accce4da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.749+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce49e00 0x55accce4dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.749+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce4e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.749+0000 7f096fc39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acccd30560 con 0x55accce4a720 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096fc39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acccd30200 con 0x55accce49e00 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096fc39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acccc176a0 con 0x55accce44890 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096fc39b80 10 client.0 _ll_register_callbacks cb 0x55acccd39b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096201c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce4e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096201c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce4e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52030/0 (socket says 172.21.15.12:52030) 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096201c700 1 -- 172.21.15.12:0/951259306 learned_addr learned my addr 172.21.15.12:0/951259306 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096101a700 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce49e00 0x55accce4dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096181b700 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce44890 0x55accce4da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096201c700 1 -- 172.21.15.12:0/951259306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce44890 msgr2=0x55accce4da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096201c700 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce44890 0x55accce4da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096201c700 1 -- 172.21.15.12:0/951259306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce49e00 msgr2=0x55accce4dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096201c700 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce49e00 0x55accce4dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096201c700 1 -- 172.21.15.12:0/951259306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09540097a0 con 0x55accce4a720 2024-05-08T23:47:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f096201c700 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce4e510 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0954000c00 tx=0x7f095400efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcfd5e7ec8cf2b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f09457fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.750+0000 7f09457fa700 1 -- 172.21.15.12:0/951259306 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954020070 con 0x55accce4a720 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f09457fa700 1 -- 172.21.15.12:0/951259306 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0954004240 con 0x55accce4a720 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f096fc39b80 1 -- 172.21.15.12:0/951259306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acccced270 con 0x55accce4a720 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f09457fa700 1 -- 172.21.15.12:0/951259306 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0954004420 con 0x55accce4a720 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f096fc39b80 10 client.15633 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f096fc39b80 1 -- 172.21.15.12:0/951259306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55acccbc1b60 con 0x55accce4a720 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f09457fa700 1 -- 172.21.15.12:0/951259306 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0954032020 con 0x55accce4a720 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f09457fa700 1 -- 172.21.15.12:0/951259306 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f095400c070 con 0x55accce4a720 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f096fc39b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f096fc39b80 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f096fc39b80 20 client.15633 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f096fc39b80 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55accce5e100 0x55accce60520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f096fc39b80 4 client.15633 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f096fc39b80 1 -- 172.21.15.12:0/951259306 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55accce13810 con 0x55accce5e100 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.751+0000 7f096181b700 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55accce5e100 0x55accce60520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.752+0000 7f096181b700 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55accce5e100 0x55accce60520 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9845ce2f3cb24ff8 server_cookie=2e24a6c297ee6e5a in_seq=0 out_seq=0 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.752+0000 7f09457fa700 10 client.15633 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f09457fa700 1 -- 172.21.15.12:0/951259306 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55accce13810 con 0x55accce5e100 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f09457fa700 10 client.15633 handle_command_reply: tid=0 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 client.15633 shutdown 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 -- 172.21.15.12:0/951259306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 msgr2=0x55accce4e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce4e510 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0954000c00 tx=0x7f095400efa0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 -- 172.21.15.12:0/951259306 shutdown_connections 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce4e510 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce44890 0x55accce4da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55accce5e100 0x55accce60520 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 --2- 172.21.15.12:0/951259306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce49e00 0x55accce4dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f09457fa700 0 client.15633 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 -- 172.21.15.12:0/951259306 >> 172.21.15.12:0/951259306 conn(0x55acccda2210 msgr2=0x55accce3e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 -- 172.21.15.12:0/951259306 shutdown_connections 2024-05-08T23:47:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 1 -- 172.21.15.12:0/951259306 wait complete. 2024-05-08T23:47:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.753+0000 7f096fc39b80 20 client.15633 trim_cache size 0 max 16384 2024-05-08T23:47:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:06 smithi204 ceph-mon[47373]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:47:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:06 smithi160 ceph-mon[46152]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:47:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.771+0000 7f096fc39b80 1 Processor -- start 2024-05-08T23:47:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.771+0000 7f096fc39b80 1 -- start start 2024-05-08T23:47:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce44890 0x55accce44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce3ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce55160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096fc39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accce4a560 con 0x55accce4a720 2024-05-08T23:47:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096fc39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accce65730 con 0x55accce44890 2024-05-08T23:47:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096fc39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accce4bfe0 con 0x55accce49e00 2024-05-08T23:47:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096201c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce55160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096201c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce55160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52040/0 (socket says 172.21.15.12:52040) 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096201c700 1 -- 172.21.15.12:0/3962323115 learned_addr learned my addr 172.21.15.12:0/3962323115 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096181b700 1 --2- 172.21.15.12:0/3962323115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce44890 0x55accce44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096101a700 1 --2- 172.21.15.12:0/3962323115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce3ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096201c700 1 -- 172.21.15.12:0/3962323115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 msgr2=0x55accce3ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096201c700 1 --2- 172.21.15.12:0/3962323115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce3ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096201c700 1 -- 172.21.15.12:0/3962323115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce44890 msgr2=0x55accce44350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096201c700 1 --2- 172.21.15.12:0/3962323115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce44890 0x55accce44350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096201c700 1 -- 172.21.15.12:0/3962323115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acccd5a560 con 0x55accce4a720 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096101a700 1 --2- 172.21.15.12:0/3962323115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce3ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f096201c700 1 --2- 172.21.15.12:0/3962323115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce55160 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0954011b20 tx=0x7f0954005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d013eb72599a67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f0950da4700 1 -- 172.21.15.12:0/3962323115 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954020040 con 0x55accce4a720 2024-05-08T23:47:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f0950da4700 1 -- 172.21.15.12:0/3962323115 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0954032030 con 0x55accce4a720 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.772+0000 7f0950da4700 1 -- 172.21.15.12:0/3962323115 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954004670 con 0x55accce4a720 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 -- 172.21.15.12:0/3962323115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 msgr2=0x55accce55160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 --2- 172.21.15.12:0/3962323115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce55160 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0954011b20 tx=0x7f0954005f60 comp rx=0 tx=0).stop 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 -- 172.21.15.12:0/3962323115 shutdown_connections 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 --2- 172.21.15.12:0/3962323115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce4a720 0x55accce55160 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 --2- 172.21.15.12:0/3962323115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce44890 0x55accce44350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 --2- 172.21.15.12:0/3962323115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce3ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 -- 172.21.15.12:0/3962323115 >> 172.21.15.12:0/3962323115 conn(0x55acccda2210 msgr2=0x55accce44c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 -- 172.21.15.12:0/3962323115 shutdown_connections 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 -- 172.21.15.12:0/3962323115 wait complete. 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 -- start start 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.773+0000 7f096fc39b80 1 Processor -- start 2024-05-08T23:47:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce55170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce556b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096fc39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce4a720 0x55accce459a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096fc39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accce45ee0 con 0x55accce44890 2024-05-08T23:47:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096fc39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accce46050 con 0x55accce4a720 2024-05-08T23:47:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096fc39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55accce46210 con 0x55accce49e00 2024-05-08T23:47:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096fc39b80 10 client.0 _ll_register_callbacks cb 0x55acccd39b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096181b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce55170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096181b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce55170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52048/0 (socket says 172.21.15.12:52048) 2024-05-08T23:47:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096181b700 1 -- 172.21.15.12:0/3090641402 learned_addr learned my addr 172.21.15.12:0/3090641402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096201c700 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce4a720 0x55accce459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096101a700 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096181b700 1 -- 172.21.15.12:0/3090641402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 msgr2=0x55accce556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096181b700 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096181b700 1 -- 172.21.15.12:0/3090641402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce4a720 msgr2=0x55accce459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096181b700 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce4a720 0x55accce459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096181b700 1 -- 172.21.15.12:0/3090641402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acccbc1b60 con 0x55accce44890 2024-05-08T23:47:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.774+0000 7f096101a700 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f096181b700 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce55170 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0958009120 tx=0x7f0958015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef987a144898cab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f09457fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f09457fa700 1 -- 172.21.15.12:0/3090641402 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09580246e0 con 0x55accce44890 2024-05-08T23:47:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f09457fa700 1 -- 172.21.15.12:0/3090641402 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0958024840 con 0x55accce44890 2024-05-08T23:47:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f096fc39b80 1 -- 172.21.15.12:0/3090641402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f095400ec00 con 0x55accce44890 2024-05-08T23:47:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f09457fa700 1 -- 172.21.15.12:0/3090641402 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0958024a00 con 0x55accce44890 2024-05-08T23:47:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f096fc39b80 10 client.15639 fetch_fsmap learned FSMap version 24 2024-05-08T23:47:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f096fc39b80 1 -- 172.21.15.12:0/3090641402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55acccc19cd0 con 0x55accce44890 2024-05-08T23:47:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f09457fa700 1 -- 172.21.15.12:0/3090641402 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095400ec00 con 0x55accce44890 2024-05-08T23:47:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f09457fa700 1 -- 172.21.15.12:0/3090641402 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0958031c20 con 0x55accce44890 2024-05-08T23:47:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f096fc39b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:47:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f096fc39b80 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.775+0000 7f096fc39b80 20 client.15639 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.776+0000 7f096fc39b80 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55accceb54e0 0x55accceb7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.776+0000 7f096fc39b80 4 client.15639 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:47:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.776+0000 7f096fc39b80 1 -- 172.21.15.12:0/3090641402 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55acccd8cec0 con 0x55accceb54e0 2024-05-08T23:47:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.776+0000 7f096101a700 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55accceb54e0 0x55accceb7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.776+0000 7f09457fa700 10 client.15639 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.776+0000 7f096101a700 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55accceb54e0 0x55accceb7900 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4693f3ee1c989bca server_cookie=a5bae97b0d8425be in_seq=0 out_seq=0 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f09457fa700 1 -- 172.21.15.12:0/3090641402 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55acccd8cec0 con 0x55accceb54e0 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f09457fa700 10 client.15639 handle_command_reply: tid=0 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 client.15639 shutdown 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 -- 172.21.15.12:0/3090641402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 msgr2=0x55accce55170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce55170 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0958009120 tx=0x7f0958015000 comp rx=0 tx=0).stop 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 -- 172.21.15.12:0/3090641402 shutdown_connections 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55accceb54e0 0x55accceb7900 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55accce4a720 0x55accce459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f09457fa700 0 client.15639 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55accce44890 0x55accce55170 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 --2- 172.21.15.12:0/3090641402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55accce49e00 0x55accce556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 -- 172.21.15.12:0/3090641402 >> 172.21.15.12:0/3090641402 conn(0x55acccda2210 msgr2=0x55accce4c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 -- 172.21.15.12:0/3090641402 shutdown_connections 2024-05-08T23:47:06.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 1 -- 172.21.15.12:0/3090641402 wait complete. 2024-05-08T23:47:06.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f096fc39b80 20 client.15639 trim_cache size 0 max 16384 2024-05-08T23:47:06.781 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:47:06.781 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:47:06.781 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:47:06.782 INFO:teuthology.orchestra.run.smithi012.stdout: "f3398a9b-9ba6-4294-bef0-2bebd500579b": { 2024-05-08T23:47:06.782 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:47:06.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "f3398a9b-9ba6-4294-bef0-2bebd500579b", 2024-05-08T23:47:06.782 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:47:06.782 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:47:06.782 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:47:06.782 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:47:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f0963a7f700 1 -- 172.21.15.12:0/4172616886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09480a13b0 msgr2=0x7f09480a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f0963a7f700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09480a13b0 0x7f09480a3870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0958009d10 tx=0x7f0958009040 comp rx=0 tx=0).stop 2024-05-08T23:47:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f0963a7f700 1 -- 172.21.15.12:0/4172616886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f095c105170 msgr2=0x7f095c102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.777+0000 7f0963a7f700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f095c105170 0x7f095c102020 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f094c000c00 tx=0x7f094c009580 comp rx=0 tx=0).stop 2024-05-08T23:47:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.778+0000 7f0963a7f700 1 -- 172.21.15.12:0/4172616886 shutdown_connections 2024-05-08T23:47:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.778+0000 7f0963a7f700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09480a13b0 0x7f09480a3870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.778+0000 7f0963a7f700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f095c102560 0x7f095c1a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.778+0000 7f0963a7f700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095c100050 0x7f095c101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.778+0000 7f0963a7f700 1 --2- 172.21.15.12:0/4172616886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f095c105170 0x7f095c102020 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.778+0000 7f0963a7f700 1 -- 172.21.15.12:0/4172616886 >> 172.21.15.12:0/4172616886 conn(0x7f095c076fa0 msgr2=0x7f095c103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.778+0000 7f0963a7f700 1 -- 172.21.15.12:0/4172616886 shutdown_connections 2024-05-08T23:47:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:06.778+0000 7f0963a7f700 1 -- 172.21.15.12:0/4172616886 wait complete. 2024-05-08T23:47:06.785 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:06 smithi012 ceph-mon[42503]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:47:06.787 INFO:tasks.cephfs.filesystem:scrub status for tag:f3398a9b-9ba6-4294-bef0-2bebd500579b - {'path': '/', 'tag': 'f3398a9b-9ba6-4294-bef0-2bebd500579b', 'options': 'recursive,force'} 2024-05-08T23:47:07.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:07 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:47:07.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:07 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:07.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:07 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:47:07.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:07 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:07.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:07 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:47:07.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:07 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:07.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:07 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:47:07.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:07 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:07.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:07 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:47:07.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:07 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:07.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:07 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:47:07.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:07 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:08.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:08 smithi204 ceph-mon[47373]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-08T23:47:08.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:08 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:47:08.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:08 smithi160 ceph-mon[46152]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-08T23:47:08.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:08 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:47:08.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:08 smithi012 ceph-mon[42503]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-08T23:47:08.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:08 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:47:10.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:10 smithi204 ceph-mon[47373]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-08T23:47:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:10 smithi160 ceph-mon[46152]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-08T23:47:10.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:10 smithi012 ceph-mon[42503]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-08T23:47:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:12 smithi204 ceph-mon[47373]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-08T23:47:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:12 smithi160 ceph-mon[46152]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-08T23:47:12.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:12 smithi012 ceph-mon[42503]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-08T23:47:14.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:14 smithi204 ceph-mon[47373]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-08T23:47:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:14 smithi160 ceph-mon[46152]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-08T23:47:14.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:14 smithi012 ceph-mon[42503]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-08T23:47:15.294 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:15 smithi204 systemd[1]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2024-05-08T23:47:15.294 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:15 smithi204 systemd[1]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-05-08T23:47:15.294 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:15 smithi204 systemd[1]: Stopped Ceph mds.c for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:47:15.294 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:15 smithi204 systemd[1]: Starting Ceph mds.c for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-08T23:47:15.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:15 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-08T23:47:15.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:15 smithi204 ceph-mon[47373]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T23:47:15.764 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:15 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/2305522697 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558457aec1a0 con 0x558457ac2800 2024-05-08T23:47:15.764 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:15 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-08T23:47:15.764 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:15 smithi204 bash[87985]: 1394c671fc2ff907ee3e732fafdcd76f1acdb6da460244446564edac411afd9d 2024-05-08T23:47:15.764 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:47:15 smithi204 systemd[1]: Started Ceph mds.c for 00f8fb28-0d94-11ef-bc97-c7b262605968. 2024-05-08T23:47:15.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:15 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-08T23:47:15.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:15 smithi160 ceph-mon[46152]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T23:47:15.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:15 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-08T23:47:15.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:15 smithi012 ceph-mon[42503]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T23:47:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.478+0000 7ff8c5504700 1 -- 172.21.15.12:0/3398538743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c01009c0 msgr2=0x7ff8c0104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.478+0000 7ff8c5504700 1 --2- 172.21.15.12:0/3398538743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c01009c0 0x7ff8c0104a00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009990 tx=0x7ff8b4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.478+0000 7ff8c5504700 1 -- 172.21.15.12:0/3398538743 shutdown_connections 2024-05-08T23:47:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.478+0000 7ff8c5504700 1 --2- 172.21.15.12:0/3398538743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0105130 0x7ff8c010ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.478+0000 7ff8c5504700 1 --2- 172.21.15.12:0/3398538743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c0100010 0x7ff8c01003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.478+0000 7ff8c5504700 1 --2- 172.21.15.12:0/3398538743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c01009c0 0x7ff8c0104a00 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.478+0000 7ff8c5504700 1 -- 172.21.15.12:0/3398538743 >> 172.21.15.12:0/3398538743 conn(0x7ff8c00fb870 msgr2=0x7ff8c00fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.478+0000 7ff8c5504700 1 -- 172.21.15.12:0/3398538743 shutdown_connections 2024-05-08T23:47:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.478+0000 7ff8c5504700 1 -- 172.21.15.12:0/3398538743 wait complete. 2024-05-08T23:47:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.479+0000 7ff8c5504700 1 Processor -- start 2024-05-08T23:47:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.479+0000 7ff8c5504700 1 -- start start 2024-05-08T23:47:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8c5504700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0100010 0x7ff8c010d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8c5504700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c0105130 0x7ff8c010db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8c5504700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c010e070 0x7ff8c01a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8c5504700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01a5bc0 con 0x7ff8c0105130 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8c5504700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01a5d30 con 0x7ff8c0100010 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8c5504700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01a5ef0 con 0x7ff8c010e070 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8be7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c0105130 0x7ff8c010db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8be7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c0105130 0x7ff8c010db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40670/0 (socket says 172.21.15.12:40670) 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8be7fc700 1 -- 172.21.15.12:0/2927420279 learned_addr learned my addr 172.21.15.12:0/2927420279 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8beffd700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0100010 0x7ff8c010d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8bf7fe700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c010e070 0x7ff8c01a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8be7fc700 1 -- 172.21.15.12:0/2927420279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c010e070 msgr2=0x7ff8c01a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c010e070 0x7ff8c01a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8be7fc700 1 -- 172.21.15.12:0/2927420279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0100010 msgr2=0x7ff8c010d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0100010 0x7ff8c010d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8be7fc700 1 -- 172.21.15.12:0/2927420279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b40095f0 con 0x7ff8c0105130 2024-05-08T23:47:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c0105130 0x7ff8c010db30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4000c00 tx=0x7ff8b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.480+0000 7ff8a7fff700 1 -- 172.21.15.12:0/2927420279 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b401d070 con 0x7ff8c0105130 2024-05-08T23:47:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.481+0000 7ff8c5504700 1 -- 172.21.15.12:0/2927420279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c01a6170 con 0x7ff8c0105130 2024-05-08T23:47:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.481+0000 7ff8a7fff700 1 -- 172.21.15.12:0/2927420279 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b4021410 con 0x7ff8c0105130 2024-05-08T23:47:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.481+0000 7ff8a7fff700 1 -- 172.21.15.12:0/2927420279 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4017650 con 0x7ff8c0105130 2024-05-08T23:47:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.481+0000 7ff8c5504700 1 -- 172.21.15.12:0/2927420279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c01a6590 con 0x7ff8c0105130 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.483+0000 7ff8a7fff700 1 -- 172.21.15.12:0/2927420279 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7ff8b4021580 con 0x7ff8c0105130 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.484+0000 7ff8a7fff700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8a80a13b0 0x7ff8a80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.484+0000 7ff8a7fff700 1 -- 172.21.15.12:0/2927420279 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff8b40ce1b0 con 0x7ff8c0105130 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.484+0000 7ff8beffd700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8a80a13b0 0x7ff8a80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.484+0000 7ff8beffd700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8a80a13b0 0x7ff8a80a3870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009c60 tx=0x7ff8b0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8d16beb80 1 Processor -- start 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8d16beb80 1 -- start start 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161bf0950 0x556161bf0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 0x556161beb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161b420f0 con 0x556161bf1270 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161b3fab0 con 0x556161bf0950 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161b3b3a0 con 0x556161beb480 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8be7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8be7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40680/0 (socket says 172.21.15.12:40680) 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8be7fc700 1 -- 172.21.15.12:0/2693395043 learned_addr learned my addr 172.21.15.12:0/2693395043 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8beffd700 1 --2- 172.21.15.12:0/2693395043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161bf0950 0x556161bf0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8bf7fe700 1 --2- 172.21.15.12:0/2693395043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 0x556161beb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8be7fc700 1 -- 172.21.15.12:0/2693395043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 msgr2=0x556161beb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/2693395043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 0x556161beb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8be7fc700 1 -- 172.21.15.12:0/2693395043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161bf0950 msgr2=0x556161bf0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/2693395043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161bf0950 0x556161bf0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.485+0000 7ff8be7fc700 1 -- 172.21.15.12:0/2693395043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561619bc350 con 0x556161bf1270 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.486+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/2693395043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beaf40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40cbf40 tx=0x7ff8b40975a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498cbc8d472bf9cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.486+0000 7ff8a57fa700 1 -- 172.21.15.12:0/2693395043 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40ce030 con 0x556161bf1270 2024-05-08T23:47:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.486+0000 7ff8a57fa700 1 -- 172.21.15.12:0/2693395043 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b409fe90 con 0x556161bf1270 2024-05-08T23:47:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.486+0000 7ff8a57fa700 1 -- 172.21.15.12:0/2693395043 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40a3530 con 0x556161bf1270 2024-05-08T23:47:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 -- 172.21.15.12:0/2693395043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 msgr2=0x556161beaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/2693395043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beaf40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40cbf40 tx=0x7ff8b40975a0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 -- 172.21.15.12:0/2693395043 shutdown_connections 2024-05-08T23:47:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/2693395043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 0x556161beb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/2693395043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161bf0950 0x556161bf0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/2693395043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beaf40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 -- 172.21.15.12:0/2693395043 >> 172.21.15.12:0/2693395043 conn(0x556161b491b0 msgr2=0x556161be5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 -- 172.21.15.12:0/2693395043 shutdown_connections 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 -- 172.21.15.12:0/2693395043 wait complete. 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 -- start start 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.487+0000 7ff8d16beb80 1 Processor -- start 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 0x556161bf71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161bf0950 0x556161bf44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161bf4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161ad7500 con 0x556161bf1270 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161ad71a0 con 0x556161bf0950 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561619be640 con 0x556161beb480 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8d16beb80 10 client.0 _ll_register_callbacks cb 0x556161ae0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161bf4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161bf4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40694/0 (socket says 172.21.15.12:40694) 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8bf7fe700 1 -- 172.21.15.12:0/4193738846 learned_addr learned my addr 172.21.15.12:0/4193738846 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161bf0950 0x556161bf44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8beffd700 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 0x556161bf71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8bf7fe700 1 -- 172.21.15.12:0/4193738846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 msgr2=0x556161bf71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8bf7fe700 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 0x556161bf71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8bf7fe700 1 -- 172.21.15.12:0/4193738846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161bf0950 msgr2=0x556161bf44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8bf7fe700 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161bf0950 0x556161bf44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8bf7fe700 1 -- 172.21.15.12:0/4193738846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c01a6590 con 0x556161bf1270 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8beffd700 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 0x556161bf71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8bf7fe700 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161bf4bf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8004070 tx=0x7ff8b8004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e39a406bb9a6646 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.488+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/4193738846 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80079d0 con 0x556161bf1270 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/4193738846 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff8b8007b30 con 0x556161bf1270 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8d16beb80 1 -- 172.21.15.12:0/4193738846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556161a94210 con 0x556161bf1270 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/4193738846 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b8007cf0 con 0x556161bf1270 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8d16beb80 10 client.15651 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8d16beb80 1 -- 172.21.15.12:0/4193738846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556161968b00 con 0x556161bf1270 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/4193738846 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556161a94210 con 0x556161bf1270 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/4193738846 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff8b8014040 con 0x556161bf1270 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8d16beb80 10 client.15651 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8d16beb80 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8d16beb80 20 client.15651 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556161c04990 0x556161c06db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8d16beb80 4 client.15651 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8d16beb80 1 -- 172.21.15.12:0/4193738846 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556161bba7b0 con 0x556161c04990 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.489+0000 7ff8beffd700 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556161c04990 0x556161c06db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.490+0000 7ff8a2ff5700 10 client.15651 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.490+0000 7ff8beffd700 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556161c04990 0x556161c06db0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e65e8034f37e40 server_cookie=575440ca5b83ba36 in_seq=0 out_seq=0 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/4193738846 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556161bba7b0 con 0x556161c04990 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8a2ff5700 10 client.15651 handle_command_reply: tid=0 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 client.15651 shutdown 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 -- 172.21.15.12:0/4193738846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 msgr2=0x556161bf4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161bf4bf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8004070 tx=0x7ff8b8004380 comp rx=0 tx=0).stop 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 -- 172.21.15.12:0/4193738846 shutdown_connections 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161bf4bf0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161beb480 0x556161bf71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556161c04990 0x556161c06db0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/4193738846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161bf0950 0x556161bf44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8a2ff5700 0 client.15651 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 -- 172.21.15.12:0/4193738846 >> 172.21.15.12:0/4193738846 conn(0x556161b491b0 msgr2=0x556161bef4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 -- 172.21.15.12:0/4193738846 shutdown_connections 2024-05-08T23:47:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 1 -- 172.21.15.12:0/4193738846 wait complete. 2024-05-08T23:47:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.491+0000 7ff8d16beb80 20 client.15651 trim_cache size 0 max 16384 2024-05-08T23:47:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8d16beb80 1 Processor -- start 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8d16beb80 1 -- start start 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161beb480 0x556161be5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf0950 0x556161bf4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161bf4ff0 con 0x556161bf1270 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161bebda0 con 0x556161beb480 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161c07410 con 0x556161bf0950 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40708/0 (socket says 172.21.15.12:40708) 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.507+0000 7ff8bf7fe700 1 -- 172.21.15.12:0/3893853570 learned_addr learned my addr 172.21.15.12:0/3893853570 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/3893853570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf0950 0x556161bf4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8beffd700 1 --2- 172.21.15.12:0/3893853570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161beb480 0x556161be5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8bf7fe700 1 -- 172.21.15.12:0/3893853570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf0950 msgr2=0x556161bf4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8bf7fe700 1 --2- 172.21.15.12:0/3893853570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf0950 0x556161bf4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8bf7fe700 1 -- 172.21.15.12:0/3893853570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161beb480 msgr2=0x556161be5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8bf7fe700 1 --2- 172.21.15.12:0/3893853570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161beb480 0x556161be5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8bf7fe700 1 -- 172.21.15.12:0/3893853570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556161b01500 con 0x556161bf1270 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/3893853570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf0950 0x556161bf4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8bf7fe700 1 --2- 172.21.15.12:0/3893853570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beb860 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009600 tx=0x7ff8b8009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f96698d99b0d24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8a5ffb700 1 -- 172.21.15.12:0/3893853570 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8004bf0 con 0x556161bf1270 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8a5ffb700 1 -- 172.21.15.12:0/3893853570 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b800f040 con 0x556161bf1270 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.508+0000 7ff8a5ffb700 1 -- 172.21.15.12:0/3893853570 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8007ac0 con 0x556161bf1270 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 -- 172.21.15.12:0/3893853570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 msgr2=0x556161beb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/3893853570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beb860 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009600 tx=0x7ff8b8009630 comp rx=0 tx=0).stop 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 -- 172.21.15.12:0/3893853570 shutdown_connections 2024-05-08T23:47:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/3893853570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf1270 0x556161beb860 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/3893853570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161beb480 0x556161be5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/3893853570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf0950 0x556161bf4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 -- 172.21.15.12:0/3893853570 >> 172.21.15.12:0/3893853570 conn(0x556161b491b0 msgr2=0x556161be9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 -- 172.21.15.12:0/3893853570 shutdown_connections 2024-05-08T23:47:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 -- 172.21.15.12:0/3893853570 wait complete. 2024-05-08T23:47:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 -- start start 2024-05-08T23:47:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.509+0000 7ff8d16beb80 1 Processor -- start 2024-05-08T23:47:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161beb480 0x556161bf7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf0950 0x556161bf7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8d16beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf1270 0x556161bf83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161c59f70 con 0x556161bf0950 2024-05-08T23:47:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161c5a0e0 con 0x556161beb480 2024-05-08T23:47:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8d16beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161c5a2a0 con 0x556161bf1270 2024-05-08T23:47:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8d16beb80 10 client.0 _ll_register_callbacks cb 0x556161ae0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8be7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf0950 0x556161bf7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8be7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf0950 0x556161bf7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40710/0 (socket says 172.21.15.12:40710) 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8be7fc700 1 -- 172.21.15.12:0/585976352 learned_addr learned my addr 172.21.15.12:0/585976352 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8beffd700 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161beb480 0x556161bf7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8bf7fe700 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf1270 0x556161bf83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8be7fc700 1 -- 172.21.15.12:0/585976352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf1270 msgr2=0x556161bf83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf1270 0x556161bf83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8be7fc700 1 -- 172.21.15.12:0/585976352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161beb480 msgr2=0x556161bf7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161beb480 0x556161bf7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8be7fc700 1 -- 172.21.15.12:0/585976352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556161968b00 con 0x556161bf0950 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8be7fc700 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf0950 0x556161bf7e60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff8b409ae40 tx=0x7ff8b409b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab03b8f056b943a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.510+0000 7ff8a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/585976352 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b409aaa0 con 0x556161bf0950 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/585976352 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff8b409b840 con 0x556161bf0950 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8d16beb80 1 -- 172.21.15.12:0/585976352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8b409fe90 con 0x556161bf0950 2024-05-08T23:47:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/585976352 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b40b9470 con 0x556161bf0950 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8d16beb80 10 client.15657 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8d16beb80 1 -- 172.21.15.12:0/585976352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5561619c0c70 con 0x556161bf0950 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/585976352 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b409fe90 con 0x556161bf0950 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/585976352 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff8b409a630 con 0x556161bf0950 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8d16beb80 10 client.15657 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8d16beb80 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8d16beb80 20 client.15657 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556161c5b6e0 0x556161c5dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8d16beb80 4 client.15657 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8d16beb80 1 -- 172.21.15.12:0/585976352 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556161b33e60 con 0x556161c5b6e0 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.511+0000 7ff8beffd700 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556161c5b6e0 0x556161c5dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.512+0000 7ff8a2ff5700 10 client.15657 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.512+0000 7ff8beffd700 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556161c5b6e0 0x556161c5dad0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe33ece40d49806 server_cookie=26205453107f8366 in_seq=0 out_seq=0 2024-05-08T23:47:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8a2ff5700 1 -- 172.21.15.12:0/585976352 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556161b33e60 con 0x556161c5b6e0 2024-05-08T23:47:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8a2ff5700 10 client.15657 handle_command_reply: tid=0 2024-05-08T23:47:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8d16beb80 1 client.15657 shutdown 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8d16beb80 1 -- 172.21.15.12:0/585976352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf0950 msgr2=0x556161bf7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf0950 0x556161bf7e60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff8b409ae40 tx=0x7ff8b409b010 comp rx=0 tx=0).stop 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8d16beb80 1 -- 172.21.15.12:0/585976352 shutdown_connections 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556161bf1270 0x556161bf83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556161c5b6e0 0x556161c5dad0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556161beb480 0x556161bf7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8a2ff5700 0 client.15657 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8d16beb80 1 --2- 172.21.15.12:0/585976352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556161bf0950 0x556161bf7e60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.519+0000 7ff8d16beb80 1 -- 172.21.15.12:0/585976352 >> 172.21.15.12:0/585976352 conn(0x556161b491b0 msgr2=0x556161bf2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8d16beb80 1 -- 172.21.15.12:0/585976352 shutdown_connections 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8d16beb80 1 -- 172.21.15.12:0/585976352 wait complete. 2024-05-08T23:47:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8d16beb80 20 client.15657 trim_cache size 0 max 16384 2024-05-08T23:47:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:47:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:47:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:47:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8c5504700 1 -- 172.21.15.12:0/2927420279 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8a80a13b0 msgr2=0x7ff8a80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8c5504700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8a80a13b0 0x7ff8a80a3870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009c60 tx=0x7ff8b0009260 comp rx=0 tx=0).stop 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8c5504700 1 -- 172.21.15.12:0/2927420279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c0105130 msgr2=0x7ff8c010db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8c5504700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c0105130 0x7ff8c010db30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4000c00 tx=0x7ff8b4004730 comp rx=0 tx=0).stop 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8c5504700 1 -- 172.21.15.12:0/2927420279 shutdown_connections 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8c5504700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8a80a13b0 0x7ff8a80a3870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8c5504700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c010e070 0x7ff8c01a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8c5504700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0100010 0x7ff8c010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8c5504700 1 --2- 172.21.15.12:0/2927420279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c0105130 0x7ff8c010db30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.520+0000 7ff8c5504700 1 -- 172.21.15.12:0/2927420279 >> 172.21.15.12:0/2927420279 conn(0x7ff8c00fb870 msgr2=0x7ff8c00fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.521+0000 7ff8c5504700 1 -- 172.21.15.12:0/2927420279 shutdown_connections 2024-05-08T23:47:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:16.521+0000 7ff8c5504700 1 -- 172.21.15.12:0/2927420279 wait complete. 2024-05-08T23:47:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:16 smithi204 ceph-mon[47373]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-08T23:47:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:16 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/3829900343,v1:172.21.15.204:6837/3829900343] up:boot 2024-05-08T23:47:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:16 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:47:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:16 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:47:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:16 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:47:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:16 smithi160 ceph-mon[46152]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-08T23:47:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:16 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/3829900343,v1:172.21.15.204:6837/3829900343] up:boot 2024-05-08T23:47:16.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:16 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:47:16.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:16 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:47:16.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:16 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:47:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:16 smithi012 ceph-mon[42503]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-08T23:47:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:16 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/3829900343,v1:172.21.15.204:6837/3829900343] up:boot 2024-05-08T23:47:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:16 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:47:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:16 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:47:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:16 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:47:17.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:17 smithi204 ceph-mon[47373]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T23:47:17.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:17 smithi160 ceph-mon[46152]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T23:47:17.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:17 smithi012 ceph-mon[42503]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T23:47:20.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:20 smithi012 ceph-mon[42503]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T23:47:20.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:20 smithi204 ceph-mon[47373]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T23:47:20.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:20 smithi160 ceph-mon[46152]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T23:47:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:22 smithi012 ceph-mon[42503]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T23:47:22.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:22 smithi204 ceph-mon[47373]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T23:47:22.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:22 smithi160 ceph-mon[46152]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T23:47:24.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:24 smithi012 ceph-mon[42503]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-08T23:47:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:24 smithi204 ceph-mon[47373]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-08T23:47:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:24 smithi160 ceph-mon[46152]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-08T23:47:26.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:26 smithi012 ceph-mon[42503]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-08T23:47:26.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:26 smithi204 ceph-mon[47373]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-08T23:47:26.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:26 smithi160 ceph-mon[46152]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-08T23:47:28.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:28 smithi012 ceph-mon[42503]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-05-08T23:47:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:28 smithi204 ceph-mon[47373]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-05-08T23:47:28.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:28 smithi160 ceph-mon[46152]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-05-08T23:47:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.689+0000 7fbec4ab2700 1 -- 172.21.15.12:0/2671240659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbec0100280 msgr2=0x7fbec0100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.689+0000 7fbec4ab2700 1 --2- 172.21.15.12:0/2671240659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbec0100280 0x7fbec0100700 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009990 tx=0x7fbeb4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.689+0000 7fbec4ab2700 1 -- 172.21.15.12:0/2671240659 shutdown_connections 2024-05-08T23:47:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.689+0000 7fbec4ab2700 1 --2- 172.21.15.12:0/2671240659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec0100c40 0x7fbec010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.689+0000 7fbec4ab2700 1 --2- 172.21.15.12:0/2671240659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbec0105f20 0x7fbec0106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.689+0000 7fbec4ab2700 1 --2- 172.21.15.12:0/2671240659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbec0100280 0x7fbec0100700 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.689+0000 7fbec4ab2700 1 -- 172.21.15.12:0/2671240659 >> 172.21.15.12:0/2671240659 conn(0x7fbec0076bc0 msgr2=0x7fbec0076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.689+0000 7fbec4ab2700 1 -- 172.21.15.12:0/2671240659 shutdown_connections 2024-05-08T23:47:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.689+0000 7fbec4ab2700 1 -- 172.21.15.12:0/2671240659 wait complete. 2024-05-08T23:47:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.690+0000 7fbec4ab2700 1 Processor -- start 2024-05-08T23:47:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.690+0000 7fbec4ab2700 1 -- start start 2024-05-08T23:47:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.690+0000 7fbec4ab2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbec0100280 0x7fbec01972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.690+0000 7fbec4ab2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbec0100c40 0x7fbec0197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.690+0000 7fbec4ab2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec0105f20 0x7fbec019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbec4ab2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec019c1e0 con 0x7fbec0100c40 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbec4ab2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec019c350 con 0x7fbec0100280 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbec4ab2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec019c510 con 0x7fbec0105f20 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec0105f20 0x7fbec019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebe59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbec0100280 0x7fbec01972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec0105f20 0x7fbec019bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:55846/0 (socket says 172.21.15.12:55846) 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebed9d700 1 -- 172.21.15.12:0/532822190 learned_addr learned my addr 172.21.15.12:0/532822190 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebed9d700 1 -- 172.21.15.12:0/532822190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbec0100280 msgr2=0x7fbec01972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbec0100c40 0x7fbec0197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebed9d700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbec0100280 0x7fbec01972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebed9d700 1 -- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbec0100c40 msgr2=0x7fbec0197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebed9d700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbec0100c40 0x7fbec0197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebed9d700 1 -- 172.21.15.12:0/532822190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb40095f0 con 0x7fbec0105f20 2024-05-08T23:47:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebe59c700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbec0100280 0x7fbec01972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbec0100c40 0x7fbec0197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:47:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.691+0000 7fbebed9d700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec0105f20 0x7fbec019bba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800ef30 tx=0x7fbeb800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.692+0000 7fbeaf7fe700 1 -- 172.21.15.12:0/532822190 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8018070 con 0x7fbec0105f20 2024-05-08T23:47:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.692+0000 7fbeaf7fe700 1 -- 172.21.15.12:0/532822190 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb800f040 con 0x7fbec0105f20 2024-05-08T23:47:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.692+0000 7fbeaf7fe700 1 -- 172.21.15.12:0/532822190 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800cdb0 con 0x7fbec0105f20 2024-05-08T23:47:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.692+0000 7fbec4ab2700 1 -- 172.21.15.12:0/532822190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec019c790 con 0x7fbec0105f20 2024-05-08T23:47:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.692+0000 7fbec4ab2700 1 -- 172.21.15.12:0/532822190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec010d490 con 0x7fbec0105f20 2024-05-08T23:47:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.695+0000 7fbeaf7fe700 1 -- 172.21.15.12:0/532822190 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fbeb8003a00 con 0x7fbec0105f20 2024-05-08T23:47:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.695+0000 7fbeaf7fe700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbea80a1430 0x7fbea80a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.696+0000 7fbeaf7fe700 1 -- 172.21.15.12:0/532822190 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fbeb80ccb00 con 0x7fbec0105f20 2024-05-08T23:47:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.696+0000 7fbebe59c700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbea80a1430 0x7fbea80a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.696+0000 7fbebe59c700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbea80a1430 0x7fbea80a38f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0011490 tx=0x7fbeb0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbed0c6cb80 1 Processor -- start 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbed0c6cb80 1 -- start start 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d1dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1e270 0x558004d17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d18480 0x558004d188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004c6f0f0 con 0x558004d1e270 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004c6cab0 con 0x558004d1d950 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004c683a0 con 0x558004d18480 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbebe59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d1dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbebe59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d1dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34502/0 (socket says 172.21.15.12:34502) 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.697+0000 7fbebe59c700 1 -- 172.21.15.12:0/3514851010 learned_addr learned my addr 172.21.15.12:0/3514851010 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbebed9d700 1 --2- 172.21.15.12:0/3514851010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d18480 0x558004d188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbebe59c700 1 -- 172.21.15.12:0/3514851010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d18480 msgr2=0x558004d188a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/3514851010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1e270 0x558004d17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbebe59c700 1 --2- 172.21.15.12:0/3514851010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d18480 0x558004d188a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbebe59c700 1 -- 172.21.15.12:0/3514851010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1e270 msgr2=0x558004d17f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbebe59c700 1 --2- 172.21.15.12:0/3514851010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1e270 0x558004d17f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbebe59c700 1 -- 172.21.15.12:0/3514851010 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558004ae9350 con 0x558004d1d950 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/3514851010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1e270 0x558004d17f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbebe59c700 1 --2- 172.21.15.12:0/3514851010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d1dd30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00043c0 tx=0x7fbeb0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83a570cc368d518c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbeacff9700 1 -- 172.21.15.12:0/3514851010 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0003d10 con 0x558004d1d950 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbeacff9700 1 -- 172.21.15.12:0/3514851010 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb0003e70 con 0x558004d1d950 2024-05-08T23:47:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.698+0000 7fbeacff9700 1 -- 172.21.15.12:0/3514851010 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb000e7a0 con 0x558004d1d950 2024-05-08T23:47:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.699+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3514851010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 msgr2=0x558004d1dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.699+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3514851010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d1dd30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00043c0 tx=0x7fbeb0014000 comp rx=0 tx=0).stop 2024-05-08T23:47:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.699+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3514851010 shutdown_connections 2024-05-08T23:47:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.699+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3514851010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d18480 0x558004d188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.699+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3514851010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d1dd30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.699+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3514851010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1e270 0x558004d17f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.699+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3514851010 >> 172.21.15.12:0/3514851010 conn(0x558004c761b0 msgr2=0x558004d12840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.699+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3514851010 shutdown_connections 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.699+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3514851010 wait complete. 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.700+0000 7fbed0c6cb80 1 -- start start 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.700+0000 7fbed0c6cb80 1 Processor -- start 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.700+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 0x558004d21510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.700+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d21a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.700+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d21f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.700+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004c04500 con 0x558004d18480 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.700+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004c041a0 con 0x558004d1d950 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.700+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004aeb640 con 0x558004d1e270 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.700+0000 7fbed0c6cb80 10 client.0 _ll_register_callbacks cb 0x558004c0dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.700+0000 7fbebdd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d21a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbebdd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d21a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34510/0 (socket says 172.21.15.12:34510) 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbebdd9b700 1 -- 172.21.15.12:0/3841639513 learned_addr learned my addr 172.21.15.12:0/3841639513 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbebed9d700 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d21f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbebe59c700 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 0x558004d21510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbebdd9b700 1 -- 172.21.15.12:0/3841639513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 msgr2=0x558004d21f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d21f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbebdd9b700 1 -- 172.21.15.12:0/3841639513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 msgr2=0x558004d21510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 0x558004d21510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbebdd9b700 1 -- 172.21.15.12:0/3841639513 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb0017070 con 0x558004d1d950 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d21a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4000c00 tx=0x7fbeb400b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96fd31d5b5b35048 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbea5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbea5ffb700 1 -- 172.21.15.12:0/3841639513 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400f870 con 0x558004d1d950 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.701+0000 7fbea5ffb700 1 -- 172.21.15.12:0/3841639513 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fbeb400f9d0 con 0x558004d1d950 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3841639513 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558004bc1210 con 0x558004d1d950 2024-05-08T23:47:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbea5ffb700 1 -- 172.21.15.12:0/3841639513 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeb400fb90 con 0x558004d1d950 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbed0c6cb80 10 client.25517 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3841639513 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558004a95b00 con 0x558004d1d950 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbea5ffb700 1 -- 172.21.15.12:0/3841639513 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558004bc1210 con 0x558004d1d950 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbea5ffb700 1 -- 172.21.15.12:0/3841639513 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbeb402a070 con 0x558004d1d950 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbed0c6cb80 10 client.25517 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbed0c6cb80 10 client.25517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbed0c6cb80 20 client.25517 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558004d31b00 0x558004d33f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbed0c6cb80 4 client.25517 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3841639513 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558004ce77b0 con 0x558004d31b00 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.702+0000 7fbebe59c700 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558004d31b00 0x558004d33f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.703+0000 7fbebe59c700 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558004d31b00 0x558004d33f20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16d586cbddb3b40b server_cookie=397ceaa20a2d7718 in_seq=0 out_seq=0 2024-05-08T23:47:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.703+0000 7fbea5ffb700 10 client.25517 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbea5ffb700 1 -- 172.21.15.12:0/3841639513 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558004ce77b0 con 0x558004d31b00 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbea5ffb700 10 client.25517 handle_command_reply: tid=0 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 client.25517 shutdown 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3841639513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 msgr2=0x558004d21a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d21a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4000c00 tx=0x7fbeb400b980 comp rx=0 tx=0).stop 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3841639513 shutdown_connections 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d21f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 0x558004d21510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558004d31b00 0x558004d33f20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3841639513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d21a50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3841639513 >> 172.21.15.12:0/3841639513 conn(0x558004c761b0 msgr2=0x558004d16f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbea5ffb700 0 client.25517 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3841639513 shutdown_connections 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3841639513 wait complete. 2024-05-08T23:47:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.704+0000 7fbed0c6cb80 20 client.25517 trim_cache size 0 max 16384 2024-05-08T23:47:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbed0c6cb80 1 Processor -- start 2024-05-08T23:47:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbed0c6cb80 1 -- start start 2024-05-08T23:47:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d18480 0x558004d33ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1d950 0x558004d1f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d12910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004d0d7c0 con 0x558004d1d950 2024-05-08T23:47:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004d0d330 con 0x558004d18480 2024-05-08T23:47:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004d22b80 con 0x558004d1e270 2024-05-08T23:47:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbebed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d12910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbebdd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1d950 0x558004d1f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbebed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d12910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:55864/0 (socket says 172.21.15.12:55864) 2024-05-08T23:47:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbebdd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1d950 0x558004d1f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54228/0 (socket says 172.21.15.12:54228) 2024-05-08T23:47:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbebed9d700 1 -- 172.21.15.12:0/3673742319 learned_addr learned my addr 172.21.15.12:0/3673742319 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.733+0000 7fbebe59c700 1 --2- 172.21.15.12:0/3673742319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d18480 0x558004d33ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.734+0000 7fbebe59c700 1 -- 172.21.15.12:0/3673742319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 msgr2=0x558004d12910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.734+0000 7fbebe59c700 1 --2- 172.21.15.12:0/3673742319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d12910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.734+0000 7fbebe59c700 1 -- 172.21.15.12:0/3673742319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1d950 msgr2=0x558004d1f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.734+0000 7fbebe59c700 1 --2- 172.21.15.12:0/3673742319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1d950 0x558004d1f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.734+0000 7fbebe59c700 1 -- 172.21.15.12:0/3673742319 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558004c2e500 con 0x558004d18480 2024-05-08T23:47:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.734+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/3673742319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1d950 0x558004d1f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.734+0000 7fbebe59c700 1 --2- 172.21.15.12:0/3673742319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d18480 0x558004d33ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x558004d34830 tx=0x7fbeb0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7666342e867ca46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.734+0000 7fbead7fa700 1 -- 172.21.15.12:0/3673742319 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0032610 con 0x558004d18480 2024-05-08T23:47:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.735+0000 7fbead7fa700 1 -- 172.21.15.12:0/3673742319 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb002d040 con 0x558004d18480 2024-05-08T23:47:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.735+0000 7fbead7fa700 1 -- 172.21.15.12:0/3673742319 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0004930 con 0x558004d18480 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3673742319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d18480 msgr2=0x558004d33ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3673742319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d18480 0x558004d33ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x558004d34830 tx=0x7fbeb0017000 comp rx=0 tx=0).stop 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3673742319 shutdown_connections 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3673742319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d12910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3673742319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d18480 0x558004d33ef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/3673742319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d1d950 0x558004d1f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3673742319 >> 172.21.15.12:0/3673742319 conn(0x558004c761b0 msgr2=0x558004d20da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3673742319 shutdown_connections 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/3673742319 wait complete. 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 -- start start 2024-05-08T23:47:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.736+0000 7fbed0c6cb80 1 Processor -- start 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.737+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 0x558004d28d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.737+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d26130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.737+0000 7fbed0c6cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d26670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.737+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004d26bb0 con 0x558004d18480 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.737+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004d26d20 con 0x558004d1d950 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.737+0000 7fbed0c6cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558004d26ee0 con 0x558004d1e270 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.737+0000 7fbed0c6cb80 10 client.0 _ll_register_callbacks cb 0x558004c0dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.737+0000 7fbebe59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 0x558004d28d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.737+0000 7fbebe59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 0x558004d28d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54240/0 (socket says 172.21.15.12:54240) 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.737+0000 7fbebe59c700 1 -- 172.21.15.12:0/2738994507 learned_addr learned my addr 172.21.15.12:0/2738994507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbebed9d700 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d26670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d26130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbebe59c700 1 -- 172.21.15.12:0/2738994507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 msgr2=0x558004d26670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbebe59c700 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d26670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbebe59c700 1 -- 172.21.15.12:0/2738994507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 msgr2=0x558004d26130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbebe59c700 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d26130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbebe59c700 1 -- 172.21.15.12:0/2738994507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558004a95b00 con 0x558004d18480 2024-05-08T23:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbebed9d700 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d26670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbebe59c700 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 0x558004d28d00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005950 tx=0x7fbeb0011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b067f300af6841c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbea5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbea5ffb700 1 -- 172.21.15.12:0/2738994507 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0032620 con 0x558004d18480 2024-05-08T23:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbea5ffb700 1 -- 172.21.15.12:0/2738994507 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fbeb0032780 con 0x558004d18480 2024-05-08T23:47:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.738+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/2738994507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbeb0003e70 con 0x558004d18480 2024-05-08T23:47:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.739+0000 7fbea5ffb700 1 -- 172.21.15.12:0/2738994507 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeb003c420 con 0x558004d18480 2024-05-08T23:47:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.739+0000 7fbed0c6cb80 10 client.15669 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.739+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/2738994507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558004ae9350 con 0x558004d18480 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.739+0000 7fbea5ffb700 1 -- 172.21.15.12:0/2738994507 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb0003e70 con 0x558004d18480 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.740+0000 7fbea5ffb700 1 -- 172.21.15.12:0/2738994507 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbeb0024020 con 0x558004d18480 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.740+0000 7fbed0c6cb80 10 client.15669 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.740+0000 7fbed0c6cb80 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.740+0000 7fbed0c6cb80 20 client.15669 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.740+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558004d87f80 0x558004d8a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.740+0000 7fbed0c6cb80 4 client.15669 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.740+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/2738994507 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558004c60e60 con 0x558004d87f80 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.740+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558004d87f80 0x558004d8a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.741+0000 7fbea5ffb700 10 client.15669 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.741+0000 7fbebdd9b700 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558004d87f80 0x558004d8a370 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286aa1c43f9abb69 server_cookie=2e7d965d343fc0de in_seq=0 out_seq=0 2024-05-08T23:47:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbea5ffb700 1 -- 172.21.15.12:0/2738994507 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558004c60e60 con 0x558004d87f80 2024-05-08T23:47:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbea5ffb700 10 client.15669 handle_command_reply: tid=0 2024-05-08T23:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbed0c6cb80 1 client.15669 shutdown 2024-05-08T23:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/2738994507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 msgr2=0x558004d28d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 0x558004d28d00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005950 tx=0x7fbeb0011c60 comp rx=0 tx=0).stop 2024-05-08T23:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/2738994507 shutdown_connections 2024-05-08T23:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558004d1e270 0x558004d26670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558004d18480 0x558004d28d00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558004d87f80 0x558004d8a370 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbed0c6cb80 1 --2- 172.21.15.12:0/2738994507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558004d1d950 0x558004d26130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.744+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/2738994507 >> 172.21.15.12:0/2738994507 conn(0x558004c761b0 msgr2=0x558004d21df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbea5ffb700 0 client.15669 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/2738994507 shutdown_connections 2024-05-08T23:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbed0c6cb80 1 -- 172.21.15.12:0/2738994507 wait complete. 2024-05-08T23:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbed0c6cb80 20 client.15669 trim_cache size 0 max 16384 2024-05-08T23:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:47:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbec4ab2700 1 -- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbea80a1430 msgr2=0x7fbea80a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbec4ab2700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbea80a1430 0x7fbea80a38f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0011490 tx=0x7fbeb0009260 comp rx=0 tx=0).stop 2024-05-08T23:47:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbec4ab2700 1 -- 172.21.15.12:0/532822190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec0105f20 msgr2=0x7fbec019bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbec4ab2700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec0105f20 0x7fbec019bba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800ef30 tx=0x7fbeb800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbec4ab2700 1 -- 172.21.15.12:0/532822190 shutdown_connections 2024-05-08T23:47:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbec4ab2700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbea80a1430 0x7fbea80a38f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbec4ab2700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec0105f20 0x7fbec019bba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbec4ab2700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbec0100280 0x7fbec01972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbec4ab2700 1 --2- 172.21.15.12:0/532822190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbec0100c40 0x7fbec0197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.745+0000 7fbec4ab2700 1 -- 172.21.15.12:0/532822190 >> 172.21.15.12:0/532822190 conn(0x7fbec0076bc0 msgr2=0x7fbec00feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.746+0000 7fbec4ab2700 1 -- 172.21.15.12:0/532822190 shutdown_connections 2024-05-08T23:47:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:29.746+0000 7fbec4ab2700 1 -- 172.21.15.12:0/532822190 wait complete. 2024-05-08T23:47:30.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:30 smithi012 ceph-mon[42503]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:47:30.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:30 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:47:30.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:30 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:47:30.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:30 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:47:30.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:30 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:47:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:30 smithi204 ceph-mon[47373]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:47:30.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:30 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:47:30.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:30 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:47:30.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:30 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:47:30.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:30 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:47:30.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:30 smithi160 ceph-mon[46152]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:47:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:30 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:47:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:30 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:47:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:30 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:47:30.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:30 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:47:32.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:32 smithi012 ceph-mon[42503]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T23:47:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:32 smithi204 ceph-mon[47373]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T23:47:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:32 smithi160 ceph-mon[46152]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T23:47:34.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:34 smithi012 ceph-mon[42503]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-08T23:47:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:34 smithi204 ceph-mon[47373]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-08T23:47:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:34 smithi160 ceph-mon[46152]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-08T23:47:36.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:36 smithi012 ceph-mon[42503]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:47:36.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:36 smithi204 ceph-mon[47373]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:47:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:36 smithi160 ceph-mon[46152]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:47:36.788 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:47:36.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.942+0000 7f27ef106700 1 -- 172.21.15.12:0/1253336135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27e80682e0 msgr2=0x7f27e80686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:36.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.942+0000 7f27ef106700 1 --2- 172.21.15.12:0/1253336135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27e80682e0 0x7f27e80686c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0099c0 tx=0x7f27dc009cd0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.942+0000 7f27ef106700 1 -- 172.21.15.12:0/1253336135 shutdown_connections 2024-05-08T23:47:36.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.942+0000 7f27ef106700 1 --2- 172.21.15.12:0/1253336135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e8106290 0x7f27e810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.942+0000 7f27ef106700 1 --2- 172.21.15.12:0/1253336135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27e80682e0 0x7f27e80686c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.942+0000 7f27ef106700 1 --2- 172.21.15.12:0/1253336135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e8068c90 0x7f27e8105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.942+0000 7f27ef106700 1 -- 172.21.15.12:0/1253336135 >> 172.21.15.12:0/1253336135 conn(0x7f27e8076f40 msgr2=0x7f27e8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:36.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.942+0000 7f27ef106700 1 -- 172.21.15.12:0/1253336135 shutdown_connections 2024-05-08T23:47:36.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.942+0000 7f27ef106700 1 -- 172.21.15.12:0/1253336135 wait complete. 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.943+0000 7f27ef106700 1 Processor -- start 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.943+0000 7f27ef106700 1 -- start start 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.943+0000 7f27ef106700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e80682e0 0x7f27e8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.943+0000 7f27ef106700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e8106290 0x7f27e8199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.943+0000 7f27ef106700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27e819dec0 0x7f27e819e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.943+0000 7f27ef106700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e819e940 con 0x7f27e819dec0 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.943+0000 7f27ef106700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e819eab0 con 0x7f27e80682e0 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.943+0000 7f27ef106700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e819ec70 con 0x7f27e8106290 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ed6a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27e819dec0 0x7f27e819e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ed6a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27e819dec0 0x7f27e819e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54254/0 (socket says 172.21.15.12:54254) 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ed6a3700 1 -- 172.21.15.12:0/1704445140 learned_addr learned my addr 172.21.15.12:0/1704445140 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27e7fff700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e8106290 0x7f27e8199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ecea2700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e80682e0 0x7f27e8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ed6a3700 1 -- 172.21.15.12:0/1704445140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e8106290 msgr2=0x7f27e8199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ed6a3700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e8106290 0x7f27e8199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ed6a3700 1 -- 172.21.15.12:0/1704445140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e80682e0 msgr2=0x7f27e8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ed6a3700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e80682e0 0x7f27e8199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ed6a3700 1 -- 172.21.15.12:0/1704445140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc0095f0 con 0x7f27e819dec0 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ed6a3700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27e819dec0 0x7f27e819e340 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f27e000ef30 tx=0x7f27e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27e5ffb700 1 -- 172.21.15.12:0/1704445140 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0018070 con 0x7f27e819dec0 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.944+0000 7f27ef106700 1 -- 172.21.15.12:0/1704445140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e81a6170 con 0x7f27e819dec0 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.945+0000 7f27e5ffb700 1 -- 172.21.15.12:0/1704445140 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e000f040 con 0x7f27e819dec0 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.945+0000 7f27e5ffb700 1 -- 172.21.15.12:0/1704445140 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e000cd00 con 0x7f27e819dec0 2024-05-08T23:47:36.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.945+0000 7f27ef106700 1 -- 172.21.15.12:0/1704445140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e81a6650 con 0x7f27e819dec0 2024-05-08T23:47:36.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.948+0000 7f27e5ffb700 1 -- 172.21.15.12:0/1704445140 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f27e0003a00 con 0x7f27e819dec0 2024-05-08T23:47:36.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.948+0000 7f27e5ffb700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27d80a1430 0x7f27d80a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.949+0000 7f27e5ffb700 1 -- 172.21.15.12:0/1704445140 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f27e00cc9c0 con 0x7f27e819dec0 2024-05-08T23:47:36.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.949+0000 7f27ecea2700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27d80a1430 0x7f27d80a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.949+0000 7f27fb2c0b80 1 Processor -- start 2024-05-08T23:47:36.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ecea2700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27d80a1430 0x7f27d80a38f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009fd0 tx=0x7f27dc019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27fb2c0b80 1 -- start start 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d8a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d84350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318cdb150 con 0x558318d84890 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318cd8b10 con 0x558318d8a720 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318cd4400 con 0x558318d89e00 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ecea2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d8a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ed6a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ecea2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d8a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:55898/0 (socket says 172.21.15.12:55898) 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ed6a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54270/0 (socket says 172.21.15.12:54270) 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ecea2700 1 -- 172.21.15.12:0/3170818173 learned_addr learned my addr 172.21.15.12:0/3170818173 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:36.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27e7fff700 1 --2- 172.21.15.12:0/3170818173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d84350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ed6a3700 1 -- 172.21.15.12:0/3170818173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 msgr2=0x558318d8a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ed6a3700 1 --2- 172.21.15.12:0/3170818173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d8a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ed6a3700 1 -- 172.21.15.12:0/3170818173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 msgr2=0x558318d84350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ed6a3700 1 --2- 172.21.15.12:0/3170818173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d84350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ed6a3700 1 -- 172.21.15.12:0/3170818173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558318b553b0 con 0x558318d84890 2024-05-08T23:47:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.950+0000 7f27ecea2700 1 --2- 172.21.15.12:0/3170818173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d8a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.951+0000 7f27ed6a3700 1 --2- 172.21.15.12:0/3170818173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84cd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f27e00c9fd0 tx=0x7f27e0095db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e33dc9984e21a91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.951+0000 7f27d2ffd700 1 -- 172.21.15.12:0/3170818173 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0004ad0 con 0x558318d84890 2024-05-08T23:47:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.951+0000 7f27d2ffd700 1 -- 172.21.15.12:0/3170818173 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e009a050 con 0x558318d84890 2024-05-08T23:47:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.951+0000 7f27d2ffd700 1 -- 172.21.15.12:0/3170818173 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0099bb0 con 0x558318d84890 2024-05-08T23:47:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/3170818173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 msgr2=0x558318d84cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/3170818173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84cd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f27e00c9fd0 tx=0x7f27e0095db0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/3170818173 shutdown_connections 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/3170818173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84cd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/3170818173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d8a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/3170818173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d84350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/3170818173 >> 172.21.15.12:0/3170818173 conn(0x558318ce2210 msgr2=0x558318d7edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/3170818173 shutdown_connections 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/3170818173 wait complete. 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 -- start start 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.952+0000 7f27fb2c0b80 1 Processor -- start 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d8da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d8df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d8e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318c70560 con 0x558318d84890 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318c70200 con 0x558318d8a720 2024-05-08T23:47:36.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318b576a0 con 0x558318d89e00 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27fb2c0b80 10 client.0 _ll_register_callbacks cb 0x558318c79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27ecea2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d8da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27ecea2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d8da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54284/0 (socket says 172.21.15.12:54284) 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27ecea2700 1 -- 172.21.15.12:0/4122896019 learned_addr learned my addr 172.21.15.12:0/4122896019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27ed6a3700 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d8e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27e7fff700 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d8df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27ecea2700 1 -- 172.21.15.12:0/4122896019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 msgr2=0x558318d8df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27ecea2700 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d8df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27ecea2700 1 -- 172.21.15.12:0/4122896019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 msgr2=0x558318d8e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27ecea2700 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d8e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.953+0000 7f27ecea2700 1 -- 172.21.15.12:0/4122896019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e810e650 con 0x558318d84890 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.954+0000 7f27ecea2700 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d8da40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0098e0 tx=0x7f27dc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56164158c5e0cda1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:36.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.954+0000 7f27c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.954+0000 7f27c7fff700 1 -- 172.21.15.12:0/4122896019 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc00bb00 con 0x558318d84890 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.954+0000 7f27c7fff700 1 -- 172.21.15.12:0/4122896019 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f27dc02f640 con 0x558318d84890 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.954+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/4122896019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558318c2d270 con 0x558318d84890 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.954+0000 7f27c7fff700 1 -- 172.21.15.12:0/4122896019 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27dc02f800 con 0x558318d84890 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.954+0000 7f27fb2c0b80 10 client.15681 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.954+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/4122896019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f27e81a6650 con 0x558318d84890 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.955+0000 7f27c7fff700 1 -- 172.21.15.12:0/4122896019 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558318c2d270 con 0x558318d84890 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.955+0000 7f27c7fff700 1 -- 172.21.15.12:0/4122896019 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f27dc023b10 con 0x558318d84890 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.955+0000 7f27fb2c0b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.955+0000 7f27fb2c0b80 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.955+0000 7f27fb2c0b80 20 client.15681 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.955+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558318d9e090 0x558318da04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.955+0000 7f27fb2c0b80 4 client.15681 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.955+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/4122896019 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558318d53810 con 0x558318d9e090 2024-05-08T23:47:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.955+0000 7f27e7fff700 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558318d9e090 0x558318da04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.956+0000 7f27c7fff700 10 client.15681 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.956+0000 7f27e7fff700 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558318d9e090 0x558318da04b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1678f4f26518ce server_cookie=69ce84c643a56802 in_seq=0 out_seq=0 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.956+0000 7f27c7fff700 1 -- 172.21.15.12:0/4122896019 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558318d53810 con 0x558318d9e090 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.956+0000 7f27c7fff700 10 client.15681 handle_command_reply: tid=0 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 client.15681 shutdown 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/4122896019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 msgr2=0x558318d8da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d8da40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0098e0 tx=0x7f27dc009310 comp rx=0 tx=0).stop 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/4122896019 shutdown_connections 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558318d9e090 0x558318da04b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d8e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d8da40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/4122896019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d8df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/4122896019 >> 172.21.15.12:0/4122896019 conn(0x558318ce2210 msgr2=0x558318d7e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27c7fff700 0 client.15681 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/4122896019 shutdown_connections 2024-05-08T23:47:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/4122896019 wait complete. 2024-05-08T23:47:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.957+0000 7f27fb2c0b80 20 client.15681 trim_cache size 0 max 16384 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27fb2c0b80 1 Processor -- start 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27fb2c0b80 1 -- start start 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d7ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318d7bdc0 con 0x558318d84890 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318d89900 con 0x558318d8a720 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318d79800 con 0x558318d89e00 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27ecea2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27ecea2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54296/0 (socket says 172.21.15.12:54296) 2024-05-08T23:47:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27ecea2700 1 -- 172.21.15.12:0/3749186411 learned_addr learned my addr 172.21.15.12:0/3749186411 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.976+0000 7f27e7fff700 1 --2- 172.21.15.12:0/3749186411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.977+0000 7f27ecea2700 1 -- 172.21.15.12:0/3749186411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 msgr2=0x558318d893c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.977+0000 7f27ecea2700 1 --2- 172.21.15.12:0/3749186411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d893c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.977+0000 7f27ecea2700 1 -- 172.21.15.12:0/3749186411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 msgr2=0x558318d7ed20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:47:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.977+0000 7f27ecea2700 1 --2- 172.21.15.12:0/3749186411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d7ed20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.977+0000 7f27ecea2700 1 -- 172.21.15.12:0/3749186411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558318b01b60 con 0x558318d84890 2024-05-08T23:47:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.977+0000 7f27e7fff700 1 --2- 172.21.15.12:0/3749186411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.977+0000 7f27ecea2700 1 --2- 172.21.15.12:0/3749186411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84350 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f27dc005ae0 tx=0x7f27dc00bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53909b592787ca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.977+0000 7f27d37fe700 1 -- 172.21.15.12:0/3749186411 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0177f0 con 0x558318d84890 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.977+0000 7f27d37fe700 1 -- 172.21.15.12:0/3749186411 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27dc01d040 con 0x558318d84890 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.977+0000 7f27d37fe700 1 -- 172.21.15.12:0/3749186411 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc023aa0 con 0x558318d84890 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.978+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/3749186411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 msgr2=0x558318d84350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.978+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/3749186411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84350 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f27dc005ae0 tx=0x7f27dc00bae0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.978+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/3749186411 shutdown_connections 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.978+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/3749186411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d8a720 0x558318d7ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.978+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/3749186411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d84890 0x558318d84350 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.978+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/3749186411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.978+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/3749186411 >> 172.21.15.12:0/3749186411 conn(0x558318ce2210 msgr2=0x558318d8c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.978+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/3749186411 shutdown_connections 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.978+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/3749186411 wait complete. 2024-05-08T23:47:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.979+0000 7f27fb2c0b80 1 -- start start 2024-05-08T23:47:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.979+0000 7f27fb2c0b80 1 Processor -- start 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.979+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d84890 0x558318d95170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.979+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d956b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.979+0000 7f27fb2c0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d8a720 0x558318d85b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.979+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318d861f0 con 0x558318d8a720 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.979+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318d86360 con 0x558318d84890 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.979+0000 7f27fb2c0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558318d86520 con 0x558318d89e00 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.979+0000 7f27fb2c0b80 10 client.0 _ll_register_callbacks cb 0x558318c79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ed6a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d8a720 0x558318d85b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ed6a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d8a720 0x558318d85b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54300/0 (socket says 172.21.15.12:54300) 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ed6a3700 1 -- 172.21.15.12:0/17293032 learned_addr learned my addr 172.21.15.12:0/17293032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ecea2700 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d84890 0x558318d95170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27e7fff700 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ecea2700 1 -- 172.21.15.12:0/17293032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 msgr2=0x558318d956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ecea2700 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d956b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ecea2700 1 -- 172.21.15.12:0/17293032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d8a720 msgr2=0x558318d85b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ecea2700 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d8a720 0x558318d85b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ecea2700 1 -- 172.21.15.12:0/17293032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e81a6650 con 0x558318d84890 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ed6a3700 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d8a720 0x558318d85b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27e7fff700 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27ecea2700 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d84890 0x558318d95170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f27dc017410 tx=0x7f27dc00b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcaf583d44dacf9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.980+0000 7f27c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.981+0000 7f27c7fff700 1 -- 172.21.15.12:0/17293032 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0233f0 con 0x558318d84890 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.981+0000 7f27c7fff700 1 -- 172.21.15.12:0/17293032 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f27dc0177d0 con 0x558318d84890 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.981+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/17293032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27e009a050 con 0x558318d84890 2024-05-08T23:47:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.981+0000 7f27c7fff700 1 -- 172.21.15.12:0/17293032 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27dc017990 con 0x558318d84890 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.981+0000 7f27fb2c0b80 10 client.25535 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.981+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/17293032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558318c9a560 con 0x558318d84890 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.981+0000 7f27c7fff700 1 -- 172.21.15.12:0/17293032 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e009a050 con 0x558318d84890 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.982+0000 7f27c7fff700 1 -- 172.21.15.12:0/17293032 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f27dc04e990 con 0x558318d84890 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.982+0000 7f27fb2c0b80 10 client.25535 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.982+0000 7f27fb2c0b80 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.982+0000 7f27fb2c0b80 20 client.25535 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.982+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558318df5230 0x558318df7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.982+0000 7f27fb2c0b80 4 client.25535 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.982+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/17293032 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558318cccec0 con 0x558318df5230 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.982+0000 7f27e7fff700 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558318df5230 0x558318df7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.982+0000 7f27c7fff700 10 client.25535 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.982+0000 7f27e7fff700 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558318df5230 0x558318df7620 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537ab157ea3bbb8 server_cookie=8a109b4e686002 in_seq=0 out_seq=0 2024-05-08T23:47:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27c7fff700 1 -- 172.21.15.12:0/17293032 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558318cccec0 con 0x558318df5230 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27c7fff700 10 client.25535 handle_command_reply: tid=0 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 client.25535 shutdown 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/17293032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d84890 msgr2=0x558318d95170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d84890 0x558318d95170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f27dc017410 tx=0x7f27dc00b720 comp rx=0 tx=0).stop 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/17293032 shutdown_connections 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558318d8a720 0x558318d85b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558318df5230 0x558318df7620 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558318d84890 0x558318d95170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 --2- 172.21.15.12:0/17293032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558318d89e00 0x558318d956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27c7fff700 0 client.25535 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/17293032 >> 172.21.15.12:0/17293032 conn(0x558318ce2210 msgr2=0x558318d8abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/17293032 shutdown_connections 2024-05-08T23:47:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 1 -- 172.21.15.12:0/17293032 wait complete. 2024-05-08T23:47:36.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.983+0000 7f27fb2c0b80 20 client.25535 trim_cache size 0 max 16384 2024-05-08T23:47:36.983 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:47:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:47:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:47:36.983 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:47:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 -- 172.21.15.12:0/1704445140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27d80a1430 msgr2=0x7f27d80a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27d80a1430 0x7f27d80a38f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009fd0 tx=0x7f27dc019040 comp rx=0 tx=0).stop 2024-05-08T23:47:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 -- 172.21.15.12:0/1704445140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27e819dec0 msgr2=0x7f27e819e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27e819dec0 0x7f27e819e340 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f27e000ef30 tx=0x7f27e000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 -- 172.21.15.12:0/1704445140 shutdown_connections 2024-05-08T23:47:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27d80a1430 0x7f27d80a38f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27e819dec0 0x7f27e819e340 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e80682e0 0x7f27e8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 --2- 172.21.15.12:0/1704445140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e8106290 0x7f27e8199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 -- 172.21.15.12:0/1704445140 >> 172.21.15.12:0/1704445140 conn(0x7f27e8076f40 msgr2=0x7f27e810c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 -- 172.21.15.12:0/1704445140 shutdown_connections 2024-05-08T23:47:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:36.984+0000 7f27ef106700 1 -- 172.21.15.12:0/1704445140 wait complete. 2024-05-08T23:47:36.993 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:47:36.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:47:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.153+0000 7fc6b5be9700 1 -- 172.21.15.12:0/3324167953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6b0101060 msgr2=0x7fc6b0101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.153+0000 7fc6b5be9700 1 --2- 172.21.15.12:0/3324167953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6b0101060 0x7fc6b0101440 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009990 tx=0x7fc6a4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.153+0000 7fc6b5be9700 1 -- 172.21.15.12:0/3324167953 shutdown_connections 2024-05-08T23:47:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.153+0000 7fc6b5be9700 1 --2- 172.21.15.12:0/3324167953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6b0106180 0x7fc6b010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.153+0000 7fc6b5be9700 1 --2- 172.21.15.12:0/3324167953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6b0101060 0x7fc6b0101440 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.153+0000 7fc6b5be9700 1 --2- 172.21.15.12:0/3324167953 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6b0101a10 0x7fc6b0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.153+0000 7fc6b5be9700 1 -- 172.21.15.12:0/3324167953 >> 172.21.15.12:0/3324167953 conn(0x7fc6b00fc960 msgr2=0x7fc6b00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.153+0000 7fc6b5be9700 1 -- 172.21.15.12:0/3324167953 shutdown_connections 2024-05-08T23:47:37.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.153+0000 7fc6b5be9700 1 -- 172.21.15.12:0/3324167953 wait complete. 2024-05-08T23:47:37.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.154+0000 7fc6b5be9700 1 Processor -- start 2024-05-08T23:47:37.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.154+0000 7fc6b5be9700 1 -- start start 2024-05-08T23:47:37.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.154+0000 7fc6b5be9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6b0101060 0x7fc6b0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.154+0000 7fc6b5be9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6b0106180 0x7fc6b0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.154+0000 7fc6b5be9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6b019dde0 0x7fc6b019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.154+0000 7fc6b5be9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b019e860 con 0x7fc6b019dde0 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.154+0000 7fc6b5be9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b019e9d0 con 0x7fc6b0106180 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.154+0000 7fc6b5be9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b019eb90 con 0x7fc6b0101060 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6af7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6b0101060 0x7fc6b0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6affff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6b019dde0 0x7fc6b019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6af7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6b0101060 0x7fc6b0199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:55950/0 (socket says 172.21.15.12:55950) 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6affff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6b019dde0 0x7fc6b019e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54318/0 (socket says 172.21.15.12:54318) 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6af7fe700 1 -- 172.21.15.12:0/4010923531 learned_addr learned my addr 172.21.15.12:0/4010923531 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6aeffd700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6b0106180 0x7fc6b0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6af7fe700 1 -- 172.21.15.12:0/4010923531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6b0106180 msgr2=0x7fc6b0199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6af7fe700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6b0106180 0x7fc6b0199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6af7fe700 1 -- 172.21.15.12:0/4010923531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6b019dde0 msgr2=0x7fc6b019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6af7fe700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6b019dde0 0x7fc6b019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6af7fe700 1 -- 172.21.15.12:0/4010923531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a40095f0 con 0x7fc6b0101060 2024-05-08T23:47:37.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6affff700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6b019dde0 0x7fc6b019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:37.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.155+0000 7fc6af7fe700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6b0101060 0x7fc6b0199500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005770 tx=0x7fc6a40057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:37.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.156+0000 7fc6acff9700 1 -- 172.21.15.12:0/4010923531 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a401d070 con 0x7fc6b0101060 2024-05-08T23:47:37.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.156+0000 7fc6b5be9700 1 -- 172.21.15.12:0/4010923531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b01a6090 con 0x7fc6b0101060 2024-05-08T23:47:37.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.156+0000 7fc6acff9700 1 -- 172.21.15.12:0/4010923531 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6a4022470 con 0x7fc6b0101060 2024-05-08T23:47:37.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.156+0000 7fc6acff9700 1 -- 172.21.15.12:0/4010923531 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4017650 con 0x7fc6b0101060 2024-05-08T23:47:37.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.156+0000 7fc6b5be9700 1 -- 172.21.15.12:0/4010923531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b01a64c0 con 0x7fc6b0101060 2024-05-08T23:47:37.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.157+0000 7fc6b5be9700 1 -- 172.21.15.12:0/4010923531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b0102530 con 0x7fc6b0101060 2024-05-08T23:47:37.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.158+0000 7fc6acff9700 1 -- 172.21.15.12:0/4010923531 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fc6a400fb00 con 0x7fc6b0101060 2024-05-08T23:47:37.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.159+0000 7fc6acff9700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc6980a13b0 0x7fc6980a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.159+0000 7fc6acff9700 1 -- 172.21.15.12:0/4010923531 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc6a40ce1d0 con 0x7fc6b0101060 2024-05-08T23:47:37.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.159+0000 7fc6aeffd700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc6980a13b0 0x7fc6980a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.159+0000 7fc6aeffd700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc6980a13b0 0x7fc6980a3870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6b019aa50 tx=0x7fc6a0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:37.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.164+0000 7fc6acff9700 1 -- 172.21.15.12:0/4010923531 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc6a40974a0 con 0x7fc6b0101060 2024-05-08T23:47:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.551+0000 7fc6b5be9700 1 -- 172.21.15.12:0/4010923531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6b0101980 con 0x7fc6b0101060 2024-05-08T23:47:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.552+0000 7fc6acff9700 1 -- 172.21.15.12:0/4010923531 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10876 (secure 0 0 0) 0x7fc6a409aac0 con 0x7fc6b0101060 2024-05-08T23:47:37.548 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:47:37.548 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:47:15.450515+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25489":{"gid":25489,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3829900343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3829900343},{"type":"v1","addr":"172.21.15.204:6837","nonce":3829900343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:47:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 -- 172.21.15.12:0/4010923531 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc6980a13b0 msgr2=0x7fc6980a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc6980a13b0 0x7fc6980a3870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6b019aa50 tx=0x7fc6a0009260 comp rx=0 tx=0).stop 2024-05-08T23:47:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 -- 172.21.15.12:0/4010923531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6b0101060 msgr2=0x7fc6b0199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6b0101060 0x7fc6b0199500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005770 tx=0x7fc6a40057a0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 -- 172.21.15.12:0/4010923531 shutdown_connections 2024-05-08T23:47:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc6980a13b0 0x7fc6980a3870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6b019dde0 0x7fc6b019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6b0101060 0x7fc6b0199500 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 --2- 172.21.15.12:0/4010923531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6b0106180 0x7fc6b0199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 -- 172.21.15.12:0/4010923531 >> 172.21.15.12:0/4010923531 conn(0x7fc6b00fc960 msgr2=0x7fc6b00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 -- 172.21.15.12:0/4010923531 shutdown_connections 2024-05-08T23:47:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.555+0000 7fc6b5be9700 1 -- 172.21.15.12:0/4010923531 wait complete. 2024-05-08T23:47:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2024-05-08T23:47:37.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:47:37.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.744+0000 7f47bfbec700 1 -- 172.21.15.12:0/1121649966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b8069710 msgr2=0x7f47b8069b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.744+0000 7f47bfbec700 1 --2- 172.21.15.12:0/1121649966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b8069710 0x7f47b8069b90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009990 tx=0x7f47a8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.744+0000 7f47bfbec700 1 -- 172.21.15.12:0/1121649966 shutdown_connections 2024-05-08T23:47:37.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.744+0000 7f47bfbec700 1 --2- 172.21.15.12:0/1121649966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b810d7f0 0x7f47b810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.744+0000 7f47bfbec700 1 --2- 172.21.15.12:0/1121649966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b8101e20 0x7f47b80691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.744+0000 7f47bfbec700 1 --2- 172.21.15.12:0/1121649966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b8069710 0x7f47b8069b90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.744+0000 7f47bfbec700 1 -- 172.21.15.12:0/1121649966 >> 172.21.15.12:0/1121649966 conn(0x7f47b8076ba0 msgr2=0x7f47b8076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.744+0000 7f47bfbec700 1 -- 172.21.15.12:0/1121649966 shutdown_connections 2024-05-08T23:47:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.744+0000 7f47bfbec700 1 -- 172.21.15.12:0/1121649966 wait complete. 2024-05-08T23:47:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.745+0000 7f47bfbec700 1 Processor -- start 2024-05-08T23:47:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.745+0000 7f47bfbec700 1 -- start start 2024-05-08T23:47:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bfbec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b8069710 0x7f47b81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bfbec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b8101e20 0x7f47b8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bfbec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b810d7f0 0x7f47b819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bfbec700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819e4e0 con 0x7f47b8069710 2024-05-08T23:47:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bfbec700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819e650 con 0x7f47b8101e20 2024-05-08T23:47:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bfbec700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819e810 con 0x7f47b810d7f0 2024-05-08T23:47:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bd988700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b8069710 0x7f47b81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bd988700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b8069710 0x7f47b81995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54328/0 (socket says 172.21.15.12:54328) 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bd988700 1 -- 172.21.15.12:0/2467523492 learned_addr learned my addr 172.21.15.12:0/2467523492 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bd187700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b8101e20 0x7f47b8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47be189700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b810d7f0 0x7f47b819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bd187700 1 -- 172.21.15.12:0/2467523492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b810d7f0 msgr2=0x7f47b819dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bd187700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b810d7f0 0x7f47b819dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bd187700 1 -- 172.21.15.12:0/2467523492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b8069710 msgr2=0x7f47b81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bd187700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b8069710 0x7f47b81995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bd187700 1 -- 172.21.15.12:0/2467523492 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80095f0 con 0x7f47b8101e20 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.746+0000 7f47bd988700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b8069710 0x7f47b81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.747+0000 7f47bd187700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b8101e20 0x7f47b8199b00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47a80059d0 tx=0x7f47a8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.747+0000 7f47aeffd700 1 -- 172.21.15.12:0/2467523492 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a801d070 con 0x7f47b8101e20 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.747+0000 7f47aeffd700 1 -- 172.21.15.12:0/2467523492 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a8022470 con 0x7f47b8101e20 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.747+0000 7f47bfbec700 1 -- 172.21.15.12:0/2467523492 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b819ea90 con 0x7f47b8101e20 2024-05-08T23:47:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.747+0000 7f47aeffd700 1 -- 172.21.15.12:0/2467523492 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8017650 con 0x7f47b8101e20 2024-05-08T23:47:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.747+0000 7f47bfbec700 1 -- 172.21.15.12:0/2467523492 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b81a6330 con 0x7f47b8101e20 2024-05-08T23:47:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.750+0000 7f47aeffd700 1 -- 172.21.15.12:0/2467523492 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f47a80225e0 con 0x7f47b8101e20 2024-05-08T23:47:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.751+0000 7f47aeffd700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47a40a13b0 0x7f47a40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.751+0000 7f47bd988700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47a40a13b0 0x7f47a40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.751+0000 7f47aeffd700 1 -- 172.21.15.12:0/2467523492 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f47a80ce0b0 con 0x7f47b8101e20 2024-05-08T23:47:37.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.751+0000 7f47cbda6b80 1 Processor -- start 2024-05-08T23:47:37.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47bd988700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47a40a13b0 0x7f47a40a3870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47b4005950 tx=0x7f47b400b500 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47cbda6b80 1 -- start start 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb732f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb73830 0x55fc3eb6d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb6dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3eab3150 con 0x55fc3eb72f10 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3eab0b10 con 0x55fc3eb73830 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3eaac400 con 0x55fc3eb6d8a0 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47bd988700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47bd988700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb732f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54336/0 (socket says 172.21.15.12:54336) 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47bd988700 1 -- 172.21.15.12:0/3721429903 learned_addr learned my addr 172.21.15.12:0/3721429903 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.752+0000 7f47bd187700 1 --2- 172.21.15.12:0/3721429903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb73830 0x55fc3eb6d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.753+0000 7f47be189700 1 --2- 172.21.15.12:0/3721429903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb6dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.753+0000 7f47bd988700 1 -- 172.21.15.12:0/3721429903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb6d8a0 msgr2=0x55fc3eb6dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.753+0000 7f47bd988700 1 --2- 172.21.15.12:0/3721429903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb6dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.753+0000 7f47bd988700 1 -- 172.21.15.12:0/3721429903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb73830 msgr2=0x55fc3eb6d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.753+0000 7f47bd988700 1 --2- 172.21.15.12:0/3721429903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb73830 0x55fc3eb6d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.753+0000 7f47bd988700 1 -- 172.21.15.12:0/3721429903 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc3e92d3b0 con 0x55fc3eb72f10 2024-05-08T23:47:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.753+0000 7f47bd988700 1 --2- 172.21.15.12:0/3721429903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb732f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f47b40117d0 tx=0x7f47b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c38de0e62124c5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.753+0000 7f47a3fff700 1 -- 172.21.15.12:0/3721429903 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4004220 con 0x55fc3eb72f10 2024-05-08T23:47:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.753+0000 7f47a3fff700 1 -- 172.21.15.12:0/3721429903 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b4015070 con 0x55fc3eb72f10 2024-05-08T23:47:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.753+0000 7f47a3fff700 1 -- 172.21.15.12:0/3721429903 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b401e6d0 con 0x55fc3eb72f10 2024-05-08T23:47:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.754+0000 7f47cbda6b80 1 -- 172.21.15.12:0/3721429903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 msgr2=0x55fc3eb732f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.754+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/3721429903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb732f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f47b40117d0 tx=0x7f47b4013000 comp rx=0 tx=0).stop 2024-05-08T23:47:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.754+0000 7f47cbda6b80 1 -- 172.21.15.12:0/3721429903 shutdown_connections 2024-05-08T23:47:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.754+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/3721429903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb6dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.754+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/3721429903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb732f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.754+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/3721429903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb73830 0x55fc3eb6d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.754+0000 7f47cbda6b80 1 -- 172.21.15.12:0/3721429903 >> 172.21.15.12:0/3721429903 conn(0x55fc3eaba210 msgr2=0x55fc3eb56df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.754+0000 7f47cbda6b80 1 -- 172.21.15.12:0/3721429903 shutdown_connections 2024-05-08T23:47:37.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.754+0000 7f47cbda6b80 1 -- 172.21.15.12:0/3721429903 wait complete. 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.755+0000 7f47cbda6b80 1 -- start start 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.755+0000 7f47cbda6b80 1 Processor -- start 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.755+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb76c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.755+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb77170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.755+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb73830 0x55fc3eb776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.755+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3ea48560 con 0x55fc3eb72f10 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.755+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3ea48200 con 0x55fc3eb6d8a0 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.755+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3e92f6a0 con 0x55fc3eb73830 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.755+0000 7f47cbda6b80 10 client.0 _ll_register_callbacks cb 0x55fc3ea51b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47bd187700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb77170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47bd187700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb77170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54352/0 (socket says 172.21.15.12:54352) 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47bd187700 1 -- 172.21.15.12:0/701138441 learned_addr learned my addr 172.21.15.12:0/701138441 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:37.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47bd988700 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb76c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47be189700 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb73830 0x55fc3eb776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47bd187700 1 -- 172.21.15.12:0/701138441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb73830 msgr2=0x55fc3eb776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47bd187700 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb73830 0x55fc3eb776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47bd187700 1 -- 172.21.15.12:0/701138441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb6d8a0 msgr2=0x55fc3eb76c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47bd187700 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb76c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47bd187700 1 -- 172.21.15.12:0/701138441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b4011480 con 0x55fc3eb72f10 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47bd187700 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb77170 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f47a8097ab0 tx=0x7f47a8022bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc6bb8e596dd97d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.756+0000 7f47a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47a17fa700 1 -- 172.21.15.12:0/701138441 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a809a730 con 0x55fc3eb72f10 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47a17fa700 1 -- 172.21.15.12:0/701138441 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f47a809b040 con 0x55fc3eb72f10 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47cbda6b80 1 -- 172.21.15.12:0/701138441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc3ea05270 con 0x55fc3eb72f10 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47a17fa700 1 -- 172.21.15.12:0/701138441 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47a809ee70 con 0x55fc3eb72f10 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47cbda6b80 10 client.15705 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:37.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47cbda6b80 1 -- 172.21.15.12:0/701138441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fc3e8d9b60 con 0x55fc3eb72f10 2024-05-08T23:47:37.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47a17fa700 1 -- 172.21.15.12:0/701138441 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc3ea05270 con 0x55fc3eb72f10 2024-05-08T23:47:37.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47a17fa700 1 -- 172.21.15.12:0/701138441 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f47a80cb9a0 con 0x55fc3eb72f10 2024-05-08T23:47:37.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47cbda6b80 10 client.15705 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:37.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47cbda6b80 10 client.15705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:47:37.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47cbda6b80 20 client.15705 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:37.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc3eb86bf0 0x55fc3eb88fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47cbda6b80 4 client.15705 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:37.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.757+0000 7f47cbda6b80 1 -- 172.21.15.12:0/701138441 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc3eb2b810 con 0x55fc3eb86bf0 2024-05-08T23:47:37.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.758+0000 7f47bd988700 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc3eb86bf0 0x55fc3eb88fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.758+0000 7f47a17fa700 10 client.15705 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:47:37.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.758+0000 7f47bd988700 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc3eb86bf0 0x55fc3eb88fe0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69927dec7aca2066 server_cookie=46853f2011d2ee2f in_seq=0 out_seq=0 2024-05-08T23:47:37.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47a17fa700 1 -- 172.21.15.12:0/701138441 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fc3eb2b810 con 0x55fc3eb86bf0 2024-05-08T23:47:37.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47a17fa700 10 client.15705 handle_command_reply: tid=0 2024-05-08T23:47:37.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47cbda6b80 1 client.15705 shutdown 2024-05-08T23:47:37.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47cbda6b80 1 -- 172.21.15.12:0/701138441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 msgr2=0x55fc3eb77170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb77170 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f47a8097ab0 tx=0x7f47a8022bc0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47cbda6b80 1 -- 172.21.15.12:0/701138441 shutdown_connections 2024-05-08T23:47:37.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc3eb86bf0 0x55fc3eb88fe0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb73830 0x55fc3eb776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb76c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/701138441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb72f10 0x55fc3eb77170 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47cbda6b80 1 -- 172.21.15.12:0/701138441 >> 172.21.15.12:0/701138441 conn(0x55fc3eaba210 msgr2=0x55fc3eb565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:37.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.759+0000 7f47a17fa700 0 client.15705 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:47:37.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.760+0000 7f47cbda6b80 1 -- 172.21.15.12:0/701138441 shutdown_connections 2024-05-08T23:47:37.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.760+0000 7f47cbda6b80 1 -- 172.21.15.12:0/701138441 wait complete. 2024-05-08T23:47:37.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.760+0000 7f47cbda6b80 20 client.15705 trim_cache size 0 max 16384 2024-05-08T23:47:37.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.774+0000 7f47cbda6b80 1 Processor -- start 2024-05-08T23:47:37.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.774+0000 7f47cbda6b80 1 -- start start 2024-05-08T23:47:37.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.774+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb83190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.774+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 0x55fc3eb56d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.774+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb73830 0x55fc3eb774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.774+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3eb78190 con 0x55fc3eb6d8a0 2024-05-08T23:47:37.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.774+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3eb53dc0 con 0x55fc3eb73830 2024-05-08T23:47:37.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.774+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3eb72b20 con 0x55fc3eb72f10 2024-05-08T23:47:37.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.774+0000 7f47bd988700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb83190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47bd988700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb83190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54366/0 (socket says 172.21.15.12:54366) 2024-05-08T23:47:37.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47bd988700 1 -- 172.21.15.12:0/747995043 learned_addr learned my addr 172.21.15.12:0/747995043 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:37.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47bd187700 1 --2- 172.21.15.12:0/747995043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 0x55fc3eb56d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47be189700 1 --2- 172.21.15.12:0/747995043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb73830 0x55fc3eb774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47bd988700 1 -- 172.21.15.12:0/747995043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 msgr2=0x55fc3eb56d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47bd988700 1 --2- 172.21.15.12:0/747995043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 0x55fc3eb56d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47bd988700 1 -- 172.21.15.12:0/747995043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb73830 msgr2=0x55fc3eb774d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47bd988700 1 --2- 172.21.15.12:0/747995043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb73830 0x55fc3eb774d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47bd988700 1 -- 172.21.15.12:0/747995043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc3ea72560 con 0x55fc3eb6d8a0 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47bd187700 1 --2- 172.21.15.12:0/747995043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 0x55fc3eb56d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47bd988700 1 --2- 172.21.15.12:0/747995043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb83190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f47b4003f60 tx=0x7f47b400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8679b8fb2901fe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47acda4700 1 -- 172.21.15.12:0/747995043 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b40042e0 con 0x55fc3eb6d8a0 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47acda4700 1 -- 172.21.15.12:0/747995043 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b4004440 con 0x55fc3eb6d8a0 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.775+0000 7f47acda4700 1 -- 172.21.15.12:0/747995043 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4003c60 con 0x55fc3eb6d8a0 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.776+0000 7f47cbda6b80 1 -- 172.21.15.12:0/747995043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb6d8a0 msgr2=0x55fc3eb83190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.776+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/747995043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb83190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f47b4003f60 tx=0x7f47b400bf80 comp rx=0 tx=0).stop 2024-05-08T23:47:37.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.776+0000 7f47cbda6b80 1 -- 172.21.15.12:0/747995043 shutdown_connections 2024-05-08T23:47:37.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.776+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/747995043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb73830 0x55fc3eb774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.776+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/747995043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb83190 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.776+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/747995043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 0x55fc3eb56d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.776+0000 7f47cbda6b80 1 -- 172.21.15.12:0/747995043 >> 172.21.15.12:0/747995043 conn(0x55fc3eaba210 msgr2=0x55fc3eb6dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:37.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.777+0000 7f47cbda6b80 1 -- 172.21.15.12:0/747995043 shutdown_connections 2024-05-08T23:47:37.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.777+0000 7f47cbda6b80 1 -- 172.21.15.12:0/747995043 wait complete. 2024-05-08T23:47:37.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.777+0000 7f47cbda6b80 1 -- start start 2024-05-08T23:47:37.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.777+0000 7f47cbda6b80 1 Processor -- start 2024-05-08T23:47:37.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb7e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 0x55fc3eb7c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47cbda6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb73830 0x55fc3eb7c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3eb7cd00 con 0x55fc3eb73830 2024-05-08T23:47:37.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3eb7ce70 con 0x55fc3eb6d8a0 2024-05-08T23:47:37.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47cbda6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3ebdc0c0 con 0x55fc3eb72f10 2024-05-08T23:47:37.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47cbda6b80 10 client.0 _ll_register_callbacks cb 0x55fc3ea51b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:37.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47be189700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb73830 0x55fc3eb7c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47be189700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb73830 0x55fc3eb7c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54370/0 (socket says 172.21.15.12:54370) 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47be189700 1 -- 172.21.15.12:0/4054080240 learned_addr learned my addr 172.21.15.12:0/4054080240 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47bd988700 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb7e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47bd187700 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 0x55fc3eb7c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47be189700 1 -- 172.21.15.12:0/4054080240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 msgr2=0x55fc3eb7c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47be189700 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 0x55fc3eb7c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47be189700 1 -- 172.21.15.12:0/4054080240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb6d8a0 msgr2=0x55fc3eb7e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47be189700 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb7e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47be189700 1 -- 172.21.15.12:0/4054080240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc3e92d3b0 con 0x55fc3eb73830 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.778+0000 7f47bd187700 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 0x55fc3eb7c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.779+0000 7f47be189700 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb73830 0x55fc3eb7c5f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f47b0007f00 tx=0x7f47b0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529b4f1d5a62941 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.779+0000 7f47a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.779+0000 7f47a17fa700 1 -- 172.21.15.12:0/4054080240 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000e040 con 0x55fc3eb73830 2024-05-08T23:47:37.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.779+0000 7f47a17fa700 1 -- 172.21.15.12:0/4054080240 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f47b0003e10 con 0x55fc3eb73830 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.779+0000 7f47cbda6b80 1 -- 172.21.15.12:0/4054080240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47b4015070 con 0x55fc3eb73830 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.779+0000 7f47a17fa700 1 -- 172.21.15.12:0/4054080240 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b0004bb0 con 0x55fc3eb73830 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.779+0000 7f47cbda6b80 10 client.15711 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.779+0000 7f47cbda6b80 1 -- 172.21.15.12:0/4054080240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fc3e931cd0 con 0x55fc3eb73830 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.779+0000 7f47a17fa700 1 -- 172.21.15.12:0/4054080240 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b4015070 con 0x55fc3eb73830 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.780+0000 7f47a17fa700 1 -- 172.21.15.12:0/4054080240 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f47b00043f0 con 0x55fc3eb73830 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.780+0000 7f47cbda6b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.780+0000 7f47cbda6b80 10 client.15711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.780+0000 7f47cbda6b80 20 client.15711 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.780+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc3ebdd950 0x55fc3ebdfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.780+0000 7f47cbda6b80 4 client.15711 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.780+0000 7f47cbda6b80 1 -- 172.21.15.12:0/4054080240 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fc3eaa4ec0 con 0x55fc3ebdd950 2024-05-08T23:47:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.780+0000 7f47bd988700 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc3ebdd950 0x55fc3ebdfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.780+0000 7f47a17fa700 10 client.15711 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.780+0000 7f47bd988700 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc3ebdd950 0x55fc3ebdfd70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7c1d4f1ce022da2 server_cookie=23f47ce580963305 in_seq=0 out_seq=0 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47a17fa700 1 -- 172.21.15.12:0/4054080240 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fc3eaa4ec0 con 0x55fc3ebdd950 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47a17fa700 10 client.15711 handle_command_reply: tid=0 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 client.15711 shutdown 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 -- 172.21.15.12:0/4054080240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb73830 msgr2=0x55fc3eb7c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb73830 0x55fc3eb7c5f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f47b0007f00 tx=0x7f47b0007f30 comp rx=0 tx=0).stop 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 -- 172.21.15.12:0/4054080240 shutdown_connections 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc3eb73830 0x55fc3eb7c5f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc3eb6d8a0 0x55fc3eb7e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc3ebdd950 0x55fc3ebdfd70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 --2- 172.21.15.12:0/4054080240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc3eb72f10 0x55fc3eb7c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47a17fa700 0 client.15711 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 -- 172.21.15.12:0/4054080240 >> 172.21.15.12:0/4054080240 conn(0x55fc3eaba210 msgr2=0x55fc3eb83190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 -- 172.21.15.12:0/4054080240 shutdown_connections 2024-05-08T23:47:37.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 1 -- 172.21.15.12:0/4054080240 wait complete. 2024-05-08T23:47:37.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.781+0000 7f47cbda6b80 20 client.15711 trim_cache size 0 max 16384 2024-05-08T23:47:37.781 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:47:37.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 -- 172.21.15.12:0/2467523492 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47a40a13b0 msgr2=0x7f47a40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47a40a13b0 0x7f47a40a3870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47b4005950 tx=0x7f47b400b500 comp rx=0 tx=0).stop 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 -- 172.21.15.12:0/2467523492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b8101e20 msgr2=0x7f47b8199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b8101e20 0x7f47b8199b00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47a80059d0 tx=0x7f47a8004730 comp rx=0 tx=0).stop 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 -- 172.21.15.12:0/2467523492 shutdown_connections 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47a40a13b0 0x7f47a40a3870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b810d7f0 0x7f47b819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b8069710 0x7f47b81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 --2- 172.21.15.12:0/2467523492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b8101e20 0x7f47b8199b00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 -- 172.21.15.12:0/2467523492 >> 172.21.15.12:0/2467523492 conn(0x7f47b8076ba0 msgr2=0x7f47b80ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 -- 172.21.15.12:0/2467523492 shutdown_connections 2024-05-08T23:47:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.782+0000 7f47bfbec700 1 -- 172.21.15.12:0/2467523492 wait complete. 2024-05-08T23:47:37.796 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:47:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.992+0000 7f9898b29700 1 -- 172.21.15.12:0/3919185049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9894068970 msgr2=0x7f989410d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.992+0000 7f9898b29700 1 --2- 172.21.15.12:0/3919185049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9894068970 0x7f989410d400 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9888009990 tx=0x7f9888009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.992+0000 7f9898b29700 1 -- 172.21.15.12:0/3919185049 shutdown_connections 2024-05-08T23:47:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.992+0000 7f9898b29700 1 --2- 172.21.15.12:0/3919185049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f989410d940 0x7f989410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.992+0000 7f9898b29700 1 --2- 172.21.15.12:0/3919185049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9894068050 0x7f9894068430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.992+0000 7f9898b29700 1 --2- 172.21.15.12:0/3919185049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9894068970 0x7f989410d400 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.992+0000 7f9898b29700 1 -- 172.21.15.12:0/3919185049 >> 172.21.15.12:0/3919185049 conn(0x7f9894076f40 msgr2=0x7f9894077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.992+0000 7f9898b29700 1 -- 172.21.15.12:0/3919185049 shutdown_connections 2024-05-08T23:47:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.992+0000 7f9898b29700 1 -- 172.21.15.12:0/3919185049 wait complete. 2024-05-08T23:47:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.993+0000 7f9898b29700 1 Processor -- start 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.993+0000 7f9898b29700 1 -- start start 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.993+0000 7f9898b29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9894068050 0x7f9894199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.993+0000 7f9898b29700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9894068970 0x7f9894199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.993+0000 7f9898b29700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989410d940 0x7f989419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.993+0000 7f9898b29700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989419e540 con 0x7f9894068050 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.993+0000 7f9898b29700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989419e6b0 con 0x7f9894068970 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.993+0000 7f9898b29700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989419e870 con 0x7f989410d940 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f989259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9894068050 0x7f9894199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f989259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9894068050 0x7f9894199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54400/0 (socket says 172.21.15.12:54400) 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f989259c700 1 -- 172.21.15.12:0/1437054826 learned_addr learned my addr 172.21.15.12:0/1437054826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f9891d9b700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9894068970 0x7f9894199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f9892d9d700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989410d940 0x7f989419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f989259c700 1 -- 172.21.15.12:0/1437054826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989410d940 msgr2=0x7f989419df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f989259c700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989410d940 0x7f989419df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f989259c700 1 -- 172.21.15.12:0/1437054826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9894068970 msgr2=0x7f9894199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f989259c700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9894068970 0x7f9894199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f989259c700 1 -- 172.21.15.12:0/1437054826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98880095f0 con 0x7f9894068050 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f9892d9d700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989410d940 0x7f989419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.994+0000 7f989259c700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9894068050 0x7f9894199620 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f988400e9a0 tx=0x7f988400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.995+0000 7f98837fe700 1 -- 172.21.15.12:0/1437054826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884009790 con 0x7f9894068050 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.995+0000 7f9898b29700 1 -- 172.21.15.12:0/1437054826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989419eb50 con 0x7f9894068050 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.995+0000 7f98837fe700 1 -- 172.21.15.12:0/1437054826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f988400cbd0 con 0x7f9894068050 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.995+0000 7f98837fe700 1 -- 172.21.15.12:0/1437054826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884018680 con 0x7f9894068050 2024-05-08T23:47:37.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:37.995+0000 7f9898b29700 1 -- 172.21.15.12:0/1437054826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98941a6390 con 0x7f9894068050 2024-05-08T23:47:37.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.000+0000 7f98a4ce3b80 1 Processor -- start 2024-05-08T23:47:37.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.000+0000 7f98837fe700 1 -- 172.21.15.12:0/1437054826 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f988400cd40 con 0x7f9894068050 2024-05-08T23:47:37.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.001+0000 7f98a4ce3b80 1 -- start start 2024-05-08T23:47:37.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.002+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 0x55b5c40e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.002+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c40e1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.002+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e1890 0x55b5c40e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.002+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c4038150 con 0x55b5c40e6e00 2024-05-08T23:47:37.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.002+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c4035b10 con 0x55b5c40e7720 2024-05-08T23:47:37.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.002+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c4031400 con 0x55b5c40e1890 2024-05-08T23:47:37.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.002+0000 7f9892d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 0x55b5c40e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.002+0000 7f9892d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 0x55b5c40e71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54410/0 (socket says 172.21.15.12:54410) 2024-05-08T23:47:37.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.002+0000 7f9892d9d700 1 -- 172.21.15.12:0/761466888 learned_addr learned my addr 172.21.15.12:0/761466888 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.002+0000 7f989259c700 1 --2- 172.21.15.12:0/761466888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c40e1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.003+0000 7f9892d9d700 1 -- 172.21.15.12:0/761466888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e1890 msgr2=0x55b5c40e1cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:47:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.003+0000 7f9892d9d700 1 --2- 172.21.15.12:0/761466888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e1890 0x55b5c40e1cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.003+0000 7f9892d9d700 1 -- 172.21.15.12:0/761466888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 msgr2=0x55b5c40e1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.003+0000 7f9892d9d700 1 --2- 172.21.15.12:0/761466888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c40e1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.003+0000 7f9892d9d700 1 -- 172.21.15.12:0/761466888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5c3eb23b0 con 0x55b5c40e6e00 2024-05-08T23:47:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.003+0000 7f98837fe700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f987c0a13b0 0x7f987c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.003+0000 7f98837fe700 1 -- 172.21.15.12:0/1437054826 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9884014070 con 0x7f9894068050 2024-05-08T23:47:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.003+0000 7f9892d9d700 1 --2- 172.21.15.12:0/761466888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 0x55b5c40e71e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f988c0093d0 tx=0x7f988c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a56cd56c82b9f4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.004+0000 7f989259c700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f987c0a13b0 0x7f987c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.004+0000 7f9880ff9700 1 -- 172.21.15.12:0/761466888 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0153f0 con 0x55b5c40e6e00 2024-05-08T23:47:38.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.004+0000 7f9880ff9700 1 -- 172.21.15.12:0/761466888 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f988c011040 con 0x55b5c40e6e00 2024-05-08T23:47:38.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.004+0000 7f9880ff9700 1 -- 172.21.15.12:0/761466888 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0148f0 con 0x55b5c40e6e00 2024-05-08T23:47:38.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.004+0000 7f989259c700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f987c0a13b0 0x7f987c0a3870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f98840ca9d0 tx=0x7f98840d0c70 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.006+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/761466888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 msgr2=0x55b5c40e71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.006+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/761466888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 0x55b5c40e71e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f988c0093d0 tx=0x7f988c009920 comp rx=0 tx=0).stop 2024-05-08T23:47:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.006+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/761466888 shutdown_connections 2024-05-08T23:47:38.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.006+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/761466888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e1890 0x55b5c40e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.006+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/761466888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 0x55b5c40e71e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.006+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/761466888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c40e1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.006+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/761466888 >> 172.21.15.12:0/761466888 conn(0x55b5c403f210 msgr2=0x55b5c40dbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.006+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/761466888 shutdown_connections 2024-05-08T23:47:38.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.006+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/761466888 wait complete. 2024-05-08T23:47:38.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.007+0000 7f98a4ce3b80 1 -- start start 2024-05-08T23:47:38.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.007+0000 7f98a4ce3b80 1 Processor -- start 2024-05-08T23:47:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.007+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e1890 0x55b5c40eaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.007+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 0x55b5c40eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.007+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e7720 0x55b5c40eb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.007+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c3fcd560 con 0x55b5c40e6e00 2024-05-08T23:47:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.007+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c3fcd200 con 0x55b5c40e1890 2024-05-08T23:47:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.007+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c3eb46a0 con 0x55b5c40e7720 2024-05-08T23:47:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.007+0000 7f98a4ce3b80 10 client.0 _ll_register_callbacks cb 0x55b5c3fd6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.008+0000 7f9892d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e1890 0x55b5c40eaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.008+0000 7f9892d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e1890 0x55b5c40eaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34686/0 (socket says 172.21.15.12:34686) 2024-05-08T23:47:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.008+0000 7f9892d9d700 1 -- 172.21.15.12:0/1920750355 learned_addr learned my addr 172.21.15.12:0/1920750355 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.008+0000 7f9892d9d700 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e7720 0x55b5c40eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.008+0000 7f9892d9d700 1 -- 172.21.15.12:0/1920750355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e7720 msgr2=0x55b5c40eb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.008+0000 7f9892d9d700 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e7720 0x55b5c40eb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.008+0000 7f9892d9d700 1 -- 172.21.15.12:0/1920750355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 msgr2=0x55b5c40eaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.008+0000 7f9892d9d700 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 0x55b5c40eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.008+0000 7f9892d9d700 1 -- 172.21.15.12:0/1920750355 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c0090b0 con 0x55b5c40e1890 2024-05-08T23:47:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.009+0000 7f9892d9d700 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e1890 0x55b5c40eaab0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f988c014f90 tx=0x7f988c014430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34c4e4e5020c129f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.009+0000 7f9879ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:47:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.009+0000 7f9879ffb700 1 -- 172.21.15.12:0/1920750355 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c015800 con 0x55b5c40e1890 2024-05-08T23:47:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.009+0000 7f9879ffb700 1 -- 172.21.15.12:0/1920750355 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f988c015960 con 0x55b5c40e1890 2024-05-08T23:47:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.009+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1920750355 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5c3f8a270 con 0x55b5c40e1890 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.009+0000 7f9879ffb700 1 -- 172.21.15.12:0/1920750355 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f988c015b20 con 0x55b5c40e1890 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.009+0000 7f98a4ce3b80 10 client.25571 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.009+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1920750355 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b5c3e5eb60 con 0x55b5c40e1890 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.009+0000 7f9879ffb700 1 -- 172.21.15.12:0/1920750355 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5c3f8a270 con 0x55b5c40e1890 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.010+0000 7f9879ffb700 1 -- 172.21.15.12:0/1920750355 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f988c007500 con 0x55b5c40e1890 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.010+0000 7f98a4ce3b80 10 client.25571 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.010+0000 7f98a4ce3b80 10 client.25571 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.010+0000 7f98a4ce3b80 20 client.25571 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.010+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55b5c40f2020 0x55b5c40f4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.010+0000 7f98a4ce3b80 4 client.25571 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.010+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1920750355 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5c40b0810 con 0x55b5c40f2020 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.010+0000 7f9891d9b700 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55b5c40f2020 0x55b5c40f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.010+0000 7f9891d9b700 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55b5c40f2020 0x55b5c40f4440 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b7a47cf7c181d140 server_cookie=414998700848a9f1 in_seq=0 out_seq=0 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.010+0000 7f9879ffb700 10 client.25571 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f9879ffb700 1 -- 172.21.15.12:0/1920750355 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5c40b0810 con 0x55b5c40f2020 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f9879ffb700 10 client.25571 handle_command_reply: tid=0 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f98a4ce3b80 1 client.25571 shutdown 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1920750355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e1890 msgr2=0x55b5c40eaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e1890 0x55b5c40eaab0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f988c014f90 tx=0x7f988c014430 comp rx=0 tx=0).stop 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1920750355 shutdown_connections 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55b5c40f2020 0x55b5c40f4440 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e7720 0x55b5c40eb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e1890 0x55b5c40eaab0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f9879ffb700 0 client.25571 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/1920750355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e6e00 0x55b5c40eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.011+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1920750355 >> 172.21.15.12:0/1920750355 conn(0x55b5c403f210 msgr2=0x55b5c40db5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.012+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1920750355 shutdown_connections 2024-05-08T23:47:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.012+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1920750355 wait complete. 2024-05-08T23:47:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.013+0000 7f98a4ce3b80 20 client.25571 trim_cache size 0 max 16384 2024-05-08T23:47:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f98a4ce3b80 1 Processor -- start 2024-05-08T23:47:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f98a4ce3b80 1 -- start start 2024-05-08T23:47:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 0x55b5c40e1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 0x55b5c40dbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c40eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c40dd0a0 con 0x55b5c40e1890 2024-05-08T23:47:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c40d8dc0 con 0x55b5c40e7720 2024-05-08T23:47:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c40e7590 con 0x55b5c40e6e00 2024-05-08T23:47:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f9892d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c40eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f9892d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c40eb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34700/0 (socket says 172.21.15.12:34700) 2024-05-08T23:47:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.032+0000 7f9892d9d700 1 -- 172.21.15.12:0/1195637711 learned_addr learned my addr 172.21.15.12:0/1195637711 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f9892d9d700 1 --2- 172.21.15.12:0/1195637711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 0x55b5c40e1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f9891d9b700 1 --2- 172.21.15.12:0/1195637711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 0x55b5c40dbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f9892d9d700 1 -- 172.21.15.12:0/1195637711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 msgr2=0x55b5c40dbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f9892d9d700 1 --2- 172.21.15.12:0/1195637711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 0x55b5c40dbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f9892d9d700 1 -- 172.21.15.12:0/1195637711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 msgr2=0x55b5c40e1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f9892d9d700 1 --2- 172.21.15.12:0/1195637711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 0x55b5c40e1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f9892d9d700 1 -- 172.21.15.12:0/1195637711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5c40e1f40 con 0x55b5c40e7720 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f9891d9b700 1 --2- 172.21.15.12:0/1195637711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 0x55b5c40dbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f9892d9d700 1 --2- 172.21.15.12:0/1195637711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c40eb020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f988c002230 tx=0x7f988c014fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b0952b6162220a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f98817fa700 1 -- 172.21.15.12:0/1195637711 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c014b20 con 0x55b5c40e7720 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f98817fa700 1 -- 172.21.15.12:0/1195637711 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f988c0073e0 con 0x55b5c40e7720 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.033+0000 7f98817fa700 1 -- 172.21.15.12:0/1195637711 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c022920 con 0x55b5c40e7720 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.034+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1195637711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 msgr2=0x55b5c40eb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.034+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/1195637711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c40eb020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f988c002230 tx=0x7f988c014fc0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.034+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1195637711 shutdown_connections 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.034+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/1195637711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c40eb020 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.034+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/1195637711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 0x55b5c40e1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.034+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/1195637711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 0x55b5c40dbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.034+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1195637711 >> 172.21.15.12:0/1195637711 conn(0x55b5c403f210 msgr2=0x55b5c40e93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.034+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1195637711 shutdown_connections 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.034+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/1195637711 wait complete. 2024-05-08T23:47:38.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f98a4ce3b80 1 -- start start 2024-05-08T23:47:38.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f98a4ce3b80 1 Processor -- start 2024-05-08T23:47:38.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 0x55b5c40f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 0x55b5c40f8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f98a4ce3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c41611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c40f9150 con 0x55b5c40e1890 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c4161710 con 0x55b5c40e7720 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f98a4ce3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c4172860 con 0x55b5c40e6e00 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f98a4ce3b80 10 client.0 _ll_register_callbacks cb 0x55b5c3fd6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f9892d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 0x55b5c40f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f9892d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 0x55b5c40f8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54440/0 (socket says 172.21.15.12:54440) 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f9892d9d700 1 -- 172.21.15.12:0/2872208656 learned_addr learned my addr 172.21.15.12:0/2872208656 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.035+0000 7f9891d9b700 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 0x55b5c40f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9892d9d700 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c41611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9891d9b700 1 -- 172.21.15.12:0/2872208656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 msgr2=0x55b5c41611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9891d9b700 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c41611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9891d9b700 1 -- 172.21.15.12:0/2872208656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 msgr2=0x55b5c40f8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9891d9b700 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 0x55b5c40f8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9891d9b700 1 -- 172.21.15.12:0/2872208656 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5c3e5eb60 con 0x55b5c40e6e00 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9892d9d700 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c41611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9891d9b700 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 0x55b5c40f8870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f989419ab70 tx=0x7f988800b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6faffc9c5724d58c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9879ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9879ffb700 1 -- 172.21.15.12:0/2872208656 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888025600 con 0x55b5c40e6e00 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/2872208656 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f988c011040 con 0x55b5c40e6e00 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.036+0000 7f9879ffb700 1 -- 172.21.15.12:0/2872208656 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9888025760 con 0x55b5c40e6e00 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f9879ffb700 1 -- 172.21.15.12:0/2872208656 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9888025920 con 0x55b5c40e6e00 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f98a4ce3b80 10 client.25576 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/2872208656 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b5c3ff7560 con 0x55b5c40e6e00 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f9879ffb700 1 -- 172.21.15.12:0/2872208656 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f988c011040 con 0x55b5c40e6e00 2024-05-08T23:47:38.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f9879ffb700 1 -- 172.21.15.12:0/2872208656 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f98880048a0 con 0x55b5c40e6e00 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f98a4ce3b80 10 client.25576 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f98a4ce3b80 10 client.25576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f98a4ce3b80 20 client.25576 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55b5c4173fe0 0x55b5c4176400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f98a4ce3b80 4 client.25576 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.037+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/2872208656 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b5c4029ec0 con 0x55b5c4173fe0 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.038+0000 7f9892d9d700 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55b5c4173fe0 0x55b5c4176400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.038+0000 7f9892d9d700 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55b5c4173fe0 0x55b5c4176400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31d6a6003efef411 server_cookie=9bf35ca19cc21857 in_seq=0 out_seq=0 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.038+0000 7f9879ffb700 10 client.25576 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.038+0000 7f9879ffb700 1 -- 172.21.15.12:0/2872208656 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b5c4029ec0 con 0x55b5c4173fe0 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.038+0000 7f9879ffb700 10 client.25576 handle_command_reply: tid=0 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.038+0000 7f98a4ce3b80 1 client.25576 shutdown 2024-05-08T23:47:38.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/2872208656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 msgr2=0x55b5c40f8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 0x55b5c40f8870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f989419ab70 tx=0x7f988800b610 comp rx=0 tx=0).stop 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/2872208656 shutdown_connections 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55b5c4173fe0 0x55b5c4176400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b5c40e7720 0x55b5c41611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b5c40e1890 0x55b5c40f8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f9879ffb700 0 client.25576 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 1 --2- 172.21.15.12:0/2872208656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5c40e6e00 0x55b5c40f8870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/2872208656 >> 172.21.15.12:0/2872208656 conn(0x55b5c403f210 msgr2=0x55b5c40ec350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/2872208656 shutdown_connections 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 1 -- 172.21.15.12:0/2872208656 wait complete. 2024-05-08T23:47:38.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.039+0000 7f98a4ce3b80 20 client.25576 trim_cache size 0 max 16384 2024-05-08T23:47:38.039 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:47:38.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 -- 172.21.15.12:0/1437054826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f987c0a13b0 msgr2=0x7f987c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f987c0a13b0 0x7f987c0a3870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f98840ca9d0 tx=0x7f98840d0c70 comp rx=0 tx=0).stop 2024-05-08T23:47:38.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 -- 172.21.15.12:0/1437054826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9894068050 msgr2=0x7f9894199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9894068050 0x7f9894199620 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f988400e9a0 tx=0x7f988400ed60 comp rx=0 tx=0).stop 2024-05-08T23:47:38.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 -- 172.21.15.12:0/1437054826 shutdown_connections 2024-05-08T23:47:38.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f987c0a13b0 0x7f987c0a3870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989410d940 0x7f989419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9894068050 0x7f9894199620 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 --2- 172.21.15.12:0/1437054826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9894068970 0x7f9894199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 -- 172.21.15.12:0/1437054826 >> 172.21.15.12:0/1437054826 conn(0x7f9894076f40 msgr2=0x7f989410efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 -- 172.21.15.12:0/1437054826 shutdown_connections 2024-05-08T23:47:38.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.040+0000 7f9898b29700 1 -- 172.21.15.12:0/1437054826 wait complete. 2024-05-08T23:47:38.051 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:47:38.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.200+0000 7f516f7cd700 1 -- 172.21.15.12:0/1233695359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5168100060 msgr2=0x7f5168100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.200+0000 7f516f7cd700 1 --2- 172.21.15.12:0/1233695359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5168100060 0x7f5168100440 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5164009990 tx=0x7f5164009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.201+0000 7f516f7cd700 1 -- 172.21.15.12:0/1233695359 shutdown_connections 2024-05-08T23:47:38.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.201+0000 7f516f7cd700 1 --2- 172.21.15.12:0/1233695359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168105160 0x7f516810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.201+0000 7f516f7cd700 1 --2- 172.21.15.12:0/1233695359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5168100060 0x7f5168100440 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.201+0000 7f516f7cd700 1 --2- 172.21.15.12:0/1233695359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5168100a10 0x7f5168104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.201+0000 7f516f7cd700 1 -- 172.21.15.12:0/1233695359 >> 172.21.15.12:0/1233695359 conn(0x7f5168076f40 msgr2=0x7f5168077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.201+0000 7f516f7cd700 1 -- 172.21.15.12:0/1233695359 shutdown_connections 2024-05-08T23:47:38.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.201+0000 7f516f7cd700 1 -- 172.21.15.12:0/1233695359 wait complete. 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516f7cd700 1 Processor -- start 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516f7cd700 1 -- start start 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516f7cd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5168100060 0x7f51681951e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516f7cd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168105160 0x7f5168195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516f7cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5168199ac0 0x7f5168199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516f7cd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819a540 con 0x7f5168199ac0 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516f7cd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819a6b0 con 0x7f5168100060 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516f7cd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819a870 con 0x7f5168105160 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516dd6a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5168199ac0 0x7f5168199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516dd6a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5168199ac0 0x7f5168199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54454/0 (socket says 172.21.15.12:54454) 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.202+0000 7f516dd6a700 1 -- 172.21.15.12:0/1200031366 learned_addr learned my addr 172.21.15.12:0/1200031366 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f516cd68700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168105160 0x7f5168195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f516d569700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5168100060 0x7f51681951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f516dd6a700 1 -- 172.21.15.12:0/1200031366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168105160 msgr2=0x7f5168195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f516dd6a700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168105160 0x7f5168195720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f516dd6a700 1 -- 172.21.15.12:0/1200031366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5168100060 msgr2=0x7f51681951e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f516dd6a700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5168100060 0x7f51681951e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f516dd6a700 1 -- 172.21.15.12:0/1200031366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160009c40 con 0x7f5168199ac0 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f516cd68700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168105160 0x7f5168195720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f516dd6a700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5168199ac0 0x7f5168199f40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f516000c370 tx=0x7f516000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f515e7fc700 1 -- 172.21.15.12:0/1200031366 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160017070 con 0x7f5168199ac0 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f516f7cd700 1 -- 172.21.15.12:0/1200031366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51640095f0 con 0x7f5168199ac0 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f515e7fc700 1 -- 172.21.15.12:0/1200031366 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f516000f040 con 0x7f5168199ac0 2024-05-08T23:47:38.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.203+0000 7f515e7fc700 1 -- 172.21.15.12:0/1200031366 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000e050 con 0x7f5168199ac0 2024-05-08T23:47:38.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.204+0000 7f516f7cd700 1 -- 172.21.15.12:0/1200031366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51681a20c0 con 0x7f5168199ac0 2024-05-08T23:47:38.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.207+0000 7f515e7fc700 1 -- 172.21.15.12:0/1200031366 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f516000cb60 con 0x7f5168199ac0 2024-05-08T23:47:38.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.208+0000 7f515e7fc700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f51540a57c0 0x7f51540a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.208+0000 7f516d569700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f51540a57c0 0x7f51540a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.208+0000 7f515e7fc700 1 -- 172.21.15.12:0/1200031366 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f51600cc3f0 con 0x7f5168199ac0 2024-05-08T23:47:38.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.208+0000 7f517b987b80 1 Processor -- start 2024-05-08T23:47:38.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.208+0000 7f517b987b80 1 -- start start 2024-05-08T23:47:38.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516d569700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f51540a57c0 0x7f51540a7c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5164006010 tx=0x7f5164005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f517b987b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d62142f50 0x557d62143330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f517b987b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 0x557d6213b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f517b987b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213bcf0 0x557d6215bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f517b987b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d62095150 con 0x557d62142f50 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f517b987b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d62092b10 con 0x557d6213bcf0 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f517b987b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6208e400 con 0x557d6213b330 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516d569700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d62142f50 0x557d62143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516d569700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d62142f50 0x557d62143330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54464/0 (socket says 172.21.15.12:54464) 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516d569700 1 -- 172.21.15.12:0/1939775012 learned_addr learned my addr 172.21.15.12:0/1939775012 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516cd68700 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 0x557d6213b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516dd6a700 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213bcf0 0x557d6215bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516cd68700 1 -- 172.21.15.12:0/1939775012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213bcf0 msgr2=0x557d6215bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516cd68700 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213bcf0 0x557d6215bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516cd68700 1 -- 172.21.15.12:0/1939775012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d62142f50 msgr2=0x557d62143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516cd68700 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d62142f50 0x557d62143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516cd68700 1 -- 172.21.15.12:0/1939775012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d61f0f3b0 con 0x557d6213b330 2024-05-08T23:47:38.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516d569700 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d62142f50 0x557d62143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:38.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.209+0000 7f516dd6a700 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213bcf0 0x557d6215bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:38.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.210+0000 7f516cd68700 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 0x557d6213b7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5158005950 tx=0x7f5158010730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b51efbab35d0f44 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.210+0000 7f51537fe700 1 -- 172.21.15.12:0/1939775012 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515801c480 con 0x557d6213b330 2024-05-08T23:47:38.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.210+0000 7f51537fe700 1 -- 172.21.15.12:0/1939775012 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5158015070 con 0x557d6213b330 2024-05-08T23:47:38.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.210+0000 7f51537fe700 1 -- 172.21.15.12:0/1939775012 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515801a740 con 0x557d6213b330 2024-05-08T23:47:38.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.211+0000 7f517b987b80 1 -- 172.21.15.12:0/1939775012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 msgr2=0x557d6213b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.211+0000 7f517b987b80 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 0x557d6213b7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5158005950 tx=0x7f5158010730 comp rx=0 tx=0).stop 2024-05-08T23:47:38.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.212+0000 7f517b987b80 1 -- 172.21.15.12:0/1939775012 shutdown_connections 2024-05-08T23:47:38.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.213+0000 7f517b987b80 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213bcf0 0x557d6215bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.213+0000 7f517b987b80 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d62142f50 0x557d62143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.213+0000 7f517b987b80 1 --2- 172.21.15.12:0/1939775012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 0x557d6213b7b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.213+0000 7f517b987b80 1 -- 172.21.15.12:0/1939775012 >> 172.21.15.12:0/1939775012 conn(0x557d6209c210 msgr2=0x557d62138df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.213+0000 7f517b987b80 1 -- 172.21.15.12:0/1939775012 shutdown_connections 2024-05-08T23:47:38.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.213+0000 7f517b987b80 1 -- 172.21.15.12:0/1939775012 wait complete. 2024-05-08T23:47:38.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.213+0000 7f517b987b80 1 -- start start 2024-05-08T23:47:38.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.213+0000 7f517b987b80 1 Processor -- start 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f517b987b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213b330 0x557d6215bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f517b987b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f517b987b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d62142f50 0x557d62159560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f517b987b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6202a560 con 0x557d6213bcf0 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f517b987b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6202a200 con 0x557d6213b330 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f517b987b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d61f116a0 con 0x557d62142f50 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f517b987b80 10 client.0 _ll_register_callbacks cb 0x557d62033b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f516cd68700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f516cd68700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54466/0 (socket says 172.21.15.12:54466) 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f516cd68700 1 -- 172.21.15.12:0/1372021277 learned_addr learned my addr 172.21.15.12:0/1372021277 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f516dd6a700 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d62142f50 0x557d62159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f516d569700 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213b330 0x557d6215bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f516cd68700 1 -- 172.21.15.12:0/1372021277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d62142f50 msgr2=0x557d62159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f516cd68700 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d62142f50 0x557d62159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.214+0000 7f516cd68700 1 -- 172.21.15.12:0/1372021277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213b330 msgr2=0x557d6215bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.215+0000 7f516cd68700 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213b330 0x557d6215bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.215+0000 7f516cd68700 1 -- 172.21.15.12:0/1372021277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516810d550 con 0x557d6213bcf0 2024-05-08T23:47:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.215+0000 7f516cd68700 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5158005850 tx=0x7f5158011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb12c5428e61cffc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.215+0000 7f5150ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.217+0000 7f5150ff9700 1 -- 172.21.15.12:0/1372021277 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51580118b0 con 0x557d6213bcf0 2024-05-08T23:47:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.217+0000 7f517b987b80 1 -- 172.21.15.12:0/1372021277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d61fe7270 con 0x557d6213bcf0 2024-05-08T23:47:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.217+0000 7f5150ff9700 1 -- 172.21.15.12:0/1372021277 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5158017070 con 0x557d6213bcf0 2024-05-08T23:47:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.217+0000 7f5150ff9700 1 -- 172.21.15.12:0/1372021277 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5158011a10 con 0x557d6213bcf0 2024-05-08T23:47:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.217+0000 7f517b987b80 10 client.15738 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.217+0000 7f517b987b80 1 -- 172.21.15.12:0/1372021277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f515800bcd0 con 0x557d6213bcf0 2024-05-08T23:47:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.217+0000 7f5150ff9700 1 -- 172.21.15.12:0/1372021277 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d61fe7270 con 0x557d6213bcf0 2024-05-08T23:47:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.217+0000 7f5150ff9700 1 -- 172.21.15.12:0/1372021277 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f515801c480 con 0x557d6213bcf0 2024-05-08T23:47:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.218+0000 7f517b987b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.218+0000 7f517b987b80 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.218+0000 7f517b987b80 20 client.15738 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.218+0000 7f517b987b80 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557d6215ef60 0x557d62161380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.218+0000 7f517b987b80 4 client.15738 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.218+0000 7f517b987b80 1 -- 172.21.15.12:0/1372021277 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d6210d810 con 0x557d6215ef60 2024-05-08T23:47:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.218+0000 7f516d569700 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557d6215ef60 0x557d62161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.218+0000 7f5150ff9700 10 client.15738 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.218+0000 7f516d569700 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557d6215ef60 0x557d62161380 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c405fb3d394e3031 server_cookie=a7c1605cf2382b26 in_seq=0 out_seq=0 2024-05-08T23:47:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.219+0000 7f5150ff9700 1 -- 172.21.15.12:0/1372021277 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d6210d810 con 0x557d6215ef60 2024-05-08T23:47:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.219+0000 7f5150ff9700 10 client.15738 handle_command_reply: tid=0 2024-05-08T23:47:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.219+0000 7f517b987b80 1 client.15738 shutdown 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.219+0000 7f517b987b80 1 -- 172.21.15.12:0/1372021277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 msgr2=0x557d62158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.219+0000 7f517b987b80 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5158005850 tx=0x7f5158011dc0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.220+0000 7f517b987b80 1 -- 172.21.15.12:0/1372021277 shutdown_connections 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.220+0000 7f517b987b80 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d62142f50 0x557d62159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.220+0000 7f517b987b80 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557d6215ef60 0x557d62161380 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.220+0000 7f517b987b80 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d6213b330 0x557d6215bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.220+0000 7f5150ff9700 0 client.15738 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.220+0000 7f517b987b80 1 --2- 172.21.15.12:0/1372021277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158e20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.220+0000 7f517b987b80 1 -- 172.21.15.12:0/1372021277 >> 172.21.15.12:0/1372021277 conn(0x557d6209c210 msgr2=0x557d621385a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.220+0000 7f517b987b80 1 -- 172.21.15.12:0/1372021277 shutdown_connections 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.220+0000 7f517b987b80 1 -- 172.21.15.12:0/1372021277 wait complete. 2024-05-08T23:47:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.221+0000 7f517b987b80 20 client.15738 trim_cache size 0 max 16384 2024-05-08T23:47:38.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f517b987b80 1 Processor -- start 2024-05-08T23:47:38.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f517b987b80 1 -- start start 2024-05-08T23:47:38.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f517b987b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 0x557d62161350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f517b987b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f517b987b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d62142f50 0x557d62175cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f517b987b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d621665c0 con 0x557d6213bcf0 2024-05-08T23:47:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f517b987b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d62140900 con 0x557d62142f50 2024-05-08T23:47:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f517b987b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d62140330 con 0x557d6213b330 2024-05-08T23:47:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f516cd68700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f516cd68700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54482/0 (socket says 172.21.15.12:54482) 2024-05-08T23:47:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f516cd68700 1 -- 172.21.15.12:0/1619991639 learned_addr learned my addr 172.21.15.12:0/1619991639 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f516d569700 1 --2- 172.21.15.12:0/1619991639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 0x557d62161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f516dd6a700 1 --2- 172.21.15.12:0/1619991639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d62142f50 0x557d62175cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f516cd68700 1 -- 172.21.15.12:0/1619991639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 msgr2=0x557d62161350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f516cd68700 1 --2- 172.21.15.12:0/1619991639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 0x557d62161350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f516cd68700 1 -- 172.21.15.12:0/1619991639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d62142f50 msgr2=0x557d62175cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.247+0000 7f516cd68700 1 --2- 172.21.15.12:0/1619991639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d62142f50 0x557d62175cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.248+0000 7f516cd68700 1 -- 172.21.15.12:0/1619991639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d61ebbb60 con 0x557d6213bcf0 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.248+0000 7f516cd68700 1 --2- 172.21.15.12:0/1619991639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158de0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5158005d90 tx=0x7f515800b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d2220dd62e6ddd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.248+0000 7f5153fff700 1 -- 172.21.15.12:0/1619991639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51580114f0 con 0x557d6213bcf0 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.248+0000 7f5153fff700 1 -- 172.21.15.12:0/1619991639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f515801cb20 con 0x557d6213bcf0 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.248+0000 7f5153fff700 1 -- 172.21.15.12:0/1619991639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158011b10 con 0x557d6213bcf0 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.249+0000 7f517b987b80 1 -- 172.21.15.12:0/1619991639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 msgr2=0x557d62158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.249+0000 7f517b987b80 1 --2- 172.21.15.12:0/1619991639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158de0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5158005d90 tx=0x7f515800b670 comp rx=0 tx=0).stop 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.249+0000 7f517b987b80 1 -- 172.21.15.12:0/1619991639 shutdown_connections 2024-05-08T23:47:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.249+0000 7f517b987b80 1 --2- 172.21.15.12:0/1619991639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d62142f50 0x557d62175cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.249+0000 7f517b987b80 1 --2- 172.21.15.12:0/1619991639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213b330 0x557d62161350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.249+0000 7f517b987b80 1 --2- 172.21.15.12:0/1619991639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213bcf0 0x557d62158de0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.249+0000 7f517b987b80 1 -- 172.21.15.12:0/1619991639 >> 172.21.15.12:0/1619991639 conn(0x557d6209c210 msgr2=0x557d6213d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.249+0000 7f517b987b80 1 -- 172.21.15.12:0/1619991639 shutdown_connections 2024-05-08T23:47:38.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.249+0000 7f517b987b80 1 -- 172.21.15.12:0/1619991639 wait complete. 2024-05-08T23:47:38.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.249+0000 7f517b987b80 1 -- start start 2024-05-08T23:47:38.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f517b987b80 1 Processor -- start 2024-05-08T23:47:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f517b987b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213b330 0x557d6216cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f517b987b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213bcf0 0x557d6216d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f517b987b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d62142f50 0x557d6216d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f517b987b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d621e0590 con 0x557d6213b330 2024-05-08T23:47:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f517b987b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d621e0700 con 0x557d62142f50 2024-05-08T23:47:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f517b987b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d621e08c0 con 0x557d6213bcf0 2024-05-08T23:47:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f517b987b80 10 client.0 _ll_register_callbacks cb 0x557d62033b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f516d569700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213b330 0x557d6216cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f516d569700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213b330 0x557d6216cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54486/0 (socket says 172.21.15.12:54486) 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f516d569700 1 -- 172.21.15.12:0/602159845 learned_addr learned my addr 172.21.15.12:0/602159845 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.250+0000 7f516cd68700 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213bcf0 0x557d6216d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.251+0000 7f516dd6a700 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d62142f50 0x557d6216d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.251+0000 7f516cd68700 1 -- 172.21.15.12:0/602159845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d62142f50 msgr2=0x557d6216d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.251+0000 7f516cd68700 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d62142f50 0x557d6216d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.251+0000 7f516cd68700 1 -- 172.21.15.12:0/602159845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213b330 msgr2=0x557d6216cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.251+0000 7f516cd68700 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213b330 0x557d6216cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.251+0000 7f516cd68700 1 -- 172.21.15.12:0/602159845 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515800bcd0 con 0x557d6213bcf0 2024-05-08T23:47:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.251+0000 7f516d569700 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213b330 0x557d6216cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.251+0000 7f516cd68700 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213bcf0 0x557d6216d250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x557d6215ed90 tx=0x7f515801c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c72379b8b79fa76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.251+0000 7f5150ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.251+0000 7f5150ff9700 1 -- 172.21.15.12:0/602159845 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158011ac0 con 0x557d6213bcf0 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.252+0000 7f5150ff9700 1 -- 172.21.15.12:0/602159845 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5158017040 con 0x557d6213bcf0 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.252+0000 7f517b987b80 1 -- 172.21.15.12:0/602159845 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5158015070 con 0x557d6213bcf0 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.252+0000 7f5150ff9700 1 -- 172.21.15.12:0/602159845 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5158011c20 con 0x557d6213bcf0 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.252+0000 7f517b987b80 10 client.25594 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.252+0000 7f517b987b80 1 -- 172.21.15.12:0/602159845 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557d62054560 con 0x557d6213bcf0 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.252+0000 7f5150ff9700 1 -- 172.21.15.12:0/602159845 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5158015070 con 0x557d6213bcf0 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.252+0000 7f5150ff9700 1 -- 172.21.15.12:0/602159845 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5158003b20 con 0x557d6213bcf0 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.253+0000 7f517b987b80 10 client.25594 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.253+0000 7f517b987b80 10 client.25594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.253+0000 7f517b987b80 20 client.25594 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.253+0000 7f517b987b80 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557d621e1bc0 0x557d621e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.253+0000 7f517b987b80 4 client.25594 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.253+0000 7f517b987b80 1 -- 172.21.15.12:0/602159845 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d62086ec0 con 0x557d621e1bc0 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.253+0000 7f516d569700 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557d621e1bc0 0x557d621e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.253+0000 7f516d569700 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557d621e1bc0 0x557d621e3fb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d78050cc74736e27 server_cookie=3c092bfc8c29ad5e in_seq=0 out_seq=0 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.253+0000 7f5150ff9700 10 client.25594 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f5150ff9700 1 -- 172.21.15.12:0/602159845 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d62086ec0 con 0x557d621e1bc0 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f5150ff9700 10 client.25594 handle_command_reply: tid=0 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 client.25594 shutdown 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 -- 172.21.15.12:0/602159845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213bcf0 msgr2=0x557d6216d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213bcf0 0x557d6216d250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x557d6215ed90 tx=0x7f515801c6b0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 -- 172.21.15.12:0/602159845 shutdown_connections 2024-05-08T23:47:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557d621e1bc0 0x557d621e3fb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557d62142f50 0x557d6216d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d6213b330 0x557d6216cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f5150ff9700 0 client.25594 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 --2- 172.21.15.12:0/602159845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d6213bcf0 0x557d6216d250 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 -- 172.21.15.12:0/602159845 >> 172.21.15.12:0/602159845 conn(0x557d6209c210 msgr2=0x557d62145b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 -- 172.21.15.12:0/602159845 shutdown_connections 2024-05-08T23:47:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 1 -- 172.21.15.12:0/602159845 wait complete. 2024-05-08T23:47:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.254+0000 7f517b987b80 20 client.25594 trim_cache size 0 max 16384 2024-05-08T23:47:38.255 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:47:38.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.255+0000 7f516f7cd700 1 -- 172.21.15.12:0/1200031366 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f51540a57c0 msgr2=0x7f51540a7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.255+0000 7f516f7cd700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f51540a57c0 0x7f51540a7c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5164006010 tx=0x7f5164005960 comp rx=0 tx=0).stop 2024-05-08T23:47:38.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.255+0000 7f516f7cd700 1 -- 172.21.15.12:0/1200031366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5168199ac0 msgr2=0x7f5168199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.255+0000 7f516f7cd700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5168199ac0 0x7f5168199f40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f516000c370 tx=0x7f516000c680 comp rx=0 tx=0).stop 2024-05-08T23:47:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.256+0000 7f516f7cd700 1 -- 172.21.15.12:0/1200031366 shutdown_connections 2024-05-08T23:47:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.256+0000 7f516f7cd700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f51540a57c0 0x7f51540a7c80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.256+0000 7f516f7cd700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5168199ac0 0x7f5168199f40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.256+0000 7f516f7cd700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5168100060 0x7f51681951e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.256+0000 7f516f7cd700 1 --2- 172.21.15.12:0/1200031366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168105160 0x7f5168195720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.256+0000 7f516f7cd700 1 -- 172.21.15.12:0/1200031366 >> 172.21.15.12:0/1200031366 conn(0x7f5168076f40 msgr2=0x7f51680fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.256+0000 7f516f7cd700 1 -- 172.21.15.12:0/1200031366 shutdown_connections 2024-05-08T23:47:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:38.256+0000 7f516f7cd700 1 -- 172.21.15.12:0/1200031366 wait complete. 2024-05-08T23:47:38.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:38 smithi012 ceph-mon[42503]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T23:47:38.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:38 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/4010923531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:47:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:38 smithi204 ceph-mon[47373]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T23:47:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:38 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/4010923531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:47:38.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:38 smithi160 ceph-mon[46152]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T23:47:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:38 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/4010923531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:47:39.271 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:47:39.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:47:39.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.497+0000 7f5b0c338700 1 -- 172.21.15.12:0/4184402978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b040ff410 msgr2=0x7f5b040ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.497+0000 7f5b0c338700 1 --2- 172.21.15.12:0/4184402978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b040ff410 0x7f5b040ff890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009990 tx=0x7f5af4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.497+0000 7f5b0c338700 1 -- 172.21.15.12:0/4184402978 shutdown_connections 2024-05-08T23:47:39.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.497+0000 7f5b0c338700 1 --2- 172.21.15.12:0/4184402978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b040ffdd0 0x7f5b0410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.497+0000 7f5b0c338700 1 --2- 172.21.15.12:0/4184402978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b04105ef0 0x7f5b041062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.497+0000 7f5b0c338700 1 --2- 172.21.15.12:0/4184402978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b040ff410 0x7f5b040ff890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.497+0000 7f5b0c338700 1 -- 172.21.15.12:0/4184402978 >> 172.21.15.12:0/4184402978 conn(0x7f5b040763a0 msgr2=0x7f5b040fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.497+0000 7f5b0c338700 1 -- 172.21.15.12:0/4184402978 shutdown_connections 2024-05-08T23:47:39.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.497+0000 7f5b0c338700 1 -- 172.21.15.12:0/4184402978 wait complete. 2024-05-08T23:47:39.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.498+0000 7f5b0c338700 1 Processor -- start 2024-05-08T23:47:39.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.498+0000 7f5b0c338700 1 -- start start 2024-05-08T23:47:39.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0c338700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b040ff410 0x7f5b0410cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0c338700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b040ffdd0 0x7f5b0410d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0c338700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b04105ef0 0x7f5b041b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0c338700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b041b23b0 con 0x7f5b040ff410 2024-05-08T23:47:39.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0c338700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0406a9f0 con 0x7f5b040ffdd0 2024-05-08T23:47:39.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0c338700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0406abb0 con 0x7f5b04105ef0 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0a8d5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b04105ef0 0x7f5b041b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b098d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b040ffdd0 0x7f5b0410d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0a8d5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b04105ef0 0x7f5b041b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38660/0 (socket says 172.21.15.12:38660) 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b098d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b040ffdd0 0x7f5b0410d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52908/0 (socket says 172.21.15.12:52908) 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2354306256 learned_addr learned my addr 172.21.15.12:0/2354306256 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0a0d4700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b040ff410 0x7f5b0410cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2354306256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b040ffdd0 msgr2=0x7f5b0410d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b040ffdd0 0x7f5b0410d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2354306256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b040ff410 msgr2=0x7f5b0410cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b040ff410 0x7f5b0410cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.499+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2354306256 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40095f0 con 0x7f5b04105ef0 2024-05-08T23:47:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.500+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b04105ef0 0x7f5b041b1de0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00ef30 tx=0x7f5afc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.500+0000 7f5afb7fe700 1 -- 172.21.15.12:0/2354306256 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc018070 con 0x7f5b04105ef0 2024-05-08T23:47:39.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.500+0000 7f5afb7fe700 1 -- 172.21.15.12:0/2354306256 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5afc00f040 con 0x7f5b04105ef0 2024-05-08T23:47:39.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.500+0000 7f5afb7fe700 1 -- 172.21.15.12:0/2354306256 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00cdb0 con 0x7f5b04105ef0 2024-05-08T23:47:39.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.500+0000 7f5b0c338700 1 -- 172.21.15.12:0/2354306256 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0406ae30 con 0x7f5b04105ef0 2024-05-08T23:47:39.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.500+0000 7f5b0c338700 1 -- 172.21.15.12:0/2354306256 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0406b2f0 con 0x7f5b04105ef0 2024-05-08T23:47:39.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.503+0000 7f5afb7fe700 1 -- 172.21.15.12:0/2354306256 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f5afc003a00 con 0x7f5b04105ef0 2024-05-08T23:47:39.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.504+0000 7f5afb7fe700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5af00a13b0 0x7f5af00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.504+0000 7f5afb7fe700 1 -- 172.21.15.12:0/2354306256 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5afc0ccbe0 con 0x7f5b04105ef0 2024-05-08T23:47:39.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.504+0000 7f5b0a0d4700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5af00a13b0 0x7f5af00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.505+0000 7f5b0a0d4700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5af00a13b0 0x7f5af00a3870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009bf0 tx=0x7f5b00009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.509+0000 7f5b184f2b80 1 Processor -- start 2024-05-08T23:47:39.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.509+0000 7f5b184f2b80 1 -- start start 2024-05-08T23:47:39.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.509+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.509+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.509+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.509+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c2071d0 con 0x56342c2b0910 2024-05-08T23:47:39.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.509+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c204b90 con 0x56342c2b67a0 2024-05-08T23:47:39.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.509+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c200480 con 0x56342c2b5e80 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b0a8d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b0a8d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48368/0 (socket says 172.21.15.12:48368) 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2335326892 learned_addr learned my addr 172.21.15.12:0/2335326892 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b0a0d4700 1 --2- 172.21.15.12:0/2335326892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b098d3700 1 --2- 172.21.15.12:0/2335326892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2335326892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 msgr2=0x56342c2b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2335326892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2335326892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 msgr2=0x56342c2b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2335326892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2335326892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56342c081430 con 0x56342c2b0910 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.510+0000 7f5b098d3700 1 --2- 172.21.15.12:0/2335326892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.511+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2335326892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b0d50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5afc096010 tx=0x7f5afc099650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5245251f2ba206 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.511+0000 7f5af8da4700 1 -- 172.21.15.12:0/2335326892 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc01b320 con 0x56342c2b0910 2024-05-08T23:47:39.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.511+0000 7f5af8da4700 1 -- 172.21.15.12:0/2335326892 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5afc09a210 con 0x56342c2b0910 2024-05-08T23:47:39.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.511+0000 7f5af8da4700 1 -- 172.21.15.12:0/2335326892 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc099d70 con 0x56342c2b0910 2024-05-08T23:47:39.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.512+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2335326892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 msgr2=0x56342c2b0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.512+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2335326892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b0d50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5afc096010 tx=0x7f5afc099650 comp rx=0 tx=0).stop 2024-05-08T23:47:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.512+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2335326892 shutdown_connections 2024-05-08T23:47:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.512+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2335326892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b0d50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.512+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2335326892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.512+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2335326892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.512+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2335326892 >> 172.21.15.12:0/2335326892 conn(0x56342c20e290 msgr2=0x56342c2aae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.512+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2335326892 shutdown_connections 2024-05-08T23:47:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.512+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2335326892 wait complete. 2024-05-08T23:47:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.513+0000 7f5b184f2b80 1 -- start start 2024-05-08T23:47:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.513+0000 7f5b184f2b80 1 Processor -- start 2024-05-08T23:47:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.514+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.514+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2ba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.514+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2ba660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.514+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c19c5e0 con 0x56342c2b0910 2024-05-08T23:47:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.514+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c19c280 con 0x56342c2b67a0 2024-05-08T23:47:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.514+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c083720 con 0x56342c2b5e80 2024-05-08T23:47:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.514+0000 7f5b184f2b80 10 client.0 _ll_register_callbacks cb 0x56342c1a5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b0a0d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b0a0d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48376/0 (socket says 172.21.15.12:48376) 2024-05-08T23:47:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b0a0d4700 1 -- 172.21.15.12:0/175902654 learned_addr learned my addr 172.21.15.12:0/175902654 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b098d3700 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2ba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2ba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b098d3700 1 -- 172.21.15.12:0/175902654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 msgr2=0x56342c2ba660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b098d3700 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2ba660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b098d3700 1 -- 172.21.15.12:0/175902654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 msgr2=0x56342c2b9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b098d3700 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b098d3700 1 -- 172.21.15.12:0/175902654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0406b2f0 con 0x56342c2b5e80 2024-05-08T23:47:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2ba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:47:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5b098d3700 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2ba120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5af400b2e0 tx=0x7f5af4005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4020b963485611f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.515+0000 7f5aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:47:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5aedffb700 1 -- 172.21.15.12:0/175902654 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af401d070 con 0x56342c2b5e80 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5aedffb700 1 -- 172.21.15.12:0/175902654 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5af40058e0 con 0x56342c2b5e80 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5b184f2b80 1 -- 172.21.15.12:0/175902654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56342c1592f0 con 0x56342c2b5e80 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5aedffb700 1 -- 172.21.15.12:0/175902654 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af4004500 con 0x56342c2b5e80 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5b184f2b80 10 client.25606 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5b184f2b80 1 -- 172.21.15.12:0/175902654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56342c02dbe0 con 0x56342c2b5e80 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5aedffb700 1 -- 172.21.15.12:0/175902654 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56342c1592f0 con 0x56342c2b5e80 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5aedffb700 1 -- 172.21.15.12:0/175902654 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5af4024640 con 0x56342c2b5e80 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5b184f2b80 10 client.25606 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5b184f2b80 10 client.25606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5b184f2b80 20 client.25606 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56342c2ca2a0 0x56342c2cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.516+0000 7f5b184f2b80 4 client.25606 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.517+0000 7f5b184f2b80 1 -- 172.21.15.12:0/175902654 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56342c27f890 con 0x56342c2ca2a0 2024-05-08T23:47:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.517+0000 7f5b0a0d4700 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56342c2ca2a0 0x56342c2cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.517+0000 7f5aedffb700 10 client.25606 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:39.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.517+0000 7f5b0a0d4700 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56342c2ca2a0 0x56342c2cc6c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd41941483bebede server_cookie=ec9c3e2758fbf2c2 in_seq=0 out_seq=0 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5aedffb700 1 -- 172.21.15.12:0/175902654 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56342c27f890 con 0x56342c2ca2a0 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5aedffb700 10 client.25606 handle_command_reply: tid=0 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5b184f2b80 1 client.25606 shutdown 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5b184f2b80 1 -- 172.21.15.12:0/175902654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 msgr2=0x56342c2ba120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2ba120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5af400b2e0 tx=0x7f5af4005750 comp rx=0 tx=0).stop 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5b184f2b80 1 -- 172.21.15.12:0/175902654 shutdown_connections 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56342c2ca2a0 0x56342c2cc6c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2ba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2b9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/175902654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2ba120 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5aedffb700 0 client.25606 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5b184f2b80 1 -- 172.21.15.12:0/175902654 >> 172.21.15.12:0/175902654 conn(0x56342c20e290 msgr2=0x56342c2aa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.518+0000 7f5b184f2b80 1 -- 172.21.15.12:0/175902654 shutdown_connections 2024-05-08T23:47:39.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.519+0000 7f5b184f2b80 1 -- 172.21.15.12:0/175902654 wait complete. 2024-05-08T23:47:39.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.519+0000 7f5b184f2b80 20 client.25606 trim_cache size 0 max 16384 2024-05-08T23:47:39.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.554+0000 7f5b184f2b80 1 Processor -- start 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.554+0000 7f5b184f2b80 1 -- start start 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b0910 0x56342c2b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b5e80 0x56342c2b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b67a0 0x56342c2aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c2bb200 con 0x56342c2b5e80 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c2abf00 con 0x56342c2b0910 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c2ac0f0 con 0x56342c2b67a0 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b0a8d5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b67a0 0x56342c2aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b0a8d5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b67a0 0x56342c2aada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38688/0 (socket says 172.21.15.12:38688) 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2183989835 learned_addr learned my addr 172.21.15.12:0/2183989835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b098d3700 1 --2- 172.21.15.12:0/2183989835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b5e80 0x56342c2b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2183989835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b0910 msgr2=0x56342c2b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2183989835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b0910 0x56342c2b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.555+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2183989835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b5e80 msgr2=0x56342c2b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.556+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2183989835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b5e80 0x56342c2b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.556+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2183989835 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56342c1c65e0 con 0x56342c2b67a0 2024-05-08T23:47:39.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.556+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2183989835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b67a0 0x56342c2aada0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0a19d0 tx=0x7f5afc009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f966846b8d6ebfd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.556+0000 7f5af95a5700 1 -- 172.21.15.12:0/2183989835 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc01b320 con 0x56342c2b67a0 2024-05-08T23:47:39.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.556+0000 7f5af95a5700 1 -- 172.21.15.12:0/2183989835 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5afc09d940 con 0x56342c2b67a0 2024-05-08T23:47:39.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.556+0000 7f5af95a5700 1 -- 172.21.15.12:0/2183989835 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc099260 con 0x56342c2b67a0 2024-05-08T23:47:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.557+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2183989835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b67a0 msgr2=0x56342c2aada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.557+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2183989835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b67a0 0x56342c2aada0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0a19d0 tx=0x7f5afc009ed0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.557+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2183989835 shutdown_connections 2024-05-08T23:47:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.557+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2183989835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b67a0 0x56342c2aada0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.557+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2183989835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b0910 0x56342c2b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.557+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2183989835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b5e80 0x56342c2b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.557+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2183989835 >> 172.21.15.12:0/2183989835 conn(0x56342c20e290 msgr2=0x56342c2b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.557+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2183989835 shutdown_connections 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.557+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2183989835 wait complete. 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.558+0000 7f5b184f2b80 1 -- start start 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.558+0000 7f5b184f2b80 1 Processor -- start 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.558+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.558+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2c1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.558+0000 7f5b184f2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.558+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c2b25c0 con 0x56342c2b0910 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.558+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c2b2730 con 0x56342c2b67a0 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.558+0000 7f5b184f2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342c2b28f0 con 0x56342c2b5e80 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.558+0000 7f5b184f2b80 10 client.0 _ll_register_callbacks cb 0x56342c1a5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.559+0000 7f5b0a8d5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.559+0000 7f5b0a8d5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52942/0 (socket says 172.21.15.12:52942) 2024-05-08T23:47:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.559+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2858102259 learned_addr learned my addr 172.21.15.12:0/2858102259 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.559+0000 7f5b098d3700 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.559+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2858102259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 msgr2=0x56342c2c1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.559+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2c1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.559+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2858102259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 msgr2=0x56342c2c11f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:47:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.559+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2c11f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.559+0000 7f5b0a8d5700 1 -- 172.21.15.12:0/2858102259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56342c081430 con 0x56342c2b67a0 2024-05-08T23:47:39.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.559+0000 7f5b0a8d5700 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b1ee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00c7a0 tx=0x7f5afc09a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e29af4bacc20791 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.560+0000 7f5aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:47:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.560+0000 7f5aedffb700 1 -- 172.21.15.12:0/2858102259 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc099470 con 0x56342c2b67a0 2024-05-08T23:47:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.560+0000 7f5aedffb700 1 -- 172.21.15.12:0/2858102259 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5afc0a1f00 con 0x56342c2b67a0 2024-05-08T23:47:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.560+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2858102259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5afc09a210 con 0x56342c2b67a0 2024-05-08T23:47:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.560+0000 7f5aedffb700 1 -- 172.21.15.12:0/2858102259 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5afc099000 con 0x56342c2b67a0 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.560+0000 7f5b184f2b80 10 client.25607 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.560+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2858102259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56342c085d50 con 0x56342c2b67a0 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.560+0000 7f5aedffb700 1 -- 172.21.15.12:0/2858102259 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5afc09a210 con 0x56342c2b67a0 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.560+0000 7f5aedffb700 1 -- 172.21.15.12:0/2858102259 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5afc0ca470 con 0x56342c2b67a0 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.560+0000 7f5b184f2b80 10 client.25607 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.561+0000 7f5b184f2b80 10 client.25607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.561+0000 7f5b184f2b80 20 client.25607 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.561+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56342c321640 0x56342c323a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.561+0000 7f5b184f2b80 4 client.25607 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.561+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2858102259 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56342c1f8f40 con 0x56342c321640 2024-05-08T23:47:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.561+0000 7f5b0a0d4700 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56342c321640 0x56342c323a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.561+0000 7f5aedffb700 10 client.25607 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.561+0000 7f5b0a0d4700 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56342c321640 0x56342c323a30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f180d9333e67ed3f server_cookie=4f2aa5a9b51af74c in_seq=0 out_seq=0 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5aedffb700 1 -- 172.21.15.12:0/2858102259 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56342c1f8f40 con 0x56342c321640 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5aedffb700 10 client.25607 handle_command_reply: tid=0 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5b184f2b80 1 client.25607 shutdown 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2858102259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 msgr2=0x56342c2b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b1ee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00c7a0 tx=0x7f5afc09a380 comp rx=0 tx=0).stop 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2858102259 shutdown_connections 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56342c321640 0x56342c323a30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56342c2b67a0 0x56342c2b1ee0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56342c2b0910 0x56342c2c11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5b184f2b80 1 --2- 172.21.15.12:0/2858102259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56342c2b5e80 0x56342c2c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2858102259 >> 172.21.15.12:0/2858102259 conn(0x56342c20e290 msgr2=0x56342c2b6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5aedffb700 0 client.25607 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.563+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2858102259 shutdown_connections 2024-05-08T23:47:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b184f2b80 1 -- 172.21.15.12:0/2858102259 wait complete. 2024-05-08T23:47:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b184f2b80 20 client.25607 trim_cache size 0 max 16384 2024-05-08T23:47:39.566 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:47:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:47:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "1a632a93-be7f-42c9-9171-fbbf64559c57", 2024-05-08T23:47:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:47:39.566 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:47:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b0c338700 1 -- 172.21.15.12:0/2354306256 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5af00a13b0 msgr2=0x7f5af00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b0c338700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5af00a13b0 0x7f5af00a3870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009bf0 tx=0x7f5b00009260 comp rx=0 tx=0).stop 2024-05-08T23:47:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b0c338700 1 -- 172.21.15.12:0/2354306256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b04105ef0 msgr2=0x7f5b041b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b0c338700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b04105ef0 0x7f5b041b1de0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00ef30 tx=0x7f5afc00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b0c338700 1 -- 172.21.15.12:0/2354306256 shutdown_connections 2024-05-08T23:47:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b0c338700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5af00a13b0 0x7f5af00a3870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b0c338700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b04105ef0 0x7f5b041b1de0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b0c338700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b040ff410 0x7f5b0410cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b0c338700 1 --2- 172.21.15.12:0/2354306256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b040ffdd0 0x7f5b0410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.564+0000 7f5b0c338700 1 -- 172.21.15.12:0/2354306256 >> 172.21.15.12:0/2354306256 conn(0x7f5b040763a0 msgr2=0x7f5b04101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.569+0000 7f5b0c338700 1 -- 172.21.15.12:0/2354306256 shutdown_connections 2024-05-08T23:47:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.569+0000 7f5b0c338700 1 -- 172.21.15.12:0/2354306256 wait complete. 2024-05-08T23:47:39.582 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:47:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.732+0000 7f373ee2b700 1 -- 172.21.15.12:0/525524473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37381062d0 msgr2=0x7f37380696d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.732+0000 7f373ee2b700 1 --2- 172.21.15.12:0/525524473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37381062d0 0x7f37380696d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3724009990 tx=0x7f3724009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.732+0000 7f373ee2b700 1 -- 172.21.15.12:0/525524473 shutdown_connections 2024-05-08T23:47:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.732+0000 7f373ee2b700 1 --2- 172.21.15.12:0/525524473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3738103960 0x7f3738103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.732+0000 7f373ee2b700 1 --2- 172.21.15.12:0/525524473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3738105e60 0x7f3738069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.732+0000 7f373ee2b700 1 --2- 172.21.15.12:0/525524473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37381062d0 0x7f37380696d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.732+0000 7f373ee2b700 1 -- 172.21.15.12:0/525524473 >> 172.21.15.12:0/525524473 conn(0x7f3738076ba0 msgr2=0x7f3738076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.732+0000 7f373ee2b700 1 -- 172.21.15.12:0/525524473 shutdown_connections 2024-05-08T23:47:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.732+0000 7f373ee2b700 1 -- 172.21.15.12:0/525524473 wait complete. 2024-05-08T23:47:39.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.733+0000 7f373ee2b700 1 Processor -- start 2024-05-08T23:47:39.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.733+0000 7f373ee2b700 1 -- start start 2024-05-08T23:47:39.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.733+0000 7f373ee2b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3738103960 0x7f37381a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.733+0000 7f373ee2b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3738105e60 0x7f37381a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.733+0000 7f373ee2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37381062d0 0x7f373819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.733+0000 7f373ee2b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373819ddb0 con 0x7f37381062d0 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.733+0000 7f373ee2b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373819df20 con 0x7f3738105e60 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.733+0000 7f373ee2b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373819e0e0 con 0x7f3738103960 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373d3c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37381062d0 0x7f373819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373d3c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37381062d0 0x7f373819d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48406/0 (socket says 172.21.15.12:48406) 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373d3c8700 1 -- 172.21.15.12:0/2596110397 learned_addr learned my addr 172.21.15.12:0/2596110397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373cbc7700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3738103960 0x7f37381a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373d3c8700 1 -- 172.21.15.12:0/2596110397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3738103960 msgr2=0x7f37381a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373d3c8700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3738103960 0x7f37381a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373d3c8700 1 -- 172.21.15.12:0/2596110397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3738105e60 msgr2=0x7f37381a3b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373d3c8700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3738105e60 0x7f37381a3b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373d3c8700 1 -- 172.21.15.12:0/2596110397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240095f0 con 0x7f37381062d0 2024-05-08T23:47:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373cbc7700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3738103960 0x7f37381a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f373d3c8700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37381062d0 0x7f373819d750 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f373000ef30 tx=0x7f373000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.734+0000 7f3735ffb700 1 -- 172.21.15.12:0/2596110397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730018070 con 0x7f37381062d0 2024-05-08T23:47:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.735+0000 7f373ee2b700 1 -- 172.21.15.12:0/2596110397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373819e360 con 0x7f37381062d0 2024-05-08T23:47:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.735+0000 7f3735ffb700 1 -- 172.21.15.12:0/2596110397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f373000f040 con 0x7f37381062d0 2024-05-08T23:47:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.735+0000 7f3735ffb700 1 -- 172.21.15.12:0/2596110397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373000cdb0 con 0x7f37381062d0 2024-05-08T23:47:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.735+0000 7f373ee2b700 1 -- 172.21.15.12:0/2596110397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37381aa610 con 0x7f37381062d0 2024-05-08T23:47:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.738+0000 7f3735ffb700 1 -- 172.21.15.12:0/2596110397 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f3730004000 con 0x7f37381062d0 2024-05-08T23:47:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.739+0000 7f3735ffb700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37280a1430 0x7f37280a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.739+0000 7f3735ffb700 1 -- 172.21.15.12:0/2596110397 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f37300ccb10 con 0x7f37381062d0 2024-05-08T23:47:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.740+0000 7f374afe5b80 1 Processor -- start 2024-05-08T23:47:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.740+0000 7f374afe5b80 1 -- start start 2024-05-08T23:47:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.740+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3abe00 0x55812d3ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.740+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.740+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.740+0000 7f374afe5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d2fd150 con 0x55812d3abe00 2024-05-08T23:47:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.740+0000 7f374afe5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d2fab10 con 0x55812d3ac720 2024-05-08T23:47:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.740+0000 7f374afe5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d2f6400 con 0x55812d3a6890 2024-05-08T23:47:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373cbc7700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37280a1430 0x7f37280a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373d3c8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373cbc7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3abe00 0x55812d3ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373d3c8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38732/0 (socket says 172.21.15.12:38732) 2024-05-08T23:47:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373cbc7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3abe00 0x55812d3ac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48422/0 (socket says 172.21.15.12:48422) 2024-05-08T23:47:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373d3c8700 1 -- 172.21.15.12:0/518163880 learned_addr learned my addr 172.21.15.12:0/518163880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f3737fff700 1 --2- 172.21.15.12:0/518163880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373d3c8700 1 -- 172.21.15.12:0/518163880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 msgr2=0x55812d3a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373d3c8700 1 --2- 172.21.15.12:0/518163880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373d3c8700 1 -- 172.21.15.12:0/518163880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3abe00 msgr2=0x55812d3ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373d3c8700 1 --2- 172.21.15.12:0/518163880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3abe00 0x55812d3ac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.741+0000 7f373d3c8700 1 -- 172.21.15.12:0/518163880 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55812d1773b0 con 0x55812d3a6890 2024-05-08T23:47:39.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.742+0000 7f373d3c8700 1 --2- 172.21.15.12:0/518163880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6cd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f373000c7a0 tx=0x7f3730099870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5e577c360a01eb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.742+0000 7f3722ffd700 1 -- 172.21.15.12:0/518163880 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730004ad0 con 0x55812d3a6890 2024-05-08T23:47:39.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.742+0000 7f3722ffd700 1 -- 172.21.15.12:0/518163880 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f373009a1a0 con 0x55812d3a6890 2024-05-08T23:47:39.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.742+0000 7f3722ffd700 1 -- 172.21.15.12:0/518163880 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730099d00 con 0x55812d3a6890 2024-05-08T23:47:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f374afe5b80 1 -- 172.21.15.12:0/518163880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 msgr2=0x55812d3a6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f374afe5b80 1 --2- 172.21.15.12:0/518163880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6cd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f373000c7a0 tx=0x7f3730099870 comp rx=0 tx=0).stop 2024-05-08T23:47:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f373cbc7700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37280a1430 0x7f37280a38f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f372c0097c0 tx=0x7f372c009750 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f374afe5b80 1 -- 172.21.15.12:0/518163880 shutdown_connections 2024-05-08T23:47:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f374afe5b80 1 --2- 172.21.15.12:0/518163880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6cd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f374afe5b80 1 --2- 172.21.15.12:0/518163880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3abe00 0x55812d3ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f374afe5b80 1 --2- 172.21.15.12:0/518163880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f374afe5b80 1 -- 172.21.15.12:0/518163880 >> 172.21.15.12:0/518163880 conn(0x55812d304210 msgr2=0x55812d3a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f374afe5b80 1 -- 172.21.15.12:0/518163880 shutdown_connections 2024-05-08T23:47:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f374afe5b80 1 -- 172.21.15.12:0/518163880 wait complete. 2024-05-08T23:47:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.743+0000 7f374afe5b80 1 -- start start 2024-05-08T23:47:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.744+0000 7f374afe5b80 1 Processor -- start 2024-05-08T23:47:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.744+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3a6890 0x55812d3afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.744+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3abe00 0x55812d3b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.744+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.744+0000 7f374afe5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d292560 con 0x55812d3a6890 2024-05-08T23:47:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.744+0000 7f374afe5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d292200 con 0x55812d3ac720 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.744+0000 7f374afe5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d1796a0 con 0x55812d3abe00 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.744+0000 7f374afe5b80 10 client.0 _ll_register_callbacks cb 0x55812d29bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f373d3c8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f373d3c8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52954/0 (socket says 172.21.15.12:52954) 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f373d3c8700 1 -- 172.21.15.12:0/3691958022 learned_addr learned my addr 172.21.15.12:0/3691958022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f373cbc7700 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3a6890 0x55812d3afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f3737fff700 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3abe00 0x55812d3b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f373d3c8700 1 -- 172.21.15.12:0/3691958022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3abe00 msgr2=0x55812d3b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f373d3c8700 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3abe00 0x55812d3b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f373d3c8700 1 -- 172.21.15.12:0/3691958022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3a6890 msgr2=0x55812d3afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f373d3c8700 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3a6890 0x55812d3afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f373d3c8700 1 -- 172.21.15.12:0/3691958022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37381aa610 con 0x55812d3ac720 2024-05-08T23:47:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f373d3c8700 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3b0600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3730099a30 tx=0x7f373009a310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa421df3ac74575f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.745+0000 7f3717fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f3717fff700 1 -- 172.21.15.12:0/3691958022 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730004ad0 con 0x55812d3ac720 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f3717fff700 1 -- 172.21.15.12:0/3691958022 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3730099aa0 con 0x55812d3ac720 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f374afe5b80 1 -- 172.21.15.12:0/3691958022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55812d24f270 con 0x55812d3ac720 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f3717fff700 1 -- 172.21.15.12:0/3691958022 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37300a1e90 con 0x55812d3ac720 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f374afe5b80 10 client.25616 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f374afe5b80 1 -- 172.21.15.12:0/3691958022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55812d123b60 con 0x55812d3ac720 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f3717fff700 1 -- 172.21.15.12:0/3691958022 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55812d24f270 con 0x55812d3ac720 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f3717fff700 1 -- 172.21.15.12:0/3691958022 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f37300ca490 con 0x55812d3ac720 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f374afe5b80 10 client.25616 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f374afe5b80 10 client.25616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.746+0000 7f374afe5b80 20 client.25616 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.747+0000 7f374afe5b80 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55812d3d0d70 0x55812d3d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.747+0000 7f374afe5b80 4 client.25616 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.747+0000 7f374afe5b80 1 -- 172.21.15.12:0/3691958022 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55812d375810 con 0x55812d3d0d70 2024-05-08T23:47:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.747+0000 7f373cbc7700 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55812d3d0d70 0x55812d3d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.747+0000 7f3717fff700 10 client.25616 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.747+0000 7f373cbc7700 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55812d3d0d70 0x55812d3d3160 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8a960454d60788 server_cookie=fbdfbe794d29101e in_seq=0 out_seq=0 2024-05-08T23:47:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f3717fff700 1 -- 172.21.15.12:0/3691958022 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55812d375810 con 0x55812d3d0d70 2024-05-08T23:47:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f3717fff700 10 client.25616 handle_command_reply: tid=0 2024-05-08T23:47:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f374afe5b80 1 client.25616 shutdown 2024-05-08T23:47:39.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f374afe5b80 1 -- 172.21.15.12:0/3691958022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 msgr2=0x55812d3b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f374afe5b80 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3b0600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3730099a30 tx=0x7f373009a310 comp rx=0 tx=0).stop 2024-05-08T23:47:39.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f374afe5b80 1 -- 172.21.15.12:0/3691958022 shutdown_connections 2024-05-08T23:47:39.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f374afe5b80 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55812d3d0d70 0x55812d3d3160 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f374afe5b80 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3b0600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f3717fff700 0 client.25616 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:39.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f374afe5b80 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3a6890 0x55812d3afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f374afe5b80 1 --2- 172.21.15.12:0/3691958022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3abe00 0x55812d3b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.748+0000 7f374afe5b80 1 -- 172.21.15.12:0/3691958022 >> 172.21.15.12:0/3691958022 conn(0x55812d304210 msgr2=0x55812d3a05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.749+0000 7f374afe5b80 1 -- 172.21.15.12:0/3691958022 shutdown_connections 2024-05-08T23:47:39.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.749+0000 7f374afe5b80 1 -- 172.21.15.12:0/3691958022 wait complete. 2024-05-08T23:47:39.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.749+0000 7f374afe5b80 20 client.25616 trim_cache size 0 max 16384 2024-05-08T23:47:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.784+0000 7f374afe5b80 1 Processor -- start 2024-05-08T23:47:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.784+0000 7f374afe5b80 1 -- start start 2024-05-08T23:47:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3abe00 0x55812d3ab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3ac720 0x55812d3a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f374afe5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d375810 con 0x55812d3ac720 2024-05-08T23:47:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f374afe5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d24f270 con 0x55812d3abe00 2024-05-08T23:47:39.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f374afe5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d3b0ae0 con 0x55812d3a6890 2024-05-08T23:47:39.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f373d3c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3ac720 0x55812d3a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f373d3c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3ac720 0x55812d3a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48432/0 (socket says 172.21.15.12:48432) 2024-05-08T23:47:39.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f373d3c8700 1 -- 172.21.15.12:0/976268222 learned_addr learned my addr 172.21.15.12:0/976268222 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:39.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f373cbc7700 1 --2- 172.21.15.12:0/976268222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.785+0000 7f3737fff700 1 --2- 172.21.15.12:0/976268222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3abe00 0x55812d3ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.786+0000 7f373cbc7700 1 -- 172.21.15.12:0/976268222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3abe00 msgr2=0x55812d3ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.786+0000 7f373cbc7700 1 --2- 172.21.15.12:0/976268222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3abe00 0x55812d3ab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.786+0000 7f373cbc7700 1 -- 172.21.15.12:0/976268222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3ac720 msgr2=0x55812d3a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.786+0000 7f373cbc7700 1 --2- 172.21.15.12:0/976268222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3ac720 0x55812d3a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.786+0000 7f373cbc7700 1 -- 172.21.15.12:0/976268222 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55812d3a6f40 con 0x55812d3a6890 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.786+0000 7f373cbc7700 1 --2- 172.21.15.12:0/976268222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55812d1797b0 tx=0x7f372c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b35eb7b7517a85b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.786+0000 7f37237fe700 1 -- 172.21.15.12:0/976268222 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c011de0 con 0x55812d3a6890 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.786+0000 7f37237fe700 1 -- 172.21.15.12:0/976268222 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f372c028040 con 0x55812d3a6890 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.786+0000 7f37237fe700 1 -- 172.21.15.12:0/976268222 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0296e0 con 0x55812d3a6890 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.787+0000 7f374afe5b80 1 -- 172.21.15.12:0/976268222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 msgr2=0x55812d3a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.787+0000 7f374afe5b80 1 --2- 172.21.15.12:0/976268222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55812d1797b0 tx=0x7f372c009260 comp rx=0 tx=0).stop 2024-05-08T23:47:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.787+0000 7f374afe5b80 1 -- 172.21.15.12:0/976268222 shutdown_connections 2024-05-08T23:47:39.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.787+0000 7f374afe5b80 1 --2- 172.21.15.12:0/976268222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3ac720 0x55812d3a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.787+0000 7f374afe5b80 1 --2- 172.21.15.12:0/976268222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3a6350 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.787+0000 7f374afe5b80 1 --2- 172.21.15.12:0/976268222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3abe00 0x55812d3ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.787+0000 7f374afe5b80 1 -- 172.21.15.12:0/976268222 >> 172.21.15.12:0/976268222 conn(0x55812d304210 msgr2=0x55812d3aac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.788+0000 7f374afe5b80 1 -- 172.21.15.12:0/976268222 shutdown_connections 2024-05-08T23:47:39.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.788+0000 7f374afe5b80 1 -- 172.21.15.12:0/976268222 wait complete. 2024-05-08T23:47:39.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.788+0000 7f374afe5b80 1 -- start start 2024-05-08T23:47:39.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.788+0000 7f374afe5b80 1 Processor -- start 2024-05-08T23:47:39.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3abe00 0x55812d3c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f374afe5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f374afe5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d3c7860 con 0x55812d3abe00 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f374afe5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d4271e0 con 0x55812d3ac720 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f374afe5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812d438330 con 0x55812d3a6890 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f374afe5b80 10 client.0 _ll_register_callbacks cb 0x55812d29bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f373cbc7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f373cbc7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38766/0 (socket says 172.21.15.12:38766) 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f373cbc7700 1 -- 172.21.15.12:0/1165430472 learned_addr learned my addr 172.21.15.12:0/1165430472 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.789+0000 7f373d3c8700 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f373cbc7700 1 -- 172.21.15.12:0/1165430472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 msgr2=0x55812d3c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f373cbc7700 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f373cbc7700 1 -- 172.21.15.12:0/1165430472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3abe00 msgr2=0x55812d3c6a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:47:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f373cbc7700 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3abe00 0x55812d3c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f373cbc7700 1 -- 172.21.15.12:0/1165430472 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372c01b040 con 0x55812d3a6890 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f373cbc7700 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3c8620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f372c016040 tx=0x7f372c0133a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2f5ffcc9d0561f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f3717fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f3717fff700 1 -- 172.21.15.12:0/1165430472 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c011ee0 con 0x55812d3a6890 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f3717fff700 1 -- 172.21.15.12:0/1165430472 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f372c005250 con 0x55812d3a6890 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f374afe5b80 1 -- 172.21.15.12:0/1165430472 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f373009a1a0 con 0x55812d3a6890 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.790+0000 7f3717fff700 1 -- 172.21.15.12:0/1165430472 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f372c005410 con 0x55812d3a6890 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f374afe5b80 10 client.25630 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f374afe5b80 1 -- 172.21.15.12:0/1165430472 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f37380674e0 con 0x55812d3a6890 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f3717fff700 1 -- 172.21.15.12:0/1165430472 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f373009a1a0 con 0x55812d3a6890 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f3717fff700 1 -- 172.21.15.12:0/1165430472 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f372c0444f0 con 0x55812d3a6890 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f374afe5b80 10 client.25630 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f374afe5b80 10 client.25630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f374afe5b80 20 client.25630 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f374afe5b80 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55812d4399f0 0x55812d43be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f374afe5b80 4 client.25630 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f374afe5b80 1 -- 172.21.15.12:0/1165430472 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55812d2eeec0 con 0x55812d4399f0 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.791+0000 7f3737fff700 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55812d4399f0 0x55812d43be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f3717fff700 10 client.25630 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f3737fff700 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55812d4399f0 0x55812d43be10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e3ebd3718f7e249 server_cookie=9fc00a84f2e788da in_seq=0 out_seq=0 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f3717fff700 1 -- 172.21.15.12:0/1165430472 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55812d2eeec0 con 0x55812d4399f0 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f3717fff700 10 client.25630 handle_command_reply: tid=0 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f374afe5b80 1 client.25630 shutdown 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f374afe5b80 1 -- 172.21.15.12:0/1165430472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 msgr2=0x55812d3c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f374afe5b80 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3c8620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f372c016040 tx=0x7f372c0133a0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f374afe5b80 1 -- 172.21.15.12:0/1165430472 shutdown_connections 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f374afe5b80 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55812d4399f0 0x55812d43be10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f374afe5b80 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55812d3ac720 0x55812d3c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f374afe5b80 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55812d3a6890 0x55812d3c8620 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f3717fff700 0 client.25630 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f374afe5b80 1 --2- 172.21.15.12:0/1165430472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55812d3abe00 0x55812d3c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.792+0000 7f374afe5b80 1 -- 172.21.15.12:0/1165430472 >> 172.21.15.12:0/1165430472 conn(0x55812d304210 msgr2=0x55812d3b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.793+0000 7f374afe5b80 1 -- 172.21.15.12:0/1165430472 shutdown_connections 2024-05-08T23:47:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.793+0000 7f374afe5b80 1 -- 172.21.15.12:0/1165430472 wait complete. 2024-05-08T23:47:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.793+0000 7f374afe5b80 20 client.25630 trim_cache size 0 max 16384 2024-05-08T23:47:39.792 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:47:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:47:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:47:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "1a632a93-be7f-42c9-9171-fbbf64559c57": { 2024-05-08T23:47:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:47:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "1a632a93-be7f-42c9-9171-fbbf64559c57", 2024-05-08T23:47:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:47:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:47:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:47:39.793 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 -- 172.21.15.12:0/2596110397 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37280a1430 msgr2=0x7f37280a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37280a1430 0x7f37280a38f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f372c0097c0 tx=0x7f372c009750 comp rx=0 tx=0).stop 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 -- 172.21.15.12:0/2596110397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37381062d0 msgr2=0x7f373819d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37381062d0 0x7f373819d750 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f373000ef30 tx=0x7f373000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 -- 172.21.15.12:0/2596110397 shutdown_connections 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37280a1430 0x7f37280a38f0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f372c0097c0 tx=0x7f372c009750 comp rx=0 tx=0).stop 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37381062d0 0x7f373819d750 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3738103960 0x7f37381a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 --2- 172.21.15.12:0/2596110397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3738105e60 0x7f37381a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 -- 172.21.15.12:0/2596110397 >> 172.21.15.12:0/2596110397 conn(0x7f3738076ba0 msgr2=0x7f37380ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 -- 172.21.15.12:0/2596110397 shutdown_connections 2024-05-08T23:47:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:47:39.794+0000 7f373ee2b700 1 -- 172.21.15.12:0/2596110397 wait complete. 2024-05-08T23:47:39.809 INFO:tasks.cephfs.filesystem:scrub status for tag:1a632a93-be7f-42c9-9171-fbbf64559c57 - {'path': '/', 'tag': '1a632a93-be7f-42c9-9171-fbbf64559c57', 'options': 'recursive,force'} 2024-05-08T23:47:40.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:40 smithi012 ceph-mon[42503]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:47:40.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:40 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:47:40.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:40 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:40.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:40 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:47:40.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:40 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:40.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:40 smithi204 ceph-mon[47373]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:47:40.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:40 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:47:40.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:40 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:40.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:40 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:47:40.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:40 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:40.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:40 smithi160 ceph-mon[46152]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:47:40.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:40 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:47:40.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:40 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:40.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:40 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:47:40.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:40 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:47:42.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:42 smithi012 ceph-mon[42503]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:47:42.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:42 smithi204 ceph-mon[47373]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:47:42.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:42 smithi160 ceph-mon[46152]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:47:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.870+0000 7ff18b595700 1 -- 172.21.15.12:0/1804309588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff184101d90 msgr2=0x7ff184102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.870+0000 7ff18b595700 1 --2- 172.21.15.12:0/1804309588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff184101d90 0x7ff184102170 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff180009990 tx=0x7ff180009ca0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.870+0000 7ff18b595700 1 -- 172.21.15.12:0/1804309588 shutdown_connections 2024-05-08T23:47:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.870+0000 7ff18b595700 1 --2- 172.21.15.12:0/1804309588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff18410db90 0x7ff18410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.871+0000 7ff18b595700 1 --2- 172.21.15.12:0/1804309588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff184101d90 0x7ff184102170 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.871+0000 7ff18b595700 1 --2- 172.21.15.12:0/1804309588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1840ff4a0 0x7ff1840ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.871+0000 7ff18b595700 1 -- 172.21.15.12:0/1804309588 >> 172.21.15.12:0/1804309588 conn(0x7ff1840763a0 msgr2=0x7ff1840fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.871+0000 7ff18b595700 1 -- 172.21.15.12:0/1804309588 shutdown_connections 2024-05-08T23:47:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.871+0000 7ff18b595700 1 -- 172.21.15.12:0/1804309588 wait complete. 2024-05-08T23:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.871+0000 7ff18b595700 1 Processor -- start 2024-05-08T23:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff18b595700 1 -- start start 2024-05-08T23:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff18b595700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1840ff4a0 0x7ff184199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff18b595700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184101d90 0x7ff184199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff18b595700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff18410db90 0x7ff18419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff18b595700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18419e440 con 0x7ff18410db90 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff18b595700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18419e5b0 con 0x7ff1840ff4a0 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff18b595700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18419e770 con 0x7ff184101d90 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff189b32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff18410db90 0x7ff18419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff189b32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff18410db90 0x7ff18419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48456/0 (socket says 172.21.15.12:48456) 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff189b32700 1 -- 172.21.15.12:0/100278319 learned_addr learned my addr 172.21.15.12:0/100278319 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff188b30700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184101d90 0x7ff184199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff189331700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1840ff4a0 0x7ff184199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff189b32700 1 -- 172.21.15.12:0/100278319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184101d90 msgr2=0x7ff184199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff189b32700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184101d90 0x7ff184199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff189b32700 1 -- 172.21.15.12:0/100278319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1840ff4a0 msgr2=0x7ff184199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff189b32700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1840ff4a0 0x7ff184199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff189b32700 1 -- 172.21.15.12:0/100278319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1800095f0 con 0x7ff18410db90 2024-05-08T23:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.872+0000 7ff189331700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1840ff4a0 0x7ff184199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:47:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.873+0000 7ff189b32700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff18410db90 0x7ff18419de00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00ada0 tx=0x7ff17c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.873+0000 7ff17a7fc700 1 -- 172.21.15.12:0/100278319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00f070 con 0x7ff18410db90 2024-05-08T23:47:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.873+0000 7ff18b595700 1 -- 172.21.15.12:0/100278319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18419ea50 con 0x7ff18410db90 2024-05-08T23:47:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.873+0000 7ff17a7fc700 1 -- 172.21.15.12:0/100278319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff17c007b00 con 0x7ff18410db90 2024-05-08T23:47:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.873+0000 7ff17a7fc700 1 -- 172.21.15.12:0/100278319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0053a0 con 0x7ff18410db90 2024-05-08T23:47:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.873+0000 7ff18b595700 1 -- 172.21.15.12:0/100278319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18406aba0 con 0x7ff18410db90 2024-05-08T23:47:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.877+0000 7ff17a7fc700 1 -- 172.21.15.12:0/100278319 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7ff17c00c980 con 0x7ff18410db90 2024-05-08T23:47:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.878+0000 7ff17a7fc700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1700a13b0 0x7ff1700a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.878+0000 7ff19774fb80 1 Processor -- start 2024-05-08T23:47:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.878+0000 7ff17a7fc700 1 -- 172.21.15.12:0/100278319 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff17c0cc4b0 con 0x7ff18410db90 2024-05-08T23:47:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.878+0000 7ff189331700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1700a13b0 0x7ff1700a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff19774fb80 1 -- start start 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed398940 0x555eed398d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed399260 0x555eed39cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff19774fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed2e70f0 con 0x555eed39d240 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff19774fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed2e4ab0 con 0x555eed399260 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff189331700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1700a13b0 0x7ff1700a3870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff180006010 tx=0x7ff18001a040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff189331700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed398940 0x555eed398d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff189b32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff189331700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed398940 0x555eed398d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38800/0 (socket says 172.21.15.12:38800) 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff189331700 1 -- 172.21.15.12:0/102985628 learned_addr learned my addr 172.21.15.12:0/102985628 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff19774fb80 1 -- 172.21.15.12:0/102985628 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed2e03a0 con 0x555eed398940 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.879+0000 7ff188b30700 1 --2- 172.21.15.12:0/102985628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed399260 0x555eed39cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff189b32700 1 -- 172.21.15.12:0/102985628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed398940 msgr2=0x555eed398d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff189b32700 1 --2- 172.21.15.12:0/102985628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed398940 0x555eed398d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff189b32700 1 -- 172.21.15.12:0/102985628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed399260 msgr2=0x555eed39cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff189b32700 1 --2- 172.21.15.12:0/102985628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed399260 0x555eed39cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff189b32700 1 -- 172.21.15.12:0/102985628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eed161350 con 0x555eed39d240 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff188b30700 1 --2- 172.21.15.12:0/102985628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed399260 0x555eed39cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff189331700 1 --2- 172.21.15.12:0/102985628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed398940 0x555eed398d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff189b32700 1 --2- 172.21.15.12:0/102985628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39d6a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0c9ac0 tx=0x7ff17c099000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a85fda35b87b146 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff16f7fe700 1 -- 172.21.15.12:0/102985628 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0840e0 con 0x555eed39d240 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff16f7fe700 1 -- 172.21.15.12:0/102985628 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff17c099b40 con 0x555eed39d240 2024-05-08T23:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.880+0000 7ff16f7fe700 1 -- 172.21.15.12:0/102985628 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00a260 con 0x555eed39d240 2024-05-08T23:47:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.881+0000 7ff19774fb80 1 -- 172.21.15.12:0/102985628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 msgr2=0x555eed39d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.881+0000 7ff19774fb80 1 --2- 172.21.15.12:0/102985628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39d6a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0c9ac0 tx=0x7ff17c099000 comp rx=0 tx=0).stop 2024-05-08T23:47:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.881+0000 7ff19774fb80 1 -- 172.21.15.12:0/102985628 shutdown_connections 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.881+0000 7ff19774fb80 1 --2- 172.21.15.12:0/102985628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39d6a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.881+0000 7ff19774fb80 1 --2- 172.21.15.12:0/102985628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed398940 0x555eed398d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.881+0000 7ff19774fb80 1 --2- 172.21.15.12:0/102985628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed399260 0x555eed39cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.881+0000 7ff19774fb80 1 -- 172.21.15.12:0/102985628 >> 172.21.15.12:0/102985628 conn(0x555eed2ee1b0 msgr2=0x555eed38a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.882+0000 7ff19774fb80 1 -- 172.21.15.12:0/102985628 shutdown_connections 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.882+0000 7ff19774fb80 1 -- 172.21.15.12:0/102985628 wait complete. 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.882+0000 7ff19774fb80 1 -- start start 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.882+0000 7ff19774fb80 1 Processor -- start 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.883+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed3a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.883+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed399260 0x555eed3a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.883+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed39d240 0x555eed3a2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.883+0000 7ff19774fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed27c500 con 0x555eed399260 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.883+0000 7ff19774fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed27c1a0 con 0x555eed398940 2024-05-08T23:47:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.883+0000 7ff19774fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed163640 con 0x555eed39d240 2024-05-08T23:47:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.883+0000 7ff19774fb80 10 client.0 _ll_register_callbacks cb 0x555eed285ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.883+0000 7ff189b32700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed39d240 0x555eed3a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.883+0000 7ff189b32700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed39d240 0x555eed3a2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38808/0 (socket says 172.21.15.12:38808) 2024-05-08T23:47:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.883+0000 7ff189b32700 1 -- 172.21.15.12:0/1870012697 learned_addr learned my addr 172.21.15.12:0/1870012697 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.884+0000 7ff189b32700 1 -- 172.21.15.12:0/1870012697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 msgr2=0x555eed3a1e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:47:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.884+0000 7ff189b32700 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed3a1e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.884+0000 7ff189b32700 1 -- 172.21.15.12:0/1870012697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed399260 msgr2=0x555eed3a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.884+0000 7ff189b32700 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed399260 0x555eed3a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.884+0000 7ff189b32700 1 -- 172.21.15.12:0/1870012697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1740078a0 con 0x555eed39d240 2024-05-08T23:47:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.884+0000 7ff189b32700 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed39d240 0x555eed3a2890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff17c099140 tx=0x7ff17c006060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ab2325b475fc89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.886+0000 7ff16cff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.886+0000 7ff16cff9700 1 -- 172.21.15.12:0/1870012697 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0a1830 con 0x555eed39d240 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.886+0000 7ff19774fb80 1 -- 172.21.15.12:0/1870012697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eed239210 con 0x555eed39d240 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.886+0000 7ff16cff9700 1 -- 172.21.15.12:0/1870012697 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff17c09f050 con 0x555eed39d240 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.886+0000 7ff16cff9700 1 -- 172.21.15.12:0/1870012697 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff17c00a1d0 con 0x555eed39d240 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.886+0000 7ff19774fb80 10 client.25639 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.886+0000 7ff19774fb80 1 -- 172.21.15.12:0/1870012697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7ff18406aba0 con 0x555eed39d240 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.886+0000 7ff16cff9700 1 -- 172.21.15.12:0/1870012697 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555eed239210 con 0x555eed39d240 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.887+0000 7ff16cff9700 1 -- 172.21.15.12:0/1870012697 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff17c0996a0 con 0x555eed39d240 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.887+0000 7ff19774fb80 10 client.25639 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.887+0000 7ff19774fb80 10 client.25639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.887+0000 7ff19774fb80 20 client.25639 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.887+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555eed3ba860 0x555eed3bcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.887+0000 7ff19774fb80 4 client.25639 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.887+0000 7ff19774fb80 1 -- 172.21.15.12:0/1870012697 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eed35f7b0 con 0x555eed3ba860 2024-05-08T23:47:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.888+0000 7ff189331700 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555eed3ba860 0x555eed3bcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.888+0000 7ff16cff9700 10 client.25639 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.888+0000 7ff189331700 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555eed3ba860 0x555eed3bcc80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a264d7a80b840254 server_cookie=612567e0b8f99449 in_seq=0 out_seq=0 2024-05-08T23:47:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff16cff9700 1 -- 172.21.15.12:0/1870012697 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555eed35f7b0 con 0x555eed3ba860 2024-05-08T23:47:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff16cff9700 10 client.25639 handle_command_reply: tid=0 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 client.25639 shutdown 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 -- 172.21.15.12:0/1870012697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed39d240 msgr2=0x555eed3a2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed39d240 0x555eed3a2890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff17c099140 tx=0x7ff17c006060 comp rx=0 tx=0).stop 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 -- 172.21.15.12:0/1870012697 shutdown_connections 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed39d240 0x555eed3a2890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555eed3ba860 0x555eed3bcc80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed3a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1870012697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed399260 0x555eed3a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 -- 172.21.15.12:0/1870012697 >> 172.21.15.12:0/1870012697 conn(0x555eed2ee1b0 msgr2=0x555eed39de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff16cff9700 0 client.25639 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 -- 172.21.15.12:0/1870012697 shutdown_connections 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.889+0000 7ff19774fb80 1 -- 172.21.15.12:0/1870012697 wait complete. 2024-05-08T23:47:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.890+0000 7ff19774fb80 20 client.25639 trim_cache size 0 max 16384 2024-05-08T23:47:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.920+0000 7ff19774fb80 1 Processor -- start 2024-05-08T23:47:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.920+0000 7ff19774fb80 1 -- start start 2024-05-08T23:47:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.920+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed38a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.920+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 0x555eed3974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.920+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.920+0000 7ff19774fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed39db80 con 0x555eed39d240 2024-05-08T23:47:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.920+0000 7ff19774fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed38b330 con 0x555eed398940 2024-05-08T23:47:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.920+0000 7ff19774fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed382b90 con 0x555eed399260 2024-05-08T23:47:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff188b30700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 0x555eed3974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff189331700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed38a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff189b32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff188b30700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 0x555eed3974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38814/0 (socket says 172.21.15.12:38814) 2024-05-08T23:47:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff188b30700 1 -- 172.21.15.12:0/3253139883 learned_addr learned my addr 172.21.15.12:0/3253139883 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff188b30700 1 -- 172.21.15.12:0/3253139883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 msgr2=0x555eed38a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff188b30700 1 --2- 172.21.15.12:0/3253139883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed38a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff188b30700 1 -- 172.21.15.12:0/3253139883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 msgr2=0x555eed39d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff188b30700 1 --2- 172.21.15.12:0/3253139883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff188b30700 1 -- 172.21.15.12:0/3253139883 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eed10db00 con 0x555eed399260 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff189331700 1 --2- 172.21.15.12:0/3253139883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed38a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff188b30700 1 --2- 172.21.15.12:0/3253139883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 0x555eed3974b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff174007e30 tx=0x7ff174011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1abe5ae9238c2115 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.921+0000 7ff16ffff700 1 -- 172.21.15.12:0/3253139883 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174016070 con 0x555eed399260 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff16ffff700 1 -- 172.21.15.12:0/3253139883 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1740044e0 con 0x555eed399260 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff16ffff700 1 -- 172.21.15.12:0/3253139883 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174003d40 con 0x555eed399260 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff19774fb80 1 -- 172.21.15.12:0/3253139883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 msgr2=0x555eed3974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff19774fb80 1 --2- 172.21.15.12:0/3253139883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 0x555eed3974b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff174007e30 tx=0x7ff174011720 comp rx=0 tx=0).stop 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff19774fb80 1 -- 172.21.15.12:0/3253139883 shutdown_connections 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff19774fb80 1 --2- 172.21.15.12:0/3253139883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff19774fb80 1 --2- 172.21.15.12:0/3253139883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed38a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff19774fb80 1 --2- 172.21.15.12:0/3253139883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 0x555eed3974b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff19774fb80 1 -- 172.21.15.12:0/3253139883 >> 172.21.15.12:0/3253139883 conn(0x555eed2ee1b0 msgr2=0x555eed3c15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff19774fb80 1 -- 172.21.15.12:0/3253139883 shutdown_connections 2024-05-08T23:47:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.922+0000 7ff19774fb80 1 -- 172.21.15.12:0/3253139883 wait complete. 2024-05-08T23:47:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.923+0000 7ff19774fb80 1 -- start start 2024-05-08T23:47:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.923+0000 7ff19774fb80 1 Processor -- start 2024-05-08T23:47:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.923+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed394370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.923+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 0x555eed3948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.923+0000 7ff19774fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.923+0000 7ff19774fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed39fa90 con 0x555eed39d240 2024-05-08T23:47:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.923+0000 7ff19774fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed39fc00 con 0x555eed398940 2024-05-08T23:47:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.923+0000 7ff19774fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eed39fdc0 con 0x555eed399260 2024-05-08T23:47:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff19774fb80 10 client.0 _ll_register_callbacks cb 0x555eed285ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff189b32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff189331700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed394370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff189331700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed394370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:53018/0 (socket says 172.21.15.12:53018) 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff189b32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48488/0 (socket says 172.21.15.12:48488) 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff189331700 1 -- 172.21.15.12:0/1945825847 learned_addr learned my addr 172.21.15.12:0/1945825847 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff189331700 1 -- 172.21.15.12:0/1945825847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 msgr2=0x555eed3948b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff189331700 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 0x555eed3948b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff189331700 1 -- 172.21.15.12:0/1945825847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 msgr2=0x555eed39f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff189331700 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.924+0000 7ff189331700 1 -- 172.21.15.12:0/1945825847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18000ba20 con 0x555eed398940 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.925+0000 7ff189331700 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed394370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff180000c00 tx=0x7ff180009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72914e5079159288 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.925+0000 7ff16cff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.925+0000 7ff16cff9700 1 -- 172.21.15.12:0/1945825847 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18001d040 con 0x555eed398940 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.925+0000 7ff16cff9700 1 -- 172.21.15.12:0/1945825847 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff1800052d0 con 0x555eed398940 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.925+0000 7ff19774fb80 1 -- 172.21.15.12:0/1945825847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff17c099b40 con 0x555eed398940 2024-05-08T23:47:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.925+0000 7ff16cff9700 1 -- 172.21.15.12:0/1945825847 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff180005490 con 0x555eed398940 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.925+0000 7ff19774fb80 10 client.25637 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.925+0000 7ff19774fb80 1 -- 172.21.15.12:0/1945825847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7ff174007ae0 con 0x555eed398940 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.925+0000 7ff16cff9700 1 -- 172.21.15.12:0/1945825847 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff17c099b40 con 0x555eed398940 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.925+0000 7ff16cff9700 1 -- 172.21.15.12:0/1945825847 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff180036020 con 0x555eed398940 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.926+0000 7ff19774fb80 10 client.25637 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.926+0000 7ff19774fb80 10 client.25637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.926+0000 7ff19774fb80 20 client.25637 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.926+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555eed411180 0x555eed4135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.926+0000 7ff19774fb80 4 client.25637 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.926+0000 7ff19774fb80 1 -- 172.21.15.12:0/1945825847 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555eed2d8e60 con 0x555eed411180 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.926+0000 7ff188b30700 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555eed411180 0x555eed4135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.926+0000 7ff16cff9700 10 client.25637 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.926+0000 7ff188b30700 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555eed411180 0x555eed4135a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1eb6cc5c6ba3b76 server_cookie=272e07fe6165f4e8 in_seq=0 out_seq=0 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.930+0000 7ff16cff9700 1 -- 172.21.15.12:0/1945825847 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555eed2d8e60 con 0x555eed411180 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.930+0000 7ff16cff9700 10 client.25637 handle_command_reply: tid=0 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.930+0000 7ff19774fb80 1 client.25637 shutdown 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.930+0000 7ff19774fb80 1 -- 172.21.15.12:0/1945825847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 msgr2=0x555eed394370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.930+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed394370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff180000c00 tx=0x7ff180009310 comp rx=0 tx=0).stop 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.930+0000 7ff19774fb80 1 -- 172.21.15.12:0/1945825847 shutdown_connections 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.930+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eed39d240 0x555eed39f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.930+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555eed411180 0x555eed4135a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.930+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555eed398940 0x555eed394370 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.931+0000 7ff19774fb80 1 --2- 172.21.15.12:0/1945825847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555eed399260 0x555eed3948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.931+0000 7ff19774fb80 1 -- 172.21.15.12:0/1945825847 >> 172.21.15.12:0/1945825847 conn(0x555eed2ee1b0 msgr2=0x555eed3c15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.931+0000 7ff16cff9700 0 client.25637 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.931+0000 7ff19774fb80 1 -- 172.21.15.12:0/1945825847 shutdown_connections 2024-05-08T23:47:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.931+0000 7ff19774fb80 1 -- 172.21.15.12:0/1945825847 wait complete. 2024-05-08T23:47:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.931+0000 7ff19774fb80 20 client.25637 trim_cache size 0 max 16384 2024-05-08T23:47:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:47:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:47:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:47:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:47:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 -- 172.21.15.12:0/100278319 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1700a13b0 msgr2=0x7ff1700a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1700a13b0 0x7ff1700a3870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff180006010 tx=0x7ff18001a040 comp rx=0 tx=0).stop 2024-05-08T23:47:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 -- 172.21.15.12:0/100278319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff18410db90 msgr2=0x7ff18419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff18410db90 0x7ff18419de00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00ada0 tx=0x7ff17c00c330 comp rx=0 tx=0).stop 2024-05-08T23:47:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 -- 172.21.15.12:0/100278319 shutdown_connections 2024-05-08T23:47:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1700a13b0 0x7ff1700a3870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff18410db90 0x7ff18419de00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1840ff4a0 0x7ff184199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 --2- 172.21.15.12:0/100278319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff184101d90 0x7ff184199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 -- 172.21.15.12:0/100278319 >> 172.21.15.12:0/100278319 conn(0x7ff1840763a0 msgr2=0x7ff18410f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 -- 172.21.15.12:0/100278319 shutdown_connections 2024-05-08T23:47:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:42.932+0000 7ff18b595700 1 -- 172.21.15.12:0/100278319 wait complete. 2024-05-08T23:47:43.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:43 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:47:43.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:43 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:47:43.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:43 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:47:44.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:44 smithi012 ceph-mon[42503]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T23:47:44.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:44 smithi204 ceph-mon[47373]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T23:47:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:44 smithi160 ceph-mon[46152]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T23:47:46.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:46 smithi012 ceph-mon[42503]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-08T23:47:46.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:46 smithi204 ceph-mon[47373]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-08T23:47:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:46 smithi160 ceph-mon[46152]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-08T23:47:48.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:48 smithi012 ceph-mon[42503]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T23:47:48.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:48 smithi204 ceph-mon[47373]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T23:47:48.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:48 smithi160 ceph-mon[46152]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T23:47:50.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:50 smithi204 ceph-mon[47373]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:47:50.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:50 smithi160 ceph-mon[46152]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:47:50.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:50 smithi012 ceph-mon[42503]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:47:52.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:52 smithi204 ceph-mon[47373]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:47:52.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:52 smithi160 ceph-mon[46152]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:47:52.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:52 smithi012 ceph-mon[42503]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T23:47:54.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:54 smithi204 ceph-mon[47373]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T23:47:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:54 smithi160 ceph-mon[46152]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T23:47:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:54 smithi012 ceph-mon[42503]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T23:47:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.117+0000 7f7dd6941700 1 -- 172.21.15.12:0/330735460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd0100760 msgr2=0x7f7dd010cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.117+0000 7f7dd6941700 1 --2- 172.21.15.12:0/330735460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd0100760 0x7f7dd010cc00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800b0a0 tx=0x7f7dc800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.117+0000 7f7dd6941700 1 -- 172.21.15.12:0/330735460 shutdown_connections 2024-05-08T23:47:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.117+0000 7f7dd6941700 1 --2- 172.21.15.12:0/330735460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd0100760 0x7f7dd010cc00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.117+0000 7f7dd6941700 1 --2- 172.21.15.12:0/330735460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd0105e60 0x7f7dd0106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.117+0000 7f7dd6941700 1 --2- 172.21.15.12:0/330735460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd00ffda0 0x7f7dd0100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.117+0000 7f7dd6941700 1 -- 172.21.15.12:0/330735460 >> 172.21.15.12:0/330735460 conn(0x7f7dd0076f10 msgr2=0x7f7dd0077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.117+0000 7f7dd6941700 1 -- 172.21.15.12:0/330735460 shutdown_connections 2024-05-08T23:47:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.117+0000 7f7dd6941700 1 -- 172.21.15.12:0/330735460 wait complete. 2024-05-08T23:47:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.118+0000 7f7dd6941700 1 Processor -- start 2024-05-08T23:47:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.118+0000 7f7dd6941700 1 -- start start 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd6941700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd00ffda0 0x7f7dd0195150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd6941700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd0100760 0x7f7dd0195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd6941700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd0105e60 0x7f7dd0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd6941700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd019a070 con 0x7f7dd0105e60 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd6941700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd019a1e0 con 0x7f7dd00ffda0 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd6941700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd019a3a0 con 0x7f7dd0100760 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd4ede700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd0105e60 0x7f7dd0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd4ede700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd0105e60 0x7f7dd0199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58410/0 (socket says 172.21.15.12:58410) 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd4ede700 1 -- 172.21.15.12:0/2750906348 learned_addr learned my addr 172.21.15.12:0/2750906348 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd0100760 0x7f7dd0195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dcffff700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd00ffda0 0x7f7dd0195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd4ede700 1 -- 172.21.15.12:0/2750906348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd0100760 msgr2=0x7f7dd0195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd4ede700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd0100760 0x7f7dd0195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd4ede700 1 -- 172.21.15.12:0/2750906348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd00ffda0 msgr2=0x7f7dd0195150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd4ede700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd00ffda0 0x7f7dd0195150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dd4ede700 1 -- 172.21.15.12:0/2750906348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8009d00 con 0x7f7dd0105e60 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.119+0000 7f7dcffff700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd00ffda0 0x7f7dd0195150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.120+0000 7f7dd4ede700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd0105e60 0x7f7dd0199a30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800b780 tx=0x7f7dc8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.120+0000 7f7dcd7fa700 1 -- 172.21.15.12:0/2750906348 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80079d0 con 0x7f7dd0105e60 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.120+0000 7f7dd6941700 1 -- 172.21.15.12:0/2750906348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd019a620 con 0x7f7dd0105e60 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.120+0000 7f7dcd7fa700 1 -- 172.21.15.12:0/2750906348 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc8008ef0 con 0x7f7dd0105e60 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.120+0000 7f7dcd7fa700 1 -- 172.21.15.12:0/2750906348 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80216a0 con 0x7f7dd0105e60 2024-05-08T23:47:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.120+0000 7f7dd6941700 1 -- 172.21.15.12:0/2750906348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd01a1ec0 con 0x7f7dd0105e60 2024-05-08T23:47:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.124+0000 7f7de2afbb80 1 Processor -- start 2024-05-08T23:47:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.125+0000 7f7dcd7fa700 1 -- 172.21.15.12:0/2750906348 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f7dc80104d0 con 0x7f7dd0105e60 2024-05-08T23:47:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.125+0000 7f7de2afbb80 1 -- start start 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.125+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.125+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c7270 0x55d7787c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.125+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.125+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7787180f0 con 0x55d7787c1480 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.125+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d778715ab0 con 0x55d7787c7270 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.125+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7787113a0 con 0x55d7787c6950 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787c18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58420/0 (socket says 172.21.15.12:58420) 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/3509412230 learned_addr learned my addr 172.21.15.12:0/3509412230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/3509412230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcffff700 1 --2- 172.21.15.12:0/3509412230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c7270 0x55d7787c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcd7fa700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7db80a1430 0x7f7db80a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/3509412230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 msgr2=0x55d7787c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/3509412230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/3509412230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c7270 msgr2=0x55d7787c0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/3509412230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c7270 0x55d7787c0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.126+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/3509412230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d778592350 con 0x55d7787c1480 2024-05-08T23:47:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.127+0000 7f7dcd7fa700 1 -- 172.21.15.12:0/2750906348 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7dc80d5170 con 0x7f7dd0105e60 2024-05-08T23:47:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.127+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/3509412230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787c18a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009720 tx=0x7f7dc40058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8364220c38604dc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.127+0000 7f7db67fc700 1 -- 172.21.15.12:0/3509412230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4009d40 con 0x55d7787c1480 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.127+0000 7f7dd4ede700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7db80a1430 0x7f7db80a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.127+0000 7f7db67fc700 1 -- 172.21.15.12:0/3509412230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc40044f0 con 0x55d7787c1480 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.127+0000 7f7db67fc700 1 -- 172.21.15.12:0/3509412230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4003e40 con 0x55d7787c1480 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3509412230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 msgr2=0x55d7787c18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7dd4ede700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7db80a1430 0x7f7db80a38f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80cf460 tx=0x7f7dc8010b00 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/3509412230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787c18a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009720 tx=0x7f7dc40058e0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3509412230 shutdown_connections 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/3509412230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787c18a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/3509412230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/3509412230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c7270 0x55d7787c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3509412230 >> 172.21.15.12:0/3509412230 conn(0x55d77871f1b0 msgr2=0x55d7787bb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3509412230 shutdown_connections 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3509412230 wait complete. 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.128+0000 7f7de2afbb80 1 -- start start 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.129+0000 7f7de2afbb80 1 Processor -- start 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.129+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c1480 0x55d7787cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.129+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c6950 0x55d7787ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.129+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 0x55d7787cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.129+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7786ad500 con 0x55d7787c7270 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.129+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7786ad1a0 con 0x55d7787c6950 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.129+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d778594640 con 0x55d7787c1480 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.129+0000 7f7de2afbb80 10 client.0 _ll_register_callbacks cb 0x55d7786b6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.129+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 0x55d7787cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 0x55d7787cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58434/0 (socket says 172.21.15.12:58434) 2024-05-08T23:47:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/2703335945 learned_addr learned my addr 172.21.15.12:0/2703335945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c1480 0x55d7787cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7dcffff700 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c6950 0x55d7787ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/2703335945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c1480 msgr2=0x55d7787cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c1480 0x55d7787cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/2703335945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c6950 msgr2=0x55d7787ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c6950 0x55d7787ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/2703335945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc4015040 con 0x55d7787c7270 2024-05-08T23:47:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 0x55d7787cacb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40059d0 tx=0x7f7dc4004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf7ad751d5db105 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7daf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7daf7fe700 1 -- 172.21.15.12:0/2703335945 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4022030 con 0x55d7787c7270 2024-05-08T23:47:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7daf7fe700 1 -- 172.21.15.12:0/2703335945 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7dc4009da0 con 0x55d7787c7270 2024-05-08T23:47:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.130+0000 7f7de2afbb80 1 -- 172.21.15.12:0/2703335945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d77866a210 con 0x55d7787c7270 2024-05-08T23:47:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7daf7fe700 1 -- 172.21.15.12:0/2703335945 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc4003e30 con 0x55d7787c7270 2024-05-08T23:47:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7de2afbb80 10 client.15801 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7de2afbb80 1 -- 172.21.15.12:0/2703335945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d77853eb00 con 0x55d7787c7270 2024-05-08T23:47:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7daf7fe700 1 -- 172.21.15.12:0/2703335945 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d77866a210 con 0x55d7787c7270 2024-05-08T23:47:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7daf7fe700 1 -- 172.21.15.12:0/2703335945 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7dc401eb00 con 0x55d7787c7270 2024-05-08T23:47:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7de2afbb80 10 client.15801 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7de2afbb80 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7de2afbb80 20 client.15801 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7787eb9f0 0x55d7787ede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7de2afbb80 4 client.15801 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:47:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7de2afbb80 1 -- 172.21.15.12:0/2703335945 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7787907b0 con 0x55d7787eb9f0 2024-05-08T23:47:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.131+0000 7f7dcffff700 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7787eb9f0 0x55d7787ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.132+0000 7f7daf7fe700 10 client.15801 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.132+0000 7f7dcffff700 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7787eb9f0 0x55d7787ede10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a9beae8ba48e85d server_cookie=65c604853f8f0e15 in_seq=0 out_seq=0 2024-05-08T23:47:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7daf7fe700 1 -- 172.21.15.12:0/2703335945 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7787907b0 con 0x55d7787eb9f0 2024-05-08T23:47:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7daf7fe700 10 client.15801 handle_command_reply: tid=0 2024-05-08T23:47:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 client.15801 shutdown 2024-05-08T23:47:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 -- 172.21.15.12:0/2703335945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 msgr2=0x55d7787cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 0x55d7787cacb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40059d0 tx=0x7f7dc4004680 comp rx=0 tx=0).stop 2024-05-08T23:47:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 -- 172.21.15.12:0/2703335945 shutdown_connections 2024-05-08T23:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 0x55d7787cacb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c1480 0x55d7787cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7787eb9f0 0x55d7787ede10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/2703335945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c6950 0x55d7787ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7daf7fe700 0 client.15801 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 -- 172.21.15.12:0/2703335945 >> 172.21.15.12:0/2703335945 conn(0x55d77871f1b0 msgr2=0x55d7787c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 -- 172.21.15.12:0/2703335945 shutdown_connections 2024-05-08T23:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 1 -- 172.21.15.12:0/2703335945 wait complete. 2024-05-08T23:47:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.133+0000 7f7de2afbb80 20 client.15801 trim_cache size 0 max 16384 2024-05-08T23:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.161+0000 7f7de2afbb80 1 Processor -- start 2024-05-08T23:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.162+0000 7f7de2afbb80 1 -- start start 2024-05-08T23:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.162+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c1480 0x55d7787c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.162+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.162+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 0x55d7787e7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.162+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7787cb4c0 con 0x55d7787c7270 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.162+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7787cb2b0 con 0x55d7787c1480 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.162+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7787f3020 con 0x55d7787c6950 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.162+0000 7f7dcffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.162+0000 7f7dcffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787bb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38624/0 (socket says 172.21.15.12:38624) 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.162+0000 7f7dcffff700 1 -- 172.21.15.12:0/835499549 learned_addr learned my addr 172.21.15.12:0/835499549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/835499549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 0x55d7787e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/835499549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c1480 0x55d7787c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7dcffff700 1 -- 172.21.15.12:0/835499549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c1480 msgr2=0x55d7787c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7dcffff700 1 --2- 172.21.15.12:0/835499549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c1480 0x55d7787c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7dcffff700 1 -- 172.21.15.12:0/835499549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 msgr2=0x55d7787e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7dcffff700 1 --2- 172.21.15.12:0/835499549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 0x55d7787e7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7dcffff700 1 -- 172.21.15.12:0/835499549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7786d7500 con 0x55d7787c6950 2024-05-08T23:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/835499549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c1480 0x55d7787c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7dcffff700 1 --2- 172.21.15.12:0/835499549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787bb910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55d7787ee720 tx=0x7f7dc0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76db0380ee1bc873 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7db6ffd700 1 -- 172.21.15.12:0/835499549 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0017070 con 0x55d7787c6950 2024-05-08T23:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7db6ffd700 1 -- 172.21.15.12:0/835499549 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc0018040 con 0x55d7787c6950 2024-05-08T23:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.163+0000 7f7db6ffd700 1 -- 172.21.15.12:0/835499549 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0005280 con 0x55d7787c6950 2024-05-08T23:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.164+0000 7f7de2afbb80 1 -- 172.21.15.12:0/835499549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 msgr2=0x55d7787bb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.164+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/835499549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787bb910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55d7787ee720 tx=0x7f7dc0009260 comp rx=0 tx=0).stop 2024-05-08T23:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.164+0000 7f7de2afbb80 1 -- 172.21.15.12:0/835499549 shutdown_connections 2024-05-08T23:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.164+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/835499549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c7270 0x55d7787e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.164+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/835499549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c1480 0x55d7787c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.164+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/835499549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787bb910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.164+0000 7f7de2afbb80 1 -- 172.21.15.12:0/835499549 >> 172.21.15.12:0/835499549 conn(0x55d77871f1b0 msgr2=0x55d7787c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.165+0000 7f7de2afbb80 1 -- 172.21.15.12:0/835499549 shutdown_connections 2024-05-08T23:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.165+0000 7f7de2afbb80 1 -- 172.21.15.12:0/835499549 wait complete. 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.165+0000 7f7de2afbb80 1 -- start start 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.165+0000 7f7de2afbb80 1 Processor -- start 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787e2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7de2afbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c7270 0x55d7787df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7787df6f0 con 0x55d7787c1480 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7787df860 con 0x55d7787c7270 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7de2afbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7787dfa20 con 0x55d7787c6950 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7de2afbb80 10 client.0 _ll_register_callbacks cb 0x55d7786b6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787e2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787e2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58440/0 (socket says 172.21.15.12:58440) 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/3445189565 learned_addr learned my addr 172.21.15.12:0/3445189565 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c7270 0x55d7787df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7dcffff700 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/3445189565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 msgr2=0x55d7787dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787dec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/3445189565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c7270 msgr2=0x55d7787df160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c7270 0x55d7787df160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.166+0000 7f7dcf7fe700 1 -- 172.21.15.12:0/3445189565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc0012170 con 0x55d7787c1480 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7dcf7fe700 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787e2e50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4005d60 tx=0x7f7dc402f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8273e5af043f51a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7daf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7daf7fe700 1 -- 172.21.15.12:0/3445189565 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4022040 con 0x55d7787c1480 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7daf7fe700 1 -- 172.21.15.12:0/3445189565 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7dc4003e30 con 0x55d7787c1480 2024-05-08T23:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3445189565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dc40044f0 con 0x55d7787c1480 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7daf7fe700 1 -- 172.21.15.12:0/3445189565 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc4003ff0 con 0x55d7787c1480 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7de2afbb80 10 client.15807 fetch_fsmap learned FSMap version 26 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3445189565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d778596c70 con 0x55d7787c1480 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7daf7fe700 1 -- 172.21.15.12:0/3445189565 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc4048080 con 0x55d7787c1480 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7daf7fe700 1 -- 172.21.15.12:0/3445189565 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7dc4005030 con 0x55d7787c1480 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7de2afbb80 10 client.15807 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.167+0000 7f7de2afbb80 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.168+0000 7f7de2afbb80 20 client.15807 populate_metadata read hostname 'smithi012' 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.168+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7788420e0 0x55d778844500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.168+0000 7f7de2afbb80 4 client.15807 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.168+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3445189565 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d778709e60 con 0x55d7788420e0 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.168+0000 7f7dcffff700 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7788420e0 0x55d778844500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.168+0000 7f7dcffff700 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7788420e0 0x55d778844500 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2be6f6b3e134e2d server_cookie=4df3636b143bd47c in_seq=0 out_seq=0 2024-05-08T23:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.168+0000 7f7daf7fe700 10 client.15807 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7daf7fe700 1 -- 172.21.15.12:0/3445189565 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d778709e60 con 0x55d7788420e0 2024-05-08T23:47:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7daf7fe700 10 client.15807 handle_command_reply: tid=0 2024-05-08T23:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 client.15807 shutdown 2024-05-08T23:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3445189565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 msgr2=0x55d7787e2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787e2e50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4005d60 tx=0x7f7dc402f040 comp rx=0 tx=0).stop 2024-05-08T23:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3445189565 shutdown_connections 2024-05-08T23:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7788420e0 0x55d778844500 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7787c7270 0x55d7787df160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7787c1480 0x55d7787e2e50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 --2- 172.21.15.12:0/3445189565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7787c6950 0x55d7787dec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3445189565 >> 172.21.15.12:0/3445189565 conn(0x55d77871f1b0 msgr2=0x55d7787bf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7daf7fe700 0 client.15807 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3445189565 shutdown_connections 2024-05-08T23:47:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 1 -- 172.21.15.12:0/3445189565 wait complete. 2024-05-08T23:47:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.174+0000 7f7de2afbb80 20 client.15807 trim_cache size 0 max 16384 2024-05-08T23:47:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:47:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:47:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:47:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 -- 172.21.15.12:0/2750906348 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7db80a1430 msgr2=0x7f7db80a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7db80a1430 0x7f7db80a38f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80cf460 tx=0x7f7dc8010b00 comp rx=0 tx=0).stop 2024-05-08T23:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 -- 172.21.15.12:0/2750906348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd0105e60 msgr2=0x7f7dd0199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd0105e60 0x7f7dd0199a30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800b780 tx=0x7f7dc8004d00 comp rx=0 tx=0).stop 2024-05-08T23:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 -- 172.21.15.12:0/2750906348 shutdown_connections 2024-05-08T23:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7db80a1430 0x7f7db80a38f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dd0105e60 0x7f7dd0199a30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7dd00ffda0 0x7f7dd0195150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 --2- 172.21.15.12:0/2750906348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dd0100760 0x7f7dd0195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 -- 172.21.15.12:0/2750906348 >> 172.21.15.12:0/2750906348 conn(0x7f7dd0076f10 msgr2=0x7f7dd00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 -- 172.21.15.12:0/2750906348 shutdown_connections 2024-05-08T23:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:47:56.175+0000 7f7dd6941700 1 -- 172.21.15.12:0/2750906348 wait complete. 2024-05-08T23:47:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:56 smithi204 ceph-mon[47373]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:47:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:56 smithi160 ceph-mon[46152]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:47:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:56 smithi012 ceph-mon[42503]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T23:47:57.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:57 smithi204 ceph-mon[47373]: mgrmap e25: x(active, since 91s), standbys: y, z 2024-05-08T23:47:57.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:57 smithi160 ceph-mon[46152]: mgrmap e25: x(active, since 91s), standbys: y, z 2024-05-08T23:47:57.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:57 smithi012 ceph-mon[42503]: mgrmap e25: x(active, since 91s), standbys: y, z 2024-05-08T23:47:58.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:47:58 smithi204 ceph-mon[47373]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T23:47:58.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:47:58 smithi160 ceph-mon[46152]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T23:47:58.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:47:58 smithi012 ceph-mon[42503]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T23:48:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:00 smithi204 ceph-mon[47373]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T23:48:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:00 smithi160 ceph-mon[46152]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T23:48:00.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:00 smithi012 ceph-mon[42503]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T23:48:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:02 smithi204 ceph-mon[47373]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:48:02.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:02 smithi160 ceph-mon[46152]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:48:02.884 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 83% (10275/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T23:48:02.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:02 smithi012 ceph-mon[42503]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:48:03.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:03 smithi204 ceph-mon[47373]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-08T23:48:03.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:03 smithi160 ceph-mon[46152]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-08T23:48:03.868 INFO:tasks.workunit.client.0.smithi012.stderr:Note: switching to 'd21e4f961318b56f0d915caa2490c3711bf13c34'. 2024-05-08T23:48:03.868 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-08T23:48:03.868 INFO:tasks.workunit.client.0.smithi012.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T23:48:03.868 INFO:tasks.workunit.client.0.smithi012.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T23:48:03.868 INFO:tasks.workunit.client.0.smithi012.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T23:48:03.868 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr: git switch -c 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr:Or undo this operation with: 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr: git switch - 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-05-08T23:48:03.869 INFO:tasks.workunit.client.0.smithi012.stderr:HEAD is now at d21e4f96131 Merge PR #57343 into wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:48:03.875 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T23:48:03.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:03 smithi012 ceph-mon[42503]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-08T23:48:03.944 INFO:tasks.workunit.client.0.smithi012.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T23:48:03.946 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T23:48:03.946 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T23:48:04.418 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T23:48:04.508 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T23:48:04.552 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T23:48:04.554 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T23:48:04.554 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T23:48:04.596 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T23:48:04.599 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-08T23:48:04.599 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T23:48:04.659 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-08T23:48:04.661 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-08T23:48:04.661 DEBUG:teuthology.orchestra.run.smithi012:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-08T23:48:04.744 INFO:tasks.workunit.client.0.smithi012.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-08T23:48:04.744 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-05-08T23:48:04.744 INFO:tasks.workunit.client.0.smithi012.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-08T23:48:04.745 INFO:tasks.workunit.client.0.smithi012.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-08T23:48:04.845 INFO:tasks.workunit.client.0.smithi012.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-08T23:48:04.845 INFO:tasks.workunit.client.0.smithi012.stdout:releasing clients 2024-05-08T23:48:05.845 INFO:tasks.workunit.client.0.smithi012.stdout: 1 3760 39.76 MB/sec warmup 1 sec latency 5.707 ms 2024-05-08T23:48:06.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:06 smithi012 ceph-mon[42503]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:48:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:06 smithi204 ceph-mon[47373]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:48:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:06 smithi160 ceph-mon[46152]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:48:06.845 INFO:tasks.workunit.client.0.smithi012.stdout: 1 6898 27.25 MB/sec warmup 2 sec latency 28.158 ms 2024-05-08T23:48:07.847 INFO:tasks.workunit.client.0.smithi012.stdout: 1 10030 23.91 MB/sec warmup 3 sec latency 11.969 ms 2024-05-08T23:48:08.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:08 smithi012 ceph-mon[42503]: pgmap v55: 97 pgs: 97 active+clean; 4.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 280 KiB/s wr, 53 op/s 2024-05-08T23:48:08.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:08 smithi204 ceph-mon[47373]: pgmap v55: 97 pgs: 97 active+clean; 4.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 280 KiB/s wr, 53 op/s 2024-05-08T23:48:08.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:08 smithi160 ceph-mon[46152]: pgmap v55: 97 pgs: 97 active+clean; 4.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 280 KiB/s wr, 53 op/s 2024-05-08T23:48:08.845 INFO:tasks.workunit.client.0.smithi012.stdout: 1 13234 21.58 MB/sec warmup 4 sec latency 5.791 ms 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 -- 172.21.15.12:0/1201757321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9320105f20 msgr2=0x7f9320106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 --2- 172.21.15.12:0/1201757321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9320105f20 0x7f9320106300 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9310009990 tx=0x7f9310009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 -- 172.21.15.12:0/1201757321 shutdown_connections 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 --2- 172.21.15.12:0/1201757321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9320100c40 0x7f932010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 --2- 172.21.15.12:0/1201757321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9320105f20 0x7f9320106300 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 --2- 172.21.15.12:0/1201757321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9320100280 0x7f9320100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 -- 172.21.15.12:0/1201757321 >> 172.21.15.12:0/1201757321 conn(0x7f9320076bc0 msgr2=0x7f9320076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 -- 172.21.15.12:0/1201757321 shutdown_connections 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 -- 172.21.15.12:0/1201757321 wait complete. 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 Processor -- start 2024-05-08T23:48:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.308+0000 7f93269e6700 1 -- start start 2024-05-08T23:48:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f93269e6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9320100280 0x7f9320197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f93269e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9320100c40 0x7f93201977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f93269e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9320105f20 0x7f932019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f93269e6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932019c140 con 0x7f9320100280 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f93269e6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932019c2b0 con 0x7f9320100c40 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f93269e6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932019c470 con 0x7f9320105f20 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f931ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9320100280 0x7f9320197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f931ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9320100280 0x7f9320197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37802/0 (socket says 172.21.15.12:37802) 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f931ffff700 1 -- 172.21.15.12:0/2482427673 learned_addr learned my addr 172.21.15.12:0/2482427673 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f931f7fe700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9320100c40 0x7f93201977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f9324f83700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9320105f20 0x7f932019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f931ffff700 1 -- 172.21.15.12:0/2482427673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9320105f20 msgr2=0x7f932019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f931ffff700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9320105f20 0x7f932019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f931ffff700 1 -- 172.21.15.12:0/2482427673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9320100c40 msgr2=0x7f93201977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f931ffff700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9320100c40 0x7f93201977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.309+0000 7f931ffff700 1 -- 172.21.15.12:0/2482427673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93100095f0 con 0x7f9320100280 2024-05-08T23:48:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.310+0000 7f931ffff700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9320100280 0x7f9320197270 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9310005770 tx=0x7f93100057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.310+0000 7f931d7fa700 1 -- 172.21.15.12:0/2482427673 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931001d070 con 0x7f9320100280 2024-05-08T23:48:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.310+0000 7f931d7fa700 1 -- 172.21.15.12:0/2482427673 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9310021410 con 0x7f9320100280 2024-05-08T23:48:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.310+0000 7f93269e6700 1 -- 172.21.15.12:0/2482427673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932019c6f0 con 0x7f9320100280 2024-05-08T23:48:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.310+0000 7f931d7fa700 1 -- 172.21.15.12:0/2482427673 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310017650 con 0x7f9320100280 2024-05-08T23:48:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.310+0000 7f93269e6700 1 -- 172.21.15.12:0/2482427673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93201a3f90 con 0x7f9320100280 2024-05-08T23:48:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.312+0000 7f931d7fa700 1 -- 172.21.15.12:0/2482427673 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f93100217f0 con 0x7f9320100280 2024-05-08T23:48:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.313+0000 7f931d7fa700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93080a1290 0x7f93080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.313+0000 7f931d7fa700 1 -- 172.21.15.12:0/2482427673 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f93100cdff0 con 0x7f9320100280 2024-05-08T23:48:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.313+0000 7f931f7fe700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93080a1290 0x7f93080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f931f7fe700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93080a1290 0x7f93080a3750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9320198770 tx=0x7f9314009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f9332ba0b80 1 Processor -- start 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f9332ba0b80 1 -- start start 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6aa270 0x561eca6a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a4480 0x561eca6a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca5fb0f0 con 0x561eca6a9950 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca5f8ab0 con 0x561eca6aa270 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca5f43a0 con 0x561eca6a4480 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f931ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f931ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6a9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37816/0 (socket says 172.21.15.12:37816) 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f931f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6aa270 0x561eca6a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.314+0000 7f931ffff700 1 -- 172.21.15.12:0/3131821371 learned_addr learned my addr 172.21.15.12:0/3131821371 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f9324f83700 1 --2- 172.21.15.12:0/3131821371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a4480 0x561eca6a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f931ffff700 1 -- 172.21.15.12:0/3131821371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a4480 msgr2=0x561eca6a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f931ffff700 1 --2- 172.21.15.12:0/3131821371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a4480 0x561eca6a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f931ffff700 1 -- 172.21.15.12:0/3131821371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6aa270 msgr2=0x561eca6a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f931ffff700 1 --2- 172.21.15.12:0/3131821371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6aa270 0x561eca6a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f931ffff700 1 -- 172.21.15.12:0/3131821371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eca475350 con 0x561eca6a9950 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f931f7fe700 1 --2- 172.21.15.12:0/3131821371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6aa270 0x561eca6a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f9324f83700 1 --2- 172.21.15.12:0/3131821371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a4480 0x561eca6a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f931ffff700 1 --2- 172.21.15.12:0/3131821371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6a9d30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f93100098e0 tx=0x7f9310097600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7349f2c2aa9b7239 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f93067fc700 1 -- 172.21.15.12:0/3131821371 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931009b1b0 con 0x561eca6a9950 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f93067fc700 1 -- 172.21.15.12:0/3131821371 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f931009efe0 con 0x561eca6a9950 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.315+0000 7f93067fc700 1 -- 172.21.15.12:0/3131821371 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931009b610 con 0x561eca6a9950 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.316+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3131821371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 msgr2=0x561eca6a9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.316+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/3131821371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6a9d30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f93100098e0 tx=0x7f9310097600 comp rx=0 tx=0).stop 2024-05-08T23:48:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.316+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3131821371 shutdown_connections 2024-05-08T23:48:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.316+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/3131821371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a4480 0x561eca6a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.316+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/3131821371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6a9d30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.316+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/3131821371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6aa270 0x561eca6a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.316+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3131821371 >> 172.21.15.12:0/3131821371 conn(0x561eca6021b0 msgr2=0x561eca69e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.316+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3131821371 shutdown_connections 2024-05-08T23:48:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.316+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3131821371 wait complete. 2024-05-08T23:48:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.316+0000 7f9332ba0b80 1 -- start start 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f9332ba0b80 1 Processor -- start 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca6b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6aa270 0x561eca6adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca590500 con 0x561eca6a9950 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca5901a0 con 0x561eca6a4480 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca477640 con 0x561eca6aa270 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f9332ba0b80 10 client.0 _ll_register_callbacks cb 0x561eca599ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f931f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f931ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca6b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f931f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6ad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37820/0 (socket says 172.21.15.12:37820) 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f931ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca6b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52444/0 (socket says 172.21.15.12:52444) 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f9324f83700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6aa270 0x561eca6adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f931ffff700 1 -- 172.21.15.12:0/3297874480 learned_addr learned my addr 172.21.15.12:0/3297874480 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f931ffff700 1 -- 172.21.15.12:0/3297874480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6aa270 msgr2=0x561eca6adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f931ffff700 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6aa270 0x561eca6adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f931ffff700 1 -- 172.21.15.12:0/3297874480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 msgr2=0x561eca6ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f931ffff700 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.317+0000 7f931ffff700 1 -- 172.21.15.12:0/3297874480 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9314017070 con 0x561eca6a4480 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f931ffff700 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca6b01f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f931009aeb0 tx=0x7f9310005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e05dc581d7301436 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f92ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f92ff7fe700 1 -- 172.21.15.12:0/3297874480 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931009a8a0 con 0x561eca6a4480 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f92ff7fe700 1 -- 172.21.15.12:0/3297874480 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f931009b1b0 con 0x561eca6a4480 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3297874480 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561eca54d210 con 0x561eca6a4480 2024-05-08T23:48:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f92ff7fe700 1 -- 172.21.15.12:0/3297874480 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f931009b410 con 0x561eca6a4480 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f9332ba0b80 10 client.25664 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3297874480 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f93201a3f90 con 0x561eca6a4480 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f92ff7fe700 1 -- 172.21.15.12:0/3297874480 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561eca54d210 con 0x561eca6a4480 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f92ff7fe700 1 -- 172.21.15.12:0/3297874480 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9310097080 con 0x561eca6a4480 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f9332ba0b80 10 client.25664 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f9332ba0b80 10 client.25664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f9332ba0b80 20 client.25664 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561eca6bd490 0x561eca6bf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f9332ba0b80 4 client.25664 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.318+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3297874480 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561eca6737b0 con 0x561eca6bd490 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.319+0000 7f931f7fe700 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561eca6bd490 0x561eca6bf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.319+0000 7f931f7fe700 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561eca6bd490 0x561eca6bf8b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5953637d7e31ba3 server_cookie=770ffc16189fe2bb in_seq=0 out_seq=0 2024-05-08T23:48:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.319+0000 7f92ff7fe700 10 client.25664 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f92ff7fe700 1 -- 172.21.15.12:0/3297874480 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561eca6737b0 con 0x561eca6bd490 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f92ff7fe700 10 client.25664 handle_command_reply: tid=0 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 client.25664 shutdown 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3297874480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 msgr2=0x561eca6b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca6b01f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f931009aeb0 tx=0x7f9310005810 comp rx=0 tx=0).stop 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3297874480 shutdown_connections 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6aa270 0x561eca6adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561eca6bd490 0x561eca6bf8b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca6b01f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/3297874480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6a9950 0x561eca6ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f92ff7fe700 0 client.25664 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3297874480 >> 172.21.15.12:0/3297874480 conn(0x561eca6021b0 msgr2=0x561eca6a2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3297874480 shutdown_connections 2024-05-08T23:48:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 1 -- 172.21.15.12:0/3297874480 wait complete. 2024-05-08T23:48:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.320+0000 7f9332ba0b80 20 client.25664 trim_cache size 0 max 16384 2024-05-08T23:48:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9332ba0b80 1 Processor -- start 2024-05-08T23:48:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9332ba0b80 1 -- start start 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca69e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 0x561eca6acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca6ad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca6c4a80 con 0x561eca6aa270 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca69fa30 con 0x561eca6a4480 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca6ae580 con 0x561eca6a9950 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9324f83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca6ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f931ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca69e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9324f83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca6ad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37832/0 (socket says 172.21.15.12:37832) 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f9324f83700 1 -- 172.21.15.12:0/4143553611 learned_addr learned my addr 172.21.15.12:0/4143553611 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.337+0000 7f931f7fe700 1 --2- 172.21.15.12:0/4143553611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 0x561eca6acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.338+0000 7f9324f83700 1 -- 172.21.15.12:0/4143553611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 msgr2=0x561eca6acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.338+0000 7f9324f83700 1 --2- 172.21.15.12:0/4143553611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 0x561eca6acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.338+0000 7f9324f83700 1 -- 172.21.15.12:0/4143553611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 msgr2=0x561eca69e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.338+0000 7f9324f83700 1 --2- 172.21.15.12:0/4143553611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca69e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.338+0000 7f9324f83700 1 -- 172.21.15.12:0/4143553611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eca421b00 con 0x561eca6aa270 2024-05-08T23:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.338+0000 7f931ffff700 1 --2- 172.21.15.12:0/4143553611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca69e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.338+0000 7f9324f83700 1 --2- 172.21.15.12:0/4143553611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca6ad240 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9318009f00 tx=0x7f93180077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5e437643c5e707 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.338+0000 7f9306ffd700 1 -- 172.21.15.12:0/4143553611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318007e00 con 0x561eca6aa270 2024-05-08T23:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.338+0000 7f9306ffd700 1 -- 172.21.15.12:0/4143553611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f931800f070 con 0x561eca6aa270 2024-05-08T23:48:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.338+0000 7f9306ffd700 1 -- 172.21.15.12:0/4143553611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318004020 con 0x561eca6aa270 2024-05-08T23:48:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 -- 172.21.15.12:0/4143553611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 msgr2=0x561eca6ad240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/4143553611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca6ad240 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9318009f00 tx=0x7f93180077e0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 -- 172.21.15.12:0/4143553611 shutdown_connections 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/4143553611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca6ad240 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/4143553611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca69e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/4143553611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 0x561eca6acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 -- 172.21.15.12:0/4143553611 >> 172.21.15.12:0/4143553611 conn(0x561eca6021b0 msgr2=0x561eca6afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 -- 172.21.15.12:0/4143553611 shutdown_connections 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 -- 172.21.15.12:0/4143553611 wait complete. 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 -- start start 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.339+0000 7f9332ba0b80 1 Processor -- start 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca6b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 0x561eca6b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9332ba0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca712950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca6b3750 con 0x561eca6aa270 2024-05-08T23:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca712e90 con 0x561eca6a4480 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9332ba0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eca712fd0 con 0x561eca6a9950 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9332ba0b80 10 client.0 _ll_register_callbacks cb 0x561eca599ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9324f83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca712950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f931f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 0x561eca6b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9324f83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca712950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37840/0 (socket says 172.21.15.12:37840) 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f931ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca6b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9324f83700 1 -- 172.21.15.12:0/536070095 learned_addr learned my addr 172.21.15.12:0/536070095 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9324f83700 1 -- 172.21.15.12:0/536070095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 msgr2=0x561eca6b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9324f83700 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 0x561eca6b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9324f83700 1 -- 172.21.15.12:0/536070095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 msgr2=0x561eca6b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9324f83700 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca6b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9324f83700 1 -- 172.21.15.12:0/536070095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93201a3f90 con 0x561eca6aa270 2024-05-08T23:48:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f931f7fe700 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 0x561eca6b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9324f83700 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca712950 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f931800c010 tx=0x7f9318003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2292e8092b8c8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f92ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f92ff7fe700 1 -- 172.21.15.12:0/536070095 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318024070 con 0x561eca6aa270 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f92ff7fe700 1 -- 172.21.15.12:0/536070095 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9318017650 con 0x561eca6aa270 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.340+0000 7f9332ba0b80 1 -- 172.21.15.12:0/536070095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f931009efe0 con 0x561eca6aa270 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f92ff7fe700 1 -- 172.21.15.12:0/536070095 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9318017950 con 0x561eca6aa270 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f9332ba0b80 10 client.15825 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f9332ba0b80 1 -- 172.21.15.12:0/536070095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f9318009bb0 con 0x561eca6aa270 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f92ff7fe700 1 -- 172.21.15.12:0/536070095 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f931009efe0 con 0x561eca6aa270 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f92ff7fe700 1 -- 172.21.15.12:0/536070095 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9318014050 con 0x561eca6aa270 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f9332ba0b80 10 client.15825 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f9332ba0b80 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f9332ba0b80 20 client.15825 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561eca714780 0x561eca716ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f9332ba0b80 4 client.15825 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f9332ba0b80 1 -- 172.21.15.12:0/536070095 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561eca5ece60 con 0x561eca714780 2024-05-08T23:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.341+0000 7f931ffff700 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561eca714780 0x561eca716ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.342+0000 7f931ffff700 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561eca714780 0x561eca716ba0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=615cc0e69b87623e server_cookie=29677d77ed1f5a75 in_seq=0 out_seq=0 2024-05-08T23:48:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.342+0000 7f92ff7fe700 10 client.15825 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f92ff7fe700 1 -- 172.21.15.12:0/536070095 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561eca5ece60 con 0x561eca714780 2024-05-08T23:48:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f92ff7fe700 10 client.15825 handle_command_reply: tid=0 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f9332ba0b80 1 client.15825 shutdown 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f9332ba0b80 1 -- 172.21.15.12:0/536070095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 msgr2=0x561eca712950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca712950 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f931800c010 tx=0x7f9318003c30 comp rx=0 tx=0).stop 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f9332ba0b80 1 -- 172.21.15.12:0/536070095 shutdown_connections 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561eca6aa270 0x561eca712950 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561eca714780 0x561eca716ba0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561eca6a4480 0x561eca6b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f9332ba0b80 1 --2- 172.21.15.12:0/536070095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561eca6a9950 0x561eca6b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f92ff7fe700 0 client.15825 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f9332ba0b80 1 -- 172.21.15.12:0/536070095 >> 172.21.15.12:0/536070095 conn(0x561eca6021b0 msgr2=0x561eca6a85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.435+0000 7f9332ba0b80 1 -- 172.21.15.12:0/536070095 shutdown_connections 2024-05-08T23:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f9332ba0b80 1 -- 172.21.15.12:0/536070095 wait complete. 2024-05-08T23:48:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f9332ba0b80 20 client.15825 trim_cache size 0 max 16384 2024-05-08T23:48:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:48:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:48:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:48:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:48:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f93269e6700 1 -- 172.21.15.12:0/2482427673 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93080a1290 msgr2=0x7f93080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f93269e6700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93080a1290 0x7f93080a3750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9320198770 tx=0x7f9314009260 comp rx=0 tx=0).stop 2024-05-08T23:48:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f93269e6700 1 -- 172.21.15.12:0/2482427673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9320100280 msgr2=0x7f9320197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f93269e6700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9320100280 0x7f9320197270 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9310005770 tx=0x7f93100057a0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f93269e6700 1 -- 172.21.15.12:0/2482427673 shutdown_connections 2024-05-08T23:48:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f93269e6700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93080a1290 0x7f93080a3750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f93269e6700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9320105f20 0x7f932019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f93269e6700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9320100280 0x7f9320197270 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f93269e6700 1 --2- 172.21.15.12:0/2482427673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9320100c40 0x7f93201977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.436+0000 7f93269e6700 1 -- 172.21.15.12:0/2482427673 >> 172.21.15.12:0/2482427673 conn(0x7f9320076bc0 msgr2=0x7f93200692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.437+0000 7f93269e6700 1 -- 172.21.15.12:0/2482427673 shutdown_connections 2024-05-08T23:48:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:09.437+0000 7f93269e6700 1 -- 172.21.15.12:0/2482427673 wait complete. 2024-05-08T23:48:09.810 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:48:09.845 INFO:tasks.workunit.client.0.smithi012.stdout: 1 15854 20.54 MB/sec warmup 5 sec latency 45.791 ms 2024-05-08T23:48:09.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.965+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2395031568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c80696f0 msgr2=0x7f19c8069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.965+0000 7f19cfcfb700 1 --2- 172.21.15.12:0/2395031568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c80696f0 0x7f19c8069b70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009990 tx=0x7f19b8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.965+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2395031568 shutdown_connections 2024-05-08T23:48:09.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.965+0000 7f19cfcfb700 1 --2- 172.21.15.12:0/2395031568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c810d7f0 0x7f19c810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.965+0000 7f19cfcfb700 1 --2- 172.21.15.12:0/2395031568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19c8101e20 0x7f19c80691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.965+0000 7f19cfcfb700 1 --2- 172.21.15.12:0/2395031568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c80696f0 0x7f19c8069b70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.965+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2395031568 >> 172.21.15.12:0/2395031568 conn(0x7f19c8076ba0 msgr2=0x7f19c8076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:09.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.966+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2395031568 shutdown_connections 2024-05-08T23:48:09.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.966+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2395031568 wait complete. 2024-05-08T23:48:09.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.966+0000 7f19cfcfb700 1 Processor -- start 2024-05-08T23:48:09.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.966+0000 7f19cfcfb700 1 -- start start 2024-05-08T23:48:09.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19cfcfb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c80696f0 0x7f19c81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19cfcfb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19c8101e20 0x7f19c8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19cfcfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c810d7f0 0x7f19c819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19cfcfb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c819e4d0 con 0x7f19c810d7f0 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19cfcfb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c819e640 con 0x7f19c8101e20 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19cfcfb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c819e800 con 0x7f19c80696f0 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19cda97700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c80696f0 0x7f19c81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19cd296700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19c8101e20 0x7f19c8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19ce298700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c810d7f0 0x7f19c819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19cd296700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19c8101e20 0x7f19c8199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52492/0 (socket says 172.21.15.12:52492) 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19cd296700 1 -- 172.21.15.12:0/2779395942 learned_addr learned my addr 172.21.15.12:0/2779395942 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19ce298700 1 -- 172.21.15.12:0/2779395942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c80696f0 msgr2=0x7f19c81995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19ce298700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c80696f0 0x7f19c81995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19ce298700 1 -- 172.21.15.12:0/2779395942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19c8101e20 msgr2=0x7f19c8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19ce298700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19c8101e20 0x7f19c8199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19ce298700 1 -- 172.21.15.12:0/2779395942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c4009520 con 0x7f19c810d7f0 2024-05-08T23:48:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.967+0000 7f19ce298700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c810d7f0 0x7f19c819de90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f19c000ef90 tx=0x7f19c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.968+0000 7f19beffd700 1 -- 172.21.15.12:0/2779395942 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c0018070 con 0x7f19c810d7f0 2024-05-08T23:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.968+0000 7f19beffd700 1 -- 172.21.15.12:0/2779395942 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19c000f040 con 0x7f19c810d7f0 2024-05-08T23:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.968+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2779395942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b80095f0 con 0x7f19c810d7f0 2024-05-08T23:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.968+0000 7f19beffd700 1 -- 172.21.15.12:0/2779395942 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c000cc30 con 0x7f19c810d7f0 2024-05-08T23:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.968+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2779395942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c81a6140 con 0x7f19c810d7f0 2024-05-08T23:48:09.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.971+0000 7f19beffd700 1 -- 172.21.15.12:0/2779395942 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f19c0003a00 con 0x7f19c810d7f0 2024-05-08T23:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.971+0000 7f19beffd700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19b40a1290 0x7f19b40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.971+0000 7f19beffd700 1 -- 172.21.15.12:0/2779395942 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f19c00cc7d0 con 0x7f19c810d7f0 2024-05-08T23:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.971+0000 7f19cda97700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19b40a1290 0x7f19b40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.972+0000 7f19cda97700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19b40a1290 0x7f19b40a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009df0 tx=0x7f19c4009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.972+0000 7f19dbeb5b80 1 Processor -- start 2024-05-08T23:48:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.972+0000 7f19dbeb5b80 1 -- start start 2024-05-08T23:48:09.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.972+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acc720 0x556cd2ac6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2ac6890 0x556cd2ac6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd2a1d150 con 0x556cd2acbe00 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd2a1ab10 con 0x556cd2acc720 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd2a16400 con 0x556cd2ac6890 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cda97700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cda97700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37870/0 (socket says 172.21.15.12:37870) 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cda97700 1 -- 172.21.15.12:0/1308866301 learned_addr learned my addr 172.21.15.12:0/1308866301 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cd296700 1 --2- 172.21.15.12:0/1308866301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acc720 0x556cd2ac6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19ce298700 1 --2- 172.21.15.12:0/1308866301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2ac6890 0x556cd2ac6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cd296700 1 -- 172.21.15.12:0/1308866301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2ac6890 msgr2=0x556cd2ac6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cd296700 1 --2- 172.21.15.12:0/1308866301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2ac6890 0x556cd2ac6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cd296700 1 -- 172.21.15.12:0/1308866301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 msgr2=0x556cd2acc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cd296700 1 --2- 172.21.15.12:0/1308866301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cd296700 1 -- 172.21.15.12:0/1308866301 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cd28973b0 con 0x556cd2acc720 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cda97700 1 --2- 172.21.15.12:0/1308866301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19cd296700 1 --2- 172.21.15.12:0/1308866301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acc720 0x556cd2ac6350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f19b8006010 tx=0x7f19b800b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=368857c4085365e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19b3fff700 1 -- 172.21.15.12:0/1308866301 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b801d070 con 0x556cd2acc720 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19b3fff700 1 -- 172.21.15.12:0/1308866301 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b801e040 con 0x556cd2acc720 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.973+0000 7f19b3fff700 1 -- 172.21.15.12:0/1308866301 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b8004790 con 0x556cd2acc720 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.974+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/1308866301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acc720 msgr2=0x556cd2ac6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.974+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/1308866301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acc720 0x556cd2ac6350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f19b8006010 tx=0x7f19b800b610 comp rx=0 tx=0).stop 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.974+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/1308866301 shutdown_connections 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.974+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/1308866301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2ac6890 0x556cd2ac6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.974+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/1308866301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.974+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/1308866301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acc720 0x556cd2ac6350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.974+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/1308866301 >> 172.21.15.12:0/1308866301 conn(0x556cd2a24210 msgr2=0x556cd2ac0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.974+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/1308866301 shutdown_connections 2024-05-08T23:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.974+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/1308866301 wait complete. 2024-05-08T23:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.975+0000 7f19dbeb5b80 1 -- start start 2024-05-08T23:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.975+0000 7f19dbeb5b80 1 Processor -- start 2024-05-08T23:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.975+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2acfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.975+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.975+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 0x556cd2ad0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.975+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd29b2560 con 0x556cd2acbe00 2024-05-08T23:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.975+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd29b2200 con 0x556cd2ac6890 2024-05-08T23:48:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.975+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd28996a0 con 0x556cd2acc720 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19dbeb5b80 10 client.0 _ll_register_callbacks cb 0x556cd29bbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19cd296700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19cda97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2acfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19ce298700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 0x556cd2ad0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19cd296700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37886/0 (socket says 172.21.15.12:37886) 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19cda97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2acfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52502/0 (socket says 172.21.15.12:52502) 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19cd296700 1 -- 172.21.15.12:0/4009166035 learned_addr learned my addr 172.21.15.12:0/4009166035 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19ce298700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 0x556cd2ad0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44450/0 (socket says 172.21.15.12:44450) 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19ce298700 1 -- 172.21.15.12:0/4009166035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 msgr2=0x556cd2acfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19ce298700 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2acfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19ce298700 1 -- 172.21.15.12:0/4009166035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 msgr2=0x556cd2acff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19ce298700 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19ce298700 1 -- 172.21.15.12:0/4009166035 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c4017070 con 0x556cd2acc720 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19ce298700 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 0x556cd2ad0490 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f19c00c9f60 tx=0x7f19c001b320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfe051fee0d0966e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19b17fa700 1 -- 172.21.15.12:0/4009166035 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c009d7e0 con 0x556cd2acc720 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/4009166035 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cd296f270 con 0x556cd2acc720 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.976+0000 7f19b17fa700 1 -- 172.21.15.12:0/4009166035 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f19c00a1da0 con 0x556cd2acc720 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19b17fa700 1 -- 172.21.15.12:0/4009166035 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19c00b2280 con 0x556cd2acc720 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19dbeb5b80 10 client.25690 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/4009166035 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f19c8110880 con 0x556cd2acc720 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19b17fa700 1 -- 172.21.15.12:0/4009166035 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19c00b6080 con 0x556cd2acc720 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19b17fa700 1 -- 172.21.15.12:0/4009166035 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f19c0098ea0 con 0x556cd2acc720 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19dbeb5b80 10 client.25690 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19dbeb5b80 10 client.25690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19dbeb5b80 20 client.25690 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556cd2ae0220 0x556cd2ae2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19dbeb5b80 4 client.25690 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/4009166035 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cd2a95810 con 0x556cd2ae0220 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.977+0000 7f19cda97700 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556cd2ae0220 0x556cd2ae2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.978+0000 7f19b17fa700 10 client.25690 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.978+0000 7f19cda97700 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556cd2ae0220 0x556cd2ae2640 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa04bc5d31c61e1f server_cookie=91ceaf85e992afd7 in_seq=0 out_seq=0 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.978+0000 7f19b17fa700 1 -- 172.21.15.12:0/4009166035 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556cd2a95810 con 0x556cd2ae0220 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.978+0000 7f19b17fa700 10 client.25690 handle_command_reply: tid=0 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.978+0000 7f19dbeb5b80 1 client.25690 shutdown 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.978+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/4009166035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 msgr2=0x556cd2ad0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.978+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 0x556cd2ad0490 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f19c00c9f60 tx=0x7f19c001b320 comp rx=0 tx=0).stop 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.979+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/4009166035 shutdown_connections 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.979+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 0x556cd2ad0490 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.979+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556cd2ae0220 0x556cd2ae2640 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.979+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2acfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.979+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/4009166035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acbe00 0x556cd2acff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.979+0000 7f19b17fa700 0 client.25690 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:09.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.979+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/4009166035 >> 172.21.15.12:0/4009166035 conn(0x556cd2a24210 msgr2=0x556cd2ac05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:09.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.979+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/4009166035 shutdown_connections 2024-05-08T23:48:09.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.979+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/4009166035 wait complete. 2024-05-08T23:48:09.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.979+0000 7f19dbeb5b80 20 client.25690 trim_cache size 0 max 16384 2024-05-08T23:48:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19dbeb5b80 1 Processor -- start 2024-05-08T23:48:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19dbeb5b80 1 -- start start 2024-05-08T23:48:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2ac0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acbe00 0x556cd2ac2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acc720 0x556cd2ace300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd2ae7850 con 0x556cd2acc720 2024-05-08T23:48:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd2abb800 con 0x556cd2ac6890 2024-05-08T23:48:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd2acdf80 con 0x556cd2acbe00 2024-05-08T23:48:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19cda97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2ac0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19ce298700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acc720 0x556cd2ace300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19cda97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2ac0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52518/0 (socket says 172.21.15.12:52518) 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19cda97700 1 -- 172.21.15.12:0/2883518421 learned_addr learned my addr 172.21.15.12:0/2883518421 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19ce298700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acc720 0x556cd2ace300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37888/0 (socket says 172.21.15.12:37888) 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:09.999+0000 7f19cd296700 1 --2- 172.21.15.12:0/2883518421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acbe00 0x556cd2ac2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.000+0000 7f19cda97700 1 -- 172.21.15.12:0/2883518421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acbe00 msgr2=0x556cd2ac2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.000+0000 7f19cda97700 1 --2- 172.21.15.12:0/2883518421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acbe00 0x556cd2ac2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.000+0000 7f19cda97700 1 -- 172.21.15.12:0/2883518421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acc720 msgr2=0x556cd2ace300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.000+0000 7f19cda97700 1 --2- 172.21.15.12:0/2883518421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acc720 0x556cd2ace300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.000+0000 7f19cda97700 1 -- 172.21.15.12:0/2883518421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cd2843b60 con 0x556cd2ac6890 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.000+0000 7f19ce298700 1 --2- 172.21.15.12:0/2883518421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acc720 0x556cd2ace300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.000+0000 7f19cda97700 1 --2- 172.21.15.12:0/2883518421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2ac0d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19c4011ab0 tx=0x7f19c4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9bf3c5e8d58674 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.000+0000 7f19bcff9700 1 -- 172.21.15.12:0/2883518421 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4022070 con 0x556cd2ac6890 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.000+0000 7f19bcff9700 1 -- 172.21.15.12:0/2883518421 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19c4023040 con 0x556cd2ac6890 2024-05-08T23:48:09.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.000+0000 7f19bcff9700 1 -- 172.21.15.12:0/2883518421 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400e890 con 0x556cd2ac6890 2024-05-08T23:48:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/2883518421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 msgr2=0x556cd2ac0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/2883518421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2ac0d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19c4011ab0 tx=0x7f19c4014000 comp rx=0 tx=0).stop 2024-05-08T23:48:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/2883518421 shutdown_connections 2024-05-08T23:48:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/2883518421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2acc720 0x556cd2ace300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/2883518421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2ac6890 0x556cd2ac0d20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/2883518421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acbe00 0x556cd2ac2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/2883518421 >> 172.21.15.12:0/2883518421 conn(0x556cd2a24210 msgr2=0x556cd2ac6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/2883518421 shutdown_connections 2024-05-08T23:48:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/2883518421 wait complete. 2024-05-08T23:48:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 -- start start 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.001+0000 7f19dbeb5b80 1 Processor -- start 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2ac6890 0x556cd2ad50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acbe00 0x556cd2ad5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19dbeb5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 0x556cd2ad5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd2b358f0 con 0x556cd2ac6890 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd2b35a60 con 0x556cd2acbe00 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19dbeb5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd2b35c20 con 0x556cd2acc720 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19dbeb5b80 10 client.0 _ll_register_callbacks cb 0x556cd29bbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19cda97700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2ac6890 0x556cd2ad50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19cda97700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2ac6890 0x556cd2ad50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37898/0 (socket says 172.21.15.12:37898) 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19cda97700 1 -- 172.21.15.12:0/411805236 learned_addr learned my addr 172.21.15.12:0/411805236 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19cd296700 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acbe00 0x556cd2ad5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19ce298700 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 0x556cd2ad5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19cda97700 1 -- 172.21.15.12:0/411805236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 msgr2=0x556cd2ad5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19cda97700 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 0x556cd2ad5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19cda97700 1 -- 172.21.15.12:0/411805236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acbe00 msgr2=0x556cd2ad5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19cda97700 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acbe00 0x556cd2ad5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19cda97700 1 -- 172.21.15.12:0/411805236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c8110880 con 0x556cd2ac6890 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19cda97700 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2ac6890 0x556cd2ad50f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f19c40094f0 tx=0x7f19c4003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bfc7c2d8a9e3df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.002+0000 7f19b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19b17fa700 1 -- 172.21.15.12:0/411805236 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4022040 con 0x556cd2ac6890 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19b17fa700 1 -- 172.21.15.12:0/411805236 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f19c402b2b0 con 0x556cd2ac6890 2024-05-08T23:48:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/411805236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19b801e040 con 0x556cd2ac6890 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19b17fa700 1 -- 172.21.15.12:0/411805236 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19c402b470 con 0x556cd2ac6890 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19dbeb5b80 10 client.15843 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/411805236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f19c40040e0 con 0x556cd2ac6890 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19b17fa700 1 -- 172.21.15.12:0/411805236 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b801e040 con 0x556cd2ac6890 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19b17fa700 1 -- 172.21.15.12:0/411805236 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f19c4035450 con 0x556cd2ac6890 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19dbeb5b80 10 client.15843 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19dbeb5b80 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19dbeb5b80 20 client.15843 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556cd2b372e0 0x556cd2b396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19dbeb5b80 4 client.15843 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.003+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/411805236 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556cd2a0eec0 con 0x556cd2b372e0 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19cd296700 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556cd2b372e0 0x556cd2b396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19cd296700 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556cd2b372e0 0x556cd2b396d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2471a65eb0fd1b7f server_cookie=b7705db80673088a in_seq=0 out_seq=0 2024-05-08T23:48:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19b17fa700 10 client.15843 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19b17fa700 1 -- 172.21.15.12:0/411805236 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556cd2a0eec0 con 0x556cd2b372e0 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19b17fa700 10 client.15843 handle_command_reply: tid=0 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19dbeb5b80 1 client.15843 shutdown 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/411805236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2ac6890 msgr2=0x556cd2ad50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2ac6890 0x556cd2ad50f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f19c40094f0 tx=0x7f19c4003cc0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/411805236 shutdown_connections 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd2acc720 0x556cd2ad5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556cd2ac6890 0x556cd2ad50f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556cd2b372e0 0x556cd2b396d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19dbeb5b80 1 --2- 172.21.15.12:0/411805236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556cd2acbe00 0x556cd2ad5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/411805236 >> 172.21.15.12:0/411805236 conn(0x556cd2a24210 msgr2=0x556cd2ac9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.004+0000 7f19b17fa700 0 client.15843 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/411805236 shutdown_connections 2024-05-08T23:48:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19dbeb5b80 1 -- 172.21.15.12:0/411805236 wait complete. 2024-05-08T23:48:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19dbeb5b80 20 client.15843 trim_cache size 0 max 16384 2024-05-08T23:48:10.007 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:48:10.007 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:48:10.007 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:48:10.007 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:48:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2779395942 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19b40a1290 msgr2=0x7f19b40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19b40a1290 0x7f19b40a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009df0 tx=0x7f19c4009260 comp rx=0 tx=0).stop 2024-05-08T23:48:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2779395942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c810d7f0 msgr2=0x7f19c819de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c810d7f0 0x7f19c819de90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f19c000ef90 tx=0x7f19c000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2779395942 shutdown_connections 2024-05-08T23:48:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19b40a1290 0x7f19b40a3750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c810d7f0 0x7f19c819de90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c80696f0 0x7f19c81995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 --2- 172.21.15.12:0/2779395942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19c8101e20 0x7f19c8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2779395942 >> 172.21.15.12:0/2779395942 conn(0x7f19c8076ba0 msgr2=0x7f19c80ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2779395942 shutdown_connections 2024-05-08T23:48:10.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.005+0000 7f19cfcfb700 1 -- 172.21.15.12:0/2779395942 wait complete. 2024-05-08T23:48:10.014 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:48:10.015 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:48:10.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.216+0000 7fdaecc45700 1 -- 172.21.15.12:0/596489223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdae8101060 msgr2=0x7fdae8101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.216+0000 7fdaecc45700 1 --2- 172.21.15.12:0/596489223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdae8101060 0x7fdae8101440 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdad80098f0 tx=0x7fdad8009c00 comp rx=0 tx=0).stop 2024-05-08T23:48:10.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.216+0000 7fdaecc45700 1 -- 172.21.15.12:0/596489223 shutdown_connections 2024-05-08T23:48:10.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.216+0000 7fdaecc45700 1 --2- 172.21.15.12:0/596489223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdae8106180 0x7fdae810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.216+0000 7fdaecc45700 1 --2- 172.21.15.12:0/596489223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdae8101060 0x7fdae8101440 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.216+0000 7fdaecc45700 1 --2- 172.21.15.12:0/596489223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdae8101a10 0x7fdae8105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.216+0000 7fdaecc45700 1 -- 172.21.15.12:0/596489223 >> 172.21.15.12:0/596489223 conn(0x7fdae80fc920 msgr2=0x7fdae80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.216+0000 7fdaecc45700 1 -- 172.21.15.12:0/596489223 shutdown_connections 2024-05-08T23:48:10.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.216+0000 7fdaecc45700 1 -- 172.21.15.12:0/596489223 wait complete. 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.217+0000 7fdaecc45700 1 Processor -- start 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.217+0000 7fdaecc45700 1 -- start start 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdaecc45700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdae8101060 0x7fdae81973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdaecc45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdae8106180 0x7fdae81978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdaecc45700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdae819bc90 0x7fdae819c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdaecc45700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae819c710 con 0x7fdae8106180 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdaecc45700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae819c880 con 0x7fdae819bc90 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdaecc45700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae819ca40 con 0x7fdae8101060 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdae8101060 0x7fdae81973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdae8101060 0x7fdae81973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44486/0 (socket says 172.21.15.12:44486) 2024-05-08T23:48:10.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae659c700 1 -- 172.21.15.12:0/1664675010 learned_addr learned my addr 172.21.15.12:0/1664675010 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdae8106180 0x7fdae81978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae6d9d700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdae819bc90 0x7fdae819c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae659c700 1 -- 172.21.15.12:0/1664675010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdae819bc90 msgr2=0x7fdae819c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae659c700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdae819bc90 0x7fdae819c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae659c700 1 -- 172.21.15.12:0/1664675010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdae8106180 msgr2=0x7fdae81978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae659c700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdae8106180 0x7fdae81978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae659c700 1 -- 172.21.15.12:0/1664675010 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad8009520 con 0x7fdae8101060 2024-05-08T23:48:10.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae6d9d700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdae819bc90 0x7fdae819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:10.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae5d9b700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdae8106180 0x7fdae81978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:10.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.218+0000 7fdae659c700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdae8101060 0x7fdae81973b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdad8005950 tx=0x7fdad800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.219+0000 7fdad77fe700 1 -- 172.21.15.12:0/1664675010 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad801c070 con 0x7fdae8101060 2024-05-08T23:48:10.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.219+0000 7fdaecc45700 1 -- 172.21.15.12:0/1664675010 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae81a3f40 con 0x7fdae8101060 2024-05-08T23:48:10.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.219+0000 7fdad77fe700 1 -- 172.21.15.12:0/1664675010 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdad800b9c0 con 0x7fdae8101060 2024-05-08T23:48:10.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.219+0000 7fdad77fe700 1 -- 172.21.15.12:0/1664675010 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad80038e0 con 0x7fdae8101060 2024-05-08T23:48:10.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.219+0000 7fdaecc45700 1 -- 172.21.15.12:0/1664675010 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae81a4360 con 0x7fdae8101060 2024-05-08T23:48:10.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.220+0000 7fdaecc45700 1 -- 172.21.15.12:0/1664675010 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae81093f0 con 0x7fdae8101060 2024-05-08T23:48:10.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.221+0000 7fdad77fe700 1 -- 172.21.15.12:0/1664675010 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdad8020030 con 0x7fdae8101060 2024-05-08T23:48:10.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.222+0000 7fdad77fe700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdad00a1290 0x7fdad00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.222+0000 7fdae5d9b700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdad00a1290 0x7fdad00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.222+0000 7fdad77fe700 1 -- 172.21.15.12:0/1664675010 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fdad80cdb50 con 0x7fdae8101060 2024-05-08T23:48:10.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.222+0000 7fdae5d9b700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdad00a1290 0x7fdad00a3750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdae8198900 tx=0x7fdadc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.225+0000 7fdad77fe700 1 -- 172.21.15.12:0/1664675010 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdad8097070 con 0x7fdae8101060 2024-05-08T23:48:10.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.498+0000 7fdaecc45700 1 -- 172.21.15.12:0/1664675010 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdae8102530 con 0x7fdae8101060 2024-05-08T23:48:10.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.499+0000 7fdad77fe700 1 -- 172.21.15.12:0/1664675010 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10876 (secure 0 0 0) 0x7fdad809a690 con 0x7fdae8101060 2024-05-08T23:48:10.494 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:48:10.494 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:47:15.450515+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25489":{"gid":25489,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3829900343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3829900343},{"type":"v1","addr":"172.21.15.204:6837","nonce":3829900343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:48:10.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 -- 172.21.15.12:0/1664675010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdad00a1290 msgr2=0x7fdad00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdad00a1290 0x7fdad00a3750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdae8198900 tx=0x7fdadc009260 comp rx=0 tx=0).stop 2024-05-08T23:48:10.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 -- 172.21.15.12:0/1664675010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdae8101060 msgr2=0x7fdae81973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdae8101060 0x7fdae81973b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdad8005950 tx=0x7fdad800fa00 comp rx=0 tx=0).stop 2024-05-08T23:48:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 -- 172.21.15.12:0/1664675010 shutdown_connections 2024-05-08T23:48:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdad00a1290 0x7fdad00a3750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdae819bc90 0x7fdae819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdae8101060 0x7fdae81973b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 --2- 172.21.15.12:0/1664675010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdae8106180 0x7fdae81978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 -- 172.21.15.12:0/1664675010 >> 172.21.15.12:0/1664675010 conn(0x7fdae80fc920 msgr2=0x7fdae80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 -- 172.21.15.12:0/1664675010 shutdown_connections 2024-05-08T23:48:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.501+0000 7fdaecc45700 1 -- 172.21.15.12:0/1664675010 wait complete. 2024-05-08T23:48:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2024-05-08T23:48:10.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:48:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.656+0000 7f2d363aa700 1 -- 172.21.15.12:0/4246170896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d300682e0 msgr2=0x7f2d300686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.656+0000 7f2d363aa700 1 --2- 172.21.15.12:0/4246170896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d300682e0 0x7f2d300686c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009990 tx=0x7f2d20009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.656+0000 7f2d363aa700 1 -- 172.21.15.12:0/4246170896 shutdown_connections 2024-05-08T23:48:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.656+0000 7f2d363aa700 1 --2- 172.21.15.12:0/4246170896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d30106290 0x7f2d3010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.656+0000 7f2d363aa700 1 --2- 172.21.15.12:0/4246170896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d300682e0 0x7f2d300686c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.656+0000 7f2d363aa700 1 --2- 172.21.15.12:0/4246170896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d30068c90 0x7f2d30105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.656+0000 7f2d363aa700 1 -- 172.21.15.12:0/4246170896 >> 172.21.15.12:0/4246170896 conn(0x7f2d30076f40 msgr2=0x7f2d30077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.656+0000 7f2d363aa700 1 -- 172.21.15.12:0/4246170896 shutdown_connections 2024-05-08T23:48:10.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.656+0000 7f2d363aa700 1 -- 172.21.15.12:0/4246170896 wait complete. 2024-05-08T23:48:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.657+0000 7f2d363aa700 1 Processor -- start 2024-05-08T23:48:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.657+0000 7f2d363aa700 1 -- start start 2024-05-08T23:48:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d363aa700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300682e0 0x7f2d301973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d363aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d30106290 0x7f2d30197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d363aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d3019bca0 0x7f2d3019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d363aa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3019c720 con 0x7f2d30106290 2024-05-08T23:48:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d363aa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3019c890 con 0x7f2d3019bca0 2024-05-08T23:48:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d363aa700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3019ca50 con 0x7f2d300682e0 2024-05-08T23:48:10.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d30106290 0x7f2d30197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300682e0 0x7f2d301973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d34947700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d3019bca0 0x7f2d3019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d30106290 0x7f2d30197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37938/0 (socket says 172.21.15.12:37938) 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300682e0 0x7f2d301973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44512/0 (socket says 172.21.15.12:44512) 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/1329088993 learned_addr learned my addr 172.21.15.12:0/1329088993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/1329088993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300682e0 msgr2=0x7f2d301973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2f7fe700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300682e0 0x7f2d301973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/1329088993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d3019bca0 msgr2=0x7f2d3019c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2f7fe700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d3019bca0 0x7f2d3019c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/1329088993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28009c40 con 0x7f2d30106290 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d34947700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d3019bca0 0x7f2d3019c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2f7fe700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d30106290 0x7f2d30197900 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e970 tx=0x7f2d2400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.658+0000 7f2d2d7fa700 1 -- 172.21.15.12:0/1329088993 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24009790 con 0x7f2d30106290 2024-05-08T23:48:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.659+0000 7f2d363aa700 1 -- 172.21.15.12:0/1329088993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d200095f0 con 0x7f2d30106290 2024-05-08T23:48:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.659+0000 7f2d2d7fa700 1 -- 172.21.15.12:0/1329088993 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d2400c7c0 con 0x7f2d30106290 2024-05-08T23:48:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.659+0000 7f2d2d7fa700 1 -- 172.21.15.12:0/1329088993 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24005460 con 0x7f2d30106290 2024-05-08T23:48:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.659+0000 7f2d363aa700 1 -- 172.21.15.12:0/1329088993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d301a42a0 con 0x7f2d30106290 2024-05-08T23:48:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.661+0000 7f2d2d7fa700 1 -- 172.21.15.12:0/1329088993 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2d24005600 con 0x7f2d30106290 2024-05-08T23:48:10.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.661+0000 7f2d2d7fa700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d180a1290 0x7f2d180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.662+0000 7f2d2ffff700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d180a1290 0x7f2d180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.662+0000 7f2d2d7fa700 1 -- 172.21.15.12:0/1329088993 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2d24014070 con 0x7f2d30106290 2024-05-08T23:48:10.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.662+0000 7f2d2ffff700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d180a1290 0x7f2d180a3750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000b6f0 tx=0x7f2d2000b280 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d42564b80 1 Processor -- start 2024-05-08T23:48:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d42564b80 1 -- start start 2024-05-08T23:48:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc9458e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458e720 0x55fc94588350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc94588cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d42564b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc944df150 con 0x55fc9458de00 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d42564b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc944dcb10 con 0x55fc9458e720 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d42564b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc944d8400 con 0x55fc94588890 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d2ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc9458e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d2f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458e720 0x55fc94588350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d34947700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc94588cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d2ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc9458e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37954/0 (socket says 172.21.15.12:37954) 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d2ffff700 1 -- 172.21.15.12:0/1160538020 learned_addr learned my addr 172.21.15.12:0/1160538020 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d34947700 1 -- 172.21.15.12:0/1160538020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458e720 msgr2=0x55fc94588350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d34947700 1 --2- 172.21.15.12:0/1160538020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458e720 0x55fc94588350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d34947700 1 -- 172.21.15.12:0/1160538020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 msgr2=0x55fc9458e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d34947700 1 --2- 172.21.15.12:0/1160538020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc9458e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.663+0000 7f2d34947700 1 -- 172.21.15.12:0/1160538020 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc943593b0 con 0x55fc94588890 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.664+0000 7f2d34947700 1 --2- 172.21.15.12:0/1160538020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc94588cd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2d280094d0 tx=0x7f2d28009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2eb93eca4cd7c2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.664+0000 7f2d167fc700 1 -- 172.21.15.12:0/1160538020 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d28004db0 con 0x55fc94588890 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.664+0000 7f2d167fc700 1 -- 172.21.15.12:0/1160538020 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d2800f040 con 0x55fc94588890 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.664+0000 7f2d167fc700 1 -- 172.21.15.12:0/1160538020 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d28005490 con 0x55fc94588890 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.665+0000 7f2d42564b80 1 -- 172.21.15.12:0/1160538020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 msgr2=0x55fc94588cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.665+0000 7f2d42564b80 1 --2- 172.21.15.12:0/1160538020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc94588cd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2d280094d0 tx=0x7f2d28009920 comp rx=0 tx=0).stop 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.665+0000 7f2d42564b80 1 -- 172.21.15.12:0/1160538020 shutdown_connections 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.665+0000 7f2d42564b80 1 --2- 172.21.15.12:0/1160538020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc94588cd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.665+0000 7f2d42564b80 1 --2- 172.21.15.12:0/1160538020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc9458e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.665+0000 7f2d42564b80 1 --2- 172.21.15.12:0/1160538020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458e720 0x55fc94588350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.665+0000 7f2d42564b80 1 -- 172.21.15.12:0/1160538020 >> 172.21.15.12:0/1160538020 conn(0x55fc944e6210 msgr2=0x55fc94582df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.665+0000 7f2d42564b80 1 -- 172.21.15.12:0/1160538020 shutdown_connections 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.665+0000 7f2d42564b80 1 -- 172.21.15.12:0/1160538020 wait complete. 2024-05-08T23:48:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.665+0000 7f2d42564b80 1 -- start start 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d42564b80 1 Processor -- start 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc94588890 0x55fc94591b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc9458de00 0x55fc94592070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc945925b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d42564b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc94474560 con 0x55fc9458e720 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d42564b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc94474200 con 0x55fc94588890 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d42564b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9435b6a0 con 0x55fc9458de00 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d42564b80 10 client.0 _ll_register_callbacks cb 0x55fc9447db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d34947700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc945925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d2ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc94588890 0x55fc94591b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d2f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc9458de00 0x55fc94592070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d34947700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc945925b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37968/0 (socket says 172.21.15.12:37968) 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d34947700 1 -- 172.21.15.12:0/4173014721 learned_addr learned my addr 172.21.15.12:0/4173014721 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d2ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc94588890 0x55fc94591b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52576/0 (socket says 172.21.15.12:52576) 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/4173014721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc94588890 msgr2=0x55fc94591b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d2f7fe700 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc94588890 0x55fc94591b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/4173014721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 msgr2=0x55fc945925b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d2f7fe700 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc945925b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.666+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/4173014721 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d280091b0 con 0x55fc9458de00 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d2f7fe700 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc9458de00 0x55fc94592070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d240ca5b0 tx=0x7f2d2409a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154f22e027b90c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/4173014721 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24016330 con 0x55fc9458de00 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d42564b80 1 -- 172.21.15.12:0/4173014721 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc94431270 con 0x55fc9458de00 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/4173014721 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2d240ade40 con 0x55fc9458de00 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/4173014721 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d24016490 con 0x55fc9458de00 2024-05-08T23:48:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d42564b80 10 client.25714 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d42564b80 1 -- 172.21.15.12:0/4173014721 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fc94305b60 con 0x55fc9458de00 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/4173014721 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc94431270 con 0x55fc9458de00 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/4173014721 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2d24099460 con 0x55fc9458de00 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d42564b80 10 client.25714 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d42564b80 10 client.25714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d42564b80 20 client.25714 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d42564b80 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc945a21f0 0x55fc945a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d42564b80 4 client.25714 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.667+0000 7f2d42564b80 1 -- 172.21.15.12:0/4173014721 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc94557810 con 0x55fc945a21f0 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.668+0000 7f2d2ffff700 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc945a21f0 0x55fc945a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.668+0000 7f2d0f7fe700 10 client.25714 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:48:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.668+0000 7f2d2ffff700 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc945a21f0 0x55fc945a4610 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94d0ad08cce003cb server_cookie=16a3f8e473c4c26 in_seq=0 out_seq=0 2024-05-08T23:48:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.669+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/4173014721 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fc94557810 con 0x55fc945a21f0 2024-05-08T23:48:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.669+0000 7f2d0f7fe700 10 client.25714 handle_command_reply: tid=0 2024-05-08T23:48:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.669+0000 7f2d42564b80 1 client.25714 shutdown 2024-05-08T23:48:10.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:10 smithi012 ceph-mon[42503]: pgmap v56: 97 pgs: 97 active+clean; 4.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 280 KiB/s wr, 49 op/s 2024-05-08T23:48:10.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.669+0000 7f2d42564b80 1 -- 172.21.15.12:0/4173014721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc9458de00 msgr2=0x55fc94592070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.669+0000 7f2d42564b80 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc9458de00 0x55fc94592070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d240ca5b0 tx=0x7f2d2409a0b0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.670+0000 7f2d42564b80 1 -- 172.21.15.12:0/4173014721 shutdown_connections 2024-05-08T23:48:10.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.670+0000 7f2d42564b80 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc945925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.670+0000 7f2d42564b80 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc94588890 0x55fc94591b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.670+0000 7f2d42564b80 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc945a21f0 0x55fc945a4610 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.670+0000 7f2d42564b80 1 --2- 172.21.15.12:0/4173014721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc9458de00 0x55fc94592070 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.670+0000 7f2d0f7fe700 0 client.25714 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:48:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.670+0000 7f2d42564b80 1 -- 172.21.15.12:0/4173014721 >> 172.21.15.12:0/4173014721 conn(0x55fc944e6210 msgr2=0x55fc945825a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.670+0000 7f2d42564b80 1 -- 172.21.15.12:0/4173014721 shutdown_connections 2024-05-08T23:48:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.670+0000 7f2d42564b80 1 -- 172.21.15.12:0/4173014721 wait complete. 2024-05-08T23:48:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.670+0000 7f2d42564b80 20 client.25714 trim_cache size 0 max 16384 2024-05-08T23:48:10.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.684+0000 7f2d42564b80 1 Processor -- start 2024-05-08T23:48:10.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.684+0000 7f2d42564b80 1 -- start start 2024-05-08T23:48:10.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.684+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc94588350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.684+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458de00 0x55fc9459dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.684+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc94582d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.684+0000 7f2d42564b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc945838e0 con 0x55fc9458e720 2024-05-08T23:48:10.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.684+0000 7f2d42564b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9457fdc0 con 0x55fc9458de00 2024-05-08T23:48:10.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.684+0000 7f2d42564b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc945a9820 con 0x55fc94588890 2024-05-08T23:48:10.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d34947700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc94582d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d2f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458de00 0x55fc9459dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d34947700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc94582d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37976/0 (socket says 172.21.15.12:37976) 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d2f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458de00 0x55fc9459dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52592/0 (socket says 172.21.15.12:52592) 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d2ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc94588350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d34947700 1 -- 172.21.15.12:0/1066406967 learned_addr learned my addr 172.21.15.12:0/1066406967 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d34947700 1 -- 172.21.15.12:0/1066406967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 msgr2=0x55fc94588350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d34947700 1 --2- 172.21.15.12:0/1066406967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc94588350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d34947700 1 -- 172.21.15.12:0/1066406967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458de00 msgr2=0x55fc9459dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d34947700 1 --2- 172.21.15.12:0/1066406967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458de00 0x55fc9459dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d34947700 1 -- 172.21.15.12:0/1066406967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc9449e560 con 0x55fc9458e720 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d2f7fe700 1 --2- 172.21.15.12:0/1066406967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458de00 0x55fc9459dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d2ffff700 1 --2- 172.21.15.12:0/1066406967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc94588350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d34947700 1 --2- 172.21.15.12:0/1066406967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc94582d20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2d28006030 tx=0x7f2d28017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595c94223491dbbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d16ffd700 1 -- 172.21.15.12:0/1066406967 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d28022040 con 0x55fc9458e720 2024-05-08T23:48:10.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d16ffd700 1 -- 172.21.15.12:0/1066406967 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d28003c00 con 0x55fc9458e720 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.685+0000 7f2d16ffd700 1 -- 172.21.15.12:0/1066406967 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800e750 con 0x55fc9458e720 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.686+0000 7f2d42564b80 1 -- 172.21.15.12:0/1066406967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 msgr2=0x55fc94582d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.686+0000 7f2d42564b80 1 --2- 172.21.15.12:0/1066406967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc94582d20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2d28006030 tx=0x7f2d28017040 comp rx=0 tx=0).stop 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.686+0000 7f2d42564b80 1 -- 172.21.15.12:0/1066406967 shutdown_connections 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.686+0000 7f2d42564b80 1 --2- 172.21.15.12:0/1066406967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458e720 0x55fc94582d20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.686+0000 7f2d42564b80 1 --2- 172.21.15.12:0/1066406967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc94588350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.686+0000 7f2d42564b80 1 --2- 172.21.15.12:0/1066406967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458de00 0x55fc9459dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.686+0000 7f2d42564b80 1 -- 172.21.15.12:0/1066406967 >> 172.21.15.12:0/1066406967 conn(0x55fc944e6210 msgr2=0x55fc9457f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.686+0000 7f2d42564b80 1 -- 172.21.15.12:0/1066406967 shutdown_connections 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.686+0000 7f2d42564b80 1 -- 172.21.15.12:0/1066406967 wait complete. 2024-05-08T23:48:10.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d42564b80 1 -- start start 2024-05-08T23:48:10.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d42564b80 1 Processor -- start 2024-05-08T23:48:10.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc945f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc945f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d42564b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458e720 0x55fc945f1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d42564b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc945fbb50 con 0x55fc9458de00 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d42564b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc945fbcc0 con 0x55fc9458e720 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d42564b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc945fbe80 con 0x55fc94588890 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d42564b80 10 client.0 _ll_register_callbacks cb 0x55fc9447db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d2f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc945f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d2f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc945f0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37984/0 (socket says 172.21.15.12:37984) 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/659146461 learned_addr learned my addr 172.21.15.12:0/659146461 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d34947700 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458e720 0x55fc945f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.687+0000 7f2d2ffff700 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc945f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/659146461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 msgr2=0x55fc945f0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d2f7fe700 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc945f0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/659146461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458e720 msgr2=0x55fc945f1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d2f7fe700 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458e720 0x55fc945f1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d2f7fe700 1 -- 172.21.15.12:0/659146461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3010e650 con 0x55fc9458de00 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d2f7fe700 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc945f0fd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c5b0 tx=0x7f2d2400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a90fa051e11fe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/659146461 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24096680 con 0x55fc9458de00 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/659146461 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2d240ade40 con 0x55fc9458de00 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d42564b80 1 -- 172.21.15.12:0/659146461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d2800f040 con 0x55fc9458de00 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/659146461 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d240967e0 con 0x55fc9458de00 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d42564b80 10 client.15867 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d42564b80 1 -- 172.21.15.12:0/659146461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fc94305b60 con 0x55fc9458de00 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.688+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/659146461 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d2800f040 con 0x55fc9458de00 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/659146461 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2d24094c10 con 0x55fc9458de00 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d42564b80 10 client.15867 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d42564b80 10 client.15867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d42564b80 20 client.15867 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d42564b80 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc945fd150 0x55fc945ff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d42564b80 4 client.15867 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:48:10.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d42564b80 1 -- 172.21.15.12:0/659146461 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fc944d0ec0 con 0x55fc945fd150 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d2ffff700 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc945fd150 0x55fc945ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d2ffff700 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc945fd150 0x55fc945ff540 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d47d084c6766f610 server_cookie=f5b2d53e08d42c15 in_seq=0 out_seq=0 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d0f7fe700 10 client.15867 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d0f7fe700 1 -- 172.21.15.12:0/659146461 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fc944d0ec0 con 0x55fc945fd150 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d0f7fe700 10 client.15867 handle_command_reply: tid=0 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.689+0000 7f2d42564b80 1 client.15867 shutdown 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 1 -- 172.21.15.12:0/659146461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 msgr2=0x55fc945f0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc945f0fd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c5b0 tx=0x7f2d2400cf00 comp rx=0 tx=0).stop 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 1 -- 172.21.15.12:0/659146461 shutdown_connections 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc9458e720 0x55fc945f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fc945fd150 0x55fc945ff540 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc94588890 0x55fc945f0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 1 --2- 172.21.15.12:0/659146461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc9458de00 0x55fc945f0fd0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d0f7fe700 0 client.15867 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:48:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 1 -- 172.21.15.12:0/659146461 >> 172.21.15.12:0/659146461 conn(0x55fc944e6210 msgr2=0x55fc94592410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 1 -- 172.21.15.12:0/659146461 shutdown_connections 2024-05-08T23:48:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 1 -- 172.21.15.12:0/659146461 wait complete. 2024-05-08T23:48:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d42564b80 20 client.15867 trim_cache size 0 max 16384 2024-05-08T23:48:10.692 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:48:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d363aa700 1 -- 172.21.15.12:0/1329088993 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d180a1290 msgr2=0x7f2d180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d363aa700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d180a1290 0x7f2d180a3750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000b6f0 tx=0x7f2d2000b280 comp rx=0 tx=0).stop 2024-05-08T23:48:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d363aa700 1 -- 172.21.15.12:0/1329088993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d30106290 msgr2=0x7f2d30197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d363aa700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d30106290 0x7f2d30197900 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e970 tx=0x7f2d2400ec80 comp rx=0 tx=0).stop 2024-05-08T23:48:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d363aa700 1 -- 172.21.15.12:0/1329088993 shutdown_connections 2024-05-08T23:48:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d363aa700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d180a1290 0x7f2d180a3750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d363aa700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d3019bca0 0x7f2d3019c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d363aa700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300682e0 0x7f2d301973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d363aa700 1 --2- 172.21.15.12:0/1329088993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d30106290 0x7f2d30197900 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.690+0000 7f2d363aa700 1 -- 172.21.15.12:0/1329088993 >> 172.21.15.12:0/1329088993 conn(0x7f2d30076f40 msgr2=0x7f2d300fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.691+0000 7f2d363aa700 1 -- 172.21.15.12:0/1329088993 shutdown_connections 2024-05-08T23:48:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.691+0000 7f2d363aa700 1 -- 172.21.15.12:0/1329088993 wait complete. 2024-05-08T23:48:10.700 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:48:10.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:10 smithi204 ceph-mon[47373]: pgmap v56: 97 pgs: 97 active+clean; 4.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 280 KiB/s wr, 49 op/s 2024-05-08T23:48:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:10 smithi160 ceph-mon[46152]: pgmap v56: 97 pgs: 97 active+clean; 4.0 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 280 KiB/s wr, 49 op/s 2024-05-08T23:48:10.845 INFO:tasks.workunit.client.0.smithi012.stdout: 1 18723 19.57 MB/sec warmup 6 sec latency 16.633 ms 2024-05-08T23:48:10.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.852+0000 7f59e7cd8700 1 -- 172.21.15.12:0/3438833640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e0100a10 msgr2=0x7f59e0104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.852+0000 7f59e7cd8700 1 --2- 172.21.15.12:0/3438833640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e0100a10 0x7f59e0104a30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009990 tx=0x7f59d0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.852+0000 7f59e7cd8700 1 -- 172.21.15.12:0/3438833640 shutdown_connections 2024-05-08T23:48:10.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.852+0000 7f59e7cd8700 1 --2- 172.21.15.12:0/3438833640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e0105160 0x7f59e010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.852+0000 7f59e7cd8700 1 --2- 172.21.15.12:0/3438833640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e0100060 0x7f59e0100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.852+0000 7f59e7cd8700 1 --2- 172.21.15.12:0/3438833640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e0100a10 0x7f59e0104a30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.852+0000 7f59e7cd8700 1 -- 172.21.15.12:0/3438833640 >> 172.21.15.12:0/3438833640 conn(0x7f59e0076f40 msgr2=0x7f59e0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.852+0000 7f59e7cd8700 1 -- 172.21.15.12:0/3438833640 shutdown_connections 2024-05-08T23:48:10.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.852+0000 7f59e7cd8700 1 -- 172.21.15.12:0/3438833640 wait complete. 2024-05-08T23:48:10.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e7cd8700 1 Processor -- start 2024-05-08T23:48:10.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e7cd8700 1 -- start start 2024-05-08T23:48:10.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e7cd8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e0100060 0x7f59e010d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e7cd8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e0105160 0x7f59e010db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e7cd8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e010e0a0 0x7f59e01a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e7cd8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01a5b10 con 0x7f59e0100060 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e7cd8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01a5c80 con 0x7f59e0105160 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e7cd8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01a5e40 con 0x7f59e010e0a0 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e5a74700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e0100060 0x7f59e010d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e5a74700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e0100060 0x7f59e010d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37994/0 (socket says 172.21.15.12:37994) 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e5a74700 1 -- 172.21.15.12:0/4185111465 learned_addr learned my addr 172.21.15.12:0/4185111465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e5273700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e0105160 0x7f59e010db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.853+0000 7f59e6275700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e010e0a0 0x7f59e01a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59e5a74700 1 -- 172.21.15.12:0/4185111465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e010e0a0 msgr2=0x7f59e01a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59e5a74700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e010e0a0 0x7f59e01a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59e5a74700 1 -- 172.21.15.12:0/4185111465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e0105160 msgr2=0x7f59e010db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59e5a74700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e0105160 0x7f59e010db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59e5a74700 1 -- 172.21.15.12:0/4185111465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc009c40 con 0x7f59e0100060 2024-05-08T23:48:10.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59e5273700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e0105160 0x7f59e010db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:10.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59e5a74700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e0100060 0x7f59e010d620 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c370 tx=0x7f59dc00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59d6ffd700 1 -- 172.21.15.12:0/4185111465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc017070 con 0x7f59e0100060 2024-05-08T23:48:10.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59e7cd8700 1 -- 172.21.15.12:0/4185111465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d00095f0 con 0x7f59e0100060 2024-05-08T23:48:10.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59d6ffd700 1 -- 172.21.15.12:0/4185111465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59dc00f040 con 0x7f59e0100060 2024-05-08T23:48:10.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59d6ffd700 1 -- 172.21.15.12:0/4185111465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc00e050 con 0x7f59e0100060 2024-05-08T23:48:10.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.854+0000 7f59e7cd8700 1 -- 172.21.15.12:0/4185111465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e01a6490 con 0x7f59e0100060 2024-05-08T23:48:10.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.857+0000 7f59d6ffd700 1 -- 172.21.15.12:0/4185111465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f59dc004440 con 0x7f59e0100060 2024-05-08T23:48:10.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.857+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59cc0a1290 0x7f59cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.857+0000 7f59d6ffd700 1 -- 172.21.15.12:0/4185111465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f59dc0cc630 con 0x7f59e0100060 2024-05-08T23:48:10.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.857+0000 7f59e5273700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59cc0a1290 0x7f59cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.858+0000 7f59f3e92b80 1 Processor -- start 2024-05-08T23:48:10.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.858+0000 7f59e5273700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59cc0a1290 0x7f59cc0a3750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59e010ed50 tx=0x7f59d0005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.858+0000 7f59f3e92b80 1 -- start start 2024-05-08T23:48:10.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41efe00 0x5649d41f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41f0720 0x5649d41ea350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41eacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d4141150 con 0x5649d41ea890 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d413eb10 con 0x5649d41f0720 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d413a400 con 0x5649d41efe00 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e5a74700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41efe00 0x5649d41f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e6275700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41eacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e5273700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41f0720 0x5649d41ea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e5a74700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41efe00 0x5649d41f01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44568/0 (socket says 172.21.15.12:44568) 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e5a74700 1 -- 172.21.15.12:0/3277972703 learned_addr learned my addr 172.21.15.12:0/3277972703 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e6275700 1 --2- 172.21.15.12:0/3277972703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41eacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38004/0 (socket says 172.21.15.12:38004) 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e5a74700 1 -- 172.21.15.12:0/3277972703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41f0720 msgr2=0x5649d41ea350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e5a74700 1 --2- 172.21.15.12:0/3277972703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41f0720 0x5649d41ea350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e5a74700 1 -- 172.21.15.12:0/3277972703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 msgr2=0x5649d41eacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e5a74700 1 --2- 172.21.15.12:0/3277972703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41eacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e5a74700 1 -- 172.21.15.12:0/3277972703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d3fbb3b0 con 0x5649d41efe00 2024-05-08T23:48:10.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.859+0000 7f59e5a74700 1 --2- 172.21.15.12:0/3277972703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41efe00 0x5649d41f01e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0961f0 tx=0x7f59dc0995d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b7f11f138ea84a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.860+0000 7f59cbfff700 1 -- 172.21.15.12:0/3277972703 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0192c0 con 0x5649d41efe00 2024-05-08T23:48:10.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.860+0000 7f59cbfff700 1 -- 172.21.15.12:0/3277972703 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59dc099e80 con 0x5649d41efe00 2024-05-08T23:48:10.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.860+0000 7f59cbfff700 1 -- 172.21.15.12:0/3277972703 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0046f0 con 0x5649d41efe00 2024-05-08T23:48:10.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.860+0000 7f59f3e92b80 1 -- 172.21.15.12:0/3277972703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41efe00 msgr2=0x5649d41f01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.860+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/3277972703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41efe00 0x5649d41f01e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0961f0 tx=0x7f59dc0995d0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.860+0000 7f59f3e92b80 1 -- 172.21.15.12:0/3277972703 shutdown_connections 2024-05-08T23:48:10.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.860+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/3277972703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41eacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.860+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/3277972703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41efe00 0x5649d41f01e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.860+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/3277972703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41f0720 0x5649d41ea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.860+0000 7f59f3e92b80 1 -- 172.21.15.12:0/3277972703 >> 172.21.15.12:0/3277972703 conn(0x5649d4148210 msgr2=0x5649d41e4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.861+0000 7f59f3e92b80 1 -- 172.21.15.12:0/3277972703 shutdown_connections 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.861+0000 7f59f3e92b80 1 -- 172.21.15.12:0/3277972703 wait complete. 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.861+0000 7f59f3e92b80 1 -- start start 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.861+0000 7f59f3e92b80 1 Processor -- start 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.861+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.861+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41f4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.861+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 0x5649d41f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d40d6560 con 0x5649d41ea890 2024-05-08T23:48:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d40d6200 con 0x5649d41efe00 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d3fbd6a0 con 0x5649d41f0720 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59f3e92b80 10 client.0 _ll_register_callbacks cb 0x5649d40dfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e5a74700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e5a74700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38008/0 (socket says 172.21.15.12:38008) 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e5a74700 1 -- 172.21.15.12:0/2903390134 learned_addr learned my addr 172.21.15.12:0/2903390134 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e6275700 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 0x5649d41f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e5273700 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e5a74700 1 -- 172.21.15.12:0/2903390134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 msgr2=0x5649d41f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e5a74700 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 0x5649d41f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e5a74700 1 -- 172.21.15.12:0/2903390134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 msgr2=0x5649d41f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e5a74700 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e5a74700 1 -- 172.21.15.12:0/2903390134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e01a6490 con 0x5649d41ea890 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e6275700 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 0x5649d41f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59e5a74700 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41f3bf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c9d0 tx=0x7f59dc0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f68f6c5cab5589 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59c97fa700 1 -- 172.21.15.12:0/2903390134 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0b2170 con 0x5649d41ea890 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.862+0000 7f59c97fa700 1 -- 172.21.15.12:0/2903390134 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f59dc09d5b0 con 0x5649d41ea890 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2903390134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649d4093270 con 0x5649d41ea890 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59c97fa700 1 -- 172.21.15.12:0/2903390134 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59dc0a1c00 con 0x5649d41ea890 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59f3e92b80 10 client.15879 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2903390134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5649d3f67b60 con 0x5649d41ea890 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59c97fa700 1 -- 172.21.15.12:0/2903390134 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649d4093270 con 0x5649d41ea890 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59c97fa700 1 -- 172.21.15.12:0/2903390134 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f59dc09d120 con 0x5649d41ea890 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59f3e92b80 10 client.15879 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59f3e92b80 10 client.15879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59f3e92b80 20 client.15879 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5649d4203d80 0x5649d4206170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59f3e92b80 4 client.15879 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2903390134 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649d41b9810 con 0x5649d4203d80 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.863+0000 7f59e5273700 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5649d4203d80 0x5649d4206170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.864+0000 7f59c97fa700 10 client.15879 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.864+0000 7f59e5273700 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5649d4203d80 0x5649d4206170 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b562b307f0279f8f server_cookie=2fa161aad9a17096 in_seq=0 out_seq=0 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.864+0000 7f59c97fa700 1 -- 172.21.15.12:0/2903390134 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5649d41b9810 con 0x5649d4203d80 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.864+0000 7f59c97fa700 10 client.15879 handle_command_reply: tid=0 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.864+0000 7f59f3e92b80 1 client.15879 shutdown 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2903390134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 msgr2=0x5649d41f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41f3bf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c9d0 tx=0x7f59dc0192c0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2903390134 shutdown_connections 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5649d4203d80 0x5649d4206170 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 0x5649d41f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41f3bf0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2903390134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59c97fa700 0 client.15879 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:48:10.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2903390134 >> 172.21.15.12:0/2903390134 conn(0x5649d4148210 msgr2=0x5649d41e45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2903390134 shutdown_connections 2024-05-08T23:48:10.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2903390134 wait complete. 2024-05-08T23:48:10.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.865+0000 7f59f3e92b80 20 client.15879 trim_cache size 0 max 16384 2024-05-08T23:48:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.879+0000 7f59f3e92b80 1 Processor -- start 2024-05-08T23:48:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.879+0000 7f59f3e92b80 1 -- start start 2024-05-08T23:48:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41ea350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 0x5649d41fb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d41f0430 con 0x5649d41ea890 2024-05-08T23:48:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d41f1b10 con 0x5649d41efe00 2024-05-08T23:48:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d41ee5f0 con 0x5649d41f0720 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e5a74700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41ea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e5a74700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41ea350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38010/0 (socket says 172.21.15.12:38010) 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e5a74700 1 -- 172.21.15.12:0/3206741625 learned_addr learned my addr 172.21.15.12:0/3206741625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e5273700 1 --2- 172.21.15.12:0/3206741625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e6275700 1 --2- 172.21.15.12:0/3206741625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 0x5649d41fb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e5a74700 1 -- 172.21.15.12:0/3206741625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 msgr2=0x5649d41fb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e5a74700 1 --2- 172.21.15.12:0/3206741625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 0x5649d41fb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e5a74700 1 -- 172.21.15.12:0/3206741625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 msgr2=0x5649d41e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e5a74700 1 --2- 172.21.15.12:0/3206741625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41e4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e5a74700 1 -- 172.21.15.12:0/3206741625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d4100560 con 0x5649d41ea890 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59e5a74700 1 --2- 172.21.15.12:0/3206741625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41ea350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f59dc095c80 tx=0x7f59dc00e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434d7a589a01f2b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59d4ff9700 1 -- 172.21.15.12:0/3206741625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0b2170 con 0x5649d41ea890 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59d4ff9700 1 -- 172.21.15.12:0/3206741625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59dc0a1c00 con 0x5649d41ea890 2024-05-08T23:48:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.880+0000 7f59d4ff9700 1 -- 172.21.15.12:0/3206741625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc004e30 con 0x5649d41ea890 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.881+0000 7f59f3e92b80 1 -- 172.21.15.12:0/3206741625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 msgr2=0x5649d41ea350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.881+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/3206741625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41ea350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f59dc095c80 tx=0x7f59dc00e1d0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.881+0000 7f59f3e92b80 1 -- 172.21.15.12:0/3206741625 shutdown_connections 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.881+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/3206741625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41f0720 0x5649d41fb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.881+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/3206741625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41ea890 0x5649d41ea350 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.881+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/3206741625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41e4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.881+0000 7f59f3e92b80 1 -- 172.21.15.12:0/3206741625 >> 172.21.15.12:0/3206741625 conn(0x5649d4148210 msgr2=0x5649d41f44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.881+0000 7f59f3e92b80 1 -- 172.21.15.12:0/3206741625 shutdown_connections 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.881+0000 7f59f3e92b80 1 -- 172.21.15.12:0/3206741625 wait complete. 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.881+0000 7f59f3e92b80 1 -- start start 2024-05-08T23:48:10.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59f3e92b80 1 Processor -- start 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41ea890 0x5649d41f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59f3e92b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41f0720 0x5649d41f9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d41fba50 con 0x5649d41f0720 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d42591f0 con 0x5649d41efe00 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59f3e92b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d4259330 con 0x5649d41ea890 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59f3e92b80 10 client.0 _ll_register_callbacks cb 0x5649d40dfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59e6275700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41f0720 0x5649d41f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59e6275700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41f0720 0x5649d41f9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38018/0 (socket says 172.21.15.12:38018) 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59e5273700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59e6275700 1 -- 172.21.15.12:0/2249614984 learned_addr learned my addr 172.21.15.12:0/2249614984 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:10.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.882+0000 7f59e5a74700 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41ea890 0x5649d41f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59e5273700 1 -- 172.21.15.12:0/2249614984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41ea890 msgr2=0x5649d41f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59e5273700 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41ea890 0x5649d41f8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59e5273700 1 -- 172.21.15.12:0/2249614984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41f0720 msgr2=0x5649d41f9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59e5273700 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41f0720 0x5649d41f9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59e5273700 1 -- 172.21.15.12:0/2249614984 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649d3f67b60 con 0x5649d41efe00 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59e5273700 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41f90d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f59d00059d0 tx=0x7f59d0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b936731866ad8ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59c97fa700 1 -- 172.21.15.12:0/2249614984 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d002d040 con 0x5649d41efe00 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59c97fa700 1 -- 172.21.15.12:0/2249614984 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f59d002b050 con 0x5649d41efe00 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2249614984 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59dc099e80 con 0x5649d41efe00 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59c97fa700 1 -- 172.21.15.12:0/2249614984 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59d0004dc0 con 0x5649d41efe00 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59f3e92b80 10 client.25730 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2249614984 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5649d3fbfcd0 con 0x5649d41efe00 2024-05-08T23:48:10.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.883+0000 7f59c97fa700 1 -- 172.21.15.12:0/2249614984 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59dc099e80 con 0x5649d41efe00 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59c97fa700 1 -- 172.21.15.12:0/2249614984 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f59d000f460 con 0x5649d41efe00 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59f3e92b80 10 client.25730 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59f3e92b80 10 client.25730 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59f3e92b80 20 client.25730 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5649d425aaa0 0x5649d425ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59f3e92b80 4 client.25730 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2249614984 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5649d4132ec0 con 0x5649d425aaa0 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59e5a74700 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5649d425aaa0 0x5649d425ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59e5a74700 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5649d425aaa0 0x5649d425ce90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e194351b8c0825f9 server_cookie=435cbd166d92360d in_seq=0 out_seq=0 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59c97fa700 10 client.25730 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59c97fa700 1 -- 172.21.15.12:0/2249614984 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5649d4132ec0 con 0x5649d425aaa0 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59c97fa700 10 client.25730 handle_command_reply: tid=0 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.884+0000 7f59f3e92b80 1 client.25730 shutdown 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2249614984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 msgr2=0x5649d41f90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41f90d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f59d00059d0 tx=0x7f59d0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2249614984 shutdown_connections 2024-05-08T23:48:10.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5649d425aaa0 0x5649d425ce90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649d41f0720 0x5649d41f9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649d41ea890 0x5649d41f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 1 --2- 172.21.15.12:0/2249614984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649d41efe00 0x5649d41f90d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59c97fa700 0 client.25730 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:48:10.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2249614984 >> 172.21.15.12:0/2249614984 conn(0x5649d4148210 msgr2=0x5649d41f44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2249614984 shutdown_connections 2024-05-08T23:48:10.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 1 -- 172.21.15.12:0/2249614984 wait complete. 2024-05-08T23:48:10.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59f3e92b80 20 client.25730 trim_cache size 0 max 16384 2024-05-08T23:48:10.886 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:48:10.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59e7cd8700 1 -- 172.21.15.12:0/4185111465 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59cc0a1290 msgr2=0x7f59cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59e7cd8700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59cc0a1290 0x7f59cc0a3750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59e010ed50 tx=0x7f59d0005960 comp rx=0 tx=0).stop 2024-05-08T23:48:10.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59e7cd8700 1 -- 172.21.15.12:0/4185111465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e0100060 msgr2=0x7f59e010d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:10.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.885+0000 7f59e7cd8700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e0100060 0x7f59e010d620 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c370 tx=0x7f59dc00c680 comp rx=0 tx=0).stop 2024-05-08T23:48:10.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.886+0000 7f59e7cd8700 1 -- 172.21.15.12:0/4185111465 shutdown_connections 2024-05-08T23:48:10.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.886+0000 7f59e7cd8700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59cc0a1290 0x7f59cc0a3750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.886+0000 7f59e7cd8700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e010e0a0 0x7f59e01a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.886+0000 7f59e7cd8700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e0100060 0x7f59e010d620 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.886+0000 7f59e7cd8700 1 --2- 172.21.15.12:0/4185111465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e0105160 0x7f59e010db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:10.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.886+0000 7f59e7cd8700 1 -- 172.21.15.12:0/4185111465 >> 172.21.15.12:0/4185111465 conn(0x7f59e0076f40 msgr2=0x7f59e0103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:10.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.886+0000 7f59e7cd8700 1 -- 172.21.15.12:0/4185111465 shutdown_connections 2024-05-08T23:48:10.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:10.886+0000 7f59e7cd8700 1 -- 172.21.15.12:0/4185111465 wait complete. 2024-05-08T23:48:10.895 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:48:11.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.044+0000 7f5dcb032700 1 -- 172.21.15.12:0/741911921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc4100a10 msgr2=0x7f5dc4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:11.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.044+0000 7f5dcb032700 1 --2- 172.21.15.12:0/741911921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc4100a10 0x7f5dc4104a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5db00098f0 tx=0x7f5db0009c00 comp rx=0 tx=0).stop 2024-05-08T23:48:11.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.044+0000 7f5dcb032700 1 -- 172.21.15.12:0/741911921 shutdown_connections 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.044+0000 7f5dcb032700 1 --2- 172.21.15.12:0/741911921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dc4105160 0x7f5dc410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.044+0000 7f5dcb032700 1 --2- 172.21.15.12:0/741911921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc4100060 0x7f5dc4100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.044+0000 7f5dcb032700 1 --2- 172.21.15.12:0/741911921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc4100a10 0x7f5dc4104a30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.044+0000 7f5dcb032700 1 -- 172.21.15.12:0/741911921 >> 172.21.15.12:0/741911921 conn(0x7f5dc4076f40 msgr2=0x7f5dc4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.044+0000 7f5dcb032700 1 -- 172.21.15.12:0/741911921 shutdown_connections 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.044+0000 7f5dcb032700 1 -- 172.21.15.12:0/741911921 wait complete. 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.044+0000 7f5dcb032700 1 Processor -- start 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dcb032700 1 -- start start 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dcb032700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dc4100060 0x7f5dc41973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dcb032700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc4105160 0x7f5dc41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dcb032700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc419bc90 0x7f5dc419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dcb032700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc419c710 con 0x7f5dc4100060 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dcb032700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc419c880 con 0x7f5dc419bc90 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dcb032700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc419ca40 con 0x7f5dc4105160 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc8dce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dc4100060 0x7f5dc41973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc8dce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dc4100060 0x7f5dc41973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38030/0 (socket says 172.21.15.12:38030) 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc8dce700 1 -- 172.21.15.12:0/1184311082 learned_addr learned my addr 172.21.15.12:0/1184311082 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc419bc90 0x7f5dc419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc3fff700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc4105160 0x7f5dc41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc8dce700 1 -- 172.21.15.12:0/1184311082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc4105160 msgr2=0x7f5dc41978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc8dce700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc4105160 0x7f5dc41978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc8dce700 1 -- 172.21.15.12:0/1184311082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc419bc90 msgr2=0x7f5dc419c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc8dce700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc419bc90 0x7f5dc419c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc8dce700 1 -- 172.21.15.12:0/1184311082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc009d00 con 0x7f5dc4100060 2024-05-08T23:48:11.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.045+0000 7f5dc8dce700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dc4100060 0x7f5dc41973b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5db800e9a0 tx=0x7f5db800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:11.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.046+0000 7f5dc1ffb700 1 -- 172.21.15.12:0/1184311082 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8009790 con 0x7f5dc4100060 2024-05-08T23:48:11.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.046+0000 7f5dc1ffb700 1 -- 172.21.15.12:0/1184311082 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db8010430 con 0x7f5dc4100060 2024-05-08T23:48:11.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.046+0000 7f5dcb032700 1 -- 172.21.15.12:0/1184311082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db0009520 con 0x7f5dc4100060 2024-05-08T23:48:11.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.046+0000 7f5dc1ffb700 1 -- 172.21.15.12:0/1184311082 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8018680 con 0x7f5dc4100060 2024-05-08T23:48:11.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.046+0000 7f5dcb032700 1 -- 172.21.15.12:0/1184311082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc41a4230 con 0x7f5dc4100060 2024-05-08T23:48:11.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.050+0000 7f5dd71ecb80 1 Processor -- start 2024-05-08T23:48:11.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.050+0000 7f5dd71ecb80 1 -- start start 2024-05-08T23:48:11.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.050+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.050+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 0x5596866eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.050+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866ebca0 0x5596866ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.050+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968663e150 con 0x5596866f11f0 2024-05-08T23:48:11.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.050+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968663bb10 con 0x5596866ebca0 2024-05-08T23:48:11.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.050+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559686637400 con 0x5596866f1b10 2024-05-08T23:48:11.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.050+0000 7f5dc95cf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.050+0000 7f5dc95cf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38038/0 (socket says 172.21.15.12:38038) 2024-05-08T23:48:11.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3894963530 learned_addr learned my addr 172.21.15.12:0/3894963530 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:11.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc1ffb700 1 -- 172.21.15.12:0/1184311082 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5db80105a0 con 0x7f5dc4100060 2024-05-08T23:48:11.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc3fff700 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866ebca0 0x5596866ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc8dce700 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 0x5596866eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3894963530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 msgr2=0x5596866eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:11.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 0x5596866eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3894963530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866ebca0 msgr2=0x5596866ec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:11.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866ebca0 0x5596866ec100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3894963530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596864b83b0 con 0x5596866f11f0 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc3fff700 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866ebca0 0x5596866ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc8dce700 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 0x5596866eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f15d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009530 tx=0x7f5dbc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9e547700ffba91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5daeffd700 1 -- 172.21.15.12:0/3894963530 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00e070 con 0x5596866f11f0 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5daeffd700 1 -- 172.21.15.12:0/3894963530 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dbc00f040 con 0x5596866f11f0 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5daeffd700 1 -- 172.21.15.12:0/3894963530 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc017680 con 0x5596866f11f0 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.051+0000 7f5dc1ffb700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5db40a1290 0x7f5db40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dc8dce700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5db40a1290 0x7f5db40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dc1ffb700 1 -- 172.21.15.12:0/1184311082 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5db8014070 con 0x7f5dc4100060 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3894963530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 msgr2=0x5596866f15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f15d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009530 tx=0x7f5dbc0099e0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dc8dce700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5db40a1290 0x7f5db40a3750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5db80ca630 tx=0x7f5db8010c80 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:11.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3894963530 shutdown_connections 2024-05-08T23:48:11.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866ebca0 0x5596866ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f15d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3894963530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 0x5596866eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3894963530 >> 172.21.15.12:0/3894963530 conn(0x559686645210 msgr2=0x5596866e1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:11.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3894963530 shutdown_connections 2024-05-08T23:48:11.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.052+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3894963530 wait complete. 2024-05-08T23:48:11.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dd71ecb80 1 -- start start 2024-05-08T23:48:11.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dd71ecb80 1 Processor -- start 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 0x5596866f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 0x5596866f5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596865d3560 con 0x5596866f11f0 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596865d3200 con 0x5596866f1b10 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596864ba6a0 con 0x5596866ebca0 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dd71ecb80 10 client.0 _ll_register_callbacks cb 0x5596865dcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dc3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dc95cf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 0x5596866f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dc3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38046/0 (socket says 172.21.15.12:38046) 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.053+0000 7f5dc3fff700 1 -- 172.21.15.12:0/3772566136 learned_addr learned my addr 172.21.15.12:0/3772566136 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:11.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 0x5596866f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3772566136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 msgr2=0x5596866f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 0x5596866f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3772566136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 msgr2=0x5596866f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3772566136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc410d550 con 0x5596866f1b10 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dc3fff700 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 0x5596866f5930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009530 tx=0x7f5dbc004890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6757958730d70045 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5da3fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5da3fff700 1 -- 172.21.15.12:0/3772566136 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00e050 con 0x5596866f1b10 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5da3fff700 1 -- 172.21.15.12:0/3772566136 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5dbc025220 con 0x5596866f1b10 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3772566136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559686590270 con 0x5596866f1b10 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5da3fff700 1 -- 172.21.15.12:0/3772566136 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dbc003c10 con 0x5596866f1b10 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dd71ecb80 10 client.25742 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:11.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.054+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3772566136 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f5dbc0091e0 con 0x5596866f1b10 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5da3fff700 1 -- 172.21.15.12:0/3772566136 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559686590270 con 0x5596866f1b10 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5da3fff700 1 -- 172.21.15.12:0/3772566136 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5dbc01e070 con 0x5596866f1b10 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5dd71ecb80 10 client.25742 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5dd71ecb80 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5dd71ecb80 20 client.25742 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559686705500 0x5596867078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5dd71ecb80 4 client.25742 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3772566136 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596866b6810 con 0x559686705500 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5dc3fff700 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559686705500 0x5596867078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5dc3fff700 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559686705500 0x5596867078f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996ae8770dde804f server_cookie=4939450378e00f38 in_seq=0 out_seq=0 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.055+0000 7f5da3fff700 10 client.25742 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5da3fff700 1 -- 172.21.15.12:0/3772566136 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596866b6810 con 0x559686705500 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5da3fff700 10 client.25742 handle_command_reply: tid=0 2024-05-08T23:48:11.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5dd71ecb80 1 client.25742 shutdown 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3772566136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 msgr2=0x5596866f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 0x5596866f5930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009530 tx=0x7f5dbc004890 comp rx=0 tx=0).stop 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3772566136 shutdown_connections 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559686705500 0x5596867078f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 0x5596866f5930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5da3fff700 0 client.25742 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 0x5596866f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3772566136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.056+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3772566136 >> 172.21.15.12:0/3772566136 conn(0x559686645210 msgr2=0x5596866e15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.057+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3772566136 shutdown_connections 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.057+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3772566136 wait complete. 2024-05-08T23:48:11.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.057+0000 7f5dd71ecb80 20 client.25742 trim_cache size 0 max 16384 2024-05-08T23:48:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dd71ecb80 1 Processor -- start 2024-05-08T23:48:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dd71ecb80 1 -- start start 2024-05-08T23:48:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866ebca0 0x5596867016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f11f0 0x5596866e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 0x5596866effd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596866f6520 con 0x5596866ebca0 2024-05-08T23:48:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596866e3e50 con 0x5596866f11f0 2024-05-08T23:48:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596866e3640 con 0x5596866f1b10 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dc95cf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866ebca0 0x5596867016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dc95cf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866ebca0 0x5596867016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38062/0 (socket says 172.21.15.12:38062) 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3542293538 learned_addr learned my addr 172.21.15.12:0/3542293538 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dc3fff700 1 --2- 172.21.15.12:0/3542293538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f11f0 0x5596866e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.071+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3542293538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 0x5596866effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.072+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3542293538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 msgr2=0x5596866effd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.072+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3542293538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 0x5596866effd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.072+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3542293538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f11f0 msgr2=0x5596866e1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.072+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3542293538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f11f0 0x5596866e1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.072+0000 7f5dc95cf700 1 -- 172.21.15.12:0/3542293538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559686464b60 con 0x5596866ebca0 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.072+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/3542293538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866ebca0 0x5596867016d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc017c00 tx=0x7f5dbc00e140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49181060f157b09d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.072+0000 7f5daf7fe700 1 -- 172.21.15.12:0/3542293538 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc0049f0 con 0x5596866ebca0 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.072+0000 7f5daf7fe700 1 -- 172.21.15.12:0/3542293538 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dbc025500 con 0x5596866ebca0 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.072+0000 7f5daf7fe700 1 -- 172.21.15.12:0/3542293538 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc0156c0 con 0x5596866ebca0 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3542293538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866ebca0 msgr2=0x5596867016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3542293538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866ebca0 0x5596867016d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc017c00 tx=0x7f5dbc00e140 comp rx=0 tx=0).stop 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3542293538 shutdown_connections 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3542293538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866f1b10 0x5596866effd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3542293538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866ebca0 0x5596867016d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/3542293538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f11f0 0x5596866e1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3542293538 >> 172.21.15.12:0/3542293538 conn(0x559686645210 msgr2=0x5596866de210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3542293538 shutdown_connections 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/3542293538 wait complete. 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 -- start start 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.073+0000 7f5dd71ecb80 1 Processor -- start 2024-05-08T23:48:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 0x5596866fc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866fa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dd71ecb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 0x5596866faaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596866fb190 con 0x5596866f11f0 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596866fb300 con 0x5596866f1b10 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dd71ecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968675a9d0 con 0x5596866ebca0 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dd71ecb80 10 client.0 _ll_register_callbacks cb 0x5596865dcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc95cf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 0x5596866fc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866fa560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38064/0 (socket says 172.21.15.12:38064) 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc95cf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 0x5596866fc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44648/0 (socket says 172.21.15.12:44648) 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc3fff700 1 -- 172.21.15.12:0/1623055995 learned_addr learned my addr 172.21.15.12:0/1623055995 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 0x5596866faaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc3fff700 1 -- 172.21.15.12:0/1623055995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 msgr2=0x5596866fc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc3fff700 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 0x5596866fc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc3fff700 1 -- 172.21.15.12:0/1623055995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 msgr2=0x5596866faaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc3fff700 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 0x5596866faaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc3fff700 1 -- 172.21.15.12:0/1623055995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc0091e0 con 0x5596866f11f0 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5dc3fff700 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866fa560 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5db00098c0 tx=0x7f5db000f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395fc6ff1884c0dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.074+0000 7f5da3fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5da3fff700 1 -- 172.21.15.12:0/1623055995 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db000b830 con 0x5596866f11f0 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5da3fff700 1 -- 172.21.15.12:0/1623055995 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5db000b990 con 0x5596866f11f0 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/1623055995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5dbc00f040 con 0x5596866f11f0 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5da3fff700 1 -- 172.21.15.12:0/1623055995 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5db000bb70 con 0x5596866f11f0 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5dd71ecb80 10 client.15903 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/1623055995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5596865fd560 con 0x5596866f11f0 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5da3fff700 1 -- 172.21.15.12:0/1623055995 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dbc00f040 con 0x5596866f11f0 2024-05-08T23:48:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5da3fff700 1 -- 172.21.15.12:0/1623055995 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5db0009920 con 0x5596866f11f0 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5dd71ecb80 10 client.15903 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5dd71ecb80 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5dd71ecb80 20 client.15903 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55968675c0f0 0x55968675e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5dd71ecb80 4 client.15903 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/1623055995 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55968662fec0 con 0x55968675c0f0 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.075+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55968675c0f0 0x55968675e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.076+0000 7f5da3fff700 10 client.15903 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.076+0000 7f5dc95cf700 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55968675c0f0 0x55968675e510 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78ea04eeb457ce6 server_cookie=b27c58d7396ab911 in_seq=0 out_seq=0 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.076+0000 7f5da3fff700 1 -- 172.21.15.12:0/1623055995 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55968662fec0 con 0x55968675c0f0 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.076+0000 7f5da3fff700 10 client.15903 handle_command_reply: tid=0 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.076+0000 7f5dd71ecb80 1 client.15903 shutdown 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.076+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/1623055995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 msgr2=0x5596866fa560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.076+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866fa560 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5db00098c0 tx=0x7f5db000f980 comp rx=0 tx=0).stop 2024-05-08T23:48:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/1623055995 shutdown_connections 2024-05-08T23:48:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55968675c0f0 0x55968675e510 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5596866f1b10 0x5596866faaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596866ebca0 0x5596866fc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5da3fff700 0 client.15903 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dd71ecb80 1 --2- 172.21.15.12:0/1623055995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596866f11f0 0x5596866fa560 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/1623055995 >> 172.21.15.12:0/1623055995 conn(0x559686645210 msgr2=0x5596866f3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/1623055995 shutdown_connections 2024-05-08T23:48:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dd71ecb80 1 -- 172.21.15.12:0/1623055995 wait complete. 2024-05-08T23:48:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dd71ecb80 20 client.15903 trim_cache size 0 max 16384 2024-05-08T23:48:11.080 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dcb032700 1 -- 172.21.15.12:0/1184311082 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5db40a1290 msgr2=0x7f5db40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dcb032700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5db40a1290 0x7f5db40a3750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5db80ca630 tx=0x7f5db8010c80 comp rx=0 tx=0).stop 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dcb032700 1 -- 172.21.15.12:0/1184311082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dc4100060 msgr2=0x7f5dc41973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dcb032700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dc4100060 0x7f5dc41973b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5db800e9a0 tx=0x7f5db800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dcb032700 1 -- 172.21.15.12:0/1184311082 shutdown_connections 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dcb032700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5db40a1290 0x7f5db40a3750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dcb032700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc419bc90 0x7f5dc419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dcb032700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dc4100060 0x7f5dc41973b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dcb032700 1 --2- 172.21.15.12:0/1184311082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dc4105160 0x7f5dc41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.077+0000 7f5dcb032700 1 -- 172.21.15.12:0/1184311082 >> 172.21.15.12:0/1184311082 conn(0x7f5dc4076f40 msgr2=0x7f5dc4068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.078+0000 7f5dcb032700 1 -- 172.21.15.12:0/1184311082 shutdown_connections 2024-05-08T23:48:11.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:11.078+0000 7f5dcb032700 1 -- 172.21.15.12:0/1184311082 wait complete. 2024-05-08T23:48:11.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:11 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1664675010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:48:11.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:11 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1664675010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:48:11.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:11 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1664675010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:48:11.845 INFO:tasks.workunit.client.0.smithi012.stdout: 1 22062 19.44 MB/sec warmup 7 sec latency 12.907 ms 2024-05-08T23:48:12.087 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:48:12.088 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:48:12.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.265+0000 7f0885e32700 1 -- 172.21.15.12:0/2248498506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0880100a10 msgr2=0x7f0880104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.265+0000 7f0885e32700 1 --2- 172.21.15.12:0/2248498506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0880100a10 0x7f0880104a30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0868009990 tx=0x7f0868009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.265+0000 7f0885e32700 1 -- 172.21.15.12:0/2248498506 shutdown_connections 2024-05-08T23:48:12.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.265+0000 7f0885e32700 1 --2- 172.21.15.12:0/2248498506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0880105160 0x7f088010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.265+0000 7f0885e32700 1 --2- 172.21.15.12:0/2248498506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0880100060 0x7f0880100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.265+0000 7f0885e32700 1 --2- 172.21.15.12:0/2248498506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0880100a10 0x7f0880104a30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.265+0000 7f0885e32700 1 -- 172.21.15.12:0/2248498506 >> 172.21.15.12:0/2248498506 conn(0x7f0880076f40 msgr2=0x7f0880077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.265+0000 7f0885e32700 1 -- 172.21.15.12:0/2248498506 shutdown_connections 2024-05-08T23:48:12.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.265+0000 7f0885e32700 1 -- 172.21.15.12:0/2248498506 wait complete. 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.266+0000 7f0885e32700 1 Processor -- start 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.266+0000 7f0885e32700 1 -- start start 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f0885e32700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0880100060 0x7f0880101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f0885e32700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0880105160 0x7f0880102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f0885e32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0880102550 0x7f08801a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f0885e32700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08801a5bf0 con 0x7f0880102550 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f0885e32700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08801a5d60 con 0x7f0880100060 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f0885e32700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08801a5f20 con 0x7f0880105160 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0880100060 0x7f0880101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0880105160 0x7f0880102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0880102550 0x7f08801a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0880100060 0x7f0880101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52730/0 (socket says 172.21.15.12:52730) 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087f7fe700 1 -- 172.21.15.12:0/2431985687 learned_addr learned my addr 172.21.15.12:0/2431985687 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0880105160 0x7f0880102010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44660/0 (socket says 172.21.15.12:44660) 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087f7fe700 1 -- 172.21.15.12:0/2431985687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0880105160 msgr2=0x7f0880102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087f7fe700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0880105160 0x7f0880102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087f7fe700 1 -- 172.21.15.12:0/2431985687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0880102550 msgr2=0x7f08801a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087f7fe700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0880102550 0x7f08801a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087f7fe700 1 -- 172.21.15.12:0/2431985687 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680095f0 con 0x7f0880100060 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087ffff700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0880102550 0x7f08801a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087f7fe700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0880100060 0x7f0880101ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f087000ef10 tx=0x7f087000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.267+0000 7f087cff9700 1 -- 172.21.15.12:0/2431985687 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000c9c0 con 0x7f0880100060 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.268+0000 7f087cff9700 1 -- 172.21.15.12:0/2431985687 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f087000f040 con 0x7f0880100060 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.268+0000 7f0885e32700 1 -- 172.21.15.12:0/2431985687 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08801a61a0 con 0x7f0880100060 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.268+0000 7f087cff9700 1 -- 172.21.15.12:0/2431985687 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870014480 con 0x7f0880100060 2024-05-08T23:48:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.268+0000 7f0885e32700 1 -- 172.21.15.12:0/2431985687 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08801a6680 con 0x7f0880100060 2024-05-08T23:48:12.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.270+0000 7f087cff9700 1 -- 172.21.15.12:0/2431985687 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0870007690 con 0x7f0880100060 2024-05-08T23:48:12.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.270+0000 7f087cff9700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f086c0a1290 0x7f086c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.271+0000 7f087cff9700 1 -- 172.21.15.12:0/2431985687 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f08700ccf50 con 0x7f0880100060 2024-05-08T23:48:12.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.271+0000 7f087effd700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f086c0a1290 0x7f086c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.271+0000 7f087effd700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f086c0a1290 0x7f086c0a3750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f086800b2e0 tx=0x7f0868005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.271+0000 7f0891fecb80 1 Processor -- start 2024-05-08T23:48:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.271+0000 7f0891fecb80 1 -- start start 2024-05-08T23:48:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0ea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e4910 0x55bf7a0e4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f0891fecb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf7a03b1d0 con 0x55bf7a0e9e80 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f0891fecb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf7a038b90 con 0x55bf7a0e4910 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f0891fecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf7a034480 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0ea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0ea260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38086/0 (socket says 172.21.15.12:38086) 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087f7fe700 1 -- 172.21.15.12:0/542675531 learned_addr learned my addr 172.21.15.12:0/542675531 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087ffff700 1 --2- 172.21.15.12:0/542675531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e4910 0x55bf7a0e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087effd700 1 -- 172.21.15.12:0/542675531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e4910 msgr2=0x55bf7a0e4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087effd700 1 --2- 172.21.15.12:0/542675531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e4910 0x55bf7a0e4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087effd700 1 -- 172.21.15.12:0/542675531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e9e80 msgr2=0x55bf7a0ea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087effd700 1 --2- 172.21.15.12:0/542675531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0ea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087effd700 1 -- 172.21.15.12:0/542675531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf79eb5430 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.272+0000 7f087f7fe700 1 --2- 172.21.15.12:0/542675531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0ea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.273+0000 7f087effd700 1 --2- 172.21.15.12:0/542675531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0e43d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0868006010 tx=0x7f0868020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e1bca036c68900c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.273+0000 7f0875ffb700 1 -- 172.21.15.12:0/542675531 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680056c0 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.273+0000 7f0875ffb700 1 -- 172.21.15.12:0/542675531 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f086801e040 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.273+0000 7f0875ffb700 1 -- 172.21.15.12:0/542675531 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086800f900 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.273+0000 7f0891fecb80 1 -- 172.21.15.12:0/542675531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0ea7a0 msgr2=0x55bf7a0e43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.274+0000 7f0891fecb80 1 --2- 172.21.15.12:0/542675531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0e43d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0868006010 tx=0x7f0868020000 comp rx=0 tx=0).stop 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.274+0000 7f0891fecb80 1 -- 172.21.15.12:0/542675531 shutdown_connections 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.274+0000 7f0891fecb80 1 --2- 172.21.15.12:0/542675531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e4910 0x55bf7a0e4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.274+0000 7f0891fecb80 1 --2- 172.21.15.12:0/542675531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0ea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.274+0000 7f0891fecb80 1 --2- 172.21.15.12:0/542675531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0e43d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.274+0000 7f0891fecb80 1 -- 172.21.15.12:0/542675531 >> 172.21.15.12:0/542675531 conn(0x55bf7a042290 msgr2=0x55bf7a0dee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.274+0000 7f0891fecb80 1 -- 172.21.15.12:0/542675531 shutdown_connections 2024-05-08T23:48:12.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.274+0000 7f0891fecb80 1 -- 172.21.15.12:0/542675531 wait complete. 2024-05-08T23:48:12.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.274+0000 7f0891fecb80 1 -- start start 2024-05-08T23:48:12.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.274+0000 7f0891fecb80 1 Processor -- start 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0ee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f0891fecb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf79fd05e0 con 0x55bf7a0e4910 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f0891fecb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf79fd0280 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f0891fecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf79eb7720 con 0x55bf7a0e9e80 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f0891fecb80 10 client.0 _ll_register_callbacks cb 0x55bf79fd9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0edbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38092/0 (socket says 172.21.15.12:38092) 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087f7fe700 1 -- 172.21.15.12:0/2489746356 learned_addr learned my addr 172.21.15.12:0/2489746356 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:12.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087effd700 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087ffff700 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087f7fe700 1 -- 172.21.15.12:0/2489746356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 msgr2=0x55bf7a0ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087f7fe700 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0ee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087f7fe700 1 -- 172.21.15.12:0/2489746356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 msgr2=0x55bf7a0ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087f7fe700 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0ee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087f7fe700 1 -- 172.21.15.12:0/2489746356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08801a6680 con 0x55bf7a0e4910 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f087f7fe700 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0edbe0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f08700cace0 tx=0x7f0870099a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da9460311c422b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f085effd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f085effd700 1 -- 172.21.15.12:0/2489746356 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870099790 con 0x55bf7a0e4910 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f085effd700 1 -- 172.21.15.12:0/2489746356 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f087009a0a0 con 0x55bf7a0e4910 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.275+0000 7f0891fecb80 1 -- 172.21.15.12:0/2489746356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf79f8d2f0 con 0x55bf7a0e4910 2024-05-08T23:48:12.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f085effd700 1 -- 172.21.15.12:0/2489746356 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f087009ded0 con 0x55bf7a0e4910 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f0891fecb80 10 client.15915 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f0891fecb80 1 -- 172.21.15.12:0/2489746356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f088010d550 con 0x55bf7a0e4910 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f085effd700 1 -- 172.21.15.12:0/2489746356 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0870063080 con 0x55bf7a0e4910 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f085effd700 1 -- 172.21.15.12:0/2489746356 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f08700b8170 con 0x55bf7a0e4910 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f0891fecb80 10 client.15915 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f0891fecb80 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f0891fecb80 20 client.15915 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f0891fecb80 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf7a0fe180 0x55bf7a1005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f0891fecb80 4 client.15915 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f0891fecb80 1 -- 172.21.15.12:0/2489746356 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf7a0b3890 con 0x55bf7a0fe180 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.276+0000 7f087effd700 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf7a0fe180 0x55bf7a1005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.277+0000 7f087effd700 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf7a0fe180 0x55bf7a1005a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26cec0db04adecb server_cookie=32e0ad80adbba021 in_seq=0 out_seq=0 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.277+0000 7f085effd700 10 client.15915 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.278+0000 7f085effd700 1 -- 172.21.15.12:0/2489746356 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf7a0b3890 con 0x55bf7a0fe180 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.278+0000 7f085effd700 10 client.15915 handle_command_reply: tid=0 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.278+0000 7f0891fecb80 1 client.15915 shutdown 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 1 -- 172.21.15.12:0/2489746356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 msgr2=0x55bf7a0edbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0edbe0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f08700cace0 tx=0x7f0870099a80 comp rx=0 tx=0).stop 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 1 -- 172.21.15.12:0/2489746356 shutdown_connections 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf7a0fe180 0x55bf7a1005a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0edbe0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 1 --2- 172.21.15.12:0/2489746356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 1 -- 172.21.15.12:0/2489746356 >> 172.21.15.12:0/2489746356 conn(0x55bf7a042290 msgr2=0x55bf7a0de620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f085effd700 0 client.15915 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 1 -- 172.21.15.12:0/2489746356 shutdown_connections 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 1 -- 172.21.15.12:0/2489746356 wait complete. 2024-05-08T23:48:12.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.279+0000 7f0891fecb80 20 client.15915 trim_cache size 0 max 16384 2024-05-08T23:48:12.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f0891fecb80 1 Processor -- start 2024-05-08T23:48:12.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f0891fecb80 1 -- start start 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e4910 0x55bf7a0e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0e9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f0891fecb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf7a0ea660 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f0891fecb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf7a0ef1c0 con 0x55bf7a0e9e80 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f0891fecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf7a0dbe40 con 0x55bf7a0e4910 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f087ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f087effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f087ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0deda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38102/0 (socket says 172.21.15.12:38102) 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f087f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e4910 0x55bf7a0e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f087ffff700 1 -- 172.21.15.12:0/4191086266 learned_addr learned my addr 172.21.15.12:0/4191086266 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.299+0000 7f087effd700 1 --2- 172.21.15.12:0/4191086266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0e9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52752/0 (socket says 172.21.15.12:52752) 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f087effd700 1 -- 172.21.15.12:0/4191086266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e4910 msgr2=0x55bf7a0e43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f087effd700 1 --2- 172.21.15.12:0/4191086266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e4910 0x55bf7a0e43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f087effd700 1 -- 172.21.15.12:0/4191086266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0ea7a0 msgr2=0x55bf7a0deda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f087effd700 1 --2- 172.21.15.12:0/4191086266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0deda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f087effd700 1 -- 172.21.15.12:0/4191086266 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf79e61be0 con 0x55bf7a0e9e80 2024-05-08T23:48:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f087ffff700 1 --2- 172.21.15.12:0/4191086266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f087f7fe700 1 --2- 172.21.15.12:0/4191086266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e4910 0x55bf7a0e43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f087effd700 1 --2- 172.21.15.12:0/4191086266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0e9510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0868005f50 tx=0x7f086801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65143d00131251ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f08767fc700 1 -- 172.21.15.12:0/4191086266 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868004500 con 0x55bf7a0e9e80 2024-05-08T23:48:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f08767fc700 1 -- 172.21.15.12:0/4191086266 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0868004b20 con 0x55bf7a0e9e80 2024-05-08T23:48:12.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.300+0000 7f08767fc700 1 -- 172.21.15.12:0/4191086266 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086800f980 con 0x55bf7a0e9e80 2024-05-08T23:48:12.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.301+0000 7f0891fecb80 1 -- 172.21.15.12:0/4191086266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e9e80 msgr2=0x55bf7a0e9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.301+0000 7f0891fecb80 1 --2- 172.21.15.12:0/4191086266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0e9510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0868005f50 tx=0x7f086801a040 comp rx=0 tx=0).stop 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.301+0000 7f0891fecb80 1 -- 172.21.15.12:0/4191086266 shutdown_connections 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.301+0000 7f0891fecb80 1 --2- 172.21.15.12:0/4191086266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.301+0000 7f0891fecb80 1 --2- 172.21.15.12:0/4191086266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e4910 0x55bf7a0e43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.301+0000 7f0891fecb80 1 --2- 172.21.15.12:0/4191086266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0e9510 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.301+0000 7f0891fecb80 1 -- 172.21.15.12:0/4191086266 >> 172.21.15.12:0/4191086266 conn(0x55bf7a042290 msgr2=0x55bf7a0ec4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.301+0000 7f0891fecb80 1 -- 172.21.15.12:0/4191086266 shutdown_connections 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.301+0000 7f0891fecb80 1 -- 172.21.15.12:0/4191086266 wait complete. 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.301+0000 7f0891fecb80 1 -- start start 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f0891fecb80 1 Processor -- start 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f0891fecb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f0891fecb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf7a0f4120 con 0x55bf7a0e4910 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f0891fecb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf7a0f4290 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f0891fecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf7a0f4450 con 0x55bf7a0e9e80 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f0891fecb80 10 client.0 _ll_register_callbacks cb 0x55bf79fd9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f087ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f087f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f087effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f087ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52760/0 (socket says 172.21.15.12:52760) 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f087effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44708/0 (socket says 172.21.15.12:44708) 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.302+0000 7f087ffff700 1 -- 172.21.15.12:0/3494749032 learned_addr learned my addr 172.21.15.12:0/3494749032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f087ffff700 1 -- 172.21.15.12:0/3494749032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 msgr2=0x55bf7a0f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f087ffff700 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f087ffff700 1 -- 172.21.15.12:0/3494749032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 msgr2=0x55bf7a0f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f087ffff700 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f087ffff700 1 -- 172.21.15.12:0/3494749032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088010d550 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f087f7fe700 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f087ffff700 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0f3be0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0878003a70 tx=0x7f0878003d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c0988c11bbcfb42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f085effd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f085effd700 1 -- 172.21.15.12:0/3494749032 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878012a60 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f085effd700 1 -- 172.21.15.12:0/3494749032 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0878012bc0 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f0891fecb80 1 -- 172.21.15.12:0/3494749032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f086801e040 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f085effd700 1 -- 172.21.15.12:0/3494749032 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08780122e0 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f0891fecb80 10 client.25766 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f0891fecb80 1 -- 172.21.15.12:0/3494749032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f0878004040 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.303+0000 7f085effd700 1 -- 172.21.15.12:0/3494749032 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f087802a020 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.304+0000 7f085effd700 1 -- 172.21.15.12:0/3494749032 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f08780052b0 con 0x55bf7a0ea7a0 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.304+0000 7f0891fecb80 10 client.25766 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.304+0000 7f0891fecb80 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.304+0000 7f0891fecb80 20 client.25766 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.304+0000 7f0891fecb80 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf7a155b70 0x55bf7a157f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.304+0000 7f0891fecb80 4 client.25766 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:48:12.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.304+0000 7f0891fecb80 1 -- 172.21.15.12:0/3494749032 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bf7a02cf40 con 0x55bf7a155b70 2024-05-08T23:48:12.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.304+0000 7f087f7fe700 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf7a155b70 0x55bf7a157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.304+0000 7f087f7fe700 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf7a155b70 0x55bf7a157f90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7692700c35ac8afc server_cookie=7cbc77f07fdc8536 in_seq=0 out_seq=0 2024-05-08T23:48:12.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.304+0000 7f085effd700 10 client.25766 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.306+0000 7f085effd700 1 -- 172.21.15.12:0/3494749032 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bf7a02cf40 con 0x55bf7a155b70 2024-05-08T23:48:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.306+0000 7f085effd700 10 client.25766 handle_command_reply: tid=0 2024-05-08T23:48:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.306+0000 7f0891fecb80 1 client.25766 shutdown 2024-05-08T23:48:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 1 -- 172.21.15.12:0/3494749032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 msgr2=0x55bf7a0f3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0f3be0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0878003a70 tx=0x7f0878003d80 comp rx=0 tx=0).stop 2024-05-08T23:48:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 1 -- 172.21.15.12:0/3494749032 shutdown_connections 2024-05-08T23:48:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf7a155b70 0x55bf7a157f90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf7a0ea7a0 0x55bf7a0f3be0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf7a0e4910 0x55bf7a0f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f085effd700 0 client.25766 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:12.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 1 --2- 172.21.15.12:0/3494749032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf7a0e9e80 0x55bf7a0f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 1 -- 172.21.15.12:0/3494749032 >> 172.21.15.12:0/3494749032 conn(0x55bf7a042290 msgr2=0x55bf7a0ee4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 1 -- 172.21.15.12:0/3494749032 shutdown_connections 2024-05-08T23:48:12.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 1 -- 172.21.15.12:0/3494749032 wait complete. 2024-05-08T23:48:12.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0891fecb80 20 client.25766 trim_cache size 0 max 16384 2024-05-08T23:48:12.306 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:48:12.306 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:48:12.306 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "dc6bbe79-b8cc-4981-9f5a-894396866f5c", 2024-05-08T23:48:12.306 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:48:12.306 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:48:12.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0885e32700 1 -- 172.21.15.12:0/2431985687 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f086c0a1290 msgr2=0x7f086c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0885e32700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f086c0a1290 0x7f086c0a3750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f086800b2e0 tx=0x7f0868005960 comp rx=0 tx=0).stop 2024-05-08T23:48:12.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0885e32700 1 -- 172.21.15.12:0/2431985687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0880100060 msgr2=0x7f0880101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0885e32700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0880100060 0x7f0880101ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f087000ef10 tx=0x7f087000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0885e32700 1 -- 172.21.15.12:0/2431985687 shutdown_connections 2024-05-08T23:48:12.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0885e32700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f086c0a1290 0x7f086c0a3750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0885e32700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0880102550 0x7f08801a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0885e32700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0880100060 0x7f0880101ad0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0885e32700 1 --2- 172.21.15.12:0/2431985687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0880105160 0x7f0880102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.307+0000 7f0885e32700 1 -- 172.21.15.12:0/2431985687 >> 172.21.15.12:0/2431985687 conn(0x7f0880076f40 msgr2=0x7f0880068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.308+0000 7f0885e32700 1 -- 172.21.15.12:0/2431985687 shutdown_connections 2024-05-08T23:48:12.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.308+0000 7f0885e32700 1 -- 172.21.15.12:0/2431985687 wait complete. 2024-05-08T23:48:12.317 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:48:12.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.465+0000 7f537d2cf700 1 -- 172.21.15.12:0/1816458164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53781062d0 msgr2=0x7f53780696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.465+0000 7f537d2cf700 1 --2- 172.21.15.12:0/1816458164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53781062d0 0x7f53780696f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f536c009990 tx=0x7f536c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.465+0000 7f537d2cf700 1 -- 172.21.15.12:0/1816458164 shutdown_connections 2024-05-08T23:48:12.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.465+0000 7f537d2cf700 1 --2- 172.21.15.12:0/1816458164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5378103960 0x7f5378103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.465+0000 7f537d2cf700 1 --2- 172.21.15.12:0/1816458164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5378105e60 0x7f53780691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.465+0000 7f537d2cf700 1 --2- 172.21.15.12:0/1816458164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53781062d0 0x7f53780696f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.465+0000 7f537d2cf700 1 -- 172.21.15.12:0/1816458164 >> 172.21.15.12:0/1816458164 conn(0x7f5378076ba0 msgr2=0x7f5378076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.465+0000 7f537d2cf700 1 -- 172.21.15.12:0/1816458164 shutdown_connections 2024-05-08T23:48:12.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.465+0000 7f537d2cf700 1 -- 172.21.15.12:0/1816458164 wait complete. 2024-05-08T23:48:12.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.465+0000 7f537d2cf700 1 Processor -- start 2024-05-08T23:48:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f537d2cf700 1 -- start start 2024-05-08T23:48:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f537d2cf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5378103960 0x7f53781a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f537d2cf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5378105e60 0x7f53781a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f537d2cf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53781062d0 0x7f537819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f537d2cf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537819ddd0 con 0x7f5378103960 2024-05-08T23:48:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f537d2cf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537819df40 con 0x7f53781062d0 2024-05-08T23:48:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f537d2cf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537819e100 con 0x7f5378105e60 2024-05-08T23:48:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5378103960 0x7f53781a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5378103960 0x7f53781a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38126/0 (socket says 172.21.15.12:38126) 2024-05-08T23:48:12.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f5376ffd700 1 -- 172.21.15.12:0/216811810 learned_addr learned my addr 172.21.15.12:0/216811810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f53767fc700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5378105e60 0x7f53781a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f53777fe700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53781062d0 0x7f537819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f5376ffd700 1 -- 172.21.15.12:0/216811810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5378105e60 msgr2=0x7f53781a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f5376ffd700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5378105e60 0x7f53781a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.466+0000 7f5376ffd700 1 -- 172.21.15.12:0/216811810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53781062d0 msgr2=0x7f537819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.467+0000 7f5376ffd700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53781062d0 0x7f537819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.467+0000 7f5376ffd700 1 -- 172.21.15.12:0/216811810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0095f0 con 0x7f5378103960 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.467+0000 7f53767fc700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5378105e60 0x7f53781a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.467+0000 7f5376ffd700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5378103960 0x7f53781a3640 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f536800e9a0 tx=0x7f536800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.467+0000 7f535ffff700 1 -- 172.21.15.12:0/216811810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368009790 con 0x7f5378103960 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.467+0000 7f537d2cf700 1 -- 172.21.15.12:0/216811810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537819e3e0 con 0x7f5378103960 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.467+0000 7f535ffff700 1 -- 172.21.15.12:0/216811810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5368010430 con 0x7f5378103960 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.467+0000 7f535ffff700 1 -- 172.21.15.12:0/216811810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368018680 con 0x7f5378103960 2024-05-08T23:48:12.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.467+0000 7f537d2cf700 1 -- 172.21.15.12:0/216811810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53781aa630 con 0x7f5378103960 2024-05-08T23:48:12.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.470+0000 7f535ffff700 1 -- 172.21.15.12:0/216811810 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f53680105a0 con 0x7f5378103960 2024-05-08T23:48:12.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.470+0000 7f535ffff700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53600a1290 0x7f53600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.470+0000 7f535ffff700 1 -- 172.21.15.12:0/216811810 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5368014070 con 0x7f5378103960 2024-05-08T23:48:12.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.470+0000 7f53767fc700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53600a1290 0x7f53600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.471+0000 7f53767fc700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53600a1290 0x7f53600a3750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b2e0 tx=0x7f536c005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5389489b80 1 Processor -- start 2024-05-08T23:48:12.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5389489b80 1 -- start start 2024-05-08T23:48:12.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5389489b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c121e00 0x560c5c1221e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5389489b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c122720 0x560c5c11c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5389489b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 0x560c5c11ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5389489b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c073150 con 0x560c5c121e00 2024-05-08T23:48:12.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5389489b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c070b10 con 0x560c5c11c890 2024-05-08T23:48:12.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5389489b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c06c400 con 0x560c5c122720 2024-05-08T23:48:12.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c121e00 0x560c5c1221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c121e00 0x560c5c1221e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38136/0 (socket says 172.21.15.12:38136) 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f5376ffd700 1 -- 172.21.15.12:0/2286199828 learned_addr learned my addr 172.21.15.12:0/2286199828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f53777fe700 1 --2- 172.21.15.12:0/2286199828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 0x560c5c11ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.472+0000 7f53767fc700 1 --2- 172.21.15.12:0/2286199828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c122720 0x560c5c11c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.473+0000 7f5376ffd700 1 -- 172.21.15.12:0/2286199828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c122720 msgr2=0x560c5c11c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.473+0000 7f5376ffd700 1 --2- 172.21.15.12:0/2286199828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c122720 0x560c5c11c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.473+0000 7f5376ffd700 1 -- 172.21.15.12:0/2286199828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 msgr2=0x560c5c11ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.473+0000 7f5376ffd700 1 --2- 172.21.15.12:0/2286199828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 0x560c5c11ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.473+0000 7f5376ffd700 1 -- 172.21.15.12:0/2286199828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c5beed3b0 con 0x560c5c121e00 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.473+0000 7f53777fe700 1 --2- 172.21.15.12:0/2286199828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 0x560c5c11ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.473+0000 7f5376ffd700 1 --2- 172.21.15.12:0/2286199828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c121e00 0x560c5c1221e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f53680a1eb0 tx=0x7f5368099940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100b6f9a7d6d34d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.473+0000 7f535d7fa700 1 -- 172.21.15.12:0/2286199828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536809dd90 con 0x560c5c121e00 2024-05-08T23:48:12.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.473+0000 7f535d7fa700 1 -- 172.21.15.12:0/2286199828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53680a23e0 con 0x560c5c121e00 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.473+0000 7f535d7fa700 1 -- 172.21.15.12:0/2286199828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53680996b0 con 0x560c5c121e00 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 -- 172.21.15.12:0/2286199828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c121e00 msgr2=0x560c5c1221e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 --2- 172.21.15.12:0/2286199828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c121e00 0x560c5c1221e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f53680a1eb0 tx=0x7f5368099940 comp rx=0 tx=0).stop 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 -- 172.21.15.12:0/2286199828 shutdown_connections 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 --2- 172.21.15.12:0/2286199828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 0x560c5c11ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 --2- 172.21.15.12:0/2286199828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c121e00 0x560c5c1221e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 --2- 172.21.15.12:0/2286199828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c122720 0x560c5c11c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 -- 172.21.15.12:0/2286199828 >> 172.21.15.12:0/2286199828 conn(0x560c5c07a210 msgr2=0x560c5c116df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 -- 172.21.15.12:0/2286199828 shutdown_connections 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 -- 172.21.15.12:0/2286199828 wait complete. 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 -- start start 2024-05-08T23:48:12.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.474+0000 7f5389489b80 1 Processor -- start 2024-05-08T23:48:12.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5389489b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c11c890 0x560c5c125b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5389489b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c121e00 0x560c5c126070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5389489b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c122720 0x560c5c1265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5389489b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c008560 con 0x560c5c11c890 2024-05-08T23:48:12.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5389489b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c008200 con 0x560c5c121e00 2024-05-08T23:48:12.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5389489b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5beef6a0 con 0x560c5c122720 2024-05-08T23:48:12.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c11c890 0x560c5c125b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5389489b80 10 client.0 _ll_register_callbacks cb 0x560c5c011b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:12.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c11c890 0x560c5c125b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38146/0 (socket says 172.21.15.12:38146) 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5376ffd700 1 -- 172.21.15.12:0/2896810047 learned_addr learned my addr 172.21.15.12:0/2896810047 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f53767fc700 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c121e00 0x560c5c126070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f53777fe700 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c122720 0x560c5c1265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5376ffd700 1 -- 172.21.15.12:0/2896810047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c122720 msgr2=0x560c5c1265b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5376ffd700 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c122720 0x560c5c1265b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5376ffd700 1 -- 172.21.15.12:0/2896810047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c121e00 msgr2=0x560c5c126070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5376ffd700 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c121e00 0x560c5c126070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.475+0000 7f5376ffd700 1 -- 172.21.15.12:0/2896810047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5370011230 con 0x560c5c11c890 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f5376ffd700 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c11c890 0x560c5c125b30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f53680ccf30 tx=0x7f5368010ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef749c4fe3b5e6e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f535aff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f535aff5700 1 -- 172.21.15.12:0/2896810047 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536809de10 con 0x560c5c11c890 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f535aff5700 1 -- 172.21.15.12:0/2896810047 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5368099660 con 0x560c5c11c890 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f5389489b80 1 -- 172.21.15.12:0/2896810047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c5bfc5270 con 0x560c5c11c890 2024-05-08T23:48:12.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f535aff5700 1 -- 172.21.15.12:0/2896810047 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53680b81b0 con 0x560c5c11c890 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f5389489b80 10 client.15933 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f5389489b80 1 -- 172.21.15.12:0/2896810047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f53781aa630 con 0x560c5c11c890 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f535aff5700 1 -- 172.21.15.12:0/2896810047 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c5bfc5270 con 0x560c5c11c890 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f535aff5700 1 -- 172.21.15.12:0/2896810047 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5368095d50 con 0x560c5c11c890 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f5389489b80 10 client.15933 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f5389489b80 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f5389489b80 20 client.15933 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f5389489b80 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c5c135ed0 0x560c5c1382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f5389489b80 4 client.15933 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.476+0000 7f5389489b80 1 -- 172.21.15.12:0/2896810047 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c5c0eb810 con 0x560c5c135ed0 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.477+0000 7f53767fc700 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c5c135ed0 0x560c5c1382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.477+0000 7f53767fc700 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c5c135ed0 0x560c5c1382f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad98ac334e4949e server_cookie=994e48d15fe6a9db in_seq=0 out_seq=0 2024-05-08T23:48:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.477+0000 7f535aff5700 10 client.15933 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f535aff5700 1 -- 172.21.15.12:0/2896810047 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c5c0eb810 con 0x560c5c135ed0 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f535aff5700 10 client.15933 handle_command_reply: tid=0 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 client.15933 shutdown 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 -- 172.21.15.12:0/2896810047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c11c890 msgr2=0x560c5c125b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c11c890 0x560c5c125b30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f53680ccf30 tx=0x7f5368010ab0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 -- 172.21.15.12:0/2896810047 shutdown_connections 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c122720 0x560c5c1265b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c11c890 0x560c5c125b30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c5c135ed0 0x560c5c1382f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 --2- 172.21.15.12:0/2896810047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c121e00 0x560c5c126070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 -- 172.21.15.12:0/2896810047 >> 172.21.15.12:0/2896810047 conn(0x560c5c07a210 msgr2=0x560c5c1165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f535aff5700 0 client.15933 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 -- 172.21.15.12:0/2896810047 shutdown_connections 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 1 -- 172.21.15.12:0/2896810047 wait complete. 2024-05-08T23:48:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.479+0000 7f5389489b80 20 client.15933 trim_cache size 0 max 16384 2024-05-08T23:48:12.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.497+0000 7f5389489b80 1 Processor -- start 2024-05-08T23:48:12.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f5389489b80 1 -- start start 2024-05-08T23:48:12.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f5389489b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 0x560c5c131bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f5389489b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c121e00 0x560c5c116d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f5389489b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 0x560c5c126080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f5389489b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c118640 con 0x560c5c122720 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f5389489b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c13d500 con 0x560c5c11c890 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f5389489b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c111800 con 0x560c5c121e00 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f53777fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 0x560c5c126080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f53767fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c121e00 0x560c5c116d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 0x560c5c131bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f53777fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 0x560c5c126080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38150/0 (socket says 172.21.15.12:38150) 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f53777fe700 1 -- 172.21.15.12:0/1266125879 learned_addr learned my addr 172.21.15.12:0/1266125879 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f53767fc700 1 -- 172.21.15.12:0/1266125879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 msgr2=0x560c5c131bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f53767fc700 1 --2- 172.21.15.12:0/1266125879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 0x560c5c131bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f53767fc700 1 -- 172.21.15.12:0/1266125879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 msgr2=0x560c5c126080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f53767fc700 1 --2- 172.21.15.12:0/1266125879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 0x560c5c126080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f53767fc700 1 -- 172.21.15.12:0/1266125879 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c5be99b60 con 0x560c5c121e00 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.498+0000 7f53777fe700 1 --2- 172.21.15.12:0/1266125879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 0x560c5c126080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.499+0000 7f53767fc700 1 --2- 172.21.15.12:0/1266125879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c121e00 0x560c5c116d20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f536c0059d0 tx=0x7f536c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8023683a28941dbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.499+0000 7f535dffb700 1 -- 172.21.15.12:0/1266125879 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c037070 con 0x560c5c121e00 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.499+0000 7f535dffb700 1 -- 172.21.15.12:0/1266125879 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f536c02c040 con 0x560c5c121e00 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.499+0000 7f535dffb700 1 -- 172.21.15.12:0/1266125879 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c02b040 con 0x560c5c121e00 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 -- 172.21.15.12:0/1266125879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c121e00 msgr2=0x560c5c116d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 --2- 172.21.15.12:0/1266125879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c121e00 0x560c5c116d20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f536c0059d0 tx=0x7f536c020000 comp rx=0 tx=0).stop 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 -- 172.21.15.12:0/1266125879 shutdown_connections 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 --2- 172.21.15.12:0/1266125879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 0x560c5c126080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 --2- 172.21.15.12:0/1266125879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c11c890 0x560c5c131bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 --2- 172.21.15.12:0/1266125879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c121e00 0x560c5c116d20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 -- 172.21.15.12:0/1266125879 >> 172.21.15.12:0/1266125879 conn(0x560c5c07a210 msgr2=0x560c5c11cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 -- 172.21.15.12:0/1266125879 shutdown_connections 2024-05-08T23:48:12.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 -- 172.21.15.12:0/1266125879 wait complete. 2024-05-08T23:48:12.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 -- start start 2024-05-08T23:48:12.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.500+0000 7f5389489b80 1 Processor -- start 2024-05-08T23:48:12.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5389489b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c11c890 0x560c5c12b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5389489b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c121e00 0x560c5c12b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5389489b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 0x560c5c18b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5389489b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c12bf40 con 0x560c5c122720 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5389489b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c18b910 con 0x560c5c121e00 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5389489b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5c18ba50 con 0x560c5c11c890 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5389489b80 10 client.0 _ll_register_callbacks cb 0x560c5c011b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c11c890 0x560c5c12b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c11c890 0x560c5c12b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44772/0 (socket says 172.21.15.12:44772) 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f53767fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c121e00 0x560c5c12b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5376ffd700 1 -- 172.21.15.12:0/1955165395 learned_addr learned my addr 172.21.15.12:0/1955165395 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f53777fe700 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 0x560c5c18b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5376ffd700 1 -- 172.21.15.12:0/1955165395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c121e00 msgr2=0x560c5c12b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5376ffd700 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c121e00 0x560c5c12b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5376ffd700 1 -- 172.21.15.12:0/1955165395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 msgr2=0x560c5c18b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5376ffd700 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 0x560c5c18b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5376ffd700 1 -- 172.21.15.12:0/1955165395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53700092d0 con 0x560c5c11c890 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5376ffd700 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c11c890 0x560c5c12b0f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5368099a30 tx=0x7f5368005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92bd801a6356f939 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f535aff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f535aff5700 1 -- 172.21.15.12:0/1955165395 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536809dd90 con 0x560c5c11c890 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f535aff5700 1 -- 172.21.15.12:0/1955165395 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f536809a1c0 con 0x560c5c11c890 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.501+0000 7f5389489b80 1 -- 172.21.15.12:0/1955165395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53680a23e0 con 0x560c5c11c890 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f535aff5700 1 -- 172.21.15.12:0/1955165395 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5368065070 con 0x560c5c11c890 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f5389489b80 10 client.25789 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f5389489b80 1 -- 172.21.15.12:0/1955165395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f5378114f50 con 0x560c5c11c890 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f535aff5700 1 -- 172.21.15.12:0/1955165395 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53680a23e0 con 0x560c5c11c890 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f535aff5700 1 -- 172.21.15.12:0/1955165395 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5368094b70 con 0x560c5c11c890 2024-05-08T23:48:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f5389489b80 10 client.25789 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f5389489b80 10 client.25789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f5389489b80 20 client.25789 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f5389489b80 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c5c18cca0 0x560c5c18f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f5389489b80 4 client.25789 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f5389489b80 1 -- 172.21.15.12:0/1955165395 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c5c064ec0 con 0x560c5c18cca0 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.502+0000 7f53767fc700 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c5c18cca0 0x560c5c18f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f53767fc700 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c5c18cca0 0x560c5c18f090 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20cc29e77475f6b6 server_cookie=70648c1b70c44dfb in_seq=0 out_seq=0 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f535aff5700 10 client.25789 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f535aff5700 1 -- 172.21.15.12:0/1955165395 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560c5c064ec0 con 0x560c5c18cca0 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f535aff5700 10 client.25789 handle_command_reply: tid=0 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 client.25789 shutdown 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 -- 172.21.15.12:0/1955165395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c11c890 msgr2=0x560c5c12b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c11c890 0x560c5c12b0f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5368099a30 tx=0x7f5368005760 comp rx=0 tx=0).stop 2024-05-08T23:48:12.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 -- 172.21.15.12:0/1955165395 shutdown_connections 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c5c122720 0x560c5c18b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c5c11c890 0x560c5c12b0f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c5c18cca0 0x560c5c18f090 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 --2- 172.21.15.12:0/1955165395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c5c121e00 0x560c5c12b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f535aff5700 0 client.25789 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 -- 172.21.15.12:0/1955165395 >> 172.21.15.12:0/1955165395 conn(0x560c5c07a210 msgr2=0x560c5c124340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 -- 172.21.15.12:0/1955165395 shutdown_connections 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 1 -- 172.21.15.12:0/1955165395 wait complete. 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.503+0000 7f5389489b80 20 client.25789 trim_cache size 0 max 16384 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:48:12.505 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:48:12.506 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:48:12.506 INFO:teuthology.orchestra.run.smithi012.stdout: "dc6bbe79-b8cc-4981-9f5a-894396866f5c": { 2024-05-08T23:48:12.506 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:48:12.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "dc6bbe79-b8cc-4981-9f5a-894396866f5c", 2024-05-08T23:48:12.506 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:48:12.506 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:48:12.506 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:48:12.506 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:48:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 -- 172.21.15.12:0/216811810 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53600a1290 msgr2=0x7f53600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53600a1290 0x7f53600a3750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b2e0 tx=0x7f536c005960 comp rx=0 tx=0).stop 2024-05-08T23:48:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 -- 172.21.15.12:0/216811810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5378103960 msgr2=0x7f53781a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5378103960 0x7f53781a3640 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f536800e9a0 tx=0x7f536800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 -- 172.21.15.12:0/216811810 shutdown_connections 2024-05-08T23:48:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53600a1290 0x7f53600a3750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53781062d0 0x7f537819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5378103960 0x7f53781a3640 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 --2- 172.21.15.12:0/216811810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5378105e60 0x7f53781a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 -- 172.21.15.12:0/216811810 >> 172.21.15.12:0/216811810 conn(0x7f5378076ba0 msgr2=0x7f5378100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:12.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 -- 172.21.15.12:0/216811810 shutdown_connections 2024-05-08T23:48:12.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:12.504+0000 7f537d2cf700 1 -- 172.21.15.12:0/216811810 wait complete. 2024-05-08T23:48:12.513 INFO:tasks.cephfs.filesystem:scrub status for tag:dc6bbe79-b8cc-4981-9f5a-894396866f5c - {'path': '/', 'tag': 'dc6bbe79-b8cc-4981-9f5a-894396866f5c', 'options': 'recursive,force'} 2024-05-08T23:48:12.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:12 smithi012 ceph-mon[42503]: pgmap v57: 97 pgs: 97 active+clean; 8.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 756 KiB/s wr, 185 op/s 2024-05-08T23:48:12.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:12 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:48:12.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:12 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:12.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:12 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:48:12.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:12 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:12 smithi204 ceph-mon[47373]: pgmap v57: 97 pgs: 97 active+clean; 8.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 756 KiB/s wr, 185 op/s 2024-05-08T23:48:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:12 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:48:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:12 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:12 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:48:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:12 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:12 smithi160 ceph-mon[46152]: pgmap v57: 97 pgs: 97 active+clean; 8.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 756 KiB/s wr, 185 op/s 2024-05-08T23:48:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:12 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:48:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:12 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:12 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:48:12.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:12 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:12.845 INFO:tasks.workunit.client.0.smithi012.stdout: 1 25214 19.35 MB/sec warmup 8 sec latency 59.864 ms 2024-05-08T23:48:13.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:13 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:48:13.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:13 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:48:13.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:13 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:48:13.845 INFO:tasks.workunit.client.0.smithi012.stdout: 1 28500 19.17 MB/sec warmup 9 sec latency 86.258 ms 2024-05-08T23:48:14.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:14 smithi012 ceph-mon[42503]: pgmap v58: 97 pgs: 97 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1020 KiB/s wr, 226 op/s 2024-05-08T23:48:14.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:14 smithi204 ceph-mon[47373]: pgmap v58: 97 pgs: 97 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1020 KiB/s wr, 226 op/s 2024-05-08T23:48:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:14 smithi160 ceph-mon[46152]: pgmap v58: 97 pgs: 97 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1020 KiB/s wr, 226 op/s 2024-05-08T23:48:14.845 INFO:tasks.workunit.client.0.smithi012.stdout: 1 31774 18.85 MB/sec warmup 10 sec latency 5.826 ms 2024-05-08T23:48:15.845 INFO:tasks.workunit.client.0.smithi012.stdout: 1 34884 18.69 MB/sec warmup 11 sec latency 12.450 ms 2024-05-08T23:48:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:16 smithi204 ceph-mon[47373]: pgmap v59: 97 pgs: 97 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1020 KiB/s wr, 224 op/s 2024-05-08T23:48:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:16 smithi160 ceph-mon[46152]: pgmap v59: 97 pgs: 97 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1020 KiB/s wr, 224 op/s 2024-05-08T23:48:16.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 37916 18.27 MB/sec warmup 12 sec latency 14.013 ms 2024-05-08T23:48:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:16 smithi012 ceph-mon[42503]: pgmap v59: 97 pgs: 97 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1020 KiB/s wr, 224 op/s 2024-05-08T23:48:17.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 40579 18.19 MB/sec warmup 13 sec latency 5.980 ms 2024-05-08T23:48:18.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:18 smithi204 ceph-mon[47373]: pgmap v60: 97 pgs: 97 active+clean; 18 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.7 MiB/s wr, 329 op/s 2024-05-08T23:48:18.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:18 smithi160 ceph-mon[46152]: pgmap v60: 97 pgs: 97 active+clean; 18 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.7 MiB/s wr, 329 op/s 2024-05-08T23:48:18.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 43063 17.89 MB/sec warmup 14 sec latency 131.991 ms 2024-05-08T23:48:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:18 smithi012 ceph-mon[42503]: pgmap v60: 97 pgs: 97 active+clean; 18 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.7 MiB/s wr, 329 op/s 2024-05-08T23:48:19.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 46369 17.95 MB/sec warmup 15 sec latency 21.737 ms 2024-05-08T23:48:20.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:20 smithi204 ceph-mon[47373]: pgmap v61: 97 pgs: 97 active+clean; 18 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.5 MiB/s wr, 296 op/s 2024-05-08T23:48:20.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:20 smithi160 ceph-mon[46152]: pgmap v61: 97 pgs: 97 active+clean; 18 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.5 MiB/s wr, 296 op/s 2024-05-08T23:48:20.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 49044 17.59 MB/sec warmup 16 sec latency 36.982 ms 2024-05-08T23:48:20.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:20 smithi012 ceph-mon[42503]: pgmap v61: 97 pgs: 97 active+clean; 18 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.5 MiB/s wr, 296 op/s 2024-05-08T23:48:21.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 51466 17.29 MB/sec warmup 17 sec latency 21.250 ms 2024-05-08T23:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.568+0000 7f181cfbc700 1 -- 172.21.15.12:0/2006567641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1818105e60 msgr2=0x7f1818106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.568+0000 7f181cfbc700 1 --2- 172.21.15.12:0/2006567641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1818105e60 0x7f1818106240 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1808009990 tx=0x7f1808009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.568+0000 7f181cfbc700 1 -- 172.21.15.12:0/2006567641 shutdown_connections 2024-05-08T23:48:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.568+0000 7f181cfbc700 1 --2- 172.21.15.12:0/2006567641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818100760 0x7f181810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.568+0000 7f181cfbc700 1 --2- 172.21.15.12:0/2006567641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1818105e60 0x7f1818106240 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.568+0000 7f181cfbc700 1 --2- 172.21.15.12:0/2006567641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18180ffda0 0x7f1818100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.568+0000 7f181cfbc700 1 -- 172.21.15.12:0/2006567641 >> 172.21.15.12:0/2006567641 conn(0x7f1818076f10 msgr2=0x7f1818077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.569+0000 7f181cfbc700 1 -- 172.21.15.12:0/2006567641 shutdown_connections 2024-05-08T23:48:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.569+0000 7f181cfbc700 1 -- 172.21.15.12:0/2006567641 wait complete. 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.569+0000 7f181cfbc700 1 Processor -- start 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f181cfbc700 1 -- start start 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f181cfbc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18180ffda0 0x7f1818197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f181cfbc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818100760 0x7f18181977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f181cfbc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818105e60 0x7f181819bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f181cfbc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181819c1a0 con 0x7f18180ffda0 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f181cfbc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181819c310 con 0x7f1818105e60 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f181cfbc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181819c4d0 con 0x7f1818100760 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1816d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818105e60 0x7f181819bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1815d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818100760 0x7f18181977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f181659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18180ffda0 0x7f1818197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1816d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818105e60 0x7f181819bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54434/0 (socket says 172.21.15.12:54434) 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1815d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818100760 0x7f18181977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:60960/0 (socket says 172.21.15.12:60960) 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1816d9d700 1 -- 172.21.15.12:0/752265321 learned_addr learned my addr 172.21.15.12:0/752265321 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1816d9d700 1 -- 172.21.15.12:0/752265321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818100760 msgr2=0x7f18181977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1816d9d700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818100760 0x7f18181977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1816d9d700 1 -- 172.21.15.12:0/752265321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18180ffda0 msgr2=0x7f1818197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1816d9d700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18180ffda0 0x7f1818197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1816d9d700 1 -- 172.21.15.12:0/752265321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18080095f0 con 0x7f1818105e60 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f1815d9b700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818100760 0x7f18181977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.570+0000 7f181659c700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18180ffda0 0x7f1818197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.571+0000 7f1816d9d700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818105e60 0x7f181819bb60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1810007ee0 tx=0x7f181000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.571+0000 7f18077fe700 1 -- 172.21.15.12:0/752265321 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810017070 con 0x7f1818105e60 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.571+0000 7f181cfbc700 1 -- 172.21.15.12:0/752265321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181819c7b0 con 0x7f1818105e60 2024-05-08T23:48:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.571+0000 7f18077fe700 1 -- 172.21.15.12:0/752265321 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f181000f040 con 0x7f1818105e60 2024-05-08T23:48:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.571+0000 7f18077fe700 1 -- 172.21.15.12:0/752265321 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810005580 con 0x7f1818105e60 2024-05-08T23:48:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.571+0000 7f181cfbc700 1 -- 172.21.15.12:0/752265321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18181a3ff0 con 0x7f1818105e60 2024-05-08T23:48:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.573+0000 7f18077fe700 1 -- 172.21.15.12:0/752265321 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f181000db20 con 0x7f1818105e60 2024-05-08T23:48:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.574+0000 7f18077fe700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18000a1290 0x7f18000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.574+0000 7f18077fe700 1 -- 172.21.15.12:0/752265321 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f18100cc440 con 0x7f1818105e60 2024-05-08T23:48:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.574+0000 7f181659c700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18000a1290 0x7f18000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.574+0000 7f181659c700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18000a1290 0x7f18000a3750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f180800b2e0 tx=0x7f1808005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.574+0000 7f1829176b80 1 Processor -- start 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f1829176b80 1 -- start start 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f1829176b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f1829176b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x5571797337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f1829176b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733d10 0x55717973fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f1829176b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571796860f0 con 0x557179733350 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f1829176b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179683ab0 con 0x557179737ce0 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f1829176b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717967f3a0 con 0x557179733d10 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f181659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f1815d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x5571797337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f181659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797380c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54448/0 (socket says 172.21.15.12:54448) 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f181659c700 1 -- 172.21.15.12:0/4145193649 learned_addr learned my addr 172.21.15.12:0/4145193649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.575+0000 7f1816d9d700 1 --2- 172.21.15.12:0/4145193649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733d10 0x55717973fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.576+0000 7f181659c700 1 -- 172.21.15.12:0/4145193649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733d10 msgr2=0x55717973fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.576+0000 7f181659c700 1 --2- 172.21.15.12:0/4145193649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733d10 0x55717973fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.576+0000 7f181659c700 1 -- 172.21.15.12:0/4145193649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 msgr2=0x5571797337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.576+0000 7f181659c700 1 --2- 172.21.15.12:0/4145193649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x5571797337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.576+0000 7f181659c700 1 -- 172.21.15.12:0/4145193649 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557179500350 con 0x557179737ce0 2024-05-08T23:48:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.576+0000 7f1815d9b700 1 --2- 172.21.15.12:0/4145193649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x5571797337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.576+0000 7f181659c700 1 --2- 172.21.15.12:0/4145193649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797380c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1808009990 tx=0x7f1808020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d95a33dffcc46b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.576+0000 7f1804ff9700 1 -- 172.21.15.12:0/4145193649 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180802b070 con 0x557179737ce0 2024-05-08T23:48:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.576+0000 7f1804ff9700 1 -- 172.21.15.12:0/4145193649 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f180801e040 con 0x557179737ce0 2024-05-08T23:48:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.576+0000 7f1804ff9700 1 -- 172.21.15.12:0/4145193649 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808024e80 con 0x557179737ce0 2024-05-08T23:48:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.577+0000 7f1829176b80 1 -- 172.21.15.12:0/4145193649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 msgr2=0x5571797380c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.577+0000 7f1829176b80 1 --2- 172.21.15.12:0/4145193649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797380c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1808009990 tx=0x7f1808020000 comp rx=0 tx=0).stop 2024-05-08T23:48:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.577+0000 7f1829176b80 1 -- 172.21.15.12:0/4145193649 shutdown_connections 2024-05-08T23:48:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.577+0000 7f1829176b80 1 --2- 172.21.15.12:0/4145193649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733d10 0x55717973fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.577+0000 7f1829176b80 1 --2- 172.21.15.12:0/4145193649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797380c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.577+0000 7f1829176b80 1 --2- 172.21.15.12:0/4145193649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x5571797337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.577+0000 7f1829176b80 1 -- 172.21.15.12:0/4145193649 >> 172.21.15.12:0/4145193649 conn(0x55717968d1b0 msgr2=0x557179729840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.577+0000 7f1829176b80 1 -- 172.21.15.12:0/4145193649 shutdown_connections 2024-05-08T23:48:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.577+0000 7f1829176b80 1 -- 172.21.15.12:0/4145193649 wait complete. 2024-05-08T23:48:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1829176b80 1 -- start start 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1829176b80 1 Processor -- start 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1829176b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 0x55717973f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1829176b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x55717973cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1829176b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x55717973d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1829176b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717961b500 con 0x557179737ce0 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1829176b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717961b1a0 con 0x557179733d10 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1829176b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179502640 con 0x557179733350 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1829176b80 10 client.0 _ll_register_callbacks cb 0x557179624ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1816d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x55717973d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f181659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 0x55717973f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1815d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x55717973cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1816d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x55717973d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49374/0 (socket says 172.21.15.12:49374) 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f181659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 0x55717973f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:60978/0 (socket says 172.21.15.12:60978) 2024-05-08T23:48:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1816d9d700 1 -- 172.21.15.12:0/3106623961 learned_addr learned my addr 172.21.15.12:0/3106623961 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.578+0000 7f1815d9b700 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x55717973cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54452/0 (socket says 172.21.15.12:54452) 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f1815d9b700 1 -- 172.21.15.12:0/3106623961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 msgr2=0x55717973f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f1815d9b700 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 0x55717973f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f1815d9b700 1 -- 172.21.15.12:0/3106623961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 msgr2=0x55717973d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f1815d9b700 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x55717973d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f1815d9b700 1 -- 172.21.15.12:0/3106623961 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c009c90 con 0x557179733d10 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f181659c700 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 0x55717973f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f1816d9d700 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x55717973d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f1815d9b700 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x55717973cd10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f18181987d0 tx=0x7f180c00f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73d28929b70de331 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f17fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f17fdffb700 1 -- 172.21.15.12:0/3106623961 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c017070 con 0x557179733d10 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f17fdffb700 1 -- 172.21.15.12:0/3106623961 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f180c00f960 con 0x557179733d10 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f1829176b80 1 -- 172.21.15.12:0/3106623961 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571795d8210 con 0x557179733d10 2024-05-08T23:48:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f17fdffb700 1 -- 172.21.15.12:0/3106623961 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f180c00fb20 con 0x557179733d10 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f1829176b80 10 client.25796 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f1829176b80 1 -- 172.21.15.12:0/3106623961 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f181810d4f0 con 0x557179733d10 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.579+0000 7f17fdffb700 1 -- 172.21.15.12:0/3106623961 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571795d8210 con 0x557179733d10 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.580+0000 7f17fdffb700 1 -- 172.21.15.12:0/3106623961 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f180c003a60 con 0x557179733d10 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.580+0000 7f1829176b80 10 client.25796 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.580+0000 7f1829176b80 10 client.25796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.580+0000 7f1829176b80 20 client.25796 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.580+0000 7f1829176b80 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55717974d040 0x55717974f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.580+0000 7f1829176b80 4 client.25796 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.580+0000 7f1829176b80 1 -- 172.21.15.12:0/3106623961 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571796fe7b0 con 0x55717974d040 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.580+0000 7f181659c700 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55717974d040 0x55717974f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.580+0000 7f181659c700 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55717974d040 0x55717974f460 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6676a2e353a6eff6 server_cookie=91bd9e06a8af626a in_seq=0 out_seq=0 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.580+0000 7f17fdffb700 10 client.25796 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f17fdffb700 1 -- 172.21.15.12:0/3106623961 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5571796fe7b0 con 0x55717974d040 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f17fdffb700 10 client.25796 handle_command_reply: tid=0 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 client.25796 shutdown 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 -- 172.21.15.12:0/3106623961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 msgr2=0x55717973cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x55717973cd10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f18181987d0 tx=0x7f180c00f3c0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 -- 172.21.15.12:0/3106623961 shutdown_connections 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x55717973d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 0x55717973f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55717974d040 0x55717974f460 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 --2- 172.21.15.12:0/3106623961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x55717973cd10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f17fdffb700 0 client.25796 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 -- 172.21.15.12:0/3106623961 >> 172.21.15.12:0/3106623961 conn(0x55717968d1b0 msgr2=0x55717972df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 -- 172.21.15.12:0/3106623961 shutdown_connections 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 1 -- 172.21.15.12:0/3106623961 wait complete. 2024-05-08T23:48:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.582+0000 7f1829176b80 20 client.25796 trim_cache size 0 max 16384 2024-05-08T23:48:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1829176b80 1 Processor -- start 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1829176b80 1 -- start start 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1829176b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 0x55717973f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1829176b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x557179743f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1829176b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x5571797444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1829176b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179754650 con 0x557179737ce0 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1829176b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179736b20 con 0x557179733d10 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1829176b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717973dc00 con 0x557179733350 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1816d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x5571797444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1816d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x5571797444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49388/0 (socket says 172.21.15.12:49388) 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1816d9d700 1 -- 172.21.15.12:0/1933873853 learned_addr learned my addr 172.21.15.12:0/1933873853 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f1815d9b700 1 --2- 172.21.15.12:0/1933873853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x557179743f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.599+0000 7f181659c700 1 --2- 172.21.15.12:0/1933873853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 0x55717973f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.600+0000 7f1816d9d700 1 -- 172.21.15.12:0/1933873853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 msgr2=0x55717973f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.600+0000 7f1816d9d700 1 --2- 172.21.15.12:0/1933873853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 0x55717973f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.600+0000 7f1816d9d700 1 -- 172.21.15.12:0/1933873853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 msgr2=0x557179743f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.600+0000 7f1816d9d700 1 --2- 172.21.15.12:0/1933873853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x557179743f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.600+0000 7f1816d9d700 1 -- 172.21.15.12:0/1933873853 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571794acb00 con 0x557179737ce0 2024-05-08T23:48:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.600+0000 7f1815d9b700 1 --2- 172.21.15.12:0/1933873853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x557179743f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.600+0000 7f1816d9d700 1 --2- 172.21.15.12:0/1933873853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x5571797444d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f18100c9b40 tx=0x7f1810084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502b584e5ab90b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.600+0000 7f18057fa700 1 -- 172.21.15.12:0/1933873853 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181009d3c0 con 0x557179737ce0 2024-05-08T23:48:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.600+0000 7f18057fa700 1 -- 172.21.15.12:0/1933873853 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18100a1a10 con 0x557179737ce0 2024-05-08T23:48:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.600+0000 7f18057fa700 1 -- 172.21.15.12:0/1933873853 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810022250 con 0x557179737ce0 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 -- 172.21.15.12:0/1933873853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 msgr2=0x5571797444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 --2- 172.21.15.12:0/1933873853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x5571797444d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f18100c9b40 tx=0x7f1810084040 comp rx=0 tx=0).stop 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 -- 172.21.15.12:0/1933873853 shutdown_connections 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 --2- 172.21.15.12:0/1933873853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179737ce0 0x5571797444d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 --2- 172.21.15.12:0/1933873853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733350 0x55717973f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 --2- 172.21.15.12:0/1933873853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179733d10 0x557179743f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 -- 172.21.15.12:0/1933873853 >> 172.21.15.12:0/1933873853 conn(0x55717968d1b0 msgr2=0x557179737010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 -- 172.21.15.12:0/1933873853 shutdown_connections 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 -- 172.21.15.12:0/1933873853 wait complete. 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 -- start start 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 Processor -- start 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.601+0000 7f1829176b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x557179741960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1829176b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733d10 0x557179741ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1829176b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1829176b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571797448b0 con 0x557179733350 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1829176b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571797a2540 con 0x557179737ce0 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1829176b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571797a26b0 con 0x557179733d10 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1829176b80 10 client.0 _ll_register_callbacks cb 0x557179624ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f181659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x557179741960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f181659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x557179741960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49400/0 (socket says 172.21.15.12:49400) 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1815d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733d10 0x557179741ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1816d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f181659c700 1 -- 172.21.15.12:0/2629727104 learned_addr learned my addr 172.21.15.12:0/2629727104 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1816d9d700 1 -- 172.21.15.12:0/2629727104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733d10 msgr2=0x557179741ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1816d9d700 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733d10 0x557179741ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1816d9d700 1 -- 172.21.15.12:0/2629727104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 msgr2=0x557179741960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1816d9d700 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x557179741960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1816d9d700 1 -- 172.21.15.12:0/2629727104 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181810d4f0 con 0x557179737ce0 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f181659c700 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x557179741960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1816d9d700 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797423e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18100c9c80 tx=0x7f1810005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc82523f2dc51369 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f17fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f17fdffb700 1 -- 172.21.15.12:0/2629727104 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100976d0 con 0x557179737ce0 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f17fdffb700 1 -- 172.21.15.12:0/2629727104 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f181009d3c0 con 0x557179737ce0 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.602+0000 7f1829176b80 1 -- 172.21.15.12:0/2629727104 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f180801e040 con 0x557179737ce0 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f17fdffb700 1 -- 172.21.15.12:0/2629727104 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1810022250 con 0x557179737ce0 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f1829176b80 10 client.25802 fetch_fsmap learned FSMap version 26 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f1829176b80 1 -- 172.21.15.12:0/2629727104 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f18181a3ff0 con 0x557179737ce0 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f17fdffb700 1 -- 172.21.15.12:0/2629727104 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f180801e040 con 0x557179737ce0 2024-05-08T23:48:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f17fdffb700 1 -- 172.21.15.12:0/2629727104 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1810098b10 con 0x557179737ce0 2024-05-08T23:48:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f1829176b80 10 client.25802 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:48:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f1829176b80 10 client.25802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f1829176b80 20 client.25802 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f1829176b80 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571797a3ad0 0x5571797a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f1829176b80 4 client.25802 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:48:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f1829176b80 1 -- 172.21.15.12:0/2629727104 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557179677e60 con 0x5571797a3ad0 2024-05-08T23:48:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.603+0000 7f181659c700 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571797a3ad0 0x5571797a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.604+0000 7f181659c700 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571797a3ad0 0x5571797a5ec0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d767a50acd6dc39f server_cookie=43d2be8ef26f8e81 in_seq=0 out_seq=0 2024-05-08T23:48:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.604+0000 7f17fdffb700 10 client.25802 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f17fdffb700 1 -- 172.21.15.12:0/2629727104 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557179677e60 con 0x5571797a3ad0 2024-05-08T23:48:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f17fdffb700 10 client.25802 handle_command_reply: tid=0 2024-05-08T23:48:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 client.25802 shutdown 2024-05-08T23:48:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 -- 172.21.15.12:0/2629727104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 msgr2=0x5571797423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797423e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18100c9c80 tx=0x7f1810005020 comp rx=0 tx=0).stop 2024-05-08T23:48:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 -- 172.21.15.12:0/2629727104 shutdown_connections 2024-05-08T23:48:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571797a3ad0 0x5571797a5ec0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557179737ce0 0x5571797423e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557179733350 0x557179741960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f17fdffb700 0 client.25802 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 --2- 172.21.15.12:0/2629727104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557179733d10 0x557179741ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 -- 172.21.15.12:0/2629727104 >> 172.21.15.12:0/2629727104 conn(0x55717968d1b0 msgr2=0x557179748a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 -- 172.21.15.12:0/2629727104 shutdown_connections 2024-05-08T23:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 1 -- 172.21.15.12:0/2629727104 wait complete. 2024-05-08T23:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.729+0000 7f1829176b80 20 client.25802 trim_cache size 0 max 16384 2024-05-08T23:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:48:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:48:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 -- 172.21.15.12:0/752265321 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18000a1290 msgr2=0x7f18000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18000a1290 0x7f18000a3750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f180800b2e0 tx=0x7f1808005960 comp rx=0 tx=0).stop 2024-05-08T23:48:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 -- 172.21.15.12:0/752265321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818105e60 msgr2=0x7f181819bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818105e60 0x7f181819bb60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1810007ee0 tx=0x7f181000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 -- 172.21.15.12:0/752265321 shutdown_connections 2024-05-08T23:48:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18000a1290 0x7f18000a3750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818105e60 0x7f181819bb60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18180ffda0 0x7f1818197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 --2- 172.21.15.12:0/752265321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818100760 0x7f18181977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 -- 172.21.15.12:0/752265321 >> 172.21.15.12:0/752265321 conn(0x7f1818076f10 msgr2=0x7f18180ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 -- 172.21.15.12:0/752265321 shutdown_connections 2024-05-08T23:48:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:22.730+0000 7f181cfbc700 1 -- 172.21.15.12:0/752265321 wait complete. 2024-05-08T23:48:22.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:22 smithi204 ceph-mon[47373]: pgmap v62: 97 pgs: 97 active+clean; 25 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 384 op/s 2024-05-08T23:48:22.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:22 smithi160 ceph-mon[46152]: pgmap v62: 97 pgs: 97 active+clean; 25 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 384 op/s 2024-05-08T23:48:22.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 53931 17.11 MB/sec warmup 18 sec latency 70.519 ms 2024-05-08T23:48:22.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:22 smithi012 ceph-mon[42503]: pgmap v62: 97 pgs: 97 active+clean; 25 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 384 op/s 2024-05-08T23:48:23.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 56444 16.90 MB/sec warmup 19 sec latency 144.464 ms 2024-05-08T23:48:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:24 smithi204 ceph-mon[47373]: pgmap v63: 97 pgs: 97 active+clean; 25 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2024-05-08T23:48:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:24 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:48:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:24 smithi160 ceph-mon[46152]: pgmap v63: 97 pgs: 97 active+clean; 25 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2024-05-08T23:48:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:24 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:48:24.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 56444 16.06 MB/sec warmup 20 sec latency 1144.505 ms 2024-05-08T23:48:24.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:24 smithi012 ceph-mon[42503]: pgmap v63: 97 pgs: 97 active+clean; 25 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2024-05-08T23:48:24.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:24 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:48:25.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 56444 15.29 MB/sec warmup 21 sec latency 2144.614 ms 2024-05-08T23:48:26.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:26 smithi204 ceph-mon[47373]: pgmap v64: 97 pgs: 97 active+clean; 25 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-05-08T23:48:26.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:26 smithi160 ceph-mon[46152]: pgmap v64: 97 pgs: 97 active+clean; 25 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-05-08T23:48:26.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 56444 14.60 MB/sec warmup 22 sec latency 3144.716 ms 2024-05-08T23:48:26.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:26 smithi012 ceph-mon[42503]: pgmap v64: 97 pgs: 97 active+clean; 25 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-05-08T23:48:27.846 INFO:tasks.workunit.client.0.smithi012.stdout: 1 56746 14.14 MB/sec warmup 23 sec latency 4061.468 ms 2024-05-08T23:48:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:28 smithi204 ceph-mon[47373]: pgmap v65: 97 pgs: 97 active+clean; 16 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.8 MiB/s wr, 366 op/s 2024-05-08T23:48:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:28 smithi160 ceph-mon[46152]: pgmap v65: 97 pgs: 97 active+clean; 16 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.8 MiB/s wr, 366 op/s 2024-05-08T23:48:28.847 INFO:tasks.workunit.client.0.smithi012.stdout: 1 57108 13.61 MB/sec warmup 24 sec latency 696.776 ms 2024-05-08T23:48:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:28 smithi012 ceph-mon[42503]: pgmap v65: 97 pgs: 97 active+clean; 16 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.8 MiB/s wr, 366 op/s 2024-05-08T23:48:29.847 INFO:tasks.workunit.client.0.smithi012.stdout: 1 57108 13.06 MB/sec warmup 25 sec latency 1696.881 ms 2024-05-08T23:48:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:30 smithi204 ceph-mon[47373]: pgmap v66: 97 pgs: 97 active+clean; 16 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-05-08T23:48:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:30 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:48:30.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:30 smithi160 ceph-mon[46152]: pgmap v66: 97 pgs: 97 active+clean; 16 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-05-08T23:48:30.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:30 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:48:30.847 INFO:tasks.workunit.client.0.smithi012.stdout: 1 57108 12.56 MB/sec warmup 26 sec latency 2696.944 ms 2024-05-08T23:48:30.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:30 smithi012 ceph-mon[42503]: pgmap v66: 97 pgs: 97 active+clean; 16 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-05-08T23:48:30.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:30 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:48:31.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:31 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:48:31.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:31 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:48:31.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:31 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:48:31.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:31 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:48:31.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:31 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:48:31.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:31 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:48:31.847 INFO:tasks.workunit.client.0.smithi012.stdout: 1 57108 12.10 MB/sec warmup 27 sec latency 3697.020 ms 2024-05-08T23:48:31.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:31 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:48:31.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:31 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:48:31.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:31 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:48:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:32 smithi204 ceph-mon[47373]: pgmap v67: 97 pgs: 97 active+clean; 16 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.1 MiB/s wr, 274 op/s 2024-05-08T23:48:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:32 smithi160 ceph-mon[46152]: pgmap v67: 97 pgs: 97 active+clean; 16 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.1 MiB/s wr, 274 op/s 2024-05-08T23:48:32.847 INFO:tasks.workunit.client.0.smithi012.stdout: 1 57168 11.68 MB/sec warmup 28 sec latency 4600.931 ms 2024-05-08T23:48:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:32 smithi012 ceph-mon[42503]: pgmap v67: 97 pgs: 97 active+clean; 16 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.1 MiB/s wr, 274 op/s 2024-05-08T23:48:33.847 INFO:tasks.workunit.client.0.smithi012.stdout: 1 57168 11.27 MB/sec warmup 29 sec latency 1037.058 ms 2024-05-08T23:48:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:34 smithi204 ceph-mon[47373]: pgmap v68: 97 pgs: 97 active+clean; 16 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 454 KiB/s wr, 186 op/s 2024-05-08T23:48:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:34 smithi160 ceph-mon[46152]: pgmap v68: 97 pgs: 97 active+clean; 16 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 454 KiB/s wr, 186 op/s 2024-05-08T23:48:34.847 INFO:tasks.workunit.client.0.smithi012.stdout: 1 57168 10.90 MB/sec warmup 30 sec latency 2037.157 ms 2024-05-08T23:48:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:34 smithi012 ceph-mon[42503]: pgmap v68: 97 pgs: 97 active+clean; 16 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 454 KiB/s wr, 186 op/s 2024-05-08T23:48:35.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 57168 10.55 MB/sec warmup 31 sec latency 3037.611 ms 2024-05-08T23:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.905+0000 7f09134e6700 1 -- 172.21.15.12:0/2984964225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f090c101110 msgr2=0x7f090c1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.905+0000 7f09134e6700 1 --2- 172.21.15.12:0/2984964225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f090c101110 0x7f090c1014f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0908009990 tx=0x7f0908009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.906+0000 7f09134e6700 1 -- 172.21.15.12:0/2984964225 shutdown_connections 2024-05-08T23:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.906+0000 7f09134e6700 1 --2- 172.21.15.12:0/2984964225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c106230 0x7f090c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.906+0000 7f09134e6700 1 --2- 172.21.15.12:0/2984964225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f090c101110 0x7f090c1014f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.906+0000 7f09134e6700 1 --2- 172.21.15.12:0/2984964225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f090c101ac0 0x7f090c105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.906+0000 7f09134e6700 1 -- 172.21.15.12:0/2984964225 >> 172.21.15.12:0/2984964225 conn(0x7f090c0fc9b0 msgr2=0x7f090c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.907+0000 7f09134e6700 1 -- 172.21.15.12:0/2984964225 shutdown_connections 2024-05-08T23:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.907+0000 7f09134e6700 1 -- 172.21.15.12:0/2984964225 wait complete. 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.908+0000 7f09134e6700 1 Processor -- start 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.908+0000 7f09134e6700 1 -- start start 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.908+0000 7f09134e6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f090c101110 0x7f090c10e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.908+0000 7f09134e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f090c106230 0x7f090c10e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.908+0000 7f09134e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c10edf0 0x7f090c071700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.908+0000 7f09134e6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c112570 con 0x7f090c101110 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.908+0000 7f09134e6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c071c40 con 0x7f090c106230 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.908+0000 7f09134e6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c071e00 con 0x7f090c10edf0 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.909+0000 7f0911282700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f090c101110 0x7f090c10e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.909+0000 7f0911282700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f090c101110 0x7f090c10e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42788/0 (socket says 172.21.15.12:42788) 2024-05-08T23:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.909+0000 7f0911282700 1 -- 172.21.15.12:0/2992084996 learned_addr learned my addr 172.21.15.12:0/2992084996 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.910+0000 7f0911a83700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c10edf0 0x7f090c071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.911+0000 7f0910a81700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f090c106230 0x7f090c10e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.911+0000 7f0911a83700 1 -- 172.21.15.12:0/2992084996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f090c106230 msgr2=0x7f090c10e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.911+0000 7f0911a83700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f090c106230 0x7f090c10e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.911+0000 7f0911a83700 1 -- 172.21.15.12:0/2992084996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f090c101110 msgr2=0x7f090c10e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.911+0000 7f0911a83700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f090c101110 0x7f090c10e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.911+0000 7f0911a83700 1 -- 172.21.15.12:0/2992084996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09080095f0 con 0x7f090c10edf0 2024-05-08T23:48:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.916+0000 7f0911a83700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c10edf0 0x7f090c071700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f090400bb00 tx=0x7f090400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.923+0000 7f09027fc700 1 -- 172.21.15.12:0/2992084996 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090400ca30 con 0x7f090c10edf0 2024-05-08T23:48:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.923+0000 7f09027fc700 1 -- 172.21.15.12:0/2992084996 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f090400cb90 con 0x7f090c10edf0 2024-05-08T23:48:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.923+0000 7f09027fc700 1 -- 172.21.15.12:0/2992084996 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09040125a0 con 0x7f090c10edf0 2024-05-08T23:48:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.924+0000 7f09134e6700 1 -- 172.21.15.12:0/2992084996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c0720e0 con 0x7f090c10edf0 2024-05-08T23:48:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.924+0000 7f09134e6700 1 -- 172.21.15.12:0/2992084996 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c0725a0 con 0x7f090c10edf0 2024-05-08T23:48:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.931+0000 7f091f6a0b80 1 Processor -- start 2024-05-08T23:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.931+0000 7f091f6a0b80 1 -- start start 2024-05-08T23:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.931+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 0x557277649d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.931+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 0x557277643f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.931+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x5572776448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.931+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55727759b0f0 con 0x557277644480 2024-05-08T23:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.931+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557277598ab0 con 0x55727764a270 2024-05-08T23:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.931+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572775943a0 con 0x557277649950 2024-05-08T23:48:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.932+0000 7f0910a81700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x5572776448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.932+0000 7f0910a81700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x5572776448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42790/0 (socket says 172.21.15.12:42790) 2024-05-08T23:48:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.932+0000 7f0910a81700 1 -- 172.21.15.12:0/3644017013 learned_addr learned my addr 172.21.15.12:0/3644017013 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.932+0000 7f0911282700 1 --2- 172.21.15.12:0/3644017013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 0x557277643f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.932+0000 7f0910a81700 1 -- 172.21.15.12:0/3644017013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 msgr2=0x557277649d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.932+0000 7f0910a81700 1 --2- 172.21.15.12:0/3644017013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 0x557277649d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.932+0000 7f0910a81700 1 -- 172.21.15.12:0/3644017013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 msgr2=0x557277643f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.932+0000 7f0910a81700 1 --2- 172.21.15.12:0/3644017013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 0x557277643f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.932+0000 7f0910a81700 1 -- 172.21.15.12:0/3644017013 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557277415350 con 0x557277644480 2024-05-08T23:48:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.933+0000 7f0910a81700 1 --2- 172.21.15.12:0/3644017013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x5572776448a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00ccf0 tx=0x7f08fc0113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b534d45f69f5092b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.936+0000 7f08f77fe700 1 -- 172.21.15.12:0/3644017013 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc011d10 con 0x557277644480 2024-05-08T23:48:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.936+0000 7f08f77fe700 1 -- 172.21.15.12:0/3644017013 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08fc011e70 con 0x557277644480 2024-05-08T23:48:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.936+0000 7f08f77fe700 1 -- 172.21.15.12:0/3644017013 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc012450 con 0x557277644480 2024-05-08T23:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.936+0000 7f09027fc700 1 -- 172.21.15.12:0/2992084996 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f090400cd00 con 0x7f090c10edf0 2024-05-08T23:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.937+0000 7f09027fc700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08f80a1420 0x7f08f80a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.937+0000 7f09027fc700 1 -- 172.21.15.12:0/2992084996 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f09040cc110 con 0x7f090c10edf0 2024-05-08T23:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.937+0000 7f0911282700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08f80a1420 0x7f08f80a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.938+0000 7f091f6a0b80 1 -- 172.21.15.12:0/3644017013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 msgr2=0x5572776448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.938+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/3644017013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x5572776448a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00ccf0 tx=0x7f08fc0113e0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.938+0000 7f091f6a0b80 1 -- 172.21.15.12:0/3644017013 shutdown_connections 2024-05-08T23:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.938+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/3644017013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x5572776448a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.938+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/3644017013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 0x557277649d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.938+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/3644017013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 0x557277643f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.938+0000 7f091f6a0b80 1 -- 172.21.15.12:0/3644017013 >> 172.21.15.12:0/3644017013 conn(0x5572775a21b0 msgr2=0x55727763e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.938+0000 7f0911282700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08f80a1420 0x7f08f80a38e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0908003800 tx=0x7f090801a040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.938+0000 7f091f6a0b80 1 -- 172.21.15.12:0/3644017013 shutdown_connections 2024-05-08T23:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.938+0000 7f091f6a0b80 1 -- 172.21.15.12:0/3644017013 wait complete. 2024-05-08T23:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.939+0000 7f091f6a0b80 1 -- start start 2024-05-08T23:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.939+0000 7f091f6a0b80 1 Processor -- start 2024-05-08T23:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.940+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557277644480 0x55727764d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.940+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 0x55727764db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.940+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55727764a270 0x55727764e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.940+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557277530500 con 0x55727764a270 2024-05-08T23:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.940+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572775301a0 con 0x557277644480 2024-05-08T23:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.940+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557277417640 con 0x557277649950 2024-05-08T23:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.940+0000 7f091f6a0b80 10 client.0 _ll_register_callbacks cb 0x557277539ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.941+0000 7f0911a83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55727764a270 0x55727764e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.941+0000 7f0911a83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55727764a270 0x55727764e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42796/0 (socket says 172.21.15.12:42796) 2024-05-08T23:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.941+0000 7f0911a83700 1 -- 172.21.15.12:0/2463603997 learned_addr learned my addr 172.21.15.12:0/2463603997 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.942+0000 7f0911282700 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557277644480 0x55727764d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.942+0000 7f0911a83700 1 -- 172.21.15.12:0/2463603997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 msgr2=0x55727764db30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:48:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.942+0000 7f0911a83700 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 0x55727764db30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.942+0000 7f0911a83700 1 -- 172.21.15.12:0/2463603997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557277644480 msgr2=0x55727764d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.942+0000 7f0911a83700 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557277644480 0x55727764d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.943+0000 7f0911a83700 1 -- 172.21.15.12:0/2463603997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc00c9d0 con 0x55727764a270 2024-05-08T23:48:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.946+0000 7f0911a83700 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55727764a270 0x55727764e070 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f090400c530 tx=0x7f0904099b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b492a011ba4e419e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.946+0000 7f08f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.951+0000 7f08f4ff9700 1 -- 172.21.15.12:0/2463603997 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09040a1790 con 0x55727764a270 2024-05-08T23:48:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.951+0000 7f08f4ff9700 1 -- 172.21.15.12:0/2463603997 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0904098800 con 0x55727764a270 2024-05-08T23:48:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.951+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2463603997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572774ed210 con 0x55727764a270 2024-05-08T23:48:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.951+0000 7f08f4ff9700 1 -- 172.21.15.12:0/2463603997 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09040989c0 con 0x55727764a270 2024-05-08T23:48:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.956+0000 7f091f6a0b80 10 client.15969 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.956+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2463603997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5572773c1b00 con 0x55727764a270 2024-05-08T23:48:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.957+0000 7f08f4ff9700 1 -- 172.21.15.12:0/2463603997 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09040a5080 con 0x55727764a270 2024-05-08T23:48:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.957+0000 7f08f4ff9700 1 -- 172.21.15.12:0/2463603997 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f090409e060 con 0x55727764a270 2024-05-08T23:48:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.960+0000 7f091f6a0b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.960+0000 7f091f6a0b80 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.960+0000 7f091f6a0b80 20 client.15969 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.960+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55727766ed80 0x5572776711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.960+0000 7f091f6a0b80 4 client.15969 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.960+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2463603997 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572776137b0 con 0x55727766ed80 2024-05-08T23:48:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.960+0000 7f0911282700 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55727766ed80 0x5572776711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.963+0000 7f0911282700 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55727766ed80 0x5572776711a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d5366aca7b8883 server_cookie=418b0b3018ec15b4 in_seq=0 out_seq=0 2024-05-08T23:48:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.963+0000 7f08f4ff9700 10 client.15969 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.966+0000 7f08f4ff9700 1 -- 172.21.15.12:0/2463603997 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5572776137b0 con 0x55727766ed80 2024-05-08T23:48:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.966+0000 7f08f4ff9700 10 client.15969 handle_command_reply: tid=0 2024-05-08T23:48:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.966+0000 7f091f6a0b80 1 client.15969 shutdown 2024-05-08T23:48:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.966+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2463603997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55727764a270 msgr2=0x55727764e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.966+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55727764a270 0x55727764e070 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f090400c530 tx=0x7f0904099b00 comp rx=0 tx=0).stop 2024-05-08T23:48:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.967+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2463603997 shutdown_connections 2024-05-08T23:48:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.967+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55727764a270 0x55727764e070 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.967+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55727766ed80 0x5572776711a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.967+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557277644480 0x55727764d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.967+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/2463603997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 0x55727764db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.967+0000 7f08f4ff9700 0 client.15969 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.967+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2463603997 >> 172.21.15.12:0/2463603997 conn(0x5572775a21b0 msgr2=0x5572776484a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.968+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2463603997 shutdown_connections 2024-05-08T23:48:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.968+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2463603997 wait complete. 2024-05-08T23:48:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:35.970+0000 7f091f6a0b80 20 client.15969 trim_cache size 0 max 16384 2024-05-08T23:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.007+0000 7f091f6a0b80 1 Processor -- start 2024-05-08T23:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.008+0000 7f091f6a0b80 1 -- start start 2024-05-08T23:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.008+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277644480 0x557277644860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.008+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277649950 0x55727763e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.008+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 0x557277643cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.008+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557277676390 con 0x557277649950 2024-05-08T23:48:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.008+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55727766c380 con 0x55727764a270 2024-05-08T23:48:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.008+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55727764bb40 con 0x557277644480 2024-05-08T23:48:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.009+0000 7f0910a81700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277649950 0x55727763e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.009+0000 7f0910a81700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277649950 0x55727763e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42810/0 (socket says 172.21.15.12:42810) 2024-05-08T23:48:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.009+0000 7f0910a81700 1 -- 172.21.15.12:0/2964654312 learned_addr learned my addr 172.21.15.12:0/2964654312 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.010+0000 7f0910a81700 1 -- 172.21.15.12:0/2964654312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277644480 msgr2=0x557277644860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:48:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.010+0000 7f0910a81700 1 --2- 172.21.15.12:0/2964654312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277644480 0x557277644860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.010+0000 7f0910a81700 1 -- 172.21.15.12:0/2964654312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 msgr2=0x557277643cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.010+0000 7f0910a81700 1 --2- 172.21.15.12:0/2964654312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 0x557277643cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.010+0000 7f0910a81700 1 -- 172.21.15.12:0/2964654312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55727755a500 con 0x557277649950 2024-05-08T23:48:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.010+0000 7f0910a81700 1 --2- 172.21.15.12:0/2964654312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277649950 0x55727763e910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0117e0 tx=0x7f08fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd2bf2556bdce5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.011+0000 7f08f7fff700 1 -- 172.21.15.12:0/2964654312 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc011d10 con 0x557277649950 2024-05-08T23:48:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.011+0000 7f08f7fff700 1 -- 172.21.15.12:0/2964654312 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08fc005500 con 0x557277649950 2024-05-08T23:48:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.011+0000 7f08f7fff700 1 -- 172.21.15.12:0/2964654312 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc012450 con 0x557277649950 2024-05-08T23:48:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.012+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2964654312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277649950 msgr2=0x55727763e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.012+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/2964654312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277649950 0x55727763e910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0117e0 tx=0x7f08fc00c2d0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.012+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2964654312 shutdown_connections 2024-05-08T23:48:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.012+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/2964654312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 0x557277643cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.012+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/2964654312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277644480 0x557277644860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.012+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/2964654312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277649950 0x55727763e910 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.012+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2964654312 >> 172.21.15.12:0/2964654312 conn(0x5572775a21b0 msgr2=0x5572776435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.013+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2964654312 shutdown_connections 2024-05-08T23:48:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.013+0000 7f091f6a0b80 1 -- 172.21.15.12:0/2964654312 wait complete. 2024-05-08T23:48:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.014+0000 7f091f6a0b80 1 -- start start 2024-05-08T23:48:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.014+0000 7f091f6a0b80 1 Processor -- start 2024-05-08T23:48:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.015+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x557277661c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.015+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 0x557277662160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.015+0000 7f091f6a0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 0x5572776c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.015+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557277662aa0 con 0x557277644480 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.015+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572776c4790 con 0x55727764a270 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.015+0000 7f091f6a0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572776c48d0 con 0x557277649950 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.015+0000 7f091f6a0b80 10 client.0 _ll_register_callbacks cb 0x557277539ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.015+0000 7f0911282700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x557277661c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.015+0000 7f0911282700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x557277661c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42824/0 (socket says 172.21.15.12:42824) 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.015+0000 7f0911282700 1 -- 172.21.15.12:0/652826482 learned_addr learned my addr 172.21.15.12:0/652826482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.016+0000 7f0911282700 1 -- 172.21.15.12:0/652826482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 msgr2=0x557277662160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.016+0000 7f0911282700 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 0x557277662160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.016+0000 7f0911282700 1 -- 172.21.15.12:0/652826482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 msgr2=0x5572776c4250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.016+0000 7f0911282700 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 0x5572776c4250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.016+0000 7f0911282700 1 -- 172.21.15.12:0/652826482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557277415350 con 0x557277644480 2024-05-08T23:48:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.016+0000 7f0911282700 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x557277661c20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f09080098e0 tx=0x7f0908009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c04b05d1fa169f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.016+0000 7f08f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.016+0000 7f08f4ff9700 1 -- 172.21.15.12:0/652826482 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090801d040 con 0x557277644480 2024-05-08T23:48:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.017+0000 7f08f4ff9700 1 -- 172.21.15.12:0/652826482 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f090805b070 con 0x557277644480 2024-05-08T23:48:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.017+0000 7f091f6a0b80 1 -- 172.21.15.12:0/652826482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08fc011e70 con 0x557277644480 2024-05-08T23:48:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.017+0000 7f08f4ff9700 1 -- 172.21.15.12:0/652826482 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0908025e40 con 0x557277644480 2024-05-08T23:48:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.017+0000 7f091f6a0b80 10 client.15975 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.017+0000 7f091f6a0b80 1 -- 172.21.15.12:0/652826482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557277419c70 con 0x557277644480 2024-05-08T23:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.018+0000 7f08f4ff9700 1 -- 172.21.15.12:0/652826482 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08fc011e70 con 0x557277644480 2024-05-08T23:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.018+0000 7f08f4ff9700 1 -- 172.21.15.12:0/652826482 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f090800f840 con 0x557277644480 2024-05-08T23:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.018+0000 7f091f6a0b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.018+0000 7f091f6a0b80 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.018+0000 7f091f6a0b80 20 client.15975 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.018+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5572776c6130 0x5572776c8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.018+0000 7f091f6a0b80 4 client.15975 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:48:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.018+0000 7f091f6a0b80 1 -- 172.21.15.12:0/652826482 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55727758ce60 con 0x5572776c6130 2024-05-08T23:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.022+0000 7f0910a81700 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5572776c6130 0x5572776c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.027+0000 7f08f4ff9700 10 client.15975 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.027+0000 7f0910a81700 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5572776c6130 0x5572776c8550 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18c27eb2af421f3 server_cookie=bd19c082d6f77e07 in_seq=0 out_seq=0 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.490+0000 7f08f4ff9700 1 -- 172.21.15.12:0/652826482 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55727758ce60 con 0x5572776c6130 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.490+0000 7f08f4ff9700 10 client.15975 handle_command_reply: tid=0 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.490+0000 7f091f6a0b80 1 client.15975 shutdown 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 1 -- 172.21.15.12:0/652826482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 msgr2=0x557277661c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x557277661c20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f09080098e0 tx=0x7f0908009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 1 -- 172.21.15.12:0/652826482 shutdown_connections 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5572776c6130 0x5572776c8550 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55727764a270 0x5572776c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557277644480 0x557277661c20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 1 --2- 172.21.15.12:0/652826482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557277649950 0x557277662160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f08f4ff9700 0 client.15975 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 1 -- 172.21.15.12:0/652826482 >> 172.21.15.12:0/652826482 conn(0x5572775a21b0 msgr2=0x55727764ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 1 -- 172.21.15.12:0/652826482 shutdown_connections 2024-05-08T23:48:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 1 -- 172.21.15.12:0/652826482 wait complete. 2024-05-08T23:48:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.491+0000 7f091f6a0b80 20 client.15975 trim_cache size 0 max 16384 2024-05-08T23:48:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:48:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:48:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:48:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:48:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 -- 172.21.15.12:0/2992084996 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08f80a1420 msgr2=0x7f08f80a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08f80a1420 0x7f08f80a38e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0908003800 tx=0x7f090801a040 comp rx=0 tx=0).stop 2024-05-08T23:48:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 -- 172.21.15.12:0/2992084996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c10edf0 msgr2=0x7f090c071700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c10edf0 0x7f090c071700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f090400bb00 tx=0x7f090400be10 comp rx=0 tx=0).stop 2024-05-08T23:48:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 -- 172.21.15.12:0/2992084996 shutdown_connections 2024-05-08T23:48:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08f80a1420 0x7f08f80a38e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c10edf0 0x7f090c071700 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f090c101110 0x7f090c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 --2- 172.21.15.12:0/2992084996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f090c106230 0x7f090c10e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 -- 172.21.15.12:0/2992084996 >> 172.21.15.12:0/2992084996 conn(0x7f090c0fc9b0 msgr2=0x7f090c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 -- 172.21.15.12:0/2992084996 shutdown_connections 2024-05-08T23:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:36.492+0000 7f09134e6700 1 -- 172.21.15.12:0/2992084996 wait complete. 2024-05-08T23:48:36.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:36 smithi204 ceph-mon[47373]: pgmap v69: 97 pgs: 97 active+clean; 16 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 454 KiB/s wr, 186 op/s 2024-05-08T23:48:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:36 smithi160 ceph-mon[46152]: pgmap v69: 97 pgs: 97 active+clean; 16 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 454 KiB/s wr, 186 op/s 2024-05-08T23:48:36.847 INFO:tasks.workunit.client.0.smithi012.stdout: 1 57788 10.25 MB/sec warmup 32 sec latency 3665.335 ms 2024-05-08T23:48:36.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:36 smithi012 ceph-mon[42503]: pgmap v69: 97 pgs: 97 active+clean; 16 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 454 KiB/s wr, 186 op/s 2024-05-08T23:48:37.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:37 smithi204 ceph-mon[47373]: pgmap v70: 97 pgs: 97 active+clean; 67 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.8 MiB/s wr, 280 op/s 2024-05-08T23:48:37.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:37 smithi160 ceph-mon[46152]: pgmap v70: 97 pgs: 97 active+clean; 67 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.8 MiB/s wr, 280 op/s 2024-05-08T23:48:37.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 59751 10.29 MB/sec warmup 33 sec latency 235.973 ms 2024-05-08T23:48:37.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:37 smithi012 ceph-mon[42503]: pgmap v70: 97 pgs: 97 active+clean; 67 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.8 MiB/s wr, 280 op/s 2024-05-08T23:48:38.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 59751 9.98 MB/sec warmup 34 sec latency 1006.959 ms 2024-05-08T23:48:39.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 59751 9.70 MB/sec warmup 35 sec latency 2007.079 ms 2024-05-08T23:48:40.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:40 smithi012 ceph-mon[42503]: pgmap v71: 97 pgs: 97 active+clean; 67 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.4 MiB/s wr, 105 op/s 2024-05-08T23:48:40.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:40 smithi204 ceph-mon[47373]: pgmap v71: 97 pgs: 97 active+clean; 67 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.4 MiB/s wr, 105 op/s 2024-05-08T23:48:40.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:40 smithi160 ceph-mon[46152]: pgmap v71: 97 pgs: 97 active+clean; 67 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.4 MiB/s wr, 105 op/s 2024-05-08T23:48:40.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 59751 9.43 MB/sec warmup 36 sec latency 3007.184 ms 2024-05-08T23:48:41.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 59751 9.17 MB/sec warmup 37 sec latency 4007.230 ms 2024-05-08T23:48:42.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:48:42.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:42 smithi012 ceph-mon[42503]: pgmap v72: 97 pgs: 97 active+clean; 26 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 5.7 MiB/s wr, 202 op/s 2024-05-08T23:48:42.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.683+0000 7feba6520700 1 -- 172.21.15.12:0/1047717903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feba0100ab0 msgr2=0x7feba010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.683+0000 7feba6520700 1 --2- 172.21.15.12:0/1047717903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feba0100ab0 0x7feba010ccc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feb9800b0d0 tx=0x7feb9800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.683+0000 7feba6520700 1 -- 172.21.15.12:0/1047717903 shutdown_connections 2024-05-08T23:48:42.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.683+0000 7feba6520700 1 --2- 172.21.15.12:0/1047717903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feba0100ab0 0x7feba010ccc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.683+0000 7feba6520700 1 --2- 172.21.15.12:0/1047717903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feba0105f90 0x7feba0106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.683+0000 7feba6520700 1 --2- 172.21.15.12:0/1047717903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba01000f0 0x7feba0100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.683+0000 7feba6520700 1 -- 172.21.15.12:0/1047717903 >> 172.21.15.12:0/1047717903 conn(0x7feba0076ba0 msgr2=0x7feba0076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:42.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.684+0000 7feba6520700 1 -- 172.21.15.12:0/1047717903 shutdown_connections 2024-05-08T23:48:42.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.684+0000 7feba6520700 1 -- 172.21.15.12:0/1047717903 wait complete. 2024-05-08T23:48:42.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.684+0000 7feba6520700 1 Processor -- start 2024-05-08T23:48:42.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.684+0000 7feba6520700 1 -- start start 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feba6520700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feba01000f0 0x7feba0197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feba6520700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feba0100ab0 0x7feba0197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feba6520700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba0105f90 0x7feba019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feba6520700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba019c370 con 0x7feba01000f0 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feba6520700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba019c4e0 con 0x7feba0100ab0 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feba6520700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba019c6a0 con 0x7feba0105f90 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feb9ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feba01000f0 0x7feba0197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feb9ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feba01000f0 0x7feba0197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47648/0 (socket says 172.21.15.12:47648) 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feb9ffff700 1 -- 172.21.15.12:0/187572091 learned_addr learned my addr 172.21.15.12:0/187572091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feba0100ab0 0x7feba0197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feba4abd700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba0105f90 0x7feba019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feb9ffff700 1 -- 172.21.15.12:0/187572091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba0105f90 msgr2=0x7feba019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feb9ffff700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba0105f90 0x7feba019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feb9ffff700 1 -- 172.21.15.12:0/187572091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feba0100ab0 msgr2=0x7feba0197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feb9ffff700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feba0100ab0 0x7feba0197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feb9ffff700 1 -- 172.21.15.12:0/187572091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90009520 con 0x7feba01000f0 2024-05-08T23:48:42.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.685+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feba0100ab0 0x7feba0197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:42.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.686+0000 7feb9ffff700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feba01000f0 0x7feba0197450 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7feb9000ea50 tx=0x7feb9000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:42.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.686+0000 7feb9d7fa700 1 -- 172.21.15.12:0/187572091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9000cbd0 con 0x7feba01000f0 2024-05-08T23:48:42.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.686+0000 7feba6520700 1 -- 172.21.15.12:0/187572091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb98009d00 con 0x7feba01000f0 2024-05-08T23:48:42.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.686+0000 7feb9d7fa700 1 -- 172.21.15.12:0/187572091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb9000cd30 con 0x7feba01000f0 2024-05-08T23:48:42.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.686+0000 7feb9d7fa700 1 -- 172.21.15.12:0/187572091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb90018680 con 0x7feba01000f0 2024-05-08T23:48:42.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.686+0000 7feba6520700 1 -- 172.21.15.12:0/187572091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba019cb70 con 0x7feba01000f0 2024-05-08T23:48:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.688+0000 7feb9d7fa700 1 -- 172.21.15.12:0/187572091 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7feb90010430 con 0x7feba01000f0 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.689+0000 7feb9d7fa700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feb880a1290 0x7feb880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.689+0000 7feb9d7fa700 1 -- 172.21.15.12:0/187572091 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7feb90014070 con 0x7feba01000f0 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.689+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feb880a1290 0x7feb880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.689+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feb880a1290 0x7feb880a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feba01989a0 tx=0x7feb9400b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7febb26dab80 1 Processor -- start 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7febb26dab80 1 -- start start 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7febb26dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x5633486835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7febb26dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 0x56334867d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7febb26dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 0x56334867e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7febb26dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633485d0150 con 0x563348683210 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7febb26dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633485cdb10 con 0x563348683b30 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7febb26dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633485c9400 con 0x56334867dca0 2024-05-08T23:48:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7feb9ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x5633486835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7feb9ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x5633486835f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47656/0 (socket says 172.21.15.12:47656) 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7feb9ffff700 1 -- 172.21.15.12:0/2136928528 learned_addr learned my addr 172.21.15.12:0/2136928528 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/2136928528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 0x56334867d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7feba4abd700 1 --2- 172.21.15.12:0/2136928528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 0x56334867e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7feb9ffff700 1 -- 172.21.15.12:0/2136928528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 msgr2=0x56334867e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7feb9ffff700 1 --2- 172.21.15.12:0/2136928528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 0x56334867e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7feb9ffff700 1 -- 172.21.15.12:0/2136928528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 msgr2=0x56334867d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7feb9ffff700 1 --2- 172.21.15.12:0/2136928528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 0x56334867d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.690+0000 7feb9ffff700 1 -- 172.21.15.12:0/2136928528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56334844a3b0 con 0x563348683210 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.691+0000 7feb9ffff700 1 --2- 172.21.15.12:0/2136928528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x5633486835f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feb900a21f0 tx=0x7feb90099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9df57ce44b60f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.691+0000 7feb867fc700 1 -- 172.21.15.12:0/2136928528 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9009e0d0 con 0x563348683210 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.691+0000 7feb867fc700 1 -- 172.21.15.12:0/2136928528 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb900a2720 con 0x563348683210 2024-05-08T23:48:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.691+0000 7feb867fc700 1 -- 172.21.15.12:0/2136928528 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb900999d0 con 0x563348683210 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.691+0000 7febb26dab80 1 -- 172.21.15.12:0/2136928528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 msgr2=0x5633486835f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.691+0000 7febb26dab80 1 --2- 172.21.15.12:0/2136928528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x5633486835f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feb900a21f0 tx=0x7feb90099f10 comp rx=0 tx=0).stop 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.692+0000 7febb26dab80 1 -- 172.21.15.12:0/2136928528 shutdown_connections 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.692+0000 7febb26dab80 1 --2- 172.21.15.12:0/2136928528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 0x56334867e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.692+0000 7febb26dab80 1 --2- 172.21.15.12:0/2136928528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x5633486835f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.692+0000 7febb26dab80 1 --2- 172.21.15.12:0/2136928528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 0x56334867d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.692+0000 7febb26dab80 1 -- 172.21.15.12:0/2136928528 >> 172.21.15.12:0/2136928528 conn(0x5633485d7210 msgr2=0x563348673df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.692+0000 7febb26dab80 1 -- 172.21.15.12:0/2136928528 shutdown_connections 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.692+0000 7febb26dab80 1 -- 172.21.15.12:0/2136928528 wait complete. 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.692+0000 7febb26dab80 1 -- start start 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.692+0000 7febb26dab80 1 Processor -- start 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7febb26dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334867dca0 0x563348689b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7febb26dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x563348686e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7febb26dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 0x563348687520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7febb26dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563348565560 con 0x563348683210 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7febb26dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563348565200 con 0x56334867dca0 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7febb26dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56334844c6a0 con 0x563348683b30 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7febb26dab80 10 client.0 _ll_register_callbacks cb 0x56334856eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb9f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x563348686e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb9f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x563348686e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47658/0 (socket says 172.21.15.12:47658) 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb9f7fe700 1 -- 172.21.15.12:0/4189895174 learned_addr learned my addr 172.21.15.12:0/4189895174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb9ffff700 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334867dca0 0x563348689b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feba4abd700 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 0x563348687520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb9f7fe700 1 -- 172.21.15.12:0/4189895174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 msgr2=0x563348687520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 0x563348687520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb9f7fe700 1 -- 172.21.15.12:0/4189895174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334867dca0 msgr2=0x563348689b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:42.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334867dca0 0x563348689b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb9f7fe700 1 -- 172.21.15.12:0/4189895174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba019cb70 con 0x563348683210 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x563348686e30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feb940119e0 tx=0x7feb940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dd04787f30623e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb7f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb7f7fe700 1 -- 172.21.15.12:0/4189895174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94011ea0 con 0x563348683210 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7feb7f7fe700 1 -- 172.21.15.12:0/4189895174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7feb94017070 con 0x563348683210 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.693+0000 7febb26dab80 1 -- 172.21.15.12:0/4189895174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563348522270 con 0x563348683210 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7feb7f7fe700 1 -- 172.21.15.12:0/4189895174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb94004a30 con 0x563348683210 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7febb26dab80 10 client.15987 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7febb26dab80 1 -- 172.21.15.12:0/4189895174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5633483f6b60 con 0x563348683210 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7feb7f7fe700 1 -- 172.21.15.12:0/4189895174 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563348522270 con 0x563348683210 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7feb7f7fe700 1 -- 172.21.15.12:0/4189895174 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feb9401f800 con 0x563348683210 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7febb26dab80 10 client.15987 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7febb26dab80 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7febb26dab80 20 client.15987 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7febb26dab80 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563348697580 0x5633486999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7febb26dab80 4 client.15987 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7febb26dab80 1 -- 172.21.15.12:0/4189895174 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563348648810 con 0x563348697580 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.694+0000 7feb9ffff700 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563348697580 0x5633486999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.695+0000 7feb9ffff700 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563348697580 0x5633486999a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d00aed713aacdb8d server_cookie=679290fe7091d78d in_seq=0 out_seq=0 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.695+0000 7feb7f7fe700 10 client.15987 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7feb7f7fe700 1 -- 172.21.15.12:0/4189895174 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563348648810 con 0x563348697580 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7feb7f7fe700 10 client.15987 handle_command_reply: tid=0 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 client.15987 shutdown 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 -- 172.21.15.12:0/4189895174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 msgr2=0x563348686e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x563348686e30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feb940119e0 tx=0x7feb940047c0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 -- 172.21.15.12:0/4189895174 shutdown_connections 2024-05-08T23:48:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 0x563348687520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563348697580 0x5633486999a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334867dca0 0x563348689b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7feb7f7fe700 0 client.15987 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 --2- 172.21.15.12:0/4189895174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x563348686e30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 -- 172.21.15.12:0/4189895174 >> 172.21.15.12:0/4189895174 conn(0x5633485d7210 msgr2=0x5633486735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 -- 172.21.15.12:0/4189895174 shutdown_connections 2024-05-08T23:48:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 1 -- 172.21.15.12:0/4189895174 wait complete. 2024-05-08T23:48:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.696+0000 7febb26dab80 20 client.15987 trim_cache size 0 max 16384 2024-05-08T23:48:42.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.714+0000 7febb26dab80 1 Processor -- start 2024-05-08T23:48:42.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.714+0000 7febb26dab80 1 -- start start 2024-05-08T23:48:42.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.714+0000 7febb26dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 0x563348673d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.714+0000 7febb26dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x56334869e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.714+0000 7febb26dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 0x563348680f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.714+0000 7febb26dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56334866e800 con 0x563348683210 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.714+0000 7febb26dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563348686c00 con 0x563348683b30 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.714+0000 7febb26dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563348674260 con 0x56334867dca0 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x56334869e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x56334869e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47660/0 (socket says 172.21.15.12:47660) 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9f7fe700 1 -- 172.21.15.12:0/3488474561 learned_addr learned my addr 172.21.15.12:0/3488474561 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9ffff700 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 0x563348673d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9f7fe700 1 -- 172.21.15.12:0/3488474561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 msgr2=0x563348673d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feba4abd700 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 0x563348680f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 0x563348673d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9f7fe700 1 -- 172.21.15.12:0/3488474561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 msgr2=0x563348680f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 0x563348680f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9f7fe700 1 -- 172.21.15.12:0/3488474561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56334858f560 con 0x563348683210 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9ffff700 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 0x563348673d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feba4abd700 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 0x563348680f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x56334869e300 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feb94004710 tx=0x7feb94004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32411f4cd9d82a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb86ffd700 1 -- 172.21.15.12:0/3488474561 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94010e80 con 0x563348683210 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb86ffd700 1 -- 172.21.15.12:0/3488474561 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb94015070 con 0x563348683210 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.715+0000 7feb86ffd700 1 -- 172.21.15.12:0/3488474561 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94005280 con 0x563348683210 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.716+0000 7febb26dab80 1 -- 172.21.15.12:0/3488474561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 msgr2=0x56334869e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.716+0000 7febb26dab80 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x56334869e300 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feb94004710 tx=0x7feb94004740 comp rx=0 tx=0).stop 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.716+0000 7febb26dab80 1 -- 172.21.15.12:0/3488474561 shutdown_connections 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.716+0000 7febb26dab80 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563348683b30 0x563348680f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.716+0000 7febb26dab80 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56334867dca0 0x563348673d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.716+0000 7febb26dab80 1 --2- 172.21.15.12:0/3488474561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563348683210 0x56334869e300 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.716+0000 7febb26dab80 1 -- 172.21.15.12:0/3488474561 >> 172.21.15.12:0/3488474561 conn(0x5633485d7210 msgr2=0x5633486757e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:42.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.716+0000 7febb26dab80 1 -- 172.21.15.12:0/3488474561 shutdown_connections 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.716+0000 7febb26dab80 1 -- 172.21.15.12:0/3488474561 wait complete. 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.716+0000 7febb26dab80 1 -- start start 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7febb26dab80 1 Processor -- start 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7febb26dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56334867dca0 0x56334868e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7febb26dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 0x56334868ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7febb26dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334868b050 0x56334868b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7febb26dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56334868ede0 con 0x56334867dca0 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7febb26dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633486eda80 con 0x56334868b050 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7febb26dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633486edbc0 con 0x563348683b30 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7febb26dab80 10 client.0 _ll_register_callbacks cb 0x56334856eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feb9ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56334867dca0 0x56334868e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feb9ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56334867dca0 0x56334868e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47664/0 (socket says 172.21.15.12:47664) 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feb9ffff700 1 -- 172.21.15.12:0/1216159139 learned_addr learned my addr 172.21.15.12:0/1216159139 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:42.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feba4abd700 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334868b050 0x56334868b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 0x56334868ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feb9ffff700 1 -- 172.21.15.12:0/1216159139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 msgr2=0x56334868ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feb9ffff700 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 0x56334868ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feb9ffff700 1 -- 172.21.15.12:0/1216159139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334868b050 msgr2=0x56334868b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feb9ffff700 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334868b050 0x56334868b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feb9ffff700 1 -- 172.21.15.12:0/1216159139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633483f6b60 con 0x56334867dca0 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feba4abd700 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334868b050 0x56334868b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.717+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 0x56334868ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7feb9ffff700 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56334867dca0 0x56334868e6b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feb90096d10 tx=0x7feb9009a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1188351fc895b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7feb7f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7feb7f7fe700 1 -- 172.21.15.12:0/1216159139 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb900b8040 con 0x56334867dca0 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7feb7f7fe700 1 -- 172.21.15.12:0/1216159139 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7feb900b7040 con 0x56334867dca0 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7febb26dab80 1 -- 172.21.15.12:0/1216159139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb900a2720 con 0x56334867dca0 2024-05-08T23:48:42.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7feb7f7fe700 1 -- 172.21.15.12:0/1216159139 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb9001e2c0 con 0x56334867dca0 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7febb26dab80 10 client.15993 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7febb26dab80 1 -- 172.21.15.12:0/1216159139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56334844ecd0 con 0x56334867dca0 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7feb7f7fe700 1 -- 172.21.15.12:0/1216159139 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb900a2720 con 0x56334867dca0 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7feb7f7fe700 1 -- 172.21.15.12:0/1216159139 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feb9009c070 con 0x56334867dca0 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7febb26dab80 10 client.15993 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7febb26dab80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7febb26dab80 20 client.15993 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7febb26dab80 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5633486ef0f0 0x5633486f1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7febb26dab80 4 client.15993 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.718+0000 7febb26dab80 1 -- 172.21.15.12:0/1216159139 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633485c1ec0 con 0x5633486ef0f0 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.719+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5633486ef0f0 0x5633486f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.719+0000 7feb7f7fe700 10 client.15993 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.719+0000 7feb9f7fe700 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5633486ef0f0 0x5633486f1510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f6e58b5ed72d7a server_cookie=1aa322012d8ef5e9 in_seq=0 out_seq=0 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.719+0000 7feb7f7fe700 1 -- 172.21.15.12:0/1216159139 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5633485c1ec0 con 0x5633486ef0f0 2024-05-08T23:48:42.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.719+0000 7feb7f7fe700 10 client.15993 handle_command_reply: tid=0 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.719+0000 7febb26dab80 1 client.15993 shutdown 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 1 -- 172.21.15.12:0/1216159139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56334867dca0 msgr2=0x56334868e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56334867dca0 0x56334868e6b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feb90096d10 tx=0x7feb9009a420 comp rx=0 tx=0).stop 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 1 -- 172.21.15.12:0/1216159139 shutdown_connections 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5633486ef0f0 0x5633486f1510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56334868b050 0x56334868b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feb7f7fe700 0 client.15993 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56334867dca0 0x56334868e6b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 1 --2- 172.21.15.12:0/1216159139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563348683b30 0x56334868ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 1 -- 172.21.15.12:0/1216159139 >> 172.21.15.12:0/1216159139 conn(0x5633485d7210 msgr2=0x56334869e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 1 -- 172.21.15.12:0/1216159139 shutdown_connections 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 1 -- 172.21.15.12:0/1216159139 wait complete. 2024-05-08T23:48:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7febb26dab80 20 client.15993 trim_cache size 0 max 16384 2024-05-08T23:48:42.723 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:48:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:48:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:48:42.723 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:48:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feba6520700 1 -- 172.21.15.12:0/187572091 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feb880a1290 msgr2=0x7feb880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feba6520700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feb880a1290 0x7feb880a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feba01989a0 tx=0x7feb9400b410 comp rx=0 tx=0).stop 2024-05-08T23:48:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feba6520700 1 -- 172.21.15.12:0/187572091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feba01000f0 msgr2=0x7feba0197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feba6520700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feba01000f0 0x7feba0197450 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7feb9000ea50 tx=0x7feb9000ed60 comp rx=0 tx=0).stop 2024-05-08T23:48:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feba6520700 1 -- 172.21.15.12:0/187572091 shutdown_connections 2024-05-08T23:48:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feba6520700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feb880a1290 0x7feb880a3750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feba6520700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba0105f90 0x7feba019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feba6520700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feba01000f0 0x7feba0197450 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feba6520700 1 --2- 172.21.15.12:0/187572091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feba0100ab0 0x7feba0197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.720+0000 7feba6520700 1 -- 172.21.15.12:0/187572091 >> 172.21.15.12:0/187572091 conn(0x7feba0076ba0 msgr2=0x7feba00691a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:42.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.721+0000 7feba6520700 1 -- 172.21.15.12:0/187572091 shutdown_connections 2024-05-08T23:48:42.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.721+0000 7feba6520700 1 -- 172.21.15.12:0/187572091 wait complete. 2024-05-08T23:48:42.729 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:48:42.729 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:48:42.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:42 smithi204 ceph-mon[47373]: pgmap v72: 97 pgs: 97 active+clean; 26 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 5.7 MiB/s wr, 202 op/s 2024-05-08T23:48:42.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:42 smithi160 ceph-mon[46152]: pgmap v72: 97 pgs: 97 active+clean; 26 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 5.7 MiB/s wr, 202 op/s 2024-05-08T23:48:42.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 59801 8.93 MB/sec warmup 38 sec latency 4913.379 ms 2024-05-08T23:48:42.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.875+0000 7f77589e2700 1 -- 172.21.15.12:0/649137354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7754105060 msgr2=0x7f775410cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.875+0000 7f77589e2700 1 --2- 172.21.15.12:0/649137354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7754105060 0x7f775410cbe0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f774c00b0a0 tx=0x7f774c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.875+0000 7f77589e2700 1 -- 172.21.15.12:0/649137354 shutdown_connections 2024-05-08T23:48:42.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.875+0000 7f77589e2700 1 --2- 172.21.15.12:0/649137354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7754105060 0x7f775410cbe0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.875+0000 7f77589e2700 1 --2- 172.21.15.12:0/649137354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77540fff40 0x7f7754100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.875+0000 7f77589e2700 1 --2- 172.21.15.12:0/649137354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77541008f0 0x7f7754104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.875+0000 7f77589e2700 1 -- 172.21.15.12:0/649137354 >> 172.21.15.12:0/649137354 conn(0x7f77540fb7e0 msgr2=0x7f77540fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:42.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.875+0000 7f77589e2700 1 -- 172.21.15.12:0/649137354 shutdown_connections 2024-05-08T23:48:42.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.875+0000 7f77589e2700 1 -- 172.21.15.12:0/649137354 wait complete. 2024-05-08T23:48:42.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.876+0000 7f77589e2700 1 Processor -- start 2024-05-08T23:48:42.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f77589e2700 1 -- start start 2024-05-08T23:48:42.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f77589e2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77540fff40 0x7f7754197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f77589e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7754105060 0x7f7754197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f77589e2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775419be30 0x7f775419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f77589e2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775419c8b0 con 0x7f7754105060 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f77589e2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775419ca20 con 0x7f77540fff40 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f77589e2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775419cbe0 con 0x7f775419be30 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f7751d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7754105060 0x7f7754197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f7751d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7754105060 0x7f7754197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47690/0 (socket says 172.21.15.12:47690) 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f7751d9b700 1 -- 172.21.15.12:0/4261400856 learned_addr learned my addr 172.21.15.12:0/4261400856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f7752d9d700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775419be30 0x7f775419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f775259c700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77540fff40 0x7f7754197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f7751d9b700 1 -- 172.21.15.12:0/4261400856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775419be30 msgr2=0x7f775419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f7751d9b700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775419be30 0x7f775419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f7751d9b700 1 -- 172.21.15.12:0/4261400856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77540fff40 msgr2=0x7f7754197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:42.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f7751d9b700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77540fff40 0x7f7754197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f7751d9b700 1 -- 172.21.15.12:0/4261400856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c009d00 con 0x7f7754105060 2024-05-08T23:48:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.877+0000 7f7752d9d700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775419be30 0x7f775419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.878+0000 7f7751d9b700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7754105060 0x7f7754197a90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f774800e970 tx=0x7f774800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.878+0000 7f77437fe700 1 -- 172.21.15.12:0/4261400856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748009790 con 0x7f7754105060 2024-05-08T23:48:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.878+0000 7f77589e2700 1 -- 172.21.15.12:0/4261400856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77541a4140 con 0x7f7754105060 2024-05-08T23:48:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.878+0000 7f77437fe700 1 -- 172.21.15.12:0/4261400856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f774800c7e0 con 0x7f7754105060 2024-05-08T23:48:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.878+0000 7f77437fe700 1 -- 172.21.15.12:0/4261400856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77480054e0 con 0x7f7754105060 2024-05-08T23:48:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.878+0000 7f77589e2700 1 -- 172.21.15.12:0/4261400856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77541a4670 con 0x7f7754105060 2024-05-08T23:48:42.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.879+0000 7f77589e2700 1 -- 172.21.15.12:0/4261400856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7754109340 con 0x7f7754105060 2024-05-08T23:48:42.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.880+0000 7f77437fe700 1 -- 172.21.15.12:0/4261400856 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f774800c950 con 0x7f7754105060 2024-05-08T23:48:42.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.881+0000 7f77437fe700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f773c0a1290 0x7f773c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:42.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.881+0000 7f77437fe700 1 -- 172.21.15.12:0/4261400856 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7748014070 con 0x7f7754105060 2024-05-08T23:48:42.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.881+0000 7f775259c700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f773c0a1290 0x7f773c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:42.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.881+0000 7f775259c700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f773c0a1290 0x7f773c0a3750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7744009c60 tx=0x7f7744009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:42.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:42.885+0000 7f77437fe700 1 -- 172.21.15.12:0/4261400856 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7748096510 con 0x7f7754105060 2024-05-08T23:48:43.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.153+0000 7f77589e2700 1 -- 172.21.15.12:0/4261400856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7754198850 con 0x7f7754105060 2024-05-08T23:48:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.154+0000 7f77437fe700 1 -- 172.21.15.12:0/4261400856 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10879 (secure 0 0 0) 0x7f7748099b30 con 0x7f7754105060 2024-05-08T23:48:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:48:43.149 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:48:23.429180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25489":{"gid":25489,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3829900343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3829900343},{"type":"v1","addr":"172.21.15.204:6837","nonce":3829900343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 -- 172.21.15.12:0/4261400856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f773c0a1290 msgr2=0x7f773c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f773c0a1290 0x7f773c0a3750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7744009c60 tx=0x7f7744009260 comp rx=0 tx=0).stop 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 -- 172.21.15.12:0/4261400856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7754105060 msgr2=0x7f7754197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7754105060 0x7f7754197a90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f774800e970 tx=0x7f774800ec80 comp rx=0 tx=0).stop 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 -- 172.21.15.12:0/4261400856 shutdown_connections 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f773c0a1290 0x7f773c0a3750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775419be30 0x7f775419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77540fff40 0x7f7754197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 --2- 172.21.15.12:0/4261400856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7754105060 0x7f7754197a90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 -- 172.21.15.12:0/4261400856 >> 172.21.15.12:0/4261400856 conn(0x7f77540fb7e0 msgr2=0x7f7754068460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 -- 172.21.15.12:0/4261400856 shutdown_connections 2024-05-08T23:48:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.156+0000 7f77589e2700 1 -- 172.21.15.12:0/4261400856 wait complete. 2024-05-08T23:48:43.153 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-05-08T23:48:43.165 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:48:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.308+0000 7fc905b6f700 1 -- 172.21.15.12:0/1059330330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900069dd0 msgr2=0x7fc900105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.308+0000 7fc905b6f700 1 --2- 172.21.15.12:0/1059330330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900069dd0 0x7fc900105b70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009990 tx=0x7fc8e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.308+0000 7fc905b6f700 1 -- 172.21.15.12:0/1059330330 shutdown_connections 2024-05-08T23:48:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.308+0000 7fc905b6f700 1 --2- 172.21.15.12:0/1059330330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9001062a0 0x7fc90010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.308+0000 7fc905b6f700 1 --2- 172.21.15.12:0/1059330330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc900069420 0x7fc900069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.308+0000 7fc905b6f700 1 --2- 172.21.15.12:0/1059330330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900069dd0 0x7fc900105b70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.308+0000 7fc905b6f700 1 -- 172.21.15.12:0/1059330330 >> 172.21.15.12:0/1059330330 conn(0x7fc900076ba0 msgr2=0x7fc900076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.308+0000 7fc905b6f700 1 -- 172.21.15.12:0/1059330330 shutdown_connections 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.308+0000 7fc905b6f700 1 -- 172.21.15.12:0/1059330330 wait complete. 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.309+0000 7fc905b6f700 1 Processor -- start 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.309+0000 7fc905b6f700 1 -- start start 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc905b6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900069420 0x7fc900197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc905b6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9001062a0 0x7fc900197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc905b6f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90019bca0 0x7fc90019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc905b6f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90019c720 con 0x7fc900069420 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc905b6f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90019c890 con 0x7fc9001062a0 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc905b6f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90019ca50 con 0x7fc90019bca0 2024-05-08T23:48:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8ff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900069420 0x7fc900197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8ff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900069420 0x7fc900197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47712/0 (socket says 172.21.15.12:47712) 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8ff7fe700 1 -- 172.21.15.12:0/647947702 learned_addr learned my addr 172.21.15.12:0/647947702 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8feffd700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9001062a0 0x7fc900197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8fffff700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90019bca0 0x7fc90019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8ff7fe700 1 -- 172.21.15.12:0/647947702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90019bca0 msgr2=0x7fc90019c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90019bca0 0x7fc90019c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8ff7fe700 1 -- 172.21.15.12:0/647947702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9001062a0 msgr2=0x7fc900197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9001062a0 0x7fc900197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8ff7fe700 1 -- 172.21.15.12:0/647947702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e80095f0 con 0x7fc900069420 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900069420 0x7fc900197410 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000e9a0 tx=0x7fc8f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8fcff9700 1 -- 172.21.15.12:0/647947702 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0009790 con 0x7fc900069420 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8fcff9700 1 -- 172.21.15.12:0/647947702 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8f0010430 con 0x7fc900069420 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc905b6f700 1 -- 172.21.15.12:0/647947702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9001a3fb0 con 0x7fc900069420 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.310+0000 7fc8fcff9700 1 -- 172.21.15.12:0/647947702 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0018680 con 0x7fc900069420 2024-05-08T23:48:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.311+0000 7fc905b6f700 1 -- 172.21.15.12:0/647947702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9001a43d0 con 0x7fc900069420 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.313+0000 7fc8fcff9700 1 -- 172.21.15.12:0/647947702 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc8f00105a0 con 0x7fc900069420 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.313+0000 7fc8fcff9700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8ec0a1290 0x7fc8ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.314+0000 7fc8feffd700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8ec0a1290 0x7fc8ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.314+0000 7fc8fcff9700 1 -- 172.21.15.12:0/647947702 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc8f0014070 con 0x7fc900069420 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.314+0000 7fc8feffd700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8ec0a1290 0x7fc8ec0a3750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800b2e0 tx=0x7fc8e8005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.314+0000 7fc911d29b80 1 Processor -- start 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.314+0000 7fc911d29b80 1 -- start start 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28616e00 0x55dc286171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28617720 0x55dc28611350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28611890 0x55dc28611cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc911d29b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc28568150 con 0x55dc28611890 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc911d29b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc28565b10 con 0x55dc28617720 2024-05-08T23:48:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc911d29b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc28561400 con 0x55dc28616e00 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28611890 0x55dc28611cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28611890 0x55dc28611cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47726/0 (socket says 172.21.15.12:47726) 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8fffff700 1 -- 172.21.15.12:0/3120252172 learned_addr learned my addr 172.21.15.12:0/3120252172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/3120252172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28616e00 0x55dc286171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8feffd700 1 --2- 172.21.15.12:0/3120252172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28617720 0x55dc28611350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8fffff700 1 -- 172.21.15.12:0/3120252172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28616e00 msgr2=0x55dc286171e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8fffff700 1 --2- 172.21.15.12:0/3120252172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28616e00 0x55dc286171e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8fffff700 1 -- 172.21.15.12:0/3120252172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28617720 msgr2=0x55dc28611350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8fffff700 1 --2- 172.21.15.12:0/3120252172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28617720 0x55dc28611350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8fffff700 1 -- 172.21.15.12:0/3120252172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc283e23b0 con 0x55dc28611890 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/3120252172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28616e00 0x55dc286171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8fffff700 1 --2- 172.21.15.12:0/3120252172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28611890 0x55dc28611cd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8011680 tx=0x7fc8f8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c842bd478a29ab13 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.315+0000 7fc8f5ffb700 1 -- 172.21.15.12:0/3120252172 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800e040 con 0x55dc28611890 2024-05-08T23:48:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc8f5ffb700 1 -- 172.21.15.12:0/3120252172 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8f800f040 con 0x55dc28611890 2024-05-08T23:48:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc8f5ffb700 1 -- 172.21.15.12:0/3120252172 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8004860 con 0x55dc28611890 2024-05-08T23:48:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc911d29b80 1 -- 172.21.15.12:0/3120252172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28611890 msgr2=0x55dc28611cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc911d29b80 1 --2- 172.21.15.12:0/3120252172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28611890 0x55dc28611cd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8011680 tx=0x7fc8f8009920 comp rx=0 tx=0).stop 2024-05-08T23:48:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc911d29b80 1 -- 172.21.15.12:0/3120252172 shutdown_connections 2024-05-08T23:48:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc911d29b80 1 --2- 172.21.15.12:0/3120252172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28611890 0x55dc28611cd0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc911d29b80 1 --2- 172.21.15.12:0/3120252172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28616e00 0x55dc286171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc911d29b80 1 --2- 172.21.15.12:0/3120252172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28617720 0x55dc28611350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc911d29b80 1 -- 172.21.15.12:0/3120252172 >> 172.21.15.12:0/3120252172 conn(0x55dc2856f210 msgr2=0x55dc2860bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc911d29b80 1 -- 172.21.15.12:0/3120252172 shutdown_connections 2024-05-08T23:48:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.316+0000 7fc911d29b80 1 -- 172.21.15.12:0/3120252172 wait complete. 2024-05-08T23:48:43.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.317+0000 7fc911d29b80 1 -- start start 2024-05-08T23:48:43.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.317+0000 7fc911d29b80 1 Processor -- start 2024-05-08T23:48:43.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 0x55dc2861aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc2861afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28617720 0x55dc2861b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc911d29b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc284fd560 con 0x55dc28616e00 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc911d29b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc284fd200 con 0x55dc28611890 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc911d29b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc283e46a0 con 0x55dc28617720 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc911d29b80 10 client.0 _ll_register_callbacks cb 0x55dc28506b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8feffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc2861afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8feffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc2861afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47730/0 (socket says 172.21.15.12:47730) 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8feffd700 1 -- 172.21.15.12:0/949517825 learned_addr learned my addr 172.21.15.12:0/949517825 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8fffff700 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28617720 0x55dc2861b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 0x55dc2861aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8feffd700 1 -- 172.21.15.12:0/949517825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28617720 msgr2=0x55dc2861b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8feffd700 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28617720 0x55dc2861b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8feffd700 1 -- 172.21.15.12:0/949517825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 msgr2=0x55dc2861aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8feffd700 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 0x55dc2861aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8feffd700 1 -- 172.21.15.12:0/949517825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f8011330 con 0x55dc28616e00 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8feffd700 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc2861afb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009990 tx=0x7fc8e801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3e5cbe337fcd62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8deffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.318+0000 7fc8deffd700 1 -- 172.21.15.12:0/949517825 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e800bee0 con 0x55dc28616e00 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc8deffd700 1 -- 172.21.15.12:0/949517825 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc8e8004c40 con 0x55dc28616e00 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc911d29b80 1 -- 172.21.15.12:0/949517825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc284ba270 con 0x55dc28616e00 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc8deffd700 1 -- 172.21.15.12:0/949517825 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8e8004e00 con 0x55dc28616e00 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc911d29b80 10 client.16011 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc911d29b80 1 -- 172.21.15.12:0/949517825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc2838eb60 con 0x55dc28616e00 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc8deffd700 1 -- 172.21.15.12:0/949517825 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc284ba270 con 0x55dc28616e00 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc8deffd700 1 -- 172.21.15.12:0/949517825 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8e8030540 con 0x55dc28616e00 2024-05-08T23:48:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc911d29b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc911d29b80 10 client.16011 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc911d29b80 20 client.16011 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc911d29b80 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc2862b1b0 0x55dc2862d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc911d29b80 4 client.16011 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.319+0000 7fc911d29b80 1 -- 172.21.15.12:0/949517825 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc285e0810 con 0x55dc2862b1b0 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.320+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc2862b1b0 0x55dc2862d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.320+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc2862b1b0 0x55dc2862d5d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa0e9ed643858023 server_cookie=5219332520761281 in_seq=0 out_seq=0 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.320+0000 7fc8deffd700 10 client.16011 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc8deffd700 1 -- 172.21.15.12:0/949517825 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc285e0810 con 0x55dc2862b1b0 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc8deffd700 10 client.16011 handle_command_reply: tid=0 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 client.16011 shutdown 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 -- 172.21.15.12:0/949517825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 msgr2=0x55dc2861afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc2861afb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009990 tx=0x7fc8e801a050 comp rx=0 tx=0).stop 2024-05-08T23:48:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 -- 172.21.15.12:0/949517825 shutdown_connections 2024-05-08T23:48:43.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc2862b1b0 0x55dc2862d5d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28617720 0x55dc2861b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 0x55dc2861aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc8deffd700 0 client.16011 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:48:43.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 --2- 172.21.15.12:0/949517825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc2861afb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 -- 172.21.15.12:0/949517825 >> 172.21.15.12:0/949517825 conn(0x55dc2856f210 msgr2=0x55dc2860b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 -- 172.21.15.12:0/949517825 shutdown_connections 2024-05-08T23:48:43.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 1 -- 172.21.15.12:0/949517825 wait complete. 2024-05-08T23:48:43.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.321+0000 7fc911d29b80 20 client.16011 trim_cache size 0 max 16384 2024-05-08T23:48:43.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc911d29b80 1 Processor -- start 2024-05-08T23:48:43.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc911d29b80 1 -- start start 2024-05-08T23:48:43.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28611890 0x55dc28611350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28616e00 0x55dc28626c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28617720 0x55dc2860bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc911d29b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc28617560 con 0x55dc28617720 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc911d29b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc28632800 con 0x55dc28616e00 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc911d29b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc28618fc0 con 0x55dc28611890 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc8fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28617720 0x55dc2860bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc8fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28617720 0x55dc2860bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47740/0 (socket says 172.21.15.12:47740) 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc8fffff700 1 -- 172.21.15.12:0/2815405963 learned_addr learned my addr 172.21.15.12:0/2815405963 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc8feffd700 1 --2- 172.21.15.12:0/2815405963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28616e00 0x55dc28626c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.335+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/2815405963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28611890 0x55dc28611350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.336+0000 7fc8fffff700 1 -- 172.21.15.12:0/2815405963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28611890 msgr2=0x55dc28611350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.336+0000 7fc8fffff700 1 --2- 172.21.15.12:0/2815405963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28611890 0x55dc28611350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.336+0000 7fc8fffff700 1 -- 172.21.15.12:0/2815405963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28616e00 msgr2=0x55dc28626c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.336+0000 7fc8fffff700 1 --2- 172.21.15.12:0/2815405963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28616e00 0x55dc28626c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.336+0000 7fc8fffff700 1 -- 172.21.15.12:0/2815405963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc28527560 con 0x55dc28617720 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.336+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/2815405963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28611890 0x55dc28611350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.336+0000 7fc8fffff700 1 --2- 172.21.15.12:0/2815405963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28617720 0x55dc2860bd20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8011680 tx=0x7fc8f80093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94df6655d9f3fb1c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.336+0000 7fc8f67fc700 1 -- 172.21.15.12:0/2815405963 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800e050 con 0x55dc28617720 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.336+0000 7fc8f67fc700 1 -- 172.21.15.12:0/2815405963 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8f8003ec0 con 0x55dc28617720 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.336+0000 7fc8f67fc700 1 -- 172.21.15.12:0/2815405963 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8007c10 con 0x55dc28617720 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 -- 172.21.15.12:0/2815405963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28617720 msgr2=0x55dc2860bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 --2- 172.21.15.12:0/2815405963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28617720 0x55dc2860bd20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8011680 tx=0x7fc8f80093a0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 -- 172.21.15.12:0/2815405963 shutdown_connections 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 --2- 172.21.15.12:0/2815405963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28617720 0x55dc2860bd20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 --2- 172.21.15.12:0/2815405963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28611890 0x55dc28611350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 --2- 172.21.15.12:0/2815405963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28616e00 0x55dc28626c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 -- 172.21.15.12:0/2815405963 >> 172.21.15.12:0/2815405963 conn(0x55dc2856f210 msgr2=0x55dc28611c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 -- 172.21.15.12:0/2815405963 shutdown_connections 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 -- 172.21.15.12:0/2815405963 wait complete. 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 -- start start 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.337+0000 7fc911d29b80 1 Processor -- start 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 0x55dc28620150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc28620690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc911d29b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28617720 0x55dc286806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc911d29b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc28620f30 con 0x55dc28616e00 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc911d29b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc28680c10 con 0x55dc28611890 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc911d29b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc28680d50 con 0x55dc28617720 2024-05-08T23:48:43.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc911d29b80 10 client.0 _ll_register_callbacks cb 0x55dc28506b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8feffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc28620690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8ff7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 0x55dc28620150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8ff7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 0x55dc28620150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:46180/0 (socket says 172.21.15.12:46180) 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8ff7fe700 1 -- 172.21.15.12:0/582445150 learned_addr learned my addr 172.21.15.12:0/582445150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8feffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc28620690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47742/0 (socket says 172.21.15.12:47742) 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8fffff700 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28617720 0x55dc286806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8ff7fe700 1 -- 172.21.15.12:0/582445150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28617720 msgr2=0x55dc286806d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28617720 0x55dc286806d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8ff7fe700 1 -- 172.21.15.12:0/582445150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 msgr2=0x55dc28620690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc28620690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8ff7fe700 1 -- 172.21.15.12:0/582445150 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc283e23b0 con 0x55dc28611890 2024-05-08T23:48:43.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8feffd700 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc28620690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8ff7fe700 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 0x55dc28620150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c2d0 tx=0x7fc8f0099d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9c0ab712970e89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.338+0000 7fc8deffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc8deffd700 1 -- 172.21.15.12:0/582445150 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0010880 con 0x55dc28611890 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc8deffd700 1 -- 172.21.15.12:0/582445150 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc8f009a1a0 con 0x55dc28611890 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc911d29b80 1 -- 172.21.15.12:0/582445150 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8f800f040 con 0x55dc28611890 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc8deffd700 1 -- 172.21.15.12:0/582445150 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8f00109e0 con 0x55dc28611890 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc911d29b80 10 client.25850 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc911d29b80 1 -- 172.21.15.12:0/582445150 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc283e6cd0 con 0x55dc28611890 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc8deffd700 1 -- 172.21.15.12:0/582445150 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8f0063020 con 0x55dc28611890 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc8deffd700 1 -- 172.21.15.12:0/582445150 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8f00a1f30 con 0x55dc28611890 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc911d29b80 10 client.25850 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc911d29b80 10 client.25850 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:48:43.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc911d29b80 20 client.25850 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc911d29b80 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc28682440 0x55dc28684830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc911d29b80 4 client.25850 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.339+0000 7fc911d29b80 1 -- 172.21.15.12:0/582445150 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc28559ec0 con 0x55dc28682440 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc8feffd700 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc28682440 0x55dc28684830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc8feffd700 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc28682440 0x55dc28684830 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb01bbeb91ad354c server_cookie=6912153aa01a506 in_seq=0 out_seq=0 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc8deffd700 10 client.25850 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc8deffd700 1 -- 172.21.15.12:0/582445150 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc28559ec0 con 0x55dc28682440 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc8deffd700 10 client.25850 handle_command_reply: tid=0 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 client.25850 shutdown 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 -- 172.21.15.12:0/582445150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 msgr2=0x55dc28620150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 0x55dc28620150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c2d0 tx=0x7fc8f0099d10 comp rx=0 tx=0).stop 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 -- 172.21.15.12:0/582445150 shutdown_connections 2024-05-08T23:48:43.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc28682440 0x55dc28684830 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc28617720 0x55dc286806d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc28611890 0x55dc28620150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc8deffd700 0 client.25850 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:48:43.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 --2- 172.21.15.12:0/582445150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc28616e00 0x55dc28620690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 -- 172.21.15.12:0/582445150 >> 172.21.15.12:0/582445150 conn(0x55dc2856f210 msgr2=0x55dc28619300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 -- 172.21.15.12:0/582445150 shutdown_connections 2024-05-08T23:48:43.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.340+0000 7fc911d29b80 1 -- 172.21.15.12:0/582445150 wait complete. 2024-05-08T23:48:43.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc911d29b80 20 client.25850 trim_cache size 0 max 16384 2024-05-08T23:48:43.343 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:48:43.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 -- 172.21.15.12:0/647947702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8ec0a1290 msgr2=0x7fc8ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8ec0a1290 0x7fc8ec0a3750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800b2e0 tx=0x7fc8e8005960 comp rx=0 tx=0).stop 2024-05-08T23:48:43.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 -- 172.21.15.12:0/647947702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900069420 msgr2=0x7fc900197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900069420 0x7fc900197410 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000e9a0 tx=0x7fc8f000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 -- 172.21.15.12:0/647947702 shutdown_connections 2024-05-08T23:48:43.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8ec0a1290 0x7fc8ec0a3750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc90019bca0 0x7fc90019c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900069420 0x7fc900197410 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 --2- 172.21.15.12:0/647947702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9001062a0 0x7fc900197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 -- 172.21.15.12:0/647947702 >> 172.21.15.12:0/647947702 conn(0x7fc900076ba0 msgr2=0x7fc9000ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 -- 172.21.15.12:0/647947702 shutdown_connections 2024-05-08T23:48:43.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.341+0000 7fc905b6f700 1 -- 172.21.15.12:0/647947702 wait complete. 2024-05-08T23:48:43.352 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:48:43.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:43 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/4261400856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:48:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.496+0000 7fbda7a0e700 1 -- 172.21.15.12:0/3953106527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbda0100ab0 msgr2=0x7fbda010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.496+0000 7fbda7a0e700 1 --2- 172.21.15.12:0/3953106527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbda0100ab0 0x7fbda010ccc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbd980099c0 tx=0x7fbd98009cd0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.496+0000 7fbda7a0e700 1 -- 172.21.15.12:0/3953106527 shutdown_connections 2024-05-08T23:48:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.496+0000 7fbda7a0e700 1 --2- 172.21.15.12:0/3953106527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbda0100ab0 0x7fbda010ccc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.496+0000 7fbda7a0e700 1 --2- 172.21.15.12:0/3953106527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda0105f90 0x7fbda0106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.496+0000 7fbda7a0e700 1 --2- 172.21.15.12:0/3953106527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda01000f0 0x7fbda0100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.496+0000 7fbda7a0e700 1 -- 172.21.15.12:0/3953106527 >> 172.21.15.12:0/3953106527 conn(0x7fbda0076ba0 msgr2=0x7fbda0076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.496+0000 7fbda7a0e700 1 -- 172.21.15.12:0/3953106527 shutdown_connections 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.496+0000 7fbda7a0e700 1 -- 172.21.15.12:0/3953106527 wait complete. 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda7a0e700 1 Processor -- start 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda7a0e700 1 -- start start 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda7a0e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda01000f0 0x7fbda0197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda7a0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbda0100ab0 0x7fbda0197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda7a0e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda0105f90 0x7fbda019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda7a0e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda019c210 con 0x7fbda0100ab0 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda7a0e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda019c380 con 0x7fbda01000f0 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda7a0e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda019c540 con 0x7fbda0105f90 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda4fa9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbda0100ab0 0x7fbda0197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda4fa9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbda0100ab0 0x7fbda0197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47754/0 (socket says 172.21.15.12:47754) 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda4fa9700 1 -- 172.21.15.12:0/4257821273 learned_addr learned my addr 172.21.15.12:0/4257821273 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.497+0000 7fbda57aa700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda01000f0 0x7fbda0197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbda5fab700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda0105f90 0x7fbda019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbda4fa9700 1 -- 172.21.15.12:0/4257821273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda0105f90 msgr2=0x7fbda019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda0105f90 0x7fbda019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbda4fa9700 1 -- 172.21.15.12:0/4257821273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda01000f0 msgr2=0x7fbda0197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda01000f0 0x7fbda0197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbda4fa9700 1 -- 172.21.15.12:0/4257821273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd980095f0 con 0x7fbda0100ab0 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbda0100ab0 0x7fbda0197880 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400d950 tx=0x7fbd9400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbd9e7fc700 1 -- 172.21.15.12:0/4257821273 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400f890 con 0x7fbda0100ab0 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbd9e7fc700 1 -- 172.21.15.12:0/4257821273 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd94010460 con 0x7fbda0100ab0 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbda7a0e700 1 -- 172.21.15.12:0/4257821273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda019c820 con 0x7fbda0100ab0 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbd9e7fc700 1 -- 172.21.15.12:0/4257821273 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400e610 con 0x7fbda0100ab0 2024-05-08T23:48:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.498+0000 7fbda7a0e700 1 -- 172.21.15.12:0/4257821273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda01a4060 con 0x7fbda0100ab0 2024-05-08T23:48:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.500+0000 7fbd9e7fc700 1 -- 172.21.15.12:0/4257821273 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbd940108d0 con 0x7fbda0100ab0 2024-05-08T23:48:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.501+0000 7fbd9e7fc700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbd880a1310 0x7fbd880a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.501+0000 7fbda57aa700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbd880a1310 0x7fbd880a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.501+0000 7fbd9e7fc700 1 -- 172.21.15.12:0/4257821273 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fbd940cca40 con 0x7fbda0100ab0 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbda57aa700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbd880a1310 0x7fbd880a37d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbda0067bb0 tx=0x7fbd9000b040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbdb3bc8b80 1 Processor -- start 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbdb3bc8b80 1 -- start start 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 0x556ee6c82350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c82890 0x556ee6c82cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6bd9150 con 0x556ee6c87e00 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6bd6b10 con 0x556ee6c82890 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6bd2400 con 0x556ee6c88720 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbda57aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbda57aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c881e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47766/0 (socket says 172.21.15.12:47766) 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbda57aa700 1 -- 172.21.15.12:0/853224626 learned_addr learned my addr 172.21.15.12:0/853224626 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.502+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/853224626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 0x556ee6c82350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbda5fab700 1 --2- 172.21.15.12:0/853224626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c82890 0x556ee6c82cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbda57aa700 1 -- 172.21.15.12:0/853224626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 msgr2=0x556ee6c82350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbda57aa700 1 --2- 172.21.15.12:0/853224626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 0x556ee6c82350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbda57aa700 1 -- 172.21.15.12:0/853224626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c82890 msgr2=0x556ee6c82cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbda57aa700 1 --2- 172.21.15.12:0/853224626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c82890 0x556ee6c82cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbda57aa700 1 -- 172.21.15.12:0/853224626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ee6a533b0 con 0x556ee6c87e00 2024-05-08T23:48:43.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/853224626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 0x556ee6c82350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:43.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbda57aa700 1 --2- 172.21.15.12:0/853224626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c881e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbd900176e0 tx=0x7fbd90014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3189e67de6979e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbd877fe700 1 -- 172.21.15.12:0/853224626 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000e040 con 0x556ee6c87e00 2024-05-08T23:48:43.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbd877fe700 1 -- 172.21.15.12:0/853224626 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd9000f040 con 0x556ee6c87e00 2024-05-08T23:48:43.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.503+0000 7fbd877fe700 1 -- 172.21.15.12:0/853224626 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90008770 con 0x556ee6c87e00 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/853224626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 msgr2=0x556ee6c881e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/853224626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c881e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbd900176e0 tx=0x7fbd90014000 comp rx=0 tx=0).stop 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/853224626 shutdown_connections 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/853224626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c82890 0x556ee6c82cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/853224626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c881e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/853224626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 0x556ee6c82350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/853224626 >> 172.21.15.12:0/853224626 conn(0x556ee6be0210 msgr2=0x556ee6c7cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/853224626 shutdown_connections 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/853224626 wait complete. 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 -- start start 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.504+0000 7fbdb3bc8b80 1 Processor -- start 2024-05-08T23:48:43.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c8e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 0x556ee6c8b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 0x556ee6c8c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6b6e560 con 0x556ee6c82890 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6b6e200 con 0x556ee6c88720 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6a556a0 con 0x556ee6c87e00 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbdb3bc8b80 10 client.0 _ll_register_callbacks cb 0x556ee6b77b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda57aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c8e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda57aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c8e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47770/0 (socket says 172.21.15.12:47770) 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda57aa700 1 -- 172.21.15.12:0/1932380506 learned_addr learned my addr 172.21.15.12:0/1932380506 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda5fab700 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 0x556ee6c8c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 0x556ee6c8b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda57aa700 1 -- 172.21.15.12:0/1932380506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 msgr2=0x556ee6c8b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda57aa700 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 0x556ee6c8b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda57aa700 1 -- 172.21.15.12:0/1932380506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 msgr2=0x556ee6c8c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda57aa700 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 0x556ee6c8c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda57aa700 1 -- 172.21.15.12:0/1932380506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda01a4060 con 0x556ee6c82890 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 0x556ee6c8b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.505+0000 7fbda5fab700 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 0x556ee6c8c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbda57aa700 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c8e720 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000fc60 tx=0x7fbd90016fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7d6e1da19c931e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbd84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbd84ff9700 1 -- 172.21.15.12:0/1932380506 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000e040 con 0x556ee6c82890 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbd84ff9700 1 -- 172.21.15.12:0/1932380506 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fbd9001c040 con 0x556ee6c82890 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1932380506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ee6b2b270 con 0x556ee6c82890 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbd84ff9700 1 -- 172.21.15.12:0/1932380506 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd900159a0 con 0x556ee6c82890 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbdb3bc8b80 10 client.16029 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1932380506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fbd90017440 con 0x556ee6c82890 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbd84ff9700 1 -- 172.21.15.12:0/1932380506 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ee6b2b270 con 0x556ee6c82890 2024-05-08T23:48:43.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbd84ff9700 1 -- 172.21.15.12:0/1932380506 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbd90027020 con 0x556ee6c82890 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbdb3bc8b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbdb3bc8b80 10 client.16029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbdb3bc8b80 20 client.16029 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556ee6c9bf10 0x556ee6c9e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbdb3bc8b80 4 client.16029 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.506+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1932380506 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ee6c51810 con 0x556ee6c9bf10 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.507+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556ee6c9bf10 0x556ee6c9e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.507+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556ee6c9bf10 0x556ee6c9e300 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc9f2d8c7a54dec8 server_cookie=855cec1719ae57fd in_seq=0 out_seq=0 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.507+0000 7fbd84ff9700 10 client.16029 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbd84ff9700 1 -- 172.21.15.12:0/1932380506 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ee6c51810 con 0x556ee6c9bf10 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbd84ff9700 10 client.16029 handle_command_reply: tid=0 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 client.16029 shutdown 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1932380506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 msgr2=0x556ee6c8e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c8e720 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000fc60 tx=0x7fbd90016fc0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1932380506 shutdown_connections 2024-05-08T23:48:43.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556ee6c9bf10 0x556ee6c9e300 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 0x556ee6c8c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c8e720 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/1932380506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 0x556ee6c8b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbd84ff9700 0 client.16029 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:48:43.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1932380506 >> 172.21.15.12:0/1932380506 conn(0x556ee6be0210 msgr2=0x556ee6c7c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1932380506 shutdown_connections 2024-05-08T23:48:43.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.508+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1932380506 wait complete. 2024-05-08T23:48:43.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.509+0000 7fbdb3bc8b80 20 client.16029 trim_cache size 0 max 16384 2024-05-08T23:48:43.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbdb3bc8b80 1 Processor -- start 2024-05-08T23:48:43.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbdb3bc8b80 1 -- start start 2024-05-08T23:48:43.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c82890 0x556ee6c7cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c8e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 0x556ee6c8e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6c8d010 con 0x556ee6c87e00 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6c86410 con 0x556ee6c82890 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6c881e0 con 0x556ee6c88720 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbda4fa9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c8e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbda4fa9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c8e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47776/0 (socket says 172.21.15.12:47776) 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbda4fa9700 1 -- 172.21.15.12:0/1181747547 learned_addr learned my addr 172.21.15.12:0/1181747547 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbda57aa700 1 --2- 172.21.15.12:0/1181747547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c82890 0x556ee6c7cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbda5fab700 1 --2- 172.21.15.12:0/1181747547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 0x556ee6c8e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbda4fa9700 1 -- 172.21.15.12:0/1181747547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 msgr2=0x556ee6c8e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/1181747547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 0x556ee6c8e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbda4fa9700 1 -- 172.21.15.12:0/1181747547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c82890 msgr2=0x556ee6c7cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/1181747547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c82890 0x556ee6c7cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.523+0000 7fbda4fa9700 1 -- 172.21.15.12:0/1181747547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ee6c82f40 con 0x556ee6c87e00 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/1181747547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c8e250 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbd94099ac0 tx=0x7fbd94010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d891b43b2b38b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbd87fff700 1 -- 172.21.15.12:0/1181747547 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd940b1070 con 0x556ee6c87e00 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbd87fff700 1 -- 172.21.15.12:0/1181747547 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd94097c70 con 0x556ee6c87e00 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbd87fff700 1 -- 172.21.15.12:0/1181747547 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94099d90 con 0x556ee6c87e00 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1181747547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 msgr2=0x556ee6c8e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/1181747547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c8e250 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbd94099ac0 tx=0x7fbd94010cd0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1181747547 shutdown_connections 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/1181747547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c88720 0x556ee6c8e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/1181747547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c82890 0x556ee6c7cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/1181747547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c87e00 0x556ee6c8e250 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1181747547 >> 172.21.15.12:0/1181747547 conn(0x556ee6be0210 msgr2=0x556ee6c97c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.524+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1181747547 shutdown_connections 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.525+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/1181747547 wait complete. 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.525+0000 7fbdb3bc8b80 1 -- start start 2024-05-08T23:48:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.525+0000 7fbdb3bc8b80 1 Processor -- start 2024-05-08T23:48:43.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.525+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c93170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.525+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 0x556ee6c936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.525+0000 7fbdb3bc8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 0x556ee6c83ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.525+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6c845d0 con 0x556ee6c82890 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.525+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6c84740 con 0x556ee6c88720 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.525+0000 7fbdb3bc8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6c84900 con 0x556ee6c87e00 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbdb3bc8b80 10 client.0 _ll_register_callbacks cb 0x556ee6b77b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda57aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c93170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda57aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c93170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47788/0 (socket says 172.21.15.12:47788) 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda57aa700 1 -- 172.21.15.12:0/3730843116 learned_addr learned my addr 172.21.15.12:0/3730843116 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda5fab700 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 0x556ee6c83ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 0x556ee6c936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda57aa700 1 -- 172.21.15.12:0/3730843116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 msgr2=0x556ee6c936b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda57aa700 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 0x556ee6c936b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda57aa700 1 -- 172.21.15.12:0/3730843116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 msgr2=0x556ee6c83ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda57aa700 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 0x556ee6c83ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda57aa700 1 -- 172.21.15.12:0/3730843116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda019c820 con 0x556ee6c82890 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 0x556ee6c936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbda57aa700 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c93170 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000fc60 tx=0x7fbd90017cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff91ca3d7aa5c690 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbd84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbd84ff9700 1 -- 172.21.15.12:0/3730843116 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000e070 con 0x556ee6c82890 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbd84ff9700 1 -- 172.21.15.12:0/3730843116 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fbd90009250 con 0x556ee6c82890 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.526+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/3730843116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd9000f040 con 0x556ee6c82890 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbd84ff9700 1 -- 172.21.15.12:0/3730843116 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd90009410 con 0x556ee6c82890 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbdb3bc8b80 10 client.16035 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/3730843116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556ee69ffb60 con 0x556ee6c82890 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbd84ff9700 1 -- 172.21.15.12:0/3730843116 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd9000f040 con 0x556ee6c82890 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbd84ff9700 1 -- 172.21.15.12:0/3730843116 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbd90027070 con 0x556ee6c82890 2024-05-08T23:48:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbdb3bc8b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbdb3bc8b80 10 client.16035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbdb3bc8b80 20 client.16035 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556ee6cf3240 0x556ee6cf5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbdb3bc8b80 4 client.16035 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/3730843116 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556ee6bcaec0 con 0x556ee6cf3240 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.527+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556ee6cf3240 0x556ee6cf5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbd84ff9700 10 client.16035 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbda4fa9700 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556ee6cf3240 0x556ee6cf5660 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=225142c400a83f28 server_cookie=d61c8074f9fc6cef in_seq=0 out_seq=0 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbd84ff9700 1 -- 172.21.15.12:0/3730843116 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556ee6bcaec0 con 0x556ee6cf3240 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbd84ff9700 10 client.16035 handle_command_reply: tid=0 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 client.16035 shutdown 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/3730843116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 msgr2=0x556ee6c93170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c93170 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000fc60 tx=0x7fbd90017cd0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/3730843116 shutdown_connections 2024-05-08T23:48:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556ee6cf3240 0x556ee6cf5660 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ee6c88720 0x556ee6c83ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ee6c82890 0x556ee6c93170 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbd84ff9700 0 client.16035 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:48:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 --2- 172.21.15.12:0/3730843116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ee6c87e00 0x556ee6c936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/3730843116 >> 172.21.15.12:0/3730843116 conn(0x556ee6be0210 msgr2=0x556ee6c86a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/3730843116 shutdown_connections 2024-05-08T23:48:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 1 -- 172.21.15.12:0/3730843116 wait complete. 2024-05-08T23:48:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.528+0000 7fbdb3bc8b80 20 client.16035 trim_cache size 0 max 16384 2024-05-08T23:48:43.531 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:48:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 -- 172.21.15.12:0/4257821273 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbd880a1310 msgr2=0x7fbd880a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbd880a1310 0x7fbd880a37d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbda0067bb0 tx=0x7fbd9000b040 comp rx=0 tx=0).stop 2024-05-08T23:48:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 -- 172.21.15.12:0/4257821273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbda0100ab0 msgr2=0x7fbda0197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbda0100ab0 0x7fbda0197880 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400d950 tx=0x7fbd9400dc60 comp rx=0 tx=0).stop 2024-05-08T23:48:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 -- 172.21.15.12:0/4257821273 shutdown_connections 2024-05-08T23:48:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbd880a1310 0x7fbd880a37d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbda0105f90 0x7fbda019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda01000f0 0x7fbda0197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 --2- 172.21.15.12:0/4257821273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbda0100ab0 0x7fbda0197880 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 -- 172.21.15.12:0/4257821273 >> 172.21.15.12:0/4257821273 conn(0x7fbda0076ba0 msgr2=0x7fbda0069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 -- 172.21.15.12:0/4257821273 shutdown_connections 2024-05-08T23:48:43.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.529+0000 7fbda7a0e700 1 -- 172.21.15.12:0/4257821273 wait complete. 2024-05-08T23:48:43.538 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:48:43.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.681+0000 7fcbe724a700 1 -- 172.21.15.12:0/2724904820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbe0106290 msgr2=0x7fcbe010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.681+0000 7fcbe724a700 1 --2- 172.21.15.12:0/2724904820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbe0106290 0x7fcbe010de10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800b0d0 tx=0x7fcbd800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.681+0000 7fcbe724a700 1 -- 172.21.15.12:0/2724904820 shutdown_connections 2024-05-08T23:48:43.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.681+0000 7fcbe724a700 1 --2- 172.21.15.12:0/2724904820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbe0106290 0x7fcbe010de10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.681+0000 7fcbe724a700 1 --2- 172.21.15.12:0/2724904820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe00682e0 0x7fcbe00686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.681+0000 7fcbe724a700 1 --2- 172.21.15.12:0/2724904820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbe0068c90 0x7fcbe0105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.681+0000 7fcbe724a700 1 -- 172.21.15.12:0/2724904820 >> 172.21.15.12:0/2724904820 conn(0x7fcbe0076f40 msgr2=0x7fcbe0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.682+0000 7fcbe724a700 1 -- 172.21.15.12:0/2724904820 shutdown_connections 2024-05-08T23:48:43.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.682+0000 7fcbe724a700 1 -- 172.21.15.12:0/2724904820 wait complete. 2024-05-08T23:48:43.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.682+0000 7fcbe724a700 1 Processor -- start 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.682+0000 7fcbe724a700 1 -- start start 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.682+0000 7fcbe724a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe00682e0 0x7fcbe0197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.682+0000 7fcbe724a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbe0106290 0x7fcbe01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.682+0000 7fcbe724a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbe019bd60 0x7fcbe019c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe724a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe019c7e0 con 0x7fcbe019bd60 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe724a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe019c950 con 0x7fcbe00682e0 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe724a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe019cb10 con 0x7fcbe0106290 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe57e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbe019bd60 0x7fcbe019c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe57e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbe019bd60 0x7fcbe019c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47802/0 (socket says 172.21.15.12:47802) 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe57e7700 1 -- 172.21.15.12:0/4055360757 learned_addr learned my addr 172.21.15.12:0/4055360757 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe00682e0 0x7fcbe0197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbe0106290 0x7fcbe01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe57e7700 1 -- 172.21.15.12:0/4055360757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbe0106290 msgr2=0x7fcbe01979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe57e7700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbe0106290 0x7fcbe01979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe57e7700 1 -- 172.21.15.12:0/4055360757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe00682e0 msgr2=0x7fcbe0197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe57e7700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe00682e0 0x7fcbe0197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe57e7700 1 -- 172.21.15.12:0/4055360757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0009520 con 0x7fcbe019bd60 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe57e7700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbe019bd60 0x7fcbe019c1e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800bd50 tx=0x7fcbd800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbd5ffb700 1 -- 172.21.15.12:0/4055360757 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd800e040 con 0x7fcbe019bd60 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbd5ffb700 1 -- 172.21.15.12:0/4055360757 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbd801b8e0 con 0x7fcbe019bd60 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbe724a700 1 -- 172.21.15.12:0/4055360757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd8009d00 con 0x7fcbe019bd60 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.683+0000 7fcbd5ffb700 1 -- 172.21.15.12:0/4055360757 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8012810 con 0x7fcbe019bd60 2024-05-08T23:48:43.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.684+0000 7fcbe724a700 1 -- 172.21.15.12:0/4055360757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe01a4300 con 0x7fcbe019bd60 2024-05-08T23:48:43.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.686+0000 7fcbd5ffb700 1 -- 172.21.15.12:0/4055360757 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcbd801ba50 con 0x7fcbe019bd60 2024-05-08T23:48:43.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.687+0000 7fcbd5ffb700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbc80a1290 0x7fcbc80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.687+0000 7fcbd5ffb700 1 -- 172.21.15.12:0/4055360757 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcbd80ce0a0 con 0x7fcbe019bd60 2024-05-08T23:48:43.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.687+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbc80a1290 0x7fcbc80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.687+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbc80a1290 0x7fcbc80a3750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009c60 tx=0x7fcbdc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbf3404b80 1 Processor -- start 2024-05-08T23:48:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbf3404b80 1 -- start start 2024-05-08T23:48:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e731e00 0x559d8e7321e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e732720 0x559d8e72c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e72c890 0x559d8e72ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e683150 con 0x559d8e732720 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e680b10 con 0x559d8e731e00 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e67c400 con 0x559d8e72c890 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbd7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e732720 0x559d8e72c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbd7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e732720 0x559d8e72c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47808/0 (socket says 172.21.15.12:47808) 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbd7fff700 1 -- 172.21.15.12:0/3047802349 learned_addr learned my addr 172.21.15.12:0/3047802349 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/3047802349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e731e00 0x559d8e7321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbe57e7700 1 --2- 172.21.15.12:0/3047802349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e72c890 0x559d8e72ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbd7fff700 1 -- 172.21.15.12:0/3047802349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e72c890 msgr2=0x559d8e72ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/3047802349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e72c890 0x559d8e72ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbd7fff700 1 -- 172.21.15.12:0/3047802349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e731e00 msgr2=0x559d8e7321e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/3047802349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e731e00 0x559d8e7321e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbd7fff700 1 -- 172.21.15.12:0/3047802349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d8e4fd3b0 con 0x559d8e732720 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.688+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/3047802349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e731e00 0x559d8e7321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.689+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/3047802349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e732720 0x559d8e72c350 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009e40 tx=0x7fcbd0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127929db5771c08d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.689+0000 7fcbc6ffd700 1 -- 172.21.15.12:0/3047802349 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0004250 con 0x559d8e732720 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.689+0000 7fcbc6ffd700 1 -- 172.21.15.12:0/3047802349 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbd00043b0 con 0x559d8e732720 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.689+0000 7fcbc6ffd700 1 -- 172.21.15.12:0/3047802349 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd00198e0 con 0x559d8e732720 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.689+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3047802349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e732720 msgr2=0x559d8e72c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.689+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/3047802349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e732720 0x559d8e72c350 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009e40 tx=0x7fcbd0011720 comp rx=0 tx=0).stop 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3047802349 shutdown_connections 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/3047802349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e72c890 0x559d8e72ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/3047802349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e731e00 0x559d8e7321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/3047802349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e732720 0x559d8e72c350 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3047802349 >> 172.21.15.12:0/3047802349 conn(0x559d8e68a210 msgr2=0x559d8e726df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3047802349 shutdown_connections 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3047802349 wait complete. 2024-05-08T23:48:43.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 -- start start 2024-05-08T23:48:43.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 Processor -- start 2024-05-08T23:48:43.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e735b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.690+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e731e00 0x559d8e7360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e7365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e618560 con 0x559d8e72c890 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e618200 con 0x559d8e732720 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e4ff6a0 con 0x559d8e731e00 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbf3404b80 10 client.0 _ll_register_callbacks cb 0x559d8e621b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe4fe6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e735b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe4fe6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e735b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47816/0 (socket says 172.21.15.12:47816) 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe4fe6700 1 -- 172.21.15.12:0/3513515484 learned_addr learned my addr 172.21.15.12:0/3513515484 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e731e00 0x559d8e7360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe57e7700 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e7365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe4fe6700 1 -- 172.21.15.12:0/3513515484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e731e00 msgr2=0x559d8e7360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e731e00 0x559d8e7360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe4fe6700 1 -- 172.21.15.12:0/3513515484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 msgr2=0x559d8e7365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e7365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe4fe6700 1 -- 172.21.15.12:0/3513515484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd0009a70 con 0x559d8e72c890 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe57e7700 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e7365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e735b70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc003980 tx=0x7fcbdc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426d0d5401592104 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbbffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbbffff700 1 -- 172.21.15.12:0/3513515484 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc01eab0 con 0x559d8e72c890 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3513515484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d8e5d5270 con 0x559d8e72c890 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.691+0000 7fcbbffff700 1 -- 172.21.15.12:0/3513515484 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcbdc01ec10 con 0x559d8e72c890 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbbffff700 1 -- 172.21.15.12:0/3513515484 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbdc01edd0 con 0x559d8e72c890 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbf3404b80 10 client.16047 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3513515484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559d8e4a9b60 con 0x559d8e72c890 2024-05-08T23:48:43.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbbffff700 1 -- 172.21.15.12:0/3513515484 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d8e5d5270 con 0x559d8e72c890 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbbffff700 1 -- 172.21.15.12:0/3513515484 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcbdc01c430 con 0x559d8e72c890 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbf3404b80 10 client.16047 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbf3404b80 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbf3404b80 20 client.16047 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559d8e746310 0x559d8e748730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbf3404b80 4 client.16047 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3513515484 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d8e6fb810 con 0x559d8e746310 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.692+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559d8e746310 0x559d8e748730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.693+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559d8e746310 0x559d8e748730 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d06ba9c1570c8a3 server_cookie=92afbe61d53b9056 in_seq=0 out_seq=0 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.693+0000 7fcbbffff700 10 client.16047 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbbffff700 1 -- 172.21.15.12:0/3513515484 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d8e6fb810 con 0x559d8e746310 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbbffff700 10 client.16047 handle_command_reply: tid=0 2024-05-08T23:48:43.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 client.16047 shutdown 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3513515484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 msgr2=0x559d8e735b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e735b70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc003980 tx=0x7fcbdc0047c0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3513515484 shutdown_connections 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559d8e746310 0x559d8e748730 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e7365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e735b70 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbbffff700 0 client.16047 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/3513515484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e731e00 0x559d8e7360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3513515484 >> 172.21.15.12:0/3513515484 conn(0x559d8e68a210 msgr2=0x559d8e7265a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3513515484 shutdown_connections 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 1 -- 172.21.15.12:0/3513515484 wait complete. 2024-05-08T23:48:43.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.694+0000 7fcbf3404b80 20 client.16047 trim_cache size 0 max 16384 2024-05-08T23:48:43.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbf3404b80 1 Processor -- start 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbf3404b80 1 -- start start 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e741c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e731e00 0x559d8e726d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e7360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e7287d0 con 0x559d8e72c890 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e72c6d0 con 0x559d8e732720 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e74d940 con 0x559d8e731e00 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbe4fe6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e741c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbe4fe6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e741c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47818/0 (socket says 172.21.15.12:47818) 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.708+0000 7fcbe4fe6700 1 -- 172.21.15.12:0/1651425427 learned_addr learned my addr 172.21.15.12:0/1651425427 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbe57e7700 1 --2- 172.21.15.12:0/1651425427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e7360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/1651425427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e731e00 0x559d8e726d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbe4fe6700 1 -- 172.21.15.12:0/1651425427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e731e00 msgr2=0x559d8e726d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/1651425427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e731e00 0x559d8e726d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbe4fe6700 1 -- 172.21.15.12:0/1651425427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 msgr2=0x559d8e7360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/1651425427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e7360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbe4fe6700 1 -- 172.21.15.12:0/1651425427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d8e642560 con 0x559d8e72c890 2024-05-08T23:48:43.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/1651425427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e741c20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009b20 tx=0x7fcbdc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb1628fd0b7f5c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbc77fe700 1 -- 172.21.15.12:0/1651425427 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc01c430 con 0x559d8e72c890 2024-05-08T23:48:43.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbc77fe700 1 -- 172.21.15.12:0/1651425427 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbdc01c590 con 0x559d8e72c890 2024-05-08T23:48:43.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.709+0000 7fcbc77fe700 1 -- 172.21.15.12:0/1651425427 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc027440 con 0x559d8e72c890 2024-05-08T23:48:43.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 -- 172.21.15.12:0/1651425427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 msgr2=0x559d8e741c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/1651425427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e741c20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009b20 tx=0x7fcbdc011ab0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 -- 172.21.15.12:0/1651425427 shutdown_connections 2024-05-08T23:48:43.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/1651425427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e7360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/1651425427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e72c890 0x559d8e741c20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/1651425427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e731e00 0x559d8e726d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 -- 172.21.15.12:0/1651425427 >> 172.21.15.12:0/1651425427 conn(0x559d8e68a210 msgr2=0x559d8e723210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 -- 172.21.15.12:0/1651425427 shutdown_connections 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 -- 172.21.15.12:0/1651425427 wait complete. 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 -- start start 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.710+0000 7fcbf3404b80 1 Processor -- start 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e72c890 0x559d8e73b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e731e00 0x559d8e73b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbf3404b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e79b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e73be20 con 0x559d8e731e00 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e73bf90 con 0x559d8e732720 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbf3404b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8e79bd50 con 0x559d8e72c890 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbf3404b80 10 client.0 _ll_register_callbacks cb 0x559d8e621b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbd7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e731e00 0x559d8e73b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbd7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e731e00 0x559d8e73b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47824/0 (socket says 172.21.15.12:47824) 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbd7fff700 1 -- 172.21.15.12:0/2873751206 learned_addr learned my addr 172.21.15.12:0/2873751206 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:43.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e72c890 0x559d8e73b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbe57e7700 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e79b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbd7fff700 1 -- 172.21.15.12:0/2873751206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e72c890 msgr2=0x559d8e73b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e72c890 0x559d8e73b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbd7fff700 1 -- 172.21.15.12:0/2873751206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 msgr2=0x559d8e79b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e79b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbd7fff700 1 -- 172.21.15.12:0/2873751206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d8e4a9b60 con 0x559d8e731e00 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.711+0000 7fcbd7fff700 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e731e00 0x559d8e73b690 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0011a70 tx=0x7fcbd0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8de9f22bbdc906 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbbffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbbffff700 1 -- 172.21.15.12:0/2873751206 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd00040d0 con 0x559d8e731e00 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbbffff700 1 -- 172.21.15.12:0/2873751206 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcbd0003680 con 0x559d8e731e00 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbf3404b80 1 -- 172.21.15.12:0/2873751206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d8e674ec0 con 0x559d8e731e00 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbbffff700 1 -- 172.21.15.12:0/2873751206 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbd0004230 con 0x559d8e731e00 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbf3404b80 10 client.16053 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:43.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbf3404b80 1 -- 172.21.15.12:0/2873751206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559d8e501cd0 con 0x559d8e731e00 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbbffff700 1 -- 172.21.15.12:0/2873751206 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d8e674ec0 con 0x559d8e731e00 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbbffff700 1 -- 172.21.15.12:0/2873751206 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcbd00046c0 con 0x559d8e731e00 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbf3404b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbf3404b80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbf3404b80 20 client.16053 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559d8e79d5a0 0x559d8e79f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbf3404b80 4 client.16053 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.712+0000 7fcbf3404b80 1 -- 172.21.15.12:0/2873751206 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d8e4cacf0 con 0x559d8e79d5a0 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559d8e79d5a0 0x559d8e79f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbbffff700 10 client.16053 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbe4fe6700 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559d8e79d5a0 0x559d8e79f9c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9423e6ed31c58d7 server_cookie=d3ed14be1428a2cb in_seq=0 out_seq=0 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbbffff700 1 -- 172.21.15.12:0/2873751206 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d8e4cacf0 con 0x559d8e79d5a0 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbbffff700 10 client.16053 handle_command_reply: tid=0 2024-05-08T23:48:43.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbf3404b80 1 client.16053 shutdown 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbf3404b80 1 -- 172.21.15.12:0/2873751206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e731e00 msgr2=0x559d8e73b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e731e00 0x559d8e73b690 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0011a70 tx=0x7fcbd0011b40 comp rx=0 tx=0).stop 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbf3404b80 1 -- 172.21.15.12:0/2873751206 shutdown_connections 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559d8e79d5a0 0x559d8e79f9c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d8e732720 0x559d8e79b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d8e72c890 0x559d8e73b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbbffff700 0 client.16053 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbf3404b80 1 --2- 172.21.15.12:0/2873751206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d8e731e00 0x559d8e73b690 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.713+0000 7fcbf3404b80 1 -- 172.21.15.12:0/2873751206 >> 172.21.15.12:0/2873751206 conn(0x559d8e68a210 msgr2=0x559d8e734420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbf3404b80 1 -- 172.21.15.12:0/2873751206 shutdown_connections 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbf3404b80 1 -- 172.21.15.12:0/2873751206 wait complete. 2024-05-08T23:48:43.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbf3404b80 20 client.16053 trim_cache size 0 max 16384 2024-05-08T23:48:43.715 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:48:43.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 -- 172.21.15.12:0/4055360757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbc80a1290 msgr2=0x7fcbc80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbc80a1290 0x7fcbc80a3750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009c60 tx=0x7fcbdc009260 comp rx=0 tx=0).stop 2024-05-08T23:48:43.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 -- 172.21.15.12:0/4055360757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbe019bd60 msgr2=0x7fcbe019c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:43.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbe019bd60 0x7fcbe019c1e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800bd50 tx=0x7fcbd800bd80 comp rx=0 tx=0).stop 2024-05-08T23:48:43.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 -- 172.21.15.12:0/4055360757 shutdown_connections 2024-05-08T23:48:43.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbc80a1290 0x7fcbc80a3750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcbe019bd60 0x7fcbe019c1e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbe00682e0 0x7fcbe0197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 --2- 172.21.15.12:0/4055360757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbe0106290 0x7fcbe01979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:43.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 -- 172.21.15.12:0/4055360757 >> 172.21.15.12:0/4055360757 conn(0x7fcbe0076f40 msgr2=0x7fcbe00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:43.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.714+0000 7fcbe724a700 1 -- 172.21.15.12:0/4055360757 shutdown_connections 2024-05-08T23:48:43.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:43.715+0000 7fcbe724a700 1 -- 172.21.15.12:0/4055360757 wait complete. 2024-05-08T23:48:43.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:43 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/4261400856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:48:43.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:43 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/4261400856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:48:43.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 59985 8.72 MB/sec warmup 39 sec latency 951.834 ms 2024-05-08T23:48:44.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:44 smithi012 ceph-mon[42503]: pgmap v73: 97 pgs: 97 active+clean; 26 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 5.7 MiB/s wr, 190 op/s 2024-05-08T23:48:44.725 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:48:44.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:48:44.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:44 smithi204 ceph-mon[47373]: pgmap v73: 97 pgs: 97 active+clean; 26 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 5.7 MiB/s wr, 190 op/s 2024-05-08T23:48:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:44 smithi160 ceph-mon[46152]: pgmap v73: 97 pgs: 97 active+clean; 26 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 5.7 MiB/s wr, 190 op/s 2024-05-08T23:48:44.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 59985 8.51 MB/sec warmup 40 sec latency 1951.903 ms 2024-05-08T23:48:44.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.880+0000 7f856ed8f700 1 -- 172.21.15.12:0/1073594397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568105f90 msgr2=0x7f8568106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.880+0000 7f856ed8f700 1 --2- 172.21.15.12:0/1073594397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568105f90 0x7f8568106370 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f85640099c0 tx=0x7f8564009cd0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.881+0000 7f856ed8f700 1 -- 172.21.15.12:0/1073594397 shutdown_connections 2024-05-08T23:48:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.881+0000 7f856ed8f700 1 --2- 172.21.15.12:0/1073594397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568100ab0 0x7f856810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.881+0000 7f856ed8f700 1 --2- 172.21.15.12:0/1073594397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568105f90 0x7f8568106370 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.881+0000 7f856ed8f700 1 --2- 172.21.15.12:0/1073594397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85681000f0 0x7f8568100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.881+0000 7f856ed8f700 1 -- 172.21.15.12:0/1073594397 >> 172.21.15.12:0/1073594397 conn(0x7f8568076ba0 msgr2=0x7f8568076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.881+0000 7f856ed8f700 1 -- 172.21.15.12:0/1073594397 shutdown_connections 2024-05-08T23:48:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.881+0000 7f856ed8f700 1 -- 172.21.15.12:0/1073594397 wait complete. 2024-05-08T23:48:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.881+0000 7f856ed8f700 1 Processor -- start 2024-05-08T23:48:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.881+0000 7f856ed8f700 1 -- start start 2024-05-08T23:48:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f856ed8f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85681000f0 0x7f8568197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f856ed8f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100ab0 0x7f8568197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f856ed8f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568105f90 0x7f856819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f856ed8f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856819c240 con 0x7f8568100ab0 2024-05-08T23:48:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f856ed8f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856819c3b0 con 0x7f85681000f0 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f856ed8f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856819c570 con 0x7f8568105f90 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100ab0 0x7f8568197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100ab0 0x7f8568197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47828/0 (socket says 172.21.15.12:47828) 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855ffff700 1 -- 172.21.15.12:0/1494464349 learned_addr learned my addr 172.21.15.12:0/1494464349 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f856cb2b700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85681000f0 0x7f8568197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f856d32c700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568105f90 0x7f856819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855ffff700 1 -- 172.21.15.12:0/1494464349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568105f90 msgr2=0x7f856819bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855ffff700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568105f90 0x7f856819bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855ffff700 1 -- 172.21.15.12:0/1494464349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85681000f0 msgr2=0x7f8568197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855ffff700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85681000f0 0x7f8568197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855ffff700 1 -- 172.21.15.12:0/1494464349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85640095f0 con 0x7f8568100ab0 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855ffff700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100ab0 0x7f8568197860 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f855800e9a0 tx=0x7f855800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855dffb700 1 -- 172.21.15.12:0/1494464349 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558009790 con 0x7f8568100ab0 2024-05-08T23:48:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855dffb700 1 -- 172.21.15.12:0/1494464349 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f855800cbd0 con 0x7f8568100ab0 2024-05-08T23:48:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f856ed8f700 1 -- 172.21.15.12:0/1494464349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f856819c850 con 0x7f8568100ab0 2024-05-08T23:48:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.882+0000 7f855dffb700 1 -- 172.21.15.12:0/1494464349 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558018680 con 0x7f8568100ab0 2024-05-08T23:48:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.883+0000 7f856ed8f700 1 -- 172.21.15.12:0/1494464349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85681a4090 con 0x7f8568100ab0 2024-05-08T23:48:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.885+0000 7f855dffb700 1 -- 172.21.15.12:0/1494464349 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f855800cd40 con 0x7f8568100ab0 2024-05-08T23:48:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.885+0000 7f855dffb700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85500a1290 0x7f85500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.886+0000 7f855dffb700 1 -- 172.21.15.12:0/1494464349 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8558014070 con 0x7f8568100ab0 2024-05-08T23:48:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.886+0000 7f856cb2b700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85500a1290 0x7f85500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.886+0000 7f856cb2b700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85500a1290 0x7f85500a3750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8564009990 tx=0x7f8564019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.886+0000 7f857af49b80 1 Processor -- start 2024-05-08T23:48:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.886+0000 7f857af49b80 1 -- start start 2024-05-08T23:48:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f857af49b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f857af49b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b47a0 0x55b2965ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f857af49b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965aed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f857af49b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2965051d0 con 0x55b2965ae910 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f857af49b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b296502b90 con 0x55b2965b3e80 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f857af49b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2964fe480 con 0x55b2965b47a0 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f856d32c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f855ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b47a0 0x55b2965ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f856d32c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47840/0 (socket says 172.21.15.12:47840) 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f856d32c700 1 -- 172.21.15.12:0/189102399 learned_addr learned my addr 172.21.15.12:0/189102399 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f856cb2b700 1 --2- 172.21.15.12:0/189102399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f856d32c700 1 -- 172.21.15.12:0/189102399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b47a0 msgr2=0x55b2965ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f856d32c700 1 --2- 172.21.15.12:0/189102399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b47a0 0x55b2965ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f856d32c700 1 -- 172.21.15.12:0/189102399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 msgr2=0x55b2965b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f856d32c700 1 --2- 172.21.15.12:0/189102399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f856d32c700 1 -- 172.21.15.12:0/189102399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b29637f430 con 0x55b2965ae910 2024-05-08T23:48:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.887+0000 7f856d32c700 1 --2- 172.21.15.12:0/189102399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965aed50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8560009440 tx=0x7f8560009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e5389d6e3cb213 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.888+0000 7f854effd700 1 -- 172.21.15.12:0/189102399 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856000e070 con 0x55b2965ae910 2024-05-08T23:48:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.888+0000 7f854effd700 1 -- 172.21.15.12:0/189102399 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f856000f040 con 0x55b2965ae910 2024-05-08T23:48:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.888+0000 7f854effd700 1 -- 172.21.15.12:0/189102399 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560015ac0 con 0x55b2965ae910 2024-05-08T23:48:44.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.888+0000 7f857af49b80 1 -- 172.21.15.12:0/189102399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 msgr2=0x55b2965aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:44.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.888+0000 7f857af49b80 1 --2- 172.21.15.12:0/189102399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965aed50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8560009440 tx=0x7f8560009920 comp rx=0 tx=0).stop 2024-05-08T23:48:44.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.888+0000 7f857af49b80 1 -- 172.21.15.12:0/189102399 shutdown_connections 2024-05-08T23:48:44.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.888+0000 7f857af49b80 1 --2- 172.21.15.12:0/189102399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965aed50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.888+0000 7f857af49b80 1 --2- 172.21.15.12:0/189102399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.888+0000 7f857af49b80 1 --2- 172.21.15.12:0/189102399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b47a0 0x55b2965ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.888+0000 7f857af49b80 1 -- 172.21.15.12:0/189102399 >> 172.21.15.12:0/189102399 conn(0x55b29650c290 msgr2=0x55b2965a8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.889+0000 7f857af49b80 1 -- 172.21.15.12:0/189102399 shutdown_connections 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.889+0000 7f857af49b80 1 -- 172.21.15.12:0/189102399 wait complete. 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.889+0000 7f857af49b80 1 -- start start 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.889+0000 7f857af49b80 1 Processor -- start 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.889+0000 7f857af49b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965ae910 0x55b2965ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.889+0000 7f857af49b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.889+0000 7f857af49b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.889+0000 7f857af49b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29649a5e0 con 0x55b2965b47a0 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.889+0000 7f857af49b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29649a280 con 0x55b2965b3e80 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.889+0000 7f857af49b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b296381720 con 0x55b2965ae910 2024-05-08T23:48:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f857af49b80 10 client.0 _ll_register_callbacks cb 0x55b2964a3bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f856d32c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f856d32c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47852/0 (socket says 172.21.15.12:47852) 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f856d32c700 1 -- 172.21.15.12:0/1268339921 learned_addr learned my addr 172.21.15.12:0/1268339921 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f855ffff700 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f856cb2b700 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965ae910 0x55b2965ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f856d32c700 1 -- 172.21.15.12:0/1268339921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965ae910 msgr2=0x55b2965ba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f856d32c700 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965ae910 0x55b2965ba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f856d32c700 1 -- 172.21.15.12:0/1268339921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 msgr2=0x55b2965b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f856d32c700 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f856d32c700 1 -- 172.21.15.12:0/1268339921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85600090f0 con 0x55b2965b47a0 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f855ffff700 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f856d32c700 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b81d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f85600156a0 tx=0x7f8560016dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c4ad17c76da4d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f8547fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f8547fff700 1 -- 172.21.15.12:0/1268339921 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856000e040 con 0x55b2965b47a0 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f8547fff700 1 -- 172.21.15.12:0/1268339921 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f85600095e0 con 0x55b2965b47a0 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.890+0000 7f857af49b80 1 -- 172.21.15.12:0/1268339921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2964572f0 con 0x55b2965b47a0 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f8547fff700 1 -- 172.21.15.12:0/1268339921 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8560004ce0 con 0x55b2965b47a0 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f857af49b80 10 client.16065 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f857af49b80 1 -- 172.21.15.12:0/1268339921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b29632bbe0 con 0x55b2965b47a0 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f8547fff700 1 -- 172.21.15.12:0/1268339921 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b2964572f0 con 0x55b2965b47a0 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f8547fff700 1 -- 172.21.15.12:0/1268339921 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f856001c070 con 0x55b2965b47a0 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f857af49b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f857af49b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f857af49b80 20 client.16065 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f857af49b80 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b2965c81c0 0x55b2965ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f857af49b80 4 client.16065 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f857af49b80 1 -- 172.21.15.12:0/1268339921 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b29657d890 con 0x55b2965c81c0 2024-05-08T23:48:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.891+0000 7f856cb2b700 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b2965c81c0 0x55b2965ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.892+0000 7f8547fff700 10 client.16065 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.892+0000 7f856cb2b700 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b2965c81c0 0x55b2965ca5e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272a1fede0846e9f server_cookie=857087fba81f559e in_seq=0 out_seq=0 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.892+0000 7f8547fff700 1 -- 172.21.15.12:0/1268339921 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b29657d890 con 0x55b2965c81c0 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.892+0000 7f8547fff700 10 client.16065 handle_command_reply: tid=0 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.892+0000 7f857af49b80 1 client.16065 shutdown 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 1 -- 172.21.15.12:0/1268339921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 msgr2=0x55b2965b81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b81d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f85600156a0 tx=0x7f8560016dd0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 1 -- 172.21.15.12:0/1268339921 shutdown_connections 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b81d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965ae910 0x55b2965ba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b2965c81c0 0x55b2965ca5e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 1 --2- 172.21.15.12:0/1268339921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f8547fff700 0 client.16065 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 1 -- 172.21.15.12:0/1268339921 >> 172.21.15.12:0/1268339921 conn(0x55b29650c290 msgr2=0x55b2965a8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:44.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 1 -- 172.21.15.12:0/1268339921 shutdown_connections 2024-05-08T23:48:44.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 1 -- 172.21.15.12:0/1268339921 wait complete. 2024-05-08T23:48:44.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.893+0000 7f857af49b80 20 client.16065 trim_cache size 0 max 16384 2024-05-08T23:48:44.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.911+0000 7f857af49b80 1 Processor -- start 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.911+0000 7f857af49b80 1 -- start start 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.911+0000 7f857af49b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.911+0000 7f857af49b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965a8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.911+0000 7f857af49b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b47a0 0x55b2965b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.911+0000 7f857af49b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2965c4d80 con 0x55b2965ae910 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.911+0000 7f857af49b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2965a9990 con 0x55b2965b3e80 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.911+0000 7f857af49b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2965b45e0 con 0x55b2965b47a0 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f856cb2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f856cb2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47854/0 (socket says 172.21.15.12:47854) 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f856cb2b700 1 -- 172.21.15.12:0/3184861230 learned_addr learned my addr 172.21.15.12:0/3184861230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f856cb2b700 1 -- 172.21.15.12:0/3184861230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b47a0 msgr2=0x55b2965b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f856d32c700 1 --2- 172.21.15.12:0/3184861230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b47a0 0x55b2965b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f855ffff700 1 --2- 172.21.15.12:0/3184861230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965a8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f856cb2b700 1 --2- 172.21.15.12:0/3184861230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b47a0 0x55b2965b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f856cb2b700 1 -- 172.21.15.12:0/3184861230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 msgr2=0x55b2965a8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f856cb2b700 1 --2- 172.21.15.12:0/3184861230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965a8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f856cb2b700 1 -- 172.21.15.12:0/3184861230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2964c45e0 con 0x55b2965ae910 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f855ffff700 1 --2- 172.21.15.12:0/3184861230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965a8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f856cb2b700 1 --2- 172.21.15.12:0/3184861230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965ae3d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f856400a010 tx=0x7f856401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16445b39ea716542 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f854f7fe700 1 -- 172.21.15.12:0/3184861230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85640055f0 con 0x55b2965ae910 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f854f7fe700 1 -- 172.21.15.12:0/3184861230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f856401d040 con 0x55b2965ae910 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.912+0000 7f854f7fe700 1 -- 172.21.15.12:0/3184861230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564035350 con 0x55b2965ae910 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 -- 172.21.15.12:0/3184861230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 msgr2=0x55b2965ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 --2- 172.21.15.12:0/3184861230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965ae3d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f856400a010 tx=0x7f856401f000 comp rx=0 tx=0).stop 2024-05-08T23:48:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 -- 172.21.15.12:0/3184861230 shutdown_connections 2024-05-08T23:48:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 --2- 172.21.15.12:0/3184861230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b47a0 0x55b2965b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 --2- 172.21.15.12:0/3184861230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965ae910 0x55b2965ae3d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 --2- 172.21.15.12:0/3184861230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965b3e80 0x55b2965a8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 -- 172.21.15.12:0/3184861230 >> 172.21.15.12:0/3184861230 conn(0x55b29650c290 msgr2=0x55b2965b63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 -- 172.21.15.12:0/3184861230 shutdown_connections 2024-05-08T23:48:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 -- 172.21.15.12:0/3184861230 wait complete. 2024-05-08T23:48:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 -- start start 2024-05-08T23:48:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.913+0000 7f857af49b80 1 Processor -- start 2024-05-08T23:48:44.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f857af49b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965ae910 0x55b2965b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f857af49b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b3e80 0x55b2965b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f857af49b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f857af49b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b296622c60 con 0x55b2965b47a0 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f857af49b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b296622dd0 con 0x55b2965ae910 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f857af49b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b296622f90 con 0x55b2965b3e80 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f857af49b80 10 client.0 _ll_register_callbacks cb 0x7f856400b910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f856d32c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f856d32c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47868/0 (socket says 172.21.15.12:47868) 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f856d32c700 1 -- 172.21.15.12:0/1232234402 learned_addr learned my addr 172.21.15.12:0/1232234402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f856cb2b700 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965ae910 0x55b2965b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.914+0000 7f855ffff700 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b3e80 0x55b2965b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f856d32c700 1 -- 172.21.15.12:0/1232234402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b3e80 msgr2=0x55b2965b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f856d32c700 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b3e80 0x55b2965b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f856d32c700 1 -- 172.21.15.12:0/1232234402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965ae910 msgr2=0x55b2965b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f856d32c700 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965ae910 0x55b2965b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f856d32c700 1 -- 172.21.15.12:0/1232234402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856810d550 con 0x55b2965b47a0 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f856d32c700 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b1170 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f85600052a0 tx=0x7f8560015590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8511972077fc3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f8547fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f8547fff700 1 -- 172.21.15.12:0/1232234402 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856000e070 con 0x55b2965b47a0 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f8547fff700 1 -- 172.21.15.12:0/1232234402 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8560016e40 con 0x55b2965b47a0 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f857af49b80 1 -- 172.21.15.12:0/1232234402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f856000f040 con 0x55b2965b47a0 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f8547fff700 1 -- 172.21.15.12:0/1232234402 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8560005510 con 0x55b2965b47a0 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f857af49b80 10 client.16071 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f857af49b80 1 -- 172.21.15.12:0/1232234402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b296383d50 con 0x55b2965b47a0 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.915+0000 7f8547fff700 1 -- 172.21.15.12:0/1232234402 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f856003f080 con 0x55b2965b47a0 2024-05-08T23:48:44.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.916+0000 7f8547fff700 1 -- 172.21.15.12:0/1232234402 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f85600077e0 con 0x55b2965b47a0 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.916+0000 7f857af49b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.916+0000 7f857af49b80 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.916+0000 7f857af49b80 20 client.16071 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.916+0000 7f857af49b80 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b296624390 0x55b2966267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.916+0000 7f857af49b80 4 client.16071 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.916+0000 7f857af49b80 1 -- 172.21.15.12:0/1232234402 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b2964f6f40 con 0x55b296624390 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.916+0000 7f856cb2b700 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b296624390 0x55b2966267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.916+0000 7f8547fff700 10 client.16071 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.916+0000 7f856cb2b700 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b296624390 0x55b2966267b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9079d7da1caf52c1 server_cookie=73df7c375b2c0da5 in_seq=0 out_seq=0 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f8547fff700 1 -- 172.21.15.12:0/1232234402 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b2964f6f40 con 0x55b296624390 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f8547fff700 10 client.16071 handle_command_reply: tid=0 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f857af49b80 1 client.16071 shutdown 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f857af49b80 1 -- 172.21.15.12:0/1232234402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 msgr2=0x55b2965b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:44.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f857af49b80 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b1170 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f85600052a0 tx=0x7f8560015590 comp rx=0 tx=0).stop 2024-05-08T23:48:44.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f857af49b80 1 -- 172.21.15.12:0/1232234402 shutdown_connections 2024-05-08T23:48:44.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f857af49b80 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2965b47a0 0x55b2965b1170 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f857af49b80 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b296624390 0x55b2966267b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f857af49b80 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b2965ae910 0x55b2965b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f857af49b80 1 --2- 172.21.15.12:0/1232234402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2965b3e80 0x55b2965b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.917+0000 7f8547fff700 0 client.16071 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:44.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f857af49b80 1 -- 172.21.15.12:0/1232234402 >> 172.21.15.12:0/1232234402 conn(0x55b29650c290 msgr2=0x55b2965b7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:44.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f857af49b80 1 -- 172.21.15.12:0/1232234402 shutdown_connections 2024-05-08T23:48:44.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f857af49b80 1 -- 172.21.15.12:0/1232234402 wait complete. 2024-05-08T23:48:44.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f857af49b80 20 client.16071 trim_cache size 0 max 16384 2024-05-08T23:48:44.919 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:48:44.919 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:48:44.919 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "e13f39d7-f448-4169-beed-9d597e8b2fca", 2024-05-08T23:48:44.919 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:48:44.919 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f856ed8f700 1 -- 172.21.15.12:0/1494464349 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85500a1290 msgr2=0x7f85500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f856ed8f700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85500a1290 0x7f85500a3750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8564009990 tx=0x7f8564019040 comp rx=0 tx=0).stop 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f856ed8f700 1 -- 172.21.15.12:0/1494464349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100ab0 msgr2=0x7f8568197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f856ed8f700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100ab0 0x7f8568197860 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f855800e9a0 tx=0x7f855800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f856ed8f700 1 -- 172.21.15.12:0/1494464349 shutdown_connections 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f856ed8f700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85500a1290 0x7f85500a3750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f856ed8f700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568105f90 0x7f856819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f856ed8f700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85681000f0 0x7f8568197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f856ed8f700 1 --2- 172.21.15.12:0/1494464349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100ab0 0x7f8568197860 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.918+0000 7f856ed8f700 1 -- 172.21.15.12:0/1494464349 >> 172.21.15.12:0/1494464349 conn(0x7f8568076ba0 msgr2=0x7f8568069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:44.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.919+0000 7f856ed8f700 1 -- 172.21.15.12:0/1494464349 shutdown_connections 2024-05-08T23:48:44.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:44.919+0000 7f856ed8f700 1 -- 172.21.15.12:0/1494464349 wait complete. 2024-05-08T23:48:44.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:48:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.080+0000 7f6340514700 1 -- 172.21.15.12:0/3846708000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6338069770 msgr2=0x7f6338069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.080+0000 7f6340514700 1 --2- 172.21.15.12:0/3846708000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6338069770 0x7f6338069b50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6334009990 tx=0x7f6334009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.080+0000 7f6340514700 1 -- 172.21.15.12:0/3846708000 shutdown_connections 2024-05-08T23:48:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.080+0000 7f6340514700 1 --2- 172.21.15.12:0/3846708000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338106280 0x7f633810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.080+0000 7f6340514700 1 --2- 172.21.15.12:0/3846708000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6338069770 0x7f6338069b50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.080+0000 7f6340514700 1 --2- 172.21.15.12:0/3846708000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f633806a120 0x7f6338105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.080+0000 7f6340514700 1 -- 172.21.15.12:0/3846708000 >> 172.21.15.12:0/3846708000 conn(0x7f6338076fa0 msgr2=0x7f63380773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:45.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.080+0000 7f6340514700 1 -- 172.21.15.12:0/3846708000 shutdown_connections 2024-05-08T23:48:45.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.080+0000 7f6340514700 1 -- 172.21.15.12:0/3846708000 wait complete. 2024-05-08T23:48:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.081+0000 7f6340514700 1 Processor -- start 2024-05-08T23:48:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.081+0000 7f6340514700 1 -- start start 2024-05-08T23:48:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.081+0000 7f6340514700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338069770 0x7f63381973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.081+0000 7f6340514700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6338106280 0x7f6338197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.081+0000 7f6340514700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633819bcc0 0x7f633819c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.081+0000 7f6340514700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633819c740 con 0x7f633819bcc0 2024-05-08T23:48:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.081+0000 7f6340514700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633819c8b0 con 0x7f6338106280 2024-05-08T23:48:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.081+0000 7f6340514700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633819ca70 con 0x7f6338069770 2024-05-08T23:48:45.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633eab1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633819bcc0 0x7f633819c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633daaf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6338106280 0x7f6338197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633daaf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6338106280 0x7f6338197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:46344/0 (socket says 172.21.15.12:46344) 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633daaf700 1 -- 172.21.15.12:0/850687094 learned_addr learned my addr 172.21.15.12:0/850687094 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633e2b0700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338069770 0x7f63381973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633eab1700 1 -- 172.21.15.12:0/850687094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338069770 msgr2=0x7f63381973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633eab1700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338069770 0x7f63381973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633eab1700 1 -- 172.21.15.12:0/850687094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6338106280 msgr2=0x7f6338197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633eab1700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6338106280 0x7f6338197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633eab1700 1 -- 172.21.15.12:0/850687094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340095f0 con 0x7f633819bcc0 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f633eab1700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633819bcc0 0x7f633819c140 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f633000efc0 tx=0x7f633000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f632f7fe700 1 -- 172.21.15.12:0/850687094 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330018070 con 0x7f633819bcc0 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f6340514700 1 -- 172.21.15.12:0/850687094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63381a3f70 con 0x7f633819bcc0 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f632f7fe700 1 -- 172.21.15.12:0/850687094 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f633000f040 con 0x7f633819bcc0 2024-05-08T23:48:45.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f632f7fe700 1 -- 172.21.15.12:0/850687094 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633000cd40 con 0x7f633819bcc0 2024-05-08T23:48:45.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.082+0000 7f6340514700 1 -- 172.21.15.12:0/850687094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63381a4400 con 0x7f633819bcc0 2024-05-08T23:48:45.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.085+0000 7f632f7fe700 1 -- 172.21.15.12:0/850687094 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6330003a00 con 0x7f633819bcc0 2024-05-08T23:48:45.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.085+0000 7f632f7fe700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63240a1290 0x7f63240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.085+0000 7f632f7fe700 1 -- 172.21.15.12:0/850687094 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f63300cc820 con 0x7f633819bcc0 2024-05-08T23:48:45.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.085+0000 7f633e2b0700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63240a1290 0x7f63240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.086+0000 7f633e2b0700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63240a1290 0x7f63240a3750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f633400b2e0 tx=0x7f6334005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f634c6ceb80 1 Processor -- start 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f634c6ceb80 1 -- start start 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd14350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd14890 0x55ec3fd14cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fc6b150 con 0x55ec3fd1a720 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fc68b10 con 0x55ec3fd19e00 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fc64400 con 0x55ec3fd14890 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f633daaf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd14350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f633daaf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd14350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47908/0 (socket says 172.21.15.12:47908) 2024-05-08T23:48:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f633daaf700 1 -- 172.21.15.12:0/754287265 learned_addr learned my addr 172.21.15.12:0/754287265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f633e2b0700 1 --2- 172.21.15.12:0/754287265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.087+0000 7f633eab1700 1 --2- 172.21.15.12:0/754287265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd14890 0x55ec3fd14cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.088+0000 7f633daaf700 1 -- 172.21.15.12:0/754287265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd14890 msgr2=0x55ec3fd14cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.088+0000 7f633daaf700 1 --2- 172.21.15.12:0/754287265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd14890 0x55ec3fd14cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.088+0000 7f633daaf700 1 -- 172.21.15.12:0/754287265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd19e00 msgr2=0x55ec3fd1a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.088+0000 7f633daaf700 1 --2- 172.21.15.12:0/754287265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.088+0000 7f633daaf700 1 -- 172.21.15.12:0/754287265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3fae53b0 con 0x55ec3fd1a720 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.088+0000 7f633daaf700 1 --2- 172.21.15.12:0/754287265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd14350 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6328009d90 tx=0x7f6328011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8525fd06eda2e07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.088+0000 7f632cff9700 1 -- 172.21.15.12:0/754287265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328004a30 con 0x55ec3fd1a720 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.088+0000 7f632cff9700 1 -- 172.21.15.12:0/754287265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63280054e0 con 0x55ec3fd1a720 2024-05-08T23:48:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.088+0000 7f632cff9700 1 -- 172.21.15.12:0/754287265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280197f0 con 0x55ec3fd1a720 2024-05-08T23:48:45.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.089+0000 7f634c6ceb80 1 -- 172.21.15.12:0/754287265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 msgr2=0x55ec3fd14350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.089+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/754287265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd14350 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6328009d90 tx=0x7f6328011720 comp rx=0 tx=0).stop 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.089+0000 7f634c6ceb80 1 -- 172.21.15.12:0/754287265 shutdown_connections 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.089+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/754287265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd14890 0x55ec3fd14cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.089+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/754287265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.089+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/754287265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd14350 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.089+0000 7f634c6ceb80 1 -- 172.21.15.12:0/754287265 >> 172.21.15.12:0/754287265 conn(0x55ec3fc72210 msgr2=0x55ec3fd0edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.089+0000 7f634c6ceb80 1 -- 172.21.15.12:0/754287265 shutdown_connections 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.089+0000 7f634c6ceb80 1 -- 172.21.15.12:0/754287265 wait complete. 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f634c6ceb80 1 -- start start 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f634c6ceb80 1 Processor -- start 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd14890 0x55ec3fd1dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd1e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fc00560 con 0x55ec3fd1a720 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fc00200 con 0x55ec3fd14890 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fae76a0 con 0x55ec3fd19e00 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f634c6ceb80 10 client.0 _ll_register_callbacks cb 0x55ec3fc09b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633eab1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd1e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633e2b0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd14890 0x55ec3fd1dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633daaf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633e2b0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd14890 0x55ec3fd1dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:46362/0 (socket says 172.21.15.12:46362) 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633e2b0700 1 -- 172.21.15.12:0/734186413 learned_addr learned my addr 172.21.15.12:0/734186413 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633eab1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd1e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47924/0 (socket says 172.21.15.12:47924) 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633eab1700 1 -- 172.21.15.12:0/734186413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd19e00 msgr2=0x55ec3fd1e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633eab1700 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633eab1700 1 -- 172.21.15.12:0/734186413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd14890 msgr2=0x55ec3fd1dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633eab1700 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd14890 0x55ec3fd1dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.090+0000 7f633eab1700 1 -- 172.21.15.12:0/734186413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63280099f0 con 0x55ec3fd1a720 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.091+0000 7f633eab1700 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd1e630 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f63300c9fb0 tx=0x7f6330099360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d770629748f373c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.091+0000 7f6321ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.091+0000 7f6321ffb700 1 -- 172.21.15.12:0/734186413 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633009a100 con 0x55ec3fd1a720 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.091+0000 7f6321ffb700 1 -- 172.21.15.12:0/734186413 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6330099a00 con 0x55ec3fd1a720 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.091+0000 7f634c6ceb80 1 -- 172.21.15.12:0/734186413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec3fbbd270 con 0x55ec3fd1a720 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.091+0000 7f6321ffb700 1 -- 172.21.15.12:0/734186413 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f633009d830 con 0x55ec3fd1a720 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.091+0000 7f634c6ceb80 10 client.16083 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.091+0000 7f634c6ceb80 1 -- 172.21.15.12:0/734186413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ec3fa91b60 con 0x55ec3fd1a720 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.091+0000 7f6321ffb700 1 -- 172.21.15.12:0/734186413 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec3fbbd270 con 0x55ec3fd1a720 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.091+0000 7f6321ffb700 1 -- 172.21.15.12:0/734186413 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6330098ff0 con 0x55ec3fd1a720 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.092+0000 7f634c6ceb80 10 client.16083 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.092+0000 7f634c6ceb80 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.092+0000 7f634c6ceb80 20 client.16083 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.092+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ec3fd2e150 0x55ec3fd30570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.092+0000 7f634c6ceb80 4 client.16083 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.092+0000 7f634c6ceb80 1 -- 172.21.15.12:0/734186413 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec3fce3810 con 0x55ec3fd2e150 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.092+0000 7f633e2b0700 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ec3fd2e150 0x55ec3fd30570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.092+0000 7f6321ffb700 10 client.16083 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.092+0000 7f633e2b0700 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ec3fd2e150 0x55ec3fd30570 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a466fa2637c876cf server_cookie=c6204719c9b9f596 in_seq=0 out_seq=0 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.093+0000 7f6321ffb700 1 -- 172.21.15.12:0/734186413 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec3fce3810 con 0x55ec3fd2e150 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.093+0000 7f6321ffb700 10 client.16083 handle_command_reply: tid=0 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.093+0000 7f634c6ceb80 1 client.16083 shutdown 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 1 -- 172.21.15.12:0/734186413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 msgr2=0x55ec3fd1e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd1e630 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f63300c9fb0 tx=0x7f6330099360 comp rx=0 tx=0).stop 2024-05-08T23:48:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 1 -- 172.21.15.12:0/734186413 shutdown_connections 2024-05-08T23:48:45.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd1a720 0x55ec3fd1e630 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ec3fd2e150 0x55ec3fd30570 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd14890 0x55ec3fd1dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/734186413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f6321ffb700 0 client.16083 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:45.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 1 -- 172.21.15.12:0/734186413 >> 172.21.15.12:0/734186413 conn(0x55ec3fc72210 msgr2=0x55ec3fd0e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:45.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 1 -- 172.21.15.12:0/734186413 shutdown_connections 2024-05-08T23:48:45.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 1 -- 172.21.15.12:0/734186413 wait complete. 2024-05-08T23:48:45.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.094+0000 7f634c6ceb80 20 client.16083 trim_cache size 0 max 16384 2024-05-08T23:48:45.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f634c6ceb80 1 Processor -- start 2024-05-08T23:48:45.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f634c6ceb80 1 -- start start 2024-05-08T23:48:45.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd14890 0x55ec3fd0ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 0x55ec3fd1c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fd0bdc0 con 0x55ec3fd14890 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fd0fca0 con 0x55ec3fd1a720 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fd10e50 con 0x55ec3fd19e00 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f633e2b0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd14890 0x55ec3fd0ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f633e2b0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd14890 0x55ec3fd0ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47932/0 (socket says 172.21.15.12:47932) 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f633e2b0700 1 -- 172.21.15.12:0/1315223793 learned_addr learned my addr 172.21.15.12:0/1315223793 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f633daaf700 1 --2- 172.21.15.12:0/1315223793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f633eab1700 1 --2- 172.21.15.12:0/1315223793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 0x55ec3fd1c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f633e2b0700 1 -- 172.21.15.12:0/1315223793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd19e00 msgr2=0x55ec3fd1e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.113+0000 7f633e2b0700 1 --2- 172.21.15.12:0/1315223793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.114+0000 7f633e2b0700 1 -- 172.21.15.12:0/1315223793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 msgr2=0x55ec3fd1c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.114+0000 7f633e2b0700 1 --2- 172.21.15.12:0/1315223793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 0x55ec3fd1c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.114+0000 7f633e2b0700 1 -- 172.21.15.12:0/1315223793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3fc2a560 con 0x55ec3fd14890 2024-05-08T23:48:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.114+0000 7f633e2b0700 1 --2- 172.21.15.12:0/1315223793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd14890 0x55ec3fd0ed20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f6334023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1744ca54edaa226 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.114+0000 7f632d7fa700 1 -- 172.21.15.12:0/1315223793 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334036160 con 0x55ec3fd14890 2024-05-08T23:48:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.114+0000 7f632d7fa700 1 -- 172.21.15.12:0/1315223793 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f633401e040 con 0x55ec3fd14890 2024-05-08T23:48:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.114+0000 7f632d7fa700 1 -- 172.21.15.12:0/1315223793 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334004bd0 con 0x55ec3fd14890 2024-05-08T23:48:45.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 -- 172.21.15.12:0/1315223793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd14890 msgr2=0x55ec3fd0ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:45.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/1315223793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd14890 0x55ec3fd0ed20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f6334023000 comp rx=0 tx=0).stop 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 -- 172.21.15.12:0/1315223793 shutdown_connections 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/1315223793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 0x55ec3fd1c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/1315223793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd14890 0x55ec3fd0ed20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/1315223793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd19e00 0x55ec3fd1e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 -- 172.21.15.12:0/1315223793 >> 172.21.15.12:0/1315223793 conn(0x55ec3fc72210 msgr2=0x55ec3fd0b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 -- 172.21.15.12:0/1315223793 shutdown_connections 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 -- 172.21.15.12:0/1315223793 wait complete. 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 -- start start 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 Processor -- start 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd14890 0x55ec3fd23340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd19e00 0x55ec3fd23880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 0x55ec3fd83650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fd83b90 con 0x55ec3fd19e00 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fd83cd0 con 0x55ec3fd1a720 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.115+0000 7f634c6ceb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3fd83e10 con 0x55ec3fd14890 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f634c6ceb80 10 client.0 _ll_register_callbacks cb 0x55ec3fc09b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633daaf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd19e00 0x55ec3fd23880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633daaf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd19e00 0x55ec3fd23880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47942/0 (socket says 172.21.15.12:47942) 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633e2b0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd14890 0x55ec3fd23340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633daaf700 1 -- 172.21.15.12:0/2769962231 learned_addr learned my addr 172.21.15.12:0/2769962231 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633eab1700 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 0x55ec3fd83650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633eab1700 1 -- 172.21.15.12:0/2769962231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd14890 msgr2=0x55ec3fd23340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633eab1700 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd14890 0x55ec3fd23340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633eab1700 1 -- 172.21.15.12:0/2769962231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd19e00 msgr2=0x55ec3fd23880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633eab1700 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd19e00 0x55ec3fd23880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633eab1700 1 -- 172.21.15.12:0/2769962231 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633810e670 con 0x55ec3fd1a720 2024-05-08T23:48:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633daaf700 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd19e00 0x55ec3fd23880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f633eab1700 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 0x55ec3fd83650 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f63300ca5b0 tx=0x7f63300098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=446adb9c7545dd84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f6321ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.116+0000 7f6321ffb700 1 -- 172.21.15.12:0/2769962231 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300b92b0 con 0x55ec3fd1a720 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f6321ffb700 1 -- 172.21.15.12:0/2769962231 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f633009a100 con 0x55ec3fd1a720 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f634c6ceb80 1 -- 172.21.15.12:0/2769962231 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63280054e0 con 0x55ec3fd1a720 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f6321ffb700 1 -- 172.21.15.12:0/2769962231 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f633001b320 con 0x55ec3fd1a720 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f634c6ceb80 10 client.25919 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f634c6ceb80 1 -- 172.21.15.12:0/2769962231 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ec3fae9cd0 con 0x55ec3fd1a720 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f6321ffb700 1 -- 172.21.15.12:0/2769962231 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63280054e0 con 0x55ec3fd1a720 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f6321ffb700 1 -- 172.21.15.12:0/2769962231 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f63300957f0 con 0x55ec3fd1a720 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f634c6ceb80 10 client.25919 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f634c6ceb80 10 client.25919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f634c6ceb80 20 client.25919 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ec3fd852e0 0x55ec3fd87700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f634c6ceb80 4 client.25919 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f634c6ceb80 1 -- 172.21.15.12:0/2769962231 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec3fc5cec0 con 0x55ec3fd852e0 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.117+0000 7f633e2b0700 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ec3fd852e0 0x55ec3fd87700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f633e2b0700 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ec3fd852e0 0x55ec3fd87700 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6105e2818e9d69b3 server_cookie=3da6c9e1ab85c3de in_seq=0 out_seq=0 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f6321ffb700 10 client.25919 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f6321ffb700 1 -- 172.21.15.12:0/2769962231 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ec3fc5cec0 con 0x55ec3fd852e0 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f6321ffb700 10 client.25919 handle_command_reply: tid=0 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 client.25919 shutdown 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 -- 172.21.15.12:0/2769962231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 msgr2=0x55ec3fd83650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 0x55ec3fd83650 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f63300ca5b0 tx=0x7f63300098a0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 -- 172.21.15.12:0/2769962231 shutdown_connections 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ec3fd852e0 0x55ec3fd87700 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec3fd1a720 0x55ec3fd83650 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f6321ffb700 0 client.25919 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:45.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3fd14890 0x55ec3fd23340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 --2- 172.21.15.12:0/2769962231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec3fd19e00 0x55ec3fd23880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 -- 172.21.15.12:0/2769962231 >> 172.21.15.12:0/2769962231 conn(0x55ec3fc72210 msgr2=0x55ec3fd1c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:45.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 -- 172.21.15.12:0/2769962231 shutdown_connections 2024-05-08T23:48:45.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 1 -- 172.21.15.12:0/2769962231 wait complete. 2024-05-08T23:48:45.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.118+0000 7f634c6ceb80 20 client.25919 trim_cache size 0 max 16384 2024-05-08T23:48:45.120 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:48:45.120 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:48:45.120 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:48:45.120 INFO:teuthology.orchestra.run.smithi012.stdout: "e13f39d7-f448-4169-beed-9d597e8b2fca": { 2024-05-08T23:48:45.121 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:48:45.121 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "e13f39d7-f448-4169-beed-9d597e8b2fca", 2024-05-08T23:48:45.121 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:48:45.121 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:48:45.121 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:48:45.121 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 -- 172.21.15.12:0/850687094 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63240a1290 msgr2=0x7f63240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63240a1290 0x7f63240a3750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f633400b2e0 tx=0x7f6334005960 comp rx=0 tx=0).stop 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 -- 172.21.15.12:0/850687094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633819bcc0 msgr2=0x7f633819c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633819bcc0 0x7f633819c140 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f633000efc0 tx=0x7f633000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 -- 172.21.15.12:0/850687094 shutdown_connections 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63240a1290 0x7f63240a3750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633819bcc0 0x7f633819c140 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338069770 0x7f63381973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 --2- 172.21.15.12:0/850687094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6338106280 0x7f6338197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 -- 172.21.15.12:0/850687094 >> 172.21.15.12:0/850687094 conn(0x7f6338076fa0 msgr2=0x7f63380fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 -- 172.21.15.12:0/850687094 shutdown_connections 2024-05-08T23:48:45.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:48:45.119+0000 7f6340514700 1 -- 172.21.15.12:0/850687094 wait complete. 2024-05-08T23:48:45.128 INFO:tasks.cephfs.filesystem:scrub status for tag:e13f39d7-f448-4169-beed-9d597e8b2fca - {'path': '/', 'tag': 'e13f39d7-f448-4169-beed-9d597e8b2fca', 'options': 'recursive,force'} 2024-05-08T23:48:45.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:45 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:48:45.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:45 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:45.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:45 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:48:45.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:45 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:48:45.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:45 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:45.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:45 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:45.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:45 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:48:45.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:45 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:45.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:45 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:48:45.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:45 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:48:45.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:45 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:45.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:45 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:45.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:45 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:48:45.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:45 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:45.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:45 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:48:45.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:45 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:48:45.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:45 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:45.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:45 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:45.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 61307 8.46 MB/sec warmup 41 sec latency 2044.896 ms 2024-05-08T23:48:46.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:46 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:48:46.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:46 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:46.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:46 smithi012 ceph-mon[42503]: pgmap v74: 97 pgs: 97 active+clean; 26 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 5.7 MiB/s wr, 190 op/s 2024-05-08T23:48:46.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:46 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:48:46.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:46 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:46.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:46 smithi204 ceph-mon[47373]: pgmap v74: 97 pgs: 97 active+clean; 26 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 5.7 MiB/s wr, 190 op/s 2024-05-08T23:48:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:46 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:48:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:46 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:48:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:46 smithi160 ceph-mon[46152]: pgmap v74: 97 pgs: 97 active+clean; 26 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 5.7 MiB/s wr, 190 op/s 2024-05-08T23:48:46.848 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63301 8.54 MB/sec warmup 42 sec latency 346.879 ms 2024-05-08T23:48:47.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63338 8.34 MB/sec warmup 43 sec latency 1249.923 ms 2024-05-08T23:48:48.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:48 smithi012 ceph-mon[42503]: pgmap v75: 97 pgs: 97 active+clean; 28 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 5.8 MiB/s wr, 234 op/s 2024-05-08T23:48:48.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:48 smithi204 ceph-mon[47373]: pgmap v75: 97 pgs: 97 active+clean; 28 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 5.8 MiB/s wr, 234 op/s 2024-05-08T23:48:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:48 smithi160 ceph-mon[46152]: pgmap v75: 97 pgs: 97 active+clean; 28 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 5.8 MiB/s wr, 234 op/s 2024-05-08T23:48:48.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63338 8.15 MB/sec warmup 44 sec latency 1065.836 ms 2024-05-08T23:48:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.622+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/3846176290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f080ffda0 msgr2=0x7f2f08100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.622+0000 7f2f0f9ba700 1 --2- 172.21.15.12:0/3846176290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f080ffda0 0x7f2f08100220 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009990 tx=0x7f2ef8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.622+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/3846176290 shutdown_connections 2024-05-08T23:48:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.622+0000 7f2f0f9ba700 1 --2- 172.21.15.12:0/3846176290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f08100760 0x7f2f0810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.622+0000 7f2f0f9ba700 1 --2- 172.21.15.12:0/3846176290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f08105e60 0x7f2f08106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.622+0000 7f2f0f9ba700 1 --2- 172.21.15.12:0/3846176290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f080ffda0 0x7f2f08100220 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.622+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/3846176290 >> 172.21.15.12:0/3846176290 conn(0x7f2f08076f10 msgr2=0x7f2f08077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.622+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/3846176290 shutdown_connections 2024-05-08T23:48:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.622+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/3846176290 wait complete. 2024-05-08T23:48:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0f9ba700 1 Processor -- start 2024-05-08T23:48:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0f9ba700 1 -- start start 2024-05-08T23:48:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0f9ba700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f080ffda0 0x7f2f081972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0f9ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f08100760 0x7f2f081977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0f9ba700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f08105e60 0x7f2f0819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0f9ba700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0819c1c0 con 0x7f2f08100760 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0f9ba700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0819c330 con 0x7f2f080ffda0 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0f9ba700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0819c4f0 con 0x7f2f08105e60 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0cf55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f08100760 0x7f2f081977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0cf55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f08100760 0x7f2f081977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33712/0 (socket says 172.21.15.12:33712) 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0cf55700 1 -- 172.21.15.12:0/1100991275 learned_addr learned my addr 172.21.15.12:0/1100991275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0d756700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f080ffda0 0x7f2f081972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0df57700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f08105e60 0x7f2f0819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0cf55700 1 -- 172.21.15.12:0/1100991275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f08105e60 msgr2=0x7f2f0819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f08105e60 0x7f2f0819bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0cf55700 1 -- 172.21.15.12:0/1100991275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f080ffda0 msgr2=0x7f2f081972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f080ffda0 0x7f2f081972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0cf55700 1 -- 172.21.15.12:0/1100991275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef80095f0 con 0x7f2f08100760 2024-05-08T23:48:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.623+0000 7f2f0d756700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f080ffda0 0x7f2f081972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.624+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f08100760 0x7f2f081977e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009fd0 tx=0x7f2ef800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.624+0000 7f2efe7fc700 1 -- 172.21.15.12:0/1100991275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef801d070 con 0x7f2f08100760 2024-05-08T23:48:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.624+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/1100991275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0819c770 con 0x7f2f08100760 2024-05-08T23:48:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.624+0000 7f2efe7fc700 1 -- 172.21.15.12:0/1100991275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef8005050 con 0x7f2f08100760 2024-05-08T23:48:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.624+0000 7f2efe7fc700 1 -- 172.21.15.12:0/1100991275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8004a90 con 0x7f2f08100760 2024-05-08T23:48:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.624+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/1100991275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f081a4010 con 0x7f2f08100760 2024-05-08T23:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.626+0000 7f2efe7fc700 1 -- 172.21.15.12:0/1100991275 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2ef8004c10 con 0x7f2f08100760 2024-05-08T23:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.627+0000 7f2efe7fc700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ef40a1290 0x7f2ef40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.627+0000 7f2efe7fc700 1 -- 172.21.15.12:0/1100991275 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2ef80cda70 con 0x7f2f08100760 2024-05-08T23:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.627+0000 7f2f0d756700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ef40a1290 0x7f2ef40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.627+0000 7f2f0d756700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ef40a1290 0x7f2ef40a3750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f04005950 tx=0x7f2f0400b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.628+0000 7f2f1bb74b80 1 Processor -- start 2024-05-08T23:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.628+0000 7f2f1bb74b80 1 -- start start 2024-05-08T23:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.628+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 0x56172bf39d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.628+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf3a270 0x56172bf33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.628+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf34480 0x56172bf348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.628+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172be8b0f0 con 0x56172bf3a270 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.628+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172be88ab0 con 0x56172bf39950 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.628+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172be843a0 con 0x56172bf34480 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0cf55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf3a270 0x56172bf33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0cf55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf3a270 0x56172bf33f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33720/0 (socket says 172.21.15.12:33720) 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0cf55700 1 -- 172.21.15.12:0/815818724 learned_addr learned my addr 172.21.15.12:0/815818724 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0d756700 1 --2- 172.21.15.12:0/815818724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 0x56172bf39d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0df57700 1 --2- 172.21.15.12:0/815818724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf34480 0x56172bf348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0cf55700 1 -- 172.21.15.12:0/815818724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf34480 msgr2=0x56172bf348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/815818724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf34480 0x56172bf348a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0cf55700 1 -- 172.21.15.12:0/815818724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 msgr2=0x56172bf39d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/815818724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 0x56172bf39d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0cf55700 1 -- 172.21.15.12:0/815818724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56172bd05350 con 0x56172bf3a270 2024-05-08T23:48:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/815818724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf3a270 0x56172bf33f40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8097150 tx=0x7f2ef809a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62efd118d153a84f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2ef37fe700 1 -- 172.21.15.12:0/815818724 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef809ea80 con 0x56172bf3a270 2024-05-08T23:48:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2ef37fe700 1 -- 172.21.15.12:0/815818724 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef80a3040 con 0x56172bf3a270 2024-05-08T23:48:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.629+0000 7f2ef37fe700 1 -- 172.21.15.12:0/815818724 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80a2bb0 con 0x56172bf3a270 2024-05-08T23:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.630+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/815818724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf3a270 msgr2=0x56172bf33f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.630+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/815818724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf3a270 0x56172bf33f40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8097150 tx=0x7f2ef809a6a0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.630+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/815818724 shutdown_connections 2024-05-08T23:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.630+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/815818724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf34480 0x56172bf348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.630+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/815818724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 0x56172bf39d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.630+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/815818724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf3a270 0x56172bf33f40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.630+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/815818724 >> 172.21.15.12:0/815818724 conn(0x56172be921b0 msgr2=0x56172bf2e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.630+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/815818724 shutdown_connections 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.630+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/815818724 wait complete. 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f1bb74b80 1 -- start start 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f1bb74b80 1 Processor -- start 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf34480 0x56172bf40160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 0x56172bf3d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf3a270 0x56172bf3dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172be20500 con 0x56172bf34480 2024-05-08T23:48:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172be201a0 con 0x56172bf39950 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172bd07640 con 0x56172bf3a270 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f1bb74b80 10 client.0 _ll_register_callbacks cb 0x56172be29ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f0d756700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf34480 0x56172bf40160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f0d756700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf34480 0x56172bf40160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33726/0 (socket says 172.21.15.12:33726) 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f0d756700 1 -- 172.21.15.12:0/2721138709 learned_addr learned my addr 172.21.15.12:0/2721138709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f0df57700 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf3a270 0x56172bf3dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.631+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 0x56172bf3d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2f0d756700 1 -- 172.21.15.12:0/2721138709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf3a270 msgr2=0x56172bf3dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2f0d756700 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf3a270 0x56172bf3dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2f0d756700 1 -- 172.21.15.12:0/2721138709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 msgr2=0x56172bf3d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2f0d756700 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 0x56172bf3d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2f0d756700 1 -- 172.21.15.12:0/2721138709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f081a4010 con 0x56172bf34480 2024-05-08T23:48:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 0x56172bf3d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2f0d756700 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf34480 0x56172bf40160 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2f040119d0 tx=0x7f2f040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cc3ea09917f0a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2ef0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2721138709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0401f840 con 0x56172bf34480 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2721138709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2f04017070 con 0x56172bf34480 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2721138709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56172bddd210 con 0x56172bf34480 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2721138709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f0401f9a0 con 0x56172bf34480 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2f1bb74b80 10 client.16101 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2721138709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56172bcb1b00 con 0x56172bf34480 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.632+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2721138709 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56172bddd210 con 0x56172bf34480 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.633+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2721138709 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2f04028bf0 con 0x56172bf34480 2024-05-08T23:48:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.633+0000 7f2f1bb74b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.633+0000 7f2f1bb74b80 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.633+0000 7f2f1bb74b80 20 client.16101 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.633+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172bf4d890 0x56172bf4fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.633+0000 7f2f1bb74b80 4 client.16101 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.633+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2721138709 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56172bf037b0 con 0x56172bf4d890 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.633+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172bf4d890 0x56172bf4fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.633+0000 7f2ef0ff9700 10 client.16101 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.633+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172bf4d890 0x56172bf4fcb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d5392dade576ae server_cookie=71e03487a5b242b8 in_seq=0 out_seq=0 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2721138709 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56172bf037b0 con 0x56172bf4d890 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2ef0ff9700 10 client.16101 handle_command_reply: tid=0 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2f1bb74b80 1 client.16101 shutdown 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2721138709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf34480 msgr2=0x56172bf40160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf34480 0x56172bf40160 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2f040119d0 tx=0x7f2f040047c0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2721138709 shutdown_connections 2024-05-08T23:48:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf3a270 0x56172bf3dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf34480 0x56172bf40160 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172bf4d890 0x56172bf4fcb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2721138709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf39950 0x56172bf3d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2721138709 >> 172.21.15.12:0/2721138709 conn(0x56172be921b0 msgr2=0x56172bf384a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.634+0000 7f2ef0ff9700 0 client.16101 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.635+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2721138709 shutdown_connections 2024-05-08T23:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.635+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2721138709 wait complete. 2024-05-08T23:48:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.635+0000 7f2f1bb74b80 20 client.16101 trim_cache size 0 max 16384 2024-05-08T23:48:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f1bb74b80 1 Processor -- start 2024-05-08T23:48:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f1bb74b80 1 -- start start 2024-05-08T23:48:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf34480 0x56172bf2e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bf38390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf3a270 0x56172bf448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172bf26a40 con 0x56172bf39950 2024-05-08T23:48:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172bf3b9a0 con 0x56172bf3a270 2024-05-08T23:48:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172bf33030 con 0x56172bf34480 2024-05-08T23:48:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0cf55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bf38390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0cf55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bf38390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33742/0 (socket says 172.21.15.12:33742) 2024-05-08T23:48:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0cf55700 1 -- 172.21.15.12:0/3617448660 learned_addr learned my addr 172.21.15.12:0/3617448660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0d756700 1 --2- 172.21.15.12:0/3617448660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf34480 0x56172bf2e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0df57700 1 --2- 172.21.15.12:0/3617448660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf3a270 0x56172bf448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0cf55700 1 -- 172.21.15.12:0/3617448660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf34480 msgr2=0x56172bf2e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/3617448660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf34480 0x56172bf2e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0cf55700 1 -- 172.21.15.12:0/3617448660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf3a270 msgr2=0x56172bf448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/3617448660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf3a270 0x56172bf448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0cf55700 1 -- 172.21.15.12:0/3617448660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56172be4a500 con 0x56172bf39950 2024-05-08T23:48:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.651+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/3617448660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bf38390 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8097150 tx=0x7f2ef809ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcf753952f70c8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2ef3fff700 1 -- 172.21.15.12:0/3617448660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef809a140 con 0x56172bf39950 2024-05-08T23:48:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2ef3fff700 1 -- 172.21.15.12:0/3617448660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef809a2a0 con 0x56172bf39950 2024-05-08T23:48:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2ef3fff700 1 -- 172.21.15.12:0/3617448660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80b34e0 con 0x56172bf39950 2024-05-08T23:48:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/3617448660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 msgr2=0x56172bf38390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/3617448660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bf38390 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8097150 tx=0x7f2ef809ec00 comp rx=0 tx=0).stop 2024-05-08T23:48:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/3617448660 shutdown_connections 2024-05-08T23:48:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/3617448660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf3a270 0x56172bf448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/3617448660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf34480 0x56172bf2e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/3617448660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bf38390 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/3617448660 >> 172.21.15.12:0/3617448660 conn(0x56172be921b0 msgr2=0x56172bf38cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/3617448660 shutdown_connections 2024-05-08T23:48:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.652+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/3617448660 wait complete. 2024-05-08T23:48:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f1bb74b80 1 -- start start 2024-05-08T23:48:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f1bb74b80 1 Processor -- start 2024-05-08T23:48:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf34480 0x56172bfa10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bfa1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f1bb74b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf3a270 0x56172bf9bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172bf9c310 con 0x56172bf39950 2024-05-08T23:48:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172bf9c450 con 0x56172bf34480 2024-05-08T23:48:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f1bb74b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172bf9c610 con 0x56172bf3a270 2024-05-08T23:48:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f1bb74b80 10 client.0 _ll_register_callbacks cb 0x56172be29ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:48:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f0d756700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf34480 0x56172bfa10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f0cf55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bfa1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f0d756700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf34480 0x56172bfa10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50942/0 (socket says 172.21.15.12:50942) 2024-05-08T23:48:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f0d756700 1 -- 172.21.15.12:0/2043615587 learned_addr learned my addr 172.21.15.12:0/2043615587 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:48:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.653+0000 7f2f0cf55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bfa1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33756/0 (socket says 172.21.15.12:33756) 2024-05-08T23:48:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f0df57700 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf3a270 0x56172bf9bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f0cf55700 1 -- 172.21.15.12:0/2043615587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf3a270 msgr2=0x56172bf9bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf3a270 0x56172bf9bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f0cf55700 1 -- 172.21.15.12:0/2043615587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf34480 msgr2=0x56172bfa10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:48:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf34480 0x56172bfa10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f0cf55700 1 -- 172.21.15.12:0/2043615587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56172bd05350 con 0x56172bf39950 2024-05-08T23:48:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f0d756700 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf34480 0x56172bfa10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:48:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f0cf55700 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bfa1600 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x56172bf3e160 tx=0x7f2ef80970a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f98c0e8ed1402d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:48:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2ef0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:48:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2043615587 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80a2cf0 con 0x56172bf39950 2024-05-08T23:48:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2043615587 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2ef809aeb0 con 0x56172bf39950 2024-05-08T23:48:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2043615587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ef80a3040 con 0x56172bf39950 2024-05-08T23:48:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2043615587 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ef80a2e50 con 0x56172bf39950 2024-05-08T23:48:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f1bb74b80 10 client.16107 fetch_fsmap learned FSMap version 27 2024-05-08T23:48:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2043615587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56172bd09c70 con 0x56172bf39950 2024-05-08T23:48:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.654+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2043615587 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef80a3040 con 0x56172bf39950 2024-05-08T23:48:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.655+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2043615587 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ef800fc70 con 0x56172bf39950 2024-05-08T23:48:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.655+0000 7f2f1bb74b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:48:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.655+0000 7f2f1bb74b80 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:48:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.655+0000 7f2f1bb74b80 20 client.16107 populate_metadata read hostname 'smithi012' 2024-05-08T23:48:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.655+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172bfa8b40 0x56172bfaaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:48:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.655+0000 7f2f1bb74b80 4 client.16107 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:48:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.655+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2043615587 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56172be7ce60 con 0x56172bfa8b40 2024-05-08T23:48:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.655+0000 7f2f0d756700 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172bfa8b40 0x56172bfaaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:48:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.655+0000 7f2ef0ff9700 10 client.16107 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.655+0000 7f2f0d756700 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172bfa8b40 0x56172bfaaf60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87b6a74fb296f3e server_cookie=8f0ad0aee1ec3ddb in_seq=0 out_seq=0 2024-05-08T23:48:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/2043615587 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56172be7ce60 con 0x56172bfa8b40 2024-05-08T23:48:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2ef0ff9700 10 client.16107 handle_command_reply: tid=0 2024-05-08T23:48:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 client.16107 shutdown 2024-05-08T23:48:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2043615587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 msgr2=0x56172bfa1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bfa1600 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x56172bf3e160 tx=0x7f2ef80970a0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2043615587 shutdown_connections 2024-05-08T23:48:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172bf3a270 0x56172bf9bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172bfa8b40 0x56172bfaaf60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172bf34480 0x56172bfa10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 --2- 172.21.15.12:0/2043615587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172bf39950 0x56172bfa1600 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2043615587 >> 172.21.15.12:0/2043615587 conn(0x56172be921b0 msgr2=0x56172bf3bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2ef0ff9700 0 client.16107 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:48:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2043615587 shutdown_connections 2024-05-08T23:48:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 1 -- 172.21.15.12:0/2043615587 wait complete. 2024-05-08T23:48:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.706+0000 7f2f1bb74b80 20 client.16107 trim_cache size 0 max 16384 2024-05-08T23:48:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:48:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:48:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:48:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:48:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/1100991275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ef40a1290 msgr2=0x7f2ef40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ef40a1290 0x7f2ef40a3750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f04005950 tx=0x7f2f0400b410 comp rx=0 tx=0).stop 2024-05-08T23:48:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/1100991275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f08100760 msgr2=0x7f2f081977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:48:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f08100760 0x7f2f081977e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009fd0 tx=0x7f2ef800f670 comp rx=0 tx=0).stop 2024-05-08T23:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/1100991275 shutdown_connections 2024-05-08T23:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ef40a1290 0x7f2ef40a3750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f08105e60 0x7f2f0819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f080ffda0 0x7f2f081972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 --2- 172.21.15.12:0/1100991275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f08100760 0x7f2f081977e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/1100991275 >> 172.21.15.12:0/1100991275 conn(0x7f2f08076f10 msgr2=0x7f2f080685c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/1100991275 shutdown_connections 2024-05-08T23:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:48:49.707+0000 7f2f0f9ba700 1 -- 172.21.15.12:0/1100991275 wait complete. 2024-05-08T23:48:49.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63338 7.97 MB/sec warmup 45 sec latency 2065.955 ms 2024-05-08T23:48:50.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:50 smithi204 ceph-mon[47373]: pgmap v76: 97 pgs: 97 active+clean; 28 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 483 KiB/s wr, 140 op/s 2024-05-08T23:48:50.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:50 smithi160 ceph-mon[46152]: pgmap v76: 97 pgs: 97 active+clean; 28 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 483 KiB/s wr, 140 op/s 2024-05-08T23:48:50.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63338 7.80 MB/sec warmup 46 sec latency 3066.048 ms 2024-05-08T23:48:50.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:50 smithi012 ceph-mon[42503]: pgmap v76: 97 pgs: 97 active+clean; 28 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 483 KiB/s wr, 140 op/s 2024-05-08T23:48:51.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63338 7.63 MB/sec warmup 47 sec latency 4066.158 ms 2024-05-08T23:48:52.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:52 smithi204 ceph-mon[47373]: pgmap v77: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 529 KiB/s wr, 173 op/s 2024-05-08T23:48:52.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:52 smithi160 ceph-mon[46152]: pgmap v77: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 529 KiB/s wr, 173 op/s 2024-05-08T23:48:52.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63535 7.49 MB/sec warmup 48 sec latency 4975.966 ms 2024-05-08T23:48:52.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:52 smithi012 ceph-mon[42503]: pgmap v77: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 529 KiB/s wr, 173 op/s 2024-05-08T23:48:53.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:53 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:48:53.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:53 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:48:53.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63535 7.33 MB/sec warmup 49 sec latency 1032.003 ms 2024-05-08T23:48:53.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:53 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:48:54.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:54 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:48:54.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:54 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:48:54.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:54 smithi204 ceph-mon[47373]: pgmap v78: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 214 KiB/s wr, 76 op/s 2024-05-08T23:48:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:54 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:48:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:54 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:48:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:54 smithi160 ceph-mon[46152]: pgmap v78: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 214 KiB/s wr, 76 op/s 2024-05-08T23:48:54.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63535 7.19 MB/sec warmup 50 sec latency 2032.130 ms 2024-05-08T23:48:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:54 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:48:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:54 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:48:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:54 smithi012 ceph-mon[42503]: pgmap v78: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 214 KiB/s wr, 76 op/s 2024-05-08T23:48:55.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63535 7.05 MB/sec warmup 51 sec latency 3032.224 ms 2024-05-08T23:48:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:56 smithi204 ceph-mon[47373]: pgmap v79: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 214 KiB/s wr, 76 op/s 2024-05-08T23:48:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:56 smithi160 ceph-mon[46152]: pgmap v79: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 214 KiB/s wr, 76 op/s 2024-05-08T23:48:56.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63535 6.91 MB/sec warmup 52 sec latency 4032.345 ms 2024-05-08T23:48:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:56 smithi012 ceph-mon[42503]: pgmap v79: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 214 KiB/s wr, 76 op/s 2024-05-08T23:48:57.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 63869 6.86 MB/sec warmup 53 sec latency 4940.965 ms 2024-05-08T23:48:58.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:48:58 smithi204 ceph-mon[47373]: pgmap v80: 97 pgs: 97 active+clean; 24 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 226 KiB/s wr, 82 op/s 2024-05-08T23:48:58.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:48:58 smithi160 ceph-mon[46152]: pgmap v80: 97 pgs: 97 active+clean; 24 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 226 KiB/s wr, 82 op/s 2024-05-08T23:48:58.849 INFO:tasks.workunit.client.0.smithi012.stdout: 1 65562 6.86 MB/sec warmup 54 sec latency 11.985 ms 2024-05-08T23:48:58.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:48:58 smithi012 ceph-mon[42503]: pgmap v80: 97 pgs: 97 active+clean; 24 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 226 KiB/s wr, 82 op/s 2024-05-08T23:48:59.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 66843 6.87 MB/sec warmup 55 sec latency 632.281 ms 2024-05-08T23:49:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:00 smithi204 ceph-mon[47373]: pgmap v81: 97 pgs: 97 active+clean; 24 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 58 KiB/s wr, 38 op/s 2024-05-08T23:49:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:00 smithi160 ceph-mon[46152]: pgmap v81: 97 pgs: 97 active+clean; 24 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 58 KiB/s wr, 38 op/s 2024-05-08T23:49:00.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 66843 6.74 MB/sec warmup 56 sec latency 1632.386 ms 2024-05-08T23:49:00.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:00 smithi012 ceph-mon[42503]: pgmap v81: 97 pgs: 97 active+clean; 24 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 58 KiB/s wr, 38 op/s 2024-05-08T23:49:01.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 67781 6.74 MB/sec warmup 57 sec latency 1804.472 ms 2024-05-08T23:49:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:02 smithi204 ceph-mon[47373]: pgmap v82: 97 pgs: 97 active+clean; 26 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 308 KiB/s wr, 102 op/s 2024-05-08T23:49:02.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:02 smithi160 ceph-mon[46152]: pgmap v82: 97 pgs: 97 active+clean; 26 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 308 KiB/s wr, 102 op/s 2024-05-08T23:49:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.841+0000 7f1a95d1a700 1 -- 172.21.15.12:0/3132211592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a90073cb0 msgr2=0x7f1a9010d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.841+0000 7f1a95d1a700 1 --2- 172.21.15.12:0/3132211592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a90073cb0 0x7f1a9010d1b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1a840098f0 tx=0x7f1a84009c00 comp rx=0 tx=0).stop 2024-05-08T23:49:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.841+0000 7f1a95d1a700 1 -- 172.21.15.12:0/3132211592 shutdown_connections 2024-05-08T23:49:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.841+0000 7f1a95d1a700 1 --2- 172.21.15.12:0/3132211592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a9010d6f0 0x7f1a9010fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.841+0000 7f1a95d1a700 1 --2- 172.21.15.12:0/3132211592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a90073390 0x7f1a90073770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.841+0000 7f1a95d1a700 1 --2- 172.21.15.12:0/3132211592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a90073cb0 0x7f1a9010d1b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.841+0000 7f1a95d1a700 1 -- 172.21.15.12:0/3132211592 >> 172.21.15.12:0/3132211592 conn(0x7f1a900fc940 msgr2=0x7f1a900fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.841+0000 7f1a95d1a700 1 -- 172.21.15.12:0/3132211592 shutdown_connections 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.841+0000 7f1a95d1a700 1 -- 172.21.15.12:0/3132211592 wait complete. 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a95d1a700 1 Processor -- start 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a95d1a700 1 -- start start 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a95d1a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a90073390 0x7f1a901995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a95d1a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a90073cb0 0x7f1a90199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a95d1a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a9010d6f0 0x7f1a9019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a95d1a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9019e510 con 0x7f1a90073390 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a95d1a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9019e680 con 0x7f1a9010d6f0 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a95d1a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9019e840 con 0x7f1a90073cb0 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a8f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a90073390 0x7f1a901995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a8f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a90073390 0x7f1a901995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58468/0 (socket says 172.21.15.12:58468) 2024-05-08T23:49:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a8f7fe700 1 -- 172.21.15.12:0/2868902255 learned_addr learned my addr 172.21.15.12:0/2868902255 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a8ffff700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a9010d6f0 0x7f1a9019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.842+0000 7f1a8effd700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a90073cb0 0x7f1a90199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a8f7fe700 1 -- 172.21.15.12:0/2868902255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a90073cb0 msgr2=0x7f1a90199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a90073cb0 0x7f1a90199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a8f7fe700 1 -- 172.21.15.12:0/2868902255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a9010d6f0 msgr2=0x7f1a9019ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a9010d6f0 0x7f1a9019ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a8f7fe700 1 -- 172.21.15.12:0/2868902255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a800095f0 con 0x7f1a90073390 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a90073390 0x7f1a901995f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000ec10 tx=0x7f1a8000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a8cff9700 1 -- 172.21.15.12:0/2868902255 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000cc80 con 0x7f1a90073390 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a8cff9700 1 -- 172.21.15.12:0/2868902255 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a800044e0 con 0x7f1a90073390 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a95d1a700 1 -- 172.21.15.12:0/2868902255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a88009c40 con 0x7f1a90073390 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a8cff9700 1 -- 172.21.15.12:0/2868902255 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80010430 con 0x7f1a90073390 2024-05-08T23:49:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.843+0000 7f1a95d1a700 1 -- 172.21.15.12:0/2868902255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a84009520 con 0x7f1a90073390 2024-05-08T23:49:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.846+0000 7f1a8cff9700 1 -- 172.21.15.12:0/2868902255 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1a80003f90 con 0x7f1a90073390 2024-05-08T23:49:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.846+0000 7f1a8cff9700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a780a1290 0x7f1a780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.847+0000 7f1a8cff9700 1 -- 172.21.15.12:0/2868902255 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1a80014070 con 0x7f1a90073390 2024-05-08T23:49:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.847+0000 7f1a8effd700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a780a1290 0x7f1a780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.847+0000 7f1a8effd700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a780a1290 0x7f1a780a3750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1a840098c0 tx=0x7f1a84019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1aa1ed4b80 1 Processor -- start 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1aa1ed4b80 1 -- start start 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 0x55646bc9ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bca08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bbf70f0 con 0x55646bca0480 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bbf4ab0 con 0x55646bca5950 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bbf03a0 con 0x55646bca6270 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1a8f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1a8f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:51130/0 (socket says 172.21.15.12:51130) 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1a8ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bca08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1a8f7fe700 1 -- 172.21.15.12:0/3887009793 learned_addr learned my addr 172.21.15.12:0/3887009793 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.848+0000 7f1a8effd700 1 --2- 172.21.15.12:0/3887009793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 0x55646bc9ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.849+0000 7f1a8f7fe700 1 -- 172.21.15.12:0/3887009793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 msgr2=0x55646bc9ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.849+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/3887009793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 0x55646bc9ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.849+0000 7f1a8f7fe700 1 -- 172.21.15.12:0/3887009793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 msgr2=0x55646bca08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.849+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/3887009793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bca08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.849+0000 7f1a8f7fe700 1 -- 172.21.15.12:0/3887009793 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55646ba71350 con 0x55646bca5950 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.849+0000 7f1a8ffff700 1 --2- 172.21.15.12:0/3887009793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bca08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.849+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/3887009793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca5d30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1a80005d70 tx=0x7f1a8009a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60ebac909d6f7eee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.849+0000 7f1a75ffb700 1 -- 172.21.15.12:0/3887009793 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80016330 con 0x55646bca5950 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.849+0000 7f1a75ffb700 1 -- 172.21.15.12:0/3887009793 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a8009a9c0 con 0x55646bca5950 2024-05-08T23:49:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.849+0000 7f1a75ffb700 1 -- 172.21.15.12:0/3887009793 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a800969d0 con 0x55646bca5950 2024-05-08T23:49:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.850+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3887009793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 msgr2=0x55646bca5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.850+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3887009793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca5d30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1a80005d70 tx=0x7f1a8009a160 comp rx=0 tx=0).stop 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.850+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3887009793 shutdown_connections 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.850+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3887009793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bca08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.850+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3887009793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca5d30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.850+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3887009793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 0x55646bc9ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.850+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3887009793 >> 172.21.15.12:0/3887009793 conn(0x55646bbfe1b0 msgr2=0x55646bc9a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.850+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3887009793 shutdown_connections 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.850+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3887009793 wait complete. 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.850+0000 7f1aa1ed4b80 1 -- start start 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.851+0000 7f1aa1ed4b80 1 Processor -- start 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.851+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bcac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.851+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca5950 0x55646bca9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.851+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca6270 0x55646bca9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.851+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bb8c500 con 0x55646bca0480 2024-05-08T23:49:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.851+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bb8c1a0 con 0x55646bca6270 2024-05-08T23:49:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.851+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646ba73640 con 0x55646bca5950 2024-05-08T23:49:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.851+0000 7f1aa1ed4b80 10 client.0 _ll_register_callbacks cb 0x55646bb95ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.851+0000 7f1a8f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bcac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bcac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58488/0 (socket says 172.21.15.12:58488) 2024-05-08T23:49:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8f7fe700 1 -- 172.21.15.12:0/3754227584 learned_addr learned my addr 172.21.15.12:0/3754227584 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8effd700 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca5950 0x55646bca9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8ffff700 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca6270 0x55646bca9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8effd700 1 -- 172.21.15.12:0/3754227584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca6270 msgr2=0x55646bca9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8effd700 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca6270 0x55646bca9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8effd700 1 -- 172.21.15.12:0/3754227584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 msgr2=0x55646bcac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8effd700 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bcac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8effd700 1 -- 172.21.15.12:0/3754227584 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8800e330 con 0x55646bca5950 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bcac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a8effd700 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca5950 0x55646bca9590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009fd0 tx=0x7f1a84009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b83878d62644dee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.852+0000 7f1a6effd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1a6effd700 1 -- 172.21.15.12:0/3754227584 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8402b040 con 0x55646bca5950 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1a6effd700 1 -- 172.21.15.12:0/3754227584 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1a8400fe70 con 0x55646bca5950 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3754227584 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55646bb49210 con 0x55646bca5950 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1a6effd700 1 -- 172.21.15.12:0/3754227584 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a8400bc20 con 0x55646bca5950 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1aa1ed4b80 10 client.25945 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3754227584 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f1a84009520 con 0x55646bca5950 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1a6effd700 1 -- 172.21.15.12:0/3754227584 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a8403c020 con 0x55646bca5950 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1a6effd700 1 -- 172.21.15.12:0/3754227584 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1a84003d00 con 0x55646bca5950 2024-05-08T23:49:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1aa1ed4b80 10 client.25945 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1aa1ed4b80 10 client.25945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1aa1ed4b80 20 client.25945 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55646bcb9c10 0x55646bcbc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1aa1ed4b80 4 client.25945 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3754227584 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55646bc6f7b0 con 0x55646bcb9c10 2024-05-08T23:49:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.853+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55646bcb9c10 0x55646bcbc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.854+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55646bcb9c10 0x55646bcbc030 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763129fd52b49d42 server_cookie=1011b4743475d7aa in_seq=0 out_seq=0 2024-05-08T23:49:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.854+0000 7f1a6effd700 10 client.25945 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:02.856 INFO:tasks.workunit.client.0.smithi012.stdout: 1 69875 6.75 MB/sec warmup 58 sec latency 25.155 ms 2024-05-08T23:49:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1a6effd700 1 -- 172.21.15.12:0/3754227584 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55646bc6f7b0 con 0x55646bcb9c10 2024-05-08T23:49:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1a6effd700 10 client.25945 handle_command_reply: tid=0 2024-05-08T23:49:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 client.25945 shutdown 2024-05-08T23:49:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3754227584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca5950 msgr2=0x55646bca9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca5950 0x55646bca9590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009fd0 tx=0x7f1a84009c00 comp rx=0 tx=0).stop 2024-05-08T23:49:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3754227584 shutdown_connections 2024-05-08T23:49:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55646bcb9c10 0x55646bcbc030 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca6270 0x55646bca9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1a6effd700 0 client.25945 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca0480 0x55646bcac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3754227584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca5950 0x55646bca9590 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3754227584 >> 172.21.15.12:0/3754227584 conn(0x55646bbfe1b0 msgr2=0x55646bc9ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3754227584 shutdown_connections 2024-05-08T23:49:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.855+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3754227584 wait complete. 2024-05-08T23:49:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.856+0000 7f1aa1ed4b80 20 client.25945 trim_cache size 0 max 16384 2024-05-08T23:49:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.873+0000 7f1aa1ed4b80 1 Processor -- start 2024-05-08T23:49:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.873+0000 7f1aa1ed4b80 1 -- start start 2024-05-08T23:49:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.873+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca0480 0x55646bc9a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.873+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.873+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca6270 0x55646bcabdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.873+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bc9ff40 con 0x55646bca6270 2024-05-08T23:49:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.873+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bc92a40 con 0x55646bca5950 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.873+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bcaab90 con 0x55646bca0480 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.873+0000 7f1a8ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca6270 0x55646bcabdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.873+0000 7f1a8ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca6270 0x55646bcabdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58492/0 (socket says 172.21.15.12:58492) 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a8ffff700 1 -- 172.21.15.12:0/4010901043 learned_addr learned my addr 172.21.15.12:0/4010901043 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a8effd700 1 --2- 172.21.15.12:0/4010901043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/4010901043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca0480 0x55646bc9a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a8effd700 1 -- 172.21.15.12:0/4010901043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca0480 msgr2=0x55646bc9a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a8effd700 1 --2- 172.21.15.12:0/4010901043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca0480 0x55646bc9a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a8effd700 1 -- 172.21.15.12:0/4010901043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca6270 msgr2=0x55646bcabdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a8effd700 1 --2- 172.21.15.12:0/4010901043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca6270 0x55646bcabdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a8effd700 1 -- 172.21.15.12:0/4010901043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55646ba1db00 con 0x55646bca5950 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a8ffff700 1 --2- 172.21.15.12:0/4010901043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca6270 0x55646bcabdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a8effd700 1 --2- 172.21.15.12:0/4010901043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca3a90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1a840254e0 tx=0x7f1a84005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d1015d9a1ed9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a767fc700 1 -- 172.21.15.12:0/4010901043 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8402b070 con 0x55646bca5950 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a767fc700 1 -- 172.21.15.12:0/4010901043 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a8403c030 con 0x55646bca5950 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.874+0000 7f1a767fc700 1 -- 172.21.15.12:0/4010901043 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84005460 con 0x55646bca5950 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.875+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/4010901043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 msgr2=0x55646bca3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.875+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/4010901043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca3a90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1a840254e0 tx=0x7f1a84005dc0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.875+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/4010901043 shutdown_connections 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.875+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/4010901043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca6270 0x55646bcabdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.875+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/4010901043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca0480 0x55646bc9a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.875+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/4010901043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca5950 0x55646bca3a90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.875+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/4010901043 >> 172.21.15.12:0/4010901043 conn(0x55646bbfe1b0 msgr2=0x55646bca3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.875+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/4010901043 shutdown_connections 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.875+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/4010901043 wait complete. 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.875+0000 7f1aa1ed4b80 1 -- start start 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1aa1ed4b80 1 Processor -- start 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca0480 0x55646bcaebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca5950 0x55646bcaf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1aa1ed4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 0x55646bd0f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bcaf8a0 con 0x55646bca5950 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bcafa10 con 0x55646bca0480 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1aa1ed4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bd0f650 con 0x55646bca6270 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1aa1ed4b80 10 client.0 _ll_register_callbacks cb 0x55646bb95ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1a8ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 0x55646bd0f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1a8effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca5950 0x55646bcaf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1a8ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 0x55646bd0f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33376/0 (socket says 172.21.15.12:33376) 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1a8ffff700 1 -- 172.21.15.12:0/3862731811 learned_addr learned my addr 172.21.15.12:0/3862731811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.876+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca0480 0x55646bcaebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a8ffff700 1 -- 172.21.15.12:0/3862731811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca0480 msgr2=0x55646bcaebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a8ffff700 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca0480 0x55646bcaebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a8ffff700 1 -- 172.21.15.12:0/3862731811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca5950 msgr2=0x55646bcaf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a8ffff700 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca5950 0x55646bcaf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a8ffff700 1 -- 172.21.15.12:0/3862731811 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a880092c0 con 0x55646bca6270 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a8effd700 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca5950 0x55646bcaf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca0480 0x55646bcaebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a8ffff700 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 0x55646bd0f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1a88004660 tx=0x7f1a88004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d58ec58dcb0f15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a6effd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a6effd700 1 -- 172.21.15.12:0/3862731811 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8801c070 con 0x55646bca6270 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a6effd700 1 -- 172.21.15.12:0/3862731811 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1a88007bc0 con 0x55646bca6270 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3862731811 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a8009a9c0 con 0x55646bca6270 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a6effd700 1 -- 172.21.15.12:0/3862731811 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a8800e040 con 0x55646bca6270 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1aa1ed4b80 10 client.25951 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3862731811 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f1a84009520 con 0x55646bca6270 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.877+0000 7f1a6effd700 1 -- 172.21.15.12:0/3862731811 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a8009a9c0 con 0x55646bca6270 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.878+0000 7f1a6effd700 1 -- 172.21.15.12:0/3862731811 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1a88021460 con 0x55646bca6270 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.878+0000 7f1aa1ed4b80 10 client.25951 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.878+0000 7f1aa1ed4b80 10 client.25951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.878+0000 7f1aa1ed4b80 20 client.25951 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.878+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55646bd10e60 0x55646bd13280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.878+0000 7f1aa1ed4b80 4 client.25951 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.878+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3862731811 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55646bbe8e60 con 0x55646bd10e60 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.878+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55646bd10e60 0x55646bd13280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.878+0000 7f1a6effd700 10 client.25951 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.878+0000 7f1a8f7fe700 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55646bd10e60 0x55646bd13280 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0de57c04762997 server_cookie=3fac13197584567b in_seq=0 out_seq=0 2024-05-08T23:49:02.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:02 smithi012 ceph-mon[42503]: pgmap v82: 97 pgs: 97 active+clean; 26 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 308 KiB/s wr, 102 op/s 2024-05-08T23:49:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1a6effd700 1 -- 172.21.15.12:0/3862731811 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55646bbe8e60 con 0x55646bd10e60 2024-05-08T23:49:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1a6effd700 10 client.25951 handle_command_reply: tid=0 2024-05-08T23:49:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1aa1ed4b80 1 client.25951 shutdown 2024-05-08T23:49:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3862731811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 msgr2=0x55646bd0f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 0x55646bd0f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1a88004660 tx=0x7f1a88004970 comp rx=0 tx=0).stop 2024-05-08T23:49:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3862731811 shutdown_connections 2024-05-08T23:49:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55646bca6270 0x55646bd0f110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55646bd10e60 0x55646bd13280 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55646bca0480 0x55646bcaebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1aa1ed4b80 1 --2- 172.21.15.12:0/3862731811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55646bca5950 0x55646bcaf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1a6effd700 0 client.25951 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3862731811 >> 172.21.15.12:0/3862731811 conn(0x55646bbfe1b0 msgr2=0x55646bca7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.920+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3862731811 shutdown_connections 2024-05-08T23:49:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1aa1ed4b80 1 -- 172.21.15.12:0/3862731811 wait complete. 2024-05-08T23:49:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1aa1ed4b80 20 client.25951 trim_cache size 0 max 16384 2024-05-08T23:49:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:49:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:49:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:49:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:49:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1a95d1a700 1 -- 172.21.15.12:0/2868902255 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a780a1290 msgr2=0x7f1a780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1a95d1a700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a780a1290 0x7f1a780a3750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1a840098c0 tx=0x7f1a84019040 comp rx=0 tx=0).stop 2024-05-08T23:49:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1a95d1a700 1 -- 172.21.15.12:0/2868902255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a90073390 msgr2=0x7f1a901995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1a95d1a700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a90073390 0x7f1a901995f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000ec10 tx=0x7f1a8000ef20 comp rx=0 tx=0).stop 2024-05-08T23:49:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1a95d1a700 1 -- 172.21.15.12:0/2868902255 shutdown_connections 2024-05-08T23:49:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1a95d1a700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a780a1290 0x7f1a780a3750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1a95d1a700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a9010d6f0 0x7f1a9019ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1a95d1a700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a90073390 0x7f1a901995f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1a95d1a700 1 --2- 172.21.15.12:0/2868902255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a90073cb0 0x7f1a90199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.921+0000 7f1a95d1a700 1 -- 172.21.15.12:0/2868902255 >> 172.21.15.12:0/2868902255 conn(0x7f1a900fc940 msgr2=0x7f1a900fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.922+0000 7f1a95d1a700 1 -- 172.21.15.12:0/2868902255 shutdown_connections 2024-05-08T23:49:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:02.922+0000 7f1a95d1a700 1 -- 172.21.15.12:0/2868902255 wait complete. 2024-05-08T23:49:03.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70178 6.70 MB/sec warmup 59 sec latency 906.667 ms 2024-05-08T23:49:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:04 smithi204 ceph-mon[47373]: pgmap v83: 97 pgs: 97 active+clean; 26 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 262 KiB/s wr, 69 op/s 2024-05-08T23:49:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:04 smithi160 ceph-mon[46152]: pgmap v83: 97 pgs: 97 active+clean; 26 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 262 KiB/s wr, 69 op/s 2024-05-08T23:49:04.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70178 6.59 MB/sec warmup 60 sec latency 1906.705 ms 2024-05-08T23:49:04.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:04 smithi012 ceph-mon[42503]: pgmap v83: 97 pgs: 97 active+clean; 26 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 262 KiB/s wr, 69 op/s 2024-05-08T23:49:05.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70178 6.48 MB/sec warmup 61 sec latency 2906.757 ms 2024-05-08T23:49:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:06 smithi204 ceph-mon[47373]: pgmap v84: 97 pgs: 97 active+clean; 26 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 262 KiB/s wr, 69 op/s 2024-05-08T23:49:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:06 smithi160 ceph-mon[46152]: pgmap v84: 97 pgs: 97 active+clean; 26 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 262 KiB/s wr, 69 op/s 2024-05-08T23:49:06.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70178 6.38 MB/sec warmup 62 sec latency 3906.867 ms 2024-05-08T23:49:06.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:06 smithi012 ceph-mon[42503]: pgmap v84: 97 pgs: 97 active+clean; 26 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 262 KiB/s wr, 69 op/s 2024-05-08T23:49:07.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70348 6.30 MB/sec warmup 63 sec latency 4814.756 ms 2024-05-08T23:49:08.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:08 smithi204 ceph-mon[47373]: pgmap v85: 97 pgs: 97 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 525 KiB/s wr, 134 op/s 2024-05-08T23:49:08.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:08 smithi160 ceph-mon[46152]: pgmap v85: 97 pgs: 97 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 525 KiB/s wr, 134 op/s 2024-05-08T23:49:08.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70348 6.20 MB/sec warmup 64 sec latency 1005.799 ms 2024-05-08T23:49:08.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:08 smithi012 ceph-mon[42503]: pgmap v85: 97 pgs: 97 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 525 KiB/s wr, 134 op/s 2024-05-08T23:49:09.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70348 6.10 MB/sec warmup 65 sec latency 2005.928 ms 2024-05-08T23:49:10.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:10 smithi204 ceph-mon[47373]: pgmap v86: 97 pgs: 97 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 512 KiB/s wr, 128 op/s 2024-05-08T23:49:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:10 smithi160 ceph-mon[46152]: pgmap v86: 97 pgs: 97 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 512 KiB/s wr, 128 op/s 2024-05-08T23:49:10.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70348 6.01 MB/sec warmup 66 sec latency 3006.000 ms 2024-05-08T23:49:10.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:10 smithi012 ceph-mon[42503]: pgmap v86: 97 pgs: 97 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 512 KiB/s wr, 128 op/s 2024-05-08T23:49:11.850 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70348 5.92 MB/sec warmup 67 sec latency 4006.067 ms 2024-05-08T23:49:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:12 smithi204 ceph-mon[47373]: pgmap v87: 97 pgs: 97 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 700 KiB/s wr, 144 op/s 2024-05-08T23:49:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:12 smithi160 ceph-mon[46152]: pgmap v87: 97 pgs: 97 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 700 KiB/s wr, 144 op/s 2024-05-08T23:49:12.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70586 5.84 MB/sec warmup 68 sec latency 4916.139 ms 2024-05-08T23:49:12.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:12 smithi012 ceph-mon[42503]: pgmap v87: 97 pgs: 97 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 700 KiB/s wr, 144 op/s 2024-05-08T23:49:13.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70626 5.76 MB/sec warmup 69 sec latency 967.223 ms 2024-05-08T23:49:14.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:14 smithi204 ceph-mon[47373]: pgmap v88: 97 pgs: 97 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 451 KiB/s wr, 80 op/s 2024-05-08T23:49:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:14 smithi160 ceph-mon[46152]: pgmap v88: 97 pgs: 97 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 451 KiB/s wr, 80 op/s 2024-05-08T23:49:14.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70626 5.68 MB/sec warmup 70 sec latency 1967.262 ms 2024-05-08T23:49:14.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:14 smithi012 ceph-mon[42503]: pgmap v88: 97 pgs: 97 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 451 KiB/s wr, 80 op/s 2024-05-08T23:49:15.130 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:49:15.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.282+0000 7fe66bac7700 1 -- 172.21.15.12:0/4104984773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe664101170 msgr2=0x7fe664101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.282+0000 7fe66bac7700 1 --2- 172.21.15.12:0/4104984773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe664101170 0x7fe664101550 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe660009990 tx=0x7fe660009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.282+0000 7fe66bac7700 1 -- 172.21.15.12:0/4104984773 shutdown_connections 2024-05-08T23:49:15.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.282+0000 7fe66bac7700 1 --2- 172.21.15.12:0/4104984773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe664106290 0x7fe66410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.282+0000 7fe66bac7700 1 --2- 172.21.15.12:0/4104984773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe664101170 0x7fe664101550 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.282+0000 7fe66bac7700 1 --2- 172.21.15.12:0/4104984773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe664101b20 0x7fe664105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.283+0000 7fe66bac7700 1 -- 172.21.15.12:0/4104984773 >> 172.21.15.12:0/4104984773 conn(0x7fe6640fc9d0 msgr2=0x7fe6640fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.283+0000 7fe66bac7700 1 -- 172.21.15.12:0/4104984773 shutdown_connections 2024-05-08T23:49:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.283+0000 7fe66bac7700 1 -- 172.21.15.12:0/4104984773 wait complete. 2024-05-08T23:49:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.283+0000 7fe66bac7700 1 Processor -- start 2024-05-08T23:49:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.283+0000 7fe66bac7700 1 -- start start 2024-05-08T23:49:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.283+0000 7fe66bac7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe664101170 0x7fe6641973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.283+0000 7fe66bac7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe664106290 0x7fe664197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe66bac7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66419bcb0 0x7fe66419c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe66bac7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66419c730 con 0x7fe664106290 2024-05-08T23:49:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe66bac7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66419c8a0 con 0x7fe664101170 2024-05-08T23:49:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe66bac7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66419ca60 con 0x7fe66419bcb0 2024-05-08T23:49:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe669062700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe664106290 0x7fe664197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe669062700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe664106290 0x7fe664197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55428/0 (socket says 172.21.15.12:55428) 2024-05-08T23:49:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe669062700 1 -- 172.21.15.12:0/2069160107 learned_addr learned my addr 172.21.15.12:0/2069160107 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe669863700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe664101170 0x7fe6641973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe66a064700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66419bcb0 0x7fe66419c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe669062700 1 -- 172.21.15.12:0/2069160107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66419bcb0 msgr2=0x7fe66419c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe669062700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66419bcb0 0x7fe66419c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe669062700 1 -- 172.21.15.12:0/2069160107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe664101170 msgr2=0x7fe6641973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe669062700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe664101170 0x7fe6641973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe669062700 1 -- 172.21.15.12:0/2069160107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600095f0 con 0x7fe664106290 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe66a064700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66419bcb0 0x7fe66419c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe669062700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe664106290 0x7fe664197910 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe65400d980 tx=0x7fe65400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe65affd700 1 -- 172.21.15.12:0/2069160107 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654009b40 con 0x7fe664106290 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe65affd700 1 -- 172.21.15.12:0/2069160107 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe65400de60 con 0x7fe664106290 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe66bac7700 1 -- 172.21.15.12:0/2069160107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6641a3fc0 con 0x7fe664106290 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.284+0000 7fe65affd700 1 -- 172.21.15.12:0/2069160107 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65400f620 con 0x7fe664106290 2024-05-08T23:49:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.285+0000 7fe66bac7700 1 -- 172.21.15.12:0/2069160107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6641a4440 con 0x7fe664106290 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.287+0000 7fe65affd700 1 -- 172.21.15.12:0/2069160107 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe654010460 con 0x7fe664106290 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.288+0000 7fe65affd700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6500a1290 0x7fe6500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.288+0000 7fe669863700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6500a1290 0x7fe6500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.288+0000 7fe65affd700 1 -- 172.21.15.12:0/2069160107 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe6540cc870 con 0x7fe664106290 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.288+0000 7fe669863700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6500a1290 0x7fe6500a3750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe660006010 tx=0x7fe66000b540 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe677c81b80 1 Processor -- start 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe677c81b80 1 -- start start 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029bc310 0x55cf029bc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029b6ca0 0x55cf029b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe677c81b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf02909150 con 0x55cf029bc310 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe677c81b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf02906b10 con 0x55cf029b6ca0 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe677c81b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf02902400 con 0x55cf029bcc30 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe669863700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029bc310 0x55cf029bc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe669863700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029bc310 0x55cf029bc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55430/0 (socket says 172.21.15.12:55430) 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe669863700 1 -- 172.21.15.12:0/2328652827 learned_addr learned my addr 172.21.15.12:0/2328652827 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.289+0000 7fe669062700 1 --2- 172.21.15.12:0/2328652827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.290+0000 7fe669863700 1 -- 172.21.15.12:0/2328652827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 msgr2=0x55cf029b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.290+0000 7fe669863700 1 --2- 172.21.15.12:0/2328652827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.290+0000 7fe669863700 1 -- 172.21.15.12:0/2328652827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029b6ca0 msgr2=0x55cf029b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.290+0000 7fe669863700 1 --2- 172.21.15.12:0/2328652827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029b6ca0 0x55cf029b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.290+0000 7fe669863700 1 -- 172.21.15.12:0/2328652827 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf027833b0 con 0x55cf029bc310 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.290+0000 7fe669863700 1 --2- 172.21.15.12:0/2328652827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029bc310 0x55cf029bc6f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe6600052b0 tx=0x7fe66001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3970b2ebc6233a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.290+0000 7fe64ffff700 1 -- 172.21.15.12:0/2328652827 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66002b070 con 0x55cf029bc310 2024-05-08T23:49:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.290+0000 7fe64ffff700 1 -- 172.21.15.12:0/2328652827 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe66001e040 con 0x55cf029bc310 2024-05-08T23:49:15.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.290+0000 7fe64ffff700 1 -- 172.21.15.12:0/2328652827 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660004a60 con 0x55cf029bc310 2024-05-08T23:49:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.291+0000 7fe677c81b80 1 -- 172.21.15.12:0/2328652827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029bc310 msgr2=0x55cf029bc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.291+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2328652827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029bc310 0x55cf029bc6f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe6600052b0 tx=0x7fe66001f000 comp rx=0 tx=0).stop 2024-05-08T23:49:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.291+0000 7fe677c81b80 1 -- 172.21.15.12:0/2328652827 shutdown_connections 2024-05-08T23:49:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.291+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2328652827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029b6ca0 0x55cf029b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.291+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2328652827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029bc310 0x55cf029bc6f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.291+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2328652827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.291+0000 7fe677c81b80 1 -- 172.21.15.12:0/2328652827 >> 172.21.15.12:0/2328652827 conn(0x55cf02910210 msgr2=0x55cf029acdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.291+0000 7fe677c81b80 1 -- 172.21.15.12:0/2328652827 shutdown_connections 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.291+0000 7fe677c81b80 1 -- 172.21.15.12:0/2328652827 wait complete. 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.291+0000 7fe677c81b80 1 -- start start 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe677c81b80 1 Processor -- start 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bc310 0x55cf029bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe677c81b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf0289e560 con 0x55cf029b6ca0 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe677c81b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf0289e200 con 0x55cf029bc310 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe677c81b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf027856a0 con 0x55cf029bcc30 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe677c81b80 10 client.0 _ll_register_callbacks cb 0x55cf028a7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe669863700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe669863700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55436/0 (socket says 172.21.15.12:55436) 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.292+0000 7fe669863700 1 -- 172.21.15.12:0/149586733 learned_addr learned my addr 172.21.15.12:0/149586733 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe669062700 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bc310 0x55cf029bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe66a064700 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe669863700 1 -- 172.21.15.12:0/149586733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 msgr2=0x55cf029c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe669863700 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029c0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe669863700 1 -- 172.21.15.12:0/149586733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bc310 msgr2=0x55cf029bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe669863700 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bc310 0x55cf029bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe669863700 1 -- 172.21.15.12:0/149586733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66410e680 con 0x55cf029b6ca0 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe669863700 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c2bd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe660005f50 tx=0x7fe6600049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428b55bf892912b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe64d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe64d7fa700 1 -- 172.21.15.12:0/149586733 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66002b040 con 0x55cf029b6ca0 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe64d7fa700 1 -- 172.21.15.12:0/149586733 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe66000bb80 con 0x55cf029b6ca0 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe677c81b80 1 -- 172.21.15.12:0/149586733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf0285b270 con 0x55cf029b6ca0 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe64d7fa700 1 -- 172.21.15.12:0/149586733 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66000bd60 con 0x55cf029b6ca0 2024-05-08T23:49:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe677c81b80 10 client.16137 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.293+0000 7fe677c81b80 1 -- 172.21.15.12:0/149586733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cf0272fb60 con 0x55cf029b6ca0 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe64d7fa700 1 -- 172.21.15.12:0/149586733 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe660045020 con 0x55cf029b6ca0 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe64d7fa700 1 -- 172.21.15.12:0/149586733 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe66000f460 con 0x55cf029b6ca0 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe677c81b80 10 client.16137 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe677c81b80 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe677c81b80 20 client.16137 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe677c81b80 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf029cffe0 0x55cf029d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe677c81b80 4 client.16137 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe677c81b80 1 -- 172.21.15.12:0/149586733 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf02981810 con 0x55cf029cffe0 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe669062700 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf029cffe0 0x55cf029d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe64d7fa700 10 client.16137 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.294+0000 7fe669062700 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf029cffe0 0x55cf029d23d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcfab9d56d95cdd8 server_cookie=5dbfebc64f2221a4 in_seq=0 out_seq=0 2024-05-08T23:49:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe64d7fa700 1 -- 172.21.15.12:0/149586733 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf02981810 con 0x55cf029cffe0 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe64d7fa700 10 client.16137 handle_command_reply: tid=0 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe677c81b80 1 client.16137 shutdown 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe677c81b80 1 -- 172.21.15.12:0/149586733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 msgr2=0x55cf029c2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe677c81b80 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c2bd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe660005f50 tx=0x7fe6600049c0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe677c81b80 1 -- 172.21.15.12:0/149586733 shutdown_connections 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe677c81b80 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe677c81b80 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c2bd0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe677c81b80 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf029cffe0 0x55cf029d23d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe677c81b80 1 --2- 172.21.15.12:0/149586733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bc310 0x55cf029bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe64d7fa700 0 client.16137 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.295+0000 7fe677c81b80 1 -- 172.21.15.12:0/149586733 >> 172.21.15.12:0/149586733 conn(0x55cf02910210 msgr2=0x55cf029ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.296+0000 7fe677c81b80 1 -- 172.21.15.12:0/149586733 shutdown_connections 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.296+0000 7fe677c81b80 1 -- 172.21.15.12:0/149586733 wait complete. 2024-05-08T23:49:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.296+0000 7fe677c81b80 20 client.16137 trim_cache size 0 max 16384 2024-05-08T23:49:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe677c81b80 1 Processor -- start 2024-05-08T23:49:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe677c81b80 1 -- start start 2024-05-08T23:49:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bc310 0x55cf029ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029ba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe677c81b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf029a7840 con 0x55cf029b6ca0 2024-05-08T23:49:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe677c81b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf029be410 con 0x55cf029bc310 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe677c81b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf029b6a60 con 0x55cf029bcc30 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe669863700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe669863700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55444/0 (socket says 172.21.15.12:55444) 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe669863700 1 -- 172.21.15.12:0/2256866751 learned_addr learned my addr 172.21.15.12:0/2256866751 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.314+0000 7fe669062700 1 --2- 172.21.15.12:0/2256866751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bc310 0x55cf029ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe66a064700 1 --2- 172.21.15.12:0/2256866751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe669863700 1 -- 172.21.15.12:0/2256866751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 msgr2=0x55cf029ba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe669863700 1 --2- 172.21.15.12:0/2256866751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029ba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe669863700 1 -- 172.21.15.12:0/2256866751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bc310 msgr2=0x55cf029ba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe669863700 1 --2- 172.21.15.12:0/2256866751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bc310 0x55cf029ba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe669863700 1 -- 172.21.15.12:0/2256866751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf028c8560 con 0x55cf029b6ca0 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe66a064700 1 --2- 172.21.15.12:0/2256866751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:49:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe669863700 1 --2- 172.21.15.12:0/2256866751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029acd20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe660009ce0 tx=0x7fe660004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efcd130d421f332 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe658da4700 1 -- 172.21.15.12:0/2256866751 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66002b070 con 0x55cf029b6ca0 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe658da4700 1 -- 172.21.15.12:0/2256866751 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe660045030 con 0x55cf029b6ca0 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe658da4700 1 -- 172.21.15.12:0/2256866751 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660018400 con 0x55cf029b6ca0 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe677c81b80 1 -- 172.21.15.12:0/2256866751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 msgr2=0x55cf029acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.315+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2256866751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029acd20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe660009ce0 tx=0x7fe660004580 comp rx=0 tx=0).stop 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 -- 172.21.15.12:0/2256866751 shutdown_connections 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2256866751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bcc30 0x55cf029ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2256866751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029acd20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2256866751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bc310 0x55cf029ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 -- 172.21.15.12:0/2256866751 >> 172.21.15.12:0/2256866751 conn(0x55cf02910210 msgr2=0x55cf029d6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 -- 172.21.15.12:0/2256866751 shutdown_connections 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 -- 172.21.15.12:0/2256866751 wait complete. 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 -- start start 2024-05-08T23:49:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 Processor -- start 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bc310 0x55cf029c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.316+0000 7fe677c81b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bcc30 0x55cf029c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe677c81b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf02a254d0 con 0x55cf029b6ca0 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe677c81b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf02a25640 con 0x55cf029bcc30 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe677c81b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf02a25800 con 0x55cf029bc310 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe677c81b80 10 client.0 _ll_register_callbacks cb 0x55cf028a7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669863700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669863700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55452/0 (socket says 172.21.15.12:55452) 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669863700 1 -- 172.21.15.12:0/2210624361 learned_addr learned my addr 172.21.15.12:0/2210624361 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe66a064700 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bcc30 0x55cf029c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669062700 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bc310 0x55cf029c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669863700 1 -- 172.21.15.12:0/2210624361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bc310 msgr2=0x55cf029c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669863700 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bc310 0x55cf029c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669863700 1 -- 172.21.15.12:0/2210624361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bcc30 msgr2=0x55cf029c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669863700 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bcc30 0x55cf029c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669863700 1 -- 172.21.15.12:0/2210624361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf0272fb60 con 0x55cf029b6ca0 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669062700 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bc310 0x55cf029c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe66a064700 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bcc30 0x55cf029c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe669863700 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c3480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe660005a80 tx=0x7fe66000bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74d2fda5bd02268 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe64d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe64d7fa700 1 -- 172.21.15.12:0/2210624361 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66002b050 con 0x55cf029b6ca0 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe64d7fa700 1 -- 172.21.15.12:0/2210624361 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe660046080 con 0x55cf029b6ca0 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.317+0000 7fe677c81b80 1 -- 172.21.15.12:0/2210624361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe66001e040 con 0x55cf029b6ca0 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe64d7fa700 1 -- 172.21.15.12:0/2210624361 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66000fa90 con 0x55cf029b6ca0 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe677c81b80 10 client.16143 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe677c81b80 1 -- 172.21.15.12:0/2210624361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cf02787cd0 con 0x55cf029b6ca0 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe64d7fa700 1 -- 172.21.15.12:0/2210624361 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe66001e040 con 0x55cf029b6ca0 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe64d7fa700 1 -- 172.21.15.12:0/2210624361 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe66003f070 con 0x55cf029b6ca0 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe677c81b80 10 client.16143 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe677c81b80 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe677c81b80 20 client.16143 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf02a26aa0 0x55cf02a28e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe677c81b80 4 client.16143 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe677c81b80 1 -- 172.21.15.12:0/2210624361 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cf028faec0 con 0x55cf02a26aa0 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.318+0000 7fe669062700 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf02a26aa0 0x55cf02a28e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe64d7fa700 10 client.16143 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe669062700 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf02a26aa0 0x55cf02a28e90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff13b780d2af149 server_cookie=ed438ac0258a2102 in_seq=0 out_seq=0 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe64d7fa700 1 -- 172.21.15.12:0/2210624361 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cf028faec0 con 0x55cf02a26aa0 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe64d7fa700 10 client.16143 handle_command_reply: tid=0 2024-05-08T23:49:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 client.16143 shutdown 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 -- 172.21.15.12:0/2210624361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 msgr2=0x55cf029c3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c3480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe660005a80 tx=0x7fe66000bc40 comp rx=0 tx=0).stop 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 -- 172.21.15.12:0/2210624361 shutdown_connections 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf02a26aa0 0x55cf02a28e90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf029bcc30 0x55cf029c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf029b6ca0 0x55cf029c3480 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 --2- 172.21.15.12:0/2210624361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf029bc310 0x55cf029c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe64d7fa700 0 client.16143 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 -- 172.21.15.12:0/2210624361 >> 172.21.15.12:0/2210624361 conn(0x55cf02910210 msgr2=0x55cf029ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 -- 172.21.15.12:0/2210624361 shutdown_connections 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 1 -- 172.21.15.12:0/2210624361 wait complete. 2024-05-08T23:49:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.319+0000 7fe677c81b80 20 client.16143 trim_cache size 0 max 16384 2024-05-08T23:49:15.322 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:49:15.322 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:49:15.322 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:49:15.322 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:49:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 -- 172.21.15.12:0/2069160107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6500a1290 msgr2=0x7fe6500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6500a1290 0x7fe6500a3750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe660006010 tx=0x7fe66000b540 comp rx=0 tx=0).stop 2024-05-08T23:49:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 -- 172.21.15.12:0/2069160107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe664106290 msgr2=0x7fe664197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe664106290 0x7fe664197910 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe65400d980 tx=0x7fe65400dd40 comp rx=0 tx=0).stop 2024-05-08T23:49:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 -- 172.21.15.12:0/2069160107 shutdown_connections 2024-05-08T23:49:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6500a1290 0x7fe6500a3750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66419bcb0 0x7fe66419c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe664101170 0x7fe6641973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 --2- 172.21.15.12:0/2069160107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe664106290 0x7fe664197910 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 -- 172.21.15.12:0/2069160107 >> 172.21.15.12:0/2069160107 conn(0x7fe6640fc9d0 msgr2=0x7fe6640fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 -- 172.21.15.12:0/2069160107 shutdown_connections 2024-05-08T23:49:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.320+0000 7fe66bac7700 1 -- 172.21.15.12:0/2069160107 wait complete. 2024-05-08T23:49:15.328 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:49:15.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:49:15.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.471+0000 7f6e78ca2700 1 -- 172.21.15.12:0/3690410073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e74101fe0 msgr2=0x7f6e741023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.471+0000 7f6e78ca2700 1 --2- 172.21.15.12:0/3690410073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e74101fe0 0x7f6e741023c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400b0a0 tx=0x7f6e6400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.471+0000 7f6e78ca2700 1 -- 172.21.15.12:0/3690410073 shutdown_connections 2024-05-08T23:49:15.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.471+0000 7f6e78ca2700 1 --2- 172.21.15.12:0/3690410073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e7410b400 0x7f6e7410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.472+0000 7f6e78ca2700 1 --2- 172.21.15.12:0/3690410073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e74101fe0 0x7f6e741023c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.472+0000 7f6e78ca2700 1 --2- 172.21.15.12:0/3690410073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e74102900 0x7f6e7410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.472+0000 7f6e78ca2700 1 -- 172.21.15.12:0/3690410073 >> 172.21.15.12:0/3690410073 conn(0x7f6e740fb7e0 msgr2=0x7f6e740fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.472+0000 7f6e78ca2700 1 -- 172.21.15.12:0/3690410073 shutdown_connections 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.472+0000 7f6e78ca2700 1 -- 172.21.15.12:0/3690410073 wait complete. 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.472+0000 7f6e78ca2700 1 Processor -- start 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.472+0000 7f6e78ca2700 1 -- start start 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e78ca2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e74101fe0 0x7f6e74197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e78ca2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e74102900 0x7f6e741979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e78ca2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e7410b400 0x7f6e7419bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e78ca2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7419c340 con 0x7f6e7410b400 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e78ca2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7419c4b0 con 0x7f6e74101fe0 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e78ca2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7419c670 con 0x7f6e74102900 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e72d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e7410b400 0x7f6e7419bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e7259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e74101fe0 0x7f6e74197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e72d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e7410b400 0x7f6e7419bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55462/0 (socket says 172.21.15.12:55462) 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e7259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e74101fe0 0x7f6e74197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41806/0 (socket says 172.21.15.12:41806) 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e72d9d700 1 -- 172.21.15.12:0/1995662036 learned_addr learned my addr 172.21.15.12:0/1995662036 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e71d9b700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e74102900 0x7f6e741979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e72d9d700 1 -- 172.21.15.12:0/1995662036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e74102900 msgr2=0x7f6e741979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e72d9d700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e74102900 0x7f6e741979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e72d9d700 1 -- 172.21.15.12:0/1995662036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e74101fe0 msgr2=0x7f6e74197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e72d9d700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e74101fe0 0x7f6e74197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e72d9d700 1 -- 172.21.15.12:0/1995662036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64009d00 con 0x7f6e7410b400 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.473+0000 7f6e72d9d700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e7410b400 0x7f6e7419bd00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00e940 tx=0x7f6e6c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.474+0000 7f6e637fe700 1 -- 172.21.15.12:0/1995662036 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0097b0 con 0x7f6e7410b400 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.474+0000 7f6e78ca2700 1 -- 172.21.15.12:0/1995662036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7419c950 con 0x7f6e7410b400 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.474+0000 7f6e637fe700 1 -- 172.21.15.12:0/1995662036 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e6c00ee90 con 0x7f6e7410b400 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.474+0000 7f6e637fe700 1 -- 172.21.15.12:0/1995662036 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c005520 con 0x7f6e7410b400 2024-05-08T23:49:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.474+0000 7f6e78ca2700 1 -- 172.21.15.12:0/1995662036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e741a4190 con 0x7f6e7410b400 2024-05-08T23:49:15.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.475+0000 7f6e78ca2700 1 -- 172.21.15.12:0/1995662036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e74106490 con 0x7f6e7410b400 2024-05-08T23:49:15.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.476+0000 7f6e637fe700 1 -- 172.21.15.12:0/1995662036 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6e6c004750 con 0x7f6e7410b400 2024-05-08T23:49:15.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.477+0000 7f6e637fe700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e5c0a1290 0x7f6e5c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.477+0000 7f6e7259c700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e5c0a1290 0x7f6e5c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.477+0000 7f6e637fe700 1 -- 172.21.15.12:0/1995662036 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6e6c014070 con 0x7f6e7410b400 2024-05-08T23:49:15.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.478+0000 7f6e7259c700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e5c0a1290 0x7f6e5c0a3750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009fd0 tx=0x7f6e6400b720 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.481+0000 7f6e637fe700 1 -- 172.21.15.12:0/1995662036 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6e6c0964b0 con 0x7f6e7410b400 2024-05-08T23:49:15.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.755+0000 7f6e78ca2700 1 -- 172.21.15.12:0/1995662036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e74068890 con 0x7f6e7410b400 2024-05-08T23:49:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.756+0000 7f6e637fe700 1 -- 172.21.15.12:0/1995662036 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10879 (secure 0 0 0) 0x7f6e6c005740 con 0x7f6e7410b400 2024-05-08T23:49:15.751 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:49:15.751 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:48:23.429180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25489":{"gid":25489,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3829900343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3829900343},{"type":"v1","addr":"172.21.15.204:6837","nonce":3829900343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 -- 172.21.15.12:0/1995662036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e5c0a1290 msgr2=0x7f6e5c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e5c0a1290 0x7f6e5c0a3750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009fd0 tx=0x7f6e6400b720 comp rx=0 tx=0).stop 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 -- 172.21.15.12:0/1995662036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e7410b400 msgr2=0x7f6e7419bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e7410b400 0x7f6e7419bd00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00e940 tx=0x7f6e6c00ec50 comp rx=0 tx=0).stop 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 -- 172.21.15.12:0/1995662036 shutdown_connections 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e5c0a1290 0x7f6e5c0a3750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e7410b400 0x7f6e7419bd00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e74101fe0 0x7f6e74197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 --2- 172.21.15.12:0/1995662036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e74102900 0x7f6e741979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 -- 172.21.15.12:0/1995662036 >> 172.21.15.12:0/1995662036 conn(0x7f6e740fb7e0 msgr2=0x7f6e740fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 -- 172.21.15.12:0/1995662036 shutdown_connections 2024-05-08T23:49:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.758+0000 7f6e78ca2700 1 -- 172.21.15.12:0/1995662036 wait complete. 2024-05-08T23:49:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-05-08T23:49:15.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:15 smithi204 ceph-mon[47373]: pgmap v89: 97 pgs: 97 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 451 KiB/s wr, 80 op/s 2024-05-08T23:49:15.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:15 smithi160 ceph-mon[46152]: pgmap v89: 97 pgs: 97 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 451 KiB/s wr, 80 op/s 2024-05-08T23:49:15.766 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:15 smithi012 ceph-mon[42503]: pgmap v89: 97 pgs: 97 active+clean; 29 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 451 KiB/s wr, 80 op/s 2024-05-08T23:49:15.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:49:15.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70626 5.60 MB/sec warmup 71 sec latency 2967.317 ms 2024-05-08T23:49:15.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.910+0000 7f9847041700 1 -- 172.21.15.12:0/1831437194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9840106280 msgr2=0x7f984010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.910+0000 7f9847041700 1 --2- 172.21.15.12:0/1831437194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9840106280 0x7f984010de00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f983800b0a0 tx=0x7f983800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.910+0000 7f9847041700 1 -- 172.21.15.12:0/1831437194 shutdown_connections 2024-05-08T23:49:15.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.911+0000 7f9847041700 1 --2- 172.21.15.12:0/1831437194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9840106280 0x7f984010de00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.911+0000 7f9847041700 1 --2- 172.21.15.12:0/1831437194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98400697d0 0x7f9840069bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.911+0000 7f9847041700 1 --2- 172.21.15.12:0/1831437194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f984006a180 0x7f9840105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.911+0000 7f9847041700 1 -- 172.21.15.12:0/1831437194 >> 172.21.15.12:0/1831437194 conn(0x7f9840077000 msgr2=0x7f9840077410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.911+0000 7f9847041700 1 -- 172.21.15.12:0/1831437194 shutdown_connections 2024-05-08T23:49:15.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.911+0000 7f9847041700 1 -- 172.21.15.12:0/1831437194 wait complete. 2024-05-08T23:49:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.911+0000 7f9847041700 1 Processor -- start 2024-05-08T23:49:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.911+0000 7f9847041700 1 -- start start 2024-05-08T23:49:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f9847041700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98400697d0 0x7f9840199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f9847041700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9840106280 0x7f9840199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f9847041700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f984019cfa0 0x7f984019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f9847041700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984019d990 con 0x7f9840106280 2024-05-08T23:49:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f9847041700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984019db00 con 0x7f98400697d0 2024-05-08T23:49:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f9847041700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984019dcc0 con 0x7f984019cfa0 2024-05-08T23:49:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f983ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9840106280 0x7f9840199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f983ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9840106280 0x7f9840199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55480/0 (socket says 172.21.15.12:55480) 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f983ffff700 1 -- 172.21.15.12:0/1065762540 learned_addr learned my addr 172.21.15.12:0/1065762540 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f9844ddd700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98400697d0 0x7f9840199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f98455de700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f984019cfa0 0x7f984019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f983ffff700 1 -- 172.21.15.12:0/1065762540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f984019cfa0 msgr2=0x7f984019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f983ffff700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f984019cfa0 0x7f984019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f983ffff700 1 -- 172.21.15.12:0/1065762540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98400697d0 msgr2=0x7f9840199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f983ffff700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98400697d0 0x7f9840199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f983ffff700 1 -- 172.21.15.12:0/1065762540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9838009d00 con 0x7f9840106280 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.912+0000 7f98455de700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f984019cfa0 0x7f984019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.913+0000 7f983ffff700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9840106280 0x7f9840199cb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f982c00e970 tx=0x7f982c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.913+0000 7f983dffb700 1 -- 172.21.15.12:0/1065762540 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c009790 con 0x7f9840106280 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.913+0000 7f9847041700 1 -- 172.21.15.12:0/1065762540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98401a6330 con 0x7f9840106280 2024-05-08T23:49:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.913+0000 7f983dffb700 1 -- 172.21.15.12:0/1065762540 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f982c004cf0 con 0x7f9840106280 2024-05-08T23:49:15.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.913+0000 7f983dffb700 1 -- 172.21.15.12:0/1065762540 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c005680 con 0x7f9840106280 2024-05-08T23:49:15.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.913+0000 7f9847041700 1 -- 172.21.15.12:0/1065762540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98401a6840 con 0x7f9840106280 2024-05-08T23:49:15.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.915+0000 7f983dffb700 1 -- 172.21.15.12:0/1065762540 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f982c004750 con 0x7f9840106280 2024-05-08T23:49:15.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.916+0000 7f983dffb700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98300a12c0 0x7f98300a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.916+0000 7f983dffb700 1 -- 172.21.15.12:0/1065762540 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f982c014070 con 0x7f9840106280 2024-05-08T23:49:15.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.916+0000 7f9844ddd700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98300a12c0 0x7f98300a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.917+0000 7f9844ddd700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98300a12c0 0x7f98300a3780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9834009bf0 tx=0x7f9834009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.917+0000 7f98531fbb80 1 Processor -- start 2024-05-08T23:49:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.917+0000 7f98531fbb80 1 -- start start 2024-05-08T23:49:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.917+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd9d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.917+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 0x55f99cd97350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.917+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd97890 0x55f99cd97cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.917+0000 7f98531fbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99ccee150 con 0x55f99cd9ce00 2024-05-08T23:49:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.917+0000 7f98531fbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99ccebb10 con 0x55f99cd97890 2024-05-08T23:49:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.917+0000 7f98531fbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99cce7400 con 0x55f99cd9d720 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.917+0000 7f9844ddd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd9d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f9844ddd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd9d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55496/0 (socket says 172.21.15.12:55496) 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f9844ddd700 1 -- 172.21.15.12:0/639041148 learned_addr learned my addr 172.21.15.12:0/639041148 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f983ffff700 1 --2- 172.21.15.12:0/639041148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 0x55f99cd97350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f98455de700 1 --2- 172.21.15.12:0/639041148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd97890 0x55f99cd97cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f9844ddd700 1 -- 172.21.15.12:0/639041148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 msgr2=0x55f99cd97350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f9844ddd700 1 --2- 172.21.15.12:0/639041148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 0x55f99cd97350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f9844ddd700 1 -- 172.21.15.12:0/639041148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd97890 msgr2=0x55f99cd97cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f9844ddd700 1 --2- 172.21.15.12:0/639041148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd97890 0x55f99cd97cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f9844ddd700 1 -- 172.21.15.12:0/639041148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f99cb683b0 con 0x55f99cd9ce00 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f983ffff700 1 --2- 172.21.15.12:0/639041148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 0x55f99cd97350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f9844ddd700 1 --2- 172.21.15.12:0/639041148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd9d1e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9834011e40 tx=0x7f9834014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8700c39eea553efc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f982affd700 1 -- 172.21.15.12:0/639041148 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834004220 con 0x55f99cd9ce00 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f982affd700 1 -- 172.21.15.12:0/639041148 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9834016070 con 0x55f99cd9ce00 2024-05-08T23:49:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.918+0000 7f982affd700 1 -- 172.21.15.12:0/639041148 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983401f680 con 0x55f99cd9ce00 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 -- 172.21.15.12:0/639041148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 msgr2=0x55f99cd9d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 --2- 172.21.15.12:0/639041148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd9d1e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9834011e40 tx=0x7f9834014000 comp rx=0 tx=0).stop 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 -- 172.21.15.12:0/639041148 shutdown_connections 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 --2- 172.21.15.12:0/639041148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd97890 0x55f99cd97cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 --2- 172.21.15.12:0/639041148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd9d1e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 --2- 172.21.15.12:0/639041148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 0x55f99cd97350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 -- 172.21.15.12:0/639041148 >> 172.21.15.12:0/639041148 conn(0x55f99ccf5210 msgr2=0x55f99cd91df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 -- 172.21.15.12:0/639041148 shutdown_connections 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 -- 172.21.15.12:0/639041148 wait complete. 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 -- start start 2024-05-08T23:49:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.919+0000 7f98531fbb80 1 Processor -- start 2024-05-08T23:49:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd97890 0x55f99cda3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9ce00 0x55f99cda09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 0x55f99cda1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f98531fbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99cc83560 con 0x55f99cd97890 2024-05-08T23:49:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f98531fbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99cc83200 con 0x55f99cd9ce00 2024-05-08T23:49:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f98531fbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99cb6a6a0 con 0x55f99cd9d720 2024-05-08T23:49:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f98531fbb80 10 client.0 _ll_register_callbacks cb 0x55f99cc8cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f9844ddd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd97890 0x55f99cda3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f9844ddd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd97890 0x55f99cda3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55498/0 (socket says 172.21.15.12:55498) 2024-05-08T23:49:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f9844ddd700 1 -- 172.21.15.12:0/3178048023 learned_addr learned my addr 172.21.15.12:0/3178048023 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f983ffff700 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9ce00 0x55f99cda09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.920+0000 7f98455de700 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 0x55f99cda1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f9844ddd700 1 -- 172.21.15.12:0/3178048023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 msgr2=0x55f99cda1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f9844ddd700 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 0x55f99cda1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f9844ddd700 1 -- 172.21.15.12:0/3178048023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9ce00 msgr2=0x55f99cda09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f9844ddd700 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9ce00 0x55f99cda09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f9844ddd700 1 -- 172.21.15.12:0/3178048023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9834011b20 con 0x55f99cd97890 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f98455de700 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 0x55f99cda1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f983ffff700 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9ce00 0x55f99cda09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f9844ddd700 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd97890 0x55f99cda3750 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9834012ac0 tx=0x7f98340040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1315410d6733d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f981ffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f981ffff700 1 -- 172.21.15.12:0/3178048023 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834005250 con 0x55f99cd97890 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f98531fbb80 1 -- 172.21.15.12:0/3178048023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f99cc40270 con 0x55f99cd97890 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f981ffff700 1 -- 172.21.15.12:0/3178048023 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9834018070 con 0x55f99cd97890 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f981ffff700 1 -- 172.21.15.12:0/3178048023 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98340053b0 con 0x55f99cd97890 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f98531fbb80 10 client.16161 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f98531fbb80 1 -- 172.21.15.12:0/3178048023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f99cb14b60 con 0x55f99cd97890 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.921+0000 7f981ffff700 1 -- 172.21.15.12:0/3178048023 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f99cc40270 con 0x55f99cd97890 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.922+0000 7f981ffff700 1 -- 172.21.15.12:0/3178048023 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9834025070 con 0x55f99cd97890 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.922+0000 7f98531fbb80 10 client.16161 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.922+0000 7f98531fbb80 10 client.16161 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.922+0000 7f98531fbb80 20 client.16161 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.922+0000 7f98531fbb80 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f99cdb10a0 0x55f99cdb3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.922+0000 7f98531fbb80 4 client.16161 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.922+0000 7f98531fbb80 1 -- 172.21.15.12:0/3178048023 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f99cd66810 con 0x55f99cdb10a0 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.922+0000 7f983ffff700 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f99cdb10a0 0x55f99cdb3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.922+0000 7f983ffff700 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f99cdb10a0 0x55f99cdb3490 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a78a0d3c12d1137 server_cookie=eb6bf587cbdd1349 in_seq=0 out_seq=0 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.922+0000 7f981ffff700 10 client.16161 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f981ffff700 1 -- 172.21.15.12:0/3178048023 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f99cd66810 con 0x55f99cdb10a0 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f981ffff700 10 client.16161 handle_command_reply: tid=0 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 client.16161 shutdown 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 -- 172.21.15.12:0/3178048023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd97890 msgr2=0x55f99cda3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd97890 0x55f99cda3750 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9834012ac0 tx=0x7f98340040b0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 -- 172.21.15.12:0/3178048023 shutdown_connections 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f99cdb10a0 0x55f99cdb3490 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd9d720 0x55f99cda1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd97890 0x55f99cda3750 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 --2- 172.21.15.12:0/3178048023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9ce00 0x55f99cda09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f981ffff700 0 client.16161 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:49:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 -- 172.21.15.12:0/3178048023 >> 172.21.15.12:0/3178048023 conn(0x55f99ccf5210 msgr2=0x55f99cd915a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 -- 172.21.15.12:0/3178048023 shutdown_connections 2024-05-08T23:49:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.923+0000 7f98531fbb80 1 -- 172.21.15.12:0/3178048023 wait complete. 2024-05-08T23:49:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.924+0000 7f98531fbb80 20 client.16161 trim_cache size 0 max 16384 2024-05-08T23:49:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f98531fbb80 1 Processor -- start 2024-05-08T23:49:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f98531fbb80 1 -- start start 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 0x55f99cda0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cdacc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9d720 0x55f99cd91d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f98531fbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99cd8edc0 con 0x55f99cd9ce00 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f98531fbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99cdb86a0 con 0x55f99cd9d720 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f98531fbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99cd8c800 con 0x55f99cd97890 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f983ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cdacc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f983ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cdacc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55512/0 (socket says 172.21.15.12:55512) 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f983ffff700 1 -- 172.21.15.12:0/2846190627 learned_addr learned my addr 172.21.15.12:0/2846190627 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f98455de700 1 --2- 172.21.15.12:0/2846190627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9d720 0x55f99cd91d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f9844ddd700 1 --2- 172.21.15.12:0/2846190627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 0x55f99cda0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f983ffff700 1 -- 172.21.15.12:0/2846190627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 msgr2=0x55f99cda0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.938+0000 7f983ffff700 1 --2- 172.21.15.12:0/2846190627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 0x55f99cda0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.939+0000 7f983ffff700 1 -- 172.21.15.12:0/2846190627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9d720 msgr2=0x55f99cd91d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.939+0000 7f983ffff700 1 --2- 172.21.15.12:0/2846190627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9d720 0x55f99cd91d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.939+0000 7f983ffff700 1 -- 172.21.15.12:0/2846190627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f99ccad560 con 0x55f99cd9ce00 2024-05-08T23:49:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.939+0000 7f983ffff700 1 --2- 172.21.15.12:0/2846190627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cdacc20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f982c099d30 tx=0x7f982c0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f551627792487e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.939+0000 7f982b7fe700 1 -- 172.21.15.12:0/2846190627 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0a24b0 con 0x55f99cd9ce00 2024-05-08T23:49:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.939+0000 7f982b7fe700 1 -- 172.21.15.12:0/2846190627 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f982c0983d0 con 0x55f99cd9ce00 2024-05-08T23:49:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.939+0000 7f982b7fe700 1 -- 172.21.15.12:0/2846190627 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c09a290 con 0x55f99cd9ce00 2024-05-08T23:49:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 -- 172.21.15.12:0/2846190627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 msgr2=0x55f99cdacc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 --2- 172.21.15.12:0/2846190627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cdacc20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f982c099d30 tx=0x7f982c0cc040 comp rx=0 tx=0).stop 2024-05-08T23:49:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 -- 172.21.15.12:0/2846190627 shutdown_connections 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 --2- 172.21.15.12:0/2846190627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9d720 0x55f99cd91d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 --2- 172.21.15.12:0/2846190627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 0x55f99cda0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 --2- 172.21.15.12:0/2846190627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cdacc20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 -- 172.21.15.12:0/2846190627 >> 172.21.15.12:0/2846190627 conn(0x55f99ccf5210 msgr2=0x55f99cda0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 -- 172.21.15.12:0/2846190627 shutdown_connections 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 -- 172.21.15.12:0/2846190627 wait complete. 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 -- start start 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.940+0000 7f98531fbb80 1 Processor -- start 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 0x55f99cda8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd99c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f98531fbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9d720 0x55f99cd9a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f98531fbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99cd98b60 con 0x55f99cd9ce00 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f98531fbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99cd98cd0 con 0x55f99cd9d720 2024-05-08T23:49:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f98531fbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99cd98e90 con 0x55f99cd97890 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f98531fbb80 10 client.0 _ll_register_callbacks cb 0x55f99cc8cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f983ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd99c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f983ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd99c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55526/0 (socket says 172.21.15.12:55526) 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f9844ddd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 0x55f99cda8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f983ffff700 1 -- 172.21.15.12:0/1022207958 learned_addr learned my addr 172.21.15.12:0/1022207958 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f98455de700 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9d720 0x55f99cd9a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f983ffff700 1 -- 172.21.15.12:0/1022207958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 msgr2=0x55f99cda8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f983ffff700 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 0x55f99cda8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f983ffff700 1 -- 172.21.15.12:0/1022207958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9d720 msgr2=0x55f99cd9a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f983ffff700 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9d720 0x55f99cd9a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f983ffff700 1 -- 172.21.15.12:0/1022207958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98401a6840 con 0x55f99cd9ce00 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f9844ddd700 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 0x55f99cda8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f983ffff700 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd99c80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f982c009230 tx=0x7f982c098540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac86f1caecbf83d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.941+0000 7f981ffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f981ffff700 1 -- 172.21.15.12:0/1022207958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0a24b0 con 0x55f99cd9ce00 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f981ffff700 1 -- 172.21.15.12:0/1022207958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f982c0b81f0 con 0x55f99cd9ce00 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f98531fbb80 1 -- 172.21.15.12:0/1022207958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9834016070 con 0x55f99cd9ce00 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f981ffff700 1 -- 172.21.15.12:0/1022207958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f982c0b83f0 con 0x55f99cd9ce00 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f98531fbb80 10 client.16167 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f98531fbb80 1 -- 172.21.15.12:0/1022207958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f99cb6ccd0 con 0x55f99cd9ce00 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f981ffff700 1 -- 172.21.15.12:0/1022207958 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9834016070 con 0x55f99cd9ce00 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f981ffff700 1 -- 172.21.15.12:0/1022207958 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f982c099670 con 0x55f99cd9ce00 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f98531fbb80 10 client.16167 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f98531fbb80 10 client.16167 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f98531fbb80 20 client.16167 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f98531fbb80 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f99ce07d80 0x55f99ce0a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f98531fbb80 4 client.16167 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.942+0000 7f98531fbb80 1 -- 172.21.15.12:0/1022207958 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f99ccdfec0 con 0x55f99ce07d80 2024-05-08T23:49:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.943+0000 7f9844ddd700 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f99ce07d80 0x55f99ce0a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.943+0000 7f981ffff700 10 client.16167 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.943+0000 7f9844ddd700 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f99ce07d80 0x55f99ce0a1a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a03fc03371ca594e server_cookie=312baaf00f18ef1c in_seq=0 out_seq=0 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.943+0000 7f981ffff700 1 -- 172.21.15.12:0/1022207958 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f99ccdfec0 con 0x55f99ce07d80 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.943+0000 7f981ffff700 10 client.16167 handle_command_reply: tid=0 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.943+0000 7f98531fbb80 1 client.16167 shutdown 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 1 -- 172.21.15.12:0/1022207958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 msgr2=0x55f99cd99c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd99c80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f982c009230 tx=0x7f982c098540 comp rx=0 tx=0).stop 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 1 -- 172.21.15.12:0/1022207958 shutdown_connections 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f99cd9d720 0x55f99cd9a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f99ce07d80 0x55f99ce0a1a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f99cd97890 0x55f99cda8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 1 --2- 172.21.15.12:0/1022207958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f99cd9ce00 0x55f99cd99c80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f981ffff700 0 client.16167 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:49:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 1 -- 172.21.15.12:0/1022207958 >> 172.21.15.12:0/1022207958 conn(0x55f99ccf5210 msgr2=0x55f99cd9e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 1 -- 172.21.15.12:0/1022207958 shutdown_connections 2024-05-08T23:49:15.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 1 -- 172.21.15.12:0/1022207958 wait complete. 2024-05-08T23:49:15.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f98531fbb80 20 client.16167 trim_cache size 0 max 16384 2024-05-08T23:49:15.945 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:49:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f9847041700 1 -- 172.21.15.12:0/1065762540 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98300a12c0 msgr2=0x7f98300a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f9847041700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98300a12c0 0x7f98300a3780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9834009bf0 tx=0x7f9834009260 comp rx=0 tx=0).stop 2024-05-08T23:49:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f9847041700 1 -- 172.21.15.12:0/1065762540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9840106280 msgr2=0x7f9840199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.944+0000 7f9847041700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9840106280 0x7f9840199cb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f982c00e970 tx=0x7f982c00ed30 comp rx=0 tx=0).stop 2024-05-08T23:49:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.945+0000 7f9847041700 1 -- 172.21.15.12:0/1065762540 shutdown_connections 2024-05-08T23:49:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.945+0000 7f9847041700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98300a12c0 0x7f98300a3780 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.945+0000 7f9847041700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f984019cfa0 0x7f984019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.945+0000 7f9847041700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98400697d0 0x7f9840199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.945+0000 7f9847041700 1 --2- 172.21.15.12:0/1065762540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9840106280 0x7f9840199cb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:15.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.945+0000 7f9847041700 1 -- 172.21.15.12:0/1065762540 >> 172.21.15.12:0/1065762540 conn(0x7f9840077000 msgr2=0x7f984010c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:15.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.945+0000 7f9847041700 1 -- 172.21.15.12:0/1065762540 shutdown_connections 2024-05-08T23:49:15.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:15.945+0000 7f9847041700 1 -- 172.21.15.12:0/1065762540 wait complete. 2024-05-08T23:49:15.954 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:49:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.048+0000 7fea1a3d3700 1 -- 172.21.15.12:0/2756000876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea14107000 msgr2=0x7fea141073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.048+0000 7fea1a3d3700 1 --2- 172.21.15.12:0/2756000876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea14107000 0x7fea141073e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fea04009990 tx=0x7fea04009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.048+0000 7fea1a3d3700 1 -- 172.21.15.12:0/2756000876 shutdown_connections 2024-05-08T23:49:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.048+0000 7fea1a3d3700 1 --2- 172.21.15.12:0/2756000876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea14068ca0 0x7fea1410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.048+0000 7fea1a3d3700 1 --2- 172.21.15.12:0/2756000876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea14107000 0x7fea141073e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.048+0000 7fea1a3d3700 1 --2- 172.21.15.12:0/2756000876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea140682e0 0x7fea14068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.048+0000 7fea1a3d3700 1 -- 172.21.15.12:0/2756000876 >> 172.21.15.12:0/2756000876 conn(0x7fea14076f10 msgr2=0x7fea14077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.048+0000 7fea1a3d3700 1 -- 172.21.15.12:0/2756000876 shutdown_connections 2024-05-08T23:49:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.048+0000 7fea1a3d3700 1 -- 172.21.15.12:0/2756000876 wait complete. 2024-05-08T23:49:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.049+0000 7fea1a3d3700 1 Processor -- start 2024-05-08T23:49:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.049+0000 7fea1a3d3700 1 -- start start 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea1a3d3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea140682e0 0x7fea141972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea1a3d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea14068ca0 0x7fea14197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea1a3d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea14107000 0x7fea1419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea1a3d3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1419c1e0 con 0x7fea14107000 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea1a3d3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1419c350 con 0x7fea14068ca0 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea1a3d3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1419c510 con 0x7fea140682e0 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea18970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea14107000 0x7fea1419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea137fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea14068ca0 0x7fea14197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea18970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea14107000 0x7fea1419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55544/0 (socket says 172.21.15.12:55544) 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea137fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea14068ca0 0x7fea14197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41896/0 (socket says 172.21.15.12:41896) 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea18970700 1 -- 172.21.15.12:0/3849446745 learned_addr learned my addr 172.21.15.12:0/3849446745 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.050+0000 7fea13fff700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea140682e0 0x7fea141972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea18970700 1 -- 172.21.15.12:0/3849446745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea140682e0 msgr2=0x7fea141972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea18970700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea140682e0 0x7fea141972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea18970700 1 -- 172.21.15.12:0/3849446745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea14068ca0 msgr2=0x7fea14197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea18970700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea14068ca0 0x7fea14197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea18970700 1 -- 172.21.15.12:0/3849446745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040095f0 con 0x7fea14107000 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea18970700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea14107000 0x7fea1419bba0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00ef30 tx=0x7fea0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea117fa700 1 -- 172.21.15.12:0/3849446745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c018070 con 0x7fea14107000 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea117fa700 1 -- 172.21.15.12:0/3849446745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0c00f040 con 0x7fea14107000 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea1a3d3700 1 -- 172.21.15.12:0/3849446745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1419c790 con 0x7fea14107000 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea117fa700 1 -- 172.21.15.12:0/3849446745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c00cd00 con 0x7fea14107000 2024-05-08T23:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.051+0000 7fea1a3d3700 1 -- 172.21.15.12:0/3849446745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea141a4030 con 0x7fea14107000 2024-05-08T23:49:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.054+0000 7fea117fa700 1 -- 172.21.15.12:0/3849446745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fea0c003a00 con 0x7fea14107000 2024-05-08T23:49:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.055+0000 7fea117fa700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9fc0a1290 0x7fe9fc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.055+0000 7fea117fa700 1 -- 172.21.15.12:0/3849446745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fea0c0cc800 con 0x7fea14107000 2024-05-08T23:49:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.055+0000 7fea13fff700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9fc0a1290 0x7fe9fc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.055+0000 7fea13fff700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9fc0a1290 0x7fe9fc0a3750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea0400b2e0 tx=0x7fea04005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea2658db80 1 Processor -- start 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea2658db80 1 -- start start 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea2658db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657cee3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea2658db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cee7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea2658db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 0x55657cee82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea2658db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657ce360f0 con 0x55657cee3e60 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea2658db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657ce33ab0 con 0x55657cee7e40 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea2658db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657ce2f3a0 con 0x55657cee3540 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cee7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cee7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55556/0 (socket says 172.21.15.12:55556) 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea137fe700 1 -- 172.21.15.12:0/1361431442 learned_addr learned my addr 172.21.15.12:0/1361431442 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea13fff700 1 --2- 172.21.15.12:0/1361431442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657cee3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea18970700 1 --2- 172.21.15.12:0/1361431442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 0x55657cee82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea137fe700 1 -- 172.21.15.12:0/1361431442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 msgr2=0x55657cee3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.056+0000 7fea137fe700 1 --2- 172.21.15.12:0/1361431442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657cee3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.057+0000 7fea137fe700 1 -- 172.21.15.12:0/1361431442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 msgr2=0x55657cee82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.057+0000 7fea137fe700 1 --2- 172.21.15.12:0/1361431442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 0x55657cee82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.057+0000 7fea137fe700 1 -- 172.21.15.12:0/1361431442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55657ccb0350 con 0x55657cee3e60 2024-05-08T23:49:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.057+0000 7fea18970700 1 --2- 172.21.15.12:0/1361431442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 0x55657cee82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.057+0000 7fea137fe700 1 --2- 172.21.15.12:0/1361431442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cee7900 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fea08007d10 tx=0x7fea080106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6f04b7e582edad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.057+0000 7fe9fa7fc700 1 -- 172.21.15.12:0/1361431442 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08010e10 con 0x55657cee3e60 2024-05-08T23:49:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.057+0000 7fe9fa7fc700 1 -- 172.21.15.12:0/1361431442 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea08015070 con 0x55657cee3e60 2024-05-08T23:49:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.057+0000 7fe9fa7fc700 1 -- 172.21.15.12:0/1361431442 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08019630 con 0x55657cee3e60 2024-05-08T23:49:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.058+0000 7fea2658db80 1 -- 172.21.15.12:0/1361431442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 msgr2=0x55657cee7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.058+0000 7fea2658db80 1 --2- 172.21.15.12:0/1361431442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cee7900 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fea08007d10 tx=0x7fea080106b0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.058+0000 7fea2658db80 1 -- 172.21.15.12:0/1361431442 shutdown_connections 2024-05-08T23:49:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.058+0000 7fea2658db80 1 --2- 172.21.15.12:0/1361431442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 0x55657cee82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.058+0000 7fea2658db80 1 --2- 172.21.15.12:0/1361431442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657cee3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.058+0000 7fea2658db80 1 --2- 172.21.15.12:0/1361431442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cee7900 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.058+0000 7fea2658db80 1 -- 172.21.15.12:0/1361431442 >> 172.21.15.12:0/1361431442 conn(0x55657ce3d1b0 msgr2=0x55657ced9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.058+0000 7fea2658db80 1 -- 172.21.15.12:0/1361431442 shutdown_connections 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.058+0000 7fea2658db80 1 -- 172.21.15.12:0/1361431442 wait complete. 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.058+0000 7fea2658db80 1 -- start start 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea2658db80 1 Processor -- start 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea2658db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657ceef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea2658db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3e60 0x55657ceec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea2658db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee7e40 0x55657ceed060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea2658db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657cdcb500 con 0x55657cee7e40 2024-05-08T23:49:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea2658db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657cdcb1a0 con 0x55657cee3e60 2024-05-08T23:49:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea2658db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657ccb2640 con 0x55657cee3540 2024-05-08T23:49:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea2658db80 10 client.0 _ll_register_callbacks cb 0x55657cdd4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea18970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee7e40 0x55657ceed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea18970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee7e40 0x55657ceed060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55558/0 (socket says 172.21.15.12:55558) 2024-05-08T23:49:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea18970700 1 -- 172.21.15.12:0/3914444941 learned_addr learned my addr 172.21.15.12:0/3914444941 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea13fff700 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657ceef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.059+0000 7fea137fe700 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3e60 0x55657ceec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fea18970700 1 -- 172.21.15.12:0/3914444941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 msgr2=0x55657ceef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fea18970700 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657ceef650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fea18970700 1 -- 172.21.15.12:0/3914444941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3e60 msgr2=0x55657ceec970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fea18970700 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3e60 0x55657ceec970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fea18970700 1 -- 172.21.15.12:0/3914444941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea08007940 con 0x55657cee7e40 2024-05-08T23:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fea18970700 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee7e40 0x55657ceed060 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0963f0 tx=0x7fea0c004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef509bf16d5a784 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fe9f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3914444941 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c099980 con 0x55657cee7e40 2024-05-08T23:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3914444941 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fea0c0b2270 con 0x55657cee7e40 2024-05-08T23:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fea2658db80 1 -- 172.21.15.12:0/3914444941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55657cd88210 con 0x55657cee7e40 2024-05-08T23:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3914444941 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea0c09d7b0 con 0x55657cee7e40 2024-05-08T23:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fea2658db80 10 client.16179 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fea2658db80 1 -- 172.21.15.12:0/3914444941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55657cc5cb00 con 0x55657cee7e40 2024-05-08T23:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.060+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3914444941 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55657cd88210 con 0x55657cee7e40 2024-05-08T23:49:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.061+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3914444941 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea0c0a18e0 con 0x55657cee7e40 2024-05-08T23:49:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.061+0000 7fea2658db80 10 client.16179 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.061+0000 7fea2658db80 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.061+0000 7fea2658db80 20 client.16179 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.061+0000 7fea2658db80 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55657cefcfe0 0x55657ceff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.061+0000 7fea2658db80 4 client.16179 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.061+0000 7fea2658db80 1 -- 172.21.15.12:0/3914444941 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55657ceae7b0 con 0x55657cefcfe0 2024-05-08T23:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.061+0000 7fea13fff700 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55657cefcfe0 0x55657ceff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.061+0000 7fea13fff700 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55657cefcfe0 0x55657ceff400 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9c367d410b6cbb server_cookie=b5f2f26eea4ba8e9 in_seq=0 out_seq=0 2024-05-08T23:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.061+0000 7fe9f37fe700 10 client.16179 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3914444941 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55657ceae7b0 con 0x55657cefcfe0 2024-05-08T23:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fe9f37fe700 10 client.16179 handle_command_reply: tid=0 2024-05-08T23:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fea2658db80 1 client.16179 shutdown 2024-05-08T23:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fea2658db80 1 -- 172.21.15.12:0/3914444941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee7e40 msgr2=0x55657ceed060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fea2658db80 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee7e40 0x55657ceed060 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0963f0 tx=0x7fea0c004c00 comp rx=0 tx=0).stop 2024-05-08T23:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fea2658db80 1 -- 172.21.15.12:0/3914444941 shutdown_connections 2024-05-08T23:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fea2658db80 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee7e40 0x55657ceed060 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fea2658db80 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657ceef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fea2658db80 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55657cefcfe0 0x55657ceff400 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fea2658db80 1 --2- 172.21.15.12:0/3914444941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3e60 0x55657ceec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fea2658db80 1 -- 172.21.15.12:0/3914444941 >> 172.21.15.12:0/3914444941 conn(0x55657ce3d1b0 msgr2=0x55657ceddf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.062+0000 7fe9f37fe700 0 client.16179 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.063+0000 7fea2658db80 1 -- 172.21.15.12:0/3914444941 shutdown_connections 2024-05-08T23:49:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.063+0000 7fea2658db80 1 -- 172.21.15.12:0/3914444941 wait complete. 2024-05-08T23:49:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.063+0000 7fea2658db80 20 client.16179 trim_cache size 0 max 16384 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea2658db80 1 Processor -- start 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea2658db80 1 -- start start 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea2658db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657cede880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea2658db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657ceef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea2658db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 0x55657cef4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea2658db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657cefa680 con 0x55657cee3e60 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea2658db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657cee6bb0 con 0x55657cee7e40 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea2658db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657ceff3d0 con 0x55657cee3540 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657ceef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657ceef180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55564/0 (socket says 172.21.15.12:55564) 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea137fe700 1 -- 172.21.15.12:0/1271754572 learned_addr learned my addr 172.21.15.12:0/1271754572 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea18970700 1 --2- 172.21.15.12:0/1271754572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 0x55657cef4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea13fff700 1 --2- 172.21.15.12:0/1271754572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657cede880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea137fe700 1 -- 172.21.15.12:0/1271754572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 msgr2=0x55657cede880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea137fe700 1 --2- 172.21.15.12:0/1271754572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657cede880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea137fe700 1 -- 172.21.15.12:0/1271754572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 msgr2=0x55657cef4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea137fe700 1 --2- 172.21.15.12:0/1271754572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 0x55657cef4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea137fe700 1 -- 172.21.15.12:0/1271754572 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55657cdf5500 con 0x55657cee3e60 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.079+0000 7fea18970700 1 --2- 172.21.15.12:0/1271754572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 0x55657cef4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.080+0000 7fea137fe700 1 --2- 172.21.15.12:0/1271754572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657ceef180 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fea08007d10 tx=0x7fea08007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62037aa81cc71958 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.080+0000 7fe9faffd700 1 -- 172.21.15.12:0/1271754572 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08010c10 con 0x55657cee3e60 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.080+0000 7fe9faffd700 1 -- 172.21.15.12:0/1271754572 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea08010d70 con 0x55657cee3e60 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.080+0000 7fe9faffd700 1 -- 172.21.15.12:0/1271754572 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08004c40 con 0x55657cee3e60 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.081+0000 7fea2658db80 1 -- 172.21.15.12:0/1271754572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 msgr2=0x55657ceef180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.081+0000 7fea2658db80 1 --2- 172.21.15.12:0/1271754572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657ceef180 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fea08007d10 tx=0x7fea08007f90 comp rx=0 tx=0).stop 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.081+0000 7fea2658db80 1 -- 172.21.15.12:0/1271754572 shutdown_connections 2024-05-08T23:49:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.081+0000 7fea2658db80 1 --2- 172.21.15.12:0/1271754572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee7e40 0x55657cef4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.081+0000 7fea2658db80 1 --2- 172.21.15.12:0/1271754572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee3540 0x55657cede880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.081+0000 7fea2658db80 1 --2- 172.21.15.12:0/1271754572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657ceef180 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.081+0000 7fea2658db80 1 -- 172.21.15.12:0/1271754572 >> 172.21.15.12:0/1271754572 conn(0x55657ce3d1b0 msgr2=0x55657cf03d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.081+0000 7fea2658db80 1 -- 172.21.15.12:0/1271754572 shutdown_connections 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.081+0000 7fea2658db80 1 -- 172.21.15.12:0/1271754572 wait complete. 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.081+0000 7fea2658db80 1 -- start start 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea2658db80 1 Processor -- start 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea2658db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3540 0x55657cef21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea2658db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cef2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea2658db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee7e40 0x55657cf524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea2658db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657cef2de0 con 0x55657cee3e60 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea2658db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657cf52a20 con 0x55657cee3540 2024-05-08T23:49:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea2658db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657cf52b60 con 0x55657cee7e40 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea2658db80 10 client.0 _ll_register_callbacks cb 0x55657cdd4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cef2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cef2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55570/0 (socket says 172.21.15.12:55570) 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea13fff700 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3540 0x55657cef21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea137fe700 1 -- 172.21.15.12:0/3289932775 learned_addr learned my addr 172.21.15.12:0/3289932775 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea18970700 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee7e40 0x55657cf524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea13fff700 1 -- 172.21.15.12:0/3289932775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee7e40 msgr2=0x55657cf524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea13fff700 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee7e40 0x55657cf524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea13fff700 1 -- 172.21.15.12:0/3289932775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 msgr2=0x55657cef2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea13fff700 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cef2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea13fff700 1 -- 172.21.15.12:0/3289932775 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55657cc5cb00 con 0x55657cee3540 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.082+0000 7fea137fe700 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cef2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.083+0000 7fea13fff700 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3540 0x55657cef21e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fea040059d0 tx=0x7fea04009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad55ac63f1fdbfe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.083+0000 7fe9f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.083+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3289932775 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea040048e0 con 0x55657cee3540 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.083+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3289932775 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fea04004a40 con 0x55657cee3540 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.083+0000 7fea2658db80 1 -- 172.21.15.12:0/3289932775 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea08015070 con 0x55657cee3540 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.083+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3289932775 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea04004c00 con 0x55657cee3540 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.083+0000 7fea2658db80 10 client.26012 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.083+0000 7fea2658db80 1 -- 172.21.15.12:0/3289932775 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fea0801e040 con 0x55657cee3540 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.083+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3289932775 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea08015070 con 0x55657cee3540 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.083+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3289932775 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea0400f460 con 0x55657cee3540 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.084+0000 7fea2658db80 10 client.26012 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.084+0000 7fea2658db80 10 client.26012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.084+0000 7fea2658db80 20 client.26012 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.084+0000 7fea2658db80 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55657cf542c0 0x55657cf566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.084+0000 7fea2658db80 4 client.26012 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:49:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.084+0000 7fea2658db80 1 -- 172.21.15.12:0/3289932775 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55657ce27e60 con 0x55657cf542c0 2024-05-08T23:49:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.084+0000 7fea137fe700 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55657cf542c0 0x55657cf566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.084+0000 7fe9f37fe700 10 client.26012 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.084+0000 7fea137fe700 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55657cf542c0 0x55657cf566e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d1ebf4fc6d3ad6 server_cookie=2f85145a55e2e5b4 in_seq=0 out_seq=0 2024-05-08T23:49:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fe9f37fe700 1 -- 172.21.15.12:0/3289932775 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55657ce27e60 con 0x55657cf542c0 2024-05-08T23:49:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fe9f37fe700 10 client.26012 handle_command_reply: tid=0 2024-05-08T23:49:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 client.26012 shutdown 2024-05-08T23:49:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 -- 172.21.15.12:0/3289932775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3540 msgr2=0x55657cef21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3540 0x55657cef21e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fea040059d0 tx=0x7fea04009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 -- 172.21.15.12:0/3289932775 shutdown_connections 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55657cee7e40 0x55657cf524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55657cf542c0 0x55657cf566e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55657cee3540 0x55657cef21e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 --2- 172.21.15.12:0/3289932775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55657cee3e60 0x55657cef2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fe9f37fe700 0 client.26012 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 -- 172.21.15.12:0/3289932775 >> 172.21.15.12:0/3289932775 conn(0x55657ce3d1b0 msgr2=0x55657cef8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 -- 172.21.15.12:0/3289932775 shutdown_connections 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 1 -- 172.21.15.12:0/3289932775 wait complete. 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.105+0000 7fea2658db80 20 client.26012 trim_cache size 0 max 16384 2024-05-08T23:49:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:49:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:49:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:49:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:49:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 -- 172.21.15.12:0/3849446745 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9fc0a1290 msgr2=0x7fe9fc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9fc0a1290 0x7fe9fc0a3750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea0400b2e0 tx=0x7fea04005960 comp rx=0 tx=0).stop 2024-05-08T23:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 -- 172.21.15.12:0/3849446745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea14107000 msgr2=0x7fea1419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea14107000 0x7fea1419bba0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00ef30 tx=0x7fea0c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 -- 172.21.15.12:0/3849446745 shutdown_connections 2024-05-08T23:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9fc0a1290 0x7fe9fc0a3750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea14107000 0x7fea1419bba0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea140682e0 0x7fea141972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 --2- 172.21.15.12:0/3849446745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea14068ca0 0x7fea14197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 -- 172.21.15.12:0/3849446745 >> 172.21.15.12:0/3849446745 conn(0x7fea14076f10 msgr2=0x7fea140fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 -- 172.21.15.12:0/3849446745 shutdown_connections 2024-05-08T23:49:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:16.106+0000 7fea1a3d3700 1 -- 172.21.15.12:0/3849446745 wait complete. 2024-05-08T23:49:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.112+0000 7f280223b700 1 -- 172.21.15.12:0/396627760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27fc0694e0 msgr2=0x7f27fc0698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.112+0000 7f280223b700 1 --2- 172.21.15.12:0/396627760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27fc0694e0 0x7f27fc0698c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0099c0 tx=0x7f27ec009cd0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.112+0000 7f280223b700 1 -- 172.21.15.12:0/396627760 shutdown_connections 2024-05-08T23:49:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.112+0000 7f280223b700 1 --2- 172.21.15.12:0/396627760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc106290 0x7f27fc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.112+0000 7f280223b700 1 --2- 172.21.15.12:0/396627760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27fc0694e0 0x7f27fc0698c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.112+0000 7f280223b700 1 --2- 172.21.15.12:0/396627760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27fc069e90 0x7f27fc105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.112+0000 7f280223b700 1 -- 172.21.15.12:0/396627760 >> 172.21.15.12:0/396627760 conn(0x7f27fc076f30 msgr2=0x7f27fc077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.112+0000 7f280223b700 1 -- 172.21.15.12:0/396627760 shutdown_connections 2024-05-08T23:49:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.112+0000 7f280223b700 1 -- 172.21.15.12:0/396627760 wait complete. 2024-05-08T23:49:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.113+0000 7f280223b700 1 Processor -- start 2024-05-08T23:49:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.113+0000 7f280223b700 1 -- start start 2024-05-08T23:49:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.113+0000 7f280223b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27fc0694e0 0x7f27fc1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f280223b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc106290 0x7f27fc197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f280223b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27fc19bcc0 0x7f27fc19c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f280223b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc19c740 con 0x7f27fc19bcc0 2024-05-08T23:49:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f280223b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc19c8b0 con 0x7f27fc0694e0 2024-05-08T23:49:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f280223b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc19ca70 con 0x7f27fc106290 2024-05-08T23:49:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27fc19bcc0 0x7f27fc19c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27fc19bcc0 0x7f27fc19c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55594/0 (socket says 172.21.15.12:55594) 2024-05-08T23:49:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27fc0694e0 0x7f27fc1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27faffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc106290 0x7f27fc197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fbfff700 1 -- 172.21.15.12:0/3882263468 learned_addr learned my addr 172.21.15.12:0/3882263468 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fb7fe700 1 -- 172.21.15.12:0/3882263468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc106290 msgr2=0x7f27fc197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc106290 0x7f27fc197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fb7fe700 1 -- 172.21.15.12:0/3882263468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27fc19bcc0 msgr2=0x7f27fc19c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27fc19bcc0 0x7f27fc19c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fb7fe700 1 -- 172.21.15.12:0/3882263468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec0095f0 con 0x7f27fc0694e0 2024-05-08T23:49:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fbfff700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27fc19bcc0 0x7f27fc19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27faffd700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc106290 0x7f27fc197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.114+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27fc0694e0 0x7f27fc1973e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0098e0 tx=0x7f27ec00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.115+0000 7f27f8ff9700 1 -- 172.21.15.12:0/3882263468 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec01c070 con 0x7f27fc0694e0 2024-05-08T23:49:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.115+0000 7f27f8ff9700 1 -- 172.21.15.12:0/3882263468 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27ec00fd80 con 0x7f27fc0694e0 2024-05-08T23:49:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.115+0000 7f280223b700 1 -- 172.21.15.12:0/3882263468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27fc1a3f70 con 0x7f27fc0694e0 2024-05-08T23:49:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.115+0000 7f27f8ff9700 1 -- 172.21.15.12:0/3882263468 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec022c10 con 0x7f27fc0694e0 2024-05-08T23:49:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.115+0000 7f280223b700 1 -- 172.21.15.12:0/3882263468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc1a4330 con 0x7f27fc0694e0 2024-05-08T23:49:16.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.117+0000 7f27f8ff9700 1 -- 172.21.15.12:0/3882263468 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f27ec020070 con 0x7f27fc0694e0 2024-05-08T23:49:16.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.118+0000 7f27f8ff9700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27e40a1290 0x7f27e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.118+0000 7f27faffd700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27e40a1290 0x7f27e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.118+0000 7f27f8ff9700 1 -- 172.21.15.12:0/3882263468 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f27ec0cda60 con 0x7f27fc0694e0 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f27faffd700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27e40a1290 0x7f27e40a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f27fc198930 tx=0x7f27f0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f280e3f5b80 1 Processor -- start 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f280e3f5b80 1 -- start start 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83369e00 0x562d8336a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d8336a720 0x562d83364350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 0x562d83364cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d832bb150 con 0x562d8336a720 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d832b8b10 con 0x562d83364890 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d832b4400 con 0x562d83369e00 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f27faffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d8336a720 0x562d83364350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f27faffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d8336a720 0x562d83364350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55610/0 (socket says 172.21.15.12:55610) 2024-05-08T23:49:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.119+0000 7f27faffd700 1 -- 172.21.15.12:0/548425744 learned_addr learned my addr 172.21.15.12:0/548425744 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/548425744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83369e00 0x562d8336a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27fbfff700 1 --2- 172.21.15.12:0/548425744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 0x562d83364cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27faffd700 1 -- 172.21.15.12:0/548425744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83369e00 msgr2=0x562d8336a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27faffd700 1 --2- 172.21.15.12:0/548425744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83369e00 0x562d8336a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27faffd700 1 -- 172.21.15.12:0/548425744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 msgr2=0x562d83364cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27faffd700 1 --2- 172.21.15.12:0/548425744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 0x562d83364cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27faffd700 1 -- 172.21.15.12:0/548425744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d831353b0 con 0x562d8336a720 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27fbfff700 1 --2- 172.21.15.12:0/548425744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 0x562d83364cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27faffd700 1 --2- 172.21.15.12:0/548425744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d8336a720 0x562d83364350 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f27f00043c0 tx=0x7f27f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86929cf26cb3cb2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27e1ffb700 1 -- 172.21.15.12:0/548425744 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0003dc0 con 0x562d8336a720 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27e1ffb700 1 -- 172.21.15.12:0/548425744 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f001e8b0 con 0x562d8336a720 2024-05-08T23:49:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.120+0000 7f27e1ffb700 1 -- 172.21.15.12:0/548425744 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f000e8e0 con 0x562d8336a720 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.121+0000 7f280e3f5b80 1 -- 172.21.15.12:0/548425744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d8336a720 msgr2=0x562d83364350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.121+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/548425744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d8336a720 0x562d83364350 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f27f00043c0 tx=0x7f27f0014000 comp rx=0 tx=0).stop 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.121+0000 7f280e3f5b80 1 -- 172.21.15.12:0/548425744 shutdown_connections 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.121+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/548425744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 0x562d83364cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.121+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/548425744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83369e00 0x562d8336a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.121+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/548425744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d8336a720 0x562d83364350 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.121+0000 7f280e3f5b80 1 -- 172.21.15.12:0/548425744 >> 172.21.15.12:0/548425744 conn(0x562d832c2210 msgr2=0x562d8335edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.121+0000 7f280e3f5b80 1 -- 172.21.15.12:0/548425744 shutdown_connections 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.121+0000 7f280e3f5b80 1 -- 172.21.15.12:0/548425744 wait complete. 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.121+0000 7f280e3f5b80 1 -- start start 2024-05-08T23:49:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f280e3f5b80 1 Processor -- start 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83364890 0x562d8336dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8336e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d8336a720 0x562d8336e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d83250560 con 0x562d83369e00 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d83250200 con 0x562d8336a720 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d831376a0 con 0x562d83364890 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f280e3f5b80 10 client.0 _ll_register_callbacks cb 0x562d83259b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f27faffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8336e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f27faffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8336e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55622/0 (socket says 172.21.15.12:55622) 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f27faffd700 1 -- 172.21.15.12:0/2700074780 learned_addr learned my addr 172.21.15.12:0/2700074780 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f27fbfff700 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d8336a720 0x562d8336e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.122+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83364890 0x562d8336dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f27faffd700 1 -- 172.21.15.12:0/2700074780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83364890 msgr2=0x562d8336dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f27faffd700 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83364890 0x562d8336dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f27faffd700 1 -- 172.21.15.12:0/2700074780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d8336a720 msgr2=0x562d8336e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f27faffd700 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d8336a720 0x562d8336e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f27faffd700 1 -- 172.21.15.12:0/2700074780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f0017070 con 0x562d83369e00 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f27faffd700 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8336e010 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f27f0005920 tx=0x7f27f000e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6f63663611a808 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f27daffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f27daffd700 1 -- 172.21.15.12:0/2700074780 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f002c070 con 0x562d83369e00 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f27daffd700 1 -- 172.21.15.12:0/2700074780 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f27f001ec90 con 0x562d83369e00 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2700074780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d8320d270 con 0x562d83369e00 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f27daffd700 1 -- 172.21.15.12:0/2700074780 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27f001e600 con 0x562d83369e00 2024-05-08T23:49:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f280e3f5b80 10 client.16197 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.123+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2700074780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562d830e1b60 con 0x562d83369e00 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f27daffd700 1 -- 172.21.15.12:0/2700074780 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d8320d270 con 0x562d83369e00 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f27daffd700 1 -- 172.21.15.12:0/2700074780 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27f0024020 con 0x562d83369e00 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f280e3f5b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f280e3f5b80 10 client.16197 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f280e3f5b80 20 client.16197 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562d8337e040 0x562d83380460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f280e3f5b80 4 client.16197 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2700074780 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d83333810 con 0x562d8337e040 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562d8337e040 0x562d83380460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f27daffd700 10 client.16197 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.124+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562d8337e040 0x562d83380460 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b7714415fc9bd42 server_cookie=7b6a0b72f086376 in_seq=0 out_seq=0 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f27daffd700 1 -- 172.21.15.12:0/2700074780 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d83333810 con 0x562d8337e040 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f27daffd700 10 client.16197 handle_command_reply: tid=0 2024-05-08T23:49:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 client.16197 shutdown 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2700074780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 msgr2=0x562d8336e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8336e010 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f27f0005920 tx=0x7f27f000e3b0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2700074780 shutdown_connections 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562d8337e040 0x562d83380460 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d8336a720 0x562d8336e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83364890 0x562d8336dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2700074780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8336e010 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f27daffd700 0 client.16197 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2700074780 >> 172.21.15.12:0/2700074780 conn(0x562d832c2210 msgr2=0x562d8335e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2700074780 shutdown_connections 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.125+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2700074780 wait complete. 2024-05-08T23:49:16.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.126+0000 7f280e3f5b80 20 client.16197 trim_cache size 0 max 16384 2024-05-08T23:49:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.139+0000 7f280e3f5b80 1 Processor -- start 2024-05-08T23:49:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.139+0000 7f280e3f5b80 1 -- start start 2024-05-08T23:49:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 0x562d83379c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8335b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d8336a720 0x562d8335ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d83370e50 con 0x562d83369e00 2024-05-08T23:49:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d83370660 con 0x562d83364890 2024-05-08T23:49:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d83385670 con 0x562d8336a720 2024-05-08T23:49:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27faffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8335b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d8336a720 0x562d8335ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d8336a720 0x562d8335ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43498/0 (socket says 172.21.15.12:43498) 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27faffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8335b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55628/0 (socket says 172.21.15.12:55628) 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fbfff700 1 -- 172.21.15.12:0/54373411 learned_addr learned my addr 172.21.15.12:0/54373411 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/54373411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 0x562d83379c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fbfff700 1 -- 172.21.15.12:0/54373411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 msgr2=0x562d83379c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fbfff700 1 --2- 172.21.15.12:0/54373411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 0x562d83379c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fbfff700 1 -- 172.21.15.12:0/54373411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 msgr2=0x562d8335b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fbfff700 1 --2- 172.21.15.12:0/54373411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8335b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fbfff700 1 -- 172.21.15.12:0/54373411 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d8327a560 con 0x562d8336a720 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/54373411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 0x562d83379c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27faffd700 1 --2- 172.21.15.12:0/54373411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8335b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.140+0000 7f27fbfff700 1 --2- 172.21.15.12:0/54373411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d8336a720 0x562d8335ed20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f27f40077d0 tx=0x7f27f400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d314ed55b98305de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f27e27fc700 1 -- 172.21.15.12:0/54373411 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4004db0 con 0x562d8336a720 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f27e27fc700 1 -- 172.21.15.12:0/54373411 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f400f040 con 0x562d8336a720 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f27e27fc700 1 -- 172.21.15.12:0/54373411 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4017ba0 con 0x562d8336a720 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f280e3f5b80 1 -- 172.21.15.12:0/54373411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d8336a720 msgr2=0x562d8335ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/54373411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d8336a720 0x562d8335ed20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f27f40077d0 tx=0x7f27f400b040 comp rx=0 tx=0).stop 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f280e3f5b80 1 -- 172.21.15.12:0/54373411 shutdown_connections 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/54373411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d8336a720 0x562d8335ed20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/54373411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d83364890 0x562d83379c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/54373411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83369e00 0x562d8335b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f280e3f5b80 1 -- 172.21.15.12:0/54373411 >> 172.21.15.12:0/54373411 conn(0x562d832c2210 msgr2=0x562d83364c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f280e3f5b80 1 -- 172.21.15.12:0/54373411 shutdown_connections 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.141+0000 7f280e3f5b80 1 -- 172.21.15.12:0/54373411 wait complete. 2024-05-08T23:49:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f280e3f5b80 1 -- start start 2024-05-08T23:49:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f280e3f5b80 1 Processor -- start 2024-05-08T23:49:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83364890 0x562d833c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83369e00 0x562d833c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f280e3f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d8336a720 0x562d833c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d833d3700 con 0x562d83364890 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d833d3870 con 0x562d8336a720 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f280e3f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d833d3a30 con 0x562d83369e00 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f280e3f5b80 10 client.0 _ll_register_callbacks cb 0x562d83259b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f27fb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83364890 0x562d833c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f27fb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83364890 0x562d833c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55638/0 (socket says 172.21.15.12:55638) 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f27fb7fe700 1 -- 172.21.15.12:0/2633431972 learned_addr learned my addr 172.21.15.12:0/2633431972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.142+0000 7f27faffd700 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83369e00 0x562d833c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27fbfff700 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d8336a720 0x562d833c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27fb7fe700 1 -- 172.21.15.12:0/2633431972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83369e00 msgr2=0x562d833c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83369e00 0x562d833c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27fb7fe700 1 -- 172.21.15.12:0/2633431972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d8336a720 msgr2=0x562d833c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d8336a720 0x562d833c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27fb7fe700 1 -- 172.21.15.12:0/2633431972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d830e1b60 con 0x562d83364890 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27fb7fe700 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83364890 0x562d833c84d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f27ec097650 tx=0x7f27ec09a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d78c848936c1ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27daffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27daffd700 1 -- 172.21.15.12:0/2633431972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec09abe0 con 0x562d83364890 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27daffd700 1 -- 172.21.15.12:0/2633431972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f27ec09ea10 con 0x562d83364890 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2633431972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27f001e8b0 con 0x562d83364890 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f27daffd700 1 -- 172.21.15.12:0/2633431972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27ec0a2fd0 con 0x562d83364890 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f280e3f5b80 10 client.16203 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.143+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2633431972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f27f4007500 con 0x562d83364890 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f27daffd700 1 -- 172.21.15.12:0/2633431972 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f001e8b0 con 0x562d83364890 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f27daffd700 1 -- 172.21.15.12:0/2633431972 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27ec09a0d0 con 0x562d83364890 2024-05-08T23:49:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f280e3f5b80 10 client.16203 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f280e3f5b80 10 client.16203 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f280e3f5b80 20 client.16203 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562d833d4e60 0x562d833d7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f280e3f5b80 4 client.16203 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2633431972 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562d832acec0 con 0x562d833d4e60 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f27faffd700 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562d833d4e60 0x562d833d7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f27daffd700 10 client.16203 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.144+0000 7f27faffd700 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562d833d4e60 0x562d833d7250 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4777b866ec2e265 server_cookie=3cdc4e07539d2665 in_seq=0 out_seq=0 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f27daffd700 1 -- 172.21.15.12:0/2633431972 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562d832acec0 con 0x562d833d4e60 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f27daffd700 10 client.16203 handle_command_reply: tid=0 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 client.16203 shutdown 2024-05-08T23:49:16.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2633431972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83364890 msgr2=0x562d833c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83364890 0x562d833c84d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f27ec097650 tx=0x7f27ec09a780 comp rx=0 tx=0).stop 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2633431972 shutdown_connections 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562d833d4e60 0x562d833d7250 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d8336a720 0x562d833c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d83364890 0x562d833c84d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f27daffd700 0 client.16203 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 --2- 172.21.15.12:0/2633431972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d83369e00 0x562d833c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2633431972 >> 172.21.15.12:0/2633431972 conn(0x562d832c2210 msgr2=0x562d83367c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2633431972 shutdown_connections 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 1 -- 172.21.15.12:0/2633431972 wait complete. 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280e3f5b80 20 client.16203 trim_cache size 0 max 16384 2024-05-08T23:49:16.147 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:49:16.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280223b700 1 -- 172.21.15.12:0/3882263468 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27e40a1290 msgr2=0x7f27e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.145+0000 7f280223b700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27e40a1290 0x7f27e40a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f27fc198930 tx=0x7f27f0009260 comp rx=0 tx=0).stop 2024-05-08T23:49:16.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.146+0000 7f280223b700 1 -- 172.21.15.12:0/3882263468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27fc0694e0 msgr2=0x7f27fc1973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.146+0000 7f280223b700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27fc0694e0 0x7f27fc1973e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0098e0 tx=0x7f27ec00f690 comp rx=0 tx=0).stop 2024-05-08T23:49:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.146+0000 7f280223b700 1 -- 172.21.15.12:0/3882263468 shutdown_connections 2024-05-08T23:49:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.146+0000 7f280223b700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f27e40a1290 0x7f27e40a3750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.146+0000 7f280223b700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27fc19bcc0 0x7f27fc19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.146+0000 7f280223b700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27fc0694e0 0x7f27fc1973e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.146+0000 7f280223b700 1 --2- 172.21.15.12:0/3882263468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc106290 0x7f27fc197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.146+0000 7f280223b700 1 -- 172.21.15.12:0/3882263468 >> 172.21.15.12:0/3882263468 conn(0x7f27fc076f30 msgr2=0x7f27fc0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.146+0000 7f280223b700 1 -- 172.21.15.12:0/3882263468 shutdown_connections 2024-05-08T23:49:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.146+0000 7f280223b700 1 -- 172.21.15.12:0/3882263468 wait complete. 2024-05-08T23:49:16.155 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:49:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.299+0000 7f30df956700 1 -- 172.21.15.12:0/1789399374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30d806a190 msgr2=0x7f30d8105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.299+0000 7f30df956700 1 --2- 172.21.15.12:0/1789399374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30d806a190 0x7f30d8105b70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0099c0 tx=0x7f30cc009cd0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.299+0000 7f30df956700 1 -- 172.21.15.12:0/1789399374 shutdown_connections 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.299+0000 7f30df956700 1 --2- 172.21.15.12:0/1789399374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30d81062a0 0x7f30d810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.299+0000 7f30df956700 1 --2- 172.21.15.12:0/1789399374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d80697e0 0x7f30d8069bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.299+0000 7f30df956700 1 --2- 172.21.15.12:0/1789399374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30d806a190 0x7f30d8105b70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.299+0000 7f30df956700 1 -- 172.21.15.12:0/1789399374 >> 172.21.15.12:0/1789399374 conn(0x7f30d8076f10 msgr2=0x7f30d8077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.299+0000 7f30df956700 1 -- 172.21.15.12:0/1789399374 shutdown_connections 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.299+0000 7f30df956700 1 -- 172.21.15.12:0/1789399374 wait complete. 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30df956700 1 Processor -- start 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30df956700 1 -- start start 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30df956700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30d80697e0 0x7f30d81973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30df956700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30d81062a0 0x7f30d8197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30df956700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d819bca0 0x7f30d819c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30df956700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d819c720 con 0x7f30d80697e0 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30df956700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d819c890 con 0x7f30d819bca0 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30df956700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d819ca50 con 0x7f30d81062a0 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30dd6f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30d80697e0 0x7f30d81973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30dd6f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30d80697e0 0x7f30d81973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55656/0 (socket says 172.21.15.12:55656) 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30dd6f2700 1 -- 172.21.15.12:0/1405106424 learned_addr learned my addr 172.21.15.12:0/1405106424 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30dcef1700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30d81062a0 0x7f30d8197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30ddef3700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d819bca0 0x7f30d819c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30dd6f2700 1 -- 172.21.15.12:0/1405106424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30d81062a0 msgr2=0x7f30d8197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30d81062a0 0x7f30d8197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30dd6f2700 1 -- 172.21.15.12:0/1405106424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d819bca0 msgr2=0x7f30d819c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d819bca0 0x7f30d819c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.300+0000 7f30dd6f2700 1 -- 172.21.15.12:0/1405106424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc0095f0 con 0x7f30d80697e0 2024-05-08T23:49:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.301+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30d80697e0 0x7f30d81973c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f30c800ef10 tx=0x7f30c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.301+0000 7f30d67fc700 1 -- 172.21.15.12:0/1405106424 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c800c9c0 con 0x7f30d80697e0 2024-05-08T23:49:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.301+0000 7f30df956700 1 -- 172.21.15.12:0/1405106424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d81a3f50 con 0x7f30d80697e0 2024-05-08T23:49:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.301+0000 7f30d67fc700 1 -- 172.21.15.12:0/1405106424 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30c800f040 con 0x7f30d80697e0 2024-05-08T23:49:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.301+0000 7f30d67fc700 1 -- 172.21.15.12:0/1405106424 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8014480 con 0x7f30d80697e0 2024-05-08T23:49:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.301+0000 7f30df956700 1 -- 172.21.15.12:0/1405106424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d81a4370 con 0x7f30d80697e0 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.303+0000 7f30d67fc700 1 -- 172.21.15.12:0/1405106424 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f30c8007690 con 0x7f30d80697e0 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.304+0000 7f30d67fc700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f30c00a1290 0x7f30c00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.304+0000 7f30d67fc700 1 -- 172.21.15.12:0/1405106424 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f30c80cceb0 con 0x7f30d80697e0 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.304+0000 7f30dcef1700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f30c00a1290 0x7f30c00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.304+0000 7f30dcef1700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f30c00a1290 0x7f30c00a3750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009990 tx=0x7f30cc019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30ebb10b80 1 Processor -- start 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30ebb10b80 1 -- start start 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 0x55607aa711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa6b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa6b890 0x55607aa6bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607a9c2150 con 0x55607aa71720 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607a9bfb10 con 0x55607aa6b890 2024-05-08T23:49:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607a9bb400 con 0x55607aa70e00 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30dcef1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa6b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30dcef1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa6b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55670/0 (socket says 172.21.15.12:55670) 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30dcef1700 1 -- 172.21.15.12:0/2602428517 learned_addr learned my addr 172.21.15.12:0/2602428517 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30ddef3700 1 --2- 172.21.15.12:0/2602428517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa6b890 0x55607aa6bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.305+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/2602428517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 0x55607aa711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.306+0000 7f30dcef1700 1 -- 172.21.15.12:0/2602428517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 msgr2=0x55607aa711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.306+0000 7f30dcef1700 1 --2- 172.21.15.12:0/2602428517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 0x55607aa711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.306+0000 7f30dcef1700 1 -- 172.21.15.12:0/2602428517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa6b890 msgr2=0x55607aa6bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.306+0000 7f30dcef1700 1 --2- 172.21.15.12:0/2602428517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa6b890 0x55607aa6bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.306+0000 7f30dcef1700 1 -- 172.21.15.12:0/2602428517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55607a83c3b0 con 0x55607aa71720 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.306+0000 7f30dcef1700 1 --2- 172.21.15.12:0/2602428517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa6b350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f30cc000c00 tx=0x7f30cc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c024c7da5d0afef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.306+0000 7f30bf7fe700 1 -- 172.21.15.12:0/2602428517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc00b9c0 con 0x55607aa71720 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.306+0000 7f30bf7fe700 1 -- 172.21.15.12:0/2602428517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30cc01d040 con 0x55607aa71720 2024-05-08T23:49:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.306+0000 7f30bf7fe700 1 -- 172.21.15.12:0/2602428517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc00f8e0 con 0x55607aa71720 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2602428517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 msgr2=0x55607aa6b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/2602428517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa6b350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f30cc000c00 tx=0x7f30cc01f000 comp rx=0 tx=0).stop 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2602428517 shutdown_connections 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/2602428517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa6b890 0x55607aa6bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/2602428517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 0x55607aa711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/2602428517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa6b350 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2602428517 >> 172.21.15.12:0/2602428517 conn(0x55607a9c9210 msgr2=0x55607aa65df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2602428517 shutdown_connections 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2602428517 wait complete. 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 -- start start 2024-05-08T23:49:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.307+0000 7f30ebb10b80 1 Processor -- start 2024-05-08T23:49:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa6b890 0x55607aa74a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa70e00 0x55607aa74f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607a957560 con 0x55607aa71720 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607a957200 con 0x55607aa70e00 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607a83e6a0 con 0x55607aa6b890 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ebb10b80 10 client.0 _ll_register_callbacks cb 0x55607a960b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ddef3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ddef3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa754d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55676/0 (socket says 172.21.15.12:55676) 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ddef3700 1 -- 172.21.15.12:0/3839894581 learned_addr learned my addr 172.21.15.12:0/3839894581 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30dcef1700 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa70e00 0x55607aa74f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa6b890 0x55607aa74a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ddef3700 1 -- 172.21.15.12:0/3839894581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa6b890 msgr2=0x55607aa74a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ddef3700 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa6b890 0x55607aa74a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ddef3700 1 -- 172.21.15.12:0/3839894581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa70e00 msgr2=0x55607aa74f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ddef3700 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa70e00 0x55607aa74f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30ddef3700 1 -- 172.21.15.12:0/3839894581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d810e660 con 0x55607aa71720 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.308+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa6b890 0x55607aa74a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30ddef3700 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa754d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009fd0 tx=0x7f30d0012a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d76c329aa33fc8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30bcff9700 1 -- 172.21.15.12:0/3839894581 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0017070 con 0x55607aa71720 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30bcff9700 1 -- 172.21.15.12:0/3839894581 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f30d000f480 con 0x55607aa71720 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30ebb10b80 1 -- 172.21.15.12:0/3839894581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55607a914270 con 0x55607aa71720 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30bcff9700 1 -- 172.21.15.12:0/3839894581 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30d00122e0 con 0x55607aa71720 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30ebb10b80 10 client.16215 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30ebb10b80 1 -- 172.21.15.12:0/3839894581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55607a7e8b60 con 0x55607aa71720 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30bcff9700 1 -- 172.21.15.12:0/3839894581 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55607a914270 con 0x55607aa71720 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30bcff9700 1 -- 172.21.15.12:0/3839894581 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f30d0010850 con 0x55607aa71720 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30ebb10b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30ebb10b80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30ebb10b80 20 client.16215 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55607aa852a0 0x55607aa876c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30ebb10b80 4 client.16215 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.309+0000 7f30ebb10b80 1 -- 172.21.15.12:0/3839894581 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55607aa3a810 con 0x55607aa852a0 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.310+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55607aa852a0 0x55607aa876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.310+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55607aa852a0 0x55607aa876c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f3a0f5e3cb75d0 server_cookie=b2cc3b3db15c5cde in_seq=0 out_seq=0 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.310+0000 7f30bcff9700 10 client.16215 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30bcff9700 1 -- 172.21.15.12:0/3839894581 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55607aa3a810 con 0x55607aa852a0 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30bcff9700 10 client.16215 handle_command_reply: tid=0 2024-05-08T23:49:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 client.16215 shutdown 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 -- 172.21.15.12:0/3839894581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 msgr2=0x55607aa754d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa754d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009fd0 tx=0x7f30d0012a20 comp rx=0 tx=0).stop 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 -- 172.21.15.12:0/3839894581 shutdown_connections 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa71720 0x55607aa754d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa6b890 0x55607aa74a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55607aa852a0 0x55607aa876c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/3839894581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa70e00 0x55607aa74f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30bcff9700 0 client.16215 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 -- 172.21.15.12:0/3839894581 >> 172.21.15.12:0/3839894581 conn(0x55607a9c9210 msgr2=0x55607aa655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 -- 172.21.15.12:0/3839894581 shutdown_connections 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.311+0000 7f30ebb10b80 1 -- 172.21.15.12:0/3839894581 wait complete. 2024-05-08T23:49:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.312+0000 7f30ebb10b80 20 client.16215 trim_cache size 0 max 16384 2024-05-08T23:49:16.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30ebb10b80 1 Processor -- start 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30ebb10b80 1 -- start start 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa6b890 0x55607aa6b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 0x55607aa81260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa71720 0x55607aa65d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607aa62dc0 con 0x55607aa6b890 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607aa67a10 con 0x55607aa71720 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607aa8c8d0 con 0x55607aa70e00 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30dd6f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa6b890 0x55607aa6b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30dd6f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa6b890 0x55607aa6b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55682/0 (socket says 172.21.15.12:55682) 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30dd6f2700 1 -- 172.21.15.12:0/57175770 learned_addr learned my addr 172.21.15.12:0/57175770 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30dcef1700 1 --2- 172.21.15.12:0/57175770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 0x55607aa81260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.326+0000 7f30ddef3700 1 --2- 172.21.15.12:0/57175770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa71720 0x55607aa65d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.327+0000 7f30dd6f2700 1 -- 172.21.15.12:0/57175770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 msgr2=0x55607aa81260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.327+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/57175770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 0x55607aa81260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.327+0000 7f30dd6f2700 1 -- 172.21.15.12:0/57175770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa71720 msgr2=0x55607aa65d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.327+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/57175770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa71720 0x55607aa65d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.327+0000 7f30dd6f2700 1 -- 172.21.15.12:0/57175770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55607a981560 con 0x55607aa6b890 2024-05-08T23:49:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.327+0000 7f30dcef1700 1 --2- 172.21.15.12:0/57175770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 0x55607aa81260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.327+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/57175770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa6b890 0x55607aa6b350 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f30c80cac40 tx=0x7f30c8007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84868e89a9d7e3ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.327+0000 7f30bffff700 1 -- 172.21.15.12:0/57175770 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c80b1070 con 0x55607aa6b890 2024-05-08T23:49:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.327+0000 7f30bffff700 1 -- 172.21.15.12:0/57175770 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30c809de30 con 0x55607aa6b890 2024-05-08T23:49:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.327+0000 7f30bffff700 1 -- 172.21.15.12:0/57175770 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c809a260 con 0x55607aa6b890 2024-05-08T23:49:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 -- 172.21.15.12:0/57175770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa6b890 msgr2=0x55607aa6b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/57175770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa6b890 0x55607aa6b350 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f30c80cac40 tx=0x7f30c8007a30 comp rx=0 tx=0).stop 2024-05-08T23:49:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 -- 172.21.15.12:0/57175770 shutdown_connections 2024-05-08T23:49:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/57175770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa71720 0x55607aa65d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/57175770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa6b890 0x55607aa6b350 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/57175770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa70e00 0x55607aa81260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 -- 172.21.15.12:0/57175770 >> 172.21.15.12:0/57175770 conn(0x55607a9c9210 msgr2=0x55607aa6bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 -- 172.21.15.12:0/57175770 shutdown_connections 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 -- 172.21.15.12:0/57175770 wait complete. 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 -- start start 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.328+0000 7f30ebb10b80 1 Processor -- start 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa6b890 0x55607aa7c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa70e00 0x55607aa6c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30ebb10b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa71720 0x55607aa6cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607aa6d580 con 0x55607aa70e00 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607aa6d6f0 con 0x55607aa6b890 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30ebb10b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55607aa6d8b0 con 0x55607aa71720 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30ebb10b80 10 client.0 _ll_register_callbacks cb 0x55607a960b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30dcef1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa70e00 0x55607aa6c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30dcef1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa70e00 0x55607aa6c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55696/0 (socket says 172.21.15.12:55696) 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30dcef1700 1 -- 172.21.15.12:0/2769673942 learned_addr learned my addr 172.21.15.12:0/2769673942 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa6b890 0x55607aa7c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.329+0000 7f30ddef3700 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa71720 0x55607aa6cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30dcef1700 1 -- 172.21.15.12:0/2769673942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa71720 msgr2=0x55607aa6cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30dcef1700 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa71720 0x55607aa6cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30dcef1700 1 -- 172.21.15.12:0/2769673942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa6b890 msgr2=0x55607aa7c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30dcef1700 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa6b890 0x55607aa7c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30dcef1700 1 -- 172.21.15.12:0/2769673942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d81a4370 con 0x55607aa70e00 2024-05-08T23:49:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30ddef3700 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa71720 0x55607aa6cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30dcef1700 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa70e00 0x55607aa6c9a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f30cc005ff0 tx=0x7f30cc00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4344034b3f8224c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30bcff9700 1 -- 172.21.15.12:0/2769673942 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc00b9c0 con 0x55607aa70e00 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30bcff9700 1 -- 172.21.15.12:0/2769673942 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f30cc00f4d0 con 0x55607aa70e00 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2769673942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30cc01d040 con 0x55607aa70e00 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30bcff9700 1 -- 172.21.15.12:0/2769673942 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30cc00f690 con 0x55607aa70e00 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30ebb10b80 10 client.16221 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2769673942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55607a840cd0 con 0x55607aa70e00 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.330+0000 7f30bcff9700 1 -- 172.21.15.12:0/2769673942 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30cc01d040 con 0x55607aa70e00 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.331+0000 7f30bcff9700 1 -- 172.21.15.12:0/2769673942 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f30cc02e770 con 0x55607aa70e00 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.331+0000 7f30ebb10b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.331+0000 7f30ebb10b80 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.331+0000 7f30ebb10b80 20 client.16221 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.331+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55607aadbb10 0x55607aaddf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.331+0000 7f30ebb10b80 4 client.16221 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.331+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2769673942 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55607a9b3ec0 con 0x55607aadbb10 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.331+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55607aadbb10 0x55607aaddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30dd6f2700 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55607aadbb10 0x55607aaddf30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=749cefbd840d0523 server_cookie=79da7a6285c98d35 in_seq=0 out_seq=0 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30bcff9700 10 client.16221 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30bcff9700 1 -- 172.21.15.12:0/2769673942 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55607a9b3ec0 con 0x55607aadbb10 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30bcff9700 10 client.16221 handle_command_reply: tid=0 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 client.16221 shutdown 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2769673942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa70e00 msgr2=0x55607aa6c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa70e00 0x55607aa6c9a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f30cc005ff0 tx=0x7f30cc00f460 comp rx=0 tx=0).stop 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2769673942 shutdown_connections 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55607aa71720 0x55607aa6cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55607aadbb10 0x55607aaddf30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55607aa6b890 0x55607aa7c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 --2- 172.21.15.12:0/2769673942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55607aa70e00 0x55607aa6c9a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2769673942 >> 172.21.15.12:0/2769673942 conn(0x55607a9c9210 msgr2=0x55607aa73040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30bcff9700 0 client.16221 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2769673942 shutdown_connections 2024-05-08T23:49:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 1 -- 172.21.15.12:0/2769673942 wait complete. 2024-05-08T23:49:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.332+0000 7f30ebb10b80 20 client.16221 trim_cache size 0 max 16384 2024-05-08T23:49:16.335 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:49:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 -- 172.21.15.12:0/1405106424 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f30c00a1290 msgr2=0x7f30c00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f30c00a1290 0x7f30c00a3750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009990 tx=0x7f30cc019040 comp rx=0 tx=0).stop 2024-05-08T23:49:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 -- 172.21.15.12:0/1405106424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30d80697e0 msgr2=0x7f30d81973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30d80697e0 0x7f30d81973c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f30c800ef10 tx=0x7f30c800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 -- 172.21.15.12:0/1405106424 shutdown_connections 2024-05-08T23:49:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f30c00a1290 0x7f30c00a3750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d819bca0 0x7f30d819c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30d80697e0 0x7f30d81973c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 --2- 172.21.15.12:0/1405106424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30d81062a0 0x7f30d8197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 -- 172.21.15.12:0/1405106424 >> 172.21.15.12:0/1405106424 conn(0x7f30d8076f10 msgr2=0x7f30d80fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 -- 172.21.15.12:0/1405106424 shutdown_connections 2024-05-08T23:49:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:16.333+0000 7f30df956700 1 -- 172.21.15.12:0/1405106424 wait complete. 2024-05-08T23:49:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:16 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1995662036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:49:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:16 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1995662036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:49:16.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 70626 5.52 MB/sec warmup 72 sec latency 3967.409 ms 2024-05-08T23:49:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:16 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1995662036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:49:17.344 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:49:17.345 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:49:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.500+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1795647069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ef0100a00 msgr2=0x7f6ef0104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.500+0000 7f6ef69e2700 1 --2- 172.21.15.12:0/1795647069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ef0100a00 0x7f6ef0104a40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009990 tx=0x7f6ee4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.500+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1795647069 shutdown_connections 2024-05-08T23:49:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.500+0000 7f6ef69e2700 1 --2- 172.21.15.12:0/1795647069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ef0105170 0x7f6ef010ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.500+0000 7f6ef69e2700 1 --2- 172.21.15.12:0/1795647069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef0100050 0x7f6ef0100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.500+0000 7f6ef69e2700 1 --2- 172.21.15.12:0/1795647069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ef0100a00 0x7f6ef0104a40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.500+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1795647069 >> 172.21.15.12:0/1795647069 conn(0x7f6ef0076fa0 msgr2=0x7f6ef00773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.500+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1795647069 shutdown_connections 2024-05-08T23:49:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.500+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1795647069 wait complete. 2024-05-08T23:49:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.501+0000 7f6ef69e2700 1 Processor -- start 2024-05-08T23:49:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.501+0000 7f6ef69e2700 1 -- start start 2024-05-08T23:49:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6ef69e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ef0100050 0x7f6ef0196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6ef69e2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef0105170 0x7f6ef0196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6ef69e2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ef019abe0 0x7f6ef019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6ef69e2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef019b660 con 0x7f6ef0100050 2024-05-08T23:49:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6ef69e2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef019b7d0 con 0x7f6ef019abe0 2024-05-08T23:49:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6ef69e2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef019b990 con 0x7f6ef0105170 2024-05-08T23:49:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ef0100050 0x7f6ef0196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ef0100050 0x7f6ef0196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55718/0 (socket says 172.21.15.12:55718) 2024-05-08T23:49:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eeffff700 1 -- 172.21.15.12:0/1163587292 learned_addr learned my addr 172.21.15.12:0/1163587292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6ef4f7f700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ef019abe0 0x7f6ef019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef0105170 0x7f6ef0196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eeffff700 1 -- 172.21.15.12:0/1163587292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef0105170 msgr2=0x7f6ef0196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eeffff700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef0105170 0x7f6ef0196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eeffff700 1 -- 172.21.15.12:0/1163587292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ef019abe0 msgr2=0x7f6ef019b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eeffff700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ef019abe0 0x7f6ef019b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eeffff700 1 -- 172.21.15.12:0/1163587292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee40095f0 con 0x7f6ef0100050 2024-05-08T23:49:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef0105170 0x7f6ef0196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:49:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.502+0000 7f6eeffff700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ef0100050 0x7f6ef0196300 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000d950 tx=0x7f6ee000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.503+0000 7f6eed7fa700 1 -- 172.21.15.12:0/1163587292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0009b40 con 0x7f6ef0100050 2024-05-08T23:49:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.503+0000 7f6eed7fa700 1 -- 172.21.15.12:0/1163587292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee0009ca0 con 0x7f6ef0100050 2024-05-08T23:49:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.503+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1163587292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef01a2ef0 con 0x7f6ef0100050 2024-05-08T23:49:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.503+0000 7f6eed7fa700 1 -- 172.21.15.12:0/1163587292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000f620 con 0x7f6ef0100050 2024-05-08T23:49:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.503+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1163587292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef01a3370 con 0x7f6ef0100050 2024-05-08T23:49:17.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.506+0000 7f6eed7fa700 1 -- 172.21.15.12:0/1163587292 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6ee0010460 con 0x7f6ef0100050 2024-05-08T23:49:17.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.506+0000 7f6eed7fa700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6ed80a1290 0x7f6ed80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.506+0000 7f6eed7fa700 1 -- 172.21.15.12:0/1163587292 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6ee00cd020 con 0x7f6ef0100050 2024-05-08T23:49:17.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.506+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6ed80a1290 0x7f6ed80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6ed80a1290 0x7f6ed80a3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400b2e0 tx=0x7f6ee4005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6f02b9cb80 1 Processor -- start 2024-05-08T23:49:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6f02b9cb80 1 -- start start 2024-05-08T23:49:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613ad7a0 0x5648613a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613a7910 0x5648613a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648612fe1d0 con 0x5648613ace80 2024-05-08T23:49:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648612fbb90 con 0x5648613ad7a0 2024-05-08T23:49:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648612f7480 con 0x5648613a7910 2024-05-08T23:49:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.507+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55720/0 (socket says 172.21.15.12:55720) 2024-05-08T23:49:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6eeffff700 1 -- 172.21.15.12:0/227490478 learned_addr learned my addr 172.21.15.12:0/227490478 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6ef4f7f700 1 --2- 172.21.15.12:0/227490478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613a7910 0x5648613a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/227490478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613ad7a0 0x5648613a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6eeffff700 1 -- 172.21.15.12:0/227490478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613a7910 msgr2=0x5648613a7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6eeffff700 1 --2- 172.21.15.12:0/227490478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613a7910 0x5648613a7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6eeffff700 1 -- 172.21.15.12:0/227490478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613ad7a0 msgr2=0x5648613a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6eeffff700 1 --2- 172.21.15.12:0/227490478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613ad7a0 0x5648613a73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6eeffff700 1 -- 172.21.15.12:0/227490478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564861178430 con 0x5648613ace80 2024-05-08T23:49:17.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6eeffff700 1 --2- 172.21.15.12:0/227490478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613ad260 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0096670 tx=0x7f6ee0099d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bf17aabe5b1767 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6ed67fc700 1 -- 172.21.15.12:0/227490478 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee009a170 con 0x5648613ace80 2024-05-08T23:49:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6ed67fc700 1 -- 172.21.15.12:0/227490478 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee009dfa0 con 0x5648613ace80 2024-05-08T23:49:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.508+0000 7f6ed67fc700 1 -- 172.21.15.12:0/227490478 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee009a5d0 con 0x5648613ace80 2024-05-08T23:49:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.509+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/227490478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 msgr2=0x5648613ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.509+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/227490478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613ad260 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0096670 tx=0x7f6ee0099d80 comp rx=0 tx=0).stop 2024-05-08T23:49:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.509+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/227490478 shutdown_connections 2024-05-08T23:49:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.509+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/227490478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613a7910 0x5648613a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.509+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/227490478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613ad260 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.509+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/227490478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613ad7a0 0x5648613a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.509+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/227490478 >> 172.21.15.12:0/227490478 conn(0x564861305290 msgr2=0x5648613a1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.509+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/227490478 shutdown_connections 2024-05-08T23:49:17.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.509+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/227490478 wait complete. 2024-05-08T23:49:17.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.509+0000 7f6f02b9cb80 1 -- start start 2024-05-08T23:49:17.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6f02b9cb80 1 Processor -- start 2024-05-08T23:49:17.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 0x5648613b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ad7a0 0x5648613b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648612935e0 con 0x5648613ace80 2024-05-08T23:49:17.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564861293280 con 0x5648613a7910 2024-05-08T23:49:17.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56486117a720 con 0x5648613ad7a0 2024-05-08T23:49:17.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6f02b9cb80 10 client.0 _ll_register_callbacks cb 0x56486129cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:17.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6eef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6eef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55734/0 (socket says 172.21.15.12:55734) 2024-05-08T23:49:17.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6eef7fe700 1 -- 172.21.15.12:0/268530420 learned_addr learned my addr 172.21.15.12:0/268530420 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:17.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6eeffff700 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 0x5648613b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6ef4f7f700 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ad7a0 0x5648613b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6eef7fe700 1 -- 172.21.15.12:0/268530420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ad7a0 msgr2=0x5648613b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ad7a0 0x5648613b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6eef7fe700 1 -- 172.21.15.12:0/268530420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 msgr2=0x5648613b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.510+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 0x5648613b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6eef7fe700 1 -- 172.21.15.12:0/268530420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef01a3370 con 0x5648613ace80 2024-05-08T23:49:17.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6eeffff700 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 0x5648613b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:17.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613b1070 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009990 tx=0x7f6ee4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d9633aaca1b173 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6ecf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/268530420 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee401e070 con 0x5648613ace80 2024-05-08T23:49:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/268530420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648612502f0 con 0x5648613ace80 2024-05-08T23:49:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/268530420 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6ee400f460 con 0x5648613ace80 2024-05-08T23:49:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/268530420 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ee400f620 con 0x5648613ace80 2024-05-08T23:49:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6f02b9cb80 10 client.16233 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/268530420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564861124be0 con 0x5648613ace80 2024-05-08T23:49:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/268530420 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee403f080 con 0x5648613ace80 2024-05-08T23:49:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.511+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/268530420 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ee403b780 con 0x5648613ace80 2024-05-08T23:49:17.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.512+0000 7f6f02b9cb80 10 client.16233 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:17.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.512+0000 7f6f02b9cb80 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:17.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.512+0000 7f6f02b9cb80 20 client.16233 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:17.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.512+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648613c1270 0x5648613c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.512+0000 7f6f02b9cb80 4 client.16233 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:17.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.512+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/268530420 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564861376890 con 0x5648613c1270 2024-05-08T23:49:17.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.512+0000 7f6eeffff700 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648613c1270 0x5648613c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.512+0000 7f6eeffff700 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648613c1270 0x5648613c3690 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caaecde709e81a2a server_cookie=ceb3524a02f50be7 in_seq=0 out_seq=0 2024-05-08T23:49:17.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.512+0000 7f6ecf7fe700 10 client.16233 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:17.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/268530420 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564861376890 con 0x5648613c1270 2024-05-08T23:49:17.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6ecf7fe700 10 client.16233 handle_command_reply: tid=0 2024-05-08T23:49:17.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 client.16233 shutdown 2024-05-08T23:49:17.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/268530420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 msgr2=0x5648613b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613b1070 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009990 tx=0x7f6ee4009310 comp rx=0 tx=0).stop 2024-05-08T23:49:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/268530420 shutdown_connections 2024-05-08T23:49:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ad7a0 0x5648613b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648613c1270 0x5648613c3690 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 0x5648613b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/268530420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ace80 0x5648613b1070 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/268530420 >> 172.21.15.12:0/268530420 conn(0x564861305290 msgr2=0x5648613a1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6ecf7fe700 0 client.16233 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/268530420 shutdown_connections 2024-05-08T23:49:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.513+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/268530420 wait complete. 2024-05-08T23:49:17.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.514+0000 7f6f02b9cb80 20 client.16233 trim_cache size 0 max 16384 2024-05-08T23:49:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6f02b9cb80 1 Processor -- start 2024-05-08T23:49:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6f02b9cb80 1 -- start start 2024-05-08T23:49:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 0x5648613afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ace80 0x5648613ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ad7a0 0x5648613a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648613b21a0 con 0x5648613ad7a0 2024-05-08T23:49:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56486139ee40 con 0x5648613a7910 2024-05-08T23:49:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648613ad5e0 con 0x5648613ace80 2024-05-08T23:49:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6ef4f7f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ad7a0 0x5648613a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6ef4f7f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ad7a0 0x5648613a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55750/0 (socket says 172.21.15.12:55750) 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6ef4f7f700 1 -- 172.21.15.12:0/4272710053 learned_addr learned my addr 172.21.15.12:0/4272710053 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.532+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/4272710053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ace80 0x5648613ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.533+0000 7f6eeffff700 1 --2- 172.21.15.12:0/4272710053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 0x5648613afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.533+0000 7f6ef4f7f700 1 -- 172.21.15.12:0/4272710053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ace80 msgr2=0x5648613ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.533+0000 7f6ef4f7f700 1 --2- 172.21.15.12:0/4272710053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ace80 0x5648613ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.533+0000 7f6ef4f7f700 1 -- 172.21.15.12:0/4272710053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 msgr2=0x5648613afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.533+0000 7f6ef4f7f700 1 --2- 172.21.15.12:0/4272710053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 0x5648613afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.533+0000 7f6ef4f7f700 1 -- 172.21.15.12:0/4272710053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648613a7fc0 con 0x5648613ad7a0 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.533+0000 7f6ef4f7f700 1 --2- 172.21.15.12:0/4272710053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ad7a0 0x5648613a1da0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8007600 tx=0x7f6ee800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d421433a57080a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.533+0000 7f6ed6ffd700 1 -- 172.21.15.12:0/4272710053 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee800e040 con 0x5648613ad7a0 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.533+0000 7f6ed6ffd700 1 -- 172.21.15.12:0/4272710053 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee800f040 con 0x5648613ad7a0 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.533+0000 7f6ed6ffd700 1 -- 172.21.15.12:0/4272710053 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8016800 con 0x5648613ad7a0 2024-05-08T23:49:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.534+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/4272710053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ad7a0 msgr2=0x5648613a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.534+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/4272710053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ad7a0 0x5648613a1da0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8007600 tx=0x7f6ee800b040 comp rx=0 tx=0).stop 2024-05-08T23:49:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.534+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/4272710053 shutdown_connections 2024-05-08T23:49:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.534+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/4272710053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613ad7a0 0x5648613a1da0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.534+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/4272710053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613a7910 0x5648613afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.534+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/4272710053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ace80 0x5648613ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.534+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/4272710053 >> 172.21.15.12:0/4272710053 conn(0x564861305290 msgr2=0x5648613adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.534+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/4272710053 shutdown_connections 2024-05-08T23:49:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.534+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/4272710053 wait complete. 2024-05-08T23:49:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.534+0000 7f6f02b9cb80 1 -- start start 2024-05-08T23:49:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6f02b9cb80 1 Processor -- start 2024-05-08T23:49:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613a7910 0x5648613b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ace80 0x5648613b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6f02b9cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613ad7a0 0x564861415bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564861416100 con 0x5648613a7910 2024-05-08T23:49:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564861416270 con 0x5648613ad7a0 2024-05-08T23:49:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6f02b9cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564861416430 con 0x5648613ace80 2024-05-08T23:49:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6f02b9cb80 10 client.0 _ll_register_callbacks cb 0x56486129cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613a7910 0x5648613b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613a7910 0x5648613b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55756/0 (socket says 172.21.15.12:55756) 2024-05-08T23:49:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6eeffff700 1 -- 172.21.15.12:0/2048238219 learned_addr learned my addr 172.21.15.12:0/2048238219 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ace80 0x5648613b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6ef4f7f700 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613ad7a0 0x564861415bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6eeffff700 1 -- 172.21.15.12:0/2048238219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ace80 msgr2=0x5648613b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6eeffff700 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ace80 0x5648613b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6eeffff700 1 -- 172.21.15.12:0/2048238219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613ad7a0 msgr2=0x564861415bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6eeffff700 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613ad7a0 0x564861415bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.535+0000 7f6eeffff700 1 -- 172.21.15.12:0/2048238219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee8009320 con 0x5648613a7910 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6eeffff700 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613a7910 0x5648613b5420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6ee009a100 tx=0x7f6ee0010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c40bdcf6bd8bd2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6ecf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/2048238219 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0065070 con 0x5648613a7910 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/2048238219 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6ee00b7070 con 0x5648613a7910 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/2048238219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ee009dfa0 con 0x5648613a7910 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/2048238219 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ee0099ad0 con 0x5648613a7910 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6f02b9cb80 10 client.16239 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/2048238219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648612bd5e0 con 0x5648613a7910 2024-05-08T23:49:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/2048238219 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee00a6080 con 0x5648613a7910 2024-05-08T23:49:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/2048238219 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ee0023070 con 0x5648613a7910 2024-05-08T23:49:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6f02b9cb80 10 client.16239 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6f02b9cb80 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6f02b9cb80 20 client.16239 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56486141d930 0x56486141fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6f02b9cb80 4 client.16239 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:49:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.536+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/2048238219 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5648612eff40 con 0x56486141d930 2024-05-08T23:49:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.537+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56486141d930 0x56486141fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.537+0000 7f6ecf7fe700 10 client.16239 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.537+0000 7f6eef7fe700 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56486141d930 0x56486141fd50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46228a01f2d0897 server_cookie=43f5f2300007de2f in_seq=0 out_seq=0 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6ecf7fe700 1 -- 172.21.15.12:0/2048238219 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5648612eff40 con 0x56486141d930 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6ecf7fe700 10 client.16239 handle_command_reply: tid=0 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6f02b9cb80 1 client.16239 shutdown 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/2048238219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613a7910 msgr2=0x5648613b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613a7910 0x5648613b5420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6ee009a100 tx=0x7f6ee0010c40 comp rx=0 tx=0).stop 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/2048238219 shutdown_connections 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56486141d930 0x56486141fd50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648613ad7a0 0x564861415bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648613a7910 0x5648613b5420 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6f02b9cb80 1 --2- 172.21.15.12:0/2048238219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648613ace80 0x5648613b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/2048238219 >> 172.21.15.12:0/2048238219 conn(0x564861305290 msgr2=0x5648613ac320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6ecf7fe700 0 client.16239 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.538+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/2048238219 shutdown_connections 2024-05-08T23:49:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6f02b9cb80 1 -- 172.21.15.12:0/2048238219 wait complete. 2024-05-08T23:49:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6f02b9cb80 20 client.16239 trim_cache size 0 max 16384 2024-05-08T23:49:17.541 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:49:17.541 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:49:17.541 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "d3c6bc9a-d7e5-4ebd-a395-1fc87be98933", 2024-05-08T23:49:17.541 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:49:17.541 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:49:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1163587292 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6ed80a1290 msgr2=0x7f6ed80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6ed80a1290 0x7f6ed80a3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400b2e0 tx=0x7f6ee4005fb0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1163587292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ef0100050 msgr2=0x7f6ef0196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ef0100050 0x7f6ef0196300 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000d950 tx=0x7f6ee000dd10 comp rx=0 tx=0).stop 2024-05-08T23:49:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1163587292 shutdown_connections 2024-05-08T23:49:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6ed80a1290 0x7f6ed80a3750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ef019abe0 0x7f6ef019b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ef0100050 0x7f6ef0196300 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 --2- 172.21.15.12:0/1163587292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef0105170 0x7f6ef0196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1163587292 >> 172.21.15.12:0/1163587292 conn(0x7f6ef0076fa0 msgr2=0x7f6ef00fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1163587292 shutdown_connections 2024-05-08T23:49:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.539+0000 7f6ef69e2700 1 -- 172.21.15.12:0/1163587292 wait complete. 2024-05-08T23:49:17.550 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:49:17.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:17 smithi012 ceph-mon[42503]: pgmap v90: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 751 KiB/s wr, 96 op/s 2024-05-08T23:49:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.704+0000 7ff751bf6700 1 -- 172.21.15.12:0/2325818459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74c100060 msgr2=0x7ff74c100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.704+0000 7ff751bf6700 1 --2- 172.21.15.12:0/2325818459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74c100060 0x7ff74c100440 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009990 tx=0x7ff73c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.704+0000 7ff751bf6700 1 -- 172.21.15.12:0/2325818459 shutdown_connections 2024-05-08T23:49:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.704+0000 7ff751bf6700 1 --2- 172.21.15.12:0/2325818459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff74c105160 0x7ff74c10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.704+0000 7ff751bf6700 1 --2- 172.21.15.12:0/2325818459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74c100060 0x7ff74c100440 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.704+0000 7ff751bf6700 1 --2- 172.21.15.12:0/2325818459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c100a10 0x7ff74c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.704+0000 7ff751bf6700 1 -- 172.21.15.12:0/2325818459 >> 172.21.15.12:0/2325818459 conn(0x7ff74c076f40 msgr2=0x7ff74c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.704+0000 7ff751bf6700 1 -- 172.21.15.12:0/2325818459 shutdown_connections 2024-05-08T23:49:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.704+0000 7ff751bf6700 1 -- 172.21.15.12:0/2325818459 wait complete. 2024-05-08T23:49:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.705+0000 7ff751bf6700 1 Processor -- start 2024-05-08T23:49:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.705+0000 7ff751bf6700 1 -- start start 2024-05-08T23:49:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.705+0000 7ff751bf6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c100060 0x7ff74c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.705+0000 7ff751bf6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74c105160 0x7ff74c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.705+0000 7ff751bf6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff74c19de80 0x7ff74c19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.705+0000 7ff751bf6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c19e900 con 0x7ff74c105160 2024-05-08T23:49:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.705+0000 7ff751bf6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c19ea70 con 0x7ff74c19de80 2024-05-08T23:49:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.705+0000 7ff751bf6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c19ec30 con 0x7ff74c100060 2024-05-08T23:49:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.705+0000 7ff74affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74c105160 0x7ff74c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff74c19de80 0x7ff74c19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c100060 0x7ff74c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74c105160 0x7ff74c199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55778/0 (socket says 172.21.15.12:55778) 2024-05-08T23:49:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74affd700 1 -- 172.21.15.12:0/1745978754 learned_addr learned my addr 172.21.15.12:0/1745978754 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74affd700 1 -- 172.21.15.12:0/1745978754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c100060 msgr2=0x7ff74c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74affd700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c100060 0x7ff74c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74affd700 1 -- 172.21.15.12:0/1745978754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff74c19de80 msgr2=0x7ff74c19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74affd700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff74c19de80 0x7ff74c19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74affd700 1 -- 172.21.15.12:0/1745978754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c0095f0 con 0x7ff74c105160 2024-05-08T23:49:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c100060 0x7ff74c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff74affd700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74c105160 0x7ff74c199ae0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff74000e970 tx=0x7ff74000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff748ff9700 1 -- 172.21.15.12:0/1745978754 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740009790 con 0x7ff74c105160 2024-05-08T23:49:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff751bf6700 1 -- 172.21.15.12:0/1745978754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74c1a6190 con 0x7ff74c105160 2024-05-08T23:49:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff748ff9700 1 -- 172.21.15.12:0/1745978754 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff74000c7e0 con 0x7ff74c105160 2024-05-08T23:49:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff748ff9700 1 -- 172.21.15.12:0/1745978754 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400054e0 con 0x7ff74c105160 2024-05-08T23:49:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.706+0000 7ff751bf6700 1 -- 172.21.15.12:0/1745978754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a6670 con 0x7ff74c105160 2024-05-08T23:49:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.709+0000 7ff748ff9700 1 -- 172.21.15.12:0/1745978754 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff74000c950 con 0x7ff74c105160 2024-05-08T23:49:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.709+0000 7ff748ff9700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7340a1290 0x7ff7340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.710+0000 7ff748ff9700 1 -- 172.21.15.12:0/1745978754 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff740014070 con 0x7ff74c105160 2024-05-08T23:49:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.710+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7340a1290 0x7ff7340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.710+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7340a1290 0x7ff7340a3750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00b690 tx=0x7ff73c005a90 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff75ddb0b80 1 Processor -- start 2024-05-08T23:49:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff75ddb0b80 1 -- start start 2024-05-08T23:49:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac528e00 0x55e0ac5291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac523350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac523890 0x55e0ac523cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac47a150 con 0x55e0ac529720 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac477b10 con 0x55e0ac523890 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac473400 con 0x55e0ac528e00 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff74affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac523350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac528e00 0x55e0ac5291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff74affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac523350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55780/0 (socket says 172.21.15.12:55780) 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff74affd700 1 -- 172.21.15.12:0/4186113720 learned_addr learned my addr 172.21.15.12:0/4186113720 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.711+0000 7ff74bfff700 1 --2- 172.21.15.12:0/4186113720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac523890 0x55e0ac523cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff74affd700 1 -- 172.21.15.12:0/4186113720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac528e00 msgr2=0x55e0ac5291e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff74affd700 1 --2- 172.21.15.12:0/4186113720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac528e00 0x55e0ac5291e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff74affd700 1 -- 172.21.15.12:0/4186113720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac523890 msgr2=0x55e0ac523cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff74affd700 1 --2- 172.21.15.12:0/4186113720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac523890 0x55e0ac523cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff74affd700 1 -- 172.21.15.12:0/4186113720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0ac2f43b0 con 0x55e0ac529720 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/4186113720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac528e00 0x55e0ac5291e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff74bfff700 1 --2- 172.21.15.12:0/4186113720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac523890 0x55e0ac523cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff74affd700 1 --2- 172.21.15.12:0/4186113720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac523350 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff7400a1f80 tx=0x7ff74000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50e1fa3b0492a61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff731ffb700 1 -- 172.21.15.12:0/4186113720 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400b1070 con 0x55e0ac529720 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff731ffb700 1 -- 172.21.15.12:0/4186113720 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff74009de60 con 0x55e0ac529720 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.712+0000 7ff731ffb700 1 -- 172.21.15.12:0/4186113720 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74009a290 con 0x55e0ac529720 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.713+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/4186113720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 msgr2=0x55e0ac523350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.713+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/4186113720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac523350 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff7400a1f80 tx=0x7ff74000cda0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.713+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/4186113720 shutdown_connections 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.713+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/4186113720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac523890 0x55e0ac523cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.713+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/4186113720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac528e00 0x55e0ac5291e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.713+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/4186113720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac523350 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.713+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/4186113720 >> 172.21.15.12:0/4186113720 conn(0x55e0ac481210 msgr2=0x55e0ac51ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.713+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/4186113720 shutdown_connections 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.713+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/4186113720 wait complete. 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.713+0000 7ff75ddb0b80 1 -- start start 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff75ddb0b80 1 Processor -- start 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac52cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac52d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac529720 0x55e0ac52d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac40f560 con 0x55e0ac528e00 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac40f200 con 0x55e0ac529720 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac2f66a0 con 0x55e0ac523890 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff75ddb0b80 10 client.0 _ll_register_callbacks cb 0x55e0ac418b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac52d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac52cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac52cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43600/0 (socket says 172.21.15.12:43600) 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74b7fe700 1 -- 172.21.15.12:0/2267665331 learned_addr learned my addr 172.21.15.12:0/2267665331 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac52d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55788/0 (socket says 172.21.15.12:55788) 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74bfff700 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac529720 0x55e0ac52d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74b7fe700 1 -- 172.21.15.12:0/2267665331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac529720 msgr2=0x55e0ac52d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac529720 0x55e0ac52d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74b7fe700 1 -- 172.21.15.12:0/2267665331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 msgr2=0x55e0ac52d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac52d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74b7fe700 1 -- 172.21.15.12:0/2267665331 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c067ff0 con 0x55e0ac523890 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.714+0000 7ff74affd700 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac52d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac52cb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009990 tx=0x7ff73c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee06a085b2127d61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff72affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff72affd700 1 -- 172.21.15.12:0/2267665331 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c005260 con 0x55e0ac523890 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff72affd700 1 -- 172.21.15.12:0/2267665331 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff73c0053c0 con 0x55e0ac523890 2024-05-08T23:49:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/2267665331 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0ac3cc270 con 0x55e0ac523890 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff72affd700 1 -- 172.21.15.12:0/2267665331 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff73c0247f0 con 0x55e0ac523890 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff75ddb0b80 10 client.26065 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/2267665331 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff74c1a6670 con 0x55e0ac523890 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff72affd700 1 -- 172.21.15.12:0/2267665331 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0ac3cc270 con 0x55e0ac523890 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff72affd700 1 -- 172.21.15.12:0/2267665331 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff73c02f070 con 0x55e0ac523890 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff75ddb0b80 10 client.26065 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff75ddb0b80 10 client.26065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff75ddb0b80 20 client.26065 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0ac53d340 0x55e0ac53f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff75ddb0b80 4 client.26065 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.715+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/2267665331 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0ac4f2810 con 0x55e0ac53d340 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.716+0000 7ff74affd700 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0ac53d340 0x55e0ac53f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.716+0000 7ff74affd700 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0ac53d340 0x55e0ac53f760 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9116dd0623b8561a server_cookie=6f4bdf6a7f43ceb1 in_seq=0 out_seq=0 2024-05-08T23:49:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.716+0000 7ff72affd700 10 client.26065 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff72affd700 1 -- 172.21.15.12:0/2267665331 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0ac4f2810 con 0x55e0ac53d340 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff72affd700 10 client.26065 handle_command_reply: tid=0 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 client.26065 shutdown 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/2267665331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 msgr2=0x55e0ac52cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac52cb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009990 tx=0x7ff73c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/2267665331 shutdown_connections 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0ac53d340 0x55e0ac53f760 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac529720 0x55e0ac52d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac52cb30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff72affd700 0 client.26065 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/2267665331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac52d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/2267665331 >> 172.21.15.12:0/2267665331 conn(0x55e0ac481210 msgr2=0x55e0ac51d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/2267665331 shutdown_connections 2024-05-08T23:49:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/2267665331 wait complete. 2024-05-08T23:49:17.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.717+0000 7ff75ddb0b80 20 client.26065 trim_cache size 0 max 16384 2024-05-08T23:49:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff75ddb0b80 1 Processor -- start 2024-05-08T23:49:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff75ddb0b80 1 -- start start 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac51fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac528e00 0x55e0ac51dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac52d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac523700 con 0x55e0ac529720 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac544970 con 0x55e0ac528e00 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac516040 con 0x55e0ac523890 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac52d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac51fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.737+0000 7ff74affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac528e00 0x55e0ac51dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac52d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55802/0 (socket says 172.21.15.12:55802) 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac51fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43614/0 (socket says 172.21.15.12:43614) 2024-05-08T23:49:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac528e00 0x55e0ac51dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42152/0 (socket says 172.21.15.12:42152) 2024-05-08T23:49:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74bfff700 1 -- 172.21.15.12:0/1083853148 learned_addr learned my addr 172.21.15.12:0/1083853148 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74affd700 1 -- 172.21.15.12:0/1083853148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 msgr2=0x55e0ac51fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74affd700 1 --2- 172.21.15.12:0/1083853148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac51fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74affd700 1 -- 172.21.15.12:0/1083853148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 msgr2=0x55e0ac52d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74affd700 1 --2- 172.21.15.12:0/1083853148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac52d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74affd700 1 -- 172.21.15.12:0/1083853148 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0ac2a0b60 con 0x55e0ac528e00 2024-05-08T23:49:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/1083853148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac51fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff74affd700 1 --2- 172.21.15.12:0/1083853148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac528e00 0x55e0ac51dd20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e0ac529b80 tx=0x7ff74009c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf731fa4607bb855 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff7327fc700 1 -- 172.21.15.12:0/1083853148 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400b1040 con 0x55e0ac528e00 2024-05-08T23:49:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff7327fc700 1 -- 172.21.15.12:0/1083853148 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff740005660 con 0x55e0ac528e00 2024-05-08T23:49:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.738+0000 7ff7327fc700 1 -- 172.21.15.12:0/1083853148 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74009a4f0 con 0x55e0ac528e00 2024-05-08T23:49:17.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.739+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1083853148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac528e00 msgr2=0x55e0ac51dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.739+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/1083853148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac528e00 0x55e0ac51dd20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e0ac529b80 tx=0x7ff74009c040 comp rx=0 tx=0).stop 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.739+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1083853148 shutdown_connections 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.739+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/1083853148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac529720 0x55e0ac52d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.739+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/1083853148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac51fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.739+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/1083853148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac528e00 0x55e0ac51dd20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.739+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1083853148 >> 172.21.15.12:0/1083853148 conn(0x55e0ac481210 msgr2=0x55e0ac51a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.739+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1083853148 shutdown_connections 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.739+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1083853148 wait complete. 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.740+0000 7ff75ddb0b80 1 -- start start 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.740+0000 7ff75ddb0b80 1 Processor -- start 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.740+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac534360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.740+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac532150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.740+0000 7ff75ddb0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac529720 0x55e0ac532690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.740+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac534a50 con 0x55e0ac528e00 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.740+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac532bd0 con 0x55e0ac529720 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.740+0000 7ff75ddb0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ac532d90 con 0x55e0ac523890 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.740+0000 7ff75ddb0b80 10 client.0 _ll_register_callbacks cb 0x55e0ac418b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac534360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac532150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac534360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43626/0 (socket says 172.21.15.12:43626) 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74b7fe700 1 -- 172.21.15.12:0/1182378326 learned_addr learned my addr 172.21.15.12:0/1182378326 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74bfff700 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac529720 0x55e0ac532690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74b7fe700 1 -- 172.21.15.12:0/1182378326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac529720 msgr2=0x55e0ac532690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac529720 0x55e0ac532690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74b7fe700 1 -- 172.21.15.12:0/1182378326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 msgr2=0x55e0ac532150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac532150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74b7fe700 1 -- 172.21.15.12:0/1182378326 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a6670 con 0x55e0ac523890 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74affd700 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac532150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff74b7fe700 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac534360 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff73c03c040 tx=0x7ff73c004f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ad735b3a2b26848 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff72affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff72affd700 1 -- 172.21.15.12:0/1182378326 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c004500 con 0x55e0ac523890 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff72affd700 1 -- 172.21.15.12:0/1182378326 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff73c004660 con 0x55e0ac523890 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.741+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1182378326 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff74009de60 con 0x55e0ac523890 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff72affd700 1 -- 172.21.15.12:0/1182378326 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff73c004860 con 0x55e0ac523890 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff75ddb0b80 10 client.26071 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1182378326 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0ac2f43b0 con 0x55e0ac523890 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff72affd700 1 -- 172.21.15.12:0/1182378326 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff74009de60 con 0x55e0ac523890 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff72affd700 1 -- 172.21.15.12:0/1182378326 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff73c00fb90 con 0x55e0ac523890 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff75ddb0b80 10 client.26071 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff75ddb0b80 10 client.26071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:17.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff75ddb0b80 20 client.26071 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0ac593850 0x55e0ac595c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff75ddb0b80 4 client.26071 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1182378326 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e0ac46bec0 con 0x55e0ac593850 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.742+0000 7ff74affd700 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0ac593850 0x55e0ac595c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff72affd700 10 client.26071 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff74affd700 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0ac593850 0x55e0ac595c70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d04c5e4c4a57cdc4 server_cookie=bc713bb5ea4c3318 in_seq=0 out_seq=0 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff72affd700 1 -- 172.21.15.12:0/1182378326 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e0ac46bec0 con 0x55e0ac593850 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff72affd700 10 client.26071 handle_command_reply: tid=0 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 client.26071 shutdown 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1182378326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 msgr2=0x55e0ac534360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac534360 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff73c03c040 tx=0x7ff73c004f50 comp rx=0 tx=0).stop 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1182378326 shutdown_connections 2024-05-08T23:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0ac593850 0x55e0ac595c70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0ac529720 0x55e0ac532690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ac523890 0x55e0ac534360 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 --2- 172.21.15.12:0/1182378326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0ac528e00 0x55e0ac532150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff72affd700 0 client.26071 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1182378326 >> 172.21.15.12:0/1182378326 conn(0x55e0ac481210 msgr2=0x55e0ac52b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1182378326 shutdown_connections 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 1 -- 172.21.15.12:0/1182378326 wait complete. 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.743+0000 7ff75ddb0b80 20 client.26071 trim_cache size 0 max 16384 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:49:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:49:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "d3c6bc9a-d7e5-4ebd-a395-1fc87be98933": { 2024-05-08T23:49:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:49:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "d3c6bc9a-d7e5-4ebd-a395-1fc87be98933", 2024-05-08T23:49:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:49:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:49:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:49:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 -- 172.21.15.12:0/1745978754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7340a1290 msgr2=0x7ff7340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7340a1290 0x7ff7340a3750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00b690 tx=0x7ff73c005a90 comp rx=0 tx=0).stop 2024-05-08T23:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 -- 172.21.15.12:0/1745978754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74c105160 msgr2=0x7ff74c199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74c105160 0x7ff74c199ae0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff74000e970 tx=0x7ff74000ec80 comp rx=0 tx=0).stop 2024-05-08T23:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 -- 172.21.15.12:0/1745978754 shutdown_connections 2024-05-08T23:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7340a1290 0x7ff7340a3750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff74c19de80 0x7ff74c19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff74c100060 0x7ff74c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 --2- 172.21.15.12:0/1745978754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74c105160 0x7ff74c199ae0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:17.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 -- 172.21.15.12:0/1745978754 >> 172.21.15.12:0/1745978754 conn(0x7ff74c076f40 msgr2=0x7ff74c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:17.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 -- 172.21.15.12:0/1745978754 shutdown_connections 2024-05-08T23:49:17.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:17.744+0000 7ff751bf6700 1 -- 172.21.15.12:0/1745978754 wait complete. 2024-05-08T23:49:17.753 INFO:tasks.cephfs.filesystem:scrub status for tag:d3c6bc9a-d7e5-4ebd-a395-1fc87be98933 - {'path': '/', 'tag': 'd3c6bc9a-d7e5-4ebd-a395-1fc87be98933', 'options': 'recursive,force'} 2024-05-08T23:49:17.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:17 smithi160 ceph-mon[46152]: pgmap v90: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 751 KiB/s wr, 96 op/s 2024-05-08T23:49:17.765 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 23:49:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-b[82741]: 2024-05-08T23:49:17.546+0000 7fd010205700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...5,head] /volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{1=1,2=1} v12961 ap=2 f(v64 m2024-05-08T23:49:02.105346+0000 11=11+0) n(v3 rc2024-05-08T23:49:02.105346+0000 b2861056 12=11+1) old_inodes=1 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15273=pAsLsXsFs/pAsLsXsFsx@1321} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55c9454f0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/74785fcb-077d-40a9-a792-1aeb808dc685 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v44>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v12961>,<0x10000000008/~dmtmp v977>,<0x10000000007/client0 v137>,<0x10000000006/clients v114>,<0x10000000005/tmp v115>,<0x100000001fb/client.0 v110>,<0x100000001fa/74785fcb-077d-40a9-a792-1aeb808dc685 v130>,<0x10000000001/sv_1 v143>,<0x10000000000/qa v132>,<0x1/volumes v123>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:49:01.631350+0000 15=15+0)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:49:02.105346+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v64 m2024-05-08T23:49:02.105346+0000 11=11+0)","memory_value.rstat":"n(v3 rc2024-05-08T23:49:02.105346+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-08T23:49:17.765 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 08 23:49:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-b[82741]: 2024-05-08T23:49:17.550+0000 7fd010205700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000229 [...5,head] /volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685/client.0/tmp/clients/client0/~dmtmp/PARADOX/ auth{1=1,2=1} v13022 ap=2 f(v104 m2024-05-08T23:49:02.757341+0000 24=24+0) n(v5 rc2024-05-08T23:49:02.757341+0000 b5201920 25=24+1) old_inodes=1 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15273=pAsLsXsFs/pAsLsXsFsx@2347} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x55c945505080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000229:[<0x10000000009/PARADOX v2418>,<0x10000000008/~dmtmp v38>,<0x10000000007/client0 v22>,<0x10000000006/clients v12>,<0x10000000005/tmp v12>,<0x100000001fb/client.0 v16>,<0x100000001fa/74785fcb-077d-40a9-a792-1aeb808dc685 v32>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v44>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000229:[<0x10000000009/PARADOX v13022>,<0x10000000008/~dmtmp v977>,<0x10000000007/client0 v137>,<0x10000000006/clients v114>,<0x10000000005/tmp v115>,<0x100000001fb/client.0 v110>,<0x100000001fa/74785fcb-077d-40a9-a792-1aeb808dc685 v130>,<0x10000000001/sv_1 v143>,<0x10000000000/qa v132>,<0x1/volumes v123>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-08T23:49:02.720341+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-05-08T23:49:02.757341+0000 b5201920 25=24+1)","memory_value.dirstat":"f(v104 m2024-05-08T23:49:02.757341+0000 24=24+0)","memory_value.rstat":"n(v5 rc2024-05-08T23:49:02.757341+0000 b5201920 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-08T23:49:17.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 71202 5.52 MB/sec warmup 73 sec latency 4674.391 ms 2024-05-08T23:49:18.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:17 smithi204 ceph-mon[47373]: pgmap v90: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 751 KiB/s wr, 96 op/s 2024-05-08T23:49:18.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:18 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:49:18.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:18 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:18.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:18 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:49:18.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:18 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:49:18.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:18 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:49:18.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:18 smithi160 ceph-mon[46152]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-05-08T23:49:18.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:18 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:18.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:18 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:18.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:18 smithi160 ceph-mon[46152]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-05-08T23:49:18.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:18 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:18.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 73343 5.55 MB/sec warmup 74 sec latency 17.398 ms 2024-05-08T23:49:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:18 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:49:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:18 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:18 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:49:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:18 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:49:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:18 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:49:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:18 smithi012 ceph-mon[42503]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-05-08T23:49:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:18 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:18 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:18 smithi012 ceph-mon[42503]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-05-08T23:49:18.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:18 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:19.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:18 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:49:19.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:18 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:19.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:18 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:49:19.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:18 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:49:19.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:18 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:49:19.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:18 smithi204 ceph-mon[47373]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-05-08T23:49:19.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:18 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:19.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:18 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:19.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:18 smithi204 ceph-mon[47373]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/74785fcb-077d-40a9-a792-1aeb808dc685/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-05-08T23:49:19.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:18 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:19.854 INFO:tasks.workunit.client.0.smithi012.stdout: 1 73755 5.53 MB/sec warmup 75 sec latency 840.117 ms 2024-05-08T23:49:19.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:19 smithi012 ceph-mon[42503]: pgmap v91: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 488 KiB/s wr, 31 op/s 2024-05-08T23:49:20.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:19 smithi204 ceph-mon[47373]: pgmap v91: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 488 KiB/s wr, 31 op/s 2024-05-08T23:49:20.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:19 smithi160 ceph-mon[46152]: pgmap v91: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 488 KiB/s wr, 31 op/s 2024-05-08T23:49:20.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 73755 5.45 MB/sec warmup 76 sec latency 1840.208 ms 2024-05-08T23:49:21.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 73755 5.38 MB/sec warmup 77 sec latency 2840.327 ms 2024-05-08T23:49:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:22 smithi012 ceph-mon[42503]: pgmap v92: 97 pgs: 97 active+clean; 33 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 926 KiB/s wr, 86 op/s 2024-05-08T23:49:22.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:22 smithi204 ceph-mon[47373]: pgmap v92: 97 pgs: 97 active+clean; 33 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 926 KiB/s wr, 86 op/s 2024-05-08T23:49:22.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:22 smithi160 ceph-mon[46152]: pgmap v92: 97 pgs: 97 active+clean; 33 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 926 KiB/s wr, 86 op/s 2024-05-08T23:49:22.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 73966 5.33 MB/sec warmup 78 sec latency 3741.981 ms 2024-05-08T23:49:23.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:23 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:49:23.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:23 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:49:23.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:23 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:49:23.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74005 5.27 MB/sec warmup 79 sec latency 963.610 ms 2024-05-08T23:49:24.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:49:24.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:49:24.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:24 smithi012 ceph-mon[42503]: pgmap v93: 97 pgs: 97 active+clean; 33 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 738 KiB/s wr, 70 op/s 2024-05-08T23:49:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:49:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:49:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:24 smithi204 ceph-mon[47373]: pgmap v93: 97 pgs: 97 active+clean; 33 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 738 KiB/s wr, 70 op/s 2024-05-08T23:49:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:49:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:49:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:24 smithi160 ceph-mon[46152]: pgmap v93: 97 pgs: 97 active+clean; 33 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 738 KiB/s wr, 70 op/s 2024-05-08T23:49:24.851 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74005 5.20 MB/sec warmup 80 sec latency 1963.699 ms 2024-05-08T23:49:25.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74005 5.14 MB/sec warmup 81 sec latency 2963.803 ms 2024-05-08T23:49:26.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:26 smithi204 ceph-mon[47373]: pgmap v94: 97 pgs: 97 active+clean; 33 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 738 KiB/s wr, 70 op/s 2024-05-08T23:49:26.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:26 smithi160 ceph-mon[46152]: pgmap v94: 97 pgs: 97 active+clean; 33 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 738 KiB/s wr, 70 op/s 2024-05-08T23:49:26.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74005 5.07 MB/sec warmup 82 sec latency 3963.923 ms 2024-05-08T23:49:26.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:26 smithi012 ceph-mon[42503]: pgmap v94: 97 pgs: 97 active+clean; 33 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 738 KiB/s wr, 70 op/s 2024-05-08T23:49:27.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74202 5.02 MB/sec warmup 83 sec latency 4870.218 ms 2024-05-08T23:49:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:28 smithi204 ceph-mon[47373]: pgmap v95: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 790 KiB/s wr, 80 op/s 2024-05-08T23:49:28.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:28 smithi160 ceph-mon[46152]: pgmap v95: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 790 KiB/s wr, 80 op/s 2024-05-08T23:49:28.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74202 4.96 MB/sec warmup 84 sec latency 1032.001 ms 2024-05-08T23:49:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:28 smithi012 ceph-mon[42503]: pgmap v95: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 790 KiB/s wr, 80 op/s 2024-05-08T23:49:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.235+0000 7f3e00503700 1 -- 172.21.15.12:0/276486995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3df8107000 msgr2=0x7f3df81073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.235+0000 7f3e00503700 1 --2- 172.21.15.12:0/276486995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3df8107000 0x7f3df81073e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009990 tx=0x7f3df4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.235+0000 7f3e00503700 1 -- 172.21.15.12:0/276486995 shutdown_connections 2024-05-08T23:49:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.235+0000 7f3e00503700 1 --2- 172.21.15.12:0/276486995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df8068ca0 0x7f3df810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.235+0000 7f3e00503700 1 --2- 172.21.15.12:0/276486995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3df8107000 0x7f3df81073e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.235+0000 7f3e00503700 1 --2- 172.21.15.12:0/276486995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3df80682e0 0x7f3df8068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.235+0000 7f3e00503700 1 -- 172.21.15.12:0/276486995 >> 172.21.15.12:0/276486995 conn(0x7f3df8076f10 msgr2=0x7f3df8077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.235+0000 7f3e00503700 1 -- 172.21.15.12:0/276486995 shutdown_connections 2024-05-08T23:49:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.235+0000 7f3e00503700 1 -- 172.21.15.12:0/276486995 wait complete. 2024-05-08T23:49:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3e00503700 1 Processor -- start 2024-05-08T23:49:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3e00503700 1 -- start start 2024-05-08T23:49:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3e00503700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df80682e0 0x7f3df80ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3e00503700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3df8068ca0 0x7f3df8100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3e00503700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3df8107000 0x7f3df8100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3e00503700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df81a5680 con 0x7f3df8107000 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3e00503700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df81a57f0 con 0x7f3df8068ca0 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3e00503700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df81a59b0 con 0x7f3df80682e0 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3dfeaa0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3df8107000 0x7f3df8100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3dfeaa0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3df8107000 0x7f3df8100890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47492/0 (socket says 172.21.15.12:47492) 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3dfeaa0700 1 -- 172.21.15.12:0/775750616 learned_addr learned my addr 172.21.15.12:0/775750616 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.236+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df80682e0 0x7f3df80ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3df8068ca0 0x7f3df8100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3dfeaa0700 1 -- 172.21.15.12:0/775750616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df80682e0 msgr2=0x7f3df80ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3dfeaa0700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df80682e0 0x7f3df80ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3dfeaa0700 1 -- 172.21.15.12:0/775750616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3df8068ca0 msgr2=0x7f3df8100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3dfeaa0700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3df8068ca0 0x7f3df8100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3dfeaa0700 1 -- 172.21.15.12:0/775750616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df40095f0 con 0x7f3df8107000 2024-05-08T23:49:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3df8068ca0 0x7f3df8100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3dfeaa0700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3df8107000 0x7f3df8100890 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3df000ef30 tx=0x7f3df000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3def7fe700 1 -- 172.21.15.12:0/775750616 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0018070 con 0x7f3df8107000 2024-05-08T23:49:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3e00503700 1 -- 172.21.15.12:0/775750616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df81a5c30 con 0x7f3df8107000 2024-05-08T23:49:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3def7fe700 1 -- 172.21.15.12:0/775750616 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3df000f040 con 0x7f3df8107000 2024-05-08T23:49:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3def7fe700 1 -- 172.21.15.12:0/775750616 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df000cdb0 con 0x7f3df8107000 2024-05-08T23:49:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.237+0000 7f3e00503700 1 -- 172.21.15.12:0/775750616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df81a6140 con 0x7f3df8107000 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.239+0000 7f3def7fe700 1 -- 172.21.15.12:0/775750616 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3df0003a00 con 0x7f3df8107000 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.240+0000 7f3def7fe700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3de40a1290 0x7f3de40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.240+0000 7f3def7fe700 1 -- 172.21.15.12:0/775750616 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3df00cc930 con 0x7f3df8107000 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.240+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3de40a1290 0x7f3de40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.241+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3de40a1290 0x7f3de40a3750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b2e0 tx=0x7f3df4005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.241+0000 7f3e0c6bdb80 1 Processor -- start 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.241+0000 7f3e0c6bdb80 1 -- start start 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.241+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ee950 0x56429f3eed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.241+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.241+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3e9480 0x56429f3e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.241+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f3400f0 con 0x56429f3ef270 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.241+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f33dab0 con 0x56429f3ee950 2024-05-08T23:49:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.241+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f3393a0 con 0x56429f3e9480 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfda9e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfda9e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3e8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47506/0 (socket says 172.21.15.12:47506) 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfda9e700 1 -- 172.21.15.12:0/3284087259 learned_addr learned my addr 172.21.15.12:0/3284087259 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/3284087259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ee950 0x56429f3eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfeaa0700 1 --2- 172.21.15.12:0/3284087259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3e9480 0x56429f3e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfda9e700 1 -- 172.21.15.12:0/3284087259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3e9480 msgr2=0x56429f3e98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/3284087259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3e9480 0x56429f3e98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfda9e700 1 -- 172.21.15.12:0/3284087259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ee950 msgr2=0x56429f3eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/3284087259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ee950 0x56429f3eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfda9e700 1 -- 172.21.15.12:0/3284087259 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429f1ba350 con 0x56429f3ef270 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/3284087259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3e8f40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3de800bd60 tx=0x7f3de800f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69bf0f335635b74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3decff9700 1 -- 172.21.15.12:0/3284087259 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de800fda0 con 0x56429f3ef270 2024-05-08T23:49:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3decff9700 1 -- 172.21.15.12:0/3284087259 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de8011900 con 0x56429f3ef270 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.242+0000 7f3decff9700 1 -- 172.21.15.12:0/3284087259 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80106c0 con 0x56429f3ef270 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.243+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/3284087259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 msgr2=0x56429f3e8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.243+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/3284087259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3e8f40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3de800bd60 tx=0x7f3de800f640 comp rx=0 tx=0).stop 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.243+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/3284087259 shutdown_connections 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.243+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/3284087259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3e9480 0x56429f3e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.243+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/3284087259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ee950 0x56429f3eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.243+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/3284087259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3e8f40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.243+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/3284087259 >> 172.21.15.12:0/3284087259 conn(0x56429f3471b0 msgr2=0x56429f3e3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.243+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/3284087259 shutdown_connections 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.243+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/3284087259 wait complete. 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3e0c6bdb80 1 -- start start 2024-05-08T23:49:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3e0c6bdb80 1 Processor -- start 2024-05-08T23:49:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3e9480 0x56429f3f5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3ee950 0x56429f3f2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f2d5500 con 0x56429f3ef270 2024-05-08T23:49:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f2d51a0 con 0x56429f3e9480 2024-05-08T23:49:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f1bc640 con 0x56429f3ee950 2024-05-08T23:49:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3e0c6bdb80 10 client.0 _ll_register_callbacks cb 0x56429f2dead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3dfeaa0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3dfeaa0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47514/0 (socket says 172.21.15.12:47514) 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.244+0000 7f3dfeaa0700 1 -- 172.21.15.12:0/4170601989 learned_addr learned my addr 172.21.15.12:0/4170601989 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.245+0000 7f3dfeaa0700 1 -- 172.21.15.12:0/4170601989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3ee950 msgr2=0x56429f3f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.245+0000 7f3dfeaa0700 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3ee950 0x56429f3f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.245+0000 7f3dfeaa0700 1 -- 172.21.15.12:0/4170601989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3e9480 msgr2=0x56429f3f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.245+0000 7f3dfeaa0700 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3e9480 0x56429f3f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.245+0000 7f3dfeaa0700 1 -- 172.21.15.12:0/4170601989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de800b9c0 con 0x56429f3ef270 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.245+0000 7f3dfeaa0700 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3f2b60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3df000ef00 tx=0x7f3df0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de02c19395fe74c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.245+0000 7f3de1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.245+0000 7f3de1ffb700 1 -- 172.21.15.12:0/4170601989 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0099ab0 con 0x56429f3ef270 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.245+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/4170601989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56429f292210 con 0x56429f3ef270 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.245+0000 7f3de1ffb700 1 -- 172.21.15.12:0/4170601989 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3df009d8e0 con 0x56429f3ef270 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3de1ffb700 1 -- 172.21.15.12:0/4170601989 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3df00a1ea0 con 0x56429f3ef270 2024-05-08T23:49:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3e0c6bdb80 10 client.16269 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/4170601989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56429f166b00 con 0x56429f3ef270 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3de1ffb700 1 -- 172.21.15.12:0/4170601989 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3df0063020 con 0x56429f3ef270 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3de1ffb700 1 -- 172.21.15.12:0/4170601989 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3df0084050 con 0x56429f3ef270 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3e0c6bdb80 10 client.16269 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3e0c6bdb80 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3e0c6bdb80 20 client.16269 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56429f402ac0 0x56429f404f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3e0c6bdb80 4 client.16269 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.246+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/4170601989 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56429f3b87b0 con 0x56429f402ac0 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.247+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56429f402ac0 0x56429f404f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.247+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56429f402ac0 0x56429f404f00 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89cf6bf87a37b83c server_cookie=de31a3aaa11d44d6 in_seq=0 out_seq=0 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.247+0000 7f3de1ffb700 10 client.16269 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.248+0000 7f3de1ffb700 1 -- 172.21.15.12:0/4170601989 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56429f3b87b0 con 0x56429f402ac0 2024-05-08T23:49:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.248+0000 7f3de1ffb700 10 client.16269 handle_command_reply: tid=0 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.248+0000 7f3e0c6bdb80 1 client.16269 shutdown 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/4170601989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 msgr2=0x56429f3f2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3f2b60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3df000ef00 tx=0x7f3df0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/4170601989 shutdown_connections 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ef270 0x56429f3f2b60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56429f402ac0 0x56429f404f00 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3e9480 0x56429f3f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/4170601989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3ee950 0x56429f3f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3de1ffb700 0 client.16269 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/4170601989 >> 172.21.15.12:0/4170601989 conn(0x56429f3471b0 msgr2=0x56429f3ed4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/4170601989 shutdown_connections 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/4170601989 wait complete. 2024-05-08T23:49:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.249+0000 7f3e0c6bdb80 20 client.16269 trim_cache size 0 max 16384 2024-05-08T23:49:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.265+0000 7f3e0c6bdb80 1 Processor -- start 2024-05-08T23:49:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.265+0000 7f3e0c6bdb80 1 -- start start 2024-05-08T23:49:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3e9480 0x56429f404eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3e3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ef270 0x56429f3f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f3e4330 con 0x56429f3ee950 2024-05-08T23:49:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f3ff6d0 con 0x56429f3ef270 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f3f3b00 con 0x56429f3e9480 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfda9e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfda9e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3e3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47516/0 (socket says 172.21.15.12:47516) 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfda9e700 1 -- 172.21.15.12:0/3485731326 learned_addr learned my addr 172.21.15.12:0/3485731326 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/3485731326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3e9480 0x56429f404eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfeaa0700 1 --2- 172.21.15.12:0/3485731326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ef270 0x56429f3f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfda9e700 1 -- 172.21.15.12:0/3485731326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3e9480 msgr2=0x56429f404eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/3485731326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3e9480 0x56429f404eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfda9e700 1 -- 172.21.15.12:0/3485731326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ef270 msgr2=0x56429f3f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/3485731326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ef270 0x56429f3f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfda9e700 1 -- 172.21.15.12:0/3485731326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429f2ff500 con 0x56429f3ee950 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfeaa0700 1 --2- 172.21.15.12:0/3485731326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ef270 0x56429f3f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/3485731326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3e3910 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3de800bd00 tx=0x7f3de800b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeddf670a1478e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.266+0000 7f3ded7fa700 1 -- 172.21.15.12:0/3485731326 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de800fda0 con 0x56429f3ee950 2024-05-08T23:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3ded7fa700 1 -- 172.21.15.12:0/3485731326 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de800beb0 con 0x56429f3ee950 2024-05-08T23:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3ded7fa700 1 -- 172.21.15.12:0/3485731326 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80043f0 con 0x56429f3ee950 2024-05-08T23:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/3485731326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 msgr2=0x56429f3e3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/3485731326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3e3910 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3de800bd00 tx=0x7f3de800b2e0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/3485731326 shutdown_connections 2024-05-08T23:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/3485731326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3ef270 0x56429f3f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/3485731326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3e9480 0x56429f404eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/3485731326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3e3910 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/3485731326 >> 172.21.15.12:0/3485731326 conn(0x56429f3471b0 msgr2=0x56429f3f1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/3485731326 shutdown_connections 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.267+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/3485731326 wait complete. 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3e0c6bdb80 1 -- start start 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3e0c6bdb80 1 Processor -- start 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3e9480 0x56429f3f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3f8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3e0c6bdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3ef270 0x56429f3f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f458020 con 0x56429f3ee950 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f458160 con 0x56429f3e9480 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3e0c6bdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429f4582a0 con 0x56429f3ef270 2024-05-08T23:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3e0c6bdb80 10 client.0 _ll_register_callbacks cb 0x56429f2dead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3dfda9e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3dfda9e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47518/0 (socket says 172.21.15.12:47518) 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3dfda9e700 1 -- 172.21.15.12:0/287587724 learned_addr learned my addr 172.21.15.12:0/287587724 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3dfeaa0700 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3ef270 0x56429f3f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.268+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3e9480 0x56429f3f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3dfda9e700 1 -- 172.21.15.12:0/287587724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3ef270 msgr2=0x56429f3f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3ef270 0x56429f3f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3dfda9e700 1 -- 172.21.15.12:0/287587724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3e9480 msgr2=0x56429f3f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3e9480 0x56429f3f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3dfda9e700 1 -- 172.21.15.12:0/287587724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56429f1ba350 con 0x56429f3ee950 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3e9480 0x56429f3f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3dfda9e700 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3f8190 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x56429f3b88f0 tx=0x7f3de8011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e31606c3e9f2e8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3de1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3de1ffb700 1 -- 172.21.15.12:0/287587724 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de800fe70 con 0x56429f3ee950 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/287587724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3de8011900 con 0x56429f3ee950 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3de1ffb700 1 -- 172.21.15.12:0/287587724 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3de8011440 con 0x56429f3ee950 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3de1ffb700 1 -- 172.21.15.12:0/287587724 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3de8011600 con 0x56429f3ee950 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3e0c6bdb80 10 client.16275 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/287587724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56429f1bec70 con 0x56429f3ee950 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.269+0000 7f3de1ffb700 1 -- 172.21.15.12:0/287587724 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de8011900 con 0x56429f3ee950 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.270+0000 7f3de1ffb700 1 -- 172.21.15.12:0/287587724 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3de8003b50 con 0x56429f3ee950 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.270+0000 7f3e0c6bdb80 10 client.16275 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.270+0000 7f3e0c6bdb80 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.270+0000 7f3e0c6bdb80 20 client.16275 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.270+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56429f459670 0x56429f45ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.270+0000 7f3e0c6bdb80 4 client.16275 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:49:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.270+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/287587724 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56429f331e60 con 0x56429f459670 2024-05-08T23:49:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.270+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56429f459670 0x56429f45ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.270+0000 7f3de1ffb700 10 client.16275 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.270+0000 7f3dfe29f700 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56429f459670 0x56429f45ba90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8215b5c0b64a5c1f server_cookie=9cc211f39f4b78df in_seq=0 out_seq=0 2024-05-08T23:49:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3de1ffb700 1 -- 172.21.15.12:0/287587724 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56429f331e60 con 0x56429f459670 2024-05-08T23:49:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3de1ffb700 10 client.16275 handle_command_reply: tid=0 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 client.16275 shutdown 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/287587724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 msgr2=0x56429f3f8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3f8190 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x56429f3b88f0 tx=0x7f3de8011ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/287587724 shutdown_connections 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56429f3ef270 0x56429f3f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56429f459670 0x56429f45ba90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56429f3e9480 0x56429f3f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 --2- 172.21.15.12:0/287587724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56429f3ee950 0x56429f3f8190 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3de1ffb700 0 client.16275 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/287587724 >> 172.21.15.12:0/287587724 conn(0x56429f3471b0 msgr2=0x56429f3f0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/287587724 shutdown_connections 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 1 -- 172.21.15.12:0/287587724 wait complete. 2024-05-08T23:49:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.295+0000 7f3e0c6bdb80 20 client.16275 trim_cache size 0 max 16384 2024-05-08T23:49:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:49:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:49:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:49:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 -- 172.21.15.12:0/775750616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3de40a1290 msgr2=0x7f3de40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3de40a1290 0x7f3de40a3750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b2e0 tx=0x7f3df4005960 comp rx=0 tx=0).stop 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 -- 172.21.15.12:0/775750616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3df8107000 msgr2=0x7f3df8100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3df8107000 0x7f3df8100890 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3df000ef30 tx=0x7f3df000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 -- 172.21.15.12:0/775750616 shutdown_connections 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3de40a1290 0x7f3de40a3750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3df8107000 0x7f3df8100890 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df80682e0 0x7f3df80ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 --2- 172.21.15.12:0/775750616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3df8068ca0 0x7f3df8100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 -- 172.21.15.12:0/775750616 >> 172.21.15.12:0/775750616 conn(0x7f3df8076f10 msgr2=0x7f3df80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 -- 172.21.15.12:0/775750616 shutdown_connections 2024-05-08T23:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:29.296+0000 7f3e00503700 1 -- 172.21.15.12:0/775750616 wait complete. 2024-05-08T23:49:29.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74202 4.90 MB/sec warmup 85 sec latency 2032.065 ms 2024-05-08T23:49:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:30 smithi204 ceph-mon[47373]: pgmap v96: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 489 KiB/s wr, 64 op/s 2024-05-08T23:49:30.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:30 smithi160 ceph-mon[46152]: pgmap v96: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 489 KiB/s wr, 64 op/s 2024-05-08T23:49:30.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74202 4.85 MB/sec warmup 86 sec latency 3032.193 ms 2024-05-08T23:49:30.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:30 smithi012 ceph-mon[42503]: pgmap v96: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 489 KiB/s wr, 64 op/s 2024-05-08T23:49:31.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:31 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:49:31.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:31 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:49:31.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74202 4.79 MB/sec warmup 87 sec latency 4032.283 ms 2024-05-08T23:49:31.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:31 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:49:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:32 smithi204 ceph-mon[47373]: pgmap v97: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 529 KiB/s wr, 83 op/s 2024-05-08T23:49:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:32 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:49:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:32 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:49:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:32 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:49:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:32 smithi160 ceph-mon[46152]: pgmap v97: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 529 KiB/s wr, 83 op/s 2024-05-08T23:49:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:32 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:49:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:32 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:49:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:32 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:49:32.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74538 4.78 MB/sec warmup 88 sec latency 4940.404 ms 2024-05-08T23:49:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:32 smithi012 ceph-mon[42503]: pgmap v97: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 529 KiB/s wr, 83 op/s 2024-05-08T23:49:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:32 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:49:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:32 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:49:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:32 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:49:33.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 75523 4.76 MB/sec warmup 89 sec latency 130.334 ms 2024-05-08T23:49:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:34 smithi204 ceph-mon[47373]: pgmap v98: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 28 op/s 2024-05-08T23:49:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:34 smithi160 ceph-mon[46152]: pgmap v98: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 28 op/s 2024-05-08T23:49:34.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 77473 4.83 MB/sec warmup 90 sec latency 304.185 ms 2024-05-08T23:49:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:34 smithi012 ceph-mon[42503]: pgmap v98: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 28 op/s 2024-05-08T23:49:35.852 INFO:tasks.workunit.client.0.smithi012.stdout: 1 77473 4.78 MB/sec warmup 91 sec latency 1304.285 ms 2024-05-08T23:49:36.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:36 smithi204 ceph-mon[47373]: pgmap v99: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 28 op/s 2024-05-08T23:49:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:36 smithi160 ceph-mon[46152]: pgmap v99: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 28 op/s 2024-05-08T23:49:36.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 77473 4.73 MB/sec warmup 92 sec latency 2304.399 ms 2024-05-08T23:49:36.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:36 smithi012 ceph-mon[42503]: pgmap v99: 97 pgs: 97 active+clean; 33 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 28 op/s 2024-05-08T23:49:37.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 77711 4.69 MB/sec warmup 93 sec latency 3217.505 ms 2024-05-08T23:49:38.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:38 smithi204 ceph-mon[47373]: pgmap v100: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 948 KiB/s wr, 93 op/s 2024-05-08T23:49:38.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:38 smithi160 ceph-mon[46152]: pgmap v100: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 948 KiB/s wr, 93 op/s 2024-05-08T23:49:38.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 79582 4.75 MB/sec warmup 94 sec latency 11.871 ms 2024-05-08T23:49:38.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:38 smithi012 ceph-mon[42503]: pgmap v100: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 948 KiB/s wr, 93 op/s 2024-05-08T23:49:39.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 80845 4.77 MB/sec warmup 95 sec latency 652.026 ms 2024-05-08T23:49:40.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:40 smithi204 ceph-mon[47373]: pgmap v101: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 897 KiB/s wr, 83 op/s 2024-05-08T23:49:40.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:40 smithi160 ceph-mon[46152]: pgmap v101: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 897 KiB/s wr, 83 op/s 2024-05-08T23:49:40.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 80845 4.72 MB/sec warmup 96 sec latency 1652.090 ms 2024-05-08T23:49:40.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:40 smithi012 ceph-mon[42503]: pgmap v101: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 897 KiB/s wr, 83 op/s 2024-05-08T23:49:41.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 80845 4.67 MB/sec warmup 97 sec latency 2652.190 ms 2024-05-08T23:49:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.422+0000 7f316dd50700 1 -- 172.21.15.12:0/946111720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168100010 msgr2=0x7f31681003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.422+0000 7f316dd50700 1 --2- 172.21.15.12:0/946111720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168100010 0x7f31681003f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3158009990 tx=0x7f3158009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.422+0000 7f316dd50700 1 -- 172.21.15.12:0/946111720 shutdown_connections 2024-05-08T23:49:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.422+0000 7f316dd50700 1 --2- 172.21.15.12:0/946111720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168105130 0x7f316810ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.422+0000 7f316dd50700 1 --2- 172.21.15.12:0/946111720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168100010 0x7f31681003f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.422+0000 7f316dd50700 1 --2- 172.21.15.12:0/946111720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31681009c0 0x7f3168104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.422+0000 7f316dd50700 1 -- 172.21.15.12:0/946111720 >> 172.21.15.12:0/946111720 conn(0x7f3168076f40 msgr2=0x7f3168077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.422+0000 7f316dd50700 1 -- 172.21.15.12:0/946111720 shutdown_connections 2024-05-08T23:49:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.422+0000 7f316dd50700 1 -- 172.21.15.12:0/946111720 wait complete. 2024-05-08T23:49:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.423+0000 7f316dd50700 1 Processor -- start 2024-05-08T23:49:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.423+0000 7f316dd50700 1 -- start start 2024-05-08T23:49:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.423+0000 7f316dd50700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168100010 0x7f3168101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f316dd50700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3168105130 0x7f3168101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f316dd50700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168102520 0x7f31681a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f316dd50700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31681a5b90 con 0x7f3168100010 2024-05-08T23:49:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f316dd50700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31681a5cd0 con 0x7f3168105130 2024-05-08T23:49:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f316dd50700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31681a5e90 con 0x7f3168102520 2024-05-08T23:49:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f31677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168100010 0x7f3168101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f31677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168100010 0x7f3168101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35054/0 (socket says 172.21.15.12:35054) 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f31677fe700 1 -- 172.21.15.12:0/574434143 learned_addr learned my addr 172.21.15.12:0/574434143 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f3166ffd700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3168105130 0x7f3168101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f3167fff700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168102520 0x7f31681a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f31677fe700 1 -- 172.21.15.12:0/574434143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168102520 msgr2=0x7f31681a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f31677fe700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168102520 0x7f31681a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f31677fe700 1 -- 172.21.15.12:0/574434143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3168105130 msgr2=0x7f3168101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f31677fe700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3168105130 0x7f3168101fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f31677fe700 1 -- 172.21.15.12:0/574434143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31580095f0 con 0x7f3168100010 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f31677fe700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168100010 0x7f3168101aa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3158004760 tx=0x7f3158004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f3164ff9700 1 -- 172.21.15.12:0/574434143 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315801d070 con 0x7f3168100010 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f3164ff9700 1 -- 172.21.15.12:0/574434143 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3158022470 con 0x7f3168100010 2024-05-08T23:49:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f3164ff9700 1 -- 172.21.15.12:0/574434143 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158017650 con 0x7f3168100010 2024-05-08T23:49:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.424+0000 7f316dd50700 1 -- 172.21.15.12:0/574434143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31681a6110 con 0x7f3168100010 2024-05-08T23:49:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.425+0000 7f316dd50700 1 -- 172.21.15.12:0/574434143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31681a64e0 con 0x7f3168100010 2024-05-08T23:49:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.427+0000 7f3164ff9700 1 -- 172.21.15.12:0/574434143 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f31580225e0 con 0x7f3168100010 2024-05-08T23:49:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.428+0000 7f3164ff9700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31540a1290 0x7f31540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.428+0000 7f3164ff9700 1 -- 172.21.15.12:0/574434143 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f31580cde60 con 0x7f3168100010 2024-05-08T23:49:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.428+0000 7f3166ffd700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31540a1290 0x7f31540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.428+0000 7f3166ffd700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31540a1290 0x7f31540a3750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3168103180 tx=0x7f3150009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.428+0000 7f3179f0ab80 1 Processor -- start 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f3179f0ab80 1 -- start start 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d910894950 0x55d910894d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910895270 0x55d91088ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d91088f480 0x55d91088f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9107e60f0 con 0x55d910894950 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9107e3ab0 con 0x55d91088f480 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9107df3a0 con 0x55d910895270 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f31677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d910894950 0x55d910894d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f31677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d910894950 0x55d910894d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35058/0 (socket says 172.21.15.12:35058) 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f31677fe700 1 -- 172.21.15.12:0/3619336965 learned_addr learned my addr 172.21.15.12:0/3619336965 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f3166ffd700 1 --2- 172.21.15.12:0/3619336965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910895270 0x55d91088ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f3167fff700 1 --2- 172.21.15.12:0/3619336965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d91088f480 0x55d91088f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f31677fe700 1 -- 172.21.15.12:0/3619336965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910895270 msgr2=0x55d91088ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f31677fe700 1 --2- 172.21.15.12:0/3619336965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910895270 0x55d91088ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f31677fe700 1 -- 172.21.15.12:0/3619336965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d91088f480 msgr2=0x55d91088f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f31677fe700 1 --2- 172.21.15.12:0/3619336965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d91088f480 0x55d91088f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f31677fe700 1 -- 172.21.15.12:0/3619336965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d910660350 con 0x55d910894950 2024-05-08T23:49:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.429+0000 7f31677fe700 1 --2- 172.21.15.12:0/3619336965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d910894950 0x55d910894d30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f31580a2f00 tx=0x7f31580974a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9619840ef1f63ed8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.430+0000 7f315dffb700 1 -- 172.21.15.12:0/3619336965 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315809b740 con 0x55d910894950 2024-05-08T23:49:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.430+0000 7f315dffb700 1 -- 172.21.15.12:0/3619336965 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f315809b040 con 0x55d910894950 2024-05-08T23:49:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.430+0000 7f315dffb700 1 -- 172.21.15.12:0/3619336965 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315809b2a0 con 0x55d910894950 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.430+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3619336965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d910894950 msgr2=0x55d910894d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.430+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/3619336965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d910894950 0x55d910894d30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f31580a2f00 tx=0x7f31580974a0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.431+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3619336965 shutdown_connections 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.431+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/3619336965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d91088f480 0x55d91088f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.431+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/3619336965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d910894950 0x55d910894d30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.431+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/3619336965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910895270 0x55d91088ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.431+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3619336965 >> 172.21.15.12:0/3619336965 conn(0x55d9107ed1b0 msgr2=0x55d910889840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.431+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3619336965 shutdown_connections 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.431+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3619336965 wait complete. 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.431+0000 7f3179f0ab80 1 -- start start 2024-05-08T23:49:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.431+0000 7f3179f0ab80 1 Processor -- start 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d91089b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910894950 0x55d910898470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910895270 0x55d910898bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d91077b500 con 0x55d91088f480 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d91077b1a0 con 0x55d910895270 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d910662640 con 0x55d910894950 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3179f0ab80 10 client.0 _ll_register_callbacks cb 0x55d910784ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f31677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d91089b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f31677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d91089b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35070/0 (socket says 172.21.15.12:35070) 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f31677fe700 1 -- 172.21.15.12:0/2801183551 learned_addr learned my addr 172.21.15.12:0/2801183551 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3166ffd700 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910894950 0x55d910898470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3167fff700 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910895270 0x55d910898bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f31677fe700 1 -- 172.21.15.12:0/2801183551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910894950 msgr2=0x55d910898470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f31677fe700 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910894950 0x55d910898470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f31677fe700 1 -- 172.21.15.12:0/2801183551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910895270 msgr2=0x55d910898bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f31677fe700 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910895270 0x55d910898bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f31677fe700 1 -- 172.21.15.12:0/2801183551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31681a64e0 con 0x55d91088f480 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f31677fe700 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d91089b1f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3158005f50 tx=0x7f31580094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5467eaf39345e210 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3146ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.432+0000 7f3146ffd700 1 -- 172.21.15.12:0/2801183551 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315809b740 con 0x55d91088f480 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3146ffd700 1 -- 172.21.15.12:0/2801183551 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f315809ee70 con 0x55d91088f480 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3179f0ab80 1 -- 172.21.15.12:0/2801183551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d910738210 con 0x55d91088f480 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3146ffd700 1 -- 172.21.15.12:0/2801183551 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31580b92c0 con 0x55d91088f480 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3179f0ab80 10 client.16287 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3179f0ab80 1 -- 172.21.15.12:0/2801183551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d91060cb00 con 0x55d91088f480 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3146ffd700 1 -- 172.21.15.12:0/2801183551 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d910738210 con 0x55d91088f480 2024-05-08T23:49:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3146ffd700 1 -- 172.21.15.12:0/2801183551 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31580a3030 con 0x55d91088f480 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3179f0ab80 10 client.16287 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3179f0ab80 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3179f0ab80 20 client.16287 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d9108a8510 0x55d9108aa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3179f0ab80 4 client.16287 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.433+0000 7f3179f0ab80 1 -- 172.21.15.12:0/2801183551 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d91085e7b0 con 0x55d9108a8510 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.434+0000 7f3166ffd700 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d9108a8510 0x55d9108aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.434+0000 7f3146ffd700 10 client.16287 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.434+0000 7f3166ffd700 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d9108a8510 0x55d9108aa930 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc484c86ac12a678 server_cookie=a2f2836533595e87 in_seq=0 out_seq=0 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3146ffd700 1 -- 172.21.15.12:0/2801183551 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d91085e7b0 con 0x55d9108a8510 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3146ffd700 10 client.16287 handle_command_reply: tid=0 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 client.16287 shutdown 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 -- 172.21.15.12:0/2801183551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 msgr2=0x55d91089b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d91089b1f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3158005f50 tx=0x7f31580094b0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 -- 172.21.15.12:0/2801183551 shutdown_connections 2024-05-08T23:49:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d9108a8510 0x55d9108aa930 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910895270 0x55d910898bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d91089b1f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/2801183551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910894950 0x55d910898470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3146ffd700 0 client.16287 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 -- 172.21.15.12:0/2801183551 >> 172.21.15.12:0/2801183551 conn(0x55d9107ed1b0 msgr2=0x55d91088df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 -- 172.21.15.12:0/2801183551 shutdown_connections 2024-05-08T23:49:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 1 -- 172.21.15.12:0/2801183551 wait complete. 2024-05-08T23:49:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.435+0000 7f3179f0ab80 20 client.16287 trim_cache size 0 max 16384 2024-05-08T23:49:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.451+0000 7f3179f0ab80 1 Processor -- start 2024-05-08T23:49:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.451+0000 7f3179f0ab80 1 -- start start 2024-05-08T23:49:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.451+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d910896660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.451+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910894950 0x55d91088e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.451+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910895270 0x55d91089dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.451+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9108afb00 con 0x55d91088f480 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.451+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d91088a580 con 0x55d910895270 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.451+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d910893100 con 0x55d910894950 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f31677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d910896660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f31677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d910896660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35074/0 (socket says 172.21.15.12:35074) 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f31677fe700 1 -- 172.21.15.12:0/529789507 learned_addr learned my addr 172.21.15.12:0/529789507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f3167fff700 1 --2- 172.21.15.12:0/529789507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910895270 0x55d91089dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f3166ffd700 1 --2- 172.21.15.12:0/529789507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910894950 0x55d91088e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f31677fe700 1 -- 172.21.15.12:0/529789507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910894950 msgr2=0x55d91088e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f31677fe700 1 --2- 172.21.15.12:0/529789507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910894950 0x55d91088e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f31677fe700 1 -- 172.21.15.12:0/529789507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910895270 msgr2=0x55d91089dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f31677fe700 1 --2- 172.21.15.12:0/529789507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910895270 0x55d91089dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f31677fe700 1 -- 172.21.15.12:0/529789507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9107a5500 con 0x55d91088f480 2024-05-08T23:49:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.452+0000 7f31677fe700 1 --2- 172.21.15.12:0/529789507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d910896660 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3158005cb0 tx=0x7f3158022cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d80df54f9896d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.453+0000 7f315e7fc700 1 -- 172.21.15.12:0/529789507 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315809b740 con 0x55d91088f480 2024-05-08T23:49:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.453+0000 7f315e7fc700 1 -- 172.21.15.12:0/529789507 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f315809ee70 con 0x55d91088f480 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.453+0000 7f315e7fc700 1 -- 172.21.15.12:0/529789507 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315809b500 con 0x55d91088f480 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 -- 172.21.15.12:0/529789507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 msgr2=0x55d910896660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/529789507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d910896660 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3158005cb0 tx=0x7f3158022cb0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 -- 172.21.15.12:0/529789507 shutdown_connections 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/529789507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910895270 0x55d91089dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/529789507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d910896660 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/529789507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910894950 0x55d91088e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 -- 172.21.15.12:0/529789507 >> 172.21.15.12:0/529789507 conn(0x55d9107ed1b0 msgr2=0x55d910885e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 -- 172.21.15.12:0/529789507 shutdown_connections 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 -- 172.21.15.12:0/529789507 wait complete. 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 -- start start 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.454+0000 7f3179f0ab80 1 Processor -- start 2024-05-08T23:49:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d9108f2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910894950 0x55d9108f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3179f0ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910895270 0x55d9108f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9108fdb10 con 0x55d91088f480 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9108fdc50 con 0x55d910894950 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3179f0ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9108fde10 con 0x55d910895270 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3179f0ab80 10 client.0 _ll_register_callbacks cb 0x55d910784ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f31677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d9108f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f31677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d9108f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35076/0 (socket says 172.21.15.12:35076) 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f31677fe700 1 -- 172.21.15.12:0/3861611505 learned_addr learned my addr 172.21.15.12:0/3861611505 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3167fff700 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910895270 0x55d9108f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3166ffd700 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910894950 0x55d9108f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f31677fe700 1 -- 172.21.15.12:0/3861611505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910895270 msgr2=0x55d9108f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f31677fe700 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910895270 0x55d9108f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f31677fe700 1 -- 172.21.15.12:0/3861611505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910894950 msgr2=0x55d9108f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f31677fe700 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910894950 0x55d9108f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f31677fe700 1 -- 172.21.15.12:0/3861611505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d91060cb00 con 0x55d91088f480 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f31677fe700 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d9108f2960 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f315809a040 tx=0x7f315800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7540f19e20922b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3146ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3146ffd700 1 -- 172.21.15.12:0/3861611505 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315809b4a0 con 0x55d91088f480 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3146ffd700 1 -- 172.21.15.12:0/3861611505 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f31580a3430 con 0x55d91088f480 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.455+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3861611505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f315809b040 con 0x55d91088f480 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3146ffd700 1 -- 172.21.15.12:0/3861611505 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f315809b740 con 0x55d91088f480 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3179f0ab80 10 client.16293 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3861611505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d910664c70 con 0x55d91088f480 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3146ffd700 1 -- 172.21.15.12:0/3861611505 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f315809b040 con 0x55d91088f480 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3146ffd700 1 -- 172.21.15.12:0/3861611505 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f315800fb90 con 0x55d91088f480 2024-05-08T23:49:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3179f0ab80 10 client.16293 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3179f0ab80 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3179f0ab80 20 client.16293 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d9108ff0f0 0x55d9109014e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3179f0ab80 4 client.16293 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:49:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3861611505 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d9107d7e60 con 0x55d9108ff0f0 2024-05-08T23:49:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.456+0000 7f3166ffd700 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d9108ff0f0 0x55d9109014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.457+0000 7f3146ffd700 10 client.16293 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.457+0000 7f3166ffd700 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d9108ff0f0 0x55d9109014e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f380d2a2cf0a1807 server_cookie=5b0aa54fc3ad1d1c in_seq=0 out_seq=0 2024-05-08T23:49:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.488+0000 7f3146ffd700 1 -- 172.21.15.12:0/3861611505 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d9107d7e60 con 0x55d9108ff0f0 2024-05-08T23:49:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.488+0000 7f3146ffd700 10 client.16293 handle_command_reply: tid=0 2024-05-08T23:49:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.488+0000 7f3179f0ab80 1 client.16293 shutdown 2024-05-08T23:49:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3861611505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 msgr2=0x55d9108f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d9108f2960 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f315809a040 tx=0x7f315800bf30 comp rx=0 tx=0).stop 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3861611505 shutdown_connections 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d910895270 0x55d9108f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d91088f480 0x55d9108f2960 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d9108ff0f0 0x55d9109014e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 1 --2- 172.21.15.12:0/3861611505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d910894950 0x55d9108f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3861611505 >> 172.21.15.12:0/3861611505 conn(0x55d9107ed1b0 msgr2=0x55d9108923f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3146ffd700 0 client.16293 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3861611505 shutdown_connections 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 1 -- 172.21.15.12:0/3861611505 wait complete. 2024-05-08T23:49:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.489+0000 7f3179f0ab80 20 client.16293 trim_cache size 0 max 16384 2024-05-08T23:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 -- 172.21.15.12:0/574434143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31540a1290 msgr2=0x7f31540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31540a1290 0x7f31540a3750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3168103180 tx=0x7f3150009260 comp rx=0 tx=0).stop 2024-05-08T23:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 -- 172.21.15.12:0/574434143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168100010 msgr2=0x7f3168101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168100010 0x7f3168101aa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3158004760 tx=0x7f3158004790 comp rx=0 tx=0).stop 2024-05-08T23:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 -- 172.21.15.12:0/574434143 shutdown_connections 2024-05-08T23:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31540a1290 0x7f31540a3750 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3168103180 tx=0x7f3150009260 comp rx=0 tx=0).stop 2024-05-08T23:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3168102520 0x7f31681a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168100010 0x7f3168101aa0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 --2- 172.21.15.12:0/574434143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3168105130 0x7f3168101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 -- 172.21.15.12:0/574434143 >> 172.21.15.12:0/574434143 conn(0x7f3168076f40 msgr2=0x7f3168069570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 -- 172.21.15.12:0/574434143 shutdown_connections 2024-05-08T23:49:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:42.490+0000 7f316dd50700 1 -- 172.21.15.12:0/574434143 wait complete. 2024-05-08T23:49:42.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:42 smithi204 ceph-mon[47373]: pgmap v102: 97 pgs: 97 active+clean; 36 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-05-08T23:49:42.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:42 smithi160 ceph-mon[46152]: pgmap v102: 97 pgs: 97 active+clean; 36 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-05-08T23:49:42.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 81014 4.63 MB/sec warmup 98 sec latency 3555.260 ms 2024-05-08T23:49:42.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:42 smithi012 ceph-mon[42503]: pgmap v102: 97 pgs: 97 active+clean; 36 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-05-08T23:49:43.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 81014 4.59 MB/sec warmup 99 sec latency 1017.286 ms 2024-05-08T23:49:44.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:44 smithi204 ceph-mon[47373]: pgmap v103: 97 pgs: 97 active+clean; 36 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-05-08T23:49:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:44 smithi160 ceph-mon[46152]: pgmap v103: 97 pgs: 97 active+clean; 36 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-05-08T23:49:44.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 81014 4.54 MB/sec warmup 100 sec latency 2017.350 ms 2024-05-08T23:49:44.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:44 smithi012 ceph-mon[42503]: pgmap v103: 97 pgs: 97 active+clean; 36 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-05-08T23:49:45.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 81014 4.50 MB/sec warmup 101 sec latency 3017.427 ms 2024-05-08T23:49:46.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:46 smithi204 ceph-mon[47373]: pgmap v104: 97 pgs: 97 active+clean; 36 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-05-08T23:49:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:46 smithi160 ceph-mon[46152]: pgmap v104: 97 pgs: 97 active+clean; 36 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-05-08T23:49:46.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 81014 4.45 MB/sec warmup 102 sec latency 4017.495 ms 2024-05-08T23:49:46.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:46 smithi012 ceph-mon[42503]: pgmap v104: 97 pgs: 97 active+clean; 36 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-05-08T23:49:47.754 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:49:47.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 81108 4.41 MB/sec warmup 103 sec latency 4920.990 ms 2024-05-08T23:49:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.908+0000 7f02f9cef700 1 -- 172.21.15.12:0/2115275048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f40ff410 msgr2=0x7f02f40ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.908+0000 7f02f9cef700 1 --2- 172.21.15.12:0/2115275048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f40ff410 0x7f02f40ff890 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009990 tx=0x7f02e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.908+0000 7f02f9cef700 1 -- 172.21.15.12:0/2115275048 shutdown_connections 2024-05-08T23:49:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.908+0000 7f02f9cef700 1 --2- 172.21.15.12:0/2115275048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02f40ffdd0 0x7f02f410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.908+0000 7f02f9cef700 1 --2- 172.21.15.12:0/2115275048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f4105ef0 0x7f02f41062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.908+0000 7f02f9cef700 1 --2- 172.21.15.12:0/2115275048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f40ff410 0x7f02f40ff890 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.908+0000 7f02f9cef700 1 -- 172.21.15.12:0/2115275048 >> 172.21.15.12:0/2115275048 conn(0x7f02f40763a0 msgr2=0x7f02f40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.909+0000 7f02f9cef700 1 -- 172.21.15.12:0/2115275048 shutdown_connections 2024-05-08T23:49:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.909+0000 7f02f9cef700 1 -- 172.21.15.12:0/2115275048 wait complete. 2024-05-08T23:49:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.909+0000 7f02f9cef700 1 Processor -- start 2024-05-08T23:49:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.909+0000 7f02f9cef700 1 -- start start 2024-05-08T23:49:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f9cef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f40ff410 0x7f02f41005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f9cef700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02f40ffdd0 0x7f02f4100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f9cef700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f4105ef0 0x7f02f4101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f9cef700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f41a5890 con 0x7f02f40ff410 2024-05-08T23:49:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f9cef700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f41a5a00 con 0x7f02f40ffdd0 2024-05-08T23:49:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f9cef700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f41a5bc0 con 0x7f02f4105ef0 2024-05-08T23:49:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f40ff410 0x7f02f41005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f40ff410 0x7f02f41005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35094/0 (socket says 172.21.15.12:35094) 2024-05-08T23:49:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f37fe700 1 -- 172.21.15.12:0/1425808315 learned_addr learned my addr 172.21.15.12:0/1425808315 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f2ffd700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02f40ffdd0 0x7f02f4100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f3fff700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f4105ef0 0x7f02f4101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f37fe700 1 -- 172.21.15.12:0/1425808315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f4105ef0 msgr2=0x7f02f4101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f37fe700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f4105ef0 0x7f02f4101090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f37fe700 1 -- 172.21.15.12:0/1425808315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02f40ffdd0 msgr2=0x7f02f4100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f37fe700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02f40ffdd0 0x7f02f4100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f37fe700 1 -- 172.21.15.12:0/1425808315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec009c40 con 0x7f02f40ff410 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f3fff700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f4105ef0 0x7f02f4101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.910+0000 7f02f37fe700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f40ff410 0x7f02f41005f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9a0 tx=0x7f02e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.911+0000 7f02f0ff9700 1 -- 172.21.15.12:0/1425808315 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4009790 con 0x7f02f40ff410 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.911+0000 7f02f9cef700 1 -- 172.21.15.12:0/1425808315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e80095f0 con 0x7f02f40ff410 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.911+0000 7f02f0ff9700 1 -- 172.21.15.12:0/1425808315 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e400cbd0 con 0x7f02f40ff410 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.911+0000 7f02f0ff9700 1 -- 172.21.15.12:0/1425808315 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4018680 con 0x7f02f40ff410 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.911+0000 7f02f9cef700 1 -- 172.21.15.12:0/1425808315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f41a61e0 con 0x7f02f40ff410 2024-05-08T23:49:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.913+0000 7f02f0ff9700 1 -- 172.21.15.12:0/1425808315 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f02e400cd40 con 0x7f02f40ff410 2024-05-08T23:49:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.913+0000 7f02f0ff9700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02dc0a1290 0x7f02dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.914+0000 7f02f0ff9700 1 -- 172.21.15.12:0/1425808315 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f02e4014070 con 0x7f02f40ff410 2024-05-08T23:49:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.914+0000 7f02f2ffd700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02dc0a1290 0x7f02dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.914+0000 7f02f2ffd700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02dc0a1290 0x7f02dc0a3750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02e800b2e0 tx=0x7f02e8005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f0305ea9b80 1 Processor -- start 2024-05-08T23:49:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f0305ea9b80 1 -- start start 2024-05-08T23:49:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 0x561b7b3f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3f0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f0890 0x561b7b3f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b347150 con 0x561b7b3f6720 2024-05-08T23:49:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b344b10 con 0x561b7b3f5e00 2024-05-08T23:49:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b340400 con 0x561b7b3f0890 2024-05-08T23:49:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f02f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f02f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3f0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35106/0 (socket says 172.21.15.12:35106) 2024-05-08T23:49:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.915+0000 7f02f2ffd700 1 -- 172.21.15.12:0/2720833092 learned_addr learned my addr 172.21.15.12:0/2720833092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02f2ffd700 1 -- 172.21.15.12:0/2720833092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f0890 msgr2=0x561b7b3f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02f2ffd700 1 --2- 172.21.15.12:0/2720833092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f0890 0x561b7b3f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02f37fe700 1 --2- 172.21.15.12:0/2720833092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 0x561b7b3f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02f2ffd700 1 -- 172.21.15.12:0/2720833092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 msgr2=0x561b7b3f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02f2ffd700 1 --2- 172.21.15.12:0/2720833092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 0x561b7b3f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02f2ffd700 1 -- 172.21.15.12:0/2720833092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b7b1c13b0 con 0x561b7b3f6720 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02f37fe700 1 --2- 172.21.15.12:0/2720833092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 0x561b7b3f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02f2ffd700 1 --2- 172.21.15.12:0/2720833092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3f0350 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009990 tx=0x7f02e8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3e78de2beca1b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02d9ffb700 1 -- 172.21.15.12:0/2720833092 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8004500 con 0x561b7b3f6720 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02d9ffb700 1 -- 172.21.15.12:0/2720833092 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e801e040 con 0x561b7b3f6720 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.916+0000 7f02d9ffb700 1 -- 172.21.15.12:0/2720833092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800f920 con 0x561b7b3f6720 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.917+0000 7f0305ea9b80 1 -- 172.21.15.12:0/2720833092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 msgr2=0x561b7b3f0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:47.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.917+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/2720833092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3f0350 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009990 tx=0x7f02e8020000 comp rx=0 tx=0).stop 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.917+0000 7f0305ea9b80 1 -- 172.21.15.12:0/2720833092 shutdown_connections 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.917+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/2720833092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f0890 0x561b7b3f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.917+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/2720833092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 0x561b7b3f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.917+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/2720833092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3f0350 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.917+0000 7f0305ea9b80 1 -- 172.21.15.12:0/2720833092 >> 172.21.15.12:0/2720833092 conn(0x561b7b34e210 msgr2=0x561b7b3eadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.917+0000 7f0305ea9b80 1 -- 172.21.15.12:0/2720833092 shutdown_connections 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.917+0000 7f0305ea9b80 1 -- 172.21.15.12:0/2720833092 wait complete. 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.917+0000 7f0305ea9b80 1 -- start start 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f0305ea9b80 1 Processor -- start 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f5e00 0x561b7b3fa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 0x561b7b3fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b2dc560 con 0x561b7b3f0890 2024-05-08T23:49:47.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b2dc200 con 0x561b7b3f6720 2024-05-08T23:49:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b1c36a0 con 0x561b7b3f5e00 2024-05-08T23:49:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f0305ea9b80 10 client.0 _ll_register_callbacks cb 0x561b7b2e5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f02f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f02f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f5e00 0x561b7b3fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f02f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3f9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35116/0 (socket says 172.21.15.12:35116) 2024-05-08T23:49:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f02f3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 0x561b7b3fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.918+0000 7f02f37fe700 1 -- 172.21.15.12:0/160472665 learned_addr learned my addr 172.21.15.12:0/160472665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02f3fff700 1 -- 172.21.15.12:0/160472665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f5e00 msgr2=0x561b7b3fa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02f3fff700 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f5e00 0x561b7b3fa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02f3fff700 1 -- 172.21.15.12:0/160472665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 msgr2=0x561b7b3f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02f3fff700 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02f3fff700 1 -- 172.21.15.12:0/160472665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f410e750 con 0x561b7b3f6720 2024-05-08T23:49:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02f37fe700 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02f3fff700 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 0x561b7b3fa670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02f41015d0 tx=0x7f02ec007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105f5abb06fe3d80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:49:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02d2ffd700 1 -- 172.21.15.12:0/160472665 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec00e070 con 0x561b7b3f6720 2024-05-08T23:49:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02d2ffd700 1 -- 172.21.15.12:0/160472665 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f02ec003a00 con 0x561b7b3f6720 2024-05-08T23:49:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f0305ea9b80 1 -- 172.21.15.12:0/160472665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b7b299270 con 0x561b7b3f6720 2024-05-08T23:49:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f02d2ffd700 1 -- 172.21.15.12:0/160472665 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02ec003bc0 con 0x561b7b3f6720 2024-05-08T23:49:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f0305ea9b80 10 client.26123 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.919+0000 7f0305ea9b80 1 -- 172.21.15.12:0/160472665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561b7b16db60 con 0x561b7b3f6720 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.920+0000 7f02d2ffd700 1 -- 172.21.15.12:0/160472665 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b7b299270 con 0x561b7b3f6720 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.920+0000 7f02d2ffd700 1 -- 172.21.15.12:0/160472665 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02ec004380 con 0x561b7b3f6720 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.920+0000 7f0305ea9b80 10 client.26123 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.920+0000 7f0305ea9b80 10 client.26123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.920+0000 7f0305ea9b80 20 client.26123 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.920+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b7b40a2f0 0x561b7b40c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.920+0000 7f0305ea9b80 4 client.26123 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.920+0000 7f0305ea9b80 1 -- 172.21.15.12:0/160472665 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b7b3bf810 con 0x561b7b40a2f0 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.920+0000 7f02f37fe700 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b7b40a2f0 0x561b7b40c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.921+0000 7f02f37fe700 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b7b40a2f0 0x561b7b40c710 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697e175784663a3f server_cookie=74c8960d66c60870 in_seq=0 out_seq=0 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.921+0000 7f02d2ffd700 10 client.26123 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.921+0000 7f02d2ffd700 1 -- 172.21.15.12:0/160472665 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b7b3bf810 con 0x561b7b40a2f0 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.921+0000 7f02d2ffd700 10 client.26123 handle_command_reply: tid=0 2024-05-08T23:49:47.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.921+0000 7f0305ea9b80 1 client.26123 shutdown 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 1 -- 172.21.15.12:0/160472665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 msgr2=0x561b7b3fa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 0x561b7b3fa670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02f41015d0 tx=0x7f02ec007e40 comp rx=0 tx=0).stop 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 1 -- 172.21.15.12:0/160472665 shutdown_connections 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b7b40a2f0 0x561b7b40c710 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 0x561b7b3fa670 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/160472665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f5e00 0x561b7b3fa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f02d2ffd700 0 client.26123 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 1 -- 172.21.15.12:0/160472665 >> 172.21.15.12:0/160472665 conn(0x561b7b34e210 msgr2=0x561b7b3ea5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 1 -- 172.21.15.12:0/160472665 shutdown_connections 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 1 -- 172.21.15.12:0/160472665 wait complete. 2024-05-08T23:49:47.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.922+0000 7f0305ea9b80 20 client.26123 trim_cache size 0 max 16384 2024-05-08T23:49:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f0305ea9b80 1 Processor -- start 2024-05-08T23:49:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f0305ea9b80 1 -- start start 2024-05-08T23:49:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f0890 0x561b7b3f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 0x561b7b405bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b3ece50 con 0x561b7b3f6720 2024-05-08T23:49:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b3faf40 con 0x561b7b3f5e00 2024-05-08T23:49:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b411900 con 0x561b7b3f0890 2024-05-08T23:49:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f02f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f02f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3ead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35120/0 (socket says 172.21.15.12:35120) 2024-05-08T23:49:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f02f3fff700 1 -- 172.21.15.12:0/2592417992 learned_addr learned my addr 172.21.15.12:0/2592417992 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f02f2ffd700 1 --2- 172.21.15.12:0/2592417992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 0x561b7b405bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f02f37fe700 1 --2- 172.21.15.12:0/2592417992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f0890 0x561b7b3f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f02f3fff700 1 -- 172.21.15.12:0/2592417992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f0890 msgr2=0x561b7b3f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f02f3fff700 1 --2- 172.21.15.12:0/2592417992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f0890 0x561b7b3f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f02f3fff700 1 -- 172.21.15.12:0/2592417992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 msgr2=0x561b7b405bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f02f3fff700 1 --2- 172.21.15.12:0/2592417992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 0x561b7b405bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.940+0000 7f02f3fff700 1 -- 172.21.15.12:0/2592417992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b7b306560 con 0x561b7b3f6720 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f02f3fff700 1 --2- 172.21.15.12:0/2592417992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3ead20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009c00 tx=0x7f02ec004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bd49c2317953d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f02da7fc700 1 -- 172.21.15.12:0/2592417992 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec00e040 con 0x561b7b3f6720 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f02da7fc700 1 -- 172.21.15.12:0/2592417992 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02ec00f040 con 0x561b7b3f6720 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f02da7fc700 1 -- 172.21.15.12:0/2592417992 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec0158e0 con 0x561b7b3f6720 2024-05-08T23:49:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f0305ea9b80 1 -- 172.21.15.12:0/2592417992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 msgr2=0x561b7b3ead20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/2592417992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3ead20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009c00 tx=0x7f02ec004900 comp rx=0 tx=0).stop 2024-05-08T23:49:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f0305ea9b80 1 -- 172.21.15.12:0/2592417992 shutdown_connections 2024-05-08T23:49:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/2592417992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f6720 0x561b7b3ead20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/2592417992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f0890 0x561b7b3f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/2592417992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f5e00 0x561b7b405bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.941+0000 7f0305ea9b80 1 -- 172.21.15.12:0/2592417992 >> 172.21.15.12:0/2592417992 conn(0x561b7b34e210 msgr2=0x561b7b3f84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 1 -- 172.21.15.12:0/2592417992 shutdown_connections 2024-05-08T23:49:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 1 -- 172.21.15.12:0/2592417992 wait complete. 2024-05-08T23:49:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 1 -- start start 2024-05-08T23:49:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 1 Processor -- start 2024-05-08T23:49:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f5e00 0x561b7b3ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 0x561b7b45f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b3ffc90 con 0x561b7b3f0890 2024-05-08T23:49:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b3ffe00 con 0x561b7b3f6720 2024-05-08T23:49:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7b45fd40 con 0x561b7b3f5e00 2024-05-08T23:49:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.942+0000 7f0305ea9b80 10 client.0 _ll_register_callbacks cb 0x561b7b2e5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3fefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35134/0 (socket says 172.21.15.12:35134) 2024-05-08T23:49:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f37fe700 1 -- 172.21.15.12:0/3956835491 learned_addr learned my addr 172.21.15.12:0/3956835491 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f3fff700 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 0x561b7b45f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f2ffd700 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f5e00 0x561b7b3ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f37fe700 1 -- 172.21.15.12:0/3956835491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f5e00 msgr2=0x561b7b3ff4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f37fe700 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f5e00 0x561b7b3ff4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f37fe700 1 -- 172.21.15.12:0/3956835491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 msgr2=0x561b7b45f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f37fe700 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 0x561b7b45f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f37fe700 1 -- 172.21.15.12:0/3956835491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b7b16db60 con 0x561b7b3f0890 2024-05-08T23:49:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02f37fe700 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3fefa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f02e4096500 tx=0x7f02e4099960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459a1da1fbb7c4d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02d2ffd700 1 -- 172.21.15.12:0/3956835491 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e40996f0 con 0x561b7b3f0890 2024-05-08T23:49:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02d2ffd700 1 -- 172.21.15.12:0/3956835491 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f02e409de30 con 0x561b7b3f0890 2024-05-08T23:49:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f0305ea9b80 1 -- 172.21.15.12:0/3956835491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02e801e040 con 0x561b7b3f0890 2024-05-08T23:49:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f02d2ffd700 1 -- 172.21.15.12:0/3956835491 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02e40a2480 con 0x561b7b3f0890 2024-05-08T23:49:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f0305ea9b80 10 client.16311 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.943+0000 7f0305ea9b80 1 -- 172.21.15.12:0/3956835491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561b7b1c5cd0 con 0x561b7b3f0890 2024-05-08T23:49:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.944+0000 7f02d2ffd700 1 -- 172.21.15.12:0/3956835491 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e801e040 con 0x561b7b3f0890 2024-05-08T23:49:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.944+0000 7f02d2ffd700 1 -- 172.21.15.12:0/3956835491 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02e4098f60 con 0x561b7b3f0890 2024-05-08T23:49:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.944+0000 7f0305ea9b80 10 client.16311 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.944+0000 7f0305ea9b80 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.944+0000 7f0305ea9b80 20 client.16311 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.944+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b7b461500 0x561b7b463920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.944+0000 7f0305ea9b80 4 client.16311 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:49:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.944+0000 7f0305ea9b80 1 -- 172.21.15.12:0/3956835491 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561b7b338ec0 con 0x561b7b461500 2024-05-08T23:49:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.944+0000 7f02f2ffd700 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b7b461500 0x561b7b463920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f02d2ffd700 10 client.16311 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f02f2ffd700 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b7b461500 0x561b7b463920 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b8318cc89962e7c server_cookie=a087f891fb830069 in_seq=0 out_seq=0 2024-05-08T23:49:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f02d2ffd700 1 -- 172.21.15.12:0/3956835491 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561b7b338ec0 con 0x561b7b461500 2024-05-08T23:49:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f02d2ffd700 10 client.16311 handle_command_reply: tid=0 2024-05-08T23:49:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 client.16311 shutdown 2024-05-08T23:49:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 -- 172.21.15.12:0/3956835491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 msgr2=0x561b7b3fefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3fefa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f02e4096500 tx=0x7f02e4099960 comp rx=0 tx=0).stop 2024-05-08T23:49:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 -- 172.21.15.12:0/3956835491 shutdown_connections 2024-05-08T23:49:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b7b461500 0x561b7b463920 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b7b3f6720 0x561b7b45f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b7b3f0890 0x561b7b3fefa0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f02d2ffd700 0 client.16311 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:47.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 --2- 172.21.15.12:0/3956835491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b7b3f5e00 0x561b7b3ff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 -- 172.21.15.12:0/3956835491 >> 172.21.15.12:0/3956835491 conn(0x561b7b34e210 msgr2=0x561b7b3f53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:47.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 -- 172.21.15.12:0/3956835491 shutdown_connections 2024-05-08T23:49:47.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 1 -- 172.21.15.12:0/3956835491 wait complete. 2024-05-08T23:49:47.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.945+0000 7f0305ea9b80 20 client.16311 trim_cache size 0 max 16384 2024-05-08T23:49:47.951 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:49:47.951 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:49:47.951 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:49:47.952 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:49:47.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 -- 172.21.15.12:0/1425808315 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02dc0a1290 msgr2=0x7f02dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:47.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02dc0a1290 0x7f02dc0a3750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02e800b2e0 tx=0x7f02e8005960 comp rx=0 tx=0).stop 2024-05-08T23:49:47.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 -- 172.21.15.12:0/1425808315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f40ff410 msgr2=0x7f02f41005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f40ff410 0x7f02f41005f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9a0 tx=0x7f02e400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 -- 172.21.15.12:0/1425808315 shutdown_connections 2024-05-08T23:49:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02dc0a1290 0x7f02dc0a3750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02f4105ef0 0x7f02f4101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f40ff410 0x7f02f41005f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 --2- 172.21.15.12:0/1425808315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02f40ffdd0 0x7f02f4100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 -- 172.21.15.12:0/1425808315 >> 172.21.15.12:0/1425808315 conn(0x7f02f40763a0 msgr2=0x7f02f4104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 -- 172.21.15.12:0/1425808315 shutdown_connections 2024-05-08T23:49:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:47.946+0000 7f02f9cef700 1 -- 172.21.15.12:0/1425808315 wait complete. 2024-05-08T23:49:47.956 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:49:47.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:49:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.096+0000 7ff5489a2700 1 -- 172.21.15.12:0/95450964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5440fff40 msgr2=0x7ff544100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.096+0000 7ff5489a2700 1 --2- 172.21.15.12:0/95450964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5440fff40 0x7ff544100320 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff534009990 tx=0x7ff534009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.096+0000 7ff5489a2700 1 -- 172.21.15.12:0/95450964 shutdown_connections 2024-05-08T23:49:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.096+0000 7ff5489a2700 1 --2- 172.21.15.12:0/95450964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff544105060 0x7ff54410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.096+0000 7ff5489a2700 1 --2- 172.21.15.12:0/95450964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5440fff40 0x7ff544100320 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.096+0000 7ff5489a2700 1 --2- 172.21.15.12:0/95450964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5441008f0 0x7ff544104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.096+0000 7ff5489a2700 1 -- 172.21.15.12:0/95450964 >> 172.21.15.12:0/95450964 conn(0x7ff5440fb7e0 msgr2=0x7ff5440fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.096+0000 7ff5489a2700 1 -- 172.21.15.12:0/95450964 shutdown_connections 2024-05-08T23:49:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.096+0000 7ff5489a2700 1 -- 172.21.15.12:0/95450964 wait complete. 2024-05-08T23:49:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff5489a2700 1 Processor -- start 2024-05-08T23:49:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff5489a2700 1 -- start start 2024-05-08T23:49:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff5489a2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5440fff40 0x7ff5441019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff5489a2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff544105060 0x7ff544101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff5489a2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff544102450 0x7ff5441a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff5489a2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5441a5b50 con 0x7ff5440fff40 2024-05-08T23:49:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff5489a2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5441a5c90 con 0x7ff544102450 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff5489a2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5441a5e50 con 0x7ff544105060 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff54259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5440fff40 0x7ff5441019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff54259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5440fff40 0x7ff5441019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35156/0 (socket says 172.21.15.12:35156) 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff54259c700 1 -- 172.21.15.12:0/3401994923 learned_addr learned my addr 172.21.15.12:0/3401994923 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff542d9d700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff544102450 0x7ff5441a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff541d9b700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff544105060 0x7ff544101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff54259c700 1 -- 172.21.15.12:0/3401994923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff544105060 msgr2=0x7ff544101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff54259c700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff544105060 0x7ff544101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff54259c700 1 -- 172.21.15.12:0/3401994923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff544102450 msgr2=0x7ff5441a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff54259c700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff544102450 0x7ff5441a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.097+0000 7ff54259c700 1 -- 172.21.15.12:0/3401994923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340095f0 con 0x7ff5440fff40 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.098+0000 7ff54259c700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5440fff40 0x7ff5441019d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff534004760 tx=0x7ff534004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.098+0000 7ff53b7fe700 1 -- 172.21.15.12:0/3401994923 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53401d070 con 0x7ff5440fff40 2024-05-08T23:49:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.098+0000 7ff53b7fe700 1 -- 172.21.15.12:0/3401994923 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff53400fb30 con 0x7ff5440fff40 2024-05-08T23:49:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.098+0000 7ff5489a2700 1 -- 172.21.15.12:0/3401994923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5441a60d0 con 0x7ff5440fff40 2024-05-08T23:49:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.098+0000 7ff53b7fe700 1 -- 172.21.15.12:0/3401994923 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534022db0 con 0x7ff5440fff40 2024-05-08T23:49:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.098+0000 7ff5489a2700 1 -- 172.21.15.12:0/3401994923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5441a6550 con 0x7ff5440fff40 2024-05-08T23:49:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.099+0000 7ff5489a2700 1 -- 172.21.15.12:0/3401994923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5441092e0 con 0x7ff5440fff40 2024-05-08T23:49:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.100+0000 7ff53b7fe700 1 -- 172.21.15.12:0/3401994923 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff53400fca0 con 0x7ff5440fff40 2024-05-08T23:49:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.101+0000 7ff53b7fe700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5300a1310 0x7ff5300a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.101+0000 7ff53b7fe700 1 -- 172.21.15.12:0/3401994923 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff5340ce240 con 0x7ff5440fff40 2024-05-08T23:49:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.101+0000 7ff541d9b700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5300a1310 0x7ff5300a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.101+0000 7ff541d9b700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5300a1310 0x7ff5300a37d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff5441030b0 tx=0x7ff52c00b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.105+0000 7ff53b7fe700 1 -- 172.21.15.12:0/3401994923 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff534097760 con 0x7ff5440fff40 2024-05-08T23:49:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.374+0000 7ff5489a2700 1 -- 172.21.15.12:0/3401994923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff544101410 con 0x7ff5440fff40 2024-05-08T23:49:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.375+0000 7ff53b7fe700 1 -- 172.21.15.12:0/3401994923 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10879 (secure 0 0 0) 0x7ff5340a1070 con 0x7ff5440fff40 2024-05-08T23:49:48.371 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:49:48.371 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:48:23.429180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25489":{"gid":25489,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3829900343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3829900343},{"type":"v1","addr":"172.21.15.204:6837","nonce":3829900343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:49:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 -- 172.21.15.12:0/3401994923 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5300a1310 msgr2=0x7ff5300a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5300a1310 0x7ff5300a37d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff5441030b0 tx=0x7ff52c00b410 comp rx=0 tx=0).stop 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 -- 172.21.15.12:0/3401994923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5440fff40 msgr2=0x7ff5441019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5440fff40 0x7ff5441019d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff534004760 tx=0x7ff534004790 comp rx=0 tx=0).stop 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 -- 172.21.15.12:0/3401994923 shutdown_connections 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff5300a1310 0x7ff5300a37d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff544102450 0x7ff5441a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5440fff40 0x7ff5441019d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 --2- 172.21.15.12:0/3401994923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff544105060 0x7ff544101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 -- 172.21.15.12:0/3401994923 >> 172.21.15.12:0/3401994923 conn(0x7ff5440fb7e0 msgr2=0x7ff544068490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 -- 172.21.15.12:0/3401994923 shutdown_connections 2024-05-08T23:49:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.377+0000 7ff5489a2700 1 -- 172.21.15.12:0/3401994923 wait complete. 2024-05-08T23:49:48.375 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-05-08T23:49:48.387 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:49:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.531+0000 7f4ef716d700 1 -- 172.21.15.12:0/2679085216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ef0101160 msgr2=0x7f4ef0101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.531+0000 7f4ef716d700 1 --2- 172.21.15.12:0/2679085216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ef0101160 0x7f4ef0101540 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009990 tx=0x7f4ee4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.531+0000 7f4ef716d700 1 -- 172.21.15.12:0/2679085216 shutdown_connections 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.531+0000 7f4ef716d700 1 --2- 172.21.15.12:0/2679085216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ef0106280 0x7f4ef010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.531+0000 7f4ef716d700 1 --2- 172.21.15.12:0/2679085216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ef0101160 0x7f4ef0101540 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.531+0000 7f4ef716d700 1 --2- 172.21.15.12:0/2679085216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0101b10 0x7f4ef0105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.531+0000 7f4ef716d700 1 -- 172.21.15.12:0/2679085216 >> 172.21.15.12:0/2679085216 conn(0x7f4ef00fc9e0 msgr2=0x7f4ef00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.531+0000 7f4ef716d700 1 -- 172.21.15.12:0/2679085216 shutdown_connections 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.531+0000 7f4ef716d700 1 -- 172.21.15.12:0/2679085216 wait complete. 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4ef716d700 1 Processor -- start 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4ef716d700 1 -- start start 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4ef716d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0101160 0x7f4ef0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4ef716d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ef0106280 0x7f4ef0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4ef716d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ef019bd10 0x7f4ef019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4ef716d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef019c790 con 0x7f4ef0106280 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4ef716d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef019c900 con 0x7f4ef0101160 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4ef716d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef019cac0 con 0x7f4ef019bd10 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ef0106280 0x7f4ef0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ef0106280 0x7f4ef0197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55188/0 (socket says 172.21.15.12:55188) 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4eeffff700 1 -- 172.21.15.12:0/1576134968 learned_addr learned my addr 172.21.15.12:0/1576134968 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4ef570a700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ef019bd10 0x7f4ef019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0101160 0x7f4ef0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4eeffff700 1 -- 172.21.15.12:0/1576134968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ef019bd10 msgr2=0x7f4ef019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4eeffff700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ef019bd10 0x7f4ef019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4eeffff700 1 -- 172.21.15.12:0/1576134968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0101160 msgr2=0x7f4ef0197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4eeffff700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0101160 0x7f4ef0197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.532+0000 7f4eeffff700 1 -- 172.21.15.12:0/1576134968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee40095f0 con 0x7f4ef0106280 2024-05-08T23:49:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.533+0000 7f4eeffff700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ef0106280 0x7f4ef0197970 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00d980 tx=0x7f4edc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.533+0000 7f4eedffb700 1 -- 172.21.15.12:0/1576134968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00f890 con 0x7f4ef0106280 2024-05-08T23:49:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.533+0000 7f4eedffb700 1 -- 172.21.15.12:0/1576134968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4edc010460 con 0x7f4ef0106280 2024-05-08T23:49:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.533+0000 7f4ef716d700 1 -- 172.21.15.12:0/1576134968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef01a3fc0 con 0x7f4ef0106280 2024-05-08T23:49:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.533+0000 7f4eedffb700 1 -- 172.21.15.12:0/1576134968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00e610 con 0x7f4ef0106280 2024-05-08T23:49:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.533+0000 7f4ef716d700 1 -- 172.21.15.12:0/1576134968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef01a44f0 con 0x7f4ef0106280 2024-05-08T23:49:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.535+0000 7f4eedffb700 1 -- 172.21.15.12:0/1576134968 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4edc0105d0 con 0x7f4ef0106280 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.536+0000 7f4eedffb700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4ee00a1310 0x7f4ee00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.536+0000 7f4eedffb700 1 -- 172.21.15.12:0/1576134968 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4edc0cc600 con 0x7f4ef0106280 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.536+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4ee00a1310 0x7f4ee00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4ee00a1310 0x7f4ee00a37d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b5c0 tx=0x7f4ee4005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4f03327b80 1 Processor -- start 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4f03327b80 1 -- start start 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b49e00 0x5650f0b4a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 0x5650f0b44cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4f03327b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0a9b150 con 0x5650f0b4a720 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4f03327b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0a98b10 con 0x5650f0b44890 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4f03327b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0a94400 con 0x5650f0b49e00 2024-05-08T23:49:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b44350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55200/0 (socket says 172.21.15.12:55200) 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.537+0000 7f4eeffff700 1 -- 172.21.15.12:0/1278796431 learned_addr learned my addr 172.21.15.12:0/1278796431 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/1278796431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b49e00 0x5650f0b4a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4ef570a700 1 --2- 172.21.15.12:0/1278796431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 0x5650f0b44cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4eeffff700 1 -- 172.21.15.12:0/1278796431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b49e00 msgr2=0x5650f0b4a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4eeffff700 1 --2- 172.21.15.12:0/1278796431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b49e00 0x5650f0b4a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4eeffff700 1 -- 172.21.15.12:0/1278796431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 msgr2=0x5650f0b44cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4eeffff700 1 --2- 172.21.15.12:0/1278796431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 0x5650f0b44cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4eeffff700 1 -- 172.21.15.12:0/1278796431 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f09153b0 con 0x5650f0b4a720 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4eeffff700 1 --2- 172.21.15.12:0/1278796431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b44350 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0996e0 tx=0x7f4edc099130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6347195618710 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4edaffd700 1 -- 172.21.15.12:0/1278796431 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc09d580 con 0x5650f0b4a720 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4edaffd700 1 -- 172.21.15.12:0/1278796431 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4edc0a1bd0 con 0x5650f0b4a720 2024-05-08T23:49:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.538+0000 7f4edaffd700 1 -- 172.21.15.12:0/1278796431 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc099c10 con 0x5650f0b4a720 2024-05-08T23:49:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 -- 172.21.15.12:0/1278796431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 msgr2=0x5650f0b44350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 --2- 172.21.15.12:0/1278796431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b44350 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0996e0 tx=0x7f4edc099130 comp rx=0 tx=0).stop 2024-05-08T23:49:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 -- 172.21.15.12:0/1278796431 shutdown_connections 2024-05-08T23:49:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 --2- 172.21.15.12:0/1278796431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 0x5650f0b44cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 --2- 172.21.15.12:0/1278796431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b49e00 0x5650f0b4a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 --2- 172.21.15.12:0/1278796431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b44350 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 -- 172.21.15.12:0/1278796431 >> 172.21.15.12:0/1278796431 conn(0x5650f0aa2210 msgr2=0x5650f0b3edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 -- 172.21.15.12:0/1278796431 shutdown_connections 2024-05-08T23:49:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 -- 172.21.15.12:0/1278796431 wait complete. 2024-05-08T23:49:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 -- start start 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.539+0000 7f4f03327b80 1 Processor -- start 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b44890 0x5650f0b4db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b49e00 0x5650f0b4e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b4a720 0x5650f0b4e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4f03327b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0a30560 con 0x5650f0b44890 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4f03327b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0a30200 con 0x5650f0b49e00 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4f03327b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f09176a0 con 0x5650f0b4a720 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4f03327b80 10 client.0 _ll_register_callbacks cb 0x5650f0a39b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4ef4f09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b44890 0x5650f0b4db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4ef4f09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b44890 0x5650f0b4db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55206/0 (socket says 172.21.15.12:55206) 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4ef4f09700 1 -- 172.21.15.12:0/3771846586 learned_addr learned my addr 172.21.15.12:0/3771846586 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4eeffff700 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b49e00 0x5650f0b4e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4ef570a700 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b4a720 0x5650f0b4e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4ef4f09700 1 -- 172.21.15.12:0/3771846586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b4a720 msgr2=0x5650f0b4e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b4a720 0x5650f0b4e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4ef4f09700 1 -- 172.21.15.12:0/3771846586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b49e00 msgr2=0x5650f0b4e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b49e00 0x5650f0b4e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.540+0000 7f4ef4f09700 1 -- 172.21.15.12:0/3771846586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef01a44f0 con 0x5650f0b44890 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b44890 0x5650f0b4db60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400ba20 tx=0x7f4ee401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676c54255460f406 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4ecffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4ecffff700 1 -- 172.21.15.12:0/3771846586 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4004990 con 0x5650f0b44890 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4ecffff700 1 -- 172.21.15.12:0/3771846586 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4ee4004af0 con 0x5650f0b44890 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4f03327b80 1 -- 172.21.15.12:0/3771846586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650f09ed270 con 0x5650f0b44890 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4ecffff700 1 -- 172.21.15.12:0/3771846586 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee4004cb0 con 0x5650f0b44890 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4f03327b80 10 client.16329 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4f03327b80 1 -- 172.21.15.12:0/3771846586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5650f08c1b60 con 0x5650f0b44890 2024-05-08T23:49:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4ecffff700 1 -- 172.21.15.12:0/3771846586 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650f09ed270 con 0x5650f0b44890 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4ecffff700 1 -- 172.21.15.12:0/3771846586 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ee4030420 con 0x5650f0b44890 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4f03327b80 10 client.16329 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4f03327b80 10 client.16329 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4f03327b80 20 client.16329 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4f03327b80 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5650f0b5e1d0 0x5650f0b605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4f03327b80 4 client.16329 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.541+0000 7f4f03327b80 1 -- 172.21.15.12:0/3771846586 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650f0b13810 con 0x5650f0b5e1d0 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.542+0000 7f4eeffff700 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5650f0b5e1d0 0x5650f0b605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.542+0000 7f4eeffff700 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5650f0b5e1d0 0x5650f0b605f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8e8d4cc7835e9a8 server_cookie=fc4c214ae30ed59c in_seq=0 out_seq=0 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.542+0000 7f4ecffff700 10 client.16329 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4ecffff700 1 -- 172.21.15.12:0/3771846586 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650f0b13810 con 0x5650f0b5e1d0 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4ecffff700 10 client.16329 handle_command_reply: tid=0 2024-05-08T23:49:48.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 client.16329 shutdown 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 -- 172.21.15.12:0/3771846586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b44890 msgr2=0x5650f0b4db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b44890 0x5650f0b4db60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400ba20 tx=0x7f4ee401a050 comp rx=0 tx=0).stop 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 -- 172.21.15.12:0/3771846586 shutdown_connections 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5650f0b5e1d0 0x5650f0b605f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b4a720 0x5650f0b4e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b44890 0x5650f0b4db60 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 --2- 172.21.15.12:0/3771846586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b49e00 0x5650f0b4e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4ecffff700 0 client.16329 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 -- 172.21.15.12:0/3771846586 >> 172.21.15.12:0/3771846586 conn(0x5650f0aa2210 msgr2=0x5650f0b3e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 -- 172.21.15.12:0/3771846586 shutdown_connections 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 1 -- 172.21.15.12:0/3771846586 wait complete. 2024-05-08T23:49:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.544+0000 7f4f03327b80 20 client.16329 trim_cache size 0 max 16384 2024-05-08T23:49:48.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4f03327b80 1 Processor -- start 2024-05-08T23:49:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4f03327b80 1 -- start start 2024-05-08T23:49:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 0x5650f0b44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b49e00 0x5650f0b40a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b4a720 0x5650f0b3ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4f03327b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0b4a560 con 0x5650f0b49e00 2024-05-08T23:49:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4f03327b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0b65800 con 0x5650f0b44890 2024-05-08T23:49:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4f03327b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0b39800 con 0x5650f0b4a720 2024-05-08T23:49:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b49e00 0x5650f0b40a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b49e00 0x5650f0b40a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55216/0 (socket says 172.21.15.12:55216) 2024-05-08T23:49:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4eeffff700 1 -- 172.21.15.12:0/3662108891 learned_addr learned my addr 172.21.15.12:0/3662108891 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4ef570a700 1 --2- 172.21.15.12:0/3662108891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b4a720 0x5650f0b3ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/3662108891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 0x5650f0b44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4eeffff700 1 -- 172.21.15.12:0/3662108891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b4a720 msgr2=0x5650f0b3ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4eeffff700 1 --2- 172.21.15.12:0/3662108891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b4a720 0x5650f0b3ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4eeffff700 1 -- 172.21.15.12:0/3662108891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 msgr2=0x5650f0b44350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4eeffff700 1 --2- 172.21.15.12:0/3662108891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 0x5650f0b44350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4eeffff700 1 -- 172.21.15.12:0/3662108891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f0a5a560 con 0x5650f0b49e00 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/3662108891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 0x5650f0b44350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4eeffff700 1 --2- 172.21.15.12:0/3662108891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b49e00 0x5650f0b40a70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0ca390 tx=0x7f4edc095bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1075bc1501476f9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.558+0000 7f4edb7fe700 1 -- 172.21.15.12:0/3662108891 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc0108b0 con 0x5650f0b49e00 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4edb7fe700 1 -- 172.21.15.12:0/3662108891 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4edc010a10 con 0x5650f0b49e00 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4edb7fe700 1 -- 172.21.15.12:0/3662108891 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc098ea0 con 0x5650f0b49e00 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4f03327b80 1 -- 172.21.15.12:0/3662108891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b49e00 msgr2=0x5650f0b40a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4f03327b80 1 --2- 172.21.15.12:0/3662108891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b49e00 0x5650f0b40a70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0ca390 tx=0x7f4edc095bc0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4f03327b80 1 -- 172.21.15.12:0/3662108891 shutdown_connections 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4f03327b80 1 --2- 172.21.15.12:0/3662108891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b4a720 0x5650f0b3ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4f03327b80 1 --2- 172.21.15.12:0/3662108891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b44890 0x5650f0b44350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4f03327b80 1 --2- 172.21.15.12:0/3662108891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b49e00 0x5650f0b40a70 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4f03327b80 1 -- 172.21.15.12:0/3662108891 >> 172.21.15.12:0/3662108891 conn(0x5650f0aa2210 msgr2=0x5650f0b44c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4f03327b80 1 -- 172.21.15.12:0/3662108891 shutdown_connections 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.559+0000 7f4f03327b80 1 -- 172.21.15.12:0/3662108891 wait complete. 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4f03327b80 1 -- start start 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4f03327b80 1 Processor -- start 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b44890 0x5650f0b552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b49e00 0x5650f0b51080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4f03327b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4f03327b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0b51b00 con 0x5650f0b4a720 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4f03327b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0b51c70 con 0x5650f0b49e00 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4f03327b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f0b51e30 con 0x5650f0b44890 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4f03327b80 10 client.0 _ll_register_callbacks cb 0x7f4ee40055b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4ef570a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4ef570a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55222/0 (socket says 172.21.15.12:55222) 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4ef570a700 1 -- 172.21.15.12:0/446920473 learned_addr learned my addr 172.21.15.12:0/446920473 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.560+0000 7f4eeffff700 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b49e00 0x5650f0b51080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b44890 0x5650f0b552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ef570a700 1 -- 172.21.15.12:0/446920473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b44890 msgr2=0x5650f0b552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ef570a700 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b44890 0x5650f0b552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ef570a700 1 -- 172.21.15.12:0/446920473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b49e00 msgr2=0x5650f0b51080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ef570a700 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b49e00 0x5650f0b51080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ef570a700 1 -- 172.21.15.12:0/446920473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650f09153b0 con 0x5650f0b4a720 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b44890 0x5650f0b552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ef570a700 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b515c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8006070 tx=0x7f4ee80042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4030255531069202 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ecffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ecffff700 1 -- 172.21.15.12:0/446920473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8003dc0 con 0x5650f0b4a720 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ecffff700 1 -- 172.21.15.12:0/446920473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4ee8007910 con 0x5650f0b4a720 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4f03327b80 1 -- 172.21.15.12:0/446920473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4edc0a1bd0 con 0x5650f0b4a720 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ecffff700 1 -- 172.21.15.12:0/446920473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee8007ad0 con 0x5650f0b4a720 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4f03327b80 10 client.16335 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4f03327b80 1 -- 172.21.15.12:0/446920473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5650f0919cd0 con 0x5650f0b4a720 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ecffff700 1 -- 172.21.15.12:0/446920473 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4edc0a1bd0 con 0x5650f0b4a720 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4ecffff700 1 -- 172.21.15.12:0/446920473 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ee8017410 con 0x5650f0b4a720 2024-05-08T23:49:48.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4f03327b80 10 client.16335 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4f03327b80 10 client.16335 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4f03327b80 20 client.16335 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4f03327b80 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5650f0bb8fd0 0x5650f0bbb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4f03327b80 4 client.16335 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.561+0000 7f4f03327b80 1 -- 172.21.15.12:0/446920473 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5650f0a8cec0 con 0x5650f0bb8fd0 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.562+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5650f0bb8fd0 0x5650f0bbb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.562+0000 7f4ecffff700 10 client.16335 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.562+0000 7f4ef4f09700 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5650f0bb8fd0 0x5650f0bbb3f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39c007217528f2b8 server_cookie=218f5c3e0fa58d0 in_seq=0 out_seq=0 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.562+0000 7f4ecffff700 1 -- 172.21.15.12:0/446920473 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5650f0a8cec0 con 0x5650f0bb8fd0 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.562+0000 7f4ecffff700 10 client.16335 handle_command_reply: tid=0 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.562+0000 7f4f03327b80 1 client.16335 shutdown 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.562+0000 7f4f03327b80 1 -- 172.21.15.12:0/446920473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 msgr2=0x5650f0b515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.562+0000 7f4f03327b80 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b515c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8006070 tx=0x7f4ee80042e0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4f03327b80 1 -- 172.21.15.12:0/446920473 shutdown_connections 2024-05-08T23:49:48.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4f03327b80 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650f0b4a720 0x5650f0b515c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4f03327b80 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5650f0bb8fd0 0x5650f0bbb3f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4f03327b80 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650f0b44890 0x5650f0b552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4f03327b80 1 --2- 172.21.15.12:0/446920473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650f0b49e00 0x5650f0b51080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ecffff700 0 client.16335 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:49:48.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4f03327b80 1 -- 172.21.15.12:0/446920473 >> 172.21.15.12:0/446920473 conn(0x5650f0aa2210 msgr2=0x5650f0b48c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4f03327b80 1 -- 172.21.15.12:0/446920473 shutdown_connections 2024-05-08T23:49:48.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4f03327b80 1 -- 172.21.15.12:0/446920473 wait complete. 2024-05-08T23:49:48.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4f03327b80 20 client.16335 trim_cache size 0 max 16384 2024-05-08T23:49:48.566 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:49:48.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ef716d700 1 -- 172.21.15.12:0/1576134968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4ee00a1310 msgr2=0x7f4ee00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ef716d700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4ee00a1310 0x7f4ee00a37d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b5c0 tx=0x7f4ee4005fd0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ef716d700 1 -- 172.21.15.12:0/1576134968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ef0106280 msgr2=0x7f4ef0197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ef716d700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ef0106280 0x7f4ef0197970 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00d980 tx=0x7f4edc00dc90 comp rx=0 tx=0).stop 2024-05-08T23:49:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ef716d700 1 -- 172.21.15.12:0/1576134968 shutdown_connections 2024-05-08T23:49:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ef716d700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4ee00a1310 0x7f4ee00a37d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ef716d700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ef019bd10 0x7f4ef019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ef716d700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef0101160 0x7f4ef0197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ef716d700 1 --2- 172.21.15.12:0/1576134968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ef0106280 0x7f4ef0197970 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.563+0000 7f4ef716d700 1 -- 172.21.15.12:0/1576134968 >> 172.21.15.12:0/1576134968 conn(0x7f4ef00fc9e0 msgr2=0x7f4ef00fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.564+0000 7f4ef716d700 1 -- 172.21.15.12:0/1576134968 shutdown_connections 2024-05-08T23:49:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.564+0000 7f4ef716d700 1 -- 172.21.15.12:0/1576134968 wait complete. 2024-05-08T23:49:48.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:49:48.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:48 smithi012 ceph-mon[42503]: pgmap v105: 97 pgs: 97 active+clean; 32 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-05-08T23:49:48.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:48 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3401994923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:49:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.715+0000 7ff3e39f2700 1 -- 172.21.15.12:0/1434639170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3dc1062a0 msgr2=0x7ff3dc10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.715+0000 7ff3e39f2700 1 --2- 172.21.15.12:0/1434639170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3dc1062a0 0x7ff3dc10de20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400b0a0 tx=0x7ff3d400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.716+0000 7ff3e39f2700 1 -- 172.21.15.12:0/1434639170 shutdown_connections 2024-05-08T23:49:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.716+0000 7ff3e39f2700 1 --2- 172.21.15.12:0/1434639170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3dc1062a0 0x7ff3dc10de20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.716+0000 7ff3e39f2700 1 --2- 172.21.15.12:0/1434639170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3dc101180 0x7ff3dc101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.716+0000 7ff3e39f2700 1 --2- 172.21.15.12:0/1434639170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3dc101b30 0x7ff3dc105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.716+0000 7ff3e39f2700 1 -- 172.21.15.12:0/1434639170 >> 172.21.15.12:0/1434639170 conn(0x7ff3dc0fc9e0 msgr2=0x7ff3dc0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.716+0000 7ff3e39f2700 1 -- 172.21.15.12:0/1434639170 shutdown_connections 2024-05-08T23:49:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.716+0000 7ff3e39f2700 1 -- 172.21.15.12:0/1434639170 wait complete. 2024-05-08T23:49:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.716+0000 7ff3e39f2700 1 Processor -- start 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.716+0000 7ff3e39f2700 1 -- start start 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e39f2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3dc101180 0x7ff3dc199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e39f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3dc1062a0 0x7ff3dc199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e39f2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3dc19e020 0x7ff3dc19e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e39f2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc19eaa0 con 0x7ff3dc1062a0 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e39f2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc19ec10 con 0x7ff3dc101180 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e39f2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc19edd0 con 0x7ff3dc19e020 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e0f8d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3dc1062a0 0x7ff3dc199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e0f8d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3dc1062a0 0x7ff3dc199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55242/0 (socket says 172.21.15.12:55242) 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/3753443041 learned_addr learned my addr 172.21.15.12:0/3753443041 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e1f8f700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3dc19e020 0x7ff3dc19e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e178e700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3dc101180 0x7ff3dc199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/3753443041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3dc19e020 msgr2=0x7ff3dc19e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3dc19e020 0x7ff3dc19e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/3753443041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3dc101180 msgr2=0x7ff3dc199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3dc101180 0x7ff3dc199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/3753443041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4009d00 con 0x7ff3dc1062a0 2024-05-08T23:49:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3dc1062a0 0x7ff3dc199c80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00d950 tx=0x7ff3cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.717+0000 7ff3d27fc700 1 -- 172.21.15.12:0/3753443041 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc004220 con 0x7ff3dc1062a0 2024-05-08T23:49:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.718+0000 7ff3d27fc700 1 -- 172.21.15.12:0/3753443041 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3cc007620 con 0x7ff3dc1062a0 2024-05-08T23:49:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.718+0000 7ff3e39f2700 1 -- 172.21.15.12:0/3753443041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3dc1a6330 con 0x7ff3dc1062a0 2024-05-08T23:49:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.718+0000 7ff3d27fc700 1 -- 172.21.15.12:0/3753443041 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc005670 con 0x7ff3dc1062a0 2024-05-08T23:49:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.718+0000 7ff3e39f2700 1 -- 172.21.15.12:0/3753443041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3dc1a6810 con 0x7ff3dc1062a0 2024-05-08T23:49:48.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.720+0000 7ff3d27fc700 1 -- 172.21.15.12:0/3753443041 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff3cc003680 con 0x7ff3dc1062a0 2024-05-08T23:49:48.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.721+0000 7ff3d27fc700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3c80a1290 0x7ff3c80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.721+0000 7ff3e178e700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3c80a1290 0x7ff3c80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.721+0000 7ff3d27fc700 1 -- 172.21.15.12:0/3753443041 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff3cc0cc990 con 0x7ff3dc1062a0 2024-05-08T23:49:48.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.721+0000 7ff3e178e700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3c80a1290 0x7ff3c80a3750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8005950 tx=0x7ff3d800b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3efbacb80 1 Processor -- start 2024-05-08T23:49:48.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3efbacb80 1 -- start start 2024-05-08T23:49:48.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b880e00 0x55cb4b8811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b881720 0x55cb4b87b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b87b890 0x55cb4b87bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b7d2150 con 0x55cb4b87b890 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b7cfb10 con 0x55cb4b880e00 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b7cb400 con 0x55cb4b881720 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e1f8f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b87b890 0x55cb4b87bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e1f8f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b87b890 0x55cb4b87bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55244/0 (socket says 172.21.15.12:55244) 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e1f8f700 1 -- 172.21.15.12:0/2124585849 learned_addr learned my addr 172.21.15.12:0/2124585849 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e178e700 1 --2- 172.21.15.12:0/2124585849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b880e00 0x55cb4b8811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/2124585849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b881720 0x55cb4b87b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e1f8f700 1 -- 172.21.15.12:0/2124585849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b881720 msgr2=0x55cb4b87b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e1f8f700 1 --2- 172.21.15.12:0/2124585849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b881720 0x55cb4b87b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e1f8f700 1 -- 172.21.15.12:0/2124585849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b880e00 msgr2=0x55cb4b8811e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e1f8f700 1 --2- 172.21.15.12:0/2124585849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b880e00 0x55cb4b8811e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e1f8f700 1 -- 172.21.15.12:0/2124585849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb4b64c3b0 con 0x55cb4b87b890 2024-05-08T23:49:48.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.722+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/2124585849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b881720 0x55cb4b87b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:49:48.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.723+0000 7ff3e1f8f700 1 --2- 172.21.15.12:0/2124585849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b87b890 0x55cb4b87bcd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400b0a0 tx=0x7ff3d4007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a12e06f326f1e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.723+0000 7ff3c77fe700 1 -- 172.21.15.12:0/2124585849 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400e040 con 0x55cb4b87b890 2024-05-08T23:49:48.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.723+0000 7ff3c77fe700 1 -- 172.21.15.12:0/2124585849 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3d4007e80 con 0x55cb4b87b890 2024-05-08T23:49:48.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.723+0000 7ff3c77fe700 1 -- 172.21.15.12:0/2124585849 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d401c6a0 con 0x55cb4b87b890 2024-05-08T23:49:48.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.723+0000 7ff3efbacb80 1 -- 172.21.15.12:0/2124585849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b87b890 msgr2=0x55cb4b87bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.723+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/2124585849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b87b890 0x55cb4b87bcd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400b0a0 tx=0x7ff3d4007be0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.724+0000 7ff3efbacb80 1 -- 172.21.15.12:0/2124585849 shutdown_connections 2024-05-08T23:49:48.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.724+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/2124585849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b87b890 0x55cb4b87bcd0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.724+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/2124585849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b880e00 0x55cb4b8811e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.724+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/2124585849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b881720 0x55cb4b87b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.724+0000 7ff3efbacb80 1 -- 172.21.15.12:0/2124585849 >> 172.21.15.12:0/2124585849 conn(0x55cb4b7d9210 msgr2=0x55cb4b875df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.724+0000 7ff3efbacb80 1 -- 172.21.15.12:0/2124585849 shutdown_connections 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.724+0000 7ff3efbacb80 1 -- 172.21.15.12:0/2124585849 wait complete. 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.724+0000 7ff3efbacb80 1 -- start start 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.724+0000 7ff3efbacb80 1 Processor -- start 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b87b890 0x55cb4b884b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b8850b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b881720 0x55cb4b8855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b767560 con 0x55cb4b880e00 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b767200 con 0x55cb4b881720 2024-05-08T23:49:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b64e6a0 con 0x55cb4b87b890 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3efbacb80 10 client.0 _ll_register_callbacks cb 0x55cb4b770b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3e0f8d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b8850b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3e0f8d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b8850b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55258/0 (socket says 172.21.15.12:55258) 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/877461799 learned_addr learned my addr 172.21.15.12:0/877461799 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3e178e700 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b87b890 0x55cb4b884b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/877461799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b87b890 msgr2=0x55cb4b884b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b87b890 0x55cb4b884b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/877461799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b881720 msgr2=0x55cb4b8855f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b881720 0x55cb4b8855f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/877461799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3dc10e690 con 0x55cb4b880e00 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b8850b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0a19d0 tx=0x7ff3cc099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a93fa8907fb3593 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/877461799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc099ee0 con 0x55cb4b880e00 2024-05-08T23:49:48.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/877461799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff3cc098f70 con 0x55cb4b880e00 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.725+0000 7ff3efbacb80 1 -- 172.21.15.12:0/877461799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb4b84a810 con 0x55cb4b880e00 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/877461799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3cc099130 con 0x55cb4b880e00 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3efbacb80 10 client.16347 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3efbacb80 1 -- 172.21.15.12:0/877461799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cb4b5f8b60 con 0x55cb4b880e00 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/877461799 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3cc0b6020 con 0x55cb4b880e00 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/877461799 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff3cc0989e0 con 0x55cb4b880e00 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3efbacb80 10 client.16347 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3efbacb80 10 client.16347 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3efbacb80 20 client.16347 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb4b895030 0x55cb4b897450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3efbacb80 4 client.16347 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.726+0000 7ff3efbacb80 1 -- 172.21.15.12:0/877461799 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb4b7c3ec0 con 0x55cb4b895030 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.727+0000 7ff3e178e700 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb4b895030 0x55cb4b897450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.727+0000 7ff3e178e700 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb4b895030 0x55cb4b897450 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8afcda8e90f072cd server_cookie=13beae184838c9a in_seq=0 out_seq=0 2024-05-08T23:49:48.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.727+0000 7ff3c4ff9700 10 client.16347 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/877461799 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb4b7c3ec0 con 0x55cb4b895030 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3c4ff9700 10 client.16347 handle_command_reply: tid=0 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 client.16347 shutdown 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 -- 172.21.15.12:0/877461799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 msgr2=0x55cb4b8850b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b8850b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0a19d0 tx=0x7ff3cc099bb0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 -- 172.21.15.12:0/877461799 shutdown_connections 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb4b895030 0x55cb4b897450 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b881720 0x55cb4b8855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b87b890 0x55cb4b884b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/877461799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b8850b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3c4ff9700 0 client.16347 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 -- 172.21.15.12:0/877461799 >> 172.21.15.12:0/877461799 conn(0x55cb4b7d9210 msgr2=0x55cb4b8755a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 -- 172.21.15.12:0/877461799 shutdown_connections 2024-05-08T23:49:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.728+0000 7ff3efbacb80 1 -- 172.21.15.12:0/877461799 wait complete. 2024-05-08T23:49:48.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.729+0000 7ff3efbacb80 20 client.16347 trim_cache size 0 max 16384 2024-05-08T23:49:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3efbacb80 1 Processor -- start 2024-05-08T23:49:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3efbacb80 1 -- start start 2024-05-08T23:49:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b87b890 0x55cb4b87b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b897450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b881720 0x55cb4b890bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b872dc0 con 0x55cb4b880e00 2024-05-08T23:49:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b877640 con 0x55cb4b881720 2024-05-08T23:49:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b89c660 con 0x55cb4b87b890 2024-05-08T23:49:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3e0f8d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b897450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3e0f8d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b897450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55264/0 (socket says 172.21.15.12:55264) 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/1270501068 learned_addr learned my addr 172.21.15.12:0/1270501068 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3e178e700 1 --2- 172.21.15.12:0/1270501068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b87b890 0x55cb4b87b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/1270501068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b87b890 msgr2=0x55cb4b87b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/1270501068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b87b890 0x55cb4b87b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/1270501068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b881720 msgr2=0x55cb4b890bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/1270501068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b881720 0x55cb4b890bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.743+0000 7ff3e0f8d700 1 -- 172.21.15.12:0/1270501068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb4b791560 con 0x55cb4b880e00 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/1270501068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b897450 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0964f0 tx=0x7ff3cc098030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8783e6502afdd9ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3c7fff700 1 -- 172.21.15.12:0/1270501068 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc014070 con 0x55cb4b880e00 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3c7fff700 1 -- 172.21.15.12:0/1270501068 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3cc0b6030 con 0x55cb4b880e00 2024-05-08T23:49:48.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3c7fff700 1 -- 172.21.15.12:0/1270501068 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc0991d0 con 0x55cb4b880e00 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3efbacb80 1 -- 172.21.15.12:0/1270501068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 msgr2=0x55cb4b897450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/1270501068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b897450 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0964f0 tx=0x7ff3cc098030 comp rx=0 tx=0).stop 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3efbacb80 1 -- 172.21.15.12:0/1270501068 shutdown_connections 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/1270501068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b881720 0x55cb4b890bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/1270501068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b87b890 0x55cb4b87b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/1270501068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b880e00 0x55cb4b897450 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.744+0000 7ff3efbacb80 1 -- 172.21.15.12:0/1270501068 >> 172.21.15.12:0/1270501068 conn(0x55cb4b7d9210 msgr2=0x55cb4b883460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.745+0000 7ff3efbacb80 1 -- 172.21.15.12:0/1270501068 shutdown_connections 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.745+0000 7ff3efbacb80 1 -- 172.21.15.12:0/1270501068 wait complete. 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.745+0000 7ff3efbacb80 1 -- start start 2024-05-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.745+0000 7ff3efbacb80 1 Processor -- start 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.745+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b87b890 0x55cb4b88c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.745+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b880e00 0x55cb4b88a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.745+0000 7ff3efbacb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b881720 0x55cb4b88a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.745+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b88abb0 con 0x55cb4b881720 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.745+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b88ad20 con 0x55cb4b87b890 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.745+0000 7ff3efbacb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4b88aee0 con 0x55cb4b880e00 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3efbacb80 10 client.0 _ll_register_callbacks cb 0x55cb4b770b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e1f8f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b881720 0x55cb4b88a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e1f8f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b881720 0x55cb4b88a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55272/0 (socket says 172.21.15.12:55272) 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e1f8f700 1 -- 172.21.15.12:0/695920378 learned_addr learned my addr 172.21.15.12:0/695920378 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b880e00 0x55cb4b88a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e178e700 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b87b890 0x55cb4b88c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e1f8f700 1 -- 172.21.15.12:0/695920378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b880e00 msgr2=0x55cb4b88a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e1f8f700 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b880e00 0x55cb4b88a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e1f8f700 1 -- 172.21.15.12:0/695920378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b87b890 msgr2=0x55cb4b88c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e1f8f700 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b87b890 0x55cb4b88c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e1f8f700 1 -- 172.21.15.12:0/695920378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb4b5f8b60 con 0x55cb4b881720 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e0f8d700 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b880e00 0x55cb4b88a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3e1f8f700 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b881720 0x55cb4b88a630 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400d010 tx=0x7ff3d40094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7c474f2a6dcd85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/695920378 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400e040 con 0x55cb4b881720 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/695920378 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff3d40074a0 con 0x55cb4b881720 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.746+0000 7ff3efbacb80 1 -- 172.21.15.12:0/695920378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb4b724270 con 0x55cb4b881720 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/695920378 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3d4007660 con 0x55cb4b881720 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3efbacb80 10 client.16353 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3efbacb80 1 -- 172.21.15.12:0/695920378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cb4b650cd0 con 0x55cb4b881720 2024-05-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/695920378 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb4b724270 con 0x55cb4b881720 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/695920378 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff3d4019070 con 0x55cb4b881720 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3efbacb80 10 client.16353 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3efbacb80 10 client.16353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3efbacb80 20 client.16353 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb4b8eb7e0 0x55cb4b8edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3efbacb80 4 client.16353 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3efbacb80 1 -- 172.21.15.12:0/695920378 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb4b619cf0 con 0x55cb4b8eb7e0 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.747+0000 7ff3e178e700 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb4b8eb7e0 0x55cb4b8edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.748+0000 7ff3c4ff9700 10 client.16353 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.748+0000 7ff3e178e700 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb4b8eb7e0 0x55cb4b8edbd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17d9f2006bd68fc3 server_cookie=333f243f9fcdb0b8 in_seq=0 out_seq=0 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.748+0000 7ff3c4ff9700 1 -- 172.21.15.12:0/695920378 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb4b619cf0 con 0x55cb4b8eb7e0 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.748+0000 7ff3c4ff9700 10 client.16353 handle_command_reply: tid=0 2024-05-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.748+0000 7ff3efbacb80 1 client.16353 shutdown 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.748+0000 7ff3efbacb80 1 -- 172.21.15.12:0/695920378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b881720 msgr2=0x55cb4b88a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.748+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b881720 0x55cb4b88a630 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400d010 tx=0x7ff3d40094f0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3efbacb80 1 -- 172.21.15.12:0/695920378 shutdown_connections 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb4b8eb7e0 0x55cb4b8edbd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb4b881720 0x55cb4b88a630 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb4b87b890 0x55cb4b88c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3c4ff9700 0 client.16353 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3efbacb80 1 --2- 172.21.15.12:0/695920378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb4b880e00 0x55cb4b88a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3efbacb80 1 -- 172.21.15.12:0/695920378 >> 172.21.15.12:0/695920378 conn(0x55cb4b7d9210 msgr2=0x55cb4b87fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3efbacb80 1 -- 172.21.15.12:0/695920378 shutdown_connections 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3efbacb80 1 -- 172.21.15.12:0/695920378 wait complete. 2024-05-08T23:49:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3efbacb80 20 client.16353 trim_cache size 0 max 16384 2024-05-08T23:49:48.751 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:49:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3e39f2700 1 -- 172.21.15.12:0/3753443041 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3c80a1290 msgr2=0x7ff3c80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3e39f2700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3c80a1290 0x7ff3c80a3750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8005950 tx=0x7ff3d800b410 comp rx=0 tx=0).stop 2024-05-08T23:49:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3e39f2700 1 -- 172.21.15.12:0/3753443041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3dc1062a0 msgr2=0x7ff3dc199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3e39f2700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3dc1062a0 0x7ff3dc199c80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00d950 tx=0x7ff3cc00dc60 comp rx=0 tx=0).stop 2024-05-08T23:49:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3e39f2700 1 -- 172.21.15.12:0/3753443041 shutdown_connections 2024-05-08T23:49:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3e39f2700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3c80a1290 0x7ff3c80a3750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3e39f2700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3dc19e020 0x7ff3dc19e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3e39f2700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3dc101180 0x7ff3dc199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3e39f2700 1 --2- 172.21.15.12:0/3753443041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3dc1062a0 0x7ff3dc199c80 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.749+0000 7ff3e39f2700 1 -- 172.21.15.12:0/3753443041 >> 172.21.15.12:0/3753443041 conn(0x7ff3dc0fc9e0 msgr2=0x7ff3dc0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.750+0000 7ff3e39f2700 1 -- 172.21.15.12:0/3753443041 shutdown_connections 2024-05-08T23:49:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.750+0000 7ff3e39f2700 1 -- 172.21.15.12:0/3753443041 wait complete. 2024-05-08T23:49:48.759 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:49:48.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:48 smithi204 ceph-mon[47373]: pgmap v105: 97 pgs: 97 active+clean; 32 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-05-08T23:49:48.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:48 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3401994923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:49:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:48 smithi160 ceph-mon[46152]: pgmap v105: 97 pgs: 97 active+clean; 32 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-05-08T23:49:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:48 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3401994923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:49:48.853 INFO:tasks.workunit.client.0.smithi012.stdout: 1 81292 4.38 MB/sec warmup 104 sec latency 934.575 ms 2024-05-08T23:49:48.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.902+0000 7f69edd9b700 1 -- 172.21.15.12:0/3282953094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69e8100a10 msgr2=0x7f69e8104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.902+0000 7f69edd9b700 1 --2- 172.21.15.12:0/3282953094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69e8100a10 0x7f69e8104a40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009990 tx=0x7f69d0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.902+0000 7f69edd9b700 1 -- 172.21.15.12:0/3282953094 shutdown_connections 2024-05-08T23:49:48.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.902+0000 7f69edd9b700 1 --2- 172.21.15.12:0/3282953094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e8105170 0x7f69e810ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.902+0000 7f69edd9b700 1 --2- 172.21.15.12:0/3282953094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e8100060 0x7f69e8100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.902+0000 7f69edd9b700 1 --2- 172.21.15.12:0/3282953094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69e8100a10 0x7f69e8104a40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.902+0000 7f69edd9b700 1 -- 172.21.15.12:0/3282953094 >> 172.21.15.12:0/3282953094 conn(0x7f69e8076f30 msgr2=0x7f69e8077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.902+0000 7f69edd9b700 1 -- 172.21.15.12:0/3282953094 shutdown_connections 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.902+0000 7f69edd9b700 1 -- 172.21.15.12:0/3282953094 wait complete. 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69edd9b700 1 Processor -- start 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69edd9b700 1 -- start start 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69edd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e8100060 0x7f69e8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69edd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69e8105170 0x7f69e8199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69edd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e819df60 0x7f69e819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69edd9b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e819e9e0 con 0x7f69e8105170 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69edd9b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e819eb50 con 0x7f69e819df60 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69edd9b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e819ed10 con 0x7f69e8100060 2024-05-08T23:49:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69e8105170 0x7f69e8199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69e8105170 0x7f69e8199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55296/0 (socket says 172.21.15.12:55296) 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e6ffd700 1 -- 172.21.15.12:0/115564578 learned_addr learned my addr 172.21.15.12:0/115564578 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e77fe700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e8100060 0x7f69e8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e7fff700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e819df60 0x7f69e819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e6ffd700 1 -- 172.21.15.12:0/115564578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e8100060 msgr2=0x7f69e8199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e8100060 0x7f69e8199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e6ffd700 1 -- 172.21.15.12:0/115564578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e819df60 msgr2=0x7f69e819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e819df60 0x7f69e819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e6ffd700 1 -- 172.21.15.12:0/115564578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d00095f0 con 0x7f69e8105170 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.903+0000 7f69e77fe700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e8100060 0x7f69e8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:49:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.904+0000 7f69e7fff700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e819df60 0x7f69e819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:49:48.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.904+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69e8105170 0x7f69e8199bc0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f69d00092e0 tx=0x7f69d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.904+0000 7f69e4ff9700 1 -- 172.21.15.12:0/115564578 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d001d070 con 0x7f69e8105170 2024-05-08T23:49:48.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.904+0000 7f69edd9b700 1 -- 172.21.15.12:0/115564578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e81a6210 con 0x7f69e8105170 2024-05-08T23:49:48.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.904+0000 7f69e4ff9700 1 -- 172.21.15.12:0/115564578 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d000fbb0 con 0x7f69e8105170 2024-05-08T23:49:48.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.904+0000 7f69e4ff9700 1 -- 172.21.15.12:0/115564578 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0022da0 con 0x7f69e8105170 2024-05-08T23:49:48.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.904+0000 7f69edd9b700 1 -- 172.21.15.12:0/115564578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e81a6690 con 0x7f69e8105170 2024-05-08T23:49:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.906+0000 7f69e4ff9700 1 -- 172.21.15.12:0/115564578 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f69d000fd20 con 0x7f69e8105170 2024-05-08T23:49:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.907+0000 7f69e4ff9700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69d40a1290 0x7f69d40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.907+0000 7f69e4ff9700 1 -- 172.21.15.12:0/115564578 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f69d00cdfc0 con 0x7f69e8105170 2024-05-08T23:49:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.907+0000 7f69e77fe700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69d40a1290 0x7f69d40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.907+0000 7f69e77fe700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69d40a1290 0x7f69d40a3750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f69d8005fd0 tx=0x7f69d8005ee0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69f9f55b80 1 Processor -- start 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69f9f55b80 1 -- start start 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 0x55e0b93e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e2720 0x55e0b93dc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93dc890 0x55e0b93dccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b9333150 con 0x55e0b93dc890 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b9330b10 con 0x55e0b93e2720 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b932c400 con 0x55e0b93e1e00 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93dc890 0x55e0b93dccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69e77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 0x55e0b93e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.908+0000 7f69e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93dc890 0x55e0b93dccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55304/0 (socket says 172.21.15.12:55304) 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69e7fff700 1 -- 172.21.15.12:0/1797515612 learned_addr learned my addr 172.21.15.12:0/1797515612 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69e77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 0x55e0b93e21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:32986/0 (socket says 172.21.15.12:32986) 2024-05-08T23:49:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/1797515612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e2720 0x55e0b93dc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69e77fe700 1 -- 172.21.15.12:0/1797515612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e2720 msgr2=0x55e0b93dc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69e77fe700 1 --2- 172.21.15.12:0/1797515612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e2720 0x55e0b93dc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69e77fe700 1 -- 172.21.15.12:0/1797515612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93dc890 msgr2=0x55e0b93dccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69e77fe700 1 --2- 172.21.15.12:0/1797515612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93dc890 0x55e0b93dccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69e77fe700 1 -- 172.21.15.12:0/1797515612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0b91ad3b0 con 0x55e0b93e1e00 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69e7fff700 1 --2- 172.21.15.12:0/1797515612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93dc890 0x55e0b93dccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69e77fe700 1 --2- 172.21.15.12:0/1797515612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 0x55e0b93e21e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009410 tx=0x7f69d8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80928a83273f624b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69ddffb700 1 -- 172.21.15.12:0/1797515612 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8022070 con 0x55e0b93e1e00 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69ddffb700 1 -- 172.21.15.12:0/1797515612 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d8011d90 con 0x55e0b93e1e00 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.909+0000 7f69ddffb700 1 -- 172.21.15.12:0/1797515612 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8005310 con 0x55e0b93e1e00 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.910+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1797515612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 msgr2=0x55e0b93e21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.910+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/1797515612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 0x55e0b93e21e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009410 tx=0x7f69d8009f90 comp rx=0 tx=0).stop 2024-05-08T23:49:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.910+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1797515612 shutdown_connections 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.910+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/1797515612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93dc890 0x55e0b93dccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.910+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/1797515612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 0x55e0b93e21e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.910+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/1797515612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e2720 0x55e0b93dc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.910+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1797515612 >> 172.21.15.12:0/1797515612 conn(0x55e0b933a210 msgr2=0x55e0b93d6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.910+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1797515612 shutdown_connections 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.910+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1797515612 wait complete. 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.910+0000 7f69f9f55b80 1 -- start start 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69f9f55b80 1 Processor -- start 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93dc890 0x55e0b93e5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e1e00 0x55e0b93e60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93e6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b92c8560 con 0x55e0b93e2720 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b92c8200 con 0x55e0b93e1e00 2024-05-08T23:49:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b91af6a0 con 0x55e0b93dc890 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69f9f55b80 10 client.0 _ll_register_callbacks cb 0x55e0b92d1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69e6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e1e00 0x55e0b93e60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69e6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e1e00 0x55e0b93e60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50040/0 (socket says 172.21.15.12:50040) 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69e6ffd700 1 -- 172.21.15.12:0/4008546845 learned_addr learned my addr 172.21.15.12:0/4008546845 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93e6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55312/0 (socket says 172.21.15.12:55312) 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.911+0000 7f69e77fe700 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93dc890 0x55e0b93e5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69e6ffd700 1 -- 172.21.15.12:0/4008546845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93dc890 msgr2=0x55e0b93e5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93dc890 0x55e0b93e5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69e6ffd700 1 -- 172.21.15.12:0/4008546845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 msgr2=0x55e0b93e6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93e6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69e6ffd700 1 -- 172.21.15.12:0/4008546845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e0007b60 con 0x55e0b93e1e00 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e1e00 0x55e0b93e60d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f69d0097650 tx=0x7f69d009b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3aff5365b5a38b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69c6ffd700 1 -- 172.21.15.12:0/4008546845 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d00cbea0 con 0x55e0b93e1e00 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69f9f55b80 1 -- 172.21.15.12:0/4008546845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0b9285270 con 0x55e0b93e1e00 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69c6ffd700 1 -- 172.21.15.12:0/4008546845 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f69d00a3590 con 0x55e0b93e1e00 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69c6ffd700 1 -- 172.21.15.12:0/4008546845 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69d009b600 con 0x55e0b93e1e00 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69f9f55b80 10 client.26171 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.912+0000 7f69f9f55b80 1 -- 172.21.15.12:0/4008546845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f69d800ea20 con 0x55e0b93e1e00 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.913+0000 7f69c6ffd700 1 -- 172.21.15.12:0/4008546845 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0b9285270 con 0x55e0b93e1e00 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.913+0000 7f69c6ffd700 1 -- 172.21.15.12:0/4008546845 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f69d00a3100 con 0x55e0b93e1e00 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.913+0000 7f69f9f55b80 10 client.26171 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.913+0000 7f69f9f55b80 10 client.26171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.913+0000 7f69f9f55b80 20 client.26171 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.913+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0b93f5f30 0x55e0b93f8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.913+0000 7f69f9f55b80 4 client.26171 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.913+0000 7f69f9f55b80 1 -- 172.21.15.12:0/4008546845 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0b93ab810 con 0x55e0b93f5f30 2024-05-08T23:49:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.913+0000 7f69e77fe700 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0b93f5f30 0x55e0b93f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69e77fe700 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0b93f5f30 0x55e0b93f8350 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc0b300154e145a server_cookie=ea70fe4f96cff418 in_seq=0 out_seq=0 2024-05-08T23:49:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69c6ffd700 10 client.26171 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69c6ffd700 1 -- 172.21.15.12:0/4008546845 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0b93ab810 con 0x55e0b93f5f30 2024-05-08T23:49:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69c6ffd700 10 client.26171 handle_command_reply: tid=0 2024-05-08T23:49:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 client.26171 shutdown 2024-05-08T23:49:48.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 -- 172.21.15.12:0/4008546845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e1e00 msgr2=0x55e0b93e60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e1e00 0x55e0b93e60d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f69d0097650 tx=0x7f69d009b970 comp rx=0 tx=0).stop 2024-05-08T23:49:48.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 -- 172.21.15.12:0/4008546845 shutdown_connections 2024-05-08T23:49:48.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93dc890 0x55e0b93e5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0b93f5f30 0x55e0b93f8350 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/4008546845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93e1e00 0x55e0b93e60d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69c6ffd700 0 client.26171 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:48.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 -- 172.21.15.12:0/4008546845 >> 172.21.15.12:0/4008546845 conn(0x55e0b933a210 msgr2=0x55e0b93d65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 -- 172.21.15.12:0/4008546845 shutdown_connections 2024-05-08T23:49:48.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 1 -- 172.21.15.12:0/4008546845 wait complete. 2024-05-08T23:49:48.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.915+0000 7f69f9f55b80 20 client.26171 trim_cache size 0 max 16384 2024-05-08T23:49:48.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69f9f55b80 1 Processor -- start 2024-05-08T23:49:48.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69f9f55b80 1 -- start start 2024-05-08T23:49:48.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 0x55e0b93f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 0x55e0b93e1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93d6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b93cf110 con 0x55e0b93e2720 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b93dd210 con 0x55e0b93dc890 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b93ab810 con 0x55e0b93e1e00 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93d6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93d6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55324/0 (socket says 172.21.15.12:55324) 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69e7fff700 1 -- 172.21.15.12:0/2754888779 learned_addr learned my addr 172.21.15.12:0/2754888779 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69e77fe700 1 --2- 172.21.15.12:0/2754888779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 0x55e0b93f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/2754888779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 0x55e0b93e1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69e7fff700 1 -- 172.21.15.12:0/2754888779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 msgr2=0x55e0b93e1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.929+0000 7f69e7fff700 1 --2- 172.21.15.12:0/2754888779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 0x55e0b93e1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69e7fff700 1 -- 172.21.15.12:0/2754888779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 msgr2=0x55e0b93f1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69e7fff700 1 --2- 172.21.15.12:0/2754888779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 0x55e0b93f1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69e7fff700 1 -- 172.21.15.12:0/2754888779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0b9159b60 con 0x55e0b93e2720 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69e7fff700 1 --2- 172.21.15.12:0/2754888779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93d6d20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f69e00106e0 tx=0x7f69e0010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107de0fcae84910e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69de7fc700 1 -- 172.21.15.12:0/2754888779 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0015070 con 0x55e0b93e2720 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69de7fc700 1 -- 172.21.15.12:0/2754888779 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69e000f8d0 con 0x55e0b93e2720 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69de7fc700 1 -- 172.21.15.12:0/2754888779 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0019890 con 0x55e0b93e2720 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69f9f55b80 1 -- 172.21.15.12:0/2754888779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 msgr2=0x55e0b93d6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/2754888779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93d6d20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f69e00106e0 tx=0x7f69e0010710 comp rx=0 tx=0).stop 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69f9f55b80 1 -- 172.21.15.12:0/2754888779 shutdown_connections 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/2754888779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e2720 0x55e0b93d6d20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/2754888779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 0x55e0b93f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.930+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/2754888779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e1e00 0x55e0b93e1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 -- 172.21.15.12:0/2754888779 >> 172.21.15.12:0/2754888779 conn(0x55e0b933a210 msgr2=0x55e0b93e0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 -- 172.21.15.12:0/2754888779 shutdown_connections 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 -- 172.21.15.12:0/2754888779 wait complete. 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 -- start start 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 Processor -- start 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 0x55e0b93ead70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e1e00 0x55e0b93eb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e2720 0x55e0b93eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b944b4c0 con 0x55e0b93e1e00 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b944b630 con 0x55e0b93dc890 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b944b7f0 con 0x55e0b93e2720 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.931+0000 7f69f9f55b80 10 client.0 _ll_register_callbacks cb 0x55e0b92d1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e1e00 0x55e0b93eb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e1e00 0x55e0b93eb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55330/0 (socket says 172.21.15.12:55330) 2024-05-08T23:49:48.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e6ffd700 1 -- 172.21.15.12:0/1827944795 learned_addr learned my addr 172.21.15.12:0/1827944795 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e77fe700 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 0x55e0b93ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e7fff700 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e2720 0x55e0b93eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e6ffd700 1 -- 172.21.15.12:0/1827944795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e2720 msgr2=0x55e0b93eb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e2720 0x55e0b93eb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e6ffd700 1 -- 172.21.15.12:0/1827944795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 msgr2=0x55e0b93ead70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 0x55e0b93ead70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e6ffd700 1 -- 172.21.15.12:0/1827944795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e0010420 con 0x55e0b93e1e00 2024-05-08T23:49:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e7fff700 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e2720 0x55e0b93eb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e77fe700 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 0x55e0b93ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69e6ffd700 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e1e00 0x55e0b93eb2b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f69d009b130 tx=0x7f69d0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737e3af87aabf40b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69c6ffd700 1 -- 172.21.15.12:0/1827944795 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d00cbea0 con 0x55e0b93e1e00 2024-05-08T23:49:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69c6ffd700 1 -- 172.21.15.12:0/1827944795 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f69d00a3590 con 0x55e0b93e1e00 2024-05-08T23:49:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1827944795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69d8011d90 con 0x55e0b93e1e00 2024-05-08T23:49:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69c6ffd700 1 -- 172.21.15.12:0/1827944795 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69d009af70 con 0x55e0b93e1e00 2024-05-08T23:49:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69f9f55b80 10 client.16371 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.932+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1827944795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0b92f2560 con 0x55e0b93e1e00 2024-05-08T23:49:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.933+0000 7f69c6ffd700 1 -- 172.21.15.12:0/1827944795 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d8011d90 con 0x55e0b93e1e00 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.933+0000 7f69c6ffd700 1 -- 172.21.15.12:0/1827944795 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f69d0097030 con 0x55e0b93e1e00 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.933+0000 7f69f9f55b80 10 client.16371 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.933+0000 7f69f9f55b80 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.933+0000 7f69f9f55b80 20 client.16371 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.933+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0b944caa0 0x55e0b944ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.933+0000 7f69f9f55b80 4 client.16371 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.933+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1827944795 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e0b9324ec0 con 0x55e0b944caa0 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.933+0000 7f69e77fe700 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0b944caa0 0x55e0b944ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69c6ffd700 10 client.16371 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69e77fe700 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0b944caa0 0x55e0b944ee90 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1cb2d6c1cfccce7 server_cookie=4490daec71e79acb in_seq=0 out_seq=0 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69c6ffd700 1 -- 172.21.15.12:0/1827944795 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e0b9324ec0 con 0x55e0b944caa0 2024-05-08T23:49:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69c6ffd700 10 client.16371 handle_command_reply: tid=0 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 client.16371 shutdown 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1827944795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e1e00 msgr2=0x55e0b93eb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e1e00 0x55e0b93eb2b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f69d009b130 tx=0x7f69d0009310 comp rx=0 tx=0).stop 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1827944795 shutdown_connections 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0b93e2720 0x55e0b93eb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0b944caa0 0x55e0b944ee90 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0b93dc890 0x55e0b93ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 --2- 172.21.15.12:0/1827944795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0b93e1e00 0x55e0b93eb2b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69c6ffd700 0 client.16371 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1827944795 >> 172.21.15.12:0/1827944795 conn(0x55e0b933a210 msgr2=0x55e0b93e6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1827944795 shutdown_connections 2024-05-08T23:49:48.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 1 -- 172.21.15.12:0/1827944795 wait complete. 2024-05-08T23:49:48.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.934+0000 7f69f9f55b80 20 client.16371 trim_cache size 0 max 16384 2024-05-08T23:49:48.942 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:49:48.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 -- 172.21.15.12:0/115564578 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69d40a1290 msgr2=0x7f69d40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69d40a1290 0x7f69d40a3750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f69d8005fd0 tx=0x7f69d8005ee0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 -- 172.21.15.12:0/115564578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69e8105170 msgr2=0x7f69e8199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69e8105170 0x7f69e8199bc0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f69d00092e0 tx=0x7f69d00057f0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 -- 172.21.15.12:0/115564578 shutdown_connections 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69d40a1290 0x7f69d40a3750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e819df60 0x7f69e819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e8100060 0x7f69e8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 --2- 172.21.15.12:0/115564578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69e8105170 0x7f69e8199bc0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 -- 172.21.15.12:0/115564578 >> 172.21.15.12:0/115564578 conn(0x7f69e8076f30 msgr2=0x7f69e80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 -- 172.21.15.12:0/115564578 shutdown_connections 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:48.935+0000 7f69edd9b700 1 -- 172.21.15.12:0/115564578 wait complete. 2024-05-08T23:49:49.854 INFO:tasks.workunit.client.0.smithi012.stdout: 1 81292 4.33 MB/sec warmup 105 sec latency 1934.681 ms 2024-05-08T23:49:49.949 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:49:49.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:49:50.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.104+0000 7f8001684700 1 -- 172.21.15.12:0/1121658829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc0697e0 msgr2=0x7f7ffc069bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.104+0000 7f8001684700 1 --2- 172.21.15.12:0/1121658829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc0697e0 0x7f7ffc069bc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0099c0 tx=0x7f7fec009cd0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.104+0000 7f8001684700 1 -- 172.21.15.12:0/1121658829 shutdown_connections 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.104+0000 7f8001684700 1 --2- 172.21.15.12:0/1121658829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc1062a0 0x7f7ffc10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.104+0000 7f8001684700 1 --2- 172.21.15.12:0/1121658829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc0697e0 0x7f7ffc069bc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.104+0000 7f8001684700 1 --2- 172.21.15.12:0/1121658829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc06a190 0x7f7ffc105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.104+0000 7f8001684700 1 -- 172.21.15.12:0/1121658829 >> 172.21.15.12:0/1121658829 conn(0x7f7ffc076f40 msgr2=0x7f7ffc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.104+0000 7f8001684700 1 -- 172.21.15.12:0/1121658829 shutdown_connections 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.104+0000 7f8001684700 1 -- 172.21.15.12:0/1121658829 wait complete. 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.104+0000 7f8001684700 1 Processor -- start 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f8001684700 1 -- start start 2024-05-08T23:49:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f8001684700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc0697e0 0x7f7ffc0ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f8001684700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc1062a0 0x7f7ffc1003c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f8001684700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc100900 0x7f7ffc1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f8001684700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc1a5b60 con 0x7f7ffc100900 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f8001684700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc1a5cd0 con 0x7f7ffc1062a0 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f8001684700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc1a5e90 con 0x7f7ffc0697e0 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f7ffb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc100900 0x7f7ffc1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f7ffb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc100900 0x7f7ffc1a5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55358/0 (socket says 172.21.15.12:55358) 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f7ffb7fe700 1 -- 172.21.15.12:0/2841970106 learned_addr learned my addr 172.21.15.12:0/2841970106 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc0697e0 0x7f7ffc0ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f7ffa7fc700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc1062a0 0x7f7ffc1003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f7ffb7fe700 1 -- 172.21.15.12:0/2841970106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc0697e0 msgr2=0x7f7ffc0ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f7ffb7fe700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc0697e0 0x7f7ffc0ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f7ffb7fe700 1 -- 172.21.15.12:0/2841970106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc1062a0 msgr2=0x7f7ffc1003c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f7ffb7fe700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc1062a0 0x7f7ffc1003c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.105+0000 7f7ffb7fe700 1 -- 172.21.15.12:0/2841970106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec0095f0 con 0x7f7ffc100900 2024-05-08T23:49:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.106+0000 7f7ffb7fe700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc100900 0x7f7ffc1a5620 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4007ee0 tx=0x7f7ff400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.106+0000 7f7ff3fff700 1 -- 172.21.15.12:0/2841970106 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4017070 con 0x7f7ffc100900 2024-05-08T23:49:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.106+0000 7f8001684700 1 -- 172.21.15.12:0/2841970106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ffc1a6170 con 0x7f7ffc100900 2024-05-08T23:49:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.106+0000 7f7ff3fff700 1 -- 172.21.15.12:0/2841970106 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ff400f040 con 0x7f7ffc100900 2024-05-08T23:49:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.106+0000 7f7ff3fff700 1 -- 172.21.15.12:0/2841970106 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4005580 con 0x7f7ffc100900 2024-05-08T23:49:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.106+0000 7f8001684700 1 -- 172.21.15.12:0/2841970106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ffc1a6680 con 0x7f7ffc100900 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.108+0000 7f7ff3fff700 1 -- 172.21.15.12:0/2841970106 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7ff400db20 con 0x7f7ffc100900 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.109+0000 7f7ff3fff700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7fe80a12c0 0x7f7fe80a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.109+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7fe80a12c0 0x7f7fe80a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.109+0000 7f7ff3fff700 1 -- 172.21.15.12:0/2841970106 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7ff40cc360 con 0x7f7ffc100900 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.110+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7fe80a12c0 0x7f7fe80a3780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009990 tx=0x7f7fec003680 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.110+0000 7f800d83eb80 1 Processor -- start 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.110+0000 7f800d83eb80 1 -- start start 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5ae80 0x555596a5b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5b7a0 0x555596a553d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 0x555596a55d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f800d83eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555969ac1d0 con 0x555596a5b7a0 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f800d83eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555969a9b90 con 0x555596a5ae80 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f800d83eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555969a5480 con 0x555596a55910 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5b7a0 0x555596a553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5b7a0 0x555596a553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55366/0 (socket says 172.21.15.12:55366) 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/1543892292 learned_addr learned my addr 172.21.15.12:0/1543892292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffb7fe700 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 0x555596a55d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5ae80 0x555596a5b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/1543892292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 msgr2=0x555596a55d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffa7fc700 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 0x555596a55d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/1543892292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5ae80 msgr2=0x555596a5b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffa7fc700 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5ae80 0x555596a5b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/1543892292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555596826430 con 0x555596a5b7a0 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffb7fe700 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 0x555596a55d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5ae80 0x555596a5b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.111+0000 7f7ffa7fc700 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5b7a0 0x555596a553d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400bcf0 tx=0x7f7fe40103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a8fff09dd56842 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.112+0000 7f7ff17fa700 1 -- 172.21.15.12:0/1543892292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4010d10 con 0x555596a5b7a0 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.112+0000 7f7ff17fa700 1 -- 172.21.15.12:0/1543892292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fe4015070 con 0x555596a5b7a0 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.112+0000 7f7ff17fa700 1 -- 172.21.15.12:0/1543892292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4005020 con 0x555596a5b7a0 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.112+0000 7f800d83eb80 1 -- 172.21.15.12:0/1543892292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5b7a0 msgr2=0x555596a553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.112+0000 7f800d83eb80 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5b7a0 0x555596a553d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400bcf0 tx=0x7f7fe40103d0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 -- 172.21.15.12:0/1543892292 shutdown_connections 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 0x555596a55d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5ae80 0x555596a5b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 --2- 172.21.15.12:0/1543892292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5b7a0 0x555596a553d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 -- 172.21.15.12:0/1543892292 >> 172.21.15.12:0/1543892292 conn(0x5555969b3290 msgr2=0x555596a4fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 -- 172.21.15.12:0/1543892292 shutdown_connections 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 -- 172.21.15.12:0/1543892292 wait complete. 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 -- start start 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 Processor -- start 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a55910 0x555596a5eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a5b7a0 0x555596a5f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555969415e0 con 0x555596a5ae80 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555596941280 con 0x555596a55910 2024-05-08T23:49:50.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.113+0000 7f800d83eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555596828720 con 0x555596a5b7a0 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f800d83eb80 10 client.0 _ll_register_callbacks cb 0x55559694abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a5f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55370/0 (socket says 172.21.15.12:55370) 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/424034371 learned_addr learned my addr 172.21.15.12:0/424034371 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a55910 0x555596a5eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffb7fe700 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a5b7a0 0x555596a5f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/424034371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a5b7a0 msgr2=0x555596a5f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffa7fc700 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a5b7a0 0x555596a5f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/424034371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a55910 msgr2=0x555596a5eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffa7fc700 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a55910 0x555596a5eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/424034371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ffc1a6680 con 0x555596a5ae80 2024-05-08T23:49:50.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7ffa7fc700 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a5f090 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4004b70 tx=0x7f7fe4004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b07948db135f86d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7fe27fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7fe27fc700 1 -- 172.21.15.12:0/424034371 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4020070 con 0x555596a5ae80 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f7fe27fc700 1 -- 172.21.15.12:0/424034371 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7fe4017070 con 0x555596a5ae80 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.114+0000 7f800d83eb80 1 -- 172.21.15.12:0/424034371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555968fe2f0 con 0x555596a5ae80 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f7fe27fc700 1 -- 172.21.15.12:0/424034371 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fe4010ea0 con 0x555596a5ae80 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f800d83eb80 10 client.16383 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f800d83eb80 1 -- 172.21.15.12:0/424034371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f7fe400b920 con 0x555596a5ae80 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f7fe27fc700 1 -- 172.21.15.12:0/424034371 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555968fe2f0 con 0x555596a5ae80 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f7fe27fc700 1 -- 172.21.15.12:0/424034371 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7fe4011440 con 0x555596a5ae80 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f800d83eb80 10 client.16383 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f800d83eb80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f800d83eb80 20 client.16383 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f800d83eb80 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555596a6f2a0 0x555596a716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f800d83eb80 4 client.16383 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.115+0000 7f800d83eb80 1 -- 172.21.15.12:0/424034371 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555596a24890 con 0x555596a6f2a0 2024-05-08T23:49:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.116+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555596a6f2a0 0x555596a716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.119+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555596a6f2a0 0x555596a716c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9cbaa3ead00e31b server_cookie=562e5a0b217b072d in_seq=0 out_seq=0 2024-05-08T23:49:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.119+0000 7f7fe27fc700 10 client.16383 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.119+0000 7f7fe27fc700 1 -- 172.21.15.12:0/424034371 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555596a24890 con 0x555596a6f2a0 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.119+0000 7f7fe27fc700 10 client.16383 handle_command_reply: tid=0 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 client.16383 shutdown 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 -- 172.21.15.12:0/424034371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 msgr2=0x555596a5f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a5f090 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4004b70 tx=0x7f7fe4004a80 comp rx=0 tx=0).stop 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 -- 172.21.15.12:0/424034371 shutdown_connections 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a5b7a0 0x555596a5f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555596a6f2a0 0x555596a716c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a55910 0x555596a5eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 --2- 172.21.15.12:0/424034371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a5f090 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f7fe27fc700 0 client.16383 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 -- 172.21.15.12:0/424034371 >> 172.21.15.12:0/424034371 conn(0x5555969b3290 msgr2=0x555596a4f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 -- 172.21.15.12:0/424034371 shutdown_connections 2024-05-08T23:49:50.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 1 -- 172.21.15.12:0/424034371 wait complete. 2024-05-08T23:49:50.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.120+0000 7f800d83eb80 20 client.16383 trim_cache size 0 max 16384 2024-05-08T23:49:50.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f800d83eb80 1 Processor -- start 2024-05-08T23:49:50.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f800d83eb80 1 -- start start 2024-05-08T23:49:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a55910 0x555596a4fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5ae80 0x555596a66210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a5b7a0 0x555596a66750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f800d83eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555596a61ed0 con 0x555596a55910 2024-05-08T23:49:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f800d83eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555596a616e0 con 0x555596a5ae80 2024-05-08T23:49:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f800d83eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555596a4ce40 con 0x555596a5b7a0 2024-05-08T23:49:50.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f7ffaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a55910 0x555596a4fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f7ffaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a55910 0x555596a4fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55382/0 (socket says 172.21.15.12:55382) 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f7ffaffd700 1 -- 172.21.15.12:0/3695336945 learned_addr learned my addr 172.21.15.12:0/3695336945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f7ffb7fe700 1 --2- 172.21.15.12:0/3695336945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a5b7a0 0x555596a66750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f7ffaffd700 1 -- 172.21.15.12:0/3695336945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a5b7a0 msgr2=0x555596a66750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/3695336945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a5b7a0 0x555596a66750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f7ffaffd700 1 -- 172.21.15.12:0/3695336945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5ae80 msgr2=0x555596a66210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/3695336945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5ae80 0x555596a66210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.139+0000 7f7ffaffd700 1 -- 172.21.15.12:0/3695336945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555596a4c560 con 0x555596a55910 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.140+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/3695336945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a55910 0x555596a4fda0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0098e0 tx=0x7f7fec020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3363242d162c3404 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.140+0000 7f7ff1ffb700 1 -- 172.21.15.12:0/3695336945 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec005430 con 0x555596a55910 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.140+0000 7f7ff1ffb700 1 -- 172.21.15.12:0/3695336945 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fec01e040 con 0x555596a55910 2024-05-08T23:49:50.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.140+0000 7f7ff1ffb700 1 -- 172.21.15.12:0/3695336945 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec004a60 con 0x555596a55910 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.140+0000 7f800d83eb80 1 -- 172.21.15.12:0/3695336945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a55910 msgr2=0x555596a4fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.140+0000 7f800d83eb80 1 --2- 172.21.15.12:0/3695336945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a55910 0x555596a4fda0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0098e0 tx=0x7f7fec020000 comp rx=0 tx=0).stop 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 -- 172.21.15.12:0/3695336945 shutdown_connections 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 --2- 172.21.15.12:0/3695336945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a5b7a0 0x555596a66750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 --2- 172.21.15.12:0/3695336945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a55910 0x555596a4fda0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 --2- 172.21.15.12:0/3695336945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5ae80 0x555596a66210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 -- 172.21.15.12:0/3695336945 >> 172.21.15.12:0/3695336945 conn(0x5555969b3290 msgr2=0x555596a5d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 -- 172.21.15.12:0/3695336945 shutdown_connections 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 -- 172.21.15.12:0/3695336945 wait complete. 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 -- start start 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 Processor -- start 2024-05-08T23:49:50.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 0x555596a57270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5b7a0 0x555596a57cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555596a66b10 con 0x555596a5ae80 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555596ac5760 con 0x555596a5b7a0 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555596ac58a0 con 0x555596a55910 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.141+0000 7f800d83eb80 10 client.0 _ll_register_callbacks cb 0x55559694abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55394/0 (socket says 172.21.15.12:55394) 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/1886931421 learned_addr learned my addr 172.21.15.12:0/1886931421 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 0x555596a57270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffb7fe700 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5b7a0 0x555596a57cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/1886931421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 msgr2=0x555596a57270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffa7fc700 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 0x555596a57270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/1886931421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5b7a0 msgr2=0x555596a57cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffa7fc700 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5b7a0 0x555596a57cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffa7fc700 1 -- 172.21.15.12:0/1886931421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec02c040 con 0x555596a5ae80 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7ffa7fc700 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a577b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4004690 tx=0x7f7fe4011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a905d5dad82847c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7fe27fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7fe27fc700 1 -- 172.21.15.12:0/1886931421 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4020040 con 0x555596a5ae80 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7fe27fc700 1 -- 172.21.15.12:0/1886931421 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7fe4000e40 con 0x555596a5ae80 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f800d83eb80 1 -- 172.21.15.12:0/1886931421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fe4015070 con 0x555596a5ae80 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f7fe27fc700 1 -- 172.21.15.12:0/1886931421 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fe4010ea0 con 0x555596a5ae80 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f800d83eb80 10 client.16389 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:50.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.142+0000 7f800d83eb80 1 -- 172.21.15.12:0/1886931421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5555967d2be0 con 0x555596a5ae80 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.143+0000 7f7fe27fc700 1 -- 172.21.15.12:0/1886931421 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fe4015070 con 0x555596a5ae80 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.143+0000 7f7fe27fc700 1 -- 172.21.15.12:0/1886931421 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7fe4011d20 con 0x555596a5ae80 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.143+0000 7f800d83eb80 10 client.16389 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.143+0000 7f800d83eb80 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.143+0000 7f800d83eb80 20 client.16389 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.143+0000 7f800d83eb80 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555596ac6e00 0x555596ac9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.143+0000 7f800d83eb80 4 client.16389 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.143+0000 7f800d83eb80 1 -- 172.21.15.12:0/1886931421 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55559699df40 con 0x555596ac6e00 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.143+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555596ac6e00 0x555596ac9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.144+0000 7f7fe27fc700 10 client.16389 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.144+0000 7f7ffaffd700 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555596ac6e00 0x555596ac9220 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acac134f77a201ca server_cookie=32347f32e044948a in_seq=0 out_seq=0 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f7fe27fc700 1 -- 172.21.15.12:0/1886931421 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55559699df40 con 0x555596ac6e00 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f7fe27fc700 10 client.16389 handle_command_reply: tid=0 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 client.16389 shutdown 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 -- 172.21.15.12:0/1886931421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 msgr2=0x555596a577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a577b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4004690 tx=0x7f7fe4011670 comp rx=0 tx=0).stop 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 -- 172.21.15.12:0/1886931421 shutdown_connections 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555596ac6e00 0x555596ac9220 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555596a5b7a0 0x555596a57cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555596a55910 0x555596a57270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 --2- 172.21.15.12:0/1886931421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555596a5ae80 0x555596a577b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 -- 172.21.15.12:0/1886931421 >> 172.21.15.12:0/1886931421 conn(0x5555969b3290 msgr2=0x555596a59c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f7fe27fc700 0 client.16389 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:50.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 -- 172.21.15.12:0/1886931421 shutdown_connections 2024-05-08T23:49:50.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 1 -- 172.21.15.12:0/1886931421 wait complete. 2024-05-08T23:49:50.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.145+0000 7f800d83eb80 20 client.16389 trim_cache size 0 max 16384 2024-05-08T23:49:50.151 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:49:50.151 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:49:50.151 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "b3750343-be5b-4489-964a-d3f63f117dc3", 2024-05-08T23:49:50.151 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:49:50.151 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:49:50.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 -- 172.21.15.12:0/2841970106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7fe80a12c0 msgr2=0x7f7fe80a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7fe80a12c0 0x7f7fe80a3780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009990 tx=0x7f7fec003680 comp rx=0 tx=0).stop 2024-05-08T23:49:50.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 -- 172.21.15.12:0/2841970106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc100900 msgr2=0x7f7ffc1a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc100900 0x7f7ffc1a5620 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4007ee0 tx=0x7f7ff400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 -- 172.21.15.12:0/2841970106 shutdown_connections 2024-05-08T23:49:50.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7fe80a12c0 0x7f7fe80a3780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ffc100900 0x7f7ffc1a5620 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc0697e0 0x7f7ffc0ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 --2- 172.21.15.12:0/2841970106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ffc1062a0 0x7f7ffc1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 -- 172.21.15.12:0/2841970106 >> 172.21.15.12:0/2841970106 conn(0x7f7ffc076f40 msgr2=0x7f7ffc0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 -- 172.21.15.12:0/2841970106 shutdown_connections 2024-05-08T23:49:50.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.146+0000 7f8001684700 1 -- 172.21.15.12:0/2841970106 wait complete. 2024-05-08T23:49:50.155 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:49:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.302+0000 7fc861c82700 1 -- 172.21.15.12:0/2250232397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc85c105180 msgr2=0x7fc85c10cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.302+0000 7fc861c82700 1 --2- 172.21.15.12:0/2250232397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc85c105180 0x7fc85c10cd00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc85400b0a0 tx=0x7fc85400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.302+0000 7fc861c82700 1 -- 172.21.15.12:0/2250232397 shutdown_connections 2024-05-08T23:49:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.302+0000 7fc861c82700 1 --2- 172.21.15.12:0/2250232397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc85c105180 0x7fc85c10cd00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.302+0000 7fc861c82700 1 --2- 172.21.15.12:0/2250232397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c100060 0x7fc85c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.302+0000 7fc861c82700 1 --2- 172.21.15.12:0/2250232397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc85c100a10 0x7fc85c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.302+0000 7fc861c82700 1 -- 172.21.15.12:0/2250232397 >> 172.21.15.12:0/2250232397 conn(0x7fc85c076fa0 msgr2=0x7fc85c0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.302+0000 7fc861c82700 1 -- 172.21.15.12:0/2250232397 shutdown_connections 2024-05-08T23:49:50.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.302+0000 7fc861c82700 1 -- 172.21.15.12:0/2250232397 wait complete. 2024-05-08T23:49:50.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.303+0000 7fc861c82700 1 Processor -- start 2024-05-08T23:49:50.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.303+0000 7fc861c82700 1 -- start start 2024-05-08T23:49:50.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.303+0000 7fc861c82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc85c100060 0x7fc85c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.303+0000 7fc861c82700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc85c105180 0x7fc85c102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.303+0000 7fc861c82700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c102570 0x7fc85c1a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.303+0000 7fc861c82700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c1a5d00 con 0x7fc85c100060 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.303+0000 7fc861c82700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c1a5e70 con 0x7fc85c102570 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.303+0000 7fc861c82700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c1a6030 con 0x7fc85c105180 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc85b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc85c100060 0x7fc85c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc85b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc85c100060 0x7fc85c101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55418/0 (socket says 172.21.15.12:55418) 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc85b7fe700 1 -- 172.21.15.12:0/2275580334 learned_addr learned my addr 172.21.15.12:0/2275580334 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc85b7fe700 1 -- 172.21.15.12:0/2275580334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc85c105180 msgr2=0x7fc85c102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc85c105180 0x7fc85c102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc85b7fe700 1 -- 172.21.15.12:0/2275580334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c102570 msgr2=0x7fc85c1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c102570 0x7fc85c1a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc85b7fe700 1 -- 172.21.15.12:0/2275580334 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc854009d00 con 0x7fc85c100060 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc85c100060 0x7fc85c101af0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00b550 tx=0x7fc84c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc858ff9700 1 -- 172.21.15.12:0/2275580334 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c011830 con 0x7fc85c100060 2024-05-08T23:49:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc861c82700 1 -- 172.21.15.12:0/2275580334 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c1a6310 con 0x7fc85c100060 2024-05-08T23:49:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc858ff9700 1 -- 172.21.15.12:0/2275580334 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc84c011e50 con 0x7fc85c100060 2024-05-08T23:49:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc858ff9700 1 -- 172.21.15.12:0/2275580334 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00f2e0 con 0x7fc85c100060 2024-05-08T23:49:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.304+0000 7fc861c82700 1 -- 172.21.15.12:0/2275580334 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c1a6820 con 0x7fc85c100060 2024-05-08T23:49:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.307+0000 7fc858ff9700 1 -- 172.21.15.12:0/2275580334 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc84c010580 con 0x7fc85c100060 2024-05-08T23:49:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.308+0000 7fc858ff9700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8440a1290 0x7fc8440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.308+0000 7fc85affd700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8440a1290 0x7fc8440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.308+0000 7fc858ff9700 1 -- 172.21.15.12:0/2275580334 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc84c0cc6d0 con 0x7fc85c100060 2024-05-08T23:49:50.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.308+0000 7fc85affd700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8440a1290 0x7fc8440a3750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc85c103220 tx=0x7fc850006cd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc86de3cb80 1 Processor -- start 2024-05-08T23:49:50.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc86de3cb80 1 -- start start 2024-05-08T23:49:50.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 0x55ac86c37350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c37890 0x55ac86c37cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86b8e150 con 0x55ac86c37890 2024-05-08T23:49:50.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86b8bb10 con 0x55ac86c3ce00 2024-05-08T23:49:50.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86b87400 con 0x55ac86c3d720 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c37890 0x55ac86c37cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c37890 0x55ac86c37cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55424/0 (socket says 172.21.15.12:55424) 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc85bfff700 1 -- 172.21.15.12:0/3331343115 learned_addr learned my addr 172.21.15.12:0/3331343115 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/3331343115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc85affd700 1 --2- 172.21.15.12:0/3331343115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 0x55ac86c37350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc85bfff700 1 -- 172.21.15.12:0/3331343115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 msgr2=0x55ac86c37350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc85bfff700 1 --2- 172.21.15.12:0/3331343115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 0x55ac86c37350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc85bfff700 1 -- 172.21.15.12:0/3331343115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 msgr2=0x55ac86c3d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc85bfff700 1 --2- 172.21.15.12:0/3331343115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c3d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.309+0000 7fc85bfff700 1 -- 172.21.15.12:0/3331343115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac86a083b0 con 0x55ac86c37890 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.310+0000 7fc85bfff700 1 --2- 172.21.15.12:0/3331343115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c37890 0x55ac86c37cd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc854006070 tx=0x7fc85400cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff60bfc9d41fa67b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.310+0000 7fc841ffb700 1 -- 172.21.15.12:0/3331343115 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc854020070 con 0x55ac86c37890 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.310+0000 7fc841ffb700 1 -- 172.21.15.12:0/3331343115 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc854009280 con 0x55ac86c37890 2024-05-08T23:49:50.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.310+0000 7fc841ffb700 1 -- 172.21.15.12:0/3331343115 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85400e040 con 0x55ac86c37890 2024-05-08T23:49:50.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.311+0000 7fc86de3cb80 1 -- 172.21.15.12:0/3331343115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c37890 msgr2=0x55ac86c37cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.311+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/3331343115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c37890 0x55ac86c37cd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc854006070 tx=0x7fc85400cfb0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.311+0000 7fc86de3cb80 1 -- 172.21.15.12:0/3331343115 shutdown_connections 2024-05-08T23:49:50.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.311+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/3331343115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c37890 0x55ac86c37cd0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.311+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/3331343115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c3d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.311+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/3331343115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 0x55ac86c37350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.311+0000 7fc86de3cb80 1 -- 172.21.15.12:0/3331343115 >> 172.21.15.12:0/3331343115 conn(0x55ac86b95210 msgr2=0x55ac86c31df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.311+0000 7fc86de3cb80 1 -- 172.21.15.12:0/3331343115 shutdown_connections 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.311+0000 7fc86de3cb80 1 -- 172.21.15.12:0/3331343115 wait complete. 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.311+0000 7fc86de3cb80 1 -- start start 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc86de3cb80 1 Processor -- start 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 0x55ac86c40b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c41070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 0x55ac86c415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86b23560 con 0x55ac86c3d720 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86b23200 con 0x55ac86c3ce00 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86a0a6a0 con 0x55ac86c37890 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc86de3cb80 10 client.0 _ll_register_callbacks cb 0x55ac86b2cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 0x55ac86c415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 0x55ac86c415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55426/0 (socket says 172.21.15.12:55426) 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc85bfff700 1 -- 172.21.15.12:0/516620120 learned_addr learned my addr 172.21.15.12:0/516620120 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:50.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc85bfff700 1 -- 172.21.15.12:0/516620120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 msgr2=0x55ac86c40b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc85bfff700 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 0x55ac86c40b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc85bfff700 1 -- 172.21.15.12:0/516620120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 msgr2=0x55ac86c41070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.312+0000 7fc85bfff700 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c41070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc85bfff700 1 -- 172.21.15.12:0/516620120 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c10d570 con 0x55ac86c3d720 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc85bfff700 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 0x55ac86c415b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc854009850 tx=0x7fc854007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb813469d13ac2f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc83affd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc83affd700 1 -- 172.21.15.12:0/516620120 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc854028070 con 0x55ac86c3d720 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc83affd700 1 -- 172.21.15.12:0/516620120 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc854003c10 con 0x55ac86c3d720 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc86de3cb80 1 -- 172.21.15.12:0/516620120 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac86ae0270 con 0x55ac86c3d720 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc83affd700 1 -- 172.21.15.12:0/516620120 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc854020070 con 0x55ac86c3d720 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc86de3cb80 10 client.16401 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc86de3cb80 1 -- 172.21.15.12:0/516620120 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ac869b4b60 con 0x55ac86c3d720 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc83affd700 1 -- 172.21.15.12:0/516620120 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac86ae0270 con 0x55ac86c3d720 2024-05-08T23:49:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.313+0000 7fc83affd700 1 -- 172.21.15.12:0/516620120 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc85402b030 con 0x55ac86c3d720 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.314+0000 7fc86de3cb80 10 client.16401 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.314+0000 7fc86de3cb80 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.314+0000 7fc86de3cb80 20 client.16401 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.314+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ac86c50ee0 0x55ac86c53300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.314+0000 7fc86de3cb80 4 client.16401 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.314+0000 7fc86de3cb80 1 -- 172.21.15.12:0/516620120 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac86c06810 con 0x55ac86c50ee0 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.314+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ac86c50ee0 0x55ac86c53300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.314+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ac86c50ee0 0x55ac86c53300 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64fa1df72df403 server_cookie=28122d536273810a in_seq=0 out_seq=0 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.314+0000 7fc83affd700 10 client.16401 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc83affd700 1 -- 172.21.15.12:0/516620120 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ac86c06810 con 0x55ac86c50ee0 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc83affd700 10 client.16401 handle_command_reply: tid=0 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 client.16401 shutdown 2024-05-08T23:49:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 -- 172.21.15.12:0/516620120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 msgr2=0x55ac86c415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 0x55ac86c415b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc854009850 tx=0x7fc854007b60 comp rx=0 tx=0).stop 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 -- 172.21.15.12:0/516620120 shutdown_connections 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 0x55ac86c415b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 0x55ac86c40b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ac86c50ee0 0x55ac86c53300 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/516620120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c41070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc83affd700 0 client.16401 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 -- 172.21.15.12:0/516620120 >> 172.21.15.12:0/516620120 conn(0x55ac86b95210 msgr2=0x55ac86c315a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 -- 172.21.15.12:0/516620120 shutdown_connections 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.315+0000 7fc86de3cb80 1 -- 172.21.15.12:0/516620120 wait complete. 2024-05-08T23:49:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.316+0000 7fc86de3cb80 20 client.16401 trim_cache size 0 max 16384 2024-05-08T23:49:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.334+0000 7fc86de3cb80 1 Processor -- start 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.334+0000 7fc86de3cb80 1 -- start start 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.334+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c37890 0x55ac86c37350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.334+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3ce00 0x55ac86c4cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.334+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 0x55ac86c31d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.334+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86ae0270 con 0x55ac86c3ce00 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.334+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86c436c0 con 0x55ac86c37890 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.334+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86c4dd40 con 0x55ac86c3d720 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.334+0000 7fc85affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3ce00 0x55ac86c4cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3ce00 0x55ac86c4cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55440/0 (socket says 172.21.15.12:55440) 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85affd700 1 -- 172.21.15.12:0/3011131867 learned_addr learned my addr 172.21.15.12:0/3011131867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85bfff700 1 --2- 172.21.15.12:0/3011131867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 0x55ac86c31d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/3011131867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c37890 0x55ac86c37350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85affd700 1 -- 172.21.15.12:0/3011131867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 msgr2=0x55ac86c31d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85affd700 1 --2- 172.21.15.12:0/3011131867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 0x55ac86c31d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85affd700 1 -- 172.21.15.12:0/3011131867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c37890 msgr2=0x55ac86c37350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85affd700 1 --2- 172.21.15.12:0/3011131867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c37890 0x55ac86c37350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85affd700 1 -- 172.21.15.12:0/3011131867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac86b4d560 con 0x55ac86c3ce00 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85bfff700 1 --2- 172.21.15.12:0/3011131867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 0x55ac86c31d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc85affd700 1 --2- 172.21.15.12:0/3011131867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3ce00 0x55ac86c4cbc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0692c0 tx=0x7fc85000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1124f488de0cddf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc8427fc700 1 -- 172.21.15.12:0/3011131867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850012870 con 0x55ac86c3ce00 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc8427fc700 1 -- 172.21.15.12:0/3011131867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc85001b040 con 0x55ac86c3ce00 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.335+0000 7fc8427fc700 1 -- 172.21.15.12:0/3011131867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85001fd00 con 0x55ac86c3ce00 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.336+0000 7fc86de3cb80 1 -- 172.21.15.12:0/3011131867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3ce00 msgr2=0x55ac86c4cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.336+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/3011131867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3ce00 0x55ac86c4cbc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0692c0 tx=0x7fc85000e000 comp rx=0 tx=0).stop 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.336+0000 7fc86de3cb80 1 -- 172.21.15.12:0/3011131867 shutdown_connections 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.336+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/3011131867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c3d720 0x55ac86c31d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.336+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/3011131867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c37890 0x55ac86c37350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.336+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/3011131867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3ce00 0x55ac86c4cbc0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.336+0000 7fc86de3cb80 1 -- 172.21.15.12:0/3011131867 >> 172.21.15.12:0/3011131867 conn(0x55ac86b95210 msgr2=0x55ac86c2e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.336+0000 7fc86de3cb80 1 -- 172.21.15.12:0/3011131867 shutdown_connections 2024-05-08T23:49:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.336+0000 7fc86de3cb80 1 -- 172.21.15.12:0/3011131867 wait complete. 2024-05-08T23:49:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.336+0000 7fc86de3cb80 1 -- start start 2024-05-08T23:49:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc86de3cb80 1 Processor -- start 2024-05-08T23:49:50.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 0x55ac86c47330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c47870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc86de3cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 0x55ac86c47db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86ca63e0 con 0x55ac86c3d720 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86ca6520 con 0x55ac86c3ce00 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc86de3cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac86ca6660 con 0x55ac86c37890 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc86de3cb80 10 client.0 _ll_register_callbacks cb 0x55ac86b2cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 0x55ac86c47330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c47870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 0x55ac86c47db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 0x55ac86c47330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33156/0 (socket says 172.21.15.12:33156) 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c47870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50176/0 (socket says 172.21.15.12:50176) 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85b7fe700 1 -- 172.21.15.12:0/2726125215 learned_addr learned my addr 172.21.15.12:0/2726125215 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85b7fe700 1 -- 172.21.15.12:0/2726125215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 msgr2=0x55ac86c47870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c47870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85b7fe700 1 -- 172.21.15.12:0/2726125215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 msgr2=0x55ac86c47db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 0x55ac86c47db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.337+0000 7fc85b7fe700 1 -- 172.21.15.12:0/2726125215 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac869b4b60 con 0x55ac86c37890 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc85b7fe700 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 0x55ac86c47330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00bd10 tx=0x7fc84c00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf597915590b8b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc83affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc83affd700 1 -- 172.21.15.12:0/2726125215 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c084040 con 0x55ac86c37890 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc86de3cb80 1 -- 172.21.15.12:0/2726125215 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc854009280 con 0x55ac86c37890 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc83affd700 1 -- 172.21.15.12:0/2726125215 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc84c099930 con 0x55ac86c37890 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc83affd700 1 -- 172.21.15.12:0/2726125215 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc84c09d760 con 0x55ac86c37890 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc86de3cb80 10 client.26194 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc86de3cb80 1 -- 172.21.15.12:0/2726125215 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc850013700 con 0x55ac86c37890 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc83affd700 1 -- 172.21.15.12:0/2726125215 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc854009280 con 0x55ac86c37890 2024-05-08T23:49:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc83affd700 1 -- 172.21.15.12:0/2726125215 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc84c0a1920 con 0x55ac86c37890 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc86de3cb80 10 client.26194 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc86de3cb80 10 client.26194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.338+0000 7fc86de3cb80 20 client.26194 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.339+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ac86ca7ec0 0x55ac86caa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.339+0000 7fc86de3cb80 4 client.26194 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.339+0000 7fc86de3cb80 1 -- 172.21.15.12:0/2726125215 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ac86b7fec0 con 0x55ac86ca7ec0 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.339+0000 7fc85affd700 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ac86ca7ec0 0x55ac86caa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.339+0000 7fc85affd700 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ac86ca7ec0 0x55ac86caa2e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6734b2adccdd5d5 server_cookie=238bd4b854464cf2 in_seq=0 out_seq=0 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.339+0000 7fc83affd700 10 client.26194 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc83affd700 1 -- 172.21.15.12:0/2726125215 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ac86b7fec0 con 0x55ac86ca7ec0 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc83affd700 10 client.26194 handle_command_reply: tid=0 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 client.26194 shutdown 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 -- 172.21.15.12:0/2726125215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 msgr2=0x55ac86c47330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 0x55ac86c47330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00bd10 tx=0x7fc84c00bf60 comp rx=0 tx=0).stop 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 -- 172.21.15.12:0/2726125215 shutdown_connections 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac86c3d720 0x55ac86c47db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac86c37890 0x55ac86c47330 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ac86ca7ec0 0x55ac86caa2e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 --2- 172.21.15.12:0/2726125215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ac86c3ce00 0x55ac86c47870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 -- 172.21.15.12:0/2726125215 >> 172.21.15.12:0/2726125215 conn(0x55ac86b95210 msgr2=0x55ac86c3bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc83affd700 0 client.26194 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 -- 172.21.15.12:0/2726125215 shutdown_connections 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 1 -- 172.21.15.12:0/2726125215 wait complete. 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.340+0000 7fc86de3cb80 20 client.26194 trim_cache size 0 max 16384 2024-05-08T23:49:50.341 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:49:50.342 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:49:50.342 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:49:50.342 INFO:teuthology.orchestra.run.smithi012.stdout: "b3750343-be5b-4489-964a-d3f63f117dc3": { 2024-05-08T23:49:50.342 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:49:50.342 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "b3750343-be5b-4489-964a-d3f63f117dc3", 2024-05-08T23:49:50.342 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:49:50.342 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:49:50.342 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:49:50.342 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:49:50.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 -- 172.21.15.12:0/2275580334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8440a1290 msgr2=0x7fc8440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8440a1290 0x7fc8440a3750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc85c103220 tx=0x7fc850006cd0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 -- 172.21.15.12:0/2275580334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc85c100060 msgr2=0x7fc85c101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:50.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc85c100060 0x7fc85c101af0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00b550 tx=0x7fc84c00b910 comp rx=0 tx=0).stop 2024-05-08T23:49:50.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 -- 172.21.15.12:0/2275580334 shutdown_connections 2024-05-08T23:49:50.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8440a1290 0x7fc8440a3750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c102570 0x7fc85c1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc85c100060 0x7fc85c101af0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 --2- 172.21.15.12:0/2275580334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc85c105180 0x7fc85c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:50.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 -- 172.21.15.12:0/2275580334 >> 172.21.15.12:0/2275580334 conn(0x7fc85c076fa0 msgr2=0x7fc85c0697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:50.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 -- 172.21.15.12:0/2275580334 shutdown_connections 2024-05-08T23:49:50.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:49:50.341+0000 7fc861c82700 1 -- 172.21.15.12:0/2275580334 wait complete. 2024-05-08T23:49:50.349 INFO:tasks.cephfs.filesystem:scrub status for tag:b3750343-be5b-4489-964a-d3f63f117dc3 - {'path': '/', 'tag': 'b3750343-be5b-4489-964a-d3f63f117dc3', 'options': 'recursive,force'} 2024-05-08T23:49:50.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:50 smithi012 ceph-mon[42503]: pgmap v106: 97 pgs: 97 active+clean; 32 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 243 KiB/s wr, 57 op/s 2024-05-08T23:49:50.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:50 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:49:50.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:50 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:50.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:50 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:49:50.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:50 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:50.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:50 smithi204 ceph-mon[47373]: pgmap v106: 97 pgs: 97 active+clean; 32 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 243 KiB/s wr, 57 op/s 2024-05-08T23:49:50.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:50 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:49:50.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:50 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:50.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:50 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:49:50.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:50 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:50.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:50 smithi160 ceph-mon[46152]: pgmap v106: 97 pgs: 97 active+clean; 32 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 243 KiB/s wr, 57 op/s 2024-05-08T23:49:50.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:50 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:49:50.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:50 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:50.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:50 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:49:50.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:50 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:50.854 INFO:tasks.workunit.client.0.smithi012.stdout: 1 82192 4.35 MB/sec warmup 106 sec latency 2245.778 ms 2024-05-08T23:49:51.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:51 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:49:51.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:51 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:49:51.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:51 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:51.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:51 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:51.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:51 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:49:51.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:51 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:49:51.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:51 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:51.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:51 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:51.854 INFO:tasks.workunit.client.0.smithi012.stdout: 1 84394 4.41 MB/sec warmup 107 sec latency 287.411 ms 2024-05-08T23:49:51.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:51 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:49:51.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:51 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:49:51.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:51 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:51.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:51 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:49:52.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:52 smithi204 ceph-mon[47373]: pgmap v107: 97 pgs: 97 active+clean; 33 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 397 KiB/s wr, 95 op/s 2024-05-08T23:49:52.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:52 smithi160 ceph-mon[46152]: pgmap v107: 97 pgs: 97 active+clean; 33 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 397 KiB/s wr, 95 op/s 2024-05-08T23:49:52.854 INFO:tasks.workunit.client.0.smithi012.stdout: 1 84563 4.38 MB/sec warmup 108 sec latency 1194.961 ms 2024-05-08T23:49:52.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:52 smithi012 ceph-mon[42503]: pgmap v107: 97 pgs: 97 active+clean; 33 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 397 KiB/s wr, 95 op/s 2024-05-08T23:49:53.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:53 smithi204 ceph-mon[47373]: pgmap v108: 97 pgs: 97 active+clean; 33 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 183 KiB/s wr, 58 op/s 2024-05-08T23:49:53.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:53 smithi160 ceph-mon[46152]: pgmap v108: 97 pgs: 97 active+clean; 33 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 183 KiB/s wr, 58 op/s 2024-05-08T23:49:53.855 INFO:tasks.workunit.client.0.smithi012.stdout: 1 84563 4.34 MB/sec warmup 109 sec latency 1012.888 ms 2024-05-08T23:49:53.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:53 smithi012 ceph-mon[42503]: pgmap v108: 97 pgs: 97 active+clean; 33 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 183 KiB/s wr, 58 op/s 2024-05-08T23:49:54.854 INFO:tasks.workunit.client.0.smithi012.stdout: 1 84563 4.30 MB/sec warmup 110 sec latency 2013.009 ms 2024-05-08T23:49:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.617+0000 7f2c14ff2700 1 -- 172.21.15.12:0/1202538565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c10105e60 msgr2=0x7f2c10106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.617+0000 7f2c14ff2700 1 --2- 172.21.15.12:0/1202538565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c10105e60 0x7f2c10106240 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009990 tx=0x7f2c00009ca0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.617+0000 7f2c14ff2700 1 -- 172.21.15.12:0/1202538565 shutdown_connections 2024-05-08T23:49:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.617+0000 7f2c14ff2700 1 --2- 172.21.15.12:0/1202538565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c10100760 0x7f2c1010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.617+0000 7f2c14ff2700 1 --2- 172.21.15.12:0/1202538565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c10105e60 0x7f2c10106240 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.617+0000 7f2c14ff2700 1 --2- 172.21.15.12:0/1202538565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c100ffda0 0x7f2c10100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.617+0000 7f2c14ff2700 1 -- 172.21.15.12:0/1202538565 >> 172.21.15.12:0/1202538565 conn(0x7f2c10076f10 msgr2=0x7f2c10077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.617+0000 7f2c14ff2700 1 -- 172.21.15.12:0/1202538565 shutdown_connections 2024-05-08T23:49:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.617+0000 7f2c14ff2700 1 -- 172.21.15.12:0/1202538565 wait complete. 2024-05-08T23:49:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.618+0000 7f2c14ff2700 1 Processor -- start 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.618+0000 7f2c14ff2700 1 -- start start 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.618+0000 7f2c14ff2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c100ffda0 0x7f2c101972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.618+0000 7f2c14ff2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c10100760 0x7f2c10197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c14ff2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c10105e60 0x7f2c1019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c14ff2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1019c1e0 con 0x7f2c100ffda0 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c14ff2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1019c350 con 0x7f2c10105e60 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c14ff2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1019c510 con 0x7f2c10100760 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c0e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c100ffda0 0x7f2c101972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c0e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c100ffda0 0x7f2c101972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55468/0 (socket says 172.21.15.12:55468) 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c0e59c700 1 -- 172.21.15.12:0/2272746387 learned_addr learned my addr 172.21.15.12:0/2272746387 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c0ed9d700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c10105e60 0x7f2c1019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c0e59c700 1 -- 172.21.15.12:0/2272746387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c10100760 msgr2=0x7f2c10197800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c10100760 0x7f2c10197800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c0e59c700 1 -- 172.21.15.12:0/2272746387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c10105e60 msgr2=0x7f2c1019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c10105e60 0x7f2c1019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c0e59c700 1 -- 172.21.15.12:0/2272746387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c000095f0 con 0x7f2c100ffda0 2024-05-08T23:49:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c100ffda0 0x7f2c101972c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000b5c0 tx=0x7f2c0000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2bff7fe700 1 -- 172.21.15.12:0/2272746387 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0001d070 con 0x7f2c100ffda0 2024-05-08T23:49:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2bff7fe700 1 -- 172.21.15.12:0/2272746387 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0000fb60 con 0x7f2c100ffda0 2024-05-08T23:49:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2c14ff2700 1 -- 172.21.15.12:0/2272746387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1019c790 con 0x7f2c100ffda0 2024-05-08T23:49:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.619+0000 7f2bff7fe700 1 -- 172.21.15.12:0/2272746387 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00017650 con 0x7f2c100ffda0 2024-05-08T23:49:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.620+0000 7f2c14ff2700 1 -- 172.21.15.12:0/2272746387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c101a4030 con 0x7f2c100ffda0 2024-05-08T23:49:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.622+0000 7f2bff7fe700 1 -- 172.21.15.12:0/2272746387 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2c0000fcd0 con 0x7f2c100ffda0 2024-05-08T23:49:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.622+0000 7f2bff7fe700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2bf80a1290 0x7f2bf80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.622+0000 7f2bff7fe700 1 -- 172.21.15.12:0/2272746387 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2c000cdf90 con 0x7f2c100ffda0 2024-05-08T23:49:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.623+0000 7f2c0dd9b700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2bf80a1290 0x7f2bf80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.623+0000 7f2c0dd9b700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2bf80a1290 0x7f2bf80a3750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c10198810 tx=0x7f2c0400b3f0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.623+0000 7f2c211acb80 1 Processor -- start 2024-05-08T23:49:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c211acb80 1 -- start start 2024-05-08T23:49:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a9950 0x55cc379a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379aa270 0x55cc379a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c211acb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc378fb0f0 con 0x55cc379a4480 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c211acb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc378f8ab0 con 0x55cc379aa270 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c211acb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc378f43a0 con 0x55cc379a9950 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55482/0 (socket says 172.21.15.12:55482) 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/2735457275 learned_addr learned my addr 172.21.15.12:0/2735457275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0dd9b700 1 --2- 172.21.15.12:0/2735457275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379aa270 0x55cc379a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/2735457275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a9950 0x55cc379a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/2735457275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a9950 msgr2=0x55cc379a9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0ed9d700 1 --2- 172.21.15.12:0/2735457275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a9950 0x55cc379a9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/2735457275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379aa270 msgr2=0x55cc379a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0ed9d700 1 --2- 172.21.15.12:0/2735457275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379aa270 0x55cc379a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/2735457275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc37775350 con 0x55cc379a4480 2024-05-08T23:49:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.624+0000 7f2c0ed9d700 1 --2- 172.21.15.12:0/2735457275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a48a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2c080098f0 tx=0x7f2c08009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3ba2620437b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.625+0000 7f2bfcff9700 1 -- 172.21.15.12:0/2735457275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08019070 con 0x55cc379a4480 2024-05-08T23:49:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.625+0000 7f2bfcff9700 1 -- 172.21.15.12:0/2735457275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0800f040 con 0x55cc379a4480 2024-05-08T23:49:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.625+0000 7f2bfcff9700 1 -- 172.21.15.12:0/2735457275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08007c00 con 0x55cc379a4480 2024-05-08T23:49:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.625+0000 7f2c211acb80 1 -- 172.21.15.12:0/2735457275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 msgr2=0x55cc379a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.625+0000 7f2c211acb80 1 --2- 172.21.15.12:0/2735457275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a48a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2c080098f0 tx=0x7f2c08009c00 comp rx=0 tx=0).stop 2024-05-08T23:49:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.625+0000 7f2c211acb80 1 -- 172.21.15.12:0/2735457275 shutdown_connections 2024-05-08T23:49:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.625+0000 7f2c211acb80 1 --2- 172.21.15.12:0/2735457275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a48a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 --2- 172.21.15.12:0/2735457275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a9950 0x55cc379a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 --2- 172.21.15.12:0/2735457275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379aa270 0x55cc379a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 -- 172.21.15.12:0/2735457275 >> 172.21.15.12:0/2735457275 conn(0x55cc379021b0 msgr2=0x55cc3799e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 -- 172.21.15.12:0/2735457275 shutdown_connections 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 -- 172.21.15.12:0/2735457275 wait complete. 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 -- start start 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 Processor -- start 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a4480 0x55cc379b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc379ad550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc37890500 con 0x55cc379aa270 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc378901a0 con 0x55cc379a9950 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc37777640 con 0x55cc379a4480 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.626+0000 7f2c211acb80 10 client.0 _ll_register_callbacks cb 0x55cc37899ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c0ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c0ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55486/0 (socket says 172.21.15.12:55486) 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/2873086935 learned_addr learned my addr 172.21.15.12:0/2873086935 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c0dd9b700 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc379ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/2873086935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a4480 msgr2=0x55cc379b02d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c0ed9d700 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a4480 0x55cc379b02d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/2873086935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 msgr2=0x55cc379ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c0ed9d700 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc379ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/2873086935 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c08003c50 con 0x55cc379aa270 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c0ed9d700 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379adc90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2c08007820 tx=0x7f2c08007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b9355e1bd8d9fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2bf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/2873086935 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08019070 con 0x55cc379aa270 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c211acb80 1 -- 172.21.15.12:0/2873086935 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc3784d210 con 0x55cc379aa270 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/2873086935 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2c08009460 con 0x55cc379aa270 2024-05-08T23:49:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/2873086935 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c08004cd0 con 0x55cc379aa270 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c211acb80 10 client.16419 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.627+0000 7f2c211acb80 1 -- 172.21.15.12:0/2873086935 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cc37721b00 con 0x55cc379aa270 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.628+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/2873086935 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc3784d210 con 0x55cc379aa270 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.628+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/2873086935 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c08015480 con 0x55cc379aa270 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.628+0000 7f2c211acb80 10 client.16419 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.628+0000 7f2c211acb80 10 client.16419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.628+0000 7f2c211acb80 20 client.16419 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.628+0000 7f2c211acb80 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cc379bdbe0 0x55cc379c0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.628+0000 7f2c211acb80 4 client.16419 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.628+0000 7f2c211acb80 1 -- 172.21.15.12:0/2873086935 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc379737b0 con 0x55cc379bdbe0 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.628+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cc379bdbe0 0x55cc379c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.629+0000 7f2bf5ffb700 10 client.16419 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.629+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cc379bdbe0 0x55cc379c0000 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c03c319bdecc3f1f server_cookie=1f8e7e4509ce729d in_seq=0 out_seq=0 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.629+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/2873086935 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc379737b0 con 0x55cc379bdbe0 2024-05-08T23:49:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.629+0000 7f2bf5ffb700 10 client.16419 handle_command_reply: tid=0 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.629+0000 7f2c211acb80 1 client.16419 shutdown 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 1 -- 172.21.15.12:0/2873086935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 msgr2=0x55cc379adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379adc90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2c08007820 tx=0x7f2c08007ab0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 1 -- 172.21.15.12:0/2873086935 shutdown_connections 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379adc90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a4480 0x55cc379b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cc379bdbe0 0x55cc379c0000 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 1 --2- 172.21.15.12:0/2873086935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc379ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2bf5ffb700 0 client.16419 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 1 -- 172.21.15.12:0/2873086935 >> 172.21.15.12:0/2873086935 conn(0x55cc379021b0 msgr2=0x55cc379a7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 1 -- 172.21.15.12:0/2873086935 shutdown_connections 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 1 -- 172.21.15.12:0/2873086935 wait complete. 2024-05-08T23:49:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.630+0000 7f2c211acb80 20 client.16419 trim_cache size 0 max 16384 2024-05-08T23:49:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c211acb80 1 Processor -- start 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c211acb80 1 -- start start 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc3799e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379aa270 0x55cc379a4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c211acb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc379c51f0 con 0x55cc379a4480 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c211acb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc379a3140 con 0x55cc379a9950 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c211acb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc379a0630 con 0x55cc379aa270 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c0e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c0e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55502/0 (socket says 172.21.15.12:55502) 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c0e59c700 1 -- 172.21.15.12:0/1366961340 learned_addr learned my addr 172.21.15.12:0/1366961340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c0e59c700 1 -- 172.21.15.12:0/1366961340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379aa270 msgr2=0x55cc379a4860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/1366961340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379aa270 0x55cc379a4860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c0e59c700 1 -- 172.21.15.12:0/1366961340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 msgr2=0x55cc3799e910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/1366961340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc3799e910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c0e59c700 1 -- 172.21.15.12:0/1366961340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc378ba500 con 0x55cc379a4480 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.646+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/1366961340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a9d30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2c000cb780 tx=0x7f2c00022b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8acdc51d6e0718 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2bfd7fa700 1 -- 172.21.15.12:0/1366961340 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0009efa0 con 0x55cc379a4480 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2bfd7fa700 1 -- 172.21.15.12:0/1366961340 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c000a3560 con 0x55cc379a4480 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2bfd7fa700 1 -- 172.21.15.12:0/1366961340 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0009a660 con 0x55cc379a4480 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2c211acb80 1 -- 172.21.15.12:0/1366961340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 msgr2=0x55cc379a9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2c211acb80 1 --2- 172.21.15.12:0/1366961340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a9d30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2c000cb780 tx=0x7f2c00022b80 comp rx=0 tx=0).stop 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2c211acb80 1 -- 172.21.15.12:0/1366961340 shutdown_connections 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2c211acb80 1 --2- 172.21.15.12:0/1366961340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379aa270 0x55cc379a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2c211acb80 1 --2- 172.21.15.12:0/1366961340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379a4480 0x55cc379a9d30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2c211acb80 1 --2- 172.21.15.12:0/1366961340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc3799e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2c211acb80 1 -- 172.21.15.12:0/1366961340 >> 172.21.15.12:0/1366961340 conn(0x55cc379021b0 msgr2=0x55cc3799ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2c211acb80 1 -- 172.21.15.12:0/1366961340 shutdown_connections 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.647+0000 7f2c211acb80 1 -- 172.21.15.12:0/1366961340 wait complete. 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c211acb80 1 -- start start 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c211acb80 1 Processor -- start 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a4480 0x55cc379b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc379b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c211acb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c211acb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc379b3890 con 0x55cc379aa270 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c211acb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc379b3a00 con 0x55cc379a9950 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c211acb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc37a174f0 con 0x55cc379a4480 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c211acb80 10 client.0 _ll_register_callbacks cb 0x55cc37899ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:49:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c0ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c0ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55518/0 (socket says 172.21.15.12:55518) 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/349471315 learned_addr learned my addr 172.21.15.12:0/349471315 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c0dd9b700 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc379b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.648+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a4480 0x55cc379b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/349471315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a4480 msgr2=0x55cc379b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c0ed9d700 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a4480 0x55cc379b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/349471315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 msgr2=0x55cc379b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c0ed9d700 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc379b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c0ed9d700 1 -- 172.21.15.12:0/349471315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c101a4030 con 0x55cc379aa270 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c0dd9b700 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc379b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a4480 0x55cc379b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c0ed9d700 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379b3180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009c40 tx=0x7f2c08007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28622c11e9182b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2bf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:49:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/349471315 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08019050 con 0x55cc379aa270 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c211acb80 1 -- 172.21.15.12:0/349471315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c0800f040 con 0x55cc379aa270 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/349471315 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2c080073e0 con 0x55cc379aa270 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/349471315 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c08015b00 con 0x55cc379aa270 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c211acb80 10 client.16425 fetch_fsmap learned FSMap version 27 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2c211acb80 1 -- 172.21.15.12:0/349471315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cc37779c70 con 0x55cc379aa270 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/349471315 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0800f040 con 0x55cc379aa270 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.649+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/349471315 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c0801e630 con 0x55cc379aa270 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.650+0000 7f2c211acb80 10 client.16425 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.650+0000 7f2c211acb80 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.650+0000 7f2c211acb80 20 client.16425 populate_metadata read hostname 'smithi012' 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.650+0000 7f2c211acb80 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cc37a18910 0x55cc37a1ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.650+0000 7f2c211acb80 4 client.16425 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:49:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.650+0000 7f2c211acb80 1 -- 172.21.15.12:0/349471315 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc378ece60 con 0x55cc37a18910 2024-05-08T23:49:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.650+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cc37a18910 0x55cc37a1ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:49:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.650+0000 7f2c0e59c700 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cc37a18910 0x55cc37a1ad30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a738bdcb5f887665 server_cookie=d8823859387d84f2 in_seq=0 out_seq=0 2024-05-08T23:49:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.650+0000 7f2bf5ffb700 10 client.16425 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2bf5ffb700 1 -- 172.21.15.12:0/349471315 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc378ece60 con 0x55cc37a18910 2024-05-08T23:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2bf5ffb700 10 client.16425 handle_command_reply: tid=0 2024-05-08T23:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 client.16425 shutdown 2024-05-08T23:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 -- 172.21.15.12:0/349471315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 msgr2=0x55cc379b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379b3180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009c40 tx=0x7f2c08007bd0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 -- 172.21.15.12:0/349471315 shutdown_connections 2024-05-08T23:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc379aa270 0x55cc379b3180 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc379a4480 0x55cc379b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cc37a18910 0x55cc37a1ad30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 --2- 172.21.15.12:0/349471315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cc379a9950 0x55cc379b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2bf5ffb700 0 client.16425 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 -- 172.21.15.12:0/349471315 >> 172.21.15.12:0/349471315 conn(0x55cc379021b0 msgr2=0x55cc379a25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 -- 172.21.15.12:0/349471315 shutdown_connections 2024-05-08T23:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 1 -- 172.21.15.12:0/349471315 wait complete. 2024-05-08T23:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.677+0000 7f2c211acb80 20 client.16425 trim_cache size 0 max 16384 2024-05-08T23:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:49:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 -- 172.21.15.12:0/2272746387 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2bf80a1290 msgr2=0x7f2bf80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2bf80a1290 0x7f2bf80a3750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c10198810 tx=0x7f2c0400b3f0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 -- 172.21.15.12:0/2272746387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c100ffda0 msgr2=0x7f2c101972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c100ffda0 0x7f2c101972c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000b5c0 tx=0x7f2c0000f670 comp rx=0 tx=0).stop 2024-05-08T23:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 -- 172.21.15.12:0/2272746387 shutdown_connections 2024-05-08T23:49:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2bf80a1290 0x7f2bf80a3750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c10105e60 0x7f2c1019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c100ffda0 0x7f2c101972c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 --2- 172.21.15.12:0/2272746387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c10100760 0x7f2c10197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 -- 172.21.15.12:0/2272746387 >> 172.21.15.12:0/2272746387 conn(0x7f2c10076f10 msgr2=0x7f2c100fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 -- 172.21.15.12:0/2272746387 shutdown_connections 2024-05-08T23:49:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:49:55.678+0000 7f2c14ff2700 1 -- 172.21.15.12:0/2272746387 wait complete. 2024-05-08T23:49:55.854 INFO:tasks.workunit.client.0.smithi012.stdout: 1 84563 4.26 MB/sec warmup 111 sec latency 3013.133 ms 2024-05-08T23:49:56.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:56 smithi012 ceph-mon[42503]: pgmap v109: 97 pgs: 97 active+clean; 33 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 183 KiB/s wr, 58 op/s 2024-05-08T23:49:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:56 smithi204 ceph-mon[47373]: pgmap v109: 97 pgs: 97 active+clean; 33 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 183 KiB/s wr, 58 op/s 2024-05-08T23:49:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:56 smithi160 ceph-mon[46152]: pgmap v109: 97 pgs: 97 active+clean; 33 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 183 KiB/s wr, 58 op/s 2024-05-08T23:49:56.854 INFO:tasks.workunit.client.0.smithi012.stdout: 1 84563 4.23 MB/sec warmup 112 sec latency 4013.245 ms 2024-05-08T23:49:57.854 INFO:tasks.workunit.client.0.smithi012.stdout: 1 84810 4.19 MB/sec warmup 113 sec latency 4918.096 ms 2024-05-08T23:49:58.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:58 smithi204 ceph-mon[47373]: pgmap v110: 97 pgs: 97 active+clean; 34 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 262 KiB/s wr, 82 op/s 2024-05-08T23:49:58.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:58 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:49:58.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:58 smithi160 ceph-mon[46152]: pgmap v110: 97 pgs: 97 active+clean; 34 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 262 KiB/s wr, 82 op/s 2024-05-08T23:49:58.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:58 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:49:58.855 INFO:tasks.workunit.client.0.smithi012.stdout: 1 84842 4.16 MB/sec warmup 114 sec latency 982.721 ms 2024-05-08T23:49:58.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:58 smithi012 ceph-mon[42503]: pgmap v110: 97 pgs: 97 active+clean; 34 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 262 KiB/s wr, 82 op/s 2024-05-08T23:49:58.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:58 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:49:59.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:59 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:49:59.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:49:59 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:49:59.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:59 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:49:59.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:49:59 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:49:59.854 INFO:tasks.workunit.client.0.smithi012.stdout: 1 86368 4.19 MB/sec warmup 115 sec latency 990.806 ms 2024-05-08T23:49:59.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:59 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:49:59.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:49:59 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:50:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:00 smithi204 ceph-mon[47373]: pgmap v111: 97 pgs: 97 active+clean; 34 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 233 KiB/s wr, 61 op/s 2024-05-08T23:50:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:00 smithi204 ceph-mon[47373]: overall HEALTH_OK 2024-05-08T23:50:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:00 smithi160 ceph-mon[46152]: pgmap v111: 97 pgs: 97 active+clean; 34 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 233 KiB/s wr, 61 op/s 2024-05-08T23:50:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:00 smithi160 ceph-mon[46152]: overall HEALTH_OK 2024-05-08T23:50:00.855 INFO:tasks.workunit.client.0.smithi012.stdout: 1 88187 4.25 MB/sec warmup 116 sec latency 29.069 ms 2024-05-08T23:50:00.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:00 smithi012 ceph-mon[42503]: pgmap v111: 97 pgs: 97 active+clean; 34 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 233 KiB/s wr, 61 op/s 2024-05-08T23:50:00.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:00 smithi012 ceph-mon[42503]: overall HEALTH_OK 2024-05-08T23:50:01.855 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89448 4.27 MB/sec warmup 117 sec latency 221.286 ms 2024-05-08T23:50:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:02 smithi204 ceph-mon[47373]: pgmap v112: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-08T23:50:02.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:02 smithi160 ceph-mon[46152]: pgmap v112: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-08T23:50:02.855 INFO:tasks.workunit.client.0.smithi012.stdout: 1 92331 4.38 MB/sec warmup 118 sec latency 15.839 ms 2024-05-08T23:50:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:02 smithi012 ceph-mon[42503]: pgmap v112: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-08T23:50:03.855 INFO:tasks.workunit.client.0.smithi012.stdout: 1 93937 4.40 MB/sec warmup 119 sec latency 12.029 ms 2024-05-08T23:50:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:04 smithi204 ceph-mon[47373]: pgmap v113: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-08T23:50:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:04 smithi160 ceph-mon[46152]: pgmap v113: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-08T23:50:04.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:04 smithi012 ceph-mon[42503]: pgmap v113: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-08T23:50:05.855 INFO:tasks.workunit.client.0.smithi012.stdout: 1 98797 12.57 MB/sec execute 1 sec latency 13.504 ms 2024-05-08T23:50:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:06 smithi204 ceph-mon[47373]: pgmap v114: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-08T23:50:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:06 smithi160 ceph-mon[46152]: pgmap v114: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-08T23:50:06.855 INFO:tasks.workunit.client.0.smithi012.stdout: 1 100430 10.55 MB/sec execute 2 sec latency 10.946 ms 2024-05-08T23:50:06.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:06 smithi012 ceph-mon[42503]: pgmap v114: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-08T23:50:07.855 INFO:tasks.workunit.client.0.smithi012.stdout: 1 103124 12.65 MB/sec execute 3 sec latency 14.496 ms 2024-05-08T23:50:08.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:08 smithi204 ceph-mon[47373]: pgmap v115: 97 pgs: 97 active+clean; 42 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-05-08T23:50:08.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:08 smithi160 ceph-mon[46152]: pgmap v115: 97 pgs: 97 active+clean; 42 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-05-08T23:50:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.807+0000 7fcd54a9b700 1 -- 172.21.15.12:0/3848221401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd500ffda0 msgr2=0x7fcd50100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.807+0000 7fcd54a9b700 1 --2- 172.21.15.12:0/3848221401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd500ffda0 0x7fcd50100220 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009990 tx=0x7fcd44009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.807+0000 7fcd54a9b700 1 -- 172.21.15.12:0/3848221401 shutdown_connections 2024-05-08T23:50:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.807+0000 7fcd54a9b700 1 --2- 172.21.15.12:0/3848221401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd50100760 0x7fcd5010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.807+0000 7fcd54a9b700 1 --2- 172.21.15.12:0/3848221401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd50105e60 0x7fcd50106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.807+0000 7fcd54a9b700 1 --2- 172.21.15.12:0/3848221401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd500ffda0 0x7fcd50100220 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.807+0000 7fcd54a9b700 1 -- 172.21.15.12:0/3848221401 >> 172.21.15.12:0/3848221401 conn(0x7fcd50076f10 msgr2=0x7fcd50077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.807+0000 7fcd54a9b700 1 -- 172.21.15.12:0/3848221401 shutdown_connections 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.807+0000 7fcd54a9b700 1 -- 172.21.15.12:0/3848221401 wait complete. 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd54a9b700 1 Processor -- start 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd54a9b700 1 -- start start 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd54a9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd500ffda0 0x7fcd501994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd54a9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd50100760 0x7fcd501999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd54a9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd50105e60 0x7fcd5019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd54a9b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5019e3c0 con 0x7fcd500ffda0 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd54a9b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5019e530 con 0x7fcd50100760 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd54a9b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5019e6f0 con 0x7fcd50105e60 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd4e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd500ffda0 0x7fcd501994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd4e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd500ffda0 0x7fcd501994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43416/0 (socket says 172.21.15.12:43416) 2024-05-08T23:50:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd4e59c700 1 -- 172.21.15.12:0/2247975639 learned_addr learned my addr 172.21.15.12:0/2247975639 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.808+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd50100760 0x7fcd501999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd4ed9d700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd50105e60 0x7fcd5019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd4e59c700 1 -- 172.21.15.12:0/2247975639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd50105e60 msgr2=0x7fcd5019dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd50105e60 0x7fcd5019dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd4e59c700 1 -- 172.21.15.12:0/2247975639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd50100760 msgr2=0x7fcd501999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd50100760 0x7fcd501999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd4e59c700 1 -- 172.21.15.12:0/2247975639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440095f0 con 0x7fcd500ffda0 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd50100760 0x7fcd501999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd500ffda0 0x7fcd501994a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000e9a0 tx=0x7fcd4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd3f7fe700 1 -- 172.21.15.12:0/2247975639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40009790 con 0x7fcd500ffda0 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd54a9b700 1 -- 172.21.15.12:0/2247975639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5019e9d0 con 0x7fcd500ffda0 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd3f7fe700 1 -- 172.21.15.12:0/2247975639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd40010430 con 0x7fcd500ffda0 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd3f7fe700 1 -- 172.21.15.12:0/2247975639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40018680 con 0x7fcd500ffda0 2024-05-08T23:50:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.809+0000 7fcd54a9b700 1 -- 172.21.15.12:0/2247975639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd501a6210 con 0x7fcd500ffda0 2024-05-08T23:50:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.811+0000 7fcd3f7fe700 1 -- 172.21.15.12:0/2247975639 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcd400105a0 con 0x7fcd500ffda0 2024-05-08T23:50:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.812+0000 7fcd3f7fe700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcd380a1290 0x7fcd380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.812+0000 7fcd3f7fe700 1 -- 172.21.15.12:0/2247975639 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcd40014070 con 0x7fcd500ffda0 2024-05-08T23:50:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.812+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcd380a1290 0x7fcd380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.813+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcd380a1290 0x7fcd380a3750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400b2e0 tx=0x7fcd44005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.813+0000 7fcd60c55b80 1 Processor -- start 2024-05-08T23:50:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.813+0000 7fcd60c55b80 1 -- start start 2024-05-08T23:50:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.813+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 0x5591dedbcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.813+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbd270 0x5591dedb6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.813+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 0x5591dedb78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.813+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ded0e0f0 con 0x5591dedbd270 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.813+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ded0bab0 con 0x5591dedb7480 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.813+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ded073a0 con 0x5591dedbc950 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbd270 0x5591dedb6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 0x5591dedb78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbd270 0x5591dedb6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43424/0 (socket says 172.21.15.12:43424) 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4dd9b700 1 -- 172.21.15.12:0/2746672690 learned_addr learned my addr 172.21.15.12:0/2746672690 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/2746672690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 0x5591dedbcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4ed9d700 1 -- 172.21.15.12:0/2746672690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 msgr2=0x5591dedbcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4ed9d700 1 --2- 172.21.15.12:0/2746672690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 0x5591dedbcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4ed9d700 1 -- 172.21.15.12:0/2746672690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbd270 msgr2=0x5591dedb6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4ed9d700 1 --2- 172.21.15.12:0/2746672690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbd270 0x5591dedb6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4ed9d700 1 -- 172.21.15.12:0/2746672690 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591deb88350 con 0x5591dedb7480 2024-05-08T23:50:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd4ed9d700 1 --2- 172.21.15.12:0/2746672690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 0x5591dedb78a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcd48011580 tx=0x7fcd48009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1e1d09dd9099944 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd3cff9700 1 -- 172.21.15.12:0/2746672690 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4800e040 con 0x5591dedb7480 2024-05-08T23:50:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd3cff9700 1 -- 172.21.15.12:0/2746672690 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4800f040 con 0x5591dedb7480 2024-05-08T23:50:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.814+0000 7fcd3cff9700 1 -- 172.21.15.12:0/2746672690 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48004360 con 0x5591dedb7480 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.815+0000 7fcd60c55b80 1 -- 172.21.15.12:0/2746672690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 msgr2=0x5591dedb78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.815+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/2746672690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 0x5591dedb78a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcd48011580 tx=0x7fcd48009920 comp rx=0 tx=0).stop 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.815+0000 7fcd60c55b80 1 -- 172.21.15.12:0/2746672690 shutdown_connections 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.815+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/2746672690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 0x5591dedb78a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.815+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/2746672690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 0x5591dedbcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.815+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/2746672690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbd270 0x5591dedb6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.815+0000 7fcd60c55b80 1 -- 172.21.15.12:0/2746672690 >> 172.21.15.12:0/2746672690 conn(0x5591ded151b0 msgr2=0x5591dedb1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.815+0000 7fcd60c55b80 1 -- 172.21.15.12:0/2746672690 shutdown_connections 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.815+0000 7fcd60c55b80 1 -- 172.21.15.12:0/2746672690 wait complete. 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd60c55b80 1 -- start start 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd60c55b80 1 Processor -- start 2024-05-08T23:50:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 0x5591dedc0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedbd270 0x5591dedc0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591deca3500 con 0x5591dedb7480 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591deca31a0 con 0x5591dedbd270 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591deb8a640 con 0x5591dedbc950 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd60c55b80 10 client.0 _ll_register_callbacks cb 0x5591decacad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd4e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd4e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43434/0 (socket says 172.21.15.12:43434) 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.816+0000 7fcd4e59c700 1 -- 172.21.15.12:0/1293883315 learned_addr learned my addr 172.21.15.12:0/1293883315 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 0x5591dedc0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd4ed9d700 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedbd270 0x5591dedc0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd4e59c700 1 -- 172.21.15.12:0/1293883315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 msgr2=0x5591dedc0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 0x5591dedc0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd4e59c700 1 -- 172.21.15.12:0/1293883315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedbd270 msgr2=0x5591dedc0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedbd270 0x5591dedc0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd4e59c700 1 -- 172.21.15.12:0/1293883315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd48011230 con 0x5591dedb7480 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 0x5591dedc0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc32a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcd40099cc0 tx=0x7fcd40099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b374295bd0c0cddc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd35ffb700 1 -- 172.21.15.12:0/1293883315 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400996b0 con 0x5591dedb7480 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd35ffb700 1 -- 172.21.15.12:0/1293883315 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcd400a2440 con 0x5591dedb7480 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1293883315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591dec60210 con 0x5591dedb7480 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd35ffb700 1 -- 172.21.15.12:0/1293883315 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd400b8170 con 0x5591dedb7480 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd60c55b80 10 client.16437 fetch_fsmap learned FSMap version 27 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.817+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1293883315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5591deb34b00 con 0x5591dedb7480 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd35ffb700 1 -- 172.21.15.12:0/1293883315 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd40063080 con 0x5591dedb7480 2024-05-08T23:50:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd35ffb700 1 -- 172.21.15.12:0/1293883315 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcd40098f20 con 0x5591dedb7480 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd60c55b80 10 client.16437 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd60c55b80 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd60c55b80 20 client.16437 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591dedd0bc0 0x5591dedd2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd60c55b80 4 client.16437 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1293883315 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591ded867b0 con 0x5591dedd0bc0 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591dedd0bc0 0x5591dedd2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd35ffb700 10 client.16437 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.818+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591dedd0bc0 0x5591dedd2fe0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd9ac5d442174dd server_cookie=1fdbf3aec3475feb in_seq=0 out_seq=0 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd35ffb700 1 -- 172.21.15.12:0/1293883315 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5591ded867b0 con 0x5591dedd0bc0 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd35ffb700 10 client.16437 handle_command_reply: tid=0 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 client.16437 shutdown 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1293883315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 msgr2=0x5591dedc32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc32a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcd40099cc0 tx=0x7fcd40099fc0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1293883315 shutdown_connections 2024-05-08T23:50:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591dedd0bc0 0x5591dedd2fe0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedbd270 0x5591dedc0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc32a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/1293883315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbc950 0x5591dedc0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd35ffb700 0 client.16437 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1293883315 >> 172.21.15.12:0/1293883315 conn(0x5591ded151b0 msgr2=0x5591dedbb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1293883315 shutdown_connections 2024-05-08T23:50:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1293883315 wait complete. 2024-05-08T23:50:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.819+0000 7fcd60c55b80 20 client.16437 trim_cache size 0 max 16384 2024-05-08T23:50:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd60c55b80 1 Processor -- start 2024-05-08T23:50:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd60c55b80 1 -- start start 2024-05-08T23:50:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 0x5591dedc0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbc950 0x5591dedb3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 0x5591dedd2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591dedce300 con 0x5591dedbc950 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591dedb6240 con 0x5591dedb7480 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591dedcd590 con 0x5591dedbd270 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd4dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbc950 0x5591dedb3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd4e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 0x5591dedc0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd4dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbc950 0x5591dedb3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43438/0 (socket says 172.21.15.12:43438) 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd4dd9b700 1 -- 172.21.15.12:0/1673475681 learned_addr learned my addr 172.21.15.12:0/1673475681 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.836+0000 7fcd4ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 0x5591dedd2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.837+0000 7fcd4e59c700 1 -- 172.21.15.12:0/1673475681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 msgr2=0x5591dedd2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.837+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/1673475681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 0x5591dedd2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.837+0000 7fcd4e59c700 1 -- 172.21.15.12:0/1673475681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbc950 msgr2=0x5591dedb3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.837+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/1673475681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbc950 0x5591dedb3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.837+0000 7fcd4e59c700 1 -- 172.21.15.12:0/1673475681 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591deccd500 con 0x5591dedb7480 2024-05-08T23:50:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.837+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/1673475681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbc950 0x5591dedb3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.837+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/1673475681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 0x5591dedc0d70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcd400a1f10 tx=0x7fcd40010a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6894da986a8df5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.837+0000 7fcd3d7fa700 1 -- 172.21.15.12:0/1673475681 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400a2440 con 0x5591dedb7480 2024-05-08T23:50:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.837+0000 7fcd3d7fa700 1 -- 172.21.15.12:0/1673475681 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5591dec60210 con 0x5591dedb7480 2024-05-08T23:50:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.837+0000 7fcd3d7fa700 1 -- 172.21.15.12:0/1673475681 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40094ea0 con 0x5591dedb7480 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1673475681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 msgr2=0x5591dedc0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/1673475681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 0x5591dedc0d70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcd400a1f10 tx=0x7fcd40010a90 comp rx=0 tx=0).stop 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1673475681 shutdown_connections 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/1673475681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 0x5591dedd2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/1673475681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedb7480 0x5591dedc0d70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/1673475681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedbc950 0x5591dedb3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1673475681 >> 172.21.15.12:0/1673475681 conn(0x5591ded151b0 msgr2=0x5591dedbfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1673475681 shutdown_connections 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 -- 172.21.15.12:0/1673475681 wait complete. 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 -- start start 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.838+0000 7fcd60c55b80 1 Processor -- start 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedbc950 0x5591dedc8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd60c55b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 0x5591dedba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591dedbaad0 con 0x5591dedb7480 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591dedbac40 con 0x5591dedbc950 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd60c55b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591dedbae00 con 0x5591dedbd270 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd60c55b80 10 client.0 _ll_register_callbacks cb 0x5591decacad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 0x5591dedba3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedbc950 0x5591dedc8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43448/0 (socket says 172.21.15.12:43448) 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4e59c700 1 -- 172.21.15.12:0/3374780052 learned_addr learned my addr 172.21.15.12:0/3374780052 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4e59c700 1 -- 172.21.15.12:0/3374780052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 msgr2=0x5591dedba3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 0x5591dedba3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4e59c700 1 -- 172.21.15.12:0/3374780052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedbc950 msgr2=0x5591dedc8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedbc950 0x5591dedc8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4e59c700 1 -- 172.21.15.12:0/3374780052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5010d4f0 con 0x5591dedb7480 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4ed9d700 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 0x5591dedba3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd4e59c700 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc7c10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000e970 tx=0x7fcd40010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a5e64d50d15582 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.839+0000 7fcd35ffb700 1 -- 172.21.15.12:0/3374780052 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400996b0 con 0x5591dedb7480 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd35ffb700 1 -- 172.21.15.12:0/3374780052 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcd400b2b70 con 0x5591dedb7480 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd60c55b80 1 -- 172.21.15.12:0/3374780052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd4800f040 con 0x5591dedb7480 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd35ffb700 1 -- 172.21.15.12:0/3374780052 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd400a24c0 con 0x5591dedb7480 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd60c55b80 10 client.16443 fetch_fsmap learned FSMap version 27 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd60c55b80 1 -- 172.21.15.12:0/3374780052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5591deb34b00 con 0x5591dedb7480 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd35ffb700 1 -- 172.21.15.12:0/3374780052 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4007b080 con 0x5591dedb7480 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd35ffb700 1 -- 172.21.15.12:0/3374780052 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcd4009ed10 con 0x5591dedb7480 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd60c55b80 10 client.16443 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:50:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd60c55b80 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd60c55b80 20 client.16443 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591dee270c0 0x5591dee294b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd60c55b80 4 client.16443 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:50:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd60c55b80 1 -- 172.21.15.12:0/3374780052 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5591decffe60 con 0x5591dee270c0 2024-05-08T23:50:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.840+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591dee270c0 0x5591dee294b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.841+0000 7fcd35ffb700 10 client.16443 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.841+0000 7fcd4dd9b700 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591dee270c0 0x5591dee294b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689cc9a078811d72 server_cookie=23b2a8023d9e3e7e in_seq=0 out_seq=0 2024-05-08T23:50:08.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 104532 10.87 MB/sec execute 4 sec latency 80.704 ms 2024-05-08T23:50:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd35ffb700 1 -- 172.21.15.12:0/3374780052 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5591decffe60 con 0x5591dee270c0 2024-05-08T23:50:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd35ffb700 10 client.16443 handle_command_reply: tid=0 2024-05-08T23:50:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd60c55b80 1 client.16443 shutdown 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd60c55b80 1 -- 172.21.15.12:0/3374780052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 msgr2=0x5591dedc7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc7c10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000e970 tx=0x7fcd40010c80 comp rx=0 tx=0).stop 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd60c55b80 1 -- 172.21.15.12:0/3374780052 shutdown_connections 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591dedbd270 0x5591dedba3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591dedb7480 0x5591dedc7c10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591dee270c0 0x5591dee294b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd60c55b80 1 --2- 172.21.15.12:0/3374780052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591dedbc950 0x5591dedc8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd35ffb700 0 client.16443 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.897+0000 7fcd60c55b80 1 -- 172.21.15.12:0/3374780052 >> 172.21.15.12:0/3374780052 conn(0x5591ded151b0 msgr2=0x5591dedb6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd60c55b80 1 -- 172.21.15.12:0/3374780052 shutdown_connections 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd60c55b80 1 -- 172.21.15.12:0/3374780052 wait complete. 2024-05-08T23:50:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd60c55b80 20 client.16443 trim_cache size 0 max 16384 2024-05-08T23:50:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:50:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:50:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:50:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 -- 172.21.15.12:0/2247975639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcd380a1290 msgr2=0x7fcd380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcd380a1290 0x7fcd380a3750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400b2e0 tx=0x7fcd44005960 comp rx=0 tx=0).stop 2024-05-08T23:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 -- 172.21.15.12:0/2247975639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd500ffda0 msgr2=0x7fcd501994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd500ffda0 0x7fcd501994a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000e9a0 tx=0x7fcd4000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 -- 172.21.15.12:0/2247975639 shutdown_connections 2024-05-08T23:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcd380a1290 0x7fcd380a3750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd50105e60 0x7fcd5019dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd500ffda0 0x7fcd501994a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 --2- 172.21.15.12:0/2247975639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd50100760 0x7fcd501999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 -- 172.21.15.12:0/2247975639 >> 172.21.15.12:0/2247975639 conn(0x7fcd50076f10 msgr2=0x7fcd50068310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.898+0000 7fcd54a9b700 1 -- 172.21.15.12:0/2247975639 shutdown_connections 2024-05-08T23:50:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:08.899+0000 7fcd54a9b700 1 -- 172.21.15.12:0/2247975639 wait complete. 2024-05-08T23:50:08.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:08 smithi012 ceph-mon[42503]: pgmap v115: 97 pgs: 97 active+clean; 42 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-05-08T23:50:09.439 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:50:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-08T23:50:09.439 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:50:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-08T23:50:09.439 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:50:09 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/905050622 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aef1e661a0 con 0x55aef1e3d000 2024-05-08T23:50:09.439 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:50:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-08T23:50:09.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:09 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-08T23:50:09.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:09 smithi204 ceph-mon[47373]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T23:50:09.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:09 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-08T23:50:09.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:09 smithi160 ceph-mon[46152]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T23:50:09.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 106780 11.41 MB/sec execute 5 sec latency 30.236 ms 2024-05-08T23:50:09.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:09 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-08T23:50:09.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:09 smithi012 ceph-mon[42503]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T23:50:10.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:10 smithi204 ceph-mon[47373]: pgmap v117: 97 pgs: 97 active+clean; 42 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 254 op/s 2024-05-08T23:50:10.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:10 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/131208281,v1:172.21.15.204:6837/131208281] up:boot 2024-05-08T23:50:10.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:10 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:50:10.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:10 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:50:10.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:10 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:50:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:10 smithi160 ceph-mon[46152]: pgmap v117: 97 pgs: 97 active+clean; 42 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 254 op/s 2024-05-08T23:50:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:10 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/131208281,v1:172.21.15.204:6837/131208281] up:boot 2024-05-08T23:50:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:10 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:50:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:10 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:50:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:10 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:50:10.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 109052 10.90 MB/sec execute 6 sec latency 13.139 ms 2024-05-08T23:50:10.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:10 smithi012 ceph-mon[42503]: pgmap v117: 97 pgs: 97 active+clean; 42 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 254 op/s 2024-05-08T23:50:10.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:10 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/131208281,v1:172.21.15.204:6837/131208281] up:boot 2024-05-08T23:50:10.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:10 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:50:10.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:10 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:50:10.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:10 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:50:11.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 110662 10.94 MB/sec execute 7 sec latency 13.324 ms 2024-05-08T23:50:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:12 smithi204 ceph-mon[47373]: pgmap v118: 97 pgs: 97 active+clean; 40 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:12 smithi160 ceph-mon[46152]: pgmap v118: 97 pgs: 97 active+clean; 40 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:12.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 113620 11.77 MB/sec execute 8 sec latency 15.742 ms 2024-05-08T23:50:12.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:12 smithi012 ceph-mon[42503]: pgmap v118: 97 pgs: 97 active+clean; 40 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:13.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 115031 11.19 MB/sec execute 9 sec latency 43.962 ms 2024-05-08T23:50:14.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:14 smithi204 ceph-mon[47373]: pgmap v119: 97 pgs: 97 active+clean; 40 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:14 smithi160 ceph-mon[46152]: pgmap v119: 97 pgs: 97 active+clean; 40 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:14.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117451 11.47 MB/sec execute 10 sec latency 15.788 ms 2024-05-08T23:50:14.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:14 smithi012 ceph-mon[42503]: pgmap v119: 97 pgs: 97 active+clean; 40 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:15.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 119842 11.43 MB/sec execute 11 sec latency 12.579 ms 2024-05-08T23:50:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:16 smithi204 ceph-mon[47373]: pgmap v120: 97 pgs: 97 active+clean; 40 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:16 smithi160 ceph-mon[46152]: pgmap v120: 97 pgs: 97 active+clean; 40 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:16.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121355 11.17 MB/sec execute 12 sec latency 16.276 ms 2024-05-08T23:50:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:16 smithi012 ceph-mon[42503]: pgmap v120: 97 pgs: 97 active+clean; 40 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:17.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124185 11.64 MB/sec execute 13 sec latency 15.098 ms 2024-05-08T23:50:18.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:18 smithi204 ceph-mon[47373]: pgmap v121: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:18.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:18 smithi160 ceph-mon[46152]: pgmap v121: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:18.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 125692 11.29 MB/sec execute 14 sec latency 15.166 ms 2024-05-08T23:50:18.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:18 smithi012 ceph-mon[42503]: pgmap v121: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-05-08T23:50:19.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 128007 11.45 MB/sec execute 15 sec latency 35.584 ms 2024-05-08T23:50:20.351 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:50:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.514+0000 7f459afaf700 1 -- 172.21.15.12:0/4107780828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4594100060 msgr2=0x7f4594100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.514+0000 7f459afaf700 1 --2- 172.21.15.12:0/4107780828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4594100060 0x7f4594100440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4588009990 tx=0x7f4588009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.514+0000 7f459afaf700 1 -- 172.21.15.12:0/4107780828 shutdown_connections 2024-05-08T23:50:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.514+0000 7f459afaf700 1 --2- 172.21.15.12:0/4107780828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4594105160 0x7f459410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.514+0000 7f459afaf700 1 --2- 172.21.15.12:0/4107780828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4594100060 0x7f4594100440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.514+0000 7f459afaf700 1 --2- 172.21.15.12:0/4107780828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4594100a10 0x7f4594104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.514+0000 7f459afaf700 1 -- 172.21.15.12:0/4107780828 >> 172.21.15.12:0/4107780828 conn(0x7f4594076f40 msgr2=0x7f4594077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.514+0000 7f459afaf700 1 -- 172.21.15.12:0/4107780828 shutdown_connections 2024-05-08T23:50:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.514+0000 7f459afaf700 1 -- 172.21.15.12:0/4107780828 wait complete. 2024-05-08T23:50:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.515+0000 7f459afaf700 1 Processor -- start 2024-05-08T23:50:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459afaf700 1 -- start start 2024-05-08T23:50:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459afaf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4594100060 0x7f45941995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459afaf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4594105160 0x7f4594199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459afaf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f459419dec0 0x7f459419e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459afaf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459419e940 con 0x7f459419dec0 2024-05-08T23:50:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459afaf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459419eab0 con 0x7f4594105160 2024-05-08T23:50:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459afaf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459419ec70 con 0x7f4594100060 2024-05-08T23:50:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459954c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f459419dec0 0x7f459419e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459954c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f459419dec0 0x7f459419e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56778/0 (socket says 172.21.15.12:56778) 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459954c700 1 -- 172.21.15.12:0/2652198591 learned_addr learned my addr 172.21.15.12:0/2652198591 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f4593fff700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4594105160 0x7f4594199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f4598d4b700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4594100060 0x7f45941995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459954c700 1 -- 172.21.15.12:0/2652198591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4594100060 msgr2=0x7f45941995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459954c700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4594100060 0x7f45941995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459954c700 1 -- 172.21.15.12:0/2652198591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4594105160 msgr2=0x7f4594199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459954c700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4594105160 0x7f4594199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.516+0000 7f459954c700 1 -- 172.21.15.12:0/2652198591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45880095f0 con 0x7f459419dec0 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.517+0000 7f459954c700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f459419dec0 0x7f459419e340 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f458c00ef30 tx=0x7f458c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.517+0000 7f4591ffb700 1 -- 172.21.15.12:0/2652198591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c018070 con 0x7f459419dec0 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.517+0000 7f459afaf700 1 -- 172.21.15.12:0/2652198591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45941a6170 con 0x7f459419dec0 2024-05-08T23:50:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.517+0000 7f4591ffb700 1 -- 172.21.15.12:0/2652198591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458c00f040 con 0x7f459419dec0 2024-05-08T23:50:20.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.517+0000 7f4591ffb700 1 -- 172.21.15.12:0/2652198591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c00cd00 con 0x7f459419dec0 2024-05-08T23:50:20.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.517+0000 7f459afaf700 1 -- 172.21.15.12:0/2652198591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45941a66a0 con 0x7f459419dec0 2024-05-08T23:50:20.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.519+0000 7f4591ffb700 1 -- 172.21.15.12:0/2652198591 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f458c003a00 con 0x7f459419dec0 2024-05-08T23:50:20.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.520+0000 7f4591ffb700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45840a1290 0x7f45840a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.520+0000 7f4591ffb700 1 -- 172.21.15.12:0/2652198591 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f458c0cc7c0 con 0x7f459419dec0 2024-05-08T23:50:20.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.520+0000 7f4598d4b700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45840a1290 0x7f45840a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f4598d4b700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45840a1290 0x7f45840a3750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4588006010 tx=0x7f4588005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:20.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f45a7169b80 1 Processor -- start 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f45a7169b80 1 -- start start 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b31240ee00 0x55b31240f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312409350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b312409890 0x55b312409cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f45a7169b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b312360150 con 0x55b312409890 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f45a7169b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b31235db10 con 0x55b31240ee00 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f45a7169b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b312359400 con 0x55b31240f720 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f459954c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b312409890 0x55b312409cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.521+0000 7f4598d4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b31240ee00 0x55b31240f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f4598d4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b31240ee00 0x55b31240f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:40852/0 (socket says 172.21.15.12:40852) 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f459954c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b312409890 0x55b312409cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56790/0 (socket says 172.21.15.12:56790) 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f4598d4b700 1 -- 172.21.15.12:0/1159162893 learned_addr learned my addr 172.21.15.12:0/1159162893 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:20.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f4593fff700 1 --2- 172.21.15.12:0/1159162893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312409350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f459954c700 1 -- 172.21.15.12:0/1159162893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 msgr2=0x55b312409350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f459954c700 1 --2- 172.21.15.12:0/1159162893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312409350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f459954c700 1 -- 172.21.15.12:0/1159162893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b31240ee00 msgr2=0x55b31240f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f459954c700 1 --2- 172.21.15.12:0/1159162893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b31240ee00 0x55b31240f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f459954c700 1 -- 172.21.15.12:0/1159162893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3121da3b0 con 0x55b312409890 2024-05-08T23:50:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f4598d4b700 1 --2- 172.21.15.12:0/1159162893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b31240ee00 0x55b31240f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f459954c700 1 --2- 172.21.15.12:0/1159162893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b312409890 0x55b312409cd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f458c0c9ef0 tx=0x7f458c095db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e389eee65b227c04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f457effd700 1 -- 172.21.15.12:0/1159162893 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c0b7ea0 con 0x55b312409890 2024-05-08T23:50:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f457effd700 1 -- 172.21.15.12:0/1159162893 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458c084040 con 0x55b312409890 2024-05-08T23:50:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.522+0000 7f457effd700 1 -- 172.21.15.12:0/1159162893 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c09d770 con 0x55b312409890 2024-05-08T23:50:20.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.523+0000 7f45a7169b80 1 -- 172.21.15.12:0/1159162893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b312409890 msgr2=0x55b312409cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.523+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1159162893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b312409890 0x55b312409cd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f458c0c9ef0 tx=0x7f458c095db0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.523+0000 7f45a7169b80 1 -- 172.21.15.12:0/1159162893 shutdown_connections 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.523+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1159162893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b312409890 0x55b312409cd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.523+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1159162893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b31240ee00 0x55b31240f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.523+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1159162893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312409350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.523+0000 7f45a7169b80 1 -- 172.21.15.12:0/1159162893 >> 172.21.15.12:0/1159162893 conn(0x55b312367210 msgr2=0x55b312403df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.523+0000 7f45a7169b80 1 -- 172.21.15.12:0/1159162893 shutdown_connections 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.523+0000 7f45a7169b80 1 -- 172.21.15.12:0/1159162893 wait complete. 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f45a7169b80 1 -- start start 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f45a7169b80 1 Processor -- start 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312412b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240ee00 0x55b3124130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240f720 0x55b312413610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f45a7169b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3122f5560 con 0x55b31240f720 2024-05-08T23:50:20.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f45a7169b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3122f5200 con 0x55b312409890 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f45a7169b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3121dc6a0 con 0x55b31240ee00 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f45a7169b80 10 client.0 _ll_register_callbacks cb 0x55b3122feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f4593fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240ee00 0x55b3124130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f4598d4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312412b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f4593fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240ee00 0x55b3124130d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34530/0 (socket says 172.21.15.12:34530) 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f4593fff700 1 -- 172.21.15.12:0/1520761799 learned_addr learned my addr 172.21.15.12:0/1520761799 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.524+0000 7f459954c700 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240f720 0x55b312413610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f459954c700 1 -- 172.21.15.12:0/1520761799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240ee00 msgr2=0x55b3124130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f459954c700 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240ee00 0x55b3124130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f459954c700 1 -- 172.21.15.12:0/1520761799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 msgr2=0x55b312412b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f459954c700 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312412b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f459954c700 1 -- 172.21.15.12:0/1520761799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459410d550 con 0x55b31240f720 2024-05-08T23:50:20.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f4598d4b700 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312412b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f459954c700 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240f720 0x55b312413610 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f458c099640 tx=0x7f458c099dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68e8e9b5831e5f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f4573fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f4573fff700 1 -- 172.21.15.12:0/1520761799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c0b6070 con 0x55b31240f720 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f4573fff700 1 -- 172.21.15.12:0/1520761799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f458c01b320 con 0x55b31240f720 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f45a7169b80 1 -- 172.21.15.12:0/1520761799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3122b2270 con 0x55b31240f720 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f4573fff700 1 -- 172.21.15.12:0/1520761799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f458c0b7ea0 con 0x55b31240f720 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f45a7169b80 10 client.16458 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.525+0000 7f45a7169b80 1 -- 172.21.15.12:0/1520761799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f45941a66a0 con 0x55b31240f720 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f4573fff700 1 -- 172.21.15.12:0/1520761799 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458c07c080 con 0x55b31240f720 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f4573fff700 1 -- 172.21.15.12:0/1520761799 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f458c0a18a0 con 0x55b31240f720 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f45a7169b80 10 client.16458 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f45a7169b80 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f45a7169b80 20 client.16458 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b312422dc0 0x55b3124251b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f45a7169b80 4 client.16458 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f45a7169b80 1 -- 172.21.15.12:0/1520761799 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3123d8810 con 0x55b312422dc0 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f4598d4b700 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b312422dc0 0x55b3124251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f4573fff700 10 client.16458 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.526+0000 7f4598d4b700 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b312422dc0 0x55b3124251b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66469883b076185 server_cookie=1e6b3d58ab64a795 in_seq=0 out_seq=0 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f4573fff700 1 -- 172.21.15.12:0/1520761799 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b3123d8810 con 0x55b312422dc0 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f4573fff700 10 client.16458 handle_command_reply: tid=0 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 client.16458 shutdown 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 -- 172.21.15.12:0/1520761799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240f720 msgr2=0x55b312413610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240f720 0x55b312413610 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f458c099640 tx=0x7f458c099dd0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 -- 172.21.15.12:0/1520761799 shutdown_connections 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240f720 0x55b312413610 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b312422dc0 0x55b3124251b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312412b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1520761799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240ee00 0x55b3124130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f4573fff700 0 client.16458 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 -- 172.21.15.12:0/1520761799 >> 172.21.15.12:0/1520761799 conn(0x55b312367210 msgr2=0x55b3124035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 -- 172.21.15.12:0/1520761799 shutdown_connections 2024-05-08T23:50:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 1 -- 172.21.15.12:0/1520761799 wait complete. 2024-05-08T23:50:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.527+0000 7f45a7169b80 20 client.16458 trim_cache size 0 max 16384 2024-05-08T23:50:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.546+0000 7f45a7169b80 1 Processor -- start 2024-05-08T23:50:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.546+0000 7f45a7169b80 1 -- start start 2024-05-08T23:50:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.546+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312409350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.546+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b31240e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.546+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312403d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.546+0000 7f45a7169b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b312415e50 con 0x55b31240ee00 2024-05-08T23:50:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.546+0000 7f45a7169b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b312414140 con 0x55b312409890 2024-05-08T23:50:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.546+0000 7f45a7169b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b312415c80 con 0x55b31240f720 2024-05-08T23:50:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4593fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b31240e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4598d4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312409350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4593fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b31240e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56820/0 (socket says 172.21.15.12:56820) 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4593fff700 1 -- 172.21.15.12:0/1200959051 learned_addr learned my addr 172.21.15.12:0/1200959051 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f459954c700 1 --2- 172.21.15.12:0/1200959051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312403d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4598d4b700 1 -- 172.21.15.12:0/1200959051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 msgr2=0x55b312403d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4598d4b700 1 --2- 172.21.15.12:0/1200959051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312403d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4598d4b700 1 -- 172.21.15.12:0/1200959051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 msgr2=0x55b31240e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4598d4b700 1 --2- 172.21.15.12:0/1200959051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b31240e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4598d4b700 1 -- 172.21.15.12:0/1200959051 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b312186b60 con 0x55b312409890 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4593fff700 1 --2- 172.21.15.12:0/1200959051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b31240e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f4598d4b700 1 --2- 172.21.15.12:0/1200959051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312409350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4588000c00 tx=0x7f4588020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c7e8f63801f059 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f457f7fe700 1 -- 172.21.15.12:0/1200959051 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45880048a0 con 0x55b312409890 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f457f7fe700 1 -- 172.21.15.12:0/1200959051 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458801e040 con 0x55b312409890 2024-05-08T23:50:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.547+0000 7f457f7fe700 1 -- 172.21.15.12:0/1200959051 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588036880 con 0x55b312409890 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.548+0000 7f45a7169b80 1 -- 172.21.15.12:0/1200959051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 msgr2=0x55b312409350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.548+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1200959051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312409350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4588000c00 tx=0x7f4588020000 comp rx=0 tx=0).stop 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.548+0000 7f45a7169b80 1 -- 172.21.15.12:0/1200959051 shutdown_connections 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.548+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1200959051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312403d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.548+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1200959051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312409350 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.548+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1200959051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b31240e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.548+0000 7f45a7169b80 1 -- 172.21.15.12:0/1200959051 >> 172.21.15.12:0/1200959051 conn(0x55b312367210 msgr2=0x55b31240dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.548+0000 7f45a7169b80 1 -- 172.21.15.12:0/1200959051 shutdown_connections 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.548+0000 7f45a7169b80 1 -- 172.21.15.12:0/1200959051 wait complete. 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.549+0000 7f45a7169b80 1 -- start start 2024-05-08T23:50:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.549+0000 7f45a7169b80 1 Processor -- start 2024-05-08T23:50:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.549+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312418390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.549+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b3124188d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.549+0000 7f45a7169b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312478250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.549+0000 7f45a7169b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b312419020 con 0x55b31240ee00 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.549+0000 7f45a7169b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b312478790 con 0x55b312409890 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.549+0000 7f45a7169b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b312478950 con 0x55b31240f720 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f45a7169b80 10 client.0 _ll_register_callbacks cb 0x55b3122feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f4593fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b3124188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f459954c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312478250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f4598d4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312418390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f459954c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312478250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34558/0 (socket says 172.21.15.12:34558) 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f459954c700 1 -- 172.21.15.12:0/1972014961 learned_addr learned my addr 172.21.15.12:0/1972014961 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f4593fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b3124188d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56824/0 (socket says 172.21.15.12:56824) 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f459954c700 1 -- 172.21.15.12:0/1972014961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 msgr2=0x55b312418390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f459954c700 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312418390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f459954c700 1 -- 172.21.15.12:0/1972014961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 msgr2=0x55b3124188d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f459954c700 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b3124188d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f459954c700 1 -- 172.21.15.12:0/1972014961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4580009c20 con 0x55b31240f720 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f4598d4b700 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312418390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f4593fff700 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b3124188d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f459954c700 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312478250 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f458c0a1800 tx=0x7f458c0060e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c3fb7a32565d985 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f4573fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f4573fff700 1 -- 172.21.15.12:0/1972014961 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c0b6060 con 0x55b31240f720 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f4573fff700 1 -- 172.21.15.12:0/1972014961 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f458c01b320 con 0x55b31240f720 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.550+0000 7f45a7169b80 1 -- 172.21.15.12:0/1972014961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f458c084040 con 0x55b31240f720 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f4573fff700 1 -- 172.21.15.12:0/1972014961 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f458c0b7ea0 con 0x55b31240f720 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f45a7169b80 10 client.26242 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f45a7169b80 1 -- 172.21.15.12:0/1972014961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f458800ba20 con 0x55b31240f720 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f4573fff700 1 -- 172.21.15.12:0/1972014961 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458c084040 con 0x55b31240f720 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f4573fff700 1 -- 172.21.15.12:0/1972014961 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f458c0a18a0 con 0x55b31240f720 2024-05-08T23:50:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f45a7169b80 10 client.26242 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f45a7169b80 10 client.26242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f45a7169b80 20 client.26242 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b312479de0 0x55b31247c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f45a7169b80 4 client.26242 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f45a7169b80 1 -- 172.21.15.12:0/1972014961 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b312351ec0 con 0x55b312479de0 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.551+0000 7f4598d4b700 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b312479de0 0x55b31247c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f4573fff700 10 client.26242 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f4598d4b700 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b312479de0 0x55b31247c200 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2bb82fa1e3d9562 server_cookie=5815b743930fe4f6 in_seq=0 out_seq=0 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f4573fff700 1 -- 172.21.15.12:0/1972014961 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b312351ec0 con 0x55b312479de0 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f4573fff700 10 client.26242 handle_command_reply: tid=0 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 client.26242 shutdown 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 -- 172.21.15.12:0/1972014961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 msgr2=0x55b312478250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312478250 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f458c0a1800 tx=0x7f458c0060e0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 -- 172.21.15.12:0/1972014961 shutdown_connections 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b31240f720 0x55b312478250 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b312479de0 0x55b31247c200 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b312409890 0x55b312418390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f4573fff700 0 client.26242 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 --2- 172.21.15.12:0/1972014961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b31240ee00 0x55b3124188d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 -- 172.21.15.12:0/1972014961 >> 172.21.15.12:0/1972014961 conn(0x55b312367210 msgr2=0x55b312413470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 -- 172.21.15.12:0/1972014961 shutdown_connections 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 1 -- 172.21.15.12:0/1972014961 wait complete. 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.552+0000 7f45a7169b80 20 client.26242 trim_cache size 0 max 16384 2024-05-08T23:50:20.554 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:50:20.555 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:50:20.555 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:50:20.555 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 -- 172.21.15.12:0/2652198591 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45840a1290 msgr2=0x7f45840a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45840a1290 0x7f45840a3750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4588006010 tx=0x7f4588005960 comp rx=0 tx=0).stop 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 -- 172.21.15.12:0/2652198591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f459419dec0 msgr2=0x7f459419e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f459419dec0 0x7f459419e340 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f458c00ef30 tx=0x7f458c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 -- 172.21.15.12:0/2652198591 shutdown_connections 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45840a1290 0x7f45840a3750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f459419dec0 0x7f459419e340 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4594100060 0x7f45941995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 --2- 172.21.15.12:0/2652198591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4594105160 0x7f4594199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 -- 172.21.15.12:0/2652198591 >> 172.21.15.12:0/2652198591 conn(0x7f4594076f40 msgr2=0x7f45940ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 -- 172.21.15.12:0/2652198591 shutdown_connections 2024-05-08T23:50:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.553+0000 7f459afaf700 1 -- 172.21.15.12:0/2652198591 wait complete. 2024-05-08T23:50:20.562 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:50:20.563 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:50:20.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:20 smithi012 ceph-mon[42503]: pgmap v122: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 298 op/s 2024-05-08T23:50:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.711+0000 7fbcae949700 1 -- 172.21.15.12:0/1972524265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810b3d0 msgr2=0x7fbca810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.711+0000 7fbcae949700 1 --2- 172.21.15.12:0/1972524265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810b3d0 0x7fbca810d870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009f90 tx=0x7fbca000b330 comp rx=0 tx=0).stop 2024-05-08T23:50:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.711+0000 7fbcae949700 1 -- 172.21.15.12:0/1972524265 shutdown_connections 2024-05-08T23:50:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.711+0000 7fbcae949700 1 --2- 172.21.15.12:0/1972524265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810b3d0 0x7fbca810d870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.711+0000 7fbcae949700 1 --2- 172.21.15.12:0/1972524265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca8102000 0x7fbca81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.711+0000 7fbcae949700 1 --2- 172.21.15.12:0/1972524265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca8102920 0x7fbca810ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.711+0000 7fbcae949700 1 -- 172.21.15.12:0/1972524265 >> 172.21.15.12:0/1972524265 conn(0x7fbca80fb7e0 msgr2=0x7fbca80fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.711+0000 7fbcae949700 1 -- 172.21.15.12:0/1972524265 shutdown_connections 2024-05-08T23:50:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.711+0000 7fbcae949700 1 -- 172.21.15.12:0/1972524265 wait complete. 2024-05-08T23:50:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbcae949700 1 Processor -- start 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbcae949700 1 -- start start 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbcae949700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca8102000 0x7fbca80ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbcae949700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca8102920 0x7fbca8100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbcae949700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810b3d0 0x7fbca81006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbcae949700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca81a5760 con 0x7fbca8102000 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbcae949700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca81a58d0 con 0x7fbca810b3d0 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbcae949700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca81a5a90 con 0x7fbca8102920 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbca7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca8102000 0x7fbca80ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbca7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca8102000 0x7fbca80ffc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56838/0 (socket says 172.21.15.12:56838) 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbca7fff700 1 -- 172.21.15.12:0/3359151282 learned_addr learned my addr 172.21.15.12:0/3359151282 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.712+0000 7fbcacee6700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810b3d0 0x7fbca81006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.713+0000 7fbca77fe700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca8102920 0x7fbca8100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.713+0000 7fbca7fff700 1 -- 172.21.15.12:0/3359151282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca8102920 msgr2=0x7fbca8100190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.713+0000 7fbca7fff700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca8102920 0x7fbca8100190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.713+0000 7fbca7fff700 1 -- 172.21.15.12:0/3359151282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810b3d0 msgr2=0x7fbca81006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.713+0000 7fbca7fff700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810b3d0 0x7fbca81006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.713+0000 7fbca7fff700 1 -- 172.21.15.12:0/3359151282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0095f0 con 0x7fbca8102000 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.713+0000 7fbcacee6700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810b3d0 0x7fbca81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.713+0000 7fbca7fff700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca8102000 0x7fbca80ffc50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800d950 tx=0x7fbc9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.714+0000 7fbca57fa700 1 -- 172.21.15.12:0/3359151282 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98009940 con 0x7fbca8102000 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.714+0000 7fbcae949700 1 -- 172.21.15.12:0/3359151282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca0009c40 con 0x7fbca8102000 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.714+0000 7fbca57fa700 1 -- 172.21.15.12:0/3359151282 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc98010460 con 0x7fbca8102000 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.714+0000 7fbca57fa700 1 -- 172.21.15.12:0/3359151282 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9800f620 con 0x7fbca8102000 2024-05-08T23:50:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.714+0000 7fbcae949700 1 -- 172.21.15.12:0/3359151282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca81a6140 con 0x7fbca8102000 2024-05-08T23:50:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.715+0000 7fbcae949700 1 -- 172.21.15.12:0/3359151282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca8106550 con 0x7fbca8102000 2024-05-08T23:50:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.716+0000 7fbca57fa700 1 -- 172.21.15.12:0/3359151282 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbc980108d0 con 0x7fbca8102000 2024-05-08T23:50:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.717+0000 7fbca57fa700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc900a1290 0x7fbc900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.717+0000 7fbca77fe700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc900a1290 0x7fbc900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.717+0000 7fbca57fa700 1 -- 172.21.15.12:0/3359151282 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fbc980ccb60 con 0x7fbca8102000 2024-05-08T23:50:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.718+0000 7fbca77fe700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc900a1290 0x7fbc900a3750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbca8101380 tx=0x7fbc9c009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:20.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.721+0000 7fbca57fa700 1 -- 172.21.15.12:0/3359151282 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbc98095f90 con 0x7fbca8102000 2024-05-08T23:50:20.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:20 smithi204 ceph-mon[47373]: pgmap v122: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 298 op/s 2024-05-08T23:50:20.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:20 smithi160 ceph-mon[46152]: pgmap v122: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 298 op/s 2024-05-08T23:50:20.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 129644 11.10 MB/sec execute 16 sec latency 23.046 ms 2024-05-08T23:50:20.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.996+0000 7fbcae949700 1 -- 172.21.15.12:0/3359151282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbca81011d0 con 0x7fbca8102000 2024-05-08T23:50:20.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:20.997+0000 7fbca57fa700 1 -- 172.21.15.12:0/3359151282 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10876 (secure 0 0 0) 0x7fbc98016090 con 0x7fbca8102000 2024-05-08T23:50:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:50:20.993 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:50:09.452032+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_26224":{"gid":26224,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/131208281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":131208281},{"type":"v1","addr":"172.21.15.204:6837","nonce":131208281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:50:20.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 -- 172.21.15.12:0/3359151282 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc900a1290 msgr2=0x7fbc900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc900a1290 0x7fbc900a3750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbca8101380 tx=0x7fbc9c009310 comp rx=0 tx=0).stop 2024-05-08T23:50:20.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 -- 172.21.15.12:0/3359151282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca8102000 msgr2=0x7fbca80ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:20.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca8102000 0x7fbca80ffc50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800d950 tx=0x7fbc9800dc60 comp rx=0 tx=0).stop 2024-05-08T23:50:20.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 -- 172.21.15.12:0/3359151282 shutdown_connections 2024-05-08T23:50:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc900a1290 0x7fbc900a3750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca810b3d0 0x7fbca81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca8102000 0x7fbca80ffc50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 --2- 172.21.15.12:0/3359151282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca8102920 0x7fbca8100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 -- 172.21.15.12:0/3359151282 >> 172.21.15.12:0/3359151282 conn(0x7fbca80fb7e0 msgr2=0x7fbca80fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 -- 172.21.15.12:0/3359151282 shutdown_connections 2024-05-08T23:50:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.000+0000 7fbcae949700 1 -- 172.21.15.12:0/3359151282 wait complete. 2024-05-08T23:50:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-05-08T23:50:21.010 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:50:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.160+0000 7f4c16122700 1 -- 172.21.15.12:0/712536318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10105160 msgr2=0x7f4c1010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.160+0000 7f4c16122700 1 --2- 172.21.15.12:0/712536318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10105160 0x7f4c1010cce0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009990 tx=0x7f4c08009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.160+0000 7f4c16122700 1 -- 172.21.15.12:0/712536318 shutdown_connections 2024-05-08T23:50:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.160+0000 7f4c16122700 1 --2- 172.21.15.12:0/712536318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10105160 0x7f4c1010cce0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.160+0000 7f4c16122700 1 --2- 172.21.15.12:0/712536318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c10100060 0x7f4c10100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.160+0000 7f4c16122700 1 --2- 172.21.15.12:0/712536318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c10100a10 0x7f4c10104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.160+0000 7f4c16122700 1 -- 172.21.15.12:0/712536318 >> 172.21.15.12:0/712536318 conn(0x7f4c10076f40 msgr2=0x7f4c10077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.160+0000 7f4c16122700 1 -- 172.21.15.12:0/712536318 shutdown_connections 2024-05-08T23:50:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.160+0000 7f4c16122700 1 -- 172.21.15.12:0/712536318 wait complete. 2024-05-08T23:50:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.161+0000 7f4c16122700 1 Processor -- start 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.161+0000 7f4c16122700 1 -- start start 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.161+0000 7f4c16122700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10100060 0x7f4c10101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.161+0000 7f4c16122700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c10105160 0x7f4c10102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.161+0000 7f4c16122700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c10102550 0x7f4c101a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.161+0000 7f4c16122700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c101a5b20 con 0x7f4c10100060 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.161+0000 7f4c16122700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c101a5c90 con 0x7f4c10105160 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.161+0000 7f4c16122700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c101a5e50 con 0x7f4c10102550 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.161+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10100060 0x7f4c10101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.161+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10100060 0x7f4c10101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56854/0 (socket says 172.21.15.12:56854) 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/3397317924 learned_addr learned my addr 172.21.15.12:0/3397317924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0effd700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c10105160 0x7f4c10102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0ffff700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c10102550 0x7f4c101a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/3397317924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c10102550 msgr2=0x7f4c101a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c10102550 0x7f4c101a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/3397317924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c10105160 msgr2=0x7f4c10102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c10105160 0x7f4c10102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/3397317924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080095f0 con 0x7f4c10100060 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0ffff700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c10102550 0x7f4c101a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0effd700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c10105160 0x7f4c10102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10100060 0x7f4c10101ad0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000ef10 tx=0x7f4c0000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0cff9700 1 -- 172.21.15.12:0/3397317924 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000ca70 con 0x7f4c10100060 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0cff9700 1 -- 172.21.15.12:0/3397317924 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c0000f040 con 0x7f4c10100060 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c0cff9700 1 -- 172.21.15.12:0/3397317924 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00014480 con 0x7f4c10100060 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c16122700 1 -- 172.21.15.12:0/3397317924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c101a60d0 con 0x7f4c10100060 2024-05-08T23:50:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.162+0000 7f4c16122700 1 -- 172.21.15.12:0/3397317924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c101a6590 con 0x7f4c10100060 2024-05-08T23:50:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.165+0000 7f4c0cff9700 1 -- 172.21.15.12:0/3397317924 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4c00007500 con 0x7f4c10100060 2024-05-08T23:50:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.166+0000 7f4c0cff9700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4bf80a1290 0x7f4bf80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.166+0000 7f4c0cff9700 1 -- 172.21.15.12:0/3397317924 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4c000cd090 con 0x7f4c10100060 2024-05-08T23:50:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.166+0000 7f4c0effd700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4bf80a1290 0x7f4bf80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.166+0000 7f4c0effd700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4bf80a1290 0x7f4bf80a3750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4c10103200 tx=0x7f4c04009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.166+0000 7f4c222dcb80 1 Processor -- start 2024-05-08T23:50:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.166+0000 7f4c222dcb80 1 -- start start 2024-05-08T23:50:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x5631336261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x563133620350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133620cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563133577150 con 0x563133625e00 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563133574b10 con 0x563133626720 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563133570400 con 0x563133620890 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x5631336261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x563133620350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133620cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x5631336261e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56868/0 (socket says 172.21.15.12:56868) 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/4009539960 learned_addr learned my addr 172.21.15.12:0/4009539960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/4009539960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 msgr2=0x563133620cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/4009539960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133620cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/4009539960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 msgr2=0x563133620350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/4009539960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x563133620350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/4009539960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631333f13b0 con 0x563133625e00 2024-05-08T23:50:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0ffff700 1 --2- 172.21.15.12:0/4009539960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133620cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:21.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0effd700 1 --2- 172.21.15.12:0/4009539960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x563133620350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:21.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.167+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/4009539960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x5631336261e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4c000a20d0 tx=0x7f4c00099ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5199ac23b983470f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4bf5ffb700 1 -- 172.21.15.12:0/4009539960 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0009dfb0 con 0x563133625e00 2024-05-08T23:50:21.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4bf5ffb700 1 -- 172.21.15.12:0/4009539960 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c000b1070 con 0x563133625e00 2024-05-08T23:50:21.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4bf5ffb700 1 -- 172.21.15.12:0/4009539960 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0009a600 con 0x563133625e00 2024-05-08T23:50:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4c222dcb80 1 -- 172.21.15.12:0/4009539960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 msgr2=0x5631336261e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/4009539960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x5631336261e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4c000a20d0 tx=0x7f4c00099ae0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4c222dcb80 1 -- 172.21.15.12:0/4009539960 shutdown_connections 2024-05-08T23:50:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/4009539960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133620cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/4009539960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x5631336261e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/4009539960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x563133620350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4c222dcb80 1 -- 172.21.15.12:0/4009539960 >> 172.21.15.12:0/4009539960 conn(0x56313357e210 msgr2=0x56313361adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4c222dcb80 1 -- 172.21.15.12:0/4009539960 shutdown_connections 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.168+0000 7f4c222dcb80 1 -- 172.21.15.12:0/4009539960 wait complete. 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.169+0000 7f4c222dcb80 1 -- start start 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.169+0000 7f4c222dcb80 1 Processor -- start 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.169+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133620890 0x563133629ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.169+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133629ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133626720 0x56313362a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313350c560 con 0x563133626720 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313350c200 con 0x563133620890 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631333f36a0 con 0x563133625e00 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c222dcb80 10 client.0 _ll_register_callbacks cb 0x563133515b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133629ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133629ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34630/0 (socket says 172.21.15.12:34630) 2024-05-08T23:50:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0effd700 1 -- 172.21.15.12:0/38659212 learned_addr learned my addr 172.21.15.12:0/38659212 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133620890 0x563133629ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0ffff700 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133626720 0x56313362a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0effd700 1 -- 172.21.15.12:0/38659212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133620890 msgr2=0x563133629ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0effd700 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133620890 0x563133629ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0effd700 1 -- 172.21.15.12:0/38659212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133626720 msgr2=0x56313362a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0effd700 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133626720 0x56313362a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0effd700 1 -- 172.21.15.12:0/38659212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c04017070 con 0x563133625e00 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0ffff700 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133626720 0x56313362a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4c0effd700 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133629ff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400ebf0 tx=0x7f4c040047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4c1a2a25635f7ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.170+0000 7f4beeffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4beeffd700 1 -- 172.21.15.12:0/38659212 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04004520 con 0x563133625e00 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4beeffd700 1 -- 172.21.15.12:0/38659212 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4c0401f890 con 0x563133625e00 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4c222dcb80 1 -- 172.21.15.12:0/38659212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631334c9270 con 0x563133625e00 2024-05-08T23:50:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4beeffd700 1 -- 172.21.15.12:0/38659212 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c0401fa50 con 0x563133625e00 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4c222dcb80 10 client.26257 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4c222dcb80 1 -- 172.21.15.12:0/38659212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f4c101a6590 con 0x563133625e00 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4beeffd700 1 -- 172.21.15.12:0/38659212 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5631334c9270 con 0x563133625e00 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4beeffd700 1 -- 172.21.15.12:0/38659212 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c04031440 con 0x563133625e00 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4c222dcb80 10 client.26257 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4c222dcb80 10 client.26257 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4c222dcb80 20 client.26257 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56313363a260 0x56313363c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4c222dcb80 4 client.26257 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4c222dcb80 1 -- 172.21.15.12:0/38659212 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631335ef810 con 0x56313363a260 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.171+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56313363a260 0x56313363c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.172+0000 7f4beeffd700 10 client.26257 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.172+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56313363a260 0x56313363c680 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9165689cebdaf511 server_cookie=7cf9bdf7ec689a20 in_seq=0 out_seq=0 2024-05-08T23:50:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.172+0000 7f4beeffd700 1 -- 172.21.15.12:0/38659212 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631335ef810 con 0x56313363a260 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.172+0000 7f4beeffd700 10 client.26257 handle_command_reply: tid=0 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.172+0000 7f4c222dcb80 1 client.26257 shutdown 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 1 -- 172.21.15.12:0/38659212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 msgr2=0x563133629ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133629ff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400ebf0 tx=0x7f4c040047c0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 1 -- 172.21.15.12:0/38659212 shutdown_connections 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133626720 0x56313362a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133620890 0x563133629ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56313363a260 0x56313363c680 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/38659212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133629ff0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 1 -- 172.21.15.12:0/38659212 >> 172.21.15.12:0/38659212 conn(0x56313357e210 msgr2=0x56313361a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4beeffd700 0 client.26257 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 1 -- 172.21.15.12:0/38659212 shutdown_connections 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 1 -- 172.21.15.12:0/38659212 wait complete. 2024-05-08T23:50:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.173+0000 7f4c222dcb80 20 client.26257 trim_cache size 0 max 16384 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.187+0000 7f4c222dcb80 1 Processor -- start 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.187+0000 7f4c222dcb80 1 -- start start 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.187+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133620890 0x5631336362f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.187+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133635c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.187+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x56313361ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.187+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313362c660 con 0x563133620890 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.187+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563133617df0 con 0x563133626720 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.187+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563133641910 con 0x563133625e00 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133620890 0x5631336362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133635c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x56313361ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133620890 0x5631336362f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56886/0 (socket says 172.21.15.12:56886) 2024-05-08T23:50:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/3828991002 learned_addr learned my addr 172.21.15.12:0/3828991002 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0effd700 1 -- 172.21.15.12:0/3828991002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 msgr2=0x56313361ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0effd700 1 --2- 172.21.15.12:0/3828991002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x56313361ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0effd700 1 -- 172.21.15.12:0/3828991002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133620890 msgr2=0x5631336362f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0effd700 1 --2- 172.21.15.12:0/3828991002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133620890 0x5631336362f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0effd700 1 -- 172.21.15.12:0/3828991002 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56313339db60 con 0x563133625e00 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/3828991002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133620890 0x5631336362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4c0effd700 1 --2- 172.21.15.12:0/3828991002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133635c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009fd0 tx=0x7f4c04011c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d47597eb96da476 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4bf67fc700 1 -- 172.21.15.12:0/3828991002 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04031440 con 0x563133625e00 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4bf67fc700 1 -- 172.21.15.12:0/3828991002 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c040315a0 con 0x563133625e00 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.188+0000 7f4bf67fc700 1 -- 172.21.15.12:0/3828991002 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04003680 con 0x563133625e00 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.189+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3828991002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 msgr2=0x563133635c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.189+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/3828991002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133635c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009fd0 tx=0x7f4c04011c60 comp rx=0 tx=0).stop 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.189+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3828991002 shutdown_connections 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.189+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/3828991002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x56313361ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.189+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/3828991002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133620890 0x5631336362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.189+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/3828991002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133625e00 0x563133635c20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.189+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3828991002 >> 172.21.15.12:0/3828991002 conn(0x56313357e210 msgr2=0x5631336283a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.189+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3828991002 shutdown_connections 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.189+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3828991002 wait complete. 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.189+0000 7f4c222dcb80 1 -- start start 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c222dcb80 1 Processor -- start 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133631690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x563133622b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c222dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x5631336232b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563133621930 con 0x563133625e00 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563133621a70 con 0x563133626720 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c222dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563133621c30 con 0x563133620890 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c222dcb80 10 client.0 _ll_register_callbacks cb 0x563133515b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133631690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c0effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x563133622b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c0ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x5631336232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c0effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x563133622b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56902/0 (socket says 172.21.15.12:56902) 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133631690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34646/0 (socket says 172.21.15.12:34646) 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.190+0000 7f4c0effd700 1 -- 172.21.15.12:0/3036570413 learned_addr learned my addr 172.21.15.12:0/3036570413 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/3036570413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 msgr2=0x5631336232b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x5631336232b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/3036570413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 msgr2=0x563133622b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x563133622b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4c0f7fe700 1 -- 172.21.15.12:0/3036570413 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631333f13b0 con 0x563133620890 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4c0effd700 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x563133622b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4c0f7fe700 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133631690 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4c00096bf0 tx=0x7f4c00009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f0b42639c62af3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4beeffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4beeffd700 1 -- 172.21.15.12:0/3036570413 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0009dfb0 con 0x563133620890 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4beeffd700 1 -- 172.21.15.12:0/3036570413 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4c0009c070 con 0x563133620890 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3036570413 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c000b1070 con 0x563133620890 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4beeffd700 1 -- 172.21.15.12:0/3036570413 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c0000cd60 con 0x563133620890 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4c222dcb80 10 client.26263 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3036570413 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f4c101a6590 con 0x563133620890 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.191+0000 7f4beeffd700 1 -- 172.21.15.12:0/3036570413 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c000b1070 con 0x563133620890 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4beeffd700 1 -- 172.21.15.12:0/3036570413 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c0009a3e0 con 0x563133620890 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4c222dcb80 10 client.26263 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4c222dcb80 10 client.26263 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4c222dcb80 20 client.26263 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563133690ee0 0x563133693300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4c222dcb80 4 client.26263 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3036570413 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563133568ec0 con 0x563133690ee0 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4c0effd700 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563133690ee0 0x563133693300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4beeffd700 10 client.26263 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4c0effd700 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563133690ee0 0x563133693300 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c06d8533aad53142 server_cookie=17d67ba32544c1de in_seq=0 out_seq=0 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4beeffd700 1 -- 172.21.15.12:0/3036570413 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563133568ec0 con 0x563133690ee0 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4beeffd700 10 client.26263 handle_command_reply: tid=0 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.192+0000 7f4c222dcb80 1 client.26263 shutdown 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3036570413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 msgr2=0x563133631690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133631690 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4c00096bf0 tx=0x7f4c00009ac0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3036570413 shutdown_connections 2024-05-08T23:50:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563133626720 0x5631336232b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563133690ee0 0x563133693300 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563133620890 0x563133631690 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 1 --2- 172.21.15.12:0/3036570413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563133625e00 0x563133622b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4beeffd700 0 client.26263 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:50:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3036570413 >> 172.21.15.12:0/3036570413 conn(0x56313357e210 msgr2=0x563133624ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3036570413 shutdown_connections 2024-05-08T23:50:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 1 -- 172.21.15.12:0/3036570413 wait complete. 2024-05-08T23:50:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c222dcb80 20 client.26263 trim_cache size 0 max 16384 2024-05-08T23:50:21.194 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:50:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c16122700 1 -- 172.21.15.12:0/3397317924 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4bf80a1290 msgr2=0x7f4bf80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c16122700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4bf80a1290 0x7f4bf80a3750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4c10103200 tx=0x7f4c04009260 comp rx=0 tx=0).stop 2024-05-08T23:50:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c16122700 1 -- 172.21.15.12:0/3397317924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10100060 msgr2=0x7f4c10101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.193+0000 7f4c16122700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10100060 0x7f4c10101ad0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000ef10 tx=0x7f4c0000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.194+0000 7f4c16122700 1 -- 172.21.15.12:0/3397317924 shutdown_connections 2024-05-08T23:50:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.194+0000 7f4c16122700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4bf80a1290 0x7f4bf80a3750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.194+0000 7f4c16122700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c10102550 0x7f4c101a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.194+0000 7f4c16122700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10100060 0x7f4c10101ad0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.194+0000 7f4c16122700 1 --2- 172.21.15.12:0/3397317924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c10105160 0x7f4c10102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.194+0000 7f4c16122700 1 -- 172.21.15.12:0/3397317924 >> 172.21.15.12:0/3397317924 conn(0x7f4c10076f40 msgr2=0x7f4c10068330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.194+0000 7f4c16122700 1 -- 172.21.15.12:0/3397317924 shutdown_connections 2024-05-08T23:50:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.194+0000 7f4c16122700 1 -- 172.21.15.12:0/3397317924 wait complete. 2024-05-08T23:50:21.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:50:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.354+0000 7fe2bd691700 1 -- 172.21.15.12:0/2771483441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2b80682e0 msgr2=0x7fe2b80686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.354+0000 7fe2bd691700 1 --2- 172.21.15.12:0/2771483441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2b80682e0 0x7fe2b80686c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80099c0 tx=0x7fe2a8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.354+0000 7fe2bd691700 1 -- 172.21.15.12:0/2771483441 shutdown_connections 2024-05-08T23:50:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.354+0000 7fe2bd691700 1 --2- 172.21.15.12:0/2771483441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106290 0x7fe2b810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.354+0000 7fe2bd691700 1 --2- 172.21.15.12:0/2771483441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2b80682e0 0x7fe2b80686c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.354+0000 7fe2bd691700 1 --2- 172.21.15.12:0/2771483441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2b8068c90 0x7fe2b8105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.354+0000 7fe2bd691700 1 -- 172.21.15.12:0/2771483441 >> 172.21.15.12:0/2771483441 conn(0x7fe2b8076f40 msgr2=0x7fe2b8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.354+0000 7fe2bd691700 1 -- 172.21.15.12:0/2771483441 shutdown_connections 2024-05-08T23:50:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.354+0000 7fe2bd691700 1 -- 172.21.15.12:0/2771483441 wait complete. 2024-05-08T23:50:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.355+0000 7fe2bd691700 1 Processor -- start 2024-05-08T23:50:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.355+0000 7fe2bd691700 1 -- start start 2024-05-08T23:50:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.355+0000 7fe2bd691700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2b80682e0 0x7fe2b81973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.355+0000 7fe2bd691700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106290 0x7fe2b81978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.355+0000 7fe2bd691700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2b819bc80 0x7fe2b819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.355+0000 7fe2bd691700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b819c700 con 0x7fe2b80682e0 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.355+0000 7fe2bd691700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b819c870 con 0x7fe2b819bc80 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.355+0000 7fe2bd691700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b819ca30 con 0x7fe2b8106290 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2b80682e0 0x7fe2b81973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2b80682e0 0x7fe2b81973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56926/0 (socket says 172.21.15.12:56926) 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/178983786 learned_addr learned my addr 172.21.15.12:0/178983786 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106290 0x7fe2b81978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b77fe700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2b819bc80 0x7fe2b819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/178983786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106290 msgr2=0x7fe2b81978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106290 0x7fe2b81978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/178983786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2b819bc80 msgr2=0x7fe2b819c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2b819bc80 0x7fe2b819c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/178983786 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a80095f0 con 0x7fe2b80682e0 2024-05-08T23:50:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2b80682e0 0x7fe2b81973a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800f690 tx=0x7fe2a800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe29ffff700 1 -- 172.21.15.12:0/178983786 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a801c070 con 0x7fe2b80682e0 2024-05-08T23:50:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2bd691700 1 -- 172.21.15.12:0/178983786 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b81a3f30 con 0x7fe2b80682e0 2024-05-08T23:50:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe29ffff700 1 -- 172.21.15.12:0/178983786 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2a800fcf0 con 0x7fe2b80682e0 2024-05-08T23:50:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe29ffff700 1 -- 172.21.15.12:0/178983786 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8022c60 con 0x7fe2b80682e0 2024-05-08T23:50:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.356+0000 7fe2bd691700 1 -- 172.21.15.12:0/178983786 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b81a42f0 con 0x7fe2b80682e0 2024-05-08T23:50:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.359+0000 7fe29ffff700 1 -- 172.21.15.12:0/178983786 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe2a8020070 con 0x7fe2b80682e0 2024-05-08T23:50:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.360+0000 7fe29ffff700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2a00a1290 0x7fe2a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.360+0000 7fe29ffff700 1 -- 172.21.15.12:0/178983786 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe2a80cdc10 con 0x7fe2b80682e0 2024-05-08T23:50:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.360+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2a00a1290 0x7fe2a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.360+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2a00a1290 0x7fe2a00a3750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe2b81988f0 tx=0x7fe2ac009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2c984bb80 1 Processor -- start 2024-05-08T23:50:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2c984bb80 1 -- start start 2024-05-08T23:50:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d167541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1674e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 0x561d1674ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d166a5150 con 0x561d16754720 2024-05-08T23:50:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d166a2b10 con 0x561d16753e00 2024-05-08T23:50:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d1669e400 con 0x561d1674e890 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2b67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1674e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2b67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1674e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56938/0 (socket says 172.21.15.12:56938) 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2b67fc700 1 -- 172.21.15.12:0/3585490802 learned_addr learned my addr 172.21.15.12:0/3585490802 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2b77fe700 1 --2- 172.21.15.12:0/3585490802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 0x561d1674ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/3585490802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d167541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2b67fc700 1 -- 172.21.15.12:0/3585490802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 msgr2=0x561d1674ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/3585490802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 0x561d1674ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2b67fc700 1 -- 172.21.15.12:0/3585490802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 msgr2=0x561d167541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/3585490802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d167541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.361+0000 7fe2b67fc700 1 -- 172.21.15.12:0/3585490802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d1651f3b0 con 0x561d16754720 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.362+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/3585490802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1674e350 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac011d20 tx=0x7fe2ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef975c2635ebdce1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.362+0000 7fe29d7fa700 1 -- 172.21.15.12:0/3585490802 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac00eb20 con 0x561d16754720 2024-05-08T23:50:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.362+0000 7fe29d7fa700 1 -- 172.21.15.12:0/3585490802 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2ac021070 con 0x561d16754720 2024-05-08T23:50:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.362+0000 7fe29d7fa700 1 -- 172.21.15.12:0/3585490802 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac01d410 con 0x561d16754720 2024-05-08T23:50:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 -- 172.21.15.12:0/3585490802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 msgr2=0x561d1674e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/3585490802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1674e350 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac011d20 tx=0x7fe2ac014000 comp rx=0 tx=0).stop 2024-05-08T23:50:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 -- 172.21.15.12:0/3585490802 shutdown_connections 2024-05-08T23:50:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/3585490802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 0x561d1674ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/3585490802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d167541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/3585490802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1674e350 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 -- 172.21.15.12:0/3585490802 >> 172.21.15.12:0/3585490802 conn(0x561d166ac210 msgr2=0x561d16748df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 -- 172.21.15.12:0/3585490802 shutdown_connections 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 -- 172.21.15.12:0/3585490802 wait complete. 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 -- start start 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.363+0000 7fe2c984bb80 1 Processor -- start 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d16757b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d167580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d16754720 0x561d167585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d1663a560 con 0x561d1674e890 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d1663a200 con 0x561d16753e00 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d165216a0 con 0x561d16754720 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2c984bb80 10 client.0 _ll_register_callbacks cb 0x561d16643b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d16757b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d16757b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56954/0 (socket says 172.21.15.12:56954) 2024-05-08T23:50:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/726833566 learned_addr learned my addr 172.21.15.12:0/726833566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d167580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b77fe700 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d16754720 0x561d167585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/726833566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d16754720 msgr2=0x561d167585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d16754720 0x561d167585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/726833566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 msgr2=0x561d167580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d167580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/726833566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac017070 con 0x561d1674e890 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d167580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d16757b60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe2a809aea0 tx=0x7fe2a809a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7627590849f367f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe29aff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe29aff5700 1 -- 172.21.15.12:0/726833566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80b2070 con 0x561d1674e890 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe29aff5700 1 -- 172.21.15.12:0/726833566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe2a809af10 con 0x561d1674e890 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.364+0000 7fe2c984bb80 1 -- 172.21.15.12:0/726833566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d165f7270 con 0x561d1674e890 2024-05-08T23:50:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe29aff5700 1 -- 172.21.15.12:0/726833566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2a809ed40 con 0x561d1674e890 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe2c984bb80 10 client.16500 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe2c984bb80 1 -- 172.21.15.12:0/726833566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561d164cbb60 con 0x561d1674e890 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe29aff5700 1 -- 172.21.15.12:0/726833566 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2a8064080 con 0x561d1674e890 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe29aff5700 1 -- 172.21.15.12:0/726833566 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe2a809a490 con 0x561d1674e890 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe2c984bb80 10 client.16500 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe2c984bb80 10 client.16500 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe2c984bb80 20 client.16500 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561d16767dd0 0x561d1676a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe2c984bb80 4 client.16500 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe2c984bb80 1 -- 172.21.15.12:0/726833566 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d1671d810 con 0x561d16767dd0 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.365+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561d16767dd0 0x561d1676a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.366+0000 7fe29aff5700 10 client.16500 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:50:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.366+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561d16767dd0 0x561d1676a1c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d81b130b5295d1fc server_cookie=ddb5754ff17f4d08 in_seq=0 out_seq=0 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.366+0000 7fe29aff5700 1 -- 172.21.15.12:0/726833566 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d1671d810 con 0x561d16767dd0 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.366+0000 7fe29aff5700 10 client.16500 handle_command_reply: tid=0 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.366+0000 7fe2c984bb80 1 client.16500 shutdown 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.366+0000 7fe2c984bb80 1 -- 172.21.15.12:0/726833566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 msgr2=0x561d16757b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.366+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d16757b60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe2a809aea0 tx=0x7fe2a809a8f0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.367+0000 7fe2c984bb80 1 -- 172.21.15.12:0/726833566 shutdown_connections 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.367+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561d16767dd0 0x561d1676a1c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.367+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d16754720 0x561d167585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.367+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d16757b60 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.367+0000 7fe29aff5700 0 client.16500 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.367+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/726833566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d167580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.367+0000 7fe2c984bb80 1 -- 172.21.15.12:0/726833566 >> 172.21.15.12:0/726833566 conn(0x561d166ac210 msgr2=0x561d167485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.367+0000 7fe2c984bb80 1 -- 172.21.15.12:0/726833566 shutdown_connections 2024-05-08T23:50:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.367+0000 7fe2c984bb80 1 -- 172.21.15.12:0/726833566 wait complete. 2024-05-08T23:50:21.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.367+0000 7fe2c984bb80 20 client.16500 trim_cache size 0 max 16384 2024-05-08T23:50:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2c984bb80 1 Processor -- start 2024-05-08T23:50:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2c984bb80 1 -- start start 2024-05-08T23:50:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d1674e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d16753e00 0x561d16748d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16754720 0x561d16758400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d1675abd0 con 0x561d1674e890 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d16754560 con 0x561d16754720 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d1676f3d0 con 0x561d16753e00 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d1674e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d1674e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56960/0 (socket says 172.21.15.12:56960) 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/662250256 learned_addr learned my addr 172.21.15.12:0/662250256 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2b77fe700 1 --2- 172.21.15.12:0/662250256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16754720 0x561d16758400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/662250256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d16753e00 0x561d16748d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/662250256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d16753e00 msgr2=0x561d16748d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/662250256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d16753e00 0x561d16748d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/662250256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16754720 msgr2=0x561d16758400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/662250256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16754720 0x561d16758400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.383+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/662250256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d16664560 con 0x561d1674e890 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.384+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/662250256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d1674e350 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe2a809ae50 tx=0x7fe2a80a1160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0d4234a4ba1700 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.384+0000 7fe29dffb700 1 -- 172.21.15.12:0/662250256 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80b2070 con 0x561d1674e890 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.384+0000 7fe29dffb700 1 -- 172.21.15.12:0/662250256 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561d165f7270 con 0x561d1674e890 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.384+0000 7fe29dffb700 1 -- 172.21.15.12:0/662250256 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80a3300 con 0x561d1674e890 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 -- 172.21.15.12:0/662250256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 msgr2=0x561d1674e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/662250256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d1674e350 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe2a809ae50 tx=0x7fe2a80a1160 comp rx=0 tx=0).stop 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 -- 172.21.15.12:0/662250256 shutdown_connections 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/662250256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16754720 0x561d16758400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/662250256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d1674e890 0x561d1674e350 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/662250256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d16753e00 0x561d16748d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 -- 172.21.15.12:0/662250256 >> 172.21.15.12:0/662250256 conn(0x561d166ac210 msgr2=0x561d16756430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 -- 172.21.15.12:0/662250256 shutdown_connections 2024-05-08T23:50:21.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 -- 172.21.15.12:0/662250256 wait complete. 2024-05-08T23:50:21.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 -- start start 2024-05-08T23:50:21.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.385+0000 7fe2c984bb80 1 Processor -- start 2024-05-08T23:50:21.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 0x561d1675f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d1675d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2c984bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1675d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d1675de30 con 0x561d16754720 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d1675dfa0 con 0x561d16753e00 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2c984bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d167bd2a0 con 0x561d1674e890 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2c984bb80 10 client.0 _ll_register_callbacks cb 0x561d16643b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1675d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 0x561d1675f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d1675d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 0x561d1675f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34712/0 (socket says 172.21.15.12:34712) 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1675d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56972/0 (socket says 172.21.15.12:56972) 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/1652243815 learned_addr learned my addr 172.21.15.12:0/1652243815 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/1652243815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 msgr2=0x561d1675d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d1675d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/1652243815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 msgr2=0x561d1675d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1675d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b6ffd700 1 -- 172.21.15.12:0/1652243815 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d164cbb60 con 0x561d1674e890 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b77fe700 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1675d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe2b6ffd700 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 0x561d1675f5e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009fd0 tx=0x7fe2a809ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f97229ed868b733 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.386+0000 7fe29aff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe29aff5700 1 -- 172.21.15.12:0/1652243815 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80b2070 con 0x561d1674e890 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe29aff5700 1 -- 172.21.15.12:0/1652243815 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe2a80a3300 con 0x561d1674e890 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe2c984bb80 1 -- 172.21.15.12:0/1652243815 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2ac021070 con 0x561d1674e890 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe29aff5700 1 -- 172.21.15.12:0/1652243815 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2a80a2f10 con 0x561d1674e890 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe2c984bb80 10 client.26281 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe2c984bb80 1 -- 172.21.15.12:0/1652243815 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561d16523cd0 con 0x561d1674e890 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe29aff5700 1 -- 172.21.15.12:0/1652243815 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2ac021070 con 0x561d1674e890 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe29aff5700 1 -- 172.21.15.12:0/1652243815 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe2a8096f00 con 0x561d1674e890 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe2c984bb80 10 client.26281 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe2c984bb80 10 client.26281 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe2c984bb80 20 client.26281 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561d167be590 0x561d167c0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe2c984bb80 4 client.26281 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.387+0000 7fe2c984bb80 1 -- 172.21.15.12:0/1652243815 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561d16696ec0 con 0x561d167be590 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561d167be590 0x561d167c0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe29aff5700 10 client.26281 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2b67fc700 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561d167be590 0x561d167c0980 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=531770b203730548 server_cookie=40eecddd7860e215 in_seq=0 out_seq=0 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe29aff5700 1 -- 172.21.15.12:0/1652243815 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561d16696ec0 con 0x561d167be590 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe29aff5700 10 client.26281 handle_command_reply: tid=0 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 client.26281 shutdown 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 -- 172.21.15.12:0/1652243815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 msgr2=0x561d1675f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 0x561d1675f5e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009fd0 tx=0x7fe2a809ace0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 -- 172.21.15.12:0/1652243815 shutdown_connections 2024-05-08T23:50:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561d167be590 0x561d167c0980 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d16754720 0x561d1675d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe29aff5700 0 client.26281 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:50:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d1674e890 0x561d1675f5e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 --2- 172.21.15.12:0/1652243815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d16753e00 0x561d1675d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 -- 172.21.15.12:0/1652243815 >> 172.21.15.12:0/1652243815 conn(0x561d166ac210 msgr2=0x561d16752c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 -- 172.21.15.12:0/1652243815 shutdown_connections 2024-05-08T23:50:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.388+0000 7fe2c984bb80 1 -- 172.21.15.12:0/1652243815 wait complete. 2024-05-08T23:50:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2c984bb80 20 client.26281 trim_cache size 0 max 16384 2024-05-08T23:50:21.390 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:50:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 -- 172.21.15.12:0/178983786 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2a00a1290 msgr2=0x7fe2a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2a00a1290 0x7fe2a00a3750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe2b81988f0 tx=0x7fe2ac009260 comp rx=0 tx=0).stop 2024-05-08T23:50:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 -- 172.21.15.12:0/178983786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2b80682e0 msgr2=0x7fe2b81973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2b80682e0 0x7fe2b81973a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800f690 tx=0x7fe2a800bfd0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 -- 172.21.15.12:0/178983786 shutdown_connections 2024-05-08T23:50:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2a00a1290 0x7fe2a00a3750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2b819bc80 0x7fe2b819c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2b80682e0 0x7fe2b81973a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 --2- 172.21.15.12:0/178983786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106290 0x7fe2b81978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 -- 172.21.15.12:0/178983786 >> 172.21.15.12:0/178983786 conn(0x7fe2b8076f40 msgr2=0x7fe2b80fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 -- 172.21.15.12:0/178983786 shutdown_connections 2024-05-08T23:50:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.389+0000 7fe2bd691700 1 -- 172.21.15.12:0/178983786 wait complete. 2024-05-08T23:50:21.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:50:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.549+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/2693656614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc98101160 msgr2=0x7fdc98101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.549+0000 7fdc9c8fe700 1 --2- 172.21.15.12:0/2693656614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc98101160 0x7fdc98101540 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009990 tx=0x7fdc88009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.549+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/2693656614 shutdown_connections 2024-05-08T23:50:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.549+0000 7fdc9c8fe700 1 --2- 172.21.15.12:0/2693656614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc98106280 0x7fdc9810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.549+0000 7fdc9c8fe700 1 --2- 172.21.15.12:0/2693656614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc98101160 0x7fdc98101540 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.549+0000 7fdc9c8fe700 1 --2- 172.21.15.12:0/2693656614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc98101b10 0x7fdc98105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.549+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/2693656614 >> 172.21.15.12:0/2693656614 conn(0x7fdc980fc9e0 msgr2=0x7fdc980fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.549+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/2693656614 shutdown_connections 2024-05-08T23:50:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.549+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/2693656614 wait complete. 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9c8fe700 1 Processor -- start 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9c8fe700 1 -- start start 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9c8fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc98101160 0x7fdc980732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9c8fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc98106280 0x7fdc980737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9c8fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc98073d20 0x7fdc981a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9c8fe700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981a5be0 con 0x7fdc98106280 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9c8fe700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981a5d20 con 0x7fdc98073d20 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9c8fe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981a5ee0 con 0x7fdc98101160 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc95d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc98106280 0x7fdc980737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc98101160 0x7fdc980732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc98101160 0x7fdc980732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34726/0 (socket says 172.21.15.12:34726) 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.550+0000 7fdc9659c700 1 -- 172.21.15.12:0/1367922148 learned_addr learned my addr 172.21.15.12:0/1367922148 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc96d9d700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc98073d20 0x7fdc981a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc95d9b700 1 -- 172.21.15.12:0/1367922148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc98101160 msgr2=0x7fdc980732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc95d9b700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc98101160 0x7fdc980732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc95d9b700 1 -- 172.21.15.12:0/1367922148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc98073d20 msgr2=0x7fdc981a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc95d9b700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc98073d20 0x7fdc981a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc95d9b700 1 -- 172.21.15.12:0/1367922148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc880095f0 con 0x7fdc98106280 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc9659c700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc98101160 0x7fdc980732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc95d9b700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc98106280 0x7fdc980737e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00e9c0 tx=0x7fdc8c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc877fe700 1 -- 172.21.15.12:0/1367922148 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00c9c0 con 0x7fdc98106280 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc877fe700 1 -- 172.21.15.12:0/1367922148 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8c010430 con 0x7fdc98106280 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/1367922148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc981a61c0 con 0x7fdc98106280 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc877fe700 1 -- 172.21.15.12:0/1367922148 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c018680 con 0x7fdc98106280 2024-05-08T23:50:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.551+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/1367922148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc981a6720 con 0x7fdc98106280 2024-05-08T23:50:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.554+0000 7fdc877fe700 1 -- 172.21.15.12:0/1367922148 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdc8c00cb20 con 0x7fdc98106280 2024-05-08T23:50:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.554+0000 7fdc877fe700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdc800a1310 0x7fdc800a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.555+0000 7fdc9659c700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdc800a1310 0x7fdc800a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.555+0000 7fdc877fe700 1 -- 172.21.15.12:0/1367922148 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fdc8c014070 con 0x7fdc98106280 2024-05-08T23:50:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.555+0000 7fdca8ab8b80 1 Processor -- start 2024-05-08T23:50:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.555+0000 7fdc9659c700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdc800a1310 0x7fdc800a37d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdc88006010 tx=0x7fdc8800b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.555+0000 7fdca8ab8b80 1 -- start start 2024-05-08T23:50:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d6f9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 0x5585d6f9ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6ef3150 con 0x5585d6fa1e00 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6ef0b10 con 0x5585d6f9c890 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6eec400 con 0x5585d6fa2720 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc9659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc95d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d6f9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc9659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56990/0 (socket says 172.21.15.12:56990) 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc9659c700 1 -- 172.21.15.12:0/1523055109 learned_addr learned my addr 172.21.15.12:0/1523055109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc95d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d6f9c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34728/0 (socket says 172.21.15.12:34728) 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc96d9d700 1 --2- 172.21.15.12:0/1523055109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 0x5585d6f9ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc9659c700 1 -- 172.21.15.12:0/1523055109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 msgr2=0x5585d6f9c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc9659c700 1 --2- 172.21.15.12:0/1523055109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d6f9c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc9659c700 1 -- 172.21.15.12:0/1523055109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 msgr2=0x5585d6f9ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc9659c700 1 --2- 172.21.15.12:0/1523055109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 0x5585d6f9ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc9659c700 1 -- 172.21.15.12:0/1523055109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585d6d6d3b0 con 0x5585d6fa1e00 2024-05-08T23:50:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc95d9b700 1 --2- 172.21.15.12:0/1523055109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d6f9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc9659c700 1 --2- 172.21.15.12:0/1523055109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa21e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdc880056e0 tx=0x7fdc88020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb07341ebffdaf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.556+0000 7fdc84ff9700 1 -- 172.21.15.12:0/1523055109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8802b070 con 0x5585d6fa1e00 2024-05-08T23:50:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdc84ff9700 1 -- 172.21.15.12:0/1523055109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8801e040 con 0x5585d6fa1e00 2024-05-08T23:50:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdc84ff9700 1 -- 172.21.15.12:0/1523055109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88004a60 con 0x5585d6fa1e00 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1523055109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 msgr2=0x5585d6fa21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/1523055109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa21e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdc880056e0 tx=0x7fdc88020000 comp rx=0 tx=0).stop 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1523055109 shutdown_connections 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/1523055109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 0x5585d6f9ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/1523055109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa21e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/1523055109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d6f9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1523055109 >> 172.21.15.12:0/1523055109 conn(0x5585d6efa210 msgr2=0x5585d6f96df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1523055109 shutdown_connections 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.557+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1523055109 wait complete. 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.558+0000 7fdca8ab8b80 1 -- start start 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.558+0000 7fdca8ab8b80 1 Processor -- start 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.558+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 0x5585d6fa5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.558+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa1e00 0x5585d6fa5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.558+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa2720 0x5585d6fa64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.558+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6e88560 con 0x5585d6fa2720 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.558+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6e88200 con 0x5585d6f9c890 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.558+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6d6f6a0 con 0x5585d6fa1e00 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.558+0000 7fdca8ab8b80 10 client.0 _ll_register_callbacks cb 0x5585d6e91b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc96d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa2720 0x5585d6fa64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc9659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 0x5585d6fa5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc9659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 0x5585d6fa5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41046/0 (socket says 172.21.15.12:41046) 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc9659c700 1 -- 172.21.15.12:0/1671304800 learned_addr learned my addr 172.21.15.12:0/1671304800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc95d9b700 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa1e00 0x5585d6fa5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc9659c700 1 -- 172.21.15.12:0/1671304800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa1e00 msgr2=0x5585d6fa5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc9659c700 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa1e00 0x5585d6fa5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc9659c700 1 -- 172.21.15.12:0/1671304800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa2720 msgr2=0x5585d6fa64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc9659c700 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa2720 0x5585d6fa64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc9659c700 1 -- 172.21.15.12:0/1671304800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc981a6720 con 0x5585d6f9c890 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc9659c700 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 0x5585d6fa5a40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc880056b0 tx=0x7fdc8800fe00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d4d2b2d7569308c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc7dffb700 1 -- 172.21.15.12:0/1671304800 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8802b040 con 0x5585d6f9c890 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc7dffb700 1 -- 172.21.15.12:0/1671304800 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fdc88005050 con 0x5585d6f9c890 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1671304800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585d6e45270 con 0x5585d6f9c890 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.559+0000 7fdc7dffb700 1 -- 172.21.15.12:0/1671304800 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc88005210 con 0x5585d6f9c890 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdca8ab8b80 10 client.26312 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1671304800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fdc9810e670 con 0x5585d6f9c890 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdc7dffb700 1 -- 172.21.15.12:0/1671304800 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5585d6e45270 con 0x5585d6f9c890 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdc7dffb700 1 -- 172.21.15.12:0/1671304800 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc88037030 con 0x5585d6f9c890 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdca8ab8b80 10 client.26312 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdca8ab8b80 10 client.26312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdca8ab8b80 20 client.26312 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5585d6fb5cd0 0x5585d6fb80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdca8ab8b80 4 client.26312 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1671304800 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585d6f6b810 con 0x5585d6fb5cd0 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.560+0000 7fdc95d9b700 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5585d6fb5cd0 0x5585d6fb80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.561+0000 7fdc7dffb700 10 client.26312 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.561+0000 7fdc95d9b700 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5585d6fb5cd0 0x5585d6fb80c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428cd49e4d839a88 server_cookie=d6c83b560c4195a9 in_seq=0 out_seq=0 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdc7dffb700 1 -- 172.21.15.12:0/1671304800 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585d6f6b810 con 0x5585d6fb5cd0 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdc7dffb700 10 client.26312 handle_command_reply: tid=0 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 client.26312 shutdown 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1671304800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 msgr2=0x5585d6fa5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 0x5585d6fa5a40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc880056b0 tx=0x7fdc8800fe00 comp rx=0 tx=0).stop 2024-05-08T23:50:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1671304800 shutdown_connections 2024-05-08T23:50:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa2720 0x5585d6fa64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5585d6fb5cd0 0x5585d6fb80c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6f9c890 0x5585d6fa5a40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/1671304800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa1e00 0x5585d6fa5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1671304800 >> 172.21.15.12:0/1671304800 conn(0x5585d6efa210 msgr2=0x5585d6f965a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdc7dffb700 0 client.26312 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1671304800 shutdown_connections 2024-05-08T23:50:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/1671304800 wait complete. 2024-05-08T23:50:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.562+0000 7fdca8ab8b80 20 client.26312 trim_cache size 0 max 16384 2024-05-08T23:50:21.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdca8ab8b80 1 Processor -- start 2024-05-08T23:50:21.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdca8ab8b80 1 -- start start 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6f9c890 0x5585d6f9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6fa2720 0x5585d6f96d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6f97ae0 con 0x5585d6fa1e00 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6fbd290 con 0x5585d6fa2720 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6fa1900 con 0x5585d6f9c890 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdc95d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdc9659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6f9c890 0x5585d6f9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.577+0000 7fdc95d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57012/0 (socket says 172.21.15.12:57012) 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc95d9b700 1 -- 172.21.15.12:0/717371966 learned_addr learned my addr 172.21.15.12:0/717371966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc96d9d700 1 --2- 172.21.15.12:0/717371966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6fa2720 0x5585d6f96d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc9659c700 1 -- 172.21.15.12:0/717371966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6fa2720 msgr2=0x5585d6f96d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc9659c700 1 --2- 172.21.15.12:0/717371966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6fa2720 0x5585d6f96d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc9659c700 1 -- 172.21.15.12:0/717371966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 msgr2=0x5585d6fa13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc9659c700 1 --2- 172.21.15.12:0/717371966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc9659c700 1 -- 172.21.15.12:0/717371966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585d6f9cf40 con 0x5585d6f9c890 2024-05-08T23:50:21.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc9659c700 1 --2- 172.21.15.12:0/717371966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6f9c890 0x5585d6f9c350 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdc88004610 tx=0x7fdc88009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dca8c62f26370b10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc857fa700 1 -- 172.21.15.12:0/717371966 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8802b070 con 0x5585d6f9c890 2024-05-08T23:50:21.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc857fa700 1 -- 172.21.15.12:0/717371966 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8800fe70 con 0x5585d6f9c890 2024-05-08T23:50:21.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.578+0000 7fdc857fa700 1 -- 172.21.15.12:0/717371966 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88042b00 con 0x5585d6f9c890 2024-05-08T23:50:21.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.579+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/717371966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6f9c890 msgr2=0x5585d6f9c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.579+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/717371966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6f9c890 0x5585d6f9c350 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdc88004610 tx=0x7fdc88009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.579+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/717371966 shutdown_connections 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.579+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/717371966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6fa2720 0x5585d6f96d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.579+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/717371966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6f9c890 0x5585d6f9c350 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.579+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/717371966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6fa1e00 0x5585d6fa13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.579+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/717371966 >> 172.21.15.12:0/717371966 conn(0x5585d6efa210 msgr2=0x5585d6fa41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.579+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/717371966 shutdown_connections 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.579+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/717371966 wait complete. 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.579+0000 7fdca8ab8b80 1 -- start start 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdca8ab8b80 1 Processor -- start 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6f9c890 0x5585d6fa90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6fa1e00 0x5585d6fa9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdca8ab8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d700b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6fa9db0 con 0x5585d6f9c890 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d6fa9f20 con 0x5585d6fa1e00 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdca8ab8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d700b690 con 0x5585d6fa2720 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdca8ab8b80 10 client.0 _ll_register_callbacks cb 0x5585d6e91b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdc9659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6f9c890 0x5585d6fa90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdc9659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6f9c890 0x5585d6fa90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57026/0 (socket says 172.21.15.12:57026) 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdc9659c700 1 -- 172.21.15.12:0/4133132743 learned_addr learned my addr 172.21.15.12:0/4133132743 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdc96d9d700 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d700b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.580+0000 7fdc95d9b700 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6fa1e00 0x5585d6fa9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc9659c700 1 -- 172.21.15.12:0/4133132743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 msgr2=0x5585d700b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc9659c700 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d700b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc9659c700 1 -- 172.21.15.12:0/4133132743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6fa1e00 msgr2=0x5585d6fa9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc9659c700 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6fa1e00 0x5585d6fa9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc9659c700 1 -- 172.21.15.12:0/4133132743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9810e670 con 0x5585d6f9c890 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc96d9d700 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d700b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc9659c700 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6f9c890 0x5585d6fa90e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800b690 tx=0x7fdc8800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302f4a7020c38919 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc7dffb700 1 -- 172.21.15.12:0/4133132743 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8802b040 con 0x5585d6f9c890 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/4133132743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc8801e040 con 0x5585d6f9c890 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc7dffb700 1 -- 172.21.15.12:0/4133132743 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fdc880424c0 con 0x5585d6f9c890 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc7dffb700 1 -- 172.21.15.12:0/4133132743 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc880426b0 con 0x5585d6f9c890 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdca8ab8b80 10 client.16524 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/4133132743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5585d6d19b60 con 0x5585d6f9c890 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.581+0000 7fdc7dffb700 1 -- 172.21.15.12:0/4133132743 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8801e040 con 0x5585d6f9c890 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdc7dffb700 1 -- 172.21.15.12:0/4133132743 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc88037030 con 0x5585d6f9c890 2024-05-08T23:50:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdca8ab8b80 10 client.16524 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdca8ab8b80 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdca8ab8b80 20 client.16524 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5585d700c980 0x5585d700ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdca8ab8b80 4 client.16524 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/4133132743 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5585d6ee4ec0 con 0x5585d700c980 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdc95d9b700 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5585d700c980 0x5585d700ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdc7dffb700 10 client.16524 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdc95d9b700 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5585d700c980 0x5585d700ed70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86fa14a57cdcfe56 server_cookie=6ef4fd4ef0725fe5 in_seq=0 out_seq=0 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdc7dffb700 1 -- 172.21.15.12:0/4133132743 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5585d6ee4ec0 con 0x5585d700c980 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdc7dffb700 10 client.16524 handle_command_reply: tid=0 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.582+0000 7fdca8ab8b80 1 client.16524 shutdown 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/4133132743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6f9c890 msgr2=0x5585d6fa90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6f9c890 0x5585d6fa90e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800b690 tx=0x7fdc8800bdc0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/4133132743 shutdown_connections 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585d6fa2720 0x5585d700b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585d6f9c890 0x5585d6fa90e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5585d700c980 0x5585d700ed70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 1 --2- 172.21.15.12:0/4133132743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5585d6fa1e00 0x5585d6fa9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdc7dffb700 0 client.16524 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/4133132743 >> 172.21.15.12:0/4133132743 conn(0x5585d6efa210 msgr2=0x5585d6fa4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/4133132743 shutdown_connections 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 1 -- 172.21.15.12:0/4133132743 wait complete. 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdca8ab8b80 20 client.16524 trim_cache size 0 max 16384 2024-05-08T23:50:21.584 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:50:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/1367922148 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdc800a1310 msgr2=0x7fdc800a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdc9c8fe700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdc800a1310 0x7fdc800a37d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdc88006010 tx=0x7fdc8800b260 comp rx=0 tx=0).stop 2024-05-08T23:50:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/1367922148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc98106280 msgr2=0x7fdc980737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdc9c8fe700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc98106280 0x7fdc980737e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00e9c0 tx=0x7fdc8c00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/1367922148 shutdown_connections 2024-05-08T23:50:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdc9c8fe700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdc800a1310 0x7fdc800a37d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdc9c8fe700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc98073d20 0x7fdc981a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdc9c8fe700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc98101160 0x7fdc980732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.583+0000 7fdc9c8fe700 1 --2- 172.21.15.12:0/1367922148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc98106280 0x7fdc980737e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.584+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/1367922148 >> 172.21.15.12:0/1367922148 conn(0x7fdc980fc9e0 msgr2=0x7fdc980fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.584+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/1367922148 shutdown_connections 2024-05-08T23:50:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:21.584+0000 7fdc9c8fe700 1 -- 172.21.15.12:0/1367922148 wait complete. 2024-05-08T23:50:21.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:21 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3359151282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:50:21.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:21 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3359151282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:50:21.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:21 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3359151282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:50:21.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 131644 11.25 MB/sec execute 17 sec latency 30.718 ms 2024-05-08T23:50:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.030+0000 7fc9f9d35700 1 -- 172.21.15.12:0/3142955697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f4107000 msgr2=0x7fc9f41073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.030+0000 7fc9f9d35700 1 --2- 172.21.15.12:0/3142955697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f4107000 0x7fc9f41073e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40098f0 tx=0x7fc9e4009c00 comp rx=0 tx=0).stop 2024-05-08T23:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.030+0000 7fc9f9d35700 1 -- 172.21.15.12:0/3142955697 shutdown_connections 2024-05-08T23:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.030+0000 7fc9f9d35700 1 --2- 172.21.15.12:0/3142955697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f4068ca0 0x7fc9f410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.030+0000 7fc9f9d35700 1 --2- 172.21.15.12:0/3142955697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f4107000 0x7fc9f41073e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.030+0000 7fc9f9d35700 1 --2- 172.21.15.12:0/3142955697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f40682e0 0x7fc9f4068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.030+0000 7fc9f9d35700 1 -- 172.21.15.12:0/3142955697 >> 172.21.15.12:0/3142955697 conn(0x7fc9f4076f10 msgr2=0x7fc9f4077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.030+0000 7fc9f9d35700 1 -- 172.21.15.12:0/3142955697 shutdown_connections 2024-05-08T23:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.030+0000 7fc9f9d35700 1 -- 172.21.15.12:0/3142955697 wait complete. 2024-05-08T23:50:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.030+0000 7fc9f9d35700 1 Processor -- start 2024-05-08T23:50:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f9d35700 1 -- start start 2024-05-08T23:50:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f9d35700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f40682e0 0x7fc9f4197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f9d35700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f4068ca0 0x7fc9f4197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f9d35700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f4107000 0x7fc9f419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f9d35700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f419c270 con 0x7fc9f40682e0 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f9d35700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f419c3e0 con 0x7fc9f4107000 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f9d35700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f419c5a0 con 0x7fc9f4068ca0 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f40682e0 0x7fc9f4197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f4068ca0 0x7fc9f4197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f40682e0 0x7fc9f4197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57048/0 (socket says 172.21.15.12:57048) 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f4068ca0 0x7fc9f4197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34766/0 (socket says 172.21.15.12:34766) 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f37fe700 1 -- 172.21.15.12:0/2118954490 learned_addr learned my addr 172.21.15.12:0/2118954490 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f3fff700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f4107000 0x7fc9f419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f37fe700 1 -- 172.21.15.12:0/2118954490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f4068ca0 msgr2=0x7fc9f4197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.031+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f4068ca0 0x7fc9f4197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f37fe700 1 -- 172.21.15.12:0/2118954490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f4107000 msgr2=0x7fc9f419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f4107000 0x7fc9f419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f37fe700 1 -- 172.21.15.12:0/2118954490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec009d00 con 0x7fc9f40682e0 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f3fff700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f4107000 0x7fc9f419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f2ffd700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f4068ca0 0x7fc9f4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f40682e0 0x7fc9f4197350 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4005950 tx=0x7fc9e400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f0ff9700 1 -- 172.21.15.12:0/2118954490 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e401c070 con 0x7fc9f40682e0 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f0ff9700 1 -- 172.21.15.12:0/2118954490 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e400bb20 con 0x7fc9f40682e0 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f9d35700 1 -- 172.21.15.12:0/2118954490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e4009520 con 0x7fc9f40682e0 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f0ff9700 1 -- 172.21.15.12:0/2118954490 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e40038e0 con 0x7fc9f40682e0 2024-05-08T23:50:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.032+0000 7fc9f9d35700 1 -- 172.21.15.12:0/2118954490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f419cab0 con 0x7fc9f40682e0 2024-05-08T23:50:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.035+0000 7fc9f0ff9700 1 -- 172.21.15.12:0/2118954490 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc9e4020030 con 0x7fc9f40682e0 2024-05-08T23:50:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.035+0000 7fc9f0ff9700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1290 0x7fc9dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.035+0000 7fc9f0ff9700 1 -- 172.21.15.12:0/2118954490 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc9e40cdb50 con 0x7fc9f40682e0 2024-05-08T23:50:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.035+0000 7fc9f2ffd700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1290 0x7fc9dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.036+0000 7fc9f2ffd700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1290 0x7fc9dc0a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc9f41988a0 tx=0x7fc9e8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.036+0000 7fca05eefb80 1 Processor -- start 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.036+0000 7fca05eefb80 1 -- start start 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.036+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace416950 0x556ace416d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.036+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace410f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.036+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace411480 0x556ace4118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.036+0000 7fca05eefb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace3680f0 con 0x556ace411480 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.036+0000 7fca05eefb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace365ab0 con 0x556ace416950 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fca05eefb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace3613a0 con 0x556ace417270 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace410f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace411480 0x556ace4118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace410f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34774/0 (socket says 172.21.15.12:34774) 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f2ffd700 1 -- 172.21.15.12:0/212501548 learned_addr learned my addr 172.21.15.12:0/212501548 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/212501548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace416950 0x556ace416d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f2ffd700 1 -- 172.21.15.12:0/212501548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace416950 msgr2=0x556ace416d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f2ffd700 1 --2- 172.21.15.12:0/212501548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace416950 0x556ace416d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f2ffd700 1 -- 172.21.15.12:0/212501548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace411480 msgr2=0x556ace4118a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f2ffd700 1 --2- 172.21.15.12:0/212501548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace411480 0x556ace4118a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f2ffd700 1 -- 172.21.15.12:0/212501548 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ace1e2350 con 0x556ace417270 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9f2ffd700 1 --2- 172.21.15.12:0/212501548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace410f40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80043c0 tx=0x7fc9e8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9565c03540c754c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9d9ffb700 1 -- 172.21.15.12:0/212501548 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8003d10 con 0x556ace417270 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9d9ffb700 1 -- 172.21.15.12:0/212501548 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e8003e70 con 0x556ace417270 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.037+0000 7fc9d9ffb700 1 -- 172.21.15.12:0/212501548 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e801e7e0 con 0x556ace417270 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.038+0000 7fca05eefb80 1 -- 172.21.15.12:0/212501548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 msgr2=0x556ace410f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.038+0000 7fca05eefb80 1 --2- 172.21.15.12:0/212501548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace410f40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80043c0 tx=0x7fc9e8014000 comp rx=0 tx=0).stop 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.038+0000 7fca05eefb80 1 -- 172.21.15.12:0/212501548 shutdown_connections 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.038+0000 7fca05eefb80 1 --2- 172.21.15.12:0/212501548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace411480 0x556ace4118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.038+0000 7fca05eefb80 1 --2- 172.21.15.12:0/212501548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace416950 0x556ace416d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.038+0000 7fca05eefb80 1 --2- 172.21.15.12:0/212501548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace410f40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.038+0000 7fca05eefb80 1 -- 172.21.15.12:0/212501548 >> 172.21.15.12:0/212501548 conn(0x556ace36f1b0 msgr2=0x556ace40b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.038+0000 7fca05eefb80 1 -- 172.21.15.12:0/212501548 shutdown_connections 2024-05-08T23:50:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.038+0000 7fca05eefb80 1 -- 172.21.15.12:0/212501548 wait complete. 2024-05-08T23:50:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.039+0000 7fca05eefb80 1 -- start start 2024-05-08T23:50:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.039+0000 7fca05eefb80 1 Processor -- start 2024-05-08T23:50:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.039+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace41a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.039+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace416950 0x556ace41ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.039+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 0x556ace41b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.039+0000 7fca05eefb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace2fd500 con 0x556ace417270 2024-05-08T23:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.039+0000 7fca05eefb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace2fd1a0 con 0x556ace411480 2024-05-08T23:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.039+0000 7fca05eefb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace1e4640 con 0x556ace416950 2024-05-08T23:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fca05eefb80 10 client.0 _ll_register_callbacks cb 0x556ace306ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 0x556ace41b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace41a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace416950 0x556ace41ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 0x556ace41b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57056/0 (socket says 172.21.15.12:57056) 2024-05-08T23:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f3fff700 1 -- 172.21.15.12:0/2335238358 learned_addr learned my addr 172.21.15.12:0/2335238358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f3fff700 1 -- 172.21.15.12:0/2335238358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace416950 msgr2=0x556ace41ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f3fff700 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace416950 0x556ace41ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f3fff700 1 -- 172.21.15.12:0/2335238358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 msgr2=0x556ace41a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f3fff700 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace41a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f3fff700 1 -- 172.21.15.12:0/2335238358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e8017070 con 0x556ace417270 2024-05-08T23:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9f3fff700 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 0x556ace41b0a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0093b0 tx=0x7fc9ec0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7116b961d1707893 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2335238358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec00e050 con 0x556ace417270 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2335238358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc9ec004db0 con 0x556ace417270 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.040+0000 7fca05eefb80 1 -- 172.21.15.12:0/2335238358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ace2ba210 con 0x556ace417270 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2335238358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9ec013930 con 0x556ace417270 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fca05eefb80 10 client.16536 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fca05eefb80 1 -- 172.21.15.12:0/2335238358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556ace18eb00 con 0x556ace417270 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2335238358 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ace2ba210 con 0x556ace417270 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2335238358 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9ec01e020 con 0x556ace417270 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fca05eefb80 10 client.16536 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fca05eefb80 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fca05eefb80 20 client.16536 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556ace42ad60 0x556ace42d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fca05eefb80 4 client.16536 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fca05eefb80 1 -- 172.21.15.12:0/2335238358 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ace3e07b0 con 0x556ace42ad60 2024-05-08T23:50:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.041+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556ace42ad60 0x556ace42d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.042+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556ace42ad60 0x556ace42d180 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9a3300a2ad01ba server_cookie=a0f6f225e54de1a3 in_seq=0 out_seq=0 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.042+0000 7fc9d2ffd700 10 client.16536 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2335238358 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ace3e07b0 con 0x556ace42ad60 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fc9d2ffd700 10 client.16536 handle_command_reply: tid=0 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 client.16536 shutdown 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 -- 172.21.15.12:0/2335238358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 msgr2=0x556ace41b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 0x556ace41b0a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0093b0 tx=0x7fc9ec0093e0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 -- 172.21.15.12:0/2335238358 shutdown_connections 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 0x556ace41b0a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556ace42ad60 0x556ace42d180 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace41a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2335238358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace416950 0x556ace41ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fc9d2ffd700 0 client.16536 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 -- 172.21.15.12:0/2335238358 >> 172.21.15.12:0/2335238358 conn(0x556ace36f1b0 msgr2=0x556ace4154a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 -- 172.21.15.12:0/2335238358 shutdown_connections 2024-05-08T23:50:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 1 -- 172.21.15.12:0/2335238358 wait complete. 2024-05-08T23:50:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.043+0000 7fca05eefb80 20 client.16536 trim_cache size 0 max 16384 2024-05-08T23:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.060+0000 7fca05eefb80 1 Processor -- start 2024-05-08T23:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.060+0000 7fca05eefb80 1 -- start start 2024-05-08T23:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.060+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace40ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.060+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace416950 0x556ace416d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.060+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace40b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.060+0000 7fca05eefb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace41bbc0 con 0x556ace416950 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.060+0000 7fca05eefb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace403a70 con 0x556ace411480 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.060+0000 7fca05eefb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace432380 con 0x556ace417270 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace416950 0x556ace416d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace40ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace416950 0x556ace416d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57070/0 (socket says 172.21.15.12:57070) 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f2ffd700 1 -- 172.21.15.12:0/1833838885 learned_addr learned my addr 172.21.15.12:0/1833838885 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f3fff700 1 --2- 172.21.15.12:0/1833838885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace40b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f37fe700 1 -- 172.21.15.12:0/1833838885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 msgr2=0x556ace40b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/1833838885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace40b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f37fe700 1 -- 172.21.15.12:0/1833838885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace416950 msgr2=0x556ace416d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/1833838885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace416950 0x556ace416d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f37fe700 1 -- 172.21.15.12:0/1833838885 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ace327500 con 0x556ace411480 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f3fff700 1 --2- 172.21.15.12:0/1833838885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace40b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f2ffd700 1 --2- 172.21.15.12:0/1833838885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace416950 0x556ace416d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/1833838885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace40ef30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40cb2e0 tx=0x7fc9e4019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d70079cbe47a05ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9da7fc700 1 -- 172.21.15.12:0/1833838885 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e409ad30 con 0x556ace411480 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9da7fc700 1 -- 172.21.15.12:0/1833838885 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e409eb60 con 0x556ace411480 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.061+0000 7fc9da7fc700 1 -- 172.21.15.12:0/1833838885 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e40051d0 con 0x556ace411480 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.062+0000 7fca05eefb80 1 -- 172.21.15.12:0/1833838885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 msgr2=0x556ace40ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.062+0000 7fca05eefb80 1 --2- 172.21.15.12:0/1833838885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace40ef30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40cb2e0 tx=0x7fc9e4019040 comp rx=0 tx=0).stop 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.062+0000 7fca05eefb80 1 -- 172.21.15.12:0/1833838885 shutdown_connections 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.062+0000 7fca05eefb80 1 --2- 172.21.15.12:0/1833838885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace417270 0x556ace40b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.062+0000 7fca05eefb80 1 --2- 172.21.15.12:0/1833838885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace40ef30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.062+0000 7fca05eefb80 1 --2- 172.21.15.12:0/1833838885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace416950 0x556ace416d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.062+0000 7fca05eefb80 1 -- 172.21.15.12:0/1833838885 >> 172.21.15.12:0/1833838885 conn(0x556ace36f1b0 msgr2=0x556ace41af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.062+0000 7fca05eefb80 1 -- 172.21.15.12:0/1833838885 shutdown_connections 2024-05-08T23:50:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.062+0000 7fca05eefb80 1 -- 172.21.15.12:0/1833838885 wait complete. 2024-05-08T23:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.062+0000 7fca05eefb80 1 -- start start 2024-05-08T23:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fca05eefb80 1 Processor -- start 2024-05-08T23:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace421cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace416950 0x556ace41f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fca05eefb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 0x556ace41f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fca05eefb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace41fad0 con 0x556ace417270 2024-05-08T23:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fca05eefb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace41fc40 con 0x556ace411480 2024-05-08T23:50:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fca05eefb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ace41fe00 con 0x556ace416950 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fca05eefb80 10 client.0 _ll_register_callbacks cb 0x556ace306ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fc9f37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace421cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fc9f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace416950 0x556ace41f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fc9f37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace421cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41112/0 (socket says 172.21.15.12:41112) 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fc9f37fe700 1 -- 172.21.15.12:0/2776246075 learned_addr learned my addr 172.21.15.12:0/2776246075 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.063+0000 7fc9f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 0x556ace41f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9f37fe700 1 -- 172.21.15.12:0/2776246075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace416950 msgr2=0x556ace41f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace416950 0x556ace41f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9f37fe700 1 -- 172.21.15.12:0/2776246075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 msgr2=0x556ace41f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 0x556ace41f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9f37fe700 1 -- 172.21.15.12:0/2776246075 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ace1e2350 con 0x556ace411480 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9f37fe700 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace421cc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40a2bf0 tx=0x7fc9e4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4614c36e62031647 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:50:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2776246075 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e40051d0 con 0x556ace411480 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2776246075 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc9e4005330 con 0x556ace411480 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fca05eefb80 1 -- 172.21.15.12:0/2776246075 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9e8003e70 con 0x556ace411480 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2776246075 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e4021380 con 0x556ace411480 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fca05eefb80 10 client.26336 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fca05eefb80 1 -- 172.21.15.12:0/2776246075 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556ace18eb00 con 0x556ace411480 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2776246075 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e8003e70 con 0x556ace411480 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.064+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2776246075 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9e409a220 con 0x556ace411480 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.065+0000 7fca05eefb80 10 client.26336 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.065+0000 7fca05eefb80 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.065+0000 7fca05eefb80 20 client.26336 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.065+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556ace4811a0 0x556ace483590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.065+0000 7fca05eefb80 4 client.26336 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:50:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.065+0000 7fca05eefb80 1 -- 172.21.15.12:0/2776246075 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ace359e60 con 0x556ace4811a0 2024-05-08T23:50:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.065+0000 7fc9f2ffd700 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556ace4811a0 0x556ace483590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.065+0000 7fc9f2ffd700 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556ace4811a0 0x556ace483590 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ba7074fd76fc1 server_cookie=4151d73642ca2135 in_seq=0 out_seq=0 2024-05-08T23:50:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.065+0000 7fc9d2ffd700 10 client.26336 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fc9d2ffd700 1 -- 172.21.15.12:0/2776246075 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ace359e60 con 0x556ace4811a0 2024-05-08T23:50:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fc9d2ffd700 10 client.26336 handle_command_reply: tid=0 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 client.26336 shutdown 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 -- 172.21.15.12:0/2776246075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 msgr2=0x556ace421cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace421cc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40a2bf0 tx=0x7fc9e4009c00 comp rx=0 tx=0).stop 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 -- 172.21.15.12:0/2776246075 shutdown_connections 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556ace417270 0x556ace41f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556ace4811a0 0x556ace483590 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556ace411480 0x556ace421cc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 --2- 172.21.15.12:0/2776246075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ace416950 0x556ace41f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fc9d2ffd700 0 client.26336 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 -- 172.21.15.12:0/2776246075 >> 172.21.15.12:0/2776246075 conn(0x556ace36f1b0 msgr2=0x556ace410660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 -- 172.21.15.12:0/2776246075 shutdown_connections 2024-05-08T23:50:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 1 -- 172.21.15.12:0/2776246075 wait complete. 2024-05-08T23:50:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.123+0000 7fca05eefb80 20 client.26336 trim_cache size 0 max 16384 2024-05-08T23:50:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:50:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:50:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:50:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:50:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 -- 172.21.15.12:0/2118954490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1290 msgr2=0x7fc9dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1290 0x7fc9dc0a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc9f41988a0 tx=0x7fc9e8009260 comp rx=0 tx=0).stop 2024-05-08T23:50:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 -- 172.21.15.12:0/2118954490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f40682e0 msgr2=0x7fc9f4197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f40682e0 0x7fc9f4197350 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4005950 tx=0x7fc9e400fa00 comp rx=0 tx=0).stop 2024-05-08T23:50:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 -- 172.21.15.12:0/2118954490 shutdown_connections 2024-05-08T23:50:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9dc0a1290 0x7fc9dc0a3750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9f4107000 0x7fc9f419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f40682e0 0x7fc9f4197350 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 --2- 172.21.15.12:0/2118954490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f4068ca0 0x7fc9f4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 -- 172.21.15.12:0/2118954490 >> 172.21.15.12:0/2118954490 conn(0x7fc9f4076f10 msgr2=0x7fc9f40fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 -- 172.21.15.12:0/2118954490 shutdown_connections 2024-05-08T23:50:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:22.124+0000 7fc9f9d35700 1 -- 172.21.15.12:0/2118954490 wait complete. 2024-05-08T23:50:22.593 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:50:22.593 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:50:22.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.754+0000 7f1e92222700 1 -- 172.21.15.12:0/1291870721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8c100a10 msgr2=0x7f1e8c104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.754+0000 7f1e92222700 1 --2- 172.21.15.12:0/1291870721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8c100a10 0x7f1e8c104a30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009990 tx=0x7f1e74009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.754+0000 7f1e92222700 1 -- 172.21.15.12:0/1291870721 shutdown_connections 2024-05-08T23:50:22.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.754+0000 7f1e92222700 1 --2- 172.21.15.12:0/1291870721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e8c105160 0x7f1e8c10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.754+0000 7f1e92222700 1 --2- 172.21.15.12:0/1291870721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e8c100060 0x7f1e8c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.754+0000 7f1e92222700 1 --2- 172.21.15.12:0/1291870721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8c100a10 0x7f1e8c104a30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.754+0000 7f1e92222700 1 -- 172.21.15.12:0/1291870721 >> 172.21.15.12:0/1291870721 conn(0x7f1e8c076f40 msgr2=0x7f1e8c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.755+0000 7f1e92222700 1 -- 172.21.15.12:0/1291870721 shutdown_connections 2024-05-08T23:50:22.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.755+0000 7f1e92222700 1 -- 172.21.15.12:0/1291870721 wait complete. 2024-05-08T23:50:22.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.755+0000 7f1e92222700 1 Processor -- start 2024-05-08T23:50:22.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.755+0000 7f1e92222700 1 -- start start 2024-05-08T23:50:22.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e92222700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8c100060 0x7f1e8c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e92222700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e8c105160 0x7f1e8c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e92222700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e8c19bc70 0x7f1e8c19c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e92222700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c19c6f0 con 0x7f1e8c100060 2024-05-08T23:50:22.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e92222700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c19c860 con 0x7f1e8c105160 2024-05-08T23:50:22.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e92222700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c19ca20 con 0x7f1e8c19bc70 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8c100060 0x7f1e8c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e8c19bc70 0x7f1e8c19c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8c100060 0x7f1e8c1973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57094/0 (socket says 172.21.15.12:57094) 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e8c105160 0x7f1e8c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/2380955742 learned_addr learned my addr 172.21.15.12:0/2380955742 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/2380955742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e8c19bc70 msgr2=0x7f1e8c19c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e8c19bc70 0x7f1e8c19c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/2380955742 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e8c105160 msgr2=0x7f1e8c197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e8c105160 0x7f1e8c197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/2380955742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740095f0 con 0x7f1e8c100060 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8bfff700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e8c19bc70 0x7f1e8c19c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.756+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8c100060 0x7f1e8c1973e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00e9a0 tx=0x7f1e7c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.757+0000 7f1e88ff9700 1 -- 172.21.15.12:0/2380955742 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c009790 con 0x7f1e8c100060 2024-05-08T23:50:22.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.757+0000 7f1e92222700 1 -- 172.21.15.12:0/2380955742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c1a3f80 con 0x7f1e8c100060 2024-05-08T23:50:22.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.757+0000 7f1e88ff9700 1 -- 172.21.15.12:0/2380955742 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e7c010430 con 0x7f1e8c100060 2024-05-08T23:50:22.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.757+0000 7f1e88ff9700 1 -- 172.21.15.12:0/2380955742 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c018680 con 0x7f1e8c100060 2024-05-08T23:50:22.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.757+0000 7f1e92222700 1 -- 172.21.15.12:0/2380955742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c1a43b0 con 0x7f1e8c100060 2024-05-08T23:50:22.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.759+0000 7f1e88ff9700 1 -- 172.21.15.12:0/2380955742 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1e7c0187e0 con 0x7f1e8c100060 2024-05-08T23:50:22.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.760+0000 7f1e88ff9700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1e780a1290 0x7f1e780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.760+0000 7f1e88ff9700 1 -- 172.21.15.12:0/2380955742 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1e7c014070 con 0x7f1e8c100060 2024-05-08T23:50:22.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.760+0000 7f1e8affd700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1e780a1290 0x7f1e780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e8affd700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1e780a1290 0x7f1e780a3750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400b2e0 tx=0x7f1e74005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e9e3dcb80 1 Processor -- start 2024-05-08T23:50:22.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e9e3dcb80 1 -- start start 2024-05-08T23:50:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767588290 0x559767588670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x5597675827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767582d20 0x559767583160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597674d51d0 con 0x559767588290 2024-05-08T23:50:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597674d2b90 con 0x559767582d20 2024-05-08T23:50:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597674ce480 con 0x559767588bb0 2024-05-08T23:50:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767588290 0x559767588670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767588290 0x559767588670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57110/0 (socket says 172.21.15.12:57110) 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/755758332 learned_addr learned my addr 172.21.15.12:0/755758332 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e8bfff700 1 --2- 172.21.15.12:0/755758332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767582d20 0x559767583160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.761+0000 7f1e8affd700 1 --2- 172.21.15.12:0/755758332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x5597675827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.762+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/755758332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 msgr2=0x5597675827e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.762+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/755758332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x5597675827e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.762+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/755758332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767582d20 msgr2=0x559767583160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.762+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/755758332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767582d20 0x559767583160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.762+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/755758332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55976734f430 con 0x559767588290 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.762+0000 7f1e8bfff700 1 --2- 172.21.15.12:0/755758332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767582d20 0x559767583160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.762+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/755758332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767588290 0x559767588670 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0a1e80 tx=0x7f1e7c09a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b422dea7eba26f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.762+0000 7f1e81ffb700 1 -- 172.21.15.12:0/755758332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0a23b0 con 0x559767588290 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.762+0000 7f1e81ffb700 1 -- 172.21.15.12:0/755758332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e7c0105a0 con 0x559767588290 2024-05-08T23:50:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.762+0000 7f1e81ffb700 1 -- 172.21.15.12:0/755758332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c099420 con 0x559767588290 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.763+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/755758332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767588290 msgr2=0x559767588670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.763+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/755758332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767588290 0x559767588670 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0a1e80 tx=0x7f1e7c09a630 comp rx=0 tx=0).stop 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.763+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/755758332 shutdown_connections 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.763+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/755758332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767582d20 0x559767583160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.763+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/755758332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767588290 0x559767588670 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.763+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/755758332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x5597675827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.763+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/755758332 >> 172.21.15.12:0/755758332 conn(0x5597674dc290 msgr2=0x559767578e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.763+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/755758332 shutdown_connections 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.763+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/755758332 wait complete. 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.763+0000 7f1e9e3dcb80 1 -- start start 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e9e3dcb80 1 Processor -- start 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x55976758bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588290 0x55976758c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588bb0 0x55976758c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55976746a5e0 con 0x559767582d20 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55976746a280 con 0x559767588bb0 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559767351720 con 0x559767588290 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e9e3dcb80 10 client.0 _ll_register_callbacks cb 0x559767473bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e8affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588290 0x55976758c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x55976758bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e8affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588290 0x55976758c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34838/0 (socket says 172.21.15.12:34838) 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e8affd700 1 -- 172.21.15.12:0/374551823 learned_addr learned my addr 172.21.15.12:0/374551823 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e8bfff700 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588bb0 0x55976758c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e8affd700 1 -- 172.21.15.12:0/374551823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588bb0 msgr2=0x55976758c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e8affd700 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588bb0 0x55976758c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e8affd700 1 -- 172.21.15.12:0/374551823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 msgr2=0x55976758bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e8affd700 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x55976758bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.764+0000 7f1e8affd700 1 -- 172.21.15.12:0/374551823 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e84003a60 con 0x559767588290 2024-05-08T23:50:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e8affd700 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588290 0x55976758c420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009990 tx=0x7f1e74009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=768c66029cca8611 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e6affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e6affd700 1 -- 172.21.15.12:0/374551823 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74004c40 con 0x559767588290 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e6affd700 1 -- 172.21.15.12:0/374551823 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1e74004da0 con 0x559767588290 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/374551823 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597674272f0 con 0x559767588290 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e6affd700 1 -- 172.21.15.12:0/374551823 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e7400f460 con 0x559767588290 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e9e3dcb80 10 client.26329 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/374551823 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f1e8c1a43b0 con 0x559767588290 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e6affd700 1 -- 172.21.15.12:0/374551823 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5597674272f0 con 0x559767588290 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e6affd700 1 -- 172.21.15.12:0/374551823 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1e7402f070 con 0x559767588290 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e9e3dcb80 10 client.26329 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e9e3dcb80 10 client.26329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e9e3dcb80 20 client.26329 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55976759c6d0 0x55976759eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e9e3dcb80 4 client.26329 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.765+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/374551823 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55976754d890 con 0x55976759c6d0 2024-05-08T23:50:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.766+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55976759c6d0 0x55976759eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:22 smithi160 ceph-mon[46152]: pgmap v123: 97 pgs: 97 active+clean; 54 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 355 op/s 2024-05-08T23:50:22.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:22 smithi204 ceph-mon[47373]: pgmap v123: 97 pgs: 97 active+clean; 54 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 355 op/s 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.766+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55976759c6d0 0x55976759eaf0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a01c0836a28b06 server_cookie=20ed2bbee1f1c5a3 in_seq=0 out_seq=0 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.766+0000 7f1e6affd700 10 client.26329 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e6affd700 1 -- 172.21.15.12:0/374551823 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55976754d890 con 0x55976759c6d0 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e6affd700 10 client.26329 handle_command_reply: tid=0 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 client.26329 shutdown 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/374551823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588290 msgr2=0x55976758c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588290 0x55976758c420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009990 tx=0x7f1e74009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/374551823 shutdown_connections 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55976759c6d0 0x55976759eaf0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588bb0 0x55976758c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x55976758bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/374551823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588290 0x55976758c420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e6affd700 0 client.26329 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/374551823 >> 172.21.15.12:0/374551823 conn(0x5597674dc290 msgr2=0x559767578620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/374551823 shutdown_connections 2024-05-08T23:50:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/374551823 wait complete. 2024-05-08T23:50:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.767+0000 7f1e9e3dcb80 20 client.26329 trim_cache size 0 max 16384 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.787+0000 7f1e9e3dcb80 1 Processor -- start 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.787+0000 7f1e9e3dcb80 1 -- start start 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.787+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767578da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.787+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 0x5597675876c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.787+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x55976758c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559767573880 con 0x559767582d20 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597675a3d00 con 0x559767588290 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55976758a430 con 0x559767588bb0 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767578da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x55976758c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 0x5597675876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767578da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57122/0 (socket says 172.21.15.12:57122) 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/2943098150 learned_addr learned my addr 172.21.15.12:0/2943098150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8bfff700 1 -- 172.21.15.12:0/2943098150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 msgr2=0x5597675876c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8bfff700 1 --2- 172.21.15.12:0/2943098150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 0x5597675876c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8bfff700 1 -- 172.21.15.12:0/2943098150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 msgr2=0x559767578da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8bfff700 1 --2- 172.21.15.12:0/2943098150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767578da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8bfff700 1 -- 172.21.15.12:0/2943098150 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597672fbbe0 con 0x559767588bb0 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e8bfff700 1 --2- 172.21.15.12:0/2943098150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x55976758c780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400e9a0 tx=0x7f1e84009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dafd4ed0ec97542 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e827fc700 1 -- 172.21.15.12:0/2943098150 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84004db0 con 0x559767588bb0 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e827fc700 1 -- 172.21.15.12:0/2943098150 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e8400f040 con 0x559767588bb0 2024-05-08T23:50:22.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.788+0000 7f1e827fc700 1 -- 172.21.15.12:0/2943098150 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84007900 con 0x559767588bb0 2024-05-08T23:50:22.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.789+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/2943098150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 msgr2=0x55976758c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.789+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/2943098150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x55976758c780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400e9a0 tx=0x7f1e84009920 comp rx=0 tx=0).stop 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.789+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/2943098150 shutdown_connections 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.789+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/2943098150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x55976758c780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.789+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/2943098150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767578da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.789+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/2943098150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 0x5597675876c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.789+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/2943098150 >> 172.21.15.12:0/2943098150 conn(0x5597674dc290 msgr2=0x559767586fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.789+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/2943098150 shutdown_connections 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.789+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/2943098150 wait complete. 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e9e3dcb80 1 -- start start 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e9e3dcb80 1 Processor -- start 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767584b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 0x559767585040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e9e3dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x5597675f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559767585950 con 0x559767582d20 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597675f3110 con 0x559767588290 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e9e3dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597675f3250 con 0x559767588bb0 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e9e3dcb80 10 client.0 _ll_register_callbacks cb 0x559767473bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767584b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767584b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57136/0 (socket says 172.21.15.12:57136) 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e8affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 0x559767585040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/1454484398 learned_addr learned my addr 172.21.15.12:0/1454484398 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.790+0000 7f1e8bfff700 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x5597675f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/1454484398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 msgr2=0x5597675f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x5597675f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/1454484398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 msgr2=0x559767585040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 0x559767585040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e8b7fe700 1 -- 172.21.15.12:0/1454484398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8400e680 con 0x559767582d20 2024-05-08T23:50:22.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e8bfff700 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x5597675f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e8affd700 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 0x559767585040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e8b7fe700 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767584b00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c096430 tx=0x7f1e7c099960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b124ec8d19c0eef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e6affd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e6affd700 1 -- 172.21.15.12:0/1454484398 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c099d00 con 0x559767582d20 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e6affd700 1 -- 172.21.15.12:0/1454484398 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1e7c0a23b0 con 0x559767582d20 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/1454484398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e7c0105a0 con 0x559767582d20 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e6affd700 1 -- 172.21.15.12:0/1454484398 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e7c096270 con 0x559767582d20 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e9e3dcb80 10 client.16560 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/1454484398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5597674945e0 con 0x559767582d20 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.791+0000 7f1e6affd700 1 -- 172.21.15.12:0/1454484398 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e7c0105a0 con 0x559767582d20 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.792+0000 7f1e6affd700 1 -- 172.21.15.12:0/1454484398 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1e7c09a190 con 0x559767582d20 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.792+0000 7f1e9e3dcb80 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.792+0000 7f1e9e3dcb80 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.792+0000 7f1e9e3dcb80 20 client.16560 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.792+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5597675f4830 0x5597675f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.792+0000 7f1e9e3dcb80 4 client.16560 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.792+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/1454484398 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5597674c6f40 con 0x5597675f4830 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.792+0000 7f1e8affd700 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5597675f4830 0x5597675f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.792+0000 7f1e6affd700 10 client.16560 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.792+0000 7f1e8affd700 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5597675f4830 0x5597675f6c20 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c02d2762a9afcf server_cookie=31345fad0619e4c7 in_seq=0 out_seq=0 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.793+0000 7f1e6affd700 1 -- 172.21.15.12:0/1454484398 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5597674c6f40 con 0x5597675f4830 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.793+0000 7f1e6affd700 10 client.16560 handle_command_reply: tid=0 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.793+0000 7f1e9e3dcb80 1 client.16560 shutdown 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/1454484398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 msgr2=0x559767584b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767584b00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c096430 tx=0x7f1e7c099960 comp rx=0 tx=0).stop 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/1454484398 shutdown_connections 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559767588bb0 0x5597675f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559767582d20 0x559767584b00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5597675f4830 0x5597675f6c20 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 1 --2- 172.21.15.12:0/1454484398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559767588290 0x559767585040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e6affd700 0 client.16560 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/1454484398 >> 172.21.15.12:0/1454484398 conn(0x5597674dc290 msgr2=0x55976757dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/1454484398 shutdown_connections 2024-05-08T23:50:22.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 1 -- 172.21.15.12:0/1454484398 wait complete. 2024-05-08T23:50:22.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e9e3dcb80 20 client.16560 trim_cache size 0 max 16384 2024-05-08T23:50:22.794 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:50:22.794 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:50:22.795 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "a800bf9d-b88a-434b-997f-ddb566868129", 2024-05-08T23:50:22.795 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:50:22.795 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:50:22.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e92222700 1 -- 172.21.15.12:0/2380955742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1e780a1290 msgr2=0x7f1e780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e92222700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1e780a1290 0x7f1e780a3750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400b2e0 tx=0x7f1e74005960 comp rx=0 tx=0).stop 2024-05-08T23:50:22.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e92222700 1 -- 172.21.15.12:0/2380955742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8c100060 msgr2=0x7f1e8c1973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.794+0000 7f1e92222700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8c100060 0x7f1e8c1973e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00e9a0 tx=0x7f1e7c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.795+0000 7f1e92222700 1 -- 172.21.15.12:0/2380955742 shutdown_connections 2024-05-08T23:50:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.795+0000 7f1e92222700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1e780a1290 0x7f1e780a3750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.795+0000 7f1e92222700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e8c19bc70 0x7f1e8c19c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.795+0000 7f1e92222700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8c100060 0x7f1e8c1973e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.795+0000 7f1e92222700 1 --2- 172.21.15.12:0/2380955742 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e8c105160 0x7f1e8c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.795+0000 7f1e92222700 1 -- 172.21.15.12:0/2380955742 >> 172.21.15.12:0/2380955742 conn(0x7f1e8c076f40 msgr2=0x7f1e8c068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.795+0000 7f1e92222700 1 -- 172.21.15.12:0/2380955742 shutdown_connections 2024-05-08T23:50:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.795+0000 7f1e92222700 1 -- 172.21.15.12:0/2380955742 wait complete. 2024-05-08T23:50:22.803 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:22 smithi012 ceph-mon[42503]: pgmap v123: 97 pgs: 97 active+clean; 54 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 355 op/s 2024-05-08T23:50:22.805 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:50:22.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 133613 11.01 MB/sec execute 18 sec latency 59.175 ms 2024-05-08T23:50:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.954+0000 7f1c37084700 1 -- 172.21.15.12:0/903620341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c30101180 msgr2=0x7f1c30101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.954+0000 7f1c37084700 1 --2- 172.21.15.12:0/903620341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c30101180 0x7f1c30101560 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009990 tx=0x7f1c24009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.954+0000 7f1c37084700 1 -- 172.21.15.12:0/903620341 shutdown_connections 2024-05-08T23:50:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.954+0000 7f1c37084700 1 --2- 172.21.15.12:0/903620341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c301062a0 0x7f1c3010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.954+0000 7f1c37084700 1 --2- 172.21.15.12:0/903620341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c30101180 0x7f1c30101560 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.954+0000 7f1c37084700 1 --2- 172.21.15.12:0/903620341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c30101b30 0x7f1c30105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.954+0000 7f1c37084700 1 -- 172.21.15.12:0/903620341 >> 172.21.15.12:0/903620341 conn(0x7f1c300fc9e0 msgr2=0x7f1c300fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.954+0000 7f1c37084700 1 -- 172.21.15.12:0/903620341 shutdown_connections 2024-05-08T23:50:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.954+0000 7f1c37084700 1 -- 172.21.15.12:0/903620341 wait complete. 2024-05-08T23:50:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.955+0000 7f1c37084700 1 Processor -- start 2024-05-08T23:50:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.955+0000 7f1c37084700 1 -- start start 2024-05-08T23:50:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.955+0000 7f1c37084700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c30101180 0x7f1c30197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.955+0000 7f1c37084700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c301062a0 0x7f1c30197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c37084700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3019bdf0 0x7f1c3019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c37084700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3019c870 con 0x7f1c30101180 2024-05-08T23:50:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c37084700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3019c9e0 con 0x7f1c301062a0 2024-05-08T23:50:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c37084700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3019cba0 con 0x7f1c3019bdf0 2024-05-08T23:50:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c34e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c30101180 0x7f1c30197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c34e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c30101180 0x7f1c30197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57160/0 (socket says 172.21.15.12:57160) 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c34e20700 1 -- 172.21.15.12:0/269002789 learned_addr learned my addr 172.21.15.12:0/269002789 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c301062a0 0x7f1c30197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c35621700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3019bdf0 0x7f1c3019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c34e20700 1 -- 172.21.15.12:0/269002789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3019bdf0 msgr2=0x7f1c3019c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c34e20700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3019bdf0 0x7f1c3019c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c34e20700 1 -- 172.21.15.12:0/269002789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c301062a0 msgr2=0x7f1c30197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c34e20700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c301062a0 0x7f1c30197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c34e20700 1 -- 172.21.15.12:0/269002789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c240095f0 con 0x7f1c30101180 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c34e20700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c30101180 0x7f1c30197510 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005910 tx=0x7f1c24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c2dffb700 1 -- 172.21.15.12:0/269002789 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2401d070 con 0x7f1c30101180 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c37084700 1 -- 172.21.15.12:0/269002789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c301a40a0 con 0x7f1c30101180 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c2dffb700 1 -- 172.21.15.12:0/269002789 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c24022470 con 0x7f1c30101180 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.956+0000 7f1c2dffb700 1 -- 172.21.15.12:0/269002789 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24017650 con 0x7f1c30101180 2024-05-08T23:50:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.957+0000 7f1c37084700 1 -- 172.21.15.12:0/269002789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c301a44c0 con 0x7f1c30101180 2024-05-08T23:50:22.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.959+0000 7f1c2dffb700 1 -- 172.21.15.12:0/269002789 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1c240225e0 con 0x7f1c30101180 2024-05-08T23:50:22.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.960+0000 7f1c2dffb700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c200a1290 0x7f1c200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.960+0000 7f1c2dffb700 1 -- 172.21.15.12:0/269002789 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1c240cdfc0 con 0x7f1c30101180 2024-05-08T23:50:22.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.960+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c200a1290 0x7f1c200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.960+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c200a1290 0x7f1c200a3750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c30198a60 tx=0x7f1c1c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c4323eb80 1 Processor -- start 2024-05-08T23:50:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c4323eb80 1 -- start start 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 0x5558ec47d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47db30 0x5558ec477760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec4780e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec3ca150 con 0x5558ec477ca0 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec3c7b10 con 0x5558ec47d210 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec3c3400 con 0x5558ec47db30 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c35621700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec4780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c34e20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 0x5558ec47d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c35621700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec4780e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57176/0 (socket says 172.21.15.12:57176) 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c35621700 1 -- 172.21.15.12:0/3844337026 learned_addr learned my addr 172.21.15.12:0/3844337026 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/3844337026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47db30 0x5558ec477760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c34e20700 1 -- 172.21.15.12:0/3844337026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47db30 msgr2=0x5558ec477760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c34e20700 1 --2- 172.21.15.12:0/3844337026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47db30 0x5558ec477760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c34e20700 1 -- 172.21.15.12:0/3844337026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 msgr2=0x5558ec4780e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c34e20700 1 --2- 172.21.15.12:0/3844337026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec4780e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.961+0000 7f1c34e20700 1 -- 172.21.15.12:0/3844337026 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558ec2443b0 con 0x5558ec47d210 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.962+0000 7f1c34e20700 1 --2- 172.21.15.12:0/3844337026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 0x5558ec47d5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c240cb750 tx=0x7f1c24022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c36fbbc0ab6b968 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.962+0000 7f1c1affd700 1 -- 172.21.15.12:0/3844337026 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c240cbea0 con 0x5558ec47d210 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.962+0000 7f1c1affd700 1 -- 172.21.15.12:0/3844337026 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c2409ac20 con 0x5558ec47d210 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.962+0000 7f1c1affd700 1 -- 172.21.15.12:0/3844337026 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2409b520 con 0x5558ec47d210 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3844337026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 msgr2=0x5558ec47d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/3844337026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 0x5558ec47d5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c240cb750 tx=0x7f1c24022f90 comp rx=0 tx=0).stop 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3844337026 shutdown_connections 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/3844337026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec4780e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/3844337026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 0x5558ec47d5f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/3844337026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47db30 0x5558ec477760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3844337026 >> 172.21.15.12:0/3844337026 conn(0x5558ec3d1210 msgr2=0x5558ec46ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3844337026 shutdown_connections 2024-05-08T23:50:22.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3844337026 wait complete. 2024-05-08T23:50:22.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 -- start start 2024-05-08T23:50:22.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.963+0000 7f1c4323eb80 1 Processor -- start 2024-05-08T23:50:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec477ca0 0x5558ec483b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 0x5558ec480e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec35f560 con 0x5558ec47db30 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec35f200 con 0x5558ec47d210 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec2466a0 con 0x5558ec477ca0 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c4323eb80 10 client.0 _ll_register_callbacks cb 0x5558ec368b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c35621700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c2ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 0x5558ec480e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c35621700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4815c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57190/0 (socket says 172.21.15.12:57190) 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c35621700 1 -- 172.21.15.12:0/1911711130 learned_addr learned my addr 172.21.15.12:0/1911711130 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c34e20700 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec477ca0 0x5558ec483b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c2ffff700 1 -- 172.21.15.12:0/1911711130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec477ca0 msgr2=0x5558ec483b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec477ca0 0x5558ec483b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c2ffff700 1 -- 172.21.15.12:0/1911711130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 msgr2=0x5558ec4815c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4815c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.964+0000 7f1c2ffff700 1 -- 172.21.15.12:0/1911711130 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c301a44c0 con 0x5558ec47d210 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c35621700 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4815c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 0x5558ec480e80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097c0 tx=0x7f1c1c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b4d81df0dde0bb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c0ffff700 1 -- 172.21.15.12:0/1911711130 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c022070 con 0x5558ec47d210 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c0ffff700 1 -- 172.21.15.12:0/1911711130 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1c1c003d40 con 0x5558ec47d210 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1911711130 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558ec31c270 con 0x5558ec47d210 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c0ffff700 1 -- 172.21.15.12:0/1911711130 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c1c027420 con 0x5558ec47d210 2024-05-08T23:50:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c4323eb80 10 client.26366 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1911711130 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5558ec1f0b60 con 0x5558ec47d210 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c0ffff700 1 -- 172.21.15.12:0/1911711130 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558ec31c270 con 0x5558ec47d210 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.965+0000 7f1c0ffff700 1 -- 172.21.15.12:0/1911711130 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c1c01d5d0 con 0x5558ec47d210 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.966+0000 7f1c4323eb80 10 client.26366 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.966+0000 7f1c4323eb80 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.966+0000 7f1c4323eb80 20 client.26366 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.966+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558ec4913d0 0x5558ec4937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.966+0000 7f1c4323eb80 4 client.26366 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.966+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1911711130 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558ec442810 con 0x5558ec4913d0 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.966+0000 7f1c34e20700 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558ec4913d0 0x5558ec4937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.966+0000 7f1c0ffff700 10 client.26366 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.966+0000 7f1c34e20700 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558ec4913d0 0x5558ec4937f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf0de335f576462 server_cookie=703de5901e5ec700 in_seq=0 out_seq=0 2024-05-08T23:50:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c0ffff700 1 -- 172.21.15.12:0/1911711130 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558ec442810 con 0x5558ec4913d0 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c0ffff700 10 client.26366 handle_command_reply: tid=0 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c4323eb80 1 client.26366 shutdown 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1911711130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 msgr2=0x5558ec480e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 0x5558ec480e80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097c0 tx=0x7f1c1c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1911711130 shutdown_connections 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4815c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec477ca0 0x5558ec483b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558ec4913d0 0x5558ec4937f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/1911711130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47d210 0x5558ec480e80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c0ffff700 0 client.26366 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.967+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1911711130 >> 172.21.15.12:0/1911711130 conn(0x5558ec3d1210 msgr2=0x5558ec46d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.968+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1911711130 shutdown_connections 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.968+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1911711130 wait complete. 2024-05-08T23:50:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.968+0000 7f1c4323eb80 20 client.26366 trim_cache size 0 max 16384 2024-05-08T23:50:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.986+0000 7f1c4323eb80 1 Processor -- start 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.986+0000 7f1c4323eb80 1 -- start start 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.986+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec46f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.986+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47d210 0x5558ec478080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.986+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47db30 0x5558ec46dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.986+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec480cd0 con 0x5558ec477ca0 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.986+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec47f400 con 0x5558ec47db30 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.986+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec468a90 con 0x5558ec47d210 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c34e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec46f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c34e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec46f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57204/0 (socket says 172.21.15.12:57204) 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c34e20700 1 -- 172.21.15.12:0/1131795084 learned_addr learned my addr 172.21.15.12:0/1131795084 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c35621700 1 --2- 172.21.15.12:0/1131795084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47db30 0x5558ec46dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/1131795084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47d210 0x5558ec478080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c34e20700 1 -- 172.21.15.12:0/1131795084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47d210 msgr2=0x5558ec478080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c34e20700 1 --2- 172.21.15.12:0/1131795084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47d210 0x5558ec478080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c34e20700 1 -- 172.21.15.12:0/1131795084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47db30 msgr2=0x5558ec46dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c34e20700 1 --2- 172.21.15.12:0/1131795084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47db30 0x5558ec46dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c34e20700 1 -- 172.21.15.12:0/1131795084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558ec389560 con 0x5558ec477ca0 2024-05-08T23:50:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c34e20700 1 --2- 172.21.15.12:0/1131795084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec46f870 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005f50 tx=0x7f1c2401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a14ad333e9bd834 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c1b7fe700 1 -- 172.21.15.12:0/1131795084 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2409b9c0 con 0x5558ec477ca0 2024-05-08T23:50:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c1b7fe700 1 -- 172.21.15.12:0/1131795084 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c2409f0f0 con 0x5558ec477ca0 2024-05-08T23:50:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.987+0000 7f1c1b7fe700 1 -- 172.21.15.12:0/1131795084 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2409b780 con 0x5558ec477ca0 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.988+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1131795084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 msgr2=0x5558ec46f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.988+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/1131795084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec46f870 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005f50 tx=0x7f1c2401a040 comp rx=0 tx=0).stop 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.988+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1131795084 shutdown_connections 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.988+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/1131795084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec47db30 0x5558ec46dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.988+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/1131795084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec477ca0 0x5558ec46f870 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.988+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/1131795084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47d210 0x5558ec478080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.988+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1131795084 >> 172.21.15.12:0/1131795084 conn(0x5558ec3d1210 msgr2=0x5558ec46a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.988+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1131795084 shutdown_connections 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.988+0000 7f1c4323eb80 1 -- 172.21.15.12:0/1131795084 wait complete. 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.988+0000 7f1c4323eb80 1 -- start start 2024-05-08T23:50:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c4323eb80 1 Processor -- start 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec477ca0 0x5558ec4862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47d210 0x5558ec486800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c4323eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec487110 con 0x5558ec47db30 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec4e6e10 con 0x5558ec477ca0 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c4323eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ec4e6f50 con 0x5558ec47d210 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c4323eb80 10 client.0 _ll_register_callbacks cb 0x5558ec368b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c34e20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec477ca0 0x5558ec4862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c35621700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c34e20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec477ca0 0x5558ec4862c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41240/0 (socket says 172.21.15.12:41240) 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c34e20700 1 -- 172.21.15.12:0/3325424546 learned_addr learned my addr 172.21.15.12:0/3325424546 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.989+0000 7f1c35621700 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4e68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57212/0 (socket says 172.21.15.12:57212) 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47d210 0x5558ec486800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c34e20700 1 -- 172.21.15.12:0/3325424546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47d210 msgr2=0x5558ec486800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c34e20700 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47d210 0x5558ec486800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c34e20700 1 -- 172.21.15.12:0/3325424546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 msgr2=0x5558ec4e68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c34e20700 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4e68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c34e20700 1 -- 172.21.15.12:0/3325424546 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558ec2443b0 con 0x5558ec477ca0 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c35621700 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c34e20700 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec477ca0 0x5558ec4862c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1c240977c0 tx=0x7f1c24005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eafde3a1d8ff69b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c0ffff700 1 -- 172.21.15.12:0/3325424546 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2409b9c0 con 0x5558ec477ca0 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3325424546 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c2409ac20 con 0x5558ec477ca0 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c0ffff700 1 -- 172.21.15.12:0/3325424546 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1c240b8040 con 0x5558ec477ca0 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c0ffff700 1 -- 172.21.15.12:0/3325424546 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c2409a850 con 0x5558ec477ca0 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c4323eb80 10 client.26372 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.990+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3325424546 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5558ec248cd0 con 0x5558ec477ca0 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c0ffff700 1 -- 172.21.15.12:0/3325424546 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c2409ac20 con 0x5558ec477ca0 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c0ffff700 1 -- 172.21.15.12:0/3325424546 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c24083020 con 0x5558ec477ca0 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c4323eb80 10 client.26372 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c4323eb80 10 client.26372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c4323eb80 20 client.26372 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558ec4e8250 0x5558ec4ea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c4323eb80 4 client.26372 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3325424546 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5558ec3bbec0 con 0x5558ec4e8250 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558ec4e8250 0x5558ec4ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c2ffff700 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558ec4e8250 0x5558ec4ea640 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb9d6fda6bde79b server_cookie=5b3f03699ef79ed6 in_seq=0 out_seq=0 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.991+0000 7f1c0ffff700 10 client.26372 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c0ffff700 1 -- 172.21.15.12:0/3325424546 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5558ec3bbec0 con 0x5558ec4e8250 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c0ffff700 10 client.26372 handle_command_reply: tid=0 2024-05-08T23:50:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 client.26372 shutdown 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3325424546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec477ca0 msgr2=0x5558ec4862c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec477ca0 0x5558ec4862c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1c240977c0 tx=0x7f1c24005c70 comp rx=0 tx=0).stop 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3325424546 shutdown_connections 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558ec47db30 0x5558ec4e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558ec4e8250 0x5558ec4ea640 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558ec477ca0 0x5558ec4862c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 --2- 172.21.15.12:0/3325424546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558ec47d210 0x5558ec486800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c0ffff700 0 client.26372 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3325424546 >> 172.21.15.12:0/3325424546 conn(0x5558ec3d1210 msgr2=0x5558ec478080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3325424546 shutdown_connections 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 1 -- 172.21.15.12:0/3325424546 wait complete. 2024-05-08T23:50:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.992+0000 7f1c4323eb80 20 client.26372 trim_cache size 0 max 16384 2024-05-08T23:50:22.993 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:50:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:50:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:50:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "a800bf9d-b88a-434b-997f-ddb566868129": { 2024-05-08T23:50:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:50:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "a800bf9d-b88a-434b-997f-ddb566868129", 2024-05-08T23:50:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:50:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:50:22.993 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:50:22.993 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:50:22.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 -- 172.21.15.12:0/269002789 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c200a1290 msgr2=0x7f1c200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c200a1290 0x7f1c200a3750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c30198a60 tx=0x7f1c1c009260 comp rx=0 tx=0).stop 2024-05-08T23:50:22.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 -- 172.21.15.12:0/269002789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c30101180 msgr2=0x7f1c30197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:22.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c30101180 0x7f1c30197510 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005910 tx=0x7f1c24004730 comp rx=0 tx=0).stop 2024-05-08T23:50:22.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 -- 172.21.15.12:0/269002789 shutdown_connections 2024-05-08T23:50:22.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c200a1290 0x7f1c200a3750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3019bdf0 0x7f1c3019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c30101180 0x7f1c30197510 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 --2- 172.21.15.12:0/269002789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c301062a0 0x7f1c30197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:22.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 -- 172.21.15.12:0/269002789 >> 172.21.15.12:0/269002789 conn(0x7f1c300fc9e0 msgr2=0x7f1c300fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:22.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 -- 172.21.15.12:0/269002789 shutdown_connections 2024-05-08T23:50:22.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:22.993+0000 7f1c37084700 1 -- 172.21.15.12:0/269002789 wait complete. 2024-05-08T23:50:23.002 INFO:tasks.cephfs.filesystem:scrub status for tag:a800bf9d-b88a-434b-997f-ddb566868129 - {'path': '/', 'tag': 'a800bf9d-b88a-434b-997f-ddb566868129', 'options': 'recursive,force'} 2024-05-08T23:50:23.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:23 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:50:23.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:23 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:23.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:23 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:50:23.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:23 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:50:23.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:23 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:23.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:23 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:23.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:23 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:50:23.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:23 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:23.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:23 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:50:23.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:23 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:50:23.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:23 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:23.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:23 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:23.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 135326 11.07 MB/sec execute 19 sec latency 14.189 ms 2024-05-08T23:50:23.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:23 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:50:23.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:23 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:23.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:23 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:50:23.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:23 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:50:23.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:23 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:23.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:23 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:24 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:50:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:24 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:24 smithi204 ceph-mon[47373]: pgmap v124: 97 pgs: 97 active+clean; 54 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-05-08T23:50:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:24 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:50:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:24 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:24 smithi160 ceph-mon[46152]: pgmap v124: 97 pgs: 97 active+clean; 54 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-05-08T23:50:24.857 INFO:tasks.workunit.client.0.smithi012.stdout: 1 137830 11.15 MB/sec execute 20 sec latency 13.593 ms 2024-05-08T23:50:24.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:24 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:50:24.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:24 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:24.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:24 smithi012 ceph-mon[42503]: pgmap v124: 97 pgs: 97 active+clean; 54 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-05-08T23:50:25.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:25 smithi204 ceph-mon[47373]: pgmap v125: 97 pgs: 97 active+clean; 54 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-05-08T23:50:25.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:25 smithi160 ceph-mon[46152]: pgmap v125: 97 pgs: 97 active+clean; 54 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-05-08T23:50:25.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 139440 11.02 MB/sec execute 21 sec latency 37.089 ms 2024-05-08T23:50:25.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:25 smithi012 ceph-mon[42503]: pgmap v125: 97 pgs: 97 active+clean; 54 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-05-08T23:50:26.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 142151 11.29 MB/sec execute 22 sec latency 12.447 ms 2024-05-08T23:50:27.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 143890 11.05 MB/sec execute 23 sec latency 12.043 ms 2024-05-08T23:50:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:28 smithi204 ceph-mon[47373]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 356 op/s 2024-05-08T23:50:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:28 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:50:28.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:28 smithi160 ceph-mon[46152]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 356 op/s 2024-05-08T23:50:28.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:28 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:50:28.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 145900 11.15 MB/sec execute 24 sec latency 10.813 ms 2024-05-08T23:50:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:28 smithi012 ceph-mon[42503]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 356 op/s 2024-05-08T23:50:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:28 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:50:29.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:50:29.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:50:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:50:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:50:29.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 148469 11.21 MB/sec execute 25 sec latency 14.815 ms 2024-05-08T23:50:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:50:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:50:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:30 smithi204 ceph-mon[47373]: pgmap v127: 97 pgs: 97 active+clean; 52 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 250 op/s 2024-05-08T23:50:30.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:30 smithi160 ceph-mon[46152]: pgmap v127: 97 pgs: 97 active+clean; 52 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 250 op/s 2024-05-08T23:50:30.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 150142 11.11 MB/sec execute 26 sec latency 26.890 ms 2024-05-08T23:50:30.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:30 smithi012 ceph-mon[42503]: pgmap v127: 97 pgs: 97 active+clean; 52 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 250 op/s 2024-05-08T23:50:31.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 152796 11.32 MB/sec execute 27 sec latency 23.560 ms 2024-05-08T23:50:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:32 smithi204 ceph-mon[47373]: pgmap v128: 97 pgs: 97 active+clean; 59 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 357 op/s 2024-05-08T23:50:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:32 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:50:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:32 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:32 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:32 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:32 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:32 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:32 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:32 smithi160 ceph-mon[46152]: pgmap v128: 97 pgs: 97 active+clean; 59 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 357 op/s 2024-05-08T23:50:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:32 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:50:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:32 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:32 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:32 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:32 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:32 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:32 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 154326 11.12 MB/sec execute 28 sec latency 17.610 ms 2024-05-08T23:50:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:32 smithi012 ceph-mon[42503]: pgmap v128: 97 pgs: 97 active+clean; 59 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 357 op/s 2024-05-08T23:50:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:32 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:50:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:32 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:32 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:32 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:32 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:32 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:32 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:33.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:33 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:50:33.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:33 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:50:33.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:33 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:33.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:33 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:50:33.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:33 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:50:33.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:33 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:33.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156517 11.21 MB/sec execute 29 sec latency 10.267 ms 2024-05-08T23:50:33.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:33 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:50:33.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:33 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:50:33.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:33 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:50:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:34 smithi204 ceph-mon[47373]: pgmap v129: 97 pgs: 97 active+clean; 59 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-05-08T23:50:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:34 smithi160 ceph-mon[46152]: pgmap v129: 97 pgs: 97 active+clean; 59 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-05-08T23:50:34.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 158859 11.20 MB/sec execute 30 sec latency 29.222 ms 2024-05-08T23:50:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:34 smithi012 ceph-mon[42503]: pgmap v129: 97 pgs: 97 active+clean; 59 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-05-08T23:50:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.258+0000 7fd19d4f9700 1 -- 172.21.15.12:0/37216728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1980ffda0 msgr2=0x7fd198100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.258+0000 7fd19d4f9700 1 --2- 172.21.15.12:0/37216728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1980ffda0 0x7fd198100220 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009990 tx=0x7fd18c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.258+0000 7fd19d4f9700 1 -- 172.21.15.12:0/37216728 shutdown_connections 2024-05-08T23:50:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.258+0000 7fd19d4f9700 1 --2- 172.21.15.12:0/37216728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd198100760 0x7fd19810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.258+0000 7fd19d4f9700 1 --2- 172.21.15.12:0/37216728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd198105e60 0x7fd198106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.258+0000 7fd19d4f9700 1 --2- 172.21.15.12:0/37216728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1980ffda0 0x7fd198100220 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.258+0000 7fd19d4f9700 1 -- 172.21.15.12:0/37216728 >> 172.21.15.12:0/37216728 conn(0x7fd198076f10 msgr2=0x7fd198077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.258+0000 7fd19d4f9700 1 -- 172.21.15.12:0/37216728 shutdown_connections 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.258+0000 7fd19d4f9700 1 -- 172.21.15.12:0/37216728 wait complete. 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.259+0000 7fd19d4f9700 1 Processor -- start 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.259+0000 7fd19d4f9700 1 -- start start 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.259+0000 7fd19d4f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1980ffda0 0x7fd198197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.259+0000 7fd19d4f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd198100760 0x7fd1981978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.259+0000 7fd19d4f9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd198105e60 0x7fd19819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.259+0000 7fd19d4f9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19819c230 con 0x7fd198100760 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.259+0000 7fd19d4f9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19819c3a0 con 0x7fd1980ffda0 2024-05-08T23:50:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.259+0000 7fd19d4f9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19819c560 con 0x7fd198105e60 2024-05-08T23:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd1967fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd198100760 0x7fd1981978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1980ffda0 0x7fd198197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1980ffda0 0x7fd198197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47686/0 (socket says 172.21.15.12:47686) 2024-05-08T23:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd196ffd700 1 -- 172.21.15.12:0/371537015 learned_addr learned my addr 172.21.15.12:0/371537015 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd1967fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd198100760 0x7fd1981978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50444/0 (socket says 172.21.15.12:50444) 2024-05-08T23:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd1977fe700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd198105e60 0x7fd19819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd196ffd700 1 -- 172.21.15.12:0/371537015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd198105e60 msgr2=0x7fd19819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd196ffd700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd198105e60 0x7fd19819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd196ffd700 1 -- 172.21.15.12:0/371537015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd198100760 msgr2=0x7fd1981978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd196ffd700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd198100760 0x7fd1981978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd196ffd700 1 -- 172.21.15.12:0/371537015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd190009520 con 0x7fd1980ffda0 2024-05-08T23:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd1967fc700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd198100760 0x7fd1981978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd196ffd700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1980ffda0 0x7fd198197360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd18800ef10 tx=0x7fd18800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd17ffff700 1 -- 172.21.15.12:0/371537015 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18800c9b0 con 0x7fd1980ffda0 2024-05-08T23:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd17ffff700 1 -- 172.21.15.12:0/371537015 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd18800f040 con 0x7fd1980ffda0 2024-05-08T23:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd19d4f9700 1 -- 172.21.15.12:0/371537015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c0095f0 con 0x7fd1980ffda0 2024-05-08T23:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd17ffff700 1 -- 172.21.15.12:0/371537015 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188014480 con 0x7fd1980ffda0 2024-05-08T23:50:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.260+0000 7fd19d4f9700 1 -- 172.21.15.12:0/371537015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1981a3ea0 con 0x7fd1980ffda0 2024-05-08T23:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.263+0000 7fd17ffff700 1 -- 172.21.15.12:0/371537015 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd188007690 con 0x7fd1980ffda0 2024-05-08T23:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.264+0000 7fd17ffff700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1800a1290 0x7fd1800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.264+0000 7fd1967fc700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1800a1290 0x7fd1800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.264+0000 7fd17ffff700 1 -- 172.21.15.12:0/371537015 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd1880ccf90 con 0x7fd1980ffda0 2024-05-08T23:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1a96b3b80 1 Processor -- start 2024-05-08T23:50:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1a96b3b80 1 -- start start 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1967fc700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1800a1290 0x7fd1800a3750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00b2e0 tx=0x7fd18c005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x55602572cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025726f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257278a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602567e0f0 con 0x556025727480 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602567bab0 con 0x55602572c950 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560256773a0 con 0x55602572d270 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257278a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x55602572cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257278a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50460/0 (socket says 172.21.15.12:50460) 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1977fe700 1 -- 172.21.15.12:0/579816140 learned_addr learned my addr 172.21.15.12:0/579816140 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd1967fc700 1 --2- 172.21.15.12:0/579816140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025726f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd196ffd700 1 -- 172.21.15.12:0/579816140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 msgr2=0x556025726f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd196ffd700 1 --2- 172.21.15.12:0/579816140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025726f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd196ffd700 1 -- 172.21.15.12:0/579816140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 msgr2=0x5560257278a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd196ffd700 1 --2- 172.21.15.12:0/579816140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257278a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.265+0000 7fd196ffd700 1 -- 172.21.15.12:0/579816140 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560254f8350 con 0x55602572c950 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.266+0000 7fd196ffd700 1 --2- 172.21.15.12:0/579816140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x55602572cd30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd1880ca6c0 tx=0x7fd188099c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2392093e0323cd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.266+0000 7fd17d7fa700 1 -- 172.21.15.12:0/579816140 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18809a080 con 0x55602572c950 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.266+0000 7fd17d7fa700 1 -- 172.21.15.12:0/579816140 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd18809deb0 con 0x55602572c950 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.266+0000 7fd17d7fa700 1 -- 172.21.15.12:0/579816140 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880997b0 con 0x55602572c950 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/579816140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 msgr2=0x55602572cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/579816140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x55602572cd30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd1880ca6c0 tx=0x7fd188099c40 comp rx=0 tx=0).stop 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/579816140 shutdown_connections 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/579816140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257278a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/579816140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x55602572cd30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/579816140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025726f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/579816140 >> 172.21.15.12:0/579816140 conn(0x5560256851b0 msgr2=0x556025721840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/579816140 shutdown_connections 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/579816140 wait complete. 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 -- start start 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 Processor -- start 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.267+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025730b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025731070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556025613500 con 0x556025727480 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560256131a0 con 0x55602572c950 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560254fa640 con 0x55602572d270 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1a96b3b80 10 client.0 _ll_register_callbacks cb 0x55602561cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1967fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025730b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257305f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50474/0 (socket says 172.21.15.12:50474) 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd196ffd700 1 -- 172.21.15.12:0/1408848550 learned_addr learned my addr 172.21.15.12:0/1408848550 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1977fe700 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025731070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1967fc700 1 -- 172.21.15.12:0/1408848550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 msgr2=0x556025731070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1967fc700 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025731070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1967fc700 1 -- 172.21.15.12:0/1408848550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 msgr2=0x5560257305f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1967fc700 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257305f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1967fc700 1 -- 172.21.15.12:0/1408848550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1981a3ea0 con 0x55602572c950 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd196ffd700 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1977fe700 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025731070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd1967fc700 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025730b30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0059d0 tx=0x7fd18c01a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41105929ac5b4bd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd17aff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.268+0000 7fd17aff5700 1 -- 172.21.15.12:0/1408848550 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c032070 con 0x55602572c950 2024-05-08T23:50:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/1408848550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560255d0210 con 0x55602572c950 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd17aff5700 1 -- 172.21.15.12:0/1408848550 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd18c0056a0 con 0x55602572c950 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd17aff5700 1 -- 172.21.15.12:0/1408848550 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd18c02d9c0 con 0x55602572c950 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd1a96b3b80 10 client.26384 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/1408848550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5560254a4b00 con 0x55602572c950 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd17aff5700 1 -- 172.21.15.12:0/1408848550 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5560255d0210 con 0x55602572c950 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd17aff5700 1 -- 172.21.15.12:0/1408848550 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd18c037b10 con 0x55602572c950 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd1a96b3b80 10 client.26384 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd1a96b3b80 10 client.26384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd1a96b3b80 20 client.26384 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556025740cc0 0x5560257430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd1a96b3b80 4 client.26384 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/1408848550 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560256f67b0 con 0x556025740cc0 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.269+0000 7fd196ffd700 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556025740cc0 0x5560257430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.270+0000 7fd196ffd700 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556025740cc0 0x5560257430e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7d39e624d680b7 server_cookie=65dca868dbf7d86f in_seq=0 out_seq=0 2024-05-08T23:50:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.270+0000 7fd17aff5700 10 client.26384 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd17aff5700 1 -- 172.21.15.12:0/1408848550 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5560256f67b0 con 0x556025740cc0 2024-05-08T23:50:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd17aff5700 10 client.26384 handle_command_reply: tid=0 2024-05-08T23:50:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 client.26384 shutdown 2024-05-08T23:50:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/1408848550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 msgr2=0x556025730b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025730b30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0059d0 tx=0x7fd18c01a050 comp rx=0 tx=0).stop 2024-05-08T23:50:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/1408848550 shutdown_connections 2024-05-08T23:50:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025731070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x5560257305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556025740cc0 0x5560257430e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/1408848550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025730b30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/1408848550 >> 172.21.15.12:0/1408848550 conn(0x5560256851b0 msgr2=0x556025725f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd17aff5700 0 client.26384 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/1408848550 shutdown_connections 2024-05-08T23:50:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/1408848550 wait complete. 2024-05-08T23:50:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.271+0000 7fd1a96b3b80 20 client.26384 trim_cache size 0 max 16384 2024-05-08T23:50:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1a96b3b80 1 Processor -- start 2024-05-08T23:50:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1a96b3b80 1 -- start start 2024-05-08T23:50:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556025727480 0x556025721910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025730e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55602572d270 0x556025747a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556025723120 con 0x55602572d270 2024-05-08T23:50:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556025723350 con 0x55602572c950 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556025722530 con 0x556025727480 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55602572d270 0x556025747a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55602572d270 0x556025747a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50476/0 (socket says 172.21.15.12:50476) 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1977fe700 1 -- 172.21.15.12:0/456717259 learned_addr learned my addr 172.21.15.12:0/456717259 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1967fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025730e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd196ffd700 1 --2- 172.21.15.12:0/456717259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556025727480 0x556025721910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1977fe700 1 -- 172.21.15.12:0/456717259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556025727480 msgr2=0x556025721910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1977fe700 1 --2- 172.21.15.12:0/456717259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556025727480 0x556025721910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1977fe700 1 -- 172.21.15.12:0/456717259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 msgr2=0x556025730e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1977fe700 1 --2- 172.21.15.12:0/456717259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025730e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1977fe700 1 -- 172.21.15.12:0/456717259 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55602563d500 con 0x55602572d270 2024-05-08T23:50:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.297+0000 7fd1967fc700 1 --2- 172.21.15.12:0/456717259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025730e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.298+0000 7fd1977fe700 1 --2- 172.21.15.12:0/456717259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55602572d270 0x556025747a40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd190004570 tx=0x7fd190003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5335791a6c164f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.298+0000 7fd17dffb700 1 -- 172.21.15.12:0/456717259 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190016070 con 0x55602572d270 2024-05-08T23:50:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.298+0000 7fd17dffb700 1 -- 172.21.15.12:0/456717259 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd190017040 con 0x55602572d270 2024-05-08T23:50:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.298+0000 7fd17dffb700 1 -- 172.21.15.12:0/456717259 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19001b7c0 con 0x55602572d270 2024-05-08T23:50:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/456717259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55602572d270 msgr2=0x556025747a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/456717259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55602572d270 0x556025747a40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd190004570 tx=0x7fd190003960 comp rx=0 tx=0).stop 2024-05-08T23:50:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/456717259 shutdown_connections 2024-05-08T23:50:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/456717259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55602572d270 0x556025747a40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/456717259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556025727480 0x556025721910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/456717259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025730e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/456717259 >> 172.21.15.12:0/456717259 conn(0x5560256851b0 msgr2=0x5560257268c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/456717259 shutdown_connections 2024-05-08T23:50:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/456717259 wait complete. 2024-05-08T23:50:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 -- start start 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.299+0000 7fd1a96b3b80 1 Processor -- start 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x556025737f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025735bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd1a96b3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025736100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556025736810 con 0x556025727480 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556025736980 con 0x55602572c950 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd1a96b3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560257961c0 con 0x55602572d270 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd1a96b3b80 10 client.0 _ll_register_callbacks cb 0x55602561cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x556025737f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd1977fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025736100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x556025737f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50490/0 (socket says 172.21.15.12:50490) 2024-05-08T23:50:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd196ffd700 1 -- 172.21.15.12:0/4043973959 learned_addr learned my addr 172.21.15.12:0/4043973959 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd1967fc700 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025735bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd196ffd700 1 -- 172.21.15.12:0/4043973959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 msgr2=0x556025736100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd196ffd700 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025736100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd196ffd700 1 -- 172.21.15.12:0/4043973959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 msgr2=0x556025735bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd196ffd700 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025735bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd196ffd700 1 -- 172.21.15.12:0/4043973959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560254a4b00 con 0x556025727480 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd1967fc700 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025735bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd196ffd700 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x556025737f20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd1880983d0 tx=0x7fd188009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a72300293948b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.300+0000 7fd17aff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd17aff5700 1 -- 172.21.15.12:0/4043973959 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188007500 con 0x556025727480 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd17aff5700 1 -- 172.21.15.12:0/4043973959 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd18809a080 con 0x556025727480 2024-05-08T23:50:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/4043973959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd18809deb0 con 0x556025727480 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd17aff5700 1 -- 172.21.15.12:0/4043973959 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1880a2580 con 0x556025727480 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd1a96b3b80 10 client.16596 fetch_fsmap learned FSMap version 29 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/4043973959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fd190004250 con 0x556025727480 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd17aff5700 1 -- 172.21.15.12:0/4043973959 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd18809deb0 con 0x556025727480 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd17aff5700 1 -- 172.21.15.12:0/4043973959 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd1880a2070 con 0x556025727480 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd1a96b3b80 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd1a96b3b80 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd1a96b3b80 20 client.16596 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556025797530 0x556025799950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd1a96b3b80 4 client.16596 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:50:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/4043973959 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55602566fe60 con 0x556025797530 2024-05-08T23:50:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.301+0000 7fd1967fc700 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556025797530 0x556025799950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.302+0000 7fd17aff5700 10 client.16596 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.302+0000 7fd1967fc700 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556025797530 0x556025799950 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204de24f31527c45 server_cookie=eaa7500272522b0e in_seq=0 out_seq=0 2024-05-08T23:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.388+0000 7fd17aff5700 1 -- 172.21.15.12:0/4043973959 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55602566fe60 con 0x556025797530 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd17aff5700 10 client.16596 handle_command_reply: tid=0 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 client.16596 shutdown 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/4043973959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 msgr2=0x556025737f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x556025737f20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd1880983d0 tx=0x7fd188009eb0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/4043973959 shutdown_connections 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55602572d270 0x556025736100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556025727480 0x556025737f20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556025797530 0x556025799950 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 --2- 172.21.15.12:0/4043973959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55602572c950 0x556025735bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/4043973959 >> 172.21.15.12:0/4043973959 conn(0x5560256851b0 msgr2=0x556025747a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd17aff5700 0 client.16596 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/4043973959 shutdown_connections 2024-05-08T23:50:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 1 -- 172.21.15.12:0/4043973959 wait complete. 2024-05-08T23:50:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.389+0000 7fd1a96b3b80 20 client.16596 trim_cache size 0 max 16384 2024-05-08T23:50:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:50:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:50:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:50:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:50:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 -- 172.21.15.12:0/371537015 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1800a1290 msgr2=0x7fd1800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1800a1290 0x7fd1800a3750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00b2e0 tx=0x7fd18c005960 comp rx=0 tx=0).stop 2024-05-08T23:50:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 -- 172.21.15.12:0/371537015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1980ffda0 msgr2=0x7fd198197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1980ffda0 0x7fd198197360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd18800ef10 tx=0x7fd18800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 -- 172.21.15.12:0/371537015 shutdown_connections 2024-05-08T23:50:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1800a1290 0x7fd1800a3750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd198105e60 0x7fd19819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1980ffda0 0x7fd198197360 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 --2- 172.21.15.12:0/371537015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd198100760 0x7fd1981978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 -- 172.21.15.12:0/371537015 >> 172.21.15.12:0/371537015 conn(0x7fd198076f10 msgr2=0x7fd198068500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 -- 172.21.15.12:0/371537015 shutdown_connections 2024-05-08T23:50:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:35.390+0000 7fd19d4f9700 1 -- 172.21.15.12:0/371537015 wait complete. 2024-05-08T23:50:35.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 160372 11.11 MB/sec execute 31 sec latency 49.703 ms 2024-05-08T23:50:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:36 smithi204 ceph-mon[47373]: pgmap v130: 97 pgs: 97 active+clean; 59 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-05-08T23:50:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:36 smithi160 ceph-mon[46152]: pgmap v130: 97 pgs: 97 active+clean; 59 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-05-08T23:50:36.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 163016 11.21 MB/sec execute 32 sec latency 62.598 ms 2024-05-08T23:50:36.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:36 smithi012 ceph-mon[42503]: pgmap v130: 97 pgs: 97 active+clean; 59 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-05-08T23:50:37.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 164609 11.16 MB/sec execute 33 sec latency 11.310 ms 2024-05-08T23:50:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:38 smithi204 ceph-mon[47373]: pgmap v131: 97 pgs: 97 active+clean; 56 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 426 op/s 2024-05-08T23:50:38.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:38 smithi160 ceph-mon[46152]: pgmap v131: 97 pgs: 97 active+clean; 56 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 426 op/s 2024-05-08T23:50:38.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 167014 11.24 MB/sec execute 34 sec latency 28.168 ms 2024-05-08T23:50:38.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:38 smithi012 ceph-mon[42503]: pgmap v131: 97 pgs: 97 active+clean; 56 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 426 op/s 2024-05-08T23:50:39.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:39 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:50:39.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:39 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:50:39.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 168824 11.09 MB/sec execute 35 sec latency 39.007 ms 2024-05-08T23:50:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:39 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:50:40.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:40 smithi204 ceph-mon[47373]: pgmap v132: 97 pgs: 97 active+clean; 56 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-05-08T23:50:40.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:40 smithi160 ceph-mon[46152]: pgmap v132: 97 pgs: 97 active+clean; 56 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-05-08T23:50:40.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 170692 11.15 MB/sec execute 36 sec latency 14.968 ms 2024-05-08T23:50:40.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:40 smithi012 ceph-mon[42503]: pgmap v132: 97 pgs: 97 active+clean; 56 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-05-08T23:50:41.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 173168 11.17 MB/sec execute 37 sec latency 86.522 ms 2024-05-08T23:50:42.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:42 smithi204 ceph-mon[47373]: pgmap v133: 97 pgs: 97 active+clean; 61 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-05-08T23:50:42.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:42 smithi160 ceph-mon[46152]: pgmap v133: 97 pgs: 97 active+clean; 61 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-05-08T23:50:42.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 174766 11.09 MB/sec execute 38 sec latency 103.879 ms 2024-05-08T23:50:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:42 smithi012 ceph-mon[42503]: pgmap v133: 97 pgs: 97 active+clean; 61 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-05-08T23:50:43.858 INFO:tasks.workunit.client.0.smithi012.stdout: 1 177604 11.27 MB/sec execute 39 sec latency 10.535 ms 2024-05-08T23:50:44.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:44 smithi204 ceph-mon[47373]: pgmap v134: 97 pgs: 97 active+clean; 61 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-05-08T23:50:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:44 smithi160 ceph-mon[46152]: pgmap v134: 97 pgs: 97 active+clean; 61 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-05-08T23:50:44.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 179665 11.16 MB/sec execute 40 sec latency 12.501 ms 2024-05-08T23:50:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:44 smithi012 ceph-mon[42503]: pgmap v134: 97 pgs: 97 active+clean; 61 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-05-08T23:50:45.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 181359 11.19 MB/sec execute 41 sec latency 20.283 ms 2024-05-08T23:50:46.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:46 smithi204 ceph-mon[47373]: pgmap v135: 97 pgs: 97 active+clean; 61 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-05-08T23:50:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:46 smithi160 ceph-mon[46152]: pgmap v135: 97 pgs: 97 active+clean; 61 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-05-08T23:50:46.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 183924 11.22 MB/sec execute 42 sec latency 13.689 ms 2024-05-08T23:50:46.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:46 smithi012 ceph-mon[42503]: pgmap v135: 97 pgs: 97 active+clean; 61 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-05-08T23:50:47.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 185236 11.13 MB/sec execute 43 sec latency 136.226 ms 2024-05-08T23:50:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.525+0000 7f86d8060700 1 -- 172.21.15.12:0/3631385219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d0101ac0 msgr2=0x7f86d0105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.525+0000 7f86d8060700 1 --2- 172.21.15.12:0/3631385219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d0101ac0 0x7f86d0105b00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f86c00098f0 tx=0x7f86c0009c00 comp rx=0 tx=0).stop 2024-05-08T23:50:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.525+0000 7f86d8060700 1 -- 172.21.15.12:0/3631385219 shutdown_connections 2024-05-08T23:50:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.525+0000 7f86d8060700 1 --2- 172.21.15.12:0/3631385219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0106230 0x7f86d010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.525+0000 7f86d8060700 1 --2- 172.21.15.12:0/3631385219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0101110 0x7f86d01014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.525+0000 7f86d8060700 1 --2- 172.21.15.12:0/3631385219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d0101ac0 0x7f86d0105b00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.525+0000 7f86d8060700 1 -- 172.21.15.12:0/3631385219 >> 172.21.15.12:0/3631385219 conn(0x7f86d00fc9b0 msgr2=0x7f86d00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.525+0000 7f86d8060700 1 -- 172.21.15.12:0/3631385219 shutdown_connections 2024-05-08T23:50:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.525+0000 7f86d8060700 1 -- 172.21.15.12:0/3631385219 wait complete. 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.526+0000 7f86d8060700 1 Processor -- start 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.526+0000 7f86d8060700 1 -- start start 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.526+0000 7f86d8060700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101110 0x7f86d00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.526+0000 7f86d8060700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0106230 0x7f86d00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.526+0000 7f86d8060700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d0073d20 0x7f86d01a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.526+0000 7f86d8060700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01a5bf0 con 0x7f86d0073d20 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.526+0000 7f86d8060700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01a5d60 con 0x7f86d0106230 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.526+0000 7f86d8060700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01a5f20 con 0x7f86d0101110 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d65fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d0073d20 0x7f86d01a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d5dfc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101110 0x7f86d00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d65fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d0073d20 0x7f86d01a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57786/0 (socket says 172.21.15.12:57786) 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d65fd700 1 -- 172.21.15.12:0/527369386 learned_addr learned my addr 172.21.15.12:0/527369386 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d5dfc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101110 0x7f86d00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47132/0 (socket says 172.21.15.12:47132) 2024-05-08T23:50:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d55fb700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0106230 0x7f86d00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d65fd700 1 -- 172.21.15.12:0/527369386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101110 msgr2=0x7f86d00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d65fd700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101110 0x7f86d00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d65fd700 1 -- 172.21.15.12:0/527369386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0106230 msgr2=0x7f86d00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d65fd700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0106230 0x7f86d00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d65fd700 1 -- 172.21.15.12:0/527369386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc0095f0 con 0x7f86d0073d20 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d55fb700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0106230 0x7f86d00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d65fd700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d0073d20 0x7f86d01a55f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f86c800ef30 tx=0x7f86c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86c6ffd700 1 -- 172.21.15.12:0/527369386 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8018070 con 0x7f86d0073d20 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86c6ffd700 1 -- 172.21.15.12:0/527369386 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c800f040 con 0x7f86d0073d20 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86c6ffd700 1 -- 172.21.15.12:0/527369386 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c800cd00 con 0x7f86d0073d20 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d8060700 1 -- 172.21.15.12:0/527369386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c0009520 con 0x7f86d0073d20 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.527+0000 7f86d8060700 1 -- 172.21.15.12:0/527369386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d01a64a0 con 0x7f86d0073d20 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.530+0000 7f86c6ffd700 1 -- 172.21.15.12:0/527369386 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f86c8003a00 con 0x7f86d0073d20 2024-05-08T23:50:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.531+0000 7f86c6ffd700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86bc0a1310 0x7f86bc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.531+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86bc0a1310 0x7f86bc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.531+0000 7f86c6ffd700 1 -- 172.21.15.12:0/527369386 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f86c80cc700 con 0x7f86d0073d20 2024-05-08T23:50:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.531+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86bc0a1310 0x7f86bc0a37d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f86cc005fd0 tx=0x7f86cc009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.532+0000 7f86e421ab80 1 Processor -- start 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.532+0000 7f86e421ab80 1 -- start start 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.532+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.532+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b90a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.532+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 0x5625b90a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.532+0000 7f86e421ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b8ff80f0 con 0x5625b90a6950 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.532+0000 7f86e421ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b8ff5ab0 con 0x5625b90a1480 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.532+0000 7f86e421ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b8ff13a0 con 0x5625b90a7270 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d5dfc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d5dfc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90a6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57794/0 (socket says 172.21.15.12:57794) 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d5dfc700 1 -- 172.21.15.12:0/1709171497 learned_addr learned my addr 172.21.15.12:0/1709171497 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d65fd700 1 --2- 172.21.15.12:0/1709171497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 0x5625b90a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d55fb700 1 --2- 172.21.15.12:0/1709171497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b90a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d65fd700 1 -- 172.21.15.12:0/1709171497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 msgr2=0x5625b90a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d65fd700 1 --2- 172.21.15.12:0/1709171497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b90a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d65fd700 1 -- 172.21.15.12:0/1709171497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 msgr2=0x5625b90a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d65fd700 1 --2- 172.21.15.12:0/1709171497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d65fd700 1 -- 172.21.15.12:0/1709171497 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b8e72350 con 0x5625b90a1480 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/1709171497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86d65fd700 1 --2- 172.21.15.12:0/1709171497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 0x5625b90a18a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f86c80c9e90 tx=0x7f86c8099420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=475d1805c43511e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86bbfff700 1 -- 172.21.15.12:0/1709171497 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8004ad0 con 0x5625b90a1480 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86bbfff700 1 -- 172.21.15.12:0/1709171497 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c801b320 con 0x5625b90a1480 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.533+0000 7f86bbfff700 1 -- 172.21.15.12:0/1709171497 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c80b90c0 con 0x5625b90a1480 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.534+0000 7f86e421ab80 1 -- 172.21.15.12:0/1709171497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 msgr2=0x5625b90a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.534+0000 7f86e421ab80 1 --2- 172.21.15.12:0/1709171497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 0x5625b90a18a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f86c80c9e90 tx=0x7f86c8099420 comp rx=0 tx=0).stop 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.534+0000 7f86e421ab80 1 -- 172.21.15.12:0/1709171497 shutdown_connections 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.534+0000 7f86e421ab80 1 --2- 172.21.15.12:0/1709171497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 0x5625b90a18a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.534+0000 7f86e421ab80 1 --2- 172.21.15.12:0/1709171497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.534+0000 7f86e421ab80 1 --2- 172.21.15.12:0/1709171497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b90a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.534+0000 7f86e421ab80 1 -- 172.21.15.12:0/1709171497 >> 172.21.15.12:0/1709171497 conn(0x5625b8fff1b0 msgr2=0x5625b909b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.534+0000 7f86e421ab80 1 -- 172.21.15.12:0/1709171497 shutdown_connections 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.534+0000 7f86e421ab80 1 -- 172.21.15.12:0/1709171497 wait complete. 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.535+0000 7f86e421ab80 1 -- start start 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.535+0000 7f86e421ab80 1 Processor -- start 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 0x5625b90ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90aa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b90aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86e421ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b8f8d500 con 0x5625b90a6950 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86e421ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b8f8d1a0 con 0x5625b90a1480 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86e421ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b8e74640 con 0x5625b90a7270 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86e421ab80 10 client.0 _ll_register_callbacks cb 0x5625b8f96ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d55fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d55fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90aa550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57810/0 (socket says 172.21.15.12:57810) 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d5dfc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 0x5625b90ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d55fb700 1 -- 172.21.15.12:0/262014409 learned_addr learned my addr 172.21.15.12:0/262014409 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d65fd700 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b90aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d5dfc700 1 -- 172.21.15.12:0/262014409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 msgr2=0x5625b90aac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b90aac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d5dfc700 1 -- 172.21.15.12:0/262014409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 msgr2=0x5625b90aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d5dfc700 1 -- 172.21.15.12:0/262014409 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc017070 con 0x5625b90a1480 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 0x5625b90ad2d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00ed40 tx=0x7f86cc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bf80825e1df2182 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.536+0000 7f86b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86b97fa700 1 -- 172.21.15.12:0/262014409 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc022070 con 0x5625b90a1480 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86b97fa700 1 -- 172.21.15.12:0/262014409 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f86cc01d820 con 0x5625b90a1480 2024-05-08T23:50:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86e421ab80 1 -- 172.21.15.12:0/262014409 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625b8f4a210 con 0x5625b90a1480 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86b97fa700 1 -- 172.21.15.12:0/262014409 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86cc01d9e0 con 0x5625b90a1480 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86e421ab80 10 client.26402 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86e421ab80 1 -- 172.21.15.12:0/262014409 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f86d01a64a0 con 0x5625b90a1480 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86b97fa700 1 -- 172.21.15.12:0/262014409 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625b8f4a210 con 0x5625b90a1480 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86b97fa700 1 -- 172.21.15.12:0/262014409 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f86cc02e070 con 0x5625b90a1480 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86e421ab80 10 client.26402 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86e421ab80 10 client.26402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86e421ab80 20 client.26402 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86e421ab80 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b90bac40 0x5625b90bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86e421ab80 4 client.26402 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.537+0000 7f86e421ab80 1 -- 172.21.15.12:0/262014409 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625b90707b0 con 0x5625b90bac40 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.538+0000 7f86d55fb700 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b90bac40 0x5625b90bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.538+0000 7f86b97fa700 10 client.26402 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.538+0000 7f86d55fb700 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b90bac40 0x5625b90bd060 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c757788b634395 server_cookie=789e6adeee7cc56e in_seq=0 out_seq=0 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86b97fa700 1 -- 172.21.15.12:0/262014409 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625b90707b0 con 0x5625b90bac40 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86b97fa700 10 client.26402 handle_command_reply: tid=0 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 client.26402 shutdown 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 -- 172.21.15.12:0/262014409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 msgr2=0x5625b90ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 0x5625b90ad2d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00ed40 tx=0x7f86cc0047c0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 -- 172.21.15.12:0/262014409 shutdown_connections 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b90aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b90bac40 0x5625b90bd060 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a1480 0x5625b90ad2d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 --2- 172.21.15.12:0/262014409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a6950 0x5625b90aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86b97fa700 0 client.26402 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 -- 172.21.15.12:0/262014409 >> 172.21.15.12:0/262014409 conn(0x5625b8fff1b0 msgr2=0x5625b909ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 -- 172.21.15.12:0/262014409 shutdown_connections 2024-05-08T23:50:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 1 -- 172.21.15.12:0/262014409 wait complete. 2024-05-08T23:50:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.539+0000 7f86e421ab80 20 client.26402 trim_cache size 0 max 16384 2024-05-08T23:50:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86e421ab80 1 Processor -- start 2024-05-08T23:50:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86e421ab80 1 -- start start 2024-05-08T23:50:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a1480 0x5625b90aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 0x5625b90aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b909b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86e421ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b90b82f0 con 0x5625b90a1480 2024-05-08T23:50:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86e421ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b909d4b0 con 0x5625b90a6950 2024-05-08T23:50:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86e421ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b90a8aa0 con 0x5625b90a7270 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d5dfc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a1480 0x5625b90aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d5dfc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a1480 0x5625b90aac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57816/0 (socket says 172.21.15.12:57816) 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d5dfc700 1 -- 172.21.15.12:0/2977208806 learned_addr learned my addr 172.21.15.12:0/2977208806 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d55fb700 1 --2- 172.21.15.12:0/2977208806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 0x5625b90aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d65fd700 1 --2- 172.21.15.12:0/2977208806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b909b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d5dfc700 1 -- 172.21.15.12:0/2977208806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 msgr2=0x5625b909b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/2977208806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b909b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d5dfc700 1 -- 172.21.15.12:0/2977208806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 msgr2=0x5625b90aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/2977208806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 0x5625b90aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d5dfc700 1 -- 172.21.15.12:0/2977208806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b8e1eb00 con 0x5625b90a1480 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.557+0000 7f86d55fb700 1 --2- 172.21.15.12:0/2977208806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 0x5625b90aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.558+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/2977208806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a1480 0x5625b90aac90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f86cc016040 tx=0x7f86cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc14d77b089661 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.558+0000 7f86c4ff9700 1 -- 172.21.15.12:0/2977208806 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc022070 con 0x5625b90a1480 2024-05-08T23:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.558+0000 7f86c4ff9700 1 -- 172.21.15.12:0/2977208806 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86cc01c400 con 0x5625b90a1480 2024-05-08T23:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.558+0000 7f86c4ff9700 1 -- 172.21.15.12:0/2977208806 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc01de10 con 0x5625b90a1480 2024-05-08T23:50:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 -- 172.21.15.12:0/2977208806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a1480 msgr2=0x5625b90aac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 --2- 172.21.15.12:0/2977208806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a1480 0x5625b90aac90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f86cc016040 tx=0x7f86cc005880 comp rx=0 tx=0).stop 2024-05-08T23:50:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 -- 172.21.15.12:0/2977208806 shutdown_connections 2024-05-08T23:50:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 --2- 172.21.15.12:0/2977208806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a7270 0x5625b909b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 --2- 172.21.15.12:0/2977208806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a1480 0x5625b90aac90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 --2- 172.21.15.12:0/2977208806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 0x5625b90aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 -- 172.21.15.12:0/2977208806 >> 172.21.15.12:0/2977208806 conn(0x5625b8fff1b0 msgr2=0x5625b90a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 -- 172.21.15.12:0/2977208806 shutdown_connections 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 -- 172.21.15.12:0/2977208806 wait complete. 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 -- start start 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.559+0000 7f86e421ab80 1 Processor -- start 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a1480 0x5625b90afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 0x5625b90b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86e421ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a7270 0x5625b9110140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86e421ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b90b0890 con 0x5625b90a7270 2024-05-08T23:50:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86e421ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b90b0a00 con 0x5625b90a6950 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86e421ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b9110680 con 0x5625b90a1480 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86e421ab80 10 client.0 _ll_register_callbacks cb 0x5625b8f96ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d65fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a7270 0x5625b9110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d5dfc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a1480 0x5625b90afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d55fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 0x5625b90b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d65fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a7270 0x5625b9110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57830/0 (socket says 172.21.15.12:57830) 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d65fd700 1 -- 172.21.15.12:0/1375965590 learned_addr learned my addr 172.21.15.12:0/1375965590 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a1480 0x5625b90afbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47170/0 (socket says 172.21.15.12:47170) 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d65fd700 1 -- 172.21.15.12:0/1375965590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a1480 msgr2=0x5625b90afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d65fd700 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a1480 0x5625b90afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d65fd700 1 -- 172.21.15.12:0/1375965590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 msgr2=0x5625b90b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d65fd700 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 0x5625b90b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d65fd700 1 -- 172.21.15.12:0/1375965590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d010e620 con 0x5625b90a7270 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a1480 0x5625b90afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86d65fd700 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a7270 0x5625b9110140 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f86c80ca490 tx=0x7f86c8004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3e00b25a484b2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.560+0000 7f86b97fa700 1 -- 172.21.15.12:0/1375965590 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c80b2240 con 0x5625b90a7270 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86b97fa700 1 -- 172.21.15.12:0/1375965590 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f86c80b7e40 con 0x5625b90a7270 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86e421ab80 1 -- 172.21.15.12:0/1375965590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86c801b320 con 0x5625b90a7270 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86b97fa700 1 -- 172.21.15.12:0/1375965590 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c8099fe0 con 0x5625b90a7270 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86e421ab80 10 client.16614 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86e421ab80 1 -- 172.21.15.12:0/1375965590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f86d01a64a0 con 0x5625b90a7270 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86b97fa700 1 -- 172.21.15.12:0/1375965590 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c80a5020 con 0x5625b90a7270 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86b97fa700 1 -- 172.21.15.12:0/1375965590 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f86c80b91a0 con 0x5625b90a7270 2024-05-08T23:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86e421ab80 10 client.16614 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86e421ab80 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86e421ab80 20 client.16614 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86e421ab80 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b9111980 0x5625b9113da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86e421ab80 4 client.16614 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86e421ab80 1 -- 172.21.15.12:0/1375965590 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5625b8fe9e60 con 0x5625b9111980 2024-05-08T23:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.561+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b9111980 0x5625b9113da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.562+0000 7f86b97fa700 10 client.16614 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.562+0000 7f86d5dfc700 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b9111980 0x5625b9113da0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1371c757d7e5fd09 server_cookie=c06e80a72bc9fb26 in_seq=0 out_seq=0 2024-05-08T23:50:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86b97fa700 1 -- 172.21.15.12:0/1375965590 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5625b8fe9e60 con 0x5625b9111980 2024-05-08T23:50:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86b97fa700 10 client.16614 handle_command_reply: tid=0 2024-05-08T23:50:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86e421ab80 1 client.16614 shutdown 2024-05-08T23:50:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86e421ab80 1 -- 172.21.15.12:0/1375965590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a7270 msgr2=0x5625b9110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86e421ab80 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a7270 0x5625b9110140 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f86c80ca490 tx=0x7f86c8004dc0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86e421ab80 1 -- 172.21.15.12:0/1375965590 shutdown_connections 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86e421ab80 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b90a7270 0x5625b9110140 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86e421ab80 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b90a1480 0x5625b90afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86e421ab80 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b9111980 0x5625b9113da0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86e421ab80 1 --2- 172.21.15.12:0/1375965590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b90a6950 0x5625b90b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86b97fa700 0 client.16614 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.619+0000 7f86e421ab80 1 -- 172.21.15.12:0/1375965590 >> 172.21.15.12:0/1375965590 conn(0x5625b8fff1b0 msgr2=0x5625b909ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86e421ab80 1 -- 172.21.15.12:0/1375965590 shutdown_connections 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86e421ab80 1 -- 172.21.15.12:0/1375965590 wait complete. 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86e421ab80 20 client.16614 trim_cache size 0 max 16384 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:50:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:50:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:50:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:50:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86d8060700 1 -- 172.21.15.12:0/527369386 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86bc0a1310 msgr2=0x7f86bc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86d8060700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86bc0a1310 0x7f86bc0a37d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f86cc005fd0 tx=0x7f86cc009310 comp rx=0 tx=0).stop 2024-05-08T23:50:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86d8060700 1 -- 172.21.15.12:0/527369386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d0073d20 msgr2=0x7f86d01a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86d8060700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d0073d20 0x7f86d01a55f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f86c800ef30 tx=0x7f86c800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86d8060700 1 -- 172.21.15.12:0/527369386 shutdown_connections 2024-05-08T23:50:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86d8060700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86bc0a1310 0x7f86bc0a37d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86d8060700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d0073d20 0x7f86d01a55f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86d8060700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101110 0x7f86d00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86d8060700 1 --2- 172.21.15.12:0/527369386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86d0106230 0x7f86d00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.620+0000 7f86d8060700 1 -- 172.21.15.12:0/527369386 >> 172.21.15.12:0/527369386 conn(0x7f86d00fc9b0 msgr2=0x7f86d00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.621+0000 7f86d8060700 1 -- 172.21.15.12:0/527369386 shutdown_connections 2024-05-08T23:50:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:50:48.621+0000 7f86d8060700 1 -- 172.21.15.12:0/527369386 wait complete. 2024-05-08T23:50:48.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:48 smithi204 ceph-mon[47373]: pgmap v136: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-05-08T23:50:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:48 smithi160 ceph-mon[46152]: pgmap v136: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-05-08T23:50:48.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 188014 11.26 MB/sec execute 44 sec latency 52.729 ms 2024-05-08T23:50:48.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:48 smithi012 ceph-mon[42503]: pgmap v136: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-05-08T23:50:49.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:49 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:50:49.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:49 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:50:49.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 189565 11.16 MB/sec execute 45 sec latency 14.152 ms 2024-05-08T23:50:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:49 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:50:50.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:50 smithi204 ceph-mon[47373]: pgmap v137: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 250 op/s 2024-05-08T23:50:50.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:50 smithi160 ceph-mon[46152]: pgmap v137: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 250 op/s 2024-05-08T23:50:50.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 191835 11.22 MB/sec execute 46 sec latency 31.620 ms 2024-05-08T23:50:50.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:50 smithi012 ceph-mon[42503]: pgmap v137: 97 pgs: 97 active+clean; 67 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 250 op/s 2024-05-08T23:50:51.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:51 smithi204 ceph-mon[47373]: pgmap v138: 97 pgs: 97 active+clean; 64 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 423 op/s 2024-05-08T23:50:51.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:51 smithi160 ceph-mon[46152]: pgmap v138: 97 pgs: 97 active+clean; 64 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 423 op/s 2024-05-08T23:50:51.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 194118 11.15 MB/sec execute 47 sec latency 12.392 ms 2024-05-08T23:50:51.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:51 smithi012 ceph-mon[42503]: pgmap v138: 97 pgs: 97 active+clean; 64 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 423 op/s 2024-05-08T23:50:52.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 195681 11.15 MB/sec execute 48 sec latency 23.083 ms 2024-05-08T23:50:53.003 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:50:53.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.164+0000 7f0ed1640700 1 -- 172.21.15.12:0/821003040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ecc100060 msgr2=0x7f0ecc100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.164+0000 7f0ed1640700 1 --2- 172.21.15.12:0/821003040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ecc100060 0x7f0ecc100440 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b0a0 tx=0x7f0ebc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.164+0000 7f0ed1640700 1 -- 172.21.15.12:0/821003040 shutdown_connections 2024-05-08T23:50:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.164+0000 7f0ed1640700 1 --2- 172.21.15.12:0/821003040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ecc105180 0x7f0ecc10cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.164+0000 7f0ed1640700 1 --2- 172.21.15.12:0/821003040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ecc100060 0x7f0ecc100440 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.164+0000 7f0ed1640700 1 --2- 172.21.15.12:0/821003040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ecc100a10 0x7f0ecc104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.164+0000 7f0ed1640700 1 -- 172.21.15.12:0/821003040 >> 172.21.15.12:0/821003040 conn(0x7f0ecc076f10 msgr2=0x7f0ecc077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.164+0000 7f0ed1640700 1 -- 172.21.15.12:0/821003040 shutdown_connections 2024-05-08T23:50:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.164+0000 7f0ed1640700 1 -- 172.21.15.12:0/821003040 wait complete. 2024-05-08T23:50:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ed1640700 1 Processor -- start 2024-05-08T23:50:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ed1640700 1 -- start start 2024-05-08T23:50:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ed1640700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ecc100060 0x7f0ecc198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ed1640700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ecc105180 0x7f0ecc198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ed1640700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ecc198bc0 0x7f0ecc1a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ed1640700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc1a38a0 con 0x7f0ecc100060 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ed1640700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc1a3a10 con 0x7f0ecc198bc0 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ed1640700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc1a3bd0 con 0x7f0ecc105180 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ecc198bc0 0x7f0ecc1a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ecc100060 0x7f0ecc198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ecc198bc0 0x7f0ecc1a31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49746/0 (socket says 172.21.15.12:49746) 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ecc100060 0x7f0ecc198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57856/0 (socket says 172.21.15.12:57856) 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0eca7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ecc105180 0x7f0ecc198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.165+0000 7f0ecb7fe700 1 -- 172.21.15.12:0/3299144988 learned_addr learned my addr 172.21.15.12:0/3299144988 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0eca7fc700 1 -- 172.21.15.12:0/3299144988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ecc198bc0 msgr2=0x7f0ecc1a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ecc198bc0 0x7f0ecc1a31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0eca7fc700 1 -- 172.21.15.12:0/3299144988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ecc100060 msgr2=0x7f0ecc198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ecc100060 0x7f0ecc198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0eca7fc700 1 -- 172.21.15.12:0/3299144988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc009d00 con 0x7f0ecc105180 2024-05-08T23:50:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ecc105180 0x7f0ecc198680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400e9c0 tx=0x7f0eb400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0ec3fff700 1 -- 172.21.15.12:0/3299144988 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400c9c0 con 0x7f0ecc105180 2024-05-08T23:50:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0ed1640700 1 -- 172.21.15.12:0/3299144988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc1a3eb0 con 0x7f0ecc105180 2024-05-08T23:50:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0ec3fff700 1 -- 172.21.15.12:0/3299144988 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eb4010430 con 0x7f0ecc105180 2024-05-08T23:50:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0ec3fff700 1 -- 172.21.15.12:0/3299144988 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4018680 con 0x7f0ecc105180 2024-05-08T23:50:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.166+0000 7f0ed1640700 1 -- 172.21.15.12:0/3299144988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc1a4330 con 0x7f0ecc105180 2024-05-08T23:50:53.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.168+0000 7f0ec3fff700 1 -- 172.21.15.12:0/3299144988 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0eb400cb20 con 0x7f0ecc105180 2024-05-08T23:50:53.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.169+0000 7f0ec3fff700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0eb80a1290 0x7f0eb80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.169+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0eb80a1290 0x7f0eb80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.169+0000 7f0ec3fff700 1 -- 172.21.15.12:0/3299144988 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0eb4014070 con 0x7f0ecc105180 2024-05-08T23:50:53.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.170+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0eb80a1290 0x7f0eb80a3750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009610 tx=0x7f0ebc006210 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.170+0000 7f0edd7fab80 1 Processor -- start 2024-05-08T23:50:53.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.170+0000 7f0edd7fab80 1 -- start start 2024-05-08T23:50:53.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 0x56357a5a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a1ca0 0x56357a5a2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a4f4150 con 0x56357a5a7b10 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a4f1b10 con 0x56357a5a1ca0 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a4ed400 con 0x56357a5a71f0 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0eca7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 0x56357a5a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5a75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47186/0 (socket says 172.21.15.12:47186) 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0eca7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 0x56357a5a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57858/0 (socket says 172.21.15.12:57858) 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecaffd700 1 -- 172.21.15.12:0/356343018 learned_addr learned my addr 172.21.15.12:0/356343018 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecb7fe700 1 --2- 172.21.15.12:0/356343018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a1ca0 0x56357a5a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecaffd700 1 -- 172.21.15.12:0/356343018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a1ca0 msgr2=0x56357a5a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/356343018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a1ca0 0x56357a5a2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecaffd700 1 -- 172.21.15.12:0/356343018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 msgr2=0x56357a5a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/356343018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 0x56357a5a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecaffd700 1 -- 172.21.15.12:0/356343018 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56357a36e3b0 con 0x56357a5a71f0 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecb7fe700 1 --2- 172.21.15.12:0/356343018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a1ca0 0x56357a5a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.171+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/356343018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5a75d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc015000 tx=0x7f0ebc01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f399c07417f0c98a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.172+0000 7f0ec17fa700 1 -- 172.21.15.12:0/356343018 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc00e040 con 0x56357a5a71f0 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.172+0000 7f0ec17fa700 1 -- 172.21.15.12:0/356343018 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ebc0079a0 con 0x56357a5a71f0 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.172+0000 7f0ec17fa700 1 -- 172.21.15.12:0/356343018 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc01ce20 con 0x56357a5a71f0 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.172+0000 7f0edd7fab80 1 -- 172.21.15.12:0/356343018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 msgr2=0x56357a5a75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.172+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/356343018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5a75d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc015000 tx=0x7f0ebc01c000 comp rx=0 tx=0).stop 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.172+0000 7f0edd7fab80 1 -- 172.21.15.12:0/356343018 shutdown_connections 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.172+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/356343018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a1ca0 0x56357a5a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.172+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/356343018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5a75d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.172+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/356343018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 0x56357a5a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 -- 172.21.15.12:0/356343018 >> 172.21.15.12:0/356343018 conn(0x56357a4fb210 msgr2=0x56357a597df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 -- 172.21.15.12:0/356343018 shutdown_connections 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 -- 172.21.15.12:0/356343018 wait complete. 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 -- start start 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 Processor -- start 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a1ca0 0x56357a5aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 0x56357a5ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 0x56357a5aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a489560 con 0x56357a5a7b10 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a489200 con 0x56357a5a71f0 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a3706a0 con 0x56357a5a1ca0 2024-05-08T23:50:53.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.173+0000 7f0edd7fab80 10 client.0 _ll_register_callbacks cb 0x56357a492b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 0x56357a5aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a1ca0 0x56357a5aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0eca7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 0x56357a5ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 0x56357a5aba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57874/0 (socket says 172.21.15.12:57874) 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a1ca0 0x56357a5aaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47202/0 (socket says 172.21.15.12:47202) 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecb7fe700 1 -- 172.21.15.12:0/798407154 learned_addr learned my addr 172.21.15.12:0/798407154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecaffd700 1 -- 172.21.15.12:0/798407154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 msgr2=0x56357a5ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 0x56357a5ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecaffd700 1 -- 172.21.15.12:0/798407154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 msgr2=0x56357a5aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 0x56357a5aba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecaffd700 1 -- 172.21.15.12:0/798407154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec4009930 con 0x56357a5a1ca0 2024-05-08T23:50:53.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 0x56357a5ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a1ca0 0x56357a5aaf90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009fd0 tx=0x7f0ebc01c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9847e0a74cbf0eaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0eb27fc700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0eb27fc700 1 -- 172.21.15.12:0/798407154 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc01caa0 con 0x56357a5a1ca0 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0eb27fc700 1 -- 172.21.15.12:0/798407154 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0ebc01cc00 con 0x56357a5a1ca0 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.174+0000 7f0edd7fab80 1 -- 172.21.15.12:0/798407154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56357a446270 con 0x56357a5a1ca0 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0eb27fc700 1 -- 172.21.15.12:0/798407154 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ebc01cdc0 con 0x56357a5a1ca0 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0edd7fab80 10 client.26401 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0edd7fab80 1 -- 172.21.15.12:0/798407154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f0ecc10d570 con 0x56357a5a1ca0 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0eb27fc700 1 -- 172.21.15.12:0/798407154 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56357a446270 con 0x56357a5a1ca0 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0eb27fc700 1 -- 172.21.15.12:0/798407154 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0ebc027400 con 0x56357a5a1ca0 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0edd7fab80 10 client.26401 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0edd7fab80 10 client.26401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0edd7fab80 20 client.26401 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:53.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56357a5bb100 0x56357a5bd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0edd7fab80 4 client.26401 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.175+0000 7f0edd7fab80 1 -- 172.21.15.12:0/798407154 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56357a56c810 con 0x56357a5bb100 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.176+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56357a5bb100 0x56357a5bd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.176+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56357a5bb100 0x56357a5bd4f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba963ac2cfa8094 server_cookie=a879a41f7228499c in_seq=0 out_seq=0 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.176+0000 7f0eb27fc700 10 client.26401 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.176+0000 7f0eb27fc700 1 -- 172.21.15.12:0/798407154 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56357a56c810 con 0x56357a5bb100 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.176+0000 7f0eb27fc700 10 client.26401 handle_command_reply: tid=0 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 client.26401 shutdown 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 -- 172.21.15.12:0/798407154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a1ca0 msgr2=0x56357a5aaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a1ca0 0x56357a5aaf90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009fd0 tx=0x7f0ebc01c960 comp rx=0 tx=0).stop 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 -- 172.21.15.12:0/798407154 shutdown_connections 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a7b10 0x56357a5aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a1ca0 0x56357a5aaf90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56357a5bb100 0x56357a5bd4f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/798407154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 0x56357a5ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 -- 172.21.15.12:0/798407154 >> 172.21.15.12:0/798407154 conn(0x56357a4fb210 msgr2=0x56357a5975a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0eb27fc700 0 client.26401 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:53.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 -- 172.21.15.12:0/798407154 shutdown_connections 2024-05-08T23:50:53.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 1 -- 172.21.15.12:0/798407154 wait complete. 2024-05-08T23:50:53.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.177+0000 7f0edd7fab80 20 client.26401 trim_cache size 0 max 16384 2024-05-08T23:50:53.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.196+0000 7f0edd7fab80 1 Processor -- start 2024-05-08T23:50:53.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.196+0000 7f0edd7fab80 1 -- start start 2024-05-08T23:50:53.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.196+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a599870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.196+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 0x56357a597d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.196+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a7b10 0x56357a5ab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.196+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a5ac510 con 0x56357a5a1ca0 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.196+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a5c2670 con 0x56357a5a71f0 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.196+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a590040 con 0x56357a5a7b10 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a599870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a599870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57890/0 (socket says 172.21.15.12:57890) 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ecaffd700 1 -- 172.21.15.12:0/183361157 learned_addr learned my addr 172.21.15.12:0/183361157 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/183361157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 0x56357a597d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ecb7fe700 1 --2- 172.21.15.12:0/183361157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a7b10 0x56357a5ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ecaffd700 1 -- 172.21.15.12:0/183361157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a7b10 msgr2=0x56357a5ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/183361157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a7b10 0x56357a5ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ecaffd700 1 -- 172.21.15.12:0/183361157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 msgr2=0x56357a597d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/183361157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 0x56357a597d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ecaffd700 1 -- 172.21.15.12:0/183361157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56357a31ab60 con 0x56357a5a1ca0 2024-05-08T23:50:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/183361157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a599870 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b0a0 tx=0x7f0ebc007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e603337da049d8d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ec1ffb700 1 -- 172.21.15.12:0/183361157 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc00e040 con 0x56357a5a1ca0 2024-05-08T23:50:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ec1ffb700 1 -- 172.21.15.12:0/183361157 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ebc003ce0 con 0x56357a5a1ca0 2024-05-08T23:50:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.197+0000 7f0ec1ffb700 1 -- 172.21.15.12:0/183361157 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc027590 con 0x56357a5a1ca0 2024-05-08T23:50:53.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.198+0000 7f0edd7fab80 1 -- 172.21.15.12:0/183361157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 msgr2=0x56357a599870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.198+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/183361157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a599870 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b0a0 tx=0x7f0ebc007b80 comp rx=0 tx=0).stop 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.198+0000 7f0edd7fab80 1 -- 172.21.15.12:0/183361157 shutdown_connections 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.198+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/183361157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a7b10 0x56357a5ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.198+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/183361157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a599870 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.198+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/183361157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a71f0 0x56357a597d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.198+0000 7f0edd7fab80 1 -- 172.21.15.12:0/183361157 >> 172.21.15.12:0/183361157 conn(0x56357a4fb210 msgr2=0x56357a594210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.198+0000 7f0edd7fab80 1 -- 172.21.15.12:0/183361157 shutdown_connections 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.198+0000 7f0edd7fab80 1 -- 172.21.15.12:0/183361157 wait complete. 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0edd7fab80 1 -- start start 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0edd7fab80 1 Processor -- start 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a5b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0edd7fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a7b10 0x56357a5b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a5b2dc0 con 0x56357a5a1ca0 2024-05-08T23:50:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a610540 con 0x56357a5a7b10 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0edd7fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357a610680 con 0x56357a5a71f0 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0edd7fab80 10 client.0 _ll_register_callbacks cb 0x56357a492b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a5b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a5b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57896/0 (socket says 172.21.15.12:57896) 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0ecaffd700 1 -- 172.21.15.12:0/3391751006 learned_addr learned my addr 172.21.15.12:0/3391751006 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.199+0000 7f0ecaffd700 1 -- 172.21.15.12:0/3391751006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 msgr2=0x56357a5b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0ecaffd700 1 -- 172.21.15.12:0/3391751006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a7b10 msgr2=0x56357a5b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0ecb7fe700 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a7b10 0x56357a5b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a7b10 0x56357a5b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0ecaffd700 1 -- 172.21.15.12:0/3391751006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc10d570 con 0x56357a5a1ca0 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0ecaffd700 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a5b0500 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0097c0 tx=0x7f0ebc027d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ae401d432f6409 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0eb27fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0eb27fc700 1 -- 172.21.15.12:0/3391751006 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc00e070 con 0x56357a5a1ca0 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0eb27fc700 1 -- 172.21.15.12:0/3391751006 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0ebc01cb60 con 0x56357a5a1ca0 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0edd7fab80 1 -- 172.21.15.12:0/3391751006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ebc0079a0 con 0x56357a5a1ca0 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0eb27fc700 1 -- 172.21.15.12:0/3391751006 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ebc01cd20 con 0x56357a5a1ca0 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0edd7fab80 10 client.16632 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0edd7fab80 1 -- 172.21.15.12:0/3391751006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56357a4b3560 con 0x56357a5a1ca0 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0eb27fc700 1 -- 172.21.15.12:0/3391751006 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ebc0079a0 con 0x56357a5a1ca0 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.200+0000 7f0eb27fc700 1 -- 172.21.15.12:0/3391751006 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0ebc0374d0 con 0x56357a5a1ca0 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0edd7fab80 10 client.16632 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0edd7fab80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0edd7fab80 20 client.16632 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56357a611900 0x56357a613cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0edd7fab80 4 client.16632 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0edd7fab80 1 -- 172.21.15.12:0/3391751006 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56357a4e5ec0 con 0x56357a611900 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56357a611900 0x56357a613cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0eb27fc700 10 client.16632 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0eca7fc700 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56357a611900 0x56357a613cf0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806ff313e9c18d6c server_cookie=ee117895c1c5220c in_seq=0 out_seq=0 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0eb27fc700 1 -- 172.21.15.12:0/3391751006 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56357a4e5ec0 con 0x56357a611900 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0eb27fc700 10 client.16632 handle_command_reply: tid=0 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.201+0000 7f0edd7fab80 1 client.16632 shutdown 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 1 -- 172.21.15.12:0/3391751006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 msgr2=0x56357a5b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a5b0500 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0097c0 tx=0x7f0ebc027d00 comp rx=0 tx=0).stop 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 1 -- 172.21.15.12:0/3391751006 shutdown_connections 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56357a611900 0x56357a613cf0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56357a5a7b10 0x56357a5b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0eb27fc700 0 client.16632 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:53.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56357a5a1ca0 0x56357a5b0500 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 1 --2- 172.21.15.12:0/3391751006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56357a5a71f0 0x56357a5b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 1 -- 172.21.15.12:0/3391751006 >> 172.21.15.12:0/3391751006 conn(0x56357a4fb210 msgr2=0x56357a599680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 1 -- 172.21.15.12:0/3391751006 shutdown_connections 2024-05-08T23:50:53.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 1 -- 172.21.15.12:0/3391751006 wait complete. 2024-05-08T23:50:53.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0edd7fab80 20 client.16632 trim_cache size 0 max 16384 2024-05-08T23:50:53.203 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:50:53.203 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:50:53.203 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:50:53.203 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:50:53.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0ed1640700 1 -- 172.21.15.12:0/3299144988 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0eb80a1290 msgr2=0x7f0eb80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0ed1640700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0eb80a1290 0x7f0eb80a3750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009610 tx=0x7f0ebc006210 comp rx=0 tx=0).stop 2024-05-08T23:50:53.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0ed1640700 1 -- 172.21.15.12:0/3299144988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ecc105180 msgr2=0x7f0ecc198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0ed1640700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ecc105180 0x7f0ecc198680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400e9c0 tx=0x7f0eb400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0ed1640700 1 -- 172.21.15.12:0/3299144988 shutdown_connections 2024-05-08T23:50:53.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0ed1640700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0eb80a1290 0x7f0eb80a3750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0ed1640700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ecc198bc0 0x7f0ecc1a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0ed1640700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ecc100060 0x7f0ecc198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.202+0000 7f0ed1640700 1 --2- 172.21.15.12:0/3299144988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ecc105180 0x7f0ecc198680 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.203+0000 7f0ed1640700 1 -- 172.21.15.12:0/3299144988 >> 172.21.15.12:0/3299144988 conn(0x7f0ecc076f10 msgr2=0x7f0ecc0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.203+0000 7f0ed1640700 1 -- 172.21.15.12:0/3299144988 shutdown_connections 2024-05-08T23:50:53.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.203+0000 7f0ed1640700 1 -- 172.21.15.12:0/3299144988 wait complete. 2024-05-08T23:50:53.211 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:50:53.211 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:50:53.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.365+0000 7f442ce2c700 1 -- 172.21.15.12:0/1210727066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44280ff5a0 msgr2=0x7f44280ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.365+0000 7f442ce2c700 1 --2- 172.21.15.12:0/1210727066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44280ff5a0 0x7f44280ff980 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4418009990 tx=0x7f4418009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.365+0000 7f442ce2c700 1 -- 172.21.15.12:0/1210727066 shutdown_connections 2024-05-08T23:50:53.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.365+0000 7f442ce2c700 1 --2- 172.21.15.12:0/1210727066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4428106160 0x7f442810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.365+0000 7f442ce2c700 1 --2- 172.21.15.12:0/1210727066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44280ff5a0 0x7f44280ff980 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.365+0000 7f442ce2c700 1 --2- 172.21.15.12:0/1210727066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44280fff50 0x7f4428105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.365+0000 7f442ce2c700 1 -- 172.21.15.12:0/1210727066 >> 172.21.15.12:0/1210727066 conn(0x7f4428076270 msgr2=0x7f44280fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.365+0000 7f442ce2c700 1 -- 172.21.15.12:0/1210727066 shutdown_connections 2024-05-08T23:50:53.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.365+0000 7f442ce2c700 1 -- 172.21.15.12:0/1210727066 wait complete. 2024-05-08T23:50:53.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f442ce2c700 1 Processor -- start 2024-05-08T23:50:53.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f442ce2c700 1 -- start start 2024-05-08T23:50:53.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f442ce2c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44280ff5a0 0x7f4428197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f442ce2c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4428106160 0x7f4428197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f442ce2c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442819bc20 0x7f442819c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f442ce2c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442819c6a0 con 0x7f4428106160 2024-05-08T23:50:53.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f442ce2c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442819c810 con 0x7f44280ff5a0 2024-05-08T23:50:53.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f442ce2c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442819c9d0 con 0x7f442819bc20 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f4425d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4428106160 0x7f4428197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f442659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44280ff5a0 0x7f4428197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f4425d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4428106160 0x7f4428197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57922/0 (socket says 172.21.15.12:57922) 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f4425d9b700 1 -- 172.21.15.12:0/1874231505 learned_addr learned my addr 172.21.15.12:0/1874231505 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.366+0000 7f4426d9d700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442819bc20 0x7f442819c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f442659c700 1 -- 172.21.15.12:0/1874231505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442819bc20 msgr2=0x7f442819c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f442659c700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442819bc20 0x7f442819c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f442659c700 1 -- 172.21.15.12:0/1874231505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4428106160 msgr2=0x7f4428197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f442659c700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4428106160 0x7f4428197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f442659c700 1 -- 172.21.15.12:0/1874231505 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44180095f0 con 0x7f44280ff5a0 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f4425d9b700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4428106160 0x7f4428197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f442659c700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44280ff5a0 0x7f4428197340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4418000c00 tx=0x7f4418009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f441f7fe700 1 -- 172.21.15.12:0/1874231505 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441801d070 con 0x7f44280ff5a0 2024-05-08T23:50:53.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f441f7fe700 1 -- 172.21.15.12:0/1874231505 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4418021410 con 0x7f44280ff5a0 2024-05-08T23:50:53.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f442ce2c700 1 -- 172.21.15.12:0/1874231505 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44281a3ed0 con 0x7f44280ff5a0 2024-05-08T23:50:53.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f441f7fe700 1 -- 172.21.15.12:0/1874231505 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418017650 con 0x7f44280ff5a0 2024-05-08T23:50:53.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.367+0000 7f442ce2c700 1 -- 172.21.15.12:0/1874231505 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44281a4290 con 0x7f44280ff5a0 2024-05-08T23:50:53.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.368+0000 7f442ce2c700 1 -- 172.21.15.12:0/1874231505 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44281093f0 con 0x7f44280ff5a0 2024-05-08T23:50:53.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.369+0000 7f441f7fe700 1 -- 172.21.15.12:0/1874231505 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4418021580 con 0x7f44280ff5a0 2024-05-08T23:50:53.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.370+0000 7f441f7fe700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44140a1290 0x7f44140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.370+0000 7f4425d9b700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44140a1290 0x7f44140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.370+0000 7f441f7fe700 1 -- 172.21.15.12:0/1874231505 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f44180ce060 con 0x7f44280ff5a0 2024-05-08T23:50:53.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.371+0000 7f4425d9b700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44140a1290 0x7f44140a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4428198890 tx=0x7f441000b460 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.374+0000 7f441f7fe700 1 -- 172.21.15.12:0/1874231505 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4418097580 con 0x7f44280ff5a0 2024-05-08T23:50:53.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.660+0000 7f442ce2c700 1 -- 172.21.15.12:0/1874231505 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4428102650 con 0x7f44280ff5a0 2024-05-08T23:50:53.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.660+0000 7f441f7fe700 1 -- 172.21.15.12:0/1874231505 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10873 (secure 0 0 0) 0x7f441809aba0 con 0x7f44280ff5a0 2024-05-08T23:50:53.656 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:50:53.656 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:50:48.465221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_26224":{"gid":26224,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/131208281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":131208281},{"type":"v1","addr":"172.21.15.204:6837","nonce":131208281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:50:53.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 -- 172.21.15.12:0/1874231505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44140a1290 msgr2=0x7f44140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44140a1290 0x7f44140a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4428198890 tx=0x7f441000b460 comp rx=0 tx=0).stop 2024-05-08T23:50:53.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 -- 172.21.15.12:0/1874231505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44280ff5a0 msgr2=0x7f4428197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44280ff5a0 0x7f4428197340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4418000c00 tx=0x7f4418009580 comp rx=0 tx=0).stop 2024-05-08T23:50:53.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 -- 172.21.15.12:0/1874231505 shutdown_connections 2024-05-08T23:50:53.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44140a1290 0x7f44140a3750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442819bc20 0x7f442819c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44280ff5a0 0x7f4428197340 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 --2- 172.21.15.12:0/1874231505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4428106160 0x7f4428197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 -- 172.21.15.12:0/1874231505 >> 172.21.15.12:0/1874231505 conn(0x7f4428076270 msgr2=0x7f44281045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 -- 172.21.15.12:0/1874231505 shutdown_connections 2024-05-08T23:50:53.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.663+0000 7f442ce2c700 1 -- 172.21.15.12:0/1874231505 wait complete. 2024-05-08T23:50:53.659 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 31 2024-05-08T23:50:53.672 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:50:53.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.823+0000 7f98b1cdd700 1 -- 172.21.15.12:0/1605462357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ac100060 msgr2=0x7f98ac100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.823+0000 7f98b1cdd700 1 --2- 172.21.15.12:0/1605462357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ac100060 0x7f98ac100440 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f989c009990 tx=0x7f989c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.823+0000 7f98b1cdd700 1 -- 172.21.15.12:0/1605462357 shutdown_connections 2024-05-08T23:50:53.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.823+0000 7f98b1cdd700 1 --2- 172.21.15.12:0/1605462357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac105160 0x7f98ac10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.823+0000 7f98b1cdd700 1 --2- 172.21.15.12:0/1605462357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ac100060 0x7f98ac100440 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.823+0000 7f98b1cdd700 1 --2- 172.21.15.12:0/1605462357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98ac100a10 0x7f98ac104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.823+0000 7f98b1cdd700 1 -- 172.21.15.12:0/1605462357 >> 172.21.15.12:0/1605462357 conn(0x7f98ac076f40 msgr2=0x7f98ac077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.823+0000 7f98b1cdd700 1 -- 172.21.15.12:0/1605462357 shutdown_connections 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.823+0000 7f98b1cdd700 1 -- 172.21.15.12:0/1605462357 wait complete. 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98b1cdd700 1 Processor -- start 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98b1cdd700 1 -- start start 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98b1cdd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac100060 0x7f98ac196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98b1cdd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ac105160 0x7f98ac196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98b1cdd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98ac196a80 0x7f98ac1a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98b1cdd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac1a5bc0 con 0x7f98ac105160 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98b1cdd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac1a5d00 con 0x7f98ac100060 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98b1cdd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac1a5e70 con 0x7f98ac196a80 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ac105160 0x7f98ac196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ac105160 0x7f98ac196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57938/0 (socket says 172.21.15.12:57938) 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98a2ffd700 1 -- 172.21.15.12:0/3527425090 learned_addr learned my addr 172.21.15.12:0/3527425090 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98abfff700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98ac196a80 0x7f98ac1a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.824+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac100060 0x7f98ac196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98a2ffd700 1 -- 172.21.15.12:0/3527425090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98ac196a80 msgr2=0x7f98ac1a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98ac196a80 0x7f98ac1a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98a2ffd700 1 -- 172.21.15.12:0/3527425090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac100060 msgr2=0x7f98ac196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac100060 0x7f98ac196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98a2ffd700 1 -- 172.21.15.12:0/3527425090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c0095f0 con 0x7f98ac105160 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ac105160 0x7f98ac196540 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f989400e970 tx=0x7f989400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98a97fa700 1 -- 172.21.15.12:0/3527425090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894009790 con 0x7f98ac105160 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98a97fa700 1 -- 172.21.15.12:0/3527425090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f989400c7e0 con 0x7f98ac105160 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98b1cdd700 1 -- 172.21.15.12:0/3527425090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac1a6150 con 0x7f98ac105160 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98a97fa700 1 -- 172.21.15.12:0/3527425090 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940054e0 con 0x7f98ac105160 2024-05-08T23:50:53.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.825+0000 7f98b1cdd700 1 -- 172.21.15.12:0/3527425090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac1a66b0 con 0x7f98ac105160 2024-05-08T23:50:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.827+0000 7f98a97fa700 1 -- 172.21.15.12:0/3527425090 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f989400c950 con 0x7f98ac105160 2024-05-08T23:50:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.828+0000 7f98a97fa700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98980a1310 0x7f98980a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.828+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98980a1310 0x7f98980a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.828+0000 7f98a97fa700 1 -- 172.21.15.12:0/3527425090 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9894014070 con 0x7f98ac105160 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98980a1310 0x7f98980a37d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f989c00b2e0 tx=0x7f989c005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98bde97b80 1 Processor -- start 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98bde97b80 1 -- start start 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca8e00 0x562d94ca91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca9720 0x562d94ca3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94ca3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98bde97b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94bfa150 con 0x562d94ca8e00 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98bde97b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94bf7b10 con 0x562d94ca3890 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98bde97b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94bf3400 con 0x562d94ca9720 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca8e00 0x562d94ca91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca8e00 0x562d94ca91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57952/0 (socket says 172.21.15.12:57952) 2024-05-08T23:50:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98ab7fe700 1 -- 172.21.15.12:0/4172026280 learned_addr learned my addr 172.21.15.12:0/4172026280 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/4172026280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca9720 0x562d94ca3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.829+0000 7f98abfff700 1 --2- 172.21.15.12:0/4172026280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94ca3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.830+0000 7f98a2ffd700 1 -- 172.21.15.12:0/4172026280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 msgr2=0x562d94ca3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.830+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/4172026280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94ca3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.830+0000 7f98a2ffd700 1 -- 172.21.15.12:0/4172026280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca8e00 msgr2=0x562d94ca91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.830+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/4172026280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca8e00 0x562d94ca91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.830+0000 7f98a2ffd700 1 -- 172.21.15.12:0/4172026280 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d94a743b0 con 0x562d94ca9720 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.830+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/4172026280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca8e00 0x562d94ca91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.830+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/4172026280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca9720 0x562d94ca3350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98940ca650 tx=0x7f989400cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e49056336dd901f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.830+0000 7f98a1ffb700 1 -- 172.21.15.12:0/4172026280 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989409de40 con 0x562d94ca9720 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.830+0000 7f98a1ffb700 1 -- 172.21.15.12:0/4172026280 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98940a2490 con 0x562d94ca9720 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.830+0000 7f98a1ffb700 1 -- 172.21.15.12:0/4172026280 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989409a270 con 0x562d94ca9720 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.831+0000 7f98bde97b80 1 -- 172.21.15.12:0/4172026280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca9720 msgr2=0x562d94ca3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.831+0000 7f98bde97b80 1 --2- 172.21.15.12:0/4172026280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca9720 0x562d94ca3350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98940ca650 tx=0x7f989400cc50 comp rx=0 tx=0).stop 2024-05-08T23:50:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.831+0000 7f98bde97b80 1 -- 172.21.15.12:0/4172026280 shutdown_connections 2024-05-08T23:50:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.831+0000 7f98bde97b80 1 --2- 172.21.15.12:0/4172026280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94ca3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.831+0000 7f98bde97b80 1 --2- 172.21.15.12:0/4172026280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca8e00 0x562d94ca91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.831+0000 7f98bde97b80 1 --2- 172.21.15.12:0/4172026280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca9720 0x562d94ca3350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.831+0000 7f98bde97b80 1 -- 172.21.15.12:0/4172026280 >> 172.21.15.12:0/4172026280 conn(0x562d94c01210 msgr2=0x562d94c9ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.831+0000 7f98bde97b80 1 -- 172.21.15.12:0/4172026280 shutdown_connections 2024-05-08T23:50:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.831+0000 7f98bde97b80 1 -- 172.21.15.12:0/4172026280 wait complete. 2024-05-08T23:50:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.831+0000 7f98bde97b80 1 -- start start 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98bde97b80 1 Processor -- start 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94cacab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94cacff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94cad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98bde97b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94b8f560 con 0x562d94ca9720 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98bde97b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94b8f200 con 0x562d94ca3890 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98bde97b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94a766a0 con 0x562d94ca8e00 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98bde97b80 10 client.0 _ll_register_callbacks cb 0x562d94b98b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98abfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94cad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98ab7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94cacab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98a2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94cacff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98ab7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94cacab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49818/0 (socket says 172.21.15.12:49818) 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98abfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94cad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57964/0 (socket says 172.21.15.12:57964) 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98a2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94cacff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47260/0 (socket says 172.21.15.12:47260) 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.832+0000 7f98ab7fe700 1 -- 172.21.15.12:0/1037502231 learned_addr learned my addr 172.21.15.12:0/1037502231 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f98abfff700 1 -- 172.21.15.12:0/1037502231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 msgr2=0x562d94cacff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f98abfff700 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94cacff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f98abfff700 1 -- 172.21.15.12:0/1037502231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 msgr2=0x562d94cacab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f98abfff700 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94cacab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f98abfff700 1 -- 172.21.15.12:0/1037502231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac10d550 con 0x562d94ca9720 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94cacab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f98abfff700 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94cad530 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f98a4012da0 tx=0x7f98a4010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1049d1df7bbc5b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f988affd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f988affd700 1 -- 172.21.15.12:0/1037502231 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4004bb0 con 0x562d94ca9720 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f988affd700 1 -- 172.21.15.12:0/1037502231 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f98a4004d10 con 0x562d94ca9720 2024-05-08T23:50:53.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f98bde97b80 1 -- 172.21.15.12:0/1037502231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d94b4c270 con 0x562d94ca9720 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f988affd700 1 -- 172.21.15.12:0/1037502231 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98a4010a80 con 0x562d94ca9720 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f98bde97b80 10 client.16650 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.833+0000 7f98bde97b80 1 -- 172.21.15.12:0/1037502231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f98ac1a66b0 con 0x562d94ca9720 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.834+0000 7f988affd700 1 -- 172.21.15.12:0/1037502231 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d94b4c270 con 0x562d94ca9720 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.834+0000 7f988affd700 1 -- 172.21.15.12:0/1037502231 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f98a40143a0 con 0x562d94ca9720 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.834+0000 7f98bde97b80 10 client.16650 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.834+0000 7f98bde97b80 10 client.16650 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.834+0000 7f98bde97b80 20 client.16650 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.834+0000 7f98bde97b80 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x562d94cbd270 0x562d94cbf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.834+0000 7f98bde97b80 4 client.16650 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.834+0000 7f98bde97b80 1 -- 172.21.15.12:0/1037502231 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d94c72810 con 0x562d94cbd270 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.834+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x562d94cbd270 0x562d94cbf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.835+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x562d94cbd270 0x562d94cbf690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=197bbda3afd01933 server_cookie=c4200f8e193d82b6 in_seq=0 out_seq=0 2024-05-08T23:50:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.835+0000 7f988affd700 10 client.16650 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.835+0000 7f988affd700 1 -- 172.21.15.12:0/1037502231 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d94c72810 con 0x562d94cbd270 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.835+0000 7f988affd700 10 client.16650 handle_command_reply: tid=0 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.835+0000 7f98bde97b80 1 client.16650 shutdown 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.835+0000 7f98bde97b80 1 -- 172.21.15.12:0/1037502231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 msgr2=0x562d94cad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.835+0000 7f98bde97b80 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94cad530 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f98a4012da0 tx=0x7f98a4010520 comp rx=0 tx=0).stop 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.836+0000 7f98bde97b80 1 -- 172.21.15.12:0/1037502231 shutdown_connections 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.836+0000 7f98bde97b80 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94cad530 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.836+0000 7f98bde97b80 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94cacab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.836+0000 7f98bde97b80 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x562d94cbd270 0x562d94cbf690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.836+0000 7f98bde97b80 1 --2- 172.21.15.12:0/1037502231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94cacff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.836+0000 7f988affd700 0 client.16650 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.836+0000 7f98bde97b80 1 -- 172.21.15.12:0/1037502231 >> 172.21.15.12:0/1037502231 conn(0x562d94c01210 msgr2=0x562d94c9d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.836+0000 7f98bde97b80 1 -- 172.21.15.12:0/1037502231 shutdown_connections 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.836+0000 7f98bde97b80 1 -- 172.21.15.12:0/1037502231 wait complete. 2024-05-08T23:50:53.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.836+0000 7f98bde97b80 20 client.16650 trim_cache size 0 max 16384 2024-05-08T23:50:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.850+0000 7f98bde97b80 1 Processor -- start 2024-05-08T23:50:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.850+0000 7f98bde97b80 1 -- start start 2024-05-08T23:50:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.850+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94ca3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.850+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94ca8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.850+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94c9dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.850+0000 7f98bde97b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94c96110 con 0x562d94ca9720 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.850+0000 7f98bde97b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94c98800 con 0x562d94ca3890 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.850+0000 7f98bde97b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94cc4890 con 0x562d94ca8e00 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.850+0000 7f98abfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94c9dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.850+0000 7f98a2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94ca8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98abfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94c9dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57968/0 (socket says 172.21.15.12:57968) 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98abfff700 1 -- 172.21.15.12:0/3720242469 learned_addr learned my addr 172.21.15.12:0/3720242469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/3720242469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94ca3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98a2ffd700 1 -- 172.21.15.12:0/3720242469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 msgr2=0x562d94ca3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/3720242469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94ca3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98a2ffd700 1 -- 172.21.15.12:0/3720242469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 msgr2=0x562d94c9dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/3720242469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94c9dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98a2ffd700 1 -- 172.21.15.12:0/3720242469 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d94ca3f40 con 0x562d94ca8e00 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98abfff700 1 --2- 172.21.15.12:0/3720242469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94c9dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/3720242469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94ca8290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f989409a8c0 tx=0x7f9894005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5272a4217f21f15a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98a27fc700 1 -- 172.21.15.12:0/3720242469 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894005660 con 0x562d94ca8e00 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98a27fc700 1 -- 172.21.15.12:0/3720242469 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98940983b0 con 0x562d94ca8e00 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.851+0000 7f98a27fc700 1 -- 172.21.15.12:0/3720242469 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894099500 con 0x562d94ca8e00 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.852+0000 7f98bde97b80 1 -- 172.21.15.12:0/3720242469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 msgr2=0x562d94ca8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.852+0000 7f98bde97b80 1 --2- 172.21.15.12:0/3720242469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94ca8290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f989409a8c0 tx=0x7f9894005910 comp rx=0 tx=0).stop 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.852+0000 7f98bde97b80 1 -- 172.21.15.12:0/3720242469 shutdown_connections 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.852+0000 7f98bde97b80 1 --2- 172.21.15.12:0/3720242469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca9720 0x562d94c9dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.852+0000 7f98bde97b80 1 --2- 172.21.15.12:0/3720242469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca3890 0x562d94ca3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.852+0000 7f98bde97b80 1 --2- 172.21.15.12:0/3720242469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94ca8290 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.852+0000 7f98bde97b80 1 -- 172.21.15.12:0/3720242469 >> 172.21.15.12:0/3720242469 conn(0x562d94c01210 msgr2=0x562d94ca7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.852+0000 7f98bde97b80 1 -- 172.21.15.12:0/3720242469 shutdown_connections 2024-05-08T23:50:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.852+0000 7f98bde97b80 1 -- 172.21.15.12:0/3720242469 wait complete. 2024-05-08T23:50:53.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.852+0000 7f98bde97b80 1 -- start start 2024-05-08T23:50:53.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98bde97b80 1 Processor -- start 2024-05-08T23:50:53.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca3890 0x562d94cb0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94cb06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98bde97b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca9720 0x562d94d12720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98bde97b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94cb0e70 con 0x562d94ca3890 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98bde97b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94cb0fe0 con 0x562d94ca9720 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98bde97b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d94d12c60 con 0x562d94ca8e00 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98bde97b80 10 client.0 _ll_register_callbacks cb 0x562d94b98b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca3890 0x562d94cb0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca3890 0x562d94cb0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57984/0 (socket says 172.21.15.12:57984) 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98ab7fe700 1 -- 172.21.15.12:0/2883438664 learned_addr learned my addr 172.21.15.12:0/2883438664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94cb06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98abfff700 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca9720 0x562d94d12720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.853+0000 7f98ab7fe700 1 -- 172.21.15.12:0/2883438664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 msgr2=0x562d94cb06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94cb06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f98ab7fe700 1 -- 172.21.15.12:0/2883438664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca9720 msgr2=0x562d94d12720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:53.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca9720 0x562d94d12720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f98ab7fe700 1 -- 172.21.15.12:0/2883438664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac1a66b0 con 0x562d94ca3890 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94cb06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f98abfff700 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca9720 0x562d94d12720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f98ab7fe700 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca3890 0x562d94cb0190 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f989c0095c0 tx=0x7f989c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa8e34561210f3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f988affd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f988affd700 1 -- 172.21.15.12:0/2883438664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c036630 con 0x562d94ca3890 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f988affd700 1 -- 172.21.15.12:0/2883438664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f989c036790 con 0x562d94ca3890 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f98bde97b80 1 -- 172.21.15.12:0/2883438664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98940a2490 con 0x562d94ca3890 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f988affd700 1 -- 172.21.15.12:0/2883438664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f989c004500 con 0x562d94ca3890 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f98bde97b80 10 client.16656 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.854+0000 7f98bde97b80 1 -- 172.21.15.12:0/2883438664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562d94a743b0 con 0x562d94ca3890 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f988affd700 1 -- 172.21.15.12:0/2883438664 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98940a2490 con 0x562d94ca3890 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f988affd700 1 -- 172.21.15.12:0/2883438664 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f989c03f630 con 0x562d94ca3890 2024-05-08T23:50:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f98bde97b80 10 client.16656 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f98bde97b80 10 client.16656 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f98bde97b80 20 client.16656 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f98bde97b80 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x562d94d14380 0x562d94d167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f98bde97b80 4 client.16656 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f98bde97b80 1 -- 172.21.15.12:0/2883438664 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562d94bebec0 con 0x562d94d14380 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x562d94d14380 0x562d94d167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f988affd700 10 client.16656 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f98a2ffd700 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x562d94d14380 0x562d94d167a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5638f4ed11cd728c server_cookie=730a335c793cec7c in_seq=0 out_seq=0 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f988affd700 1 -- 172.21.15.12:0/2883438664 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562d94bebec0 con 0x562d94d14380 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.855+0000 7f988affd700 10 client.16656 handle_command_reply: tid=0 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 client.16656 shutdown 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 -- 172.21.15.12:0/2883438664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca3890 msgr2=0x562d94cb0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca3890 0x562d94cb0190 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f989c0095c0 tx=0x7f989c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 -- 172.21.15.12:0/2883438664 shutdown_connections 2024-05-08T23:50:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d94ca9720 0x562d94d12720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d94ca3890 0x562d94cb0190 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x562d94d14380 0x562d94d167a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 --2- 172.21.15.12:0/2883438664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d94ca8e00 0x562d94cb06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f988affd700 0 client.16656 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:50:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 -- 172.21.15.12:0/2883438664 >> 172.21.15.12:0/2883438664 conn(0x562d94c01210 msgr2=0x562d94ca8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 -- 172.21.15.12:0/2883438664 shutdown_connections 2024-05-08T23:50:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 1 -- 172.21.15.12:0/2883438664 wait complete. 2024-05-08T23:50:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.856+0000 7f98bde97b80 20 client.16656 trim_cache size 0 max 16384 2024-05-08T23:50:53.857 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:50:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 -- 172.21.15.12:0/3527425090 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98980a1310 msgr2=0x7f98980a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98980a1310 0x7f98980a37d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f989c00b2e0 tx=0x7f989c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 -- 172.21.15.12:0/3527425090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ac105160 msgr2=0x7f98ac196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ac105160 0x7f98ac196540 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f989400e970 tx=0x7f989400ec80 comp rx=0 tx=0).stop 2024-05-08T23:50:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 -- 172.21.15.12:0/3527425090 shutdown_connections 2024-05-08T23:50:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f98980a1310 0x7f98980a37d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98ac196a80 0x7f98ac1a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac100060 0x7f98ac196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 --2- 172.21.15.12:0/3527425090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ac105160 0x7f98ac196540 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 -- 172.21.15.12:0/3527425090 >> 172.21.15.12:0/3527425090 conn(0x7f98ac076f40 msgr2=0x7f98ac0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 -- 172.21.15.12:0/3527425090 shutdown_connections 2024-05-08T23:50:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:53.857+0000 7f98b1cdd700 1 -- 172.21.15.12:0/3527425090 wait complete. 2024-05-08T23:50:53.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 198281 11.20 MB/sec execute 49 sec latency 19.322 ms 2024-05-08T23:50:53.866 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:50:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.015+0000 7ff27b391700 1 -- 172.21.15.12:0/2781616730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff274101160 msgr2=0x7ff274101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.015+0000 7ff27b391700 1 --2- 172.21.15.12:0/2781616730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff274101160 0x7ff274101540 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff270009990 tx=0x7ff270009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.015+0000 7ff27b391700 1 -- 172.21.15.12:0/2781616730 shutdown_connections 2024-05-08T23:50:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.015+0000 7ff27b391700 1 --2- 172.21.15.12:0/2781616730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff274106280 0x7ff27410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.015+0000 7ff27b391700 1 --2- 172.21.15.12:0/2781616730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff274101160 0x7ff274101540 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.015+0000 7ff27b391700 1 --2- 172.21.15.12:0/2781616730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff274101b10 0x7ff274105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.015+0000 7ff27b391700 1 -- 172.21.15.12:0/2781616730 >> 172.21.15.12:0/2781616730 conn(0x7ff2740fc9e0 msgr2=0x7ff2740fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.015+0000 7ff27b391700 1 -- 172.21.15.12:0/2781616730 shutdown_connections 2024-05-08T23:50:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.015+0000 7ff27b391700 1 -- 172.21.15.12:0/2781616730 wait complete. 2024-05-08T23:50:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27b391700 1 Processor -- start 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27b391700 1 -- start start 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27b391700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff274101160 0x7ff2740732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27b391700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff274106280 0x7ff2740737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27b391700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff274073d20 0x7ff2741a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27b391700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2741a5bc0 con 0x7ff274106280 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27b391700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2741a5d00 con 0x7ff274101160 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27b391700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2741a5ec0 con 0x7ff274073d20 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27892c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff274106280 0x7ff2740737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27912d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff274101160 0x7ff2740732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27892c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff274106280 0x7ff2740737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57992/0 (socket says 172.21.15.12:57992) 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27892c700 1 -- 172.21.15.12:0/4115159005 learned_addr learned my addr 172.21.15.12:0/4115159005 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.016+0000 7ff27992e700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff274073d20 0x7ff2741a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff27912d700 1 -- 172.21.15.12:0/4115159005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff274073d20 msgr2=0x7ff2741a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff27912d700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff274073d20 0x7ff2741a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff27912d700 1 -- 172.21.15.12:0/4115159005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff274106280 msgr2=0x7ff2740737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff27912d700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff274106280 0x7ff2740737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff27912d700 1 -- 172.21.15.12:0/4115159005 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2700095f0 con 0x7ff274101160 2024-05-08T23:50:54.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff27892c700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff274106280 0x7ff2740737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:54.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff27912d700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff274101160 0x7ff2740732a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff2700059d0 tx=0x7ff270004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff26a7fc700 1 -- 172.21.15.12:0/4115159005 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27001d070 con 0x7ff274101160 2024-05-08T23:50:54.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff26a7fc700 1 -- 172.21.15.12:0/4115159005 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff270022470 con 0x7ff274101160 2024-05-08T23:50:54.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff27b391700 1 -- 172.21.15.12:0/4115159005 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2741a6140 con 0x7ff274101160 2024-05-08T23:50:54.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff26a7fc700 1 -- 172.21.15.12:0/4115159005 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270017650 con 0x7ff274101160 2024-05-08T23:50:54.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.017+0000 7ff27b391700 1 -- 172.21.15.12:0/4115159005 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2741a6560 con 0x7ff274101160 2024-05-08T23:50:54.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.019+0000 7ff26a7fc700 1 -- 172.21.15.12:0/4115159005 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff270022870 con 0x7ff274101160 2024-05-08T23:50:54.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.020+0000 7ff26a7fc700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff2600a1290 0x7ff2600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.021+0000 7ff27892c700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff2600a1290 0x7ff2600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.021+0000 7ff26a7fc700 1 -- 172.21.15.12:0/4115159005 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff2700cdee0 con 0x7ff274101160 2024-05-08T23:50:54.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.021+0000 7ff27892c700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff2600a1290 0x7ff2600a3750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff274074980 tx=0x7ff264009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff28754bb80 1 Processor -- start 2024-05-08T23:50:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff28754bb80 1 -- start start 2024-05-08T23:50:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea92e00 0x556d0ea931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea93720 0x556d0ea8d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 0x556d0ea8dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff28754bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0e9e4150 con 0x556d0ea93720 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff28754bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0e9e1b10 con 0x556d0ea8d890 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff28754bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0e9dd400 con 0x556d0ea92e00 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27892c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea93720 0x556d0ea8d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27912d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea92e00 0x556d0ea931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27892c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea93720 0x556d0ea8d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58008/0 (socket says 172.21.15.12:58008) 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27912d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea92e00 0x556d0ea931e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47318/0 (socket says 172.21.15.12:47318) 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27892c700 1 -- 172.21.15.12:0/1454010167 learned_addr learned my addr 172.21.15.12:0/1454010167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27992e700 1 --2- 172.21.15.12:0/1454010167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 0x556d0ea8dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27912d700 1 -- 172.21.15.12:0/1454010167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 msgr2=0x556d0ea8dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27912d700 1 --2- 172.21.15.12:0/1454010167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 0x556d0ea8dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27912d700 1 -- 172.21.15.12:0/1454010167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea93720 msgr2=0x556d0ea8d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27912d700 1 --2- 172.21.15.12:0/1454010167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea93720 0x556d0ea8d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.022+0000 7ff27912d700 1 -- 172.21.15.12:0/1454010167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d0e85e3b0 con 0x556d0ea92e00 2024-05-08T23:50:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.023+0000 7ff27912d700 1 --2- 172.21.15.12:0/1454010167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea92e00 0x556d0ea931e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2700cb610 tx=0x7ff27009aeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c05a9cabf89057 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.023+0000 7ff25f7fe700 1 -- 172.21.15.12:0/1454010167 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700aeea0 con 0x556d0ea92e00 2024-05-08T23:50:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.023+0000 7ff25f7fe700 1 -- 172.21.15.12:0/1454010167 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27009b060 con 0x556d0ea92e00 2024-05-08T23:50:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.023+0000 7ff25f7fe700 1 -- 172.21.15.12:0/1454010167 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27009b2c0 con 0x556d0ea92e00 2024-05-08T23:50:54.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 -- 172.21.15.12:0/1454010167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea92e00 msgr2=0x556d0ea931e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 --2- 172.21.15.12:0/1454010167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea92e00 0x556d0ea931e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2700cb610 tx=0x7ff27009aeb0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 -- 172.21.15.12:0/1454010167 shutdown_connections 2024-05-08T23:50:54.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 --2- 172.21.15.12:0/1454010167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 0x556d0ea8dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 --2- 172.21.15.12:0/1454010167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea92e00 0x556d0ea931e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 --2- 172.21.15.12:0/1454010167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea93720 0x556d0ea8d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 -- 172.21.15.12:0/1454010167 >> 172.21.15.12:0/1454010167 conn(0x556d0e9eb210 msgr2=0x556d0ea87df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 -- 172.21.15.12:0/1454010167 shutdown_connections 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 -- 172.21.15.12:0/1454010167 wait complete. 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 -- start start 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.024+0000 7ff28754bb80 1 Processor -- start 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea8d890 0x556d0ea99740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea92e00 0x556d0ea96a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea97170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff28754bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0e979560 con 0x556d0ea8d890 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff28754bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0e979200 con 0x556d0ea92e00 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff28754bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0e8606a0 con 0x556d0ea93720 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff28754bb80 10 client.0 _ll_register_callbacks cb 0x556d0e982b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff27992e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea97170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff27912d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea8d890 0x556d0ea99740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff27992e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea97170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47332/0 (socket says 172.21.15.12:47332) 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff27992e700 1 -- 172.21.15.12:0/4198939875 learned_addr learned my addr 172.21.15.12:0/4198939875 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff27892c700 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea92e00 0x556d0ea96a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff27992e700 1 -- 172.21.15.12:0/4198939875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea92e00 msgr2=0x556d0ea96a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff27992e700 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea92e00 0x556d0ea96a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff27992e700 1 -- 172.21.15.12:0/4198939875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea8d890 msgr2=0x556d0ea99740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff27992e700 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea8d890 0x556d0ea99740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.025+0000 7ff27992e700 1 -- 172.21.15.12:0/4198939875 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2741a6560 con 0x556d0ea93720 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff27992e700 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea97170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009790 tx=0x7ff26c007c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a72560607121f609 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff25cff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff25cff9700 1 -- 172.21.15.12:0/4198939875 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c00e040 con 0x556d0ea93720 2024-05-08T23:50:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff25cff9700 1 -- 172.21.15.12:0/4198939875 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff26c0138c0 con 0x556d0ea93720 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff28754bb80 1 -- 172.21.15.12:0/4198939875 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d0e936270 con 0x556d0ea93720 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff25cff9700 1 -- 172.21.15.12:0/4198939875 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26c013a80 con 0x556d0ea93720 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff28754bb80 10 client.26443 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff28754bb80 1 -- 172.21.15.12:0/4198939875 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556d0e80ab60 con 0x556d0ea93720 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff25cff9700 1 -- 172.21.15.12:0/4198939875 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d0e936270 con 0x556d0ea93720 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff25cff9700 1 -- 172.21.15.12:0/4198939875 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff26c015390 con 0x556d0ea93720 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff28754bb80 10 client.26443 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff28754bb80 10 client.26443 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff28754bb80 20 client.26443 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff28754bb80 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556d0eaa6f30 0x556d0eaa9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff28754bb80 4 client.26443 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.026+0000 7ff28754bb80 1 -- 172.21.15.12:0/4198939875 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d0ea5c810 con 0x556d0eaa6f30 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.027+0000 7ff27912d700 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556d0eaa6f30 0x556d0eaa9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.027+0000 7ff25cff9700 10 client.26443 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.027+0000 7ff27912d700 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556d0eaa6f30 0x556d0eaa9350 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28d0fabd3cb7decc server_cookie=38379d8635b01958 in_seq=0 out_seq=0 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff25cff9700 1 -- 172.21.15.12:0/4198939875 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d0ea5c810 con 0x556d0eaa6f30 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff25cff9700 10 client.26443 handle_command_reply: tid=0 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 client.26443 shutdown 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 -- 172.21.15.12:0/4198939875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 msgr2=0x556d0ea97170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea97170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009790 tx=0x7ff26c007c30 comp rx=0 tx=0).stop 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 -- 172.21.15.12:0/4198939875 shutdown_connections 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556d0eaa6f30 0x556d0eaa9350 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea97170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea8d890 0x556d0ea99740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff25cff9700 0 client.26443 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 --2- 172.21.15.12:0/4198939875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea92e00 0x556d0ea96a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 -- 172.21.15.12:0/4198939875 >> 172.21.15.12:0/4198939875 conn(0x556d0e9eb210 msgr2=0x556d0ea875a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 -- 172.21.15.12:0/4198939875 shutdown_connections 2024-05-08T23:50:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 1 -- 172.21.15.12:0/4198939875 wait complete. 2024-05-08T23:50:54.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.028+0000 7ff28754bb80 20 client.26443 trim_cache size 0 max 16384 2024-05-08T23:50:54.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.042+0000 7ff28754bb80 1 Processor -- start 2024-05-08T23:50:54.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.042+0000 7ff28754bb80 1 -- start start 2024-05-08T23:50:54.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.042+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea8d890 0x556d0ea8d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.042+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea87d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.042+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea93720 0x556d0ea95300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.042+0000 7ff28754bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0eaa3a00 con 0x556d0ea92e00 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.042+0000 7ff28754bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0ea84dc0 con 0x556d0ea93720 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.042+0000 7ff28754bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0ea82a90 con 0x556d0ea8d890 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27892c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea87d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27912d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea8d890 0x556d0ea8d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27892c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea87d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58018/0 (socket says 172.21.15.12:58018) 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27892c700 1 -- 172.21.15.12:0/3576091657 learned_addr learned my addr 172.21.15.12:0/3576091657 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27912d700 1 -- 172.21.15.12:0/3576091657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea93720 msgr2=0x556d0ea95300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27912d700 1 --2- 172.21.15.12:0/3576091657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea93720 0x556d0ea95300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27912d700 1 -- 172.21.15.12:0/3576091657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 msgr2=0x556d0ea87d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27912d700 1 --2- 172.21.15.12:0/3576091657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea87d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27912d700 1 -- 172.21.15.12:0/3576091657 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d0e9a3560 con 0x556d0ea8d890 2024-05-08T23:50:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27892c700 1 --2- 172.21.15.12:0/3576091657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea87d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:54.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff27912d700 1 --2- 172.21.15.12:0/3576091657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea8d890 0x556d0ea8d350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff2700a3610 tx=0x7ff27001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca2d96d2c55eb038 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff25ffff700 1 -- 172.21.15.12:0/3576091657 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700aeea0 con 0x556d0ea8d890 2024-05-08T23:50:54.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff25ffff700 1 -- 172.21.15.12:0/3576091657 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2700a3450 con 0x556d0ea8d890 2024-05-08T23:50:54.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.043+0000 7ff25ffff700 1 -- 172.21.15.12:0/3576091657 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27009a7b0 con 0x556d0ea8d890 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.044+0000 7ff28754bb80 1 -- 172.21.15.12:0/3576091657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea8d890 msgr2=0x556d0ea8d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.044+0000 7ff28754bb80 1 --2- 172.21.15.12:0/3576091657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea8d890 0x556d0ea8d350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff2700a3610 tx=0x7ff27001a040 comp rx=0 tx=0).stop 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.044+0000 7ff28754bb80 1 -- 172.21.15.12:0/3576091657 shutdown_connections 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.044+0000 7ff28754bb80 1 --2- 172.21.15.12:0/3576091657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea93720 0x556d0ea95300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.044+0000 7ff28754bb80 1 --2- 172.21.15.12:0/3576091657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea8d890 0x556d0ea8d350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.044+0000 7ff28754bb80 1 --2- 172.21.15.12:0/3576091657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea87d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.044+0000 7ff28754bb80 1 -- 172.21.15.12:0/3576091657 >> 172.21.15.12:0/3576091657 conn(0x556d0e9eb210 msgr2=0x556d0ea90bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.044+0000 7ff28754bb80 1 -- 172.21.15.12:0/3576091657 shutdown_connections 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.044+0000 7ff28754bb80 1 -- 172.21.15.12:0/3576091657 wait complete. 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff28754bb80 1 -- start start 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff28754bb80 1 Processor -- start 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 0x556d0ea9dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea9e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff28754bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea8ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff28754bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0ea8ef40 con 0x556d0ea92e00 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff28754bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0ea8f080 con 0x556d0ea8d890 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff28754bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d0ea8f240 con 0x556d0ea93720 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff28754bb80 10 client.0 _ll_register_callbacks cb 0x556d0e982b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff27892c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea9e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff27992e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea8ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff27992e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea8ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47354/0 (socket says 172.21.15.12:47354) 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff27992e700 1 -- 172.21.15.12:0/756720458 learned_addr learned my addr 172.21.15.12:0/756720458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.045+0000 7ff27912d700 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 0x556d0ea9dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff27892c700 1 -- 172.21.15.12:0/756720458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 msgr2=0x556d0ea8ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff27892c700 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea8ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff27892c700 1 -- 172.21.15.12:0/756720458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 msgr2=0x556d0ea9dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff27892c700 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 0x556d0ea9dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff27892c700 1 -- 172.21.15.12:0/756720458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff264017070 con 0x556d0ea92e00 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff27912d700 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 0x556d0ea9dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff27892c700 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea9e420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x556d0ea97810 tx=0x7ff2640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232b77f2e8f26af8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff25cff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff25cff9700 1 -- 172.21.15.12:0/756720458 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26401e960 con 0x556d0ea92e00 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff25cff9700 1 -- 172.21.15.12:0/756720458 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff26401eac0 con 0x556d0ea92e00 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff28754bb80 1 -- 172.21.15.12:0/756720458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff27009b060 con 0x556d0ea92e00 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff25cff9700 1 -- 172.21.15.12:0/756720458 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26401ec80 con 0x556d0ea92e00 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff28754bb80 10 client.16674 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff28754bb80 1 -- 172.21.15.12:0/756720458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556d0e85e3b0 con 0x556d0ea92e00 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.046+0000 7ff25cff9700 1 -- 172.21.15.12:0/756720458 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27009b060 con 0x556d0ea92e00 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff25cff9700 1 -- 172.21.15.12:0/756720458 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff2640313d0 con 0x556d0ea92e00 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff28754bb80 10 client.16674 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff28754bb80 10 client.16674 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff28754bb80 20 client.16674 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff28754bb80 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556d0eafe4d0 0x556d0eb008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff28754bb80 4 client.16674 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff28754bb80 1 -- 172.21.15.12:0/756720458 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556d0e9d5ec0 con 0x556d0eafe4d0 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff27912d700 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556d0eafe4d0 0x556d0eb008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff25cff9700 10 client.16674 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff27912d700 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556d0eafe4d0 0x556d0eb008f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d63280f9c29d25d server_cookie=10bcbce16b365081 in_seq=0 out_seq=0 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff25cff9700 1 -- 172.21.15.12:0/756720458 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556d0e9d5ec0 con 0x556d0eafe4d0 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff25cff9700 10 client.16674 handle_command_reply: tid=0 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff28754bb80 1 client.16674 shutdown 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff28754bb80 1 -- 172.21.15.12:0/756720458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 msgr2=0x556d0ea9e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.047+0000 7ff28754bb80 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea9e420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x556d0ea97810 tx=0x7ff2640047c0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff28754bb80 1 -- 172.21.15.12:0/756720458 shutdown_connections 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff28754bb80 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556d0eafe4d0 0x556d0eb008f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff28754bb80 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d0ea93720 0x556d0ea8ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff28754bb80 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556d0ea8d890 0x556d0ea9dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff28754bb80 1 --2- 172.21.15.12:0/756720458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0ea92e00 0x556d0ea9e420 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff25cff9700 0 client.16674 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:50:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff28754bb80 1 -- 172.21.15.12:0/756720458 >> 172.21.15.12:0/756720458 conn(0x556d0e9eb210 msgr2=0x556d0ea95290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff28754bb80 1 -- 172.21.15.12:0/756720458 shutdown_connections 2024-05-08T23:50:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff28754bb80 1 -- 172.21.15.12:0/756720458 wait complete. 2024-05-08T23:50:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff28754bb80 20 client.16674 trim_cache size 0 max 16384 2024-05-08T23:50:54.048 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:50:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 -- 172.21.15.12:0/4115159005 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff2600a1290 msgr2=0x7ff2600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff2600a1290 0x7ff2600a3750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff274074980 tx=0x7ff264009260 comp rx=0 tx=0).stop 2024-05-08T23:50:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 -- 172.21.15.12:0/4115159005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff274101160 msgr2=0x7ff2740732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff274101160 0x7ff2740732a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff2700059d0 tx=0x7ff270004730 comp rx=0 tx=0).stop 2024-05-08T23:50:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 -- 172.21.15.12:0/4115159005 shutdown_connections 2024-05-08T23:50:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff2600a1290 0x7ff2600a3750 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff274073d20 0x7ff2741a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff274101160 0x7ff2740732a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 --2- 172.21.15.12:0/4115159005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff274106280 0x7ff2740737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 -- 172.21.15.12:0/4115159005 >> 172.21.15.12:0/4115159005 conn(0x7ff2740fc9e0 msgr2=0x7ff2740fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 -- 172.21.15.12:0/4115159005 shutdown_connections 2024-05-08T23:50:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.048+0000 7ff27b391700 1 -- 172.21.15.12:0/4115159005 wait complete. 2024-05-08T23:50:54.058 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:50:54.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.206+0000 7fc51bed3700 1 -- 172.21.15.12:0/3685509138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc514100060 msgr2=0x7fc514100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.206+0000 7fc51bed3700 1 --2- 172.21.15.12:0/3685509138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc514100060 0x7fc514100440 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc504009990 tx=0x7fc504009ca0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.206+0000 7fc51bed3700 1 -- 172.21.15.12:0/3685509138 shutdown_connections 2024-05-08T23:50:54.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.206+0000 7fc51bed3700 1 --2- 172.21.15.12:0/3685509138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc514105180 0x7fc51410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.206+0000 7fc51bed3700 1 --2- 172.21.15.12:0/3685509138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc514100060 0x7fc514100440 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.206+0000 7fc51bed3700 1 --2- 172.21.15.12:0/3685509138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc514100a10 0x7fc514104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.206+0000 7fc51bed3700 1 -- 172.21.15.12:0/3685509138 >> 172.21.15.12:0/3685509138 conn(0x7fc514076fa0 msgr2=0x7fc5140773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.206+0000 7fc51bed3700 1 -- 172.21.15.12:0/3685509138 shutdown_connections 2024-05-08T23:50:54.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.206+0000 7fc51bed3700 1 -- 172.21.15.12:0/3685509138 wait complete. 2024-05-08T23:50:54.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.207+0000 7fc51bed3700 1 Processor -- start 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.207+0000 7fc51bed3700 1 -- start start 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc51bed3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc514100060 0x7fc514197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc51bed3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc514105180 0x7fc514197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc51bed3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc51419bce0 0x7fc51419c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc51bed3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51419c760 con 0x7fc51419bce0 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc51bed3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51419c8d0 con 0x7fc514100060 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc51bed3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51419ca90 con 0x7fc514105180 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc519c6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc514100060 0x7fc514197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc519c6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc514100060 0x7fc514197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49886/0 (socket says 172.21.15.12:49886) 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc519c6f700 1 -- 172.21.15.12:0/724022717 learned_addr learned my addr 172.21.15.12:0/724022717 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc51a470700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc51419bce0 0x7fc51419c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc51946e700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc514105180 0x7fc514197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc519c6f700 1 -- 172.21.15.12:0/724022717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc514105180 msgr2=0x7fc514197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc519c6f700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc514105180 0x7fc514197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc519c6f700 1 -- 172.21.15.12:0/724022717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc51419bce0 msgr2=0x7fc51419c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc519c6f700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc51419bce0 0x7fc51419c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc519c6f700 1 -- 172.21.15.12:0/724022717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5040095f0 con 0x7fc514100060 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc519c6f700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc514100060 0x7fc514197400 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc504005910 tx=0x7fc504004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.208+0000 7fc50affd700 1 -- 172.21.15.12:0/724022717 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50401d070 con 0x7fc514100060 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.209+0000 7fc51bed3700 1 -- 172.21.15.12:0/724022717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5141a3f90 con 0x7fc514100060 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.209+0000 7fc50affd700 1 -- 172.21.15.12:0/724022717 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc504022470 con 0x7fc514100060 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.209+0000 7fc50affd700 1 -- 172.21.15.12:0/724022717 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504017650 con 0x7fc514100060 2024-05-08T23:50:54.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.209+0000 7fc51bed3700 1 -- 172.21.15.12:0/724022717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5141a43b0 con 0x7fc514100060 2024-05-08T23:50:54.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.211+0000 7fc50affd700 1 -- 172.21.15.12:0/724022717 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc50400fac0 con 0x7fc514100060 2024-05-08T23:50:54.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.212+0000 7fc50affd700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5000a1290 0x7fc5000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.212+0000 7fc50affd700 1 -- 172.21.15.12:0/724022717 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc5040cddd0 con 0x7fc514100060 2024-05-08T23:50:54.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.212+0000 7fc51946e700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5000a1290 0x7fc5000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.213+0000 7fc51946e700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5000a1290 0x7fc5000a3750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc514198950 tx=0x7fc510009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.213+0000 7fc52808db80 1 Processor -- start 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.213+0000 7fc52808db80 1 -- start start 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc52808db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 0x55904116b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc52808db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116bb10 0x559041165760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc52808db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 0x559041166100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc52808db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590410b8150 con 0x55904116bb10 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc52808db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590410b5b10 con 0x55904116b1f0 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc52808db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590410b1400 con 0x559041165ca0 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51946e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116bb10 0x559041165760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51946e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116bb10 0x559041165760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58058/0 (socket says 172.21.15.12:58058) 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51946e700 1 -- 172.21.15.12:0/2979187401 learned_addr learned my addr 172.21.15.12:0/2979187401 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:54.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc519c6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 0x55904116b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51a470700 1 --2- 172.21.15.12:0/2979187401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 0x559041166100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51a470700 1 -- 172.21.15.12:0/2979187401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 msgr2=0x55904116b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51a470700 1 --2- 172.21.15.12:0/2979187401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 0x55904116b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51a470700 1 -- 172.21.15.12:0/2979187401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116bb10 msgr2=0x559041165760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51a470700 1 --2- 172.21.15.12:0/2979187401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116bb10 0x559041165760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51a470700 1 -- 172.21.15.12:0/2979187401 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559040f323b0 con 0x559041165ca0 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc519c6f700 1 --2- 172.21.15.12:0/2979187401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 0x55904116b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51946e700 1 --2- 172.21.15.12:0/2979187401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116bb10 0x559041165760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc51a470700 1 --2- 172.21.15.12:0/2979187401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 0x559041166100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc50c011580 tx=0x7fc50c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa207b816c9acaa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.214+0000 7fc4fffff700 1 -- 172.21.15.12:0/2979187401 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00e040 con 0x559041165ca0 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc4fffff700 1 -- 172.21.15.12:0/2979187401 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc50c00f040 con 0x559041165ca0 2024-05-08T23:50:54.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc4fffff700 1 -- 172.21.15.12:0/2979187401 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0044f0 con 0x559041165ca0 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc52808db80 1 -- 172.21.15.12:0/2979187401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 msgr2=0x559041166100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc52808db80 1 --2- 172.21.15.12:0/2979187401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 0x559041166100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc50c011580 tx=0x7fc50c009920 comp rx=0 tx=0).stop 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc52808db80 1 -- 172.21.15.12:0/2979187401 shutdown_connections 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc52808db80 1 --2- 172.21.15.12:0/2979187401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 0x559041166100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc52808db80 1 --2- 172.21.15.12:0/2979187401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 0x55904116b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc52808db80 1 --2- 172.21.15.12:0/2979187401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116bb10 0x559041165760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc52808db80 1 -- 172.21.15.12:0/2979187401 >> 172.21.15.12:0/2979187401 conn(0x5590410bf210 msgr2=0x55904115bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc52808db80 1 -- 172.21.15.12:0/2979187401 shutdown_connections 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.215+0000 7fc52808db80 1 -- 172.21.15.12:0/2979187401 wait complete. 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.216+0000 7fc52808db80 1 -- start start 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.216+0000 7fc52808db80 1 Processor -- start 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.216+0000 7fc52808db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 0x559041171b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.216+0000 7fc52808db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x55904116ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.216+0000 7fc52808db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116bb10 0x55904116f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.216+0000 7fc52808db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904104d560 con 0x55904116b1f0 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.216+0000 7fc52808db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904104d200 con 0x55904116bb10 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.216+0000 7fc52808db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559040f346a0 con 0x559041165ca0 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.216+0000 7fc52808db80 10 client.0 _ll_register_callbacks cb 0x559041056b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51946e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x55904116ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51946e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x55904116ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58072/0 (socket says 172.21.15.12:58072) 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51946e700 1 -- 172.21.15.12:0/3510147331 learned_addr learned my addr 172.21.15.12:0/3510147331 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc519c6f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 0x559041171b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51a470700 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116bb10 0x55904116f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51946e700 1 -- 172.21.15.12:0/3510147331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 msgr2=0x559041171b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51946e700 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 0x559041171b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51946e700 1 -- 172.21.15.12:0/3510147331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116bb10 msgr2=0x55904116f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51946e700 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116bb10 0x55904116f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51946e700 1 -- 172.21.15.12:0/3510147331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc510017070 con 0x55904116b1f0 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51a470700 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116bb10 0x55904116f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc51946e700 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x55904116ee00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc510004520 tx=0x7fc5100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc61632de9ea976e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc4fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/3510147331 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510022070 con 0x55904116b1f0 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/3510147331 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc510003d70 con 0x55904116b1f0 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.217+0000 7fc52808db80 1 -- 172.21.15.12:0/3510147331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55904100a270 con 0x55904116b1f0 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/3510147331 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc510027420 con 0x55904116b1f0 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc52808db80 10 client.16686 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc52808db80 1 -- 172.21.15.12:0/3510147331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fc50c011230 con 0x55904116b1f0 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/3510147331 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55904100a270 con 0x55904116b1f0 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/3510147331 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc510030480 con 0x55904116b1f0 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc52808db80 10 client.16686 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc52808db80 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:54.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc52808db80 20 client.16686 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc52808db80 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55904117f5b0 0x5590411819d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc52808db80 4 client.16686 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc52808db80 1 -- 172.21.15.12:0/3510147331 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559041130810 con 0x55904117f5b0 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.218+0000 7fc519c6f700 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55904117f5b0 0x5590411819d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.219+0000 7fc4fd7fa700 10 client.16686 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.219+0000 7fc519c6f700 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55904117f5b0 0x5590411819d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6895a052cc4dccd server_cookie=e296cbc129de6359 in_seq=0 out_seq=0 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.219+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/3510147331 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559041130810 con 0x55904117f5b0 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.219+0000 7fc4fd7fa700 10 client.16686 handle_command_reply: tid=0 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 client.16686 shutdown 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 -- 172.21.15.12:0/3510147331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 msgr2=0x55904116ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x55904116ee00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc510004520 tx=0x7fc5100047c0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 -- 172.21.15.12:0/3510147331 shutdown_connections 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55904117f5b0 0x5590411819d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116bb10 0x55904116f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559041165ca0 0x559041171b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 --2- 172.21.15.12:0/3510147331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x55904116ee00 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc4fd7fa700 0 client.16686 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:54.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 -- 172.21.15.12:0/3510147331 >> 172.21.15.12:0/3510147331 conn(0x5590410bf210 msgr2=0x55904115b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 -- 172.21.15.12:0/3510147331 shutdown_connections 2024-05-08T23:50:54.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 1 -- 172.21.15.12:0/3510147331 wait complete. 2024-05-08T23:50:54.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.220+0000 7fc52808db80 20 client.16686 trim_cache size 0 max 16384 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc52808db80 1 Processor -- start 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc52808db80 1 -- start start 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc52808db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559041165ca0 0x55904116edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc52808db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 0x55904115bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc52808db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 0x55904117b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc52808db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904117ccd0 con 0x559041165ca0 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc52808db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559041166660 con 0x55904116b1f0 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc52808db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559041156800 con 0x55904116bb10 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc519c6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559041165ca0 0x55904116edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc51a470700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 0x55904117b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc519c6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559041165ca0 0x55904116edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58084/0 (socket says 172.21.15.12:58084) 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc519c6f700 1 -- 172.21.15.12:0/1557945653 learned_addr learned my addr 172.21.15.12:0/1557945653 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc51946e700 1 --2- 172.21.15.12:0/1557945653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 0x55904115bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc51a470700 1 -- 172.21.15.12:0/1557945653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 msgr2=0x55904115bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc51a470700 1 --2- 172.21.15.12:0/1557945653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 0x55904115bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc51a470700 1 -- 172.21.15.12:0/1557945653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559041165ca0 msgr2=0x55904116edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc51a470700 1 --2- 172.21.15.12:0/1557945653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559041165ca0 0x55904116edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.235+0000 7fc51a470700 1 -- 172.21.15.12:0/1557945653 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559040edeb60 con 0x55904116bb10 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.236+0000 7fc51a470700 1 --2- 172.21.15.12:0/1557945653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 0x55904117b0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc50c000f80 tx=0x7fc50c009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=723c2656cd035792 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.236+0000 7fc508ff9700 1 -- 172.21.15.12:0/1557945653 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00e040 con 0x55904116bb10 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.236+0000 7fc508ff9700 1 -- 172.21.15.12:0/1557945653 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc50c004050 con 0x55904116bb10 2024-05-08T23:50:54.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.236+0000 7fc508ff9700 1 -- 172.21.15.12:0/1557945653 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c007740 con 0x55904116bb10 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.236+0000 7fc52808db80 1 -- 172.21.15.12:0/1557945653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 msgr2=0x55904117b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.236+0000 7fc52808db80 1 --2- 172.21.15.12:0/1557945653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 0x55904117b0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc50c000f80 tx=0x7fc50c009e10 comp rx=0 tx=0).stop 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.237+0000 7fc52808db80 1 -- 172.21.15.12:0/1557945653 shutdown_connections 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.237+0000 7fc52808db80 1 --2- 172.21.15.12:0/1557945653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 0x55904117b0c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.237+0000 7fc52808db80 1 --2- 172.21.15.12:0/1557945653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559041165ca0 0x55904116edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.237+0000 7fc52808db80 1 --2- 172.21.15.12:0/1557945653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55904116b1f0 0x55904115bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.237+0000 7fc52808db80 1 -- 172.21.15.12:0/1557945653 >> 172.21.15.12:0/1557945653 conn(0x5590410bf210 msgr2=0x559041186330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.237+0000 7fc52808db80 1 -- 172.21.15.12:0/1557945653 shutdown_connections 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.237+0000 7fc52808db80 1 -- 172.21.15.12:0/1557945653 wait complete. 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.237+0000 7fc52808db80 1 -- start start 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.237+0000 7fc52808db80 1 Processor -- start 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc52808db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559041165ca0 0x559041176610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc52808db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x559041176b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc52808db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 0x559041166d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc52808db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559041167290 con 0x55904116b1f0 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc52808db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590411673d0 con 0x559041165ca0 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc52808db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559041167590 con 0x55904116bb10 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc52808db80 10 client.0 _ll_register_callbacks cb 0x559041056b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc51946e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x559041176b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc51946e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x559041176b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58098/0 (socket says 172.21.15.12:58098) 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc51946e700 1 -- 172.21.15.12:0/1783700933 learned_addr learned my addr 172.21.15.12:0/1783700933 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc519c6f700 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559041165ca0 0x559041176610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc51a470700 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 0x559041166d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc51946e700 1 -- 172.21.15.12:0/1783700933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 msgr2=0x559041166d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc51946e700 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 0x559041166d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc51946e700 1 -- 172.21.15.12:0/1783700933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559041165ca0 msgr2=0x559041176610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc51946e700 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559041165ca0 0x559041176610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc51946e700 1 -- 172.21.15.12:0/1783700933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559040f323b0 con 0x55904116b1f0 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc51a470700 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 0x559041166d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.238+0000 7fc519c6f700 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559041165ca0 0x559041176610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.239+0000 7fc51946e700 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x559041176b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x559041154180 tx=0x7fc510003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f20e542b4fe713 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.239+0000 7fc4fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.239+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/1783700933 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510022040 con 0x55904116b1f0 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.239+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/1783700933 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc510030ab0 con 0x55904116b1f0 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.239+0000 7fc52808db80 1 -- 172.21.15.12:0/1783700933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc50c00f040 con 0x55904116b1f0 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.239+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/1783700933 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc510004610 con 0x55904116b1f0 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.239+0000 7fc52808db80 10 client.16692 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.239+0000 7fc52808db80 1 -- 172.21.15.12:0/1783700933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559041077560 con 0x55904116b1f0 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.239+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/1783700933 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc510046020 con 0x55904116b1f0 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.239+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/1783700933 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc510027420 con 0x55904116b1f0 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc52808db80 10 client.16692 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc52808db80 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc52808db80 20 client.16692 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:54.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc52808db80 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5590411d6af0 0x5590411d8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc52808db80 4 client.16692 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc52808db80 1 -- 172.21.15.12:0/1783700933 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5590410a9ec0 con 0x5590411d6af0 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc519c6f700 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5590411d6af0 0x5590411d8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc4fd7fa700 10 client.16692 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc519c6f700 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5590411d6af0 0x5590411d8f10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7b731add98e44e server_cookie=68d7d7a7c1c0c0b4 in_seq=0 out_seq=0 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc4fd7fa700 1 -- 172.21.15.12:0/1783700933 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5590410a9ec0 con 0x5590411d6af0 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc4fd7fa700 10 client.16692 handle_command_reply: tid=0 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc52808db80 1 client.16692 shutdown 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc52808db80 1 -- 172.21.15.12:0/1783700933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 msgr2=0x559041176b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.240+0000 7fc52808db80 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x559041176b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x559041154180 tx=0x7fc510003ef0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc52808db80 1 -- 172.21.15.12:0/1783700933 shutdown_connections 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc52808db80 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55904116bb10 0x559041166d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc52808db80 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5590411d6af0 0x5590411d8f10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc52808db80 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559041165ca0 0x559041176610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc52808db80 1 --2- 172.21.15.12:0/1783700933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904116b1f0 0x559041176b50 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc4fd7fa700 0 client.16692 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:54.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc52808db80 1 -- 172.21.15.12:0/1783700933 >> 172.21.15.12:0/1783700933 conn(0x5590410bf210 msgr2=0x559041168f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc52808db80 1 -- 172.21.15.12:0/1783700933 shutdown_connections 2024-05-08T23:50:54.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc52808db80 1 -- 172.21.15.12:0/1783700933 wait complete. 2024-05-08T23:50:54.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc52808db80 20 client.16692 trim_cache size 0 max 16384 2024-05-08T23:50:54.241 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc51bed3700 1 -- 172.21.15.12:0/724022717 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5000a1290 msgr2=0x7fc5000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc51bed3700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5000a1290 0x7fc5000a3750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc514198950 tx=0x7fc510009260 comp rx=0 tx=0).stop 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc51bed3700 1 -- 172.21.15.12:0/724022717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc514100060 msgr2=0x7fc514197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc51bed3700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc514100060 0x7fc514197400 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc504005910 tx=0x7fc504004730 comp rx=0 tx=0).stop 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc51bed3700 1 -- 172.21.15.12:0/724022717 shutdown_connections 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc51bed3700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5000a1290 0x7fc5000a3750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc51bed3700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc51419bce0 0x7fc51419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc51bed3700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc514100060 0x7fc514197400 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc51bed3700 1 --2- 172.21.15.12:0/724022717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc514105180 0x7fc514197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.241+0000 7fc51bed3700 1 -- 172.21.15.12:0/724022717 >> 172.21.15.12:0/724022717 conn(0x7fc514076fa0 msgr2=0x7fc5140fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:54.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.242+0000 7fc51bed3700 1 -- 172.21.15.12:0/724022717 shutdown_connections 2024-05-08T23:50:54.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:54.242+0000 7fc51bed3700 1 -- 172.21.15.12:0/724022717 wait complete. 2024-05-08T23:50:54.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:54 smithi204 ceph-mon[47373]: pgmap v139: 97 pgs: 97 active+clean; 64 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-08T23:50:54.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:54 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1874231505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:50:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:54 smithi160 ceph-mon[46152]: pgmap v139: 97 pgs: 97 active+clean; 64 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-08T23:50:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:54 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1874231505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:50:54.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 199860 11.13 MB/sec execute 50 sec latency 13.613 ms 2024-05-08T23:50:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:54 smithi012 ceph-mon[42503]: pgmap v139: 97 pgs: 97 active+clean; 64 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-08T23:50:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:54 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1874231505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:50:55.250 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:50:55.251 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:50:55.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.427+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/3319625336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ae0106290 msgr2=0x7f9ae010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.427+0000 7f9ae5c1b700 1 --2- 172.21.15.12:0/3319625336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ae0106290 0x7f9ae010de10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800b0d0 tx=0x7f9ad800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.427+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/3319625336 shutdown_connections 2024-05-08T23:50:55.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.427+0000 7f9ae5c1b700 1 --2- 172.21.15.12:0/3319625336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ae0106290 0x7f9ae010de10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.427+0000 7f9ae5c1b700 1 --2- 172.21.15.12:0/3319625336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ae00682e0 0x7f9ae00686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.427+0000 7f9ae5c1b700 1 --2- 172.21.15.12:0/3319625336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ae0068c90 0x7f9ae0105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.427+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/3319625336 >> 172.21.15.12:0/3319625336 conn(0x7f9ae0076f40 msgr2=0x7f9ae0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.427+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/3319625336 shutdown_connections 2024-05-08T23:50:55.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.427+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/3319625336 wait complete. 2024-05-08T23:50:55.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9ae5c1b700 1 Processor -- start 2024-05-08T23:50:55.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9ae5c1b700 1 -- start start 2024-05-08T23:50:55.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9ae5c1b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ae00682e0 0x7f9ae0197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9ae5c1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ae0106290 0x7f9ae0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9ae5c1b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ae019be10 0x7f9ae019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9ae5c1b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae019c890 con 0x7f9ae0106290 2024-05-08T23:50:55.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9ae5c1b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae019ca00 con 0x7f9ae019be10 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9ae5c1b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae019cbc0 con 0x7f9ae00682e0 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9adeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ae0106290 0x7f9ae0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9adffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ae019be10 0x7f9ae019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9adffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ae019be10 0x7f9ae019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49952/0 (socket says 172.21.15.12:49952) 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9adeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ae0106290 0x7f9ae0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58116/0 (socket says 172.21.15.12:58116) 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9adffff700 1 -- 172.21.15.12:0/2809465817 learned_addr learned my addr 172.21.15.12:0/2809465817 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.428+0000 7f9adf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ae00682e0 0x7f9ae0197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9adffff700 1 -- 172.21.15.12:0/2809465817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ae00682e0 msgr2=0x7f9ae0197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9adffff700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ae00682e0 0x7f9ae0197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9adffff700 1 -- 172.21.15.12:0/2809465817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ae0106290 msgr2=0x7f9ae0197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9adffff700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ae0106290 0x7f9ae0197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9adffff700 1 -- 172.21.15.12:0/2809465817 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad8009d00 con 0x7f9ae019be10 2024-05-08T23:50:55.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9adeffd700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ae0106290 0x7f9ae0197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:55.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9adffff700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ae019be10 0x7f9ae019c290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800bcb0 tx=0x7f9ad800bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9adcff9700 1 -- 172.21.15.12:0/2809465817 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad800e040 con 0x7f9ae019be10 2024-05-08T23:50:55.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9adcff9700 1 -- 172.21.15.12:0/2809465817 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ad8005070 con 0x7f9ae019be10 2024-05-08T23:50:55.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/2809465817 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae01a40c0 con 0x7f9ae019be10 2024-05-08T23:50:55.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9adcff9700 1 -- 172.21.15.12:0/2809465817 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad801b6a0 con 0x7f9ae019be10 2024-05-08T23:50:55.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.429+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/2809465817 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae01a4540 con 0x7f9ae019be10 2024-05-08T23:50:55.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.432+0000 7f9adcff9700 1 -- 172.21.15.12:0/2809465817 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9ad8003a00 con 0x7f9ae019be10 2024-05-08T23:50:55.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.433+0000 7f9adcff9700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9acc0a1310 0x7f9acc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.433+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9acc0a1310 0x7f9acc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.433+0000 7f9adcff9700 1 -- 172.21.15.12:0/2809465817 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9ad80cd790 con 0x7f9ae019be10 2024-05-08T23:50:55.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.433+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9acc0a1310 0x7f9acc0a37d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005950 tx=0x7f9ac800b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9af1dd5b80 1 Processor -- start 2024-05-08T23:50:55.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9af1dd5b80 1 -- start start 2024-05-08T23:50:55.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0898e80 0x55cad0899260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad08997a0 0x55cad08933d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0893d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad07ea1d0 con 0x55cad0893910 2024-05-08T23:50:55.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad07e7b90 con 0x55cad08997a0 2024-05-08T23:50:55.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad07e3480 con 0x55cad0898e80 2024-05-08T23:50:55.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9adffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0893d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9adf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0898e80 0x55cad0899260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9adffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0893d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58118/0 (socket says 172.21.15.12:58118) 2024-05-08T23:50:55.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.434+0000 7f9adffff700 1 -- 172.21.15.12:0/2687807647 learned_addr learned my addr 172.21.15.12:0/2687807647 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:55.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.435+0000 7f9adf7fe700 1 -- 172.21.15.12:0/2687807647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad08997a0 msgr2=0x55cad08933d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:50:55.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.435+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/2687807647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad08997a0 0x55cad08933d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.435+0000 7f9adf7fe700 1 -- 172.21.15.12:0/2687807647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 msgr2=0x55cad0893d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.435+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/2687807647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0893d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.435+0000 7f9adf7fe700 1 -- 172.21.15.12:0/2687807647 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cad0664430 con 0x55cad0898e80 2024-05-08T23:50:55.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.435+0000 7f9adffff700 1 --2- 172.21.15.12:0/2687807647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0893d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:55.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.435+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/2687807647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0898e80 0x55cad0899260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005d90 tx=0x7f9ac8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e28428651b4969 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.437+0000 7f9ad5ffb700 1 -- 172.21.15.12:0/2687807647 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8004830 con 0x55cad0898e80 2024-05-08T23:50:55.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.437+0000 7f9ad5ffb700 1 -- 172.21.15.12:0/2687807647 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ac8015070 con 0x55cad0898e80 2024-05-08T23:50:55.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.437+0000 7f9ad5ffb700 1 -- 172.21.15.12:0/2687807647 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac801e680 con 0x55cad0898e80 2024-05-08T23:50:55.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.437+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/2687807647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0898e80 msgr2=0x55cad0899260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.437+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/2687807647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0898e80 0x55cad0899260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005d90 tx=0x7f9ac8013000 comp rx=0 tx=0).stop 2024-05-08T23:50:55.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.437+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/2687807647 shutdown_connections 2024-05-08T23:50:55.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.437+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/2687807647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0893d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.437+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/2687807647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0898e80 0x55cad0899260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.437+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/2687807647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad08997a0 0x55cad08933d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.437+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/2687807647 >> 172.21.15.12:0/2687807647 conn(0x55cad07f1290 msgr2=0x55cad088de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.438+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/2687807647 shutdown_connections 2024-05-08T23:50:55.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.438+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/2687807647 wait complete. 2024-05-08T23:50:55.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.438+0000 7f9af1dd5b80 1 -- start start 2024-05-08T23:50:55.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.438+0000 7f9af1dd5b80 1 Processor -- start 2024-05-08T23:50:55.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.438+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0893910 0x55cad089f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad089caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 0x55cad089d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad077f5e0 con 0x55cad0898e80 2024-05-08T23:50:55.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad077f280 con 0x55cad0893910 2024-05-08T23:50:55.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad0666720 con 0x55cad08997a0 2024-05-08T23:50:55.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9af1dd5b80 10 client.0 _ll_register_callbacks cb 0x55cad0788bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad089caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad089caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58128/0 (socket says 172.21.15.12:58128) 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adeffd700 1 -- 172.21.15.12:0/1237116377 learned_addr learned my addr 172.21.15.12:0/1237116377 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adf7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0893910 0x55cad089f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adffff700 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 0x55cad089d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adeffd700 1 -- 172.21.15.12:0/1237116377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 msgr2=0x55cad089d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adeffd700 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 0x55cad089d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adeffd700 1 -- 172.21.15.12:0/1237116377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0893910 msgr2=0x55cad089f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adeffd700 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0893910 0x55cad089f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adeffd700 1 -- 172.21.15.12:0/1237116377 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad80b8e70 con 0x55cad0898e80 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0893910 0x55cad089f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9adeffd700 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad089caf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0011a00 tx=0x7f9ad0011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef21439425360c34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9abeffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:55.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.439+0000 7f9abeffd700 1 -- 172.21.15.12:0/1237116377 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0011ea0 con 0x55cad0898e80 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9abeffd700 1 -- 172.21.15.12:0/1237116377 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9ad000fca0 con 0x55cad0898e80 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/1237116377 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cad073c2f0 con 0x55cad0898e80 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9abeffd700 1 -- 172.21.15.12:0/1237116377 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ad000fe60 con 0x55cad0898e80 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9af1dd5b80 10 client.16704 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/1237116377 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f9ac8010c50 con 0x55cad0898e80 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9abeffd700 1 -- 172.21.15.12:0/1237116377 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cad073c2f0 con 0x55cad0898e80 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9abeffd700 1 -- 172.21.15.12:0/1237116377 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9ad0019420 con 0x55cad0898e80 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9af1dd5b80 10 client.16704 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9af1dd5b80 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9af1dd5b80 20 client.16704 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cad08abcd0 0x55cad08ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9af1dd5b80 4 client.16704 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:55.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/1237116377 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cad0862890 con 0x55cad08abcd0 2024-05-08T23:50:55.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.440+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cad08abcd0 0x55cad08ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.441+0000 7f9abeffd700 10 client.16704 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:55.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.441+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cad08abcd0 0x55cad08ac0e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a413bdf788a93e server_cookie=5d9a5b010b31798c in_seq=0 out_seq=0 2024-05-08T23:50:55.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9abeffd700 1 -- 172.21.15.12:0/1237116377 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cad0862890 con 0x55cad08abcd0 2024-05-08T23:50:55.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9abeffd700 10 client.16704 handle_command_reply: tid=0 2024-05-08T23:50:55.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9af1dd5b80 1 client.16704 shutdown 2024-05-08T23:50:55.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/1237116377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 msgr2=0x55cad089caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad089caf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0011a00 tx=0x7f9ad0011d10 comp rx=0 tx=0).stop 2024-05-08T23:50:55.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/1237116377 shutdown_connections 2024-05-08T23:50:55.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 0x55cad089d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cad08abcd0 0x55cad08ac0e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0893910 0x55cad089f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/1237116377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad089caf0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9abeffd700 0 client.16704 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:55.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.442+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/1237116377 >> 172.21.15.12:0/1237116377 conn(0x55cad07f1290 msgr2=0x55cad088d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.443+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/1237116377 shutdown_connections 2024-05-08T23:50:55.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.443+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/1237116377 wait complete. 2024-05-08T23:50:55.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.443+0000 7f9af1dd5b80 20 client.16704 trim_cache size 0 max 16384 2024-05-08T23:50:55.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9af1dd5b80 1 Processor -- start 2024-05-08T23:50:55.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9af1dd5b80 1 -- start start 2024-05-08T23:50:55.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0898350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0898e80 0x55cad088dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 0x55cad08bd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad08ae190 con 0x55cad0893910 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad088e960 con 0x55cad0898e80 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad088ae40 con 0x55cad08997a0 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0898350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0898350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58144/0 (socket says 172.21.15.12:58144) 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adf7fe700 1 -- 172.21.15.12:0/2318818424 learned_addr learned my addr 172.21.15.12:0/2318818424 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adeffd700 1 --2- 172.21.15.12:0/2318818424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0898e80 0x55cad088dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adffff700 1 --2- 172.21.15.12:0/2318818424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 0x55cad08bd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adf7fe700 1 -- 172.21.15.12:0/2318818424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 msgr2=0x55cad08bd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/2318818424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 0x55cad08bd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adf7fe700 1 -- 172.21.15.12:0/2318818424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0898e80 msgr2=0x55cad088dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/2318818424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0898e80 0x55cad088dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adf7fe700 1 -- 172.21.15.12:0/2318818424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cad088a560 con 0x55cad0893910 2024-05-08T23:50:55.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.462+0000 7f9adffff700 1 --2- 172.21.15.12:0/2318818424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 0x55cad08bd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:50:55.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.463+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/2318818424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0898350 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005500 tx=0x7f9ac800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c738d6e3ac67e51a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.463+0000 7f9ad67fc700 1 -- 172.21.15.12:0/2318818424 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8005660 con 0x55cad0893910 2024-05-08T23:50:55.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.463+0000 7f9ad67fc700 1 -- 172.21.15.12:0/2318818424 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ac8035530 con 0x55cad0893910 2024-05-08T23:50:55.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.463+0000 7f9ad67fc700 1 -- 172.21.15.12:0/2318818424 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8026ae0 con 0x55cad0893910 2024-05-08T23:50:55.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/2318818424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 msgr2=0x55cad0898350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/2318818424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0898350 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005500 tx=0x7f9ac800b4a0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/2318818424 shutdown_connections 2024-05-08T23:50:55.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/2318818424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad08997a0 0x55cad08bd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/2318818424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0893910 0x55cad0898350 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/2318818424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad0898e80 0x55cad088dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/2318818424 >> 172.21.15.12:0/2318818424 conn(0x55cad07f1290 msgr2=0x55cad0897c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/2318818424 shutdown_connections 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/2318818424 wait complete. 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 -- start start 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.464+0000 7f9af1dd5b80 1 Processor -- start 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0893910 0x55cad08bd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad08b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9af1dd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad08997a0 0x55cad08b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad08b9fc0 con 0x55cad0898e80 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad08ba130 con 0x55cad08997a0 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9af1dd5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad08ba2f0 con 0x55cad0893910 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9af1dd5b80 10 client.0 _ll_register_callbacks cb 0x55cad0788bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad08b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0893910 0x55cad08bd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad08b9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58152/0 (socket says 172.21.15.12:58152) 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adeffd700 1 -- 172.21.15.12:0/465184387 learned_addr learned my addr 172.21.15.12:0/465184387 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adffff700 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad08997a0 0x55cad08b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adeffd700 1 -- 172.21.15.12:0/465184387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0893910 msgr2=0x55cad08bd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adeffd700 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0893910 0x55cad08bd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adeffd700 1 -- 172.21.15.12:0/465184387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad08997a0 msgr2=0x55cad08b9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adeffd700 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad08997a0 0x55cad08b9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adeffd700 1 -- 172.21.15.12:0/465184387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cad0664430 con 0x55cad0898e80 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adffff700 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad08997a0 0x55cad08b9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0893910 0x55cad08bd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9adeffd700 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad08b9540 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00119d0 tx=0x7f9ad0019bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f430a8637c5fb3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.465+0000 7f9abeffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9abeffd700 1 -- 172.21.15.12:0/465184387 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000fee0 con 0x55cad0898e80 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9abeffd700 1 -- 172.21.15.12:0/465184387 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9ad00052b0 con 0x55cad0898e80 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/465184387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ac8015070 con 0x55cad0898e80 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9abeffd700 1 -- 172.21.15.12:0/465184387 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ad0005470 con 0x55cad0898e80 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9af1dd5b80 10 client.16710 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/465184387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55cad0610be0 con 0x55cad0898e80 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9abeffd700 1 -- 172.21.15.12:0/465184387 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ac8015070 con 0x55cad0898e80 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9abeffd700 1 -- 172.21.15.12:0/465184387 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9ad002b070 con 0x55cad0898e80 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9af1dd5b80 10 client.16710 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9af1dd5b80 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9af1dd5b80 20 client.16710 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cad090d260 0x55cad090f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9af1dd5b80 4 client.16710 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/465184387 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cad07dbf40 con 0x55cad090d260 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.466+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cad090d260 0x55cad090f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.467+0000 7f9abeffd700 10 client.16710 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.467+0000 7f9adf7fe700 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cad090d260 0x55cad090f680 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a2c92cfc003f129 server_cookie=f3381547811d88c9 in_seq=0 out_seq=0 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9abeffd700 1 -- 172.21.15.12:0/465184387 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cad07dbf40 con 0x55cad090d260 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9abeffd700 10 client.16710 handle_command_reply: tid=0 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9af1dd5b80 1 client.16710 shutdown 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/465184387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 msgr2=0x55cad08b9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad08b9540 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00119d0 tx=0x7f9ad0019bd0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/465184387 shutdown_connections 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cad090d260 0x55cad090f680 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cad08997a0 0x55cad08b9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cad0893910 0x55cad08bd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9abeffd700 0 client.16710 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9af1dd5b80 1 --2- 172.21.15.12:0/465184387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cad0898e80 0x55cad08b9540 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.468+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/465184387 >> 172.21.15.12:0/465184387 conn(0x55cad07f1290 msgr2=0x55cad08984d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/465184387 shutdown_connections 2024-05-08T23:50:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9af1dd5b80 1 -- 172.21.15.12:0/465184387 wait complete. 2024-05-08T23:50:55.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9af1dd5b80 20 client.16710 trim_cache size 0 max 16384 2024-05-08T23:50:55.469 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:50:55.469 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:50:55.469 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "baf4bc94-b53c-46b7-a0d8-670627781d49", 2024-05-08T23:50:55.469 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:50:55.469 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/2809465817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9acc0a1310 msgr2=0x7f9acc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9acc0a1310 0x7f9acc0a37d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005950 tx=0x7f9ac800b410 comp rx=0 tx=0).stop 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/2809465817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ae019be10 msgr2=0x7f9ae019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ae019be10 0x7f9ae019c290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800bcb0 tx=0x7f9ad800bce0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/2809465817 shutdown_connections 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9acc0a1310 0x7f9acc0a37d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ae019be10 0x7f9ae019c290 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ae00682e0 0x7f9ae0197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 --2- 172.21.15.12:0/2809465817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ae0106290 0x7f9ae0197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/2809465817 >> 172.21.15.12:0/2809465817 conn(0x7f9ae0076f40 msgr2=0x7f9ae00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/2809465817 shutdown_connections 2024-05-08T23:50:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.469+0000 7f9ae5c1b700 1 -- 172.21.15.12:0/2809465817 wait complete. 2024-05-08T23:50:55.480 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:50:55.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.633+0000 7f62993b7700 1 -- 172.21.15.12:0/2532293753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6294106280 msgr2=0x7f629410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.633+0000 7f62993b7700 1 --2- 172.21.15.12:0/2532293753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6294106280 0x7f629410de00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f628c009f90 tx=0x7f628c00b330 comp rx=0 tx=0).stop 2024-05-08T23:50:55.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.633+0000 7f62993b7700 1 -- 172.21.15.12:0/2532293753 shutdown_connections 2024-05-08T23:50:55.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.633+0000 7f62993b7700 1 --2- 172.21.15.12:0/2532293753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6294106280 0x7f629410de00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.633+0000 7f62993b7700 1 --2- 172.21.15.12:0/2532293753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294069770 0x7f6294069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.633+0000 7f62993b7700 1 --2- 172.21.15.12:0/2532293753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f629406a120 0x7f6294105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.633+0000 7f62993b7700 1 -- 172.21.15.12:0/2532293753 >> 172.21.15.12:0/2532293753 conn(0x7f6294076fa0 msgr2=0x7f62940773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.634+0000 7f62993b7700 1 -- 172.21.15.12:0/2532293753 shutdown_connections 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.634+0000 7f62993b7700 1 -- 172.21.15.12:0/2532293753 wait complete. 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.634+0000 7f62993b7700 1 Processor -- start 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.634+0000 7f62993b7700 1 -- start start 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.634+0000 7f62993b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6294069770 0x7f6294197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.634+0000 7f62993b7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294106280 0x7f6294197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f62993b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f629419be10 0x7f629419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f62993b7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629419c890 con 0x7f6294106280 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f62993b7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629419ca00 con 0x7f629419be10 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f62993b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629419cbc0 con 0x7f6294069770 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f628bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294106280 0x7f6294197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f6292ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6294069770 0x7f6294197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f628bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294106280 0x7f6294197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58172/0 (socket says 172.21.15.12:58172) 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f628bfff700 1 -- 172.21.15.12:0/3900638512 learned_addr learned my addr 172.21.15.12:0/3900638512 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:55.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f62937fe700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f629419be10 0x7f629419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f628bfff700 1 -- 172.21.15.12:0/3900638512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6294069770 msgr2=0x7f6294197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f628bfff700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6294069770 0x7f6294197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f628bfff700 1 -- 172.21.15.12:0/3900638512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f629419be10 msgr2=0x7f629419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f628bfff700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f629419be10 0x7f629419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f628bfff700 1 -- 172.21.15.12:0/3900638512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c009c40 con 0x7f6294106280 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f628bfff700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294106280 0x7f6294197a70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f627c00e970 tx=0x7f627c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f6290ff9700 1 -- 172.21.15.12:0/3900638512 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c009790 con 0x7f6294106280 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f62993b7700 1 -- 172.21.15.12:0/3900638512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62941a4120 con 0x7f6294106280 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f6290ff9700 1 -- 172.21.15.12:0/3900638512 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f627c00c7c0 con 0x7f6294106280 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.635+0000 7f6290ff9700 1 -- 172.21.15.12:0/3900638512 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c005460 con 0x7f6294106280 2024-05-08T23:50:55.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.636+0000 7f62993b7700 1 -- 172.21.15.12:0/3900638512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62941a4650 con 0x7f6294106280 2024-05-08T23:50:55.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.638+0000 7f6290ff9700 1 -- 172.21.15.12:0/3900638512 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f627c005600 con 0x7f6294106280 2024-05-08T23:50:55.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.639+0000 7f6290ff9700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62800a1290 0x7f62800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.639+0000 7f6290ff9700 1 -- 172.21.15.12:0/3900638512 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f627c014070 con 0x7f6294106280 2024-05-08T23:50:55.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.639+0000 7f6292ffd700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62800a1290 0x7f62800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.639+0000 7f6292ffd700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62800a1290 0x7f62800a3750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6284005fd0 tx=0x7f6284009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f62a5571b80 1 Processor -- start 2024-05-08T23:50:55.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f62a5571b80 1 -- start start 2024-05-08T23:50:55.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d5175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d511760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 0x556b6d512100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f62a5571b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d464150 con 0x556b6d511ca0 2024-05-08T23:50:55.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f62a5571b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d461b10 con 0x556b6d517b10 2024-05-08T23:50:55.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f62a5571b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d45d400 con 0x556b6d5171f0 2024-05-08T23:50:55.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f6292ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d5175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f6292ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d5175d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47498/0 (socket says 172.21.15.12:47498) 2024-05-08T23:50:55.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f62937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 0x556b6d512100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f6292ffd700 1 -- 172.21.15.12:0/3763233676 learned_addr learned my addr 172.21.15.12:0/3763233676 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.640+0000 7f628bfff700 1 --2- 172.21.15.12:0/3763233676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d511760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.641+0000 7f6292ffd700 1 -- 172.21.15.12:0/3763233676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 msgr2=0x556b6d511760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.641+0000 7f6292ffd700 1 --2- 172.21.15.12:0/3763233676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d511760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.641+0000 7f6292ffd700 1 -- 172.21.15.12:0/3763233676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 msgr2=0x556b6d512100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.641+0000 7f6292ffd700 1 --2- 172.21.15.12:0/3763233676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 0x556b6d512100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.641+0000 7f6292ffd700 1 -- 172.21.15.12:0/3763233676 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b6d2de3b0 con 0x556b6d5171f0 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.641+0000 7f6292ffd700 1 --2- 172.21.15.12:0/3763233676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d5175d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f62840041d0 tx=0x7f6284014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c693b7f97ddb36cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.641+0000 7f62897fa700 1 -- 172.21.15.12:0/3763233676 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284009e60 con 0x556b6d5171f0 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.641+0000 7f62897fa700 1 -- 172.21.15.12:0/3763233676 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6284004830 con 0x556b6d5171f0 2024-05-08T23:50:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.641+0000 7f62897fa700 1 -- 172.21.15.12:0/3763233676 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284005560 con 0x556b6d5171f0 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.642+0000 7f62a5571b80 1 -- 172.21.15.12:0/3763233676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 msgr2=0x556b6d5175d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.642+0000 7f62a5571b80 1 --2- 172.21.15.12:0/3763233676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d5175d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f62840041d0 tx=0x7f6284014000 comp rx=0 tx=0).stop 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.642+0000 7f62a5571b80 1 -- 172.21.15.12:0/3763233676 shutdown_connections 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.642+0000 7f62a5571b80 1 --2- 172.21.15.12:0/3763233676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 0x556b6d512100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.642+0000 7f62a5571b80 1 --2- 172.21.15.12:0/3763233676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d5175d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.642+0000 7f62a5571b80 1 --2- 172.21.15.12:0/3763233676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d511760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.642+0000 7f62a5571b80 1 -- 172.21.15.12:0/3763233676 >> 172.21.15.12:0/3763233676 conn(0x556b6d46b210 msgr2=0x556b6d507df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.642+0000 7f62a5571b80 1 -- 172.21.15.12:0/3763233676 shutdown_connections 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.642+0000 7f62a5571b80 1 -- 172.21.15.12:0/3763233676 wait complete. 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f62a5571b80 1 -- start start 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f62a5571b80 1 Processor -- start 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d511ca0 0x556b6d51dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d5171f0 0x556b6d51ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d51b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f62a5571b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d3f9560 con 0x556b6d5171f0 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f62a5571b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d3f9200 con 0x556b6d517b10 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f62a5571b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d2e06a0 con 0x556b6d511ca0 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f62a5571b80 10 client.0 _ll_register_callbacks cb 0x556b6d402b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f628bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d5171f0 0x556b6d51ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f6292ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d511ca0 0x556b6d51dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f628bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d5171f0 0x556b6d51ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58194/0 (socket says 172.21.15.12:58194) 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f628bfff700 1 -- 172.21.15.12:0/4160794113 learned_addr learned my addr 172.21.15.12:0/4160794113 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.643+0000 7f62937fe700 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d51b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f6292ffd700 1 -- 172.21.15.12:0/4160794113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 msgr2=0x556b6d51b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f6292ffd700 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d51b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f6292ffd700 1 -- 172.21.15.12:0/4160794113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d5171f0 msgr2=0x556b6d51ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f6292ffd700 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d5171f0 0x556b6d51ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f6292ffd700 1 -- 172.21.15.12:0/4160794113 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6284017070 con 0x556b6d511ca0 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f62937fe700 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d51b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f628bfff700 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d5171f0 0x556b6d51ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f6292ffd700 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d511ca0 0x556b6d51dbf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6284003960 tx=0x7f6284003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=547f9467b5d37b56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f627a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f627a7fc700 1 -- 172.21.15.12:0/4160794113 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628402c070 con 0x556b6d511ca0 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f627a7fc700 1 -- 172.21.15.12:0/4160794113 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6284005410 con 0x556b6d511ca0 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f62a5571b80 1 -- 172.21.15.12:0/4160794113 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b6d3b6270 con 0x556b6d511ca0 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f627a7fc700 1 -- 172.21.15.12:0/4160794113 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6284003e30 con 0x556b6d511ca0 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f62a5571b80 10 client.26497 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.644+0000 7f62a5571b80 1 -- 172.21.15.12:0/4160794113 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556b6d28ab60 con 0x556b6d511ca0 2024-05-08T23:50:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f627a7fc700 1 -- 172.21.15.12:0/4160794113 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b6d3b6270 con 0x556b6d511ca0 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f627a7fc700 1 -- 172.21.15.12:0/4160794113 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6284024030 con 0x556b6d511ca0 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f62a5571b80 10 client.26497 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f62a5571b80 10 client.26497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f62a5571b80 20 client.26497 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556b6d52b440 0x556b6d52d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f62a5571b80 4 client.26497 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f62a5571b80 1 -- 172.21.15.12:0/4160794113 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b6d4dc810 con 0x556b6d52b440 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f628bfff700 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556b6d52b440 0x556b6d52d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f628bfff700 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556b6d52b440 0x556b6d52d830 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a193c95354dc934 server_cookie=eaabdcd747b39dff in_seq=0 out_seq=0 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.645+0000 7f627a7fc700 10 client.26497 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f627a7fc700 1 -- 172.21.15.12:0/4160794113 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b6d4dc810 con 0x556b6d52b440 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f627a7fc700 10 client.26497 handle_command_reply: tid=0 2024-05-08T23:50:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 client.26497 shutdown 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 -- 172.21.15.12:0/4160794113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d511ca0 msgr2=0x556b6d51dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d511ca0 0x556b6d51dbf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6284003960 tx=0x7f6284003d40 comp rx=0 tx=0).stop 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 -- 172.21.15.12:0/4160794113 shutdown_connections 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556b6d52b440 0x556b6d52d830 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d51b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d511ca0 0x556b6d51dbf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4160794113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d5171f0 0x556b6d51ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f627a7fc700 0 client.26497 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 -- 172.21.15.12:0/4160794113 >> 172.21.15.12:0/4160794113 conn(0x556b6d46b210 msgr2=0x556b6d5075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 -- 172.21.15.12:0/4160794113 shutdown_connections 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.646+0000 7f62a5571b80 1 -- 172.21.15.12:0/4160794113 wait complete. 2024-05-08T23:50:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.647+0000 7f62a5571b80 20 client.26497 trim_cache size 0 max 16384 2024-05-08T23:50:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.667+0000 7f62a5571b80 1 Processor -- start 2024-05-08T23:50:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.667+0000 7f62a5571b80 1 -- start start 2024-05-08T23:50:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.667+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d511ca0 0x556b6d51b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.667+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d51ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.667+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d517b10 0x556b6d507d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.667+0000 7f62a5571b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d517910 con 0x556b6d517b10 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.667+0000 7f62a5571b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d5193e0 con 0x556b6d511ca0 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.667+0000 7f62a5571b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d500040 con 0x556b6d5171f0 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6292ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d511ca0 0x556b6d51b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f62937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d517b10 0x556b6d507d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6292ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d511ca0 0x556b6d51b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50028/0 (socket says 172.21.15.12:50028) 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f62937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d517b10 0x556b6d507d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58198/0 (socket says 172.21.15.12:58198) 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6292ffd700 1 -- 172.21.15.12:0/2686238626 learned_addr learned my addr 172.21.15.12:0/2686238626 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f628bfff700 1 --2- 172.21.15.12:0/2686238626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d51ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6292ffd700 1 -- 172.21.15.12:0/2686238626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 msgr2=0x556b6d51ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6292ffd700 1 --2- 172.21.15.12:0/2686238626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d51ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6292ffd700 1 -- 172.21.15.12:0/2686238626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d517b10 msgr2=0x556b6d507d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6292ffd700 1 --2- 172.21.15.12:0/2686238626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d517b10 0x556b6d507d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6292ffd700 1 -- 172.21.15.12:0/2686238626 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b6d423560 con 0x556b6d511ca0 2024-05-08T23:50:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6292ffd700 1 --2- 172.21.15.12:0/2686238626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d511ca0 0x556b6d51b5b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6284011ab0 tx=0x7f62840058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d0aa5ad37dcc0a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6289ffb700 1 -- 172.21.15.12:0/2686238626 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628402c070 con 0x556b6d511ca0 2024-05-08T23:50:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6289ffb700 1 -- 172.21.15.12:0/2686238626 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6284003eb0 con 0x556b6d511ca0 2024-05-08T23:50:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.668+0000 7f6289ffb700 1 -- 172.21.15.12:0/2686238626 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62840049a0 con 0x556b6d511ca0 2024-05-08T23:50:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.669+0000 7f62a5571b80 1 -- 172.21.15.12:0/2686238626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d511ca0 msgr2=0x556b6d51b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.669+0000 7f62a5571b80 1 --2- 172.21.15.12:0/2686238626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d511ca0 0x556b6d51b5b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6284011ab0 tx=0x7f62840058e0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.669+0000 7f62a5571b80 1 -- 172.21.15.12:0/2686238626 shutdown_connections 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.669+0000 7f62a5571b80 1 --2- 172.21.15.12:0/2686238626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d517b10 0x556b6d507d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.669+0000 7f62a5571b80 1 --2- 172.21.15.12:0/2686238626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d511ca0 0x556b6d51b5b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.669+0000 7f62a5571b80 1 --2- 172.21.15.12:0/2686238626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d51ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.669+0000 7f62a5571b80 1 -- 172.21.15.12:0/2686238626 >> 172.21.15.12:0/2686238626 conn(0x556b6d46b210 msgr2=0x556b6d51a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 1 -- 172.21.15.12:0/2686238626 shutdown_connections 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 1 -- 172.21.15.12:0/2686238626 wait complete. 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 1 -- start start 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 1 Processor -- start 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 0x556b6d520790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d520cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d580900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d580e40 con 0x556b6d511ca0 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d580f80 con 0x556b6d517b10 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b6d5810c0 con 0x556b6d5171f0 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62a5571b80 10 client.0 _ll_register_callbacks cb 0x556b6d402b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f6292ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 0x556b6d520790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f62937fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d580900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f6292ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 0x556b6d520790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58210/0 (socket says 172.21.15.12:58210) 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.670+0000 7f6292ffd700 1 -- 172.21.15.12:0/4204432452 learned_addr learned my addr 172.21.15.12:0/4204432452 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f628bfff700 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d520cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f6292ffd700 1 -- 172.21.15.12:0/4204432452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 msgr2=0x556b6d520cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f6292ffd700 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d520cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f6292ffd700 1 -- 172.21.15.12:0/4204432452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 msgr2=0x556b6d580900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f6292ffd700 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d580900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f6292ffd700 1 -- 172.21.15.12:0/4204432452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b6d28ab60 con 0x556b6d511ca0 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f62937fe700 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d580900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:55.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f628bfff700 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d520cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f6292ffd700 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 0x556b6d520790 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6284009f50 tx=0x7f62840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5860eb21a457bf15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f627a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f627a7fc700 1 -- 172.21.15.12:0/4204432452 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628402c040 con 0x556b6d511ca0 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f627a7fc700 1 -- 172.21.15.12:0/4204432452 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6284031cf0 con 0x556b6d511ca0 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.671+0000 7f62a5571b80 1 -- 172.21.15.12:0/4204432452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6284004830 con 0x556b6d511ca0 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f627a7fc700 1 -- 172.21.15.12:0/4204432452 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6284005090 con 0x556b6d511ca0 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f62a5571b80 10 client.16728 fetch_fsmap learned FSMap version 31 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f62a5571b80 1 -- 172.21.15.12:0/4204432452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556b6d2e2cd0 con 0x556b6d511ca0 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f627a7fc700 1 -- 172.21.15.12:0/4204432452 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6284004830 con 0x556b6d511ca0 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f627a7fc700 1 -- 172.21.15.12:0/4204432452 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6284003680 con 0x556b6d511ca0 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f62a5571b80 10 client.16728 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f62a5571b80 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f62a5571b80 20 client.16728 populate_metadata read hostname 'smithi012' 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556b6d5823c0 0x556b6d5847b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:50:55.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f62a5571b80 4 client.16728 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f62a5571b80 1 -- 172.21.15.12:0/4204432452 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556b6d455ec0 con 0x556b6d5823c0 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.672+0000 7f628bfff700 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556b6d5823c0 0x556b6d5847b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f627a7fc700 10 client.16728 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f628bfff700 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556b6d5823c0 0x556b6d5847b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf3b1771c444314 server_cookie=a7038124921bba8e in_seq=0 out_seq=0 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f627a7fc700 1 -- 172.21.15.12:0/4204432452 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556b6d455ec0 con 0x556b6d5823c0 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f627a7fc700 10 client.16728 handle_command_reply: tid=0 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 client.16728 shutdown 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 -- 172.21.15.12:0/4204432452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 msgr2=0x556b6d520790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 0x556b6d520790 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6284009f50 tx=0x7f62840057f0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 -- 172.21.15.12:0/4204432452 shutdown_connections 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556b6d5823c0 0x556b6d5847b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b6d517b10 0x556b6d580900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b6d511ca0 0x556b6d520790 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 --2- 172.21.15.12:0/4204432452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556b6d5171f0 0x556b6d520cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 -- 172.21.15.12:0/4204432452 >> 172.21.15.12:0/4204432452 conn(0x556b6d46b210 msgr2=0x556b6d50c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f627a7fc700 0 client.16728 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:50:55.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 -- 172.21.15.12:0/4204432452 shutdown_connections 2024-05-08T23:50:55.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 1 -- 172.21.15.12:0/4204432452 wait complete. 2024-05-08T23:50:55.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.673+0000 7f62a5571b80 20 client.16728 trim_cache size 0 max 16384 2024-05-08T23:50:55.674 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:50:55.674 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:50:55.674 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:50:55.675 INFO:teuthology.orchestra.run.smithi012.stdout: "baf4bc94-b53c-46b7-a0d8-670627781d49": { 2024-05-08T23:50:55.675 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:50:55.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "baf4bc94-b53c-46b7-a0d8-670627781d49", 2024-05-08T23:50:55.675 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:50:55.675 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:50:55.675 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:50:55.675 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:50:55.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 -- 172.21.15.12:0/3900638512 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62800a1290 msgr2=0x7f62800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62800a1290 0x7f62800a3750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6284005fd0 tx=0x7f6284009310 comp rx=0 tx=0).stop 2024-05-08T23:50:55.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 -- 172.21.15.12:0/3900638512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294106280 msgr2=0x7f6294197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:50:55.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294106280 0x7f6294197a70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f627c00e970 tx=0x7f627c00ec80 comp rx=0 tx=0).stop 2024-05-08T23:50:55.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 -- 172.21.15.12:0/3900638512 shutdown_connections 2024-05-08T23:50:55.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62800a1290 0x7f62800a3750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f629419be10 0x7f629419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6294069770 0x7f6294197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 --2- 172.21.15.12:0/3900638512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294106280 0x7f6294197a70 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:50:55.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 -- 172.21.15.12:0/3900638512 >> 172.21.15.12:0/3900638512 conn(0x7f6294076fa0 msgr2=0x7f62940fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:50:55.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 -- 172.21.15.12:0/3900638512 shutdown_connections 2024-05-08T23:50:55.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:50:55.674+0000 7f62993b7700 1 -- 172.21.15.12:0/3900638512 wait complete. 2024-05-08T23:50:55.684 INFO:tasks.cephfs.filesystem:scrub status for tag:baf4bc94-b53c-46b7-a0d8-670627781d49 - {'path': '/', 'tag': 'baf4bc94-b53c-46b7-a0d8-670627781d49', 'options': 'recursive,force'} 2024-05-08T23:50:55.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 202399 11.24 MB/sec execute 51 sec latency 32.194 ms 2024-05-08T23:50:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:56 smithi204 ceph-mon[47373]: pgmap v140: 97 pgs: 97 active+clean; 64 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-08T23:50:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:56 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:50:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:56 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:56 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:50:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:56 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:50:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:56 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:50:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:56 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:56 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:56 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:56 smithi160 ceph-mon[46152]: pgmap v140: 97 pgs: 97 active+clean; 64 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-08T23:50:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:56 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:50:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:56 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:56 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:50:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:56 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:50:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:56 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:50:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:56 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:56 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:56 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 204277 11.14 MB/sec execute 52 sec latency 10.869 ms 2024-05-08T23:50:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:56 smithi012 ceph-mon[42503]: pgmap v140: 97 pgs: 97 active+clean; 64 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-08T23:50:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:56 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:50:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:56 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:56 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:50:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:56 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:50:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:56 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:50:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:56 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:56 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:56 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:50:57.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 206134 11.18 MB/sec execute 53 sec latency 24.058 ms 2024-05-08T23:50:58.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:50:58 smithi204 ceph-mon[47373]: pgmap v141: 97 pgs: 97 active+clean; 68 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 435 op/s 2024-05-08T23:50:58.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:50:58 smithi160 ceph-mon[46152]: pgmap v141: 97 pgs: 97 active+clean; 68 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 435 op/s 2024-05-08T23:50:58.861 INFO:tasks.workunit.client.0.smithi012.stdout: 1 208551 11.18 MB/sec execute 54 sec latency 13.672 ms 2024-05-08T23:50:58.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:50:58 smithi012 ceph-mon[42503]: pgmap v141: 97 pgs: 97 active+clean; 68 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 435 op/s 2024-05-08T23:50:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 210059 11.13 MB/sec execute 55 sec latency 10.154 ms 2024-05-08T23:51:00.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:00 smithi204 ceph-mon[47373]: pgmap v142: 97 pgs: 97 active+clean; 68 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-05-08T23:51:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:00 smithi160 ceph-mon[46152]: pgmap v142: 97 pgs: 97 active+clean; 68 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-05-08T23:51:00.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 212484 11.16 MB/sec execute 56 sec latency 58.246 ms 2024-05-08T23:51:00.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:00 smithi012 ceph-mon[42503]: pgmap v142: 97 pgs: 97 active+clean; 68 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-05-08T23:51:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.757+0000 7f5af6c28700 1 -- 172.21.15.12:0/35155091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af00682e0 msgr2=0x7f5af0068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.757+0000 7f5af6c28700 1 --2- 172.21.15.12:0/35155091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af00682e0 0x7f5af0068760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0098f0 tx=0x7f5adc009c00 comp rx=0 tx=0).stop 2024-05-08T23:51:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.757+0000 7f5af6c28700 1 -- 172.21.15.12:0/35155091 shutdown_connections 2024-05-08T23:51:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.757+0000 7f5af6c28700 1 --2- 172.21.15.12:0/35155091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0068ca0 0x7f5af010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.757+0000 7f5af6c28700 1 --2- 172.21.15.12:0/35155091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af0107000 0x7f5af01073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.757+0000 7f5af6c28700 1 --2- 172.21.15.12:0/35155091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af00682e0 0x7f5af0068760 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.757+0000 7f5af6c28700 1 -- 172.21.15.12:0/35155091 >> 172.21.15.12:0/35155091 conn(0x7f5af0076f10 msgr2=0x7f5af0077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.757+0000 7f5af6c28700 1 -- 172.21.15.12:0/35155091 shutdown_connections 2024-05-08T23:51:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.757+0000 7f5af6c28700 1 -- 172.21.15.12:0/35155091 wait complete. 2024-05-08T23:51:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af6c28700 1 Processor -- start 2024-05-08T23:51:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af6c28700 1 -- start start 2024-05-08T23:51:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af6c28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af00682e0 0x7f5af0197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af6c28700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0068ca0 0x7f5af0197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af6c28700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af0107000 0x7f5af019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af6c28700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af019c240 con 0x7f5af00682e0 2024-05-08T23:51:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af6c28700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af019c3b0 con 0x7f5af0107000 2024-05-08T23:51:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af6c28700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af019c570 con 0x7f5af0068ca0 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af49c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af00682e0 0x7f5af0197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af49c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af00682e0 0x7f5af0197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37452/0 (socket says 172.21.15.12:37452) 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af49c4700 1 -- 172.21.15.12:0/2756029419 learned_addr learned my addr 172.21.15.12:0/2756029419 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5aeffff700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0068ca0 0x7f5af0197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.758+0000 7f5af51c5700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af0107000 0x7f5af019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5af49c4700 1 -- 172.21.15.12:0/2756029419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0068ca0 msgr2=0x7f5af0197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5af49c4700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0068ca0 0x7f5af0197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5af49c4700 1 -- 172.21.15.12:0/2756029419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af0107000 msgr2=0x7f5af019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5af49c4700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af0107000 0x7f5af019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5af49c4700 1 -- 172.21.15.12:0/2756029419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8009d00 con 0x7f5af00682e0 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5aeffff700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0068ca0 0x7f5af0197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5af49c4700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af00682e0 0x7f5af0197320 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400e9a0 tx=0x7f5ae400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5aedffb700 1 -- 172.21.15.12:0/2756029419 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4009790 con 0x7f5af00682e0 2024-05-08T23:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5af6c28700 1 -- 172.21.15.12:0/2756029419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc009520 con 0x7f5af00682e0 2024-05-08T23:51:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5aedffb700 1 -- 172.21.15.12:0/2756029419 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ae4010430 con 0x7f5af00682e0 2024-05-08T23:51:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.759+0000 7f5aedffb700 1 -- 172.21.15.12:0/2756029419 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4018680 con 0x7f5af00682e0 2024-05-08T23:51:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.760+0000 7f5af6c28700 1 -- 172.21.15.12:0/2756029419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af01a3eb0 con 0x7f5af00682e0 2024-05-08T23:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.762+0000 7f5aedffb700 1 -- 172.21.15.12:0/2756029419 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5ae40105a0 con 0x7f5af00682e0 2024-05-08T23:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.762+0000 7f5aedffb700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ae00a1290 0x7f5ae00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.763+0000 7f5aedffb700 1 -- 172.21.15.12:0/2756029419 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5ae4014070 con 0x7f5af00682e0 2024-05-08T23:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.763+0000 7f5aeffff700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ae00a1290 0x7f5ae00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.763+0000 7f5aeffff700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ae00a1290 0x7f5ae00a3750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009fd0 tx=0x7f5adc019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.763+0000 7f5b02de2b80 1 Processor -- start 2024-05-08T23:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.763+0000 7f5b02de2b80 1 -- start start 2024-05-08T23:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35d68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e352d0f0 con 0x55d0e35dc270 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e352aab0 con 0x55d0e35db950 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e35263a0 con 0x55d0e35d6480 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af49c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5aeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af49c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35dbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56310/0 (socket says 172.21.15.12:56310) 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af49c4700 1 -- 172.21.15.12:0/1887524204 learned_addr learned my addr 172.21.15.12:0/1887524204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af51c5700 1 --2- 172.21.15.12:0/1887524204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35d68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af51c5700 1 -- 172.21.15.12:0/1887524204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 msgr2=0x55d0e35dbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af51c5700 1 --2- 172.21.15.12:0/1887524204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35dbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af51c5700 1 -- 172.21.15.12:0/1887524204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 msgr2=0x55d0e35d5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af51c5700 1 --2- 172.21.15.12:0/1887524204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35d5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af51c5700 1 -- 172.21.15.12:0/1887524204 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0e33a7350 con 0x55d0e35d6480 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af49c4700 1 --2- 172.21.15.12:0/1887524204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.764+0000 7f5af51c5700 1 --2- 172.21.15.12:0/1887524204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35d68a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009cd0 tx=0x7f5ae80099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b16483e1b68e8f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.765+0000 7f5adaffd700 1 -- 172.21.15.12:0/1887524204 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae800e030 con 0x55d0e35d6480 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.765+0000 7f5adaffd700 1 -- 172.21.15.12:0/1887524204 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ae800f040 con 0x55d0e35d6480 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.765+0000 7f5adaffd700 1 -- 172.21.15.12:0/1887524204 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8005280 con 0x55d0e35d6480 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.765+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1887524204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 msgr2=0x55d0e35d68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.765+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/1887524204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35d68a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009cd0 tx=0x7f5ae80099e0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.765+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1887524204 shutdown_connections 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.765+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/1887524204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35d68a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.765+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/1887524204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.765+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/1887524204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.765+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1887524204 >> 172.21.15.12:0/1887524204 conn(0x55d0e35341b0 msgr2=0x55d0e35d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.766+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1887524204 shutdown_connections 2024-05-08T23:51:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.766+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1887524204 wait complete. 2024-05-08T23:51:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.766+0000 7f5b02de2b80 1 -- start start 2024-05-08T23:51:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.766+0000 7f5b02de2b80 1 Processor -- start 2024-05-08T23:51:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35e2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e34c2500 con 0x55d0e35dc270 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e34c21a0 con 0x55d0e35db950 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e33a9640 con 0x55d0e35d6480 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5b02de2b80 10 client.0 _ll_register_callbacks cb 0x55d0e34cbad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5af51c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5aeffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5aeffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35df4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56326/0 (socket says 172.21.15.12:56326) 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5af51c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35dfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37470/0 (socket says 172.21.15.12:37470) 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5aeffff700 1 -- 172.21.15.12:0/102064882 learned_addr learned my addr 172.21.15.12:0/102064882 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5af49c4700 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35e2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5af51c5700 1 -- 172.21.15.12:0/102064882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 msgr2=0x55d0e35e2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5af51c5700 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35e2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5af51c5700 1 -- 172.21.15.12:0/102064882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 msgr2=0x55d0e35df4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5af51c5700 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35df4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5af51c5700 1 -- 172.21.15.12:0/102064882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af01a3eb0 con 0x55d0e35dc270 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5aeffff700 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35df4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5af51c5700 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35dfc30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8007b80 tx=0x7f5ae80061f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd2219efbf2c63c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.767+0000 7f5acffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5acffff700 1 -- 172.21.15.12:0/102064882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8017870 con 0x55d0e35dc270 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5acffff700 1 -- 172.21.15.12:0/102064882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5ae8005040 con 0x55d0e35dc270 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5b02de2b80 1 -- 172.21.15.12:0/102064882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0e347f210 con 0x55d0e35dc270 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5acffff700 1 -- 172.21.15.12:0/102064882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ae8005200 con 0x55d0e35dc270 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5b02de2b80 10 client.16740 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5b02de2b80 1 -- 172.21.15.12:0/102064882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f5ae8009160 con 0x55d0e35dc270 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5acffff700 1 -- 172.21.15.12:0/102064882 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0e347f210 con 0x55d0e35dc270 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5acffff700 1 -- 172.21.15.12:0/102064882 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5ae801f7e0 con 0x55d0e35dc270 2024-05-08T23:51:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5b02de2b80 10 client.16740 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5b02de2b80 10 client.16740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5b02de2b80 20 client.16740 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.768+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d0e35efad0 0x55d0e35f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.769+0000 7f5b02de2b80 4 client.16740 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.769+0000 7f5b02de2b80 1 -- 172.21.15.12:0/102064882 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0e35a57b0 con 0x55d0e35efad0 2024-05-08T23:51:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.769+0000 7f5af49c4700 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d0e35efad0 0x55d0e35f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.769+0000 7f5acffff700 10 client.16740 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.769+0000 7f5af49c4700 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d0e35efad0 0x55d0e35f1ef0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d49f804d14ead54 server_cookie=aee57ab8fc24e511 in_seq=0 out_seq=0 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.770+0000 7f5acffff700 1 -- 172.21.15.12:0/102064882 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0e35a57b0 con 0x55d0e35efad0 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.770+0000 7f5acffff700 10 client.16740 handle_command_reply: tid=0 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.770+0000 7f5b02de2b80 1 client.16740 shutdown 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 1 -- 172.21.15.12:0/102064882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 msgr2=0x55d0e35dfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35dfc30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8007b80 tx=0x7f5ae80061f0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 1 -- 172.21.15.12:0/102064882 shutdown_connections 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35dfc30 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35e2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d0e35efad0 0x55d0e35f1ef0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/102064882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35df4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 1 -- 172.21.15.12:0/102064882 >> 172.21.15.12:0/102064882 conn(0x55d0e35341b0 msgr2=0x55d0e35da4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5acffff700 0 client.16740 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 1 -- 172.21.15.12:0/102064882 shutdown_connections 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 1 -- 172.21.15.12:0/102064882 wait complete. 2024-05-08T23:51:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.771+0000 7f5b02de2b80 20 client.16740 trim_cache size 0 max 16384 2024-05-08T23:51:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5b02de2b80 1 Processor -- start 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5b02de2b80 1 -- start start 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35d6480 0x55d0e35d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35db950 0x55d0e35d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35dc270 0x55d0e35e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e35d1960 con 0x55d0e35db950 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e35f7100 con 0x55d0e35d6480 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e35db390 con 0x55d0e35dc270 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5af49c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35d6480 0x55d0e35d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5af49c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35d6480 0x55d0e35d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56334/0 (socket says 172.21.15.12:56334) 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5aeffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35db950 0x55d0e35d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5af49c4700 1 -- 172.21.15.12:0/2631824660 learned_addr learned my addr 172.21.15.12:0/2631824660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.790+0000 7f5af51c5700 1 --2- 172.21.15.12:0/2631824660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35dc270 0x55d0e35e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.791+0000 7f5af49c4700 1 -- 172.21.15.12:0/2631824660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35dc270 msgr2=0x55d0e35e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.791+0000 7f5af49c4700 1 --2- 172.21.15.12:0/2631824660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35dc270 0x55d0e35e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.791+0000 7f5af49c4700 1 -- 172.21.15.12:0/2631824660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35db950 msgr2=0x55d0e35d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.791+0000 7f5af49c4700 1 --2- 172.21.15.12:0/2631824660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35db950 0x55d0e35d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.791+0000 7f5af49c4700 1 -- 172.21.15.12:0/2631824660 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0e3353b00 con 0x55d0e35d6480 2024-05-08T23:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.791+0000 7f5af49c4700 1 --2- 172.21.15.12:0/2631824660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35d6480 0x55d0e35d0910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4099c80 tx=0x7f5ae4023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fa49e31a28b1ba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.791+0000 7f5adb7fe700 1 -- 172.21.15.12:0/2631824660 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40adea0 con 0x55d0e35d6480 2024-05-08T23:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.791+0000 7f5adb7fe700 1 -- 172.21.15.12:0/2631824660 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ae4099950 con 0x55d0e35d6480 2024-05-08T23:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.791+0000 7f5adb7fe700 1 -- 172.21.15.12:0/2631824660 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4094e40 con 0x55d0e35d6480 2024-05-08T23:51:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.792+0000 7f5b02de2b80 1 -- 172.21.15.12:0/2631824660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35d6480 msgr2=0x55d0e35d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.792+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/2631824660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35d6480 0x55d0e35d0910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4099c80 tx=0x7f5ae4023040 comp rx=0 tx=0).stop 2024-05-08T23:51:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.792+0000 7f5b02de2b80 1 -- 172.21.15.12:0/2631824660 shutdown_connections 2024-05-08T23:51:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.792+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/2631824660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35dc270 0x55d0e35e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.792+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/2631824660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35d6480 0x55d0e35d0910 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.792+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/2631824660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35db950 0x55d0e35d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.792+0000 7f5b02de2b80 1 -- 172.21.15.12:0/2631824660 >> 172.21.15.12:0/2631824660 conn(0x55d0e35341b0 msgr2=0x55d0e35cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.792+0000 7f5b02de2b80 1 -- 172.21.15.12:0/2631824660 shutdown_connections 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.792+0000 7f5b02de2b80 1 -- 172.21.15.12:0/2631824660 wait complete. 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.792+0000 7f5b02de2b80 1 -- start start 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5b02de2b80 1 Processor -- start 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5b02de2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35e5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e35e5700 con 0x55d0e35dc270 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e35e5870 con 0x55d0e35db950 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5b02de2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e35e5a30 con 0x55d0e35d6480 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5b02de2b80 10 client.0 _ll_register_callbacks cb 0x55d0e34cbad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5aeffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5af51c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35e5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5aeffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35e4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56340/0 (socket says 172.21.15.12:56340) 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5af49c4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.793+0000 7f5aeffff700 1 -- 172.21.15.12:0/1436501493 learned_addr learned my addr 172.21.15.12:0/1436501493 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5af51c5700 1 -- 172.21.15.12:0/1436501493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 msgr2=0x55d0e35e6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5af51c5700 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35e6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5af51c5700 1 -- 172.21.15.12:0/1436501493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 msgr2=0x55d0e35e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5af51c5700 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35e4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5af51c5700 1 -- 172.21.15.12:0/1436501493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc009520 con 0x55d0e35dc270 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5af49c4700 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35e6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5af51c5700 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35e5130 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8000f80 tx=0x7f5ae80178a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587d82ed4e48c285 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5acffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5acffff700 1 -- 172.21.15.12:0/1436501493 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8003e10 con 0x55d0e35dc270 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5acffff700 1 -- 172.21.15.12:0/1436501493 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5ae8006000 con 0x55d0e35dc270 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1436501493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0e351ee60 con 0x55d0e35dc270 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5acffff700 1 -- 172.21.15.12:0/1436501493 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ae801f410 con 0x55d0e35dc270 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5b02de2b80 10 client.16746 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1436501493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d0e34ec500 con 0x55d0e35dc270 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.794+0000 7f5acffff700 1 -- 172.21.15.12:0/1436501493 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0e351ee60 con 0x55d0e35dc270 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.795+0000 7f5acffff700 1 -- 172.21.15.12:0/1436501493 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5ae80045d0 con 0x55d0e35dc270 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.795+0000 7f5b02de2b80 10 client.16746 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.795+0000 7f5b02de2b80 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.795+0000 7f5b02de2b80 20 client.16746 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.795+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d0e36461c0 0x55d0e36485e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.795+0000 7f5b02de2b80 4 client.16746 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.795+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1436501493 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d0e3374c90 con 0x55d0e36461c0 2024-05-08T23:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.795+0000 7f5af49c4700 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d0e36461c0 0x55d0e36485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.795+0000 7f5acffff700 10 client.16746 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.795+0000 7f5af49c4700 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d0e36461c0 0x55d0e36485e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad825157574e08f6 server_cookie=903e1a8179864d6f in_seq=0 out_seq=0 2024-05-08T23:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5acffff700 1 -- 172.21.15.12:0/1436501493 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d0e3374c90 con 0x55d0e36461c0 2024-05-08T23:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5acffff700 10 client.16746 handle_command_reply: tid=0 2024-05-08T23:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5b02de2b80 1 client.16746 shutdown 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1436501493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 msgr2=0x55d0e35e5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35e5130 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8000f80 tx=0x7f5ae80178a0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1436501493 shutdown_connections 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e35dc270 0x55d0e35e5130 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0e35d6480 0x55d0e35e6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d0e36461c0 0x55d0e36485e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5b02de2b80 1 --2- 172.21.15.12:0/1436501493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0e35db950 0x55d0e35e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.865+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1436501493 >> 172.21.15.12:0/1436501493 conn(0x55d0e35341b0 msgr2=0x55d0e35d54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.866+0000 7f5acffff700 0 client.16746 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.866+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1436501493 shutdown_connections 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.866+0000 7f5b02de2b80 1 -- 172.21.15.12:0/1436501493 wait complete. 2024-05-08T23:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.866+0000 7f5b02de2b80 20 client.16746 trim_cache size 0 max 16384 2024-05-08T23:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:51:01.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 213916 11.10 MB/sec execute 57 sec latency 23.249 ms 2024-05-08T23:51:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 -- 172.21.15.12:0/2756029419 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ae00a1290 msgr2=0x7f5ae00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ae00a1290 0x7f5ae00a3750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009fd0 tx=0x7f5adc019040 comp rx=0 tx=0).stop 2024-05-08T23:51:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 -- 172.21.15.12:0/2756029419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af00682e0 msgr2=0x7f5af0197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af00682e0 0x7f5af0197320 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400e9a0 tx=0x7f5ae400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 -- 172.21.15.12:0/2756029419 shutdown_connections 2024-05-08T23:51:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ae00a1290 0x7f5ae00a3750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af0107000 0x7f5af019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af00682e0 0x7f5af0197320 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 --2- 172.21.15.12:0/2756029419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0068ca0 0x7f5af0197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 -- 172.21.15.12:0/2756029419 >> 172.21.15.12:0/2756029419 conn(0x7f5af0076f10 msgr2=0x7f5af00fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 -- 172.21.15.12:0/2756029419 shutdown_connections 2024-05-08T23:51:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:01.867+0000 7f5af6c28700 1 -- 172.21.15.12:0/2756029419 wait complete. 2024-05-08T23:51:02.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:02 smithi204 ceph-mon[47373]: pgmap v143: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 449 op/s 2024-05-08T23:51:02.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:02 smithi160 ceph-mon[46152]: pgmap v143: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 449 op/s 2024-05-08T23:51:02.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 216631 11.20 MB/sec execute 58 sec latency 17.155 ms 2024-05-08T23:51:02.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:02 smithi012 ceph-mon[42503]: pgmap v143: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 449 op/s 2024-05-08T23:51:03.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:03 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:51:03.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:03 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:51:03.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 218126 11.11 MB/sec execute 59 sec latency 12.476 ms 2024-05-08T23:51:03.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:03 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:51:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:51:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:51:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:04 smithi204 ceph-mon[47373]: pgmap v144: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 276 op/s 2024-05-08T23:51:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:51:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:51:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:04 smithi160 ceph-mon[46152]: pgmap v144: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 276 op/s 2024-05-08T23:51:04.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 220369 11.15 MB/sec execute 60 sec latency 18.076 ms 2024-05-08T23:51:04.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:51:04.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:51:04.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:04 smithi012 ceph-mon[42503]: pgmap v144: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 276 op/s 2024-05-08T23:51:05.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 222739 11.15 MB/sec execute 61 sec latency 21.076 ms 2024-05-08T23:51:06.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:06 smithi204 ceph-mon[47373]: pgmap v145: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 276 op/s 2024-05-08T23:51:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:06 smithi160 ceph-mon[46152]: pgmap v145: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 276 op/s 2024-05-08T23:51:06.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 224274 11.11 MB/sec execute 62 sec latency 19.351 ms 2024-05-08T23:51:06.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:06 smithi012 ceph-mon[42503]: pgmap v145: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 276 op/s 2024-05-08T23:51:07.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 227031 11.20 MB/sec execute 63 sec latency 14.988 ms 2024-05-08T23:51:08.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:08 smithi204 ceph-mon[47373]: pgmap v146: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 436 op/s 2024-05-08T23:51:08.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:08 smithi160 ceph-mon[46152]: pgmap v146: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 436 op/s 2024-05-08T23:51:08.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 228549 11.13 MB/sec execute 64 sec latency 18.521 ms 2024-05-08T23:51:08.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:08 smithi012 ceph-mon[42503]: pgmap v146: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 436 op/s 2024-05-08T23:51:09.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 230812 11.17 MB/sec execute 65 sec latency 77.393 ms 2024-05-08T23:51:10.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:10 smithi204 ceph-mon[47373]: pgmap v147: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-05-08T23:51:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:10 smithi160 ceph-mon[46152]: pgmap v147: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-05-08T23:51:10.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 232380 11.09 MB/sec execute 66 sec latency 33.474 ms 2024-05-08T23:51:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:10 smithi012 ceph-mon[42503]: pgmap v147: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-05-08T23:51:11.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 234562 11.13 MB/sec execute 67 sec latency 13.466 ms 2024-05-08T23:51:12.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:12 smithi204 ceph-mon[47373]: pgmap v148: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 425 op/s 2024-05-08T23:51:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:12 smithi160 ceph-mon[46152]: pgmap v148: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 425 op/s 2024-05-08T23:51:12.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 236907 11.13 MB/sec execute 68 sec latency 15.893 ms 2024-05-08T23:51:12.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:12 smithi012 ceph-mon[42503]: pgmap v148: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 425 op/s 2024-05-08T23:51:13.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 238497 11.09 MB/sec execute 69 sec latency 17.124 ms 2024-05-08T23:51:14.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:14 smithi204 ceph-mon[47373]: pgmap v149: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-08T23:51:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:14 smithi160 ceph-mon[46152]: pgmap v149: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-08T23:51:14.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 240905 11.12 MB/sec execute 70 sec latency 44.877 ms 2024-05-08T23:51:14.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:14 smithi012 ceph-mon[42503]: pgmap v149: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-08T23:51:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.005+0000 7fe979c0f700 1 -- 172.21.15.12:0/4018428109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974104d80 msgr2=0x7fe974105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.005+0000 7fe979c0f700 1 --2- 172.21.15.12:0/4018428109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974104d80 0x7fe974105160 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe964009990 tx=0x7fe964009ca0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.005+0000 7fe979c0f700 1 -- 172.21.15.12:0/4018428109 shutdown_connections 2024-05-08T23:51:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.005+0000 7fe979c0f700 1 --2- 172.21.15.12:0/4018428109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974100ee0 0x7fe97410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.005+0000 7fe979c0f700 1 --2- 172.21.15.12:0/4018428109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974104d80 0x7fe974105160 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.005+0000 7fe979c0f700 1 --2- 172.21.15.12:0/4018428109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe974100520 0x7fe9741009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.005+0000 7fe979c0f700 1 -- 172.21.15.12:0/4018428109 >> 172.21.15.12:0/4018428109 conn(0x7fe9740763a0 msgr2=0x7fe9740fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.005+0000 7fe979c0f700 1 -- 172.21.15.12:0/4018428109 shutdown_connections 2024-05-08T23:51:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.005+0000 7fe979c0f700 1 -- 172.21.15.12:0/4018428109 wait complete. 2024-05-08T23:51:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.006+0000 7fe979c0f700 1 Processor -- start 2024-05-08T23:51:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.006+0000 7fe979c0f700 1 -- start start 2024-05-08T23:51:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.006+0000 7fe979c0f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974100520 0x7fe974198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.006+0000 7fe979c0f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe974100ee0 0x7fe9741987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.006+0000 7fe979c0f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974104d80 0x7fe9741a3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.006+0000 7fe979c0f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974198ff0 con 0x7fe974104d80 2024-05-08T23:51:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe979c0f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974196070 con 0x7fe974100520 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe979c0f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974196230 con 0x7fe974100ee0 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe972ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe974100ee0 0x7fe9741987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe973fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974104d80 0x7fe9741a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe9737fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974100520 0x7fe974198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe972ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe974100ee0 0x7fe9741987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:58582/0 (socket says 172.21.15.12:58582) 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe972ffd700 1 -- 172.21.15.12:0/1614278788 learned_addr learned my addr 172.21.15.12:0/1614278788 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe9737fe700 1 -- 172.21.15.12:0/1614278788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe974100ee0 msgr2=0x7fe9741987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe9737fe700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe974100ee0 0x7fe9741987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe9737fe700 1 -- 172.21.15.12:0/1614278788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974104d80 msgr2=0x7fe9741a3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe9737fe700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974104d80 0x7fe9741a3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe9737fe700 1 -- 172.21.15.12:0/1614278788 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640095f0 con 0x7fe974100520 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe9737fe700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974100520 0x7fe974198280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe9640098e0 tx=0x7fe964005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.007+0000 7fe970ff9700 1 -- 172.21.15.12:0/1614278788 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96401d070 con 0x7fe974100520 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.008+0000 7fe970ff9700 1 -- 172.21.15.12:0/1614278788 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe964022470 con 0x7fe974100520 2024-05-08T23:51:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.008+0000 7fe979c0f700 1 -- 172.21.15.12:0/1614278788 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9741964b0 con 0x7fe974100520 2024-05-08T23:51:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.008+0000 7fe970ff9700 1 -- 172.21.15.12:0/1614278788 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964017650 con 0x7fe974100520 2024-05-08T23:51:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.008+0000 7fe979c0f700 1 -- 172.21.15.12:0/1614278788 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9741969b0 con 0x7fe974100520 2024-05-08T23:51:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.010+0000 7fe970ff9700 1 -- 172.21.15.12:0/1614278788 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe9640225e0 con 0x7fe974100520 2024-05-08T23:51:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.011+0000 7fe970ff9700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe95c0a1310 0x7fe95c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.011+0000 7fe970ff9700 1 -- 172.21.15.12:0/1614278788 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe9640ce1f0 con 0x7fe974100520 2024-05-08T23:51:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.011+0000 7fe972ffd700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe95c0a1310 0x7fe95c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe972ffd700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe95c0a1310 0x7fe95c0a37d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe9740ffbd0 tx=0x7fe968007480 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe985dc9b80 1 Processor -- start 2024-05-08T23:51:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe985dc9b80 1 -- start start 2024-05-08T23:51:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a9668d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 0x5562a9662f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a96638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a95ba0f0 con 0x5562a9663480 2024-05-08T23:51:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a95b7ab0 con 0x5562a9668950 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a95b33a0 con 0x5562a9669270 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe972ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 0x5562a9662f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe973fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a96638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe972ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 0x5562a9662f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:58586/0 (socket says 172.21.15.12:58586) 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe973fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a96638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43278/0 (socket says 172.21.15.12:43278) 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe972ffd700 1 -- 172.21.15.12:0/4066036423 learned_addr learned my addr 172.21.15.12:0/4066036423 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.012+0000 7fe9737fe700 1 --2- 172.21.15.12:0/4066036423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a9668d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.013+0000 7fe972ffd700 1 -- 172.21.15.12:0/4066036423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 msgr2=0x5562a9668d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.013+0000 7fe972ffd700 1 --2- 172.21.15.12:0/4066036423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a9668d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.013+0000 7fe972ffd700 1 -- 172.21.15.12:0/4066036423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 msgr2=0x5562a96638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.013+0000 7fe972ffd700 1 --2- 172.21.15.12:0/4066036423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a96638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.013+0000 7fe972ffd700 1 -- 172.21.15.12:0/4066036423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562a9434350 con 0x5562a9669270 2024-05-08T23:51:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.013+0000 7fe972ffd700 1 --2- 172.21.15.12:0/4066036423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 0x5562a9662f40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe968010f20 tx=0x7fe968013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d064ea1de1a216 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.013+0000 7fe959ffb700 1 -- 172.21.15.12:0/4066036423 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968011db0 con 0x5562a9669270 2024-05-08T23:51:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.013+0000 7fe959ffb700 1 -- 172.21.15.12:0/4066036423 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe968015070 con 0x5562a9669270 2024-05-08T23:51:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.013+0000 7fe959ffb700 1 -- 172.21.15.12:0/4066036423 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96801c5f0 con 0x5562a9669270 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.014+0000 7fe985dc9b80 1 -- 172.21.15.12:0/4066036423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 msgr2=0x5562a9662f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.014+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/4066036423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 0x5562a9662f40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe968010f20 tx=0x7fe968013000 comp rx=0 tx=0).stop 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.014+0000 7fe985dc9b80 1 -- 172.21.15.12:0/4066036423 shutdown_connections 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.014+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/4066036423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a96638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.014+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/4066036423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a9668d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.014+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/4066036423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 0x5562a9662f40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.014+0000 7fe985dc9b80 1 -- 172.21.15.12:0/4066036423 >> 172.21.15.12:0/4066036423 conn(0x5562a95c11b0 msgr2=0x5562a965d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.014+0000 7fe985dc9b80 1 -- 172.21.15.12:0/4066036423 shutdown_connections 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.014+0000 7fe985dc9b80 1 -- 172.21.15.12:0/4066036423 wait complete. 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.014+0000 7fe985dc9b80 1 -- start start 2024-05-08T23:51:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe985dc9b80 1 Processor -- start 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9663480 0x5562a966f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a966c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9669270 0x5562a966cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a954f500 con 0x5562a9669270 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a954f1a0 con 0x5562a9668950 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a9436640 con 0x5562a9663480 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe985dc9b80 10 client.0 _ll_register_callbacks cb 0x5562a9558ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe973fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9669270 0x5562a966cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe9737fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9663480 0x5562a966f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe973fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9669270 0x5562a966cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43280/0 (socket says 172.21.15.12:43280) 2024-05-08T23:51:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe973fff700 1 -- 172.21.15.12:0/3652153728 learned_addr learned my addr 172.21.15.12:0/3652153728 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe9737fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9663480 0x5562a966f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:58594/0 (socket says 172.21.15.12:58594) 2024-05-08T23:51:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.015+0000 7fe972ffd700 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a966c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe973fff700 1 -- 172.21.15.12:0/3652153728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9663480 msgr2=0x5562a966f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe973fff700 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9663480 0x5562a966f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe973fff700 1 -- 172.21.15.12:0/3652153728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 msgr2=0x5562a966c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe973fff700 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a966c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe973fff700 1 -- 172.21.15.12:0/3652153728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe968010c50 con 0x5562a9669270 2024-05-08T23:51:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe973fff700 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9669270 0x5562a966cb50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe96c011ef0 tx=0x7fe96c014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea6ac15263f2cff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe952ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe952ffd700 1 -- 172.21.15.12:0/3652153728 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00e040 con 0x5562a9669270 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe985dc9b80 1 -- 172.21.15.12:0/3652153728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562a950c210 con 0x5562a9669270 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe952ffd700 1 -- 172.21.15.12:0/3652153728 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe96c014b90 con 0x5562a9669270 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe952ffd700 1 -- 172.21.15.12:0/3652153728 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe96c014d90 con 0x5562a9669270 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe985dc9b80 10 client.16758 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.016+0000 7fe985dc9b80 1 -- 172.21.15.12:0/3652153728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5562a93e0b00 con 0x5562a9669270 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.017+0000 7fe952ffd700 1 -- 172.21.15.12:0/3652153728 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5562a950c210 con 0x5562a9669270 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.017+0000 7fe952ffd700 1 -- 172.21.15.12:0/3652153728 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe96c01c020 con 0x5562a9669270 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.017+0000 7fe985dc9b80 10 client.16758 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.017+0000 7fe985dc9b80 10 client.16758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.017+0000 7fe985dc9b80 20 client.16758 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.017+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5562a967cc70 0x5562a967f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.017+0000 7fe985dc9b80 4 client.16758 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.017+0000 7fe985dc9b80 1 -- 172.21.15.12:0/3652153728 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562a96327b0 con 0x5562a967cc70 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.017+0000 7fe9737fe700 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5562a967cc70 0x5562a967f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.018+0000 7fe952ffd700 10 client.16758 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.018+0000 7fe9737fe700 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5562a967cc70 0x5562a967f090 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8744296e7ae9dbbc server_cookie=c086662d6d25e5fa in_seq=0 out_seq=0 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.018+0000 7fe952ffd700 1 -- 172.21.15.12:0/3652153728 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5562a96327b0 con 0x5562a967cc70 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.018+0000 7fe952ffd700 10 client.16758 handle_command_reply: tid=0 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.018+0000 7fe985dc9b80 1 client.16758 shutdown 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.018+0000 7fe985dc9b80 1 -- 172.21.15.12:0/3652153728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9669270 msgr2=0x5562a966cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.018+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9669270 0x5562a966cb50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe96c011ef0 tx=0x7fe96c014600 comp rx=0 tx=0).stop 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.019+0000 7fe985dc9b80 1 -- 172.21.15.12:0/3652153728 shutdown_connections 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.019+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9669270 0x5562a966cb50 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.019+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9663480 0x5562a966f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.019+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5562a967cc70 0x5562a967f090 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.019+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/3652153728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a966c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.019+0000 7fe952ffd700 0 client.16758 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.019+0000 7fe985dc9b80 1 -- 172.21.15.12:0/3652153728 >> 172.21.15.12:0/3652153728 conn(0x5562a95c11b0 msgr2=0x5562a96674a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.019+0000 7fe985dc9b80 1 -- 172.21.15.12:0/3652153728 shutdown_connections 2024-05-08T23:51:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.019+0000 7fe985dc9b80 1 -- 172.21.15.12:0/3652153728 wait complete. 2024-05-08T23:51:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.019+0000 7fe985dc9b80 20 client.16758 trim_cache size 0 max 16384 2024-05-08T23:51:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.035+0000 7fe985dc9b80 1 Processor -- start 2024-05-08T23:51:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.035+0000 7fe985dc9b80 1 -- start start 2024-05-08T23:51:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9668950 0x5562a965d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9669270 0x5562a9667390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a965f0f0 con 0x5562a9663480 2024-05-08T23:51:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a967a300 con 0x5562a9669270 2024-05-08T23:51:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a966d130 con 0x5562a9668950 2024-05-08T23:51:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe9737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe973fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9669270 0x5562a9667390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe972ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9668950 0x5562a965d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe9737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43284/0 (socket says 172.21.15.12:43284) 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe9737fe700 1 -- 172.21.15.12:0/881918457 learned_addr learned my addr 172.21.15.12:0/881918457 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe973fff700 1 -- 172.21.15.12:0/881918457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9668950 msgr2=0x5562a965d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe973fff700 1 --2- 172.21.15.12:0/881918457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9668950 0x5562a965d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe973fff700 1 -- 172.21.15.12:0/881918457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 msgr2=0x5562a966ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe973fff700 1 --2- 172.21.15.12:0/881918457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe973fff700 1 -- 172.21.15.12:0/881918457 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562a9579500 con 0x5562a9669270 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe9737fe700 1 --2- 172.21.15.12:0/881918457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.036+0000 7fe973fff700 1 --2- 172.21.15.12:0/881918457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9669270 0x5562a9667390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b010 tx=0x7fe96c014a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8816b224f3bac4d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe95a7fc700 1 -- 172.21.15.12:0/881918457 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00e030 con 0x5562a9669270 2024-05-08T23:51:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe95a7fc700 1 -- 172.21.15.12:0/881918457 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96c00f040 con 0x5562a9669270 2024-05-08T23:51:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe95a7fc700 1 -- 172.21.15.12:0/881918457 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c007580 con 0x5562a9669270 2024-05-08T23:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe985dc9b80 1 -- 172.21.15.12:0/881918457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9669270 msgr2=0x5562a9667390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/881918457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9669270 0x5562a9667390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b010 tx=0x7fe96c014a00 comp rx=0 tx=0).stop 2024-05-08T23:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe985dc9b80 1 -- 172.21.15.12:0/881918457 shutdown_connections 2024-05-08T23:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/881918457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9669270 0x5562a9667390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/881918457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/881918457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9668950 0x5562a965d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe985dc9b80 1 -- 172.21.15.12:0/881918457 >> 172.21.15.12:0/881918457 conn(0x5562a95c11b0 msgr2=0x5562a9667cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.037+0000 7fe985dc9b80 1 -- 172.21.15.12:0/881918457 shutdown_connections 2024-05-08T23:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.038+0000 7fe985dc9b80 1 -- 172.21.15.12:0/881918457 wait complete. 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.038+0000 7fe985dc9b80 1 -- start start 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.038+0000 7fe985dc9b80 1 Processor -- start 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.038+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.038+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a9670150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.038+0000 7fe985dc9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 0x5562a9670690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.038+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a96d21f0 con 0x5562a9663480 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.038+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a96d2360 con 0x5562a9668950 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.038+0000 7fe985dc9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562a96d2520 con 0x5562a9669270 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe985dc9b80 10 client.0 _ll_register_callbacks cb 0x5562a9558ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe9737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe972ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a9670150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe9737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43294/0 (socket says 172.21.15.12:43294) 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe9737fe700 1 -- 172.21.15.12:0/2499215313 learned_addr learned my addr 172.21.15.12:0/2499215313 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe973fff700 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 0x5562a9670690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe972ffd700 1 -- 172.21.15.12:0/2499215313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 msgr2=0x5562a9670690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe972ffd700 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 0x5562a9670690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe972ffd700 1 -- 172.21.15.12:0/2499215313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 msgr2=0x5562a966fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe972ffd700 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe972ffd700 1 -- 172.21.15.12:0/2499215313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9741969b0 con 0x5562a9668950 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe9737fe700 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe972ffd700 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a9670150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe968011a00 tx=0x7fe968011ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=853cb47013e3cc96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe952ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe952ffd700 1 -- 172.21.15.12:0/2499215313 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96801e730 con 0x5562a9668950 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe952ffd700 1 -- 172.21.15.12:0/2499215313 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe968017070 con 0x5562a9668950 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.039+0000 7fe985dc9b80 1 -- 172.21.15.12:0/2499215313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe968015070 con 0x5562a9668950 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe952ffd700 1 -- 172.21.15.12:0/2499215313 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe96801e890 con 0x5562a9668950 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe985dc9b80 10 client.26546 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe985dc9b80 1 -- 172.21.15.12:0/2499215313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5562a93e0b00 con 0x5562a9668950 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe952ffd700 1 -- 172.21.15.12:0/2499215313 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe968015070 con 0x5562a9668950 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe952ffd700 1 -- 172.21.15.12:0/2499215313 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe9680263f0 con 0x5562a9668950 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe985dc9b80 10 client.26546 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe985dc9b80 10 client.26546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe985dc9b80 20 client.26546 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5562a96d3c90 0x5562a96d60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe985dc9b80 4 client.26546 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:51:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe985dc9b80 1 -- 172.21.15.12:0/2499215313 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5562a95abe60 con 0x5562a96d3c90 2024-05-08T23:51:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.040+0000 7fe9737fe700 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5562a96d3c90 0x5562a96d60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.041+0000 7fe9737fe700 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5562a96d3c90 0x5562a96d60b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698b90046d0ba34f server_cookie=5dcd4926a9c78117 in_seq=0 out_seq=0 2024-05-08T23:51:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.041+0000 7fe952ffd700 10 client.26546 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe952ffd700 1 -- 172.21.15.12:0/2499215313 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5562a95abe60 con 0x5562a96d3c90 2024-05-08T23:51:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe952ffd700 10 client.26546 handle_command_reply: tid=0 2024-05-08T23:51:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 client.26546 shutdown 2024-05-08T23:51:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 -- 172.21.15.12:0/2499215313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 msgr2=0x5562a9670150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a9670150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe968011a00 tx=0x7fe968011ea0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 -- 172.21.15.12:0/2499215313 shutdown_connections 2024-05-08T23:51:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562a9669270 0x5562a9670690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562a9663480 0x5562a966fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5562a96d3c90 0x5562a96d60b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe952ffd700 0 client.26546 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 --2- 172.21.15.12:0/2499215313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5562a9668950 0x5562a9670150 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 -- 172.21.15.12:0/2499215313 >> 172.21.15.12:0/2499215313 conn(0x5562a95c11b0 msgr2=0x5562a96786f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 -- 172.21.15.12:0/2499215313 shutdown_connections 2024-05-08T23:51:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.108+0000 7fe985dc9b80 1 -- 172.21.15.12:0/2499215313 wait complete. 2024-05-08T23:51:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe985dc9b80 20 client.26546 trim_cache size 0 max 16384 2024-05-08T23:51:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:51:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:51:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:51:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:51:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 -- 172.21.15.12:0/1614278788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe95c0a1310 msgr2=0x7fe95c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe95c0a1310 0x7fe95c0a37d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe9740ffbd0 tx=0x7fe968007480 comp rx=0 tx=0).stop 2024-05-08T23:51:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 -- 172.21.15.12:0/1614278788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974100520 msgr2=0x7fe974198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974100520 0x7fe974198280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe9640098e0 tx=0x7fe964005740 comp rx=0 tx=0).stop 2024-05-08T23:51:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 -- 172.21.15.12:0/1614278788 shutdown_connections 2024-05-08T23:51:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe95c0a1310 0x7fe95c0a37d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974104d80 0x7fe9741a3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974100520 0x7fe974198280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 --2- 172.21.15.12:0/1614278788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe974100ee0 0x7fe9741987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 -- 172.21.15.12:0/1614278788 >> 172.21.15.12:0/1614278788 conn(0x7fe9740763a0 msgr2=0x7fe974102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 -- 172.21.15.12:0/1614278788 shutdown_connections 2024-05-08T23:51:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:15.109+0000 7fe979c0f700 1 -- 172.21.15.12:0/1614278788 wait complete. 2024-05-08T23:51:15.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 242141 11.05 MB/sec execute 71 sec latency 49.912 ms 2024-05-08T23:51:15.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:15 smithi012 ceph-mon[42503]: pgmap v150: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-08T23:51:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:15 smithi204 ceph-mon[47373]: pgmap v150: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-08T23:51:16.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:15 smithi160 ceph-mon[46152]: pgmap v150: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-08T23:51:16.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244994 11.15 MB/sec execute 72 sec latency 17.677 ms 2024-05-08T23:51:17.862 INFO:tasks.workunit.client.0.smithi012.stdout: 1 246597 11.07 MB/sec execute 73 sec latency 10.753 ms 2024-05-08T23:51:18.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:18 smithi204 ceph-mon[47373]: pgmap v151: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 443 op/s 2024-05-08T23:51:18.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:18 smithi160 ceph-mon[46152]: pgmap v151: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 443 op/s 2024-05-08T23:51:18.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 248772 11.11 MB/sec execute 74 sec latency 9.806 ms 2024-05-08T23:51:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:18 smithi012 ceph-mon[42503]: pgmap v151: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 443 op/s 2024-05-08T23:51:19.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 251310 11.13 MB/sec execute 75 sec latency 12.762 ms 2024-05-08T23:51:20.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:20 smithi204 ceph-mon[47373]: pgmap v152: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-08T23:51:20.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:20 smithi160 ceph-mon[46152]: pgmap v152: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-08T23:51:20.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 252858 11.08 MB/sec execute 76 sec latency 21.316 ms 2024-05-08T23:51:20.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:20 smithi012 ceph-mon[42503]: pgmap v152: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-08T23:51:21.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 255562 11.16 MB/sec execute 77 sec latency 41.150 ms 2024-05-08T23:51:22.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:22 smithi204 ceph-mon[47373]: pgmap v153: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 405 op/s 2024-05-08T23:51:22.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:22 smithi160 ceph-mon[46152]: pgmap v153: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 405 op/s 2024-05-08T23:51:22.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 257113 11.10 MB/sec execute 78 sec latency 12.270 ms 2024-05-08T23:51:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:22 smithi012 ceph-mon[42503]: pgmap v153: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 405 op/s 2024-05-08T23:51:23.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 259354 11.13 MB/sec execute 79 sec latency 11.795 ms 2024-05-08T23:51:24.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:24 smithi204 ceph-mon[47373]: pgmap v154: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-05-08T23:51:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:24 smithi160 ceph-mon[46152]: pgmap v154: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-05-08T23:51:24.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 261522 11.09 MB/sec execute 80 sec latency 12.348 ms 2024-05-08T23:51:24.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:24 smithi012 ceph-mon[42503]: pgmap v154: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-05-08T23:51:25.685 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:51:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.857+0000 7f29892d7700 1 -- 172.21.15.12:0/1934182176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2984107060 msgr2=0x7f2984107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.857+0000 7f29892d7700 1 --2- 172.21.15.12:0/1934182176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2984107060 0x7f2984107440 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f297400b0a0 tx=0x7f297400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.857+0000 7f29892d7700 1 -- 172.21.15.12:0/1934182176 shutdown_connections 2024-05-08T23:51:25.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.857+0000 7f29892d7700 1 --2- 172.21.15.12:0/1934182176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2984069f10 0x7f298410dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.857+0000 7f29892d7700 1 --2- 172.21.15.12:0/1934182176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2984107060 0x7f2984107440 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.857+0000 7f29892d7700 1 --2- 172.21.15.12:0/1934182176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2984069550 0x7f29840699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.857+0000 7f29892d7700 1 -- 172.21.15.12:0/1934182176 >> 172.21.15.12:0/1934182176 conn(0x7f2984076f10 msgr2=0x7f2984077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:25.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.857+0000 7f29892d7700 1 -- 172.21.15.12:0/1934182176 shutdown_connections 2024-05-08T23:51:25.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.857+0000 7f29892d7700 1 -- 172.21.15.12:0/1934182176 wait complete. 2024-05-08T23:51:25.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.857+0000 7f29892d7700 1 Processor -- start 2024-05-08T23:51:25.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29892d7700 1 -- start start 2024-05-08T23:51:25.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29892d7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2984069550 0x7f2984197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29892d7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2984069f10 0x7f2984197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29892d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2984107060 0x7f298419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29892d7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298419c360 con 0x7f2984107060 2024-05-08T23:51:25.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29892d7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298419c4d0 con 0x7f2984069f10 2024-05-08T23:51:25.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29892d7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298419c690 con 0x7f2984069550 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2984107060 0x7f298419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29827fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2984069f10 0x7f2984197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2984107060 0x7f298419bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53154/0 (socket says 172.21.15.12:53154) 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f2982ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2984069550 0x7f2984197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29837fe700 1 -- 172.21.15.12:0/3751092441 learned_addr learned my addr 172.21.15.12:0/3751092441 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f2982ffd700 1 -- 172.21.15.12:0/3751092441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2984069f10 msgr2=0x7f2984197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f2982ffd700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2984069f10 0x7f2984197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f2982ffd700 1 -- 172.21.15.12:0/3751092441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2984107060 msgr2=0x7f298419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f2982ffd700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2984107060 0x7f298419bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f2982ffd700 1 -- 172.21.15.12:0/3751092441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2974009d00 con 0x7f2984069550 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29827fc700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2984069f10 0x7f2984197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.858+0000 7f29837fe700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2984107060 0x7f298419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.859+0000 7f2982ffd700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2984069550 0x7f2984197440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f297400b070 tx=0x7f29740078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:25.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.859+0000 7f297bfff700 1 -- 172.21.15.12:0/3751092441 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297400e040 con 0x7f2984069550 2024-05-08T23:51:25.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.859+0000 7f297bfff700 1 -- 172.21.15.12:0/3751092441 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f297401d470 con 0x7f2984069550 2024-05-08T23:51:25.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.859+0000 7f29892d7700 1 -- 172.21.15.12:0/3751092441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298419c910 con 0x7f2984069550 2024-05-08T23:51:25.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.859+0000 7f297bfff700 1 -- 172.21.15.12:0/3751092441 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29740052a0 con 0x7f2984069550 2024-05-08T23:51:25.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.859+0000 7f29892d7700 1 -- 172.21.15.12:0/3751092441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29841a41b0 con 0x7f2984069550 2024-05-08T23:51:25.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.861+0000 7f297bfff700 1 -- 172.21.15.12:0/3751092441 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f297401d5e0 con 0x7f2984069550 2024-05-08T23:51:25.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.862+0000 7f297bfff700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29700a1290 0x7f29700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.862+0000 7f297bfff700 1 -- 172.21.15.12:0/3751092441 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f29740cdeb0 con 0x7f2984069550 2024-05-08T23:51:25.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.862+0000 7f29827fc700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29700a1290 0x7f29700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.863+0000 7f29827fc700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29700a1290 0x7f29700a3750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2984198990 tx=0x7f296c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:25.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.863+0000 7f2995491b80 1 Processor -- start 2024-05-08T23:51:25.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.863+0000 7f2995491b80 1 -- start start 2024-05-08T23:51:25.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.863+0000 7f2995491b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f8d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.863+0000 7f2995491b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.863+0000 7f2995491b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f87cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.863+0000 7f2995491b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62ede150 con 0x55ea62f87890 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.863+0000 7f2995491b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62edbb10 con 0x55ea62f8ce00 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.863+0000 7f2995491b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62ed7400 con 0x55ea62f8d720 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f2982ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f8d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f29837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f87cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f29827fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f2982ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f8d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48994/0 (socket says 172.21.15.12:48994) 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f2982ffd700 1 -- 172.21.15.12:0/1900211808 learned_addr learned my addr 172.21.15.12:0/1900211808 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f2982ffd700 1 -- 172.21.15.12:0/1900211808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 msgr2=0x55ea62f87350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f2982ffd700 1 --2- 172.21.15.12:0/1900211808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f87350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f2982ffd700 1 -- 172.21.15.12:0/1900211808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 msgr2=0x55ea62f87cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f2982ffd700 1 --2- 172.21.15.12:0/1900211808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f87cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f2982ffd700 1 -- 172.21.15.12:0/1900211808 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea62d583b0 con 0x55ea62f8ce00 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f2982ffd700 1 --2- 172.21.15.12:0/1900211808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f8d1e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f297409b020 tx=0x7f297400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87cfaf3708b421c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f29797fa700 1 -- 172.21.15.12:0/1900211808 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297409b790 con 0x55ea62f8ce00 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f29797fa700 1 -- 172.21.15.12:0/1900211808 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f297409b090 con 0x55ea62f8ce00 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.864+0000 7f29797fa700 1 -- 172.21.15.12:0/1900211808 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29740cba30 con 0x55ea62f8ce00 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.865+0000 7f2995491b80 1 -- 172.21.15.12:0/1900211808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 msgr2=0x55ea62f8d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.865+0000 7f2995491b80 1 --2- 172.21.15.12:0/1900211808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f8d1e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f297409b020 tx=0x7f297400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.865+0000 7f2995491b80 1 -- 172.21.15.12:0/1900211808 shutdown_connections 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.865+0000 7f2995491b80 1 --2- 172.21.15.12:0/1900211808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f87cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.865+0000 7f2995491b80 1 --2- 172.21.15.12:0/1900211808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f8d1e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.865+0000 7f2995491b80 1 --2- 172.21.15.12:0/1900211808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f87350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.865+0000 7f2995491b80 1 -- 172.21.15.12:0/1900211808 >> 172.21.15.12:0/1900211808 conn(0x55ea62ee5210 msgr2=0x55ea62f81df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.865+0000 7f2995491b80 1 -- 172.21.15.12:0/1900211808 shutdown_connections 2024-05-08T23:51:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.865+0000 7f2995491b80 1 -- 172.21.15.12:0/1900211808 wait complete. 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2995491b80 1 -- start start 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2995491b80 1 Processor -- start 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2995491b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f87890 0x55ea62f90aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2995491b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f90fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2995491b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f8d720 0x55ea62f91520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2995491b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62e73560 con 0x55ea62f8d720 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2995491b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62e73200 con 0x55ea62f8ce00 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2995491b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62d5a6a0 con 0x55ea62f87890 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2995491b80 10 client.0 _ll_register_callbacks cb 0x55ea62e7cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f29827fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f90fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f29837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f8d720 0x55ea62f91520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2982ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f87890 0x55ea62f90aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f29837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f8d720 0x55ea62f91520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53170/0 (socket says 172.21.15.12:53170) 2024-05-08T23:51:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f29837fe700 1 -- 172.21.15.12:0/1812845809 learned_addr learned my addr 172.21.15.12:0/1812845809 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f2982ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f87890 0x55ea62f90aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33466/0 (socket says 172.21.15.12:33466) 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.866+0000 7f29827fc700 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f90fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48998/0 (socket says 172.21.15.12:48998) 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2982ffd700 1 -- 172.21.15.12:0/1812845809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 msgr2=0x55ea62f90fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2982ffd700 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f90fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2982ffd700 1 -- 172.21.15.12:0/1812845809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f8d720 msgr2=0x55ea62f91520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2982ffd700 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f8d720 0x55ea62f91520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2982ffd700 1 -- 172.21.15.12:0/1812845809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29841a41b0 con 0x55ea62f87890 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f29837fe700 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f8d720 0x55ea62f91520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f29827fc700 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f90fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2982ffd700 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f87890 0x55ea62f90aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f297409ad40 tx=0x7f297409b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14946ae6659b30e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f296a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f296a7fc700 1 -- 172.21.15.12:0/1812845809 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29740a3480 con 0x55ea62f87890 2024-05-08T23:51:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f296a7fc700 1 -- 172.21.15.12:0/1812845809 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f29740977a0 con 0x55ea62f87890 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2995491b80 1 -- 172.21.15.12:0/1812845809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea62e30270 con 0x55ea62f87890 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f296a7fc700 1 -- 172.21.15.12:0/1812845809 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2974097980 con 0x55ea62f87890 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2995491b80 10 client.26551 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2995491b80 1 -- 172.21.15.12:0/1812845809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ea62d04b60 con 0x55ea62f87890 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f296a7fc700 1 -- 172.21.15.12:0/1812845809 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea62e30270 con 0x55ea62f87890 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f296a7fc700 1 -- 172.21.15.12:0/1812845809 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f297409a580 con 0x55ea62f87890 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2995491b80 10 client.26551 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2995491b80 10 client.26551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.867+0000 7f2995491b80 20 client.26551 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.868+0000 7f2995491b80 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ea62fa0ad0 0x55ea62fa2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.868+0000 7f2995491b80 4 client.26551 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.868+0000 7f2995491b80 1 -- 172.21.15.12:0/1812845809 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea62f56810 con 0x55ea62fa0ad0 2024-05-08T23:51:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.868+0000 7f29827fc700 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ea62fa0ad0 0x55ea62fa2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 263092 11.10 MB/sec execute 81 sec latency 47.647 ms 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.868+0000 7f296a7fc700 10 client.26551 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.868+0000 7f29827fc700 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ea62fa0ad0 0x55ea62fa2ef0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43b893aa86ec3ad9 server_cookie=635a20c6d6d4bb08 in_seq=0 out_seq=0 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f296a7fc700 1 -- 172.21.15.12:0/1812845809 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea62f56810 con 0x55ea62fa0ad0 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f296a7fc700 10 client.26551 handle_command_reply: tid=0 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 client.26551 shutdown 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 -- 172.21.15.12:0/1812845809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f87890 msgr2=0x55ea62f90aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f87890 0x55ea62f90aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f297409ad40 tx=0x7f297409b790 comp rx=0 tx=0).stop 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 -- 172.21.15.12:0/1812845809 shutdown_connections 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f8d720 0x55ea62f91520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f87890 0x55ea62f90aa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ea62fa0ad0 0x55ea62fa2ef0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 --2- 172.21.15.12:0/1812845809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f90fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f296a7fc700 0 client.26551 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:25.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 -- 172.21.15.12:0/1812845809 >> 172.21.15.12:0/1812845809 conn(0x55ea62ee5210 msgr2=0x55ea62f815a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:25.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 -- 172.21.15.12:0/1812845809 shutdown_connections 2024-05-08T23:51:25.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 1 -- 172.21.15.12:0/1812845809 wait complete. 2024-05-08T23:51:25.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.870+0000 7f2995491b80 20 client.26551 trim_cache size 0 max 16384 2024-05-08T23:51:25.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2995491b80 1 Processor -- start 2024-05-08T23:51:25.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2995491b80 1 -- start start 2024-05-08T23:51:25.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2995491b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2995491b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2995491b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f91340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2995491b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62f7ee30 con 0x55ea62f87890 2024-05-08T23:51:25.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2995491b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62f82910 con 0x55ea62f8ce00 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2995491b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62fa80f0 con 0x55ea62f8d720 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2982ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2982ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f87350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53174/0 (socket says 172.21.15.12:53174) 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f2982ffd700 1 -- 172.21.15.12:0/569686255 learned_addr learned my addr 172.21.15.12:0/569686255 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f29827fc700 1 --2- 172.21.15.12:0/569686255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.889+0000 7f29837fe700 1 --2- 172.21.15.12:0/569686255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f91340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.890+0000 7f2982ffd700 1 -- 172.21.15.12:0/569686255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 msgr2=0x55ea62f91340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.890+0000 7f2982ffd700 1 --2- 172.21.15.12:0/569686255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f91340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.890+0000 7f2982ffd700 1 -- 172.21.15.12:0/569686255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 msgr2=0x55ea62f81d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.890+0000 7f2982ffd700 1 --2- 172.21.15.12:0/569686255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f81d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.890+0000 7f2982ffd700 1 -- 172.21.15.12:0/569686255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea62e9d560 con 0x55ea62f87890 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.890+0000 7f2982ffd700 1 --2- 172.21.15.12:0/569686255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f87350 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f29740993e0 tx=0x7f2974009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e7cbf660632d78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.890+0000 7f2979ffb700 1 -- 172.21.15.12:0/569686255 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297409b800 con 0x55ea62f87890 2024-05-08T23:51:25.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.890+0000 7f2979ffb700 1 -- 172.21.15.12:0/569686255 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29740a3480 con 0x55ea62f87890 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.890+0000 7f2979ffb700 1 -- 172.21.15.12:0/569686255 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297409a580 con 0x55ea62f87890 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 -- 172.21.15.12:0/569686255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 msgr2=0x55ea62f87350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 --2- 172.21.15.12:0/569686255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f87350 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f29740993e0 tx=0x7f2974009b80 comp rx=0 tx=0).stop 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 -- 172.21.15.12:0/569686255 shutdown_connections 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 --2- 172.21.15.12:0/569686255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f91340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 --2- 172.21.15.12:0/569686255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f87350 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 --2- 172.21.15.12:0/569686255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f81d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 -- 172.21.15.12:0/569686255 >> 172.21.15.12:0/569686255 conn(0x55ea62ee5210 msgr2=0x55ea62f8f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 -- 172.21.15.12:0/569686255 shutdown_connections 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 -- 172.21.15.12:0/569686255 wait complete. 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 -- start start 2024-05-08T23:51:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.891+0000 7f2995491b80 1 Processor -- start 2024-05-08T23:51:25.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2995491b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f98160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2995491b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f95f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2995491b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f96490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2995491b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62f98830 con 0x55ea62f87890 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2995491b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62f969d0 con 0x55ea62f8ce00 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2995491b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea62f96b90 con 0x55ea62f8d720 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2995491b80 10 client.0 _ll_register_callbacks cb 0x55ea62e7cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2982ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f98160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2982ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f98160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53180/0 (socket says 172.21.15.12:53180) 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f29827fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f95f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2982ffd700 1 -- 172.21.15.12:0/2131465338 learned_addr learned my addr 172.21.15.12:0/2131465338 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f29837fe700 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f96490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f29837fe700 1 -- 172.21.15.12:0/2131465338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 msgr2=0x55ea62f95f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:25.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f29837fe700 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f95f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f29837fe700 1 -- 172.21.15.12:0/2131465338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 msgr2=0x55ea62f98160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f29837fe700 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f98160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f29837fe700 1 -- 172.21.15.12:0/2131465338 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea62d04b60 con 0x55ea62f8d720 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f29827fc700 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f95f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f2982ffd700 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f98160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f29837fe700 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f96490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f297c0096d0 tx=0x7f297c00f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a614615c6fb931c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.892+0000 7f296a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f296a7fc700 1 -- 172.21.15.12:0/2131465338 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c021070 con 0x55ea62f8d720 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f2995491b80 1 -- 172.21.15.12:0/2131465338 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f297409b090 con 0x55ea62f8d720 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f296a7fc700 1 -- 172.21.15.12:0/2131465338 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f297c00fc40 con 0x55ea62f8d720 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f296a7fc700 1 -- 172.21.15.12:0/2131465338 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f297c017040 con 0x55ea62f8d720 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f2995491b80 10 client.26557 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f2995491b80 1 -- 172.21.15.12:0/2131465338 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ea62d5ccd0 con 0x55ea62f8d720 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f296a7fc700 1 -- 172.21.15.12:0/2131465338 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f297409b090 con 0x55ea62f8d720 2024-05-08T23:51:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f296a7fc700 1 -- 172.21.15.12:0/2131465338 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f297c024030 con 0x55ea62f8d720 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f2995491b80 10 client.26557 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f2995491b80 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f2995491b80 20 client.26557 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f2995491b80 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ea62ff7390 0x55ea62ff97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f2995491b80 4 client.26557 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.893+0000 7f2995491b80 1 -- 172.21.15.12:0/2131465338 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ea62ecfec0 con 0x55ea62ff7390 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.894+0000 7f2982ffd700 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ea62ff7390 0x55ea62ff97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.894+0000 7f296a7fc700 10 client.26557 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.894+0000 7f2982ffd700 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ea62ff7390 0x55ea62ff97b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eea234b7fbe9a28 server_cookie=c09bcd6767286ad4 in_seq=0 out_seq=0 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.895+0000 7f296a7fc700 1 -- 172.21.15.12:0/2131465338 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ea62ecfec0 con 0x55ea62ff7390 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.895+0000 7f296a7fc700 10 client.26557 handle_command_reply: tid=0 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.895+0000 7f2995491b80 1 client.26557 shutdown 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 1 -- 172.21.15.12:0/2131465338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 msgr2=0x55ea62f96490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f96490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f297c0096d0 tx=0x7f297c00f6a0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 1 -- 172.21.15.12:0/2131465338 shutdown_connections 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea62f8d720 0x55ea62f96490 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea62f87890 0x55ea62f98160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ea62ff7390 0x55ea62ff97b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 1 --2- 172.21.15.12:0/2131465338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea62f8ce00 0x55ea62f95f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 1 -- 172.21.15.12:0/2131465338 >> 172.21.15.12:0/2131465338 conn(0x55ea62ee5210 msgr2=0x55ea62f8bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f296a7fc700 0 client.26557 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 1 -- 172.21.15.12:0/2131465338 shutdown_connections 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 1 -- 172.21.15.12:0/2131465338 wait complete. 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f2995491b80 20 client.26557 trim_cache size 0 max 16384 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:51:25.895 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:51:25.896 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:51:25.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f29892d7700 1 -- 172.21.15.12:0/3751092441 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29700a1290 msgr2=0x7f29700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:25.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f29892d7700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29700a1290 0x7f29700a3750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2984198990 tx=0x7f296c009260 comp rx=0 tx=0).stop 2024-05-08T23:51:25.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f29892d7700 1 -- 172.21.15.12:0/3751092441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2984069550 msgr2=0x7f2984197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:25.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f29892d7700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2984069550 0x7f2984197440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f297400b070 tx=0x7f29740078e0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f29892d7700 1 -- 172.21.15.12:0/3751092441 shutdown_connections 2024-05-08T23:51:25.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f29892d7700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29700a1290 0x7f29700a3750 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f29892d7700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2984107060 0x7f298419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f29892d7700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2984069550 0x7f2984197440 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f29892d7700 1 --2- 172.21.15.12:0/3751092441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2984069f10 0x7f2984197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.896+0000 7f29892d7700 1 -- 172.21.15.12:0/3751092441 >> 172.21.15.12:0/3751092441 conn(0x7f2984076f10 msgr2=0x7f29840feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.897+0000 7f29892d7700 1 -- 172.21.15.12:0/3751092441 shutdown_connections 2024-05-08T23:51:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:25.897+0000 7f29892d7700 1 -- 172.21.15.12:0/3751092441 wait complete. 2024-05-08T23:51:25.905 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:51:25.905 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:51:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.058+0000 7f603f74f700 1 -- 172.21.15.12:0/2807504968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6038105070 msgr2=0x7f603810cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.058+0000 7f603f74f700 1 --2- 172.21.15.12:0/2807504968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6038105070 0x7f603810cbf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f603000b0a0 tx=0x7f603000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.058+0000 7f603f74f700 1 -- 172.21.15.12:0/2807504968 shutdown_connections 2024-05-08T23:51:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.058+0000 7f603f74f700 1 --2- 172.21.15.12:0/2807504968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6038105070 0x7f603810cbf0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.058+0000 7f603f74f700 1 --2- 172.21.15.12:0/2807504968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60380fff50 0x7f6038100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.058+0000 7f603f74f700 1 --2- 172.21.15.12:0/2807504968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6038100900 0x7f6038104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.058+0000 7f603f74f700 1 -- 172.21.15.12:0/2807504968 >> 172.21.15.12:0/2807504968 conn(0x7f60380fb7f0 msgr2=0x7f60380fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.058+0000 7f603f74f700 1 -- 172.21.15.12:0/2807504968 shutdown_connections 2024-05-08T23:51:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.058+0000 7f603f74f700 1 -- 172.21.15.12:0/2807504968 wait complete. 2024-05-08T23:51:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603f74f700 1 Processor -- start 2024-05-08T23:51:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603f74f700 1 -- start start 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603f74f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60380fff50 0x7f6038197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603f74f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6038105070 0x7f6038197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603f74f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f603819bcf0 0x7f603819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603f74f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603819c770 con 0x7f60380fff50 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603f74f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603819c8e0 con 0x7f6038105070 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603f74f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603819caa0 con 0x7f603819bcf0 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603d4eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60380fff50 0x7f6038197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603d4eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60380fff50 0x7f6038197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53204/0 (socket says 172.21.15.12:53204) 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603d4eb700 1 -- 172.21.15.12:0/3528273732 learned_addr learned my addr 172.21.15.12:0/3528273732 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.059+0000 7f603ccea700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6038105070 0x7f6038197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f603dcec700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f603819bcf0 0x7f603819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f603d4eb700 1 -- 172.21.15.12:0/3528273732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f603819bcf0 msgr2=0x7f603819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f603d4eb700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f603819bcf0 0x7f603819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f603d4eb700 1 -- 172.21.15.12:0/3528273732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6038105070 msgr2=0x7f6038197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f603d4eb700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6038105070 0x7f6038197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f603d4eb700 1 -- 172.21.15.12:0/3528273732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6030009d00 con 0x7f60380fff50 2024-05-08T23:51:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f603dcec700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f603819bcf0 0x7f603819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:51:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f603d4eb700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60380fff50 0x7f6038197410 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f603400e9a0 tx=0x7f603400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f602e7fc700 1 -- 172.21.15.12:0/3528273732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034009790 con 0x7f60380fff50 2024-05-08T23:51:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f602e7fc700 1 -- 172.21.15.12:0/3528273732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f603400cbd0 con 0x7f60380fff50 2024-05-08T23:51:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f603f74f700 1 -- 172.21.15.12:0/3528273732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60381a3fa0 con 0x7f60380fff50 2024-05-08T23:51:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f602e7fc700 1 -- 172.21.15.12:0/3528273732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034018680 con 0x7f60380fff50 2024-05-08T23:51:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.060+0000 7f603f74f700 1 -- 172.21.15.12:0/3528273732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60381a4480 con 0x7f60380fff50 2024-05-08T23:51:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.061+0000 7f603f74f700 1 -- 172.21.15.12:0/3528273732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6038109350 con 0x7f60380fff50 2024-05-08T23:51:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.063+0000 7f602e7fc700 1 -- 172.21.15.12:0/3528273732 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f603400cd40 con 0x7f60380fff50 2024-05-08T23:51:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.063+0000 7f602e7fc700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f60240a1290 0x7f60240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.063+0000 7f603ccea700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f60240a1290 0x7f60240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.063+0000 7f602e7fc700 1 -- 172.21.15.12:0/3528273732 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6034014070 con 0x7f60380fff50 2024-05-08T23:51:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.064+0000 7f603ccea700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f60240a1290 0x7f60240a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6038198960 tx=0x7f6028009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.068+0000 7f602e7fc700 1 -- 172.21.15.12:0/3528273732 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6034096690 con 0x7f60380fff50 2024-05-08T23:51:26.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.362+0000 7f603f74f700 1 -- 172.21.15.12:0/3528273732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60381987b0 con 0x7f60380fff50 2024-05-08T23:51:26.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.363+0000 7f602e7fc700 1 -- 172.21.15.12:0/3528273732 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10873 (secure 0 0 0) 0x7f6034099cb0 con 0x7f60380fff50 2024-05-08T23:51:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:51:26.359 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:50:48.465221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_26224":{"gid":26224,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/131208281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":131208281},{"type":"v1","addr":"172.21.15.204:6837","nonce":131208281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:51:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 -- 172.21.15.12:0/3528273732 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f60240a1290 msgr2=0x7f60240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f60240a1290 0x7f60240a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6038198960 tx=0x7f6028009260 comp rx=0 tx=0).stop 2024-05-08T23:51:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 -- 172.21.15.12:0/3528273732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60380fff50 msgr2=0x7f6038197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60380fff50 0x7f6038197410 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f603400e9a0 tx=0x7f603400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 -- 172.21.15.12:0/3528273732 shutdown_connections 2024-05-08T23:51:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f60240a1290 0x7f60240a3750 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6038198960 tx=0x7f6028009260 comp rx=0 tx=0).stop 2024-05-08T23:51:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f603819bcf0 0x7f603819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60380fff50 0x7f6038197410 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 --2- 172.21.15.12:0/3528273732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6038105070 0x7f6038197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 -- 172.21.15.12:0/3528273732 >> 172.21.15.12:0/3528273732 conn(0x7f60380fb7f0 msgr2=0x7f6038069750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.368+0000 7f603f74f700 1 -- 172.21.15.12:0/3528273732 shutdown_connections 2024-05-08T23:51:26.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.369+0000 7f603f74f700 1 -- 172.21.15.12:0/3528273732 wait complete. 2024-05-08T23:51:26.366 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 31 2024-05-08T23:51:26.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:51:26.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.549+0000 7fe9a48ac700 1 -- 172.21.15.12:0/27854601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a0105390 msgr2=0x7fe9a010cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.549+0000 7fe9a48ac700 1 --2- 172.21.15.12:0/27854601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a0105390 0x7fe9a010cf10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe99800b0a0 tx=0x7fe99800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.549+0000 7fe9a48ac700 1 -- 172.21.15.12:0/27854601 shutdown_connections 2024-05-08T23:51:26.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.549+0000 7fe9a48ac700 1 --2- 172.21.15.12:0/27854601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a0105390 0x7fe9a010cf10 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.549+0000 7fe9a48ac700 1 --2- 172.21.15.12:0/27854601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9a0100790 0x7fe9a0100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.549+0000 7fe9a48ac700 1 --2- 172.21.15.12:0/27854601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0101140 0x7fe9a0104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.549+0000 7fe9a48ac700 1 -- 172.21.15.12:0/27854601 >> 172.21.15.12:0/27854601 conn(0x7fe9a00763a0 msgr2=0x7fe9a00fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.550+0000 7fe9a48ac700 1 -- 172.21.15.12:0/27854601 shutdown_connections 2024-05-08T23:51:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.550+0000 7fe9a48ac700 1 -- 172.21.15.12:0/27854601 wait complete. 2024-05-08T23:51:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.550+0000 7fe9a48ac700 1 Processor -- start 2024-05-08T23:51:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe9a48ac700 1 -- start start 2024-05-08T23:51:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe9a48ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0100790 0x7fe9a01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe9a48ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a0105390 0x7fe9a0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe9a48ac700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9a019bfc0 0x7fe9a019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe9a48ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a019ca40 con 0x7fe9a0105390 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe9a48ac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a019cbb0 con 0x7fe9a0100790 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe9a48ac700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a019cd70 con 0x7fe9a019bfc0 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a0105390 0x7fe9a0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0100790 0x7fe9a01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a0105390 0x7fe9a0197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53228/0 (socket says 172.21.15.12:53228) 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99dd9b700 1 -- 172.21.15.12:0/4166869700 learned_addr learned my addr 172.21.15.12:0/4166869700 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99ed9d700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9a019bfc0 0x7fe9a019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99dd9b700 1 -- 172.21.15.12:0/4166869700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9a019bfc0 msgr2=0x7fe9a019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9a019bfc0 0x7fe9a019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99dd9b700 1 -- 172.21.15.12:0/4166869700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0100790 msgr2=0x7fe9a01976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0100790 0x7fe9a01976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99dd9b700 1 -- 172.21.15.12:0/4166869700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe998009d00 con 0x7fe9a0105390 2024-05-08T23:51:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99e59c700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0100790 0x7fe9a01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.551+0000 7fe99ed9d700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9a019bfc0 0x7fe9a019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.552+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a0105390 0x7fe9a0197c20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe99400e970 tx=0x7fe99400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.552+0000 7fe98f7fe700 1 -- 172.21.15.12:0/4166869700 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994009790 con 0x7fe9a0105390 2024-05-08T23:51:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.552+0000 7fe98f7fe700 1 -- 172.21.15.12:0/4166869700 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe99400c7e0 con 0x7fe9a0105390 2024-05-08T23:51:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.552+0000 7fe9a48ac700 1 -- 172.21.15.12:0/4166869700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a01a42d0 con 0x7fe9a0105390 2024-05-08T23:51:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.552+0000 7fe98f7fe700 1 -- 172.21.15.12:0/4166869700 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940054e0 con 0x7fe9a0105390 2024-05-08T23:51:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.552+0000 7fe9a48ac700 1 -- 172.21.15.12:0/4166869700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a01a47b0 con 0x7fe9a0105390 2024-05-08T23:51:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.554+0000 7fe98f7fe700 1 -- 172.21.15.12:0/4166869700 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe99400c950 con 0x7fe9a0105390 2024-05-08T23:51:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.555+0000 7fe98f7fe700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9880a1290 0x7fe9880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.555+0000 7fe98f7fe700 1 -- 172.21.15.12:0/4166869700 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe994014070 con 0x7fe9a0105390 2024-05-08T23:51:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.555+0000 7fe99e59c700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9880a1290 0x7fe9880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.556+0000 7fe99e59c700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9880a1290 0x7fe9880a3750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe990011490 tx=0x7fe990009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.556+0000 7fe9b0a66b80 1 Processor -- start 2024-05-08T23:51:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.556+0000 7fe9b0a66b80 1 -- start start 2024-05-08T23:51:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.556+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.556+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d8720 0x5587c56d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.556+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d2890 0x5587c56d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.556+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c5629150 con 0x5587c56d8720 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c5626b10 con 0x5587c56d7e00 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c5622400 con 0x5587c56d2890 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56d81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d8720 0x5587c56d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56d81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49066/0 (socket says 172.21.15.12:49066) 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99e59c700 1 -- 172.21.15.12:0/2250286868 learned_addr learned my addr 172.21.15.12:0/2250286868 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99ed9d700 1 --2- 172.21.15.12:0/2250286868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d2890 0x5587c56d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99e59c700 1 -- 172.21.15.12:0/2250286868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d2890 msgr2=0x5587c56d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99e59c700 1 --2- 172.21.15.12:0/2250286868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d2890 0x5587c56d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99e59c700 1 -- 172.21.15.12:0/2250286868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d8720 msgr2=0x5587c56d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99e59c700 1 --2- 172.21.15.12:0/2250286868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d8720 0x5587c56d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99e59c700 1 -- 172.21.15.12:0/2250286868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587c54a33b0 con 0x5587c56d7e00 2024-05-08T23:51:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/2250286868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d8720 0x5587c56d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe99e59c700 1 --2- 172.21.15.12:0/2250286868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56d81e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe9900043c0 tx=0x7fe990014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9aa50aaf28631c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe98cff9700 1 -- 172.21.15.12:0/2250286868 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990003d10 con 0x5587c56d7e00 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.557+0000 7fe98cff9700 1 -- 172.21.15.12:0/2250286868 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe990003e70 con 0x5587c56d7e00 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.558+0000 7fe98cff9700 1 -- 172.21.15.12:0/2250286868 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99001e7d0 con 0x5587c56d7e00 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.558+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/2250286868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 msgr2=0x5587c56d81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.558+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/2250286868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56d81e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe9900043c0 tx=0x7fe990014000 comp rx=0 tx=0).stop 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.558+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/2250286868 shutdown_connections 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.558+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/2250286868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d2890 0x5587c56d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.558+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/2250286868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56d81e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.558+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/2250286868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d8720 0x5587c56d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.558+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/2250286868 >> 172.21.15.12:0/2250286868 conn(0x5587c5630210 msgr2=0x5587c56ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.558+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/2250286868 shutdown_connections 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.558+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/2250286868 wait complete. 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.559+0000 7fe9b0a66b80 1 -- start start 2024-05-08T23:51:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.559+0000 7fe9b0a66b80 1 Processor -- start 2024-05-08T23:51:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.559+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d2890 0x5587c56dba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.559+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56dbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.559+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.559+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c55be560 con 0x5587c56d2890 2024-05-08T23:51:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.559+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c55be200 con 0x5587c56d7e00 2024-05-08T23:51:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.559+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c54a56a0 con 0x5587c56d8720 2024-05-08T23:51:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.559+0000 7fe9b0a66b80 10 client.0 _ll_register_callbacks cb 0x5587c55c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe99dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56dbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe99dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56dbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49078/0 (socket says 172.21.15.12:49078) 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe99dd9b700 1 -- 172.21.15.12:0/795296915 learned_addr learned my addr 172.21.15.12:0/795296915 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe99ed9d700 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56dc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe99dd9b700 1 -- 172.21.15.12:0/795296915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 msgr2=0x5587c56dc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56dc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe99dd9b700 1 -- 172.21.15.12:0/795296915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d2890 msgr2=0x5587c56dba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d2890 0x5587c56dba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe99dd9b700 1 -- 172.21.15.12:0/795296915 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a01a47b0 con 0x5587c56d7e00 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56dbf90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9940ca6a0 tx=0x7fe99409a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29fdb427f68739ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe985ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe985ffb700 1 -- 172.21.15.12:0/795296915 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99409a210 con 0x5587c56d7e00 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe985ffb700 1 -- 172.21.15.12:0/795296915 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe99409a370 con 0x5587c56d7e00 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.560+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/795296915 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587c557b270 con 0x5587c56d7e00 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe985ffb700 1 -- 172.21.15.12:0/795296915 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9940b8250 con 0x5587c56d7e00 2024-05-08T23:51:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe9b0a66b80 10 client.26582 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/795296915 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fe990017070 con 0x5587c56d7e00 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe985ffb700 1 -- 172.21.15.12:0/795296915 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5587c557b270 con 0x5587c56d7e00 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe985ffb700 1 -- 172.21.15.12:0/795296915 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe9940a2010 con 0x5587c56d7e00 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe9b0a66b80 10 client.26582 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe9b0a66b80 10 client.26582 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe9b0a66b80 20 client.26582 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5587c56f4920 0x5587c56f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe9b0a66b80 4 client.26582 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/795296915 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587c56a1810 con 0x5587c56f4920 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.561+0000 7fe99e59c700 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5587c56f4920 0x5587c56f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.562+0000 7fe985ffb700 10 client.26582 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.562+0000 7fe99e59c700 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5587c56f4920 0x5587c56f6d40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=854b988b36eb8d73 server_cookie=dff19fc960a087f in_seq=0 out_seq=0 2024-05-08T23:51:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.562+0000 7fe985ffb700 1 -- 172.21.15.12:0/795296915 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587c56a1810 con 0x5587c56f4920 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.562+0000 7fe985ffb700 10 client.26582 handle_command_reply: tid=0 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 client.26582 shutdown 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/795296915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 msgr2=0x5587c56dbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56dbf90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9940ca6a0 tx=0x7fe99409a060 comp rx=0 tx=0).stop 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/795296915 shutdown_connections 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5587c56f4920 0x5587c56f6d40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56dc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d2890 0x5587c56dba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/795296915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d7e00 0x5587c56dbf90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe985ffb700 0 client.26582 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/795296915 >> 172.21.15.12:0/795296915 conn(0x5587c5630210 msgr2=0x5587c56cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/795296915 shutdown_connections 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/795296915 wait complete. 2024-05-08T23:51:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.563+0000 7fe9b0a66b80 20 client.26582 trim_cache size 0 max 16384 2024-05-08T23:51:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.578+0000 7fe9b0a66b80 1 Processor -- start 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.578+0000 7fe9b0a66b80 1 -- start start 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.578+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 0x5587c56d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.578+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.578+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56dbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.578+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c56de5c0 con 0x5587c56d7e00 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.578+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c56dec40 con 0x5587c56d2890 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.578+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c56c9dc0 con 0x5587c56d8720 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 0x5587c56d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53256/0 (socket says 172.21.15.12:53256) 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99dd9b700 1 -- 172.21.15.12:0/928732965 learned_addr learned my addr 172.21.15.12:0/928732965 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 0x5587c56d2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49080/0 (socket says 172.21.15.12:49080) 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99ed9d700 1 --2- 172.21.15.12:0/928732965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56dbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99dd9b700 1 -- 172.21.15.12:0/928732965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 msgr2=0x5587c56dbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/928732965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56dbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99dd9b700 1 -- 172.21.15.12:0/928732965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 msgr2=0x5587c56d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/928732965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 0x5587c56d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99dd9b700 1 -- 172.21.15.12:0/928732965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587c544fb60 con 0x5587c56d7e00 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99e59c700 1 --2- 172.21.15.12:0/928732965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 0x5587c56d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99ed9d700 1 --2- 172.21.15.12:0/928732965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56dbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/928732965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56ccd20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe994099ef0 tx=0x7fe994098560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c769166a6d4b3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe98d7fa700 1 -- 172.21.15.12:0/928732965 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994094db0 con 0x5587c56d7e00 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe98d7fa700 1 -- 172.21.15.12:0/928732965 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9940169f0 con 0x5587c56d7e00 2024-05-08T23:51:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.579+0000 7fe98d7fa700 1 -- 172.21.15.12:0/928732965 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994016f00 con 0x5587c56d7e00 2024-05-08T23:51:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.580+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/928732965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 msgr2=0x5587c56ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.580+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/928732965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56ccd20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe994099ef0 tx=0x7fe994098560 comp rx=0 tx=0).stop 2024-05-08T23:51:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.580+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/928732965 shutdown_connections 2024-05-08T23:51:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.580+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/928732965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56dbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.580+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/928732965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 0x5587c56d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.580+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/928732965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56ccd20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.580+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/928732965 >> 172.21.15.12:0/928732965 conn(0x5587c5630210 msgr2=0x5587c56d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.580+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/928732965 shutdown_connections 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.580+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/928732965 wait complete. 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe9b0a66b80 1 -- start start 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe9b0a66b80 1 Processor -- start 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 0x5587c56e1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe9b0a66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56e1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c5749e00 con 0x5587c56d7e00 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c5749f40 con 0x5587c56d2890 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe9b0a66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c574a080 con 0x5587c56d8720 2024-05-08T23:51:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe9b0a66b80 10 client.0 _ll_register_callbacks cb 0x5587c55c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe99dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe99dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56e1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53272/0 (socket says 172.21.15.12:53272) 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe99dd9b700 1 -- 172.21.15.12:0/1590776167 learned_addr learned my addr 172.21.15.12:0/1590776167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe99e59c700 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 0x5587c56e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe99ed9d700 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56e1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe99dd9b700 1 -- 172.21.15.12:0/1590776167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 msgr2=0x5587c56e1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56e1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe99dd9b700 1 -- 172.21.15.12:0/1590776167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 msgr2=0x5587c56e1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 0x5587c56e1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.581+0000 7fe99dd9b700 1 -- 172.21.15.12:0/1590776167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587c54a33b0 con 0x5587c56d7e00 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe99dd9b700 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56e1690 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe994099f40 tx=0x7fe99409a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d7c301712a3204 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe985ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe985ffb700 1 -- 172.21.15.12:0/1590776167 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994094db0 con 0x5587c56d7e00 2024-05-08T23:51:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe985ffb700 1 -- 172.21.15.12:0/1590776167 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe99409a210 con 0x5587c56d7e00 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/1590776167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe990003e70 con 0x5587c56d7e00 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe985ffb700 1 -- 172.21.15.12:0/1590776167 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe99409a3d0 con 0x5587c56d7e00 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe9b0a66b80 10 client.16803 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/1590776167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fe9a010d780 con 0x5587c56d7e00 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe985ffb700 1 -- 172.21.15.12:0/1590776167 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe990003e70 con 0x5587c56d7e00 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe985ffb700 1 -- 172.21.15.12:0/1590776167 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe994095da0 con 0x5587c56d7e00 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe9b0a66b80 10 client.16803 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe9b0a66b80 10 client.16803 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe9b0a66b80 20 client.16803 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5587c574b3a0 0x5587c574d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe9b0a66b80 4 client.16803 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.582+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/1590776167 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5587c561aec0 con 0x5587c574b3a0 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.583+0000 7fe99e59c700 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5587c574b3a0 0x5587c574d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.583+0000 7fe99e59c700 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5587c574b3a0 0x5587c574d7c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c9aa002c495e3fd server_cookie=1216c906e37546f in_seq=0 out_seq=0 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.583+0000 7fe985ffb700 10 client.16803 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.583+0000 7fe985ffb700 1 -- 172.21.15.12:0/1590776167 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587c561aec0 con 0x5587c574b3a0 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.583+0000 7fe985ffb700 10 client.16803 handle_command_reply: tid=0 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.583+0000 7fe9b0a66b80 1 client.16803 shutdown 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/1590776167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 msgr2=0x5587c56e1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56e1690 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe994099f40 tx=0x7fe99409a060 comp rx=0 tx=0).stop 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/1590776167 shutdown_connections 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5587c574b3a0 0x5587c574d7c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5587c56d8720 0x5587c56e1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe985ffb700 0 client.16803 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5587c56d2890 0x5587c56e1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 1 --2- 172.21.15.12:0/1590776167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c56d7e00 0x5587c56e1690 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/1590776167 >> 172.21.15.12:0/1590776167 conn(0x5587c5630210 msgr2=0x5587c56da300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/1590776167 shutdown_connections 2024-05-08T23:51:26.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 1 -- 172.21.15.12:0/1590776167 wait complete. 2024-05-08T23:51:26.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9b0a66b80 20 client.16803 trim_cache size 0 max 16384 2024-05-08T23:51:26.586 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:51:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9a48ac700 1 -- 172.21.15.12:0/4166869700 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9880a1290 msgr2=0x7fe9880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9a48ac700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9880a1290 0x7fe9880a3750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe990011490 tx=0x7fe990009260 comp rx=0 tx=0).stop 2024-05-08T23:51:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9a48ac700 1 -- 172.21.15.12:0/4166869700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a0105390 msgr2=0x7fe9a0197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.584+0000 7fe9a48ac700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a0105390 0x7fe9a0197c20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe99400e970 tx=0x7fe99400ec80 comp rx=0 tx=0).stop 2024-05-08T23:51:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.585+0000 7fe9a48ac700 1 -- 172.21.15.12:0/4166869700 shutdown_connections 2024-05-08T23:51:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.585+0000 7fe9a48ac700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9880a1290 0x7fe9880a3750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.585+0000 7fe9a48ac700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9a019bfc0 0x7fe9a019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.585+0000 7fe9a48ac700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0100790 0x7fe9a01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.585+0000 7fe9a48ac700 1 --2- 172.21.15.12:0/4166869700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a0105390 0x7fe9a0197c20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.585+0000 7fe9a48ac700 1 -- 172.21.15.12:0/4166869700 >> 172.21.15.12:0/4166869700 conn(0x7fe9a00763a0 msgr2=0x7fe9a00ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.585+0000 7fe9a48ac700 1 -- 172.21.15.12:0/4166869700 shutdown_connections 2024-05-08T23:51:26.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.585+0000 7fe9a48ac700 1 -- 172.21.15.12:0/4166869700 wait complete. 2024-05-08T23:51:26.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:51:26.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:26 smithi012 ceph-mon[42503]: pgmap v155: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-05-08T23:51:26.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:26 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3528273732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:51:26.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.742+0000 7fb7e24c4700 1 -- 172.21.15.12:0/485178108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7dc101180 msgr2=0x7fb7dc101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.742+0000 7fb7e24c4700 1 --2- 172.21.15.12:0/485178108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7dc101180 0x7fb7dc101560 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009990 tx=0x7fb7cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.743+0000 7fb7e24c4700 1 -- 172.21.15.12:0/485178108 shutdown_connections 2024-05-08T23:51:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.743+0000 7fb7e24c4700 1 --2- 172.21.15.12:0/485178108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7dc1062a0 0x7fb7dc10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.743+0000 7fb7e24c4700 1 --2- 172.21.15.12:0/485178108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7dc101180 0x7fb7dc101560 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.743+0000 7fb7e24c4700 1 --2- 172.21.15.12:0/485178108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc101b30 0x7fb7dc105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.743+0000 7fb7e24c4700 1 -- 172.21.15.12:0/485178108 >> 172.21.15.12:0/485178108 conn(0x7fb7dc0fc9e0 msgr2=0x7fb7dc0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.743+0000 7fb7e24c4700 1 -- 172.21.15.12:0/485178108 shutdown_connections 2024-05-08T23:51:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.743+0000 7fb7e24c4700 1 -- 172.21.15.12:0/485178108 wait complete. 2024-05-08T23:51:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.743+0000 7fb7e24c4700 1 Processor -- start 2024-05-08T23:51:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.743+0000 7fb7e24c4700 1 -- start start 2024-05-08T23:51:26.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7e24c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7dc101180 0x7fb7dc1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7e24c4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1062a0 0x7fb7dc197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7e24c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7dc19bca0 0x7fb7dc19c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7e24c4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc19c720 con 0x7fb7dc19bca0 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7e24c4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc19c890 con 0x7fb7dc101180 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7e24c4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc19ca50 con 0x7fb7dc1062a0 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7dc101180 0x7fb7dc1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7e0a61700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7dc19bca0 0x7fb7dc19c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7dc101180 0x7fb7dc1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49110/0 (socket says 172.21.15.12:49110) 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7dbfff700 1 -- 172.21.15.12:0/752694445 learned_addr learned my addr 172.21.15.12:0/752694445 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1062a0 0x7fb7dc197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7dbfff700 1 -- 172.21.15.12:0/752694445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1062a0 msgr2=0x7fb7dc197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1062a0 0x7fb7dc197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7dbfff700 1 -- 172.21.15.12:0/752694445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7dc19bca0 msgr2=0x7fb7dc19c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7dc19bca0 0x7fb7dc19c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7dbfff700 1 -- 172.21.15.12:0/752694445 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d0009520 con 0x7fb7dc101180 2024-05-08T23:51:26.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1062a0 0x7fb7dc197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:51:26.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.744+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7dc101180 0x7fb7dc1973c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc005770 tx=0x7fb7cc0057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.745+0000 7fb7d97fa700 1 -- 172.21.15.12:0/752694445 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc01d070 con 0x7fb7dc101180 2024-05-08T23:51:26.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.745+0000 7fb7e24c4700 1 -- 172.21.15.12:0/752694445 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7cc0095f0 con 0x7fb7dc101180 2024-05-08T23:51:26.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.745+0000 7fb7d97fa700 1 -- 172.21.15.12:0/752694445 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7cc022470 con 0x7fb7dc101180 2024-05-08T23:51:26.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.745+0000 7fb7d97fa700 1 -- 172.21.15.12:0/752694445 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc017650 con 0x7fb7dc101180 2024-05-08T23:51:26.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.745+0000 7fb7e24c4700 1 -- 172.21.15.12:0/752694445 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc1a41e0 con 0x7fb7dc101180 2024-05-08T23:51:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.747+0000 7fb7d97fa700 1 -- 172.21.15.12:0/752694445 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb7cc00fb00 con 0x7fb7dc101180 2024-05-08T23:51:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.748+0000 7fb7d97fa700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb7c40a1310 0x7fb7c40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.748+0000 7fb7d97fa700 1 -- 172.21.15.12:0/752694445 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb7cc0ce0d0 con 0x7fb7dc101180 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.748+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb7c40a1310 0x7fb7c40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.748+0000 7fb7ee67eb80 1 Processor -- start 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.748+0000 7fb7ee67eb80 1 -- start start 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb7c40a1310 0x7fb7c40a37d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc198910 tx=0x7fb7d0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 0x55892d8c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d815150 con 0x55892d8c8b30 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d812b10 con 0x55892d8c8210 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d80e400 con 0x55892d8c2ca0 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53304/0 (socket says 172.21.15.12:53304) 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7db7fe700 1 -- 172.21.15.12:0/2738256421 learned_addr learned my addr 172.21.15.12:0/2738256421 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/2738256421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 0x55892d8c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7e0a61700 1 --2- 172.21.15.12:0/2738256421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7dbfff700 1 -- 172.21.15.12:0/2738256421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 msgr2=0x55892d8c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/2738256421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8c30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7dbfff700 1 -- 172.21.15.12:0/2738256421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 msgr2=0x55892d8c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/2738256421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7dbfff700 1 -- 172.21.15.12:0/2738256421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55892d68f3b0 con 0x55892d8c8210 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/2738256421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.749+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/2738256421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 0x55892d8c85f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0cb800 tx=0x7fb7cc09b210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619deb80072a3f60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.750+0000 7fb7c27fc700 1 -- 172.21.15.12:0/2738256421 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc09b370 con 0x55892d8c8210 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.750+0000 7fb7c27fc700 1 -- 172.21.15.12:0/2738256421 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7cc09f1a0 con 0x55892d8c8210 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.750+0000 7fb7c27fc700 1 -- 172.21.15.12:0/2738256421 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc09b5d0 con 0x55892d8c8210 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.750+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/2738256421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 msgr2=0x55892d8c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.750+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/2738256421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 0x55892d8c85f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0cb800 tx=0x7fb7cc09b210 comp rx=0 tx=0).stop 2024-05-08T23:51:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.750+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/2738256421 shutdown_connections 2024-05-08T23:51:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.750+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/2738256421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8c30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.750+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/2738256421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 0x55892d8c85f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.750+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/2738256421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.750+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/2738256421 >> 172.21.15.12:0/2738256421 conn(0x55892d81c210 msgr2=0x55892d8b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.751+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/2738256421 shutdown_connections 2024-05-08T23:51:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.751+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/2738256421 wait complete. 2024-05-08T23:51:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.751+0000 7fb7ee67eb80 1 -- start start 2024-05-08T23:51:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.751+0000 7fb7ee67eb80 1 Processor -- start 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c2ca0 0x55892d8ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c8b30 0x55892d8cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d7aa560 con 0x55892d8c8210 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d7aa200 con 0x55892d8c2ca0 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d6916a0 con 0x55892d8c8b30 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7ee67eb80 10 client.0 _ll_register_callbacks cb 0x55892d7b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c2ca0 0x55892d8ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c2ca0 0x55892d8ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49132/0 (socket says 172.21.15.12:49132) 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7dbfff700 1 -- 172.21.15.12:0/3669387312 learned_addr learned my addr 172.21.15.12:0/3669387312 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53312/0 (socket says 172.21.15.12:53312) 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7e0a61700 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c8b30 0x55892d8cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7db7fe700 1 -- 172.21.15.12:0/3669387312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c8b30 msgr2=0x55892d8cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c8b30 0x55892d8cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7db7fe700 1 -- 172.21.15.12:0/3669387312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c2ca0 msgr2=0x55892d8ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c2ca0 0x55892d8ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7db7fe700 1 -- 172.21.15.12:0/3669387312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d0011d70 con 0x55892d8c8210 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c2ca0 0x55892d8ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8cbe40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00098c0 tx=0x7fb7d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a1e4a2bacee8aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.752+0000 7fb7bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3669387312 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d001ead0 con 0x55892d8c8210 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3669387312 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb7d0018070 con 0x55892d8c8210 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3669387312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55892d767270 con 0x55892d8c8210 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3669387312 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7d001ec30 con 0x55892d8c8210 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7ee67eb80 10 client.16815 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3669387312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fb7dc1a41e0 con 0x55892d8c8210 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3669387312 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55892d767270 con 0x55892d8c8210 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3669387312 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb7d001f420 con 0x55892d8c8210 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7ee67eb80 10 client.16815 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7ee67eb80 10 client.16815 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7ee67eb80 20 client.16815 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55892d8dc250 0x55892d8de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7ee67eb80 4 client.16815 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.753+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3669387312 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55892d88d810 con 0x55892d8dc250 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.754+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55892d8dc250 0x55892d8de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.754+0000 7fb7bb7fe700 10 client.16815 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.754+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55892d8dc250 0x55892d8de670 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de5be4b24c9ae7ac server_cookie=111785aafa33779d in_seq=0 out_seq=0 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.754+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3669387312 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55892d88d810 con 0x55892d8dc250 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7bb7fe700 10 client.16815 handle_command_reply: tid=0 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 client.16815 shutdown 2024-05-08T23:51:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3669387312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 msgr2=0x55892d8cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8cbe40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00098c0 tx=0x7fb7d00047c0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3669387312 shutdown_connections 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55892d8dc250 0x55892d8de670 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c8b30 0x55892d8cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c2ca0 0x55892d8ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3669387312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8cbe40 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7bb7fe700 0 client.16815 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3669387312 >> 172.21.15.12:0/3669387312 conn(0x55892d81c210 msgr2=0x55892d8b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3669387312 shutdown_connections 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3669387312 wait complete. 2024-05-08T23:51:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.755+0000 7fb7ee67eb80 20 client.16815 trim_cache size 0 max 16384 2024-05-08T23:51:26.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:26 smithi204 ceph-mon[47373]: pgmap v155: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-05-08T23:51:26.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:26 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3528273732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:51:26.771 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:26 smithi160 ceph-mon[46152]: pgmap v155: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-05-08T23:51:26.771 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:26 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3528273732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:51:26.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.769+0000 7fb7ee67eb80 1 Processor -- start 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.769+0000 7fb7ee67eb80 1 -- start start 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.769+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.769+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 0x55892d8e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d8b5dc0 con 0x55892d8c8b30 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d8b98e0 con 0x55892d8c8210 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d8c2760 con 0x55892d8c2ca0 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7e0a61700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7dbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7e0a61700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8d7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53318/0 (socket says 172.21.15.12:53318) 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7e0a61700 1 -- 172.21.15.12:0/1893722117 learned_addr learned my addr 172.21.15.12:0/1893722117 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/1893722117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 0x55892d8e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7dbfff700 1 -- 172.21.15.12:0/1893722117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 msgr2=0x55892d8e2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/1893722117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 0x55892d8e2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7dbfff700 1 -- 172.21.15.12:0/1893722117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 msgr2=0x55892d8d7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/1893722117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8d7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7dbfff700 1 -- 172.21.15.12:0/1893722117 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55892d63bb60 con 0x55892d8c2ca0 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/1893722117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8b8d20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0cb860 tx=0x7fb7cc022c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578bf4c06cb4a18b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7c2ffd700 1 -- 172.21.15.12:0/1893722117 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc09b3f0 con 0x55892d8c2ca0 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7c2ffd700 1 -- 172.21.15.12:0/1893722117 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7cc0b3e90 con 0x55892d8c2ca0 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.770+0000 7fb7c2ffd700 1 -- 172.21.15.12:0/1893722117 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc022870 con 0x55892d8c2ca0 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.771+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/1893722117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 msgr2=0x55892d8b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.771+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/1893722117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8b8d20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0cb860 tx=0x7fb7cc022c40 comp rx=0 tx=0).stop 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.771+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/1893722117 shutdown_connections 2024-05-08T23:51:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.771+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/1893722117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8b30 0x55892d8d7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.771+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/1893722117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8b8d20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.771+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/1893722117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8210 0x55892d8e2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.771+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/1893722117 >> 172.21.15.12:0/1893722117 conn(0x55892d81c210 msgr2=0x55892d8cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.771+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/1893722117 shutdown_connections 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.771+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/1893722117 wait complete. 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.771+0000 7fb7ee67eb80 1 -- start start 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7ee67eb80 1 Processor -- start 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8d3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7ee67eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8b30 0x55892d8c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d8c47d0 con 0x55892d8c8210 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d8c4940 con 0x55892d8c8b30 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7ee67eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892d8c4b00 con 0x55892d8c2ca0 2024-05-08T23:51:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7ee67eb80 10 client.0 _ll_register_callbacks cb 0x55892d7b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8c3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53320/0 (socket says 172.21.15.12:53320) 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7db7fe700 1 -- 172.21.15.12:0/3978073382 learned_addr learned my addr 172.21.15.12:0/3978073382 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7e0a61700 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8b30 0x55892d8c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.772+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8d3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7db7fe700 1 -- 172.21.15.12:0/3978073382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 msgr2=0x55892d8d3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8d3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7db7fe700 1 -- 172.21.15.12:0/3978073382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8b30 msgr2=0x55892d8c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8b30 0x55892d8c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7db7fe700 1 -- 172.21.15.12:0/3978073382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55892d68f3b0 con 0x55892d8c8210 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7e0a61700 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8b30 0x55892d8c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7db7fe700 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8c3c70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb7d001f8d0 tx=0x7fb7d001f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ac155c94af364a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3978073382 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d001e690 con 0x55892d8c8210 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3978073382 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb7d0018040 con 0x55892d8c8210 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3978073382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7cc09f1a0 con 0x55892d8c8210 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3978073382 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7d001f9a0 con 0x55892d8c8210 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7ee67eb80 10 client.16821 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3978073382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55892d7d4560 con 0x55892d8c8210 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.773+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3978073382 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7cc09f1a0 con 0x55892d8c8210 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.774+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3978073382 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb7d001e8d0 con 0x55892d8c8210 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.774+0000 7fb7ee67eb80 10 client.16821 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.774+0000 7fb7ee67eb80 10 client.16821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.774+0000 7fb7ee67eb80 20 client.16821 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.774+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55892d932e30 0x55892d935250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.774+0000 7fb7ee67eb80 4 client.16821 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:51:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.774+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3978073382 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55892d806ec0 con 0x55892d932e30 2024-05-08T23:51:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.774+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55892d932e30 0x55892d935250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.774+0000 7fb7bb7fe700 10 client.16821 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:51:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.774+0000 7fb7dbfff700 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55892d932e30 0x55892d935250 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e97f26aa0f8decc9 server_cookie=b54233504567cde3 in_seq=0 out_seq=0 2024-05-08T23:51:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7bb7fe700 1 -- 172.21.15.12:0/3978073382 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55892d806ec0 con 0x55892d932e30 2024-05-08T23:51:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7bb7fe700 10 client.16821 handle_command_reply: tid=0 2024-05-08T23:51:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 client.16821 shutdown 2024-05-08T23:51:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3978073382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 msgr2=0x55892d8c3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8c3c70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb7d001f8d0 tx=0x7fb7d001f650 comp rx=0 tx=0).stop 2024-05-08T23:51:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3978073382 shutdown_connections 2024-05-08T23:51:26.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55892d932e30 0x55892d935250 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55892d8c8b30 0x55892d8c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7bb7fe700 0 client.16821 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:51:26.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55892d8c2ca0 0x55892d8d3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 --2- 172.21.15.12:0/3978073382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55892d8c8210 0x55892d8c3c70 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3978073382 >> 172.21.15.12:0/3978073382 conn(0x55892d81c210 msgr2=0x55892d8cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3978073382 shutdown_connections 2024-05-08T23:51:26.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 1 -- 172.21.15.12:0/3978073382 wait complete. 2024-05-08T23:51:26.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.775+0000 7fb7ee67eb80 20 client.16821 trim_cache size 0 max 16384 2024-05-08T23:51:26.779 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:51:26.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 -- 172.21.15.12:0/752694445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb7c40a1310 msgr2=0x7fb7c40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb7c40a1310 0x7fb7c40a37d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc198910 tx=0x7fb7d0009260 comp rx=0 tx=0).stop 2024-05-08T23:51:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 -- 172.21.15.12:0/752694445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7dc101180 msgr2=0x7fb7dc1973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7dc101180 0x7fb7dc1973c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc005770 tx=0x7fb7cc0057a0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 -- 172.21.15.12:0/752694445 shutdown_connections 2024-05-08T23:51:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb7c40a1310 0x7fb7c40a37d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7dc19bca0 0x7fb7dc19c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7dc101180 0x7fb7dc1973c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 --2- 172.21.15.12:0/752694445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7dc1062a0 0x7fb7dc197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 -- 172.21.15.12:0/752694445 >> 172.21.15.12:0/752694445 conn(0x7fb7dc0fc9e0 msgr2=0x7fb7dc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 -- 172.21.15.12:0/752694445 shutdown_connections 2024-05-08T23:51:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.776+0000 7fb7e24c4700 1 -- 172.21.15.12:0/752694445 wait complete. 2024-05-08T23:51:26.784 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:51:26.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 265280 11.09 MB/sec execute 82 sec latency 58.120 ms 2024-05-08T23:51:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.931+0000 7fb3bb49f700 1 -- 172.21.15.12:0/3844460069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3b40ff410 msgr2=0x7fb3b40ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.932+0000 7fb3bb49f700 1 --2- 172.21.15.12:0/3844460069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3b40ff410 0x7fb3b40ff890 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009990 tx=0x7fb3a4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.932+0000 7fb3bb49f700 1 -- 172.21.15.12:0/3844460069 shutdown_connections 2024-05-08T23:51:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.932+0000 7fb3bb49f700 1 --2- 172.21.15.12:0/3844460069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b40ffdd0 0x7fb3b410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.932+0000 7fb3bb49f700 1 --2- 172.21.15.12:0/3844460069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b4105ef0 0x7fb3b41062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.932+0000 7fb3bb49f700 1 --2- 172.21.15.12:0/3844460069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3b40ff410 0x7fb3b40ff890 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.932+0000 7fb3bb49f700 1 -- 172.21.15.12:0/3844460069 >> 172.21.15.12:0/3844460069 conn(0x7fb3b40763a0 msgr2=0x7fb3b40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.932+0000 7fb3bb49f700 1 -- 172.21.15.12:0/3844460069 shutdown_connections 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.932+0000 7fb3bb49f700 1 -- 172.21.15.12:0/3844460069 wait complete. 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.932+0000 7fb3bb49f700 1 Processor -- start 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.932+0000 7fb3bb49f700 1 -- start start 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3bb49f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3b40ff410 0x7fb3b4197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3bb49f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b40ffdd0 0x7fb3b4197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3bb49f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b4105ef0 0x7fb3b419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3bb49f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b419c350 con 0x7fb3b40ff410 2024-05-08T23:51:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3bb49f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b419c4c0 con 0x7fb3b40ffdd0 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3bb49f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b419c680 con 0x7fb3b4105ef0 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b923b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3b40ff410 0x7fb3b4197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b923b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3b40ff410 0x7fb3b4197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53348/0 (socket says 172.21.15.12:53348) 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b923b700 1 -- 172.21.15.12:0/2223549251 learned_addr learned my addr 172.21.15.12:0/2223549251 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b40ffdd0 0x7fb3b4197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b9a3c700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b4105ef0 0x7fb3b419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b8a3a700 1 -- 172.21.15.12:0/2223549251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b4105ef0 msgr2=0x7fb3b419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b4105ef0 0x7fb3b419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b8a3a700 1 -- 172.21.15.12:0/2223549251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3b40ff410 msgr2=0x7fb3b4197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3b40ff410 0x7fb3b4197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b8a3a700 1 -- 172.21.15.12:0/2223549251 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40095f0 con 0x7fb3b40ffdd0 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b923b700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3b40ff410 0x7fb3b4197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.933+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b40ffdd0 0x7fb3b4197970 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40059d0 tx=0x7fb3a4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.934+0000 7fb3aa7fc700 1 -- 172.21.15.12:0/2223549251 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a401d070 con 0x7fb3b40ffdd0 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.934+0000 7fb3aa7fc700 1 -- 172.21.15.12:0/2223549251 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3a4022470 con 0x7fb3b40ffdd0 2024-05-08T23:51:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.934+0000 7fb3bb49f700 1 -- 172.21.15.12:0/2223549251 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b419c900 con 0x7fb3b40ffdd0 2024-05-08T23:51:26.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.934+0000 7fb3aa7fc700 1 -- 172.21.15.12:0/2223549251 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4017650 con 0x7fb3b40ffdd0 2024-05-08T23:51:26.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.934+0000 7fb3bb49f700 1 -- 172.21.15.12:0/2223549251 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b41a41a0 con 0x7fb3b40ffdd0 2024-05-08T23:51:26.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.936+0000 7fb3aa7fc700 1 -- 172.21.15.12:0/2223549251 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb3a40225e0 con 0x7fb3b40ffdd0 2024-05-08T23:51:26.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.936+0000 7fb3aa7fc700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3a00a1310 0x7fb3a00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.937+0000 7fb3b923b700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3a00a1310 0x7fb3a00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.937+0000 7fb3aa7fc700 1 -- 172.21.15.12:0/2223549251 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb3a40cdcf0 con 0x7fb3b40ffdd0 2024-05-08T23:51:26.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.937+0000 7fb3b923b700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3a00a1310 0x7fb3a00a37d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0011550 tx=0x7fb3b0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3c7659b80 1 Processor -- start 2024-05-08T23:51:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3c7659b80 1 -- start start 2024-05-08T23:51:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785de00 0x55768785e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785e720 0x557687858350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687858cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576877af150 con 0x55768785de00 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576877acb10 con 0x55768785e720 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576877a8400 con 0x557687858890 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3b923b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785de00 0x55768785e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3b923b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785de00 0x55768785e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53358/0 (socket says 172.21.15.12:53358) 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3b923b700 1 -- 172.21.15.12:0/3307398527 learned_addr learned my addr 172.21.15.12:0/3307398527 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/3307398527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785e720 0x557687858350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.938+0000 7fb3b9a3c700 1 --2- 172.21.15.12:0/3307398527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687858cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.939+0000 7fb3b923b700 1 -- 172.21.15.12:0/3307398527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 msgr2=0x557687858cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.939+0000 7fb3b923b700 1 --2- 172.21.15.12:0/3307398527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687858cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.939+0000 7fb3b923b700 1 -- 172.21.15.12:0/3307398527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785e720 msgr2=0x557687858350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.939+0000 7fb3b923b700 1 --2- 172.21.15.12:0/3307398527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785e720 0x557687858350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.939+0000 7fb3b923b700 1 -- 172.21.15.12:0/3307398527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576876293b0 con 0x55768785de00 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.939+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/3307398527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785e720 0x557687858350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.939+0000 7fb3b923b700 1 --2- 172.21.15.12:0/3307398527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785de00 0x55768785e1e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009880 tx=0x7fb3b0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868fc1b075ada417 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.939+0000 7fb39f7fe700 1 -- 172.21.15.12:0/3307398527 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0026630 con 0x55768785de00 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.939+0000 7fb39f7fe700 1 -- 172.21.15.12:0/3307398527 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3b0021070 con 0x55768785de00 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.939+0000 7fb39f7fe700 1 -- 172.21.15.12:0/3307398527 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b001d670 con 0x55768785de00 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.940+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3307398527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785de00 msgr2=0x55768785e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.940+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3307398527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785de00 0x55768785e1e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009880 tx=0x7fb3b0014000 comp rx=0 tx=0).stop 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.940+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3307398527 shutdown_connections 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.940+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3307398527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687858cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.940+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3307398527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785de00 0x55768785e1e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.940+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3307398527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785e720 0x557687858350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.940+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3307398527 >> 172.21.15.12:0/3307398527 conn(0x5576877b6210 msgr2=0x557687852df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.940+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3307398527 shutdown_connections 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.940+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3307398527 wait complete. 2024-05-08T23:51:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.940+0000 7fb3c7659b80 1 -- start start 2024-05-08T23:51:26.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3c7659b80 1 Processor -- start 2024-05-08T23:51:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687861a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687861f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687862480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557687744560 con 0x55768785e720 2024-05-08T23:51:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557687744200 con 0x55768785de00 2024-05-08T23:51:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768762b6a0 con 0x557687858890 2024-05-08T23:51:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3c7659b80 10 client.0 _ll_register_callbacks cb 0x55768774db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3b9a3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687862480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3b923b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687861a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3b9a3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687862480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53362/0 (socket says 172.21.15.12:53362) 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3b923b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687861a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33654/0 (socket says 172.21.15.12:33654) 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3b9a3c700 1 -- 172.21.15.12:0/3385230364 learned_addr learned my addr 172.21.15.12:0/3385230364 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.941+0000 7fb3b8a3a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687861f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3b9a3c700 1 -- 172.21.15.12:0/3385230364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 msgr2=0x557687861a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3b9a3c700 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687861a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3b9a3c700 1 -- 172.21.15.12:0/3385230364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 msgr2=0x557687861f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3b9a3c700 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687861f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3b9a3c700 1 -- 172.21.15.12:0/3385230364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b41a41a0 con 0x55768785e720 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3b923b700 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687861a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687861f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3b9a3c700 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687862480 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac011f60 tx=0x7fb3ac014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a832f25ab0c1dc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb39cff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb39cff9700 1 -- 172.21.15.12:0/3385230364 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00e040 con 0x55768785e720 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb39cff9700 1 -- 172.21.15.12:0/3385230364 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb3ac014ba0 con 0x55768785e720 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3385230364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557687701270 con 0x55768785e720 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb39cff9700 1 -- 172.21.15.12:0/3385230364 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3ac014d60 con 0x55768785e720 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3c7659b80 10 client.16833 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.942+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3385230364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fb3b0017070 con 0x55768785e720 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb39cff9700 1 -- 172.21.15.12:0/3385230364 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557687701270 con 0x55768785e720 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb39cff9700 1 -- 172.21.15.12:0/3385230364 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb3ac01c020 con 0x55768785e720 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb3c7659b80 10 client.16833 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb3c7659b80 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb3c7659b80 20 client.16833 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557687872200 0x557687874620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb3c7659b80 4 client.16833 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3385230364 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557687827810 con 0x557687872200 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb3b923b700 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557687872200 0x557687874620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb39cff9700 10 client.16833 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.943+0000 7fb3b923b700 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557687872200 0x557687874620 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69cee26bf3ac6bff server_cookie=dbd2ed6efa7be10f in_seq=0 out_seq=0 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb39cff9700 1 -- 172.21.15.12:0/3385230364 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557687827810 con 0x557687872200 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb39cff9700 10 client.16833 handle_command_reply: tid=0 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb3c7659b80 1 client.16833 shutdown 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3385230364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 msgr2=0x557687862480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687862480 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac011f60 tx=0x7fb3ac014600 comp rx=0 tx=0).stop 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3385230364 shutdown_connections 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687862480 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687861a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557687872200 0x557687874620 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3385230364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687861f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3385230364 >> 172.21.15.12:0/3385230364 conn(0x5576877b6210 msgr2=0x5576878525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb39cff9700 0 client.16833 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:26.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.944+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3385230364 shutdown_connections 2024-05-08T23:51:26.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.945+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3385230364 wait complete. 2024-05-08T23:51:26.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.945+0000 7fb3c7659b80 20 client.16833 trim_cache size 0 max 16384 2024-05-08T23:51:26.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.959+0000 7fb3c7659b80 1 Processor -- start 2024-05-08T23:51:26.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.959+0000 7fb3c7659b80 1 -- start start 2024-05-08T23:51:26.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.959+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557687858890 0x557687858350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.959+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55768785de00 0x557687852d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.959+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x5576878602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.959+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768784fdf0 con 0x55768785e720 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.959+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768784d800 con 0x557687858890 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.959+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557687879810 con 0x55768785de00 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.959+0000 7fb3b9a3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x5576878602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.959+0000 7fb3b923b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557687858890 0x557687858350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb3b9a3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x5576878602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53376/0 (socket says 172.21.15.12:53376) 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb3b9a3c700 1 -- 172.21.15.12:0/3399404481 learned_addr learned my addr 172.21.15.12:0/3399404481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/3399404481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55768785de00 0x557687852d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb3b9a3c700 1 -- 172.21.15.12:0/3399404481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55768785de00 msgr2=0x557687852d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb3b9a3c700 1 --2- 172.21.15.12:0/3399404481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55768785de00 0x557687852d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb3b9a3c700 1 -- 172.21.15.12:0/3399404481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557687858890 msgr2=0x557687858350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb3b9a3c700 1 --2- 172.21.15.12:0/3399404481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557687858890 0x557687858350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb3b9a3c700 1 -- 172.21.15.12:0/3399404481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576875d5b60 con 0x55768785e720 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb3b923b700 1 --2- 172.21.15.12:0/3399404481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557687858890 0x557687858350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb3b9a3c700 1 --2- 172.21.15.12:0/3399404481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x5576878602d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0149d0 tx=0x7fb3ac014a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d4741d1ed51dcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb39ffff700 1 -- 172.21.15.12:0/3399404481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00e070 con 0x55768785e720 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb39ffff700 1 -- 172.21.15.12:0/3399404481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3ac00f040 con 0x55768785e720 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.960+0000 7fb39ffff700 1 -- 172.21.15.12:0/3399404481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac007650 con 0x55768785e720 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.961+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3399404481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 msgr2=0x5576878602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.961+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3399404481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x5576878602d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0149d0 tx=0x7fb3ac014a00 comp rx=0 tx=0).stop 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.961+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3399404481 shutdown_connections 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.961+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3399404481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x5576878602d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.961+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3399404481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557687858890 0x557687858350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.961+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/3399404481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55768785de00 0x557687852d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.961+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3399404481 >> 172.21.15.12:0/3399404481 conn(0x5576877b6210 msgr2=0x55768785bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.961+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3399404481 shutdown_connections 2024-05-08T23:51:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.961+0000 7fb3c7659b80 1 -- 172.21.15.12:0/3399404481 wait complete. 2024-05-08T23:51:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.961+0000 7fb3c7659b80 1 -- start start 2024-05-08T23:51:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.962+0000 7fb3c7659b80 1 Processor -- start 2024-05-08T23:51:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.962+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687867150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.962+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687867690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.962+0000 7fb3c7659b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687867bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.962+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576878c7700 con 0x55768785e720 2024-05-08T23:51:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.962+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576878c7840 con 0x55768785de00 2024-05-08T23:51:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.962+0000 7fb3c7659b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576878c7980 con 0x557687858890 2024-05-08T23:51:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.962+0000 7fb3c7659b80 10 client.0 _ll_register_callbacks cb 0x55768774db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.962+0000 7fb3b923b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687867150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b8a3a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687867690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b9a3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687867bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b8a3a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687867690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49202/0 (socket says 172.21.15.12:49202) 2024-05-08T23:51:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b923b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687867150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33672/0 (socket says 172.21.15.12:33672) 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b9a3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687867bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53392/0 (socket says 172.21.15.12:53392) 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b8a3a700 1 -- 172.21.15.12:0/1344397929 learned_addr learned my addr 172.21.15.12:0/1344397929 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b923b700 1 -- 172.21.15.12:0/1344397929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 msgr2=0x557687867690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b923b700 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687867690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b923b700 1 -- 172.21.15.12:0/1344397929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 msgr2=0x557687867bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b923b700 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687867bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b923b700 1 -- 172.21.15.12:0/1344397929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b002a040 con 0x557687858890 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687867690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3b923b700 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687867150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3b001ccb0 tx=0x7fb3b0009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=addaedf749112f91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb39cff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:51:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb39cff9700 1 -- 172.21.15.12:0/1344397929 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b00092d0 con 0x557687858890 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb39cff9700 1 -- 172.21.15.12:0/1344397929 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb3b000ee40 con 0x557687858890 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3c7659b80 1 -- 172.21.15.12:0/1344397929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3b0021070 con 0x557687858890 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb39cff9700 1 -- 172.21.15.12:0/1344397929 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3b0005040 con 0x557687858890 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3c7659b80 10 client.26614 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.963+0000 7fb3c7659b80 1 -- 172.21.15.12:0/1344397929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fb3b0017070 con 0x557687858890 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb39cff9700 1 -- 172.21.15.12:0/1344397929 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3b0021070 con 0x557687858890 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb39cff9700 1 -- 172.21.15.12:0/1344397929 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb3b00331b0 con 0x557687858890 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb3c7659b80 10 client.26614 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb3c7659b80 10 client.26614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb3c7659b80 20 client.26614 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576878c9030 0x5576878cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb3c7659b80 4 client.26614 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb3c7659b80 1 -- 172.21.15.12:0/1344397929 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5576877a0ec0 con 0x5576878c9030 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576878c9030 0x5576878cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb39cff9700 10 client.26614 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.964+0000 7fb3b8a3a700 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576878c9030 0x5576878cb420 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c5ae7b9d7039b server_cookie=715ad9d041ba644a in_seq=0 out_seq=0 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb39cff9700 1 -- 172.21.15.12:0/1344397929 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5576877a0ec0 con 0x5576878c9030 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb39cff9700 10 client.26614 handle_command_reply: tid=0 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 client.26614 shutdown 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 -- 172.21.15.12:0/1344397929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 msgr2=0x557687867150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687867150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3b001ccb0 tx=0x7fb3b0009ed0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 -- 172.21.15.12:0/1344397929 shutdown_connections 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55768785e720 0x557687867bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557687858890 0x557687867150 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576878c9030 0x5576878cb420 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 --2- 172.21.15.12:0/1344397929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55768785de00 0x557687867690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb39cff9700 0 client.26614 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 -- 172.21.15.12:0/1344397929 >> 172.21.15.12:0/1344397929 conn(0x5576877b6210 msgr2=0x557687860140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 -- 172.21.15.12:0/1344397929 shutdown_connections 2024-05-08T23:51:26.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 1 -- 172.21.15.12:0/1344397929 wait complete. 2024-05-08T23:51:26.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.965+0000 7fb3c7659b80 20 client.26614 trim_cache size 0 max 16384 2024-05-08T23:51:26.967 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 -- 172.21.15.12:0/2223549251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3a00a1310 msgr2=0x7fb3a00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3a00a1310 0x7fb3a00a37d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0011550 tx=0x7fb3b0009260 comp rx=0 tx=0).stop 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 -- 172.21.15.12:0/2223549251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b40ffdd0 msgr2=0x7fb3b4197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b40ffdd0 0x7fb3b4197970 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40059d0 tx=0x7fb3a4004730 comp rx=0 tx=0).stop 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 -- 172.21.15.12:0/2223549251 shutdown_connections 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3a00a1310 0x7fb3a00a37d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b4105ef0 0x7fb3b419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3b40ff410 0x7fb3b4197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 --2- 172.21.15.12:0/2223549251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b40ffdd0 0x7fb3b4197970 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 -- 172.21.15.12:0/2223549251 >> 172.21.15.12:0/2223549251 conn(0x7fb3b40763a0 msgr2=0x7fb3b4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 -- 172.21.15.12:0/2223549251 shutdown_connections 2024-05-08T23:51:26.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:26.966+0000 7fb3bb49f700 1 -- 172.21.15.12:0/2223549251 wait complete. 2024-05-08T23:51:27.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 266973 11.07 MB/sec execute 83 sec latency 14.757 ms 2024-05-08T23:51:27.975 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:51:27.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:51:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.140+0000 7fb1e638a700 1 -- 172.21.15.12:0/1402099994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1e010d940 msgr2=0x7fb1e010fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.140+0000 7fb1e638a700 1 --2- 172.21.15.12:0/1402099994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1e010d940 0x7fb1e010fd30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009f90 tx=0x7fb1d800b330 comp rx=0 tx=0).stop 2024-05-08T23:51:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.140+0000 7fb1e638a700 1 -- 172.21.15.12:0/1402099994 shutdown_connections 2024-05-08T23:51:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.140+0000 7fb1e638a700 1 --2- 172.21.15.12:0/1402099994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1e010d940 0x7fb1e010fd30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.140+0000 7fb1e638a700 1 --2- 172.21.15.12:0/1402099994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1e0068050 0x7fb1e0068430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.140+0000 7fb1e638a700 1 --2- 172.21.15.12:0/1402099994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e0068970 0x7fb1e010d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.140+0000 7fb1e638a700 1 -- 172.21.15.12:0/1402099994 >> 172.21.15.12:0/1402099994 conn(0x7fb1e0076f40 msgr2=0x7fb1e0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.140+0000 7fb1e638a700 1 -- 172.21.15.12:0/1402099994 shutdown_connections 2024-05-08T23:51:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.140+0000 7fb1e638a700 1 -- 172.21.15.12:0/1402099994 wait complete. 2024-05-08T23:51:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.141+0000 7fb1e638a700 1 Processor -- start 2024-05-08T23:51:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.141+0000 7fb1e638a700 1 -- start start 2024-05-08T23:51:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.141+0000 7fb1e638a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1e0068050 0x7fb1e01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.141+0000 7fb1e638a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1e0068970 0x7fb1e0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.141+0000 7fb1e638a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e010d940 0x7fb1e019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.141+0000 7fb1e638a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e019e5c0 con 0x7fb1e010d940 2024-05-08T23:51:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.141+0000 7fb1e638a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e019e730 con 0x7fb1e0068050 2024-05-08T23:51:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.141+0000 7fb1e638a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e019e8f0 con 0x7fb1e0068970 2024-05-08T23:51:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e4927700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e010d940 0x7fb1e019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1dffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1e0068050 0x7fb1e01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e4927700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e010d940 0x7fb1e019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53400/0 (socket says 172.21.15.12:53400) 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e4927700 1 -- 172.21.15.12:0/2587317327 learned_addr learned my addr 172.21.15.12:0/2587317327 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1dffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1e0068050 0x7fb1e01996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49218/0 (socket says 172.21.15.12:49218) 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1df7fe700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1e0068970 0x7fb1e0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e4927700 1 -- 172.21.15.12:0/2587317327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1e0068970 msgr2=0x7fb1e0199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e4927700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1e0068970 0x7fb1e0199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e4927700 1 -- 172.21.15.12:0/2587317327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1e0068050 msgr2=0x7fb1e01996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e4927700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1e0068050 0x7fb1e01996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e4927700 1 -- 172.21.15.12:0/2587317327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40095f0 con 0x7fb1e010d940 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e4927700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e010d940 0x7fb1e019df80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009f60 tx=0x7fb1d8007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1dd7fa700 1 -- 172.21.15.12:0/2587317327 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800e040 con 0x7fb1e010d940 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e638a700 1 -- 172.21.15.12:0/2587317327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d8009c40 con 0x7fb1e010d940 2024-05-08T23:51:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1dd7fa700 1 -- 172.21.15.12:0/2587317327 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1d8007e50 con 0x7fb1e010d940 2024-05-08T23:51:28.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1dd7fa700 1 -- 172.21.15.12:0/2587317327 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80052a0 con 0x7fb1e010d940 2024-05-08T23:51:28.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.142+0000 7fb1e638a700 1 -- 172.21.15.12:0/2587317327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e01a6230 con 0x7fb1e010d940 2024-05-08T23:51:28.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.146+0000 7fb1dd7fa700 1 -- 172.21.15.12:0/2587317327 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb1d801b450 con 0x7fb1e010d940 2024-05-08T23:51:28.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.146+0000 7fb1dd7fa700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1c80a1290 0x7fb1c80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1dd7fa700 1 -- 172.21.15.12:0/2587317327 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb1d80ce0b0 con 0x7fb1e010d940 2024-05-08T23:51:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1dffff700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1c80a1290 0x7fb1c80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1f2544b80 1 Processor -- start 2024-05-08T23:51:28.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1f2544b80 1 -- start start 2024-05-08T23:51:28.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1dffff700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1c80a1290 0x7fb1c80a3750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0007be0 tx=0x7fb1d00073d0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bdda6e0 0x558e0bddaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bddb000 0x558e0bdd4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdd5170 0x558e0bdd55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bd231d0 con 0x558e0bdda6e0 2024-05-08T23:51:28.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bd20b90 con 0x558e0bdd5170 2024-05-08T23:51:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.147+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bd1c480 con 0x558e0bddb000 2024-05-08T23:51:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.148+0000 7fb1e4927700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdd5170 0x558e0bdd55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.148+0000 7fb1e4927700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdd5170 0x558e0bdd55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49226/0 (socket says 172.21.15.12:49226) 2024-05-08T23:51:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.148+0000 7fb1e4927700 1 -- 172.21.15.12:0/2994697613 learned_addr learned my addr 172.21.15.12:0/2994697613 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.148+0000 7fb1e4927700 1 -- 172.21.15.12:0/2994697613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bddb000 msgr2=0x558e0bdd4be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:51:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.148+0000 7fb1e4927700 1 --2- 172.21.15.12:0/2994697613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bddb000 0x558e0bdd4be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.148+0000 7fb1e4927700 1 -- 172.21.15.12:0/2994697613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bdda6e0 msgr2=0x558e0bddaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.148+0000 7fb1e4927700 1 --2- 172.21.15.12:0/2994697613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bdda6e0 0x558e0bddaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.148+0000 7fb1e4927700 1 -- 172.21.15.12:0/2994697613 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e0bb9d430 con 0x558e0bdd5170 2024-05-08T23:51:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.148+0000 7fb1e4927700 1 --2- 172.21.15.12:0/2994697613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdd5170 0x558e0bdd55f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1d809b030 tx=0x7fb1d8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cccb2879288991a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.149+0000 7fb1c67fc700 1 -- 172.21.15.12:0/2994697613 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d809ac90 con 0x558e0bdd5170 2024-05-08T23:51:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.149+0000 7fb1c67fc700 1 -- 172.21.15.12:0/2994697613 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1d800bbf0 con 0x558e0bdd5170 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.149+0000 7fb1c67fc700 1 -- 172.21.15.12:0/2994697613 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d809f160 con 0x558e0bdd5170 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.150+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2994697613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdd5170 msgr2=0x558e0bdd55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.150+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/2994697613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdd5170 0x558e0bdd55f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1d809b030 tx=0x7fb1d8009920 comp rx=0 tx=0).stop 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.150+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2994697613 shutdown_connections 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.150+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/2994697613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdd5170 0x558e0bdd55f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.150+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/2994697613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bdda6e0 0x558e0bddaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.150+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/2994697613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bddb000 0x558e0bdd4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.150+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2994697613 >> 172.21.15.12:0/2994697613 conn(0x558e0bd2a290 msgr2=0x558e0bdc6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.150+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2994697613 shutdown_connections 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.150+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2994697613 wait complete. 2024-05-08T23:51:28.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.150+0000 7fb1f2544b80 1 -- start start 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1f2544b80 1 Processor -- start 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bddea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bddefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdd36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bcb85e0 con 0x558e0bddb000 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bcb8280 con 0x558e0bdda6e0 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bb9f720 con 0x558e0bdd5170 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1f2544b80 10 client.0 _ll_register_callbacks cb 0x558e0bcc1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1df7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bddefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1df7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bddefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49240/0 (socket says 172.21.15.12:49240) 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1df7fe700 1 -- 172.21.15.12:0/3035429695 learned_addr learned my addr 172.21.15.12:0/3035429695 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1e4927700 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdd36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.151+0000 7fb1dffff700 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bddea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1df7fe700 1 -- 172.21.15.12:0/3035429695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 msgr2=0x558e0bddea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1df7fe700 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bddea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1df7fe700 1 -- 172.21.15.12:0/3035429695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 msgr2=0x558e0bdd36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1df7fe700 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdd36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1df7fe700 1 -- 172.21.15.12:0/3035429695 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e01a6230 con 0x558e0bdda6e0 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1df7fe700 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bddefa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4011800 tx=0x7fb1d4011b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f00a8f4aca74bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/3035429695 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4004520 con 0x558e0bdda6e0 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/3035429695 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb1d4003680 con 0x558e0bdda6e0 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1f2544b80 1 -- 172.21.15.12:0/3035429695 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e0bc752f0 con 0x558e0bdda6e0 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/3035429695 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1d4003840 con 0x558e0bdda6e0 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1f2544b80 10 client.26636 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.152+0000 7fb1f2544b80 1 -- 172.21.15.12:0/3035429695 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558e0bb49be0 con 0x558e0bdda6e0 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/3035429695 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e0bc752f0 con 0x558e0bdda6e0 2024-05-08T23:51:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/3035429695 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb1d400fca0 con 0x558e0bdda6e0 2024-05-08T23:51:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1f2544b80 10 client.26636 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1f2544b80 10 client.26636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1f2544b80 20 client.26636 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558e0bdf7170 0x558e0bdf9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1f2544b80 4 client.26636 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1f2544b80 1 -- 172.21.15.12:0/3035429695 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e0bd9b890 con 0x558e0bdf7170 2024-05-08T23:51:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1dffff700 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558e0bdf7170 0x558e0bdf9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1bf7fe700 10 client.26636 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.153+0000 7fb1dffff700 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558e0bdf7170 0x558e0bdf9590 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9365e12fc14a6e server_cookie=8bd8342b0f33706b in_seq=0 out_seq=0 2024-05-08T23:51:28.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/3035429695 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e0bd9b890 con 0x558e0bdf7170 2024-05-08T23:51:28.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1bf7fe700 10 client.26636 handle_command_reply: tid=0 2024-05-08T23:51:28.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 client.26636 shutdown 2024-05-08T23:51:28.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 -- 172.21.15.12:0/3035429695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 msgr2=0x558e0bddefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bddefa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4011800 tx=0x7fb1d4011b10 comp rx=0 tx=0).stop 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 -- 172.21.15.12:0/3035429695 shutdown_connections 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdd36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bddea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558e0bdf7170 0x558e0bdf9590 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/3035429695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bddefa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1bf7fe700 0 client.26636 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 -- 172.21.15.12:0/3035429695 >> 172.21.15.12:0/3035429695 conn(0x558e0bd2a290 msgr2=0x558e0bdc6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 -- 172.21.15.12:0/3035429695 shutdown_connections 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.154+0000 7fb1f2544b80 1 -- 172.21.15.12:0/3035429695 wait complete. 2024-05-08T23:51:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.155+0000 7fb1f2544b80 20 client.26636 trim_cache size 0 max 16384 2024-05-08T23:51:28.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.174+0000 7fb1f2544b80 1 Processor -- start 2024-05-08T23:51:28.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1f2544b80 1 -- start start 2024-05-08T23:51:28.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bddc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bdde150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdc6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bdd4780 con 0x558e0bddb000 2024-05-08T23:51:28.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bddfee0 con 0x558e0bdda6e0 2024-05-08T23:51:28.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bdc3e40 con 0x558e0bdd5170 2024-05-08T23:51:28.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1e4927700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdc6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1df7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bdde150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1dffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bddc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1df7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bdde150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49256/0 (socket says 172.21.15.12:49256) 2024-05-08T23:51:28.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1df7fe700 1 -- 172.21.15.12:0/733710193 learned_addr learned my addr 172.21.15.12:0/733710193 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1e4927700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdc6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53432/0 (socket says 172.21.15.12:53432) 2024-05-08T23:51:28.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1df7fe700 1 -- 172.21.15.12:0/733710193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 msgr2=0x558e0bddc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1df7fe700 1 --2- 172.21.15.12:0/733710193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bddc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1df7fe700 1 -- 172.21.15.12:0/733710193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 msgr2=0x558e0bdc6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1df7fe700 1 --2- 172.21.15.12:0/733710193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdc6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.175+0000 7fb1df7fe700 1 -- 172.21.15.12:0/733710193 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e0bce25e0 con 0x558e0bdda6e0 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.176+0000 7fb1df7fe700 1 --2- 172.21.15.12:0/733710193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bdde150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4001eb0 tx=0x7fb1d40214e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ba9ded9ec4902d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.176+0000 7fb1c6ffd700 1 -- 172.21.15.12:0/733710193 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4005590 con 0x558e0bdda6e0 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.176+0000 7fb1c6ffd700 1 -- 172.21.15.12:0/733710193 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1d4009380 con 0x558e0bdda6e0 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.176+0000 7fb1c6ffd700 1 -- 172.21.15.12:0/733710193 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4003b40 con 0x558e0bdda6e0 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.177+0000 7fb1f2544b80 1 -- 172.21.15.12:0/733710193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 msgr2=0x558e0bdde150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.177+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/733710193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bdde150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4001eb0 tx=0x7fb1d40214e0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.177+0000 7fb1f2544b80 1 -- 172.21.15.12:0/733710193 shutdown_connections 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.177+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/733710193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdc6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.177+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/733710193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bddc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.177+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/733710193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bdde150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.177+0000 7fb1f2544b80 1 -- 172.21.15.12:0/733710193 >> 172.21.15.12:0/733710193 conn(0x558e0bd2a290 msgr2=0x558e0bdd8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.177+0000 7fb1f2544b80 1 -- 172.21.15.12:0/733710193 shutdown_connections 2024-05-08T23:51:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.177+0000 7fb1f2544b80 1 -- 172.21.15.12:0/733710193 wait complete. 2024-05-08T23:51:28.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.177+0000 7fb1f2544b80 1 -- start start 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1f2544b80 1 Processor -- start 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bdd0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bdd0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1f2544b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bdced40 con 0x558e0bddb000 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bdceeb0 con 0x558e0bdda6e0 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1f2544b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bdcf070 con 0x558e0bdd5170 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1f2544b80 10 client.0 _ll_register_callbacks cb 0x558e0bcc1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1dffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bdd0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1e4927700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1e4927700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53448/0 (socket says 172.21.15.12:53448) 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.178+0000 7fb1e4927700 1 -- 172.21.15.12:0/2198893744 learned_addr learned my addr 172.21.15.12:0/2198893744 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1df7fe700 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bdd0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1e4927700 1 -- 172.21.15.12:0/2198893744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 msgr2=0x558e0bdd0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1e4927700 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bdd0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1e4927700 1 -- 172.21.15.12:0/2198893744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 msgr2=0x558e0bdd0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1e4927700 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bdd0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1e4927700 1 -- 172.21.15.12:0/2198893744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e0bb49be0 con 0x558e0bddb000 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1e4927700 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdce630 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800d010 tx=0x7fb1d80cbf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becb179e13cdc037 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/2198893744 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d809ac90 con 0x558e0bddb000 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/2198893744 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb1d809ba30 con 0x558e0bddb000 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2198893744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1d800bbf0 con 0x558e0bddb000 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/2198893744 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1d801be90 con 0x558e0bddb000 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1f2544b80 10 client.16854 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.179+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2198893744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558e0bba1d50 con 0x558e0bddb000 2024-05-08T23:51:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.180+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/2198893744 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1d800bbf0 con 0x558e0bddb000 2024-05-08T23:51:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.180+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/2198893744 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb1d809a820 con 0x558e0bddb000 2024-05-08T23:51:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.180+0000 7fb1f2544b80 10 client.16854 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.180+0000 7fb1f2544b80 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.180+0000 7fb1f2544b80 20 client.16854 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.180+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558e0be4f3f0 0x558e0be51810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.180+0000 7fb1f2544b80 4 client.16854 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:51:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.180+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2198893744 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558e0bd14f40 con 0x558e0be4f3f0 2024-05-08T23:51:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.180+0000 7fb1dffff700 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558e0be4f3f0 0x558e0be51810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.180+0000 7fb1bf7fe700 10 client.16854 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.181+0000 7fb1dffff700 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558e0be4f3f0 0x558e0be51810 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0ebba23c466fd13 server_cookie=327f6a15cc1ea375 in_seq=0 out_seq=0 2024-05-08T23:51:28.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.181+0000 7fb1bf7fe700 1 -- 172.21.15.12:0/2198893744 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558e0bd14f40 con 0x558e0be4f3f0 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.181+0000 7fb1bf7fe700 10 client.16854 handle_command_reply: tid=0 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.181+0000 7fb1f2544b80 1 client.16854 shutdown 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2198893744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 msgr2=0x558e0bdce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdce630 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800d010 tx=0x7fb1d80cbf90 comp rx=0 tx=0).stop 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2198893744 shutdown_connections 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e0bddb000 0x558e0bdce630 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558e0bdd5170 0x558e0bdd0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558e0be4f3f0 0x558e0be51810 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 1 --2- 172.21.15.12:0/2198893744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558e0bdda6e0 0x558e0bdd0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2198893744 >> 172.21.15.12:0/2198893744 conn(0x558e0bd2a290 msgr2=0x558e0bdde810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1bf7fe700 0 client.16854 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2198893744 shutdown_connections 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 1 -- 172.21.15.12:0/2198893744 wait complete. 2024-05-08T23:51:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1f2544b80 20 client.16854 trim_cache size 0 max 16384 2024-05-08T23:51:28.181 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:51:28.181 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:51:28.181 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "bcc06c4e-129d-4e73-b61d-0bce038d32b5", 2024-05-08T23:51:28.181 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:51:28.181 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:51:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1e638a700 1 -- 172.21.15.12:0/2587317327 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1c80a1290 msgr2=0x7fb1c80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1e638a700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1c80a1290 0x7fb1c80a3750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0007be0 tx=0x7fb1d00073d0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.182+0000 7fb1e638a700 1 -- 172.21.15.12:0/2587317327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e010d940 msgr2=0x7fb1e019df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.183+0000 7fb1e638a700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e010d940 0x7fb1e019df80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009f60 tx=0x7fb1d8007a50 comp rx=0 tx=0).stop 2024-05-08T23:51:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.183+0000 7fb1e638a700 1 -- 172.21.15.12:0/2587317327 shutdown_connections 2024-05-08T23:51:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.183+0000 7fb1e638a700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1c80a1290 0x7fb1c80a3750 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.183+0000 7fb1e638a700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1e010d940 0x7fb1e019df80 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.183+0000 7fb1e638a700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1e0068050 0x7fb1e01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.183+0000 7fb1e638a700 1 --2- 172.21.15.12:0/2587317327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1e0068970 0x7fb1e0199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.183+0000 7fb1e638a700 1 -- 172.21.15.12:0/2587317327 >> 172.21.15.12:0/2587317327 conn(0x7fb1e0076f40 msgr2=0x7fb1e010eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.183+0000 7fb1e638a700 1 -- 172.21.15.12:0/2587317327 shutdown_connections 2024-05-08T23:51:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.183+0000 7fb1e638a700 1 -- 172.21.15.12:0/2587317327 wait complete. 2024-05-08T23:51:28.194 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:51:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.248+0000 7f6e5b535700 1 -- 172.21.15.12:0/1577730753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e540ffda0 msgr2=0x7f6e54100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.248+0000 7f6e5b535700 1 --2- 172.21.15.12:0/1577730753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e540ffda0 0x7f6e54100220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009990 tx=0x7f6e50009ca0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.248+0000 7f6e5b535700 1 -- 172.21.15.12:0/1577730753 shutdown_connections 2024-05-08T23:51:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.248+0000 7f6e5b535700 1 --2- 172.21.15.12:0/1577730753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e54100760 0x7f6e5410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.248+0000 7f6e5b535700 1 --2- 172.21.15.12:0/1577730753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e54105e60 0x7f6e54106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.248+0000 7f6e5b535700 1 --2- 172.21.15.12:0/1577730753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e540ffda0 0x7f6e54100220 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.248+0000 7f6e5b535700 1 -- 172.21.15.12:0/1577730753 >> 172.21.15.12:0/1577730753 conn(0x7f6e54076f10 msgr2=0x7f6e54077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.248+0000 7f6e5b535700 1 -- 172.21.15.12:0/1577730753 shutdown_connections 2024-05-08T23:51:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.248+0000 7f6e5b535700 1 -- 172.21.15.12:0/1577730753 wait complete. 2024-05-08T23:51:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e5b535700 1 Processor -- start 2024-05-08T23:51:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e5b535700 1 -- start start 2024-05-08T23:51:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e5b535700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e540ffda0 0x7f6e541973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e5b535700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e54100760 0x7f6e54197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e5b535700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e54105e60 0x7f6e5419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e5b535700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5419c290 con 0x7f6e540ffda0 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e5b535700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5419c400 con 0x7f6e54100760 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e5b535700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5419c5c0 con 0x7f6e54105e60 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e592d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e540ffda0 0x7f6e541973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e592d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e540ffda0 0x7f6e541973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53466/0 (socket says 172.21.15.12:53466) 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e592d1700 1 -- 172.21.15.12:0/423677471 learned_addr learned my addr 172.21.15.12:0/423677471 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e54100760 0x7f6e54197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e59ad2700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e54105e60 0x7f6e5419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e592d1700 1 -- 172.21.15.12:0/423677471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e54105e60 msgr2=0x7f6e5419bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e592d1700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e54105e60 0x7f6e5419bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e592d1700 1 -- 172.21.15.12:0/423677471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e54100760 msgr2=0x7f6e54197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e592d1700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e54100760 0x7f6e54197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e592d1700 1 -- 172.21.15.12:0/423677471 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e500095f0 con 0x7f6e540ffda0 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.249+0000 7f6e59ad2700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e54105e60 0x7f6e5419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:51:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.250+0000 7f6e592d1700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e540ffda0 0x7f6e541973c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800e9a0 tx=0x7f6e4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.250+0000 7f6e467fc700 1 -- 172.21.15.12:0/423677471 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48009790 con 0x7f6e540ffda0 2024-05-08T23:51:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.250+0000 7f6e5b535700 1 -- 172.21.15.12:0/423677471 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5419c8a0 con 0x7f6e540ffda0 2024-05-08T23:51:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.250+0000 7f6e467fc700 1 -- 172.21.15.12:0/423677471 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4800cbd0 con 0x7f6e540ffda0 2024-05-08T23:51:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.250+0000 7f6e467fc700 1 -- 172.21.15.12:0/423677471 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48018680 con 0x7f6e540ffda0 2024-05-08T23:51:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.250+0000 7f6e5b535700 1 -- 172.21.15.12:0/423677471 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e541a40e0 con 0x7f6e540ffda0 2024-05-08T23:51:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.253+0000 7f6e467fc700 1 -- 172.21.15.12:0/423677471 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6e480187e0 con 0x7f6e540ffda0 2024-05-08T23:51:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.253+0000 7f6e467fc700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e400a1290 0x7f6e400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.253+0000 7f6e467fc700 1 -- 172.21.15.12:0/423677471 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6e48014070 con 0x7f6e540ffda0 2024-05-08T23:51:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.253+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e400a1290 0x7f6e400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.254+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e400a1290 0x7f6e400a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b2e0 tx=0x7f6e50005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.254+0000 7f6e676efb80 1 Processor -- start 2024-05-08T23:51:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.254+0000 7f6e676efb80 1 -- start start 2024-05-08T23:51:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 0x55b9b606cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6066f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 0x55b9b60678a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e676efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b5fbe0f0 con 0x55b9b606d270 2024-05-08T23:51:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e676efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b5fbbab0 con 0x55b9b6067480 2024-05-08T23:51:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e676efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b5fb73a0 con 0x55b9b606c950 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e58ad0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6066f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e58ad0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6066f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53474/0 (socket says 172.21.15.12:53474) 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e58ad0700 1 -- 172.21.15.12:0/3496559675 learned_addr learned my addr 172.21.15.12:0/3496559675 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e592d1700 1 --2- 172.21.15.12:0/3496559675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 0x55b9b606cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e59ad2700 1 --2- 172.21.15.12:0/3496559675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 0x55b9b60678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e58ad0700 1 -- 172.21.15.12:0/3496559675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 msgr2=0x55b9b606cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/3496559675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 0x55b9b606cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e58ad0700 1 -- 172.21.15.12:0/3496559675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 msgr2=0x55b9b60678a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/3496559675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 0x55b9b60678a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e58ad0700 1 -- 172.21.15.12:0/3496559675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9b5e38350 con 0x55b9b606d270 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.255+0000 7f6e592d1700 1 --2- 172.21.15.12:0/3496559675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 0x55b9b606cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.256+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/3496559675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6066f40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009990 tx=0x7f6e50020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b313454220058ff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.256+0000 7f6e3f7fe700 1 -- 172.21.15.12:0/3496559675 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50004500 con 0x55b9b606d270 2024-05-08T23:51:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.256+0000 7f6e3f7fe700 1 -- 172.21.15.12:0/3496559675 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e5001e040 con 0x55b9b606d270 2024-05-08T23:51:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.256+0000 7f6e3f7fe700 1 -- 172.21.15.12:0/3496559675 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000f8c0 con 0x55b9b606d270 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 -- 172.21.15.12:0/3496559675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 msgr2=0x55b9b6066f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 --2- 172.21.15.12:0/3496559675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6066f40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009990 tx=0x7f6e50020000 comp rx=0 tx=0).stop 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 -- 172.21.15.12:0/3496559675 shutdown_connections 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 --2- 172.21.15.12:0/3496559675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 0x55b9b60678a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 --2- 172.21.15.12:0/3496559675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 0x55b9b606cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 --2- 172.21.15.12:0/3496559675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6066f40 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 -- 172.21.15.12:0/3496559675 >> 172.21.15.12:0/3496559675 conn(0x55b9b5fc51b0 msgr2=0x55b9b6061840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 -- 172.21.15.12:0/3496559675 shutdown_connections 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 -- 172.21.15.12:0/3496559675 wait complete. 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 -- start start 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.257+0000 7f6e676efb80 1 Processor -- start 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b6067480 0x55b9b60732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e676efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b5f53500 con 0x55b9b606d270 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e676efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b5f531a0 con 0x55b9b606c950 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e676efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b5e3a640 con 0x55b9b6067480 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e676efb80 10 client.0 _ll_register_callbacks cb 0x55b9b5f5cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e58ad0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e58ad0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49284/0 (socket says 172.21.15.12:49284) 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e592d1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b6067480 0x55b9b60732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e58ad0700 1 -- 172.21.15.12:0/3249783600 learned_addr learned my addr 172.21.15.12:0/3249783600 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e59ad2700 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e58ad0700 1 -- 172.21.15.12:0/3249783600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b6067480 msgr2=0x55b9b60732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b6067480 0x55b9b60732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e58ad0700 1 -- 172.21.15.12:0/3249783600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 msgr2=0x55b9b6070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.258+0000 7f6e58ad0700 1 -- 172.21.15.12:0/3249783600 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e541a40e0 con 0x55b9b606c950 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6070590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009ce0 tx=0x7f6e50004b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee588cba35dc691 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e3cff9700 1 -- 172.21.15.12:0/3249783600 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50004ca0 con 0x55b9b606c950 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e3cff9700 1 -- 172.21.15.12:0/3249783600 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6e5000f7b0 con 0x55b9b606c950 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e676efb80 1 -- 172.21.15.12:0/3249783600 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9b5f10210 con 0x55b9b606c950 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e3cff9700 1 -- 172.21.15.12:0/3249783600 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e5000f970 con 0x55b9b606c950 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e676efb80 10 client.26654 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e676efb80 1 -- 172.21.15.12:0/3249783600 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f6e5410d4f0 con 0x55b9b606c950 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e3cff9700 1 -- 172.21.15.12:0/3249783600 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9b5f10210 con 0x55b9b606c950 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e3cff9700 1 -- 172.21.15.12:0/3249783600 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6e500244a0 con 0x55b9b606c950 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e676efb80 10 client.26654 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e676efb80 10 client.26654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e676efb80 20 client.26654 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e676efb80 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9b60807d0 0x55b9b6082bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e676efb80 4 client.26654 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.259+0000 7f6e676efb80 1 -- 172.21.15.12:0/3249783600 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9b60367b0 con 0x55b9b60807d0 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.260+0000 7f6e592d1700 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9b60807d0 0x55b9b6082bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.260+0000 7f6e3cff9700 10 client.26654 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.260+0000 7f6e592d1700 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9b60807d0 0x55b9b6082bc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee189767e899c26d server_cookie=45e167688b13abfc in_seq=0 out_seq=0 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e3cff9700 1 -- 172.21.15.12:0/3249783600 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9b60367b0 con 0x55b9b60807d0 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e3cff9700 10 client.26654 handle_command_reply: tid=0 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 client.26654 shutdown 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 -- 172.21.15.12:0/3249783600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 msgr2=0x55b9b6070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6070590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009ce0 tx=0x7f6e50004b30 comp rx=0 tx=0).stop 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 -- 172.21.15.12:0/3249783600 shutdown_connections 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b6067480 0x55b9b60732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9b60807d0 0x55b9b6082bc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 --2- 172.21.15.12:0/3249783600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6070590 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 -- 172.21.15.12:0/3249783600 >> 172.21.15.12:0/3249783600 conn(0x55b9b5fc51b0 msgr2=0x55b9b606b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e3cff9700 0 client.26654 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 -- 172.21.15.12:0/3249783600 shutdown_connections 2024-05-08T23:51:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 1 -- 172.21.15.12:0/3249783600 wait complete. 2024-05-08T23:51:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.261+0000 7f6e676efb80 20 client.26654 trim_cache size 0 max 16384 2024-05-08T23:51:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e676efb80 1 Processor -- start 2024-05-08T23:51:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e676efb80 1 -- start start 2024-05-08T23:51:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 0x55b9b6061910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 0x55b9b606b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e676efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b6066170 con 0x55b9b606d270 2024-05-08T23:51:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e676efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b6067860 con 0x55b9b6067480 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e676efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b6067a20 con 0x55b9b606c950 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e592d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 0x55b9b6061910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e59ad2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e592d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 0x55b9b6061910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49286/0 (socket says 172.21.15.12:49286) 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e592d1700 1 -- 172.21.15.12:0/1831583934 learned_addr learned my addr 172.21.15.12:0/1831583934 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/1831583934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 0x55b9b606b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e592d1700 1 -- 172.21.15.12:0/1831583934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 msgr2=0x55b9b606b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e592d1700 1 --2- 172.21.15.12:0/1831583934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 0x55b9b606b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e592d1700 1 -- 172.21.15.12:0/1831583934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 msgr2=0x55b9b6072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e592d1700 1 --2- 172.21.15.12:0/1831583934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6072dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.279+0000 7f6e592d1700 1 -- 172.21.15.12:0/1831583934 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9b5de4b00 con 0x55b9b6067480 2024-05-08T23:51:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.280+0000 7f6e592d1700 1 --2- 172.21.15.12:0/1831583934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 0x55b9b6061910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800c5b0 tx=0x7f6e4801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6edcadc5f8b84732 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.280+0000 7f6e3ffff700 1 -- 172.21.15.12:0/1831583934 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e480adea0 con 0x55b9b6067480 2024-05-08T23:51:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.280+0000 7f6e3ffff700 1 -- 172.21.15.12:0/1831583934 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e480a2450 con 0x55b9b6067480 2024-05-08T23:51:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.280+0000 7f6e3ffff700 1 -- 172.21.15.12:0/1831583934 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48099720 con 0x55b9b6067480 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 -- 172.21.15.12:0/1831583934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 msgr2=0x55b9b6061910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 --2- 172.21.15.12:0/1831583934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 0x55b9b6061910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800c5b0 tx=0x7f6e4801a040 comp rx=0 tx=0).stop 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 -- 172.21.15.12:0/1831583934 shutdown_connections 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 --2- 172.21.15.12:0/1831583934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b6072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 --2- 172.21.15.12:0/1831583934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b6067480 0x55b9b6061910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 --2- 172.21.15.12:0/1831583934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b606c950 0x55b9b606b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 -- 172.21.15.12:0/1831583934 >> 172.21.15.12:0/1831583934 conn(0x55b9b5fc51b0 msgr2=0x55b9b60874e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 -- 172.21.15.12:0/1831583934 shutdown_connections 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 -- 172.21.15.12:0/1831583934 wait complete. 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 -- start start 2024-05-08T23:51:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.281+0000 7f6e676efb80 1 Processor -- start 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b6067480 0x55b9b6073be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6074120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e676efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b60d5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e676efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b6074900 con 0x55b9b606d270 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e676efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b6074a70 con 0x55b9b606c950 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e676efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b60d6190 con 0x55b9b6067480 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e676efb80 10 client.0 _ll_register_callbacks cb 0x55b9b5f5cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e58ad0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e58ad0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6074120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49300/0 (socket says 172.21.15.12:49300) 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e58ad0700 1 -- 172.21.15.12:0/570496073 learned_addr learned my addr 172.21.15.12:0/570496073 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e59ad2700 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b60d5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e592d1700 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b6067480 0x55b9b6073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e58ad0700 1 -- 172.21.15.12:0/570496073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b6067480 msgr2=0x55b9b6073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b6067480 0x55b9b6073be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e58ad0700 1 -- 172.21.15.12:0/570496073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 msgr2=0x55b9b60d5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b60d5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.282+0000 7f6e58ad0700 1 -- 172.21.15.12:0/570496073 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5419c8a0 con 0x55b9b606c950 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e58ad0700 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6074120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e870 tx=0x7f6e50009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c734431bd375d545 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e3cff9700 1 -- 172.21.15.12:0/570496073 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000e5e0 con 0x55b9b606c950 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e3cff9700 1 -- 172.21.15.12:0/570496073 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6e50024ad0 con 0x55b9b606c950 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e676efb80 1 -- 172.21.15.12:0/570496073 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e5001e040 con 0x55b9b606c950 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e3cff9700 1 -- 172.21.15.12:0/570496073 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e5000bd10 con 0x55b9b606c950 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e676efb80 10 client.26660 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e676efb80 1 -- 172.21.15.12:0/570496073 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b9b5f7d500 con 0x55b9b606c950 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e3cff9700 1 -- 172.21.15.12:0/570496073 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e5001e040 con 0x55b9b606c950 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e3cff9700 1 -- 172.21.15.12:0/570496073 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6e5000f7b0 con 0x55b9b606c950 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e676efb80 10 client.26660 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e676efb80 10 client.26660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e676efb80 20 client.26660 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e676efb80 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9b60d7560 0x55b9b60d9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e676efb80 4 client.26660 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e676efb80 1 -- 172.21.15.12:0/570496073 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b9b5fafe60 con 0x55b9b60d7560 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.283+0000 7f6e592d1700 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9b60d7560 0x55b9b60d9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.284+0000 7f6e592d1700 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9b60d7560 0x55b9b60d9980 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bad9da5d7dce89f server_cookie=d80108f7f7e6edbe in_seq=0 out_seq=0 2024-05-08T23:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.284+0000 7f6e3cff9700 10 client.26660 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.341+0000 7f6e3cff9700 1 -- 172.21.15.12:0/570496073 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b9b5fafe60 con 0x55b9b60d7560 2024-05-08T23:51:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.341+0000 7f6e3cff9700 10 client.26660 handle_command_reply: tid=0 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.341+0000 7f6e676efb80 1 client.26660 shutdown 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 1 -- 172.21.15.12:0/570496073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 msgr2=0x55b9b6074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6074120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e870 tx=0x7f6e50009300 comp rx=0 tx=0).stop 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 1 -- 172.21.15.12:0/570496073 shutdown_connections 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9b606d270 0x55b9b60d5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9b6067480 0x55b9b6073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9b60d7560 0x55b9b60d9980 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 1 --2- 172.21.15.12:0/570496073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9b606c950 0x55b9b6074120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e3cff9700 0 client.26660 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 1 -- 172.21.15.12:0/570496073 >> 172.21.15.12:0/570496073 conn(0x55b9b5fc51b0 msgr2=0x55b9b607c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 1 -- 172.21.15.12:0/570496073 shutdown_connections 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 1 -- 172.21.15.12:0/570496073 wait complete. 2024-05-08T23:51:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.342+0000 7f6e676efb80 20 client.26660 trim_cache size 0 max 16384 2024-05-08T23:51:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:51:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:51:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:51:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:51:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 -- 172.21.15.12:0/423677471 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e400a1290 msgr2=0x7f6e400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e400a1290 0x7f6e400a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b2e0 tx=0x7f6e50005960 comp rx=0 tx=0).stop 2024-05-08T23:51:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 -- 172.21.15.12:0/423677471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e540ffda0 msgr2=0x7f6e541973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e540ffda0 0x7f6e541973c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800e9a0 tx=0x7f6e4800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 -- 172.21.15.12:0/423677471 shutdown_connections 2024-05-08T23:51:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e400a1290 0x7f6e400a3750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e54105e60 0x7f6e5419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e540ffda0 0x7f6e541973c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 --2- 172.21.15.12:0/423677471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e54100760 0x7f6e54197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 -- 172.21.15.12:0/423677471 >> 172.21.15.12:0/423677471 conn(0x7f6e54076f10 msgr2=0x7f6e54068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 -- 172.21.15.12:0/423677471 shutdown_connections 2024-05-08T23:51:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:28.343+0000 7f6e5b535700 1 -- 172.21.15.12:0/423677471 wait complete. 2024-05-08T23:51:28.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.366+0000 7fedc1ff8700 1 -- 172.21.15.12:0/46947106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedbc06a150 msgr2=0x7fedbc105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.366+0000 7fedc1ff8700 1 --2- 172.21.15.12:0/46947106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedbc06a150 0x7fedbc105b70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7feda40099c0 tx=0x7feda4009cd0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.366+0000 7fedc1ff8700 1 -- 172.21.15.12:0/46947106 shutdown_connections 2024-05-08T23:51:28.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.366+0000 7fedc1ff8700 1 --2- 172.21.15.12:0/46947106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc1062a0 0x7fedbc10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.366+0000 7fedc1ff8700 1 --2- 172.21.15.12:0/46947106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedbc0697a0 0x7fedbc069b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.366+0000 7fedc1ff8700 1 --2- 172.21.15.12:0/46947106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedbc06a150 0x7fedbc105b70 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.366+0000 7fedc1ff8700 1 -- 172.21.15.12:0/46947106 >> 172.21.15.12:0/46947106 conn(0x7fedbc076fa0 msgr2=0x7fedbc0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.366+0000 7fedc1ff8700 1 -- 172.21.15.12:0/46947106 shutdown_connections 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.366+0000 7fedc1ff8700 1 -- 172.21.15.12:0/46947106 wait complete. 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.367+0000 7fedc1ff8700 1 Processor -- start 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.367+0000 7fedc1ff8700 1 -- start start 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.367+0000 7fedc1ff8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedbc0697a0 0x7fedbc0ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.367+0000 7fedc1ff8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedbc1062a0 0x7fedbc1003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.367+0000 7fedc1ff8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc101b20 0x7fedbc100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.367+0000 7fedc1ff8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc1a9ff0 con 0x7fedbc1062a0 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.367+0000 7fedc1ff8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc1aa160 con 0x7fedbc0697a0 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.367+0000 7fedc1ff8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc1aa320 con 0x7fedbc101b20 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedbc1062a0 0x7fedbc1003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedbc0697a0 0x7fedbc0ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedbc1062a0 0x7fedbc1003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53528/0 (socket says 172.21.15.12:53528) 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbaffd700 1 -- 172.21.15.12:0/2861275417 learned_addr learned my addr 172.21.15.12:0/2861275417 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbbfff700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc101b20 0x7fedbc100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbaffd700 1 -- 172.21.15.12:0/2861275417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc101b20 msgr2=0x7fedbc100920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbaffd700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc101b20 0x7fedbc100920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbaffd700 1 -- 172.21.15.12:0/2861275417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedbc0697a0 msgr2=0x7fedbc0ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbaffd700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedbc0697a0 0x7fedbc0ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbaffd700 1 -- 172.21.15.12:0/2861275417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac009520 con 0x7fedbc1062a0 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedbc0697a0 0x7fedbc0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedbaffd700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedbc1062a0 0x7fedbc1003a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7feda40059d0 tx=0x7feda400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedb8ff9700 1 -- 172.21.15.12:0/2861275417 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda401d070 con 0x7fedbc1062a0 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedb8ff9700 1 -- 172.21.15.12:0/2861275417 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda400beb0 con 0x7fedbc1062a0 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.368+0000 7fedc1ff8700 1 -- 172.21.15.12:0/2861275417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb4009c40 con 0x7fedbc1062a0 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.369+0000 7fedb8ff9700 1 -- 172.21.15.12:0/2861275417 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda400fa30 con 0x7fedbc1062a0 2024-05-08T23:51:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.369+0000 7fedc1ff8700 1 -- 172.21.15.12:0/2861275417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda40095f0 con 0x7fedbc1062a0 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.371+0000 7fedb8ff9700 1 -- 172.21.15.12:0/2861275417 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7feda4004cd0 con 0x7fedbc1062a0 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.372+0000 7fedb8ff9700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda80a13e0 0x7feda80a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.372+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda80a13e0 0x7feda80a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.372+0000 7fedb8ff9700 1 -- 172.21.15.12:0/2861275417 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7feda40cd5b0 con 0x7fedbc1062a0 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda80a13e0 0x7feda80a38a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fedac009fd0 tx=0x7fedac009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedce1b2b80 1 Processor -- start 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedce1b2b80 1 -- start start 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554ce00 0x5610f554d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554d720 0x5610f5547350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 0x5610f5547cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f549e150 con 0x5610f5547890 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f549bb10 con 0x5610f554ce00 2024-05-08T23:51:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f5497400 con 0x5610f554d720 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedbb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554ce00 0x5610f554d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.373+0000 7fedbb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554ce00 0x5610f554d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49330/0 (socket says 172.21.15.12:49330) 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedbb7fe700 1 -- 172.21.15.12:0/1341990758 learned_addr learned my addr 172.21.15.12:0/1341990758 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedbaffd700 1 --2- 172.21.15.12:0/1341990758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554d720 0x5610f5547350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedbb7fe700 1 -- 172.21.15.12:0/1341990758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554d720 msgr2=0x5610f5547350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/1341990758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554d720 0x5610f5547350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedbb7fe700 1 -- 172.21.15.12:0/1341990758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 msgr2=0x5610f5547cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedbbfff700 1 --2- 172.21.15.12:0/1341990758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 0x5610f5547cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/1341990758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 0x5610f5547cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedbb7fe700 1 -- 172.21.15.12:0/1341990758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610f53183b0 con 0x5610f554ce00 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedbbfff700 1 --2- 172.21.15.12:0/1341990758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 0x5610f5547cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/1341990758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554ce00 0x5610f554d1e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedac012fd0 tx=0x7fedac014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f24781109fdc649 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedb1ffb700 1 -- 172.21.15.12:0/1341990758 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac01b600 con 0x5610f554ce00 2024-05-08T23:51:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedb1ffb700 1 -- 172.21.15.12:0/1341990758 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedac00ee40 con 0x5610f554ce00 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.374+0000 7fedb1ffb700 1 -- 172.21.15.12:0/1341990758 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac026b60 con 0x5610f554ce00 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.375+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1341990758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554ce00 msgr2=0x5610f554d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.375+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1341990758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554ce00 0x5610f554d1e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedac012fd0 tx=0x7fedac014000 comp rx=0 tx=0).stop 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.376+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1341990758 shutdown_connections 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.376+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1341990758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 0x5610f5547cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.376+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1341990758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554ce00 0x5610f554d1e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.376+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1341990758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554d720 0x5610f5547350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.376+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1341990758 >> 172.21.15.12:0/1341990758 conn(0x5610f54a5210 msgr2=0x5610f5541df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.376+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1341990758 shutdown_connections 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.376+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1341990758 wait complete. 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.376+0000 7fedce1b2b80 1 -- start start 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedce1b2b80 1 Processor -- start 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 0x5610f5550b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f55510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f55515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f5433560 con 0x5610f554ce00 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f5433200 con 0x5610f554d720 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f531a6a0 con 0x5610f5547890 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedce1b2b80 10 client.0 _ll_register_callbacks cb 0x5610f543cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f55510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f55510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53550/0 (socket says 172.21.15.12:53550) 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbaffd700 1 -- 172.21.15.12:0/1652893087 learned_addr learned my addr 172.21.15.12:0/1652893087 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 0x5610f5550b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbbfff700 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f55515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbbfff700 1 -- 172.21.15.12:0/1652893087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 msgr2=0x5610f5550b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbbfff700 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 0x5610f5550b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbbfff700 1 -- 172.21.15.12:0/1652893087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 msgr2=0x5610f55510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbbfff700 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f55510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbbfff700 1 -- 172.21.15.12:0/1652893087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac017050 con 0x5610f554d720 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 0x5610f5550b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.377+0000 7fedbaffd700 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f55510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.378+0000 7fedbbfff700 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f55515f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009fd0 tx=0x7fedb4007f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e510f2abe3a7c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.378+0000 7fed9affd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.378+0000 7fed9affd700 1 -- 172.21.15.12:0/1652893087 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb400e040 con 0x5610f554d720 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.378+0000 7fed9affd700 1 -- 172.21.15.12:0/1652893087 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fedb4003e10 con 0x5610f554d720 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.378+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1652893087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610f53f0270 con 0x5610f554d720 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.378+0000 7fed9affd700 1 -- 172.21.15.12:0/1652893087 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedb40127a0 con 0x5610f554d720 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.378+0000 7fedce1b2b80 10 client.26672 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.378+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1652893087 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5610f52c4b60 con 0x5610f554d720 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.378+0000 7fed9affd700 1 -- 172.21.15.12:0/1652893087 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5610f53f0270 con 0x5610f554d720 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.378+0000 7fed9affd700 1 -- 172.21.15.12:0/1652893087 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fedb4014040 con 0x5610f554d720 2024-05-08T23:51:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.379+0000 7fedce1b2b80 10 client.26672 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.379+0000 7fedce1b2b80 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.379+0000 7fedce1b2b80 20 client.26672 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.379+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5610f5561270 0x5610f5563690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.379+0000 7fedce1b2b80 4 client.26672 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.379+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1652893087 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610f5516810 con 0x5610f5561270 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.379+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5610f5561270 0x5610f5563690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.379+0000 7fed9affd700 10 client.26672 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.379+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5610f5561270 0x5610f5563690 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bceaffeb5ce64b8c server_cookie=3afc565cad234db0 in_seq=0 out_seq=0 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fed9affd700 1 -- 172.21.15.12:0/1652893087 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5610f5516810 con 0x5610f5561270 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fed9affd700 10 client.26672 handle_command_reply: tid=0 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 client.26672 shutdown 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1652893087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 msgr2=0x5610f55515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f55515f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009fd0 tx=0x7fedb4007f60 comp rx=0 tx=0).stop 2024-05-08T23:51:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1652893087 shutdown_connections 2024-05-08T23:51:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5610f5561270 0x5610f5563690 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f55515f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 0x5610f5550b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fed9affd700 0 client.26672 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1652893087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f55510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1652893087 >> 172.21.15.12:0/1652893087 conn(0x5610f54a5210 msgr2=0x5610f55415a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1652893087 shutdown_connections 2024-05-08T23:51:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1652893087 wait complete. 2024-05-08T23:51:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.380+0000 7fedce1b2b80 20 client.26672 trim_cache size 0 max 16384 2024-05-08T23:51:28.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.400+0000 7fedce1b2b80 1 Processor -- start 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.400+0000 7fedce1b2b80 1 -- start start 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.400+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 0x5610f555cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.400+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f5547c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.400+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5541d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.400+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f55521e0 con 0x5610f554ce00 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.400+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f555dde0 con 0x5610f554d720 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.400+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f553edc0 con 0x5610f5547890 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f5547c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f5547c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33474/0 (socket says 172.21.15.12:33474) 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbaffd700 1 -- 172.21.15.12:0/1078163183 learned_addr learned my addr 172.21.15.12:0/1078163183 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/1078163183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 0x5610f555cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbbfff700 1 --2- 172.21.15.12:0/1078163183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5541d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbaffd700 1 -- 172.21.15.12:0/1078163183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 msgr2=0x5610f555cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbaffd700 1 --2- 172.21.15.12:0/1078163183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 0x5610f555cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbaffd700 1 -- 172.21.15.12:0/1078163183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 msgr2=0x5610f5541d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbaffd700 1 --2- 172.21.15.12:0/1078163183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5541d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbaffd700 1 -- 172.21.15.12:0/1078163183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610f545d560 con 0x5610f554ce00 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbbfff700 1 --2- 172.21.15.12:0/1078163183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5541d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedbaffd700 1 --2- 172.21.15.12:0/1078163183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f5547c70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feda4009fd0 tx=0x7feda40cad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824903a985fd1cae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.401+0000 7fedb27fc700 1 -- 172.21.15.12:0/1078163183 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda40b2030 con 0x5610f554ce00 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.402+0000 7fedb27fc700 1 -- 172.21.15.12:0/1078163183 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda409a0f0 con 0x5610f554ce00 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.402+0000 7fedb27fc700 1 -- 172.21.15.12:0/1078163183 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4099c80 con 0x5610f554ce00 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.402+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1078163183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 msgr2=0x5610f5547c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.402+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1078163183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f5547c70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feda4009fd0 tx=0x7feda40cad60 comp rx=0 tx=0).stop 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.403+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1078163183 shutdown_connections 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.403+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1078163183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5541d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.403+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1078163183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f5547890 0x5610f555cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.403+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/1078163183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f554ce00 0x5610f5547c70 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.403+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1078163183 >> 172.21.15.12:0/1078163183 conn(0x5610f54a5210 msgr2=0x5610f553e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.403+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1078163183 shutdown_connections 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.403+0000 7fedce1b2b80 1 -- 172.21.15.12:0/1078163183 wait complete. 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.403+0000 7fedce1b2b80 1 -- start start 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.403+0000 7fedce1b2b80 1 Processor -- start 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 0x5610f5558460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554ce00 0x5610f5556150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedce1b2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5556690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f5556d70 con 0x5610f5547890 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f5556ee0 con 0x5610f554d720 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedce1b2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610f5557020 con 0x5610f554ce00 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedce1b2b80 10 client.0 _ll_register_callbacks cb 0x5610f543cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 0x5610f5558460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5556690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554ce00 0x5610f5556150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 0x5610f5558460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33476/0 (socket says 172.21.15.12:33476) 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbb7fe700 1 -- 172.21.15.12:0/3351034347 learned_addr learned my addr 172.21.15.12:0/3351034347 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbbfff700 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5556690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34782/0 (socket says 172.21.15.12:34782) 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbbfff700 1 -- 172.21.15.12:0/3351034347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554ce00 msgr2=0x5610f5556150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbbfff700 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554ce00 0x5610f5556150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbbfff700 1 -- 172.21.15.12:0/3351034347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 msgr2=0x5610f5558460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbbfff700 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 0x5610f5558460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbbfff700 1 -- 172.21.15.12:0/3351034347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda40095f0 con 0x5610f554d720 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbaffd700 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554ce00 0x5610f5556150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fedbbfff700 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5556690 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fedb4002230 tx=0x7fedb4012a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=776a973f1036b7f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fed9affd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.404+0000 7fed9affd700 1 -- 172.21.15.12:0/3351034347 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb400e040 con 0x5610f554d720 2024-05-08T23:51:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fedce1b2b80 1 -- 172.21.15.12:0/3351034347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedac00ee40 con 0x5610f554d720 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fed9affd700 1 -- 172.21.15.12:0/3351034347 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fedb4007420 con 0x5610f554d720 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fed9affd700 1 -- 172.21.15.12:0/3351034347 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedb4016420 con 0x5610f554d720 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fedce1b2b80 10 client.26678 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fedce1b2b80 1 -- 172.21.15.12:0/3351034347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5610f531ccd0 con 0x5610f554d720 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fed9affd700 1 -- 172.21.15.12:0/3351034347 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedb4037020 con 0x5610f554d720 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fed9affd700 1 -- 172.21.15.12:0/3351034347 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fedb4014070 con 0x5610f554d720 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fedce1b2b80 10 client.26678 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fedce1b2b80 10 client.26678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fedce1b2b80 20 client.26678 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5610f55c8e10 0x5610f55cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fedce1b2b80 4 client.26678 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fedce1b2b80 1 -- 172.21.15.12:0/3351034347 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5610f548fec0 con 0x5610f55c8e10 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.405+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5610f55c8e10 0x5610f55cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fed9affd700 10 client.26678 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedbb7fe700 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5610f55c8e10 0x5610f55cb230 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc29440201bd7cc6 server_cookie=73b042dc232ba681 in_seq=0 out_seq=0 2024-05-08T23:51:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fed9affd700 1 -- 172.21.15.12:0/3351034347 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5610f548fec0 con 0x5610f55c8e10 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fed9affd700 10 client.26678 handle_command_reply: tid=0 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 client.26678 shutdown 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 -- 172.21.15.12:0/3351034347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 msgr2=0x5610f5556690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5556690 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fedb4002230 tx=0x7fedb4012a50 comp rx=0 tx=0).stop 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 -- 172.21.15.12:0/3351034347 shutdown_connections 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5610f55c8e10 0x5610f55cb230 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610f554d720 0x5610f5556690 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610f5547890 0x5610f5558460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 --2- 172.21.15.12:0/3351034347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610f554ce00 0x5610f5556150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fed9affd700 0 client.26678 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 -- 172.21.15.12:0/3351034347 >> 172.21.15.12:0/3351034347 conn(0x5610f54a5210 msgr2=0x5610f5547c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 -- 172.21.15.12:0/3351034347 shutdown_connections 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 1 -- 172.21.15.12:0/3351034347 wait complete. 2024-05-08T23:51:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.406+0000 7fedce1b2b80 20 client.26678 trim_cache size 0 max 16384 2024-05-08T23:51:28.409 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:51:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:51:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:51:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "bcc06c4e-129d-4e73-b61d-0bce038d32b5": { 2024-05-08T23:51:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:51:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "bcc06c4e-129d-4e73-b61d-0bce038d32b5", 2024-05-08T23:51:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:51:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:51:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:51:28.409 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 -- 172.21.15.12:0/2861275417 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda80a13e0 msgr2=0x7feda80a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda80a13e0 0x7feda80a38a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fedac009fd0 tx=0x7fedac009260 comp rx=0 tx=0).stop 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 -- 172.21.15.12:0/2861275417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedbc1062a0 msgr2=0x7fedbc1003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedbc1062a0 0x7fedbc1003a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7feda40059d0 tx=0x7feda400b9c0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 -- 172.21.15.12:0/2861275417 shutdown_connections 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda80a13e0 0x7feda80a38a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedbc101b20 0x7fedbc100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedbc0697a0 0x7fedbc0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 --2- 172.21.15.12:0/2861275417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedbc1062a0 0x7fedbc1003a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 -- 172.21.15.12:0/2861275417 >> 172.21.15.12:0/2861275417 conn(0x7fedbc076fa0 msgr2=0x7fedbc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 -- 172.21.15.12:0/2861275417 shutdown_connections 2024-05-08T23:51:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:28.407+0000 7fedc1ff8700 1 -- 172.21.15.12:0/2861275417 wait complete. 2024-05-08T23:51:28.416 INFO:tasks.cephfs.filesystem:scrub status for tag:bcc06c4e-129d-4e73-b61d-0bce038d32b5 - {'path': '/', 'tag': 'bcc06c4e-129d-4e73-b61d-0bce038d32b5', 'options': 'recursive,force'} 2024-05-08T23:51:28.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:28 smithi204 ceph-mon[47373]: pgmap v156: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 399 op/s 2024-05-08T23:51:28.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:28 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:51:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:28 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:28 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:51:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:28 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:28 smithi160 ceph-mon[46152]: pgmap v156: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 399 op/s 2024-05-08T23:51:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:28 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:51:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:28 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:28 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:51:28.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:28 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:28.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 269604 11.13 MB/sec execute 84 sec latency 33.801 ms 2024-05-08T23:51:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:28 smithi012 ceph-mon[42503]: pgmap v156: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 399 op/s 2024-05-08T23:51:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:28 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:51:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:28 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:28 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:51:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:28 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:29.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:29 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:51:29.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:29 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:51:29.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:29 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:29.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:29 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:29 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:51:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:29 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:51:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:29 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:29 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:29.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 271193 11.08 MB/sec execute 85 sec latency 13.091 ms 2024-05-08T23:51:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:29 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:51:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:29 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:51:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:29 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:29 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:51:30.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:30 smithi204 ceph-mon[47373]: pgmap v157: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 239 op/s 2024-05-08T23:51:30.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:30 smithi160 ceph-mon[46152]: pgmap v157: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 239 op/s 2024-05-08T23:51:30.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 273518 11.12 MB/sec execute 86 sec latency 13.643 ms 2024-05-08T23:51:30.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:30 smithi012 ceph-mon[42503]: pgmap v157: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 239 op/s 2024-05-08T23:51:31.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 275554 11.07 MB/sec execute 87 sec latency 53.155 ms 2024-05-08T23:51:32.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:32 smithi204 ceph-mon[47373]: pgmap v158: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 409 op/s 2024-05-08T23:51:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:32 smithi160 ceph-mon[46152]: pgmap v158: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 409 op/s 2024-05-08T23:51:32.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 277282 11.08 MB/sec execute 88 sec latency 13.267 ms 2024-05-08T23:51:32.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:32 smithi012 ceph-mon[42503]: pgmap v158: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 409 op/s 2024-05-08T23:51:33.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:33 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:51:33.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:33 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:51:33.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:33 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:51:33.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:33 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:51:33.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:33 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:51:33.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:33 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:51:33.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:33 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:51:33.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:33 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:51:33.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:33 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:51:33.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:33 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:51:33.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 279703 11.10 MB/sec execute 89 sec latency 19.468 ms 2024-05-08T23:51:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:33 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:51:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:33 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:51:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:33 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:51:33.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:33 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:51:33.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:33 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:51:34.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:34 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:51:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:34 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:51:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:34 smithi204 ceph-mon[47373]: pgmap v159: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 286 op/s 2024-05-08T23:51:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:34 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:51:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:34 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:51:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:34 smithi160 ceph-mon[46152]: pgmap v159: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 286 op/s 2024-05-08T23:51:34.863 INFO:tasks.workunit.client.0.smithi012.stdout: 1 281232 11.06 MB/sec execute 90 sec latency 17.369 ms 2024-05-08T23:51:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:34 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:51:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:34 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:51:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:34 smithi012 ceph-mon[42503]: pgmap v159: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 286 op/s 2024-05-08T23:51:35.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 283924 11.12 MB/sec execute 91 sec latency 12.925 ms 2024-05-08T23:51:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:36 smithi204 ceph-mon[47373]: pgmap v160: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 286 op/s 2024-05-08T23:51:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:36 smithi160 ceph-mon[46152]: pgmap v160: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 286 op/s 2024-05-08T23:51:36.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 285280 11.07 MB/sec execute 92 sec latency 45.260 ms 2024-05-08T23:51:36.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:36 smithi012 ceph-mon[42503]: pgmap v160: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 286 op/s 2024-05-08T23:51:37.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 287668 11.10 MB/sec execute 93 sec latency 13.564 ms 2024-05-08T23:51:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:38 smithi204 ceph-mon[47373]: pgmap v161: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 401 op/s 2024-05-08T23:51:38.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:38 smithi160 ceph-mon[46152]: pgmap v161: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 401 op/s 2024-05-08T23:51:38.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 289730 11.06 MB/sec execute 94 sec latency 23.534 ms 2024-05-08T23:51:38.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:38 smithi012 ceph-mon[42503]: pgmap v161: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 401 op/s 2024-05-08T23:51:39.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 291333 11.07 MB/sec execute 95 sec latency 16.532 ms 2024-05-08T23:51:40.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:40 smithi204 ceph-mon[47373]: pgmap v162: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 285 op/s 2024-05-08T23:51:40.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:40 smithi160 ceph-mon[46152]: pgmap v162: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 285 op/s 2024-05-08T23:51:40.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 293753 11.07 MB/sec execute 96 sec latency 12.388 ms 2024-05-08T23:51:40.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:40 smithi012 ceph-mon[42503]: pgmap v162: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 285 op/s 2024-05-08T23:51:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.497+0000 7f57af4c9700 1 -- 172.21.15.12:0/742436122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a8101110 msgr2=0x7f57a81014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.497+0000 7f57af4c9700 1 --2- 172.21.15.12:0/742436122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a8101110 0x7f57a81014f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f57a40098f0 tx=0x7f57a4009c00 comp rx=0 tx=0).stop 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.497+0000 7f57af4c9700 1 -- 172.21.15.12:0/742436122 shutdown_connections 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.497+0000 7f57af4c9700 1 --2- 172.21.15.12:0/742436122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57a8106230 0x7f57a810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.497+0000 7f57af4c9700 1 --2- 172.21.15.12:0/742436122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a8101110 0x7f57a81014f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.497+0000 7f57af4c9700 1 --2- 172.21.15.12:0/742436122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57a8101ac0 0x7f57a8105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.497+0000 7f57af4c9700 1 -- 172.21.15.12:0/742436122 >> 172.21.15.12:0/742436122 conn(0x7f57a80fc9b0 msgr2=0x7f57a80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.498+0000 7f57af4c9700 1 -- 172.21.15.12:0/742436122 shutdown_connections 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.498+0000 7f57af4c9700 1 -- 172.21.15.12:0/742436122 wait complete. 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.498+0000 7f57af4c9700 1 Processor -- start 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.498+0000 7f57af4c9700 1 -- start start 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.498+0000 7f57af4c9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a8101110 0x7f57a8197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.498+0000 7f57af4c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57a8106230 0x7f57a81979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.498+0000 7f57af4c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57a819bcf0 0x7f57a819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.498+0000 7f57af4c9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a819c770 con 0x7f57a819bcf0 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.498+0000 7f57af4c9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a819c8e0 con 0x7f57a8106230 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.498+0000 7f57af4c9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a819caa0 con 0x7f57a8101110 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57aca64700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57a8106230 0x7f57a81979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57ad265700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a8101110 0x7f57a8197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57ada66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57a819bcf0 0x7f57a819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57aca64700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57a8106230 0x7f57a81979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:35088/0 (socket says 172.21.15.12:35088) 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57aca64700 1 -- 172.21.15.12:0/1373120513 learned_addr learned my addr 172.21.15.12:0/1373120513 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57ad265700 1 -- 172.21.15.12:0/1373120513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57a8106230 msgr2=0x7f57a81979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57ad265700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57a8106230 0x7f57a81979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57ad265700 1 -- 172.21.15.12:0/1373120513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57a819bcf0 msgr2=0x7f57a819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57ad265700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57a819bcf0 0x7f57a819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57ad265700 1 -- 172.21.15.12:0/1373120513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57980095f0 con 0x7f57a8101110 2024-05-08T23:51:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57ada66700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57a819bcf0 0x7f57a819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57ad265700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a8101110 0x7f57a8197460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009fd0 tx=0x7f57a400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f579e7fc700 1 -- 172.21.15.12:0/1373120513 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a401c070 con 0x7f57a8101110 2024-05-08T23:51:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f579e7fc700 1 -- 172.21.15.12:0/1373120513 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a400b9c0 con 0x7f57a8101110 2024-05-08T23:51:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.499+0000 7f57af4c9700 1 -- 172.21.15.12:0/1373120513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a4009520 con 0x7f57a8101110 2024-05-08T23:51:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.500+0000 7f579e7fc700 1 -- 172.21.15.12:0/1373120513 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a40038e0 con 0x7f57a8101110 2024-05-08T23:51:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.500+0000 7f57af4c9700 1 -- 172.21.15.12:0/1373120513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a81a4230 con 0x7f57a8101110 2024-05-08T23:51:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.502+0000 7f579e7fc700 1 -- 172.21.15.12:0/1373120513 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f57a4020030 con 0x7f57a8101110 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.502+0000 7f579e7fc700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57940a1310 0x7f57940a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.502+0000 7f579e7fc700 1 -- 172.21.15.12:0/1373120513 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f57a40cd9a0 con 0x7f57a8101110 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.502+0000 7f57aca64700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57940a1310 0x7f57940a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.503+0000 7f57aca64700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57940a1310 0x7f57940a37d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f57a8198960 tx=0x7f5798009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.503+0000 7f57bb683b80 1 Processor -- start 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57bb683b80 1 -- start start 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620a950 0x563e2620ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e26204f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e26205480 0x563e262058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57bb683b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2615c0f0 con 0x563e2620a950 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57bb683b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e26159ab0 con 0x563e2620b270 2024-05-08T23:51:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57bb683b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e261553a0 con 0x563e26205480 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ad265700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620a950 0x563e2620ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ad265700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620a950 0x563e2620ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41018/0 (socket says 172.21.15.12:41018) 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ad265700 1 -- 172.21.15.12:0/3749421324 learned_addr learned my addr 172.21.15.12:0/3749421324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ada66700 1 --2- 172.21.15.12:0/3749421324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e26205480 0x563e262058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57aca64700 1 --2- 172.21.15.12:0/3749421324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e26204f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ad265700 1 -- 172.21.15.12:0/3749421324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e26205480 msgr2=0x563e262058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ad265700 1 --2- 172.21.15.12:0/3749421324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e26205480 0x563e262058a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ad265700 1 -- 172.21.15.12:0/3749421324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 msgr2=0x563e26204f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ad265700 1 --2- 172.21.15.12:0/3749421324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e26204f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ad265700 1 -- 172.21.15.12:0/3749421324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e25fd6350 con 0x563e2620a950 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ada66700 1 --2- 172.21.15.12:0/3749421324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e26205480 0x563e262058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.504+0000 7f57ad265700 1 --2- 172.21.15.12:0/3749421324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620a950 0x563e2620ad30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f57a40a2a40 tx=0x7f57a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb528eb3e4f6d49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.505+0000 7f57937fe700 1 -- 172.21.15.12:0/3749421324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a409b280 con 0x563e2620a950 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.505+0000 7f57937fe700 1 -- 172.21.15.12:0/3749421324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a409ab80 con 0x563e2620a950 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.505+0000 7f57937fe700 1 -- 172.21.15.12:0/3749421324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4097290 con 0x563e2620a950 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.505+0000 7f57bb683b80 1 -- 172.21.15.12:0/3749421324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620a950 msgr2=0x563e2620ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.505+0000 7f57bb683b80 1 --2- 172.21.15.12:0/3749421324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620a950 0x563e2620ad30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f57a40a2a40 tx=0x7f57a4009c00 comp rx=0 tx=0).stop 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.505+0000 7f57bb683b80 1 -- 172.21.15.12:0/3749421324 shutdown_connections 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.505+0000 7f57bb683b80 1 --2- 172.21.15.12:0/3749421324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e26205480 0x563e262058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.505+0000 7f57bb683b80 1 --2- 172.21.15.12:0/3749421324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620a950 0x563e2620ad30 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.505+0000 7f57bb683b80 1 --2- 172.21.15.12:0/3749421324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e26204f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.505+0000 7f57bb683b80 1 -- 172.21.15.12:0/3749421324 >> 172.21.15.12:0/3749421324 conn(0x563e261631b0 msgr2=0x563e261ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.506+0000 7f57bb683b80 1 -- 172.21.15.12:0/3749421324 shutdown_connections 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.506+0000 7f57bb683b80 1 -- 172.21.15.12:0/3749421324 wait complete. 2024-05-08T23:51:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.506+0000 7f57bb683b80 1 -- start start 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.506+0000 7f57bb683b80 1 Processor -- start 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.506+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e26205480 0x563e26211170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.506+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e2620e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.506+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620b270 0x563e2620ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.506+0000 7f57bb683b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e260f1500 con 0x563e2620b270 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.506+0000 7f57bb683b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e260f11a0 con 0x563e26205480 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.506+0000 7f57bb683b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e25fd8640 con 0x563e2620a950 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57bb683b80 10 client.0 _ll_register_callbacks cb 0x563e260faad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57ada66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620b270 0x563e2620ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57aca64700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e2620e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57ada66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620b270 0x563e2620ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41022/0 (socket says 172.21.15.12:41022) 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57ada66700 1 -- 172.21.15.12:0/594231960 learned_addr learned my addr 172.21.15.12:0/594231960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57ad265700 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e26205480 0x563e26211170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57aca64700 1 -- 172.21.15.12:0/594231960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e26205480 msgr2=0x563e26211170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57aca64700 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e26205480 0x563e26211170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57aca64700 1 -- 172.21.15.12:0/594231960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620b270 msgr2=0x563e2620ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57aca64700 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620b270 0x563e2620ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57aca64700 1 -- 172.21.15.12:0/594231960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a0003a60 con 0x563e2620a950 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57ad265700 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e26205480 0x563e26211170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57ada66700 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620b270 0x563e2620ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57aca64700 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e2620e4b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5798011b60 tx=0x7f57980047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b99468e41695c974 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f5790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f5790ff9700 1 -- 172.21.15.12:0/594231960 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579800e900 con 0x563e2620a950 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f5790ff9700 1 -- 172.21.15.12:0/594231960 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f579800ea60 con 0x563e2620a950 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.507+0000 7f57bb683b80 1 -- 172.21.15.12:0/594231960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e260ae210 con 0x563e2620a950 2024-05-08T23:51:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f5790ff9700 1 -- 172.21.15.12:0/594231960 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f579800ec20 con 0x563e2620a950 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f57bb683b80 10 client.26674 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f57bb683b80 1 -- 172.21.15.12:0/594231960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f57a81a4230 con 0x563e2620a950 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f5790ff9700 1 -- 172.21.15.12:0/594231960 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e260ae210 con 0x563e2620a950 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f5790ff9700 1 -- 172.21.15.12:0/594231960 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5798031440 con 0x563e2620a950 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f57bb683b80 10 client.26674 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f57bb683b80 10 client.26674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f57bb683b80 20 client.26674 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f57bb683b80 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e2621e7d0 0x563e26220bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f57bb683b80 4 client.26674 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f57bb683b80 1 -- 172.21.15.12:0/594231960 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e261d47b0 con 0x563e2621e7d0 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.508+0000 7f57ad265700 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e2621e7d0 0x563e26220bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.509+0000 7f5790ff9700 10 client.26674 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.509+0000 7f57ad265700 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e2621e7d0 0x563e26220bf0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd63b0cf424f2088 server_cookie=a3881553e8429991 in_seq=0 out_seq=0 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.509+0000 7f5790ff9700 1 -- 172.21.15.12:0/594231960 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e261d47b0 con 0x563e2621e7d0 2024-05-08T23:51:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.509+0000 7f5790ff9700 10 client.26674 handle_command_reply: tid=0 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.509+0000 7f57bb683b80 1 client.26674 shutdown 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 1 -- 172.21.15.12:0/594231960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 msgr2=0x563e2620e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e2620e4b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5798011b60 tx=0x7f57980047c0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 1 -- 172.21.15.12:0/594231960 shutdown_connections 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e2620b270 0x563e2620ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e2621e7d0 0x563e26220bf0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e26205480 0x563e26211170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 1 --2- 172.21.15.12:0/594231960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e2620e4b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f5790ff9700 0 client.26674 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 1 -- 172.21.15.12:0/594231960 >> 172.21.15.12:0/594231960 conn(0x563e261631b0 msgr2=0x563e26203f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 1 -- 172.21.15.12:0/594231960 shutdown_connections 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 1 -- 172.21.15.12:0/594231960 wait complete. 2024-05-08T23:51:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.510+0000 7f57bb683b80 20 client.26674 trim_cache size 0 max 16384 2024-05-08T23:51:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.527+0000 7f57bb683b80 1 Processor -- start 2024-05-08T23:51:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.527+0000 7f57bb683b80 1 -- start start 2024-05-08T23:51:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 0x563e261ff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e262157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e26215ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57bb683b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e26200ac0 con 0x563e26205480 2024-05-08T23:51:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57bb683b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e26225e10 con 0x563e2620b270 2024-05-08T23:51:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57bb683b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2620ca30 con 0x563e2620a950 2024-05-08T23:51:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57ad265700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 0x563e261ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57ada66700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e26215ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57ad265700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 0x563e261ff910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41024/0 (socket says 172.21.15.12:41024) 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57ada66700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e26215ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:35110/0 (socket says 172.21.15.12:35110) 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57ad265700 1 -- 172.21.15.12:0/1656819945 learned_addr learned my addr 172.21.15.12:0/1656819945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57aca64700 1 --2- 172.21.15.12:0/1656819945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e262157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57aca64700 1 -- 172.21.15.12:0/1656819945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 msgr2=0x563e26215ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57aca64700 1 --2- 172.21.15.12:0/1656819945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e26215ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57aca64700 1 -- 172.21.15.12:0/1656819945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 msgr2=0x563e261ff910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57aca64700 1 --2- 172.21.15.12:0/1656819945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 0x563e261ff910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.528+0000 7f57aca64700 1 -- 172.21.15.12:0/1656819945 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e25f82b00 con 0x563e2620a950 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.529+0000 7f57aca64700 1 --2- 172.21.15.12:0/1656819945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e262157a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57980317a0 tx=0x7f5798004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db59ec53ebf0bdc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.529+0000 7f5793fff700 1 -- 172.21.15.12:0/1656819945 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798031a90 con 0x563e2620a950 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.529+0000 7f5793fff700 1 -- 172.21.15.12:0/1656819945 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5798031440 con 0x563e2620a950 2024-05-08T23:51:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.529+0000 7f5793fff700 1 -- 172.21.15.12:0/1656819945 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980286d0 con 0x563e2620a950 2024-05-08T23:51:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 -- 172.21.15.12:0/1656819945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 msgr2=0x563e262157a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 --2- 172.21.15.12:0/1656819945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e262157a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57980317a0 tx=0x7f5798004740 comp rx=0 tx=0).stop 2024-05-08T23:51:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 -- 172.21.15.12:0/1656819945 shutdown_connections 2024-05-08T23:51:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 --2- 172.21.15.12:0/1656819945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e26215ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 --2- 172.21.15.12:0/1656819945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 0x563e261ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 --2- 172.21.15.12:0/1656819945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e262157a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 -- 172.21.15.12:0/1656819945 >> 172.21.15.12:0/1656819945 conn(0x563e261631b0 msgr2=0x563e2621a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 -- 172.21.15.12:0/1656819945 shutdown_connections 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 -- 172.21.15.12:0/1656819945 wait complete. 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 -- start start 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.530+0000 7f57bb683b80 1 Processor -- start 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 0x563e26213760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e26213ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57bb683b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e262141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57bb683b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e26273d60 con 0x563e26205480 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57bb683b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e26273ed0 con 0x563e2620b270 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57bb683b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e26274090 con 0x563e2620a950 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57bb683b80 10 client.0 _ll_register_callbacks cb 0x563e260faad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57ad265700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 0x563e26213760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57ada66700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e262141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57ad265700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 0x563e26213760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41036/0 (socket says 172.21.15.12:41036) 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57ad265700 1 -- 172.21.15.12:0/975526478 learned_addr learned my addr 172.21.15.12:0/975526478 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57aca64700 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e26213ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57ada66700 1 -- 172.21.15.12:0/975526478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 msgr2=0x563e26213ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57ada66700 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e26213ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57ada66700 1 -- 172.21.15.12:0/975526478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 msgr2=0x563e26213760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57ada66700 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 0x563e26213760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57ada66700 1 -- 172.21.15.12:0/975526478 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a81a4230 con 0x563e2620b270 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.531+0000 7f57aca64700 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e26213ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f57ada66700 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e262141e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57a0004940 tx=0x7f57a00077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e6ecf60c040fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f5790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f5790ff9700 1 -- 172.21.15.12:0/975526478 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a001c070 con 0x563e2620b270 2024-05-08T23:51:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f5790ff9700 1 -- 172.21.15.12:0/975526478 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f57a0004bb0 con 0x563e2620b270 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f57bb683b80 1 -- 172.21.15.12:0/975526478 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57a409ab80 con 0x563e2620b270 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f5790ff9700 1 -- 172.21.15.12:0/975526478 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57a0007e50 con 0x563e2620b270 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f57bb683b80 10 client.26696 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f57bb683b80 1 -- 172.21.15.12:0/975526478 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563e2611b500 con 0x563e2620b270 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f5790ff9700 1 -- 172.21.15.12:0/975526478 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a409ab80 con 0x563e2620b270 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f5790ff9700 1 -- 172.21.15.12:0/975526478 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f57a0020670 con 0x563e2620b270 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f57bb683b80 10 client.26696 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f57bb683b80 10 client.26696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.532+0000 7f57bb683b80 20 client.26696 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.533+0000 7f57bb683b80 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e262758b0 0x563e26277cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.533+0000 7f57bb683b80 4 client.26696 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.533+0000 7f57bb683b80 1 -- 172.21.15.12:0/975526478 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e2614de60 con 0x563e262758b0 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.533+0000 7f57ad265700 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e262758b0 0x563e26277cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.533+0000 7f5790ff9700 10 client.26696 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.533+0000 7f57ad265700 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e262758b0 0x563e26277cd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ddc65d309eabf3 server_cookie=d9ce1a89ed2bb9cb in_seq=0 out_seq=0 2024-05-08T23:51:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f5790ff9700 1 -- 172.21.15.12:0/975526478 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e2614de60 con 0x563e262758b0 2024-05-08T23:51:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f5790ff9700 10 client.26696 handle_command_reply: tid=0 2024-05-08T23:51:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 client.26696 shutdown 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 -- 172.21.15.12:0/975526478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 msgr2=0x563e262141e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e262141e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57a0004940 tx=0x7f57a00077e0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 -- 172.21.15.12:0/975526478 shutdown_connections 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e262758b0 0x563e26277cd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e2620b270 0x563e262141e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e26205480 0x563e26213760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 --2- 172.21.15.12:0/975526478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e2620a950 0x563e26213ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f5790ff9700 0 client.26696 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 -- 172.21.15.12:0/975526478 >> 172.21.15.12:0/975526478 conn(0x563e261631b0 msgr2=0x563e2620ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 -- 172.21.15.12:0/975526478 shutdown_connections 2024-05-08T23:51:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.589+0000 7f57bb683b80 1 -- 172.21.15.12:0/975526478 wait complete. 2024-05-08T23:51:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57bb683b80 20 client.26696 trim_cache size 0 max 16384 2024-05-08T23:51:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:51:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:51:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:51:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:51:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 -- 172.21.15.12:0/1373120513 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57940a1310 msgr2=0x7f57940a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57940a1310 0x7f57940a37d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f57a8198960 tx=0x7f5798009310 comp rx=0 tx=0).stop 2024-05-08T23:51:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 -- 172.21.15.12:0/1373120513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a8101110 msgr2=0x7f57a8197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a8101110 0x7f57a8197460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009fd0 tx=0x7f57a400fa00 comp rx=0 tx=0).stop 2024-05-08T23:51:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 -- 172.21.15.12:0/1373120513 shutdown_connections 2024-05-08T23:51:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57940a1310 0x7f57940a37d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57a819bcf0 0x7f57a819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a8101110 0x7f57a8197460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 --2- 172.21.15.12:0/1373120513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57a8106230 0x7f57a81979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 -- 172.21.15.12:0/1373120513 >> 172.21.15.12:0/1373120513 conn(0x7f57a80fc9b0 msgr2=0x7f57a80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 -- 172.21.15.12:0/1373120513 shutdown_connections 2024-05-08T23:51:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:41.590+0000 7f57af4c9700 1 -- 172.21.15.12:0/1373120513 wait complete. 2024-05-08T23:51:41.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 294993 11.04 MB/sec execute 97 sec latency 36.489 ms 2024-05-08T23:51:42.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:42 smithi204 ceph-mon[47373]: pgmap v163: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 403 op/s 2024-05-08T23:51:42.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:42 smithi160 ceph-mon[46152]: pgmap v163: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 403 op/s 2024-05-08T23:51:42.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 297638 11.06 MB/sec execute 98 sec latency 20.758 ms 2024-05-08T23:51:42.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:42 smithi012 ceph-mon[42503]: pgmap v163: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 403 op/s 2024-05-08T23:51:43.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 299376 11.06 MB/sec execute 99 sec latency 11.305 ms 2024-05-08T23:51:44.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:44 smithi204 ceph-mon[47373]: pgmap v164: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2024-05-08T23:51:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:44 smithi160 ceph-mon[46152]: pgmap v164: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2024-05-08T23:51:44.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 301764 11.09 MB/sec execute 100 sec latency 12.110 ms 2024-05-08T23:51:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:44 smithi012 ceph-mon[42503]: pgmap v164: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2024-05-08T23:51:45.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 303392 11.03 MB/sec execute 101 sec latency 13.848 ms 2024-05-08T23:51:46.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:46 smithi204 ceph-mon[47373]: pgmap v165: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2024-05-08T23:51:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:46 smithi160 ceph-mon[46152]: pgmap v165: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2024-05-08T23:51:46.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 305392 11.06 MB/sec execute 102 sec latency 38.484 ms 2024-05-08T23:51:46.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:46 smithi012 ceph-mon[42503]: pgmap v165: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2024-05-08T23:51:47.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 307556 11.03 MB/sec execute 103 sec latency 20.787 ms 2024-05-08T23:51:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:47 smithi012 ceph-mon[42503]: pgmap v166: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 389 op/s 2024-05-08T23:51:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:47 smithi204 ceph-mon[47373]: pgmap v166: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 389 op/s 2024-05-08T23:51:48.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:47 smithi160 ceph-mon[46152]: pgmap v166: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 389 op/s 2024-05-08T23:51:48.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 309159 11.03 MB/sec execute 104 sec latency 13.995 ms 2024-05-08T23:51:49.864 INFO:tasks.workunit.client.0.smithi012.stdout: 1 311969 11.06 MB/sec execute 105 sec latency 17.552 ms 2024-05-08T23:51:50.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:50 smithi204 ceph-mon[47373]: pgmap v167: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 275 op/s 2024-05-08T23:51:50.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:50 smithi160 ceph-mon[46152]: pgmap v167: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 275 op/s 2024-05-08T23:51:50.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 313557 11.05 MB/sec execute 106 sec latency 18.801 ms 2024-05-08T23:51:50.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:50 smithi012 ceph-mon[42503]: pgmap v167: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 275 op/s 2024-05-08T23:51:51.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 315840 11.07 MB/sec execute 107 sec latency 50.278 ms 2024-05-08T23:51:52.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:52 smithi204 ceph-mon[47373]: pgmap v168: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 380 op/s 2024-05-08T23:51:52.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:52 smithi160 ceph-mon[46152]: pgmap v168: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 380 op/s 2024-05-08T23:51:52.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 317223 11.02 MB/sec execute 108 sec latency 18.136 ms 2024-05-08T23:51:52.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:52 smithi012 ceph-mon[42503]: pgmap v168: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 380 op/s 2024-05-08T23:51:53.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 319355 11.04 MB/sec execute 109 sec latency 167.574 ms 2024-05-08T23:51:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.729+0000 7f7819c9e700 1 -- 172.21.15.12:0/746680548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7814105e60 msgr2=0x7f7814106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.729+0000 7f7819c9e700 1 --2- 172.21.15.12:0/746680548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7814105e60 0x7f7814106240 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7804009990 tx=0x7f7804009ca0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.729+0000 7f7819c9e700 1 -- 172.21.15.12:0/746680548 shutdown_connections 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.729+0000 7f7819c9e700 1 --2- 172.21.15.12:0/746680548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7814100760 0x7f781410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.729+0000 7f7819c9e700 1 --2- 172.21.15.12:0/746680548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7814105e60 0x7f7814106240 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.729+0000 7f7819c9e700 1 --2- 172.21.15.12:0/746680548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78140ffda0 0x7f7814100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.729+0000 7f7819c9e700 1 -- 172.21.15.12:0/746680548 >> 172.21.15.12:0/746680548 conn(0x7f7814076f10 msgr2=0x7f7814077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.729+0000 7f7819c9e700 1 -- 172.21.15.12:0/746680548 shutdown_connections 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.729+0000 7f7819c9e700 1 -- 172.21.15.12:0/746680548 wait complete. 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.729+0000 7f7819c9e700 1 Processor -- start 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f7819c9e700 1 -- start start 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f7819c9e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78140ffda0 0x7f781419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f7819c9e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7814100760 0x7f781419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f7819c9e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7814105e60 0x7f78141960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f7819c9e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7814196760 con 0x7f78140ffda0 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f7819c9e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78141968d0 con 0x7f7814105e60 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f7819c9e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7814196a90 con 0x7f7814100760 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f78137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78140ffda0 0x7f781419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f78137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78140ffda0 0x7f781419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46718/0 (socket says 172.21.15.12:46718) 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f78137fe700 1 -- 172.21.15.12:0/3204584093 learned_addr learned my addr 172.21.15.12:0/3204584093 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f7813fff700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7814105e60 0x7f78141960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f780affd700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7814100760 0x7f781419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f78137fe700 1 -- 172.21.15.12:0/3204584093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7814100760 msgr2=0x7f781419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f78137fe700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7814100760 0x7f781419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f78137fe700 1 -- 172.21.15.12:0/3204584093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7814105e60 msgr2=0x7f78141960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f78137fe700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7814105e60 0x7f78141960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f78137fe700 1 -- 172.21.15.12:0/3204584093 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c009c40 con 0x7f78140ffda0 2024-05-08T23:51:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.730+0000 7f780affd700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7814100760 0x7f781419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:51:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.731+0000 7f78137fe700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78140ffda0 0x7f781419adf0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7804000c00 tx=0x7f7804005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.731+0000 7f78117fa700 1 -- 172.21.15.12:0/3204584093 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780401f070 con 0x7f78140ffda0 2024-05-08T23:51:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.731+0000 7f78117fa700 1 -- 172.21.15.12:0/3204584093 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f780400f460 con 0x7f78140ffda0 2024-05-08T23:51:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.731+0000 7f7819c9e700 1 -- 172.21.15.12:0/3204584093 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78040095f0 con 0x7f78140ffda0 2024-05-08T23:51:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.731+0000 7f78117fa700 1 -- 172.21.15.12:0/3204584093 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804025680 con 0x7f78140ffda0 2024-05-08T23:51:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.731+0000 7f7819c9e700 1 -- 172.21.15.12:0/3204584093 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7814068280 con 0x7f78140ffda0 2024-05-08T23:51:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.733+0000 7f78117fa700 1 -- 172.21.15.12:0/3204584093 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7804023070 con 0x7f78140ffda0 2024-05-08T23:51:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.734+0000 7f78117fa700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78000a1290 0x7f78000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.734+0000 7f78117fa700 1 -- 172.21.15.12:0/3204584093 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f78040cfee0 con 0x7f78140ffda0 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.734+0000 7f780affd700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78000a1290 0x7f78000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.735+0000 7f780affd700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78000a1290 0x7f78000a3750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f77fc005950 tx=0x7f77fc0058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.735+0000 7f7825e58b80 1 Processor -- start 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.735+0000 7f7825e58b80 1 -- start start 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.735+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.735+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebd2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.735+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebd32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.735+0000 7f7825e58b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913eb210f0 con 0x55913ebcee60 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.735+0000 7f7825e58b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913eb1eab0 con 0x55913ebce540 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.735+0000 7f7825e58b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913eb1a3a0 con 0x55913ebd2e40 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f780affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebd2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f78137fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f780affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebd2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46726/0 (socket says 172.21.15.12:46726) 2024-05-08T23:51:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f780affd700 1 -- 172.21.15.12:0/3842677184 learned_addr learned my addr 172.21.15.12:0/3842677184 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f7813fff700 1 --2- 172.21.15.12:0/3842677184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebd32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f78137fe700 1 -- 172.21.15.12:0/3842677184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 msgr2=0x55913ebd32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f78137fe700 1 --2- 172.21.15.12:0/3842677184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebd32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f78137fe700 1 -- 172.21.15.12:0/3842677184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 msgr2=0x55913ebd2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f78137fe700 1 --2- 172.21.15.12:0/3842677184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebd2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f78137fe700 1 -- 172.21.15.12:0/3842677184 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55913e99b350 con 0x55913ebce540 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f780affd700 1 --2- 172.21.15.12:0/3842677184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebd2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f78137fe700 1 --2- 172.21.15.12:0/3842677184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebce920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78040cd670 tx=0x7f780409cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d96f84026accdfbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f7809ffb700 1 -- 172.21.15.12:0/3842677184 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780409d0c0 con 0x55913ebce540 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f7809ffb700 1 -- 172.21.15.12:0/3842677184 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78040a0ef0 con 0x55913ebce540 2024-05-08T23:51:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.736+0000 7f7809ffb700 1 -- 172.21.15.12:0/3842677184 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804099920 con 0x55913ebce540 2024-05-08T23:51:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.737+0000 7f7825e58b80 1 -- 172.21.15.12:0/3842677184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 msgr2=0x55913ebce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.737+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3842677184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebce920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78040cd670 tx=0x7f780409cf10 comp rx=0 tx=0).stop 2024-05-08T23:51:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.737+0000 7f7825e58b80 1 -- 172.21.15.12:0/3842677184 shutdown_connections 2024-05-08T23:51:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.737+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3842677184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebd32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.737+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3842677184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebce920 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.737+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3842677184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebd2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.737+0000 7f7825e58b80 1 -- 172.21.15.12:0/3842677184 >> 172.21.15.12:0/3842677184 conn(0x55913eb281b0 msgr2=0x55913ebc4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.737+0000 7f7825e58b80 1 -- 172.21.15.12:0/3842677184 shutdown_connections 2024-05-08T23:51:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.737+0000 7f7825e58b80 1 -- 172.21.15.12:0/3842677184 wait complete. 2024-05-08T23:51:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f7825e58b80 1 -- start start 2024-05-08T23:51:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f7825e58b80 1 Processor -- start 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebce540 0x55913ebda5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebcee60 0x55913ebd78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebd8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f7825e58b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913eab6500 con 0x55913ebce540 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f7825e58b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913eab61a0 con 0x55913ebcee60 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f7825e58b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913e99d640 con 0x55913ebd2e40 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f7825e58b80 10 client.0 _ll_register_callbacks cb 0x55913eabfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f78137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebce540 0x55913ebda5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f780affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebcee60 0x55913ebd78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f78137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebce540 0x55913ebda5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46728/0 (socket says 172.21.15.12:46728) 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f78137fe700 1 -- 172.21.15.12:0/3094538999 learned_addr learned my addr 172.21.15.12:0/3094538999 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.738+0000 7f780affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebcee60 0x55913ebd78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47378/0 (socket says 172.21.15.12:47378) 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f7813fff700 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebd8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f78137fe700 1 -- 172.21.15.12:0/3094538999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 msgr2=0x55913ebd8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f78137fe700 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebd8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f78137fe700 1 -- 172.21.15.12:0/3094538999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebcee60 msgr2=0x55913ebd78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f78137fe700 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebcee60 0x55913ebd78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f78137fe700 1 -- 172.21.15.12:0/3094538999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77fc00fbb0 con 0x55913ebce540 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f780affd700 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebcee60 0x55913ebd78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f78137fe700 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebce540 0x55913ebda5e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f780409cdc0 tx=0x7f7804099a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f56929a7c0d2a9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f77f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f77f2ffd700 1 -- 172.21.15.12:0/3094538999 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040a54b0 con 0x55913ebce540 2024-05-08T23:51:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f77f2ffd700 1 -- 172.21.15.12:0/3094538999 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f780400f8d0 con 0x55913ebce540 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f7825e58b80 1 -- 172.21.15.12:0/3094538999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55913ea73210 con 0x55913ebce540 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f77f2ffd700 1 -- 172.21.15.12:0/3094538999 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f780400fa90 con 0x55913ebce540 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f7825e58b80 10 client.16917 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.739+0000 7f7825e58b80 1 -- 172.21.15.12:0/3094538999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f7814068280 con 0x55913ebce540 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.740+0000 7f77f2ffd700 1 -- 172.21.15.12:0/3094538999 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55913ea73210 con 0x55913ebce540 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.740+0000 7f77f2ffd700 1 -- 172.21.15.12:0/3094538999 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f780409c5b0 con 0x55913ebce540 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.740+0000 7f7825e58b80 10 client.16917 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.740+0000 7f7825e58b80 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.740+0000 7f7825e58b80 20 client.16917 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.740+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55913ebe7800 0x55913ebe9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.740+0000 7f7825e58b80 4 client.16917 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.740+0000 7f7825e58b80 1 -- 172.21.15.12:0/3094538999 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55913eb997b0 con 0x55913ebe7800 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.740+0000 7f780affd700 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55913ebe7800 0x55913ebe9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.741+0000 7f77f2ffd700 10 client.16917 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.741+0000 7f780affd700 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55913ebe7800 0x55913ebe9bf0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7abe5e1f653319c1 server_cookie=81abd8ffebb9c8e8 in_seq=0 out_seq=0 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.741+0000 7f77f2ffd700 1 -- 172.21.15.12:0/3094538999 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55913eb997b0 con 0x55913ebe7800 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.741+0000 7f77f2ffd700 10 client.16917 handle_command_reply: tid=0 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.741+0000 7f7825e58b80 1 client.16917 shutdown 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.741+0000 7f7825e58b80 1 -- 172.21.15.12:0/3094538999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebce540 msgr2=0x55913ebda5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.741+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebce540 0x55913ebda5e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f780409cdc0 tx=0x7f7804099a80 comp rx=0 tx=0).stop 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.742+0000 7f7825e58b80 1 -- 172.21.15.12:0/3094538999 shutdown_connections 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.742+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebd8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.742+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebce540 0x55913ebda5e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.742+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55913ebe7800 0x55913ebe9bf0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.742+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3094538999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebcee60 0x55913ebd78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.742+0000 7f77f2ffd700 0 client.16917 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.742+0000 7f7825e58b80 1 -- 172.21.15.12:0/3094538999 >> 172.21.15.12:0/3094538999 conn(0x55913eb281b0 msgr2=0x55913ebc8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.742+0000 7f7825e58b80 1 -- 172.21.15.12:0/3094538999 shutdown_connections 2024-05-08T23:51:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.742+0000 7f7825e58b80 1 -- 172.21.15.12:0/3094538999 wait complete. 2024-05-08T23:51:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.742+0000 7f7825e58b80 20 client.16917 trim_cache size 0 max 16384 2024-05-08T23:51:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.759+0000 7f7825e58b80 1 Processor -- start 2024-05-08T23:51:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.759+0000 7f7825e58b80 1 -- start start 2024-05-08T23:51:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebe9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebc4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebc8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7825e58b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913ebdaaa0 con 0x55913ebcee60 2024-05-08T23:51:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7825e58b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913ebbca40 con 0x55913ebce540 2024-05-08T23:51:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7825e58b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913ebc5a30 con 0x55913ebd2e40 2024-05-08T23:51:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f780affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebc4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7813fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebc8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7813fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebc8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57850/0 (socket says 172.21.15.12:57850) 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f780affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebc4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46736/0 (socket says 172.21.15.12:46736) 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7813fff700 1 -- 172.21.15.12:0/3212453036 learned_addr learned my addr 172.21.15.12:0/3212453036 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f78137fe700 1 --2- 172.21.15.12:0/3212453036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebe9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7813fff700 1 -- 172.21.15.12:0/3212453036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 msgr2=0x55913ebe9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7813fff700 1 --2- 172.21.15.12:0/3212453036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebe9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7813fff700 1 -- 172.21.15.12:0/3212453036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 msgr2=0x55913ebc4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7813fff700 1 --2- 172.21.15.12:0/3212453036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebc4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7813fff700 1 -- 172.21.15.12:0/3212453036 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55913e947b00 con 0x55913ebd2e40 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.760+0000 7f7813fff700 1 --2- 172.21.15.12:0/3212453036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebc8f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f780c007630 tx=0x7f780c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d82e768544180154 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.761+0000 7f780a7fc700 1 -- 172.21.15.12:0/3212453036 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c00e040 con 0x55913ebd2e40 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.761+0000 7f780a7fc700 1 -- 172.21.15.12:0/3212453036 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f780c01b070 con 0x55913ebd2e40 2024-05-08T23:51:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.761+0000 7f780a7fc700 1 -- 172.21.15.12:0/3212453036 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c020480 con 0x55913ebd2e40 2024-05-08T23:51:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 -- 172.21.15.12:0/3212453036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 msgr2=0x55913ebc8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3212453036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebc8f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f780c007630 tx=0x7f780c009920 comp rx=0 tx=0).stop 2024-05-08T23:51:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 -- 172.21.15.12:0/3212453036 shutdown_connections 2024-05-08T23:51:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3212453036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebd2e40 0x55913ebc8f30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3212453036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebe9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 --2- 172.21.15.12:0/3212453036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebcee60 0x55913ebc4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 -- 172.21.15.12:0/3212453036 >> 172.21.15.12:0/3212453036 conn(0x55913eb281b0 msgr2=0x55913ebc0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 -- 172.21.15.12:0/3212453036 shutdown_connections 2024-05-08T23:51:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 -- 172.21.15.12:0/3212453036 wait complete. 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 -- start start 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.762+0000 7f7825e58b80 1 Processor -- start 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebdeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebcee60 0x55913ebdacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f7825e58b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebd2e40 0x55913ebdb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f7825e58b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913ebdb740 con 0x55913ebd2e40 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f7825e58b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913ebdb880 con 0x55913ebce540 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f7825e58b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913ebdba40 con 0x55913ebcee60 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f7825e58b80 10 client.0 _ll_register_callbacks cb 0x55913eabfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f78137fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebdeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f7813fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebd2e40 0x55913ebdb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f78137fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebdeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47400/0 (socket says 172.21.15.12:47400) 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f78137fe700 1 -- 172.21.15.12:0/2225602039 learned_addr learned my addr 172.21.15.12:0/2225602039 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f780affd700 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebcee60 0x55913ebdacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f78137fe700 1 -- 172.21.15.12:0/2225602039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebcee60 msgr2=0x55913ebdacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f78137fe700 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebcee60 0x55913ebdacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f78137fe700 1 -- 172.21.15.12:0/2225602039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebd2e40 msgr2=0x55913ebdb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f78137fe700 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebd2e40 0x55913ebdb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f78137fe700 1 -- 172.21.15.12:0/2225602039 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c009280 con 0x55913ebce540 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f78137fe700 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebdeef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f78040098e0 tx=0x7f7804009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd3be407d903d834 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.763+0000 7f77f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f77f2ffd700 1 -- 172.21.15.12:0/2225602039 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040a54b0 con 0x55913ebce540 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f77f2ffd700 1 -- 172.21.15.12:0/2225602039 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f780409d320 con 0x55913ebce540 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f7825e58b80 1 -- 172.21.15.12:0/2225602039 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78040a0ef0 con 0x55913ebce540 2024-05-08T23:51:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f77f2ffd700 1 -- 172.21.15.12:0/2225602039 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f780409d4e0 con 0x55913ebce540 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f7825e58b80 10 client.26714 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f7825e58b80 1 -- 172.21.15.12:0/2225602039 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55913eae0500 con 0x55913ebce540 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f77f2ffd700 1 -- 172.21.15.12:0/2225602039 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78040a0ef0 con 0x55913ebce540 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f77f2ffd700 1 -- 172.21.15.12:0/2225602039 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f78040a5020 con 0x55913ebce540 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f7825e58b80 10 client.26714 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f7825e58b80 10 client.26714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f7825e58b80 20 client.26714 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f7825e58b80 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55913ec3dd10 0x55913ec40100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f7825e58b80 4 client.26714 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f7825e58b80 1 -- 172.21.15.12:0/2225602039 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55913eb12e60 con 0x55913ec3dd10 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.764+0000 7f780affd700 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55913ec3dd10 0x55913ec40100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.765+0000 7f780affd700 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55913ec3dd10 0x55913ec40100 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0519e047b7760f9 server_cookie=c398ed12cb8857f8 in_seq=0 out_seq=0 2024-05-08T23:51:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.765+0000 7f77f2ffd700 10 client.26714 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:54 smithi160 ceph-mon[46152]: pgmap v169: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-05-08T23:51:54.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:54 smithi204 ceph-mon[47373]: pgmap v169: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-05-08T23:51:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.835+0000 7f77f2ffd700 1 -- 172.21.15.12:0/2225602039 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55913eb12e60 con 0x55913ec3dd10 2024-05-08T23:51:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.835+0000 7f77f2ffd700 10 client.26714 handle_command_reply: tid=0 2024-05-08T23:51:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.835+0000 7f7825e58b80 1 client.26714 shutdown 2024-05-08T23:51:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.835+0000 7f7825e58b80 1 -- 172.21.15.12:0/2225602039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 msgr2=0x55913ebdeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.835+0000 7f7825e58b80 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebdeef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f78040098e0 tx=0x7f7804009ca0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7825e58b80 1 -- 172.21.15.12:0/2225602039 shutdown_connections 2024-05-08T23:51:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7825e58b80 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55913ebd2e40 0x55913ebdb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7825e58b80 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55913ec3dd10 0x55913ec40100 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7825e58b80 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55913ebce540 0x55913ebdeef0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7825e58b80 1 --2- 172.21.15.12:0/2225602039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55913ebcee60 0x55913ebdacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f77f2ffd700 0 client.26714 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7825e58b80 1 -- 172.21.15.12:0/2225602039 >> 172.21.15.12:0/2225602039 conn(0x55913eb281b0 msgr2=0x55913ebc60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7825e58b80 1 -- 172.21.15.12:0/2225602039 shutdown_connections 2024-05-08T23:51:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7825e58b80 1 -- 172.21.15.12:0/2225602039 wait complete. 2024-05-08T23:51:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7825e58b80 20 client.26714 trim_cache size 0 max 16384 2024-05-08T23:51:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:51:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:51:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:51:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7819c9e700 1 -- 172.21.15.12:0/3204584093 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78000a1290 msgr2=0x7f78000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7819c9e700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78000a1290 0x7f78000a3750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f77fc005950 tx=0x7f77fc0058e0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7819c9e700 1 -- 172.21.15.12:0/3204584093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78140ffda0 msgr2=0x7f781419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.836+0000 7f7819c9e700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78140ffda0 0x7f781419adf0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7804000c00 tx=0x7f7804005ef0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.837+0000 7f7819c9e700 1 -- 172.21.15.12:0/3204584093 shutdown_connections 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.837+0000 7f7819c9e700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78000a1290 0x7f78000a3750 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.837+0000 7f7819c9e700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7814105e60 0x7f78141960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.837+0000 7f7819c9e700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78140ffda0 0x7f781419adf0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.837+0000 7f7819c9e700 1 --2- 172.21.15.12:0/3204584093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7814100760 0x7f781419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.837+0000 7f7819c9e700 1 -- 172.21.15.12:0/3204584093 >> 172.21.15.12:0/3204584093 conn(0x7f7814076f10 msgr2=0x7f78140fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.837+0000 7f7819c9e700 1 -- 172.21.15.12:0/3204584093 shutdown_connections 2024-05-08T23:51:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:51:54.837+0000 7f7819c9e700 1 -- 172.21.15.12:0/3204584093 wait complete. 2024-05-08T23:51:54.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 320656 11.00 MB/sec execute 110 sec latency 24.654 ms 2024-05-08T23:51:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:54 smithi012 ceph-mon[42503]: pgmap v169: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-05-08T23:51:55.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 323032 11.02 MB/sec execute 111 sec latency 23.005 ms 2024-05-08T23:51:56.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:56 smithi204 ceph-mon[47373]: pgmap v170: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-05-08T23:51:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:56 smithi160 ceph-mon[46152]: pgmap v170: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-05-08T23:51:56.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 324774 10.98 MB/sec execute 112 sec latency 28.138 ms 2024-05-08T23:51:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:56 smithi012 ceph-mon[42503]: pgmap v170: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-05-08T23:51:57.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 326782 11.00 MB/sec execute 113 sec latency 21.821 ms 2024-05-08T23:51:58.417 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:51:58.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.576+0000 7fd0d368c700 1 -- 172.21.15.12:0/3448543267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0cc1000f0 msgr2=0x7fd0cc100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.576+0000 7fd0d368c700 1 --2- 172.21.15.12:0/3448543267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0cc1000f0 0x7fd0cc100570 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0099c0 tx=0x7fd0bc009cd0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.576+0000 7fd0d368c700 1 -- 172.21.15.12:0/3448543267 shutdown_connections 2024-05-08T23:51:58.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.576+0000 7fd0d368c700 1 --2- 172.21.15.12:0/3448543267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0cc100ab0 0x7fd0cc10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.576+0000 7fd0d368c700 1 --2- 172.21.15.12:0/3448543267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0cc105f90 0x7fd0cc106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.576+0000 7fd0d368c700 1 --2- 172.21.15.12:0/3448543267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0cc1000f0 0x7fd0cc100570 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.576+0000 7fd0d368c700 1 -- 172.21.15.12:0/3448543267 >> 172.21.15.12:0/3448543267 conn(0x7fd0cc076ba0 msgr2=0x7fd0cc076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:58.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.576+0000 7fd0d368c700 1 -- 172.21.15.12:0/3448543267 shutdown_connections 2024-05-08T23:51:58.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.576+0000 7fd0d368c700 1 -- 172.21.15.12:0/3448543267 wait complete. 2024-05-08T23:51:58.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d368c700 1 Processor -- start 2024-05-08T23:51:58.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d368c700 1 -- start start 2024-05-08T23:51:58.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d368c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0cc1000f0 0x7fd0cc196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d368c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0cc100ab0 0x7fd0cc1967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d368c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0cc105f90 0x7fd0cc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d368c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc19b190 con 0x7fd0cc1000f0 2024-05-08T23:51:58.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d368c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc19b300 con 0x7fd0cc100ab0 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d368c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc19b4c0 con 0x7fd0cc105f90 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d1428700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0cc1000f0 0x7fd0cc196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d1428700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0cc1000f0 0x7fd0cc196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42040/0 (socket says 172.21.15.12:42040) 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d1428700 1 -- 172.21.15.12:0/2310284911 learned_addr learned my addr 172.21.15.12:0/2310284911 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0cc100ab0 0x7fd0cc1967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.577+0000 7fd0d1c29700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0cc105f90 0x7fd0cc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0d1428700 1 -- 172.21.15.12:0/2310284911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0cc105f90 msgr2=0x7fd0cc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0d1428700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0cc105f90 0x7fd0cc19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0d1428700 1 -- 172.21.15.12:0/2310284911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0cc100ab0 msgr2=0x7fd0cc1967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0d1428700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0cc100ab0 0x7fd0cc1967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0d1428700 1 -- 172.21.15.12:0/2310284911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc0095f0 con 0x7fd0cc1000f0 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0cc100ab0 0x7fd0cc1967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:58.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0d1428700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0cc1000f0 0x7fd0cc196270 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800d980 tx=0x7fd0c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:58.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0c27fc700 1 -- 172.21.15.12:0/2310284911 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8004020 con 0x7fd0cc1000f0 2024-05-08T23:51:58.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0d368c700 1 -- 172.21.15.12:0/2310284911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc19b7a0 con 0x7fd0cc1000f0 2024-05-08T23:51:58.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0d368c700 1 -- 172.21.15.12:0/2310284911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc1a2fe0 con 0x7fd0cc1000f0 2024-05-08T23:51:58.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0c27fc700 1 -- 172.21.15.12:0/2310284911 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0c800f690 con 0x7fd0cc1000f0 2024-05-08T23:51:58.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.578+0000 7fd0c27fc700 1 -- 172.21.15.12:0/2310284911 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80038e0 con 0x7fd0cc1000f0 2024-05-08T23:51:58.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.581+0000 7fd0c27fc700 1 -- 172.21.15.12:0/2310284911 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd0c8003a40 con 0x7fd0cc1000f0 2024-05-08T23:51:58.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.581+0000 7fd0c27fc700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd0b80a1310 0x7fd0b80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.581+0000 7fd0c27fc700 1 -- 172.21.15.12:0/2310284911 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd0c80cc470 con 0x7fd0cc1000f0 2024-05-08T23:51:58.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.581+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd0b80a1310 0x7fd0b80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.582+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd0b80a1310 0x7fd0b80a37d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009fd0 tx=0x7fd0bc005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:58.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.582+0000 7fd0df846b80 1 Processor -- start 2024-05-08T23:51:58.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.582+0000 7fd0df846b80 1 -- start start 2024-05-08T23:51:58.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.582+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe7e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe78350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 0x562c6fe78cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0df846b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fdcf150 con 0x562c6fe7e720 2024-05-08T23:51:58.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0df846b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fdccb10 con 0x562c6fe7de00 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0df846b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fdc8400 con 0x562c6fe78890 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d0c27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe78350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d1428700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe7e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d0c27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe78350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42042/0 (socket says 172.21.15.12:42042) 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d1428700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe7e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:32894/0 (socket says 172.21.15.12:32894) 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d0c27700 1 -- 172.21.15.12:0/871688481 learned_addr learned my addr 172.21.15.12:0/871688481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d1c29700 1 --2- 172.21.15.12:0/871688481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 0x562c6fe78cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d1428700 1 -- 172.21.15.12:0/871688481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 msgr2=0x562c6fe78cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d1428700 1 --2- 172.21.15.12:0/871688481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 0x562c6fe78cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d1428700 1 -- 172.21.15.12:0/871688481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 msgr2=0x562c6fe78350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d1428700 1 --2- 172.21.15.12:0/871688481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe78350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d1428700 1 -- 172.21.15.12:0/871688481 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c6fc493b0 con 0x562c6fe7de00 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d1c29700 1 --2- 172.21.15.12:0/871688481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 0x562c6fe78cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0d1428700 1 --2- 172.21.15.12:0/871688481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe7e1e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80c9c00 tx=0x7fd0c8098f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b9a8a7a2f89254a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:58.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0b77fe700 1 -- 172.21.15.12:0/871688481 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80995c0 con 0x562c6fe7de00 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.583+0000 7fd0b77fe700 1 -- 172.21.15.12:0/871688481 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0c809d3f0 con 0x562c6fe7de00 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.584+0000 7fd0b77fe700 1 -- 172.21.15.12:0/871688481 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8095cd0 con 0x562c6fe7de00 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.584+0000 7fd0df846b80 1 -- 172.21.15.12:0/871688481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 msgr2=0x562c6fe7e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.584+0000 7fd0df846b80 1 --2- 172.21.15.12:0/871688481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe7e1e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80c9c00 tx=0x7fd0c8098f20 comp rx=0 tx=0).stop 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.584+0000 7fd0df846b80 1 -- 172.21.15.12:0/871688481 shutdown_connections 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.584+0000 7fd0df846b80 1 --2- 172.21.15.12:0/871688481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 0x562c6fe78cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.584+0000 7fd0df846b80 1 --2- 172.21.15.12:0/871688481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe7e1e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.584+0000 7fd0df846b80 1 --2- 172.21.15.12:0/871688481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe78350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.584+0000 7fd0df846b80 1 -- 172.21.15.12:0/871688481 >> 172.21.15.12:0/871688481 conn(0x562c6fdd6210 msgr2=0x562c6fe72df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.584+0000 7fd0df846b80 1 -- 172.21.15.12:0/871688481 shutdown_connections 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.584+0000 7fd0df846b80 1 -- 172.21.15.12:0/871688481 wait complete. 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.585+0000 7fd0df846b80 1 -- start start 2024-05-08T23:51:58.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.585+0000 7fd0df846b80 1 Processor -- start 2024-05-08T23:51:58.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.585+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 0x562c6fe84830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe81b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe82260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0df846b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fd64560 con 0x562c6fe7e720 2024-05-08T23:51:58.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0df846b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fd64200 con 0x562c6fe7de00 2024-05-08T23:51:58.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0df846b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fc4b6a0 con 0x562c6fe78890 2024-05-08T23:51:58.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0df846b80 10 client.0 _ll_register_callbacks cb 0x562c6fd6db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:58.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d1c29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe82260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d0c27700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe81b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d1428700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 0x562c6fe84830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d1c29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe82260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42046/0 (socket says 172.21.15.12:42046) 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d1c29700 1 -- 172.21.15.12:0/2668342868 learned_addr learned my addr 172.21.15.12:0/2668342868 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d0c27700 1 -- 172.21.15.12:0/2668342868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 msgr2=0x562c6fe84830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 0x562c6fe84830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d0c27700 1 -- 172.21.15.12:0/2668342868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 msgr2=0x562c6fe82260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe82260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d0c27700 1 -- 172.21.15.12:0/2668342868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc1a2fe0 con 0x562c6fe7de00 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d1c29700 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe82260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe81b20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x562c6fe74bd0 tx=0x7fd0bc019050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3733c2dd71ebb27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.586+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/2668342868 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc02a050 con 0x562c6fe7de00 2024-05-08T23:51:58.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0df846b80 1 -- 172.21.15.12:0/2668342868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c6fd21270 con 0x562c6fe7de00 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/2668342868 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd0bc00f660 con 0x562c6fe7de00 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/2668342868 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0bc00f820 con 0x562c6fe7de00 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0df846b80 10 client.26726 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0df846b80 1 -- 172.21.15.12:0/2668342868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562c6fbf5b60 con 0x562c6fe7de00 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/2668342868 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c6fd21270 con 0x562c6fe7de00 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/2668342868 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd0bc023b30 con 0x562c6fe7de00 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0df846b80 10 client.26726 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0df846b80 10 client.26726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0df846b80 20 client.26726 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0df846b80 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6fe92230 0x562c6fe94650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0df846b80 4 client.26726 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0df846b80 1 -- 172.21.15.12:0/2668342868 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c6fe47810 con 0x562c6fe92230 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.587+0000 7fd0d1428700 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6fe92230 0x562c6fe94650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.588+0000 7fd0d1428700 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6fe92230 0x562c6fe94650 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a71712239af62158 server_cookie=2b84a180f2fc623a in_seq=0 out_seq=0 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.588+0000 7fd0b4ff9700 10 client.26726 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.589+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/2668342868 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c6fe47810 con 0x562c6fe92230 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.589+0000 7fd0b4ff9700 10 client.26726 handle_command_reply: tid=0 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.589+0000 7fd0df846b80 1 client.26726 shutdown 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.589+0000 7fd0df846b80 1 -- 172.21.15.12:0/2668342868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 msgr2=0x562c6fe81b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.589+0000 7fd0df846b80 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe81b20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x562c6fe74bd0 tx=0x7fd0bc019050 comp rx=0 tx=0).stop 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.590+0000 7fd0df846b80 1 -- 172.21.15.12:0/2668342868 shutdown_connections 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.590+0000 7fd0df846b80 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe7e720 0x562c6fe82260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.590+0000 7fd0df846b80 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe78890 0x562c6fe84830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.590+0000 7fd0df846b80 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6fe92230 0x562c6fe94650 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.590+0000 7fd0df846b80 1 --2- 172.21.15.12:0/2668342868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fe81b20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.590+0000 7fd0b4ff9700 0 client.26726 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.590+0000 7fd0df846b80 1 -- 172.21.15.12:0/2668342868 >> 172.21.15.12:0/2668342868 conn(0x562c6fdd6210 msgr2=0x562c6fe725a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:58.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.590+0000 7fd0df846b80 1 -- 172.21.15.12:0/2668342868 shutdown_connections 2024-05-08T23:51:58.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.590+0000 7fd0df846b80 1 -- 172.21.15.12:0/2668342868 wait complete. 2024-05-08T23:51:58.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.590+0000 7fd0df846b80 20 client.26726 trim_cache size 0 max 16384 2024-05-08T23:51:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.608+0000 7fd0df846b80 1 Processor -- start 2024-05-08T23:51:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.608+0000 7fd0df846b80 1 -- start start 2024-05-08T23:51:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fe81ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe7de00 0x562c6fe72d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7e720 0x562c6fe89390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0df846b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fe74e50 con 0x562c6fe78890 2024-05-08T23:51:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0df846b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fe82860 con 0x562c6fe7e720 2024-05-08T23:51:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0df846b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fe99860 con 0x562c6fe7de00 2024-05-08T23:51:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d1428700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fe81ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d1428700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fe81ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42052/0 (socket says 172.21.15.12:42052) 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d1428700 1 -- 172.21.15.12:0/981650091 learned_addr learned my addr 172.21.15.12:0/981650091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d1c29700 1 --2- 172.21.15.12:0/981650091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7e720 0x562c6fe89390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/981650091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe7de00 0x562c6fe72d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d1428700 1 -- 172.21.15.12:0/981650091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe7de00 msgr2=0x562c6fe72d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d1428700 1 --2- 172.21.15.12:0/981650091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe7de00 0x562c6fe72d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d1428700 1 -- 172.21.15.12:0/981650091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7e720 msgr2=0x562c6fe89390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d1428700 1 --2- 172.21.15.12:0/981650091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7e720 0x562c6fe89390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d1428700 1 -- 172.21.15.12:0/981650091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c6fd8e560 con 0x562c6fe78890 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0d1428700 1 --2- 172.21.15.12:0/981650091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fe81ac0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80ca180 tx=0x7fd0c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fed07b625adf1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.609+0000 7fd0b7fff700 1 -- 172.21.15.12:0/981650091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8022e60 con 0x562c6fe78890 2024-05-08T23:51:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0b7fff700 1 -- 172.21.15.12:0/981650091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0c809d160 con 0x562c6fe78890 2024-05-08T23:51:58.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0b7fff700 1 -- 172.21.15.12:0/981650091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80840b0 con 0x562c6fe78890 2024-05-08T23:51:58.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0df846b80 1 -- 172.21.15.12:0/981650091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 msgr2=0x562c6fe81ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0df846b80 1 --2- 172.21.15.12:0/981650091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fe81ac0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80ca180 tx=0x7fd0c8000bc0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0df846b80 1 -- 172.21.15.12:0/981650091 shutdown_connections 2024-05-08T23:51:58.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0df846b80 1 --2- 172.21.15.12:0/981650091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7e720 0x562c6fe89390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0df846b80 1 --2- 172.21.15.12:0/981650091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fe81ac0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0df846b80 1 --2- 172.21.15.12:0/981650091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe7de00 0x562c6fe72d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0df846b80 1 -- 172.21.15.12:0/981650091 >> 172.21.15.12:0/981650091 conn(0x562c6fdd6210 msgr2=0x562c6fe813b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:58.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0df846b80 1 -- 172.21.15.12:0/981650091 shutdown_connections 2024-05-08T23:51:58.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.610+0000 7fd0df846b80 1 -- 172.21.15.12:0/981650091 wait complete. 2024-05-08T23:51:58.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0df846b80 1 -- start start 2024-05-08T23:51:58.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0df846b80 1 Processor -- start 2024-05-08T23:51:58.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fede8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fedee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0df846b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe7e720 0x562c6fee29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0df846b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fee2f90 con 0x562c6fe78890 2024-05-08T23:51:58.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0df846b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fee3100 con 0x562c6fe7de00 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0df846b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fee32c0 con 0x562c6fe7e720 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0df846b80 10 client.0 _ll_register_callbacks cb 0x562c6fd6db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0d1428700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fede8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0d1428700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fede8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42062/0 (socket says 172.21.15.12:42062) 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.611+0000 7fd0d1428700 1 -- 172.21.15.12:0/3156735472 learned_addr learned my addr 172.21.15.12:0/3156735472 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fedee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0d1c29700 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe7e720 0x562c6fee29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0d1428700 1 -- 172.21.15.12:0/3156735472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe7e720 msgr2=0x562c6fee29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0d1428700 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe7e720 0x562c6fee29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0d1428700 1 -- 172.21.15.12:0/3156735472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 msgr2=0x562c6fedee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0d1428700 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fedee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0d1428700 1 -- 172.21.15.12:0/3156735472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c6fc493b0 con 0x562c6fe78890 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0d1428700 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fede8c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8003ff0 tx=0x7fd0c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d65b94acd42ee5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:58.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/3156735472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8022e60 con 0x562c6fe78890 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/3156735472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd0c8095cd0 con 0x562c6fe78890 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0df846b80 1 -- 172.21.15.12:0/3156735472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0c809d3f0 con 0x562c6fe78890 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/3156735472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0c8095e90 con 0x562c6fe78890 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0df846b80 10 client.16941 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.612+0000 7fd0df846b80 1 -- 172.21.15.12:0/3156735472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562c6fc4dcd0 con 0x562c6fe78890 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/3156735472 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0c809d3f0 con 0x562c6fe78890 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/3156735472 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd0c8098ab0 con 0x562c6fe78890 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0df846b80 10 client.16941 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0df846b80 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0df846b80 20 client.16941 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0df846b80 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6feeb840 0x562c6feedc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0df846b80 4 client.16941 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0df846b80 1 -- 172.21.15.12:0/3156735472 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562c6fdc0ec0 con 0x562c6feeb840 2024-05-08T23:51:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6feeb840 0x562c6feedc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0d0c27700 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6feeb840 0x562c6feedc30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ca1636d5c8f950 server_cookie=d4c02aafd567d198 in_seq=0 out_seq=0 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.613+0000 7fd0b4ff9700 10 client.16941 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0b4ff9700 1 -- 172.21.15.12:0/3156735472 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562c6fdc0ec0 con 0x562c6feeb840 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0b4ff9700 10 client.16941 handle_command_reply: tid=0 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 client.16941 shutdown 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 -- 172.21.15.12:0/3156735472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 msgr2=0x562c6fede8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fede8c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8003ff0 tx=0x7fd0c8004750 comp rx=0 tx=0).stop 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 -- 172.21.15.12:0/3156735472 shutdown_connections 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6fe7e720 0x562c6fee29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6fe78890 0x562c6fede8c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6feeb840 0x562c6feedc30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 --2- 172.21.15.12:0/3156735472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6fe7de00 0x562c6fedee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0b4ff9700 0 client.16941 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 -- 172.21.15.12:0/3156735472 >> 172.21.15.12:0/3156735472 conn(0x562c6fdd6210 msgr2=0x562c6fe813b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 -- 172.21.15.12:0/3156735472 shutdown_connections 2024-05-08T23:51:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 1 -- 172.21.15.12:0/3156735472 wait complete. 2024-05-08T23:51:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.614+0000 7fd0df846b80 20 client.16941 trim_cache size 0 max 16384 2024-05-08T23:51:58.615 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:51:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:51:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:51:58.615 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:51:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 -- 172.21.15.12:0/2310284911 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd0b80a1310 msgr2=0x7fd0b80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd0b80a1310 0x7fd0b80a37d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009fd0 tx=0x7fd0bc005fb0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 -- 172.21.15.12:0/2310284911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0cc1000f0 msgr2=0x7fd0cc196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0cc1000f0 0x7fd0cc196270 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800d980 tx=0x7fd0c800dc90 comp rx=0 tx=0).stop 2024-05-08T23:51:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 -- 172.21.15.12:0/2310284911 shutdown_connections 2024-05-08T23:51:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd0b80a1310 0x7fd0b80a37d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0cc105f90 0x7fd0cc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0cc1000f0 0x7fd0cc196270 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 --2- 172.21.15.12:0/2310284911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0cc100ab0 0x7fd0cc1967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 -- 172.21.15.12:0/2310284911 >> 172.21.15.12:0/2310284911 conn(0x7fd0cc076ba0 msgr2=0x7fd0cc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 -- 172.21.15.12:0/2310284911 shutdown_connections 2024-05-08T23:51:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.615+0000 7fd0d368c700 1 -- 172.21.15.12:0/2310284911 wait complete. 2024-05-08T23:51:58.624 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:51:58.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:51:58.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:58 smithi204 ceph-mon[47373]: pgmap v171: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-05-08T23:51:58.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:58 smithi160 ceph-mon[46152]: pgmap v171: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-05-08T23:51:58.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.773+0000 7fa58c10e700 1 -- 172.21.15.12:0/258200292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5841008f0 msgr2=0x7fa584104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.773+0000 7fa58c10e700 1 --2- 172.21.15.12:0/258200292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5841008f0 0x7fa584104930 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa574009990 tx=0x7fa574009ca0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.773+0000 7fa58c10e700 1 -- 172.21.15.12:0/258200292 shutdown_connections 2024-05-08T23:51:58.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.773+0000 7fa58c10e700 1 --2- 172.21.15.12:0/258200292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa584105060 0x7fa58410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.773+0000 7fa58c10e700 1 --2- 172.21.15.12:0/258200292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5840fff40 0x7fa584100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.773+0000 7fa58c10e700 1 --2- 172.21.15.12:0/258200292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5841008f0 0x7fa584104930 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.773+0000 7fa58c10e700 1 -- 172.21.15.12:0/258200292 >> 172.21.15.12:0/258200292 conn(0x7fa5840fb7e0 msgr2=0x7fa5840fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:58.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.774+0000 7fa58c10e700 1 -- 172.21.15.12:0/258200292 shutdown_connections 2024-05-08T23:51:58.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.774+0000 7fa58c10e700 1 -- 172.21.15.12:0/258200292 wait complete. 2024-05-08T23:51:58.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.774+0000 7fa58c10e700 1 Processor -- start 2024-05-08T23:51:58.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.774+0000 7fa58c10e700 1 -- start start 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa58c10e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840fff40 0x7fa584197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa58c10e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa584105060 0x7fa5841978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa58c10e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa58419bc50 0x7fa58419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa58c10e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58419c6d0 con 0x7fa5840fff40 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa58c10e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58419c840 con 0x7fa58419bc50 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa58c10e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58419ca00 con 0x7fa584105060 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa589eaa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840fff40 0x7fa584197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa589eaa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840fff40 0x7fa584197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42070/0 (socket says 172.21.15.12:42070) 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa589eaa700 1 -- 172.21.15.12:0/534803219 learned_addr learned my addr 172.21.15.12:0/534803219 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa58a6ab700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa58419bc50 0x7fa58419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa5896a9700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa584105060 0x7fa5841978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa589eaa700 1 -- 172.21.15.12:0/534803219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa584105060 msgr2=0x7fa5841978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa589eaa700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa584105060 0x7fa5841978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa589eaa700 1 -- 172.21.15.12:0/534803219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa58419bc50 msgr2=0x7fa58419c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa589eaa700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa58419bc50 0x7fa58419c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa589eaa700 1 -- 172.21.15.12:0/534803219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740095f0 con 0x7fa5840fff40 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa5896a9700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa584105060 0x7fa5841978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.775+0000 7fa58a6ab700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa58419bc50 0x7fa58419c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.776+0000 7fa589eaa700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840fff40 0x7fa584197370 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e9a0 tx=0x7fa58000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.776+0000 7fa57affd700 1 -- 172.21.15.12:0/534803219 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580009790 con 0x7fa5840fff40 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.776+0000 7fa58c10e700 1 -- 172.21.15.12:0/534803219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa584101960 con 0x7fa5840fff40 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.776+0000 7fa57affd700 1 -- 172.21.15.12:0/534803219 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa58000cbd0 con 0x7fa5840fff40 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.776+0000 7fa57affd700 1 -- 172.21.15.12:0/534803219 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580018680 con 0x7fa5840fff40 2024-05-08T23:51:58.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.776+0000 7fa58c10e700 1 -- 172.21.15.12:0/534803219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa584101ec0 con 0x7fa5840fff40 2024-05-08T23:51:58.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.777+0000 7fa58c10e700 1 -- 172.21.15.12:0/534803219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa584101410 con 0x7fa5840fff40 2024-05-08T23:51:58.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.778+0000 7fa57affd700 1 -- 172.21.15.12:0/534803219 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa58000cd40 con 0x7fa5840fff40 2024-05-08T23:51:58.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.779+0000 7fa57affd700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5700a1290 0x7fa5700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:58.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.779+0000 7fa5896a9700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5700a1290 0x7fa5700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:58.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.779+0000 7fa57affd700 1 -- 172.21.15.12:0/534803219 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fa580014070 con 0x7fa5840fff40 2024-05-08T23:51:58.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.780+0000 7fa5896a9700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5700a1290 0x7fa5700a3750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa574006010 tx=0x7fa57400b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:58.785+0000 7fa57affd700 1 -- 172.21.15.12:0/534803219 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa580052d60 con 0x7fa5840fff40 2024-05-08T23:51:58.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 328946 10.97 MB/sec execute 114 sec latency 30.922 ms 2024-05-08T23:51:58.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:58 smithi012 ceph-mon[42503]: pgmap v171: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-05-08T23:51:59.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.069+0000 7fa58c10e700 1 -- 172.21.15.12:0/534803219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa584198730 con 0x7fa5840fff40 2024-05-08T23:51:59.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.070+0000 7fa57affd700 1 -- 172.21.15.12:0/534803219 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10873 (secure 0 0 0) 0x7fa5800963b0 con 0x7fa5840fff40 2024-05-08T23:51:59.081 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:51:59.081 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:50:48.465221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_26224":{"gid":26224,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/131208281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":131208281},{"type":"v1","addr":"172.21.15.204:6837","nonce":131208281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:51:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 -- 172.21.15.12:0/534803219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5700a1290 msgr2=0x7fa5700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5700a1290 0x7fa5700a3750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa574006010 tx=0x7fa57400b260 comp rx=0 tx=0).stop 2024-05-08T23:51:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 -- 172.21.15.12:0/534803219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840fff40 msgr2=0x7fa584197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840fff40 0x7fa584197370 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e9a0 tx=0x7fa58000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 -- 172.21.15.12:0/534803219 shutdown_connections 2024-05-08T23:51:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5700a1290 0x7fa5700a3750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa58419bc50 0x7fa58419c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840fff40 0x7fa584197370 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 --2- 172.21.15.12:0/534803219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa584105060 0x7fa5841978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 -- 172.21.15.12:0/534803219 >> 172.21.15.12:0/534803219 conn(0x7fa5840fb7e0 msgr2=0x7fa584068420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 -- 172.21.15.12:0/534803219 shutdown_connections 2024-05-08T23:51:59.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.073+0000 7fa58c10e700 1 -- 172.21.15.12:0/534803219 wait complete. 2024-05-08T23:51:59.084 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 31 2024-05-08T23:51:59.084 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:51:59.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.236+0000 7f42d3b4f700 1 -- 172.21.15.12:0/70198524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42cc101b20 msgr2=0x7f42cc105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.236+0000 7f42d3b4f700 1 --2- 172.21.15.12:0/70198524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42cc101b20 0x7f42cc105b60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0098f0 tx=0x7f42bc009c00 comp rx=0 tx=0).stop 2024-05-08T23:51:59.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.237+0000 7f42d3b4f700 1 -- 172.21.15.12:0/70198524 shutdown_connections 2024-05-08T23:51:59.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.237+0000 7f42d3b4f700 1 --2- 172.21.15.12:0/70198524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42cc106290 0x7f42cc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.237+0000 7f42d3b4f700 1 --2- 172.21.15.12:0/70198524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42cc101170 0x7f42cc101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.237+0000 7f42d3b4f700 1 --2- 172.21.15.12:0/70198524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42cc101b20 0x7f42cc105b60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.237+0000 7f42d3b4f700 1 -- 172.21.15.12:0/70198524 >> 172.21.15.12:0/70198524 conn(0x7f42cc0fc9d0 msgr2=0x7f42cc0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.237+0000 7f42d3b4f700 1 -- 172.21.15.12:0/70198524 shutdown_connections 2024-05-08T23:51:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.237+0000 7f42d3b4f700 1 -- 172.21.15.12:0/70198524 wait complete. 2024-05-08T23:51:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.237+0000 7f42d3b4f700 1 Processor -- start 2024-05-08T23:51:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.237+0000 7f42d3b4f700 1 -- start start 2024-05-08T23:51:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d3b4f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42cc101170 0x7f42cc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d3b4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42cc106290 0x7f42cc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d3b4f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42cc073d20 0x7f42cc1a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d3b4f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc1a5c70 con 0x7f42cc106290 2024-05-08T23:51:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d3b4f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc1a5db0 con 0x7f42cc073d20 2024-05-08T23:51:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d3b4f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc1a5f70 con 0x7f42cc101170 2024-05-08T23:51:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d10ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42cc106290 0x7f42cc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d18eb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42cc101170 0x7f42cc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d18eb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42cc101170 0x7f42cc0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50608/0 (socket says 172.21.15.12:50608) 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d18eb700 1 -- 172.21.15.12:0/2669645447 learned_addr learned my addr 172.21.15.12:0/2669645447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d10ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42cc106290 0x7f42cc0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42084/0 (socket says 172.21.15.12:42084) 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d20ec700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42cc073d20 0x7f42cc1a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d18eb700 1 -- 172.21.15.12:0/2669645447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42cc073d20 msgr2=0x7f42cc1a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d18eb700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42cc073d20 0x7f42cc1a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d18eb700 1 -- 172.21.15.12:0/2669645447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42cc106290 msgr2=0x7f42cc0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d18eb700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42cc106290 0x7f42cc0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d18eb700 1 -- 172.21.15.12:0/2669645447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c4009d00 con 0x7f42cc101170 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42d18eb700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42cc101170 0x7f42cc0732a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f42c800e9a0 tx=0x7f42c800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.238+0000 7f42c2ffd700 1 -- 172.21.15.12:0/2669645447 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8009790 con 0x7f42cc101170 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.239+0000 7f42c2ffd700 1 -- 172.21.15.12:0/2669645447 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c8010430 con 0x7f42cc101170 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.239+0000 7f42d3b4f700 1 -- 172.21.15.12:0/2669645447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc009520 con 0x7f42cc101170 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.239+0000 7f42c2ffd700 1 -- 172.21.15.12:0/2669645447 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8018680 con 0x7f42cc101170 2024-05-08T23:51:59.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.239+0000 7f42d3b4f700 1 -- 172.21.15.12:0/2669645447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc1a65a0 con 0x7f42cc101170 2024-05-08T23:51:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.241+0000 7f42c2ffd700 1 -- 172.21.15.12:0/2669645447 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f42c80105a0 con 0x7f42cc101170 2024-05-08T23:51:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.242+0000 7f42c2ffd700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42b80a1290 0x7f42b80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.242+0000 7f42c2ffd700 1 -- 172.21.15.12:0/2669645447 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f42c8014070 con 0x7f42cc101170 2024-05-08T23:51:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.242+0000 7f42d10ea700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42b80a1290 0x7f42b80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.242+0000 7f42d10ea700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42b80a1290 0x7f42b80a3750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f42bc005e90 tx=0x7f42bc019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.242+0000 7f42dfd09b80 1 Processor -- start 2024-05-08T23:51:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42dfd09b80 1 -- start start 2024-05-08T23:51:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 0x55fa32fbe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fbecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32f15150 con 0x55fa32fbe890 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32f12b10 con 0x55fa32fc4720 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32f0e400 con 0x55fa32fc3e00 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d18eb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d10ea700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 0x55fa32fbe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d20ec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fbecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d18eb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50612/0 (socket says 172.21.15.12:50612) 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d18eb700 1 -- 172.21.15.12:0/1431968183 learned_addr learned my addr 172.21.15.12:0/1431968183 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d18eb700 1 -- 172.21.15.12:0/1431968183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 msgr2=0x55fa32fbe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d18eb700 1 --2- 172.21.15.12:0/1431968183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 0x55fa32fbe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d18eb700 1 -- 172.21.15.12:0/1431968183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 msgr2=0x55fa32fbecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d18eb700 1 --2- 172.21.15.12:0/1431968183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fbecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d18eb700 1 -- 172.21.15.12:0/1431968183 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa32d8f3b0 con 0x55fa32fc3e00 2024-05-08T23:51:59.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d20ec700 1 --2- 172.21.15.12:0/1431968183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fbecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:59.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.243+0000 7f42d18eb700 1 --2- 172.21.15.12:0/1431968183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc41e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f42c80964c0 tx=0x7f42c809a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6217f86c9f4c0a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.244+0000 7f42b7fff700 1 -- 172.21.15.12:0/1431968183 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c80a2440 con 0x55fa32fc3e00 2024-05-08T23:51:59.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.244+0000 7f42b7fff700 1 -- 172.21.15.12:0/1431968183 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c8010880 con 0x55fa32fc3e00 2024-05-08T23:51:59.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.244+0000 7f42b7fff700 1 -- 172.21.15.12:0/1431968183 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8094bd0 con 0x55fa32fc3e00 2024-05-08T23:51:59.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.244+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1431968183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 msgr2=0x55fa32fc41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.244+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/1431968183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc41e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f42c80964c0 tx=0x7f42c809a6c0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.244+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1431968183 shutdown_connections 2024-05-08T23:51:59.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.244+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/1431968183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fbecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.244+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/1431968183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc41e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.244+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/1431968183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 0x55fa32fbe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.244+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1431968183 >> 172.21.15.12:0/1431968183 conn(0x55fa32f1c210 msgr2=0x55fa32fb8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.245+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1431968183 shutdown_connections 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.245+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1431968183 wait complete. 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.245+0000 7f42dfd09b80 1 -- start start 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.245+0000 7f42dfd09b80 1 Processor -- start 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.245+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fbe890 0x55fa32fc7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.245+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.245+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc4720 0x55fa32fc8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.245+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32eaa560 con 0x55fa32fc4720 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.245+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32eaa200 con 0x55fa32fbe890 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.245+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32d916a0 con 0x55fa32fc3e00 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42dfd09b80 10 client.0 _ll_register_callbacks cb 0x55fa32eb3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d10ea700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d18eb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fbe890 0x55fa32fc7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d10ea700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50616/0 (socket says 172.21.15.12:50616) 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d10ea700 1 -- 172.21.15.12:0/3196107233 learned_addr learned my addr 172.21.15.12:0/3196107233 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d20ec700 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc4720 0x55fa32fc8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d10ea700 1 -- 172.21.15.12:0/3196107233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fbe890 msgr2=0x55fa32fc7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d10ea700 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fbe890 0x55fa32fc7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d10ea700 1 -- 172.21.15.12:0/3196107233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc4720 msgr2=0x55fa32fc8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d10ea700 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc4720 0x55fa32fc8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d10ea700 1 -- 172.21.15.12:0/3196107233 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c4009160 con 0x55fa32fc3e00 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42d10ea700 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc80e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0098f0 tx=0x7f42bc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e8743330aabd46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42b57fa700 1 -- 172.21.15.12:0/3196107233 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc02b040 con 0x55fa32fc3e00 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42b57fa700 1 -- 172.21.15.12:0/3196107233 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f42bc005240 con 0x55fa32fc3e00 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.246+0000 7f42dfd09b80 1 -- 172.21.15.12:0/3196107233 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa32e67270 con 0x55fa32fc3e00 2024-05-08T23:51:59.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42b57fa700 1 -- 172.21.15.12:0/3196107233 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42bc00bbc0 con 0x55fa32fc3e00 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42dfd09b80 10 client.26734 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42dfd09b80 1 -- 172.21.15.12:0/3196107233 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f42cc1a65a0 con 0x55fa32fc3e00 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42b57fa700 1 -- 172.21.15.12:0/3196107233 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fa32e67270 con 0x55fa32fc3e00 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42b57fa700 1 -- 172.21.15.12:0/3196107233 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f42bc003d00 con 0x55fa32fc3e00 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42dfd09b80 10 client.26734 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42dfd09b80 10 client.26734 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42dfd09b80 20 client.26734 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fa32fd8280 0x55fa32fda6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42dfd09b80 4 client.26734 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42dfd09b80 1 -- 172.21.15.12:0/3196107233 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa32f8d810 con 0x55fa32fd8280 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.247+0000 7f42d18eb700 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fa32fd8280 0x55fa32fda6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.248+0000 7f42b57fa700 10 client.26734 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.248+0000 7f42d18eb700 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fa32fd8280 0x55fa32fda6a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1f8cce7a2c96b58 server_cookie=f66c4d3126d273b7 in_seq=0 out_seq=0 2024-05-08T23:51:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.248+0000 7f42b57fa700 1 -- 172.21.15.12:0/3196107233 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa32f8d810 con 0x55fa32fd8280 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.248+0000 7f42b57fa700 10 client.26734 handle_command_reply: tid=0 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.248+0000 7f42dfd09b80 1 client.26734 shutdown 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 1 -- 172.21.15.12:0/3196107233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 msgr2=0x55fa32fc80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc80e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0098f0 tx=0x7f42bc009c00 comp rx=0 tx=0).stop 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 1 -- 172.21.15.12:0/3196107233 shutdown_connections 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc4720 0x55fa32fc8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fbe890 0x55fa32fc7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fa32fd8280 0x55fa32fda6a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/3196107233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc3e00 0x55fa32fc80e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 1 -- 172.21.15.12:0/3196107233 >> 172.21.15.12:0/3196107233 conn(0x55fa32f1c210 msgr2=0x55fa32fb85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42b57fa700 0 client.26734 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 1 -- 172.21.15.12:0/3196107233 shutdown_connections 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 1 -- 172.21.15.12:0/3196107233 wait complete. 2024-05-08T23:51:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.249+0000 7f42dfd09b80 20 client.26734 trim_cache size 0 max 16384 2024-05-08T23:51:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42dfd09b80 1 Processor -- start 2024-05-08T23:51:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42dfd09b80 1 -- start start 2024-05-08T23:51:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fbe890 0x55fa32fda670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc3e00 0x55fa32fbec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 0x55fa32fd3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32fc9030 con 0x55fa32fc3e00 2024-05-08T23:51:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32fca6a0 con 0x55fa32fc4720 2024-05-08T23:51:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32fdf8b0 con 0x55fa32fbe890 2024-05-08T23:51:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d10ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc3e00 0x55fa32fbec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d20ec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 0x55fa32fd3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d10ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc3e00 0x55fa32fbec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42114/0 (socket says 172.21.15.12:42114) 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d10ea700 1 -- 172.21.15.12:0/2634449845 learned_addr learned my addr 172.21.15.12:0/2634449845 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d18eb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fbe890 0x55fa32fda670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d20ec700 1 -- 172.21.15.12:0/2634449845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fbe890 msgr2=0x55fa32fda670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d20ec700 1 --2- 172.21.15.12:0/2634449845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fbe890 0x55fa32fda670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d20ec700 1 -- 172.21.15.12:0/2634449845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc3e00 msgr2=0x55fa32fbec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d20ec700 1 --2- 172.21.15.12:0/2634449845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc3e00 0x55fa32fbec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d20ec700 1 -- 172.21.15.12:0/2634449845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa32d3bb60 con 0x55fa32fc4720 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.263+0000 7f42d10ea700 1 --2- 172.21.15.12:0/2634449845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc3e00 0x55fa32fbec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42d20ec700 1 --2- 172.21.15.12:0/2634449845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 0x55fa32fd3c20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009cd0 tx=0x7f42c40099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5bc3a52da452331 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42c0da4700 1 -- 172.21.15.12:0/2634449845 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4004bb0 con 0x55fa32fc4720 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42c0da4700 1 -- 172.21.15.12:0/2634449845 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c401b070 con 0x55fa32fc4720 2024-05-08T23:51:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42c0da4700 1 -- 172.21.15.12:0/2634449845 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4020480 con 0x55fa32fc4720 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42dfd09b80 1 -- 172.21.15.12:0/2634449845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 msgr2=0x55fa32fd3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/2634449845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 0x55fa32fd3c20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009cd0 tx=0x7f42c40099e0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42dfd09b80 1 -- 172.21.15.12:0/2634449845 shutdown_connections 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/2634449845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc4720 0x55fa32fd3c20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/2634449845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fbe890 0x55fa32fda670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/2634449845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fc3e00 0x55fa32fbec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.264+0000 7f42dfd09b80 1 -- 172.21.15.12:0/2634449845 >> 172.21.15.12:0/2634449845 conn(0x55fa32f1c210 msgr2=0x55fa32fc6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 1 -- 172.21.15.12:0/2634449845 shutdown_connections 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 1 -- 172.21.15.12:0/2634449845 wait complete. 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 1 -- start start 2024-05-08T23:51:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 1 Processor -- start 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fcf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc3e00 0x55fa32fcd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc4720 0x55fa32fcd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32fcdd70 con 0x55fa32fbe890 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa32fcdee0 con 0x55fa32fc3e00 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3302d780 con 0x55fa32fc4720 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.265+0000 7f42dfd09b80 10 client.0 _ll_register_callbacks cb 0x55fa32eb3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d18eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fcf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d20ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc4720 0x55fa32fcd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d18eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fcf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42130/0 (socket says 172.21.15.12:42130) 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d20ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc4720 0x55fa32fcd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50632/0 (socket says 172.21.15.12:50632) 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d18eb700 1 -- 172.21.15.12:0/1869634661 learned_addr learned my addr 172.21.15.12:0/1869634661 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d10ea700 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc3e00 0x55fa32fcd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d18eb700 1 -- 172.21.15.12:0/1869634661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc4720 msgr2=0x55fa32fcd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d18eb700 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc4720 0x55fa32fcd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d18eb700 1 -- 172.21.15.12:0/1869634661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc3e00 msgr2=0x55fa32fcd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d18eb700 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc3e00 0x55fa32fcd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d18eb700 1 -- 172.21.15.12:0/1869634661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc1a65a0 con 0x55fa32fbe890 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d10ea700 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc3e00 0x55fa32fcd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42d18eb700 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fcf4f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f42c8099c70 tx=0x7f42c8099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56d874cfde84a8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42b57fa700 1 -- 172.21.15.12:0/1869634661 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c809a420 con 0x55fa32fbe890 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42b57fa700 1 -- 172.21.15.12:0/1869634661 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f42c80994b0 con 0x55fa32fbe890 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1869634661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42c8010880 con 0x55fa32fbe890 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42b57fa700 1 -- 172.21.15.12:0/1869634661 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42c8096300 con 0x55fa32fbe890 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42dfd09b80 10 client.16965 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.266+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1869634661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f42c4003aa0 con 0x55fa32fbe890 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42b57fa700 1 -- 172.21.15.12:0/1869634661 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c8063080 con 0x55fa32fbe890 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42b57fa700 1 -- 172.21.15.12:0/1869634661 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f42c80a2020 con 0x55fa32fbe890 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42dfd09b80 10 client.16965 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42dfd09b80 10 client.16965 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42dfd09b80 20 client.16965 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fa3302eb50 0x55fa33030f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42dfd09b80 4 client.16965 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1869634661 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fa32f06ec0 con 0x55fa3302eb50 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42d10ea700 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fa3302eb50 0x55fa33030f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42b57fa700 10 client.16965 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.267+0000 7f42d10ea700 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fa3302eb50 0x55fa33030f40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e493dc693c8b2df3 server_cookie=b2323d1db7cec8a6 in_seq=0 out_seq=0 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42b57fa700 1 -- 172.21.15.12:0/1869634661 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fa32f06ec0 con 0x55fa3302eb50 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42b57fa700 10 client.16965 handle_command_reply: tid=0 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 client.16965 shutdown 2024-05-08T23:51:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1869634661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 msgr2=0x55fa32fcf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fcf4f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f42c8099c70 tx=0x7f42c8099fc0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1869634661 shutdown_connections 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fa3302eb50 0x55fa33030f40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa32fc4720 0x55fa32fcd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42b57fa700 0 client.16965 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa32fbe890 0x55fa32fcf4f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 --2- 172.21.15.12:0/1869634661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa32fc3e00 0x55fa32fcd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1869634661 >> 172.21.15.12:0/1869634661 conn(0x55fa32f1c210 msgr2=0x55fa32fc2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1869634661 shutdown_connections 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 1 -- 172.21.15.12:0/1869634661 wait complete. 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.268+0000 7f42dfd09b80 20 client.16965 trim_cache size 0 max 16384 2024-05-08T23:51:59.269 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:51:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 -- 172.21.15.12:0/2669645447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42b80a1290 msgr2=0x7f42b80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42b80a1290 0x7f42b80a3750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f42bc005e90 tx=0x7f42bc019040 comp rx=0 tx=0).stop 2024-05-08T23:51:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 -- 172.21.15.12:0/2669645447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42cc101170 msgr2=0x7f42cc0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42cc101170 0x7f42cc0732a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f42c800e9a0 tx=0x7f42c800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 -- 172.21.15.12:0/2669645447 shutdown_connections 2024-05-08T23:51:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42b80a1290 0x7f42b80a3750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42cc073d20 0x7f42cc1a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42cc101170 0x7f42cc0732a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 --2- 172.21.15.12:0/2669645447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42cc106290 0x7f42cc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 -- 172.21.15.12:0/2669645447 >> 172.21.15.12:0/2669645447 conn(0x7f42cc0fc9d0 msgr2=0x7f42cc0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 -- 172.21.15.12:0/2669645447 shutdown_connections 2024-05-08T23:51:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.269+0000 7f42d3b4f700 1 -- 172.21.15.12:0/2669645447 wait complete. 2024-05-08T23:51:59.277 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:51:59.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.429+0000 7f3ec9878700 1 -- 172.21.15.12:0/1990083610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ec4105f90 msgr2=0x7f3ec4106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.429+0000 7f3ec9878700 1 --2- 172.21.15.12:0/1990083610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ec4105f90 0x7f3ec4106370 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40099c0 tx=0x7f3eb4009cd0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.429+0000 7f3ec9878700 1 -- 172.21.15.12:0/1990083610 shutdown_connections 2024-05-08T23:51:59.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.429+0000 7f3ec9878700 1 --2- 172.21.15.12:0/1990083610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec4100ab0 0x7f3ec410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.429+0000 7f3ec9878700 1 --2- 172.21.15.12:0/1990083610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ec4105f90 0x7f3ec4106370 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.429+0000 7f3ec9878700 1 --2- 172.21.15.12:0/1990083610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ec41000f0 0x7f3ec4100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.429+0000 7f3ec9878700 1 -- 172.21.15.12:0/1990083610 >> 172.21.15.12:0/1990083610 conn(0x7f3ec4076ba0 msgr2=0x7f3ec4076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.429+0000 7f3ec9878700 1 -- 172.21.15.12:0/1990083610 shutdown_connections 2024-05-08T23:51:59.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.429+0000 7f3ec9878700 1 -- 172.21.15.12:0/1990083610 wait complete. 2024-05-08T23:51:59.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.430+0000 7f3ec9878700 1 Processor -- start 2024-05-08T23:51:59.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.430+0000 7f3ec9878700 1 -- start start 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec9878700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ec41000f0 0x7f3ec410d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec9878700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ec4100ab0 0x7f3ec410db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec9878700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec4105f90 0x7f3ec410e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec9878700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec41a5710 con 0x7f3ec4100ab0 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec9878700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec41a5880 con 0x7f3ec41000f0 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec9878700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec41a5a40 con 0x7f3ec4105f90 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ec4100ab0 0x7f3ec410db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ec41000f0 0x7f3ec410d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ec41000f0 0x7f3ec410d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:32992/0 (socket says 172.21.15.12:32992) 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec2ffd700 1 -- 172.21.15.12:0/37465667 learned_addr learned my addr 172.21.15.12:0/37465667 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec4105f90 0x7f3ec410e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec2ffd700 1 -- 172.21.15.12:0/37465667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec4105f90 msgr2=0x7f3ec410e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec4105f90 0x7f3ec410e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec2ffd700 1 -- 172.21.15.12:0/37465667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ec4100ab0 msgr2=0x7f3ec410db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ec4100ab0 0x7f3ec410db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec2ffd700 1 -- 172.21.15.12:0/37465667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc009c40 con 0x7f3ec41000f0 2024-05-08T23:51:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec27fc700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ec4100ab0 0x7f3ec410db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.431+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ec41000f0 0x7f3ec410d620 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40059d0 tx=0x7f3eb400b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.432+0000 7f3ec8876700 1 -- 172.21.15.12:0/37465667 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb401d070 con 0x7f3ec41000f0 2024-05-08T23:51:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.432+0000 7f3ec9878700 1 -- 172.21.15.12:0/37465667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb40095f0 con 0x7f3ec41000f0 2024-05-08T23:51:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.432+0000 7f3ec8876700 1 -- 172.21.15.12:0/37465667 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb400beb0 con 0x7f3ec41000f0 2024-05-08T23:51:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.432+0000 7f3ec8876700 1 -- 172.21.15.12:0/37465667 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400fa30 con 0x7f3ec41000f0 2024-05-08T23:51:59.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.432+0000 7f3ec9878700 1 -- 172.21.15.12:0/37465667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec41a6030 con 0x7f3ec41000f0 2024-05-08T23:51:59.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.434+0000 7f3ec8876700 1 -- 172.21.15.12:0/37465667 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3eb400fbb0 con 0x7f3ec41000f0 2024-05-08T23:51:59.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.435+0000 7f3ec8876700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3eac0a1290 0x7f3eac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.435+0000 7f3ec27fc700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3eac0a1290 0x7f3eac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.435+0000 7f3ec8876700 1 -- 172.21.15.12:0/37465667 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3eb40cd8a0 con 0x7f3ec41000f0 2024-05-08T23:51:59.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.436+0000 7f3ec27fc700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3eac0a1290 0x7f3eac0a3750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ec410ed50 tx=0x7f3eb8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.437+0000 7f3ed5a32b80 1 Processor -- start 2024-05-08T23:51:59.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.437+0000 7f3ed5a32b80 1 -- start start 2024-05-08T23:51:59.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.437+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a015e00 0x55f31a0161e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.437+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 0x55f31a010350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.437+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a010890 0x55f31a010cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.437+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f319f67150 con 0x55f31a010890 2024-05-08T23:51:59.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.437+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f319f64b10 con 0x55f31a015e00 2024-05-08T23:51:59.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.437+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f319f60400 con 0x55f31a016720 2024-05-08T23:51:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 0x55f31a010350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 0x55f31a010350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50656/0 (socket says 172.21.15.12:50656) 2024-05-08T23:51:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a010890 0x55f31a010cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec27fc700 1 -- 172.21.15.12:0/2729345236 learned_addr learned my addr 172.21.15.12:0/2729345236 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/2729345236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a015e00 0x55f31a0161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec27fc700 1 -- 172.21.15.12:0/2729345236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a015e00 msgr2=0x55f31a0161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec27fc700 1 --2- 172.21.15.12:0/2729345236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a015e00 0x55f31a0161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec27fc700 1 -- 172.21.15.12:0/2729345236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a010890 msgr2=0x55f31a010cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec27fc700 1 --2- 172.21.15.12:0/2729345236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a010890 0x55f31a010cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec27fc700 1 -- 172.21.15.12:0/2729345236 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f319de13b0 con 0x55f31a016720 2024-05-08T23:51:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.438+0000 7f3ec27fc700 1 --2- 172.21.15.12:0/2729345236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 0x55f31a010350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80043c0 tx=0x7f3eb8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7474d1c3711d6cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.439+0000 7f3ea9ffb700 1 -- 172.21.15.12:0/2729345236 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8003d10 con 0x55f31a016720 2024-05-08T23:51:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.439+0000 7f3ea9ffb700 1 -- 172.21.15.12:0/2729345236 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb8003e70 con 0x55f31a016720 2024-05-08T23:51:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.439+0000 7f3ea9ffb700 1 -- 172.21.15.12:0/2729345236 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb801e7e0 con 0x55f31a016720 2024-05-08T23:51:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.440+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/2729345236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 msgr2=0x55f31a010350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.440+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/2729345236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 0x55f31a010350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80043c0 tx=0x7f3eb8014000 comp rx=0 tx=0).stop 2024-05-08T23:51:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.440+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/2729345236 shutdown_connections 2024-05-08T23:51:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.440+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/2729345236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a010890 0x55f31a010cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.440+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/2729345236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a015e00 0x55f31a0161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.440+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/2729345236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 0x55f31a010350 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.440+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/2729345236 >> 172.21.15.12:0/2729345236 conn(0x55f319f6e210 msgr2=0x55f31a00adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.440+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/2729345236 shutdown_connections 2024-05-08T23:51:59.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.441+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/2729345236 wait complete. 2024-05-08T23:51:59.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.441+0000 7f3ed5a32b80 1 -- start start 2024-05-08T23:51:59.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.441+0000 7f3ed5a32b80 1 Processor -- start 2024-05-08T23:51:59.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.442+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 0x55f31a019b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.442+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a015e00 0x55f31a01a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.442+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a01a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.442+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f319efc560 con 0x55f31a016720 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f319efc200 con 0x55f31a010890 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f319de36a0 con 0x55f31a015e00 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ed5a32b80 10 client.0 _ll_register_callbacks cb 0x55f319f05b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a01a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a015e00 0x55f31a01a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 0x55f31a019b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a015e00 0x55f31a01a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50666/0 (socket says 172.21.15.12:50666) 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 0x55f31a019b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33018/0 (socket says 172.21.15.12:33018) 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec27fc700 1 -- 172.21.15.12:0/300694420 learned_addr learned my addr 172.21.15.12:0/300694420 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a01a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42186/0 (socket says 172.21.15.12:42186) 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec37fe700 1 -- 172.21.15.12:0/300694420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a015e00 msgr2=0x55f31a01a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec37fe700 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a015e00 0x55f31a01a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec37fe700 1 -- 172.21.15.12:0/300694420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 msgr2=0x55f31a019b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec37fe700 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 0x55f31a019b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec37fe700 1 -- 172.21.15.12:0/300694420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb8017070 con 0x55f31a016720 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec27fc700 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a015e00 0x55f31a01a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 0x55f31a019b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ec37fe700 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a01a5a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00eee0 tx=0x7f3ebc014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1db57b8274e4d68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.443+0000 7f3ea2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/300694420 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00e040 con 0x55f31a016720 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/300694420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f319eb9270 con 0x55f31a016720 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/300694420 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3ebc01a070 con 0x55f31a016720 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/300694420 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ebc014b90 con 0x55f31a016720 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ed5a32b80 10 client.16977 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/300694420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f319d8db60 con 0x55f31a016720 2024-05-08T23:51:59.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/300694420 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f319eb9270 con 0x55f31a016720 2024-05-08T23:51:59.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/300694420 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ebc01f480 con 0x55f31a016720 2024-05-08T23:51:59.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ed5a32b80 10 client.16977 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:59.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ed5a32b80 10 client.16977 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:51:59.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ed5a32b80 20 client.16977 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:59.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f31a03b240 0x55f31a03d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ed5a32b80 4 client.16977 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:59.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.444+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/300694420 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f319fdf810 con 0x55f31a03b240 2024-05-08T23:51:59.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.445+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f31a03b240 0x55f31a03d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.445+0000 7f3ea2ffd700 10 client.16977 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:51:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.445+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f31a03b240 0x55f31a03d660 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af7289a8d3900702 server_cookie=c9f788840c6c61d0 in_seq=0 out_seq=0 2024-05-08T23:51:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/300694420 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f319fdf810 con 0x55f31a03b240 2024-05-08T23:51:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ea2ffd700 10 client.16977 handle_command_reply: tid=0 2024-05-08T23:51:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 client.16977 shutdown 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/300694420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 msgr2=0x55f31a01a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a01a5a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00eee0 tx=0x7f3ebc014600 comp rx=0 tx=0).stop 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/300694420 shutdown_connections 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f31a03b240 0x55f31a03d660 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a01a5a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 0x55f31a019b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/300694420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a015e00 0x55f31a01a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/300694420 >> 172.21.15.12:0/300694420 conn(0x55f319f6e210 msgr2=0x55f31a00a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ea2ffd700 0 client.16977 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/300694420 shutdown_connections 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.446+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/300694420 wait complete. 2024-05-08T23:51:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.447+0000 7f3ed5a32b80 20 client.16977 trim_cache size 0 max 16384 2024-05-08T23:51:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.460+0000 7f3ed5a32b80 1 Processor -- start 2024-05-08T23:51:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ed5a32b80 1 -- start start 2024-05-08T23:51:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 0x55f31a010350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a015e00 0x55f31a00ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 0x55f31a01a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31a005b00 con 0x55f31a015e00 2024-05-08T23:51:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31a01b190 con 0x55f31a010890 2024-05-08T23:51:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31a01cb70 con 0x55f31a016720 2024-05-08T23:51:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a015e00 0x55f31a00ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a015e00 0x55f31a00ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42194/0 (socket says 172.21.15.12:42194) 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec27fc700 1 -- 172.21.15.12:0/3329234071 learned_addr learned my addr 172.21.15.12:0/3329234071 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/3329234071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 0x55f31a010350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec37fe700 1 --2- 172.21.15.12:0/3329234071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 0x55f31a01a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec27fc700 1 -- 172.21.15.12:0/3329234071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 msgr2=0x55f31a01a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec27fc700 1 --2- 172.21.15.12:0/3329234071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 0x55f31a01a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec27fc700 1 -- 172.21.15.12:0/3329234071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 msgr2=0x55f31a010350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec27fc700 1 --2- 172.21.15.12:0/3329234071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 0x55f31a010350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec27fc700 1 -- 172.21.15.12:0/3329234071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f31a010f40 con 0x55f31a015e00 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.461+0000 7f3ec27fc700 1 --2- 172.21.15.12:0/3329234071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a015e00 0x55f31a00ad20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8003e40 tx=0x7f3eb80092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f23fe84eba8103a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3eaa7fc700 1 -- 172.21.15.12:0/3329234071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb802a050 con 0x55f31a015e00 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3eaa7fc700 1 -- 172.21.15.12:0/3329234071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb802b040 con 0x55f31a015e00 2024-05-08T23:51:59.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3eaa7fc700 1 -- 172.21.15.12:0/3329234071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8026660 con 0x55f31a015e00 2024-05-08T23:51:59.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/3329234071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a015e00 msgr2=0x55f31a00ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/3329234071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a015e00 0x55f31a00ad20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8003e40 tx=0x7f3eb80092d0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/3329234071 shutdown_connections 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/3329234071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a016720 0x55f31a01a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/3329234071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a010890 0x55f31a010350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/3329234071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a015e00 0x55f31a00ad20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/3329234071 >> 172.21.15.12:0/3329234071 conn(0x55f319f6e210 msgr2=0x55f31a0183f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/3329234071 shutdown_connections 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.462+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/3329234071 wait complete. 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.463+0000 7f3ed5a32b80 1 -- start start 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.463+0000 7f3ed5a32b80 1 Processor -- start 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.463+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a010890 0x55f31a030120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.463+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a015e00 0x55f31a030660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.463+0000 7f3ed5a32b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a030ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.463+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31a0907a0 con 0x55f31a016720 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.463+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31a0908e0 con 0x55f31a015e00 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.463+0000 7f3ed5a32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31a090aa0 con 0x55f31a010890 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.463+0000 7f3ed5a32b80 10 client.0 _ll_register_callbacks cb 0x55f319f05b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.463+0000 7f3ec37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a030ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ec37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a030ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42198/0 (socket says 172.21.15.12:42198) 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ec27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a015e00 0x55f31a030660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ec37fe700 1 -- 172.21.15.12:0/745187985 learned_addr learned my addr 172.21.15.12:0/745187985 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a010890 0x55f31a030120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ec37fe700 1 -- 172.21.15.12:0/745187985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a010890 msgr2=0x55f31a030120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ec37fe700 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a010890 0x55f31a030120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ec37fe700 1 -- 172.21.15.12:0/745187985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a015e00 msgr2=0x55f31a030660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ec37fe700 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a015e00 0x55f31a030660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ec37fe700 1 -- 172.21.15.12:0/745187985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f319de13b0 con 0x55f31a016720 2024-05-08T23:51:59.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ec37fe700 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a030ba0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc000f80 tx=0x7f3ebc01f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82de461d9223aecf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ea2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/745187985 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc014b90 con 0x55f31a016720 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/745187985 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3ebc01a040 con 0x55f31a016720 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/745187985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3eb8003e70 con 0x55f31a016720 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/745187985 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ebc014cf0 con 0x55f31a016720 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ed5a32b80 10 client.16983 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.464+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/745187985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f319f26560 con 0x55f31a016720 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/745187985 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb8003e70 con 0x55f31a016720 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/745187985 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ebc0079f0 con 0x55f31a016720 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ed5a32b80 10 client.16983 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ed5a32b80 10 client.16983 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ed5a32b80 20 client.16983 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f31a0921f0 0x55f31a094610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ed5a32b80 4 client.16983 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/745187985 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f319f58ec0 con 0x55f31a0921f0 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f31a0921f0 0x55f31a094610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ec2ffd700 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f31a0921f0 0x55f31a094610 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76a613fdc83e835f server_cookie=15193f28849e5280 in_seq=0 out_seq=0 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.465+0000 7f3ea2ffd700 10 client.16983 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ea2ffd700 1 -- 172.21.15.12:0/745187985 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f319f58ec0 con 0x55f31a0921f0 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ea2ffd700 10 client.16983 handle_command_reply: tid=0 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 client.16983 shutdown 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/745187985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 msgr2=0x55f31a030ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a030ba0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc000f80 tx=0x7f3ebc01f480 comp rx=0 tx=0).stop 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/745187985 shutdown_connections 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f31a0921f0 0x55f31a094610 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f31a016720 0x55f31a030ba0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f31a010890 0x55f31a030120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 --2- 172.21.15.12:0/745187985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f31a015e00 0x55f31a030660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/745187985 >> 172.21.15.12:0/745187985 conn(0x55f319f6e210 msgr2=0x55f31a014bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ea2ffd700 0 client.16983 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:51:59.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/745187985 shutdown_connections 2024-05-08T23:51:59.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 1 -- 172.21.15.12:0/745187985 wait complete. 2024-05-08T23:51:59.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.466+0000 7f3ed5a32b80 20 client.16983 trim_cache size 0 max 16384 2024-05-08T23:51:59.467 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:51:59.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 -- 172.21.15.12:0/37465667 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3eac0a1290 msgr2=0x7f3eac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3eac0a1290 0x7f3eac0a3750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ec410ed50 tx=0x7f3eb8009260 comp rx=0 tx=0).stop 2024-05-08T23:51:59.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 -- 172.21.15.12:0/37465667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ec41000f0 msgr2=0x7f3ec410d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ec41000f0 0x7f3ec410d620 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40059d0 tx=0x7f3eb400b9c0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 -- 172.21.15.12:0/37465667 shutdown_connections 2024-05-08T23:51:59.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3eac0a1290 0x7f3eac0a3750 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ec4105f90 0x7f3ec410e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ec41000f0 0x7f3ec410d620 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 --2- 172.21.15.12:0/37465667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ec4100ab0 0x7f3ec410db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 -- 172.21.15.12:0/37465667 >> 172.21.15.12:0/37465667 conn(0x7f3ec4076ba0 msgr2=0x7f3ec40feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 -- 172.21.15.12:0/37465667 shutdown_connections 2024-05-08T23:51:59.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.467+0000 7f3ec9878700 1 -- 172.21.15.12:0/37465667 wait complete. 2024-05-08T23:51:59.476 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:51:59.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 -- 172.21.15.12:0/356874557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec100a00 msgr2=0x7fb6ec104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 --2- 172.21.15.12:0/356874557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec100a00 0x7fb6ec104a40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00098f0 tx=0x7fb6e0009c00 comp rx=0 tx=0).stop 2024-05-08T23:51:59.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 -- 172.21.15.12:0/356874557 shutdown_connections 2024-05-08T23:51:59.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 --2- 172.21.15.12:0/356874557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6ec105170 0x7fb6ec10ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 --2- 172.21.15.12:0/356874557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6ec100050 0x7fb6ec100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 --2- 172.21.15.12:0/356874557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec100a00 0x7fb6ec104a40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 -- 172.21.15.12:0/356874557 >> 172.21.15.12:0/356874557 conn(0x7fb6ec076fa0 msgr2=0x7fb6ec0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 -- 172.21.15.12:0/356874557 shutdown_connections 2024-05-08T23:51:59.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 -- 172.21.15.12:0/356874557 wait complete. 2024-05-08T23:51:59.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 Processor -- start 2024-05-08T23:51:59.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.624+0000 7fb6f2534700 1 -- start start 2024-05-08T23:51:59.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6f2534700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6ec100050 0x7fb6ec197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6f2534700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6ec105170 0x7fb6ec1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6f2534700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec19bd60 0x7fb6ec19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6f2534700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec19c7e0 con 0x7fb6ec105170 2024-05-08T23:51:59.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6f2534700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec19c950 con 0x7fb6ec100050 2024-05-08T23:51:59.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6f2534700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec19cb10 con 0x7fb6ec19bd60 2024-05-08T23:51:59.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6ec105170 0x7fb6ec1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6ec105170 0x7fb6ec1979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42218/0 (socket says 172.21.15.12:42218) 2024-05-08T23:51:59.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6eb7fe700 1 -- 172.21.15.12:0/1722981033 learned_addr learned my addr 172.21.15.12:0/1722981033 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6f0ad1700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec19bd60 0x7fb6ec19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6ec100050 0x7fb6ec197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6eb7fe700 1 -- 172.21.15.12:0/1722981033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec19bd60 msgr2=0x7fb6ec19c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec19bd60 0x7fb6ec19c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6eb7fe700 1 -- 172.21.15.12:0/1722981033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6ec100050 msgr2=0x7fb6ec197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6ec100050 0x7fb6ec197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.625+0000 7fb6eb7fe700 1 -- 172.21.15.12:0/1722981033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc0095f0 con 0x7fb6ec105170 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.626+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6ec105170 0x7fb6ec1979c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009230 tx=0x7fb6e000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.626+0000 7fb6e97fa700 1 -- 172.21.15.12:0/1722981033 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e001c070 con 0x7fb6ec105170 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.626+0000 7fb6e97fa700 1 -- 172.21.15.12:0/1722981033 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6e000b9c0 con 0x7fb6ec105170 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.626+0000 7fb6f2534700 1 -- 172.21.15.12:0/1722981033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e0009520 con 0x7fb6ec105170 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.626+0000 7fb6e97fa700 1 -- 172.21.15.12:0/1722981033 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00038e0 con 0x7fb6ec105170 2024-05-08T23:51:59.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.626+0000 7fb6f2534700 1 -- 172.21.15.12:0/1722981033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec1a42a0 con 0x7fb6ec105170 2024-05-08T23:51:59.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.628+0000 7fb6e97fa700 1 -- 172.21.15.12:0/1722981033 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb6e0020030 con 0x7fb6ec105170 2024-05-08T23:51:59.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.629+0000 7fb6e97fa700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6d40a1290 0x7fb6d40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.629+0000 7fb6e97fa700 1 -- 172.21.15.12:0/1722981033 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb6e00cda60 con 0x7fb6ec105170 2024-05-08T23:51:59.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.629+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6d40a1290 0x7fb6d40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.629+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6d40a1290 0x7fb6d40a3750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005fd0 tx=0x7fb6dc009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.630+0000 7fb6fe6eeb80 1 Processor -- start 2024-05-08T23:51:59.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.630+0000 7fb6fe6eeb80 1 -- start start 2024-05-08T23:51:59.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.630+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2479e00 0x559ad247a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.630+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 0x559ad2474350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.630+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 0x559ad2474cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.630+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad23cb150 con 0x559ad2474890 2024-05-08T23:51:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.630+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad23c8b10 con 0x559ad2479e00 2024-05-08T23:51:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.630+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad23c4400 con 0x559ad247a720 2024-05-08T23:51:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6ebfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2479e00 0x559ad247a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6ebfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2479e00 0x559ad247a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33068/0 (socket says 172.21.15.12:33068) 2024-05-08T23:51:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6ebfff700 1 -- 172.21.15.12:0/840459230 learned_addr learned my addr 172.21.15.12:0/840459230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/840459230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 0x559ad2474350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6f0ad1700 1 --2- 172.21.15.12:0/840459230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 0x559ad2474cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6ebfff700 1 -- 172.21.15.12:0/840459230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 msgr2=0x559ad2474350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/840459230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 0x559ad2474350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6ebfff700 1 -- 172.21.15.12:0/840459230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 msgr2=0x559ad2474cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/840459230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 0x559ad2474cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6ebfff700 1 -- 172.21.15.12:0/840459230 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ad22453b0 con 0x559ad2479e00 2024-05-08T23:51:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/840459230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 0x559ad2474350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/840459230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2479e00 0x559ad247a1e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00e670 tx=0x7fb6dc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56fb641848e4d867 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6d27fc700 1 -- 172.21.15.12:0/840459230 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00ed30 con 0x559ad2479e00 2024-05-08T23:51:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6d27fc700 1 -- 172.21.15.12:0/840459230 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6dc021070 con 0x559ad2479e00 2024-05-08T23:51:59.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.631+0000 7fb6d27fc700 1 -- 172.21.15.12:0/840459230 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc01d670 con 0x559ad2479e00 2024-05-08T23:51:59.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.632+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/840459230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2479e00 msgr2=0x559ad247a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.632+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/840459230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2479e00 0x559ad247a1e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00e670 tx=0x7fb6dc014000 comp rx=0 tx=0).stop 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.632+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/840459230 shutdown_connections 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.632+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/840459230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 0x559ad2474cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.632+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/840459230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2479e00 0x559ad247a1e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.632+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/840459230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 0x559ad2474350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.632+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/840459230 >> 172.21.15.12:0/840459230 conn(0x559ad23d2210 msgr2=0x559ad246edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.632+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/840459230 shutdown_connections 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.632+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/840459230 wait complete. 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6fe6eeb80 1 -- start start 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6fe6eeb80 1 Processor -- start 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2474890 0x559ad247db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad247e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 0x559ad247e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad2360560 con 0x559ad2479e00 2024-05-08T23:51:59.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad2360200 con 0x559ad2474890 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad22476a0 con 0x559ad247a720 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6fe6eeb80 10 client.0 _ll_register_callbacks cb 0x559ad2369b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad247e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6ebfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2474890 0x559ad247db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad247e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42250/0 (socket says 172.21.15.12:42250) 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6eb7fe700 1 -- 172.21.15.12:0/3120444815 learned_addr learned my addr 172.21.15.12:0/3120444815 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.633+0000 7fb6f0ad1700 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 0x559ad247e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6ebfff700 1 -- 172.21.15.12:0/3120444815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 msgr2=0x559ad247e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 0x559ad247e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6ebfff700 1 -- 172.21.15.12:0/3120444815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 msgr2=0x559ad247e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad247e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6ebfff700 1 -- 172.21.15.12:0/3120444815 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec1a42a0 con 0x559ad2474890 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2474890 0x559ad247db00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00e640 tx=0x7fb6dc00eb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5af4b20b81d8bdb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/3120444815 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc005040 con 0x559ad2474890 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/3120444815 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb6dc0051a0 con 0x559ad2474890 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/3120444815 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ad231d270 con 0x559ad2474890 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/3120444815 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6dc005380 con 0x559ad2474890 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6fe6eeb80 10 client.26786 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/3120444815 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fb6dc017070 con 0x559ad2474890 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.634+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/3120444815 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6dc03d020 con 0x559ad2474890 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.635+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/3120444815 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6dc01c710 con 0x559ad2474890 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.635+0000 7fb6fe6eeb80 10 client.26786 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.635+0000 7fb6fe6eeb80 10 client.26786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.635+0000 7fb6fe6eeb80 20 client.26786 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.635+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559ad24924c0 0x559ad24948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.635+0000 7fb6fe6eeb80 4 client.26786 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.635+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/3120444815 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ad2443810 con 0x559ad24924c0 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.635+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559ad24924c0 0x559ad24948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.635+0000 7fb6cb7fe700 10 client.26786 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.635+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559ad24924c0 0x559ad24948b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf8aae5295afc6 server_cookie=fbba39768f401419 in_seq=0 out_seq=0 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/3120444815 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559ad2443810 con 0x559ad24924c0 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6cb7fe700 10 client.26786 handle_command_reply: tid=0 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 client.26786 shutdown 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/3120444815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2474890 msgr2=0x559ad247db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2474890 0x559ad247db00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00e640 tx=0x7fb6dc00eb30 comp rx=0 tx=0).stop 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/3120444815 shutdown_connections 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad247a720 0x559ad247e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559ad24924c0 0x559ad24948b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad2474890 0x559ad247db00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/3120444815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad247e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6cb7fe700 0 client.26786 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:59.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/3120444815 >> 172.21.15.12:0/3120444815 conn(0x559ad23d2210 msgr2=0x559ad246e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/3120444815 shutdown_connections 2024-05-08T23:51:59.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/3120444815 wait complete. 2024-05-08T23:51:59.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.636+0000 7fb6fe6eeb80 20 client.26786 trim_cache size 0 max 16384 2024-05-08T23:51:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6fe6eeb80 1 Processor -- start 2024-05-08T23:51:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6fe6eeb80 1 -- start start 2024-05-08T23:51:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2474890 0x559ad2474350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad24794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad247a720 0x559ad246ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad246fae0 con 0x559ad2479e00 2024-05-08T23:51:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad2499ac0 con 0x559ad247a720 2024-05-08T23:51:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad246f8e0 con 0x559ad2474890 2024-05-08T23:51:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad24794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad24794a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42254/0 (socket says 172.21.15.12:42254) 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6ebfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2474890 0x559ad2474350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6eb7fe700 1 -- 172.21.15.12:0/2783705572 learned_addr learned my addr 172.21.15.12:0/2783705572 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6f0ad1700 1 --2- 172.21.15.12:0/2783705572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad247a720 0x559ad246ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6eb7fe700 1 -- 172.21.15.12:0/2783705572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2474890 msgr2=0x559ad2474350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/2783705572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2474890 0x559ad2474350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6eb7fe700 1 -- 172.21.15.12:0/2783705572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad247a720 msgr2=0x559ad246ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/2783705572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad247a720 0x559ad246ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.651+0000 7fb6eb7fe700 1 -- 172.21.15.12:0/2783705572 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ad2474f40 con 0x559ad2479e00 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/2783705572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad24794a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb6e009ada0 tx=0x7fb6e0005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8970e7cf81448901 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6d2ffd700 1 -- 172.21.15.12:0/2783705572 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00a2fd0 con 0x559ad2479e00 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6d2ffd700 1 -- 172.21.15.12:0/2783705572 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6e00ba040 con 0x559ad2479e00 2024-05-08T23:51:59.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6d2ffd700 1 -- 172.21.15.12:0/2783705572 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00050d0 con 0x559ad2479e00 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/2783705572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 msgr2=0x559ad24794a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/2783705572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad24794a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb6e009ada0 tx=0x7fb6e0005470 comp rx=0 tx=0).stop 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/2783705572 shutdown_connections 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/2783705572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad247a720 0x559ad246ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/2783705572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2474890 0x559ad2474350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/2783705572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2479e00 0x559ad24794a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.652+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/2783705572 >> 172.21.15.12:0/2783705572 conn(0x559ad23d2210 msgr2=0x559ad247c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.653+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/2783705572 shutdown_connections 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.653+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/2783705572 wait complete. 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.653+0000 7fb6fe6eeb80 1 -- start start 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.653+0000 7fb6fe6eeb80 1 Processor -- start 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.653+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 0x559ad2485160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.653+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2479e00 0x559ad24856a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.653+0000 7fb6fe6eeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad247a720 0x559ad24836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.653+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad2483dd0 con 0x559ad2474890 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.653+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad2483f40 con 0x559ad247a720 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.653+0000 7fb6fe6eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad2484100 con 0x559ad2479e00 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6fe6eeb80 10 client.0 _ll_register_callbacks cb 0x559ad2369b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6ebfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 0x559ad2485160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6eb7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2479e00 0x559ad24856a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6ebfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 0x559ad2485160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42262/0 (socket says 172.21.15.12:42262) 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6ebfff700 1 -- 172.21.15.12:0/1642826310 learned_addr learned my addr 172.21.15.12:0/1642826310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6f0ad1700 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad247a720 0x559ad24836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6ebfff700 1 -- 172.21.15.12:0/1642826310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2479e00 msgr2=0x559ad24856a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2479e00 0x559ad24856a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6ebfff700 1 -- 172.21.15.12:0/1642826310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad247a720 msgr2=0x559ad24836c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad247a720 0x559ad24836c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6ebfff700 1 -- 172.21.15.12:0/1642826310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e0009520 con 0x559ad2474890 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2479e00 0x559ad24856a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6ebfff700 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 0x559ad2485160 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc01ca70 tx=0x7fb6dc01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908c76a629539dcc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/1642826310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00e980 con 0x559ad2474890 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/1642826310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb6dc03e080 con 0x559ad2474890 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.654+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/1642826310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6dc021070 con 0x559ad2474890 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/1642826310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6dc01c620 con 0x559ad2474890 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6fe6eeb80 10 client.17001 fetch_fsmap learned FSMap version 31 2024-05-08T23:51:59.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/1642826310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559ad21f1b60 con 0x559ad2474890 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/1642826310 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6dc021070 con 0x559ad2474890 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/1642826310 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6dc00e6a0 con 0x559ad2474890 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6fe6eeb80 10 client.17001 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6fe6eeb80 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6fe6eeb80 20 client.17001 populate_metadata read hostname 'smithi012' 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559ad24e9a40 0x559ad24ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6fe6eeb80 4 client.17001 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/1642826310 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559ad23bcec0 con 0x559ad24e9a40 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.655+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559ad24e9a40 0x559ad24ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6cb7fe700 10 client.17001 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6eb7fe700 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559ad24e9a40 0x559ad24ebe60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=753938d2aa9fe4d5 server_cookie=9552ed611973a5c0 in_seq=0 out_seq=0 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6cb7fe700 1 -- 172.21.15.12:0/1642826310 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559ad23bcec0 con 0x559ad24e9a40 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6cb7fe700 10 client.17001 handle_command_reply: tid=0 2024-05-08T23:51:59.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6fe6eeb80 1 client.17001 shutdown 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/1642826310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 msgr2=0x559ad2485160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 0x559ad2485160 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc01ca70 tx=0x7fb6dc01ce40 comp rx=0 tx=0).stop 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/1642826310 shutdown_connections 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559ad24e9a40 0x559ad24ebe60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ad247a720 0x559ad24836c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559ad2474890 0x559ad2485160 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6cb7fe700 0 client.17001 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6fe6eeb80 1 --2- 172.21.15.12:0/1642826310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ad2479e00 0x559ad24856a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.656+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/1642826310 >> 172.21.15.12:0/1642826310 conn(0x559ad23d2210 msgr2=0x559ad2479310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/1642826310 shutdown_connections 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6fe6eeb80 1 -- 172.21.15.12:0/1642826310 wait complete. 2024-05-08T23:51:59.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6fe6eeb80 20 client.17001 trim_cache size 0 max 16384 2024-05-08T23:51:59.658 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 -- 172.21.15.12:0/1722981033 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6d40a1290 msgr2=0x7fb6d40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6d40a1290 0x7fb6d40a3750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005fd0 tx=0x7fb6dc009310 comp rx=0 tx=0).stop 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 -- 172.21.15.12:0/1722981033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6ec105170 msgr2=0x7fb6ec1979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6ec105170 0x7fb6ec1979c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009230 tx=0x7fb6e000fa00 comp rx=0 tx=0).stop 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 -- 172.21.15.12:0/1722981033 shutdown_connections 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6d40a1290 0x7fb6d40a3750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec19bd60 0x7fb6ec19c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6ec100050 0x7fb6ec197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 --2- 172.21.15.12:0/1722981033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6ec105170 0x7fb6ec1979c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 -- 172.21.15.12:0/1722981033 >> 172.21.15.12:0/1722981033 conn(0x7fb6ec076fa0 msgr2=0x7fb6ec069930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 -- 172.21.15.12:0/1722981033 shutdown_connections 2024-05-08T23:51:59.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:51:59.657+0000 7fb6f2534700 1 -- 172.21.15.12:0/1722981033 wait complete. 2024-05-08T23:51:59.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:51:59 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/534803219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:51:59.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:51:59 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/534803219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:51:59.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 330473 10.97 MB/sec execute 115 sec latency 31.047 ms 2024-05-08T23:51:59.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:51:59 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/534803219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:52:00.667 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:52:00.668 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:52:00.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:00 smithi204 ceph-mon[47373]: pgmap v172: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-05-08T23:52:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:00 smithi160 ceph-mon[46152]: pgmap v172: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-05-08T23:52:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 -- 172.21.15.12:0/545435983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c105160 msgr2=0x7f8f4c10cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 --2- 172.21.15.12:0/545435983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c105160 0x7f8f4c10cce0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009f90 tx=0x7f8f4400b330 comp rx=0 tx=0).stop 2024-05-08T23:52:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 -- 172.21.15.12:0/545435983 shutdown_connections 2024-05-08T23:52:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 --2- 172.21.15.12:0/545435983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c105160 0x7f8f4c10cce0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 --2- 172.21.15.12:0/545435983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4c100060 0x7f8f4c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 --2- 172.21.15.12:0/545435983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f4c100a10 0x7f8f4c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 -- 172.21.15.12:0/545435983 >> 172.21.15.12:0/545435983 conn(0x7f8f4c076f40 msgr2=0x7f8f4c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 -- 172.21.15.12:0/545435983 shutdown_connections 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 -- 172.21.15.12:0/545435983 wait complete. 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 Processor -- start 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.820+0000 7f8f534a9700 1 -- start start 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f534a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4c100060 0x7f8f4c101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f534a9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c105160 0x7f8f4c102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f534a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f4c102550 0x7f8f4c1a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f534a9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1a5d60 con 0x7f8f4c100060 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f534a9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1a5ed0 con 0x7f8f4c102550 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f534a9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1a6090 con 0x7f8f4c105160 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51245700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4c100060 0x7f8f4c101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f50a44700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c105160 0x7f8f4c102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51a46700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f4c102550 0x7f8f4c1a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51245700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4c100060 0x7f8f4c101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42276/0 (socket says 172.21.15.12:42276) 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f50a44700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c105160 0x7f8f4c102010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50736/0 (socket says 172.21.15.12:50736) 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51245700 1 -- 172.21.15.12:0/2920193625 learned_addr learned my addr 172.21.15.12:0/2920193625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51a46700 1 -- 172.21.15.12:0/2920193625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c105160 msgr2=0x7f8f4c102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51a46700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c105160 0x7f8f4c102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51a46700 1 -- 172.21.15.12:0/2920193625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4c100060 msgr2=0x7f8f4c101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51a46700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4c100060 0x7f8f4c101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51a46700 1 -- 172.21.15.12:0/2920193625 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f44009c40 con 0x7f8f4c102550 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51245700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4c100060 0x7f8f4c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.821+0000 7f8f51a46700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f4c102550 0x7f8f4c1a5720 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400b6c0 tx=0x7f8f44008a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.822+0000 7f8f3e7fc700 1 -- 172.21.15.12:0/2920193625 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4400e040 con 0x7f8f4c102550 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.822+0000 7f8f534a9700 1 -- 172.21.15.12:0/2920193625 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c1a6310 con 0x7f8f4c102550 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.822+0000 7f8f3e7fc700 1 -- 172.21.15.12:0/2920193625 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f4400be50 con 0x7f8f4c102550 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.822+0000 7f8f3e7fc700 1 -- 172.21.15.12:0/2920193625 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44005040 con 0x7f8f4c102550 2024-05-08T23:52:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.822+0000 7f8f534a9700 1 -- 172.21.15.12:0/2920193625 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c1a67e0 con 0x7f8f4c102550 2024-05-08T23:52:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.825+0000 7f8f3e7fc700 1 -- 172.21.15.12:0/2920193625 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8f4401d6e0 con 0x7f8f4c102550 2024-05-08T23:52:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.825+0000 7f8f3e7fc700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f380a1290 0x7f8f380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.825+0000 7f8f51245700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f380a1290 0x7f8f380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.825+0000 7f8f3e7fc700 1 -- 172.21.15.12:0/2920193625 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8f440cdf90 con 0x7f8f4c102550 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.826+0000 7f8f51245700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f380a1290 0x7f8f380a3750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f48005950 tx=0x7f8f48009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.826+0000 7f8f5f663b80 1 Processor -- start 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.826+0000 7f8f5f663b80 1 -- start start 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.826+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd1e80 0x564a8ecd2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.826+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd27a0 0x564a8eccc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.826+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.826+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ec231d0 con 0x564a8eccc910 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.826+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ec20b90 con 0x564a8ecd27a0 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.826+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ec1c480 con 0x564a8ecd1e80 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f51a46700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f51a46700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42290/0 (socket says 172.21.15.12:42290) 2024-05-08T23:52:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f51245700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd1e80 0x564a8ecd2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f51a46700 1 -- 172.21.15.12:0/357744210 learned_addr learned my addr 172.21.15.12:0/357744210 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f50a44700 1 --2- 172.21.15.12:0/357744210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd27a0 0x564a8eccc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f51a46700 1 -- 172.21.15.12:0/357744210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd1e80 msgr2=0x564a8ecd2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f51a46700 1 --2- 172.21.15.12:0/357744210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd1e80 0x564a8ecd2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f51a46700 1 -- 172.21.15.12:0/357744210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd27a0 msgr2=0x564a8eccc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f51a46700 1 --2- 172.21.15.12:0/357744210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd27a0 0x564a8eccc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f51a46700 1 -- 172.21.15.12:0/357744210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a8ea9d430 con 0x564a8eccc910 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f50a44700 1 --2- 172.21.15.12:0/357744210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd27a0 0x564a8eccc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f51a46700 1 --2- 172.21.15.12:0/357744210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecccd50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8f44097be0 tx=0x7f8f4401da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da6b1c09f25e754 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f377fe700 1 -- 172.21.15.12:0/357744210 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4401d470 con 0x564a8eccc910 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f377fe700 1 -- 172.21.15.12:0/357744210 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f4409b870 con 0x564a8eccc910 2024-05-08T23:52:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.827+0000 7f8f377fe700 1 -- 172.21.15.12:0/357744210 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f440a1070 con 0x564a8eccc910 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.828+0000 7f8f5f663b80 1 -- 172.21.15.12:0/357744210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 msgr2=0x564a8ecccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.828+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/357744210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecccd50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8f44097be0 tx=0x7f8f4401da80 comp rx=0 tx=0).stop 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.828+0000 7f8f5f663b80 1 -- 172.21.15.12:0/357744210 shutdown_connections 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.828+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/357744210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecccd50 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.828+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/357744210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd1e80 0x564a8ecd2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.828+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/357744210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd27a0 0x564a8eccc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.828+0000 7f8f5f663b80 1 -- 172.21.15.12:0/357744210 >> 172.21.15.12:0/357744210 conn(0x564a8ec2a290 msgr2=0x564a8ecc6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.828+0000 7f8f5f663b80 1 -- 172.21.15.12:0/357744210 shutdown_connections 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.828+0000 7f8f5f663b80 1 -- 172.21.15.12:0/357744210 wait complete. 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f5f663b80 1 -- start start 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f5f663b80 1 Processor -- start 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8eccc910 0x564a8ecd5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 0x564a8ecd6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd27a0 0x564a8ecd65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ebb85e0 con 0x564a8ecd27a0 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ebb8280 con 0x564a8ecd1e80 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ea9f720 con 0x564a8eccc910 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f5f663b80 10 client.0 _ll_register_callbacks cb 0x564a8ebc1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f51a46700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd27a0 0x564a8ecd65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f51a46700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd27a0 0x564a8ecd65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42306/0 (socket says 172.21.15.12:42306) 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f51a46700 1 -- 172.21.15.12:0/1035050864 learned_addr learned my addr 172.21.15.12:0/1035050864 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.829+0000 7f8f51245700 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8eccc910 0x564a8ecd5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f50a44700 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 0x564a8ecd6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f51a46700 1 -- 172.21.15.12:0/1035050864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8eccc910 msgr2=0x564a8ecd5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f51a46700 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8eccc910 0x564a8ecd5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f51a46700 1 -- 172.21.15.12:0/1035050864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 msgr2=0x564a8ecd6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f51a46700 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 0x564a8ecd6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f51a46700 1 -- 172.21.15.12:0/1035050864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f400098e0 con 0x564a8ecd27a0 2024-05-08T23:52:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f50a44700 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 0x564a8ecd6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f51a46700 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd27a0 0x564a8ecd65b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009790 tx=0x7f8f44009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda5d79c4f48cc2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1035050864 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f440a12d0 con 0x564a8ecd27a0 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1035050864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a8eb752f0 con 0x564a8ecd27a0 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1035050864 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8f4409efa0 con 0x564a8ecd27a0 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1035050864 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f440a3560 con 0x564a8ecd27a0 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f5f663b80 10 client.17013 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1035050864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f8f4c1a67e0 con 0x564a8ecd27a0 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.830+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1035050864 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a8eb752f0 con 0x564a8ecd27a0 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.831+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1035050864 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f4407c070 con 0x564a8ecd27a0 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.831+0000 7f8f5f663b80 10 client.17013 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.831+0000 7f8f5f663b80 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.831+0000 7f8f5f663b80 20 client.17013 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.831+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8ece5bc0 0x564a8ece7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.831+0000 7f8f5f663b80 4 client.17013 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.831+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1035050864 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a8ec9b890 con 0x564a8ece5bc0 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.831+0000 7f8f51245700 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8ece5bc0 0x564a8ece7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.831+0000 7f8f34ff9700 10 client.17013 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.831+0000 7f8f51245700 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8ece5bc0 0x564a8ece7fe0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef8d1b6c73fb282 server_cookie=de76b8698eb30153 in_seq=0 out_seq=0 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1035050864 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a8ec9b890 con 0x564a8ece5bc0 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f34ff9700 10 client.17013 handle_command_reply: tid=0 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 client.17013 shutdown 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1035050864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd27a0 msgr2=0x564a8ecd65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd27a0 0x564a8ecd65b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009790 tx=0x7f8f44009920 comp rx=0 tx=0).stop 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1035050864 shutdown_connections 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd27a0 0x564a8ecd65b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8eccc910 0x564a8ecd5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8ece5bc0 0x564a8ece7fe0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1035050864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 0x564a8ecd6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f34ff9700 0 client.17013 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1035050864 >> 172.21.15.12:0/1035050864 conn(0x564a8ec2a290 msgr2=0x564a8ecc6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1035050864 shutdown_connections 2024-05-08T23:52:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.832+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1035050864 wait complete. 2024-05-08T23:52:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.833+0000 7f8f5f663b80 20 client.17013 trim_cache size 0 max 16384 2024-05-08T23:52:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.852+0000 7f8f5f663b80 1 Processor -- start 2024-05-08T23:52:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.852+0000 7f8f5f663b80 1 -- start start 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.852+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8eccc910 0x564a8eccc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.852+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd1e80 0x564a8ecd14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.852+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 0x564a8ecc6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.852+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ecd2640 con 0x564a8ecd1e80 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.852+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ec9b8c0 con 0x564a8eccc910 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.852+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ecbf190 con 0x564a8ecd27a0 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f50a44700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd1e80 0x564a8ecd14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f50a44700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd1e80 0x564a8ecd14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42308/0 (socket says 172.21.15.12:42308) 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f50a44700 1 -- 172.21.15.12:0/2297285407 learned_addr learned my addr 172.21.15.12:0/2297285407 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f51245700 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8eccc910 0x564a8eccc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f51a46700 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 0x564a8ecc6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f50a44700 1 -- 172.21.15.12:0/2297285407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 msgr2=0x564a8ecc6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f50a44700 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 0x564a8ecc6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f50a44700 1 -- 172.21.15.12:0/2297285407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8eccc910 msgr2=0x564a8eccc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f50a44700 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8eccc910 0x564a8eccc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f50a44700 1 -- 172.21.15.12:0/2297285407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a8ecd5690 con 0x564a8ecd1e80 2024-05-08T23:52:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f51a46700 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 0x564a8ecc6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:52:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f51245700 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8eccc910 0x564a8eccc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f50a44700 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd1e80 0x564a8ecd14b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c103200 tx=0x7f8f40004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64b11387b86a231 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f37fff700 1 -- 172.21.15.12:0/2297285407 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40017070 con 0x564a8ecd1e80 2024-05-08T23:52:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f37fff700 1 -- 172.21.15.12:0/2297285407 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f40004cf0 con 0x564a8ecd1e80 2024-05-08T23:52:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.853+0000 7f8f37fff700 1 -- 172.21.15.12:0/2297285407 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40005280 con 0x564a8ecd1e80 2024-05-08T23:52:00.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.854+0000 7f8f5f663b80 1 -- 172.21.15.12:0/2297285407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd1e80 msgr2=0x564a8ecd14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.854+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd1e80 0x564a8ecd14b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c103200 tx=0x7f8f40004320 comp rx=0 tx=0).stop 2024-05-08T23:52:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.854+0000 7f8f5f663b80 1 -- 172.21.15.12:0/2297285407 shutdown_connections 2024-05-08T23:52:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.854+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 0x564a8ecc6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.854+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8eccc910 0x564a8eccc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.854+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/2297285407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8ecd1e80 0x564a8ecd14b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.854+0000 7f8f5f663b80 1 -- 172.21.15.12:0/2297285407 >> 172.21.15.12:0/2297285407 conn(0x564a8ec2a290 msgr2=0x564a8ecd4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.854+0000 7f8f5f663b80 1 -- 172.21.15.12:0/2297285407 shutdown_connections 2024-05-08T23:52:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.854+0000 7f8f5f663b80 1 -- 172.21.15.12:0/2297285407 wait complete. 2024-05-08T23:52:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.854+0000 7f8f5f663b80 1 -- start start 2024-05-08T23:52:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f5f663b80 1 Processor -- start 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 0x564a8eccec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f5f663b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 0x564a8eccf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ed3c1e0 con 0x564a8eccc910 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ed3c350 con 0x564a8ecd1e80 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f5f663b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ed3c510 con 0x564a8ecd27a0 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f5f663b80 10 client.0 _ll_register_callbacks cb 0x564a8ebc1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f51245700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f51245700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42318/0 (socket says 172.21.15.12:42318) 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f51245700 1 -- 172.21.15.12:0/1184785055 learned_addr learned my addr 172.21.15.12:0/1184785055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f50a44700 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 0x564a8eccec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.855+0000 7f8f51a46700 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 0x564a8eccf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f51245700 1 -- 172.21.15.12:0/1184785055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 msgr2=0x564a8eccf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f51245700 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 0x564a8eccf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f51245700 1 -- 172.21.15.12:0/1184785055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 msgr2=0x564a8eccec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f51245700 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 0x564a8eccec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f51245700 1 -- 172.21.15.12:0/1184785055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f48017070 con 0x564a8eccc910 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f51a46700 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 0x564a8eccf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f50a44700 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 0x564a8eccec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f51245700 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecce6f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800e540 tx=0x7f8f480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a4d542e8cab758 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1184785055 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48022040 con 0x564a8eccc910 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1184785055 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8f4801dab0 con 0x564a8eccc910 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1184785055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f4409b870 con 0x564a8eccc910 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1184785055 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f4801dc70 con 0x564a8eccc910 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f5f663b80 10 client.17019 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1184785055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f8f40009b80 con 0x564a8eccc910 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1184785055 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f4409b870 con 0x564a8eccc910 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.856+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1184785055 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f48005290 con 0x564a8eccc910 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.857+0000 7f8f5f663b80 10 client.17019 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.857+0000 7f8f5f663b80 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.857+0000 7f8f5f663b80 20 client.17019 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.857+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8ed3dcb0 0x564a8ed400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.857+0000 7f8f5f663b80 4 client.17019 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.857+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1184785055 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564a8ec14f40 con 0x564a8ed3dcb0 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.857+0000 7f8f50a44700 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8ed3dcb0 0x564a8ed400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.857+0000 7f8f50a44700 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8ed3dcb0 0x564a8ed400d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc23763127be9f0c server_cookie=75fe2e8c2d4d1b64 in_seq=0 out_seq=0 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.857+0000 7f8f34ff9700 10 client.17019 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f34ff9700 1 -- 172.21.15.12:0/1184785055 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564a8ec14f40 con 0x564a8ed3dcb0 2024-05-08T23:52:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f34ff9700 10 client.17019 handle_command_reply: tid=0 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f5f663b80 1 client.17019 shutdown 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1184785055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 msgr2=0x564a8ecce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecce6f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800e540 tx=0x7f8f480047c0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1184785055 shutdown_connections 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a8ecd27a0 0x564a8eccf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a8eccc910 0x564a8ecce6f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564a8ed3dcb0 0x564a8ed400d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f5f663b80 1 --2- 172.21.15.12:0/1184785055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564a8ecd1e80 0x564a8eccec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1184785055 >> 172.21.15.12:0/1184785055 conn(0x564a8ec2a290 msgr2=0x564a8ecd6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.858+0000 7f8f34ff9700 0 client.17019 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1184785055 shutdown_connections 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f5f663b80 1 -- 172.21.15.12:0/1184785055 wait complete. 2024-05-08T23:52:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f5f663b80 20 client.17019 trim_cache size 0 max 16384 2024-05-08T23:52:00.860 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:52:00.860 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:52:00.860 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "91b1d2f3-6762-4a06-afd8-fb0827a23286", 2024-05-08T23:52:00.860 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:52:00.860 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:52:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 -- 172.21.15.12:0/2920193625 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f380a1290 msgr2=0x7f8f380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f380a1290 0x7f8f380a3750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f48005950 tx=0x7f8f48009310 comp rx=0 tx=0).stop 2024-05-08T23:52:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 -- 172.21.15.12:0/2920193625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f4c102550 msgr2=0x7f8f4c1a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f4c102550 0x7f8f4c1a5720 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400b6c0 tx=0x7f8f44008a10 comp rx=0 tx=0).stop 2024-05-08T23:52:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 -- 172.21.15.12:0/2920193625 shutdown_connections 2024-05-08T23:52:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f380a1290 0x7f8f380a3750 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f4c102550 0x7f8f4c1a5720 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4c100060 0x7f8f4c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 --2- 172.21.15.12:0/2920193625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c105160 0x7f8f4c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:00.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 -- 172.21.15.12:0/2920193625 >> 172.21.15.12:0/2920193625 conn(0x7f8f4c076f40 msgr2=0x7f8f4c0682d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:00.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 -- 172.21.15.12:0/2920193625 shutdown_connections 2024-05-08T23:52:00.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:00.859+0000 7f8f534a9700 1 -- 172.21.15.12:0/2920193625 wait complete. 2024-05-08T23:52:00.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 332232 10.94 MB/sec execute 116 sec latency 27.708 ms 2024-05-08T23:52:00.868 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:00 smithi012 ceph-mon[42503]: pgmap v172: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-05-08T23:52:00.869 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:52:01.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.015+0000 7f19ff5f7700 1 -- 172.21.15.12:0/245744052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8100060 msgr2=0x7f19f8100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.015+0000 7f19ff5f7700 1 --2- 172.21.15.12:0/245744052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8100060 0x7f19f8100440 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009990 tx=0x7f19f4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.015+0000 7f19ff5f7700 1 -- 172.21.15.12:0/245744052 shutdown_connections 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.015+0000 7f19ff5f7700 1 --2- 172.21.15.12:0/245744052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f8105160 0x7f19f810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.015+0000 7f19ff5f7700 1 --2- 172.21.15.12:0/245744052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f8100060 0x7f19f8100440 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.015+0000 7f19ff5f7700 1 --2- 172.21.15.12:0/245744052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f8100a10 0x7f19f8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.015+0000 7f19ff5f7700 1 -- 172.21.15.12:0/245744052 >> 172.21.15.12:0/245744052 conn(0x7f19f8076f40 msgr2=0x7f19f8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.015+0000 7f19ff5f7700 1 -- 172.21.15.12:0/245744052 shutdown_connections 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.015+0000 7f19ff5f7700 1 -- 172.21.15.12:0/245744052 wait complete. 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19ff5f7700 1 Processor -- start 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19ff5f7700 1 -- start start 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19ff5f7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f8100060 0x7f19f81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19ff5f7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f8105160 0x7f19f8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19ff5f7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f819de90 0x7f19f819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19ff5f7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f819e910 con 0x7f19f819de90 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19ff5f7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f819ea80 con 0x7f19f8100060 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19ff5f7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f819ec40 con 0x7f19f8105160 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19fdb94700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f819de90 0x7f19f819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19fdb94700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f819de90 0x7f19f819e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42336/0 (socket says 172.21.15.12:42336) 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19fdb94700 1 -- 172.21.15.12:0/118844044 learned_addr learned my addr 172.21.15.12:0/118844044 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19fd393700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f8100060 0x7f19f81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.016+0000 7f19fcb92700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f8105160 0x7f19f8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19fdb94700 1 -- 172.21.15.12:0/118844044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f8105160 msgr2=0x7f19f8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19fdb94700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f8105160 0x7f19f8199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19fdb94700 1 -- 172.21.15.12:0/118844044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f8100060 msgr2=0x7f19f81995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19fdb94700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f8100060 0x7f19f81995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19fdb94700 1 -- 172.21.15.12:0/118844044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40095f0 con 0x7f19f819de90 2024-05-08T23:52:01.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19fdb94700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f819de90 0x7f19f819e310 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f19f000ef30 tx=0x7f19f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:01.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19ee7fc700 1 -- 172.21.15.12:0/118844044 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0018070 con 0x7f19f819de90 2024-05-08T23:52:01.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19ff5f7700 1 -- 172.21.15.12:0/118844044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f81a6140 con 0x7f19f819de90 2024-05-08T23:52:01.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19ee7fc700 1 -- 172.21.15.12:0/118844044 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19f000f040 con 0x7f19f819de90 2024-05-08T23:52:01.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19ee7fc700 1 -- 172.21.15.12:0/118844044 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f000cd00 con 0x7f19f819de90 2024-05-08T23:52:01.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.017+0000 7f19ff5f7700 1 -- 172.21.15.12:0/118844044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f81a6620 con 0x7f19f819de90 2024-05-08T23:52:01.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.020+0000 7f19ee7fc700 1 -- 172.21.15.12:0/118844044 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f19f0003a00 con 0x7f19f819de90 2024-05-08T23:52:01.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.021+0000 7f19ee7fc700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19e40a1290 0x7f19e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.021+0000 7f19ee7fc700 1 -- 172.21.15.12:0/118844044 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f19f00cc7a0 con 0x7f19f819de90 2024-05-08T23:52:01.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.021+0000 7f19fd393700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19e40a1290 0x7f19e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f19fd393700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19e40a1290 0x7f19e40a3750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b2e0 tx=0x7f19f4005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:01.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f1a0b7b1b80 1 Processor -- start 2024-05-08T23:52:01.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f1a0b7b1b80 1 -- start start 2024-05-08T23:52:01.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef3e00 0x5652c1ef41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1ef4720 0x5652c1eee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1eee890 0x5652c1eeecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1e45150 con 0x5652c1eee890 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1e42b10 con 0x5652c1ef4720 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1e3e400 con 0x5652c1ef3e00 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f19fd393700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef3e00 0x5652c1ef41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f19fdb94700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1eee890 0x5652c1eeecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f19fcb92700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1ef4720 0x5652c1eee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f19fd393700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef3e00 0x5652c1ef41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50804/0 (socket says 172.21.15.12:50804) 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f19fcb92700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1ef4720 0x5652c1eee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33190/0 (socket says 172.21.15.12:33190) 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.022+0000 7f19fd393700 1 -- 172.21.15.12:0/251457495 learned_addr learned my addr 172.21.15.12:0/251457495 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.023+0000 7f19fdb94700 1 -- 172.21.15.12:0/251457495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef3e00 msgr2=0x5652c1ef41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.023+0000 7f19fdb94700 1 --2- 172.21.15.12:0/251457495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef3e00 0x5652c1ef41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.023+0000 7f19fdb94700 1 -- 172.21.15.12:0/251457495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1ef4720 msgr2=0x5652c1eee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:01.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.023+0000 7f19fdb94700 1 --2- 172.21.15.12:0/251457495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1ef4720 0x5652c1eee350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.023+0000 7f19fdb94700 1 -- 172.21.15.12:0/251457495 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652c1cbf3b0 con 0x5652c1eee890 2024-05-08T23:52:01.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.023+0000 7f19fcb92700 1 --2- 172.21.15.12:0/251457495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1ef4720 0x5652c1eee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:01.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.023+0000 7f19fdb94700 1 --2- 172.21.15.12:0/251457495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1eee890 0x5652c1eeecd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f19f00c9f30 tx=0x7f19f00994c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170b2a7cdff8986c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:01.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.023+0000 7f19e37fe700 1 -- 172.21.15.12:0/251457495 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0004ad0 con 0x5652c1eee890 2024-05-08T23:52:01.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.023+0000 7f19e37fe700 1 -- 172.21.15.12:0/251457495 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19f001b320 con 0x5652c1eee890 2024-05-08T23:52:01.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.023+0000 7f19e37fe700 1 -- 172.21.15.12:0/251457495 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0099be0 con 0x5652c1eee890 2024-05-08T23:52:01.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.024+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/251457495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1eee890 msgr2=0x5652c1eeecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:01.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.024+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/251457495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1eee890 0x5652c1eeecd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f19f00c9f30 tx=0x7f19f00994c0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.024+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/251457495 shutdown_connections 2024-05-08T23:52:01.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.024+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/251457495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1eee890 0x5652c1eeecd0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.024+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/251457495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef3e00 0x5652c1ef41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.024+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/251457495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1ef4720 0x5652c1eee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.024+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/251457495 >> 172.21.15.12:0/251457495 conn(0x5652c1e4c210 msgr2=0x5652c1ee8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:01.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.024+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/251457495 shutdown_connections 2024-05-08T23:52:01.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.024+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/251457495 wait complete. 2024-05-08T23:52:01.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f1a0b7b1b80 1 -- start start 2024-05-08T23:52:01.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f1a0b7b1b80 1 Processor -- start 2024-05-08T23:52:01.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1ef7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ef8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1dda560 con 0x5652c1ef3e00 2024-05-08T23:52:01.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1dda200 con 0x5652c1eee890 2024-05-08T23:52:01.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1cc16a0 con 0x5652c1ef4720 2024-05-08T23:52:01.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f1a0b7b1b80 10 client.0 _ll_register_callbacks cb 0x5652c1de3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f19fcb92700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f19fcb92700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42350/0 (socket says 172.21.15.12:42350) 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f19fcb92700 1 -- 172.21.15.12:0/2659859725 learned_addr learned my addr 172.21.15.12:0/2659859725 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f19fd393700 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1ef7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.025+0000 7f19fdb94700 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ef8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f19fcb92700 1 -- 172.21.15.12:0/2659859725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 msgr2=0x5652c1ef8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f19fcb92700 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ef8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f19fcb92700 1 -- 172.21.15.12:0/2659859725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 msgr2=0x5652c1ef7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f19fcb92700 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1ef7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f19fcb92700 1 -- 172.21.15.12:0/2659859725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e800b8e0 con 0x5652c1ef3e00 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f19fcb92700 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef8020 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f19e80042c0 tx=0x7f19e80045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a861df4d4191ad2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f19e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f19e0ff9700 1 -- 172.21.15.12:0/2659859725 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8016070 con 0x5652c1ef3e00 2024-05-08T23:52:01.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f19e0ff9700 1 -- 172.21.15.12:0/2659859725 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f19e8003d00 con 0x5652c1ef3e00 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2659859725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652c1d97270 con 0x5652c1ef3e00 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f19e0ff9700 1 -- 172.21.15.12:0/2659859725 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19e8004830 con 0x5652c1ef3e00 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f1a0b7b1b80 10 client.17031 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.026+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2659859725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f19f81a6620 con 0x5652c1ef3e00 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f19e0ff9700 1 -- 172.21.15.12:0/2659859725 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5652c1d97270 con 0x5652c1ef3e00 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f19e0ff9700 1 -- 172.21.15.12:0/2659859725 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f19e80244d0 con 0x5652c1ef3e00 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f1a0b7b1b80 10 client.17031 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f1a0b7b1b80 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f1a0b7b1b80 20 client.17031 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5652c1f08280 0x5652c1f0a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f1a0b7b1b80 4 client.17031 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2659859725 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652c1ebd810 con 0x5652c1f08280 2024-05-08T23:52:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f19fd393700 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5652c1f08280 0x5652c1f0a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f19e0ff9700 10 client.17031 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.027+0000 7f19fd393700 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5652c1f08280 0x5652c1f0a6a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44305250f0188af5 server_cookie=4d1e7edc65bcd6f1 in_seq=0 out_seq=0 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f19e0ff9700 1 -- 172.21.15.12:0/2659859725 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652c1ebd810 con 0x5652c1f08280 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f19e0ff9700 10 client.17031 handle_command_reply: tid=0 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f1a0b7b1b80 1 client.17031 shutdown 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2659859725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 msgr2=0x5652c1ef8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef8020 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f19e80042c0 tx=0x7f19e80045d0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2659859725 shutdown_connections 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ef8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5652c1f08280 0x5652c1f0a6a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1ef7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/2659859725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef8020 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f19e0ff9700 0 client.17031 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.028+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2659859725 >> 172.21.15.12:0/2659859725 conn(0x5652c1e4c210 msgr2=0x5652c1ee85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:01.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.029+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2659859725 shutdown_connections 2024-05-08T23:52:01.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.029+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2659859725 wait complete. 2024-05-08T23:52:01.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.029+0000 7f1a0b7b1b80 20 client.17031 trim_cache size 0 max 16384 2024-05-08T23:52:01.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.047+0000 7f1a0b7b1b80 1 Processor -- start 2024-05-08T23:52:01.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.047+0000 7f1a0b7b1b80 1 -- start start 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.047+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1eee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.047+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1f03c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.047+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ee8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.047+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1f04dd0 con 0x5652c1ef3e00 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.047+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1ef9180 con 0x5652c1eee890 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.047+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1eea070 con 0x5652c1ef4720 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fcb92700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1f03c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fd393700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1eee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fcb92700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1f03c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42358/0 (socket says 172.21.15.12:42358) 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fcb92700 1 -- 172.21.15.12:0/2080021624 learned_addr learned my addr 172.21.15.12:0/2080021624 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fdb94700 1 --2- 172.21.15.12:0/2080021624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ee8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fcb92700 1 -- 172.21.15.12:0/2080021624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 msgr2=0x5652c1ee8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fcb92700 1 --2- 172.21.15.12:0/2080021624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ee8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fcb92700 1 -- 172.21.15.12:0/2080021624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 msgr2=0x5652c1eee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fcb92700 1 --2- 172.21.15.12:0/2080021624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1eee350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fcb92700 1 -- 172.21.15.12:0/2080021624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652c1c6bb60 con 0x5652c1ef3e00 2024-05-08T23:52:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19fcb92700 1 --2- 172.21.15.12:0/2080021624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1f03c20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f19e8000c00 tx=0x7f19e80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6165de8348510704 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19e3fff700 1 -- 172.21.15.12:0/2080021624 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8016070 con 0x5652c1ef3e00 2024-05-08T23:52:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19e3fff700 1 -- 172.21.15.12:0/2080021624 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e8017040 con 0x5652c1ef3e00 2024-05-08T23:52:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.048+0000 7f19e3fff700 1 -- 172.21.15.12:0/2080021624 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e801bb90 con 0x5652c1ef3e00 2024-05-08T23:52:01.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2080021624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 msgr2=0x5652c1f03c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:01.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/2080021624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1f03c20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f19e8000c00 tx=0x7f19e80039e0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2080021624 shutdown_connections 2024-05-08T23:52:01.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/2080021624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ee8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/2080021624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1eee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/2080021624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1f03c20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2080021624 >> 172.21.15.12:0/2080021624 conn(0x5652c1e4c210 msgr2=0x5652c1ef2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2080021624 shutdown_connections 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/2080021624 wait complete. 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 -- start start 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.049+0000 7f1a0b7b1b80 1 Processor -- start 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1ef05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f1a0b7b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ef1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1f5e7e0 con 0x5652c1ef3e00 2024-05-08T23:52:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1f5e920 con 0x5652c1eee890 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f1a0b7b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c1f5eae0 con 0x5652c1ef4720 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f1a0b7b1b80 10 client.0 _ll_register_callbacks cb 0x5652c1de3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f19fcb92700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f19fd393700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1ef05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f19fcb92700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42374/0 (socket says 172.21.15.12:42374) 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f19fcb92700 1 -- 172.21.15.12:0/1759010390 learned_addr learned my addr 172.21.15.12:0/1759010390 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.050+0000 7f19fdb94700 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ef1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19fcb92700 1 -- 172.21.15.12:0/1759010390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 msgr2=0x5652c1ef1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19fcb92700 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ef1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19fcb92700 1 -- 172.21.15.12:0/1759010390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 msgr2=0x5652c1ef05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19fcb92700 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1ef05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19fcb92700 1 -- 172.21.15.12:0/1759010390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f81a6620 con 0x5652c1ef3e00 2024-05-08T23:52:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19fdb94700 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ef1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19fd393700 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1ef05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19fcb92700 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef0af0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f19e80059d0 tx=0x7f19e8024a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d880737e504809a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19e0ff9700 1 -- 172.21.15.12:0/1759010390 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8016070 con 0x5652c1ef3e00 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19e0ff9700 1 -- 172.21.15.12:0/1759010390 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f19e8029070 con 0x5652c1ef3e00 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/1759010390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19f001b320 con 0x5652c1ef3e00 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19e0ff9700 1 -- 172.21.15.12:0/1759010390 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19e801b880 con 0x5652c1ef3e00 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f1a0b7b1b80 10 client.17037 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/1759010390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5652c1e04560 con 0x5652c1ef3e00 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.051+0000 7f19e0ff9700 1 -- 172.21.15.12:0/1759010390 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19f001b320 con 0x5652c1ef3e00 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f19e0ff9700 1 -- 172.21.15.12:0/1759010390 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f19e8039b50 con 0x5652c1ef3e00 2024-05-08T23:52:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f1a0b7b1b80 10 client.17037 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f1a0b7b1b80 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f1a0b7b1b80 20 client.17037 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5652c1f5fe50 0x5652c1f62270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f1a0b7b1b80 4 client.17037 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/1759010390 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652c1e36ec0 con 0x5652c1f5fe50 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f19fd393700 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5652c1f5fe50 0x5652c1f62270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f19fd393700 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5652c1f5fe50 0x5652c1f62270 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cddccef77fd83a5 server_cookie=376a117d4b9d02eb in_seq=0 out_seq=0 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f19e0ff9700 10 client.17037 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f19e0ff9700 1 -- 172.21.15.12:0/1759010390 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5652c1e36ec0 con 0x5652c1f5fe50 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f19e0ff9700 10 client.17037 handle_command_reply: tid=0 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.052+0000 7f1a0b7b1b80 1 client.17037 shutdown 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/1759010390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 msgr2=0x5652c1ef0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef0af0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f19e80059d0 tx=0x7f19e8024a70 comp rx=0 tx=0).stop 2024-05-08T23:52:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/1759010390 shutdown_connections 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652c1ef4720 0x5652c1ef1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5652c1f5fe50 0x5652c1f62270 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5652c1eee890 0x5652c1ef05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 1 --2- 172.21.15.12:0/1759010390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5652c1ef3e00 0x5652c1ef0af0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19e0ff9700 0 client.17037 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/1759010390 >> 172.21.15.12:0/1759010390 conn(0x5652c1e4c210 msgr2=0x5652c1ef4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/1759010390 shutdown_connections 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 1 -- 172.21.15.12:0/1759010390 wait complete. 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f1a0b7b1b80 20 client.17037 trim_cache size 0 max 16384 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:52:01.055 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:52:01.056 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:52:01.056 INFO:teuthology.orchestra.run.smithi012.stdout: "91b1d2f3-6762-4a06-afd8-fb0827a23286": { 2024-05-08T23:52:01.056 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:52:01.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "91b1d2f3-6762-4a06-afd8-fb0827a23286", 2024-05-08T23:52:01.056 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:52:01.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:52:01.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:52:01.056 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19ff5f7700 1 -- 172.21.15.12:0/118844044 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19e40a1290 msgr2=0x7f19e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19ff5f7700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19e40a1290 0x7f19e40a3750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b2e0 tx=0x7f19f4005960 comp rx=0 tx=0).stop 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19ff5f7700 1 -- 172.21.15.12:0/118844044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f819de90 msgr2=0x7f19f819e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19ff5f7700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f819de90 0x7f19f819e310 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f19f000ef30 tx=0x7f19f000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19ff5f7700 1 -- 172.21.15.12:0/118844044 shutdown_connections 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19ff5f7700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f19e40a1290 0x7f19e40a3750 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19ff5f7700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19f819de90 0x7f19f819e310 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19ff5f7700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f8100060 0x7f19f81995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19ff5f7700 1 --2- 172.21.15.12:0/118844044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f8105160 0x7f19f8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.053+0000 7f19ff5f7700 1 -- 172.21.15.12:0/118844044 >> 172.21.15.12:0/118844044 conn(0x7f19f8076f40 msgr2=0x7f19f80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:01.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.054+0000 7f19ff5f7700 1 -- 172.21.15.12:0/118844044 shutdown_connections 2024-05-08T23:52:01.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:01.054+0000 7f19ff5f7700 1 -- 172.21.15.12:0/118844044 wait complete. 2024-05-08T23:52:01.062 INFO:tasks.cephfs.filesystem:scrub status for tag:91b1d2f3-6762-4a06-afd8-fb0827a23286 - {'path': '/', 'tag': '91b1d2f3-6762-4a06-afd8-fb0827a23286', 'options': 'recursive,force'} 2024-05-08T23:52:01.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:01 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:52:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:01 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:01 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:52:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:01 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:52:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:01 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:01 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:52:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:01 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:52:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:01 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:01 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:01 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:01 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:52:01.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:01 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:01 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:52:01.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:01 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:52:01.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:01 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:01 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:52:01.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:01 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:52:01.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:01 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:01 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:01 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 333973 10.95 MB/sec execute 117 sec latency 50.403 ms 2024-05-08T23:52:01.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:01 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:52:01.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:01 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:01 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:52:01.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:01 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:52:01.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:01 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:01 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:52:01.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:01 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:52:01.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:01 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:01 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:01.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:01 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:02.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:02 smithi204 ceph-mon[47373]: pgmap v173: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-05-08T23:52:02.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:02 smithi160 ceph-mon[46152]: pgmap v173: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-05-08T23:52:02.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 336634 10.97 MB/sec execute 118 sec latency 22.466 ms 2024-05-08T23:52:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:02 smithi012 ceph-mon[42503]: pgmap v173: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-05-08T23:52:03.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 338257 10.94 MB/sec execute 119 sec latency 63.980 ms 2024-05-08T23:52:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:04 smithi204 ceph-mon[47373]: pgmap v174: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-05-08T23:52:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:04 smithi160 ceph-mon[46152]: pgmap v174: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-05-08T23:52:04.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 340814 10.99 MB/sec execute 120 sec latency 13.673 ms 2024-05-08T23:52:04.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:04 smithi012 ceph-mon[42503]: pgmap v174: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-05-08T23:52:05.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:05 smithi204 ceph-mon[47373]: pgmap v175: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-05-08T23:52:05.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:05 smithi160 ceph-mon[46152]: pgmap v175: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-05-08T23:52:05.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 342948 10.96 MB/sec execute 121 sec latency 13.995 ms 2024-05-08T23:52:05.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:05 smithi012 ceph-mon[42503]: pgmap v175: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-05-08T23:52:06.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 344601 10.97 MB/sec execute 122 sec latency 52.141 ms 2024-05-08T23:52:07.865 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.92 MB/sec execute 123 sec latency 529.797 ms 2024-05-08T23:52:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.963+0000 7f59dd420700 1 -- 172.21.15.12:0/661721252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59d8106230 msgr2=0x7f59d810ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.963+0000 7f59dd420700 1 --2- 172.21.15.12:0/661721252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59d8106230 0x7f59d810ddb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f59d000b0d0 tx=0x7f59d000b3e0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.963+0000 7f59dd420700 1 -- 172.21.15.12:0/661721252 shutdown_connections 2024-05-08T23:52:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.963+0000 7f59dd420700 1 --2- 172.21.15.12:0/661721252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59d8106230 0x7f59d810ddb0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.964+0000 7f59dd420700 1 --2- 172.21.15.12:0/661721252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59d80699e0 0x7f59d8069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.964+0000 7f59dd420700 1 --2- 172.21.15.12:0/661721252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59d806a390 0x7f59d8105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.964+0000 7f59dd420700 1 -- 172.21.15.12:0/661721252 >> 172.21.15.12:0/661721252 conn(0x7f59d8076f40 msgr2=0x7f59d8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.964+0000 7f59dd420700 1 -- 172.21.15.12:0/661721252 shutdown_connections 2024-05-08T23:52:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.964+0000 7f59dd420700 1 -- 172.21.15.12:0/661721252 wait complete. 2024-05-08T23:52:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.964+0000 7f59dd420700 1 Processor -- start 2024-05-08T23:52:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.964+0000 7f59dd420700 1 -- start start 2024-05-08T23:52:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59dd420700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59d80699e0 0x7f59d8197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59dd420700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59d8106230 0x7f59d8197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59dd420700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59d819bcd0 0x7f59d819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59dd420700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d819c750 con 0x7f59d8106230 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59dd420700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d819c8c0 con 0x7f59d80699e0 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59dd420700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d819ca80 con 0x7f59d819bcd0 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59d8106230 0x7f59d8197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59d8106230 0x7f59d8197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42392/0 (socket says 172.21.15.12:42392) 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d67fc700 1 -- 172.21.15.12:0/664753854 learned_addr learned my addr 172.21.15.12:0/664753854 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59d80699e0 0x7f59d8197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d77fe700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59d819bcd0 0x7f59d819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d67fc700 1 -- 172.21.15.12:0/664753854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59d819bcd0 msgr2=0x7f59d819c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d67fc700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59d819bcd0 0x7f59d819c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d67fc700 1 -- 172.21.15.12:0/664753854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59d80699e0 msgr2=0x7f59d8197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d67fc700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59d80699e0 0x7f59d8197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d67fc700 1 -- 172.21.15.12:0/664753854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0009d00 con 0x7f59d8106230 2024-05-08T23:52:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.965+0000 7f59d67fc700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59d8106230 0x7f59d8197980 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00e970 tx=0x7f59cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.966+0000 7f59bffff700 1 -- 172.21.15.12:0/664753854 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc009790 con 0x7f59d8106230 2024-05-08T23:52:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.966+0000 7f59bffff700 1 -- 172.21.15.12:0/664753854 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59cc00c7e0 con 0x7f59d8106230 2024-05-08T23:52:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.966+0000 7f59dd420700 1 -- 172.21.15.12:0/664753854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d81a3fe0 con 0x7f59d8106230 2024-05-08T23:52:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.966+0000 7f59bffff700 1 -- 172.21.15.12:0/664753854 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0054e0 con 0x7f59d8106230 2024-05-08T23:52:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.966+0000 7f59dd420700 1 -- 172.21.15.12:0/664753854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d81a44b0 con 0x7f59d8106230 2024-05-08T23:52:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.969+0000 7f59bffff700 1 -- 172.21.15.12:0/664753854 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f59cc00c950 con 0x7f59d8106230 2024-05-08T23:52:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.969+0000 7f59bffff700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59c00a1310 0x7f59c00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.969+0000 7f59bffff700 1 -- 172.21.15.12:0/664753854 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f59cc014070 con 0x7f59d8106230 2024-05-08T23:52:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.969+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59c00a1310 0x7f59c00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.970+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59c00a1310 0x7f59c00a37d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009c60 tx=0x7f59c8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.970+0000 7f59e95dab80 1 Processor -- start 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.970+0000 7f59e95dab80 1 -- start start 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.970+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 0x55fafbc360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.970+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc317d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.970+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31d10 0x55fafbc3da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.970+0000 7f59e95dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbb840f0 con 0x55fafbc31350 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.970+0000 7f59e95dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbb81ab0 con 0x55fafbc31d10 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.970+0000 7f59e95dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbb7d3a0 con 0x55fafbc35ce0 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc317d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42406/0 (socket says 172.21.15.12:42406) 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d67fc700 1 -- 172.21.15.12:0/1007601286 learned_addr learned my addr 172.21.15.12:0/1007601286 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d77fe700 1 --2- 172.21.15.12:0/1007601286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31d10 0x55fafbc3da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/1007601286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 0x55fafbc360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d67fc700 1 -- 172.21.15.12:0/1007601286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 msgr2=0x55fafbc360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d67fc700 1 --2- 172.21.15.12:0/1007601286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 0x55fafbc360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d67fc700 1 -- 172.21.15.12:0/1007601286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31d10 msgr2=0x55fafbc3da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d67fc700 1 --2- 172.21.15.12:0/1007601286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31d10 0x55fafbc3da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d67fc700 1 -- 172.21.15.12:0/1007601286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fafb9fe350 con 0x55fafbc31350 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d77fe700 1 --2- 172.21.15.12:0/1007601286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31d10 0x55fafbc3da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59d67fc700 1 --2- 172.21.15.12:0/1007601286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc317d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009230 tx=0x7f59cc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1121c31d3a9a909b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59bd7fa700 1 -- 172.21.15.12:0/1007601286 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc09dd40 con 0x55fafbc31350 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59bd7fa700 1 -- 172.21.15.12:0/1007601286 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59cc0a2390 con 0x55fafbc31350 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.971+0000 7f59bd7fa700 1 -- 172.21.15.12:0/1007601286 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc09a3b0 con 0x55fafbc31350 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.972+0000 7f59e95dab80 1 -- 172.21.15.12:0/1007601286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 msgr2=0x55fafbc317d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.972+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1007601286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc317d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009230 tx=0x7f59cc00cc50 comp rx=0 tx=0).stop 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.972+0000 7f59e95dab80 1 -- 172.21.15.12:0/1007601286 shutdown_connections 2024-05-08T23:52:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.972+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1007601286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31d10 0x55fafbc3da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.972+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1007601286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 0x55fafbc360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.972+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1007601286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc317d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.972+0000 7f59e95dab80 1 -- 172.21.15.12:0/1007601286 >> 172.21.15.12:0/1007601286 conn(0x55fafbb8b1b0 msgr2=0x55fafbc27840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.972+0000 7f59e95dab80 1 -- 172.21.15.12:0/1007601286 shutdown_connections 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.972+0000 7f59e95dab80 1 -- 172.21.15.12:0/1007601286 wait complete. 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.972+0000 7f59e95dab80 1 -- start start 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59e95dab80 1 Processor -- start 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31350 0x55fafbc3ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 0x55fafbc3b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc35ce0 0x55fafbc3b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59e95dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbb19500 con 0x55fafbc35ce0 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59e95dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbb191a0 con 0x55fafbc31350 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59e95dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafba00640 con 0x55fafbc31d10 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59e95dab80 10 client.0 _ll_register_callbacks cb 0x55fafbb22ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc35ce0 0x55fafbc3b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc35ce0 0x55fafbc3b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42420/0 (socket says 172.21.15.12:42420) 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d77fe700 1 -- 172.21.15.12:0/1661476569 learned_addr learned my addr 172.21.15.12:0/1661476569 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31350 0x55fafbc3ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d67fc700 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 0x55fafbc3b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d77fe700 1 -- 172.21.15.12:0/1661476569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 msgr2=0x55fafbc3b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d77fe700 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 0x55fafbc3b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d77fe700 1 -- 172.21.15.12:0/1661476569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31350 msgr2=0x55fafbc3ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d77fe700 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31350 0x55fafbc3ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d77fe700 1 -- 172.21.15.12:0/1661476569 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d810e5f0 con 0x55fafbc35ce0 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.973+0000 7f59d67fc700 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 0x55fafbc3b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59d77fe700 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc35ce0 0x55fafbc3b810 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f59d0008e80 tx=0x7f59d0008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d78eb0bc53c29e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59baff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59baff5700 1 -- 172.21.15.12:0/1661476569 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d000e040 con 0x55fafbc35ce0 2024-05-08T23:52:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59baff5700 1 -- 172.21.15.12:0/1661476569 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f59d00074e0 con 0x55fafbc35ce0 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59e95dab80 1 -- 172.21.15.12:0/1661476569 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fafbad6210 con 0x55fafbc35ce0 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59baff5700 1 -- 172.21.15.12:0/1661476569 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59d00076a0 con 0x55fafbc35ce0 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59e95dab80 10 client.17049 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59e95dab80 1 -- 172.21.15.12:0/1661476569 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f59d81a44b0 con 0x55fafbc35ce0 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59baff5700 1 -- 172.21.15.12:0/1661476569 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fafbad6210 con 0x55fafbc35ce0 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59baff5700 1 -- 172.21.15.12:0/1661476569 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f59d000b100 con 0x55fafbc35ce0 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59e95dab80 10 client.17049 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59e95dab80 10 client.17049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.974+0000 7f59e95dab80 20 client.17049 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.975+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55fafbc4b000 0x55fafbc4d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.975+0000 7f59e95dab80 4 client.17049 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.975+0000 7f59e95dab80 1 -- 172.21.15.12:0/1661476569 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fafbbfc7b0 con 0x55fafbc4b000 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.975+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55fafbc4b000 0x55fafbc4d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.975+0000 7f59baff5700 10 client.17049 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.975+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55fafbc4b000 0x55fafbc4d420 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbee2e7c34a6fa0b server_cookie=17f663d2824dec94 in_seq=0 out_seq=0 2024-05-08T23:52:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59baff5700 1 -- 172.21.15.12:0/1661476569 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fafbbfc7b0 con 0x55fafbc4b000 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59baff5700 10 client.17049 handle_command_reply: tid=0 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 client.17049 shutdown 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 -- 172.21.15.12:0/1661476569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc35ce0 msgr2=0x55fafbc3b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc35ce0 0x55fafbc3b810 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f59d0008e80 tx=0x7f59d0008eb0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 -- 172.21.15.12:0/1661476569 shutdown_connections 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc35ce0 0x55fafbc3b810 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55fafbc4b000 0x55fafbc4d420 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31350 0x55fafbc3ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1661476569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 0x55fafbc3b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59baff5700 0 client.17049 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 -- 172.21.15.12:0/1661476569 >> 172.21.15.12:0/1661476569 conn(0x55fafbb8b1b0 msgr2=0x55fafbc2bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 -- 172.21.15.12:0/1661476569 shutdown_connections 2024-05-08T23:52:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 1 -- 172.21.15.12:0/1661476569 wait complete. 2024-05-08T23:52:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.976+0000 7f59e95dab80 20 client.17049 trim_cache size 0 max 16384 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.992+0000 7f59e95dab80 1 Processor -- start 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.992+0000 7f59e95dab80 1 -- start start 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31350 0x55fafbc4d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31d10 0x55fafbc31730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 0x55fafbc23e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59e95dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbc24380 con 0x55fafbc31d10 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59e95dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbc36bf0 con 0x55fafbc31350 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59e95dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbc36970 con 0x55fafbc35ce0 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31d10 0x55fafbc31730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31d10 0x55fafbc31730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42436/0 (socket says 172.21.15.12:42436) 2024-05-08T23:52:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d67fc700 1 -- 172.21.15.12:0/3565400208 learned_addr learned my addr 172.21.15.12:0/3565400208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/3565400208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31350 0x55fafbc4d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d77fe700 1 --2- 172.21.15.12:0/3565400208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 0x55fafbc23e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d67fc700 1 -- 172.21.15.12:0/3565400208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 msgr2=0x55fafbc23e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d67fc700 1 --2- 172.21.15.12:0/3565400208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 0x55fafbc23e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d67fc700 1 -- 172.21.15.12:0/3565400208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31350 msgr2=0x55fafbc4d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d67fc700 1 --2- 172.21.15.12:0/3565400208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31350 0x55fafbc4d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d67fc700 1 -- 172.21.15.12:0/3565400208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fafb9aab00 con 0x55fafbc31d10 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d77fe700 1 --2- 172.21.15.12:0/3565400208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 0x55fafbc23e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59d67fc700 1 --2- 172.21.15.12:0/3565400208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31d10 0x55fafbc31730 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00cc00 tx=0x7f59cc00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed607792fb168f1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59bdffb700 1 -- 172.21.15.12:0/3565400208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc09dd40 con 0x55fafbc31d10 2024-05-08T23:52:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59bdffb700 1 -- 172.21.15.12:0/3565400208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59cc005660 con 0x55fafbc31d10 2024-05-08T23:52:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.993+0000 7f59bdffb700 1 -- 172.21.15.12:0/3565400208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0b82e0 con 0x55fafbc31d10 2024-05-08T23:52:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.994+0000 7f59e95dab80 1 -- 172.21.15.12:0/3565400208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31d10 msgr2=0x55fafbc31730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.994+0000 7f59e95dab80 1 --2- 172.21.15.12:0/3565400208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31d10 0x55fafbc31730 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00cc00 tx=0x7f59cc00cf50 comp rx=0 tx=0).stop 2024-05-08T23:52:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.994+0000 7f59e95dab80 1 -- 172.21.15.12:0/3565400208 shutdown_connections 2024-05-08T23:52:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.994+0000 7f59e95dab80 1 --2- 172.21.15.12:0/3565400208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc35ce0 0x55fafbc23e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.994+0000 7f59e95dab80 1 --2- 172.21.15.12:0/3565400208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc31350 0x55fafbc4d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.994+0000 7f59e95dab80 1 --2- 172.21.15.12:0/3565400208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31d10 0x55fafbc31730 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.994+0000 7f59e95dab80 1 -- 172.21.15.12:0/3565400208 >> 172.21.15.12:0/3565400208 conn(0x55fafbb8b1b0 msgr2=0x55fafbc2c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.994+0000 7f59e95dab80 1 -- 172.21.15.12:0/3565400208 shutdown_connections 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.994+0000 7f59e95dab80 1 -- 172.21.15.12:0/3565400208 wait complete. 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.994+0000 7f59e95dab80 1 -- start start 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59e95dab80 1 Processor -- start 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc41f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 0x55fafbc3fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59e95dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc35ce0 0x55fafbc40290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59e95dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbc426b0 con 0x55fafbc31350 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59e95dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbc42820 con 0x55fafbc35ce0 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59e95dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafbc40850 con 0x55fafbc31d10 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59e95dab80 10 client.0 _ll_register_callbacks cb 0x55fafbb22ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59d6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc41f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59d6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc41f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42442/0 (socket says 172.21.15.12:42442) 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59d6ffd700 1 -- 172.21.15.12:0/1569251698 learned_addr learned my addr 172.21.15.12:0/1569251698 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59d77fe700 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc35ce0 0x55fafbc40290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59d67fc700 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 0x55fafbc3fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59d6ffd700 1 -- 172.21.15.12:0/1569251698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 msgr2=0x55fafbc3fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 0x55fafbc3fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59d6ffd700 1 -- 172.21.15.12:0/1569251698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc35ce0 msgr2=0x55fafbc40290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc35ce0 0x55fafbc40290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.995+0000 7f59d6ffd700 1 -- 172.21.15.12:0/1569251698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fafb9fe350 con 0x55fafbc31350 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59d6ffd700 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc41f60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55fafbc4dd30 tx=0x7f59c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe4a17b2ebc62a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59baff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59baff5700 1 -- 172.21.15.12:0/1569251698 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8022070 con 0x55fafbc31350 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59baff5700 1 -- 172.21.15.12:0/1569251698 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f59c802b0b0 con 0x55fafbc31350 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59e95dab80 1 -- 172.21.15.12:0/1569251698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59cc0a2390 con 0x55fafbc31350 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59baff5700 1 -- 172.21.15.12:0/1569251698 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59c802b270 con 0x55fafbc31350 2024-05-08T23:52:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59e95dab80 10 client.17055 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59e95dab80 1 -- 172.21.15.12:0/1569251698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55fafbb43500 con 0x55fafbc31350 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59baff5700 1 -- 172.21.15.12:0/1569251698 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59cc0a2390 con 0x55fafbc31350 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59baff5700 1 -- 172.21.15.12:0/1569251698 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f59c8029960 con 0x55fafbc31350 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59e95dab80 10 client.17055 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59e95dab80 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59e95dab80 20 client.17055 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55fafbca1780 0x55fafbca3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59e95dab80 4 client.17055 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.996+0000 7f59e95dab80 1 -- 172.21.15.12:0/1569251698 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fafbb75e60 con 0x55fafbca1780 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.997+0000 7f59d67fc700 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55fafbca1780 0x55fafbca3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.997+0000 7f59baff5700 10 client.17055 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:07.997+0000 7f59d67fc700 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55fafbca1780 0x55fafbca3ba0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2522b1b87cf97239 server_cookie=28dc8f56ad1b5fb7 in_seq=0 out_seq=0 2024-05-08T23:52:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.052+0000 7f59baff5700 1 -- 172.21.15.12:0/1569251698 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fafbb75e60 con 0x55fafbca1780 2024-05-08T23:52:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.052+0000 7f59baff5700 10 client.17055 handle_command_reply: tid=0 2024-05-08T23:52:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.052+0000 7f59e95dab80 1 client.17055 shutdown 2024-05-08T23:52:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 1 -- 172.21.15.12:0/1569251698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 msgr2=0x55fafbc41f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc41f60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55fafbc4dd30 tx=0x7f59c80047c0 comp rx=0 tx=0).stop 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 1 -- 172.21.15.12:0/1569251698 shutdown_connections 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55fafbca1780 0x55fafbca3ba0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fafbc35ce0 0x55fafbc40290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fafbc31350 0x55fafbc41f60 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 1 --2- 172.21.15.12:0/1569251698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fafbc31d10 0x55fafbc3fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 1 -- 172.21.15.12:0/1569251698 >> 172.21.15.12:0/1569251698 conn(0x55fafbb8b1b0 msgr2=0x55fafbc39620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59baff5700 0 client.17055 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 1 -- 172.21.15.12:0/1569251698 shutdown_connections 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 1 -- 172.21.15.12:0/1569251698 wait complete. 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.053+0000 7f59e95dab80 20 client.17055 trim_cache size 0 max 16384 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:52:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:52:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:52:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:52:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 -- 172.21.15.12:0/664753854 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59c00a1310 msgr2=0x7f59c00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59c00a1310 0x7f59c00a37d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009c60 tx=0x7f59c8009260 comp rx=0 tx=0).stop 2024-05-08T23:52:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 -- 172.21.15.12:0/664753854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59d8106230 msgr2=0x7f59d8197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59d8106230 0x7f59d8197980 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00e970 tx=0x7f59cc00ec80 comp rx=0 tx=0).stop 2024-05-08T23:52:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 -- 172.21.15.12:0/664753854 shutdown_connections 2024-05-08T23:52:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59c00a1310 0x7f59c00a37d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59d819bcd0 0x7f59d819c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59d80699e0 0x7f59d8197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 --2- 172.21.15.12:0/664753854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59d8106230 0x7f59d8197980 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 -- 172.21.15.12:0/664753854 >> 172.21.15.12:0/664753854 conn(0x7f59d8076f40 msgr2=0x7f59d80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 -- 172.21.15.12:0/664753854 shutdown_connections 2024-05-08T23:52:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:08.054+0000 7f59dd420700 1 -- 172.21.15.12:0/664753854 wait complete. 2024-05-08T23:52:08.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:08 smithi204 ceph-mon[47373]: pgmap v176: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 432 op/s 2024-05-08T23:52:08.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:08 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:52:08.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:08 smithi160 ceph-mon[46152]: pgmap v176: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 432 op/s 2024-05-08T23:52:08.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:08 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:52:08.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.84 MB/sec execute 124 sec latency 1529.914 ms 2024-05-08T23:52:08.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:08 smithi012 ceph-mon[42503]: pgmap v176: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 432 op/s 2024-05-08T23:52:08.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:08 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:52:09.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:09 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:52:09.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:09 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:52:09.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:09 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:52:09.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:09 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:52:09.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.75 MB/sec execute 125 sec latency 2529.959 ms 2024-05-08T23:52:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:09 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:52:09.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:09 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:52:10.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:10 smithi204 ceph-mon[47373]: pgmap v177: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-05-08T23:52:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:10 smithi160 ceph-mon[46152]: pgmap v177: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-05-08T23:52:10.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.66 MB/sec execute 126 sec latency 3530.047 ms 2024-05-08T23:52:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:10 smithi012 ceph-mon[42503]: pgmap v177: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-05-08T23:52:11.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.58 MB/sec execute 127 sec latency 4530.140 ms 2024-05-08T23:52:12.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:12 smithi204 ceph-mon[47373]: pgmap v178: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 366 op/s 2024-05-08T23:52:12.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:12 smithi160 ceph-mon[46152]: pgmap v178: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 366 op/s 2024-05-08T23:52:12.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.50 MB/sec execute 128 sec latency 5530.218 ms 2024-05-08T23:52:12.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:12 smithi012 ceph-mon[42503]: pgmap v178: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 366 op/s 2024-05-08T23:52:13.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.42 MB/sec execute 129 sec latency 6530.319 ms 2024-05-08T23:52:14.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:14 smithi204 ceph-mon[47373]: pgmap v179: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 2.5 MiB/s wr, 212 op/s 2024-05-08T23:52:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:14 smithi160 ceph-mon[46152]: pgmap v179: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 2.5 MiB/s wr, 212 op/s 2024-05-08T23:52:14.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.34 MB/sec execute 130 sec latency 7530.421 ms 2024-05-08T23:52:14.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:14 smithi012 ceph-mon[42503]: pgmap v179: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 2.5 MiB/s wr, 212 op/s 2024-05-08T23:52:15.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.26 MB/sec execute 131 sec latency 8530.515 ms 2024-05-08T23:52:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:16 smithi204 ceph-mon[47373]: pgmap v180: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 2.5 MiB/s wr, 212 op/s 2024-05-08T23:52:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:16 smithi160 ceph-mon[46152]: pgmap v180: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 2.5 MiB/s wr, 212 op/s 2024-05-08T23:52:16.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.18 MB/sec execute 132 sec latency 9530.645 ms 2024-05-08T23:52:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:16 smithi012 ceph-mon[42503]: pgmap v180: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 2.5 MiB/s wr, 212 op/s 2024-05-08T23:52:17.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.10 MB/sec execute 133 sec latency 10530.716 ms 2024-05-08T23:52:18.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:18 smithi204 ceph-mon[47373]: pgmap v181: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-05-08T23:52:18.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:18 smithi160 ceph-mon[46152]: pgmap v181: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-05-08T23:52:18.866 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 10.03 MB/sec execute 134 sec latency 11530.782 ms 2024-05-08T23:52:18.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:18 smithi012 ceph-mon[42503]: pgmap v181: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-05-08T23:52:19.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 9.95 MB/sec execute 135 sec latency 12530.854 ms 2024-05-08T23:52:20.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:20 smithi204 ceph-mon[47373]: pgmap v182: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.7 MiB/s wr, 120 op/s 2024-05-08T23:52:20.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:20 smithi160 ceph-mon[46152]: pgmap v182: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.7 MiB/s wr, 120 op/s 2024-05-08T23:52:20.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345599 9.88 MB/sec execute 136 sec latency 13530.949 ms 2024-05-08T23:52:20.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:20 smithi012 ceph-mon[42503]: pgmap v182: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.7 MiB/s wr, 120 op/s 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.188+0000 7f8296868700 1 -- 172.21.15.12:0/1631047369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8290068ca0 msgr2=0x7f829010dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.188+0000 7f8296868700 1 --2- 172.21.15.12:0/1631047369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8290068ca0 0x7f829010dca0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f828800b0a0 tx=0x7f828800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.188+0000 7f8296868700 1 -- 172.21.15.12:0/1631047369 shutdown_connections 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.188+0000 7f8296868700 1 --2- 172.21.15.12:0/1631047369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8290068ca0 0x7f829010dca0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.188+0000 7f8296868700 1 --2- 172.21.15.12:0/1631047369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290107000 0x7f82901073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.188+0000 7f8296868700 1 --2- 172.21.15.12:0/1631047369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82900682e0 0x7f8290068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.188+0000 7f8296868700 1 -- 172.21.15.12:0/1631047369 >> 172.21.15.12:0/1631047369 conn(0x7f8290076f10 msgr2=0x7f8290077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.188+0000 7f8296868700 1 -- 172.21.15.12:0/1631047369 shutdown_connections 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.188+0000 7f8296868700 1 -- 172.21.15.12:0/1631047369 wait complete. 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f8296868700 1 Processor -- start 2024-05-08T23:52:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f8296868700 1 -- start start 2024-05-08T23:52:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f8296868700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82900682e0 0x7f8290199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f8296868700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290068ca0 0x7f8290199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f8296868700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8290107000 0x7f829019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f8296868700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829019e560 con 0x7f82900682e0 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f8296868700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829019e6d0 con 0x7f8290107000 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f8296868700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829019e890 con 0x7f8290068ca0 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f828ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82900682e0 0x7f8290199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f828ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82900682e0 0x7f8290199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42456/0 (socket says 172.21.15.12:42456) 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f828ffff700 1 -- 172.21.15.12:0/933138385 learned_addr learned my addr 172.21.15.12:0/933138385 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.189+0000 7f8294e05700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8290107000 0x7f829019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f828f7fe700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290068ca0 0x7f8290199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f828ffff700 1 -- 172.21.15.12:0/933138385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290068ca0 msgr2=0x7f8290199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f828ffff700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290068ca0 0x7f8290199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f828ffff700 1 -- 172.21.15.12:0/933138385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8290107000 msgr2=0x7f829019df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f828ffff700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8290107000 0x7f829019df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f828ffff700 1 -- 172.21.15.12:0/933138385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288009d00 con 0x7f82900682e0 2024-05-08T23:52:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f828ffff700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82900682e0 0x7f8290199690 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f828000e9a0 tx=0x7f828000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f828d7fa700 1 -- 172.21.15.12:0/933138385 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280009790 con 0x7f82900682e0 2024-05-08T23:52:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f8296868700 1 -- 172.21.15.12:0/933138385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829019eb70 con 0x7f82900682e0 2024-05-08T23:52:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f828d7fa700 1 -- 172.21.15.12:0/933138385 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8280010430 con 0x7f82900682e0 2024-05-08T23:52:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f828d7fa700 1 -- 172.21.15.12:0/933138385 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280018680 con 0x7f82900682e0 2024-05-08T23:52:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.190+0000 7f8296868700 1 -- 172.21.15.12:0/933138385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82901a63b0 con 0x7f82900682e0 2024-05-08T23:52:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.193+0000 7f828d7fa700 1 -- 172.21.15.12:0/933138385 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f82800105a0 con 0x7f82900682e0 2024-05-08T23:52:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.194+0000 7f828d7fa700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f827c0a1290 0x7f827c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.194+0000 7f828d7fa700 1 -- 172.21.15.12:0/933138385 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8280014070 con 0x7f82900682e0 2024-05-08T23:52:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.194+0000 7f828f7fe700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f827c0a1290 0x7f827c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f828f7fe700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f827c0a1290 0x7f827c0a3750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f829019ab90 tx=0x7f8278009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f82a2a22b80 1 Processor -- start 2024-05-08T23:52:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f82a2a22b80 1 -- start start 2024-05-08T23:52:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca023950 0x562bca023d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca024270 0x562bca01df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca01e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc9f750f0 con 0x562bca023950 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc9f72ab0 con 0x562bca01e480 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc9f6e3a0 con 0x562bca024270 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f828ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca023950 0x562bca023d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f828ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca023950 0x562bca023d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42468/0 (socket says 172.21.15.12:42468) 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f828ffff700 1 -- 172.21.15.12:0/3998421589 learned_addr learned my addr 172.21.15.12:0/3998421589 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f8294e05700 1 --2- 172.21.15.12:0/3998421589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca01e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.195+0000 7f828f7fe700 1 --2- 172.21.15.12:0/3998421589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca024270 0x562bca01df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.196+0000 7f828ffff700 1 -- 172.21.15.12:0/3998421589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca024270 msgr2=0x562bca01df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.196+0000 7f828ffff700 1 --2- 172.21.15.12:0/3998421589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca024270 0x562bca01df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.196+0000 7f828ffff700 1 -- 172.21.15.12:0/3998421589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 msgr2=0x562bca01e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.196+0000 7f828ffff700 1 --2- 172.21.15.12:0/3998421589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca01e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.196+0000 7f828ffff700 1 -- 172.21.15.12:0/3998421589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc9def350 con 0x562bca023950 2024-05-08T23:52:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.196+0000 7f8294e05700 1 --2- 172.21.15.12:0/3998421589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca01e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.196+0000 7f828ffff700 1 --2- 172.21.15.12:0/3998421589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca023950 0x562bca023d30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f82800a1f10 tx=0x7f828009a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733b05a582f510c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.196+0000 7f82867fc700 1 -- 172.21.15.12:0/3998421589 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82800a2440 con 0x562bca023950 2024-05-08T23:52:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.196+0000 7f82867fc700 1 -- 172.21.15.12:0/3998421589 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8280010880 con 0x562bca023950 2024-05-08T23:52:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.196+0000 7f82867fc700 1 -- 172.21.15.12:0/3998421589 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280099760 con 0x562bca023950 2024-05-08T23:52:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.197+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3998421589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca023950 msgr2=0x562bca023d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.197+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/3998421589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca023950 0x562bca023d30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f82800a1f10 tx=0x7f828009a6c0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.197+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3998421589 shutdown_connections 2024-05-08T23:52:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.197+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/3998421589 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca01e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.197+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/3998421589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca023950 0x562bca023d30 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.197+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/3998421589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca024270 0x562bca01df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.197+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3998421589 >> 172.21.15.12:0/3998421589 conn(0x562bc9f7c1b0 msgr2=0x562bca018840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.197+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3998421589 shutdown_connections 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.197+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3998421589 wait complete. 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.197+0000 7f82a2a22b80 1 -- start start 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f82a2a22b80 1 Processor -- start 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca0275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca023950 0x562bca027b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca028070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc9f0a500 con 0x562bca024270 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc9f0a1a0 con 0x562bca01e480 2024-05-08T23:52:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc9df1640 con 0x562bca023950 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f82a2a22b80 10 client.0 _ll_register_callbacks cb 0x562bc9f13ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f828ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca0275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f8294e05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca028070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f828ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca0275f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:57320/0 (socket says 172.21.15.12:57320) 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f8294e05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca028070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42470/0 (socket says 172.21.15.12:42470) 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f828ffff700 1 -- 172.21.15.12:0/3685647117 learned_addr learned my addr 172.21.15.12:0/3685647117 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f828f7fe700 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca023950 0x562bca027b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f828ffff700 1 -- 172.21.15.12:0/3685647117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca023950 msgr2=0x562bca027b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f828ffff700 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca023950 0x562bca027b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f828ffff700 1 -- 172.21.15.12:0/3685647117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 msgr2=0x562bca028070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f828ffff700 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca028070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.198+0000 7f828ffff700 1 -- 172.21.15.12:0/3685647117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829010e630 con 0x562bca01e480 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f828ffff700 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca0275f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8280099f00 tx=0x7f8280099760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e418902e80e5350c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f826f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f826f7fe700 1 -- 172.21.15.12:0/3685647117 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82800a2440 con 0x562bca01e480 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3685647117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bc9ec7210 con 0x562bca01e480 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f826f7fe700 1 -- 172.21.15.12:0/3685647117 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8280094de0 con 0x562bca01e480 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f826f7fe700 1 -- 172.21.15.12:0/3685647117 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82800a1fb0 con 0x562bca01e480 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f82a2a22b80 10 client.26855 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3685647117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f82901a63b0 con 0x562bca01e480 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f826f7fe700 1 -- 172.21.15.12:0/3685647117 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562bc9ec7210 con 0x562bca01e480 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f826f7fe700 1 -- 172.21.15.12:0/3685647117 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8280098f20 con 0x562bca01e480 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f82a2a22b80 10 client.26855 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f82a2a22b80 10 client.26855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f82a2a22b80 20 client.26855 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562bca037970 0x562bca039d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f82a2a22b80 4 client.26855 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.199+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3685647117 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bc9fed7b0 con 0x562bca037970 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.200+0000 7f828f7fe700 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562bca037970 0x562bca039d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.200+0000 7f828f7fe700 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562bca037970 0x562bca039d90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87641ead5ee1fed server_cookie=39b0d92e2c42516b in_seq=0 out_seq=0 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.200+0000 7f826f7fe700 10 client.26855 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f826f7fe700 1 -- 172.21.15.12:0/3685647117 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562bc9fed7b0 con 0x562bca037970 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f826f7fe700 10 client.26855 handle_command_reply: tid=0 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f82a2a22b80 1 client.26855 shutdown 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3685647117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 msgr2=0x562bca0275f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca0275f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8280099f00 tx=0x7f8280099760 comp rx=0 tx=0).stop 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3685647117 shutdown_connections 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca028070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562bca037970 0x562bca039d90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca01e480 0x562bca0275f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/3685647117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca023950 0x562bca027b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f826f7fe700 0 client.26855 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3685647117 >> 172.21.15.12:0/3685647117 conn(0x562bc9f7c1b0 msgr2=0x562bca01cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.201+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3685647117 shutdown_connections 2024-05-08T23:52:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.202+0000 7f82a2a22b80 1 -- 172.21.15.12:0/3685647117 wait complete. 2024-05-08T23:52:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.202+0000 7f82a2a22b80 20 client.26855 trim_cache size 0 max 16384 2024-05-08T23:52:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.218+0000 7f82a2a22b80 1 Processor -- start 2024-05-08T23:52:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.218+0000 7f82a2a22b80 1 -- start start 2024-05-08T23:52:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.218+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 0x562bca01a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.218+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 0x562bca018910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.218+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca033690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.218+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bca019350 con 0x562bca024270 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.218+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bca03ef60 con 0x562bca023950 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.218+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bca023d30 con 0x562bca01e480 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8294e05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca033690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f828ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 0x562bca01a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8294e05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca033690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42484/0 (socket says 172.21.15.12:42484) 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f828f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 0x562bca018910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8294e05700 1 -- 172.21.15.12:0/677657880 learned_addr learned my addr 172.21.15.12:0/677657880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f828ffff700 1 --2- 172.21.15.12:0/677657880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 0x562bca01a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53080/0 (socket says 172.21.15.12:53080) 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8294e05700 1 -- 172.21.15.12:0/677657880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 msgr2=0x562bca01a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8294e05700 1 --2- 172.21.15.12:0/677657880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 0x562bca01a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8294e05700 1 -- 172.21.15.12:0/677657880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 msgr2=0x562bca018910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8294e05700 1 --2- 172.21.15.12:0/677657880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 0x562bca018910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8294e05700 1 -- 172.21.15.12:0/677657880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc9d9bb00 con 0x562bca024270 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f828f7fe700 1 --2- 172.21.15.12:0/677657880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 0x562bca018910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8294e05700 1 --2- 172.21.15.12:0/677657880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca033690 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8288015000 tx=0x7f8288007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f35db7a150c55e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8286ffd700 1 -- 172.21.15.12:0/677657880 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828800e040 con 0x562bca024270 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8286ffd700 1 -- 172.21.15.12:0/677657880 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8288003a00 con 0x562bca024270 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.219+0000 7f8286ffd700 1 -- 172.21.15.12:0/677657880 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288005040 con 0x562bca024270 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.220+0000 7f82a2a22b80 1 -- 172.21.15.12:0/677657880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 msgr2=0x562bca033690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.220+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/677657880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca033690 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8288015000 tx=0x7f8288007be0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.220+0000 7f82a2a22b80 1 -- 172.21.15.12:0/677657880 shutdown_connections 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.220+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/677657880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca033690 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.220+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/677657880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 0x562bca01a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.220+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/677657880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 0x562bca018910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.220+0000 7f82a2a22b80 1 -- 172.21.15.12:0/677657880 >> 172.21.15.12:0/677657880 conn(0x562bc9f7c1b0 msgr2=0x562bca014e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.220+0000 7f82a2a22b80 1 -- 172.21.15.12:0/677657880 shutdown_connections 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.220+0000 7f82a2a22b80 1 -- 172.21.15.12:0/677657880 wait complete. 2024-05-08T23:52:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.220+0000 7f82a2a22b80 1 -- start start 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f82a2a22b80 1 Processor -- start 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 0x562bca02efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 0x562bca02cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f82a2a22b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca02d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bca02d870 con 0x562bca024270 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bca02d9e0 con 0x562bca023950 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f82a2a22b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bca08ce30 con 0x562bca01e480 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f82a2a22b80 10 client.0 _ll_register_callbacks cb 0x562bc9f13ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f8294e05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca02d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f828ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 0x562bca02efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f8294e05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca02d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42488/0 (socket says 172.21.15.12:42488) 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f828f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 0x562bca02cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.221+0000 7f8294e05700 1 -- 172.21.15.12:0/2050750826 learned_addr learned my addr 172.21.15.12:0/2050750826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f8294e05700 1 -- 172.21.15.12:0/2050750826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 msgr2=0x562bca02efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f8294e05700 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 0x562bca02efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f8294e05700 1 -- 172.21.15.12:0/2050750826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 msgr2=0x562bca02cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f8294e05700 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 0x562bca02cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f8294e05700 1 -- 172.21.15.12:0/2050750826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8278017070 con 0x562bca024270 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f828f7fe700 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 0x562bca02cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f8294e05700 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca02d160 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f82880061a0 tx=0x7f8288007970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27012e1ce0f0950b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f826f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f826f7fe700 1 -- 172.21.15.12:0/2050750826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828800e040 con 0x562bca024270 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f826f7fe700 1 -- 172.21.15.12:0/2050750826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f828800bab0 con 0x562bca024270 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f82a2a22b80 1 -- 172.21.15.12:0/2050750826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8280010880 con 0x562bca024270 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f826f7fe700 1 -- 172.21.15.12:0/2050750826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f828800bc70 con 0x562bca024270 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f82a2a22b80 10 client.17073 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.222+0000 7f82a2a22b80 1 -- 172.21.15.12:0/2050750826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562bc9f34500 con 0x562bca024270 2024-05-08T23:52:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f826f7fe700 1 -- 172.21.15.12:0/2050750826 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8280010880 con 0x562bca024270 2024-05-08T23:52:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f826f7fe700 1 -- 172.21.15.12:0/2050750826 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8288019070 con 0x562bca024270 2024-05-08T23:52:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f82a2a22b80 10 client.17073 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f82a2a22b80 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f82a2a22b80 20 client.17073 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562bca08e290 0x562bca090680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f82a2a22b80 4 client.17073 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:52:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f82a2a22b80 1 -- 172.21.15.12:0/2050750826 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562bc9f66e60 con 0x562bca08e290 2024-05-08T23:52:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f828ffff700 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562bca08e290 0x562bca090680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f828ffff700 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562bca08e290 0x562bca090680 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7475a764deacd1e6 server_cookie=f55feede04fa7400 in_seq=0 out_seq=0 2024-05-08T23:52:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.223+0000 7f826f7fe700 10 client.17073 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.249+0000 7f826f7fe700 1 -- 172.21.15.12:0/2050750826 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562bc9f66e60 con 0x562bca08e290 2024-05-08T23:52:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.249+0000 7f826f7fe700 10 client.17073 handle_command_reply: tid=0 2024-05-08T23:52:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.249+0000 7f82a2a22b80 1 client.17073 shutdown 2024-05-08T23:52:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.249+0000 7f82a2a22b80 1 -- 172.21.15.12:0/2050750826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 msgr2=0x562bca02d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.249+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca02d160 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f82880061a0 tx=0x7f8288007970 comp rx=0 tx=0).stop 2024-05-08T23:52:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f82a2a22b80 1 -- 172.21.15.12:0/2050750826 shutdown_connections 2024-05-08T23:52:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562bca024270 0x562bca02d160 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bca01e480 0x562bca02efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562bca08e290 0x562bca090680 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f82a2a22b80 1 --2- 172.21.15.12:0/2050750826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562bca023950 0x562bca02cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f826f7fe700 0 client.17073 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f82a2a22b80 1 -- 172.21.15.12:0/2050750826 >> 172.21.15.12:0/2050750826 conn(0x562bc9f7c1b0 msgr2=0x562bca01a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f82a2a22b80 1 -- 172.21.15.12:0/2050750826 shutdown_connections 2024-05-08T23:52:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f82a2a22b80 1 -- 172.21.15.12:0/2050750826 wait complete. 2024-05-08T23:52:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f82a2a22b80 20 client.17073 trim_cache size 0 max 16384 2024-05-08T23:52:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:52:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:52:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:52:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:52:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f8296868700 1 -- 172.21.15.12:0/933138385 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f827c0a1290 msgr2=0x7f827c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f8296868700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f827c0a1290 0x7f827c0a3750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f829019ab90 tx=0x7f8278009260 comp rx=0 tx=0).stop 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f8296868700 1 -- 172.21.15.12:0/933138385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82900682e0 msgr2=0x7f8290199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.250+0000 7f8296868700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82900682e0 0x7f8290199690 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f828000e9a0 tx=0x7f828000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.251+0000 7f8296868700 1 -- 172.21.15.12:0/933138385 shutdown_connections 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.251+0000 7f8296868700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f827c0a1290 0x7f827c0a3750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.251+0000 7f8296868700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8290107000 0x7f829019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.251+0000 7f8296868700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82900682e0 0x7f8290199690 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.251+0000 7f8296868700 1 --2- 172.21.15.12:0/933138385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290068ca0 0x7f8290199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.251+0000 7f8296868700 1 -- 172.21.15.12:0/933138385 >> 172.21.15.12:0/933138385 conn(0x7f8290076f10 msgr2=0x7f82900fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.251+0000 7f8296868700 1 -- 172.21.15.12:0/933138385 shutdown_connections 2024-05-08T23:52:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:21.251+0000 7f8296868700 1 -- 172.21.15.12:0/933138385 wait complete. 2024-05-08T23:52:21.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 347682 9.90 MB/sec execute 137 sec latency 13724.200 ms 2024-05-08T23:52:22.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:22 smithi204 ceph-mon[47373]: pgmap v183: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 2.5 MiB/s wr, 164 op/s 2024-05-08T23:52:22.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:22 smithi160 ceph-mon[46152]: pgmap v183: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 2.5 MiB/s wr, 164 op/s 2024-05-08T23:52:22.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 349045 9.87 MB/sec execute 138 sec latency 42.609 ms 2024-05-08T23:52:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:22 smithi012 ceph-mon[42503]: pgmap v183: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 2.5 MiB/s wr, 164 op/s 2024-05-08T23:52:23.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 351418 9.90 MB/sec execute 139 sec latency 13.635 ms 2024-05-08T23:52:23.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:23 smithi012 ceph-mon[42503]: pgmap v184: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.6 MiB/s wr, 74 op/s 2024-05-08T23:52:24.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:23 smithi204 ceph-mon[47373]: pgmap v184: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.6 MiB/s wr, 74 op/s 2024-05-08T23:52:24.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:23 smithi160 ceph-mon[46152]: pgmap v184: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.6 MiB/s wr, 74 op/s 2024-05-08T23:52:24.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 352749 9.87 MB/sec execute 140 sec latency 21.232 ms 2024-05-08T23:52:25.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 354929 9.90 MB/sec execute 141 sec latency 12.967 ms 2024-05-08T23:52:26.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:26 smithi204 ceph-mon[47373]: pgmap v185: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.6 MiB/s wr, 74 op/s 2024-05-08T23:52:26.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:26 smithi160 ceph-mon[46152]: pgmap v185: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.6 MiB/s wr, 74 op/s 2024-05-08T23:52:26.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 356476 9.87 MB/sec execute 142 sec latency 14.154 ms 2024-05-08T23:52:26.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:26 smithi012 ceph-mon[42503]: pgmap v185: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.6 MiB/s wr, 74 op/s 2024-05-08T23:52:27.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 358567 9.89 MB/sec execute 143 sec latency 67.327 ms 2024-05-08T23:52:28.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:28 smithi204 ceph-mon[47373]: pgmap v186: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-05-08T23:52:28.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:28 smithi160 ceph-mon[46152]: pgmap v186: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-05-08T23:52:28.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 360585 9.87 MB/sec execute 144 sec latency 17.531 ms 2024-05-08T23:52:28.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:28 smithi012 ceph-mon[42503]: pgmap v186: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-05-08T23:52:29.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 362277 9.89 MB/sec execute 145 sec latency 13.667 ms 2024-05-08T23:52:30.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:30 smithi204 ceph-mon[47373]: pgmap v187: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-05-08T23:52:30.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:30 smithi160 ceph-mon[46152]: pgmap v187: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-05-08T23:52:30.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 363991 9.86 MB/sec execute 146 sec latency 18.991 ms 2024-05-08T23:52:30.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:30 smithi012 ceph-mon[42503]: pgmap v187: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-05-08T23:52:31.063 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:52:31.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.223+0000 7f3433871700 1 -- 172.21.15.12:0/2592632280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c100a10 msgr2=0x7f342c104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.223+0000 7f3433871700 1 --2- 172.21.15.12:0/2592632280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c100a10 0x7f342c104a30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f341c0098f0 tx=0x7f341c009c00 comp rx=0 tx=0).stop 2024-05-08T23:52:31.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.223+0000 7f3433871700 1 -- 172.21.15.12:0/2592632280 shutdown_connections 2024-05-08T23:52:31.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.223+0000 7f3433871700 1 --2- 172.21.15.12:0/2592632280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f342c105160 0x7f342c10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.223+0000 7f3433871700 1 --2- 172.21.15.12:0/2592632280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342c100060 0x7f342c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.223+0000 7f3433871700 1 --2- 172.21.15.12:0/2592632280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c100a10 0x7f342c104a30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.223+0000 7f3433871700 1 -- 172.21.15.12:0/2592632280 >> 172.21.15.12:0/2592632280 conn(0x7f342c076f40 msgr2=0x7f342c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.223+0000 7f3433871700 1 -- 172.21.15.12:0/2592632280 shutdown_connections 2024-05-08T23:52:31.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.223+0000 7f3433871700 1 -- 172.21.15.12:0/2592632280 wait complete. 2024-05-08T23:52:31.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.223+0000 7f3433871700 1 Processor -- start 2024-05-08T23:52:31.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f3433871700 1 -- start start 2024-05-08T23:52:31.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f3433871700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f342c100060 0x7f342c10d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f3433871700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342c105160 0x7f342c10db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f3433871700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c10e0a0 0x7f342c1a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f3433871700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1a5bb0 con 0x7f342c100060 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f3433871700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1a5d20 con 0x7f342c10e0a0 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f3433871700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1a5ee0 con 0x7f342c105160 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f343160d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f342c100060 0x7f342c10d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f343160d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f342c100060 0x7f342c10d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35548/0 (socket says 172.21.15.12:35548) 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f3431e0e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c10e0a0 0x7f342c1a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f343160d700 1 -- 172.21.15.12:0/2663357140 learned_addr learned my addr 172.21.15.12:0/2663357140 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f3430e0c700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342c105160 0x7f342c10db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f343160d700 1 -- 172.21.15.12:0/2663357140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342c105160 msgr2=0x7f342c10db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f343160d700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342c105160 0x7f342c10db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f343160d700 1 -- 172.21.15.12:0/2663357140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c10e0a0 msgr2=0x7f342c1a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f343160d700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c10e0a0 0x7f342c1a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f343160d700 1 -- 172.21.15.12:0/2663357140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424009c40 con 0x7f342c100060 2024-05-08T23:52:31.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.224+0000 7f343160d700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f342c100060 0x7f342c10d620 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f342800e9c0 tx=0x7f342800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.225+0000 7f34227fc700 1 -- 172.21.15.12:0/2663357140 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428004a30 con 0x7f342c100060 2024-05-08T23:52:31.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.225+0000 7f34227fc700 1 -- 172.21.15.12:0/2663357140 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f342800cc80 con 0x7f342c100060 2024-05-08T23:52:31.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.225+0000 7f3433871700 1 -- 172.21.15.12:0/2663357140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c009520 con 0x7f342c100060 2024-05-08T23:52:31.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.225+0000 7f34227fc700 1 -- 172.21.15.12:0/2663357140 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34280054b0 con 0x7f342c100060 2024-05-08T23:52:31.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.225+0000 7f3433871700 1 -- 172.21.15.12:0/2663357140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c1a6560 con 0x7f342c100060 2024-05-08T23:52:31.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.227+0000 7f34227fc700 1 -- 172.21.15.12:0/2663357140 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3428005610 con 0x7f342c100060 2024-05-08T23:52:31.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.228+0000 7f34227fc700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34180a1310 0x7f34180a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.228+0000 7f3430e0c700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34180a1310 0x7f34180a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.228+0000 7f34227fc700 1 -- 172.21.15.12:0/2663357140 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3428014070 con 0x7f342c100060 2024-05-08T23:52:31.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.229+0000 7f3430e0c700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34180a1310 0x7f34180a37d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f341c009810 tx=0x7f341c019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.229+0000 7f343fa2bb80 1 Processor -- start 2024-05-08T23:52:31.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.229+0000 7f343fa2bb80 1 -- start start 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.229+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc97e00 0x55bdfcc981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.229+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 0x55bdfcc92350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc92890 0x55bdfcc92cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcbe9150 con 0x55bdfcc97e00 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcbe6b10 con 0x55bdfcc98720 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcbe2400 con 0x55bdfcc92890 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f343160d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc97e00 0x55bdfcc981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f343160d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc97e00 0x55bdfcc981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35552/0 (socket says 172.21.15.12:35552) 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f343160d700 1 -- 172.21.15.12:0/1946580518 learned_addr learned my addr 172.21.15.12:0/1946580518 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f3430e0c700 1 --2- 172.21.15.12:0/1946580518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 0x55bdfcc92350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f3431e0e700 1 --2- 172.21.15.12:0/1946580518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc92890 0x55bdfcc92cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f3430e0c700 1 -- 172.21.15.12:0/1946580518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc92890 msgr2=0x55bdfcc92cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f3430e0c700 1 --2- 172.21.15.12:0/1946580518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc92890 0x55bdfcc92cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f3430e0c700 1 -- 172.21.15.12:0/1946580518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc97e00 msgr2=0x55bdfcc981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f3430e0c700 1 --2- 172.21.15.12:0/1946580518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc97e00 0x55bdfcc981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f3430e0c700 1 -- 172.21.15.12:0/1946580518 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdfca633b0 con 0x55bdfcc98720 2024-05-08T23:52:31.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f343160d700 1 --2- 172.21.15.12:0/1946580518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc97e00 0x55bdfcc981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:31.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f3430e0c700 1 --2- 172.21.15.12:0/1946580518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 0x55bdfcc92350 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f341c000c00 tx=0x7f341c01e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8922f7a2896196 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.230+0000 7f34177fe700 1 -- 172.21.15.12:0/1946580518 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c00f760 con 0x55bdfcc98720 2024-05-08T23:52:31.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.231+0000 7f34177fe700 1 -- 172.21.15.12:0/1946580518 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f341c01d040 con 0x55bdfcc98720 2024-05-08T23:52:31.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.231+0000 7f34177fe700 1 -- 172.21.15.12:0/1946580518 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c005570 con 0x55bdfcc98720 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.231+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1946580518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 msgr2=0x55bdfcc92350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.231+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/1946580518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 0x55bdfcc92350 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f341c000c00 tx=0x7f341c01e000 comp rx=0 tx=0).stop 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.231+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1946580518 shutdown_connections 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.231+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/1946580518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc92890 0x55bdfcc92cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.231+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/1946580518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc97e00 0x55bdfcc981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.231+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/1946580518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 0x55bdfcc92350 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.231+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1946580518 >> 172.21.15.12:0/1946580518 conn(0x55bdfcbf0210 msgr2=0x55bdfcc8cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.232+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1946580518 shutdown_connections 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.232+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1946580518 wait complete. 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.232+0000 7f343fa2bb80 1 -- start start 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.232+0000 7f343fa2bb80 1 Processor -- start 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.232+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc92890 0x55bdfcc9bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.232+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc9c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.232+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 0x55bdfcc9c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcb7e560 con 0x55bdfcc92890 2024-05-08T23:52:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcb7e200 con 0x55bdfcc98720 2024-05-08T23:52:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfca656a0 con 0x55bdfcc97e00 2024-05-08T23:52:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343fa2bb80 10 client.0 _ll_register_callbacks cb 0x55bdfcb87b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343160d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc92890 0x55bdfcc9bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f3430e0c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc9c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343160d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc92890 0x55bdfcc9bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35566/0 (socket says 172.21.15.12:35566) 2024-05-08T23:52:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343160d700 1 -- 172.21.15.12:0/2609511991 learned_addr learned my addr 172.21.15.12:0/2609511991 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f3431e0e700 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 0x55bdfcc9c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343160d700 1 -- 172.21.15.12:0/2609511991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 msgr2=0x55bdfcc9c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343160d700 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc9c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343160d700 1 -- 172.21.15.12:0/2609511991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 msgr2=0x55bdfcc9c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343160d700 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 0x55bdfcc9c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343160d700 1 -- 172.21.15.12:0/2609511991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c1a6560 con 0x55bdfcc92890 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f3430e0c700 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f3431e0e700 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 0x55bdfcc9c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343160d700 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc92890 0x55bdfcc9bb60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f342800cfd0 tx=0x7f342800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcd7adb03476ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f3414ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f3414ff9700 1 -- 172.21.15.12:0/2609511991 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428016530 con 0x55bdfcc92890 2024-05-08T23:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f3414ff9700 1 -- 172.21.15.12:0/2609511991 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3428099e80 con 0x55bdfcc92890 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.233+0000 7f343fa2bb80 1 -- 172.21.15.12:0/2609511991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdfcb3b270 con 0x55bdfcc92890 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f3414ff9700 1 -- 172.21.15.12:0/2609511991 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3428004da0 con 0x55bdfcc92890 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f343fa2bb80 10 client.17085 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f343fa2bb80 1 -- 172.21.15.12:0/2609511991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f342c067ff0 con 0x55bdfcc92890 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f3414ff9700 1 -- 172.21.15.12:0/2609511991 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bdfcb3b270 con 0x55bdfcc92890 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f3414ff9700 1 -- 172.21.15.12:0/2609511991 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3428099370 con 0x55bdfcc92890 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f343fa2bb80 10 client.17085 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f343fa2bb80 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f343fa2bb80 20 client.17085 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bdfccac1d0 0x55bdfccae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f343fa2bb80 4 client.17085 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f343fa2bb80 1 -- 172.21.15.12:0/2609511991 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdfcc61810 con 0x55bdfccac1d0 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.234+0000 7f3430e0c700 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bdfccac1d0 0x55bdfccae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.235+0000 7f3414ff9700 10 client.17085 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.235+0000 7f3430e0c700 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bdfccac1d0 0x55bdfccae5f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c56847dd6d4fab server_cookie=dae683c33c2f0a in_seq=0 out_seq=0 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.235+0000 7f3414ff9700 1 -- 172.21.15.12:0/2609511991 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bdfcc61810 con 0x55bdfccac1d0 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.235+0000 7f3414ff9700 10 client.17085 handle_command_reply: tid=0 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.235+0000 7f343fa2bb80 1 client.17085 shutdown 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 1 -- 172.21.15.12:0/2609511991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc92890 msgr2=0x55bdfcc9bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc92890 0x55bdfcc9bb60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f342800cfd0 tx=0x7f342800ce70 comp rx=0 tx=0).stop 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 1 -- 172.21.15.12:0/2609511991 shutdown_connections 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bdfccac1d0 0x55bdfccae5f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc98720 0x55bdfcc9c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc92890 0x55bdfcc9bb60 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/2609511991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f3414ff9700 0 client.17085 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 1 -- 172.21.15.12:0/2609511991 >> 172.21.15.12:0/2609511991 conn(0x55bdfcbf0210 msgr2=0x55bdfcc8c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 1 -- 172.21.15.12:0/2609511991 shutdown_connections 2024-05-08T23:52:31.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 1 -- 172.21.15.12:0/2609511991 wait complete. 2024-05-08T23:52:31.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.236+0000 7f343fa2bb80 20 client.17085 trim_cache size 0 max 16384 2024-05-08T23:52:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.255+0000 7f343fa2bb80 1 Processor -- start 2024-05-08T23:52:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.255+0000 7f343fa2bb80 1 -- start start 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.255+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcc92350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.255+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc8cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.255+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 0x55bdfcc9c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.255+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcca9990 con 0x55bdfcc98720 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.255+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcc9d180 con 0x55bdfcc92890 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.255+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcc8ea30 con 0x55bdfcc97e00 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f343160d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcc92350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f3430e0c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc8cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f3431e0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 0x55bdfcc9c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f343160d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcc92350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49750/0 (socket says 172.21.15.12:49750) 2024-05-08T23:52:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f343160d700 1 -- 172.21.15.12:0/3621763535 learned_addr learned my addr 172.21.15.12:0/3621763535 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f343160d700 1 -- 172.21.15.12:0/3621763535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 msgr2=0x55bdfcc8cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f343160d700 1 --2- 172.21.15.12:0/3621763535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc8cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f343160d700 1 -- 172.21.15.12:0/3621763535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 msgr2=0x55bdfcc9c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f343160d700 1 --2- 172.21.15.12:0/3621763535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 0x55bdfcc9c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f343160d700 1 -- 172.21.15.12:0/3621763535 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdfca0fb60 con 0x55bdfcc92890 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f3430e0c700 1 --2- 172.21.15.12:0/3621763535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc8cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f3431e0e700 1 --2- 172.21.15.12:0/3621763535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 0x55bdfcc9c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f343160d700 1 --2- 172.21.15.12:0/3621763535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcc92350 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3428099b80 tx=0x7f3428099d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d50833a8c6e3450 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f3417fff700 1 -- 172.21.15.12:0/3621763535 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428004da0 con 0x55bdfcc92890 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f3417fff700 1 -- 172.21.15.12:0/3621763535 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f342809ae90 con 0x55bdfcc92890 2024-05-08T23:52:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.256+0000 7f3417fff700 1 -- 172.21.15.12:0/3621763535 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34280a2300 con 0x55bdfcc92890 2024-05-08T23:52:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.257+0000 7f343fa2bb80 1 -- 172.21.15.12:0/3621763535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 msgr2=0x55bdfcc92350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.257+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/3621763535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcc92350 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3428099b80 tx=0x7f3428099d80 comp rx=0 tx=0).stop 2024-05-08T23:52:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.257+0000 7f343fa2bb80 1 -- 172.21.15.12:0/3621763535 shutdown_connections 2024-05-08T23:52:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.257+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/3621763535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 0x55bdfcc9c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.257+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/3621763535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcc92350 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.257+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/3621763535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc8cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.257+0000 7f343fa2bb80 1 -- 172.21.15.12:0/3621763535 >> 172.21.15.12:0/3621763535 conn(0x55bdfcbf0210 msgr2=0x55bdfcc9a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.257+0000 7f343fa2bb80 1 -- 172.21.15.12:0/3621763535 shutdown_connections 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.257+0000 7f343fa2bb80 1 -- 172.21.15.12:0/3621763535 wait complete. 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.257+0000 7f343fa2bb80 1 -- start start 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343fa2bb80 1 Processor -- start 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcca35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc94dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343fa2bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 0x55bdfcc939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcc956d0 con 0x55bdfcc98720 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcc93ee0 con 0x55bdfcc92890 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343fa2bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfcc940a0 con 0x55bdfcc97e00 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343fa2bb80 10 client.0 _ll_register_callbacks cb 0x55bdfcb87b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343160d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcca35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f3431e0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 0x55bdfcc939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343160d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcca35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49752/0 (socket says 172.21.15.12:49752) 2024-05-08T23:52:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f343160d700 1 -- 172.21.15.12:0/1097539242 learned_addr learned my addr 172.21.15.12:0/1097539242 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.258+0000 7f3430e0c700 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc94dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f343160d700 1 -- 172.21.15.12:0/1097539242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 msgr2=0x55bdfcc94dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f343160d700 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc94dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f343160d700 1 -- 172.21.15.12:0/1097539242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 msgr2=0x55bdfcc939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f343160d700 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 0x55bdfcc939a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f343160d700 1 -- 172.21.15.12:0/1097539242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdfca633b0 con 0x55bdfcc92890 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f343160d700 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcca35e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f342801b040 tx=0x7f3428004f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f900ccdde314a3b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f3414ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f3414ff9700 1 -- 172.21.15.12:0/1097539242 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428099570 con 0x55bdfcc92890 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f3414ff9700 1 -- 172.21.15.12:0/1097539242 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f34280a20d0 con 0x55bdfcc92890 2024-05-08T23:52:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1097539242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f341c01d040 con 0x55bdfcc92890 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f3414ff9700 1 -- 172.21.15.12:0/1097539242 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3428004da0 con 0x55bdfcc92890 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f343fa2bb80 10 client.26879 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1097539242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f342400e890 con 0x55bdfcc92890 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.259+0000 7f3414ff9700 1 -- 172.21.15.12:0/1097539242 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f341c01d040 con 0x55bdfcc92890 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f3414ff9700 1 -- 172.21.15.12:0/1097539242 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3428098e70 con 0x55bdfcc92890 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f343fa2bb80 10 client.26879 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f343fa2bb80 10 client.26879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f343fa2bb80 20 client.26879 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bdfcd02ec0 0x55bdfcd052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f343fa2bb80 4 client.26879 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1097539242 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bdfcbdaec0 con 0x55bdfcd02ec0 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f3430e0c700 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bdfcd02ec0 0x55bdfcd052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f3414ff9700 10 client.26879 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f3430e0c700 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bdfcd02ec0 0x55bdfcd052b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84a7edd84776277 server_cookie=556f7ce13f22dbeb in_seq=0 out_seq=0 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f3414ff9700 1 -- 172.21.15.12:0/1097539242 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bdfcbdaec0 con 0x55bdfcd02ec0 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f3414ff9700 10 client.26879 handle_command_reply: tid=0 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.260+0000 7f343fa2bb80 1 client.26879 shutdown 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1097539242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 msgr2=0x55bdfcca35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcca35e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f342801b040 tx=0x7f3428004f30 comp rx=0 tx=0).stop 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1097539242 shutdown_connections 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdfcc98720 0x55bdfcc939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bdfcd02ec0 0x55bdfcd052b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bdfcc92890 0x55bdfcca35e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 1 --2- 172.21.15.12:0/1097539242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdfcc97e00 0x55bdfcc94dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f3414ff9700 0 client.26879 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1097539242 >> 172.21.15.12:0/1097539242 conn(0x55bdfcbf0210 msgr2=0x55bdfcc9a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1097539242 shutdown_connections 2024-05-08T23:52:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 1 -- 172.21.15.12:0/1097539242 wait complete. 2024-05-08T23:52:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f343fa2bb80 20 client.26879 trim_cache size 0 max 16384 2024-05-08T23:52:31.263 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:52:31.263 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:52:31.263 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:52:31.263 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:52:31.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f3433871700 1 -- 172.21.15.12:0/2663357140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34180a1310 msgr2=0x7f34180a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f3433871700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34180a1310 0x7f34180a37d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f341c009810 tx=0x7f341c019040 comp rx=0 tx=0).stop 2024-05-08T23:52:31.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f3433871700 1 -- 172.21.15.12:0/2663357140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f342c100060 msgr2=0x7f342c10d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.261+0000 7f3433871700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f342c100060 0x7f342c10d620 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f342800e9c0 tx=0x7f342800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.262+0000 7f3433871700 1 -- 172.21.15.12:0/2663357140 shutdown_connections 2024-05-08T23:52:31.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.262+0000 7f3433871700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34180a1310 0x7f34180a37d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.262+0000 7f3433871700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c10e0a0 0x7f342c1a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.262+0000 7f3433871700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f342c100060 0x7f342c10d620 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.262+0000 7f3433871700 1 --2- 172.21.15.12:0/2663357140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f342c105160 0x7f342c10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.262+0000 7f3433871700 1 -- 172.21.15.12:0/2663357140 >> 172.21.15.12:0/2663357140 conn(0x7f342c076f40 msgr2=0x7f342c0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.262+0000 7f3433871700 1 -- 172.21.15.12:0/2663357140 shutdown_connections 2024-05-08T23:52:31.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.262+0000 7f3433871700 1 -- 172.21.15.12:0/2663357140 wait complete. 2024-05-08T23:52:31.270 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:52:31.270 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:52:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.418+0000 7faa8886d700 1 -- 172.21.15.12:0/3196863934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa841028c0 msgr2=0x7faa8410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.418+0000 7faa8886d700 1 --2- 172.21.15.12:0/3196863934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa841028c0 0x7faa8410ada0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7faa78009990 tx=0x7faa78009ca0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.419+0000 7faa8886d700 1 -- 172.21.15.12:0/3196863934 shutdown_connections 2024-05-08T23:52:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.419+0000 7faa8886d700 1 --2- 172.21.15.12:0/3196863934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa8410b3c0 0x7faa8410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.419+0000 7faa8886d700 1 --2- 172.21.15.12:0/3196863934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa84101fa0 0x7faa84102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.419+0000 7faa8886d700 1 --2- 172.21.15.12:0/3196863934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa841028c0 0x7faa8410ada0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.419+0000 7faa8886d700 1 -- 172.21.15.12:0/3196863934 >> 172.21.15.12:0/3196863934 conn(0x7faa840fb7e0 msgr2=0x7faa840fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.419+0000 7faa8886d700 1 -- 172.21.15.12:0/3196863934 shutdown_connections 2024-05-08T23:52:31.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.419+0000 7faa8886d700 1 -- 172.21.15.12:0/3196863934 wait complete. 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8886d700 1 Processor -- start 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8886d700 1 -- start start 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8886d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa84101fa0 0x7faa84197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8886d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa841028c0 0x7faa84197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8886d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa8410b3c0 0x7faa8419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8886d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8419c260 con 0x7faa84101fa0 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8886d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8419c3d0 con 0x7faa8410b3c0 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8886d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8419c590 con 0x7faa841028c0 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa84101fa0 0x7faa84197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa84101fa0 0x7faa84197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35596/0 (socket says 172.21.15.12:35596) 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8259c700 1 -- 172.21.15.12:0/3284944769 learned_addr learned my addr 172.21.15.12:0/3284944769 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa81d9b700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa841028c0 0x7faa84197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa82d9d700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa8410b3c0 0x7faa8419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8259c700 1 -- 172.21.15.12:0/3284944769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa841028c0 msgr2=0x7faa84197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8259c700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa841028c0 0x7faa84197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8259c700 1 -- 172.21.15.12:0/3284944769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa8410b3c0 msgr2=0x7faa8419bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8259c700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa8410b3c0 0x7faa8419bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa8259c700 1 -- 172.21.15.12:0/3284944769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa780095f0 con 0x7faa84101fa0 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa81d9b700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa841028c0 0x7faa84197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.420+0000 7faa82d9d700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa8410b3c0 0x7faa8419bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.421+0000 7faa8259c700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa84101fa0 0x7faa84197340 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7faa7400e9a0 tx=0x7faa7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.421+0000 7faa737fe700 1 -- 172.21.15.12:0/3284944769 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74009790 con 0x7faa84101fa0 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.421+0000 7faa8886d700 1 -- 172.21.15.12:0/3284944769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8419c870 con 0x7faa84101fa0 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.421+0000 7faa737fe700 1 -- 172.21.15.12:0/3284944769 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa74010430 con 0x7faa84101fa0 2024-05-08T23:52:31.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.421+0000 7faa737fe700 1 -- 172.21.15.12:0/3284944769 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74018680 con 0x7faa84101fa0 2024-05-08T23:52:31.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.421+0000 7faa8886d700 1 -- 172.21.15.12:0/3284944769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa841a40b0 con 0x7faa84101fa0 2024-05-08T23:52:31.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.422+0000 7faa8886d700 1 -- 172.21.15.12:0/3284944769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa84106490 con 0x7faa84101fa0 2024-05-08T23:52:31.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.423+0000 7faa737fe700 1 -- 172.21.15.12:0/3284944769 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faa740105a0 con 0x7faa84101fa0 2024-05-08T23:52:31.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.424+0000 7faa737fe700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa6c0a1290 0x7faa6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.424+0000 7faa81d9b700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa6c0a1290 0x7faa6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.424+0000 7faa737fe700 1 -- 172.21.15.12:0/3284944769 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7faa74014070 con 0x7faa84101fa0 2024-05-08T23:52:31.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.425+0000 7faa81d9b700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa6c0a1290 0x7faa6c0a3750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa7800b2e0 tx=0x7faa78005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.429+0000 7faa737fe700 1 -- 172.21.15.12:0/3284944769 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faa74052d60 con 0x7faa84101fa0 2024-05-08T23:52:31.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.708+0000 7faa8886d700 1 -- 172.21.15.12:0/3284944769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa840688d0 con 0x7faa84101fa0 2024-05-08T23:52:31.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.709+0000 7faa737fe700 1 -- 172.21.15.12:0/3284944769 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10873 (secure 0 0 0) 0x7faa740d2020 con 0x7faa84101fa0 2024-05-08T23:52:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:52:31.704 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:50:48.465221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_26224":{"gid":26224,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/131208281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":131208281},{"type":"v1","addr":"172.21.15.204:6837","nonce":131208281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:52:31.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.711+0000 7faa8886d700 1 -- 172.21.15.12:0/3284944769 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa6c0a1290 msgr2=0x7faa6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.711+0000 7faa8886d700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa6c0a1290 0x7faa6c0a3750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa7800b2e0 tx=0x7faa78005960 comp rx=0 tx=0).stop 2024-05-08T23:52:31.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.711+0000 7faa8886d700 1 -- 172.21.15.12:0/3284944769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa84101fa0 msgr2=0x7faa84197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.711+0000 7faa8886d700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa84101fa0 0x7faa84197340 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7faa7400e9a0 tx=0x7faa7400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.712+0000 7faa8886d700 1 -- 172.21.15.12:0/3284944769 shutdown_connections 2024-05-08T23:52:31.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.712+0000 7faa8886d700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa6c0a1290 0x7faa6c0a3750 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.712+0000 7faa8886d700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa8410b3c0 0x7faa8419bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.712+0000 7faa8886d700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa84101fa0 0x7faa84197340 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.712+0000 7faa8886d700 1 --2- 172.21.15.12:0/3284944769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa841028c0 0x7faa84197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.712+0000 7faa8886d700 1 -- 172.21.15.12:0/3284944769 >> 172.21.15.12:0/3284944769 conn(0x7faa840fb7e0 msgr2=0x7faa840fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.712+0000 7faa8886d700 1 -- 172.21.15.12:0/3284944769 shutdown_connections 2024-05-08T23:52:31.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.712+0000 7faa8886d700 1 -- 172.21.15.12:0/3284944769 wait complete. 2024-05-08T23:52:31.708 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 31 2024-05-08T23:52:31.720 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:52:31.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.867+0000 7fc1533ce700 1 -- 172.21.15.12:0/298271320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc14c101b10 msgr2=0x7fc14c105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.867+0000 7fc1533ce700 1 --2- 172.21.15.12:0/298271320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc14c101b10 0x7fc14c105b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc1480098f0 tx=0x7fc148009c00 comp rx=0 tx=0).stop 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.867+0000 7fc1533ce700 1 -- 172.21.15.12:0/298271320 shutdown_connections 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.867+0000 7fc1533ce700 1 --2- 172.21.15.12:0/298271320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc14c106280 0x7fc14c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.867+0000 7fc1533ce700 1 --2- 172.21.15.12:0/298271320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc14c101160 0x7fc14c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.867+0000 7fc1533ce700 1 --2- 172.21.15.12:0/298271320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc14c101b10 0x7fc14c105b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.867+0000 7fc1533ce700 1 -- 172.21.15.12:0/298271320 >> 172.21.15.12:0/298271320 conn(0x7fc14c0fc9e0 msgr2=0x7fc14c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.867+0000 7fc1533ce700 1 -- 172.21.15.12:0/298271320 shutdown_connections 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.867+0000 7fc1533ce700 1 -- 172.21.15.12:0/298271320 wait complete. 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.868+0000 7fc1533ce700 1 Processor -- start 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.868+0000 7fc1533ce700 1 -- start start 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.868+0000 7fc1533ce700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc14c101160 0x7fc14c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.868+0000 7fc1533ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc14c106280 0x7fc14c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.868+0000 7fc1533ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc14c073d20 0x7fc14c1a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.868+0000 7fc1533ce700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c1a5c00 con 0x7fc14c106280 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.868+0000 7fc1533ce700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c1a5d70 con 0x7fc14c073d20 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.868+0000 7fc1533ce700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c1a5f30 con 0x7fc14c101160 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.868+0000 7fc150969700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc14c106280 0x7fc14c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc15116a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc14c101160 0x7fc14c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc15196b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc14c073d20 0x7fc14c1a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc150969700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc14c106280 0x7fc14c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35612/0 (socket says 172.21.15.12:35612) 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc150969700 1 -- 172.21.15.12:0/1108226181 learned_addr learned my addr 172.21.15.12:0/1108226181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc150969700 1 -- 172.21.15.12:0/1108226181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc14c101160 msgr2=0x7fc14c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc150969700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc14c101160 0x7fc14c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc150969700 1 -- 172.21.15.12:0/1108226181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc14c073d20 msgr2=0x7fc14c1a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc150969700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc14c073d20 0x7fc14c1a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc150969700 1 -- 172.21.15.12:0/1108226181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1400095f0 con 0x7fc14c106280 2024-05-08T23:52:31.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc150969700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc14c106280 0x7fc14c0737e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc148009230 tx=0x7fc14800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc13e7fc700 1 -- 172.21.15.12:0/1108226181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14801c070 con 0x7fc14c106280 2024-05-08T23:52:31.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc1533ce700 1 -- 172.21.15.12:0/1108226181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc148009520 con 0x7fc14c106280 2024-05-08T23:52:31.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc13e7fc700 1 -- 172.21.15.12:0/1108226181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc14800ba20 con 0x7fc14c106280 2024-05-08T23:52:31.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc13e7fc700 1 -- 172.21.15.12:0/1108226181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480038e0 con 0x7fc14c106280 2024-05-08T23:52:31.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.869+0000 7fc1533ce700 1 -- 172.21.15.12:0/1108226181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14c1a64f0 con 0x7fc14c106280 2024-05-08T23:52:31.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.871+0000 7fc13e7fc700 1 -- 172.21.15.12:0/1108226181 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc148020030 con 0x7fc14c106280 2024-05-08T23:52:31.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.872+0000 7fc13e7fc700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1380a1290 0x7fc1380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 365733 9.89 MB/sec execute 147 sec latency 41.672 ms 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.872+0000 7fc13e7fc700 1 -- 172.21.15.12:0/1108226181 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc1480cda00 con 0x7fc14c106280 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.872+0000 7fc15116a700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1380a1290 0x7fc1380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.873+0000 7fc15116a700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1380a1290 0x7fc1380a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc140005fd0 tx=0x7fc140009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.873+0000 7fc15f588b80 1 Processor -- start 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.873+0000 7fc15f588b80 1 -- start start 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8660 0x5613348b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 0x5613348b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15f588b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334801150 con 0x5613348b8f80 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15f588b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613347feb10 con 0x5613348b8660 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15f588b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613347fa400 con 0x5613348b30f0 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15116a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8660 0x5613348b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15196b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc150969700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 0x5613348b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15116a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8660 0x5613348b8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49804/0 (socket says 172.21.15.12:49804) 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15116a700 1 -- 172.21.15.12:0/2828637673 learned_addr learned my addr 172.21.15.12:0/2828637673 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15196b700 1 -- 172.21.15.12:0/2828637673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8660 msgr2=0x5613348b8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15196b700 1 --2- 172.21.15.12:0/2828637673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8660 0x5613348b8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15196b700 1 -- 172.21.15.12:0/2828637673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 msgr2=0x5613348b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15196b700 1 --2- 172.21.15.12:0/2828637673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 0x5613348b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15196b700 1 -- 172.21.15.12:0/2828637673 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56133467b3b0 con 0x5613348b30f0 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc150969700 1 --2- 172.21.15.12:0/2828637673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 0x5613348b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.874+0000 7fc15196b700 1 --2- 172.21.15.12:0/2828637673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348b3570 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc144011580 tx=0x7fc144009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84310a9f4e9ec219 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.875+0000 7fc1377fe700 1 -- 172.21.15.12:0/2828637673 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14400e040 con 0x5613348b30f0 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.875+0000 7fc1377fe700 1 -- 172.21.15.12:0/2828637673 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc14400f040 con 0x5613348b30f0 2024-05-08T23:52:31.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.875+0000 7fc1377fe700 1 -- 172.21.15.12:0/2828637673 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc144004390 con 0x5613348b30f0 2024-05-08T23:52:31.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.875+0000 7fc15f588b80 1 -- 172.21.15.12:0/2828637673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 msgr2=0x5613348b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.875+0000 7fc15f588b80 1 --2- 172.21.15.12:0/2828637673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348b3570 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc144011580 tx=0x7fc144009920 comp rx=0 tx=0).stop 2024-05-08T23:52:31.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.875+0000 7fc15f588b80 1 -- 172.21.15.12:0/2828637673 shutdown_connections 2024-05-08T23:52:31.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.875+0000 7fc15f588b80 1 --2- 172.21.15.12:0/2828637673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348b3570 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.875+0000 7fc15f588b80 1 --2- 172.21.15.12:0/2828637673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8660 0x5613348b8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.875+0000 7fc15f588b80 1 --2- 172.21.15.12:0/2828637673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 0x5613348b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.875+0000 7fc15f588b80 1 -- 172.21.15.12:0/2828637673 >> 172.21.15.12:0/2828637673 conn(0x561334808210 msgr2=0x5613348a4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.876+0000 7fc15f588b80 1 -- 172.21.15.12:0/2828637673 shutdown_connections 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.876+0000 7fc15f588b80 1 -- 172.21.15.12:0/2828637673 wait complete. 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.876+0000 7fc15f588b80 1 -- start start 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.876+0000 7fc15f588b80 1 Processor -- start 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b30f0 0x5613348bc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 0x5613348bc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 0x5613348bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15f588b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334796560 con 0x5613348b8f80 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15f588b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334796200 con 0x5613348b30f0 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15f588b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56133467d6a0 con 0x5613348b8660 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15f588b80 10 client.0 _ll_register_callbacks cb 0x56133479fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15196b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 0x5613348bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15116a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b30f0 0x5613348bc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc150969700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 0x5613348bc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15116a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b30f0 0x5613348bc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49808/0 (socket says 172.21.15.12:49808) 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15116a700 1 -- 172.21.15.12:0/1487536857 learned_addr learned my addr 172.21.15.12:0/1487536857 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15116a700 1 -- 172.21.15.12:0/1487536857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 msgr2=0x5613348bc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15116a700 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 0x5613348bc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15116a700 1 -- 172.21.15.12:0/1487536857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 msgr2=0x5613348bcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15116a700 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 0x5613348bcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15116a700 1 -- 172.21.15.12:0/1487536857 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14c1a64f0 con 0x5613348b30f0 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15196b700 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 0x5613348bcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15116a700 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b30f0 0x5613348bc260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc14000edb0 tx=0x7fc1400047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca616235d56b240b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc134ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc134ff9700 1 -- 172.21.15.12:0/1487536857 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140022070 con 0x5613348b30f0 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc134ff9700 1 -- 172.21.15.12:0/1487536857 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc14001da30 con 0x5613348b30f0 2024-05-08T23:52:31.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.877+0000 7fc15f588b80 1 -- 172.21.15.12:0/1487536857 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561334753270 con 0x5613348b30f0 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc134ff9700 1 -- 172.21.15.12:0/1487536857 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc14000ee90 con 0x5613348b30f0 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc15f588b80 10 client.26897 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc15f588b80 1 -- 172.21.15.12:0/1487536857 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fc144011230 con 0x5613348b30f0 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc134ff9700 1 -- 172.21.15.12:0/1487536857 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561334753270 con 0x5613348b30f0 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc134ff9700 1 -- 172.21.15.12:0/1487536857 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc140026ca0 con 0x5613348b30f0 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc15f588b80 10 client.26897 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc15f588b80 10 client.26897 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc15f588b80 20 client.26897 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc15f588b80 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5613348cca20 0x5613348cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc15f588b80 4 client.26897 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc15f588b80 1 -- 172.21.15.12:0/1487536857 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561334879810 con 0x5613348cca20 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.878+0000 7fc150969700 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5613348cca20 0x5613348cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.879+0000 7fc150969700 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5613348cca20 0x5613348cee40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=107c215ec8f59506 server_cookie=41a84070939d7bf7 in_seq=0 out_seq=0 2024-05-08T23:52:31.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.879+0000 7fc134ff9700 10 client.26897 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc134ff9700 1 -- 172.21.15.12:0/1487536857 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561334879810 con 0x5613348cca20 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc134ff9700 10 client.26897 handle_command_reply: tid=0 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 client.26897 shutdown 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 -- 172.21.15.12:0/1487536857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b30f0 msgr2=0x5613348bc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b30f0 0x5613348bc260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc14000edb0 tx=0x7fc1400047c0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 -- 172.21.15.12:0/1487536857 shutdown_connections 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8f80 0x5613348bcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b30f0 0x5613348bc260 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5613348cca20 0x5613348cee40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 --2- 172.21.15.12:0/1487536857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 0x5613348bc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 -- 172.21.15.12:0/1487536857 >> 172.21.15.12:0/1487536857 conn(0x561334808210 msgr2=0x5613348a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc134ff9700 0 client.26897 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 -- 172.21.15.12:0/1487536857 shutdown_connections 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 1 -- 172.21.15.12:0/1487536857 wait complete. 2024-05-08T23:52:31.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.880+0000 7fc15f588b80 20 client.26897 trim_cache size 0 max 16384 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.894+0000 7fc15f588b80 1 Processor -- start 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15f588b80 1 -- start start 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b30f0 0x5613348cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 0x5613348c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8f80 0x5613348a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15f588b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613348ca150 con 0x5613348b30f0 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15f588b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613348bedd0 con 0x5613348b8f80 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15f588b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613348a1dc0 con 0x5613348b8660 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15116a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b30f0 0x5613348cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc150969700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 0x5613348c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15116a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b30f0 0x5613348cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35648/0 (socket says 172.21.15.12:35648) 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15116a700 1 -- 172.21.15.12:0/1763588341 learned_addr learned my addr 172.21.15.12:0/1763588341 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc15196b700 1 --2- 172.21.15.12:0/1763588341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8f80 0x5613348a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc150969700 1 -- 172.21.15.12:0/1763588341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8f80 msgr2=0x5613348a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc150969700 1 --2- 172.21.15.12:0/1763588341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8f80 0x5613348a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc150969700 1 -- 172.21.15.12:0/1763588341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b30f0 msgr2=0x5613348cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc150969700 1 --2- 172.21.15.12:0/1763588341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b30f0 0x5613348cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc150969700 1 -- 172.21.15.12:0/1763588341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561334627b60 con 0x5613348b8660 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.895+0000 7fc150969700 1 --2- 172.21.15.12:0/1763588341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 0x5613348c8420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc1480cb160 tx=0x7fc148005470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=933e6ee302cc9932 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc137fff700 1 -- 172.21.15.12:0/1763588341 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14809abe0 con 0x5613348b8660 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc137fff700 1 -- 172.21.15.12:0/1763588341 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1480ba040 con 0x5613348b8660 2024-05-08T23:52:31.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc137fff700 1 -- 172.21.15.12:0/1763588341 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480990a0 con 0x5613348b8660 2024-05-08T23:52:31.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc15f588b80 1 -- 172.21.15.12:0/1763588341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 msgr2=0x5613348c8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc15f588b80 1 --2- 172.21.15.12:0/1763588341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 0x5613348c8420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc1480cb160 tx=0x7fc148005470 comp rx=0 tx=0).stop 2024-05-08T23:52:31.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc15f588b80 1 -- 172.21.15.12:0/1763588341 shutdown_connections 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc15f588b80 1 --2- 172.21.15.12:0/1763588341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8f80 0x5613348a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc15f588b80 1 --2- 172.21.15.12:0/1763588341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b30f0 0x5613348cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc15f588b80 1 --2- 172.21.15.12:0/1763588341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b8660 0x5613348c8420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc15f588b80 1 -- 172.21.15.12:0/1763588341 >> 172.21.15.12:0/1763588341 conn(0x561334808210 msgr2=0x5613348bcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc15f588b80 1 -- 172.21.15.12:0/1763588341 shutdown_connections 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.896+0000 7fc15f588b80 1 -- 172.21.15.12:0/1763588341 wait complete. 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.897+0000 7fc15f588b80 1 -- start start 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.897+0000 7fc15f588b80 1 Processor -- start 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.897+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.897+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8660 0x5613348c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.897+0000 7fc15f588b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8f80 0x5613348c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.897+0000 7fc15f588b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613348c4290 con 0x5613348b8660 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.897+0000 7fc15f588b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613348c23d0 con 0x5613348b8f80 2024-05-08T23:52:31.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.897+0000 7fc15f588b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613348c2590 con 0x5613348b30f0 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.897+0000 7fc15f588b80 10 client.0 _ll_register_callbacks cb 0x56133479fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15116a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc150969700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8660 0x5613348c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15116a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348c3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45394/0 (socket says 172.21.15.12:45394) 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15196b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8f80 0x5613348c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc150969700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8660 0x5613348c1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35658/0 (socket says 172.21.15.12:35658) 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15116a700 1 -- 172.21.15.12:0/663681083 learned_addr learned my addr 172.21.15.12:0/663681083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15116a700 1 -- 172.21.15.12:0/663681083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8f80 msgr2=0x5613348c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15116a700 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8f80 0x5613348c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15116a700 1 -- 172.21.15.12:0/663681083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8660 msgr2=0x5613348c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15116a700 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8660 0x5613348c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15116a700 1 -- 172.21.15.12:0/663681083 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc148009520 con 0x5613348b30f0 2024-05-08T23:52:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc150969700 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8660 0x5613348c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15116a700 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348c3b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc14000edb0 tx=0x7fc140009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d4910567a7bae41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc134ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc134ff9700 1 -- 172.21.15.12:0/663681083 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140022070 con 0x5613348b30f0 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc134ff9700 1 -- 172.21.15.12:0/663681083 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc14001c6f0 con 0x5613348b30f0 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15f588b80 1 -- 172.21.15.12:0/663681083 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc14400f040 con 0x5613348b30f0 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc134ff9700 1 -- 172.21.15.12:0/663681083 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc14000ee90 con 0x5613348b30f0 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15f588b80 10 client.26887 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.898+0000 7fc15f588b80 1 -- 172.21.15.12:0/663681083 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5613347c0560 con 0x5613348b30f0 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc134ff9700 1 -- 172.21.15.12:0/663681083 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc14400f040 con 0x5613348b30f0 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc134ff9700 1 -- 172.21.15.12:0/663681083 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc14003abb0 con 0x5613348b30f0 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc15f588b80 10 client.26887 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc15f588b80 10 client.26887 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc15f588b80 20 client.26887 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc15f588b80 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561334923310 0x561334925730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc15f588b80 4 client.26887 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc15f588b80 1 -- 172.21.15.12:0/663681083 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5613347f2ec0 con 0x561334923310 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc150969700 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561334923310 0x561334925730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc134ff9700 10 client.26887 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.899+0000 7fc150969700 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561334923310 0x561334925730 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2db77e961d09f2f server_cookie=67f66bd1267fa4 in_seq=0 out_seq=0 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc134ff9700 1 -- 172.21.15.12:0/663681083 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5613347f2ec0 con 0x561334923310 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc134ff9700 10 client.26887 handle_command_reply: tid=0 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 client.26887 shutdown 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 -- 172.21.15.12:0/663681083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 msgr2=0x5613348c3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348c3b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc14000edb0 tx=0x7fc140009380 comp rx=0 tx=0).stop 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 -- 172.21.15.12:0/663681083 shutdown_connections 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5613348b8f80 0x5613348c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561334923310 0x561334925730 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613348b30f0 0x5613348c3b60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 --2- 172.21.15.12:0/663681083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5613348b8660 0x5613348c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc134ff9700 0 client.26887 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:52:31.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 -- 172.21.15.12:0/663681083 >> 172.21.15.12:0/663681083 conn(0x561334808210 msgr2=0x5613348b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 -- 172.21.15.12:0/663681083 shutdown_connections 2024-05-08T23:52:31.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 1 -- 172.21.15.12:0/663681083 wait complete. 2024-05-08T23:52:31.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.900+0000 7fc15f588b80 20 client.26887 trim_cache size 0 max 16384 2024-05-08T23:52:31.901 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:52:31.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 -- 172.21.15.12:0/1108226181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1380a1290 msgr2=0x7fc1380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1380a1290 0x7fc1380a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc140005fd0 tx=0x7fc140009310 comp rx=0 tx=0).stop 2024-05-08T23:52:31.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 -- 172.21.15.12:0/1108226181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc14c106280 msgr2=0x7fc14c0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:31.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc14c106280 0x7fc14c0737e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc148009230 tx=0x7fc14800fa00 comp rx=0 tx=0).stop 2024-05-08T23:52:31.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 -- 172.21.15.12:0/1108226181 shutdown_connections 2024-05-08T23:52:31.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1380a1290 0x7fc1380a3750 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc14c073d20 0x7fc14c1a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc14c101160 0x7fc14c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 --2- 172.21.15.12:0/1108226181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc14c106280 0x7fc14c0737e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:31.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 -- 172.21.15.12:0/1108226181 >> 172.21.15.12:0/1108226181 conn(0x7fc14c0fc9e0 msgr2=0x7fc14c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:31.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 -- 172.21.15.12:0/1108226181 shutdown_connections 2024-05-08T23:52:31.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:31.901+0000 7fc1533ce700 1 -- 172.21.15.12:0/1108226181 wait complete. 2024-05-08T23:52:31.909 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:52:32.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.057+0000 7f4b66a46700 1 -- 172.21.15.12:0/4120702481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b60068970 msgr2=0x7f4b6010d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.057+0000 7f4b66a46700 1 --2- 172.21.15.12:0/4120702481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b60068970 0x7f4b6010d400 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009990 tx=0x7f4b50009ca0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.057+0000 7f4b66a46700 1 -- 172.21.15.12:0/4120702481 shutdown_connections 2024-05-08T23:52:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.057+0000 7f4b66a46700 1 --2- 172.21.15.12:0/4120702481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b6010d940 0x7f4b6010fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.057+0000 7f4b66a46700 1 --2- 172.21.15.12:0/4120702481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60068050 0x7f4b60068430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.057+0000 7f4b66a46700 1 --2- 172.21.15.12:0/4120702481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b60068970 0x7f4b6010d400 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.057+0000 7f4b66a46700 1 -- 172.21.15.12:0/4120702481 >> 172.21.15.12:0/4120702481 conn(0x7f4b60076f40 msgr2=0x7f4b60077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.058+0000 7f4b66a46700 1 -- 172.21.15.12:0/4120702481 shutdown_connections 2024-05-08T23:52:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.058+0000 7f4b66a46700 1 -- 172.21.15.12:0/4120702481 wait complete. 2024-05-08T23:52:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.058+0000 7f4b66a46700 1 Processor -- start 2024-05-08T23:52:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.058+0000 7f4b66a46700 1 -- start start 2024-05-08T23:52:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b66a46700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60068050 0x7f4b60199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b66a46700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b60068970 0x7f4b60199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b66a46700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b6010d940 0x7f4b6019df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b66a46700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6019e540 con 0x7f4b60068970 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b66a46700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6019e6b0 con 0x7f4b6010d940 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b66a46700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6019e870 con 0x7f4b60068050 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60068050 0x7f4b60199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b60068970 0x7f4b60199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60068050 0x7f4b60199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45420/0 (socket says 172.21.15.12:45420) 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b5ffff700 1 -- 172.21.15.12:0/2446847413 learned_addr learned my addr 172.21.15.12:0/2446847413 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b64fe3700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b6010d940 0x7f4b6019df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b5ffff700 1 -- 172.21.15.12:0/2446847413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b6010d940 msgr2=0x7f4b6019df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b6010d940 0x7f4b6019df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b5ffff700 1 -- 172.21.15.12:0/2446847413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b60068970 msgr2=0x7f4b60199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b60068970 0x7f4b60199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b5ffff700 1 -- 172.21.15.12:0/2446847413 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500095f0 con 0x7f4b60068050 2024-05-08T23:52:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b64fe3700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b6010d940 0x7f4b6019df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.059+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b60068970 0x7f4b60199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.060+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60068050 0x7f4b60199620 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400e9a0 tx=0x7f4b5400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.060+0000 7f4b5d7fa700 1 -- 172.21.15.12:0/2446847413 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54009790 con 0x7f4b60068050 2024-05-08T23:52:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.060+0000 7f4b5d7fa700 1 -- 172.21.15.12:0/2446847413 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b54010430 con 0x7f4b60068050 2024-05-08T23:52:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.060+0000 7f4b66a46700 1 -- 172.21.15.12:0/2446847413 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6019eb50 con 0x7f4b60068050 2024-05-08T23:52:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.060+0000 7f4b5d7fa700 1 -- 172.21.15.12:0/2446847413 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54018680 con 0x7f4b60068050 2024-05-08T23:52:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.060+0000 7f4b66a46700 1 -- 172.21.15.12:0/2446847413 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b601a6390 con 0x7f4b60068050 2024-05-08T23:52:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.063+0000 7f4b5d7fa700 1 -- 172.21.15.12:0/2446847413 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4b540105a0 con 0x7f4b60068050 2024-05-08T23:52:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.064+0000 7f4b5d7fa700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4b480a1310 0x7f4b480a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.064+0000 7f4b5d7fa700 1 -- 172.21.15.12:0/2446847413 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4b54014070 con 0x7f4b60068050 2024-05-08T23:52:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.064+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4b480a1310 0x7f4b480a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.065+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4b480a1310 0x7f4b480a37d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000b2e0 tx=0x7f4b50005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.065+0000 7f4b72c00b80 1 Processor -- start 2024-05-08T23:52:32.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.065+0000 7f4b72c00b80 1 -- start start 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.065+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.065+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.065+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 0x55f2cd1cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.065+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd121150 con 0x55f2cd1d0720 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.065+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd11eb10 con 0x55f2cd1ca890 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.065+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd11a400 con 0x55f2cd1cfe00 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45424/0 (socket says 172.21.15.12:45424) 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b5ffff700 1 -- 172.21.15.12:0/3212585028 learned_addr learned my addr 172.21.15.12:0/3212585028 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b64fe3700 1 --2- 172.21.15.12:0/3212585028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 0x55f2cd1cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b5ffff700 1 -- 172.21.15.12:0/3212585028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 msgr2=0x55f2cd1cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/3212585028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 0x55f2cd1cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b5ffff700 1 -- 172.21.15.12:0/3212585028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 msgr2=0x55f2cd1ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/3212585028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b5ffff700 1 -- 172.21.15.12:0/3212585028 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2ccf9b3b0 con 0x55f2cd1cfe00 2024-05-08T23:52:32.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/3212585028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d01e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4b540a1e90 tx=0x7f4b5409a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecf5bb5a3e12377 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.066+0000 7f4b467fc700 1 -- 172.21.15.12:0/3212585028 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540a23c0 con 0x55f2cd1cfe00 2024-05-08T23:52:32.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b467fc700 1 -- 172.21.15.12:0/3212585028 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b54010880 con 0x55f2cd1cfe00 2024-05-08T23:52:32.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b467fc700 1 -- 172.21.15.12:0/3212585028 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54099700 con 0x55f2cd1cfe00 2024-05-08T23:52:32.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b72c00b80 1 -- 172.21.15.12:0/3212585028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1cfe00 msgr2=0x55f2cd1d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/3212585028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d01e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4b540a1e90 tx=0x7f4b5409a640 comp rx=0 tx=0).stop 2024-05-08T23:52:32.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b72c00b80 1 -- 172.21.15.12:0/3212585028 shutdown_connections 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/3212585028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 0x55f2cd1cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/3212585028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d01e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/3212585028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b72c00b80 1 -- 172.21.15.12:0/3212585028 >> 172.21.15.12:0/3212585028 conn(0x55f2cd128210 msgr2=0x55f2cd1c4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b72c00b80 1 -- 172.21.15.12:0/3212585028 shutdown_connections 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.067+0000 7f4b72c00b80 1 -- 172.21.15.12:0/3212585028 wait complete. 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.068+0000 7f4b72c00b80 1 -- start start 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.068+0000 7f4b72c00b80 1 Processor -- start 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.068+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.068+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.068+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1d45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.068+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd0b6560 con 0x55f2cd1d0720 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.068+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd0b6200 con 0x55f2cd1cfe00 2024-05-08T23:52:32.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.068+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ccf9d6a0 con 0x55f2cd1ca890 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b72c00b80 10 client.0 _ll_register_callbacks cb 0x55f2cd0bfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b64fe3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1d45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45434/0 (socket says 172.21.15.12:45434) 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b5ffff700 1 -- 172.21.15.12:0/152889430 learned_addr learned my addr 172.21.15.12:0/152889430 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b64fe3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1d45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35708/0 (socket says 172.21.15.12:35708) 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b5ffff700 1 -- 172.21.15.12:0/152889430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 msgr2=0x55f2cd1d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b5ffff700 1 -- 172.21.15.12:0/152889430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 msgr2=0x55f2cd1d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1d45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b5ffff700 1 -- 172.21.15.12:0/152889430 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b601a6390 con 0x55f2cd1ca890 2024-05-08T23:52:32.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b64fe3700 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1d45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d3b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4b540ca630 tx=0x7f4b540996e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcbb5877ec70d010 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b3f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.069+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/152889430 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540a23c0 con 0x55f2cd1ca890 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/152889430 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4b540a2140 con 0x55f2cd1ca890 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b72c00b80 1 -- 172.21.15.12:0/152889430 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2cd073270 con 0x55f2cd1ca890 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/152889430 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b540b82c0 con 0x55f2cd1ca890 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b72c00b80 10 client.26899 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b72c00b80 1 -- 172.21.15.12:0/152889430 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f2ccf47b60 con 0x55f2cd1ca890 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/152889430 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2cd073270 con 0x55f2cd1ca890 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/152889430 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4b54098ea0 con 0x55f2cd1ca890 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b72c00b80 10 client.26899 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b72c00b80 10 client.26899 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b72c00b80 20 client.26899 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f2cd1e3d60 0x55f2cd1e6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b72c00b80 4 client.26899 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:32.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.070+0000 7f4b72c00b80 1 -- 172.21.15.12:0/152889430 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2cd199810 con 0x55f2cd1e3d60 2024-05-08T23:52:32.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.071+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f2cd1e3d60 0x55f2cd1e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.071+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f2cd1e3d60 0x55f2cd1e6150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=535c26de408873ec server_cookie=890efd4549f1dd13 in_seq=0 out_seq=0 2024-05-08T23:52:32.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.071+0000 7f4b3f7fe700 10 client.26899 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:52:32.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/152889430 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2cd199810 con 0x55f2cd1e3d60 2024-05-08T23:52:32.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b3f7fe700 10 client.26899 handle_command_reply: tid=0 2024-05-08T23:52:32.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b72c00b80 1 client.26899 shutdown 2024-05-08T23:52:32.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b72c00b80 1 -- 172.21.15.12:0/152889430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 msgr2=0x55f2cd1d3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d3b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4b540ca630 tx=0x7f4b540996e0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b72c00b80 1 -- 172.21.15.12:0/152889430 shutdown_connections 2024-05-08T23:52:32.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f2cd1e3d60 0x55f2cd1e6150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1d45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d3b40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/152889430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.072+0000 7f4b72c00b80 1 -- 172.21.15.12:0/152889430 >> 172.21.15.12:0/152889430 conn(0x55f2cd128210 msgr2=0x55f2cd1c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.073+0000 7f4b3f7fe700 0 client.26899 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:52:32.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.073+0000 7f4b72c00b80 1 -- 172.21.15.12:0/152889430 shutdown_connections 2024-05-08T23:52:32.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.073+0000 7f4b72c00b80 1 -- 172.21.15.12:0/152889430 wait complete. 2024-05-08T23:52:32.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.073+0000 7f4b72c00b80 20 client.26899 trim_cache size 0 max 16384 2024-05-08T23:52:32.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.088+0000 7f4b72c00b80 1 Processor -- start 2024-05-08T23:52:32.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.088+0000 7f4b72c00b80 1 -- start start 2024-05-08T23:52:32.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.088+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 0x55f2cd1ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.088+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.088+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.088+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd1bf800 con 0x55f2cd1d0720 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.088+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd1d6b10 con 0x55f2cd1cfe00 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.088+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd1c58e0 con 0x55f2cd1ca890 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.088+0000 7f4b64fe3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b64fe3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35724/0 (socket says 172.21.15.12:35724) 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b64fe3700 1 -- 172.21.15.12:0/509680277 learned_addr learned my addr 172.21.15.12:0/509680277 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/509680277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 0x55f2cd1ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/509680277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b64fe3700 1 -- 172.21.15.12:0/509680277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 msgr2=0x55f2cd1ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b64fe3700 1 --2- 172.21.15.12:0/509680277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 0x55f2cd1ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b64fe3700 1 -- 172.21.15.12:0/509680277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 msgr2=0x55f2cd1cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b64fe3700 1 --2- 172.21.15.12:0/509680277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b64fe3700 1 -- 172.21.15.12:0/509680277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2cd1caf40 con 0x55f2cd1d0720 2024-05-08T23:52:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/509680277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b64fe3700 1 --2- 172.21.15.12:0/509680277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1c4d20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800ea90 tx=0x7f4b58009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f819c616e97a2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b46ffd700 1 -- 172.21.15.12:0/509680277 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58004db0 con 0x55f2cd1d0720 2024-05-08T23:52:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b46ffd700 1 -- 172.21.15.12:0/509680277 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5801b070 con 0x55f2cd1d0720 2024-05-08T23:52:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.089+0000 7f4b46ffd700 1 -- 172.21.15.12:0/509680277 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58020810 con 0x55f2cd1d0720 2024-05-08T23:52:32.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.090+0000 7f4b72c00b80 1 -- 172.21.15.12:0/509680277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 msgr2=0x55f2cd1c4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.090+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/509680277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1c4d20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800ea90 tx=0x7f4b58009920 comp rx=0 tx=0).stop 2024-05-08T23:52:32.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.090+0000 7f4b72c00b80 1 -- 172.21.15.12:0/509680277 shutdown_connections 2024-05-08T23:52:32.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.090+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/509680277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1d0720 0x55f2cd1c4d20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.090+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/509680277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1ca890 0x55f2cd1ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.090+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/509680277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.090+0000 7f4b72c00b80 1 -- 172.21.15.12:0/509680277 >> 172.21.15.12:0/509680277 conn(0x55f2cd128210 msgr2=0x55f2cd1cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.090+0000 7f4b72c00b80 1 -- 172.21.15.12:0/509680277 shutdown_connections 2024-05-08T23:52:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.090+0000 7f4b72c00b80 1 -- 172.21.15.12:0/509680277 wait complete. 2024-05-08T23:52:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.090+0000 7f4b72c00b80 1 -- start start 2024-05-08T23:52:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b72c00b80 1 Processor -- start 2024-05-08T23:52:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b72c00b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1d0720 0x55f2cd1d96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd2391f0 con 0x55f2cd1cfe00 2024-05-08T23:52:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd239330 con 0x55f2cd1ca890 2024-05-08T23:52:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b72c00b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd2394f0 con 0x55f2cd1d0720 2024-05-08T23:52:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b72c00b80 10 client.0 _ll_register_callbacks cb 0x55f2cd0bfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35732/0 (socket says 172.21.15.12:35732) 2024-05-08T23:52:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b5f7fe700 1 -- 172.21.15.12:0/2354006290 learned_addr learned my addr 172.21.15.12:0/2354006290 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.091+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b64fe3700 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1d0720 0x55f2cd1d96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b5ffff700 1 -- 172.21.15.12:0/2354006290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1d0720 msgr2=0x55f2cd1d96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1d0720 0x55f2cd1d96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b5ffff700 1 -- 172.21.15.12:0/2354006290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1cfe00 msgr2=0x55f2cd1d9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b5ffff700 1 -- 172.21.15.12:0/2354006290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b60110860 con 0x55f2cd1ca890 2024-05-08T23:52:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b5ffff700 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d8c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4b5409a5a0 tx=0x7f4b54099640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e15bbc6516041fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b3f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:52:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/2354006290 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5409a640 con 0x55f2cd1ca890 2024-05-08T23:52:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b72c00b80 1 -- 172.21.15.12:0/2354006290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b54010880 con 0x55f2cd1ca890 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/2354006290 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4b540a23c0 con 0x55f2cd1ca890 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/2354006290 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b54096280 con 0x55f2cd1ca890 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b72c00b80 10 client.26921 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.092+0000 7f4b72c00b80 1 -- 172.21.15.12:0/2354006290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f4b5800e740 con 0x55f2cd1ca890 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/2354006290 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b54010880 con 0x55f2cd1ca890 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/2354006290 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4b54095d30 con 0x55f2cd1ca890 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b72c00b80 10 client.26921 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b72c00b80 10 client.26921 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b72c00b80 20 client.26921 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f2cd23a7c0 0x55f2cd23cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b72c00b80 4 client.26921 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b72c00b80 1 -- 172.21.15.12:0/2354006290 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f2cd112ec0 con 0x55f2cd23a7c0 2024-05-08T23:52:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f2cd23a7c0 0x55f2cd23cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b3f7fe700 10 client.26921 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.093+0000 7f4b5f7fe700 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f2cd23a7c0 0x55f2cd23cbb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eaf478f9849667c3 server_cookie=2644d2ed47d4ea0f in_seq=0 out_seq=0 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b3f7fe700 1 -- 172.21.15.12:0/2354006290 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f2cd112ec0 con 0x55f2cd23a7c0 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b3f7fe700 10 client.26921 handle_command_reply: tid=0 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 client.26921 shutdown 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 -- 172.21.15.12:0/2354006290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 msgr2=0x55f2cd1d8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d8c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4b5409a5a0 tx=0x7f4b54099640 comp rx=0 tx=0).stop 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 -- 172.21.15.12:0/2354006290 shutdown_connections 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f2cd23a7c0 0x55f2cd23cbb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2cd1d0720 0x55f2cd1d96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f2cd1ca890 0x55f2cd1d8c30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 --2- 172.21.15.12:0/2354006290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2cd1cfe00 0x55f2cd1d9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b3f7fe700 0 client.26921 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 -- 172.21.15.12:0/2354006290 >> 172.21.15.12:0/2354006290 conn(0x55f2cd128210 msgr2=0x55f2cd1d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 -- 172.21.15.12:0/2354006290 shutdown_connections 2024-05-08T23:52:32.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 1 -- 172.21.15.12:0/2354006290 wait complete. 2024-05-08T23:52:32.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.094+0000 7f4b72c00b80 20 client.26921 trim_cache size 0 max 16384 2024-05-08T23:52:32.097 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:52:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 -- 172.21.15.12:0/2446847413 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4b480a1310 msgr2=0x7f4b480a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4b480a1310 0x7f4b480a37d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000b2e0 tx=0x7f4b50005960 comp rx=0 tx=0).stop 2024-05-08T23:52:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 -- 172.21.15.12:0/2446847413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60068050 msgr2=0x7f4b60199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60068050 0x7f4b60199620 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400e9a0 tx=0x7f4b5400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 -- 172.21.15.12:0/2446847413 shutdown_connections 2024-05-08T23:52:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4b480a1310 0x7f4b480a37d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b6010d940 0x7f4b6019df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60068050 0x7f4b60199620 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 --2- 172.21.15.12:0/2446847413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b60068970 0x7f4b60199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 -- 172.21.15.12:0/2446847413 >> 172.21.15.12:0/2446847413 conn(0x7f4b60076f40 msgr2=0x7f4b6010efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 -- 172.21.15.12:0/2446847413 shutdown_connections 2024-05-08T23:52:32.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.095+0000 7f4b66a46700 1 -- 172.21.15.12:0/2446847413 wait complete. 2024-05-08T23:52:32.104 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:52:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.253+0000 7f05097cf700 1 -- 172.21.15.12:0/3701729719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05041062b0 msgr2=0x7f050410de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.253+0000 7f05097cf700 1 --2- 172.21.15.12:0/3701729719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05041062b0 0x7f050410de40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00b0a0 tx=0x7f04fc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.253+0000 7f05097cf700 1 -- 172.21.15.12:0/3701729719 shutdown_connections 2024-05-08T23:52:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.253+0000 7f05097cf700 1 --2- 172.21.15.12:0/3701729719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05041062b0 0x7f050410de40 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.254+0000 7f05097cf700 1 --2- 172.21.15.12:0/3701729719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05040ffa80 0x7f05040ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.254+0000 7f05097cf700 1 --2- 172.21.15.12:0/3701729719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0504100430 0x7f0504105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.254+0000 7f05097cf700 1 -- 172.21.15.12:0/3701729719 >> 172.21.15.12:0/3701729719 conn(0x7f0504076110 msgr2=0x7f0504076520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.254+0000 7f05097cf700 1 -- 172.21.15.12:0/3701729719 shutdown_connections 2024-05-08T23:52:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.254+0000 7f05097cf700 1 -- 172.21.15.12:0/3701729719 wait complete. 2024-05-08T23:52:32.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.254+0000 7f05097cf700 1 Processor -- start 2024-05-08T23:52:32.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05097cf700 1 -- start start 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05097cf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05040ffa80 0x7f05041974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05097cf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05041062b0 0x7f05041979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05097cf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f050419bd80 0x7f050419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05097cf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050419c800 con 0x7f05041062b0 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05097cf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050419c970 con 0x7f050419bd80 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05097cf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050419cb30 con 0x7f05040ffa80 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05027fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05041062b0 0x7f05041979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05040ffa80 0x7f05041974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05027fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05041062b0 0x7f05041979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35748/0 (socket says 172.21.15.12:35748) 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05027fc700 1 -- 172.21.15.12:0/3585964800 learned_addr learned my addr 172.21.15.12:0/3585964800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05027fc700 1 -- 172.21.15.12:0/3585964800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05040ffa80 msgr2=0x7f05041974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05027fc700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05040ffa80 0x7f05041974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05027fc700 1 -- 172.21.15.12:0/3585964800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f050419bd80 msgr2=0x7f050419c200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:52:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05027fc700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f050419bd80 0x7f050419c200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f05027fc700 1 -- 172.21.15.12:0/3585964800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f8009520 con 0x7f05041062b0 2024-05-08T23:52:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.255+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05040ffa80 0x7f05041974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.256+0000 7f05027fc700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05041062b0 0x7f05041979e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009fd0 tx=0x7f04f800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.256+0000 7f04ebfff700 1 -- 172.21.15.12:0/3585964800 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f8009820 con 0x7f05041062b0 2024-05-08T23:52:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.256+0000 7f05097cf700 1 -- 172.21.15.12:0/3585964800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc009d00 con 0x7f05041062b0 2024-05-08T23:52:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.256+0000 7f04ebfff700 1 -- 172.21.15.12:0/3585964800 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04f80044e0 con 0x7f05041062b0 2024-05-08T23:52:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.256+0000 7f04ebfff700 1 -- 172.21.15.12:0/3585964800 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f8003d90 con 0x7f05041062b0 2024-05-08T23:52:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.256+0000 7f05097cf700 1 -- 172.21.15.12:0/3585964800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0504100d50 con 0x7f05041062b0 2024-05-08T23:52:32.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.259+0000 7f04ebfff700 1 -- 172.21.15.12:0/3585964800 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f04f8003f10 con 0x7f05041062b0 2024-05-08T23:52:32.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.260+0000 7f04ebfff700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04ec0a1290 0x7f04ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.260+0000 7f04ebfff700 1 -- 172.21.15.12:0/3585964800 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f04f8014070 con 0x7f05041062b0 2024-05-08T23:52:32.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.260+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04ec0a1290 0x7f04ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.260+0000 7f0515989b80 1 Processor -- start 2024-05-08T23:52:32.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04ec0a1290 0x7f04ec0a3750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f04f4011490 tx=0x7f04f4009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f0515989b80 1 -- start start 2024-05-08T23:52:32.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f0515989b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 0x5632477481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f0515989b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f0515989b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247742890 0x563247742cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f0515989b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563247699150 con 0x563247742890 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f0515989b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563247696b10 con 0x563247748720 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f0515989b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563247692400 con 0x563247747e00 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f05037fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247742890 0x563247742cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 0x5632477481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f05027fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f05037fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247742890 0x563247742cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35756/0 (socket says 172.21.15.12:35756) 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f05027fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49902/0 (socket says 172.21.15.12:49902) 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 0x5632477481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45480/0 (socket says 172.21.15.12:45480) 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.261+0000 7f05037fe700 1 -- 172.21.15.12:0/2813198378 learned_addr learned my addr 172.21.15.12:0/2813198378 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f0502ffd700 1 -- 172.21.15.12:0/2813198378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 msgr2=0x563247742350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f0502ffd700 1 --2- 172.21.15.12:0/2813198378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f0502ffd700 1 -- 172.21.15.12:0/2813198378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247742890 msgr2=0x563247742cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f0502ffd700 1 --2- 172.21.15.12:0/2813198378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247742890 0x563247742cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f0502ffd700 1 -- 172.21.15.12:0/2813198378 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632475133b0 con 0x563247747e00 2024-05-08T23:52:32.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f05037fe700 1 --2- 172.21.15.12:0/2813198378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247742890 0x563247742cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f05027fc700 1 --2- 172.21.15.12:0/2813198378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f0502ffd700 1 --2- 172.21.15.12:0/2813198378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 0x5632477481e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04f40042f0 tx=0x7f04f4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee01a277a29fe350 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f04e97fa700 1 -- 172.21.15.12:0/2813198378 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f4003d10 con 0x563247747e00 2024-05-08T23:52:32.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f04e97fa700 1 -- 172.21.15.12:0/2813198378 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04f4003e70 con 0x563247747e00 2024-05-08T23:52:32.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.262+0000 7f04e97fa700 1 -- 172.21.15.12:0/2813198378 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f401e760 con 0x563247747e00 2024-05-08T23:52:32.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.263+0000 7f0515989b80 1 -- 172.21.15.12:0/2813198378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 msgr2=0x5632477481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.263+0000 7f0515989b80 1 --2- 172.21.15.12:0/2813198378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 0x5632477481e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04f40042f0 tx=0x7f04f4014000 comp rx=0 tx=0).stop 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.263+0000 7f0515989b80 1 -- 172.21.15.12:0/2813198378 shutdown_connections 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.263+0000 7f0515989b80 1 --2- 172.21.15.12:0/2813198378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247742890 0x563247742cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.263+0000 7f0515989b80 1 --2- 172.21.15.12:0/2813198378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 0x5632477481e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.263+0000 7f0515989b80 1 --2- 172.21.15.12:0/2813198378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.263+0000 7f0515989b80 1 -- 172.21.15.12:0/2813198378 >> 172.21.15.12:0/2813198378 conn(0x5632476a0210 msgr2=0x56324773cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.263+0000 7f0515989b80 1 -- 172.21.15.12:0/2813198378 shutdown_connections 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.263+0000 7f0515989b80 1 -- 172.21.15.12:0/2813198378 wait complete. 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.263+0000 7f0515989b80 1 -- start start 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0515989b80 1 Processor -- start 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0515989b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324774bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0515989b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 0x56324774c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0515989b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247748720 0x56324774c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0515989b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324762e560 con 0x563247748720 2024-05-08T23:52:32.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0515989b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324762e200 con 0x563247742890 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0515989b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632475156a0 con 0x563247747e00 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324774bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0515989b80 10 client.0 _ll_register_callbacks cb 0x563247637b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f05037fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247748720 0x56324774c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324774bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49906/0 (socket says 172.21.15.12:49906) 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f0502ffd700 1 -- 172.21.15.12:0/3259875071 learned_addr learned my addr 172.21.15.12:0/3259875071 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.264+0000 7f05027fc700 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 0x56324774c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f05037fe700 1 -- 172.21.15.12:0/3259875071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 msgr2=0x56324774c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f05037fe700 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 0x56324774c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f05037fe700 1 -- 172.21.15.12:0/3259875071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 msgr2=0x56324774bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f05037fe700 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324774bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f05037fe700 1 -- 172.21.15.12:0/3259875071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0504100d50 con 0x563247748720 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324774bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f05037fe700 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247748720 0x56324774c5f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0062a0 tx=0x7f04fc007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c62b5a9777f00bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f04e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3259875071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc00e040 con 0x563247748720 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f0515989b80 1 -- 172.21.15.12:0/3259875071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632475eb270 con 0x563247748720 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3259875071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f04fc006020 con 0x563247748720 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3259875071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04fc003a00 con 0x563247748720 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f0515989b80 10 client.17145 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f0515989b80 1 -- 172.21.15.12:0/3259875071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f04f4017070 con 0x563247748720 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.265+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3259875071 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632475eb270 con 0x563247748720 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.266+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3259875071 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f04fc019070 con 0x563247748720 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.266+0000 7f0515989b80 10 client.17145 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.266+0000 7f0515989b80 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.266+0000 7f0515989b80 20 client.17145 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.266+0000 7f0515989b80 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563247764970 0x563247766d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.266+0000 7f0515989b80 4 client.17145 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.266+0000 7f0515989b80 1 -- 172.21.15.12:0/3259875071 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563247711810 con 0x563247764970 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.266+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563247764970 0x563247766d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.266+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563247764970 0x563247766d90 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b07e13ab89b931 server_cookie=aa986d40cffa5252 in_seq=0 out_seq=0 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.266+0000 7f04e6ff5700 10 client.17145 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.267+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3259875071 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563247711810 con 0x563247764970 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f04e6ff5700 10 client.17145 handle_command_reply: tid=0 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 client.17145 shutdown 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 -- 172.21.15.12:0/3259875071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247748720 msgr2=0x56324774c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247748720 0x56324774c5f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0062a0 tx=0x7f04fc007dd0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 -- 172.21.15.12:0/3259875071 shutdown_connections 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247748720 0x56324774c5f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563247764970 0x563247766d90 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324774bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 --2- 172.21.15.12:0/3259875071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247747e00 0x56324774c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 -- 172.21.15.12:0/3259875071 >> 172.21.15.12:0/3259875071 conn(0x5632476a0210 msgr2=0x56324773c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f04e6ff5700 0 client.17145 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:32.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 -- 172.21.15.12:0/3259875071 shutdown_connections 2024-05-08T23:52:32.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 1 -- 172.21.15.12:0/3259875071 wait complete. 2024-05-08T23:52:32.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.268+0000 7f0515989b80 20 client.17145 trim_cache size 0 max 16384 2024-05-08T23:52:32.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.282+0000 7f0515989b80 1 Processor -- start 2024-05-08T23:52:32.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.282+0000 7f0515989b80 1 -- start start 2024-05-08T23:52:32.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0515989b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247742890 0x5632477481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0515989b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247760430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0515989b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0515989b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632477615f0 con 0x563247747e00 2024-05-08T23:52:32.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0515989b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324774d1e0 con 0x563247748720 2024-05-08T23:52:32.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0515989b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563247739dc0 con 0x563247742890 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f05027fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247760430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247742890 0x5632477481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f05037fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247742890 0x5632477481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45508/0 (socket says 172.21.15.12:45508) 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f05037fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49912/0 (socket says 172.21.15.12:49912) 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f05027fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247760430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35760/0 (socket says 172.21.15.12:35760) 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0502ffd700 1 -- 172.21.15.12:0/3486076992 learned_addr learned my addr 172.21.15.12:0/3486076992 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0502ffd700 1 -- 172.21.15.12:0/3486076992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 msgr2=0x563247742c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3486076992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0502ffd700 1 -- 172.21.15.12:0/3486076992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 msgr2=0x563247760430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3486076992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247760430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0502ffd700 1 -- 172.21.15.12:0/3486076992 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632474bfb60 con 0x563247742890 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f05037fe700 1 --2- 172.21.15.12:0/3486076992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.283+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3486076992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247742890 0x5632477481e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f04f4026ea0 tx=0x7f04f4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12cc190ddfd1bcfb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f04e9ffb700 1 -- 172.21.15.12:0/3486076992 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f402b070 con 0x563247742890 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f04e9ffb700 1 -- 172.21.15.12:0/3486076992 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04f402c040 con 0x563247742890 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f04e9ffb700 1 -- 172.21.15.12:0/3486076992 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f4033070 con 0x563247742890 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f0515989b80 1 -- 172.21.15.12:0/3486076992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247742890 msgr2=0x5632477481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f0515989b80 1 --2- 172.21.15.12:0/3486076992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247742890 0x5632477481e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f04f4026ea0 tx=0x7f04f4017000 comp rx=0 tx=0).stop 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f0515989b80 1 -- 172.21.15.12:0/3486076992 shutdown_connections 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f0515989b80 1 --2- 172.21.15.12:0/3486076992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247748720 0x563247742c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f0515989b80 1 --2- 172.21.15.12:0/3486076992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247742890 0x5632477481e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f0515989b80 1 --2- 172.21.15.12:0/3486076992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247760430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f0515989b80 1 -- 172.21.15.12:0/3486076992 >> 172.21.15.12:0/3486076992 conn(0x5632476a0210 msgr2=0x563247739210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f0515989b80 1 -- 172.21.15.12:0/3486076992 shutdown_connections 2024-05-08T23:52:32.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.284+0000 7f0515989b80 1 -- 172.21.15.12:0/3486076992 wait complete. 2024-05-08T23:52:32.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.285+0000 7f0515989b80 1 -- start start 2024-05-08T23:52:32.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.285+0000 7f0515989b80 1 Processor -- start 2024-05-08T23:52:32.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.285+0000 7f0515989b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324775bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.285+0000 7f0515989b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247759960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.285+0000 7f0515989b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247748720 0x563247759ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.285+0000 7f0515989b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324775c2c0 con 0x563247747e00 2024-05-08T23:52:32.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.285+0000 7f0515989b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324775a410 con 0x563247742890 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.285+0000 7f0515989b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324775a5d0 con 0x563247748720 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.285+0000 7f0515989b80 10 client.0 _ll_register_callbacks cb 0x563247637b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f05027fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247759960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324775bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f0502ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324775bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49926/0 (socket says 172.21.15.12:49926) 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f0502ffd700 1 -- 172.21.15.12:0/3585413797 learned_addr learned my addr 172.21.15.12:0/3585413797 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f05027fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247759960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35772/0 (socket says 172.21.15.12:35772) 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f05037fe700 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247748720 0x563247759ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f0502ffd700 1 -- 172.21.15.12:0/3585413797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247748720 msgr2=0x563247759ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247748720 0x563247759ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f0502ffd700 1 -- 172.21.15.12:0/3585413797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 msgr2=0x563247759960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247759960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f0502ffd700 1 -- 172.21.15.12:0/3585413797 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f4017070 con 0x563247742890 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f05027fc700 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247759960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f0502ffd700 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324775bb70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f04f4011d20 tx=0x7f04f4005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad3e937c1991d4bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f04e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3585413797 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f402b070 con 0x563247742890 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f0515989b80 1 -- 172.21.15.12:0/3585413797 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04f4003e70 con 0x563247742890 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.286+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3585413797 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f04f40053b0 con 0x563247742890 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3585413797 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04f4004f30 con 0x563247742890 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f0515989b80 10 client.26936 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f0515989b80 1 -- 172.21.15.12:0/3585413797 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5632475133b0 con 0x563247742890 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3585413797 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04f4003e70 con 0x563247742890 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3585413797 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f04f4024020 con 0x563247742890 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f0515989b80 10 client.26936 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f0515989b80 10 client.26936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:32.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f0515989b80 20 client.26936 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f0515989b80 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5632477baf00 0x5632477bd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f0515989b80 4 client.26936 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f0515989b80 1 -- 172.21.15.12:0/3585413797 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56324768aec0 con 0x5632477baf00 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.287+0000 7f05027fc700 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5632477baf00 0x5632477bd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f05027fc700 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5632477baf00 0x5632477bd320 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab66c0fa20767345 server_cookie=517d434ca7a6f3cc in_seq=0 out_seq=0 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f04e6ff5700 10 client.26936 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f04e6ff5700 1 -- 172.21.15.12:0/3585413797 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56324768aec0 con 0x5632477baf00 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f04e6ff5700 10 client.26936 handle_command_reply: tid=0 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f0515989b80 1 client.26936 shutdown 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f0515989b80 1 -- 172.21.15.12:0/3585413797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 msgr2=0x56324775bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f0515989b80 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324775bb70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f04f4011d20 tx=0x7f04f4005c80 comp rx=0 tx=0).stop 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f0515989b80 1 -- 172.21.15.12:0/3585413797 shutdown_connections 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f0515989b80 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563247748720 0x563247759ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f0515989b80 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5632477baf00 0x5632477bd320 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f0515989b80 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563247742890 0x56324775bb70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f0515989b80 1 --2- 172.21.15.12:0/3585413797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563247747e00 0x563247759960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f04e6ff5700 0 client.26936 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:32.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.288+0000 7f0515989b80 1 -- 172.21.15.12:0/3585413797 >> 172.21.15.12:0/3585413797 conn(0x5632476a0210 msgr2=0x563247742c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f0515989b80 1 -- 172.21.15.12:0/3585413797 shutdown_connections 2024-05-08T23:52:32.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f0515989b80 1 -- 172.21.15.12:0/3585413797 wait complete. 2024-05-08T23:52:32.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f0515989b80 20 client.26936 trim_cache size 0 max 16384 2024-05-08T23:52:32.290 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:52:32.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f05097cf700 1 -- 172.21.15.12:0/3585964800 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04ec0a1290 msgr2=0x7f04ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f05097cf700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04ec0a1290 0x7f04ec0a3750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f04f4011490 tx=0x7f04f4009260 comp rx=0 tx=0).stop 2024-05-08T23:52:32.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f05097cf700 1 -- 172.21.15.12:0/3585964800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05041062b0 msgr2=0x7f05041979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:32.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f05097cf700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05041062b0 0x7f05041979e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009fd0 tx=0x7f04f800ed10 comp rx=0 tx=0).stop 2024-05-08T23:52:32.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f05097cf700 1 -- 172.21.15.12:0/3585964800 shutdown_connections 2024-05-08T23:52:32.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f05097cf700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04ec0a1290 0x7f04ec0a3750 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f05097cf700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f050419bd80 0x7f050419c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f05097cf700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05040ffa80 0x7f05041974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f05097cf700 1 --2- 172.21.15.12:0/3585964800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05041062b0 0x7f05041979e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:32.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.289+0000 7f05097cf700 1 -- 172.21.15.12:0/3585964800 >> 172.21.15.12:0/3585964800 conn(0x7f0504076110 msgr2=0x7f05040fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:32.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.290+0000 7f05097cf700 1 -- 172.21.15.12:0/3585964800 shutdown_connections 2024-05-08T23:52:32.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:32.290+0000 7f05097cf700 1 -- 172.21.15.12:0/3585964800 wait complete. 2024-05-08T23:52:32.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:32 smithi204 ceph-mon[47373]: pgmap v188: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-05-08T23:52:32.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:32 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3284944769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:52:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:32 smithi160 ceph-mon[46152]: pgmap v188: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-05-08T23:52:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:32 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3284944769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:52:32.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 367234 9.86 MB/sec execute 148 sec latency 64.762 ms 2024-05-08T23:52:32.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:32 smithi012 ceph-mon[42503]: pgmap v188: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-05-08T23:52:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:32 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3284944769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:52:33.299 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:52:33.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:52:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.466+0000 7f440b8fc700 1 -- 172.21.15.12:0/279482930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4404073370 msgr2=0x7f4404073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.466+0000 7f440b8fc700 1 --2- 172.21.15.12:0/279482930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4404073370 0x7f4404073750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4400009990 tx=0x7f4400009ca0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.466+0000 7f440b8fc700 1 -- 172.21.15.12:0/279482930 shutdown_connections 2024-05-08T23:52:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.466+0000 7f440b8fc700 1 --2- 172.21.15.12:0/279482930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f440410d790 0x7f440410fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.466+0000 7f440b8fc700 1 --2- 172.21.15.12:0/279482930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4404073370 0x7f4404073750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.466+0000 7f440b8fc700 1 --2- 172.21.15.12:0/279482930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4404073c90 0x7f440410d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.466+0000 7f440b8fc700 1 -- 172.21.15.12:0/279482930 >> 172.21.15.12:0/279482930 conn(0x7f44040fc9e0 msgr2=0x7f44040fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.467+0000 7f440b8fc700 1 -- 172.21.15.12:0/279482930 shutdown_connections 2024-05-08T23:52:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.467+0000 7f440b8fc700 1 -- 172.21.15.12:0/279482930 wait complete. 2024-05-08T23:52:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f440b8fc700 1 Processor -- start 2024-05-08T23:52:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f440b8fc700 1 -- start start 2024-05-08T23:52:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f440b8fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4404073370 0x7f4404100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f440b8fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4404073c90 0x7f44041013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f440b8fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f440410d790 0x7f4404101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f440b8fc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44041a9c20 con 0x7f4404073370 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f440b8fc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4404199330 con 0x7f440410d790 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f440b8fc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44041994a0 con 0x7f4404073c90 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f4409698700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4404073370 0x7f4404100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f4408e97700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4404073c90 0x7f44041013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f4409698700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4404073370 0x7f4404100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35802/0 (socket says 172.21.15.12:35802) 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f4409698700 1 -- 172.21.15.12:0/3420158288 learned_addr learned my addr 172.21.15.12:0/3420158288 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f4408e97700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4404073c90 0x7f44041013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45524/0 (socket says 172.21.15.12:45524) 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f4409e99700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f440410d790 0x7f4404101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f4408e97700 1 -- 172.21.15.12:0/3420158288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f440410d790 msgr2=0x7f4404101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f4408e97700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f440410d790 0x7f4404101930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f4408e97700 1 -- 172.21.15.12:0/3420158288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4404073370 msgr2=0x7f4404100eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.468+0000 7f4408e97700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4404073370 0x7f4404100eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.469+0000 7f4408e97700 1 -- 172.21.15.12:0/3420158288 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44000095f0 con 0x7f4404073c90 2024-05-08T23:52:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.469+0000 7f4408e97700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4404073c90 0x7f44041013f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f43f400e9f0 tx=0x7f43f400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.469+0000 7f43fa7fc700 1 -- 172.21.15.12:0/3420158288 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f400c9c0 con 0x7f4404073c90 2024-05-08T23:52:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.469+0000 7f43fa7fc700 1 -- 172.21.15.12:0/3420158288 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f4010430 con 0x7f4404073c90 2024-05-08T23:52:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.469+0000 7f440b8fc700 1 -- 172.21.15.12:0/3420158288 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4404199780 con 0x7f4404073c90 2024-05-08T23:52:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.469+0000 7f43fa7fc700 1 -- 172.21.15.12:0/3420158288 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4018680 con 0x7f4404073c90 2024-05-08T23:52:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.469+0000 7f440b8fc700 1 -- 172.21.15.12:0/3420158288 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4404199ce0 con 0x7f4404073c90 2024-05-08T23:52:33.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.472+0000 7f43fa7fc700 1 -- 172.21.15.12:0/3420158288 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f43f4010860 con 0x7f4404073c90 2024-05-08T23:52:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.473+0000 7f43fa7fc700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f43f00a1310 0x7f43f00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.473+0000 7f43fa7fc700 1 -- 172.21.15.12:0/3420158288 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f43f4014070 con 0x7f4404073c90 2024-05-08T23:52:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.473+0000 7f4409698700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f43f00a1310 0x7f43f00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.474+0000 7f4409698700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f43f00a1310 0x7f43f00a37d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f440000b6f0 tx=0x7f4400005bc0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.474+0000 7f4417ab6b80 1 Processor -- start 2024-05-08T23:52:33.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.474+0000 7f4417ab6b80 1 -- start start 2024-05-08T23:52:33.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.474+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.474+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.474+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87c8910 0x5648f87c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.474+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f871f1d0 con 0x5648f87c8910 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.474+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f871cb90 con 0x5648f87ce7a0 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.474+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f8718480 con 0x5648f87cde80 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.474+0000 7f4409698700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f4408e97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f4409698700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45532/0 (socket says 172.21.15.12:45532) 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f4409698700 1 -- 172.21.15.12:0/3221997290 learned_addr learned my addr 172.21.15.12:0/3221997290 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f4408e97700 1 -- 172.21.15.12:0/3221997290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 msgr2=0x5648f87ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f4408e97700 1 --2- 172.21.15.12:0/3221997290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f4408e97700 1 -- 172.21.15.12:0/3221997290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87c8910 msgr2=0x5648f87c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f4408e97700 1 --2- 172.21.15.12:0/3221997290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87c8910 0x5648f87c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f4408e97700 1 -- 172.21.15.12:0/3221997290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648f8599430 con 0x5648f87ce7a0 2024-05-08T23:52:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f4409698700 1 --2- 172.21.15.12:0/3221997290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f4408e97700 1 --2- 172.21.15.12:0/3221997290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87c83d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f43f40ca510 tx=0x7f43f4096340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b9df9d73288ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f43ef7fe700 1 -- 172.21.15.12:0/3221997290 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4099ed0 con 0x5648f87ce7a0 2024-05-08T23:52:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f43ef7fe700 1 -- 172.21.15.12:0/3221997290 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f409dd00 con 0x5648f87ce7a0 2024-05-08T23:52:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.475+0000 7f43ef7fe700 1 -- 172.21.15.12:0/3221997290 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4099620 con 0x5648f87ce7a0 2024-05-08T23:52:33.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.476+0000 7f4417ab6b80 1 -- 172.21.15.12:0/3221997290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 msgr2=0x5648f87c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.476+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/3221997290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87c83d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f43f40ca510 tx=0x7f43f4096340 comp rx=0 tx=0).stop 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.476+0000 7f4417ab6b80 1 -- 172.21.15.12:0/3221997290 shutdown_connections 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.476+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/3221997290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87c8910 0x5648f87c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.476+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/3221997290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.476+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/3221997290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87c83d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.476+0000 7f4417ab6b80 1 -- 172.21.15.12:0/3221997290 >> 172.21.15.12:0/3221997290 conn(0x5648f8726290 msgr2=0x5648f87c2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.476+0000 7f4417ab6b80 1 -- 172.21.15.12:0/3221997290 shutdown_connections 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.476+0000 7f4417ab6b80 1 -- 172.21.15.12:0/3221997290 wait complete. 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.477+0000 7f4417ab6b80 1 -- start start 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.477+0000 7f4417ab6b80 1 Processor -- start 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87c8910 0x5648f87d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f86b45e0 con 0x5648f87c8910 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f86b4280 con 0x5648f87ce7a0 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f859b720 con 0x5648f87cde80 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4417ab6b80 10 client.0 _ll_register_callbacks cb 0x5648f86bdbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409e99700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4408e97700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409e99700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87d2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49958/0 (socket says 172.21.15.12:49958) 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409e99700 1 -- 172.21.15.12:0/2861068324 learned_addr learned my addr 172.21.15.12:0/2861068324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409698700 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87c8910 0x5648f87d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409e99700 1 -- 172.21.15.12:0/2861068324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 msgr2=0x5648f87d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409e99700 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409e99700 1 -- 172.21.15.12:0/2861068324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87c8910 msgr2=0x5648f87d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409e99700 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87c8910 0x5648f87d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409e99700 1 -- 172.21.15.12:0/2861068324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44041108d0 con 0x5648f87ce7a0 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409698700 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87c8910 0x5648f87d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f4409e99700 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87d2640 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f43fc011d00 tx=0x7f43fc00f490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5462973cdb7bdd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.478+0000 7f43ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.479+0000 7f43ecff9700 1 -- 172.21.15.12:0/2861068324 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc00e040 con 0x5648f87ce7a0 2024-05-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.479+0000 7f43ecff9700 1 -- 172.21.15.12:0/2861068324 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f43fc003a00 con 0x5648f87ce7a0 2024-05-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.479+0000 7f4417ab6b80 1 -- 172.21.15.12:0/2861068324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648f86712f0 con 0x5648f87ce7a0 2024-05-08T23:52:33.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.485+0000 7f43ecff9700 1 -- 172.21.15.12:0/2861068324 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43fc003bc0 con 0x5648f87ce7a0 2024-05-08T23:52:33.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.486+0000 7f4417ab6b80 10 client.26948 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:33.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.486+0000 7f4417ab6b80 1 -- 172.21.15.12:0/2861068324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f4404199ce0 con 0x5648f87ce7a0 2024-05-08T23:52:33.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.486+0000 7f43ecff9700 1 -- 172.21.15.12:0/2861068324 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648f86712f0 con 0x5648f87ce7a0 2024-05-08T23:52:33.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.486+0000 7f43ecff9700 1 -- 172.21.15.12:0/2861068324 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f43fc007b20 con 0x5648f87ce7a0 2024-05-08T23:52:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.487+0000 7f4417ab6b80 10 client.26948 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.487+0000 7f4417ab6b80 10 client.26948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.487+0000 7f4417ab6b80 20 client.26948 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.487+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648f87e22b0 0x5648f87e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.487+0000 7f4417ab6b80 4 client.26948 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.487+0000 7f4417ab6b80 1 -- 172.21.15.12:0/2861068324 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648f8797890 con 0x5648f87e22b0 2024-05-08T23:52:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.487+0000 7f4409698700 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648f87e22b0 0x5648f87e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.487+0000 7f43ecff9700 10 client.26948 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.487+0000 7f4409698700 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648f87e22b0 0x5648f87e46d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e261db118388584 server_cookie=47172d23e8cdc7ed in_seq=0 out_seq=0 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.488+0000 7f43ecff9700 1 -- 172.21.15.12:0/2861068324 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648f8797890 con 0x5648f87e22b0 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.488+0000 7f43ecff9700 10 client.26948 handle_command_reply: tid=0 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.488+0000 7f4417ab6b80 1 client.26948 shutdown 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.488+0000 7f4417ab6b80 1 -- 172.21.15.12:0/2861068324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 msgr2=0x5648f87d2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.488+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87d2640 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f43fc011d00 tx=0x7f43fc00f490 comp rx=0 tx=0).stop 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.489+0000 7f4417ab6b80 1 -- 172.21.15.12:0/2861068324 shutdown_connections 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.489+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648f87e22b0 0x5648f87e46d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.489+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87ce7a0 0x5648f87d2640 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.489+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87c8910 0x5648f87d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.489+0000 7f43ecff9700 0 client.26948 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.489+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/2861068324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.489+0000 7f4417ab6b80 1 -- 172.21.15.12:0/2861068324 >> 172.21.15.12:0/2861068324 conn(0x5648f8726290 msgr2=0x5648f87c2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.489+0000 7f4417ab6b80 1 -- 172.21.15.12:0/2861068324 shutdown_connections 2024-05-08T23:52:33.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.489+0000 7f4417ab6b80 1 -- 172.21.15.12:0/2861068324 wait complete. 2024-05-08T23:52:33.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.489+0000 7f4417ab6b80 20 client.26948 trim_cache size 0 max 16384 2024-05-08T23:52:33.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.509+0000 7f4417ab6b80 1 Processor -- start 2024-05-08T23:52:33.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4417ab6b80 1 -- start start 2024-05-08T23:52:33.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 0x5648f87d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f87e9900 con 0x5648f87ce7a0 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f87d4be0 con 0x5648f87c8910 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f87d0130 con 0x5648f87cde80 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4409698700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4409e99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 0x5648f87d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4408e97700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4409698700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49962/0 (socket says 172.21.15.12:49962) 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4408e97700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45544/0 (socket says 172.21.15.12:45544) 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4409e99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 0x5648f87d2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35830/0 (socket says 172.21.15.12:35830) 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4409698700 1 -- 172.21.15.12:0/3148995235 learned_addr learned my addr 172.21.15.12:0/3148995235 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4409698700 1 -- 172.21.15.12:0/3148995235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 msgr2=0x5648f87c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.510+0000 7f4409698700 1 --2- 172.21.15.12:0/3148995235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.511+0000 7f4409698700 1 -- 172.21.15.12:0/3148995235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 msgr2=0x5648f87d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.511+0000 7f4409698700 1 --2- 172.21.15.12:0/3148995235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 0x5648f87d2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.511+0000 7f4409698700 1 -- 172.21.15.12:0/3148995235 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648f87c8fc0 con 0x5648f87c8910 2024-05-08T23:52:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.511+0000 7f4409e99700 1 --2- 172.21.15.12:0/3148995235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 0x5648f87d2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.511+0000 7f4408e97700 1 --2- 172.21.15.12:0/3148995235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.511+0000 7f4409698700 1 --2- 172.21.15.12:0/3148995235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87cd560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4400005210 tx=0x7f4400020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926ddd3f8bfbd499 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.511+0000 7f43effff700 1 -- 172.21.15.12:0/3148995235 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400004860 con 0x5648f87c8910 2024-05-08T23:52:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.511+0000 7f43effff700 1 -- 172.21.15.12:0/3148995235 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f440001e040 con 0x5648f87c8910 2024-05-08T23:52:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.511+0000 7f43effff700 1 -- 172.21.15.12:0/3148995235 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000f460 con 0x5648f87c8910 2024-05-08T23:52:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 -- 172.21.15.12:0/3148995235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 msgr2=0x5648f87cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/3148995235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87cd560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4400005210 tx=0x7f4400020000 comp rx=0 tx=0).stop 2024-05-08T23:52:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 -- 172.21.15.12:0/3148995235 shutdown_connections 2024-05-08T23:52:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/3148995235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 0x5648f87d2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/3148995235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87cd560 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/3148995235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 -- 172.21.15.12:0/3148995235 >> 172.21.15.12:0/3148995235 conn(0x5648f8726290 msgr2=0x5648f87d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 -- 172.21.15.12:0/3148995235 shutdown_connections 2024-05-08T23:52:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 -- 172.21.15.12:0/3148995235 wait complete. 2024-05-08T23:52:33.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 -- start start 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.512+0000 7f4417ab6b80 1 Processor -- start 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87d7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4417ab6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 0x5648f87d87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f88387e0 con 0x5648f87ce7a0 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f8838920 con 0x5648f87c8910 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4417ab6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f8838a60 con 0x5648f87cde80 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4417ab6b80 10 client.0 _ll_register_callbacks cb 0x5648f86bdbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4409698700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87d7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4409e99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 0x5648f87d87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4409698700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87d7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49976/0 (socket says 172.21.15.12:49976) 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4409698700 1 -- 172.21.15.12:0/1811535863 learned_addr learned my addr 172.21.15.12:0/1811535863 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4408e97700 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4409698700 1 -- 172.21.15.12:0/1811535863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 msgr2=0x5648f87d82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4409698700 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87d82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4409698700 1 -- 172.21.15.12:0/1811535863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 msgr2=0x5648f87d87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4409698700 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 0x5648f87d87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.513+0000 7f4409698700 1 -- 172.21.15.12:0/1811535863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440000bab0 con 0x5648f87c8910 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.514+0000 7f4409698700 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87d7d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4400009ce0 tx=0x7f4400005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f81fb8187463913b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.514+0000 7f43ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.514+0000 7f43ecff9700 1 -- 172.21.15.12:0/1811535863 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44000350f0 con 0x5648f87c8910 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.514+0000 7f43ecff9700 1 -- 172.21.15.12:0/1811535863 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4400035250 con 0x5648f87c8910 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.514+0000 7f4417ab6b80 1 -- 172.21.15.12:0/1811535863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43f409dd00 con 0x5648f87c8910 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.514+0000 7f43ecff9700 1 -- 172.21.15.12:0/1811535863 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4400035410 con 0x5648f87c8910 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.514+0000 7f4417ab6b80 10 client.26954 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.514+0000 7f4417ab6b80 1 -- 172.21.15.12:0/1811535863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5648f87bf560 con 0x5648f87c8910 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.514+0000 7f43ecff9700 1 -- 172.21.15.12:0/1811535863 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f409dd00 con 0x5648f87c8910 2024-05-08T23:52:33.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.515+0000 7f43ecff9700 1 -- 172.21.15.12:0/1811535863 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f440000fa30 con 0x5648f87c8910 2024-05-08T23:52:33.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.515+0000 7f4417ab6b80 10 client.26954 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:33.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.515+0000 7f4417ab6b80 10 client.26954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:33.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.515+0000 7f4417ab6b80 20 client.26954 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:33.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.515+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648f8839d90 0x5648f883c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.515+0000 7f4417ab6b80 4 client.26954 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:52:33.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.515+0000 7f4417ab6b80 1 -- 172.21.15.12:0/1811535863 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5648f8710f40 con 0x5648f8839d90 2024-05-08T23:52:33.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.515+0000 7f4408e97700 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648f8839d90 0x5648f883c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.515+0000 7f43ecff9700 10 client.26954 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:33.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.515+0000 7f4408e97700 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648f8839d90 0x5648f883c180 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33585ab50df58f49 server_cookie=50dc32328d57b3a3 in_seq=0 out_seq=0 2024-05-08T23:52:33.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f43ecff9700 1 -- 172.21.15.12:0/1811535863 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5648f8710f40 con 0x5648f8839d90 2024-05-08T23:52:33.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f43ecff9700 10 client.26954 handle_command_reply: tid=0 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 client.26954 shutdown 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 -- 172.21.15.12:0/1811535863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 msgr2=0x5648f87d7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87d7d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4400009ce0 tx=0x7f4400005a90 comp rx=0 tx=0).stop 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 -- 172.21.15.12:0/1811535863 shutdown_connections 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648f87ce7a0 0x5648f87d87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5648f8839d90 0x5648f883c180 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5648f87c8910 0x5648f87d7d60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 --2- 172.21.15.12:0/1811535863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648f87cde80 0x5648f87d82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f43ecff9700 0 client.26954 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 -- 172.21.15.12:0/1811535863 >> 172.21.15.12:0/1811535863 conn(0x5648f8726290 msgr2=0x5648f87cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 -- 172.21.15.12:0/1811535863 shutdown_connections 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 1 -- 172.21.15.12:0/1811535863 wait complete. 2024-05-08T23:52:33.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.517+0000 7f4417ab6b80 20 client.26954 trim_cache size 0 max 16384 2024-05-08T23:52:33.517 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:52:33.517 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:52:33.517 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "bc5fadf5-e391-45b2-add5-35eba79cdb28", 2024-05-08T23:52:33.517 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:52:33.517 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 -- 172.21.15.12:0/3420158288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f43f00a1310 msgr2=0x7f43f00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f43f00a1310 0x7f43f00a37d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f440000b6f0 tx=0x7f4400005bc0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 -- 172.21.15.12:0/3420158288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4404073c90 msgr2=0x7f44041013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4404073c90 0x7f44041013f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f43f400e9f0 tx=0x7f43f400ed00 comp rx=0 tx=0).stop 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 -- 172.21.15.12:0/3420158288 shutdown_connections 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f43f00a1310 0x7f43f00a37d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f440410d790 0x7f4404101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4404073370 0x7f4404100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 --2- 172.21.15.12:0/3420158288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4404073c90 0x7f44041013f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 -- 172.21.15.12:0/3420158288 >> 172.21.15.12:0/3420158288 conn(0x7f44040fc9e0 msgr2=0x7f44040fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 -- 172.21.15.12:0/3420158288 shutdown_connections 2024-05-08T23:52:33.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.518+0000 7f440b8fc700 1 -- 172.21.15.12:0/3420158288 wait complete. 2024-05-08T23:52:33.530 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:52:33.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:33 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:52:33.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.683+0000 7f68e8572700 1 -- 172.21.15.12:0/4192795741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e00682e0 msgr2=0x7f68e00686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.683+0000 7f68e8572700 1 --2- 172.21.15.12:0/4192795741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e00682e0 0x7f68e00686c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009990 tx=0x7f68dc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.683+0000 7f68e8572700 1 -- 172.21.15.12:0/4192795741 shutdown_connections 2024-05-08T23:52:33.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.683+0000 7f68e8572700 1 --2- 172.21.15.12:0/4192795741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68e0106290 0x7f68e010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.683+0000 7f68e8572700 1 --2- 172.21.15.12:0/4192795741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e00682e0 0x7f68e00686c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.683+0000 7f68e8572700 1 --2- 172.21.15.12:0/4192795741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68e0068c90 0x7f68e0105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.683+0000 7f68e8572700 1 -- 172.21.15.12:0/4192795741 >> 172.21.15.12:0/4192795741 conn(0x7f68e0076f40 msgr2=0x7f68e0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.683+0000 7f68e8572700 1 -- 172.21.15.12:0/4192795741 shutdown_connections 2024-05-08T23:52:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.683+0000 7f68e8572700 1 -- 172.21.15.12:0/4192795741 wait complete. 2024-05-08T23:52:33.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e8572700 1 Processor -- start 2024-05-08T23:52:33.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e8572700 1 -- start start 2024-05-08T23:52:33.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e8572700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68e00682e0 0x7f68e01973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e8572700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e0106290 0x7f68e0197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e8572700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68e019bca0 0x7f68e019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e8572700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e019c720 con 0x7f68e019bca0 2024-05-08T23:52:33.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e8572700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e019c890 con 0x7f68e0106290 2024-05-08T23:52:33.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e8572700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e019ca50 con 0x7f68e00682e0 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e6b0f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68e019bca0 0x7f68e019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e5b0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e0106290 0x7f68e0197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e6b0f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68e019bca0 0x7f68e019c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35860/0 (socket says 172.21.15.12:35860) 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e6b0f700 1 -- 172.21.15.12:0/1094040109 learned_addr learned my addr 172.21.15.12:0/1094040109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e0106290 0x7f68e0197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50006/0 (socket says 172.21.15.12:50006) 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.684+0000 7f68e630e700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68e00682e0 0x7f68e01973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68e6b0f700 1 -- 172.21.15.12:0/1094040109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68e00682e0 msgr2=0x7f68e01973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68e6b0f700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68e00682e0 0x7f68e01973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68e6b0f700 1 -- 172.21.15.12:0/1094040109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e0106290 msgr2=0x7f68e0197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68e6b0f700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e0106290 0x7f68e0197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68e6b0f700 1 -- 172.21.15.12:0/1094040109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d8009c40 con 0x7f68e019bca0 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e0106290 0x7f68e0197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68e630e700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68e00682e0 0x7f68e01973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:33.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68e6b0f700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68e019bca0 0x7f68e019c120 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f68d800c370 tx=0x7f68d800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68d77fe700 1 -- 172.21.15.12:0/1094040109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8017070 con 0x7f68e019bca0 2024-05-08T23:52:33.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68e8572700 1 -- 172.21.15.12:0/1094040109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68dc0095f0 con 0x7f68e019bca0 2024-05-08T23:52:33.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68d77fe700 1 -- 172.21.15.12:0/1094040109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68d800f040 con 0x7f68e019bca0 2024-05-08T23:52:33.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68d77fe700 1 -- 172.21.15.12:0/1094040109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d800e050 con 0x7f68e019bca0 2024-05-08T23:52:33.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.685+0000 7f68e8572700 1 -- 172.21.15.12:0/1094040109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e01a4250 con 0x7f68e019bca0 2024-05-08T23:52:33.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.689+0000 7f68d77fe700 1 -- 172.21.15.12:0/1094040109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f68d800cb60 con 0x7f68e019bca0 2024-05-08T23:52:33.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.690+0000 7f68d77fe700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68cc0a1290 0x7f68cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.690+0000 7f68d77fe700 1 -- 172.21.15.12:0/1094040109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f68d80cc580 con 0x7f68e019bca0 2024-05-08T23:52:33.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.690+0000 7f68f472cb80 1 Processor -- start 2024-05-08T23:52:33.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68f472cb80 1 -- start start 2024-05-08T23:52:33.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c5220be00 0x564c5220c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52206350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c52206890 0x564c52206cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68f472cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c5215d150 con 0x564c5220be00 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68f472cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c5215ab10 con 0x564c5220c720 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68f472cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c52156400 con 0x564c52206890 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68e5b0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52206350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68e6b0f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c52206890 0x564c52206cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68e5b0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52206350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50008/0 (socket says 172.21.15.12:50008) 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.691+0000 7f68e5b0d700 1 -- 172.21.15.12:0/1652163004 learned_addr learned my addr 172.21.15.12:0/1652163004 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68e630e700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68cc0a1290 0x7f68cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68e5b0d700 1 -- 172.21.15.12:0/1652163004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c52206890 msgr2=0x564c52206cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/1652163004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c52206890 0x564c52206cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68e5b0d700 1 -- 172.21.15.12:0/1652163004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c5220be00 msgr2=0x564c5220c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/1652163004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c5220be00 0x564c5220c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68e5b0d700 1 -- 172.21.15.12:0/1652163004 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c51fd73b0 con 0x564c5220c720 2024-05-08T23:52:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/1652163004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52206350 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68d0011540 tx=0x7f68d0011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44591fac5cf9786c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68e630e700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68cc0a1290 0x7f68cc0a3750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0098e0 tx=0x7f68dc00b650 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68d4ff9700 1 -- 172.21.15.12:0/1652163004 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0004a30 con 0x564c5220c720 2024-05-08T23:52:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68d4ff9700 1 -- 172.21.15.12:0/1652163004 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68d00054e0 con 0x564c5220c720 2024-05-08T23:52:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.692+0000 7f68d4ff9700 1 -- 172.21.15.12:0/1652163004 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d000e7e0 con 0x564c5220c720 2024-05-08T23:52:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.693+0000 7f68f472cb80 1 -- 172.21.15.12:0/1652163004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 msgr2=0x564c52206350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.693+0000 7f68f472cb80 1 --2- 172.21.15.12:0/1652163004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52206350 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68d0011540 tx=0x7f68d0011850 comp rx=0 tx=0).stop 2024-05-08T23:52:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.694+0000 7f68f472cb80 1 -- 172.21.15.12:0/1652163004 shutdown_connections 2024-05-08T23:52:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.694+0000 7f68f472cb80 1 --2- 172.21.15.12:0/1652163004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c52206890 0x564c52206cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.694+0000 7f68f472cb80 1 --2- 172.21.15.12:0/1652163004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c5220be00 0x564c5220c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.694+0000 7f68f472cb80 1 --2- 172.21.15.12:0/1652163004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52206350 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.694+0000 7f68f472cb80 1 -- 172.21.15.12:0/1652163004 >> 172.21.15.12:0/1652163004 conn(0x564c52164210 msgr2=0x564c52200df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.694+0000 7f68f472cb80 1 -- 172.21.15.12:0/1652163004 shutdown_connections 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.694+0000 7f68f472cb80 1 -- 172.21.15.12:0/1652163004 wait complete. 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.694+0000 7f68f472cb80 1 -- start start 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.694+0000 7f68f472cb80 1 Processor -- start 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5220fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52210490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68f472cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c520f2560 con 0x564c52206890 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68f472cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c520f2200 con 0x564c5220c720 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68f472cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c51fd96a0 con 0x564c5220be00 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68f472cb80 10 client.0 _ll_register_callbacks cb 0x564c520fbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e5b0d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e5b0d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45594/0 (socket says 172.21.15.12:45594) 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e5b0d700 1 -- 172.21.15.12:0/3082779546 learned_addr learned my addr 172.21.15.12:0/3082779546 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e6b0f700 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52210490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e630e700 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5220fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e5b0d700 1 -- 172.21.15.12:0/3082779546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 msgr2=0x564c52210490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52210490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e5b0d700 1 -- 172.21.15.12:0/3082779546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 msgr2=0x564c5220fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5220fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e5b0d700 1 -- 172.21.15.12:0/3082779546 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d0009d10 con 0x564c5220be00 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.695+0000 7f68e630e700 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5220fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.696+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220ff50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f68d001b040 tx=0x7f68d0004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fa613db222c78a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.696+0000 7f68c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.696+0000 7f68c9ffb700 1 -- 172.21.15.12:0/3082779546 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0005170 con 0x564c5220be00 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.696+0000 7f68c9ffb700 1 -- 172.21.15.12:0/3082779546 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f68d00052d0 con 0x564c5220be00 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.696+0000 7f68f472cb80 1 -- 172.21.15.12:0/3082779546 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c520af270 con 0x564c5220be00 2024-05-08T23:52:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.696+0000 7f68c9ffb700 1 -- 172.21.15.12:0/3082779546 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68d0004020 con 0x564c5220be00 2024-05-08T23:52:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.696+0000 7f68f472cb80 10 client.26953 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.696+0000 7f68f472cb80 1 -- 172.21.15.12:0/3082779546 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564c51f83b60 con 0x564c5220be00 2024-05-08T23:52:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.696+0000 7f68c9ffb700 1 -- 172.21.15.12:0/3082779546 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c520af270 con 0x564c5220be00 2024-05-08T23:52:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.697+0000 7f68c9ffb700 1 -- 172.21.15.12:0/3082779546 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f68d0024430 con 0x564c5220be00 2024-05-08T23:52:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.698+0000 7f68f472cb80 10 client.26953 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.698+0000 7f68f472cb80 10 client.26953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.698+0000 7f68f472cb80 20 client.26953 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.698+0000 7f68f472cb80 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564c5221f560 0x564c5221f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.698+0000 7f68f472cb80 4 client.26953 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.698+0000 7f68f472cb80 1 -- 172.21.15.12:0/3082779546 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c521d5810 con 0x564c5221f560 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.698+0000 7f68e630e700 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564c5221f560 0x564c5221f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.698+0000 7f68c9ffb700 10 client.26953 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.698+0000 7f68e630e700 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564c5221f560 0x564c5221f970 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba528b361943ae server_cookie=4d0282268b54a838 in_seq=0 out_seq=0 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68c9ffb700 1 -- 172.21.15.12:0/3082779546 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c521d5810 con 0x564c5221f560 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68c9ffb700 10 client.26953 handle_command_reply: tid=0 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 client.26953 shutdown 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 -- 172.21.15.12:0/3082779546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 msgr2=0x564c5220ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220ff50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f68d001b040 tx=0x7f68d0004be0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 -- 172.21.15.12:0/3082779546 shutdown_connections 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564c5221f560 0x564c5221f970 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52210490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5220fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 --2- 172.21.15.12:0/3082779546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220ff50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 -- 172.21.15.12:0/3082779546 >> 172.21.15.12:0/3082779546 conn(0x564c52164210 msgr2=0x564c522005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68c9ffb700 0 client.26953 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:33.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 -- 172.21.15.12:0/3082779546 shutdown_connections 2024-05-08T23:52:33.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 1 -- 172.21.15.12:0/3082779546 wait complete. 2024-05-08T23:52:33.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.699+0000 7f68f472cb80 20 client.26953 trim_cache size 0 max 16384 2024-05-08T23:52:33.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68f472cb80 1 Processor -- start 2024-05-08T23:52:33.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68f472cb80 1 -- start start 2024-05-08T23:52:33.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c52206350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52200d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68f472cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c522018e0 con 0x564c52206890 2024-05-08T23:52:33.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68f472cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c52211080 con 0x564c5220c720 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68f472cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c52202e30 con 0x564c5220be00 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68e630e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c52206350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68e5b0d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68e6b0f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52200d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68e630e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c52206350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35874/0 (socket says 172.21.15.12:35874) 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.722+0000 7f68e630e700 1 -- 172.21.15.12:0/2242669555 learned_addr learned my addr 172.21.15.12:0/2242669555 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68e630e700 1 -- 172.21.15.12:0/2242669555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 msgr2=0x564c5220b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68e630e700 1 --2- 172.21.15.12:0/2242669555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68e630e700 1 -- 172.21.15.12:0/2242669555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 msgr2=0x564c52200d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68e630e700 1 --2- 172.21.15.12:0/2242669555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52200d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68e630e700 1 -- 172.21.15.12:0/2242669555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c52206f40 con 0x564c52206890 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/2242669555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68e6b0f700 1 --2- 172.21.15.12:0/2242669555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52200d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68e630e700 1 --2- 172.21.15.12:0/2242669555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c52206350 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009990 tx=0x7f68dc022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39e498f2b68d9e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68d57fa700 1 -- 172.21.15.12:0/2242669555 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc02d040 con 0x564c52206890 2024-05-08T23:52:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68d57fa700 1 -- 172.21.15.12:0/2242669555 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68dc0056a0 con 0x564c52206890 2024-05-08T23:52:33.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.723+0000 7f68d57fa700 1 -- 172.21.15.12:0/2242669555 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc01d070 con 0x564c52206890 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.724+0000 7f68f472cb80 1 -- 172.21.15.12:0/2242669555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 msgr2=0x564c52206350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.724+0000 7f68f472cb80 1 --2- 172.21.15.12:0/2242669555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c52206350 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009990 tx=0x7f68dc022000 comp rx=0 tx=0).stop 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.725+0000 7f68f472cb80 1 -- 172.21.15.12:0/2242669555 shutdown_connections 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.725+0000 7f68f472cb80 1 --2- 172.21.15.12:0/2242669555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220c720 0x564c52200d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.725+0000 7f68f472cb80 1 --2- 172.21.15.12:0/2242669555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c52206350 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.725+0000 7f68f472cb80 1 --2- 172.21.15.12:0/2242669555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220be00 0x564c5220b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.725+0000 7f68f472cb80 1 -- 172.21.15.12:0/2242669555 >> 172.21.15.12:0/2242669555 conn(0x564c52164210 msgr2=0x564c5220e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.725+0000 7f68f472cb80 1 -- 172.21.15.12:0/2242669555 shutdown_connections 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.725+0000 7f68f472cb80 1 -- 172.21.15.12:0/2242669555 wait complete. 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.725+0000 7f68f472cb80 1 -- start start 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68f472cb80 1 Processor -- start 2024-05-08T23:52:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5222fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220be00 0x564c522079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68f472cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220c720 0x564c52207ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68f472cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c5222ff90 con 0x564c52206890 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68f472cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c52230100 con 0x564c5220be00 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68f472cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c522302c0 con 0x564c5220c720 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68f472cb80 10 client.0 _ll_register_callbacks cb 0x564c520fbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68e630e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5222fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68e5b0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220be00 0x564c522079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68e630e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5222fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35890/0 (socket says 172.21.15.12:35890) 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68e630e700 1 -- 172.21.15.12:0/845098255 learned_addr learned my addr 172.21.15.12:0/845098255 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.726+0000 7f68e6b0f700 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220c720 0x564c52207ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68e630e700 1 -- 172.21.15.12:0/845098255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220c720 msgr2=0x564c52207ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68e630e700 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220c720 0x564c52207ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68e630e700 1 -- 172.21.15.12:0/845098255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220be00 msgr2=0x564c522079a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68e630e700 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220be00 0x564c522079a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68e630e700 1 -- 172.21.15.12:0/845098255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c51f83b60 con 0x564c52206890 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220be00 0x564c522079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68e630e700 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5222fa00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009ce0 tx=0x7f68dc005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61f78d0bfefae03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68c9ffb700 1 -- 172.21.15.12:0/845098255 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc01d040 con 0x564c52206890 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68c9ffb700 1 -- 172.21.15.12:0/845098255 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f68dc004e80 con 0x564c52206890 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68f472cb80 1 -- 172.21.15.12:0/845098255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68d00054e0 con 0x564c52206890 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68c9ffb700 1 -- 172.21.15.12:0/845098255 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68dc02d040 con 0x564c52206890 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68f472cb80 10 client.17178 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68f472cb80 1 -- 172.21.15.12:0/845098255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f68e01a4250 con 0x564c52206890 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.727+0000 7f68c9ffb700 1 -- 172.21.15.12:0/845098255 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68dc049080 con 0x564c52206890 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.728+0000 7f68c9ffb700 1 -- 172.21.15.12:0/845098255 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f68dc03a030 con 0x564c52206890 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.728+0000 7f68f472cb80 10 client.17178 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.728+0000 7f68f472cb80 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.728+0000 7f68f472cb80 20 client.17178 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.728+0000 7f68f472cb80 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564c5227fd90 0x564c52282180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.728+0000 7f68f472cb80 4 client.17178 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:52:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.728+0000 7f68f472cb80 1 -- 172.21.15.12:0/845098255 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c5214eec0 con 0x564c5227fd90 2024-05-08T23:52:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.728+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564c5227fd90 0x564c52282180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:33.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.728+0000 7f68e5b0d700 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564c5227fd90 0x564c52282180 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb56ab6620c61e8c server_cookie=ebb76d800011eaec in_seq=0 out_seq=0 2024-05-08T23:52:33.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.728+0000 7f68c9ffb700 10 client.17178 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:33.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68c9ffb700 1 -- 172.21.15.12:0/845098255 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564c5214eec0 con 0x564c5227fd90 2024-05-08T23:52:33.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68c9ffb700 10 client.17178 handle_command_reply: tid=0 2024-05-08T23:52:33.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 client.17178 shutdown 2024-05-08T23:52:33.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 -- 172.21.15.12:0/845098255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 msgr2=0x564c5222fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5222fa00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009ce0 tx=0x7f68dc005810 comp rx=0 tx=0).stop 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 -- 172.21.15.12:0/845098255 shutdown_connections 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c5220c720 0x564c52207ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564c52206890 0x564c5222fa00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564c5227fd90 0x564c52282180 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 --2- 172.21.15.12:0/845098255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564c5220be00 0x564c522079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68c9ffb700 0 client.17178 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 -- 172.21.15.12:0/845098255 >> 172.21.15.12:0/845098255 conn(0x564c52164210 msgr2=0x564c5221b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 -- 172.21.15.12:0/845098255 shutdown_connections 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 1 -- 172.21.15.12:0/845098255 wait complete. 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.729+0000 7f68f472cb80 20 client.17178 trim_cache size 0 max 16384 2024-05-08T23:52:33.728 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:52:33.729 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:52:33.729 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:52:33.729 INFO:teuthology.orchestra.run.smithi012.stdout: "bc5fadf5-e391-45b2-add5-35eba79cdb28": { 2024-05-08T23:52:33.729 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:52:33.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "bc5fadf5-e391-45b2-add5-35eba79cdb28", 2024-05-08T23:52:33.729 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:52:33.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:52:33.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:52:33.729 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:52:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 -- 172.21.15.12:0/1094040109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68cc0a1290 msgr2=0x7f68cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68cc0a1290 0x7f68cc0a3750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0098e0 tx=0x7f68dc00b650 comp rx=0 tx=0).stop 2024-05-08T23:52:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 -- 172.21.15.12:0/1094040109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68e019bca0 msgr2=0x7f68e019c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68e019bca0 0x7f68e019c120 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f68d800c370 tx=0x7f68d800c680 comp rx=0 tx=0).stop 2024-05-08T23:52:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 -- 172.21.15.12:0/1094040109 shutdown_connections 2024-05-08T23:52:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68cc0a1290 0x7f68cc0a3750 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68e019bca0 0x7f68e019c120 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68e00682e0 0x7f68e01973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 --2- 172.21.15.12:0/1094040109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68e0106290 0x7f68e0197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 -- 172.21.15.12:0/1094040109 >> 172.21.15.12:0/1094040109 conn(0x7f68e0076f40 msgr2=0x7f68e00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 -- 172.21.15.12:0/1094040109 shutdown_connections 2024-05-08T23:52:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:52:33.730+0000 7f68e8572700 1 -- 172.21.15.12:0/1094040109 wait complete. 2024-05-08T23:52:33.739 INFO:tasks.cephfs.filesystem:scrub status for tag:bc5fadf5-e391-45b2-add5-35eba79cdb28 - {'path': '/', 'tag': 'bc5fadf5-e391-45b2-add5-35eba79cdb28', 'options': 'recursive,force'} 2024-05-08T23:52:33.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:33 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:52:33.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:33 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:52:33.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 369252 9.88 MB/sec execute 149 sec latency 16.021 ms 2024-05-08T23:52:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.383+0000 7f5aa3b16700 1 -- 172.21.15.12:0/1262963024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a9c0ffdd0 msgr2=0x7f5a9c100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.383+0000 7f5aa3b16700 1 --2- 172.21.15.12:0/1262963024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a9c0ffdd0 0x7f5a9c100250 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009990 tx=0x7f5a8c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.383+0000 7f5aa3b16700 1 -- 172.21.15.12:0/1262963024 shutdown_connections 2024-05-08T23:52:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.383+0000 7f5aa3b16700 1 --2- 172.21.15.12:0/1262963024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a9c100790 0x7f5a9c10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.383+0000 7f5aa3b16700 1 --2- 172.21.15.12:0/1262963024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a9c105e90 0x7f5a9c106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.383+0000 7f5aa3b16700 1 --2- 172.21.15.12:0/1262963024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a9c0ffdd0 0x7f5a9c100250 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.383+0000 7f5aa3b16700 1 -- 172.21.15.12:0/1262963024 >> 172.21.15.12:0/1262963024 conn(0x7f5a9c076fd0 msgr2=0x7f5a9c0773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.383+0000 7f5aa3b16700 1 -- 172.21.15.12:0/1262963024 shutdown_connections 2024-05-08T23:52:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.383+0000 7f5aa3b16700 1 -- 172.21.15.12:0/1262963024 wait complete. 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa3b16700 1 Processor -- start 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa3b16700 1 -- start start 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa3b16700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a9c0ffdd0 0x7f5a9c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa3b16700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a9c100790 0x7f5a9c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa3b16700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a9c105e90 0x7f5a9c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa3b16700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c19c240 con 0x7f5a9c0ffdd0 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa3b16700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c19c3b0 con 0x7f5a9c105e90 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa3b16700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c19c570 con 0x7f5a9c100790 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa18b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a9c0ffdd0 0x7f5a9c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa18b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a9c0ffdd0 0x7f5a9c197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35910/0 (socket says 172.21.15.12:35910) 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa18b2700 1 -- 172.21.15.12:0/2343890300 learned_addr learned my addr 172.21.15.12:0/2343890300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa10b1700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a9c100790 0x7f5a9c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a9c105e90 0x7f5a9c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa18b2700 1 -- 172.21.15.12:0/2343890300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a9c100790 msgr2=0x7f5a9c197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa18b2700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a9c100790 0x7f5a9c197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa18b2700 1 -- 172.21.15.12:0/2343890300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a9c105e90 msgr2=0x7f5a9c19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa18b2700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a9c105e90 0x7f5a9c19bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.384+0000 7f5aa18b2700 1 -- 172.21.15.12:0/2343890300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0095f0 con 0x7f5a9c0ffdd0 2024-05-08T23:52:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.385+0000 7f5aa18b2700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a9c0ffdd0 0x7f5a9c197320 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800e9a0 tx=0x7f5a9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.385+0000 7f5a92ffd700 1 -- 172.21.15.12:0/2343890300 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98009790 con 0x7f5a9c0ffdd0 2024-05-08T23:52:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.385+0000 7f5a92ffd700 1 -- 172.21.15.12:0/2343890300 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a98010430 con 0x7f5a9c0ffdd0 2024-05-08T23:52:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.385+0000 7f5a92ffd700 1 -- 172.21.15.12:0/2343890300 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98018680 con 0x7f5a9c0ffdd0 2024-05-08T23:52:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.385+0000 7f5aa3b16700 1 -- 172.21.15.12:0/2343890300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c19c850 con 0x7f5a9c0ffdd0 2024-05-08T23:52:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.385+0000 7f5aa3b16700 1 -- 172.21.15.12:0/2343890300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c1a4090 con 0x7f5a9c0ffdd0 2024-05-08T23:52:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.387+0000 7f5a92ffd700 1 -- 172.21.15.12:0/2343890300 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5a980105a0 con 0x7f5a9c0ffdd0 2024-05-08T23:52:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.388+0000 7f5a92ffd700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5a880a1310 0x7f5a880a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.389+0000 7f5aa10b1700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5a880a1310 0x7f5a880a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.389+0000 7f5a92ffd700 1 -- 172.21.15.12:0/2343890300 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5a98014070 con 0x7f5a9c0ffdd0 2024-05-08T23:52:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.389+0000 7f5aa10b1700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5a880a1310 0x7f5a880a37d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c198870 tx=0x7f5a8c005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.389+0000 7f5aafcd0b80 1 Processor -- start 2024-05-08T23:52:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aafcd0b80 1 -- start start 2024-05-08T23:52:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0b540 0x55666be0b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be0f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0fe40 0x55666be102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666bd5e0f0 con 0x55666be0be60 2024-05-08T23:52:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666bd5bab0 con 0x55666be0fe40 2024-05-08T23:52:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666bd573a0 con 0x55666be0b540 2024-05-08T23:52:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa10b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be0f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa10b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be0f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35924/0 (socket says 172.21.15.12:35924) 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa18b2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0b540 0x55666be0b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa10b1700 1 -- 172.21.15.12:0/625533547 learned_addr learned my addr 172.21.15.12:0/625533547 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/625533547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0fe40 0x55666be102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa20b3700 1 -- 172.21.15.12:0/625533547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0b540 msgr2=0x55666be0b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/625533547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0b540 0x55666be0b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa20b3700 1 -- 172.21.15.12:0/625533547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 msgr2=0x55666be0f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/625533547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be0f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa20b3700 1 -- 172.21.15.12:0/625533547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55666bbd8350 con 0x55666be0fe40 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa10b1700 1 --2- 172.21.15.12:0/625533547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be0f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.390+0000 7f5aa18b2700 1 --2- 172.21.15.12:0/625533547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0b540 0x55666be0b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.391+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/625533547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0fe40 0x55666be102a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a94003f00 tx=0x7f5a94009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eda5328c3aa7690 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.391+0000 7f5a87fff700 1 -- 172.21.15.12:0/625533547 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94019070 con 0x55666be0fe40 2024-05-08T23:52:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.391+0000 7f5a87fff700 1 -- 172.21.15.12:0/625533547 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9400f040 con 0x55666be0fe40 2024-05-08T23:52:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.391+0000 7f5a87fff700 1 -- 172.21.15.12:0/625533547 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94007aa0 con 0x55666be0fe40 2024-05-08T23:52:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/625533547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0fe40 msgr2=0x55666be102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/625533547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0fe40 0x55666be102a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a94003f00 tx=0x7f5a94009c20 comp rx=0 tx=0).stop 2024-05-08T23:52:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/625533547 shutdown_connections 2024-05-08T23:52:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/625533547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0fe40 0x55666be102a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/625533547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0b540 0x55666be0b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/625533547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be0f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/625533547 >> 172.21.15.12:0/625533547 conn(0x55666bd651b0 msgr2=0x55666be01840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/625533547 shutdown_connections 2024-05-08T23:52:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/625533547 wait complete. 2024-05-08T23:52:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 -- start start 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.392+0000 7f5aafcd0b80 1 Processor -- start 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0be60 0x55666be14930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0fe40 0x55666be15070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666bcf3500 con 0x55666be0fe40 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666bcf31a0 con 0x55666be0b540 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666bbda640 con 0x55666be0be60 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aafcd0b80 10 client.0 _ll_register_callbacks cb 0x55666bcfcad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aa18b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aa18b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50112/0 (socket says 172.21.15.12:50112) 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aa18b2700 1 -- 172.21.15.12:0/1698885119 learned_addr learned my addr 172.21.15.12:0/1698885119 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0fe40 0x55666be15070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aa10b1700 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0be60 0x55666be14930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aa18b2700 1 -- 172.21.15.12:0/1698885119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0be60 msgr2=0x55666be14930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.393+0000 7f5aa18b2700 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0be60 0x55666be14930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5aa18b2700 1 -- 172.21.15.12:0/1698885119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0fe40 msgr2=0x55666be15070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5aa18b2700 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0fe40 0x55666be15070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5aa18b2700 1 -- 172.21.15.12:0/1698885119 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c10d520 con 0x55666be0b540 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5aa18b2700 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a98096370 tx=0x7f5a9809a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95816810efab2cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5a857fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5a857fa700 1 -- 172.21.15.12:0/1698885119 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a980995b0 con 0x55666be0b540 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5a857fa700 1 -- 172.21.15.12:0/1698885119 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5a9809dcf0 con 0x55666be0b540 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/1698885119 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55666bcb0210 con 0x55666be0b540 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5a857fa700 1 -- 172.21.15.12:0/1698885119 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a980b82d0 con 0x55666be0b540 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5aafcd0b80 10 client.26984 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/1698885119 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f5a94003b60 con 0x55666be0b540 2024-05-08T23:52:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.394+0000 7f5a857fa700 1 -- 172.21.15.12:0/1698885119 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9807c020 con 0x55666be0b540 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.395+0000 7f5a857fa700 1 -- 172.21.15.12:0/1698885119 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5a98098f20 con 0x55666be0b540 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.395+0000 7f5aafcd0b80 10 client.26984 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.395+0000 7f5aafcd0b80 10 client.26984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.395+0000 7f5aafcd0b80 20 client.26984 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.395+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55666be25020 0x55666be27440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.395+0000 7f5aafcd0b80 4 client.26984 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.395+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/1698885119 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55666bdd67b0 con 0x55666be25020 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.395+0000 7f5aa10b1700 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55666be25020 0x55666be27440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.395+0000 7f5a857fa700 10 client.26984 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.395+0000 7f5aa10b1700 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55666be25020 0x55666be27440 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c75c015e354edc27 server_cookie=bc928c9d0053cb00 in_seq=0 out_seq=0 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5a857fa700 1 -- 172.21.15.12:0/1698885119 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55666bdd67b0 con 0x55666be25020 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5a857fa700 10 client.26984 handle_command_reply: tid=0 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 client.26984 shutdown 2024-05-08T23:52:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/1698885119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 msgr2=0x55666be17640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a98096370 tx=0x7f5a9809a640 comp rx=0 tx=0).stop 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/1698885119 shutdown_connections 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0fe40 0x55666be15070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55666be25020 0x55666be27440 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17640 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/1698885119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0be60 0x55666be14930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5a857fa700 0 client.26984 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/1698885119 >> 172.21.15.12:0/1698885119 conn(0x55666bd651b0 msgr2=0x55666be10220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/1698885119 shutdown_connections 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/1698885119 wait complete. 2024-05-08T23:52:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.396+0000 7f5aafcd0b80 20 client.26984 trim_cache size 0 max 16384 2024-05-08T23:52:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.413+0000 7f5aafcd0b80 1 Processor -- start 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.413+0000 7f5aafcd0b80 1 -- start start 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.413+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be0d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.413+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be15030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 0x55666be01910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666be02440 con 0x55666be0be60 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666be0eb90 con 0x55666be0b540 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666be026d0 con 0x55666be0fe40 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa10b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be15030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa10b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be15030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35930/0 (socket says 172.21.15.12:35930) 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa10b1700 1 -- 172.21.15.12:0/2487407008 learned_addr learned my addr 172.21.15.12:0/2487407008 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa18b2700 1 --2- 172.21.15.12:0/2487407008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be0d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/2487407008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 0x55666be01910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa10b1700 1 -- 172.21.15.12:0/2487407008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 msgr2=0x55666be01910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa10b1700 1 --2- 172.21.15.12:0/2487407008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 0x55666be01910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa10b1700 1 -- 172.21.15.12:0/2487407008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 msgr2=0x55666be0d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa10b1700 1 --2- 172.21.15.12:0/2487407008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be0d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa10b1700 1 -- 172.21.15.12:0/2487407008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55666bb84b00 con 0x55666be0be60 2024-05-08T23:52:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5aa10b1700 1 --2- 172.21.15.12:0/2487407008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be15030 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c005f50 tx=0x7f5a8c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903b664f9688a0f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5a90da4700 1 -- 172.21.15.12:0/2487407008 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c005600 con 0x55666be0be60 2024-05-08T23:52:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5a90da4700 1 -- 172.21.15.12:0/2487407008 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a8c01e040 con 0x55666be0be60 2024-05-08T23:52:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.414+0000 7f5a90da4700 1 -- 172.21.15.12:0/2487407008 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c036870 con 0x55666be0be60 2024-05-08T23:52:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.415+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2487407008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 msgr2=0x55666be15030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.415+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/2487407008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be15030 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c005f50 tx=0x7f5a8c023000 comp rx=0 tx=0).stop 2024-05-08T23:52:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.415+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2487407008 shutdown_connections 2024-05-08T23:52:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.415+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/2487407008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 0x55666be01910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.415+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/2487407008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be0d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.415+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/2487407008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be15030 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.415+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2487407008 >> 172.21.15.12:0/2487407008 conn(0x55666bd651b0 msgr2=0x55666be0e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.415+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2487407008 shutdown_connections 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.415+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2487407008 wait complete. 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.416+0000 7f5aafcd0b80 1 -- start start 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.416+0000 7f5aafcd0b80 1 Processor -- start 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.416+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.416+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.416+0000 7f5aafcd0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 0x55666be18a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.416+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666be7a4d0 con 0x55666be0be60 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.416+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666be7a610 con 0x55666be0b540 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.416+0000 7f5aafcd0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666be7a750 con 0x55666be0fe40 2024-05-08T23:52:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aafcd0b80 10 client.0 _ll_register_callbacks cb 0x55666bcfcad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa18b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa10b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa20b3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 0x55666be18a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa18b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50130/0 (socket says 172.21.15.12:50130) 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa10b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be184f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35938/0 (socket says 172.21.15.12:35938) 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa18b2700 1 -- 172.21.15.12:0/2429983163 learned_addr learned my addr 172.21.15.12:0/2429983163 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 0x55666be18a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45670/0 (socket says 172.21.15.12:45670) 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa20b3700 1 -- 172.21.15.12:0/2429983163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 msgr2=0x55666be17fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa20b3700 1 -- 172.21.15.12:0/2429983163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 msgr2=0x55666be184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa20b3700 1 -- 172.21.15.12:0/2429983163 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c00b9f0 con 0x55666be0fe40 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aa20b3700 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 0x55666be18a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5a94007540 tx=0x7f5a94007840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f7e069dd7fe26f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5a857fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5a857fa700 1 -- 172.21.15.12:0/2429983163 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94019040 con 0x55666be0fe40 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5a857fa700 1 -- 172.21.15.12:0/2429983163 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5a94009550 con 0x55666be0fe40 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.417+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2429983163 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a9400f040 con 0x55666be0fe40 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5a857fa700 1 -- 172.21.15.12:0/2429983163 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a94005a10 con 0x55666be0fe40 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5aafcd0b80 10 client.26977 fetch_fsmap learned FSMap version 31 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2429983163 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55666bd1d500 con 0x55666be0fe40 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5a857fa700 1 -- 172.21.15.12:0/2429983163 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9400f040 con 0x55666be0fe40 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5a857fa700 1 -- 172.21.15.12:0/2429983163 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5a94028450 con 0x55666be0fe40 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5aafcd0b80 10 client.26977 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5aafcd0b80 10 client.26977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5aafcd0b80 20 client.26977 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55666be7bdb0 0x55666be7e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5aafcd0b80 4 client.26977 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2429983163 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55666bd4fe60 con 0x55666be7bdb0 2024-05-08T23:52:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.418+0000 7f5aa18b2700 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55666be7bdb0 0x55666be7e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.419+0000 7f5a857fa700 10 client.26977 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.419+0000 7f5aa18b2700 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55666be7bdb0 0x55666be7e1d0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36fe7daa1fdf0cd9 server_cookie=c448e21608c5f630 in_seq=0 out_seq=0 2024-05-08T23:52:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.485+0000 7f5a857fa700 1 -- 172.21.15.12:0/2429983163 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55666bd4fe60 con 0x55666be7bdb0 2024-05-08T23:52:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.485+0000 7f5a857fa700 10 client.26977 handle_command_reply: tid=0 2024-05-08T23:52:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.485+0000 7f5aafcd0b80 1 client.26977 shutdown 2024-05-08T23:52:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.485+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2429983163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 msgr2=0x55666be18a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.485+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 0x55666be18a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5a94007540 tx=0x7f5a94007840 comp rx=0 tx=0).stop 2024-05-08T23:52:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.486+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2429983163 shutdown_connections 2024-05-08T23:52:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.486+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55666be0fe40 0x55666be18a30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.486+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55666be7bdb0 0x55666be7e1d0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.486+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55666be0b540 0x55666be17fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.486+0000 7f5aafcd0b80 1 --2- 172.21.15.12:0/2429983163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55666be0be60 0x55666be184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.486+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2429983163 >> 172.21.15.12:0/2429983163 conn(0x55666bd651b0 msgr2=0x55666be12230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.486+0000 7f5a857fa700 0 client.26977 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.486+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2429983163 shutdown_connections 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.486+0000 7f5aafcd0b80 1 -- 172.21.15.12:0/2429983163 wait complete. 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.486+0000 7f5aafcd0b80 20 client.26977 trim_cache size 0 max 16384 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:52:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:52:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:52:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 -- 172.21.15.12:0/2343890300 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5a880a1310 msgr2=0x7f5a880a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5a880a1310 0x7f5a880a37d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c198870 tx=0x7f5a8c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 -- 172.21.15.12:0/2343890300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a9c0ffdd0 msgr2=0x7f5a9c197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a9c0ffdd0 0x7f5a9c197320 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800e9a0 tx=0x7f5a9800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 -- 172.21.15.12:0/2343890300 shutdown_connections 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5a880a1310 0x7f5a880a37d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a9c105e90 0x7f5a9c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a9c0ffdd0 0x7f5a9c197320 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 --2- 172.21.15.12:0/2343890300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a9c100790 0x7f5a9c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 -- 172.21.15.12:0/2343890300 >> 172.21.15.12:0/2343890300 conn(0x7f5a9c076fd0 msgr2=0x7f5a9c069830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 -- 172.21.15.12:0/2343890300 shutdown_connections 2024-05-08T23:52:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:34.487+0000 7f5aa3b16700 1 -- 172.21.15.12:0/2343890300 wait complete. 2024-05-08T23:52:34.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: pgmap v189: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:52:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:34 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: pgmap v189: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:52:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:34 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:52:34.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 370504 9.85 MB/sec execute 150 sec latency 29.937 ms 2024-05-08T23:52:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: pgmap v189: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:52:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:34 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:52:35.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 372652 9.87 MB/sec execute 151 sec latency 16.823 ms 2024-05-08T23:52:36.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:36 smithi204 ceph-mon[47373]: pgmap v190: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-05-08T23:52:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:36 smithi160 ceph-mon[46152]: pgmap v190: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-05-08T23:52:36.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 373990 9.85 MB/sec execute 152 sec latency 19.184 ms 2024-05-08T23:52:36.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:36 smithi012 ceph-mon[42503]: pgmap v190: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-05-08T23:52:37.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:37 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:52:37.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:37 smithi204 ceph-mon[47373]: pgmap v191: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 367 op/s 2024-05-08T23:52:37.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:37 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:52:37.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:37 smithi160 ceph-mon[46152]: pgmap v191: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 367 op/s 2024-05-08T23:52:37.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 376059 9.86 MB/sec execute 153 sec latency 118.544 ms 2024-05-08T23:52:37.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:37 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:52:37.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:37 smithi012 ceph-mon[42503]: pgmap v191: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 367 op/s 2024-05-08T23:52:38.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 377427 9.84 MB/sec execute 154 sec latency 32.791 ms 2024-05-08T23:52:39.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 379807 9.87 MB/sec execute 155 sec latency 25.339 ms 2024-05-08T23:52:40.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:40 smithi204 ceph-mon[47373]: pgmap v192: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 256 op/s 2024-05-08T23:52:40.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:40 smithi160 ceph-mon[46152]: pgmap v192: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 256 op/s 2024-05-08T23:52:40.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 381184 9.84 MB/sec execute 156 sec latency 18.022 ms 2024-05-08T23:52:40.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:40 smithi012 ceph-mon[42503]: pgmap v192: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 256 op/s 2024-05-08T23:52:41.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 383340 9.86 MB/sec execute 157 sec latency 15.100 ms 2024-05-08T23:52:42.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:42 smithi204 ceph-mon[47373]: pgmap v193: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 365 op/s 2024-05-08T23:52:42.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:42 smithi160 ceph-mon[46152]: pgmap v193: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 365 op/s 2024-05-08T23:52:42.867 INFO:tasks.workunit.client.0.smithi012.stdout: 1 384663 9.84 MB/sec execute 158 sec latency 44.861 ms 2024-05-08T23:52:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:42 smithi012 ceph-mon[42503]: pgmap v193: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 365 op/s 2024-05-08T23:52:43.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:43 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:52:43.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:43 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:52:43.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 386901 9.86 MB/sec execute 159 sec latency 16.044 ms 2024-05-08T23:52:43.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:43 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:52:44.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:44 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:52:44.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:44 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:52:44.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:44 smithi204 ceph-mon[47373]: pgmap v194: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-05-08T23:52:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:44 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:52:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:44 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:52:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:44 smithi160 ceph-mon[46152]: pgmap v194: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-05-08T23:52:44.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 388172 9.83 MB/sec execute 160 sec latency 19.169 ms 2024-05-08T23:52:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:44 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:52:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:44 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:52:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:44 smithi012 ceph-mon[42503]: pgmap v194: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-05-08T23:52:45.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 390308 9.85 MB/sec execute 161 sec latency 13.060 ms 2024-05-08T23:52:46.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:46 smithi204 ceph-mon[47373]: pgmap v195: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-05-08T23:52:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:46 smithi160 ceph-mon[46152]: pgmap v195: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-05-08T23:52:46.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 391672 9.83 MB/sec execute 162 sec latency 21.656 ms 2024-05-08T23:52:46.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:46 smithi012 ceph-mon[42503]: pgmap v195: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-05-08T23:52:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.620+0000 7f72d61c3700 1 -- 172.21.15.12:0/1366223230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72d00ffda0 msgr2=0x7f72d0100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.620+0000 7f72d61c3700 1 --2- 172.21.15.12:0/1366223230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72d00ffda0 0x7f72d0100220 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009990 tx=0x7f72c4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.620+0000 7f72d61c3700 1 -- 172.21.15.12:0/1366223230 shutdown_connections 2024-05-08T23:52:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.620+0000 7f72d61c3700 1 --2- 172.21.15.12:0/1366223230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72d0100760 0x7f72d010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.620+0000 7f72d61c3700 1 --2- 172.21.15.12:0/1366223230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d0105e60 0x7f72d0106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.620+0000 7f72d61c3700 1 --2- 172.21.15.12:0/1366223230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72d00ffda0 0x7f72d0100220 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.620+0000 7f72d61c3700 1 -- 172.21.15.12:0/1366223230 >> 172.21.15.12:0/1366223230 conn(0x7f72d0076f10 msgr2=0x7f72d0077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.620+0000 7f72d61c3700 1 -- 172.21.15.12:0/1366223230 shutdown_connections 2024-05-08T23:52:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.620+0000 7f72d61c3700 1 -- 172.21.15.12:0/1366223230 wait complete. 2024-05-08T23:52:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.620+0000 7f72d61c3700 1 Processor -- start 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72d61c3700 1 -- start start 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72d61c3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72d00ffda0 0x7f72d0198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72d61c3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72d0100760 0x7f72d0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72d61c3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d0105e60 0x7f72d01a3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72d61c3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d0198ea0 con 0x7f72d0100760 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72d61c3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d01a36d0 con 0x7f72d00ffda0 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72d61c3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d01a3810 con 0x7f72d0105e60 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72ceffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72d0100760 0x7f72d0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72ceffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72d0100760 0x7f72d0198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36236/0 (socket says 172.21.15.12:36236) 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72ceffd700 1 -- 172.21.15.12:0/153802344 learned_addr learned my addr 172.21.15.12:0/153802344 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72cffff700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d0105e60 0x7f72d01a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72d00ffda0 0x7f72d0198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72ceffd700 1 -- 172.21.15.12:0/153802344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d0105e60 msgr2=0x7f72d01a3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72ceffd700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d0105e60 0x7f72d01a3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72ceffd700 1 -- 172.21.15.12:0/153802344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72d00ffda0 msgr2=0x7f72d0198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72ceffd700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72d00ffda0 0x7f72d0198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.621+0000 7f72ceffd700 1 -- 172.21.15.12:0/153802344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c40095f0 con 0x7f72d0100760 2024-05-08T23:52:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.622+0000 7f72ceffd700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72d0100760 0x7f72d0198650 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f72c40092e0 tx=0x7f72c400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.622+0000 7f72ccff9700 1 -- 172.21.15.12:0/153802344 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c401d070 con 0x7f72d0100760 2024-05-08T23:52:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.622+0000 7f72d61c3700 1 -- 172.21.15.12:0/153802344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d01a3a10 con 0x7f72d0100760 2024-05-08T23:52:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.622+0000 7f72d61c3700 1 -- 172.21.15.12:0/153802344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d01a3f10 con 0x7f72d0100760 2024-05-08T23:52:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.624+0000 7f72ccff9700 1 -- 172.21.15.12:0/153802344 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72c4021410 con 0x7f72d0100760 2024-05-08T23:52:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.624+0000 7f72ccff9700 1 -- 172.21.15.12:0/153802344 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4017650 con 0x7f72d0100760 2024-05-08T23:52:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.625+0000 7f72ccff9700 1 -- 172.21.15.12:0/153802344 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f72c4017830 con 0x7f72d0100760 2024-05-08T23:52:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.626+0000 7f72ccff9700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72b80a1310 0x7f72b80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.626+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72b80a1310 0x7f72b80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.626+0000 7f72ccff9700 1 -- 172.21.15.12:0/153802344 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f72c40cdcf0 con 0x7f72d0100760 2024-05-08T23:52:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.626+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72b80a1310 0x7f72b80a37d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009380 tx=0x7f72c000b040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.627+0000 7f72e237db80 1 Processor -- start 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.627+0000 7f72e237db80 1 -- start start 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.627+0000 7f72e237db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 0x56529c6cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.627+0000 7f72e237db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cfe60 0x56529c6d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.627+0000 7f72e237db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 0x56529c6d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.627+0000 7f72e237db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c6220f0 con 0x56529c6cfe60 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.627+0000 7f72e237db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c61fab0 con 0x56529c6cf540 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.627+0000 7f72e237db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c61b3a0 con 0x56529c6d3e40 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72ceffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cfe60 0x56529c6d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72ceffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cfe60 0x56529c6d3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36240/0 (socket says 172.21.15.12:36240) 2024-05-08T23:52:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72ceffd700 1 -- 172.21.15.12:0/1462554916 learned_addr learned my addr 172.21.15.12:0/1462554916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72cffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 0x56529c6d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/1462554916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 0x56529c6cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72ceffd700 1 -- 172.21.15.12:0/1462554916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 msgr2=0x56529c6d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72ceffd700 1 --2- 172.21.15.12:0/1462554916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 0x56529c6d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72ceffd700 1 -- 172.21.15.12:0/1462554916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 msgr2=0x56529c6cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72ceffd700 1 --2- 172.21.15.12:0/1462554916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 0x56529c6cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72ceffd700 1 -- 172.21.15.12:0/1462554916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56529c49c350 con 0x56529c6cfe60 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72cffff700 1 --2- 172.21.15.12:0/1462554916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 0x56529c6d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72ceffd700 1 --2- 172.21.15.12:0/1462554916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cfe60 0x56529c6d3900 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f72c409bf90 tx=0x7f72c409bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c580fb2297ce9a8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72b5ffb700 1 -- 172.21.15.12:0/1462554916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c409c700 con 0x56529c6cfe60 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72b5ffb700 1 -- 172.21.15.12:0/1462554916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72c409c000 con 0x56529c6cfe60 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.628+0000 7f72b5ffb700 1 -- 172.21.15.12:0/1462554916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c409b4f0 con 0x56529c6cfe60 2024-05-08T23:52:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.629+0000 7f72e237db80 1 -- 172.21.15.12:0/1462554916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cfe60 msgr2=0x56529c6d3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.629+0000 7f72e237db80 1 --2- 172.21.15.12:0/1462554916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cfe60 0x56529c6d3900 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f72c409bf90 tx=0x7f72c409bee0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.629+0000 7f72e237db80 1 -- 172.21.15.12:0/1462554916 shutdown_connections 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.629+0000 7f72e237db80 1 --2- 172.21.15.12:0/1462554916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 0x56529c6d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.629+0000 7f72e237db80 1 --2- 172.21.15.12:0/1462554916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 0x56529c6cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.629+0000 7f72e237db80 1 --2- 172.21.15.12:0/1462554916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cfe60 0x56529c6d3900 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.629+0000 7f72e237db80 1 -- 172.21.15.12:0/1462554916 >> 172.21.15.12:0/1462554916 conn(0x56529c6291b0 msgr2=0x56529c6c5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.629+0000 7f72e237db80 1 -- 172.21.15.12:0/1462554916 shutdown_connections 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.629+0000 7f72e237db80 1 -- 172.21.15.12:0/1462554916 wait complete. 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72e237db80 1 -- start start 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72e237db80 1 Processor -- start 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72e237db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 0x56529c6d8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72e237db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6d8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72e237db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6d3e40 0x56529c6d9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72e237db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c5b7500 con 0x56529c6d3e40 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72e237db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c5b71a0 con 0x56529c6cf540 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72e237db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c49e640 con 0x56529c6cfe60 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72e237db80 10 client.0 _ll_register_callbacks cb 0x56529c5c0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72cf7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 0x56529c6d8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72ceffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6d8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72cf7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 0x56529c6d8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:37844/0 (socket says 172.21.15.12:37844) 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72cf7fe700 1 -- 172.21.15.12:0/4243549098 learned_addr learned my addr 172.21.15.12:0/4243549098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.630+0000 7f72cffff700 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6d3e40 0x56529c6d9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72cf7fe700 1 -- 172.21.15.12:0/4243549098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 msgr2=0x56529c6d8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6d8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72cf7fe700 1 -- 172.21.15.12:0/4243549098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6d3e40 msgr2=0x56529c6d9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6d3e40 0x56529c6d9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72cf7fe700 1 -- 172.21.15.12:0/4243549098 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c80098c0 con 0x56529c6cf540 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72ceffd700 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6d8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72cffff700 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6d3e40 0x56529c6d9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 0x56529c6d8990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f72c0015e40 tx=0x7f72c0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa1f21971b3546e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72aeffd700 1 -- 172.21.15.12:0/4243549098 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c000e050 con 0x56529c6cf540 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72aeffd700 1 -- 172.21.15.12:0/4243549098 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f72c0003a00 con 0x56529c6cf540 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72e237db80 1 -- 172.21.15.12:0/4243549098 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56529c574210 con 0x56529c6cf540 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72aeffd700 1 -- 172.21.15.12:0/4243549098 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72c0007e90 con 0x56529c6cf540 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.631+0000 7f72e237db80 10 client.27002 fetch_fsmap learned FSMap version 32 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.632+0000 7f72e237db80 1 -- 172.21.15.12:0/4243549098 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f72d01a3f10 con 0x56529c6cf540 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.632+0000 7f72aeffd700 1 -- 172.21.15.12:0/4243549098 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56529c574210 con 0x56529c6cf540 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.632+0000 7f72aeffd700 1 -- 172.21.15.12:0/4243549098 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f72c00223f0 con 0x56529c6cf540 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.632+0000 7f72e237db80 10 client.27002 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:52:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.632+0000 7f72e237db80 10 client.27002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.632+0000 7f72e237db80 20 client.27002 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.632+0000 7f72e237db80 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56529c6e8de0 0x56529c6eb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.632+0000 7f72e237db80 4 client.27002 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.632+0000 7f72e237db80 1 -- 172.21.15.12:0/4243549098 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56529c69a7b0 con 0x56529c6e8de0 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.632+0000 7f72ceffd700 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56529c6e8de0 0x56529c6eb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.633+0000 7f72ceffd700 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56529c6e8de0 0x56529c6eb200 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7437259950302a53 server_cookie=6f4f1fb72a1b8a51 in_seq=0 out_seq=0 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.633+0000 7f72aeffd700 10 client.27002 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72aeffd700 1 -- 172.21.15.12:0/4243549098 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56529c69a7b0 con 0x56529c6e8de0 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72aeffd700 10 client.27002 handle_command_reply: tid=0 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 client.27002 shutdown 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 -- 172.21.15.12:0/4243549098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 msgr2=0x56529c6d8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 0x56529c6d8990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f72c0015e40 tx=0x7f72c0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 -- 172.21.15.12:0/4243549098 shutdown_connections 2024-05-08T23:52:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6d3e40 0x56529c6d9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56529c6e8de0 0x56529c6eb200 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cf540 0x56529c6d8990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72aeffd700 0 client.27002 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 --2- 172.21.15.12:0/4243549098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6d8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 -- 172.21.15.12:0/4243549098 >> 172.21.15.12:0/4243549098 conn(0x56529c6291b0 msgr2=0x56529c6c9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 -- 172.21.15.12:0/4243549098 shutdown_connections 2024-05-08T23:52:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 1 -- 172.21.15.12:0/4243549098 wait complete. 2024-05-08T23:52:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.634+0000 7f72e237db80 20 client.27002 trim_cache size 0 max 16384 2024-05-08T23:52:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72e237db80 1 Processor -- start 2024-05-08T23:52:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72e237db80 1 -- start start 2024-05-08T23:52:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72e237db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72e237db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cfe60 0x56529c6c5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72e237db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 0x56529c6d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72e237db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c6d4760 con 0x56529c6cf540 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72e237db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c6db9d0 con 0x56529c6cfe60 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72e237db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c6d6ee0 con 0x56529c6d3e40 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72cf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72cf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36260/0 (socket says 172.21.15.12:36260) 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72ceffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cfe60 0x56529c6c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72cf7fe700 1 -- 172.21.15.12:0/2529556786 learned_addr learned my addr 172.21.15.12:0/2529556786 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.651+0000 7f72cffff700 1 --2- 172.21.15.12:0/2529556786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 0x56529c6d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.652+0000 7f72ceffd700 1 -- 172.21.15.12:0/2529556786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 msgr2=0x56529c6d4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.652+0000 7f72ceffd700 1 --2- 172.21.15.12:0/2529556786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 0x56529c6d4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.652+0000 7f72ceffd700 1 -- 172.21.15.12:0/2529556786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 msgr2=0x56529c6c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.652+0000 7f72ceffd700 1 --2- 172.21.15.12:0/2529556786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.652+0000 7f72ceffd700 1 -- 172.21.15.12:0/2529556786 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56529c448b00 con 0x56529c6cfe60 2024-05-08T23:52:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.652+0000 7f72ceffd700 1 --2- 172.21.15.12:0/2529556786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cfe60 0x56529c6c5910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x56529c6d0330 tx=0x7f72c4098470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c368350ecf5bd30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.652+0000 7f72b67fc700 1 -- 172.21.15.12:0/2529556786 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c409c780 con 0x56529c6cfe60 2024-05-08T23:52:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.652+0000 7f72b67fc700 1 -- 172.21.15.12:0/2529556786 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72c409fe30 con 0x56529c6cfe60 2024-05-08T23:52:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.652+0000 7f72b67fc700 1 -- 172.21.15.12:0/2529556786 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c40cbbd0 con 0x56529c6cfe60 2024-05-08T23:52:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.653+0000 7f72e237db80 1 -- 172.21.15.12:0/2529556786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cfe60 msgr2=0x56529c6c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.653+0000 7f72e237db80 1 --2- 172.21.15.12:0/2529556786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cfe60 0x56529c6c5910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x56529c6d0330 tx=0x7f72c4098470 comp rx=0 tx=0).stop 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.653+0000 7f72e237db80 1 -- 172.21.15.12:0/2529556786 shutdown_connections 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.653+0000 7f72e237db80 1 --2- 172.21.15.12:0/2529556786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6d3e40 0x56529c6d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.653+0000 7f72e237db80 1 --2- 172.21.15.12:0/2529556786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.653+0000 7f72e237db80 1 --2- 172.21.15.12:0/2529556786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6cfe60 0x56529c6c5910 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.653+0000 7f72e237db80 1 -- 172.21.15.12:0/2529556786 >> 172.21.15.12:0/2529556786 conn(0x56529c6291b0 msgr2=0x56529c6d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.653+0000 7f72e237db80 1 -- 172.21.15.12:0/2529556786 shutdown_connections 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.653+0000 7f72e237db80 1 -- 172.21.15.12:0/2529556786 wait complete. 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72e237db80 1 -- start start 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72e237db80 1 Processor -- start 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72e237db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6dfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72e237db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6e0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72e237db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6d3e40 0x56529c6dde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72e237db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c6de4e0 con 0x56529c6cf540 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72e237db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c6de650 con 0x56529c6d3e40 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72e237db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529c6de810 con 0x56529c6cfe60 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72e237db80 10 client.0 _ll_register_callbacks cb 0x56529c5c0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72cf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6dfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72ceffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6e0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72ceffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6e0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47990/0 (socket says 172.21.15.12:47990) 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72ceffd700 1 -- 172.21.15.12:0/4261690266 learned_addr learned my addr 172.21.15.12:0/4261690266 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72cffff700 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6d3e40 0x56529c6dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.654+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6dfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36276/0 (socket says 172.21.15.12:36276) 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72ceffd700 1 -- 172.21.15.12:0/4261690266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6d3e40 msgr2=0x56529c6dde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72ceffd700 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6d3e40 0x56529c6dde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72ceffd700 1 -- 172.21.15.12:0/4261690266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 msgr2=0x56529c6dfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72ceffd700 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6dfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72ceffd700 1 -- 172.21.15.12:0/4261690266 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56529c49c350 con 0x56529c6cfe60 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6dfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72ceffd700 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6e0290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72c40a3ec0 tx=0x7f72c409be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0d4862e5e2492e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72aeffd700 1 -- 172.21.15.12:0/4261690266 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c40a43f0 con 0x56529c6cfe60 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72aeffd700 1 -- 172.21.15.12:0/4261690266 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f72c40b9280 con 0x56529c6cfe60 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72e237db80 1 -- 172.21.15.12:0/4261690266 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72c409c000 con 0x56529c6cfe60 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72aeffd700 1 -- 172.21.15.12:0/4261690266 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72c409c260 con 0x56529c6cfe60 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72e237db80 10 client.26995 fetch_fsmap learned FSMap version 32 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.655+0000 7f72e237db80 1 -- 172.21.15.12:0/4261690266 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56529c5e1500 con 0x56529c6cfe60 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.656+0000 7f72aeffd700 1 -- 172.21.15.12:0/4261690266 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72c409c000 con 0x56529c6cfe60 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.656+0000 7f72aeffd700 1 -- 172.21.15.12:0/4261690266 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f72c40a3f60 con 0x56529c6cfe60 2024-05-08T23:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.656+0000 7f72e237db80 10 client.26995 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.656+0000 7f72e237db80 10 client.26995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.656+0000 7f72e237db80 20 client.26995 populate_metadata read hostname 'smithi012' 2024-05-08T23:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.656+0000 7f72e237db80 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56529c73f0b0 0x56529c7414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.656+0000 7f72e237db80 4 client.26995 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.656+0000 7f72e237db80 1 -- 172.21.15.12:0/4261690266 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56529c613e60 con 0x56529c73f0b0 2024-05-08T23:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.656+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56529c73f0b0 0x56529c7414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.657+0000 7f72aeffd700 10 client.26995 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.657+0000 7f72cf7fe700 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56529c73f0b0 0x56529c7414a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4370e9a64666045 server_cookie=95181772ecf904b8 in_seq=0 out_seq=0 2024-05-08T23:52:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72aeffd700 1 -- 172.21.15.12:0/4261690266 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56529c613e60 con 0x56529c73f0b0 2024-05-08T23:52:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72aeffd700 10 client.26995 handle_command_reply: tid=0 2024-05-08T23:52:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 client.26995 shutdown 2024-05-08T23:52:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 -- 172.21.15.12:0/4261690266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 msgr2=0x56529c6e0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6e0290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72c40a3ec0 tx=0x7f72c409be80 comp rx=0 tx=0).stop 2024-05-08T23:52:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 -- 172.21.15.12:0/4261690266 shutdown_connections 2024-05-08T23:52:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56529c73f0b0 0x56529c7414a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56529c6d3e40 0x56529c6dde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56529c6cf540 0x56529c6dfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 --2- 172.21.15.12:0/4261690266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56529c6cfe60 0x56529c6e0290 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 -- 172.21.15.12:0/4261690266 >> 172.21.15.12:0/4261690266 conn(0x56529c6291b0 msgr2=0x56529c6d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72aeffd700 0 client.26995 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:52:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 -- 172.21.15.12:0/4261690266 shutdown_connections 2024-05-08T23:52:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.718+0000 7f72e237db80 1 -- 172.21.15.12:0/4261690266 wait complete. 2024-05-08T23:52:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.719+0000 7f72e237db80 20 client.26995 trim_cache size 0 max 16384 2024-05-08T23:52:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:52:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:52:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:52:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:52:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.719+0000 7f72d61c3700 1 -- 172.21.15.12:0/153802344 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72b80a1310 msgr2=0x7f72b80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.719+0000 7f72d61c3700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72b80a1310 0x7f72b80a37d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009380 tx=0x7f72c000b040 comp rx=0 tx=0).stop 2024-05-08T23:52:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.719+0000 7f72d61c3700 1 -- 172.21.15.12:0/153802344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72d0100760 msgr2=0x7f72d0198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:52:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.719+0000 7f72d61c3700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72d0100760 0x7f72d0198650 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f72c40092e0 tx=0x7f72c400bfd0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.720+0000 7f72d61c3700 1 -- 172.21.15.12:0/153802344 shutdown_connections 2024-05-08T23:52:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.720+0000 7f72d61c3700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72b80a1310 0x7f72b80a37d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.720+0000 7f72d61c3700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72d0105e60 0x7f72d01a3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.720+0000 7f72d61c3700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72d00ffda0 0x7f72d0198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.720+0000 7f72d61c3700 1 --2- 172.21.15.12:0/153802344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72d0100760 0x7f72d0198650 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:52:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.720+0000 7f72d61c3700 1 -- 172.21.15.12:0/153802344 >> 172.21.15.12:0/153802344 conn(0x7f72d0076f10 msgr2=0x7f72d00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:52:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.720+0000 7f72d61c3700 1 -- 172.21.15.12:0/153802344 shutdown_connections 2024-05-08T23:52:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:52:47.720+0000 7f72d61c3700 1 -- 172.21.15.12:0/153802344 wait complete. 2024-05-08T23:52:47.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393223 9.81 MB/sec execute 163 sec latency 45.347 ms 2024-05-08T23:52:48.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:48 smithi204 ceph-mon[47373]: pgmap v196: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 334 op/s 2024-05-08T23:52:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:48 smithi160 ceph-mon[46152]: pgmap v196: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 334 op/s 2024-05-08T23:52:48.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 394674 9.80 MB/sec execute 164 sec latency 14.967 ms 2024-05-08T23:52:48.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:48 smithi012 ceph-mon[42503]: pgmap v196: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 334 op/s 2024-05-08T23:52:49.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 397383 9.84 MB/sec execute 165 sec latency 16.044 ms 2024-05-08T23:52:50.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:50 smithi204 ceph-mon[47373]: pgmap v197: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-05-08T23:52:50.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:50 smithi160 ceph-mon[46152]: pgmap v197: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-05-08T23:52:50.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 398744 9.82 MB/sec execute 166 sec latency 14.949 ms 2024-05-08T23:52:50.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:50 smithi012 ceph-mon[42503]: pgmap v197: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-05-08T23:52:51.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 400822 9.84 MB/sec execute 167 sec latency 17.244 ms 2024-05-08T23:52:52.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:52 smithi204 ceph-mon[47373]: pgmap v198: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-05-08T23:52:52.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:52 smithi160 ceph-mon[46152]: pgmap v198: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-05-08T23:52:52.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 402000 9.80 MB/sec execute 168 sec latency 17.423 ms 2024-05-08T23:52:52.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:52 smithi012 ceph-mon[42503]: pgmap v198: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-05-08T23:52:53.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 404509 9.84 MB/sec execute 169 sec latency 25.688 ms 2024-05-08T23:52:54.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:54 smithi204 ceph-mon[47373]: pgmap v199: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-05-08T23:52:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:54 smithi160 ceph-mon[46152]: pgmap v199: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-05-08T23:52:54.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 405958 9.82 MB/sec execute 170 sec latency 14.629 ms 2024-05-08T23:52:54.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:54 smithi012 ceph-mon[42503]: pgmap v199: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-05-08T23:52:55.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:55 smithi204 ceph-mon[47373]: pgmap v200: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-05-08T23:52:55.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:55 smithi160 ceph-mon[46152]: pgmap v200: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-05-08T23:52:55.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 408225 9.84 MB/sec execute 171 sec latency 16.811 ms 2024-05-08T23:52:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:55 smithi012 ceph-mon[42503]: pgmap v200: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-05-08T23:52:56.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 409569 9.81 MB/sec execute 172 sec latency 17.028 ms 2024-05-08T23:52:57.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 411731 9.84 MB/sec execute 173 sec latency 12.996 ms 2024-05-08T23:52:58.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:52:58 smithi204 ceph-mon[47373]: pgmap v201: 97 pgs: 97 active+clean; 117 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-05-08T23:52:58.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:52:58 smithi160 ceph-mon[46152]: pgmap v201: 97 pgs: 97 active+clean; 117 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-05-08T23:52:58.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 413184 9.81 MB/sec execute 174 sec latency 33.162 ms 2024-05-08T23:52:58.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:52:58 smithi012 ceph-mon[42503]: pgmap v201: 97 pgs: 97 active+clean; 117 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-05-08T23:52:59.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 415341 9.83 MB/sec execute 175 sec latency 50.648 ms 2024-05-08T23:53:00.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:00 smithi204 ceph-mon[47373]: pgmap v202: 97 pgs: 97 active+clean; 117 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 222 op/s 2024-05-08T23:53:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:00 smithi160 ceph-mon[46152]: pgmap v202: 97 pgs: 97 active+clean; 117 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 222 op/s 2024-05-08T23:53:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.850+0000 7f155d364700 1 -- 172.21.15.12:0/500006020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1558069730 msgr2=0x7f1558069b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.850+0000 7f155d364700 1 --2- 172.21.15.12:0/500006020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1558069730 0x7f1558069b90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f154c0098f0 tx=0x7f154c009c00 comp rx=0 tx=0).stop 2024-05-08T23:53:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.850+0000 7f155d364700 1 -- 172.21.15.12:0/500006020 shutdown_connections 2024-05-08T23:53:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.850+0000 7f155d364700 1 --2- 172.21.15.12:0/500006020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f155806a0d0 0x7f155810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.850+0000 7f155d364700 1 --2- 172.21.15.12:0/500006020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1558106f50 0x7f1558107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.850+0000 7f155d364700 1 --2- 172.21.15.12:0/500006020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1558069730 0x7f1558069b90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.850+0000 7f155d364700 1 -- 172.21.15.12:0/500006020 >> 172.21.15.12:0/500006020 conn(0x7f1558076fd0 msgr2=0x7f15580773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.850+0000 7f155d364700 1 -- 172.21.15.12:0/500006020 shutdown_connections 2024-05-08T23:53:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.850+0000 7f155d364700 1 -- 172.21.15.12:0/500006020 wait complete. 2024-05-08T23:53:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.850+0000 7f155d364700 1 Processor -- start 2024-05-08T23:53:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f155d364700 1 -- start start 2024-05-08T23:53:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f155d364700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1558069730 0x7f1558197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f155d364700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f155806a0d0 0x7f15581978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f155d364700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1558106f50 0x7f155819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f155d364700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155819c280 con 0x7f1558106f50 2024-05-08T23:53:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f155d364700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155819c3f0 con 0x7f155806a0d0 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f155d364700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155819c5b0 con 0x7f1558069730 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f15567fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f155806a0d0 0x7f15581978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f15577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1558106f50 0x7f155819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f15567fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f155806a0d0 0x7f15581978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:38946/0 (socket says 172.21.15.12:38946) 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f15567fc700 1 -- 172.21.15.12:0/3234351985 learned_addr learned my addr 172.21.15.12:0/3234351985 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.851+0000 7f1556ffd700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1558069730 0x7f1558197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f15577fe700 1 -- 172.21.15.12:0/3234351985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1558069730 msgr2=0x7f1558197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f15577fe700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1558069730 0x7f1558197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f15577fe700 1 -- 172.21.15.12:0/3234351985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f155806a0d0 msgr2=0x7f15581978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f15577fe700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f155806a0d0 0x7f15581978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f15577fe700 1 -- 172.21.15.12:0/3234351985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c009520 con 0x7f1558106f50 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f15567fc700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f155806a0d0 0x7f15581978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f15577fe700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1558106f50 0x7f155819bc40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f155000efc0 tx=0x7f155000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f153ffff700 1 -- 172.21.15.12:0/3234351985 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550018070 con 0x7f1558106f50 2024-05-08T23:53:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f155d364700 1 -- 172.21.15.12:0/3234351985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155819c830 con 0x7f1558106f50 2024-05-08T23:53:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f153ffff700 1 -- 172.21.15.12:0/3234351985 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f155000f040 con 0x7f1558106f50 2024-05-08T23:53:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f153ffff700 1 -- 172.21.15.12:0/3234351985 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155000cbb0 con 0x7f1558106f50 2024-05-08T23:53:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.852+0000 7f155d364700 1 -- 172.21.15.12:0/3234351985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15581a40d0 con 0x7f1558106f50 2024-05-08T23:53:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.854+0000 7f153ffff700 1 -- 172.21.15.12:0/3234351985 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1550003a00 con 0x7f1558106f50 2024-05-08T23:53:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.855+0000 7f153ffff700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15400a1290 0x7f15400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.855+0000 7f1556ffd700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15400a1290 0x7f15400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.855+0000 7f153ffff700 1 -- 172.21.15.12:0/3234351985 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f15500cc890 con 0x7f1558106f50 2024-05-08T23:53:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.855+0000 7f1556ffd700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15400a1290 0x7f15400a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1548005fd0 tx=0x7f1548009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f156951eb80 1 Processor -- start 2024-05-08T23:53:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f156951eb80 1 -- start start 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f156951eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2856950 0x5581d2856d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f156951eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 0x5581d2850f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f156951eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d28518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f156951eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d27a80f0 con 0x5581d2851480 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f156951eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d27a5ab0 con 0x5581d2856950 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f156951eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d27a13a0 con 0x5581d2857270 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f15577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d28518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2856950 0x5581d2856d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f15577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d28518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55806/0 (socket says 172.21.15.12:55806) 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2856950 0x5581d2856d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:38954/0 (socket says 172.21.15.12:38954) 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f15577fe700 1 -- 172.21.15.12:0/3789570624 learned_addr learned my addr 172.21.15.12:0/3789570624 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.856+0000 7f15567fc700 1 --2- 172.21.15.12:0/3789570624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 0x5581d2850f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.857+0000 7f15577fe700 1 -- 172.21.15.12:0/3789570624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 msgr2=0x5581d2850f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.857+0000 7f15577fe700 1 --2- 172.21.15.12:0/3789570624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 0x5581d2850f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.857+0000 7f15577fe700 1 -- 172.21.15.12:0/3789570624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2856950 msgr2=0x5581d2856d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.857+0000 7f15577fe700 1 --2- 172.21.15.12:0/3789570624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2856950 0x5581d2856d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.857+0000 7f15577fe700 1 -- 172.21.15.12:0/3789570624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581d2622350 con 0x5581d2851480 2024-05-08T23:53:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.857+0000 7f15577fe700 1 --2- 172.21.15.12:0/3789570624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d28518a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f15500090a0 tx=0x7f15500992d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959c8d395b87a421 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.857+0000 7f153d7fa700 1 -- 172.21.15.12:0/3789570624 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155009a0b0 con 0x5581d2851480 2024-05-08T23:53:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.857+0000 7f153d7fa700 1 -- 172.21.15.12:0/3789570624 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1550099970 con 0x5581d2851480 2024-05-08T23:53:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.857+0000 7f153d7fa700 1 -- 172.21.15.12:0/3789570624 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550099e30 con 0x5581d2851480 2024-05-08T23:53:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 -- 172.21.15.12:0/3789570624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 msgr2=0x5581d28518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 --2- 172.21.15.12:0/3789570624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d28518a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f15500090a0 tx=0x7f15500992d0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 -- 172.21.15.12:0/3789570624 shutdown_connections 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 --2- 172.21.15.12:0/3789570624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d28518a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 --2- 172.21.15.12:0/3789570624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2856950 0x5581d2856d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 --2- 172.21.15.12:0/3789570624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 0x5581d2850f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 -- 172.21.15.12:0/3789570624 >> 172.21.15.12:0/3789570624 conn(0x5581d27af1b0 msgr2=0x5581d284b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 -- 172.21.15.12:0/3789570624 shutdown_connections 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 -- 172.21.15.12:0/3789570624 wait complete. 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 -- start start 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.858+0000 7f156951eb80 1 Processor -- start 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f156951eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2851480 0x5581d285d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f156951eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d285a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f156951eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 0x5581d285acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f156951eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d273d500 con 0x5581d2856950 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f156951eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d273d1a0 con 0x5581d2851480 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f156951eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d2624640 con 0x5581d2857270 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f156951eb80 10 client.0 _ll_register_callbacks cb 0x5581d2746ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15567fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d285a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2851480 0x5581d285d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15567fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d285a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55810/0 (socket says 172.21.15.12:55810) 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15567fc700 1 -- 172.21.15.12:0/2804467247 learned_addr learned my addr 172.21.15.12:0/2804467247 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15577fe700 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 0x5581d285acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15567fc700 1 -- 172.21.15.12:0/2804467247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 msgr2=0x5581d285acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15567fc700 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 0x5581d285acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15567fc700 1 -- 172.21.15.12:0/2804467247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2851480 msgr2=0x5581d285d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15567fc700 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2851480 0x5581d285d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15567fc700 1 -- 172.21.15.12:0/2804467247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15581a40d0 con 0x5581d2856950 2024-05-08T23:53:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f1556ffd700 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2851480 0x5581d285d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15577fe700 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 0x5581d285acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f15567fc700 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d285a590 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f154c009230 tx=0x7f154c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3baddd31389dd8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.859+0000 7f153aff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f153aff5700 1 -- 172.21.15.12:0/2804467247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c01c040 con 0x5581d2856950 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f156951eb80 1 -- 172.21.15.12:0/2804467247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581d26fa210 con 0x5581d2856950 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f153aff5700 1 -- 172.21.15.12:0/2804467247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f154c005240 con 0x5581d2856950 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f153aff5700 1 -- 172.21.15.12:0/2804467247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f154c00f980 con 0x5581d2856950 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f156951eb80 10 client.17226 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f156951eb80 1 -- 172.21.15.12:0/2804467247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5581d25ceb00 con 0x5581d2856950 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f153aff5700 1 -- 172.21.15.12:0/2804467247 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581d26fa210 con 0x5581d2856950 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f153aff5700 1 -- 172.21.15.12:0/2804467247 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f154c015020 con 0x5581d2856950 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f156951eb80 10 client.17226 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f156951eb80 10 client.17226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f156951eb80 20 client.17226 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f156951eb80 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581d286ac90 0x5581d286d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f156951eb80 4 client.17226 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f156951eb80 1 -- 172.21.15.12:0/2804467247 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581d28207b0 con 0x5581d286ac90 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.860+0000 7f1556ffd700 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581d286ac90 0x5581d286d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.861+0000 7f1556ffd700 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581d286ac90 0x5581d286d0b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99983fb84ea03ded server_cookie=f78d04cb383b5563 in_seq=0 out_seq=0 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.861+0000 7f153aff5700 10 client.17226 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.861+0000 7f153aff5700 1 -- 172.21.15.12:0/2804467247 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581d28207b0 con 0x5581d286ac90 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.861+0000 7f153aff5700 10 client.17226 handle_command_reply: tid=0 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 client.17226 shutdown 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 -- 172.21.15.12:0/2804467247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 msgr2=0x5581d285a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d285a590 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f154c009230 tx=0x7f154c00b790 comp rx=0 tx=0).stop 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 -- 172.21.15.12:0/2804467247 shutdown_connections 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2857270 0x5581d285acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581d286ac90 0x5581d286d0b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2851480 0x5581d285d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 --2- 172.21.15.12:0/2804467247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d285a590 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f153aff5700 0 client.17226 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 -- 172.21.15.12:0/2804467247 >> 172.21.15.12:0/2804467247 conn(0x5581d27af1b0 msgr2=0x5581d28554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 -- 172.21.15.12:0/2804467247 shutdown_connections 2024-05-08T23:53:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 1 -- 172.21.15.12:0/2804467247 wait complete. 2024-05-08T23:53:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.862+0000 7f156951eb80 20 client.17226 trim_cache size 0 max 16384 2024-05-08T23:53:00.868 INFO:tasks.workunit.client.0.smithi012.stdout: 1 416465 9.81 MB/sec execute 176 sec latency 32.285 ms 2024-05-08T23:53:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f156951eb80 1 Processor -- start 2024-05-08T23:53:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f156951eb80 1 -- start start 2024-05-08T23:53:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f156951eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d285ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f156951eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2856950 0x5581d284b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f156951eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2857270 0x5581d285cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f156951eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d28722c0 con 0x5581d2851480 2024-05-08T23:53:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f156951eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d284caf0 con 0x5581d2857270 2024-05-08T23:53:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f156951eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d2858ae0 con 0x5581d2856950 2024-05-08T23:53:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d285ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d285ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55822/0 (socket says 172.21.15.12:55822) 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f1556ffd700 1 -- 172.21.15.12:0/3195534842 learned_addr learned my addr 172.21.15.12:0/3195534842 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f15577fe700 1 --2- 172.21.15.12:0/3195534842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2857270 0x5581d285cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f15567fc700 1 --2- 172.21.15.12:0/3195534842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2856950 0x5581d284b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f1556ffd700 1 -- 172.21.15.12:0/3195534842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2856950 msgr2=0x5581d284b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f1556ffd700 1 --2- 172.21.15.12:0/3195534842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2856950 0x5581d284b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f1556ffd700 1 -- 172.21.15.12:0/3195534842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2857270 msgr2=0x5581d285cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.878+0000 7f1556ffd700 1 --2- 172.21.15.12:0/3195534842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2857270 0x5581d285cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.879+0000 7f1556ffd700 1 -- 172.21.15.12:0/3195534842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581d2767500 con 0x5581d2851480 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.879+0000 7f1556ffd700 1 --2- 172.21.15.12:0/3195534842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d285ac90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x5581d286d9c0 tx=0x7f1548014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15416a7b1e661002 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.879+0000 7f153dffb700 1 -- 172.21.15.12:0/3195534842 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548024040 con 0x5581d2851480 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.879+0000 7f153dffb700 1 -- 172.21.15.12:0/3195534842 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1548029920 con 0x5581d2851480 2024-05-08T23:53:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.879+0000 7f153dffb700 1 -- 172.21.15.12:0/3195534842 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154801ee90 con 0x5581d2851480 2024-05-08T23:53:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 -- 172.21.15.12:0/3195534842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 msgr2=0x5581d285ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 --2- 172.21.15.12:0/3195534842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d285ac90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x5581d286d9c0 tx=0x7f1548014000 comp rx=0 tx=0).stop 2024-05-08T23:53:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 -- 172.21.15.12:0/3195534842 shutdown_connections 2024-05-08T23:53:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 --2- 172.21.15.12:0/3195534842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2857270 0x5581d285cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 --2- 172.21.15.12:0/3195534842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2851480 0x5581d285ac90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 --2- 172.21.15.12:0/3195534842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2856950 0x5581d284b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 -- 172.21.15.12:0/3195534842 >> 172.21.15.12:0/3195534842 conn(0x5581d27af1b0 msgr2=0x5581d28666c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 -- 172.21.15.12:0/3195534842 shutdown_connections 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 -- 172.21.15.12:0/3195534842 wait complete. 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 -- start start 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.880+0000 7f156951eb80 1 Processor -- start 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f156951eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2851480 0x5581d2862040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f156951eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d2855400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f156951eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2857270 0x5581d2855a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f156951eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d2856110 con 0x5581d2856950 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f156951eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d2854380 con 0x5581d2857270 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f156951eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d2854540 con 0x5581d2851480 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f156951eb80 10 client.0 _ll_register_callbacks cb 0x5581d2746ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f15567fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d2855400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f15567fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d2855400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55824/0 (socket says 172.21.15.12:55824) 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f15567fc700 1 -- 172.21.15.12:0/3929942375 learned_addr learned my addr 172.21.15.12:0/3929942375 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2851480 0x5581d2862040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f15577fe700 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2857270 0x5581d2855a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f15567fc700 1 -- 172.21.15.12:0/3929942375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2851480 msgr2=0x5581d2862040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f15567fc700 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2851480 0x5581d2862040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f15567fc700 1 -- 172.21.15.12:0/3929942375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2857270 msgr2=0x5581d2855a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f15567fc700 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2857270 0x5581d2855a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f15567fc700 1 -- 172.21.15.12:0/3929942375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1548017070 con 0x5581d2856950 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f15567fc700 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d2855400 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f154c009810 tx=0x7f154c0234b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fce8c6f63920902 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f153aff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f153aff5700 1 -- 172.21.15.12:0/3929942375 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c01c040 con 0x5581d2856950 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f153aff5700 1 -- 172.21.15.12:0/3929942375 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f154c000e40 con 0x5581d2856950 2024-05-08T23:53:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.881+0000 7f156951eb80 1 -- 172.21.15.12:0/3929942375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1550099970 con 0x5581d2856950 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f153aff5700 1 -- 172.21.15.12:0/3929942375 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f154c005b00 con 0x5581d2856950 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f156951eb80 10 client.17232 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f156951eb80 1 -- 172.21.15.12:0/3929942375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5581d2626c70 con 0x5581d2856950 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f153aff5700 1 -- 172.21.15.12:0/3929942375 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f154c044020 con 0x5581d2856950 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f153aff5700 1 -- 172.21.15.12:0/3929942375 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f154c009c70 con 0x5581d2856950 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f156951eb80 10 client.17232 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f156951eb80 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f156951eb80 20 client.17232 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f156951eb80 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581d28c1650 0x5581d28c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f156951eb80 4 client.17232 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f156951eb80 1 -- 172.21.15.12:0/3929942375 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5581d2799e60 con 0x5581d28c1650 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.882+0000 7f1556ffd700 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581d28c1650 0x5581d28c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.883+0000 7f1556ffd700 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581d28c1650 0x5581d28c3a70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1554ce2c10047440 server_cookie=f7f1ea8d76f805de in_seq=0 out_seq=0 2024-05-08T23:53:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.883+0000 7f153aff5700 10 client.17232 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:00.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:00 smithi012 ceph-mon[42503]: pgmap v202: 97 pgs: 97 active+clean; 117 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 222 op/s 2024-05-08T23:53:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f153aff5700 1 -- 172.21.15.12:0/3929942375 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5581d2799e60 con 0x5581d28c1650 2024-05-08T23:53:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f153aff5700 10 client.17232 handle_command_reply: tid=0 2024-05-08T23:53:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f156951eb80 1 client.17232 shutdown 2024-05-08T23:53:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f156951eb80 1 -- 172.21.15.12:0/3929942375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 msgr2=0x5581d2855400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f156951eb80 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d2855400 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f154c009810 tx=0x7f154c0234b0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f156951eb80 1 -- 172.21.15.12:0/3929942375 shutdown_connections 2024-05-08T23:53:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f156951eb80 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581d28c1650 0x5581d28c3a70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f156951eb80 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581d2857270 0x5581d2855a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f156951eb80 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581d2851480 0x5581d2862040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f156951eb80 1 --2- 172.21.15.12:0/3929942375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d2856950 0x5581d2855400 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f156951eb80 1 -- 172.21.15.12:0/3929942375 >> 172.21.15.12:0/3929942375 conn(0x5581d27af1b0 msgr2=0x5581d28505b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.942+0000 7f153aff5700 0 client.17232 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.946+0000 7f156951eb80 1 -- 172.21.15.12:0/3929942375 shutdown_connections 2024-05-08T23:53:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.946+0000 7f156951eb80 1 -- 172.21.15.12:0/3929942375 wait complete. 2024-05-08T23:53:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.946+0000 7f156951eb80 20 client.17232 trim_cache size 0 max 16384 2024-05-08T23:53:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:53:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:53:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:53:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:53:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 -- 172.21.15.12:0/3234351985 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15400a1290 msgr2=0x7f15400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15400a1290 0x7f15400a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1548005fd0 tx=0x7f1548009310 comp rx=0 tx=0).stop 2024-05-08T23:53:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 -- 172.21.15.12:0/3234351985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1558106f50 msgr2=0x7f155819bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1558106f50 0x7f155819bc40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f155000efc0 tx=0x7f155000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 -- 172.21.15.12:0/3234351985 shutdown_connections 2024-05-08T23:53:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15400a1290 0x7f15400a3750 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1558106f50 0x7f155819bc40 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1558069730 0x7f1558197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 --2- 172.21.15.12:0/3234351985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f155806a0d0 0x7f15581978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 -- 172.21.15.12:0/3234351985 >> 172.21.15.12:0/3234351985 conn(0x7f1558076fd0 msgr2=0x7f15580fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 -- 172.21.15.12:0/3234351985 shutdown_connections 2024-05-08T23:53:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:00.947+0000 7f155d364700 1 -- 172.21.15.12:0/3234351985 wait complete. 2024-05-08T23:53:01.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 418663 9.82 MB/sec execute 177 sec latency 28.641 ms 2024-05-08T23:53:02.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:02 smithi204 ceph-mon[47373]: pgmap v203: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 405 op/s 2024-05-08T23:53:02.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:02 smithi160 ceph-mon[46152]: pgmap v203: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 405 op/s 2024-05-08T23:53:02.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 419993 9.80 MB/sec execute 178 sec latency 15.529 ms 2024-05-08T23:53:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:02 smithi012 ceph-mon[42503]: pgmap v203: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 405 op/s 2024-05-08T23:53:03.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:53:03.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 421377 9.78 MB/sec execute 179 sec latency 63.563 ms 2024-05-08T23:53:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.953+0000 7fe68cdbd700 1 -- 172.21.15.12:0/3381790516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe688101b90 msgr2=0x7fe688105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.953+0000 7fe68cdbd700 1 --2- 172.21.15.12:0/3381790516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe688101b90 0x7fe688105bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009990 tx=0x7fe67c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.953+0000 7fe68cdbd700 1 -- 172.21.15.12:0/3381790516 shutdown_connections 2024-05-08T23:53:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.953+0000 7fe68cdbd700 1 --2- 172.21.15.12:0/3381790516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe688106300 0x7fe68810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.953+0000 7fe68cdbd700 1 --2- 172.21.15.12:0/3381790516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6881011e0 0x7fe6881015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.954+0000 7fe68cdbd700 1 --2- 172.21.15.12:0/3381790516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe688101b90 0x7fe688105bd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.954+0000 7fe68cdbd700 1 -- 172.21.15.12:0/3381790516 >> 172.21.15.12:0/3381790516 conn(0x7fe6880fca80 msgr2=0x7fe6880feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.954+0000 7fe68cdbd700 1 -- 172.21.15.12:0/3381790516 shutdown_connections 2024-05-08T23:53:03.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.954+0000 7fe68cdbd700 1 -- 172.21.15.12:0/3381790516 wait complete. 2024-05-08T23:53:03.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68cdbd700 1 Processor -- start 2024-05-08T23:53:03.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68cdbd700 1 -- start start 2024-05-08T23:53:03.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68cdbd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6881011e0 0x7fe68810dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68cdbd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe688106300 0x7fe68810ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68cdbd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68810b2c0 0x7fe68810b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68cdbd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881aa0c0 con 0x7fe6881011e0 2024-05-08T23:53:03.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68cdbd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881aa230 con 0x7fe688106300 2024-05-08T23:53:03.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68cdbd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881aa3f0 con 0x7fe68810b2c0 2024-05-08T23:53:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6881011e0 0x7fe68810dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6881011e0 0x7fe68810dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55840/0 (socket says 172.21.15.12:55840) 2024-05-08T23:53:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.955+0000 7fe68659c700 1 -- 172.21.15.12:0/2515716336 learned_addr learned my addr 172.21.15.12:0/2515716336 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.956+0000 7fe686d9d700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68810b2c0 0x7fe68810b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.956+0000 7fe68659c700 1 -- 172.21.15.12:0/2515716336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68810b2c0 msgr2=0x7fe68810b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.956+0000 7fe68659c700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68810b2c0 0x7fe68810b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.956+0000 7fe68659c700 1 -- 172.21.15.12:0/2515716336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe688106300 msgr2=0x7fe68810ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.956+0000 7fe68659c700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe688106300 0x7fe68810ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.956+0000 7fe68659c700 1 -- 172.21.15.12:0/2515716336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c0095f0 con 0x7fe6881011e0 2024-05-08T23:53:03.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.956+0000 7fe68659c700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6881011e0 0x7fe68810dcb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b550 tx=0x7fe67800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:03.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.956+0000 7fe6777fe700 1 -- 172.21.15.12:0/2515716336 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678011830 con 0x7fe6881011e0 2024-05-08T23:53:03.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.957+0000 7fe6777fe700 1 -- 172.21.15.12:0/2515716336 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe678011e50 con 0x7fe6881011e0 2024-05-08T23:53:03.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.957+0000 7fe68cdbd700 1 -- 172.21.15.12:0/2515716336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6881aa6d0 con 0x7fe6881011e0 2024-05-08T23:53:03.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.957+0000 7fe6777fe700 1 -- 172.21.15.12:0/2515716336 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800f2e0 con 0x7fe6881011e0 2024-05-08T23:53:03.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.957+0000 7fe68cdbd700 1 -- 172.21.15.12:0/2515716336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6881aab90 con 0x7fe6881011e0 2024-05-08T23:53:03.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.959+0000 7fe6777fe700 1 -- 172.21.15.12:0/2515716336 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe678011990 con 0x7fe6881011e0 2024-05-08T23:53:03.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.960+0000 7fe6777fe700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6700a1310 0x7fe6700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.960+0000 7fe6777fe700 1 -- 172.21.15.12:0/2515716336 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe6780cc600 con 0x7fe6881011e0 2024-05-08T23:53:03.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.960+0000 7fe685d9b700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6700a1310 0x7fe6700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.961+0000 7fe685d9b700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6700a1310 0x7fe6700a37d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00b5c0 tx=0x7fe67c005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:03.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.961+0000 7fe698f77b80 1 Processor -- start 2024-05-08T23:53:03.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.962+0000 7fe698f77b80 1 -- start start 2024-05-08T23:53:03.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.962+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c691e00 0x55dd2c6921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.962+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c692720 0x55dd2c68c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.962+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.962+0000 7fe698f77b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c5e3150 con 0x55dd2c692720 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.962+0000 7fe698f77b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c5e0b10 con 0x55dd2c691e00 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.962+0000 7fe698f77b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c5dc400 con 0x55dd2c68c890 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.962+0000 7fe686d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.962+0000 7fe686d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41506/0 (socket says 172.21.15.12:41506) 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.962+0000 7fe686d9d700 1 -- 172.21.15.12:0/3084799840 learned_addr learned my addr 172.21.15.12:0/3084799840 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.963+0000 7fe686d9d700 1 -- 172.21.15.12:0/3084799840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c691e00 msgr2=0x55dd2c6921e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.963+0000 7fe686d9d700 1 --2- 172.21.15.12:0/3084799840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c691e00 0x55dd2c6921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.963+0000 7fe686d9d700 1 -- 172.21.15.12:0/3084799840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c692720 msgr2=0x55dd2c68c350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:53:03.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.963+0000 7fe686d9d700 1 --2- 172.21.15.12:0/3084799840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c692720 0x55dd2c68c350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.963+0000 7fe686d9d700 1 -- 172.21.15.12:0/3084799840 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd2c45d3b0 con 0x55dd2c68c890 2024-05-08T23:53:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.963+0000 7fe686d9d700 1 --2- 172.21.15.12:0/3084799840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68ccd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe68000f5a0 tx=0x7fe68000f8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37e8708f9acab4a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.963+0000 7fe674ff9700 1 -- 172.21.15.12:0/3084799840 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680007e30 con 0x55dd2c68c890 2024-05-08T23:53:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.964+0000 7fe674ff9700 1 -- 172.21.15.12:0/3084799840 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe680004bb0 con 0x55dd2c68c890 2024-05-08T23:53:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.964+0000 7fe674ff9700 1 -- 172.21.15.12:0/3084799840 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6800052c0 con 0x55dd2c68c890 2024-05-08T23:53:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.965+0000 7fe698f77b80 1 -- 172.21.15.12:0/3084799840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 msgr2=0x55dd2c68ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.965+0000 7fe698f77b80 1 --2- 172.21.15.12:0/3084799840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68ccd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe68000f5a0 tx=0x7fe68000f8b0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.965+0000 7fe698f77b80 1 -- 172.21.15.12:0/3084799840 shutdown_connections 2024-05-08T23:53:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.965+0000 7fe698f77b80 1 --2- 172.21.15.12:0/3084799840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68ccd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.965+0000 7fe698f77b80 1 --2- 172.21.15.12:0/3084799840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c691e00 0x55dd2c6921e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.965+0000 7fe698f77b80 1 --2- 172.21.15.12:0/3084799840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c692720 0x55dd2c68c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.965+0000 7fe698f77b80 1 -- 172.21.15.12:0/3084799840 >> 172.21.15.12:0/3084799840 conn(0x55dd2c5ea210 msgr2=0x55dd2c686df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.965+0000 7fe698f77b80 1 -- 172.21.15.12:0/3084799840 shutdown_connections 2024-05-08T23:53:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.965+0000 7fe698f77b80 1 -- 172.21.15.12:0/3084799840 wait complete. 2024-05-08T23:53:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe698f77b80 1 -- start start 2024-05-08T23:53:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe698f77b80 1 Processor -- start 2024-05-08T23:53:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c695b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c691e00 0x55dd2c6960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c692720 0x55dd2c696600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe698f77b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c578560 con 0x55dd2c692720 2024-05-08T23:53:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe698f77b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c578200 con 0x55dd2c691e00 2024-05-08T23:53:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe698f77b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c45f6a0 con 0x55dd2c68c890 2024-05-08T23:53:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe698f77b80 10 client.0 _ll_register_callbacks cb 0x55dd2c581b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe68659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c695b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe68659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c695b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41508/0 (socket says 172.21.15.12:41508) 2024-05-08T23:53:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.966+0000 7fe68659c700 1 -- 172.21.15.12:0/975674252 learned_addr learned my addr 172.21.15.12:0/975674252 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.967+0000 7fe686d9d700 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c692720 0x55dd2c696600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.967+0000 7fe68659c700 1 -- 172.21.15.12:0/975674252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c691e00 msgr2=0x55dd2c6960c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:53:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.967+0000 7fe68659c700 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c691e00 0x55dd2c6960c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.967+0000 7fe68659c700 1 -- 172.21.15.12:0/975674252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c692720 msgr2=0x55dd2c696600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.967+0000 7fe68659c700 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c692720 0x55dd2c696600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.967+0000 7fe68659c700 1 -- 172.21.15.12:0/975674252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68000f1d0 con 0x55dd2c68c890 2024-05-08T23:53:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.968+0000 7fe68659c700 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c695b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe67800bc60 tx=0x7fe678099200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed7b7386d1fa05e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.968+0000 7fe66dffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:53:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.968+0000 7fe66dffb700 1 -- 172.21.15.12:0/975674252 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67809d5e0 con 0x55dd2c68c890 2024-05-08T23:53:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.968+0000 7fe698f77b80 1 -- 172.21.15.12:0/975674252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd2c535270 con 0x55dd2c68c890 2024-05-08T23:53:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.968+0000 7fe66dffb700 1 -- 172.21.15.12:0/975674252 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe6780a1c30 con 0x55dd2c68c890 2024-05-08T23:53:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.968+0000 7fe66dffb700 1 -- 172.21.15.12:0/975674252 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6780172c0 con 0x55dd2c68c890 2024-05-08T23:53:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.968+0000 7fe698f77b80 10 client.27025 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.968+0000 7fe698f77b80 1 -- 172.21.15.12:0/975674252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dd2c409b60 con 0x55dd2c68c890 2024-05-08T23:53:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.969+0000 7fe66dffb700 1 -- 172.21.15.12:0/975674252 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd2c535270 con 0x55dd2c68c890 2024-05-08T23:53:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.969+0000 7fe66dffb700 1 -- 172.21.15.12:0/975674252 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe678098ca0 con 0x55dd2c68c890 2024-05-08T23:53:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.969+0000 7fe698f77b80 10 client.27025 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.969+0000 7fe698f77b80 10 client.27025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.969+0000 7fe698f77b80 20 client.27025 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.969+0000 7fe698f77b80 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dd2c6a6170 0x55dd2c6a8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.969+0000 7fe698f77b80 4 client.27025 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.969+0000 7fe698f77b80 1 -- 172.21.15.12:0/975674252 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd2c65b810 con 0x55dd2c6a6170 2024-05-08T23:53:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.969+0000 7fe685d9b700 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dd2c6a6170 0x55dd2c6a8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.970+0000 7fe685d9b700 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dd2c6a6170 0x55dd2c6a8590 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8392a32813202c server_cookie=8018d5353c4a0f42 in_seq=0 out_seq=0 2024-05-08T23:53:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.970+0000 7fe66dffb700 10 client.27025 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.971+0000 7fe66dffb700 1 -- 172.21.15.12:0/975674252 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd2c65b810 con 0x55dd2c6a6170 2024-05-08T23:53:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.971+0000 7fe66dffb700 10 client.27025 handle_command_reply: tid=0 2024-05-08T23:53:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.971+0000 7fe698f77b80 1 client.27025 shutdown 2024-05-08T23:53:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.972+0000 7fe698f77b80 1 -- 172.21.15.12:0/975674252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 msgr2=0x55dd2c695b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.972+0000 7fe698f77b80 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c695b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe67800bc60 tx=0x7fe678099200 comp rx=0 tx=0).stop 2024-05-08T23:53:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.973+0000 7fe698f77b80 1 -- 172.21.15.12:0/975674252 shutdown_connections 2024-05-08T23:53:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.973+0000 7fe698f77b80 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c692720 0x55dd2c696600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.973+0000 7fe698f77b80 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c695b80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.973+0000 7fe698f77b80 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dd2c6a6170 0x55dd2c6a8590 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.973+0000 7fe698f77b80 1 --2- 172.21.15.12:0/975674252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c691e00 0x55dd2c6960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.973+0000 7fe698f77b80 1 -- 172.21.15.12:0/975674252 >> 172.21.15.12:0/975674252 conn(0x55dd2c5ea210 msgr2=0x55dd2c6865a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:03.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.973+0000 7fe66dffb700 0 client.27025 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:03.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.973+0000 7fe698f77b80 1 -- 172.21.15.12:0/975674252 shutdown_connections 2024-05-08T23:53:03.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.973+0000 7fe698f77b80 1 -- 172.21.15.12:0/975674252 wait complete. 2024-05-08T23:53:03.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.973+0000 7fe698f77b80 20 client.27025 trim_cache size 0 max 16384 2024-05-08T23:53:03.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.994+0000 7fe698f77b80 1 Processor -- start 2024-05-08T23:53:03.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe698f77b80 1 -- start start 2024-05-08T23:53:03.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 0x55dd2c691350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c692720 0x55dd2c686d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe698f77b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c697180 con 0x55dd2c691e00 2024-05-08T23:53:03.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe698f77b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c688e60 con 0x55dd2c692720 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe698f77b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c6ad7a0 con 0x55dd2c68c890 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe68659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe68659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41518/0 (socket says 172.21.15.12:41518) 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe68659c700 1 -- 172.21.15.12:0/1636020550 learned_addr learned my addr 172.21.15.12:0/1636020550 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe686d9d700 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c692720 0x55dd2c686d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe685d9b700 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 0x55dd2c691350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe686d9d700 1 -- 172.21.15.12:0/1636020550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 msgr2=0x55dd2c68c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe686d9d700 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe686d9d700 1 -- 172.21.15.12:0/1636020550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 msgr2=0x55dd2c691350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe686d9d700 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 0x55dd2c691350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe686d9d700 1 -- 172.21.15.12:0/1636020550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd2c5a2560 con 0x55dd2c692720 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe68659c700 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.995+0000 7fe685d9b700 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 0x55dd2c691350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.996+0000 7fe686d9d700 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c692720 0x55dd2c686d20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe680011010 tx=0x7fe68000f4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2fb5c331cda28a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.996+0000 7fe6757fa700 1 -- 172.21.15.12:0/1636020550 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680014d30 con 0x55dd2c692720 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.996+0000 7fe6757fa700 1 -- 172.21.15.12:0/1636020550 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe68000b030 con 0x55dd2c692720 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.996+0000 7fe6757fa700 1 -- 172.21.15.12:0/1636020550 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68001dd60 con 0x55dd2c692720 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 -- 172.21.15.12:0/1636020550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c692720 msgr2=0x55dd2c686d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c692720 0x55dd2c686d20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe680011010 tx=0x7fe68000f4e0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 -- 172.21.15.12:0/1636020550 shutdown_connections 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c692720 0x55dd2c686d20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c68c890 0x55dd2c68c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 --2- 172.21.15.12:0/1636020550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 0x55dd2c691350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 -- 172.21.15.12:0/1636020550 >> 172.21.15.12:0/1636020550 conn(0x55dd2c5ea210 msgr2=0x55dd2c690c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:03.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 -- 172.21.15.12:0/1636020550 shutdown_connections 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 -- 172.21.15.12:0/1636020550 wait complete. 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 -- start start 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.997+0000 7fe698f77b80 1 Processor -- start 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c68c890 0x55dd2c69b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 0x55dd2c69b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe698f77b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c692720 0x55dd2c6fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe698f77b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c6fbbb0 con 0x55dd2c691e00 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe698f77b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c6fbcf0 con 0x55dd2c68c890 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe698f77b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c6fbe30 con 0x55dd2c692720 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe698f77b80 10 client.0 _ll_register_callbacks cb 0x55dd2c581b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe68659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c68c890 0x55dd2c69b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe68659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c68c890 0x55dd2c69b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39020/0 (socket says 172.21.15.12:39020) 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe686d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c692720 0x55dd2c6fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe68659c700 1 -- 172.21.15.12:0/524828365 learned_addr learned my addr 172.21.15.12:0/524828365 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:03.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe685d9b700 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 0x55dd2c69b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe686d9d700 1 -- 172.21.15.12:0/524828365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c68c890 msgr2=0x55dd2c69b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe686d9d700 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c68c890 0x55dd2c69b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe686d9d700 1 -- 172.21.15.12:0/524828365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 msgr2=0x55dd2c69b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe686d9d700 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 0x55dd2c69b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe686d9d700 1 -- 172.21.15.12:0/524828365 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd2c409b60 con 0x55dd2c692720 2024-05-08T23:53:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe685d9b700 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 0x55dd2c69b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe686d9d700 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c692720 0x55dd2c6fb670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe680011010 tx=0x7fe680014730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=186a0451cedb9066 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:03.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe66dffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.998+0000 7fe66dffb700 1 -- 172.21.15.12:0/524828365 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68001d960 con 0x55dd2c692720 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe698f77b80 1 -- 172.21.15.12:0/524828365 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe680004bb0 con 0x55dd2c692720 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe66dffb700 1 -- 172.21.15.12:0/524828365 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe68001dac0 con 0x55dd2c692720 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe66dffb700 1 -- 172.21.15.12:0/524828365 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe68001dc80 con 0x55dd2c692720 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe698f77b80 10 client.27031 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe698f77b80 1 -- 172.21.15.12:0/524828365 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dd2c45d3b0 con 0x55dd2c692720 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe66dffb700 1 -- 172.21.15.12:0/524828365 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe68003c020 con 0x55dd2c692720 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe66dffb700 1 -- 172.21.15.12:0/524828365 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe680012070 con 0x55dd2c692720 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe698f77b80 10 client.27031 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe698f77b80 10 client.27031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe698f77b80 20 client.27031 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:04.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe698f77b80 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dd2c6fd2f0 0x55dd2c6ff710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe698f77b80 4 client.27031 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:53:04.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe698f77b80 1 -- 172.21.15.12:0/524828365 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dd2c5d4ec0 con 0x55dd2c6fd2f0 2024-05-08T23:53:04.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:03.999+0000 7fe68659c700 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dd2c6fd2f0 0x55dd2c6ff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe68659c700 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dd2c6fd2f0 0x55dd2c6ff710 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6da5ce098bbbed server_cookie=407ca348eb969533 in_seq=0 out_seq=0 2024-05-08T23:53:04.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe66dffb700 10 client.27031 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:04.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe66dffb700 1 -- 172.21.15.12:0/524828365 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55dd2c5d4ec0 con 0x55dd2c6fd2f0 2024-05-08T23:53:04.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe66dffb700 10 client.27031 handle_command_reply: tid=0 2024-05-08T23:53:04.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 client.27031 shutdown 2024-05-08T23:53:04.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 -- 172.21.15.12:0/524828365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c692720 msgr2=0x55dd2c6fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c692720 0x55dd2c6fb670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe680011010 tx=0x7fe680014730 comp rx=0 tx=0).stop 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 -- 172.21.15.12:0/524828365 shutdown_connections 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd2c692720 0x55dd2c6fb670 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dd2c6fd2f0 0x55dd2c6ff710 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd2c68c890 0x55dd2c69b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 --2- 172.21.15.12:0/524828365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd2c691e00 0x55dd2c69b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe66dffb700 0 client.27031 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 -- 172.21.15.12:0/524828365 >> 172.21.15.12:0/524828365 conn(0x55dd2c5ea210 msgr2=0x55dd2c694450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 -- 172.21.15.12:0/524828365 shutdown_connections 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.000+0000 7fe698f77b80 1 -- 172.21.15.12:0/524828365 wait complete. 2024-05-08T23:53:04.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe698f77b80 20 client.27031 trim_cache size 0 max 16384 2024-05-08T23:53:04.003 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:53:04.003 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:53:04.003 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:53:04.003 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 -- 172.21.15.12:0/2515716336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6700a1310 msgr2=0x7fe6700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6700a1310 0x7fe6700a37d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00b5c0 tx=0x7fe67c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 -- 172.21.15.12:0/2515716336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6881011e0 msgr2=0x7fe68810dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6881011e0 0x7fe68810dcb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b550 tx=0x7fe67800b860 comp rx=0 tx=0).stop 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 -- 172.21.15.12:0/2515716336 shutdown_connections 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe6700a1310 0x7fe6700a37d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68810b2c0 0x7fe68810b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6881011e0 0x7fe68810dcb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 --2- 172.21.15.12:0/2515716336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe688106300 0x7fe68810ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 -- 172.21.15.12:0/2515716336 >> 172.21.15.12:0/2515716336 conn(0x7fe6880fca80 msgr2=0x7fe6880fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 -- 172.21.15.12:0/2515716336 shutdown_connections 2024-05-08T23:53:04.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.001+0000 7fe68cdbd700 1 -- 172.21.15.12:0/2515716336 wait complete. 2024-05-08T23:53:04.011 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:53:04.011 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:53:04.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.165+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1736880380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae34102000 msgr2=0x7fae341023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.165+0000 7fae3b4cc700 1 --2- 172.21.15.12:0/1736880380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae34102000 0x7fae341023e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fae30009990 tx=0x7fae30009ca0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.165+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1736880380 shutdown_connections 2024-05-08T23:53:04.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.165+0000 7fae3b4cc700 1 --2- 172.21.15.12:0/1736880380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae3410b3d0 0x7fae3410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.165+0000 7fae3b4cc700 1 --2- 172.21.15.12:0/1736880380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae34102000 0x7fae341023e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.165+0000 7fae3b4cc700 1 --2- 172.21.15.12:0/1736880380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae34102920 0x7fae3410ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.165+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1736880380 >> 172.21.15.12:0/1736880380 conn(0x7fae340fb7e0 msgr2=0x7fae340fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.165+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1736880380 shutdown_connections 2024-05-08T23:53:04.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.165+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1736880380 wait complete. 2024-05-08T23:53:04.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae3b4cc700 1 Processor -- start 2024-05-08T23:53:04.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae3b4cc700 1 -- start start 2024-05-08T23:53:04.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae3b4cc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae34102000 0x7fae34197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae3b4cc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae34102920 0x7fae341978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae3b4cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae3410b3d0 0x7fae3419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae3b4cc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3419c260 con 0x7fae3410b3d0 2024-05-08T23:53:04.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae3b4cc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3419c3d0 con 0x7fae34102920 2024-05-08T23:53:04.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae3b4cc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3419c590 con 0x7fae34102000 2024-05-08T23:53:04.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae39a69700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae3410b3d0 0x7fae3419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae39a69700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae3410b3d0 0x7fae3419bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55878/0 (socket says 172.21.15.12:55878) 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae38a67700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae34102920 0x7fae341978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae39a69700 1 -- 172.21.15.12:0/1359187722 learned_addr learned my addr 172.21.15.12:0/1359187722 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.166+0000 7fae39268700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae34102000 0x7fae34197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae39a69700 1 -- 172.21.15.12:0/1359187722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae34102000 msgr2=0x7fae34197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae39a69700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae34102000 0x7fae34197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae39a69700 1 -- 172.21.15.12:0/1359187722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae34102920 msgr2=0x7fae341978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae39a69700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae34102920 0x7fae341978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae39a69700 1 -- 172.21.15.12:0/1359187722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae300095f0 con 0x7fae3410b3d0 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae38a67700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae34102920 0x7fae341978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae39a69700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae3410b3d0 0x7fae3419bc20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00ef30 tx=0x7fae2c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae2a7fc700 1 -- 172.21.15.12:0/1359187722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c018070 con 0x7fae3410b3d0 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1359187722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3419c810 con 0x7fae3410b3d0 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae2a7fc700 1 -- 172.21.15.12:0/1359187722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae2c00f040 con 0x7fae3410b3d0 2024-05-08T23:53:04.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae2a7fc700 1 -- 172.21.15.12:0/1359187722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00cd00 con 0x7fae3410b3d0 2024-05-08T23:53:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.167+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1359187722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae341a40b0 con 0x7fae3410b3d0 2024-05-08T23:53:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.168+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1359187722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae341064b0 con 0x7fae3410b3d0 2024-05-08T23:53:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.170+0000 7fae2a7fc700 1 -- 172.21.15.12:0/1359187722 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fae2c003a00 con 0x7fae3410b3d0 2024-05-08T23:53:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.171+0000 7fae2a7fc700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae200a1290 0x7fae200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.171+0000 7fae2a7fc700 1 -- 172.21.15.12:0/1359187722 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fae2c0cd190 con 0x7fae3410b3d0 2024-05-08T23:53:04.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.171+0000 7fae39268700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae200a1290 0x7fae200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.171+0000 7fae39268700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae200a1290 0x7fae200a3750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fae3000b2e0 tx=0x7fae30005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.174+0000 7fae2a7fc700 1 -- 172.21.15.12:0/1359187722 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fae2c096600 con 0x7fae3410b3d0 2024-05-08T23:53:04.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.466+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1359187722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae34068870 con 0x7fae3410b3d0 2024-05-08T23:53:04.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.467+0000 7fae2a7fc700 1 -- 172.21.15.12:0/1359187722 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10874 (secure 0 0 0) 0x7fae2c025380 con 0x7fae3410b3d0 2024-05-08T23:53:04.462 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:53:04.462 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:52:36.483244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_26224":{"gid":26224,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/131208281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":131208281},{"type":"v1","addr":"172.21.15.204:6837","nonce":131208281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:53:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1359187722 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae200a1290 msgr2=0x7fae200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae200a1290 0x7fae200a3750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fae3000b2e0 tx=0x7fae30005960 comp rx=0 tx=0).stop 2024-05-08T23:53:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1359187722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae3410b3d0 msgr2=0x7fae3419bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae3410b3d0 0x7fae3419bc20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00ef30 tx=0x7fae2c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1359187722 shutdown_connections 2024-05-08T23:53:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae200a1290 0x7fae200a3750 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae3410b3d0 0x7fae3419bc20 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae34102000 0x7fae34197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 --2- 172.21.15.12:0/1359187722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae34102920 0x7fae341978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1359187722 >> 172.21.15.12:0/1359187722 conn(0x7fae340fb7e0 msgr2=0x7fae340fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1359187722 shutdown_connections 2024-05-08T23:53:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.470+0000 7fae3b4cc700 1 -- 172.21.15.12:0/1359187722 wait complete. 2024-05-08T23:53:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2024-05-08T23:53:04.481 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:53:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.638+0000 7f09fb304700 1 -- 172.21.15.12:0/781409878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f41062e0 msgr2=0x7f09f410dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.638+0000 7f09fb304700 1 --2- 172.21.15.12:0/781409878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f41062e0 0x7f09f410dea0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00b0a0 tx=0x7f09ec00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.638+0000 7f09fb304700 1 -- 172.21.15.12:0/781409878 shutdown_connections 2024-05-08T23:53:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.638+0000 7f09fb304700 1 --2- 172.21.15.12:0/781409878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f41062e0 0x7f09f410dea0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.638+0000 7f09fb304700 1 --2- 172.21.15.12:0/781409878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09f40ffad0 0x7f09f40ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.638+0000 7f09fb304700 1 --2- 172.21.15.12:0/781409878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f4100480 0x7f09f4105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.638+0000 7f09fb304700 1 -- 172.21.15.12:0/781409878 >> 172.21.15.12:0/781409878 conn(0x7f09f40750c0 msgr2=0x7f09f40754d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.638+0000 7f09fb304700 1 -- 172.21.15.12:0/781409878 shutdown_connections 2024-05-08T23:53:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.638+0000 7f09fb304700 1 -- 172.21.15.12:0/781409878 wait complete. 2024-05-08T23:53:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09fb304700 1 Processor -- start 2024-05-08T23:53:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09fb304700 1 -- start start 2024-05-08T23:53:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09fb304700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09f40ffad0 0x7f09f4197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09fb304700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f41062e0 0x7f09f4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09fb304700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f419be40 0x7f09f419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09fb304700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f419c8c0 con 0x7f09f41062e0 2024-05-08T23:53:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09fb304700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f419ca30 con 0x7f09f419be40 2024-05-08T23:53:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09fb304700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f419cbf0 con 0x7f09f40ffad0 2024-05-08T23:53:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f889f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f41062e0 0x7f09f4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f90a0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09f40ffad0 0x7f09f4197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f889f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f41062e0 0x7f09f4197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55902/0 (socket says 172.21.15.12:55902) 2024-05-08T23:53:04.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f889f700 1 -- 172.21.15.12:0/1883270763 learned_addr learned my addr 172.21.15.12:0/1883270763 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f98a1700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f419be40 0x7f09f419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f90a0700 1 -- 172.21.15.12:0/1883270763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f419be40 msgr2=0x7f09f419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f90a0700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f419be40 0x7f09f419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f90a0700 1 -- 172.21.15.12:0/1883270763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f41062e0 msgr2=0x7f09f4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f90a0700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f41062e0 0x7f09f4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f90a0700 1 -- 172.21.15.12:0/1883270763 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec009d00 con 0x7f09f40ffad0 2024-05-08T23:53:04.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.639+0000 7f09f889f700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f41062e0 0x7f09f4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:04.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.640+0000 7f09f90a0700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09f40ffad0 0x7f09f4197560 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f09e800e9f0 tx=0x7f09e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.640+0000 7f09e67fc700 1 -- 172.21.15.12:0/1883270763 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e800c9c0 con 0x7f09f40ffad0 2024-05-08T23:53:04.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.640+0000 7f09e67fc700 1 -- 172.21.15.12:0/1883270763 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09e8010430 con 0x7f09f40ffad0 2024-05-08T23:53:04.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.640+0000 7f09fb304700 1 -- 172.21.15.12:0/1883270763 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f41a4150 con 0x7f09f40ffad0 2024-05-08T23:53:04.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.640+0000 7f09e67fc700 1 -- 172.21.15.12:0/1883270763 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e8018680 con 0x7f09f40ffad0 2024-05-08T23:53:04.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.640+0000 7f09fb304700 1 -- 172.21.15.12:0/1883270763 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f41a4630 con 0x7f09f40ffad0 2024-05-08T23:53:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.642+0000 7f09e67fc700 1 -- 172.21.15.12:0/1883270763 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f09e8010860 con 0x7f09f40ffad0 2024-05-08T23:53:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.643+0000 7f09e67fc700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09e00a1290 0x7f09e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.643+0000 7f09f889f700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09e00a1290 0x7f09e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.643+0000 7f09e67fc700 1 -- 172.21.15.12:0/1883270763 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f09e8014070 con 0x7f09f40ffad0 2024-05-08T23:53:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.643+0000 7f09f889f700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09e00a1290 0x7f09e00a3750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09f4198ab0 tx=0x7f09f0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f0a074beb80 1 Processor -- start 2024-05-08T23:53:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f0a074beb80 1 -- start start 2024-05-08T23:53:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 0x564f0b45e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 0x564f0b458350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b458890 0x564f0b458cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f0a074beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b3af150 con 0x564f0b458890 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f0a074beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b3acb10 con 0x564f0b45e720 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f0a074beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b3a8400 con 0x564f0b45de00 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f09f98a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b458890 0x564f0b458cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f09f889f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 0x564f0b458350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f09f98a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b458890 0x564f0b458cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55910/0 (socket says 172.21.15.12:55910) 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f09f98a1700 1 -- 172.21.15.12:0/2413369104 learned_addr learned my addr 172.21.15.12:0/2413369104 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.644+0000 7f09f90a0700 1 --2- 172.21.15.12:0/2413369104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 0x564f0b45e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.645+0000 7f09f889f700 1 -- 172.21.15.12:0/2413369104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 msgr2=0x564f0b45e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.645+0000 7f09f889f700 1 --2- 172.21.15.12:0/2413369104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 0x564f0b45e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.645+0000 7f09f889f700 1 -- 172.21.15.12:0/2413369104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b458890 msgr2=0x564f0b458cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.645+0000 7f09f889f700 1 --2- 172.21.15.12:0/2413369104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b458890 0x564f0b458cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.645+0000 7f09f889f700 1 -- 172.21.15.12:0/2413369104 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f0b2293b0 con 0x564f0b45e720 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.645+0000 7f09f98a1700 1 --2- 172.21.15.12:0/2413369104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b458890 0x564f0b458cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.645+0000 7f09f889f700 1 --2- 172.21.15.12:0/2413369104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 0x564f0b458350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f09f0004300 tx=0x7f09f0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=412cd0b4c5c2c02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.645+0000 7f09df7fe700 1 -- 172.21.15.12:0/2413369104 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0003d10 con 0x564f0b45e720 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.645+0000 7f09df7fe700 1 -- 172.21.15.12:0/2413369104 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f0003e70 con 0x564f0b45e720 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.645+0000 7f09df7fe700 1 -- 172.21.15.12:0/2413369104 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f001e7f0 con 0x564f0b45e720 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 -- 172.21.15.12:0/2413369104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 msgr2=0x564f0b458350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 --2- 172.21.15.12:0/2413369104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 0x564f0b458350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f09f0004300 tx=0x7f09f0014000 comp rx=0 tx=0).stop 2024-05-08T23:53:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 -- 172.21.15.12:0/2413369104 shutdown_connections 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 --2- 172.21.15.12:0/2413369104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b458890 0x564f0b458cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 --2- 172.21.15.12:0/2413369104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 0x564f0b45e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 --2- 172.21.15.12:0/2413369104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 0x564f0b458350 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 -- 172.21.15.12:0/2413369104 >> 172.21.15.12:0/2413369104 conn(0x564f0b3b6210 msgr2=0x564f0b452df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 -- 172.21.15.12:0/2413369104 shutdown_connections 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 -- 172.21.15.12:0/2413369104 wait complete. 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 -- start start 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.646+0000 7f0a074beb80 1 Processor -- start 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 0x564f0b461bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45de00 0x564f0b4620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b462630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f0a074beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b344560 con 0x564f0b45e720 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f0a074beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b344200 con 0x564f0b45de00 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f0a074beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b22b6a0 con 0x564f0b458890 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f0a074beb80 10 client.0 _ll_register_callbacks cb 0x564f0b34db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f98a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b462630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f98a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b462630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55914/0 (socket says 172.21.15.12:55914) 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f889f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45de00 0x564f0b4620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f98a1700 1 -- 172.21.15.12:0/1916088017 learned_addr learned my addr 172.21.15.12:0/1916088017 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f90a0700 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 0x564f0b461bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f98a1700 1 -- 172.21.15.12:0/1916088017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 msgr2=0x564f0b461bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f98a1700 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 0x564f0b461bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f98a1700 1 -- 172.21.15.12:0/1916088017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45de00 msgr2=0x564f0b4620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f98a1700 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45de00 0x564f0b4620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f98a1700 1 -- 172.21.15.12:0/1916088017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f410e710 con 0x564f0b45e720 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.647+0000 7f09f889f700 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45de00 0x564f0b4620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f09f98a1700 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b462630 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f09ec007f70 tx=0x7f09ec00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1242a84b65f93a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f09dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f09dcff9700 1 -- 172.21.15.12:0/1916088017 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00e040 con 0x564f0b45e720 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f09dcff9700 1 -- 172.21.15.12:0/1916088017 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f09ec0076e0 con 0x564f0b45e720 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f0a074beb80 1 -- 172.21.15.12:0/1916088017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f0b301270 con 0x564f0b45e720 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f09dcff9700 1 -- 172.21.15.12:0/1916088017 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09ec009ba0 con 0x564f0b45e720 2024-05-08T23:53:04.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f0a074beb80 10 client.17262 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f0a074beb80 1 -- 172.21.15.12:0/1916088017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f09f0017070 con 0x564f0b45e720 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f09dcff9700 1 -- 172.21.15.12:0/1916088017 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f0b301270 con 0x564f0b45e720 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f09dcff9700 1 -- 172.21.15.12:0/1916088017 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f09ec023020 con 0x564f0b45e720 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f0a074beb80 10 client.17262 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.648+0000 7f0a074beb80 10 client.17262 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.649+0000 7f0a074beb80 20 client.17262 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.649+0000 7f0a074beb80 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f0b472270 0x564f0b474690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.649+0000 7f0a074beb80 4 client.17262 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.649+0000 7f0a074beb80 1 -- 172.21.15.12:0/1916088017 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f0b427810 con 0x564f0b472270 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.649+0000 7f09f90a0700 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f0b472270 0x564f0b474690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.649+0000 7f09f90a0700 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f0b472270 0x564f0b474690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca1d6ad17a2d74de server_cookie=d932ff84e1825aba in_seq=0 out_seq=0 2024-05-08T23:53:04.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.649+0000 7f09dcff9700 10 client.17262 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:53:04.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f09dcff9700 1 -- 172.21.15.12:0/1916088017 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564f0b427810 con 0x564f0b472270 2024-05-08T23:53:04.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f09dcff9700 10 client.17262 handle_command_reply: tid=0 2024-05-08T23:53:04.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 client.17262 shutdown 2024-05-08T23:53:04.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 -- 172.21.15.12:0/1916088017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 msgr2=0x564f0b462630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b462630 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f09ec007f70 tx=0x7f09ec00ba50 comp rx=0 tx=0).stop 2024-05-08T23:53:04.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 -- 172.21.15.12:0/1916088017 shutdown_connections 2024-05-08T23:53:04.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b462630 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f0b472270 0x564f0b474690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 0x564f0b461bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 --2- 172.21.15.12:0/1916088017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45de00 0x564f0b4620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f09dcff9700 0 client.17262 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:53:04.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 -- 172.21.15.12:0/1916088017 >> 172.21.15.12:0/1916088017 conn(0x564f0b3b6210 msgr2=0x564f0b4525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 -- 172.21.15.12:0/1916088017 shutdown_connections 2024-05-08T23:53:04.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 1 -- 172.21.15.12:0/1916088017 wait complete. 2024-05-08T23:53:04.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.650+0000 7f0a074beb80 20 client.17262 trim_cache size 0 max 16384 2024-05-08T23:53:04.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.665+0000 7f0a074beb80 1 Processor -- start 2024-05-08T23:53:04.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f0a074beb80 1 -- start start 2024-05-08T23:53:04.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b458890 0x564f0b46dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 0x564f0b452d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b460480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f0a074beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b4631b0 con 0x564f0b45e720 2024-05-08T23:53:04.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f0a074beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b44fdc0 con 0x564f0b458890 2024-05-08T23:53:04.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f0a074beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b4619a0 con 0x564f0b45de00 2024-05-08T23:53:04.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f98a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b460480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f98a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b460480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55916/0 (socket says 172.21.15.12:55916) 2024-05-08T23:53:04.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f98a1700 1 -- 172.21.15.12:0/4044300309 learned_addr learned my addr 172.21.15.12:0/4044300309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f90a0700 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b458890 0x564f0b46dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f889f700 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 0x564f0b452d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f90a0700 1 -- 172.21.15.12:0/4044300309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 msgr2=0x564f0b452d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f90a0700 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 0x564f0b452d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f90a0700 1 -- 172.21.15.12:0/4044300309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 msgr2=0x564f0b460480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f90a0700 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b460480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f90a0700 1 -- 172.21.15.12:0/4044300309 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f0b1d5b60 con 0x564f0b458890 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f889f700 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 0x564f0b452d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f98a1700 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b460480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.666+0000 7f09f90a0700 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b458890 0x564f0b46dc20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f09e80ca550 tx=0x7f09e8096380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=173e62c92496043e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.667+0000 7f09dffff700 1 -- 172.21.15.12:0/4044300309 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e809dd40 con 0x564f0b458890 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.667+0000 7f09dffff700 1 -- 172.21.15.12:0/4044300309 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09e80a2390 con 0x564f0b458890 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.667+0000 7f09dffff700 1 -- 172.21.15.12:0/4044300309 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e8099400 con 0x564f0b458890 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.667+0000 7f0a074beb80 1 -- 172.21.15.12:0/4044300309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b458890 msgr2=0x564f0b46dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.668+0000 7f0a074beb80 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b458890 0x564f0b46dc20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f09e80ca550 tx=0x7f09e8096380 comp rx=0 tx=0).stop 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.668+0000 7f0a074beb80 1 -- 172.21.15.12:0/4044300309 shutdown_connections 2024-05-08T23:53:04.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.668+0000 7f0a074beb80 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45e720 0x564f0b460480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.668+0000 7f0a074beb80 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b458890 0x564f0b46dc20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.668+0000 7f0a074beb80 1 --2- 172.21.15.12:0/4044300309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b45de00 0x564f0b452d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.668+0000 7f0a074beb80 1 -- 172.21.15.12:0/4044300309 >> 172.21.15.12:0/4044300309 conn(0x564f0b3b6210 msgr2=0x564f0b45bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.668+0000 7f0a074beb80 1 -- 172.21.15.12:0/4044300309 shutdown_connections 2024-05-08T23:53:04.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.668+0000 7f0a074beb80 1 -- 172.21.15.12:0/4044300309 wait complete. 2024-05-08T23:53:04.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.668+0000 7f0a074beb80 1 -- start start 2024-05-08T23:53:04.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.668+0000 7f0a074beb80 1 Processor -- start 2024-05-08T23:53:04.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 0x564f0b467150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45de00 0x564f0b467690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f0a074beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 0x564f0b467bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f0a074beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b469a00 con 0x564f0b45de00 2024-05-08T23:53:04.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f0a074beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b4c7770 con 0x564f0b45e720 2024-05-08T23:53:04.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f0a074beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0b4c78b0 con 0x564f0b458890 2024-05-08T23:53:04.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f0a074beb80 10 client.0 _ll_register_callbacks cb 0x564f0b34db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:04.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f90a0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 0x564f0b467150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f889f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45de00 0x564f0b467690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f90a0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 0x564f0b467150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41612/0 (socket says 172.21.15.12:41612) 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f90a0700 1 -- 172.21.15.12:0/2935259123 learned_addr learned my addr 172.21.15.12:0/2935259123 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f98a1700 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 0x564f0b467bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f98a1700 1 -- 172.21.15.12:0/2935259123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 msgr2=0x564f0b467150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f98a1700 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 0x564f0b467150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f98a1700 1 -- 172.21.15.12:0/2935259123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45de00 msgr2=0x564f0b467690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f98a1700 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45de00 0x564f0b467690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f98a1700 1 -- 172.21.15.12:0/2935259123 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f0017070 con 0x564f0b45e720 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f90a0700 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 0x564f0b467150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f889f700 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45de00 0x564f0b467690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09f98a1700 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 0x564f0b467bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f09ec002230 tx=0x7f09ec005a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12d0e6b9d67434b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:53:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09dcff9700 1 -- 172.21.15.12:0/2935259123 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0078b0 con 0x564f0b45e720 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f0a074beb80 1 -- 172.21.15.12:0/2935259123 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09f0003e70 con 0x564f0b45e720 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.669+0000 7f09dcff9700 1 -- 172.21.15.12:0/2935259123 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f09ec003a00 con 0x564f0b45e720 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f09dcff9700 1 -- 172.21.15.12:0/2935259123 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09ec00bac0 con 0x564f0b45e720 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f0a074beb80 10 client.27056 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f0a074beb80 1 -- 172.21.15.12:0/2935259123 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f09f41a4630 con 0x564f0b45e720 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f09dcff9700 1 -- 172.21.15.12:0/2935259123 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f0003e70 con 0x564f0b45e720 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f09dcff9700 1 -- 172.21.15.12:0/2935259123 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f09ec034020 con 0x564f0b45e720 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f0a074beb80 10 client.27056 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f0a074beb80 10 client.27056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f0a074beb80 20 client.27056 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:04.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f0a074beb80 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f0b4c8c20 0x564f0b4cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f0a074beb80 4 client.27056 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f0a074beb80 1 -- 172.21.15.12:0/2935259123 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f0b3a0ec0 con 0x564f0b4c8c20 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.670+0000 7f09f90a0700 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f0b4c8c20 0x564f0b4cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.671+0000 7f09dcff9700 10 client.27056 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.671+0000 7f09f90a0700 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f0b4c8c20 0x564f0b4cb040 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bd8a034c69b87aa server_cookie=afaf3fd376546f09 in_seq=0 out_seq=0 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.671+0000 7f09dcff9700 1 -- 172.21.15.12:0/2935259123 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f0b3a0ec0 con 0x564f0b4c8c20 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.671+0000 7f09dcff9700 10 client.27056 handle_command_reply: tid=0 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.671+0000 7f0a074beb80 1 client.27056 shutdown 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 1 -- 172.21.15.12:0/2935259123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 msgr2=0x564f0b467bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 0x564f0b467bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f09ec002230 tx=0x7f09ec005a10 comp rx=0 tx=0).stop 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 1 -- 172.21.15.12:0/2935259123 shutdown_connections 2024-05-08T23:53:04.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f0b45e720 0x564f0b467bd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f0b4c8c20 0x564f0b4cb040 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f0b458890 0x564f0b467150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 1 --2- 172.21.15.12:0/2935259123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f0b45de00 0x564f0b467690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f09dcff9700 0 client.27056 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:53:04.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 1 -- 172.21.15.12:0/2935259123 >> 172.21.15.12:0/2935259123 conn(0x564f0b3b6210 msgr2=0x564f0b4603c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 1 -- 172.21.15.12:0/2935259123 shutdown_connections 2024-05-08T23:53:04.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 1 -- 172.21.15.12:0/2935259123 wait complete. 2024-05-08T23:53:04.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f0a074beb80 20 client.27056 trim_cache size 0 max 16384 2024-05-08T23:53:04.674 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:53:04.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f09fb304700 1 -- 172.21.15.12:0/1883270763 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09e00a1290 msgr2=0x7f09e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f09fb304700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09e00a1290 0x7f09e00a3750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09f4198ab0 tx=0x7f09f0009260 comp rx=0 tx=0).stop 2024-05-08T23:53:04.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f09fb304700 1 -- 172.21.15.12:0/1883270763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09f40ffad0 msgr2=0x7f09f4197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.672+0000 7f09fb304700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09f40ffad0 0x7f09f4197560 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f09e800e9f0 tx=0x7f09e800ed00 comp rx=0 tx=0).stop 2024-05-08T23:53:04.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.673+0000 7f09fb304700 1 -- 172.21.15.12:0/1883270763 shutdown_connections 2024-05-08T23:53:04.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.673+0000 7f09fb304700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f09e00a1290 0x7f09e00a3750 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.673+0000 7f09fb304700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09f419be40 0x7f09f419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.673+0000 7f09fb304700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09f40ffad0 0x7f09f4197560 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.673+0000 7f09fb304700 1 --2- 172.21.15.12:0/1883270763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f41062e0 0x7f09f4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.673+0000 7f09fb304700 1 -- 172.21.15.12:0/1883270763 >> 172.21.15.12:0/1883270763 conn(0x7f09f40750c0 msgr2=0x7f09f40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.673+0000 7f09fb304700 1 -- 172.21.15.12:0/1883270763 shutdown_connections 2024-05-08T23:53:04.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.673+0000 7f09fb304700 1 -- 172.21.15.12:0/1883270763 wait complete. 2024-05-08T23:53:04.682 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:53:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:04 smithi204 ceph-mon[47373]: pgmap v204: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 278 op/s 2024-05-08T23:53:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:04 smithi204 ceph-mon[47373]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T23:53:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:04 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1359187722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:53:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:04 smithi160 ceph-mon[46152]: pgmap v204: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 278 op/s 2024-05-08T23:53:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:04 smithi160 ceph-mon[46152]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T23:53:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:04 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1359187722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:53:04.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.834+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/3723202633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b40682e0 msgr2=0x7fb5b40686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.834+0000 7fb5bb4e8700 1 --2- 172.21.15.12:0/3723202633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b40682e0 0x7fb5b40686c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00099c0 tx=0x7fb5b0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.834+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/3723202633 shutdown_connections 2024-05-08T23:53:04.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.834+0000 7fb5bb4e8700 1 --2- 172.21.15.12:0/3723202633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b4106290 0x7fb5b410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.834+0000 7fb5bb4e8700 1 --2- 172.21.15.12:0/3723202633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b40682e0 0x7fb5b40686c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.834+0000 7fb5bb4e8700 1 --2- 172.21.15.12:0/3723202633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4068c90 0x7fb5b4105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.834+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/3723202633 >> 172.21.15.12:0/3723202633 conn(0x7fb5b4076f40 msgr2=0x7fb5b4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.834+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/3723202633 shutdown_connections 2024-05-08T23:53:04.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.834+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/3723202633 wait complete. 2024-05-08T23:53:04.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.835+0000 7fb5bb4e8700 1 Processor -- start 2024-05-08T23:53:04.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.835+0000 7fb5bb4e8700 1 -- start start 2024-05-08T23:53:04.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.835+0000 7fb5bb4e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b40682e0 0x7fb5b40ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.835+0000 7fb5bb4e8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4106290 0x7fb5b41003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.835+0000 7fb5bb4e8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b41008e0 0x7fb5b41a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.835+0000 7fb5bb4e8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b41a5bc0 con 0x7fb5b40682e0 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.835+0000 7fb5bb4e8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b41a5d00 con 0x7fb5b41008e0 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.835+0000 7fb5bb4e8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b41a5ec0 con 0x7fb5b4106290 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b9284700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b40682e0 0x7fb5b40ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b9284700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b40682e0 0x7fb5b40ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55954/0 (socket says 172.21.15.12:55954) 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b9284700 1 -- 172.21.15.12:0/2277446086 learned_addr learned my addr 172.21.15.12:0/2277446086 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4106290 0x7fb5b41003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b9a85700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b41008e0 0x7fb5b41a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b9284700 1 -- 172.21.15.12:0/2277446086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4106290 msgr2=0x7fb5b41003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b9284700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4106290 0x7fb5b41003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b9284700 1 -- 172.21.15.12:0/2277446086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b41008e0 msgr2=0x7fb5b41a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b9284700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b41008e0 0x7fb5b41a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b9284700 1 -- 172.21.15.12:0/2277446086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b00095f0 con 0x7fb5b40682e0 2024-05-08T23:53:04.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5b9284700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b40682e0 0x7fb5b40ffe60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000f690 tx=0x7fb5b000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5aa7fc700 1 -- 172.21.15.12:0/2277446086 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b001c070 con 0x7fb5b40682e0 2024-05-08T23:53:04.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/2277446086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b41a6140 con 0x7fb5b40682e0 2024-05-08T23:53:04.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5aa7fc700 1 -- 172.21.15.12:0/2277446086 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5b000fcf0 con 0x7fb5b40682e0 2024-05-08T23:53:04.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5aa7fc700 1 -- 172.21.15.12:0/2277446086 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0022c60 con 0x7fb5b40682e0 2024-05-08T23:53:04.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.836+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/2277446086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b41a6570 con 0x7fb5b40682e0 2024-05-08T23:53:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.839+0000 7fb5aa7fc700 1 -- 172.21.15.12:0/2277446086 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb5b0020070 con 0x7fb5b40682e0 2024-05-08T23:53:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.840+0000 7fb5aa7fc700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a1290 0x7fb5a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.840+0000 7fb5aa7fc700 1 -- 172.21.15.12:0/2277446086 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb5b00cdc60 con 0x7fb5b40682e0 2024-05-08T23:53:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.840+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a1290 0x7fb5a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.841+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a1290 0x7fb5a00a3750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4101540 tx=0x7fb5a4009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.841+0000 7fb5c76a2b80 1 Processor -- start 2024-05-08T23:53:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.841+0000 7fb5c76a2b80 1 -- start start 2024-05-08T23:53:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.841+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 0x55ddf3a131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.841+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 0x55ddf3a0d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.841+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.841+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf3964150 con 0x55ddf3a13720 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.841+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf3961b10 con 0x55ddf3a0d890 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.841+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf395d400 con 0x55ddf3a12e00 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b9a85700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b8a83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 0x55ddf3a0d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b9a85700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39108/0 (socket says 172.21.15.12:39108) 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b9a85700 1 -- 172.21.15.12:0/269196659 learned_addr learned my addr 172.21.15.12:0/269196659 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b9284700 1 --2- 172.21.15.12:0/269196659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 0x55ddf3a131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b9a85700 1 -- 172.21.15.12:0/269196659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 msgr2=0x55ddf3a131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b9a85700 1 --2- 172.21.15.12:0/269196659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 0x55ddf3a131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b9a85700 1 -- 172.21.15.12:0/269196659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 msgr2=0x55ddf3a0d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b9a85700 1 --2- 172.21.15.12:0/269196659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 0x55ddf3a0d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b9a85700 1 -- 172.21.15.12:0/269196659 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf37de3b0 con 0x55ddf3a0d890 2024-05-08T23:53:04.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb5b9a85700 1 --2- 172.21.15.12:0/269196659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0dcd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac011680 tx=0x7fb5ac009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b561a18963e4d57d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.842+0000 7fb59f7fe700 1 -- 172.21.15.12:0/269196659 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac00e040 con 0x55ddf3a0d890 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.843+0000 7fb59f7fe700 1 -- 172.21.15.12:0/269196659 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5ac00f040 con 0x55ddf3a0d890 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.843+0000 7fb59f7fe700 1 -- 172.21.15.12:0/269196659 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac004860 con 0x55ddf3a0d890 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.843+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/269196659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 msgr2=0x55ddf3a0dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.843+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/269196659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0dcd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac011680 tx=0x7fb5ac009920 comp rx=0 tx=0).stop 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.843+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/269196659 shutdown_connections 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.843+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/269196659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0dcd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.843+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/269196659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 0x55ddf3a131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.843+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/269196659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 0x55ddf3a0d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.843+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/269196659 >> 172.21.15.12:0/269196659 conn(0x55ddf396b210 msgr2=0x55ddf3a07df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.844+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/269196659 shutdown_connections 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.844+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/269196659 wait complete. 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.844+0000 7fb5c76a2b80 1 -- start start 2024-05-08T23:53:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.844+0000 7fb5c76a2b80 1 Processor -- start 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.844+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a16a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.844+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 0x55ddf3a16fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.844+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 0x55ddf3a174f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.844+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf38f9560 con 0x55ddf3a13720 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.844+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf38f9200 con 0x55ddf3a0d890 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.844+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf37e06a0 con 0x55ddf3a12e00 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5c76a2b80 10 client.0 _ll_register_callbacks cb 0x55ddf3902b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b9a85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 0x55ddf3a174f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b9284700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a16a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b8a83700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 0x55ddf3a16fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b9a85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 0x55ddf3a174f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55960/0 (socket says 172.21.15.12:55960) 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b9284700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a16a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39118/0 (socket says 172.21.15.12:39118) 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b9a85700 1 -- 172.21.15.12:0/4031416167 learned_addr learned my addr 172.21.15.12:0/4031416167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b8a83700 1 -- 172.21.15.12:0/4031416167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 msgr2=0x55ddf3a16a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a16a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b8a83700 1 -- 172.21.15.12:0/4031416167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 msgr2=0x55ddf3a174f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 0x55ddf3a174f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b8a83700 1 -- 172.21.15.12:0/4031416167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ac011330 con 0x55ddf3a12e00 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b9284700 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a16a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 0x55ddf3a16fb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40097c0 tx=0x7fb5a40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96f0d90de0b34e6b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb59cff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:53:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb59cff9700 1 -- 172.21.15.12:0/4031416167 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4022070 con 0x55ddf3a12e00 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb59cff9700 1 -- 172.21.15.12:0/4031416167 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb5a40044f0 con 0x55ddf3a12e00 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.845+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/4031416167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddf38b6270 con 0x55ddf3a12e00 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb59cff9700 1 -- 172.21.15.12:0/4031416167 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5a4003d40 con 0x55ddf3a12e00 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb5c76a2b80 10 client.27067 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/4031416167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ddf378ab60 con 0x55ddf3a12e00 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb59cff9700 1 -- 172.21.15.12:0/4031416167 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ddf38b6270 con 0x55ddf3a12e00 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb59cff9700 1 -- 172.21.15.12:0/4031416167 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb5a401d530 con 0x55ddf3a12e00 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb5c76a2b80 10 client.27067 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb5c76a2b80 10 client.27067 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb5c76a2b80 20 client.27067 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ddf3a27250 0x55ddf3a29670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb5c76a2b80 4 client.27067 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/4031416167 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddf39dc810 con 0x55ddf3a27250 2024-05-08T23:53:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.846+0000 7fb5b9284700 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ddf3a27250 0x55ddf3a29670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb59cff9700 10 client.27067 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb5b9284700 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ddf3a27250 0x55ddf3a29670 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=364541e220bc6c4b server_cookie=531b3c00f5953722 in_seq=0 out_seq=0 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb59cff9700 1 -- 172.21.15.12:0/4031416167 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ddf39dc810 con 0x55ddf3a27250 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb59cff9700 10 client.27067 handle_command_reply: tid=0 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb5c76a2b80 1 client.27067 shutdown 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/4031416167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 msgr2=0x55ddf3a16fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 0x55ddf3a16fb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40097c0 tx=0x7fb5a40047c0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/4031416167 shutdown_connections 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ddf3a27250 0x55ddf3a29670 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a13720 0x55ddf3a174f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a0d890 0x55ddf3a16a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb59cff9700 0 client.27067 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/4031416167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a12e00 0x55ddf3a16fb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.847+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/4031416167 >> 172.21.15.12:0/4031416167 conn(0x55ddf396b210 msgr2=0x55ddf3a075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.848+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/4031416167 shutdown_connections 2024-05-08T23:53:04.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.848+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/4031416167 wait complete. 2024-05-08T23:53:04.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.848+0000 7fb5c76a2b80 20 client.27067 trim_cache size 0 max 16384 2024-05-08T23:53:04.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.862+0000 7fb5c76a2b80 1 Processor -- start 2024-05-08T23:53:04.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.862+0000 7fb5c76a2b80 1 -- start start 2024-05-08T23:53:04.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.862+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.862+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a29640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.862+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 0x55ddf3a07d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.862+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf3a24920 con 0x55ddf3a12e00 2024-05-08T23:53:04.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.862+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf3a2e860 con 0x55ddf3a13720 2024-05-08T23:53:04.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.862+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf3a18040 con 0x55ddf3a0d890 2024-05-08T23:53:04.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b8a83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a29640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b9284700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b8a83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a29640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55964/0 (socket says 172.21.15.12:55964) 2024-05-08T23:53:04.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b9284700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41666/0 (socket says 172.21.15.12:41666) 2024-05-08T23:53:04.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b8a83700 1 -- 172.21.15.12:0/2555132676 learned_addr learned my addr 172.21.15.12:0/2555132676 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b9a85700 1 --2- 172.21.15.12:0/2555132676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 0x55ddf3a07d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b8a83700 1 -- 172.21.15.12:0/2555132676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 msgr2=0x55ddf3a0d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/2555132676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b8a83700 1 -- 172.21.15.12:0/2555132676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 msgr2=0x55ddf3a07d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/2555132676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 0x55ddf3a07d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b8a83700 1 -- 172.21.15.12:0/2555132676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf3923560 con 0x55ddf3a12e00 2024-05-08T23:53:04.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b9284700 1 --2- 172.21.15.12:0/2555132676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:04.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b9a85700 1 --2- 172.21.15.12:0/2555132676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 0x55ddf3a07d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:04.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/2555132676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a29640 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4016040 tx=0x7fb5a4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6678416d283f61dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb59ffff700 1 -- 172.21.15.12:0/2555132676 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4022040 con 0x55ddf3a12e00 2024-05-08T23:53:04.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb59ffff700 1 -- 172.21.15.12:0/2555132676 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a401dcb0 con 0x55ddf3a12e00 2024-05-08T23:53:04.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.863+0000 7fb59ffff700 1 -- 172.21.15.12:0/2555132676 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4027c60 con 0x55ddf3a12e00 2024-05-08T23:53:04.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.864+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/2555132676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 msgr2=0x55ddf3a29640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.864+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/2555132676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a29640 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4016040 tx=0x7fb5a4004570 comp rx=0 tx=0).stop 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.864+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/2555132676 shutdown_connections 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.864+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/2555132676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 0x55ddf3a07d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.864+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/2555132676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 0x55ddf3a0d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.864+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/2555132676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a29640 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.864+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/2555132676 >> 172.21.15.12:0/2555132676 conn(0x55ddf396b210 msgr2=0x55ddf3a04210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.864+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/2555132676 shutdown_connections 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.864+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/2555132676 wait complete. 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5c76a2b80 1 -- start start 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5c76a2b80 1 Processor -- start 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 0x55ddf3a1e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a1e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5c76a2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 0x55ddf3a0e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf3a0f0b0 con 0x55ddf3a12e00 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf3a0f220 con 0x55ddf3a13720 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5c76a2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf3a0f3e0 con 0x55ddf3a0d890 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5c76a2b80 10 client.0 _ll_register_callbacks cb 0x55ddf3902b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5b8a83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a1e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5b8a83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a1e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55980/0 (socket says 172.21.15.12:55980) 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5b8a83700 1 -- 172.21.15.12:0/1311683430 learned_addr learned my addr 172.21.15.12:0/1311683430 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5b9284700 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 0x55ddf3a1e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.865+0000 7fb5b9a85700 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 0x55ddf3a0e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb5b8a83700 1 -- 172.21.15.12:0/1311683430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 msgr2=0x55ddf3a1e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 0x55ddf3a1e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb5b8a83700 1 -- 172.21.15.12:0/1311683430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 msgr2=0x55ddf3a0e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:04.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 0x55ddf3a0e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb5b8a83700 1 -- 172.21.15.12:0/1311683430 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b41a6570 con 0x55ddf3a12e00 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb5b8a83700 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a1e6b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb5a401d960 tx=0x7fb5a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b8dd300aa1d59c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb59cff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb59cff9700 1 -- 172.21.15.12:0/1311683430 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4022040 con 0x55ddf3a12e00 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb59cff9700 1 -- 172.21.15.12:0/1311683430 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb5a40278f0 con 0x55ddf3a12e00 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/1311683430 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5ac00f040 con 0x55ddf3a12e00 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb59cff9700 1 -- 172.21.15.12:0/1311683430 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5a4027af0 con 0x55ddf3a12e00 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb5c76a2b80 10 client.17286 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/1311683430 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ddf37de3b0 con 0x55ddf3a12e00 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.866+0000 7fb59cff9700 1 -- 172.21.15.12:0/1311683430 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a404d080 con 0x55ddf3a12e00 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb59cff9700 1 -- 172.21.15.12:0/1311683430 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb5a403f030 con 0x55ddf3a12e00 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5c76a2b80 10 client.17286 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:04.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5c76a2b80 10 client.17286 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5c76a2b80 20 client.17286 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ddf3a7d750 0x55ddf3a7fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5c76a2b80 4 client.17286 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/1311683430 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ddf3955ec0 con 0x55ddf3a7d750 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5b9284700 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ddf3a7d750 0x55ddf3a7fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5b9284700 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ddf3a7d750 0x55ddf3a7fb40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c8f8f2db7c6596f server_cookie=435890b4c9a77d5a in_seq=0 out_seq=0 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb59cff9700 10 client.17286 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb59cff9700 1 -- 172.21.15.12:0/1311683430 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ddf3955ec0 con 0x55ddf3a7d750 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb59cff9700 10 client.17286 handle_command_reply: tid=0 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5c76a2b80 1 client.17286 shutdown 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/1311683430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 msgr2=0x55ddf3a1e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.867+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a1e6b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb5a401d960 tx=0x7fb5a4014000 comp rx=0 tx=0).stop 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/1311683430 shutdown_connections 2024-05-08T23:53:04.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ddf3a7d750 0x55ddf3a7fb40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ddf3a13720 0x55ddf3a0e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ddf3a0d890 0x55ddf3a1e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb59cff9700 0 client.17286 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:53:04.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5c76a2b80 1 --2- 172.21.15.12:0/1311683430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf3a12e00 0x55ddf3a1e6b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/1311683430 >> 172.21.15.12:0/1311683430 conn(0x55ddf396b210 msgr2=0x55ddf3a10bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/1311683430 shutdown_connections 2024-05-08T23:53:04.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5c76a2b80 1 -- 172.21.15.12:0/1311683430 wait complete. 2024-05-08T23:53:04.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5c76a2b80 20 client.17286 trim_cache size 0 max 16384 2024-05-08T23:53:04.870 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:53:04.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/2277446086 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a1290 msgr2=0x7fb5a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5bb4e8700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a1290 0x7fb5a00a3750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4101540 tx=0x7fb5a4009260 comp rx=0 tx=0).stop 2024-05-08T23:53:04.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/2277446086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b40682e0 msgr2=0x7fb5b40ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:04.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5bb4e8700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b40682e0 0x7fb5b40ffe60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000f690 tx=0x7fb5b000bfd0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/2277446086 shutdown_connections 2024-05-08T23:53:04.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5bb4e8700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5a00a1290 0x7fb5a00a3750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5bb4e8700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5b41008e0 0x7fb5b41a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5bb4e8700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b40682e0 0x7fb5b40ffe60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5bb4e8700 1 --2- 172.21.15.12:0/2277446086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5b4106290 0x7fb5b41003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:04.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.868+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/2277446086 >> 172.21.15.12:0/2277446086 conn(0x7fb5b4076f40 msgr2=0x7fb5b40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:04.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.869+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/2277446086 shutdown_connections 2024-05-08T23:53:04.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:04.869+0000 7fb5bb4e8700 1 -- 172.21.15.12:0/2277446086 wait complete. 2024-05-08T23:53:04.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 422725 9.77 MB/sec execute 180 sec latency 44.223 ms 2024-05-08T23:53:04.876 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:04 smithi012 ceph-mon[42503]: pgmap v204: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 278 op/s 2024-05-08T23:53:04.876 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:04 smithi012 ceph-mon[42503]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T23:53:04.876 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:04 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1359187722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:53:04.877 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:53:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.037+0000 7f6e559a7700 1 -- 172.21.15.12:0/2153496910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e5006a110 msgr2=0x7f6e50105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.037+0000 7f6e559a7700 1 --2- 172.21.15.12:0/2153496910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e5006a110 0x7f6e50105bc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6e440099c0 tx=0x7f6e44009cd0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.037+0000 7f6e559a7700 1 -- 172.21.15.12:0/2153496910 shutdown_connections 2024-05-08T23:53:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.037+0000 7f6e559a7700 1 --2- 172.21.15.12:0/2153496910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e501062f0 0x7f6e5010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.037+0000 7f6e559a7700 1 --2- 172.21.15.12:0/2153496910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50069760 0x7f6e50069b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.037+0000 7f6e559a7700 1 --2- 172.21.15.12:0/2153496910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e5006a110 0x7f6e50105bc0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.037+0000 7f6e559a7700 1 -- 172.21.15.12:0/2153496910 >> 172.21.15.12:0/2153496910 conn(0x7f6e50076fa0 msgr2=0x7f6e500773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.037+0000 7f6e559a7700 1 -- 172.21.15.12:0/2153496910 shutdown_connections 2024-05-08T23:53:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.037+0000 7f6e559a7700 1 -- 172.21.15.12:0/2153496910 wait complete. 2024-05-08T23:53:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.038+0000 7f6e559a7700 1 Processor -- start 2024-05-08T23:53:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.038+0000 7f6e559a7700 1 -- start start 2024-05-08T23:53:05.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.038+0000 7f6e559a7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50069760 0x7f6e500ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.038+0000 7f6e559a7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e501062f0 0x7f6e50100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.038+0000 7f6e559a7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50100890 0x7f6e501a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e559a7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e501a5b30 con 0x7f6e50100890 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e559a7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e501a5ca0 con 0x7f6e50069760 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e559a7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e501a5e60 con 0x7f6e501062f0 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50100890 0x7f6e501a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50069760 0x7f6e500ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50100890 0x7f6e501a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55986/0 (socket says 172.21.15.12:55986) 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50069760 0x7f6e500ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39170/0 (socket says 172.21.15.12:39170) 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4f7fe700 1 -- 172.21.15.12:0/2552650339 learned_addr learned my addr 172.21.15.12:0/2552650339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4e7fc700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e501062f0 0x7f6e50100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4f7fe700 1 -- 172.21.15.12:0/2552650339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e501062f0 msgr2=0x7f6e50100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4f7fe700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e501062f0 0x7f6e50100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4f7fe700 1 -- 172.21.15.12:0/2552650339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50069760 msgr2=0x7f6e500ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4f7fe700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50069760 0x7f6e500ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4f7fe700 1 -- 172.21.15.12:0/2552650339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40009520 con 0x7f6e50100890 2024-05-08T23:53:05.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4e7fc700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e501062f0 0x7f6e50100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:05.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4effd700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50069760 0x7f6e500ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:05.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e4f7fe700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50100890 0x7f6e501a5560 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800ef30 tx=0x7f6e4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:05.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.039+0000 7f6e549a5700 1 -- 172.21.15.12:0/2552650339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48018070 con 0x7f6e50100890 2024-05-08T23:53:05.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.040+0000 7f6e559a7700 1 -- 172.21.15.12:0/2552650339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e440095f0 con 0x7f6e50100890 2024-05-08T23:53:05.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.040+0000 7f6e549a5700 1 -- 172.21.15.12:0/2552650339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4800f040 con 0x7f6e50100890 2024-05-08T23:53:05.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.040+0000 7f6e549a5700 1 -- 172.21.15.12:0/2552650339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800cd00 con 0x7f6e50100890 2024-05-08T23:53:05.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.040+0000 7f6e559a7700 1 -- 172.21.15.12:0/2552650339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e501a6430 con 0x7f6e50100890 2024-05-08T23:53:05.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.042+0000 7f6e549a5700 1 -- 172.21.15.12:0/2552650339 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6e48003a00 con 0x7f6e50100890 2024-05-08T23:53:05.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.043+0000 7f6e549a5700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e380a1290 0x7f6e380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.043+0000 7f6e549a5700 1 -- 172.21.15.12:0/2552650339 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6e480cc830 con 0x7f6e50100890 2024-05-08T23:53:05.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.043+0000 7f6e4effd700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e380a1290 0x7f6e380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.043+0000 7f6e4effd700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e380a1290 0x7f6e380a3750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009fd0 tx=0x7f6e40009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:05.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e61b61b80 1 Processor -- start 2024-05-08T23:53:05.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e61b61b80 1 -- start start 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaada1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaad4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaad4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaa2b150 con 0x55edaaad4890 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaa28b10 con 0x55edaaad9e00 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaa24400 con 0x55edaaada720 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e4effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaada1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaad4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e4e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaad4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.044+0000 7f6e4effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaada1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39176/0 (socket says 172.21.15.12:39176) 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaad4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55998/0 (socket says 172.21.15.12:55998) 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e4e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaad4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41718/0 (socket says 172.21.15.12:41718) 2024-05-08T23:53:05.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e4effd700 1 -- 172.21.15.12:0/1856504542 learned_addr learned my addr 172.21.15.12:0/1856504542 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:05.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e4effd700 1 -- 172.21.15.12:0/1856504542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 msgr2=0x55edaaad4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:05.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e4effd700 1 --2- 172.21.15.12:0/1856504542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaad4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e4effd700 1 -- 172.21.15.12:0/1856504542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 msgr2=0x55edaaad4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:05.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e4effd700 1 --2- 172.21.15.12:0/1856504542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaad4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e4effd700 1 -- 172.21.15.12:0/1856504542 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edaa8a53b0 con 0x55edaaad9e00 2024-05-08T23:53:05.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e4effd700 1 --2- 172.21.15.12:0/1856504542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaada1e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e40012fd0 tx=0x7f6e40014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac48944c70e0c186 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e35ffb700 1 -- 172.21.15.12:0/1856504542 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4001b600 con 0x55edaaad9e00 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e35ffb700 1 -- 172.21.15.12:0/1856504542 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4000ee40 con 0x55edaaad9e00 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.045+0000 7f6e35ffb700 1 -- 172.21.15.12:0/1856504542 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40026b40 con 0x55edaaad9e00 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.046+0000 7f6e61b61b80 1 -- 172.21.15.12:0/1856504542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 msgr2=0x55edaaada1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.046+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/1856504542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaada1e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e40012fd0 tx=0x7f6e40014000 comp rx=0 tx=0).stop 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.046+0000 7f6e61b61b80 1 -- 172.21.15.12:0/1856504542 shutdown_connections 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.046+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/1856504542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaad4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.046+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/1856504542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaada1e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.046+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/1856504542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaad4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.046+0000 7f6e61b61b80 1 -- 172.21.15.12:0/1856504542 >> 172.21.15.12:0/1856504542 conn(0x55edaaa32210 msgr2=0x55edaaacedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.046+0000 7f6e61b61b80 1 -- 172.21.15.12:0/1856504542 shutdown_connections 2024-05-08T23:53:05.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.046+0000 7f6e61b61b80 1 -- 172.21.15.12:0/1856504542 wait complete. 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.047+0000 7f6e61b61b80 1 -- start start 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.047+0000 7f6e61b61b80 1 Processor -- start 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaaddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaade100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaade640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaa9c0560 con 0x55edaaad4890 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaa9c0200 con 0x55edaaad9e00 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaa8a76a0 con 0x55edaaada720 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e61b61b80 10 client.0 _ll_register_callbacks cb 0x55edaa9c9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaaddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaade100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaaddbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56004/0 (socket says 172.21.15.12:56004) 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaade640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4effd700 1 -- 172.21.15.12:0/2369194064 learned_addr learned my addr 172.21.15.12:0/2369194064 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4e7fc700 1 -- 172.21.15.12:0/2369194064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 msgr2=0x55edaaade640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4e7fc700 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaade640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4e7fc700 1 -- 172.21.15.12:0/2369194064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 msgr2=0x55edaaaddbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4e7fc700 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaaddbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4e7fc700 1 -- 172.21.15.12:0/2369194064 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e40017050 con 0x55edaaad9e00 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4effd700 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaaddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4f7fe700 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaade640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e4e7fc700 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaade100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e440098e0 tx=0x7f6e440052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5b4c82c88e9c77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.048+0000 7f6e2effd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e2effd700 1 -- 172.21.15.12:0/2369194064 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4401c070 con 0x55edaaad9e00 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e2effd700 1 -- 172.21.15.12:0/2369194064 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6e44003880 con 0x55edaaad9e00 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e61b61b80 1 -- 172.21.15.12:0/2369194064 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edaa97d270 con 0x55edaaad9e00 2024-05-08T23:53:05.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e2effd700 1 -- 172.21.15.12:0/2369194064 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e44005720 con 0x55edaaad9e00 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e61b61b80 10 client.27086 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e61b61b80 1 -- 172.21.15.12:0/2369194064 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55edaa851b60 con 0x55edaaad9e00 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e2effd700 1 -- 172.21.15.12:0/2369194064 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e44032020 con 0x55edaaad9e00 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e2effd700 1 -- 172.21.15.12:0/2369194064 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6e4400fb50 con 0x55edaaad9e00 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e61b61b80 10 client.27086 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e61b61b80 10 client.27086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e61b61b80 20 client.27086 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55edaaaee0c0 0x55edaaaf04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e61b61b80 4 client.27086 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e61b61b80 1 -- 172.21.15.12:0/2369194064 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edaaaa3810 con 0x55edaaaee0c0 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.049+0000 7f6e4effd700 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55edaaaee0c0 0x55edaaaf04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.050+0000 7f6e2effd700 10 client.27086 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.050+0000 7f6e4effd700 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55edaaaee0c0 0x55edaaaf04e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e931874b40903e03 server_cookie=8257fbe0b35501a3 in_seq=0 out_seq=0 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.051+0000 7f6e2effd700 1 -- 172.21.15.12:0/2369194064 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55edaaaa3810 con 0x55edaaaee0c0 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.051+0000 7f6e2effd700 10 client.27086 handle_command_reply: tid=0 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.051+0000 7f6e61b61b80 1 client.27086 shutdown 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 1 -- 172.21.15.12:0/2369194064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 msgr2=0x55edaaade100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaade100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e440098e0 tx=0x7f6e440052e0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 1 -- 172.21.15.12:0/2369194064 shutdown_connections 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaade640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaaddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55edaaaee0c0 0x55edaaaf04e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/2369194064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaade100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e2effd700 0 client.27086 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 1 -- 172.21.15.12:0/2369194064 >> 172.21.15.12:0/2369194064 conn(0x55edaaa32210 msgr2=0x55edaaace5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 1 -- 172.21.15.12:0/2369194064 shutdown_connections 2024-05-08T23:53:05.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 1 -- 172.21.15.12:0/2369194064 wait complete. 2024-05-08T23:53:05.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.052+0000 7f6e61b61b80 20 client.27086 trim_cache size 0 max 16384 2024-05-08T23:53:05.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.066+0000 7f6e61b61b80 1 Processor -- start 2024-05-08T23:53:05.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.066+0000 7f6e61b61b80 1 -- start start 2024-05-08T23:53:05.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaad4890 0x55edaaaea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaaced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaada720 0x55edaaade460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaacbdc0 con 0x55edaaada720 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaaf56f0 con 0x55edaaad9e00 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaad8630 con 0x55edaaad4890 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaad4890 0x55edaaaea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaada720 0x55edaaade460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaaced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaaced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39200/0 (socket says 172.21.15.12:39200) 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaad4890 0x55edaaaea2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41728/0 (socket says 172.21.15.12:41728) 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaada720 0x55edaaade460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56014/0 (socket says 172.21.15.12:56014) 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4e7fc700 1 -- 172.21.15.12:0/1058292106 learned_addr learned my addr 172.21.15.12:0/1058292106 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:05.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4f7fe700 1 -- 172.21.15.12:0/1058292106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaad4890 msgr2=0x55edaaaea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4f7fe700 1 --2- 172.21.15.12:0/1058292106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaad4890 0x55edaaaea2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4f7fe700 1 -- 172.21.15.12:0/1058292106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 msgr2=0x55edaaaced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4f7fe700 1 --2- 172.21.15.12:0/1058292106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaaced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4f7fe700 1 -- 172.21.15.12:0/1058292106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edaa9ea560 con 0x55edaaada720 2024-05-08T23:53:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4e7fc700 1 --2- 172.21.15.12:0/1058292106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaaced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e4f7fe700 1 --2- 172.21.15.12:0/1058292106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaada720 0x55edaaade460 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6e480c9f70 tx=0x7f6e48095690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65cc0772d5097e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.067+0000 7f6e367fc700 1 -- 172.21.15.12:0/1058292106 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4801b320 con 0x55edaaada720 2024-05-08T23:53:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e367fc700 1 -- 172.21.15.12:0/1058292106 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4809a0c0 con 0x55edaaada720 2024-05-08T23:53:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e367fc700 1 -- 172.21.15.12:0/1058292106 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48099e80 con 0x55edaaada720 2024-05-08T23:53:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e61b61b80 1 -- 172.21.15.12:0/1058292106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaada720 msgr2=0x55edaaade460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/1058292106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaada720 0x55edaaade460 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6e480c9f70 tx=0x7f6e48095690 comp rx=0 tx=0).stop 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e61b61b80 1 -- 172.21.15.12:0/1058292106 shutdown_connections 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/1058292106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaada720 0x55edaaade460 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/1058292106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaad4890 0x55edaaaea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/1058292106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaaced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e61b61b80 1 -- 172.21.15.12:0/1058292106 >> 172.21.15.12:0/1058292106 conn(0x55edaaa32210 msgr2=0x55edaaad0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e61b61b80 1 -- 172.21.15.12:0/1058292106 shutdown_connections 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.068+0000 7f6e61b61b80 1 -- 172.21.15.12:0/1058292106 wait complete. 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.069+0000 7f6e61b61b80 1 -- start start 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.069+0000 7f6e61b61b80 1 Processor -- start 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaae4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaae4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e61b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaae4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaab435c0 con 0x55edaaad4890 2024-05-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaab43700 con 0x55edaaad9e00 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e61b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaab43840 con 0x55edaaada720 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e61b61b80 10 client.0 _ll_register_callbacks cb 0x55edaa9c9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaae4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaae4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaae4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39214/0 (socket says 172.21.15.12:39214) 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaae4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56022/0 (socket says 172.21.15.12:56022) 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4e7fc700 1 -- 172.21.15.12:0/3095282014 learned_addr learned my addr 172.21.15.12:0/3095282014 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4f7fe700 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaae4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4effd700 1 -- 172.21.15.12:0/3095282014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 msgr2=0x55edaaae4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4effd700 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaae4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4effd700 1 -- 172.21.15.12:0/3095282014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 msgr2=0x55edaaae4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4effd700 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaae4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4effd700 1 -- 172.21.15.12:0/3095282014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edaa851b60 con 0x55edaaad4890 2024-05-08T23:53:05.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4e7fc700 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaae4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e4effd700 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaae4320 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6e40026760 tx=0x7f6e40011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa0029a53979630 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.070+0000 7f6e2effd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e2effd700 1 -- 172.21.15.12:0/3095282014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40037070 con 0x55edaaad4890 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e2effd700 1 -- 172.21.15.12:0/3095282014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6e400264e0 con 0x55edaaad4890 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e61b61b80 1 -- 172.21.15.12:0/3095282014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e4000ee40 con 0x55edaaad4890 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e2effd700 1 -- 172.21.15.12:0/3095282014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e400054d0 con 0x55edaaad4890 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e61b61b80 10 client.17304 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e61b61b80 1 -- 172.21.15.12:0/3095282014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f6e501a6430 con 0x55edaaad4890 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e2effd700 1 -- 172.21.15.12:0/3095282014 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4000ee40 con 0x55edaaad4890 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e2effd700 1 -- 172.21.15.12:0/3095282014 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6e4000e6d0 con 0x55edaaad4890 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e61b61b80 10 client.17304 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e61b61b80 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e61b61b80 20 client.17304 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55edaab44fe0 0x55edaab47400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e61b61b80 4 client.17304 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:53:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.071+0000 7f6e61b61b80 1 -- 172.21.15.12:0/3095282014 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55edaaa1cec0 con 0x55edaab44fe0 2024-05-08T23:53:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.072+0000 7f6e4e7fc700 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55edaab44fe0 0x55edaab47400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.072+0000 7f6e2effd700 10 client.17304 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.072+0000 7f6e4e7fc700 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55edaab44fe0 0x55edaab47400 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c3b0cd440bb1dc server_cookie=c4abafccb4990bf in_seq=0 out_seq=0 2024-05-08T23:53:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.073+0000 7f6e2effd700 1 -- 172.21.15.12:0/3095282014 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55edaaa1cec0 con 0x55edaab44fe0 2024-05-08T23:53:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.073+0000 7f6e2effd700 10 client.17304 handle_command_reply: tid=0 2024-05-08T23:53:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.073+0000 7f6e61b61b80 1 client.17304 shutdown 2024-05-08T23:53:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 1 -- 172.21.15.12:0/3095282014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 msgr2=0x55edaaae4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaae4320 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6e40026760 tx=0x7f6e40011d50 comp rx=0 tx=0).stop 2024-05-08T23:53:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 1 -- 172.21.15.12:0/3095282014 shutdown_connections 2024-05-08T23:53:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55edaaada720 0x55edaaae4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55edaaad4890 0x55edaaae4320 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55edaab44fe0 0x55edaab47400 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 1 --2- 172.21.15.12:0/3095282014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55edaaad9e00 0x55edaaae4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e2effd700 0 client.17304 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 1 -- 172.21.15.12:0/3095282014 >> 172.21.15.12:0/3095282014 conn(0x55edaaa32210 msgr2=0x55edaaad8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 1 -- 172.21.15.12:0/3095282014 shutdown_connections 2024-05-08T23:53:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 1 -- 172.21.15.12:0/3095282014 wait complete. 2024-05-08T23:53:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e61b61b80 20 client.17304 trim_cache size 0 max 16384 2024-05-08T23:53:05.073 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:53:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e559a7700 1 -- 172.21.15.12:0/2552650339 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e380a1290 msgr2=0x7f6e380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e559a7700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e380a1290 0x7f6e380a3750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009fd0 tx=0x7f6e40009260 comp rx=0 tx=0).stop 2024-05-08T23:53:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e559a7700 1 -- 172.21.15.12:0/2552650339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50100890 msgr2=0x7f6e501a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.074+0000 7f6e559a7700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50100890 0x7f6e501a5560 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800ef30 tx=0x7f6e4800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.075+0000 7f6e559a7700 1 -- 172.21.15.12:0/2552650339 shutdown_connections 2024-05-08T23:53:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.075+0000 7f6e559a7700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e380a1290 0x7f6e380a3750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.075+0000 7f6e559a7700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50100890 0x7f6e501a5560 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.075+0000 7f6e559a7700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50069760 0x7f6e500ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.075+0000 7f6e559a7700 1 --2- 172.21.15.12:0/2552650339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e501062f0 0x7f6e50100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:05.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.075+0000 7f6e559a7700 1 -- 172.21.15.12:0/2552650339 >> 172.21.15.12:0/2552650339 conn(0x7f6e50076fa0 msgr2=0x7f6e500fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:05.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.075+0000 7f6e559a7700 1 -- 172.21.15.12:0/2552650339 shutdown_connections 2024-05-08T23:53:05.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:05.075+0000 7f6e559a7700 1 -- 172.21.15.12:0/2552650339 wait complete. 2024-05-08T23:53:05.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 425063 9.79 MB/sec execute 181 sec latency 16.609 ms 2024-05-08T23:53:06.085 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:53:06.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:53:06.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.249+0000 7f0ba3757700 1 -- 172.21.15.12:0/99408116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b9c101160 msgr2=0x7f0b9c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.249+0000 7f0ba3757700 1 --2- 172.21.15.12:0/99408116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b9c101160 0x7f0b9c101540 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b980098f0 tx=0x7f0b98009c00 comp rx=0 tx=0).stop 2024-05-08T23:53:06.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.249+0000 7f0ba3757700 1 -- 172.21.15.12:0/99408116 shutdown_connections 2024-05-08T23:53:06.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.249+0000 7f0ba3757700 1 --2- 172.21.15.12:0/99408116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b9c106280 0x7f0b9c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.249+0000 7f0ba3757700 1 --2- 172.21.15.12:0/99408116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b9c101160 0x7f0b9c101540 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.249+0000 7f0ba3757700 1 --2- 172.21.15.12:0/99408116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c101b10 0x7f0b9c105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.249+0000 7f0ba3757700 1 -- 172.21.15.12:0/99408116 >> 172.21.15.12:0/99408116 conn(0x7f0b9c0fc9e0 msgr2=0x7f0b9c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.249+0000 7f0ba3757700 1 -- 172.21.15.12:0/99408116 shutdown_connections 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.249+0000 7f0ba3757700 1 -- 172.21.15.12:0/99408116 wait complete. 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba3757700 1 Processor -- start 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba3757700 1 -- start start 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba3757700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c101160 0x7f0b9c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba3757700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b9c106280 0x7f0b9c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba3757700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b9c073d20 0x7f0b9c1a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba3757700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1a5bd0 con 0x7f0b9c101160 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba3757700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1a5d10 con 0x7f0b9c073d20 2024-05-08T23:53:06.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba3757700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1a5ed0 con 0x7f0b9c106280 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba14f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c101160 0x7f0b9c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba0cf2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b9c106280 0x7f0b9c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba1cf4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b9c073d20 0x7f0b9c1a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba14f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c101160 0x7f0b9c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56044/0 (socket says 172.21.15.12:56044) 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba14f3700 1 -- 172.21.15.12:0/2116021532 learned_addr learned my addr 172.21.15.12:0/2116021532 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba14f3700 1 -- 172.21.15.12:0/2116021532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b9c106280 msgr2=0x7f0b9c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b9c106280 0x7f0b9c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba14f3700 1 -- 172.21.15.12:0/2116021532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b9c073d20 msgr2=0x7f0b9c1a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b9c073d20 0x7f0b9c1a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba14f3700 1 -- 172.21.15.12:0/2116021532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b98009520 con 0x7f0b9c101160 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba1cf4700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b9c073d20 0x7f0b9c1a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.250+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c101160 0x7f0b9c0732a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0b98005950 tx=0x7f0b9800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.251+0000 7f0b927fc700 1 -- 172.21.15.12:0/2116021532 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9801c070 con 0x7f0b9c101160 2024-05-08T23:53:06.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.251+0000 7f0ba3757700 1 -- 172.21.15.12:0/2116021532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c1a6150 con 0x7f0b9c101160 2024-05-08T23:53:06.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.251+0000 7f0b927fc700 1 -- 172.21.15.12:0/2116021532 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9800bb20 con 0x7f0b9c101160 2024-05-08T23:53:06.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.251+0000 7f0b927fc700 1 -- 172.21.15.12:0/2116021532 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b980038e0 con 0x7f0b9c101160 2024-05-08T23:53:06.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.251+0000 7f0ba3757700 1 -- 172.21.15.12:0/2116021532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c1a6620 con 0x7f0b9c101160 2024-05-08T23:53:06.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.254+0000 7f0b927fc700 1 -- 172.21.15.12:0/2116021532 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0b98020030 con 0x7f0b9c101160 2024-05-08T23:53:06.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.254+0000 7f0b927fc700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0b880a1290 0x7f0b880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.255+0000 7f0b927fc700 1 -- 172.21.15.12:0/2116021532 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0b980cdaa0 con 0x7f0b9c101160 2024-05-08T23:53:06.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.255+0000 7f0ba0cf2700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0b880a1290 0x7f0b880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.255+0000 7f0ba0cf2700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0b880a1290 0x7f0b880a3750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0fce00 tx=0x7f0b8c009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.255+0000 7f0baf911b80 1 Processor -- start 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.255+0000 7f0baf911b80 1 -- start start 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c87260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c877a0 0x55f7b7c813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c81d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0baf911b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7bd81d0 con 0x55f7b7c86e80 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0baf911b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7bd5b90 con 0x55f7b7c81910 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0baf911b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7bd1480 con 0x55f7b7c877a0 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba14f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c87260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba0cf2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c877a0 0x55f7b7c813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba14f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c87260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56054/0 (socket says 172.21.15.12:56054) 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba1cf4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c81d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba14f3700 1 -- 172.21.15.12:0/1650828346 learned_addr learned my addr 172.21.15.12:0/1650828346 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba14f3700 1 -- 172.21.15.12:0/1650828346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c877a0 msgr2=0x55f7b7c813d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/1650828346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c877a0 0x55f7b7c813d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba14f3700 1 -- 172.21.15.12:0/1650828346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 msgr2=0x55f7b7c81d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/1650828346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c81d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba14f3700 1 -- 172.21.15.12:0/1650828346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7b7a52430 con 0x55f7b7c86e80 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba1cf4700 1 --2- 172.21.15.12:0/1650828346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c81d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/1650828346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c87260 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0b98097130 tx=0x7f0b980054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4362b7eb7ea7f84b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.256+0000 7f0b877fe700 1 -- 172.21.15.12:0/1650828346 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9809b330 con 0x55f7b7c86e80 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.257+0000 7f0b877fe700 1 -- 172.21.15.12:0/1650828346 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9809ac30 con 0x55f7b7c86e80 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.257+0000 7f0b877fe700 1 -- 172.21.15.12:0/1650828346 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98005140 con 0x55f7b7c86e80 2024-05-08T23:53:06.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.257+0000 7f0baf911b80 1 -- 172.21.15.12:0/1650828346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 msgr2=0x55f7b7c87260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.257+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1650828346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c87260 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0b98097130 tx=0x7f0b980054e0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.257+0000 7f0baf911b80 1 -- 172.21.15.12:0/1650828346 shutdown_connections 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.257+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1650828346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c81d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.257+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1650828346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c87260 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.257+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1650828346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c877a0 0x55f7b7c813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.257+0000 7f0baf911b80 1 -- 172.21.15.12:0/1650828346 >> 172.21.15.12:0/1650828346 conn(0x55f7b7bdf290 msgr2=0x55f7b7c7be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.257+0000 7f0baf911b80 1 -- 172.21.15.12:0/1650828346 shutdown_connections 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.258+0000 7f0baf911b80 1 -- 172.21.15.12:0/1650828346 wait complete. 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.258+0000 7f0baf911b80 1 -- start start 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.258+0000 7f0baf911b80 1 Processor -- start 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.258+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 0x55f7b7c8aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.258+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c8afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.258+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.258+0000 7f0baf911b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7b6d5e0 con 0x55f7b7c86e80 2024-05-08T23:53:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.258+0000 7f0baf911b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7b6d280 con 0x55f7b7c877a0 2024-05-08T23:53:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.258+0000 7f0baf911b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7a54720 con 0x55f7b7c81910 2024-05-08T23:53:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.258+0000 7f0baf911b80 10 client.0 _ll_register_callbacks cb 0x55f7b7b76bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba0cf2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c8afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba14f3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 0x55f7b7c8aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba0cf2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c8afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56066/0 (socket says 172.21.15.12:56066) 2024-05-08T23:53:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba1cf4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba0cf2700 1 -- 172.21.15.12:0/1480795811 learned_addr learned my addr 172.21.15.12:0/1480795811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba0cf2700 1 -- 172.21.15.12:0/1480795811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 msgr2=0x55f7b7c8aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba0cf2700 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 0x55f7b7c8aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba0cf2700 1 -- 172.21.15.12:0/1480795811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 msgr2=0x55f7b7c8b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba0cf2700 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba0cf2700 1 -- 172.21.15.12:0/1480795811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b94003a60 con 0x55f7b7c86e80 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 0x55f7b7c8aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba1cf4700 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0ba0cf2700 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c8afb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009fd0 tx=0x7f0b8c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85259b29bb476a9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0b84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1480795811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c004a30 con 0x55f7b7c86e80 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1480795811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0b8c004b90 con 0x55f7b7c86e80 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.259+0000 7f0baf911b80 1 -- 172.21.15.12:0/1480795811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7b7b2a2f0 con 0x55f7b7c86e80 2024-05-08T23:53:06.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1480795811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b8c004d50 con 0x55f7b7c86e80 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0baf911b80 10 client.17316 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0baf911b80 1 -- 172.21.15.12:0/1480795811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f0b9c1a6620 con 0x55f7b7c86e80 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1480795811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7b7b2a2f0 con 0x55f7b7c86e80 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1480795811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0b8c01e900 con 0x55f7b7c86e80 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0baf911b80 10 client.17316 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0baf911b80 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0baf911b80 20 client.17316 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f7b7c9af70 0x55f7b7c9d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0baf911b80 4 client.17316 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0baf911b80 1 -- 172.21.15.12:0/1480795811 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7b7c50890 con 0x55f7b7c9af70 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.260+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f7b7c9af70 0x55f7b7c9d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f7b7c9af70 0x55f7b7c9d390 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129c3e97d53234ae server_cookie=b51f2c1dce98e1a6 in_seq=0 out_seq=0 2024-05-08T23:53:06.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0b84ff9700 10 client.17316 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1480795811 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f7b7c50890 con 0x55f7b7c9af70 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0b84ff9700 10 client.17316 handle_command_reply: tid=0 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0baf911b80 1 client.17316 shutdown 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0baf911b80 1 -- 172.21.15.12:0/1480795811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 msgr2=0x55f7b7c8afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c8afb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009fd0 tx=0x7f0b8c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0baf911b80 1 -- 172.21.15.12:0/1480795811 shutdown_connections 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f7b7c9af70 0x55f7b7c9d390 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 0x55f7b7c8aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0b84ff9700 0 client.17316 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1480795811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c8afb0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.261+0000 7f0baf911b80 1 -- 172.21.15.12:0/1480795811 >> 172.21.15.12:0/1480795811 conn(0x55f7b7bdf290 msgr2=0x55f7b7c7b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.262+0000 7f0baf911b80 1 -- 172.21.15.12:0/1480795811 shutdown_connections 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.262+0000 7f0baf911b80 1 -- 172.21.15.12:0/1480795811 wait complete. 2024-05-08T23:53:06.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.262+0000 7f0baf911b80 20 client.17316 trim_cache size 0 max 16384 2024-05-08T23:53:06.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.282+0000 7f0baf911b80 1 Processor -- start 2024-05-08T23:53:06.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.282+0000 7f0baf911b80 1 -- start start 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.282+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 0x55f7b7c89bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.282+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c7bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.282+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.282+0000 7f0baf911b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7c8a110 con 0x55f7b7c86e80 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.282+0000 7f0baf911b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7c76880 con 0x55f7b7c877a0 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.282+0000 7f0baf911b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7c813d0 con 0x55f7b7c81910 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba0cf2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c7bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba1cf4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba1cf4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39252/0 (socket says 172.21.15.12:39252) 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba0cf2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c7bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56078/0 (socket says 172.21.15.12:56078) 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba1cf4700 1 -- 172.21.15.12:0/734608198 learned_addr learned my addr 172.21.15.12:0/734608198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/734608198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 0x55f7b7c89bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba1cf4700 1 -- 172.21.15.12:0/734608198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 msgr2=0x55f7b7c89bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba1cf4700 1 --2- 172.21.15.12:0/734608198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 0x55f7b7c89bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba1cf4700 1 -- 172.21.15.12:0/734608198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 msgr2=0x55f7b7c7bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba1cf4700 1 --2- 172.21.15.12:0/734608198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c7bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba1cf4700 1 -- 172.21.15.12:0/734608198 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7b7c81fc0 con 0x55f7b7c877a0 2024-05-08T23:53:06.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0ba1cf4700 1 --2- 172.21.15.12:0/734608198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b940045d0 tx=0x7f0b94009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3edec2a02da0cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0b87fff700 1 -- 172.21.15.12:0/734608198 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400e960 con 0x55f7b7c877a0 2024-05-08T23:53:06.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0b87fff700 1 -- 172.21.15.12:0/734608198 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9400f040 con 0x55f7b7c877a0 2024-05-08T23:53:06.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.283+0000 7f0b87fff700 1 -- 172.21.15.12:0/734608198 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94020440 con 0x55f7b7c877a0 2024-05-08T23:53:06.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.284+0000 7f0baf911b80 1 -- 172.21.15.12:0/734608198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 msgr2=0x55f7b7c8b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.284+0000 7f0baf911b80 1 --2- 172.21.15.12:0/734608198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b940045d0 tx=0x7f0b94009920 comp rx=0 tx=0).stop 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.284+0000 7f0baf911b80 1 -- 172.21.15.12:0/734608198 shutdown_connections 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.284+0000 7f0baf911b80 1 --2- 172.21.15.12:0/734608198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8b310 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.284+0000 7f0baf911b80 1 --2- 172.21.15.12:0/734608198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c81910 0x55f7b7c89bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.284+0000 7f0baf911b80 1 --2- 172.21.15.12:0/734608198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c86e80 0x55f7b7c7bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.284+0000 7f0baf911b80 1 -- 172.21.15.12:0/734608198 >> 172.21.15.12:0/734608198 conn(0x55f7b7bdf290 msgr2=0x55f7b7c861e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.284+0000 7f0baf911b80 1 -- 172.21.15.12:0/734608198 shutdown_connections 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.284+0000 7f0baf911b80 1 -- 172.21.15.12:0/734608198 wait complete. 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.285+0000 7f0baf911b80 1 -- start start 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.285+0000 7f0baf911b80 1 Processor -- start 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c91f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c86e80 0x55f7b7c92440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0baf911b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0baf911b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7c905f0 con 0x55f7b7c877a0 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0baf911b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7c90760 con 0x55f7b7c81910 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0baf911b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b7c90920 con 0x55f7b7c86e80 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0baf911b80 10 client.0 _ll_register_callbacks cb 0x55f7b7b76bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba14f3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c91f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba0cf2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c86e80 0x55f7b7c92440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba0cf2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c86e80 0x55f7b7c92440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41786/0 (socket says 172.21.15.12:41786) 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba0cf2700 1 -- 172.21.15.12:0/1247364907 learned_addr learned my addr 172.21.15.12:0/1247364907 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba14f3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c91f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39254/0 (socket says 172.21.15.12:39254) 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba1cf4700 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba0cf2700 1 -- 172.21.15.12:0/1247364907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 msgr2=0x55f7b7c91f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba0cf2700 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c91f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba0cf2700 1 -- 172.21.15.12:0/1247364907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c877a0 msgr2=0x55f7b7c8fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba0cf2700 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba0cf2700 1 -- 172.21.15.12:0/1247364907 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b94004280 con 0x55f7b7c86e80 2024-05-08T23:53:06.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c91f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:06.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba1cf4700 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:06.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0ba0cf2700 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c86e80 0x55f7b7c92440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c016040 tx=0x7f0b8c004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c864c14a7c0c5d14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0b84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:53:06.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.286+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1247364907 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c005240 con 0x55f7b7c86e80 2024-05-08T23:53:06.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1247364907 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0b8c035070 con 0x55f7b7c86e80 2024-05-08T23:53:06.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0baf911b80 1 -- 172.21.15.12:0/1247364907 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b9809ac30 con 0x55f7b7c86e80 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1247364907 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b8c003ec0 con 0x55f7b7c86e80 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0baf911b80 10 client.27109 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0baf911b80 1 -- 172.21.15.12:0/1247364907 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f7b79febe0 con 0x55f7b7c86e80 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1247364907 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9809ac30 con 0x55f7b7c86e80 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1247364907 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0b8c0307c0 con 0x55f7b7c86e80 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0baf911b80 10 client.27109 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0baf911b80 10 client.27109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0baf911b80 20 client.27109 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f7b7cf35d0 0x55f7b7cf59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0baf911b80 4 client.27109 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0baf911b80 1 -- 172.21.15.12:0/1247364907 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f7b7bc9f40 con 0x55f7b7cf35d0 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.287+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f7b7cf35d0 0x55f7b7cf59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.288+0000 7f0b84ff9700 10 client.27109 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:06.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.288+0000 7f0ba14f3700 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f7b7cf35d0 0x55f7b7cf59f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aadcf89c7730fea server_cookie=ce3c890c17a42581 in_seq=0 out_seq=0 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0b84ff9700 1 -- 172.21.15.12:0/1247364907 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f7b7bc9f40 con 0x55f7b7cf35d0 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0b84ff9700 10 client.27109 handle_command_reply: tid=0 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 client.27109 shutdown 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 -- 172.21.15.12:0/1247364907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c86e80 msgr2=0x55f7b7c92440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c86e80 0x55f7b7c92440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c016040 tx=0x7f0b8c004a60 comp rx=0 tx=0).stop 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 -- 172.21.15.12:0/1247364907 shutdown_connections 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7b7c877a0 0x55f7b7c8fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f7b7cf35d0 0x55f7b7cf59f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f7b7c81910 0x55f7b7c91f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 --2- 172.21.15.12:0/1247364907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7b7c86e80 0x55f7b7c92440 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0b84ff9700 0 client.27109 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 -- 172.21.15.12:0/1247364907 >> 172.21.15.12:0/1247364907 conn(0x55f7b7bdf290 msgr2=0x55f7b7c803c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 -- 172.21.15.12:0/1247364907 shutdown_connections 2024-05-08T23:53:06.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.289+0000 7f0baf911b80 1 -- 172.21.15.12:0/1247364907 wait complete. 2024-05-08T23:53:06.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0baf911b80 20 client.27109 trim_cache size 0 max 16384 2024-05-08T23:53:06.290 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:53:06.290 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:53:06.290 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "33412203-63ab-4b3f-aea0-1c528bc40ed9", 2024-05-08T23:53:06.290 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:53:06.290 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:53:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 -- 172.21.15.12:0/2116021532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0b880a1290 msgr2=0x7f0b880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0b880a1290 0x7f0b880a3750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0fce00 tx=0x7f0b8c009310 comp rx=0 tx=0).stop 2024-05-08T23:53:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 -- 172.21.15.12:0/2116021532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c101160 msgr2=0x7f0b9c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c101160 0x7f0b9c0732a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0b98005950 tx=0x7f0b9800fa00 comp rx=0 tx=0).stop 2024-05-08T23:53:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 -- 172.21.15.12:0/2116021532 shutdown_connections 2024-05-08T23:53:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0b880a1290 0x7f0b880a3750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b9c073d20 0x7f0b9c1a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c101160 0x7f0b9c0732a0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 --2- 172.21.15.12:0/2116021532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b9c106280 0x7f0b9c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 -- 172.21.15.12:0/2116021532 >> 172.21.15.12:0/2116021532 conn(0x7f0b9c0fc9e0 msgr2=0x7f0b9c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 -- 172.21.15.12:0/2116021532 shutdown_connections 2024-05-08T23:53:06.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.290+0000 7f0ba3757700 1 -- 172.21.15.12:0/2116021532 wait complete. 2024-05-08T23:53:06.302 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.458+0000 7f9a02350700 1 -- 172.21.15.12:0/947804670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc105170 msgr2=0x7f99fc10ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.458+0000 7f9a02350700 1 --2- 172.21.15.12:0/947804670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc105170 0x7f99fc10ccf0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f99f400b0a0 tx=0x7f99f400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.458+0000 7f9a02350700 1 -- 172.21.15.12:0/947804670 shutdown_connections 2024-05-08T23:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.458+0000 7f9a02350700 1 --2- 172.21.15.12:0/947804670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc105170 0x7f99fc10ccf0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.458+0000 7f9a02350700 1 --2- 172.21.15.12:0/947804670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99fc100050 0x7f99fc100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.458+0000 7f9a02350700 1 --2- 172.21.15.12:0/947804670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99fc100a00 0x7f99fc104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.458+0000 7f9a02350700 1 -- 172.21.15.12:0/947804670 >> 172.21.15.12:0/947804670 conn(0x7f99fc076fa0 msgr2=0x7f99fc0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.458+0000 7f9a02350700 1 -- 172.21.15.12:0/947804670 shutdown_connections 2024-05-08T23:53:06.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.458+0000 7f9a02350700 1 -- 172.21.15.12:0/947804670 wait complete. 2024-05-08T23:53:06.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.459+0000 7f9a02350700 1 Processor -- start 2024-05-08T23:53:06.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.459+0000 7f9a02350700 1 -- start start 2024-05-08T23:53:06.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.459+0000 7f9a02350700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99fc100050 0x7f99fc1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f9a02350700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc105170 0x7f99fc197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f9a02350700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99fc19bdd0 0x7f99fc19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f9a02350700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc19c850 con 0x7f99fc105170 2024-05-08T23:53:06.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f9a02350700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc19c9c0 con 0x7f99fc100050 2024-05-08T23:53:06.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f9a02350700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc19cb80 con 0x7f99fc19bdd0 2024-05-08T23:53:06.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99fc100050 0x7f99fc1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc105170 0x7f99fc197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99fc100050 0x7f99fc1974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39270/0 (socket says 172.21.15.12:39270) 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fbfff700 1 -- 172.21.15.12:0/1115994023 learned_addr learned my addr 172.21.15.12:0/1115994023 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f9a008ed700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99fc19bdd0 0x7f99fc19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fbfff700 1 -- 172.21.15.12:0/1115994023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99fc19bdd0 msgr2=0x7f99fc19c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fbfff700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99fc19bdd0 0x7f99fc19c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fbfff700 1 -- 172.21.15.12:0/1115994023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc105170 msgr2=0x7f99fc197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fbfff700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc105170 0x7f99fc197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fbfff700 1 -- 172.21.15.12:0/1115994023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f4009d00 con 0x7f99fc100050 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f9a008ed700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99fc19bdd0 0x7f99fc19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc105170 0x7f99fc197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.460+0000 7f99fbfff700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99fc100050 0x7f99fc1974f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00e9a0 tx=0x7f99ec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.461+0000 7f99f97fa700 1 -- 172.21.15.12:0/1115994023 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec009790 con 0x7f99fc100050 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.461+0000 7f99f97fa700 1 -- 172.21.15.12:0/1115994023 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ec010430 con 0x7f99fc100050 2024-05-08T23:53:06.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.461+0000 7f9a02350700 1 -- 172.21.15.12:0/1115994023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc1a40e0 con 0x7f99fc100050 2024-05-08T23:53:06.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.461+0000 7f99f97fa700 1 -- 172.21.15.12:0/1115994023 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec018680 con 0x7f99fc100050 2024-05-08T23:53:06.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.461+0000 7f9a02350700 1 -- 172.21.15.12:0/1115994023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc1a45c0 con 0x7f99fc100050 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.463+0000 7f99f97fa700 1 -- 172.21.15.12:0/1115994023 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f99ec0105a0 con 0x7f99fc100050 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.464+0000 7f99f97fa700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99e40a1290 0x7f99e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.464+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99e40a1290 0x7f99e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.464+0000 7f99f97fa700 1 -- 172.21.15.12:0/1115994023 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f99ec014070 con 0x7f99fc100050 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.464+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99e40a1290 0x7f99e40a3750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99fc198a40 tx=0x7f99f0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f9a0e50ab80 1 Processor -- start 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f9a0e50ab80 1 -- start start 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52c720 0x560a0e526350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e526cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e47d150 con 0x560a0e526890 2024-05-08T23:53:06.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e47ab10 con 0x560a0e52c720 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e476400 con 0x560a0e52be00 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f9a008ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e526cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41822/0 (socket says 172.21.15.12:41822) 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f99fbfff700 1 -- 172.21.15.12:0/319320207 learned_addr learned my addr 172.21.15.12:0/319320207 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/319320207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52c720 0x560a0e526350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f99fbfff700 1 -- 172.21.15.12:0/319320207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52c720 msgr2=0x560a0e526350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f99fbfff700 1 --2- 172.21.15.12:0/319320207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52c720 0x560a0e526350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f99fbfff700 1 -- 172.21.15.12:0/319320207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 msgr2=0x560a0e526cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f99fbfff700 1 --2- 172.21.15.12:0/319320207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e526cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.465+0000 7f99fbfff700 1 -- 172.21.15.12:0/319320207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a0e2f73b0 con 0x560a0e52be00 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.466+0000 7f99fbfff700 1 --2- 172.21.15.12:0/319320207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52c1e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f99ec096a40 tx=0x7f99ec099c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67aa6b00ac05e239 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.466+0000 7f99e27fc700 1 -- 172.21.15.12:0/319320207 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0adea0 con 0x560a0e52be00 2024-05-08T23:53:06.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.466+0000 7f99e27fc700 1 -- 172.21.15.12:0/319320207 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ec09de00 con 0x560a0e52be00 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.466+0000 7f99e27fc700 1 -- 172.21.15.12:0/319320207 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec09a430 con 0x560a0e52be00 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/319320207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 msgr2=0x560a0e52c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/319320207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52c1e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f99ec096a40 tx=0x7f99ec099c40 comp rx=0 tx=0).stop 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/319320207 shutdown_connections 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/319320207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e526cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/319320207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52c1e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/319320207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52c720 0x560a0e526350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/319320207 >> 172.21.15.12:0/319320207 conn(0x560a0e484210 msgr2=0x560a0e520df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/319320207 shutdown_connections 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/319320207 wait complete. 2024-05-08T23:53:06.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 -- start start 2024-05-08T23:53:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.467+0000 7f9a0e50ab80 1 Processor -- start 2024-05-08T23:53:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e52fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52c720 0x560a0e530530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e412560 con 0x560a0e526890 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e412200 con 0x560a0e52c720 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e2f96a0 con 0x560a0e52be00 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f9a0e50ab80 10 client.0 _ll_register_callbacks cb 0x560a0e41bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e52fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fb7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e52fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56144/0 (socket says 172.21.15.12:56144) 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fbfff700 1 -- 172.21.15.12:0/200084929 learned_addr learned my addr 172.21.15.12:0/200084929 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f9a008ed700 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52c720 0x560a0e530530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fb7fe700 1 -- 172.21.15.12:0/200084929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52c720 msgr2=0x560a0e530530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52c720 0x560a0e530530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fb7fe700 1 -- 172.21.15.12:0/200084929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 msgr2=0x560a0e52fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e52fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fb7fe700 1 -- 172.21.15.12:0/200084929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc1a45c0 con 0x560a0e52be00 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fbfff700 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e52fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52fff0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x560a0e522bd0 tx=0x7f99f00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637b15f13dd058f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.468+0000 7f99db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f99db7fe700 1 -- 172.21.15.12:0/200084929 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0004460 con 0x560a0e52be00 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f99db7fe700 1 -- 172.21.15.12:0/200084929 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f99f001f860 con 0x560a0e52be00 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/200084929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a0e3cf270 con 0x560a0e52be00 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f99db7fe700 1 -- 172.21.15.12:0/200084929 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f001fa20 con 0x560a0e52be00 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f9a0e50ab80 10 client.27121 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/200084929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560a0e2a3b60 con 0x560a0e52be00 2024-05-08T23:53:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f99db7fe700 1 -- 172.21.15.12:0/200084929 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a0e3cf270 con 0x560a0e52be00 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f99db7fe700 1 -- 172.21.15.12:0/200084929 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f99f0003c90 con 0x560a0e52be00 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f9a0e50ab80 10 client.27121 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f9a0e50ab80 10 client.27121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f9a0e50ab80 20 client.27121 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a0e5401f0 0x560a0e542610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f9a0e50ab80 4 client.27121 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.469+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/200084929 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a0e4f5810 con 0x560a0e5401f0 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.470+0000 7f99fbfff700 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a0e5401f0 0x560a0e542610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.470+0000 7f99db7fe700 10 client.27121 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.470+0000 7f99fbfff700 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a0e5401f0 0x560a0e542610 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dae846d4a366e99b server_cookie=b463f33c4a5f0b37 in_seq=0 out_seq=0 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f99db7fe700 1 -- 172.21.15.12:0/200084929 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a0e4f5810 con 0x560a0e5401f0 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f99db7fe700 10 client.27121 handle_command_reply: tid=0 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 client.27121 shutdown 2024-05-08T23:53:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/200084929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 msgr2=0x560a0e52fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52fff0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x560a0e522bd0 tx=0x7f99f00047c0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/200084929 shutdown_connections 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a0e5401f0 0x560a0e542610 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52c720 0x560a0e530530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e52fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/200084929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52be00 0x560a0e52fff0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f99db7fe700 0 client.27121 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/200084929 >> 172.21.15.12:0/200084929 conn(0x560a0e484210 msgr2=0x560a0e5205a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/200084929 shutdown_connections 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/200084929 wait complete. 2024-05-08T23:53:06.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.471+0000 7f9a0e50ab80 20 client.27121 trim_cache size 0 max 16384 2024-05-08T23:53:06.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a0e50ab80 1 Processor -- start 2024-05-08T23:53:06.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a0e50ab80 1 -- start start 2024-05-08T23:53:06.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e526890 0x560a0e526350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 0x560a0e52b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e52c720 0x560a0e520d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e51ddc0 con 0x560a0e52c720 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e522c00 con 0x560a0e52be00 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e5190d0 con 0x560a0e526890 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a008ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e52c720 0x560a0e520d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e526890 0x560a0e526350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a008ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e52c720 0x560a0e520d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56150/0 (socket says 172.21.15.12:56150) 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f9a008ed700 1 -- 172.21.15.12:0/3719881275 learned_addr learned my addr 172.21.15.12:0/3719881275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.494+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/3719881275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 0x560a0e52b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.495+0000 7f9a008ed700 1 -- 172.21.15.12:0/3719881275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e526890 msgr2=0x560a0e526350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.495+0000 7f9a008ed700 1 --2- 172.21.15.12:0/3719881275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e526890 0x560a0e526350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.495+0000 7f9a008ed700 1 -- 172.21.15.12:0/3719881275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 msgr2=0x560a0e52b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.495+0000 7f9a008ed700 1 --2- 172.21.15.12:0/3719881275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 0x560a0e52b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.495+0000 7f9a008ed700 1 -- 172.21.15.12:0/3719881275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a0e526f40 con 0x560a0e52c720 2024-05-08T23:53:06.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.495+0000 7f99fbfff700 1 --2- 172.21.15.12:0/3719881275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e526890 0x560a0e526350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:06.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.495+0000 7f9a008ed700 1 --2- 172.21.15.12:0/3719881275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e52c720 0x560a0e520d20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f99f4015000 tx=0x7f99f4009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14258ef68b03cd12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.495+0000 7f99e2ffd700 1 -- 172.21.15.12:0/3719881275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f400e040 con 0x560a0e52c720 2024-05-08T23:53:06.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.495+0000 7f99e2ffd700 1 -- 172.21.15.12:0/3719881275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99f4007cc0 con 0x560a0e52c720 2024-05-08T23:53:06.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.495+0000 7f99e2ffd700 1 -- 172.21.15.12:0/3719881275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f40052a0 con 0x560a0e52c720 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.496+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/3719881275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e52c720 msgr2=0x560a0e520d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.496+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/3719881275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e52c720 0x560a0e520d20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f99f4015000 tx=0x7f99f4009390 comp rx=0 tx=0).stop 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.496+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/3719881275 shutdown_connections 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.496+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/3719881275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e52c720 0x560a0e520d20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.496+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/3719881275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e526890 0x560a0e526350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.496+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/3719881275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 0x560a0e52b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.496+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/3719881275 >> 172.21.15.12:0/3719881275 conn(0x560a0e484210 msgr2=0x560a0e52e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.496+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/3719881275 shutdown_connections 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.496+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/3719881275 wait complete. 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.496+0000 7f9a0e50ab80 1 -- start start 2024-05-08T23:53:06.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f9a0e50ab80 1 Processor -- start 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e537540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 0x560a0e528060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f9a0e50ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52c720 0x560a0e5285a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e528cb0 con 0x560a0e526890 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e528e20 con 0x560a0e52be00 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f9a0e50ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0e59ab00 con 0x560a0e52c720 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f9a0e50ab80 10 client.0 _ll_register_callbacks cb 0x560a0e41bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e537540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e537540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56160/0 (socket says 172.21.15.12:56160) 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f99fbfff700 1 -- 172.21.15.12:0/1744691653 learned_addr learned my addr 172.21.15.12:0/1744691653 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 0x560a0e528060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f9a008ed700 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52c720 0x560a0e5285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f99fbfff700 1 -- 172.21.15.12:0/1744691653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52c720 msgr2=0x560a0e5285a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f99fbfff700 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52c720 0x560a0e5285a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f99fbfff700 1 -- 172.21.15.12:0/1744691653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 msgr2=0x560a0e528060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f99fbfff700 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 0x560a0e528060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f99fbfff700 1 -- 172.21.15.12:0/1744691653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a0e2f73b0 con 0x560a0e526890 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.497+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 0x560a0e528060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f99fbfff700 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e537540 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f99ec099f60 tx=0x7f99ec09a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416e8671f8c03b05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f99db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f99db7fe700 1 -- 172.21.15.12:0/1744691653 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0adea0 con 0x560a0e526890 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f99db7fe700 1 -- 172.21.15.12:0/1744691653 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f99ec094de0 con 0x560a0e526890 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/1744691653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99ec09de00 con 0x560a0e526890 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f99db7fe700 1 -- 172.21.15.12:0/1744691653 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99ec0b8320 con 0x560a0e526890 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f9a0e50ab80 10 client.17340 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/1744691653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f99fc10d560 con 0x560a0e526890 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f99db7fe700 1 -- 172.21.15.12:0/1744691653 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ec09de00 con 0x560a0e526890 2024-05-08T23:53:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f99db7fe700 1 -- 172.21.15.12:0/1744691653 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f99ec0b2b80 con 0x560a0e526890 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f9a0e50ab80 10 client.17340 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f9a0e50ab80 10 client.17340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f9a0e50ab80 20 client.17340 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a0e59be10 0x560a0e59e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f9a0e50ab80 4 client.17340 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.498+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/1744691653 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560a0e46eec0 con 0x560a0e59be10 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.499+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a0e59be10 0x560a0e59e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.499+0000 7f99fb7fe700 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a0e59be10 0x560a0e59e200 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e71002507bd39beb server_cookie=eb91d894ba005c32 in_seq=0 out_seq=0 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.499+0000 7f99db7fe700 10 client.17340 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.499+0000 7f99db7fe700 1 -- 172.21.15.12:0/1744691653 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560a0e46eec0 con 0x560a0e59be10 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.499+0000 7f99db7fe700 10 client.17340 handle_command_reply: tid=0 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.499+0000 7f9a0e50ab80 1 client.17340 shutdown 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.499+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/1744691653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 msgr2=0x560a0e537540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e537540 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f99ec099f60 tx=0x7f99ec09a7d0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/1744691653 shutdown_connections 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a0e52c720 0x560a0e5285a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a0e526890 0x560a0e537540 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a0e59be10 0x560a0e59e200 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stdout: "33412203-63ab-4b3f-aea0-1c528bc40ed9": { 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "33412203-63ab-4b3f-aea0-1c528bc40ed9", 2024-05-08T23:53:06.500 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:53:06.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:53:06.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:53:06.501 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:53:06.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a0e50ab80 1 --2- 172.21.15.12:0/1744691653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a0e52be00 0x560a0e528060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/1744691653 >> 172.21.15.12:0/1744691653 conn(0x560a0e484210 msgr2=0x560a0e52b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f99db7fe700 0 client.17340 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/1744691653 shutdown_connections 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a0e50ab80 1 -- 172.21.15.12:0/1744691653 wait complete. 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a0e50ab80 20 client.17340 trim_cache size 0 max 16384 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a02350700 1 -- 172.21.15.12:0/1115994023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99e40a1290 msgr2=0x7f99e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a02350700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99e40a1290 0x7f99e40a3750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99fc198a40 tx=0x7f99f0009260 comp rx=0 tx=0).stop 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a02350700 1 -- 172.21.15.12:0/1115994023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99fc100050 msgr2=0x7f99fc1974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.500+0000 7f9a02350700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99fc100050 0x7f99fc1974f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00e9a0 tx=0x7f99ec00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.501+0000 7f9a02350700 1 -- 172.21.15.12:0/1115994023 shutdown_connections 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.501+0000 7f9a02350700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99e40a1290 0x7f99e40a3750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.501+0000 7f9a02350700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99fc19bdd0 0x7f99fc19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.501+0000 7f9a02350700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99fc100050 0x7f99fc1974f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.501+0000 7f9a02350700 1 --2- 172.21.15.12:0/1115994023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc105170 0x7f99fc197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:06.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.501+0000 7f9a02350700 1 -- 172.21.15.12:0/1115994023 >> 172.21.15.12:0/1115994023 conn(0x7f99fc076fa0 msgr2=0x7f99fc069800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:06.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.501+0000 7f9a02350700 1 -- 172.21.15.12:0/1115994023 shutdown_connections 2024-05-08T23:53:06.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:06.501+0000 7f9a02350700 1 -- 172.21.15.12:0/1115994023 wait complete. 2024-05-08T23:53:06.510 INFO:tasks.cephfs.filesystem:scrub status for tag:33412203-63ab-4b3f-aea0-1c528bc40ed9 - {'path': '/', 'tag': '33412203-63ab-4b3f-aea0-1c528bc40ed9', 'options': 'recursive,force'} 2024-05-08T23:53:06.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:06 smithi204 ceph-mon[47373]: pgmap v206: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-05-08T23:53:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:06 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:53:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:06 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:06 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:53:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:06 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:06 smithi160 ceph-mon[46152]: pgmap v206: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-05-08T23:53:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:06 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:53:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:06 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:06 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:53:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:06 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:06.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 426610 9.78 MB/sec execute 182 sec latency 15.362 ms 2024-05-08T23:53:06.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:06 smithi012 ceph-mon[42503]: pgmap v206: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-05-08T23:53:06.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:06 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:53:06.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:06 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:06.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:06 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:53:06.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:06 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:07.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:07 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:53:07.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:07 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:53:07.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:07 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:07.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:07 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:07.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:07 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:53:07.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:07 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:53:07.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:07 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:07.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:07 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:07.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 428432 9.78 MB/sec execute 183 sec latency 34.123 ms 2024-05-08T23:53:07.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:07 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:53:07.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:07 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:53:07.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:07 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:07.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:07 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:08.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:08 smithi204 ceph-mon[47373]: pgmap v207: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 354 op/s 2024-05-08T23:53:08.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:08 smithi160 ceph-mon[46152]: pgmap v207: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 354 op/s 2024-05-08T23:53:08.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 429697 9.77 MB/sec execute 184 sec latency 30.010 ms 2024-05-08T23:53:08.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:08 smithi012 ceph-mon[42503]: pgmap v207: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 354 op/s 2024-05-08T23:53:09.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 432208 9.78 MB/sec execute 185 sec latency 17.624 ms 2024-05-08T23:53:10.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:10 smithi204 ceph-mon[47373]: pgmap v208: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 354 op/s 2024-05-08T23:53:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:10 smithi160 ceph-mon[46152]: pgmap v208: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 354 op/s 2024-05-08T23:53:10.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 433824 9.78 MB/sec execute 186 sec latency 15.186 ms 2024-05-08T23:53:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:10 smithi012 ceph-mon[42503]: pgmap v208: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 354 op/s 2024-05-08T23:53:11.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 435744 9.78 MB/sec execute 187 sec latency 16.410 ms 2024-05-08T23:53:12.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:11 smithi204 ceph-mon[47373]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T23:53:12.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:11 smithi204 ceph-mon[47373]: pgmap v210: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 321 op/s 2024-05-08T23:53:12.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:11 smithi160 ceph-mon[46152]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T23:53:12.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:11 smithi160 ceph-mon[46152]: pgmap v210: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 321 op/s 2024-05-08T23:53:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:11 smithi012 ceph-mon[42503]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T23:53:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:11 smithi012 ceph-mon[42503]: pgmap v210: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 321 op/s 2024-05-08T23:53:12.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 437221 9.77 MB/sec execute 188 sec latency 15.266 ms 2024-05-08T23:53:13.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:13 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:53:13.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:13 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:53:13.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 439228 9.78 MB/sec execute 189 sec latency 53.924 ms 2024-05-08T23:53:13.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:13 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:53:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.085+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2583440778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2efc100280 msgr2=0x7f2efc100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.085+0000 7f2f03f7d700 1 --2- 172.21.15.12:0/2583440778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2efc100280 0x7f2efc100700 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009990 tx=0x7f2eec009ca0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.086+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2583440778 shutdown_connections 2024-05-08T23:53:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.086+0000 7f2f03f7d700 1 --2- 172.21.15.12:0/2583440778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2efc100c40 0x7f2efc10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.086+0000 7f2f03f7d700 1 --2- 172.21.15.12:0/2583440778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2efc105f20 0x7f2efc106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.086+0000 7f2f03f7d700 1 --2- 172.21.15.12:0/2583440778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2efc100280 0x7f2efc100700 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.086+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2583440778 >> 172.21.15.12:0/2583440778 conn(0x7f2efc076bc0 msgr2=0x7f2efc076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.086+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2583440778 shutdown_connections 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.086+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2583440778 wait complete. 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.086+0000 7f2f03f7d700 1 Processor -- start 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.086+0000 7f2f03f7d700 1 -- start start 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f03f7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2efc100280 0x7f2efc1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f03f7d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2efc100c40 0x7f2efc197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f03f7d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2efc105f20 0x7f2efc19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f03f7d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc19c1f0 con 0x7f2efc100280 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f03f7d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc19c360 con 0x7f2efc105f20 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f03f7d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc19c520 con 0x7f2efc100c40 2024-05-08T23:53:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f01d19700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2efc100280 0x7f2efc1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f01d19700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2efc100280 0x7f2efc1972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34616/0 (socket says 172.21.15.12:34616) 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f0251a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2efc105f20 0x7f2efc19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f01d19700 1 -- 172.21.15.12:0/2397454480 learned_addr learned my addr 172.21.15.12:0/2397454480 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f01518700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2efc100c40 0x7f2efc197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f01d19700 1 -- 172.21.15.12:0/2397454480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2efc100c40 msgr2=0x7f2efc197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f01d19700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2efc100c40 0x7f2efc197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f01d19700 1 -- 172.21.15.12:0/2397454480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2efc105f20 msgr2=0x7f2efc19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f01d19700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2efc105f20 0x7f2efc19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f01d19700 1 -- 172.21.15.12:0/2397454480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec0095f0 con 0x7f2efc100280 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f0251a700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2efc105f20 0x7f2efc19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2f01d19700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2efc100280 0x7f2efc1972d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800e9a0 tx=0x7f2ef800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.087+0000 7f2ef2ffd700 1 -- 172.21.15.12:0/2397454480 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8009790 con 0x7f2efc100280 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.088+0000 7f2ef2ffd700 1 -- 172.21.15.12:0/2397454480 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef8010430 con 0x7f2efc100280 2024-05-08T23:53:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.088+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2397454480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc19c800 con 0x7f2efc100280 2024-05-08T23:53:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.088+0000 7f2ef2ffd700 1 -- 172.21.15.12:0/2397454480 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8018680 con 0x7f2efc100280 2024-05-08T23:53:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.088+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2397454480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2efc1a4040 con 0x7f2efc100280 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.090+0000 7f2ef2ffd700 1 -- 172.21.15.12:0/2397454480 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2ef80105a0 con 0x7f2efc100280 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.090+0000 7f2ef2ffd700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ee80a1310 0x7f2ee80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.090+0000 7f2ef2ffd700 1 -- 172.21.15.12:0/2397454480 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2ef8014070 con 0x7f2efc100280 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.091+0000 7f2f01518700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ee80a1310 0x7f2ee80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.091+0000 7f2f01518700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ee80a1310 0x7f2ee80a37d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b2e0 tx=0x7f2eec005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f10137b80 1 Processor -- start 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f10137b80 1 -- start start 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cb3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb4270 0x560a53cadf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 0x560a53cae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f10137b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53c050f0 con 0x560a53cb3950 2024-05-08T23:53:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f10137b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53c02ab0 con 0x560a53cb4270 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f10137b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53bfe3a0 con 0x560a53cae480 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f01d19700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cb3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f01518700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb4270 0x560a53cadf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f01d19700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cb3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34622/0 (socket says 172.21.15.12:34622) 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f01d19700 1 -- 172.21.15.12:0/3038134399 learned_addr learned my addr 172.21.15.12:0/3038134399 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.092+0000 7f2f0251a700 1 --2- 172.21.15.12:0/3038134399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 0x560a53cae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.093+0000 7f2f01d19700 1 -- 172.21.15.12:0/3038134399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 msgr2=0x560a53cae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.093+0000 7f2f01d19700 1 --2- 172.21.15.12:0/3038134399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 0x560a53cae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.093+0000 7f2f01d19700 1 -- 172.21.15.12:0/3038134399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb4270 msgr2=0x560a53cadf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.093+0000 7f2f01d19700 1 --2- 172.21.15.12:0/3038134399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb4270 0x560a53cadf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.093+0000 7f2f01d19700 1 -- 172.21.15.12:0/3038134399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a53a7f350 con 0x560a53cb3950 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.093+0000 7f2f01518700 1 --2- 172.21.15.12:0/3038134399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb4270 0x560a53cadf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.093+0000 7f2f01d19700 1 --2- 172.21.15.12:0/3038134399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cb3d30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80a1e10 tx=0x7f2ef80998a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e184c73a24dd74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.093+0000 7f2ee7fff700 1 -- 172.21.15.12:0/3038134399 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef809dcf0 con 0x560a53cb3950 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.093+0000 7f2ee7fff700 1 -- 172.21.15.12:0/3038134399 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef80a2340 con 0x560a53cb3950 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.093+0000 7f2ee7fff700 1 -- 172.21.15.12:0/3038134399 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80995d0 con 0x560a53cb3950 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.094+0000 7f2f10137b80 1 -- 172.21.15.12:0/3038134399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 msgr2=0x560a53cb3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.094+0000 7f2f10137b80 1 --2- 172.21.15.12:0/3038134399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cb3d30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80a1e10 tx=0x7f2ef80998a0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.094+0000 7f2f10137b80 1 -- 172.21.15.12:0/3038134399 shutdown_connections 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.094+0000 7f2f10137b80 1 --2- 172.21.15.12:0/3038134399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 0x560a53cae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.094+0000 7f2f10137b80 1 --2- 172.21.15.12:0/3038134399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cb3d30 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.094+0000 7f2f10137b80 1 --2- 172.21.15.12:0/3038134399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb4270 0x560a53cadf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.094+0000 7f2f10137b80 1 -- 172.21.15.12:0/3038134399 >> 172.21.15.12:0/3038134399 conn(0x560a53c0c1b0 msgr2=0x560a53ca8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.094+0000 7f2f10137b80 1 -- 172.21.15.12:0/3038134399 shutdown_connections 2024-05-08T23:53:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.094+0000 7f2f10137b80 1 -- 172.21.15.12:0/3038134399 wait complete. 2024-05-08T23:53:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.094+0000 7f2f10137b80 1 -- start start 2024-05-08T23:53:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f10137b80 1 Processor -- start 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 0x560a53cba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb3950 0x560a53cb75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb4270 0x560a53cb7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f10137b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53b9a500 con 0x560a53cb4270 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f10137b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53b9a1a0 con 0x560a53cb3950 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f10137b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53a81640 con 0x560a53cae480 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f10137b80 10 client.0 _ll_register_callbacks cb 0x560a53ba3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f01d19700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 0x560a53cba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f01518700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb3950 0x560a53cb75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f01d19700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 0x560a53cba2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57458/0 (socket says 172.21.15.12:57458) 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f01d19700 1 -- 172.21.15.12:0/1921088847 learned_addr learned my addr 172.21.15.12:0/1921088847 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.095+0000 7f2f0251a700 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb4270 0x560a53cb7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2f01d19700 1 -- 172.21.15.12:0/1921088847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb3950 msgr2=0x560a53cb75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2f01d19700 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb3950 0x560a53cb75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2f01d19700 1 -- 172.21.15.12:0/1921088847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb4270 msgr2=0x560a53cb7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2f01d19700 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb4270 0x560a53cb7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2f01d19700 1 -- 172.21.15.12:0/1921088847 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2efc10d530 con 0x560a53cae480 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2f01518700 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb3950 0x560a53cb75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2f01d19700 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 0x560a53cba2d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80ca5b0 tx=0x7f2ef8010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4c39ac778c22e7d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2ee57fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2ee57fa700 1 -- 172.21.15.12:0/1921088847 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef809a330 con 0x560a53cae480 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2ee57fa700 1 -- 172.21.15.12:0/1921088847 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2ef80b81c0 con 0x560a53cae480 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2f10137b80 1 -- 172.21.15.12:0/1921088847 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a53b57210 con 0x560a53cae480 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2ee57fa700 1 -- 172.21.15.12:0/1921088847 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ef80a1eb0 con 0x560a53cae480 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2f10137b80 10 client.27139 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.096+0000 7f2f10137b80 1 -- 172.21.15.12:0/1921088847 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f2efc1a4040 con 0x560a53cae480 2024-05-08T23:53:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.097+0000 7f2ee57fa700 1 -- 172.21.15.12:0/1921088847 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a53b57210 con 0x560a53cae480 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.097+0000 7f2ee57fa700 1 -- 172.21.15.12:0/1921088847 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2ef807e020 con 0x560a53cae480 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.097+0000 7f2f10137b80 10 client.27139 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.097+0000 7f2f10137b80 10 client.27139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.097+0000 7f2f10137b80 20 client.27139 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.097+0000 7f2f10137b80 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a53cc7770 0x560a53cc9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.097+0000 7f2f10137b80 4 client.27139 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.097+0000 7f2f10137b80 1 -- 172.21.15.12:0/1921088847 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a53c7d7b0 con 0x560a53cc7770 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.097+0000 7f2f01518700 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a53cc7770 0x560a53cc9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.098+0000 7f2ee57fa700 10 client.27139 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.098+0000 7f2f01518700 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a53cc7770 0x560a53cc9b60 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd7b5171605c19c server_cookie=b43d7d98910e1da7 in_seq=0 out_seq=0 2024-05-08T23:53:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.098+0000 7f2ee57fa700 1 -- 172.21.15.12:0/1921088847 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a53c7d7b0 con 0x560a53cc7770 2024-05-08T23:53:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.098+0000 7f2ee57fa700 10 client.27139 handle_command_reply: tid=0 2024-05-08T23:53:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.098+0000 7f2f10137b80 1 client.27139 shutdown 2024-05-08T23:53:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 1 -- 172.21.15.12:0/1921088847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 msgr2=0x560a53cba2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 0x560a53cba2d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80ca5b0 tx=0x7f2ef8010ab0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 1 -- 172.21.15.12:0/1921088847 shutdown_connections 2024-05-08T23:53:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb4270 0x560a53cb7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cae480 0x560a53cba2d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a53cc7770 0x560a53cc9b60 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 1 --2- 172.21.15.12:0/1921088847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cb3950 0x560a53cb75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2ee57fa700 0 client.27139 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 1 -- 172.21.15.12:0/1921088847 >> 172.21.15.12:0/1921088847 conn(0x560a53c0c1b0 msgr2=0x560a53cacf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 1 -- 172.21.15.12:0/1921088847 shutdown_connections 2024-05-08T23:53:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 1 -- 172.21.15.12:0/1921088847 wait complete. 2024-05-08T23:53:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.099+0000 7f2f10137b80 20 client.27139 trim_cache size 0 max 16384 2024-05-08T23:53:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.116+0000 7f2f10137b80 1 Processor -- start 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.116+0000 7f2f10137b80 1 -- start start 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.116+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cc9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.116+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.116+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 0x560a53cbeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.116+0000 7f2f10137b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53cced30 con 0x560a53cb3950 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.116+0000 7f2f10137b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53ca9330 con 0x560a53cae480 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.116+0000 7f2f10137b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53cb5ad0 con 0x560a53cb4270 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01d19700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cc9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01518700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01d19700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cc9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49842/0 (socket says 172.21.15.12:49842) 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01d19700 1 -- 172.21.15.12:0/1388706105 learned_addr learned my addr 172.21.15.12:0/1388706105 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f0251a700 1 --2- 172.21.15.12:0/1388706105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 0x560a53cbeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01d19700 1 -- 172.21.15.12:0/1388706105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 msgr2=0x560a53cbeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01d19700 1 --2- 172.21.15.12:0/1388706105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 0x560a53cbeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01d19700 1 -- 172.21.15.12:0/1388706105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 msgr2=0x560a53cae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01d19700 1 --2- 172.21.15.12:0/1388706105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01d19700 1 -- 172.21.15.12:0/1388706105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a53a2bb00 con 0x560a53cae480 2024-05-08T23:53:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01518700 1 --2- 172.21.15.12:0/1388706105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2f01d19700 1 --2- 172.21.15.12:0/1388706105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cc9b60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8096930 tx=0x7f2ef80ca5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b48368176c42e3a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/1388706105 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef809a330 con 0x560a53cae480 2024-05-08T23:53:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/1388706105 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef80b81c0 con 0x560a53cae480 2024-05-08T23:53:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.117+0000 7f2ef0ff9700 1 -- 172.21.15.12:0/1388706105 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8095f10 con 0x560a53cae480 2024-05-08T23:53:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.118+0000 7f2f10137b80 1 -- 172.21.15.12:0/1388706105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 msgr2=0x560a53cc9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.118+0000 7f2f10137b80 1 --2- 172.21.15.12:0/1388706105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cc9b60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8096930 tx=0x7f2ef80ca5e0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.118+0000 7f2f10137b80 1 -- 172.21.15.12:0/1388706105 shutdown_connections 2024-05-08T23:53:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.118+0000 7f2f10137b80 1 --2- 172.21.15.12:0/1388706105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 0x560a53cbeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.118+0000 7f2f10137b80 1 --2- 172.21.15.12:0/1388706105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cc9b60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.118+0000 7f2f10137b80 1 --2- 172.21.15.12:0/1388706105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.119+0000 7f2f10137b80 1 -- 172.21.15.12:0/1388706105 >> 172.21.15.12:0/1388706105 conn(0x560a53c0c1b0 msgr2=0x560a53cad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.119+0000 7f2f10137b80 1 -- 172.21.15.12:0/1388706105 shutdown_connections 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.119+0000 7f2f10137b80 1 -- 172.21.15.12:0/1388706105 wait complete. 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.119+0000 7f2f10137b80 1 -- start start 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.119+0000 7f2f10137b80 1 Processor -- start 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.119+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cbeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cbf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f10137b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 0x560a53cbcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f10137b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53cbd0a0 con 0x560a53cb3950 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f10137b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53cbd1e0 con 0x560a53cae480 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f10137b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a53cbd3a0 con 0x560a53cb4270 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f10137b80 10 client.0 _ll_register_callbacks cb 0x560a53ba3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01518700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cbf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01d19700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cbeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f0251a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 0x560a53cbcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01518700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cbf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34632/0 (socket says 172.21.15.12:34632) 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f0251a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 0x560a53cbcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57478/0 (socket says 172.21.15.12:57478) 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01518700 1 -- 172.21.15.12:0/3464959266 learned_addr learned my addr 172.21.15.12:0/3464959266 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01d19700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cbeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49844/0 (socket says 172.21.15.12:49844) 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01d19700 1 -- 172.21.15.12:0/3464959266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 msgr2=0x560a53cbcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01d19700 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 0x560a53cbcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01d19700 1 -- 172.21.15.12:0/3464959266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 msgr2=0x560a53cbf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01d19700 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cbf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01d19700 1 -- 172.21.15.12:0/3464959266 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eec00ba80 con 0x560a53cae480 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01518700 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cbf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f01d19700 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cbeb80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80963c0 tx=0x7f2ef8010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12f0de58bb9dbe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2ee57fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2ee57fa700 1 -- 172.21.15.12:0/3464959266 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8099030 con 0x560a53cae480 2024-05-08T23:53:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2ee57fa700 1 -- 172.21.15.12:0/3464959266 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2ef8095cb0 con 0x560a53cae480 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.120+0000 7f2f10137b80 1 -- 172.21.15.12:0/3464959266 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ef80a2340 con 0x560a53cae480 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2ee57fa700 1 -- 172.21.15.12:0/3464959266 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ef8099190 con 0x560a53cae480 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2f10137b80 10 client.27146 fetch_fsmap learned FSMap version 32 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2f10137b80 1 -- 172.21.15.12:0/3464959266 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f2efc1a4040 con 0x560a53cae480 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2ee57fa700 1 -- 172.21.15.12:0/3464959266 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef80a2340 con 0x560a53cae480 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2ee57fa700 1 -- 172.21.15.12:0/3464959266 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2ef809ec10 con 0x560a53cae480 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2f10137b80 10 client.27146 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2f10137b80 10 client.27146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2f10137b80 20 client.27146 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2f10137b80 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a53d1d710 0x560a53d1fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2f10137b80 4 client.27146 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2f10137b80 1 -- 172.21.15.12:0/3464959266 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a53bf6e60 con 0x560a53d1d710 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.121+0000 7f2f01518700 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a53d1d710 0x560a53d1fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.122+0000 7f2f01518700 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a53d1d710 0x560a53d1fb00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e0826bb3a46965 server_cookie=38ffd5a46447acd1 in_seq=0 out_seq=0 2024-05-08T23:53:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.122+0000 7f2ee57fa700 10 client.27146 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.184+0000 7f2ee57fa700 1 -- 172.21.15.12:0/3464959266 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a53bf6e60 con 0x560a53d1d710 2024-05-08T23:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.184+0000 7f2ee57fa700 10 client.27146 handle_command_reply: tid=0 2024-05-08T23:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.184+0000 7f2f10137b80 1 client.27146 shutdown 2024-05-08T23:53:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 1 -- 172.21.15.12:0/3464959266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 msgr2=0x560a53cbeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cbeb80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80963c0 tx=0x7f2ef8010ab0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 1 -- 172.21.15.12:0/3464959266 shutdown_connections 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a53cb4270 0x560a53cbcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a53d1d710 0x560a53d1fb00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a53cae480 0x560a53cbeb80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 1 --2- 172.21.15.12:0/3464959266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a53cb3950 0x560a53cbf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 1 -- 172.21.15.12:0/3464959266 >> 172.21.15.12:0/3464959266 conn(0x560a53c0c1b0 msgr2=0x560a53caa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2ee57fa700 0 client.27146 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 1 -- 172.21.15.12:0/3464959266 shutdown_connections 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 1 -- 172.21.15.12:0/3464959266 wait complete. 2024-05-08T23:53:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.185+0000 7f2f10137b80 20 client.27146 trim_cache size 0 max 16384 2024-05-08T23:53:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:53:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:53:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:53:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2397454480 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ee80a1310 msgr2=0x7f2ee80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ee80a1310 0x7f2ee80a37d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b2e0 tx=0x7f2eec005960 comp rx=0 tx=0).stop 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2397454480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2efc100280 msgr2=0x7f2efc1972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2efc100280 0x7f2efc1972d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800e9a0 tx=0x7f2ef800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2397454480 shutdown_connections 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ee80a1310 0x7f2ee80a37d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2efc105f20 0x7f2efc19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2efc100280 0x7f2efc1972d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 --2- 172.21.15.12:0/2397454480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2efc100c40 0x7f2efc197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2397454480 >> 172.21.15.12:0/2397454480 conn(0x7f2efc076bc0 msgr2=0x7f2efc069350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2397454480 shutdown_connections 2024-05-08T23:53:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:14.186+0000 7f2f03f7d700 1 -- 172.21.15.12:0/2397454480 wait complete. 2024-05-08T23:53:14.512 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:53:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-08T23:53:14.513 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:53:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-08T23:53:14.513 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:53:14 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/2995720750 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5613170c61a0 con 0x56131709c800 2024-05-08T23:53:14.513 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:53:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-08T23:53:14.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:14 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:53:14.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:14 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:53:14.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:14 smithi204 ceph-mon[47373]: pgmap v211: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-05-08T23:53:14.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:14 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-08T23:53:14.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:14 smithi204 ceph-mon[47373]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T23:53:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:14 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:53:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:14 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:53:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:14 smithi160 ceph-mon[46152]: pgmap v211: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-05-08T23:53:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:14 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-08T23:53:14.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:14 smithi160 ceph-mon[46152]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T23:53:14.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 440593 9.76 MB/sec execute 190 sec latency 54.742 ms 2024-05-08T23:53:14.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:14 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:53:14.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:14 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:53:14.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:14 smithi012 ceph-mon[42503]: pgmap v211: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-05-08T23:53:14.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:14 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-08T23:53:14.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:14 smithi012 ceph-mon[42503]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T23:53:15.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:15 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/2083654353,v1:172.21.15.204:6837/2083654353] up:boot 2024-05-08T23:53:15.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:15 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:53:15.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:15 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:53:15.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:15 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:15.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:15 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/2083654353,v1:172.21.15.204:6837/2083654353] up:boot 2024-05-08T23:53:15.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:15 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:53:15.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:15 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:53:15.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:15 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:15.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 442349 9.75 MB/sec execute 191 sec latency 17.428 ms 2024-05-08T23:53:15.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:15 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/2083654353,v1:172.21.15.204:6837/2083654353] up:boot 2024-05-08T23:53:15.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:15 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:53:15.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:15 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:53:15.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:15 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:16.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:16 smithi204 ceph-mon[47373]: pgmap v213: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-05-08T23:53:16.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:16 smithi160 ceph-mon[46152]: pgmap v213: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-05-08T23:53:16.869 INFO:tasks.workunit.client.0.smithi012.stdout: 1 443843 9.76 MB/sec execute 192 sec latency 100.862 ms 2024-05-08T23:53:16.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:16 smithi012 ceph-mon[42503]: pgmap v213: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-05-08T23:53:17.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 445322 9.74 MB/sec execute 193 sec latency 48.394 ms 2024-05-08T23:53:18.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:18 smithi204 ceph-mon[47373]: pgmap v214: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 366 op/s 2024-05-08T23:53:18.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:18 smithi160 ceph-mon[46152]: pgmap v214: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 366 op/s 2024-05-08T23:53:18.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 447270 9.76 MB/sec execute 194 sec latency 52.151 ms 2024-05-08T23:53:18.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:18 smithi012 ceph-mon[42503]: pgmap v214: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 366 op/s 2024-05-08T23:53:19.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 448861 9.74 MB/sec execute 195 sec latency 17.392 ms 2024-05-08T23:53:20.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:20 smithi204 ceph-mon[47373]: pgmap v215: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 338 op/s 2024-05-08T23:53:20.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:20 smithi160 ceph-mon[46152]: pgmap v215: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 338 op/s 2024-05-08T23:53:20.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 450874 9.76 MB/sec execute 196 sec latency 15.738 ms 2024-05-08T23:53:20.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:20 smithi012 ceph-mon[42503]: pgmap v215: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 338 op/s 2024-05-08T23:53:21.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 452676 9.74 MB/sec execute 197 sec latency 32.004 ms 2024-05-08T23:53:22.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:22 smithi204 ceph-mon[47373]: pgmap v216: 97 pgs: 97 active+clean; 125 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-05-08T23:53:22.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:22 smithi160 ceph-mon[46152]: pgmap v216: 97 pgs: 97 active+clean; 125 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-05-08T23:53:22.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 454471 9.75 MB/sec execute 198 sec latency 14.853 ms 2024-05-08T23:53:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:22 smithi012 ceph-mon[42503]: pgmap v216: 97 pgs: 97 active+clean; 125 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-05-08T23:53:23.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 456223 9.73 MB/sec execute 199 sec latency 45.806 ms 2024-05-08T23:53:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:24 smithi204 ceph-mon[47373]: pgmap v217: 97 pgs: 97 active+clean; 125 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-05-08T23:53:24.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:24 smithi160 ceph-mon[46152]: pgmap v217: 97 pgs: 97 active+clean; 125 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-05-08T23:53:24.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 457980 9.75 MB/sec execute 200 sec latency 16.245 ms 2024-05-08T23:53:24.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:24 smithi012 ceph-mon[42503]: pgmap v217: 97 pgs: 97 active+clean; 125 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-05-08T23:53:25.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 458829 9.79 MB/sec execute 201 sec latency 65.656 ms 2024-05-08T23:53:25.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:25 smithi012 ceph-mon[42503]: pgmap v218: 97 pgs: 97 active+clean; 125 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-05-08T23:53:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:25 smithi204 ceph-mon[47373]: pgmap v218: 97 pgs: 97 active+clean; 125 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-05-08T23:53:26.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:25 smithi160 ceph-mon[46152]: pgmap v218: 97 pgs: 97 active+clean; 125 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-05-08T23:53:26.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 461337 9.82 MB/sec execute 202 sec latency 40.829 ms 2024-05-08T23:53:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.319+0000 7f6daf469700 1 -- 172.21.15.12:0/2792721712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da8105e60 msgr2=0x7f6da8106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.319+0000 7f6daf469700 1 --2- 172.21.15.12:0/2792721712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da8105e60 0x7f6da8106240 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009990 tx=0x7f6da4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.319+0000 7f6daf469700 1 -- 172.21.15.12:0/2792721712 shutdown_connections 2024-05-08T23:53:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.319+0000 7f6daf469700 1 --2- 172.21.15.12:0/2792721712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da8100760 0x7f6da810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.319+0000 7f6daf469700 1 --2- 172.21.15.12:0/2792721712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da8105e60 0x7f6da8106240 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.319+0000 7f6daf469700 1 --2- 172.21.15.12:0/2792721712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6da80ffda0 0x7f6da8100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.319+0000 7f6daf469700 1 -- 172.21.15.12:0/2792721712 >> 172.21.15.12:0/2792721712 conn(0x7f6da8077000 msgr2=0x7f6da8077410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.319+0000 7f6daf469700 1 -- 172.21.15.12:0/2792721712 shutdown_connections 2024-05-08T23:53:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.319+0000 7f6daf469700 1 -- 172.21.15.12:0/2792721712 wait complete. 2024-05-08T23:53:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daf469700 1 Processor -- start 2024-05-08T23:53:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daf469700 1 -- start start 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daf469700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da80ffda0 0x7f6da81961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daf469700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da8100760 0x7f6da8196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daf469700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6da8105e60 0x7f6da819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daf469700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da819b110 con 0x7f6da8100760 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daf469700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da819b280 con 0x7f6da8105e60 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daf469700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da819b440 con 0x7f6da80ffda0 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daca04700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da8100760 0x7f6da8196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6dada06700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6da8105e60 0x7f6da819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6dad205700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da80ffda0 0x7f6da81961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daca04700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da8100760 0x7f6da8196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45202/0 (socket says 172.21.15.12:45202) 2024-05-08T23:53:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6dada06700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6da8105e60 0x7f6da819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:37614/0 (socket says 172.21.15.12:37614) 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6daca04700 1 -- 172.21.15.12:0/1567682022 learned_addr learned my addr 172.21.15.12:0/1567682022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6dad205700 1 -- 172.21.15.12:0/1567682022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6da8105e60 msgr2=0x7f6da819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6dad205700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6da8105e60 0x7f6da819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6dad205700 1 -- 172.21.15.12:0/1567682022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da8100760 msgr2=0x7f6da8196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.320+0000 7f6dad205700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da8100760 0x7f6da8196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.321+0000 7f6dad205700 1 -- 172.21.15.12:0/1567682022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da40095f0 con 0x7f6da80ffda0 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.321+0000 7f6dad205700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da80ffda0 0x7f6da81961f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6da40059d0 tx=0x7f6da4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.321+0000 7f6d9e7fc700 1 -- 172.21.15.12:0/1567682022 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da401d070 con 0x7f6da80ffda0 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.321+0000 7f6d9e7fc700 1 -- 172.21.15.12:0/1567682022 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6da4022470 con 0x7f6da80ffda0 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.321+0000 7f6daf469700 1 -- 172.21.15.12:0/1567682022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da819b6c0 con 0x7f6da80ffda0 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.321+0000 7f6d9e7fc700 1 -- 172.21.15.12:0/1567682022 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4017650 con 0x7f6da80ffda0 2024-05-08T23:53:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.321+0000 7f6daf469700 1 -- 172.21.15.12:0/1567682022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da81a2f60 con 0x7f6da80ffda0 2024-05-08T23:53:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.323+0000 7f6d9e7fc700 1 -- 172.21.15.12:0/1567682022 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6da40225e0 con 0x7f6da80ffda0 2024-05-08T23:53:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.324+0000 7f6d9e7fc700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d940a1290 0x7f6d940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.324+0000 7f6d9e7fc700 1 -- 172.21.15.12:0/1567682022 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6da40cddb0 con 0x7f6da80ffda0 2024-05-08T23:53:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.324+0000 7f6daca04700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d940a1290 0x7f6d940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.325+0000 7f6daca04700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d940a1290 0x7f6d940a3750 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6da8197740 tx=0x7f6d9800b500 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.325+0000 7f6dbb623b80 1 Processor -- start 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.325+0000 7f6dbb623b80 1 -- start start 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 0x5588b25f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 0x5588b25f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b254b0f0 con 0x5588b25f9950 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b2548ab0 con 0x5588b25f4480 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b25443a0 con 0x5588b25fa270 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dad205700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6daca04700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 0x5588b25f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dad205700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45212/0 (socket says 172.21.15.12:45212) 2024-05-08T23:53:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dad205700 1 -- 172.21.15.12:0/4092918259 learned_addr learned my addr 172.21.15.12:0/4092918259 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dada06700 1 --2- 172.21.15.12:0/4092918259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 0x5588b25f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6daca04700 1 -- 172.21.15.12:0/4092918259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 msgr2=0x5588b25f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6daca04700 1 --2- 172.21.15.12:0/4092918259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 0x5588b25f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6daca04700 1 -- 172.21.15.12:0/4092918259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 msgr2=0x5588b25f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6daca04700 1 --2- 172.21.15.12:0/4092918259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6daca04700 1 -- 172.21.15.12:0/4092918259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588b23c5350 con 0x5588b25fa270 2024-05-08T23:53:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6dad205700 1 --2- 172.21.15.12:0/4092918259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6daca04700 1 --2- 172.21.15.12:0/4092918259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 0x5588b25f3f40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6d98005950 tx=0x7f6d98013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd768f730fb6db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.326+0000 7f6d937fe700 1 -- 172.21.15.12:0/4092918259 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98011d30 con 0x5588b25fa270 2024-05-08T23:53:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6d937fe700 1 -- 172.21.15.12:0/4092918259 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d98015070 con 0x5588b25fa270 2024-05-08T23:53:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6d937fe700 1 -- 172.21.15.12:0/4092918259 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9801c610 con 0x5588b25fa270 2024-05-08T23:53:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6dbb623b80 1 -- 172.21.15.12:0/4092918259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 msgr2=0x5588b25f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/4092918259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 0x5588b25f3f40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6d98005950 tx=0x7f6d98013000 comp rx=0 tx=0).stop 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6dbb623b80 1 -- 172.21.15.12:0/4092918259 shutdown_connections 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/4092918259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 0x5588b25f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/4092918259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/4092918259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 0x5588b25f3f40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6dbb623b80 1 -- 172.21.15.12:0/4092918259 >> 172.21.15.12:0/4092918259 conn(0x5588b25521b0 msgr2=0x5588b25ee840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6dbb623b80 1 -- 172.21.15.12:0/4092918259 shutdown_connections 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.327+0000 7f6dbb623b80 1 -- 172.21.15.12:0/4092918259 wait complete. 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.328+0000 7f6dbb623b80 1 -- start start 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.328+0000 7f6dbb623b80 1 Processor -- start 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.328+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f4480 0x5588b26001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.328+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f9950 0x5588b25fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.328+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25fa270 0x5588b25fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.328+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b24e0500 con 0x5588b25f4480 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.328+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b24e01a0 con 0x5588b25fa270 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.328+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b23c7640 con 0x5588b25f9950 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6dbb623b80 10 client.0 _ll_register_callbacks cb 0x5588b24e9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6dad205700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f4480 0x5588b26001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6dad205700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f4480 0x5588b26001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45222/0 (socket says 172.21.15.12:45222) 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6dad205700 1 -- 172.21.15.12:0/2740183821 learned_addr learned my addr 172.21.15.12:0/2740183821 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6daca04700 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f9950 0x5588b25fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6dada06700 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25fa270 0x5588b25fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6daca04700 1 -- 172.21.15.12:0/2740183821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25fa270 msgr2=0x5588b25fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6daca04700 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25fa270 0x5588b25fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6daca04700 1 -- 172.21.15.12:0/2740183821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f4480 msgr2=0x5588b26001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6daca04700 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f4480 0x5588b26001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6daca04700 1 -- 172.21.15.12:0/2740183821 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da81a2f60 con 0x5588b25f9950 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6dad205700 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f4480 0x5588b26001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6daca04700 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f9950 0x5588b25fd4b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6d9801eda0 tx=0x7f6d98012d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574eb77184a17b76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6d90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6d90ff9700 1 -- 172.21.15.12:0/2740183821 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9802b070 con 0x5588b25f9950 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6d90ff9700 1 -- 172.21.15.12:0/2740183821 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6d98017070 con 0x5588b25f9950 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.329+0000 7f6dbb623b80 1 -- 172.21.15.12:0/2740183821 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588b249d210 con 0x5588b25f9950 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6d90ff9700 1 -- 172.21.15.12:0/2740183821 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d9801eea0 con 0x5588b25f9950 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6dbb623b80 10 client.27163 fetch_fsmap learned FSMap version 34 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6dbb623b80 1 -- 172.21.15.12:0/2740183821 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f6d98010c20 con 0x5588b25f9950 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6d90ff9700 1 -- 172.21.15.12:0/2740183821 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5588b249d210 con 0x5588b25f9950 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6d90ff9700 1 -- 172.21.15.12:0/2740183821 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6d98003680 con 0x5588b25f9950 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6dbb623b80 10 client.27163 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T23:53:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6dbb623b80 10 client.27163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6dbb623b80 20 client.27163 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588b2611e10 0x5588b2614200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6dbb623b80 4 client.27163 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6dbb623b80 1 -- 172.21.15.12:0/2740183821 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588b25c37b0 con 0x5588b2611e10 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.330+0000 7f6dad205700 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588b2611e10 0x5588b2614200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.331+0000 7f6dad205700 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588b2611e10 0x5588b2614200 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f55d7b43431cf9d5 server_cookie=68c7763edb2bdd49 in_seq=0 out_seq=0 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.331+0000 7f6d90ff9700 10 client.27163 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6d90ff9700 1 -- 172.21.15.12:0/2740183821 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588b25c37b0 con 0x5588b2611e10 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6d90ff9700 10 client.27163 handle_command_reply: tid=0 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 client.27163 shutdown 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 -- 172.21.15.12:0/2740183821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f9950 msgr2=0x5588b25fd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f9950 0x5588b25fd4b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6d9801eda0 tx=0x7f6d98012d50 comp rx=0 tx=0).stop 2024-05-08T23:53:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 -- 172.21.15.12:0/2740183821 shutdown_connections 2024-05-08T23:53:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588b2611e10 0x5588b2614200 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25fa270 0x5588b25fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f4480 0x5588b26001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/2740183821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f9950 0x5588b25fd4b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 -- 172.21.15.12:0/2740183821 >> 172.21.15.12:0/2740183821 conn(0x5588b25521b0 msgr2=0x5588b25f84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6d90ff9700 0 client.27163 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 -- 172.21.15.12:0/2740183821 shutdown_connections 2024-05-08T23:53:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.332+0000 7f6dbb623b80 1 -- 172.21.15.12:0/2740183821 wait complete. 2024-05-08T23:53:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.333+0000 7f6dbb623b80 20 client.27163 trim_cache size 0 max 16384 2024-05-08T23:53:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6dbb623b80 1 Processor -- start 2024-05-08T23:53:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6dbb623b80 1 -- start start 2024-05-08T23:53:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f4480 0x5588b2614200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25ee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25fa270 0x5588b25f4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b2619410 con 0x5588b25f9950 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b25f3050 con 0x5588b25fa270 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b25fe370 con 0x5588b25f4480 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6daca04700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6daca04700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25ee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45234/0 (socket says 172.21.15.12:45234) 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6daca04700 1 -- 172.21.15.12:0/825211481 learned_addr learned my addr 172.21.15.12:0/825211481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6dad205700 1 --2- 172.21.15.12:0/825211481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f4480 0x5588b2614200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6dada06700 1 --2- 172.21.15.12:0/825211481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25fa270 0x5588b25f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6daca04700 1 -- 172.21.15.12:0/825211481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f4480 msgr2=0x5588b2614200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6daca04700 1 --2- 172.21.15.12:0/825211481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f4480 0x5588b2614200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.353+0000 7f6daca04700 1 -- 172.21.15.12:0/825211481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25fa270 msgr2=0x5588b25f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.354+0000 7f6daca04700 1 --2- 172.21.15.12:0/825211481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25fa270 0x5588b25f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.354+0000 7f6daca04700 1 -- 172.21.15.12:0/825211481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588b2371b00 con 0x5588b25f9950 2024-05-08T23:53:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.354+0000 7f6daca04700 1 --2- 172.21.15.12:0/825211481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25ee910 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6d9801edf0 tx=0x7f6d98003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8a9244c6e96204 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.354+0000 7f6d93fff700 1 -- 172.21.15.12:0/825211481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9802b070 con 0x5588b25f9950 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.354+0000 7f6d93fff700 1 -- 172.21.15.12:0/825211481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d9801ee20 con 0x5588b25f9950 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.354+0000 7f6d93fff700 1 -- 172.21.15.12:0/825211481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9801e850 con 0x5588b25f9950 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 -- 172.21.15.12:0/825211481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 msgr2=0x5588b25ee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/825211481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25ee910 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6d9801edf0 tx=0x7f6d98003b80 comp rx=0 tx=0).stop 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 -- 172.21.15.12:0/825211481 shutdown_connections 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/825211481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25fa270 0x5588b25f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/825211481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25f4480 0x5588b2614200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/825211481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b25ee910 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 -- 172.21.15.12:0/825211481 >> 172.21.15.12:0/825211481 conn(0x5588b25521b0 msgr2=0x5588b25f00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 -- 172.21.15.12:0/825211481 shutdown_connections 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 -- 172.21.15.12:0/825211481 wait complete. 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 -- start start 2024-05-08T23:53:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.355+0000 7f6dbb623b80 1 Processor -- start 2024-05-08T23:53:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 0x5588b2604c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b26051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6dbb623b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 0x5588b2600c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b2601180 con 0x5588b25f9950 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b26012f0 con 0x5588b25f4480 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6dbb623b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b26014b0 con 0x5588b25fa270 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6dbb623b80 10 client.0 _ll_register_callbacks cb 0x5588b24e9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6daca04700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b26051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6daca04700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b26051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45236/0 (socket says 172.21.15.12:45236) 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6dad205700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 0x5588b2604c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6daca04700 1 -- 172.21.15.12:0/1387200804 learned_addr learned my addr 172.21.15.12:0/1387200804 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.356+0000 7f6dada06700 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 0x5588b2600c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6daca04700 1 -- 172.21.15.12:0/1387200804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 msgr2=0x5588b2600c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6daca04700 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 0x5588b2600c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6daca04700 1 -- 172.21.15.12:0/1387200804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 msgr2=0x5588b2604c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6daca04700 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 0x5588b2604c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6daca04700 1 -- 172.21.15.12:0/1387200804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d98010c20 con 0x5588b25f9950 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6dad205700 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 0x5588b2604c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6daca04700 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b26051c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800b5b0 tx=0x7f6d98005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406d4f578f33fe7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6d90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6d90ff9700 1 -- 172.21.15.12:0/1387200804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9802b070 con 0x5588b25f9950 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6dbb623b80 1 -- 172.21.15.12:0/1387200804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d98015070 con 0x5588b25f9950 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6d90ff9700 1 -- 172.21.15.12:0/1387200804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6d98017040 con 0x5588b25f9950 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6d90ff9700 1 -- 172.21.15.12:0/1387200804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d9801e630 con 0x5588b25f9950 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6dbb623b80 10 client.17382 fetch_fsmap learned FSMap version 34 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6dbb623b80 1 -- 172.21.15.12:0/1387200804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5588b250a500 con 0x5588b25f9950 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.357+0000 7f6d90ff9700 1 -- 172.21.15.12:0/1387200804 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d98015070 con 0x5588b25f9950 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.358+0000 7f6d90ff9700 1 -- 172.21.15.12:0/1387200804 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6d980399d0 con 0x5588b25f9950 2024-05-08T23:53:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.358+0000 7f6dbb623b80 10 client.17382 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T23:53:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.358+0000 7f6dbb623b80 10 client.17382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.358+0000 7f6dbb623b80 20 client.17382 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.358+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588b2667d60 0x5588b266a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.358+0000 7f6dbb623b80 4 client.17382 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:53:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.358+0000 7f6dbb623b80 1 -- 172.21.15.12:0/1387200804 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588b253ce60 con 0x5588b2667d60 2024-05-08T23:53:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.358+0000 7f6dad205700 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588b2667d60 0x5588b266a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.358+0000 7f6d90ff9700 10 client.17382 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.358+0000 7f6dad205700 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588b2667d60 0x5588b266a150 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d411a3fb54068338 server_cookie=c29dc726fe89eba in_seq=0 out_seq=0 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6d90ff9700 1 -- 172.21.15.12:0/1387200804 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588b253ce60 con 0x5588b2667d60 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6d90ff9700 10 client.17382 handle_command_reply: tid=0 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6dbb623b80 1 client.17382 shutdown 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6dbb623b80 1 -- 172.21.15.12:0/1387200804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 msgr2=0x5588b26051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b26051c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800b5b0 tx=0x7f6d98005c20 comp rx=0 tx=0).stop 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6dbb623b80 1 -- 172.21.15.12:0/1387200804 shutdown_connections 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588b25fa270 0x5588b2600c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588b2667d60 0x5588b266a150 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5588b25f4480 0x5588b2604c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6dbb623b80 1 --2- 172.21.15.12:0/1387200804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588b25f9950 0x5588b26051c0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.421+0000 7f6dbb623b80 1 -- 172.21.15.12:0/1387200804 >> 172.21.15.12:0/1387200804 conn(0x5588b25521b0 msgr2=0x5588b25fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.423+0000 7f6d90ff9700 0 client.17382 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.423+0000 7f6dbb623b80 1 -- 172.21.15.12:0/1387200804 shutdown_connections 2024-05-08T23:53:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.423+0000 7f6dbb623b80 1 -- 172.21.15.12:0/1387200804 wait complete. 2024-05-08T23:53:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.423+0000 7f6dbb623b80 20 client.17382 trim_cache size 0 max 16384 2024-05-08T23:53:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:53:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:53:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:53:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:53:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.424+0000 7f6daf469700 1 -- 172.21.15.12:0/1567682022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d940a1290 msgr2=0x7f6d940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.424+0000 7f6daf469700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d940a1290 0x7f6d940a3750 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6da8197740 tx=0x7f6d9800b500 comp rx=0 tx=0).stop 2024-05-08T23:53:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.424+0000 7f6daf469700 1 -- 172.21.15.12:0/1567682022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da80ffda0 msgr2=0x7f6da81961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.424+0000 7f6daf469700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da80ffda0 0x7f6da81961f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6da40059d0 tx=0x7f6da4004730 comp rx=0 tx=0).stop 2024-05-08T23:53:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.424+0000 7f6daf469700 1 -- 172.21.15.12:0/1567682022 shutdown_connections 2024-05-08T23:53:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.424+0000 7f6daf469700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d940a1290 0x7f6d940a3750 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.424+0000 7f6daf469700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6da8105e60 0x7f6da819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.424+0000 7f6daf469700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da80ffda0 0x7f6da81961f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.424+0000 7f6daf469700 1 --2- 172.21.15.12:0/1567682022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da8100760 0x7f6da8196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.424+0000 7f6daf469700 1 -- 172.21.15.12:0/1567682022 >> 172.21.15.12:0/1567682022 conn(0x7f6da8077000 msgr2=0x7f6da80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.425+0000 7f6daf469700 1 -- 172.21.15.12:0/1567682022 shutdown_connections 2024-05-08T23:53:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:27.425+0000 7f6daf469700 1 -- 172.21.15.12:0/1567682022 wait complete. 2024-05-08T23:53:27.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 462716 9.81 MB/sec execute 203 sec latency 43.670 ms 2024-05-08T23:53:28.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:53:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-08T23:53:28.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:53:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-08T23:53:28.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:53:27 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/764444975 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ee9104a1a0 con 0x55ee91020c00 2024-05-08T23:53:28.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:53:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-08T23:53:28.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:28 smithi204 ceph-mon[47373]: pgmap v219: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 351 op/s 2024-05-08T23:53:28.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:28 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-08T23:53:28.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:28 smithi204 ceph-mon[47373]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T23:53:28.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:28 smithi160 ceph-mon[46152]: pgmap v219: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 351 op/s 2024-05-08T23:53:28.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:28 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-08T23:53:28.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:28 smithi160 ceph-mon[46152]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T23:53:28.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 464830 9.82 MB/sec execute 204 sec latency 197.692 ms 2024-05-08T23:53:28.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:28 smithi012 ceph-mon[42503]: pgmap v219: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 351 op/s 2024-05-08T23:53:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:28 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-08T23:53:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:28 smithi012 ceph-mon[42503]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T23:53:29.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:29 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/2842389121,v1:172.21.15.204:6837/2842389121] up:boot 2024-05-08T23:53:29.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:29 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:53:29.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:53:29.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:29 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:29 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/2842389121,v1:172.21.15.204:6837/2842389121] up:boot 2024-05-08T23:53:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:29 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:53:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:53:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:29 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:29.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 464830 9.77 MB/sec execute 205 sec latency 1197.787 ms 2024-05-08T23:53:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:29 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/2842389121,v1:172.21.15.204:6837/2842389121] up:boot 2024-05-08T23:53:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:29 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:53:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:53:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:29 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:30.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:30 smithi204 ceph-mon[47373]: pgmap v221: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-05-08T23:53:30.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:30 smithi160 ceph-mon[46152]: pgmap v221: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-05-08T23:53:30.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 464830 9.72 MB/sec execute 206 sec latency 2197.890 ms 2024-05-08T23:53:30.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:30 smithi012 ceph-mon[42503]: pgmap v221: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-05-08T23:53:31.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 464830 9.68 MB/sec execute 207 sec latency 3197.990 ms 2024-05-08T23:53:32.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:32 smithi204 ceph-mon[47373]: pgmap v222: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-05-08T23:53:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:32 smithi160 ceph-mon[46152]: pgmap v222: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-05-08T23:53:32.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 464874 9.63 MB/sec execute 208 sec latency 4092.433 ms 2024-05-08T23:53:32.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:32 smithi012 ceph-mon[42503]: pgmap v222: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-05-08T23:53:33.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 464911 9.58 MB/sec execute 209 sec latency 566.114 ms 2024-05-08T23:53:34.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:34 smithi204 ceph-mon[47373]: pgmap v223: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-05-08T23:53:34.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:34 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:53:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:34 smithi160 ceph-mon[46152]: pgmap v223: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-05-08T23:53:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:34 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:53:34.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 464911 9.54 MB/sec execute 210 sec latency 1472.745 ms 2024-05-08T23:53:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:34 smithi012 ceph-mon[42503]: pgmap v223: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-05-08T23:53:34.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:34 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:53:35.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:35 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:53:35.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:35 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:53:35.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:35 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:53:35.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:35 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:53:35.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:35 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:53:35.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:35 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:53:35.871 INFO:tasks.workunit.client.0.smithi012.stdout: 1 464911 9.49 MB/sec execute 211 sec latency 2472.813 ms 2024-05-08T23:53:35.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:35 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:53:35.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:35 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:53:35.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:35 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:53:36.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:53:36.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.674+0000 7f8fe33b7700 1 -- 172.21.15.12:0/2845002622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fdc100920 msgr2=0x7f8fdc104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.674+0000 7f8fe33b7700 1 --2- 172.21.15.12:0/2845002622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fdc100920 0x7f8fdc104960 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009990 tx=0x7f8fcc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.674+0000 7f8fe33b7700 1 -- 172.21.15.12:0/2845002622 shutdown_connections 2024-05-08T23:53:36.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.674+0000 7f8fe33b7700 1 --2- 172.21.15.12:0/2845002622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fdc105090 0x7f8fdc10cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.674+0000 7f8fe33b7700 1 --2- 172.21.15.12:0/2845002622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc0fff70 0x7f8fdc100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.674+0000 7f8fe33b7700 1 --2- 172.21.15.12:0/2845002622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fdc100920 0x7f8fdc104960 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.674+0000 7f8fe33b7700 1 -- 172.21.15.12:0/2845002622 >> 172.21.15.12:0/2845002622 conn(0x7f8fdc0fb7b0 msgr2=0x7f8fdc0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:36.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.674+0000 7f8fe33b7700 1 -- 172.21.15.12:0/2845002622 shutdown_connections 2024-05-08T23:53:36.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.674+0000 7f8fe33b7700 1 -- 172.21.15.12:0/2845002622 wait complete. 2024-05-08T23:53:36.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe33b7700 1 Processor -- start 2024-05-08T23:53:36.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe33b7700 1 -- start start 2024-05-08T23:53:36.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe33b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc0fff70 0x7f8fdc1962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe33b7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fdc105090 0x7f8fdc1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe33b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fdc19ab40 0x7f8fdc19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe33b7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc19b5c0 con 0x7f8fdc105090 2024-05-08T23:53:36.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe33b7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc19b730 con 0x7f8fdc19ab40 2024-05-08T23:53:36.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe33b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc19b8f0 con 0x7f8fdc0fff70 2024-05-08T23:53:36.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe0952700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fdc105090 0x7f8fdc1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe0952700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fdc105090 0x7f8fdc1967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43876/0 (socket says 172.21.15.12:43876) 2024-05-08T23:53:36.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe0952700 1 -- 172.21.15.12:0/3487400203 learned_addr learned my addr 172.21.15.12:0/3487400203 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:36.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.675+0000 7f8fe1153700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc0fff70 0x7f8fdc1962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fe1954700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fdc19ab40 0x7f8fdc19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fe0952700 1 -- 172.21.15.12:0/3487400203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc0fff70 msgr2=0x7f8fdc1962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fe0952700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc0fff70 0x7f8fdc1962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fe0952700 1 -- 172.21.15.12:0/3487400203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fdc19ab40 msgr2=0x7f8fdc19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fe0952700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fdc19ab40 0x7f8fdc19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fe0952700 1 -- 172.21.15.12:0/3487400203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc0095f0 con 0x7f8fdc105090 2024-05-08T23:53:36.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fe0952700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fdc105090 0x7f8fdc1967f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc000c00 tx=0x7f8fcc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:36.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fd27fc700 1 -- 172.21.15.12:0/3487400203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc01d070 con 0x7f8fdc105090 2024-05-08T23:53:36.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fd27fc700 1 -- 172.21.15.12:0/3487400203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fcc00fb30 con 0x7f8fdc105090 2024-05-08T23:53:36.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fe33b7700 1 -- 172.21.15.12:0/3487400203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc1a2df0 con 0x7f8fdc105090 2024-05-08T23:53:36.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fd27fc700 1 -- 172.21.15.12:0/3487400203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc022bb0 con 0x7f8fdc105090 2024-05-08T23:53:36.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.676+0000 7f8fe33b7700 1 -- 172.21.15.12:0/3487400203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc1a31c0 con 0x7f8fdc105090 2024-05-08T23:53:36.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fef571b80 1 Processor -- start 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fef571b80 1 -- start start 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab3910 0x55ad01aad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01aadf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fef571b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad01a02150 con 0x55ad01ab2ff0 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fef571b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad019ffb10 con 0x55ad01ab3910 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fef571b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad019fb400 con 0x55ad01aadaa0 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fe1954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fe1954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43878/0 (socket says 172.21.15.12:43878) 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fe1954700 1 -- 172.21.15.12:0/2953110836 learned_addr learned my addr 172.21.15.12:0/2953110836 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.680+0000 7f8fe1153700 1 --2- 172.21.15.12:0/2953110836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab3910 0x55ad01aad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.681+0000 7f8fe1954700 1 -- 172.21.15.12:0/2953110836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 msgr2=0x55ad01aadf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.681+0000 7f8fe1954700 1 --2- 172.21.15.12:0/2953110836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01aadf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.681+0000 7f8fe1954700 1 -- 172.21.15.12:0/2953110836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab3910 msgr2=0x55ad01aad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.681+0000 7f8fe1954700 1 --2- 172.21.15.12:0/2953110836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab3910 0x55ad01aad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.681+0000 7f8fe1954700 1 -- 172.21.15.12:0/2953110836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad0187c3b0 con 0x55ad01ab2ff0 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.681+0000 7f8fe1954700 1 --2- 172.21.15.12:0/2953110836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab33d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009d20 tx=0x7f8fd40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704eda87d0a07ec2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.681+0000 7f8fc77fe700 1 -- 172.21.15.12:0/2953110836 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4018070 con 0x55ad01ab2ff0 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.681+0000 7f8fc77fe700 1 -- 172.21.15.12:0/2953110836 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd400f050 con 0x55ad01ab2ff0 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.681+0000 7f8fc77fe700 1 -- 172.21.15.12:0/2953110836 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4014410 con 0x55ad01ab2ff0 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.681+0000 7f8fd27fc700 1 -- 172.21.15.12:0/3487400203 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8fcc00fca0 con 0x7f8fdc105090 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fef571b80 1 -- 172.21.15.12:0/2953110836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab2ff0 msgr2=0x55ad01ab33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fef571b80 1 --2- 172.21.15.12:0/2953110836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab33d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009d20 tx=0x7f8fd40077e0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fd27fc700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8fc80a1280 0x7f8fc80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fef571b80 1 -- 172.21.15.12:0/2953110836 shutdown_connections 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fef571b80 1 --2- 172.21.15.12:0/2953110836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01aadf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fef571b80 1 --2- 172.21.15.12:0/2953110836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab33d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fef571b80 1 --2- 172.21.15.12:0/2953110836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab3910 0x55ad01aad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fef571b80 1 -- 172.21.15.12:0/2953110836 >> 172.21.15.12:0/2953110836 conn(0x55ad01a09210 msgr2=0x55ad01aa5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fe1954700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8fc80a1280 0x7f8fc80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fef571b80 1 -- 172.21.15.12:0/2953110836 shutdown_connections 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fd27fc700 1 -- 172.21.15.12:0/3487400203 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8fcc0ce2f0 con 0x7f8fdc105090 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.682+0000 7f8fef571b80 1 -- 172.21.15.12:0/2953110836 wait complete. 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fef571b80 1 -- start start 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fe1954700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8fc80a1280 0x7f8fc80a3740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009680 tx=0x7f8fd40055f0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:36.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fef571b80 1 Processor -- start 2024-05-08T23:53:36.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01ab6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fef571b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad01997560 con 0x55ad01ab3910 2024-05-08T23:53:36.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fef571b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad01997200 con 0x55ad01ab2ff0 2024-05-08T23:53:36.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fef571b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0187e6a0 con 0x55ad01aadaa0 2024-05-08T23:53:36.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fef571b80 10 client.0 _ll_register_callbacks cb 0x55ad019a0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fe1954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fe0952700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fe0952700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47360/0 (socket says 172.21.15.12:47360) 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fe0952700 1 -- 172.21.15.12:0/2533584024 learned_addr learned my addr 172.21.15.12:0/2533584024 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.683+0000 7f8fe1153700 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01ab6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fe1954700 1 -- 172.21.15.12:0/2533584024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 msgr2=0x55ad01ab6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fe1954700 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01ab6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fe1954700 1 -- 172.21.15.12:0/2533584024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 msgr2=0x55ad01ab71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fe1954700 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fe1954700 1 -- 172.21.15.12:0/2533584024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd4009980 con 0x55ad01ab3910 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fe0952700 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:36.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fe1954700 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab76e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4007b30 tx=0x7f8fd40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e551d9fb2c58c25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/2533584024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4018070 con 0x55ad01ab3910 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/2533584024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8fd4006020 con 0x55ad01ab3910 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fef571b80 1 -- 172.21.15.12:0/2533584024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad01954270 con 0x55ad01ab3910 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/2533584024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd401eab0 con 0x55ad01ab3910 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fef571b80 10 client.17400 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.684+0000 7f8fef571b80 1 -- 172.21.15.12:0/2533584024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ad01828b60 con 0x55ad01ab3910 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/2533584024 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad01954270 con 0x55ad01ab3910 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/2533584024 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8fd4029400 con 0x55ad01ab3910 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fef571b80 10 client.17400 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fef571b80 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fef571b80 20 client.17400 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:36.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fef571b80 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad01ac72c0 0x55ad01ac96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fef571b80 4 client.17400 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fef571b80 1 -- 172.21.15.12:0/2533584024 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad01a7a810 con 0x55ad01ac72c0 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fe1153700 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad01ac72c0 0x55ad01ac96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fc4ff9700 10 client.17400 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.685+0000 7f8fe1153700 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad01ac72c0 0x55ad01ac96e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53b7f247545c8da server_cookie=eefefb10103189e9 in_seq=0 out_seq=0 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/2533584024 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad01a7a810 con 0x55ad01ac72c0 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fc4ff9700 10 client.17400 handle_command_reply: tid=0 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fef571b80 1 client.17400 shutdown 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fef571b80 1 -- 172.21.15.12:0/2533584024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 msgr2=0x55ad01ab76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fef571b80 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab76e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4007b30 tx=0x7f8fd40077e0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fef571b80 1 -- 172.21.15.12:0/2533584024 shutdown_connections 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fef571b80 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab76e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fef571b80 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01ab6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fef571b80 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad01ac72c0 0x55ad01ac96e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fef571b80 1 --2- 172.21.15.12:0/2533584024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fc4ff9700 0 client.17400 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:36.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fef571b80 1 -- 172.21.15.12:0/2533584024 >> 172.21.15.12:0/2533584024 conn(0x55ad01a09210 msgr2=0x55ad01aa55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:36.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.686+0000 7f8fef571b80 1 -- 172.21.15.12:0/2533584024 shutdown_connections 2024-05-08T23:53:36.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.687+0000 7f8fef571b80 1 -- 172.21.15.12:0/2533584024 wait complete. 2024-05-08T23:53:36.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.687+0000 7f8fef571b80 20 client.17400 trim_cache size 0 max 16384 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fef571b80 1 Processor -- start 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fef571b80 1 -- start start 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01aa5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fef571b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad01ab8260 con 0x55ad01ab3910 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fef571b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad01aa6e80 con 0x55ad01ab2ff0 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fef571b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad01aa7640 con 0x55ad01aadaa0 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fe1954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fe1954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43900/0 (socket says 172.21.15.12:43900) 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fe1954700 1 -- 172.21.15.12:0/3164256043 learned_addr learned my addr 172.21.15.12:0/3164256043 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:36.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fe0952700 1 --2- 172.21.15.12:0/3164256043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fe1153700 1 --2- 172.21.15.12:0/3164256043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01aa5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fe1954700 1 -- 172.21.15.12:0/3164256043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 msgr2=0x55ad01aa5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.705+0000 7f8fe1954700 1 --2- 172.21.15.12:0/3164256043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01aa5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.706+0000 7f8fe1954700 1 -- 172.21.15.12:0/3164256043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 msgr2=0x55ad01ab5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.706+0000 7f8fe1954700 1 --2- 172.21.15.12:0/3164256043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.706+0000 7f8fe1954700 1 -- 172.21.15.12:0/3164256043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad019c1560 con 0x55ad01ab3910 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.706+0000 7f8fe1954700 1 --2- 172.21.15.12:0/3164256043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab5a70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4005c30 tx=0x7f8fd4029930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b833142b6461d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.706+0000 7f8fc7fff700 1 -- 172.21.15.12:0/3164256043 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4029a80 con 0x55ad01ab3910 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.706+0000 7f8fc7fff700 1 -- 172.21.15.12:0/3164256043 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd4019040 con 0x55ad01ab3910 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.706+0000 7f8fc7fff700 1 -- 172.21.15.12:0/3164256043 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40327c0 con 0x55ad01ab3910 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.706+0000 7f8fef571b80 1 -- 172.21.15.12:0/3164256043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 msgr2=0x55ad01ab5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.706+0000 7f8fef571b80 1 --2- 172.21.15.12:0/3164256043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab5a70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4005c30 tx=0x7f8fd4029930 comp rx=0 tx=0).stop 2024-05-08T23:53:36.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 -- 172.21.15.12:0/3164256043 shutdown_connections 2024-05-08T23:53:36.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 --2- 172.21.15.12:0/3164256043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01ab3910 0x55ad01ab5a70 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 --2- 172.21.15.12:0/3164256043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01aadaa0 0x55ad01aa5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 --2- 172.21.15.12:0/3164256043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab2ff0 0x55ad01ab5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 -- 172.21.15.12:0/3164256043 >> 172.21.15.12:0/3164256043 conn(0x55ad01a09210 msgr2=0x55ad01aa2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:36.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 -- 172.21.15.12:0/3164256043 shutdown_connections 2024-05-08T23:53:36.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 -- 172.21.15.12:0/3164256043 wait complete. 2024-05-08T23:53:36.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 -- start start 2024-05-08T23:53:36.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 Processor -- start 2024-05-08T23:53:36.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01aadaa0 0x55ad01abc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01ab2ff0 0x55ad01abc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab3910 0x55ad01abcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad01abec70 con 0x55ad01aadaa0 2024-05-08T23:53:36.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad01b1c7c0 con 0x55ad01ab3910 2024-05-08T23:53:36.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.707+0000 7f8fef571b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad01b1c930 con 0x55ad01ab2ff0 2024-05-08T23:53:36.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fef571b80 10 client.0 _ll_register_callbacks cb 0x55ad019a0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:36.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe1153700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01aadaa0 0x55ad01abc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe1153700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01aadaa0 0x55ad01abc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43916/0 (socket says 172.21.15.12:43916) 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe1153700 1 -- 172.21.15.12:0/1838621009 learned_addr learned my addr 172.21.15.12:0/1838621009 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe1954700 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab3910 0x55ad01abcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe0952700 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01ab2ff0 0x55ad01abc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe1153700 1 -- 172.21.15.12:0/1838621009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01ab2ff0 msgr2=0x55ad01abc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe1153700 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01ab2ff0 0x55ad01abc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe1153700 1 -- 172.21.15.12:0/1838621009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab3910 msgr2=0x55ad01abcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe1153700 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab3910 0x55ad01abcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe1153700 1 -- 172.21.15.12:0/1838621009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad01828b60 con 0x55ad01aadaa0 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fe1153700 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01aadaa0 0x55ad01abc340 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x55ad01ac9ff0 tx=0x7f8fd800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8505ed6e801763 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/1838621009 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8016040 con 0x55ad01aadaa0 2024-05-08T23:53:36.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/1838621009 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8fd8003e70 con 0x55ad01aadaa0 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fef571b80 1 -- 172.21.15.12:0/1838621009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fd400f050 con 0x55ad01aadaa0 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/1838621009 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd8004830 con 0x55ad01aadaa0 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fef571b80 10 client.17406 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.708+0000 7f8fef571b80 1 -- 172.21.15.12:0/1838621009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ad01880cd0 con 0x55ad01aadaa0 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.709+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/1838621009 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd400f050 con 0x55ad01aadaa0 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.709+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/1838621009 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8fd8011810 con 0x55ad01aadaa0 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.709+0000 7f8fef571b80 10 client.17406 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.709+0000 7f8fef571b80 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.709+0000 7f8fef571b80 20 client.17406 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.709+0000 7f8fef571b80 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad01b1e1b0 0x55ad01b205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.709+0000 7f8fef571b80 4 client.17406 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.709+0000 7f8fef571b80 1 -- 172.21.15.12:0/1838621009 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad019f3ec0 con 0x55ad01b1e1b0 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.709+0000 7f8fe0952700 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad01b1e1b0 0x55ad01b205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fc4ff9700 10 client.17406 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fe0952700 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad01b1e1b0 0x55ad01b205d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1acf99e7fbc86b0 server_cookie=a7c9f6ccd9aeb5e9 in_seq=0 out_seq=0 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fc4ff9700 1 -- 172.21.15.12:0/1838621009 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ad019f3ec0 con 0x55ad01b1e1b0 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fc4ff9700 10 client.17406 handle_command_reply: tid=0 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 client.17406 shutdown 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 -- 172.21.15.12:0/1838621009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01aadaa0 msgr2=0x55ad01abc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01aadaa0 0x55ad01abc340 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x55ad01ac9ff0 tx=0x7f8fd800bdb0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 -- 172.21.15.12:0/1838621009 shutdown_connections 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad01b1e1b0 0x55ad01b205d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad01ab3910 0x55ad01abcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fc4ff9700 0 client.17406 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad01aadaa0 0x55ad01abc340 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 --2- 172.21.15.12:0/1838621009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad01ab2ff0 0x55ad01abc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 -- 172.21.15.12:0/1838621009 >> 172.21.15.12:0/1838621009 conn(0x55ad01a09210 msgr2=0x55ad01ab54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:36.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 -- 172.21.15.12:0/1838621009 shutdown_connections 2024-05-08T23:53:36.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 1 -- 172.21.15.12:0/1838621009 wait complete. 2024-05-08T23:53:36.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.710+0000 7f8fef571b80 20 client.17406 trim_cache size 0 max 16384 2024-05-08T23:53:36.712 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:53:36.712 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:53:36.712 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:53:36.712 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:53:36.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 -- 172.21.15.12:0/3487400203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8fc80a1280 msgr2=0x7f8fc80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8fc80a1280 0x7f8fc80a3740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009680 tx=0x7f8fd40055f0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 -- 172.21.15.12:0/3487400203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fdc105090 msgr2=0x7f8fdc1967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fdc105090 0x7f8fdc1967f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc000c00 tx=0x7f8fcc0036f0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 -- 172.21.15.12:0/3487400203 shutdown_connections 2024-05-08T23:53:36.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8fc80a1280 0x7f8fc80a3740 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fdc19ab40 0x7f8fdc19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc0fff70 0x7f8fdc1962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 --2- 172.21.15.12:0/3487400203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fdc105090 0x7f8fdc1967f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 -- 172.21.15.12:0/3487400203 >> 172.21.15.12:0/3487400203 conn(0x7f8fdc0fb7b0 msgr2=0x7f8fdc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:36.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 -- 172.21.15.12:0/3487400203 shutdown_connections 2024-05-08T23:53:36.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.711+0000 7f8fe33b7700 1 -- 172.21.15.12:0/3487400203 wait complete. 2024-05-08T23:53:36.719 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:53:36.719 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:53:36.746 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:36 smithi012 ceph-mon[42503]: pgmap v224: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-05-08T23:53:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:36 smithi204 ceph-mon[47373]: pgmap v224: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-05-08T23:53:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:36 smithi160 ceph-mon[46152]: pgmap v224: 97 pgs: 97 active+clean; 128 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-05-08T23:53:36.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.862+0000 7feec513d700 1 -- 172.21.15.12:0/3360315528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feec0105500 msgr2=0x7feec01058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.862+0000 7feec513d700 1 --2- 172.21.15.12:0/3360315528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feec0105500 0x7feec01058e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7feea800aaa0 tx=0x7feea800adb0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.862+0000 7feec513d700 1 -- 172.21.15.12:0/3360315528 shutdown_connections 2024-05-08T23:53:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.862+0000 7feec513d700 1 --2- 172.21.15.12:0/3360315528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec00ffe20 0x7feec010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.862+0000 7feec513d700 1 --2- 172.21.15.12:0/3360315528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feec0105500 0x7feec01058e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.862+0000 7feec513d700 1 --2- 172.21.15.12:0/3360315528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feec00ff460 0x7feec00ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.862+0000 7feec513d700 1 -- 172.21.15.12:0/3360315528 >> 172.21.15.12:0/3360315528 conn(0x7feec0076270 msgr2=0x7feec00fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.862+0000 7feec513d700 1 -- 172.21.15.12:0/3360315528 shutdown_connections 2024-05-08T23:53:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.862+0000 7feec513d700 1 -- 172.21.15.12:0/3360315528 wait complete. 2024-05-08T23:53:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.862+0000 7feec513d700 1 Processor -- start 2024-05-08T23:53:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feec513d700 1 -- start start 2024-05-08T23:53:36.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feec513d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec00ff460 0x7feec0199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feec513d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feec00ffe20 0x7feec01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feec513d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feec0105500 0x7feec019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feec513d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec019e3b0 con 0x7feec00ffe20 2024-05-08T23:53:36.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feec513d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec019e520 con 0x7feec0105500 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feec513d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec019e6e0 con 0x7feec00ff460 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feebe59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feec00ffe20 0x7feec01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feebe59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feec00ffe20 0x7feec01999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43924/0 (socket says 172.21.15.12:43924) 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feebe59c700 1 -- 172.21.15.12:0/261408994 learned_addr learned my addr 172.21.15.12:0/261408994 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feebf59e700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feec0105500 0x7feec019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feebed9d700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec00ff460 0x7feec0199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feebe59c700 1 -- 172.21.15.12:0/261408994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec00ff460 msgr2=0x7feec0199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feebe59c700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec00ff460 0x7feec0199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feebe59c700 1 -- 172.21.15.12:0/261408994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feec0105500 msgr2=0x7feec019dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.863+0000 7feebe59c700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feec0105500 0x7feec019dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.864+0000 7feebe59c700 1 -- 172.21.15.12:0/261408994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea800a700 con 0x7feec00ffe20 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.864+0000 7feebf59e700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feec0105500 0x7feec019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.864+0000 7feebe59c700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feec00ffe20 0x7feec01999d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7feeb000e9a0 tx=0x7feeb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.864+0000 7feeb7fff700 1 -- 172.21.15.12:0/261408994 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb0009790 con 0x7feec00ffe20 2024-05-08T23:53:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.864+0000 7feec513d700 1 -- 172.21.15.12:0/261408994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec019e9c0 con 0x7feec00ffe20 2024-05-08T23:53:36.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.864+0000 7feeb7fff700 1 -- 172.21.15.12:0/261408994 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feeb000cbd0 con 0x7feec00ffe20 2024-05-08T23:53:36.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.864+0000 7feeb7fff700 1 -- 172.21.15.12:0/261408994 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb0018680 con 0x7feec00ffe20 2024-05-08T23:53:36.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.864+0000 7feec513d700 1 -- 172.21.15.12:0/261408994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec01a6200 con 0x7feec00ffe20 2024-05-08T23:53:36.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.865+0000 7feec513d700 1 -- 172.21.15.12:0/261408994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec0100ba0 con 0x7feec00ffe20 2024-05-08T23:53:36.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.866+0000 7feeb7fff700 1 -- 172.21.15.12:0/261408994 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7feeb000cd40 con 0x7feec00ffe20 2024-05-08T23:53:36.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.867+0000 7feeb7fff700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feeac0a1290 0x7feeac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:36.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.867+0000 7feebed9d700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feeac0a1290 0x7feeac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:36.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.867+0000 7feeb7fff700 1 -- 172.21.15.12:0/261408994 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7feeb0014070 con 0x7feec00ffe20 2024-05-08T23:53:36.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.868+0000 7feebed9d700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feeac0a1290 0x7feeac0a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feea800c6d0 tx=0x7feea801b040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:36.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:36.871+0000 7feeb7fff700 1 -- 172.21.15.12:0/261408994 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feeb00966c0 con 0x7feec00ffe20 2024-05-08T23:53:36.870 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465075 9.45 MB/sec execute 212 sec latency 2626.804 ms 2024-05-08T23:53:37.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.139+0000 7feec513d700 1 -- 172.21.15.12:0/261408994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feec0067380 con 0x7feec00ffe20 2024-05-08T23:53:37.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.140+0000 7feeb7fff700 1 -- 172.21.15.12:0/261408994 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10878 (secure 0 0 0) 0x7feeb0099ce0 con 0x7feec00ffe20 2024-05-08T23:53:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:53:37.135 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:53:28.485482+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27175":{"gid":27175,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2842389121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2842389121},{"type":"v1","addr":"172.21.15.204:6837","nonce":2842389121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:53:37.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 -- 172.21.15.12:0/261408994 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feeac0a1290 msgr2=0x7feeac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feeac0a1290 0x7feeac0a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feea800c6d0 tx=0x7feea801b040 comp rx=0 tx=0).stop 2024-05-08T23:53:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 -- 172.21.15.12:0/261408994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feec00ffe20 msgr2=0x7feec01999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feec00ffe20 0x7feec01999d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7feeb000e9a0 tx=0x7feeb000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 -- 172.21.15.12:0/261408994 shutdown_connections 2024-05-08T23:53:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feeac0a1290 0x7feeac0a3750 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feec0105500 0x7feec019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec00ff460 0x7feec0199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 --2- 172.21.15.12:0/261408994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feec00ffe20 0x7feec01999d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 -- 172.21.15.12:0/261408994 >> 172.21.15.12:0/261408994 conn(0x7feec0076270 msgr2=0x7feec010b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 -- 172.21.15.12:0/261408994 shutdown_connections 2024-05-08T23:53:37.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.142+0000 7feec513d700 1 -- 172.21.15.12:0/261408994 wait complete. 2024-05-08T23:53:37.139 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 36 2024-05-08T23:53:37.151 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:53:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.337+0000 7f31fd9ca700 1 -- 172.21.15.12:0/1581550181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f81011e0 msgr2=0x7f31f81015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.337+0000 7f31fd9ca700 1 --2- 172.21.15.12:0/1581550181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f81011e0 0x7f31f81015c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009990 tx=0x7f31e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.337+0000 7f31fd9ca700 1 -- 172.21.15.12:0/1581550181 shutdown_connections 2024-05-08T23:53:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.337+0000 7f31fd9ca700 1 --2- 172.21.15.12:0/1581550181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f8106300 0x7f31f810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.337+0000 7f31fd9ca700 1 --2- 172.21.15.12:0/1581550181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f81011e0 0x7f31f81015c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.337+0000 7f31fd9ca700 1 --2- 172.21.15.12:0/1581550181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31f8101b90 0x7f31f8105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.337+0000 7f31fd9ca700 1 -- 172.21.15.12:0/1581550181 >> 172.21.15.12:0/1581550181 conn(0x7f31f80fca80 msgr2=0x7f31f80feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.337+0000 7f31fd9ca700 1 -- 172.21.15.12:0/1581550181 shutdown_connections 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.337+0000 7f31fd9ca700 1 -- 172.21.15.12:0/1581550181 wait complete. 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31fd9ca700 1 Processor -- start 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31fd9ca700 1 -- start start 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31fd9ca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31f81011e0 0x7f31f81975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31fd9ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f8106300 0x7f31f8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31fd9ca700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f819bed0 0x7f31f819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31fd9ca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f819c950 con 0x7f31f8106300 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31fd9ca700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f819cac0 con 0x7f31f81011e0 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31fd9ca700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f819cc80 con 0x7f31f819bed0 2024-05-08T23:53:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31f67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f8106300 0x7f31f8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31f6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31f81011e0 0x7f31f81975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31f67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f8106300 0x7f31f8197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43946/0 (socket says 172.21.15.12:43946) 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.338+0000 7f31f67fc700 1 -- 172.21.15.12:0/3277199416 learned_addr learned my addr 172.21.15.12:0/3277199416 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31f77fe700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f819bed0 0x7f31f819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31f67fc700 1 -- 172.21.15.12:0/3277199416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f819bed0 msgr2=0x7f31f819c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31f67fc700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f819bed0 0x7f31f819c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31f67fc700 1 -- 172.21.15.12:0/3277199416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31f81011e0 msgr2=0x7f31f81975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31f67fc700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31f81011e0 0x7f31f81975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31f67fc700 1 -- 172.21.15.12:0/3277199416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e0009520 con 0x7f31f8106300 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31f67fc700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f8106300 0x7f31f8197b30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f31e000ea30 tx=0x7f31e000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31fc9c8700 1 -- 172.21.15.12:0/3277199416 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0009820 con 0x7f31f8106300 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31fd9ca700 1 -- 172.21.15.12:0/3277199416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f0009c40 con 0x7f31f8106300 2024-05-08T23:53:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31fc9c8700 1 -- 172.21.15.12:0/3277199416 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31e00044e0 con 0x7f31f8106300 2024-05-08T23:53:37.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31fc9c8700 1 -- 172.21.15.12:0/3277199416 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0003dc0 con 0x7f31f8106300 2024-05-08T23:53:37.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.339+0000 7f31fd9ca700 1 -- 172.21.15.12:0/3277199416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e80095f0 con 0x7f31f8106300 2024-05-08T23:53:37.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.342+0000 7f31fc9c8700 1 -- 172.21.15.12:0/3277199416 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f31e0003f80 con 0x7f31f8106300 2024-05-08T23:53:37.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.342+0000 7f31fc9c8700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31e40a1290 0x7f31e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.343+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31e40a1290 0x7f31e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.343+0000 7f31fc9c8700 1 -- 172.21.15.12:0/3277199416 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f31e0014070 con 0x7f31f8106300 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.343+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31e40a1290 0x7f31e40a3750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31e8006010 tx=0x7f31e800b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.343+0000 7f3209b84b80 1 Processor -- start 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f3209b84b80 1 -- start start 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba600e00 0x55ffba6011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 0x55ffba5fb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f3209b84b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba552150 con 0x55ffba5fb890 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f3209b84b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba54fb10 con 0x55ffba601720 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f3209b84b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba54b400 con 0x55ffba600e00 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43950/0 (socket says 172.21.15.12:43950) 2024-05-08T23:53:37.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f77fe700 1 -- 172.21.15.12:0/2826856738 learned_addr learned my addr 172.21.15.12:0/2826856738 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 0x55ffba5fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/2826856738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba600e00 0x55ffba6011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f77fe700 1 -- 172.21.15.12:0/2826856738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba600e00 msgr2=0x55ffba6011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f77fe700 1 --2- 172.21.15.12:0/2826856738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba600e00 0x55ffba6011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f77fe700 1 -- 172.21.15.12:0/2826856738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 msgr2=0x55ffba5fb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f77fe700 1 --2- 172.21.15.12:0/2826856738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 0x55ffba5fb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f77fe700 1 -- 172.21.15.12:0/2826856738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffba3cc3b0 con 0x55ffba5fb890 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f67fc700 1 --2- 172.21.15.12:0/2826856738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 0x55ffba5fb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.344+0000 7f31f77fe700 1 --2- 172.21.15.12:0/2826856738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fbcd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f31f0007600 tx=0x7f31f0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57448f4acf262ccb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f31edffb700 1 -- 172.21.15.12:0/2826856738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f000e070 con 0x55ffba5fb890 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f31edffb700 1 -- 172.21.15.12:0/2826856738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31f000f040 con 0x55ffba5fb890 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f31edffb700 1 -- 172.21.15.12:0/2826856738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0015b90 con 0x55ffba5fb890 2024-05-08T23:53:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f3209b84b80 1 -- 172.21.15.12:0/2826856738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 msgr2=0x55ffba5fbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f3209b84b80 1 --2- 172.21.15.12:0/2826856738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fbcd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f31f0007600 tx=0x7f31f0009920 comp rx=0 tx=0).stop 2024-05-08T23:53:37.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f3209b84b80 1 -- 172.21.15.12:0/2826856738 shutdown_connections 2024-05-08T23:53:37.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f3209b84b80 1 --2- 172.21.15.12:0/2826856738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fbcd0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f3209b84b80 1 --2- 172.21.15.12:0/2826856738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba600e00 0x55ffba6011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f3209b84b80 1 --2- 172.21.15.12:0/2826856738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 0x55ffba5fb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f3209b84b80 1 -- 172.21.15.12:0/2826856738 >> 172.21.15.12:0/2826856738 conn(0x55ffba559210 msgr2=0x55ffba5f5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f3209b84b80 1 -- 172.21.15.12:0/2826856738 shutdown_connections 2024-05-08T23:53:37.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.345+0000 7f3209b84b80 1 -- 172.21.15.12:0/2826856738 wait complete. 2024-05-08T23:53:37.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.346+0000 7f3209b84b80 1 -- start start 2024-05-08T23:53:37.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.346+0000 7f3209b84b80 1 Processor -- start 2024-05-08T23:53:37.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba5fb890 0x55ffba604ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba600e00 0x55ffba604ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 0x55ffba605530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f3209b84b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba4e7560 con 0x55ffba600e00 2024-05-08T23:53:37.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f3209b84b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba4e7200 con 0x55ffba601720 2024-05-08T23:53:37.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f3209b84b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba3ce6a0 con 0x55ffba5fb890 2024-05-08T23:53:37.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f3209b84b80 10 client.0 _ll_register_callbacks cb 0x55ffba4f0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:37.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba600e00 0x55ffba604ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba600e00 0x55ffba604ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43954/0 (socket says 172.21.15.12:43954) 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f67fc700 1 -- 172.21.15.12:0/2107993137 learned_addr learned my addr 172.21.15.12:0/2107993137 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba5fb890 0x55ffba604ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f77fe700 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 0x55ffba605530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f67fc700 1 -- 172.21.15.12:0/2107993137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba5fb890 msgr2=0x55ffba604ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f67fc700 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba5fb890 0x55ffba604ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f67fc700 1 -- 172.21.15.12:0/2107993137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 msgr2=0x55ffba605530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f67fc700 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 0x55ffba605530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f67fc700 1 -- 172.21.15.12:0/2107993137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e80095f0 con 0x55ffba600e00 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31f67fc700 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba600e00 0x55ffba604ff0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f31e0096690 tx=0x7f31e009a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9d00cadccdf0ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31d6ffd700 1 -- 172.21.15.12:0/2107993137 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e00a2690 con 0x55ffba600e00 2024-05-08T23:53:37.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f31d6ffd700 1 -- 172.21.15.12:0/2107993137 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f31e0004230 con 0x55ffba600e00 2024-05-08T23:53:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.347+0000 7f3209b84b80 1 -- 172.21.15.12:0/2107993137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffba4a4270 con 0x55ffba600e00 2024-05-08T23:53:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f31d6ffd700 1 -- 172.21.15.12:0/2107993137 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e009a3f0 con 0x55ffba600e00 2024-05-08T23:53:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f3209b84b80 10 client.17424 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f3209b84b80 1 -- 172.21.15.12:0/2107993137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f31f00092e0 con 0x55ffba600e00 2024-05-08T23:53:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f31d6ffd700 1 -- 172.21.15.12:0/2107993137 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffba4a4270 con 0x55ffba600e00 2024-05-08T23:53:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f31d6ffd700 1 -- 172.21.15.12:0/2107993137 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31e00a2180 con 0x55ffba600e00 2024-05-08T23:53:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f3209b84b80 10 client.17424 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f3209b84b80 10 client.17424 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:53:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f3209b84b80 20 client.17424 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:37.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f3209b84b80 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ffba615200 0x55ffba617620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f3209b84b80 4 client.17424 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f3209b84b80 1 -- 172.21.15.12:0/2107993137 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffba5ca810 con 0x55ffba615200 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.348+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ffba615200 0x55ffba617620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.349+0000 7f31d6ffd700 10 client.17424 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.349+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ffba615200 0x55ffba617620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49d02895daf1cfac server_cookie=223df15798683037 in_seq=0 out_seq=0 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f31d6ffd700 1 -- 172.21.15.12:0/2107993137 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffba5ca810 con 0x55ffba615200 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f31d6ffd700 10 client.17424 handle_command_reply: tid=0 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 client.17424 shutdown 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 -- 172.21.15.12:0/2107993137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba600e00 msgr2=0x55ffba604ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba600e00 0x55ffba604ff0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f31e0096690 tx=0x7f31e009a2c0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 -- 172.21.15.12:0/2107993137 shutdown_connections 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba601720 0x55ffba605530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ffba615200 0x55ffba617620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba5fb890 0x55ffba604ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 --2- 172.21.15.12:0/2107993137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba600e00 0x55ffba604ff0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 -- 172.21.15.12:0/2107993137 >> 172.21.15.12:0/2107993137 conn(0x55ffba559210 msgr2=0x55ffba5f55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f31d6ffd700 0 client.17424 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:53:37.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 -- 172.21.15.12:0/2107993137 shutdown_connections 2024-05-08T23:53:37.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 1 -- 172.21.15.12:0/2107993137 wait complete. 2024-05-08T23:53:37.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.350+0000 7f3209b84b80 20 client.17424 trim_cache size 0 max 16384 2024-05-08T23:53:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.364+0000 7f3209b84b80 1 Processor -- start 2024-05-08T23:53:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.364+0000 7f3209b84b80 1 -- start start 2024-05-08T23:53:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.364+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.364+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 0x55ffba6175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.364+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 0x55ffba5fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.364+0000 7f3209b84b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba611cd0 con 0x55ffba5fb890 2024-05-08T23:53:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.364+0000 7f3209b84b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba606120 con 0x55ffba600e00 2024-05-08T23:53:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.364+0000 7f3209b84b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba6128b0 con 0x55ffba601720 2024-05-08T23:53:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.364+0000 7f31f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43970/0 (socket says 172.21.15.12:43970) 2024-05-08T23:53:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f6ffd700 1 -- 172.21.15.12:0/1511622535 learned_addr learned my addr 172.21.15.12:0/1511622535 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f67fc700 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 0x55ffba6175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f77fe700 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 0x55ffba5fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f6ffd700 1 -- 172.21.15.12:0/1511622535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 msgr2=0x55ffba5fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 0x55ffba5fbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f6ffd700 1 -- 172.21.15.12:0/1511622535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 msgr2=0x55ffba6175f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 0x55ffba6175f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f6ffd700 1 -- 172.21.15.12:0/1511622535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffba378b60 con 0x55ffba5fb890 2024-05-08T23:53:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f77fe700 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 0x55ffba5fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:53:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f67fc700 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 0x55ffba6175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fb350 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009990 tx=0x7f31e8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50eaa38a75847f22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31ee7fc700 1 -- 172.21.15.12:0/1511622535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e800be50 con 0x55ffba5fb890 2024-05-08T23:53:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31ee7fc700 1 -- 172.21.15.12:0/1511622535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31e801e040 con 0x55ffba5fb890 2024-05-08T23:53:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.365+0000 7f31ee7fc700 1 -- 172.21.15.12:0/1511622535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80049c0 con 0x55ffba5fb890 2024-05-08T23:53:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.366+0000 7f3209b84b80 1 -- 172.21.15.12:0/1511622535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 msgr2=0x55ffba5fb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.366+0000 7f3209b84b80 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fb350 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009990 tx=0x7f31e8020000 comp rx=0 tx=0).stop 2024-05-08T23:53:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.366+0000 7f3209b84b80 1 -- 172.21.15.12:0/1511622535 shutdown_connections 2024-05-08T23:53:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.366+0000 7f3209b84b80 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 0x55ffba5fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.366+0000 7f3209b84b80 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba5fb350 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.366+0000 7f3209b84b80 1 --2- 172.21.15.12:0/1511622535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 0x55ffba6175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.366+0000 7f3209b84b80 1 -- 172.21.15.12:0/1511622535 >> 172.21.15.12:0/1511622535 conn(0x55ffba559210 msgr2=0x55ffba5f2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.366+0000 7f3209b84b80 1 -- 172.21.15.12:0/1511622535 shutdown_connections 2024-05-08T23:53:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.366+0000 7f3209b84b80 1 -- 172.21.15.12:0/1511622535 wait complete. 2024-05-08T23:53:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.366+0000 7f3209b84b80 1 -- start start 2024-05-08T23:53:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f3209b84b80 1 Processor -- start 2024-05-08T23:53:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba60b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 0x55ffba60b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f3209b84b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 0x55ffba60bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f3209b84b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba66a7c0 con 0x55ffba5fb890 2024-05-08T23:53:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f3209b84b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba66a930 con 0x55ffba600e00 2024-05-08T23:53:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f3209b84b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffba66aaf0 con 0x55ffba601720 2024-05-08T23:53:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f3209b84b80 10 client.0 _ll_register_callbacks cb 0x55ffba4f0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:37.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba60b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba60b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43976/0 (socket says 172.21.15.12:43976) 2024-05-08T23:53:37.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f6ffd700 1 -- 172.21.15.12:0/85673152 learned_addr learned my addr 172.21.15.12:0/85673152 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f67fc700 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 0x55ffba60b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f77fe700 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 0x55ffba60bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f6ffd700 1 -- 172.21.15.12:0/85673152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 msgr2=0x55ffba60bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 0x55ffba60bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f6ffd700 1 -- 172.21.15.12:0/85673152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 msgr2=0x55ffba60b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 0x55ffba60b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f6ffd700 1 -- 172.21.15.12:0/85673152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f810e6f0 con 0x55ffba5fb890 2024-05-08T23:53:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f77fe700 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 0x55ffba60bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31f6ffd700 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba60b4b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005920 tx=0x7f31e800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9813d42c1d9203 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.367+0000 7f31d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f31d6ffd700 1 -- 172.21.15.12:0/85673152 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80367a0 con 0x55ffba5fb890 2024-05-08T23:53:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f3209b84b80 1 -- 172.21.15.12:0/85673152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31f000f040 con 0x55ffba5fb890 2024-05-08T23:53:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f31d6ffd700 1 -- 172.21.15.12:0/85673152 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f31e8004640 con 0x55ffba5fb890 2024-05-08T23:53:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f31d6ffd700 1 -- 172.21.15.12:0/85673152 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e8036900 con 0x55ffba5fb890 2024-05-08T23:53:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f3209b84b80 10 client.17430 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f3209b84b80 1 -- 172.21.15.12:0/85673152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ffba511560 con 0x55ffba5fb890 2024-05-08T23:53:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f31d6ffd700 1 -- 172.21.15.12:0/85673152 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31f000f040 con 0x55ffba5fb890 2024-05-08T23:53:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f31d6ffd700 1 -- 172.21.15.12:0/85673152 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31e800f840 con 0x55ffba5fb890 2024-05-08T23:53:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f3209b84b80 10 client.17430 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f3209b84b80 10 client.17430 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:53:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f3209b84b80 20 client.17430 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f3209b84b80 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ffba66be50 0x55ffba66e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f3209b84b80 4 client.17430 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:53:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.368+0000 7f3209b84b80 1 -- 172.21.15.12:0/85673152 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ffba543ec0 con 0x55ffba66be50 2024-05-08T23:53:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f31f67fc700 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ffba66be50 0x55ffba66e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f31f67fc700 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ffba66be50 0x55ffba66e240 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9ce2481724841d6 server_cookie=74cc21cfb850f62d in_seq=0 out_seq=0 2024-05-08T23:53:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f31d6ffd700 10 client.17430 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:53:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f31d6ffd700 1 -- 172.21.15.12:0/85673152 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ffba543ec0 con 0x55ffba66be50 2024-05-08T23:53:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f31d6ffd700 10 client.17430 handle_command_reply: tid=0 2024-05-08T23:53:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f3209b84b80 1 client.17430 shutdown 2024-05-08T23:53:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f3209b84b80 1 -- 172.21.15.12:0/85673152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 msgr2=0x55ffba60b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f3209b84b80 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba60b4b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005920 tx=0x7f31e800bed0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f3209b84b80 1 -- 172.21.15.12:0/85673152 shutdown_connections 2024-05-08T23:53:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f3209b84b80 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ffba66be50 0x55ffba66e240 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f3209b84b80 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffba601720 0x55ffba60bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f3209b84b80 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffba5fb890 0x55ffba60b4b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f31d6ffd700 0 client.17430 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:53:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f3209b84b80 1 --2- 172.21.15.12:0/85673152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffba600e00 0x55ffba60b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f3209b84b80 1 -- 172.21.15.12:0/85673152 >> 172.21.15.12:0/85673152 conn(0x55ffba559210 msgr2=0x55ffba5fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.369+0000 7f3209b84b80 1 -- 172.21.15.12:0/85673152 shutdown_connections 2024-05-08T23:53:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f3209b84b80 1 -- 172.21.15.12:0/85673152 wait complete. 2024-05-08T23:53:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f3209b84b80 20 client.17430 trim_cache size 0 max 16384 2024-05-08T23:53:37.381 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:53:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 -- 172.21.15.12:0/3277199416 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31e40a1290 msgr2=0x7f31e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31e40a1290 0x7f31e40a3750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31e8006010 tx=0x7f31e800b260 comp rx=0 tx=0).stop 2024-05-08T23:53:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 -- 172.21.15.12:0/3277199416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f8106300 msgr2=0x7f31f8197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f8106300 0x7f31f8197b30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f31e000ea30 tx=0x7f31e000ed40 comp rx=0 tx=0).stop 2024-05-08T23:53:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 -- 172.21.15.12:0/3277199416 shutdown_connections 2024-05-08T23:53:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f31e40a1290 0x7f31e40a3750 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31f819bed0 0x7f31f819c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31f81011e0 0x7f31f81975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 --2- 172.21.15.12:0/3277199416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31f8106300 0x7f31f8197b30 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 -- 172.21.15.12:0/3277199416 >> 172.21.15.12:0/3277199416 conn(0x7f31f80fca80 msgr2=0x7f31f80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 -- 172.21.15.12:0/3277199416 shutdown_connections 2024-05-08T23:53:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.370+0000 7f31fd9ca700 1 -- 172.21.15.12:0/3277199416 wait complete. 2024-05-08T23:53:37.383 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:53:37.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.528+0000 7f7d42a94700 1 -- 172.21.15.12:0/4135316153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d3c101b10 msgr2=0x7f7d3c105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.528+0000 7f7d42a94700 1 --2- 172.21.15.12:0/4135316153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d3c101b10 0x7f7d3c105b50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009990 tx=0x7f7d24009ca0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.528+0000 7f7d42a94700 1 -- 172.21.15.12:0/4135316153 shutdown_connections 2024-05-08T23:53:37.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.528+0000 7f7d42a94700 1 --2- 172.21.15.12:0/4135316153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d3c106280 0x7f7d3c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.528+0000 7f7d42a94700 1 --2- 172.21.15.12:0/4135316153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d3c101160 0x7f7d3c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.528+0000 7f7d42a94700 1 --2- 172.21.15.12:0/4135316153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d3c101b10 0x7f7d3c105b50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 -- 172.21.15.12:0/4135316153 >> 172.21.15.12:0/4135316153 conn(0x7f7d3c0fc9e0 msgr2=0x7f7d3c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 -- 172.21.15.12:0/4135316153 shutdown_connections 2024-05-08T23:53:37.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 -- 172.21.15.12:0/4135316153 wait complete. 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 Processor -- start 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 -- start start 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d3c101160 0x7f7d3c1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d3c106280 0x7f7d3c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d3c19bd30 0x7f7d3c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c19c7b0 con 0x7f7d3c101160 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c19c920 con 0x7f7d3c106280 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.529+0000 7f7d42a94700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c19cae0 con 0x7f7d3c19bd30 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d40830700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d3c101160 0x7f7d3c1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d40830700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d3c101160 0x7f7d3c1974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43996/0 (socket says 172.21.15.12:43996) 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d40830700 1 -- 172.21.15.12:0/2311133004 learned_addr learned my addr 172.21.15.12:0/2311133004 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d3bfff700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d3c106280 0x7f7d3c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d41031700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d3c19bd30 0x7f7d3c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d40830700 1 -- 172.21.15.12:0/2311133004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d3c19bd30 msgr2=0x7f7d3c19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d40830700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d3c19bd30 0x7f7d3c19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d40830700 1 -- 172.21.15.12:0/2311133004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d3c106280 msgr2=0x7f7d3c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d40830700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d3c106280 0x7f7d3c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d40830700 1 -- 172.21.15.12:0/2311133004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d240095f0 con 0x7f7d3c101160 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d40830700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d3c101160 0x7f7d3c1974a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e9a0 tx=0x7f7d2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d39ffb700 1 -- 172.21.15.12:0/2311133004 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c009790 con 0x7f7d3c101160 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d39ffb700 1 -- 172.21.15.12:0/2311133004 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d2c010430 con 0x7f7d3c101160 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d42a94700 1 -- 172.21.15.12:0/2311133004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c1a3fe0 con 0x7f7d3c101160 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.530+0000 7f7d39ffb700 1 -- 172.21.15.12:0/2311133004 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c018680 con 0x7f7d3c101160 2024-05-08T23:53:37.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.531+0000 7f7d42a94700 1 -- 172.21.15.12:0/2311133004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c1a4510 con 0x7f7d3c101160 2024-05-08T23:53:37.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.533+0000 7f7d39ffb700 1 -- 172.21.15.12:0/2311133004 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7d2c0105a0 con 0x7f7d3c101160 2024-05-08T23:53:37.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.533+0000 7f7d39ffb700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d280a1310 0x7f7d280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.534+0000 7f7d39ffb700 1 -- 172.21.15.12:0/2311133004 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7d2c014070 con 0x7f7d3c101160 2024-05-08T23:53:37.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.534+0000 7f7d3bfff700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d280a1310 0x7f7d280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.534+0000 7f7d3bfff700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d280a1310 0x7f7d280a37d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c1989a0 tx=0x7f7d24005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d4ec4eb80 1 Processor -- start 2024-05-08T23:53:37.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d4ec4eb80 1 -- start start 2024-05-08T23:53:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758fed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758fedcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563758f44150 con 0x563758ff3720 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563758f41b10 con 0x563758fed890 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563758f3d400 con 0x563758ff2e00 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758fed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758fed350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44004/0 (socket says 172.21.15.12:44004) 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d3bfff700 1 -- 172.21.15.12:0/607028898 learned_addr learned my addr 172.21.15.12:0/607028898 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d40830700 1 --2- 172.21.15.12:0/607028898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.535+0000 7f7d41031700 1 --2- 172.21.15.12:0/607028898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758fedcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.536+0000 7f7d3bfff700 1 -- 172.21.15.12:0/607028898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 msgr2=0x563758ff31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.536+0000 7f7d3bfff700 1 --2- 172.21.15.12:0/607028898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.536+0000 7f7d3bfff700 1 -- 172.21.15.12:0/607028898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 msgr2=0x563758fedcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.536+0000 7f7d3bfff700 1 --2- 172.21.15.12:0/607028898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758fedcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.536+0000 7f7d3bfff700 1 -- 172.21.15.12:0/607028898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563758dbe3b0 con 0x563758ff3720 2024-05-08T23:53:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.536+0000 7f7d3bfff700 1 --2- 172.21.15.12:0/607028898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758fed350 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7d240095c0 tx=0x7f7d24020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c8987c0005e7e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.536+0000 7f7d32ffd700 1 -- 172.21.15.12:0/607028898 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24005660 con 0x563758ff3720 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.536+0000 7f7d32ffd700 1 -- 172.21.15.12:0/607028898 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d2401e040 con 0x563758ff3720 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.536+0000 7f7d32ffd700 1 -- 172.21.15.12:0/607028898 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400f7d0 con 0x563758ff3720 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/607028898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 msgr2=0x563758fed350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/607028898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758fed350 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7d240095c0 tx=0x7f7d24020000 comp rx=0 tx=0).stop 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/607028898 shutdown_connections 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/607028898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758fedcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/607028898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/607028898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758fed350 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/607028898 >> 172.21.15.12:0/607028898 conn(0x563758f4b210 msgr2=0x563758fe7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/607028898 shutdown_connections 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/607028898 wait complete. 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 -- start start 2024-05-08T23:53:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.537+0000 7f7d4ec4eb80 1 Processor -- start 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758ff9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758ff70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563758ed9560 con 0x563758ff3720 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563758ed9200 con 0x563758fed890 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563758dc06a0 con 0x563758ff2e00 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d4ec4eb80 10 client.0 _ll_register_callbacks cb 0x563758ee2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d41031700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758ff70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d40830700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758ff9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d41031700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758ff70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44020/0 (socket says 172.21.15.12:44020) 2024-05-08T23:53:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d41031700 1 -- 172.21.15.12:0/2742249329 learned_addr learned my addr 172.21.15.12:0/2742249329 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d3bfff700 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d41031700 1 -- 172.21.15.12:0/2742249329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 msgr2=0x563758ff6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d41031700 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d41031700 1 -- 172.21.15.12:0/2742249329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 msgr2=0x563758ff9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d41031700 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758ff9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.538+0000 7f7d41031700 1 -- 172.21.15.12:0/2742249329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c10e670 con 0x563758ff3720 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d41031700 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758ff70f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7d34011e80 tx=0x7f7d34014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7865f89bfbb374d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2742249329 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400e040 con 0x563758ff3720 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2742249329 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7d3401a070 con 0x563758ff3720 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2742249329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563758e96270 con 0x563758ff3720 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2742249329 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d34027420 con 0x563758ff3720 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d4ec4eb80 10 client.17442 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2742249329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563758d6ab60 con 0x563758ff3720 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2742249329 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563758e96270 con 0x563758ff3720 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2742249329 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7d34016470 con 0x563758ff3720 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d4ec4eb80 10 client.17442 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d4ec4eb80 10 client.17442 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d4ec4eb80 20 client.17442 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563759007200 0x563759009620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d4ec4eb80 4 client.17442 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.539+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2742249329 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563758fbc810 con 0x563759007200 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.540+0000 7f7d40830700 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563759007200 0x563759009620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.540+0000 7f7d40830700 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563759007200 0x563759009620 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a422dd27494197d3 server_cookie=aca5d8c6c5e0fb29 in_seq=0 out_seq=0 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.540+0000 7f7d1bfff700 10 client.17442 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2742249329 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563758fbc810 con 0x563759007200 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d1bfff700 10 client.17442 handle_command_reply: tid=0 2024-05-08T23:53:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 client.17442 shutdown 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2742249329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 msgr2=0x563758ff70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758ff70f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7d34011e80 tx=0x7f7d34014600 comp rx=0 tx=0).stop 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2742249329 shutdown_connections 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563759007200 0x563759009620 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x563758ff70f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758ff9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d1bfff700 0 client.17442 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2742249329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2742249329 >> 172.21.15.12:0/2742249329 conn(0x563758f4b210 msgr2=0x563758fe75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2742249329 shutdown_connections 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2742249329 wait complete. 2024-05-08T23:53:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.541+0000 7f7d4ec4eb80 20 client.17442 trim_cache size 0 max 16384 2024-05-08T23:53:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.555+0000 7f7d4ec4eb80 1 Processor -- start 2024-05-08T23:53:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.555+0000 7f7d4ec4eb80 1 -- start start 2024-05-08T23:53:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.555+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758fed890 0x563758fed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.555+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.555+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758ff3720 0x563758fe7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.555+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563758fe9a80 con 0x563758fed890 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.555+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563758fe2830 con 0x563758ff3720 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.555+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563758fe2ac0 con 0x563758ff2e00 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d40830700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758fed890 0x563758fed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d40830700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758fed890 0x563758fed350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44026/0 (socket says 172.21.15.12:44026) 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d40830700 1 -- 172.21.15.12:0/1049950754 learned_addr learned my addr 172.21.15.12:0/1049950754 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d3bfff700 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d41031700 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758ff3720 0x563758fe7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d40830700 1 -- 172.21.15.12:0/1049950754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 msgr2=0x563758ff7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d40830700 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d40830700 1 -- 172.21.15.12:0/1049950754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758ff3720 msgr2=0x563758fe7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d40830700 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758ff3720 0x563758fe7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d40830700 1 -- 172.21.15.12:0/1049950754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563758fe44e0 con 0x563758fed890 2024-05-08T23:53:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d3bfff700 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d41031700 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758ff3720 0x563758fe7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d40830700 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758fed890 0x563758fed350 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0ca5e0 tx=0x7f7d2c0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b285d222953929c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d337fe700 1 -- 172.21.15.12:0/1049950754 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0a23c0 con 0x563758fed890 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d337fe700 1 -- 172.21.15.12:0/1049950754 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d2c0982e0 con 0x563758fed890 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.556+0000 7f7d337fe700 1 -- 172.21.15.12:0/1049950754 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c099690 con 0x563758fed890 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/1049950754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758fed890 msgr2=0x563758fed350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758fed890 0x563758fed350 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0ca5e0 tx=0x7f7d2c0cc040 comp rx=0 tx=0).stop 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/1049950754 shutdown_connections 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758ff3720 0x563758fe7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758fed890 0x563758fed350 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/1049950754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ff7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/1049950754 >> 172.21.15.12:0/1049950754 conn(0x563758f4b210 msgr2=0x563759002bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/1049950754 shutdown_connections 2024-05-08T23:53:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/1049950754 wait complete. 2024-05-08T23:53:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 -- start start 2024-05-08T23:53:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.557+0000 7f7d4ec4eb80 1 Processor -- start 2024-05-08T23:53:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758ffc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ffc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d4ec4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x56375905c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56375905cc40 con 0x563758ff3720 2024-05-08T23:53:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56375905cd80 con 0x563758fed890 2024-05-08T23:53:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d4ec4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56375905cef0 con 0x563758ff2e00 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d4ec4eb80 10 client.0 _ll_register_callbacks cb 0x7f7d340117b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d41031700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x56375905c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d41031700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x56375905c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44040/0 (socket says 172.21.15.12:44040) 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d41031700 1 -- 172.21.15.12:0/2446087261 learned_addr learned my addr 172.21.15.12:0/2446087261 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d40830700 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758ffc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d3bfff700 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ffc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d41031700 1 -- 172.21.15.12:0/2446087261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 msgr2=0x563758ffc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d41031700 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ffc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d41031700 1 -- 172.21.15.12:0/2446087261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 msgr2=0x563758ffc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d41031700 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758ffc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.558+0000 7f7d41031700 1 -- 172.21.15.12:0/2446087261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c1a4510 con 0x563758ff3720 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d41031700 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x56375905c700 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7d34000f80 tx=0x7f7d34009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322155818c64b2a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2446087261 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400e070 con 0x563758ff3720 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2446087261 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7d3401a040 con 0x563758ff3720 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2446087261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d2401e040 con 0x563758ff3720 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2446087261 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d34016610 con 0x563758ff3720 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d4ec4eb80 10 client.17448 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2446087261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563758f03560 con 0x563758ff3720 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2446087261 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d2401e040 con 0x563758ff3720 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2446087261 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7d34005980 con 0x563758ff3720 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d4ec4eb80 10 client.17448 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d4ec4eb80 10 client.17448 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d4ec4eb80 20 client.17448 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56375905e690 0x563759060ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d4ec4eb80 4 client.17448 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:53:37.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.559+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2446087261 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563758f35ec0 con 0x56375905e690 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.560+0000 7f7d40830700 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56375905e690 0x563759060ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.560+0000 7f7d1bfff700 10 client.17448 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.560+0000 7f7d40830700 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56375905e690 0x563759060ab0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b77ecc6fed4731c9 server_cookie=7172727b5bac06d0 in_seq=0 out_seq=0 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.560+0000 7f7d1bfff700 1 -- 172.21.15.12:0/2446087261 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563758f35ec0 con 0x56375905e690 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.560+0000 7f7d1bfff700 10 client.17448 handle_command_reply: tid=0 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.560+0000 7f7d4ec4eb80 1 client.17448 shutdown 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.560+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2446087261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 msgr2=0x56375905c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.560+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x56375905c700 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7d34000f80 tx=0x7f7d34009920 comp rx=0 tx=0).stop 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.560+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2446087261 shutdown_connections 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.560+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56375905e690 0x563759060ab0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563758ff3720 0x56375905c700 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563758fed890 0x563758ffc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d4ec4eb80 1 --2- 172.21.15.12:0/2446087261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563758ff2e00 0x563758ffc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d1bfff700 0 client.17448 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:53:37.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2446087261 >> 172.21.15.12:0/2446087261 conn(0x563758f4b210 msgr2=0x563759002bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2446087261 shutdown_connections 2024-05-08T23:53:37.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d4ec4eb80 1 -- 172.21.15.12:0/2446087261 wait complete. 2024-05-08T23:53:37.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d4ec4eb80 20 client.17448 trim_cache size 0 max 16384 2024-05-08T23:53:37.562 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:53:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 -- 172.21.15.12:0/2311133004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d280a1310 msgr2=0x7f7d280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d280a1310 0x7f7d280a37d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c1989a0 tx=0x7f7d24005fd0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 -- 172.21.15.12:0/2311133004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d3c101160 msgr2=0x7f7d3c1974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d3c101160 0x7f7d3c1974a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e9a0 tx=0x7f7d2c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 -- 172.21.15.12:0/2311133004 shutdown_connections 2024-05-08T23:53:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d280a1310 0x7f7d280a37d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d3c19bd30 0x7f7d3c19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d3c101160 0x7f7d3c1974a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 --2- 172.21.15.12:0/2311133004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d3c106280 0x7f7d3c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 -- 172.21.15.12:0/2311133004 >> 172.21.15.12:0/2311133004 conn(0x7f7d3c0fc9e0 msgr2=0x7f7d3c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 -- 172.21.15.12:0/2311133004 shutdown_connections 2024-05-08T23:53:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.561+0000 7f7d42a94700 1 -- 172.21.15.12:0/2311133004 wait complete. 2024-05-08T23:53:37.570 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:53:37.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:37 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/261408994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:53:37.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:37 smithi012 ceph-mon[42503]: pgmap v225: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-08T23:53:37.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.714+0000 7fc3d9156700 1 -- 172.21.15.12:0/1566771137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3d4100220 msgr2=0x7fc3d410ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.714+0000 7fc3d9156700 1 --2- 172.21.15.12:0/1566771137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3d4100220 0x7fc3d410ddf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00b0a0 tx=0x7fc3cc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.714+0000 7fc3d9156700 1 -- 172.21.15.12:0/1566771137 shutdown_connections 2024-05-08T23:53:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.714+0000 7fc3d9156700 1 --2- 172.21.15.12:0/1566771137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3d4100220 0x7fc3d410ddf0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.714+0000 7fc3d9156700 1 --2- 172.21.15.12:0/1566771137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3d4105f40 0x7fc3d4106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.714+0000 7fc3d9156700 1 --2- 172.21.15.12:0/1566771137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3d40ff860 0x7fc3d40ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.714+0000 7fc3d9156700 1 -- 172.21.15.12:0/1566771137 >> 172.21.15.12:0/1566771137 conn(0x7fc3d40751c0 msgr2=0x7fc3d40755d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.714+0000 7fc3d9156700 1 -- 172.21.15.12:0/1566771137 shutdown_connections 2024-05-08T23:53:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.714+0000 7fc3d9156700 1 -- 172.21.15.12:0/1566771137 wait complete. 2024-05-08T23:53:37.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.715+0000 7fc3d9156700 1 Processor -- start 2024-05-08T23:53:37.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.715+0000 7fc3d9156700 1 -- start start 2024-05-08T23:53:37.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.715+0000 7fc3d9156700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3d40ff860 0x7fc3d41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.715+0000 7fc3d9156700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3d4100220 0x7fc3d4199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.715+0000 7fc3d9156700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3d4105f40 0x7fc3d419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.715+0000 7fc3d9156700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419d5b0 con 0x7fc3d4100220 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.715+0000 7fc3d9156700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419d720 con 0x7fc3d40ff860 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.715+0000 7fc3d9156700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419d8e0 con 0x7fc3d4105f40 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3d4100220 0x7fc3d4199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3d4100220 0x7fc3d4199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44064/0 (socket says 172.21.15.12:44064) 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d259c700 1 -- 172.21.15.12:0/2625750427 learned_addr learned my addr 172.21.15.12:0/2625750427 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3d40ff860 0x7fc3d41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d359e700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3d4105f40 0x7fc3d419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d259c700 1 -- 172.21.15.12:0/2625750427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3d4105f40 msgr2=0x7fc3d419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d259c700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3d4105f40 0x7fc3d419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d259c700 1 -- 172.21.15.12:0/2625750427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3d40ff860 msgr2=0x7fc3d41997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d259c700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3d40ff860 0x7fc3d41997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d259c700 1 -- 172.21.15.12:0/2625750427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc009d00 con 0x7fc3d4100220 2024-05-08T23:53:37.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3d40ff860 0x7fc3d41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:37.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d259c700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3d4100220 0x7fc3d4199d10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400e9a0 tx=0x7fc3c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2625750427 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4009790 con 0x7fc3d4100220 2024-05-08T23:53:37.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2625750427 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c400cbd0 con 0x7fc3d4100220 2024-05-08T23:53:37.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d9156700 1 -- 172.21.15.12:0/2625750427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d419dbc0 con 0x7fc3d4100220 2024-05-08T23:53:37.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2625750427 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4018680 con 0x7fc3d4100220 2024-05-08T23:53:37.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.716+0000 7fc3d9156700 1 -- 172.21.15.12:0/2625750427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d41a6520 con 0x7fc3d4100220 2024-05-08T23:53:37.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.719+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2625750427 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc3c400cd40 con 0x7fc3d4100220 2024-05-08T23:53:37.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.719+0000 7fc3bbfff700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3bc0a1290 0x7fc3bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.719+0000 7fc3bbfff700 1 -- 172.21.15.12:0/2625750427 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc3c4014070 con 0x7fc3d4100220 2024-05-08T23:53:37.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.720+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3bc0a1290 0x7fc3bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.720+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3bc0a1290 0x7fc3bc0a3750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8011490 tx=0x7fc3c8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.720+0000 7fc3e5310b80 1 Processor -- start 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3e5310b80 1 -- start start 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d055e00 0x55629d0561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 0x55629d050350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d050890 0x55629d050cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629cfa7150 con 0x55629d055e00 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629cfa4b10 con 0x55629d056720 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629cfa0400 con 0x55629d050890 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d055e00 0x55629d0561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d055e00 0x55629d0561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44074/0 (socket says 172.21.15.12:44074) 2024-05-08T23:53:37.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/2319041829 learned_addr learned my addr 172.21.15.12:0/2319041829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d259c700 1 --2- 172.21.15.12:0/2319041829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 0x55629d050350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d359e700 1 --2- 172.21.15.12:0/2319041829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d050890 0x55629d050cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/2319041829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d050890 msgr2=0x55629d050cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/2319041829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d050890 0x55629d050cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/2319041829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 msgr2=0x55629d050350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/2319041829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 0x55629d050350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/2319041829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55629ce213b0 con 0x55629d055e00 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d259c700 1 --2- 172.21.15.12:0/2319041829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 0x55629d050350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.721+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/2319041829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d055e00 0x55629d0561e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80043c0 tx=0x7fc3c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a536e998db6b463d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3b97fa700 1 -- 172.21.15.12:0/2319041829 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8003d10 con 0x55629d055e00 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3b97fa700 1 -- 172.21.15.12:0/2319041829 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c8003e70 con 0x55629d055e00 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3b97fa700 1 -- 172.21.15.12:0/2319041829 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c801e780 con 0x55629d055e00 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3e5310b80 1 -- 172.21.15.12:0/2319041829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d055e00 msgr2=0x55629d0561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/2319041829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d055e00 0x55629d0561e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80043c0 tx=0x7fc3c8014000 comp rx=0 tx=0).stop 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3e5310b80 1 -- 172.21.15.12:0/2319041829 shutdown_connections 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/2319041829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d050890 0x55629d050cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/2319041829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d055e00 0x55629d0561e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/2319041829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 0x55629d050350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3e5310b80 1 -- 172.21.15.12:0/2319041829 >> 172.21.15.12:0/2319041829 conn(0x55629cfae210 msgr2=0x55629d04adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.722+0000 7fc3e5310b80 1 -- 172.21.15.12:0/2319041829 shutdown_connections 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3e5310b80 1 -- 172.21.15.12:0/2319041829 wait complete. 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3e5310b80 1 -- start start 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3e5310b80 1 Processor -- start 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d059ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d055e00 0x55629d05a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 0x55629d05a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629cf3c560 con 0x55629d050890 2024-05-08T23:53:37.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629cf3c200 con 0x55629d056720 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629ce236a0 con 0x55629d055e00 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3e5310b80 10 client.0 _ll_register_callbacks cb 0x55629cf45b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d059ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d059ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44086/0 (socket says 172.21.15.12:44086) 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.723+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/206302091 learned_addr learned my addr 172.21.15.12:0/206302091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3d259c700 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d055e00 0x55629d05a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3d359e700 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 0x55629d05a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/206302091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d055e00 msgr2=0x55629d05a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d055e00 0x55629d05a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/206302091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 msgr2=0x55629d05a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 0x55629d05a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/206302091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d41a6520 con 0x55629d050890 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3d259c700 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d055e00 0x55629d05a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:37.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d059ad0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8004390 tx=0x7fc3c8026820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ada35476c5c7b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/206302091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c802b070 con 0x55629d050890 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/206302091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc3c8026cd0 con 0x55629d050890 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3e5310b80 1 -- 172.21.15.12:0/206302091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55629cef9270 con 0x55629d050890 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/206302091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3c8030420 con 0x55629d050890 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3e5310b80 10 client.17460 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.724+0000 7fc3e5310b80 1 -- 172.21.15.12:0/206302091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fc3c8017070 con 0x55629d050890 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/206302091 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55629cef9270 con 0x55629d050890 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/206302091 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc3c8024020 con 0x55629d050890 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3e5310b80 10 client.17460 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3e5310b80 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3e5310b80 20 client.17460 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:37.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55629d06a160 0x55629d06c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3e5310b80 4 client.17460 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3e5310b80 1 -- 172.21.15.12:0/206302091 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55629d01f810 con 0x55629d06a160 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3d259c700 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55629d06a160 0x55629d06c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3b6ff5700 10 client.17460 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.725+0000 7fc3d259c700 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55629d06a160 0x55629d06c550 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222615eadc39fb9c server_cookie=c9da0b25ecbf0a86 in_seq=0 out_seq=0 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/206302091 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55629d01f810 con 0x55629d06a160 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3b6ff5700 10 client.17460 handle_command_reply: tid=0 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 client.17460 shutdown 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 -- 172.21.15.12:0/206302091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 msgr2=0x55629d059ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d059ad0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8004390 tx=0x7fc3c8026820 comp rx=0 tx=0).stop 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 -- 172.21.15.12:0/206302091 shutdown_connections 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55629d06a160 0x55629d06c550 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d056720 0x55629d05a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d059ad0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3b6ff5700 0 client.17460 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:37.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/206302091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d055e00 0x55629d05a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 -- 172.21.15.12:0/206302091 >> 172.21.15.12:0/206302091 conn(0x55629cfae210 msgr2=0x55629d04a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 -- 172.21.15.12:0/206302091 shutdown_connections 2024-05-08T23:53:37.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.726+0000 7fc3e5310b80 1 -- 172.21.15.12:0/206302091 wait complete. 2024-05-08T23:53:37.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.727+0000 7fc3e5310b80 20 client.17460 trim_cache size 0 max 16384 2024-05-08T23:53:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.740+0000 7fc3e5310b80 1 Processor -- start 2024-05-08T23:53:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.740+0000 7fc3e5310b80 1 -- start start 2024-05-08T23:53:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d050350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d055e00 0x55629d04ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 0x55629d061160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629d05c640 con 0x55629d050890 2024-05-08T23:53:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629d056560 con 0x55629d055e00 2024-05-08T23:53:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629d071760 con 0x55629d056720 2024-05-08T23:53:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d050350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d050350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44088/0 (socket says 172.21.15.12:44088) 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/871732615 learned_addr learned my addr 172.21.15.12:0/871732615 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d359e700 1 --2- 172.21.15.12:0/871732615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 0x55629d061160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d259c700 1 --2- 172.21.15.12:0/871732615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d055e00 0x55629d04ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/871732615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 msgr2=0x55629d061160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/871732615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 0x55629d061160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/871732615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d055e00 msgr2=0x55629d04ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/871732615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d055e00 0x55629d04ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/871732615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55629cdcdb60 con 0x55629d050890 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/871732615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d050350 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8004360 tx=0x7fc3c8011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0f5713e201d7f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3b9ffb700 1 -- 172.21.15.12:0/871732615 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c802b020 con 0x55629d050890 2024-05-08T23:53:37.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3b9ffb700 1 -- 172.21.15.12:0/871732615 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c80054f0 con 0x55629d050890 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.741+0000 7fc3b9ffb700 1 -- 172.21.15.12:0/871732615 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8004a50 con 0x55629d050890 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 -- 172.21.15.12:0/871732615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 msgr2=0x55629d050350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/871732615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d050350 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8004360 tx=0x7fc3c8011be0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 -- 172.21.15.12:0/871732615 shutdown_connections 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/871732615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 0x55629d061160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/871732615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d050350 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/871732615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d055e00 0x55629d04ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 -- 172.21.15.12:0/871732615 >> 172.21.15.12:0/871732615 conn(0x55629cfae210 msgr2=0x55629d0583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 -- 172.21.15.12:0/871732615 shutdown_connections 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 -- 172.21.15.12:0/871732615 wait complete. 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 -- start start 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.742+0000 7fc3e5310b80 1 Processor -- start 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d05d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d055e00 0x55629d05d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3e5310b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 0x55629d05dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629d0bf620 con 0x55629d050890 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629d0bf760 con 0x55629d055e00 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3e5310b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55629d0bf8a0 con 0x55629d056720 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3e5310b80 10 client.0 _ll_register_callbacks cb 0x55629cf45b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d05d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d05d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44090/0 (socket says 172.21.15.12:44090) 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/3584984587 learned_addr learned my addr 172.21.15.12:0/3584984587 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d259c700 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d055e00 0x55629d05d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d359e700 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 0x55629d05dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/3584984587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 msgr2=0x55629d05dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 0x55629d05dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/3584984587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d055e00 msgr2=0x55629d05d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d055e00 0x55629d05d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d2d9d700 1 -- 172.21.15.12:0/3584984587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c8017070 con 0x55629d050890 2024-05-08T23:53:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d359e700 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 0x55629d05dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:53:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3d2d9d700 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d05d140 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc3c801ebe0 tx=0x7fc3c8026820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48adcac8f102ca40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.743+0000 7fc3b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/3584984587 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c802b030 con 0x55629d050890 2024-05-08T23:53:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/3584984587 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc3c8026cd0 con 0x55629d050890 2024-05-08T23:53:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3e5310b80 1 -- 172.21.15.12:0/3584984587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3c8003e70 con 0x55629d050890 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/3584984587 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3c801e880 con 0x55629d050890 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3e5310b80 10 client.17466 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3e5310b80 1 -- 172.21.15.12:0/3584984587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55629cf66560 con 0x55629d050890 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/3584984587 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c8003e70 con 0x55629d050890 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/3584984587 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc3c8024070 con 0x55629d050890 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3e5310b80 10 client.17466 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3e5310b80 10 client.17466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3e5310b80 20 client.17466 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55629d0c0b00 0x55629d0c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3e5310b80 4 client.17466 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3e5310b80 1 -- 172.21.15.12:0/3584984587 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55629cf98ec0 con 0x55629d0c0b00 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.744+0000 7fc3d259c700 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55629d0c0b00 0x55629d0c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:37.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3b6ff5700 10 client.17466 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3d259c700 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55629d0c0b00 0x55629d0c2f20 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a96ad3e9a35478f7 server_cookie=4d968a20f63426b9 in_seq=0 out_seq=0 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3b6ff5700 1 -- 172.21.15.12:0/3584984587 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55629cf98ec0 con 0x55629d0c0b00 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3b6ff5700 10 client.17466 handle_command_reply: tid=0 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 client.17466 shutdown 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 -- 172.21.15.12:0/3584984587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 msgr2=0x55629d05d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d05d140 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc3c801ebe0 tx=0x7fc3c8026820 comp rx=0 tx=0).stop 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 -- 172.21.15.12:0/3584984587 shutdown_connections 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55629d056720 0x55629d05dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629d050890 0x55629d05d140 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55629d0c0b00 0x55629d0c2f20 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 --2- 172.21.15.12:0/3584984587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55629d055e00 0x55629d05d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3b6ff5700 0 client.17466 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:37.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 -- 172.21.15.12:0/3584984587 >> 172.21.15.12:0/3584984587 conn(0x55629cfae210 msgr2=0x55629d054bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 -- 172.21.15.12:0/3584984587 shutdown_connections 2024-05-08T23:53:37.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.745+0000 7fc3e5310b80 1 -- 172.21.15.12:0/3584984587 wait complete. 2024-05-08T23:53:37.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3e5310b80 20 client.17466 trim_cache size 0 max 16384 2024-05-08T23:53:37.748 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:53:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 -- 172.21.15.12:0/2625750427 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3bc0a1290 msgr2=0x7fc3bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3bc0a1290 0x7fc3bc0a3750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8011490 tx=0x7fc3c8009260 comp rx=0 tx=0).stop 2024-05-08T23:53:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 -- 172.21.15.12:0/2625750427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3d4100220 msgr2=0x7fc3d4199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3d4100220 0x7fc3d4199d10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400e9a0 tx=0x7fc3c400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 -- 172.21.15.12:0/2625750427 shutdown_connections 2024-05-08T23:53:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3bc0a1290 0x7fc3bc0a3750 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3d4105f40 0x7fc3d419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3d40ff860 0x7fc3d41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 --2- 172.21.15.12:0/2625750427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3d4100220 0x7fc3d4199d10 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 -- 172.21.15.12:0/2625750427 >> 172.21.15.12:0/2625750427 conn(0x7fc3d40751c0 msgr2=0x7fc3d410c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 -- 172.21.15.12:0/2625750427 shutdown_connections 2024-05-08T23:53:37.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:37.746+0000 7fc3d9156700 1 -- 172.21.15.12:0/2625750427 wait complete. 2024-05-08T23:53:37.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:37 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/261408994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:53:37.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:37 smithi204 ceph-mon[47373]: pgmap v225: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-08T23:53:37.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:37 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/261408994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:53:37.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:37 smithi160 ceph-mon[46152]: pgmap v225: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-08T23:53:37.871 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465075 9.41 MB/sec execute 213 sec latency 1796.199 ms 2024-05-08T23:53:38.755 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:53:38.756 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:53:38.871 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465656 9.39 MB/sec execute 214 sec latency 2282.710 ms 2024-05-08T23:53:38.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.910+0000 7fa872482700 1 -- 172.21.15.12:0/1683254084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa86c105160 msgr2=0x7fa86c10cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:38.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.910+0000 7fa872482700 1 --2- 172.21.15.12:0/1683254084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa86c105160 0x7fa86c10cce0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b0a0 tx=0x7fa86400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.911+0000 7fa872482700 1 -- 172.21.15.12:0/1683254084 shutdown_connections 2024-05-08T23:53:38.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.911+0000 7fa872482700 1 --2- 172.21.15.12:0/1683254084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa86c105160 0x7fa86c10cce0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.911+0000 7fa872482700 1 --2- 172.21.15.12:0/1683254084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa86c100060 0x7fa86c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.911+0000 7fa872482700 1 --2- 172.21.15.12:0/1683254084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c100a10 0x7fa86c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.911+0000 7fa872482700 1 -- 172.21.15.12:0/1683254084 >> 172.21.15.12:0/1683254084 conn(0x7fa86c076f40 msgr2=0x7fa86c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:38.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.911+0000 7fa872482700 1 -- 172.21.15.12:0/1683254084 shutdown_connections 2024-05-08T23:53:38.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.911+0000 7fa872482700 1 -- 172.21.15.12:0/1683254084 wait complete. 2024-05-08T23:53:38.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.911+0000 7fa872482700 1 Processor -- start 2024-05-08T23:53:38.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.911+0000 7fa872482700 1 -- start start 2024-05-08T23:53:38.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa872482700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa86c100060 0x7fa86c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa872482700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c105160 0x7fa86c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa872482700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa86c19bda0 0x7fa86c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa872482700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c19c820 con 0x7fa86c100060 2024-05-08T23:53:38.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa872482700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c19c990 con 0x7fa86c19bda0 2024-05-08T23:53:38.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa872482700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c19cb50 con 0x7fa86c105160 2024-05-08T23:53:38.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa86bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa86c100060 0x7fa86c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa86bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa86c100060 0x7fa86c1974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48354/0 (socket says 172.21.15.12:48354) 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa86bfff700 1 -- 172.21.15.12:0/2025707576 learned_addr learned my addr 172.21.15.12:0/2025707576 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa870a1f700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa86c19bda0 0x7fa86c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa86b7fe700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c105160 0x7fa86c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa86bfff700 1 -- 172.21.15.12:0/2025707576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c105160 msgr2=0x7fa86c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa86bfff700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c105160 0x7fa86c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa86bfff700 1 -- 172.21.15.12:0/2025707576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa86c19bda0 msgr2=0x7fa86c19c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa86bfff700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa86c19bda0 0x7fa86c19c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa86bfff700 1 -- 172.21.15.12:0/2025707576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c009520 con 0x7fa86c100060 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa86bfff700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa86c100060 0x7fa86c1974c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00eb30 tx=0x7fa85c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.912+0000 7fa8697fa700 1 -- 172.21.15.12:0/2025707576 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c009880 con 0x7fa86c100060 2024-05-08T23:53:38.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.913+0000 7fa872482700 1 -- 172.21.15.12:0/2025707576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa864009d00 con 0x7fa86c100060 2024-05-08T23:53:38.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.913+0000 7fa8697fa700 1 -- 172.21.15.12:0/2025707576 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa85c0044e0 con 0x7fa86c100060 2024-05-08T23:53:38.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.913+0000 7fa8697fa700 1 -- 172.21.15.12:0/2025707576 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c003e90 con 0x7fa86c100060 2024-05-08T23:53:38.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.913+0000 7fa872482700 1 -- 172.21.15.12:0/2025707576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c1a4450 con 0x7fa86c100060 2024-05-08T23:53:38.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.915+0000 7fa8697fa700 1 -- 172.21.15.12:0/2025707576 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa85c004000 con 0x7fa86c100060 2024-05-08T23:53:38.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.915+0000 7fa8697fa700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8540a1290 0x7fa8540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.916+0000 7fa8697fa700 1 -- 172.21.15.12:0/2025707576 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fa85c014070 con 0x7fa86c100060 2024-05-08T23:53:38.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.916+0000 7fa86b7fe700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8540a1290 0x7fa8540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.916+0000 7fa86b7fe700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8540a1290 0x7fa8540a3750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa860005950 tx=0x7fa8600058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:38.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.916+0000 7fa87e63cb80 1 Processor -- start 2024-05-08T23:53:38.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.916+0000 7fa87e63cb80 1 -- start start 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 0x55e699c89260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c897a0 0x55e699c833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 0x55e699c83d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699bda1d0 con 0x55e699c88e80 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699bd7b90 con 0x55e699c83910 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699bd3480 con 0x55e699c897a0 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa86bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 0x55e699c89260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa86bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 0x55e699c89260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48368/0 (socket says 172.21.15.12:48368) 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa86bfff700 1 -- 172.21.15.12:0/3680496313 learned_addr learned my addr 172.21.15.12:0/3680496313 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa870a1f700 1 --2- 172.21.15.12:0/3680496313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 0x55e699c83d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa86b7fe700 1 --2- 172.21.15.12:0/3680496313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c897a0 0x55e699c833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa870a1f700 1 -- 172.21.15.12:0/3680496313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c897a0 msgr2=0x55e699c833d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa870a1f700 1 --2- 172.21.15.12:0/3680496313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c897a0 0x55e699c833d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa870a1f700 1 -- 172.21.15.12:0/3680496313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 msgr2=0x55e699c89260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa870a1f700 1 --2- 172.21.15.12:0/3680496313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 0x55e699c89260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa870a1f700 1 -- 172.21.15.12:0/3680496313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e699a54430 con 0x55e699c83910 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa86bfff700 1 --2- 172.21.15.12:0/3680496313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 0x55e699c89260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.917+0000 7fa870a1f700 1 --2- 172.21.15.12:0/3680496313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 0x55e699c83d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b780 tx=0x7fa864009410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3ab8b5a77e90145 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.918+0000 7fa8527fc700 1 -- 172.21.15.12:0/3680496313 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400e040 con 0x55e699c83910 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.918+0000 7fa8527fc700 1 -- 172.21.15.12:0/3680496313 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa864006000 con 0x55e699c83910 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.918+0000 7fa8527fc700 1 -- 172.21.15.12:0/3680496313 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864007920 con 0x55e699c83910 2024-05-08T23:53:38.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.918+0000 7fa87e63cb80 1 -- 172.21.15.12:0/3680496313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 msgr2=0x55e699c83d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:38.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.918+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/3680496313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 0x55e699c83d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b780 tx=0x7fa864009410 comp rx=0 tx=0).stop 2024-05-08T23:53:38.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 -- 172.21.15.12:0/3680496313 shutdown_connections 2024-05-08T23:53:38.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/3680496313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 0x55e699c83d50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/3680496313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 0x55e699c89260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/3680496313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c897a0 0x55e699c833d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 -- 172.21.15.12:0/3680496313 >> 172.21.15.12:0/3680496313 conn(0x55e699be1290 msgr2=0x55e699c7de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:38.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 -- 172.21.15.12:0/3680496313 shutdown_connections 2024-05-08T23:53:38.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 -- 172.21.15.12:0/3680496313 wait complete. 2024-05-08T23:53:38.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 -- start start 2024-05-08T23:53:38.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 Processor -- start 2024-05-08T23:53:38.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 0x55e699c8cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c88e80 0x55e699c8d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 0x55e699c8d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699b6f5e0 con 0x55e699c897a0 2024-05-08T23:53:38.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.919+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699b6f280 con 0x55e699c83910 2024-05-08T23:53:38.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699a56720 con 0x55e699c88e80 2024-05-08T23:53:38.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa87e63cb80 10 client.0 _ll_register_callbacks cb 0x55e699b78bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa870a1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 0x55e699c8d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa870a1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 0x55e699c8d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48382/0 (socket says 172.21.15.12:48382) 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa870a1f700 1 -- 172.21.15.12:0/236949730 learned_addr learned my addr 172.21.15.12:0/236949730 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa86b7fe700 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c88e80 0x55e699c8d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa86bfff700 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 0x55e699c8cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa870a1f700 1 -- 172.21.15.12:0/236949730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c88e80 msgr2=0x55e699c8d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa870a1f700 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c88e80 0x55e699c8d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa870a1f700 1 -- 172.21.15.12:0/236949730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 msgr2=0x55e699c8cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa870a1f700 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 0x55e699c8cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa870a1f700 1 -- 172.21.15.12:0/236949730 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c1a4450 con 0x55e699c897a0 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa870a1f700 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 0x55e699c8d590 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b3f0 tx=0x7fa86400ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4d203285c67af6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa84b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa84b7fe700 1 -- 172.21.15.12:0/236949730 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400e040 con 0x55e699c897a0 2024-05-08T23:53:38.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa84b7fe700 1 -- 172.21.15.12:0/236949730 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fa8640076b0 con 0x55e699c897a0 2024-05-08T23:53:38.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.920+0000 7fa87e63cb80 1 -- 172.21.15.12:0/236949730 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e699b2c2f0 con 0x55e699c897a0 2024-05-08T23:53:38.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa84b7fe700 1 -- 172.21.15.12:0/236949730 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa864007870 con 0x55e699c897a0 2024-05-08T23:53:38.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa87e63cb80 10 client.17478 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:38.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa87e63cb80 1 -- 172.21.15.12:0/236949730 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fa86c067ff0 con 0x55e699c897a0 2024-05-08T23:53:38.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa84b7fe700 1 -- 172.21.15.12:0/236949730 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e699b2c2f0 con 0x55e699c897a0 2024-05-08T23:53:38.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa84b7fe700 1 -- 172.21.15.12:0/236949730 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa864019070 con 0x55e699c897a0 2024-05-08T23:53:38.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa87e63cb80 10 client.17478 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:38.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa87e63cb80 10 client.17478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:38.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa87e63cb80 20 client.17478 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e699c9cdb0 0x55e699c9f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa87e63cb80 4 client.17478 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa87e63cb80 1 -- 172.21.15.12:0/236949730 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e699c52890 con 0x55e699c9cdb0 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.921+0000 7fa86bfff700 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e699c9cdb0 0x55e699c9f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.922+0000 7fa86bfff700 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e699c9cdb0 0x55e699c9f1a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3430b66b836e7668 server_cookie=430fd9427ec3201c in_seq=0 out_seq=0 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.922+0000 7fa84b7fe700 10 client.17478 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.922+0000 7fa84b7fe700 1 -- 172.21.15.12:0/236949730 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fa85c065a90 con 0x55e699c9cdb0 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.922+0000 7fa84b7fe700 10 client.17478 handle_command_reply: tid=0 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.922+0000 7fa87e63cb80 1 client.17478 shutdown 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.922+0000 7fa87e63cb80 1 -- 172.21.15.12:0/236949730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 msgr2=0x55e699c8d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.922+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 0x55e699c8d590 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b3f0 tx=0x7fa86400ce20 comp rx=0 tx=0).stop 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.923+0000 7fa87e63cb80 1 -- 172.21.15.12:0/236949730 shutdown_connections 2024-05-08T23:53:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.923+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 0x55e699c8d590 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.923+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e699c9cdb0 0x55e699c9f1a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.923+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c83910 0x55e699c8cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.923+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/236949730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c88e80 0x55e699c8d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.923+0000 7fa84b7fe700 0 client.17478 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.923+0000 7fa87e63cb80 1 -- 172.21.15.12:0/236949730 >> 172.21.15.12:0/236949730 conn(0x55e699be1290 msgr2=0x55e699c7d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.923+0000 7fa87e63cb80 1 -- 172.21.15.12:0/236949730 shutdown_connections 2024-05-08T23:53:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.923+0000 7fa87e63cb80 1 -- 172.21.15.12:0/236949730 wait complete. 2024-05-08T23:53:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.923+0000 7fa87e63cb80 20 client.17478 trim_cache size 0 max 16384 2024-05-08T23:53:38.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.953+0000 7fa87e63cb80 1 Processor -- start 2024-05-08T23:53:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.955+0000 7fa87e63cb80 1 -- start start 2024-05-08T23:53:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.956+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 0x55e699c833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.956+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c88e80 0x55e699c7dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.956+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 0x55e699c882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.956+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699c8f6b0 con 0x55e699c897a0 2024-05-08T23:53:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.956+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699ca43b0 con 0x55e699c88e80 2024-05-08T23:53:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.956+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699c78880 con 0x55e699c83910 2024-05-08T23:53:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.956+0000 7fa86bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 0x55e699c833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.956+0000 7fa86bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 0x55e699c833d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:55520/0 (socket says 172.21.15.12:55520) 2024-05-08T23:53:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.956+0000 7fa86bfff700 1 -- 172.21.15.12:0/1222917830 learned_addr learned my addr 172.21.15.12:0/1222917830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.956+0000 7fa86b7fe700 1 --2- 172.21.15.12:0/1222917830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c88e80 0x55e699c7dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.957+0000 7fa86bfff700 1 -- 172.21.15.12:0/1222917830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c88e80 msgr2=0x55e699c7dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.957+0000 7fa86bfff700 1 --2- 172.21.15.12:0/1222917830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c88e80 0x55e699c7dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.957+0000 7fa86bfff700 1 -- 172.21.15.12:0/1222917830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 msgr2=0x55e699c882f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:53:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.957+0000 7fa86bfff700 1 --2- 172.21.15.12:0/1222917830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 0x55e699c882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.958+0000 7fa86bfff700 1 -- 172.21.15.12:0/1222917830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e699c83fc0 con 0x55e699c83910 2024-05-08T23:53:38.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.961+0000 7fa86bfff700 1 --2- 172.21.15.12:0/1222917830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 0x55e699c833d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0ca980 tx=0x7fa85c0cbf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43b6deb227060b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:38.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.962+0000 7fa852ffd700 1 -- 172.21.15.12:0/1222917830 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c09a990 con 0x55e699c83910 2024-05-08T23:53:38.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.962+0000 7fa852ffd700 1 -- 172.21.15.12:0/1222917830 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa85c061070 con 0x55e699c83910 2024-05-08T23:53:38.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.962+0000 7fa852ffd700 1 -- 172.21.15.12:0/1222917830 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0a2710 con 0x55e699c83910 2024-05-08T23:53:38.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.963+0000 7fa87e63cb80 1 -- 172.21.15.12:0/1222917830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 msgr2=0x55e699c833d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:38.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.963+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/1222917830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 0x55e699c833d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0ca980 tx=0x7fa85c0cbf90 comp rx=0 tx=0).stop 2024-05-08T23:53:38.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.963+0000 7fa87e63cb80 1 -- 172.21.15.12:0/1222917830 shutdown_connections 2024-05-08T23:53:38.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.963+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/1222917830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c897a0 0x55e699c882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.963+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/1222917830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 0x55e699c833d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.963+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/1222917830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c88e80 0x55e699c7dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.963+0000 7fa87e63cb80 1 -- 172.21.15.12:0/1222917830 >> 172.21.15.12:0/1222917830 conn(0x55e699be1290 msgr2=0x55e699c87be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:38.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.963+0000 7fa87e63cb80 1 -- 172.21.15.12:0/1222917830 shutdown_connections 2024-05-08T23:53:38.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.963+0000 7fa87e63cb80 1 -- 172.21.15.12:0/1222917830 wait complete. 2024-05-08T23:53:38.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.963+0000 7fa87e63cb80 1 -- start start 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa87e63cb80 1 Processor -- start 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 0x55e699c91500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 0x55e699c91a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa87e63cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c897a0 0x55e699ced250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699ced790 con 0x55e699c88e80 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699ced900 con 0x55e699c897a0 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa87e63cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e699cedac0 con 0x55e699c83910 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa87e63cb80 10 client.0 _ll_register_callbacks cb 0x55e699b78bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa870a1f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c897a0 0x55e699ced250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa86b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 0x55e699c91a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa870a1f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c897a0 0x55e699ced250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50836/0 (socket says 172.21.15.12:50836) 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa870a1f700 1 -- 172.21.15.12:0/2170558775 learned_addr learned my addr 172.21.15.12:0/2170558775 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.964+0000 7fa86bfff700 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 0x55e699c91500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa870a1f700 1 -- 172.21.15.12:0/2170558775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 msgr2=0x55e699c91500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa870a1f700 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 0x55e699c91500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa870a1f700 1 -- 172.21.15.12:0/2170558775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 msgr2=0x55e699c91a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa870a1f700 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 0x55e699c91a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa870a1f700 1 -- 172.21.15.12:0/2170558775 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c0af040 con 0x55e699c897a0 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa870a1f700 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c897a0 0x55e699ced250 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b0a0 tx=0x7fa864000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999dd312aaa4259c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa84b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa84b7fe700 1 -- 172.21.15.12:0/2170558775 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400e040 con 0x55e699c897a0 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa84b7fe700 1 -- 172.21.15.12:0/2170558775 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fa864003c10 con 0x55e699c897a0 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa87e63cb80 1 -- 172.21.15.12:0/2170558775 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa864006000 con 0x55e699c897a0 2024-05-08T23:53:38.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa84b7fe700 1 -- 172.21.15.12:0/2170558775 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa864003e10 con 0x55e699c897a0 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa87e63cb80 10 client.27269 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.965+0000 7fa87e63cb80 1 -- 172.21.15.12:0/2170558775 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e699a00be0 con 0x55e699c897a0 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa84b7fe700 1 -- 172.21.15.12:0/2170558775 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa864006000 con 0x55e699c897a0 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa84b7fe700 1 -- 172.21.15.12:0/2170558775 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa864019070 con 0x55e699c897a0 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa87e63cb80 10 client.27269 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa87e63cb80 10 client.27269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa87e63cb80 20 client.27269 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e699cf4f00 0x55e699cf72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa87e63cb80 4 client.27269 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa87e63cb80 1 -- 172.21.15.12:0/2170558775 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e699bcbf40 con 0x55e699cf4f00 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa86bfff700 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e699cf4f00 0x55e699cf72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa84b7fe700 10 client.27269 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:38.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.966+0000 7fa86bfff700 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e699cf4f00 0x55e699cf72f0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46bb939feec4db67 server_cookie=f10ccf34bbbf8c11 in_seq=0 out_seq=0 2024-05-08T23:53:38.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.968+0000 7fa84b7fe700 1 -- 172.21.15.12:0/2170558775 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e699bcbf40 con 0x55e699cf4f00 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.968+0000 7fa84b7fe700 10 client.27269 handle_command_reply: tid=0 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.968+0000 7fa87e63cb80 1 client.27269 shutdown 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.968+0000 7fa87e63cb80 1 -- 172.21.15.12:0/2170558775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c897a0 msgr2=0x55e699ced250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.968+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c897a0 0x55e699ced250 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b0a0 tx=0x7fa864000f40 comp rx=0 tx=0).stop 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa87e63cb80 1 -- 172.21.15.12:0/2170558775 shutdown_connections 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e699cf4f00 0x55e699cf72f0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e699c897a0 0x55e699ced250 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e699c83910 0x55e699c91500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa87e63cb80 1 --2- 172.21.15.12:0/2170558775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e699c88e80 0x55e699c91a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa84b7fe700 0 client.27269 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa87e63cb80 1 -- 172.21.15.12:0/2170558775 >> 172.21.15.12:0/2170558775 conn(0x55e699be1290 msgr2=0x55e699c88470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa87e63cb80 1 -- 172.21.15.12:0/2170558775 shutdown_connections 2024-05-08T23:53:38.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa87e63cb80 1 -- 172.21.15.12:0/2170558775 wait complete. 2024-05-08T23:53:38.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa87e63cb80 20 client.27269 trim_cache size 0 max 16384 2024-05-08T23:53:38.967 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:53:38.967 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:53:38.967 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "b6593e18-0212-4d3d-a85d-9cbe6e22b015", 2024-05-08T23:53:38.967 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:53:38.967 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:53:38.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 -- 172.21.15.12:0/2025707576 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8540a1290 msgr2=0x7fa8540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:38.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8540a1290 0x7fa8540a3750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa860005950 tx=0x7fa8600058e0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 -- 172.21.15.12:0/2025707576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa86c100060 msgr2=0x7fa86c1974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:38.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa86c100060 0x7fa86c1974c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00eb30 tx=0x7fa85c00ee40 comp rx=0 tx=0).stop 2024-05-08T23:53:38.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 -- 172.21.15.12:0/2025707576 shutdown_connections 2024-05-08T23:53:38.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8540a1290 0x7fa8540a3750 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa86c19bda0 0x7fa86c19c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa86c100060 0x7fa86c1974c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 --2- 172.21.15.12:0/2025707576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c105160 0x7fa86c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 -- 172.21.15.12:0/2025707576 >> 172.21.15.12:0/2025707576 conn(0x7fa86c076f40 msgr2=0x7fa86c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.969+0000 7fa872482700 1 -- 172.21.15.12:0/2025707576 shutdown_connections 2024-05-08T23:53:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:38.970+0000 7fa872482700 1 -- 172.21.15.12:0/2025707576 wait complete. 2024-05-08T23:53:38.978 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.126+0000 7f4ecb152700 1 -- 172.21.15.12:0/3879744782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ec41004f0 msgr2=0x7f4ec4105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.126+0000 7f4ecb152700 1 --2- 172.21.15.12:0/3879744782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ec41004f0 0x7f4ec4105c20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00099c0 tx=0x7f4eb0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.126+0000 7f4ecb152700 1 -- 172.21.15.12:0/3879744782 shutdown_connections 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.126+0000 7f4ecb152700 1 --2- 172.21.15.12:0/3879744782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec4106350 0x7f4ec410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.126+0000 7f4ecb152700 1 --2- 172.21.15.12:0/3879744782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ec40ffb40 0x7f4ec40fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.126+0000 7f4ecb152700 1 --2- 172.21.15.12:0/3879744782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ec41004f0 0x7f4ec4105c20 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.126+0000 7f4ecb152700 1 -- 172.21.15.12:0/3879744782 >> 172.21.15.12:0/3879744782 conn(0x7f4ec4075070 msgr2=0x7f4ec4075480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.126+0000 7f4ecb152700 1 -- 172.21.15.12:0/3879744782 shutdown_connections 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.126+0000 7f4ecb152700 1 -- 172.21.15.12:0/3879744782 wait complete. 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ecb152700 1 Processor -- start 2024-05-08T23:53:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ecb152700 1 -- start start 2024-05-08T23:53:39.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ecb152700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ec40ffb40 0x7f4ec4100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ecb152700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ec4106350 0x7f4ec4100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ecb152700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec4101530 0x7f4ec41a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ecb152700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41a5b50 con 0x7f4ec4106350 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ecb152700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41a5cc0 con 0x7f4ec4101530 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ecb152700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41a5e80 con 0x7f4ec40ffb40 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ec4106350 0x7f4ec4100ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ec4106350 0x7f4ec4100ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48424/0 (socket says 172.21.15.12:48424) 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ec3fff700 1 -- 172.21.15.12:0/2131949551 learned_addr learned my addr 172.21.15.12:0/2131949551 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.127+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ec40ffb40 0x7f4ec4100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec96ef700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec4101530 0x7f4ec41a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec3fff700 1 -- 172.21.15.12:0/2131949551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ec40ffb40 msgr2=0x7f4ec4100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ec40ffb40 0x7f4ec4100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec3fff700 1 -- 172.21.15.12:0/2131949551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec4101530 msgr2=0x7f4ec41a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec4101530 0x7f4ec41a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec3fff700 1 -- 172.21.15.12:0/2131949551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb00095f0 con 0x7f4ec4106350 2024-05-08T23:53:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ec40ffb40 0x7f4ec4100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ec4106350 0x7f4ec4100ff0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b580 tx=0x7f4eb000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec1ffb700 1 -- 172.21.15.12:0/2131949551 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb001d070 con 0x7f4ec4106350 2024-05-08T23:53:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec1ffb700 1 -- 172.21.15.12:0/2131949551 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb0005510 con 0x7f4ec4106350 2024-05-08T23:53:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ecb152700 1 -- 172.21.15.12:0/2131949551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec41a6100 con 0x7f4ec4106350 2024-05-08T23:53:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ec1ffb700 1 -- 172.21.15.12:0/2131949551 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0004b10 con 0x7f4ec4106350 2024-05-08T23:53:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.128+0000 7f4ecb152700 1 -- 172.21.15.12:0/2131949551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec41a6580 con 0x7f4ec4106350 2024-05-08T23:53:39.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.131+0000 7f4ec1ffb700 1 -- 172.21.15.12:0/2131949551 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4eb000fbb0 con 0x7f4ec4106350 2024-05-08T23:53:39.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.132+0000 7f4ec1ffb700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4eb40a1310 0x7f4eb40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.132+0000 7f4ec1ffb700 1 -- 172.21.15.12:0/2131949551 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4eb00cd990 con 0x7f4ec4106350 2024-05-08T23:53:39.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.132+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4eb40a1310 0x7f4eb40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.132+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4eb40a1310 0x7f4eb40a37d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8011490 tx=0x7f4eb8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ed730cb80 1 Processor -- start 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ed730cb80 1 -- start start 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 0x5629637ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562963734150 con 0x5629637e3720 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562963731b10 con 0x5629637e2e00 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296372d400 con 0x5629637dd890 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ec8eee700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ec8eee700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637e31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50868/0 (socket says 172.21.15.12:50868) 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ec8eee700 1 -- 172.21.15.12:0/4184708998 learned_addr learned my addr 172.21.15.12:0/4184708998 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.133+0000 7f4ec96ef700 1 --2- 172.21.15.12:0/4184708998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 0x5629637ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.134+0000 7f4ec3fff700 1 -- 172.21.15.12:0/4184708998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 msgr2=0x5629637ddcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.134+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/4184708998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 0x5629637ddcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.134+0000 7f4ec3fff700 1 -- 172.21.15.12:0/4184708998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 msgr2=0x5629637e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.134+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/4184708998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637e31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.134+0000 7f4ec3fff700 1 -- 172.21.15.12:0/4184708998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629635ae3b0 con 0x5629637e3720 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.134+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/4184708998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637dd350 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00cb0c0 tx=0x7f4eb009a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c68a9d6e52ace00 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.134+0000 7f4eaeffd700 1 -- 172.21.15.12:0/4184708998 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb009ab10 con 0x5629637e3720 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.134+0000 7f4eaeffd700 1 -- 172.21.15.12:0/4184708998 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb009e940 con 0x5629637e3720 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.134+0000 7f4eaeffd700 1 -- 172.21.15.12:0/4184708998 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb009a000 con 0x5629637e3720 2024-05-08T23:53:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 -- 172.21.15.12:0/4184708998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 msgr2=0x5629637dd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/4184708998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637dd350 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00cb0c0 tx=0x7f4eb009a470 comp rx=0 tx=0).stop 2024-05-08T23:53:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 -- 172.21.15.12:0/4184708998 shutdown_connections 2024-05-08T23:53:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/4184708998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 0x5629637ddcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/4184708998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/4184708998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637dd350 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 -- 172.21.15.12:0/4184708998 >> 172.21.15.12:0/4184708998 conn(0x56296373b210 msgr2=0x5629637d7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 -- 172.21.15.12:0/4184708998 shutdown_connections 2024-05-08T23:53:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 -- 172.21.15.12:0/4184708998 wait complete. 2024-05-08T23:53:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 -- start start 2024-05-08T23:53:39.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.135+0000 7f4ed730cb80 1 Processor -- start 2024-05-08T23:53:39.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 0x5629637e9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637e6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629636c9560 con 0x5629637e3720 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629636c9200 con 0x5629637e2e00 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629635b06a0 con 0x5629637dd890 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ed730cb80 10 client.0 _ll_register_callbacks cb 0x5629636d2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ec96ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ec96ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48434/0 (socket says 172.21.15.12:48434) 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ec96ef700 1 -- 172.21.15.12:0/3610783480 learned_addr learned my addr 172.21.15.12:0/3610783480 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 0x5629637e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ec3fff700 1 -- 172.21.15.12:0/3610783480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 msgr2=0x5629637e9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 0x5629637e9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ec3fff700 1 -- 172.21.15.12:0/3610783480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 msgr2=0x5629637e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.136+0000 7f4ec3fff700 1 -- 172.21.15.12:0/3610783480 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec41a6580 con 0x5629637e2e00 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ec96ef700 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 0x5629637e9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637e6990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0097010 tx=0x7f4eb009a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=772fe6b376e84ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ea3fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ea3fff700 1 -- 172.21.15.12:0/3610783480 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00b7070 con 0x5629637e2e00 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ea3fff700 1 -- 172.21.15.12:0/3610783480 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4eb009ab10 con 0x5629637e2e00 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3610783480 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562963686270 con 0x5629637e2e00 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ea3fff700 1 -- 172.21.15.12:0/3610783480 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eb00a2f00 con 0x5629637e2e00 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ed730cb80 10 client.27281 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3610783480 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56296355ab60 con 0x5629637e2e00 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ea3fff700 1 -- 172.21.15.12:0/3610783480 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562963686270 con 0x5629637e2e00 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ea3fff700 1 -- 172.21.15.12:0/3610783480 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4eb00b3210 con 0x5629637e2e00 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ed730cb80 10 client.27281 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:39.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ed730cb80 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ed730cb80 20 client.27281 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5629637f6960 0x5629637f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ed730cb80 4 client.27281 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.137+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3610783480 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629637ac810 con 0x5629637f6960 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.138+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5629637f6960 0x5629637f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.138+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5629637f6960 0x5629637f8d80 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bec5d9ce63bbadc server_cookie=47b02166da026b42 in_seq=0 out_seq=0 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.138+0000 7f4ea3fff700 10 client.27281 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ea3fff700 1 -- 172.21.15.12:0/3610783480 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629637ac810 con 0x5629637f6960 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ea3fff700 10 client.27281 handle_command_reply: tid=0 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 client.27281 shutdown 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3610783480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 msgr2=0x5629637e6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637e6990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0097010 tx=0x7f4eb009a230 comp rx=0 tx=0).stop 2024-05-08T23:53:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3610783480 shutdown_connections 2024-05-08T23:53:39.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637dd890 0x5629637e9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5629637f6960 0x5629637f8d80 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/3610783480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637e6990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3610783480 >> 172.21.15.12:0/3610783480 conn(0x56296373b210 msgr2=0x5629637d75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:39.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ea3fff700 0 client.27281 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:39.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3610783480 shutdown_connections 2024-05-08T23:53:39.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3610783480 wait complete. 2024-05-08T23:53:39.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.139+0000 7f4ed730cb80 20 client.27281 trim_cache size 0 max 16384 2024-05-08T23:53:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.157+0000 7f4ed730cb80 1 Processor -- start 2024-05-08T23:53:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.157+0000 7f4ed730cb80 1 -- start start 2024-05-08T23:53:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.157+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637dd890 0x5629637dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.157+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637e2e00 0x5629637d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.157+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.157+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629637d2800 con 0x5629637e3720 2024-05-08T23:53:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.157+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629637e7680 con 0x5629637dd890 2024-05-08T23:53:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.157+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629637e31e0 con 0x5629637e2e00 2024-05-08T23:53:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec96ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec96ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48436/0 (socket says 172.21.15.12:48436) 2024-05-08T23:53:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec96ef700 1 -- 172.21.15.12:0/3157326604 learned_addr learned my addr 172.21.15.12:0/3157326604 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/3157326604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637dd890 0x5629637dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/3157326604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637e2e00 0x5629637d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec96ef700 1 -- 172.21.15.12:0/3157326604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637e2e00 msgr2=0x5629637d7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec96ef700 1 --2- 172.21.15.12:0/3157326604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637e2e00 0x5629637d7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec96ef700 1 -- 172.21.15.12:0/3157326604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637dd890 msgr2=0x5629637dd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec96ef700 1 --2- 172.21.15.12:0/3157326604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637dd890 0x5629637dd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec96ef700 1 -- 172.21.15.12:0/3157326604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629637ddf40 con 0x5629637e3720 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4ec96ef700 1 --2- 172.21.15.12:0/3157326604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e2190 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0076c0 tx=0x7f4ebc0076f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c681b040beb15040 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4eaf7fe700 1 -- 172.21.15.12:0/3157326604 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc017610 con 0x5629637e3720 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4eaf7fe700 1 -- 172.21.15.12:0/3157326604 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ebc00f040 con 0x5629637e3720 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.158+0000 7f4eaf7fe700 1 -- 172.21.15.12:0/3157326604 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc01fa70 con 0x5629637e3720 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3157326604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 msgr2=0x5629637e2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/3157326604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e2190 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0076c0 tx=0x7f4ebc0076f0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3157326604 shutdown_connections 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/3157326604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637e3720 0x5629637e2190 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/3157326604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637dd890 0x5629637dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/3157326604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637e2e00 0x5629637d7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3157326604 >> 172.21.15.12:0/3157326604 conn(0x56296373b210 msgr2=0x5629637e1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3157326604 shutdown_connections 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 -- 172.21.15.12:0/3157326604 wait complete. 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 -- start start 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.159+0000 7f4ed730cb80 1 Processor -- start 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637dd890 0x5629637ec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637eabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ed730cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637e3720 0x5629637eb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629637eb8c0 con 0x5629637dd890 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629637eba30 con 0x5629637e2e00 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ed730cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296384ce60 con 0x5629637e3720 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ed730cb80 10 client.0 _ll_register_callbacks cb 0x5629636d2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec8eee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637dd890 0x5629637ec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637eabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637eabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50902/0 (socket says 172.21.15.12:50902) 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec8eee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637dd890 0x5629637ec6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48438/0 (socket says 172.21.15.12:48438) 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec3fff700 1 -- 172.21.15.12:0/2961722347 learned_addr learned my addr 172.21.15.12:0/2961722347 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec96ef700 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637e3720 0x5629637eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec3fff700 1 -- 172.21.15.12:0/2961722347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637e3720 msgr2=0x5629637eb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:39.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637e3720 0x5629637eb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec3fff700 1 -- 172.21.15.12:0/2961722347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637dd890 msgr2=0x5629637ec6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637dd890 0x5629637ec6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec3fff700 1 -- 172.21.15.12:0/2961722347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc009e70 con 0x5629637e2e00 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637dd890 0x5629637ec6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ea3fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.160+0000 7f4ec3fff700 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637eabd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0000c00 tx=0x7f4eb000b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=309d6b959b8f4ee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ea3fff700 1 -- 172.21.15.12:0/2961722347 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb009a2a0 con 0x5629637e2e00 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ea3fff700 1 -- 172.21.15.12:0/2961722347 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4eb009ab10 con 0x5629637e2e00 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ed730cb80 1 -- 172.21.15.12:0/2961722347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4eb009e940 con 0x5629637e2e00 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ea3fff700 1 -- 172.21.15.12:0/2961722347 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eb009a470 con 0x5629637e2e00 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ed730cb80 10 client.27287 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:39.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ed730cb80 1 -- 172.21.15.12:0/2961722347 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5629636f3560 con 0x5629637e2e00 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ea3fff700 1 -- 172.21.15.12:0/2961722347 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb009e940 con 0x5629637e2e00 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ea3fff700 1 -- 172.21.15.12:0/2961722347 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4eb009bc50 con 0x5629637e2e00 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ed730cb80 10 client.27287 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ed730cb80 10 client.27287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ed730cb80 20 client.27287 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56296384e140 0x562963850530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ed730cb80 4 client.27287 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ed730cb80 1 -- 172.21.15.12:0/2961722347 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562963725ec0 con 0x56296384e140 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.161+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56296384e140 0x562963850530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ea3fff700 10 client.27287 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ec8eee700 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56296384e140 0x562963850530 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8cbf44e4905d6e server_cookie=a29cff3c58002f55 in_seq=0 out_seq=0 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ea3fff700 1 -- 172.21.15.12:0/2961722347 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562963725ec0 con 0x56296384e140 2024-05-08T23:53:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ea3fff700 10 client.27287 handle_command_reply: tid=0 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 client.27287 shutdown 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 -- 172.21.15.12:0/2961722347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 msgr2=0x5629637eabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637eabd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0000c00 tx=0x7f4eb000b5b0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 -- 172.21.15.12:0/2961722347 shutdown_connections 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629637e3720 0x5629637eb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629637dd890 0x5629637ec6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56296384e140 0x562963850530 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 --2- 172.21.15.12:0/2961722347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5629637e2e00 0x5629637eabd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 -- 172.21.15.12:0/2961722347 >> 172.21.15.12:0/2961722347 conn(0x56296373b210 msgr2=0x5629637e2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ea3fff700 0 client.27287 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 -- 172.21.15.12:0/2961722347 shutdown_connections 2024-05-08T23:53:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 1 -- 172.21.15.12:0/2961722347 wait complete. 2024-05-08T23:53:39.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.162+0000 7f4ed730cb80 20 client.27287 trim_cache size 0 max 16384 2024-05-08T23:53:39.167 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:53:39.167 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:53:39.167 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:53:39.167 INFO:teuthology.orchestra.run.smithi012.stdout: "b6593e18-0212-4d3d-a85d-9cbe6e22b015": { 2024-05-08T23:53:39.167 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:53:39.167 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "b6593e18-0212-4d3d-a85d-9cbe6e22b015", 2024-05-08T23:53:39.167 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:53:39.167 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:53:39.168 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:53:39.168 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:53:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 -- 172.21.15.12:0/2131949551 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4eb40a1310 msgr2=0x7f4eb40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4eb40a1310 0x7f4eb40a37d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8011490 tx=0x7f4eb8009260 comp rx=0 tx=0).stop 2024-05-08T23:53:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 -- 172.21.15.12:0/2131949551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ec4106350 msgr2=0x7f4ec4100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ec4106350 0x7f4ec4100ff0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b580 tx=0x7f4eb000f690 comp rx=0 tx=0).stop 2024-05-08T23:53:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 -- 172.21.15.12:0/2131949551 shutdown_connections 2024-05-08T23:53:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4eb40a1310 0x7f4eb40a37d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec4101530 0x7f4ec41a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ec40ffb40 0x7f4ec4100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 --2- 172.21.15.12:0/2131949551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ec4106350 0x7f4ec4100ff0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 -- 172.21.15.12:0/2131949551 >> 172.21.15.12:0/2131949551 conn(0x7f4ec4075070 msgr2=0x7f4ec40fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 -- 172.21.15.12:0/2131949551 shutdown_connections 2024-05-08T23:53:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:53:39.163+0000 7f4ecb152700 1 -- 172.21.15.12:0/2131949551 wait complete. 2024-05-08T23:53:39.171 INFO:tasks.cephfs.filesystem:scrub status for tag:b6593e18-0212-4d3d-a85d-9cbe6e22b015 - {'path': '/', 'tag': 'b6593e18-0212-4d3d-a85d-9cbe6e22b015', 'options': 'recursive,force'} 2024-05-08T23:53:39.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:39 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:53:39.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:39 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:39.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:39 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:53:39.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:39 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:39.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:39 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:53:39.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:39 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:39.871 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465685 9.35 MB/sec execute 215 sec latency 830.795 ms 2024-05-08T23:53:40.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:40 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:53:40.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:40 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:40.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:40 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:53:40.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:40 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:53:40.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:40 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:40.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:40 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:40.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:40 smithi012 ceph-mon[42503]: pgmap v226: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-05-08T23:53:40.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:40 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:53:40.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:40 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:40.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:40 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:53:40.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:40 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:53:40.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:40 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:40.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:40 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:40.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:40 smithi204 ceph-mon[47373]: pgmap v226: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-05-08T23:53:40.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:40 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:53:40.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:40 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:40.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:40 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:53:40.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:40 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:53:40.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:40 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:40.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:40 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:53:40.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:40 smithi160 ceph-mon[46152]: pgmap v226: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-05-08T23:53:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.552+0000 7f780c51c700 1 -- 172.21.15.12:0/372128195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78041010f0 msgr2=0x7f78041014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.552+0000 7f780c51c700 1 --2- 172.21.15.12:0/372128195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78041010f0 0x7f78041014d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f77f400b0a0 tx=0x7f77f400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.552+0000 7f780c51c700 1 -- 172.21.15.12:0/372128195 shutdown_connections 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.552+0000 7f780c51c700 1 --2- 172.21.15.12:0/372128195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7804106210 0x7f780410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.552+0000 7f780c51c700 1 --2- 172.21.15.12:0/372128195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78041010f0 0x7f78041014d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.552+0000 7f780c51c700 1 --2- 172.21.15.12:0/372128195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7804101aa0 0x7f7804105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.552+0000 7f780c51c700 1 -- 172.21.15.12:0/372128195 >> 172.21.15.12:0/372128195 conn(0x7f78040fc9d0 msgr2=0x7f78040fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.552+0000 7f780c51c700 1 -- 172.21.15.12:0/372128195 shutdown_connections 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.552+0000 7f780c51c700 1 -- 172.21.15.12:0/372128195 wait complete. 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780c51c700 1 Processor -- start 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780c51c700 1 -- start start 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780c51c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78041010f0 0x7f78040732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780c51c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7804106210 0x7f78040737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780c51c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804073d20 0x7f78041a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780c51c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78041a5c70 con 0x7f7804073d20 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780c51c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78041a5de0 con 0x7f7804106210 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780c51c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78041a5fa0 con 0x7f78041010f0 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780aab9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804073d20 0x7f78041a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780aab9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804073d20 0x7f78041a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48462/0 (socket says 172.21.15.12:48462) 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.553+0000 7f780aab9700 1 -- 172.21.15.12:0/2367292876 learned_addr learned my addr 172.21.15.12:0/2367292876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f7809ab7700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7804106210 0x7f78040737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f780a2b8700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78041010f0 0x7f78040732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f780aab9700 1 -- 172.21.15.12:0/2367292876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78041010f0 msgr2=0x7f78040732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f780aab9700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78041010f0 0x7f78040732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f780aab9700 1 -- 172.21.15.12:0/2367292876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7804106210 msgr2=0x7f78040737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f780aab9700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7804106210 0x7f78040737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f780aab9700 1 -- 172.21.15.12:0/2367292876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4009d00 con 0x7f7804073d20 2024-05-08T23:53:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f7809ab7700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7804106210 0x7f78040737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f780aab9700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804073d20 0x7f78041a56a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e940 tx=0x7f77fc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f78037fe700 1 -- 172.21.15.12:0/2367292876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0097b0 con 0x7f7804073d20 2024-05-08T23:53:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f78037fe700 1 -- 172.21.15.12:0/2367292876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77fc004cf0 con 0x7f7804073d20 2024-05-08T23:53:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f780c51c700 1 -- 172.21.15.12:0/2367292876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78041a6280 con 0x7f7804073d20 2024-05-08T23:53:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f78037fe700 1 -- 172.21.15.12:0/2367292876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0055d0 con 0x7f7804073d20 2024-05-08T23:53:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.554+0000 7f780c51c700 1 -- 172.21.15.12:0/2367292876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78041a6740 con 0x7f7804073d20 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.556+0000 7f78037fe700 1 -- 172.21.15.12:0/2367292876 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f77fc00cc60 con 0x7f7804073d20 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.557+0000 7f78037fe700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f77ec0a1290 0x7f77ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.557+0000 7f78037fe700 1 -- 172.21.15.12:0/2367292876 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f77fc014070 con 0x7f7804073d20 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.557+0000 7f780a2b8700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f77ec0a1290 0x7f77ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.558+0000 7f780a2b8700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f77ec0a1290 0x7f77ec0a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f77f4000f80 tx=0x7f77f400b700 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.558+0000 7f78186d6b80 1 Processor -- start 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.558+0000 7f78186d6b80 1 -- start start 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.558+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e105950 0x55d03e105d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.558+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e106270 0x55d03e0fff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.558+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 0x55d03e1008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.558+0000 7f78186d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e0570f0 con 0x55d03e105950 2024-05-08T23:53:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.558+0000 7f78186d6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e054ab0 con 0x55d03e106270 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.558+0000 7f78186d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e0503a0 con 0x55d03e100480 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780a2b8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e105950 0x55d03e105d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780a2b8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e105950 0x55d03e105d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48464/0 (socket says 172.21.15.12:48464) 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780a2b8700 1 -- 172.21.15.12:0/1948994313 learned_addr learned my addr 172.21.15.12:0/1948994313 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f7809ab7700 1 --2- 172.21.15.12:0/1948994313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e106270 0x55d03e0fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780aab9700 1 --2- 172.21.15.12:0/1948994313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 0x55d03e1008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780a2b8700 1 -- 172.21.15.12:0/1948994313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 msgr2=0x55d03e1008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780a2b8700 1 --2- 172.21.15.12:0/1948994313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 0x55d03e1008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780a2b8700 1 -- 172.21.15.12:0/1948994313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e106270 msgr2=0x55d03e0fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780a2b8700 1 --2- 172.21.15.12:0/1948994313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e106270 0x55d03e0fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780a2b8700 1 -- 172.21.15.12:0/1948994313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d03ded1350 con 0x55d03e105950 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780aab9700 1 --2- 172.21.15.12:0/1948994313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 0x55d03e1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f780a2b8700 1 --2- 172.21.15.12:0/1948994313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e105950 0x55d03e105d30 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f77f4007d00 tx=0x7f77f401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f19f8c5c7f1767 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f7800ff9700 1 -- 172.21.15.12:0/1948994313 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f402c070 con 0x55d03e105950 2024-05-08T23:53:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f7800ff9700 1 -- 172.21.15.12:0/1948994313 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77f4007d30 con 0x55d03e105950 2024-05-08T23:53:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.559+0000 7f7800ff9700 1 -- 172.21.15.12:0/1948994313 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f40052a0 con 0x55d03e105950 2024-05-08T23:53:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.560+0000 7f78186d6b80 1 -- 172.21.15.12:0/1948994313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e105950 msgr2=0x55d03e105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.560+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1948994313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e105950 0x55d03e105d30 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f77f4007d00 tx=0x7f77f401b000 comp rx=0 tx=0).stop 2024-05-08T23:53:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.560+0000 7f78186d6b80 1 -- 172.21.15.12:0/1948994313 shutdown_connections 2024-05-08T23:53:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.560+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1948994313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 0x55d03e1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.560+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1948994313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e105950 0x55d03e105d30 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.560+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1948994313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e106270 0x55d03e0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.560+0000 7f78186d6b80 1 -- 172.21.15.12:0/1948994313 >> 172.21.15.12:0/1948994313 conn(0x55d03e05e1b0 msgr2=0x55d03e0fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.560+0000 7f78186d6b80 1 -- 172.21.15.12:0/1948994313 shutdown_connections 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.560+0000 7f78186d6b80 1 -- 172.21.15.12:0/1948994313 wait complete. 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.561+0000 7f78186d6b80 1 -- start start 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.561+0000 7f78186d6b80 1 Processor -- start 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.561+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 0x55d03e10c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.561+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 0x55d03e1094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.561+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.561+0000 7f78186d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03dfec500 con 0x55d03e106270 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.561+0000 7f78186d6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03dfec1a0 con 0x55d03e100480 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.561+0000 7f78186d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03ded3640 con 0x55d03e105950 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.561+0000 7f78186d6b80 10 client.0 _ll_register_callbacks cb 0x55d03dff5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f780aab9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f780aab9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e109ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48474/0 (socket says 172.21.15.12:48474) 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f780aab9700 1 -- 172.21.15.12:0/1210829610 learned_addr learned my addr 172.21.15.12:0/1210829610 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f7809ab7700 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 0x55d03e1094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f780a2b8700 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 0x55d03e10c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f780aab9700 1 -- 172.21.15.12:0/1210829610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 msgr2=0x55d03e1094b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f780aab9700 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 0x55d03e1094b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f780aab9700 1 -- 172.21.15.12:0/1210829610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 msgr2=0x55d03e10c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f780aab9700 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 0x55d03e10c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f780aab9700 1 -- 172.21.15.12:0/1210829610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780410e600 con 0x55d03e106270 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f780aab9700 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e109ba0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009fd0 tx=0x7f77fc099a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8927c0f8fd3f8b35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f77e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1210829610 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc09d070 con 0x55d03e106270 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1210829610 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f77fc09a080 con 0x55d03e106270 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f78186d6b80 1 -- 172.21.15.12:0/1210829610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d03dfa9210 con 0x55d03e106270 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1210829610 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77fc09deb0 con 0x55d03e106270 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f78186d6b80 10 client.17511 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.562+0000 7f78186d6b80 1 -- 172.21.15.12:0/1210829610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d03de7db00 con 0x55d03e106270 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.563+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1210829610 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d03dfa9210 con 0x55d03e106270 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.563+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1210829610 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f77fc099570 con 0x55d03e106270 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.563+0000 7f78186d6b80 10 client.17511 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.563+0000 7f78186d6b80 10 client.17511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.563+0000 7f78186d6b80 20 client.17511 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.563+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d03e119b60 0x55d03e11bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.563+0000 7f78186d6b80 4 client.17511 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.563+0000 7f78186d6b80 1 -- 172.21.15.12:0/1210829610 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d03e0cf7b0 con 0x55d03e119b60 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.563+0000 7f780a2b8700 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d03e119b60 0x55d03e11bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f780a2b8700 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d03e119b60 0x55d03e11bf80 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ce96a0a4cc55f9 server_cookie=843274b808523855 in_seq=0 out_seq=0 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f77e9ffb700 10 client.17511 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1210829610 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d03e0cf7b0 con 0x55d03e119b60 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f77e9ffb700 10 client.17511 handle_command_reply: tid=0 2024-05-08T23:53:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f78186d6b80 1 client.17511 shutdown 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f78186d6b80 1 -- 172.21.15.12:0/1210829610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 msgr2=0x55d03e109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e109ba0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009fd0 tx=0x7f77fc099a60 comp rx=0 tx=0).stop 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f78186d6b80 1 -- 172.21.15.12:0/1210829610 shutdown_connections 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e109ba0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d03e119b60 0x55d03e11bf80 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 0x55d03e10c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f77e9ffb700 0 client.17511 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1210829610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 0x55d03e1094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f78186d6b80 1 -- 172.21.15.12:0/1210829610 >> 172.21.15.12:0/1210829610 conn(0x55d03e05e1b0 msgr2=0x55d03e0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.564+0000 7f78186d6b80 1 -- 172.21.15.12:0/1210829610 shutdown_connections 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.565+0000 7f78186d6b80 1 -- 172.21.15.12:0/1210829610 wait complete. 2024-05-08T23:53:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.565+0000 7f78186d6b80 20 client.17511 trim_cache size 0 max 16384 2024-05-08T23:53:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.580+0000 7f78186d6b80 1 Processor -- start 2024-05-08T23:53:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.580+0000 7f78186d6b80 1 -- start start 2024-05-08T23:53:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 0x55d03e0fa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e105950 0x55d03e10bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e110be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f78186d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03dfa9210 con 0x55d03e106270 2024-05-08T23:53:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f78186d6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e0f5330 con 0x55d03e105950 2024-05-08T23:53:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f78186d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e1002e0 con 0x55d03e100480 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f780aab9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e110be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f780aab9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e110be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48480/0 (socket says 172.21.15.12:48480) 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f780aab9700 1 -- 172.21.15.12:0/592013753 learned_addr learned my addr 172.21.15.12:0/592013753 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f7809ab7700 1 --2- 172.21.15.12:0/592013753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e105950 0x55d03e10bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f780a2b8700 1 --2- 172.21.15.12:0/592013753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 0x55d03e0fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f780aab9700 1 -- 172.21.15.12:0/592013753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 msgr2=0x55d03e0fa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f780aab9700 1 --2- 172.21.15.12:0/592013753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 0x55d03e0fa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f780aab9700 1 -- 172.21.15.12:0/592013753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e105950 msgr2=0x55d03e10bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f780aab9700 1 --2- 172.21.15.12:0/592013753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e105950 0x55d03e10bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f780aab9700 1 -- 172.21.15.12:0/592013753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d03e016500 con 0x55d03e106270 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.581+0000 7f780aab9700 1 --2- 172.21.15.12:0/592013753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e110be0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f77fc096580 tx=0x7f77fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8358d4c26c8f1859 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78017fa700 1 -- 172.21.15.12:0/592013753 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc09d070 con 0x55d03e106270 2024-05-08T23:53:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78017fa700 1 -- 172.21.15.12:0/592013753 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77fc09a5a0 con 0x55d03e106270 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78017fa700 1 -- 172.21.15.12:0/592013753 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc099570 con 0x55d03e106270 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78186d6b80 1 -- 172.21.15.12:0/592013753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 msgr2=0x55d03e110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78186d6b80 1 --2- 172.21.15.12:0/592013753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e110be0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f77fc096580 tx=0x7f77fc017000 comp rx=0 tx=0).stop 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78186d6b80 1 -- 172.21.15.12:0/592013753 shutdown_connections 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78186d6b80 1 --2- 172.21.15.12:0/592013753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e110be0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78186d6b80 1 --2- 172.21.15.12:0/592013753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e100480 0x55d03e0fa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78186d6b80 1 --2- 172.21.15.12:0/592013753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e105950 0x55d03e10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78186d6b80 1 -- 172.21.15.12:0/592013753 >> 172.21.15.12:0/592013753 conn(0x55d03e05e1b0 msgr2=0x55d03e107d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78186d6b80 1 -- 172.21.15.12:0/592013753 shutdown_connections 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.582+0000 7f78186d6b80 1 -- 172.21.15.12:0/592013753 wait complete. 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f78186d6b80 1 -- start start 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f78186d6b80 1 Processor -- start 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 0x55d03e10eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 0x55d03e10f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f78186d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e16f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f78186d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e16f5a0 con 0x55d03e106270 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f78186d6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e16f6e0 con 0x55d03e100480 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f78186d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e16f870 con 0x55d03e105950 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f78186d6b80 10 client.0 _ll_register_callbacks cb 0x55d03dff5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f780aab9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e16f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f780aab9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e16f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48492/0 (socket says 172.21.15.12:48492) 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f780aab9700 1 -- 172.21.15.12:0/1672906782 learned_addr learned my addr 172.21.15.12:0/1672906782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.583+0000 7f780a2b8700 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 0x55d03e10eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f7809ab7700 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 0x55d03e10f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f780aab9700 1 -- 172.21.15.12:0/1672906782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 msgr2=0x55d03e10f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f780aab9700 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 0x55d03e10f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f780aab9700 1 -- 172.21.15.12:0/1672906782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 msgr2=0x55d03e10eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f780aab9700 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 0x55d03e10eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f780aab9700 1 -- 172.21.15.12:0/1672906782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d03de7db00 con 0x55d03e106270 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f780a2b8700 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 0x55d03e10eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f7809ab7700 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 0x55d03e10f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f780aab9700 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e16f060 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f77fc099d80 tx=0x7f77fc0ca7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f9e07a781d615b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f77e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1672906782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc09d050 con 0x55d03e106270 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1672906782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f77fc0a2500 con 0x55d03e106270 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f78186d6b80 1 -- 172.21.15.12:0/1672906782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77f4007d30 con 0x55d03e106270 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1672906782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77fc09a780 con 0x55d03e106270 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f78186d6b80 10 client.17517 fetch_fsmap learned FSMap version 36 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f78186d6b80 1 -- 172.21.15.12:0/1672906782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d03ded5c70 con 0x55d03e106270 2024-05-08T23:53:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.584+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1672906782 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77f4007d30 con 0x55d03e106270 2024-05-08T23:53:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.585+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1672906782 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f77fc098fe0 con 0x55d03e106270 2024-05-08T23:53:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.585+0000 7f78186d6b80 10 client.17517 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:53:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.585+0000 7f78186d6b80 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.585+0000 7f78186d6b80 20 client.17517 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.585+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d03e170b70 0x55d03e172f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.585+0000 7f78186d6b80 4 client.17517 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:53:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.585+0000 7f78186d6b80 1 -- 172.21.15.12:0/1672906782 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d03e048e60 con 0x55d03e170b70 2024-05-08T23:53:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.585+0000 7f780a2b8700 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d03e170b70 0x55d03e172f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.585+0000 7f77e9ffb700 10 client.17517 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.585+0000 7f780a2b8700 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d03e170b70 0x55d03e172f60 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f355e240499e33e2 server_cookie=1b1ef722af6ab697 in_seq=0 out_seq=0 2024-05-08T23:53:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.644+0000 7f77e9ffb700 1 -- 172.21.15.12:0/1672906782 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d03e048e60 con 0x55d03e170b70 2024-05-08T23:53:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.644+0000 7f77e9ffb700 10 client.17517 handle_command_reply: tid=0 2024-05-08T23:53:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.644+0000 7f78186d6b80 1 client.17517 shutdown 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 1 -- 172.21.15.12:0/1672906782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 msgr2=0x55d03e16f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e16f060 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f77fc099d80 tx=0x7f77fc0ca7a0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 1 -- 172.21.15.12:0/1672906782 shutdown_connections 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d03e106270 0x55d03e16f060 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d03e170b70 0x55d03e172f60 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d03e100480 0x55d03e10eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 1 --2- 172.21.15.12:0/1672906782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d03e105950 0x55d03e10f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f77e9ffb700 0 client.17517 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 1 -- 172.21.15.12:0/1672906782 >> 172.21.15.12:0/1672906782 conn(0x55d03e05e1b0 msgr2=0x55d03e107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 1 -- 172.21.15.12:0/1672906782 shutdown_connections 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 1 -- 172.21.15.12:0/1672906782 wait complete. 2024-05-08T23:53:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.645+0000 7f78186d6b80 20 client.17517 trim_cache size 0 max 16384 2024-05-08T23:53:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:53:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:53:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:53:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 -- 172.21.15.12:0/2367292876 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f77ec0a1290 msgr2=0x7f77ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f77ec0a1290 0x7f77ec0a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f77f4000f80 tx=0x7f77f400b700 comp rx=0 tx=0).stop 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 -- 172.21.15.12:0/2367292876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804073d20 msgr2=0x7f78041a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804073d20 0x7f78041a56a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e940 tx=0x7f77fc00ed00 comp rx=0 tx=0).stop 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 -- 172.21.15.12:0/2367292876 shutdown_connections 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f77ec0a1290 0x7f77ec0a3750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804073d20 0x7f78041a56a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78041010f0 0x7f78040732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 --2- 172.21.15.12:0/2367292876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7804106210 0x7f78040737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 -- 172.21.15.12:0/2367292876 >> 172.21.15.12:0/2367292876 conn(0x7f78040fc9d0 msgr2=0x7f78040fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 -- 172.21.15.12:0/2367292876 shutdown_connections 2024-05-08T23:53:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:40.646+0000 7f780c51c700 1 -- 172.21.15.12:0/2367292876 wait complete. 2024-05-08T23:53:40.871 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465772 9.30 MB/sec execute 216 sec latency 1551.889 ms 2024-05-08T23:53:41.871 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465772 9.26 MB/sec execute 217 sec latency 1137.561 ms 2024-05-08T23:53:42.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:42 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:42.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:42 smithi012 ceph-mon[42503]: pgmap v227: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 2.5 MiB/s wr, 198 op/s 2024-05-08T23:53:42.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:42 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:42.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:42 smithi204 ceph-mon[47373]: pgmap v227: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 2.5 MiB/s wr, 198 op/s 2024-05-08T23:53:42.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:42 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:42.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:42 smithi160 ceph-mon[46152]: pgmap v227: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 2.5 MiB/s wr, 198 op/s 2024-05-08T23:53:42.871 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465772 9.22 MB/sec execute 218 sec latency 2137.671 ms 2024-05-08T23:53:43.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465820 9.18 MB/sec execute 219 sec latency 2620.011 ms 2024-05-08T23:53:44.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:44 smithi204 ceph-mon[47373]: pgmap v228: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-05-08T23:53:44.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:44 smithi160 ceph-mon[46152]: pgmap v228: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-05-08T23:53:44.871 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465820 9.14 MB/sec execute 220 sec latency 1478.976 ms 2024-05-08T23:53:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:44 smithi012 ceph-mon[42503]: pgmap v228: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-05-08T23:53:45.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465820 9.09 MB/sec execute 221 sec latency 2479.087 ms 2024-05-08T23:53:46.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:46 smithi204 ceph-mon[47373]: pgmap v229: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-05-08T23:53:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:46 smithi160 ceph-mon[46152]: pgmap v229: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-05-08T23:53:46.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465820 9.05 MB/sec execute 222 sec latency 3479.140 ms 2024-05-08T23:53:46.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:46 smithi012 ceph-mon[42503]: pgmap v229: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-05-08T23:53:47.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465820 9.01 MB/sec execute 223 sec latency 4479.234 ms 2024-05-08T23:53:48.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:48 smithi204 ceph-mon[47373]: pgmap v230: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-05-08T23:53:48.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:48 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:53:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:48 smithi160 ceph-mon[46152]: pgmap v230: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-05-08T23:53:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:48 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:53:48.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465832 8.97 MB/sec execute 224 sec latency 4966.476 ms 2024-05-08T23:53:48.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:48 smithi012 ceph-mon[42503]: pgmap v230: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-05-08T23:53:48.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:48 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:53:49.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:49 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:53:49.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:49 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:53:49.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:49 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:53:49.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:49 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:53:49.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465832 8.93 MB/sec execute 225 sec latency 1489.307 ms 2024-05-08T23:53:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:49 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:53:49.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:49 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:53:50.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:50 smithi204 ceph-mon[47373]: pgmap v231: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-05-08T23:53:50.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:50 smithi160 ceph-mon[46152]: pgmap v231: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-05-08T23:53:50.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465832 8.89 MB/sec execute 226 sec latency 2489.410 ms 2024-05-08T23:53:50.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:50 smithi012 ceph-mon[42503]: pgmap v231: 97 pgs: 97 active+clean; 129 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-05-08T23:53:51.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465832 8.85 MB/sec execute 227 sec latency 3489.519 ms 2024-05-08T23:53:51.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:51 smithi012 ceph-mon[42503]: pgmap v232: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-05-08T23:53:52.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:51 smithi204 ceph-mon[47373]: pgmap v232: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-05-08T23:53:52.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:51 smithi160 ceph-mon[46152]: pgmap v232: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-05-08T23:53:52.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465832 8.81 MB/sec execute 228 sec latency 4489.567 ms 2024-05-08T23:53:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.780+0000 7f101c0cb700 1 -- 172.21.15.12:0/2716373024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1014101aa0 msgr2=0x7f1014105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.780+0000 7f101c0cb700 1 --2- 172.21.15.12:0/2716373024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1014101aa0 0x7f1014105ae0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1004009990 tx=0x7f1004009ca0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.780+0000 7f101c0cb700 1 -- 172.21.15.12:0/2716373024 shutdown_connections 2024-05-08T23:53:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.780+0000 7f101c0cb700 1 --2- 172.21.15.12:0/2716373024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1014106210 0x7f101410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.780+0000 7f101c0cb700 1 --2- 172.21.15.12:0/2716373024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10141010f0 0x7f10141014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.780+0000 7f101c0cb700 1 --2- 172.21.15.12:0/2716373024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1014101aa0 0x7f1014105ae0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.780+0000 7f101c0cb700 1 -- 172.21.15.12:0/2716373024 >> 172.21.15.12:0/2716373024 conn(0x7f10140fc970 msgr2=0x7f10140fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.780+0000 7f101c0cb700 1 -- 172.21.15.12:0/2716373024 shutdown_connections 2024-05-08T23:53:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.780+0000 7f101c0cb700 1 -- 172.21.15.12:0/2716373024 wait complete. 2024-05-08T23:53:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f101c0cb700 1 Processor -- start 2024-05-08T23:53:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f101c0cb700 1 -- start start 2024-05-08T23:53:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f101c0cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10141010f0 0x7f1014197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f101c0cb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1014106210 0x7f1014197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f101c0cb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101419bc20 0x7f101419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f101c0cb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101419c6a0 con 0x7f10141010f0 2024-05-08T23:53:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f101c0cb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101419c810 con 0x7f101419bc20 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f101c0cb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101419c9d0 con 0x7f1014106210 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f1019e67700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10141010f0 0x7f1014197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f1019e67700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10141010f0 0x7f1014197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44136/0 (socket says 172.21.15.12:44136) 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f1019e67700 1 -- 172.21.15.12:0/3103894502 learned_addr learned my addr 172.21.15.12:0/3103894502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f1019666700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1014106210 0x7f1014197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.781+0000 7f101a668700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101419bc20 0x7f101419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f1019e67700 1 -- 172.21.15.12:0/3103894502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1014106210 msgr2=0x7f1014197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f1019e67700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1014106210 0x7f1014197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f1019e67700 1 -- 172.21.15.12:0/3103894502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101419bc20 msgr2=0x7f101419c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f1019e67700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101419bc20 0x7f101419c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f1019e67700 1 -- 172.21.15.12:0/3103894502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010009520 con 0x7f10141010f0 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f1019e67700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10141010f0 0x7f1014197340 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f101000eb30 tx=0x7f101000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f100affd700 1 -- 172.21.15.12:0/3103894502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010009880 con 0x7f10141010f0 2024-05-08T23:53:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f101c0cb700 1 -- 172.21.15.12:0/3103894502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10040095f0 con 0x7f10141010f0 2024-05-08T23:53:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f100affd700 1 -- 172.21.15.12:0/3103894502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10100044e0 con 0x7f10141010f0 2024-05-08T23:53:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f100affd700 1 -- 172.21.15.12:0/3103894502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010003d80 con 0x7f10141010f0 2024-05-08T23:53:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.782+0000 7f101c0cb700 1 -- 172.21.15.12:0/3103894502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10141a4270 con 0x7f10141010f0 2024-05-08T23:53:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.784+0000 7f100affd700 1 -- 172.21.15.12:0/3103894502 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1010004000 con 0x7f10141010f0 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.785+0000 7f100affd700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10000a1290 0x7f10000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.785+0000 7f100affd700 1 -- 172.21.15.12:0/3103894502 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1010014070 con 0x7f10141010f0 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.785+0000 7f1019666700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10000a1290 0x7f10000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.785+0000 7f1019666700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10000a1290 0x7f10000a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f100400b2e0 tx=0x7f1004005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.786+0000 7f1028285b80 1 Processor -- start 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.786+0000 7f1028285b80 1 -- start start 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.786+0000 7f1028285b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 0x55883fc16d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.786+0000 7f1028285b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 0x55883fc10f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.786+0000 7f1028285b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.786+0000 7f1028285b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fb680f0 con 0x55883fc17270 2024-05-08T23:53:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.786+0000 7f1028285b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fb65ab0 con 0x55883fc11480 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.786+0000 7f1028285b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fb613a0 con 0x55883fc16950 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f101a668700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f1019666700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 0x55883fc10f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f101a668700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc118a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49228/0 (socket says 172.21.15.12:49228) 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f101a668700 1 -- 172.21.15.12:0/3009939271 learned_addr learned my addr 172.21.15.12:0/3009939271 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f1019e67700 1 --2- 172.21.15.12:0/3009939271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 0x55883fc16d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f101a668700 1 -- 172.21.15.12:0/3009939271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 msgr2=0x55883fc16d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f101a668700 1 --2- 172.21.15.12:0/3009939271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 0x55883fc16d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f101a668700 1 -- 172.21.15.12:0/3009939271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 msgr2=0x55883fc10f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f101a668700 1 --2- 172.21.15.12:0/3009939271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 0x55883fc10f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f101a668700 1 -- 172.21.15.12:0/3009939271 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55883f9e2350 con 0x55883fc11480 2024-05-08T23:53:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f1019e67700 1 --2- 172.21.15.12:0/3009939271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 0x55883fc16d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f101a668700 1 --2- 172.21.15.12:0/3009939271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc118a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f100c011680 tx=0x7f100c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=473d434ff128b4cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f0ffffff700 1 -- 172.21.15.12:0/3009939271 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c00e040 con 0x55883fc11480 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f0ffffff700 1 -- 172.21.15.12:0/3009939271 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100c00f040 con 0x55883fc11480 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.787+0000 7f0ffffff700 1 -- 172.21.15.12:0/3009939271 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c004860 con 0x55883fc11480 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.788+0000 7f1028285b80 1 -- 172.21.15.12:0/3009939271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 msgr2=0x55883fc118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.788+0000 7f1028285b80 1 --2- 172.21.15.12:0/3009939271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc118a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f100c011680 tx=0x7f100c009920 comp rx=0 tx=0).stop 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.788+0000 7f1028285b80 1 -- 172.21.15.12:0/3009939271 shutdown_connections 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.788+0000 7f1028285b80 1 --2- 172.21.15.12:0/3009939271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc118a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.788+0000 7f1028285b80 1 --2- 172.21.15.12:0/3009939271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 0x55883fc16d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.788+0000 7f1028285b80 1 --2- 172.21.15.12:0/3009939271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 0x55883fc10f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.788+0000 7f1028285b80 1 -- 172.21.15.12:0/3009939271 >> 172.21.15.12:0/3009939271 conn(0x55883fb6f1b0 msgr2=0x55883fc0b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.788+0000 7f1028285b80 1 -- 172.21.15.12:0/3009939271 shutdown_connections 2024-05-08T23:53:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.788+0000 7f1028285b80 1 -- 172.21.15.12:0/3009939271 wait complete. 2024-05-08T23:53:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.789+0000 7f1028285b80 1 -- start start 2024-05-08T23:53:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.789+0000 7f1028285b80 1 Processor -- start 2024-05-08T23:53:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.789+0000 7f1028285b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc11480 0x55883fc1d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.789+0000 7f1028285b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc1a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.789+0000 7f1028285b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc17270 0x55883fc1ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.789+0000 7f1028285b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fafd500 con 0x55883fc16950 2024-05-08T23:53:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.789+0000 7f1028285b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fafd1a0 con 0x55883fc17270 2024-05-08T23:53:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.789+0000 7f1028285b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883f9e4640 con 0x55883fc11480 2024-05-08T23:53:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1028285b80 10 client.0 _ll_register_callbacks cb 0x55883fb06ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019666700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc1a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f101a668700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc17270 0x55883fc1ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019e67700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc11480 0x55883fc1d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019666700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc1a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44150/0 (socket says 172.21.15.12:44150) 2024-05-08T23:53:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019666700 1 -- 172.21.15.12:0/2877155410 learned_addr learned my addr 172.21.15.12:0/2877155410 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019666700 1 -- 172.21.15.12:0/2877155410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc11480 msgr2=0x55883fc1d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019666700 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc11480 0x55883fc1d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019666700 1 -- 172.21.15.12:0/2877155410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc17270 msgr2=0x55883fc1ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019666700 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc17270 0x55883fc1ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019666700 1 -- 172.21.15.12:0/2877155410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c011330 con 0x55883fc16950 2024-05-08T23:53:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019e67700 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc11480 0x55883fc1d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1019666700 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc1a550 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1004009990 tx=0x7f1004009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62b521e738eb414 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f0ffd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2877155410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004004c40 con 0x55883fc16950 2024-05-08T23:53:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f1028285b80 1 -- 172.21.15.12:0/2877155410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55883faba210 con 0x55883fc16950 2024-05-08T23:53:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.790+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2877155410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1004004da0 con 0x55883fc16950 2024-05-08T23:53:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2877155410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f100400f460 con 0x55883fc16950 2024-05-08T23:53:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f1028285b80 10 client.17529 fetch_fsmap learned FSMap version 38 2024-05-08T23:53:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f1028285b80 1 -- 172.21.15.12:0/2877155410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55883f98eb00 con 0x55883fc16950 2024-05-08T23:53:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2877155410 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55883faba210 con 0x55883fc16950 2024-05-08T23:53:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2877155410 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f100402f070 con 0x55883fc16950 2024-05-08T23:53:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f1028285b80 10 client.17529 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:53:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f1028285b80 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f1028285b80 20 client.17529 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f1028285b80 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55883fc2ab70 0x55883fc2cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f1028285b80 4 client.17529 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:53:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f1028285b80 1 -- 172.21.15.12:0/2877155410 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55883fbe07b0 con 0x55883fc2ab70 2024-05-08T23:53:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.791+0000 7f1019e67700 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55883fc2ab70 0x55883fc2cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.792+0000 7f1019e67700 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55883fc2ab70 0x55883fc2cf90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167edc986b8e264f server_cookie=9a29fb528bc04706 in_seq=0 out_seq=0 2024-05-08T23:53:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.792+0000 7f0ffd7fa700 10 client.17529 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.792+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2877155410 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55883fbe07b0 con 0x55883fc2ab70 2024-05-08T23:53:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.792+0000 7f0ffd7fa700 10 client.17529 handle_command_reply: tid=0 2024-05-08T23:53:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.792+0000 7f1028285b80 1 client.17529 shutdown 2024-05-08T23:53:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 1 -- 172.21.15.12:0/2877155410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 msgr2=0x55883fc1a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc1a550 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1004009990 tx=0x7f1004009ca0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 1 -- 172.21.15.12:0/2877155410 shutdown_connections 2024-05-08T23:53:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55883fc2ab70 0x55883fc2cf90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc17270 0x55883fc1ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f0ffd7fa700 0 client.17529 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc11480 0x55883fc1d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 1 --2- 172.21.15.12:0/2877155410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc1a550 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 1 -- 172.21.15.12:0/2877155410 >> 172.21.15.12:0/2877155410 conn(0x55883fb6f1b0 msgr2=0x55883fc154a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 1 -- 172.21.15.12:0/2877155410 shutdown_connections 2024-05-08T23:53:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 1 -- 172.21.15.12:0/2877155410 wait complete. 2024-05-08T23:53:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.793+0000 7f1028285b80 20 client.17529 trim_cache size 0 max 16384 2024-05-08T23:53:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f1028285b80 1 Processor -- start 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f1028285b80 1 -- start start 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f1028285b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc16d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f1028285b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 0x55883fc0b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f1028285b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 0x55883fc1cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f1028285b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fc28210 con 0x55883fc17270 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f1028285b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fc27670 con 0x55883fc11480 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f1028285b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fc0c330 con 0x55883fc16950 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f101a668700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 0x55883fc1cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f1019666700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 0x55883fc0b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f101a668700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 0x55883fc1cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44160/0 (socket says 172.21.15.12:44160) 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f101a668700 1 -- 172.21.15.12:0/2719947785 learned_addr learned my addr 172.21.15.12:0/2719947785 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.810+0000 7f1019e67700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc16d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.811+0000 7f1019666700 1 -- 172.21.15.12:0/2719947785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 msgr2=0x55883fc16d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.811+0000 7f1019666700 1 --2- 172.21.15.12:0/2719947785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc16d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.811+0000 7f1019666700 1 -- 172.21.15.12:0/2719947785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 msgr2=0x55883fc1cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.811+0000 7f1019666700 1 --2- 172.21.15.12:0/2719947785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 0x55883fc1cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.811+0000 7f1019666700 1 -- 172.21.15.12:0/2719947785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55883fb27500 con 0x55883fc16950 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.811+0000 7f101a668700 1 --2- 172.21.15.12:0/2719947785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 0x55883fc1cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.811+0000 7f1019666700 1 --2- 172.21.15.12:0/2719947785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 0x55883fc0b910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1004000c00 tx=0x7f100400f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0037d6c2a344d8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.811+0000 7f1008ff9700 1 -- 172.21.15.12:0/2719947785 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10040055c0 con 0x55883fc16950 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.811+0000 7f1008ff9700 1 -- 172.21.15.12:0/2719947785 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1004028070 con 0x55883fc16950 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.811+0000 7f1008ff9700 1 -- 172.21.15.12:0/2719947785 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100400fc70 con 0x55883fc16950 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 -- 172.21.15.12:0/2719947785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 msgr2=0x55883fc0b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 --2- 172.21.15.12:0/2719947785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 0x55883fc0b910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1004000c00 tx=0x7f100400f460 comp rx=0 tx=0).stop 2024-05-08T23:53:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 -- 172.21.15.12:0/2719947785 shutdown_connections 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 --2- 172.21.15.12:0/2719947785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc17270 0x55883fc1cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 --2- 172.21.15.12:0/2719947785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc16d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 --2- 172.21.15.12:0/2719947785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc16950 0x55883fc0b910 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 -- 172.21.15.12:0/2719947785 >> 172.21.15.12:0/2719947785 conn(0x55883fb6f1b0 msgr2=0x55883fc0d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 -- 172.21.15.12:0/2719947785 shutdown_connections 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 -- 172.21.15.12:0/2719947785 wait complete. 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 -- start start 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.812+0000 7f1028285b80 1 Processor -- start 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1028285b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc21c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1028285b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc22150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1028285b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc17270 0x55883fc1fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1028285b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fc20240 con 0x55883fc16950 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1028285b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fc203b0 con 0x55883fc11480 2024-05-08T23:53:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1028285b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883fc20570 con 0x55883fc17270 2024-05-08T23:53:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1028285b80 10 client.0 _ll_register_callbacks cb 0x55883fb06ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:53:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1019666700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc22150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1019666700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc22150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44162/0 (socket says 172.21.15.12:44162) 2024-05-08T23:53:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1019666700 1 -- 172.21.15.12:0/2371921746 learned_addr learned my addr 172.21.15.12:0/2371921746 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:53:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1019e67700 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc21c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f101a668700 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc17270 0x55883fc1fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1019666700 1 -- 172.21.15.12:0/2371921746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc17270 msgr2=0x55883fc1fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1019666700 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc17270 0x55883fc1fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1019666700 1 -- 172.21.15.12:0/2371921746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 msgr2=0x55883fc21c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1019666700 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc21c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.813+0000 7f1019666700 1 -- 172.21.15.12:0/2371921746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55883f9e2350 con 0x55883fc16950 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f1019666700 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc22150 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f10040093f0 tx=0x7f1004024ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac345838904af34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f0ffd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2371921746 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100404e070 con 0x55883fc16950 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2371921746 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1004004ac0 con 0x55883fc16950 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f1028285b80 1 -- 172.21.15.12:0/2371921746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f100c00f040 con 0x55883fc16950 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2371921746 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1004004c80 con 0x55883fc16950 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f1028285b80 10 client.17535 fetch_fsmap learned FSMap version 38 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f1028285b80 1 -- 172.21.15.12:0/2371921746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55883f98eb00 con 0x55883fc16950 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2371921746 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1004057020 con 0x55883fc16950 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2371921746 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1004049070 con 0x55883fc16950 2024-05-08T23:53:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f1028285b80 10 client.17535 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:53:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f1028285b80 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:53:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f1028285b80 20 client.17535 populate_metadata read hostname 'smithi012' 2024-05-08T23:53:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f1028285b80 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55883fc850d0 0x55883fc874f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:53:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f1028285b80 4 client.17535 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:53:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.814+0000 7f1028285b80 1 -- 172.21.15.12:0/2371921746 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55883fb59e60 con 0x55883fc850d0 2024-05-08T23:53:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.815+0000 7f1019e67700 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55883fc850d0 0x55883fc874f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:53:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.815+0000 7f1019e67700 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55883fc850d0 0x55883fc874f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4836cc646a6fd339 server_cookie=c16a88946cb28daa in_seq=0 out_seq=0 2024-05-08T23:53:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.815+0000 7f0ffd7fa700 10 client.17535 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.843+0000 7f0ffd7fa700 1 -- 172.21.15.12:0/2371921746 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55883fb59e60 con 0x55883fc850d0 2024-05-08T23:53:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.843+0000 7f0ffd7fa700 10 client.17535 handle_command_reply: tid=0 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 client.17535 shutdown 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 -- 172.21.15.12:0/2371921746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 msgr2=0x55883fc22150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc22150 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f10040093f0 tx=0x7f1004024ab0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 -- 172.21.15.12:0/2371921746 shutdown_connections 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55883fc17270 0x55883fc1fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55883fc850d0 0x55883fc874f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55883fc11480 0x55883fc21c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f0ffd7fa700 0 client.17535 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 --2- 172.21.15.12:0/2371921746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55883fc16950 0x55883fc22150 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 -- 172.21.15.12:0/2371921746 >> 172.21.15.12:0/2371921746 conn(0x55883fb6f1b0 msgr2=0x55883fc0f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 -- 172.21.15.12:0/2371921746 shutdown_connections 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 1 -- 172.21.15.12:0/2371921746 wait complete. 2024-05-08T23:53:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.844+0000 7f1028285b80 20 client.17535 trim_cache size 0 max 16384 2024-05-08T23:53:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:53:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:53:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:53:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:53:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 -- 172.21.15.12:0/3103894502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10000a1290 msgr2=0x7f10000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10000a1290 0x7f10000a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f100400b2e0 tx=0x7f1004005960 comp rx=0 tx=0).stop 2024-05-08T23:53:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 -- 172.21.15.12:0/3103894502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10141010f0 msgr2=0x7f1014197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:53:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10141010f0 0x7f1014197340 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f101000eb30 tx=0x7f101000ee40 comp rx=0 tx=0).stop 2024-05-08T23:53:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 -- 172.21.15.12:0/3103894502 shutdown_connections 2024-05-08T23:53:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10000a1290 0x7f10000a3750 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101419bc20 0x7f101419c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10141010f0 0x7f1014197340 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 --2- 172.21.15.12:0/3103894502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1014106210 0x7f1014197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:53:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 -- 172.21.15.12:0/3103894502 >> 172.21.15.12:0/3103894502 conn(0x7f10140fc970 msgr2=0x7f10140fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:53:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 -- 172.21.15.12:0/3103894502 shutdown_connections 2024-05-08T23:53:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:53:53.845+0000 7f101c0cb700 1 -- 172.21.15.12:0/3103894502 wait complete. 2024-05-08T23:53:53.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 466518 8.78 MB/sec execute 229 sec latency 4976.926 ms 2024-05-08T23:53:54.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:54 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:54.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:54 smithi012 ceph-mon[42503]: pgmap v233: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-05-08T23:53:54.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:54 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:54.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:54 smithi204 ceph-mon[47373]: pgmap v233: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-05-08T23:53:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:54 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:53:54.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:54 smithi160 ceph-mon[46152]: pgmap v233: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-05-08T23:53:54.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 467138 8.76 MB/sec execute 230 sec latency 705.342 ms 2024-05-08T23:53:55.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 467138 8.72 MB/sec execute 231 sec latency 1705.461 ms 2024-05-08T23:53:56.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:56 smithi204 ceph-mon[47373]: pgmap v234: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-05-08T23:53:56.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:56 smithi160 ceph-mon[46152]: pgmap v234: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-05-08T23:53:56.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 469242 8.74 MB/sec execute 232 sec latency 1881.279 ms 2024-05-08T23:53:56.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:56 smithi012 ceph-mon[42503]: pgmap v234: 97 pgs: 97 active+clean; 126 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-05-08T23:53:57.872 INFO:tasks.workunit.client.0.smithi012.stdout: 1 470685 8.73 MB/sec execute 233 sec latency 138.925 ms 2024-05-08T23:53:58.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:53:58 smithi204 ceph-mon[47373]: pgmap v235: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-05-08T23:53:58.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:53:58 smithi160 ceph-mon[46152]: pgmap v235: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-05-08T23:53:58.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 470685 8.69 MB/sec execute 234 sec latency 1139.014 ms 2024-05-08T23:53:58.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:53:58 smithi012 ceph-mon[42503]: pgmap v235: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-05-08T23:53:59.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 470685 8.66 MB/sec execute 235 sec latency 2139.137 ms 2024-05-08T23:54:00.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:00 smithi204 ceph-mon[47373]: pgmap v236: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-05-08T23:54:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:00 smithi160 ceph-mon[46152]: pgmap v236: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-05-08T23:54:00.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 470685 8.62 MB/sec execute 236 sec latency 3139.270 ms 2024-05-08T23:54:00.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:00 smithi012 ceph-mon[42503]: pgmap v236: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-05-08T23:54:01.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 472851 8.64 MB/sec execute 237 sec latency 3296.447 ms 2024-05-08T23:54:02.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:02 smithi204 ceph-mon[47373]: pgmap v237: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 3.9 MiB/s wr, 165 op/s 2024-05-08T23:54:02.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:02 smithi160 ceph-mon[46152]: pgmap v237: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 3.9 MiB/s wr, 165 op/s 2024-05-08T23:54:02.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 474231 8.63 MB/sec execute 238 sec latency 16.650 ms 2024-05-08T23:54:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:02 smithi012 ceph-mon[42503]: pgmap v237: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 3.9 MiB/s wr, 165 op/s 2024-05-08T23:54:03.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 474231 8.59 MB/sec execute 239 sec latency 1011.324 ms 2024-05-08T23:54:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:04 smithi204 ceph-mon[47373]: pgmap v238: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 3.4 MiB/s wr, 139 op/s 2024-05-08T23:54:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:04 smithi204 ceph-mon[47373]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T23:54:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:04 smithi160 ceph-mon[46152]: pgmap v238: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 3.4 MiB/s wr, 139 op/s 2024-05-08T23:54:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:04 smithi160 ceph-mon[46152]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T23:54:04.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 474231 8.56 MB/sec execute 240 sec latency 2011.422 ms 2024-05-08T23:54:04.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:04 smithi012 ceph-mon[42503]: pgmap v238: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 3.4 MiB/s wr, 139 op/s 2024-05-08T23:54:04.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:04 smithi012 ceph-mon[42503]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T23:54:05.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 474231 8.52 MB/sec execute 241 sec latency 3011.538 ms 2024-05-08T23:54:05.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:05 smithi012 ceph-mon[42503]: pgmap v240: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 4.0 MiB/s wr, 167 op/s 2024-05-08T23:54:06.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:05 smithi204 ceph-mon[47373]: pgmap v240: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 4.0 MiB/s wr, 167 op/s 2024-05-08T23:54:06.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:05 smithi160 ceph-mon[46152]: pgmap v240: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 4.0 MiB/s wr, 167 op/s 2024-05-08T23:54:06.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 476332 8.54 MB/sec execute 242 sec latency 3164.443 ms 2024-05-08T23:54:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.979+0000 7ff34f264700 1 -- 172.21.15.12:0/4231702584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff348105e60 msgr2=0x7ff348106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.979+0000 7ff34f264700 1 --2- 172.21.15.12:0/4231702584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff348105e60 0x7ff348106240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0098f0 tx=0x7ff33c009c00 comp rx=0 tx=0).stop 2024-05-08T23:54:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.979+0000 7ff34f264700 1 -- 172.21.15.12:0/4231702584 shutdown_connections 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.979+0000 7ff34f264700 1 --2- 172.21.15.12:0/4231702584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff348100760 0x7ff34810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.979+0000 7ff34f264700 1 --2- 172.21.15.12:0/4231702584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff348105e60 0x7ff348106240 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.979+0000 7ff34f264700 1 --2- 172.21.15.12:0/4231702584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3480ffda0 0x7ff348100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.979+0000 7ff34f264700 1 -- 172.21.15.12:0/4231702584 >> 172.21.15.12:0/4231702584 conn(0x7ff348076f10 msgr2=0x7ff348077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.979+0000 7ff34f264700 1 -- 172.21.15.12:0/4231702584 shutdown_connections 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.979+0000 7ff34f264700 1 -- 172.21.15.12:0/4231702584 wait complete. 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34f264700 1 Processor -- start 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34f264700 1 -- start start 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34f264700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3480ffda0 0x7ff348197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34f264700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff348100760 0x7ff348197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34f264700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff348105e60 0x7ff34819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34f264700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34819c220 con 0x7ff348105e60 2024-05-08T23:54:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34f264700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34819c390 con 0x7ff348100760 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34f264700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34819c550 con 0x7ff3480ffda0 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34d801700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff348105e60 0x7ff34819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34d801700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff348105e60 0x7ff34819bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35058/0 (socket says 172.21.15.12:35058) 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34d000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3480ffda0 0x7ff348197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff34d801700 1 -- 172.21.15.12:0/2946416463 learned_addr learned my addr 172.21.15.12:0/2946416463 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.980+0000 7ff347fff700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff348100760 0x7ff348197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff34d000700 1 -- 172.21.15.12:0/2946416463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff348100760 msgr2=0x7ff348197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff34d000700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff348100760 0x7ff348197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff34d000700 1 -- 172.21.15.12:0/2946416463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff348105e60 msgr2=0x7ff34819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff34d000700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff348105e60 0x7ff34819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff34d000700 1 -- 172.21.15.12:0/2946416463 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c009520 con 0x7ff3480ffda0 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff34d801700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff348105e60 0x7ff34819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff347fff700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff348100760 0x7ff348197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff34d000700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3480ffda0 0x7ff348197300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009230 tx=0x7ff33c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff345ffb700 1 -- 172.21.15.12:0/2946416463 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c01c070 con 0x7ff3480ffda0 2024-05-08T23:54:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff345ffb700 1 -- 172.21.15.12:0/2946416463 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33c00bba0 con 0x7ff3480ffda0 2024-05-08T23:54:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff34f264700 1 -- 172.21.15.12:0/2946416463 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34819c7d0 con 0x7ff3480ffda0 2024-05-08T23:54:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff345ffb700 1 -- 172.21.15.12:0/2946416463 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0038e0 con 0x7ff3480ffda0 2024-05-08T23:54:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.981+0000 7ff34f264700 1 -- 172.21.15.12:0/2946416463 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3481a4070 con 0x7ff3480ffda0 2024-05-08T23:54:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.983+0000 7ff345ffb700 1 -- 172.21.15.12:0/2946416463 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff33c020030 con 0x7ff3480ffda0 2024-05-08T23:54:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.984+0000 7ff345ffb700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3380a1310 0x7ff3380a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.984+0000 7ff345ffb700 1 -- 172.21.15.12:0/2946416463 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff33c0cdc50 con 0x7ff3480ffda0 2024-05-08T23:54:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.984+0000 7ff347fff700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3380a1310 0x7ff3380a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.985+0000 7ff347fff700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3380a1310 0x7ff3380a37d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff348198850 tx=0x7ff334009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.985+0000 7ff35b41eb80 1 Processor -- start 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.985+0000 7ff35b41eb80 1 -- start start 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.985+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 0x5605a9ce0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.985+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 0x5605a9cdaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.985+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cdb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.985+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9c320f0 con 0x5605a9cdb480 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.985+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9c2fab0 con 0x5605a9ce1270 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.985+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9c2b3a0 con 0x5605a9ce0950 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d801700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cdb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff347fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 0x5605a9cdaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d801700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cdb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35070/0 (socket says 172.21.15.12:35070) 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d801700 1 -- 172.21.15.12:0/650750125 learned_addr learned my addr 172.21.15.12:0/650750125 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d000700 1 --2- 172.21.15.12:0/650750125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 0x5605a9ce0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d801700 1 -- 172.21.15.12:0/650750125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 msgr2=0x5605a9ce0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d801700 1 --2- 172.21.15.12:0/650750125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 0x5605a9ce0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d801700 1 -- 172.21.15.12:0/650750125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 msgr2=0x5605a9cdaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d801700 1 --2- 172.21.15.12:0/650750125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 0x5605a9cdaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d801700 1 -- 172.21.15.12:0/650750125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605a9aac350 con 0x5605a9cdb480 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d000700 1 --2- 172.21.15.12:0/650750125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 0x5605a9ce0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff34d801700 1 --2- 172.21.15.12:0/650750125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cdb8a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff3400078d0 tx=0x7ff3400091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d80472b4b818da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff332ffd700 1 -- 172.21.15.12:0/650750125 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340015850 con 0x5605a9cdb480 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff332ffd700 1 -- 172.21.15.12:0/650750125 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff34000f040 con 0x5605a9cdb480 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.986+0000 7ff332ffd700 1 -- 172.21.15.12:0/650750125 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340014d00 con 0x5605a9cdb480 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.987+0000 7ff35b41eb80 1 -- 172.21.15.12:0/650750125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 msgr2=0x5605a9cdb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.987+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/650750125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cdb8a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff3400078d0 tx=0x7ff3400091b0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.987+0000 7ff35b41eb80 1 -- 172.21.15.12:0/650750125 shutdown_connections 2024-05-08T23:54:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.987+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/650750125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cdb8a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.987+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/650750125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 0x5605a9ce0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.987+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/650750125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 0x5605a9cdaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.987+0000 7ff35b41eb80 1 -- 172.21.15.12:0/650750125 >> 172.21.15.12:0/650750125 conn(0x5605a9c391b0 msgr2=0x5605a9cd5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.987+0000 7ff35b41eb80 1 -- 172.21.15.12:0/650750125 shutdown_connections 2024-05-08T23:54:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.987+0000 7ff35b41eb80 1 -- 172.21.15.12:0/650750125 wait complete. 2024-05-08T23:54:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff35b41eb80 1 -- start start 2024-05-08T23:54:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff35b41eb80 1 Processor -- start 2024-05-08T23:54:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9cdb480 0x5605a9ce7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 0x5605a9ce44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9ce1270 0x5605a9ce4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9bc7500 con 0x5605a9ce1270 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9bc71a0 con 0x5605a9ce0950 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9aae640 con 0x5605a9cdb480 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff35b41eb80 10 client.0 _ll_register_callbacks cb 0x5605a9bd0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff34d000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9cdb480 0x5605a9ce7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff34d000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9cdb480 0x5605a9ce7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51002/0 (socket says 172.21.15.12:51002) 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff347fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 0x5605a9ce44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff34d000700 1 -- 172.21.15.12:0/1174606255 learned_addr learned my addr 172.21.15.12:0/1174606255 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.988+0000 7ff34d801700 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9ce1270 0x5605a9ce4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff34d000700 1 -- 172.21.15.12:0/1174606255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 msgr2=0x5605a9ce44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff34d000700 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 0x5605a9ce44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff34d000700 1 -- 172.21.15.12:0/1174606255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9ce1270 msgr2=0x5605a9ce4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff34d000700 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9ce1270 0x5605a9ce4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff34d000700 1 -- 172.21.15.12:0/1174606255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3481a4070 con 0x5605a9cdb480 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff347fff700 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 0x5605a9ce44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff34d000700 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9cdb480 0x5605a9ce7200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009fd0 tx=0x7ff33c09a740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7345ee3c1b214fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff327fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff327fff700 1 -- 172.21.15.12:0/1174606255 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0b2070 con 0x5605a9cdb480 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff327fff700 1 -- 172.21.15.12:0/1174606255 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff33c09ec10 con 0x5605a9cdb480 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff35b41eb80 1 -- 172.21.15.12:0/1174606255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605a9b84210 con 0x5605a9cdb480 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff327fff700 1 -- 172.21.15.12:0/1174606255 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff33c0a31d0 con 0x5605a9cdb480 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff35b41eb80 10 client.27325 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.989+0000 7ff35b41eb80 1 -- 172.21.15.12:0/1174606255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7ff340007500 con 0x5605a9cdb480 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.990+0000 7ff327fff700 1 -- 172.21.15.12:0/1174606255 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33c07d080 con 0x5605a9cdb480 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.990+0000 7ff327fff700 1 -- 172.21.15.12:0/1174606255 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff33c005140 con 0x5605a9cdb480 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.990+0000 7ff35b41eb80 10 client.27325 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.990+0000 7ff35b41eb80 10 client.27325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.990+0000 7ff35b41eb80 20 client.27325 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.990+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5605a9cf4860 0x5605a9cf6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.990+0000 7ff35b41eb80 4 client.27325 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.990+0000 7ff35b41eb80 1 -- 172.21.15.12:0/1174606255 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605a9caa7b0 con 0x5605a9cf4860 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.990+0000 7ff347fff700 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5605a9cf4860 0x5605a9cf6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.991+0000 7ff327fff700 10 client.27325 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.991+0000 7ff347fff700 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5605a9cf4860 0x5605a9cf6c80 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e213b3a16f8bae server_cookie=5e9907a02724bd87 in_seq=0 out_seq=0 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff327fff700 1 -- 172.21.15.12:0/1174606255 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5605a9caa7b0 con 0x5605a9cf4860 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff327fff700 10 client.27325 handle_command_reply: tid=0 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff35b41eb80 1 client.27325 shutdown 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff35b41eb80 1 -- 172.21.15.12:0/1174606255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9cdb480 msgr2=0x5605a9ce7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9cdb480 0x5605a9ce7200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009fd0 tx=0x7ff33c09a740 comp rx=0 tx=0).stop 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff35b41eb80 1 -- 172.21.15.12:0/1174606255 shutdown_connections 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9ce1270 0x5605a9ce4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9cdb480 0x5605a9ce7200 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5605a9cf4860 0x5605a9cf6c80 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/1174606255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 0x5605a9ce44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff327fff700 0 client.27325 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.992+0000 7ff35b41eb80 1 -- 172.21.15.12:0/1174606255 >> 172.21.15.12:0/1174606255 conn(0x5605a9c391b0 msgr2=0x5605a9cd9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.993+0000 7ff35b41eb80 1 -- 172.21.15.12:0/1174606255 shutdown_connections 2024-05-08T23:54:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.993+0000 7ff35b41eb80 1 -- 172.21.15.12:0/1174606255 wait complete. 2024-05-08T23:54:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:06.993+0000 7ff35b41eb80 20 client.27325 trim_cache size 0 max 16384 2024-05-08T23:54:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff35b41eb80 1 Processor -- start 2024-05-08T23:54:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff35b41eb80 1 -- start start 2024-05-08T23:54:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cf6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 0x5605a9cf06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 0x5605a9cd5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9cd0330 con 0x5605a9cdb480 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9ce5af0 con 0x5605a9ce1270 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9ce5470 con 0x5605a9ce0950 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff34d000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cf6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff34d000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cf6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35086/0 (socket says 172.21.15.12:35086) 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.009+0000 7ff34d000700 1 -- 172.21.15.12:0/983267285 learned_addr learned my addr 172.21.15.12:0/983267285 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff34d801700 1 --2- 172.21.15.12:0/983267285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 0x5605a9cd5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff347fff700 1 --2- 172.21.15.12:0/983267285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 0x5605a9cf06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff34d000700 1 -- 172.21.15.12:0/983267285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 msgr2=0x5605a9cf06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff34d000700 1 --2- 172.21.15.12:0/983267285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 0x5605a9cf06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff34d000700 1 -- 172.21.15.12:0/983267285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 msgr2=0x5605a9cd5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff34d000700 1 --2- 172.21.15.12:0/983267285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 0x5605a9cd5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff34d000700 1 -- 172.21.15.12:0/983267285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605a9a58b00 con 0x5605a9cdb480 2024-05-08T23:54:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff34d801700 1 --2- 172.21.15.12:0/983267285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 0x5605a9cd5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff34d000700 1 --2- 172.21.15.12:0/983267285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cf6c50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff33c005bf0 tx=0x7ff33c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffec54a9e9a74fca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff3337fe700 1 -- 172.21.15.12:0/983267285 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0b2050 con 0x5605a9cdb480 2024-05-08T23:54:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff3337fe700 1 -- 172.21.15.12:0/983267285 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5605a9b84210 con 0x5605a9cdb480 2024-05-08T23:54:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.010+0000 7ff3337fe700 1 -- 172.21.15.12:0/983267285 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c005140 con 0x5605a9cdb480 2024-05-08T23:54:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 -- 172.21.15.12:0/983267285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 msgr2=0x5605a9cf6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/983267285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cf6c50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff33c005bf0 tx=0x7ff33c00fa70 comp rx=0 tx=0).stop 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 -- 172.21.15.12:0/983267285 shutdown_connections 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/983267285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce1270 0x5605a9cd5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/983267285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cf6c50 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/983267285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce0950 0x5605a9cf06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 -- 172.21.15.12:0/983267285 >> 172.21.15.12:0/983267285 conn(0x5605a9c391b0 msgr2=0x5605a9ce3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 -- 172.21.15.12:0/983267285 shutdown_connections 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 -- 172.21.15.12:0/983267285 wait complete. 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 -- start start 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.011+0000 7ff35b41eb80 1 Processor -- start 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 0x5605a9ce9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff35b41eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce1270 0x5605a9cea160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9cec580 con 0x5605a9cdb480 2024-05-08T23:54:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9cea6d0 con 0x5605a9ce0950 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff35b41eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9cea890 con 0x5605a9ce1270 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff35b41eb80 10 client.0 _ll_register_callbacks cb 0x5605a9bd0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff34d000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff34d000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cebe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35098/0 (socket says 172.21.15.12:35098) 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff34d000700 1 -- 172.21.15.12:0/804451709 learned_addr learned my addr 172.21.15.12:0/804451709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff347fff700 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 0x5605a9ce9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.012+0000 7ff34d801700 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce1270 0x5605a9cea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff347fff700 1 -- 172.21.15.12:0/804451709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce1270 msgr2=0x5605a9cea160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff347fff700 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce1270 0x5605a9cea160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff347fff700 1 -- 172.21.15.12:0/804451709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 msgr2=0x5605a9cebe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff347fff700 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cebe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff347fff700 1 -- 172.21.15.12:0/804451709 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605a9aac350 con 0x5605a9ce0950 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff34d000700 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff34d801700 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce1270 0x5605a9cea160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff347fff700 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 0x5605a9ce9c20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5605a9cf7590 tx=0x7ff3340047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae4dc1a9ac96777a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff327fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff327fff700 1 -- 172.21.15.12:0/804451709 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334026040 con 0x5605a9ce0950 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff327fff700 1 -- 172.21.15.12:0/804451709 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff3340039e0 con 0x5605a9ce0950 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff35b41eb80 1 -- 172.21.15.12:0/804451709 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff34000f040 con 0x5605a9ce0950 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff327fff700 1 -- 172.21.15.12:0/804451709 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff334003ba0 con 0x5605a9ce0950 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff35b41eb80 10 client.27341 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.013+0000 7ff35b41eb80 1 -- 172.21.15.12:0/804451709 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7ff340007500 con 0x5605a9ce0950 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff327fff700 1 -- 172.21.15.12:0/804451709 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff334044080 con 0x5605a9ce0950 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff327fff700 1 -- 172.21.15.12:0/804451709 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff334038430 con 0x5605a9ce0950 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff35b41eb80 10 client.27341 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff35b41eb80 10 client.27341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff35b41eb80 20 client.27341 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5605a9d4b250 0x5605a9d4d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff35b41eb80 4 client.27341 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:54:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff35b41eb80 1 -- 172.21.15.12:0/804451709 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5605a9c23e60 con 0x5605a9d4b250 2024-05-08T23:54:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff34d000700 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5605a9d4b250 0x5605a9d4d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff34d000700 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5605a9d4b250 0x5605a9d4d670 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e603245218ef31 server_cookie=6d7caafc718a5b0a in_seq=0 out_seq=0 2024-05-08T23:54:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:07.014+0000 7ff327fff700 10 client.27341 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:07.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 476612 8.51 MB/sec execute 243 sec latency 684.459 ms 2024-05-08T23:54:08.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:08 smithi204 ceph-mon[47373]: pgmap v241: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.3 MiB/s wr, 137 op/s 2024-05-08T23:54:08.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:08 smithi160 ceph-mon[46152]: pgmap v241: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.3 MiB/s wr, 137 op/s 2024-05-08T23:54:08.874 INFO:tasks.workunit.client.0.smithi012.stdout: 1 476612 8.47 MB/sec execute 244 sec latency 1684.546 ms 2024-05-08T23:54:08.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:08 smithi012 ceph-mon[42503]: pgmap v241: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.3 MiB/s wr, 137 op/s 2024-05-08T23:54:09.172 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:54:09.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.339+0000 7fc371a11700 1 -- 172.21.15.12:0/2231658864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc36c0ffe10 msgr2=0x7fc36c100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:09.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.339+0000 7fc371a11700 1 --2- 172.21.15.12:0/2231658864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc36c0ffe10 0x7fc36c100290 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc360009990 tx=0x7fc360009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.339+0000 7fc371a11700 1 -- 172.21.15.12:0/2231658864 shutdown_connections 2024-05-08T23:54:09.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.339+0000 7fc371a11700 1 --2- 172.21.15.12:0/2231658864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36c1007d0 0x7fc36c10cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.339+0000 7fc371a11700 1 --2- 172.21.15.12:0/2231658864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36c105ed0 0x7fc36c1062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.339+0000 7fc371a11700 1 --2- 172.21.15.12:0/2231658864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc36c0ffe10 0x7fc36c100290 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.339+0000 7fc371a11700 1 -- 172.21.15.12:0/2231658864 >> 172.21.15.12:0/2231658864 conn(0x7fc36c076f10 msgr2=0x7fc36c077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:09.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.339+0000 7fc371a11700 1 -- 172.21.15.12:0/2231658864 shutdown_connections 2024-05-08T23:54:09.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.339+0000 7fc371a11700 1 -- 172.21.15.12:0/2231658864 wait complete. 2024-05-08T23:54:09.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc371a11700 1 Processor -- start 2024-05-08T23:54:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc371a11700 1 -- start start 2024-05-08T23:54:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc371a11700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36c0ffe10 0x7fc36c101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc371a11700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36c1007d0 0x7fc36c102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc371a11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc36c105ed0 0x7fc36c102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc371a11700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1a5750 con 0x7fc36c105ed0 2024-05-08T23:54:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc371a11700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1a58c0 con 0x7fc36c0ffe10 2024-05-08T23:54:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc371a11700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1a5a80 con 0x7fc36c1007d0 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc36c105ed0 0x7fc36c102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc36c105ed0 0x7fc36c102560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35980/0 (socket says 172.21.15.12:35980) 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc36b7fe700 1 -- 172.21.15.12:0/3496193595 learned_addr learned my addr 172.21.15.12:0/3496193595 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc36a7fc700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36c1007d0 0x7fc36c102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.340+0000 7fc36affd700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36c0ffe10 0x7fc36c101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc36b7fe700 1 -- 172.21.15.12:0/3496193595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36c1007d0 msgr2=0x7fc36c102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36c1007d0 0x7fc36c102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc36b7fe700 1 -- 172.21.15.12:0/3496193595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36c0ffe10 msgr2=0x7fc36c101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36c0ffe10 0x7fc36c101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc36b7fe700 1 -- 172.21.15.12:0/3496193595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3600095f0 con 0x7fc36c105ed0 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc36affd700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36c0ffe10 0x7fc36c101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc36c105ed0 0x7fc36c102560 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc364007ee0 tx=0x7fc36400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc370a0f700 1 -- 172.21.15.12:0/3496193595 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364017070 con 0x7fc36c105ed0 2024-05-08T23:54:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc370a0f700 1 -- 172.21.15.12:0/3496193595 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc36400f040 con 0x7fc36c105ed0 2024-05-08T23:54:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc371a11700 1 -- 172.21.15.12:0/3496193595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c1a5d60 con 0x7fc36c105ed0 2024-05-08T23:54:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc370a0f700 1 -- 172.21.15.12:0/3496193595 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364005740 con 0x7fc36c105ed0 2024-05-08T23:54:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.341+0000 7fc371a11700 1 -- 172.21.15.12:0/3496193595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c1a61f0 con 0x7fc36c105ed0 2024-05-08T23:54:09.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.343+0000 7fc370a0f700 1 -- 172.21.15.12:0/3496193595 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc36400dc60 con 0x7fc36c105ed0 2024-05-08T23:54:09.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.344+0000 7fc370a0f700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3540a1290 0x7fc3540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.344+0000 7fc36affd700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3540a1290 0x7fc3540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.344+0000 7fc370a0f700 1 -- 172.21.15.12:0/3496193595 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc3640cc560 con 0x7fc36c105ed0 2024-05-08T23:54:09.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc36affd700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3540a1290 0x7fc3540a3750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009c90 tx=0x7fc35c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:09.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc37dbcbb80 1 Processor -- start 2024-05-08T23:54:09.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc37dbcbb80 1 -- start start 2024-05-08T23:54:09.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a2fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fd720 0x561b8a2f7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a24e150 con 0x561b8a2f7890 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a24bb10 con 0x561b8a2fd720 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a247400 con 0x561b8a2fce00 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35992/0 (socket says 172.21.15.12:35992) 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc36b7fe700 1 -- 172.21.15.12:0/1603994337 learned_addr learned my addr 172.21.15.12:0/1603994337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc36affd700 1 --2- 172.21.15.12:0/1603994337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a2fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc36a7fc700 1 --2- 172.21.15.12:0/1603994337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fd720 0x561b8a2f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc36b7fe700 1 -- 172.21.15.12:0/1603994337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 msgr2=0x561b8a2fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/1603994337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a2fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc36b7fe700 1 -- 172.21.15.12:0/1603994337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fd720 msgr2=0x561b8a2f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.345+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/1603994337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fd720 0x561b8a2f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.346+0000 7fc36b7fe700 1 -- 172.21.15.12:0/1603994337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b8a0c83b0 con 0x561b8a2f7890 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.346+0000 7fc36a7fc700 1 --2- 172.21.15.12:0/1603994337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fd720 0x561b8a2f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.346+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/1603994337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7cd0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc3640a1550 tx=0x7fc36400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51abed86963fd6c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.346+0000 7fc351ffb700 1 -- 172.21.15.12:0/1603994337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640b90d0 con 0x561b8a2f7890 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.346+0000 7fc351ffb700 1 -- 172.21.15.12:0/1603994337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc36409d430 con 0x561b8a2f7890 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.346+0000 7fc351ffb700 1 -- 172.21.15.12:0/1603994337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640840c0 con 0x561b8a2f7890 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1603994337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 msgr2=0x561b8a2f7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/1603994337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7cd0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc3640a1550 tx=0x7fc36400df60 comp rx=0 tx=0).stop 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1603994337 shutdown_connections 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/1603994337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7cd0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/1603994337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a2fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/1603994337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fd720 0x561b8a2f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1603994337 >> 172.21.15.12:0/1603994337 conn(0x561b8a255210 msgr2=0x561b8a2f1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:09.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1603994337 shutdown_connections 2024-05-08T23:54:09.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1603994337 wait complete. 2024-05-08T23:54:09.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 -- start start 2024-05-08T23:54:09.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.347+0000 7fc37dbcbb80 1 Processor -- start 2024-05-08T23:54:09.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 0x561b8a300b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a301040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 0x561b8a301580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a1e3560 con 0x561b8a2fd720 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a1e3200 con 0x561b8a2f7890 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a0ca6a0 con 0x561b8a2fce00 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc37dbcbb80 10 client.0 _ll_register_callbacks cb 0x561b8a1ecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 0x561b8a301580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 0x561b8a301580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36008/0 (socket says 172.21.15.12:36008) 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36b7fe700 1 -- 172.21.15.12:0/3309439258 learned_addr learned my addr 172.21.15.12:0/3309439258 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36a7fc700 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a301040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36affd700 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 0x561b8a300b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36b7fe700 1 -- 172.21.15.12:0/3309439258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 msgr2=0x561b8a301040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a301040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36b7fe700 1 -- 172.21.15.12:0/3309439258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 msgr2=0x561b8a300b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 0x561b8a300b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36b7fe700 1 -- 172.21.15.12:0/3309439258 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c1a61f0 con 0x561b8a2fd720 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36a7fc700 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a301040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 0x561b8a301580 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc364095ab0 tx=0x7fc3640b90d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48644cc7c0499905 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc34affd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.348+0000 7fc34affd700 1 -- 172.21.15.12:0/3309439258 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640058a0 con 0x561b8a2fd720 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc34affd700 1 -- 172.21.15.12:0/3309439258 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc364005a00 con 0x561b8a2fd720 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/3309439258 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b8a1a0270 con 0x561b8a2fd720 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc34affd700 1 -- 172.21.15.12:0/3309439258 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc364084050 con 0x561b8a2fd720 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc37dbcbb80 10 client.17565 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/3309439258 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561b8a074b60 con 0x561b8a2fd720 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc34affd700 1 -- 172.21.15.12:0/3309439258 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b8a1a0270 con 0x561b8a2fd720 2024-05-08T23:54:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc34affd700 1 -- 172.21.15.12:0/3309439258 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc3640b5020 con 0x561b8a2fd720 2024-05-08T23:54:09.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc37dbcbb80 10 client.17565 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:09.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc37dbcbb80 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:09.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc37dbcbb80 20 client.17565 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:09.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b8a310e60 0x561b8a313280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:09.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc37dbcbb80 4 client.17565 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:09.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.349+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/3309439258 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b8a2c6810 con 0x561b8a310e60 2024-05-08T23:54:09.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.350+0000 7fc36affd700 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b8a310e60 0x561b8a313280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:09.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.350+0000 7fc36affd700 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b8a310e60 0x561b8a313280 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f61db8ef5a3dd252 server_cookie=be4471da860e8904 in_seq=0 out_seq=0 2024-05-08T23:54:09.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:09.350+0000 7fc34affd700 10 client.17565 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:09.873 INFO:tasks.workunit.client.0.smithi012.stdout: 1 476612 8.44 MB/sec execute 245 sec latency 2684.606 ms 2024-05-08T23:54:10.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:10 smithi204 ceph-mon[47373]: pgmap v242: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.3 MiB/s wr, 137 op/s 2024-05-08T23:54:10.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:10 smithi160 ceph-mon[46152]: pgmap v242: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.3 MiB/s wr, 137 op/s 2024-05-08T23:54:10.874 INFO:tasks.workunit.client.0.smithi012.stdout: 1 476612 8.40 MB/sec execute 246 sec latency 3684.738 ms 2024-05-08T23:54:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:10 smithi012 ceph-mon[42503]: pgmap v242: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.3 MiB/s wr, 137 op/s 2024-05-08T23:54:11.874 INFO:tasks.workunit.client.0.smithi012.stdout: 1 476612 8.37 MB/sec execute 247 sec latency 4684.812 ms 2024-05-08T23:54:12.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:11 smithi012 ceph-mon[42503]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T23:54:12.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:11 smithi012 ceph-mon[42503]: pgmap v244: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 739 KiB/s wr, 114 op/s 2024-05-08T23:54:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:11 smithi204 ceph-mon[47373]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T23:54:12.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:11 smithi204 ceph-mon[47373]: pgmap v244: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 739 KiB/s wr, 114 op/s 2024-05-08T23:54:12.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:11 smithi160 ceph-mon[46152]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T23:54:12.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:11 smithi160 ceph-mon[46152]: pgmap v244: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 739 KiB/s wr, 114 op/s 2024-05-08T23:54:12.874 INFO:tasks.workunit.client.0.smithi012.stdout: 1 476612 8.34 MB/sec execute 248 sec latency 5684.940 ms 2024-05-08T23:54:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.385+0000 7ff327fff700 1 -- 172.21.15.12:0/804451709 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5605a9c23e60 con 0x5605a9d4b250 2024-05-08T23:54:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.385+0000 7ff327fff700 10 client.27341 handle_command_reply: tid=0 2024-05-08T23:54:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.385+0000 7ff35b41eb80 1 client.27341 shutdown 2024-05-08T23:54:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.385+0000 7ff35b41eb80 1 -- 172.21.15.12:0/804451709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 msgr2=0x5605a9ce9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.385+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 0x5605a9ce9c20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5605a9cf7590 tx=0x7ff3340047c0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.386+0000 7ff35b41eb80 1 -- 172.21.15.12:0/804451709 shutdown_connections 2024-05-08T23:54:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.386+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605a9ce1270 0x5605a9cea160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.386+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5605a9cdb480 0x5605a9cebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.386+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5605a9d4b250 0x5605a9d4d670 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.386+0000 7ff35b41eb80 1 --2- 172.21.15.12:0/804451709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5605a9ce0950 0x5605a9ce9c20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.386+0000 7ff327fff700 0 client.27341 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.386+0000 7ff35b41eb80 1 -- 172.21.15.12:0/804451709 >> 172.21.15.12:0/804451709 conn(0x5605a9c391b0 msgr2=0x5605a9ce1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:13.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc34affd700 1 -- 172.21.15.12:0/3309439258 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b8a2c6810 con 0x561b8a310e60 2024-05-08T23:54:13.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc34affd700 10 client.17565 handle_command_reply: tid=0 2024-05-08T23:54:13.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc37dbcbb80 1 client.17565 shutdown 2024-05-08T23:54:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.386+0000 7ff35b41eb80 1 -- 172.21.15.12:0/804451709 shutdown_connections 2024-05-08T23:54:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.386+0000 7ff35b41eb80 1 -- 172.21.15.12:0/804451709 wait complete. 2024-05-08T23:54:13.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/3309439258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 msgr2=0x561b8a301580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 0x561b8a301580 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc364095ab0 tx=0x7fc3640b90d0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.386+0000 7ff35b41eb80 20 client.27341 trim_cache size 0 max 16384 2024-05-08T23:54:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/3309439258 shutdown_connections 2024-05-08T23:54:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 0x561b8a301580 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b8a310e60 0x561b8a313280 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 0x561b8a300b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/3309439258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a301040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/3309439258 >> 172.21.15.12:0/3309439258 conn(0x561b8a255210 msgr2=0x561b8a2f15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.386+0000 7fc34affd700 0 client.17565 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.387+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/3309439258 shutdown_connections 2024-05-08T23:54:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.387+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/3309439258 wait complete. 2024-05-08T23:54:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.387+0000 7fc37dbcbb80 20 client.17565 trim_cache size 0 max 16384 2024-05-08T23:54:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:54:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:54:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:54:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 -- 172.21.15.12:0/2946416463 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3380a1310 msgr2=0x7ff3380a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3380a1310 0x7ff3380a37d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff348198850 tx=0x7ff334009310 comp rx=0 tx=0).stop 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 -- 172.21.15.12:0/2946416463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3480ffda0 msgr2=0x7ff348197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3480ffda0 0x7ff348197300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009230 tx=0x7ff33c00fa00 comp rx=0 tx=0).stop 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 -- 172.21.15.12:0/2946416463 shutdown_connections 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3380a1310 0x7ff3380a37d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff348105e60 0x7ff34819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3480ffda0 0x7ff348197300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 --2- 172.21.15.12:0/2946416463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff348100760 0x7ff348197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 -- 172.21.15.12:0/2946416463 >> 172.21.15.12:0/2946416463 conn(0x7ff348076f10 msgr2=0x7ff348068480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 -- 172.21.15.12:0/2946416463 shutdown_connections 2024-05-08T23:54:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:13.388+0000 7ff34f264700 1 -- 172.21.15.12:0/2946416463 wait complete. 2024-05-08T23:54:13.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.407+0000 7fc37dbcbb80 1 Processor -- start 2024-05-08T23:54:13.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.407+0000 7fc37dbcbb80 1 -- start start 2024-05-08T23:54:13.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fce00 0x561b8a2f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fd720 0x561b8a308160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a2f3070 con 0x561b8a2f7890 2024-05-08T23:54:13.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a2eedc0 con 0x561b8a2fce00 2024-05-08T23:54:13.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a2fd560 con 0x561b8a2fd720 2024-05-08T23:54:13.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36012/0 (socket says 172.21.15.12:36012) 2024-05-08T23:54:13.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36affd700 1 -- 172.21.15.12:0/531517639 learned_addr learned my addr 172.21.15.12:0/531517639 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:13.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36a7fc700 1 --2- 172.21.15.12:0/531517639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fce00 0x561b8a2f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/531517639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fd720 0x561b8a308160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36affd700 1 -- 172.21.15.12:0/531517639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fd720 msgr2=0x561b8a308160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:13.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36affd700 1 --2- 172.21.15.12:0/531517639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fd720 0x561b8a308160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36affd700 1 -- 172.21.15.12:0/531517639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fce00 msgr2=0x561b8a2f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36affd700 1 --2- 172.21.15.12:0/531517639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fce00 0x561b8a2f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36affd700 1 -- 172.21.15.12:0/531517639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b8a20d560 con 0x561b8a2f7890 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.408+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/531517639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fd720 0x561b8a308160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.409+0000 7fc36affd700 1 --2- 172.21.15.12:0/531517639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7350 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0097f0 tx=0x7fc35c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d375fc5082619b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.409+0000 7fc3527fc700 1 -- 172.21.15.12:0/531517639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c025070 con 0x561b8a2f7890 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.409+0000 7fc3527fc700 1 -- 172.21.15.12:0/531517639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c015070 con 0x561b8a2f7890 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.409+0000 7fc3527fc700 1 -- 172.21.15.12:0/531517639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0116a0 con 0x561b8a2f7890 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.410+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/531517639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 msgr2=0x561b8a2f7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.410+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/531517639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7350 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0097f0 tx=0x7fc35c013000 comp rx=0 tx=0).stop 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.410+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/531517639 shutdown_connections 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.410+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/531517639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fd720 0x561b8a308160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.410+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/531517639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2f7890 0x561b8a2f7350 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.410+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/531517639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2fce00 0x561b8a2f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.410+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/531517639 >> 172.21.15.12:0/531517639 conn(0x561b8a255210 msgr2=0x561b8a2ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:13.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.410+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/531517639 shutdown_connections 2024-05-08T23:54:13.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.410+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/531517639 wait complete. 2024-05-08T23:54:13.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.410+0000 7fc37dbcbb80 1 -- start start 2024-05-08T23:54:13.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc37dbcbb80 1 Processor -- start 2024-05-08T23:54:13.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 0x561b8a3084c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a2f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc37dbcbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 0x561b8a2f9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a2f9d20 con 0x561b8a2fd720 2024-05-08T23:54:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a2f9e90 con 0x561b8a2f7890 2024-05-08T23:54:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc37dbcbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a367360 con 0x561b8a2fce00 2024-05-08T23:54:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc37dbcbb80 10 client.0 _ll_register_callbacks cb 0x561b8a1ecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc36affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 0x561b8a3084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc36affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 0x561b8a3084c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45778/0 (socket says 172.21.15.12:45778) 2024-05-08T23:54:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc36a7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a2f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc36affd700 1 -- 172.21.15.12:0/1964052548 learned_addr learned my addr 172.21.15.12:0/1964052548 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.411+0000 7fc36b7fe700 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 0x561b8a2f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc36affd700 1 -- 172.21.15.12:0/1964052548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 msgr2=0x561b8a2f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc36affd700 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a2f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc36affd700 1 -- 172.21.15.12:0/1964052548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 msgr2=0x561b8a2f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc36affd700 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 0x561b8a2f9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc36affd700 1 -- 172.21.15.12:0/1964052548 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c0204c0 con 0x561b8a2f7890 2024-05-08T23:54:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc36affd700 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 0x561b8a3084c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc35c000c00 tx=0x7fc35c027560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628a13a4a3f8885f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc34affd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:54:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc34affd700 1 -- 172.21.15.12:0/1964052548 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c025040 con 0x561b8a2f7890 2024-05-08T23:54:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc34affd700 1 -- 172.21.15.12:0/1964052548 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc35c017070 con 0x561b8a2f7890 2024-05-08T23:54:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1964052548 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc36409d430 con 0x561b8a2f7890 2024-05-08T23:54:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc34affd700 1 -- 172.21.15.12:0/1964052548 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc35c005450 con 0x561b8a2f7890 2024-05-08T23:54:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc37dbcbb80 10 client.27359 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.412+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1964052548 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561b8a0cccd0 con 0x561b8a2f7890 2024-05-08T23:54:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.413+0000 7fc34affd700 1 -- 172.21.15.12:0/1964052548 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c03f020 con 0x561b8a2f7890 2024-05-08T23:54:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.413+0000 7fc34affd700 1 -- 172.21.15.12:0/1964052548 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc35c02e070 con 0x561b8a2f7890 2024-05-08T23:54:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.413+0000 7fc37dbcbb80 10 client.27359 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.413+0000 7fc37dbcbb80 10 client.27359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.413+0000 7fc37dbcbb80 20 client.27359 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.413+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b8a368a10 0x561b8a36ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.413+0000 7fc37dbcbb80 4 client.27359 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:54:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.413+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1964052548 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561b8a23fec0 con 0x561b8a368a10 2024-05-08T23:54:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.413+0000 7fc36a7fc700 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b8a368a10 0x561b8a36ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc36a7fc700 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b8a368a10 0x561b8a36ae30 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed45b27ac07e0c6d server_cookie=93e27c57a52e16 in_seq=0 out_seq=0 2024-05-08T23:54:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc34affd700 10 client.27359 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc34affd700 1 -- 172.21.15.12:0/1964052548 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561b8a23fec0 con 0x561b8a368a10 2024-05-08T23:54:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc34affd700 10 client.27359 handle_command_reply: tid=0 2024-05-08T23:54:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 client.27359 shutdown 2024-05-08T23:54:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1964052548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 msgr2=0x561b8a3084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 0x561b8a3084c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc35c000c00 tx=0x7fc35c027560 comp rx=0 tx=0).stop 2024-05-08T23:54:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1964052548 shutdown_connections 2024-05-08T23:54:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b8a2fd720 0x561b8a2f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b8a368a10 0x561b8a36ae30 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b8a2f7890 0x561b8a3084c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 --2- 172.21.15.12:0/1964052548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b8a2fce00 0x561b8a2f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1964052548 >> 172.21.15.12:0/1964052548 conn(0x561b8a255210 msgr2=0x561b8a2fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc34affd700 0 client.27359 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1964052548 shutdown_connections 2024-05-08T23:54:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 1 -- 172.21.15.12:0/1964052548 wait complete. 2024-05-08T23:54:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.414+0000 7fc37dbcbb80 20 client.27359 trim_cache size 0 max 16384 2024-05-08T23:54:13.418 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:54:13.418 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:54:13.418 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:54:13.418 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:54:13.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 -- 172.21.15.12:0/3496193595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3540a1290 msgr2=0x7fc3540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3540a1290 0x7fc3540a3750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009c90 tx=0x7fc35c009260 comp rx=0 tx=0).stop 2024-05-08T23:54:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 -- 172.21.15.12:0/3496193595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc36c105ed0 msgr2=0x7fc36c102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc36c105ed0 0x7fc36c102560 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc364007ee0 tx=0x7fc36400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 -- 172.21.15.12:0/3496193595 shutdown_connections 2024-05-08T23:54:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc3540a1290 0x7fc3540a3750 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc36c105ed0 0x7fc36c102560 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36c0ffe10 0x7fc36c101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 --2- 172.21.15.12:0/3496193595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36c1007d0 0x7fc36c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 -- 172.21.15.12:0/3496193595 >> 172.21.15.12:0/3496193595 conn(0x7fc36c076f10 msgr2=0x7fc36c069790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:13.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 -- 172.21.15.12:0/3496193595 shutdown_connections 2024-05-08T23:54:13.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.415+0000 7fc371a11700 1 -- 172.21.15.12:0/3496193595 wait complete. 2024-05-08T23:54:13.426 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:54:13.427 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:54:13.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.589+0000 7ff75c80a700 1 -- 172.21.15.12:0/3036422999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff758073370 msgr2=0x7ff758073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.589+0000 7ff75c80a700 1 --2- 172.21.15.12:0/3036422999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff758073370 0x7ff758073750 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff748009990 tx=0x7ff748009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.589+0000 7ff75c80a700 1 -- 172.21.15.12:0/3036422999 shutdown_connections 2024-05-08T23:54:13.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.589+0000 7ff75c80a700 1 --2- 172.21.15.12:0/3036422999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75810d690 0x7ff75810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.589+0000 7ff75c80a700 1 --2- 172.21.15.12:0/3036422999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff758073370 0x7ff758073750 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.589+0000 7ff75c80a700 1 --2- 172.21.15.12:0/3036422999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758073c90 0x7ff75810d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.589+0000 7ff75c80a700 1 -- 172.21.15.12:0/3036422999 >> 172.21.15.12:0/3036422999 conn(0x7ff7580fc960 msgr2=0x7ff7580fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:13.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.590+0000 7ff75c80a700 1 -- 172.21.15.12:0/3036422999 shutdown_connections 2024-05-08T23:54:13.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.590+0000 7ff75c80a700 1 -- 172.21.15.12:0/3036422999 wait complete. 2024-05-08T23:54:13.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75c80a700 1 Processor -- start 2024-05-08T23:54:13.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75c80a700 1 -- start start 2024-05-08T23:54:13.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75c80a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758073370 0x7ff758199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75c80a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758073c90 0x7ff758199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75c80a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff75810d690 0x7ff75819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75c80a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75819e420 con 0x7ff75810d690 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75c80a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75819e590 con 0x7ff758073370 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75c80a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75819e750 con 0x7ff758073c90 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff756d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff75810d690 0x7ff75819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758073370 0x7ff758199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758073370 0x7ff758199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45802/0 (socket says 172.21.15.12:45802) 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff756d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff75810d690 0x7ff75819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36038/0 (socket says 172.21.15.12:36038) 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff75659c700 1 -- 172.21.15.12:0/2556722747 learned_addr learned my addr 172.21.15.12:0/2556722747 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.591+0000 7ff755d9b700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758073c90 0x7ff758199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff75659c700 1 -- 172.21.15.12:0/2556722747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758073c90 msgr2=0x7ff758199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff75659c700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758073c90 0x7ff758199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff75659c700 1 -- 172.21.15.12:0/2556722747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff75810d690 msgr2=0x7ff75819dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff75659c700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff75810d690 0x7ff75819dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff75659c700 1 -- 172.21.15.12:0/2556722747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480095f0 con 0x7ff758073370 2024-05-08T23:54:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff75659c700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758073370 0x7ff758199500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff748005770 tx=0x7ff7480057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff7477fe700 1 -- 172.21.15.12:0/2556722747 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74801d070 con 0x7ff758073370 2024-05-08T23:54:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff7477fe700 1 -- 172.21.15.12:0/2556722747 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff748021410 con 0x7ff758073370 2024-05-08T23:54:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff7477fe700 1 -- 172.21.15.12:0/2556722747 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748017650 con 0x7ff758073370 2024-05-08T23:54:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff75c80a700 1 -- 172.21.15.12:0/2556722747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75819e9d0 con 0x7ff758073370 2024-05-08T23:54:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.592+0000 7ff75c80a700 1 -- 172.21.15.12:0/2556722747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7581a6270 con 0x7ff758073370 2024-05-08T23:54:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.593+0000 7ff75c80a700 1 -- 172.21.15.12:0/2556722747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7581088a0 con 0x7ff758073370 2024-05-08T23:54:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.595+0000 7ff7477fe700 1 -- 172.21.15.12:0/2556722747 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff74800fb00 con 0x7ff758073370 2024-05-08T23:54:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.595+0000 7ff7477fe700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7400a1310 0x7ff7400a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.596+0000 7ff7477fe700 1 -- 172.21.15.12:0/2556722747 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff7480cdf00 con 0x7ff758073370 2024-05-08T23:54:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.596+0000 7ff755d9b700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7400a1310 0x7ff7400a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.596+0000 7ff755d9b700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7400a1310 0x7ff7400a37d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff75819aa50 tx=0x7ff74c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:13.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.602+0000 7ff7477fe700 1 -- 172.21.15.12:0/2556722747 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff7480973f0 con 0x7ff758073370 2024-05-08T23:54:13.874 INFO:tasks.workunit.client.0.smithi012.stdout: 1 477498 8.32 MB/sec execute 249 sec latency 6190.196 ms 2024-05-08T23:54:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.897+0000 7ff75c80a700 1 -- 172.21.15.12:0/2556722747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff758067c10 con 0x7ff758073370 2024-05-08T23:54:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.898+0000 7ff7477fe700 1 -- 172.21.15.12:0/2556722747 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+10883 (secure 0 0 0) 0x7ff74809aa10 con 0x7ff758073370 2024-05-08T23:54:13.893 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:54:13.893 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:53:53.404590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27175":{"gid":27175,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2842389121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2842389121},{"type":"v1","addr":"172.21.15.204:6837","nonce":2842389121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:54:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 -- 172.21.15.12:0/2556722747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7400a1310 msgr2=0x7ff7400a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7400a1310 0x7ff7400a37d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff75819aa50 tx=0x7ff74c009260 comp rx=0 tx=0).stop 2024-05-08T23:54:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 -- 172.21.15.12:0/2556722747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758073370 msgr2=0x7ff758199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758073370 0x7ff758199500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff748005770 tx=0x7ff7480057a0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 -- 172.21.15.12:0/2556722747 shutdown_connections 2024-05-08T23:54:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff7400a1310 0x7ff7400a37d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff75810d690 0x7ff75819dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758073370 0x7ff758199500 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 --2- 172.21.15.12:0/2556722747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758073c90 0x7ff758199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 -- 172.21.15.12:0/2556722747 >> 172.21.15.12:0/2556722747 conn(0x7ff7580fc960 msgr2=0x7ff7580fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:13.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 -- 172.21.15.12:0/2556722747 shutdown_connections 2024-05-08T23:54:13.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:13.901+0000 7ff75c80a700 1 -- 172.21.15.12:0/2556722747 wait complete. 2024-05-08T23:54:13.897 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2024-05-08T23:54:13.913 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:54:14.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.065+0000 7f266edd5700 1 -- 172.21.15.12:0/2256665009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26681011e0 msgr2=0x7f26681015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.065+0000 7f266edd5700 1 --2- 172.21.15.12:0/2256665009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26681011e0 0x7f26681015c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f265c009990 tx=0x7f265c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.065+0000 7f266edd5700 1 -- 172.21.15.12:0/2256665009 shutdown_connections 2024-05-08T23:54:14.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.065+0000 7f266edd5700 1 --2- 172.21.15.12:0/2256665009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2668106300 0x7f266810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.065+0000 7f266edd5700 1 --2- 172.21.15.12:0/2256665009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26681011e0 0x7f26681015c0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.065+0000 7f266edd5700 1 --2- 172.21.15.12:0/2256665009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2668101b90 0x7f2668105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.065+0000 7f266edd5700 1 -- 172.21.15.12:0/2256665009 >> 172.21.15.12:0/2256665009 conn(0x7f26680fca80 msgr2=0x7f26680feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.065+0000 7f266edd5700 1 -- 172.21.15.12:0/2256665009 shutdown_connections 2024-05-08T23:54:14.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.065+0000 7f266edd5700 1 -- 172.21.15.12:0/2256665009 wait complete. 2024-05-08T23:54:14.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.066+0000 7f266edd5700 1 Processor -- start 2024-05-08T23:54:14.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.066+0000 7f266edd5700 1 -- start start 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.066+0000 7f266edd5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26681011e0 0x7f26680732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.066+0000 7f266edd5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2668106300 0x7f26680737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.066+0000 7f266edd5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2668074f60 0x7f2668073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.066+0000 7f266edd5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26681aa0a0 con 0x7f26681011e0 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.066+0000 7f266edd5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26681aa210 con 0x7f2668106300 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.066+0000 7f266edd5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26681aa3d0 con 0x7f2668074f60 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266cb71700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26681011e0 0x7f26680732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266cb71700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26681011e0 0x7f26680732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36058/0 (socket says 172.21.15.12:36058) 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266cb71700 1 -- 172.21.15.12:0/4176612161 learned_addr learned my addr 172.21.15.12:0/4176612161 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f2667fff700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2668106300 0x7f26680737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266d372700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2668074f60 0x7f2668073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266cb71700 1 -- 172.21.15.12:0/4176612161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2668074f60 msgr2=0x7f2668073d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266cb71700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2668074f60 0x7f2668073d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266cb71700 1 -- 172.21.15.12:0/4176612161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2668106300 msgr2=0x7f26680737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266cb71700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2668106300 0x7f26680737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266cb71700 1 -- 172.21.15.12:0/4176612161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c0095f0 con 0x7f26681011e0 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266cb71700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26681011e0 0x7f26680732a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f265c004760 tx=0x7f265c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f2665ffb700 1 -- 172.21.15.12:0/4176612161 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c01d070 con 0x7f26681011e0 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f2665ffb700 1 -- 172.21.15.12:0/4176612161 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f265c022470 con 0x7f26681011e0 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f266edd5700 1 -- 172.21.15.12:0/4176612161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26681aa650 con 0x7f26681011e0 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.067+0000 7f2665ffb700 1 -- 172.21.15.12:0/4176612161 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c017650 con 0x7f26681011e0 2024-05-08T23:54:14.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.068+0000 7f266edd5700 1 -- 172.21.15.12:0/4176612161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26681aaad0 con 0x7f26681011e0 2024-05-08T23:54:14.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.070+0000 7f2665ffb700 1 -- 172.21.15.12:0/4176612161 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f265c0225e0 con 0x7f26681011e0 2024-05-08T23:54:14.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.070+0000 7f2665ffb700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26580a1310 0x7f26580a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.071+0000 7f2665ffb700 1 -- 172.21.15.12:0/4176612161 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f265c0cdd60 con 0x7f26681011e0 2024-05-08T23:54:14.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.071+0000 7f2667fff700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26580a1310 0x7f26580a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.071+0000 7f2667fff700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26580a1310 0x7f26580a37d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2668074a80 tx=0x7f2654009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.071+0000 7f267af8fb80 1 Processor -- start 2024-05-08T23:54:14.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f267af8fb80 1 -- start start 2024-05-08T23:54:14.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af8e00 0x5622f9af91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af9720 0x5622f9af3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af3890 0x5622f9af3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f267af8fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f9a4a150 con 0x5622f9af9720 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f267af8fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f9a47b10 con 0x5622f9af8e00 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f267af8fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f9a43400 con 0x5622f9af3890 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f2667fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af9720 0x5622f9af3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f266cb71700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af8e00 0x5622f9af91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f2667fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af9720 0x5622f9af3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36074/0 (socket says 172.21.15.12:36074) 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f266cb71700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af8e00 0x5622f9af91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45826/0 (socket says 172.21.15.12:45826) 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f2667fff700 1 -- 172.21.15.12:0/1193185857 learned_addr learned my addr 172.21.15.12:0/1193185857 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f266d372700 1 --2- 172.21.15.12:0/1193185857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af3890 0x5622f9af3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f266cb71700 1 -- 172.21.15.12:0/1193185857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af3890 msgr2=0x5622f9af3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f266cb71700 1 --2- 172.21.15.12:0/1193185857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af3890 0x5622f9af3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f266cb71700 1 -- 172.21.15.12:0/1193185857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af9720 msgr2=0x5622f9af3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f266cb71700 1 --2- 172.21.15.12:0/1193185857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af9720 0x5622f9af3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f266cb71700 1 -- 172.21.15.12:0/1193185857 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622f98c43b0 con 0x5622f9af8e00 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.072+0000 7f2667fff700 1 --2- 172.21.15.12:0/1193185857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af9720 0x5622f9af3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.073+0000 7f266cb71700 1 --2- 172.21.15.12:0/1193185857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af8e00 0x5622f9af91e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f265c0cb4c0 tx=0x7f265c09adb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3261e19126b91de4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.073+0000 7f2652ffd700 1 -- 172.21.15.12:0/1193185857 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c09b610 con 0x5622f9af8e00 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.073+0000 7f2652ffd700 1 -- 172.21.15.12:0/1193185857 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f265c09af10 con 0x5622f9af8e00 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.073+0000 7f2652ffd700 1 -- 172.21.15.12:0/1193185857 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c09b3b0 con 0x5622f9af8e00 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.073+0000 7f267af8fb80 1 -- 172.21.15.12:0/1193185857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af8e00 msgr2=0x5622f9af91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.073+0000 7f267af8fb80 1 --2- 172.21.15.12:0/1193185857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af8e00 0x5622f9af91e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f265c0cb4c0 tx=0x7f265c09adb0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.074+0000 7f267af8fb80 1 -- 172.21.15.12:0/1193185857 shutdown_connections 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.074+0000 7f267af8fb80 1 --2- 172.21.15.12:0/1193185857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af3890 0x5622f9af3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.074+0000 7f267af8fb80 1 --2- 172.21.15.12:0/1193185857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af8e00 0x5622f9af91e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.074+0000 7f267af8fb80 1 --2- 172.21.15.12:0/1193185857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af9720 0x5622f9af3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.074+0000 7f267af8fb80 1 -- 172.21.15.12:0/1193185857 >> 172.21.15.12:0/1193185857 conn(0x5622f9a51210 msgr2=0x5622f9aeddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.074+0000 7f267af8fb80 1 -- 172.21.15.12:0/1193185857 shutdown_connections 2024-05-08T23:54:14.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.074+0000 7f267af8fb80 1 -- 172.21.15.12:0/1193185857 wait complete. 2024-05-08T23:54:14.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.074+0000 7f267af8fb80 1 -- start start 2024-05-08T23:54:14.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.074+0000 7f267af8fb80 1 Processor -- start 2024-05-08T23:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af3890 0x5622f9afcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9afcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 0x5622f9afd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f267af8fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f99df560 con 0x5622f9af8e00 2024-05-08T23:54:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f267af8fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f99df200 con 0x5622f9af9720 2024-05-08T23:54:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f267af8fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f98c66a0 con 0x5622f9af3890 2024-05-08T23:54:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f267af8fb80 10 client.0 _ll_register_callbacks cb 0x5622f99e8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f2667fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9afcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f266d372700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 0x5622f9afd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f2667fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9afcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36076/0 (socket says 172.21.15.12:36076) 2024-05-08T23:54:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f2667fff700 1 -- 172.21.15.12:0/3543833090 learned_addr learned my addr 172.21.15.12:0/3543833090 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f266cb71700 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af3890 0x5622f9afcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f266d372700 1 -- 172.21.15.12:0/3543833090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af3890 msgr2=0x5622f9afcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f266d372700 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af3890 0x5622f9afcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f266d372700 1 -- 172.21.15.12:0/3543833090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 msgr2=0x5622f9afcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f266d372700 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9afcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.075+0000 7f266d372700 1 -- 172.21.15.12:0/3543833090 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2654017070 con 0x5622f9af9720 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f266d372700 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 0x5622f9afd520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2660007af0 tx=0x7f2660007b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60d226e386dfbe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f2647fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f2647fff700 1 -- 172.21.15.12:0/3543833090 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266000e040 con 0x5622f9af9720 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f2647fff700 1 -- 172.21.15.12:0/3543833090 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2660004db0 con 0x5622f9af9720 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f267af8fb80 1 -- 172.21.15.12:0/3543833090 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622f999c270 con 0x5622f9af9720 2024-05-08T23:54:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f2647fff700 1 -- 172.21.15.12:0/3543833090 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2660016700 con 0x5622f9af9720 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f267af8fb80 10 client.27377 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f267af8fb80 1 -- 172.21.15.12:0/3543833090 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f26681aaad0 con 0x5622f9af9720 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f2647fff700 1 -- 172.21.15.12:0/3543833090 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5622f999c270 con 0x5622f9af9720 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f2647fff700 1 -- 172.21.15.12:0/3543833090 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f26600054e0 con 0x5622f9af9720 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f267af8fb80 10 client.27377 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f267af8fb80 10 client.27377 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f267af8fb80 20 client.27377 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f267af8fb80 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5622f9b0cf40 0x5622f9b0f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f267af8fb80 4 client.27377 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.076+0000 7f267af8fb80 1 -- 172.21.15.12:0/3543833090 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622f9ac2810 con 0x5622f9b0cf40 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.077+0000 7f266cb71700 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5622f9b0cf40 0x5622f9b0f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.077+0000 7f2647fff700 10 client.27377 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:54:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.077+0000 7f266cb71700 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5622f9b0cf40 0x5622f9b0f360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8302aa17bbd18338 server_cookie=dd7c4c56a10209d4 in_seq=0 out_seq=0 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f2647fff700 1 -- 172.21.15.12:0/3543833090 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622f9ac2810 con 0x5622f9b0cf40 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f2647fff700 10 client.27377 handle_command_reply: tid=0 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 client.27377 shutdown 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 -- 172.21.15.12:0/3543833090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 msgr2=0x5622f9afd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 0x5622f9afd520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2660007af0 tx=0x7f2660007b20 comp rx=0 tx=0).stop 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 -- 172.21.15.12:0/3543833090 shutdown_connections 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 0x5622f9afd520 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5622f9b0cf40 0x5622f9b0f360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af3890 0x5622f9afcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 --2- 172.21.15.12:0/3543833090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9afcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f2647fff700 0 client.27377 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 -- 172.21.15.12:0/3543833090 >> 172.21.15.12:0/3543833090 conn(0x5622f9a51210 msgr2=0x5622f9aed5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 -- 172.21.15.12:0/3543833090 shutdown_connections 2024-05-08T23:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 1 -- 172.21.15.12:0/3543833090 wait complete. 2024-05-08T23:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.078+0000 7f267af8fb80 20 client.27377 trim_cache size 0 max 16384 2024-05-08T23:54:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.093+0000 7f267af8fb80 1 Processor -- start 2024-05-08T23:54:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.093+0000 7f267af8fb80 1 -- start start 2024-05-08T23:54:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.093+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af3890 0x5622f9af3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.093+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af8e00 0x5622f9aedd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.093+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 0x5622f9b03ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.093+0000 7f267af8fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f9afe0d0 con 0x5622f9af3890 2024-05-08T23:54:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.093+0000 7f267af8fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f9aeeeb0 con 0x5622f9af9720 2024-05-08T23:54:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.093+0000 7f267af8fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f9aef0c0 con 0x5622f9af8e00 2024-05-08T23:54:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.093+0000 7f266cb71700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af3890 0x5622f9af3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f266d372700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 0x5622f9b03ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f2667fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af8e00 0x5622f9aedd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f266cb71700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af3890 0x5622f9af3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36088/0 (socket says 172.21.15.12:36088) 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f266cb71700 1 -- 172.21.15.12:0/4181136394 learned_addr learned my addr 172.21.15.12:0/4181136394 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f266d372700 1 -- 172.21.15.12:0/4181136394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af8e00 msgr2=0x5622f9aedd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f266d372700 1 --2- 172.21.15.12:0/4181136394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af8e00 0x5622f9aedd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f266d372700 1 -- 172.21.15.12:0/4181136394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af3890 msgr2=0x5622f9af3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f266d372700 1 --2- 172.21.15.12:0/4181136394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af3890 0x5622f9af3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f266d372700 1 -- 172.21.15.12:0/4181136394 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622f9870b60 con 0x5622f9af9720 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f2667fff700 1 --2- 172.21.15.12:0/4181136394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af8e00 0x5622f9aedd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f266d372700 1 --2- 172.21.15.12:0/4181136394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 0x5622f9b03ed0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f266000b010 tx=0x7f2660015390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71bc36b1b06660b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f26537fe700 1 -- 172.21.15.12:0/4181136394 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266000e040 con 0x5622f9af9720 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f26537fe700 1 -- 172.21.15.12:0/4181136394 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2660022070 con 0x5622f9af9720 2024-05-08T23:54:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.094+0000 7f26537fe700 1 -- 172.21.15.12:0/4181136394 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660016960 con 0x5622f9af9720 2024-05-08T23:54:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 -- 172.21.15.12:0/4181136394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 msgr2=0x5622f9b03ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 --2- 172.21.15.12:0/4181136394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 0x5622f9b03ed0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f266000b010 tx=0x7f2660015390 comp rx=0 tx=0).stop 2024-05-08T23:54:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 -- 172.21.15.12:0/4181136394 shutdown_connections 2024-05-08T23:54:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 --2- 172.21.15.12:0/4181136394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af9720 0x5622f9b03ed0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 --2- 172.21.15.12:0/4181136394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af3890 0x5622f9af3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 --2- 172.21.15.12:0/4181136394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af8e00 0x5622f9aedd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 -- 172.21.15.12:0/4181136394 >> 172.21.15.12:0/4181136394 conn(0x5622f9a51210 msgr2=0x5622f9afb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 -- 172.21.15.12:0/4181136394 shutdown_connections 2024-05-08T23:54:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 -- 172.21.15.12:0/4181136394 wait complete. 2024-05-08T23:54:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 -- start start 2024-05-08T23:54:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.095+0000 7f267af8fb80 1 Processor -- start 2024-05-08T23:54:14.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af3890 0x5622f9af6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9b03e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f267af8fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af9720 0x5622f9af49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f267af8fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f9b04770 con 0x5622f9af8e00 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f267af8fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f9af4ee0 con 0x5622f9af3890 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f267af8fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f9af50a0 con 0x5622f9af9720 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f267af8fb80 10 client.0 _ll_register_callbacks cb 0x5622f99e8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f2667fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9b03e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f2667fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9b03e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36098/0 (socket says 172.21.15.12:36098) 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f266cb71700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af3890 0x5622f9af6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f2667fff700 1 -- 172.21.15.12:0/2151302611 learned_addr learned my addr 172.21.15.12:0/2151302611 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f266d372700 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af9720 0x5622f9af49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f2667fff700 1 -- 172.21.15.12:0/2151302611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af9720 msgr2=0x5622f9af49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f2667fff700 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af9720 0x5622f9af49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f2667fff700 1 -- 172.21.15.12:0/2151302611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af3890 msgr2=0x5622f9af6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f2667fff700 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af3890 0x5622f9af6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f2667fff700 1 -- 172.21.15.12:0/2151302611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2654011c60 con 0x5622f9af8e00 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.096+0000 7f266cb71700 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af3890 0x5622f9af6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f2667fff700 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9b03e80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x5622f9aef9b0 tx=0x7f26540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46768bcd07cf9f22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f2647fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f2647fff700 1 -- 172.21.15.12:0/2151302611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265401e620 con 0x5622f9af8e00 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f2647fff700 1 -- 172.21.15.12:0/2151302611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f265401e780 con 0x5622f9af8e00 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f267af8fb80 1 -- 172.21.15.12:0/2151302611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f265c09af10 con 0x5622f9af8e00 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f2647fff700 1 -- 172.21.15.12:0/2151302611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2654004580 con 0x5622f9af8e00 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f267af8fb80 10 client.17595 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f267af8fb80 1 -- 172.21.15.12:0/2151302611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f26681aaad0 con 0x5622f9af8e00 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f2647fff700 1 -- 172.21.15.12:0/2151302611 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f265c09af10 con 0x5622f9af8e00 2024-05-08T23:54:14.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f2647fff700 1 -- 172.21.15.12:0/2151302611 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2654029a60 con 0x5622f9af8e00 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f267af8fb80 10 client.17595 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f267af8fb80 10 client.17595 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f267af8fb80 20 client.17595 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f267af8fb80 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5622f9b642c0 0x5622f9b666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f267af8fb80 4 client.17595 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.097+0000 7f267af8fb80 1 -- 172.21.15.12:0/2151302611 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5622f9a3bec0 con 0x5622f9b642c0 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f266cb71700 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5622f9b642c0 0x5622f9b666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f266cb71700 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5622f9b642c0 0x5622f9b666e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e98d2bc340373111 server_cookie=bec996084e0e88d9 in_seq=0 out_seq=0 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f2647fff700 10 client.17595 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f2647fff700 1 -- 172.21.15.12:0/2151302611 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5622f9a3bec0 con 0x5622f9b642c0 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f2647fff700 10 client.17595 handle_command_reply: tid=0 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f267af8fb80 1 client.17595 shutdown 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f267af8fb80 1 -- 172.21.15.12:0/2151302611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 msgr2=0x5622f9b03e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f267af8fb80 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9b03e80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x5622f9aef9b0 tx=0x7f26540047c0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f267af8fb80 1 -- 172.21.15.12:0/2151302611 shutdown_connections 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f267af8fb80 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5622f9b642c0 0x5622f9b666e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f267af8fb80 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622f9af9720 0x5622f9af49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f267af8fb80 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5622f9af3890 0x5622f9af6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f2647fff700 0 client.17595 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f267af8fb80 1 --2- 172.21.15.12:0/2151302611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622f9af8e00 0x5622f9b03e80 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.098+0000 7f267af8fb80 1 -- 172.21.15.12:0/2151302611 >> 172.21.15.12:0/2151302611 conn(0x5622f9a51210 msgr2=0x5622f9af7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f267af8fb80 1 -- 172.21.15.12:0/2151302611 shutdown_connections 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f267af8fb80 1 -- 172.21.15.12:0/2151302611 wait complete. 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f267af8fb80 20 client.17595 trim_cache size 0 max 16384 2024-05-08T23:54:14.100 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:54:14.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 -- 172.21.15.12:0/4176612161 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26580a1310 msgr2=0x7f26580a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26580a1310 0x7f26580a37d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2668074a80 tx=0x7f2654009260 comp rx=0 tx=0).stop 2024-05-08T23:54:14.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 -- 172.21.15.12:0/4176612161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26681011e0 msgr2=0x7f26680732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26681011e0 0x7f26680732a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f265c004760 tx=0x7f265c004790 comp rx=0 tx=0).stop 2024-05-08T23:54:14.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 -- 172.21.15.12:0/4176612161 shutdown_connections 2024-05-08T23:54:14.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26580a1310 0x7f26580a37d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2668074f60 0x7f2668073d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26681011e0 0x7f26680732a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 --2- 172.21.15.12:0/4176612161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2668106300 0x7f26680737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 -- 172.21.15.12:0/4176612161 >> 172.21.15.12:0/4176612161 conn(0x7f26680fca80 msgr2=0x7f26680fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.099+0000 7f266edd5700 1 -- 172.21.15.12:0/4176612161 shutdown_connections 2024-05-08T23:54:14.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.100+0000 7f266edd5700 1 -- 172.21.15.12:0/4176612161 wait complete. 2024-05-08T23:54:14.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:54:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.252+0000 7f6846ce7700 1 -- 172.21.15.12:0/2596511111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6840105160 msgr2=0x7f684010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.252+0000 7f6846ce7700 1 --2- 172.21.15.12:0/2596511111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6840105160 0x7f684010cce0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f683800b0a0 tx=0x7f683800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.252+0000 7f6846ce7700 1 -- 172.21.15.12:0/2596511111 shutdown_connections 2024-05-08T23:54:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.252+0000 7f6846ce7700 1 --2- 172.21.15.12:0/2596511111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6840105160 0x7f684010cce0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.252+0000 7f6846ce7700 1 --2- 172.21.15.12:0/2596511111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6840100060 0x7f6840100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.252+0000 7f6846ce7700 1 --2- 172.21.15.12:0/2596511111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6840100a10 0x7f6840104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.252+0000 7f6846ce7700 1 -- 172.21.15.12:0/2596511111 >> 172.21.15.12:0/2596511111 conn(0x7f6840076f40 msgr2=0x7f6840077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.252+0000 7f6846ce7700 1 -- 172.21.15.12:0/2596511111 shutdown_connections 2024-05-08T23:54:14.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.252+0000 7f6846ce7700 1 -- 172.21.15.12:0/2596511111 wait complete. 2024-05-08T23:54:14.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6846ce7700 1 Processor -- start 2024-05-08T23:54:14.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6846ce7700 1 -- start start 2024-05-08T23:54:14.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6846ce7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6840100060 0x7f6840196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6846ce7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6840105160 0x7f68401969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6846ce7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f684019ad00 0x7f684019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6846ce7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684019b780 con 0x7f684019ad00 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6846ce7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684019b8f0 con 0x7f6840105160 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6846ce7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684019bab0 con 0x7f6840100060 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6845284700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f684019ad00 0x7f684019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6845284700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f684019ad00 0x7f684019b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36116/0 (socket says 172.21.15.12:36116) 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f6845284700 1 -- 172.21.15.12:0/116147650 learned_addr learned my addr 172.21.15.12:0/116147650 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.253+0000 7f683ffff700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6840105160 0x7f68401969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f6844a83700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6840100060 0x7f6840196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f6845284700 1 -- 172.21.15.12:0/116147650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6840100060 msgr2=0x7f6840196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f6845284700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6840100060 0x7f6840196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f6845284700 1 -- 172.21.15.12:0/116147650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6840105160 msgr2=0x7f68401969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f6845284700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6840105160 0x7f68401969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f6845284700 1 -- 172.21.15.12:0/116147650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6838009d00 con 0x7f684019ad00 2024-05-08T23:54:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f683ffff700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6840105160 0x7f68401969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f6845284700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f684019ad00 0x7f684019b180 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f683800b070 tx=0x7f6838007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f683dffb700 1 -- 172.21.15.12:0/116147650 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683800e040 con 0x7f684019ad00 2024-05-08T23:54:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f683dffb700 1 -- 172.21.15.12:0/116147650 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f683800bdc0 con 0x7f684019ad00 2024-05-08T23:54:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f6846ce7700 1 -- 172.21.15.12:0/116147650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68401a2fb0 con 0x7f684019ad00 2024-05-08T23:54:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f683dffb700 1 -- 172.21.15.12:0/116147650 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68380052a0 con 0x7f684019ad00 2024-05-08T23:54:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.254+0000 7f6846ce7700 1 -- 172.21.15.12:0/116147650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68401a33d0 con 0x7f684019ad00 2024-05-08T23:54:14.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.257+0000 7f683dffb700 1 -- 172.21.15.12:0/116147650 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f683801d480 con 0x7f684019ad00 2024-05-08T23:54:14.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.257+0000 7f683dffb700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68300a1290 0x7f68300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.257+0000 7f683dffb700 1 -- 172.21.15.12:0/116147650 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f68380cdc50 con 0x7f684019ad00 2024-05-08T23:54:14.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.257+0000 7f6844a83700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68300a1290 0x7f68300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.258+0000 7f6844a83700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68300a1290 0x7f68300a3750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6834005950 tx=0x7f68340058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.258+0000 7f6852ea1b80 1 Processor -- start 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.258+0000 7f6852ea1b80 1 -- start start 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 0x56025fd9a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd94560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fd94f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fce9150 con 0x56025fd9a910 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fce6b10 con 0x56025fd99ff0 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fce2400 con 0x56025fd94aa0 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f683ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd94560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f683ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd94560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36126/0 (socket says 172.21.15.12:36126) 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f683ffff700 1 -- 172.21.15.12:0/397336607 learned_addr learned my addr 172.21.15.12:0/397336607 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f6845284700 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fd94f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f6844a83700 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 0x56025fd9a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f683ffff700 1 -- 172.21.15.12:0/397336607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 msgr2=0x56025fd94f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f683ffff700 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fd94f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f683ffff700 1 -- 172.21.15.12:0/397336607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 msgr2=0x56025fd9a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f683ffff700 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 0x56025fd9a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f683ffff700 1 -- 172.21.15.12:0/397336607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56025fb633b0 con 0x56025fd9a910 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f6845284700 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fd94f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f6844a83700 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 0x56025fd9a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f683ffff700 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd94560 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f682c0114f0 tx=0x7f682c011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5846bc47b701fce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.259+0000 7f682affd700 1 -- 172.21.15.12:0/397336607 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c004a30 con 0x56025fd9a910 2024-05-08T23:54:14.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f682affd700 1 -- 172.21.15.12:0/397336607 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f682c0054e0 con 0x56025fd9a910 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f682affd700 1 -- 172.21.15.12:0/397336607 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c00e8b0 con 0x56025fd9a910 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f6852ea1b80 1 -- 172.21.15.12:0/397336607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 msgr2=0x56025fd94560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd94560 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f682c0114f0 tx=0x7f682c011800 comp rx=0 tx=0).stop 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f6852ea1b80 1 -- 172.21.15.12:0/397336607 shutdown_connections 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fd94f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 0x56025fd9a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/397336607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd94560 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f6852ea1b80 1 -- 172.21.15.12:0/397336607 >> 172.21.15.12:0/397336607 conn(0x56025fcf0210 msgr2=0x56025fd8cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f6852ea1b80 1 -- 172.21.15.12:0/397336607 shutdown_connections 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.260+0000 7f6852ea1b80 1 -- 172.21.15.12:0/397336607 wait complete. 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.261+0000 7f6852ea1b80 1 -- start start 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.261+0000 7f6852ea1b80 1 Processor -- start 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.261+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fd9dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.261+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fd9e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.261+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 0x56025fd9e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.261+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fc7e560 con 0x56025fd99ff0 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.261+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fc7e200 con 0x56025fd9a910 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.261+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fb656a0 con 0x56025fd94aa0 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.261+0000 7f6852ea1b80 10 client.0 _ll_register_callbacks cb 0x56025fc87b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.261+0000 7f683ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fd9e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f6844a83700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fd9dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f683ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fd9e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36132/0 (socket says 172.21.15.12:36132) 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f683ffff700 1 -- 172.21.15.12:0/2521338408 learned_addr learned my addr 172.21.15.12:0/2521338408 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f6845284700 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 0x56025fd9e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f683ffff700 1 -- 172.21.15.12:0/2521338408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 msgr2=0x56025fd9dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f683ffff700 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fd9dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f683ffff700 1 -- 172.21.15.12:0/2521338408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 msgr2=0x56025fd9e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f683ffff700 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 0x56025fd9e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f683ffff700 1 -- 172.21.15.12:0/2521338408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68401a33d0 con 0x56025fd99ff0 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f6845284700 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 0x56025fd9e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f6844a83700 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fd9dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f683ffff700 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fd9e1c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f682c00e4b0 tx=0x7f682c004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b7332228229675 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f681ffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f681ffff700 1 -- 172.21.15.12:0/2521338408 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c004a20 con 0x56025fd99ff0 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f681ffff700 1 -- 172.21.15.12:0/2521338408 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f682c004b80 con 0x56025fd99ff0 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2521338408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56025fc3b270 con 0x56025fd99ff0 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f681ffff700 1 -- 172.21.15.12:0/2521338408 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f682c004020 con 0x56025fd99ff0 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.262+0000 7f6852ea1b80 10 client.17607 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.263+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2521338408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f682c009c90 con 0x56025fd99ff0 2024-05-08T23:54:14.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.263+0000 7f681ffff700 1 -- 172.21.15.12:0/2521338408 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56025fc3b270 con 0x56025fd99ff0 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.263+0000 7f681ffff700 1 -- 172.21.15.12:0/2521338408 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f682c021040 con 0x56025fd99ff0 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.263+0000 7f6852ea1b80 10 client.17607 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.263+0000 7f6852ea1b80 10 client.17607 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.263+0000 7f6852ea1b80 20 client.17607 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.263+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56025fdae460 0x56025fdb0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.263+0000 7f6852ea1b80 4 client.17607 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.263+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2521338408 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56025fd61810 con 0x56025fdae460 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.263+0000 7f6844a83700 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56025fdae460 0x56025fdb0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.264+0000 7f6844a83700 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56025fdae460 0x56025fdb0880 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e177448934b13c0 server_cookie=bcba5579679cb066 in_seq=0 out_seq=0 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.264+0000 7f681ffff700 10 client.17607 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.264+0000 7f681ffff700 1 -- 172.21.15.12:0/2521338408 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56025fd61810 con 0x56025fdae460 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.264+0000 7f681ffff700 10 client.17607 handle_command_reply: tid=0 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 client.17607 shutdown 2024-05-08T23:54:14.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2521338408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 msgr2=0x56025fd9e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fd9e1c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f682c00e4b0 tx=0x7f682c004df0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2521338408 shutdown_connections 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56025fdae460 0x56025fdb0880 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 0x56025fd9e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fd9dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/2521338408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fd9e1c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2521338408 >> 172.21.15.12:0/2521338408 conn(0x56025fcf0210 msgr2=0x56025fd8c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f681ffff700 0 client.17607 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2521338408 shutdown_connections 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2521338408 wait complete. 2024-05-08T23:54:14.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.265+0000 7f6852ea1b80 20 client.17607 trim_cache size 0 max 16384 2024-05-08T23:54:14.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.279+0000 7f6852ea1b80 1 Processor -- start 2024-05-08T23:54:14.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.279+0000 7f6852ea1b80 1 -- start start 2024-05-08T23:54:14.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.279+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fda9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 0x56025fd8cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd97df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fd9bd00 con 0x56025fd9a910 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fd95430 con 0x56025fd99ff0 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fdb5a30 con 0x56025fd94aa0 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6845284700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd97df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6845284700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd97df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36148/0 (socket says 172.21.15.12:36148) 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6845284700 1 -- 172.21.15.12:0/2302721540 learned_addr learned my addr 172.21.15.12:0/2302721540 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f683ffff700 1 --2- 172.21.15.12:0/2302721540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 0x56025fd8cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6844a83700 1 --2- 172.21.15.12:0/2302721540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fda9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6845284700 1 -- 172.21.15.12:0/2302721540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 msgr2=0x56025fda9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6845284700 1 --2- 172.21.15.12:0/2302721540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fda9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6845284700 1 -- 172.21.15.12:0/2302721540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 msgr2=0x56025fd8cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6845284700 1 --2- 172.21.15.12:0/2302721540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 0x56025fd8cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6845284700 1 -- 172.21.15.12:0/2302721540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56025fb0fb60 con 0x56025fd9a910 2024-05-08T23:54:14.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f6845284700 1 --2- 172.21.15.12:0/2302721540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd97df0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6838097300 tx=0x7f6838003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603c54de30dfcd21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f682b7fe700 1 -- 172.21.15.12:0/2302721540 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683809a940 con 0x56025fd9a910 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f682b7fe700 1 -- 172.21.15.12:0/2302721540 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f683809a760 con 0x56025fd9a910 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.280+0000 7f682b7fe700 1 -- 172.21.15.12:0/2302721540 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683809b060 con 0x56025fd9a910 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2302721540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 msgr2=0x56025fd97df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/2302721540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd97df0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6838097300 tx=0x7f6838003e10 comp rx=0 tx=0).stop 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2302721540 shutdown_connections 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/2302721540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd9a910 0x56025fd97df0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/2302721540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fda9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/2302721540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd99ff0 0x56025fd8cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2302721540 >> 172.21.15.12:0/2302721540 conn(0x56025fcf0210 msgr2=0x56025fd89210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2302721540 shutdown_connections 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 -- 172.21.15.12:0/2302721540 wait complete. 2024-05-08T23:54:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 -- start start 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.281+0000 7f6852ea1b80 1 Processor -- start 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fda5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fda3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f6852ea1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 0x56025fda3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fda5c60 con 0x56025fd99ff0 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fda3dc0 con 0x56025fd9a910 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f6852ea1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025fda3f80 con 0x56025fd94aa0 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f6852ea1b80 10 client.0 _ll_register_callbacks cb 0x56025fc87b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f683ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fda3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f683ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fda3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36154/0 (socket says 172.21.15.12:36154) 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f683ffff700 1 -- 172.21.15.12:0/3534613030 learned_addr learned my addr 172.21.15.12:0/3534613030 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f6845284700 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 0x56025fda3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f6844a83700 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fda5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f683ffff700 1 -- 172.21.15.12:0/3534613030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 msgr2=0x56025fda5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f683ffff700 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fda5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f683ffff700 1 -- 172.21.15.12:0/3534613030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 msgr2=0x56025fda3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.282+0000 7f683ffff700 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 0x56025fda3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f683ffff700 1 -- 172.21.15.12:0/3534613030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683802b070 con 0x56025fd99ff0 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f6845284700 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 0x56025fda3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f683ffff700 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fda3340 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x56025fd9f1c0 tx=0x7f682c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7ef82553b2c38d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f681ffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f681ffff700 1 -- 172.21.15.12:0/3534613030 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c00e660 con 0x56025fd99ff0 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f681ffff700 1 -- 172.21.15.12:0/3534613030 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f682c004c30 con 0x56025fd99ff0 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f6852ea1b80 1 -- 172.21.15.12:0/3534613030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f682c0054e0 con 0x56025fd99ff0 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f681ffff700 1 -- 172.21.15.12:0/3534613030 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f682c003680 con 0x56025fd99ff0 2024-05-08T23:54:14.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f6852ea1b80 10 client.17613 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f6852ea1b80 1 -- 172.21.15.12:0/3534613030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56025fca8560 con 0x56025fd99ff0 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f681ffff700 1 -- 172.21.15.12:0/3534613030 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f682c044020 con 0x56025fd99ff0 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f681ffff700 1 -- 172.21.15.12:0/3534613030 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f682c021070 con 0x56025fd99ff0 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f6852ea1b80 10 client.17613 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f6852ea1b80 10 client.17613 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f6852ea1b80 20 client.17613 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56025fe04940 0x56025fe06d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f6852ea1b80 4 client.17613 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.283+0000 7f6852ea1b80 1 -- 172.21.15.12:0/3534613030 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56025fcdaec0 con 0x56025fe04940 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.284+0000 7f6844a83700 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56025fe04940 0x56025fe06d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.284+0000 7f6844a83700 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56025fe04940 0x56025fe06d60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e44050c163edd70d server_cookie=26b3dac038ee267f in_seq=0 out_seq=0 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.284+0000 7f681ffff700 10 client.17613 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.284+0000 7f681ffff700 1 -- 172.21.15.12:0/3534613030 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56025fcdaec0 con 0x56025fe04940 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.284+0000 7f681ffff700 10 client.17613 handle_command_reply: tid=0 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.284+0000 7f6852ea1b80 1 client.17613 shutdown 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 1 -- 172.21.15.12:0/3534613030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 msgr2=0x56025fda3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fda3340 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x56025fd9f1c0 tx=0x7f682c005e50 comp rx=0 tx=0).stop 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 1 -- 172.21.15.12:0/3534613030 shutdown_connections 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56025fe04940 0x56025fe06d60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56025fd9a910 0x56025fda3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56025fd94aa0 0x56025fda5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f681ffff700 0 client.17613 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:54:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 1 --2- 172.21.15.12:0/3534613030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56025fd99ff0 0x56025fda3340 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 1 -- 172.21.15.12:0/3534613030 >> 172.21.15.12:0/3534613030 conn(0x56025fcf0210 msgr2=0x56025fd97df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 1 -- 172.21.15.12:0/3534613030 shutdown_connections 2024-05-08T23:54:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 1 -- 172.21.15.12:0/3534613030 wait complete. 2024-05-08T23:54:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6852ea1b80 20 client.17613 trim_cache size 0 max 16384 2024-05-08T23:54:14.287 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6846ce7700 1 -- 172.21.15.12:0/116147650 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68300a1290 msgr2=0x7f68300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6846ce7700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68300a1290 0x7f68300a3750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6834005950 tx=0x7f68340058e0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6846ce7700 1 -- 172.21.15.12:0/116147650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f684019ad00 msgr2=0x7f684019b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.285+0000 7f6846ce7700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f684019ad00 0x7f684019b180 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f683800b070 tx=0x7f6838007cf0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.286+0000 7f6846ce7700 1 -- 172.21.15.12:0/116147650 shutdown_connections 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.286+0000 7f6846ce7700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f68300a1290 0x7f68300a3750 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.286+0000 7f6846ce7700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f684019ad00 0x7f684019b180 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.286+0000 7f6846ce7700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6840100060 0x7f6840196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.286+0000 7f6846ce7700 1 --2- 172.21.15.12:0/116147650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6840105160 0x7f68401969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.286+0000 7f6846ce7700 1 -- 172.21.15.12:0/116147650 >> 172.21.15.12:0/116147650 conn(0x7f6840076f40 msgr2=0x7f68400fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.286+0000 7f6846ce7700 1 -- 172.21.15.12:0/116147650 shutdown_connections 2024-05-08T23:54:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.286+0000 7f6846ce7700 1 -- 172.21.15.12:0/116147650 wait complete. 2024-05-08T23:54:14.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:54:14.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.439+0000 7fe11c4e3700 1 -- 172.21.15.12:0/791706832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114106290 msgr2=0x7fe11410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.439+0000 7fe11c4e3700 1 --2- 172.21.15.12:0/791706832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114106290 0x7fe11410de10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00b0d0 tx=0x7fe10c00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.439+0000 7fe11c4e3700 1 -- 172.21.15.12:0/791706832 shutdown_connections 2024-05-08T23:54:14.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.439+0000 7fe11c4e3700 1 --2- 172.21.15.12:0/791706832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114106290 0x7fe11410de10 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.439+0000 7fe11c4e3700 1 --2- 172.21.15.12:0/791706832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1140682e0 0x7fe1140686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.439+0000 7fe11c4e3700 1 --2- 172.21.15.12:0/791706832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe114068c90 0x7fe114105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.439+0000 7fe11c4e3700 1 -- 172.21.15.12:0/791706832 >> 172.21.15.12:0/791706832 conn(0x7fe114076f40 msgr2=0x7fe114077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.439+0000 7fe11c4e3700 1 -- 172.21.15.12:0/791706832 shutdown_connections 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.439+0000 7fe11c4e3700 1 -- 172.21.15.12:0/791706832 wait complete. 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe11c4e3700 1 Processor -- start 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe11c4e3700 1 -- start start 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe11c4e3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1140682e0 0x7fe1141974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe11c4e3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114106290 0x7fe114197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe11c4e3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe11419bdc0 0x7fe11419c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe11c4e3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11419c840 con 0x7fe114106290 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe11c4e3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11419c9b0 con 0x7fe11419bdc0 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe11c4e3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11419cb70 con 0x7fe1140682e0 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe119a7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114106290 0x7fe114197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe119a7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114106290 0x7fe114197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36170/0 (socket says 172.21.15.12:36170) 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.440+0000 7fe119a7e700 1 -- 172.21.15.12:0/2719227660 learned_addr learned my addr 172.21.15.12:0/2719227660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe11a27f700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1140682e0 0x7fe1141974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe11aa80700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe11419bdc0 0x7fe11419c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe119a7e700 1 -- 172.21.15.12:0/2719227660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1140682e0 msgr2=0x7fe1141974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe119a7e700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1140682e0 0x7fe1141974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe119a7e700 1 -- 172.21.15.12:0/2719227660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe11419bdc0 msgr2=0x7fe11419c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe119a7e700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe11419bdc0 0x7fe11419c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe119a7e700 1 -- 172.21.15.12:0/2719227660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c009d00 con 0x7fe114106290 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe119a7e700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114106290 0x7fe114197a20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe11000e970 tx=0x7fe11000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe10b7fe700 1 -- 172.21.15.12:0/2719227660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110009790 con 0x7fe114106290 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe10b7fe700 1 -- 172.21.15.12:0/2719227660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe11000eec0 con 0x7fe114106290 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe10b7fe700 1 -- 172.21.15.12:0/2719227660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110005590 con 0x7fe114106290 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe11c4e3700 1 -- 172.21.15.12:0/2719227660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1141a40d0 con 0x7fe114106290 2024-05-08T23:54:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.441+0000 7fe11c4e3700 1 -- 172.21.15.12:0/2719227660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1141a4560 con 0x7fe114106290 2024-05-08T23:54:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.444+0000 7fe10b7fe700 1 -- 172.21.15.12:0/2719227660 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe11000cbd0 con 0x7fe114106290 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.444+0000 7fe10b7fe700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1000a1290 0x7fe1000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.444+0000 7fe10b7fe700 1 -- 172.21.15.12:0/2719227660 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe110014070 con 0x7fe114106290 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.444+0000 7fe11a27f700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1000a1290 0x7fe1000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.445+0000 7fe11a27f700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1000a1290 0x7fe1000a3750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe104005950 tx=0x7fe10400b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.445+0000 7fe12869db80 1 Processor -- start 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.445+0000 7fe12869db80 1 -- start start 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe12869db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe12869db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e40720 0x55ae64e3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe12869db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e3acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe12869db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64d91150 con 0x55ae64e3a890 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe12869db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64d8eb10 con 0x55ae64e40720 2024-05-08T23:54:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe12869db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64d8a400 con 0x55ae64e3fe00 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11aa80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e3acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11a27f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11aa80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e3acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36180/0 (socket says 172.21.15.12:36180) 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11aa80700 1 -- 172.21.15.12:0/3609608930 learned_addr learned my addr 172.21.15.12:0/3609608930 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe119a7e700 1 --2- 172.21.15.12:0/3609608930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e40720 0x55ae64e3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11aa80700 1 -- 172.21.15.12:0/3609608930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 msgr2=0x55ae64e401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11aa80700 1 --2- 172.21.15.12:0/3609608930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11aa80700 1 -- 172.21.15.12:0/3609608930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e40720 msgr2=0x55ae64e3a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11aa80700 1 --2- 172.21.15.12:0/3609608930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e40720 0x55ae64e3a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11aa80700 1 -- 172.21.15.12:0/3609608930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae64c0b3b0 con 0x55ae64e3a890 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11a27f700 1 --2- 172.21.15.12:0/3609608930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.446+0000 7fe11aa80700 1 --2- 172.21.15.12:0/3609608930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e3acd0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009c70 tx=0x7fe10c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ecf989bc3528bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.447+0000 7fe108ff9700 1 -- 172.21.15.12:0/3609608930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00e040 con 0x55ae64e3a890 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.447+0000 7fe108ff9700 1 -- 172.21.15.12:0/3609608930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10c007d00 con 0x55ae64e3a890 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.447+0000 7fe108ff9700 1 -- 172.21.15.12:0/3609608930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c007740 con 0x55ae64e3a890 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.447+0000 7fe12869db80 1 -- 172.21.15.12:0/3609608930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 msgr2=0x55ae64e3acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.447+0000 7fe12869db80 1 --2- 172.21.15.12:0/3609608930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e3acd0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009c70 tx=0x7fe10c0092e0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.447+0000 7fe12869db80 1 -- 172.21.15.12:0/3609608930 shutdown_connections 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.447+0000 7fe12869db80 1 --2- 172.21.15.12:0/3609608930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e3acd0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.447+0000 7fe12869db80 1 --2- 172.21.15.12:0/3609608930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.447+0000 7fe12869db80 1 --2- 172.21.15.12:0/3609608930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e40720 0x55ae64e3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.447+0000 7fe12869db80 1 -- 172.21.15.12:0/3609608930 >> 172.21.15.12:0/3609608930 conn(0x55ae64d98210 msgr2=0x55ae64e34df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.448+0000 7fe12869db80 1 -- 172.21.15.12:0/3609608930 shutdown_connections 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.448+0000 7fe12869db80 1 -- 172.21.15.12:0/3609608930 wait complete. 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.448+0000 7fe12869db80 1 -- start start 2024-05-08T23:54:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.448+0000 7fe12869db80 1 Processor -- start 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.448+0000 7fe12869db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3a890 0x55ae64e43a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.448+0000 7fe12869db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e43fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.448+0000 7fe12869db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e44510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.448+0000 7fe12869db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64d26560 con 0x55ae64e40720 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.448+0000 7fe12869db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64d26200 con 0x55ae64e3a890 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.448+0000 7fe12869db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64c0d6a0 con 0x55ae64e3fe00 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe12869db80 10 client.0 _ll_register_callbacks cb 0x55ae64d2fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe11aa80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e44510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe119a7e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e43fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe11aa80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e44510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36182/0 (socket says 172.21.15.12:36182) 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe11aa80700 1 -- 172.21.15.12:0/4110190287 learned_addr learned my addr 172.21.15.12:0/4110190287 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe11a27f700 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3a890 0x55ae64e43a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe119a7e700 1 -- 172.21.15.12:0/4110190287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3a890 msgr2=0x55ae64e43a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe119a7e700 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3a890 0x55ae64e43a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe119a7e700 1 -- 172.21.15.12:0/4110190287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 msgr2=0x55ae64e44510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe119a7e700 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e44510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe119a7e700 1 -- 172.21.15.12:0/4110190287 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe104010c50 con 0x55ae64e3fe00 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe11aa80700 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e44510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe119a7e700 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e43fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1100ca630 tx=0x7fe110099ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db78fd4dda9b9429 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe0fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe0fdffb700 1 -- 172.21.15.12:0/4110190287 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11009a450 con 0x55ae64e3fe00 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe0fdffb700 1 -- 172.21.15.12:0/4110190287 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe110016540 con 0x55ae64e3fe00 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.449+0000 7fe12869db80 1 -- 172.21.15.12:0/4110190287 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae64ce3270 con 0x55ae64e3fe00 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe0fdffb700 1 -- 172.21.15.12:0/4110190287 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1100a2470 con 0x55ae64e3fe00 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe12869db80 10 client.27403 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe12869db80 1 -- 172.21.15.12:0/4110190287 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fe11410e650 con 0x55ae64e3fe00 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe0fdffb700 1 -- 172.21.15.12:0/4110190287 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae64ce3270 con 0x55ae64e3fe00 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe0fdffb700 1 -- 172.21.15.12:0/4110190287 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe1100a1fe0 con 0x55ae64e3fe00 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe12869db80 10 client.27403 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe12869db80 10 client.27403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe12869db80 20 client.27403 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe12869db80 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ae64e54020 0x55ae64e56440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe12869db80 4 client.27403 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe12869db80 1 -- 172.21.15.12:0/4110190287 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae64e09810 con 0x55ae64e54020 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.450+0000 7fe11a27f700 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ae64e54020 0x55ae64e56440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.451+0000 7fe0fdffb700 10 client.27403 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.451+0000 7fe11a27f700 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ae64e54020 0x55ae64e56440 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b4b9530913b912 server_cookie=f0f5df93f87f2615 in_seq=0 out_seq=0 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.451+0000 7fe0fdffb700 1 -- 172.21.15.12:0/4110190287 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae64e09810 con 0x55ae64e54020 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.451+0000 7fe0fdffb700 10 client.27403 handle_command_reply: tid=0 2024-05-08T23:54:14.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.451+0000 7fe12869db80 1 client.27403 shutdown 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 1 -- 172.21.15.12:0/4110190287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 msgr2=0x55ae64e43fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e43fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1100ca630 tx=0x7fe110099ff0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 1 -- 172.21.15.12:0/4110190287 shutdown_connections 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e44510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ae64e54020 0x55ae64e56440 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3a890 0x55ae64e43a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 1 --2- 172.21.15.12:0/4110190287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e43fd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe0fdffb700 0 client.27403 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 1 -- 172.21.15.12:0/4110190287 >> 172.21.15.12:0/4110190287 conn(0x55ae64d98210 msgr2=0x55ae64e345a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 1 -- 172.21.15.12:0/4110190287 shutdown_connections 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 1 -- 172.21.15.12:0/4110190287 wait complete. 2024-05-08T23:54:14.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.452+0000 7fe12869db80 20 client.27403 trim_cache size 0 max 16384 2024-05-08T23:54:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe12869db80 1 Processor -- start 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe12869db80 1 -- start start 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe12869db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3a890 0x55ae64e3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe12869db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3fe00 0x55ae64e3ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe12869db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe12869db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64e31e10 con 0x55ae64e40720 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe12869db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64e40590 con 0x55ae64e3fe00 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe12869db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64e5b680 con 0x55ae64e3a890 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe11aa80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe119a7e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3fe00 0x55ae64e3ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe11aa80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e34d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36198/0 (socket says 172.21.15.12:36198) 2024-05-08T23:54:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe11aa80700 1 -- 172.21.15.12:0/1722912892 learned_addr learned my addr 172.21.15.12:0/1722912892 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe11a27f700 1 --2- 172.21.15.12:0/1722912892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3a890 0x55ae64e3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe11aa80700 1 -- 172.21.15.12:0/1722912892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3a890 msgr2=0x55ae64e3a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe11aa80700 1 --2- 172.21.15.12:0/1722912892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3a890 0x55ae64e3a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe11aa80700 1 -- 172.21.15.12:0/1722912892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3fe00 msgr2=0x55ae64e3ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe11aa80700 1 --2- 172.21.15.12:0/1722912892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3fe00 0x55ae64e3ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe11aa80700 1 -- 172.21.15.12:0/1722912892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae64bb7b60 con 0x55ae64e40720 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe11a27f700 1 --2- 172.21.15.12:0/1722912892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3a890 0x55ae64e3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.466+0000 7fe119a7e700 1 --2- 172.21.15.12:0/1722912892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3fe00 0x55ae64e3ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe11aa80700 1 --2- 172.21.15.12:0/1722912892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e34d20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009fd0 tx=0x7fe10c004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130f089fb3c271ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe1097fa700 1 -- 172.21.15.12:0/1722912892 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00e040 con 0x55ae64e40720 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe1097fa700 1 -- 172.21.15.12:0/1722912892 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10c007e70 con 0x55ae64e40720 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe1097fa700 1 -- 172.21.15.12:0/1722912892 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0256b0 con 0x55ae64e40720 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe12869db80 1 -- 172.21.15.12:0/1722912892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 msgr2=0x55ae64e34d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe12869db80 1 --2- 172.21.15.12:0/1722912892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e34d20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009fd0 tx=0x7fe10c004e60 comp rx=0 tx=0).stop 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe12869db80 1 -- 172.21.15.12:0/1722912892 shutdown_connections 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe12869db80 1 --2- 172.21.15.12:0/1722912892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e40720 0x55ae64e34d20 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe12869db80 1 --2- 172.21.15.12:0/1722912892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3a890 0x55ae64e3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe12869db80 1 --2- 172.21.15.12:0/1722912892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e3fe00 0x55ae64e3ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe12869db80 1 -- 172.21.15.12:0/1722912892 >> 172.21.15.12:0/1722912892 conn(0x55ae64d98210 msgr2=0x55ae64e31210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe12869db80 1 -- 172.21.15.12:0/1722912892 shutdown_connections 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.467+0000 7fe12869db80 1 -- 172.21.15.12:0/1722912892 wait complete. 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe12869db80 1 -- start start 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe12869db80 1 Processor -- start 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe12869db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e4b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe12869db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe12869db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e40720 0x55ae64e49630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe12869db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64e4ba50 con 0x55ae64e3a890 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe12869db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64e49ba0 con 0x55ae64e40720 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe12869db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae64e49d60 con 0x55ae64e3fe00 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe12869db80 10 client.0 _ll_register_callbacks cb 0x55ae64d2fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe11a27f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e4b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe11a27f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e4b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36210/0 (socket says 172.21.15.12:36210) 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe11a27f700 1 -- 172.21.15.12:0/778303572 learned_addr learned my addr 172.21.15.12:0/778303572 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe11aa80700 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e40720 0x55ae64e49630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe119a7e700 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe11a27f700 1 -- 172.21.15.12:0/778303572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 msgr2=0x55ae64e490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe11a27f700 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe11a27f700 1 -- 172.21.15.12:0/778303572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e40720 msgr2=0x55ae64e49630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.468+0000 7fe11a27f700 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e40720 0x55ae64e49630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe11a27f700 1 -- 172.21.15.12:0/778303572 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11410e650 con 0x55ae64e3a890 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe11a27f700 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e4b300 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe104005d90 tx=0x7fe104004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1847e270cda7a239 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe0fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe0fdffb700 1 -- 172.21.15.12:0/778303572 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104024040 con 0x55ae64e3a890 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe0fdffb700 1 -- 172.21.15.12:0/778303572 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe104017070 con 0x55ae64e3a890 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe12869db80 1 -- 172.21.15.12:0/778303572 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe10c007d00 con 0x55ae64e3a890 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe0fdffb700 1 -- 172.21.15.12:0/778303572 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe104011e10 con 0x55ae64e3a890 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe12869db80 10 client.17631 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe12869db80 1 -- 172.21.15.12:0/778303572 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55ae64c0b3b0 con 0x55ae64e3a890 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe0fdffb700 1 -- 172.21.15.12:0/778303572 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10c007d00 con 0x55ae64e3a890 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe0fdffb700 1 -- 172.21.15.12:0/778303572 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe10403acb0 con 0x55ae64e3a890 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe12869db80 10 client.17631 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe12869db80 10 client.17631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.469+0000 7fe12869db80 20 client.17631 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ae64eaee20 0x55ae64eb1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 4 client.17631 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 -- 172.21.15.12:0/778303572 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae64d82ec0 con 0x55ae64eaee20 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe119a7e700 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ae64eaee20 0x55ae64eb1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe119a7e700 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ae64eaee20 0x55ae64eb1240 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f23ecf082a1cbaee server_cookie=cf7257b3f9f1db4f in_seq=0 out_seq=0 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe0fdffb700 10 client.17631 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe0fdffb700 1 -- 172.21.15.12:0/778303572 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae64d82ec0 con 0x55ae64eaee20 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe0fdffb700 10 client.17631 handle_command_reply: tid=0 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 client.17631 shutdown 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 -- 172.21.15.12:0/778303572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 msgr2=0x55ae64e4b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e4b300 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe104005d90 tx=0x7fe104004750 comp rx=0 tx=0).stop 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 -- 172.21.15.12:0/778303572 shutdown_connections 2024-05-08T23:54:14.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ae64eaee20 0x55ae64eb1240 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ae64e40720 0x55ae64e49630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ae64e3a890 0x55ae64e4b300 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe0fdffb700 0 client.17631 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 --2- 172.21.15.12:0/778303572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ae64e3fe00 0x55ae64e490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 -- 172.21.15.12:0/778303572 >> 172.21.15.12:0/778303572 conn(0x55ae64d98210 msgr2=0x55ae64e42320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.470+0000 7fe12869db80 1 -- 172.21.15.12:0/778303572 shutdown_connections 2024-05-08T23:54:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe12869db80 1 -- 172.21.15.12:0/778303572 wait complete. 2024-05-08T23:54:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe12869db80 20 client.17631 trim_cache size 0 max 16384 2024-05-08T23:54:14.473 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:54:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 -- 172.21.15.12:0/2719227660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1000a1290 msgr2=0x7fe1000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1000a1290 0x7fe1000a3750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe104005950 tx=0x7fe10400b410 comp rx=0 tx=0).stop 2024-05-08T23:54:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 -- 172.21.15.12:0/2719227660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114106290 msgr2=0x7fe114197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114106290 0x7fe114197a20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe11000e970 tx=0x7fe11000ec80 comp rx=0 tx=0).stop 2024-05-08T23:54:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 -- 172.21.15.12:0/2719227660 shutdown_connections 2024-05-08T23:54:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1000a1290 0x7fe1000a3750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe11419bdc0 0x7fe11419c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1140682e0 0x7fe1141974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 --2- 172.21.15.12:0/2719227660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114106290 0x7fe114197a20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 -- 172.21.15.12:0/2719227660 >> 172.21.15.12:0/2719227660 conn(0x7fe114076f40 msgr2=0x7fe1140fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 -- 172.21.15.12:0/2719227660 shutdown_connections 2024-05-08T23:54:14.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:14.471+0000 7fe11c4e3700 1 -- 172.21.15.12:0/2719227660 wait complete. 2024-05-08T23:54:14.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:14 smithi012 ceph-mon[42503]: pgmap v245: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 595 KiB/s wr, 92 op/s 2024-05-08T23:54:14.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:14 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2556722747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:54:14.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:14 smithi204 ceph-mon[47373]: pgmap v245: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 595 KiB/s wr, 92 op/s 2024-05-08T23:54:14.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:14 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2556722747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:54:14.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:14 smithi160 ceph-mon[46152]: pgmap v245: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 595 KiB/s wr, 92 op/s 2024-05-08T23:54:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:14 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2556722747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:54:14.874 INFO:tasks.workunit.client.0.smithi012.stdout: 1 477780 8.29 MB/sec execute 250 sec latency 807.213 ms 2024-05-08T23:54:15.480 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:54:15.480 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:54:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.636+0000 7f382e8bf700 1 -- 172.21.15.12:0/3974918270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38280694b0 msgr2=0x7f3828069890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.636+0000 7f382e8bf700 1 --2- 172.21.15.12:0/3974918270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38280694b0 0x7f3828069890 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3818009990 tx=0x7f3818009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.637+0000 7f382e8bf700 1 -- 172.21.15.12:0/3974918270 shutdown_connections 2024-05-08T23:54:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.637+0000 7f382e8bf700 1 --2- 172.21.15.12:0/3974918270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f382810d940 0x7f382810fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.637+0000 7f382e8bf700 1 --2- 172.21.15.12:0/3974918270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38280694b0 0x7f3828069890 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.637+0000 7f382e8bf700 1 --2- 172.21.15.12:0/3974918270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3828069dd0 0x7f382810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.637+0000 7f382e8bf700 1 -- 172.21.15.12:0/3974918270 >> 172.21.15.12:0/3974918270 conn(0x7f3828076f70 msgr2=0x7f3828077380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.637+0000 7f382e8bf700 1 -- 172.21.15.12:0/3974918270 shutdown_connections 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.637+0000 7f382e8bf700 1 -- 172.21.15.12:0/3974918270 wait complete. 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.637+0000 7f382e8bf700 1 Processor -- start 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.637+0000 7f382e8bf700 1 -- start start 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f382e8bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38280694b0 0x7f3828199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f382e8bf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3828069dd0 0x7f3828199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f382e8bf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f382810d940 0x7f382819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f382e8bf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382819e4f0 con 0x7f38280694b0 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f382e8bf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382819e660 con 0x7f3828069dd0 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f382e8bf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382819e820 con 0x7f382810d940 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f3827fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38280694b0 0x7f3828199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f3827fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38280694b0 0x7f3828199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36226/0 (socket says 172.21.15.12:36226) 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f3827fff700 1 -- 172.21.15.12:0/27034520 learned_addr learned my addr 172.21.15.12:0/27034520 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f38277fe700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3828069dd0 0x7f3828199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f382ce5c700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f382810d940 0x7f382819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f3827fff700 1 -- 172.21.15.12:0/27034520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f382810d940 msgr2=0x7f382819deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f3827fff700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f382810d940 0x7f382819deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f3827fff700 1 -- 172.21.15.12:0/27034520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3828069dd0 msgr2=0x7f3828199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f3827fff700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3828069dd0 0x7f3828199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f3827fff700 1 -- 172.21.15.12:0/27034520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38180095f0 con 0x7f38280694b0 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f3827fff700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38280694b0 0x7f3828199620 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3818005910 tx=0x7f3818004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.638+0000 7f38257fa700 1 -- 172.21.15.12:0/27034520 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381801d070 con 0x7f38280694b0 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.639+0000 7f382e8bf700 1 -- 172.21.15.12:0/27034520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382819eaa0 con 0x7f38280694b0 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.639+0000 7f38257fa700 1 -- 172.21.15.12:0/27034520 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3818022470 con 0x7f38280694b0 2024-05-08T23:54:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.639+0000 7f38257fa700 1 -- 172.21.15.12:0/27034520 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3818017650 con 0x7f38280694b0 2024-05-08T23:54:15.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.639+0000 7f382e8bf700 1 -- 172.21.15.12:0/27034520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38281a6340 con 0x7f38280694b0 2024-05-08T23:54:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.641+0000 7f38257fa700 1 -- 172.21.15.12:0/27034520 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f38180225e0 con 0x7f38280694b0 2024-05-08T23:54:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.641+0000 7f38257fa700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38100a1290 0x7f38100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.642+0000 7f38257fa700 1 -- 172.21.15.12:0/27034520 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f38180cdff0 con 0x7f38280694b0 2024-05-08T23:54:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.642+0000 7f38277fe700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38100a1290 0x7f38100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.642+0000 7f38277fe700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38100a1290 0x7f38100a3750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f382819ab20 tx=0x7f381c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f383aa79b80 1 Processor -- start 2024-05-08T23:54:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f383aa79b80 1 -- start start 2024-05-08T23:54:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 0x557331c1d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c17d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f383aa79b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331b6e1d0 con 0x557331c1d7a0 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f383aa79b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331b6bb90 con 0x557331c1ce80 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f383aa79b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331b67480 con 0x557331c17910 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f38277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f38277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c173d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36232/0 (socket says 172.21.15.12:36232) 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f38277fe700 1 -- 172.21.15.12:0/2977218662 learned_addr learned my addr 172.21.15.12:0/2977218662 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f3827fff700 1 --2- 172.21.15.12:0/2977218662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 0x557331c1d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.643+0000 7f382ce5c700 1 --2- 172.21.15.12:0/2977218662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c17d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.644+0000 7f38277fe700 1 -- 172.21.15.12:0/2977218662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 msgr2=0x557331c17d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.644+0000 7f38277fe700 1 --2- 172.21.15.12:0/2977218662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c17d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.644+0000 7f38277fe700 1 -- 172.21.15.12:0/2977218662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 msgr2=0x557331c1d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.644+0000 7f38277fe700 1 --2- 172.21.15.12:0/2977218662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 0x557331c1d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.644+0000 7f38277fe700 1 -- 172.21.15.12:0/2977218662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573319e8430 con 0x557331c1d7a0 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.644+0000 7f3827fff700 1 --2- 172.21.15.12:0/2977218662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 0x557331c1d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.644+0000 7f38277fe700 1 --2- 172.21.15.12:0/2977218662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c173d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f381c011d50 tx=0x7f381c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10e49837dc24d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.644+0000 7f380e7fc700 1 -- 172.21.15.12:0/2977218662 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c00eb20 con 0x557331c1d7a0 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.644+0000 7f380e7fc700 1 -- 172.21.15.12:0/2977218662 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f381c021070 con 0x557331c1d7a0 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.644+0000 7f380e7fc700 1 -- 172.21.15.12:0/2977218662 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c01d670 con 0x557331c1d7a0 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.645+0000 7f383aa79b80 1 -- 172.21.15.12:0/2977218662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 msgr2=0x557331c173d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.645+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2977218662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c173d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f381c011d50 tx=0x7f381c014000 comp rx=0 tx=0).stop 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.645+0000 7f383aa79b80 1 -- 172.21.15.12:0/2977218662 shutdown_connections 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.645+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2977218662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c17d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.645+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2977218662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 0x557331c1d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.645+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2977218662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c173d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.645+0000 7f383aa79b80 1 -- 172.21.15.12:0/2977218662 >> 172.21.15.12:0/2977218662 conn(0x557331b75290 msgr2=0x557331c11e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.645+0000 7f383aa79b80 1 -- 172.21.15.12:0/2977218662 shutdown_connections 2024-05-08T23:54:15.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.645+0000 7f383aa79b80 1 -- 172.21.15.12:0/2977218662 wait complete. 2024-05-08T23:54:15.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.645+0000 7f383aa79b80 1 -- start start 2024-05-08T23:54:15.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f383aa79b80 1 Processor -- start 2024-05-08T23:54:15.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c20ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1d7a0 0x557331c21200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f383aa79b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331b035e0 con 0x557331c1ce80 2024-05-08T23:54:15.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f383aa79b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331b03280 con 0x557331c1d7a0 2024-05-08T23:54:15.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f383aa79b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573319ea720 con 0x557331c17910 2024-05-08T23:54:15.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f383aa79b80 10 client.0 _ll_register_callbacks cb 0x557331b0cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:15.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f38277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c20ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f38277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c20ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36240/0 (socket says 172.21.15.12:36240) 2024-05-08T23:54:15.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f38277fe700 1 -- 172.21.15.12:0/2172170907 learned_addr learned my addr 172.21.15.12:0/2172170907 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:15.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f3827fff700 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.646+0000 7f382ce5c700 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1d7a0 0x557331c21200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38277fe700 1 -- 172.21.15.12:0/2172170907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 msgr2=0x557331c237d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38277fe700 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c237d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38277fe700 1 -- 172.21.15.12:0/2172170907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1d7a0 msgr2=0x557331c21200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38277fe700 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1d7a0 0x557331c21200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38277fe700 1 -- 172.21.15.12:0/2172170907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38281a6340 con 0x557331c1ce80 2024-05-08T23:54:15.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f382ce5c700 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1d7a0 0x557331c21200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:54:15.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38277fe700 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c20ac0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f381c00e970 tx=0x7f381c011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb8703b2cc05ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38077fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:15.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38077fe700 1 -- 172.21.15.12:0/2172170907 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c00ed30 con 0x557331c1ce80 2024-05-08T23:54:15.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f383aa79b80 1 -- 172.21.15.12:0/2172170907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557331ac02f0 con 0x557331c1ce80 2024-05-08T23:54:15.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38077fe700 1 -- 172.21.15.12:0/2172170907 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f381c01c410 con 0x557331c1ce80 2024-05-08T23:54:15.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38077fe700 1 -- 172.21.15.12:0/2172170907 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f381c00ee90 con 0x557331c1ce80 2024-05-08T23:54:15.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f383aa79b80 10 client.17643 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:15.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f383aa79b80 1 -- 172.21.15.12:0/2172170907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f381c017070 con 0x557331c1ce80 2024-05-08T23:54:15.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38077fe700 1 -- 172.21.15.12:0/2172170907 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f381c03b020 con 0x557331c1ce80 2024-05-08T23:54:15.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.647+0000 7f38077fe700 1 -- 172.21.15.12:0/2172170907 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f381c003710 con 0x557331c1ce80 2024-05-08T23:54:15.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.648+0000 7f383aa79b80 10 client.17643 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:15.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.648+0000 7f383aa79b80 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:15.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.648+0000 7f383aa79b80 20 client.17643 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:15.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.648+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557331c31200 0x557331c33620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.648+0000 7f383aa79b80 4 client.17643 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:15.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.648+0000 7f383aa79b80 1 -- 172.21.15.12:0/2172170907 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557331be6890 con 0x557331c31200 2024-05-08T23:54:15.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.648+0000 7f3827fff700 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557331c31200 0x557331c33620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.648+0000 7f38077fe700 10 client.17643 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:15.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.648+0000 7f3827fff700 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557331c31200 0x557331c33620 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97dd10c1057c021b server_cookie=dcdb1730f4363a32 in_seq=0 out_seq=0 2024-05-08T23:54:15.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f38077fe700 1 -- 172.21.15.12:0/2172170907 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557331be6890 con 0x557331c31200 2024-05-08T23:54:15.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f38077fe700 10 client.17643 handle_command_reply: tid=0 2024-05-08T23:54:15.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 client.17643 shutdown 2024-05-08T23:54:15.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 -- 172.21.15.12:0/2172170907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 msgr2=0x557331c20ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c20ac0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f381c00e970 tx=0x7f381c011e30 comp rx=0 tx=0).stop 2024-05-08T23:54:15.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 -- 172.21.15.12:0/2172170907 shutdown_connections 2024-05-08T23:54:15.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557331c31200 0x557331c33620 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1d7a0 0x557331c21200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f38077fe700 0 client.17643 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:15.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2172170907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c20ac0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 -- 172.21.15.12:0/2172170907 >> 172.21.15.12:0/2172170907 conn(0x557331b75290 msgr2=0x557331c11620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 -- 172.21.15.12:0/2172170907 shutdown_connections 2024-05-08T23:54:15.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 1 -- 172.21.15.12:0/2172170907 wait complete. 2024-05-08T23:54:15.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.649+0000 7f383aa79b80 20 client.17643 trim_cache size 0 max 16384 2024-05-08T23:54:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f383aa79b80 1 Processor -- start 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f383aa79b80 1 -- start start 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c17910 0x557331c173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c11da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c1d7a0 0x557331c281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f383aa79b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331c0cb10 con 0x557331c1ce80 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f383aa79b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331c0ee40 con 0x557331c17910 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f383aa79b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331c0a150 con 0x557331c1d7a0 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f38277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c11da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f38277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c11da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36244/0 (socket says 172.21.15.12:36244) 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f38277fe700 1 -- 172.21.15.12:0/3152372949 learned_addr learned my addr 172.21.15.12:0/3152372949 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f3827fff700 1 --2- 172.21.15.12:0/3152372949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c17910 0x557331c173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f382ce5c700 1 --2- 172.21.15.12:0/3152372949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c1d7a0 0x557331c281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f38277fe700 1 -- 172.21.15.12:0/3152372949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c1d7a0 msgr2=0x557331c281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f38277fe700 1 --2- 172.21.15.12:0/3152372949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c1d7a0 0x557331c281e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f38277fe700 1 -- 172.21.15.12:0/3152372949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c17910 msgr2=0x557331c173d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f38277fe700 1 --2- 172.21.15.12:0/3152372949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c17910 0x557331c173d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f38277fe700 1 -- 172.21.15.12:0/3152372949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557331994be0 con 0x557331c1ce80 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.669+0000 7f382ce5c700 1 --2- 172.21.15.12:0/3152372949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c1d7a0 0x557331c281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.670+0000 7f38277fe700 1 --2- 172.21.15.12:0/3152372949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c11da0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f381c009b20 tx=0x7f381c011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b93632f445e134c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.670+0000 7f380effd700 1 -- 172.21.15.12:0/3152372949 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c01cce0 con 0x557331c1ce80 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.670+0000 7f380effd700 1 -- 172.21.15.12:0/3152372949 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f381c03b070 con 0x557331c1ce80 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.670+0000 7f380effd700 1 -- 172.21.15.12:0/3152372949 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c00ee30 con 0x557331c1ce80 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.670+0000 7f383aa79b80 1 -- 172.21.15.12:0/3152372949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 msgr2=0x557331c11da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.670+0000 7f383aa79b80 1 --2- 172.21.15.12:0/3152372949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c11da0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f381c009b20 tx=0x7f381c011cd0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.670+0000 7f383aa79b80 1 -- 172.21.15.12:0/3152372949 shutdown_connections 2024-05-08T23:54:15.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.670+0000 7f383aa79b80 1 --2- 172.21.15.12:0/3152372949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c1d7a0 0x557331c281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.670+0000 7f383aa79b80 1 --2- 172.21.15.12:0/3152372949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c17910 0x557331c173d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.670+0000 7f383aa79b80 1 --2- 172.21.15.12:0/3152372949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1ce80 0x557331c11da0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.671+0000 7f383aa79b80 1 -- 172.21.15.12:0/3152372949 >> 172.21.15.12:0/3152372949 conn(0x557331b75290 msgr2=0x557331c37f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.671+0000 7f383aa79b80 1 -- 172.21.15.12:0/3152372949 shutdown_connections 2024-05-08T23:54:15.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.671+0000 7f383aa79b80 1 -- 172.21.15.12:0/3152372949 wait complete. 2024-05-08T23:54:15.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.671+0000 7f383aa79b80 1 -- start start 2024-05-08T23:54:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.671+0000 7f383aa79b80 1 Processor -- start 2024-05-08T23:54:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c28200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 0x557331c28740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f383aa79b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f383aa79b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331c250f0 con 0x557331c1d7a0 2024-05-08T23:54:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f383aa79b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331c25260 con 0x557331c1ce80 2024-05-08T23:54:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f383aa79b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557331c25420 con 0x557331c17910 2024-05-08T23:54:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f383aa79b80 10 client.0 _ll_register_callbacks cb 0x557331b0cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f382ce5c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f382ce5c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c249e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36250/0 (socket says 172.21.15.12:36250) 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f3827fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c28200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f382ce5c700 1 -- 172.21.15.12:0/2718818550 learned_addr learned my addr 172.21.15.12:0/2718818550 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f38277fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 0x557331c28740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f382ce5c700 1 -- 172.21.15.12:0/2718818550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 msgr2=0x557331c28200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f382ce5c700 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c28200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f382ce5c700 1 -- 172.21.15.12:0/2718818550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 msgr2=0x557331c28740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f382ce5c700 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 0x557331c28740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f382ce5c700 1 -- 172.21.15.12:0/2718818550 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c017070 con 0x557331c1d7a0 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f382ce5c700 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c249e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f3820009550 tx=0x7f3820007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67ec4e71f69e828 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.672+0000 7f38077fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f38077fe700 1 -- 172.21.15.12:0/2718818550 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382000e070 con 0x557331c1d7a0 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f38077fe700 1 -- 172.21.15.12:0/2718818550 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3820014b40 con 0x557331c1d7a0 2024-05-08T23:54:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f383aa79b80 1 -- 172.21.15.12:0/2718818550 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f381c021070 con 0x557331c1d7a0 2024-05-08T23:54:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f38077fe700 1 -- 172.21.15.12:0/2718818550 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3820014d00 con 0x557331c1d7a0 2024-05-08T23:54:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f383aa79b80 10 client.17649 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f383aa79b80 1 -- 172.21.15.12:0/2718818550 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557331b2d5e0 con 0x557331c1d7a0 2024-05-08T23:54:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f38077fe700 1 -- 172.21.15.12:0/2718818550 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f381c021070 con 0x557331c1d7a0 2024-05-08T23:54:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f38077fe700 1 -- 172.21.15.12:0/2718818550 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f382001f4b0 con 0x557331c1d7a0 2024-05-08T23:54:15.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f383aa79b80 10 client.17649 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:15.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f383aa79b80 10 client.17649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:15.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f383aa79b80 20 client.17649 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:15.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557331c88b20 0x557331c8af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f383aa79b80 4 client.17649 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:54:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f383aa79b80 1 -- 172.21.15.12:0/2718818550 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557331b5ff40 con 0x557331c88b20 2024-05-08T23:54:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.673+0000 7f3827fff700 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557331c88b20 0x557331c8af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.674+0000 7f3827fff700 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557331c88b20 0x557331c8af40 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99e03ed63b26654 server_cookie=e7ca31a077db39ca in_seq=0 out_seq=0 2024-05-08T23:54:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.674+0000 7f38077fe700 10 client.17649 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f38077fe700 1 -- 172.21.15.12:0/2718818550 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557331b5ff40 con 0x557331c88b20 2024-05-08T23:54:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f38077fe700 10 client.17649 handle_command_reply: tid=0 2024-05-08T23:54:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 client.17649 shutdown 2024-05-08T23:54:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 -- 172.21.15.12:0/2718818550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 msgr2=0x557331c249e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c249e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f3820009550 tx=0x7f3820007e30 comp rx=0 tx=0).stop 2024-05-08T23:54:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 -- 172.21.15.12:0/2718818550 shutdown_connections 2024-05-08T23:54:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557331c1d7a0 0x557331c249e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557331c17910 0x557331c28200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557331c88b20 0x557331c8af40 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 --2- 172.21.15.12:0/2718818550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557331c1ce80 0x557331c28740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 -- 172.21.15.12:0/2718818550 >> 172.21.15.12:0/2718818550 conn(0x557331b75290 msgr2=0x557331c1c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f38077fe700 0 client.17649 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 -- 172.21.15.12:0/2718818550 shutdown_connections 2024-05-08T23:54:15.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.675+0000 7f383aa79b80 1 -- 172.21.15.12:0/2718818550 wait complete. 2024-05-08T23:54:15.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f383aa79b80 20 client.17649 trim_cache size 0 max 16384 2024-05-08T23:54:15.680 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:54:15.681 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:54:15.681 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "8057275f-ecb7-444f-a8cb-bb524c2e4eb5", 2024-05-08T23:54:15.681 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:54:15.681 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:54:15.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 -- 172.21.15.12:0/27034520 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38100a1290 msgr2=0x7f38100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38100a1290 0x7f38100a3750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f382819ab20 tx=0x7f381c009260 comp rx=0 tx=0).stop 2024-05-08T23:54:15.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 -- 172.21.15.12:0/27034520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38280694b0 msgr2=0x7f3828199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38280694b0 0x7f3828199620 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3818005910 tx=0x7f3818004730 comp rx=0 tx=0).stop 2024-05-08T23:54:15.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 -- 172.21.15.12:0/27034520 shutdown_connections 2024-05-08T23:54:15.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38100a1290 0x7f38100a3750 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f382810d940 0x7f382819deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38280694b0 0x7f3828199620 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 --2- 172.21.15.12:0/27034520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3828069dd0 0x7f3828199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 -- 172.21.15.12:0/27034520 >> 172.21.15.12:0/27034520 conn(0x7f3828076f70 msgr2=0x7f382810eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 -- 172.21.15.12:0/27034520 shutdown_connections 2024-05-08T23:54:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.676+0000 7f382e8bf700 1 -- 172.21.15.12:0/27034520 wait complete. 2024-05-08T23:54:15.687 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:54:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.841+0000 7f1607821700 1 -- 172.21.15.12:0/2344483597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16001062a0 msgr2=0x7f160010de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.841+0000 7f1607821700 1 --2- 172.21.15.12:0/2344483597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16001062a0 0x7f160010de20 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f15f800b0a0 tx=0x7f15f800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.841+0000 7f1607821700 1 -- 172.21.15.12:0/2344483597 shutdown_connections 2024-05-08T23:54:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.841+0000 7f1607821700 1 --2- 172.21.15.12:0/2344483597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16001062a0 0x7f160010de20 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.841+0000 7f1607821700 1 --2- 172.21.15.12:0/2344483597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1600069420 0x7f1600069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.841+0000 7f1607821700 1 --2- 172.21.15.12:0/2344483597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1600069dd0 0x7f1600105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.841+0000 7f1607821700 1 -- 172.21.15.12:0/2344483597 >> 172.21.15.12:0/2344483597 conn(0x7f1600076ba0 msgr2=0x7f1600076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.841+0000 7f1607821700 1 -- 172.21.15.12:0/2344483597 shutdown_connections 2024-05-08T23:54:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.841+0000 7f1607821700 1 -- 172.21.15.12:0/2344483597 wait complete. 2024-05-08T23:54:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.841+0000 7f1607821700 1 Processor -- start 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1607821700 1 -- start start 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1607821700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1600069420 0x7f16001001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1607821700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16001062a0 0x7f1600100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1607821700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1600100c40 0x7f16001a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1607821700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16001a5d10 con 0x7f1600100c40 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1607821700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16001a5e50 con 0x7f16001062a0 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1607821700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16001a6010 con 0x7f1600069420 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1605dbe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1600100c40 0x7f16001a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1605dbe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1600100c40 0x7f16001a5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36274/0 (socket says 172.21.15.12:36274) 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1604dbc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16001062a0 0x7f1600100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1605dbe700 1 -- 172.21.15.12:0/1586192886 learned_addr learned my addr 172.21.15.12:0/1586192886 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f16055bd700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1600069420 0x7f16001001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1605dbe700 1 -- 172.21.15.12:0/1586192886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1600069420 msgr2=0x7f16001001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1605dbe700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1600069420 0x7f16001001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1605dbe700 1 -- 172.21.15.12:0/1586192886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16001062a0 msgr2=0x7f1600100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1605dbe700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16001062a0 0x7f1600100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1605dbe700 1 -- 172.21.15.12:0/1586192886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f8009d00 con 0x7f1600100c40 2024-05-08T23:54:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.842+0000 7f1605dbe700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1600100c40 0x7f16001a5740 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f15f800b070 tx=0x7f15f800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.843+0000 7f15f67fc700 1 -- 172.21.15.12:0/1586192886 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f800e040 con 0x7f1600100c40 2024-05-08T23:54:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.843+0000 7f1607821700 1 -- 172.21.15.12:0/1586192886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16001a6290 con 0x7f1600100c40 2024-05-08T23:54:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.843+0000 7f15f67fc700 1 -- 172.21.15.12:0/1586192886 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15f801d480 con 0x7f1600100c40 2024-05-08T23:54:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.843+0000 7f15f67fc700 1 -- 172.21.15.12:0/1586192886 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f80052a0 con 0x7f1600100c40 2024-05-08T23:54:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.843+0000 7f1607821700 1 -- 172.21.15.12:0/1586192886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16001a6710 con 0x7f1600100c40 2024-05-08T23:54:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.846+0000 7f15f67fc700 1 -- 172.21.15.12:0/1586192886 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f15f801d5f0 con 0x7f1600100c40 2024-05-08T23:54:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.846+0000 7f15f67fc700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15ec0a1310 0x7f15ec0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.846+0000 7f15f67fc700 1 -- 172.21.15.12:0/1586192886 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f15f80cdb30 con 0x7f1600100c40 2024-05-08T23:54:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.846+0000 7f16055bd700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15ec0a1310 0x7f15ec0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.847+0000 7f16055bd700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15ec0a1310 0x7f15ec0a37d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009bf0 tx=0x7f15fc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.847+0000 7f16139dbb80 1 Processor -- start 2024-05-08T23:54:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16139dbb80 1 -- start start 2024-05-08T23:54:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f62820f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 0x55f628209350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f628209890 0x55f628209cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16139dbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f628160150 con 0x55f62820ee00 2024-05-08T23:54:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16139dbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f62815db10 con 0x55f628209890 2024-05-08T23:54:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16139dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f628159400 con 0x55f62820f720 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16055bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f62820f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f1604dbc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 0x55f628209350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16055bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f62820f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36280/0 (socket says 172.21.15.12:36280) 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16055bd700 1 -- 172.21.15.12:0/3511882975 learned_addr learned my addr 172.21.15.12:0/3511882975 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f1605dbe700 1 --2- 172.21.15.12:0/3511882975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f628209890 0x55f628209cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16055bd700 1 -- 172.21.15.12:0/3511882975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 msgr2=0x55f628209350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16055bd700 1 --2- 172.21.15.12:0/3511882975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 0x55f628209350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16055bd700 1 -- 172.21.15.12:0/3511882975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f628209890 msgr2=0x55f628209cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16055bd700 1 --2- 172.21.15.12:0/3511882975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f628209890 0x55f628209cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f16055bd700 1 -- 172.21.15.12:0/3511882975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f627fda3b0 con 0x55f62820ee00 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f1604dbc700 1 --2- 172.21.15.12:0/3511882975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 0x55f628209350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.848+0000 7f1605dbe700 1 --2- 172.21.15.12:0/3511882975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f628209890 0x55f628209cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.849+0000 7f16055bd700 1 --2- 172.21.15.12:0/3511882975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f62820f1e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f15fc011e40 tx=0x7f15fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f18941c10e7d47a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.849+0000 7f15eb7fe700 1 -- 172.21.15.12:0/3511882975 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc012dc0 con 0x55f62820ee00 2024-05-08T23:54:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.849+0000 7f15eb7fe700 1 -- 172.21.15.12:0/3511882975 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15fc016070 con 0x55f62820ee00 2024-05-08T23:54:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.849+0000 7f15eb7fe700 1 -- 172.21.15.12:0/3511882975 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc003d20 con 0x55f62820ee00 2024-05-08T23:54:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 -- 172.21.15.12:0/3511882975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 msgr2=0x55f62820f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 --2- 172.21.15.12:0/3511882975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f62820f1e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f15fc011e40 tx=0x7f15fc014000 comp rx=0 tx=0).stop 2024-05-08T23:54:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 -- 172.21.15.12:0/3511882975 shutdown_connections 2024-05-08T23:54:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 --2- 172.21.15.12:0/3511882975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f628209890 0x55f628209cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 --2- 172.21.15.12:0/3511882975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f62820f1e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 --2- 172.21.15.12:0/3511882975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 0x55f628209350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 -- 172.21.15.12:0/3511882975 >> 172.21.15.12:0/3511882975 conn(0x55f628167210 msgr2=0x55f628203df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 -- 172.21.15.12:0/3511882975 shutdown_connections 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 -- 172.21.15.12:0/3511882975 wait complete. 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 -- start start 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.850+0000 7f16139dbb80 1 Processor -- start 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 0x55f628212b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f6282130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820f720 0x55f6282135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f16139dbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6280f5560 con 0x55f62820f720 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f16139dbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6280f5200 con 0x55f62820ee00 2024-05-08T23:54:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f16139dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f627fdc6a0 con 0x55f628209890 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f16139dbb80 10 client.0 _ll_register_callbacks cb 0x55f6280feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f1604dbc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f6282130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f1604dbc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f6282130b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:46052/0 (socket says 172.21.15.12:46052) 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f1604dbc700 1 -- 172.21.15.12:0/2377862339 learned_addr learned my addr 172.21.15.12:0/2377862339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f16055bd700 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 0x55f628212b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f1604dbc700 1 -- 172.21.15.12:0/2377862339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 msgr2=0x55f628212b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f1604dbc700 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 0x55f628212b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f1604dbc700 1 -- 172.21.15.12:0/2377862339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820f720 msgr2=0x55f6282135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f1604dbc700 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820f720 0x55f6282135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.851+0000 7f1604dbc700 1 -- 172.21.15.12:0/2377862339 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f00098e0 con 0x55f62820ee00 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.852+0000 7f1604dbc700 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f6282130b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f15f00119a0 tx=0x7f15f0011cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64530e738069fdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.852+0000 7f15e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.852+0000 7f15e8ff9700 1 -- 172.21.15.12:0/2377862339 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0016070 con 0x55f62820ee00 2024-05-08T23:54:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.852+0000 7f15e8ff9700 1 -- 172.21.15.12:0/2377862339 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f15f0004300 con 0x55f62820ee00 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.852+0000 7f16139dbb80 1 -- 172.21.15.12:0/2377862339 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6280b2270 con 0x55f62820ee00 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.852+0000 7f15e8ff9700 1 -- 172.21.15.12:0/2377862339 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15f00044c0 con 0x55f62820ee00 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.852+0000 7f16139dbb80 10 client.27446 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.852+0000 7f16139dbb80 1 -- 172.21.15.12:0/2377862339 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f15fc011b20 con 0x55f62820ee00 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.852+0000 7f15e8ff9700 1 -- 172.21.15.12:0/2377862339 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6280b2270 con 0x55f62820ee00 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.852+0000 7f15e8ff9700 1 -- 172.21.15.12:0/2377862339 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f15f0003b30 con 0x55f62820ee00 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.853+0000 7f16139dbb80 10 client.27446 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.853+0000 7f16139dbb80 10 client.27446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.853+0000 7f16139dbb80 20 client.27446 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.853+0000 7f16139dbb80 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f628223450 0x55f628225870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.853+0000 7f16139dbb80 4 client.27446 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.853+0000 7f16139dbb80 1 -- 172.21.15.12:0/2377862339 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6281d8810 con 0x55f628223450 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.853+0000 7f16055bd700 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f628223450 0x55f628225870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.853+0000 7f15e8ff9700 10 client.27446 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.853+0000 7f16055bd700 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f628223450 0x55f628225870 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=243949a9c941aff3 server_cookie=8ddf0742234f3e in_seq=0 out_seq=0 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f15e8ff9700 1 -- 172.21.15.12:0/2377862339 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f6281d8810 con 0x55f628223450 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f15e8ff9700 10 client.27446 handle_command_reply: tid=0 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 client.27446 shutdown 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 -- 172.21.15.12:0/2377862339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 msgr2=0x55f6282130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f6282130b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f15f00119a0 tx=0x7f15f0011cb0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 -- 172.21.15.12:0/2377862339 shutdown_connections 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820f720 0x55f6282135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 0x55f628212b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f628223450 0x55f628225870 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 --2- 172.21.15.12:0/2377862339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f6282130b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f15e8ff9700 0 client.27446 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 -- 172.21.15.12:0/2377862339 >> 172.21.15.12:0/2377862339 conn(0x55f628167210 msgr2=0x55f6282035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 -- 172.21.15.12:0/2377862339 shutdown_connections 2024-05-08T23:54:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.854+0000 7f16139dbb80 1 -- 172.21.15.12:0/2377862339 wait complete. 2024-05-08T23:54:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.855+0000 7f16139dbb80 20 client.27446 trim_cache size 0 max 16384 2024-05-08T23:54:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f16139dbb80 1 Processor -- start 2024-05-08T23:54:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f16139dbb80 1 -- start start 2024-05-08T23:54:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 0x55f62821ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f628203d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820f720 0x55f628211460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f16139dbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f628220970 con 0x55f62820ee00 2024-05-08T23:54:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f16139dbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f62821fde0 con 0x55f62820f720 2024-05-08T23:54:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f16139dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f628214170 con 0x55f628209890 2024-05-08T23:54:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f1604dbc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f628203d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f1604dbc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f628203d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36296/0 (socket says 172.21.15.12:36296) 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f1604dbc700 1 -- 172.21.15.12:0/952336119 learned_addr learned my addr 172.21.15.12:0/952336119 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f16055bd700 1 --2- 172.21.15.12:0/952336119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 0x55f62821ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.874+0000 7f1605dbe700 1 --2- 172.21.15.12:0/952336119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820f720 0x55f628211460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.875+0000 7f1604dbc700 1 -- 172.21.15.12:0/952336119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 msgr2=0x55f62821ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.875+0000 7f1604dbc700 1 --2- 172.21.15.12:0/952336119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 0x55f62821ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.875+0000 7f1604dbc700 1 -- 172.21.15.12:0/952336119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820f720 msgr2=0x55f628211460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.875+0000 7f1604dbc700 1 --2- 172.21.15.12:0/952336119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820f720 0x55f628211460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.875+0000 7f1604dbc700 1 -- 172.21.15.12:0/952336119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f627f86b60 con 0x55f62820ee00 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.875+0000 7f16055bd700 1 --2- 172.21.15.12:0/952336119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 0x55f62821ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.875+0000 7f1604dbc700 1 --2- 172.21.15.12:0/952336119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f628203d20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009340 tx=0x7f15f0009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fd083f0a61e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.875+0000 7f15ebfff700 1 -- 172.21.15.12:0/952336119 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0016040 con 0x55f62820ee00 2024-05-08T23:54:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.875+0000 7f15ebfff700 1 -- 172.21.15.12:0/952336119 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15f001a610 con 0x55f62820ee00 2024-05-08T23:54:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.875+0000 7f15ebfff700 1 -- 172.21.15.12:0/952336119 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f001ad20 con 0x55f62820ee00 2024-05-08T23:54:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 -- 172.21.15.12:0/952336119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 msgr2=0x55f628203d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 --2- 172.21.15.12:0/952336119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f628203d20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009340 tx=0x7f15f0009770 comp rx=0 tx=0).stop 2024-05-08T23:54:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 -- 172.21.15.12:0/952336119 shutdown_connections 2024-05-08T23:54:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 --2- 172.21.15.12:0/952336119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820f720 0x55f628211460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 --2- 172.21.15.12:0/952336119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f628209890 0x55f62821ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 --2- 172.21.15.12:0/952336119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f62820ee00 0x55f628203d20 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 -- 172.21.15.12:0/952336119 >> 172.21.15.12:0/952336119 conn(0x55f628167210 msgr2=0x55f62820cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 -- 172.21.15.12:0/952336119 shutdown_connections 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 -- 172.21.15.12:0/952336119 wait complete. 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 -- start start 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.876+0000 7f16139dbb80 1 Processor -- start 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f628209890 0x55f628218150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f628218690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f16139dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 0x55f628278950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f16139dbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f628218e30 con 0x55f628209890 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f16139dbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f628218fa0 con 0x55f62820ee00 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f16139dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f628278e90 con 0x55f62820f720 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f16139dbb80 10 client.0 _ll_register_callbacks cb 0x55f6280feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f16055bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f628209890 0x55f628218150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f16055bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f628209890 0x55f628218150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36300/0 (socket says 172.21.15.12:36300) 2024-05-08T23:54:15.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f16055bd700 1 -- 172.21.15.12:0/3797065787 learned_addr learned my addr 172.21.15.12:0/3797065787 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f1604dbc700 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f628218690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.877+0000 7f1605dbe700 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 0x55f628278950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f16055bd700 1 -- 172.21.15.12:0/3797065787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 msgr2=0x55f628278950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f16055bd700 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 0x55f628278950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f16055bd700 1 -- 172.21.15.12:0/3797065787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 msgr2=0x55f628218690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f16055bd700 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f628218690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f16055bd700 1 -- 172.21.15.12:0/3797065787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f627fda3b0 con 0x55f628209890 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f1604dbc700 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f628218690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f16055bd700 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f628209890 0x55f628218150 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f15fc005d90 tx=0x7f15fc0118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acc4c72aa2f0a83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f15e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f15e8ff9700 1 -- 172.21.15.12:0/3797065787 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc02d040 con 0x55f628209890 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f15e8ff9700 1 -- 172.21.15.12:0/3797065787 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f15fc018070 con 0x55f628209890 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f16139dbb80 1 -- 172.21.15.12:0/3797065787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15fc016070 con 0x55f628209890 2024-05-08T23:54:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f15e8ff9700 1 -- 172.21.15.12:0/3797065787 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15fc034480 con 0x55f628209890 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f16139dbb80 10 client.17664 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f16139dbb80 1 -- 172.21.15.12:0/3797065787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f15fc011b20 con 0x55f628209890 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.878+0000 7f15e8ff9700 1 -- 172.21.15.12:0/3797065787 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15fc016070 con 0x55f628209890 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.879+0000 7f15e8ff9700 1 -- 172.21.15.12:0/3797065787 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f15fc025070 con 0x55f628209890 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.879+0000 7f16139dbb80 10 client.17664 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.879+0000 7f16139dbb80 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.879+0000 7f16139dbb80 20 client.17664 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.879+0000 7f16139dbb80 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f62827a580 0x55f62827c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.879+0000 7f16139dbb80 4 client.17664 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.879+0000 7f16139dbb80 1 -- 172.21.15.12:0/3797065787 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f628151ec0 con 0x55f62827a580 2024-05-08T23:54:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.879+0000 7f1604dbc700 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f62827a580 0x55f62827c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:15.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 478329 8.26 MB/sec execute 251 sec latency 1624.685 ms 2024-05-08T23:54:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.879+0000 7f15e8ff9700 10 client.17664 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.879+0000 7f1604dbc700 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f62827a580 0x55f62827c9a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed3420da3dfba53 server_cookie=48e2ca331f07dd1a in_seq=0 out_seq=0 2024-05-08T23:54:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f15e8ff9700 1 -- 172.21.15.12:0/3797065787 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f628151ec0 con 0x55f62827a580 2024-05-08T23:54:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f15e8ff9700 10 client.17664 handle_command_reply: tid=0 2024-05-08T23:54:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 client.17664 shutdown 2024-05-08T23:54:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 -- 172.21.15.12:0/3797065787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f628209890 msgr2=0x55f628218150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f628209890 0x55f628218150 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f15fc005d90 tx=0x7f15fc0118a0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 -- 172.21.15.12:0/3797065787 shutdown_connections 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f62820f720 0x55f628278950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f628209890 0x55f628218150 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f62827a580 0x55f62827c9a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 --2- 172.21.15.12:0/3797065787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f62820ee00 0x55f628218690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 -- 172.21.15.12:0/3797065787 >> 172.21.15.12:0/3797065787 conn(0x55f628167210 msgr2=0x55f628211370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f15e8ff9700 0 client.17664 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 -- 172.21.15.12:0/3797065787 shutdown_connections 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 1 -- 172.21.15.12:0/3797065787 wait complete. 2024-05-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.880+0000 7f16139dbb80 20 client.17664 trim_cache size 0 max 16384 2024-05-08T23:54:15.883 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:54:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:54:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:54:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "8057275f-ecb7-444f-a8cb-bb524c2e4eb5": { 2024-05-08T23:54:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:54:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "8057275f-ecb7-444f-a8cb-bb524c2e4eb5", 2024-05-08T23:54:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:54:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:54:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:54:15.883 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:54:15.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 -- 172.21.15.12:0/1586192886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15ec0a1310 msgr2=0x7f15ec0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15ec0a1310 0x7f15ec0a37d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009bf0 tx=0x7f15fc009260 comp rx=0 tx=0).stop 2024-05-08T23:54:15.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 -- 172.21.15.12:0/1586192886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1600100c40 msgr2=0x7f16001a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:15.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1600100c40 0x7f16001a5740 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f15f800b070 tx=0x7f15f800bda0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 -- 172.21.15.12:0/1586192886 shutdown_connections 2024-05-08T23:54:15.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f15ec0a1310 0x7f15ec0a37d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1600100c40 0x7f16001a5740 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1600069420 0x7f16001001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 --2- 172.21.15.12:0/1586192886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16001062a0 0x7f1600100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:15.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 -- 172.21.15.12:0/1586192886 >> 172.21.15.12:0/1586192886 conn(0x7f1600076ba0 msgr2=0x7f16000fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:15.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 -- 172.21.15.12:0/1586192886 shutdown_connections 2024-05-08T23:54:15.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:15.881+0000 7f1607821700 1 -- 172.21.15.12:0/1586192886 wait complete. 2024-05-08T23:54:15.889 INFO:tasks.cephfs.filesystem:scrub status for tag:8057275f-ecb7-444f-a8cb-bb524c2e4eb5 - {'path': '/', 'tag': '8057275f-ecb7-444f-a8cb-bb524c2e4eb5', 'options': 'recursive,force'} 2024-05-08T23:54:16.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: pgmap v246: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 591 KiB/s wr, 91 op/s 2024-05-08T23:54:16.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:54:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:54:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:54:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:54:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:54:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: pgmap v246: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 591 KiB/s wr, 91 op/s 2024-05-08T23:54:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:54:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:54:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:54:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:54:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:54:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.874 INFO:tasks.workunit.client.0.smithi012.stdout: 1 480210 8.28 MB/sec execute 252 sec latency 17.371 ms 2024-05-08T23:54:16.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: pgmap v246: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 591 KiB/s wr, 91 op/s 2024-05-08T23:54:16.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:54:16.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:54:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:54:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:54:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:54:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:16.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:17.874 INFO:tasks.workunit.client.0.smithi012.stdout: 1 481360 8.27 MB/sec execute 253 sec latency 286.980 ms 2024-05-08T23:54:18.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:18 smithi204 ceph-mon[47373]: pgmap v247: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 552 KiB/s wr, 99 op/s 2024-05-08T23:54:18.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:18 smithi160 ceph-mon[46152]: pgmap v247: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 552 KiB/s wr, 99 op/s 2024-05-08T23:54:18.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 481360 8.23 MB/sec execute 254 sec latency 1287.118 ms 2024-05-08T23:54:18.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:18 smithi012 ceph-mon[42503]: pgmap v247: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 552 KiB/s wr, 99 op/s 2024-05-08T23:54:19.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 481360 8.20 MB/sec execute 255 sec latency 2287.218 ms 2024-05-08T23:54:20.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:20 smithi204 ceph-mon[47373]: pgmap v248: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 552 KiB/s wr, 99 op/s 2024-05-08T23:54:20.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:20 smithi160 ceph-mon[46152]: pgmap v248: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 552 KiB/s wr, 99 op/s 2024-05-08T23:54:20.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 481360 8.17 MB/sec execute 256 sec latency 3287.273 ms 2024-05-08T23:54:20.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:20 smithi012 ceph-mon[42503]: pgmap v248: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 552 KiB/s wr, 99 op/s 2024-05-08T23:54:21.874 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483487 8.19 MB/sec execute 257 sec latency 3432.229 ms 2024-05-08T23:54:22.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:22 smithi204 ceph-mon[47373]: pgmap v249: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 823 KiB/s wr, 126 op/s 2024-05-08T23:54:22.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:22 smithi160 ceph-mon[46152]: pgmap v249: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 823 KiB/s wr, 126 op/s 2024-05-08T23:54:22.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 484859 8.18 MB/sec execute 258 sec latency 39.661 ms 2024-05-08T23:54:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:22 smithi012 ceph-mon[42503]: pgmap v249: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 823 KiB/s wr, 126 op/s 2024-05-08T23:54:23.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 484911 8.15 MB/sec execute 259 sec latency 955.248 ms 2024-05-08T23:54:23.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:23 smithi012 ceph-mon[42503]: pgmap v250: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 557 KiB/s wr, 69 op/s 2024-05-08T23:54:24.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:23 smithi204 ceph-mon[47373]: pgmap v250: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 557 KiB/s wr, 69 op/s 2024-05-08T23:54:24.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:23 smithi160 ceph-mon[46152]: pgmap v250: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 557 KiB/s wr, 69 op/s 2024-05-08T23:54:24.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 484911 8.12 MB/sec execute 260 sec latency 1955.368 ms 2024-05-08T23:54:25.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 484911 8.09 MB/sec execute 261 sec latency 2955.487 ms 2024-05-08T23:54:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 -- 172.21.15.12:0/3082322347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3244105e40 msgr2=0x7f3244106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 --2- 172.21.15.12:0/3082322347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3244105e40 0x7f3244106220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f32340098f0 tx=0x7f3234009c00 comp rx=0 tx=0).stop 2024-05-08T23:54:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 -- 172.21.15.12:0/3082322347 shutdown_connections 2024-05-08T23:54:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 --2- 172.21.15.12:0/3082322347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3244100740 0x7f324410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 --2- 172.21.15.12:0/3082322347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3244105e40 0x7f3244106220 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 --2- 172.21.15.12:0/3082322347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32440ffd80 0x7f3244100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 -- 172.21.15.12:0/3082322347 >> 172.21.15.12:0/3082322347 conn(0x7f3244076f00 msgr2=0x7f3244077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 -- 172.21.15.12:0/3082322347 shutdown_connections 2024-05-08T23:54:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 -- 172.21.15.12:0/3082322347 wait complete. 2024-05-08T23:54:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 Processor -- start 2024-05-08T23:54:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.525+0000 7f324a916700 1 -- start start 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f324a916700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32440ffd80 0x7f32441997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f324a916700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3244100740 0x7f3244199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f324a916700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3244105e40 0x7f324419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f324a916700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324419e4b0 con 0x7f3244105e40 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f324a916700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324419e620 con 0x7f32440ffd80 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f324a916700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324419e7e0 con 0x7f3244100740 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f3243fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32440ffd80 0x7f32441997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f3243fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32440ffd80 0x7f32441997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:59352/0 (socket says 172.21.15.12:59352) 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f3243fff700 1 -- 172.21.15.12:0/3441719143 learned_addr learned my addr 172.21.15.12:0/3441719143 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f3248eb3700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3244105e40 0x7f324419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f32437fe700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3244100740 0x7f3244199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f3243fff700 1 -- 172.21.15.12:0/3441719143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3244100740 msgr2=0x7f3244199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f3243fff700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3244100740 0x7f3244199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f3243fff700 1 -- 172.21.15.12:0/3441719143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3244105e40 msgr2=0x7f324419dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f3243fff700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3244105e40 0x7f324419dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.526+0000 7f3243fff700 1 -- 172.21.15.12:0/3441719143 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3234009520 con 0x7f32440ffd80 2024-05-08T23:54:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.527+0000 7f3243fff700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32440ffd80 0x7f32441997a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3234005950 tx=0x7f323400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.527+0000 7f32417fa700 1 -- 172.21.15.12:0/3441719143 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323401c070 con 0x7f32440ffd80 2024-05-08T23:54:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.527+0000 7f324a916700 1 -- 172.21.15.12:0/3441719143 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324419ea60 con 0x7f32440ffd80 2024-05-08T23:54:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.527+0000 7f32417fa700 1 -- 172.21.15.12:0/3441719143 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f323400bb20 con 0x7f32440ffd80 2024-05-08T23:54:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.527+0000 7f32417fa700 1 -- 172.21.15.12:0/3441719143 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340038e0 con 0x7f32440ffd80 2024-05-08T23:54:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.527+0000 7f324a916700 1 -- 172.21.15.12:0/3441719143 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32441a6330 con 0x7f32440ffd80 2024-05-08T23:54:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.529+0000 7f32417fa700 1 -- 172.21.15.12:0/3441719143 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3234020030 con 0x7f32440ffd80 2024-05-08T23:54:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.530+0000 7f32417fa700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f322c0a1290 0x7f322c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.530+0000 7f32437fe700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f322c0a1290 0x7f322c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.530+0000 7f32417fa700 1 -- 172.21.15.12:0/3441719143 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f32340cdb40 con 0x7f32440ffd80 2024-05-08T23:54:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.530+0000 7f32437fe700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f322c0a1290 0x7f322c0a3750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f324419ab10 tx=0x7f3238009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.531+0000 7f3256ad0b80 1 Processor -- start 2024-05-08T23:54:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.531+0000 7f3256ad0b80 1 -- start start 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e44950 0x557b36e44d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e45270 0x557b36e3ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e3f480 0x557b36e3f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36d960f0 con 0x557b36e3f480 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36d93ab0 con 0x557b36e44950 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36d8f3a0 con 0x557b36e45270 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f32437fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e45270 0x557b36e3ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f3248eb3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e3f480 0x557b36e3f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f3243fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e44950 0x557b36e44d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f32437fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e45270 0x557b36e3ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44784/0 (socket says 172.21.15.12:44784) 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f3248eb3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e3f480 0x557b36e3f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39272/0 (socket says 172.21.15.12:39272) 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f32437fe700 1 -- 172.21.15.12:0/138004710 learned_addr learned my addr 172.21.15.12:0/138004710 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f32437fe700 1 -- 172.21.15.12:0/138004710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e44950 msgr2=0x557b36e44d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f32437fe700 1 --2- 172.21.15.12:0/138004710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e44950 0x557b36e44d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f32437fe700 1 -- 172.21.15.12:0/138004710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e3f480 msgr2=0x557b36e3f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f32437fe700 1 --2- 172.21.15.12:0/138004710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e3f480 0x557b36e3f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f32437fe700 1 -- 172.21.15.12:0/138004710 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b36c10350 con 0x557b36e45270 2024-05-08T23:54:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f3248eb3700 1 --2- 172.21.15.12:0/138004710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e3f480 0x557b36e3f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.532+0000 7f32437fe700 1 --2- 172.21.15.12:0/138004710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e45270 0x557b36e3ef40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f32380041d0 tx=0x7f3238014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c8f44bc31be414 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.533+0000 7f322a7fc700 1 -- 172.21.15.12:0/138004710 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238009e60 con 0x557b36e45270 2024-05-08T23:54:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.533+0000 7f322a7fc700 1 -- 172.21.15.12:0/138004710 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3238004830 con 0x557b36e45270 2024-05-08T23:54:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.533+0000 7f322a7fc700 1 -- 172.21.15.12:0/138004710 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32380055c0 con 0x557b36e45270 2024-05-08T23:54:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.533+0000 7f3256ad0b80 1 -- 172.21.15.12:0/138004710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e45270 msgr2=0x557b36e3ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.533+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/138004710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e45270 0x557b36e3ef40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f32380041d0 tx=0x7f3238014000 comp rx=0 tx=0).stop 2024-05-08T23:54:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.533+0000 7f3256ad0b80 1 -- 172.21.15.12:0/138004710 shutdown_connections 2024-05-08T23:54:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.533+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/138004710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e3f480 0x557b36e3f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.533+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/138004710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e44950 0x557b36e44d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.533+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/138004710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e45270 0x557b36e3ef40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.533+0000 7f3256ad0b80 1 -- 172.21.15.12:0/138004710 >> 172.21.15.12:0/138004710 conn(0x557b36d9d1b0 msgr2=0x557b36e39840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.534+0000 7f3256ad0b80 1 -- 172.21.15.12:0/138004710 shutdown_connections 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.534+0000 7f3256ad0b80 1 -- 172.21.15.12:0/138004710 wait complete. 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.534+0000 7f3256ad0b80 1 -- start start 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.534+0000 7f3256ad0b80 1 Processor -- start 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e3f480 0x557b36e4b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e44950 0x557b36e48530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e45270 0x557b36e48c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36d2b500 con 0x557b36e45270 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36d2b1a0 con 0x557b36e3f480 2024-05-08T23:54:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36c12640 con 0x557b36e44950 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3256ad0b80 10 client.0 _ll_register_callbacks cb 0x557b36d34ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f32437fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e44950 0x557b36e48530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3243fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e3f480 0x557b36e4b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3243fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e3f480 0x557b36e4b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:59364/0 (socket says 172.21.15.12:59364) 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f32437fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e44950 0x557b36e48530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44794/0 (socket says 172.21.15.12:44794) 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3243fff700 1 -- 172.21.15.12:0/3358235640 learned_addr learned my addr 172.21.15.12:0/3358235640 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3248eb3700 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e45270 0x557b36e48c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f32437fe700 1 -- 172.21.15.12:0/3358235640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e3f480 msgr2=0x557b36e4b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f32437fe700 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e3f480 0x557b36e4b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f32437fe700 1 -- 172.21.15.12:0/3358235640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e45270 msgr2=0x557b36e48c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f32437fe700 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e45270 0x557b36e48c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f32437fe700 1 -- 172.21.15.12:0/3358235640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c011230 con 0x557b36e44950 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3243fff700 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e3f480 0x557b36e4b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f3248eb3700 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e45270 0x557b36e48c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f32437fe700 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e44950 0x557b36e48530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3238005100 tx=0x7f3238005430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ae3a371a271a6b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.535+0000 7f32237fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f32237fe700 1 -- 172.21.15.12:0/3358235640 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323802c070 con 0x557b36e44950 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f32237fe700 1 -- 172.21.15.12:0/3358235640 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3238009e60 con 0x557b36e44950 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3358235640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b36ce8210 con 0x557b36e44950 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f32237fe700 1 -- 172.21.15.12:0/3358235640 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32380044e0 con 0x557b36e44950 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f3256ad0b80 10 client.27454 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3358235640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f3238017070 con 0x557b36e44950 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f32237fe700 1 -- 172.21.15.12:0/3358235640 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f323803d080 con 0x557b36e44950 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f32237fe700 1 -- 172.21.15.12:0/3358235640 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f3238024030 con 0x557b36e44950 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f3256ad0b80 10 client.27454 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f3256ad0b80 10 client.27454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f3256ad0b80 20 client.27454 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557b36e58c70 0x557b36e5b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f3256ad0b80 4 client.27454 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3358235640 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b36e0e7b0 con 0x557b36e58c70 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.536+0000 7f3243fff700 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557b36e58c70 0x557b36e5b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.537+0000 7f32237fe700 10 client.27454 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.537+0000 7f3243fff700 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557b36e58c70 0x557b36e5b090 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbfe67dea6b4438d server_cookie=8b0eb13a36f242f2 in_seq=0 out_seq=0 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f32237fe700 1 -- 172.21.15.12:0/3358235640 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b36e0e7b0 con 0x557b36e58c70 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f32237fe700 10 client.27454 handle_command_reply: tid=0 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 client.27454 shutdown 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3358235640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e44950 msgr2=0x557b36e48530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e44950 0x557b36e48530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3238005100 tx=0x7f3238005430 comp rx=0 tx=0).stop 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3358235640 shutdown_connections 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e45270 0x557b36e48c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557b36e58c70 0x557b36e5b090 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e3f480 0x557b36e4b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3358235640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e44950 0x557b36e48530 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f32237fe700 0 client.27454 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3358235640 >> 172.21.15.12:0/3358235640 conn(0x557b36d9d1b0 msgr2=0x557b36e434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3358235640 shutdown_connections 2024-05-08T23:54:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3358235640 wait complete. 2024-05-08T23:54:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.538+0000 7f3256ad0b80 20 client.27454 trim_cache size 0 max 16384 2024-05-08T23:54:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3256ad0b80 1 Processor -- start 2024-05-08T23:54:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3256ad0b80 1 -- start start 2024-05-08T23:54:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 0x557b36e39910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e47d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 0x557b36e546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36e369b0 con 0x557b36e44950 2024-05-08T23:54:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36e34330 con 0x557b36e45270 2024-05-08T23:54:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36e4b720 con 0x557b36e3f480 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f32437fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e47d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f32437fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e47d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39304/0 (socket says 172.21.15.12:39304) 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f32437fe700 1 -- 172.21.15.12:0/1507060538 learned_addr learned my addr 172.21.15.12:0/1507060538 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3243fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 0x557b36e39910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3248eb3700 1 --2- 172.21.15.12:0/1507060538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 0x557b36e546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f32437fe700 1 -- 172.21.15.12:0/1507060538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 msgr2=0x557b36e39910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f32437fe700 1 --2- 172.21.15.12:0/1507060538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 0x557b36e39910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f32437fe700 1 -- 172.21.15.12:0/1507060538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 msgr2=0x557b36e546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f32437fe700 1 --2- 172.21.15.12:0/1507060538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 0x557b36e546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f32437fe700 1 -- 172.21.15.12:0/1507060538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b36bbcb00 con 0x557b36e44950 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.556+0000 7f3248eb3700 1 --2- 172.21.15.12:0/1507060538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 0x557b36e546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.557+0000 7f32437fe700 1 --2- 172.21.15.12:0/1507060538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e47d00 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f32380051b0 tx=0x7f323801ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c3af018320aea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.557+0000 7f322affd700 1 -- 172.21.15.12:0/1507060538 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323802c020 con 0x557b36e44950 2024-05-08T23:54:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.557+0000 7f322affd700 1 -- 172.21.15.12:0/1507060538 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b36ce8210 con 0x557b36e44950 2024-05-08T23:54:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.557+0000 7f322affd700 1 -- 172.21.15.12:0/1507060538 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323801e420 con 0x557b36e44950 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 -- 172.21.15.12:0/1507060538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 msgr2=0x557b36e47d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/1507060538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e47d00 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f32380051b0 tx=0x7f323801ea20 comp rx=0 tx=0).stop 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 -- 172.21.15.12:0/1507060538 shutdown_connections 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/1507060538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 0x557b36e546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/1507060538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 0x557b36e39910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/1507060538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e47d00 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 -- 172.21.15.12:0/1507060538 >> 172.21.15.12:0/1507060538 conn(0x557b36d9d1b0 msgr2=0x557b36e3d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 -- 172.21.15.12:0/1507060538 shutdown_connections 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 -- 172.21.15.12:0/1507060538 wait complete. 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 -- start start 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.558+0000 7f3256ad0b80 1 Processor -- start 2024-05-08T23:54:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 0x557b36e4fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e4dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3256ad0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 0x557b36e4e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36e504f0 con 0x557b36e44950 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36e4e670 con 0x557b36e45270 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3256ad0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36e4e830 con 0x557b36e3f480 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3256ad0b80 10 client.0 _ll_register_callbacks cb 0x7f323c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f32437fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e4dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3243fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 0x557b36e4fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3243fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 0x557b36e4fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44812/0 (socket says 172.21.15.12:44812) 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f32437fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e4dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39316/0 (socket says 172.21.15.12:39316) 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3243fff700 1 -- 172.21.15.12:0/3446461091 learned_addr learned my addr 172.21.15.12:0/3446461091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3248eb3700 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 0x557b36e4e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3243fff700 1 -- 172.21.15.12:0/3446461091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 msgr2=0x557b36e4e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3243fff700 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 0x557b36e4e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3243fff700 1 -- 172.21.15.12:0/3446461091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 msgr2=0x557b36e4dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3243fff700 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e4dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.559+0000 7f3243fff700 1 -- 172.21.15.12:0/3446461091 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3238017070 con 0x557b36e3f480 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.560+0000 7f3243fff700 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 0x557b36e4fe00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3234009810 tx=0x7f323400bc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40529beaddd5269 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.560+0000 7f32237fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.560+0000 7f32237fe700 1 -- 172.21.15.12:0/3446461091 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340a3080 con 0x557b36e3f480 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.560+0000 7f32237fe700 1 -- 172.21.15.12:0/3446461091 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f32340b3070 con 0x557b36e3f480 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.560+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3446461091 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3238004830 con 0x557b36e3f480 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.560+0000 7f32237fe700 1 -- 172.21.15.12:0/3446461091 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32340cb4c0 con 0x557b36e3f480 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.560+0000 7f3256ad0b80 10 client.27460 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.560+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3446461091 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557b36d55500 con 0x557b36e3f480 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.560+0000 7f32237fe700 1 -- 172.21.15.12:0/3446461091 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3238004830 con 0x557b36e3f480 2024-05-08T23:54:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.560+0000 7f32237fe700 1 -- 172.21.15.12:0/3446461091 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f3234021380 con 0x557b36e3f480 2024-05-08T23:54:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.561+0000 7f3256ad0b80 10 client.27460 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.561+0000 7f3256ad0b80 10 client.27460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.561+0000 7f3256ad0b80 20 client.27460 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.561+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557b36eaefd0 0x557b36eb13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.561+0000 7f3256ad0b80 4 client.27460 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:54:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.561+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3446461091 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b36d87e60 con 0x557b36eaefd0 2024-05-08T23:54:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.561+0000 7f32437fe700 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557b36eaefd0 0x557b36eb13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.561+0000 7f32237fe700 10 client.27460 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.561+0000 7f32437fe700 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557b36eaefd0 0x557b36eb13c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1ae84342a5235a server_cookie=6a929ed655ab216b in_seq=0 out_seq=0 2024-05-08T23:54:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f32237fe700 1 -- 172.21.15.12:0/3446461091 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557b36d87e60 con 0x557b36eaefd0 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f32237fe700 10 client.27460 handle_command_reply: tid=0 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 client.27460 shutdown 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3446461091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 msgr2=0x557b36e4fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 0x557b36e4fe00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3234009810 tx=0x7f323400bc90 comp rx=0 tx=0).stop 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3446461091 shutdown_connections 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557b36eaefd0 0x557b36eb13c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b36e45270 0x557b36e4e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557b36e3f480 0x557b36e4fe00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 --2- 172.21.15.12:0/3446461091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b36e44950 0x557b36e4dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3446461091 >> 172.21.15.12:0/3446461091 conn(0x557b36d9d1b0 msgr2=0x557b36e546c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f32237fe700 0 client.27460 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3446461091 shutdown_connections 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 1 -- 172.21.15.12:0/3446461091 wait complete. 2024-05-08T23:54:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.608+0000 7f3256ad0b80 20 client.27460 trim_cache size 0 max 16384 2024-05-08T23:54:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:54:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:54:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:54:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:54:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.609+0000 7f324a916700 1 -- 172.21.15.12:0/3441719143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f322c0a1290 msgr2=0x7f322c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.609+0000 7f324a916700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f322c0a1290 0x7f322c0a3750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f324419ab10 tx=0x7f3238009310 comp rx=0 tx=0).stop 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.609+0000 7f324a916700 1 -- 172.21.15.12:0/3441719143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32440ffd80 msgr2=0x7f32441997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.609+0000 7f324a916700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32440ffd80 0x7f32441997a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3234005950 tx=0x7f323400fa00 comp rx=0 tx=0).stop 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.610+0000 7f324a916700 1 -- 172.21.15.12:0/3441719143 shutdown_connections 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.610+0000 7f324a916700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f322c0a1290 0x7f322c0a3750 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.610+0000 7f324a916700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3244105e40 0x7f324419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.610+0000 7f324a916700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32440ffd80 0x7f32441997a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.610+0000 7f324a916700 1 --2- 172.21.15.12:0/3441719143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3244100740 0x7f3244199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.610+0000 7f324a916700 1 -- 172.21.15.12:0/3441719143 >> 172.21.15.12:0/3441719143 conn(0x7f3244076f00 msgr2=0x7f32440fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.610+0000 7f324a916700 1 -- 172.21.15.12:0/3441719143 shutdown_connections 2024-05-08T23:54:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:26.611+0000 7f324a916700 1 -- 172.21.15.12:0/3441719143 wait complete. 2024-05-08T23:54:26.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:26 smithi204 ceph-mon[47373]: pgmap v251: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 557 KiB/s wr, 69 op/s 2024-05-08T23:54:26.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:26 smithi160 ceph-mon[46152]: pgmap v251: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 557 KiB/s wr, 69 op/s 2024-05-08T23:54:26.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 486920 8.10 MB/sec execute 262 sec latency 3106.089 ms 2024-05-08T23:54:26.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:26 smithi012 ceph-mon[42503]: pgmap v251: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 557 KiB/s wr, 69 op/s 2024-05-08T23:54:27.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 488249 8.10 MB/sec execute 263 sec latency 28.044 ms 2024-05-08T23:54:28.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:28 smithi204 ceph-mon[47373]: pgmap v252: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 897 KiB/s wr, 107 op/s 2024-05-08T23:54:28.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:28 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:54:28.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:28 smithi160 ceph-mon[46152]: pgmap v252: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 897 KiB/s wr, 107 op/s 2024-05-08T23:54:28.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:28 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:54:28.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 488453 8.07 MB/sec execute 264 sec latency 862.786 ms 2024-05-08T23:54:28.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:28 smithi012 ceph-mon[42503]: pgmap v252: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 897 KiB/s wr, 107 op/s 2024-05-08T23:54:28.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:28 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:54:29.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:54:29.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:54:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:54:29.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:54:29.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 488453 8.04 MB/sec execute 265 sec latency 1862.863 ms 2024-05-08T23:54:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:54:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:54:30.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:30 smithi204 ceph-mon[47373]: pgmap v253: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 611 KiB/s wr, 67 op/s 2024-05-08T23:54:30.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:30 smithi160 ceph-mon[46152]: pgmap v253: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 611 KiB/s wr, 67 op/s 2024-05-08T23:54:30.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 488453 8.01 MB/sec execute 266 sec latency 2862.914 ms 2024-05-08T23:54:30.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:30 smithi012 ceph-mon[42503]: pgmap v253: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 611 KiB/s wr, 67 op/s 2024-05-08T23:54:31.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:31 smithi204 ceph-mon[47373]: pgmap v254: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-08T23:54:31.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:31 smithi160 ceph-mon[46152]: pgmap v254: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-08T23:54:31.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 490562 8.03 MB/sec execute 267 sec latency 3012.350 ms 2024-05-08T23:54:31.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:31 smithi012 ceph-mon[42503]: pgmap v254: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-08T23:54:32.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 491976 8.02 MB/sec execute 268 sec latency 22.185 ms 2024-05-08T23:54:33.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 492000 7.99 MB/sec execute 269 sec latency 987.848 ms 2024-05-08T23:54:34.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:34 smithi204 ceph-mon[47373]: pgmap v255: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 929 KiB/s wr, 99 op/s 2024-05-08T23:54:34.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:34 smithi160 ceph-mon[46152]: pgmap v255: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 929 KiB/s wr, 99 op/s 2024-05-08T23:54:34.875 INFO:tasks.workunit.client.0.smithi012.stdout: 1 492000 7.96 MB/sec execute 270 sec latency 1987.889 ms 2024-05-08T23:54:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:34 smithi012 ceph-mon[42503]: pgmap v255: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 929 KiB/s wr, 99 op/s 2024-05-08T23:54:35.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:35 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:54:35.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:35 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:54:35.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:35 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:54:35.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:35 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:54:35.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:35 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:54:35.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:35 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:54:35.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:35 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:54:35.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:35 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:54:35.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 492000 7.93 MB/sec execute 271 sec latency 2987.937 ms 2024-05-08T23:54:35.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:35 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:54:35.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:35 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:54:35.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:35 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:54:35.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:35 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:54:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:36 smithi204 ceph-mon[47373]: pgmap v256: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 929 KiB/s wr, 99 op/s 2024-05-08T23:54:36.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:36 smithi160 ceph-mon[46152]: pgmap v256: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 929 KiB/s wr, 99 op/s 2024-05-08T23:54:36.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 494123 7.95 MB/sec execute 272 sec latency 3136.984 ms 2024-05-08T23:54:36.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:36 smithi012 ceph-mon[42503]: pgmap v256: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 929 KiB/s wr, 99 op/s 2024-05-08T23:54:37.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 495375 7.94 MB/sec execute 273 sec latency 17.169 ms 2024-05-08T23:54:38.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:38 smithi204 ceph-mon[47373]: pgmap v257: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-05-08T23:54:38.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:38 smithi160 ceph-mon[46152]: pgmap v257: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-05-08T23:54:38.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 495544 7.91 MB/sec execute 274 sec latency 793.389 ms 2024-05-08T23:54:38.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:38 smithi012 ceph-mon[42503]: pgmap v257: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-05-08T23:54:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.743+0000 7f7b9179f700 1 -- 172.21.15.12:0/1219204622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b8c100010 msgr2=0x7f7b8c1003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.743+0000 7f7b9179f700 1 --2- 172.21.15.12:0/1219204622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b8c100010 0x7f7b8c1003f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009990 tx=0x7f7b7c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.743+0000 7f7b9179f700 1 -- 172.21.15.12:0/1219204622 shutdown_connections 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.743+0000 7f7b9179f700 1 --2- 172.21.15.12:0/1219204622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b8c105130 0x7f7b8c10ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.743+0000 7f7b9179f700 1 --2- 172.21.15.12:0/1219204622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b8c100010 0x7f7b8c1003f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.743+0000 7f7b9179f700 1 --2- 172.21.15.12:0/1219204622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c1009c0 0x7f7b8c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.743+0000 7f7b9179f700 1 -- 172.21.15.12:0/1219204622 >> 172.21.15.12:0/1219204622 conn(0x7f7b8c076f40 msgr2=0x7f7b8c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.743+0000 7f7b9179f700 1 -- 172.21.15.12:0/1219204622 shutdown_connections 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.743+0000 7f7b9179f700 1 -- 172.21.15.12:0/1219204622 wait complete. 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.744+0000 7f7b9179f700 1 Processor -- start 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.744+0000 7f7b9179f700 1 -- start start 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.744+0000 7f7b9179f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b8c100010 0x7f7b8c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.744+0000 7f7b9179f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b8c105130 0x7f7b8c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.744+0000 7f7b9179f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c19bc00 0x7f7b8c19c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.744+0000 7f7b9179f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19c680 con 0x7f7b8c105130 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.744+0000 7f7b9179f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19c7f0 con 0x7f7b8c100010 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.744+0000 7f7b9179f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19c9b0 con 0x7f7b8c19bc00 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b8c105130 0x7f7b8c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b8c105130 0x7f7b8c1978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58982/0 (socket says 172.21.15.12:58982) 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b827fc700 1 -- 172.21.15.12:0/918126138 learned_addr learned my addr 172.21.15.12:0/918126138 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b8b7fe700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c19bc00 0x7f7b8c19c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b8affd700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b8c100010 0x7f7b8c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b827fc700 1 -- 172.21.15.12:0/918126138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c19bc00 msgr2=0x7f7b8c19c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b827fc700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c19bc00 0x7f7b8c19c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b827fc700 1 -- 172.21.15.12:0/918126138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b8c100010 msgr2=0x7f7b8c197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b827fc700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b8c100010 0x7f7b8c197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b827fc700 1 -- 172.21.15.12:0/918126138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c0095f0 con 0x7f7b8c105130 2024-05-08T23:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b8affd700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b8c100010 0x7f7b8c197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b827fc700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b8c105130 0x7f7b8c1978b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400e970 tx=0x7f7b7400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b88ff9700 1 -- 172.21.15.12:0/918126138 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74009790 con 0x7f7b8c105130 2024-05-08T23:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b88ff9700 1 -- 172.21.15.12:0/918126138 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b7400eec0 con 0x7f7b8c105130 2024-05-08T23:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b9179f700 1 -- 172.21.15.12:0/918126138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8c1a3f10 con 0x7f7b8c105130 2024-05-08T23:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b88ff9700 1 -- 172.21.15.12:0/918126138 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74005590 con 0x7f7b8c105130 2024-05-08T23:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.745+0000 7f7b9179f700 1 -- 172.21.15.12:0/918126138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c1a4390 con 0x7f7b8c105130 2024-05-08T23:54:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.748+0000 7f7b88ff9700 1 -- 172.21.15.12:0/918126138 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7b7400cbd0 con 0x7f7b8c105130 2024-05-08T23:54:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.748+0000 7f7b88ff9700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b780a1290 0x7f7b780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.748+0000 7f7b8affd700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b780a1290 0x7f7b780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.748+0000 7f7b88ff9700 1 -- 172.21.15.12:0/918126138 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7b74014070 con 0x7f7b8c105130 2024-05-08T23:54:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.749+0000 7f7b8affd700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b780a1290 0x7f7b780a3750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00b2e0 tx=0x7f7b7c005940 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.749+0000 7f7b9d959b80 1 Processor -- start 2024-05-08T23:54:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.749+0000 7f7b9d959b80 1 -- start start 2024-05-08T23:54:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc51950 0x55d7bbc51d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc4c480 0x55d7bbc4c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbba30f0 con 0x55d7bbc52270 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbba0ab0 con 0x55d7bbc4c480 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbb9c3a0 con 0x55d7bbc51950 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58998/0 (socket says 172.21.15.12:58998) 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b827fc700 1 -- 172.21.15.12:0/2658869504 learned_addr learned my addr 172.21.15.12:0/2658869504 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b8b7fe700 1 --2- 172.21.15.12:0/2658869504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc4c480 0x55d7bbc4c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b8affd700 1 --2- 172.21.15.12:0/2658869504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc51950 0x55d7bbc51d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b827fc700 1 -- 172.21.15.12:0/2658869504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc51950 msgr2=0x55d7bbc51d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b827fc700 1 --2- 172.21.15.12:0/2658869504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc51950 0x55d7bbc51d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b827fc700 1 -- 172.21.15.12:0/2658869504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc4c480 msgr2=0x55d7bbc4c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b827fc700 1 --2- 172.21.15.12:0/2658869504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc4c480 0x55d7bbc4c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b827fc700 1 -- 172.21.15.12:0/2658869504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7bba1d350 con 0x55d7bbc52270 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.750+0000 7f7b827fc700 1 --2- 172.21.15.12:0/2658869504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4bf40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7b74096580 tx=0x7f7b7409a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e965c06e08e5ab57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.751+0000 7f7b817fa700 1 -- 172.21.15.12:0/2658869504 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740a2500 con 0x55d7bbc52270 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.751+0000 7f7b817fa700 1 -- 172.21.15.12:0/2658869504 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b74096790 con 0x55d7bbc52270 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.751+0000 7f7b817fa700 1 -- 172.21.15.12:0/2658869504 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7409a520 con 0x55d7bbc52270 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.751+0000 7f7b9d959b80 1 -- 172.21.15.12:0/2658869504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 msgr2=0x55d7bbc4bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.751+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/2658869504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4bf40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7b74096580 tx=0x7f7b7409a800 comp rx=0 tx=0).stop 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 -- 172.21.15.12:0/2658869504 shutdown_connections 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/2658869504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc4c480 0x55d7bbc4c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/2658869504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc51950 0x55d7bbc51d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/2658869504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4bf40 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 -- 172.21.15.12:0/2658869504 >> 172.21.15.12:0/2658869504 conn(0x55d7bbbaa1b0 msgr2=0x55d7bbc46840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 -- 172.21.15.12:0/2658869504 shutdown_connections 2024-05-08T23:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 -- 172.21.15.12:0/2658869504 wait complete. 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 -- start start 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 Processor -- start 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc55570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc51950 0x55d7bbc55ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc52270 0x55d7bbc55ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbb38500 con 0x55d7bbc51950 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbb381a0 con 0x55d7bbc52270 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bba1f640 con 0x55d7bbc4c480 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b9d959b80 10 client.0 _ll_register_callbacks cb 0x55d7bbb41ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.752+0000 7f7b827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc51950 0x55d7bbc55ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc51950 0x55d7bbc55ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59014/0 (socket says 172.21.15.12:59014) 2024-05-08T23:54:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b827fc700 1 -- 172.21.15.12:0/3722142237 learned_addr learned my addr 172.21.15.12:0/3722142237 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b8affd700 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc55570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b8b7fe700 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc52270 0x55d7bbc55ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b827fc700 1 -- 172.21.15.12:0/3722142237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 msgr2=0x55d7bbc55570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b827fc700 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc55570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b827fc700 1 -- 172.21.15.12:0/3722142237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc52270 msgr2=0x55d7bbc55ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b827fc700 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc52270 0x55d7bbc55ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b827fc700 1 -- 172.21.15.12:0/3722142237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c1a4390 con 0x55d7bbc51950 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b8b7fe700 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc52270 0x55d7bbc55ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b827fc700 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc51950 0x55d7bbc55ab0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7b74099d30 tx=0x7f7b740997a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f425bdbfe32907f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b727fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b727fc700 1 -- 172.21.15.12:0/3722142237 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740963c0 con 0x55d7bbc51950 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b727fc700 1 -- 172.21.15.12:0/3722142237 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7b740a2500 con 0x55d7bbc51950 2024-05-08T23:54:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b9d959b80 1 -- 172.21.15.12:0/3722142237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7bbaf5210 con 0x55d7bbc51950 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b727fc700 1 -- 172.21.15.12:0/3722142237 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b7409a2e0 con 0x55d7bbc51950 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b9d959b80 10 client.17694 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.753+0000 7f7b9d959b80 1 -- 172.21.15.12:0/3722142237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55d7bb9c9b00 con 0x55d7bbc51950 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.754+0000 7f7b727fc700 1 -- 172.21.15.12:0/3722142237 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7bbaf5210 con 0x55d7bbc51950 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.754+0000 7f7b727fc700 1 -- 172.21.15.12:0/3722142237 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7b74099070 con 0x55d7bbc51950 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.754+0000 7f7b9d959b80 10 client.17694 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.754+0000 7f7b9d959b80 10 client.17694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.754+0000 7f7b9d959b80 20 client.17694 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.754+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7bbc657c0 0x55d7bbc67bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.754+0000 7f7b9d959b80 4 client.17694 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.754+0000 7f7b9d959b80 1 -- 172.21.15.12:0/3722142237 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7bbc1b7b0 con 0x55d7bbc657c0 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.754+0000 7f7b8affd700 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7bbc657c0 0x55d7bbc67bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b8affd700 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7bbc657c0 0x55d7bbc67bb0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bcfb674b469af89 server_cookie=e64c802937b5c537 in_seq=0 out_seq=0 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b727fc700 10 client.17694 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b727fc700 1 -- 172.21.15.12:0/3722142237 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7bbc1b7b0 con 0x55d7bbc657c0 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b727fc700 10 client.17694 handle_command_reply: tid=0 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b9d959b80 1 client.17694 shutdown 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b9d959b80 1 -- 172.21.15.12:0/3722142237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc51950 msgr2=0x55d7bbc55ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc51950 0x55d7bbc55ab0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7b74099d30 tx=0x7f7b740997a0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b9d959b80 1 -- 172.21.15.12:0/3722142237 shutdown_connections 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7bbc657c0 0x55d7bbc67bb0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc52270 0x55d7bbc55ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc55570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b727fc700 0 client.17694 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/3722142237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc51950 0x55d7bbc55ab0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.755+0000 7f7b9d959b80 1 -- 172.21.15.12:0/3722142237 >> 172.21.15.12:0/3722142237 conn(0x55d7bbbaa1b0 msgr2=0x55d7bbc504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.756+0000 7f7b9d959b80 1 -- 172.21.15.12:0/3722142237 shutdown_connections 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.756+0000 7f7b9d959b80 1 -- 172.21.15.12:0/3722142237 wait complete. 2024-05-08T23:54:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.756+0000 7f7b9d959b80 20 client.17694 trim_cache size 0 max 16384 2024-05-08T23:54:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.771+0000 7f7b9d959b80 1 Processor -- start 2024-05-08T23:54:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b9d959b80 1 -- start start 2024-05-08T23:54:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc51d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 0x55d7bbc46910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbc4cda0 con 0x55d7bbc52270 2024-05-08T23:54:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbc480f0 con 0x55d7bbc51950 2024-05-08T23:54:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbc4b1a0 con 0x55d7bbc4c480 2024-05-08T23:54:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59016/0 (socket says 172.21.15.12:59016) 2024-05-08T23:54:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b8b7fe700 1 -- 172.21.15.12:0/1525775960 learned_addr learned my addr 172.21.15.12:0/1525775960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b8affd700 1 --2- 172.21.15.12:0/1525775960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc51d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.772+0000 7f7b827fc700 1 --2- 172.21.15.12:0/1525775960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 0x55d7bbc46910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.773+0000 7f7b8b7fe700 1 -- 172.21.15.12:0/1525775960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 msgr2=0x55d7bbc51d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.773+0000 7f7b8b7fe700 1 --2- 172.21.15.12:0/1525775960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc51d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.773+0000 7f7b8b7fe700 1 -- 172.21.15.12:0/1525775960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 msgr2=0x55d7bbc46910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.773+0000 7f7b8b7fe700 1 --2- 172.21.15.12:0/1525775960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 0x55d7bbc46910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.773+0000 7f7b8b7fe700 1 -- 172.21.15.12:0/1525775960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7bbb62500 con 0x55d7bbc52270 2024-05-08T23:54:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.773+0000 7f7b827fc700 1 --2- 172.21.15.12:0/1525775960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 0x55d7bbc46910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.773+0000 7f7b8b7fe700 1 --2- 172.21.15.12:0/1525775960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4c860 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7b840099c0 tx=0x7f7b840099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ffcbce9f3f45ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.773+0000 7f7b81ffb700 1 -- 172.21.15.12:0/1525775960 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84004450 con 0x55d7bbc52270 2024-05-08T23:54:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.773+0000 7f7b81ffb700 1 -- 172.21.15.12:0/1525775960 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b8400f040 con 0x55d7bbc52270 2024-05-08T23:54:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.773+0000 7f7b81ffb700 1 -- 172.21.15.12:0/1525775960 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b840073e0 con 0x55d7bbc52270 2024-05-08T23:54:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 -- 172.21.15.12:0/1525775960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 msgr2=0x55d7bbc4c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/1525775960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4c860 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7b840099c0 tx=0x7f7b840099f0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 -- 172.21.15.12:0/1525775960 shutdown_connections 2024-05-08T23:54:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/1525775960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc4c860 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/1525775960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc51d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/1525775960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 0x55d7bbc46910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 -- 172.21.15.12:0/1525775960 >> 172.21.15.12:0/1525775960 conn(0x55d7bbbaa1b0 msgr2=0x55d7bbc6c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 -- 172.21.15.12:0/1525775960 shutdown_connections 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 -- 172.21.15.12:0/1525775960 wait complete. 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 -- start start 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.774+0000 7f7b9d959b80 1 Processor -- start 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc5ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 0x55d7bbc58c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b9d959b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc59150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbc59720 con 0x55d7bbc52270 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbc59890 con 0x55d7bbc51950 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b9d959b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7bbc59a50 con 0x55d7bbc4c480 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b9d959b80 10 client.0 _ll_register_callbacks cb 0x55d7bbb41ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc59150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b8affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc5ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc59150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59028/0 (socket says 172.21.15.12:59028) 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b8b7fe700 1 -- 172.21.15.12:0/638195444 learned_addr learned my addr 172.21.15.12:0/638195444 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b827fc700 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 0x55d7bbc58c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b8affd700 1 -- 172.21.15.12:0/638195444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 msgr2=0x55d7bbc58c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b8affd700 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 0x55d7bbc58c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b8affd700 1 -- 172.21.15.12:0/638195444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 msgr2=0x55d7bbc59150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b8affd700 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc59150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b8affd700 1 -- 172.21.15.12:0/638195444 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b84003d20 con 0x55d7bbc4c480 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b8affd700 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc5ceb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c006010 tx=0x7f7b7c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=116c98491b51b3a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.775+0000 7f7b727fc700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b727fc700 1 -- 172.21.15.12:0/638195444 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c03d470 con 0x55d7bbc4c480 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b9d959b80 1 -- 172.21.15.12:0/638195444 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b74096790 con 0x55d7bbc4c480 2024-05-08T23:54:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b727fc700 1 -- 172.21.15.12:0/638195444 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7b7c03d5d0 con 0x55d7bbc4c480 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b727fc700 1 -- 172.21.15.12:0/638195444 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b7c03d790 con 0x55d7bbc4c480 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b9d959b80 10 client.27478 fetch_fsmap learned FSMap version 38 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b9d959b80 1 -- 172.21.15.12:0/638195444 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55d7bba21c70 con 0x55d7bbc4c480 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b727fc700 1 -- 172.21.15.12:0/638195444 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b74096790 con 0x55d7bbc4c480 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b727fc700 1 -- 172.21.15.12:0/638195444 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7b7c044070 con 0x55d7bbc4c480 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b9d959b80 10 client.27478 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b9d959b80 10 client.27478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b9d959b80 20 client.27478 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7bbcbc220 0x55d7bbcbe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b9d959b80 4 client.27478 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.776+0000 7f7b9d959b80 1 -- 172.21.15.12:0/638195444 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7bbb94e60 con 0x55d7bbcbc220 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.777+0000 7f7b827fc700 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7bbcbc220 0x55d7bbcbe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.777+0000 7f7b827fc700 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7bbcbc220 0x55d7bbcbe640 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a14d74814316566 server_cookie=5c393de8733b43e7 in_seq=0 out_seq=0 2024-05-08T23:54:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.777+0000 7f7b727fc700 10 client.27478 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.829+0000 7f7b727fc700 1 -- 172.21.15.12:0/638195444 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7bbb94e60 con 0x55d7bbcbc220 2024-05-08T23:54:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.829+0000 7f7b727fc700 10 client.27478 handle_command_reply: tid=0 2024-05-08T23:54:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.829+0000 7f7b9d959b80 1 client.27478 shutdown 2024-05-08T23:54:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.829+0000 7f7b9d959b80 1 -- 172.21.15.12:0/638195444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 msgr2=0x55d7bbc5ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.829+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc5ceb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c006010 tx=0x7f7b7c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.830+0000 7f7b9d959b80 1 -- 172.21.15.12:0/638195444 shutdown_connections 2024-05-08T23:54:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.830+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7bbc52270 0x55d7bbc59150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.830+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7bbc4c480 0x55d7bbc5ceb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.830+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d7bbcbc220 0x55d7bbcbe640 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.830+0000 7f7b9d959b80 1 --2- 172.21.15.12:0/638195444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7bbc51950 0x55d7bbc58c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.830+0000 7f7b9d959b80 1 -- 172.21.15.12:0/638195444 >> 172.21.15.12:0/638195444 conn(0x55d7bbbaa1b0 msgr2=0x55d7bbc53cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.830+0000 7f7b727fc700 0 client.27478 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.830+0000 7f7b9d959b80 1 -- 172.21.15.12:0/638195444 shutdown_connections 2024-05-08T23:54:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.830+0000 7f7b9d959b80 1 -- 172.21.15.12:0/638195444 wait complete. 2024-05-08T23:54:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.830+0000 7f7b9d959b80 20 client.27478 trim_cache size 0 max 16384 2024-05-08T23:54:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:54:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:54:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:54:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:54:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 -- 172.21.15.12:0/918126138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b780a1290 msgr2=0x7f7b780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b780a1290 0x7f7b780a3750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00b2e0 tx=0x7f7b7c005940 comp rx=0 tx=0).stop 2024-05-08T23:54:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 -- 172.21.15.12:0/918126138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b8c105130 msgr2=0x7f7b8c1978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b8c105130 0x7f7b8c1978b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400e970 tx=0x7f7b7400ec80 comp rx=0 tx=0).stop 2024-05-08T23:54:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 -- 172.21.15.12:0/918126138 shutdown_connections 2024-05-08T23:54:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b780a1290 0x7f7b780a3750 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c19bc00 0x7f7b8c19c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b8c100010 0x7f7b8c197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 --2- 172.21.15.12:0/918126138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b8c105130 0x7f7b8c1978b0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 -- 172.21.15.12:0/918126138 >> 172.21.15.12:0/918126138 conn(0x7f7b8c076f40 msgr2=0x7f7b8c069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 -- 172.21.15.12:0/918126138 shutdown_connections 2024-05-08T23:54:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:39.831+0000 7f7b9179f700 1 -- 172.21.15.12:0/918126138 wait complete. 2024-05-08T23:54:39.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 495544 7.88 MB/sec execute 275 sec latency 1793.490 ms 2024-05-08T23:54:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:39 smithi012 ceph-mon[42503]: pgmap v258: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-05-08T23:54:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:39 smithi204 ceph-mon[47373]: pgmap v258: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-05-08T23:54:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:39 smithi160 ceph-mon[46152]: pgmap v258: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-05-08T23:54:40.512 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:54:40 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-08T23:54:40.512 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:54:40 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-08T23:54:40.513 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:54:40 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/1798175504 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b1550161a0 con 0x55b154fed000 2024-05-08T23:54:40.513 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:54:40 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-08T23:54:40.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 495544 7.86 MB/sec execute 276 sec latency 2793.600 ms 2024-05-08T23:54:40.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:40 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-08T23:54:40.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:40 smithi012 ceph-mon[42503]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T23:54:41.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:40 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-08T23:54:41.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:40 smithi204 ceph-mon[47373]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T23:54:41.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:40 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-08T23:54:41.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:40 smithi160 ceph-mon[46152]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T23:54:41.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 497671 7.88 MB/sec execute 277 sec latency 2957.510 ms 2024-05-08T23:54:41.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:41 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/3332734891,v1:172.21.15.204:6837/3332734891] up:boot 2024-05-08T23:54:41.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:41 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:54:41.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:54:41.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:41 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:54:41.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:41 smithi012 ceph-mon[42503]: pgmap v260: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-08T23:54:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:41 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/3332734891,v1:172.21.15.204:6837/3332734891] up:boot 2024-05-08T23:54:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:41 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:54:42.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:54:42.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:41 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:54:42.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:41 smithi204 ceph-mon[47373]: pgmap v260: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-08T23:54:42.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:41 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/3332734891,v1:172.21.15.204:6837/3332734891] up:boot 2024-05-08T23:54:42.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:41 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:54:42.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:54:42.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:41 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:54:42.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:41 smithi160 ceph-mon[46152]: pgmap v260: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-08T23:54:42.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 498990 7.87 MB/sec execute 278 sec latency 20.275 ms 2024-05-08T23:54:43.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499092 7.84 MB/sec execute 279 sec latency 864.841 ms 2024-05-08T23:54:44.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:44 smithi204 ceph-mon[47373]: pgmap v261: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-08T23:54:44.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:44 smithi160 ceph-mon[46152]: pgmap v261: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-08T23:54:44.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499092 7.81 MB/sec execute 280 sec latency 1864.950 ms 2024-05-08T23:54:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:44 smithi012 ceph-mon[42503]: pgmap v261: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-08T23:54:45.876 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499092 7.78 MB/sec execute 281 sec latency 2865.013 ms 2024-05-08T23:54:45.891 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:54:46.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.050+0000 7f16c0fba700 1 -- 172.21.15.12:0/2734991830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc06a120 msgr2=0x7f16bc105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.050+0000 7f16c0fba700 1 --2- 172.21.15.12:0/2734991830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc06a120 0x7f16bc105b50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009990 tx=0x7f16b0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.050+0000 7f16c0fba700 1 -- 172.21.15.12:0/2734991830 shutdown_connections 2024-05-08T23:54:46.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.050+0000 7f16c0fba700 1 --2- 172.21.15.12:0/2734991830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16bc106280 0x7f16bc10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.050+0000 7f16c0fba700 1 --2- 172.21.15.12:0/2734991830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16bc069770 0x7f16bc069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.050+0000 7f16c0fba700 1 --2- 172.21.15.12:0/2734991830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc06a120 0x7f16bc105b50 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.050+0000 7f16c0fba700 1 -- 172.21.15.12:0/2734991830 >> 172.21.15.12:0/2734991830 conn(0x7f16bc076fa0 msgr2=0x7f16bc0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.050+0000 7f16c0fba700 1 -- 172.21.15.12:0/2734991830 shutdown_connections 2024-05-08T23:54:46.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.050+0000 7f16c0fba700 1 -- 172.21.15.12:0/2734991830 wait complete. 2024-05-08T23:54:46.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.051+0000 7f16c0fba700 1 Processor -- start 2024-05-08T23:54:46.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.051+0000 7f16c0fba700 1 -- start start 2024-05-08T23:54:46.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16c0fba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc069770 0x7f16bc197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16c0fba700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16bc106280 0x7f16bc197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16c0fba700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16bc19bce0 0x7f16bc19c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16c0fba700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19c760 con 0x7f16bc069770 2024-05-08T23:54:46.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16c0fba700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19c8d0 con 0x7f16bc106280 2024-05-08T23:54:46.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16c0fba700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19ca90 con 0x7f16bc19bce0 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc069770 0x7f16bc197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc069770 0x7f16bc197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59050/0 (socket says 172.21.15.12:59050) 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16b9d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16bc106280 0x7f16bc197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16bad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16bc19bce0 0x7f16bc19c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16ba59c700 1 -- 172.21.15.12:0/1805324388 learned_addr learned my addr 172.21.15.12:0/1805324388 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16b9d9b700 1 -- 172.21.15.12:0/1805324388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16bc19bce0 msgr2=0x7f16bc19c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16bc19bce0 0x7f16bc19c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16b9d9b700 1 -- 172.21.15.12:0/1805324388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc069770 msgr2=0x7f16bc197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc069770 0x7f16bc197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16b9d9b700 1 -- 172.21.15.12:0/1805324388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac009520 con 0x7f16bc106280 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16ba59c700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc069770 0x7f16bc197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.052+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16bc106280 0x7f16bc197940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f16b0000c00 tx=0x7f16b0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.053+0000 7f16ab7fe700 1 -- 172.21.15.12:0/1805324388 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b001d070 con 0x7f16bc106280 2024-05-08T23:54:46.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.053+0000 7f16c0fba700 1 -- 172.21.15.12:0/1805324388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b00095f0 con 0x7f16bc106280 2024-05-08T23:54:46.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.053+0000 7f16ab7fe700 1 -- 172.21.15.12:0/1805324388 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b0021410 con 0x7f16bc106280 2024-05-08T23:54:46.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.053+0000 7f16ab7fe700 1 -- 172.21.15.12:0/1805324388 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0017650 con 0x7f16bc106280 2024-05-08T23:54:46.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.053+0000 7f16c0fba700 1 -- 172.21.15.12:0/1805324388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc1a4220 con 0x7f16bc106280 2024-05-08T23:54:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.055+0000 7f16ab7fe700 1 -- 172.21.15.12:0/1805324388 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f16b0021580 con 0x7f16bc106280 2024-05-08T23:54:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.056+0000 7f16ab7fe700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f16a40a1290 0x7f16a40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.056+0000 7f16ab7fe700 1 -- 172.21.15.12:0/1805324388 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f16b00cde70 con 0x7f16bc106280 2024-05-08T23:54:46.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.056+0000 7f16ba59c700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f16a40a1290 0x7f16a40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.057+0000 7f16ba59c700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f16a40a1290 0x7f16a40a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009fd0 tx=0x7f16ac009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.057+0000 7f16cd174b80 1 Processor -- start 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.057+0000 7f16cd174b80 1 -- start start 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.057+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582155057e0 0x558215505bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.057+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582154f7350 0x5582154f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.057+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x55821550c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.057+0000 7f16cd174b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821544e150 con 0x5582155057e0 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.057+0000 7f16cd174b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821544bb10 con 0x5582154f7d10 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.057+0000 7f16cd174b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558215447400 con 0x5582154f7350 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.057+0000 7f16b9d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582154f7350 0x5582154f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16b9d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582154f7350 0x5582154f77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39332/0 (socket says 172.21.15.12:39332) 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16b9d9b700 1 -- 172.21.15.12:0/1815107871 learned_addr learned my addr 172.21.15.12:0/1815107871 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16b9d9b700 1 -- 172.21.15.12:0/1815107871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 msgr2=0x55821550c4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/1815107871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x55821550c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16b9d9b700 1 -- 172.21.15.12:0/1815107871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582155057e0 msgr2=0x558215505bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/1815107871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582155057e0 0x558215505bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16b9d9b700 1 -- 172.21.15.12:0/1815107871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582152c83b0 con 0x5582154f7350 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/1815107871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582154f7350 0x5582154f77d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16b00974f0 tx=0x7f16b009adc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81c5dcffc37fc4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16a8ff9700 1 -- 172.21.15.12:0/1815107871 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b009b6f0 con 0x5582154f7350 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16a8ff9700 1 -- 172.21.15.12:0/1815107871 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b009aff0 con 0x5582154f7350 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.058+0000 7f16a8ff9700 1 -- 172.21.15.12:0/1815107871 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b009b250 con 0x5582154f7350 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.059+0000 7f16cd174b80 1 -- 172.21.15.12:0/1815107871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582154f7350 msgr2=0x5582154f77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.059+0000 7f16cd174b80 1 --2- 172.21.15.12:0/1815107871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582154f7350 0x5582154f77d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16b00974f0 tx=0x7f16b009adc0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.060+0000 7f16cd174b80 1 -- 172.21.15.12:0/1815107871 shutdown_connections 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.060+0000 7f16cd174b80 1 --2- 172.21.15.12:0/1815107871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x55821550c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.060+0000 7f16cd174b80 1 --2- 172.21.15.12:0/1815107871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582155057e0 0x558215505bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.060+0000 7f16cd174b80 1 --2- 172.21.15.12:0/1815107871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582154f7350 0x5582154f77d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.060+0000 7f16cd174b80 1 -- 172.21.15.12:0/1815107871 >> 172.21.15.12:0/1815107871 conn(0x558215455210 msgr2=0x5582154f1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.060+0000 7f16cd174b80 1 -- 172.21.15.12:0/1815107871 shutdown_connections 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.060+0000 7f16cd174b80 1 -- 172.21.15.12:0/1815107871 wait complete. 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.060+0000 7f16cd174b80 1 -- start start 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.060+0000 7f16cd174b80 1 Processor -- start 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x55821550f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x55821550f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x558215509540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16cd174b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582153e3560 con 0x5582154f7350 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16cd174b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582153e3200 con 0x5582154f7d10 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16cd174b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582152ca6a0 con 0x5582155057e0 2024-05-08T23:54:46.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16cd174b80 10 client.0 _ll_register_callbacks cb 0x5582153ecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x55821550f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x55821550f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59062/0 (socket says 172.21.15.12:59062) 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16ba59c700 1 -- 172.21.15.12:0/4144837766 learned_addr learned my addr 172.21.15.12:0/4144837766 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16bad9d700 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x558215509540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x55821550f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16ba59c700 1 -- 172.21.15.12:0/4144837766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 msgr2=0x558215509540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16ba59c700 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x558215509540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16ba59c700 1 -- 172.21.15.12:0/4144837766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 msgr2=0x55821550f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16ba59c700 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x55821550f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16ba59c700 1 -- 172.21.15.12:0/4144837766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc1a4220 con 0x5582154f7350 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.061+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x55821550f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.062+0000 7f16ba59c700 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x55821550f280 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00fbe0 tx=0x7f16ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adefc675f9f503a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.062+0000 7f16a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.062+0000 7f16a1ffb700 1 -- 172.21.15.12:0/4144837766 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac01dc40 con 0x5582154f7350 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.062+0000 7f16a1ffb700 1 -- 172.21.15.12:0/4144837766 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f16ac01dda0 con 0x5582154f7350 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.062+0000 7f16cd174b80 1 -- 172.21.15.12:0/4144837766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582153a0270 con 0x5582154f7350 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.062+0000 7f16a1ffb700 1 -- 172.21.15.12:0/4144837766 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16ac030420 con 0x5582154f7350 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.062+0000 7f16cd174b80 10 client.17715 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.062+0000 7f16cd174b80 1 -- 172.21.15.12:0/4144837766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558215274b60 con 0x5582154f7350 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.062+0000 7f16a1ffb700 1 -- 172.21.15.12:0/4144837766 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16ac034020 con 0x5582154f7350 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.063+0000 7f16a1ffb700 1 -- 172.21.15.12:0/4144837766 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f16ac01b6b0 con 0x5582154f7350 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.063+0000 7f16cd174b80 10 client.17715 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.063+0000 7f16cd174b80 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.063+0000 7f16cd174b80 20 client.17715 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.063+0000 7f16cd174b80 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558215519820 0x55821551bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.063+0000 7f16cd174b80 4 client.17715 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.063+0000 7f16cd174b80 1 -- 172.21.15.12:0/4144837766 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582154c6810 con 0x558215519820 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.063+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558215519820 0x55821551bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.063+0000 7f16a1ffb700 10 client.17715 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.063+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558215519820 0x55821551bc40 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262ebb3cf35407c server_cookie=1a6fce95db08d621 in_seq=0 out_seq=0 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16a1ffb700 1 -- 172.21.15.12:0/4144837766 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582154c6810 con 0x558215519820 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16a1ffb700 10 client.17715 handle_command_reply: tid=0 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 client.17715 shutdown 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 -- 172.21.15.12:0/4144837766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 msgr2=0x55821550f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x55821550f280 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00fbe0 tx=0x7f16ac004750 comp rx=0 tx=0).stop 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 -- 172.21.15.12:0/4144837766 shutdown_connections 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x558215509540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x55821550f280 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558215519820 0x55821551bc40 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 --2- 172.21.15.12:0/4144837766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x55821550f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16a1ffb700 0 client.17715 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:46.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 -- 172.21.15.12:0/4144837766 >> 172.21.15.12:0/4144837766 conn(0x558215455210 msgr2=0x5582154f1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 -- 172.21.15.12:0/4144837766 shutdown_connections 2024-05-08T23:54:46.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.064+0000 7f16cd174b80 1 -- 172.21.15.12:0/4144837766 wait complete. 2024-05-08T23:54:46.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.065+0000 7f16cd174b80 20 client.17715 trim_cache size 0 max 16384 2024-05-08T23:54:46.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.086+0000 7f16cd174b80 1 Processor -- start 2024-05-08T23:54:46.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.086+0000 7f16cd174b80 1 -- start start 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7350 0x5582154f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7d10 0x558215504930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x558215504e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16cd174b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582154ea040 con 0x5582154f7d10 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16cd174b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558215516e90 con 0x5582154f7350 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16cd174b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582154f2260 con 0x5582155057e0 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16b9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7d10 0x558215504930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16b9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7d10 0x558215504930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59064/0 (socket says 172.21.15.12:59064) 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16b9d9b700 1 -- 172.21.15.12:0/2819459629 learned_addr learned my addr 172.21.15.12:0/2819459629 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16bad9d700 1 --2- 172.21.15.12:0/2819459629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x558215504e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16ba59c700 1 --2- 172.21.15.12:0/2819459629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7350 0x5582154f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16b9d9b700 1 -- 172.21.15.12:0/2819459629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 msgr2=0x558215504e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/2819459629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x558215504e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16b9d9b700 1 -- 172.21.15.12:0/2819459629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7350 msgr2=0x5582154f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/2819459629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7350 0x5582154f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16b9d9b700 1 -- 172.21.15.12:0/2819459629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55821540d560 con 0x5582154f7d10 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.087+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/2819459629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7d10 0x558215504930 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f16b000b2e0 tx=0x7f16b0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bd30e226b62756 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16a97fa700 1 -- 172.21.15.12:0/2819459629 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0060ea0 con 0x5582154f7d10 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16a97fa700 1 -- 172.21.15.12:0/2819459629 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b009ee20 con 0x5582154f7d10 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16a97fa700 1 -- 172.21.15.12:0/2819459629 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b009a4e0 con 0x5582154f7d10 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16cd174b80 1 -- 172.21.15.12:0/2819459629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7d10 msgr2=0x558215504930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16cd174b80 1 --2- 172.21.15.12:0/2819459629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7d10 0x558215504930 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f16b000b2e0 tx=0x7f16b0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16cd174b80 1 -- 172.21.15.12:0/2819459629 shutdown_connections 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16cd174b80 1 --2- 172.21.15.12:0/2819459629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x558215504e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16cd174b80 1 --2- 172.21.15.12:0/2819459629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7350 0x5582154f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16cd174b80 1 --2- 172.21.15.12:0/2819459629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7d10 0x558215504930 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16cd174b80 1 -- 172.21.15.12:0/2819459629 >> 172.21.15.12:0/2819459629 conn(0x558215455210 msgr2=0x558215507e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16cd174b80 1 -- 172.21.15.12:0/2819459629 shutdown_connections 2024-05-08T23:54:46.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.088+0000 7f16cd174b80 1 -- 172.21.15.12:0/2819459629 wait complete. 2024-05-08T23:54:46.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16cd174b80 1 -- start start 2024-05-08T23:54:46.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16cd174b80 1 Processor -- start 2024-05-08T23:54:46.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x558215513ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x558215514010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16cd174b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x55821556ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16cd174b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821556f260 con 0x5582154f7350 2024-05-08T23:54:46.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16cd174b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821556f3a0 con 0x5582154f7d10 2024-05-08T23:54:46.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16cd174b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821556f560 con 0x5582155057e0 2024-05-08T23:54:46.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16cd174b80 10 client.0 _ll_register_callbacks cb 0x5582153ecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:46.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x558215513ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.089+0000 7f16ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x558215513ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59080/0 (socket says 172.21.15.12:59080) 2024-05-08T23:54:46.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16ba59c700 1 -- 172.21.15.12:0/2784746628 learned_addr learned my addr 172.21.15.12:0/2784746628 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16bad9d700 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x55821556ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x558215514010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16ba59c700 1 -- 172.21.15.12:0/2784746628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 msgr2=0x55821556ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16ba59c700 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x55821556ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16ba59c700 1 -- 172.21.15.12:0/2784746628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 msgr2=0x558215514010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16ba59c700 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x558215514010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16ba59c700 1 -- 172.21.15.12:0/2784746628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582152c83b0 con 0x5582154f7350 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16ba59c700 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x558215513ad0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f16ac016040 tx=0x7f16ac01b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392e11ca9a6c81fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16a1ffb700 1 -- 172.21.15.12:0/2784746628 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac01b830 con 0x5582154f7350 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16a1ffb700 1 -- 172.21.15.12:0/2784746628 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f16ac035080 con 0x5582154f7350 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16cd174b80 1 -- 172.21.15.12:0/2784746628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16b009aff0 con 0x5582154f7350 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16a1ffb700 1 -- 172.21.15.12:0/2784746628 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16ac01de50 con 0x5582154f7350 2024-05-08T23:54:46.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16cd174b80 10 client.17721 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.090+0000 7f16cd174b80 1 -- 172.21.15.12:0/2784746628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5582152cccd0 con 0x5582154f7350 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16a1ffb700 1 -- 172.21.15.12:0/2784746628 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b009aff0 con 0x5582154f7350 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16a1ffb700 1 -- 172.21.15.12:0/2784746628 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f16ac004c00 con 0x5582154f7350 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16cd174b80 10 client.17721 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16cd174b80 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16cd174b80 20 client.17721 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16cd174b80 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558215570dd0 0x5582155731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16cd174b80 4 client.17721 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16cd174b80 1 -- 172.21.15.12:0/2784746628 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55821543fec0 con 0x558215570dd0 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558215570dd0 0x5582155731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16b9d9b700 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558215570dd0 0x5582155731f0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e83c53ca00756b7 server_cookie=7922860c2e94e99d in_seq=0 out_seq=0 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.091+0000 7f16a1ffb700 10 client.17721 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16a1ffb700 1 -- 172.21.15.12:0/2784746628 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55821543fec0 con 0x558215570dd0 2024-05-08T23:54:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16a1ffb700 10 client.17721 handle_command_reply: tid=0 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 client.17721 shutdown 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 -- 172.21.15.12:0/2784746628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 msgr2=0x558215513ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x558215513ad0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f16ac016040 tx=0x7f16ac01b6c0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 -- 172.21.15.12:0/2784746628 shutdown_connections 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5582155057e0 0x55821556ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582154f7350 0x558215513ad0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558215570dd0 0x5582155731f0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 --2- 172.21.15.12:0/2784746628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582154f7d10 0x558215514010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16a1ffb700 0 client.17721 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 -- 172.21.15.12:0/2784746628 >> 172.21.15.12:0/2784746628 conn(0x558215455210 msgr2=0x558215504930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 -- 172.21.15.12:0/2784746628 shutdown_connections 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 1 -- 172.21.15.12:0/2784746628 wait complete. 2024-05-08T23:54:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16cd174b80 20 client.17721 trim_cache size 0 max 16384 2024-05-08T23:54:46.094 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:54:46.094 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:54:46.094 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:54:46.094 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:54:46.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16c0fba700 1 -- 172.21.15.12:0/1805324388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f16a40a1290 msgr2=0x7f16a40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16c0fba700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f16a40a1290 0x7f16a40a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009fd0 tx=0x7f16ac009260 comp rx=0 tx=0).stop 2024-05-08T23:54:46.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16c0fba700 1 -- 172.21.15.12:0/1805324388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16bc106280 msgr2=0x7f16bc197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.092+0000 7f16c0fba700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16bc106280 0x7f16bc197940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f16b0000c00 tx=0x7f16b0004730 comp rx=0 tx=0).stop 2024-05-08T23:54:46.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.093+0000 7f16c0fba700 1 -- 172.21.15.12:0/1805324388 shutdown_connections 2024-05-08T23:54:46.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.093+0000 7f16c0fba700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f16a40a1290 0x7f16a40a3750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.093+0000 7f16c0fba700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16bc19bce0 0x7f16bc19c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.093+0000 7f16c0fba700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16bc069770 0x7f16bc197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.093+0000 7f16c0fba700 1 --2- 172.21.15.12:0/1805324388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16bc106280 0x7f16bc197940 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.093+0000 7f16c0fba700 1 -- 172.21.15.12:0/1805324388 >> 172.21.15.12:0/1805324388 conn(0x7f16bc076fa0 msgr2=0x7f16bc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.093+0000 7f16c0fba700 1 -- 172.21.15.12:0/1805324388 shutdown_connections 2024-05-08T23:54:46.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.093+0000 7f16c0fba700 1 -- 172.21.15.12:0/1805324388 wait complete. 2024-05-08T23:54:46.100 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:54:46.101 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:54:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.243+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2881638748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d081008f0 msgr2=0x7f2d08104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.243+0000 7f2d0cfd8700 1 --2- 172.21.15.12:0/2881638748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d081008f0 0x7f2d08104930 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009990 tx=0x7f2cfc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.243+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2881638748 shutdown_connections 2024-05-08T23:54:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.243+0000 7f2d0cfd8700 1 --2- 172.21.15.12:0/2881638748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d08105060 0x7f2d0810cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.243+0000 7f2d0cfd8700 1 --2- 172.21.15.12:0/2881638748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d080fff40 0x7f2d08100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.243+0000 7f2d0cfd8700 1 --2- 172.21.15.12:0/2881638748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d081008f0 0x7f2d08104930 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.243+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2881638748 >> 172.21.15.12:0/2881638748 conn(0x7f2d080fb7e0 msgr2=0x7f2d080fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.243+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2881638748 shutdown_connections 2024-05-08T23:54:46.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.243+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2881638748 wait complete. 2024-05-08T23:54:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0cfd8700 1 Processor -- start 2024-05-08T23:54:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0cfd8700 1 -- start start 2024-05-08T23:54:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0cfd8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d080fff40 0x7f2d08199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0cfd8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d08105060 0x7f2d08199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0cfd8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0819de30 0x7f2d0819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0cfd8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819e8b0 con 0x7f2d080fff40 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0cfd8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819ea20 con 0x7f2d08105060 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0cfd8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819ebe0 con 0x7f2d0819de30 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d080fff40 0x7f2d08199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d05d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d08105060 0x7f2d08199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d080fff40 0x7f2d08199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59098/0 (socket says 172.21.15.12:59098) 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d0659c700 1 -- 172.21.15.12:0/2407375388 learned_addr learned my addr 172.21.15.12:0/2407375388 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.244+0000 7f2d06d9d700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0819de30 0x7f2d0819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2d05d9b700 1 -- 172.21.15.12:0/2407375388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0819de30 msgr2=0x7f2d0819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2d05d9b700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0819de30 0x7f2d0819e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2d05d9b700 1 -- 172.21.15.12:0/2407375388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d080fff40 msgr2=0x7f2d08199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2d05d9b700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d080fff40 0x7f2d08199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2d05d9b700 1 -- 172.21.15.12:0/2407375388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc0095f0 con 0x7f2d08105060 2024-05-08T23:54:46.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2d0659c700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d080fff40 0x7f2d08199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:46.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2d05d9b700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d08105060 0x7f2d08199a90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006010 tx=0x7f2cfc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2cf77fe700 1 -- 172.21.15.12:0/2407375388 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc01d070 con 0x7f2d08105060 2024-05-08T23:54:46.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2407375388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d081a60e0 con 0x7f2d08105060 2024-05-08T23:54:46.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2cf77fe700 1 -- 172.21.15.12:0/2407375388 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cfc022470 con 0x7f2d08105060 2024-05-08T23:54:46.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2cf77fe700 1 -- 172.21.15.12:0/2407375388 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc017650 con 0x7f2d08105060 2024-05-08T23:54:46.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.245+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2407375388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d081a6560 con 0x7f2d08105060 2024-05-08T23:54:46.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.246+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2407375388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d081030a0 con 0x7f2d08105060 2024-05-08T23:54:46.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.248+0000 7f2cf77fe700 1 -- 172.21.15.12:0/2407375388 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2cfc022870 con 0x7f2d08105060 2024-05-08T23:54:46.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.248+0000 7f2cf77fe700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2cf00a1290 0x7f2cf00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.248+0000 7f2d0659c700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2cf00a1290 0x7f2cf00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.248+0000 7f2cf77fe700 1 -- 172.21.15.12:0/2407375388 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2cfc0ce270 con 0x7f2d08105060 2024-05-08T23:54:46.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.249+0000 7f2d0659c700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2cf00a1290 0x7f2cf00a3750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8011550 tx=0x7f2cf8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.252+0000 7f2cf77fe700 1 -- 172.21.15.12:0/2407375388 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2cfc097790 con 0x7f2d08105060 2024-05-08T23:54:46.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.526+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2407375388 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d0819a850 con 0x7f2d08105060 2024-05-08T23:54:46.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.527+0000 7f2cf77fe700 1 -- 172.21.15.12:0/2407375388 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+10883 (secure 0 0 0) 0x7f2cfc09adb0 con 0x7f2d08105060 2024-05-08T23:54:46.522 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:54:46.522 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:54:40.565738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27484":{"gid":27484,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3332734891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3332734891},{"type":"v1","addr":"172.21.15.204:6837","nonce":3332734891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:54:46.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2407375388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2cf00a1290 msgr2=0x7f2cf00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2cf00a1290 0x7f2cf00a3750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8011550 tx=0x7f2cf8009260 comp rx=0 tx=0).stop 2024-05-08T23:54:46.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2407375388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d08105060 msgr2=0x7f2d08199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d08105060 0x7f2d08199a90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006010 tx=0x7f2cfc004730 comp rx=0 tx=0).stop 2024-05-08T23:54:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2407375388 shutdown_connections 2024-05-08T23:54:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2cf00a1290 0x7f2cf00a3750 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0819de30 0x7f2d0819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d080fff40 0x7f2d08199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 --2- 172.21.15.12:0/2407375388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d08105060 0x7f2d08199a90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2407375388 >> 172.21.15.12:0/2407375388 conn(0x7f2d080fb7e0 msgr2=0x7f2d080fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2407375388 shutdown_connections 2024-05-08T23:54:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.529+0000 7f2d0cfd8700 1 -- 172.21.15.12:0/2407375388 wait complete. 2024-05-08T23:54:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 40 2024-05-08T23:54:46.538 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:54:46.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.683+0000 7fb59e17e700 1 -- 172.21.15.12:0/2898203091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5981000f0 msgr2=0x7fb598100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.683+0000 7fb59e17e700 1 --2- 172.21.15.12:0/2898203091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5981000f0 0x7fb598100570 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0099c0 tx=0x7fb58c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.683+0000 7fb59e17e700 1 -- 172.21.15.12:0/2898203091 shutdown_connections 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.683+0000 7fb59e17e700 1 --2- 172.21.15.12:0/2898203091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb598100ab0 0x7fb59810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.683+0000 7fb59e17e700 1 --2- 172.21.15.12:0/2898203091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb598105f90 0x7fb598106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.683+0000 7fb59e17e700 1 --2- 172.21.15.12:0/2898203091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5981000f0 0x7fb598100570 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.683+0000 7fb59e17e700 1 -- 172.21.15.12:0/2898203091 >> 172.21.15.12:0/2898203091 conn(0x7fb598076ba0 msgr2=0x7fb598076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.683+0000 7fb59e17e700 1 -- 172.21.15.12:0/2898203091 shutdown_connections 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.683+0000 7fb59e17e700 1 -- 172.21.15.12:0/2898203091 wait complete. 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb59e17e700 1 Processor -- start 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb59e17e700 1 -- start start 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb59e17e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5981000f0 0x7fb5981962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb59e17e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb598100ab0 0x7fb5981967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb59e17e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb598105f90 0x7fb59819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb59e17e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59819b1d0 con 0x7fb5981000f0 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb59e17e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59819b340 con 0x7fb598100ab0 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb59e17e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59819b500 con 0x7fb598105f90 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb5977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5981000f0 0x7fb5981962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb5977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5981000f0 0x7fb5981962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59110/0 (socket says 172.21.15.12:59110) 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.684+0000 7fb5977fe700 1 -- 172.21.15.12:0/2303088064 learned_addr learned my addr 172.21.15.12:0/2303088064 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb596ffd700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb598100ab0 0x7fb5981967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb597fff700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb598105f90 0x7fb59819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb5977fe700 1 -- 172.21.15.12:0/2303088064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb598105f90 msgr2=0x7fb59819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb5977fe700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb598105f90 0x7fb59819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb5977fe700 1 -- 172.21.15.12:0/2303088064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb598100ab0 msgr2=0x7fb5981967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb5977fe700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb598100ab0 0x7fb5981967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb5977fe700 1 -- 172.21.15.12:0/2303088064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c0095f0 con 0x7fb5981000f0 2024-05-08T23:54:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb5977fe700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5981000f0 0x7fb5981962b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb588007b50 tx=0x7fb588007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb594ff9700 1 -- 172.21.15.12:0/2303088064 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588011070 con 0x7fb5981000f0 2024-05-08T23:54:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb594ff9700 1 -- 172.21.15.12:0/2303088064 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb58800c2e0 con 0x7fb5981000f0 2024-05-08T23:54:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb59e17e700 1 -- 172.21.15.12:0/2303088064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59819b7e0 con 0x7fb5981000f0 2024-05-08T23:54:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb594ff9700 1 -- 172.21.15.12:0/2303088064 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800e500 con 0x7fb5981000f0 2024-05-08T23:54:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.685+0000 7fb59e17e700 1 -- 172.21.15.12:0/2303088064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb598069350 con 0x7fb5981000f0 2024-05-08T23:54:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.688+0000 7fb594ff9700 1 -- 172.21.15.12:0/2303088064 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb588003f70 con 0x7fb5981000f0 2024-05-08T23:54:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.689+0000 7fb594ff9700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5800a12d0 0x7fb5800a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.689+0000 7fb594ff9700 1 -- 172.21.15.12:0/2303088064 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb5880cdef0 con 0x7fb5981000f0 2024-05-08T23:54:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.689+0000 7fb596ffd700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5800a12d0 0x7fb5800a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb596ffd700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5800a12d0 0x7fb5800a3790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb58c000c00 tx=0x7fb58c005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb5aa338b80 1 Processor -- start 2024-05-08T23:54:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb5aa338b80 1 -- start start 2024-05-08T23:54:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435f991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 0x561435f93350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435f93cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561435eea150 con 0x561435f93890 2024-05-08T23:54:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561435ee7b10 con 0x561435f99720 2024-05-08T23:54:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561435ee3400 con 0x561435f98e00 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb597fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435f93cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb597fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435f93cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59118/0 (socket says 172.21.15.12:59118) 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb597fff700 1 -- 172.21.15.12:0/789518077 learned_addr learned my addr 172.21.15.12:0/789518077 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb5977fe700 1 --2- 172.21.15.12:0/789518077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435f991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb596ffd700 1 --2- 172.21.15.12:0/789518077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 0x561435f93350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb597fff700 1 -- 172.21.15.12:0/789518077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 msgr2=0x561435f991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb597fff700 1 --2- 172.21.15.12:0/789518077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435f991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb597fff700 1 -- 172.21.15.12:0/789518077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 msgr2=0x561435f93350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb597fff700 1 --2- 172.21.15.12:0/789518077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 0x561435f93350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb597fff700 1 -- 172.21.15.12:0/789518077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561435d643b0 con 0x561435f93890 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.690+0000 7fb596ffd700 1 --2- 172.21.15.12:0/789518077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 0x561435f93350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.691+0000 7fb597fff700 1 --2- 172.21.15.12:0/789518077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435f93cd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb59000f5a0 tx=0x7fb59000f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9b7e918ab8c442 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.691+0000 7fb57dffb700 1 -- 172.21.15.12:0/789518077 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590004bb0 con 0x561435f93890 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.691+0000 7fb57dffb700 1 -- 172.21.15.12:0/789518077 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb590004d10 con 0x561435f93890 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.691+0000 7fb57dffb700 1 -- 172.21.15.12:0/789518077 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590005040 con 0x561435f93890 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 -- 172.21.15.12:0/789518077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 msgr2=0x561435f93cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/789518077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435f93cd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb59000f5a0 tx=0x7fb59000f8b0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 -- 172.21.15.12:0/789518077 shutdown_connections 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/789518077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435f93cd0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/789518077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435f991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/789518077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 0x561435f93350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 -- 172.21.15.12:0/789518077 >> 172.21.15.12:0/789518077 conn(0x561435ef1210 msgr2=0x561435f8ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 -- 172.21.15.12:0/789518077 shutdown_connections 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 -- 172.21.15.12:0/789518077 wait complete. 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 -- start start 2024-05-08T23:54:46.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.692+0000 7fb5aa338b80 1 Processor -- start 2024-05-08T23:54:46.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f93890 0x561435f9cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435f9d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435f9d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561435e7f560 con 0x561435f99720 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561435e7f200 con 0x561435f93890 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561435d666a0 con 0x561435f98e00 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb5aa338b80 10 client.0 _ll_register_callbacks cb 0x561435e88b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb597fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435f9d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb597fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435f9d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59124/0 (socket says 172.21.15.12:59124) 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb597fff700 1 -- 172.21.15.12:0/3090609964 learned_addr learned my addr 172.21.15.12:0/3090609964 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb5977fe700 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f93890 0x561435f9cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb596ffd700 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435f9d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb597fff700 1 -- 172.21.15.12:0/3090609964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 msgr2=0x561435f9d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb597fff700 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435f9d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb597fff700 1 -- 172.21.15.12:0/3090609964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f93890 msgr2=0x561435f9cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb597fff700 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f93890 0x561435f9cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb597fff700 1 -- 172.21.15.12:0/3090609964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59000f1d0 con 0x561435f99720 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb596ffd700 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435f9d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb597fff700 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435f9d550 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb590014ea0 tx=0x7fb590004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33352eb5844718ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.693+0000 7fb576ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb576ffd700 1 -- 172.21.15.12:0/3090609964 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590014680 con 0x561435f99720 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb576ffd700 1 -- 172.21.15.12:0/3090609964 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb5900147e0 con 0x561435f99720 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3090609964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561435e3c270 con 0x561435f99720 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb576ffd700 1 -- 172.21.15.12:0/3090609964 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5900149a0 con 0x561435f99720 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb5aa338b80 10 client.17739 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3090609964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561435d10b60 con 0x561435f99720 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb576ffd700 1 -- 172.21.15.12:0/3090609964 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561435e3c270 con 0x561435f99720 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb576ffd700 1 -- 172.21.15.12:0/3090609964 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb590023020 con 0x561435f99720 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb5aa338b80 10 client.17739 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb5aa338b80 10 client.17739 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb5aa338b80 20 client.17739 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:46.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561435fad030 0x561435faf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb5aa338b80 4 client.17739 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.694+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3090609964 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561435f62810 con 0x561435fad030 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.695+0000 7fb5977fe700 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561435fad030 0x561435faf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.695+0000 7fb5977fe700 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561435fad030 0x561435faf450 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa540bf4be6c4e14 server_cookie=6cc90cf653c19a96 in_seq=0 out_seq=0 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.695+0000 7fb576ffd700 10 client.17739 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb576ffd700 1 -- 172.21.15.12:0/3090609964 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561435f62810 con 0x561435fad030 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb576ffd700 10 client.17739 handle_command_reply: tid=0 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 client.17739 shutdown 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3090609964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 msgr2=0x561435f9d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435f9d550 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb590014ea0 tx=0x7fb590004f60 comp rx=0 tx=0).stop 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3090609964 shutdown_connections 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435f9d550 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f93890 0x561435f9cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561435fad030 0x561435faf450 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3090609964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435f9d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3090609964 >> 172.21.15.12:0/3090609964 conn(0x561435ef1210 msgr2=0x561435f8d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb576ffd700 0 client.17739 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:54:46.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3090609964 shutdown_connections 2024-05-08T23:54:46.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3090609964 wait complete. 2024-05-08T23:54:46.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.696+0000 7fb5aa338b80 20 client.17739 trim_cache size 0 max 16384 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb5aa338b80 1 Processor -- start 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb5aa338b80 1 -- start start 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f93890 0x561435f93350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f98e00 0x561435f8dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435fa4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561435face60 con 0x561435f99720 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561435f9e0f0 con 0x561435f98e00 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561435f99580 con 0x561435f93890 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb597fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435fa4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb597fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435fa4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59138/0 (socket says 172.21.15.12:59138) 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb597fff700 1 -- 172.21.15.12:0/629900358 learned_addr learned my addr 172.21.15.12:0/629900358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb596ffd700 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f98e00 0x561435f8dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.710+0000 7fb5977fe700 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f93890 0x561435f93350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb597fff700 1 -- 172.21.15.12:0/629900358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f93890 msgr2=0x561435f93350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb597fff700 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f93890 0x561435f93350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb597fff700 1 -- 172.21.15.12:0/629900358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f98e00 msgr2=0x561435f8dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb597fff700 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f98e00 0x561435f8dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb597fff700 1 -- 172.21.15.12:0/629900358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561435ea9560 con 0x561435f99720 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb596ffd700 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f98e00 0x561435f8dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb5977fe700 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f93890 0x561435f93350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb597fff700 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435fa4160 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb59000f4c0 tx=0x7fb59002e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86ca013c55400ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb57e7fc700 1 -- 172.21.15.12:0/629900358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59002eaf0 con 0x561435f99720 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb57e7fc700 1 -- 172.21.15.12:0/629900358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb59000b070 con 0x561435f99720 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.711+0000 7fb57e7fc700 1 -- 172.21.15.12:0/629900358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590003f90 con 0x561435f99720 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 -- 172.21.15.12:0/629900358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 msgr2=0x561435fa4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435fa4160 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb59000f4c0 tx=0x7fb59002e7e0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 -- 172.21.15.12:0/629900358 shutdown_connections 2024-05-08T23:54:46.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f99720 0x561435fa4160 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f93890 0x561435f93350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/629900358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f98e00 0x561435f8dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 -- 172.21.15.12:0/629900358 >> 172.21.15.12:0/629900358 conn(0x561435ef1210 msgr2=0x561435f8f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 -- 172.21.15.12:0/629900358 shutdown_connections 2024-05-08T23:54:46.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 -- 172.21.15.12:0/629900358 wait complete. 2024-05-08T23:54:46.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 -- start start 2024-05-08T23:54:46.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 Processor -- start 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.712+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435fa0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435fa0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5aa338b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 0x561435fa0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561436002560 con 0x561435f93890 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614360026d0 con 0x561435f99720 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5aa338b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561436002890 con 0x561435f98e00 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5aa338b80 10 client.0 _ll_register_callbacks cb 0x561435e88b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435fa0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435fa0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59152/0 (socket says 172.21.15.12:59152) 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5977fe700 1 -- 172.21.15.12:0/3630408749 learned_addr learned my addr 172.21.15.12:0/3630408749 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb596ffd700 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435fa0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb597fff700 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 0x561435fa0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5977fe700 1 -- 172.21.15.12:0/3630408749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 msgr2=0x561435fa0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5977fe700 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435fa0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5977fe700 1 -- 172.21.15.12:0/3630408749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 msgr2=0x561435fa0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5977fe700 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 0x561435fa0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5977fe700 1 -- 172.21.15.12:0/3630408749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561435d10b60 con 0x561435f93890 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb597fff700 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 0x561435fa0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5977fe700 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435fa0140 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb58800e330 tx=0x7fb588005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bb5cbd3ca1350e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb576ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb576ffd700 1 -- 172.21.15.12:0/3630408749 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58809eea0 con 0x561435f93890 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb576ffd700 1 -- 172.21.15.12:0/3630408749 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb5880a3460 con 0x561435f93890 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.713+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3630408749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb590004d10 con 0x561435f93890 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb576ffd700 1 -- 172.21.15.12:0/3630408749 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb58801cda0 con 0x561435f93890 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb5aa338b80 10 client.17745 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3630408749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561435d68cd0 con 0x561435f93890 2024-05-08T23:54:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb576ffd700 1 -- 172.21.15.12:0/3630408749 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb590004d10 con 0x561435f93890 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb576ffd700 1 -- 172.21.15.12:0/3630408749 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb5880b9200 con 0x561435f93890 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb5aa338b80 10 client.17745 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb5aa338b80 10 client.17745 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb5aa338b80 20 client.17745 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561436003e70 0x561436006290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb5aa338b80 4 client.17745 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3630408749 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561435edbec0 con 0x561436003e70 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.714+0000 7fb596ffd700 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561436003e70 0x561436006290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb576ffd700 10 client.17745 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb596ffd700 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561436003e70 0x561436006290 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17fabf3dbe585c2c server_cookie=c3552f1d0dbece3c in_seq=0 out_seq=0 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb576ffd700 1 -- 172.21.15.12:0/3630408749 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561435edbec0 con 0x561436003e70 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb576ffd700 10 client.17745 handle_command_reply: tid=0 2024-05-08T23:54:46.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 client.17745 shutdown 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3630408749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 msgr2=0x561435fa0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435fa0140 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb58800e330 tx=0x7fb588005d80 comp rx=0 tx=0).stop 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3630408749 shutdown_connections 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561435f99720 0x561435fa0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561435f93890 0x561435fa0140 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561436003e70 0x561436006290 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 --2- 172.21.15.12:0/3630408749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561435f98e00 0x561435fa0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb576ffd700 0 client.17745 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3630408749 >> 172.21.15.12:0/3630408749 conn(0x561435ef1210 msgr2=0x561435f97b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3630408749 shutdown_connections 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 1 -- 172.21.15.12:0/3630408749 wait complete. 2024-05-08T23:54:46.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.715+0000 7fb5aa338b80 20 client.17745 trim_cache size 0 max 16384 2024-05-08T23:54:46.717 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:54:46.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 -- 172.21.15.12:0/2303088064 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5800a12d0 msgr2=0x7fb5800a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5800a12d0 0x7fb5800a3790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb58c000c00 tx=0x7fb58c005fb0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 -- 172.21.15.12:0/2303088064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5981000f0 msgr2=0x7fb5981962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5981000f0 0x7fb5981962b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb588007b50 tx=0x7fb588007e60 comp rx=0 tx=0).stop 2024-05-08T23:54:46.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 -- 172.21.15.12:0/2303088064 shutdown_connections 2024-05-08T23:54:46.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb5800a12d0 0x7fb5800a3790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb598105f90 0x7fb59819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5981000f0 0x7fb5981962b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 --2- 172.21.15.12:0/2303088064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb598100ab0 0x7fb5981967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 -- 172.21.15.12:0/2303088064 >> 172.21.15.12:0/2303088064 conn(0x7fb598076ba0 msgr2=0x7fb5980fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 -- 172.21.15.12:0/2303088064 shutdown_connections 2024-05-08T23:54:46.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.716+0000 7fb59e17e700 1 -- 172.21.15.12:0/2303088064 wait complete. 2024-05-08T23:54:46.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:54:46.750 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:46 smithi012 ceph-mon[42503]: pgmap v262: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-08T23:54:46.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:46 smithi204 ceph-mon[47373]: pgmap v262: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-08T23:54:46.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:46 smithi160 ceph-mon[46152]: pgmap v262: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-05-08T23:54:46.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.871+0000 7fc090e22700 1 -- 172.21.15.12:0/512340618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08c0682e0 msgr2=0x7fc08c0686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.871+0000 7fc090e22700 1 --2- 172.21.15.12:0/512340618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08c0682e0 0x7fc08c0686c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0099c0 tx=0x7fc07c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.871+0000 7fc090e22700 1 -- 172.21.15.12:0/512340618 shutdown_connections 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.871+0000 7fc090e22700 1 --2- 172.21.15.12:0/512340618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc08c106290 0x7fc08c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.871+0000 7fc090e22700 1 --2- 172.21.15.12:0/512340618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08c0682e0 0x7fc08c0686c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.871+0000 7fc090e22700 1 --2- 172.21.15.12:0/512340618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc08c068c90 0x7fc08c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.871+0000 7fc090e22700 1 -- 172.21.15.12:0/512340618 >> 172.21.15.12:0/512340618 conn(0x7fc08c076f40 msgr2=0x7fc08c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.871+0000 7fc090e22700 1 -- 172.21.15.12:0/512340618 shutdown_connections 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.871+0000 7fc090e22700 1 -- 172.21.15.12:0/512340618 wait complete. 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc090e22700 1 Processor -- start 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc090e22700 1 -- start start 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc090e22700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc08c0682e0 0x7fc08c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc090e22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08c106290 0x7fc08c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc090e22700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc08c19bcb0 0x7fc08c19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc090e22700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c19c730 con 0x7fc08c106290 2024-05-08T23:54:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc090e22700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c19c8a0 con 0x7fc08c19bcb0 2024-05-08T23:54:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc090e22700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c19ca60 con 0x7fc08c0682e0 2024-05-08T23:54:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc089d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08c106290 0x7fc08c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc089d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08c106290 0x7fc08c197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59162/0 (socket says 172.21.15.12:59162) 2024-05-08T23:54:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc089d9b700 1 -- 172.21.15.12:0/437787106 learned_addr learned my addr 172.21.15.12:0/437787106 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc08a59c700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc08c0682e0 0x7fc08c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.872+0000 7fc08ad9d700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc08c19bcb0 0x7fc08c19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc089d9b700 1 -- 172.21.15.12:0/437787106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc08c0682e0 msgr2=0x7fc08c197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc089d9b700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc08c0682e0 0x7fc08c197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc089d9b700 1 -- 172.21.15.12:0/437787106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc08c19bcb0 msgr2=0x7fc08c19c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc089d9b700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc08c19bcb0 0x7fc08c19c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc089d9b700 1 -- 172.21.15.12:0/437787106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0095f0 con 0x7fc08c106290 2024-05-08T23:54:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc089d9b700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08c106290 0x7fc08c197960 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc08000e970 tx=0x7fc08000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc07b7fe700 1 -- 172.21.15.12:0/437787106 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080009790 con 0x7fc08c106290 2024-05-08T23:54:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc07b7fe700 1 -- 172.21.15.12:0/437787106 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08000eec0 con 0x7fc08c106290 2024-05-08T23:54:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc090e22700 1 -- 172.21.15.12:0/437787106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c1a3fc0 con 0x7fc08c106290 2024-05-08T23:54:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc07b7fe700 1 -- 172.21.15.12:0/437787106 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080005590 con 0x7fc08c106290 2024-05-08T23:54:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.873+0000 7fc090e22700 1 -- 172.21.15.12:0/437787106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c1a4490 con 0x7fc08c106290 2024-05-08T23:54:46.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.875+0000 7fc07b7fe700 1 -- 172.21.15.12:0/437787106 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc08000cbd0 con 0x7fc08c106290 2024-05-08T23:54:46.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.876+0000 7fc07b7fe700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0740a1310 0x7fc0740a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.876+0000 7fc07b7fe700 1 -- 172.21.15.12:0/437787106 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc080014070 con 0x7fc08c106290 2024-05-08T23:54:46.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.876+0000 7fc08a59c700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0740a1310 0x7fc0740a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc08a59c700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0740a1310 0x7fc0740a37d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009fd0 tx=0x7fc07c019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc09cfdcb80 1 Processor -- start 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc09cfdcb80 1 -- start start 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e01720 0x564cf2dfb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2dfbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2d52150 con 0x564cf2e00e00 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2d4fb10 con 0x564cf2dfb890 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2d4b400 con 0x564cf2e01720 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc08a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc08a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59168/0 (socket says 172.21.15.12:59168) 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.877+0000 7fc08a59c700 1 -- 172.21.15.12:0/59184811 learned_addr learned my addr 172.21.15.12:0/59184811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc089d9b700 1 --2- 172.21.15.12:0/59184811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e01720 0x564cf2dfb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc08ad9d700 1 --2- 172.21.15.12:0/59184811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2dfbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc08a59c700 1 -- 172.21.15.12:0/59184811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e01720 msgr2=0x564cf2dfb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc08a59c700 1 --2- 172.21.15.12:0/59184811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e01720 0x564cf2dfb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc08a59c700 1 -- 172.21.15.12:0/59184811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 msgr2=0x564cf2dfbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc08a59c700 1 --2- 172.21.15.12:0/59184811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2dfbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc08a59c700 1 -- 172.21.15.12:0/59184811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf2bcc3b0 con 0x564cf2e00e00 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc08ad9d700 1 --2- 172.21.15.12:0/59184811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2dfbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc08a59c700 1 --2- 172.21.15.12:0/59184811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e011e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0099c0 tx=0x7fc07c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d299def89d9705d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc078ff9700 1 -- 172.21.15.12:0/59184811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00fac0 con 0x564cf2e00e00 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc078ff9700 1 -- 172.21.15.12:0/59184811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c01d040 con 0x564cf2e00e00 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.878+0000 7fc078ff9700 1 -- 172.21.15.12:0/59184811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c023de0 con 0x564cf2e00e00 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.879+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/59184811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 msgr2=0x564cf2e011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.879+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/59184811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e011e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0099c0 tx=0x7fc07c01f000 comp rx=0 tx=0).stop 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.879+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/59184811 shutdown_connections 2024-05-08T23:54:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.879+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/59184811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2dfbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.879+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/59184811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e011e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.879+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/59184811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e01720 0x564cf2dfb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.879+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/59184811 >> 172.21.15.12:0/59184811 conn(0x564cf2d59210 msgr2=0x564cf2df5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.879+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/59184811 shutdown_connections 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.879+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/59184811 wait complete. 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc09cfdcb80 1 -- start start 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc09cfdcb80 1 Processor -- start 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2dfb890 0x564cf2e04b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e05060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2e01720 0x564cf2e055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2ce7560 con 0x564cf2e00e00 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2ce7200 con 0x564cf2e01720 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2bce6a0 con 0x564cf2dfb890 2024-05-08T23:54:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc09cfdcb80 10 client.0 _ll_register_callbacks cb 0x564cf2cf0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc089d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e05060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc089d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e05060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59176/0 (socket says 172.21.15.12:59176) 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc089d9b700 1 -- 172.21.15.12:0/2972674098 learned_addr learned my addr 172.21.15.12:0/2972674098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc08a59c700 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2dfb890 0x564cf2e04b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc08ad9d700 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2e01720 0x564cf2e055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc089d9b700 1 -- 172.21.15.12:0/2972674098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2dfb890 msgr2=0x564cf2e04b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc089d9b700 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2dfb890 0x564cf2e04b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc089d9b700 1 -- 172.21.15.12:0/2972674098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2e01720 msgr2=0x564cf2e055a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc089d9b700 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2e01720 0x564cf2e055a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.880+0000 7fc089d9b700 1 -- 172.21.15.12:0/2972674098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c10e650 con 0x564cf2e00e00 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc089d9b700 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e05060 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc0800ca660 tx=0x7fc080099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc697ec1f25fbf20 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc071ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc071ffb700 1 -- 172.21.15.12:0/2972674098 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08009a220 con 0x564cf2e00e00 2024-05-08T23:54:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc071ffb700 1 -- 172.21.15.12:0/2972674098 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc08009a380 con 0x564cf2e00e00 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2972674098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564cf2ca4270 con 0x564cf2e00e00 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc071ffb700 1 -- 172.21.15.12:0/2972674098 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0800b8250 con 0x564cf2e00e00 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc09cfdcb80 10 client.17757 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2972674098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564cf2b78b60 con 0x564cf2e00e00 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc071ffb700 1 -- 172.21.15.12:0/2972674098 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564cf2ca4270 con 0x564cf2e00e00 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc071ffb700 1 -- 172.21.15.12:0/2972674098 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc080094bd0 con 0x564cf2e00e00 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc09cfdcb80 10 client.17757 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc09cfdcb80 10 client.17757 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc09cfdcb80 20 client.17757 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf2e19600 0x564cf2e1ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc09cfdcb80 4 client.17757 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.881+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2972674098 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cf2dca810 con 0x564cf2e19600 2024-05-08T23:54:46.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499095 7.76 MB/sec execute 282 sec latency 3039.960 ms 2024-05-08T23:54:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.882+0000 7fc08a59c700 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf2e19600 0x564cf2e1ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.882+0000 7fc08a59c700 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf2e19600 0x564cf2e1ba20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3582f1c8799437a5 server_cookie=82a3870bf1623d5e in_seq=0 out_seq=0 2024-05-08T23:54:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.882+0000 7fc071ffb700 10 client.17757 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:54:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc071ffb700 1 -- 172.21.15.12:0/2972674098 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cf2dca810 con 0x564cf2e19600 2024-05-08T23:54:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc071ffb700 10 client.17757 handle_command_reply: tid=0 2024-05-08T23:54:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 client.17757 shutdown 2024-05-08T23:54:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2972674098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 msgr2=0x564cf2e05060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e05060 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc0800ca660 tx=0x7fc080099fc0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2972674098 shutdown_connections 2024-05-08T23:54:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf2e19600 0x564cf2e1ba20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2e01720 0x564cf2e055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2dfb890 0x564cf2e04b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc071ffb700 0 client.17757 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:54:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/2972674098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e05060 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2972674098 >> 172.21.15.12:0/2972674098 conn(0x564cf2d59210 msgr2=0x564cf2df55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2972674098 shutdown_connections 2024-05-08T23:54:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2972674098 wait complete. 2024-05-08T23:54:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.883+0000 7fc09cfdcb80 20 client.17757 trim_cache size 0 max 16384 2024-05-08T23:54:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.897+0000 7fc09cfdcb80 1 Processor -- start 2024-05-08T23:54:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.897+0000 7fc09cfdcb80 1 -- start start 2024-05-08T23:54:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.897+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2e1b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.897+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e00e00 0x564cf2df7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.897+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e01720 0x564cf2df5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2df0800 con 0x564cf2e01720 2024-05-08T23:54:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2e02b10 con 0x564cf2dfb890 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2e16cb0 con 0x564cf2e00e00 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc08ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e01720 0x564cf2df5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc08ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e01720 0x564cf2df5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59192/0 (socket says 172.21.15.12:59192) 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc08ad9d700 1 -- 172.21.15.12:0/2338414566 learned_addr learned my addr 172.21.15.12:0/2338414566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc089d9b700 1 --2- 172.21.15.12:0/2338414566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e00e00 0x564cf2df7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc08a59c700 1 --2- 172.21.15.12:0/2338414566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2e1b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc08ad9d700 1 -- 172.21.15.12:0/2338414566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e00e00 msgr2=0x564cf2df7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc08ad9d700 1 --2- 172.21.15.12:0/2338414566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e00e00 0x564cf2df7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc08ad9d700 1 -- 172.21.15.12:0/2338414566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 msgr2=0x564cf2e1b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc08ad9d700 1 --2- 172.21.15.12:0/2338414566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2e1b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc08ad9d700 1 -- 172.21.15.12:0/2338414566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf2d11560 con 0x564cf2e01720 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc08ad9d700 1 --2- 172.21.15.12:0/2338414566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e01720 0x564cf2df5d20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc084009df0 tx=0x7fc0840076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a49bef4961ac11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc0797fa700 1 -- 172.21.15.12:0/2338414566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084007e30 con 0x564cf2e01720 2024-05-08T23:54:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc0797fa700 1 -- 172.21.15.12:0/2338414566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08400f040 con 0x564cf2e01720 2024-05-08T23:54:46.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.898+0000 7fc0797fa700 1 -- 172.21.15.12:0/2338414566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840167d0 con 0x564cf2e01720 2024-05-08T23:54:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2338414566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e01720 msgr2=0x564cf2df5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/2338414566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e01720 0x564cf2df5d20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc084009df0 tx=0x7fc0840076c0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2338414566 shutdown_connections 2024-05-08T23:54:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/2338414566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e01720 0x564cf2df5d20 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/2338414566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2e1b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/2338414566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e00e00 0x564cf2df7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2338414566 >> 172.21.15.12:0/2338414566 conn(0x564cf2d59210 msgr2=0x564cf2df2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2338414566 shutdown_connections 2024-05-08T23:54:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/2338414566 wait complete. 2024-05-08T23:54:46.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 -- start start 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.899+0000 7fc09cfdcb80 1 Processor -- start 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2e10900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e0e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc09cfdcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e01720 0x564cf2e0eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2e0f1d0 con 0x564cf2e00e00 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2e0f340 con 0x564cf2dfb890 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc09cfdcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf2e6eb00 con 0x564cf2e01720 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc09cfdcb80 10 client.0 _ll_register_callbacks cb 0x564cf2cf0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc089d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc08a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2e10900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc089d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e0e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59204/0 (socket says 172.21.15.12:59204) 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc089d9b700 1 -- 172.21.15.12:0/598614368 learned_addr learned my addr 172.21.15.12:0/598614368 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc08ad9d700 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e01720 0x564cf2e0eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc08a59c700 1 -- 172.21.15.12:0/598614368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e01720 msgr2=0x564cf2e0eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc08a59c700 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e01720 0x564cf2e0eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc08a59c700 1 -- 172.21.15.12:0/598614368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 msgr2=0x564cf2e0e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc08a59c700 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e0e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc08a59c700 1 -- 172.21.15.12:0/598614368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc084009a20 con 0x564cf2dfb890 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc089d9b700 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e0e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc08a59c700 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2e10900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0098e0 tx=0x7fc07c00be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a2fee3b78f7ec2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.900+0000 7fc071ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc071ffb700 1 -- 172.21.15.12:0/598614368 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c045070 con 0x564cf2dfb890 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/598614368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc07c01d040 con 0x564cf2dfb890 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc071ffb700 1 -- 172.21.15.12:0/598614368 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc07c0233f0 con 0x564cf2dfb890 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc071ffb700 1 -- 172.21.15.12:0/598614368 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc07c03c040 con 0x564cf2dfb890 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc09cfdcb80 10 client.27545 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/598614368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564cf2bd0cd0 con 0x564cf2dfb890 2024-05-08T23:54:46.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc071ffb700 1 -- 172.21.15.12:0/598614368 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c01d040 con 0x564cf2dfb890 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc071ffb700 1 -- 172.21.15.12:0/598614368 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc07c048030 con 0x564cf2dfb890 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc09cfdcb80 10 client.27545 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc09cfdcb80 10 client.27545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc09cfdcb80 20 client.27545 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf2e6fd60 0x564cf2e72150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc09cfdcb80 4 client.27545 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.901+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/598614368 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564cf2d43ec0 con 0x564cf2e6fd60 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc089d9b700 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf2e6fd60 0x564cf2e72150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc071ffb700 10 client.27545 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc089d9b700 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf2e6fd60 0x564cf2e72150 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f75bac006987c982 server_cookie=e9caab304a32cdd2 in_seq=0 out_seq=0 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc071ffb700 1 -- 172.21.15.12:0/598614368 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564cf2d43ec0 con 0x564cf2e6fd60 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc071ffb700 10 client.27545 handle_command_reply: tid=0 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 client.27545 shutdown 2024-05-08T23:54:46.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/598614368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 msgr2=0x564cf2e10900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2e10900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0098e0 tx=0x7fc07c00be90 comp rx=0 tx=0).stop 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/598614368 shutdown_connections 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x564cf2e6fd60 0x564cf2e72150 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cf2e01720 0x564cf2e0eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cf2dfb890 0x564cf2e10900 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 --2- 172.21.15.12:0/598614368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cf2e00e00 0x564cf2e0e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc071ffb700 0 client.27545 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/598614368 >> 172.21.15.12:0/598614368 conn(0x564cf2d59210 msgr2=0x564cf2e03340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/598614368 shutdown_connections 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.902+0000 7fc09cfdcb80 1 -- 172.21.15.12:0/598614368 wait complete. 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc09cfdcb80 20 client.27545 trim_cache size 0 max 16384 2024-05-08T23:54:46.903 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:54:46.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 -- 172.21.15.12:0/437787106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0740a1310 msgr2=0x7fc0740a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0740a1310 0x7fc0740a37d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009fd0 tx=0x7fc07c019040 comp rx=0 tx=0).stop 2024-05-08T23:54:46.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 -- 172.21.15.12:0/437787106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08c106290 msgr2=0x7fc08c197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08c106290 0x7fc08c197960 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc08000e970 tx=0x7fc08000ec80 comp rx=0 tx=0).stop 2024-05-08T23:54:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 -- 172.21.15.12:0/437787106 shutdown_connections 2024-05-08T23:54:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0740a1310 0x7fc0740a37d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc08c19bcb0 0x7fc08c19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc08c0682e0 0x7fc08c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 --2- 172.21.15.12:0/437787106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08c106290 0x7fc08c197960 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 -- 172.21.15.12:0/437787106 >> 172.21.15.12:0/437787106 conn(0x7fc08c076f40 msgr2=0x7fc08c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 -- 172.21.15.12:0/437787106 shutdown_connections 2024-05-08T23:54:46.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:46.903+0000 7fc090e22700 1 -- 172.21.15.12:0/437787106 wait complete. 2024-05-08T23:54:46.911 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:54:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.100+0000 7fe1998fb700 1 -- 172.21.15.12:0/3501478166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe194100a10 msgr2=0x7fe194104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.100+0000 7fe1998fb700 1 --2- 172.21.15.12:0/3501478166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe194100a10 0x7fe194104a50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe188009990 tx=0x7fe188009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.100+0000 7fe1998fb700 1 -- 172.21.15.12:0/3501478166 shutdown_connections 2024-05-08T23:54:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.100+0000 7fe1998fb700 1 --2- 172.21.15.12:0/3501478166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe194105180 0x7fe19410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.100+0000 7fe1998fb700 1 --2- 172.21.15.12:0/3501478166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe194100060 0x7fe194100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.100+0000 7fe1998fb700 1 --2- 172.21.15.12:0/3501478166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe194100a10 0x7fe194104a50 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.100+0000 7fe1998fb700 1 -- 172.21.15.12:0/3501478166 >> 172.21.15.12:0/3501478166 conn(0x7fe194076f40 msgr2=0x7fe194077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.100+0000 7fe1998fb700 1 -- 172.21.15.12:0/3501478166 shutdown_connections 2024-05-08T23:54:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.100+0000 7fe1998fb700 1 -- 172.21.15.12:0/3501478166 wait complete. 2024-05-08T23:54:47.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1998fb700 1 Processor -- start 2024-05-08T23:54:47.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1998fb700 1 -- start start 2024-05-08T23:54:47.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1998fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe194100060 0x7fe194197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1998fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe194105180 0x7fe1941979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1998fb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe19419bcf0 0x7fe19419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1998fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19419c770 con 0x7fe194105180 2024-05-08T23:54:47.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1998fb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19419c8e0 con 0x7fe194100060 2024-05-08T23:54:47.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1998fb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19419caa0 con 0x7fe19419bcf0 2024-05-08T23:54:47.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1927fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe194105180 0x7fe1941979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1927fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe194105180 0x7fe1941979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59212/0 (socket says 172.21.15.12:59212) 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1927fc700 1 -- 172.21.15.12:0/479427491 learned_addr learned my addr 172.21.15.12:0/479427491 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe192ffd700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe194100060 0x7fe194197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1937fe700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe19419bcf0 0x7fe19419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1927fc700 1 -- 172.21.15.12:0/479427491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe19419bcf0 msgr2=0x7fe19419c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1927fc700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe19419bcf0 0x7fe19419c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1927fc700 1 -- 172.21.15.12:0/479427491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe194100060 msgr2=0x7fe194197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1927fc700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe194100060 0x7fe194197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.101+0000 7fe1927fc700 1 -- 172.21.15.12:0/479427491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1880095f0 con 0x7fe194105180 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.102+0000 7fe1937fe700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe19419bcf0 0x7fe19419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.102+0000 7fe192ffd700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe194100060 0x7fe194197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.102+0000 7fe1927fc700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe194105180 0x7fe1941979a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe188005770 tx=0x7fe188009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.102+0000 7fe1988f9700 1 -- 172.21.15.12:0/479427491 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18801d070 con 0x7fe194105180 2024-05-08T23:54:47.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.102+0000 7fe1988f9700 1 -- 172.21.15.12:0/479427491 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe18800fbb0 con 0x7fe194105180 2024-05-08T23:54:47.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.102+0000 7fe1998fb700 1 -- 172.21.15.12:0/479427491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1941a3fa0 con 0x7fe194105180 2024-05-08T23:54:47.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.102+0000 7fe1988f9700 1 -- 172.21.15.12:0/479427491 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188022c00 con 0x7fe194105180 2024-05-08T23:54:47.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.102+0000 7fe1998fb700 1 -- 172.21.15.12:0/479427491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1941a43c0 con 0x7fe194105180 2024-05-08T23:54:47.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.105+0000 7fe1988f9700 1 -- 172.21.15.12:0/479427491 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe18800fd20 con 0x7fe194105180 2024-05-08T23:54:47.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.106+0000 7fe1988f9700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe17c0a1290 0x7fe17c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.106+0000 7fe1988f9700 1 -- 172.21.15.12:0/479427491 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe1880ce220 con 0x7fe194105180 2024-05-08T23:54:47.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.106+0000 7fe192ffd700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe17c0a1290 0x7fe17c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.106+0000 7fe1a5ab5b80 1 Processor -- start 2024-05-08T23:54:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.106+0000 7fe1a5ab5b80 1 -- start start 2024-05-08T23:54:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.106+0000 7fe192ffd700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe17c0a1290 0x7fe17c0a3750 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe184005950 tx=0x7fe1840058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7ee00 0x562dede7f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7f720 0x562dede79350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede79cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deddd0150 con 0x562dede79890 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deddcdb10 con 0x562dede7ee00 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562deddc9400 con 0x562dede7f720 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede79cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede79cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59214/0 (socket says 172.21.15.12:59214) 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1937fe700 1 -- 172.21.15.12:0/322317647 learned_addr learned my addr 172.21.15.12:0/322317647 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe192ffd700 1 --2- 172.21.15.12:0/322317647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7ee00 0x562dede7f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1927fc700 1 --2- 172.21.15.12:0/322317647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7f720 0x562dede79350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1937fe700 1 -- 172.21.15.12:0/322317647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7f720 msgr2=0x562dede79350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1937fe700 1 --2- 172.21.15.12:0/322317647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7f720 0x562dede79350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1937fe700 1 -- 172.21.15.12:0/322317647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7ee00 msgr2=0x562dede7f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1937fe700 1 --2- 172.21.15.12:0/322317647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7ee00 0x562dede7f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1937fe700 1 -- 172.21.15.12:0/322317647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dedc4a3b0 con 0x562dede79890 2024-05-08T23:54:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe192ffd700 1 --2- 172.21.15.12:0/322317647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7ee00 0x562dede7f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe1937fe700 1 --2- 172.21.15.12:0/322317647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede79cd0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009cd0 tx=0x7fe18c007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76cfaf1e9edbd01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.107+0000 7fe179ffb700 1 -- 172.21.15.12:0/322317647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c014400 con 0x562dede79890 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe179ffb700 1 -- 172.21.15.12:0/322317647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe18c014a20 con 0x562dede79890 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe179ffb700 1 -- 172.21.15.12:0/322317647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c013a00 con 0x562dede79890 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/322317647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 msgr2=0x562dede79cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/322317647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede79cd0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009cd0 tx=0x7fe18c007500 comp rx=0 tx=0).stop 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/322317647 shutdown_connections 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/322317647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede79cd0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/322317647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7ee00 0x562dede7f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/322317647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7f720 0x562dede79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/322317647 >> 172.21.15.12:0/322317647 conn(0x562deddd7210 msgr2=0x562dede73df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/322317647 shutdown_connections 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.108+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/322317647 wait complete. 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe1a5ab5b80 1 -- start start 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe1a5ab5b80 1 Processor -- start 2024-05-08T23:54:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede82bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7ee00 0x562dede83110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7f720 0x562dede83650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dedd65560 con 0x562dede79890 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dedd65200 con 0x562dede7f720 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dedc4c6a0 con 0x562dede7ee00 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe1a5ab5b80 10 client.0 _ll_register_callbacks cb 0x562dedd6eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe192ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede82bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe192ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede82bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59230/0 (socket says 172.21.15.12:59230) 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.109+0000 7fe192ffd700 1 -- 172.21.15.12:0/1573146397 learned_addr learned my addr 172.21.15.12:0/1573146397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe1937fe700 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7f720 0x562dede83650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe1927fc700 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7ee00 0x562dede83110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe192ffd700 1 -- 172.21.15.12:0/1573146397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7ee00 msgr2=0x562dede83110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe192ffd700 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7ee00 0x562dede83110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe192ffd700 1 -- 172.21.15.12:0/1573146397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7f720 msgr2=0x562dede83650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe192ffd700 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7f720 0x562dede83650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe192ffd700 1 -- 172.21.15.12:0/1573146397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18400be50 con 0x562dede79890 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe192ffd700 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede82bd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe1840042e0 tx=0x7fe184004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c6db3dcbaf36b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe172ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe172ffd700 1 -- 172.21.15.12:0/1573146397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184017040 con 0x562dede79890 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe172ffd700 1 -- 172.21.15.12:0/1573146397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe184003cd0 con 0x562dede79890 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1573146397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562dedd22270 con 0x562dede79890 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe172ffd700 1 -- 172.21.15.12:0/1573146397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe184003e90 con 0x562dede79890 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe1a5ab5b80 10 client.17775 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.110+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1573146397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fe18c009900 con 0x562dede79890 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.111+0000 7fe172ffd700 1 -- 172.21.15.12:0/1573146397 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562dedd22270 con 0x562dede79890 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.111+0000 7fe172ffd700 1 -- 172.21.15.12:0/1573146397 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe1840205d0 con 0x562dede79890 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.111+0000 7fe1a5ab5b80 10 client.17775 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.111+0000 7fe1a5ab5b80 10 client.17775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.111+0000 7fe1a5ab5b80 20 client.17775 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:47.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.111+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dede93270 0x562dede95690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.111+0000 7fe1a5ab5b80 4 client.17775 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:47.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.111+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1573146397 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562dede48810 con 0x562dede93270 2024-05-08T23:54:47.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.111+0000 7fe1927fc700 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dede93270 0x562dede95690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe172ffd700 10 client.17775 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:47.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe1927fc700 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dede93270 0x562dede95690 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3fff77663a19f85 server_cookie=f0966e2b05dfa259 in_seq=0 out_seq=0 2024-05-08T23:54:47.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe172ffd700 1 -- 172.21.15.12:0/1573146397 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562dede48810 con 0x562dede93270 2024-05-08T23:54:47.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe172ffd700 10 client.17775 handle_command_reply: tid=0 2024-05-08T23:54:47.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe1a5ab5b80 1 client.17775 shutdown 2024-05-08T23:54:47.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1573146397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 msgr2=0x562dede82bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:47.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede82bd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe1840042e0 tx=0x7fe184004750 comp rx=0 tx=0).stop 2024-05-08T23:54:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1573146397 shutdown_connections 2024-05-08T23:54:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dede93270 0x562dede95690 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede7f720 0x562dede83650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede79890 0x562dede82bd0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/1573146397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7ee00 0x562dede83110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe172ffd700 0 client.17775 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.112+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1573146397 >> 172.21.15.12:0/1573146397 conn(0x562deddd7210 msgr2=0x562dede735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.113+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1573146397 shutdown_connections 2024-05-08T23:54:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.113+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1573146397 wait complete. 2024-05-08T23:54:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.113+0000 7fe1a5ab5b80 20 client.17775 trim_cache size 0 max 16384 2024-05-08T23:54:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1a5ab5b80 1 Processor -- start 2024-05-08T23:54:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1a5ab5b80 1 -- start start 2024-05-08T23:54:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 0x562dede79350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7ee00 0x562dede75a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7f720 0x562dede73d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dede84180 con 0x562dede7ee00 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dede81140 con 0x562dede79890 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dede7f430 con 0x562dede7f720 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1927fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7ee00 0x562dede75a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1927fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7ee00 0x562dede75a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59240/0 (socket says 172.21.15.12:59240) 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1927fc700 1 -- 172.21.15.12:0/1356154654 learned_addr learned my addr 172.21.15.12:0/1356154654 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe192ffd700 1 --2- 172.21.15.12:0/1356154654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 0x562dede79350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1937fe700 1 --2- 172.21.15.12:0/1356154654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7f720 0x562dede73d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1927fc700 1 -- 172.21.15.12:0/1356154654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7f720 msgr2=0x562dede73d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1927fc700 1 --2- 172.21.15.12:0/1356154654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7f720 0x562dede73d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1927fc700 1 -- 172.21.15.12:0/1356154654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 msgr2=0x562dede79350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1927fc700 1 --2- 172.21.15.12:0/1356154654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 0x562dede79350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.127+0000 7fe1927fc700 1 -- 172.21.15.12:0/1356154654 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dedbf6b60 con 0x562dede7ee00 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.128+0000 7fe192ffd700 1 --2- 172.21.15.12:0/1356154654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 0x562dede79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.128+0000 7fe1927fc700 1 --2- 172.21.15.12:0/1356154654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7ee00 0x562dede75a00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe18809b0a0 tx=0x7fe188086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4322440ef0bc6326 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.128+0000 7fe17a7fc700 1 -- 172.21.15.12:0/1356154654 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18809baa0 con 0x562dede7ee00 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.128+0000 7fe17a7fc700 1 -- 172.21.15.12:0/1356154654 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe18809b3a0 con 0x562dede7ee00 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.128+0000 7fe17a7fc700 1 -- 172.21.15.12:0/1356154654 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18809b600 con 0x562dede7ee00 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.128+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1356154654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7ee00 msgr2=0x562dede75a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.128+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/1356154654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7ee00 0x562dede75a00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe18809b0a0 tx=0x7fe188086040 comp rx=0 tx=0).stop 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1356154654 shutdown_connections 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/1356154654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7f720 0x562dede73d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/1356154654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 0x562dede79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/1356154654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7ee00 0x562dede75a00 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1356154654 >> 172.21.15.12:0/1356154654 conn(0x562deddd7210 msgr2=0x562dede79c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1356154654 shutdown_connections 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/1356154654 wait complete. 2024-05-08T23:54:47.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 -- start start 2024-05-08T23:54:47.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 Processor -- start 2024-05-08T23:54:47.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 0x562dede8a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7ee00 0x562dede88150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7f720 0x562dede88890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dede88f60 con 0x562dede7f720 2024-05-08T23:54:47.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dedee8770 con 0x562dede79890 2024-05-08T23:54:47.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.129+0000 7fe1a5ab5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dedee88b0 con 0x562dede7ee00 2024-05-08T23:54:47.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1a5ab5b80 10 client.0 _ll_register_callbacks cb 0x562dedd6eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:47.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7f720 0x562dede88890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7f720 0x562dede88890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59248/0 (socket says 172.21.15.12:59248) 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1937fe700 1 -- 172.21.15.12:0/3754514119 learned_addr learned my addr 172.21.15.12:0/3754514119 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe192ffd700 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 0x562dede8a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1927fc700 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7ee00 0x562dede88150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1937fe700 1 -- 172.21.15.12:0/3754514119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7ee00 msgr2=0x562dede88150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1937fe700 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7ee00 0x562dede88150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1937fe700 1 -- 172.21.15.12:0/3754514119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 msgr2=0x562dede8a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1937fe700 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 0x562dede8a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1937fe700 1 -- 172.21.15.12:0/3754514119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1941a43c0 con 0x562dede7f720 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1937fe700 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7f720 0x562dede88890 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009cd0 tx=0x7fe18c004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c973444c3952b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe172ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe172ffd700 1 -- 172.21.15.12:0/3754514119 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0095e0 con 0x562dede7f720 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe172ffd700 1 -- 172.21.15.12:0/3754514119 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe18c0133a0 con 0x562dede7f720 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.130+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/3754514119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe18c014a20 con 0x562dede7f720 2024-05-08T23:54:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe172ffd700 1 -- 172.21.15.12:0/3754514119 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe18c014810 con 0x562dede7f720 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe1a5ab5b80 10 client.17781 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/3754514119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562dedd8f560 con 0x562dede7f720 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe172ffd700 1 -- 172.21.15.12:0/3754514119 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe18c014a20 con 0x562dede7f720 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe172ffd700 1 -- 172.21.15.12:0/3754514119 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe18c025070 con 0x562dede7f720 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe1a5ab5b80 10 client.17781 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe1a5ab5b80 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe1a5ab5b80 20 client.17781 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dedeea030 0x562dedeec420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe1a5ab5b80 4 client.17781 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/3754514119 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562deddc1ec0 con 0x562dedeea030 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.131+0000 7fe192ffd700 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dedeea030 0x562dedeec420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe192ffd700 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dedeea030 0x562dedeec420 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7972d630df024d6 server_cookie=6c018bb282f07021 in_seq=0 out_seq=0 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe172ffd700 10 client.17781 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:47.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe172ffd700 1 -- 172.21.15.12:0/3754514119 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562deddc1ec0 con 0x562dedeea030 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe172ffd700 10 client.17781 handle_command_reply: tid=0 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 client.17781 shutdown 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/3754514119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7f720 msgr2=0x562dede88890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7f720 0x562dede88890 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009cd0 tx=0x7fe18c004ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/3754514119 shutdown_connections 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dede7f720 0x562dede88890 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dedeea030 0x562dedeec420 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dede79890 0x562dede8a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 --2- 172.21.15.12:0/3754514119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dede7ee00 0x562dede88150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe172ffd700 0 client.17781 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/3754514119 >> 172.21.15.12:0/3754514119 conn(0x562deddd7210 msgr2=0x562dede81480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/3754514119 shutdown_connections 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 1 -- 172.21.15.12:0/3754514119 wait complete. 2024-05-08T23:54:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.132+0000 7fe1a5ab5b80 20 client.17781 trim_cache size 0 max 16384 2024-05-08T23:54:47.134 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:54:47.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 -- 172.21.15.12:0/479427491 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe17c0a1290 msgr2=0x7fe17c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:47.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe17c0a1290 0x7fe17c0a3750 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe184005950 tx=0x7fe1840058e0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 -- 172.21.15.12:0/479427491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe194105180 msgr2=0x7fe1941979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe194105180 0x7fe1941979a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe188005770 tx=0x7fe188009580 comp rx=0 tx=0).stop 2024-05-08T23:54:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 -- 172.21.15.12:0/479427491 shutdown_connections 2024-05-08T23:54:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe17c0a1290 0x7fe17c0a3750 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe19419bcf0 0x7fe19419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe194100060 0x7fe194197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 --2- 172.21.15.12:0/479427491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe194105180 0x7fe1941979a0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 -- 172.21.15.12:0/479427491 >> 172.21.15.12:0/479427491 conn(0x7fe194076f40 msgr2=0x7fe1940698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 -- 172.21.15.12:0/479427491 shutdown_connections 2024-05-08T23:54:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:47.133+0000 7fe1998fb700 1 -- 172.21.15.12:0/479427491 wait complete. 2024-05-08T23:54:47.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:47 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2407375388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:54:47.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:47 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2407375388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:54:47.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499095 7.73 MB/sec execute 283 sec latency 1825.152 ms 2024-05-08T23:54:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:47 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2407375388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:54:48.142 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:54:48.143 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:54:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.299+0000 7f2f81b72700 1 -- 172.21.15.12:0/2027972665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c101b90 msgr2=0x7f2f7c105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.299+0000 7f2f81b72700 1 --2- 172.21.15.12:0/2027972665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c101b90 0x7f2f7c105bd0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009990 tx=0x7f2f64009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.300+0000 7f2f81b72700 1 -- 172.21.15.12:0/2027972665 shutdown_connections 2024-05-08T23:54:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.300+0000 7f2f81b72700 1 --2- 172.21.15.12:0/2027972665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f7c106300 0x7f2f7c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.300+0000 7f2f81b72700 1 --2- 172.21.15.12:0/2027972665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7c1011e0 0x7f2f7c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.300+0000 7f2f81b72700 1 --2- 172.21.15.12:0/2027972665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c101b90 0x7f2f7c105bd0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.300+0000 7f2f81b72700 1 -- 172.21.15.12:0/2027972665 >> 172.21.15.12:0/2027972665 conn(0x7f2f7c0fca80 msgr2=0x7f2f7c0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.300+0000 7f2f81b72700 1 -- 172.21.15.12:0/2027972665 shutdown_connections 2024-05-08T23:54:48.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.300+0000 7f2f81b72700 1 -- 172.21.15.12:0/2027972665 wait complete. 2024-05-08T23:54:48.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.300+0000 7f2f81b72700 1 Processor -- start 2024-05-08T23:54:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f81b72700 1 -- start start 2024-05-08T23:54:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f81b72700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c1011e0 0x7f2f7c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f81b72700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f7c106300 0x7f2f7c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f81b72700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7c073d20 0x7f2f7c1a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f81b72700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c1a5d20 con 0x7f2f7c1011e0 2024-05-08T23:54:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f81b72700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c1a5e60 con 0x7f2f7c073d20 2024-05-08T23:54:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f81b72700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c1a6020 con 0x7f2f7c106300 2024-05-08T23:54:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c1011e0 0x7f2f7c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c1011e0 0x7f2f7c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59268/0 (socket says 172.21.15.12:59268) 2024-05-08T23:54:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/3020249470 learned_addr learned my addr 172.21.15.12:0/3020249470 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7bfff700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7c073d20 0x7f2f7c1a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7affd700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f7c106300 0x7f2f7c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/3020249470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f7c106300 msgr2=0x7f2f7c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f7c106300 0x7f2f7c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/3020249470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7c073d20 msgr2=0x7f2f7c1a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7c073d20 0x7f2f7c1a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/3020249470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f640095f0 con 0x7f2f7c1011e0 2024-05-08T23:54:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7affd700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f7c106300 0x7f2f7c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.301+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c1011e0 0x7f2f7c0732a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e9a0 tx=0x7f2f6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.302+0000 7f2f78ff9700 1 -- 172.21.15.12:0/3020249470 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c009790 con 0x7f2f7c1011e0 2024-05-08T23:54:48.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.302+0000 7f2f78ff9700 1 -- 172.21.15.12:0/3020249470 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f6c00cbd0 con 0x7f2f7c1011e0 2024-05-08T23:54:48.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.302+0000 7f2f81b72700 1 -- 172.21.15.12:0/3020249470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c1a6300 con 0x7f2f7c1011e0 2024-05-08T23:54:48.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.302+0000 7f2f78ff9700 1 -- 172.21.15.12:0/3020249470 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c018680 con 0x7f2f7c1011e0 2024-05-08T23:54:48.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.302+0000 7f2f81b72700 1 -- 172.21.15.12:0/3020249470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c1a6810 con 0x7f2f7c1011e0 2024-05-08T23:54:48.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.304+0000 7f2f78ff9700 1 -- 172.21.15.12:0/3020249470 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2f6c00cd40 con 0x7f2f7c1011e0 2024-05-08T23:54:48.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.305+0000 7f2f78ff9700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2f680a1290 0x7f2f680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.305+0000 7f2f78ff9700 1 -- 172.21.15.12:0/3020249470 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2f6c014070 con 0x7f2f7c1011e0 2024-05-08T23:54:48.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.305+0000 7f2f7affd700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2f680a1290 0x7f2f680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.305+0000 7f2f7affd700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2f680a1290 0x7f2f680a3750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400b2e0 tx=0x7f2f64005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f8dd2cb80 1 Processor -- start 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f8dd2cb80 1 -- start start 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2c07a0 0x56172d2ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 0x56172d2bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d2111d0 con 0x56172d2bfe80 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d20eb90 con 0x56172d2ba910 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d20a480 con 0x56172d2c07a0 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f7b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f7b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59276/0 (socket says 172.21.15.12:59276) 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.306+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/2807708486 learned_addr learned my addr 172.21.15.12:0/2807708486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f7bfff700 1 --2- 172.21.15.12:0/2807708486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 0x56172d2bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f7affd700 1 --2- 172.21.15.12:0/2807708486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2c07a0 0x56172d2ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/2807708486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2c07a0 msgr2=0x56172d2ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/2807708486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2c07a0 0x56172d2ba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/2807708486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 msgr2=0x56172d2bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/2807708486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 0x56172d2bad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/2807708486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56172d08b430 con 0x56172d2bfe80 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/2807708486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c0260 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0a1fc0 tx=0x7f2f6c099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4262b623b3be1b74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f71ffb700 1 -- 172.21.15.12:0/2807708486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c09dea0 con 0x56172d2bfe80 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f71ffb700 1 -- 172.21.15.12:0/2807708486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f6c0a24f0 con 0x56172d2bfe80 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.307+0000 7f2f71ffb700 1 -- 172.21.15.12:0/2807708486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c099760 con 0x56172d2bfe80 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.308+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/2807708486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 msgr2=0x56172d2c0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.308+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/2807708486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c0260 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0a1fc0 tx=0x7f2f6c099a50 comp rx=0 tx=0).stop 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.308+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/2807708486 shutdown_connections 2024-05-08T23:54:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.308+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/2807708486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 0x56172d2bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.308+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/2807708486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c0260 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.308+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/2807708486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2c07a0 0x56172d2ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.308+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/2807708486 >> 172.21.15.12:0/2807708486 conn(0x56172d218290 msgr2=0x56172d2b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.308+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/2807708486 shutdown_connections 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.308+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/2807708486 wait complete. 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.308+0000 7f2f8dd2cb80 1 -- start start 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f8dd2cb80 1 Processor -- start 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2ba910 0x56172d2c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2c07a0 0x56172d2c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d1a65e0 con 0x56172d2bfe80 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d1a6280 con 0x56172d2c07a0 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d08d720 con 0x56172d2ba910 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f8dd2cb80 10 client.0 _ll_register_callbacks cb 0x56172d1afbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f7affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f7affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59282/0 (socket says 172.21.15.12:59282) 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f7affd700 1 -- 172.21.15.12:0/1846146829 learned_addr learned my addr 172.21.15.12:0/1846146829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f7bfff700 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2c07a0 0x56172d2c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2ba910 0x56172d2c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f7affd700 1 -- 172.21.15.12:0/1846146829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2ba910 msgr2=0x56172d2c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f7affd700 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2ba910 0x56172d2c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f7affd700 1 -- 172.21.15.12:0/1846146829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2c07a0 msgr2=0x56172d2c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f7affd700 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2c07a0 0x56172d2c45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.309+0000 7f2f7affd700 1 -- 172.21.15.12:0/1846146829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c1a6810 con 0x56172d2bfe80 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.310+0000 7f2f7affd700 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c40a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c06a780 tx=0x7f2f64009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24129f7474fb5d7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.310+0000 7f2f5affd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.310+0000 7f2f5affd700 1 -- 172.21.15.12:0/1846146829 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64030630 con 0x56172d2bfe80 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.310+0000 7f2f5affd700 1 -- 172.21.15.12:0/1846146829 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2f64030790 con 0x56172d2bfe80 2024-05-08T23:54:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.310+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1846146829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56172d1632f0 con 0x56172d2bfe80 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.310+0000 7f2f5affd700 1 -- 172.21.15.12:0/1846146829 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f64030950 con 0x56172d2bfe80 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.310+0000 7f2f8dd2cb80 10 client.17793 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.310+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1846146829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56172d037be0 con 0x56172d2bfe80 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.310+0000 7f2f5affd700 1 -- 172.21.15.12:0/1846146829 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f6403e020 con 0x56172d2bfe80 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.310+0000 7f2f5affd700 1 -- 172.21.15.12:0/1846146829 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2f6403ab90 con 0x56172d2bfe80 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.311+0000 7f2f8dd2cb80 10 client.17793 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.311+0000 7f2f8dd2cb80 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.311+0000 7f2f8dd2cb80 20 client.17793 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.311+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172d2d4350 0x56172d2d6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.311+0000 7f2f8dd2cb80 4 client.17793 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.311+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1846146829 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56172d289890 con 0x56172d2d4350 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.311+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172d2d4350 0x56172d2d6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.311+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172d2d4350 0x56172d2d6770 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05632b194dfc7d server_cookie=e72c2d3cedef09e5 in_seq=0 out_seq=0 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.311+0000 7f2f5affd700 10 client.17793 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f5affd700 1 -- 172.21.15.12:0/1846146829 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56172d289890 con 0x56172d2d4350 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f5affd700 10 client.17793 handle_command_reply: tid=0 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 client.17793 shutdown 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1846146829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 msgr2=0x56172d2c40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c40a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c06a780 tx=0x7f2f64009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1846146829 shutdown_connections 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172d2d4350 0x56172d2d6770 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2c07a0 0x56172d2c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2ba910 0x56172d2c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f5affd700 0 client.17793 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1846146829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2bfe80 0x56172d2c40a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1846146829 >> 172.21.15.12:0/1846146829 conn(0x56172d218290 msgr2=0x56172d2b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1846146829 shutdown_connections 2024-05-08T23:54:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1846146829 wait complete. 2024-05-08T23:54:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.312+0000 7f2f8dd2cb80 20 client.17793 trim_cache size 0 max 16384 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f8dd2cb80 1 Processor -- start 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f8dd2cb80 1 -- start start 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 0x56172d2ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 0x56172d2b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2c07a0 0x56172d2bf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d2d1960 con 0x56172d2c07a0 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d2c66d0 con 0x56172d2ba910 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d2b1e70 con 0x56172d2bfe80 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f7bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2c07a0 0x56172d2bf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f7bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2c07a0 0x56172d2bf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59298/0 (socket says 172.21.15.12:59298) 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f7bfff700 1 -- 172.21.15.12:0/2607465439 learned_addr learned my addr 172.21.15.12:0/2607465439 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/2607465439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 0x56172d2ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f7affd700 1 --2- 172.21.15.12:0/2607465439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 0x56172d2b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f7bfff700 1 -- 172.21.15.12:0/2607465439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 msgr2=0x56172d2b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f7bfff700 1 --2- 172.21.15.12:0/2607465439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 0x56172d2b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f7bfff700 1 -- 172.21.15.12:0/2607465439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 msgr2=0x56172d2ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f7bfff700 1 --2- 172.21.15.12:0/2607465439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 0x56172d2ba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.331+0000 7f2f7bfff700 1 -- 172.21.15.12:0/2607465439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56172d2b1560 con 0x56172d2c07a0 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.332+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/2607465439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 0x56172d2ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.332+0000 7f2f7bfff700 1 --2- 172.21.15.12:0/2607465439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2c07a0 0x56172d2bf360 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f2f74012900 tx=0x7f2f74009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3e7b79b7418d21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.332+0000 7f2f727fc700 1 -- 172.21.15.12:0/2607465439 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400e040 con 0x56172d2c07a0 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.332+0000 7f2f727fc700 1 -- 172.21.15.12:0/2607465439 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f7400f040 con 0x56172d2c07a0 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.332+0000 7f2f727fc700 1 -- 172.21.15.12:0/2607465439 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74007a60 con 0x56172d2c07a0 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/2607465439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2c07a0 msgr2=0x56172d2bf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/2607465439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2c07a0 0x56172d2bf360 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f2f74012900 tx=0x7f2f74009920 comp rx=0 tx=0).stop 2024-05-08T23:54:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/2607465439 shutdown_connections 2024-05-08T23:54:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/2607465439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2c07a0 0x56172d2bf360 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/2607465439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2ba910 0x56172d2ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/2607465439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 0x56172d2b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/2607465439 >> 172.21.15.12:0/2607465439 conn(0x56172d218290 msgr2=0x56172d2bec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/2607465439 shutdown_connections 2024-05-08T23:54:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/2607465439 wait complete. 2024-05-08T23:54:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 -- start start 2024-05-08T23:54:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.333+0000 7f2f8dd2cb80 1 Processor -- start 2024-05-08T23:54:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2ba910 0x56172d2bc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 0x56172d2bcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f8dd2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2c07a0 0x56172d2bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d32a890 con 0x56172d2ba910 2024-05-08T23:54:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d32aa00 con 0x56172d2c07a0 2024-05-08T23:54:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f8dd2cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172d32abc0 con 0x56172d2bfe80 2024-05-08T23:54:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f8dd2cb80 10 client.0 _ll_register_callbacks cb 0x56172d1afbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2ba910 0x56172d2bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2ba910 0x56172d2bc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59302/0 (socket says 172.21.15.12:59302) 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/1319364198 learned_addr learned my addr 172.21.15.12:0/1319364198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7bfff700 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2c07a0 0x56172d2bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7affd700 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 0x56172d2bcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/1319364198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 msgr2=0x56172d2bcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 0x56172d2bcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/1319364198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2c07a0 msgr2=0x56172d2bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2c07a0 0x56172d2bd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7b7fe700 1 -- 172.21.15.12:0/1319364198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f740125e0 con 0x56172d2ba910 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7affd700 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 0x56172d2bcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f7b7fe700 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2ba910 0x56172d2bc610 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0ca760 tx=0x7f2f6c0ca6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e376b4527e9a12ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.334+0000 7f2f5affd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f5affd700 1 -- 172.21.15.12:0/1319364198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c099760 con 0x56172d2ba910 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f5affd700 1 -- 172.21.15.12:0/1319364198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2f6c09a2d0 con 0x56172d2ba910 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1319364198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f6c0a24f0 con 0x56172d2ba910 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f5affd700 1 -- 172.21.15.12:0/1319364198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f6c0963b0 con 0x56172d2ba910 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f8dd2cb80 10 client.17799 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1319364198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56172d2bafc0 con 0x56172d2ba910 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f5affd700 1 -- 172.21.15.12:0/1319364198 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f6c0a24f0 con 0x56172d2ba910 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f5affd700 1 -- 172.21.15.12:0/1319364198 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2f6c0a2060 con 0x56172d2ba910 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f8dd2cb80 10 client.17799 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f8dd2cb80 10 client.17799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f8dd2cb80 20 client.17799 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172d32c0f0 0x56172d32e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f8dd2cb80 4 client.17799 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.335+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1319364198 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56172d202f40 con 0x56172d32c0f0 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.336+0000 7f2f7affd700 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172d32c0f0 0x56172d32e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.336+0000 7f2f5affd700 10 client.17799 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.336+0000 7f2f7affd700 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172d32c0f0 0x56172d32e510 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f2c5784739be47e server_cookie=a8daeef8d796c35 in_seq=0 out_seq=0 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f5affd700 1 -- 172.21.15.12:0/1319364198 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56172d202f40 con 0x56172d32c0f0 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f5affd700 10 client.17799 handle_command_reply: tid=0 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 client.17799 shutdown 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1319364198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2ba910 msgr2=0x56172d2bc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2ba910 0x56172d2bc610 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0ca760 tx=0x7f2f6c0ca6a0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1319364198 shutdown_connections 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56172d32c0f0 0x56172d32e510 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56172d2c07a0 0x56172d2bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56172d2ba910 0x56172d2bc610 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 --2- 172.21.15.12:0/1319364198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56172d2bfe80 0x56172d2bcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f5affd700 0 client.17799 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1319364198 >> 172.21.15.12:0/1319364198 conn(0x56172d218290 msgr2=0x56172d2bf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1319364198 shutdown_connections 2024-05-08T23:54:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.337+0000 7f2f8dd2cb80 1 -- 172.21.15.12:0/1319364198 wait complete. 2024-05-08T23:54:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f8dd2cb80 20 client.17799 trim_cache size 0 max 16384 2024-05-08T23:54:48.338 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:54:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:54:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "756675f6-95f7-4560-98b6-eb4ad2ace9d5", 2024-05-08T23:54:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:54:48.338 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:54:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 -- 172.21.15.12:0/3020249470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2f680a1290 msgr2=0x7f2f680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2f680a1290 0x7f2f680a3750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400b2e0 tx=0x7f2f64005fb0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 -- 172.21.15.12:0/3020249470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c1011e0 msgr2=0x7f2f7c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c1011e0 0x7f2f7c0732a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e9a0 tx=0x7f2f6c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 -- 172.21.15.12:0/3020249470 shutdown_connections 2024-05-08T23:54:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2f680a1290 0x7f2f680a3750 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7c073d20 0x7f2f7c1a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c1011e0 0x7f2f7c0732a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 --2- 172.21.15.12:0/3020249470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f7c106300 0x7f2f7c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 -- 172.21.15.12:0/3020249470 >> 172.21.15.12:0/3020249470 conn(0x7f2f7c0fca80 msgr2=0x7f2f7c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 -- 172.21.15.12:0/3020249470 shutdown_connections 2024-05-08T23:54:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.338+0000 7f2f81b72700 1 -- 172.21.15.12:0/3020249470 wait complete. 2024-05-08T23:54:48.347 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:54:48.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 -- 172.21.15.12:0/428268535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff20100430 msgr2=0x7eff20104220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 --2- 172.21.15.12:0/428268535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff20100430 0x7eff20104220 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7eff100099c0 tx=0x7eff10009cd0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 -- 172.21.15.12:0/428268535 shutdown_connections 2024-05-08T23:54:48.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 --2- 172.21.15.12:0/428268535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff20104950 0x7eff2010c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 --2- 172.21.15.12:0/428268535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff200ffa80 0x7eff200ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 --2- 172.21.15.12:0/428268535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff20100430 0x7eff20104220 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 -- 172.21.15.12:0/428268535 >> 172.21.15.12:0/428268535 conn(0x7eff20076110 msgr2=0x7eff20076520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 -- 172.21.15.12:0/428268535 shutdown_connections 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 -- 172.21.15.12:0/428268535 wait complete. 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 Processor -- start 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.494+0000 7eff27976700 1 -- start start 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff27976700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff200ffa80 0x7eff20197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff27976700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff20104950 0x7eff201978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff27976700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2019bc60 0x7eff2019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff27976700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2019c6e0 con 0x7eff2019bc60 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff27976700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2019c850 con 0x7eff200ffa80 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff27976700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2019ca10 con 0x7eff20104950 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25f13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2019bc60 0x7eff2019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25f13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2019bc60 0x7eff2019c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56308/0 (socket says 172.21.15.12:56308) 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25f13700 1 -- 172.21.15.12:0/3953598862 learned_addr learned my addr 172.21.15.12:0/3953598862 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25712700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff200ffa80 0x7eff20197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff24f11700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff20104950 0x7eff201978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25f13700 1 -- 172.21.15.12:0/3953598862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff20104950 msgr2=0x7eff201978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25f13700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff20104950 0x7eff201978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25f13700 1 -- 172.21.15.12:0/3953598862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff200ffa80 msgr2=0x7eff20197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25f13700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff200ffa80 0x7eff20197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25f13700 1 -- 172.21.15.12:0/3953598862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff100095f0 con 0x7eff2019bc60 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25712700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff200ffa80 0x7eff20197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.495+0000 7eff25f13700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2019bc60 0x7eff2019c0e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7eff1800ef30 tx=0x7eff1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.496+0000 7eff167fc700 1 -- 172.21.15.12:0/3953598862 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18018070 con 0x7eff2019bc60 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.496+0000 7eff167fc700 1 -- 172.21.15.12:0/3953598862 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff1800f040 con 0x7eff2019bc60 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.496+0000 7eff27976700 1 -- 172.21.15.12:0/3953598862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff201a3f10 con 0x7eff2019bc60 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.496+0000 7eff167fc700 1 -- 172.21.15.12:0/3953598862 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1800cdb0 con 0x7eff2019bc60 2024-05-08T23:54:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.496+0000 7eff27976700 1 -- 172.21.15.12:0/3953598862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff201a4340 con 0x7eff2019bc60 2024-05-08T23:54:48.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.498+0000 7eff167fc700 1 -- 172.21.15.12:0/3953598862 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7eff18003a00 con 0x7eff2019bc60 2024-05-08T23:54:48.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.499+0000 7eff167fc700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7eff0c0a1290 0x7eff0c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.499+0000 7eff167fc700 1 -- 172.21.15.12:0/3953598862 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7eff180cc890 con 0x7eff2019bc60 2024-05-08T23:54:48.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.499+0000 7eff25712700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7eff0c0a1290 0x7eff0c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.499+0000 7eff25712700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7eff0c0a1290 0x7eff0c0a3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff1c005950 tx=0x7eff1c00b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff33b30b80 1 Processor -- start 2024-05-08T23:54:48.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff33b30b80 1 -- start start 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060baede00 0x56060baee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060bae8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060bae8890 0x56060bae8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff33b30b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060ba3f150 con 0x56060baee720 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff33b30b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060ba3cb10 con 0x56060baede00 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff33b30b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060ba38400 con 0x56060bae8890 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff24f11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060bae8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff24f11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060bae8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56320/0 (socket says 172.21.15.12:56320) 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff24f11700 1 -- 172.21.15.12:0/2165711238 learned_addr learned my addr 172.21.15.12:0/2165711238 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff25712700 1 --2- 172.21.15.12:0/2165711238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060baede00 0x56060baee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.500+0000 7eff25f13700 1 --2- 172.21.15.12:0/2165711238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060bae8890 0x56060bae8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.501+0000 7eff24f11700 1 -- 172.21.15.12:0/2165711238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060bae8890 msgr2=0x56060bae8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.501+0000 7eff24f11700 1 --2- 172.21.15.12:0/2165711238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060bae8890 0x56060bae8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.501+0000 7eff24f11700 1 -- 172.21.15.12:0/2165711238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060baede00 msgr2=0x56060baee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.501+0000 7eff24f11700 1 --2- 172.21.15.12:0/2165711238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060baede00 0x56060baee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.501+0000 7eff24f11700 1 -- 172.21.15.12:0/2165711238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56060b8b93b0 con 0x56060baee720 2024-05-08T23:54:48.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.501+0000 7eff24f11700 1 --2- 172.21.15.12:0/2165711238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060bae8350 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7eff20073690 tx=0x7eff10005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d82b319c9a8994 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.501+0000 7eff0b7fe700 1 -- 172.21.15.12:0/2165711238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1001d040 con 0x56060baee720 2024-05-08T23:54:48.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.501+0000 7eff0b7fe700 1 -- 172.21.15.12:0/2165711238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff10003720 con 0x56060baee720 2024-05-08T23:54:48.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.501+0000 7eff0b7fe700 1 -- 172.21.15.12:0/2165711238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1000bab0 con 0x56060baee720 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 -- 172.21.15.12:0/2165711238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 msgr2=0x56060bae8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 --2- 172.21.15.12:0/2165711238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060bae8350 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7eff20073690 tx=0x7eff10005f00 comp rx=0 tx=0).stop 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 -- 172.21.15.12:0/2165711238 shutdown_connections 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 --2- 172.21.15.12:0/2165711238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060bae8890 0x56060bae8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 --2- 172.21.15.12:0/2165711238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060baede00 0x56060baee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 --2- 172.21.15.12:0/2165711238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060bae8350 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 -- 172.21.15.12:0/2165711238 >> 172.21.15.12:0/2165711238 conn(0x56060ba46210 msgr2=0x56060bae2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 -- 172.21.15.12:0/2165711238 shutdown_connections 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 -- 172.21.15.12:0/2165711238 wait complete. 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 -- start start 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.502+0000 7eff33b30b80 1 Processor -- start 2024-05-08T23:54:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 0x56060baf1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baede00 0x56060baf20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff33b30b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b9d4560 con 0x56060baee720 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff33b30b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b9d4200 con 0x56060bae8890 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff33b30b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b8bb6a0 con 0x56060baede00 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff33b30b80 10 client.0 _ll_register_callbacks cb 0x56060b9ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff25f13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff25f13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56330/0 (socket says 172.21.15.12:56330) 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff25f13700 1 -- 172.21.15.12:0/412144596 learned_addr learned my addr 172.21.15.12:0/412144596 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff24f11700 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baede00 0x56060baf20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff25712700 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 0x56060baf1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff25f13700 1 -- 172.21.15.12:0/412144596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baede00 msgr2=0x56060baf20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff25f13700 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baede00 0x56060baf20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff25f13700 1 -- 172.21.15.12:0/412144596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 msgr2=0x56060baf1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff25f13700 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 0x56060baf1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff25f13700 1 -- 172.21.15.12:0/412144596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2010cce0 con 0x56060baee720 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.503+0000 7eff25712700 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 0x56060baf1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff25f13700 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf2630 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7eff18099a00 tx=0x7eff18099450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b611c1a9b765bbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff08ff9700 1 -- 172.21.15.12:0/412144596 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1809a170 con 0x56060baee720 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff08ff9700 1 -- 172.21.15.12:0/412144596 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7eff18099a70 con 0x56060baee720 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff33b30b80 1 -- 172.21.15.12:0/412144596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56060b991270 con 0x56060baee720 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff08ff9700 1 -- 172.21.15.12:0/412144596 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff1809d8a0 con 0x56060baee720 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff33b30b80 10 client.17811 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff33b30b80 1 -- 172.21.15.12:0/412144596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56060b865b60 con 0x56060baee720 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff08ff9700 1 -- 172.21.15.12:0/412144596 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff180b6080 con 0x56060baee720 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff08ff9700 1 -- 172.21.15.12:0/412144596 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7eff180a19d0 con 0x56060baee720 2024-05-08T23:54:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff33b30b80 10 client.17811 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff33b30b80 10 client.17811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff33b30b80 20 client.17811 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff33b30b80 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56060bb02260 0x56060bb04680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff33b30b80 4 client.17811 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.504+0000 7eff33b30b80 1 -- 172.21.15.12:0/412144596 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56060bab7810 con 0x56060bb02260 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.505+0000 7eff25712700 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56060bb02260 0x56060bb04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.505+0000 7eff25712700 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56060bb02260 0x56060bb04680 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0c0fb9138a8f9f server_cookie=392bf1730342622f in_seq=0 out_seq=0 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.505+0000 7eff08ff9700 10 client.17811 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff08ff9700 1 -- 172.21.15.12:0/412144596 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56060bab7810 con 0x56060bb02260 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff08ff9700 10 client.17811 handle_command_reply: tid=0 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 client.17811 shutdown 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 -- 172.21.15.12:0/412144596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 msgr2=0x56060baf2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf2630 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7eff18099a00 tx=0x7eff18099450 comp rx=0 tx=0).stop 2024-05-08T23:54:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 -- 172.21.15.12:0/412144596 shutdown_connections 2024-05-08T23:54:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf2630 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56060bb02260 0x56060bb04680 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 0x56060baf1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 --2- 172.21.15.12:0/412144596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baede00 0x56060baf20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff08ff9700 0 client.17811 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 -- 172.21.15.12:0/412144596 >> 172.21.15.12:0/412144596 conn(0x56060ba46210 msgr2=0x56060bae25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 -- 172.21.15.12:0/412144596 shutdown_connections 2024-05-08T23:54:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 1 -- 172.21.15.12:0/412144596 wait complete. 2024-05-08T23:54:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.506+0000 7eff33b30b80 20 client.17811 trim_cache size 0 max 16384 2024-05-08T23:54:48.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.525+0000 7eff33b30b80 1 Processor -- start 2024-05-08T23:54:48.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.525+0000 7eff33b30b80 1 -- start start 2024-05-08T23:54:48.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 0x56060bae2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baede00 0x56060baf2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baee720 0x56060baecce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff33b30b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060baff930 con 0x56060baede00 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff33b30b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060bae3e80 con 0x56060bae8890 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff33b30b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060badfdc0 con 0x56060baee720 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff24f11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baede00 0x56060baf2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff24f11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baede00 0x56060baf2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56338/0 (socket says 172.21.15.12:56338) 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff24f11700 1 -- 172.21.15.12:0/920524489 learned_addr learned my addr 172.21.15.12:0/920524489 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff25712700 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 0x56060bae2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff25f13700 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baee720 0x56060baecce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff24f11700 1 -- 172.21.15.12:0/920524489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baee720 msgr2=0x56060baecce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff24f11700 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baee720 0x56060baecce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff24f11700 1 -- 172.21.15.12:0/920524489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 msgr2=0x56060bae2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff24f11700 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 0x56060bae2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff24f11700 1 -- 172.21.15.12:0/920524489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56060b9fe560 con 0x56060baede00 2024-05-08T23:54:48.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff25712700 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 0x56060bae2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:48.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff25f13700 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baee720 0x56060baecce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:48.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.526+0000 7eff24f11700 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baede00 0x56060baf2450 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x56060baeeb80 tx=0x7eff10005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e073d2cf265e9534 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff0bfff700 1 -- 172.21.15.12:0/920524489 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1001d040 con 0x56060baede00 2024-05-08T23:54:48.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff0bfff700 1 -- 172.21.15.12:0/920524489 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff10003d50 con 0x56060baede00 2024-05-08T23:54:48.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff0bfff700 1 -- 172.21.15.12:0/920524489 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1000bc90 con 0x56060baede00 2024-05-08T23:54:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff33b30b80 1 -- 172.21.15.12:0/920524489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baede00 msgr2=0x56060baf2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff33b30b80 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baede00 0x56060baf2450 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x56060baeeb80 tx=0x7eff10005040 comp rx=0 tx=0).stop 2024-05-08T23:54:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff33b30b80 1 -- 172.21.15.12:0/920524489 shutdown_connections 2024-05-08T23:54:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff33b30b80 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060baee720 0x56060baecce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff33b30b80 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060bae8890 0x56060bae2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff33b30b80 1 --2- 172.21.15.12:0/920524489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baede00 0x56060baf2450 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff33b30b80 1 -- 172.21.15.12:0/920524489 >> 172.21.15.12:0/920524489 conn(0x56060ba46210 msgr2=0x56060baf04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.527+0000 7eff33b30b80 1 -- 172.21.15.12:0/920524489 shutdown_connections 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.528+0000 7eff33b30b80 1 -- 172.21.15.12:0/920524489 wait complete. 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.528+0000 7eff33b30b80 1 -- start start 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.528+0000 7eff33b30b80 1 Processor -- start 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.528+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060bae8890 0x56060baf70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.528+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060baede00 0x56060baf7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.528+0000 7eff33b30b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.528+0000 7eff33b30b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060bb577f0 con 0x56060baee720 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.528+0000 7eff33b30b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060bb57960 con 0x56060baede00 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.528+0000 7eff33b30b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060bb57b20 con 0x56060bae8890 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.528+0000 7eff33b30b80 10 client.0 _ll_register_callbacks cb 0x56060b9ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff25f13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff25f13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56350/0 (socket says 172.21.15.12:56350) 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff25f13700 1 -- 172.21.15.12:0/3314018452 learned_addr learned my addr 172.21.15.12:0/3314018452 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff25712700 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060bae8890 0x56060baf70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff24f11700 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060baede00 0x56060baf7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff25f13700 1 -- 172.21.15.12:0/3314018452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060bae8890 msgr2=0x56060baf70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff25f13700 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060bae8890 0x56060baf70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff25f13700 1 -- 172.21.15.12:0/3314018452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060baede00 msgr2=0x56060baf7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff25f13700 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060baede00 0x56060baf7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff25f13700 1 -- 172.21.15.12:0/3314018452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c0118c0 con 0x56060baee720 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff25f13700 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf7b70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7eff1800c7a0 tx=0x7eff18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68c8116aaaede17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff08ff9700 1 -- 172.21.15.12:0/3314018452 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1809a170 con 0x56060baee720 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff08ff9700 1 -- 172.21.15.12:0/3314018452 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7eff18099a70 con 0x56060baee720 2024-05-08T23:54:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff33b30b80 1 -- 172.21.15.12:0/3314018452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff10003720 con 0x56060baee720 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff08ff9700 1 -- 172.21.15.12:0/3314018452 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff1809d8a0 con 0x56060baee720 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff33b30b80 10 client.17817 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.529+0000 7eff33b30b80 1 -- 172.21.15.12:0/3314018452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56060b8b93b0 con 0x56060baee720 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff08ff9700 1 -- 172.21.15.12:0/3314018452 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff18065020 con 0x56060baee720 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff08ff9700 1 -- 172.21.15.12:0/3314018452 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7eff18065190 con 0x56060baee720 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff33b30b80 10 client.17817 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff33b30b80 10 client.17817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff33b30b80 20 client.17817 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff33b30b80 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56060bb58ff0 0x56060bb5b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff33b30b80 4 client.17817 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff33b30b80 1 -- 172.21.15.12:0/3314018452 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56060ba30ec0 con 0x56060bb58ff0 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff25712700 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56060bb58ff0 0x56060bb5b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff25712700 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56060bb58ff0 0x56060bb5b410 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7977d836f04a69bd server_cookie=81989a6e0d253e46 in_seq=0 out_seq=0 2024-05-08T23:54:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.530+0000 7eff08ff9700 10 client.17817 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff08ff9700 1 -- 172.21.15.12:0/3314018452 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56060ba30ec0 con 0x56060bb58ff0 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff08ff9700 10 client.17817 handle_command_reply: tid=0 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 client.17817 shutdown 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 -- 172.21.15.12:0/3314018452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 msgr2=0x56060baf7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf7b70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7eff1800c7a0 tx=0x7eff18004ad0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 -- 172.21.15.12:0/3314018452 shutdown_connections 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56060baee720 0x56060baf7b70 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060bae8890 0x56060baf70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56060bb58ff0 0x56060bb5b410 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 --2- 172.21.15.12:0/3314018452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56060baede00 0x56060baf7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff08ff9700 0 client.17817 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 -- 172.21.15.12:0/3314018452 >> 172.21.15.12:0/3314018452 conn(0x56060ba46210 msgr2=0x56060baecce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 -- 172.21.15.12:0/3314018452 shutdown_connections 2024-05-08T23:54:48.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 1 -- 172.21.15.12:0/3314018452 wait complete. 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.531+0000 7eff33b30b80 20 client.17817 trim_cache size 0 max 16384 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (1 inodes in the stack)", 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stdout: "756675f6-95f7-4560-98b6-eb4ad2ace9d5": { 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "756675f6-95f7-4560-98b6-eb4ad2ace9d5", 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:54:48.532 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:54:48.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 -- 172.21.15.12:0/3953598862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7eff0c0a1290 msgr2=0x7eff0c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7eff0c0a1290 0x7eff0c0a3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff1c005950 tx=0x7eff1c00b410 comp rx=0 tx=0).stop 2024-05-08T23:54:48.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 -- 172.21.15.12:0/3953598862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2019bc60 msgr2=0x7eff2019c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:48.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2019bc60 0x7eff2019c0e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7eff1800ef30 tx=0x7eff1800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 -- 172.21.15.12:0/3953598862 shutdown_connections 2024-05-08T23:54:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7eff0c0a1290 0x7eff0c0a3750 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff2019bc60 0x7eff2019c0e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff200ffa80 0x7eff20197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 --2- 172.21.15.12:0/3953598862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff20104950 0x7eff201978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 -- 172.21.15.12:0/3953598862 >> 172.21.15.12:0/3953598862 conn(0x7eff20076110 msgr2=0x7eff20103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 -- 172.21.15.12:0/3953598862 shutdown_connections 2024-05-08T23:54:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:54:48.532+0000 7eff27976700 1 -- 172.21.15.12:0/3953598862 wait complete. 2024-05-08T23:54:48.540 INFO:tasks.cephfs.filesystem:scrub status for tag:756675f6-95f7-4560-98b6-eb4ad2ace9d5 - {'path': '/', 'tag': '756675f6-95f7-4560-98b6-eb4ad2ace9d5', 'options': 'recursive,force'} 2024-05-08T23:54:48.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:48 smithi204 ceph-mon[47373]: pgmap v263: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-05-08T23:54:48.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:48 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:54:48.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:48 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:48.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:48 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:54:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:48 smithi160 ceph-mon[46152]: pgmap v263: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-05-08T23:54:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:48 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:54:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:48 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:48.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:48 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:54:48.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499095 7.70 MB/sec execute 284 sec latency 2825.255 ms 2024-05-08T23:54:48.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:48 smithi012 ceph-mon[42503]: pgmap v263: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-05-08T23:54:48.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:48 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:54:48.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:48 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:48.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:48 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:54:49.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:49 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:54:49.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:49 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:54:49.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:49 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:49.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:49 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:49.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:49 smithi204 ceph-mon[47373]: pgmap v264: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-05-08T23:54:49.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:49 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:54:49.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:49 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:54:49.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:49 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:49.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:49 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:49.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:49 smithi160 ceph-mon[46152]: pgmap v264: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-05-08T23:54:49.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499095 7.67 MB/sec execute 285 sec latency 3825.343 ms 2024-05-08T23:54:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:49 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:54:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:49 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:54:49.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:49 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:49.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:49 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:49.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:49 smithi012 ceph-mon[42503]: pgmap v264: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-05-08T23:54:50.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499095 7.65 MB/sec execute 286 sec latency 4825.442 ms 2024-05-08T23:54:51.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499095 7.62 MB/sec execute 287 sec latency 5825.569 ms 2024-05-08T23:54:52.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:52 smithi204 ceph-mon[47373]: pgmap v265: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-05-08T23:54:52.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:52 smithi160 ceph-mon[46152]: pgmap v265: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-05-08T23:54:52.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499095 7.59 MB/sec execute 288 sec latency 6825.644 ms 2024-05-08T23:54:52.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:52 smithi012 ceph-mon[42503]: pgmap v265: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-05-08T23:54:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.959+0000 7fc319f7c700 1 -- 172.21.15.12:0/1302656715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc314069750 msgr2=0x7fc314069bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.959+0000 7fc319f7c700 1 --2- 172.21.15.12:0/1302656715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc314069750 0x7fc314069bd0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc308009990 tx=0x7fc308009ca0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.959+0000 7fc319f7c700 1 -- 172.21.15.12:0/1302656715 shutdown_connections 2024-05-08T23:54:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.959+0000 7fc319f7c700 1 --2- 172.21.15.12:0/1302656715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc31406a110 0x7fc31410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.959+0000 7fc319f7c700 1 --2- 172.21.15.12:0/1302656715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc314106fb0 0x7fc314107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.959+0000 7fc319f7c700 1 --2- 172.21.15.12:0/1302656715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc314069750 0x7fc314069bd0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.959+0000 7fc319f7c700 1 -- 172.21.15.12:0/1302656715 >> 172.21.15.12:0/1302656715 conn(0x7fc314077000 msgr2=0x7fc314077410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.959+0000 7fc319f7c700 1 -- 172.21.15.12:0/1302656715 shutdown_connections 2024-05-08T23:54:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.959+0000 7fc319f7c700 1 -- 172.21.15.12:0/1302656715 wait complete. 2024-05-08T23:54:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.960+0000 7fc319f7c700 1 Processor -- start 2024-05-08T23:54:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.960+0000 7fc319f7c700 1 -- start start 2024-05-08T23:54:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.960+0000 7fc319f7c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc314069750 0x7fc314197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc319f7c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc31406a110 0x7fc314197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc319f7c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc314106fb0 0x7fc31419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc319f7c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31419c220 con 0x7fc31406a110 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc319f7c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31419c390 con 0x7fc314106fb0 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc319f7c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31419c550 con 0x7fc314069750 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc312ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc31406a110 0x7fc314197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc312ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc31406a110 0x7fc314197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56370/0 (socket says 172.21.15.12:56370) 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc312ffd700 1 -- 172.21.15.12:0/2772498790 learned_addr learned my addr 172.21.15.12:0/2772498790 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc3137fe700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc314069750 0x7fc314197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc313fff700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc314106fb0 0x7fc31419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc312ffd700 1 -- 172.21.15.12:0/2772498790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc314069750 msgr2=0x7fc314197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc312ffd700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc314069750 0x7fc314197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc312ffd700 1 -- 172.21.15.12:0/2772498790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc314106fb0 msgr2=0x7fc31419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc312ffd700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc314106fb0 0x7fc31419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc312ffd700 1 -- 172.21.15.12:0/2772498790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3080095f0 con 0x7fc31406a110 2024-05-08T23:54:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc3137fe700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc314069750 0x7fc314197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc313fff700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc314106fb0 0x7fc31419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc312ffd700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc31406a110 0x7fc314197840 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc308000c00 tx=0x7fc308005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc310ff9700 1 -- 172.21.15.12:0/2772498790 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30801d070 con 0x7fc31406a110 2024-05-08T23:54:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc310ff9700 1 -- 172.21.15.12:0/2772498790 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc308022470 con 0x7fc31406a110 2024-05-08T23:54:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.961+0000 7fc319f7c700 1 -- 172.21.15.12:0/2772498790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31419c7d0 con 0x7fc31406a110 2024-05-08T23:54:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.962+0000 7fc310ff9700 1 -- 172.21.15.12:0/2772498790 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308017650 con 0x7fc31406a110 2024-05-08T23:54:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.962+0000 7fc319f7c700 1 -- 172.21.15.12:0/2772498790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3141a4070 con 0x7fc31406a110 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.964+0000 7fc310ff9700 1 -- 172.21.15.12:0/2772498790 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc3080225e0 con 0x7fc31406a110 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.965+0000 7fc310ff9700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc2fc0a1310 0x7fc2fc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.965+0000 7fc310ff9700 1 -- 172.21.15.12:0/2772498790 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc3080ce180 con 0x7fc31406a110 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.965+0000 7fc3137fe700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc2fc0a1310 0x7fc2fc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.965+0000 7fc3137fe700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc2fc0a1310 0x7fc2fc0a37d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc304005e10 tx=0x7fc304005d60 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.965+0000 7fc326136b80 1 Processor -- start 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc326136b80 1 -- start start 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc326136b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1a7540 0x55e94e1a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc326136b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 0x55e94e1ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc326136b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1abe40 0x55e94e1ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc326136b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e0fa0f0 con 0x55e94e1abe40 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc326136b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e0f7ab0 con 0x55e94e1a7e60 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc326136b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e0f33a0 con 0x55e94e1a7540 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc313fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1abe40 0x55e94e1ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc313fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1abe40 0x55e94e1ac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56380/0 (socket says 172.21.15.12:56380) 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc313fff700 1 -- 172.21.15.12:0/1011898010 learned_addr learned my addr 172.21.15.12:0/1011898010 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc312ffd700 1 --2- 172.21.15.12:0/1011898010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 0x55e94e1ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc3137fe700 1 --2- 172.21.15.12:0/1011898010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1a7540 0x55e94e1a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc313fff700 1 -- 172.21.15.12:0/1011898010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1a7540 msgr2=0x55e94e1a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc313fff700 1 --2- 172.21.15.12:0/1011898010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1a7540 0x55e94e1a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc313fff700 1 -- 172.21.15.12:0/1011898010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 msgr2=0x55e94e1ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc313fff700 1 --2- 172.21.15.12:0/1011898010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 0x55e94e1ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc313fff700 1 -- 172.21.15.12:0/1011898010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e94df74350 con 0x55e94e1abe40 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.966+0000 7fc312ffd700 1 --2- 172.21.15.12:0/1011898010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 0x55e94e1ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:54:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.967+0000 7fc313fff700 1 --2- 172.21.15.12:0/1011898010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1abe40 0x55e94e1ac2a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009440 tx=0x7fc30c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b386e0efea1a9ab7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.967+0000 7fc2f9ffb700 1 -- 172.21.15.12:0/1011898010 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00e070 con 0x55e94e1abe40 2024-05-08T23:54:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.967+0000 7fc2f9ffb700 1 -- 172.21.15.12:0/1011898010 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc30c00f040 con 0x55e94e1abe40 2024-05-08T23:54:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.967+0000 7fc2f9ffb700 1 -- 172.21.15.12:0/1011898010 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c015b70 con 0x55e94e1abe40 2024-05-08T23:54:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.967+0000 7fc326136b80 1 -- 172.21.15.12:0/1011898010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1abe40 msgr2=0x55e94e1ac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.967+0000 7fc326136b80 1 --2- 172.21.15.12:0/1011898010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1abe40 0x55e94e1ac2a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009440 tx=0x7fc30c009920 comp rx=0 tx=0).stop 2024-05-08T23:54:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 -- 172.21.15.12:0/1011898010 shutdown_connections 2024-05-08T23:54:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 --2- 172.21.15.12:0/1011898010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1abe40 0x55e94e1ac2a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 --2- 172.21.15.12:0/1011898010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1a7540 0x55e94e1a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 --2- 172.21.15.12:0/1011898010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 0x55e94e1ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 -- 172.21.15.12:0/1011898010 >> 172.21.15.12:0/1011898010 conn(0x55e94e1011b0 msgr2=0x55e94e19d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 -- 172.21.15.12:0/1011898010 shutdown_connections 2024-05-08T23:54:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 -- 172.21.15.12:0/1011898010 wait complete. 2024-05-08T23:54:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 -- start start 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 Processor -- start 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.968+0000 7fc326136b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1a7e60 0x55e94e1b0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc326136b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1abe40 0x55e94e1b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc326136b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e08f500 con 0x55e94e1a7540 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc326136b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e08f1a0 con 0x55e94e1abe40 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc326136b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94df76640 con 0x55e94e1a7e60 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc326136b80 10 client.0 _ll_register_callbacks cb 0x55e94e098ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc3137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc3137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56392/0 (socket says 172.21.15.12:56392) 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc3137fe700 1 -- 172.21.15.12:0/2879627447 learned_addr learned my addr 172.21.15.12:0/2879627447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc313fff700 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1abe40 0x55e94e1b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc312ffd700 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1a7e60 0x55e94e1b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc3137fe700 1 -- 172.21.15.12:0/2879627447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1a7e60 msgr2=0x55e94e1b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc3137fe700 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1a7e60 0x55e94e1b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc3137fe700 1 -- 172.21.15.12:0/2879627447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1abe40 msgr2=0x55e94e1b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc3137fe700 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1abe40 0x55e94e1b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc3137fe700 1 -- 172.21.15.12:0/2879627447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3141a4070 con 0x55e94e1a7540 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc3137fe700 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b35d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc30400fb80 tx=0x7fc304004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ff6f7eefe49301 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc2f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2879627447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304017040 con 0x55e94e1a7540 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2879627447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc304003a90 con 0x55e94e1a7540 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.969+0000 7fc326136b80 1 -- 172.21.15.12:0/2879627447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e94e04c210 con 0x55e94e1a7540 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2879627447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc304004560 con 0x55e94e1a7540 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc326136b80 10 client.17829 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc326136b80 1 -- 172.21.15.12:0/2879627447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fc30c0090f0 con 0x55e94e1a7540 2024-05-08T23:54:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2879627447 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e94e04c210 con 0x55e94e1a7540 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2879627447 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc304020410 con 0x55e94e1a7540 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc326136b80 10 client.17829 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc326136b80 10 client.17829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc326136b80 20 client.17829 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc326136b80 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e94e1c1050 0x55e94e1c3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc326136b80 4 client.17829 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc326136b80 1 -- 172.21.15.12:0/2879627447 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e94e1727b0 con 0x55e94e1c1050 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.970+0000 7fc312ffd700 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e94e1c1050 0x55e94e1c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.971+0000 7fc312ffd700 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e94e1c1050 0x55e94e1c3470 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6632803be23934c5 server_cookie=7b2e17353134dc90 in_seq=0 out_seq=0 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.971+0000 7fc2f2ffd700 10 client.17829 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.971+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2879627447 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e94e1727b0 con 0x55e94e1c1050 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.971+0000 7fc2f2ffd700 10 client.17829 handle_command_reply: tid=0 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.971+0000 7fc326136b80 1 client.17829 shutdown 2024-05-08T23:54:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 1 -- 172.21.15.12:0/2879627447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 msgr2=0x55e94e1b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b35d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc30400fb80 tx=0x7fc304004750 comp rx=0 tx=0).stop 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 1 -- 172.21.15.12:0/2879627447 shutdown_connections 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e94e1c1050 0x55e94e1c3470 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1abe40 0x55e94e1b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b35d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 1 --2- 172.21.15.12:0/2879627447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1a7e60 0x55e94e1b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc2f2ffd700 0 client.17829 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 1 -- 172.21.15.12:0/2879627447 >> 172.21.15.12:0/2879627447 conn(0x55e94e1011b0 msgr2=0x55e94e1a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 1 -- 172.21.15.12:0/2879627447 shutdown_connections 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 1 -- 172.21.15.12:0/2879627447 wait complete. 2024-05-08T23:54:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.972+0000 7fc326136b80 20 client.17829 trim_cache size 0 max 16384 2024-05-08T23:54:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.987+0000 7fc326136b80 1 Processor -- start 2024-05-08T23:54:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc326136b80 1 -- start start 2024-05-08T23:54:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc326136b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7540 0x55e94e19d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc326136b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7e60 0x55e94e1b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc326136b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1abe40 0x55e94e1c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc326136b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e1b1ed0 con 0x55e94e1a7e60 2024-05-08T23:54:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc326136b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e1c3690 con 0x55e94e1a7540 2024-05-08T23:54:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc326136b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e1c3850 con 0x55e94e1abe40 2024-05-08T23:54:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc312ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7e60 0x55e94e1b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc312ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7e60 0x55e94e1b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56394/0 (socket says 172.21.15.12:56394) 2024-05-08T23:54:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc312ffd700 1 -- 172.21.15.12:0/3442112876 learned_addr learned my addr 172.21.15.12:0/3442112876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc3137fe700 1 --2- 172.21.15.12:0/3442112876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7540 0x55e94e19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc313fff700 1 --2- 172.21.15.12:0/3442112876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1abe40 0x55e94e1c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc312ffd700 1 -- 172.21.15.12:0/3442112876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1abe40 msgr2=0x55e94e1c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc312ffd700 1 --2- 172.21.15.12:0/3442112876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1abe40 0x55e94e1c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc312ffd700 1 -- 172.21.15.12:0/3442112876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7540 msgr2=0x55e94e19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc312ffd700 1 --2- 172.21.15.12:0/3442112876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7540 0x55e94e19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc312ffd700 1 -- 172.21.15.12:0/3442112876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e94df20b00 con 0x55e94e1a7e60 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc3137fe700 1 --2- 172.21.15.12:0/3442112876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7540 0x55e94e19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.988+0000 7fc312ffd700 1 --2- 172.21.15.12:0/3442112876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7e60 0x55e94e1b31e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc308006010 tx=0x7fc308009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606f7200e78ee0ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc2fa7fc700 1 -- 172.21.15.12:0/3442112876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30807fea0 con 0x55e94e1a7e60 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc2fa7fc700 1 -- 172.21.15.12:0/3442112876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc30809f130 con 0x55e94e1a7e60 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc2fa7fc700 1 -- 172.21.15.12:0/3442112876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30809b7c0 con 0x55e94e1a7e60 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc326136b80 1 -- 172.21.15.12:0/3442112876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7e60 msgr2=0x55e94e1b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc326136b80 1 --2- 172.21.15.12:0/3442112876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7e60 0x55e94e1b31e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc308006010 tx=0x7fc308009310 comp rx=0 tx=0).stop 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc326136b80 1 -- 172.21.15.12:0/3442112876 shutdown_connections 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc326136b80 1 --2- 172.21.15.12:0/3442112876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1abe40 0x55e94e1c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc326136b80 1 --2- 172.21.15.12:0/3442112876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7540 0x55e94e19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc326136b80 1 --2- 172.21.15.12:0/3442112876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7e60 0x55e94e1b31e0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc326136b80 1 -- 172.21.15.12:0/3442112876 >> 172.21.15.12:0/3442112876 conn(0x55e94e1011b0 msgr2=0x55e94e199e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc326136b80 1 -- 172.21.15.12:0/3442112876 shutdown_connections 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.989+0000 7fc326136b80 1 -- 172.21.15.12:0/3442112876 wait complete. 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.990+0000 7fc326136b80 1 -- start start 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.990+0000 7fc326136b80 1 Processor -- start 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.990+0000 7fc326136b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.990+0000 7fc326136b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 0x55e94e1b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.990+0000 7fc326136b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1abe40 0x55e94e1b6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.990+0000 7fc326136b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e1b6c60 con 0x55e94e1a7540 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.990+0000 7fc326136b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e1b6dd0 con 0x55e94e1a7e60 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.990+0000 7fc326136b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e216550 con 0x55e94e1abe40 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc326136b80 10 client.0 _ll_register_callbacks cb 0x55e94e098ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc3137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc3137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56408/0 (socket says 172.21.15.12:56408) 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc3137fe700 1 -- 172.21.15.12:0/2556454322 learned_addr learned my addr 172.21.15.12:0/2556454322 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:54:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc313fff700 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1abe40 0x55e94e1b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc312ffd700 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 0x55e94e1b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc3137fe700 1 -- 172.21.15.12:0/2556454322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1abe40 msgr2=0x55e94e1b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc3137fe700 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1abe40 0x55e94e1b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc3137fe700 1 -- 172.21.15.12:0/2556454322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 msgr2=0x55e94e1b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc3137fe700 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 0x55e94e1b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc3137fe700 1 -- 172.21.15.12:0/2556454322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31419c7d0 con 0x55e94e1a7540 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc312ffd700 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 0x55e94e1b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc3137fe700 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b8400 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc30400fb50 tx=0x7fc30400b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f7e5b062332095 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc2f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2556454322 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304003ca0 con 0x55e94e1a7540 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2556454322 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc304017040 con 0x55e94e1a7540 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc326136b80 1 -- 172.21.15.12:0/2556454322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc30c00f040 con 0x55e94e1a7540 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2556454322 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc304004540 con 0x55e94e1a7540 2024-05-08T23:54:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc326136b80 10 client.17835 fetch_fsmap learned FSMap version 40 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.991+0000 7fc326136b80 1 -- 172.21.15.12:0/2556454322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e94e0b9500 con 0x55e94e1a7540 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.992+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2556454322 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc30c00f040 con 0x55e94e1a7540 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.992+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2556454322 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc304011c20 con 0x55e94e1a7540 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.992+0000 7fc326136b80 10 client.17835 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.992+0000 7fc326136b80 10 client.17835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.992+0000 7fc326136b80 20 client.17835 populate_metadata read hostname 'smithi012' 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.992+0000 7fc326136b80 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e94e217bc0 0x55e94e219fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.992+0000 7fc326136b80 4 client.17835 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.992+0000 7fc326136b80 1 -- 172.21.15.12:0/2556454322 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e94e0ebe60 con 0x55e94e217bc0 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.992+0000 7fc312ffd700 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e94e217bc0 0x55e94e219fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.993+0000 7fc2f2ffd700 10 client.17835 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:52.993+0000 7fc312ffd700 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e94e217bc0 0x55e94e219fe0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48149f9963a2ff68 server_cookie=97c6a21a046090be in_seq=0 out_seq=0 2024-05-08T23:54:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc2f2ffd700 1 -- 172.21.15.12:0/2556454322 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e94e0ebe60 con 0x55e94e217bc0 2024-05-08T23:54:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc2f2ffd700 10 client.17835 handle_command_reply: tid=0 2024-05-08T23:54:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 client.17835 shutdown 2024-05-08T23:54:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 -- 172.21.15.12:0/2556454322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 msgr2=0x55e94e1b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b8400 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc30400fb50 tx=0x7fc30400b580 comp rx=0 tx=0).stop 2024-05-08T23:54:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 -- 172.21.15.12:0/2556454322 shutdown_connections 2024-05-08T23:54:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e94e1abe40 0x55e94e1b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e94e1a7540 0x55e94e1b8400 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e94e217bc0 0x55e94e219fe0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 --2- 172.21.15.12:0/2556454322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e94e1a7e60 0x55e94e1b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc2f2ffd700 0 client.17835 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:54:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 -- 172.21.15.12:0/2556454322 >> 172.21.15.12:0/2556454322 conn(0x55e94e1011b0 msgr2=0x55e94e1c7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 -- 172.21.15.12:0/2556454322 shutdown_connections 2024-05-08T23:54:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 1 -- 172.21.15.12:0/2556454322 wait complete. 2024-05-08T23:54:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.024+0000 7fc326136b80 20 client.17835 trim_cache size 0 max 16384 2024-05-08T23:54:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:54:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:54:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:54:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:54:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 -- 172.21.15.12:0/2772498790 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc2fc0a1310 msgr2=0x7fc2fc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc2fc0a1310 0x7fc2fc0a37d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc304005e10 tx=0x7fc304005d60 comp rx=0 tx=0).stop 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 -- 172.21.15.12:0/2772498790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc31406a110 msgr2=0x7fc314197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc31406a110 0x7fc314197840 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc308000c00 tx=0x7fc308005ef0 comp rx=0 tx=0).stop 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 -- 172.21.15.12:0/2772498790 shutdown_connections 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc2fc0a1310 0x7fc2fc0a37d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc314106fb0 0x7fc31419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc314069750 0x7fc314197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 --2- 172.21.15.12:0/2772498790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc31406a110 0x7fc314197840 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 -- 172.21.15.12:0/2772498790 >> 172.21.15.12:0/2772498790 conn(0x7fc314077000 msgr2=0x7fc3140fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 -- 172.21.15.12:0/2772498790 shutdown_connections 2024-05-08T23:54:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:54:53.025+0000 7fc319f7c700 1 -- 172.21.15.12:0/2772498790 wait complete. 2024-05-08T23:54:53.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499095 7.57 MB/sec execute 289 sec latency 7825.746 ms 2024-05-08T23:54:54.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:54 smithi204 ceph-mon[47373]: pgmap v266: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 627 KiB/s wr, 53 op/s 2024-05-08T23:54:54.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:54 smithi160 ceph-mon[46152]: pgmap v266: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 627 KiB/s wr, 53 op/s 2024-05-08T23:54:54.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499095 7.54 MB/sec execute 290 sec latency 8825.863 ms 2024-05-08T23:54:54.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:54 smithi012 ceph-mon[42503]: pgmap v266: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 627 KiB/s wr, 53 op/s 2024-05-08T23:54:55.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 499095 7.52 MB/sec execute 291 sec latency 9826.008 ms 2024-05-08T23:54:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:55 smithi012 ceph-mon[42503]: pgmap v267: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 627 KiB/s wr, 53 op/s 2024-05-08T23:54:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:55 smithi204 ceph-mon[47373]: pgmap v267: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 627 KiB/s wr, 53 op/s 2024-05-08T23:54:56.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:55 smithi160 ceph-mon[46152]: pgmap v267: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 627 KiB/s wr, 53 op/s 2024-05-08T23:54:56.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 501242 7.54 MB/sec execute 292 sec latency 9981.785 ms 2024-05-08T23:54:56.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:56 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:54:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:56 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:56.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:56 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:57.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:56 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:54:57.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:56 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:57.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:56 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:57.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:56 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:54:57.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:56 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:57.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:56 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:54:57.877 INFO:tasks.workunit.client.0.smithi012.stdout: 1 502533 7.53 MB/sec execute 293 sec latency 23.194 ms 2024-05-08T23:54:57.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:54:57 smithi012 ceph-mon[42503]: pgmap v268: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 981 KiB/s wr, 82 op/s 2024-05-08T23:54:58.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:54:57 smithi204 ceph-mon[47373]: pgmap v268: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 981 KiB/s wr, 82 op/s 2024-05-08T23:54:58.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:54:57 smithi160 ceph-mon[46152]: pgmap v268: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 981 KiB/s wr, 82 op/s 2024-05-08T23:54:58.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 502633 7.50 MB/sec execute 294 sec latency 886.649 ms 2024-05-08T23:54:59.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 502633 7.48 MB/sec execute 295 sec latency 1886.732 ms 2024-05-08T23:55:00.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:00 smithi204 ceph-mon[47373]: pgmap v269: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 698 KiB/s wr, 45 op/s 2024-05-08T23:55:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:00 smithi160 ceph-mon[46152]: pgmap v269: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 698 KiB/s wr, 45 op/s 2024-05-08T23:55:00.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 502633 7.45 MB/sec execute 296 sec latency 2886.814 ms 2024-05-08T23:55:00.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:00 smithi012 ceph-mon[42503]: pgmap v269: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 698 KiB/s wr, 45 op/s 2024-05-08T23:55:01.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 503921 7.45 MB/sec execute 297 sec latency 3040.413 ms 2024-05-08T23:55:02.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:02 smithi204 ceph-mon[47373]: pgmap v270: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-05-08T23:55:02.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:02 smithi160 ceph-mon[46152]: pgmap v270: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-05-08T23:55:02.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 505550 7.46 MB/sec execute 298 sec latency 12.576 ms 2024-05-08T23:55:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:02 smithi012 ceph-mon[42503]: pgmap v270: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-05-08T23:55:03.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:03 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:55:03.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:03 smithi204 ceph-mon[47373]: pgmap v271: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 947 KiB/s wr, 94 op/s 2024-05-08T23:55:03.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:03 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:55:03.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:03 smithi160 ceph-mon[46152]: pgmap v271: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 947 KiB/s wr, 94 op/s 2024-05-08T23:55:03.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 506183 7.44 MB/sec execute 299 sec latency 443.516 ms 2024-05-08T23:55:03.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:03 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:55:03.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:03 smithi012 ceph-mon[42503]: pgmap v271: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 947 KiB/s wr, 94 op/s 2024-05-08T23:55:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:55:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:55:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:04 smithi204 ceph-mon[47373]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T23:55:04.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:55:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:55:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:04 smithi160 ceph-mon[46152]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T23:55:04.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 506183 7.42 MB/sec execute 300 sec latency 1443.583 ms 2024-05-08T23:55:04.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:55:04.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:55:04.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:04 smithi012 ceph-mon[42503]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T23:55:05.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:05 smithi160 ceph-mon[46152]: pgmap v273: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-05-08T23:55:05.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 506183 7.39 MB/sec execute 301 sec latency 2443.681 ms 2024-05-08T23:55:05.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:05 smithi012 ceph-mon[42503]: pgmap v273: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-05-08T23:55:06.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:05 smithi204 ceph-mon[47373]: pgmap v273: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-05-08T23:55:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.156+0000 7f32a9acf700 1 -- 172.21.15.12:0/4185129278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32a4105f40 msgr2=0x7f32a4106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.156+0000 7f32a9acf700 1 --2- 172.21.15.12:0/4185129278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32a4105f40 0x7f32a4106320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3294009990 tx=0x7f3294009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.156+0000 7f32a9acf700 1 -- 172.21.15.12:0/4185129278 shutdown_connections 2024-05-08T23:55:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.156+0000 7f32a9acf700 1 --2- 172.21.15.12:0/4185129278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a4100c60 0x7f32a410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.156+0000 7f32a9acf700 1 --2- 172.21.15.12:0/4185129278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32a4105f40 0x7f32a4106320 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.156+0000 7f32a9acf700 1 --2- 172.21.15.12:0/4185129278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32a41002a0 0x7f32a4100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.156+0000 7f32a9acf700 1 -- 172.21.15.12:0/4185129278 >> 172.21.15.12:0/4185129278 conn(0x7f32a4076c10 msgr2=0x7f32a4077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.156+0000 7f32a9acf700 1 -- 172.21.15.12:0/4185129278 shutdown_connections 2024-05-08T23:55:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.156+0000 7f32a9acf700 1 -- 172.21.15.12:0/4185129278 wait complete. 2024-05-08T23:55:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.156+0000 7f32a9acf700 1 Processor -- start 2024-05-08T23:55:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a9acf700 1 -- start start 2024-05-08T23:55:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a9acf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32a41002a0 0x7f32a41972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a9acf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a4100c60 0x7f32a41977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a9acf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32a4105f40 0x7f32a419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a9acf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a419c1c0 con 0x7f32a41002a0 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a9acf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a419c330 con 0x7f32a4105f40 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a9acf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a419c4f0 con 0x7f32a4100c60 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32a41002a0 0x7f32a41972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32a4105f40 0x7f32a419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32a41002a0 0x7f32a41972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46036/0 (socket says 172.21.15.12:46036) 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a37fe700 1 -- 172.21.15.12:0/3849164189 learned_addr learned my addr 172.21.15.12:0/3849164189 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.157+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a4100c60 0x7f32a41977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a37fe700 1 -- 172.21.15.12:0/3849164189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a4100c60 msgr2=0x7f32a41977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a37fe700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a4100c60 0x7f32a41977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a37fe700 1 -- 172.21.15.12:0/3849164189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32a4105f40 msgr2=0x7f32a419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a37fe700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32a4105f40 0x7f32a419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a37fe700 1 -- 172.21.15.12:0/3849164189 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32940095f0 con 0x7f32a41002a0 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a3fff700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32a4105f40 0x7f32a419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a37fe700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32a41002a0 0x7f32a41972a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3294004760 tx=0x7f3294004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a0ff9700 1 -- 172.21.15.12:0/3849164189 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329401d070 con 0x7f32a41002a0 2024-05-08T23:55:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a9acf700 1 -- 172.21.15.12:0/3849164189 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a419c770 con 0x7f32a41002a0 2024-05-08T23:55:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a0ff9700 1 -- 172.21.15.12:0/3849164189 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3294022470 con 0x7f32a41002a0 2024-05-08T23:55:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a0ff9700 1 -- 172.21.15.12:0/3849164189 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294017650 con 0x7f32a41002a0 2024-05-08T23:55:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.158+0000 7f32a9acf700 1 -- 172.21.15.12:0/3849164189 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a41a4010 con 0x7f32a41002a0 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.161+0000 7f32a0ff9700 1 -- 172.21.15.12:0/3849164189 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f32940177b0 con 0x7f32a41002a0 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.162+0000 7f32a0ff9700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f328c0a1290 0x7f328c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.162+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f328c0a1290 0x7f328c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.162+0000 7f32a0ff9700 1 -- 172.21.15.12:0/3849164189 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f32940cde30 con 0x7f32a41002a0 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.162+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f328c0a1290 0x7f328c0a3750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f32a41987f0 tx=0x7f3298009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32b5c89b80 1 Processor -- start 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32b5c89b80 1 -- start start 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d8950 0x55e0d01d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 0x55e0d01d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d012a0f0 con 0x55e0d01d9270 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d0127ab0 con 0x55e0d01d3480 2024-05-08T23:55:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d01233a0 con 0x55e0d01d8950 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d8950 0x55e0d01d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46046/0 (socket says 172.21.15.12:46046) 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32a2ffd700 1 -- 172.21.15.12:0/1801487447 learned_addr learned my addr 172.21.15.12:0/1801487447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32a3fff700 1 --2- 172.21.15.12:0/1801487447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 0x55e0d01d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32a37fe700 1 -- 172.21.15.12:0/1801487447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 msgr2=0x55e0d01d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32a37fe700 1 --2- 172.21.15.12:0/1801487447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 0x55e0d01d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32a37fe700 1 -- 172.21.15.12:0/1801487447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 msgr2=0x55e0d01d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32a37fe700 1 --2- 172.21.15.12:0/1801487447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.163+0000 7f32a37fe700 1 -- 172.21.15.12:0/1801487447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0cffa4350 con 0x55e0d01d8950 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.164+0000 7f32a37fe700 1 --2- 172.21.15.12:0/1801487447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d8950 0x55e0d01d8d30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f32940cb570 tx=0x7f329409afc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c369ba81972a3fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.164+0000 7f3289ffb700 1 -- 172.21.15.12:0/1801487447 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32940a33b0 con 0x55e0d01d8950 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.164+0000 7f3289ffb700 1 -- 172.21.15.12:0/1801487447 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32940228e0 con 0x55e0d01d8950 2024-05-08T23:55:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.164+0000 7f3289ffb700 1 -- 172.21.15.12:0/1801487447 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329409a940 con 0x55e0d01d8950 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.165+0000 7f32b5c89b80 1 -- 172.21.15.12:0/1801487447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d8950 msgr2=0x55e0d01d8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.165+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/1801487447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d8950 0x55e0d01d8d30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f32940cb570 tx=0x7f329409afc0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.165+0000 7f32b5c89b80 1 -- 172.21.15.12:0/1801487447 shutdown_connections 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.165+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/1801487447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 0x55e0d01d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.165+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/1801487447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d8950 0x55e0d01d8d30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.165+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/1801487447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.165+0000 7f32b5c89b80 1 -- 172.21.15.12:0/1801487447 >> 172.21.15.12:0/1801487447 conn(0x55e0d01311b0 msgr2=0x55e0d01cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.165+0000 7f32b5c89b80 1 -- 172.21.15.12:0/1801487447 shutdown_connections 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.165+0000 7f32b5c89b80 1 -- 172.21.15.12:0/1801487447 wait complete. 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.165+0000 7f32b5c89b80 1 -- start start 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32b5c89b80 1 Processor -- start 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 0x55e0d01df150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d8950 0x55e0d01dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01dca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d00bf500 con 0x55e0d01d9270 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d00bf1a0 con 0x55e0d01d8950 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0cffa6640 con 0x55e0d01d3480 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32b5c89b80 10 client.0 _ll_register_callbacks cb 0x55e0d00c8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d8950 0x55e0d01dc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01dca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 0x55e0d01df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d8950 0x55e0d01dc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33776/0 (socket says 172.21.15.12:33776) 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a2ffd700 1 -- 172.21.15.12:0/568416649 learned_addr learned my addr 172.21.15.12:0/568416649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a37fe700 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 0x55e0d01df150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41204/0 (socket says 172.21.15.12:41204) 2024-05-08T23:55:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a3fff700 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01dca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46052/0 (socket says 172.21.15.12:46052) 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a37fe700 1 -- 172.21.15.12:0/568416649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d8950 msgr2=0x55e0d01dc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a37fe700 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d8950 0x55e0d01dc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a37fe700 1 -- 172.21.15.12:0/568416649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 msgr2=0x55e0d01dca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a37fe700 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01dca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a37fe700 1 -- 172.21.15.12:0/568416649 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a41a4010 con 0x55e0d01d3480 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d8950 0x55e0d01dc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.166+0000 7f32a3fff700 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01dca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f32a37fe700 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 0x55e0d01df150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3294099540 tx=0x7f329409a910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e37665f7566a99da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f3282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f3282ffd700 1 -- 172.21.15.12:0/568416649 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32940a33b0 con 0x55e0d01d3480 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f3282ffd700 1 -- 172.21.15.12:0/568416649 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f3294099590 con 0x55e0d01d3480 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f32b5c89b80 1 -- 172.21.15.12:0/568416649 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0d007c210 con 0x55e0d01d3480 2024-05-08T23:55:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f3282ffd700 1 -- 172.21.15.12:0/568416649 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32940b9310 con 0x55e0d01d3480 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f32b5c89b80 10 client.27619 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f32b5c89b80 1 -- 172.21.15.12:0/568416649 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e0cff50b00 con 0x55e0d01d3480 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f3282ffd700 1 -- 172.21.15.12:0/568416649 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0d007c210 con 0x55e0d01d3480 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f3282ffd700 1 -- 172.21.15.12:0/568416649 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f3294017bf0 con 0x55e0d01d3480 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f32b5c89b80 10 client.27619 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f32b5c89b80 10 client.27619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f32b5c89b80 20 client.27619 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d01ec4c0 0x55e0d01ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f32b5c89b80 4 client.27619 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.167+0000 7f32b5c89b80 1 -- 172.21.15.12:0/568416649 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0d01a27b0 con 0x55e0d01ec4c0 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.168+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d01ec4c0 0x55e0d01ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.168+0000 7f3282ffd700 10 client.27619 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.168+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d01ec4c0 0x55e0d01ee8e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f687046c5bb186 server_cookie=421081860b0e4d19 in_seq=0 out_seq=0 2024-05-08T23:55:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f3282ffd700 1 -- 172.21.15.12:0/568416649 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0d01a27b0 con 0x55e0d01ec4c0 2024-05-08T23:55:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f3282ffd700 10 client.27619 handle_command_reply: tid=0 2024-05-08T23:55:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 client.27619 shutdown 2024-05-08T23:55:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 -- 172.21.15.12:0/568416649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 msgr2=0x55e0d01df150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 0x55e0d01df150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3294099540 tx=0x7f329409a910 comp rx=0 tx=0).stop 2024-05-08T23:55:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 -- 172.21.15.12:0/568416649 shutdown_connections 2024-05-08T23:55:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d9270 0x55e0d01dca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 0x55e0d01df150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d01ec4c0 0x55e0d01ee8e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/568416649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d8950 0x55e0d01dc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f3282ffd700 0 client.27619 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 -- 172.21.15.12:0/568416649 >> 172.21.15.12:0/568416649 conn(0x55e0d01311b0 msgr2=0x55e0d01d1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 -- 172.21.15.12:0/568416649 shutdown_connections 2024-05-08T23:55:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 1 -- 172.21.15.12:0/568416649 wait complete. 2024-05-08T23:55:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.169+0000 7f32b5c89b80 20 client.27619 trim_cache size 0 max 16384 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32b5c89b80 1 Processor -- start 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32b5c89b80 1 -- start start 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 0x55e0d01cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d9270 0x55e0d01e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d01dd050 con 0x55e0d01d8950 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d01c5a40 con 0x55e0d01d9270 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d01f3a90 con 0x55e0d01d3480 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 0x55e0d01cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46064/0 (socket says 172.21.15.12:46064) 2024-05-08T23:55:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d9270 0x55e0d01e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32a2ffd700 1 -- 172.21.15.12:0/828492597 learned_addr learned my addr 172.21.15.12:0/828492597 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32a2ffd700 1 -- 172.21.15.12:0/828492597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 msgr2=0x55e0d01cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/828492597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 0x55e0d01cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32a2ffd700 1 -- 172.21.15.12:0/828492597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d9270 msgr2=0x55e0d01e3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/828492597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d9270 0x55e0d01e3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.186+0000 7f32a2ffd700 1 -- 172.21.15.12:0/828492597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0d00e9500 con 0x55e0d01d8950 2024-05-08T23:55:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/828492597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e37f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3298013010 tx=0x7f3298014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa299341f66212d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f328a7fc700 1 -- 172.21.15.12:0/828492597 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3298022030 con 0x55e0d01d8950 2024-05-08T23:55:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f328a7fc700 1 -- 172.21.15.12:0/828492597 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3298023040 con 0x55e0d01d8950 2024-05-08T23:55:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f328a7fc700 1 -- 172.21.15.12:0/828492597 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329801d410 con 0x55e0d01d8950 2024-05-08T23:55:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f32b5c89b80 1 -- 172.21.15.12:0/828492597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 msgr2=0x55e0d01e37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/828492597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e37f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3298013010 tx=0x7f3298014000 comp rx=0 tx=0).stop 2024-05-08T23:55:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f32b5c89b80 1 -- 172.21.15.12:0/828492597 shutdown_connections 2024-05-08T23:55:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/828492597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d9270 0x55e0d01e3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/828492597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d3480 0x55e0d01cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/828492597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e37f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.187+0000 7f32b5c89b80 1 -- 172.21.15.12:0/828492597 >> 172.21.15.12:0/828492597 conn(0x55e0d01311b0 msgr2=0x55e0d01d2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.188+0000 7f32b5c89b80 1 -- 172.21.15.12:0/828492597 shutdown_connections 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.188+0000 7f32b5c89b80 1 -- 172.21.15.12:0/828492597 wait complete. 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.188+0000 7f32b5c89b80 1 -- start start 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.188+0000 7f32b5c89b80 1 Processor -- start 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.188+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 0x55e0d01e1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.188+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.188+0000 7f32b5c89b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d9270 0x55e0d0241980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.188+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d01e25e0 con 0x55e0d01d8950 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.188+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d0241ec0 con 0x55e0d01d3480 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.188+0000 7f32b5c89b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d0242080 con 0x55e0d01d9270 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32b5c89b80 10 client.0 _ll_register_callbacks cb 0x55e0d00c8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 0x55e0d01e1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 0x55e0d01e1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33782/0 (socket says 172.21.15.12:33782) 2024-05-08T23:55:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a37fe700 1 -- 172.21.15.12:0/244620529 learned_addr learned my addr 172.21.15.12:0/244620529 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46072/0 (socket says 172.21.15.12:46072) 2024-05-08T23:55:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a3fff700 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d9270 0x55e0d0241980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a37fe700 1 -- 172.21.15.12:0/244620529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d9270 msgr2=0x55e0d0241980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a37fe700 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d9270 0x55e0d0241980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a37fe700 1 -- 172.21.15.12:0/244620529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 msgr2=0x55e0d01e1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a37fe700 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a37fe700 1 -- 172.21.15.12:0/244620529 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3298017070 con 0x55e0d01d3480 2024-05-08T23:55:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a3fff700 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d9270 0x55e0d0241980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32a37fe700 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 0x55e0d01e1930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3294022c60 tx=0x7f3294005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a48f4794e31e8c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f3282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f3282ffd700 1 -- 172.21.15.12:0/244620529 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32940b9310 con 0x55e0d01d3480 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f3282ffd700 1 -- 172.21.15.12:0/244620529 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f32940a33b0 con 0x55e0d01d3480 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.189+0000 7f32b5c89b80 1 -- 172.21.15.12:0/244620529 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32940228e0 con 0x55e0d01d3480 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f3282ffd700 1 -- 172.21.15.12:0/244620529 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f329409a980 con 0x55e0d01d3480 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f32b5c89b80 10 client.27632 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f32b5c89b80 1 -- 172.21.15.12:0/244620529 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e0cffa8c70 con 0x55e0d01d3480 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f3282ffd700 1 -- 172.21.15.12:0/244620529 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32940228e0 con 0x55e0d01d3480 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f3282ffd700 1 -- 172.21.15.12:0/244620529 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f329409a150 con 0x55e0d01d3480 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f32b5c89b80 10 client.27632 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f32b5c89b80 10 client.27632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f32b5c89b80 20 client.27632 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d0243380 0x55e0d0245770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f32b5c89b80 4 client.27632 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f32b5c89b80 1 -- 172.21.15.12:0/244620529 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e0d011be60 con 0x55e0d0243380 2024-05-08T23:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.190+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d0243380 0x55e0d0245770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.191+0000 7f3282ffd700 10 client.27632 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.191+0000 7f32a2ffd700 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d0243380 0x55e0d0245770 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18cadfb27b1d5c2e server_cookie=a7346ce3a8e29cf6 in_seq=0 out_seq=0 2024-05-08T23:55:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.234+0000 7f3282ffd700 1 -- 172.21.15.12:0/244620529 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e0d011be60 con 0x55e0d0243380 2024-05-08T23:55:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.234+0000 7f3282ffd700 10 client.27632 handle_command_reply: tid=0 2024-05-08T23:55:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.234+0000 7f32b5c89b80 1 client.27632 shutdown 2024-05-08T23:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 1 -- 172.21.15.12:0/244620529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 msgr2=0x55e0d01e1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 0x55e0d01e1930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3294022c60 tx=0x7f3294005fb0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 1 -- 172.21.15.12:0/244620529 shutdown_connections 2024-05-08T23:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d01d9270 0x55e0d0241980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d0243380 0x55e0d0245770 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d01d3480 0x55e0d01e1930 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f3282ffd700 0 client.27632 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 1 --2- 172.21.15.12:0/244620529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d01d8950 0x55e0d01e1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 1 -- 172.21.15.12:0/244620529 >> 172.21.15.12:0/244620529 conn(0x55e0d01311b0 msgr2=0x55e0d01e82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 1 -- 172.21.15.12:0/244620529 shutdown_connections 2024-05-08T23:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 1 -- 172.21.15.12:0/244620529 wait complete. 2024-05-08T23:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.235+0000 7f32b5c89b80 20 client.27632 trim_cache size 0 max 16384 2024-05-08T23:55:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:55:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:55:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:55:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:55:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 -- 172.21.15.12:0/3849164189 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f328c0a1290 msgr2=0x7f328c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f328c0a1290 0x7f328c0a3750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f32a41987f0 tx=0x7f3298009260 comp rx=0 tx=0).stop 2024-05-08T23:55:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 -- 172.21.15.12:0/3849164189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32a41002a0 msgr2=0x7f32a41972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32a41002a0 0x7f32a41972a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3294004760 tx=0x7f3294004790 comp rx=0 tx=0).stop 2024-05-08T23:55:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 -- 172.21.15.12:0/3849164189 shutdown_connections 2024-05-08T23:55:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f328c0a1290 0x7f328c0a3750 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32a4105f40 0x7f32a419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32a41002a0 0x7f32a41972a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 --2- 172.21.15.12:0/3849164189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32a4100c60 0x7f32a41977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 -- 172.21.15.12:0/3849164189 >> 172.21.15.12:0/3849164189 conn(0x7f32a4076c10 msgr2=0x7f32a4104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 -- 172.21.15.12:0/3849164189 shutdown_connections 2024-05-08T23:55:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:06.236+0000 7f32a9acf700 1 -- 172.21.15.12:0/3849164189 wait complete. 2024-05-08T23:55:06.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 508191 7.41 MB/sec execute 302 sec latency 2592.504 ms 2024-05-08T23:55:07.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 509593 7.41 MB/sec execute 303 sec latency 16.974 ms 2024-05-08T23:55:08.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:08 smithi204 ceph-mon[47373]: pgmap v274: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-08T23:55:08.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:08 smithi160 ceph-mon[46152]: pgmap v274: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-08T23:55:08.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 509726 7.38 MB/sec execute 304 sec latency 931.709 ms 2024-05-08T23:55:08.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:08 smithi012 ceph-mon[42503]: pgmap v274: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-08T23:55:09.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 509726 7.36 MB/sec execute 305 sec latency 1931.832 ms 2024-05-08T23:55:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:09 smithi012 ceph-mon[42503]: pgmap v275: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-08T23:55:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:09 smithi204 ceph-mon[47373]: pgmap v275: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-08T23:55:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:09 smithi160 ceph-mon[46152]: pgmap v275: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-08T23:55:10.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 509726 7.34 MB/sec execute 306 sec latency 2931.872 ms 2024-05-08T23:55:11.878 INFO:tasks.workunit.client.0.smithi012.stdout: 1 511803 7.36 MB/sec execute 307 sec latency 3076.212 ms 2024-05-08T23:55:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:11 smithi012 ceph-mon[42503]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T23:55:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:11 smithi012 ceph-mon[42503]: pgmap v277: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-05-08T23:55:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:11 smithi204 ceph-mon[47373]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T23:55:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:11 smithi204 ceph-mon[47373]: pgmap v277: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-05-08T23:55:12.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:11 smithi160 ceph-mon[46152]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T23:55:12.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:11 smithi160 ceph-mon[46152]: pgmap v277: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-05-08T23:55:12.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 513272 7.35 MB/sec execute 308 sec latency 17.814 ms 2024-05-08T23:55:13.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 513272 7.33 MB/sec execute 309 sec latency 1009.561 ms 2024-05-08T23:55:14.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:14 smithi204 ceph-mon[47373]: pgmap v278: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-05-08T23:55:14.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:14 smithi160 ceph-mon[46152]: pgmap v278: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-05-08T23:55:14.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 513272 7.30 MB/sec execute 310 sec latency 2009.655 ms 2024-05-08T23:55:14.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:14 smithi012 ceph-mon[42503]: pgmap v278: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-05-08T23:55:15.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:15 smithi160 ceph-mon[46152]: pgmap v279: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.6 MiB/s wr, 154 op/s 2024-05-08T23:55:15.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 513272 7.28 MB/sec execute 311 sec latency 3009.727 ms 2024-05-08T23:55:15.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:15 smithi012 ceph-mon[42503]: pgmap v279: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.6 MiB/s wr, 154 op/s 2024-05-08T23:55:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:15 smithi204 ceph-mon[47373]: pgmap v279: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.6 MiB/s wr, 154 op/s 2024-05-08T23:55:16.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 515384 7.30 MB/sec execute 312 sec latency 3156.462 ms 2024-05-08T23:55:17.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 516783 7.29 MB/sec execute 313 sec latency 66.789 ms 2024-05-08T23:55:18.541 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:55:18.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.695+0000 7f83b570e700 1 -- 172.21.15.12:0/2990568947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b0100a10 msgr2=0x7f83b0104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.695+0000 7f83b570e700 1 --2- 172.21.15.12:0/2990568947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b0100a10 0x7f83b0104a50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8398009990 tx=0x7f8398009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.695+0000 7f83b570e700 1 -- 172.21.15.12:0/2990568947 shutdown_connections 2024-05-08T23:55:18.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.695+0000 7f83b570e700 1 --2- 172.21.15.12:0/2990568947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83b0105180 0x7f83b010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.695+0000 7f83b570e700 1 --2- 172.21.15.12:0/2990568947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83b0100060 0x7f83b0100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.695+0000 7f83b570e700 1 --2- 172.21.15.12:0/2990568947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b0100a10 0x7f83b0104a50 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.695+0000 7f83b570e700 1 -- 172.21.15.12:0/2990568947 >> 172.21.15.12:0/2990568947 conn(0x7f83b0076fa0 msgr2=0x7f83b00773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.695+0000 7f83b570e700 1 -- 172.21.15.12:0/2990568947 shutdown_connections 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.695+0000 7f83b570e700 1 -- 172.21.15.12:0/2990568947 wait complete. 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.696+0000 7f83b570e700 1 Processor -- start 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.696+0000 7f83b570e700 1 -- start start 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.696+0000 7f83b570e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83b0100060 0x7f83b0101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.696+0000 7f83b570e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b0105180 0x7f83b0102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.696+0000 7f83b570e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83b0102570 0x7f83b01a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.696+0000 7f83b570e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01a5c10 con 0x7f83b0105180 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.696+0000 7f83b570e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01a5d80 con 0x7f83b0100060 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.696+0000 7f83b570e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01a5f40 con 0x7f83b0102570 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83a67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b0105180 0x7f83b0102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83b0100060 0x7f83b0101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83a67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b0105180 0x7f83b0102030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53926/0 (socket says 172.21.15.12:53926) 2024-05-08T23:55:18.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83a67fc700 1 -- 172.21.15.12:0/2648245925 learned_addr learned my addr 172.21.15.12:0/2648245925 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83af7fe700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83b0102570 0x7f83b01a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83aeffd700 1 -- 172.21.15.12:0/2648245925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83b0102570 msgr2=0x7f83b01a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83aeffd700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83b0102570 0x7f83b01a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83aeffd700 1 -- 172.21.15.12:0/2648245925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b0105180 msgr2=0x7f83b0102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83aeffd700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b0105180 0x7f83b0102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83aeffd700 1 -- 172.21.15.12:0/2648245925 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980095f0 con 0x7f83b0100060 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83aeffd700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83b0100060 0x7f83b0101af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83a0007f10 tx=0x7f83a000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83acff9700 1 -- 172.21.15.12:0/2648245925 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0004db0 con 0x7f83b0100060 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83b570e700 1 -- 172.21.15.12:0/2648245925 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b01a6220 con 0x7f83b0100060 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83acff9700 1 -- 172.21.15.12:0/2648245925 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a000f040 con 0x7f83b0100060 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83acff9700 1 -- 172.21.15.12:0/2648245925 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0015400 con 0x7f83b0100060 2024-05-08T23:55:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.697+0000 7f83b570e700 1 -- 172.21.15.12:0/2648245925 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b01a6730 con 0x7f83b0100060 2024-05-08T23:55:18.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:18 smithi160 ceph-mon[46152]: pgmap v280: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-05-08T23:55:18.769 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:18 smithi204 ceph-mon[47373]: pgmap v280: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-05-08T23:55:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.700+0000 7f83acff9700 1 -- 172.21.15.12:0/2648245925 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f83a000dc00 con 0x7f83b0100060 2024-05-08T23:55:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.701+0000 7f83acff9700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f839c0a1290 0x7f839c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.701+0000 7f83acff9700 1 -- 172.21.15.12:0/2648245925 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f83a00cc740 con 0x7f83b0100060 2024-05-08T23:55:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.701+0000 7f83a67fc700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f839c0a1290 0x7f839c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.701+0000 7f83a67fc700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f839c0a1290 0x7f839c0a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f83b01031d0 tx=0x7f8398005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.701+0000 7f83c18c8b80 1 Processor -- start 2024-05-08T23:55:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.701+0000 7f83c18c8b80 1 -- start start 2024-05-08T23:55:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad21350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 0x55e05ad21cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ac78150 con 0x55e05ad27720 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ac75b10 con 0x55e05ad21890 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ac71400 con 0x55e05ad26e00 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83a67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad21350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83a67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad21350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53934/0 (socket says 172.21.15.12:53934) 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83a67fc700 1 -- 172.21.15.12:0/1156759025 learned_addr learned my addr 172.21.15.12:0/1156759025 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83af7fe700 1 --2- 172.21.15.12:0/1156759025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 0x55e05ad21cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83aeffd700 1 --2- 172.21.15.12:0/1156759025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83a67fc700 1 -- 172.21.15.12:0/1156759025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 msgr2=0x55e05ad271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83a67fc700 1 --2- 172.21.15.12:0/1156759025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad271e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83a67fc700 1 -- 172.21.15.12:0/1156759025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 msgr2=0x55e05ad21cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83a67fc700 1 --2- 172.21.15.12:0/1156759025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 0x55e05ad21cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83a67fc700 1 -- 172.21.15.12:0/1156759025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e05aaf23b0 con 0x55e05ad27720 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.702+0000 7f83af7fe700 1 --2- 172.21.15.12:0/1156759025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 0x55e05ad21cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.703+0000 7f83a67fc700 1 --2- 172.21.15.12:0/1156759025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad21350 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f839800b660 tx=0x7f8398020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2ce8b57611db25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.703+0000 7f83a57fa700 1 -- 172.21.15.12:0/1156759025 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398004500 con 0x55e05ad27720 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.703+0000 7f83a57fa700 1 -- 172.21.15.12:0/1156759025 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f839801e040 con 0x55e05ad27720 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.703+0000 7f83a57fa700 1 -- 172.21.15.12:0/1156759025 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800fa40 con 0x55e05ad27720 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.703+0000 7f83c18c8b80 1 -- 172.21.15.12:0/1156759025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 msgr2=0x55e05ad21350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.703+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/1156759025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad21350 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f839800b660 tx=0x7f8398020000 comp rx=0 tx=0).stop 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 -- 172.21.15.12:0/1156759025 shutdown_connections 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/1156759025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 0x55e05ad21cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/1156759025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad271e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/1156759025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad21350 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 -- 172.21.15.12:0/1156759025 >> 172.21.15.12:0/1156759025 conn(0x55e05ac7f210 msgr2=0x55e05ad1bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 -- 172.21.15.12:0/1156759025 shutdown_connections 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 -- 172.21.15.12:0/1156759025 wait complete. 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 -- start start 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 Processor -- start 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad2d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad26e00 0x55e05ad2aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad27720 0x55e05ad2b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ac0d560 con 0x55e05ad21890 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ac0d200 con 0x55e05ad26e00 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05aaf46a0 con 0x55e05ad27720 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83c18c8b80 10 client.0 _ll_register_callbacks cb 0x55e05ac16b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad2d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad2d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53950/0 (socket says 172.21.15.12:53950) 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.704+0000 7f83aeffd700 1 -- 172.21.15.12:0/2643733349 learned_addr learned my addr 172.21.15.12:0/2643733349 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:18.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83a67fc700 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad26e00 0x55e05ad2aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83af7fe700 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad27720 0x55e05ad2b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83aeffd700 1 -- 172.21.15.12:0/2643733349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad27720 msgr2=0x55e05ad2b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83aeffd700 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad27720 0x55e05ad2b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83aeffd700 1 -- 172.21.15.12:0/2643733349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad26e00 msgr2=0x55e05ad2aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83aeffd700 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad26e00 0x55e05ad2aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83aeffd700 1 -- 172.21.15.12:0/2643733349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b010d570 con 0x55e05ad21890 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83aeffd700 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad2d720 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f83a0095cb0 tx=0x7f83a0099280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac18380c23eb8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83967fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83967fc700 1 -- 172.21.15.12:0/2643733349 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a00b92b0 con 0x55e05ad21890 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83967fc700 1 -- 172.21.15.12:0/2643733349 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f83a009d630 con 0x55e05ad21890 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83c18c8b80 1 -- 172.21.15.12:0/2643733349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e05abca270 con 0x55e05ad21890 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83967fc700 1 -- 172.21.15.12:0/2643733349 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83a00a1c80 con 0x55e05ad21890 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83c18c8b80 10 client.17865 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83c18c8b80 1 -- 172.21.15.12:0/2643733349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e05aa9eb60 con 0x55e05ad21890 2024-05-08T23:55:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.705+0000 7f83967fc700 1 -- 172.21.15.12:0/2643733349 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e05abca270 con 0x55e05ad21890 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.706+0000 7f83967fc700 1 -- 172.21.15.12:0/2643733349 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f83a0098d60 con 0x55e05ad21890 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.706+0000 7f83c18c8b80 10 client.17865 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.706+0000 7f83c18c8b80 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.706+0000 7f83c18c8b80 20 client.17865 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.706+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e05ad3b1d0 0x55e05ad3d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.706+0000 7f83c18c8b80 4 client.17865 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.706+0000 7f83c18c8b80 1 -- 172.21.15.12:0/2643733349 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e05acf0810 con 0x55e05ad3b1d0 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.706+0000 7f83a67fc700 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e05ad3b1d0 0x55e05ad3d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.706+0000 7f83967fc700 10 client.17865 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.706+0000 7f83a67fc700 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e05ad3b1d0 0x55e05ad3d5f0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f05d69ef4c08f2 server_cookie=9c6f9aa2d4668820 in_seq=0 out_seq=0 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83967fc700 1 -- 172.21.15.12:0/2643733349 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e05acf0810 con 0x55e05ad3b1d0 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83967fc700 10 client.17865 handle_command_reply: tid=0 2024-05-08T23:55:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 client.17865 shutdown 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 -- 172.21.15.12:0/2643733349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 msgr2=0x55e05ad2d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad2d720 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f83a0095cb0 tx=0x7f83a0099280 comp rx=0 tx=0).stop 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 -- 172.21.15.12:0/2643733349 shutdown_connections 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad27720 0x55e05ad2b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad2d720 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e05ad3b1d0 0x55e05ad3d5f0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/2643733349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad26e00 0x55e05ad2aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83967fc700 0 client.17865 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 -- 172.21.15.12:0/2643733349 >> 172.21.15.12:0/2643733349 conn(0x55e05ac7f210 msgr2=0x55e05ad1b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 -- 172.21.15.12:0/2643733349 shutdown_connections 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.707+0000 7f83c18c8b80 1 -- 172.21.15.12:0/2643733349 wait complete. 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.708+0000 7f83c18c8b80 20 client.17865 trim_cache size 0 max 16384 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.726+0000 7f83c18c8b80 1 Processor -- start 2024-05-08T23:55:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.726+0000 7f83c18c8b80 1 -- start start 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.726+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 0x55e05ad1bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.726+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad32110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.726+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad32650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.726+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ad216d0 con 0x55e05ad27720 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.726+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ad42800 con 0x55e05ad21890 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.726+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ad3afb0 con 0x55e05ad26e00 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.726+0000 7f83af7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad32650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83af7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad32650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53962/0 (socket says 172.21.15.12:53962) 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83af7fe700 1 -- 172.21.15.12:0/1507712327 learned_addr learned my addr 172.21.15.12:0/1507712327 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83aeffd700 1 --2- 172.21.15.12:0/1507712327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 0x55e05ad1bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83a67fc700 1 --2- 172.21.15.12:0/1507712327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad32110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83af7fe700 1 -- 172.21.15.12:0/1507712327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 msgr2=0x55e05ad32110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83af7fe700 1 --2- 172.21.15.12:0/1507712327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad32110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83af7fe700 1 -- 172.21.15.12:0/1507712327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 msgr2=0x55e05ad1bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83af7fe700 1 --2- 172.21.15.12:0/1507712327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 0x55e05ad1bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83af7fe700 1 -- 172.21.15.12:0/1507712327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e05ad21f40 con 0x55e05ad27720 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83aeffd700 1 --2- 172.21.15.12:0/1507712327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 0x55e05ad1bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83af7fe700 1 --2- 172.21.15.12:0/1507712327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad32650 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f83a80096d0 tx=0x7f83a80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c78e3ac77746a67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83a5ffb700 1 -- 172.21.15.12:0/1507712327 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8017070 con 0x55e05ad27720 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83a5ffb700 1 -- 172.21.15.12:0/1507712327 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a8003c00 con 0x55e05ad27720 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.727+0000 7f83a5ffb700 1 -- 172.21.15.12:0/1507712327 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8010c60 con 0x55e05ad27720 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 -- 172.21.15.12:0/1507712327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 msgr2=0x55e05ad32650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/1507712327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad32650 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f83a80096d0 tx=0x7f83a80099e0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 -- 172.21.15.12:0/1507712327 shutdown_connections 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/1507712327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad27720 0x55e05ad32650 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/1507712327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad21890 0x55e05ad1bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/1507712327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad32110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 -- 172.21.15.12:0/1507712327 >> 172.21.15.12:0/1507712327 conn(0x55e05ac7f210 msgr2=0x55e05ad36bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 -- 172.21.15.12:0/1507712327 shutdown_connections 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 -- 172.21.15.12:0/1507712327 wait complete. 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 -- start start 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.728+0000 7f83c18c8b80 1 Processor -- start 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad30d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83c18c8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad27720 0x55e05ad31ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ad320e0 con 0x55e05ad21890 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ad32250 con 0x55e05ad27720 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83c18c8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05ad32410 con 0x55e05ad26e00 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83c18c8b80 10 client.0 _ll_register_callbacks cb 0x55e05ac16b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad307d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53968/0 (socket says 172.21.15.12:53968) 2024-05-08T23:55:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83aeffd700 1 -- 172.21.15.12:0/662712932 learned_addr learned my addr 172.21.15.12:0/662712932 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83a67fc700 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad30d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83af7fe700 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad27720 0x55e05ad31ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83aeffd700 1 -- 172.21.15.12:0/662712932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 msgr2=0x55e05ad30d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83aeffd700 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad30d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83aeffd700 1 -- 172.21.15.12:0/662712932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad27720 msgr2=0x55e05ad31ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83aeffd700 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad27720 0x55e05ad31ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83aeffd700 1 -- 172.21.15.12:0/662712932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a80043d0 con 0x55e05ad21890 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83a67fc700 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad30d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83aeffd700 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad307d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f83a0099790 tx=0x7f83a00b9360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac3c55c6a95bf58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.729+0000 7f83967fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83967fc700 1 -- 172.21.15.12:0/662712932 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a009d630 con 0x55e05ad21890 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83967fc700 1 -- 172.21.15.12:0/662712932 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f83a00a1c80 con 0x55e05ad21890 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83c18c8b80 1 -- 172.21.15.12:0/662712932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f839801e040 con 0x55e05ad21890 2024-05-08T23:55:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83967fc700 1 -- 172.21.15.12:0/662712932 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83a00a1990 con 0x55e05ad21890 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83c18c8b80 10 client.17871 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83c18c8b80 1 -- 172.21.15.12:0/662712932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e05ac37560 con 0x55e05ad21890 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83967fc700 1 -- 172.21.15.12:0/662712932 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f839801e040 con 0x55e05ad21890 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83967fc700 1 -- 172.21.15.12:0/662712932 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f83a0005000 con 0x55e05ad21890 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83c18c8b80 10 client.17871 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83c18c8b80 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83c18c8b80 20 client.17871 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e05ad92510 0x55e05ad94930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83c18c8b80 4 client.17871 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83c18c8b80 1 -- 172.21.15.12:0/662712932 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e05ac69ec0 con 0x55e05ad92510 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.730+0000 7f83a67fc700 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e05ad92510 0x55e05ad94930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83a67fc700 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e05ad92510 0x55e05ad94930 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50a0fec5b121b4a server_cookie=49bba3ec55207872 in_seq=0 out_seq=0 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83967fc700 10 client.17871 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83967fc700 1 -- 172.21.15.12:0/662712932 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e05ac69ec0 con 0x55e05ad92510 2024-05-08T23:55:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83967fc700 10 client.17871 handle_command_reply: tid=0 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 client.17871 shutdown 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 -- 172.21.15.12:0/662712932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 msgr2=0x55e05ad307d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad307d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f83a0099790 tx=0x7f83a00b9360 comp rx=0 tx=0).stop 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 -- 172.21.15.12:0/662712932 shutdown_connections 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e05ad92510 0x55e05ad94930 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e05ad27720 0x55e05ad31ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e05ad21890 0x55e05ad307d0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83967fc700 0 client.17871 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 --2- 172.21.15.12:0/662712932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e05ad26e00 0x55e05ad30d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 -- 172.21.15.12:0/662712932 >> 172.21.15.12:0/662712932 conn(0x55e05ac7f210 msgr2=0x55e05ad29130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 -- 172.21.15.12:0/662712932 shutdown_connections 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 1 -- 172.21.15.12:0/662712932 wait complete. 2024-05-08T23:55:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.731+0000 7f83c18c8b80 20 client.17871 trim_cache size 0 max 16384 2024-05-08T23:55:18.794 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:55:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:55:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:55:18.794 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:55:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 -- 172.21.15.12:0/2648245925 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f839c0a1290 msgr2=0x7f839c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f839c0a1290 0x7f839c0a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f83b01031d0 tx=0x7f8398005960 comp rx=0 tx=0).stop 2024-05-08T23:55:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 -- 172.21.15.12:0/2648245925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83b0100060 msgr2=0x7f83b0101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83b0100060 0x7f83b0101af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83a0007f10 tx=0x7f83a000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 -- 172.21.15.12:0/2648245925 shutdown_connections 2024-05-08T23:55:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f839c0a1290 0x7f839c0a3750 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83b0102570 0x7f83b01a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83b0100060 0x7f83b0101af0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 --2- 172.21.15.12:0/2648245925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b0105180 0x7f83b0102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 -- 172.21.15.12:0/2648245925 >> 172.21.15.12:0/2648245925 conn(0x7f83b0076fa0 msgr2=0x7f83b0069870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 -- 172.21.15.12:0/2648245925 shutdown_connections 2024-05-08T23:55:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.732+0000 7f83b570e700 1 -- 172.21.15.12:0/2648245925 wait complete. 2024-05-08T23:55:18.796 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:55:18.796 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:55:18.829 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:18 smithi012 ceph-mon[42503]: pgmap v280: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-05-08T23:55:18.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 516816 7.27 MB/sec execute 314 sec latency 983.610 ms 2024-05-08T23:55:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.946+0000 7fb3ffceb700 1 -- 172.21.15.12:0/4285505402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f80fff80 msgr2=0x7fb3f8100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.946+0000 7fb3ffceb700 1 --2- 172.21.15.12:0/4285505402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f80fff80 0x7fb3f8100360 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009990 tx=0x7fb3f4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.946+0000 7fb3ffceb700 1 -- 172.21.15.12:0/4285505402 shutdown_connections 2024-05-08T23:55:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.946+0000 7fb3ffceb700 1 --2- 172.21.15.12:0/4285505402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f81050a0 0x7fb3f810cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.946+0000 7fb3ffceb700 1 --2- 172.21.15.12:0/4285505402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f80fff80 0x7fb3f8100360 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.946+0000 7fb3ffceb700 1 --2- 172.21.15.12:0/4285505402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3f8100930 0x7fb3f8104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.946+0000 7fb3ffceb700 1 -- 172.21.15.12:0/4285505402 >> 172.21.15.12:0/4285505402 conn(0x7fb3f80fb840 msgr2=0x7fb3f80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.946+0000 7fb3ffceb700 1 -- 172.21.15.12:0/4285505402 shutdown_connections 2024-05-08T23:55:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.946+0000 7fb3ffceb700 1 -- 172.21.15.12:0/4285505402 wait complete. 2024-05-08T23:55:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3ffceb700 1 Processor -- start 2024-05-08T23:55:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3ffceb700 1 -- start start 2024-05-08T23:55:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3ffceb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f80fff80 0x7fb3f8195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3ffceb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f81050a0 0x7fb3f81964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3ffceb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3f81969f0 0x7fb3f81a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3ffceb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81a5b30 con 0x7fb3f81050a0 2024-05-08T23:55:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3ffceb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81a5c70 con 0x7fb3f80fff80 2024-05-08T23:55:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3ffceb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81a5de0 con 0x7fb3f81969f0 2024-05-08T23:55:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fd286700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f81050a0 0x7fb3f81964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fd286700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f81050a0 0x7fb3f81964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53988/0 (socket says 172.21.15.12:53988) 2024-05-08T23:55:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fd286700 1 -- 172.21.15.12:0/1733875618 learned_addr learned my addr 172.21.15.12:0/1733875618 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fda87700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f80fff80 0x7fb3f8195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fe288700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3f81969f0 0x7fb3f81a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fd286700 1 -- 172.21.15.12:0/1733875618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3f81969f0 msgr2=0x7fb3f81a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fd286700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3f81969f0 0x7fb3f81a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fd286700 1 -- 172.21.15.12:0/1733875618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f80fff80 msgr2=0x7fb3f8195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fd286700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f80fff80 0x7fb3f8195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fd286700 1 -- 172.21.15.12:0/1733875618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f40095f0 con 0x7fb3f81050a0 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.947+0000 7fb3fda87700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f80fff80 0x7fb3f8195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.948+0000 7fb3fd286700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f81050a0 0x7fb3f81964b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800e970 tx=0x7fb3e800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.948+0000 7fb3eeffd700 1 -- 172.21.15.12:0/1733875618 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8009790 con 0x7fb3f81050a0 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.948+0000 7fb3ffceb700 1 -- 172.21.15.12:0/1733875618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f81a60c0 con 0x7fb3f81050a0 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.948+0000 7fb3eeffd700 1 -- 172.21.15.12:0/1733875618 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e8004cf0 con 0x7fb3f81050a0 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.948+0000 7fb3eeffd700 1 -- 172.21.15.12:0/1733875618 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8005680 con 0x7fb3f81050a0 2024-05-08T23:55:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.948+0000 7fb3ffceb700 1 -- 172.21.15.12:0/1733875618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f81a65d0 con 0x7fb3f81050a0 2024-05-08T23:55:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.949+0000 7fb3ffceb700 1 -- 172.21.15.12:0/1733875618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f8108310 con 0x7fb3f81050a0 2024-05-08T23:55:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.950+0000 7fb3eeffd700 1 -- 172.21.15.12:0/1733875618 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb3e8004750 con 0x7fb3f81050a0 2024-05-08T23:55:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.951+0000 7fb3eeffd700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3e40a1290 0x7fb3e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.951+0000 7fb3fda87700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3e40a1290 0x7fb3e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.951+0000 7fb3eeffd700 1 -- 172.21.15.12:0/1733875618 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb3e8014070 con 0x7fb3f81050a0 2024-05-08T23:55:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.951+0000 7fb3fda87700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3e40a1290 0x7fb3e40a3750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400b2e0 tx=0x7fb3f4005940 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:18.956+0000 7fb3eeffd700 1 -- 172.21.15.12:0/1733875618 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb3e8096440 con 0x7fb3f81050a0 2024-05-08T23:55:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.228+0000 7fb3ffceb700 1 -- 172.21.15.12:0/1733875618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3f8101450 con 0x7fb3f81050a0 2024-05-08T23:55:19.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.229+0000 7fb3eeffd700 1 -- 172.21.15.12:0/1733875618 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+10883 (secure 0 0 0) 0x7fb3e8099a60 con 0x7fb3f81050a0 2024-05-08T23:55:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:55:19.224 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:54:40.565738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27484":{"gid":27484,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3332734891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3332734891},{"type":"v1","addr":"172.21.15.204:6837","nonce":3332734891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:55:19.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 -- 172.21.15.12:0/1733875618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3e40a1290 msgr2=0x7fb3e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3e40a1290 0x7fb3e40a3750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400b2e0 tx=0x7fb3f4005940 comp rx=0 tx=0).stop 2024-05-08T23:55:19.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 -- 172.21.15.12:0/1733875618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f81050a0 msgr2=0x7fb3f81964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f81050a0 0x7fb3f81964b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800e970 tx=0x7fb3e800ed30 comp rx=0 tx=0).stop 2024-05-08T23:55:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 -- 172.21.15.12:0/1733875618 shutdown_connections 2024-05-08T23:55:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3e40a1290 0x7fb3e40a3750 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3f81969f0 0x7fb3f81a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3f80fff80 0x7fb3f8195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 --2- 172.21.15.12:0/1733875618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f81050a0 0x7fb3f81964b0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 -- 172.21.15.12:0/1733875618 >> 172.21.15.12:0/1733875618 conn(0x7fb3f80fb840 msgr2=0x7fb3f80fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 -- 172.21.15.12:0/1733875618 shutdown_connections 2024-05-08T23:55:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.231+0000 7fb3ffceb700 1 -- 172.21.15.12:0/1733875618 wait complete. 2024-05-08T23:55:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 40 2024-05-08T23:55:19.240 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:55:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 -- 172.21.15.12:0/2357708916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde54068ca0 msgr2=0x7fde5410dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 --2- 172.21.15.12:0/2357708916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde54068ca0 0x7fde5410dca0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00b0a0 tx=0x7fde4c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 -- 172.21.15.12:0/2357708916 shutdown_connections 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 --2- 172.21.15.12:0/2357708916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde54068ca0 0x7fde5410dca0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 --2- 172.21.15.12:0/2357708916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54107000 0x7fde541073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 --2- 172.21.15.12:0/2357708916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde540682e0 0x7fde54068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 -- 172.21.15.12:0/2357708916 >> 172.21.15.12:0/2357708916 conn(0x7fde54076f10 msgr2=0x7fde54077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 -- 172.21.15.12:0/2357708916 shutdown_connections 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 -- 172.21.15.12:0/2357708916 wait complete. 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 Processor -- start 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.358+0000 7fde593e1700 1 -- start start 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde593e1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde540682e0 0x7fde540ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde593e1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde54068ca0 0x7fde54100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde593e1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54107000 0x7fde54100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde593e1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde541a5780 con 0x7fde540682e0 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde593e1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde541a58f0 con 0x7fde54068ca0 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde593e1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde541a5ab0 con 0x7fde54107000 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde52ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde540682e0 0x7fde540ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde52ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde540682e0 0x7fde540ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54002/0 (socket says 172.21.15.12:54002) 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde52ffd700 1 -- 172.21.15.12:0/2297784918 learned_addr learned my addr 172.21.15.12:0/2297784918 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde537fe700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54107000 0x7fde54100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde4bfff700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde54068ca0 0x7fde54100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde52ffd700 1 -- 172.21.15.12:0/2297784918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54107000 msgr2=0x7fde54100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde52ffd700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54107000 0x7fde54100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde52ffd700 1 -- 172.21.15.12:0/2297784918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde54068ca0 msgr2=0x7fde54100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde52ffd700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde54068ca0 0x7fde54100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde52ffd700 1 -- 172.21.15.12:0/2297784918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c009d00 con 0x7fde540682e0 2024-05-08T23:55:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde537fe700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54107000 0x7fde54100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.359+0000 7fde52ffd700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde540682e0 0x7fde540ffe10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fde4400e9a0 tx=0x7fde4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.360+0000 7fde50ff9700 1 -- 172.21.15.12:0/2297784918 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde44009790 con 0x7fde540682e0 2024-05-08T23:55:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.360+0000 7fde593e1700 1 -- 172.21.15.12:0/2297784918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde541a5d90 con 0x7fde540682e0 2024-05-08T23:55:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.360+0000 7fde50ff9700 1 -- 172.21.15.12:0/2297784918 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde44010430 con 0x7fde540682e0 2024-05-08T23:55:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.360+0000 7fde50ff9700 1 -- 172.21.15.12:0/2297784918 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde44018680 con 0x7fde540682e0 2024-05-08T23:55:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.360+0000 7fde593e1700 1 -- 172.21.15.12:0/2297784918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde541a6240 con 0x7fde540682e0 2024-05-08T23:55:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.363+0000 7fde6559bb80 1 Processor -- start 2024-05-08T23:55:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.363+0000 7fde6559bb80 1 -- start start 2024-05-08T23:55:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.363+0000 7fde50ff9700 1 -- 172.21.15.12:0/2297784918 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fde440105a0 con 0x7fde540682e0 2024-05-08T23:55:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x559159453d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159454270 0x55915944df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 0x55915944e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde6559bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591593a50f0 con 0x559159453950 2024-05-08T23:55:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde6559bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591593a2ab0 con 0x559159454270 2024-05-08T23:55:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde6559bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915939e3a0 con 0x55915944e480 2024-05-08T23:55:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde52ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159454270 0x55915944df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x559159453d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x559159453d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54004/0 (socket says 172.21.15.12:54004) 2024-05-08T23:55:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde52ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159454270 0x55915944df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34620/0 (socket says 172.21.15.12:34620) 2024-05-08T23:55:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde537fe700 1 -- 172.21.15.12:0/3406687684 learned_addr learned my addr 172.21.15.12:0/3406687684 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.364+0000 7fde4bfff700 1 --2- 172.21.15.12:0/3406687684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 0x55915944e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde50ff9700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde400a1290 0x7fde400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde537fe700 1 -- 172.21.15.12:0/3406687684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 msgr2=0x55915944e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde537fe700 1 --2- 172.21.15.12:0/3406687684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 0x55915944e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde537fe700 1 -- 172.21.15.12:0/3406687684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159454270 msgr2=0x55915944df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde537fe700 1 --2- 172.21.15.12:0/3406687684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159454270 0x55915944df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde537fe700 1 -- 172.21.15.12:0/3406687684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55915921f350 con 0x559159453950 2024-05-08T23:55:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde4bfff700 1 --2- 172.21.15.12:0/3406687684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 0x55915944e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde50ff9700 1 -- 172.21.15.12:0/2297784918 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fde44014070 con 0x7fde540682e0 2024-05-08T23:55:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde537fe700 1 --2- 172.21.15.12:0/3406687684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x559159453d30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009040 tx=0x7fde4c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68e214c00224c6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde537fe700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde400a1290 0x7fde400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde497fa700 1 -- 172.21.15.12:0/3406687684 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00e040 con 0x559159453950 2024-05-08T23:55:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde497fa700 1 -- 172.21.15.12:0/3406687684 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde4c00bdf0 con 0x559159453950 2024-05-08T23:55:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.365+0000 7fde497fa700 1 -- 172.21.15.12:0/3406687684 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c005600 con 0x559159453950 2024-05-08T23:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.366+0000 7fde6559bb80 1 -- 172.21.15.12:0/3406687684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 msgr2=0x559159453d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.366+0000 7fde537fe700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde400a1290 0x7fde400a3750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0061e0 tx=0x7fde4c005920 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.366+0000 7fde6559bb80 1 --2- 172.21.15.12:0/3406687684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x559159453d30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009040 tx=0x7fde4c009b80 comp rx=0 tx=0).stop 2024-05-08T23:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.366+0000 7fde6559bb80 1 -- 172.21.15.12:0/3406687684 shutdown_connections 2024-05-08T23:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.366+0000 7fde6559bb80 1 --2- 172.21.15.12:0/3406687684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 0x55915944e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.366+0000 7fde6559bb80 1 --2- 172.21.15.12:0/3406687684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x559159453d30 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.366+0000 7fde6559bb80 1 --2- 172.21.15.12:0/3406687684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159454270 0x55915944df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.366+0000 7fde6559bb80 1 -- 172.21.15.12:0/3406687684 >> 172.21.15.12:0/3406687684 conn(0x5591593ac1b0 msgr2=0x559159448840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.366+0000 7fde6559bb80 1 -- 172.21.15.12:0/3406687684 shutdown_connections 2024-05-08T23:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.366+0000 7fde6559bb80 1 -- 172.21.15.12:0/3406687684 wait complete. 2024-05-08T23:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde6559bb80 1 -- start start 2024-05-08T23:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde6559bb80 1 Processor -- start 2024-05-08T23:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55915944e480 0x55915945a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159453950 0x559159457570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 0x559159457b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde6559bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915933a500 con 0x55915944e480 2024-05-08T23:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde6559bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915933a1a0 con 0x559159453950 2024-05-08T23:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde6559bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559159221640 con 0x559159454270 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde6559bb80 10 client.0 _ll_register_callbacks cb 0x559159343ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde52ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55915944e480 0x55915945a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde52ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55915944e480 0x55915945a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54012/0 (socket says 172.21.15.12:54012) 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde52ffd700 1 -- 172.21.15.12:0/1616178388 learned_addr learned my addr 172.21.15.12:0/1616178388 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde4bfff700 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159453950 0x559159457570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde537fe700 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 0x559159457b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde52ffd700 1 -- 172.21.15.12:0/1616178388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 msgr2=0x559159457b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde52ffd700 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 0x559159457b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde52ffd700 1 -- 172.21.15.12:0/1616178388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159453950 msgr2=0x559159457570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde52ffd700 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159453950 0x559159457570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde52ffd700 1 -- 172.21.15.12:0/1616178388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c0098e0 con 0x55915944e480 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.367+0000 7fde537fe700 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 0x559159457b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde52ffd700 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55915944e480 0x55915945a250 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fde44000c00 tx=0x7fde440961b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6454bda2dd8aa7a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde3a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde3a7fc700 1 -- 172.21.15.12:0/1616178388 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4409a2d0 con 0x55915944e480 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde3a7fc700 1 -- 172.21.15.12:0/1616178388 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fde4409a570 con 0x55915944e480 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde6559bb80 1 -- 172.21.15.12:0/1616178388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591592f7210 con 0x55915944e480 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde3a7fc700 1 -- 172.21.15.12:0/1616178388 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde440997d0 con 0x55915944e480 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde6559bb80 10 client.17889 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde6559bb80 1 -- 172.21.15.12:0/1616178388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fde5410e630 con 0x55915944e480 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde3a7fc700 1 -- 172.21.15.12:0/1616178388 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5591592f7210 con 0x55915944e480 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde3a7fc700 1 -- 172.21.15.12:0/1616178388 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fde44099360 con 0x55915944e480 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde6559bb80 10 client.17889 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde6559bb80 10 client.17889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde6559bb80 20 client.17889 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.368+0000 7fde6559bb80 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559159467b30 0x559159469f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.369+0000 7fde6559bb80 4 client.17889 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.369+0000 7fde6559bb80 1 -- 172.21.15.12:0/1616178388 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55915941d7b0 con 0x559159467b30 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.369+0000 7fde4bfff700 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559159467b30 0x559159469f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.369+0000 7fde3a7fc700 10 client.17889 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.369+0000 7fde4bfff700 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559159467b30 0x559159469f50 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c10d743db0881c server_cookie=b90a6580cef52549 in_seq=0 out_seq=0 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde3a7fc700 1 -- 172.21.15.12:0/1616178388 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55915941d7b0 con 0x559159467b30 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde3a7fc700 10 client.17889 handle_command_reply: tid=0 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 client.17889 shutdown 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 -- 172.21.15.12:0/1616178388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55915944e480 msgr2=0x55915945a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55915944e480 0x55915945a250 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fde44000c00 tx=0x7fde440961b0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 -- 172.21.15.12:0/1616178388 shutdown_connections 2024-05-08T23:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 0x559159457b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55915944e480 0x55915945a250 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559159467b30 0x559159469f50 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 --2- 172.21.15.12:0/1616178388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159453950 0x559159457570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde3a7fc700 0 client.17889 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 -- 172.21.15.12:0/1616178388 >> 172.21.15.12:0/1616178388 conn(0x5591593ac1b0 msgr2=0x55915944cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 -- 172.21.15.12:0/1616178388 shutdown_connections 2024-05-08T23:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 1 -- 172.21.15.12:0/1616178388 wait complete. 2024-05-08T23:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.370+0000 7fde6559bb80 20 client.17889 trim_cache size 0 max 16384 2024-05-08T23:55:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde6559bb80 1 Processor -- start 2024-05-08T23:55:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde6559bb80 1 -- start start 2024-05-08T23:55:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 0x559159448910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159453950 0x55915944e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159454270 0x559159459d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde6559bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915945a830 con 0x559159454270 2024-05-08T23:55:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde6559bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915944eda0 con 0x559159453950 2024-05-08T23:55:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde6559bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915946f120 con 0x55915944e480 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159454270 0x559159459d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159454270 0x559159459d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54016/0 (socket says 172.21.15.12:54016) 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde537fe700 1 -- 172.21.15.12:0/2315371297 learned_addr learned my addr 172.21.15.12:0/2315371297 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde52ffd700 1 --2- 172.21.15.12:0/2315371297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 0x559159448910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.387+0000 7fde4bfff700 1 --2- 172.21.15.12:0/2315371297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159453950 0x55915944e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.388+0000 7fde537fe700 1 -- 172.21.15.12:0/2315371297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 msgr2=0x559159448910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.388+0000 7fde537fe700 1 --2- 172.21.15.12:0/2315371297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 0x559159448910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.388+0000 7fde537fe700 1 -- 172.21.15.12:0/2315371297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159453950 msgr2=0x55915944e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.388+0000 7fde537fe700 1 --2- 172.21.15.12:0/2315371297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159453950 0x55915944e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.388+0000 7fde537fe700 1 -- 172.21.15.12:0/2315371297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591591cbb00 con 0x559159454270 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.388+0000 7fde537fe700 1 --2- 172.21.15.12:0/2315371297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159454270 0x559159459d80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0057a0 tx=0x7fde4c02d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57410477fe1612c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.388+0000 7fde49ffb700 1 -- 172.21.15.12:0/2315371297 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00e040 con 0x559159454270 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.388+0000 7fde49ffb700 1 -- 172.21.15.12:0/2315371297 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde4c02c040 con 0x559159454270 2024-05-08T23:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.388+0000 7fde49ffb700 1 -- 172.21.15.12:0/2315371297 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c007700 con 0x559159454270 2024-05-08T23:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.389+0000 7fde6559bb80 1 -- 172.21.15.12:0/2315371297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159454270 msgr2=0x559159459d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.389+0000 7fde6559bb80 1 --2- 172.21.15.12:0/2315371297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159454270 0x559159459d80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0057a0 tx=0x7fde4c02d000 comp rx=0 tx=0).stop 2024-05-08T23:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.389+0000 7fde6559bb80 1 -- 172.21.15.12:0/2315371297 shutdown_connections 2024-05-08T23:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.389+0000 7fde6559bb80 1 --2- 172.21.15.12:0/2315371297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159454270 0x559159459d80 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.389+0000 7fde6559bb80 1 --2- 172.21.15.12:0/2315371297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55915944e480 0x559159448910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.389+0000 7fde6559bb80 1 --2- 172.21.15.12:0/2315371297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559159453950 0x55915944e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.389+0000 7fde6559bb80 1 -- 172.21.15.12:0/2315371297 >> 172.21.15.12:0/2315371297 conn(0x5591593ac1b0 msgr2=0x559159455e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.389+0000 7fde6559bb80 1 -- 172.21.15.12:0/2315371297 shutdown_connections 2024-05-08T23:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.389+0000 7fde6559bb80 1 -- 172.21.15.12:0/2315371297 wait complete. 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.389+0000 7fde6559bb80 1 -- start start 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde6559bb80 1 Processor -- start 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55915944e480 0x55915945f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x55915945cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde6559bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 0x55915945d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde6559bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915945da10 con 0x559159453950 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde6559bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591594bcff0 con 0x55915944e480 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde6559bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591594bd130 con 0x559159454270 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde6559bb80 10 client.0 _ll_register_callbacks cb 0x559159343ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x55915945cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x55915945cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54022/0 (socket says 172.21.15.12:54022) 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde4bfff700 1 -- 172.21.15.12:0/772248713 learned_addr learned my addr 172.21.15.12:0/772248713 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde52ffd700 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55915944e480 0x55915945f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde537fe700 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 0x55915945d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde4bfff700 1 -- 172.21.15.12:0/772248713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 msgr2=0x55915945d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde4bfff700 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 0x55915945d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde4bfff700 1 -- 172.21.15.12:0/772248713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55915944e480 msgr2=0x55915945f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde4bfff700 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55915944e480 0x55915945f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.390+0000 7fde4bfff700 1 -- 172.21.15.12:0/772248713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55915921f350 con 0x559159453950 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde4bfff700 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x55915945cbc0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fde54101540 tx=0x7fde3c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b7418b3584b773 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde3a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde3a7fc700 1 -- 172.21.15.12:0/772248713 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c017030 con 0x559159453950 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde3a7fc700 1 -- 172.21.15.12:0/772248713 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fde3c003b00 con 0x559159453950 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde6559bb80 1 -- 172.21.15.12:0/772248713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde4c00bdf0 con 0x559159453950 2024-05-08T23:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde3a7fc700 1 -- 172.21.15.12:0/772248713 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde3c003cc0 con 0x559159453950 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde6559bb80 10 client.17895 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde6559bb80 1 -- 172.21.15.12:0/772248713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559159364500 con 0x559159453950 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde3a7fc700 1 -- 172.21.15.12:0/772248713 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde4c00bdf0 con 0x559159453950 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde3a7fc700 1 -- 172.21.15.12:0/772248713 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fde3c012760 con 0x559159453950 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde6559bb80 10 client.17895 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde6559bb80 10 client.17895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde6559bb80 20 client.17895 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde6559bb80 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591594be950 0x5591594c0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde6559bb80 4 client.17895 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.391+0000 7fde6559bb80 1 -- 172.21.15.12:0/772248713 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559159396e60 con 0x5591594be950 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.392+0000 7fde52ffd700 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591594be950 0x5591594c0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.392+0000 7fde3a7fc700 10 client.17895 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.392+0000 7fde52ffd700 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591594be950 0x5591594c0d70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c57a8182c803b8 server_cookie=53e9bc9bdd15b5c4 in_seq=0 out_seq=0 2024-05-08T23:55:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.440+0000 7fde3a7fc700 1 -- 172.21.15.12:0/772248713 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559159396e60 con 0x5591594be950 2024-05-08T23:55:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.440+0000 7fde3a7fc700 10 client.17895 handle_command_reply: tid=0 2024-05-08T23:55:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.440+0000 7fde6559bb80 1 client.17895 shutdown 2024-05-08T23:55:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.441+0000 7fde6559bb80 1 -- 172.21.15.12:0/772248713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 msgr2=0x55915945cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.441+0000 7fde6559bb80 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x55915945cbc0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fde54101540 tx=0x7fde3c0043b0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.442+0000 7f5a0c27c700 1 -- 172.21.15.12:0/2429349122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a04069e90 msgr2=0x7f5a04105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.442+0000 7f5a0c27c700 1 --2- 172.21.15.12:0/2429349122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a04069e90 0x7f5a04105b60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009990 tx=0x7f5a00009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.442+0000 7f5a0c27c700 1 -- 172.21.15.12:0/2429349122 shutdown_connections 2024-05-08T23:55:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.442+0000 7f5a0c27c700 1 --2- 172.21.15.12:0/2429349122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a04106290 0x7f5a0410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.442+0000 7f5a0c27c700 1 --2- 172.21.15.12:0/2429349122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a040694e0 0x7f5a040698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.442+0000 7f5a0c27c700 1 --2- 172.21.15.12:0/2429349122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a04069e90 0x7f5a04105b60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.442+0000 7f5a0c27c700 1 -- 172.21.15.12:0/2429349122 >> 172.21.15.12:0/2429349122 conn(0x7f5a04076f30 msgr2=0x7f5a04077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.443+0000 7f5a0c27c700 1 -- 172.21.15.12:0/2429349122 shutdown_connections 2024-05-08T23:55:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.443+0000 7f5a0c27c700 1 -- 172.21.15.12:0/2429349122 wait complete. 2024-05-08T23:55:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.443+0000 7f5a0c27c700 1 Processor -- start 2024-05-08T23:55:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.443+0000 7f5a0c27c700 1 -- start start 2024-05-08T23:55:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.443+0000 7fde6559bb80 1 -- 172.21.15.12:0/772248713 shutdown_connections 2024-05-08T23:55:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.443+0000 7fde6559bb80 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559159454270 0x55915945d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.443+0000 7fde6559bb80 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591594be950 0x5591594c0d70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.443+0000 7fde6559bb80 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55915944e480 0x55915945f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.443+0000 7fde6559bb80 1 --2- 172.21.15.12:0/772248713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559159453950 0x55915945cbc0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.443+0000 7fde6559bb80 1 -- 172.21.15.12:0/772248713 >> 172.21.15.12:0/772248713 conn(0x5591593ac1b0 msgr2=0x5591594513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.443+0000 7fde3a7fc700 0 client.17895 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.444+0000 7fde6559bb80 1 -- 172.21.15.12:0/772248713 shutdown_connections 2024-05-08T23:55:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.444+0000 7fde6559bb80 1 -- 172.21.15.12:0/772248713 wait complete. 2024-05-08T23:55:19.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0c27c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a040694e0 0x7f5a04109950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.444+0000 7fde6559bb80 20 client.17895 trim_cache size 0 max 16384 2024-05-08T23:55:19.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0c27c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a04106290 0x7f5a04109ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0c27c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a0410d2b0 0x7f5a0410d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0c27c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0410dd30 con 0x7f5a0410d2b0 2024-05-08T23:55:19.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0c27c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0410dea0 con 0x7f5a04106290 2024-05-08T23:55:19.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0c27c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0410e060 con 0x7f5a040694e0 2024-05-08T23:55:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:55:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:55:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:55:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:55:19.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0a018700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a040694e0 0x7f5a04109950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0a819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a0410d2b0 0x7f5a0410d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0a018700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a040694e0 0x7f5a04109950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:36872/0 (socket says 172.21.15.12:36872) 2024-05-08T23:55:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0a018700 1 -- 172.21.15.12:0/1019102494 learned_addr learned my addr 172.21.15.12:0/1019102494 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0a819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a0410d2b0 0x7f5a0410d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54050/0 (socket says 172.21.15.12:54050) 2024-05-08T23:55:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a09817700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a04106290 0x7f5a04109ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0a819700 1 -- 172.21.15.12:0/1019102494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a040694e0 msgr2=0x7f5a04109950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0a819700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a040694e0 0x7f5a04109950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0a819700 1 -- 172.21.15.12:0/1019102494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a04106290 msgr2=0x7f5a04109ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0a819700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a04106290 0x7f5a04109ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.444+0000 7f5a0a819700 1 -- 172.21.15.12:0/1019102494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc009c40 con 0x7f5a0410d2b0 2024-05-08T23:55:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.444+0000 7fde593e1700 1 -- 172.21.15.12:0/2297784918 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde400a1290 msgr2=0x7fde400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.444+0000 7fde593e1700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde400a1290 0x7fde400a3750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0061e0 tx=0x7fde4c005920 comp rx=0 tx=0).stop 2024-05-08T23:55:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.444+0000 7fde593e1700 1 -- 172.21.15.12:0/2297784918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde540682e0 msgr2=0x7fde540ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.444+0000 7fde593e1700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde540682e0 0x7fde540ffe10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fde4400e9a0 tx=0x7fde4400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.445+0000 7fde593e1700 1 -- 172.21.15.12:0/2297784918 shutdown_connections 2024-05-08T23:55:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.445+0000 7fde593e1700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde400a1290 0x7fde400a3750 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.445+0000 7fde593e1700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde54107000 0x7fde54100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.445+0000 7fde593e1700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde540682e0 0x7fde540ffe10 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.445+0000 7fde593e1700 1 --2- 172.21.15.12:0/2297784918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde54068ca0 0x7fde54100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.445+0000 7fde593e1700 1 -- 172.21.15.12:0/2297784918 >> 172.21.15.12:0/2297784918 conn(0x7fde54076f10 msgr2=0x7fde540feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.445+0000 7f5a0a819700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a0410d2b0 0x7f5a0410d760 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00efb0 tx=0x7f59fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.445+0000 7fde593e1700 1 -- 172.21.15.12:0/2297784918 shutdown_connections 2024-05-08T23:55:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:19.445+0000 7fde593e1700 1 -- 172.21.15.12:0/2297784918 wait complete. 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.445+0000 7f59fb7fe700 1 -- 172.21.15.12:0/1019102494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc003a00 con 0x7f5a0410d2b0 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.445+0000 7f59fb7fe700 1 -- 172.21.15.12:0/1019102494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59fc00f040 con 0x7f5a0410d2b0 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.445+0000 7f5a0c27c700 1 -- 172.21.15.12:0/1019102494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a000095f0 con 0x7f5a0410d2b0 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.445+0000 7f59fb7fe700 1 -- 172.21.15.12:0/1019102494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc004540 con 0x7f5a0410d2b0 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.445+0000 7f5a0c27c700 1 -- 172.21.15.12:0/1019102494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a041bdb50 con 0x7f5a0410d2b0 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.448+0000 7f59fb7fe700 1 -- 172.21.15.12:0/1019102494 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f59fc003bf0 con 0x7f5a0410d2b0 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.449+0000 7f5a18436b80 1 Processor -- start 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.449+0000 7f5a18436b80 1 -- start start 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.449+0000 7f59fb7fe700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59f00a1280 0x7f59f00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 0x55c8113c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c4720 0x55c8113be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a18436b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c811315150 con 0x55c8113be890 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a18436b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c811312b10 con 0x55c8113c3e00 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a18436b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81130e400 con 0x55c8113c4720 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a09817700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59f00a1280 0x7f59f00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f59fb7fe700 1 -- 172.21.15.12:0/1019102494 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f59fc0cc8e0 con 0x7f5a0410d2b0 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a018700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 0x55c8113c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113becd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54062/0 (socket says 172.21.15.12:54062) 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a018700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 0x55c8113c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34690/0 (socket says 172.21.15.12:34690) 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a819700 1 -- 172.21.15.12:0/3754795152 learned_addr learned my addr 172.21.15.12:0/3754795152 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a09817700 1 --2- 172.21.15.12:0/3754795152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c4720 0x55c8113be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a018700 1 -- 172.21.15.12:0/3754795152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c4720 msgr2=0x55c8113be350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a018700 1 --2- 172.21.15.12:0/3754795152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c4720 0x55c8113be350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a018700 1 -- 172.21.15.12:0/3754795152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 msgr2=0x55c8113becd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a018700 1 --2- 172.21.15.12:0/3754795152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113becd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a018700 1 -- 172.21.15.12:0/3754795152 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c81118f3b0 con 0x55c8113c3e00 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a0a819700 1 --2- 172.21.15.12:0/3754795152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113becd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.450+0000 7f5a09817700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59f00a1280 0x7f59f00a3740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5a00006010 tx=0x7f5a000053b0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.451+0000 7f5a0a018700 1 --2- 172.21.15.12:0/3754795152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 0x55c8113c41e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009d10 tx=0x7f59f4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=880e5dacae18b077 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.451+0000 7f59f8da4700 1 -- 172.21.15.12:0/3754795152 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4004250 con 0x55c8113c3e00 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.451+0000 7f59f8da4700 1 -- 172.21.15.12:0/3754795152 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f4004cf0 con 0x55c8113c3e00 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.451+0000 7f59f8da4700 1 -- 172.21.15.12:0/3754795152 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f40055c0 con 0x55c8113c3e00 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 -- 172.21.15.12:0/3754795152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 msgr2=0x55c8113c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3754795152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 0x55c8113c41e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009d10 tx=0x7f59f4011720 comp rx=0 tx=0).stop 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 -- 172.21.15.12:0/3754795152 shutdown_connections 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3754795152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113becd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3754795152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 0x55c8113c41e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3754795152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c4720 0x55c8113be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 -- 172.21.15.12:0/3754795152 >> 172.21.15.12:0/3754795152 conn(0x55c81131c210 msgr2=0x55c8113b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 -- 172.21.15.12:0/3754795152 shutdown_connections 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 -- 172.21.15.12:0/3754795152 wait complete. 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 -- start start 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.452+0000 7f5a18436b80 1 Processor -- start 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113be890 0x55c8113c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 0x55c8113c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c4720 0x55c8113c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a18436b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8112aa560 con 0x55c8113c4720 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a18436b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8112aa200 con 0x55c8113c3e00 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a18436b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8111916a0 con 0x55c8113be890 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a18436b80 10 client.0 _ll_register_callbacks cb 0x55c8112b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:19.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a0a819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c4720 0x55c8113c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a0a018700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113be890 0x55c8113c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a0a819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c4720 0x55c8113c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54068/0 (socket says 172.21.15.12:54068) 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a0a819700 1 -- 172.21.15.12:0/3395911042 learned_addr learned my addr 172.21.15.12:0/3395911042 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a09817700 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 0x55c8113c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a0a018700 1 -- 172.21.15.12:0/3395911042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 msgr2=0x55c8113c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a0a018700 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 0x55c8113c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a0a018700 1 -- 172.21.15.12:0/3395911042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c4720 msgr2=0x55c8113c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a0a018700 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c4720 0x55c8113c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a0a018700 1 -- 172.21.15.12:0/3395911042 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a041bdb50 con 0x55c8113be890 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f5a0a018700 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113be890 0x55c8113c7b30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f59f4000c00 tx=0x7f59f4004c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9b49ffd1c18ca0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.453+0000 7f59edffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:55:19.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f59edffb700 1 -- 172.21.15.12:0/3395911042 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4005410 con 0x55c8113be890 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f59edffb700 1 -- 172.21.15.12:0/3395911042 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f59f4005570 con 0x55c8113be890 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f5a18436b80 1 -- 172.21.15.12:0/3395911042 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c811267270 con 0x55c8113be890 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f59edffb700 1 -- 172.21.15.12:0/3395911042 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59f4004050 con 0x55c8113be890 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f5a18436b80 10 client.27679 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f5a18436b80 1 -- 172.21.15.12:0/3395911042 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f59f4009970 con 0x55c8113be890 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f59edffb700 1 -- 172.21.15.12:0/3395911042 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c811267270 con 0x55c8113be890 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f59edffb700 1 -- 172.21.15.12:0/3395911042 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f59f4003bf0 con 0x55c8113be890 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f5a18436b80 10 client.27679 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f5a18436b80 10 client.27679 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f5a18436b80 20 client.27679 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c8113dc520 0x55c8113de940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f5a18436b80 4 client.27679 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.454+0000 7f5a18436b80 1 -- 172.21.15.12:0/3395911042 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c81138d810 con 0x55c8113dc520 2024-05-08T23:55:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.455+0000 7f5a09817700 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c8113dc520 0x55c8113de940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.455+0000 7f59edffb700 10 client.27679 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.455+0000 7f5a09817700 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c8113dc520 0x55c8113de940 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a924296065c6b74 server_cookie=53c5fab906888101 in_seq=0 out_seq=0 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f59edffb700 1 -- 172.21.15.12:0/3395911042 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c81138d810 con 0x55c8113dc520 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f59edffb700 10 client.27679 handle_command_reply: tid=0 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f5a18436b80 1 client.27679 shutdown 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f5a18436b80 1 -- 172.21.15.12:0/3395911042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113be890 msgr2=0x55c8113c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113be890 0x55c8113c7b30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f59f4000c00 tx=0x7f59f4004c30 comp rx=0 tx=0).stop 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f5a18436b80 1 -- 172.21.15.12:0/3395911042 shutdown_connections 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c4720 0x55c8113c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c8113dc520 0x55c8113de940 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113be890 0x55c8113c7b30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3395911042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c3e00 0x55c8113c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f59edffb700 0 client.27679 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:55:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.456+0000 7f5a18436b80 1 -- 172.21.15.12:0/3395911042 >> 172.21.15.12:0/3395911042 conn(0x55c81131c210 msgr2=0x55c8113b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.457+0000 7f5a18436b80 1 -- 172.21.15.12:0/3395911042 shutdown_connections 2024-05-08T23:55:19.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.457+0000 7f5a18436b80 1 -- 172.21.15.12:0/3395911042 wait complete. 2024-05-08T23:55:19.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.457+0000 7f5a18436b80 20 client.27679 trim_cache size 0 max 16384 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a18436b80 1 Processor -- start 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a18436b80 1 -- start start 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113d8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c3e00 0x55c8113b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c4720 0x55c8113c3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a18436b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8113e3b50 con 0x55c8113be890 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a18436b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8113b3800 con 0x55c8113c4720 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a18436b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8113be690 con 0x55c8113c3e00 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a0a018700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113d8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a0a018700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113d8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54076/0 (socket says 172.21.15.12:54076) 2024-05-08T23:55:19.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a0a018700 1 -- 172.21.15.12:0/3703169058 learned_addr learned my addr 172.21.15.12:0/3703169058 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a09817700 1 --2- 172.21.15.12:0/3703169058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c3e00 0x55c8113b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.471+0000 7f5a0a819700 1 --2- 172.21.15.12:0/3703169058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c4720 0x55c8113c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.472+0000 7f5a0a018700 1 -- 172.21.15.12:0/3703169058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c3e00 msgr2=0x55c8113b8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.472+0000 7f5a0a018700 1 --2- 172.21.15.12:0/3703169058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c3e00 0x55c8113b8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.472+0000 7f5a0a018700 1 -- 172.21.15.12:0/3703169058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c4720 msgr2=0x55c8113c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.472+0000 7f5a0a018700 1 --2- 172.21.15.12:0/3703169058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c4720 0x55c8113c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.472+0000 7f5a0a018700 1 -- 172.21.15.12:0/3703169058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c81113bb60 con 0x55c8113be890 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.472+0000 7f5a0a018700 1 --2- 172.21.15.12:0/3703169058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113d8030 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009800 tx=0x7f59f4009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01e7298fd742d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.472+0000 7f59f95a5700 1 -- 172.21.15.12:0/3703169058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4022610 con 0x55c8113be890 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.472+0000 7f59f95a5700 1 -- 172.21.15.12:0/3703169058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f4022d10 con 0x55c8113be890 2024-05-08T23:55:19.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.472+0000 7f59f95a5700 1 -- 172.21.15.12:0/3703169058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4005670 con 0x55c8113be890 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 -- 172.21.15.12:0/3703169058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 msgr2=0x55c8113d8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3703169058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113d8030 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009800 tx=0x7f59f4009f20 comp rx=0 tx=0).stop 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 -- 172.21.15.12:0/3703169058 shutdown_connections 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3703169058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113c4720 0x55c8113c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3703169058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113be890 0x55c8113d8030 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 --2- 172.21.15.12:0/3703169058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c3e00 0x55c8113b8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 -- 172.21.15.12:0/3703169058 >> 172.21.15.12:0/3703169058 conn(0x55c81131c210 msgr2=0x55c8113c2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 -- 172.21.15.12:0/3703169058 shutdown_connections 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 -- 172.21.15.12:0/3703169058 wait complete. 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 -- start start 2024-05-08T23:55:19.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 Processor -- start 2024-05-08T23:55:19.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113be890 0x55c8113d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c3e00 0x55c8113d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c4720 0x55c8113d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c811431a20 con 0x55c8113c3e00 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c811431b60 con 0x55c8113be890 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.473+0000 7f5a18436b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c811431ca0 con 0x55c8113c4720 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a18436b80 10 client.0 _ll_register_callbacks cb 0x55c8112b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a09817700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c3e00 0x55c8113d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a0a018700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113be890 0x55c8113d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a09817700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c3e00 0x55c8113d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54078/0 (socket says 172.21.15.12:54078) 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a09817700 1 -- 172.21.15.12:0/408765782 learned_addr learned my addr 172.21.15.12:0/408765782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a0a819700 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c4720 0x55c8113d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a09817700 1 -- 172.21.15.12:0/408765782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c4720 msgr2=0x55c8113d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a09817700 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c4720 0x55c8113d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a09817700 1 -- 172.21.15.12:0/408765782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113be890 msgr2=0x55c8113d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a09817700 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113be890 0x55c8113d1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a09817700 1 -- 172.21.15.12:0/408765782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f4009970 con 0x55c8113c3e00 2024-05-08T23:55:19.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a0a018700 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113be890 0x55c8113d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:19.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a09817700 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c3e00 0x55c8113d15c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5a00015000 tx=0x7f5a00009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4399acf7adeff1fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f59edffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:19.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f59edffb700 1 -- 172.21.15.12:0/408765782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0001d040 con 0x55c8113c3e00 2024-05-08T23:55:19.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f59edffb700 1 -- 172.21.15.12:0/408765782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5a00035450 con 0x55c8113c3e00 2024-05-08T23:55:19.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f5a18436b80 1 -- 172.21.15.12:0/408765782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59f4004cf0 con 0x55c8113c3e00 2024-05-08T23:55:19.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.474+0000 7f59edffb700 1 -- 172.21.15.12:0/408765782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a00035610 con 0x55c8113c3e00 2024-05-08T23:55:19.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f5a18436b80 10 client.17913 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f5a18436b80 1 -- 172.21.15.12:0/408765782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c8112d4560 con 0x55c8113c3e00 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f59edffb700 1 -- 172.21.15.12:0/408765782 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f4004cf0 con 0x55c8113c3e00 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f59edffb700 1 -- 172.21.15.12:0/408765782 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5a00004cc0 con 0x55c8113c3e00 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f5a18436b80 10 client.17913 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f5a18436b80 10 client.17913 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f5a18436b80 20 client.17913 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f5a18436b80 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c8114332f0 0x55c811435710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f5a18436b80 4 client.17913 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f5a18436b80 1 -- 172.21.15.12:0/408765782 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c811306ec0 con 0x55c8114332f0 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.475+0000 7f5a0a018700 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c8114332f0 0x55c811435710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a0a018700 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c8114332f0 0x55c811435710 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc3dd5faae626be6 server_cookie=2240debb18bab97d in_seq=0 out_seq=0 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f59edffb700 10 client.17913 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f59edffb700 1 -- 172.21.15.12:0/408765782 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c811306ec0 con 0x55c8114332f0 2024-05-08T23:55:19.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f59edffb700 10 client.17913 handle_command_reply: tid=0 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 client.17913 shutdown 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 -- 172.21.15.12:0/408765782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c3e00 msgr2=0x55c8113d15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c3e00 0x55c8113d15c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5a00015000 tx=0x7f5a00009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 -- 172.21.15.12:0/408765782 shutdown_connections 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c8114332f0 0x55c811435710 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c8113c4720 0x55c8113d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c8113be890 0x55c8113d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f59edffb700 0 client.17913 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 --2- 172.21.15.12:0/408765782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8113c3e00 0x55c8113d15c0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 -- 172.21.15.12:0/408765782 >> 172.21.15.12:0/408765782 conn(0x55c81131c210 msgr2=0x55c8113c8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 -- 172.21.15.12:0/408765782 shutdown_connections 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 1 -- 172.21.15.12:0/408765782 wait complete. 2024-05-08T23:55:19.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.476+0000 7f5a18436b80 20 client.17913 trim_cache size 0 max 16384 2024-05-08T23:55:19.479 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:55:19.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 -- 172.21.15.12:0/1019102494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59f00a1280 msgr2=0x7f59f00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59f00a1280 0x7f59f00a3740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5a00006010 tx=0x7f5a000053b0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 -- 172.21.15.12:0/1019102494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a0410d2b0 msgr2=0x7f5a0410d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a0410d2b0 0x7f5a0410d760 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00efb0 tx=0x7f59fc00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 -- 172.21.15.12:0/1019102494 shutdown_connections 2024-05-08T23:55:19.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59f00a1280 0x7f59f00a3740 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a0410d2b0 0x7f5a0410d760 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a040694e0 0x7f5a04109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 --2- 172.21.15.12:0/1019102494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a04106290 0x7f5a04109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 -- 172.21.15.12:0/1019102494 >> 172.21.15.12:0/1019102494 conn(0x7f5a04076f30 msgr2=0x7f5a040fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 -- 172.21.15.12:0/1019102494 shutdown_connections 2024-05-08T23:55:19.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.477+0000 7f5a0c27c700 1 -- 172.21.15.12:0/1019102494 wait complete. 2024-05-08T23:55:19.485 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:55:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.632+0000 7f4151204700 1 -- 172.21.15.12:0/2933468407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f414c101140 msgr2=0x7f414c104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.632+0000 7f4151204700 1 --2- 172.21.15.12:0/2933468407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f414c101140 0x7f414c104c60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4134009990 tx=0x7f4134009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.632+0000 7f4151204700 1 -- 172.21.15.12:0/2933468407 shutdown_connections 2024-05-08T23:55:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.632+0000 7f4151204700 1 --2- 172.21.15.12:0/2933468407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414c105390 0x7f414c10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.632+0000 7f4151204700 1 --2- 172.21.15.12:0/2933468407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f414c100790 0x7f414c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.632+0000 7f4151204700 1 --2- 172.21.15.12:0/2933468407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f414c101140 0x7f414c104c60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.632+0000 7f4151204700 1 -- 172.21.15.12:0/2933468407 >> 172.21.15.12:0/2933468407 conn(0x7f414c0763a0 msgr2=0x7f414c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.632+0000 7f4151204700 1 -- 172.21.15.12:0/2933468407 shutdown_connections 2024-05-08T23:55:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.632+0000 7f4151204700 1 -- 172.21.15.12:0/2933468407 wait complete. 2024-05-08T23:55:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.633+0000 7f4151204700 1 Processor -- start 2024-05-08T23:55:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.633+0000 7f4151204700 1 -- start start 2024-05-08T23:55:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.633+0000 7f4151204700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f414c100790 0x7f414c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.633+0000 7f4151204700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414c105390 0x7f414c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.633+0000 7f4151204700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f414c198f00 0x7f414c1a3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.633+0000 7f4151204700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c1a3ba0 con 0x7f414c100790 2024-05-08T23:55:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.633+0000 7f4151204700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c1a3d10 con 0x7f414c198f00 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.633+0000 7f4151204700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c1a3ed0 con 0x7f414c105390 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f414c100790 0x7f414c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f414c100790 0x7f414c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54094/0 (socket says 172.21.15.12:54094) 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414ad9d700 1 -- 172.21.15.12:0/1092054109 learned_addr learned my addr 172.21.15.12:0/1092054109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414a59c700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414c105390 0x7f414c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414b59e700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f414c198f00 0x7f414c1a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414ad9d700 1 -- 172.21.15.12:0/1092054109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414c105390 msgr2=0x7f414c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414ad9d700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414c105390 0x7f414c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414ad9d700 1 -- 172.21.15.12:0/1092054109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f414c198f00 msgr2=0x7f414c1a3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414ad9d700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f414c198f00 0x7f414c1a3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414ad9d700 1 -- 172.21.15.12:0/1092054109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41340095f0 con 0x7f414c100790 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414b59e700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f414c198f00 0x7f414c1a3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:55:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414a59c700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414c105390 0x7f414c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:55:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f414ad9d700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f414c100790 0x7f414c198480 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f413c00d980 tx=0x7f413c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f4143fff700 1 -- 172.21.15.12:0/1092054109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c00bb60 con 0x7f414c100790 2024-05-08T23:55:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f4151204700 1 -- 172.21.15.12:0/1092054109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414c1a41b0 con 0x7f414c100790 2024-05-08T23:55:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f4143fff700 1 -- 172.21.15.12:0/1092054109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f413c007640 con 0x7f414c100790 2024-05-08T23:55:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f4143fff700 1 -- 172.21.15.12:0/1092054109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c004e90 con 0x7f414c100790 2024-05-08T23:55:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.634+0000 7f4151204700 1 -- 172.21.15.12:0/1092054109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c1a4710 con 0x7f414c100790 2024-05-08T23:55:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.638+0000 7f4143fff700 1 -- 172.21.15.12:0/1092054109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f413c005020 con 0x7f414c100790 2024-05-08T23:55:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.638+0000 7f4143fff700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41380a1290 0x7f41380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.638+0000 7f414a59c700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41380a1290 0x7f41380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.638+0000 7f4143fff700 1 -- 172.21.15.12:0/1092054109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f413c0ccc70 con 0x7f414c100790 2024-05-08T23:55:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f415d3beb80 1 Processor -- start 2024-05-08T23:55:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f415d3beb80 1 -- start start 2024-05-08T23:55:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f414a59c700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41380a1290 0x7f41380a3750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4134000c00 tx=0x7f4134005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a47720 0x55e8d4a41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a41cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f415d3beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d4998150 con 0x55e8d4a46e00 2024-05-08T23:55:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f415d3beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d4995b10 con 0x55e8d4a41890 2024-05-08T23:55:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f415d3beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d4991400 con 0x55e8d4a47720 2024-05-08T23:55:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f414ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f414ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54096/0 (socket says 172.21.15.12:54096) 2024-05-08T23:55:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.639+0000 7f414ad9d700 1 -- 172.21.15.12:0/2086099098 learned_addr learned my addr 172.21.15.12:0/2086099098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f414b59e700 1 --2- 172.21.15.12:0/2086099098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a41cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f414a59c700 1 --2- 172.21.15.12:0/2086099098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a47720 0x55e8d4a41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f414ad9d700 1 -- 172.21.15.12:0/2086099098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a47720 msgr2=0x55e8d4a41350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f414ad9d700 1 --2- 172.21.15.12:0/2086099098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a47720 0x55e8d4a41350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f414ad9d700 1 -- 172.21.15.12:0/2086099098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 msgr2=0x55e8d4a41cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f414ad9d700 1 --2- 172.21.15.12:0/2086099098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a41cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f414ad9d700 1 -- 172.21.15.12:0/2086099098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d48123b0 con 0x55e8d4a46e00 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f414ad9d700 1 --2- 172.21.15.12:0/2086099098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a471e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f413c0967a0 tx=0x7f413c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68131d9b5dae92cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f41417fa700 1 -- 172.21.15.12:0/2086099098 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c09db60 con 0x55e8d4a46e00 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f41417fa700 1 -- 172.21.15.12:0/2086099098 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f413c0a21b0 con 0x55e8d4a46e00 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.640+0000 7f41417fa700 1 -- 172.21.15.12:0/2086099098 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c099220 con 0x55e8d4a46e00 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.641+0000 7f415d3beb80 1 -- 172.21.15.12:0/2086099098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 msgr2=0x55e8d4a471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.641+0000 7f415d3beb80 1 --2- 172.21.15.12:0/2086099098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a471e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f413c0967a0 tx=0x7f413c0048a0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.641+0000 7f415d3beb80 1 -- 172.21.15.12:0/2086099098 shutdown_connections 2024-05-08T23:55:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.641+0000 7f415d3beb80 1 --2- 172.21.15.12:0/2086099098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a41cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.641+0000 7f415d3beb80 1 --2- 172.21.15.12:0/2086099098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a471e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.641+0000 7f415d3beb80 1 --2- 172.21.15.12:0/2086099098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a47720 0x55e8d4a41350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.641+0000 7f415d3beb80 1 -- 172.21.15.12:0/2086099098 >> 172.21.15.12:0/2086099098 conn(0x55e8d499f210 msgr2=0x55e8d4a3bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.641+0000 7f415d3beb80 1 -- 172.21.15.12:0/2086099098 shutdown_connections 2024-05-08T23:55:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.641+0000 7f415d3beb80 1 -- 172.21.15.12:0/2086099098 wait complete. 2024-05-08T23:55:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.641+0000 7f415d3beb80 1 -- start start 2024-05-08T23:55:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f415d3beb80 1 Processor -- start 2024-05-08T23:55:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a4aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a46e00 0x55e8d4a4b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4a4b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f415d3beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d492d560 con 0x55e8d4a47720 2024-05-08T23:55:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f415d3beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d492d200 con 0x55e8d4a41890 2024-05-08T23:55:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f415d3beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d48146a0 con 0x55e8d4a46e00 2024-05-08T23:55:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f415d3beb80 10 client.0 _ll_register_callbacks cb 0x55e8d4936b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414b59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4a4b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414ad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a4aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414ad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a4aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34762/0 (socket says 172.21.15.12:34762) 2024-05-08T23:55:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414ad9d700 1 -- 172.21.15.12:0/1639388278 learned_addr learned my addr 172.21.15.12:0/1639388278 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414b59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4a4b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54098/0 (socket says 172.21.15.12:54098) 2024-05-08T23:55:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414a59c700 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a46e00 0x55e8d4a4b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414b59e700 1 -- 172.21.15.12:0/1639388278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a46e00 msgr2=0x55e8d4a4b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414b59e700 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a46e00 0x55e8d4a4b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414b59e700 1 -- 172.21.15.12:0/1639388278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 msgr2=0x55e8d4a4aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414b59e700 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a4aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414b59e700 1 -- 172.21.15.12:0/1639388278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c1a4710 con 0x55e8d4a47720 2024-05-08T23:55:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.642+0000 7f414ad9d700 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a4aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f414b59e700 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4a4b550 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4144007970 tx=0x7f4144007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bddeaeefd609119 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f41327fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f41327fc700 1 -- 172.21.15.12:0/1639388278 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440155f0 con 0x55e8d4a47720 2024-05-08T23:55:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f41327fc700 1 -- 172.21.15.12:0/1639388278 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4144015750 con 0x55e8d4a47720 2024-05-08T23:55:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f415d3beb80 1 -- 172.21.15.12:0/1639388278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8d48ea270 con 0x55e8d4a47720 2024-05-08T23:55:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f41327fc700 1 -- 172.21.15.12:0/1639388278 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4144015910 con 0x55e8d4a47720 2024-05-08T23:55:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f415d3beb80 10 client.17925 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f415d3beb80 1 -- 172.21.15.12:0/1639388278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e8d47beb60 con 0x55e8d4a47720 2024-05-08T23:55:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f41327fc700 1 -- 172.21.15.12:0/1639388278 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8d48ea270 con 0x55e8d4a47720 2024-05-08T23:55:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f41327fc700 1 -- 172.21.15.12:0/1639388278 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f4144016c10 con 0x55e8d4a47720 2024-05-08T23:55:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f415d3beb80 10 client.17925 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f415d3beb80 10 client.17925 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:55:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f415d3beb80 20 client.17925 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f415d3beb80 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e8d4a5f640 0x55e8d4a61a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f415d3beb80 4 client.17925 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.643+0000 7f415d3beb80 1 -- 172.21.15.12:0/1639388278 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8d4a10810 con 0x55e8d4a5f640 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.644+0000 7f414ad9d700 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e8d4a5f640 0x55e8d4a61a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.644+0000 7f41327fc700 10 client.17925 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.644+0000 7f414ad9d700 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e8d4a5f640 0x55e8d4a61a60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ae37fe4d03594d7 server_cookie=7bb90e34f5c50179 in_seq=0 out_seq=0 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f41327fc700 1 -- 172.21.15.12:0/1639388278 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8d4a10810 con 0x55e8d4a5f640 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f41327fc700 10 client.17925 handle_command_reply: tid=0 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 client.17925 shutdown 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 -- 172.21.15.12:0/1639388278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 msgr2=0x55e8d4a4b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4a4b550 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4144007970 tx=0x7f4144007c80 comp rx=0 tx=0).stop 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 -- 172.21.15.12:0/1639388278 shutdown_connections 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e8d4a5f640 0x55e8d4a61a60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4a4b550 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a4aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 --2- 172.21.15.12:0/1639388278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a46e00 0x55e8d4a4b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f41327fc700 0 client.17925 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:55:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 -- 172.21.15.12:0/1639388278 >> 172.21.15.12:0/1639388278 conn(0x55e8d499f210 msgr2=0x55e8d4a3b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 -- 172.21.15.12:0/1639388278 shutdown_connections 2024-05-08T23:55:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 1 -- 172.21.15.12:0/1639388278 wait complete. 2024-05-08T23:55:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.645+0000 7f415d3beb80 20 client.17925 trim_cache size 0 max 16384 2024-05-08T23:55:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.659+0000 7f415d3beb80 1 Processor -- start 2024-05-08T23:55:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.659+0000 7f415d3beb80 1 -- start start 2024-05-08T23:55:19.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a3bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a47720 0x55e8d4a45c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f415d3beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d4a10810 con 0x55e8d4a46e00 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f415d3beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d4a47430 con 0x55e8d4a41890 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f415d3beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d4a4a8c0 con 0x55e8d4a47720 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a3bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a3bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54108/0 (socket says 172.21.15.12:54108) 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414a59c700 1 -- 172.21.15.12:0/1480243648 learned_addr learned my addr 172.21.15.12:0/1480243648 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414b59e700 1 --2- 172.21.15.12:0/1480243648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a47720 0x55e8d4a45c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414ad9d700 1 --2- 172.21.15.12:0/1480243648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414a59c700 1 -- 172.21.15.12:0/1480243648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a47720 msgr2=0x55e8d4a45c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414a59c700 1 --2- 172.21.15.12:0/1480243648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a47720 0x55e8d4a45c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414a59c700 1 -- 172.21.15.12:0/1480243648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 msgr2=0x55e8d4a41350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414a59c700 1 --2- 172.21.15.12:0/1480243648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a41350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414a59c700 1 -- 172.21.15.12:0/1480243648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d4a41f40 con 0x55e8d4a46e00 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.660+0000 7f414a59c700 1 --2- 172.21.15.12:0/1480243648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a3bd20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4134009990 tx=0x7f4134021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86512e9ede54eaa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f4141ffb700 1 -- 172.21.15.12:0/1480243648 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413400be10 con 0x55e8d4a46e00 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f4141ffb700 1 -- 172.21.15.12:0/1480243648 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f413402b050 con 0x55e8d4a46e00 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f4141ffb700 1 -- 172.21.15.12:0/1480243648 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41340307b0 con 0x55e8d4a46e00 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f415d3beb80 1 -- 172.21.15.12:0/1480243648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 msgr2=0x55e8d4a3bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f415d3beb80 1 --2- 172.21.15.12:0/1480243648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a3bd20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4134009990 tx=0x7f4134021000 comp rx=0 tx=0).stop 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f415d3beb80 1 -- 172.21.15.12:0/1480243648 shutdown_connections 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f415d3beb80 1 --2- 172.21.15.12:0/1480243648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a47720 0x55e8d4a45c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f415d3beb80 1 --2- 172.21.15.12:0/1480243648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a41890 0x55e8d4a41350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f415d3beb80 1 --2- 172.21.15.12:0/1480243648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a46e00 0x55e8d4a3bd20 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f415d3beb80 1 -- 172.21.15.12:0/1480243648 >> 172.21.15.12:0/1480243648 conn(0x55e8d499f210 msgr2=0x55e8d4a493a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f415d3beb80 1 -- 172.21.15.12:0/1480243648 shutdown_connections 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.661+0000 7f415d3beb80 1 -- 172.21.15.12:0/1480243648 wait complete. 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f415d3beb80 1 -- start start 2024-05-08T23:55:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f415d3beb80 1 Processor -- start 2024-05-08T23:55:19.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a41890 0x55e8d4a54580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a46e00 0x55e8d4a54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f415d3beb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4ab4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f415d3beb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d4a553d0 con 0x55e8d4a47720 2024-05-08T23:55:19.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f415d3beb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d4ab5040 con 0x55e8d4a46e00 2024-05-08T23:55:19.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f415d3beb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d4ab5180 con 0x55e8d4a41890 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f415d3beb80 10 client.0 _ll_register_callbacks cb 0x55e8d4936b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f414b59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4ab4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f414b59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4ab4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54110/0 (socket says 172.21.15.12:54110) 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f414b59e700 1 -- 172.21.15.12:0/3681214488 learned_addr learned my addr 172.21.15.12:0/3681214488 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f414a59c700 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a46e00 0x55e8d4a54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.662+0000 7f414ad9d700 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a41890 0x55e8d4a54580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f414b59e700 1 -- 172.21.15.12:0/3681214488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a41890 msgr2=0x55e8d4a54580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f414b59e700 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a41890 0x55e8d4a54580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f414b59e700 1 -- 172.21.15.12:0/3681214488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a46e00 msgr2=0x55e8d4a54ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f414b59e700 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a46e00 0x55e8d4a54ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f414b59e700 1 -- 172.21.15.12:0/3681214488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c10d780 con 0x55e8d4a47720 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f414b59e700 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4ab4b00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f414401f3b0 tx=0x7f4144004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736957c0445c9dc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f41327fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f41327fc700 1 -- 172.21.15.12:0/3681214488 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414400d040 con 0x55e8d4a47720 2024-05-08T23:55:19.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f41327fc700 1 -- 172.21.15.12:0/3681214488 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f414401f620 con 0x55e8d4a47720 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f415d3beb80 1 -- 172.21.15.12:0/3681214488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f413c0a21b0 con 0x55e8d4a47720 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f41327fc700 1 -- 172.21.15.12:0/3681214488 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4144004e50 con 0x55e8d4a47720 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f415d3beb80 10 client.17931 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f415d3beb80 1 -- 172.21.15.12:0/3681214488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e8d4957560 con 0x55e8d4a47720 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.663+0000 7f41327fc700 1 -- 172.21.15.12:0/3681214488 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f413c0a21b0 con 0x55e8d4a47720 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f41327fc700 1 -- 172.21.15.12:0/3681214488 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f4144015c40 con 0x55e8d4a47720 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f415d3beb80 10 client.17931 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f415d3beb80 10 client.17931 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f415d3beb80 20 client.17931 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f415d3beb80 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e8d4ab6920 0x55e8d4ab8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f415d3beb80 4 client.17931 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f415d3beb80 1 -- 172.21.15.12:0/3681214488 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8d4989ec0 con 0x55e8d4ab6920 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f414ad9d700 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e8d4ab6920 0x55e8d4ab8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f414ad9d700 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e8d4ab6920 0x55e8d4ab8d40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db007133081142df server_cookie=4c932b57f1b9e495 in_seq=0 out_seq=0 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f41327fc700 10 client.17931 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f41327fc700 1 -- 172.21.15.12:0/3681214488 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8d4989ec0 con 0x55e8d4ab6920 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f41327fc700 10 client.17931 handle_command_reply: tid=0 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.664+0000 7f415d3beb80 1 client.17931 shutdown 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 1 -- 172.21.15.12:0/3681214488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 msgr2=0x55e8d4ab4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4ab4b00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f414401f3b0 tx=0x7f4144004c60 comp rx=0 tx=0).stop 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 1 -- 172.21.15.12:0/3681214488 shutdown_connections 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e8d4ab6920 0x55e8d4ab8d40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8d4a47720 0x55e8d4ab4b00 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d4a41890 0x55e8d4a54580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 1 --2- 172.21.15.12:0/3681214488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e8d4a46e00 0x55e8d4a54ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f41327fc700 0 client.17931 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 1 -- 172.21.15.12:0/3681214488 >> 172.21.15.12:0/3681214488 conn(0x55e8d499f210 msgr2=0x55e8d4a45c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 1 -- 172.21.15.12:0/3681214488 shutdown_connections 2024-05-08T23:55:19.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 1 -- 172.21.15.12:0/3681214488 wait complete. 2024-05-08T23:55:19.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f415d3beb80 20 client.17931 trim_cache size 0 max 16384 2024-05-08T23:55:19.666 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f4151204700 1 -- 172.21.15.12:0/1092054109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41380a1290 msgr2=0x7f41380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f4151204700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41380a1290 0x7f41380a3750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4134000c00 tx=0x7f4134005fb0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f4151204700 1 -- 172.21.15.12:0/1092054109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f414c100790 msgr2=0x7f414c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f4151204700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f414c100790 0x7f414c198480 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f413c00d980 tx=0x7f413c00dd40 comp rx=0 tx=0).stop 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f4151204700 1 -- 172.21.15.12:0/1092054109 shutdown_connections 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f4151204700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41380a1290 0x7f41380a3750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f4151204700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f414c198f00 0x7f414c1a3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f4151204700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f414c100790 0x7f414c198480 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f4151204700 1 --2- 172.21.15.12:0/1092054109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f414c105390 0x7f414c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.665+0000 7f4151204700 1 -- 172.21.15.12:0/1092054109 >> 172.21.15.12:0/1092054109 conn(0x7f414c0763a0 msgr2=0x7f414c1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.666+0000 7f4151204700 1 -- 172.21.15.12:0/1092054109 shutdown_connections 2024-05-08T23:55:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.666+0000 7f4151204700 1 -- 172.21.15.12:0/1092054109 wait complete. 2024-05-08T23:55:19.674 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:55:19.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:19 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1733875618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:55:19.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:19 smithi204 ceph-mon[47373]: pgmap v281: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-05-08T23:55:19.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:19 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1733875618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:55:19.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:19 smithi160 ceph-mon[46152]: pgmap v281: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-05-08T23:55:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.818+0000 7f9b615c4700 1 -- 172.21.15.12:0/2537464762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c106300 msgr2=0x7f9b5c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.819+0000 7f9b615c4700 1 --2- 172.21.15.12:0/2537464762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c106300 0x7f9b5c10de80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009990 tx=0x7f9b54009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.819+0000 7f9b615c4700 1 -- 172.21.15.12:0/2537464762 shutdown_connections 2024-05-08T23:55:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.819+0000 7f9b615c4700 1 --2- 172.21.15.12:0/2537464762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c106300 0x7f9b5c10de80 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.819+0000 7f9b615c4700 1 --2- 172.21.15.12:0/2537464762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b5c1011e0 0x7f9b5c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.819+0000 7f9b615c4700 1 --2- 172.21.15.12:0/2537464762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b5c101b90 0x7f9b5c105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.819+0000 7f9b615c4700 1 -- 172.21.15.12:0/2537464762 >> 172.21.15.12:0/2537464762 conn(0x7f9b5c0fca80 msgr2=0x7f9b5c0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.819+0000 7f9b615c4700 1 -- 172.21.15.12:0/2537464762 shutdown_connections 2024-05-08T23:55:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.819+0000 7f9b615c4700 1 -- 172.21.15.12:0/2537464762 wait complete. 2024-05-08T23:55:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.819+0000 7f9b615c4700 1 Processor -- start 2024-05-08T23:55:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b615c4700 1 -- start start 2024-05-08T23:55:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b615c4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b5c1011e0 0x7f9b5c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b615c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c106300 0x7f9b5c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b615c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b5c19dfa0 0x7f9b5c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b615c4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c19ea20 con 0x7f9b5c106300 2024-05-08T23:55:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b615c4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c19eb90 con 0x7f9b5c19dfa0 2024-05-08T23:55:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b615c4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c19ed50 con 0x7f9b5c1011e0 2024-05-08T23:55:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c106300 0x7f9b5c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c106300 0x7f9b5c199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54136/0 (socket says 172.21.15.12:54136) 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/2109450518 learned_addr learned my addr 172.21.15.12:0/2109450518 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5affd700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b5c1011e0 0x7f9b5c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5b7fe700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b5c19dfa0 0x7f9b5c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/2109450518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b5c1011e0 msgr2=0x7f9b5c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b5c1011e0 0x7f9b5c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/2109450518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b5c19dfa0 msgr2=0x7f9b5c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b5c19dfa0 0x7f9b5c19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/2109450518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b540095f0 con 0x7f9b5c106300 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5b7fe700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b5c19dfa0 0x7f9b5c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.820+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c106300 0x7f9b5c199c00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000e970 tx=0x7f9b5000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.821+0000 7f9b43fff700 1 -- 172.21.15.12:0/2109450518 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50009790 con 0x7f9b5c106300 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.821+0000 7f9b615c4700 1 -- 172.21.15.12:0/2109450518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c1a62b0 con 0x7f9b5c106300 2024-05-08T23:55:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.821+0000 7f9b43fff700 1 -- 172.21.15.12:0/2109450518 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b5000eec0 con 0x7f9b5c106300 2024-05-08T23:55:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.821+0000 7f9b43fff700 1 -- 172.21.15.12:0/2109450518 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50005590 con 0x7f9b5c106300 2024-05-08T23:55:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.821+0000 7f9b615c4700 1 -- 172.21.15.12:0/2109450518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c1a67e0 con 0x7f9b5c106300 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.823+0000 7f9b43fff700 1 -- 172.21.15.12:0/2109450518 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9b5000cbd0 con 0x7f9b5c106300 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.824+0000 7f9b43fff700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b440a1310 0x7f9b440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.824+0000 7f9b5affd700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b440a1310 0x7f9b440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.824+0000 7f9b43fff700 1 -- 172.21.15.12:0/2109450518 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9b50014070 con 0x7f9b5c106300 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b5affd700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b440a1310 0x7f9b440a37d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009550 tx=0x7f9b4c0091b0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b6d77eb80 1 Processor -- start 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b6d77eb80 1 -- start start 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223dde00 0x55c2223de1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223d8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 0x55c2223d8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22232f150 con 0x55c2223de720 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22232cb10 con 0x55c2223dde00 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c222328400 con 0x55c2223d8890 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223d8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223d8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54140/0 (socket says 172.21.15.12:54140) 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/3636290046 learned_addr learned my addr 172.21.15.12:0/3636290046 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b5affd700 1 --2- 172.21.15.12:0/3636290046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223dde00 0x55c2223de1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b5b7fe700 1 --2- 172.21.15.12:0/3636290046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 0x55c2223d8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/3636290046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 msgr2=0x55c2223d8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/3636290046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 0x55c2223d8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.825+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/3636290046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223dde00 msgr2=0x55c2223de1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.826+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/3636290046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223dde00 0x55c2223de1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.826+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/3636290046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2221a93b0 con 0x55c2223de720 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.826+0000 7f9b5b7fe700 1 --2- 172.21.15.12:0/3636290046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 0x55c2223d8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.826+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/3636290046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223d8350 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9b500969c0 tx=0x7f9b5009a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f9783570a334c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.826+0000 7f9b417fa700 1 -- 172.21.15.12:0/3636290046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b500a23d0 con 0x55c2223de720 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.826+0000 7f9b417fa700 1 -- 172.21.15.12:0/3636290046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b500982f0 con 0x55c2223de720 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.826+0000 7f9b417fa700 1 -- 172.21.15.12:0/3636290046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50099440 con 0x55c2223de720 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/3636290046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 msgr2=0x55c2223d8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/3636290046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223d8350 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9b500969c0 tx=0x7f9b5009a6d0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/3636290046 shutdown_connections 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/3636290046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 0x55c2223d8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/3636290046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223dde00 0x55c2223de1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/3636290046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223d8350 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/3636290046 >> 172.21.15.12:0/3636290046 conn(0x55c222336210 msgr2=0x55c2223d2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/3636290046 shutdown_connections 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/3636290046 wait complete. 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 -- start start 2024-05-08T23:55:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 Processor -- start 2024-05-08T23:55:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 0x55c2223e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223dde00 0x55c2223e1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223e2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.827+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2222c4560 con 0x55c2223de720 2024-05-08T23:55:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2222c4200 con 0x55c2223dde00 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2221ab6a0 con 0x55c2223d8890 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b6d77eb80 10 client.0 _ll_register_callbacks cb 0x55c2222cdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223e2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223e2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54146/0 (socket says 172.21.15.12:54146) 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5b7fe700 1 -- 172.21.15.12:0/1150205631 learned_addr learned my addr 172.21.15.12:0/1150205631 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223dde00 0x55c2223e1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5affd700 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 0x55c2223e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5b7fe700 1 -- 172.21.15.12:0/1150205631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 msgr2=0x55c2223e1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5b7fe700 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 0x55c2223e1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5b7fe700 1 -- 172.21.15.12:0/1150205631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223dde00 msgr2=0x55c2223e1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5b7fe700 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223dde00 0x55c2223e1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5b7fe700 1 -- 172.21.15.12:0/1150205631 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c1a67e0 con 0x55c2223de720 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5affd700 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 0x55c2223e1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b5b7fe700 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223e2510 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9b54015000 tx=0x7f9b5400ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484a0b85c94df5d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1150205631 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400f930 con 0x55c2223de720 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1150205631 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c222281270 con 0x55c2223de720 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.828+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1150205631 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9b5400fa90 con 0x55c2223de720 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1150205631 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b5400fc50 con 0x55c2223de720 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b6d77eb80 10 client.17943 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1150205631 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c222155b60 con 0x55c2223de720 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1150205631 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b54031020 con 0x55c2223de720 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1150205631 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9b5400e3e0 con 0x55c2223de720 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b6d77eb80 10 client.17943 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b6d77eb80 10 client.17943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b6d77eb80 20 client.17943 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c2223f2270 0x55c2223f4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b6d77eb80 4 client.17943 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1150205631 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2223a7810 con 0x55c2223f2270 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.829+0000 7f9b5affd700 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c2223f2270 0x55c2223f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.830+0000 7f9b3eff5700 10 client.17943 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.830+0000 7f9b5affd700 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c2223f2270 0x55c2223f4690 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91cf06e91697aa2 server_cookie=34c0e08df2e442 in_seq=0 out_seq=0 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.830+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1150205631 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2223a7810 con 0x55c2223f2270 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.830+0000 7f9b3eff5700 10 client.17943 handle_command_reply: tid=0 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.830+0000 7f9b6d77eb80 1 client.17943 shutdown 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1150205631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 msgr2=0x55c2223e2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223e2510 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9b54015000 tx=0x7f9b5400ba60 comp rx=0 tx=0).stop 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1150205631 shutdown_connections 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223de720 0x55c2223e2510 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223d8890 0x55c2223e1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c2223f2270 0x55c2223f4690 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1150205631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223dde00 0x55c2223e1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b3eff5700 0 client.17943 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1150205631 >> 172.21.15.12:0/1150205631 conn(0x55c222336210 msgr2=0x55c2223d25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1150205631 shutdown_connections 2024-05-08T23:55:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1150205631 wait complete. 2024-05-08T23:55:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.831+0000 7f9b6d77eb80 20 client.17943 trim_cache size 0 max 16384 2024-05-08T23:55:19.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b6d77eb80 1 Processor -- start 2024-05-08T23:55:19.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b6d77eb80 1 -- start start 2024-05-08T23:55:19.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223d8890 0x55c2223d2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223dde00 0x55c2223e9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223de720 0x55c2223e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2223f9860 con 0x55c2223dde00 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2223cd800 con 0x55c2223d8890 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2223f4cf0 con 0x55c2223de720 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223dde00 0x55c2223e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223dde00 0x55c2223e9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54154/0 (socket says 172.21.15.12:54154) 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/146494178 learned_addr learned my addr 172.21.15.12:0/146494178 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b5affd700 1 --2- 172.21.15.12:0/146494178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223d8890 0x55c2223d2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.845+0000 7f9b5b7fe700 1 --2- 172.21.15.12:0/146494178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223de720 0x55c2223e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/146494178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223de720 msgr2=0x55c2223e96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/146494178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223de720 0x55c2223e96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/146494178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223d8890 msgr2=0x55c2223d2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/146494178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223d8890 0x55c2223d2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b5a7fc700 1 -- 172.21.15.12:0/146494178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2222ee560 con 0x55c2223dde00 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b5affd700 1 --2- 172.21.15.12:0/146494178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223d8890 0x55c2223d2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/146494178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223dde00 0x55c2223e9170 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9b500ca640 tx=0x7f9b5009de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c815acf8edd1f6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b41ffb700 1 -- 172.21.15.12:0/146494178 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b500a23d0 con 0x55c2223dde00 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b41ffb700 1 -- 172.21.15.12:0/146494178 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b50096660 con 0x55c2223dde00 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b41ffb700 1 -- 172.21.15.12:0/146494178 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5009a410 con 0x55c2223dde00 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/146494178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223dde00 msgr2=0x55c2223e9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.846+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/146494178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223dde00 0x55c2223e9170 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9b500ca640 tx=0x7f9b5009de80 comp rx=0 tx=0).stop 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/146494178 shutdown_connections 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/146494178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223de720 0x55c2223e96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/146494178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223d8890 0x55c2223d2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/146494178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223dde00 0x55c2223e9170 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/146494178 >> 172.21.15.12:0/146494178 conn(0x55c222336210 msgr2=0x55c2223d47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/146494178 shutdown_connections 2024-05-08T23:55:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/146494178 wait complete. 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 -- start start 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 Processor -- start 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223d8890 0x55c22243c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223dde00 0x55c22243cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223de720 0x55c222447740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22243d4e0 con 0x55c2223d8890 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c222447c80 con 0x55c2223de720 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c222447dc0 con 0x55c2223dde00 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b6d77eb80 10 client.0 _ll_register_callbacks cb 0x55c2222cdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223d8890 0x55c22243c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223d8890 0x55c22243c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54166/0 (socket says 172.21.15.12:54166) 2024-05-08T23:55:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.847+0000 7f9b5affd700 1 -- 172.21.15.12:0/1695137559 learned_addr learned my addr 172.21.15.12:0/1695137559 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223dde00 0x55c22243cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b5b7fe700 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223de720 0x55c222447740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b5affd700 1 -- 172.21.15.12:0/1695137559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223dde00 msgr2=0x55c22243cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b5affd700 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223dde00 0x55c22243cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b5affd700 1 -- 172.21.15.12:0/1695137559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223de720 msgr2=0x55c222447740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b5affd700 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223de720 0x55c222447740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b5affd700 1 -- 172.21.15.12:0/1695137559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2221a93b0 con 0x55c2223d8890 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b5affd700 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223d8890 0x55c22243c8b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009690 tx=0x7f9b4c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779d2a460011cd2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1695137559 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c029070 con 0x55c2223d8890 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1695137559 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9b4c024d00 con 0x55c2223d8890 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1695137559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b500982f0 con 0x55c2223d8890 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1695137559 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b4c0291d0 con 0x55c2223d8890 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b6d77eb80 10 client.17949 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.848+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1695137559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c2221adcd0 con 0x55c2223d8890 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1695137559 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b500982f0 con 0x55c2223d8890 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1695137559 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9b4c039070 con 0x55c2223d8890 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b6d77eb80 10 client.17949 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b6d77eb80 10 client.17949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b6d77eb80 20 client.17949 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c222449650 0x55c22244ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b6d77eb80 4 client.17949 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1695137559 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c222320ec0 con 0x55c222449650 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c222449650 0x55c22244ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b3eff5700 10 client.17949 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.849+0000 7f9b5a7fc700 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c222449650 0x55c22244ba70 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5062ddcdd44d44b server_cookie=df4d870db888a997 in_seq=0 out_seq=0 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b3eff5700 1 -- 172.21.15.12:0/1695137559 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c222320ec0 con 0x55c222449650 2024-05-08T23:55:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b3eff5700 10 client.17949 handle_command_reply: tid=0 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 client.17949 shutdown 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1695137559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223d8890 msgr2=0x55c22243c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223d8890 0x55c22243c8b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009690 tx=0x7f9b4c004b40 comp rx=0 tx=0).stop 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1695137559 shutdown_connections 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c222449650 0x55c22244ba70 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c2223de720 0x55c222447740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2223d8890 0x55c22243c8b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b3eff5700 0 client.17949 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 --2- 172.21.15.12:0/1695137559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c2223dde00 0x55c22243cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1695137559 >> 172.21.15.12:0/1695137559 conn(0x55c222336210 msgr2=0x55c2223dcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1695137559 shutdown_connections 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 1 -- 172.21.15.12:0/1695137559 wait complete. 2024-05-08T23:55:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b6d77eb80 20 client.17949 trim_cache size 0 max 16384 2024-05-08T23:55:19.852 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:55:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b615c4700 1 -- 172.21.15.12:0/2109450518 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b440a1310 msgr2=0x7f9b440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b615c4700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b440a1310 0x7f9b440a37d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009550 tx=0x7f9b4c0091b0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b615c4700 1 -- 172.21.15.12:0/2109450518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c106300 msgr2=0x7f9b5c199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b615c4700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c106300 0x7f9b5c199c00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000e970 tx=0x7f9b5000ec80 comp rx=0 tx=0).stop 2024-05-08T23:55:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b615c4700 1 -- 172.21.15.12:0/2109450518 shutdown_connections 2024-05-08T23:55:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.850+0000 7f9b615c4700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b440a1310 0x7f9b440a37d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.851+0000 7f9b615c4700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b5c19dfa0 0x7f9b5c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.851+0000 7f9b615c4700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b5c1011e0 0x7f9b5c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.851+0000 7f9b615c4700 1 --2- 172.21.15.12:0/2109450518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c106300 0x7f9b5c199c00 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.851+0000 7f9b615c4700 1 -- 172.21.15.12:0/2109450518 >> 172.21.15.12:0/2109450518 conn(0x7f9b5c0fca80 msgr2=0x7f9b5c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.851+0000 7f9b615c4700 1 -- 172.21.15.12:0/2109450518 shutdown_connections 2024-05-08T23:55:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:19.851+0000 7f9b615c4700 1 -- 172.21.15.12:0/2109450518 wait complete. 2024-05-08T23:55:19.857 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:19 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1733875618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:55:19.857 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:19 smithi012 ceph-mon[42503]: pgmap v281: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-05-08T23:55:19.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 516816 7.25 MB/sec execute 315 sec latency 1983.642 ms 2024-05-08T23:55:20.859 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:55:20.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:55:20.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 516816 7.22 MB/sec execute 316 sec latency 2983.687 ms 2024-05-08T23:55:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.015+0000 7f671c57d700 1 -- 172.21.15.12:0/1464468943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6714106280 msgr2=0x7f671410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.015+0000 7f671c57d700 1 --2- 172.21.15.12:0/1464468943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6714106280 0x7f671410de00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f670c00b0a0 tx=0x7f670c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.016+0000 7f671c57d700 1 -- 172.21.15.12:0/1464468943 shutdown_connections 2024-05-08T23:55:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.016+0000 7f671c57d700 1 --2- 172.21.15.12:0/1464468943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6714106280 0x7f671410de00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.016+0000 7f671c57d700 1 --2- 172.21.15.12:0/1464468943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6714101160 0x7f6714101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.016+0000 7f671c57d700 1 --2- 172.21.15.12:0/1464468943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6714101b10 0x7f6714105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.016+0000 7f671c57d700 1 -- 172.21.15.12:0/1464468943 >> 172.21.15.12:0/1464468943 conn(0x7f67140fc9e0 msgr2=0x7f67140fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.016+0000 7f671c57d700 1 -- 172.21.15.12:0/1464468943 shutdown_connections 2024-05-08T23:55:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.016+0000 7f671c57d700 1 -- 172.21.15.12:0/1464468943 wait complete. 2024-05-08T23:55:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.016+0000 7f671c57d700 1 Processor -- start 2024-05-08T23:55:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.016+0000 7f671c57d700 1 -- start start 2024-05-08T23:55:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f671c57d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6714101160 0x7f67141974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f671c57d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6714106280 0x7f6714197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f671c57d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f671419bdb0 0x7f671419c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f671c57d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419c830 con 0x7f6714106280 2024-05-08T23:55:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f671c57d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419c9a0 con 0x7f6714101160 2024-05-08T23:55:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f671c57d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419cb60 con 0x7f671419bdb0 2024-05-08T23:55:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f6719b18700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6714106280 0x7f6714197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f6719b18700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6714106280 0x7f6714197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54192/0 (socket says 172.21.15.12:54192) 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f6719b18700 1 -- 172.21.15.12:0/4169683616 learned_addr learned my addr 172.21.15.12:0/4169683616 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f671a319700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6714101160 0x7f67141974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f671ab1a700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f671419bdb0 0x7f671419c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f6719b18700 1 -- 172.21.15.12:0/4169683616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f671419bdb0 msgr2=0x7f671419c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f6719b18700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f671419bdb0 0x7f671419c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f6719b18700 1 -- 172.21.15.12:0/4169683616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6714101160 msgr2=0x7f67141974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f6719b18700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6714101160 0x7f67141974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f6719b18700 1 -- 172.21.15.12:0/4169683616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c009d00 con 0x7f6714106280 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f6719b18700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6714106280 0x7f6714197a10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f670400d980 tx=0x7f670400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f670b7fe700 1 -- 172.21.15.12:0/4169683616 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670400f890 con 0x7f6714106280 2024-05-08T23:55:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f670b7fe700 1 -- 172.21.15.12:0/4169683616 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6704010460 con 0x7f6714106280 2024-05-08T23:55:21.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.017+0000 7f670b7fe700 1 -- 172.21.15.12:0/4169683616 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670400e610 con 0x7f6714106280 2024-05-08T23:55:21.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.018+0000 7f671c57d700 1 -- 172.21.15.12:0/4169683616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67141a40c0 con 0x7f6714106280 2024-05-08T23:55:21.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.018+0000 7f671c57d700 1 -- 172.21.15.12:0/4169683616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67141a45a0 con 0x7f6714106280 2024-05-08T23:55:21.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.020+0000 7f670b7fe700 1 -- 172.21.15.12:0/4169683616 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f67040105d0 con 0x7f6714106280 2024-05-08T23:55:21.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.021+0000 7f670b7fe700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67000a1310 0x7f67000a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.021+0000 7f671a319700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67000a1310 0x7f67000a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.021+0000 7f670b7fe700 1 -- 172.21.15.12:0/4169683616 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f67040cc5c0 con 0x7f6714106280 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.022+0000 7f671a319700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67000a1310 0x7f67000a37d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f671000ac50 tx=0x7f671000a380 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.022+0000 7f6728737b80 1 Processor -- start 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.022+0000 7f6728737b80 1 -- start start 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.022+0000 7f6728737b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e40e80 0x5580e6e41260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.022+0000 7f6728737b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e3b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.022+0000 7f6728737b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e3b910 0x5580e6e3bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.022+0000 7f6728737b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6d921d0 con 0x5580e6e40e80 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.022+0000 7f6728737b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6d8fb90 con 0x5580e6e3b910 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.022+0000 7f6728737b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6d8b480 con 0x5580e6e417a0 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f671a319700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e40e80 0x5580e6e41260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f671a319700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e40e80 0x5580e6e41260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54206/0 (socket says 172.21.15.12:54206) 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f671a319700 1 -- 172.21.15.12:0/1041560063 learned_addr learned my addr 172.21.15.12:0/1041560063 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f671ab1a700 1 --2- 172.21.15.12:0/1041560063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e3b910 0x5580e6e3bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f6719b18700 1 --2- 172.21.15.12:0/1041560063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e3b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f671a319700 1 -- 172.21.15.12:0/1041560063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 msgr2=0x5580e6e3b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f671a319700 1 --2- 172.21.15.12:0/1041560063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e3b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f671a319700 1 -- 172.21.15.12:0/1041560063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e3b910 msgr2=0x5580e6e3bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f671a319700 1 --2- 172.21.15.12:0/1041560063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e3b910 0x5580e6e3bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f671a319700 1 -- 172.21.15.12:0/1041560063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580e6c0c430 con 0x5580e6e40e80 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f671a319700 1 --2- 172.21.15.12:0/1041560063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e40e80 0x5580e6e41260 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f671000ff90 tx=0x7f6710012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e7324e21952efd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f6708ff9700 1 -- 172.21.15.12:0/1041560063 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710003a90 con 0x5580e6e40e80 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f6708ff9700 1 -- 172.21.15.12:0/1041560063 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6710003bf0 con 0x5580e6e40e80 2024-05-08T23:55:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.023+0000 7f6708ff9700 1 -- 172.21.15.12:0/1041560063 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671001d660 con 0x5580e6e40e80 2024-05-08T23:55:21.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.024+0000 7f6728737b80 1 -- 172.21.15.12:0/1041560063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e40e80 msgr2=0x5580e6e41260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.024+0000 7f6728737b80 1 --2- 172.21.15.12:0/1041560063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e40e80 0x5580e6e41260 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f671000ff90 tx=0x7f6710012000 comp rx=0 tx=0).stop 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.024+0000 7f6728737b80 1 -- 172.21.15.12:0/1041560063 shutdown_connections 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.024+0000 7f6728737b80 1 --2- 172.21.15.12:0/1041560063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e3b910 0x5580e6e3bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.024+0000 7f6728737b80 1 --2- 172.21.15.12:0/1041560063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e40e80 0x5580e6e41260 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.024+0000 7f6728737b80 1 --2- 172.21.15.12:0/1041560063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e3b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.024+0000 7f6728737b80 1 -- 172.21.15.12:0/1041560063 >> 172.21.15.12:0/1041560063 conn(0x5580e6d99290 msgr2=0x5580e6e35e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.024+0000 7f6728737b80 1 -- 172.21.15.12:0/1041560063 shutdown_connections 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f6728737b80 1 -- 172.21.15.12:0/1041560063 wait complete. 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f6728737b80 1 -- start start 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f6728737b80 1 Processor -- start 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f6728737b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e44a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f6728737b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 0x5580e6e44fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f6728737b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f6728737b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6d275e0 con 0x5580e6e3b910 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f6728737b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6d27280 con 0x5580e6e40e80 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f6728737b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6c0e720 con 0x5580e6e417a0 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f6728737b80 10 client.0 _ll_register_callbacks cb 0x5580e6d30bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f671a319700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e44a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f671a319700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e44a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54222/0 (socket says 172.21.15.12:54222) 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.025+0000 7f671a319700 1 -- 172.21.15.12:0/452190255 learned_addr learned my addr 172.21.15.12:0/452190255 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f671ab1a700 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f6719b18700 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 0x5580e6e44fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f671a319700 1 -- 172.21.15.12:0/452190255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 msgr2=0x5580e6e454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f671a319700 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f671a319700 1 -- 172.21.15.12:0/452190255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 msgr2=0x5580e6e44fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f671a319700 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 0x5580e6e44fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f671a319700 1 -- 172.21.15.12:0/452190255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671000fc40 con 0x5580e6e3b910 2024-05-08T23:55:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f671a319700 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e44a70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f671000fa30 tx=0x7f6710003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6060c2975f8d2fc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f66fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f66fdffb700 1 -- 172.21.15.12:0/452190255 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671002b070 con 0x5580e6e3b910 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f66fdffb700 1 -- 172.21.15.12:0/452190255 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6710010430 con 0x5580e6e3b910 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f6728737b80 1 -- 172.21.15.12:0/452190255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580e6ce42f0 con 0x5580e6e3b910 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f66fdffb700 1 -- 172.21.15.12:0/452190255 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67100105f0 con 0x5580e6e3b910 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f6728737b80 10 client.17961 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.026+0000 7f6728737b80 1 -- 172.21.15.12:0/452190255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5580e6bb8be0 con 0x5580e6e3b910 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.027+0000 7f66fdffb700 1 -- 172.21.15.12:0/452190255 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671003c020 con 0x5580e6e3b910 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.027+0000 7f66fdffb700 1 -- 172.21.15.12:0/452190255 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f67100253f0 con 0x5580e6e3b910 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.027+0000 7f6728737b80 10 client.17961 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.027+0000 7f6728737b80 10 client.17961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.027+0000 7f6728737b80 20 client.17961 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.027+0000 7f6728737b80 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5580e6e55010 0x5580e6e57430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.027+0000 7f6728737b80 4 client.17961 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.027+0000 7f6728737b80 1 -- 172.21.15.12:0/452190255 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580e6e0a890 con 0x5580e6e55010 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.027+0000 7f6719b18700 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5580e6e55010 0x5580e6e57430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.028+0000 7f66fdffb700 10 client.17961 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.028+0000 7f6719b18700 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5580e6e55010 0x5580e6e57430 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d4a26c809636e5 server_cookie=ca36d69083a9572e in_seq=0 out_seq=0 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.028+0000 7f66fdffb700 1 -- 172.21.15.12:0/452190255 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5580e6e0a890 con 0x5580e6e55010 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f66fdffb700 10 client.17961 handle_command_reply: tid=0 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 client.17961 shutdown 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 -- 172.21.15.12:0/452190255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 msgr2=0x5580e6e44a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e44a70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f671000fa30 tx=0x7f6710003f90 comp rx=0 tx=0).stop 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 -- 172.21.15.12:0/452190255 shutdown_connections 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e44a70 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5580e6e55010 0x5580e6e57430 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 --2- 172.21.15.12:0/452190255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 0x5580e6e44fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f66fdffb700 0 client.17961 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 -- 172.21.15.12:0/452190255 >> 172.21.15.12:0/452190255 conn(0x5580e6d99290 msgr2=0x5580e6e35620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 -- 172.21.15.12:0/452190255 shutdown_connections 2024-05-08T23:55:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 1 -- 172.21.15.12:0/452190255 wait complete. 2024-05-08T23:55:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.029+0000 7f6728737b80 20 client.17961 trim_cache size 0 max 16384 2024-05-08T23:55:21.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.047+0000 7f6728737b80 1 Processor -- start 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.047+0000 7f6728737b80 1 -- start start 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.047+0000 7f6728737b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e3b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.047+0000 7f6728737b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e40e80 0x5580e6e403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.047+0000 7f6728737b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e417a0 0x5580e6e35da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.047+0000 7f6728737b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6e47d90 con 0x5580e6e3b910 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.047+0000 7f6728737b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6e36b90 con 0x5580e6e417a0 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.047+0000 7f6728737b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6e308b0 con 0x5580e6e40e80 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f671a319700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e3b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f671a319700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e3b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54230/0 (socket says 172.21.15.12:54230) 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f671a319700 1 -- 172.21.15.12:0/1742722917 learned_addr learned my addr 172.21.15.12:0/1742722917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f671ab1a700 1 --2- 172.21.15.12:0/1742722917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e417a0 0x5580e6e35da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f6719b18700 1 --2- 172.21.15.12:0/1742722917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e40e80 0x5580e6e403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f671a319700 1 -- 172.21.15.12:0/1742722917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e40e80 msgr2=0x5580e6e403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f671a319700 1 --2- 172.21.15.12:0/1742722917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e40e80 0x5580e6e403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f671a319700 1 -- 172.21.15.12:0/1742722917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e417a0 msgr2=0x5580e6e35da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f671a319700 1 --2- 172.21.15.12:0/1742722917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e417a0 0x5580e6e35da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f671a319700 1 -- 172.21.15.12:0/1742722917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580e6e32560 con 0x5580e6e3b910 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f671a319700 1 --2- 172.21.15.12:0/1742722917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e3b3d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6710013040 tx=0x7f67100259a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9571275c1db0bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f67097fa700 1 -- 172.21.15.12:0/1742722917 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671002b020 con 0x5580e6e3b910 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f67097fa700 1 -- 172.21.15.12:0/1742722917 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671003c030 con 0x5580e6e3b910 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.048+0000 7f67097fa700 1 -- 172.21.15.12:0/1742722917 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710010690 con 0x5580e6e3b910 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 -- 172.21.15.12:0/1742722917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 msgr2=0x5580e6e3b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 --2- 172.21.15.12:0/1742722917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e3b3d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6710013040 tx=0x7f67100259a0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 -- 172.21.15.12:0/1742722917 shutdown_connections 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 --2- 172.21.15.12:0/1742722917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e417a0 0x5580e6e35da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 --2- 172.21.15.12:0/1742722917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e3b3d0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 --2- 172.21.15.12:0/1742722917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e40e80 0x5580e6e403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 -- 172.21.15.12:0/1742722917 >> 172.21.15.12:0/1742722917 conn(0x5580e6d99290 msgr2=0x5580e6e43340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 -- 172.21.15.12:0/1742722917 shutdown_connections 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 -- 172.21.15.12:0/1742722917 wait complete. 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 -- start start 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.049+0000 7f6728737b80 1 Processor -- start 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f6728737b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e4c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f6728737b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 0x5580e6e48980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f6728737b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e48ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f6728737b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6e495d0 con 0x5580e6e3b910 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f6728737b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6e49740 con 0x5580e6e40e80 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f6728737b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e6eab510 con 0x5580e6e417a0 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f6728737b80 10 client.0 _ll_register_callbacks cb 0x5580e6d30bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671a319700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e4c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671a319700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e4c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54238/0 (socket says 172.21.15.12:54238) 2024-05-08T23:55:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671a319700 1 -- 172.21.15.12:0/3999416252 learned_addr learned my addr 172.21.15.12:0/3999416252 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671ab1a700 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e48ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f6719b18700 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 0x5580e6e48980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671a319700 1 -- 172.21.15.12:0/3999416252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 msgr2=0x5580e6e48ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671a319700 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e48ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671a319700 1 -- 172.21.15.12:0/3999416252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 msgr2=0x5580e6e48980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671a319700 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 0x5580e6e48980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671a319700 1 -- 172.21.15.12:0/3999416252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580e6bb8be0 con 0x5580e6e3b910 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f6719b18700 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 0x5580e6e48980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671ab1a700 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e48ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f671a319700 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e4c530 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f671000fa30 tx=0x7f67100253f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfc1948abd1111b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.050+0000 7f66fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f66fdffb700 1 -- 172.21.15.12:0/3999416252 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671002b030 con 0x5580e6e3b910 2024-05-08T23:55:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f66fdffb700 1 -- 172.21.15.12:0/3999416252 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f671003d080 con 0x5580e6e3b910 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f6728737b80 1 -- 172.21.15.12:0/3999416252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6710003bf0 con 0x5580e6e3b910 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f66fdffb700 1 -- 172.21.15.12:0/3999416252 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6710010630 con 0x5580e6e3b910 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f6728737b80 10 client.17967 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f6728737b80 1 -- 172.21.15.12:0/3999416252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5580e6d515e0 con 0x5580e6e3b910 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f66fdffb700 1 -- 172.21.15.12:0/3999416252 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6710003bf0 con 0x5580e6e3b910 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f66fdffb700 1 -- 172.21.15.12:0/3999416252 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f6710004b80 con 0x5580e6e3b910 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f6728737b80 10 client.17967 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f6728737b80 10 client.17967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f6728737b80 20 client.17967 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f6728737b80 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5580e6eac740 0x5580e6eaeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f6728737b80 4 client.17967 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f6728737b80 1 -- 172.21.15.12:0/3999416252 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5580e6d83f40 con 0x5580e6eac740 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.051+0000 7f6719b18700 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5580e6eac740 0x5580e6eaeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.052+0000 7f6719b18700 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5580e6eac740 0x5580e6eaeb30 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=477d84f8e5552ae8 server_cookie=8e52231dabb89c8e in_seq=0 out_seq=0 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.052+0000 7f66fdffb700 10 client.17967 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f66fdffb700 1 -- 172.21.15.12:0/3999416252 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5580e6d83f40 con 0x5580e6eac740 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f66fdffb700 10 client.17967 handle_command_reply: tid=0 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 client.17967 shutdown 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 -- 172.21.15.12:0/3999416252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 msgr2=0x5580e6e4c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e4c530 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f671000fa30 tx=0x7f67100253f0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 -- 172.21.15.12:0/3999416252 shutdown_connections 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580e6e417a0 0x5580e6e48ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5580e6e3b910 0x5580e6e4c530 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5580e6eac740 0x5580e6eaeb30 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 --2- 172.21.15.12:0/3999416252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580e6e40e80 0x5580e6e48980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 -- 172.21.15.12:0/3999416252 >> 172.21.15.12:0/3999416252 conn(0x5580e6d99290 msgr2=0x5580e6e45350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f66fdffb700 0 client.17967 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 -- 172.21.15.12:0/3999416252 shutdown_connections 2024-05-08T23:55:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.053+0000 7f6728737b80 1 -- 172.21.15.12:0/3999416252 wait complete. 2024-05-08T23:55:21.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f6728737b80 20 client.17967 trim_cache size 0 max 16384 2024-05-08T23:55:21.057 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:55:21.057 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:55:21.057 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "88e35ffd-5f3d-49b3-9a73-52576e93cebb", 2024-05-08T23:55:21.057 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:55:21.057 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:55:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 -- 172.21.15.12:0/4169683616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67000a1310 msgr2=0x7f67000a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67000a1310 0x7f67000a37d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f671000ac50 tx=0x7f671000a380 comp rx=0 tx=0).stop 2024-05-08T23:55:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 -- 172.21.15.12:0/4169683616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6714106280 msgr2=0x7f6714197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6714106280 0x7f6714197a10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f670400d980 tx=0x7f670400dc90 comp rx=0 tx=0).stop 2024-05-08T23:55:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 -- 172.21.15.12:0/4169683616 shutdown_connections 2024-05-08T23:55:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67000a1310 0x7f67000a37d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f671419bdb0 0x7f671419c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6714101160 0x7f67141974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 --2- 172.21.15.12:0/4169683616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6714106280 0x7f6714197a10 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 -- 172.21.15.12:0/4169683616 >> 172.21.15.12:0/4169683616 conn(0x7f67140fc9e0 msgr2=0x7f67140feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 -- 172.21.15.12:0/4169683616 shutdown_connections 2024-05-08T23:55:21.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.054+0000 7f671c57d700 1 -- 172.21.15.12:0/4169683616 wait complete. 2024-05-08T23:55:21.064 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:55:21.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:21 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:55:21.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:21 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.257+0000 7ffaf50ca700 1 -- 172.21.15.12:0/2460551321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaf01011e0 msgr2=0x7ffaf01015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.257+0000 7ffaf50ca700 1 --2- 172.21.15.12:0/2460551321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaf01011e0 0x7ffaf01015c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009990 tx=0x7ffae0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.257+0000 7ffaf50ca700 1 -- 172.21.15.12:0/2460551321 shutdown_connections 2024-05-08T23:55:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.257+0000 7ffaf50ca700 1 --2- 172.21.15.12:0/2460551321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf0106300 0x7ffaf010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.257+0000 7ffaf50ca700 1 --2- 172.21.15.12:0/2460551321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaf01011e0 0x7ffaf01015c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.257+0000 7ffaf50ca700 1 --2- 172.21.15.12:0/2460551321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffaf0101b90 0x7ffaf0105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.257+0000 7ffaf50ca700 1 -- 172.21.15.12:0/2460551321 >> 172.21.15.12:0/2460551321 conn(0x7ffaf00fca80 msgr2=0x7ffaf00feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.257+0000 7ffaf50ca700 1 -- 172.21.15.12:0/2460551321 shutdown_connections 2024-05-08T23:55:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.257+0000 7ffaf50ca700 1 -- 172.21.15.12:0/2460551321 wait complete. 2024-05-08T23:55:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaf50ca700 1 Processor -- start 2024-05-08T23:55:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaf50ca700 1 -- start start 2024-05-08T23:55:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaf50ca700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf01011e0 0x7ffaf01975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaf50ca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffaf0106300 0x7ffaf0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaf50ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaf019beb0 0x7ffaf019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaf50ca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf019c930 con 0x7ffaf019beb0 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaf50ca700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf019caa0 con 0x7ffaf0106300 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaf50ca700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf019cc60 con 0x7ffaf01011e0 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaeed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf01011e0 0x7ffaf01975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaef59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaf019beb0 0x7ffaf019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaee59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffaf0106300 0x7ffaf0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaef59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaf019beb0 0x7ffaf019c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54258/0 (socket says 172.21.15.12:54258) 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaeed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf01011e0 0x7ffaf01975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:37052/0 (socket says 172.21.15.12:37052) 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.259+0000 7ffaef59e700 1 -- 172.21.15.12:0/746336026 learned_addr learned my addr 172.21.15.12:0/746336026 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffaef59e700 1 -- 172.21.15.12:0/746336026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf01011e0 msgr2=0x7ffaf01975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffaef59e700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf01011e0 0x7ffaf01975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffaef59e700 1 -- 172.21.15.12:0/746336026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffaf0106300 msgr2=0x7ffaf0197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffaef59e700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffaf0106300 0x7ffaf0197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffaef59e700 1 -- 172.21.15.12:0/746336026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae00095f0 con 0x7ffaf019beb0 2024-05-08T23:55:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffaee59c700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffaf0106300 0x7ffaf0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf01011e0 0x7ffaf01975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffaef59e700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaf019beb0 0x7ffaf019c330 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ffae8007ee0 tx=0x7ffae800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffad7fff700 1 -- 172.21.15.12:0/746336026 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8017070 con 0x7ffaf019beb0 2024-05-08T23:55:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffaf50ca700 1 -- 172.21.15.12:0/746336026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf01a41c0 con 0x7ffaf019beb0 2024-05-08T23:55:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffad7fff700 1 -- 172.21.15.12:0/746336026 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffae800f040 con 0x7ffaf019beb0 2024-05-08T23:55:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffad7fff700 1 -- 172.21.15.12:0/746336026 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8005580 con 0x7ffaf019beb0 2024-05-08T23:55:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.260+0000 7ffaf50ca700 1 -- 172.21.15.12:0/746336026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf01a46a0 con 0x7ffaf019beb0 2024-05-08T23:55:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.263+0000 7ffad7fff700 1 -- 172.21.15.12:0/746336026 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ffae80056e0 con 0x7ffaf019beb0 2024-05-08T23:55:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.263+0000 7ffad7fff700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffad80a1290 0x7ffad80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.263+0000 7ffad7fff700 1 -- 172.21.15.12:0/746336026 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ffae80cc380 con 0x7ffaf019beb0 2024-05-08T23:55:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.263+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffad80a1290 0x7ffad80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.264+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffad80a1290 0x7ffad80a3750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffae000b2e0 tx=0x7ffae0005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.264+0000 7ffb01284b80 1 Processor -- start 2024-05-08T23:55:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.264+0000 7ffb01284b80 1 -- start start 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.264+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c9751f0 0x55cf4c9755d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.264+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c975b10 0x55cf4c96f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.264+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c970100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.264+0000 7ffb01284b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c8c2150 con 0x55cf4c9751f0 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.264+0000 7ffb01284b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c8bfb10 con 0x55cf4c96fca0 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.264+0000 7ffb01284b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c8bb400 con 0x55cf4c975b10 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaeed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c9751f0 0x55cf4c9755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaef59e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c970100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaeed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c9751f0 0x55cf4c9755d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54268/0 (socket says 172.21.15.12:54268) 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaeed9d700 1 -- 172.21.15.12:0/380081870 learned_addr learned my addr 172.21.15.12:0/380081870 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaee59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c975b10 0x55cf4c96f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaeed9d700 1 -- 172.21.15.12:0/380081870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c975b10 msgr2=0x55cf4c96f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/380081870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c975b10 0x55cf4c96f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaeed9d700 1 -- 172.21.15.12:0/380081870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 msgr2=0x55cf4c970100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/380081870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c970100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaeed9d700 1 -- 172.21.15.12:0/380081870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf4c73c3b0 con 0x55cf4c9751f0 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaef59e700 1 --2- 172.21.15.12:0/380081870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c970100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/380081870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c9751f0 0x55cf4c9755d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ffae00095c0 tx=0x7ffae0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c81ed77ba136b95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffad57fa700 1 -- 172.21.15.12:0/380081870 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae002b070 con 0x55cf4c9751f0 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffad57fa700 1 -- 172.21.15.12:0/380081870 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffae001e040 con 0x55cf4c9751f0 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.265+0000 7ffad57fa700 1 -- 172.21.15.12:0/380081870 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0024d30 con 0x55cf4c9751f0 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.266+0000 7ffb01284b80 1 -- 172.21.15.12:0/380081870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c9751f0 msgr2=0x55cf4c9755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.266+0000 7ffb01284b80 1 --2- 172.21.15.12:0/380081870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c9751f0 0x55cf4c9755d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ffae00095c0 tx=0x7ffae0020000 comp rx=0 tx=0).stop 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.266+0000 7ffb01284b80 1 -- 172.21.15.12:0/380081870 shutdown_connections 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.266+0000 7ffb01284b80 1 --2- 172.21.15.12:0/380081870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c970100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.266+0000 7ffb01284b80 1 --2- 172.21.15.12:0/380081870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c9751f0 0x55cf4c9755d0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.266+0000 7ffb01284b80 1 --2- 172.21.15.12:0/380081870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c975b10 0x55cf4c96f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.266+0000 7ffb01284b80 1 -- 172.21.15.12:0/380081870 >> 172.21.15.12:0/380081870 conn(0x55cf4c8c9210 msgr2=0x55cf4c965df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.266+0000 7ffb01284b80 1 -- 172.21.15.12:0/380081870 shutdown_connections 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.266+0000 7ffb01284b80 1 -- 172.21.15.12:0/380081870 wait complete. 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffb01284b80 1 -- start start 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffb01284b80 1 Processor -- start 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c978fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 0x55cf4c979500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c979a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffb01284b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c857560 con 0x55cf4c975b10 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffb01284b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c857200 con 0x55cf4c96fca0 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffb01284b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c73e6a0 con 0x55cf4c9751f0 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffb01284b80 10 client.0 _ll_register_callbacks cb 0x55cf4c860b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffaef59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c979a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffaeed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c978fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffaef59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c979a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54276/0 (socket says 172.21.15.12:54276) 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffaeed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c978fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34918/0 (socket says 172.21.15.12:34918) 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffaef59e700 1 -- 172.21.15.12:0/2731705570 learned_addr learned my addr 172.21.15.12:0/2731705570 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.267+0000 7ffaee59c700 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 0x55cf4c979500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffaef59e700 1 -- 172.21.15.12:0/2731705570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 msgr2=0x55cf4c979500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffaef59e700 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 0x55cf4c979500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffaef59e700 1 -- 172.21.15.12:0/2731705570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 msgr2=0x55cf4c978fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffaef59e700 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c978fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffaef59e700 1 -- 172.21.15.12:0/2731705570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf01a46a0 con 0x55cf4c975b10 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffaee59c700 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 0x55cf4c979500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c978fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffaef59e700 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c979a40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ffae80c9b90 tx=0x7ffae8004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c56e87aaa3aabc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffad2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffad2ff5700 1 -- 172.21.15.12:0/2731705570 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8099ba0 con 0x55cf4c975b10 2024-05-08T23:55:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffad2ff5700 1 -- 172.21.15.12:0/2731705570 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ffae80994a0 con 0x55cf4c975b10 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffb01284b80 1 -- 172.21.15.12:0/2731705570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf4c814270 con 0x55cf4c975b10 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffad2ff5700 1 -- 172.21.15.12:0/2731705570 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffae809d2d0 con 0x55cf4c975b10 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffb01284b80 10 client.17979 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffb01284b80 1 -- 172.21.15.12:0/2731705570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7ffaf010e6f0 con 0x55cf4c975b10 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.268+0000 7ffad2ff5700 1 -- 172.21.15.12:0/2731705570 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf4c814270 con 0x55cf4c975b10 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.269+0000 7ffad2ff5700 1 -- 172.21.15.12:0/2731705570 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ffae8099770 con 0x55cf4c975b10 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.269+0000 7ffb01284b80 10 client.17979 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.269+0000 7ffb01284b80 10 client.17979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.269+0000 7ffb01284b80 20 client.17979 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.269+0000 7ffb01284b80 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf4c989770 0x55cf4c98bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.269+0000 7ffb01284b80 4 client.17979 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.269+0000 7ffb01284b80 1 -- 172.21.15.12:0/2731705570 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf4c93a810 con 0x55cf4c989770 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.269+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf4c989770 0x55cf4c98bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.269+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf4c989770 0x55cf4c98bb90 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aecb15e1f2e9b9d0 server_cookie=7b5ac6ec9c0a8e49 in_seq=0 out_seq=0 2024-05-08T23:55:21.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.269+0000 7ffad2ff5700 10 client.17979 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffad2ff5700 1 -- 172.21.15.12:0/2731705570 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf4c93a810 con 0x55cf4c989770 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffad2ff5700 10 client.17979 handle_command_reply: tid=0 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffb01284b80 1 client.17979 shutdown 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffb01284b80 1 -- 172.21.15.12:0/2731705570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 msgr2=0x55cf4c979a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffb01284b80 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c979a40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ffae80c9b90 tx=0x7ffae8004f10 comp rx=0 tx=0).stop 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffb01284b80 1 -- 172.21.15.12:0/2731705570 shutdown_connections 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffb01284b80 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c979a40 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffb01284b80 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf4c989770 0x55cf4c98bb90 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffb01284b80 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c978fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffb01284b80 1 --2- 172.21.15.12:0/2731705570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 0x55cf4c979500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffb01284b80 1 -- 172.21.15.12:0/2731705570 >> 172.21.15.12:0/2731705570 conn(0x55cf4c8c9210 msgr2=0x55cf4c9655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.271+0000 7ffad2ff5700 0 client.17979 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.272+0000 7ffb01284b80 1 -- 172.21.15.12:0/2731705570 shutdown_connections 2024-05-08T23:55:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.272+0000 7ffb01284b80 1 -- 172.21.15.12:0/2731705570 wait complete. 2024-05-08T23:55:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.272+0000 7ffb01284b80 20 client.17979 trim_cache size 0 max 16384 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffb01284b80 1 Processor -- start 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffb01284b80 1 -- start start 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c96fca0 0x55cf4c9747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c9751f0 0x55cf4c965d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c9778b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffb01284b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c967640 con 0x55cf4c975b10 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffb01284b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c976fb0 con 0x55cf4c9751f0 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffb01284b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c967e00 con 0x55cf4c96fca0 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffaef59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c9778b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffaee59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c9751f0 0x55cf4c965d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffaef59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c9778b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54282/0 (socket says 172.21.15.12:54282) 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffaeed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c96fca0 0x55cf4c9747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.290+0000 7ffaef59e700 1 -- 172.21.15.12:0/59564897 learned_addr learned my addr 172.21.15.12:0/59564897 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.291+0000 7ffaef59e700 1 -- 172.21.15.12:0/59564897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c96fca0 msgr2=0x55cf4c9747d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.291+0000 7ffaef59e700 1 --2- 172.21.15.12:0/59564897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c96fca0 0x55cf4c9747d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.291+0000 7ffaef59e700 1 -- 172.21.15.12:0/59564897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c9751f0 msgr2=0x55cf4c965d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.291+0000 7ffaef59e700 1 --2- 172.21.15.12:0/59564897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c9751f0 0x55cf4c965d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.291+0000 7ffaef59e700 1 -- 172.21.15.12:0/59564897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf4c6e8b60 con 0x55cf4c975b10 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.291+0000 7ffaee59c700 1 --2- 172.21.15.12:0/59564897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c9751f0 0x55cf4c965d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.291+0000 7ffaef59e700 1 --2- 172.21.15.12:0/59564897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c9778b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ffae8095f10 tx=0x7ffae8097a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8db92eb0bb04f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.291+0000 7ffad5ffb700 1 -- 172.21.15.12:0/59564897 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8098a60 con 0x55cf4c975b10 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.291+0000 7ffad5ffb700 1 -- 172.21.15.12:0/59564897 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffae8099ba0 con 0x55cf4c975b10 2024-05-08T23:55:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.291+0000 7ffad5ffb700 1 -- 172.21.15.12:0/59564897 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae80a1920 con 0x55cf4c975b10 2024-05-08T23:55:21.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 -- 172.21.15.12:0/59564897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 msgr2=0x55cf4c9778b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 --2- 172.21.15.12:0/59564897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c9778b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ffae8095f10 tx=0x7ffae8097a50 comp rx=0 tx=0).stop 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 -- 172.21.15.12:0/59564897 shutdown_connections 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 --2- 172.21.15.12:0/59564897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c9778b0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 --2- 172.21.15.12:0/59564897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c96fca0 0x55cf4c9747d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 --2- 172.21.15.12:0/59564897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c9751f0 0x55cf4c965d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 -- 172.21.15.12:0/59564897 >> 172.21.15.12:0/59564897 conn(0x55cf4c8c9210 msgr2=0x55cf4c9740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 -- 172.21.15.12:0/59564897 shutdown_connections 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 -- 172.21.15.12:0/59564897 wait complete. 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 -- start start 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.292+0000 7ffb01284b80 1 Processor -- start 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c97e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 0x55cf4c97e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffb01284b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c97ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffb01284b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c9decb0 con 0x55cf4c975b10 2024-05-08T23:55:21.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffb01284b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c9dee20 con 0x55cf4c96fca0 2024-05-08T23:55:21.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffb01284b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4c9defe0 con 0x55cf4c9751f0 2024-05-08T23:55:21.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffb01284b80 10 client.0 _ll_register_callbacks cb 0x7ffae8098e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:21.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaef59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c97ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaeed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c97e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaef59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c97ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54290/0 (socket says 172.21.15.12:54290) 2024-05-08T23:55:21.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaef59e700 1 -- 172.21.15.12:0/3459004434 learned_addr learned my addr 172.21.15.12:0/3459004434 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:21.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaee59c700 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 0x55cf4c97e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaef59e700 1 -- 172.21.15.12:0/3459004434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 msgr2=0x55cf4c97e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaef59e700 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 0x55cf4c97e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaef59e700 1 -- 172.21.15.12:0/3459004434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 msgr2=0x55cf4c97e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaef59e700 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c97e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaef59e700 1 -- 172.21.15.12:0/3459004434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae4012760 con 0x55cf4c975b10 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffaef59e700 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c97ef00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ffae80977f0 tx=0x7ffae8004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9981911aacb72bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.293+0000 7ffad2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffad2ff5700 1 -- 172.21.15.12:0/3459004434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8098a60 con 0x55cf4c975b10 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffad2ff5700 1 -- 172.21.15.12:0/3459004434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ffae80a1920 con 0x55cf4c975b10 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffb01284b80 1 -- 172.21.15.12:0/3459004434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffae001e040 con 0x55cf4c975b10 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffad2ff5700 1 -- 172.21.15.12:0/3459004434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffae809d2d0 con 0x55cf4c975b10 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffb01284b80 10 client.17985 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffb01284b80 1 -- 172.21.15.12:0/3459004434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7ffaf010e6f0 con 0x55cf4c975b10 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffad2ff5700 1 -- 172.21.15.12:0/3459004434 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffae001e040 con 0x55cf4c975b10 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffad2ff5700 1 -- 172.21.15.12:0/3459004434 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ffae8095290 con 0x55cf4c975b10 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffb01284b80 10 client.17985 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffb01284b80 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffb01284b80 20 client.17985 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffb01284b80 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf4c9e0280 0x55cf4c9e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffb01284b80 4 client.17985 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffb01284b80 1 -- 172.21.15.12:0/3459004434 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cf4c8b3ec0 con 0x55cf4c9e0280 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.294+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf4c9e0280 0x55cf4c9e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffaeed9d700 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf4c9e0280 0x55cf4c9e2670 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c3a7dfa7761f54b server_cookie=30052292022d68c2 in_seq=0 out_seq=0 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffad2ff5700 10 client.17985 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffad2ff5700 1 -- 172.21.15.12:0/3459004434 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cf4c8b3ec0 con 0x55cf4c9e0280 2024-05-08T23:55:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffad2ff5700 10 client.17985 handle_command_reply: tid=0 2024-05-08T23:55:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 client.17985 shutdown 2024-05-08T23:55:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 -- 172.21.15.12:0/3459004434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 msgr2=0x55cf4c97ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c97ef00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ffae80977f0 tx=0x7ffae8004f10 comp rx=0 tx=0).stop 2024-05-08T23:55:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 -- 172.21.15.12:0/3459004434 shutdown_connections 2024-05-08T23:55:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf4c975b10 0x55cf4c97ef00 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cf4c9e0280 0x55cf4c9e2670 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cf4c96fca0 0x55cf4c97e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 --2- 172.21.15.12:0/3459004434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cf4c9751f0 0x55cf4c97e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffad2ff5700 0 client.17985 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:21.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 -- 172.21.15.12:0/3459004434 >> 172.21.15.12:0/3459004434 conn(0x55cf4c8c9210 msgr2=0x55cf4c96a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 -- 172.21.15.12:0/3459004434 shutdown_connections 2024-05-08T23:55:21.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 1 -- 172.21.15.12:0/3459004434 wait complete. 2024-05-08T23:55:21.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.295+0000 7ffb01284b80 20 client.17985 trim_cache size 0 max 16384 2024-05-08T23:55:21.298 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:55:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:55:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:55:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "88e35ffd-5f3d-49b3-9a73-52576e93cebb": { 2024-05-08T23:55:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:55:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "88e35ffd-5f3d-49b3-9a73-52576e93cebb", 2024-05-08T23:55:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:55:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:55:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:55:21.299 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:55:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 -- 172.21.15.12:0/746336026 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffad80a1290 msgr2=0x7ffad80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffad80a1290 0x7ffad80a3750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffae000b2e0 tx=0x7ffae0005fd0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 -- 172.21.15.12:0/746336026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaf019beb0 msgr2=0x7ffaf019c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaf019beb0 0x7ffaf019c330 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ffae8007ee0 tx=0x7ffae800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 -- 172.21.15.12:0/746336026 shutdown_connections 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffad80a1290 0x7ffad80a3750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaf019beb0 0x7ffaf019c330 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf01011e0 0x7ffaf01975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 --2- 172.21.15.12:0/746336026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffaf0106300 0x7ffaf0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 -- 172.21.15.12:0/746336026 >> 172.21.15.12:0/746336026 conn(0x7ffaf00fca80 msgr2=0x7ffaf00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 -- 172.21.15.12:0/746336026 shutdown_connections 2024-05-08T23:55:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:21.296+0000 7ffaf50ca700 1 -- 172.21.15.12:0/746336026 wait complete. 2024-05-08T23:55:21.304 INFO:tasks.cephfs.filesystem:scrub status for tag:88e35ffd-5f3d-49b3-9a73-52576e93cebb - {'path': '/', 'tag': '88e35ffd-5f3d-49b3-9a73-52576e93cebb', 'options': 'recursive,force'} 2024-05-08T23:55:21.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:21 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:55:21.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:21 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:21.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:21 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:55:21.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:21 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:21.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 518996 7.24 MB/sec execute 317 sec latency 3126.178 ms 2024-05-08T23:55:22.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:22 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:55:22.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:22.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:22 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:55:22.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:22 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:55:22.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:22.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:22.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:22 smithi012 ceph-mon[42503]: pgmap v282: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-05-08T23:55:22.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:22 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:55:22.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:22.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:22 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:55:22.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:22 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:55:22.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:22.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:22.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:22 smithi204 ceph-mon[47373]: pgmap v282: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-05-08T23:55:22.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:22 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:55:22.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:22.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:22 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:55:22.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:22 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:55:22.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:22.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:22.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:22 smithi160 ceph-mon[46152]: pgmap v282: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-05-08T23:55:22.879 INFO:tasks.workunit.client.0.smithi012.stdout: 1 520282 7.24 MB/sec execute 318 sec latency 22.622 ms 2024-05-08T23:55:23.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 520365 7.22 MB/sec execute 319 sec latency 947.984 ms 2024-05-08T23:55:23.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:23 smithi012 ceph-mon[42503]: pgmap v283: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 906 KiB/s wr, 112 op/s 2024-05-08T23:55:24.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:23 smithi204 ceph-mon[47373]: pgmap v283: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 906 KiB/s wr, 112 op/s 2024-05-08T23:55:24.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:23 smithi160 ceph-mon[46152]: pgmap v283: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 906 KiB/s wr, 112 op/s 2024-05-08T23:55:24.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 520365 7.19 MB/sec execute 320 sec latency 1948.111 ms 2024-05-08T23:55:25.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 520365 7.17 MB/sec execute 321 sec latency 2948.211 ms 2024-05-08T23:55:25.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:25 smithi012 ceph-mon[42503]: pgmap v284: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 906 KiB/s wr, 112 op/s 2024-05-08T23:55:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:25 smithi204 ceph-mon[47373]: pgmap v284: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 906 KiB/s wr, 112 op/s 2024-05-08T23:55:26.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:25 smithi160 ceph-mon[46152]: pgmap v284: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 906 KiB/s wr, 112 op/s 2024-05-08T23:55:26.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 522517 7.19 MB/sec execute 322 sec latency 3098.355 ms 2024-05-08T23:55:27.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 523906 7.19 MB/sec execute 323 sec latency 91.438 ms 2024-05-08T23:55:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:27 smithi012 ceph-mon[42503]: pgmap v285: 97 pgs: 97 active+clean; 152 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-08T23:55:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:27 smithi204 ceph-mon[47373]: pgmap v285: 97 pgs: 97 active+clean; 152 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-08T23:55:28.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:27 smithi160 ceph-mon[46152]: pgmap v285: 97 pgs: 97 active+clean; 152 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-08T23:55:28.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 523906 7.16 MB/sec execute 324 sec latency 1091.486 ms 2024-05-08T23:55:28.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:28 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:55:29.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:28 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:55:29.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:28 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:55:29.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 523906 7.14 MB/sec execute 325 sec latency 2091.597 ms 2024-05-08T23:55:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:55:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:55:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:29 smithi012 ceph-mon[42503]: pgmap v286: 97 pgs: 97 active+clean; 152 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 729 KiB/s wr, 105 op/s 2024-05-08T23:55:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:55:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:55:30.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:29 smithi204 ceph-mon[47373]: pgmap v286: 97 pgs: 97 active+clean; 152 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 729 KiB/s wr, 105 op/s 2024-05-08T23:55:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:55:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:55:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:29 smithi160 ceph-mon[46152]: pgmap v286: 97 pgs: 97 active+clean; 152 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 729 KiB/s wr, 105 op/s 2024-05-08T23:55:30.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 523906 7.12 MB/sec execute 326 sec latency 3091.693 ms 2024-05-08T23:55:31.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 526006 7.14 MB/sec execute 327 sec latency 3281.220 ms 2024-05-08T23:55:31.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:31 smithi012 ceph-mon[42503]: pgmap v287: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-08T23:55:32.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:31 smithi204 ceph-mon[47373]: pgmap v287: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-08T23:55:32.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:31 smithi160 ceph-mon[46152]: pgmap v287: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-08T23:55:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.579+0000 7f79a134c700 1 -- 172.21.15.12:0/432839039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f799c1010f0 msgr2=0x7f799c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.579+0000 7f79a134c700 1 --2- 172.21.15.12:0/432839039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f799c1010f0 0x7f799c1014d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7984009990 tx=0x7f7984009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.579+0000 7f79a134c700 1 -- 172.21.15.12:0/432839039 shutdown_connections 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.579+0000 7f79a134c700 1 --2- 172.21.15.12:0/432839039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c106210 0x7f799c10dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.579+0000 7f79a134c700 1 --2- 172.21.15.12:0/432839039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f799c1010f0 0x7f799c1014d0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.579+0000 7f79a134c700 1 --2- 172.21.15.12:0/432839039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f799c101aa0 0x7f799c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.579+0000 7f79a134c700 1 -- 172.21.15.12:0/432839039 >> 172.21.15.12:0/432839039 conn(0x7f799c0fc9d0 msgr2=0x7f799c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.579+0000 7f79a134c700 1 -- 172.21.15.12:0/432839039 shutdown_connections 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.579+0000 7f79a134c700 1 -- 172.21.15.12:0/432839039 wait complete. 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f79a134c700 1 Processor -- start 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f79a134c700 1 -- start start 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f79a134c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c1010f0 0x7f799c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f79a134c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f799c106210 0x7f799c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f79a134c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f799c19df00 0x7f799c19e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f79a134c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c19e980 con 0x7f799c19df00 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f79a134c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c19eaf0 con 0x7f799c106210 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f79a134c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c19ecb0 con 0x7f799c1010f0 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f799b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f799c19df00 0x7f799c19e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f799affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c1010f0 0x7f799c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f799a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f799c106210 0x7f799c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f799affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c1010f0 0x7f799c199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51848/0 (socket says 172.21.15.12:51848) 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.580+0000 7f799affd700 1 -- 172.21.15.12:0/2122898439 learned_addr learned my addr 172.21.15.12:0/2122898439 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f799a7fc700 1 -- 172.21.15.12:0/2122898439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c1010f0 msgr2=0x7f799c199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f799a7fc700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c1010f0 0x7f799c199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f799a7fc700 1 -- 172.21.15.12:0/2122898439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f799c19df00 msgr2=0x7f799c19e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f799a7fc700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f799c19df00 0x7f799c19e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f799a7fc700 1 -- 172.21.15.12:0/2122898439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79840095f0 con 0x7f799c106210 2024-05-08T23:55:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f799a7fc700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f799c106210 0x7f799c199b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f798c00e9c0 tx=0x7f798c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f7993fff700 1 -- 172.21.15.12:0/2122898439 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c00c9c0 con 0x7f799c106210 2024-05-08T23:55:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f7993fff700 1 -- 172.21.15.12:0/2122898439 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f798c010430 con 0x7f799c106210 2024-05-08T23:55:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f79a134c700 1 -- 172.21.15.12:0/2122898439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c1a6210 con 0x7f799c106210 2024-05-08T23:55:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f7993fff700 1 -- 172.21.15.12:0/2122898439 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c018680 con 0x7f799c106210 2024-05-08T23:55:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.581+0000 7f79a134c700 1 -- 172.21.15.12:0/2122898439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c1a66f0 con 0x7f799c106210 2024-05-08T23:55:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.584+0000 7f7993fff700 1 -- 172.21.15.12:0/2122898439 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f798c00cb20 con 0x7f799c106210 2024-05-08T23:55:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.584+0000 7f7993fff700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f79880a1290 0x7f79880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.584+0000 7f799affd700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f79880a1290 0x7f79880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.584+0000 7f7993fff700 1 -- 172.21.15.12:0/2122898439 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f798c014070 con 0x7f799c106210 2024-05-08T23:55:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.585+0000 7f799affd700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f79880a1290 0x7f79880a3750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7984006010 tx=0x7f798400b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.585+0000 7f79ad506b80 1 Processor -- start 2024-05-08T23:55:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.585+0000 7f79ad506b80 1 -- start start 2024-05-08T23:55:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7203d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf71fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf71fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f79ad506b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf71550f0 con 0x55dcf7203950 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f79ad506b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf7152ab0 con 0x55dcf71fe480 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f79ad506b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf714e3a0 con 0x55dcf7204270 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7203d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7203d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33958/0 (socket says 172.21.15.12:33958) 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799affd700 1 -- 172.21.15.12:0/2163295242 learned_addr learned my addr 172.21.15.12:0/2163295242 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799b7fe700 1 --2- 172.21.15.12:0/2163295242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf71fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799a7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf71fdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799affd700 1 -- 172.21.15.12:0/2163295242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 msgr2=0x55dcf71fdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799affd700 1 --2- 172.21.15.12:0/2163295242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf71fdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799affd700 1 -- 172.21.15.12:0/2163295242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 msgr2=0x55dcf71fe8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799affd700 1 --2- 172.21.15.12:0/2163295242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf71fe8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799affd700 1 -- 172.21.15.12:0/2163295242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcf6fcf350 con 0x55dcf7203950 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799b7fe700 1 --2- 172.21.15.12:0/2163295242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf71fe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799a7fc700 1 --2- 172.21.15.12:0/2163295242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf71fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.586+0000 7f799affd700 1 --2- 172.21.15.12:0/2163295242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7203d30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7984005780 tx=0x7f7984020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c173ec6e3d81f8b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.587+0000 7f79917fa700 1 -- 172.21.15.12:0/2163295242 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984004700 con 0x55dcf7203950 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.587+0000 7f79917fa700 1 -- 172.21.15.12:0/2163295242 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f798401e040 con 0x55dcf7203950 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.587+0000 7f79917fa700 1 -- 172.21.15.12:0/2163295242 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798400fa70 con 0x55dcf7203950 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.587+0000 7f79ad506b80 1 -- 172.21.15.12:0/2163295242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 msgr2=0x55dcf7203d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.587+0000 7f79ad506b80 1 --2- 172.21.15.12:0/2163295242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7203d30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7984005780 tx=0x7f7984020000 comp rx=0 tx=0).stop 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 -- 172.21.15.12:0/2163295242 shutdown_connections 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 --2- 172.21.15.12:0/2163295242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf71fe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 --2- 172.21.15.12:0/2163295242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7203d30 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 --2- 172.21.15.12:0/2163295242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf71fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 -- 172.21.15.12:0/2163295242 >> 172.21.15.12:0/2163295242 conn(0x55dcf715c1b0 msgr2=0x55dcf71f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 -- 172.21.15.12:0/2163295242 shutdown_connections 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 -- 172.21.15.12:0/2163295242 wait complete. 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 -- start start 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 Processor -- start 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf720a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7207550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf7207c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf70ea500 con 0x55dcf7203950 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf70ea1a0 con 0x55dcf71fe480 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf6fd1640 con 0x55dcf7204270 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f79ad506b80 10 client.0 _ll_register_callbacks cb 0x55dcf70f3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f799affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf720a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.588+0000 7f799a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7207550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f799affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf720a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:36628/0 (socket says 172.21.15.12:36628) 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f799affd700 1 -- 172.21.15.12:0/3867103615 learned_addr learned my addr 172.21.15.12:0/3867103615 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f799b7fe700 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf7207c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f799affd700 1 -- 172.21.15.12:0/3867103615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 msgr2=0x55dcf7207c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f799affd700 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf7207c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f799affd700 1 -- 172.21.15.12:0/3867103615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 msgr2=0x55dcf7207550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f799affd700 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7207550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f799affd700 1 -- 172.21.15.12:0/3867103615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c1a66f0 con 0x55dcf71fe480 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f799a7fc700 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7207550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f799affd700 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf720a260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f79840053e0 tx=0x7f798400f9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f85fa534bec275 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f79827fc700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f79827fc700 1 -- 172.21.15.12:0/3867103615 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984004910 con 0x55dcf71fe480 2024-05-08T23:55:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f79ad506b80 1 -- 172.21.15.12:0/3867103615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcf70a7210 con 0x55dcf71fe480 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f79827fc700 1 -- 172.21.15.12:0/3867103615 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f798400be60 con 0x55dcf71fe480 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f79827fc700 1 -- 172.21.15.12:0/3867103615 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7984004a70 con 0x55dcf71fe480 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f79ad506b80 10 client.27770 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.589+0000 7f79ad506b80 1 -- 172.21.15.12:0/3867103615 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f799c10e600 con 0x55dcf71fe480 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f79827fc700 1 -- 172.21.15.12:0/3867103615 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7984047020 con 0x55dcf71fe480 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f79827fc700 1 -- 172.21.15.12:0/3867103615 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7984038c00 con 0x55dcf71fe480 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f79ad506b80 10 client.27770 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f79ad506b80 10 client.27770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f79ad506b80 20 client.27770 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dcf7217690 0x55dcf7219a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f79ad506b80 4 client.27770 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f79ad506b80 1 -- 172.21.15.12:0/3867103615 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcf71cd7b0 con 0x55dcf7217690 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f799a7fc700 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dcf7217690 0x55dcf7219a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f79827fc700 10 client.27770 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.590+0000 7f799a7fc700 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dcf7217690 0x55dcf7219a80 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f61f9c535556be server_cookie=7ea4096d0c3f8c4d in_seq=0 out_seq=0 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79827fc700 1 -- 172.21.15.12:0/3867103615 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dcf71cd7b0 con 0x55dcf7217690 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79827fc700 10 client.27770 handle_command_reply: tid=0 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 client.27770 shutdown 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 -- 172.21.15.12:0/3867103615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 msgr2=0x55dcf720a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf720a260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f79840053e0 tx=0x7f798400f9e0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 -- 172.21.15.12:0/3867103615 shutdown_connections 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf7207c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dcf7217690 0x55dcf7219a80 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf71fe480 0x55dcf720a260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3867103615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf7207550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79827fc700 0 client.27770 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 -- 172.21.15.12:0/3867103615 >> 172.21.15.12:0/3867103615 conn(0x55dcf715c1b0 msgr2=0x55dcf71fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 -- 172.21.15.12:0/3867103615 shutdown_connections 2024-05-08T23:55:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.591+0000 7f79ad506b80 1 -- 172.21.15.12:0/3867103615 wait complete. 2024-05-08T23:55:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.592+0000 7f79ad506b80 20 client.27770 trim_cache size 0 max 16384 2024-05-08T23:55:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f79ad506b80 1 Processor -- start 2024-05-08T23:55:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f79ad506b80 1 -- start start 2024-05-08T23:55:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf71fe480 0x55dcf71f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7203950 0x55dcf7213630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf71fe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f79ad506b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf71feda0 con 0x55dcf71fe480 2024-05-08T23:55:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f79ad506b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf721ebd0 con 0x55dcf7203950 2024-05-08T23:55:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f79ad506b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf7205aa0 con 0x55dcf7204270 2024-05-08T23:55:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f799affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf71fe480 0x55dcf71f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f799affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf71fe480 0x55dcf71f8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33984/0 (socket says 172.21.15.12:33984) 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f799a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7203950 0x55dcf7213630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f799affd700 1 -- 172.21.15.12:0/1421621686 learned_addr learned my addr 172.21.15.12:0/1421621686 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.611+0000 7f799b7fe700 1 --2- 172.21.15.12:0/1421621686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf71fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f799a7fc700 1 -- 172.21.15.12:0/1421621686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 msgr2=0x55dcf71fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f799a7fc700 1 --2- 172.21.15.12:0/1421621686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf71fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f799a7fc700 1 -- 172.21.15.12:0/1421621686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf71fe480 msgr2=0x55dcf71f8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f799a7fc700 1 --2- 172.21.15.12:0/1421621686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf71fe480 0x55dcf71f8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f799a7fc700 1 -- 172.21.15.12:0/1421621686 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcf6f7bb00 con 0x55dcf7203950 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f799b7fe700 1 --2- 172.21.15.12:0/1421621686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf71fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f799affd700 1 --2- 172.21.15.12:0/1421621686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf71fe480 0x55dcf71f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f799a7fc700 1 --2- 172.21.15.12:0/1421621686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7203950 0x55dcf7213630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f798c0ca550 tx=0x7f798c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36fca1292baa618 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f7991ffb700 1 -- 172.21.15.12:0/1421621686 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c09dd40 con 0x55dcf7203950 2024-05-08T23:55:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f7991ffb700 1 -- 172.21.15.12:0/1421621686 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f798c0a2390 con 0x55dcf7203950 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.612+0000 7f7991ffb700 1 -- 172.21.15.12:0/1421621686 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c0106c0 con 0x55dcf7203950 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.613+0000 7f79ad506b80 1 -- 172.21.15.12:0/1421621686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7203950 msgr2=0x55dcf7213630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.613+0000 7f79ad506b80 1 --2- 172.21.15.12:0/1421621686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7203950 0x55dcf7213630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f798c0ca550 tx=0x7f798c01a040 comp rx=0 tx=0).stop 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.613+0000 7f79ad506b80 1 -- 172.21.15.12:0/1421621686 shutdown_connections 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.613+0000 7f79ad506b80 1 --2- 172.21.15.12:0/1421621686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf7204270 0x55dcf71fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.613+0000 7f79ad506b80 1 --2- 172.21.15.12:0/1421621686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf71fe480 0x55dcf71f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.613+0000 7f79ad506b80 1 --2- 172.21.15.12:0/1421621686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7203950 0x55dcf7213630 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.613+0000 7f79ad506b80 1 -- 172.21.15.12:0/1421621686 >> 172.21.15.12:0/1421621686 conn(0x55dcf715c1b0 msgr2=0x55dcf71fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.613+0000 7f79ad506b80 1 -- 172.21.15.12:0/1421621686 shutdown_connections 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.613+0000 7f79ad506b80 1 -- 172.21.15.12:0/1421621686 wait complete. 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.613+0000 7f79ad506b80 1 -- start start 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f79ad506b80 1 Processor -- start 2024-05-08T23:55:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf71fe480 0x55dcf720cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf720d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f79ad506b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7204270 0x55dcf720d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f79ad506b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf726cba0 con 0x55dcf7203950 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f79ad506b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf726cd10 con 0x55dcf7204270 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f79ad506b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf726ced0 con 0x55dcf71fe480 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f79ad506b80 10 client.0 _ll_register_callbacks cb 0x55dcf70f3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f799a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf720d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f799a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf720d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33986/0 (socket says 172.21.15.12:33986) 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f799a7fc700 1 -- 172.21.15.12:0/3389299811 learned_addr learned my addr 172.21.15.12:0/3389299811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f799b7fe700 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7204270 0x55dcf720d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f799affd700 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf71fe480 0x55dcf720cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f799a7fc700 1 -- 172.21.15.12:0/3389299811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf71fe480 msgr2=0x55dcf720cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f799a7fc700 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf71fe480 0x55dcf720cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f799a7fc700 1 -- 172.21.15.12:0/3389299811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7204270 msgr2=0x55dcf720d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f799a7fc700 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7204270 0x55dcf720d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.614+0000 7f799a7fc700 1 -- 172.21.15.12:0/3389299811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c10e600 con 0x55dcf7203950 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f799a7fc700 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf720d0a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f798c09a7c0 tx=0x7f798c099ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3e1ab5e7a0a145 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79827fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79827fc700 1 -- 172.21.15.12:0/3389299811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c09dd40 con 0x55dcf7203950 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79827fc700 1 -- 172.21.15.12:0/3389299811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f798c0982b0 con 0x55dcf7203950 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79ad506b80 1 -- 172.21.15.12:0/3389299811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f798401e040 con 0x55dcf7203950 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79827fc700 1 -- 172.21.15.12:0/3389299811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f798c010860 con 0x55dcf7203950 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79ad506b80 10 client.18003 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79ad506b80 1 -- 172.21.15.12:0/3389299811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f7994017430 con 0x55dcf7203950 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79827fc700 1 -- 172.21.15.12:0/3389299811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f798401e040 con 0x55dcf7203950 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79827fc700 1 -- 172.21.15.12:0/3389299811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f798c080080 con 0x55dcf7203950 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79ad506b80 10 client.18003 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79ad506b80 10 client.18003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79ad506b80 20 client.18003 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dcf726e3a0 0x55dcf72707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79ad506b80 4 client.18003 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:55:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.615+0000 7f79ad506b80 1 -- 172.21.15.12:0/3389299811 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dcf7146e60 con 0x55dcf726e3a0 2024-05-08T23:55:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.616+0000 7f799affd700 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dcf726e3a0 0x55dcf72707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.616+0000 7f799affd700 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dcf726e3a0 0x55dcf72707c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ca31ab79dd750d server_cookie=6e24008a42b07f1c in_seq=0 out_seq=0 2024-05-08T23:55:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.616+0000 7f79827fc700 10 client.18003 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79827fc700 1 -- 172.21.15.12:0/3389299811 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dcf7146e60 con 0x55dcf726e3a0 2024-05-08T23:55:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79827fc700 10 client.18003 handle_command_reply: tid=0 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 client.18003 shutdown 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 -- 172.21.15.12:0/3389299811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 msgr2=0x55dcf720d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf720d0a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f798c09a7c0 tx=0x7f798c099ff0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 -- 172.21.15.12:0/3389299811 shutdown_connections 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55dcf726e3a0 0x55dcf72707c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dcf7204270 0x55dcf720d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dcf71fe480 0x55dcf720cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 --2- 172.21.15.12:0/3389299811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dcf7203950 0x55dcf720d0a0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79827fc700 0 client.18003 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 -- 172.21.15.12:0/3389299811 >> 172.21.15.12:0/3389299811 conn(0x55dcf715c1b0 msgr2=0x55dcf72013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 -- 172.21.15.12:0/3389299811 shutdown_connections 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 1 -- 172.21.15.12:0/3389299811 wait complete. 2024-05-08T23:55:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.658+0000 7f79ad506b80 20 client.18003 trim_cache size 0 max 16384 2024-05-08T23:55:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:55:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:55:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:55:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 -- 172.21.15.12:0/2122898439 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f79880a1290 msgr2=0x7f79880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f79880a1290 0x7f79880a3750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7984006010 tx=0x7f798400b260 comp rx=0 tx=0).stop 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 -- 172.21.15.12:0/2122898439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f799c106210 msgr2=0x7f799c199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f799c106210 0x7f799c199b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f798c00e9c0 tx=0x7f798c00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 -- 172.21.15.12:0/2122898439 shutdown_connections 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f79880a1290 0x7f79880a3750 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f799c19df00 0x7f799c19e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c1010f0 0x7f799c199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 --2- 172.21.15.12:0/2122898439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f799c106210 0x7f799c199b60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 -- 172.21.15.12:0/2122898439 >> 172.21.15.12:0/2122898439 conn(0x7f799c0fc9d0 msgr2=0x7f799c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 -- 172.21.15.12:0/2122898439 shutdown_connections 2024-05-08T23:55:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:32.659+0000 7f79a134c700 1 -- 172.21.15.12:0/2122898439 wait complete. 2024-05-08T23:55:32.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 527373 7.13 MB/sec execute 328 sec latency 36.442 ms 2024-05-08T23:55:33.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 527456 7.11 MB/sec execute 329 sec latency 944.048 ms 2024-05-08T23:55:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:33 smithi012 ceph-mon[42503]: pgmap v288: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 935 KiB/s wr, 76 op/s 2024-05-08T23:55:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:33 smithi204 ceph-mon[47373]: pgmap v288: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 935 KiB/s wr, 76 op/s 2024-05-08T23:55:34.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:33 smithi160 ceph-mon[46152]: pgmap v288: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 935 KiB/s wr, 76 op/s 2024-05-08T23:55:34.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 527456 7.09 MB/sec execute 330 sec latency 1944.142 ms 2024-05-08T23:55:35.880 INFO:tasks.workunit.client.0.smithi012.stdout: 1 527456 7.07 MB/sec execute 331 sec latency 2944.193 ms 2024-05-08T23:55:35.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:35 smithi012 ceph-mon[42503]: pgmap v289: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 935 KiB/s wr, 76 op/s 2024-05-08T23:55:35.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:35 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:55:36.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:35 smithi204 ceph-mon[47373]: pgmap v289: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 935 KiB/s wr, 76 op/s 2024-05-08T23:55:36.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:35 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:55:36.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:35 smithi160 ceph-mon[46152]: pgmap v289: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 935 KiB/s wr, 76 op/s 2024-05-08T23:55:36.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:35 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:55:36.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 529576 7.09 MB/sec execute 332 sec latency 3087.885 ms 2024-05-08T23:55:36.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:36 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:55:36.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:36 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:55:36.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:36 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:55:37.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:36 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:55:37.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:36 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:55:37.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:36 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:55:37.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:36 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:55:37.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:36 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:55:37.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:36 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:55:37.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 530999 7.08 MB/sec execute 333 sec latency 24.776 ms 2024-05-08T23:55:37.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:37 smithi012 ceph-mon[42503]: pgmap v290: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-08T23:55:38.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:37 smithi204 ceph-mon[47373]: pgmap v290: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-08T23:55:38.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:37 smithi160 ceph-mon[46152]: pgmap v290: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-08T23:55:38.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 530999 7.06 MB/sec execute 334 sec latency 1009.167 ms 2024-05-08T23:55:39.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 530999 7.04 MB/sec execute 335 sec latency 2009.264 ms 2024-05-08T23:55:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:39 smithi012 ceph-mon[42503]: pgmap v291: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 981 KiB/s wr, 106 op/s 2024-05-08T23:55:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:39 smithi204 ceph-mon[47373]: pgmap v291: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 981 KiB/s wr, 106 op/s 2024-05-08T23:55:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:39 smithi160 ceph-mon[46152]: pgmap v291: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 981 KiB/s wr, 106 op/s 2024-05-08T23:55:40.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 530999 7.02 MB/sec execute 336 sec latency 3009.317 ms 2024-05-08T23:55:41.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 533096 7.04 MB/sec execute 337 sec latency 3160.274 ms 2024-05-08T23:55:41.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:41 smithi012 ceph-mon[42503]: pgmap v292: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.7 MiB/s wr, 154 op/s 2024-05-08T23:55:42.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:41 smithi204 ceph-mon[47373]: pgmap v292: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.7 MiB/s wr, 154 op/s 2024-05-08T23:55:42.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:41 smithi160 ceph-mon[46152]: pgmap v292: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.7 MiB/s wr, 154 op/s 2024-05-08T23:55:42.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 534417 7.04 MB/sec execute 338 sec latency 18.602 ms 2024-05-08T23:55:43.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 534579 7.02 MB/sec execute 339 sec latency 898.959 ms 2024-05-08T23:55:43.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:43 smithi012 ceph-mon[42503]: pgmap v293: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-05-08T23:55:44.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:43 smithi204 ceph-mon[47373]: pgmap v293: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-05-08T23:55:44.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:43 smithi160 ceph-mon[46152]: pgmap v293: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-05-08T23:55:44.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 534579 7.00 MB/sec execute 340 sec latency 1899.108 ms 2024-05-08T23:55:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.791+0000 7f39c98e4700 1 -- 172.21.15.12:0/498007495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c4105e60 msgr2=0x7f39c4106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.791+0000 7f39c98e4700 1 --2- 172.21.15.12:0/498007495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c4105e60 0x7f39c4106240 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f39b400b0a0 tx=0x7f39b400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.791+0000 7f39c98e4700 1 -- 172.21.15.12:0/498007495 shutdown_connections 2024-05-08T23:55:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.791+0000 7f39c98e4700 1 --2- 172.21.15.12:0/498007495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c4100760 0x7f39c410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.791+0000 7f39c98e4700 1 --2- 172.21.15.12:0/498007495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c4105e60 0x7f39c4106240 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.791+0000 7f39c98e4700 1 --2- 172.21.15.12:0/498007495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c40ffda0 0x7f39c4100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.792+0000 7f39c98e4700 1 -- 172.21.15.12:0/498007495 >> 172.21.15.12:0/498007495 conn(0x7f39c4077000 msgr2=0x7f39c4077410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.792+0000 7f39c98e4700 1 -- 172.21.15.12:0/498007495 shutdown_connections 2024-05-08T23:55:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.792+0000 7f39c98e4700 1 -- 172.21.15.12:0/498007495 wait complete. 2024-05-08T23:55:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.792+0000 7f39c98e4700 1 Processor -- start 2024-05-08T23:55:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.792+0000 7f39c98e4700 1 -- start start 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c98e4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c40ffda0 0x7f39c41973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c98e4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c4100760 0x7f39c4197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c98e4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c4105e60 0x7f39c419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c98e4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c419c300 con 0x7f39c4100760 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c98e4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c419c470 con 0x7f39c4105e60 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c98e4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c419c630 con 0x7f39c40ffda0 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c4100760 0x7f39c4197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c4100760 0x7f39c4197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59208/0 (socket says 172.21.15.12:59208) 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c27fc700 1 -- 172.21.15.12:0/550121320 learned_addr learned my addr 172.21.15.12:0/550121320 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c40ffda0 0x7f39c41973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c37fe700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c4105e60 0x7f39c419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c27fc700 1 -- 172.21.15.12:0/550121320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c40ffda0 msgr2=0x7f39c41973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c27fc700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c40ffda0 0x7f39c41973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c27fc700 1 -- 172.21.15.12:0/550121320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c4105e60 msgr2=0x7f39c419bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c27fc700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c4105e60 0x7f39c419bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c27fc700 1 -- 172.21.15.12:0/550121320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4009d00 con 0x7f39c4100760 2024-05-08T23:55:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c27fc700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c4100760 0x7f39c4197920 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f39b800e9a0 tx=0x7f39b800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.793+0000 7f39c88e2700 1 -- 172.21.15.12:0/550121320 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8009790 con 0x7f39c4100760 2024-05-08T23:55:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.794+0000 7f39c98e4700 1 -- 172.21.15.12:0/550121320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c419c910 con 0x7f39c4100760 2024-05-08T23:55:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.794+0000 7f39c88e2700 1 -- 172.21.15.12:0/550121320 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b800cbd0 con 0x7f39c4100760 2024-05-08T23:55:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.794+0000 7f39c88e2700 1 -- 172.21.15.12:0/550121320 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8018680 con 0x7f39c4100760 2024-05-08T23:55:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.794+0000 7f39c98e4700 1 -- 172.21.15.12:0/550121320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c41a4150 con 0x7f39c4100760 2024-05-08T23:55:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.797+0000 7f39c88e2700 1 -- 172.21.15.12:0/550121320 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f39b800cd40 con 0x7f39c4100760 2024-05-08T23:55:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.797+0000 7f39c88e2700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 0x7f39ac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.797+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 0x7f39ac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.797+0000 7f39c88e2700 1 -- 172.21.15.12:0/550121320 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f39b8014070 con 0x7f39c4100760 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 0x7f39ac0a3750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f39b4006130 tx=0x7f39b40060a0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39d5a9eb80 1 Processor -- start 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39d5a9eb80 1 -- start start 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b89e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8a2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b7f10f0 con 0x562e2b89e540 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b7eeab0 con 0x562e2b89ee60 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b7ea3a0 con 0x562e2b8a2e40 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39c2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b89e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39c2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b89e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59216/0 (socket says 172.21.15.12:59216) 2024-05-08T23:55:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3397604398 learned_addr learned my addr 172.21.15.12:0/3397604398 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.798+0000 7f39c27fc700 1 --2- 172.21.15.12:0/3397604398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39c37fe700 1 --2- 172.21.15.12:0/3397604398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3397604398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 msgr2=0x562e2b8a32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/3397604398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8a32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3397604398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 msgr2=0x562e2b8a2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/3397604398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8a2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3397604398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e2b66b350 con 0x562e2b89e540 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39c27fc700 1 --2- 172.21.15.12:0/3397604398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8a2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/3397604398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b89e920 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f39b4015000 tx=0x7f39b401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2eda5fff4b947e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39a9ffb700 1 -- 172.21.15.12:0/3397604398 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4007ea0 con 0x562e2b89e540 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39a9ffb700 1 -- 172.21.15.12:0/3397604398 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b4007960 con 0x562e2b89e540 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.799+0000 7f39a9ffb700 1 -- 172.21.15.12:0/3397604398 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4005640 con 0x562e2b89e540 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.800+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3397604398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 msgr2=0x562e2b89e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.800+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/3397604398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b89e920 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f39b4015000 tx=0x7f39b401b000 comp rx=0 tx=0).stop 2024-05-08T23:55:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.800+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3397604398 shutdown_connections 2024-05-08T23:55:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.800+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/3397604398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8a32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.800+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/3397604398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b89e920 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.800+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/3397604398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8a2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.800+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3397604398 >> 172.21.15.12:0/3397604398 conn(0x562e2b7f81b0 msgr2=0x562e2b894840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.800+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3397604398 shutdown_connections 2024-05-08T23:55:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.800+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3397604398 wait complete. 2024-05-08T23:55:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.800+0000 7f39d5a9eb80 1 -- start start 2024-05-08T23:55:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39d5a9eb80 1 Processor -- start 2024-05-08T23:55:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8aa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b786500 con 0x562e2b89e540 2024-05-08T23:55:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b7861a0 con 0x562e2b89ee60 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b66d640 con 0x562e2b8a2e40 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39d5a9eb80 10 client.0 _ll_register_callbacks cb 0x562e2b78fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39c2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39c2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8aa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59218/0 (socket says 172.21.15.12:59218) 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3286494726 learned_addr learned my addr 172.21.15.12:0/3286494726 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39c37fe700 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.801+0000 7f39c27fc700 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3286494726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 msgr2=0x562e2b8a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3286494726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 msgr2=0x562e2b8a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3286494726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c41a4150 con 0x562e2b89e540 2024-05-08T23:55:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8aa680 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f39b4007ad0 tx=0x7f39b4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18c64b7ea22ac9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39a2ffd700 1 -- 172.21.15.12:0/3286494726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4038070 con 0x562e2b89e540 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39a2ffd700 1 -- 172.21.15.12:0/3286494726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f39b4023040 con 0x562e2b89e540 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3286494726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e2b743210 con 0x562e2b89e540 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39a2ffd700 1 -- 172.21.15.12:0/3286494726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39b4005000 con 0x562e2b89e540 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39d5a9eb80 10 client.18015 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3286494726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f39c410d4f0 con 0x562e2b89e540 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.802+0000 7f39a2ffd700 1 -- 172.21.15.12:0/3286494726 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e2b743210 con 0x562e2b89e540 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.803+0000 7f39a2ffd700 1 -- 172.21.15.12:0/3286494726 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f39b403b020 con 0x562e2b89e540 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.803+0000 7f39d5a9eb80 10 client.18015 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.803+0000 7f39d5a9eb80 10 client.18015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.803+0000 7f39d5a9eb80 20 client.18015 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.803+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e2b8b7c00 0x562e2b8ba020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.803+0000 7f39d5a9eb80 4 client.18015 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.803+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3286494726 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e2b8697b0 con 0x562e2b8b7c00 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.803+0000 7f39c27fc700 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e2b8b7c00 0x562e2b8ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.803+0000 7f39c27fc700 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e2b8b7c00 0x562e2b8ba020 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b68bd1285088335 server_cookie=b48b69b10764e68c in_seq=0 out_seq=0 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.803+0000 7f39a2ffd700 10 client.18015 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39a2ffd700 1 -- 172.21.15.12:0/3286494726 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e2b8697b0 con 0x562e2b8b7c00 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39a2ffd700 10 client.18015 handle_command_reply: tid=0 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 client.18015 shutdown 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3286494726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 msgr2=0x562e2b8aa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8aa680 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f39b4007ad0 tx=0x7f39b4004d00 comp rx=0 tx=0).stop 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3286494726 shutdown_connections 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8aa680 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e2b8b7c00 0x562e2b8ba020 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/3286494726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39a2ffd700 0 client.18015 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3286494726 >> 172.21.15.12:0/3286494726 conn(0x562e2b7f81b0 msgr2=0x562e2b898f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3286494726 shutdown_connections 2024-05-08T23:55:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.804+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/3286494726 wait complete. 2024-05-08T23:55:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.805+0000 7f39d5a9eb80 20 client.18015 trim_cache size 0 max 16384 2024-05-08T23:55:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.820+0000 7f39d5a9eb80 1 Processor -- start 2024-05-08T23:55:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.820+0000 7f39d5a9eb80 1 -- start start 2024-05-08T23:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89e540 0x562e2b8a0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b89ee60 0x562e2b8b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b8a2e40 0x562e2b899880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b8a8f70 con 0x562e2b8a2e40 2024-05-08T23:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b896340 con 0x562e2b89e540 2024-05-08T23:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b8a8960 con 0x562e2b89ee60 2024-05-08T23:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b8a2e40 0x562e2b899880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b8a2e40 0x562e2b899880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59222/0 (socket says 172.21.15.12:59222) 2024-05-08T23:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c37fe700 1 -- 172.21.15.12:0/897018288 learned_addr learned my addr 172.21.15.12:0/897018288 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89e540 0x562e2b8a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c27fc700 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b89ee60 0x562e2b8b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c37fe700 1 -- 172.21.15.12:0/897018288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b89ee60 msgr2=0x562e2b8b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c37fe700 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b89ee60 0x562e2b8b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c37fe700 1 -- 172.21.15.12:0/897018288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89e540 msgr2=0x562e2b8a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c37fe700 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89e540 0x562e2b8a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c37fe700 1 -- 172.21.15.12:0/897018288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e2b617b00 con 0x562e2b8a2e40 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89e540 0x562e2b8a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c27fc700 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b89ee60 0x562e2b8b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39c37fe700 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b8a2e40 0x562e2b899880 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f39bc011b50 tx=0x7f39bc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7ddd9bc2ccbb4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39aa7fc700 1 -- 172.21.15.12:0/897018288 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc013b90 con 0x562e2b8a2e40 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.821+0000 7f39aa7fc700 1 -- 172.21.15.12:0/897018288 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39bc013cf0 con 0x562e2b8a2e40 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39aa7fc700 1 -- 172.21.15.12:0/897018288 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc00f620 con 0x562e2b8a2e40 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/897018288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b8a2e40 msgr2=0x562e2b899880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b8a2e40 0x562e2b899880 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f39bc011b50 tx=0x7f39bc011e60 comp rx=0 tx=0).stop 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/897018288 shutdown_connections 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b8a2e40 0x562e2b899880 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89e540 0x562e2b8a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/897018288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b89ee60 0x562e2b8b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/897018288 >> 172.21.15.12:0/897018288 conn(0x562e2b7f81b0 msgr2=0x562e2b8be940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/897018288 shutdown_connections 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/897018288 wait complete. 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.822+0000 7f39d5a9eb80 1 -- start start 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39d5a9eb80 1 Processor -- start 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39d5a9eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b8af8e0 con 0x562e2b89e540 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b90d0c0 con 0x562e2b89ee60 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39d5a9eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2b90d250 con 0x562e2b8a2e40 2024-05-08T23:55:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39d5a9eb80 10 client.0 _ll_register_callbacks cb 0x562e2b78fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39c2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39c2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8acf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59228/0 (socket says 172.21.15.12:59228) 2024-05-08T23:55:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39c2ffd700 1 -- 172.21.15.12:0/1639543179 learned_addr learned my addr 172.21.15.12:0/1639543179 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39c37fe700 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39c27fc700 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39c2ffd700 1 -- 172.21.15.12:0/1639543179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 msgr2=0x562e2b8ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.823+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8ada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39c2ffd700 1 -- 172.21.15.12:0/1639543179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 msgr2=0x562e2b8ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39c2ffd700 1 -- 172.21.15.12:0/1639543179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc011780 con 0x562e2b89e540 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39c27fc700 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8acf90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009fd0 tx=0x7f39b40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8228e1b98b557d2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39a2ffd700 1 -- 172.21.15.12:0/1639543179 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4038060 con 0x562e2b89e540 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39a2ffd700 1 -- 172.21.15.12:0/1639543179 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f39b4005000 con 0x562e2b89e540 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/1639543179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39b4007960 con 0x562e2b89e540 2024-05-08T23:55:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39a2ffd700 1 -- 172.21.15.12:0/1639543179 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39b4032b50 con 0x562e2b89e540 2024-05-08T23:55:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39d5a9eb80 10 client.18021 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/1639543179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562e2b7b0500 con 0x562e2b89e540 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39a2ffd700 1 -- 172.21.15.12:0/1639543179 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b4007960 con 0x562e2b89e540 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39a2ffd700 1 -- 172.21.15.12:0/1639543179 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f39b403b040 con 0x562e2b89e540 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39d5a9eb80 10 client.18021 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39d5a9eb80 10 client.18021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39d5a9eb80 20 client.18021 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e2b90e630 0x562e2b910a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39d5a9eb80 4 client.18021 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.824+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/1639543179 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562e2b7e2e60 con 0x562e2b90e630 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.825+0000 7f39c27fc700 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e2b90e630 0x562e2b910a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.825+0000 7f39c27fc700 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e2b90e630 0x562e2b910a50 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98b4556e6c16241 server_cookie=6e00d8d13b33587e in_seq=0 out_seq=0 2024-05-08T23:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.825+0000 7f39a2ffd700 10 client.18021 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39a2ffd700 1 -- 172.21.15.12:0/1639543179 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562e2b7e2e60 con 0x562e2b90e630 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39a2ffd700 10 client.18021 handle_command_reply: tid=0 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39d5a9eb80 1 client.18021 shutdown 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/1639543179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 msgr2=0x562e2b8acf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8acf90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009fd0 tx=0x7f39b40056d0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/1639543179 shutdown_connections 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e2b8a2e40 0x562e2b8ada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e2b89e540 0x562e2b8acf90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e2b90e630 0x562e2b910a50 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39d5a9eb80 1 --2- 172.21.15.12:0/1639543179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e2b89ee60 0x562e2b8ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39a2ffd700 0 client.18021 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/1639543179 >> 172.21.15.12:0/1639543179 conn(0x562e2b7f81b0 msgr2=0x562e2b896480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.862+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/1639543179 shutdown_connections 2024-05-08T23:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39d5a9eb80 1 -- 172.21.15.12:0/1639543179 wait complete. 2024-05-08T23:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39d5a9eb80 20 client.18021 trim_cache size 0 max 16384 2024-05-08T23:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:55:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 -- 172.21.15.12:0/550121320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 msgr2=0x7f39ac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 0x7f39ac0a3750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f39b4006130 tx=0x7f39b40060a0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 -- 172.21.15.12:0/550121320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c4100760 msgr2=0x7f39c4197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c4100760 0x7f39c4197920 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f39b800e9a0 tx=0x7f39b800ed60 comp rx=0 tx=0).stop 2024-05-08T23:55:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 -- 172.21.15.12:0/550121320 shutdown_connections 2024-05-08T23:55:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 0x7f39ac0a3750 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c4105e60 0x7f39c419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c40ffda0 0x7f39c41973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 --2- 172.21.15.12:0/550121320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c4100760 0x7f39c4197920 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 -- 172.21.15.12:0/550121320 >> 172.21.15.12:0/550121320 conn(0x7f39c4077000 msgr2=0x7f39c40ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 -- 172.21.15.12:0/550121320 shutdown_connections 2024-05-08T23:55:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:45.863+0000 7f39c98e4700 1 -- 172.21.15.12:0/550121320 wait complete. 2024-05-08T23:55:45.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 534579 6.97 MB/sec execute 341 sec latency 2899.155 ms 2024-05-08T23:55:45.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:45 smithi012 ceph-mon[42503]: pgmap v294: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-05-08T23:55:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:45 smithi204 ceph-mon[47373]: pgmap v294: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-05-08T23:55:46.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:45 smithi160 ceph-mon[46152]: pgmap v294: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-05-08T23:55:46.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 536655 6.99 MB/sec execute 342 sec latency 3047.236 ms 2024-05-08T23:55:47.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 537833 6.99 MB/sec execute 343 sec latency 55.886 ms 2024-05-08T23:55:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:47 smithi012 ceph-mon[42503]: pgmap v295: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-05-08T23:55:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:47 smithi204 ceph-mon[47373]: pgmap v295: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-05-08T23:55:48.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:47 smithi160 ceph-mon[46152]: pgmap v295: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-05-08T23:55:48.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 538122 6.97 MB/sec execute 344 sec latency 785.802 ms 2024-05-08T23:55:49.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 538122 6.95 MB/sec execute 345 sec latency 1785.843 ms 2024-05-08T23:55:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:49 smithi012 ceph-mon[42503]: pgmap v296: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-05-08T23:55:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:49 smithi204 ceph-mon[47373]: pgmap v296: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-05-08T23:55:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:49 smithi160 ceph-mon[46152]: pgmap v296: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-05-08T23:55:50.881 INFO:tasks.workunit.client.0.smithi012.stdout: 1 538122 6.93 MB/sec execute 346 sec latency 2785.907 ms 2024-05-08T23:55:51.306 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:55:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.465+0000 7f860684c700 1 -- 172.21.15.12:0/3272080788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8600106300 msgr2=0x7f860010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.465+0000 7f860684c700 1 --2- 172.21.15.12:0/3272080788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8600106300 0x7f860010de80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b0a0 tx=0x7f85f800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.465+0000 7f860684c700 1 -- 172.21.15.12:0/3272080788 shutdown_connections 2024-05-08T23:55:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.465+0000 7f860684c700 1 --2- 172.21.15.12:0/3272080788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8600106300 0x7f860010de80 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.465+0000 7f860684c700 1 --2- 172.21.15.12:0/3272080788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86001011e0 0x7f86001015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.465+0000 7f860684c700 1 --2- 172.21.15.12:0/3272080788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8600101b90 0x7f8600105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.465+0000 7f860684c700 1 -- 172.21.15.12:0/3272080788 >> 172.21.15.12:0/3272080788 conn(0x7f86000fca80 msgr2=0x7f86000feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.465+0000 7f860684c700 1 -- 172.21.15.12:0/3272080788 shutdown_connections 2024-05-08T23:55:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.465+0000 7f860684c700 1 -- 172.21.15.12:0/3272080788 wait complete. 2024-05-08T23:55:51.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.465+0000 7f860684c700 1 Processor -- start 2024-05-08T23:55:51.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f860684c700 1 -- start start 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f860684c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86001011e0 0x7f86001953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f860684c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8600106300 0x7f8600195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f860684c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8600199cb0 0x7f860019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f860684c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860019a730 con 0x7f8600199cb0 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f860684c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860019a8a0 con 0x7f8600106300 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f860684c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860019aa60 con 0x7f86001011e0 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f85ff7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8600106300 0x7f8600195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f85fffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86001011e0 0x7f86001953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f85ff7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8600106300 0x7f8600195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45266/0 (socket says 172.21.15.12:45266) 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f85ff7fe700 1 -- 172.21.15.12:0/2190595758 learned_addr learned my addr 172.21.15.12:0/2190595758 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f8604de9700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8600199cb0 0x7f860019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f85ff7fe700 1 -- 172.21.15.12:0/2190595758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86001011e0 msgr2=0x7f86001953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86001011e0 0x7f86001953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.466+0000 7f85ff7fe700 1 -- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8600199cb0 msgr2=0x7f860019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.467+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8600199cb0 0x7f860019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.467+0000 7f85ff7fe700 1 -- 172.21.15.12:0/2190595758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4009520 con 0x7f8600106300 2024-05-08T23:55:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.467+0000 7f85fffff700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86001011e0 0x7f86001953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.467+0000 7f8604de9700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8600199cb0 0x7f860019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.467+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8600106300 0x7f8600195910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85f400ea30 tx=0x7f85f400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.467+0000 7f85fd7fa700 1 -- 172.21.15.12:0/2190595758 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f400cbd0 con 0x7f8600106300 2024-05-08T23:55:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.467+0000 7f85fd7fa700 1 -- 172.21.15.12:0/2190595758 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85f400eed0 con 0x7f8600106300 2024-05-08T23:55:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.467+0000 7f860684c700 1 -- 172.21.15.12:0/2190595758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f8009d00 con 0x7f8600106300 2024-05-08T23:55:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.467+0000 7f85fd7fa700 1 -- 172.21.15.12:0/2190595758 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4003d20 con 0x7f8600106300 2024-05-08T23:55:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.467+0000 7f860684c700 1 -- 172.21.15.12:0/2190595758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86001a2260 con 0x7f8600106300 2024-05-08T23:55:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.470+0000 7f85fd7fa700 1 -- 172.21.15.12:0/2190595758 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f85f4003ee0 con 0x7f8600106300 2024-05-08T23:55:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.470+0000 7f85fd7fa700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85e80a1290 0x7f85e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.470+0000 7f85fffff700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85e80a1290 0x7f85e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.470+0000 7f85fd7fa700 1 -- 172.21.15.12:0/2190595758 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f85f4014070 con 0x7f8600106300 2024-05-08T23:55:51.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.471+0000 7f85fffff700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85e80a1290 0x7f85e80a3750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f85f000d5c0 tx=0x7f85f0018040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:51.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.471+0000 7f8612a06b80 1 Processor -- start 2024-05-08T23:55:51.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.471+0000 7f8612a06b80 1 -- start start 2024-05-08T23:55:51.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 0x559853d301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 0x559853d2a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2a890 0x559853d2acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8612a06b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853c81150 con 0x559853d2a890 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8612a06b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853c7eb10 con 0x559853d30720 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8612a06b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853c7a400 con 0x559853d2fe00 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8604de9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2a890 0x559853d2acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8604de9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2a890 0x559853d2acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44202/0 (socket says 172.21.15.12:44202) 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8604de9700 1 -- 172.21.15.12:0/786485199 learned_addr learned my addr 172.21.15.12:0/786485199 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f85ff7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 0x559853d2a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f85fffff700 1 --2- 172.21.15.12:0/786485199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 0x559853d301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8604de9700 1 -- 172.21.15.12:0/786485199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 msgr2=0x559853d301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8604de9700 1 --2- 172.21.15.12:0/786485199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 0x559853d301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8604de9700 1 -- 172.21.15.12:0/786485199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 msgr2=0x559853d2a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8604de9700 1 --2- 172.21.15.12:0/786485199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 0x559853d2a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8604de9700 1 -- 172.21.15.12:0/786485199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559853afb3b0 con 0x559853d2a890 2024-05-08T23:55:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/786485199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 0x559853d2a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f8604de9700 1 --2- 172.21.15.12:0/786485199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2a890 0x559853d2acd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b0a0 tx=0x7f85f8007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358a216029027ff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f85e67fc700 1 -- 172.21.15.12:0/786485199 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f800e040 con 0x559853d2a890 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f85e67fc700 1 -- 172.21.15.12:0/786485199 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85f800ba50 con 0x559853d2a890 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.472+0000 7f85e67fc700 1 -- 172.21.15.12:0/786485199 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f801c940 con 0x559853d2a890 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.473+0000 7f8612a06b80 1 -- 172.21.15.12:0/786485199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2a890 msgr2=0x559853d2acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.473+0000 7f8612a06b80 1 --2- 172.21.15.12:0/786485199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2a890 0x559853d2acd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b0a0 tx=0x7f85f8007990 comp rx=0 tx=0).stop 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.473+0000 7f8612a06b80 1 -- 172.21.15.12:0/786485199 shutdown_connections 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.473+0000 7f8612a06b80 1 --2- 172.21.15.12:0/786485199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2a890 0x559853d2acd0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.473+0000 7f8612a06b80 1 --2- 172.21.15.12:0/786485199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 0x559853d301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.473+0000 7f8612a06b80 1 --2- 172.21.15.12:0/786485199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 0x559853d2a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.473+0000 7f8612a06b80 1 -- 172.21.15.12:0/786485199 >> 172.21.15.12:0/786485199 conn(0x559853c88210 msgr2=0x559853d24df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.473+0000 7f8612a06b80 1 -- 172.21.15.12:0/786485199 shutdown_connections 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.473+0000 7f8612a06b80 1 -- 172.21.15.12:0/786485199 wait complete. 2024-05-08T23:55:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f8612a06b80 1 -- start start 2024-05-08T23:55:51.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f8612a06b80 1 Processor -- start 2024-05-08T23:55:51.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d36780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 0x559853d33a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d30720 0x559853d34140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f8612a06b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853c16560 con 0x559853d30720 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f8612a06b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853c16200 con 0x559853d2a890 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f8612a06b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853afd6a0 con 0x559853d2fe00 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f8612a06b80 10 client.0 _ll_register_callbacks cb 0x559853c1fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f85fffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d36780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f8604de9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d30720 0x559853d34140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f85fffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d36780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45282/0 (socket says 172.21.15.12:45282) 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.474+0000 7f85fffff700 1 -- 172.21.15.12:0/1317578539 learned_addr learned my addr 172.21.15.12:0/1317578539 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 0x559853d33a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85fffff700 1 -- 172.21.15.12:0/1317578539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 msgr2=0x559853d33a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85fffff700 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 0x559853d33a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85fffff700 1 -- 172.21.15.12:0/1317578539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d30720 msgr2=0x559853d34140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85fffff700 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d30720 0x559853d34140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85fffff700 1 -- 172.21.15.12:0/1317578539 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86001a2260 con 0x559853d2a890 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 0x559853d33a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85fffff700 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d36780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f85f00216c0 tx=0x7f85f00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72c805298877787a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85df7fe700 1 -- 172.21.15.12:0/1317578539 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0031070 con 0x559853d2a890 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f85df7fe700 1 -- 172.21.15.12:0/1317578539 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f85f0004830 con 0x559853d2a890 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.475+0000 7f8612a06b80 1 -- 172.21.15.12:0/1317578539 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559853bd3270 con 0x559853d2a890 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f85df7fe700 1 -- 172.21.15.12:0/1317578539 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f001b040 con 0x559853d2a890 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f8612a06b80 10 client.27803 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f8612a06b80 1 -- 172.21.15.12:0/1317578539 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f860010e6f0 con 0x559853d2a890 2024-05-08T23:55:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f85df7fe700 1 -- 172.21.15.12:0/1317578539 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559853bd3270 con 0x559853d2a890 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f85df7fe700 1 -- 172.21.15.12:0/1317578539 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f85f0034020 con 0x559853d2a890 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f8612a06b80 10 client.27803 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f8612a06b80 10 client.27803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f8612a06b80 20 client.27803 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f8612a06b80 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559853d441e0 0x559853d46600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f8612a06b80 4 client.27803 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f8612a06b80 1 -- 172.21.15.12:0/1317578539 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559853cf9810 con 0x559853d441e0 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.476+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559853d441e0 0x559853d46600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559853d441e0 0x559853d46600 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c66d274d382ecf server_cookie=32b4e1a6b7c26696 in_seq=0 out_seq=0 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f85df7fe700 10 client.27803 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f85df7fe700 1 -- 172.21.15.12:0/1317578539 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559853cf9810 con 0x559853d441e0 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f85df7fe700 10 client.27803 handle_command_reply: tid=0 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f8612a06b80 1 client.27803 shutdown 2024-05-08T23:55:51.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f8612a06b80 1 -- 172.21.15.12:0/1317578539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 msgr2=0x559853d36780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f8612a06b80 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d36780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f85f00216c0 tx=0x7f85f00047c0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f8612a06b80 1 -- 172.21.15.12:0/1317578539 shutdown_connections 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f8612a06b80 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d30720 0x559853d34140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f8612a06b80 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559853d441e0 0x559853d46600 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f8612a06b80 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d36780 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f8612a06b80 1 --2- 172.21.15.12:0/1317578539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2fe00 0x559853d33a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f85df7fe700 0 client.27803 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.477+0000 7f8612a06b80 1 -- 172.21.15.12:0/1317578539 >> 172.21.15.12:0/1317578539 conn(0x559853c88210 msgr2=0x559853d245a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.478+0000 7f8612a06b80 1 -- 172.21.15.12:0/1317578539 shutdown_connections 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.478+0000 7f8612a06b80 1 -- 172.21.15.12:0/1317578539 wait complete. 2024-05-08T23:55:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.478+0000 7f8612a06b80 20 client.27803 trim_cache size 0 max 16384 2024-05-08T23:55:51.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f8612a06b80 1 Processor -- start 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f8612a06b80 1 -- start start 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2a890 0x559853d24d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d362b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 0x559853d3b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f8612a06b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853d4b7d0 con 0x559853d2fe00 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f8612a06b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853d1f800 con 0x559853d30720 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f8612a06b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853d26640 con 0x559853d2a890 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f85ff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f85ff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d362b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44214/0 (socket says 172.21.15.12:44214) 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f85ff7fe700 1 -- 172.21.15.12:0/357980395 learned_addr learned my addr 172.21.15.12:0/357980395 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.496+0000 7f85fffff700 1 --2- 172.21.15.12:0/357980395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2a890 0x559853d24d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.497+0000 7f8604de9700 1 --2- 172.21.15.12:0/357980395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 0x559853d3b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.497+0000 7f85ff7fe700 1 -- 172.21.15.12:0/357980395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2a890 msgr2=0x559853d24d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.497+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/357980395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2a890 0x559853d24d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.497+0000 7f85ff7fe700 1 -- 172.21.15.12:0/357980395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 msgr2=0x559853d3b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.497+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/357980395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 0x559853d3b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.497+0000 7f85ff7fe700 1 -- 172.21.15.12:0/357980395 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559853aa7b60 con 0x559853d2fe00 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.497+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/357980395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d362b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009fd0 tx=0x7f85f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5c43b4b5534924 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.497+0000 7f85e6ffd700 1 -- 172.21.15.12:0/357980395 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40981d0 con 0x559853d2fe00 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.497+0000 7f85e6ffd700 1 -- 172.21.15.12:0/357980395 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85f4098430 con 0x559853d2fe00 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.497+0000 7f85e6ffd700 1 -- 172.21.15.12:0/357980395 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f409a550 con 0x559853d2fe00 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.498+0000 7f8612a06b80 1 -- 172.21.15.12:0/357980395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 msgr2=0x559853d362b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.498+0000 7f8612a06b80 1 --2- 172.21.15.12:0/357980395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d362b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009fd0 tx=0x7f85f40045e0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.498+0000 7f8612a06b80 1 -- 172.21.15.12:0/357980395 shutdown_connections 2024-05-08T23:55:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.498+0000 7f8612a06b80 1 --2- 172.21.15.12:0/357980395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d30720 0x559853d3b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.498+0000 7f8612a06b80 1 --2- 172.21.15.12:0/357980395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d2a890 0x559853d24d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.498+0000 7f8612a06b80 1 --2- 172.21.15.12:0/357980395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d362b0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.498+0000 7f8612a06b80 1 -- 172.21.15.12:0/357980395 >> 172.21.15.12:0/357980395 conn(0x559853c88210 msgr2=0x559853d3fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:51.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.498+0000 7f8612a06b80 1 -- 172.21.15.12:0/357980395 shutdown_connections 2024-05-08T23:55:51.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.498+0000 7f8612a06b80 1 -- 172.21.15.12:0/357980395 wait complete. 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.498+0000 7f8612a06b80 1 -- start start 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f8612a06b80 1 Processor -- start 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d37370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f8612a06b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d30720 0x559853d93660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f8612a06b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853d93ba0 con 0x559853d2fe00 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f8612a06b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853d93d10 con 0x559853d2a890 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f8612a06b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559853d93ed0 con 0x559853d30720 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f8612a06b80 10 client.0 _ll_register_callbacks cb 0x559853c1fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f85ff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f8604de9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d30720 0x559853d93660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f85ff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d378b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44230/0 (socket says 172.21.15.12:44230) 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f85ff7fe700 1 -- 172.21.15.12:0/778455215 learned_addr learned my addr 172.21.15.12:0/778455215 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.499+0000 7f85fffff700 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d37370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f8604de9700 1 -- 172.21.15.12:0/778455215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 msgr2=0x559853d37370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f8604de9700 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d37370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f8604de9700 1 -- 172.21.15.12:0/778455215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 msgr2=0x559853d378b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f8604de9700 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d378b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f8604de9700 1 -- 172.21.15.12:0/778455215 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f8009d00 con 0x559853d30720 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f85fffff700 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d37370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f85ff7fe700 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f8604de9700 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d30720 0x559853d93660 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b3f0 tx=0x7f85f800bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b911ce399ce0bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f85df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f85df7fe700 1 -- 172.21.15.12:0/778455215 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f800e070 con 0x559853d30720 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f85df7fe700 1 -- 172.21.15.12:0/778455215 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f85f8007d90 con 0x559853d30720 2024-05-08T23:55:51.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f8612a06b80 1 -- 172.21.15.12:0/778455215 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85f800ba50 con 0x559853d30720 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f85df7fe700 1 -- 172.21.15.12:0/778455215 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f8003a00 con 0x559853d30720 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f8612a06b80 10 client.27805 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f8612a06b80 1 -- 172.21.15.12:0/778455215 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559853c40560 con 0x559853d30720 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f85df7fe700 1 -- 172.21.15.12:0/778455215 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85f800ba50 con 0x559853d30720 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.500+0000 7f85df7fe700 1 -- 172.21.15.12:0/778455215 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f85f8019050 con 0x559853d30720 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f8612a06b80 10 client.27805 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f8612a06b80 10 client.27805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f8612a06b80 20 client.27805 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f8612a06b80 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559853d9b280 0x559853d9d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f8612a06b80 4 client.27805 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f8612a06b80 1 -- 172.21.15.12:0/778455215 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559853c72ec0 con 0x559853d9b280 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f85fffff700 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559853d9b280 0x559853d9d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f85df7fe700 10 client.27805 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:51.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f85fffff700 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559853d9b280 0x559853d9d6a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d5690bc2749e51 server_cookie=b85893b49d32d9f5 in_seq=0 out_seq=0 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f85df7fe700 1 -- 172.21.15.12:0/778455215 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559853c72ec0 con 0x559853d9b280 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f85df7fe700 10 client.27805 handle_command_reply: tid=0 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.501+0000 7f8612a06b80 1 client.27805 shutdown 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 1 -- 172.21.15.12:0/778455215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d30720 msgr2=0x559853d93660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d30720 0x559853d93660 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b3f0 tx=0x7f85f800bdf0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 1 -- 172.21.15.12:0/778455215 shutdown_connections 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559853d30720 0x559853d93660 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559853d9b280 0x559853d9d6a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559853d2a890 0x559853d37370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 1 --2- 172.21.15.12:0/778455215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559853d2fe00 0x559853d378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f85df7fe700 0 client.27805 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 1 -- 172.21.15.12:0/778455215 >> 172.21.15.12:0/778455215 conn(0x559853c88210 msgr2=0x559853d2eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 1 -- 172.21.15.12:0/778455215 shutdown_connections 2024-05-08T23:55:51.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 1 -- 172.21.15.12:0/778455215 wait complete. 2024-05-08T23:55:51.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f8612a06b80 20 client.27805 trim_cache size 0 max 16384 2024-05-08T23:55:51.503 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:55:51.503 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:55:51.503 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:55:51.503 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:55:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f860684c700 1 -- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85e80a1290 msgr2=0x7f85e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f860684c700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85e80a1290 0x7f85e80a3750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f85f000d5c0 tx=0x7f85f0018040 comp rx=0 tx=0).stop 2024-05-08T23:55:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f860684c700 1 -- 172.21.15.12:0/2190595758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8600106300 msgr2=0x7f8600195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f860684c700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8600106300 0x7f8600195910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85f400ea30 tx=0x7f85f400ed40 comp rx=0 tx=0).stop 2024-05-08T23:55:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f860684c700 1 -- 172.21.15.12:0/2190595758 shutdown_connections 2024-05-08T23:55:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f860684c700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85e80a1290 0x7f85e80a3750 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f860684c700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8600199cb0 0x7f860019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f860684c700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86001011e0 0x7f86001953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.502+0000 7f860684c700 1 --2- 172.21.15.12:0/2190595758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8600106300 0x7f8600195910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.503+0000 7f860684c700 1 -- 172.21.15.12:0/2190595758 >> 172.21.15.12:0/2190595758 conn(0x7f86000fca80 msgr2=0x7f86000fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.503+0000 7f860684c700 1 -- 172.21.15.12:0/2190595758 shutdown_connections 2024-05-08T23:55:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.503+0000 7f860684c700 1 -- 172.21.15.12:0/2190595758 wait complete. 2024-05-08T23:55:51.511 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:55:51.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:55:51.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.664+0000 7fb4d0661700 1 -- 172.21.15.12:0/488403603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c8101070 msgr2=0x7fb4c8101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.664+0000 7fb4d0661700 1 --2- 172.21.15.12:0/488403603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c8101070 0x7fb4c8101450 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009990 tx=0x7fb4bc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.664+0000 7fb4d0661700 1 -- 172.21.15.12:0/488403603 shutdown_connections 2024-05-08T23:55:51.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.664+0000 7fb4d0661700 1 --2- 172.21.15.12:0/488403603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4c8106190 0x7fb4c810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.664+0000 7fb4d0661700 1 --2- 172.21.15.12:0/488403603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c8101070 0x7fb4c8101450 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.664+0000 7fb4d0661700 1 --2- 172.21.15.12:0/488403603 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4c8101a20 0x7fb4c8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.664+0000 7fb4d0661700 1 -- 172.21.15.12:0/488403603 >> 172.21.15.12:0/488403603 conn(0x7fb4c80fc950 msgr2=0x7fb4c80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:51.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.666+0000 7fb4d0661700 1 -- 172.21.15.12:0/488403603 shutdown_connections 2024-05-08T23:55:51.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.666+0000 7fb4d0661700 1 -- 172.21.15.12:0/488403603 wait complete. 2024-05-08T23:55:51.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.666+0000 7fb4d0661700 1 Processor -- start 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4d0661700 1 -- start start 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4d0661700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4c8101070 0x7fb4c80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4d0661700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c8106190 0x7fb4c80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4d0661700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4c8073d20 0x7fb4c81a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4d0661700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c81a5af0 con 0x7fb4c8106190 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4d0661700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c81a5c60 con 0x7fb4c8073d20 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4d0661700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c81a5e20 con 0x7fb4c8101070 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4cdbfc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c8106190 0x7fb4c80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4ce3fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4c8101070 0x7fb4c80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4cdbfc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c8106190 0x7fb4c80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44244/0 (socket says 172.21.15.12:44244) 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4ce3fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4c8101070 0x7fb4c80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33690/0 (socket says 172.21.15.12:33690) 2024-05-08T23:55:51.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4cdbfc700 1 -- 172.21.15.12:0/11987682 learned_addr learned my addr 172.21.15.12:0/11987682 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4cebfe700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4c8073d20 0x7fb4c81a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4cdbfc700 1 -- 172.21.15.12:0/11987682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4c8101070 msgr2=0x7fb4c80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4cdbfc700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4c8101070 0x7fb4c80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4cdbfc700 1 -- 172.21.15.12:0/11987682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4c8073d20 msgr2=0x7fb4c81a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4cdbfc700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4c8073d20 0x7fb4c81a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4cdbfc700 1 -- 172.21.15.12:0/11987682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c0009c40 con 0x7fb4c8106190 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4ce3fd700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4c8101070 0x7fb4c80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.667+0000 7fb4cebfe700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4c8073d20 0x7fb4c81a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.668+0000 7fb4cdbfc700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c8106190 0x7fb4c80737e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400e970 tx=0x7fb4c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.668+0000 7fb4bb7fe700 1 -- 172.21.15.12:0/11987682 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4009790 con 0x7fb4c8106190 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.668+0000 7fb4bb7fe700 1 -- 172.21.15.12:0/11987682 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4c400c7c0 con 0x7fb4c8106190 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.668+0000 7fb4d0661700 1 -- 172.21.15.12:0/11987682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc0095f0 con 0x7fb4c8106190 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.668+0000 7fb4bb7fe700 1 -- 172.21.15.12:0/11987682 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40054b0 con 0x7fb4c8106190 2024-05-08T23:55:51.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.668+0000 7fb4d0661700 1 -- 172.21.15.12:0/11987682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c81a6480 con 0x7fb4c8106190 2024-05-08T23:55:51.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.669+0000 7fb4d0661700 1 -- 172.21.15.12:0/11987682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c8109400 con 0x7fb4c8106190 2024-05-08T23:55:51.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:51 smithi012 ceph-mon[42503]: pgmap v297: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.0 MiB/s wr, 151 op/s 2024-05-08T23:55:51.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.670+0000 7fb4bb7fe700 1 -- 172.21.15.12:0/11987682 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb4c4005610 con 0x7fb4c8106190 2024-05-08T23:55:51.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.671+0000 7fb4bb7fe700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4b40a1290 0x7fb4b40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:51.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.671+0000 7fb4ce3fd700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4b40a1290 0x7fb4b40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:51.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.671+0000 7fb4bb7fe700 1 -- 172.21.15.12:0/11987682 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb4c4014070 con 0x7fb4c8106190 2024-05-08T23:55:51.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.672+0000 7fb4ce3fd700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4b40a1290 0x7fb4b40a3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc006010 tx=0x7fb4bc005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:51.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.675+0000 7fb4bb7fe700 1 -- 172.21.15.12:0/11987682 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb4c40964f0 con 0x7fb4c8106190 2024-05-08T23:55:51.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 540203 6.95 MB/sec execute 347 sec latency 2930.369 ms 2024-05-08T23:55:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.958+0000 7fb4d0661700 1 -- 172.21.15.12:0/11987682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4c8102540 con 0x7fb4c8106190 2024-05-08T23:55:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.959+0000 7fb4bb7fe700 1 -- 172.21.15.12:0/11987682 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+10883 (secure 0 0 0) 0x7fb4c4099b10 con 0x7fb4c8106190 2024-05-08T23:55:51.954 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:55:51.954 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:54:40.565738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27484":{"gid":27484,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3332734891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3332734891},{"type":"v1","addr":"172.21.15.204:6837","nonce":3332734891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:55:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.961+0000 7fb4d0661700 1 -- 172.21.15.12:0/11987682 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4b40a1290 msgr2=0x7fb4b40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.961+0000 7fb4d0661700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4b40a1290 0x7fb4b40a3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc006010 tx=0x7fb4bc005960 comp rx=0 tx=0).stop 2024-05-08T23:55:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.961+0000 7fb4d0661700 1 -- 172.21.15.12:0/11987682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c8106190 msgr2=0x7fb4c80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.961+0000 7fb4d0661700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c8106190 0x7fb4c80737e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400e970 tx=0x7fb4c400ec80 comp rx=0 tx=0).stop 2024-05-08T23:55:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.961+0000 7fb4d0661700 1 -- 172.21.15.12:0/11987682 shutdown_connections 2024-05-08T23:55:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.961+0000 7fb4d0661700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4b40a1290 0x7fb4b40a3750 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.961+0000 7fb4d0661700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4c8073d20 0x7fb4c81a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.961+0000 7fb4d0661700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4c8101070 0x7fb4c80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.961+0000 7fb4d0661700 1 --2- 172.21.15.12:0/11987682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c8106190 0x7fb4c80737e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.961+0000 7fb4d0661700 1 -- 172.21.15.12:0/11987682 >> 172.21.15.12:0/11987682 conn(0x7fb4c80fc950 msgr2=0x7fb4c80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.962+0000 7fb4d0661700 1 -- 172.21.15.12:0/11987682 shutdown_connections 2024-05-08T23:55:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:51.962+0000 7fb4d0661700 1 -- 172.21.15.12:0/11987682 wait complete. 2024-05-08T23:55:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 40 2024-05-08T23:55:51.971 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:55:52.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:51 smithi204 ceph-mon[47373]: pgmap v297: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.0 MiB/s wr, 151 op/s 2024-05-08T23:55:52.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:51 smithi160 ceph-mon[46152]: pgmap v297: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.0 MiB/s wr, 151 op/s 2024-05-08T23:55:52.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.130+0000 7ff1b0236700 1 -- 172.21.15.12:0/2591475564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8100a10 msgr2=0x7ff1a8104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.130+0000 7ff1b0236700 1 --2- 172.21.15.12:0/2591475564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8100a10 0x7ff1a8104a50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff198009990 tx=0x7ff198009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.130+0000 7ff1b0236700 1 -- 172.21.15.12:0/2591475564 shutdown_connections 2024-05-08T23:55:52.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.130+0000 7ff1b0236700 1 --2- 172.21.15.12:0/2591475564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a8105180 0x7ff1a810cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.130+0000 7ff1b0236700 1 --2- 172.21.15.12:0/2591475564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a8100060 0x7ff1a8100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.130+0000 7ff1b0236700 1 --2- 172.21.15.12:0/2591475564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8100a10 0x7ff1a8104a50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.130+0000 7ff1b0236700 1 -- 172.21.15.12:0/2591475564 >> 172.21.15.12:0/2591475564 conn(0x7ff1a8076fa0 msgr2=0x7ff1a80773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.130+0000 7ff1b0236700 1 -- 172.21.15.12:0/2591475564 shutdown_connections 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.130+0000 7ff1b0236700 1 -- 172.21.15.12:0/2591475564 wait complete. 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1b0236700 1 Processor -- start 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1b0236700 1 -- start start 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1b0236700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8100060 0x7ff1a810d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1b0236700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a8105180 0x7ff1a810db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1b0236700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a810e0c0 0x7ff1a81a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1b0236700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81a5b00 con 0x7ff1a810e0c0 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1b0236700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81a5c70 con 0x7ff1a8100060 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1b0236700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81a5e30 con 0x7ff1a8105180 2024-05-08T23:55:52.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1ae7d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a810e0c0 0x7ff1a81a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1adfd2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8100060 0x7ff1a810d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1ad7d1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a8105180 0x7ff1a810db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1ae7d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a810e0c0 0x7ff1a81a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44264/0 (socket says 172.21.15.12:44264) 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1adfd2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8100060 0x7ff1a810d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45332/0 (socket says 172.21.15.12:45332) 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.131+0000 7ff1ae7d3700 1 -- 172.21.15.12:0/2641317886 learned_addr learned my addr 172.21.15.12:0/2641317886 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff1ae7d3700 1 -- 172.21.15.12:0/2641317886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a8105180 msgr2=0x7ff1a810db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff1ae7d3700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a8105180 0x7ff1a810db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff1ae7d3700 1 -- 172.21.15.12:0/2641317886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8100060 msgr2=0x7ff1a810d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff1ae7d3700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8100060 0x7ff1a810d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff1ae7d3700 1 -- 172.21.15.12:0/2641317886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980095f0 con 0x7ff1a810e0c0 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8100060 0x7ff1a810d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff1ae7d3700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a810e0c0 0x7ff1a81a5500 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000ef30 tx=0x7ff1a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff19effd700 1 -- 172.21.15.12:0/2641317886 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0018070 con 0x7ff1a810e0c0 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff1b0236700 1 -- 172.21.15.12:0/2641317886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a81a60b0 con 0x7ff1a810e0c0 2024-05-08T23:55:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff19effd700 1 -- 172.21.15.12:0/2641317886 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a000f040 con 0x7ff1a810e0c0 2024-05-08T23:55:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff19effd700 1 -- 172.21.15.12:0/2641317886 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a000cd00 con 0x7ff1a810e0c0 2024-05-08T23:55:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.132+0000 7ff1b0236700 1 -- 172.21.15.12:0/2641317886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a81a6540 con 0x7ff1a810e0c0 2024-05-08T23:55:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.135+0000 7ff19effd700 1 -- 172.21.15.12:0/2641317886 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff1a0003a00 con 0x7ff1a810e0c0 2024-05-08T23:55:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.135+0000 7ff19effd700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1940a1290 0x7ff1940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.135+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1940a1290 0x7ff1940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.135+0000 7ff19effd700 1 -- 172.21.15.12:0/2641317886 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff1a00cc7d0 con 0x7ff1a810e0c0 2024-05-08T23:55:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.136+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1940a1290 0x7ff1940a3750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4011490 tx=0x7ff1a4009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.136+0000 7ff1bc3f0b80 1 Processor -- start 2024-05-08T23:55:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.136+0000 7ff1bc3f0b80 1 -- start start 2024-05-08T23:55:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f5191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f519720 0x55643f513350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f513890 0x55643f513cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f46a150 con 0x55643f519720 2024-05-08T23:55:52.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f467b10 con 0x55643f513890 2024-05-08T23:55:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f463400 con 0x55643f518e00 2024-05-08T23:55:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1ad7d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f519720 0x55643f513350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1ae7d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f513890 0x55643f513cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1ad7d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f519720 0x55643f513350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44268/0 (socket says 172.21.15.12:44268) 2024-05-08T23:55:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1ae7d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f513890 0x55643f513cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45334/0 (socket says 172.21.15.12:45334) 2024-05-08T23:55:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1ad7d1700 1 -- 172.21.15.12:0/676988069 learned_addr learned my addr 172.21.15.12:0/676988069 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/676988069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f5191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1ae7d3700 1 -- 172.21.15.12:0/676988069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 msgr2=0x55643f5191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1ae7d3700 1 --2- 172.21.15.12:0/676988069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f5191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1ae7d3700 1 -- 172.21.15.12:0/676988069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f519720 msgr2=0x55643f513350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1ae7d3700 1 --2- 172.21.15.12:0/676988069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f519720 0x55643f513350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.137+0000 7ff1ae7d3700 1 -- 172.21.15.12:0/676988069 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55643f2e43b0 con 0x55643f513890 2024-05-08T23:55:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.138+0000 7ff1ae7d3700 1 --2- 172.21.15.12:0/676988069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f513890 0x55643f513cd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00c9f30 tx=0x7ff1a00994c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a17f76b468991ee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.138+0000 7ff193fff700 1 -- 172.21.15.12:0/676988069 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0004ad0 con 0x55643f513890 2024-05-08T23:55:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.138+0000 7ff193fff700 1 -- 172.21.15.12:0/676988069 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a001b320 con 0x55643f513890 2024-05-08T23:55:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.138+0000 7ff193fff700 1 -- 172.21.15.12:0/676988069 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0099be0 con 0x55643f513890 2024-05-08T23:55:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/676988069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f513890 msgr2=0x55643f513cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/676988069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f513890 0x55643f513cd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00c9f30 tx=0x7ff1a00994c0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/676988069 shutdown_connections 2024-05-08T23:55:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/676988069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f513890 0x55643f513cd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/676988069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f5191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/676988069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f519720 0x55643f513350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/676988069 >> 172.21.15.12:0/676988069 conn(0x55643f471210 msgr2=0x55643f50ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/676988069 shutdown_connections 2024-05-08T23:55:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/676988069 wait complete. 2024-05-08T23:55:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 -- start start 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.139+0000 7ff1bc3f0b80 1 Processor -- start 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f51ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f51cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f51d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f3ff560 con 0x55643f513890 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f3ff200 con 0x55643f519720 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f2e66a0 con 0x55643f518e00 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1bc3f0b80 10 client.0 _ll_register_callbacks cb 0x55643f408b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1adfd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f51ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1ad7d1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f51cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1adfd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f51ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44272/0 (socket says 172.21.15.12:44272) 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1adfd2700 1 -- 172.21.15.12:0/2049680005 learned_addr learned my addr 172.21.15.12:0/2049680005 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1ae7d3700 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f51d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1adfd2700 1 -- 172.21.15.12:0/2049680005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 msgr2=0x55643f51cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f51cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1adfd2700 1 -- 172.21.15.12:0/2049680005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 msgr2=0x55643f51d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f51d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1adfd2700 1 -- 172.21.15.12:0/2049680005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a81a6540 con 0x55643f513890 2024-05-08T23:55:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1ad7d1700 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f51cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f51ca90 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4003ad0 tx=0x7ff1a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563d692c5a938b6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.140+0000 7ff1917fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1917fa700 1 -- 172.21.15.12:0/2049680005 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a401fa90 con 0x55643f513890 2024-05-08T23:55:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1917fa700 1 -- 172.21.15.12:0/2049680005 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff1a401fbf0 con 0x55643f513890 2024-05-08T23:55:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2049680005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55643f3bc270 con 0x55643f513890 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1917fa700 1 -- 172.21.15.12:0/2049680005 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1a401fdb0 con 0x55643f513890 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1bc3f0b80 10 client.18057 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2049680005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55643f290b60 con 0x55643f513890 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1917fa700 1 -- 172.21.15.12:0/2049680005 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a4034080 con 0x55643f513890 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1917fa700 1 -- 172.21.15.12:0/2049680005 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff1a4030780 con 0x55643f513890 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1bc3f0b80 10 client.18057 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1bc3f0b80 10 client.18057 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1bc3f0b80 20 client.18057 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55643f52d310 0x55643f52f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1bc3f0b80 4 client.18057 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2049680005 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55643f4e2810 con 0x55643f52d310 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.141+0000 7ff1ad7d1700 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55643f52d310 0x55643f52f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.142+0000 7ff1ad7d1700 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55643f52d310 0x55643f52f730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d42b3c27744b040a server_cookie=f0e8eb6153cab7ef in_seq=0 out_seq=0 2024-05-08T23:55:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.142+0000 7ff1917fa700 10 client.18057 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:55:52.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1917fa700 1 -- 172.21.15.12:0/2049680005 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55643f4e2810 con 0x55643f52d310 2024-05-08T23:55:52.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1917fa700 10 client.18057 handle_command_reply: tid=0 2024-05-08T23:55:52.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 client.18057 shutdown 2024-05-08T23:55:52.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2049680005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 msgr2=0x55643f51ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f51ca90 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4003ad0 tx=0x7ff1a40047c0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2049680005 shutdown_connections 2024-05-08T23:55:52.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f51d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f51ca90 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55643f52d310 0x55643f52f730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2049680005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f51cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1917fa700 0 client.18057 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:55:52.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2049680005 >> 172.21.15.12:0/2049680005 conn(0x55643f471210 msgr2=0x55643f50d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2049680005 shutdown_connections 2024-05-08T23:55:52.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2049680005 wait complete. 2024-05-08T23:55:52.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.143+0000 7ff1bc3f0b80 20 client.18057 trim_cache size 0 max 16384 2024-05-08T23:55:52.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.157+0000 7ff1bc3f0b80 1 Processor -- start 2024-05-08T23:55:52.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.157+0000 7ff1bc3f0b80 1 -- start start 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f513350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f513c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f5182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f52a940 con 0x55643f513890 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f50fe50 con 0x55643f519720 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f534940 con 0x55643f518e00 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1adfd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f513350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1adfd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f513350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44284/0 (socket says 172.21.15.12:44284) 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1adfd2700 1 -- 172.21.15.12:0/4286952479 learned_addr learned my addr 172.21.15.12:0/4286952479 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1ae7d3700 1 --2- 172.21.15.12:0/4286952479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f5182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1ad7d1700 1 --2- 172.21.15.12:0/4286952479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f513c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1adfd2700 1 -- 172.21.15.12:0/4286952479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 msgr2=0x55643f513c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/4286952479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f513c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1adfd2700 1 -- 172.21.15.12:0/4286952479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 msgr2=0x55643f5182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/4286952479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f5182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1adfd2700 1 -- 172.21.15.12:0/4286952479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55643f429560 con 0x55643f513890 2024-05-08T23:55:52.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/4286952479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f513350 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4000c00 tx=0x7ff1a4011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3602f659a0ebbd12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.158+0000 7ff19cff9700 1 -- 172.21.15.12:0/4286952479 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4028850 con 0x55643f513890 2024-05-08T23:55:52.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff19cff9700 1 -- 172.21.15.12:0/4286952479 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55643f3bc270 con 0x55643f513890 2024-05-08T23:55:52.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff19cff9700 1 -- 172.21.15.12:0/4286952479 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a401fe30 con 0x55643f513890 2024-05-08T23:55:52.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/4286952479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 msgr2=0x55643f513350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/4286952479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f513350 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4000c00 tx=0x7ff1a4011d10 comp rx=0 tx=0).stop 2024-05-08T23:55:52.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/4286952479 shutdown_connections 2024-05-08T23:55:52.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/4286952479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f5182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/4286952479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f513350 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/4286952479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f513c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/4286952479 >> 172.21.15.12:0/4286952479 conn(0x55643f471210 msgr2=0x55643f517ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/4286952479 shutdown_connections 2024-05-08T23:55:52.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.159+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/4286952479 wait complete. 2024-05-08T23:55:52.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1bc3f0b80 1 -- start start 2024-05-08T23:55:52.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1bc3f0b80 1 Processor -- start 2024-05-08T23:55:52.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f524470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f520140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1bc3f0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f520680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f520d10 con 0x55643f513890 2024-05-08T23:55:52.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f520e80 con 0x55643f519720 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1bc3f0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f582800 con 0x55643f518e00 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1bc3f0b80 10 client.0 _ll_register_callbacks cb 0x55643f408b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1adfd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f524470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1adfd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f524470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44288/0 (socket says 172.21.15.12:44288) 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1adfd2700 1 -- 172.21.15.12:0/2281331053 learned_addr learned my addr 172.21.15.12:0/2281331053 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.160+0000 7ff1ae7d3700 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f520680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1ad7d1700 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f520140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1adfd2700 1 -- 172.21.15.12:0/2281331053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 msgr2=0x55643f520140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f520140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1adfd2700 1 -- 172.21.15.12:0/2281331053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 msgr2=0x55643f520680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f520680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1adfd2700 1 -- 172.21.15.12:0/2281331053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55643f290b60 con 0x55643f513890 2024-05-08T23:55:52.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1ae7d3700 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f520680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1ad7d1700 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f520140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1adfd2700 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f524470 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4030b30 tx=0x7ff1a401f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00c120790b9ce70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1917fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1917fa700 1 -- 172.21.15.12:0/2281331053 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40305f0 con 0x55643f513890 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1917fa700 1 -- 172.21.15.12:0/2281331053 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff1a4028aa0 con 0x55643f513890 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2281331053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1a001b320 con 0x55643f513890 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1917fa700 1 -- 172.21.15.12:0/2281331053 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1a4030750 con 0x55643f513890 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1bc3f0b80 10 client.18063 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2281331053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55643f2e8cd0 con 0x55643f513890 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.161+0000 7ff1917fa700 1 -- 172.21.15.12:0/2281331053 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a001b320 con 0x55643f513890 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.162+0000 7ff1917fa700 1 -- 172.21.15.12:0/2281331053 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff1a4028d20 con 0x55643f513890 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.162+0000 7ff1bc3f0b80 10 client.18063 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.162+0000 7ff1bc3f0b80 10 client.18063 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:55:52.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.162+0000 7ff1bc3f0b80 20 client.18063 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.162+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55643f583ae0 0x55643f585f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.162+0000 7ff1bc3f0b80 4 client.18063 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.162+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2281331053 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55643f45bec0 con 0x55643f583ae0 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.162+0000 7ff1ad7d1700 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55643f583ae0 0x55643f585f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.162+0000 7ff1917fa700 10 client.18063 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.162+0000 7ff1ad7d1700 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55643f583ae0 0x55643f585f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d863ec85af0780c server_cookie=d04221290a18f6f6 in_seq=0 out_seq=0 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1917fa700 1 -- 172.21.15.12:0/2281331053 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55643f45bec0 con 0x55643f583ae0 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1917fa700 10 client.18063 handle_command_reply: tid=0 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 client.18063 shutdown 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2281331053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 msgr2=0x55643f524470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f524470 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4030b30 tx=0x7ff1a401f910 comp rx=0 tx=0).stop 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2281331053 shutdown_connections 2024-05-08T23:55:52.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55643f519720 0x55643f520680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55643f513890 0x55643f524470 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55643f583ae0 0x55643f585f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 --2- 172.21.15.12:0/2281331053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55643f518e00 0x55643f520140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1917fa700 0 client.18063 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:55:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2281331053 >> 172.21.15.12:0/2281331053 conn(0x55643f471210 msgr2=0x55643f519bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2281331053 shutdown_connections 2024-05-08T23:55:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 1 -- 172.21.15.12:0/2281331053 wait complete. 2024-05-08T23:55:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.163+0000 7ff1bc3f0b80 20 client.18063 trim_cache size 0 max 16384 2024-05-08T23:55:52.165 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:55:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 -- 172.21.15.12:0/2641317886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1940a1290 msgr2=0x7ff1940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1940a1290 0x7ff1940a3750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4011490 tx=0x7ff1a4009260 comp rx=0 tx=0).stop 2024-05-08T23:55:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 -- 172.21.15.12:0/2641317886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a810e0c0 msgr2=0x7ff1a81a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a810e0c0 0x7ff1a81a5500 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000ef30 tx=0x7ff1a000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 -- 172.21.15.12:0/2641317886 shutdown_connections 2024-05-08T23:55:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1940a1290 0x7ff1940a3750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a810e0c0 0x7ff1a81a5500 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8100060 0x7ff1a810d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 --2- 172.21.15.12:0/2641317886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a8105180 0x7ff1a810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 -- 172.21.15.12:0/2641317886 >> 172.21.15.12:0/2641317886 conn(0x7ff1a8076fa0 msgr2=0x7ff1a80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 -- 172.21.15.12:0/2641317886 shutdown_connections 2024-05-08T23:55:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.164+0000 7ff1b0236700 1 -- 172.21.15.12:0/2641317886 wait complete. 2024-05-08T23:55:52.172 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:55:52.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.321+0000 7f80f8895700 1 -- 172.21.15.12:0/553890839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80f4100060 msgr2=0x7f80f4100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.321+0000 7f80f8895700 1 --2- 172.21.15.12:0/553890839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80f4100060 0x7f80f4100440 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009990 tx=0x7f80e4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.321+0000 7f80f8895700 1 -- 172.21.15.12:0/553890839 shutdown_connections 2024-05-08T23:55:52.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.321+0000 7f80f8895700 1 --2- 172.21.15.12:0/553890839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80f4105160 0x7f80f410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.321+0000 7f80f8895700 1 --2- 172.21.15.12:0/553890839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80f4100060 0x7f80f4100440 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.321+0000 7f80f8895700 1 --2- 172.21.15.12:0/553890839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f4100a10 0x7f80f4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.321+0000 7f80f8895700 1 -- 172.21.15.12:0/553890839 >> 172.21.15.12:0/553890839 conn(0x7f80f4076f40 msgr2=0x7f80f4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.321+0000 7f80f8895700 1 -- 172.21.15.12:0/553890839 shutdown_connections 2024-05-08T23:55:52.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.321+0000 7f80f8895700 1 -- 172.21.15.12:0/553890839 wait complete. 2024-05-08T23:55:52.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.322+0000 7f80f8895700 1 Processor -- start 2024-05-08T23:55:52.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.322+0000 7f80f8895700 1 -- start start 2024-05-08T23:55:52.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.322+0000 7f80f8895700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f4100060 0x7f80f4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.322+0000 7f80f8895700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80f4105160 0x7f80f4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.322+0000 7f80f8895700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80f4198cb0 0x7f80f41a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.322+0000 7f80f8895700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41a3950 con 0x7f80f4105160 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.322+0000 7f80f8895700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41a3ac0 con 0x7f80f4100060 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.322+0000 7f80f8895700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41a3c80 con 0x7f80f4198cb0 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80f4105160 0x7f80f4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80f4105160 0x7f80f4198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44304/0 (socket says 172.21.15.12:44304) 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f1d9b700 1 -- 172.21.15.12:0/2903959754 learned_addr learned my addr 172.21.15.12:0/2903959754 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80f4198cb0 0x7f80f41a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f259c700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f4100060 0x7f80f4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f1d9b700 1 -- 172.21.15.12:0/2903959754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80f4198cb0 msgr2=0x7f80f41a32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f1d9b700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80f4198cb0 0x7f80f41a32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f1d9b700 1 -- 172.21.15.12:0/2903959754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f4100060 msgr2=0x7f80f4198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f1d9b700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f4100060 0x7f80f4198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f1d9b700 1 -- 172.21.15.12:0/2903959754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40095f0 con 0x7f80f4105160 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80f4198cb0 0x7f80f41a32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f259c700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f4100060 0x7f80f4198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:52.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f1d9b700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80f4105160 0x7f80f4198770 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00e970 tx=0x7f80dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80eb7fe700 1 -- 172.21.15.12:0/2903959754 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc009790 con 0x7f80f4105160 2024-05-08T23:55:52.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80eb7fe700 1 -- 172.21.15.12:0/2903959754 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80dc00eec0 con 0x7f80f4105160 2024-05-08T23:55:52.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f8895700 1 -- 172.21.15.12:0/2903959754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f41a3f60 con 0x7f80f4105160 2024-05-08T23:55:52.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80eb7fe700 1 -- 172.21.15.12:0/2903959754 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc005590 con 0x7f80f4105160 2024-05-08T23:55:52.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.323+0000 7f80f8895700 1 -- 172.21.15.12:0/2903959754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f41a43e0 con 0x7f80f4105160 2024-05-08T23:55:52.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.326+0000 7f80eb7fe700 1 -- 172.21.15.12:0/2903959754 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f80dc021440 con 0x7f80f4105160 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.326+0000 7f80eb7fe700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f80e00a1290 0x7f80e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.327+0000 7f80eb7fe700 1 -- 172.21.15.12:0/2903959754 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f80dc014070 con 0x7f80f4105160 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.327+0000 7f80f259c700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f80e00a1290 0x7f80e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.327+0000 7f80f259c700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f80e00a1290 0x7f80e00a3750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f80e400b2e0 tx=0x7f80e4005940 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.327+0000 7f8104a4fb80 1 Processor -- start 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f8104a4fb80 1 -- start start 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d6acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819cc1150 con 0x560819d70720 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819cbeb10 con 0x560819d6a890 2024-05-08T23:55:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819cba400 con 0x560819d6fe00 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44320/0 (socket says 172.21.15.12:44320) 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f1d9b700 1 -- 172.21.15.12:0/2133725306 learned_addr learned my addr 172.21.15.12:0/2133725306 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f259c700 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d6acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f259c700 1 -- 172.21.15.12:0/2133725306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 msgr2=0x560819d6acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f259c700 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d6acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f259c700 1 -- 172.21.15.12:0/2133725306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 msgr2=0x560819d6a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f259c700 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f259c700 1 -- 172.21.15.12:0/2133725306 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560819b3b3b0 con 0x560819d6fe00 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d6acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.328+0000 7f80f1d9b700 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.329+0000 7f80f259c700 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d701e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80e4006010 tx=0x7f80e401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b749e57b4bcd5166 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.329+0000 7f80e8ff9700 1 -- 172.21.15.12:0/2133725306 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e402b070 con 0x560819d6fe00 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.329+0000 7f80e8ff9700 1 -- 172.21.15.12:0/2133725306 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e401e040 con 0x560819d6fe00 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.329+0000 7f80e8ff9700 1 -- 172.21.15.12:0/2133725306 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4004cf0 con 0x560819d6fe00 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 -- 172.21.15.12:0/2133725306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 msgr2=0x560819d701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d701e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80e4006010 tx=0x7f80e401f000 comp rx=0 tx=0).stop 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 -- 172.21.15.12:0/2133725306 shutdown_connections 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d6acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d701e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/2133725306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 -- 172.21.15.12:0/2133725306 >> 172.21.15.12:0/2133725306 conn(0x560819cc8210 msgr2=0x560819d64df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 -- 172.21.15.12:0/2133725306 shutdown_connections 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 -- 172.21.15.12:0/2133725306 wait complete. 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 -- start start 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.330+0000 7f8104a4fb80 1 Processor -- start 2024-05-08T23:55:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d6a890 0x560819d73b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d70720 0x560819d745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819c56560 con 0x560819d6a890 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819c56200 con 0x560819d70720 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819b3d6a0 con 0x560819d6fe00 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f8104a4fb80 10 client.0 _ll_register_callbacks cb 0x560819c5fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d6a890 0x560819d73b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33776/0 (socket says 172.21.15.12:33776) 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f1d9b700 1 -- 172.21.15.12:0/743851928 learned_addr learned my addr 172.21.15.12:0/743851928 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d70720 0x560819d745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f259c700 1 -- 172.21.15.12:0/743851928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 msgr2=0x560819d740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f259c700 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f259c700 1 -- 172.21.15.12:0/743851928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d70720 msgr2=0x560819d745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f259c700 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d70720 0x560819d745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f259c700 1 -- 172.21.15.12:0/743851928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f41a43e0 con 0x560819d6a890 2024-05-08T23:55:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f1d9b700 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80f259c700 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d6a890 0x560819d73b60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f80e4004fd0 tx=0x7f80e400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90de4eb2a28a773d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80d9ffb700 1 -- 172.21.15.12:0/743851928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e402b040 con 0x560819d6a890 2024-05-08T23:55:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f80d9ffb700 1 -- 172.21.15.12:0/743851928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f80e4005460 con 0x560819d6a890 2024-05-08T23:55:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.331+0000 7f8104a4fb80 1 -- 172.21.15.12:0/743851928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560819c13270 con 0x560819d6a890 2024-05-08T23:55:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f80d9ffb700 1 -- 172.21.15.12:0/743851928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e4004760 con 0x560819d6a890 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f8104a4fb80 10 client.18075 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f8104a4fb80 1 -- 172.21.15.12:0/743851928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f80f410d550 con 0x560819d6a890 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f80d9ffb700 1 -- 172.21.15.12:0/743851928 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e4041020 con 0x560819d6a890 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f80d9ffb700 1 -- 172.21.15.12:0/743851928 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f80e40306c0 con 0x560819d6a890 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f8104a4fb80 10 client.18075 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f8104a4fb80 10 client.18075 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f8104a4fb80 20 client.18075 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560819d83d30 0x560819d86120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f8104a4fb80 4 client.18075 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f8104a4fb80 1 -- 172.21.15.12:0/743851928 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560819d39810 con 0x560819d83d30 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.332+0000 7f80f1d9b700 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560819d83d30 0x560819d86120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.333+0000 7f80f1d9b700 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560819d83d30 0x560819d86120 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9a21e16fe46396f server_cookie=4112799b4392c126 in_seq=0 out_seq=0 2024-05-08T23:55:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.333+0000 7f80d9ffb700 10 client.18075 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.333+0000 7f80d9ffb700 1 -- 172.21.15.12:0/743851928 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560819d39810 con 0x560819d83d30 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.333+0000 7f80d9ffb700 10 client.18075 handle_command_reply: tid=0 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.333+0000 7f8104a4fb80 1 client.18075 shutdown 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.333+0000 7f8104a4fb80 1 -- 172.21.15.12:0/743851928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d6a890 msgr2=0x560819d73b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.333+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d6a890 0x560819d73b60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f80e4004fd0 tx=0x7f80e400bf70 comp rx=0 tx=0).stop 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.334+0000 7f8104a4fb80 1 -- 172.21.15.12:0/743851928 shutdown_connections 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.334+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560819d83d30 0x560819d86120 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.334+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d70720 0x560819d745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.334+0000 7f80d9ffb700 0 client.18075 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.334+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d6a890 0x560819d73b60 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.334+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/743851928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.334+0000 7f8104a4fb80 1 -- 172.21.15.12:0/743851928 >> 172.21.15.12:0/743851928 conn(0x560819cc8210 msgr2=0x560819d645a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.334+0000 7f8104a4fb80 1 -- 172.21.15.12:0/743851928 shutdown_connections 2024-05-08T23:55:52.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.334+0000 7f8104a4fb80 1 -- 172.21.15.12:0/743851928 wait complete. 2024-05-08T23:55:52.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.334+0000 7f8104a4fb80 20 client.18075 trim_cache size 0 max 16384 2024-05-08T23:55:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.348+0000 7f8104a4fb80 1 Processor -- start 2024-05-08T23:55:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.348+0000 7f8104a4fb80 1 -- start start 2024-05-08T23:55:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.348+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d64d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.348+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d74400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.348+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.348+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819d75130 con 0x560819d70720 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.348+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819d720b0 con 0x560819d6a890 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.348+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819d71b10 con 0x560819d6fe00 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44342/0 (socket says 172.21.15.12:44342) 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f2d9d700 1 -- 172.21.15.12:0/2026816648 learned_addr learned my addr 172.21.15.12:0/2026816648 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f259c700 1 --2- 172.21.15.12:0/2026816648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d64d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f1d9b700 1 --2- 172.21.15.12:0/2026816648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d74400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f2d9d700 1 -- 172.21.15.12:0/2026816648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 msgr2=0x560819d74400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/2026816648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d74400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f2d9d700 1 -- 172.21.15.12:0/2026816648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 msgr2=0x560819d64d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/2026816648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d64d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f2d9d700 1 -- 172.21.15.12:0/2026816648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560819ae7b60 con 0x560819d70720 2024-05-08T23:55:52.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/2026816648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6ec50 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00eaf0 tx=0x7f80ec009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc39dc2a0466f6ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80e97fa700 1 -- 172.21.15.12:0/2026816648 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec017c90 con 0x560819d70720 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80e97fa700 1 -- 172.21.15.12:0/2026816648 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80ec00f040 con 0x560819d70720 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.349+0000 7f80e97fa700 1 -- 172.21.15.12:0/2026816648 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec007640 con 0x560819d70720 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 -- 172.21.15.12:0/2026816648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 msgr2=0x560819d6ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/2026816648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6ec50 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00eaf0 tx=0x7f80ec009920 comp rx=0 tx=0).stop 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 -- 172.21.15.12:0/2026816648 shutdown_connections 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/2026816648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819d6ec50 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/2026816648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d64d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/2026816648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d74400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 -- 172.21.15.12:0/2026816648 >> 172.21.15.12:0/2026816648 conn(0x560819cc8210 msgr2=0x560819d72430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 -- 172.21.15.12:0/2026816648 shutdown_connections 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 -- 172.21.15.12:0/2026816648 wait complete. 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 -- start start 2024-05-08T23:55:52.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.350+0000 7f8104a4fb80 1 Processor -- start 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d792e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d79820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f8104a4fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819ddd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819d79ef0 con 0x560819d70720 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819dddb50 con 0x560819d6a890 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f8104a4fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560819dddc90 con 0x560819d6fe00 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f8104a4fb80 10 client.0 _ll_register_callbacks cb 0x560819c5fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819ddd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d79820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819ddd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44346/0 (socket says 172.21.15.12:44346) 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f2d9d700 1 -- 172.21.15.12:0/213351628 learned_addr learned my addr 172.21.15.12:0/213351628 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f2d9d700 1 -- 172.21.15.12:0/213351628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 msgr2=0x560819d79820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d79820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f2d9d700 1 -- 172.21.15.12:0/213351628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 msgr2=0x560819d792e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d792e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f2d9d700 1 -- 172.21.15.12:0/213351628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec00e7a0 con 0x560819d70720 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80f2d9d700 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819ddd610 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f80ec022040 tx=0x7f80ec01f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1386f59a34b163d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.351+0000 7f80d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f80d9ffb700 1 -- 172.21.15.12:0/213351628 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec004cd0 con 0x560819d70720 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f80d9ffb700 1 -- 172.21.15.12:0/213351628 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f80ec01f960 con 0x560819d70720 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f8104a4fb80 1 -- 172.21.15.12:0/213351628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80e401e040 con 0x560819d70720 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f80d9ffb700 1 -- 172.21.15.12:0/213351628 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80ec01fb50 con 0x560819d70720 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f8104a4fb80 10 client.18081 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:52.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f8104a4fb80 1 -- 172.21.15.12:0/213351628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560819c80560 con 0x560819d70720 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f80d9ffb700 1 -- 172.21.15.12:0/213351628 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e401e040 con 0x560819d70720 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f80d9ffb700 1 -- 172.21.15.12:0/213351628 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f80ec029400 con 0x560819d70720 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f8104a4fb80 10 client.18081 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f8104a4fb80 10 client.18081 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f8104a4fb80 20 client.18081 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560819ddf2e0 0x560819de1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f8104a4fb80 4 client.18081 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f8104a4fb80 1 -- 172.21.15.12:0/213351628 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560819cb2ec0 con 0x560819ddf2e0 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.352+0000 7f80f259c700 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560819ddf2e0 0x560819de1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.353+0000 7f80d9ffb700 10 client.18081 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:55:52.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.353+0000 7f80f259c700 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560819ddf2e0 0x560819de1700 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93ed7916c79e28ee server_cookie=b82c637cf3c0833d in_seq=0 out_seq=0 2024-05-08T23:55:52.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f80d9ffb700 1 -- 172.21.15.12:0/213351628 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560819cb2ec0 con 0x560819ddf2e0 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f80d9ffb700 10 client.18081 handle_command_reply: tid=0 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 client.18081 shutdown 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 -- 172.21.15.12:0/213351628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 msgr2=0x560819ddd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819ddd610 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f80ec022040 tx=0x7f80ec01f660 comp rx=0 tx=0).stop 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 -- 172.21.15.12:0/213351628 shutdown_connections 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560819ddf2e0 0x560819de1700 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560819d70720 0x560819ddd610 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560819d6a890 0x560819d792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f80d9ffb700 0 client.18081 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 --2- 172.21.15.12:0/213351628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560819d6fe00 0x560819d79820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 -- 172.21.15.12:0/213351628 >> 172.21.15.12:0/213351628 conn(0x560819cc8210 msgr2=0x560819d6ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 -- 172.21.15.12:0/213351628 shutdown_connections 2024-05-08T23:55:52.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 1 -- 172.21.15.12:0/213351628 wait complete. 2024-05-08T23:55:52.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.354+0000 7f8104a4fb80 20 client.18081 trim_cache size 0 max 16384 2024-05-08T23:55:52.356 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:55:52.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 -- 172.21.15.12:0/2903959754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f80e00a1290 msgr2=0x7f80e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f80e00a1290 0x7f80e00a3750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f80e400b2e0 tx=0x7f80e4005940 comp rx=0 tx=0).stop 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 -- 172.21.15.12:0/2903959754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80f4105160 msgr2=0x7f80f4198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80f4105160 0x7f80f4198770 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00e970 tx=0x7f80dc00ec80 comp rx=0 tx=0).stop 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 -- 172.21.15.12:0/2903959754 shutdown_connections 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f80e00a1290 0x7f80e00a3750 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80f4198cb0 0x7f80f41a32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f4100060 0x7f80f4198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 --2- 172.21.15.12:0/2903959754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80f4105160 0x7f80f4198770 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 -- 172.21.15.12:0/2903959754 >> 172.21.15.12:0/2903959754 conn(0x7f80f4076f40 msgr2=0x7f80f40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 -- 172.21.15.12:0/2903959754 shutdown_connections 2024-05-08T23:55:52.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.355+0000 7f80f8895700 1 -- 172.21.15.12:0/2903959754 wait complete. 2024-05-08T23:55:52.363 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:55:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.515+0000 7f7ef2db8700 1 -- 172.21.15.12:0/2093330236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eec101b30 msgr2=0x7f7eec105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.515+0000 7f7ef2db8700 1 --2- 172.21.15.12:0/2093330236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eec101b30 0x7f7eec105b70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009990 tx=0x7f7ed8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.515+0000 7f7ef2db8700 1 -- 172.21.15.12:0/2093330236 shutdown_connections 2024-05-08T23:55:52.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.515+0000 7f7ef2db8700 1 --2- 172.21.15.12:0/2093330236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec1062a0 0x7f7eec10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.515+0000 7f7ef2db8700 1 --2- 172.21.15.12:0/2093330236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eec101180 0x7f7eec101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.515+0000 7f7ef2db8700 1 --2- 172.21.15.12:0/2093330236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eec101b30 0x7f7eec105b70 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.515+0000 7f7ef2db8700 1 -- 172.21.15.12:0/2093330236 >> 172.21.15.12:0/2093330236 conn(0x7f7eec0fc9e0 msgr2=0x7f7eec0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.515+0000 7f7ef2db8700 1 -- 172.21.15.12:0/2093330236 shutdown_connections 2024-05-08T23:55:52.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.515+0000 7f7ef2db8700 1 -- 172.21.15.12:0/2093330236 wait complete. 2024-05-08T23:55:52.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.516+0000 7f7ef2db8700 1 Processor -- start 2024-05-08T23:55:52.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.516+0000 7f7ef2db8700 1 -- start start 2024-05-08T23:55:52.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.516+0000 7f7ef2db8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eec101180 0x7f7eec197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.516+0000 7f7ef2db8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eec1062a0 0x7f7eec1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.516+0000 7f7ef2db8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec19bd50 0x7f7eec19c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.516+0000 7f7ef2db8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec19c7d0 con 0x7f7eec1062a0 2024-05-08T23:55:52.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.516+0000 7f7ef2db8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec19c940 con 0x7f7eec19bd50 2024-05-08T23:55:52.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.516+0000 7f7ef2db8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec19cb00 con 0x7f7eec101180 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7eebfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eec1062a0 0x7f7eec1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef0b54700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eec101180 0x7f7eec197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef1355700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec19bd50 0x7f7eec19c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef0b54700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eec101180 0x7f7eec197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33816/0 (socket says 172.21.15.12:33816) 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef0b54700 1 -- 172.21.15.12:0/4031539187 learned_addr learned my addr 172.21.15.12:0/4031539187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef0b54700 1 -- 172.21.15.12:0/4031539187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec19bd50 msgr2=0x7f7eec19c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec19bd50 0x7f7eec19c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef0b54700 1 -- 172.21.15.12:0/4031539187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eec1062a0 msgr2=0x7f7eec1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eec1062a0 0x7f7eec1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef0b54700 1 -- 172.21.15.12:0/4031539187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed80095f0 con 0x7f7eec101180 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7eebfff700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eec1062a0 0x7f7eec1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eec101180 0x7f7eec197470 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000e9a0 tx=0x7f7ee000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ee9ffb700 1 -- 172.21.15.12:0/4031539187 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0009790 con 0x7f7eec101180 2024-05-08T23:55:52.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ee9ffb700 1 -- 172.21.15.12:0/4031539187 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ee0010430 con 0x7f7eec101180 2024-05-08T23:55:52.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ee9ffb700 1 -- 172.21.15.12:0/4031539187 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0018680 con 0x7f7eec101180 2024-05-08T23:55:52.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.517+0000 7f7ef2db8700 1 -- 172.21.15.12:0/4031539187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec1a4060 con 0x7f7eec101180 2024-05-08T23:55:52.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.518+0000 7f7ef2db8700 1 -- 172.21.15.12:0/4031539187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec1a4540 con 0x7f7eec101180 2024-05-08T23:55:52.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.520+0000 7f7ee9ffb700 1 -- 172.21.15.12:0/4031539187 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7ee00105a0 con 0x7f7eec101180 2024-05-08T23:55:52.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.521+0000 7f7ee9ffb700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7edc0a1310 0x7f7edc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.521+0000 7f7eebfff700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7edc0a1310 0x7f7edc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.521+0000 7f7ee9ffb700 1 -- 172.21.15.12:0/4031539187 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7ee0014070 con 0x7f7eec101180 2024-05-08T23:55:52.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.521+0000 7f7efef72b80 1 Processor -- start 2024-05-08T23:55:52.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.521+0000 7f7efef72b80 1 -- start start 2024-05-08T23:55:52.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.521+0000 7f7eebfff700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7edc0a1310 0x7f7edc0a37d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800b2e0 tx=0x7f7ed8005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 0x5604bb519350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 0x5604bb519cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7efef72b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb470150 con 0x5604bb519890 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7efef72b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb46db10 con 0x5604bb51f720 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7efef72b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb469400 con 0x5604bb51ee00 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7eebfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 0x5604bb519350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ef1355700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 0x5604bb519cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ef0b54700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7eebfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 0x5604bb519350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45418/0 (socket says 172.21.15.12:45418) 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ef1355700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 0x5604bb519cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44378/0 (socket says 172.21.15.12:44378) 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7eebfff700 1 -- 172.21.15.12:0/2396934300 learned_addr learned my addr 172.21.15.12:0/2396934300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ef0b54700 1 -- 172.21.15.12:0/2396934300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 msgr2=0x5604bb519350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/2396934300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 0x5604bb519350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ef0b54700 1 -- 172.21.15.12:0/2396934300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 msgr2=0x5604bb519cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/2396934300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 0x5604bb519cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ef0b54700 1 -- 172.21.15.12:0/2396934300 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604bb2ea3b0 con 0x5604bb51ee00 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/2396934300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51f1e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00ca580 tx=0x7f7ee0099b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ad5472185320f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ed6ffd700 1 -- 172.21.15.12:0/2396934300 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee009dd70 con 0x5604bb51ee00 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ed6ffd700 1 -- 172.21.15.12:0/2396934300 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ee00a23c0 con 0x5604bb51ee00 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.522+0000 7f7ed6ffd700 1 -- 172.21.15.12:0/2396934300 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0099630 con 0x5604bb51ee00 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.523+0000 7f7efef72b80 1 -- 172.21.15.12:0/2396934300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 msgr2=0x5604bb51f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.523+0000 7f7efef72b80 1 --2- 172.21.15.12:0/2396934300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51f1e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00ca580 tx=0x7f7ee0099b90 comp rx=0 tx=0).stop 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.523+0000 7f7efef72b80 1 -- 172.21.15.12:0/2396934300 shutdown_connections 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.523+0000 7f7efef72b80 1 --2- 172.21.15.12:0/2396934300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 0x5604bb519cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.523+0000 7f7efef72b80 1 --2- 172.21.15.12:0/2396934300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51f1e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.523+0000 7f7efef72b80 1 --2- 172.21.15.12:0/2396934300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 0x5604bb519350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.523+0000 7f7efef72b80 1 -- 172.21.15.12:0/2396934300 >> 172.21.15.12:0/2396934300 conn(0x5604bb477210 msgr2=0x5604bb513df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.523+0000 7f7efef72b80 1 -- 172.21.15.12:0/2396934300 shutdown_connections 2024-05-08T23:55:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.523+0000 7f7efef72b80 1 -- 172.21.15.12:0/2396934300 wait complete. 2024-05-08T23:55:52.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.524+0000 7f7efef72b80 1 -- start start 2024-05-08T23:55:52.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.524+0000 7f7efef72b80 1 Processor -- start 2024-05-08T23:55:52.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.524+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 0x5604bb522ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.524+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 0x5604bb523010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.524+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb523550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7efef72b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb405560 con 0x5604bb51f720 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7efef72b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb405200 con 0x5604bb51ee00 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7efef72b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb2ec6a0 con 0x5604bb519890 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7efef72b80 10 client.0 _ll_register_callbacks cb 0x5604bb40eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7ef1355700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb523550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7eebfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 0x5604bb523010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7ef1355700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb523550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44380/0 (socket says 172.21.15.12:44380) 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7ef1355700 1 -- 172.21.15.12:0/1594698115 learned_addr learned my addr 172.21.15.12:0/1594698115 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 0x5604bb522ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7eebfff700 1 -- 172.21.15.12:0/1594698115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 msgr2=0x5604bb522ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7eebfff700 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 0x5604bb522ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7eebfff700 1 -- 172.21.15.12:0/1594698115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 msgr2=0x5604bb523550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7eebfff700 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb523550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7eebfff700 1 -- 172.21.15.12:0/1594698115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec1a4540 con 0x5604bb51ee00 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7ef1355700 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb523550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7eebfff700 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 0x5604bb523010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c010 tx=0x7f7ed801a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e6e4e79fe0be09a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7ecbfff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7ecbfff700 1 -- 172.21.15.12:0/1594698115 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8034070 con 0x5604bb51ee00 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7efef72b80 1 -- 172.21.15.12:0/1594698115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604bb3c2270 con 0x5604bb51ee00 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.525+0000 7f7ecbfff700 1 -- 172.21.15.12:0/1594698115 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7ed8005020 con 0x5604bb51ee00 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7ecbfff700 1 -- 172.21.15.12:0/1594698115 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ed802b050 con 0x5604bb51ee00 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7efef72b80 10 client.27860 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7efef72b80 1 -- 172.21.15.12:0/1594698115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5604bb296b60 con 0x5604bb51ee00 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7ecbfff700 1 -- 172.21.15.12:0/1594698115 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604bb3c2270 con 0x5604bb51ee00 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7ecbfff700 1 -- 172.21.15.12:0/1594698115 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7ed8037020 con 0x5604bb51ee00 2024-05-08T23:55:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7efef72b80 10 client.27860 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7efef72b80 10 client.27860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7efef72b80 20 client.27860 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7efef72b80 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bb5332c0 0x5604bb5356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7efef72b80 4 client.27860 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7efef72b80 1 -- 172.21.15.12:0/1594698115 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604bb4e8810 con 0x5604bb5332c0 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.526+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bb5332c0 0x5604bb5356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.527+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bb5332c0 0x5604bb5356e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6634d01711c19cbb server_cookie=b2a8c83e8344a204 in_seq=0 out_seq=0 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.527+0000 7f7ecbfff700 10 client.27860 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.527+0000 7f7ecbfff700 1 -- 172.21.15.12:0/1594698115 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604bb4e8810 con 0x5604bb5332c0 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.527+0000 7f7ecbfff700 10 client.27860 handle_command_reply: tid=0 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.527+0000 7f7efef72b80 1 client.27860 shutdown 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.527+0000 7f7efef72b80 1 -- 172.21.15.12:0/1594698115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 msgr2=0x5604bb523010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.527+0000 7f7efef72b80 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 0x5604bb523010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c010 tx=0x7f7ed801a050 comp rx=0 tx=0).stop 2024-05-08T23:55:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.528+0000 7f7efef72b80 1 -- 172.21.15.12:0/1594698115 shutdown_connections 2024-05-08T23:55:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.528+0000 7f7efef72b80 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb523550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.528+0000 7f7efef72b80 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 0x5604bb522ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.528+0000 7f7efef72b80 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bb5332c0 0x5604bb5356e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.528+0000 7f7efef72b80 1 --2- 172.21.15.12:0/1594698115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 0x5604bb523010 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.528+0000 7f7efef72b80 1 -- 172.21.15.12:0/1594698115 >> 172.21.15.12:0/1594698115 conn(0x5604bb477210 msgr2=0x5604bb5135a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.528+0000 7f7ecbfff700 0 client.27860 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.528+0000 7f7efef72b80 1 -- 172.21.15.12:0/1594698115 shutdown_connections 2024-05-08T23:55:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.528+0000 7f7efef72b80 1 -- 172.21.15.12:0/1594698115 wait complete. 2024-05-08T23:55:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.528+0000 7f7efef72b80 20 client.27860 trim_cache size 0 max 16384 2024-05-08T23:55:52.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7efef72b80 1 Processor -- start 2024-05-08T23:55:52.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7efef72b80 1 -- start start 2024-05-08T23:55:52.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 0x5604bb519350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 0x5604bb513d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7efef72b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb515a20 con 0x5604bb519890 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7efef72b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb515e50 con 0x5604bb51f720 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7efef72b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb51f560 con 0x5604bb51ee00 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7ef0b54700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 0x5604bb519350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7eebfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7eebfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33830/0 (socket says 172.21.15.12:33830) 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7eebfff700 1 -- 172.21.15.12:0/3172212405 learned_addr learned my addr 172.21.15.12:0/3172212405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.543+0000 7f7ef1355700 1 --2- 172.21.15.12:0/3172212405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 0x5604bb513d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.544+0000 7f7eebfff700 1 -- 172.21.15.12:0/3172212405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 msgr2=0x5604bb513d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.544+0000 7f7eebfff700 1 --2- 172.21.15.12:0/3172212405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 0x5604bb513d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.544+0000 7f7eebfff700 1 -- 172.21.15.12:0/3172212405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 msgr2=0x5604bb519350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.544+0000 7f7eebfff700 1 --2- 172.21.15.12:0/3172212405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 0x5604bb519350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.544+0000 7f7eebfff700 1 -- 172.21.15.12:0/3172212405 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604bb519f40 con 0x5604bb51ee00 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.544+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/3172212405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 0x5604bb519350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.544+0000 7f7eebfff700 1 --2- 172.21.15.12:0/3172212405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51e450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8005370 tx=0x7f7ed8005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8f871f5da48a5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.544+0000 7f7ed77fe700 1 -- 172.21.15.12:0/3172212405 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8034070 con 0x5604bb51ee00 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.544+0000 7f7ed77fe700 1 -- 172.21.15.12:0/3172212405 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ed802c040 con 0x5604bb51ee00 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.544+0000 7f7ed77fe700 1 -- 172.21.15.12:0/3172212405 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed802b040 con 0x5604bb51ee00 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 -- 172.21.15.12:0/3172212405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 msgr2=0x5604bb51e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 --2- 172.21.15.12:0/3172212405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51e450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8005370 tx=0x7f7ed8005780 comp rx=0 tx=0).stop 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 -- 172.21.15.12:0/3172212405 shutdown_connections 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 --2- 172.21.15.12:0/3172212405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51f720 0x5604bb513d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 --2- 172.21.15.12:0/3172212405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb519890 0x5604bb519350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 --2- 172.21.15.12:0/3172212405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb51ee00 0x5604bb51e450 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 -- 172.21.15.12:0/3172212405 >> 172.21.15.12:0/3172212405 conn(0x5604bb477210 msgr2=0x5604bb5213a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 -- 172.21.15.12:0/3172212405 shutdown_connections 2024-05-08T23:55:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 -- 172.21.15.12:0/3172212405 wait complete. 2024-05-08T23:55:52.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 -- start start 2024-05-08T23:55:52.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.545+0000 7f7efef72b80 1 Processor -- start 2024-05-08T23:55:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 0x5604bb5282d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 0x5604bb528810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7efef72b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb5887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7efef72b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb528f40 con 0x5604bb51f720 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7efef72b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb588d00 con 0x5604bb51ee00 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7efef72b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bb588e90 con 0x5604bb519890 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7efef72b80 10 client.0 _ll_register_callbacks cb 0x5604bb40eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef0b54700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 0x5604bb5282d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef1355700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb5887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef0b54700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 0x5604bb5282d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33846/0 (socket says 172.21.15.12:33846) 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef1355700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb5887c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44392/0 (socket says 172.21.15.12:44392) 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7eebfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 0x5604bb528810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef0b54700 1 -- 172.21.15.12:0/2708881138 learned_addr learned my addr 172.21.15.12:0/2708881138 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:52.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef0b54700 1 -- 172.21.15.12:0/2708881138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 msgr2=0x5604bb528810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 0x5604bb528810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef0b54700 1 -- 172.21.15.12:0/2708881138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 msgr2=0x5604bb5887c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb5887c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef0b54700 1 -- 172.21.15.12:0/2708881138 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604bb2ea3b0 con 0x5604bb519890 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.546+0000 7f7ef1355700 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb5887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7ef0b54700 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 0x5604bb5282d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c2d0 tx=0x7f7ee00cab20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98b52671570b997e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7ecbfff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7ecbfff700 1 -- 172.21.15.12:0/2708881138 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee009dd70 con 0x5604bb519890 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7efef72b80 1 -- 172.21.15.12:0/2708881138 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ee00a23c0 con 0x5604bb519890 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7ecbfff700 1 -- 172.21.15.12:0/2708881138 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7ee00b81b0 con 0x5604bb519890 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7ecbfff700 1 -- 172.21.15.12:0/2708881138 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ee0099430 con 0x5604bb519890 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7efef72b80 10 client.27865 fetch_fsmap learned FSMap version 40 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7efef72b80 1 -- 172.21.15.12:0/2708881138 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5604bb296b60 con 0x5604bb519890 2024-05-08T23:55:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7ecbfff700 1 -- 172.21.15.12:0/2708881138 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ee00a23c0 con 0x5604bb519890 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7ecbfff700 1 -- 172.21.15.12:0/2708881138 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7ee00a1f30 con 0x5604bb519890 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7efef72b80 10 client.27865 fetch_fsmap finished waiting for FSMap version 40 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7efef72b80 10 client.27865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7efef72b80 20 client.27865 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7efef72b80 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bb58a370 0x5604bb58c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7efef72b80 4 client.27865 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.547+0000 7f7efef72b80 1 -- 172.21.15.12:0/2708881138 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5604bb461ec0 con 0x5604bb58a370 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7eebfff700 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bb58a370 0x5604bb58c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7ecbfff700 10 client.27865 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7eebfff700 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bb58a370 0x5604bb58c790 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef065b9058b53096 server_cookie=7626e31e0d77c21c in_seq=0 out_seq=0 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7ecbfff700 1 -- 172.21.15.12:0/2708881138 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5604bb461ec0 con 0x5604bb58a370 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7ecbfff700 10 client.27865 handle_command_reply: tid=0 2024-05-08T23:55:52.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7efef72b80 1 client.27865 shutdown 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7efef72b80 1 -- 172.21.15.12:0/2708881138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 msgr2=0x5604bb5282d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7efef72b80 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 0x5604bb5282d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c2d0 tx=0x7f7ee00cab20 comp rx=0 tx=0).stop 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7efef72b80 1 -- 172.21.15.12:0/2708881138 shutdown_connections 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7efef72b80 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bb51f720 0x5604bb5887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7efef72b80 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bb519890 0x5604bb5282d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7efef72b80 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bb58a370 0x5604bb58c790 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7efef72b80 1 --2- 172.21.15.12:0/2708881138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bb51ee00 0x5604bb528810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7efef72b80 1 -- 172.21.15.12:0/2708881138 >> 172.21.15.12:0/2708881138 conn(0x5604bb477210 msgr2=0x5604bb5233b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.548+0000 7f7ecbfff700 0 client.27865 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7efef72b80 1 -- 172.21.15.12:0/2708881138 shutdown_connections 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7efef72b80 1 -- 172.21.15.12:0/2708881138 wait complete. 2024-05-08T23:55:52.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7efef72b80 20 client.27865 trim_cache size 0 max 16384 2024-05-08T23:55:52.552 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:55:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 -- 172.21.15.12:0/4031539187 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7edc0a1310 msgr2=0x7f7edc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7edc0a1310 0x7f7edc0a37d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800b2e0 tx=0x7f7ed8005960 comp rx=0 tx=0).stop 2024-05-08T23:55:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 -- 172.21.15.12:0/4031539187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eec101180 msgr2=0x7f7eec197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:52.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eec101180 0x7f7eec197470 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000e9a0 tx=0x7f7ee000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 -- 172.21.15.12:0/4031539187 shutdown_connections 2024-05-08T23:55:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7edc0a1310 0x7f7edc0a37d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec19bd50 0x7f7eec19c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eec101180 0x7f7eec197470 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 --2- 172.21.15.12:0/4031539187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eec1062a0 0x7f7eec1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 -- 172.21.15.12:0/4031539187 >> 172.21.15.12:0/4031539187 conn(0x7f7eec0fc9e0 msgr2=0x7f7eec0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 -- 172.21.15.12:0/4031539187 shutdown_connections 2024-05-08T23:55:52.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:52.549+0000 7f7ef2db8700 1 -- 172.21.15.12:0/4031539187 wait complete. 2024-05-08T23:55:52.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:52 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/11987682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:55:52.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 541414 6.94 MB/sec execute 348 sec latency 32.653 ms 2024-05-08T23:55:53.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:52 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/11987682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:55:53.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:52 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/11987682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:55:53.559 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:55:53.559 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.713+0000 7f81db2a0700 1 -- 172.21.15.12:0/1138756554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81d4101160 msgr2=0x7f81d4101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.713+0000 7f81db2a0700 1 --2- 172.21.15.12:0/1138756554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81d4101160 0x7f81d4101540 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009990 tx=0x7f81c8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.713+0000 7f81db2a0700 1 -- 172.21.15.12:0/1138756554 shutdown_connections 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.713+0000 7f81db2a0700 1 --2- 172.21.15.12:0/1138756554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d4106280 0x7f81d410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.713+0000 7f81db2a0700 1 --2- 172.21.15.12:0/1138756554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81d4101160 0x7f81d4101540 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.713+0000 7f81db2a0700 1 --2- 172.21.15.12:0/1138756554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81d4101b10 0x7f81d4105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.713+0000 7f81db2a0700 1 -- 172.21.15.12:0/1138756554 >> 172.21.15.12:0/1138756554 conn(0x7f81d40fc9e0 msgr2=0x7f81d40fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.713+0000 7f81db2a0700 1 -- 172.21.15.12:0/1138756554 shutdown_connections 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.714+0000 7f81db2a0700 1 -- 172.21.15.12:0/1138756554 wait complete. 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.714+0000 7f81db2a0700 1 Processor -- start 2024-05-08T23:55:53.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.714+0000 7f81db2a0700 1 -- start start 2024-05-08T23:55:53.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.714+0000 7f81db2a0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81d4101160 0x7f81d40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.714+0000 7f81db2a0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d4106280 0x7f81d40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.714+0000 7f81db2a0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81d4073d20 0x7f81d41a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.714+0000 7f81db2a0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d41a5c30 con 0x7f81d4101160 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.714+0000 7f81db2a0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d41a5da0 con 0x7f81d4106280 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.714+0000 7f81db2a0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d41a5f60 con 0x7f81d4073d20 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d903c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81d4101160 0x7f81d40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d903c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81d4101160 0x7f81d40732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44410/0 (socket says 172.21.15.12:44410) 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d903c700 1 -- 172.21.15.12:0/710435991 learned_addr learned my addr 172.21.15.12:0/710435991 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d983d700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81d4073d20 0x7f81d41a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d883b700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d4106280 0x7f81d40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d903c700 1 -- 172.21.15.12:0/710435991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81d4073d20 msgr2=0x7f81d41a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d903c700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81d4073d20 0x7f81d41a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d903c700 1 -- 172.21.15.12:0/710435991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d4106280 msgr2=0x7f81d40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d903c700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d4106280 0x7f81d40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d903c700 1 -- 172.21.15.12:0/710435991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c80095f0 con 0x7f81d4101160 2024-05-08T23:55:53.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81d903c700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81d4101160 0x7f81d40732a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f81c8004760 tx=0x7f81c8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81c67fc700 1 -- 172.21.15.12:0/710435991 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c801d070 con 0x7f81d4101160 2024-05-08T23:55:53.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81c67fc700 1 -- 172.21.15.12:0/710435991 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81c8022470 con 0x7f81d4101160 2024-05-08T23:55:53.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81db2a0700 1 -- 172.21.15.12:0/710435991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d41a61e0 con 0x7f81d4101160 2024-05-08T23:55:53.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81c67fc700 1 -- 172.21.15.12:0/710435991 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8017650 con 0x7f81d4101160 2024-05-08T23:55:53.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.715+0000 7f81db2a0700 1 -- 172.21.15.12:0/710435991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d41a6600 con 0x7f81d4101160 2024-05-08T23:55:53.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.718+0000 7f81c67fc700 1 -- 172.21.15.12:0/710435991 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f81c80225e0 con 0x7f81d4101160 2024-05-08T23:55:53.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.718+0000 7f81c67fc700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f81c00a1290 0x7f81c00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.719+0000 7f81d883b700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f81c00a1290 0x7f81c00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.719+0000 7f81c67fc700 1 -- 172.21.15.12:0/710435991 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f81c80cdf10 con 0x7f81d4101160 2024-05-08T23:55:53.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.719+0000 7f81d883b700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f81c00a1290 0x7f81c00a3750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81d40749d0 tx=0x7f81d0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81e745ab80 1 Processor -- start 2024-05-08T23:55:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81e745ab80 1 -- start start 2024-05-08T23:55:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2abd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81e745ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de2021d0 con 0x5626de2ab910 2024-05-08T23:55:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81e745ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de1ffb90 con 0x5626de2b17a0 2024-05-08T23:55:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81e745ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de1fb480 con 0x5626de2b0e80 2024-05-08T23:55:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81d983d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81d983d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2abd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44420/0 (socket says 172.21.15.12:44420) 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.720+0000 7f81d983d700 1 -- 172.21.15.12:0/3997862782 learned_addr learned my addr 172.21.15.12:0/3997862782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81d903c700 1 --2- 172.21.15.12:0/3997862782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81d883b700 1 --2- 172.21.15.12:0/3997862782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81d983d700 1 -- 172.21.15.12:0/3997862782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 msgr2=0x5626de2b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81d983d700 1 --2- 172.21.15.12:0/3997862782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81d983d700 1 -- 172.21.15.12:0/3997862782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 msgr2=0x5626de2ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81d983d700 1 --2- 172.21.15.12:0/3997862782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2ab3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81d983d700 1 -- 172.21.15.12:0/3997862782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626de07c430 con 0x5626de2ab910 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81d883b700 1 --2- 172.21.15.12:0/3997862782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81d983d700 1 --2- 172.21.15.12:0/3997862782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2abd50 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f81cc003f30 tx=0x7f81cc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9966e82b808fc7f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81bf7fe700 1 -- 172.21.15.12:0/3997862782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc003a00 con 0x5626de2ab910 2024-05-08T23:55:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81bf7fe700 1 -- 172.21.15.12:0/3997862782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81cc00f040 con 0x5626de2ab910 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.721+0000 7f81bf7fe700 1 -- 172.21.15.12:0/3997862782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0156e0 con 0x5626de2ab910 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.722+0000 7f81e745ab80 1 -- 172.21.15.12:0/3997862782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 msgr2=0x5626de2abd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.722+0000 7f81e745ab80 1 --2- 172.21.15.12:0/3997862782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2abd50 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f81cc003f30 tx=0x7f81cc009c20 comp rx=0 tx=0).stop 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.722+0000 7f81e745ab80 1 -- 172.21.15.12:0/3997862782 shutdown_connections 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.722+0000 7f81e745ab80 1 --2- 172.21.15.12:0/3997862782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2abd50 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.722+0000 7f81e745ab80 1 --2- 172.21.15.12:0/3997862782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.722+0000 7f81e745ab80 1 --2- 172.21.15.12:0/3997862782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.722+0000 7f81e745ab80 1 -- 172.21.15.12:0/3997862782 >> 172.21.15.12:0/3997862782 conn(0x5626de209290 msgr2=0x5626de2a5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.722+0000 7f81e745ab80 1 -- 172.21.15.12:0/3997862782 shutdown_connections 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.722+0000 7f81e745ab80 1 -- 172.21.15.12:0/3997862782 wait complete. 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.722+0000 7f81e745ab80 1 -- start start 2024-05-08T23:55:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81e745ab80 1 Processor -- start 2024-05-08T23:55:53.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2ab910 0x5626de2b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b0e80 0x5626de2b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81e745ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de1975e0 con 0x5626de2b0e80 2024-05-08T23:55:53.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81e745ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de197280 con 0x5626de2b17a0 2024-05-08T23:55:53.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81e745ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de07e720 con 0x5626de2ab910 2024-05-08T23:55:53.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81e745ab80 10 client.0 _ll_register_callbacks cb 0x5626de1a0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:53.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81d883b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b0e80 0x5626de2b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81d883b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b0e80 0x5626de2b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44430/0 (socket says 172.21.15.12:44430) 2024-05-08T23:55:53.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81d883b700 1 -- 172.21.15.12:0/992611171 learned_addr learned my addr 172.21.15.12:0/992611171 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81d903c700 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2ab910 0x5626de2b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.723+0000 7f81d983d700 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81d883b700 1 -- 172.21.15.12:0/992611171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2ab910 msgr2=0x5626de2b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81d883b700 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2ab910 0x5626de2b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81d883b700 1 -- 172.21.15.12:0/992611171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 msgr2=0x5626de2b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81d883b700 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81d883b700 1 -- 172.21.15.12:0/992611171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc003b60 con 0x5626de2b0e80 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81d983d700 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81d883b700 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b0e80 0x5626de2b5050 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f81d001e7d0 tx=0x7f81d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee131d9d99b1ee5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81bcff9700 1 -- 172.21.15.12:0/992611171 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0004320 con 0x5626de2b0e80 2024-05-08T23:55:53.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81bcff9700 1 -- 172.21.15.12:0/992611171 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f81d0004480 con 0x5626de2b0e80 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81e745ab80 1 -- 172.21.15.12:0/992611171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626de1542f0 con 0x5626de2b0e80 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81bcff9700 1 -- 172.21.15.12:0/992611171 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81d0003680 con 0x5626de2b0e80 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81e745ab80 10 client.18111 fetch_fsmap learned FSMap version 41 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81e745ab80 1 -- 172.21.15.12:0/992611171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5626de028be0 con 0x5626de2b0e80 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.724+0000 7f81bcff9700 1 -- 172.21.15.12:0/992611171 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626de1542f0 con 0x5626de2b0e80 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.725+0000 7f81bcff9700 1 -- 172.21.15.12:0/992611171 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f81d0030960 con 0x5626de2b0e80 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.725+0000 7f81e745ab80 10 client.18111 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.725+0000 7f81e745ab80 10 client.18111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.725+0000 7f81e745ab80 20 client.18111 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.725+0000 7f81e745ab80 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5626de2c5300 0x5626de2c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.725+0000 7f81e745ab80 4 client.18111 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:53.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.725+0000 7f81e745ab80 1 -- 172.21.15.12:0/992611171 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626de27a890 con 0x5626de2c5300 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.725+0000 7f81d903c700 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5626de2c5300 0x5626de2c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.725+0000 7f81d903c700 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5626de2c5300 0x5626de2c7720 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae9131cac27cb0d server_cookie=a6469eec33bf6bf7 in_seq=0 out_seq=0 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.725+0000 7f81bcff9700 10 client.18111 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81bcff9700 1 -- 172.21.15.12:0/992611171 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626de27a890 con 0x5626de2c5300 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81bcff9700 10 client.18111 handle_command_reply: tid=0 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 client.18111 shutdown 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 -- 172.21.15.12:0/992611171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b0e80 msgr2=0x5626de2b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b0e80 0x5626de2b5050 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f81d001e7d0 tx=0x7f81d00047c0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 -- 172.21.15.12:0/992611171 shutdown_connections 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5626de2c5300 0x5626de2c7720 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2ab910 0x5626de2b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81bcff9700 0 client.18111 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:53.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 --2- 172.21.15.12:0/992611171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b0e80 0x5626de2b5050 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 -- 172.21.15.12:0/992611171 >> 172.21.15.12:0/992611171 conn(0x5626de209290 msgr2=0x5626de2a5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 -- 172.21.15.12:0/992611171 shutdown_connections 2024-05-08T23:55:53.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.726+0000 7f81e745ab80 1 -- 172.21.15.12:0/992611171 wait complete. 2024-05-08T23:55:53.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.727+0000 7f81e745ab80 20 client.18111 trim_cache size 0 max 16384 2024-05-08T23:55:53.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.745+0000 7f81e745ab80 1 Processor -- start 2024-05-08T23:55:53.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.745+0000 7f81e745ab80 1 -- start start 2024-05-08T23:55:53.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.745+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2ab910 0x5626de2ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.745+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.745+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b17a0 0x5626de2afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.745+0000 7f81e745ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de2b61b0 con 0x5626de2b17a0 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.745+0000 7f81e745ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de2a2e70 con 0x5626de2ab910 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.745+0000 7f81e745ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de2b1660 con 0x5626de2b0e80 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d983d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b17a0 0x5626de2afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d983d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b17a0 0x5626de2afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44432/0 (socket says 172.21.15.12:44432) 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d983d700 1 -- 172.21.15.12:0/1980492962 learned_addr learned my addr 172.21.15.12:0/1980492962 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d903c700 1 --2- 172.21.15.12:0/1980492962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2ab910 0x5626de2ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d883b700 1 --2- 172.21.15.12:0/1980492962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d983d700 1 -- 172.21.15.12:0/1980492962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 msgr2=0x5626de2a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d983d700 1 --2- 172.21.15.12:0/1980492962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d983d700 1 -- 172.21.15.12:0/1980492962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2ab910 msgr2=0x5626de2ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d983d700 1 --2- 172.21.15.12:0/1980492962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2ab910 0x5626de2ab3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d983d700 1 -- 172.21.15.12:0/1980492962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626de1c15e0 con 0x5626de2b17a0 2024-05-08T23:55:53.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d883b700 1 --2- 172.21.15.12:0/1980492962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81d983d700 1 --2- 172.21.15.12:0/1980492962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b17a0 0x5626de2afc00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f81cc007d00 tx=0x7f81cc003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f672a7afe8f757ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81bffff700 1 -- 172.21.15.12:0/1980492962 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc007700 con 0x5626de2b17a0 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81bffff700 1 -- 172.21.15.12:0/1980492962 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81cc007860 con 0x5626de2b17a0 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.746+0000 7f81bffff700 1 -- 172.21.15.12:0/1980492962 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0046d0 con 0x5626de2b17a0 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 -- 172.21.15.12:0/1980492962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b17a0 msgr2=0x5626de2afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 --2- 172.21.15.12:0/1980492962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b17a0 0x5626de2afc00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f81cc007d00 tx=0x7f81cc003a00 comp rx=0 tx=0).stop 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 -- 172.21.15.12:0/1980492962 shutdown_connections 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 --2- 172.21.15.12:0/1980492962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2b17a0 0x5626de2afc00 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 --2- 172.21.15.12:0/1980492962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2ab910 0x5626de2ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 --2- 172.21.15.12:0/1980492962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 -- 172.21.15.12:0/1980492962 >> 172.21.15.12:0/1980492962 conn(0x5626de209290 msgr2=0x5626de2b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 -- 172.21.15.12:0/1980492962 shutdown_connections 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 -- 172.21.15.12:0/1980492962 wait complete. 2024-05-08T23:55:53.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 -- start start 2024-05-08T23:55:53.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.747+0000 7f81e745ab80 1 Processor -- start 2024-05-08T23:55:53.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2bad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81e745ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2bb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81e745ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de31b850 con 0x5626de2ab910 2024-05-08T23:55:53.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81e745ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de31b990 con 0x5626de2b17a0 2024-05-08T23:55:53.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81e745ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626de31bb20 con 0x5626de2b0e80 2024-05-08T23:55:53.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81e745ab80 10 client.0 _ll_register_callbacks cb 0x5626de1a0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:53.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d903c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d903c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2bad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44444/0 (socket says 172.21.15.12:44444) 2024-05-08T23:55:53.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d903c700 1 -- 172.21.15.12:0/2412426803 learned_addr learned my addr 172.21.15.12:0/2412426803 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d983d700 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d883b700 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d903c700 1 -- 172.21.15.12:0/2412426803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 msgr2=0x5626de2bb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d903c700 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2bb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d903c700 1 -- 172.21.15.12:0/2412426803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 msgr2=0x5626de2bb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d903c700 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2bb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d903c700 1 -- 172.21.15.12:0/2412426803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626de07c430 con 0x5626de2ab910 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.748+0000 7f81d983d700 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81d903c700 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2bad00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f81c8097b30 tx=0x7f81c80a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83aaa04c5f98534 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81bcff9700 1 -- 172.21.15.12:0/2412426803 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c809eef0 con 0x5626de2ab910 2024-05-08T23:55:53.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81bcff9700 1 -- 172.21.15.12:0/2412426803 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f81c80a34b0 con 0x5626de2ab910 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81e745ab80 1 -- 172.21.15.12:0/2412426803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81cc00f040 con 0x5626de2ab910 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81bcff9700 1 -- 172.21.15.12:0/2412426803 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81c80a10b0 con 0x5626de2ab910 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81e745ab80 10 client.18117 fetch_fsmap learned FSMap version 41 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81e745ab80 1 -- 172.21.15.12:0/2412426803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5626de080d50 con 0x5626de2ab910 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81bcff9700 1 -- 172.21.15.12:0/2412426803 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81cc00f040 con 0x5626de2ab910 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81bcff9700 1 -- 172.21.15.12:0/2412426803 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f81c809b320 con 0x5626de2ab910 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81e745ab80 10 client.18117 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81e745ab80 10 client.18117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81e745ab80 20 client.18117 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81e745ab80 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5626de31cfc0 0x5626de31f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81e745ab80 4 client.18117 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:55:53.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.749+0000 7f81e745ab80 1 -- 172.21.15.12:0/2412426803 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5626de1f3f40 con 0x5626de31cfc0 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.750+0000 7f81d883b700 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5626de31cfc0 0x5626de31f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.750+0000 7f81d883b700 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5626de31cfc0 0x5626de31f3e0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee3a5d76bc01d server_cookie=b0005fb591184cb0 in_seq=0 out_seq=0 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.750+0000 7f81bcff9700 10 client.18117 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81bcff9700 1 -- 172.21.15.12:0/2412426803 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5626de1f3f40 con 0x5626de31cfc0 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81bcff9700 10 client.18117 handle_command_reply: tid=0 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81e745ab80 1 client.18117 shutdown 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81e745ab80 1 -- 172.21.15.12:0/2412426803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 msgr2=0x5626de2bad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81e745ab80 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2bad00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f81c8097b30 tx=0x7f81c80a1040 comp rx=0 tx=0).stop 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81e745ab80 1 -- 172.21.15.12:0/2412426803 shutdown_connections 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81e745ab80 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5626de31cfc0 0x5626de31f3e0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81e745ab80 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5626de2b17a0 0x5626de2bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81e745ab80 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5626de2ab910 0x5626de2bad00 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81bcff9700 0 client.18117 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81e745ab80 1 --2- 172.21.15.12:0/2412426803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626de2b0e80 0x5626de2bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.751+0000 7f81e745ab80 1 -- 172.21.15.12:0/2412426803 >> 172.21.15.12:0/2412426803 conn(0x5626de209290 msgr2=0x5626de2afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81e745ab80 1 -- 172.21.15.12:0/2412426803 shutdown_connections 2024-05-08T23:55:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81e745ab80 1 -- 172.21.15.12:0/2412426803 wait complete. 2024-05-08T23:55:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81e745ab80 20 client.18117 trim_cache size 0 max 16384 2024-05-08T23:55:53.753 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:55:53.753 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:55:53.754 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "ed63472c-3cd6-4fc4-ac7d-07fd7cf1c463", 2024-05-08T23:55:53.754 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:55:53.754 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:55:53.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81db2a0700 1 -- 172.21.15.12:0/710435991 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f81c00a1290 msgr2=0x7f81c00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81db2a0700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f81c00a1290 0x7f81c00a3750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81d40749d0 tx=0x7f81d0009260 comp rx=0 tx=0).stop 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81db2a0700 1 -- 172.21.15.12:0/710435991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81d4101160 msgr2=0x7f81d40732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81db2a0700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81d4101160 0x7f81d40732a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f81c8004760 tx=0x7f81c8004790 comp rx=0 tx=0).stop 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81db2a0700 1 -- 172.21.15.12:0/710435991 shutdown_connections 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81db2a0700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f81c00a1290 0x7f81c00a3750 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81db2a0700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81d4073d20 0x7f81d41a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81db2a0700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81d4101160 0x7f81d40732a0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.752+0000 7f81db2a0700 1 --2- 172.21.15.12:0/710435991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d4106280 0x7f81d40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.753+0000 7f81db2a0700 1 -- 172.21.15.12:0/710435991 >> 172.21.15.12:0/710435991 conn(0x7f81d40fc9e0 msgr2=0x7f81d40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.753+0000 7f81db2a0700 1 -- 172.21.15.12:0/710435991 shutdown_connections 2024-05-08T23:55:53.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.753+0000 7f81db2a0700 1 -- 172.21.15.12:0/710435991 wait complete. 2024-05-08T23:55:53.762 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:55:53.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 541959 6.93 MB/sec execute 349 sec latency 705.613 ms 2024-05-08T23:55:53.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.914+0000 7fad466cc700 1 -- 172.21.15.12:0/2305123363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad40105f90 msgr2=0x7fad40106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.914+0000 7fad466cc700 1 --2- 172.21.15.12:0/2305123363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad40105f90 0x7fad40106370 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fad300099c0 tx=0x7fad30009cd0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.914+0000 7fad466cc700 1 -- 172.21.15.12:0/2305123363 shutdown_connections 2024-05-08T23:55:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.914+0000 7fad466cc700 1 --2- 172.21.15.12:0/2305123363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad40100ab0 0x7fad4010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.914+0000 7fad466cc700 1 --2- 172.21.15.12:0/2305123363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad40105f90 0x7fad40106370 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.914+0000 7fad466cc700 1 --2- 172.21.15.12:0/2305123363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad401000f0 0x7fad40100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.914+0000 7fad466cc700 1 -- 172.21.15.12:0/2305123363 >> 172.21.15.12:0/2305123363 conn(0x7fad40076ba0 msgr2=0x7fad40076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.914+0000 7fad466cc700 1 -- 172.21.15.12:0/2305123363 shutdown_connections 2024-05-08T23:55:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.914+0000 7fad466cc700 1 -- 172.21.15.12:0/2305123363 wait complete. 2024-05-08T23:55:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad466cc700 1 Processor -- start 2024-05-08T23:55:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad466cc700 1 -- start start 2024-05-08T23:55:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad466cc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad401000f0 0x7fad40197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad466cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad40100ab0 0x7fad40197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad466cc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad40105f90 0x7fad4019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad466cc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4019c240 con 0x7fad40100ab0 2024-05-08T23:55:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad466cc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4019c3b0 con 0x7fad401000f0 2024-05-08T23:55:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad466cc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4019c570 con 0x7fad40105f90 2024-05-08T23:55:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad3f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad40100ab0 0x7fad40197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad3f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad40100ab0 0x7fad40197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44456/0 (socket says 172.21.15.12:44456) 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad401000f0 0x7fad40197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.915+0000 7fad3f7fe700 1 -- 172.21.15.12:0/2383011666 learned_addr learned my addr 172.21.15.12:0/2383011666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad44c69700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad40105f90 0x7fad4019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad3f7fe700 1 -- 172.21.15.12:0/2383011666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad40105f90 msgr2=0x7fad4019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad40105f90 0x7fad4019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad3f7fe700 1 -- 172.21.15.12:0/2383011666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad401000f0 msgr2=0x7fad40197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad401000f0 0x7fad40197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad3f7fe700 1 -- 172.21.15.12:0/2383011666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad300095f0 con 0x7fad40100ab0 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad40100ab0 0x7fad40197860 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fad3400e9a0 tx=0x7fad3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad3d7fa700 1 -- 172.21.15.12:0/2383011666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34009790 con 0x7fad40100ab0 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad3d7fa700 1 -- 172.21.15.12:0/2383011666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad34010430 con 0x7fad40100ab0 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad466cc700 1 -- 172.21.15.12:0/2383011666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4019c850 con 0x7fad40100ab0 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad3d7fa700 1 -- 172.21.15.12:0/2383011666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34018680 con 0x7fad40100ab0 2024-05-08T23:55:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.916+0000 7fad466cc700 1 -- 172.21.15.12:0/2383011666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad401a4090 con 0x7fad40100ab0 2024-05-08T23:55:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.919+0000 7fad3d7fa700 1 -- 172.21.15.12:0/2383011666 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fad340105a0 con 0x7fad40100ab0 2024-05-08T23:55:53.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:53 smithi012 ceph-mon[42503]: pgmap v298: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-05-08T23:55:53.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.920+0000 7fad3d7fa700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad280a1290 0x7fad280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.920+0000 7fad3ffff700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad280a1290 0x7fad280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.920+0000 7fad3d7fa700 1 -- 172.21.15.12:0/2383011666 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fad34014070 con 0x7fad40100ab0 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.920+0000 7fad3ffff700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad280a1290 0x7fad280a3750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad30009990 tx=0x7fad30019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.920+0000 7fad52886b80 1 Processor -- start 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad52886b80 1 -- start start 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad52886b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b51031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad52886b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 0x5620b50fd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad52886b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b50fdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad52886b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5054150 con 0x5620b5102e00 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad52886b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5051b10 con 0x5620b5103720 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad52886b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b504d400 con 0x5620b50fd890 2024-05-08T23:55:53.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b51031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b51031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44466/0 (socket says 172.21.15.12:44466) 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3ffff700 1 -- 172.21.15.12:0/3447784450 learned_addr learned my addr 172.21.15.12:0/3447784450 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/3447784450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 0x5620b50fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad44c69700 1 --2- 172.21.15.12:0/3447784450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b50fdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3ffff700 1 -- 172.21.15.12:0/3447784450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 msgr2=0x5620b50fdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3ffff700 1 --2- 172.21.15.12:0/3447784450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b50fdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3ffff700 1 -- 172.21.15.12:0/3447784450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 msgr2=0x5620b50fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3ffff700 1 --2- 172.21.15.12:0/3447784450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 0x5620b50fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3ffff700 1 -- 172.21.15.12:0/3447784450 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b4ece3b0 con 0x5620b5102e00 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/3447784450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 0x5620b50fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.921+0000 7fad3ffff700 1 --2- 172.21.15.12:0/3447784450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b51031e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fad30000c00 tx=0x7fad3001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5c82a5dd9a7fd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.922+0000 7fad267fc700 1 -- 172.21.15.12:0/3447784450 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3000ba70 con 0x5620b5102e00 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.922+0000 7fad267fc700 1 -- 172.21.15.12:0/3447784450 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad3001d040 con 0x5620b5102e00 2024-05-08T23:55:53.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.922+0000 7fad267fc700 1 -- 172.21.15.12:0/3447784450 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3000f910 con 0x5620b5102e00 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.922+0000 7fad52886b80 1 -- 172.21.15.12:0/3447784450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 msgr2=0x5620b51031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.922+0000 7fad52886b80 1 --2- 172.21.15.12:0/3447784450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b51031e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fad30000c00 tx=0x7fad3001f000 comp rx=0 tx=0).stop 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.923+0000 7fad52886b80 1 -- 172.21.15.12:0/3447784450 shutdown_connections 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.923+0000 7fad52886b80 1 --2- 172.21.15.12:0/3447784450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b50fdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.923+0000 7fad52886b80 1 --2- 172.21.15.12:0/3447784450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b51031e0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.923+0000 7fad52886b80 1 --2- 172.21.15.12:0/3447784450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 0x5620b50fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.923+0000 7fad52886b80 1 -- 172.21.15.12:0/3447784450 >> 172.21.15.12:0/3447784450 conn(0x5620b505b210 msgr2=0x5620b50f7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.923+0000 7fad52886b80 1 -- 172.21.15.12:0/3447784450 shutdown_connections 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.923+0000 7fad52886b80 1 -- 172.21.15.12:0/3447784450 wait complete. 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.923+0000 7fad52886b80 1 -- start start 2024-05-08T23:55:53.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.923+0000 7fad52886b80 1 Processor -- start 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad52886b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b5106a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad52886b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5102e00 0x5620b5106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad52886b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5103720 0x5620b51074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad52886b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b4fe9560 con 0x5620b5103720 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad52886b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b4fe9200 con 0x5620b5102e00 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad52886b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b4ed06a0 con 0x5620b50fd890 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad52886b80 10 client.0 _ll_register_callbacks cb 0x5620b4ff2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad44c69700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5103720 0x5620b51074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad3f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5102e00 0x5620b5106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad44c69700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5103720 0x5620b51074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44472/0 (socket says 172.21.15.12:44472) 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad44c69700 1 -- 172.21.15.12:0/4009772022 learned_addr learned my addr 172.21.15.12:0/4009772022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b5106a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad3f7fe700 1 -- 172.21.15.12:0/4009772022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 msgr2=0x5620b5106a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b5106a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad3f7fe700 1 -- 172.21.15.12:0/4009772022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5103720 msgr2=0x5620b51074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5103720 0x5620b51074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad3f7fe700 1 -- 172.21.15.12:0/4009772022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad401a4090 con 0x5620b5102e00 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad44c69700 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5103720 0x5620b51074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5102e00 0x5620b5106fb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad340ca600 tx=0x7fad3409a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cfbaa0808c293e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad1f7fe700 1 -- 172.21.15.12:0/4009772022 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad340996b0 con 0x5620b5102e00 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad1f7fe700 1 -- 172.21.15.12:0/4009772022 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fad3409ddf0 con 0x5620b5102e00 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.924+0000 7fad52886b80 1 -- 172.21.15.12:0/4009772022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620b4fa6270 con 0x5620b5102e00 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad1f7fe700 1 -- 172.21.15.12:0/4009772022 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad340a2440 con 0x5620b5102e00 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad52886b80 10 client.27896 fetch_fsmap learned FSMap version 41 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad52886b80 1 -- 172.21.15.12:0/4009772022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fad4010d550 con 0x5620b5102e00 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad1f7fe700 1 -- 172.21.15.12:0/4009772022 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620b4fa6270 con 0x5620b5102e00 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad1f7fe700 1 -- 172.21.15.12:0/4009772022 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fad34098f20 con 0x5620b5102e00 2024-05-08T23:55:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad52886b80 10 client.27896 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad52886b80 10 client.27896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad52886b80 20 client.27896 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad52886b80 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620b51171c0 0x5620b51195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad52886b80 4 client.27896 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad52886b80 1 -- 172.21.15.12:0/4009772022 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620b50cc810 con 0x5620b51171c0 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.925+0000 7fad3ffff700 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620b51171c0 0x5620b51195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.926+0000 7fad3ffff700 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620b51171c0 0x5620b51195e0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316bdaecfa354580 server_cookie=dae445ff15d16b60 in_seq=0 out_seq=0 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.926+0000 7fad1f7fe700 10 client.27896 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.926+0000 7fad1f7fe700 1 -- 172.21.15.12:0/4009772022 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620b50cc810 con 0x5620b51171c0 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.926+0000 7fad1f7fe700 10 client.27896 handle_command_reply: tid=0 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 client.27896 shutdown 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 -- 172.21.15.12:0/4009772022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5102e00 msgr2=0x5620b5106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5102e00 0x5620b5106fb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad340ca600 tx=0x7fad3409a7b0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 -- 172.21.15.12:0/4009772022 shutdown_connections 2024-05-08T23:55:53.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5103720 0x5620b51074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b5106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620b51171c0 0x5620b51195e0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 --2- 172.21.15.12:0/4009772022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5102e00 0x5620b5106fb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad1f7fe700 0 client.27896 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:53.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 -- 172.21.15.12:0/4009772022 >> 172.21.15.12:0/4009772022 conn(0x5620b505b210 msgr2=0x5620b50f75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 -- 172.21.15.12:0/4009772022 shutdown_connections 2024-05-08T23:55:53.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 1 -- 172.21.15.12:0/4009772022 wait complete. 2024-05-08T23:55:53.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.927+0000 7fad52886b80 20 client.27896 trim_cache size 0 max 16384 2024-05-08T23:55:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.946+0000 7fad52886b80 1 Processor -- start 2024-05-08T23:55:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.946+0000 7fad52886b80 1 -- start start 2024-05-08T23:55:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad52886b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b50fd890 0x5620b50fd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad52886b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b50f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad52886b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b5103720 0x5620b5107310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad52886b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b50f4dc0 con 0x5620b5102e00 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad52886b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5103560 con 0x5620b50fd890 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad52886b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b511e7f0 con 0x5620b5103720 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad3f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b50f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b50fd890 0x5620b50fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad44c69700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b5103720 0x5620b5107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad3f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b50f7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44484/0 (socket says 172.21.15.12:44484) 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b50fd890 0x5620b50fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45550/0 (socket says 172.21.15.12:45550) 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad44c69700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b5103720 0x5620b5107310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33960/0 (socket says 172.21.15.12:33960) 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad3f7fe700 1 -- 172.21.15.12:0/2768690595 learned_addr learned my addr 172.21.15.12:0/2768690595 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad44c69700 1 -- 172.21.15.12:0/2768690595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b50fd890 msgr2=0x5620b50fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad44c69700 1 --2- 172.21.15.12:0/2768690595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b50fd890 0x5620b50fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad44c69700 1 -- 172.21.15.12:0/2768690595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 msgr2=0x5620b50f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad44c69700 1 --2- 172.21.15.12:0/2768690595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b50f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad44c69700 1 -- 172.21.15.12:0/2768690595 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b50fdf40 con 0x5620b5103720 2024-05-08T23:55:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad3ffff700 1 --2- 172.21.15.12:0/2768690595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b50fd890 0x5620b50fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:55:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.947+0000 7fad44c69700 1 --2- 172.21.15.12:0/2768690595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b5103720 0x5620b5107310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad38007510 tx=0x7fad38007540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=460195ba8c1505d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad26ffd700 1 -- 172.21.15.12:0/2768690595 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38004bb0 con 0x5620b5103720 2024-05-08T23:55:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad26ffd700 1 -- 172.21.15.12:0/2768690595 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad3800f040 con 0x5620b5103720 2024-05-08T23:55:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad26ffd700 1 -- 172.21.15.12:0/2768690595 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38005080 con 0x5620b5103720 2024-05-08T23:55:53.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad52886b80 1 -- 172.21.15.12:0/2768690595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b5103720 msgr2=0x5620b5107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad52886b80 1 --2- 172.21.15.12:0/2768690595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b5103720 0x5620b5107310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad38007510 tx=0x7fad38007540 comp rx=0 tx=0).stop 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad52886b80 1 -- 172.21.15.12:0/2768690595 shutdown_connections 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad52886b80 1 --2- 172.21.15.12:0/2768690595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b5103720 0x5620b5107310 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad52886b80 1 --2- 172.21.15.12:0/2768690595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b50fd890 0x5620b50fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad52886b80 1 --2- 172.21.15.12:0/2768690595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b50f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad52886b80 1 -- 172.21.15.12:0/2768690595 >> 172.21.15.12:0/2768690595 conn(0x5620b505b210 msgr2=0x5620b5101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.948+0000 7fad52886b80 1 -- 172.21.15.12:0/2768690595 shutdown_connections 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.949+0000 7fad52886b80 1 -- 172.21.15.12:0/2768690595 wait complete. 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.949+0000 7fad52886b80 1 -- start start 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.949+0000 7fad52886b80 1 Processor -- start 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.949+0000 7fad52886b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b510e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.949+0000 7fad52886b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b510a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.949+0000 7fad52886b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 0x5620b510ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.949+0000 7fad52886b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b510b160 con 0x5620b5102e00 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.949+0000 7fad52886b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b510b2a0 con 0x5620b5103720 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.949+0000 7fad52886b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b510b460 con 0x5620b50fd890 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.949+0000 7fad52886b80 10 client.0 _ll_register_callbacks cb 0x5620b4ff2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad3f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b510a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad44c69700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 0x5620b510ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad3f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b510a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44490/0 (socket says 172.21.15.12:44490) 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad3f7fe700 1 -- 172.21.15.12:0/3349433086 learned_addr learned my addr 172.21.15.12:0/3349433086 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad3ffff700 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b510e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad3f7fe700 1 -- 172.21.15.12:0/3349433086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 msgr2=0x5620b510e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b510e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad3f7fe700 1 -- 172.21.15.12:0/3349433086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 msgr2=0x5620b510ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 0x5620b510ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad3f7fe700 1 -- 172.21.15.12:0/3349433086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620b4ece3b0 con 0x5620b5102e00 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad3f7fe700 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b510a6e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fad340b4040 tx=0x7fad34099ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91deaf0a1df3180f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad1f7fe700 1 -- 172.21.15.12:0/3349433086 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34094dd0 con 0x5620b5102e00 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad1f7fe700 1 -- 172.21.15.12:0/3349433086 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fad3409ddf0 con 0x5620b5102e00 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad52886b80 1 -- 172.21.15.12:0/3349433086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad3001d040 con 0x5620b5102e00 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.950+0000 7fad1f7fe700 1 -- 172.21.15.12:0/3349433086 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad340b8390 con 0x5620b5102e00 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad52886b80 10 client.18135 fetch_fsmap learned FSMap version 41 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad52886b80 1 -- 172.21.15.12:0/3349433086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fad38009d60 con 0x5620b5102e00 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad1f7fe700 1 -- 172.21.15.12:0/3349433086 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad3001d040 con 0x5620b5102e00 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad1f7fe700 1 -- 172.21.15.12:0/3349433086 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fad340b2c70 con 0x5620b5102e00 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad52886b80 10 client.18135 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:55:53.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad52886b80 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad52886b80 20 client.18135 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad52886b80 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620b516e790 0x5620b5170bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad52886b80 4 client.18135 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad52886b80 1 -- 172.21.15.12:0/3349433086 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5620b5045ec0 con 0x5620b516e790 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.951+0000 7fad3ffff700 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620b516e790 0x5620b5170bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad1f7fe700 10 client.18135 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad3ffff700 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620b516e790 0x5620b5170bb0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e85a62336aec68 server_cookie=2373902ee230209a in_seq=0 out_seq=0 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad1f7fe700 1 -- 172.21.15.12:0/3349433086 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5620b5045ec0 con 0x5620b516e790 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad1f7fe700 10 client.18135 handle_command_reply: tid=0 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 client.18135 shutdown 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 -- 172.21.15.12:0/3349433086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 msgr2=0x5620b510a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b510a6e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fad340b4040 tx=0x7fad34099ec0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 -- 172.21.15.12:0/3349433086 shutdown_connections 2024-05-08T23:55:53.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620b516e790 0x5620b5170bb0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620b5103720 0x5620b510ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad1f7fe700 0 client.18135 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:53.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620b50fd890 0x5620b510e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 --2- 172.21.15.12:0/3349433086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620b5102e00 0x5620b510a6e0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 -- 172.21.15.12:0/3349433086 >> 172.21.15.12:0/3349433086 conn(0x5620b505b210 msgr2=0x5620b51023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 -- 172.21.15.12:0/3349433086 shutdown_connections 2024-05-08T23:55:53.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 1 -- 172.21.15.12:0/3349433086 wait complete. 2024-05-08T23:55:53.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.952+0000 7fad52886b80 20 client.18135 trim_cache size 0 max 16384 2024-05-08T23:55:53.953 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:55:53.954 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:55:53.954 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:55:53.954 INFO:teuthology.orchestra.run.smithi012.stdout: "ed63472c-3cd6-4fc4-ac7d-07fd7cf1c463": { 2024-05-08T23:55:53.954 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:55:53.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "ed63472c-3cd6-4fc4-ac7d-07fd7cf1c463", 2024-05-08T23:55:53.954 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:55:53.954 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:55:53.954 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:55:53.954 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:55:53.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 -- 172.21.15.12:0/2383011666 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad280a1290 msgr2=0x7fad280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad280a1290 0x7fad280a3750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad30009990 tx=0x7fad30019040 comp rx=0 tx=0).stop 2024-05-08T23:55:53.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 -- 172.21.15.12:0/2383011666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad40100ab0 msgr2=0x7fad40197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:53.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad40100ab0 0x7fad40197860 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fad3400e9a0 tx=0x7fad3400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 -- 172.21.15.12:0/2383011666 shutdown_connections 2024-05-08T23:55:53.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad280a1290 0x7fad280a3750 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad40105f90 0x7fad4019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad401000f0 0x7fad40197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 --2- 172.21.15.12:0/2383011666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad40100ab0 0x7fad40197860 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:53.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 -- 172.21.15.12:0/2383011666 >> 172.21.15.12:0/2383011666 conn(0x7fad40076ba0 msgr2=0x7fad400692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:53.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 -- 172.21.15.12:0/2383011666 shutdown_connections 2024-05-08T23:55:53.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:55:53.953+0000 7fad466cc700 1 -- 172.21.15.12:0/2383011666 wait complete. 2024-05-08T23:55:53.961 INFO:tasks.cephfs.filesystem:scrub status for tag:ed63472c-3cd6-4fc4-ac7d-07fd7cf1c463 - {'path': '/', 'tag': 'ed63472c-3cd6-4fc4-ac7d-07fd7cf1c463', 'options': 'recursive,force'} 2024-05-08T23:55:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:53 smithi204 ceph-mon[47373]: pgmap v298: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-05-08T23:55:54.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:53 smithi160 ceph-mon[46152]: pgmap v298: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-05-08T23:55:54.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 544006 6.94 MB/sec execute 350 sec latency 12.539 ms 2024-05-08T23:55:54.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:55:54.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:54.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:55:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:55:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:55:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:55:55.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:55:55.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:55:55.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:55:55.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:55:55.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 545212 6.94 MB/sec execute 351 sec latency 104.458 ms 2024-05-08T23:55:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:55 smithi012 ceph-mon[42503]: pgmap v299: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-05-08T23:55:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:55 smithi204 ceph-mon[47373]: pgmap v299: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-05-08T23:55:56.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:55 smithi160 ceph-mon[46152]: pgmap v299: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-05-08T23:55:56.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 547310 6.96 MB/sec execute 352 sec latency 244.188 ms 2024-05-08T23:55:57.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 548679 6.95 MB/sec execute 353 sec latency 15.372 ms 2024-05-08T23:55:57.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:57 smithi012 ceph-mon[42503]: pgmap v300: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-05-08T23:55:58.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:57 smithi204 ceph-mon[47373]: pgmap v300: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-05-08T23:55:58.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:57 smithi160 ceph-mon[46152]: pgmap v300: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-05-08T23:55:58.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 548762 6.93 MB/sec execute 354 sec latency 935.601 ms 2024-05-08T23:55:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.989+0000 7f1068347700 1 -- 172.21.15.12:0/3765940940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1060101100 msgr2=0x7f10601014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.989+0000 7f1068347700 1 --2- 172.21.15.12:0/3765940940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1060101100 0x7f10601014e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f105c009990 tx=0x7f105c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:55:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.989+0000 7f1068347700 1 -- 172.21.15.12:0/3765940940 shutdown_connections 2024-05-08T23:55:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.989+0000 7f1068347700 1 --2- 172.21.15.12:0/3765940940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1060106220 0x7f106010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.989+0000 7f1068347700 1 --2- 172.21.15.12:0/3765940940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1060101100 0x7f10601014e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.989+0000 7f1068347700 1 --2- 172.21.15.12:0/3765940940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1060101ab0 0x7f1060105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.989+0000 7f1068347700 1 -- 172.21.15.12:0/3765940940 >> 172.21.15.12:0/3765940940 conn(0x7f10600fc9e0 msgr2=0x7f10600fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.989+0000 7f1068347700 1 -- 172.21.15.12:0/3765940940 shutdown_connections 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.989+0000 7f1068347700 1 -- 172.21.15.12:0/3765940940 wait complete. 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f1068347700 1 Processor -- start 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f1068347700 1 -- start start 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f1068347700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1060101100 0x7f10601973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f1068347700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1060106220 0x7f1060197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f1068347700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f106019bcd0 0x7f106019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f1068347700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106019c750 con 0x7f1060106220 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f1068347700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106019c8c0 con 0x7f106019bcd0 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f1068347700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106019ca80 con 0x7f1060101100 2024-05-08T23:55:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f10658e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1060106220 0x7f1060197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f10658e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1060106220 0x7f1060197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40288/0 (socket says 172.21.15.12:40288) 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f10658e2700 1 -- 172.21.15.12:0/148928702 learned_addr learned my addr 172.21.15.12:0/148928702 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f10668e4700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f106019bcd0 0x7f106019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f10660e3700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1060101100 0x7f10601973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f10658e2700 1 -- 172.21.15.12:0/148928702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1060101100 msgr2=0x7f10601973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f10658e2700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1060101100 0x7f10601973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f10658e2700 1 -- 172.21.15.12:0/148928702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f106019bcd0 msgr2=0x7f106019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f10658e2700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f106019bcd0 0x7f106019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.990+0000 7f10658e2700 1 -- 172.21.15.12:0/148928702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c0095f0 con 0x7f1060106220 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.991+0000 7f10658e2700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1060106220 0x7f1060197930 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f105000d980 tx=0x7f105000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.991+0000 7f10577fe700 1 -- 172.21.15.12:0/148928702 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105000f890 con 0x7f1060106220 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.991+0000 7f10577fe700 1 -- 172.21.15.12:0/148928702 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1050010460 con 0x7f1060106220 2024-05-08T23:55:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.991+0000 7f1068347700 1 -- 172.21.15.12:0/148928702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10601a3fe0 con 0x7f1060106220 2024-05-08T23:55:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.991+0000 7f10577fe700 1 -- 172.21.15.12:0/148928702 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105000e610 con 0x7f1060106220 2024-05-08T23:55:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.991+0000 7f1068347700 1 -- 172.21.15.12:0/148928702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10601a4460 con 0x7f1060106220 2024-05-08T23:55:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.993+0000 7f10577fe700 1 -- 172.21.15.12:0/148928702 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f10500105d0 con 0x7f1060106220 2024-05-08T23:55:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.994+0000 7f10577fe700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f104c0a1290 0x7f104c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.994+0000 7f10577fe700 1 -- 172.21.15.12:0/148928702 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f10500cc6d0 con 0x7f1060106220 2024-05-08T23:55:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.994+0000 7f10660e3700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f104c0a1290 0x7f104c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.994+0000 7f10660e3700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f104c0a1290 0x7f104c0a3750 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f105c00b5c0 tx=0x7f105c005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f1074501b80 1 Processor -- start 2024-05-08T23:55:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f1074501b80 1 -- start start 2024-05-08T23:55:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f1074501b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178ac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f1074501b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178ace60 0x5575178b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f1074501b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178b0e40 0x5575178b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f1074501b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575177ff0f0 con 0x5575178ac540 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f1074501b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575177fcab0 con 0x5575178b0e40 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f1074501b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575177f83a0 con 0x5575178ace60 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f10660e3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f10660e3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178ac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40302/0 (socket says 172.21.15.12:40302) 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.995+0000 7f10660e3700 1 -- 172.21.15.12:0/3242567400 learned_addr learned my addr 172.21.15.12:0/3242567400 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f10668e4700 1 --2- 172.21.15.12:0/3242567400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178b0e40 0x5575178b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f10658e2700 1 --2- 172.21.15.12:0/3242567400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178ace60 0x5575178b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f10660e3700 1 -- 172.21.15.12:0/3242567400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178ace60 msgr2=0x5575178b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f10660e3700 1 --2- 172.21.15.12:0/3242567400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178ace60 0x5575178b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f10660e3700 1 -- 172.21.15.12:0/3242567400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178b0e40 msgr2=0x5575178b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f10660e3700 1 --2- 172.21.15.12:0/3242567400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178b0e40 0x5575178b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f10660e3700 1 -- 172.21.15.12:0/3242567400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557517679350 con 0x5575178ac540 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f10660e3700 1 --2- 172.21.15.12:0/3242567400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178ac920 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f105c005320 tx=0x7f105c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb94ba447452469d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f1054da4700 1 -- 172.21.15.12:0/3242567400 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c00bc00 con 0x5575178ac540 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f1054da4700 1 -- 172.21.15.12:0/3242567400 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f105c01e040 con 0x5575178ac540 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.996+0000 7f1054da4700 1 -- 172.21.15.12:0/3242567400 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c00f6a0 con 0x5575178ac540 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.997+0000 7f1074501b80 1 -- 172.21.15.12:0/3242567400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 msgr2=0x5575178ac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.997+0000 7f1074501b80 1 --2- 172.21.15.12:0/3242567400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178ac920 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f105c005320 tx=0x7f105c020000 comp rx=0 tx=0).stop 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.997+0000 7f1074501b80 1 -- 172.21.15.12:0/3242567400 shutdown_connections 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.997+0000 7f1074501b80 1 --2- 172.21.15.12:0/3242567400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178b0e40 0x5575178b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.997+0000 7f1074501b80 1 --2- 172.21.15.12:0/3242567400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178ac920 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.997+0000 7f1074501b80 1 --2- 172.21.15.12:0/3242567400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178ace60 0x5575178b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.997+0000 7f1074501b80 1 -- 172.21.15.12:0/3242567400 >> 172.21.15.12:0/3242567400 conn(0x5575178061b0 msgr2=0x5575178a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.997+0000 7f1074501b80 1 -- 172.21.15.12:0/3242567400 shutdown_connections 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.997+0000 7f1074501b80 1 -- 172.21.15.12:0/3242567400 wait complete. 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f1074501b80 1 -- start start 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f1074501b80 1 Processor -- start 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f1074501b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ac540 0x5575178b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f1074501b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ace60 0x5575178b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f1074501b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f1074501b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557517794500 con 0x5575178ace60 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f1074501b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575177941a0 con 0x5575178ac540 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f1074501b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55751767b640 con 0x5575178b0e40 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f1074501b80 10 client.0 _ll_register_callbacks cb 0x55751779dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f10658e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ace60 0x5575178b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f10658e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ace60 0x5575178b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40310/0 (socket says 172.21.15.12:40310) 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f10658e2700 1 -- 172.21.15.12:0/4281416451 learned_addr learned my addr 172.21.15.12:0/4281416451 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f10660e3700 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ac540 0x5575178b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f10668e4700 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f10658e2700 1 -- 172.21.15.12:0/4281416451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 msgr2=0x5575178b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f10658e2700 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f10658e2700 1 -- 172.21.15.12:0/4281416451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ac540 msgr2=0x5575178b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f10658e2700 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ac540 0x5575178b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.998+0000 7f10658e2700 1 -- 172.21.15.12:0/4281416451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106010e610 con 0x5575178ace60 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f10658e2700 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ace60 0x5575178b5f30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1050095ca0 tx=0x7f10500997f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f0a0b997d02d03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1049ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1049ffb700 1 -- 172.21.15.12:0/4281416451 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10500b92b0 con 0x5575178ace60 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1049ffb700 1 -- 172.21.15.12:0/4281416451 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f10500a1c70 con 0x5575178ace60 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1074501b80 1 -- 172.21.15.12:0/4281416451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557517751210 con 0x5575178ace60 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1049ffb700 1 -- 172.21.15.12:0/4281416451 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1050099a50 con 0x5575178ace60 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1074501b80 10 client.18147 fetch_fsmap learned FSMap version 41 2024-05-08T23:55:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1074501b80 1 -- 172.21.15.12:0/4281416451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x557517625b00 con 0x5575178ace60 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1049ffb700 1 -- 172.21.15.12:0/4281416451 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10500b7030 con 0x5575178ace60 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1049ffb700 1 -- 172.21.15.12:0/4281416451 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f10500a17e0 con 0x5575178ace60 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1074501b80 10 client.18147 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:58.999+0000 7f1074501b80 10 client.18147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.000+0000 7f1074501b80 20 client.18147 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.000+0000 7f1074501b80 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5575178c6060 0x5575178c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.000+0000 7f1074501b80 4 client.18147 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.000+0000 7f1074501b80 1 -- 172.21.15.12:0/4281416451 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575178777b0 con 0x5575178c6060 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.000+0000 7f10660e3700 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5575178c6060 0x5575178c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.000+0000 7f1049ffb700 10 client.18147 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.000+0000 7f10660e3700 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5575178c6060 0x5575178c8480 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4720901a39284512 server_cookie=6e932d7b46d311de in_seq=0 out_seq=0 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1049ffb700 1 -- 172.21.15.12:0/4281416451 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5575178777b0 con 0x5575178c6060 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1049ffb700 10 client.18147 handle_command_reply: tid=0 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 client.18147 shutdown 2024-05-08T23:55:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 -- 172.21.15.12:0/4281416451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ace60 msgr2=0x5575178b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ace60 0x5575178b5f30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1050095ca0 tx=0x7f10500997f0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 -- 172.21.15.12:0/4281416451 shutdown_connections 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5575178c6060 0x5575178c8480 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ac540 0x5575178b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 --2- 172.21.15.12:0/4281416451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ace60 0x5575178b5f30 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1049ffb700 0 client.18147 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 -- 172.21.15.12:0/4281416451 >> 172.21.15.12:0/4281416451 conn(0x5575178061b0 msgr2=0x5575178a6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 -- 172.21.15.12:0/4281416451 shutdown_connections 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.001+0000 7f1074501b80 1 -- 172.21.15.12:0/4281416451 wait complete. 2024-05-08T23:55:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.002+0000 7f1074501b80 20 client.18147 trim_cache size 0 max 16384 2024-05-08T23:55:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f1074501b80 1 Processor -- start 2024-05-08T23:55:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f1074501b80 1 -- start start 2024-05-08T23:55:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f1074501b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f1074501b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 0x5575178a2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f1074501b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f1074501b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575178cd700 con 0x5575178ac540 2024-05-08T23:55:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f1074501b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575178afbd0 con 0x5575178ace60 2024-05-08T23:55:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f1074501b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575178a49d0 con 0x5575178b0e40 2024-05-08T23:55:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f10660e3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f10660e3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40318/0 (socket says 172.21.15.12:40318) 2024-05-08T23:55:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f10660e3700 1 -- 172.21.15.12:0/455228444 learned_addr learned my addr 172.21.15.12:0/455228444 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f10658e2700 1 --2- 172.21.15.12:0/455228444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 0x5575178a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f10660e3700 1 -- 172.21.15.12:0/455228444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 msgr2=0x5575178a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f10660e3700 1 --2- 172.21.15.12:0/455228444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f10660e3700 1 -- 172.21.15.12:0/455228444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 msgr2=0x5575178a2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f10660e3700 1 --2- 172.21.15.12:0/455228444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 0x5575178a2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.017+0000 7f10660e3700 1 -- 172.21.15.12:0/455228444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575177be500 con 0x5575178ac540 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f10658e2700 1 --2- 172.21.15.12:0/455228444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 0x5575178a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f10660e3700 1 --2- 172.21.15.12:0/455228444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178c2100 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f105c032040 tx=0x7f105c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c635c7ced0e0816 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f10555a5700 1 -- 172.21.15.12:0/455228444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c00bc00 con 0x5575178ac540 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f10555a5700 1 -- 172.21.15.12:0/455228444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f105c00bd60 con 0x5575178ac540 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f10555a5700 1 -- 172.21.15.12:0/455228444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c03b4e0 con 0x5575178ac540 2024-05-08T23:55:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f1074501b80 1 -- 172.21.15.12:0/455228444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 msgr2=0x5575178c2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f1074501b80 1 --2- 172.21.15.12:0/455228444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178c2100 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f105c032040 tx=0x7f105c01a040 comp rx=0 tx=0).stop 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f1074501b80 1 -- 172.21.15.12:0/455228444 shutdown_connections 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f1074501b80 1 --2- 172.21.15.12:0/455228444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f1074501b80 1 --2- 172.21.15.12:0/455228444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178c2100 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f1074501b80 1 --2- 172.21.15.12:0/455228444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 0x5575178a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.018+0000 7f1074501b80 1 -- 172.21.15.12:0/455228444 >> 172.21.15.12:0/455228444 conn(0x5575178061b0 msgr2=0x5575178af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 1 -- 172.21.15.12:0/455228444 shutdown_connections 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 1 -- 172.21.15.12:0/455228444 wait complete. 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 1 -- start start 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 1 Processor -- start 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 0x5575178b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575178b59b0 con 0x5575178ac540 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575178b5af0 con 0x5575178ace60 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575178b5c60 con 0x5575178b0e40 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f1074501b80 10 client.0 _ll_register_callbacks cb 0x55751779dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f10660e3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f10660e3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178bd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40330/0 (socket says 172.21.15.12:40330) 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.019+0000 7f10660e3700 1 -- 172.21.15.12:0/1094812553 learned_addr learned my addr 172.21.15.12:0/1094812553 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f10668e4700 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f10658e2700 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 0x5575178b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f10660e3700 1 -- 172.21.15.12:0/1094812553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 msgr2=0x5575178b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f10660e3700 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f10660e3700 1 -- 172.21.15.12:0/1094812553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 msgr2=0x5575178b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f10660e3700 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 0x5575178b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f10660e3700 1 -- 172.21.15.12:0/1094812553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557517679350 con 0x5575178ac540 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f10658e2700 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 0x5575178b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f10660e3700 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178bd1a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f105c000c00 tx=0x7f105c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d917100a8de371b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f1049ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f1049ffb700 1 -- 172.21.15.12:0/1094812553 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c004b10 con 0x5575178ac540 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f1049ffb700 1 -- 172.21.15.12:0/1094812553 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f105c03b160 con 0x5575178ac540 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f1074501b80 1 -- 172.21.15.12:0/1094812553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f105c01e040 con 0x5575178ac540 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f1049ffb700 1 -- 172.21.15.12:0/1094812553 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f105c004c70 con 0x5575178ac540 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f1074501b80 10 client.18153 fetch_fsmap learned FSMap version 41 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f1074501b80 1 -- 172.21.15.12:0/1094812553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55751767dc70 con 0x5575178ac540 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.020+0000 7f1049ffb700 1 -- 172.21.15.12:0/1094812553 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f105c053020 con 0x5575178ac540 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.021+0000 7f1049ffb700 1 -- 172.21.15.12:0/1094812553 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f105c044070 con 0x5575178ac540 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.021+0000 7f1074501b80 10 client.18153 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.021+0000 7f1074501b80 10 client.18153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.021+0000 7f1074501b80 20 client.18153 populate_metadata read hostname 'smithi012' 2024-05-08T23:55:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.021+0000 7f1074501b80 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55751791c8d0 0x55751791ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:55:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.021+0000 7f1074501b80 4 client.18153 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:55:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.021+0000 7f1074501b80 1 -- 172.21.15.12:0/1094812553 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5575177f0e60 con 0x55751791c8d0 2024-05-08T23:55:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.021+0000 7f10658e2700 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55751791c8d0 0x55751791ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:55:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.021+0000 7f1049ffb700 10 client.18153 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.021+0000 7f10658e2700 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55751791c8d0 0x55751791ecf0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f32becebc4f30bca server_cookie=f5f145a20e63d53d in_seq=0 out_seq=0 2024-05-08T23:55:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1049ffb700 1 -- 172.21.15.12:0/1094812553 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5575177f0e60 con 0x55751791c8d0 2024-05-08T23:55:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1049ffb700 10 client.18153 handle_command_reply: tid=0 2024-05-08T23:55:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1074501b80 1 client.18153 shutdown 2024-05-08T23:55:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1074501b80 1 -- 172.21.15.12:0/1094812553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 msgr2=0x5575178bd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1074501b80 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178bd1a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f105c000c00 tx=0x7f105c00bcb0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1074501b80 1 -- 172.21.15.12:0/1094812553 shutdown_connections 2024-05-08T23:55:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1074501b80 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5575178b0e40 0x5575178b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1074501b80 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5575178ac540 0x5575178bd1a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1074501b80 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55751791c8d0 0x55751791ecf0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1074501b80 1 --2- 172.21.15.12:0/1094812553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575178ace60 0x5575178b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.067+0000 7f1074501b80 1 -- 172.21.15.12:0/1094812553 >> 172.21.15.12:0/1094812553 conn(0x5575178061b0 msgr2=0x5575178af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.068+0000 7f1049ffb700 0 client.18153 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:55:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.068+0000 7f1074501b80 1 -- 172.21.15.12:0/1094812553 shutdown_connections 2024-05-08T23:55:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.068+0000 7f1074501b80 1 -- 172.21.15.12:0/1094812553 wait complete. 2024-05-08T23:55:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.068+0000 7f1074501b80 20 client.18153 trim_cache size 0 max 16384 2024-05-08T23:55:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:55:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:55:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:55:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:55:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.068+0000 7f1068347700 1 -- 172.21.15.12:0/148928702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f104c0a1290 msgr2=0x7f104c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.068+0000 7f1068347700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f104c0a1290 0x7f104c0a3750 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f105c00b5c0 tx=0x7f105c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.068+0000 7f1068347700 1 -- 172.21.15.12:0/148928702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1060106220 msgr2=0x7f1060197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:55:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.068+0000 7f1068347700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1060106220 0x7f1060197930 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f105000d980 tx=0x7f105000dc90 comp rx=0 tx=0).stop 2024-05-08T23:55:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.069+0000 7f1068347700 1 -- 172.21.15.12:0/148928702 shutdown_connections 2024-05-08T23:55:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.069+0000 7f1068347700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f104c0a1290 0x7f104c0a3750 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.069+0000 7f1068347700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f106019bcd0 0x7f106019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.069+0000 7f1068347700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1060101100 0x7f10601973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.069+0000 7f1068347700 1 --2- 172.21.15.12:0/148928702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1060106220 0x7f1060197930 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:55:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.069+0000 7f1068347700 1 -- 172.21.15.12:0/148928702 >> 172.21.15.12:0/148928702 conn(0x7f10600fc9e0 msgr2=0x7f10600fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:55:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.069+0000 7f1068347700 1 -- 172.21.15.12:0/148928702 shutdown_connections 2024-05-08T23:55:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:55:59.069+0000 7f1068347700 1 -- 172.21.15.12:0/148928702 wait complete. 2024-05-08T23:55:59.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 548762 6.92 MB/sec execute 355 sec latency 1935.698 ms 2024-05-08T23:55:59.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:55:59 smithi012 ceph-mon[42503]: pgmap v301: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-08T23:56:00.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:55:59 smithi204 ceph-mon[47373]: pgmap v301: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-08T23:56:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:55:59 smithi160 ceph-mon[46152]: pgmap v301: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-08T23:56:00.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 548762 6.90 MB/sec execute 356 sec latency 2935.759 ms 2024-05-08T23:56:01.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 550057 6.90 MB/sec execute 357 sec latency 3075.337 ms 2024-05-08T23:56:01.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:01 smithi012 ceph-mon[42503]: pgmap v302: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-05-08T23:56:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:01 smithi204 ceph-mon[47373]: pgmap v302: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-05-08T23:56:02.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:01 smithi160 ceph-mon[46152]: pgmap v302: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-05-08T23:56:02.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 551440 6.90 MB/sec execute 358 sec latency 29.339 ms 2024-05-08T23:56:03.882 INFO:tasks.workunit.client.0.smithi012.stdout: 1 552305 6.89 MB/sec execute 359 sec latency 485.898 ms 2024-05-08T23:56:04.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:04 smithi012 ceph-mon[42503]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T23:56:04.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:04 smithi012 ceph-mon[42503]: pgmap v304: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-05-08T23:56:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:04 smithi204 ceph-mon[47373]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T23:56:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:04 smithi204 ceph-mon[47373]: pgmap v304: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-05-08T23:56:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:04 smithi160 ceph-mon[46152]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T23:56:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:04 smithi160 ceph-mon[46152]: pgmap v304: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-05-08T23:56:04.883 INFO:tasks.workunit.client.0.smithi012.stdout: 1 552305 6.87 MB/sec execute 360 sec latency 1486.021 ms 2024-05-08T23:56:05.883 INFO:tasks.workunit.client.0.smithi012.stdout: 1 552305 6.85 MB/sec execute 361 sec latency 2486.139 ms 2024-05-08T23:56:05.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:05 smithi012 ceph-mon[42503]: pgmap v305: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-05-08T23:56:06.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:05 smithi204 ceph-mon[47373]: pgmap v305: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-05-08T23:56:06.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:05 smithi160 ceph-mon[46152]: pgmap v305: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-05-08T23:56:06.883 INFO:tasks.workunit.client.0.smithi012.stdout: 1 554373 6.87 MB/sec execute 362 sec latency 2629.639 ms 2024-05-08T23:56:07.883 INFO:tasks.workunit.client.0.smithi012.stdout: 1 555563 6.87 MB/sec execute 363 sec latency 46.733 ms 2024-05-08T23:56:07.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:07 smithi012 ceph-mon[42503]: pgmap v306: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-08T23:56:08.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:07 smithi204 ceph-mon[47373]: pgmap v306: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-08T23:56:08.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:07 smithi160 ceph-mon[46152]: pgmap v306: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-08T23:56:08.883 INFO:tasks.workunit.client.0.smithi012.stdout: 1 555852 6.85 MB/sec execute 364 sec latency 785.567 ms 2024-05-08T23:56:08.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:08 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:56:09.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:08 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:56:09.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:08 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:56:09.883 INFO:tasks.workunit.client.0.smithi012.stdout: 1 555852 6.83 MB/sec execute 365 sec latency 1785.662 ms 2024-05-08T23:56:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:09 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:56:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:09 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:56:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:09 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:56:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:09 smithi012 ceph-mon[42503]: pgmap v307: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-08T23:56:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:09 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:56:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:09 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:56:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:09 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:56:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:09 smithi204 ceph-mon[47373]: pgmap v307: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-08T23:56:10.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:09 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:56:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:09 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:56:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:09 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:56:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:09 smithi160 ceph-mon[46152]: pgmap v307: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-08T23:56:10.883 INFO:tasks.workunit.client.0.smithi012.stdout: 1 555852 6.81 MB/sec execute 366 sec latency 2785.776 ms 2024-05-08T23:56:11.883 INFO:tasks.workunit.client.0.smithi012.stdout: 1 557954 6.83 MB/sec execute 367 sec latency 2929.529 ms 2024-05-08T23:56:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:11 smithi012 ceph-mon[42503]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T23:56:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:11 smithi012 ceph-mon[42503]: pgmap v309: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-05-08T23:56:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.201+0000 7f6b502f1700 1 -- 172.21.15.12:0/2114596202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b48101ab0 msgr2=0x7f6b48105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.201+0000 7f6b502f1700 1 --2- 172.21.15.12:0/2114596202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b48101ab0 0x7f6b48105af0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009990 tx=0x7f6b38009ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.201+0000 7f6b502f1700 1 -- 172.21.15.12:0/2114596202 shutdown_connections 2024-05-08T23:56:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.201+0000 7f6b502f1700 1 --2- 172.21.15.12:0/2114596202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b48106220 0x7f6b4810dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.201+0000 7f6b502f1700 1 --2- 172.21.15.12:0/2114596202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b48101100 0x7f6b481014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.201+0000 7f6b502f1700 1 --2- 172.21.15.12:0/2114596202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b48101ab0 0x7f6b48105af0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.201+0000 7f6b502f1700 1 -- 172.21.15.12:0/2114596202 >> 172.21.15.12:0/2114596202 conn(0x7f6b480fc9a0 msgr2=0x7f6b480fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.201+0000 7f6b502f1700 1 -- 172.21.15.12:0/2114596202 shutdown_connections 2024-05-08T23:56:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.201+0000 7f6b502f1700 1 -- 172.21.15.12:0/2114596202 wait complete. 2024-05-08T23:56:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.202+0000 7f6b502f1700 1 Processor -- start 2024-05-08T23:56:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.202+0000 7f6b502f1700 1 -- start start 2024-05-08T23:56:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.202+0000 7f6b502f1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b48101100 0x7f6b480732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.202+0000 7f6b502f1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b48106220 0x7f6b480737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.202+0000 7f6b502f1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b48073d20 0x7f6b481a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.202+0000 7f6b502f1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b481a5be0 con 0x7f6b48073d20 2024-05-08T23:56:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.202+0000 7f6b502f1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b481a5d50 con 0x7f6b48101100 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.202+0000 7f6b502f1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b481a5f10 con 0x7f6b48106220 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4e88e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b48073d20 0x7f6b481a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4d88c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b48106220 0x7f6b480737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4e88e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b48073d20 0x7f6b481a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39110/0 (socket says 172.21.15.12:39110) 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4e88e700 1 -- 172.21.15.12:0/2701078306 learned_addr learned my addr 172.21.15.12:0/2701078306 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4e08d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b48101100 0x7f6b480732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4e88e700 1 -- 172.21.15.12:0/2701078306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b48106220 msgr2=0x7f6b480737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b48106220 0x7f6b480737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4e88e700 1 -- 172.21.15.12:0/2701078306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b48101100 msgr2=0x7f6b480732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b48101100 0x7f6b480732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4e88e700 1 -- 172.21.15.12:0/2701078306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b380095f0 con 0x7f6b48073d20 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b48073d20 0x7f6b481a55e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000ef30 tx=0x7f6b4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b3f7fe700 1 -- 172.21.15.12:0/2701078306 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40018070 con 0x7f6b48073d20 2024-05-08T23:56:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b3f7fe700 1 -- 172.21.15.12:0/2701078306 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b4000f040 con 0x7f6b48073d20 2024-05-08T23:56:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b502f1700 1 -- 172.21.15.12:0/2701078306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b481a6190 con 0x7f6b48073d20 2024-05-08T23:56:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b3f7fe700 1 -- 172.21.15.12:0/2701078306 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4000cd00 con 0x7f6b48073d20 2024-05-08T23:56:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.203+0000 7f6b502f1700 1 -- 172.21.15.12:0/2701078306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b481a6620 con 0x7f6b48073d20 2024-05-08T23:56:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.205+0000 7f6b3f7fe700 1 -- 172.21.15.12:0/2701078306 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6b40003a00 con 0x7f6b48073d20 2024-05-08T23:56:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.206+0000 7f6b3f7fe700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b340a1290 0x7f6b340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.206+0000 7f6b3f7fe700 1 -- 172.21.15.12:0/2701078306 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6b400cc7d0 con 0x7f6b48073d20 2024-05-08T23:56:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.206+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b340a1290 0x7f6b340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.207+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b340a1290 0x7f6b340a3750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009c60 tx=0x7f6b44009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.207+0000 7f6b5c4abb80 1 Processor -- start 2024-05-08T23:56:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.207+0000 7f6b5c4abb80 1 -- start start 2024-05-08T23:56:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03239d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03233f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba032348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0318b0f0 con 0x55ba03239950 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba03188ab0 con 0x55ba0323a270 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba031843a0 con 0x55ba03234480 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e08d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03239d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e08d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03239d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39124/0 (socket says 172.21.15.12:39124) 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e08d700 1 -- 172.21.15.12:0/4001898696 learned_addr learned my addr 172.21.15.12:0/4001898696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/4001898696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba032348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4d88c700 1 --2- 172.21.15.12:0/4001898696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03233f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e88e700 1 -- 172.21.15.12:0/4001898696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 msgr2=0x55ba03233f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/4001898696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03233f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e88e700 1 -- 172.21.15.12:0/4001898696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 msgr2=0x55ba03239d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/4001898696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03239d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e88e700 1 -- 172.21.15.12:0/4001898696 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba03005350 con 0x55ba03234480 2024-05-08T23:56:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/4001898696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03239d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.208+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/4001898696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba032348a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6b400c9f30 tx=0x7f6b400994c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a57e885d96687e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.209+0000 7f6b3cda4700 1 -- 172.21.15.12:0/4001898696 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40004ad0 con 0x55ba03234480 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.209+0000 7f6b3cda4700 1 -- 172.21.15.12:0/4001898696 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b4001b320 con 0x55ba03234480 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.209+0000 7f6b3cda4700 1 -- 172.21.15.12:0/4001898696 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40099be0 con 0x55ba03234480 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.209+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/4001898696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 msgr2=0x55ba032348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.209+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/4001898696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba032348a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6b400c9f30 tx=0x7f6b400994c0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.209+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/4001898696 shutdown_connections 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.209+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/4001898696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba032348a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.209+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/4001898696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03239d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.209+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/4001898696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03233f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.209+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/4001898696 >> 172.21.15.12:0/4001898696 conn(0x55ba031921b0 msgr2=0x55ba0322e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/4001898696 shutdown_connections 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/4001898696 wait complete. 2024-05-08T23:56:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 1 -- start start 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 1 Processor -- start 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba03240210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba03239950 0x55ba0323d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba0323a270 0x55ba0323dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba03120500 con 0x55ba0323a270 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba031201a0 con 0x55ba03239950 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba03007640 con 0x55ba03234480 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b5c4abb80 10 client.0 _ll_register_callbacks cb 0x55ba03129ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b4e88e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba0323a270 0x55ba0323dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b4d88c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba03239950 0x55ba0323d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b4e88e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba0323a270 0x55ba0323dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39132/0 (socket says 172.21.15.12:39132) 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.210+0000 7f6b4e88e700 1 -- 172.21.15.12:0/630771955 learned_addr learned my addr 172.21.15.12:0/630771955 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba03240210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b4e88e700 1 -- 172.21.15.12:0/630771955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 msgr2=0x55ba03240210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba03240210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b4e88e700 1 -- 172.21.15.12:0/630771955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba03239950 msgr2=0x55ba0323d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba03239950 0x55ba0323d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b4e88e700 1 -- 172.21.15.12:0/630771955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b44017070 con 0x55ba0323a270 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b4d88c700 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba03239950 0x55ba0323d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba03240210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba0323a270 0x55ba0323dc90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6b400ca530 tx=0x7f6b400990a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a17895c161a3232 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b31ffb700 1 -- 172.21.15.12:0/630771955 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40004ad0 con 0x55ba0323a270 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b31ffb700 1 -- 172.21.15.12:0/630771955 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6b4009a080 con 0x55ba0323a270 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/630771955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba030dd210 con 0x55ba0323a270 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b31ffb700 1 -- 172.21.15.12:0/630771955 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b400b2260 con 0x55ba0323a270 2024-05-08T23:56:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b5c4abb80 10 client.18165 fetch_fsmap learned FSMap version 42 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.211+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/630771955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f6b481a6620 con 0x55ba0323a270 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.212+0000 7f6b31ffb700 1 -- 172.21.15.12:0/630771955 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba030dd210 con 0x55ba0323a270 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.212+0000 7f6b31ffb700 1 -- 172.21.15.12:0/630771955 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f6b400a18e0 con 0x55ba0323a270 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.212+0000 7f6b5c4abb80 10 client.18165 fetch_fsmap finished waiting for FSMap version 42 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.212+0000 7f6b5c4abb80 10 client.18165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.212+0000 7f6b5c4abb80 20 client.18165 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.212+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ba0324d740 0x55ba0324fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.212+0000 7f6b5c4abb80 4 client.18165 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.212+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/630771955 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba032037b0 con 0x55ba0324d740 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.212+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ba0324d740 0x55ba0324fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.213+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ba0324d740 0x55ba0324fb30 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad31f718a1e5b56d server_cookie=28e2f447f49c220d in_seq=0 out_seq=0 2024-05-08T23:56:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.213+0000 7f6b31ffb700 10 client.18165 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.214+0000 7f6b31ffb700 1 -- 172.21.15.12:0/630771955 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba032037b0 con 0x55ba0324d740 2024-05-08T23:56:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.214+0000 7f6b31ffb700 10 client.18165 handle_command_reply: tid=0 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.214+0000 7f6b5c4abb80 1 client.18165 shutdown 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/630771955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba0323a270 msgr2=0x55ba0323dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba0323a270 0x55ba0323dc90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6b400ca530 tx=0x7f6b400990a0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/630771955 shutdown_connections 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba0323a270 0x55ba0323dc90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba03240210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ba0324d740 0x55ba0324fb30 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/630771955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba03239950 0x55ba0323d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b31ffb700 0 client.18165 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/630771955 >> 172.21.15.12:0/630771955 conn(0x55ba031921b0 msgr2=0x55ba03232f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/630771955 shutdown_connections 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/630771955 wait complete. 2024-05-08T23:56:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.215+0000 7f6b5c4abb80 20 client.18165 trim_cache size 0 max 16384 2024-05-08T23:56:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.231+0000 7f6b5c4abb80 1 Processor -- start 2024-05-08T23:56:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.231+0000 7f6b5c4abb80 1 -- start start 2024-05-08T23:56:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.231+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03234480 0x55ba032349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.231+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03239950 0x55ba0322e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.231+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03244be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.231+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba032395c0 con 0x55ba03234480 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.231+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba03254d20 con 0x55ba0323a270 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.231+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba03238190 con 0x55ba03239950 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4e08d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03234480 0x55ba032349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4e08d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03234480 0x55ba032349a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39134/0 (socket says 172.21.15.12:39134) 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4e08d700 1 -- 172.21.15.12:0/1526627862 learned_addr learned my addr 172.21.15.12:0/1526627862 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/1526627862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03244be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4d88c700 1 --2- 172.21.15.12:0/1526627862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03239950 0x55ba0322e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4e08d700 1 -- 172.21.15.12:0/1526627862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03239950 msgr2=0x55ba0322e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/1526627862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03239950 0x55ba0322e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4e08d700 1 -- 172.21.15.12:0/1526627862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 msgr2=0x55ba03244be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/1526627862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03244be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4e08d700 1 -- 172.21.15.12:0/1526627862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba02fb1b00 con 0x55ba03234480 2024-05-08T23:56:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/1526627862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03234480 0x55ba032349a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6b44013010 tx=0x7f6b44014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0e87ac97a338fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b3d5a5700 1 -- 172.21.15.12:0/1526627862 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44022030 con 0x55ba03234480 2024-05-08T23:56:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b3d5a5700 1 -- 172.21.15.12:0/1526627862 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b44023040 con 0x55ba03234480 2024-05-08T23:56:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.232+0000 7f6b3d5a5700 1 -- 172.21.15.12:0/1526627862 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4401daa0 con 0x55ba03234480 2024-05-08T23:56:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/1526627862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03234480 msgr2=0x55ba032349a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/1526627862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03234480 0x55ba032349a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6b44013010 tx=0x7f6b44014000 comp rx=0 tx=0).stop 2024-05-08T23:56:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/1526627862 shutdown_connections 2024-05-08T23:56:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/1526627862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03244be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/1526627862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03234480 0x55ba032349a0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/1526627862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03239950 0x55ba0322e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/1526627862 >> 172.21.15.12:0/1526627862 conn(0x55ba031921b0 msgr2=0x55ba0322ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/1526627862 shutdown_connections 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/1526627862 wait complete. 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 -- start start 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.233+0000 7f6b5c4abb80 1 Processor -- start 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba03242bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03243100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b5c4abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03243640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba032a2c10 con 0x55ba03239950 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba032a2d50 con 0x55ba0323a270 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b5c4abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba032a2ee0 con 0x55ba03234480 2024-05-08T23:56:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b5c4abb80 10 client.0 _ll_register_callbacks cb 0x55ba03129ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4d88c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03243100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4e88e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03243640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4d88c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03243100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39150/0 (socket says 172.21.15.12:39150) 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4e08d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba03242bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4d88c700 1 -- 172.21.15.12:0/2061477397 learned_addr learned my addr 172.21.15.12:0/2061477397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4d88c700 1 -- 172.21.15.12:0/2061477397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 msgr2=0x55ba03242bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4d88c700 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba03242bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4d88c700 1 -- 172.21.15.12:0/2061477397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 msgr2=0x55ba03243640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4d88c700 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03243640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4d88c700 1 -- 172.21.15.12:0/2061477397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b44003700 con 0x55ba03239950 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba03242bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4e88e700 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03243640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b4d88c700 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03243100 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6b38015000 tx=0x7f6b3800ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214270f61d032e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.234+0000 7f6b31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b31ffb700 1 -- 172.21.15.12:0/2061477397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3801d070 con 0x55ba03239950 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b31ffb700 1 -- 172.21.15.12:0/2061477397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6b38004910 con 0x55ba03239950 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/2061477397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b4001b320 con 0x55ba03239950 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b31ffb700 1 -- 172.21.15.12:0/2061477397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b38004ad0 con 0x55ba03239950 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b5c4abb80 10 client.18171 fetch_fsmap learned FSMap version 42 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/2061477397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55ba0314a500 con 0x55ba03239950 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b31ffb700 1 -- 172.21.15.12:0/2061477397 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b4001b320 con 0x55ba03239950 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b31ffb700 1 -- 172.21.15.12:0/2061477397 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f6b3800f460 con 0x55ba03239950 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b5c4abb80 10 client.18171 fetch_fsmap finished waiting for FSMap version 42 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b5c4abb80 10 client.18171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b5c4abb80 20 client.18171 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ba032a47a0 0x55ba032a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b5c4abb80 4 client.18171 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:56:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.235+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/2061477397 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba0317ce60 con 0x55ba032a47a0 2024-05-08T23:56:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.236+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ba032a47a0 0x55ba032a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.236+0000 7f6b31ffb700 10 client.18171 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.236+0000 7f6b4e08d700 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ba032a47a0 0x55ba032a6bc0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c84640afd78f845 server_cookie=fba1af50f976f4a9 in_seq=0 out_seq=0 2024-05-08T23:56:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:11 smithi204 ceph-mon[47373]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T23:56:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:11 smithi204 ceph-mon[47373]: pgmap v309: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-05-08T23:56:12.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:11 smithi160 ceph-mon[46152]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T23:56:12.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:11 smithi160 ceph-mon[46152]: pgmap v309: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-05-08T23:56:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b31ffb700 1 -- 172.21.15.12:0/2061477397 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba0317ce60 con 0x55ba032a47a0 2024-05-08T23:56:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b31ffb700 10 client.18171 handle_command_reply: tid=0 2024-05-08T23:56:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 client.18171 shutdown 2024-05-08T23:56:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/2061477397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 msgr2=0x55ba03243100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03243100 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6b38015000 tx=0x7f6b3800ba60 comp rx=0 tx=0).stop 2024-05-08T23:56:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/2061477397 shutdown_connections 2024-05-08T23:56:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ba032a47a0 0x55ba032a6bc0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ba0323a270 0x55ba03243640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba03234480 0x55ba03242bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b31ffb700 0 client.18171 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 --2- 172.21.15.12:0/2061477397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba03239950 0x55ba03243100 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/2061477397 >> 172.21.15.12:0/2061477397 conn(0x55ba031921b0 msgr2=0x55ba0323bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/2061477397 shutdown_connections 2024-05-08T23:56:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 1 -- 172.21.15.12:0/2061477397 wait complete. 2024-05-08T23:56:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.286+0000 7f6b5c4abb80 20 client.18171 trim_cache size 0 max 16384 2024-05-08T23:56:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:56:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:56:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:56:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:56:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 -- 172.21.15.12:0/2701078306 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b340a1290 msgr2=0x7f6b340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b340a1290 0x7f6b340a3750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009c60 tx=0x7f6b44009260 comp rx=0 tx=0).stop 2024-05-08T23:56:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 -- 172.21.15.12:0/2701078306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b48073d20 msgr2=0x7f6b481a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b48073d20 0x7f6b481a55e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000ef30 tx=0x7f6b4000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 -- 172.21.15.12:0/2701078306 shutdown_connections 2024-05-08T23:56:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b340a1290 0x7f6b340a3750 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b48073d20 0x7f6b481a55e0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b48101100 0x7f6b480732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 --2- 172.21.15.12:0/2701078306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b48106220 0x7f6b480737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 -- 172.21.15.12:0/2701078306 >> 172.21.15.12:0/2701078306 conn(0x7f6b480fc9a0 msgr2=0x7f6b480fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.287+0000 7f6b502f1700 1 -- 172.21.15.12:0/2701078306 shutdown_connections 2024-05-08T23:56:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:12.288+0000 7f6b502f1700 1 -- 172.21.15.12:0/2701078306 wait complete. 2024-05-08T23:56:12.883 INFO:tasks.workunit.client.0.smithi012.stdout: 1 559158 6.83 MB/sec execute 368 sec latency 31.212 ms 2024-05-08T23:56:13.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:12 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-08T23:56:13.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:12 smithi204 ceph-mon[47373]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T23:56:13.013 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:56:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-08T23:56:13.013 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:56:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-08T23:56:13.013 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:56:12 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/1491923641 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f8cb7501a0 con 0x55f8cb727000 2024-05-08T23:56:13.013 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:56:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-08T23:56:13.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:12 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-08T23:56:13.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:12 smithi012 ceph-mon[42503]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T23:56:13.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:12 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-08T23:56:13.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:12 smithi160 ceph-mon[46152]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T23:56:13.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 559397 6.81 MB/sec execute 369 sec latency 840.553 ms 2024-05-08T23:56:14.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:13 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/2536985900,v1:172.21.15.204:6837/2536985900] up:boot 2024-05-08T23:56:14.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:13 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:56:14.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:13 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:56:14.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:13 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:56:14.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:13 smithi012 ceph-mon[42503]: pgmap v311: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-05-08T23:56:14.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:13 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/2536985900,v1:172.21.15.204:6837/2536985900] up:boot 2024-05-08T23:56:14.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:13 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:56:14.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:13 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:56:14.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:13 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:56:14.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:13 smithi204 ceph-mon[47373]: pgmap v311: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-05-08T23:56:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:13 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/2536985900,v1:172.21.15.204:6837/2536985900] up:boot 2024-05-08T23:56:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:13 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:56:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:13 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:56:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:13 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:56:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:13 smithi160 ceph-mon[46152]: pgmap v311: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-05-08T23:56:14.883 INFO:tasks.workunit.client.0.smithi012.stdout: 1 559397 6.79 MB/sec execute 370 sec latency 1840.654 ms 2024-05-08T23:56:15.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 559397 6.77 MB/sec execute 371 sec latency 2840.763 ms 2024-05-08T23:56:15.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:15 smithi012 ceph-mon[42503]: pgmap v312: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 950 KiB/s wr, 68 op/s 2024-05-08T23:56:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:15 smithi204 ceph-mon[47373]: pgmap v312: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 950 KiB/s wr, 68 op/s 2024-05-08T23:56:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:15 smithi160 ceph-mon[46152]: pgmap v312: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 950 KiB/s wr, 68 op/s 2024-05-08T23:56:16.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 561497 6.79 MB/sec execute 372 sec latency 2981.109 ms 2024-05-08T23:56:17.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 562863 6.79 MB/sec execute 373 sec latency 39.564 ms 2024-05-08T23:56:17.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:17 smithi012 ceph-mon[42503]: pgmap v313: 97 pgs: 97 active+clean; 161 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-05-08T23:56:18.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:17 smithi204 ceph-mon[47373]: pgmap v313: 97 pgs: 97 active+clean; 161 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-05-08T23:56:18.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:17 smithi160 ceph-mon[46152]: pgmap v313: 97 pgs: 97 active+clean; 161 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-05-08T23:56:18.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 562946 6.77 MB/sec execute 374 sec latency 939.444 ms 2024-05-08T23:56:19.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 562946 6.75 MB/sec execute 375 sec latency 1939.551 ms 2024-05-08T23:56:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:19 smithi012 ceph-mon[42503]: pgmap v314: 97 pgs: 97 active+clean; 161 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-05-08T23:56:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:19 smithi204 ceph-mon[47373]: pgmap v314: 97 pgs: 97 active+clean; 161 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-05-08T23:56:20.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:19 smithi160 ceph-mon[46152]: pgmap v314: 97 pgs: 97 active+clean; 161 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-05-08T23:56:20.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 562946 6.73 MB/sec execute 376 sec latency 2939.625 ms 2024-05-08T23:56:21.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 565040 6.75 MB/sec execute 377 sec latency 3083.537 ms 2024-05-08T23:56:21.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:21 smithi012 ceph-mon[42503]: pgmap v315: 97 pgs: 97 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-05-08T23:56:22.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:21 smithi204 ceph-mon[47373]: pgmap v315: 97 pgs: 97 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-05-08T23:56:22.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:21 smithi160 ceph-mon[46152]: pgmap v315: 97 pgs: 97 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-05-08T23:56:22.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 566324 6.75 MB/sec execute 378 sec latency 34.447 ms 2024-05-08T23:56:23.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 566486 6.73 MB/sec execute 379 sec latency 911.661 ms 2024-05-08T23:56:23.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:23 smithi012 ceph-mon[42503]: pgmap v316: 97 pgs: 97 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-05-08T23:56:23.962 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:56:24.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:23 smithi204 ceph-mon[47373]: pgmap v316: 97 pgs: 97 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-05-08T23:56:24.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:23 smithi160 ceph-mon[46152]: pgmap v316: 97 pgs: 97 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-05-08T23:56:24.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.119+0000 7f91adff9700 1 -- 172.21.15.12:0/2704525879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a8100a10 msgr2=0x7f91a8104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.119+0000 7f91adff9700 1 --2- 172.21.15.12:0/2704525879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a8100a10 0x7f91a8104a30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f919c009990 tx=0x7f919c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.120+0000 7f91adff9700 1 -- 172.21.15.12:0/2704525879 shutdown_connections 2024-05-08T23:56:24.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.120+0000 7f91adff9700 1 --2- 172.21.15.12:0/2704525879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a8105160 0x7f91a810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.120+0000 7f91adff9700 1 --2- 172.21.15.12:0/2704525879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a8100060 0x7f91a8100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.120+0000 7f91adff9700 1 --2- 172.21.15.12:0/2704525879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a8100a10 0x7f91a8104a30 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.120+0000 7f91adff9700 1 -- 172.21.15.12:0/2704525879 >> 172.21.15.12:0/2704525879 conn(0x7f91a8076f40 msgr2=0x7f91a8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.120+0000 7f91adff9700 1 -- 172.21.15.12:0/2704525879 shutdown_connections 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.120+0000 7f91adff9700 1 -- 172.21.15.12:0/2704525879 wait complete. 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.120+0000 7f91adff9700 1 Processor -- start 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91adff9700 1 -- start start 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91adff9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a8100060 0x7f91a8101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91adff9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a8105160 0x7f91a8102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91adff9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a8102550 0x7f91a81a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91adff9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81a5ca0 con 0x7f91a8105160 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91adff9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81a5e10 con 0x7f91a8102550 2024-05-08T23:56:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91adff9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81a5fd0 con 0x7f91a8100060 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a8105160 0x7f91a8102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a8105160 0x7f91a8102010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38512/0 (socket says 172.21.15.12:38512) 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a6ffd700 1 -- 172.21.15.12:0/208124008 learned_addr learned my addr 172.21.15.12:0/208124008 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a7fff700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a8102550 0x7f91a81a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a77fe700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a8100060 0x7f91a8101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a6ffd700 1 -- 172.21.15.12:0/208124008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a8100060 msgr2=0x7f91a8101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a8100060 0x7f91a8101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a6ffd700 1 -- 172.21.15.12:0/208124008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a8102550 msgr2=0x7f91a81a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a8102550 0x7f91a81a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a6ffd700 1 -- 172.21.15.12:0/208124008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c0095f0 con 0x7f91a8105160 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.121+0000 7f91a7fff700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a8102550 0x7f91a81a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.122+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a8105160 0x7f91a8102010 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f919c000c00 tx=0x7f919c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.122+0000 7f91a4ff9700 1 -- 172.21.15.12:0/208124008 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c01d070 con 0x7f91a8105160 2024-05-08T23:56:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.122+0000 7f91a4ff9700 1 -- 172.21.15.12:0/208124008 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f919c022470 con 0x7f91a8105160 2024-05-08T23:56:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.122+0000 7f91adff9700 1 -- 172.21.15.12:0/208124008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a81a6250 con 0x7f91a8105160 2024-05-08T23:56:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.122+0000 7f91a4ff9700 1 -- 172.21.15.12:0/208124008 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c017650 con 0x7f91a8105160 2024-05-08T23:56:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.122+0000 7f91adff9700 1 -- 172.21.15.12:0/208124008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a81a66d0 con 0x7f91a8105160 2024-05-08T23:56:24.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.124+0000 7f91a4ff9700 1 -- 172.21.15.12:0/208124008 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f919c00fad0 con 0x7f91a8105160 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.125+0000 7f91a4ff9700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f91900a1310 0x7f91900a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.125+0000 7f91a4ff9700 1 -- 172.21.15.12:0/208124008 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f919c0cdfa0 con 0x7f91a8105160 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.125+0000 7f91a77fe700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f91900a1310 0x7f91900a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.125+0000 7f91a77fe700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f91900a1310 0x7f91900a37d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9198009c60 tx=0x7f9198009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91ba1b3b80 1 Processor -- start 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91ba1b3b80 1 -- start start 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c5e00 0x557a128c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c6720 0x557a128c0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12817150 con 0x557a128c0890 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12814b10 con 0x557a128c5e00 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12810400 con 0x557a128c6720 2024-05-08T23:56:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91a7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91a7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38514/0 (socket says 172.21.15.12:38514) 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91a7fff700 1 -- 172.21.15.12:0/311784657 learned_addr learned my addr 172.21.15.12:0/311784657 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91a77fe700 1 --2- 172.21.15.12:0/311784657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c5e00 0x557a128c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/311784657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c6720 0x557a128c0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91a7fff700 1 -- 172.21.15.12:0/311784657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c6720 msgr2=0x557a128c0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91a7fff700 1 --2- 172.21.15.12:0/311784657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c6720 0x557a128c0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.126+0000 7f91a7fff700 1 -- 172.21.15.12:0/311784657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c5e00 msgr2=0x557a128c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.127+0000 7f91a7fff700 1 --2- 172.21.15.12:0/311784657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c5e00 0x557a128c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.127+0000 7f91a7fff700 1 -- 172.21.15.12:0/311784657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a126913b0 con 0x557a128c0890 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.127+0000 7f91a77fe700 1 --2- 172.21.15.12:0/311784657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c5e00 0x557a128c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.127+0000 7f91a7fff700 1 --2- 172.21.15.12:0/311784657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0cd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f91a0007740 tx=0x7f91a0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248e2f3b534580d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.127+0000 7f918dffb700 1 -- 172.21.15.12:0/311784657 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0015850 con 0x557a128c0890 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.127+0000 7f918dffb700 1 -- 172.21.15.12:0/311784657 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91a000f040 con 0x557a128c0890 2024-05-08T23:56:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.127+0000 7f918dffb700 1 -- 172.21.15.12:0/311784657 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0014d90 con 0x557a128c0890 2024-05-08T23:56:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/311784657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 msgr2=0x557a128c0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/311784657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0cd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f91a0007740 tx=0x7f91a0009f90 comp rx=0 tx=0).stop 2024-05-08T23:56:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/311784657 shutdown_connections 2024-05-08T23:56:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/311784657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0cd0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/311784657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c5e00 0x557a128c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/311784657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c6720 0x557a128c0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/311784657 >> 172.21.15.12:0/311784657 conn(0x557a1281e210 msgr2=0x557a128badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/311784657 shutdown_connections 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/311784657 wait complete. 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 -- start start 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.128+0000 7f91ba1b3b80 1 Processor -- start 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c0890 0x557a128c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c6720 0x557a128ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a127ac560 con 0x557a128c5e00 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a127ac200 con 0x557a128c0890 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a126936a0 con 0x557a128c6720 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91ba1b3b80 10 client.0 _ll_register_callbacks cb 0x557a127b5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38530/0 (socket says 172.21.15.12:38530) 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a6ffd700 1 -- 172.21.15.12:0/3070743372 learned_addr learned my addr 172.21.15.12:0/3070743372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a77fe700 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c0890 0x557a128c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a7fff700 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c6720 0x557a128ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a6ffd700 1 -- 172.21.15.12:0/3070743372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c6720 msgr2=0x557a128ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c6720 0x557a128ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a6ffd700 1 -- 172.21.15.12:0/3070743372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c0890 msgr2=0x557a128c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c0890 0x557a128c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a6ffd700 1 -- 172.21.15.12:0/3070743372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9198017070 con 0x557a128c5e00 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128ca060 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f919c097cb0 tx=0x7f919c09b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80086dd152a5cbe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.129+0000 7f9186ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f9186ffd700 1 -- 172.21.15.12:0/3070743372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c09aba0 con 0x557a128c5e00 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f9186ffd700 1 -- 172.21.15.12:0/3070743372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f919c09b940 con 0x557a128c5e00 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/3070743372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a12769270 con 0x557a128c5e00 2024-05-08T23:56:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f9186ffd700 1 -- 172.21.15.12:0/3070743372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f919c09b240 con 0x557a128c5e00 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f91ba1b3b80 10 client.18189 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/3070743372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f91a0009e00 con 0x557a128c5e00 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f9186ffd700 1 -- 172.21.15.12:0/3070743372 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a12769270 con 0x557a128c5e00 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f9186ffd700 1 -- 172.21.15.12:0/3070743372 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f919c09e070 con 0x557a128c5e00 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f91ba1b3b80 10 client.18189 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f91ba1b3b80 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f91ba1b3b80 20 client.18189 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a128d9dc0 0x557a128dc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f91ba1b3b80 4 client.18189 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.130+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/3070743372 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a1288f810 con 0x557a128d9dc0 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.131+0000 7f91a77fe700 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a128d9dc0 0x557a128dc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.131+0000 7f91a77fe700 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a128d9dc0 0x557a128dc1e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b38895904a06f29e server_cookie=b2d396c3f752a547 in_seq=0 out_seq=0 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.131+0000 7f9186ffd700 10 client.18189 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f9186ffd700 1 -- 172.21.15.12:0/3070743372 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a1288f810 con 0x557a128d9dc0 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f9186ffd700 10 client.18189 handle_command_reply: tid=0 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 client.18189 shutdown 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/3070743372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 msgr2=0x557a128ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128ca060 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f919c097cb0 tx=0x7f919c09b010 comp rx=0 tx=0).stop 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/3070743372 shutdown_connections 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c6720 0x557a128ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a128d9dc0 0x557a128dc1e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c0890 0x557a128c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/3070743372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128ca060 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/3070743372 >> 172.21.15.12:0/3070743372 conn(0x557a1281e210 msgr2=0x557a128ba5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f9186ffd700 0 client.18189 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/3070743372 shutdown_connections 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/3070743372 wait complete. 2024-05-08T23:56:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.132+0000 7f91ba1b3b80 20 client.18189 trim_cache size 0 max 16384 2024-05-08T23:56:24.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.150+0000 7f91ba1b3b80 1 Processor -- start 2024-05-08T23:56:24.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.150+0000 7f91ba1b3b80 1 -- start start 2024-05-08T23:56:24.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.150+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.150+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c5e00 0x557a128bad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.150+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c6720 0x557a128ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.150+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a128b5800 con 0x557a128c0890 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.150+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a128cb0f0 con 0x557a128c6720 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.150+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a128b3040 con 0x557a128c5e00 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38546/0 (socket says 172.21.15.12:38546) 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a77fe700 1 -- 172.21.15.12:0/1527949236 learned_addr learned my addr 172.21.15.12:0/1527949236 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/1527949236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c5e00 0x557a128bad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a77fe700 1 -- 172.21.15.12:0/1527949236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c5e00 msgr2=0x557a128bad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a77fe700 1 --2- 172.21.15.12:0/1527949236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c5e00 0x557a128bad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a77fe700 1 -- 172.21.15.12:0/1527949236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c6720 msgr2=0x557a128ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a77fe700 1 --2- 172.21.15.12:0/1527949236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c6720 0x557a128ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a77fe700 1 -- 172.21.15.12:0/1527949236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a1263db60 con 0x557a128c0890 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/1527949236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c5e00 0x557a128bad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f91a77fe700 1 --2- 172.21.15.12:0/1527949236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0350 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x557a128dcae0 tx=0x7f9198014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dfceff779f51fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f918e7fc700 1 -- 172.21.15.12:0/1527949236 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9198022030 con 0x557a128c0890 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f918e7fc700 1 -- 172.21.15.12:0/1527949236 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9198023040 con 0x557a128c0890 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.151+0000 7f918e7fc700 1 -- 172.21.15.12:0/1527949236 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919802b0e0 con 0x557a128c0890 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.152+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1527949236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 msgr2=0x557a128c0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.152+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/1527949236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0350 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x557a128dcae0 tx=0x7f9198014000 comp rx=0 tx=0).stop 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.152+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1527949236 shutdown_connections 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.152+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/1527949236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c6720 0x557a128ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.152+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/1527949236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c0890 0x557a128c0350 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.152+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/1527949236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c5e00 0x557a128bad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.152+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1527949236 >> 172.21.15.12:0/1527949236 conn(0x557a1281e210 msgr2=0x557a128c83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.152+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1527949236 shutdown_connections 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.152+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1527949236 wait complete. 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.152+0000 7f91ba1b3b80 1 -- start start 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91ba1b3b80 1 Processor -- start 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c0890 0x557a128cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91ba1b3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c6720 0x557a1292f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a128d0020 con 0x557a128c5e00 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1292f850 con 0x557a128c6720 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91ba1b3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1292fa10 con 0x557a128c0890 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91ba1b3b80 10 client.0 _ll_register_callbacks cb 0x557a127b5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38552/0 (socket says 172.21.15.12:38552) 2024-05-08T23:56:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a6ffd700 1 -- 172.21.15.12:0/1070775939 learned_addr learned my addr 172.21.15.12:0/1070775939 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a7fff700 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c6720 0x557a1292f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a77fe700 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c0890 0x557a128cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a6ffd700 1 -- 172.21.15.12:0/1070775939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c0890 msgr2=0x557a128cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c0890 0x557a128cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a6ffd700 1 -- 172.21.15.12:0/1070775939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c6720 msgr2=0x557a1292f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c6720 0x557a1292f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a6ffd700 1 -- 172.21.15.12:0/1070775939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9198009e60 con 0x557a128c5e00 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.153+0000 7f91a7fff700 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c6720 0x557a1292f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f91a6ffd700 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128cf880 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f919c0a3100 tx=0x7f919c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d5c56ea7d4f451 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f9186ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f9186ffd700 1 -- 172.21.15.12:0/1070775939 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c0b9450 con 0x557a128c5e00 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f9186ffd700 1 -- 172.21.15.12:0/1070775939 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f919c09a730 con 0x557a128c5e00 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1070775939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91a000f040 con 0x557a128c5e00 2024-05-08T23:56:24.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f9186ffd700 1 -- 172.21.15.12:0/1070775939 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f919c0b95b0 con 0x557a128c5e00 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f91ba1b3b80 10 client.18195 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1070775939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x557a127d6560 con 0x557a128c5e00 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f9186ffd700 1 -- 172.21.15.12:0/1070775939 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91a000f040 con 0x557a128c5e00 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f9186ffd700 1 -- 172.21.15.12:0/1070775939 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f919c09e040 con 0x557a128c5e00 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f91ba1b3b80 10 client.18195 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f91ba1b3b80 10 client.18195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f91ba1b3b80 20 client.18195 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a12930c60 0x557a12933050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f91ba1b3b80 4 client.18195 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.154+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1070775939 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557a12808ec0 con 0x557a12930c60 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91a77fe700 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a12930c60 0x557a12933050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91a77fe700 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a12930c60 0x557a12933050 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9650ae5ebcc378e server_cookie=cec07c95c367e847 in_seq=0 out_seq=0 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f9186ffd700 10 client.18195 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:24.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f9186ffd700 1 -- 172.21.15.12:0/1070775939 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557a12808ec0 con 0x557a12930c60 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f9186ffd700 10 client.18195 handle_command_reply: tid=0 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91ba1b3b80 1 client.18195 shutdown 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1070775939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 msgr2=0x557a128cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128cf880 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f919c0a3100 tx=0x7f919c005fb0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1070775939 shutdown_connections 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a12930c60 0x557a12933050 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a128c6720 0x557a1292f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a128c0890 0x557a128cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f9186ffd700 0 client.18195 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91ba1b3b80 1 --2- 172.21.15.12:0/1070775939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a128c5e00 0x557a128cf880 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1070775939 >> 172.21.15.12:0/1070775939 conn(0x557a1281e210 msgr2=0x557a128c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.155+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1070775939 shutdown_connections 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91ba1b3b80 1 -- 172.21.15.12:0/1070775939 wait complete. 2024-05-08T23:56:24.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91ba1b3b80 20 client.18195 trim_cache size 0 max 16384 2024-05-08T23:56:24.157 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:56:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:56:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:56:24.157 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 -- 172.21.15.12:0/208124008 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f91900a1310 msgr2=0x7f91900a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f91900a1310 0x7f91900a37d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9198009c60 tx=0x7f9198009260 comp rx=0 tx=0).stop 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 -- 172.21.15.12:0/208124008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a8105160 msgr2=0x7f91a8102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a8105160 0x7f91a8102010 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f919c000c00 tx=0x7f919c005740 comp rx=0 tx=0).stop 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 -- 172.21.15.12:0/208124008 shutdown_connections 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f91900a1310 0x7f91900a37d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91a8102550 0x7f91a81a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a8100060 0x7f91a8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 --2- 172.21.15.12:0/208124008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91a8105160 0x7f91a8102010 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 -- 172.21.15.12:0/208124008 >> 172.21.15.12:0/208124008 conn(0x7f91a8076f40 msgr2=0x7f91a80683b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 -- 172.21.15.12:0/208124008 shutdown_connections 2024-05-08T23:56:24.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.156+0000 7f91adff9700 1 -- 172.21.15.12:0/208124008 wait complete. 2024-05-08T23:56:24.164 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:56:24.165 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:56:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.307+0000 7ff9c5bef700 1 -- 172.21.15.12:0/1573225436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9c01028d0 msgr2=0x7ff9c010adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.307+0000 7ff9c5bef700 1 --2- 172.21.15.12:0/1573225436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9c01028d0 0x7ff9c010adb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009990 tx=0x7ff9a8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.307+0000 7ff9c5bef700 1 -- 172.21.15.12:0/1573225436 shutdown_connections 2024-05-08T23:56:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.307+0000 7ff9c5bef700 1 --2- 172.21.15.12:0/1573225436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9c010b420 0x7ff9c010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.307+0000 7ff9c5bef700 1 --2- 172.21.15.12:0/1573225436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c0101fb0 0x7ff9c0102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.307+0000 7ff9c5bef700 1 --2- 172.21.15.12:0/1573225436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9c01028d0 0x7ff9c010adb0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.307+0000 7ff9c5bef700 1 -- 172.21.15.12:0/1573225436 >> 172.21.15.12:0/1573225436 conn(0x7ff9c00fb7f0 msgr2=0x7ff9c00fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.307+0000 7ff9c5bef700 1 -- 172.21.15.12:0/1573225436 shutdown_connections 2024-05-08T23:56:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.307+0000 7ff9c5bef700 1 -- 172.21.15.12:0/1573225436 wait complete. 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9c5bef700 1 Processor -- start 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9c5bef700 1 -- start start 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9c5bef700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9c0101fb0 0x7ff9c01972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9c5bef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9c01028d0 0x7ff9c0197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9c5bef700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c010b420 0x7ff9c019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9c5bef700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c019c1f0 con 0x7ff9c01028d0 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9c5bef700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c019c360 con 0x7ff9c010b420 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9c5bef700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c019c520 con 0x7ff9c0101fb0 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9beffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9c01028d0 0x7ff9c0197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9beffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9c01028d0 0x7ff9c0197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38566/0 (socket says 172.21.15.12:38566) 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9beffd700 1 -- 172.21.15.12:0/2593404167 learned_addr learned my addr 172.21.15.12:0/2593404167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.308+0000 7ff9bf7fe700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9c0101fb0 0x7ff9c01972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9bffff700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c010b420 0x7ff9c019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9beffd700 1 -- 172.21.15.12:0/2593404167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9c0101fb0 msgr2=0x7ff9c01972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9beffd700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9c0101fb0 0x7ff9c01972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9beffd700 1 -- 172.21.15.12:0/2593404167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c010b420 msgr2=0x7ff9c019bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9beffd700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c010b420 0x7ff9c019bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9beffd700 1 -- 172.21.15.12:0/2593404167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80095f0 con 0x7ff9c01028d0 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9beffd700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9c01028d0 0x7ff9c0197810 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8000c00 tx=0x7ff9a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9bcff9700 1 -- 172.21.15.12:0/2593404167 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a801d070 con 0x7ff9c01028d0 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9bcff9700 1 -- 172.21.15.12:0/2593404167 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9a8022470 con 0x7ff9c01028d0 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9bcff9700 1 -- 172.21.15.12:0/2593404167 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8017650 con 0x7ff9c01028d0 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9c5bef700 1 -- 172.21.15.12:0/2593404167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c019c7a0 con 0x7ff9c01028d0 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.309+0000 7ff9c5bef700 1 -- 172.21.15.12:0/2593404167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c01a4040 con 0x7ff9c01028d0 2024-05-08T23:56:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.310+0000 7ff9c5bef700 1 -- 172.21.15.12:0/2593404167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c0106500 con 0x7ff9c01028d0 2024-05-08T23:56:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.311+0000 7ff9bcff9700 1 -- 172.21.15.12:0/2593404167 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff9a80225e0 con 0x7ff9c01028d0 2024-05-08T23:56:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.312+0000 7ff9bcff9700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9ac0a1310 0x7ff9ac0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.312+0000 7ff9bf7fe700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9ac0a1310 0x7ff9ac0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.312+0000 7ff9bcff9700 1 -- 172.21.15.12:0/2593404167 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff9a80ce490 con 0x7ff9c01028d0 2024-05-08T23:56:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.313+0000 7ff9bf7fe700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9ac0a1310 0x7ff9ac0a37d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005950 tx=0x7ff9b000b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.316+0000 7ff9bcff9700 1 -- 172.21.15.12:0/2593404167 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff9a8097a30 con 0x7ff9c01028d0 2024-05-08T23:56:24.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.587+0000 7ff9c5bef700 1 -- 172.21.15.12:0/2593404167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9c01985d0 con 0x7ff9c01028d0 2024-05-08T23:56:24.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.588+0000 7ff9bcff9700 1 -- 172.21.15.12:0/2593404167 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+10880 (secure 0 0 0) 0x7ff9a809b050 con 0x7ff9c01028d0 2024-05-08T23:56:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:56:24.583 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:56:12.808348+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27940":{"gid":27940,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2536985900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2536985900},{"type":"v1","addr":"172.21.15.204:6837","nonce":2536985900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:56:24.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.590+0000 7ff9c5bef700 1 -- 172.21.15.12:0/2593404167 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9ac0a1310 msgr2=0x7ff9ac0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.590+0000 7ff9c5bef700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9ac0a1310 0x7ff9ac0a37d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005950 tx=0x7ff9b000b410 comp rx=0 tx=0).stop 2024-05-08T23:56:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.590+0000 7ff9c5bef700 1 -- 172.21.15.12:0/2593404167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9c01028d0 msgr2=0x7ff9c0197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.590+0000 7ff9c5bef700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9c01028d0 0x7ff9c0197810 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8000c00 tx=0x7ff9a8004730 comp rx=0 tx=0).stop 2024-05-08T23:56:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.591+0000 7ff9c5bef700 1 -- 172.21.15.12:0/2593404167 shutdown_connections 2024-05-08T23:56:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.591+0000 7ff9c5bef700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9ac0a1310 0x7ff9ac0a37d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.591+0000 7ff9c5bef700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c010b420 0x7ff9c019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.591+0000 7ff9c5bef700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9c0101fb0 0x7ff9c01972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.591+0000 7ff9c5bef700 1 --2- 172.21.15.12:0/2593404167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9c01028d0 0x7ff9c0197810 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.591+0000 7ff9c5bef700 1 -- 172.21.15.12:0/2593404167 >> 172.21.15.12:0/2593404167 conn(0x7ff9c00fb7f0 msgr2=0x7ff9c00fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.591+0000 7ff9c5bef700 1 -- 172.21.15.12:0/2593404167 shutdown_connections 2024-05-08T23:56:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.591+0000 7ff9c5bef700 1 -- 172.21.15.12:0/2593404167 wait complete. 2024-05-08T23:56:24.587 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2024-05-08T23:56:24.599 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:56:24.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.744+0000 7fbca2630700 1 -- 172.21.15.12:0/3428114477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c1011e0 msgr2=0x7fbc9c1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.744+0000 7fbca2630700 1 --2- 172.21.15.12:0/3428114477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c1011e0 0x7fbc9c1015c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009990 tx=0x7fbc8c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.744+0000 7fbca2630700 1 -- 172.21.15.12:0/3428114477 shutdown_connections 2024-05-08T23:56:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.744+0000 7fbca2630700 1 --2- 172.21.15.12:0/3428114477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9c106300 0x7fbc9c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.744+0000 7fbca2630700 1 --2- 172.21.15.12:0/3428114477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c1011e0 0x7fbc9c1015c0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.744+0000 7fbca2630700 1 --2- 172.21.15.12:0/3428114477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc9c101b90 0x7fbc9c105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.744+0000 7fbca2630700 1 -- 172.21.15.12:0/3428114477 >> 172.21.15.12:0/3428114477 conn(0x7fbc9c0fca80 msgr2=0x7fbc9c0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.744+0000 7fbca2630700 1 -- 172.21.15.12:0/3428114477 shutdown_connections 2024-05-08T23:56:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.744+0000 7fbca2630700 1 -- 172.21.15.12:0/3428114477 wait complete. 2024-05-08T23:56:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.745+0000 7fbca2630700 1 Processor -- start 2024-05-08T23:56:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.745+0000 7fbca2630700 1 -- start start 2024-05-08T23:56:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.745+0000 7fbca2630700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9c1011e0 0x7fbc9c1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.745+0000 7fbca2630700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c106300 0x7fbc9c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.745+0000 7fbca2630700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc9c19bd90 0x7fbc9c19c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.745+0000 7fbca2630700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c19c810 con 0x7fbc9c106300 2024-05-08T23:56:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.745+0000 7fbca2630700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c19c980 con 0x7fbc9c19bd90 2024-05-08T23:56:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.745+0000 7fbca2630700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c19cb40 con 0x7fbc9c1011e0 2024-05-08T23:56:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c106300 0x7fbc9c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c106300 0x7fbc9c1979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38578/0 (socket says 172.21.15.12:38578) 2024-05-08T23:56:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/1343907238 learned_addr learned my addr 172.21.15.12:0/1343907238 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc9c19bd90 0x7fbc9c19c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9c1011e0 0x7fbc9c1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/1343907238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9c1011e0 msgr2=0x7fbc9c1974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9c1011e0 0x7fbc9c1974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/1343907238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc9c19bd90 msgr2=0x7fbc9c19c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc9c19bd90 0x7fbc9c19c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/1343907238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c0095f0 con 0x7fbc9c106300 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc9c19bd90 0x7fbc9c19c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9c1011e0 0x7fbc9c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c106300 0x7fbc9c1979f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000d950 tx=0x7fbc9000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc997fa700 1 -- 172.21.15.12:0/1343907238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9000bb60 con 0x7fbc9c106300 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc997fa700 1 -- 172.21.15.12:0/1343907238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc90007620 con 0x7fbc9c106300 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbca2630700 1 -- 172.21.15.12:0/1343907238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9c1a40a0 con 0x7fbc9c106300 2024-05-08T23:56:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbc997fa700 1 -- 172.21.15.12:0/1343907238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90005560 con 0x7fbc9c106300 2024-05-08T23:56:24.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.746+0000 7fbca2630700 1 -- 172.21.15.12:0/1343907238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c1a4580 con 0x7fbc9c106300 2024-05-08T23:56:24.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.749+0000 7fbc997fa700 1 -- 172.21.15.12:0/1343907238 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbc90004750 con 0x7fbc9c106300 2024-05-08T23:56:24.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.749+0000 7fbc997fa700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc840a1290 0x7fbc840a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.749+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc840a1290 0x7fbc840a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.749+0000 7fbc997fa700 1 -- 172.21.15.12:0/1343907238 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fbc900cc8f0 con 0x7fbc9c106300 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.750+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc840a1290 0x7fbc840a3750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00b2e0 tx=0x7fbc8c005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.750+0000 7fbcae7eab80 1 Processor -- start 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.750+0000 7fbcae7eab80 1 -- start start 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.750+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb20210 0x56085cb205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.750+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 0x56085cb1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.750+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb1aca0 0x56085cb1b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.750+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085ca6d150 con 0x56085cb1aca0 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.750+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085ca6ab10 con 0x56085cb20210 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.750+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085ca66400 con 0x56085cb20b30 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbca0bcd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb1aca0 0x56085cb1b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbca0bcd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb1aca0 0x56085cb1b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38584/0 (socket says 172.21.15.12:38584) 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbca0bcd700 1 -- 172.21.15.12:0/2757547619 learned_addr learned my addr 172.21.15.12:0/2757547619 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/2757547619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 0x56085cb1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/2757547619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb20210 0x56085cb205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbca0bcd700 1 -- 172.21.15.12:0/2757547619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 msgr2=0x56085cb1a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/2757547619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 0x56085cb1a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbca0bcd700 1 -- 172.21.15.12:0/2757547619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb20210 msgr2=0x56085cb205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/2757547619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb20210 0x56085cb205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbca0bcd700 1 -- 172.21.15.12:0/2757547619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56085c8e73b0 con 0x56085cb1aca0 2024-05-08T23:56:24.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/2757547619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 0x56085cb1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:24.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/2757547619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb1aca0 0x56085cb1b0e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009440 tx=0x7fbc94009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e0768b038d2863 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbc827fc700 1 -- 172.21.15.12:0/2757547619 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9400e070 con 0x56085cb1aca0 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbc827fc700 1 -- 172.21.15.12:0/2757547619 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9400f040 con 0x56085cb1aca0 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.751+0000 7fbc827fc700 1 -- 172.21.15.12:0/2757547619 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94014a20 con 0x56085cb1aca0 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.752+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2757547619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb1aca0 msgr2=0x56085cb1b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.752+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/2757547619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb1aca0 0x56085cb1b0e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009440 tx=0x7fbc94009920 comp rx=0 tx=0).stop 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.752+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2757547619 shutdown_connections 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.752+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/2757547619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb1aca0 0x56085cb1b0e0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.752+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/2757547619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb20210 0x56085cb205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.752+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/2757547619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 0x56085cb1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.752+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2757547619 >> 172.21.15.12:0/2757547619 conn(0x56085ca74210 msgr2=0x56085cb10df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.752+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2757547619 shutdown_connections 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.752+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2757547619 wait complete. 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.752+0000 7fbcae7eab80 1 -- start start 2024-05-08T23:56:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbcae7eab80 1 Processor -- start 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 0x56085cb26b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20210 0x56085cb23e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20b30 0x56085cb24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085ca02560 con 0x56085cb20b30 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085ca02200 con 0x56085cb1aca0 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085c8e96a0 con 0x56085cb20210 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbcae7eab80 10 client.0 _ll_register_callbacks cb 0x56085ca0bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbca0bcd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20b30 0x56085cb24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbca0bcd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20b30 0x56085cb24580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38600/0 (socket says 172.21.15.12:38600) 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbca0bcd700 1 -- 172.21.15.12:0/1313038270 learned_addr learned my addr 172.21.15.12:0/1313038270 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 0x56085cb26b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.753+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20210 0x56085cb23e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbca0bcd700 1 -- 172.21.15.12:0/1313038270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20210 msgr2=0x56085cb23e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20210 0x56085cb23e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbca0bcd700 1 -- 172.21.15.12:0/1313038270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 msgr2=0x56085cb26b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 0x56085cb26b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbca0bcd700 1 -- 172.21.15.12:0/1313038270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc940090f0 con 0x56085cb20b30 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 0x56085cb26b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20b30 0x56085cb24580 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbc94016ff0 tx=0x7fbc94014980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd56a79012e793e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbc7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/1313038270 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9400e040 con 0x56085cb20b30 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/1313038270 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fbc94015640 con 0x56085cb20b30 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1313038270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56085c9bf270 con 0x56085cb20b30 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/1313038270 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc940095e0 con 0x56085cb20b30 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbcae7eab80 10 client.18213 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1313038270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56085c893b60 con 0x56085cb20b30 2024-05-08T23:56:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.754+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/1313038270 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56085c9bf270 con 0x56085cb20b30 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.755+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/1313038270 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fbc9401c050 con 0x56085cb20b30 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.755+0000 7fbcae7eab80 10 client.18213 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.755+0000 7fbcae7eab80 10 client.18213 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.755+0000 7fbcae7eab80 20 client.18213 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.755+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56085cb343d0 0x56085cb367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.755+0000 7fbcae7eab80 4 client.18213 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.755+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1313038270 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56085cae5810 con 0x56085cb343d0 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.755+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56085cb343d0 0x56085cb367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.755+0000 7fbc7b7fe700 10 client.18213 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.755+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56085cb343d0 0x56085cb367f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=742df2cb988810bd server_cookie=b9778a65c86612c in_seq=0 out_seq=0 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/1313038270 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56085cae5810 con 0x56085cb343d0 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbc7b7fe700 10 client.18213 handle_command_reply: tid=0 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 client.18213 shutdown 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1313038270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20b30 msgr2=0x56085cb24580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20b30 0x56085cb24580 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbc94016ff0 tx=0x7fbc94014980 comp rx=0 tx=0).stop 2024-05-08T23:56:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1313038270 shutdown_connections 2024-05-08T23:56:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20b30 0x56085cb24580 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 0x56085cb26b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56085cb343d0 0x56085cb367f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/1313038270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20210 0x56085cb23e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1313038270 >> 172.21.15.12:0/1313038270 conn(0x56085ca74210 msgr2=0x56085cb105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbc7b7fe700 0 client.18213 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:56:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1313038270 shutdown_connections 2024-05-08T23:56:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.756+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1313038270 wait complete. 2024-05-08T23:56:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.757+0000 7fbcae7eab80 20 client.18213 trim_cache size 0 max 16384 2024-05-08T23:56:24.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbcae7eab80 1 Processor -- start 2024-05-08T23:56:24.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbcae7eab80 1 -- start start 2024-05-08T23:56:24.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 0x56085cb205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb0d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 0x56085cb1b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085cb090d0 con 0x56085cb20210 2024-05-08T23:56:24.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085cb0b830 con 0x56085cb1aca0 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085cb11910 con 0x56085cb20b30 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbc9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb0d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbc9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb0d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38608/0 (socket says 172.21.15.12:38608) 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/1029255676 learned_addr learned my addr 172.21.15.12:0/1029255676 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/1029255676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 0x56085cb205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/1029255676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 0x56085cb1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/1029255676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 msgr2=0x56085cb1b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/1029255676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 0x56085cb1b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/1029255676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 msgr2=0x56085cb205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/1029255676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 0x56085cb205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/1029255676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56085ca2c560 con 0x56085cb20210 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.771+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/1029255676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 0x56085cb1b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:56:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/1029255676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb0d210 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fbc900999d0 tx=0x7fbc9000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68cda5c58773cae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbc82ffd700 1 -- 172.21.15.12:0/1029255676 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90099a40 con 0x56085cb20210 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbc82ffd700 1 -- 172.21.15.12:0/1029255676 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9009d870 con 0x56085cb20210 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbc82ffd700 1 -- 172.21.15.12:0/1029255676 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90004a00 con 0x56085cb20210 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1029255676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 msgr2=0x56085cb0d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/1029255676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb0d210 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fbc900999d0 tx=0x7fbc9000bd70 comp rx=0 tx=0).stop 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1029255676 shutdown_connections 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/1029255676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb20b30 0x56085cb1b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/1029255676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb1aca0 0x56085cb205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/1029255676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb0d210 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1029255676 >> 172.21.15.12:0/1029255676 conn(0x56085ca74210 msgr2=0x56085cb236d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1029255676 shutdown_connections 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.772+0000 7fbcae7eab80 1 -- 172.21.15.12:0/1029255676 wait complete. 2024-05-08T23:56:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.773+0000 7fbcae7eab80 1 -- start start 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.773+0000 7fbcae7eab80 1 Processor -- start 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb1aca0 0x56085cb2b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb29580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbcae7eab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb20b30 0x56085cb29ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085cb2a0e0 con 0x56085cb20210 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085cb2a220 con 0x56085cb20b30 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbcae7eab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085cb2a3e0 con 0x56085cb1aca0 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbcae7eab80 10 client.0 _ll_register_callbacks cb 0x56085ca0bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb29580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb29580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38624/0 (socket says 172.21.15.12:38624) 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/2472644445 learned_addr learned my addr 172.21.15.12:0/2472644445 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbca0bcd700 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb20b30 0x56085cb29ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb1aca0 0x56085cb2b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/2472644445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb1aca0 msgr2=0x56085cb2b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb1aca0 0x56085cb2b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/2472644445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb20b30 msgr2=0x56085cb29ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb20b30 0x56085cb29ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9b7fe700 1 -- 172.21.15.12:0/2472644445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c1a4580 con 0x56085cb20210 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb1aca0 0x56085cb2b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc9b7fe700 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb29580 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fbc900ca680 tx=0x7fbc900b2490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ba0ea4f147fade server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/2472644445 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90097b80 con 0x56085cb20210 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/2472644445 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fbc90004c10 con 0x56085cb20210 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.774+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2472644445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc9400f040 con 0x56085cb20210 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/2472644445 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc90099a40 con 0x56085cb20210 2024-05-08T23:56:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbcae7eab80 10 client.18219 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2472644445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56085c8ebcd0 con 0x56085cb20210 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/2472644445 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9400f040 con 0x56085cb20210 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/2472644445 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fbc90099030 con 0x56085cb20210 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbcae7eab80 10 client.18219 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbcae7eab80 10 client.18219 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbcae7eab80 20 client.18219 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56085cb8aa20 0x56085cb8ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbcae7eab80 4 client.18219 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2472644445 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56085ca5eec0 con 0x56085cb8aa20 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.775+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56085cb8aa20 0x56085cb8ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbc9bfff700 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56085cb8aa20 0x56085cb8ce10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c32065467b33b285 server_cookie=98b641d16875b2 in_seq=0 out_seq=0 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbc7b7fe700 10 client.18219 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:56:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbc7b7fe700 1 -- 172.21.15.12:0/2472644445 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56085ca5eec0 con 0x56085cb8aa20 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbc7b7fe700 10 client.18219 handle_command_reply: tid=0 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 client.18219 shutdown 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2472644445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 msgr2=0x56085cb29580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb29580 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fbc900ca680 tx=0x7fbc900b2490 comp rx=0 tx=0).stop 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2472644445 shutdown_connections 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56085cb20b30 0x56085cb29ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56085cb8aa20 0x56085cb8ce10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56085cb1aca0 0x56085cb2b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 --2- 172.21.15.12:0/2472644445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56085cb20210 0x56085cb29580 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbc7b7fe700 0 client.18219 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2472644445 >> 172.21.15.12:0/2472644445 conn(0x56085ca74210 msgr2=0x56085cb1df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2472644445 shutdown_connections 2024-05-08T23:56:24.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 1 -- 172.21.15.12:0/2472644445 wait complete. 2024-05-08T23:56:24.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.776+0000 7fbcae7eab80 20 client.18219 trim_cache size 0 max 16384 2024-05-08T23:56:24.779 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:56:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 -- 172.21.15.12:0/1343907238 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc840a1290 msgr2=0x7fbc840a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc840a1290 0x7fbc840a3750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00b2e0 tx=0x7fbc8c005fb0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 -- 172.21.15.12:0/1343907238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c106300 msgr2=0x7fbc9c1979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c106300 0x7fbc9c1979f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000d950 tx=0x7fbc9000dd10 comp rx=0 tx=0).stop 2024-05-08T23:56:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 -- 172.21.15.12:0/1343907238 shutdown_connections 2024-05-08T23:56:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbc840a1290 0x7fbc840a3750 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc9c19bd90 0x7fbc9c19c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9c1011e0 0x7fbc9c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 --2- 172.21.15.12:0/1343907238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c106300 0x7fbc9c1979f0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 -- 172.21.15.12:0/1343907238 >> 172.21.15.12:0/1343907238 conn(0x7fbc9c0fca80 msgr2=0x7fbc9c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 -- 172.21.15.12:0/1343907238 shutdown_connections 2024-05-08T23:56:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.777+0000 7fbca2630700 1 -- 172.21.15.12:0/1343907238 wait complete. 2024-05-08T23:56:24.785 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:56:24.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 566486 6.71 MB/sec execute 380 sec latency 1911.732 ms 2024-05-08T23:56:24.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:24 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2593404167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:56:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.929+0000 7f7f77aee700 1 -- 172.21.15.12:0/1197644082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f701000f0 msgr2=0x7f7f70100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.929+0000 7f7f77aee700 1 --2- 172.21.15.12:0/1197644082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f701000f0 0x7f7f70100570 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7f600099c0 tx=0x7f7f60009cd0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.929+0000 7f7f77aee700 1 -- 172.21.15.12:0/1197644082 shutdown_connections 2024-05-08T23:56:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.929+0000 7f7f77aee700 1 --2- 172.21.15.12:0/1197644082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f70100ab0 0x7f7f7010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.929+0000 7f7f77aee700 1 --2- 172.21.15.12:0/1197644082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f70105f90 0x7f7f70106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.929+0000 7f7f77aee700 1 --2- 172.21.15.12:0/1197644082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f701000f0 0x7f7f70100570 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.929+0000 7f7f77aee700 1 -- 172.21.15.12:0/1197644082 >> 172.21.15.12:0/1197644082 conn(0x7f7f70076ba0 msgr2=0x7f7f70076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.929+0000 7f7f77aee700 1 -- 172.21.15.12:0/1197644082 shutdown_connections 2024-05-08T23:56:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.929+0000 7f7f77aee700 1 -- 172.21.15.12:0/1197644082 wait complete. 2024-05-08T23:56:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f77aee700 1 Processor -- start 2024-05-08T23:56:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f77aee700 1 -- start start 2024-05-08T23:56:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f77aee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f701000f0 0x7f7f70196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f77aee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f70100ab0 0x7f7f701967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f77aee700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f70105f90 0x7f7f7019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f77aee700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7019b140 con 0x7f7f70100ab0 2024-05-08T23:56:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f77aee700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7019b2b0 con 0x7f7f70105f90 2024-05-08T23:56:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f77aee700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7019b470 con 0x7f7f701000f0 2024-05-08T23:56:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f75089700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f70100ab0 0x7f7f701967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f75089700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f70100ab0 0x7f7f701967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38638/0 (socket says 172.21.15.12:38638) 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f75089700 1 -- 172.21.15.12:0/2025880566 learned_addr learned my addr 172.21.15.12:0/2025880566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f7608b700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f70105f90 0x7f7f7019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.930+0000 7f7f7588a700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f701000f0 0x7f7f70196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f75089700 1 -- 172.21.15.12:0/2025880566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f701000f0 msgr2=0x7f7f70196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f75089700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f701000f0 0x7f7f70196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f75089700 1 -- 172.21.15.12:0/2025880566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f70105f90 msgr2=0x7f7f7019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f75089700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f70105f90 0x7f7f7019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f75089700 1 -- 172.21.15.12:0/2025880566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f600095f0 con 0x7f7f70100ab0 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f7608b700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f70105f90 0x7f7f7019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f75089700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f70100ab0 0x7f7f701967b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7f600098e0 tx=0x7f7f60003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f66ffd700 1 -- 172.21.15.12:0/2025880566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6000f970 con 0x7f7f70100ab0 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f77aee700 1 -- 172.21.15.12:0/2025880566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7019b6f0 con 0x7f7f70100ab0 2024-05-08T23:56:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f66ffd700 1 -- 172.21.15.12:0/2025880566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f60020440 con 0x7f7f70100ab0 2024-05-08T23:56:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f66ffd700 1 -- 172.21.15.12:0/2025880566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60003a20 con 0x7f7f70100ab0 2024-05-08T23:56:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.931+0000 7f7f77aee700 1 -- 172.21.15.12:0/2025880566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f700693b0 con 0x7f7f70100ab0 2024-05-08T23:56:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.933+0000 7f7f66ffd700 1 -- 172.21.15.12:0/2025880566 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7f60020840 con 0x7f7f70100ab0 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.934+0000 7f7f66ffd700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f5c0a1290 0x7f7f5c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.934+0000 7f7f66ffd700 1 -- 172.21.15.12:0/2025880566 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7f600d0880 con 0x7f7f70100ab0 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.934+0000 7f7f7588a700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f5c0a1290 0x7f7f5c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.935+0000 7f7f7588a700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f5c0a1290 0x7f7f5c0a3750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c005fd0 tx=0x7f7f6c005e50 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.935+0000 7f7f83ca8b80 1 Processor -- start 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.935+0000 7f7f83ca8b80 1 -- start start 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.935+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.935+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab0e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.935+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab0eca0 0x561a6ab0f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.935+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6aa61150 con 0x561a6ab14b10 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.935+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6aa5eb10 con 0x561a6ab141f0 2024-05-08T23:56:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6aa5a400 con 0x561a6ab0eca0 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f75089700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab0e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f75089700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab0e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38640/0 (socket says 172.21.15.12:38640) 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f75089700 1 -- 172.21.15.12:0/1203466289 learned_addr learned my addr 172.21.15.12:0/1203466289 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f7608b700 1 --2- 172.21.15.12:0/1203466289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab0eca0 0x561a6ab0f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f7588a700 1 --2- 172.21.15.12:0/1203466289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f75089700 1 -- 172.21.15.12:0/1203466289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab0eca0 msgr2=0x561a6ab0f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f75089700 1 --2- 172.21.15.12:0/1203466289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab0eca0 0x561a6ab0f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f75089700 1 -- 172.21.15.12:0/1203466289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 msgr2=0x561a6ab145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f75089700 1 --2- 172.21.15.12:0/1203466289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab145d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f75089700 1 -- 172.21.15.12:0/1203466289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a6a8db3b0 con 0x561a6ab14b10 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f75089700 1 --2- 172.21.15.12:0/1203466289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab0e760 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7f60099f00 tx=0x7f7f6009d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81562dda7c9fec8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f5bfff700 1 -- 172.21.15.12:0/1203466289 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600a1830 con 0x561a6ab14b10 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f5bfff700 1 -- 172.21.15.12:0/1203466289 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f600a5df0 con 0x561a6ab14b10 2024-05-08T23:56:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.936+0000 7f7f5bfff700 1 -- 172.21.15.12:0/1203466289 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6009d110 con 0x561a6ab14b10 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.937+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/1203466289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 msgr2=0x561a6ab0e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.937+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/1203466289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab0e760 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7f60099f00 tx=0x7f7f6009d8e0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.937+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/1203466289 shutdown_connections 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.937+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/1203466289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab0eca0 0x561a6ab0f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.937+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/1203466289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.937+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/1203466289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab0e760 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.937+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/1203466289 >> 172.21.15.12:0/1203466289 conn(0x561a6aa68210 msgr2=0x561a6ab04df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.937+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/1203466289 shutdown_connections 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.937+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/1203466289 wait complete. 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f83ca8b80 1 -- start start 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f83ca8b80 1 Processor -- start 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab0eca0 0x561a6ab1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab17ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab185c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6a9f6560 con 0x561a6ab14b10 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6a9f6200 con 0x561a6ab141f0 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6a8dd6a0 con 0x561a6ab0eca0 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f83ca8b80 10 client.0 _ll_register_callbacks cb 0x561a6a9ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f7608b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f75089700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab17ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f7608b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab185c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38656/0 (socket says 172.21.15.12:38656) 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f7608b700 1 -- 172.21.15.12:0/815688592 learned_addr learned my addr 172.21.15.12:0/815688592 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.938+0000 7f7f7588a700 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab0eca0 0x561a6ab1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f7608b700 1 -- 172.21.15.12:0/815688592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab0eca0 msgr2=0x561a6ab1ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f7608b700 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab0eca0 0x561a6ab1ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f7608b700 1 -- 172.21.15.12:0/815688592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 msgr2=0x561a6ab17ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f7608b700 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab17ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f7608b700 1 -- 172.21.15.12:0/815688592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f700693b0 con 0x561a6ab14b10 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f75089700 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab17ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f7608b700 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab185c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7f68007ba0 tx=0x7f7f68007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b2fe0d49c0bb73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f597fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f597fa700 1 -- 172.21.15.12:0/815688592 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f680118a0 con 0x561a6ab14b10 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f597fa700 1 -- 172.21.15.12:0/815688592 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7f68011a00 con 0x561a6ab14b10 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/815688592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a6a9b3270 con 0x561a6ab14b10 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f597fa700 1 -- 172.21.15.12:0/815688592 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f68011bc0 con 0x561a6ab14b10 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f83ca8b80 10 client.18231 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.939+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/815688592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x561a6a887b60 con 0x561a6ab14b10 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.940+0000 7f7f597fa700 1 -- 172.21.15.12:0/815688592 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a6a9b3270 con 0x561a6ab14b10 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.940+0000 7f7f597fa700 1 -- 172.21.15.12:0/815688592 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7f68014070 con 0x561a6ab14b10 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.940+0000 7f7f83ca8b80 10 client.18231 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.940+0000 7f7f83ca8b80 10 client.18231 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:56:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.940+0000 7f7f83ca8b80 20 client.18231 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.940+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561a6ab283b0 0x561a6ab2a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.940+0000 7f7f83ca8b80 4 client.18231 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.940+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/815688592 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a6aad9810 con 0x561a6ab283b0 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.940+0000 7f7f7588a700 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561a6ab283b0 0x561a6ab2a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f597fa700 10 client.18231 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f7588a700 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561a6ab283b0 0x561a6ab2a7d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd7bda9126a8721d server_cookie=3cdb972375d19c7c in_seq=0 out_seq=0 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f597fa700 1 -- 172.21.15.12:0/815688592 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a6aad9810 con 0x561a6ab283b0 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f597fa700 10 client.18231 handle_command_reply: tid=0 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f83ca8b80 1 client.18231 shutdown 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/815688592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 msgr2=0x561a6ab185c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab185c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7f68007ba0 tx=0x7f7f68007eb0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/815688592 shutdown_connections 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561a6ab283b0 0x561a6ab2a7d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab185c0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab0eca0 0x561a6ab1ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f597fa700 0 client.18231 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:56:24.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/815688592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab17ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.941+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/815688592 >> 172.21.15.12:0/815688592 conn(0x561a6aa68210 msgr2=0x561a6ab045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.942+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/815688592 shutdown_connections 2024-05-08T23:56:24.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.942+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/815688592 wait complete. 2024-05-08T23:56:24.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.942+0000 7f7f83ca8b80 20 client.18231 trim_cache size 0 max 16384 2024-05-08T23:56:24.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f83ca8b80 1 Processor -- start 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f83ca8b80 1 -- start start 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab0eca0 0x561a6ab04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab1f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab14b10 0x561a6ab1f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6ab149c0 con 0x561a6ab0eca0 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6ab05a00 con 0x561a6ab141f0 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6aaff830 con 0x561a6ab14b10 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f7588a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab0eca0 0x561a6ab04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f7588a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab0eca0 0x561a6ab04d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38662/0 (socket says 172.21.15.12:38662) 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f7588a700 1 -- 172.21.15.12:0/3845066247 learned_addr learned my addr 172.21.15.12:0/3845066247 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f75089700 1 --2- 172.21.15.12:0/3845066247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab1f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f7608b700 1 --2- 172.21.15.12:0/3845066247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab14b10 0x561a6ab1f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f7588a700 1 -- 172.21.15.12:0/3845066247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab14b10 msgr2=0x561a6ab1f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.956+0000 7f7f7588a700 1 --2- 172.21.15.12:0/3845066247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab14b10 0x561a6ab1f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.957+0000 7f7f7588a700 1 -- 172.21.15.12:0/3845066247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 msgr2=0x561a6ab1f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.957+0000 7f7f7588a700 1 --2- 172.21.15.12:0/3845066247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab1f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.957+0000 7f7f7588a700 1 -- 172.21.15.12:0/3845066247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a6aa20560 con 0x561a6ab0eca0 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.957+0000 7f7f7608b700 1 --2- 172.21.15.12:0/3845066247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab14b10 0x561a6ab1f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.957+0000 7f7f7588a700 1 --2- 172.21.15.12:0/3845066247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab0eca0 0x561a6ab04d20 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x561a6ab2b0e0 tx=0x7f7f6c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cefe70d20971eed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.957+0000 7f7f64da4700 1 -- 172.21.15.12:0/3845066247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c027de0 con 0x561a6ab0eca0 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.957+0000 7f7f64da4700 1 -- 172.21.15.12:0/3845066247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f6c0277e0 con 0x561a6ab0eca0 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.957+0000 7f7f64da4700 1 -- 172.21.15.12:0/3845066247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c010850 con 0x561a6ab0eca0 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.957+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/3845066247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab0eca0 msgr2=0x561a6ab04d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.957+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/3845066247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab0eca0 0x561a6ab04d20 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x561a6ab2b0e0 tx=0x7f7f6c014000 comp rx=0 tx=0).stop 2024-05-08T23:56:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.958+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/3845066247 shutdown_connections 2024-05-08T23:56:24.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.958+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/3845066247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab14b10 0x561a6ab1f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.958+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/3845066247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab0eca0 0x561a6ab04d20 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.958+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/3845066247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab141f0 0x561a6ab1f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.958+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/3845066247 >> 172.21.15.12:0/3845066247 conn(0x561a6aa68210 msgr2=0x561a6ab11ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.958+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/3845066247 shutdown_connections 2024-05-08T23:56:24.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.958+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/3845066247 wait complete. 2024-05-08T23:56:24.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.958+0000 7f7f83ca8b80 1 -- start start 2024-05-08T23:56:24.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.958+0000 7f7f83ca8b80 1 Processor -- start 2024-05-08T23:56:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab0eca0 0x561a6ab151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab141f0 0x561a6ab156f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f83ca8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab1e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6ab15e40 con 0x561a6ab14b10 2024-05-08T23:56:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6ab1e840 con 0x561a6ab0eca0 2024-05-08T23:56:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f83ca8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6ab1ea00 con 0x561a6ab141f0 2024-05-08T23:56:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f83ca8b80 10 client.0 _ll_register_callbacks cb 0x561a6a9ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f7608b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab1e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f7608b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab1e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38672/0 (socket says 172.21.15.12:38672) 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f7608b700 1 -- 172.21.15.12:0/2869031836 learned_addr learned my addr 172.21.15.12:0/2869031836 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f7588a700 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab0eca0 0x561a6ab151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f75089700 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab141f0 0x561a6ab156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f7608b700 1 -- 172.21.15.12:0/2869031836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab141f0 msgr2=0x561a6ab156f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f7608b700 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab141f0 0x561a6ab156f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f7608b700 1 -- 172.21.15.12:0/2869031836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab0eca0 msgr2=0x561a6ab151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f7608b700 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab0eca0 0x561a6ab151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f7608b700 1 -- 172.21.15.12:0/2869031836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c00e9a0 con 0x561a6ab14b10 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f7608b700 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab1e300 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7f680096f0 tx=0x7f7f68015a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ee07ad12bf1565 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f597fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.959+0000 7f7f597fa700 1 -- 172.21.15.12:0/2869031836 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6802d070 con 0x561a6ab14b10 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f597fa700 1 -- 172.21.15.12:0/2869031836 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7f680155a0 con 0x561a6ab14b10 2024-05-08T23:56:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/2869031836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f600a5df0 con 0x561a6ab14b10 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f597fa700 1 -- 172.21.15.12:0/2869031836 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f680114b0 con 0x561a6ab14b10 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f83ca8b80 10 client.18237 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/2869031836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x561a6a8dfcd0 con 0x561a6ab14b10 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f597fa700 1 -- 172.21.15.12:0/2869031836 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f68036020 con 0x561a6ab14b10 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f597fa700 1 -- 172.21.15.12:0/2869031836 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7f68014050 con 0x561a6ab14b10 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f83ca8b80 10 client.18237 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f83ca8b80 10 client.18237 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f83ca8b80 20 client.18237 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561a6ab7f6e0 0x561a6ab81b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f83ca8b80 4 client.18237 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/2869031836 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561a6aa52ec0 con 0x561a6ab7f6e0 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.960+0000 7f7f7588a700 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561a6ab7f6e0 0x561a6ab81b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f7588a700 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561a6ab7f6e0 0x561a6ab81b00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef3e1501ecb621d server_cookie=20f6a3a33d085b3a in_seq=0 out_seq=0 2024-05-08T23:56:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f597fa700 10 client.18237 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f597fa700 1 -- 172.21.15.12:0/2869031836 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561a6aa52ec0 con 0x561a6ab7f6e0 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f597fa700 10 client.18237 handle_command_reply: tid=0 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 client.18237 shutdown 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/2869031836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 msgr2=0x561a6ab1e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab1e300 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7f680096f0 tx=0x7f7f68015a90 comp rx=0 tx=0).stop 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/2869031836 shutdown_connections 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x561a6ab7f6e0 0x561a6ab81b00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6ab14b10 0x561a6ab1e300 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561a6ab0eca0 0x561a6ab151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 --2- 172.21.15.12:0/2869031836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561a6ab141f0 0x561a6ab156f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f597fa700 0 client.18237 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/2869031836 >> 172.21.15.12:0/2869031836 conn(0x561a6aa68210 msgr2=0x561a6ab166d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/2869031836 shutdown_connections 2024-05-08T23:56:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.961+0000 7f7f83ca8b80 1 -- 172.21.15.12:0/2869031836 wait complete. 2024-05-08T23:56:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f83ca8b80 20 client.18237 trim_cache size 0 max 16384 2024-05-08T23:56:24.964 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 -- 172.21.15.12:0/2025880566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f5c0a1290 msgr2=0x7f7f5c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f5c0a1290 0x7f7f5c0a3750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c005fd0 tx=0x7f7f6c005e50 comp rx=0 tx=0).stop 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 -- 172.21.15.12:0/2025880566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f70100ab0 msgr2=0x7f7f701967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f70100ab0 0x7f7f701967b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7f600098e0 tx=0x7f7f60003680 comp rx=0 tx=0).stop 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 -- 172.21.15.12:0/2025880566 shutdown_connections 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f5c0a1290 0x7f7f5c0a3750 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f70105f90 0x7f7f7019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f701000f0 0x7f7f70196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 --2- 172.21.15.12:0/2025880566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f70100ab0 0x7f7f701967b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 -- 172.21.15.12:0/2025880566 >> 172.21.15.12:0/2025880566 conn(0x7f7f70076ba0 msgr2=0x7f7f700fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 -- 172.21.15.12:0/2025880566 shutdown_connections 2024-05-08T23:56:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:24.962+0000 7f7f77aee700 1 -- 172.21.15.12:0/2025880566 wait complete. 2024-05-08T23:56:24.970 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:56:25.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:24 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2593404167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:56:25.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:24 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2593404167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:56:25.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.115+0000 7f45a6025700 1 -- 172.21.15.12:0/3082743070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45a01062a0 msgr2=0x7f45a010de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.115+0000 7f45a6025700 1 --2- 172.21.15.12:0/3082743070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45a01062a0 0x7f45a010de20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f459800b0a0 tx=0x7f459800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.115+0000 7f45a6025700 1 -- 172.21.15.12:0/3082743070 shutdown_connections 2024-05-08T23:56:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.115+0000 7f45a6025700 1 --2- 172.21.15.12:0/3082743070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45a01062a0 0x7f45a010de20 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.115+0000 7f45a6025700 1 --2- 172.21.15.12:0/3082743070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45a0069420 0x7f45a0069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.115+0000 7f45a6025700 1 --2- 172.21.15.12:0/3082743070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45a0069dd0 0x7f45a0105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.115+0000 7f45a6025700 1 -- 172.21.15.12:0/3082743070 >> 172.21.15.12:0/3082743070 conn(0x7f45a0076ba0 msgr2=0x7f45a0076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.115+0000 7f45a6025700 1 -- 172.21.15.12:0/3082743070 shutdown_connections 2024-05-08T23:56:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.115+0000 7f45a6025700 1 -- 172.21.15.12:0/3082743070 wait complete. 2024-05-08T23:56:25.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.116+0000 7f45a6025700 1 Processor -- start 2024-05-08T23:56:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.116+0000 7f45a6025700 1 -- start start 2024-05-08T23:56:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.116+0000 7f45a6025700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45a0069420 0x7f45a01001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.116+0000 7f45a6025700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45a01062a0 0x7f45a0100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.116+0000 7f45a6025700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45a0100c40 0x7f45a01a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.116+0000 7f45a6025700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a01a5cc0 con 0x7f45a01062a0 2024-05-08T23:56:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.116+0000 7f45a6025700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a01a5e00 con 0x7f45a0100c40 2024-05-08T23:56:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.116+0000 7f45a6025700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a01a5fc0 con 0x7f45a0069420 2024-05-08T23:56:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.116+0000 7f459effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45a01062a0 0x7f45a0100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.116+0000 7f459effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45a01062a0 0x7f45a0100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38678/0 (socket says 172.21.15.12:38678) 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459effd700 1 -- 172.21.15.12:0/682418848 learned_addr learned my addr 172.21.15.12:0/682418848 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459ffff700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45a0100c40 0x7f45a01a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459f7fe700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45a0069420 0x7f45a01001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459effd700 1 -- 172.21.15.12:0/682418848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45a0069420 msgr2=0x7f45a01001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459effd700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45a0069420 0x7f45a01001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459effd700 1 -- 172.21.15.12:0/682418848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45a0100c40 msgr2=0x7f45a01a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459effd700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45a0100c40 0x7f45a01a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459effd700 1 -- 172.21.15.12:0/682418848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4598009d00 con 0x7f45a01062a0 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459ffff700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45a0100c40 0x7f45a01a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459effd700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45a01062a0 0x7f45a0100700 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f458800d980 tx=0x7f458800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459cff9700 1 -- 172.21.15.12:0/682418848 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588009b40 con 0x7f45a01062a0 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459cff9700 1 -- 172.21.15.12:0/682418848 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458800de60 con 0x7f45a01062a0 2024-05-08T23:56:25.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f45a6025700 1 -- 172.21.15.12:0/682418848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a01a62a0 con 0x7f45a01062a0 2024-05-08T23:56:25.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f459cff9700 1 -- 172.21.15.12:0/682418848 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458800f620 con 0x7f45a01062a0 2024-05-08T23:56:25.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.117+0000 7f45a6025700 1 -- 172.21.15.12:0/682418848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a01a67b0 con 0x7f45a01062a0 2024-05-08T23:56:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.120+0000 7f459cff9700 1 -- 172.21.15.12:0/682418848 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4588010460 con 0x7f45a01062a0 2024-05-08T23:56:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.121+0000 7f459cff9700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f458c0a1290 0x7f458c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.121+0000 7f459cff9700 1 -- 172.21.15.12:0/682418848 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f45880cc850 con 0x7f45a01062a0 2024-05-08T23:56:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.121+0000 7f459f7fe700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f458c0a1290 0x7f458c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.121+0000 7f459f7fe700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f458c0a1290 0x7f458c0a3750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4590009bf0 tx=0x7f4590009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.121+0000 7f45b21dfb80 1 Processor -- start 2024-05-08T23:56:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.121+0000 7f45b21dfb80 1 -- start start 2024-05-08T23:56:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 0x55bc0cd3a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 0x55bc0cd34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd34890 0x55bc0cd34cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cc8b150 con 0x55bc0cd34890 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cc88b10 con 0x55bc0cd3a720 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cc84400 con 0x55bc0cd39e00 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd34890 0x55bc0cd34cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd34890 0x55bc0cd34cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38684/0 (socket says 172.21.15.12:38684) 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459ffff700 1 -- 172.21.15.12:0/2037680875 learned_addr learned my addr 172.21.15.12:0/2037680875 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459effd700 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 0x55bc0cd34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459f7fe700 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 0x55bc0cd3a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459ffff700 1 -- 172.21.15.12:0/2037680875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 msgr2=0x55bc0cd3a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459ffff700 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 0x55bc0cd3a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459ffff700 1 -- 172.21.15.12:0/2037680875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 msgr2=0x55bc0cd34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459ffff700 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 0x55bc0cd34350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459ffff700 1 -- 172.21.15.12:0/2037680875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0cb053b0 con 0x55bc0cd34890 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459effd700 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 0x55bc0cd34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459f7fe700 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 0x55bc0cd3a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.122+0000 7f459ffff700 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd34890 0x55bc0cd34cd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4598009390 tx=0x7f45980093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb5db5c45d43bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f4595ffb700 1 -- 172.21.15.12:0/2037680875 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459800e050 con 0x55bc0cd34890 2024-05-08T23:56:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f4595ffb700 1 -- 172.21.15.12:0/2037680875 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4598007cc0 con 0x55bc0cd34890 2024-05-08T23:56:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f4595ffb700 1 -- 172.21.15.12:0/2037680875 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598004c90 con 0x55bc0cd34890 2024-05-08T23:56:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f45b21dfb80 1 -- 172.21.15.12:0/2037680875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd34890 msgr2=0x55bc0cd34cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd34890 0x55bc0cd34cd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4598009390 tx=0x7f45980093c0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f45b21dfb80 1 -- 172.21.15.12:0/2037680875 shutdown_connections 2024-05-08T23:56:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd34890 0x55bc0cd34cd0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 0x55bc0cd3a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/2037680875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 0x55bc0cd34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f45b21dfb80 1 -- 172.21.15.12:0/2037680875 >> 172.21.15.12:0/2037680875 conn(0x55bc0cc92210 msgr2=0x55bc0cd2edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f45b21dfb80 1 -- 172.21.15.12:0/2037680875 shutdown_connections 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.123+0000 7f45b21dfb80 1 -- 172.21.15.12:0/2037680875 wait complete. 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.124+0000 7f45b21dfb80 1 -- start start 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.124+0000 7f45b21dfb80 1 Processor -- start 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.124+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd34890 0x55bc0cd3dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.124+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd39e00 0x55bc0cd3e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.124+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd3e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.124+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cc20560 con 0x55bc0cd3a720 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.124+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cc20200 con 0x55bc0cd39e00 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.124+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cb076a0 con 0x55bc0cd34890 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f45b21dfb80 10 client.0 _ll_register_callbacks cb 0x55bc0cc29b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd3e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd3e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38688/0 (socket says 172.21.15.12:38688) 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459ffff700 1 -- 172.21.15.12:0/157515040 learned_addr learned my addr 172.21.15.12:0/157515040 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459f7fe700 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd34890 0x55bc0cd3dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459effd700 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd39e00 0x55bc0cd3e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459ffff700 1 -- 172.21.15.12:0/157515040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd34890 msgr2=0x55bc0cd3dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459ffff700 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd34890 0x55bc0cd3dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459ffff700 1 -- 172.21.15.12:0/157515040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd39e00 msgr2=0x55bc0cd3e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459ffff700 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd39e00 0x55bc0cd3e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459ffff700 1 -- 172.21.15.12:0/157515040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a01a67b0 con 0x55bc0cd3a720 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f459ffff700 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd3e550 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f459800b3f0 tx=0x7f4598004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f30d8dd0324b95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f457effd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f457effd700 1 -- 172.21.15.12:0/157515040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45980078a0 con 0x55bc0cd3a720 2024-05-08T23:56:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f45b21dfb80 1 -- 172.21.15.12:0/157515040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc0cbdd270 con 0x55bc0cd3a720 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.125+0000 7f457effd700 1 -- 172.21.15.12:0/157515040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f459800bbe0 con 0x55bc0cd3a720 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f457effd700 1 -- 172.21.15.12:0/157515040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f459800b740 con 0x55bc0cd3a720 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f45b21dfb80 10 client.18249 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f45b21dfb80 1 -- 172.21.15.12:0/157515040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f45a010e690 con 0x55bc0cd3a720 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f457effd700 1 -- 172.21.15.12:0/157515040 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc0cbdd270 con 0x55bc0cd3a720 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f457effd700 1 -- 172.21.15.12:0/157515040 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f459800b0d0 con 0x55bc0cd3a720 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f45b21dfb80 10 client.18249 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f45b21dfb80 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f45b21dfb80 20 client.18249 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bc0cd4dc40 0x55bc0cd50060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f45b21dfb80 4 client.18249 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f45b21dfb80 1 -- 172.21.15.12:0/157515040 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc0cd03810 con 0x55bc0cd4dc40 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.126+0000 7f459f7fe700 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bc0cd4dc40 0x55bc0cd50060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.127+0000 7f459f7fe700 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bc0cd4dc40 0x55bc0cd50060 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d776843fedf26800 server_cookie=f38bf8c786c7cc1f in_seq=0 out_seq=0 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.127+0000 7f457effd700 10 client.18249 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f457effd700 1 -- 172.21.15.12:0/157515040 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc0cd03810 con 0x55bc0cd4dc40 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f457effd700 10 client.18249 handle_command_reply: tid=0 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 client.18249 shutdown 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 -- 172.21.15.12:0/157515040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 msgr2=0x55bc0cd3e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd3e550 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f459800b3f0 tx=0x7f4598004d30 comp rx=0 tx=0).stop 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 -- 172.21.15.12:0/157515040 shutdown_connections 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd3e550 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd34890 0x55bc0cd3dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bc0cd4dc40 0x55bc0cd50060 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/157515040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd39e00 0x55bc0cd3e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f457effd700 0 client.18249 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 -- 172.21.15.12:0/157515040 >> 172.21.15.12:0/157515040 conn(0x55bc0cc92210 msgr2=0x55bc0cd2e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 -- 172.21.15.12:0/157515040 shutdown_connections 2024-05-08T23:56:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 1 -- 172.21.15.12:0/157515040 wait complete. 2024-05-08T23:56:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.128+0000 7f45b21dfb80 20 client.18249 trim_cache size 0 max 16384 2024-05-08T23:56:25.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.142+0000 7f45b21dfb80 1 Processor -- start 2024-05-08T23:56:25.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.142+0000 7f45b21dfb80 1 -- start start 2024-05-08T23:56:25.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.142+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd34890 0x55bc0cd34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.142+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 0x55bc0cd392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.142+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd2ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.142+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cd3a560 con 0x55bc0cd3a720 2024-05-08T23:56:25.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.142+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cd27080 con 0x55bc0cd34890 2024-05-08T23:56:25.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.142+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cd39800 con 0x55bc0cd39e00 2024-05-08T23:56:25.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd2ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd2ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38698/0 (socket says 172.21.15.12:38698) 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459ffff700 1 -- 172.21.15.12:0/4233310976 learned_addr learned my addr 172.21.15.12:0/4233310976 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459effd700 1 --2- 172.21.15.12:0/4233310976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 0x55bc0cd392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459f7fe700 1 --2- 172.21.15.12:0/4233310976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd34890 0x55bc0cd34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459ffff700 1 -- 172.21.15.12:0/4233310976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 msgr2=0x55bc0cd392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459ffff700 1 --2- 172.21.15.12:0/4233310976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 0x55bc0cd392c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459ffff700 1 -- 172.21.15.12:0/4233310976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd34890 msgr2=0x55bc0cd34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459ffff700 1 --2- 172.21.15.12:0/4233310976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd34890 0x55bc0cd34350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459ffff700 1 -- 172.21.15.12:0/4233310976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0cd34f40 con 0x55bc0cd3a720 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f459ffff700 1 --2- 172.21.15.12:0/4233310976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd2ed20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4598009c80 tx=0x7f4598007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb835722e99cf42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f45967fc700 1 -- 172.21.15.12:0/4233310976 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459800e040 con 0x55bc0cd3a720 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f45967fc700 1 -- 172.21.15.12:0/4233310976 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f459800b700 con 0x55bc0cd3a720 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.143+0000 7f45967fc700 1 -- 172.21.15.12:0/4233310976 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45980123f0 con 0x55bc0cd3a720 2024-05-08T23:56:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.144+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4233310976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 msgr2=0x55bc0cd2ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.144+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/4233310976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd2ed20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4598009c80 tx=0x7f4598007e30 comp rx=0 tx=0).stop 2024-05-08T23:56:25.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.144+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4233310976 shutdown_connections 2024-05-08T23:56:25.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.144+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/4233310976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd3a720 0x55bc0cd2ed20 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.144+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/4233310976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd34890 0x55bc0cd34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.144+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/4233310976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd39e00 0x55bc0cd392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.144+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4233310976 >> 172.21.15.12:0/4233310976 conn(0x55bc0cc92210 msgr2=0x55bc0cd38bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.144+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4233310976 shutdown_connections 2024-05-08T23:56:25.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.144+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4233310976 wait complete. 2024-05-08T23:56:25.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.144+0000 7f45b21dfb80 1 -- start start 2024-05-08T23:56:25.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f45b21dfb80 1 Processor -- start 2024-05-08T23:56:25.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd34890 0x55bc0cd42cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd39e00 0x55bc0cd431f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f45b21dfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 0x55bc0cd43730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cda3230 con 0x55bc0cd39e00 2024-05-08T23:56:25.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cda33a0 con 0x55bc0cd3a720 2024-05-08T23:56:25.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f45b21dfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cda3560 con 0x55bc0cd34890 2024-05-08T23:56:25.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f45b21dfb80 10 client.0 _ll_register_callbacks cb 0x55bc0cc29b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd39e00 0x55bc0cd431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd39e00 0x55bc0cd431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38706/0 (socket says 172.21.15.12:38706) 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459effd700 1 -- 172.21.15.12:0/4139077168 learned_addr learned my addr 172.21.15.12:0/4139077168 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459f7fe700 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd34890 0x55bc0cd42cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459ffff700 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 0x55bc0cd43730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459effd700 1 -- 172.21.15.12:0/4139077168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd34890 msgr2=0x55bc0cd42cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459effd700 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd34890 0x55bc0cd42cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459effd700 1 -- 172.21.15.12:0/4139077168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 msgr2=0x55bc0cd43730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459effd700 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 0x55bc0cd43730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459effd700 1 -- 172.21.15.12:0/4139077168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a010e690 con 0x55bc0cd39e00 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f459effd700 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd39e00 0x55bc0cd431f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f45880c9fe0 tx=0x7f4588099940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40335165d17dc04a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.145+0000 7f457effd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f457effd700 1 -- 172.21.15.12:0/4139077168 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45880a1dc0 con 0x55bc0cd39e00 2024-05-08T23:56:25.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f457effd700 1 -- 172.21.15.12:0/4139077168 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4588099ba0 con 0x55bc0cd39e00 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4139077168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4598007cc0 con 0x55bc0cd39e00 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f457effd700 1 -- 172.21.15.12:0/4139077168 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4588099d60 con 0x55bc0cd39e00 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f45b21dfb80 10 client.18255 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4139077168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55bc0cab1b60 con 0x55bc0cd39e00 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f457effd700 1 -- 172.21.15.12:0/4139077168 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4598007cc0 con 0x55bc0cd39e00 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f457effd700 1 -- 172.21.15.12:0/4139077168 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f45880a19c0 con 0x55bc0cd39e00 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f45b21dfb80 10 client.18255 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f45b21dfb80 10 client.18255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f45b21dfb80 20 client.18255 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bc0cda4c10 0x55bc0cda7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f45b21dfb80 4 client.18255 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.146+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4139077168 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc0cc7cec0 con 0x55bc0cda4c10 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f459f7fe700 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bc0cda4c10 0x55bc0cda7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f457effd700 10 client.18255 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f459f7fe700 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bc0cda4c10 0x55bc0cda7030 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43bd52fd7633439 server_cookie=ef61175761064d79 in_seq=0 out_seq=0 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f457effd700 1 -- 172.21.15.12:0/4139077168 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc0cc7cec0 con 0x55bc0cda4c10 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f457effd700 10 client.18255 handle_command_reply: tid=0 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f45b21dfb80 1 client.18255 shutdown 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4139077168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd39e00 msgr2=0x55bc0cd431f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd39e00 0x55bc0cd431f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f45880c9fe0 tx=0x7f4588099940 comp rx=0 tx=0).stop 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4139077168 shutdown_connections 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bc0cda4c10 0x55bc0cda7030 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc0cd3a720 0x55bc0cd43730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc0cd34890 0x55bc0cd42cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f457effd700 0 client.18255 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f45b21dfb80 1 --2- 172.21.15.12:0/4139077168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc0cd39e00 0x55bc0cd431f0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.147+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4139077168 >> 172.21.15.12:0/4139077168 conn(0x55bc0cc92210 msgr2=0x55bc0cd3e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4139077168 shutdown_connections 2024-05-08T23:56:25.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45b21dfb80 1 -- 172.21.15.12:0/4139077168 wait complete. 2024-05-08T23:56:25.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45b21dfb80 20 client.18255 trim_cache size 0 max 16384 2024-05-08T23:56:25.149 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 -- 172.21.15.12:0/682418848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f458c0a1290 msgr2=0x7f458c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f458c0a1290 0x7f458c0a3750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4590009bf0 tx=0x7f4590009260 comp rx=0 tx=0).stop 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 -- 172.21.15.12:0/682418848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45a01062a0 msgr2=0x7f45a0100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45a01062a0 0x7f45a0100700 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f458800d980 tx=0x7f458800dd40 comp rx=0 tx=0).stop 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 -- 172.21.15.12:0/682418848 shutdown_connections 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f458c0a1290 0x7f458c0a3750 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45a0100c40 0x7f45a01a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45a0069420 0x7f45a01001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 --2- 172.21.15.12:0/682418848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45a01062a0 0x7f45a0100700 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 -- 172.21.15.12:0/682418848 >> 172.21.15.12:0/682418848 conn(0x7f45a0076ba0 msgr2=0x7f45a00fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 -- 172.21.15.12:0/682418848 shutdown_connections 2024-05-08T23:56:25.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:25.148+0000 7f45a6025700 1 -- 172.21.15.12:0/682418848 wait complete. 2024-05-08T23:56:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.416+0000 7f33697bd700 1 -- 172.21.15.12:0/1911157063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33641011b0 msgr2=0x7f3364104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.416+0000 7f33697bd700 1 --2- 172.21.15.12:0/1911157063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33641011b0 0x7f3364104990 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3358009990 tx=0x7f3358009ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.416+0000 7f33697bd700 1 -- 172.21.15.12:0/1911157063 shutdown_connections 2024-05-08T23:56:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.416+0000 7f33697bd700 1 --2- 172.21.15.12:0/1911157063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33641050c0 0x7f336410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.416+0000 7f33697bd700 1 --2- 172.21.15.12:0/1911157063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3364100800 0x7f3364100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.416+0000 7f33697bd700 1 --2- 172.21.15.12:0/1911157063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33641011b0 0x7f3364104990 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.416+0000 7f33697bd700 1 -- 172.21.15.12:0/1911157063 >> 172.21.15.12:0/1911157063 conn(0x7f33640773b0 msgr2=0x7f33640fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.416+0000 7f33697bd700 1 -- 172.21.15.12:0/1911157063 shutdown_connections 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.416+0000 7f33697bd700 1 -- 172.21.15.12:0/1911157063 wait complete. 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.416+0000 7f33697bd700 1 Processor -- start 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f33697bd700 1 -- start start 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f33697bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3364100800 0x7f3364199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f33697bd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33641050c0 0x7f3364199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f33697bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f336419de00 0x7f336419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f33697bd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336419e880 con 0x7f3364100800 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f33697bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336419e9f0 con 0x7f336419de00 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f33697bd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336419ebb0 con 0x7f33641050c0 2024-05-08T23:56:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f3362ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3364100800 0x7f3364199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f3362ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3364100800 0x7f3364199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38720/0 (socket says 172.21.15.12:38720) 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f3362ffd700 1 -- 172.21.15.12:0/41550766 learned_addr learned my addr 172.21.15.12:0/41550766 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f33637fe700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f336419de00 0x7f336419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f33627fc700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33641050c0 0x7f3364199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f3362ffd700 1 -- 172.21.15.12:0/41550766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33641050c0 msgr2=0x7f3364199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f3362ffd700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33641050c0 0x7f3364199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f3362ffd700 1 -- 172.21.15.12:0/41550766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f336419de00 msgr2=0x7f336419e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f3362ffd700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f336419de00 0x7f336419e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f3362ffd700 1 -- 172.21.15.12:0/41550766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33580095f0 con 0x7f3364100800 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.417+0000 7f33627fc700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33641050c0 0x7f3364199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.418+0000 7f3362ffd700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3364100800 0x7f3364199520 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f335400d980 tx=0x7f335400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.418+0000 7f334bfff700 1 -- 172.21.15.12:0/41550766 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354004220 con 0x7f3364100800 2024-05-08T23:56:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.418+0000 7f33697bd700 1 -- 172.21.15.12:0/41550766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33641a6110 con 0x7f3364100800 2024-05-08T23:56:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.418+0000 7f334bfff700 1 -- 172.21.15.12:0/41550766 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3354007640 con 0x7f3364100800 2024-05-08T23:56:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.418+0000 7f334bfff700 1 -- 172.21.15.12:0/41550766 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354005610 con 0x7f3364100800 2024-05-08T23:56:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.418+0000 7f33697bd700 1 -- 172.21.15.12:0/41550766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33641a65f0 con 0x7f3364100800 2024-05-08T23:56:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.422+0000 7f334bfff700 1 -- 172.21.15.12:0/41550766 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3354004750 con 0x7f3364100800 2024-05-08T23:56:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.422+0000 7f334bfff700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f334c0a1290 0x7f334c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.422+0000 7f334bfff700 1 -- 172.21.15.12:0/41550766 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f33540ccb40 con 0x7f3364100800 2024-05-08T23:56:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.422+0000 7f33627fc700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f334c0a1290 0x7f334c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3375977b80 1 Processor -- start 2024-05-08T23:56:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3375977b80 1 -- start start 2024-05-08T23:56:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3375977b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b89e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3375977b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e1270 0x55c3b89daf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3375977b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 0x55c3b89db8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f33627fc700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f334c0a1290 0x7f334c0a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3358000c00 tx=0x7f3358005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3375977b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b89320f0 con 0x55c3b89e0950 2024-05-08T23:56:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3375977b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b892fab0 con 0x55c3b89db480 2024-05-08T23:56:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3375977b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b892b3a0 con 0x55c3b89e1270 2024-05-08T23:56:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3362ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b89e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3362ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b89e0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38724/0 (socket says 172.21.15.12:38724) 2024-05-08T23:56:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3362ffd700 1 -- 172.21.15.12:0/1934119195 learned_addr learned my addr 172.21.15.12:0/1934119195 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f33627fc700 1 --2- 172.21.15.12:0/1934119195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e1270 0x55c3b89daf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f33637fe700 1 --2- 172.21.15.12:0/1934119195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 0x55c3b89db8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3362ffd700 1 -- 172.21.15.12:0/1934119195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e1270 msgr2=0x55c3b89daf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3362ffd700 1 --2- 172.21.15.12:0/1934119195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e1270 0x55c3b89daf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3362ffd700 1 -- 172.21.15.12:0/1934119195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 msgr2=0x55c3b89db8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3362ffd700 1 --2- 172.21.15.12:0/1934119195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 0x55c3b89db8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f3362ffd700 1 -- 172.21.15.12:0/1934119195 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b87ac350 con 0x55c3b89e0950 2024-05-08T23:56:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.423+0000 7f33637fe700 1 --2- 172.21.15.12:0/1934119195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 0x55c3b89db8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:56:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.424+0000 7f3362ffd700 1 --2- 172.21.15.12:0/1934119195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b89e0d30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f33540a1b80 tx=0x7f335409a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac14a371cd9d1102 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.424+0000 7f33497fa700 1 -- 172.21.15.12:0/1934119195 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33540a20b0 con 0x55c3b89e0950 2024-05-08T23:56:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.424+0000 7f33497fa700 1 -- 172.21.15.12:0/1934119195 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3354097fd0 con 0x55c3b89e0950 2024-05-08T23:56:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.424+0000 7f33497fa700 1 -- 172.21.15.12:0/1934119195 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354099320 con 0x55c3b89e0950 2024-05-08T23:56:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.424+0000 7f3375977b80 1 -- 172.21.15.12:0/1934119195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 msgr2=0x55c3b89e0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.424+0000 7f3375977b80 1 --2- 172.21.15.12:0/1934119195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b89e0d30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f33540a1b80 tx=0x7f335409a3b0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 -- 172.21.15.12:0/1934119195 shutdown_connections 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 --2- 172.21.15.12:0/1934119195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 0x55c3b89db8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 --2- 172.21.15.12:0/1934119195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b89e0d30 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 --2- 172.21.15.12:0/1934119195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e1270 0x55c3b89daf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 -- 172.21.15.12:0/1934119195 >> 172.21.15.12:0/1934119195 conn(0x55c3b89391b0 msgr2=0x55c3b89d5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 -- 172.21.15.12:0/1934119195 shutdown_connections 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 -- 172.21.15.12:0/1934119195 wait complete. 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 -- start start 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 Processor -- start 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89db480 0x55c3b89e4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89e0950 0x55c3b89e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e1270 0x55c3b89e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b88c7500 con 0x55c3b89e1270 2024-05-08T23:56:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b88c71a0 con 0x55c3b89e0950 2024-05-08T23:56:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b87ae640 con 0x55c3b89db480 2024-05-08T23:56:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.425+0000 7f3375977b80 10 client.0 _ll_register_callbacks cb 0x55c3b88d0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f33637fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e1270 0x55c3b89e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f33637fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e1270 0x55c3b89e4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38732/0 (socket says 172.21.15.12:38732) 2024-05-08T23:56:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f33637fe700 1 -- 172.21.15.12:0/455988190 learned_addr learned my addr 172.21.15.12:0/455988190 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f3362ffd700 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89db480 0x55c3b89e4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f33627fc700 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89e0950 0x55c3b89e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f33637fe700 1 -- 172.21.15.12:0/455988190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89db480 msgr2=0x55c3b89e4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f33637fe700 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89db480 0x55c3b89e4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f33637fe700 1 -- 172.21.15.12:0/455988190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89e0950 msgr2=0x55c3b89e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f33637fe700 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89e0950 0x55c3b89e4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f33637fe700 1 -- 172.21.15.12:0/455988190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33641a65f0 con 0x55c3b89e1270 2024-05-08T23:56:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f33637fe700 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e1270 0x55c3b89e4fd0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f335c010a30 tx=0x7f335c010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c0f986537a5a8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f3346ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f3346ff5700 1 -- 172.21.15.12:0/455988190 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c010ed0 con 0x55c3b89e1270 2024-05-08T23:56:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f3346ff5700 1 -- 172.21.15.12:0/455988190 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f335c004bb0 con 0x55c3b89e1270 2024-05-08T23:56:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f3375977b80 1 -- 172.21.15.12:0/455988190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3b8884210 con 0x55c3b89e1270 2024-05-08T23:56:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f3346ff5700 1 -- 172.21.15.12:0/455988190 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f335c007e90 con 0x55c3b89e1270 2024-05-08T23:56:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f3375977b80 10 client.18267 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.426+0000 7f3375977b80 1 -- 172.21.15.12:0/455988190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c3b8758b00 con 0x55c3b89e1270 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.427+0000 7f3346ff5700 1 -- 172.21.15.12:0/455988190 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3b8884210 con 0x55c3b89e1270 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.427+0000 7f3346ff5700 1 -- 172.21.15.12:0/455988190 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f335c015970 con 0x55c3b89e1270 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.427+0000 7f3375977b80 10 client.18267 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.427+0000 7f3375977b80 10 client.18267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.427+0000 7f3375977b80 20 client.18267 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.427+0000 7f3375977b80 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3b89f4b80 0x55c3b89f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.427+0000 7f3375977b80 4 client.18267 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.427+0000 7f3375977b80 1 -- 172.21.15.12:0/455988190 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3b89aa7b0 con 0x55c3b89f4b80 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.427+0000 7f3362ffd700 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3b89f4b80 0x55c3b89f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3346ff5700 10 client.18267 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3362ffd700 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3b89f4b80 0x55c3b89f6fa0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c26983f6312ec92 server_cookie=6d5a9b6879b91e1f in_seq=0 out_seq=0 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3346ff5700 1 -- 172.21.15.12:0/455988190 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3b89aa7b0 con 0x55c3b89f4b80 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3346ff5700 10 client.18267 handle_command_reply: tid=0 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3375977b80 1 client.18267 shutdown 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3375977b80 1 -- 172.21.15.12:0/455988190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e1270 msgr2=0x55c3b89e4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3375977b80 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e1270 0x55c3b89e4fd0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f335c010a30 tx=0x7f335c010d40 comp rx=0 tx=0).stop 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3375977b80 1 -- 172.21.15.12:0/455988190 shutdown_connections 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3375977b80 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e1270 0x55c3b89e4fd0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3375977b80 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89db480 0x55c3b89e4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3375977b80 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3b89f4b80 0x55c3b89f6fa0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3375977b80 1 --2- 172.21.15.12:0/455988190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89e0950 0x55c3b89e4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3346ff5700 0 client.18267 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3375977b80 1 -- 172.21.15.12:0/455988190 >> 172.21.15.12:0/455988190 conn(0x55c3b89391b0 msgr2=0x55c3b89d9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.428+0000 7f3375977b80 1 -- 172.21.15.12:0/455988190 shutdown_connections 2024-05-08T23:56:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.429+0000 7f3375977b80 1 -- 172.21.15.12:0/455988190 wait complete. 2024-05-08T23:56:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.429+0000 7f3375977b80 20 client.18267 trim_cache size 0 max 16384 2024-05-08T23:56:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3375977b80 1 Processor -- start 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3375977b80 1 -- start start 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3375977b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89db480 0x55c3b89f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3375977b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e0950 0x55c3b89d5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3375977b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89e1270 0x55c3b89d7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3375977b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b89d6330 con 0x55c3b89db480 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3375977b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b89f2180 con 0x55c3b89e1270 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3375977b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b89fc1b0 con 0x55c3b89e0950 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3362ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89db480 0x55c3b89f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3362ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89db480 0x55c3b89f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38734/0 (socket says 172.21.15.12:38734) 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.444+0000 7f3362ffd700 1 -- 172.21.15.12:0/1430281690 learned_addr learned my addr 172.21.15.12:0/1430281690 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f33627fc700 1 --2- 172.21.15.12:0/1430281690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e0950 0x55c3b89d5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f33637fe700 1 --2- 172.21.15.12:0/1430281690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89e1270 0x55c3b89d7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f3362ffd700 1 -- 172.21.15.12:0/1430281690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e0950 msgr2=0x55c3b89d5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f3362ffd700 1 --2- 172.21.15.12:0/1430281690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e0950 0x55c3b89d5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f3362ffd700 1 -- 172.21.15.12:0/1430281690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89e1270 msgr2=0x55c3b89d7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f3362ffd700 1 --2- 172.21.15.12:0/1430281690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89e1270 0x55c3b89d7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f3362ffd700 1 -- 172.21.15.12:0/1430281690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b88f1500 con 0x55c3b89db480 2024-05-08T23:56:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f3362ffd700 1 --2- 172.21.15.12:0/1430281690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89db480 0x55c3b89f6f70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f3354097f80 tx=0x7f3354099770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e392d054e90f317e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f3349ffb700 1 -- 172.21.15.12:0/1430281690 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354022eb0 con 0x55c3b89db480 2024-05-08T23:56:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f3349ffb700 1 -- 172.21.15.12:0/1430281690 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3354099120 con 0x55c3b89db480 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.445+0000 7f3349ffb700 1 -- 172.21.15.12:0/1430281690 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33540a1e40 con 0x55c3b89db480 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 -- 172.21.15.12:0/1430281690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89db480 msgr2=0x55c3b89f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 --2- 172.21.15.12:0/1430281690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89db480 0x55c3b89f6f70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f3354097f80 tx=0x7f3354099770 comp rx=0 tx=0).stop 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 -- 172.21.15.12:0/1430281690 shutdown_connections 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 --2- 172.21.15.12:0/1430281690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89e1270 0x55c3b89d7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 --2- 172.21.15.12:0/1430281690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89db480 0x55c3b89f6f70 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 --2- 172.21.15.12:0/1430281690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e0950 0x55c3b89d5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 -- 172.21.15.12:0/1430281690 >> 172.21.15.12:0/1430281690 conn(0x55c3b89391b0 msgr2=0x55c3b89e4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 -- 172.21.15.12:0/1430281690 shutdown_connections 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 -- 172.21.15.12:0/1430281690 wait complete. 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 -- start start 2024-05-08T23:56:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 Processor -- start 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 0x55c3b8a437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.446+0000 7f3375977b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b8a43cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f3375977b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e1270 0x55c3b89eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f3375977b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b89eb810 con 0x55c3b89e0950 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f3375977b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b89eb980 con 0x55c3b89db480 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f3375977b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b89ebb40 con 0x55c3b89e1270 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f3375977b80 10 client.0 _ll_register_callbacks cb 0x55c3b88d0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f33627fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b8a43cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f33627fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b8a43cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38746/0 (socket says 172.21.15.12:38746) 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f33627fc700 1 -- 172.21.15.12:0/1864073370 learned_addr learned my addr 172.21.15.12:0/1864073370 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f3362ffd700 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 0x55c3b8a437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f33637fe700 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e1270 0x55c3b89eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f33627fc700 1 -- 172.21.15.12:0/1864073370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e1270 msgr2=0x55c3b89eb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f33627fc700 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e1270 0x55c3b89eb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f33627fc700 1 -- 172.21.15.12:0/1864073370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 msgr2=0x55c3b8a437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f33627fc700 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 0x55c3b8a437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f33627fc700 1 -- 172.21.15.12:0/1864073370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3b87ac350 con 0x55c3b89e0950 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f33627fc700 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b8a43cf0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3358009990 tx=0x7f3358009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1566e22255755d14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f3346ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f3346ff5700 1 -- 172.21.15.12:0/1864073370 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335801e070 con 0x55c3b89e0950 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f3346ff5700 1 -- 172.21.15.12:0/1864073370 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f335800be80 con 0x55c3b89e0950 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.447+0000 7f3375977b80 1 -- 172.21.15.12:0/1864073370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3354097fd0 con 0x55c3b89e0950 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3346ff5700 1 -- 172.21.15.12:0/1864073370 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3358004710 con 0x55c3b89e0950 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3375977b80 10 client.18273 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3375977b80 1 -- 172.21.15.12:0/1864073370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c3b87b0c70 con 0x55c3b89e0950 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3346ff5700 1 -- 172.21.15.12:0/1864073370 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3354097fd0 con 0x55c3b89e0950 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3346ff5700 1 -- 172.21.15.12:0/1864073370 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f335800fac0 con 0x55c3b89e0950 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3375977b80 10 client.18273 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3375977b80 10 client.18273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3375977b80 20 client.18273 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3375977b80 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3b8a4b6e0 0x55c3b8a4db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3375977b80 4 client.18273 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3375977b80 1 -- 172.21.15.12:0/1864073370 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c3b8923e60 con 0x55c3b8a4b6e0 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.448+0000 7f3362ffd700 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3b8a4b6e0 0x55c3b8a4db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.449+0000 7f3362ffd700 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3b8a4b6e0 0x55c3b8a4db00 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0fb7bde79217d4 server_cookie=9e5781740c8c7724 in_seq=0 out_seq=0 2024-05-08T23:56:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.449+0000 7f3346ff5700 10 client.18273 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.493+0000 7f3346ff5700 1 -- 172.21.15.12:0/1864073370 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c3b8923e60 con 0x55c3b8a4b6e0 2024-05-08T23:56:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.493+0000 7f3346ff5700 10 client.18273 handle_command_reply: tid=0 2024-05-08T23:56:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.493+0000 7f3375977b80 1 client.18273 shutdown 2024-05-08T23:56:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 1 -- 172.21.15.12:0/1864073370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 msgr2=0x55c3b8a43cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b8a43cf0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3358009990 tx=0x7f3358009310 comp rx=0 tx=0).stop 2024-05-08T23:56:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 1 -- 172.21.15.12:0/1864073370 shutdown_connections 2024-05-08T23:56:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b89e1270 0x55c3b89eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3b8a4b6e0 0x55c3b8a4db00 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3b89db480 0x55c3b8a437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 1 --2- 172.21.15.12:0/1864073370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3b89e0950 0x55c3b8a43cf0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3346ff5700 0 client.18273 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 1 -- 172.21.15.12:0/1864073370 >> 172.21.15.12:0/1864073370 conn(0x55c3b89391b0 msgr2=0x55c3b89df5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 1 -- 172.21.15.12:0/1864073370 shutdown_connections 2024-05-08T23:56:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 1 -- 172.21.15.12:0/1864073370 wait complete. 2024-05-08T23:56:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.494+0000 7f3375977b80 20 client.18273 trim_cache size 0 max 16384 2024-05-08T23:56:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:56:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:56:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:56:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:56:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 -- 172.21.15.12:0/41550766 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f334c0a1290 msgr2=0x7f334c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f334c0a1290 0x7f334c0a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3358000c00 tx=0x7f3358005fd0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 -- 172.21.15.12:0/41550766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3364100800 msgr2=0x7f3364199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3364100800 0x7f3364199520 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f335400d980 tx=0x7f335400dc90 comp rx=0 tx=0).stop 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 -- 172.21.15.12:0/41550766 shutdown_connections 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f334c0a1290 0x7f334c0a3750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f336419de00 0x7f336419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3364100800 0x7f3364199520 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 --2- 172.21.15.12:0/41550766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33641050c0 0x7f3364199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 -- 172.21.15.12:0/41550766 >> 172.21.15.12:0/41550766 conn(0x7f33640773b0 msgr2=0x7f33640ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 -- 172.21.15.12:0/41550766 shutdown_connections 2024-05-08T23:56:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:25.495+0000 7f33697bd700 1 -- 172.21.15.12:0/41550766 wait complete. 2024-05-08T23:56:25.885 INFO:tasks.workunit.client.0.smithi012.stdout: 1 566486 6.69 MB/sec execute 381 sec latency 2911.828 ms 2024-05-08T23:56:25.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:25 smithi012 ceph-mon[42503]: pgmap v317: 97 pgs: 97 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-08T23:56:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:25 smithi204 ceph-mon[47373]: pgmap v317: 97 pgs: 97 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-08T23:56:26.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:25 smithi160 ceph-mon[46152]: pgmap v317: 97 pgs: 97 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-08T23:56:26.157 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:56:26.158 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:56:26.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.316+0000 7fcf293b4700 1 -- 172.21.15.12:0/889525455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf24106280 msgr2=0x7fcf2410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.316+0000 7fcf293b4700 1 --2- 172.21.15.12:0/889525455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf24106280 0x7fcf2410de00 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00b0a0 tx=0x7fcf1c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.316+0000 7fcf293b4700 1 -- 172.21.15.12:0/889525455 shutdown_connections 2024-05-08T23:56:26.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.316+0000 7fcf293b4700 1 --2- 172.21.15.12:0/889525455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf24106280 0x7fcf2410de00 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.316+0000 7fcf293b4700 1 --2- 172.21.15.12:0/889525455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf24101160 0x7fcf24101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.316+0000 7fcf293b4700 1 --2- 172.21.15.12:0/889525455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf24101b10 0x7fcf24105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.316+0000 7fcf293b4700 1 -- 172.21.15.12:0/889525455 >> 172.21.15.12:0/889525455 conn(0x7fcf240fc9e0 msgr2=0x7fcf240fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.316+0000 7fcf293b4700 1 -- 172.21.15.12:0/889525455 shutdown_connections 2024-05-08T23:56:26.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.316+0000 7fcf293b4700 1 -- 172.21.15.12:0/889525455 wait complete. 2024-05-08T23:56:26.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.317+0000 7fcf293b4700 1 Processor -- start 2024-05-08T23:56:26.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.317+0000 7fcf293b4700 1 -- start start 2024-05-08T23:56:26.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.317+0000 7fcf293b4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf24101160 0x7fcf24197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.317+0000 7fcf293b4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf24106280 0x7fcf24197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.317+0000 7fcf293b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2419be10 0x7fcf2419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.317+0000 7fcf293b4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2419c890 con 0x7fcf2419be10 2024-05-08T23:56:26.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.317+0000 7fcf293b4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2419ca00 con 0x7fcf24106280 2024-05-08T23:56:26.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.317+0000 7fcf293b4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2419cbc0 con 0x7fcf24101160 2024-05-08T23:56:26.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2419be10 0x7fcf2419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf24106280 0x7fcf24197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf24101160 0x7fcf24197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2419be10 0x7fcf2419c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38750/0 (socket says 172.21.15.12:38750) 2024-05-08T23:56:26.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf237fe700 1 -- 172.21.15.12:0/960964620 learned_addr learned my addr 172.21.15.12:0/960964620 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:26.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf237fe700 1 -- 172.21.15.12:0/960964620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf24101160 msgr2=0x7fcf24197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf237fe700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf24101160 0x7fcf24197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf237fe700 1 -- 172.21.15.12:0/960964620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf24106280 msgr2=0x7fcf24197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf237fe700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf24106280 0x7fcf24197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf237fe700 1 -- 172.21.15.12:0/960964620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c009d00 con 0x7fcf2419be10 2024-05-08T23:56:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf237fe700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2419be10 0x7fcf2419c290 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00b070 tx=0x7fcf1c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf1bfff700 1 -- 172.21.15.12:0/960964620 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00e040 con 0x7fcf2419be10 2024-05-08T23:56:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf293b4700 1 -- 172.21.15.12:0/960964620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf241a40c0 con 0x7fcf2419be10 2024-05-08T23:56:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf1bfff700 1 -- 172.21.15.12:0/960964620 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf1c01d480 con 0x7fcf2419be10 2024-05-08T23:56:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf1bfff700 1 -- 172.21.15.12:0/960964620 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0052a0 con 0x7fcf2419be10 2024-05-08T23:56:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.318+0000 7fcf293b4700 1 -- 172.21.15.12:0/960964620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf241a4540 con 0x7fcf2419be10 2024-05-08T23:56:26.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.321+0000 7fcf1bfff700 1 -- 172.21.15.12:0/960964620 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcf1c01d5f0 con 0x7fcf2419be10 2024-05-08T23:56:26.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.322+0000 7fcf1bfff700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf100a1290 0x7fcf100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.322+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf100a1290 0x7fcf100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.322+0000 7fcf1bfff700 1 -- 172.21.15.12:0/960964620 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcf1c0cda70 con 0x7fcf2419be10 2024-05-08T23:56:26.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.322+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf100a1290 0x7fcf100a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009c60 tx=0x7fcf0c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf3556eb80 1 Processor -- start 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf3556eb80 1 -- start start 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4de80 0x55e156c4e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4e7a0 0x55e156c483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c48910 0x55e156c48d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156b9f1d0 con 0x55e156c48910 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156b9cb90 con 0x55e156c4de80 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156b98480 con 0x55e156c4e7a0 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c48910 0x55e156c48d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4de80 0x55e156c4e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4de80 0x55e156c4e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54622/0 (socket says 172.21.15.12:54622) 2024-05-08T23:56:26.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4e7a0 0x55e156c483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c48910 0x55e156c48d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38762/0 (socket says 172.21.15.12:38762) 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.323+0000 7fcf22ffd700 1 -- 172.21.15.12:0/147325008 learned_addr learned my addr 172.21.15.12:0/147325008 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.324+0000 7fcf22ffd700 1 -- 172.21.15.12:0/147325008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4e7a0 msgr2=0x55e156c483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.324+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/147325008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4e7a0 0x55e156c483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.324+0000 7fcf22ffd700 1 -- 172.21.15.12:0/147325008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c48910 msgr2=0x55e156c48d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.324+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/147325008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c48910 0x55e156c48d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.324+0000 7fcf22ffd700 1 -- 172.21.15.12:0/147325008 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e156a19430 con 0x55e156c4de80 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.324+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/147325008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4de80 0x55e156c4e260 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00e4e0 tx=0x7fcf0c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39123eefd57bbf37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.324+0000 7fcf197fa700 1 -- 172.21.15.12:0/147325008 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00ee00 con 0x55e156c4de80 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.324+0000 7fcf197fa700 1 -- 172.21.15.12:0/147325008 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf0c021070 con 0x55e156c4de80 2024-05-08T23:56:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.324+0000 7fcf197fa700 1 -- 172.21.15.12:0/147325008 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c01d810 con 0x55e156c4de80 2024-05-08T23:56:26.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.325+0000 7fcf3556eb80 1 -- 172.21.15.12:0/147325008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4de80 msgr2=0x55e156c4e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.325+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/147325008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4de80 0x55e156c4e260 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00e4e0 tx=0x7fcf0c014000 comp rx=0 tx=0).stop 2024-05-08T23:56:26.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.325+0000 7fcf3556eb80 1 -- 172.21.15.12:0/147325008 shutdown_connections 2024-05-08T23:56:26.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.325+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/147325008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c48910 0x55e156c48d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.325+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/147325008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4de80 0x55e156c4e260 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.325+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/147325008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4e7a0 0x55e156c483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.325+0000 7fcf3556eb80 1 -- 172.21.15.12:0/147325008 >> 172.21.15.12:0/147325008 conn(0x55e156ba6290 msgr2=0x55e156c42e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.325+0000 7fcf3556eb80 1 -- 172.21.15.12:0/147325008 shutdown_connections 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.325+0000 7fcf3556eb80 1 -- 172.21.15.12:0/147325008 wait complete. 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.325+0000 7fcf3556eb80 1 -- start start 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf3556eb80 1 Processor -- start 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 0x55e156c51c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c52150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 0x55e156c52690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156b345e0 con 0x55e156c4de80 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156b34280 con 0x55e156c4e7a0 2024-05-08T23:56:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156a1b720 con 0x55e156c48910 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf3556eb80 10 client.0 _ll_register_callbacks cb 0x55e156b3dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c52150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c52150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38774/0 (socket says 172.21.15.12:38774) 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf227fc700 1 -- 172.21.15.12:0/3491841062 learned_addr learned my addr 172.21.15.12:0/3491841062 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf237fe700 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 0x55e156c52690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 0x55e156c51c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf227fc700 1 -- 172.21.15.12:0/3491841062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 msgr2=0x55e156c51c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf227fc700 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 0x55e156c51c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf227fc700 1 -- 172.21.15.12:0/3491841062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 msgr2=0x55e156c52690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf227fc700 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 0x55e156c52690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf227fc700 1 -- 172.21.15.12:0/3491841062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c017070 con 0x55e156c4de80 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf227fc700 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c52150 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fcf140126b0 tx=0x7fcf140129c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2e4464725fe32b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.326+0000 7fcf0a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/3491841062 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf14017070 con 0x55e156c4de80 2024-05-08T23:56:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/3491841062 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcf14010430 con 0x55e156c4de80 2024-05-08T23:56:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf3556eb80 1 -- 172.21.15.12:0/3491841062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e156af12f0 con 0x55e156c4de80 2024-05-08T23:56:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/3491841062 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf140105f0 con 0x55e156c4de80 2024-05-08T23:56:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf3556eb80 10 client.18285 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf3556eb80 1 -- 172.21.15.12:0/3491841062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55e1569c5be0 con 0x55e156c4de80 2024-05-08T23:56:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/3491841062 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e156af12f0 con 0x55e156c4de80 2024-05-08T23:56:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/3491841062 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fcf1401c890 con 0x55e156c4de80 2024-05-08T23:56:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf3556eb80 10 client.18285 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf3556eb80 10 client.18285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf3556eb80 20 client.18285 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e156c62400 0x55e156c64820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf3556eb80 4 client.18285 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.327+0000 7fcf3556eb80 1 -- 172.21.15.12:0/3491841062 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e156c17890 con 0x55e156c62400 2024-05-08T23:56:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.328+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e156c62400 0x55e156c64820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.328+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e156c62400 0x55e156c64820 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea44d7a0f279a30 server_cookie=c79636b98f701b56 in_seq=0 out_seq=0 2024-05-08T23:56:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.328+0000 7fcf0a7fc700 10 client.18285 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/3491841062 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e156c17890 con 0x55e156c62400 2024-05-08T23:56:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf0a7fc700 10 client.18285 handle_command_reply: tid=0 2024-05-08T23:56:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf3556eb80 1 client.18285 shutdown 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf3556eb80 1 -- 172.21.15.12:0/3491841062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 msgr2=0x55e156c52150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c52150 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fcf140126b0 tx=0x7fcf140129c0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf3556eb80 1 -- 172.21.15.12:0/3491841062 shutdown_connections 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e156c62400 0x55e156c64820 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 0x55e156c52690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 0x55e156c51c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/3491841062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c52150 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf0a7fc700 0 client.18285 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.330+0000 7fcf3556eb80 1 -- 172.21.15.12:0/3491841062 >> 172.21.15.12:0/3491841062 conn(0x55e156ba6290 msgr2=0x55e156c42620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.331+0000 7fcf3556eb80 1 -- 172.21.15.12:0/3491841062 shutdown_connections 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.331+0000 7fcf3556eb80 1 -- 172.21.15.12:0/3491841062 wait complete. 2024-05-08T23:56:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.331+0000 7fcf3556eb80 20 client.18285 trim_cache size 0 max 16384 2024-05-08T23:56:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.350+0000 7fcf3556eb80 1 Processor -- start 2024-05-08T23:56:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.350+0000 7fcf3556eb80 1 -- start start 2024-05-08T23:56:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c48910 0x55e156c4d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4de80 0x55e156c50b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4e7a0 0x55e156c42da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156c53280 con 0x55e156c4e7a0 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156c492b0 con 0x55e156c48910 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156c5fa50 con 0x55e156c4de80 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4de80 0x55e156c50b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c48910 0x55e156c4d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4e7a0 0x55e156c42da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4de80 0x55e156c50b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:56722/0 (socket says 172.21.15.12:56722) 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf227fc700 1 -- 172.21.15.12:0/943907176 learned_addr learned my addr 172.21.15.12:0/943907176 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf237fe700 1 -- 172.21.15.12:0/943907176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4de80 msgr2=0x55e156c50b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf237fe700 1 --2- 172.21.15.12:0/943907176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4de80 0x55e156c50b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf237fe700 1 -- 172.21.15.12:0/943907176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c48910 msgr2=0x55e156c4d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf237fe700 1 --2- 172.21.15.12:0/943907176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c48910 0x55e156c4d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf237fe700 1 -- 172.21.15.12:0/943907176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e156b5e5e0 con 0x55e156c4e7a0 2024-05-08T23:56:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf227fc700 1 --2- 172.21.15.12:0/943907176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4de80 0x55e156c50b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf237fe700 1 --2- 172.21.15.12:0/943907176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4e7a0 0x55e156c42da0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0cb1a0 tx=0x7fcf1c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb22bb875e045f68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.351+0000 7fcf19ffb700 1 -- 172.21.15.12:0/943907176 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c09abf0 con 0x55e156c4e7a0 2024-05-08T23:56:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.352+0000 7fcf19ffb700 1 -- 172.21.15.12:0/943907176 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf1c09ea20 con 0x55e156c4e7a0 2024-05-08T23:56:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.352+0000 7fcf19ffb700 1 -- 172.21.15.12:0/943907176 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c09b0b0 con 0x55e156c4e7a0 2024-05-08T23:56:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.352+0000 7fcf3556eb80 1 -- 172.21.15.12:0/943907176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4e7a0 msgr2=0x55e156c42da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.352+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/943907176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4e7a0 0x55e156c42da0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0cb1a0 tx=0x7fcf1c0099e0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.352+0000 7fcf3556eb80 1 -- 172.21.15.12:0/943907176 shutdown_connections 2024-05-08T23:56:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.352+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/943907176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4e7a0 0x55e156c42da0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.352+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/943907176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c48910 0x55e156c4d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.352+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/943907176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c4de80 0x55e156c50b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.352+0000 7fcf3556eb80 1 -- 172.21.15.12:0/943907176 >> 172.21.15.12:0/943907176 conn(0x55e156ba6290 msgr2=0x55e156c4eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 1 -- 172.21.15.12:0/943907176 shutdown_connections 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 1 -- 172.21.15.12:0/943907176 wait complete. 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 1 -- start start 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 1 Processor -- start 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 0x55e156c58700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c58c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 0x55e156cb28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156cb2e90 con 0x55e156c4de80 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156cb3000 con 0x55e156c4e7a0 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e156cb31c0 con 0x55e156c48910 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf3556eb80 10 client.0 _ll_register_callbacks cb 0x55e156b3dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c58c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf227fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c58c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38792/0 (socket says 172.21.15.12:38792) 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf227fc700 1 -- 172.21.15.12:0/4009794383 learned_addr learned my addr 172.21.15.12:0/4009794383 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 0x55e156c58700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.353+0000 7fcf237fe700 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 0x55e156cb28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf22ffd700 1 -- 172.21.15.12:0/4009794383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 msgr2=0x55e156cb28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 0x55e156cb28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf22ffd700 1 -- 172.21.15.12:0/4009794383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 msgr2=0x55e156c58c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c58c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf22ffd700 1 -- 172.21.15.12:0/4009794383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1569c5be0 con 0x55e156c48910 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf237fe700 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 0x55e156cb28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf22ffd700 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 0x55e156c58700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c011ab0 tx=0x7fcf0c005ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=665cf8f6aa47cbff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf0a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/4009794383 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00ec40 con 0x55e156c48910 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/4009794383 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcf0c0050d0 con 0x55e156c48910 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.354+0000 7fcf3556eb80 1 -- 172.21.15.12:0/4009794383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf0c021070 con 0x55e156c48910 2024-05-08T23:56:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/4009794383 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0c005290 con 0x55e156c48910 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf3556eb80 10 client.28033 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf3556eb80 1 -- 172.21.15.12:0/4009794383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7fcf241a4540 con 0x55e156c48910 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/4009794383 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf0c021070 con 0x55e156c48910 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/4009794383 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fcf0c03a060 con 0x55e156c48910 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf3556eb80 10 client.28033 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf3556eb80 10 client.28033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf3556eb80 20 client.28033 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e156cbaaa0 0x55e156cbcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf3556eb80 4 client.28033 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf3556eb80 1 -- 172.21.15.12:0/4009794383 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e156b90f40 con 0x55e156cbaaa0 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.355+0000 7fcf227fc700 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e156cbaaa0 0x55e156cbcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.356+0000 7fcf0a7fc700 10 client.28033 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.356+0000 7fcf227fc700 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e156cbaaa0 0x55e156cbcec0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2060f258f835523 server_cookie=c418f4ca747dbbac in_seq=0 out_seq=0 2024-05-08T23:56:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf0a7fc700 1 -- 172.21.15.12:0/4009794383 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e156b90f40 con 0x55e156cbaaa0 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf0a7fc700 10 client.28033 handle_command_reply: tid=0 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 client.28033 shutdown 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 -- 172.21.15.12:0/4009794383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 msgr2=0x55e156c58700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 0x55e156c58700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c011ab0 tx=0x7fcf0c005ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 -- 172.21.15.12:0/4009794383 shutdown_connections 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e156cbaaa0 0x55e156cbcec0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e156c4e7a0 0x55e156cb28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf0a7fc700 0 client.28033 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e156c48910 0x55e156c58700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 --2- 172.21.15.12:0/4009794383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e156c4de80 0x55e156c58c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 -- 172.21.15.12:0/4009794383 >> 172.21.15.12:0/4009794383 conn(0x55e156ba6290 msgr2=0x55e156c4f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 -- 172.21.15.12:0/4009794383 shutdown_connections 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 1 -- 172.21.15.12:0/4009794383 wait complete. 2024-05-08T23:56:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.357+0000 7fcf3556eb80 20 client.28033 trim_cache size 0 max 16384 2024-05-08T23:56:26.357 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:56:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:56:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "4c82b97d-f624-4ddc-ad6c-5d17e435e684", 2024-05-08T23:56:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:56:26.357 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:56:26.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 -- 172.21.15.12:0/960964620 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf100a1290 msgr2=0x7fcf100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf100a1290 0x7fcf100a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009c60 tx=0x7fcf0c009260 comp rx=0 tx=0).stop 2024-05-08T23:56:26.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 -- 172.21.15.12:0/960964620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2419be10 msgr2=0x7fcf2419c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2419be10 0x7fcf2419c290 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00b070 tx=0x7fcf1c00bda0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 -- 172.21.15.12:0/960964620 shutdown_connections 2024-05-08T23:56:26.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf100a1290 0x7fcf100a3750 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf2419be10 0x7fcf2419c290 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf24101160 0x7fcf24197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 --2- 172.21.15.12:0/960964620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf24106280 0x7fcf24197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 -- 172.21.15.12:0/960964620 >> 172.21.15.12:0/960964620 conn(0x7fcf240fc9e0 msgr2=0x7fcf240fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 -- 172.21.15.12:0/960964620 shutdown_connections 2024-05-08T23:56:26.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.358+0000 7fcf293b4700 1 -- 172.21.15.12:0/960964620 wait complete. 2024-05-08T23:56:26.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:56:26.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.518+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/1028836790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b28069d90 msgr2=0x7f6b28105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.518+0000 7f6b2c8ac700 1 --2- 172.21.15.12:0/1028836790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b28069d90 0x7f6b28105b70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009990 tx=0x7f6b10009ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.519+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/1028836790 shutdown_connections 2024-05-08T23:56:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.519+0000 7f6b2c8ac700 1 --2- 172.21.15.12:0/1028836790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b28106250 0x7f6b2810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.519+0000 7f6b2c8ac700 1 --2- 172.21.15.12:0/1028836790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b280693e0 0x7f6b280697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.519+0000 7f6b2c8ac700 1 --2- 172.21.15.12:0/1028836790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b28069d90 0x7f6b28105b70 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.519+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/1028836790 >> 172.21.15.12:0/1028836790 conn(0x7f6b28076ba0 msgr2=0x7f6b28076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.519+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/1028836790 shutdown_connections 2024-05-08T23:56:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.519+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/1028836790 wait complete. 2024-05-08T23:56:26.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b2c8ac700 1 Processor -- start 2024-05-08T23:56:26.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b2c8ac700 1 -- start start 2024-05-08T23:56:26.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b2c8ac700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b280693e0 0x7f6b281001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b2c8ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b28106250 0x7f6b28100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b2c8ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b28100c40 0x7f6b281a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b2c8ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281a5c20 con 0x7f6b28106250 2024-05-08T23:56:26.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b2c8ac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281a5d90 con 0x7f6b28100c40 2024-05-08T23:56:26.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b2c8ac700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281a5f50 con 0x7f6b280693e0 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b28106250 0x7f6b28100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b28100c40 0x7f6b281a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b28106250 0x7f6b28100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38796/0 (socket says 172.21.15.12:38796) 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b28100c40 0x7f6b281a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54660/0 (socket says 172.21.15.12:54660) 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b25d9b700 1 -- 172.21.15.12:0/4090562265 learned_addr learned my addr 172.21.15.12:0/4090562265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.520+0000 7f6b2659c700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b280693e0 0x7f6b281001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b26d9d700 1 -- 172.21.15.12:0/4090562265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b280693e0 msgr2=0x7f6b281001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b26d9d700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b280693e0 0x7f6b281001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b26d9d700 1 -- 172.21.15.12:0/4090562265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b28106250 msgr2=0x7f6b28100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b26d9d700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b28106250 0x7f6b28100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b26d9d700 1 -- 172.21.15.12:0/4090562265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100095f0 con 0x7f6b28100c40 2024-05-08T23:56:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b26d9d700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b28100c40 0x7f6b281a5650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000efc0 tx=0x7f6b2000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b1f7fe700 1 -- 172.21.15.12:0/4090562265 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20018070 con 0x7f6b28100c40 2024-05-08T23:56:26.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b1f7fe700 1 -- 172.21.15.12:0/4090562265 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b2000f040 con 0x7f6b28100c40 2024-05-08T23:56:26.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/4090562265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b281a61d0 con 0x7f6b28100c40 2024-05-08T23:56:26.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b1f7fe700 1 -- 172.21.15.12:0/4090562265 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000cd40 con 0x7f6b28100c40 2024-05-08T23:56:26.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.521+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/4090562265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b281a66e0 con 0x7f6b28100c40 2024-05-08T23:56:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.524+0000 7f6b1f7fe700 1 -- 172.21.15.12:0/4090562265 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6b20003a00 con 0x7f6b28100c40 2024-05-08T23:56:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.524+0000 7f6b1f7fe700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b140a1290 0x7f6b140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.524+0000 7f6b2659c700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b140a1290 0x7f6b140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.524+0000 7f6b1f7fe700 1 -- 172.21.15.12:0/4090562265 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6b200cc790 con 0x7f6b28100c40 2024-05-08T23:56:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.525+0000 7f6b2659c700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b140a1290 0x7f6b140a3750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009c60 tx=0x7f6b18009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.525+0000 7f6b38a66b80 1 Processor -- start 2024-05-08T23:56:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.525+0000 7f6b38a66b80 1 -- start start 2024-05-08T23:56:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.525+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 0x560677fa71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.525+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa7720 0x560677fa1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.525+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677fa1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.525+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560677ef8150 con 0x560677fa1890 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.525+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560677ef5b10 con 0x560677fa7720 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.525+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560677ef1400 con 0x560677fa6e00 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677fa1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677fa1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38806/0 (socket says 172.21.15.12:38806) 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa7720 0x560677fa1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b26d9d700 1 -- 172.21.15.12:0/1155745893 learned_addr learned my addr 172.21.15.12:0/1155745893 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b2659c700 1 --2- 172.21.15.12:0/1155745893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 0x560677fa71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b2659c700 1 -- 172.21.15.12:0/1155745893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa7720 msgr2=0x560677fa1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b2659c700 1 --2- 172.21.15.12:0/1155745893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa7720 0x560677fa1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b2659c700 1 -- 172.21.15.12:0/1155745893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 msgr2=0x560677fa1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b2659c700 1 --2- 172.21.15.12:0/1155745893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677fa1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b2659c700 1 -- 172.21.15.12:0/1155745893 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560677d723b0 con 0x560677fa6e00 2024-05-08T23:56:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b2659c700 1 --2- 172.21.15.12:0/1155745893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 0x560677fa71e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800e3b0 tx=0x7f6b18014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9cd775a797b2f26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b1cff9700 1 -- 172.21.15.12:0/1155745893 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800ed50 con 0x560677fa6e00 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b1cff9700 1 -- 172.21.15.12:0/1155745893 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b18021070 con 0x560677fa6e00 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.526+0000 7f6b1cff9700 1 -- 172.21.15.12:0/1155745893 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18003ea0 con 0x560677fa6e00 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.527+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1155745893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 msgr2=0x560677fa71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.527+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/1155745893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 0x560677fa71e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800e3b0 tx=0x7f6b18014000 comp rx=0 tx=0).stop 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.527+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1155745893 shutdown_connections 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.527+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/1155745893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677fa1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.527+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/1155745893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 0x560677fa71e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.527+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/1155745893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa7720 0x560677fa1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.527+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1155745893 >> 172.21.15.12:0/1155745893 conn(0x560677eff210 msgr2=0x560677f9bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.527+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1155745893 shutdown_connections 2024-05-08T23:56:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.527+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1155745893 wait complete. 2024-05-08T23:56:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b38a66b80 1 -- start start 2024-05-08T23:56:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b38a66b80 1 Processor -- start 2024-05-08T23:56:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 0x560677faabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 0x560677fab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa7720 0x560677fab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560677e8d560 con 0x560677fa7720 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560677e8d200 con 0x560677fa1890 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560677d746a0 con 0x560677fa6e00 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b38a66b80 10 client.0 _ll_register_callbacks cb 0x560677e96b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa7720 0x560677fab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa7720 0x560677fab670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38816/0 (socket says 172.21.15.12:38816) 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b2659c700 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 0x560677faabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b25d9b700 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 0x560677fab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b26d9d700 1 -- 172.21.15.12:0/789129592 learned_addr learned my addr 172.21.15.12:0/789129592 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b2659c700 1 -- 172.21.15.12:0/789129592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 msgr2=0x560677fab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b2659c700 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 0x560677fab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b2659c700 1 -- 172.21.15.12:0/789129592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa7720 msgr2=0x560677fab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.528+0000 7f6b2659c700 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa7720 0x560677fab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.529+0000 7f6b2659c700 1 -- 172.21.15.12:0/789129592 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b18017070 con 0x560677fa1890 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.529+0000 7f6b2659c700 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 0x560677faabf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800e9a0 tx=0x7f6b180037a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597b71cc08ebab5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.529+0000 7f6b0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.529+0000 7f6b0dffb700 1 -- 172.21.15.12:0/789129592 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1802a040 con 0x560677fa1890 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.529+0000 7f6b38a66b80 1 -- 172.21.15.12:0/789129592 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560677e4a270 con 0x560677fa1890 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.529+0000 7f6b0dffb700 1 -- 172.21.15.12:0/789129592 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6b18003e30 con 0x560677fa1890 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.529+0000 7f6b0dffb700 1 -- 172.21.15.12:0/789129592 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b180052c0 con 0x560677fa1890 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.529+0000 7f6b38a66b80 10 client.28055 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.529+0000 7f6b38a66b80 1 -- 172.21.15.12:0/789129592 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x560677d1eb60 con 0x560677fa1890 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.529+0000 7f6b0dffb700 1 -- 172.21.15.12:0/789129592 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560677e4a270 con 0x560677fa1890 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.530+0000 7f6b0dffb700 1 -- 172.21.15.12:0/789129592 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f6b18026cc0 con 0x560677fa1890 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.530+0000 7f6b38a66b80 10 client.28055 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.530+0000 7f6b38a66b80 10 client.28055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.530+0000 7f6b38a66b80 20 client.28055 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.530+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560677fbb180 0x560677fbd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.530+0000 7f6b38a66b80 4 client.28055 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.530+0000 7f6b38a66b80 1 -- 172.21.15.12:0/789129592 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560677f70810 con 0x560677fbb180 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.530+0000 7f6b25d9b700 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560677fbb180 0x560677fbd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.530+0000 7f6b0dffb700 10 client.28055 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.530+0000 7f6b25d9b700 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560677fbb180 0x560677fbd5a0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e3553b88545d64 server_cookie=ddd929de4138b8c in_seq=0 out_seq=0 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b0dffb700 1 -- 172.21.15.12:0/789129592 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560677f70810 con 0x560677fbb180 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b0dffb700 10 client.28055 handle_command_reply: tid=0 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b38a66b80 1 client.28055 shutdown 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b38a66b80 1 -- 172.21.15.12:0/789129592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 msgr2=0x560677faabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 0x560677faabf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800e9a0 tx=0x7f6b180037a0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b38a66b80 1 -- 172.21.15.12:0/789129592 shutdown_connections 2024-05-08T23:56:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa7720 0x560677fab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560677fbb180 0x560677fbd5a0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 0x560677faabf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/789129592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa6e00 0x560677fab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b0dffb700 0 client.28055 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.531+0000 7f6b38a66b80 1 -- 172.21.15.12:0/789129592 >> 172.21.15.12:0/789129592 conn(0x560677eff210 msgr2=0x560677f9b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.532+0000 7f6b38a66b80 1 -- 172.21.15.12:0/789129592 shutdown_connections 2024-05-08T23:56:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.532+0000 7f6b38a66b80 1 -- 172.21.15.12:0/789129592 wait complete. 2024-05-08T23:56:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.532+0000 7f6b38a66b80 20 client.28055 trim_cache size 0 max 16384 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b38a66b80 1 Processor -- start 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b38a66b80 1 -- start start 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 0x560677fa63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa6e00 0x560677f9bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa7720 0x560677fab490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560677f94080 con 0x560677fa6e00 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560677fa9140 con 0x560677fa1890 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560677fa2230 con 0x560677fa7720 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa6e00 0x560677f9bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa6e00 0x560677f9bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38824/0 (socket says 172.21.15.12:38824) 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.550+0000 7f6b25d9b700 1 -- 172.21.15.12:0/2899058148 learned_addr learned my addr 172.21.15.12:0/2899058148 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.551+0000 7f6b2659c700 1 --2- 172.21.15.12:0/2899058148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 0x560677fa63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.551+0000 7f6b25d9b700 1 -- 172.21.15.12:0/2899058148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa7720 msgr2=0x560677fab490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.551+0000 7f6b25d9b700 1 --2- 172.21.15.12:0/2899058148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa7720 0x560677fab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.551+0000 7f6b25d9b700 1 -- 172.21.15.12:0/2899058148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 msgr2=0x560677fa63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.551+0000 7f6b25d9b700 1 --2- 172.21.15.12:0/2899058148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 0x560677fa63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.551+0000 7f6b25d9b700 1 -- 172.21.15.12:0/2899058148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560677eb7560 con 0x560677fa6e00 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.551+0000 7f6b25d9b700 1 --2- 172.21.15.12:0/2899058148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa6e00 0x560677f9bd20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6b100098e0 tx=0x7f6b10005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ba3d438d4f30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.551+0000 7f6b1d7fa700 1 -- 172.21.15.12:0/2899058148 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1002a070 con 0x560677fa6e00 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.551+0000 7f6b1d7fa700 1 -- 172.21.15.12:0/2899058148 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1002e070 con 0x560677fa6e00 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.551+0000 7f6b1d7fa700 1 -- 172.21.15.12:0/2899058148 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1001d040 con 0x560677fa6e00 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 -- 172.21.15.12:0/2899058148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa6e00 msgr2=0x560677f9bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/2899058148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa6e00 0x560677f9bd20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6b100098e0 tx=0x7f6b10005960 comp rx=0 tx=0).stop 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 -- 172.21.15.12:0/2899058148 shutdown_connections 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/2899058148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa7720 0x560677fab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/2899058148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa1890 0x560677fa63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/2899058148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa6e00 0x560677f9bd20 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 -- 172.21.15.12:0/2899058148 >> 172.21.15.12:0/2899058148 conn(0x560677eff210 msgr2=0x560677fa5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 -- 172.21.15.12:0/2899058148 shutdown_connections 2024-05-08T23:56:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 -- 172.21.15.12:0/2899058148 wait complete. 2024-05-08T23:56:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 -- start start 2024-05-08T23:56:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.552+0000 7f6b38a66b80 1 Processor -- start 2024-05-08T23:56:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677faffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa6e00 0x560677fb0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b38a66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa7720 0x560678010680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560677fb0e20 con 0x560677fa1890 2024-05-08T23:56:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560678010bc0 con 0x560677fa6e00 2024-05-08T23:56:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b38a66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560678010d00 con 0x560677fa7720 2024-05-08T23:56:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b38a66b80 10 client.0 _ll_register_callbacks cb 0x560677e96b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677faffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677faffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38840/0 (socket says 172.21.15.12:38840) 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b2659c700 1 -- 172.21.15.12:0/1535150925 learned_addr learned my addr 172.21.15.12:0/1535150925 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b26d9d700 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa7720 0x560678010680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b25d9b700 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa6e00 0x560677fb0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b26d9d700 1 -- 172.21.15.12:0/1535150925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa6e00 msgr2=0x560677fb0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b26d9d700 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa6e00 0x560677fb0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b26d9d700 1 -- 172.21.15.12:0/1535150925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 msgr2=0x560677faffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b26d9d700 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677faffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b26d9d700 1 -- 172.21.15.12:0/1535150925 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2810e680 con 0x560677fa7720 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b2659c700 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677faffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b26d9d700 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa7720 0x560678010680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000c6f0 tx=0x7f6b200994b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bdcaa75b9056f68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.553+0000 7f6b0dffb700 1 -- 172.21.15.12:0/1535150925 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20004ad0 con 0x560677fa7720 2024-05-08T23:56:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1535150925 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b18021070 con 0x560677fa7720 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b0dffb700 1 -- 172.21.15.12:0/1535150925 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6b2009a070 con 0x560677fa7720 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b0dffb700 1 -- 172.21.15.12:0/1535150925 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b20099970 con 0x560677fa7720 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b38a66b80 10 client.28048 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1535150925 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x560677d76cd0 con 0x560677fa7720 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b0dffb700 1 -- 172.21.15.12:0/1535150925 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b20063020 con 0x560677fa7720 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b0dffb700 1 -- 172.21.15.12:0/1535150925 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f6b200a18d0 con 0x560677fa7720 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b38a66b80 10 client.28048 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b38a66b80 10 client.28048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b38a66b80 20 client.28048 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560678012540 0x560678014960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b38a66b80 4 client.28048 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1535150925 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560677ee9ec0 con 0x560678012540 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.554+0000 7f6b2659c700 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560678012540 0x560678014960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b0dffb700 10 client.28048 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b2659c700 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560678012540 0x560678014960 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c868cc078ed8cd5 server_cookie=200d5a9319360bab in_seq=0 out_seq=0 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b0dffb700 1 -- 172.21.15.12:0/1535150925 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560677ee9ec0 con 0x560678012540 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b0dffb700 10 client.28048 handle_command_reply: tid=0 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 client.28048 shutdown 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1535150925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa7720 msgr2=0x560678010680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa7720 0x560678010680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000c6f0 tx=0x7f6b200994b0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1535150925 shutdown_connections 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560677fa7720 0x560678010680 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560677fa1890 0x560677faffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560678012540 0x560678014960 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 --2- 172.21.15.12:0/1535150925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560677fa6e00 0x560677fb0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b0dffb700 0 client.28048 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1535150925 >> 172.21.15.12:0/1535150925 conn(0x560677eff210 msgr2=0x560677fab4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1535150925 shutdown_connections 2024-05-08T23:56:26.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 1 -- 172.21.15.12:0/1535150925 wait complete. 2024-05-08T23:56:26.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.555+0000 7f6b38a66b80 20 client.28048 trim_cache size 0 max 16384 2024-05-08T23:56:26.557 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:56:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:56:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:56:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "4c82b97d-f624-4ddc-ad6c-5d17e435e684": { 2024-05-08T23:56:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:56:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "4c82b97d-f624-4ddc-ad6c-5d17e435e684", 2024-05-08T23:56:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:56:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:56:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:56:26.558 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:56:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/4090562265 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b140a1290 msgr2=0x7f6b140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b140a1290 0x7f6b140a3750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009c60 tx=0x7f6b18009260 comp rx=0 tx=0).stop 2024-05-08T23:56:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/4090562265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b28100c40 msgr2=0x7f6b281a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b28100c40 0x7f6b281a5650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000efc0 tx=0x7f6b2000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/4090562265 shutdown_connections 2024-05-08T23:56:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b140a1290 0x7f6b140a3750 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b28100c40 0x7f6b281a5650 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b280693e0 0x7f6b281001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 --2- 172.21.15.12:0/4090562265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b28106250 0x7f6b28100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/4090562265 >> 172.21.15.12:0/4090562265 conn(0x7f6b28076ba0 msgr2=0x7f6b281073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/4090562265 shutdown_connections 2024-05-08T23:56:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:26.556+0000 7f6b2c8ac700 1 -- 172.21.15.12:0/4090562265 wait complete. 2024-05-08T23:56:26.564 INFO:tasks.cephfs.filesystem:scrub status for tag:4c82b97d-f624-4ddc-ad6c-5d17e435e684 - {'path': '/', 'tag': '4c82b97d-f624-4ddc-ad6c-5d17e435e684', 'options': 'recursive,force'} 2024-05-08T23:56:26.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 568577 6.71 MB/sec execute 382 sec latency 3065.845 ms 2024-05-08T23:56:26.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:26 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:56:26.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:26 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:26.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:26 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:56:26.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:26 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:27.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:26 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:56:27.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:26 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:27.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:26 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:56:27.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:26 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:27.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:26 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:56:27.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:26 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:27.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:26 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:56:27.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:26 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:27.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 569851 6.71 MB/sec execute 383 sec latency 15.249 ms 2024-05-08T23:56:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:27 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:56:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:27 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:56:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:27 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:27 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:27 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:56:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:27 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:27 smithi012 ceph-mon[42503]: pgmap v318: 97 pgs: 97 active+clean; 163 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-05-08T23:56:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:27 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:56:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:27 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:56:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:27 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:27 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:27 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:56:28.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:27 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:28.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:27 smithi204 ceph-mon[47373]: pgmap v318: 97 pgs: 97 active+clean; 163 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-05-08T23:56:28.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:27 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:56:28.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:27 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:56:28.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:27 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:28.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:27 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:28.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:27 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:56:28.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:27 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:28.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:27 smithi160 ceph-mon[46152]: pgmap v318: 97 pgs: 97 active+clean; 163 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-05-08T23:56:28.884 INFO:tasks.workunit.client.0.smithi012.stdout: 1 570036 6.69 MB/sec execute 384 sec latency 819.377 ms 2024-05-08T23:56:29.885 INFO:tasks.workunit.client.0.smithi012.stdout: 1 570036 6.67 MB/sec execute 385 sec latency 1819.484 ms 2024-05-08T23:56:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:29 smithi012 ceph-mon[42503]: pgmap v319: 97 pgs: 97 active+clean; 163 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-05-08T23:56:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:29 smithi204 ceph-mon[47373]: pgmap v319: 97 pgs: 97 active+clean; 163 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-05-08T23:56:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:29 smithi160 ceph-mon[46152]: pgmap v319: 97 pgs: 97 active+clean; 163 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-05-08T23:56:30.885 INFO:tasks.workunit.client.0.smithi012.stdout: 1 570036 6.66 MB/sec execute 386 sec latency 2819.565 ms 2024-05-08T23:56:31.885 INFO:tasks.workunit.client.0.smithi012.stdout: 1 572140 6.67 MB/sec execute 387 sec latency 2977.508 ms 2024-05-08T23:56:31.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:31 smithi012 ceph-mon[42503]: pgmap v320: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.0 MiB/s wr, 159 op/s 2024-05-08T23:56:32.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:31 smithi204 ceph-mon[47373]: pgmap v320: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.0 MiB/s wr, 159 op/s 2024-05-08T23:56:32.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:31 smithi160 ceph-mon[46152]: pgmap v320: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.0 MiB/s wr, 159 op/s 2024-05-08T23:56:32.885 INFO:tasks.workunit.client.0.smithi012.stdout: 1 573406 6.67 MB/sec execute 388 sec latency 20.954 ms 2024-05-08T23:56:33.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:33 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:56:33.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:33 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:56:33.885 INFO:tasks.workunit.client.0.smithi012.stdout: 1 573579 6.65 MB/sec execute 389 sec latency 795.196 ms 2024-05-08T23:56:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:33 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:56:34.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:34 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:56:34.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:34 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:56:34.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:34 smithi204 ceph-mon[47373]: pgmap v321: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-08T23:56:34.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:34 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:56:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:34 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:56:34.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:34 smithi160 ceph-mon[46152]: pgmap v321: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-08T23:56:34.885 INFO:tasks.workunit.client.0.smithi012.stdout: 1 573579 6.64 MB/sec execute 390 sec latency 1795.303 ms 2024-05-08T23:56:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:34 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:56:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:34 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:56:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:34 smithi012 ceph-mon[42503]: pgmap v321: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-08T23:56:35.885 INFO:tasks.workunit.client.0.smithi012.stdout: 1 573579 6.62 MB/sec execute 391 sec latency 2795.406 ms 2024-05-08T23:56:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:36 smithi204 ceph-mon[47373]: pgmap v322: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-08T23:56:36.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:36 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:56:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:36 smithi160 ceph-mon[46152]: pgmap v322: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-08T23:56:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:36 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:56:36.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 575549 6.64 MB/sec execute 392 sec latency 2943.905 ms 2024-05-08T23:56:36.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:36 smithi012 ceph-mon[42503]: pgmap v322: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-08T23:56:36.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:36 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:56:37.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 576818 6.64 MB/sec execute 393 sec latency 19.486 ms 2024-05-08T23:56:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.625+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2763955019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc1009c0 msgr2=0x7fa4bc104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.625+0000 7fa4c3d2b700 1 --2- 172.21.15.12:0/2763955019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc1009c0 0x7fa4bc104a00 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009990 tx=0x7fa4b0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.625+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2763955019 shutdown_connections 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.625+0000 7fa4c3d2b700 1 --2- 172.21.15.12:0/2763955019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4bc105130 0x7fa4bc10ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.625+0000 7fa4c3d2b700 1 --2- 172.21.15.12:0/2763955019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4bc100010 0x7fa4bc1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.625+0000 7fa4c3d2b700 1 --2- 172.21.15.12:0/2763955019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc1009c0 0x7fa4bc104a00 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.625+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2763955019 >> 172.21.15.12:0/2763955019 conn(0x7fa4bc076f40 msgr2=0x7fa4bc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.625+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2763955019 shutdown_connections 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.625+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2763955019 wait complete. 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c3d2b700 1 Processor -- start 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c3d2b700 1 -- start start 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c3d2b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4bc100010 0x7fa4bc197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c3d2b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4bc105130 0x7fa4bc1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c3d2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc19bc50 0x7fa4bc19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c3d2b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc19c6d0 con 0x7fa4bc19bc50 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c3d2b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc19c840 con 0x7fa4bc105130 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c3d2b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc19ca00 con 0x7fa4bc100010 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c22c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc19bc50 0x7fa4bc19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c22c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc19bc50 0x7fa4bc19c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45930/0 (socket says 172.21.15.12:45930) 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c22c8700 1 -- 172.21.15.12:0/2856141138 learned_addr learned my addr 172.21.15.12:0/2856141138 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.626+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4bc100010 0x7fa4bc197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4bc105130 0x7fa4bc1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4c22c8700 1 -- 172.21.15.12:0/2856141138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4bc100010 msgr2=0x7fa4bc197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4c22c8700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4bc100010 0x7fa4bc197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4c22c8700 1 -- 172.21.15.12:0/2856141138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4bc105130 msgr2=0x7fa4bc1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4c22c8700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4bc105130 0x7fa4bc1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4c22c8700 1 -- 172.21.15.12:0/2856141138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b00095f0 con 0x7fa4bc19bc50 2024-05-08T23:56:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4bc105130 0x7fa4bc1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4c22c8700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc19bc50 0x7fa4bc19c0d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4007ee0 tx=0x7fa4b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4aeffd700 1 -- 172.21.15.12:0/2856141138 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4017070 con 0x7fa4bc19bc50 2024-05-08T23:56:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4aeffd700 1 -- 172.21.15.12:0/2856141138 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b400f040 con 0x7fa4bc19bc50 2024-05-08T23:56:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2856141138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4bc1a3f60 con 0x7fa4bc19bc50 2024-05-08T23:56:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4aeffd700 1 -- 172.21.15.12:0/2856141138 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4005740 con 0x7fa4bc19bc50 2024-05-08T23:56:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.627+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2856141138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4bc1a4440 con 0x7fa4bc19bc50 2024-05-08T23:56:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.629+0000 7fa4aeffd700 1 -- 172.21.15.12:0/2856141138 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa4b400dc60 con 0x7fa4bc19bc50 2024-05-08T23:56:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.630+0000 7fa4aeffd700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa4a80a1310 0x7fa4a80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.630+0000 7fa4aeffd700 1 -- 172.21.15.12:0/2856141138 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fa4b40cc3a0 con 0x7fa4bc19bc50 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.630+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa4a80a1310 0x7fa4a80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.631+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa4a80a1310 0x7fa4a80a37d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009c60 tx=0x7fa4b8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.631+0000 7fa4cfee5b80 1 Processor -- start 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.631+0000 7fa4cfee5b80 1 -- start start 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603025950 0x555603025d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603026270 0x55560301ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603020480 0x5556030208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602f770f0 con 0x555603026270 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602f74ab0 con 0x555603020480 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602f703a0 con 0x555603025950 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c12c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603026270 0x55560301ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c12c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603026270 0x55560301ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45942/0 (socket says 172.21.15.12:45942) 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c12c6700 1 -- 172.21.15.12:0/793044664 learned_addr learned my addr 172.21.15.12:0/793044664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/793044664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603025950 0x555603025d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c22c8700 1 --2- 172.21.15.12:0/793044664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603020480 0x5556030208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c12c6700 1 -- 172.21.15.12:0/793044664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603025950 msgr2=0x555603025d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/793044664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603025950 0x555603025d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c12c6700 1 -- 172.21.15.12:0/793044664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603020480 msgr2=0x5556030208a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/793044664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603020480 0x5556030208a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c12c6700 1 -- 172.21.15.12:0/793044664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555602df1350 con 0x555603026270 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c22c8700 1 --2- 172.21.15.12:0/793044664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603020480 0x5556030208a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.632+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/793044664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603026270 0x55560301ff40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0006010 tx=0x7fa4b0005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21276b4ac14cc857 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4a7fff700 1 -- 172.21.15.12:0/793044664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00049e0 con 0x555603026270 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4a7fff700 1 -- 172.21.15.12:0/793044664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b001e040 con 0x555603026270 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4a7fff700 1 -- 172.21.15.12:0/793044664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b000b9d0 con 0x555603026270 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/793044664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603026270 msgr2=0x55560301ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/793044664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603026270 0x55560301ff40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0006010 tx=0x7fa4b0005c80 comp rx=0 tx=0).stop 2024-05-08T23:56:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/793044664 shutdown_connections 2024-05-08T23:56:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/793044664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603020480 0x5556030208a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/793044664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603025950 0x555603025d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/793044664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603026270 0x55560301ff40 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/793044664 >> 172.21.15.12:0/793044664 conn(0x555602f7e1b0 msgr2=0x55560301a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/793044664 shutdown_connections 2024-05-08T23:56:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.633+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/793044664 wait complete. 2024-05-08T23:56:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.634+0000 7fa4cfee5b80 1 -- start start 2024-05-08T23:56:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.634+0000 7fa4cfee5b80 1 Processor -- start 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.634+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x55560302c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.634+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603025950 0x555603029570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.634+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 0x555603029c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.634+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602f0c500 con 0x555603020480 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.634+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602f0c1a0 con 0x555603026270 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.634+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602df3640 con 0x555603025950 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4cfee5b80 10 client.0 _ll_register_callbacks cb 0x555602f15ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c1ac7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x55560302c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c1ac7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x55560302c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45952/0 (socket says 172.21.15.12:45952) 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c1ac7700 1 -- 172.21.15.12:0/2841421388 learned_addr learned my addr 172.21.15.12:0/2841421388 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603025950 0x555603029570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c22c8700 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 0x555603029c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c1ac7700 1 -- 172.21.15.12:0/2841421388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603025950 msgr2=0x555603029570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603025950 0x555603029570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c1ac7700 1 -- 172.21.15.12:0/2841421388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 msgr2=0x555603029c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 0x555603029c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c1ac7700 1 -- 172.21.15.12:0/2841421388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4bc10d520 con 0x555603020480 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c22c8700 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 0x555603029c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x55560302c280 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8005910 tx=0x7fa4b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994d3fb6d7f178b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4a57fa700 1 -- 172.21.15.12:0/2841421388 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8022070 con 0x555603020480 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4a57fa700 1 -- 172.21.15.12:0/2841421388 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fa4b801d820 con 0x555603020480 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/2841421388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555602ec9210 con 0x555603020480 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4a57fa700 1 -- 172.21.15.12:0/2841421388 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4b801d9e0 con 0x555603020480 2024-05-08T23:56:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.635+0000 7fa4cfee5b80 10 client.18321 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.636+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/2841421388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x555602d9db00 con 0x555603020480 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.636+0000 7fa4a57fa700 1 -- 172.21.15.12:0/2841421388 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555602ec9210 con 0x555603020480 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.636+0000 7fa4a57fa700 1 -- 172.21.15.12:0/2841421388 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fa4b8027b00 con 0x555603020480 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.636+0000 7fa4cfee5b80 10 client.18321 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.636+0000 7fa4cfee5b80 10 client.18321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.636+0000 7fa4cfee5b80 20 client.18321 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.636+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555603039ca0 0x55560303c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.636+0000 7fa4cfee5b80 4 client.18321 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.636+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/2841421388 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555602fef7b0 con 0x555603039ca0 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.636+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555603039ca0 0x55560303c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.637+0000 7fa4a57fa700 10 client.18321 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.637+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555603039ca0 0x55560303c0c0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=728650955611896f server_cookie=fc3139bb16c97786 in_seq=0 out_seq=0 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.637+0000 7fa4a57fa700 1 -- 172.21.15.12:0/2841421388 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555602fef7b0 con 0x555603039ca0 2024-05-08T23:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.637+0000 7fa4a57fa700 10 client.18321 handle_command_reply: tid=0 2024-05-08T23:56:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.637+0000 7fa4cfee5b80 1 client.18321 shutdown 2024-05-08T23:56:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.637+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/2841421388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 msgr2=0x55560302c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.637+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x55560302c280 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8005910 tx=0x7fa4b80047c0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.638+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/2841421388 shutdown_connections 2024-05-08T23:56:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.638+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555603039ca0 0x55560303c0c0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.638+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 0x555603029c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.638+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x55560302c280 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.638+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/2841421388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603025950 0x555603029570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.638+0000 7fa4a57fa700 0 client.18321 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.638+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/2841421388 >> 172.21.15.12:0/2841421388 conn(0x555602f7e1b0 msgr2=0x555603023460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.638+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/2841421388 shutdown_connections 2024-05-08T23:56:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.638+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/2841421388 wait complete. 2024-05-08T23:56:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.638+0000 7fa4cfee5b80 20 client.18321 trim_cache size 0 max 16384 2024-05-08T23:56:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4cfee5b80 1 Processor -- start 2024-05-08T23:56:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4cfee5b80 1 -- start start 2024-05-08T23:56:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x555603029be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603025950 0x55560301a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603026270 0x55560302bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560301c0f0 con 0x555603020480 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556030412d0 con 0x555603025950 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560302c810 con 0x555603026270 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4c1ac7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x555603029be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4c12c6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603025950 0x55560301a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4c1ac7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x555603029be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45966/0 (socket says 172.21.15.12:45966) 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4c1ac7700 1 -- 172.21.15.12:0/80570775 learned_addr learned my addr 172.21.15.12:0/80570775 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.656+0000 7fa4c22c8700 1 --2- 172.21.15.12:0/80570775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603026270 0x55560302bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.657+0000 7fa4c1ac7700 1 -- 172.21.15.12:0/80570775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603026270 msgr2=0x55560302bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.657+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/80570775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603026270 0x55560302bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.657+0000 7fa4c1ac7700 1 -- 172.21.15.12:0/80570775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603025950 msgr2=0x55560301a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.657+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/80570775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603025950 0x55560301a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.657+0000 7fa4c1ac7700 1 -- 172.21.15.12:0/80570775 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555602f36500 con 0x555603020480 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.657+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/80570775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603025950 0x55560301a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.657+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/80570775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x555603029be0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80058e0 tx=0x7fa4b8009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928eb646648925c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.657+0000 7fa4acff9700 1 -- 172.21.15.12:0/80570775 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8022070 con 0x555603020480 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.657+0000 7fa4acff9700 1 -- 172.21.15.12:0/80570775 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b801c460 con 0x555603020480 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.657+0000 7fa4acff9700 1 -- 172.21.15.12:0/80570775 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b801da50 con 0x555603020480 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/80570775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 msgr2=0x555603029be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/80570775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x555603029be0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80058e0 tx=0x7fa4b8009e80 comp rx=0 tx=0).stop 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/80570775 shutdown_connections 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/80570775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603026270 0x55560302bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/80570775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603020480 0x555603029be0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/80570775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603025950 0x55560301a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/80570775 >> 172.21.15.12:0/80570775 conn(0x555602f7e1b0 msgr2=0x555603035690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/80570775 shutdown_connections 2024-05-08T23:56:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/80570775 wait complete. 2024-05-08T23:56:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 -- start start 2024-05-08T23:56:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.658+0000 7fa4cfee5b80 1 Processor -- start 2024-05-08T23:56:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.659+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603020480 0x55560302ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.659+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603025950 0x55560302f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.659+0000 7fa4cfee5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 0x55560302f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.659+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556030935b0 con 0x555603025950 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.659+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556030936f0 con 0x555603026270 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.659+0000 7fa4cfee5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555603093830 con 0x555603020480 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.659+0000 7fa4cfee5b80 10 client.0 _ll_register_callbacks cb 0x555602f15ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.659+0000 7fa4c1ac7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603020480 0x55560302ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.659+0000 7fa4c1ac7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603020480 0x55560302ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52746/0 (socket says 172.21.15.12:52746) 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.659+0000 7fa4c1ac7700 1 -- 172.21.15.12:0/3967100902 learned_addr learned my addr 172.21.15.12:0/3967100902 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.660+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603025950 0x55560302f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.661+0000 7fa4c22c8700 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 0x55560302f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.661+0000 7fa4c12c6700 1 -- 172.21.15.12:0/3967100902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603020480 msgr2=0x55560302ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.661+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603020480 0x55560302ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.661+0000 7fa4c12c6700 1 -- 172.21.15.12:0/3967100902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 msgr2=0x55560302f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.661+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 0x55560302f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.661+0000 7fa4c12c6700 1 -- 172.21.15.12:0/3967100902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555602df1350 con 0x555603025950 2024-05-08T23:56:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.663+0000 7fa4c12c6700 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603025950 0x55560302f100 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0005b40 tx=0x7fa4b0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3095bcb6aadaa563 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.663+0000 7fa4a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.666+0000 7fa4a57fa700 1 -- 172.21.15.12:0/3967100902 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00300c0 con 0x555603025950 2024-05-08T23:56:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.666+0000 7fa4a57fa700 1 -- 172.21.15.12:0/3967100902 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fa4b0030220 con 0x555603025950 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.666+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/3967100902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4b001e040 con 0x555603025950 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.666+0000 7fa4a57fa700 1 -- 172.21.15.12:0/3967100902 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4b0030400 con 0x555603025950 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.666+0000 7fa4cfee5b80 10 client.18327 fetch_fsmap learned FSMap version 44 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.666+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/3967100902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x555602d9db00 con 0x555603025950 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.666+0000 7fa4a57fa700 1 -- 172.21.15.12:0/3967100902 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b001e040 con 0x555603025950 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.667+0000 7fa4a57fa700 1 -- 172.21.15.12:0/3967100902 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fa4b000fb00 con 0x555603025950 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.667+0000 7fa4cfee5b80 10 client.18327 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.667+0000 7fa4cfee5b80 10 client.18327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.667+0000 7fa4cfee5b80 20 client.18327 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.667+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555603090760 0x555603092b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.667+0000 7fa4cfee5b80 4 client.18327 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.667+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/3967100902 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555602f68e60 con 0x555603090760 2024-05-08T23:56:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.667+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555603090760 0x555603092b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.667+0000 7fa4c1ac7700 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555603090760 0x555603092b80 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b15935e5ea85f70 server_cookie=d6ed9639b576e in_seq=0 out_seq=0 2024-05-08T23:56:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.667+0000 7fa4a57fa700 10 client.18327 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: pgmap v323: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:56:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:56:38.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4a57fa700 1 -- 172.21.15.12:0/3967100902 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555602f68e60 con 0x555603090760 2024-05-08T23:56:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4a57fa700 10 client.18327 handle_command_reply: tid=0 2024-05-08T23:56:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 client.18327 shutdown 2024-05-08T23:56:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/3967100902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603025950 msgr2=0x55560302f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603025950 0x55560302f100 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0005b40 tx=0x7fa4b0005720 comp rx=0 tx=0).stop 2024-05-08T23:56:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/3967100902 shutdown_connections 2024-05-08T23:56:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555603090760 0x555603092b80 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555603026270 0x55560302f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555603020480 0x55560302ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4a57fa700 0 client.18327 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 --2- 172.21.15.12:0/3967100902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555603025950 0x55560302f100 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/3967100902 >> 172.21.15.12:0/3967100902 conn(0x555602f7e1b0 msgr2=0x55560301f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/3967100902 shutdown_connections 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 1 -- 172.21.15.12:0/3967100902 wait complete. 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.713+0000 7fa4cfee5b80 20 client.18327 trim_cache size 0 max 16384 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:56:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:56:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:56:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2856141138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa4a80a1310 msgr2=0x7fa4a80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa4a80a1310 0x7fa4a80a37d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009c60 tx=0x7fa4b8009260 comp rx=0 tx=0).stop 2024-05-08T23:56:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2856141138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc19bc50 msgr2=0x7fa4bc19c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc19bc50 0x7fa4bc19c0d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4007ee0 tx=0x7fa4b400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2856141138 shutdown_connections 2024-05-08T23:56:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa4a80a1310 0x7fa4a80a37d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc19bc50 0x7fa4bc19c0d0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4bc100010 0x7fa4bc197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 --2- 172.21.15.12:0/2856141138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4bc105130 0x7fa4bc1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2856141138 >> 172.21.15.12:0/2856141138 conn(0x7fa4bc076f40 msgr2=0x7fa4bc0694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2856141138 shutdown_connections 2024-05-08T23:56:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:38.714+0000 7fa4c3d2b700 1 -- 172.21.15.12:0/2856141138 wait complete. 2024-05-08T23:56:38.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: pgmap v323: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-08T23:56:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:56:38.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:56:38.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: pgmap v323: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:56:38.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:56:38.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:56:38.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:56:38.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 577126 6.62 MB/sec execute 394 sec latency 622.595 ms 2024-05-08T23:56:39.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:39 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi204 to 182.4M 2024-05-08T23:56:39.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:39 smithi204 ceph-mon[47373]: Unable to set osd_memory_target on smithi204 to 191345459: error parsing value: Value '191345459' is below minimum 939524096 2024-05-08T23:56:39.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:39 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-08T23:56:39.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:39 smithi204 ceph-mon[47373]: Unable to set osd_memory_target on smithi160 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:56:39.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:39 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi012 to 183.5M 2024-05-08T23:56:39.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:39 smithi204 ceph-mon[47373]: Unable to set osd_memory_target on smithi012 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-08T23:56:39.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:39 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi204 to 182.4M 2024-05-08T23:56:39.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:39 smithi160 ceph-mon[46152]: Unable to set osd_memory_target on smithi204 to 191345459: error parsing value: Value '191345459' is below minimum 939524096 2024-05-08T23:56:39.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:39 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-08T23:56:39.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:39 smithi160 ceph-mon[46152]: Unable to set osd_memory_target on smithi160 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:56:39.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:39 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi012 to 183.5M 2024-05-08T23:56:39.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:39 smithi160 ceph-mon[46152]: Unable to set osd_memory_target on smithi012 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-08T23:56:39.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 577126 6.60 MB/sec execute 395 sec latency 1622.710 ms 2024-05-08T23:56:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:39 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi204 to 182.4M 2024-05-08T23:56:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:39 smithi012 ceph-mon[42503]: Unable to set osd_memory_target on smithi204 to 191345459: error parsing value: Value '191345459' is below minimum 939524096 2024-05-08T23:56:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:39 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-08T23:56:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:39 smithi012 ceph-mon[42503]: Unable to set osd_memory_target on smithi160 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:56:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:39 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi012 to 183.5M 2024-05-08T23:56:39.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:39 smithi012 ceph-mon[42503]: Unable to set osd_memory_target on smithi012 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-08T23:56:40.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:40 smithi204 ceph-mon[47373]: pgmap v324: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-05-08T23:56:40.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:40 smithi160 ceph-mon[46152]: pgmap v324: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-05-08T23:56:40.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 577126 6.59 MB/sec execute 396 sec latency 2622.794 ms 2024-05-08T23:56:40.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:40 smithi012 ceph-mon[42503]: pgmap v324: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-05-08T23:56:41.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 579232 6.60 MB/sec execute 397 sec latency 2761.100 ms 2024-05-08T23:56:41.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:41 smithi012 ceph-mon[42503]: pgmap v325: 97 pgs: 97 active+clean; 168 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-05-08T23:56:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:41 smithi204 ceph-mon[47373]: pgmap v325: 97 pgs: 97 active+clean; 168 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-05-08T23:56:42.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:41 smithi160 ceph-mon[46152]: pgmap v325: 97 pgs: 97 active+clean; 168 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-05-08T23:56:42.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 580483 6.60 MB/sec execute 398 sec latency 18.913 ms 2024-05-08T23:56:43.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 580669 6.58 MB/sec execute 399 sec latency 778.931 ms 2024-05-08T23:56:43.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:43 smithi012 ceph-mon[42503]: pgmap v326: 97 pgs: 97 active+clean; 168 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:56:44.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:43 smithi204 ceph-mon[47373]: pgmap v326: 97 pgs: 97 active+clean; 168 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:56:44.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:43 smithi160 ceph-mon[46152]: pgmap v326: 97 pgs: 97 active+clean; 168 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:56:44.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 580669 6.57 MB/sec execute 400 sec latency 1779.016 ms 2024-05-08T23:56:45.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 580669 6.55 MB/sec execute 401 sec latency 2779.138 ms 2024-05-08T23:56:45.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:45 smithi012 ceph-mon[42503]: pgmap v327: 97 pgs: 97 active+clean; 168 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:56:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:45 smithi204 ceph-mon[47373]: pgmap v327: 97 pgs: 97 active+clean; 168 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:56:46.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:45 smithi160 ceph-mon[46152]: pgmap v327: 97 pgs: 97 active+clean; 168 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:56:46.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 582781 6.57 MB/sec execute 402 sec latency 2923.799 ms 2024-05-08T23:56:47.886 INFO:tasks.workunit.client.0.smithi012.stdout: 1 584062 6.57 MB/sec execute 403 sec latency 20.374 ms 2024-05-08T23:56:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:47 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:56:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:47 smithi012 ceph-mon[42503]: pgmap v328: 97 pgs: 97 active+clean; 171 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.7 MiB/s wr, 157 op/s 2024-05-08T23:56:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:47 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:56:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:47 smithi204 ceph-mon[47373]: pgmap v328: 97 pgs: 97 active+clean; 171 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.7 MiB/s wr, 157 op/s 2024-05-08T23:56:48.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:47 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:56:48.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:47 smithi160 ceph-mon[46152]: pgmap v328: 97 pgs: 97 active+clean; 171 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.7 MiB/s wr, 157 op/s 2024-05-08T23:56:48.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 584222 6.55 MB/sec execute 404 sec latency 844.459 ms 2024-05-08T23:56:49.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 584222 6.53 MB/sec execute 405 sec latency 1844.560 ms 2024-05-08T23:56:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:49 smithi012 ceph-mon[42503]: pgmap v329: 97 pgs: 97 active+clean; 171 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-05-08T23:56:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:49 smithi204 ceph-mon[47373]: pgmap v329: 97 pgs: 97 active+clean; 171 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-05-08T23:56:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:49 smithi160 ceph-mon[46152]: pgmap v329: 97 pgs: 97 active+clean; 171 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-05-08T23:56:50.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 584222 6.52 MB/sec execute 406 sec latency 2844.644 ms 2024-05-08T23:56:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.852+0000 7f7f9a376700 1 -- 172.21.15.12:0/3682530951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f94106250 msgr2=0x7f7f9410ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.852+0000 7f7f9a376700 1 --2- 172.21.15.12:0/3682530951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f94106250 0x7f7f9410ddd0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00b0d0 tx=0x7f7f8c00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.852+0000 7f7f9a376700 1 -- 172.21.15.12:0/3682530951 shutdown_connections 2024-05-08T23:56:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.852+0000 7f7f9a376700 1 --2- 172.21.15.12:0/3682530951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f94106250 0x7f7f9410ddd0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.852+0000 7f7f9a376700 1 --2- 172.21.15.12:0/3682530951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f94069960 0x7f7f94069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.852+0000 7f7f9a376700 1 --2- 172.21.15.12:0/3682530951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f9406a310 0x7f7f94105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.852+0000 7f7f9a376700 1 -- 172.21.15.12:0/3682530951 >> 172.21.15.12:0/3682530951 conn(0x7f7f94076f70 msgr2=0x7f7f94077380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.852+0000 7f7f9a376700 1 -- 172.21.15.12:0/3682530951 shutdown_connections 2024-05-08T23:56:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.852+0000 7f7f9a376700 1 -- 172.21.15.12:0/3682530951 wait complete. 2024-05-08T23:56:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.853+0000 7f7f9a376700 1 Processor -- start 2024-05-08T23:56:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.853+0000 7f7f9a376700 1 -- start start 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.853+0000 7f7f9a376700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f94069960 0x7f7f94197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.853+0000 7f7f9a376700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f94106250 0x7f7f941979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.853+0000 7f7f9a376700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f9419bd50 0x7f7f9419c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.853+0000 7f7f9a376700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9419c7d0 con 0x7f7f94069960 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.853+0000 7f7f9a376700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9419c940 con 0x7f7f9419bd50 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.853+0000 7f7f9a376700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9419cb00 con 0x7f7f94106250 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.853+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f94069960 0x7f7f94197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f94069960 0x7f7f94197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44194/0 (socket says 172.21.15.12:44194) 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f937fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f94106250 0x7f7f941979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f93fff700 1 -- 172.21.15.12:0/2983836812 learned_addr learned my addr 172.21.15.12:0/2983836812 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f98913700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f9419bd50 0x7f7f9419c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f937fe700 1 -- 172.21.15.12:0/2983836812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f9419bd50 msgr2=0x7f7f9419c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f937fe700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f9419bd50 0x7f7f9419c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f937fe700 1 -- 172.21.15.12:0/2983836812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f94069960 msgr2=0x7f7f94197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f937fe700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f94069960 0x7f7f94197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f937fe700 1 -- 172.21.15.12:0/2983836812 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c009d00 con 0x7f7f94106250 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f98913700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f9419bd50 0x7f7f9419c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f93fff700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f94069960 0x7f7f94197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f937fe700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f94106250 0x7f7f941979b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800e9c0 tx=0x7f7f8800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f917fa700 1 -- 172.21.15.12:0/2983836812 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8800c9c0 con 0x7f7f94106250 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f917fa700 1 -- 172.21.15.12:0/2983836812 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f88010430 con 0x7f7f94106250 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f9a376700 1 -- 172.21.15.12:0/2983836812 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f941a4060 con 0x7f7f94106250 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f917fa700 1 -- 172.21.15.12:0/2983836812 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88018680 con 0x7f7f94106250 2024-05-08T23:56:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.854+0000 7f7f9a376700 1 -- 172.21.15.12:0/2983836812 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f941a4540 con 0x7f7f94106250 2024-05-08T23:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.857+0000 7f7f917fa700 1 -- 172.21.15.12:0/2983836812 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7f8800cb20 con 0x7f7f94106250 2024-05-08T23:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.857+0000 7f7f917fa700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f7c0a1290 0x7f7f7c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.857+0000 7f7f93fff700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f7c0a1290 0x7f7f7c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.857+0000 7f7f917fa700 1 -- 172.21.15.12:0/2983836812 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7f88014070 con 0x7f7f94106250 2024-05-08T23:56:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.858+0000 7f7f93fff700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f7c0a1290 0x7f7f7c0a3750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7f84011550 tx=0x7f7f84009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7fa6530b80 1 Processor -- start 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7fa6530b80 1 -- start start 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fabd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 0x55db73fa5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fa68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73efd0f0 con 0x55db73fab950 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73efaab0 con 0x55db73fa6480 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73ef63a0 con 0x55db73fac270 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fabd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fabd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44196/0 (socket says 172.21.15.12:44196) 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f93fff700 1 -- 172.21.15.12:0/151087146 learned_addr learned my addr 172.21.15.12:0/151087146 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f937fe700 1 --2- 172.21.15.12:0/151087146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 0x55db73fa5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f98913700 1 --2- 172.21.15.12:0/151087146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fa68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f937fe700 1 -- 172.21.15.12:0/151087146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 msgr2=0x55db73fa68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f937fe700 1 --2- 172.21.15.12:0/151087146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fa68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f937fe700 1 -- 172.21.15.12:0/151087146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 msgr2=0x55db73fabd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f937fe700 1 --2- 172.21.15.12:0/151087146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fabd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f937fe700 1 -- 172.21.15.12:0/151087146 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db73d77350 con 0x55db73fac270 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.859+0000 7f7f93fff700 1 --2- 172.21.15.12:0/151087146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fabd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.860+0000 7f7f937fe700 1 --2- 172.21.15.12:0/151087146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 0x55db73fa5f40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7f880ca4e0 tx=0x7f7f88099a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=486341613726d9e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.860+0000 7f7f7a7fc700 1 -- 172.21.15.12:0/151087146 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88099ea0 con 0x55db73fac270 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.860+0000 7f7f7a7fc700 1 -- 172.21.15.12:0/151087146 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f8809dcd0 con 0x55db73fac270 2024-05-08T23:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.860+0000 7f7f7a7fc700 1 -- 172.21.15.12:0/151087146 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8809a100 con 0x55db73fac270 2024-05-08T23:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 -- 172.21.15.12:0/151087146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 msgr2=0x55db73fa5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/151087146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 0x55db73fa5f40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7f880ca4e0 tx=0x7f7f88099a60 comp rx=0 tx=0).stop 2024-05-08T23:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 -- 172.21.15.12:0/151087146 shutdown_connections 2024-05-08T23:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/151087146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fa68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/151087146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fabd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/151087146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 0x55db73fa5f40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 -- 172.21.15.12:0/151087146 >> 172.21.15.12:0/151087146 conn(0x55db73f041b0 msgr2=0x55db73fa0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 -- 172.21.15.12:0/151087146 shutdown_connections 2024-05-08T23:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 -- 172.21.15.12:0/151087146 wait complete. 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 -- start start 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.861+0000 7f7fa6530b80 1 Processor -- start 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73faf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73e92500 con 0x55db73fac270 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73e921a0 con 0x55db73fa6480 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73d79640 con 0x55db73fab950 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7fa6530b80 10 client.0 _ll_register_callbacks cb 0x55db73e9bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f937fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f98913700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73faf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f937fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51358/0 (socket says 172.21.15.12:51358) 2024-05-08T23:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f937fe700 1 -- 172.21.15.12:0/4031369497 learned_addr learned my addr 172.21.15.12:0/4031369497 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f98913700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44204/0 (socket says 172.21.15.12:44204) 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f937fe700 1 -- 172.21.15.12:0/4031369497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 msgr2=0x55db73faf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f937fe700 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73faf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f937fe700 1 -- 172.21.15.12:0/4031369497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 msgr2=0x55db73faff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f937fe700 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f937fe700 1 -- 172.21.15.12:0/4031369497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f84017070 con 0x55db73fab950 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f937fe700 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafa50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f880ca590 tx=0x7f7f88099740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff64f9ff7f21da76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.862+0000 7f7f737fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7f737fe700 1 -- 172.21.15.12:0/4031369497 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880a2320 con 0x55db73fab950 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7f737fe700 1 -- 172.21.15.12:0/4031369497 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7f88010860 con 0x55db73fab950 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7fa6530b80 1 -- 172.21.15.12:0/4031369497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db73e4f210 con 0x55db73fab950 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7f737fe700 1 -- 172.21.15.12:0/4031369497 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f880105a0 con 0x55db73fab950 2024-05-08T23:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7fa6530b80 10 client.28078 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7fa6530b80 1 -- 172.21.15.12:0/4031369497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f7f941a4540 con 0x55db73fab950 2024-05-08T23:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7f737fe700 1 -- 172.21.15.12:0/4031369497 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db73e4f210 con 0x55db73fab950 2024-05-08T23:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7f737fe700 1 -- 172.21.15.12:0/4031369497 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7f880a1e90 con 0x55db73fab950 2024-05-08T23:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7fa6530b80 10 client.28078 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7fa6530b80 10 client.28078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7fa6530b80 20 client.28078 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db73fbf880 0x55db73fc1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7fa6530b80 4 client.28078 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.863+0000 7f7fa6530b80 1 -- 172.21.15.12:0/4031369497 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db73f757b0 con 0x55db73fbf880 2024-05-08T23:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.864+0000 7f7f93fff700 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db73fbf880 0x55db73fc1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.864+0000 7f7f93fff700 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db73fbf880 0x55db73fc1ca0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=384b026527e95ee2 server_cookie=5f61bec416e11abd in_seq=0 out_seq=0 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.864+0000 7f7f737fe700 10 client.28078 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7f737fe700 1 -- 172.21.15.12:0/4031369497 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db73f757b0 con 0x55db73fbf880 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7f737fe700 10 client.28078 handle_command_reply: tid=0 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 client.28078 shutdown 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 -- 172.21.15.12:0/4031369497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 msgr2=0x55db73fafa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafa50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f880ca590 tx=0x7f7f88099740 comp rx=0 tx=0).stop 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 -- 172.21.15.12:0/4031369497 shutdown_connections 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db73fbf880 0x55db73fc1ca0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73faf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/4031369497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafa50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7f737fe700 0 client.28078 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 -- 172.21.15.12:0/4031369497 >> 172.21.15.12:0/4031369497 conn(0x55db73f041b0 msgr2=0x55db73faa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 -- 172.21.15.12:0/4031369497 shutdown_connections 2024-05-08T23:56:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 1 -- 172.21.15.12:0/4031369497 wait complete. 2024-05-08T23:56:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.865+0000 7f7fa6530b80 20 client.28078 trim_cache size 0 max 16384 2024-05-08T23:56:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.881+0000 7f7fa6530b80 1 Processor -- start 2024-05-08T23:56:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.881+0000 7f7fa6530b80 1 -- start start 2024-05-08T23:56:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.881+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fa0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.881+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.881+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.881+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73fa5100 con 0x55db73fac270 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.881+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73fbc570 con 0x55db73fa6480 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.881+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73fc2300 con 0x55db73fab950 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.881+0000 7f7f98913700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.881+0000 7f7f98913700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faa400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44218/0 (socket says 172.21.15.12:44218) 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fa0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f98913700 1 -- 172.21.15.12:0/213635411 learned_addr learned my addr 172.21.15.12:0/213635411 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f937fe700 1 --2- 172.21.15.12:0/213635411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f98913700 1 -- 172.21.15.12:0/213635411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 msgr2=0x55db73fafdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f98913700 1 --2- 172.21.15.12:0/213635411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f98913700 1 -- 172.21.15.12:0/213635411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 msgr2=0x55db73fa0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f98913700 1 --2- 172.21.15.12:0/213635411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fa0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f98913700 1 -- 172.21.15.12:0/213635411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db73d23b00 con 0x55db73fac270 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f937fe700 1 --2- 172.21.15.12:0/213635411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f93fff700 1 --2- 172.21.15.12:0/213635411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fa0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f98913700 1 --2- 172.21.15.12:0/213635411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faa400 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009c30 tx=0x7f7f8c009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74e5d0ff648cf6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f7affd700 1 -- 172.21.15.12:0/213635411 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00e040 con 0x55db73fac270 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f7affd700 1 -- 172.21.15.12:0/213635411 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f8c00baa0 con 0x55db73fac270 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.882+0000 7f7f7affd700 1 -- 172.21.15.12:0/213635411 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c005070 con 0x55db73fac270 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.883+0000 7f7fa6530b80 1 -- 172.21.15.12:0/213635411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 msgr2=0x55db73faa400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.883+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/213635411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faa400 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009c30 tx=0x7f7f8c009c60 comp rx=0 tx=0).stop 2024-05-08T23:56:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.883+0000 7f7fa6530b80 1 -- 172.21.15.12:0/213635411 shutdown_connections 2024-05-08T23:56:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.883+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/213635411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fac270 0x55db73faa400 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.883+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/213635411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fa0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.883+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/213635411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fab950 0x55db73fafdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.883+0000 7f7fa6530b80 1 -- 172.21.15.12:0/213635411 >> 172.21.15.12:0/213635411 conn(0x55db73f041b0 msgr2=0x55db73fa20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.883+0000 7f7fa6530b80 1 -- 172.21.15.12:0/213635411 shutdown_connections 2024-05-08T23:56:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.883+0000 7f7fa6530b80 1 -- 172.21.15.12:0/213635411 wait complete. 2024-05-08T23:56:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.883+0000 7f7fa6530b80 1 -- start start 2024-05-08T23:56:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7fa6530b80 1 Processor -- start 2024-05-08T23:56:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fb2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fb3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7fa6530b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 0x55db73fb35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db74014d30 con 0x55db73fab950 2024-05-08T23:56:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db74014e70 con 0x55db73fa6480 2024-05-08T23:56:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7fa6530b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db74014fb0 con 0x55db73fac270 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7fa6530b80 10 client.0 _ll_register_callbacks cb 0x55db73e9bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7f937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fb3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7f937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fb3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44224/0 (socket says 172.21.15.12:44224) 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7f937fe700 1 -- 172.21.15.12:0/3732442895 learned_addr learned my addr 172.21.15.12:0/3732442895 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.884+0000 7f7f93fff700 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fb2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f98913700 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 0x55db73fb35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f937fe700 1 -- 172.21.15.12:0/3732442895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 msgr2=0x55db73fb35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f937fe700 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 0x55db73fb35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f937fe700 1 -- 172.21.15.12:0/3732442895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 msgr2=0x55db73fb2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f937fe700 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fb2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f937fe700 1 -- 172.21.15.12:0/3732442895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8400e920 con 0x55db73fab950 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f93fff700 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fb2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f937fe700 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fb3090 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7f880981f0 tx=0x7f7f880aff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc7287d914ba393 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f737fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f737fe700 1 -- 172.21.15.12:0/3732442895 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880a2320 con 0x55db73fab950 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7fa6530b80 1 -- 172.21.15.12:0/3732442895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f8809dcd0 con 0x55db73fab950 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f737fe700 1 -- 172.21.15.12:0/3732442895 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7f880105a0 con 0x55db73fab950 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7f737fe700 1 -- 172.21.15.12:0/3732442895 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f880a1e90 con 0x55db73fab950 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7fa6530b80 10 client.18345 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.885+0000 7f7fa6530b80 1 -- 172.21.15.12:0/3732442895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f7f941a4540 con 0x55db73fab950 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7f737fe700 1 -- 172.21.15.12:0/3732442895 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f8809dcd0 con 0x55db73fab950 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7f737fe700 1 -- 172.21.15.12:0/3732442895 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7f88094ab0 con 0x55db73fab950 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7fa6530b80 10 client.18345 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7fa6530b80 10 client.18345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7fa6530b80 20 client.18345 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db74016300 0x55db74018720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7fa6530b80 4 client.18345 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:56:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7fa6530b80 1 -- 172.21.15.12:0/3732442895 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db73eeee60 con 0x55db74016300 2024-05-08T23:56:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7f93fff700 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db74016300 0x55db74018720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7f737fe700 10 client.18345 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.886+0000 7f7f93fff700 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db74016300 0x55db74018720 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d71fa5ad61055c1 server_cookie=9dffc1de8c259513 in_seq=0 out_seq=0 2024-05-08T23:56:51.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 586307 6.53 MB/sec execute 407 sec latency 2989.482 ms 2024-05-08T23:56:51.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:51 smithi012 ceph-mon[42503]: pgmap v330: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.7 MiB/s wr, 157 op/s 2024-05-08T23:56:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7f737fe700 1 -- 172.21.15.12:0/3732442895 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db73eeee60 con 0x55db74016300 2024-05-08T23:56:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7f737fe700 10 client.18345 handle_command_reply: tid=0 2024-05-08T23:56:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 client.18345 shutdown 2024-05-08T23:56:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 -- 172.21.15.12:0/3732442895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 msgr2=0x55db73fb3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fb3090 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7f880981f0 tx=0x7f7f880aff40 comp rx=0 tx=0).stop 2024-05-08T23:56:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 -- 172.21.15.12:0/3732442895 shutdown_connections 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db73fac270 0x55db73fb35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db74016300 0x55db74018720 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db73fa6480 0x55db73fb2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 --2- 172.21.15.12:0/3732442895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db73fab950 0x55db73fb3090 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7f737fe700 0 client.18345 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 -- 172.21.15.12:0/3732442895 >> 172.21.15.12:0/3732442895 conn(0x55db73f041b0 msgr2=0x55db73fa20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 -- 172.21.15.12:0/3732442895 shutdown_connections 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 1 -- 172.21.15.12:0/3732442895 wait complete. 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.928+0000 7f7fa6530b80 20 client.18345 trim_cache size 0 max 16384 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:56:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:56:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:56:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:56:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 -- 172.21.15.12:0/2983836812 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f7c0a1290 msgr2=0x7f7f7c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f7c0a1290 0x7f7f7c0a3750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7f84011550 tx=0x7f7f84009260 comp rx=0 tx=0).stop 2024-05-08T23:56:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 -- 172.21.15.12:0/2983836812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f94106250 msgr2=0x7f7f941979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f94106250 0x7f7f941979b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800e9c0 tx=0x7f7f8800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 -- 172.21.15.12:0/2983836812 shutdown_connections 2024-05-08T23:56:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7f7c0a1290 0x7f7f7c0a3750 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f9419bd50 0x7f7f9419c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f94069960 0x7f7f94197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 --2- 172.21.15.12:0/2983836812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f94106250 0x7f7f941979b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 -- 172.21.15.12:0/2983836812 >> 172.21.15.12:0/2983836812 conn(0x7f7f94076f70 msgr2=0x7f7f940fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 -- 172.21.15.12:0/2983836812 shutdown_connections 2024-05-08T23:56:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:56:51.929+0000 7f7f9a376700 1 -- 172.21.15.12:0/2983836812 wait complete. 2024-05-08T23:56:52.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:51 smithi204 ceph-mon[47373]: pgmap v330: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.7 MiB/s wr, 157 op/s 2024-05-08T23:56:52.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:51 smithi160 ceph-mon[46152]: pgmap v330: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.7 MiB/s wr, 157 op/s 2024-05-08T23:56:52.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 587665 6.53 MB/sec execute 408 sec latency 26.910 ms 2024-05-08T23:56:53.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 587763 6.52 MB/sec execute 409 sec latency 934.029 ms 2024-05-08T23:56:53.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:53 smithi012 ceph-mon[42503]: pgmap v331: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-05-08T23:56:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:53 smithi204 ceph-mon[47373]: pgmap v331: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-05-08T23:56:54.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:53 smithi160 ceph-mon[46152]: pgmap v331: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-05-08T23:56:54.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 587763 6.50 MB/sec execute 410 sec latency 1934.106 ms 2024-05-08T23:56:55.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 587763 6.48 MB/sec execute 411 sec latency 2934.238 ms 2024-05-08T23:56:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:55 smithi012 ceph-mon[42503]: pgmap v332: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-05-08T23:56:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:55 smithi204 ceph-mon[47373]: pgmap v332: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-05-08T23:56:56.025 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:55 smithi160 ceph-mon[46152]: pgmap v332: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-05-08T23:56:56.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:56:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.730+0000 7f0079219700 1 -- 172.21.15.12:0/2108354540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0074100a10 msgr2=0x7f0074104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.730+0000 7f0079219700 1 --2- 172.21.15.12:0/2108354540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0074100a10 0x7f0074104a30 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0068009990 tx=0x7f0068009ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.731+0000 7f0079219700 1 -- 172.21.15.12:0/2108354540 shutdown_connections 2024-05-08T23:56:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.731+0000 7f0079219700 1 --2- 172.21.15.12:0/2108354540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0074105160 0x7f007410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.731+0000 7f0079219700 1 --2- 172.21.15.12:0/2108354540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0074100060 0x7f0074100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.731+0000 7f0079219700 1 --2- 172.21.15.12:0/2108354540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0074100a10 0x7f0074104a30 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.731+0000 7f0079219700 1 -- 172.21.15.12:0/2108354540 >> 172.21.15.12:0/2108354540 conn(0x7f0074076f40 msgr2=0x7f0074077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.731+0000 7f0079219700 1 -- 172.21.15.12:0/2108354540 shutdown_connections 2024-05-08T23:56:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.731+0000 7f0079219700 1 -- 172.21.15.12:0/2108354540 wait complete. 2024-05-08T23:56:56.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.731+0000 7f0079219700 1 Processor -- start 2024-05-08T23:56:56.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f0079219700 1 -- start start 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f0079219700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0074100060 0x7f0074197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f0079219700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0074105160 0x7f0074197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f0079219700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f007419bd00 0x7f007419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f0079219700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007419c780 con 0x7f0074105160 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f0079219700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007419c8f0 con 0x7f0074100060 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f0079219700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007419cab0 con 0x7f007419bd00 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f007259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0074105160 0x7f0074197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f007259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0074105160 0x7f0074197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44250/0 (socket says 172.21.15.12:44250) 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f007259c700 1 -- 172.21.15.12:0/3078487517 learned_addr learned my addr 172.21.15.12:0/3078487517 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f0072d9d700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0074100060 0x7f0074197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f007359e700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f007419bd00 0x7f007419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f007259c700 1 -- 172.21.15.12:0/3078487517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f007419bd00 msgr2=0x7f007419c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f007259c700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f007419bd00 0x7f007419c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f007259c700 1 -- 172.21.15.12:0/3078487517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0074100060 msgr2=0x7f0074197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f007259c700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0074100060 0x7f0074197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f007259c700 1 -- 172.21.15.12:0/3078487517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680095f0 con 0x7f0074105160 2024-05-08T23:56:56.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.732+0000 7f0072d9d700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0074100060 0x7f0074197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:56.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.733+0000 7f007259c700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0074105160 0x7f0074197960 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c00 tx=0x7f0068004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:56.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.733+0000 7f005bfff700 1 -- 172.21.15.12:0/3078487517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006801d070 con 0x7f0074105160 2024-05-08T23:56:56.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.733+0000 7f005bfff700 1 -- 172.21.15.12:0/3078487517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0068021410 con 0x7f0074105160 2024-05-08T23:56:56.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.733+0000 7f0079219700 1 -- 172.21.15.12:0/3078487517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00741a3fb0 con 0x7f0074105160 2024-05-08T23:56:56.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.733+0000 7f005bfff700 1 -- 172.21.15.12:0/3078487517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068017650 con 0x7f0074105160 2024-05-08T23:56:56.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.733+0000 7f0079219700 1 -- 172.21.15.12:0/3078487517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00741a4370 con 0x7f0074105160 2024-05-08T23:56:56.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.735+0000 7f005bfff700 1 -- 172.21.15.12:0/3078487517 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f00680177b0 con 0x7f0074105160 2024-05-08T23:56:56.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.736+0000 7f005bfff700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f005c0a1290 0x7f005c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.736+0000 7f005bfff700 1 -- 172.21.15.12:0/3078487517 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f00680cdf40 con 0x7f0074105160 2024-05-08T23:56:56.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.736+0000 7f0072d9d700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f005c0a1290 0x7f005c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f00853d3b80 1 Processor -- start 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f0072d9d700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f005c0a1290 0x7f005c0a3750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0064011490 tx=0x7f0064009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f00853d3b80 1 -- start start 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f6e00 0x5594175f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f1890 0x5594175f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f00853d3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559417548150 con 0x5594175f7720 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f00853d3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559417545b10 con 0x5594175f1890 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f00853d3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559417541400 con 0x5594175f6e00 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f007259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f007259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175f1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44256/0 (socket says 172.21.15.12:44256) 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f007259c700 1 -- 172.21.15.12:0/2145283804 learned_addr learned my addr 172.21.15.12:0/2145283804 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f0072d9d700 1 --2- 172.21.15.12:0/2145283804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f6e00 0x5594175f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f007359e700 1 --2- 172.21.15.12:0/2145283804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f1890 0x5594175f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f007259c700 1 -- 172.21.15.12:0/2145283804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f6e00 msgr2=0x5594175f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f007259c700 1 --2- 172.21.15.12:0/2145283804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f6e00 0x5594175f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f007259c700 1 -- 172.21.15.12:0/2145283804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f1890 msgr2=0x5594175f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f007259c700 1 --2- 172.21.15.12:0/2145283804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f1890 0x5594175f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f007259c700 1 -- 172.21.15.12:0/2145283804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594173c23b0 con 0x5594175f7720 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.737+0000 7f007359e700 1 --2- 172.21.15.12:0/2145283804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f1890 0x5594175f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.738+0000 7f007259c700 1 --2- 172.21.15.12:0/2145283804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175f1350 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f00680a2f80 tx=0x7f0068009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811597ba5ce459b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.738+0000 7f00597fa700 1 -- 172.21.15.12:0/2145283804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006809e070 con 0x5594175f7720 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.738+0000 7f00597fa700 1 -- 172.21.15.12:0/2145283804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f006809b7c0 con 0x5594175f7720 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.738+0000 7f00597fa700 1 -- 172.21.15.12:0/2145283804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068021860 con 0x5594175f7720 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.738+0000 7f00853d3b80 1 -- 172.21.15.12:0/2145283804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 msgr2=0x5594175f1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.738+0000 7f00853d3b80 1 --2- 172.21.15.12:0/2145283804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175f1350 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f00680a2f80 tx=0x7f0068009310 comp rx=0 tx=0).stop 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 -- 172.21.15.12:0/2145283804 shutdown_connections 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 --2- 172.21.15.12:0/2145283804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f1890 0x5594175f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 --2- 172.21.15.12:0/2145283804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f6e00 0x5594175f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 --2- 172.21.15.12:0/2145283804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175f1350 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 -- 172.21.15.12:0/2145283804 >> 172.21.15.12:0/2145283804 conn(0x55941754f210 msgr2=0x5594175ebdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 -- 172.21.15.12:0/2145283804 shutdown_connections 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 -- 172.21.15.12:0/2145283804 wait complete. 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 -- start start 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 Processor -- start 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f1890 0x5594175faa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 0x5594175fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f7720 0x5594175fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594174dd560 con 0x5594175f1890 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594174dd200 con 0x5594175f6e00 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594173c46a0 con 0x5594175f7720 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f00853d3b80 10 client.0 _ll_register_callbacks cb 0x5594174e6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:56.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.739+0000 7f0072d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f1890 0x5594175faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0072d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f1890 0x5594175faa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44266/0 (socket says 172.21.15.12:44266) 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0072d9d700 1 -- 172.21.15.12:0/880459425 learned_addr learned my addr 172.21.15.12:0/880459425 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f007259c700 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 0x5594175fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f007359e700 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f7720 0x5594175fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0072d9d700 1 -- 172.21.15.12:0/880459425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f7720 msgr2=0x5594175fb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0072d9d700 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f7720 0x5594175fb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0072d9d700 1 -- 172.21.15.12:0/880459425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 msgr2=0x5594175fafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0072d9d700 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 0x5594175fafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0072d9d700 1 -- 172.21.15.12:0/880459425 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00741a4370 con 0x5594175f1890 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0072d9d700 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f1890 0x5594175faa60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f00640039f0 tx=0x7f00640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abf6d2a70330d58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0056ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0056ff5700 1 -- 172.21.15.12:0/880459425 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006401f9c0 con 0x5594175f1890 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f0056ff5700 1 -- 172.21.15.12:0/880459425 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f006401fb20 con 0x5594175f1890 2024-05-08T23:56:56.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.740+0000 7f00853d3b80 1 -- 172.21.15.12:0/880459425 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55941749a270 con 0x5594175f1890 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f0056ff5700 1 -- 172.21.15.12:0/880459425 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006401fce0 con 0x5594175f1890 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f00853d3b80 10 client.18357 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f00853d3b80 1 -- 172.21.15.12:0/880459425 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55941736eb60 con 0x5594175f1890 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f0056ff5700 1 -- 172.21.15.12:0/880459425 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55941749a270 con 0x5594175f1890 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f0056ff5700 1 -- 172.21.15.12:0/880459425 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0064027500 con 0x5594175f1890 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f00853d3b80 10 client.18357 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f00853d3b80 10 client.18357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f00853d3b80 20 client.18357 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f00853d3b80 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55941760aef0 0x55941760d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f00853d3b80 4 client.18357 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f00853d3b80 1 -- 172.21.15.12:0/880459425 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594175c0810 con 0x55941760aef0 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.741+0000 7f007259c700 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55941760aef0 0x55941760d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f007259c700 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55941760aef0 0x55941760d310 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ca8dde21284e4b server_cookie=6a011bef359f1510 in_seq=0 out_seq=0 2024-05-08T23:56:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f0056ff5700 10 client.18357 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f0056ff5700 1 -- 172.21.15.12:0/880459425 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5594175c0810 con 0x55941760aef0 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f0056ff5700 10 client.18357 handle_command_reply: tid=0 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 client.18357 shutdown 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 -- 172.21.15.12:0/880459425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f1890 msgr2=0x5594175faa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f1890 0x5594175faa60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f00640039f0 tx=0x7f00640047c0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 -- 172.21.15.12:0/880459425 shutdown_connections 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f7720 0x5594175fb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f1890 0x5594175faa60 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55941760aef0 0x55941760d310 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 --2- 172.21.15.12:0/880459425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 0x5594175fafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f0056ff5700 0 client.18357 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:56.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 -- 172.21.15.12:0/880459425 >> 172.21.15.12:0/880459425 conn(0x55941754f210 msgr2=0x5594175eb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:56.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 -- 172.21.15.12:0/880459425 shutdown_connections 2024-05-08T23:56:56.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.742+0000 7f00853d3b80 1 -- 172.21.15.12:0/880459425 wait complete. 2024-05-08T23:56:56.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.743+0000 7f00853d3b80 20 client.18357 trim_cache size 0 max 16384 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f00853d3b80 1 Processor -- start 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f00853d3b80 1 -- start start 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594175f6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f6e00 0x5594175ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f7720 0x5594175f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f00853d3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559417608610 con 0x5594175f6e00 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f00853d3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594175ecb10 con 0x5594175f7720 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f00853d3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559417612550 con 0x5594175f1890 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f007259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f6e00 0x5594175ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f0072d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594175f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f007259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f6e00 0x5594175ebd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44268/0 (socket says 172.21.15.12:44268) 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f007259c700 1 -- 172.21.15.12:0/1564983193 learned_addr learned my addr 172.21.15.12:0/1564983193 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.768+0000 7f007359e700 1 --2- 172.21.15.12:0/1564983193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f7720 0x5594175f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.769+0000 7f0072d9d700 1 -- 172.21.15.12:0/1564983193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f7720 msgr2=0x5594175f9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.769+0000 7f0072d9d700 1 --2- 172.21.15.12:0/1564983193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f7720 0x5594175f9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.769+0000 7f0072d9d700 1 -- 172.21.15.12:0/1564983193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f6e00 msgr2=0x5594175ebd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.769+0000 7f0072d9d700 1 --2- 172.21.15.12:0/1564983193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f6e00 0x5594175ebd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.769+0000 7f0072d9d700 1 -- 172.21.15.12:0/1564983193 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594175f1f40 con 0x5594175f1890 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.769+0000 7f007259c700 1 --2- 172.21.15.12:0/1564983193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f6e00 0x5594175ebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.769+0000 7f0072d9d700 1 --2- 172.21.15.12:0/1564983193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594175f6430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0064000c00 tx=0x7f00640058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f190150a8e6e3e74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.769+0000 7f0059ffb700 1 -- 172.21.15.12:0/1564983193 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00640273e0 con 0x5594175f1890 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.769+0000 7f0059ffb700 1 -- 172.21.15.12:0/1564983193 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0064027540 con 0x5594175f1890 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.769+0000 7f0059ffb700 1 -- 172.21.15.12:0/1564983193 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006401fbe0 con 0x5594175f1890 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.770+0000 7f00853d3b80 1 -- 172.21.15.12:0/1564983193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 msgr2=0x5594175f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.770+0000 7f00853d3b80 1 --2- 172.21.15.12:0/1564983193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594175f6430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0064000c00 tx=0x7f00640058a0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.770+0000 7f00853d3b80 1 -- 172.21.15.12:0/1564983193 shutdown_connections 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.770+0000 7f00853d3b80 1 --2- 172.21.15.12:0/1564983193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f7720 0x5594175f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.770+0000 7f00853d3b80 1 --2- 172.21.15.12:0/1564983193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594175f6430 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.770+0000 7f00853d3b80 1 --2- 172.21.15.12:0/1564983193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f6e00 0x5594175ebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.770+0000 7f00853d3b80 1 -- 172.21.15.12:0/1564983193 >> 172.21.15.12:0/1564983193 conn(0x55941754f210 msgr2=0x5594175f9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.770+0000 7f00853d3b80 1 -- 172.21.15.12:0/1564983193 shutdown_connections 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.770+0000 7f00853d3b80 1 -- 172.21.15.12:0/1564983193 wait complete. 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.770+0000 7f00853d3b80 1 -- start start 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f00853d3b80 1 Processor -- start 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594176021e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 0x5594175fe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f00853d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175fead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f00853d3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594175ff190 con 0x5594175f7720 2024-05-08T23:56:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f00853d3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594175ff300 con 0x5594175f6e00 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f00853d3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594176613f0 con 0x5594175f1890 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f00853d3b80 10 client.0 _ll_register_callbacks cb 0x5594174e6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 0x5594175fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f0072d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594176021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007359e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 0x5594175fe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:60874/0 (socket says 172.21.15.12:60874) 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f0072d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594176021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51432/0 (socket says 172.21.15.12:51432) 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007259c700 1 -- 172.21.15.12:0/425421579 learned_addr learned my addr 172.21.15.12:0/425421579 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007359e700 1 -- 172.21.15.12:0/425421579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 msgr2=0x5594176021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007359e700 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594176021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007359e700 1 -- 172.21.15.12:0/425421579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 msgr2=0x5594175fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007359e700 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 0x5594175fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007359e700 1 -- 172.21.15.12:0/425421579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594173c23b0 con 0x5594175f7720 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f0072d9d700 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594176021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007259c700 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 0x5594175fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f007359e700 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175fead0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f006c007fb0 tx=0x7f006c009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f923564f1ae599 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.771+0000 7f0056ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f0056ff5700 1 -- 172.21.15.12:0/425421579 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00e040 con 0x5594175f7720 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f0056ff5700 1 -- 172.21.15.12:0/425421579 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f006c004db0 con 0x5594175f7720 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f00853d3b80 1 -- 172.21.15.12:0/425421579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f006809b7c0 con 0x5594175f7720 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f0056ff5700 1 -- 172.21.15.12:0/425421579 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006c018b60 con 0x5594175f7720 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f00853d3b80 10 client.18363 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f00853d3b80 1 -- 172.21.15.12:0/425421579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55941736eb60 con 0x5594175f7720 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f0056ff5700 1 -- 172.21.15.12:0/425421579 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f006809b7c0 con 0x5594175f7720 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f0056ff5700 1 -- 172.21.15.12:0/425421579 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f006c016430 con 0x5594175f7720 2024-05-08T23:56:56.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f00853d3b80 10 client.18363 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f00853d3b80 10 client.18363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f00853d3b80 20 client.18363 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f00853d3b80 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559417662bc0 0x559417664fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f00853d3b80 4 client.18363 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.772+0000 7f00853d3b80 1 -- 172.21.15.12:0/425421579 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559417539ec0 con 0x559417662bc0 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f0072d9d700 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559417662bc0 0x559417664fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f0056ff5700 10 client.18363 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f0072d9d700 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559417662bc0 0x559417664fe0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc488b8acc9a472 server_cookie=73726362e7ee2248 in_seq=0 out_seq=0 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f0056ff5700 1 -- 172.21.15.12:0/425421579 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559417539ec0 con 0x559417662bc0 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f0056ff5700 10 client.18363 handle_command_reply: tid=0 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f00853d3b80 1 client.18363 shutdown 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f00853d3b80 1 -- 172.21.15.12:0/425421579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 msgr2=0x5594175fead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f00853d3b80 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175fead0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f006c007fb0 tx=0x7f006c009350 comp rx=0 tx=0).stop 2024-05-08T23:56:56.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f00853d3b80 1 -- 172.21.15.12:0/425421579 shutdown_connections 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f00853d3b80 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5594175f7720 0x5594175fead0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f00853d3b80 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594175f1890 0x5594176021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f00853d3b80 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559417662bc0 0x559417664fe0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f00853d3b80 1 --2- 172.21.15.12:0/425421579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594175f6e00 0x5594175fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f0056ff5700 0 client.18363 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.773+0000 7f00853d3b80 1 -- 172.21.15.12:0/425421579 >> 172.21.15.12:0/425421579 conn(0x55941754f210 msgr2=0x5594175fb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f00853d3b80 1 -- 172.21.15.12:0/425421579 shutdown_connections 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f00853d3b80 1 -- 172.21.15.12:0/425421579 wait complete. 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f00853d3b80 20 client.18363 trim_cache size 0 max 16384 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:56:56.777 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:56:56.778 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:56:56.778 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:56:56.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 -- 172.21.15.12:0/3078487517 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f005c0a1290 msgr2=0x7f005c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f005c0a1290 0x7f005c0a3750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0064011490 tx=0x7f0064009260 comp rx=0 tx=0).stop 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 -- 172.21.15.12:0/3078487517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0074105160 msgr2=0x7f0074197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0074105160 0x7f0074197960 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c00 tx=0x7f0068004730 comp rx=0 tx=0).stop 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 -- 172.21.15.12:0/3078487517 shutdown_connections 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f005c0a1290 0x7f005c0a3750 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f007419bd00 0x7f007419c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0074100060 0x7f0074197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 --2- 172.21.15.12:0/3078487517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0074105160 0x7f0074197960 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 -- 172.21.15.12:0/3078487517 >> 172.21.15.12:0/3078487517 conn(0x7f0074076f40 msgr2=0x7f0074068430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 -- 172.21.15.12:0/3078487517 shutdown_connections 2024-05-08T23:56:56.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.774+0000 7f0079219700 1 -- 172.21.15.12:0/3078487517 wait complete. 2024-05-08T23:56:56.782 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:56:56.782 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:56:56.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 589756 6.50 MB/sec execute 412 sec latency 3073.685 ms 2024-05-08T23:56:56.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.930+0000 7fc8166c8700 1 -- 172.21.15.12:0/2810653777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc810105060 msgr2=0x7fc81010cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.930+0000 7fc8166c8700 1 --2- 172.21.15.12:0/2810653777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc810105060 0x7fc81010cbe0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b0a0 tx=0x7fc80800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.930+0000 7fc8166c8700 1 -- 172.21.15.12:0/2810653777 shutdown_connections 2024-05-08T23:56:56.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.930+0000 7fc8166c8700 1 --2- 172.21.15.12:0/2810653777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc810105060 0x7fc81010cbe0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.930+0000 7fc8166c8700 1 --2- 172.21.15.12:0/2810653777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8100fff40 0x7fc810100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.930+0000 7fc8166c8700 1 --2- 172.21.15.12:0/2810653777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8101008f0 0x7fc810104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.930+0000 7fc8166c8700 1 -- 172.21.15.12:0/2810653777 >> 172.21.15.12:0/2810653777 conn(0x7fc8100fb7e0 msgr2=0x7fc8100fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:56.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.930+0000 7fc8166c8700 1 -- 172.21.15.12:0/2810653777 shutdown_connections 2024-05-08T23:56:56.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.930+0000 7fc8166c8700 1 -- 172.21.15.12:0/2810653777 wait complete. 2024-05-08T23:56:56.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc8166c8700 1 Processor -- start 2024-05-08T23:56:56.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc8166c8700 1 -- start start 2024-05-08T23:56:56.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc8166c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8100fff40 0x7fc810198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc8166c8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc810105060 0x7fc810198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc8166c8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc810198c90 0x7fc8101a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc8166c8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8101a39c0 con 0x7fc8100fff40 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc8166c8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8101a3b30 con 0x7fc810105060 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc8166c8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8101a3cf0 con 0x7fc810198c90 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc80ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8100fff40 0x7fc810198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc80ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8100fff40 0x7fc810198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44294/0 (socket says 172.21.15.12:44294) 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc80ffff700 1 -- 172.21.15.12:0/3086177803 learned_addr learned my addr 172.21.15.12:0/3086177803 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc814c65700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc810198c90 0x7fc8101a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.931+0000 7fc80f7fe700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc810105060 0x7fc810198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc814c65700 1 -- 172.21.15.12:0/3086177803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc810105060 msgr2=0x7fc810198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc814c65700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc810105060 0x7fc810198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc814c65700 1 -- 172.21.15.12:0/3086177803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8100fff40 msgr2=0x7fc810198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc814c65700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8100fff40 0x7fc810198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc814c65700 1 -- 172.21.15.12:0/3086177803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808009d00 con 0x7fc810198c90 2024-05-08T23:56:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc80ffff700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8100fff40 0x7fc810198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc814c65700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc810198c90 0x7fc8101a3290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc8080062a0 tx=0x7fc80800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc80d7fa700 1 -- 172.21.15.12:0/3086177803 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800e040 con 0x7fc810198c90 2024-05-08T23:56:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc8166c8700 1 -- 172.21.15.12:0/3086177803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8101a3f10 con 0x7fc810198c90 2024-05-08T23:56:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc80d7fa700 1 -- 172.21.15.12:0/3086177803 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80801d480 con 0x7fc810198c90 2024-05-08T23:56:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc80d7fa700 1 -- 172.21.15.12:0/3086177803 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80801b6d0 con 0x7fc810198c90 2024-05-08T23:56:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.932+0000 7fc8166c8700 1 -- 172.21.15.12:0/3086177803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8101a4390 con 0x7fc810198c90 2024-05-08T23:56:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.933+0000 7fc8166c8700 1 -- 172.21.15.12:0/3086177803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc810108330 con 0x7fc810198c90 2024-05-08T23:56:56.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.934+0000 7fc80d7fa700 1 -- 172.21.15.12:0/3086177803 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc80801d5f0 con 0x7fc810198c90 2024-05-08T23:56:56.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.935+0000 7fc80d7fa700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7fc0a1290 0x7fc7fc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:56.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.935+0000 7fc80ffff700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7fc0a1290 0x7fc7fc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:56.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.935+0000 7fc80d7fa700 1 -- 172.21.15.12:0/3086177803 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc8080ce230 con 0x7fc810198c90 2024-05-08T23:56:56.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.936+0000 7fc80ffff700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7fc0a1290 0x7fc7fc0a3750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc800011550 tx=0x7fc800009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:56.940+0000 7fc80d7fa700 1 -- 172.21.15.12:0/3086177803 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc808097750 con 0x7fc810198c90 2024-05-08T23:56:57.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.213+0000 7fc8166c8700 1 -- 172.21.15.12:0/3086177803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8100685a0 con 0x7fc810198c90 2024-05-08T23:56:57.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.214+0000 7fc80d7fa700 1 -- 172.21.15.12:0/3086177803 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+10878 (secure 0 0 0) 0x7fc80809ad70 con 0x7fc810198c90 2024-05-08T23:56:57.208 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:56:57.209 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:56:46.575052+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27940":{"gid":27940,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2536985900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2536985900},{"type":"v1","addr":"172.21.15.204:6837","nonce":2536985900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 -- 172.21.15.12:0/3086177803 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7fc0a1290 msgr2=0x7fc7fc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7fc0a1290 0x7fc7fc0a3750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc800011550 tx=0x7fc800009260 comp rx=0 tx=0).stop 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 -- 172.21.15.12:0/3086177803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc810198c90 msgr2=0x7fc8101a3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc810198c90 0x7fc8101a3290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc8080062a0 tx=0x7fc80800bd70 comp rx=0 tx=0).stop 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 -- 172.21.15.12:0/3086177803 shutdown_connections 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7fc0a1290 0x7fc7fc0a3750 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc810198c90 0x7fc8101a3290 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8100fff40 0x7fc810198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 --2- 172.21.15.12:0/3086177803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc810105060 0x7fc810198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 -- 172.21.15.12:0/3086177803 >> 172.21.15.12:0/3086177803 conn(0x7fc8100fb7e0 msgr2=0x7fc8100fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 -- 172.21.15.12:0/3086177803 shutdown_connections 2024-05-08T23:56:57.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.216+0000 7fc8166c8700 1 -- 172.21.15.12:0/3086177803 wait complete. 2024-05-08T23:56:57.212 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 45 2024-05-08T23:56:57.224 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:56:57.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.371+0000 7f6f53d72700 1 -- 172.21.15.12:0/1013677172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c103930 msgr2=0x7f6f4c103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.371+0000 7f6f53d72700 1 --2- 172.21.15.12:0/1013677172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c103930 0x7f6f4c103db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400b0a0 tx=0x7f6f4400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.371+0000 7f6f53d72700 1 -- 172.21.15.12:0/1013677172 shutdown_connections 2024-05-08T23:56:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.371+0000 7f6f53d72700 1 --2- 172.21.15.12:0/1013677172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c103930 0x7f6f4c103db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.371+0000 7f6f53d72700 1 --2- 172.21.15.12:0/1013677172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f4c105e30 0x7f6f4c068050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.371+0000 7f6f53d72700 1 --2- 172.21.15.12:0/1013677172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f4c068590 0x7f6f4c068a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.372+0000 7f6f53d72700 1 -- 172.21.15.12:0/1013677172 >> 172.21.15.12:0/1013677172 conn(0x7f6f4c076f40 msgr2=0x7f6f4c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.372+0000 7f6f53d72700 1 -- 172.21.15.12:0/1013677172 shutdown_connections 2024-05-08T23:56:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.372+0000 7f6f53d72700 1 -- 172.21.15.12:0/1013677172 wait complete. 2024-05-08T23:56:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.372+0000 7f6f53d72700 1 Processor -- start 2024-05-08T23:56:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f53d72700 1 -- start start 2024-05-08T23:56:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f53d72700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f4c068590 0x7f6f4c1a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f53d72700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f4c103930 0x7f6f4c1a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f53d72700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c105e30 0x7f6f4c19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f53d72700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c19df50 con 0x7f6f4c068590 2024-05-08T23:56:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f53d72700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c19e0c0 con 0x7f6f4c105e30 2024-05-08T23:56:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f53d72700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c19e280 con 0x7f6f4c103930 2024-05-08T23:56:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f51b0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f4c068590 0x7f6f4c1a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f51b0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f4c068590 0x7f6f4c1a37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44304/0 (socket says 172.21.15.12:44304) 2024-05-08T23:56:57.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f51b0e700 1 -- 172.21.15.12:0/1850213632 learned_addr learned my addr 172.21.15.12:0/1850213632 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f5230f700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c105e30 0x7f6f4c19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f5130d700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f4c103930 0x7f6f4c1a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f51b0e700 1 -- 172.21.15.12:0/1850213632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f4c103930 msgr2=0x7f6f4c1a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f4c103930 0x7f6f4c1a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f51b0e700 1 -- 172.21.15.12:0/1850213632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c105e30 msgr2=0x7f6f4c19d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c105e30 0x7f6f4c19d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f51b0e700 1 -- 172.21.15.12:0/1850213632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c009520 con 0x7f6f4c068590 2024-05-08T23:56:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.373+0000 7f6f5230f700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c105e30 0x7f6f4c19d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.374+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f4c068590 0x7f6f4c1a37c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800e9a0 tx=0x7f6f4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.374+0000 7f6f42ffd700 1 -- 172.21.15.12:0/1850213632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48009790 con 0x7f6f4c068590 2024-05-08T23:56:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.374+0000 7f6f42ffd700 1 -- 172.21.15.12:0/1850213632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f48010430 con 0x7f6f4c068590 2024-05-08T23:56:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.374+0000 7f6f53d72700 1 -- 172.21.15.12:0/1850213632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f44009d00 con 0x7f6f4c068590 2024-05-08T23:56:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.374+0000 7f6f42ffd700 1 -- 172.21.15.12:0/1850213632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48018680 con 0x7f6f4c068590 2024-05-08T23:56:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.374+0000 7f6f53d72700 1 -- 172.21.15.12:0/1850213632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c1aa5d0 con 0x7f6f4c068590 2024-05-08T23:56:57.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.376+0000 7f6f42ffd700 1 -- 172.21.15.12:0/1850213632 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6f480105a0 con 0x7f6f4c068590 2024-05-08T23:56:57.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.377+0000 7f6f42ffd700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f380a1290 0x7f6f380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.377+0000 7f6f42ffd700 1 -- 172.21.15.12:0/1850213632 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6f48014070 con 0x7f6f4c068590 2024-05-08T23:56:57.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.377+0000 7f6f5130d700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f380a1290 0x7f6f380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.378+0000 7f6f5130d700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f380a1290 0x7f6f380a3750 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c19eab0 tx=0x7f6f3c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.378+0000 7f6f5ff2cb80 1 Processor -- start 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.378+0000 7f6f5ff2cb80 1 -- start start 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.378+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 0x55d3e7a671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.378+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a61350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.378+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a61890 0x55d3e7a61cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.378+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e79b8150 con 0x55d3e7a67720 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.378+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e79b5b10 con 0x55d3e7a61890 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.378+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e79b1400 con 0x55d3e7a66e00 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5130d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a61350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5230f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a61890 0x55d3e7a61cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5130d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a61350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44316/0 (socket says 172.21.15.12:44316) 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5130d700 1 -- 172.21.15.12:0/315188901 learned_addr learned my addr 172.21.15.12:0/315188901 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/315188901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 0x55d3e7a671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5130d700 1 -- 172.21.15.12:0/315188901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 msgr2=0x55d3e7a671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5130d700 1 --2- 172.21.15.12:0/315188901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 0x55d3e7a671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5130d700 1 -- 172.21.15.12:0/315188901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a61890 msgr2=0x55d3e7a61cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5130d700 1 --2- 172.21.15.12:0/315188901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a61890 0x55d3e7a61cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5130d700 1 -- 172.21.15.12:0/315188901 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e78323b0 con 0x55d3e7a67720 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5230f700 1 --2- 172.21.15.12:0/315188901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a61890 0x55d3e7a61cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/315188901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 0x55d3e7a671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f5130d700 1 --2- 172.21.15.12:0/315188901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a61350 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e610 tx=0x7f6f3c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a23d4452402981 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f37fff700 1 -- 172.21.15.12:0/315188901 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c004220 con 0x55d3e7a67720 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f37fff700 1 -- 172.21.15.12:0/315188901 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f3c021070 con 0x55d3e7a67720 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.379+0000 7f6f37fff700 1 -- 172.21.15.12:0/315188901 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c01c670 con 0x55d3e7a67720 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.380+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/315188901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 msgr2=0x55d3e7a61350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.380+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/315188901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a61350 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e610 tx=0x7f6f3c014000 comp rx=0 tx=0).stop 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.380+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/315188901 shutdown_connections 2024-05-08T23:56:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.380+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/315188901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a61890 0x55d3e7a61cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.380+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/315188901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 0x55d3e7a671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.380+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/315188901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a61350 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.380+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/315188901 >> 172.21.15.12:0/315188901 conn(0x55d3e79bf210 msgr2=0x55d3e7a5bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.380+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/315188901 shutdown_connections 2024-05-08T23:56:57.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.380+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/315188901 wait complete. 2024-05-08T23:56:57.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f5ff2cb80 1 -- start start 2024-05-08T23:56:57.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f5ff2cb80 1 Processor -- start 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a61890 0x55d3e7a6aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 0x55d3e7a6b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a67720 0x55d3e7a6b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e794d560 con 0x55d3e7a61890 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e794d200 con 0x55d3e7a67720 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e78346a0 con 0x55d3e7a66e00 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f5ff2cb80 10 client.0 _ll_register_callbacks cb 0x55d3e7956b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f51b0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a61890 0x55d3e7a6aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f51b0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a61890 0x55d3e7a6aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44330/0 (socket says 172.21.15.12:44330) 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.381+0000 7f6f51b0e700 1 -- 172.21.15.12:0/1199952667 learned_addr learned my addr 172.21.15.12:0/1199952667 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f5130d700 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 0x55d3e7a6b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f5230f700 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a67720 0x55d3e7a6b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f51b0e700 1 -- 172.21.15.12:0/1199952667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 msgr2=0x55d3e7a6b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 0x55d3e7a6b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f51b0e700 1 -- 172.21.15.12:0/1199952667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a67720 msgr2=0x55d3e7a6b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a67720 0x55d3e7a6b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f51b0e700 1 -- 172.21.15.12:0/1199952667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c114d30 con 0x55d3e7a61890 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a61890 0x55d3e7a6aad0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6f480caba0 tx=0x7f6f48099fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d83a3ff04f996b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f357fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f357fa700 1 -- 172.21.15.12:0/1199952667 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4809a3c0 con 0x55d3e7a61890 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f357fa700 1 -- 172.21.15.12:0/1199952667 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6f480a23e0 con 0x55d3e7a61890 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/1199952667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3e790a270 con 0x55d3e7a61890 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f357fa700 1 -- 172.21.15.12:0/1199952667 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f48099450 con 0x55d3e7a61890 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f5ff2cb80 10 client.18381 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.382+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/1199952667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f6f3c017070 con 0x55d3e7a61890 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f357fa700 1 -- 172.21.15.12:0/1199952667 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3e790a270 con 0x55d3e7a61890 2024-05-08T23:56:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f357fa700 1 -- 172.21.15.12:0/1199952667 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6f48094b70 con 0x55d3e7a61890 2024-05-08T23:56:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f5ff2cb80 10 client.18381 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f5ff2cb80 10 client.18381 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:56:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f5ff2cb80 20 client.18381 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d3e7a7b200 0x55d3e7a7d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f5ff2cb80 4 client.18381 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/1199952667 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3e7a30810 con 0x55d3e7a7b200 2024-05-08T23:56:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f5130d700 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d3e7a7b200 0x55d3e7a7d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f357fa700 10 client.18381 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:56:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.383+0000 7f6f5130d700 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d3e7a7b200 0x55d3e7a7d5f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a409fba506aae226 server_cookie=a58839c10ef12bc6 in_seq=0 out_seq=0 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f357fa700 1 -- 172.21.15.12:0/1199952667 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3e7a30810 con 0x55d3e7a7b200 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f357fa700 10 client.18381 handle_command_reply: tid=0 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 client.18381 shutdown 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/1199952667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a61890 msgr2=0x55d3e7a6aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a61890 0x55d3e7a6aad0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6f480caba0 tx=0x7f6f48099fe0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/1199952667 shutdown_connections 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a67720 0x55d3e7a6b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a61890 0x55d3e7a6aad0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d3e7a7b200 0x55d3e7a7d5f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/1199952667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a66e00 0x55d3e7a6b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f357fa700 0 client.18381 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/1199952667 >> 172.21.15.12:0/1199952667 conn(0x55d3e79bf210 msgr2=0x55d3e7a5b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/1199952667 shutdown_connections 2024-05-08T23:56:57.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.384+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/1199952667 wait complete. 2024-05-08T23:56:57.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.385+0000 7f6f5ff2cb80 20 client.18381 trim_cache size 0 max 16384 2024-05-08T23:56:57.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.398+0000 7f6f5ff2cb80 1 Processor -- start 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5ff2cb80 1 -- start start 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a772c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a76c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a5bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e7a61740 con 0x55d3e7a67720 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e7a827c0 con 0x55d3e7a66e00 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e7a56830 con 0x55d3e7a61890 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5230f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a5bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f51b0e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a772c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5130d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a76c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f51b0e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a772c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51510/0 (socket says 172.21.15.12:51510) 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5230f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a5bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44340/0 (socket says 172.21.15.12:44340) 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f51b0e700 1 -- 172.21.15.12:0/4218115968 learned_addr learned my addr 172.21.15.12:0/4218115968 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5230f700 1 -- 172.21.15.12:0/4218115968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 msgr2=0x55d3e7a772c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5230f700 1 --2- 172.21.15.12:0/4218115968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a772c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5230f700 1 -- 172.21.15.12:0/4218115968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 msgr2=0x55d3e7a76c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5230f700 1 --2- 172.21.15.12:0/4218115968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a76c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5230f700 1 -- 172.21.15.12:0/4218115968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e77deb60 con 0x55d3e7a67720 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/4218115968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5130d700 1 --2- 172.21.15.12:0/4218115968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a76c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.399+0000 7f6f5230f700 1 --2- 172.21.15.12:0/4218115968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a5bd20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009fd0 tx=0x7f6f44009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3624d0d3fd81941e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f40ff9700 1 -- 172.21.15.12:0/4218115968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4400e040 con 0x55d3e7a67720 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f40ff9700 1 -- 172.21.15.12:0/4218115968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4400bd30 con 0x55d3e7a67720 2024-05-08T23:56:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f40ff9700 1 -- 172.21.15.12:0/4218115968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44005240 con 0x55d3e7a67720 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/4218115968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 msgr2=0x55d3e7a5bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/4218115968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a5bd20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009fd0 tx=0x7f6f44009520 comp rx=0 tx=0).stop 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/4218115968 shutdown_connections 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/4218115968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7a5bd20 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/4218115968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/4218115968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a76c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/4218115968 >> 172.21.15.12:0/4218115968 conn(0x55d3e79bf210 msgr2=0x55d3e7a61c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/4218115968 shutdown_connections 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.400+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/4218115968 wait complete. 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.401+0000 7f6f5ff2cb80 1 -- start start 2024-05-08T23:56:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.401+0000 7f6f5ff2cb80 1 Processor -- start 2024-05-08T23:56:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.401+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a70010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.401+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a70550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.401+0000 7f6f5ff2cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7ad0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.401+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e7a70e30 con 0x55d3e7a67720 2024-05-08T23:56:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.401+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e7ad0ba0 con 0x55d3e7a66e00 2024-05-08T23:56:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.401+0000 7f6f5ff2cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e7ad0ce0 con 0x55d3e7a61890 2024-05-08T23:56:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.401+0000 7f6f5ff2cb80 10 client.0 _ll_register_callbacks cb 0x55d3e7956b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f5130d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a70550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f51b0e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a70010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f5230f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7ad0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f51b0e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a70010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51514/0 (socket says 172.21.15.12:51514) 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f51b0e700 1 -- 172.21.15.12:0/482000651 learned_addr learned my addr 172.21.15.12:0/482000651 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f5130d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a70550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:60962/0 (socket says 172.21.15.12:60962) 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f51b0e700 1 -- 172.21.15.12:0/482000651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 msgr2=0x55d3e7a70550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a70550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f51b0e700 1 -- 172.21.15.12:0/482000651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 msgr2=0x55d3e7ad0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7ad0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f51b0e700 1 -- 172.21.15.12:0/482000651 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3c017070 con 0x55d3e7a61890 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f5130d700 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a70550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f51b0e700 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a70010 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f48099c60 tx=0x7f6f4809de00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e3b9cb31e84569 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f357fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:56:57.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f357fa700 1 -- 172.21.15.12:0/482000651 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f480a23e0 con 0x55d3e7a61890 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f357fa700 1 -- 172.21.15.12:0/482000651 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6f48099450 con 0x55d3e7a61890 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/482000651 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f3c021070 con 0x55d3e7a61890 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f357fa700 1 -- 172.21.15.12:0/482000651 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f48099610 con 0x55d3e7a61890 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f5ff2cb80 10 client.28126 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.402+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/482000651 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55d3e78323b0 con 0x55d3e7a61890 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f357fa700 1 -- 172.21.15.12:0/482000651 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f3c021070 con 0x55d3e7a61890 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f357fa700 1 -- 172.21.15.12:0/482000651 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6f48098ec0 con 0x55d3e7a61890 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f5ff2cb80 10 client.28126 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f5ff2cb80 10 client.28126 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f5ff2cb80 20 client.28126 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d3e7ad1f30 0x55d3e7ad4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f5ff2cb80 4 client.28126 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/482000651 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3e79a9ec0 con 0x55d3e7ad1f30 2024-05-08T23:56:57.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f5130d700 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d3e7ad1f30 0x55d3e7ad4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f5130d700 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d3e7ad1f30 0x55d3e7ad4350 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bc88bf08796dff2 server_cookie=80300797a1c70753 in_seq=0 out_seq=0 2024-05-08T23:56:57.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.403+0000 7f6f357fa700 10 client.28126 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:56:57.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f357fa700 1 -- 172.21.15.12:0/482000651 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3e79a9ec0 con 0x55d3e7ad1f30 2024-05-08T23:56:57.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f357fa700 10 client.28126 handle_command_reply: tid=0 2024-05-08T23:56:57.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 client.28126 shutdown 2024-05-08T23:56:57.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/482000651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 msgr2=0x55d3e7a70010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a70010 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f48099c60 tx=0x7f6f4809de00 comp rx=0 tx=0).stop 2024-05-08T23:56:57.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/482000651 shutdown_connections 2024-05-08T23:56:57.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3e7a67720 0x55d3e7ad0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d3e7ad1f30 0x55d3e7ad4350 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e7a61890 0x55d3e7a70010 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 --2- 172.21.15.12:0/482000651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d3e7a66e00 0x55d3e7a70550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f357fa700 0 client.28126 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:56:57.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/482000651 >> 172.21.15.12:0/482000651 conn(0x55d3e79bf210 msgr2=0x55d3e7a6b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/482000651 shutdown_connections 2024-05-08T23:56:57.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 1 -- 172.21.15.12:0/482000651 wait complete. 2024-05-08T23:56:57.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.404+0000 7f6f5ff2cb80 20 client.28126 trim_cache size 0 max 16384 2024-05-08T23:56:57.407 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:56:57.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 -- 172.21.15.12:0/1850213632 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f380a1290 msgr2=0x7f6f380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f380a1290 0x7f6f380a3750 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c19eab0 tx=0x7f6f3c009260 comp rx=0 tx=0).stop 2024-05-08T23:56:57.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 -- 172.21.15.12:0/1850213632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f4c068590 msgr2=0x7f6f4c1a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f4c068590 0x7f6f4c1a37c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800e9a0 tx=0x7f6f4800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 -- 172.21.15.12:0/1850213632 shutdown_connections 2024-05-08T23:56:57.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f380a1290 0x7f6f380a3750 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c105e30 0x7f6f4c19d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f4c068590 0x7f6f4c1a37c0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 --2- 172.21.15.12:0/1850213632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f4c103930 0x7f6f4c1a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 -- 172.21.15.12:0/1850213632 >> 172.21.15.12:0/1850213632 conn(0x7f6f4c076f40 msgr2=0x7f6f4c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 -- 172.21.15.12:0/1850213632 shutdown_connections 2024-05-08T23:56:57.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.405+0000 7f6f53d72700 1 -- 172.21.15.12:0/1850213632 wait complete. 2024-05-08T23:56:57.413 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:56:57.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:57 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3086177803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:56:57.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:57 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3086177803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:56:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.560+0000 7ff6d429c700 1 -- 172.21.15.12:0/4008347473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6cc1062a0 msgr2=0x7ff6cc10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.560+0000 7ff6d429c700 1 --2- 172.21.15.12:0/4008347473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6cc1062a0 0x7ff6cc10de20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400b0a0 tx=0x7ff6c400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.560+0000 7ff6d429c700 1 -- 172.21.15.12:0/4008347473 shutdown_connections 2024-05-08T23:56:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.560+0000 7ff6d429c700 1 --2- 172.21.15.12:0/4008347473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6cc1062a0 0x7ff6cc10de20 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.560+0000 7ff6d429c700 1 --2- 172.21.15.12:0/4008347473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6cc069420 0x7ff6cc069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.560+0000 7ff6d429c700 1 --2- 172.21.15.12:0/4008347473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6cc069dd0 0x7ff6cc105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.560+0000 7ff6d429c700 1 -- 172.21.15.12:0/4008347473 >> 172.21.15.12:0/4008347473 conn(0x7ff6cc076ba0 msgr2=0x7ff6cc076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.560+0000 7ff6d429c700 1 -- 172.21.15.12:0/4008347473 shutdown_connections 2024-05-08T23:56:57.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.560+0000 7ff6d429c700 1 -- 172.21.15.12:0/4008347473 wait complete. 2024-05-08T23:56:57.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d429c700 1 Processor -- start 2024-05-08T23:56:57.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d429c700 1 -- start start 2024-05-08T23:56:57.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d429c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6cc069420 0x7ff6cc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d429c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6cc1062a0 0x7ff6cc199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d429c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6cc19dfb0 0x7ff6cc19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d429c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc19ea30 con 0x7ff6cc1062a0 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d429c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc19eba0 con 0x7ff6cc069420 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d429c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc19ed60 con 0x7ff6cc19dfb0 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d1837700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6cc1062a0 0x7ff6cc199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d2038700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6cc069420 0x7ff6cc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d2038700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6cc069420 0x7ff6cc199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:60974/0 (socket says 172.21.15.12:60974) 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d2038700 1 -- 172.21.15.12:0/1389831251 learned_addr learned my addr 172.21.15.12:0/1389831251 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d2839700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6cc19dfb0 0x7ff6cc19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d2038700 1 -- 172.21.15.12:0/1389831251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6cc19dfb0 msgr2=0x7ff6cc19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d2038700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6cc19dfb0 0x7ff6cc19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d2038700 1 -- 172.21.15.12:0/1389831251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6cc1062a0 msgr2=0x7ff6cc199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d2038700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6cc1062a0 0x7ff6cc199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d2038700 1 -- 172.21.15.12:0/1389831251 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c4009d00 con 0x7ff6cc069420 2024-05-08T23:56:57.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.561+0000 7ff6d2839700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6cc19dfb0 0x7ff6cc19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:57.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.562+0000 7ff6d2038700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6cc069420 0x7ff6cc199720 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00e9a0 tx=0x7ff6bc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.562+0000 7ff6c37fe700 1 -- 172.21.15.12:0/1389831251 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc009790 con 0x7ff6cc069420 2024-05-08T23:56:57.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.562+0000 7ff6d429c700 1 -- 172.21.15.12:0/1389831251 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc1a62c0 con 0x7ff6cc069420 2024-05-08T23:56:57.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.562+0000 7ff6c37fe700 1 -- 172.21.15.12:0/1389831251 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6bc010430 con 0x7ff6cc069420 2024-05-08T23:56:57.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.562+0000 7ff6c37fe700 1 -- 172.21.15.12:0/1389831251 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc018680 con 0x7ff6cc069420 2024-05-08T23:56:57.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.562+0000 7ff6d429c700 1 -- 172.21.15.12:0/1389831251 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc1a67f0 con 0x7ff6cc069420 2024-05-08T23:56:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.564+0000 7ff6c37fe700 1 -- 172.21.15.12:0/1389831251 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff6bc0105a0 con 0x7ff6cc069420 2024-05-08T23:56:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.565+0000 7ff6c37fe700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff6b80a1310 0x7ff6b80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.565+0000 7ff6d1837700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff6b80a1310 0x7ff6b80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.565+0000 7ff6c37fe700 1 -- 172.21.15.12:0/1389831251 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff6bc014070 con 0x7ff6cc069420 2024-05-08T23:56:57.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.565+0000 7ff6e0456b80 1 Processor -- start 2024-05-08T23:56:57.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6d1837700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff6b80a1310 0x7ff6b80a37d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc19ac20 tx=0x7ff6c8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6e0456b80 1 -- start start 2024-05-08T23:56:57.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11ee00 0x55bb9d11f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 0x55bb9d119350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 0x55bb9d119cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d070150 con 0x55bb9d11ee00 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d06db10 con 0x55bb9d119890 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d069400 con 0x55bb9d11f720 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6d2038700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11ee00 0x55bb9d11f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6d2038700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11ee00 0x55bb9d11f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44372/0 (socket says 172.21.15.12:44372) 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6d2038700 1 -- 172.21.15.12:0/3088848235 learned_addr learned my addr 172.21.15.12:0/3088848235 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6d2839700 1 --2- 172.21.15.12:0/3088848235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 0x55bb9d119cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.566+0000 7ff6d1837700 1 --2- 172.21.15.12:0/3088848235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 0x55bb9d119350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.567+0000 7ff6d2038700 1 -- 172.21.15.12:0/3088848235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 msgr2=0x55bb9d119350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.567+0000 7ff6d2038700 1 --2- 172.21.15.12:0/3088848235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 0x55bb9d119350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.567+0000 7ff6d2038700 1 -- 172.21.15.12:0/3088848235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 msgr2=0x55bb9d119cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.567+0000 7ff6d2038700 1 --2- 172.21.15.12:0/3088848235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 0x55bb9d119cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.567+0000 7ff6d2038700 1 -- 172.21.15.12:0/3088848235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb9ceea3b0 con 0x55bb9d11ee00 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.567+0000 7ff6d2038700 1 --2- 172.21.15.12:0/3088848235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11ee00 0x55bb9d11f1e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0969b0 tx=0x7ff6bc099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc2753888353dbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.567+0000 7ff6c0da4700 1 -- 172.21.15.12:0/3088848235 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc099fa0 con 0x55bb9d11ee00 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.567+0000 7ff6c0da4700 1 -- 172.21.15.12:0/3088848235 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6bc09dd70 con 0x55bb9d11ee00 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.567+0000 7ff6c0da4700 1 -- 172.21.15.12:0/3088848235 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc099690 con 0x55bb9d11ee00 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 -- 172.21.15.12:0/3088848235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11ee00 msgr2=0x55bb9d11f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/3088848235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11ee00 0x55bb9d11f1e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0969b0 tx=0x7ff6bc099bb0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 -- 172.21.15.12:0/3088848235 shutdown_connections 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/3088848235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 0x55bb9d119cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/3088848235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11ee00 0x55bb9d11f1e0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/3088848235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 0x55bb9d119350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 -- 172.21.15.12:0/3088848235 >> 172.21.15.12:0/3088848235 conn(0x55bb9d077210 msgr2=0x55bb9d113df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 -- 172.21.15.12:0/3088848235 shutdown_connections 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 -- 172.21.15.12:0/3088848235 wait complete. 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 -- start start 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.568+0000 7ff6e0456b80 1 Processor -- start 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 0x55bb9d122ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11ee00 0x55bb9d123010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 0x55bb9d123550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d005560 con 0x55bb9d11f720 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d005200 con 0x55bb9d119890 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9ceec6a0 con 0x55bb9d11ee00 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6e0456b80 10 client.0 _ll_register_callbacks cb 0x55bb9d00eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2839700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 0x55bb9d123550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2038700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 0x55bb9d122ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2038700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 0x55bb9d122ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:60996/0 (socket says 172.21.15.12:60996) 2024-05-08T23:56:57.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2839700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 0x55bb9d123550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44386/0 (socket says 172.21.15.12:44386) 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2038700 1 -- 172.21.15.12:0/2759737204 learned_addr learned my addr 172.21.15.12:0/2759737204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d1837700 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11ee00 0x55bb9d123010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2038700 1 -- 172.21.15.12:0/2759737204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11ee00 msgr2=0x55bb9d123010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2038700 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11ee00 0x55bb9d123010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2038700 1 -- 172.21.15.12:0/2759737204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 msgr2=0x55bb9d123550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2038700 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 0x55bb9d123550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2038700 1 -- 172.21.15.12:0/2759737204 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc1a67f0 con 0x55bb9d119890 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2839700 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 0x55bb9d123550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6d2038700 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 0x55bb9d122ad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc099c40 tx=0x7ff6bc010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7151456a58054cc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.569+0000 7ff6b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/2759737204 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc09a3b0 con 0x55bb9d119890 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/2759737204 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff6bc0a1f30 con 0x55bb9d119890 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6e0456b80 1 -- 172.21.15.12:0/2759737204 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb9cfc2270 con 0x55bb9d119890 2024-05-08T23:56:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/2759737204 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6bc0b82c0 con 0x55bb9d119890 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6e0456b80 10 client.28148 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6e0456b80 1 -- 172.21.15.12:0/2759737204 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55bb9ce96b60 con 0x55bb9d119890 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/2759737204 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb9cfc2270 con 0x55bb9d119890 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/2759737204 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff6bc098f30 con 0x55bb9d119890 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6e0456b80 10 client.28148 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6e0456b80 10 client.28148 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6e0456b80 20 client.28148 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bb9d132e60 0x55bb9d135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6e0456b80 4 client.28148 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6e0456b80 1 -- 172.21.15.12:0/2759737204 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb9d0e8810 con 0x55bb9d132e60 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.570+0000 7ff6d1837700 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bb9d132e60 0x55bb9d135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.571+0000 7ff6d1837700 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bb9d132e60 0x55bb9d135280 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc6218a466fabf99 server_cookie=802d29dfa97dc820 in_seq=0 out_seq=0 2024-05-08T23:56:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.571+0000 7ff6b5ffb700 10 client.28148 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/2759737204 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb9d0e8810 con 0x55bb9d132e60 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6b5ffb700 10 client.28148 handle_command_reply: tid=0 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 client.28148 shutdown 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 -- 172.21.15.12:0/2759737204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 msgr2=0x55bb9d122ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 0x55bb9d122ad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc099c40 tx=0x7ff6bc010ab0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 -- 172.21.15.12:0/2759737204 shutdown_connections 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bb9d132e60 0x55bb9d135280 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 0x55bb9d123550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d119890 0x55bb9d122ad0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/2759737204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11ee00 0x55bb9d123010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 -- 172.21.15.12:0/2759737204 >> 172.21.15.12:0/2759737204 conn(0x55bb9d077210 msgr2=0x55bb9d1135a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6b5ffb700 0 client.28148 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:56:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 -- 172.21.15.12:0/2759737204 shutdown_connections 2024-05-08T23:56:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 1 -- 172.21.15.12:0/2759737204 wait complete. 2024-05-08T23:56:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.572+0000 7ff6e0456b80 20 client.28148 trim_cache size 0 max 16384 2024-05-08T23:56:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6e0456b80 1 Processor -- start 2024-05-08T23:56:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6e0456b80 1 -- start start 2024-05-08T23:56:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d119890 0x55bb9d119350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 0x55bb9d113d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 0x55bb9d123370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d10c0b0 con 0x55bb9d11f720 2024-05-08T23:56:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d115070 con 0x55bb9d11ee00 2024-05-08T23:56:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d110dc0 con 0x55bb9d119890 2024-05-08T23:56:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6d2038700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d119890 0x55bb9d119350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6d1837700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 0x55bb9d113d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6d2038700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d119890 0x55bb9d119350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51558/0 (socket says 172.21.15.12:51558) 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6d2839700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 0x55bb9d123370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.587+0000 7ff6d2038700 1 -- 172.21.15.12:0/1960229273 learned_addr learned my addr 172.21.15.12:0/1960229273 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.588+0000 7ff6d2038700 1 -- 172.21.15.12:0/1960229273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 msgr2=0x55bb9d113d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.588+0000 7ff6d2038700 1 --2- 172.21.15.12:0/1960229273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 0x55bb9d113d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.588+0000 7ff6d2038700 1 -- 172.21.15.12:0/1960229273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 msgr2=0x55bb9d123370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.588+0000 7ff6d2038700 1 --2- 172.21.15.12:0/1960229273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 0x55bb9d123370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.588+0000 7ff6d2038700 1 -- 172.21.15.12:0/1960229273 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb9d119f40 con 0x55bb9d119890 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.588+0000 7ff6d2038700 1 --2- 172.21.15.12:0/1960229273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d119890 0x55bb9d119350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0969b0 tx=0x7ff6bc099bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f8216060ce1ce7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.588+0000 7ff6c15a5700 1 -- 172.21.15.12:0/1960229273 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc09a640 con 0x55bb9d119890 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.588+0000 7ff6c15a5700 1 -- 172.21.15.12:0/1960229273 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6bc0b82c0 con 0x55bb9d119890 2024-05-08T23:56:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.588+0000 7ff6c15a5700 1 -- 172.21.15.12:0/1960229273 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0a2190 con 0x55bb9d119890 2024-05-08T23:56:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1960229273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d119890 msgr2=0x55bb9d119350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/1960229273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d119890 0x55bb9d119350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0969b0 tx=0x7ff6bc099bb0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1960229273 shutdown_connections 2024-05-08T23:56:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/1960229273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d11f720 0x55bb9d123370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/1960229273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d119890 0x55bb9d119350 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/1960229273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 0x55bb9d113d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1960229273 >> 172.21.15.12:0/1960229273 conn(0x55bb9d077210 msgr2=0x55bb9d1213a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1960229273 shutdown_connections 2024-05-08T23:56:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1960229273 wait complete. 2024-05-08T23:56:57.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 -- start start 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.589+0000 7ff6e0456b80 1 Processor -- start 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d119890 0x55bb9d129390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 0x55bb9d1298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6e0456b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 0x55bb9d129e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d188320 con 0x55bb9d119890 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d188460 con 0x55bb9d11ee00 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6e0456b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9d1885a0 con 0x55bb9d11f720 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6e0456b80 10 client.0 _ll_register_callbacks cb 0x55bb9d00eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d2038700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d119890 0x55bb9d129390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d2839700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 0x55bb9d129e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d1837700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 0x55bb9d1298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d1837700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 0x55bb9d1298d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:32780/0 (socket says 172.21.15.12:32780) 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d2839700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 0x55bb9d129e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51568/0 (socket says 172.21.15.12:51568) 2024-05-08T23:56:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d1837700 1 -- 172.21.15.12:0/1873407103 learned_addr learned my addr 172.21.15.12:0/1873407103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d1837700 1 -- 172.21.15.12:0/1873407103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 msgr2=0x55bb9d129e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d1837700 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 0x55bb9d129e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d1837700 1 -- 172.21.15.12:0/1873407103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d119890 msgr2=0x55bb9d129390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d1837700 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d119890 0x55bb9d129390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d1837700 1 -- 172.21.15.12:0/1873407103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb9ce96b60 con 0x55bb9d11ee00 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d2038700 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d119890 0x55bb9d129390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6d1837700 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 0x55bb9d1298d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8005950 tx=0x7ff6c8013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7166354014cfb34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.590+0000 7ff6b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/1873407103 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8024040 con 0x55bb9d11ee00 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/1873407103 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff6c80296a0 con 0x55bb9d11ee00 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1873407103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6bc09dd70 con 0x55bb9d11ee00 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/1873407103 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6c8029860 con 0x55bb9d11ee00 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6e0456b80 10 client.28154 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:57.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1873407103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55bb9d02f560 con 0x55bb9d11ee00 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/1873407103 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6bc09dd70 con 0x55bb9d11ee00 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/1873407103 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff6c803a7d0 con 0x55bb9d11ee00 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6e0456b80 10 client.28154 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6e0456b80 10 client.28154 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6e0456b80 20 client.28154 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bb9d189de0 0x55bb9d18c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6e0456b80 4 client.28154 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.591+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1873407103 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bb9d061ec0 con 0x55bb9d189de0 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6d2038700 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bb9d189de0 0x55bb9d18c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6d2038700 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bb9d189de0 0x55bb9d18c200 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d20a729a8fad9819 server_cookie=f08bd276952142a1 in_seq=0 out_seq=0 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6b5ffb700 10 client.28154 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6b5ffb700 1 -- 172.21.15.12:0/1873407103 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bb9d061ec0 con 0x55bb9d189de0 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6b5ffb700 10 client.28154 handle_command_reply: tid=0 2024-05-08T23:56:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 client.28154 shutdown 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1873407103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 msgr2=0x55bb9d1298d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 0x55bb9d1298d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8005950 tx=0x7ff6c8013040 comp rx=0 tx=0).stop 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1873407103 shutdown_connections 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bb9d189de0 0x55bb9d18c200 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb9d11f720 0x55bb9d129e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6b5ffb700 0 client.28154 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb9d119890 0x55bb9d129390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 --2- 172.21.15.12:0/1873407103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bb9d11ee00 0x55bb9d1298d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1873407103 >> 172.21.15.12:0/1873407103 conn(0x55bb9d077210 msgr2=0x55bb9d11dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1873407103 shutdown_connections 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.592+0000 7ff6e0456b80 1 -- 172.21.15.12:0/1873407103 wait complete. 2024-05-08T23:56:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6e0456b80 20 client.28154 trim_cache size 0 max 16384 2024-05-08T23:56:57.594 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:56:57.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 -- 172.21.15.12:0/1389831251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff6b80a1310 msgr2=0x7ff6b80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff6b80a1310 0x7ff6b80a37d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc19ac20 tx=0x7ff6c8009260 comp rx=0 tx=0).stop 2024-05-08T23:56:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 -- 172.21.15.12:0/1389831251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6cc069420 msgr2=0x7ff6cc199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6cc069420 0x7ff6cc199720 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00e9a0 tx=0x7ff6bc00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 -- 172.21.15.12:0/1389831251 shutdown_connections 2024-05-08T23:56:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff6b80a1310 0x7ff6b80a37d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6cc19dfb0 0x7ff6cc19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6cc069420 0x7ff6cc199720 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 --2- 172.21.15.12:0/1389831251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6cc1062a0 0x7ff6cc199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 -- 172.21.15.12:0/1389831251 >> 172.21.15.12:0/1389831251 conn(0x7ff6cc076ba0 msgr2=0x7ff6cc10c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 -- 172.21.15.12:0/1389831251 shutdown_connections 2024-05-08T23:56:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.593+0000 7ff6d429c700 1 -- 172.21.15.12:0/1389831251 wait complete. 2024-05-08T23:56:57.600 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:57 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3086177803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:56:57.601 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:56:57.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.751+0000 7f4197531700 1 -- 172.21.15.12:0/1613214480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41900fff80 msgr2=0x7f4190100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.751+0000 7f4197531700 1 --2- 172.21.15.12:0/1613214480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41900fff80 0x7f4190100360 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f418c009990 tx=0x7f418c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.752+0000 7f4197531700 1 -- 172.21.15.12:0/1613214480 shutdown_connections 2024-05-08T23:56:57.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.752+0000 7f4197531700 1 --2- 172.21.15.12:0/1613214480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41901050a0 0x7f419010cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.752+0000 7f4197531700 1 --2- 172.21.15.12:0/1613214480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41900fff80 0x7f4190100360 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.752+0000 7f4197531700 1 --2- 172.21.15.12:0/1613214480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4190100930 0x7f4190104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.752+0000 7f4197531700 1 -- 172.21.15.12:0/1613214480 >> 172.21.15.12:0/1613214480 conn(0x7f41900fb800 msgr2=0x7f41900fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.752+0000 7f4197531700 1 -- 172.21.15.12:0/1613214480 shutdown_connections 2024-05-08T23:56:57.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.752+0000 7f4197531700 1 -- 172.21.15.12:0/1613214480 wait complete. 2024-05-08T23:56:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.752+0000 7f4197531700 1 Processor -- start 2024-05-08T23:56:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f4197531700 1 -- start start 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f4197531700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41900fff80 0x7f4190198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f4197531700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41901050a0 0x7f4190198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f4197531700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4190198bc0 0x7f41901a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f4197531700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41901a38a0 con 0x7f41900fff80 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f4197531700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41901a3a10 con 0x7f4190198bc0 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f4197531700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41901a3bd0 con 0x7f41901050a0 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f41952cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41900fff80 0x7f4190198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f41952cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41900fff80 0x7f4190198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44416/0 (socket says 172.21.15.12:44416) 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f41952cd700 1 -- 172.21.15.12:0/2522193062 learned_addr learned my addr 172.21.15.12:0/2522193062 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f4194acc700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41901050a0 0x7f4190198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f4195ace700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4190198bc0 0x7f41901a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f41952cd700 1 -- 172.21.15.12:0/2522193062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41901050a0 msgr2=0x7f4190198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f41952cd700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41901050a0 0x7f4190198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f41952cd700 1 -- 172.21.15.12:0/2522193062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4190198bc0 msgr2=0x7f41901a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f41952cd700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4190198bc0 0x7f41901a31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f41952cd700 1 -- 172.21.15.12:0/2522193062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c0095f0 con 0x7f41900fff80 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.753+0000 7f4195ace700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4190198bc0 0x7f41901a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.754+0000 7f41952cd700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41900fff80 0x7f4190198140 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f418c004760 tx=0x7f418c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.754+0000 7f41867fc700 1 -- 172.21.15.12:0/2522193062 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c01d070 con 0x7f41900fff80 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.754+0000 7f4197531700 1 -- 172.21.15.12:0/2522193062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41901a3e50 con 0x7f41900fff80 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.754+0000 7f41867fc700 1 -- 172.21.15.12:0/2522193062 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f418c022470 con 0x7f41900fff80 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.754+0000 7f41867fc700 1 -- 172.21.15.12:0/2522193062 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c017650 con 0x7f41900fff80 2024-05-08T23:56:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.754+0000 7f4197531700 1 -- 172.21.15.12:0/2522193062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41901a4270 con 0x7f41900fff80 2024-05-08T23:56:57.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.757+0000 7f41867fc700 1 -- 172.21.15.12:0/2522193062 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f418c0225e0 con 0x7f41900fff80 2024-05-08T23:56:57.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.758+0000 7f41867fc700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f417c0a1280 0x7f417c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.758+0000 7f41867fc700 1 -- 172.21.15.12:0/2522193062 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f418c0cdeb0 con 0x7f41900fff80 2024-05-08T23:56:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.758+0000 7f41a36ebb80 1 Processor -- start 2024-05-08T23:56:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.758+0000 7f4194acc700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f417c0a1280 0x7f417c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.758+0000 7f41a36ebb80 1 -- start start 2024-05-08T23:56:57.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.758+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.758+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f55720 0x5609a9f4f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.758+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f4f890 0x5609a9f4fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9ea6150 con 0x5609a9f4f890 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9ea3b10 con 0x5609a9f54e00 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9e9f400 con 0x5609a9f55720 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f4194acc700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f417c0a1280 0x7f417c0a3740 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41900699a0 tx=0x7f4180009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f41952cd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f4194acc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f55720 0x5609a9f4f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f41952cd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:32806/0 (socket says 172.21.15.12:32806) 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f41952cd700 1 -- 172.21.15.12:0/1190506791 learned_addr learned my addr 172.21.15.12:0/1190506791 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f4194acc700 1 --2- 172.21.15.12:0/1190506791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f55720 0x5609a9f4f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51588/0 (socket says 172.21.15.12:51588) 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f4195ace700 1 --2- 172.21.15.12:0/1190506791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f4f890 0x5609a9f4fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f4194acc700 1 -- 172.21.15.12:0/1190506791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 msgr2=0x5609a9f551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f4194acc700 1 --2- 172.21.15.12:0/1190506791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f4194acc700 1 -- 172.21.15.12:0/1190506791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f4f890 msgr2=0x5609a9f4fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f4194acc700 1 --2- 172.21.15.12:0/1190506791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f4f890 0x5609a9f4fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f4194acc700 1 -- 172.21.15.12:0/1190506791 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a9d203b0 con 0x5609a9f55720 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f4194acc700 1 --2- 172.21.15.12:0/1190506791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f55720 0x5609a9f4f350 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4180011c40 tx=0x7f4180014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd55b37815f16a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f417b7fe700 1 -- 172.21.15.12:0/1190506791 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418000eb00 con 0x5609a9f55720 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f417b7fe700 1 -- 172.21.15.12:0/1190506791 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4180021070 con 0x5609a9f55720 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.759+0000 7f417b7fe700 1 -- 172.21.15.12:0/1190506791 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180005380 con 0x5609a9f55720 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.760+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1190506791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f55720 msgr2=0x5609a9f4f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.760+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/1190506791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f55720 0x5609a9f4f350 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4180011c40 tx=0x7f4180014000 comp rx=0 tx=0).stop 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.760+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1190506791 shutdown_connections 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.760+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/1190506791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f4f890 0x5609a9f4fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.760+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/1190506791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.760+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/1190506791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f55720 0x5609a9f4f350 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.760+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1190506791 >> 172.21.15.12:0/1190506791 conn(0x5609a9ead210 msgr2=0x5609a9f49df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.760+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1190506791 shutdown_connections 2024-05-08T23:56:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.760+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1190506791 wait complete. 2024-05-08T23:56:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.761+0000 7f41a36ebb80 1 -- start start 2024-05-08T23:56:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.761+0000 7f41a36ebb80 1 Processor -- start 2024-05-08T23:56:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.761+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f58c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.761+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f54e00 0x5609a9f591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.761+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 0x5609a9f596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.761+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9e3b560 con 0x5609a9f54e00 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9e3b200 con 0x5609a9f55720 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9d226a0 con 0x5609a9f4f890 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f41a36ebb80 10 client.0 _ll_register_callbacks cb 0x5609a9e44b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f4194acc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f54e00 0x5609a9f591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f4194acc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f54e00 0x5609a9f591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44434/0 (socket says 172.21.15.12:44434) 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f4194acc700 1 -- 172.21.15.12:0/1788589446 learned_addr learned my addr 172.21.15.12:0/1788589446 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f41952cd700 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f58c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f4195ace700 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 0x5609a9f596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f41952cd700 1 -- 172.21.15.12:0/1788589446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 msgr2=0x5609a9f596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f41952cd700 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 0x5609a9f596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f41952cd700 1 -- 172.21.15.12:0/1788589446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f54e00 msgr2=0x5609a9f591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f41952cd700 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f54e00 0x5609a9f591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f41952cd700 1 -- 172.21.15.12:0/1788589446 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4180017070 con 0x5609a9f4f890 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f41952cd700 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f58c60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f418c0a2ef0 tx=0x7f418c09ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813f9670e65f5e23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.762+0000 7f4178ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f4178ff9700 1 -- 172.21.15.12:0/1788589446 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c09b730 con 0x5609a9f4f890 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f4178ff9700 1 -- 172.21.15.12:0/1788589446 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f418c09b030 con 0x5609a9f4f890 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1788589446 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609a9df8270 con 0x5609a9f4f890 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f4178ff9700 1 -- 172.21.15.12:0/1788589446 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f418c09ee60 con 0x5609a9f4f890 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f41a36ebb80 10 client.28156 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1788589446 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5609a9cccb60 con 0x5609a9f4f890 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f4178ff9700 1 -- 172.21.15.12:0/1788589446 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609a9df8270 con 0x5609a9f4f890 2024-05-08T23:56:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f4178ff9700 1 -- 172.21.15.12:0/1788589446 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f418c0a2f90 con 0x5609a9f4f890 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f41a36ebb80 10 client.28156 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f41a36ebb80 10 client.28156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f41a36ebb80 20 client.28156 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a9f68e70 0x5609a9f6b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f41a36ebb80 4 client.28156 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.763+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1788589446 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609a9f1e810 con 0x5609a9f68e70 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.764+0000 7f4194acc700 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a9f68e70 0x5609a9f6b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.764+0000 7f4194acc700 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a9f68e70 0x5609a9f6b290 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738f649047eac20d server_cookie=6b8867026225c4c5 in_seq=0 out_seq=0 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.764+0000 7f4178ff9700 10 client.28156 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f4178ff9700 1 -- 172.21.15.12:0/1788589446 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609a9f1e810 con 0x5609a9f68e70 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f4178ff9700 10 client.28156 handle_command_reply: tid=0 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 client.28156 shutdown 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1788589446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 msgr2=0x5609a9f58c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f58c60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f418c0a2ef0 tx=0x7f418c09ab70 comp rx=0 tx=0).stop 2024-05-08T23:56:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1788589446 shutdown_connections 2024-05-08T23:56:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a9f68e70 0x5609a9f6b290 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 0x5609a9f596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f4178ff9700 0 client.28156 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f58c60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/1788589446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f54e00 0x5609a9f591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1788589446 >> 172.21.15.12:0/1788589446 conn(0x5609a9ead210 msgr2=0x5609a9f495a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1788589446 shutdown_connections 2024-05-08T23:56:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 1 -- 172.21.15.12:0/1788589446 wait complete. 2024-05-08T23:56:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.765+0000 7f41a36ebb80 20 client.28156 trim_cache size 0 max 16384 2024-05-08T23:56:57.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f41a36ebb80 1 Processor -- start 2024-05-08T23:56:57.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f41a36ebb80 1 -- start start 2024-05-08T23:56:57.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f4f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f49d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f55720 0x5609a9f59500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9f4ae80 con 0x5609a9f55720 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9f4be60 con 0x5609a9f54e00 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9f46dc0 con 0x5609a9f4f890 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f4194acc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f49d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f4195ace700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f55720 0x5609a9f59500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f4194acc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f49d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:32814/0 (socket says 172.21.15.12:32814) 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f4194acc700 1 -- 172.21.15.12:0/2331810881 learned_addr learned my addr 172.21.15.12:0/2331810881 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.780+0000 7f41952cd700 1 --2- 172.21.15.12:0/2331810881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f4f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.781+0000 7f4194acc700 1 -- 172.21.15.12:0/2331810881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 msgr2=0x5609a9f4f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.781+0000 7f4194acc700 1 --2- 172.21.15.12:0/2331810881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f4f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.781+0000 7f4194acc700 1 -- 172.21.15.12:0/2331810881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f55720 msgr2=0x5609a9f59500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.781+0000 7f4194acc700 1 --2- 172.21.15.12:0/2331810881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f55720 0x5609a9f59500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.781+0000 7f4194acc700 1 -- 172.21.15.12:0/2331810881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a9e65560 con 0x5609a9f54e00 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.781+0000 7f4194acc700 1 --2- 172.21.15.12:0/2331810881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f49d20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f418000efa0 tx=0x7f418001f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6da34d67c1d20f1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.781+0000 7f417bfff700 1 -- 172.21.15.12:0/2331810881 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418002b040 con 0x5609a9f54e00 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.781+0000 7f417bfff700 1 -- 172.21.15.12:0/2331810881 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f418000ee00 con 0x5609a9f54e00 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.781+0000 7f417bfff700 1 -- 172.21.15.12:0/2331810881 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180004b90 con 0x5609a9f54e00 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.782+0000 7f41a36ebb80 1 -- 172.21.15.12:0/2331810881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 msgr2=0x5609a9f49d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.782+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/2331810881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f49d20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f418000efa0 tx=0x7f418001f040 comp rx=0 tx=0).stop 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.782+0000 7f41a36ebb80 1 -- 172.21.15.12:0/2331810881 shutdown_connections 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.782+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/2331810881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f55720 0x5609a9f59500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.782+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/2331810881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f4f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.782+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/2331810881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f54e00 0x5609a9f49d20 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f418000efa0 tx=0x7f418001f040 comp rx=0 tx=0).stop 2024-05-08T23:56:57.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.782+0000 7f41a36ebb80 1 -- 172.21.15.12:0/2331810881 >> 172.21.15.12:0/2331810881 conn(0x5609a9ead210 msgr2=0x5609a9f57550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.782+0000 7f41a36ebb80 1 -- 172.21.15.12:0/2331810881 shutdown_connections 2024-05-08T23:56:57.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.782+0000 7f41a36ebb80 1 -- 172.21.15.12:0/2331810881 wait complete. 2024-05-08T23:56:57.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.783+0000 7f41a36ebb80 1 -- start start 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.783+0000 7f41a36ebb80 1 Processor -- start 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.783+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f5e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.783+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 0x5609a9f5e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.783+0000 7f41a36ebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f5ebd0 0x5609a9fc27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.783+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9fc2d20 con 0x5609a9f5ebd0 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.783+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9fc2e90 con 0x5609a9f55720 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.783+0000 7f41a36ebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a9fc3050 con 0x5609a9f4f890 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f41a36ebb80 10 client.0 _ll_register_callbacks cb 0x5609a9e44b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4194acc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 0x5609a9f5e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4195ace700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f5ebd0 0x5609a9fc27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4194acc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 0x5609a9f5e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:32818/0 (socket says 172.21.15.12:32818) 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4194acc700 1 -- 172.21.15.12:0/3129774402 learned_addr learned my addr 172.21.15.12:0/3129774402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f41952cd700 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f5e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4194acc700 1 -- 172.21.15.12:0/3129774402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 msgr2=0x5609a9f5e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4194acc700 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f5e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4194acc700 1 -- 172.21.15.12:0/3129774402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f5ebd0 msgr2=0x5609a9fc27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4194acc700 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f5ebd0 0x5609a9fc27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4194acc700 1 -- 172.21.15.12:0/3129774402 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609a9d203b0 con 0x5609a9f55720 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4195ace700 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f5ebd0 0x5609a9fc27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4194acc700 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 0x5609a9f5e690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f418000efd0 tx=0x7f418000e900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7fe56c6d5efb472 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4178ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4178ff9700 1 -- 172.21.15.12:0/3129774402 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418002b040 con 0x5609a9f55720 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f4178ff9700 1 -- 172.21.15.12:0/3129774402 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4180004240 con 0x5609a9f55720 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.784+0000 7f41a36ebb80 1 -- 172.21.15.12:0/3129774402 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4180021070 con 0x5609a9f55720 2024-05-08T23:56:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f4178ff9700 1 -- 172.21.15.12:0/3129774402 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f418000eaf0 con 0x5609a9f55720 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f41a36ebb80 10 client.28172 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f41a36ebb80 1 -- 172.21.15.12:0/3129774402 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5609a9d24cd0 con 0x5609a9f55720 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f4178ff9700 1 -- 172.21.15.12:0/3129774402 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4180021070 con 0x5609a9f55720 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f4178ff9700 1 -- 172.21.15.12:0/3129774402 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f418003a070 con 0x5609a9f55720 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f41a36ebb80 10 client.28172 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f41a36ebb80 10 client.28172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f41a36ebb80 20 client.28172 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a9fc4380 0x5609a9fc67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f41a36ebb80 4 client.28172 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f41a36ebb80 1 -- 172.21.15.12:0/3129774402 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609a9e97ec0 con 0x5609a9fc4380 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.785+0000 7f41952cd700 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a9fc4380 0x5609a9fc67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f4178ff9700 10 client.28172 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41952cd700 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a9fc4380 0x5609a9fc67a0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee35a07c4ca7fa6 server_cookie=a1aa4ab55f76ea56 in_seq=0 out_seq=0 2024-05-08T23:56:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f4178ff9700 1 -- 172.21.15.12:0/3129774402 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609a9e97ec0 con 0x5609a9fc4380 2024-05-08T23:56:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f4178ff9700 10 client.28172 handle_command_reply: tid=0 2024-05-08T23:56:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 client.28172 shutdown 2024-05-08T23:56:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 -- 172.21.15.12:0/3129774402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 msgr2=0x5609a9f5e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 0x5609a9f5e690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f418000efd0 tx=0x7f418000e900 comp rx=0 tx=0).stop 2024-05-08T23:56:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 -- 172.21.15.12:0/3129774402 shutdown_connections 2024-05-08T23:56:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a9f5ebd0 0x5609a9fc27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a9f4f890 0x5609a9f5e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a9fc4380 0x5609a9fc67a0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 --2- 172.21.15.12:0/3129774402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a9f55720 0x5609a9f5e690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f4178ff9700 0 client.28172 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 -- 172.21.15.12:0/3129774402 >> 172.21.15.12:0/3129774402 conn(0x5609a9ead210 msgr2=0x5609a9f53d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 -- 172.21.15.12:0/3129774402 shutdown_connections 2024-05-08T23:56:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 1 -- 172.21.15.12:0/3129774402 wait complete. 2024-05-08T23:56:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.786+0000 7f41a36ebb80 20 client.28172 trim_cache size 0 max 16384 2024-05-08T23:56:57.788 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:56:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 -- 172.21.15.12:0/2522193062 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f417c0a1280 msgr2=0x7f417c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f417c0a1280 0x7f417c0a3740 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41900699a0 tx=0x7f4180009260 comp rx=0 tx=0).stop 2024-05-08T23:56:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 -- 172.21.15.12:0/2522193062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41900fff80 msgr2=0x7f4190198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41900fff80 0x7f4190198140 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f418c004760 tx=0x7f418c004790 comp rx=0 tx=0).stop 2024-05-08T23:56:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 -- 172.21.15.12:0/2522193062 shutdown_connections 2024-05-08T23:56:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f417c0a1280 0x7f417c0a3740 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4190198bc0 0x7f41901a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41900fff80 0x7f4190198140 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 --2- 172.21.15.12:0/2522193062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41901050a0 0x7f4190198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 -- 172.21.15.12:0/2522193062 >> 172.21.15.12:0/2522193062 conn(0x7f41900fb800 msgr2=0x7f41900fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 -- 172.21.15.12:0/2522193062 shutdown_connections 2024-05-08T23:56:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:57.787+0000 7f4197531700 1 -- 172.21.15.12:0/2522193062 wait complete. 2024-05-08T23:56:57.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 591046 6.50 MB/sec execute 413 sec latency 26.657 ms 2024-05-08T23:56:58.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:58 smithi012 ceph-mon[42503]: pgmap v333: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-05-08T23:56:58.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:58 smithi204 ceph-mon[47373]: pgmap v333: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-05-08T23:56:58.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:58 smithi160 ceph-mon[46152]: pgmap v333: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-05-08T23:56:58.797 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:56:58.797 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:56:58.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 591312 6.48 MB/sec execute 414 sec latency 822.072 ms 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.953+0000 7f546025b700 1 -- 172.21.15.12:0/221720254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54581062a0 msgr2=0x7f545810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.953+0000 7f546025b700 1 --2- 172.21.15.12:0/221720254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54581062a0 0x7f545810de20 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f545000b0a0 tx=0x7f545000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.954+0000 7f546025b700 1 -- 172.21.15.12:0/221720254 shutdown_connections 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.954+0000 7f546025b700 1 --2- 172.21.15.12:0/221720254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54581062a0 0x7f545810de20 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.954+0000 7f546025b700 1 --2- 172.21.15.12:0/221720254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5458101180 0x7f5458101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.954+0000 7f546025b700 1 --2- 172.21.15.12:0/221720254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5458101b30 0x7f5458105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.954+0000 7f546025b700 1 -- 172.21.15.12:0/221720254 >> 172.21.15.12:0/221720254 conn(0x7f54580fc9e0 msgr2=0x7f54580fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.954+0000 7f546025b700 1 -- 172.21.15.12:0/221720254 shutdown_connections 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.954+0000 7f546025b700 1 -- 172.21.15.12:0/221720254 wait complete. 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.954+0000 7f546025b700 1 Processor -- start 2024-05-08T23:56:58.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.954+0000 7f546025b700 1 -- start start 2024-05-08T23:56:58.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.954+0000 7f546025b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5458101180 0x7f5458197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f546025b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54581062a0 0x7f5458197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f546025b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f545819be30 0x7f545819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f546025b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545819c8b0 con 0x7f5458101180 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f546025b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545819ca20 con 0x7f545819be30 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f546025b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545819cbe0 con 0x7f54581062a0 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545dff7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5458101180 0x7f5458197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545dff7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5458101180 0x7f5458197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54498/0 (socket says 172.21.15.12:54498) 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545dff7700 1 -- 172.21.15.12:0/3016745055 learned_addr learned my addr 172.21.15.12:0/3016745055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545e7f8700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f545819be30 0x7f545819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545d7f6700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54581062a0 0x7f5458197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545dff7700 1 -- 172.21.15.12:0/3016745055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54581062a0 msgr2=0x7f5458197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545dff7700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54581062a0 0x7f5458197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545dff7700 1 -- 172.21.15.12:0/3016745055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f545819be30 msgr2=0x7f545819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545dff7700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f545819be30 0x7f545819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545dff7700 1 -- 172.21.15.12:0/3016745055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5450009d00 con 0x7f5458101180 2024-05-08T23:56:58.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545e7f8700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f545819be30 0x7f545819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:58.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f545dff7700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5458101180 0x7f5458197550 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f545400d950 tx=0x7f545400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:58.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f544effd700 1 -- 172.21.15.12:0/3016745055 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454009b40 con 0x7f5458101180 2024-05-08T23:56:58.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f544effd700 1 -- 172.21.15.12:0/3016745055 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5454009ca0 con 0x7f5458101180 2024-05-08T23:56:58.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f546025b700 1 -- 172.21.15.12:0/3016745055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54581a4140 con 0x7f5458101180 2024-05-08T23:56:58.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f544effd700 1 -- 172.21.15.12:0/3016745055 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545400f620 con 0x7f5458101180 2024-05-08T23:56:58.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.955+0000 7f546025b700 1 -- 172.21.15.12:0/3016745055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54581a4620 con 0x7f5458101180 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.958+0000 7f544effd700 1 -- 172.21.15.12:0/3016745055 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5454010460 con 0x7f5458101180 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.958+0000 7f544effd700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54440a1290 0x7f54440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.958+0000 7f544effd700 1 -- 172.21.15.12:0/3016745055 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f54540ccc90 con 0x7f5458101180 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.958+0000 7f545d7f6700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54440a1290 0x7f54440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.959+0000 7f545d7f6700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54440a1290 0x7f54440a3750 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5458198aa0 tx=0x7f544800d4b0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.959+0000 7f546c415b80 1 Processor -- start 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f546c415b80 1 -- start start 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f546c415b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe9e80 0x5612aefea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f546c415b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 0x5612aefe43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f546c415b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aefe4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f546c415b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aef3b1d0 con 0x5612aefe4910 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f546c415b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aef38b90 con 0x5612aefea7a0 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f546c415b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aef34480 con 0x5612aefe9e80 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f545e7f8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aefe4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f545e7f8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aefe4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54504/0 (socket says 172.21.15.12:54504) 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f545e7f8700 1 -- 172.21.15.12:0/3031858126 learned_addr learned my addr 172.21.15.12:0/3031858126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f545d7f6700 1 --2- 172.21.15.12:0/3031858126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 0x5612aefe43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.960+0000 7f545dff7700 1 --2- 172.21.15.12:0/3031858126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe9e80 0x5612aefea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.961+0000 7f545e7f8700 1 -- 172.21.15.12:0/3031858126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe9e80 msgr2=0x5612aefea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.961+0000 7f545e7f8700 1 --2- 172.21.15.12:0/3031858126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe9e80 0x5612aefea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.961+0000 7f545e7f8700 1 -- 172.21.15.12:0/3031858126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 msgr2=0x5612aefe43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.961+0000 7f545e7f8700 1 --2- 172.21.15.12:0/3031858126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 0x5612aefe43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.961+0000 7f545e7f8700 1 -- 172.21.15.12:0/3031858126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612aedb5430 con 0x5612aefe4910 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.961+0000 7f545d7f6700 1 --2- 172.21.15.12:0/3031858126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 0x5612aefe43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.961+0000 7f545e7f8700 1 --2- 172.21.15.12:0/3031858126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aefe4d50 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f545000b0a0 tx=0x7f5450007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677c8e9ca6fd0122 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:58.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.961+0000 7f5443fff700 1 -- 172.21.15.12:0/3031858126 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545000e050 con 0x5612aefe4910 2024-05-08T23:56:58.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.961+0000 7f5443fff700 1 -- 172.21.15.12:0/3031858126 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54500074e0 con 0x5612aefe4910 2024-05-08T23:56:58.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.961+0000 7f5443fff700 1 -- 172.21.15.12:0/3031858126 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450005650 con 0x5612aefe4910 2024-05-08T23:56:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 -- 172.21.15.12:0/3031858126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 msgr2=0x5612aefe4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 --2- 172.21.15.12:0/3031858126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aefe4d50 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f545000b0a0 tx=0x7f5450007bb0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 -- 172.21.15.12:0/3031858126 shutdown_connections 2024-05-08T23:56:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 --2- 172.21.15.12:0/3031858126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aefe4d50 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 --2- 172.21.15.12:0/3031858126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe9e80 0x5612aefea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 --2- 172.21.15.12:0/3031858126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 0x5612aefe43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 -- 172.21.15.12:0/3031858126 >> 172.21.15.12:0/3031858126 conn(0x5612aef42290 msgr2=0x5612aefdee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 -- 172.21.15.12:0/3031858126 shutdown_connections 2024-05-08T23:56:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 -- 172.21.15.12:0/3031858126 wait complete. 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 -- start start 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.962+0000 7f546c415b80 1 Processor -- start 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f546c415b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe4910 0x5612aeff0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f546c415b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefe9e80 0x5612aefedb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f546c415b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefea7a0 0x5612aefee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f546c415b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aeed05e0 con 0x5612aefea7a0 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f546c415b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aeed0280 con 0x5612aefe9e80 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f546c415b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aedb7720 con 0x5612aefe4910 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f546c415b80 10 client.0 _ll_register_callbacks cb 0x5612aeed9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f545e7f8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefea7a0 0x5612aefee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f545e7f8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefea7a0 0x5612aefee290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54510/0 (socket says 172.21.15.12:54510) 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f545e7f8700 1 -- 172.21.15.12:0/3890767529 learned_addr learned my addr 172.21.15.12:0/3890767529 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f545dff7700 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe4910 0x5612aeff0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f545d7f6700 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefe9e80 0x5612aefedb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f545e7f8700 1 -- 172.21.15.12:0/3890767529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe4910 msgr2=0x5612aeff0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f545e7f8700 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe4910 0x5612aeff0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f545e7f8700 1 -- 172.21.15.12:0/3890767529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefe9e80 msgr2=0x5612aefedb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f545e7f8700 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefe9e80 0x5612aefedb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.963+0000 7f545e7f8700 1 -- 172.21.15.12:0/3890767529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544800f820 con 0x5612aefea7a0 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f545e7f8700 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefea7a0 0x5612aefee290 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5450009ca0 tx=0x7f5450009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d259c1febc549dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f54417fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f54417fa700 1 -- 172.21.15.12:0/3890767529 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450003a00 con 0x5612aefea7a0 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f54417fa700 1 -- 172.21.15.12:0/3890767529 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5450003b60 con 0x5612aefea7a0 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f546c415b80 1 -- 172.21.15.12:0/3890767529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612aee8d2f0 con 0x5612aefea7a0 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f54417fa700 1 -- 172.21.15.12:0/3890767529 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5450003d20 con 0x5612aefea7a0 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f546c415b80 10 client.18435 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f546c415b80 1 -- 172.21.15.12:0/3890767529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f545810e690 con 0x5612aefea7a0 2024-05-08T23:56:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f54417fa700 1 -- 172.21.15.12:0/3890767529 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612aee8d2f0 con 0x5612aefea7a0 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f54417fa700 1 -- 172.21.15.12:0/3890767529 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f545000b0d0 con 0x5612aefea7a0 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f546c415b80 10 client.18435 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f546c415b80 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f546c415b80 20 client.18435 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f546c415b80 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612aeffdd70 0x5612af000190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f546c415b80 4 client.18435 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.964+0000 7f546c415b80 1 -- 172.21.15.12:0/3890767529 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612aefb3890 con 0x5612aeffdd70 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.965+0000 7f545dff7700 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612aeffdd70 0x5612af000190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.965+0000 7f545dff7700 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612aeffdd70 0x5612af000190 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb697e4f482450f9 server_cookie=b7f4569cb4f62a44 in_seq=0 out_seq=0 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.965+0000 7f54417fa700 10 client.18435 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f54417fa700 1 -- 172.21.15.12:0/3890767529 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612aefb3890 con 0x5612aeffdd70 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f54417fa700 10 client.18435 handle_command_reply: tid=0 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 client.18435 shutdown 2024-05-08T23:56:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 -- 172.21.15.12:0/3890767529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefea7a0 msgr2=0x5612aefee290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefea7a0 0x5612aefee290 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5450009ca0 tx=0x7f5450009180 comp rx=0 tx=0).stop 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 -- 172.21.15.12:0/3890767529 shutdown_connections 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefea7a0 0x5612aefee290 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe4910 0x5612aeff0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612aeffdd70 0x5612af000190 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 --2- 172.21.15.12:0/3890767529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefe9e80 0x5612aefedb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f54417fa700 0 client.18435 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 -- 172.21.15.12:0/3890767529 >> 172.21.15.12:0/3890767529 conn(0x5612aef42290 msgr2=0x5612aefde620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 -- 172.21.15.12:0/3890767529 shutdown_connections 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 1 -- 172.21.15.12:0/3890767529 wait complete. 2024-05-08T23:56:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.966+0000 7f546c415b80 20 client.18435 trim_cache size 0 max 16384 2024-05-08T23:56:58.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f546c415b80 1 Processor -- start 2024-05-08T23:56:58.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f546c415b80 1 -- start start 2024-05-08T23:56:58.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f546c415b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefe4910 0x5612aefe43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f546c415b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe9e80 0x5612aefdeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f546c415b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefea7a0 0x5612aeff5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f546c415b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aee8d2f0 con 0x5612aefe9e80 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f546c415b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aeffad00 con 0x5612aefe4910 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f546c415b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aefea5e0 con 0x5612aefea7a0 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545d7f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe9e80 0x5612aefdeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545d7f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe9e80 0x5612aefdeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54512/0 (socket says 172.21.15.12:54512) 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545d7f6700 1 -- 172.21.15.12:0/100461506 learned_addr learned my addr 172.21.15.12:0/100461506 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545dff7700 1 --2- 172.21.15.12:0/100461506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefe4910 0x5612aefe43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545e7f8700 1 --2- 172.21.15.12:0/100461506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefea7a0 0x5612aeff5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545d7f6700 1 -- 172.21.15.12:0/100461506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefea7a0 msgr2=0x5612aeff5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545d7f6700 1 --2- 172.21.15.12:0/100461506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefea7a0 0x5612aeff5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545d7f6700 1 -- 172.21.15.12:0/100461506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefe4910 msgr2=0x5612aefe43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:58.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545d7f6700 1 --2- 172.21.15.12:0/100461506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefe4910 0x5612aefe43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545d7f6700 1 -- 172.21.15.12:0/100461506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612aed61be0 con 0x5612aefe9e80 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.985+0000 7f545e7f8700 1 --2- 172.21.15.12:0/100461506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefea7a0 0x5612aeff5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f545d7f6700 1 --2- 172.21.15.12:0/100461506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe9e80 0x5612aefdeda0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f544800fd90 tx=0x7f5448012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf66210336892b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f544cff9700 1 -- 172.21.15.12:0/100461506 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448029640 con 0x5612aefe9e80 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f544cff9700 1 -- 172.21.15.12:0/100461506 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f544800fe80 con 0x5612aefe9e80 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f544cff9700 1 -- 172.21.15.12:0/100461506 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544801e8b0 con 0x5612aefe9e80 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f546c415b80 1 -- 172.21.15.12:0/100461506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe9e80 msgr2=0x5612aefdeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f546c415b80 1 --2- 172.21.15.12:0/100461506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe9e80 0x5612aefdeda0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f544800fd90 tx=0x7f5448012000 comp rx=0 tx=0).stop 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f546c415b80 1 -- 172.21.15.12:0/100461506 shutdown_connections 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f546c415b80 1 --2- 172.21.15.12:0/100461506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefea7a0 0x5612aeff5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f546c415b80 1 --2- 172.21.15.12:0/100461506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefe4910 0x5612aefe43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f546c415b80 1 --2- 172.21.15.12:0/100461506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe9e80 0x5612aefdeda0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.986+0000 7f546c415b80 1 -- 172.21.15.12:0/100461506 >> 172.21.15.12:0/100461506 conn(0x5612aef42290 msgr2=0x5612aefe7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.987+0000 7f546c415b80 1 -- 172.21.15.12:0/100461506 shutdown_connections 2024-05-08T23:56:58.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.987+0000 7f546c415b80 1 -- 172.21.15.12:0/100461506 wait complete. 2024-05-08T23:56:58.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.987+0000 7f546c415b80 1 -- start start 2024-05-08T23:56:58.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.987+0000 7f546c415b80 1 Processor -- start 2024-05-08T23:56:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f546c415b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aeff53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f546c415b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe9e80 0x5612aeff18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f546c415b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 0x5612aeff1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f546c415b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aeff2360 con 0x5612aefe4910 2024-05-08T23:56:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f546c415b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aeff24d0 con 0x5612aefea7a0 2024-05-08T23:56:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f546c415b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612aeff2690 con 0x5612aefe9e80 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f546c415b80 10 client.0 _ll_register_callbacks cb 0x5612aeed9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545dff7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aeff53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545dff7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aeff53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54526/0 (socket says 172.21.15.12:54526) 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545dff7700 1 -- 172.21.15.12:0/3884541016 learned_addr learned my addr 172.21.15.12:0/3884541016 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545d7f6700 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe9e80 0x5612aeff18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545e7f8700 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 0x5612aeff1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545dff7700 1 -- 172.21.15.12:0/3884541016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe9e80 msgr2=0x5612aeff18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545dff7700 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe9e80 0x5612aeff18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545dff7700 1 -- 172.21.15.12:0/3884541016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 msgr2=0x5612aeff1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545dff7700 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 0x5612aeff1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545dff7700 1 -- 172.21.15.12:0/3884541016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544800fa70 con 0x5612aefe4910 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545e7f8700 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 0x5612aeff1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f545dff7700 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aeff53d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5454099d70 tx=0x7f5454009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928031dd72952458 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f54417fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f54417fa700 1 -- 172.21.15.12:0/3884541016 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545409dc10 con 0x5612aefe4910 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.988+0000 7f54417fa700 1 -- 172.21.15.12:0/3884541016 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f54540a2260 con 0x5612aefe4910 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f546c415b80 1 -- 172.21.15.12:0/3884541016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54500074e0 con 0x5612aefe4910 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f54417fa700 1 -- 172.21.15.12:0/3884541016 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54540b81a0 con 0x5612aefe4910 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f546c415b80 10 client.18441 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f546c415b80 1 -- 172.21.15.12:0/3884541016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5612aeefa5e0 con 0x5612aefe4910 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f54417fa700 1 -- 172.21.15.12:0/3884541016 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54500074e0 con 0x5612aefe4910 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f54417fa700 1 -- 172.21.15.12:0/3884541016 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f54540992d0 con 0x5612aefe4910 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f546c415b80 10 client.18441 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f546c415b80 10 client.18441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f546c415b80 20 client.18441 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f546c415b80 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612af055650 0x5612af057a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f546c415b80 4 client.18441 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f546c415b80 1 -- 172.21.15.12:0/3884541016 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5612aef2cf40 con 0x5612af055650 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.989+0000 7f545d7f6700 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612af055650 0x5612af057a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.990+0000 7f54417fa700 10 client.18441 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.990+0000 7f545d7f6700 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612af055650 0x5612af057a70 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebcf6368c9d4f9b9 server_cookie=b067d3299f1408a2 in_seq=0 out_seq=0 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f54417fa700 1 -- 172.21.15.12:0/3884541016 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5612aef2cf40 con 0x5612af055650 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f54417fa700 10 client.18441 handle_command_reply: tid=0 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 client.18441 shutdown 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 -- 172.21.15.12:0/3884541016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 msgr2=0x5612aeff53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aeff53d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5454099d70 tx=0x7f5454009650 comp rx=0 tx=0).stop 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 -- 172.21.15.12:0/3884541016 shutdown_connections 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612af055650 0x5612af057a70 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612aefea7a0 0x5612aeff1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f54417fa700 0 client.18441 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612aefe4910 0x5612aeff53d0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 --2- 172.21.15.12:0/3884541016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612aefe9e80 0x5612aeff18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 -- 172.21.15.12:0/3884541016 >> 172.21.15.12:0/3884541016 conn(0x5612aef42290 msgr2=0x5612aefec350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 -- 172.21.15.12:0/3884541016 shutdown_connections 2024-05-08T23:56:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 1 -- 172.21.15.12:0/3884541016 wait complete. 2024-05-08T23:56:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.991+0000 7f546c415b80 20 client.18441 trim_cache size 0 max 16384 2024-05-08T23:56:58.991 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:56:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:56:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "a9612573-3765-4f19-b78f-f636ef10c5bf", 2024-05-08T23:56:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:56:58.992 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:56:58.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 -- 172.21.15.12:0/3016745055 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54440a1290 msgr2=0x7f54440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:58.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54440a1290 0x7f54440a3750 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5458198aa0 tx=0x7f544800d4b0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 -- 172.21.15.12:0/3016745055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5458101180 msgr2=0x7f5458197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:58.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5458101180 0x7f5458197550 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f545400d950 tx=0x7f545400dd10 comp rx=0 tx=0).stop 2024-05-08T23:56:58.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 -- 172.21.15.12:0/3016745055 shutdown_connections 2024-05-08T23:56:58.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54440a1290 0x7f54440a3750 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f545819be30 0x7f545819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5458101180 0x7f5458197550 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 --2- 172.21.15.12:0/3016745055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54581062a0 0x7f5458197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:58.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 -- 172.21.15.12:0/3016745055 >> 172.21.15.12:0/3016745055 conn(0x7f54580fc9e0 msgr2=0x7f54580fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:58.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 -- 172.21.15.12:0/3016745055 shutdown_connections 2024-05-08T23:56:58.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:58.992+0000 7f546025b700 1 -- 172.21.15.12:0/3016745055 wait complete. 2024-05-08T23:56:59.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:56:59.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.151+0000 7f08f3e0d700 1 -- 172.21.15.12:0/2028148640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec0696d0 msgr2=0x7f08ec069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:59.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.151+0000 7f08f3e0d700 1 --2- 172.21.15.12:0/2028148640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec0696d0 0x7f08ec069b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009990 tx=0x7f08e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.151+0000 7f08f3e0d700 1 -- 172.21.15.12:0/2028148640 shutdown_connections 2024-05-08T23:56:59.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.151+0000 7f08f3e0d700 1 --2- 172.21.15.12:0/2028148640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08ec10d7f0 0x7f08ec10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.151+0000 7f08f3e0d700 1 --2- 172.21.15.12:0/2028148640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08ec101e20 0x7f08ec069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.151+0000 7f08f3e0d700 1 --2- 172.21.15.12:0/2028148640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec0696d0 0x7f08ec069b50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.151+0000 7f08f3e0d700 1 -- 172.21.15.12:0/2028148640 >> 172.21.15.12:0/2028148640 conn(0x7f08ec076ba0 msgr2=0x7f08ec076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.152+0000 7f08f3e0d700 1 -- 172.21.15.12:0/2028148640 shutdown_connections 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.152+0000 7f08f3e0d700 1 -- 172.21.15.12:0/2028148640 wait complete. 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.152+0000 7f08f3e0d700 1 Processor -- start 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.152+0000 7f08f3e0d700 1 -- start start 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.152+0000 7f08f3e0d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08ec0696d0 0x7f08ec1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.152+0000 7f08f3e0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08ec101e20 0x7f08ec199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.152+0000 7f08f3e0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d7f0 0x7f08ec19ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.152+0000 7f08f3e0d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec19e510 con 0x7f08ec101e20 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.152+0000 7f08f3e0d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec19e680 con 0x7f08ec10d7f0 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.152+0000 7f08f3e0d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec19e840 con 0x7f08ec0696d0 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f1ba9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08ec0696d0 0x7f08ec1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f13a8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08ec101e20 0x7f08ec199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f23aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d7f0 0x7f08ec19ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f1ba9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08ec0696d0 0x7f08ec1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57602/0 (socket says 172.21.15.12:57602) 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f1ba9700 1 -- 172.21.15.12:0/686770618 learned_addr learned my addr 172.21.15.12:0/686770618 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f13a8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08ec101e20 0x7f08ec199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54530/0 (socket says 172.21.15.12:54530) 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f13a8700 1 -- 172.21.15.12:0/686770618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08ec0696d0 msgr2=0x7f08ec1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f13a8700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08ec0696d0 0x7f08ec1995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f13a8700 1 -- 172.21.15.12:0/686770618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d7f0 msgr2=0x7f08ec19ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f13a8700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d7f0 0x7f08ec19ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f13a8700 1 -- 172.21.15.12:0/686770618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e80095f0 con 0x7f08ec101e20 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08ec0696d0 0x7f08ec1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f23aa700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d7f0 0x7f08ec19ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f13a8700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08ec101e20 0x7f08ec199b30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f08e8000c00 tx=0x7f08e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08e2ffd700 1 -- 172.21.15.12:0/686770618 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e801d070 con 0x7f08ec101e20 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f3e0d700 1 -- 172.21.15.12:0/686770618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec19eac0 con 0x7f08ec101e20 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08e2ffd700 1 -- 172.21.15.12:0/686770618 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08e8021410 con 0x7f08ec101e20 2024-05-08T23:56:59.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08e2ffd700 1 -- 172.21.15.12:0/686770618 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8017650 con 0x7f08ec101e20 2024-05-08T23:56:59.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.153+0000 7f08f3e0d700 1 -- 172.21.15.12:0/686770618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec1a6360 con 0x7f08ec101e20 2024-05-08T23:56:59.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.156+0000 7f08e2ffd700 1 -- 172.21.15.12:0/686770618 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f08e8021580 con 0x7f08ec101e20 2024-05-08T23:56:59.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.156+0000 7f08e2ffd700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08d80a1290 0x7f08d80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.157+0000 7f08e2ffd700 1 -- 172.21.15.12:0/686770618 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f08e80cdf00 con 0x7f08ec101e20 2024-05-08T23:56:59.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.157+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08d80a1290 0x7f08d80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.157+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08d80a1290 0x7f08d80a3750 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08dc011550 tx=0x7f08dc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.157+0000 7f08fffc7b80 1 Processor -- start 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08fffc7b80 1 -- start start 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 0x555d2f2e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f234150 con 0x555d2f2dd890 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f231b10 con 0x555d2f2e2e00 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f22d400 con 0x555d2f2e3720 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f1ba9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 0x555d2f2e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f1ba9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 0x555d2f2e31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34972/0 (socket says 172.21.15.12:34972) 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f1ba9700 1 -- 172.21.15.12:0/1983576822 learned_addr learned my addr 172.21.15.12:0/1983576822 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f23aa700 1 --2- 172.21.15.12:0/1983576822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f13a8700 1 --2- 172.21.15.12:0/1983576822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f1ba9700 1 -- 172.21.15.12:0/1983576822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 msgr2=0x555d2f2dd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/1983576822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2dd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f1ba9700 1 -- 172.21.15.12:0/1983576822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 msgr2=0x555d2f2ddcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/1983576822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2ddcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f1ba9700 1 -- 172.21.15.12:0/1983576822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d2f0ae3b0 con 0x555d2f2e2e00 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.158+0000 7f08f13a8700 1 --2- 172.21.15.12:0/1983576822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.159+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/1983576822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 0x555d2f2e31e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009880 tx=0x7f08dc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a251447e818d3bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.159+0000 7f08d7fff700 1 -- 172.21.15.12:0/1983576822 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc026630 con 0x555d2f2e2e00 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.159+0000 7f08d7fff700 1 -- 172.21.15.12:0/1983576822 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08dc021070 con 0x555d2f2e2e00 2024-05-08T23:56:59.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.159+0000 7f08d7fff700 1 -- 172.21.15.12:0/1983576822 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc01d650 con 0x555d2f2e2e00 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 -- 172.21.15.12:0/1983576822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 msgr2=0x555d2f2e31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/1983576822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 0x555d2f2e31e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009880 tx=0x7f08dc014000 comp rx=0 tx=0).stop 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 -- 172.21.15.12:0/1983576822 shutdown_connections 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/1983576822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2ddcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/1983576822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 0x555d2f2e31e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/1983576822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 -- 172.21.15.12:0/1983576822 >> 172.21.15.12:0/1983576822 conn(0x555d2f23b210 msgr2=0x555d2f2d7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 -- 172.21.15.12:0/1983576822 shutdown_connections 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 -- 172.21.15.12:0/1983576822 wait complete. 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 -- start start 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.160+0000 7f08fffc7b80 1 Processor -- start 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2dd890 0x555d2f2e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2e2e00 0x555d2f2e7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f1c9560 con 0x555d2f2e2e00 2024-05-08T23:56:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f1c9200 con 0x555d2f2dd890 2024-05-08T23:56:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f0b06a0 con 0x555d2f2e3720 2024-05-08T23:56:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08fffc7b80 10 client.0 _ll_register_callbacks cb 0x555d2f1d2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08f1ba9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2dd890 0x555d2f2e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08f13a8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2e2e00 0x555d2f2e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08f1ba9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2dd890 0x555d2f2e6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34984/0 (socket says 172.21.15.12:34984) 2024-05-08T23:56:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08f1ba9700 1 -- 172.21.15.12:0/230716126 learned_addr learned my addr 172.21.15.12:0/230716126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08f23aa700 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.161+0000 7f08f13a8700 1 -- 172.21.15.12:0/230716126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 msgr2=0x555d2f2e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08f13a8700 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2e75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08f13a8700 1 -- 172.21.15.12:0/230716126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2dd890 msgr2=0x555d2f2e6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08f13a8700 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2dd890 0x555d2f2e6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08f13a8700 1 -- 172.21.15.12:0/230716126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc017070 con 0x555d2f2e2e00 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2dd890 0x555d2f2e6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08f13a8700 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2e2e00 0x555d2f2e7080 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f08e80cb630 tx=0x7f08e809acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2faa5186eef45a37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08d57fa700 1 -- 172.21.15.12:0/230716126 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e809e070 con 0x555d2f2e2e00 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08d57fa700 1 -- 172.21.15.12:0/230716126 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f08e809b080 con 0x555d2f2e2e00 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08fffc7b80 1 -- 172.21.15.12:0/230716126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d2f186270 con 0x555d2f2e2e00 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08d57fa700 1 -- 172.21.15.12:0/230716126 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08e8021a70 con 0x555d2f2e2e00 2024-05-08T23:56:59.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08fffc7b80 10 client.18453 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08fffc7b80 1 -- 172.21.15.12:0/230716126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x555d2f05ab60 con 0x555d2f2e2e00 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08d57fa700 1 -- 172.21.15.12:0/230716126 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d2f186270 con 0x555d2f2e2e00 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.162+0000 7f08d57fa700 1 -- 172.21.15.12:0/230716126 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08e80b9270 con 0x555d2f2e2e00 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.163+0000 7f08fffc7b80 10 client.18453 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.163+0000 7f08fffc7b80 10 client.18453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.163+0000 7f08fffc7b80 20 client.18453 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.163+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555d2f2f6dd0 0x555d2f2f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.163+0000 7f08fffc7b80 4 client.18453 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.163+0000 7f08fffc7b80 1 -- 172.21.15.12:0/230716126 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d2f2ac810 con 0x555d2f2f6dd0 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.163+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555d2f2f6dd0 0x555d2f2f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.163+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555d2f2f6dd0 0x555d2f2f91f0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c804b43a3999246c server_cookie=54b61498a92d5b12 in_seq=0 out_seq=0 2024-05-08T23:56:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.163+0000 7f08d57fa700 10 client.18453 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.164+0000 7f08d57fa700 1 -- 172.21.15.12:0/230716126 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d2f2ac810 con 0x555d2f2f6dd0 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.164+0000 7f08d57fa700 10 client.18453 handle_command_reply: tid=0 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.164+0000 7f08fffc7b80 1 client.18453 shutdown 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.164+0000 7f08fffc7b80 1 -- 172.21.15.12:0/230716126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2e2e00 msgr2=0x555d2f2e7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.164+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2e2e00 0x555d2f2e7080 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f08e80cb630 tx=0x7f08e809acc0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.165+0000 7f08fffc7b80 1 -- 172.21.15.12:0/230716126 shutdown_connections 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.165+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.165+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555d2f2f6dd0 0x555d2f2f91f0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.165+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2dd890 0x555d2f2e6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.165+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/230716126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2e2e00 0x555d2f2e7080 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.165+0000 7f08d57fa700 0 client.18453 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.165+0000 7f08fffc7b80 1 -- 172.21.15.12:0/230716126 >> 172.21.15.12:0/230716126 conn(0x555d2f23b210 msgr2=0x555d2f2d75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.165+0000 7f08fffc7b80 1 -- 172.21.15.12:0/230716126 shutdown_connections 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.165+0000 7f08fffc7b80 1 -- 172.21.15.12:0/230716126 wait complete. 2024-05-08T23:56:59.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.165+0000 7f08fffc7b80 20 client.18453 trim_cache size 0 max 16384 2024-05-08T23:56:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.184+0000 7f08fffc7b80 1 Processor -- start 2024-05-08T23:56:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.184+0000 7f08fffc7b80 1 -- start start 2024-05-08T23:56:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.184+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.184+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e2e00 0x555d2f2e24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.184+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e3720 0x555d2f2d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.184+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f2e8160 con 0x555d2f2dd890 2024-05-08T23:56:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.184+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f2d88e0 con 0x555d2f2e3720 2024-05-08T23:56:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.184+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f2d4dc0 con 0x555d2f2e2e00 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.184+0000 7f08f1ba9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f13a8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e2e00 0x555d2f2e24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f1ba9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2dd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54552/0 (socket says 172.21.15.12:54552) 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f1ba9700 1 -- 172.21.15.12:0/4077965570 learned_addr learned my addr 172.21.15.12:0/4077965570 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f23aa700 1 --2- 172.21.15.12:0/4077965570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e3720 0x555d2f2d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f1ba9700 1 -- 172.21.15.12:0/4077965570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e2e00 msgr2=0x555d2f2e24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/4077965570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e2e00 0x555d2f2e24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f1ba9700 1 -- 172.21.15.12:0/4077965570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e3720 msgr2=0x555d2f2d7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/4077965570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e3720 0x555d2f2d7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f1ba9700 1 -- 172.21.15.12:0/4077965570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d2f2ddf40 con 0x555d2f2dd890 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f13a8700 1 --2- 172.21.15.12:0/4077965570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e2e00 0x555d2f2e24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/4077965570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2dd350 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00e6a0 tx=0x7f08dc011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7034bc976d0b0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08e0ff9700 1 -- 172.21.15.12:0/4077965570 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc01cdc0 con 0x555d2f2dd890 2024-05-08T23:56:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08e0ff9700 1 -- 172.21.15.12:0/4077965570 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08dc0055a0 con 0x555d2f2dd890 2024-05-08T23:56:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.185+0000 7f08e0ff9700 1 -- 172.21.15.12:0/4077965570 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc033890 con 0x555d2f2dd890 2024-05-08T23:56:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 -- 172.21.15.12:0/4077965570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 msgr2=0x555d2f2dd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/4077965570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2dd350 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00e6a0 tx=0x7f08dc011d50 comp rx=0 tx=0).stop 2024-05-08T23:56:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 -- 172.21.15.12:0/4077965570 shutdown_connections 2024-05-08T23:56:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/4077965570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e3720 0x555d2f2d7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/4077965570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2dd350 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/4077965570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e2e00 0x555d2f2e24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 -- 172.21.15.12:0/4077965570 >> 172.21.15.12:0/4077965570 conn(0x555d2f23b210 msgr2=0x555d2f2e5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 -- 172.21.15.12:0/4077965570 shutdown_connections 2024-05-08T23:56:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 -- 172.21.15.12:0/4077965570 wait complete. 2024-05-08T23:56:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 -- start start 2024-05-08T23:56:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.186+0000 7f08fffc7b80 1 Processor -- start 2024-05-08T23:56:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2ee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 0x555d2f2ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08fffc7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2ec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f2ed030 con 0x555d2f2dd890 2024-05-08T23:56:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f2ed1a0 con 0x555d2f2e2e00 2024-05-08T23:56:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08fffc7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2f2ed360 con 0x555d2f2e3720 2024-05-08T23:56:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08fffc7b80 10 client.0 _ll_register_callbacks cb 0x555d2f1d2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:56:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08f1ba9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08f13a8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 0x555d2f2ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08f23aa700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2ec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08f1ba9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2ee160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54554/0 (socket says 172.21.15.12:54554) 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08f1ba9700 1 -- 172.21.15.12:0/805864174 learned_addr learned my addr 172.21.15.12:0/805864174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08f1ba9700 1 -- 172.21.15.12:0/805864174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 msgr2=0x555d2f2ec950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2ec950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08f1ba9700 1 -- 172.21.15.12:0/805864174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 msgr2=0x555d2f2ee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 0x555d2f2ee6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.187+0000 7f08f1ba9700 1 -- 172.21.15.12:0/805864174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d2f05ab60 con 0x555d2f2dd890 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08f1ba9700 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2ee160 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f08dc035040 tx=0x7f08dc01cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8372caa7bd1f987 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08d57fa700 1 -- 172.21.15.12:0/805864174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00edc0 con 0x555d2f2dd890 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08d57fa700 1 -- 172.21.15.12:0/805864174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f08dc0331b0 con 0x555d2f2dd890 2024-05-08T23:56:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08fffc7b80 1 -- 172.21.15.12:0/805864174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08dc021070 con 0x555d2f2dd890 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08d57fa700 1 -- 172.21.15.12:0/805864174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08dc033370 con 0x555d2f2dd890 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08fffc7b80 10 client.18459 fetch_fsmap learned FSMap version 45 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08fffc7b80 1 -- 172.21.15.12:0/805864174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f08dc02a040 con 0x555d2f2dd890 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08d57fa700 1 -- 172.21.15.12:0/805864174 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08dc04d080 con 0x555d2f2dd890 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08d57fa700 1 -- 172.21.15.12:0/805864174 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08dc01c770 con 0x555d2f2dd890 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08fffc7b80 10 client.18459 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.188+0000 7f08fffc7b80 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08fffc7b80 20 client.18459 populate_metadata read hostname 'smithi012' 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555d2f34e250 0x555d2f350640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08fffc7b80 4 client.18459 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08fffc7b80 1 -- 172.21.15.12:0/805864174 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d2f225ec0 con 0x555d2f34e250 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08f13a8700 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555d2f34e250 0x555d2f350640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08d57fa700 10 client.18459 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08f13a8700 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555d2f34e250 0x555d2f350640 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33044c4e724af1d2 server_cookie=cea2e31a154f57f6 in_seq=0 out_seq=0 2024-05-08T23:56:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08d57fa700 1 -- 172.21.15.12:0/805864174 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555d2f225ec0 con 0x555d2f34e250 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08d57fa700 10 client.18459 handle_command_reply: tid=0 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08fffc7b80 1 client.18459 shutdown 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08fffc7b80 1 -- 172.21.15.12:0/805864174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 msgr2=0x555d2f2ee160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.189+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2ee160 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f08dc035040 tx=0x7f08dc01cd90 comp rx=0 tx=0).stop 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08fffc7b80 1 -- 172.21.15.12:0/805864174 shutdown_connections 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d2f2e3720 0x555d2f2ec950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d2f2dd890 0x555d2f2ee160 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555d2f34e250 0x555d2f350640 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08fffc7b80 1 --2- 172.21.15.12:0/805864174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d2f2e2e00 0x555d2f2ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08fffc7b80 1 -- 172.21.15.12:0/805864174 >> 172.21.15.12:0/805864174 conn(0x555d2f23b210 msgr2=0x555d2f2e2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08d57fa700 0 client.18459 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08fffc7b80 1 -- 172.21.15.12:0/805864174 shutdown_connections 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08fffc7b80 1 -- 172.21.15.12:0/805864174 wait complete. 2024-05-08T23:56:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08fffc7b80 20 client.18459 trim_cache size 0 max 16384 2024-05-08T23:56:59.191 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:56:59.191 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:56:59.191 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:56:59.191 INFO:teuthology.orchestra.run.smithi012.stdout: "a9612573-3765-4f19-b78f-f636ef10c5bf": { 2024-05-08T23:56:59.191 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:56:59.191 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "a9612573-3765-4f19-b78f-f636ef10c5bf", 2024-05-08T23:56:59.191 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:56:59.191 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:56:59.191 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:56:59.191 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:56:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 -- 172.21.15.12:0/686770618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08d80a1290 msgr2=0x7f08d80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08d80a1290 0x7f08d80a3750 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08dc011550 tx=0x7f08dc009260 comp rx=0 tx=0).stop 2024-05-08T23:56:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 -- 172.21.15.12:0/686770618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08ec101e20 msgr2=0x7f08ec199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:56:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08ec101e20 0x7f08ec199b30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f08e8000c00 tx=0x7f08e8004730 comp rx=0 tx=0).stop 2024-05-08T23:56:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 -- 172.21.15.12:0/686770618 shutdown_connections 2024-05-08T23:56:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08d80a1290 0x7f08d80a3750 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d7f0 0x7f08ec19ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08ec0696d0 0x7f08ec1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 --2- 172.21.15.12:0/686770618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08ec101e20 0x7f08ec199b30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:56:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 -- 172.21.15.12:0/686770618 >> 172.21.15.12:0/686770618 conn(0x7f08ec076ba0 msgr2=0x7f08ec10e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:56:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 -- 172.21.15.12:0/686770618 shutdown_connections 2024-05-08T23:56:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:56:59.190+0000 7f08f3e0d700 1 -- 172.21.15.12:0/686770618 wait complete. 2024-05-08T23:56:59.199 INFO:tasks.cephfs.filesystem:scrub status for tag:a9612573-3765-4f19-b78f-f636ef10c5bf - {'path': '/', 'tag': 'a9612573-3765-4f19-b78f-f636ef10c5bf', 'options': 'recursive,force'} 2024-05-08T23:56:59.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:59 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:56:59.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:56:59 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:59.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:59 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:56:59.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:56:59 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:59.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:59 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:56:59.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:56:59 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:56:59.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 593450 6.50 MB/sec execute 415 sec latency 944.974 ms 2024-05-08T23:57:00.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:57:00.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:57:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:57:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:57:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:57:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:00 smithi012 ceph-mon[42503]: pgmap v334: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-05-08T23:57:00.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:57:00.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:57:00.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:57:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:57:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:57:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:00 smithi204 ceph-mon[47373]: pgmap v334: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-05-08T23:57:00.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:57:00.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:57:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:57:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:57:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:57:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:00.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:00 smithi160 ceph-mon[46152]: pgmap v334: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-05-08T23:57:00.887 INFO:tasks.workunit.client.0.smithi012.stdout: 1 593761 6.49 MB/sec execute 416 sec latency 697.219 ms 2024-05-08T23:57:01.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 593761 6.47 MB/sec execute 417 sec latency 1697.316 ms 2024-05-08T23:57:01.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:01 smithi012 ceph-mon[42503]: pgmap v335: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-05-08T23:57:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:01 smithi204 ceph-mon[47373]: pgmap v335: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-05-08T23:57:02.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:01 smithi160 ceph-mon[46152]: pgmap v335: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-05-08T23:57:02.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 593887 6.46 MB/sec execute 418 sec latency 2602.272 ms 2024-05-08T23:57:03.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 594549 6.45 MB/sec execute 419 sec latency 88.882 ms 2024-05-08T23:57:04.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:04 smithi012 ceph-mon[42503]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T23:57:04.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:04 smithi012 ceph-mon[42503]: pgmap v337: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-05-08T23:57:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:04 smithi204 ceph-mon[47373]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T23:57:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:04 smithi204 ceph-mon[47373]: pgmap v337: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-05-08T23:57:04.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:04 smithi160 ceph-mon[46152]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T23:57:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:04 smithi160 ceph-mon[46152]: pgmap v337: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-05-08T23:57:04.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 594856 6.44 MB/sec execute 420 sec latency 779.109 ms 2024-05-08T23:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.057+0000 7f4dbd70f700 1 -- 172.21.15.12:0/3087714051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4db8105130 msgr2=0x7f4db810ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.057+0000 7f4dbd70f700 1 --2- 172.21.15.12:0/3087714051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4db8105130 0x7f4db810ccb0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4db000b0a0 tx=0x7f4db000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.057+0000 7f4dbd70f700 1 -- 172.21.15.12:0/3087714051 shutdown_connections 2024-05-08T23:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.057+0000 7f4dbd70f700 1 --2- 172.21.15.12:0/3087714051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4db8105130 0x7f4db810ccb0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.057+0000 7f4dbd70f700 1 --2- 172.21.15.12:0/3087714051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4db8100010 0x7f4db81003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.057+0000 7f4dbd70f700 1 --2- 172.21.15.12:0/3087714051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4db81009c0 0x7f4db8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.057+0000 7f4dbd70f700 1 -- 172.21.15.12:0/3087714051 >> 172.21.15.12:0/3087714051 conn(0x7f4db80fb870 msgr2=0x7f4db80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.057+0000 7f4dbd70f700 1 -- 172.21.15.12:0/3087714051 shutdown_connections 2024-05-08T23:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.057+0000 7f4dbd70f700 1 -- 172.21.15.12:0/3087714051 wait complete. 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.058+0000 7f4dbd70f700 1 Processor -- start 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.058+0000 7f4dbd70f700 1 -- start start 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.058+0000 7f4dbd70f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4db8100010 0x7f4db8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.058+0000 7f4dbd70f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4db8105130 0x7f4db81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.058+0000 7f4dbd70f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4db8198de0 0x7f4db81a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.058+0000 7f4dbd70f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db81a3af0 con 0x7f4db8100010 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.058+0000 7f4dbd70f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db81a3c60 con 0x7f4db8105130 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.058+0000 7f4dbd70f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db81a3e20 con 0x7f4db8198de0 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4db8100010 0x7f4db8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4db8100010 0x7f4db8198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54584/0 (socket says 172.21.15.12:54584) 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db6ffd700 1 -- 172.21.15.12:0/134752470 learned_addr learned my addr 172.21.15.12:0/134752470 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db67fc700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4db8105130 0x7f4db81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db77fe700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4db8198de0 0x7f4db81a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db6ffd700 1 -- 172.21.15.12:0/134752470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4db8198de0 msgr2=0x7f4db81a33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4db8198de0 0x7f4db81a33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db6ffd700 1 -- 172.21.15.12:0/134752470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4db8105130 msgr2=0x7f4db81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4db8105130 0x7f4db81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db6ffd700 1 -- 172.21.15.12:0/134752470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db0009d00 con 0x7f4db8100010 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db77fe700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4db8198de0 0x7f4db81a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4db8100010 0x7f4db8198360 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4da000e9a0 tx=0x7f4da000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4daffff700 1 -- 172.21.15.12:0/134752470 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0009790 con 0x7f4db8100010 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4daffff700 1 -- 172.21.15.12:0/134752470 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4da000cbd0 con 0x7f4db8100010 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4dbd70f700 1 -- 172.21.15.12:0/134752470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db81a4100 con 0x7f4db8100010 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4daffff700 1 -- 172.21.15.12:0/134752470 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0018680 con 0x7f4db8100010 2024-05-08T23:57:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.059+0000 7f4dbd70f700 1 -- 172.21.15.12:0/134752470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db81a4610 con 0x7f4db8100010 2024-05-08T23:57:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.062+0000 7f4daffff700 1 -- 172.21.15.12:0/134752470 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4da000cd40 con 0x7f4db8100010 2024-05-08T23:57:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.062+0000 7f4daffff700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4da40a1290 0x7f4da40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.062+0000 7f4daffff700 1 -- 172.21.15.12:0/134752470 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4da0014070 con 0x7f4db8100010 2024-05-08T23:57:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.062+0000 7f4db67fc700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4da40a1290 0x7f4da40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.063+0000 7f4db67fc700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4da40a1290 0x7f4da40a3750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4db8069e30 tx=0x7f4da8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.063+0000 7f4dc98c9b80 1 Processor -- start 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.063+0000 7f4dc98c9b80 1 -- start start 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 0x5584de3cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de31f0f0 con 0x5584de3c8480 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de31cab0 con 0x5584de3cd950 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de3183a0 con 0x5584de3ce270 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 0x5584de3cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db67fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54600/0 (socket says 172.21.15.12:54600) 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db77fe700 1 -- 172.21.15.12:0/557083482 learned_addr learned my addr 172.21.15.12:0/557083482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 0x5584de3cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:35016/0 (socket says 172.21.15.12:35016) 2024-05-08T23:57:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db6ffd700 1 -- 172.21.15.12:0/557083482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 msgr2=0x5584de3c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/557083482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db6ffd700 1 -- 172.21.15.12:0/557083482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 msgr2=0x5584de3c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/557083482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db6ffd700 1 -- 172.21.15.12:0/557083482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584de199350 con 0x5584de3cd950 2024-05-08T23:57:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db77fe700 1 --2- 172.21.15.12:0/557083482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.064+0000 7f4db67fc700 1 --2- 172.21.15.12:0/557083482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.065+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/557083482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 0x5584de3cdd30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4da00ca6b0 tx=0x7f4da00999d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19187ec449f38db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.065+0000 7f4dad7fa700 1 -- 172.21.15.12:0/557083482 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da009a070 con 0x5584de3cd950 2024-05-08T23:57:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.065+0000 7f4dad7fa700 1 -- 172.21.15.12:0/557083482 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4da009dea0 con 0x5584de3cd950 2024-05-08T23:57:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.065+0000 7f4dad7fa700 1 -- 172.21.15.12:0/557083482 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da00997c0 con 0x5584de3cd950 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.065+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/557083482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 msgr2=0x5584de3cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.065+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/557083482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 0x5584de3cdd30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4da00ca6b0 tx=0x7f4da00999d0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/557083482 shutdown_connections 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/557083482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/557083482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 0x5584de3cdd30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/557083482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/557083482 >> 172.21.15.12:0/557083482 conn(0x5584de3261b0 msgr2=0x5584de3c2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/557083482 shutdown_connections 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/557083482 wait complete. 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 -- start start 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 Processor -- start 2024-05-08T23:57:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3c8480 0x5584de3d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3ce270 0x5584de3d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de2b4500 con 0x5584de3cd950 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de2b41a0 con 0x5584de3ce270 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de19b640 con 0x5584de3c8480 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.066+0000 7f4dc98c9b80 10 client.0 _ll_register_callbacks cb 0x5584de2bdad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54616/0 (socket says 172.21.15.12:54616) 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db67fc700 1 -- 172.21.15.12:0/3538102637 learned_addr learned my addr 172.21.15.12:0/3538102637 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db77fe700 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3ce270 0x5584de3d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3c8480 0x5584de3d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db67fc700 1 -- 172.21.15.12:0/3538102637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3c8480 msgr2=0x5584de3d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db67fc700 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3c8480 0x5584de3d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db67fc700 1 -- 172.21.15.12:0/3538102637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3ce270 msgr2=0x5584de3d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db67fc700 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3ce270 0x5584de3d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db67fc700 1 -- 172.21.15.12:0/3538102637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da8017070 con 0x5584de3cd950 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4db67fc700 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3d1b30 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4db8069330 tx=0x7f4da80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e053705f8e293a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4d9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3538102637 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8004020 con 0x5584de3cd950 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3538102637 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4da8004180 con 0x5584de3cd950 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3538102637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584de271210 con 0x5584de3cd950 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.067+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3538102637 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4da8004380 con 0x5584de3cd950 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4dc98c9b80 10 client.18471 fetch_fsmap learned FSMap version 45 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3538102637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f4db81a4610 con 0x5584de3cd950 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3538102637 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4da8033080 con 0x5584de3cd950 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3538102637 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4da800e7d0 con 0x5584de3cd950 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4dc98c9b80 10 client.18471 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:57:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4dc98c9b80 10 client.18471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4dc98c9b80 20 client.18471 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584de3e1d40 0x5584de3e4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4dc98c9b80 4 client.18471 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3538102637 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584de3977b0 con 0x5584de3e1d40 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.068+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584de3e1d40 0x5584de3e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.069+0000 7f4d9e7fc700 10 client.18471 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.069+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584de3e1d40 0x5584de3e4160 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8051165a960b160 server_cookie=6ff70785c64379d4 in_seq=0 out_seq=0 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.069+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3538102637 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584de3977b0 con 0x5584de3e1d40 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.069+0000 7f4d9e7fc700 10 client.18471 handle_command_reply: tid=0 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.069+0000 7f4dc98c9b80 1 client.18471 shutdown 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3538102637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 msgr2=0x5584de3d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3d1b30 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4db8069330 tx=0x7f4da80047c0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3538102637 shutdown_connections 2024-05-08T23:57:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584de3e1d40 0x5584de3e4160 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3ce270 0x5584de3d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3c8480 0x5584de3d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4d9e7fc700 0 client.18471 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3538102637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3d1b30 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3538102637 >> 172.21.15.12:0/3538102637 conn(0x5584de3261b0 msgr2=0x5584de3c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3538102637 shutdown_connections 2024-05-08T23:57:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3538102637 wait complete. 2024-05-08T23:57:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.070+0000 7f4dc98c9b80 20 client.18471 trim_cache size 0 max 16384 2024-05-08T23:57:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4dc98c9b80 1 Processor -- start 2024-05-08T23:57:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4dc98c9b80 1 -- start start 2024-05-08T23:57:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3c8480 0x5584de3c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3bee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de3df320 con 0x5584de3cd950 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de3c40f0 con 0x5584de3c8480 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de3e9370 con 0x5584de3ce270 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4db67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4db67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54622/0 (socket says 172.21.15.12:54622) 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.086+0000 7f4db67fc700 1 -- 172.21.15.12:0/1980447709 learned_addr learned my addr 172.21.15.12:0/1980447709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4db77fe700 1 --2- 172.21.15.12:0/1980447709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/1980447709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3c8480 0x5584de3c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4db67fc700 1 -- 172.21.15.12:0/1980447709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 msgr2=0x5584de3bee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4db67fc700 1 --2- 172.21.15.12:0/1980447709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3bee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4db67fc700 1 -- 172.21.15.12:0/1980447709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3c8480 msgr2=0x5584de3c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4db67fc700 1 --2- 172.21.15.12:0/1980447709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3c8480 0x5584de3c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4db67fc700 1 -- 172.21.15.12:0/1980447709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584de145b00 con 0x5584de3cd950 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4db67fc700 1 --2- 172.21.15.12:0/1980447709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3c4520 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4da800ec80 tx=0x7f4da800e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a21f32d4d4db22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4dadffb700 1 -- 172.21.15.12:0/1980447709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da800eda0 con 0x5584de3cd950 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4dadffb700 1 -- 172.21.15.12:0/1980447709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584de271210 con 0x5584de3cd950 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.087+0000 7f4dadffb700 1 -- 172.21.15.12:0/1980447709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8003a20 con 0x5584de3cd950 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/1980447709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 msgr2=0x5584de3c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/1980447709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3c4520 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4da800ec80 tx=0x7f4da800e5c0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/1980447709 shutdown_connections 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/1980447709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3bee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/1980447709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3c8480 0x5584de3c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/1980447709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3cd950 0x5584de3c4520 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/1980447709 >> 172.21.15.12:0/1980447709 conn(0x5584de3261b0 msgr2=0x5584de3cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/1980447709 shutdown_connections 2024-05-08T23:57:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/1980447709 wait complete. 2024-05-08T23:57:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 -- start start 2024-05-08T23:57:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 Processor -- start 2024-05-08T23:57:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.088+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 0x5584de3d6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4dc98c9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de3d7680 con 0x5584de3c8480 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de3d77c0 con 0x5584de3cd950 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4dc98c9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584de3d7980 con 0x5584de3ce270 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4dc98c9b80 10 client.0 _ll_register_callbacks cb 0x5584de2bdad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3d8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54638/0 (socket says 172.21.15.12:54638) 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db6ffd700 1 -- 172.21.15.12:0/3804656113 learned_addr learned my addr 172.21.15.12:0/3804656113 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db77fe700 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db67fc700 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 0x5584de3d6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db6ffd700 1 -- 172.21.15.12:0/3804656113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 msgr2=0x5584de3d7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3d7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db6ffd700 1 -- 172.21.15.12:0/3804656113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 msgr2=0x5584de3d6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 0x5584de3d6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db6ffd700 1 -- 172.21.15.12:0/3804656113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db81a4610 con 0x5584de3c8480 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4db6ffd700 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3d8dd0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f4da00983c0 tx=0x7f4da000efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0c8ca0279a4a83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4d9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3804656113 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0099760 con 0x5584de3c8480 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3804656113 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4da00a24f0 con 0x5584de3c8480 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.089+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3804656113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4da009dea0 con 0x5584de3c8480 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3804656113 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4da00b8340 con 0x5584de3c8480 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4dc98c9b80 10 client.18477 fetch_fsmap learned FSMap version 45 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3804656113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5584de2de500 con 0x5584de3c8480 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3804656113 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4da009dea0 con 0x5584de3c8480 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3804656113 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 45) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4da0098fd0 con 0x5584de3c8480 2024-05-08T23:57:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4dc98c9b80 10 client.18477 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T23:57:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4dc98c9b80 10 client.18477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4dc98c9b80 20 client.18477 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584de43c960 0x5584de43ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4dc98c9b80 4 client.18477 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:57:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3804656113 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5584de310e60 con 0x5584de43c960 2024-05-08T23:57:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.090+0000 7f4db67fc700 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584de43c960 0x5584de43ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.091+0000 7f4d9e7fc700 10 client.18477 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.091+0000 7f4db67fc700 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584de43c960 0x5584de43ed80 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d927d70629d65e server_cookie=8a288b9aaf2bca1 in_seq=0 out_seq=0 2024-05-08T23:57:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4d9e7fc700 1 -- 172.21.15.12:0/3804656113 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5584de310e60 con 0x5584de43c960 2024-05-08T23:57:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4d9e7fc700 10 client.18477 handle_command_reply: tid=0 2024-05-08T23:57:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 client.18477 shutdown 2024-05-08T23:57:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3804656113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 msgr2=0x5584de3d8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3d8dd0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f4da00983c0 tx=0x7f4da000efb0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3804656113 shutdown_connections 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5584de3ce270 0x5584de3d7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584de3c8480 0x5584de3d8dd0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584de43c960 0x5584de43ed80 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 --2- 172.21.15.12:0/3804656113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584de3cd950 0x5584de3d6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3804656113 >> 172.21.15.12:0/3804656113 conn(0x5584de3261b0 msgr2=0x5584de3cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4d9e7fc700 0 client.18477 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3804656113 shutdown_connections 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.130+0000 7f4dc98c9b80 1 -- 172.21.15.12:0/3804656113 wait complete. 2024-05-08T23:57:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dc98c9b80 20 client.18477 trim_cache size 0 max 16384 2024-05-08T23:57:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:57:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:57:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:57:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:57:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dbd70f700 1 -- 172.21.15.12:0/134752470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4da40a1290 msgr2=0x7f4da40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dbd70f700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4da40a1290 0x7f4da40a3750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4db8069e30 tx=0x7f4da8009260 comp rx=0 tx=0).stop 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dbd70f700 1 -- 172.21.15.12:0/134752470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4db8100010 msgr2=0x7f4db8198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dbd70f700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4db8100010 0x7f4db8198360 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4da000e9a0 tx=0x7f4da000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dbd70f700 1 -- 172.21.15.12:0/134752470 shutdown_connections 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dbd70f700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4da40a1290 0x7f4da40a3750 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dbd70f700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4db8198de0 0x7f4db81a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dbd70f700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4db8100010 0x7f4db8198360 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dbd70f700 1 --2- 172.21.15.12:0/134752470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4db8105130 0x7f4db81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.131+0000 7f4dbd70f700 1 -- 172.21.15.12:0/134752470 >> 172.21.15.12:0/134752470 conn(0x7f4db80fb870 msgr2=0x7f4db80fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.132+0000 7f4dbd70f700 1 -- 172.21.15.12:0/134752470 shutdown_connections 2024-05-08T23:57:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:05.132+0000 7f4dbd70f700 1 -- 172.21.15.12:0/134752470 wait complete. 2024-05-08T23:57:05.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 594856 6.42 MB/sec execute 421 sec latency 1779.218 ms 2024-05-08T23:57:05.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:05 smithi012 ceph-mon[42503]: pgmap v338: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-05-08T23:57:06.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:05 smithi204 ceph-mon[47373]: pgmap v338: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-05-08T23:57:06.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:05 smithi160 ceph-mon[46152]: pgmap v338: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-05-08T23:57:06.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 596965 6.44 MB/sec execute 422 sec latency 1916.568 ms 2024-05-08T23:57:07.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 598270 6.44 MB/sec execute 423 sec latency 23.134 ms 2024-05-08T23:57:07.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:07 smithi012 ceph-mon[42503]: pgmap v339: 97 pgs: 97 active+clean; 170 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-05-08T23:57:08.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:07 smithi204 ceph-mon[47373]: pgmap v339: 97 pgs: 97 active+clean; 170 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-05-08T23:57:08.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:07 smithi160 ceph-mon[46152]: pgmap v339: 97 pgs: 97 active+clean; 170 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-05-08T23:57:08.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 598403 6.42 MB/sec execute 424 sec latency 926.444 ms 2024-05-08T23:57:09.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 598403 6.41 MB/sec execute 425 sec latency 1926.537 ms 2024-05-08T23:57:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:09 smithi012 ceph-mon[42503]: pgmap v340: 97 pgs: 97 active+clean; 170 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-05-08T23:57:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:09 smithi204 ceph-mon[47373]: pgmap v340: 97 pgs: 97 active+clean; 170 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-05-08T23:57:10.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:09 smithi160 ceph-mon[46152]: pgmap v340: 97 pgs: 97 active+clean; 170 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-05-08T23:57:10.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 598403 6.39 MB/sec execute 426 sec latency 2926.605 ms 2024-05-08T23:57:11.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 600495 6.41 MB/sec execute 427 sec latency 3071.912 ms 2024-05-08T23:57:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:11 smithi012 ceph-mon[42503]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T23:57:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:11 smithi012 ceph-mon[42503]: pgmap v342: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-08T23:57:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:11 smithi204 ceph-mon[47373]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T23:57:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:11 smithi204 ceph-mon[47373]: pgmap v342: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-08T23:57:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:11 smithi160 ceph-mon[46152]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T23:57:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:11 smithi160 ceph-mon[46152]: pgmap v342: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-08T23:57:12.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 601763 6.41 MB/sec execute 428 sec latency 16.610 ms 2024-05-08T23:57:13.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:12 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:13.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:12 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:13.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:12 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:13.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 601947 6.39 MB/sec execute 429 sec latency 796.543 ms 2024-05-08T23:57:14.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:13 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:57:14.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:13 smithi012 ceph-mon[42503]: pgmap v343: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-08T23:57:14.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:13 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:57:14.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:13 smithi204 ceph-mon[47373]: pgmap v343: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-08T23:57:14.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:13 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:57:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:13 smithi160 ceph-mon[46152]: pgmap v343: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-08T23:57:14.888 INFO:tasks.workunit.client.0.smithi012.stdout: 1 601947 6.38 MB/sec execute 430 sec latency 1796.677 ms 2024-05-08T23:57:15.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:14 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:57:15.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:14 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:57:15.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:14 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:57:15.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:14 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:57:15.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:14 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:57:15.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:14 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:57:15.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 601947 6.36 MB/sec execute 431 sec latency 2796.759 ms 2024-05-08T23:57:16.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:15 smithi012 ceph-mon[42503]: pgmap v344: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-08T23:57:16.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:15 smithi204 ceph-mon[47373]: pgmap v344: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-08T23:57:16.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:15 smithi160 ceph-mon[46152]: pgmap v344: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-08T23:57:16.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 603944 6.38 MB/sec execute 432 sec latency 2962.326 ms 2024-05-08T23:57:17.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 605229 6.38 MB/sec execute 433 sec latency 25.356 ms 2024-05-08T23:57:17.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:17 smithi012 ceph-mon[42503]: pgmap v345: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-05-08T23:57:18.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:17 smithi204 ceph-mon[47373]: pgmap v345: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-05-08T23:57:18.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:17 smithi160 ceph-mon[46152]: pgmap v345: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-05-08T23:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.264+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/908993642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54106250 msgr2=0x7f1b5410ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.264+0000 7f1b5c1ac700 1 --2- 172.21.15.12:0/908993642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54106250 0x7f1b5410ddd0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00b0d0 tx=0x7f1b4c00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.264+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/908993642 shutdown_connections 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.264+0000 7f1b5c1ac700 1 --2- 172.21.15.12:0/908993642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54106250 0x7f1b5410ddd0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.264+0000 7f1b5c1ac700 1 --2- 172.21.15.12:0/908993642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b54069440 0x7f1b54069820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.264+0000 7f1b5c1ac700 1 --2- 172.21.15.12:0/908993642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b54069df0 0x7f1b54105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.264+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/908993642 >> 172.21.15.12:0/908993642 conn(0x7f1b54076f40 msgr2=0x7f1b54077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.265+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/908993642 shutdown_connections 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.265+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/908993642 wait complete. 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.265+0000 7f1b5c1ac700 1 Processor -- start 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.265+0000 7f1b5c1ac700 1 -- start start 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.265+0000 7f1b5c1ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b54069440 0x7f1b540ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.265+0000 7f1b5c1ac700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b54106250 0x7f1b54100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.265+0000 7f1b5c1ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54100890 0x7f1b541a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.265+0000 7f1b5c1ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b541a5c70 con 0x7f1b54100890 2024-05-08T23:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.265+0000 7f1b5c1ac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b541a5db0 con 0x7f1b54069440 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.265+0000 7f1b5c1ac700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b541a5f70 con 0x7f1b54106250 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5a749700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54100890 0x7f1b541a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5a749700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54100890 0x7f1b541a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56344/0 (socket says 172.21.15.12:56344) 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5a749700 1 -- 172.21.15.12:0/339672177 learned_addr learned my addr 172.21.15.12:0/339672177 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b59f48700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b54069440 0x7f1b540ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b59747700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b54106250 0x7f1b54100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5a749700 1 -- 172.21.15.12:0/339672177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b54106250 msgr2=0x7f1b54100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5a749700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b54106250 0x7f1b54100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5a749700 1 -- 172.21.15.12:0/339672177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b54069440 msgr2=0x7f1b540ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5a749700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b54069440 0x7f1b540ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5a749700 1 -- 172.21.15.12:0/339672177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c009d00 con 0x7f1b54100890 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5a749700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54100890 0x7f1b541a56a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00d010 tx=0x7f1b4c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b4affd700 1 -- 172.21.15.12:0/339672177 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c00e040 con 0x7f1b54100890 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/339672177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b541a61f0 con 0x7f1b54100890 2024-05-08T23:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b4affd700 1 -- 172.21.15.12:0/339672177 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b4c005070 con 0x7f1b54100890 2024-05-08T23:57:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b4affd700 1 -- 172.21.15.12:0/339672177 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c01b6d0 con 0x7f1b54100890 2024-05-08T23:57:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.266+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/339672177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b541a6610 con 0x7f1b54100890 2024-05-08T23:57:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.269+0000 7f1b4affd700 1 -- 172.21.15.12:0/339672177 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1b4c0054e0 con 0x7f1b54100890 2024-05-08T23:57:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.269+0000 7f1b4affd700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1b400a1290 0x7f1b400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.269+0000 7f1b4affd700 1 -- 172.21.15.12:0/339672177 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1b4c0cdd70 con 0x7f1b54100890 2024-05-08T23:57:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.269+0000 7f1b59f48700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1b400a1290 0x7f1b400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.270+0000 7f1b59f48700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1b400a1290 0x7f1b400a3750 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1b50005950 tx=0x7f1b5000b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.270+0000 7f1b68366b80 1 Processor -- start 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b68366b80 1 -- start start 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5adf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b68366b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f5050f0 con 0x56267f5b4270 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b68366b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f502ab0 con 0x56267f5b3950 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b68366b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f4fe3a0 con 0x56267f5ae480 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b59747700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b59747700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56360/0 (socket says 172.21.15.12:56360) 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b59747700 1 -- 172.21.15.12:0/3452327300 learned_addr learned my addr 172.21.15.12:0/3452327300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b59f48700 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b5a749700 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b59747700 1 -- 172.21.15.12:0/3452327300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 msgr2=0x56267f5ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.271+0000 7f1b59747700 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.272+0000 7f1b59747700 1 -- 172.21.15.12:0/3452327300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 msgr2=0x56267f5b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.272+0000 7f1b59747700 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.272+0000 7f1b59747700 1 -- 172.21.15.12:0/3452327300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56267f37f350 con 0x56267f5b4270 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.272+0000 7f1b5a749700 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.272+0000 7f1b59f48700 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.272+0000 7f1b59747700 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5adf40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1b44007be0 tx=0x7f1b44007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5abcd80adfdd72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.272+0000 7f1b3ffff700 1 -- 172.21.15.12:0/3452327300 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44010cf0 con 0x56267f5b4270 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.272+0000 7f1b3ffff700 1 -- 172.21.15.12:0/3452327300 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b44010e50 con 0x56267f5b4270 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.272+0000 7f1b3ffff700 1 -- 172.21.15.12:0/3452327300 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4401a470 con 0x56267f5b4270 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 -- 172.21.15.12:0/3452327300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 msgr2=0x56267f5adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5adf40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1b44007be0 tx=0x7f1b44007ef0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 -- 172.21.15.12:0/3452327300 shutdown_connections 2024-05-08T23:57:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 --2- 172.21.15.12:0/3452327300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5adf40 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 -- 172.21.15.12:0/3452327300 >> 172.21.15.12:0/3452327300 conn(0x56267f50c1b0 msgr2=0x56267f5a8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 -- 172.21.15.12:0/3452327300 shutdown_connections 2024-05-08T23:57:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 -- 172.21.15.12:0/3452327300 wait complete. 2024-05-08T23:57:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 -- start start 2024-05-08T23:57:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.273+0000 7f1b68366b80 1 Processor -- start 2024-05-08T23:57:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5ba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b68366b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f49a500 con 0x56267f5b4270 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b68366b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f49a1a0 con 0x56267f5b3950 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b68366b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f381640 con 0x56267f5ae480 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b68366b80 10 client.0 _ll_register_callbacks cb 0x56267f4a3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b5a749700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b5a749700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5b7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56376/0 (socket says 172.21.15.12:56376) 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b59747700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b5a749700 1 -- 172.21.15.12:0/3280389920 learned_addr learned my addr 172.21.15.12:0/3280389920 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b59f48700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5ba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b5a749700 1 -- 172.21.15.12:0/3280389920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 msgr2=0x56267f5ba2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b5a749700 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5ba2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b5a749700 1 -- 172.21.15.12:0/3280389920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 msgr2=0x56267f5b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b5a749700 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5b7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b5a749700 1 -- 172.21.15.12:0/3280389920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b44007840 con 0x56267f5b4270 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b59747700 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b5a749700 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5b7cd0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009fd0 tx=0x7f1b4c09ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81698c1c6b63819 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.274+0000 7f1b3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/3280389920 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c0a3080 con 0x56267f5b4270 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/3280389920 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1b4c09af20 con 0x56267f5b4270 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b68366b80 1 -- 172.21.15.12:0/3280389920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56267f457210 con 0x56267f5b4270 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/3280389920 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b4c09ed50 con 0x56267f5b4270 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b68366b80 10 client.18489 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b68366b80 1 -- 172.21.15.12:0/3280389920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56267f32bb00 con 0x56267f5b4270 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/3280389920 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b4c064080 con 0x56267f5b4270 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/3280389920 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1b4c09a410 con 0x56267f5b4270 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b68366b80 10 client.18489 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b68366b80 10 client.18489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b68366b80 20 client.18489 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b68366b80 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56267f5c7980 0x56267f5c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b68366b80 4 client.18489 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b68366b80 1 -- 172.21.15.12:0/3280389920 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56267f57d7b0 con 0x56267f5c7980 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.275+0000 7f1b59f48700 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56267f5c7980 0x56267f5c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.276+0000 7f1b59f48700 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56267f5c7980 0x56267f5c9d70 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676c59d0d7495a19 server_cookie=891f283994996551 in_seq=0 out_seq=0 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.276+0000 7f1b3d7fa700 10 client.18489 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.276+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/3280389920 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56267f57d7b0 con 0x56267f5c7980 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.276+0000 7f1b3d7fa700 10 client.18489 handle_command_reply: tid=0 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.276+0000 7f1b68366b80 1 client.18489 shutdown 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 1 -- 172.21.15.12:0/3280389920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 msgr2=0x56267f5b7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5b7cd0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009fd0 tx=0x7f1b4c09ade0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 1 -- 172.21.15.12:0/3280389920 shutdown_connections 2024-05-08T23:57:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5b7cd0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5ba2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56267f5c7980 0x56267f5c9d70 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 1 --2- 172.21.15.12:0/3280389920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 1 -- 172.21.15.12:0/3280389920 >> 172.21.15.12:0/3280389920 conn(0x56267f50c1b0 msgr2=0x56267f5b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b3d7fa700 0 client.18489 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 1 -- 172.21.15.12:0/3280389920 shutdown_connections 2024-05-08T23:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 1 -- 172.21.15.12:0/3280389920 wait complete. 2024-05-08T23:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.277+0000 7f1b68366b80 20 client.18489 trim_cache size 0 max 16384 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b68366b80 1 Processor -- start 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b68366b80 1 -- start start 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b3950 0x56267f5a8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b4270 0x56267f5b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b68366b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f5cef50 con 0x56267f5b3950 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b68366b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f5a9580 con 0x56267f5b4270 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b68366b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f5ad180 con 0x56267f5ae480 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b59747700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b3950 0x56267f5a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b59747700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b3950 0x56267f5a8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56380/0 (socket says 172.21.15.12:56380) 2024-05-08T23:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b59747700 1 -- 172.21.15.12:0/2443196492 learned_addr learned my addr 172.21.15.12:0/2443196492 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b59f48700 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.293+0000 7f1b5a749700 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b4270 0x56267f5b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b59f48700 1 -- 172.21.15.12:0/2443196492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b4270 msgr2=0x56267f5b9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b59f48700 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b4270 0x56267f5b9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b59f48700 1 -- 172.21.15.12:0/2443196492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b3950 msgr2=0x56267f5a8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b59f48700 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b3950 0x56267f5a8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b59f48700 1 -- 172.21.15.12:0/2443196492 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56267f4c4500 con 0x56267f5ae480 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b59747700 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b3950 0x56267f5a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b5a749700 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b4270 0x56267f5b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b59f48700 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5b7530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c099270 tx=0x7f1b50013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e9e669c884d7873 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b48ff9700 1 -- 172.21.15.12:0/2443196492 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5002b070 con 0x56267f5ae480 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b48ff9700 1 -- 172.21.15.12:0/2443196492 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b50015070 con 0x56267f5ae480 2024-05-08T23:57:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.294+0000 7f1b48ff9700 1 -- 172.21.15.12:0/2443196492 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50025070 con 0x56267f5ae480 2024-05-08T23:57:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 -- 172.21.15.12:0/2443196492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 msgr2=0x56267f5b7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5b7530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c099270 tx=0x7f1b50013000 comp rx=0 tx=0).stop 2024-05-08T23:57:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 -- 172.21.15.12:0/2443196492 shutdown_connections 2024-05-08T23:57:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b4270 0x56267f5b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5b7530 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 --2- 172.21.15.12:0/2443196492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b3950 0x56267f5a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 -- 172.21.15.12:0/2443196492 >> 172.21.15.12:0/2443196492 conn(0x56267f50c1b0 msgr2=0x56267f5b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 -- 172.21.15.12:0/2443196492 shutdown_connections 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 -- 172.21.15.12:0/2443196492 wait complete. 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 -- start start 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.295+0000 7f1b68366b80 1 Processor -- start 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5bcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5bd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b68366b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5bd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b68366b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f61cfa0 con 0x56267f5b4270 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b68366b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f61d110 con 0x56267f5b3950 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b68366b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267f61d2d0 con 0x56267f5ae480 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b68366b80 10 client.0 _ll_register_callbacks cb 0x56267f4a3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b5a749700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5bd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b5a749700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5bd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56388/0 (socket says 172.21.15.12:56388) 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b5a749700 1 -- 172.21.15.12:0/2554120604 learned_addr learned my addr 172.21.15.12:0/2554120604 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b59f48700 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5bcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b59747700 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5bd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b5a749700 1 -- 172.21.15.12:0/2554120604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 msgr2=0x56267f5bcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b5a749700 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5bcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b5a749700 1 -- 172.21.15.12:0/2554120604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 msgr2=0x56267f5bd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b5a749700 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5bd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b5a749700 1 -- 172.21.15.12:0/2554120604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56267f37f350 con 0x56267f5b4270 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.296+0000 7f1b59747700 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5bd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b5a749700 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5bd6a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c09aeb0 tx=0x7f1b4c09b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f54351b85ad425 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/2554120604 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c09af20 con 0x56267f5b4270 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/2554120604 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1b4c09ed50 con 0x56267f5b4270 2024-05-08T23:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b68366b80 1 -- 172.21.15.12:0/2554120604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b44010e50 con 0x56267f5b4270 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/2554120604 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b4c0a3310 con 0x56267f5b4270 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b68366b80 10 client.18495 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b68366b80 1 -- 172.21.15.12:0/2554120604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56267f32bb00 con 0x56267f5b4270 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/2554120604 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b4c07d020 con 0x56267f5b4270 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/2554120604 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1b4c019cf0 con 0x56267f5b4270 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b68366b80 10 client.18495 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b68366b80 10 client.18495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b68366b80 20 client.18495 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b68366b80 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56267f61e740 0x56267f620b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b68366b80 4 client.18495 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.297+0000 7f1b68366b80 1 -- 172.21.15.12:0/2554120604 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56267f4f6e60 con 0x56267f61e740 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.298+0000 7f1b59f48700 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56267f61e740 0x56267f620b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.298+0000 7f1b59f48700 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56267f61e740 0x56267f620b30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33162529196b708d server_cookie=f5a939921b642853 in_seq=0 out_seq=0 2024-05-08T23:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.298+0000 7f1b3d7fa700 10 client.18495 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b3d7fa700 1 -- 172.21.15.12:0/2554120604 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56267f4f6e60 con 0x56267f61e740 2024-05-08T23:57:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b3d7fa700 10 client.18495 handle_command_reply: tid=0 2024-05-08T23:57:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 client.18495 shutdown 2024-05-08T23:57:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 -- 172.21.15.12:0/2554120604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 msgr2=0x56267f5bd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5bd6a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c09aeb0 tx=0x7f1b4c09b520 comp rx=0 tx=0).stop 2024-05-08T23:57:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 -- 172.21.15.12:0/2554120604 shutdown_connections 2024-05-08T23:57:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56267f5b4270 0x56267f5bd6a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56267f5ae480 0x56267f5bcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56267f61e740 0x56267f620b30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 --2- 172.21.15.12:0/2554120604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56267f5b3950 0x56267f5bd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b3d7fa700 0 client.18495 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 -- 172.21.15.12:0/2554120604 >> 172.21.15.12:0/2554120604 conn(0x56267f50c1b0 msgr2=0x56267f5ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 -- 172.21.15.12:0/2554120604 shutdown_connections 2024-05-08T23:57:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 1 -- 172.21.15.12:0/2554120604 wait complete. 2024-05-08T23:57:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.346+0000 7f1b68366b80 20 client.18495 trim_cache size 0 max 16384 2024-05-08T23:57:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:57:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:57:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:57:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:57:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/339672177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1b400a1290 msgr2=0x7f1b400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1b400a1290 0x7f1b400a3750 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1b50005950 tx=0x7f1b5000b410 comp rx=0 tx=0).stop 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/339672177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54100890 msgr2=0x7f1b541a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54100890 0x7f1b541a56a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00d010 tx=0x7f1b4c009440 comp rx=0 tx=0).stop 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/339672177 shutdown_connections 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1b400a1290 0x7f1b400a3750 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54100890 0x7f1b541a56a0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b54069440 0x7f1b540ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 --2- 172.21.15.12:0/339672177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b54106250 0x7f1b54100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/339672177 >> 172.21.15.12:0/339672177 conn(0x7f1b54076f40 msgr2=0x7f1b540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/339672177 shutdown_connections 2024-05-08T23:57:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:18.347+0000 7f1b5c1ac700 1 -- 172.21.15.12:0/339672177 wait complete. 2024-05-08T23:57:18.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 605495 6.36 MB/sec execute 434 sec latency 658.952 ms 2024-05-08T23:57:19.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 605495 6.35 MB/sec execute 435 sec latency 1659.079 ms 2024-05-08T23:57:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:19 smithi012 ceph-mon[42503]: pgmap v346: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-05-08T23:57:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:19 smithi204 ceph-mon[47373]: pgmap v346: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-05-08T23:57:20.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:19 smithi160 ceph-mon[46152]: pgmap v346: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-05-08T23:57:20.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 605495 6.33 MB/sec execute 436 sec latency 2659.219 ms 2024-05-08T23:57:21.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 607570 6.35 MB/sec execute 437 sec latency 2803.607 ms 2024-05-08T23:57:22.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:21 smithi012 ceph-mon[42503]: pgmap v347: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-05-08T23:57:22.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:21 smithi204 ceph-mon[47373]: pgmap v347: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-05-08T23:57:22.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:21 smithi160 ceph-mon[46152]: pgmap v347: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-05-08T23:57:22.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 608839 6.35 MB/sec execute 438 sec latency 23.110 ms 2024-05-08T23:57:23.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 609544 6.33 MB/sec execute 439 sec latency 603.654 ms 2024-05-08T23:57:23.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:23 smithi012 ceph-mon[42503]: pgmap v348: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-08T23:57:24.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:23 smithi204 ceph-mon[47373]: pgmap v348: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-08T23:57:24.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:23 smithi160 ceph-mon[46152]: pgmap v348: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-08T23:57:24.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 611434 6.35 MB/sec execute 440 sec latency 13.581 ms 2024-05-08T23:57:25.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 612586 6.35 MB/sec execute 441 sec latency 287.712 ms 2024-05-08T23:57:25.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:25 smithi012 ceph-mon[42503]: pgmap v349: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-08T23:57:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:25 smithi204 ceph-mon[47373]: pgmap v349: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-08T23:57:26.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:25 smithi160 ceph-mon[46152]: pgmap v349: 97 pgs: 97 active+clean; 183 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-08T23:57:26.889 INFO:tasks.workunit.client.0.smithi012.stdout: 1 614679 6.36 MB/sec execute 442 sec latency 421.269 ms 2024-05-08T23:57:27.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 615932 6.36 MB/sec execute 443 sec latency 23.073 ms 2024-05-08T23:57:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:27 smithi012 ceph-mon[42503]: pgmap v350: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2024-05-08T23:57:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:27 smithi204 ceph-mon[47373]: pgmap v350: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2024-05-08T23:57:28.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:27 smithi160 ceph-mon[46152]: pgmap v350: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2024-05-08T23:57:28.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 616129 6.35 MB/sec execute 444 sec latency 745.886 ms 2024-05-08T23:57:29.200 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:57:29.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.363+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/3244059283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cd4105160 msgr2=0x7f9cd410cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.363+0000 7f9cd8aeb700 1 --2- 172.21.15.12:0/3244059283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cd4105160 0x7f9cd410cce0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00b0a0 tx=0x7f9ccc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.363+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/3244059283 shutdown_connections 2024-05-08T23:57:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.363+0000 7f9cd8aeb700 1 --2- 172.21.15.12:0/3244059283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cd4105160 0x7f9cd410cce0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.363+0000 7f9cd8aeb700 1 --2- 172.21.15.12:0/3244059283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cd4100060 0x7f9cd4100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.363+0000 7f9cd8aeb700 1 --2- 172.21.15.12:0/3244059283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd4100a10 0x7f9cd4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.363+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/3244059283 >> 172.21.15.12:0/3244059283 conn(0x7f9cd4076f40 msgr2=0x7f9cd4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.363+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/3244059283 shutdown_connections 2024-05-08T23:57:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.363+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/3244059283 wait complete. 2024-05-08T23:57:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.364+0000 7f9cd8aeb700 1 Processor -- start 2024-05-08T23:57:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.364+0000 7f9cd8aeb700 1 -- start start 2024-05-08T23:57:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cd8aeb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cd4100060 0x7f9cd4197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cd8aeb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cd4105160 0x7f9cd4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cd8aeb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd419bde0 0x7f9cd419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cd8aeb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd419c860 con 0x7f9cd4105160 2024-05-08T23:57:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cd8aeb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd419c9d0 con 0x7f9cd4100060 2024-05-08T23:57:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cd8aeb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd419cb90 con 0x7f9cd419bde0 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cd4105160 0x7f9cd4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cd4105160 0x7f9cd4197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43938/0 (socket says 172.21.15.12:43938) 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/943486073 learned_addr learned my addr 172.21.15.12:0/943486073 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cd2d9d700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd419bde0 0x7f9cd419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cd259c700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cd4100060 0x7f9cd4197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/943486073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd419bde0 msgr2=0x7f9cd419c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd419bde0 0x7f9cd419c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/943486073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cd4100060 msgr2=0x7f9cd4197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cd4100060 0x7f9cd4197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/943486073 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc009520 con 0x7f9cd4105160 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cd259c700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cd4100060 0x7f9cd4197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cd4105160 0x7f9cd4197a40 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00ea30 tx=0x7f9cbc00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9ccbfff700 1 -- 172.21.15.12:0/943486073 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc009820 con 0x7f9cd4105160 2024-05-08T23:57:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9ccbfff700 1 -- 172.21.15.12:0/943486073 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cbc0044e0 con 0x7f9cd4105160 2024-05-08T23:57:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/943486073 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc009d00 con 0x7f9cd4105160 2024-05-08T23:57:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.365+0000 7f9ccbfff700 1 -- 172.21.15.12:0/943486073 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc003dc0 con 0x7f9cd4105160 2024-05-08T23:57:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.366+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/943486073 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd41a4440 con 0x7f9cd4105160 2024-05-08T23:57:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.368+0000 7f9ccbfff700 1 -- 172.21.15.12:0/943486073 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9cbc003f80 con 0x7f9cd4105160 2024-05-08T23:57:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.368+0000 7f9ccbfff700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9cc00a1290 0x7f9cc00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.369+0000 7f9ccbfff700 1 -- 172.21.15.12:0/943486073 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9cbc014070 con 0x7f9cd4105160 2024-05-08T23:57:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.369+0000 7f9cd259c700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9cc00a1290 0x7f9cc00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.369+0000 7f9cd259c700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9cc00a1290 0x7f9cc00a3750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400bc10 tx=0x7f9cc400b5f0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.369+0000 7f9ce4ca5b80 1 Processor -- start 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9ce4ca5b80 1 -- start start 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 0x563e8b6851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b685720 0x563e8b67f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 0x563e8b67fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b5d6150 con 0x563e8b685720 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b5d3b10 con 0x563e8b67f890 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b5cf400 con 0x563e8b684e00 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b685720 0x563e8b67f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b685720 0x563e8b67f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43946/0 (socket says 172.21.15.12:43946) 2024-05-08T23:57:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/285476858 learned_addr learned my addr 172.21.15.12:0/285476858 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.370+0000 7f9cd259c700 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 0x563e8b6851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cd2d9d700 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 0x563e8b67fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/285476858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 msgr2=0x563e8b6851e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 0x563e8b6851e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/285476858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 msgr2=0x563e8b67fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 0x563e8b67fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/285476858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e8b4503b0 con 0x563e8b685720 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cd2d9d700 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 0x563e8b67fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cd259c700 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 0x563e8b6851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b685720 0x563e8b67f350 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc096600 tx=0x7f9cbc0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed91ade7381d3f98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cc8d99700 1 -- 172.21.15.12:0/285476858 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc09a100 con 0x563e8b685720 2024-05-08T23:57:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cc8d99700 1 -- 172.21.15.12:0/285476858 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cbc0cae90 con 0x563e8b685720 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.371+0000 7f9cc8d99700 1 -- 172.21.15.12:0/285476858 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc004230 con 0x563e8b685720 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.372+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/285476858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b685720 msgr2=0x563e8b67f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.372+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b685720 0x563e8b67f350 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc096600 tx=0x7f9cbc0057e0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.372+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/285476858 shutdown_connections 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.372+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 0x563e8b67fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.372+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 0x563e8b6851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.372+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/285476858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b685720 0x563e8b67f350 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.372+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/285476858 >> 172.21.15.12:0/285476858 conn(0x563e8b5dd210 msgr2=0x563e8b679df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.372+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/285476858 shutdown_connections 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.372+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/285476858 wait complete. 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.372+0000 7f9ce4ca5b80 1 -- start start 2024-05-08T23:57:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9ce4ca5b80 1 Processor -- start 2024-05-08T23:57:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b688a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 0x563e8b688fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b685720 0x563e8b689510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b56b560 con 0x563e8b67f890 2024-05-08T23:57:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b56b200 con 0x563e8b685720 2024-05-08T23:57:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b4526a0 con 0x563e8b684e00 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9ce4ca5b80 10 client.0 _ll_register_callbacks cb 0x563e8b574b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cd259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b688a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cd259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b688a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43948/0 (socket says 172.21.15.12:43948) 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cd259c700 1 -- 172.21.15.12:0/1101037267 learned_addr learned my addr 172.21.15.12:0/1101037267 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cd2d9d700 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b685720 0x563e8b689510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 0x563e8b688fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/1101037267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b685720 msgr2=0x563e8b689510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b685720 0x563e8b689510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/1101037267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 msgr2=0x563e8b688a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b688a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/1101037267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd41a4440 con 0x563e8b684e00 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.373+0000 7f9cd259c700 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b688a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 0x563e8b688fd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0cd100 tx=0x7f9cbc0ca760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d7adcaf4473b440 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9cb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/1101037267 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0a2580 con 0x563e8b684e00 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/1101037267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e8b528270 con 0x563e8b684e00 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/1101037267 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9cbc09a100 con 0x563e8b684e00 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/1101037267 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cbc0984a0 con 0x563e8b684e00 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9ce4ca5b80 10 client.28237 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/1101037267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x563e8b3fcb60 con 0x563e8b684e00 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/1101037267 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cbc0a6020 con 0x563e8b684e00 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/1101037267 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9cbc0995f0 con 0x563e8b684e00 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9ce4ca5b80 10 client.28237 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9ce4ca5b80 10 client.28237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9ce4ca5b80 20 client.28237 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e8b698d40 0x563e8b69b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9ce4ca5b80 4 client.28237 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.374+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/1101037267 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e8b64e810 con 0x563e8b698d40 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.375+0000 7f9cd259c700 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e8b698d40 0x563e8b69b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.375+0000 7f9cb9ffb700 10 client.28237 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.375+0000 7f9cd259c700 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e8b698d40 0x563e8b69b130 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abaaa9ebf44b9f7f server_cookie=640f8a47e0423e0c in_seq=0 out_seq=0 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/1101037267 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e8b64e810 con 0x563e8b698d40 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9cb9ffb700 10 client.28237 handle_command_reply: tid=0 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 client.28237 shutdown 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/1101037267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 msgr2=0x563e8b688fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 0x563e8b688fd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0cd100 tx=0x7f9cbc0ca760 comp rx=0 tx=0).stop 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/1101037267 shutdown_connections 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e8b698d40 0x563e8b69b130 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b685720 0x563e8b689510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b688a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/1101037267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b684e00 0x563e8b688fd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9cb9ffb700 0 client.28237 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/1101037267 >> 172.21.15.12:0/1101037267 conn(0x563e8b5dd210 msgr2=0x563e8b6795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/1101037267 shutdown_connections 2024-05-08T23:57:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/1101037267 wait complete. 2024-05-08T23:57:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.376+0000 7f9ce4ca5b80 20 client.28237 trim_cache size 0 max 16384 2024-05-08T23:57:29.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.394+0000 7f9ce4ca5b80 1 Processor -- start 2024-05-08T23:57:29.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.394+0000 7f9ce4ca5b80 1 -- start start 2024-05-08T23:57:29.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.394+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 0x563e8b679d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.394+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b684e00 0x563e8b690160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.394+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b685720 0x563e8b6906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.394+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b68b600 con 0x563e8b684e00 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.394+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b676dc0 con 0x563e8b67f890 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.394+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b672040 con 0x563e8b685720 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b684e00 0x563e8b690160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b684e00 0x563e8b690160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43962/0 (socket says 172.21.15.12:43962) 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/2843742099 learned_addr learned my addr 172.21.15.12:0/2843742099 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cd2d9d700 1 --2- 172.21.15.12:0/2843742099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b685720 0x563e8b6906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cd259c700 1 --2- 172.21.15.12:0/2843742099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 0x563e8b679d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/2843742099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b685720 msgr2=0x563e8b6906a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/2843742099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b685720 0x563e8b6906a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/2843742099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 msgr2=0x563e8b679d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/2843742099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 0x563e8b679d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc9d9b700 1 -- 172.21.15.12:0/2843742099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e8b595560 con 0x563e8b684e00 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cd259c700 1 --2- 172.21.15.12:0/2843742099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 0x563e8b679d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/2843742099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b684e00 0x563e8b690160 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc099920 tx=0x7f9cbc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b75b2abca751e8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc959a700 1 -- 172.21.15.12:0/2843742099 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00cbd0 con 0x563e8b684e00 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc959a700 1 -- 172.21.15.12:0/2843742099 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cbc0a6070 con 0x563e8b684e00 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.395+0000 7f9cc959a700 1 -- 172.21.15.12:0/2843742099 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0995f0 con 0x563e8b684e00 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/2843742099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b684e00 msgr2=0x563e8b690160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/2843742099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b684e00 0x563e8b690160 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc099920 tx=0x7f9cbc003820 comp rx=0 tx=0).stop 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/2843742099 shutdown_connections 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/2843742099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b685720 0x563e8b6906a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/2843742099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b67f890 0x563e8b679d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/2843742099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b684e00 0x563e8b690160 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/2843742099 >> 172.21.15.12:0/2843742099 conn(0x563e8b5dd210 msgr2=0x563e8b69f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/2843742099 shutdown_connections 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/2843742099 wait complete. 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 -- start start 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.396+0000 7f9ce4ca5b80 1 Processor -- start 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b68c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b684e00 0x563e8b68c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9ce4ca5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b685720 0x563e8b6ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b68cf90 con 0x563e8b67f890 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b6ee660 con 0x563e8b684e00 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9ce4ca5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8b6ee7a0 con 0x563e8b685720 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9ce4ca5b80 10 client.0 _ll_register_callbacks cb 0x563e8b574b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cd259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b68c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cd259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b68c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43976/0 (socket says 172.21.15.12:43976) 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cd259c700 1 -- 172.21.15.12:0/4100904580 learned_addr learned my addr 172.21.15.12:0/4100904580 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b684e00 0x563e8b68c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cd2d9d700 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b685720 0x563e8b6ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cd259c700 1 -- 172.21.15.12:0/4100904580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b685720 msgr2=0x563e8b6ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cd259c700 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b685720 0x563e8b6ee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cd259c700 1 -- 172.21.15.12:0/4100904580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b684e00 msgr2=0x563e8b68c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cd259c700 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b684e00 0x563e8b68c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cd259c700 1 -- 172.21.15.12:0/4100904580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e8b3fcb60 con 0x563e8b67f890 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b684e00 0x563e8b68c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cd259c700 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b68c140 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4014040 tx=0x7f9cc40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5341aa152b054511 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.397+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/4100904580 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4025050 con 0x563e8b67f890 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/4100904580 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9cc4027d70 con 0x563e8b67f890 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/4100904580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cbc0cae90 con 0x563e8b67f890 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/4100904580 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cc4011e60 con 0x563e8b67f890 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9ce4ca5b80 10 client.18513 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/4100904580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x563e8b454cd0 con 0x563e8b67f890 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/4100904580 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cbc0cae90 con 0x563e8b67f890 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/4100904580 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9cc4037070 con 0x563e8b67f890 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9ce4ca5b80 10 client.18513 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9ce4ca5b80 10 client.18513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9ce4ca5b80 20 client.18513 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e8b6efd80 0x563e8b6f2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9ce4ca5b80 4 client.18513 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/4100904580 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563e8b5c7ec0 con 0x563e8b6efd80 2024-05-08T23:57:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.398+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e8b6efd80 0x563e8b6f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9cc9d9b700 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e8b6efd80 0x563e8b6f2170 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b665d6dd56663cd0 server_cookie=81383e6e2236a5e3 in_seq=0 out_seq=0 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9cb9ffb700 10 client.18513 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9cb9ffb700 1 -- 172.21.15.12:0/4100904580 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563e8b5c7ec0 con 0x563e8b6efd80 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9cb9ffb700 10 client.18513 handle_command_reply: tid=0 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 client.18513 shutdown 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/4100904580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 msgr2=0x563e8b68c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b68c140 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4014040 tx=0x7f9cc40047c0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/4100904580 shutdown_connections 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8b685720 0x563e8b6ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e8b67f890 0x563e8b68c140 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e8b6efd80 0x563e8b6f2170 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 --2- 172.21.15.12:0/4100904580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e8b684e00 0x563e8b68c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9cb9ffb700 0 client.18513 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/4100904580 >> 172.21.15.12:0/4100904580 conn(0x563e8b5dd210 msgr2=0x563e8b683b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/4100904580 shutdown_connections 2024-05-08T23:57:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 1 -- 172.21.15.12:0/4100904580 wait complete. 2024-05-08T23:57:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.399+0000 7f9ce4ca5b80 20 client.18513 trim_cache size 0 max 16384 2024-05-08T23:57:29.402 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:57:29.402 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:57:29.402 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:57:29.402 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:57:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/943486073 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9cc00a1290 msgr2=0x7f9cc00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9cc00a1290 0x7f9cc00a3750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400bc10 tx=0x7f9cc400b5f0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/943486073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cd4105160 msgr2=0x7f9cd4197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cd4105160 0x7f9cd4197a40 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00ea30 tx=0x7f9cbc00ed40 comp rx=0 tx=0).stop 2024-05-08T23:57:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/943486073 shutdown_connections 2024-05-08T23:57:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9cc00a1290 0x7f9cc00a3750 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd419bde0 0x7f9cd419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cd4100060 0x7f9cd4197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 --2- 172.21.15.12:0/943486073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cd4105160 0x7f9cd4197a40 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/943486073 >> 172.21.15.12:0/943486073 conn(0x7f9cd4076f40 msgr2=0x7f9cd4068370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/943486073 shutdown_connections 2024-05-08T23:57:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.400+0000 7f9cd8aeb700 1 -- 172.21.15.12:0/943486073 wait complete. 2024-05-08T23:57:29.408 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:57:29.408 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.553+0000 7fb466e6f700 1 -- 172.21.15.12:0/513993566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460073350 msgr2=0x7fb460073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.553+0000 7fb466e6f700 1 --2- 172.21.15.12:0/513993566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460073350 0x7fb460073730 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb454009990 tx=0x7fb454009ca0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.553+0000 7fb466e6f700 1 -- 172.21.15.12:0/513993566 shutdown_connections 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.553+0000 7fb466e6f700 1 --2- 172.21.15.12:0/513993566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb46010d6b0 0x7fb46010faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.553+0000 7fb466e6f700 1 --2- 172.21.15.12:0/513993566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb460073350 0x7fb460073730 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.553+0000 7fb466e6f700 1 --2- 172.21.15.12:0/513993566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460073c70 0x7fb46010d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.553+0000 7fb466e6f700 1 -- 172.21.15.12:0/513993566 >> 172.21.15.12:0/513993566 conn(0x7fb460078360 msgr2=0x7fb460078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.553+0000 7fb466e6f700 1 -- 172.21.15.12:0/513993566 shutdown_connections 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.553+0000 7fb466e6f700 1 -- 172.21.15.12:0/513993566 wait complete. 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb466e6f700 1 Processor -- start 2024-05-08T23:57:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb466e6f700 1 -- start start 2024-05-08T23:57:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb466e6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460073350 0x7fb4601994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb466e6f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460073c70 0x7fb460199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb466e6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb46010d6b0 0x7fb46019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb466e6f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46019e410 con 0x7fb46010d6b0 2024-05-08T23:57:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb466e6f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46019e580 con 0x7fb460073350 2024-05-08T23:57:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb466e6f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46019e740 con 0x7fb460073c70 2024-05-08T23:57:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb46540c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb46010d6b0 0x7fb46019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460073c70 0x7fb460199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460073c70 0x7fb460199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:56576/0 (socket says 172.21.15.12:56576) 2024-05-08T23:57:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb45ffff700 1 -- 172.21.15.12:0/3194491139 learned_addr learned my addr 172.21.15.12:0/3194491139 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb46540c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb46010d6b0 0x7fb46019ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44002/0 (socket says 172.21.15.12:44002) 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.554+0000 7fb464c0b700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460073350 0x7fb4601994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb46540c700 1 -- 172.21.15.12:0/3194491139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460073c70 msgr2=0x7fb460199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb46540c700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460073c70 0x7fb460199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb46540c700 1 -- 172.21.15.12:0/3194491139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460073350 msgr2=0x7fb4601994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb46540c700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460073350 0x7fb4601994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb46540c700 1 -- 172.21.15.12:0/3194491139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4540095f0 con 0x7fb46010d6b0 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb45ffff700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460073c70 0x7fb460199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb46540c700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb46010d6b0 0x7fb46019ddd0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb45800efc0 tx=0x7fb45800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb45dffb700 1 -- 172.21.15.12:0/3194491139 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458018070 con 0x7fb46010d6b0 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb45dffb700 1 -- 172.21.15.12:0/3194491139 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb45800f040 con 0x7fb46010d6b0 2024-05-08T23:57:29.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb466e6f700 1 -- 172.21.15.12:0/3194491139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46019e9c0 con 0x7fb46010d6b0 2024-05-08T23:57:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb45dffb700 1 -- 172.21.15.12:0/3194491139 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800cd40 con 0x7fb46010d6b0 2024-05-08T23:57:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.555+0000 7fb466e6f700 1 -- 172.21.15.12:0/3194491139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4601a6260 con 0x7fb46010d6b0 2024-05-08T23:57:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.556+0000 7fb466e6f700 1 -- 172.21.15.12:0/3194491139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4601088c0 con 0x7fb46010d6b0 2024-05-08T23:57:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.557+0000 7fb45dffb700 1 -- 172.21.15.12:0/3194491139 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb458003a00 con 0x7fb46010d6b0 2024-05-08T23:57:29.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.558+0000 7fb45dffb700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1310 0x7fb4500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:29.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.558+0000 7fb464c0b700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1310 0x7fb4500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:29.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.558+0000 7fb45dffb700 1 -- 172.21.15.12:0/3194491139 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb4580cc7d0 con 0x7fb46010d6b0 2024-05-08T23:57:29.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.559+0000 7fb464c0b700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1310 0x7fb4500a37d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb454006010 tx=0x7fb45400b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:29.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.563+0000 7fb45dffb700 1 -- 172.21.15.12:0/3194491139 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb458095cc0 con 0x7fb46010d6b0 2024-05-08T23:57:29.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.853+0000 7fb466e6f700 1 -- 172.21.15.12:0/3194491139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb46019a7f0 con 0x7fb46010d6b0 2024-05-08T23:57:29.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.853+0000 7fb45dffb700 1 -- 172.21.15.12:0/3194491139 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10877 (secure 0 0 0) 0x7fb45801b320 con 0x7fb46010d6b0 2024-05-08T23:57:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:57:29.848 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:57:11.803958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27940":{"gid":27940,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2536985900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2536985900},{"type":"v1","addr":"172.21.15.204:6837","nonce":2536985900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:57:29.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 -- 172.21.15.12:0/3194491139 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1310 msgr2=0x7fb4500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1310 0x7fb4500a37d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb454006010 tx=0x7fb45400b260 comp rx=0 tx=0).stop 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 -- 172.21.15.12:0/3194491139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb46010d6b0 msgr2=0x7fb46019ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb46010d6b0 0x7fb46019ddd0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb45800efc0 tx=0x7fb45800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 -- 172.21.15.12:0/3194491139 shutdown_connections 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4500a1310 0x7fb4500a37d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb46010d6b0 0x7fb46019ddd0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb460073350 0x7fb4601994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 --2- 172.21.15.12:0/3194491139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb460073c70 0x7fb460199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 -- 172.21.15.12:0/3194491139 >> 172.21.15.12:0/3194491139 conn(0x7fb460078360 msgr2=0x7fb4600fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 -- 172.21.15.12:0/3194491139 shutdown_connections 2024-05-08T23:57:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:29.856+0000 7fb466e6f700 1 -- 172.21.15.12:0/3194491139 wait complete. 2024-05-08T23:57:29.852 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 46 2024-05-08T23:57:29.863 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:29 smithi012 ceph-mon[42503]: pgmap v351: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-05-08T23:57:29.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:57:29.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 616129 6.33 MB/sec execute 445 sec latency 1745.938 ms 2024-05-08T23:57:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.008+0000 7f2186fb2700 1 -- 172.21.15.12:0/4250322614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2180100060 msgr2=0x7f2180100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.008+0000 7f2186fb2700 1 --2- 172.21.15.12:0/4250322614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2180100060 0x7f2180100440 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f217c009990 tx=0x7f217c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.008+0000 7f2186fb2700 1 -- 172.21.15.12:0/4250322614 shutdown_connections 2024-05-08T23:57:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.008+0000 7f2186fb2700 1 --2- 172.21.15.12:0/4250322614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2180105160 0x7f218010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.008+0000 7f2186fb2700 1 --2- 172.21.15.12:0/4250322614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2180100060 0x7f2180100440 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.008+0000 7f2186fb2700 1 --2- 172.21.15.12:0/4250322614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2180100a10 0x7f2180104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.008+0000 7f2186fb2700 1 -- 172.21.15.12:0/4250322614 >> 172.21.15.12:0/4250322614 conn(0x7f2180076f40 msgr2=0x7f2180077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.008+0000 7f2186fb2700 1 -- 172.21.15.12:0/4250322614 shutdown_connections 2024-05-08T23:57:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.008+0000 7f2186fb2700 1 -- 172.21.15.12:0/4250322614 wait complete. 2024-05-08T23:57:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f2186fb2700 1 Processor -- start 2024-05-08T23:57:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f2186fb2700 1 -- start start 2024-05-08T23:57:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f2186fb2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2180100060 0x7f21801974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f2186fb2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2180105160 0x7f21801979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f2186fb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218019bd40 0x7f218019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f2186fb2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218019c7c0 con 0x7f218019bd40 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f2186fb2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218019c930 con 0x7f2180105160 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f2186fb2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218019caf0 con 0x7f2180100060 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f218554f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218019bd40 0x7f218019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f218554f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218019bd40 0x7f218019c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44022/0 (socket says 172.21.15.12:44022) 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.009+0000 7f218554f700 1 -- 172.21.15.12:0/3973330106 learned_addr learned my addr 172.21.15.12:0/3973330106 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f2177fff700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2180105160 0x7f21801979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f2184d4e700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2180100060 0x7f21801974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f218554f700 1 -- 172.21.15.12:0/3973330106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2180100060 msgr2=0x7f21801974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f218554f700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2180100060 0x7f21801974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f218554f700 1 -- 172.21.15.12:0/3973330106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2180105160 msgr2=0x7f21801979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f218554f700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2180105160 0x7f21801979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f218554f700 1 -- 172.21.15.12:0/3973330106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0095f0 con 0x7f218019bd40 2024-05-08T23:57:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f218554f700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218019bd40 0x7f218019c1c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2178007ee0 tx=0x7f217800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f2175ffb700 1 -- 172.21.15.12:0/3973330106 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178017070 con 0x7f218019bd40 2024-05-08T23:57:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f2175ffb700 1 -- 172.21.15.12:0/3973330106 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f217800f040 con 0x7f218019bd40 2024-05-08T23:57:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f2186fb2700 1 -- 172.21.15.12:0/3973330106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21801a4050 con 0x7f218019bd40 2024-05-08T23:57:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f2175ffb700 1 -- 172.21.15.12:0/3973330106 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178005740 con 0x7f218019bd40 2024-05-08T23:57:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.010+0000 7f2186fb2700 1 -- 172.21.15.12:0/3973330106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21801a44e0 con 0x7f218019bd40 2024-05-08T23:57:30.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:29 smithi204 ceph-mon[47373]: pgmap v351: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-05-08T23:57:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.012+0000 7f2175ffb700 1 -- 172.21.15.12:0/3973330106 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f217800dc60 con 0x7f218019bd40 2024-05-08T23:57:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.013+0000 7f2175ffb700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21680a1290 0x7f21680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.013+0000 7f2175ffb700 1 -- 172.21.15.12:0/3973330106 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f21780cc450 con 0x7f218019bd40 2024-05-08T23:57:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.013+0000 7f2184d4e700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21680a1290 0x7f21680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.014+0000 7f2184d4e700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21680a1290 0x7f21680a3750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f217c00b2e0 tx=0x7f217c005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.014+0000 7f219316cb80 1 Processor -- start 2024-05-08T23:57:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.014+0000 7f219316cb80 1 -- start start 2024-05-08T23:57:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.014+0000 7f219316cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b09591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.014+0000 7f219316cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0953350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.014+0000 7f219316cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 0x5566b0953cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.014+0000 7f219316cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b08aa150 con 0x5566b0959720 2024-05-08T23:57:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.014+0000 7f219316cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b08a7b10 con 0x5566b0953890 2024-05-08T23:57:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.014+0000 7f219316cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b08a3400 con 0x5566b0958e00 2024-05-08T23:57:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2177fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0953350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2177fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0953350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44026/0 (socket says 172.21.15.12:44026) 2024-05-08T23:57:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2177fff700 1 -- 172.21.15.12:0/4011472184 learned_addr learned my addr 172.21.15.12:0/4011472184 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2184d4e700 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b09591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f218554f700 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 0x5566b0953cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2177fff700 1 -- 172.21.15.12:0/4011472184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 msgr2=0x5566b09591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2177fff700 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b09591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2177fff700 1 -- 172.21.15.12:0/4011472184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 msgr2=0x5566b0953cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2177fff700 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 0x5566b0953cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2177fff700 1 -- 172.21.15.12:0/4011472184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566b07243b0 con 0x5566b0959720 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f218554f700 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 0x5566b0953cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2184d4e700 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b09591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2177fff700 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0953350 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f2170009cf0 tx=0x7f2170011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60de6124cbff6da8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2166ffd700 1 -- 172.21.15.12:0/4011472184 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170011e30 con 0x5566b0959720 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2166ffd700 1 -- 172.21.15.12:0/4011472184 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2170004510 con 0x5566b0959720 2024-05-08T23:57:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.015+0000 7f2166ffd700 1 -- 172.21.15.12:0/4011472184 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170005280 con 0x5566b0959720 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.016+0000 7f219316cb80 1 -- 172.21.15.12:0/4011472184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 msgr2=0x5566b0953350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.016+0000 7f219316cb80 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0953350 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f2170009cf0 tx=0x7f2170011440 comp rx=0 tx=0).stop 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.016+0000 7f219316cb80 1 -- 172.21.15.12:0/4011472184 shutdown_connections 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.016+0000 7f219316cb80 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 0x5566b0953cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.016+0000 7f219316cb80 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b09591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.016+0000 7f219316cb80 1 --2- 172.21.15.12:0/4011472184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0953350 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.016+0000 7f219316cb80 1 -- 172.21.15.12:0/4011472184 >> 172.21.15.12:0/4011472184 conn(0x5566b08b1210 msgr2=0x5566b094ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.016+0000 7f219316cb80 1 -- 172.21.15.12:0/4011472184 shutdown_connections 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.016+0000 7f219316cb80 1 -- 172.21.15.12:0/4011472184 wait complete. 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.016+0000 7f219316cb80 1 -- start start 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f219316cb80 1 Processor -- start 2024-05-08T23:57:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f219316cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0953890 0x5566b095ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f219316cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0958e00 0x5566b095cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f219316cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b095d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f219316cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b083f560 con 0x5566b0959720 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f219316cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b083f200 con 0x5566b0958e00 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f219316cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b07266a0 con 0x5566b0953890 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f219316cb80 10 client.0 _ll_register_callbacks cb 0x5566b0848b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f218554f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b095d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f218554f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b095d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44036/0 (socket says 172.21.15.12:44036) 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f218554f700 1 -- 172.21.15.12:0/2093817050 learned_addr learned my addr 172.21.15.12:0/2093817050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f2184d4e700 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0953890 0x5566b095ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.017+0000 7f2177fff700 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0958e00 0x5566b095cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f218554f700 1 -- 172.21.15.12:0/2093817050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0953890 msgr2=0x5566b095ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f218554f700 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0953890 0x5566b095ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f218554f700 1 -- 172.21.15.12:0/2093817050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0958e00 msgr2=0x5566b095cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f218554f700 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0958e00 0x5566b095cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f218554f700 1 -- 172.21.15.12:0/2093817050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2170009920 con 0x5566b0959720 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f218554f700 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b095d4f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2178095fe0 tx=0x7f2178099130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b55e2ee1a15795 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f215ffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f215ffff700 1 -- 172.21.15.12:0/2093817050 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178099570 con 0x5566b0959720 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f215ffff700 1 -- 172.21.15.12:0/2093817050 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f217809d3a0 con 0x5566b0959720 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f219316cb80 1 -- 172.21.15.12:0/2093817050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566b07fc270 con 0x5566b0959720 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f215ffff700 1 -- 172.21.15.12:0/2093817050 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21780a19f0 con 0x5566b0959720 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f219316cb80 10 client.18531 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f219316cb80 1 -- 172.21.15.12:0/2093817050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5566b06d0b60 con 0x5566b0959720 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.018+0000 7f215ffff700 1 -- 172.21.15.12:0/2093817050 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5566b07fc270 con 0x5566b0959720 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.019+0000 7f215ffff700 1 -- 172.21.15.12:0/2093817050 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2178004f10 con 0x5566b0959720 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.019+0000 7f219316cb80 10 client.18531 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.019+0000 7f219316cb80 10 client.18531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:57:30.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.019+0000 7f219316cb80 20 client.18531 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.019+0000 7f219316cb80 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5566b096d110 0x5566b096f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.019+0000 7f219316cb80 4 client.18531 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.019+0000 7f219316cb80 1 -- 172.21.15.12:0/2093817050 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566b0922810 con 0x5566b096d110 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.019+0000 7f2184d4e700 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5566b096d110 0x5566b096f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.019+0000 7f215ffff700 10 client.18531 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.019+0000 7f2184d4e700 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5566b096d110 0x5566b096f530 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75a1e1f9576fbf87 server_cookie=de7b8dd5894ef690 in_seq=0 out_seq=0 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f215ffff700 1 -- 172.21.15.12:0/2093817050 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5566b0922810 con 0x5566b096d110 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f215ffff700 10 client.18531 handle_command_reply: tid=0 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 client.18531 shutdown 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 -- 172.21.15.12:0/2093817050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 msgr2=0x5566b095d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b095d4f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2178095fe0 tx=0x7f2178099130 comp rx=0 tx=0).stop 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 -- 172.21.15.12:0/2093817050 shutdown_connections 2024-05-08T23:57:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b095d4f0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5566b096d110 0x5566b096f530 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0953890 0x5566b095ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f215ffff700 0 client.18531 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:57:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 --2- 172.21.15.12:0/2093817050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0958e00 0x5566b095cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 -- 172.21.15.12:0/2093817050 >> 172.21.15.12:0/2093817050 conn(0x5566b08b1210 msgr2=0x5566b094d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 -- 172.21.15.12:0/2093817050 shutdown_connections 2024-05-08T23:57:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 1 -- 172.21.15.12:0/2093817050 wait complete. 2024-05-08T23:57:30.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.020+0000 7f219316cb80 20 client.18531 trim_cache size 0 max 16384 2024-05-08T23:57:30.023 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:29 smithi160 ceph-mon[46152]: pgmap v351: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-05-08T23:57:30.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.034+0000 7f219316cb80 1 Processor -- start 2024-05-08T23:57:30.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.034+0000 7f219316cb80 1 -- start start 2024-05-08T23:57:30.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f219316cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 0x5566b09591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f219316cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b094dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f219316cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0964160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f219316cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b096cf80 con 0x5566b0959720 2024-05-08T23:57:30.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f219316cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b095f630 con 0x5566b0953890 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f219316cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b096a890 con 0x5566b0958e00 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f218554f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0964160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f218554f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0964160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44044/0 (socket says 172.21.15.12:44044) 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f218554f700 1 -- 172.21.15.12:0/368279170 learned_addr learned my addr 172.21.15.12:0/368279170 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f2184d4e700 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 0x5566b09591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f2177fff700 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b094dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f218554f700 1 -- 172.21.15.12:0/368279170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 msgr2=0x5566b094dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f218554f700 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b094dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f218554f700 1 -- 172.21.15.12:0/368279170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 msgr2=0x5566b09591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f218554f700 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 0x5566b09591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f218554f700 1 -- 172.21.15.12:0/368279170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566b0869560 con 0x5566b0959720 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f2177fff700 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b094dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f2184d4e700 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 0x5566b09591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f218554f700 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0964160 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2178099270 tx=0x7f2178004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1eae545013b4839 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.035+0000 7f21677fe700 1 -- 172.21.15.12:0/368279170 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178098ed0 con 0x5566b0959720 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f21677fe700 1 -- 172.21.15.12:0/368279170 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2178005ab0 con 0x5566b0959720 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f21677fe700 1 -- 172.21.15.12:0/368279170 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21780b7440 con 0x5566b0959720 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f219316cb80 1 -- 172.21.15.12:0/368279170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 msgr2=0x5566b0964160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f219316cb80 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0964160 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2178099270 tx=0x7f2178004c40 comp rx=0 tx=0).stop 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f219316cb80 1 -- 172.21.15.12:0/368279170 shutdown_connections 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f219316cb80 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0959720 0x5566b0964160 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f219316cb80 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0953890 0x5566b09591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f219316cb80 1 --2- 172.21.15.12:0/368279170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b094dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f219316cb80 1 -- 172.21.15.12:0/368279170 >> 172.21.15.12:0/368279170 conn(0x5566b08b1210 msgr2=0x5566b0953c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f219316cb80 1 -- 172.21.15.12:0/368279170 shutdown_connections 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f219316cb80 1 -- 172.21.15.12:0/368279170 wait complete. 2024-05-08T23:57:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.036+0000 7f219316cb80 1 -- start start 2024-05-08T23:57:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f219316cb80 1 Processor -- start 2024-05-08T23:57:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f219316cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0953890 0x5566b0964590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f219316cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b09549a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f219316cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0959720 0x5566b0954f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f219316cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b0955650 con 0x5566b0953890 2024-05-08T23:57:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f219316cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b09557c0 con 0x5566b0959720 2024-05-08T23:57:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f219316cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b0955980 con 0x5566b0958e00 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f219316cb80 10 client.0 _ll_register_callbacks cb 0x5566b0848b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f2184d4e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0953890 0x5566b0964590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f2184d4e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0953890 0x5566b0964590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44056/0 (socket says 172.21.15.12:44056) 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f2184d4e700 1 -- 172.21.15.12:0/997676655 learned_addr learned my addr 172.21.15.12:0/997676655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f2177fff700 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b09549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.037+0000 7f218554f700 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0959720 0x5566b0954f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f2184d4e700 1 -- 172.21.15.12:0/997676655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 msgr2=0x5566b09549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f2184d4e700 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b09549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f2184d4e700 1 -- 172.21.15.12:0/997676655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0959720 msgr2=0x5566b0954f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f2184d4e700 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0959720 0x5566b0954f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f2184d4e700 1 -- 172.21.15.12:0/997676655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566b06d0b60 con 0x5566b0953890 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f2184d4e700 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0953890 0x5566b0964590 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f217c0098e0 tx=0x7f217c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b692daefec7ee1d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f215ffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f215ffff700 1 -- 172.21.15.12:0/997676655 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c034050 con 0x5566b0953890 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f215ffff700 1 -- 172.21.15.12:0/997676655 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f217c0341b0 con 0x5566b0953890 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f219316cb80 1 -- 172.21.15.12:0/997676655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2170004510 con 0x5566b0953890 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f215ffff700 1 -- 172.21.15.12:0/997676655 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f217c0343b0 con 0x5566b0953890 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f219316cb80 10 client.18537 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f219316cb80 1 -- 172.21.15.12:0/997676655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5566b0728cd0 con 0x5566b0953890 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f215ffff700 1 -- 172.21.15.12:0/997676655 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2170004510 con 0x5566b0953890 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.038+0000 7f215ffff700 1 -- 172.21.15.12:0/997676655 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f217c004c10 con 0x5566b0953890 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f219316cb80 10 client.18537 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f219316cb80 10 client.18537 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f219316cb80 20 client.18537 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f219316cb80 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5566b096d110 0x5566b09ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f219316cb80 4 client.18537 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f219316cb80 1 -- 172.21.15.12:0/997676655 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5566b089bec0 con 0x5566b096d110 2024-05-08T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f2177fff700 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5566b096d110 0x5566b09ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f215ffff700 10 client.18537 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f2177fff700 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5566b096d110 0x5566b09ca0b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1dc0966226d40d3 server_cookie=49b34e1a0ecd33b6 in_seq=0 out_seq=0 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f215ffff700 1 -- 172.21.15.12:0/997676655 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5566b089bec0 con 0x5566b096d110 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f215ffff700 10 client.18537 handle_command_reply: tid=0 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f219316cb80 1 client.18537 shutdown 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f219316cb80 1 -- 172.21.15.12:0/997676655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0953890 msgr2=0x5566b0964590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.039+0000 7f219316cb80 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0953890 0x5566b0964590 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f217c0098e0 tx=0x7f217c01a050 comp rx=0 tx=0).stop 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f219316cb80 1 -- 172.21.15.12:0/997676655 shutdown_connections 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f219316cb80 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5566b0959720 0x5566b0954f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f219316cb80 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566b0953890 0x5566b0964590 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f219316cb80 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5566b096d110 0x5566b09ca0b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f219316cb80 1 --2- 172.21.15.12:0/997676655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5566b0958e00 0x5566b09549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f215ffff700 0 client.18537 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:57:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f219316cb80 1 -- 172.21.15.12:0/997676655 >> 172.21.15.12:0/997676655 conn(0x5566b08b1210 msgr2=0x5566b095b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f219316cb80 1 -- 172.21.15.12:0/997676655 shutdown_connections 2024-05-08T23:57:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f219316cb80 1 -- 172.21.15.12:0/997676655 wait complete. 2024-05-08T23:57:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f219316cb80 20 client.18537 trim_cache size 0 max 16384 2024-05-08T23:57:30.042 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:57:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f2186fb2700 1 -- 172.21.15.12:0/3973330106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21680a1290 msgr2=0x7f21680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f2186fb2700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21680a1290 0x7f21680a3750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f217c00b2e0 tx=0x7f217c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f2186fb2700 1 -- 172.21.15.12:0/3973330106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218019bd40 msgr2=0x7f218019c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f2186fb2700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218019bd40 0x7f218019c1c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2178007ee0 tx=0x7f217800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f2186fb2700 1 -- 172.21.15.12:0/3973330106 shutdown_connections 2024-05-08T23:57:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f2186fb2700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21680a1290 0x7f21680a3750 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f2186fb2700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218019bd40 0x7f218019c1c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f2186fb2700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2180100060 0x7f21801974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f2186fb2700 1 --2- 172.21.15.12:0/3973330106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2180105160 0x7f21801979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.040+0000 7f2186fb2700 1 -- 172.21.15.12:0/3973330106 >> 172.21.15.12:0/3973330106 conn(0x7f2180076f40 msgr2=0x7f21800684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.041+0000 7f2186fb2700 1 -- 172.21.15.12:0/3973330106 shutdown_connections 2024-05-08T23:57:30.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.041+0000 7f2186fb2700 1 -- 172.21.15.12:0/3973330106 wait complete. 2024-05-08T23:57:30.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:57:30.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.236+0000 7fb295669700 1 -- 172.21.15.12:0/741461072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb290100060 msgr2=0x7fb290100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.236+0000 7fb295669700 1 --2- 172.21.15.12:0/741461072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb290100060 0x7fb290100440 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb280009990 tx=0x7fb280009ca0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.236+0000 7fb295669700 1 -- 172.21.15.12:0/741461072 shutdown_connections 2024-05-08T23:57:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.236+0000 7fb295669700 1 --2- 172.21.15.12:0/741461072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb290105180 0x7fb29010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.236+0000 7fb295669700 1 --2- 172.21.15.12:0/741461072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb290100060 0x7fb290100440 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.236+0000 7fb295669700 1 --2- 172.21.15.12:0/741461072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290100a10 0x7fb290104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.236+0000 7fb295669700 1 -- 172.21.15.12:0/741461072 >> 172.21.15.12:0/741461072 conn(0x7fb290076f40 msgr2=0x7fb290077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.236+0000 7fb295669700 1 -- 172.21.15.12:0/741461072 shutdown_connections 2024-05-08T23:57:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.236+0000 7fb295669700 1 -- 172.21.15.12:0/741461072 wait complete. 2024-05-08T23:57:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb295669700 1 Processor -- start 2024-05-08T23:57:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb295669700 1 -- start start 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb295669700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb290100060 0x7fb290199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb295669700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290105180 0x7fb290199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb295669700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29019df40 0x7fb29019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb295669700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29019e9c0 con 0x7fb29019df40 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb295669700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29019eb30 con 0x7fb290100060 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb295669700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29019ecf0 con 0x7fb290105180 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29019df40 0x7fb29019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29019df40 0x7fb29019e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44074/0 (socket says 172.21.15.12:44074) 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb28f7fe700 1 -- 172.21.15.12:0/3634453063 learned_addr learned my addr 172.21.15.12:0/3634453063 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.237+0000 7fb28effd700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb290100060 0x7fb290199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb28e7fc700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290105180 0x7fb290199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb28f7fe700 1 -- 172.21.15.12:0/3634453063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290105180 msgr2=0x7fb290199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290105180 0x7fb290199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb28f7fe700 1 -- 172.21.15.12:0/3634453063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb290100060 msgr2=0x7fb290199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb290100060 0x7fb290199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb28f7fe700 1 -- 172.21.15.12:0/3634453063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800095f0 con 0x7fb29019df40 2024-05-08T23:57:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29019df40 0x7fb29019e3c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb28800ef30 tx=0x7fb28800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb287fff700 1 -- 172.21.15.12:0/3634453063 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288018070 con 0x7fb29019df40 2024-05-08T23:57:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb287fff700 1 -- 172.21.15.12:0/3634453063 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28800f040 con 0x7fb29019df40 2024-05-08T23:57:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb295669700 1 -- 172.21.15.12:0/3634453063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2901a61f0 con 0x7fb29019df40 2024-05-08T23:57:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb287fff700 1 -- 172.21.15.12:0/3634453063 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28800cdb0 con 0x7fb29019df40 2024-05-08T23:57:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.238+0000 7fb295669700 1 -- 172.21.15.12:0/3634453063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2901a66d0 con 0x7fb29019df40 2024-05-08T23:57:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.240+0000 7fb287fff700 1 -- 172.21.15.12:0/3634453063 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb288003a00 con 0x7fb29019df40 2024-05-08T23:57:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.241+0000 7fb287fff700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb27c0a1290 0x7fb27c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.241+0000 7fb28effd700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb27c0a1290 0x7fb27c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.242+0000 7fb287fff700 1 -- 172.21.15.12:0/3634453063 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb2880cc8b0 con 0x7fb29019df40 2024-05-08T23:57:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.242+0000 7fb28effd700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb27c0a1290 0x7fb27c0a3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b2e0 tx=0x7fb280005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.242+0000 7fb2a1823b80 1 Processor -- start 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.242+0000 7fb2a1823b80 1 -- start start 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a07e00 0x55ce77a081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a02350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a02890 0x55ce77a02cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce77959150 con 0x55ce77a08720 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce77956b10 con 0x55ce77a07e00 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce77952400 con 0x55ce77a02890 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb28e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a02350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb28e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a02350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44086/0 (socket says 172.21.15.12:44086) 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb28e7fc700 1 -- 172.21.15.12:0/1928330143 learned_addr learned my addr 172.21.15.12:0/1928330143 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb28effd700 1 --2- 172.21.15.12:0/1928330143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a07e00 0x55ce77a081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.243+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/1928330143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a02890 0x55ce77a02cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.244+0000 7fb28e7fc700 1 -- 172.21.15.12:0/1928330143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a02890 msgr2=0x55ce77a02cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.244+0000 7fb28e7fc700 1 --2- 172.21.15.12:0/1928330143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a02890 0x55ce77a02cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.244+0000 7fb28e7fc700 1 -- 172.21.15.12:0/1928330143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a07e00 msgr2=0x55ce77a081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.244+0000 7fb28e7fc700 1 --2- 172.21.15.12:0/1928330143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a07e00 0x55ce77a081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.244+0000 7fb28e7fc700 1 -- 172.21.15.12:0/1928330143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce777d33b0 con 0x55ce77a08720 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.244+0000 7fb28e7fc700 1 --2- 172.21.15.12:0/1928330143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a02350 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb278009c80 tx=0x7fb278009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac3461ae955e829 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.244+0000 7fb2857fa700 1 -- 172.21.15.12:0/1928330143 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278009750 con 0x55ce77a08720 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.244+0000 7fb2857fa700 1 -- 172.21.15.12:0/1928330143 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2780044f0 con 0x55ce77a08720 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.244+0000 7fb2857fa700 1 -- 172.21.15.12:0/1928330143 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278003e70 con 0x55ce77a08720 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1928330143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 msgr2=0x55ce77a02350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/1928330143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a02350 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb278009c80 tx=0x7fb278009f90 comp rx=0 tx=0).stop 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1928330143 shutdown_connections 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/1928330143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a02890 0x55ce77a02cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/1928330143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a07e00 0x55ce77a081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/1928330143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a02350 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1928330143 >> 172.21.15.12:0/1928330143 conn(0x55ce77960210 msgr2=0x55ce779fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1928330143 shutdown_connections 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1928330143 wait complete. 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 -- start start 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.245+0000 7fb2a1823b80 1 Processor -- start 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 0x55ce77a0bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a07e00 0x55ce77a0c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a08720 0x55ce77a0c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce778ee560 con 0x55ce77a07e00 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce778ee200 con 0x55ce77a02890 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce777d56a0 con 0x55ce77a08720 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2a1823b80 10 client.0 _ll_register_callbacks cb 0x55ce778f7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a07e00 0x55ce77a0c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a07e00 0x55ce77a0c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44098/0 (socket says 172.21.15.12:44098) 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28e7fc700 1 -- 172.21.15.12:0/1478007668 learned_addr learned my addr 172.21.15.12:0/1478007668 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a08720 0x55ce77a0c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28effd700 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 0x55ce77a0bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28e7fc700 1 -- 172.21.15.12:0/1478007668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a08720 msgr2=0x55ce77a0c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28e7fc700 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a08720 0x55ce77a0c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28e7fc700 1 -- 172.21.15.12:0/1478007668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 msgr2=0x55ce77a0bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28e7fc700 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 0x55ce77a0bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28e7fc700 1 -- 172.21.15.12:0/1478007668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2780098b0 con 0x55ce77a07e00 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb28e7fc700 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a07e00 0x55ce77a0c0d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb278000c00 tx=0x7fb2780097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf19b7e36690c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2767fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2767fc700 1 -- 172.21.15.12:0/1478007668 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278003c00 con 0x55ce77a07e00 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2767fc700 1 -- 172.21.15.12:0/1478007668 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb278003d60 con 0x55ce77a07e00 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.246+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1478007668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce778ab270 con 0x55ce77a07e00 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2767fc700 1 -- 172.21.15.12:0/1478007668 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb278003f20 con 0x55ce77a07e00 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2a1823b80 10 client.18549 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1478007668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ce7777fb60 con 0x55ce77a07e00 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2767fc700 1 -- 172.21.15.12:0/1478007668 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce778ab270 con 0x55ce77a07e00 2024-05-08T23:57:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2767fc700 1 -- 172.21.15.12:0/1478007668 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb278012cb0 con 0x55ce77a07e00 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2a1823b80 10 client.18549 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2a1823b80 10 client.18549 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2a1823b80 20 client.18549 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ce77a1c1b0 0x55ce77a1e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2a1823b80 4 client.18549 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1478007668 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce779d1810 con 0x55ce77a1c1b0 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.247+0000 7fb28effd700 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ce77a1c1b0 0x55ce77a1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.248+0000 7fb2767fc700 10 client.18549 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.248+0000 7fb28effd700 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ce77a1c1b0 0x55ce77a1e5d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e5716dcf5e065b7 server_cookie=945b3d2912ae0c92 in_seq=0 out_seq=0 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.248+0000 7fb2767fc700 1 -- 172.21.15.12:0/1478007668 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ce779d1810 con 0x55ce77a1c1b0 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.248+0000 7fb2767fc700 10 client.18549 handle_command_reply: tid=0 2024-05-08T23:57:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 client.18549 shutdown 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1478007668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a07e00 msgr2=0x55ce77a0c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a07e00 0x55ce77a0c0d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb278000c00 tx=0x7fb2780097b0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1478007668 shutdown_connections 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ce77a1c1b0 0x55ce77a1e5d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a08720 0x55ce77a0c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 0x55ce77a0bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2767fc700 0 client.18549 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/1478007668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a07e00 0x55ce77a0c0d0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1478007668 >> 172.21.15.12:0/1478007668 conn(0x55ce77960210 msgr2=0x55ce779fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1478007668 shutdown_connections 2024-05-08T23:57:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 1 -- 172.21.15.12:0/1478007668 wait complete. 2024-05-08T23:57:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.249+0000 7fb2a1823b80 20 client.18549 trim_cache size 0 max 16384 2024-05-08T23:57:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb2a1823b80 1 Processor -- start 2024-05-08T23:57:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb2a1823b80 1 -- start start 2024-05-08T23:57:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 0x55ce77a18340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a07e00 0x55ce779fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a0c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce77a0d190 con 0x55ce77a08720 2024-05-08T23:57:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce779fe870 con 0x55ce77a02890 2024-05-08T23:57:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce779fd8e0 con 0x55ce77a07e00 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a0c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a0c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44100/0 (socket says 172.21.15.12:44100) 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.263+0000 7fb28f7fe700 1 -- 172.21.15.12:0/3596302942 learned_addr learned my addr 172.21.15.12:0/3596302942 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb28e7fc700 1 --2- 172.21.15.12:0/3596302942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a07e00 0x55ce779fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb28effd700 1 --2- 172.21.15.12:0/3596302942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 0x55ce77a18340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb28f7fe700 1 -- 172.21.15.12:0/3596302942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a07e00 msgr2=0x55ce779fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/3596302942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a07e00 0x55ce779fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb28f7fe700 1 -- 172.21.15.12:0/3596302942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 msgr2=0x55ce77a18340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/3596302942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 0x55ce77a18340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb28f7fe700 1 -- 172.21.15.12:0/3596302942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce77918560 con 0x55ce77a08720 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb28effd700 1 --2- 172.21.15.12:0/3596302942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 0x55ce77a18340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/3596302942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a0c430 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb288000f80 tx=0x7fb2880998f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f076ea245c59124 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb285ffb700 1 -- 172.21.15.12:0/3596302942 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288004ad0 con 0x55ce77a08720 2024-05-08T23:57:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb285ffb700 1 -- 172.21.15.12:0/3596302942 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28801b320 con 0x55ce77a08720 2024-05-08T23:57:30.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.264+0000 7fb285ffb700 1 -- 172.21.15.12:0/3596302942 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880a1df0 con 0x55ce77a08720 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.265+0000 7fb2a1823b80 1 -- 172.21.15.12:0/3596302942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 msgr2=0x55ce77a0c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.265+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/3596302942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a0c430 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb288000f80 tx=0x7fb2880998f0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.265+0000 7fb2a1823b80 1 -- 172.21.15.12:0/3596302942 shutdown_connections 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.265+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/3596302942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a0c430 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.265+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/3596302942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a02890 0x55ce77a18340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.265+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/3596302942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a07e00 0x55ce779fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.265+0000 7fb2a1823b80 1 -- 172.21.15.12:0/3596302942 >> 172.21.15.12:0/3596302942 conn(0x55ce77960210 msgr2=0x55ce779f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.265+0000 7fb2a1823b80 1 -- 172.21.15.12:0/3596302942 shutdown_connections 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.265+0000 7fb2a1823b80 1 -- 172.21.15.12:0/3596302942 wait complete. 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.265+0000 7fb2a1823b80 1 -- start start 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb2a1823b80 1 Processor -- start 2024-05-08T23:57:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a02890 0x55ce77a11150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a07e00 0x55ce77a11690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb2a1823b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce77a11f50 con 0x55ce77a08720 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce77a71bf0 con 0x55ce77a07e00 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb2a1823b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce77a71d30 con 0x55ce77a02890 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb2a1823b80 10 client.0 _ll_register_callbacks cb 0x55ce778f7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44110/0 (socket says 172.21.15.12:44110) 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28f7fe700 1 -- 172.21.15.12:0/2005835481 learned_addr learned my addr 172.21.15.12:0/2005835481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28e7fc700 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a07e00 0x55ce77a11690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28effd700 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a02890 0x55ce77a11150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28f7fe700 1 -- 172.21.15.12:0/2005835481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a02890 msgr2=0x55ce77a11150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a02890 0x55ce77a11150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28f7fe700 1 -- 172.21.15.12:0/2005835481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a07e00 msgr2=0x55ce77a11690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a07e00 0x55ce77a11690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28f7fe700 1 -- 172.21.15.12:0/2005835481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2901a66d0 con 0x55ce77a08720 2024-05-08T23:57:30.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb28f7fe700 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a716b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb288096470 tx=0x7fb28800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bdf20771c599d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.266+0000 7fb2767fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:30.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2767fc700 1 -- 172.21.15.12:0/2005835481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28809a100 con 0x55ce77a08720 2024-05-08T23:57:30.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2767fc700 1 -- 172.21.15.12:0/2005835481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb2880a1df0 con 0x55ce77a08720 2024-05-08T23:57:30.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2a1823b80 1 -- 172.21.15.12:0/2005835481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2780044f0 con 0x55ce77a08720 2024-05-08T23:57:30.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2767fc700 1 -- 172.21.15.12:0/2005835481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb288099a00 con 0x55ce77a08720 2024-05-08T23:57:30.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2a1823b80 10 client.18555 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:30.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2a1823b80 1 -- 172.21.15.12:0/2005835481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ce777d7cd0 con 0x55ce77a08720 2024-05-08T23:57:30.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2767fc700 1 -- 172.21.15.12:0/2005835481 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2780044f0 con 0x55ce77a08720 2024-05-08T23:57:30.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2767fc700 1 -- 172.21.15.12:0/2005835481 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb288098f80 con 0x55ce77a08720 2024-05-08T23:57:30.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2a1823b80 10 client.18555 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:30.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2a1823b80 10 client.18555 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:57:30.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2a1823b80 20 client.18555 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:30.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ce77a72f30 0x55ce77a75320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2a1823b80 4 client.18555 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.267+0000 7fb2a1823b80 1 -- 172.21.15.12:0/2005835481 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ce7794aec0 con 0x55ce77a72f30 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb28effd700 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ce77a72f30 0x55ce77a75320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb28effd700 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ce77a72f30 0x55ce77a75320 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7241ed0ac35ce3a0 server_cookie=6959915f494956c4 in_seq=0 out_seq=0 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2767fc700 10 client.18555 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2767fc700 1 -- 172.21.15.12:0/2005835481 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ce7794aec0 con 0x55ce77a72f30 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2767fc700 10 client.18555 handle_command_reply: tid=0 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 client.18555 shutdown 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 -- 172.21.15.12:0/2005835481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 msgr2=0x55ce77a716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a716b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb288096470 tx=0x7fb28800cf90 comp rx=0 tx=0).stop 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 -- 172.21.15.12:0/2005835481 shutdown_connections 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ce77a72f30 0x55ce77a75320 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce77a08720 0x55ce77a716b0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce77a02890 0x55ce77a11150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2767fc700 0 client.18555 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:57:30.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 --2- 172.21.15.12:0/2005835481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce77a07e00 0x55ce77a11690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 -- 172.21.15.12:0/2005835481 >> 172.21.15.12:0/2005835481 conn(0x55ce77960210 msgr2=0x55ce77a0a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 -- 172.21.15.12:0/2005835481 shutdown_connections 2024-05-08T23:57:30.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.268+0000 7fb2a1823b80 1 -- 172.21.15.12:0/2005835481 wait complete. 2024-05-08T23:57:30.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb2a1823b80 20 client.18555 trim_cache size 0 max 16384 2024-05-08T23:57:30.270 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 -- 172.21.15.12:0/3634453063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb27c0a1290 msgr2=0x7fb27c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb27c0a1290 0x7fb27c0a3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b2e0 tx=0x7fb280005960 comp rx=0 tx=0).stop 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 -- 172.21.15.12:0/3634453063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29019df40 msgr2=0x7fb29019e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29019df40 0x7fb29019e3c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb28800ef30 tx=0x7fb28800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 -- 172.21.15.12:0/3634453063 shutdown_connections 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb27c0a1290 0x7fb27c0a3750 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29019df40 0x7fb29019e3c0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb290100060 0x7fb290199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 --2- 172.21.15.12:0/3634453063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290105180 0x7fb290199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 -- 172.21.15.12:0/3634453063 >> 172.21.15.12:0/3634453063 conn(0x7fb290076f40 msgr2=0x7fb2900fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 -- 172.21.15.12:0/3634453063 shutdown_connections 2024-05-08T23:57:30.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.269+0000 7fb295669700 1 -- 172.21.15.12:0/3634453063 wait complete. 2024-05-08T23:57:30.277 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:57:30.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.422+0000 7f04d0395700 1 -- 172.21.15.12:0/4106809970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04c8100ab0 msgr2=0x7f04c810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.422+0000 7f04d0395700 1 --2- 172.21.15.12:0/4106809970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04c8100ab0 0x7f04c810ccc0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f04b80099c0 tx=0x7f04b8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.422+0000 7f04d0395700 1 -- 172.21.15.12:0/4106809970 shutdown_connections 2024-05-08T23:57:30.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.422+0000 7f04d0395700 1 --2- 172.21.15.12:0/4106809970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04c8100ab0 0x7f04c810ccc0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.422+0000 7f04d0395700 1 --2- 172.21.15.12:0/4106809970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04c8105f90 0x7f04c8106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.422+0000 7f04d0395700 1 --2- 172.21.15.12:0/4106809970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c81000f0 0x7f04c8100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.422+0000 7f04d0395700 1 -- 172.21.15.12:0/4106809970 >> 172.21.15.12:0/4106809970 conn(0x7f04c8076ba0 msgr2=0x7f04c8076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.422+0000 7f04d0395700 1 -- 172.21.15.12:0/4106809970 shutdown_connections 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.422+0000 7f04d0395700 1 -- 172.21.15.12:0/4106809970 wait complete. 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04d0395700 1 Processor -- start 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04d0395700 1 -- start start 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04d0395700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c81000f0 0x7f04c8101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04d0395700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04c8100ab0 0x7f04c8102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04d0395700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04c8105f90 0x7f04c81025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04d0395700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81a5760 con 0x7f04c8100ab0 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04d0395700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81a58d0 con 0x7f04c8105f90 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04d0395700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81a5a90 con 0x7f04c81000f0 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04cd930700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04c8100ab0 0x7f04c8102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04ce932700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04c8105f90 0x7f04c81025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04ce932700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04c8105f90 0x7f04c81025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44374/0 (socket says 172.21.15.12:44374) 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04ce932700 1 -- 172.21.15.12:0/615012447 learned_addr learned my addr 172.21.15.12:0/615012447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04cd930700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04c8100ab0 0x7f04c8102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44138/0 (socket says 172.21.15.12:44138) 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.423+0000 7f04ce131700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c81000f0 0x7f04c8101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04ce932700 1 -- 172.21.15.12:0/615012447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c81000f0 msgr2=0x7f04c8101b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04ce932700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c81000f0 0x7f04c8101b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04ce932700 1 -- 172.21.15.12:0/615012447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04c8100ab0 msgr2=0x7f04c8102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04ce932700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04c8100ab0 0x7f04c8102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04ce932700 1 -- 172.21.15.12:0/615012447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b80095f0 con 0x7f04c8105f90 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04cd930700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04c8100ab0 0x7f04c8102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04ce932700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04c8105f90 0x7f04c81025a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04b8004730 tx=0x7f04b8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04c77fe700 1 -- 172.21.15.12:0/615012447 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b801d070 con 0x7f04c8105f90 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04d0395700 1 -- 172.21.15.12:0/615012447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c81a5d10 con 0x7f04c8105f90 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04c77fe700 1 -- 172.21.15.12:0/615012447 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b8004c90 con 0x7f04c8105f90 2024-05-08T23:57:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04c77fe700 1 -- 172.21.15.12:0/615012447 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b800f690 con 0x7f04c8105f90 2024-05-08T23:57:30.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.424+0000 7f04d0395700 1 -- 172.21.15.12:0/615012447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c81a61c0 con 0x7f04c8105f90 2024-05-08T23:57:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.427+0000 7f04c77fe700 1 -- 172.21.15.12:0/615012447 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f04b8005000 con 0x7f04c8105f90 2024-05-08T23:57:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.427+0000 7f04c77fe700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04b00a1290 0x7f04b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.428+0000 7f04c77fe700 1 -- 172.21.15.12:0/615012447 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04b80cd9b0 con 0x7f04c8105f90 2024-05-08T23:57:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.428+0000 7f04ce131700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04b00a1290 0x7f04b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.428+0000 7f04ce131700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04b00a1290 0x7f04b00a3750 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009550 tx=0x7f04c00091b0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.428+0000 7f04dc54fb80 1 Processor -- start 2024-05-08T23:57:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04dc54fb80 1 -- start start 2024-05-08T23:57:30.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb09f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 0x55cbeb099350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 0x55cbeb099cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeaff0150 con 0x55cbeb09ee00 2024-05-08T23:57:30.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeafedb10 con 0x55cbeb099890 2024-05-08T23:57:30.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeafe9400 con 0x55cbeb09f720 2024-05-08T23:57:30.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04ce131700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb09f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04ce131700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb09f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44144/0 (socket says 172.21.15.12:44144) 2024-05-08T23:57:30.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04ce131700 1 -- 172.21.15.12:0/1292758355 learned_addr learned my addr 172.21.15.12:0/1292758355 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04cd930700 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 0x55cbeb099350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04ce932700 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 0x55cbeb099cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.429+0000 7f04ce131700 1 -- 172.21.15.12:0/1292758355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 msgr2=0x55cbeb099350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.430+0000 7f04ce131700 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 0x55cbeb099350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.430+0000 7f04ce131700 1 -- 172.21.15.12:0/1292758355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 msgr2=0x55cbeb099cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.430+0000 7f04ce131700 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 0x55cbeb099cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.430+0000 7f04ce131700 1 -- 172.21.15.12:0/1292758355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbeae6a3b0 con 0x55cbeb09ee00 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.430+0000 7f04ce932700 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 0x55cbeb099cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.430+0000 7f04cd930700 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 0x55cbeb099350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.430+0000 7f04ce131700 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb09f1e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f04c00154f0 tx=0x7f04c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f164580d2013e480 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.430+0000 7f04c4da4700 1 -- 172.21.15.12:0/1292758355 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0021640 con 0x55cbeb09ee00 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.430+0000 7f04c4da4700 1 -- 172.21.15.12:0/1292758355 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c001a070 con 0x55cbeb09ee00 2024-05-08T23:57:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.430+0000 7f04c4da4700 1 -- 172.21.15.12:0/1292758355 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0016a90 con 0x55cbeb09ee00 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1292758355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 msgr2=0x55cbeb09f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb09f1e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f04c00154f0 tx=0x7f04c0013000 comp rx=0 tx=0).stop 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1292758355 shutdown_connections 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 0x55cbeb099cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb09f1e0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/1292758355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 0x55cbeb099350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1292758355 >> 172.21.15.12:0/1292758355 conn(0x55cbeaff7210 msgr2=0x55cbeb093df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1292758355 shutdown_connections 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1292758355 wait complete. 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 -- start start 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.431+0000 7f04dc54fb80 1 Processor -- start 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 0x55cbeb0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb0a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 0x55cbeb0a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeaf85560 con 0x55cbeb09ee00 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeaf85200 con 0x55cbeb099890 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeae6c6a0 con 0x55cbeb09f720 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04dc54fb80 10 client.0 _ll_register_callbacks cb 0x55cbeaf8eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04cd930700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb0a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04cd930700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb0a2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44148/0 (socket says 172.21.15.12:44148) 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04cd930700 1 -- 172.21.15.12:0/2270889666 learned_addr learned my addr 172.21.15.12:0/2270889666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04ce131700 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 0x55cbeb0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04ce932700 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 0x55cbeb0a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04cd930700 1 -- 172.21.15.12:0/2270889666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 msgr2=0x55cbeb0a34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04cd930700 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 0x55cbeb0a34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04cd930700 1 -- 172.21.15.12:0/2270889666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 msgr2=0x55cbeb0a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04cd930700 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 0x55cbeb0a2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.432+0000 7f04cd930700 1 -- 172.21.15.12:0/2270889666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c0007d90 con 0x55cbeb09ee00 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04cd930700 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb0a2fb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0116e0 tx=0x7f04bc0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99f79b923669c40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04adffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04adffb700 1 -- 172.21.15.12:0/2270889666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc012920 con 0x55cbeb09ee00 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04adffb700 1 -- 172.21.15.12:0/2270889666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04bc018070 con 0x55cbeb09ee00 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04dc54fb80 1 -- 172.21.15.12:0/2270889666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbeaf42270 con 0x55cbeb09ee00 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04adffb700 1 -- 172.21.15.12:0/2270889666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04bc012a80 con 0x55cbeb09ee00 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04dc54fb80 10 client.18567 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04dc54fb80 1 -- 172.21.15.12:0/2270889666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55cbeae16b60 con 0x55cbeb09ee00 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04adffb700 1 -- 172.21.15.12:0/2270889666 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cbeaf42270 con 0x55cbeb09ee00 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04adffb700 1 -- 172.21.15.12:0/2270889666 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f04bc026470 con 0x55cbeb09ee00 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04dc54fb80 10 client.18567 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04dc54fb80 10 client.18567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04dc54fb80 20 client.18567 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbeb0b3180 0x55cbeb0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04dc54fb80 4 client.18567 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.433+0000 7f04dc54fb80 1 -- 172.21.15.12:0/2270889666 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbeb068810 con 0x55cbeb0b3180 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.434+0000 7f04ce131700 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbeb0b3180 0x55cbeb0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.434+0000 7f04ce131700 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbeb0b3180 0x55cbeb0b55a0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ae7001bdbec40a server_cookie=f1c46a11e511a955 in_seq=0 out_seq=0 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.434+0000 7f04adffb700 10 client.18567 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04adffb700 1 -- 172.21.15.12:0/2270889666 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbeb068810 con 0x55cbeb0b3180 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04adffb700 10 client.18567 handle_command_reply: tid=0 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 client.18567 shutdown 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 -- 172.21.15.12:0/2270889666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 msgr2=0x55cbeb0a2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb0a2fb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0116e0 tx=0x7f04bc0119f0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 -- 172.21.15.12:0/2270889666 shutdown_connections 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09f720 0x55cbeb0a34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbeb0b3180 0x55cbeb0b55a0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb099890 0x55cbeb0a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/2270889666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09ee00 0x55cbeb0a2fb0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 -- 172.21.15.12:0/2270889666 >> 172.21.15.12:0/2270889666 conn(0x55cbeaff7210 msgr2=0x55cbeb0935a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04adffb700 0 client.18567 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:30.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 -- 172.21.15.12:0/2270889666 shutdown_connections 2024-05-08T23:57:30.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 1 -- 172.21.15.12:0/2270889666 wait complete. 2024-05-08T23:57:30.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.435+0000 7f04dc54fb80 20 client.18567 trim_cache size 0 max 16384 2024-05-08T23:57:30.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04dc54fb80 1 Processor -- start 2024-05-08T23:57:30.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04dc54fb80 1 -- start start 2024-05-08T23:57:30.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb099890 0x55cbeb0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09ee00 0x55cbeb093d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09f720 0x55cbeb0aa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeb0a4070 con 0x55cbeb09f720 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeb0b0850 con 0x55cbeb09ee00 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeb0afd00 con 0x55cbeb099890 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04ce932700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09f720 0x55cbeb0aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04ce932700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09f720 0x55cbeb0aa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44154/0 (socket says 172.21.15.12:44154) 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04ce932700 1 -- 172.21.15.12:0/4213844080 learned_addr learned my addr 172.21.15.12:0/4213844080 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04ce131700 1 --2- 172.21.15.12:0/4213844080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb099890 0x55cbeb0aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.449+0000 7f04cd930700 1 --2- 172.21.15.12:0/4213844080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09ee00 0x55cbeb093d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04ce932700 1 -- 172.21.15.12:0/4213844080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb099890 msgr2=0x55cbeb0aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04ce932700 1 --2- 172.21.15.12:0/4213844080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb099890 0x55cbeb0aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04ce932700 1 -- 172.21.15.12:0/4213844080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09ee00 msgr2=0x55cbeb093d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04ce932700 1 --2- 172.21.15.12:0/4213844080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09ee00 0x55cbeb093d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04ce932700 1 -- 172.21.15.12:0/4213844080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbeafaf560 con 0x55cbeb09f720 2024-05-08T23:57:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04cd930700 1 --2- 172.21.15.12:0/4213844080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09ee00 0x55cbeb093d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04ce932700 1 --2- 172.21.15.12:0/4213844080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09f720 0x55cbeb0aa190 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c010 tx=0x7f04b80cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7afd80c94a227c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04c55a5700 1 -- 172.21.15.12:0/4213844080 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b809ac80 con 0x55cbeb09f720 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04c55a5700 1 -- 172.21.15.12:0/4213844080 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b809eab0 con 0x55cbeb09f720 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04c55a5700 1 -- 172.21.15.12:0/4213844080 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b809a3d0 con 0x55cbeb09f720 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04dc54fb80 1 -- 172.21.15.12:0/4213844080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09f720 msgr2=0x55cbeb0aa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.450+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/4213844080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09f720 0x55cbeb0aa190 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c010 tx=0x7f04b80cd040 comp rx=0 tx=0).stop 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 -- 172.21.15.12:0/4213844080 shutdown_connections 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/4213844080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb09f720 0x55cbeb0aa190 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/4213844080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb099890 0x55cbeb0aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/4213844080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09ee00 0x55cbeb093d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 -- 172.21.15.12:0/4213844080 >> 172.21.15.12:0/4213844080 conn(0x55cbeaff7210 msgr2=0x55cbeb090210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 -- 172.21.15.12:0/4213844080 shutdown_connections 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 -- 172.21.15.12:0/4213844080 wait complete. 2024-05-08T23:57:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 -- start start 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 Processor -- start 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb099890 0x55cbeb0aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09ee00 0x55cbeb09ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09f720 0x55cbeb09c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeb09a930 con 0x55cbeb099890 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeb09aaa0 con 0x55cbeb09f720 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbeb09ac60 con 0x55cbeb09ee00 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04dc54fb80 10 client.0 _ll_register_callbacks cb 0x55cbeaf8eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04ce131700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb099890 0x55cbeb0aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04ce131700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb099890 0x55cbeb0aa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44160/0 (socket says 172.21.15.12:44160) 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.451+0000 7f04ce131700 1 -- 172.21.15.12:0/1461941378 learned_addr learned my addr 172.21.15.12:0/1461941378 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04ce932700 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09f720 0x55cbeb09c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04ce131700 1 -- 172.21.15.12:0/1461941378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09ee00 msgr2=0x55cbeb09ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04cd930700 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09ee00 0x55cbeb09ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04ce131700 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09ee00 0x55cbeb09ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04ce131700 1 -- 172.21.15.12:0/1461941378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09f720 msgr2=0x55cbeb09c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04ce131700 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09f720 0x55cbeb09c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04ce131700 1 -- 172.21.15.12:0/1461941378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c81a61c0 con 0x55cbeb099890 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04ce932700 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09f720 0x55cbeb09c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04ce131700 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb099890 0x55cbeb0aa6c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f04c0016460 tx=0x7f04c0016610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964d9c86a98ab9be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04adffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04adffb700 1 -- 172.21.15.12:0/1461941378 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0003c00 con 0x55cbeb099890 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1461941378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04c001a070 con 0x55cbeb099890 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04adffb700 1 -- 172.21.15.12:0/1461941378 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04c0033e40 con 0x55cbeb099890 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04adffb700 1 -- 172.21.15.12:0/1461941378 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04c0003d60 con 0x55cbeb099890 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04dc54fb80 10 client.18573 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.452+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1461941378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55cbeae6ecd0 con 0x55cbeb099890 2024-05-08T23:57:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04adffb700 1 -- 172.21.15.12:0/1461941378 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c001a070 con 0x55cbeb099890 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04adffb700 1 -- 172.21.15.12:0/1461941378 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f04c0016c10 con 0x55cbeb099890 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04dc54fb80 10 client.18573 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04dc54fb80 10 client.18573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04dc54fb80 20 client.18573 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbeb109eb0 0x55cbeb10c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04dc54fb80 4 client.18573 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1461941378 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cbeafe1ec0 con 0x55cbeb109eb0 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04cd930700 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbeb109eb0 0x55cbeb10c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04adffb700 10 client.18573 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.453+0000 7f04cd930700 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbeb109eb0 0x55cbeb10c2d0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95ae894094b21339 server_cookie=844074c3b6641e36 in_seq=0 out_seq=0 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04adffb700 1 -- 172.21.15.12:0/1461941378 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cbeafe1ec0 con 0x55cbeb109eb0 2024-05-08T23:57:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04adffb700 10 client.18573 handle_command_reply: tid=0 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 client.18573 shutdown 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1461941378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb099890 msgr2=0x55cbeb0aa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb099890 0x55cbeb0aa6c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f04c0016460 tx=0x7f04c0016610 comp rx=0 tx=0).stop 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1461941378 shutdown_connections 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbeb109eb0 0x55cbeb10c2d0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbeb09f720 0x55cbeb09c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04adffb700 0 client.18573 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbeb099890 0x55cbeb0aa6c0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 --2- 172.21.15.12:0/1461941378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbeb09ee00 0x55cbeb09ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1461941378 >> 172.21.15.12:0/1461941378 conn(0x55cbeaff7210 msgr2=0x55cbeb0a12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1461941378 shutdown_connections 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 1 -- 172.21.15.12:0/1461941378 wait complete. 2024-05-08T23:57:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04dc54fb80 20 client.18573 trim_cache size 0 max 16384 2024-05-08T23:57:30.456 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04d0395700 1 -- 172.21.15.12:0/615012447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04b00a1290 msgr2=0x7f04b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04d0395700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04b00a1290 0x7f04b00a3750 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009550 tx=0x7f04c00091b0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.454+0000 7f04d0395700 1 -- 172.21.15.12:0/615012447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04c8105f90 msgr2=0x7f04c81025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.455+0000 7f04d0395700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04c8105f90 0x7f04c81025a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04b8004730 tx=0x7f04b8009580 comp rx=0 tx=0).stop 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.455+0000 7f04d0395700 1 -- 172.21.15.12:0/615012447 shutdown_connections 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.455+0000 7f04d0395700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04b00a1290 0x7f04b00a3750 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.455+0000 7f04d0395700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04c8105f90 0x7f04c81025a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.455+0000 7f04d0395700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c81000f0 0x7f04c8101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.455+0000 7f04d0395700 1 --2- 172.21.15.12:0/615012447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04c8100ab0 0x7f04c8102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.455+0000 7f04d0395700 1 -- 172.21.15.12:0/615012447 >> 172.21.15.12:0/615012447 conn(0x7f04c8076ba0 msgr2=0x7f04c80692e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.455+0000 7f04d0395700 1 -- 172.21.15.12:0/615012447 shutdown_connections 2024-05-08T23:57:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:30.455+0000 7f04d0395700 1 -- 172.21.15.12:0/615012447 wait complete. 2024-05-08T23:57:30.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 616129 6.32 MB/sec execute 446 sec latency 2746.046 ms 2024-05-08T23:57:30.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:30 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3194491139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:57:31.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:30 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3194491139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:57:31.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:30 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3194491139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:57:31.463 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:57:31.464 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.481+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/689855179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d0105e60 msgr2=0x7fc8d0106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.481+0000 7fc8d6e0b700 1 --2- 172.21.15.12:0/689855179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d0105e60 0x7fc8d0106240 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009990 tx=0x7fc8cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.481+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/689855179 shutdown_connections 2024-05-08T23:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.481+0000 7fc8d6e0b700 1 --2- 172.21.15.12:0/689855179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d0100760 0x7fc8d010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.481+0000 7fc8d6e0b700 1 --2- 172.21.15.12:0/689855179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d0105e60 0x7fc8d0106240 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.481+0000 7fc8d6e0b700 1 --2- 172.21.15.12:0/689855179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d00ffda0 0x7fc8d0100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.481+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/689855179 >> 172.21.15.12:0/689855179 conn(0x7fc8d0076f10 msgr2=0x7fc8d0077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.481+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/689855179 shutdown_connections 2024-05-08T23:57:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.481+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/689855179 wait complete. 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.482+0000 7fc8d6e0b700 1 Processor -- start 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.482+0000 7fc8d6e0b700 1 -- start start 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.482+0000 7fc8d6e0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d00ffda0 0x7fc8d019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.482+0000 7fc8d6e0b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d0100760 0x7fc8d019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d6e0b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d0105e60 0x7fc8d01953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d6e0b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d0195bc0 con 0x7fc8d00ffda0 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d6e0b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d0195d30 con 0x7fc8d0100760 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d6e0b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d0195ef0 con 0x7fc8d0105e60 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d4ba7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d00ffda0 0x7fc8d019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d4ba7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d00ffda0 0x7fc8d019d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44186/0 (socket says 172.21.15.12:44186) 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/2711360007 learned_addr learned my addr 172.21.15.12:0/2711360007 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8c7fff700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d0100760 0x7fc8d019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d53a8700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d0105e60 0x7fc8d01953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/2711360007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d0105e60 msgr2=0x7fc8d01953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d0105e60 0x7fc8d01953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/2711360007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d0100760 msgr2=0x7fc8d019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d0100760 0x7fc8d019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/2711360007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c0009520 con 0x7fc8d00ffda0 2024-05-08T23:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.483+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d00ffda0 0x7fc8d019d040 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc000c00 tx=0x7fc8cc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.484+0000 7fc8c5ffb700 1 -- 172.21.15.12:0/2711360007 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc01d070 con 0x7fc8d00ffda0 2024-05-08T23:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.484+0000 7fc8c5ffb700 1 -- 172.21.15.12:0/2711360007 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8cc022470 con 0x7fc8d00ffda0 2024-05-08T23:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.484+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/2711360007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c8009c40 con 0x7fc8d00ffda0 2024-05-08T23:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.484+0000 7fc8c5ffb700 1 -- 172.21.15.12:0/2711360007 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc017650 con 0x7fc8d00ffda0 2024-05-08T23:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.484+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/2711360007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc0095f0 con 0x7fc8d00ffda0 2024-05-08T23:57:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.488+0000 7fc8c5ffb700 1 -- 172.21.15.12:0/2711360007 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc8cc0177b0 con 0x7fc8d00ffda0 2024-05-08T23:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.488+0000 7fc8c5ffb700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8b80a13e0 0x7fc8b80a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.489+0000 7fc8c5ffb700 1 -- 172.21.15.12:0/2711360007 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc8cc0cde10 con 0x7fc8d00ffda0 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.489+0000 7fc8e2fc5b80 1 Processor -- start 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.489+0000 7fc8e2fc5b80 1 -- start start 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.489+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e16f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.489+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e16fa60 0x55743e173500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.489+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e173a40 0x55743e173ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.489+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e0b50f0 con 0x55743e173a40 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.489+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e0b2ab0 con 0x55743e16f140 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.489+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e0ae3a0 con 0x55743e16fa60 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.490+0000 7fc8d4ba7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e16f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.490+0000 7fc8d4ba7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e16f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44452/0 (socket says 172.21.15.12:44452) 2024-05-08T23:57:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.490+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/3032492190 learned_addr learned my addr 172.21.15.12:0/3032492190 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.490+0000 7fc8d53a8700 1 --2- 172.21.15.12:0/3032492190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e173a40 0x55743e173ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.491+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/3032492190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e16fa60 msgr2=0x55743e173500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:57:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.491+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/3032492190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e16fa60 0x55743e173500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.491+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/3032492190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e173a40 msgr2=0x55743e173ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.491+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/3032492190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e173a40 0x55743e173ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.491+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/3032492190 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55743df2f350 con 0x55743e16f140 2024-05-08T23:57:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.492+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/3032492190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e16f520 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0974c0 tx=0x7fc8cc096c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa3d7d55ccef5e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.492+0000 7fc8b6ffd700 1 -- 172.21.15.12:0/3032492190 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc09a920 con 0x55743e16f140 2024-05-08T23:57:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.492+0000 7fc8b6ffd700 1 -- 172.21.15.12:0/3032492190 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8cc0a10c0 con 0x55743e16f140 2024-05-08T23:57:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.492+0000 7fc8b6ffd700 1 -- 172.21.15.12:0/3032492190 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0225e0 con 0x55743e16f140 2024-05-08T23:57:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.492+0000 7fc8c7fff700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8b80a13e0 0x7fc8b80a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.493+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/3032492190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 msgr2=0x55743e16f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.493+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/3032492190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e16f520 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0974c0 tx=0x7fc8cc096c60 comp rx=0 tx=0).stop 2024-05-08T23:57:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.493+0000 7fc8c7fff700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8b80a13e0 0x7fc8b80a38a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc8d01967c0 tx=0x7fc8c0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.493+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/3032492190 shutdown_connections 2024-05-08T23:57:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.493+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/3032492190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e173a40 0x55743e173ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.493+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/3032492190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e16f520 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.493+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/3032492190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e16fa60 0x55743e173500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.493+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/3032492190 >> 172.21.15.12:0/3032492190 conn(0x55743e0bc1b0 msgr2=0x55743e158840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.494+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/3032492190 shutdown_connections 2024-05-08T23:57:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.494+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/3032492190 wait complete. 2024-05-08T23:57:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.494+0000 7fc8e2fc5b80 1 -- start start 2024-05-08T23:57:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.494+0000 7fc8e2fc5b80 1 Processor -- start 2024-05-08T23:57:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e1786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16fa60 0x55743e178c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e173a40 0x55743e179150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e04a500 con 0x55743e16f140 2024-05-08T23:57:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e04a1a0 con 0x55743e16fa60 2024-05-08T23:57:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743df31640 con 0x55743e173a40 2024-05-08T23:57:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8e2fc5b80 10 client.0 _ll_register_callbacks cb 0x55743e053ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8c7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16fa60 0x55743e178c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8d4ba7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e1786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8c7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16fa60 0x55743e178c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44462/0 (socket says 172.21.15.12:44462) 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8d4ba7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e1786d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44202/0 (socket says 172.21.15.12:44202) 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8c7fff700 1 -- 172.21.15.12:0/2125224265 learned_addr learned my addr 172.21.15.12:0/2125224265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8d53a8700 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e173a40 0x55743e179150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8c7fff700 1 -- 172.21.15.12:0/2125224265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e173a40 msgr2=0x55743e179150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8c7fff700 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e173a40 0x55743e179150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8c7fff700 1 -- 172.21.15.12:0/2125224265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 msgr2=0x55743e1786d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.495+0000 7fc8c7fff700 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e1786d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.496+0000 7fc8c7fff700 1 -- 172.21.15.12:0/2125224265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc0095f0 con 0x55743e16fa60 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.496+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e1786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.496+0000 7fc8c7fff700 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16fa60 0x55743e178c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000ebb0 tx=0x7fc8c00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71651754022a76d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.496+0000 7fc8affff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.496+0000 7fc8affff700 1 -- 172.21.15.12:0/2125224265 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c001cc40 con 0x55743e16fa60 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.496+0000 7fc8affff700 1 -- 172.21.15.12:0/2125224265 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc8c0023070 con 0x55743e16fa60 2024-05-08T23:57:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.496+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2125224265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55743e007210 con 0x55743e16fa60 2024-05-08T23:57:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.496+0000 7fc8affff700 1 -- 172.21.15.12:0/2125224265 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8c001cda0 con 0x55743e16fa60 2024-05-08T23:57:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.496+0000 7fc8e2fc5b80 10 client.28319 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.496+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2125224265 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55743dedbb00 con 0x55743e16fa60 2024-05-08T23:57:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.497+0000 7fc8affff700 1 -- 172.21.15.12:0/2125224265 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55743e007210 con 0x55743e16fa60 2024-05-08T23:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.497+0000 7fc8affff700 1 -- 172.21.15.12:0/2125224265 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc8c001d480 con 0x55743e16fa60 2024-05-08T23:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.497+0000 7fc8e2fc5b80 10 client.28319 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.497+0000 7fc8e2fc5b80 10 client.28319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.497+0000 7fc8e2fc5b80 20 client.28319 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.497+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55743e1888f0 0x55743e18ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.497+0000 7fc8e2fc5b80 4 client.28319 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.497+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2125224265 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55743e12d7b0 con 0x55743e1888f0 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.497+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55743e1888f0 0x55743e18ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.498+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55743e1888f0 0x55743e18ad10 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383d88cebecc35d3 server_cookie=b8c22a59c3cf16b9 in_seq=0 out_seq=0 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.498+0000 7fc8affff700 10 client.28319 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.499+0000 7fc8affff700 1 -- 172.21.15.12:0/2125224265 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55743e12d7b0 con 0x55743e1888f0 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.499+0000 7fc8affff700 10 client.28319 handle_command_reply: tid=0 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.499+0000 7fc8e2fc5b80 1 client.28319 shutdown 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2125224265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16fa60 msgr2=0x55743e178c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16fa60 0x55743e178c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000ebb0 tx=0x7fc8c00047c0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2125224265 shutdown_connections 2024-05-08T23:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e173a40 0x55743e179150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e1786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55743e1888f0 0x55743e18ad10 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/2125224265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16fa60 0x55743e178c10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2125224265 >> 172.21.15.12:0/2125224265 conn(0x55743e0bc1b0 msgr2=0x55743e15cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8affff700 0 client.28319 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2125224265 shutdown_connections 2024-05-08T23:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2125224265 wait complete. 2024-05-08T23:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.500+0000 7fc8e2fc5b80 20 client.28319 trim_cache size 0 max 16384 2024-05-08T23:57:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.524+0000 7fc8e2fc5b80 1 Processor -- start 2024-05-08T23:57:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8e2fc5b80 1 -- start start 2024-05-08T23:57:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e171470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16fa60 0x55743e158910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e173a40 0x55743e15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e176c00 con 0x55743e16fa60 2024-05-08T23:57:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e172860 con 0x55743e16f140 2024-05-08T23:57:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e17b580 con 0x55743e173a40 2024-05-08T23:57:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8d4ba7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e171470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8c7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16fa60 0x55743e158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8c7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16fa60 0x55743e158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44220/0 (socket says 172.21.15.12:44220) 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8c7fff700 1 -- 172.21.15.12:0/2009989847 learned_addr learned my addr 172.21.15.12:0/2009989847 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8d53a8700 1 --2- 172.21.15.12:0/2009989847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e173a40 0x55743e15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/2009989847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e173a40 msgr2=0x55743e15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/2009989847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e173a40 0x55743e15d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/2009989847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16fa60 msgr2=0x55743e158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.525+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/2009989847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16fa60 0x55743e158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.526+0000 7fc8d4ba7700 1 -- 172.21.15.12:0/2009989847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55743e074500 con 0x55743e16f140 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.526+0000 7fc8c7fff700 1 --2- 172.21.15.12:0/2009989847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16fa60 0x55743e158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.526+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/2009989847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e171470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55743e17b7b0 tx=0x7fc8cc005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97c47b6be9f65870 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.526+0000 7fc8b77fe700 1 -- 172.21.15.12:0/2009989847 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc09edf0 con 0x55743e16f140 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.526+0000 7fc8b77fe700 1 -- 172.21.15.12:0/2009989847 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8cc09a920 con 0x55743e16f140 2024-05-08T23:57:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.526+0000 7fc8b77fe700 1 -- 172.21.15.12:0/2009989847 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0993f0 con 0x55743e16f140 2024-05-08T23:57:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.527+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2009989847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 msgr2=0x55743e171470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.527+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/2009989847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e171470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55743e17b7b0 tx=0x7fc8cc005f10 comp rx=0 tx=0).stop 2024-05-08T23:57:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.527+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2009989847 shutdown_connections 2024-05-08T23:57:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.527+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/2009989847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e173a40 0x55743e15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.527+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/2009989847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e16f140 0x55743e171470 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.527+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/2009989847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16fa60 0x55743e158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.527+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2009989847 >> 172.21.15.12:0/2009989847 conn(0x55743e0bc1b0 msgr2=0x55743e18f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.527+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2009989847 shutdown_connections 2024-05-08T23:57:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.527+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/2009989847 wait complete. 2024-05-08T23:57:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.527+0000 7fc8e2fc5b80 1 -- start start 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8e2fc5b80 1 Processor -- start 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e179170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e16fa60 0x55743e17f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8e2fc5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e173a40 0x55743e177cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e1801b0 con 0x55743e16f140 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e178210 con 0x55743e173a40 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8e2fc5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743e1783d0 con 0x55743e16fa60 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8e2fc5b80 10 client.0 _ll_register_callbacks cb 0x55743e053ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8d53a8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e173a40 0x55743e177cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e16fa60 0x55743e17f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.528+0000 7fc8d4ba7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e179170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8d53a8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e173a40 0x55743e177cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44478/0 (socket says 172.21.15.12:44478) 2024-05-08T23:57:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8d53a8700 1 -- 172.21.15.12:0/1373363043 learned_addr learned my addr 172.21.15.12:0/1373363043 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8d53a8700 1 -- 172.21.15.12:0/1373363043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e16fa60 msgr2=0x55743e17f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8d53a8700 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e16fa60 0x55743e17f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8d53a8700 1 -- 172.21.15.12:0/1373363043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 msgr2=0x55743e179170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8d53a8700 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e179170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8d53a8700 1 -- 172.21.15.12:0/1373363043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55743dedbb00 con 0x55743e173a40 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8c7fff700 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e16fa60 0x55743e17f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e179170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8d53a8700 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e173a40 0x55743e177cd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8004660 tx=0x7fc8c8004690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2560864ab29f9aac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8affff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8affff700 1 -- 172.21.15.12:0/1373363043 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c801c070 con 0x55743e173a40 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8affff700 1 -- 172.21.15.12:0/1373363043 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc8c8007710 con 0x55743e173a40 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.529+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/1373363043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8cc0a10c0 con 0x55743e173a40 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8affff700 1 -- 172.21.15.12:0/1373363043 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8c800e040 con 0x55743e173a40 2024-05-08T23:57:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8e2fc5b80 10 client.28325 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/1373363043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55743df2f350 con 0x55743e173a40 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8affff700 1 -- 172.21.15.12:0/1373363043 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8cc0a10c0 con 0x55743e173a40 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8affff700 1 -- 172.21.15.12:0/1373363043 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc8c802aac0 con 0x55743e173a40 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8e2fc5b80 10 client.28325 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8e2fc5b80 10 client.28325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8e2fc5b80 20 client.28325 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55743e1dec20 0x55743e1e1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8e2fc5b80 4 client.28325 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/1373363043 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55743e0a6e60 con 0x55743e1dec20 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.530+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55743e1dec20 0x55743e1e1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.531+0000 7fc8d4ba7700 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55743e1dec20 0x55743e1e1040 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a757db7454ca0ee server_cookie=65d9d08966ae1351 in_seq=0 out_seq=0 2024-05-08T23:57:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.531+0000 7fc8affff700 10 client.28325 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8affff700 1 -- 172.21.15.12:0/1373363043 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55743e0a6e60 con 0x55743e1dec20 2024-05-08T23:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8affff700 10 client.28325 handle_command_reply: tid=0 2024-05-08T23:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8e2fc5b80 1 client.28325 shutdown 2024-05-08T23:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/1373363043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e173a40 msgr2=0x55743e177cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e173a40 0x55743e177cd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8004660 tx=0x7fc8c8004690 comp rx=0 tx=0).stop 2024-05-08T23:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/1373363043 shutdown_connections 2024-05-08T23:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55743e1dec20 0x55743e1e1040 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55743e173a40 0x55743e177cd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55743e16f140 0x55743e179170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8e2fc5b80 1 --2- 172.21.15.12:0/1373363043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55743e16fa60 0x55743e17f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.590+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/1373363043 >> 172.21.15.12:0/1373363043 conn(0x55743e0bc1b0 msgr2=0x55743e15a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.591+0000 7fc8affff700 0 client.28325 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.592+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/1373363043 shutdown_connections 2024-05-08T23:57:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.592+0000 7fc8e2fc5b80 1 -- 172.21.15.12:0/1373363043 wait complete. 2024-05-08T23:57:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.592+0000 7fc8e2fc5b80 20 client.28325 trim_cache size 0 max 16384 2024-05-08T23:57:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:57:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:57:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:57:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.592+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/2711360007 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8b80a13e0 msgr2=0x7fc8b80a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.592+0000 7fc8d6e0b700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8b80a13e0 0x7fc8b80a38a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc8d01967c0 tx=0x7fc8c0009260 comp rx=0 tx=0).stop 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.592+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/2711360007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d00ffda0 msgr2=0x7fc8d019d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.592+0000 7fc8d6e0b700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d00ffda0 0x7fc8d019d040 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc000c00 tx=0x7fc8cc005740 comp rx=0 tx=0).stop 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.593+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/2711360007 shutdown_connections 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.593+0000 7fc8d6e0b700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc8b80a13e0 0x7fc8b80a38a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.593+0000 7fc8d6e0b700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d0105e60 0x7fc8d01953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.593+0000 7fc8d6e0b700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d00ffda0 0x7fc8d019d040 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.593+0000 7fc8d6e0b700 1 --2- 172.21.15.12:0/2711360007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d0100760 0x7fc8d019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.593+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/2711360007 >> 172.21.15.12:0/2711360007 conn(0x7fc8d0076f10 msgr2=0x7fc8d0068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.593+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/2711360007 shutdown_connections 2024-05-08T23:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:31.593+0000 7fc8d6e0b700 1 -- 172.21.15.12:0/2711360007 wait complete. 2024-05-08T23:57:31.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.647+0000 7f20f4350700 1 -- 172.21.15.12:0/141274003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20ec0682e0 msgr2=0x7f20ec0686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.647+0000 7f20f4350700 1 --2- 172.21.15.12:0/141274003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20ec0682e0 0x7f20ec0686c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f20e80099c0 tx=0x7f20e8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.647+0000 7f20f4350700 1 -- 172.21.15.12:0/141274003 shutdown_connections 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.647+0000 7f20f4350700 1 --2- 172.21.15.12:0/141274003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ec106290 0x7f20ec10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.647+0000 7f20f4350700 1 --2- 172.21.15.12:0/141274003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20ec0682e0 0x7f20ec0686c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.647+0000 7f20f4350700 1 --2- 172.21.15.12:0/141274003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec068c90 0x7f20ec105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.647+0000 7f20f4350700 1 -- 172.21.15.12:0/141274003 >> 172.21.15.12:0/141274003 conn(0x7f20ec076f40 msgr2=0x7f20ec077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.647+0000 7f20f4350700 1 -- 172.21.15.12:0/141274003 shutdown_connections 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.647+0000 7f20f4350700 1 -- 172.21.15.12:0/141274003 wait complete. 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f4350700 1 Processor -- start 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f4350700 1 -- start start 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f4350700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20ec0682e0 0x7f20ec0ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f4350700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec106290 0x7f20ec1003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f4350700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ec1008e0 0x7f20ec1a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f4350700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1a5b90 con 0x7f20ec0682e0 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f4350700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1a5d00 con 0x7f20ec1008e0 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f4350700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1a5ec0 con 0x7f20ec106290 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f20ec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20ec0682e0 0x7f20ec0ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f20ec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20ec0682e0 0x7f20ec0ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44246/0 (socket says 172.21.15.12:44246) 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f20ec700 1 -- 172.21.15.12:0/339581443 learned_addr learned my addr 172.21.15.12:0/339581443 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f28ed700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ec1008e0 0x7f20ec1a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f18eb700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec106290 0x7f20ec1003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f20ec700 1 -- 172.21.15.12:0/339581443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec106290 msgr2=0x7f20ec1003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f20ec700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec106290 0x7f20ec1003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f20ec700 1 -- 172.21.15.12:0/339581443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ec1008e0 msgr2=0x7f20ec1a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f20ec700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ec1008e0 0x7f20ec1a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.648+0000 7f20f20ec700 1 -- 172.21.15.12:0/339581443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e80095f0 con 0x7f20ec0682e0 2024-05-08T23:57:31.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.649+0000 7f20f20ec700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20ec0682e0 0x7f20ec0ffe60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f20e800b9c0 tx=0x7f20e8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.649+0000 7f20e37fe700 1 -- 172.21.15.12:0/339581443 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e801d070 con 0x7f20ec0682e0 2024-05-08T23:57:31.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.649+0000 7f20e37fe700 1 -- 172.21.15.12:0/339581443 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20e800bdb0 con 0x7f20ec0682e0 2024-05-08T23:57:31.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.649+0000 7f20e37fe700 1 -- 172.21.15.12:0/339581443 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e800f990 con 0x7f20ec0682e0 2024-05-08T23:57:31.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.649+0000 7f20f4350700 1 -- 172.21.15.12:0/339581443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec1a6140 con 0x7f20ec0682e0 2024-05-08T23:57:31.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.649+0000 7f20f4350700 1 -- 172.21.15.12:0/339581443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec1a6510 con 0x7f20ec0682e0 2024-05-08T23:57:31.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.652+0000 7f20e37fe700 1 -- 172.21.15.12:0/339581443 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f20e8003720 con 0x7f20ec0682e0 2024-05-08T23:57:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.652+0000 7f20e37fe700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f20d80a1290 0x7f20d80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.653+0000 7f20f18eb700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f20d80a1290 0x7f20d80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.653+0000 7f20e37fe700 1 -- 172.21.15.12:0/339581443 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f20e80cd9b0 con 0x7f20ec0682e0 2024-05-08T23:57:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.653+0000 7f210050ab80 1 Processor -- start 2024-05-08T23:57:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f210050ab80 1 -- start start 2024-05-08T23:57:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f210050ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8b0e80 0x559bfd8b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f210050ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b17a0 0x559bfd8ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f210050ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 0x559bfd8abd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f210050ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd8021d0 con 0x559bfd8b17a0 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f210050ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd7ffb90 con 0x559bfd8ab910 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f210050ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd7fb480 con 0x559bfd8b0e80 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f20f18eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b17a0 0x559bfd8ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f20f18eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b17a0 0x559bfd8ab3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44252/0 (socket says 172.21.15.12:44252) 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f20f18eb700 1 -- 172.21.15.12:0/1628917891 learned_addr learned my addr 172.21.15.12:0/1628917891 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f20f20ec700 1 --2- 172.21.15.12:0/1628917891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8b0e80 0x559bfd8b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f20f28ed700 1 --2- 172.21.15.12:0/1628917891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 0x559bfd8abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f20f18eb700 1 -- 172.21.15.12:0/1628917891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8b0e80 msgr2=0x559bfd8b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f20f18eb700 1 --2- 172.21.15.12:0/1628917891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8b0e80 0x559bfd8b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f20f18eb700 1 -- 172.21.15.12:0/1628917891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 msgr2=0x559bfd8abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.654+0000 7f20f18eb700 1 --2- 172.21.15.12:0/1628917891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 0x559bfd8abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.655+0000 7f20f18eb700 1 -- 172.21.15.12:0/1628917891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bfd67c430 con 0x559bfd8b17a0 2024-05-08T23:57:31.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.655+0000 7f20f18eb700 1 --2- 172.21.15.12:0/1628917891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b17a0 0x559bfd8ab3d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f20dc011c60 tx=0x7f20dc011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac43f52f921eec1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.655+0000 7f20f18eb700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f20d80a1290 0x7f20d80a3750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00f6a0 tx=0x7f20dc013000 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.655+0000 7f20e0da4700 1 -- 172.21.15.12:0/1628917891 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0099d0 con 0x559bfd8b17a0 2024-05-08T23:57:31.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.655+0000 7f20e0da4700 1 -- 172.21.15.12:0/1628917891 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20dc0092d0 con 0x559bfd8b17a0 2024-05-08T23:57:31.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.655+0000 7f20e0da4700 1 -- 172.21.15.12:0/1628917891 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00ed50 con 0x559bfd8b17a0 2024-05-08T23:57:31.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.656+0000 7f210050ab80 1 -- 172.21.15.12:0/1628917891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b17a0 msgr2=0x559bfd8ab3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.656+0000 7f210050ab80 1 --2- 172.21.15.12:0/1628917891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b17a0 0x559bfd8ab3d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f20dc011c60 tx=0x7f20dc011f70 comp rx=0 tx=0).stop 2024-05-08T23:57:31.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.656+0000 7f210050ab80 1 -- 172.21.15.12:0/1628917891 shutdown_connections 2024-05-08T23:57:31.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.656+0000 7f210050ab80 1 --2- 172.21.15.12:0/1628917891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 0x559bfd8abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.656+0000 7f210050ab80 1 --2- 172.21.15.12:0/1628917891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8b0e80 0x559bfd8b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.656+0000 7f210050ab80 1 --2- 172.21.15.12:0/1628917891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b17a0 0x559bfd8ab3d0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.656+0000 7f210050ab80 1 -- 172.21.15.12:0/1628917891 >> 172.21.15.12:0/1628917891 conn(0x559bfd809290 msgr2=0x559bfd8a5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.656+0000 7f210050ab80 1 -- 172.21.15.12:0/1628917891 shutdown_connections 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.656+0000 7f210050ab80 1 -- 172.21.15.12:0/1628917891 wait complete. 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.656+0000 7f210050ab80 1 -- start start 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f210050ab80 1 Processor -- start 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f210050ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f210050ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b0e80 0x559bfd8b5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f210050ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b5770 0x559bfd8c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f210050ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd7975e0 con 0x559bfd8b5770 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f210050ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd797280 con 0x559bfd8b0e80 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f210050ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd67e720 con 0x559bfd8ab910 2024-05-08T23:57:31.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f210050ab80 10 client.0 _ll_register_callbacks cb 0x559bfd7a0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f18eb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b0e80 0x559bfd8b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f28ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b5770 0x559bfd8c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f20ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f20ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:56802/0 (socket says 172.21.15.12:56802) 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f18eb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b0e80 0x559bfd8b5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44520/0 (socket says 172.21.15.12:44520) 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f20ec700 1 -- 172.21.15.12:0/1222072311 learned_addr learned my addr 172.21.15.12:0/1222072311 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f20ec700 1 -- 172.21.15.12:0/1222072311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b0e80 msgr2=0x559bfd8b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f20ec700 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b0e80 0x559bfd8b5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f20ec700 1 -- 172.21.15.12:0/1222072311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b5770 msgr2=0x559bfd8c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f20ec700 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b5770 0x559bfd8c1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f20ec700 1 -- 172.21.15.12:0/1222072311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc011890 con 0x559bfd8ab910 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f18eb700 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b0e80 0x559bfd8b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.657+0000 7f20f28ed700 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b5770 0x559bfd8c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.658+0000 7f20f20ec700 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8b4cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f20e8097570 tx=0x7f20e809b2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb363c797ea3088e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.658+0000 7f20d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.658+0000 7f20d5ffb700 1 -- 172.21.15.12:0/1222072311 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e809e930 con 0x559bfd8ab910 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.658+0000 7f20d5ffb700 1 -- 172.21.15.12:0/1222072311 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f20e80a2ef0 con 0x559bfd8ab910 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.658+0000 7f210050ab80 1 -- 172.21.15.12:0/1222072311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bfd7542f0 con 0x559bfd8ab910 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.658+0000 7f20d5ffb700 1 -- 172.21.15.12:0/1222072311 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20e80b3200 con 0x559bfd8ab910 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.658+0000 7f210050ab80 10 client.28321 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.658+0000 7f210050ab80 1 -- 172.21.15.12:0/1222072311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x559bfd628be0 con 0x559bfd8ab910 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.658+0000 7f20d5ffb700 1 -- 172.21.15.12:0/1222072311 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20e80b7070 con 0x559bfd8ab910 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.659+0000 7f20d5ffb700 1 -- 172.21.15.12:0/1222072311 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f20e800fb50 con 0x559bfd8ab910 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.659+0000 7f210050ab80 10 client.28321 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.659+0000 7f210050ab80 10 client.28321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.659+0000 7f210050ab80 20 client.28321 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.659+0000 7f210050ab80 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559bfd8c5450 0x559bfd8c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.659+0000 7f210050ab80 4 client.28321 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:31.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.659+0000 7f210050ab80 1 -- 172.21.15.12:0/1222072311 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bfd87a890 con 0x559bfd8c5450 2024-05-08T23:57:31.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.659+0000 7f20f18eb700 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559bfd8c5450 0x559bfd8c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.659+0000 7f20d5ffb700 10 client.28321 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.659+0000 7f20f18eb700 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559bfd8c5450 0x559bfd8c7840 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ad2827e39a133b4 server_cookie=5e6d839755fff7c0 in_seq=0 out_seq=0 2024-05-08T23:57:31.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.660+0000 7f20d5ffb700 1 -- 172.21.15.12:0/1222072311 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559bfd87a890 con 0x559bfd8c5450 2024-05-08T23:57:31.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.660+0000 7f20d5ffb700 10 client.28321 handle_command_reply: tid=0 2024-05-08T23:57:31.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.660+0000 7f210050ab80 1 client.28321 shutdown 2024-05-08T23:57:31.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 1 -- 172.21.15.12:0/1222072311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 msgr2=0x559bfd8b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8b4cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f20e8097570 tx=0x7f20e809b2f0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 1 -- 172.21.15.12:0/1222072311 shutdown_connections 2024-05-08T23:57:31.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b5770 0x559bfd8c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8b4cf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559bfd8c5450 0x559bfd8c7840 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 1 --2- 172.21.15.12:0/1222072311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b0e80 0x559bfd8b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f20d5ffb700 0 client.28321 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 1 -- 172.21.15.12:0/1222072311 >> 172.21.15.12:0/1222072311 conn(0x559bfd809290 msgr2=0x559bfd8a5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 1 -- 172.21.15.12:0/1222072311 shutdown_connections 2024-05-08T23:57:31.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 1 -- 172.21.15.12:0/1222072311 wait complete. 2024-05-08T23:57:31.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.661+0000 7f210050ab80 20 client.28321 trim_cache size 0 max 16384 2024-05-08T23:57:31.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.680+0000 7f210050ab80 1 Processor -- start 2024-05-08T23:57:31.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.680+0000 7f210050ab80 1 -- start start 2024-05-08T23:57:31.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.680+0000 7f210050ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 0x559bfd8ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.680+0000 7f210050ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.680+0000 7f210050ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8b5770 0x559bfd8afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.680+0000 7f210050ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd8a0b10 con 0x559bfd8b0e80 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.680+0000 7f210050ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd8a76c0 con 0x559bfd8ab910 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.680+0000 7f210050ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd8cc9e0 con 0x559bfd8b5770 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f20ec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 0x559bfd8ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f18eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f20ec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 0x559bfd8ab3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44528/0 (socket says 172.21.15.12:44528) 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f20ec700 1 -- 172.21.15.12:0/1022018951 learned_addr learned my addr 172.21.15.12:0/1022018951 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f18eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8a5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44272/0 (socket says 172.21.15.12:44272) 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f28ed700 1 --2- 172.21.15.12:0/1022018951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8b5770 0x559bfd8afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f20ec700 1 -- 172.21.15.12:0/1022018951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8b5770 msgr2=0x559bfd8afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f20ec700 1 --2- 172.21.15.12:0/1022018951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8b5770 0x559bfd8afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f20ec700 1 -- 172.21.15.12:0/1022018951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 msgr2=0x559bfd8a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f20ec700 1 --2- 172.21.15.12:0/1022018951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f20ec700 1 -- 172.21.15.12:0/1022018951 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bfd8a2560 con 0x559bfd8ab910 2024-05-08T23:57:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f18eb700 1 --2- 172.21.15.12:0/1022018951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20f20ec700 1 --2- 172.21.15.12:0/1022018951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 0x559bfd8ab3d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20e809aa40 tx=0x7f20e809ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f58baedf74f3b32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20e15a5700 1 -- 172.21.15.12:0/1022018951 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8096c80 con 0x559bfd8ab910 2024-05-08T23:57:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20e15a5700 1 -- 172.21.15.12:0/1022018951 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20e80b7020 con 0x559bfd8ab910 2024-05-08T23:57:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.681+0000 7f20e15a5700 1 -- 172.21.15.12:0/1022018951 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80b3200 con 0x559bfd8ab910 2024-05-08T23:57:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.682+0000 7f210050ab80 1 -- 172.21.15.12:0/1022018951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 msgr2=0x559bfd8ab3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.682+0000 7f210050ab80 1 --2- 172.21.15.12:0/1022018951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 0x559bfd8ab3d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20e809aa40 tx=0x7f20e809ac80 comp rx=0 tx=0).stop 2024-05-08T23:57:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.682+0000 7f210050ab80 1 -- 172.21.15.12:0/1022018951 shutdown_connections 2024-05-08T23:57:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.682+0000 7f210050ab80 1 --2- 172.21.15.12:0/1022018951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8b5770 0x559bfd8afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.682+0000 7f210050ab80 1 --2- 172.21.15.12:0/1022018951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8ab910 0x559bfd8ab3d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.682+0000 7f210050ab80 1 --2- 172.21.15.12:0/1022018951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.682+0000 7f210050ab80 1 -- 172.21.15.12:0/1022018951 >> 172.21.15.12:0/1022018951 conn(0x559bfd809290 msgr2=0x559bfd8b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.682+0000 7f210050ab80 1 -- 172.21.15.12:0/1022018951 shutdown_connections 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.682+0000 7f210050ab80 1 -- 172.21.15.12:0/1022018951 wait complete. 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f210050ab80 1 -- start start 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f210050ab80 1 Processor -- start 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f210050ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8ba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f210050ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8baa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f210050ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b5770 0x559bfd914880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f210050ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd914dc0 con 0x559bfd8b0e80 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f210050ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd914f00 con 0x559bfd8b5770 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f210050ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bfd9150c0 con 0x559bfd8ab910 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f210050ab80 10 client.0 _ll_register_callbacks cb 0x559bfd7a0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f20f20ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f20f18eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8baa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f20f20ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8ba540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:56814/0 (socket says 172.21.15.12:56814) 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f20f20ec700 1 -- 172.21.15.12:0/3380967998 learned_addr learned my addr 172.21.15.12:0/3380967998 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.683+0000 7f20f28ed700 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b5770 0x559bfd914880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20f20ec700 1 -- 172.21.15.12:0/3380967998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b5770 msgr2=0x559bfd914880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20f20ec700 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b5770 0x559bfd914880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20f20ec700 1 -- 172.21.15.12:0/3380967998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 msgr2=0x559bfd8baa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20f20ec700 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8baa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20f20ec700 1 -- 172.21.15.12:0/3380967998 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bfd67c430 con 0x559bfd8ab910 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20f18eb700 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8baa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20f28ed700 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b5770 0x559bfd914880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20f20ec700 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8ba540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f20e8097000 tx=0x7f20e8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4302ef835f6a9033 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20d5ffb700 1 -- 172.21.15.12:0/3380967998 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8096c80 con 0x559bfd8ab910 2024-05-08T23:57:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f210050ab80 1 -- 172.21.15.12:0/3380967998 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20dc0092d0 con 0x559bfd8ab910 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20d5ffb700 1 -- 172.21.15.12:0/3380967998 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f20e80a2ef0 con 0x559bfd8ab910 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f20d5ffb700 1 -- 172.21.15.12:0/3380967998 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20e800fb50 con 0x559bfd8ab910 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f210050ab80 10 client.28330 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.684+0000 7f210050ab80 1 -- 172.21.15.12:0/3380967998 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x559bfd628be0 con 0x559bfd8ab910 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.685+0000 7f20d5ffb700 1 -- 172.21.15.12:0/3380967998 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20dc0092d0 con 0x559bfd8ab910 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.685+0000 7f20d5ffb700 1 -- 172.21.15.12:0/3380967998 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f20e8099a60 con 0x559bfd8ab910 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.685+0000 7f210050ab80 10 client.28330 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.685+0000 7f210050ab80 10 client.28330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.685+0000 7f210050ab80 20 client.28330 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.685+0000 7f210050ab80 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559bfd91c490 0x559bfd91e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.685+0000 7f210050ab80 4 client.28330 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.685+0000 7f210050ab80 1 -- 172.21.15.12:0/3380967998 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559bfd7f3f40 con 0x559bfd91c490 2024-05-08T23:57:31.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.685+0000 7f20f18eb700 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559bfd91c490 0x559bfd91e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.686+0000 7f20d5ffb700 10 client.28330 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.686+0000 7f20f18eb700 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559bfd91c490 0x559bfd91e880 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7d27f425a112b9 server_cookie=8953d1e216a3a9bc in_seq=0 out_seq=0 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.686+0000 7f20d5ffb700 1 -- 172.21.15.12:0/3380967998 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559bfd7f3f40 con 0x559bfd91c490 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.686+0000 7f20d5ffb700 10 client.28330 handle_command_reply: tid=0 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.686+0000 7f210050ab80 1 client.28330 shutdown 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 1 -- 172.21.15.12:0/3380967998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 msgr2=0x559bfd8ba540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8ba540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f20e8097000 tx=0x7f20e8009310 comp rx=0 tx=0).stop 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 1 -- 172.21.15.12:0/3380967998 shutdown_connections 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559bfd91c490 0x559bfd91e880 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bfd8b5770 0x559bfd914880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bfd8ab910 0x559bfd8ba540 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 1 --2- 172.21.15.12:0/3380967998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bfd8b0e80 0x559bfd8baa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 1 -- 172.21.15.12:0/3380967998 >> 172.21.15.12:0/3380967998 conn(0x559bfd809290 msgr2=0x559bfd8afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f20d5ffb700 0 client.28330 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 1 -- 172.21.15.12:0/3380967998 shutdown_connections 2024-05-08T23:57:31.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 1 -- 172.21.15.12:0/3380967998 wait complete. 2024-05-08T23:57:31.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.687+0000 7f210050ab80 20 client.28330 trim_cache size 0 max 16384 2024-05-08T23:57:31.687 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:57:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:57:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "4e6aa0fb-6434-4f13-aee5-67a67b8792e6", 2024-05-08T23:57:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:57:31.687 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:57:31.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 -- 172.21.15.12:0/339581443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f20d80a1290 msgr2=0x7f20d80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f20d80a1290 0x7f20d80a3750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00f6a0 tx=0x7f20dc013000 comp rx=0 tx=0).stop 2024-05-08T23:57:31.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 -- 172.21.15.12:0/339581443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20ec0682e0 msgr2=0x7f20ec0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20ec0682e0 0x7f20ec0ffe60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f20e800b9c0 tx=0x7f20e8009580 comp rx=0 tx=0).stop 2024-05-08T23:57:31.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 -- 172.21.15.12:0/339581443 shutdown_connections 2024-05-08T23:57:31.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f20d80a1290 0x7f20d80a3750 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ec1008e0 0x7f20ec1a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20ec0682e0 0x7f20ec0ffe60 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 --2- 172.21.15.12:0/339581443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec106290 0x7f20ec1003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 -- 172.21.15.12:0/339581443 >> 172.21.15.12:0/339581443 conn(0x7f20ec076f40 msgr2=0x7f20ec0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 -- 172.21.15.12:0/339581443 shutdown_connections 2024-05-08T23:57:31.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.688+0000 7f20f4350700 1 -- 172.21.15.12:0/339581443 wait complete. 2024-05-08T23:57:31.704 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:57:31.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:31 smithi012 ceph-mon[42503]: pgmap v352: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-05-08T23:57:31.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 618044 6.33 MB/sec execute 447 sec latency 2890.181 ms 2024-05-08T23:57:31.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.900+0000 7f6fa3794700 1 -- 172.21.15.12:0/2584642089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f9c101b30 msgr2=0x7f6f9c105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.900+0000 7f6fa3794700 1 --2- 172.21.15.12:0/2584642089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f9c101b30 0x7f6f9c105b70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0098f0 tx=0x7f6f8c009c00 comp rx=0 tx=0).stop 2024-05-08T23:57:31.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.900+0000 7f6fa3794700 1 -- 172.21.15.12:0/2584642089 shutdown_connections 2024-05-08T23:57:31.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.900+0000 7f6fa3794700 1 --2- 172.21.15.12:0/2584642089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f9c1062a0 0x7f6f9c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.900+0000 7f6fa3794700 1 --2- 172.21.15.12:0/2584642089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9c101180 0x7f6f9c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.900+0000 7f6fa3794700 1 --2- 172.21.15.12:0/2584642089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f9c101b30 0x7f6f9c105b70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.900+0000 7f6fa3794700 1 -- 172.21.15.12:0/2584642089 >> 172.21.15.12:0/2584642089 conn(0x7f6f9c0fc9e0 msgr2=0x7f6f9c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.900+0000 7f6fa3794700 1 -- 172.21.15.12:0/2584642089 shutdown_connections 2024-05-08T23:57:31.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.900+0000 7f6fa3794700 1 -- 172.21.15.12:0/2584642089 wait complete. 2024-05-08T23:57:31.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa3794700 1 Processor -- start 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa3794700 1 -- start start 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa3794700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f9c101180 0x7f6f9c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa3794700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f9c1062a0 0x7f6f9c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa3794700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9c19de60 0x7f6f9c19e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa3794700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c19e8e0 con 0x7f6f9c101180 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa3794700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c19ea50 con 0x7f6f9c1062a0 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa3794700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c19ec10 con 0x7f6f9c19de60 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa1530700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f9c101180 0x7f6f9c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa1530700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f9c101180 0x7f6f9c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44294/0 (socket says 172.21.15.12:44294) 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.901+0000 7f6fa1530700 1 -- 172.21.15.12:0/1850348204 learned_addr learned my addr 172.21.15.12:0/1850348204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa0d2f700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f9c1062a0 0x7f6f9c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa1d31700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9c19de60 0x7f6f9c19e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa1530700 1 -- 172.21.15.12:0/1850348204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9c19de60 msgr2=0x7f6f9c19e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa1530700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9c19de60 0x7f6f9c19e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa1530700 1 -- 172.21.15.12:0/1850348204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f9c1062a0 msgr2=0x7f6f9c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa1530700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f9c1062a0 0x7f6f9c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa1530700 1 -- 172.21.15.12:0/1850348204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94009d00 con 0x7f6f9c101180 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa1d31700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9c19de60 0x7f6f9c19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa0d2f700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f9c1062a0 0x7f6f9c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa1530700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f9c101180 0x7f6f9c1995f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800e9a0 tx=0x7f6f9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6f927fc700 1 -- 172.21.15.12:0/1850348204 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98009790 con 0x7f6f9c101180 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6f927fc700 1 -- 172.21.15.12:0/1850348204 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f98010430 con 0x7f6f9c101180 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6f927fc700 1 -- 172.21.15.12:0/1850348204 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98018680 con 0x7f6f9c101180 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa3794700 1 -- 172.21.15.12:0/1850348204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c009520 con 0x7f6f9c101180 2024-05-08T23:57:31.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.902+0000 7f6fa3794700 1 -- 172.21.15.12:0/1850348204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c1a6470 con 0x7f6f9c101180 2024-05-08T23:57:31.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.905+0000 7f6f927fc700 1 -- 172.21.15.12:0/1850348204 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6f980105a0 con 0x7f6f9c101180 2024-05-08T23:57:31.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.906+0000 7f6f927fc700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f880a1290 0x7f6f880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.906+0000 7f6f927fc700 1 -- 172.21.15.12:0/1850348204 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6f98014070 con 0x7f6f9c101180 2024-05-08T23:57:31.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.906+0000 7f6fa0d2f700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f880a1290 0x7f6f880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6faf94eb80 1 Processor -- start 2024-05-08T23:57:31.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6faf94eb80 1 -- start start 2024-05-08T23:57:31.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6fa0d2f700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f880a1290 0x7f6f880a3750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009810 tx=0x7f6f8c019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 0x5565a30c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c2720 0x5565a30bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30bc890 0x5565a30bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a3013150 con 0x5565a30c2720 2024-05-08T23:57:31.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a3010b10 con 0x5565a30c1e00 2024-05-08T23:57:31.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a300c400 con 0x5565a30bc890 2024-05-08T23:57:31.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6fa1530700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 0x5565a30c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6fa0d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c2720 0x5565a30bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6fa1530700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 0x5565a30c21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44576/0 (socket says 172.21.15.12:44576) 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6fa1530700 1 -- 172.21.15.12:0/2291314078 learned_addr learned my addr 172.21.15.12:0/2291314078 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.907+0000 7f6fa1d31700 1 --2- 172.21.15.12:0/2291314078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30bc890 0x5565a30bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.908+0000 7f6fa1530700 1 -- 172.21.15.12:0/2291314078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30bc890 msgr2=0x5565a30bccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.908+0000 7f6fa1530700 1 --2- 172.21.15.12:0/2291314078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30bc890 0x5565a30bccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.908+0000 7f6fa1530700 1 -- 172.21.15.12:0/2291314078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c2720 msgr2=0x5565a30bc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.908+0000 7f6fa1530700 1 --2- 172.21.15.12:0/2291314078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c2720 0x5565a30bc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.908+0000 7f6fa1530700 1 -- 172.21.15.12:0/2291314078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565a2e8d3b0 con 0x5565a30c1e00 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.908+0000 7f6fa1d31700 1 --2- 172.21.15.12:0/2291314078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30bc890 0x5565a30bccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.908+0000 7f6fa1530700 1 --2- 172.21.15.12:0/2291314078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 0x5565a30c21e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6f980a1f20 tx=0x7f6f9809a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab3942e7cc0015a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.908+0000 7f6f877fe700 1 -- 172.21.15.12:0/2291314078 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980adea0 con 0x5565a30c1e00 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.908+0000 7f6f877fe700 1 -- 172.21.15.12:0/2291314078 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f980a2450 con 0x5565a30c1e00 2024-05-08T23:57:31.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.908+0000 7f6f877fe700 1 -- 172.21.15.12:0/2291314078 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9809a430 con 0x5565a30c1e00 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2291314078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 msgr2=0x5565a30c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/2291314078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 0x5565a30c21e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6f980a1f20 tx=0x7f6f9809a6d0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2291314078 shutdown_connections 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/2291314078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30bc890 0x5565a30bccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/2291314078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 0x5565a30c21e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/2291314078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c2720 0x5565a30bc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2291314078 >> 172.21.15.12:0/2291314078 conn(0x5565a301a210 msgr2=0x5565a30b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2291314078 shutdown_connections 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2291314078 wait complete. 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 -- start start 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.909+0000 7f6faf94eb80 1 Processor -- start 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 0x5565a30c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a2fa8560 con 0x5565a30c1e00 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a2fa8200 con 0x5565a30bc890 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a2e8f6a0 con 0x5565a30c2720 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6faf94eb80 10 client.0 _ll_register_callbacks cb 0x5565a2fb1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa0d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa1530700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 0x5565a30c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa0d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44318/0 (socket says 172.21.15.12:44318) 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa0d2f700 1 -- 172.21.15.12:0/3963281900 learned_addr learned my addr 172.21.15.12:0/3963281900 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa1d31700 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa1530700 1 -- 172.21.15.12:0/3963281900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 msgr2=0x5565a30c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa1530700 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa1530700 1 -- 172.21.15.12:0/3963281900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 msgr2=0x5565a30c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa1530700 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa1530700 1 -- 172.21.15.12:0/3963281900 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9400e740 con 0x5565a30bc890 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa1d31700 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.910+0000 7f6fa0d2f700 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6fa1530700 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 0x5565a30c5b60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5565a30c8d40 tx=0x7f6f98010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2549122b2b6beaf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6f84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6f84ff9700 1 -- 172.21.15.12:0/3963281900 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980adea0 con 0x5565a30bc890 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6f84ff9700 1 -- 172.21.15.12:0/3963281900 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6f98098370 con 0x5565a30bc890 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6faf94eb80 1 -- 172.21.15.12:0/3963281900 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565a2f65270 con 0x5565a30bc890 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6f84ff9700 1 -- 172.21.15.12:0/3963281900 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f98094df0 con 0x5565a30bc890 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6faf94eb80 10 client.28361 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6faf94eb80 1 -- 172.21.15.12:0/3963281900 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f6f9c1a6470 con 0x5565a30bc890 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6f84ff9700 1 -- 172.21.15.12:0/3963281900 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f98063080 con 0x5565a30bc890 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6f84ff9700 1 -- 172.21.15.12:0/3963281900 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6f980a2030 con 0x5565a30bc890 2024-05-08T23:57:31.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6faf94eb80 10 client.28361 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6faf94eb80 10 client.28361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6faf94eb80 20 client.28361 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5565a30d5de0 0x5565a30d8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6faf94eb80 4 client.28361 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.911+0000 7f6faf94eb80 1 -- 172.21.15.12:0/3963281900 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565a308b810 con 0x5565a30d5de0 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.912+0000 7f6fa0d2f700 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5565a30d5de0 0x5565a30d8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.912+0000 7f6fa0d2f700 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5565a30d5de0 0x5565a30d8200 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=538d561a79ebcf76 server_cookie=bbe58a384617abc2 in_seq=0 out_seq=0 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.912+0000 7f6f84ff9700 10 client.28361 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6f84ff9700 1 -- 172.21.15.12:0/3963281900 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565a308b810 con 0x5565a30d5de0 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6f84ff9700 10 client.28361 handle_command_reply: tid=0 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 client.28361 shutdown 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 -- 172.21.15.12:0/3963281900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 msgr2=0x5565a30c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 0x5565a30c5b60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5565a30c8d40 tx=0x7f6f98010ab0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 -- 172.21.15.12:0/3963281900 shutdown_connections 2024-05-08T23:57:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5565a30d5de0 0x5565a30d8200 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 0x5565a30c5b60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6f84ff9700 0 client.28361 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/3963281900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 -- 172.21.15.12:0/3963281900 >> 172.21.15.12:0/3963281900 conn(0x5565a301a210 msgr2=0x5565a30b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 -- 172.21.15.12:0/3963281900 shutdown_connections 2024-05-08T23:57:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 1 -- 172.21.15.12:0/3963281900 wait complete. 2024-05-08T23:57:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.913+0000 7f6faf94eb80 20 client.28361 trim_cache size 0 max 16384 2024-05-08T23:57:31.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6faf94eb80 1 Processor -- start 2024-05-08T23:57:31.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6faf94eb80 1 -- start start 2024-05-08T23:57:31.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30bc890 0x5565a30b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 0x5565a30c6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a30dd440 con 0x5565a30bc890 2024-05-08T23:57:31.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a30b1800 con 0x5565a30c1e00 2024-05-08T23:57:31.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a30bc690 con 0x5565a30c2720 2024-05-08T23:57:31.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6fa1530700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30bc890 0x5565a30b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6fa1530700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30bc890 0x5565a30b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44324/0 (socket says 172.21.15.12:44324) 2024-05-08T23:57:31.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6fa1d31700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.933+0000 7f6fa1530700 1 -- 172.21.15.12:0/1572876070 learned_addr learned my addr 172.21.15.12:0/1572876070 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.934+0000 7f6fa0d2f700 1 --2- 172.21.15.12:0/1572876070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 0x5565a30c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.934+0000 7f6fa1530700 1 -- 172.21.15.12:0/1572876070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 msgr2=0x5565a30c0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.934+0000 7f6fa1530700 1 --2- 172.21.15.12:0/1572876070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30c0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.934+0000 7f6fa1530700 1 -- 172.21.15.12:0/1572876070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 msgr2=0x5565a30c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.934+0000 7f6fa1530700 1 --2- 172.21.15.12:0/1572876070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 0x5565a30c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.934+0000 7f6fa1530700 1 -- 172.21.15.12:0/1572876070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565a2e39b60 con 0x5565a30bc890 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.934+0000 7f6fa1530700 1 --2- 172.21.15.12:0/1572876070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30bc890 0x5565a30b6d20 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6f98099f60 tx=0x7f6f9809a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc1a57a45aafaeb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.934+0000 7f6f87fff700 1 -- 172.21.15.12:0/1572876070 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980adea0 con 0x5565a30bc890 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.934+0000 7f6f87fff700 1 -- 172.21.15.12:0/1572876070 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565a2f65270 con 0x5565a30bc890 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.934+0000 7f6f87fff700 1 -- 172.21.15.12:0/1572876070 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980992a0 con 0x5565a30bc890 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.935+0000 7f6faf94eb80 1 -- 172.21.15.12:0/1572876070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30bc890 msgr2=0x5565a30b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.935+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/1572876070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30bc890 0x5565a30b6d20 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6f98099f60 tx=0x7f6f9809a6d0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.935+0000 7f6faf94eb80 1 -- 172.21.15.12:0/1572876070 shutdown_connections 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.935+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/1572876070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.935+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/1572876070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30bc890 0x5565a30b6d20 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.935+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/1572876070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30c1e00 0x5565a30c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.935+0000 7f6faf94eb80 1 -- 172.21.15.12:0/1572876070 >> 172.21.15.12:0/1572876070 conn(0x5565a301a210 msgr2=0x5565a30c4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.935+0000 7f6faf94eb80 1 -- 172.21.15.12:0/1572876070 shutdown_connections 2024-05-08T23:57:31.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.935+0000 7f6faf94eb80 1 -- 172.21.15.12:0/1572876070 wait complete. 2024-05-08T23:57:31.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.935+0000 7f6faf94eb80 1 -- start start 2024-05-08T23:57:31.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6faf94eb80 1 Processor -- start 2024-05-08T23:57:31.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 0x5565a30cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6faf94eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a312b440 con 0x5565a30c1e00 2024-05-08T23:57:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a312b5b0 con 0x5565a30bc890 2024-05-08T23:57:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6faf94eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a312b770 con 0x5565a30c2720 2024-05-08T23:57:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6faf94eb80 10 client.0 _ll_register_callbacks cb 0x5565a2fb1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6fa0d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6fa1d31700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6fa0d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44342/0 (socket says 172.21.15.12:44342) 2024-05-08T23:57:31.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6fa0d2f700 1 -- 172.21.15.12:0/2911924221 learned_addr learned my addr 172.21.15.12:0/2911924221 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6fa1d31700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:56880/0 (socket says 172.21.15.12:56880) 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.936+0000 7f6fa1530700 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 0x5565a30cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6fa1d31700 1 -- 172.21.15.12:0/2911924221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 msgr2=0x5565a30cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6fa1d31700 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 0x5565a30cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6fa1d31700 1 -- 172.21.15.12:0/2911924221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 msgr2=0x5565a30cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6fa1d31700 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6fa1d31700 1 -- 172.21.15.12:0/2911924221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c1a6470 con 0x5565a30c2720 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6fa0d2f700 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6fa1d31700 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30cbbd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009610 tx=0x7f6f94007d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97c1f3c41dd06b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6f84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6f84ff9700 1 -- 172.21.15.12:0/2911924221 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9401c070 con 0x5565a30c2720 2024-05-08T23:57:31.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6f84ff9700 1 -- 172.21.15.12:0/2911924221 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6f94004bb0 con 0x5565a30c2720 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2911924221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f980a2450 con 0x5565a30c2720 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6f84ff9700 1 -- 172.21.15.12:0/2911924221 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f9400e040 con 0x5565a30c2720 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6faf94eb80 10 client.28351 fetch_fsmap learned FSMap version 46 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2911924221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5565a2fd2560 con 0x5565a30c2720 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.937+0000 7f6f84ff9700 1 -- 172.21.15.12:0/2911924221 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f980a2450 con 0x5565a30c2720 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6f84ff9700 1 -- 172.21.15.12:0/2911924221 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 46) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6f940041e0 con 0x5565a30c2720 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6faf94eb80 10 client.28351 fetch_fsmap finished waiting for FSMap version 46 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6faf94eb80 10 client.28351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6faf94eb80 20 client.28351 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5565a312cfc0 0x5565a312f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6faf94eb80 4 client.28351 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2911924221 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5565a3004ec0 con 0x5565a312cfc0 2024-05-08T23:57:31.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6fa1530700 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5565a312cfc0 0x5565a312f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6f84ff9700 10 client.28351 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6fa1530700 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5565a312cfc0 0x5565a312f3e0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a333654079c75df server_cookie=4dac1adb057f2a93 in_seq=0 out_seq=0 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6f84ff9700 1 -- 172.21.15.12:0/2911924221 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5565a3004ec0 con 0x5565a312cfc0 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6f84ff9700 10 client.28351 handle_command_reply: tid=0 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.938+0000 7f6faf94eb80 1 client.28351 shutdown 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2911924221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 msgr2=0x5565a30cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30cbbd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009610 tx=0x7f6f94007d70 comp rx=0 tx=0).stop 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2911924221 shutdown_connections 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565a30c2720 0x5565a30cbbd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5565a312cfc0 0x5565a312f3e0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565a30bc890 0x5565a30cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 1 --2- 172.21.15.12:0/2911924221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565a30c1e00 0x5565a30cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6f84ff9700 0 client.28351 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:31.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2911924221 >> 172.21.15.12:0/2911924221 conn(0x5565a301a210 msgr2=0x5565a30c0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2911924221 shutdown_connections 2024-05-08T23:57:31.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 1 -- 172.21.15.12:0/2911924221 wait complete. 2024-05-08T23:57:31.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6faf94eb80 20 client.28351 trim_cache size 0 max 16384 2024-05-08T23:57:31.942 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:57:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:57:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:57:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: "4e6aa0fb-6434-4f13-aee5-67a67b8792e6": { 2024-05-08T23:57:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:57:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "4e6aa0fb-6434-4f13-aee5-67a67b8792e6", 2024-05-08T23:57:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:57:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:57:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:57:31.943 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:57:31.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6fa3794700 1 -- 172.21.15.12:0/1850348204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f880a1290 msgr2=0x7f6f880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6fa3794700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f880a1290 0x7f6f880a3750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009810 tx=0x7f6f8c019040 comp rx=0 tx=0).stop 2024-05-08T23:57:31.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6fa3794700 1 -- 172.21.15.12:0/1850348204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f9c101180 msgr2=0x7f6f9c1995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:31.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6fa3794700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f9c101180 0x7f6f9c1995f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800e9a0 tx=0x7f6f9800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6fa3794700 1 -- 172.21.15.12:0/1850348204 shutdown_connections 2024-05-08T23:57:31.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6fa3794700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6f880a1290 0x7f6f880a3750 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6fa3794700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9c19de60 0x7f6f9c19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6fa3794700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f9c101180 0x7f6f9c1995f0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6fa3794700 1 --2- 172.21.15.12:0/1850348204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f9c1062a0 0x7f6f9c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.939+0000 7f6fa3794700 1 -- 172.21.15.12:0/1850348204 >> 172.21.15.12:0/1850348204 conn(0x7f6f9c0fc9e0 msgr2=0x7f6f9c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.940+0000 7f6fa3794700 1 -- 172.21.15.12:0/1850348204 shutdown_connections 2024-05-08T23:57:31.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:57:31.940+0000 7f6fa3794700 1 -- 172.21.15.12:0/1850348204 wait complete. 2024-05-08T23:57:31.948 INFO:tasks.cephfs.filesystem:scrub status for tag:4e6aa0fb-6434-4f13-aee5-67a67b8792e6 - {'path': '/', 'tag': '4e6aa0fb-6434-4f13-aee5-67a67b8792e6', 'options': 'recursive,force'} 2024-05-08T23:57:32.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:31 smithi204 ceph-mon[47373]: pgmap v352: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-05-08T23:57:32.012 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:57:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-08T23:57:32.012 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:57:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-08T23:57:32.013 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:57:31 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/2704026281 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5605276a81a0 con 0x56052767e800 2024-05-08T23:57:32.013 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:57:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-08T23:57:32.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:31 smithi160 ceph-mon[46152]: pgmap v352: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-05-08T23:57:32.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 619385 6.33 MB/sec execute 448 sec latency 13.429 ms 2024-05-08T23:57:32.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:57:32.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:32.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:57:32.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:57:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:57:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-08T23:57:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:57:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T23:57:32.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:32 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:57:33.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:57:33.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:57:33.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:57:33.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-08T23:57:33.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:57:33.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T23:57:33.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:32 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:57:33.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:57:33.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:57:33.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:57:33.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-08T23:57:33.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:57:33.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T23:57:33.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:32 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:57:33.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 619675 6.32 MB/sec execute 449 sec latency 637.063 ms 2024-05-08T23:57:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:33 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/2125160991,v1:172.21.15.204:6837/2125160991] up:boot 2024-05-08T23:57:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:33 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:57:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:33 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:57:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:33 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:33 smithi012 ceph-mon[42503]: pgmap v354: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-08T23:57:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:33 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/2125160991,v1:172.21.15.204:6837/2125160991] up:boot 2024-05-08T23:57:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:33 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:57:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:33 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:57:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:33 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:34.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:33 smithi204 ceph-mon[47373]: pgmap v354: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-08T23:57:34.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:33 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/2125160991,v1:172.21.15.204:6837/2125160991] up:boot 2024-05-08T23:57:34.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:33 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:57:34.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:33 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:57:34.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:33 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:34.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:33 smithi160 ceph-mon[46152]: pgmap v354: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-08T23:57:34.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 619675 6.31 MB/sec execute 450 sec latency 1637.169 ms 2024-05-08T23:57:35.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 619675 6.29 MB/sec execute 451 sec latency 2637.293 ms 2024-05-08T23:57:35.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:35 smithi012 ceph-mon[42503]: pgmap v355: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-08T23:57:36.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:35 smithi204 ceph-mon[47373]: pgmap v355: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-08T23:57:36.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:35 smithi160 ceph-mon[46152]: pgmap v355: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-08T23:57:36.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 621738 6.31 MB/sec execute 452 sec latency 2799.212 ms 2024-05-08T23:57:37.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 622962 6.31 MB/sec execute 453 sec latency 17.865 ms 2024-05-08T23:57:37.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:37 smithi012 ceph-mon[42503]: pgmap v356: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-08T23:57:38.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:37 smithi204 ceph-mon[47373]: pgmap v356: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-08T23:57:38.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:37 smithi160 ceph-mon[46152]: pgmap v356: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-08T23:57:38.890 INFO:tasks.workunit.client.0.smithi012.stdout: 1 623219 6.29 MB/sec execute 454 sec latency 663.496 ms 2024-05-08T23:57:38.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:38 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:57:38.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:57:39.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:38 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:57:39.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:57:39.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:38 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:57:39.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:57:39.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 623219 6.28 MB/sec execute 455 sec latency 1663.597 ms 2024-05-08T23:57:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:39 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:57:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:39 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:57:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:39 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:57:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:39 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:57:39.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:39 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:57:39.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:39 smithi012 ceph-mon[42503]: pgmap v357: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-08T23:57:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:39 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:57:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:39 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:57:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:39 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:57:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:39 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:57:40.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:39 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:57:40.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:39 smithi204 ceph-mon[47373]: pgmap v357: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-08T23:57:40.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:39 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:57:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:39 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:57:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:39 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:57:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:39 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:57:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:39 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:57:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:39 smithi160 ceph-mon[46152]: pgmap v357: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-08T23:57:40.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 623219 6.26 MB/sec execute 456 sec latency 2663.713 ms 2024-05-08T23:57:41.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 625320 6.28 MB/sec execute 457 sec latency 2815.919 ms 2024-05-08T23:57:41.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:41 smithi012 ceph-mon[42503]: pgmap v358: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-08T23:57:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:41 smithi204 ceph-mon[47373]: pgmap v358: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-08T23:57:42.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:41 smithi160 ceph-mon[46152]: pgmap v358: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-08T23:57:42.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 626608 6.28 MB/sec execute 458 sec latency 22.339 ms 2024-05-08T23:57:43.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 626768 6.26 MB/sec execute 459 sec latency 847.587 ms 2024-05-08T23:57:43.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:43 smithi012 ceph-mon[42503]: pgmap v359: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-05-08T23:57:44.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:43 smithi204 ceph-mon[47373]: pgmap v359: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-05-08T23:57:44.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:43 smithi160 ceph-mon[46152]: pgmap v359: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-05-08T23:57:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.731+0000 7f291d2d1700 1 -- 172.21.15.12:0/3559888232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2918101a90 msgr2=0x7f2918105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.731+0000 7f291d2d1700 1 --2- 172.21.15.12:0/3559888232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2918101a90 0x7f2918105ad0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f290c009990 tx=0x7f290c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.731+0000 7f291d2d1700 1 -- 172.21.15.12:0/3559888232 shutdown_connections 2024-05-08T23:57:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.731+0000 7f291d2d1700 1 --2- 172.21.15.12:0/3559888232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2918106200 0x7f291810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.731+0000 7f291d2d1700 1 --2- 172.21.15.12:0/3559888232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29181010e0 0x7f29181014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.731+0000 7f291d2d1700 1 --2- 172.21.15.12:0/3559888232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2918101a90 0x7f2918105ad0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.731+0000 7f291d2d1700 1 -- 172.21.15.12:0/3559888232 >> 172.21.15.12:0/3559888232 conn(0x7f29180fc940 msgr2=0x7f29180fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.731+0000 7f291d2d1700 1 -- 172.21.15.12:0/3559888232 shutdown_connections 2024-05-08T23:57:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.731+0000 7f291d2d1700 1 -- 172.21.15.12:0/3559888232 wait complete. 2024-05-08T23:57:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f291d2d1700 1 Processor -- start 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f291d2d1700 1 -- start start 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f291d2d1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29181010e0 0x7f29181973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f291d2d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2918106200 0x7f2918197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f291d2d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f291819bcc0 0x7f291819c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f291d2d1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291819c740 con 0x7f291819bcc0 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f291d2d1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291819c8b0 con 0x7f2918106200 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f291d2d1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291819ca70 con 0x7f29181010e0 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f29177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f291819bcc0 0x7f291819c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f29177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f291819bcc0 0x7f291819c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58884/0 (socket says 172.21.15.12:58884) 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f29177fe700 1 -- 172.21.15.12:0/2019048428 learned_addr learned my addr 172.21.15.12:0/2019048428 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.732+0000 7f2916ffd700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29181010e0 0x7f29181973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f29167fc700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2918106200 0x7f2918197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f29177fe700 1 -- 172.21.15.12:0/2019048428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29181010e0 msgr2=0x7f29181973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f29177fe700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29181010e0 0x7f29181973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f29177fe700 1 -- 172.21.15.12:0/2019048428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2918106200 msgr2=0x7f2918197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f29177fe700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2918106200 0x7f2918197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f29177fe700 1 -- 172.21.15.12:0/2019048428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c0095f0 con 0x7f291819bcc0 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f29167fc700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2918106200 0x7f2918197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f29177fe700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f291819bcc0 0x7f291819c140 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f2910007ee0 tx=0x7f291000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f28fffff700 1 -- 172.21.15.12:0/2019048428 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291000ddd0 con 0x7f291819bcc0 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f291d2d1700 1 -- 172.21.15.12:0/2019048428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29181a3fd0 con 0x7f291819bcc0 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f28fffff700 1 -- 172.21.15.12:0/2019048428 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f291000f040 con 0x7f291819bcc0 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f28fffff700 1 -- 172.21.15.12:0/2019048428 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29100048e0 con 0x7f291819bcc0 2024-05-08T23:57:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.733+0000 7f291d2d1700 1 -- 172.21.15.12:0/2019048428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29181a44b0 con 0x7f291819bcc0 2024-05-08T23:57:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.735+0000 7f28fffff700 1 -- 172.21.15.12:0/2019048428 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2910003a00 con 0x7f291819bcc0 2024-05-08T23:57:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.736+0000 7f28fffff700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29000a1290 0x7f29000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.736+0000 7f28fffff700 1 -- 172.21.15.12:0/2019048428 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f29100cc560 con 0x7f291819bcc0 2024-05-08T23:57:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.736+0000 7f2916ffd700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29000a1290 0x7f29000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.737+0000 7f2916ffd700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29000a1290 0x7f29000a3750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2908005950 tx=0x7f290800b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.737+0000 7f292948bb80 1 Processor -- start 2024-05-08T23:57:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.737+0000 7f292948bb80 1 -- start start 2024-05-08T23:57:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.737+0000 7f292948bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e5d950 0x562427e5dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f292948bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e57f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f292948bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e58480 0x562427e588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f292948bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427daf0f0 con 0x562427e58480 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f292948bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427dacab0 con 0x562427e5e270 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f292948bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427da83a0 con 0x562427e5d950 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f29177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e58480 0x562427e588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f29177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e58480 0x562427e588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58894/0 (socket says 172.21.15.12:58894) 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f29177fe700 1 -- 172.21.15.12:0/842846116 learned_addr learned my addr 172.21.15.12:0/842846116 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f2916ffd700 1 --2- 172.21.15.12:0/842846116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e5d950 0x562427e5dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f29167fc700 1 --2- 172.21.15.12:0/842846116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e57f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f29177fe700 1 -- 172.21.15.12:0/842846116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e5d950 msgr2=0x562427e5dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f29177fe700 1 --2- 172.21.15.12:0/842846116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e5d950 0x562427e5dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f29177fe700 1 -- 172.21.15.12:0/842846116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 msgr2=0x562427e57f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f29177fe700 1 --2- 172.21.15.12:0/842846116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e57f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f29177fe700 1 -- 172.21.15.12:0/842846116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562427c29350 con 0x562427e58480 2024-05-08T23:57:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f29177fe700 1 --2- 172.21.15.12:0/842846116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e58480 0x562427e588a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f2910099640 tx=0x7f2910095b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc03eca4fb78b05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f28fd7fa700 1 -- 172.21.15.12:0/842846116 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910099db0 con 0x562427e58480 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f28fd7fa700 1 -- 172.21.15.12:0/842846116 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29100996b0 con 0x562427e58480 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.738+0000 7f28fd7fa700 1 -- 172.21.15.12:0/842846116 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29100b9070 con 0x562427e58480 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.739+0000 7f292948bb80 1 -- 172.21.15.12:0/842846116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e58480 msgr2=0x562427e588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.739+0000 7f292948bb80 1 --2- 172.21.15.12:0/842846116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e58480 0x562427e588a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f2910099640 tx=0x7f2910095b10 comp rx=0 tx=0).stop 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.739+0000 7f292948bb80 1 -- 172.21.15.12:0/842846116 shutdown_connections 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.739+0000 7f292948bb80 1 --2- 172.21.15.12:0/842846116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e58480 0x562427e588a0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.739+0000 7f292948bb80 1 --2- 172.21.15.12:0/842846116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e5d950 0x562427e5dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.739+0000 7f292948bb80 1 --2- 172.21.15.12:0/842846116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e57f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.739+0000 7f292948bb80 1 -- 172.21.15.12:0/842846116 >> 172.21.15.12:0/842846116 conn(0x562427db61b0 msgr2=0x562427e52840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.739+0000 7f292948bb80 1 -- 172.21.15.12:0/842846116 shutdown_connections 2024-05-08T23:57:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.739+0000 7f292948bb80 1 -- 172.21.15.12:0/842846116 wait complete. 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f292948bb80 1 -- start start 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f292948bb80 1 Processor -- start 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f292948bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e58480 0x562427e64280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f292948bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e61550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f292948bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e5e270 0x562427e61c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f292948bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427d44500 con 0x562427e5d950 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f292948bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427d441a0 con 0x562427e58480 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f292948bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427c2b640 con 0x562427e5e270 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f292948bb80 10 client.0 _ll_register_callbacks cb 0x562427d4dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f29167fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e61550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f29167fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e61550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58906/0 (socket says 172.21.15.12:58906) 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f29167fc700 1 -- 172.21.15.12:0/2548193379 learned_addr learned my addr 172.21.15.12:0/2548193379 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f29177fe700 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e5e270 0x562427e61c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f2916ffd700 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e58480 0x562427e64280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f29167fc700 1 -- 172.21.15.12:0/2548193379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e5e270 msgr2=0x562427e61c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f29167fc700 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e5e270 0x562427e61c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f29167fc700 1 -- 172.21.15.12:0/2548193379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e58480 msgr2=0x562427e64280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.740+0000 7f29167fc700 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e58480 0x562427e64280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f29167fc700 1 -- 172.21.15.12:0/2548193379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29181a44b0 con 0x562427e5d950 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f2916ffd700 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e58480 0x562427e64280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f29167fc700 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e61550 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f290c009990 tx=0x7f290c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad9215ffed97b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f28faff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f28faff5700 1 -- 172.21.15.12:0/2548193379 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c00bb80 con 0x562427e5d950 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f28faff5700 1 -- 172.21.15.12:0/2548193379 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f290c00bce0 con 0x562427e5d950 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f292948bb80 1 -- 172.21.15.12:0/2548193379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562427d01210 con 0x562427e5d950 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f28faff5700 1 -- 172.21.15.12:0/2548193379 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f290c00bee0 con 0x562427e5d950 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f292948bb80 10 client.18645 fetch_fsmap learned FSMap version 48 2024-05-08T23:57:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f292948bb80 1 -- 172.21.15.12:0/2548193379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x562427bd5b00 con 0x562427e5d950 2024-05-08T23:57:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f28faff5700 1 -- 172.21.15.12:0/2548193379 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562427d01210 con 0x562427e5d950 2024-05-08T23:57:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.741+0000 7f28faff5700 1 -- 172.21.15.12:0/2548193379 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 48) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f290c0245f0 con 0x562427e5d950 2024-05-08T23:57:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.742+0000 7f292948bb80 10 client.18645 fetch_fsmap finished waiting for FSMap version 48 2024-05-08T23:57:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.742+0000 7f292948bb80 10 client.18645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.742+0000 7f292948bb80 20 client.18645 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.742+0000 7f292948bb80 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562427e76070 0x562427e78490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.742+0000 7f292948bb80 4 client.18645 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.742+0000 7f292948bb80 1 -- 172.21.15.12:0/2548193379 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562427e277b0 con 0x562427e76070 2024-05-08T23:57:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.742+0000 7f2916ffd700 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562427e76070 0x562427e78490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.742+0000 7f28faff5700 10 client.18645 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.742+0000 7f2916ffd700 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562427e76070 0x562427e78490 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a502d59a2638d41 server_cookie=c79aca65e85410b8 in_seq=0 out_seq=0 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f28faff5700 1 -- 172.21.15.12:0/2548193379 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562427e277b0 con 0x562427e76070 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f28faff5700 10 client.18645 handle_command_reply: tid=0 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 client.18645 shutdown 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 -- 172.21.15.12:0/2548193379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 msgr2=0x562427e61550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e61550 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f290c009990 tx=0x7f290c005800 comp rx=0 tx=0).stop 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 -- 172.21.15.12:0/2548193379 shutdown_connections 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e5e270 0x562427e61c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562427e76070 0x562427e78490 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e58480 0x562427e64280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 --2- 172.21.15.12:0/2548193379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e61550 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f28faff5700 0 client.18645 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 -- 172.21.15.12:0/2548193379 >> 172.21.15.12:0/2548193379 conn(0x562427db61b0 msgr2=0x562427e5c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 -- 172.21.15.12:0/2548193379 shutdown_connections 2024-05-08T23:57:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 1 -- 172.21.15.12:0/2548193379 wait complete. 2024-05-08T23:57:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.743+0000 7f292948bb80 20 client.18645 trim_cache size 0 max 16384 2024-05-08T23:57:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.759+0000 7f292948bb80 1 Processor -- start 2024-05-08T23:57:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.759+0000 7f292948bb80 1 -- start start 2024-05-08T23:57:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.759+0000 7f292948bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e58480 0x562427e58860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.759+0000 7f292948bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e52910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.759+0000 7f292948bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e63db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.759+0000 7f292948bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427e540f0 con 0x562427e5d950 2024-05-08T23:57:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.759+0000 7f292948bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427e73700 con 0x562427e5e270 2024-05-08T23:57:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.759+0000 7f292948bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427e53930 con 0x562427e58480 2024-05-08T23:57:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f29167fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e52910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f29167fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e52910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58908/0 (socket says 172.21.15.12:58908) 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f29167fc700 1 -- 172.21.15.12:0/1858235131 learned_addr learned my addr 172.21.15.12:0/1858235131 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f29177fe700 1 --2- 172.21.15.12:0/1858235131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e63db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f2916ffd700 1 --2- 172.21.15.12:0/1858235131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e58480 0x562427e58860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f29167fc700 1 -- 172.21.15.12:0/1858235131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e58480 msgr2=0x562427e58860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f29167fc700 1 --2- 172.21.15.12:0/1858235131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e58480 0x562427e58860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f29167fc700 1 -- 172.21.15.12:0/1858235131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 msgr2=0x562427e63db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f29167fc700 1 --2- 172.21.15.12:0/1858235131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e63db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f29167fc700 1 -- 172.21.15.12:0/1858235131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562427d6e500 con 0x562427e5d950 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f29167fc700 1 --2- 172.21.15.12:0/1858235131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e52910 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f290c024a50 tx=0x7f290c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62412973bf0dc2ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f28fdffb700 1 -- 172.21.15.12:0/1858235131 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c024680 con 0x562427e5d950 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f28fdffb700 1 -- 172.21.15.12:0/1858235131 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f290c01e040 con 0x562427e5d950 2024-05-08T23:57:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.760+0000 7f28fdffb700 1 -- 172.21.15.12:0/1858235131 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c0048e0 con 0x562427e5d950 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 -- 172.21.15.12:0/1858235131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 msgr2=0x562427e52910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 --2- 172.21.15.12:0/1858235131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e52910 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f290c024a50 tx=0x7f290c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 -- 172.21.15.12:0/1858235131 shutdown_connections 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 --2- 172.21.15.12:0/1858235131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e63db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 --2- 172.21.15.12:0/1858235131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e58480 0x562427e58860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 --2- 172.21.15.12:0/1858235131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e52910 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 -- 172.21.15.12:0/1858235131 >> 172.21.15.12:0/1858235131 conn(0x562427db61b0 msgr2=0x562427e565d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 -- 172.21.15.12:0/1858235131 shutdown_connections 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 -- 172.21.15.12:0/1858235131 wait complete. 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 -- start start 2024-05-08T23:57:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.761+0000 7f292948bb80 1 Processor -- start 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f292948bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e58480 0x562427e6d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f292948bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e5b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f292948bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e5b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f292948bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427e5beb0 con 0x562427e5d950 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f292948bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427e5c020 con 0x562427e5e270 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f292948bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562427e5c1e0 con 0x562427e58480 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f292948bb80 10 client.0 _ll_register_callbacks cb 0x562427d4dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f29167fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e5b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f29167fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e5b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58914/0 (socket says 172.21.15.12:58914) 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f29167fc700 1 -- 172.21.15.12:0/729613689 learned_addr learned my addr 172.21.15.12:0/729613689 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f29177fe700 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e5b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f2916ffd700 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e58480 0x562427e6d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f29167fc700 1 -- 172.21.15.12:0/729613689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e58480 msgr2=0x562427e6d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f29167fc700 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e58480 0x562427e6d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f29167fc700 1 -- 172.21.15.12:0/729613689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 msgr2=0x562427e5b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f29167fc700 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e5b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.762+0000 7f29167fc700 1 -- 172.21.15.12:0/729613689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562427bd5b00 con 0x562427e5d950 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f29167fc700 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e5b3f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f290c024af0 tx=0x7f290c00b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845de12b4d5e6abf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f28faff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f28faff5700 1 -- 172.21.15.12:0/729613689 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c00bb80 con 0x562427e5d950 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f28faff5700 1 -- 172.21.15.12:0/729613689 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f290c000e40 con 0x562427e5d950 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f292948bb80 1 -- 172.21.15.12:0/729613689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29100996b0 con 0x562427e5d950 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f28faff5700 1 -- 172.21.15.12:0/729613689 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f290c00bce0 con 0x562427e5d950 2024-05-08T23:57:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f292948bb80 10 client.18651 fetch_fsmap learned FSMap version 48 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f292948bb80 1 -- 172.21.15.12:0/729613689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x562427c2dc70 con 0x562427e5d950 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f28faff5700 1 -- 172.21.15.12:0/729613689 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29100996b0 con 0x562427e5d950 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f28faff5700 1 -- 172.21.15.12:0/729613689 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 48) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f290c00f460 con 0x562427e5d950 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f292948bb80 10 client.18651 fetch_fsmap finished waiting for FSMap version 48 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f292948bb80 10 client.18651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f292948bb80 20 client.18651 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f292948bb80 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562427ecc870 0x562427ecec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f292948bb80 4 client.18651 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.763+0000 7f292948bb80 1 -- 172.21.15.12:0/729613689 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562427da0e60 con 0x562427ecc870 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.764+0000 7f2916ffd700 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562427ecc870 0x562427ecec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.764+0000 7f2916ffd700 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562427ecc870 0x562427ecec60 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522d589875730079 server_cookie=865d108cf6614520 in_seq=0 out_seq=0 2024-05-08T23:57:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.764+0000 7f28faff5700 10 client.18651 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f28faff5700 1 -- 172.21.15.12:0/729613689 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562427da0e60 con 0x562427ecc870 2024-05-08T23:57:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f28faff5700 10 client.18651 handle_command_reply: tid=0 2024-05-08T23:57:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f292948bb80 1 client.18651 shutdown 2024-05-08T23:57:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f292948bb80 1 -- 172.21.15.12:0/729613689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 msgr2=0x562427e5b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f292948bb80 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e5b3f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f290c024af0 tx=0x7f290c00b540 comp rx=0 tx=0).stop 2024-05-08T23:57:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f292948bb80 1 -- 172.21.15.12:0/729613689 shutdown_connections 2024-05-08T23:57:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f292948bb80 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562427ecc870 0x562427ecec60 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f292948bb80 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562427e5e270 0x562427e5b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f292948bb80 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562427e58480 0x562427e6d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f292948bb80 1 --2- 172.21.15.12:0/729613689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562427e5d950 0x562427e5b3f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.813+0000 7f292948bb80 1 -- 172.21.15.12:0/729613689 >> 172.21.15.12:0/729613689 conn(0x562427db61b0 msgr2=0x562427e57570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.814+0000 7f28faff5700 0 client.18651 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.815+0000 7f292948bb80 1 -- 172.21.15.12:0/729613689 shutdown_connections 2024-05-08T23:57:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.815+0000 7f292948bb80 1 -- 172.21.15.12:0/729613689 wait complete. 2024-05-08T23:57:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f292948bb80 20 client.18651 trim_cache size 0 max 16384 2024-05-08T23:57:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:57:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:57:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:57:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:57:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f291d2d1700 1 -- 172.21.15.12:0/2019048428 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29000a1290 msgr2=0x7f29000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f291d2d1700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29000a1290 0x7f29000a3750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2908005950 tx=0x7f290800b410 comp rx=0 tx=0).stop 2024-05-08T23:57:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f291d2d1700 1 -- 172.21.15.12:0/2019048428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f291819bcc0 msgr2=0x7f291819c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f291d2d1700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f291819bcc0 0x7f291819c140 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f2910007ee0 tx=0x7f291000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f291d2d1700 1 -- 172.21.15.12:0/2019048428 shutdown_connections 2024-05-08T23:57:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f291d2d1700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f29000a1290 0x7f29000a3750 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f291d2d1700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f291819bcc0 0x7f291819c140 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f291d2d1700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29181010e0 0x7f29181973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f291d2d1700 1 --2- 172.21.15.12:0/2019048428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2918106200 0x7f2918197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.816+0000 7f291d2d1700 1 -- 172.21.15.12:0/2019048428 >> 172.21.15.12:0/2019048428 conn(0x7f29180fc940 msgr2=0x7f29180fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.817+0000 7f291d2d1700 1 -- 172.21.15.12:0/2019048428 shutdown_connections 2024-05-08T23:57:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:44.817+0000 7f291d2d1700 1 -- 172.21.15.12:0/2019048428 wait complete. 2024-05-08T23:57:44.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 626768 6.25 MB/sec execute 460 sec latency 1847.690 ms 2024-05-08T23:57:45.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 626768 6.24 MB/sec execute 461 sec latency 2847.725 ms 2024-05-08T23:57:45.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:45 smithi012 ceph-mon[42503]: pgmap v360: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-05-08T23:57:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:45 smithi204 ceph-mon[47373]: pgmap v360: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-05-08T23:57:46.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:45 smithi160 ceph-mon[46152]: pgmap v360: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-05-08T23:57:46.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 628925 6.25 MB/sec execute 462 sec latency 2988.065 ms 2024-05-08T23:57:47.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 630236 6.25 MB/sec execute 463 sec latency 22.264 ms 2024-05-08T23:57:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:47 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:47 smithi012 ceph-mon[42503]: pgmap v361: 97 pgs: 97 active+clean; 187 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-05-08T23:57:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:47 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:47 smithi204 ceph-mon[47373]: pgmap v361: 97 pgs: 97 active+clean; 187 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-05-08T23:57:48.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:47 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:48.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:47 smithi160 ceph-mon[46152]: pgmap v361: 97 pgs: 97 active+clean; 187 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-05-08T23:57:48.891 INFO:tasks.workunit.client.0.smithi012.stdout: 1 630309 6.24 MB/sec execute 464 sec latency 932.494 ms 2024-05-08T23:57:49.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 630309 6.23 MB/sec execute 465 sec latency 1932.633 ms 2024-05-08T23:57:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:49 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:49 smithi012 ceph-mon[42503]: pgmap v362: 97 pgs: 97 active+clean; 187 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-05-08T23:57:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:49 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:49 smithi204 ceph-mon[47373]: pgmap v362: 97 pgs: 97 active+clean; 187 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-05-08T23:57:50.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:49 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:57:50.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:49 smithi160 ceph-mon[46152]: pgmap v362: 97 pgs: 97 active+clean; 187 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-05-08T23:57:50.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 630309 6.21 MB/sec execute 466 sec latency 2932.709 ms 2024-05-08T23:57:51.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 632426 6.23 MB/sec execute 467 sec latency 3073.358 ms 2024-05-08T23:57:51.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:51 smithi012 ceph-mon[42503]: pgmap v363: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-05-08T23:57:52.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:51 smithi204 ceph-mon[47373]: pgmap v363: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-05-08T23:57:52.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:51 smithi160 ceph-mon[46152]: pgmap v363: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-05-08T23:57:52.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 633834 6.23 MB/sec execute 468 sec latency 17.441 ms 2024-05-08T23:57:53.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 633858 6.21 MB/sec execute 469 sec latency 996.529 ms 2024-05-08T23:57:53.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:53 smithi012 ceph-mon[42503]: pgmap v364: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:57:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:53 smithi204 ceph-mon[47373]: pgmap v364: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:57:54.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:53 smithi160 ceph-mon[46152]: pgmap v364: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:57:54.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 633858 6.20 MB/sec execute 470 sec latency 1996.648 ms 2024-05-08T23:57:55.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 633858 6.19 MB/sec execute 471 sec latency 2996.771 ms 2024-05-08T23:57:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:55 smithi012 ceph-mon[42503]: pgmap v365: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:57:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:55 smithi204 ceph-mon[47373]: pgmap v365: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:57:56.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:55 smithi160 ceph-mon[46152]: pgmap v365: 97 pgs: 97 active+clean; 192 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-08T23:57:56.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 635948 6.20 MB/sec execute 472 sec latency 3141.536 ms 2024-05-08T23:57:57.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 637268 6.20 MB/sec execute 473 sec latency 36.655 ms 2024-05-08T23:57:57.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:57 smithi012 ceph-mon[42503]: pgmap v366: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.944+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2514499859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cc8100760 msgr2=0x7f3cc810cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.945+0000 7f3ccf08a700 1 --2- 172.21.15.12:0/2514499859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cc8100760 0x7f3cc810cc00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009f90 tx=0x7f3cc000b330 comp rx=0 tx=0).stop 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.945+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2514499859 shutdown_connections 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.945+0000 7f3ccf08a700 1 --2- 172.21.15.12:0/2514499859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cc8100760 0x7f3cc810cc00 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.945+0000 7f3ccf08a700 1 --2- 172.21.15.12:0/2514499859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cc8105e60 0x7f3cc8106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.945+0000 7f3ccf08a700 1 --2- 172.21.15.12:0/2514499859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc80ffda0 0x7f3cc8100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.945+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2514499859 >> 172.21.15.12:0/2514499859 conn(0x7f3cc8076f10 msgr2=0x7f3cc8077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.945+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2514499859 shutdown_connections 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.945+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2514499859 wait complete. 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.945+0000 7f3ccf08a700 1 Processor -- start 2024-05-08T23:57:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.945+0000 7f3ccf08a700 1 -- start start 2024-05-08T23:57:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccf08a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cc80ffda0 0x7f3cc8197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccf08a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cc8100760 0x7f3cc8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccf08a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc8105e60 0x7f3cc819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccf08a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc819c340 con 0x7f3cc80ffda0 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccf08a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc819c4b0 con 0x7f3cc8100760 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccf08a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc819c670 con 0x7f3cc8105e60 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cc80ffda0 0x7f3cc8197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cc80ffda0 0x7f3cc8197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39938/0 (socket says 172.21.15.12:39938) 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccce26700 1 -- 172.21.15.12:0/2210566989 learned_addr learned my addr 172.21.15.12:0/2210566989 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cc8100760 0x7f3cc8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccd627700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc8105e60 0x7f3cc819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccce26700 1 -- 172.21.15.12:0/2210566989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc8105e60 msgr2=0x7f3cc819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccce26700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc8105e60 0x7f3cc819bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccce26700 1 -- 172.21.15.12:0/2210566989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cc8100760 msgr2=0x7f3cc8197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccce26700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cc8100760 0x7f3cc8197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccce26700 1 -- 172.21.15.12:0/2210566989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb00095f0 con 0x7f3cc80ffda0 2024-05-08T23:57:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.946+0000 7f3ccce26700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cc80ffda0 0x7f3cc8197420 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000ec10 tx=0x7f3cb000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.947+0000 7f3cc5ffb700 1 -- 172.21.15.12:0/2210566989 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000cc90 con 0x7f3cc80ffda0 2024-05-08T23:57:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.947+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2210566989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc0009c40 con 0x7f3cc80ffda0 2024-05-08T23:57:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.947+0000 7f3cc5ffb700 1 -- 172.21.15.12:0/2210566989 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cb00044e0 con 0x7f3cc80ffda0 2024-05-08T23:57:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.947+0000 7f3cc5ffb700 1 -- 172.21.15.12:0/2210566989 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0010430 con 0x7f3cc80ffda0 2024-05-08T23:57:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.947+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2210566989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc81a3fb0 con 0x7f3cc80ffda0 2024-05-08T23:57:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.949+0000 7f3cc5ffb700 1 -- 172.21.15.12:0/2210566989 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3cb0003fb0 con 0x7f3cc80ffda0 2024-05-08T23:57:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.950+0000 7f3cc5ffb700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3cb40a1290 0x7f3cb40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.950+0000 7f3cc5ffb700 1 -- 172.21.15.12:0/2210566989 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3cb0014070 con 0x7f3cc80ffda0 2024-05-08T23:57:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.950+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3cb40a1290 0x7f3cb40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.950+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3cb40a1290 0x7f3cb40a3750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8198970 tx=0x7f3cb8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3cdb244b80 1 Processor -- start 2024-05-08T23:57:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3cdb244b80 1 -- start start 2024-05-08T23:57:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdef950 0x558c0fdefd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdf0270 0x558c0fde9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdea480 0x558c0fdea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fd410f0 con 0x558c0fdef950 2024-05-08T23:57:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fd3eab0 con 0x558c0fdea480 2024-05-08T23:57:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fd3a3a0 con 0x558c0fdf0270 2024-05-08T23:57:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3ccce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdef950 0x558c0fdefd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3ccce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdef950 0x558c0fdefd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39940/0 (socket says 172.21.15.12:39940) 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3ccce26700 1 -- 172.21.15.12:0/1643500780 learned_addr learned my addr 172.21.15.12:0/1643500780 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.951+0000 7f3ccd627700 1 --2- 172.21.15.12:0/1643500780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdea480 0x558c0fdea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/1643500780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdf0270 0x558c0fde9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3ccce26700 1 -- 172.21.15.12:0/1643500780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdf0270 msgr2=0x558c0fde9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3ccce26700 1 --2- 172.21.15.12:0/1643500780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdf0270 0x558c0fde9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3ccce26700 1 -- 172.21.15.12:0/1643500780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdea480 msgr2=0x558c0fdea8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3ccce26700 1 --2- 172.21.15.12:0/1643500780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdea480 0x558c0fdea8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3ccce26700 1 -- 172.21.15.12:0/1643500780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c0fbbb350 con 0x558c0fdef950 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3ccd627700 1 --2- 172.21.15.12:0/1643500780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdea480 0x558c0fdea8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3ccce26700 1 --2- 172.21.15.12:0/1643500780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdef950 0x558c0fdefd30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00a2280 tx=0x7f3cb009a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cce9dd46eabd47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3cbeffd700 1 -- 172.21.15.12:0/1643500780 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0016330 con 0x558c0fdef950 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3cbeffd700 1 -- 172.21.15.12:0/1643500780 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cb009aa30 con 0x558c0fdef950 2024-05-08T23:57:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.952+0000 7f3cbeffd700 1 -- 172.21.15.12:0/1643500780 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0096a40 con 0x558c0fdef950 2024-05-08T23:57:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.953+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1643500780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdef950 msgr2=0x558c0fdefd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.953+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/1643500780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdef950 0x558c0fdefd30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00a2280 tx=0x7f3cb009a180 comp rx=0 tx=0).stop 2024-05-08T23:57:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.953+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1643500780 shutdown_connections 2024-05-08T23:57:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.953+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/1643500780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdea480 0x558c0fdea8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.953+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/1643500780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdef950 0x558c0fdefd30 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.953+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/1643500780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdf0270 0x558c0fde9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.953+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1643500780 >> 172.21.15.12:0/1643500780 conn(0x558c0fd481b0 msgr2=0x558c0fde4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.953+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1643500780 shutdown_connections 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.953+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1643500780 wait complete. 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.953+0000 7f3cdb244b80 1 -- start start 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3cdb244b80 1 Processor -- start 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdf3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdf3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdf0270 0x558c0fdf4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fcd6500 con 0x558c0fdea480 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fcd61a0 con 0x558c0fdf0270 2024-05-08T23:57:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fbbd640 con 0x558c0fdef950 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3cdb244b80 10 client.0 _ll_register_callbacks cb 0x558c0fcdfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3ccce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdf3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3cc7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdf3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3ccce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdf3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39946/0 (socket says 172.21.15.12:39946) 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.954+0000 7f3cc7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdf3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53480/0 (socket says 172.21.15.12:53480) 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3ccce26700 1 -- 172.21.15.12:0/1389515523 learned_addr learned my addr 172.21.15.12:0/1389515523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3ccd627700 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdf0270 0x558c0fdf4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3cc7fff700 1 -- 172.21.15.12:0/1389515523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdf0270 msgr2=0x558c0fdf4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdf0270 0x558c0fdf4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3cc7fff700 1 -- 172.21.15.12:0/1389515523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 msgr2=0x558c0fdf3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdf3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3cc7fff700 1 -- 172.21.15.12:0/1389515523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc810d4f0 con 0x558c0fdef950 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3ccce26700 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdf3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdf3ad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097c0 tx=0x7f3cb80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=300a86c346037f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3ca7fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3ca7fff700 1 -- 172.21.15.12:0/1389515523 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8022070 con 0x558c0fdef950 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1389515523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c0fc93210 con 0x558c0fdef950 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3ca7fff700 1 -- 172.21.15.12:0/1389515523 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3cb8004460 con 0x558c0fdef950 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3ca7fff700 1 -- 172.21.15.12:0/1389515523 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cb8003d40 con 0x558c0fdef950 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3cdb244b80 10 client.28381 fetch_fsmap learned FSMap version 50 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.955+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1389515523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f3cc81a3fb0 con 0x558c0fdef950 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.956+0000 7f3ca7fff700 1 -- 172.21.15.12:0/1389515523 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558c0fc93210 con 0x558c0fdef950 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.956+0000 7f3ca7fff700 1 -- 172.21.15.12:0/1389515523 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3cb801d410 con 0x558c0fdef950 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.956+0000 7f3cdb244b80 10 client.28381 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:57:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.956+0000 7f3cdb244b80 10 client.28381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.956+0000 7f3cdb244b80 20 client.28381 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.956+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c0fe03c70 0x558c0fe06090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.956+0000 7f3cdb244b80 4 client.28381 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.956+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1389515523 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c0fdb97b0 con 0x558c0fe03c70 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.956+0000 7f3ccce26700 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c0fe03c70 0x558c0fe06090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3ca7fff700 10 client.28381 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3ccce26700 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c0fe03c70 0x558c0fe06090 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad0077c93f7139bf server_cookie=18d63b674e537d8c in_seq=0 out_seq=0 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3ca7fff700 1 -- 172.21.15.12:0/1389515523 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c0fdb97b0 con 0x558c0fe03c70 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3ca7fff700 10 client.28381 handle_command_reply: tid=0 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3cdb244b80 1 client.28381 shutdown 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1389515523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 msgr2=0x558c0fdf3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdf3ad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097c0 tx=0x7f3cb80047c0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1389515523 shutdown_connections 2024-05-08T23:57:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c0fe03c70 0x558c0fe06090 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdf0270 0x558c0fdf4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdf3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/1389515523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdf3ad0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3ca7fff700 0 client.28381 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.957+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1389515523 >> 172.21.15.12:0/1389515523 conn(0x558c0fd481b0 msgr2=0x558c0fde8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.958+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1389515523 shutdown_connections 2024-05-08T23:57:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.958+0000 7f3cdb244b80 1 -- 172.21.15.12:0/1389515523 wait complete. 2024-05-08T23:57:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.958+0000 7f3cdb244b80 20 client.28381 trim_cache size 0 max 16384 2024-05-08T23:57:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.974+0000 7f3cdb244b80 1 Processor -- start 2024-05-08T23:57:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.974+0000 7f3cdb244b80 1 -- start start 2024-05-08T23:57:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdedc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdf0270 0x558c0fde4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fdf4b90 con 0x558c0fdea480 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fdf6720 con 0x558c0fdf0270 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fdf48e0 con 0x558c0fdef950 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cc7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdedc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3ccce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cc7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdedc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53484/0 (socket says 172.21.15.12:53484) 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cc7fff700 1 -- 172.21.15.12:0/3545425086 learned_addr learned my addr 172.21.15.12:0/3545425086 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3ccd627700 1 --2- 172.21.15.12:0/3545425086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdf0270 0x558c0fde4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cc7fff700 1 -- 172.21.15.12:0/3545425086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdf0270 msgr2=0x558c0fde4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/3545425086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdf0270 0x558c0fde4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cc7fff700 1 -- 172.21.15.12:0/3545425086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 msgr2=0x558c0fdffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/3545425086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cc7fff700 1 -- 172.21.15.12:0/3545425086 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c0fb67b00 con 0x558c0fdef950 2024-05-08T23:57:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.975+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/3545425086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdedc10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8013010 tx=0x7f3cb8009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=792347c211ecc894 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.976+0000 7f3cbf7fe700 1 -- 172.21.15.12:0/3545425086 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8022070 con 0x558c0fdef950 2024-05-08T23:57:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.976+0000 7f3cbf7fe700 1 -- 172.21.15.12:0/3545425086 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cb801d510 con 0x558c0fdef950 2024-05-08T23:57:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.976+0000 7f3cbf7fe700 1 -- 172.21.15.12:0/3545425086 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80278e0 con 0x558c0fdef950 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.976+0000 7f3cdb244b80 1 -- 172.21.15.12:0/3545425086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 msgr2=0x558c0fdedc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.976+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/3545425086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdedc10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8013010 tx=0x7f3cb8009450 comp rx=0 tx=0).stop 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.976+0000 7f3cdb244b80 1 -- 172.21.15.12:0/3545425086 shutdown_connections 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.976+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/3545425086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdf0270 0x558c0fde4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.976+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/3545425086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.976+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/3545425086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdef950 0x558c0fdedc10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.976+0000 7f3cdb244b80 1 -- 172.21.15.12:0/3545425086 >> 172.21.15.12:0/3545425086 conn(0x558c0fd481b0 msgr2=0x558c0fdf1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.977+0000 7f3cdb244b80 1 -- 172.21.15.12:0/3545425086 shutdown_connections 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.977+0000 7f3cdb244b80 1 -- 172.21.15.12:0/3545425086 wait complete. 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.977+0000 7f3cdb244b80 1 -- start start 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.977+0000 7f3cdb244b80 1 Processor -- start 2024-05-08T23:57:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdfadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdef950 0x558c0fdf8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3cdb244b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdf0270 0x558c0fdf9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fdf9640 con 0x558c0fdea480 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fdf97b0 con 0x558c0fdef950 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3cdb244b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c0fdf9970 con 0x558c0fdf0270 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3cdb244b80 10 client.0 _ll_register_callbacks cb 0x558c0fcdfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ccce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdfadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ccce26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdfadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39970/0 (socket says 172.21.15.12:39970) 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ccce26700 1 -- 172.21.15.12:0/720906469 learned_addr learned my addr 172.21.15.12:0/720906469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ccd627700 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdf0270 0x558c0fdf9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdef950 0x558c0fdf8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ccce26700 1 -- 172.21.15.12:0/720906469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdf0270 msgr2=0x558c0fdf9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ccce26700 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdf0270 0x558c0fdf9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ccce26700 1 -- 172.21.15.12:0/720906469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdef950 msgr2=0x558c0fdf8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:57:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ccce26700 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdef950 0x558c0fdf8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ccce26700 1 -- 172.21.15.12:0/720906469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc81a3fb0 con 0x558c0fdea480 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ccce26700 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdfadd0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005d70 tx=0x7f3cb0085120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e839fce2cd2ec150 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.978+0000 7f3ca7fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3ca7fff700 1 -- 172.21.15.12:0/720906469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0009a10 con 0x558c0fdea480 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3ca7fff700 1 -- 172.21.15.12:0/720906469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3cb0016330 con 0x558c0fdea480 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3cdb244b80 1 -- 172.21.15.12:0/720906469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cb009aa30 con 0x558c0fdea480 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3ca7fff700 1 -- 172.21.15.12:0/720906469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cb009a330 con 0x558c0fdea480 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3cdb244b80 10 client.18669 fetch_fsmap learned FSMap version 50 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3cdb244b80 1 -- 172.21.15.12:0/720906469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x558c0fd00500 con 0x558c0fdea480 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3ca7fff700 1 -- 172.21.15.12:0/720906469 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cb009aa30 con 0x558c0fdea480 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3ca7fff700 1 -- 172.21.15.12:0/720906469 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3cb009a6b0 con 0x558c0fdea480 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3cdb244b80 10 client.18669 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:57:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3cdb244b80 10 client.18669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:57:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3cdb244b80 20 client.18669 populate_metadata read hostname 'smithi012' 2024-05-08T23:57:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c0fe5a2e0 0x558c0fe5c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:57:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3cdb244b80 4 client.18669 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:57:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3cdb244b80 1 -- 172.21.15.12:0/720906469 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558c0fd32e60 con 0x558c0fe5a2e0 2024-05-08T23:57:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.979+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c0fe5a2e0 0x558c0fe5c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:57:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.980+0000 7f3ca7fff700 10 client.18669 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:57.980+0000 7f3cc7fff700 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c0fe5a2e0 0x558c0fe5c700 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ce9ae446ff81d3 server_cookie=479e8b86d53b7870 in_seq=0 out_seq=0 2024-05-08T23:57:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3ca7fff700 1 -- 172.21.15.12:0/720906469 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558c0fd32e60 con 0x558c0fe5a2e0 2024-05-08T23:57:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3ca7fff700 10 client.18669 handle_command_reply: tid=0 2024-05-08T23:57:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 client.18669 shutdown 2024-05-08T23:57:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 -- 172.21.15.12:0/720906469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 msgr2=0x558c0fdfadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdfadd0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005d70 tx=0x7f3cb0085120 comp rx=0 tx=0).stop 2024-05-08T23:57:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 -- 172.21.15.12:0/720906469 shutdown_connections 2024-05-08T23:57:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c0fdf0270 0x558c0fdf9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c0fdea480 0x558c0fdfadd0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c0fe5a2e0 0x558c0fe5c700 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 --2- 172.21.15.12:0/720906469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c0fdef950 0x558c0fdf8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3ca7fff700 0 client.18669 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:57:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 -- 172.21.15.12:0/720906469 >> 172.21.15.12:0/720906469 conn(0x558c0fd481b0 msgr2=0x558c0fdee5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 -- 172.21.15.12:0/720906469 shutdown_connections 2024-05-08T23:57:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 1 -- 172.21.15.12:0/720906469 wait complete. 2024-05-08T23:57:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.015+0000 7f3cdb244b80 20 client.18669 trim_cache size 0 max 16384 2024-05-08T23:57:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:57:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:57:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:57:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:57:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2210566989 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3cb40a1290 msgr2=0x7f3cb40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3cb40a1290 0x7f3cb40a3750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8198970 tx=0x7f3cb8009260 comp rx=0 tx=0).stop 2024-05-08T23:57:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2210566989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cc80ffda0 msgr2=0x7f3cc8197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:57:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cc80ffda0 0x7f3cc8197420 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000ec10 tx=0x7f3cb000ef20 comp rx=0 tx=0).stop 2024-05-08T23:57:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2210566989 shutdown_connections 2024-05-08T23:57:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3cb40a1290 0x7f3cb40a3750 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc8105e60 0x7f3cc819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cc80ffda0 0x7f3cc8197420 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 --2- 172.21.15.12:0/2210566989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cc8100760 0x7f3cc8197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:57:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2210566989 >> 172.21.15.12:0/2210566989 conn(0x7f3cc8076f10 msgr2=0x7f3cc80685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:57:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2210566989 shutdown_connections 2024-05-08T23:57:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:57:58.016+0000 7f3ccf08a700 1 -- 172.21.15.12:0/2210566989 wait complete. 2024-05-08T23:57:58.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:57 smithi204 ceph-mon[47373]: pgmap v366: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-05-08T23:57:58.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:57 smithi160 ceph-mon[46152]: pgmap v366: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-05-08T23:57:58.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 637402 6.19 MB/sec execute 474 sec latency 928.538 ms 2024-05-08T23:57:59.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 637402 6.17 MB/sec execute 475 sec latency 1928.645 ms 2024-05-08T23:57:59.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:57:59 smithi012 ceph-mon[42503]: pgmap v367: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-05-08T23:58:00.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:57:59 smithi204 ceph-mon[47373]: pgmap v367: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-05-08T23:58:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:57:59 smithi160 ceph-mon[46152]: pgmap v367: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-05-08T23:58:00.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 637402 6.16 MB/sec execute 476 sec latency 2928.698 ms 2024-05-08T23:58:01.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 638681 6.16 MB/sec execute 477 sec latency 3067.087 ms 2024-05-08T23:58:01.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:01 smithi012 ceph-mon[42503]: pgmap v368: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-05-08T23:58:01.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:58:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:01 smithi204 ceph-mon[47373]: pgmap v368: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-05-08T23:58:02.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:01 smithi160 ceph-mon[46152]: pgmap v368: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-05-08T23:58:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.112+0000 7f4d9c488700 1 -- 172.21.15.12:0/2007429911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d94068050 msgr2=0x7f4d94068430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.112+0000 7f4d9c488700 1 --2- 172.21.15.12:0/2007429911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d94068050 0x7f4d94068430 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009990 tx=0x7f4d90009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.112+0000 7f4d9c488700 1 -- 172.21.15.12:0/2007429911 shutdown_connections 2024-05-08T23:58:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.112+0000 7f4d9c488700 1 --2- 172.21.15.12:0/2007429911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d9410d940 0x7f4d9410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.112+0000 7f4d9c488700 1 --2- 172.21.15.12:0/2007429911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d94068050 0x7f4d94068430 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.112+0000 7f4d9c488700 1 --2- 172.21.15.12:0/2007429911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d94068970 0x7f4d9410d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.112+0000 7f4d9c488700 1 -- 172.21.15.12:0/2007429911 >> 172.21.15.12:0/2007429911 conn(0x7f4d94076f40 msgr2=0x7f4d94077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.112+0000 7f4d9c488700 1 -- 172.21.15.12:0/2007429911 shutdown_connections 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.112+0000 7f4d9c488700 1 -- 172.21.15.12:0/2007429911 wait complete. 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d9c488700 1 Processor -- start 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d9c488700 1 -- start start 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d9c488700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d94068050 0x7f4d94199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d9c488700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d94068970 0x7f4d94199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d9c488700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d9410d940 0x7f4d9419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d9c488700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9419e520 con 0x7f4d94068050 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d9c488700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9419e690 con 0x7f4d94068970 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d9c488700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9419e850 con 0x7f4d9410d940 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d9a224700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d94068050 0x7f4d94199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d99a23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d94068970 0x7f4d94199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d9aa25700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d9410d940 0x7f4d9419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d99a23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d94068970 0x7f4d94199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48748/0 (socket says 172.21.15.12:48748) 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.113+0000 7f4d99a23700 1 -- 172.21.15.12:0/2127121581 learned_addr learned my addr 172.21.15.12:0/2127121581 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d99a23700 1 -- 172.21.15.12:0/2127121581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d9410d940 msgr2=0x7f4d9419dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d99a23700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d9410d940 0x7f4d9419dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d99a23700 1 -- 172.21.15.12:0/2127121581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d94068050 msgr2=0x7f4d94199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d99a23700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d94068050 0x7f4d94199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d99a23700 1 -- 172.21.15.12:0/2127121581 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d900095f0 con 0x7f4d94068970 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d9a224700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d94068050 0x7f4d94199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d99a23700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d94068970 0x7f4d94199b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400e9a0 tx=0x7f4d8400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d8b7fe700 1 -- 172.21.15.12:0/2127121581 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d84009790 con 0x7f4d94068970 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d8b7fe700 1 -- 172.21.15.12:0/2127121581 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d84010430 con 0x7f4d94068970 2024-05-08T23:58:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d9c488700 1 -- 172.21.15.12:0/2127121581 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9419eb30 con 0x7f4d94068970 2024-05-08T23:58:02.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d8b7fe700 1 -- 172.21.15.12:0/2127121581 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d84018680 con 0x7f4d94068970 2024-05-08T23:58:02.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.114+0000 7f4d9c488700 1 -- 172.21.15.12:0/2127121581 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d941a6370 con 0x7f4d94068970 2024-05-08T23:58:02.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.117+0000 7f4d8b7fe700 1 -- 172.21.15.12:0/2127121581 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4d840105a0 con 0x7f4d94068970 2024-05-08T23:58:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.117+0000 7f4d8b7fe700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4d800a1290 0x7f4d800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.117+0000 7f4d8b7fe700 1 -- 172.21.15.12:0/2127121581 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4d84014070 con 0x7f4d94068970 2024-05-08T23:58:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.117+0000 7f4d9a224700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4d800a1290 0x7f4d800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.118+0000 7f4d9a224700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4d800a1290 0x7f4d800a3750 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000b2e0 tx=0x7f4d90005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.118+0000 7f4da8642b80 1 Processor -- start 2024-05-08T23:58:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.118+0000 7f4da8642b80 1 -- start start 2024-05-08T23:58:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 0x55f46478a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f46478a720 0x55f464784350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f464784cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4da8642b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4646db150 con 0x55f464784890 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4da8642b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4646d8b10 con 0x55f46478a720 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4da8642b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4646d4400 con 0x55f464789e00 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9aa25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f464784cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9a224700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 0x55f46478a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d99a23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f46478a720 0x55f464784350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9a224700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 0x55f46478a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46122/0 (socket says 172.21.15.12:46122) 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d99a23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f46478a720 0x55f464784350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48750/0 (socket says 172.21.15.12:48750) 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9a224700 1 -- 172.21.15.12:0/2198541055 learned_addr learned my addr 172.21.15.12:0/2198541055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9aa25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f464784cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46060/0 (socket says 172.21.15.12:46060) 2024-05-08T23:58:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9aa25700 1 -- 172.21.15.12:0/2198541055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 msgr2=0x55f46478a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9aa25700 1 --2- 172.21.15.12:0/2198541055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 0x55f46478a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9aa25700 1 -- 172.21.15.12:0/2198541055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f46478a720 msgr2=0x55f464784350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9aa25700 1 --2- 172.21.15.12:0/2198541055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f46478a720 0x55f464784350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9aa25700 1 -- 172.21.15.12:0/2198541055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4645553b0 con 0x55f464784890 2024-05-08T23:58:02.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d99a23700 1 --2- 172.21.15.12:0/2198541055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f46478a720 0x55f464784350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.119+0000 7f4d9aa25700 1 --2- 172.21.15.12:0/2198541055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f464784cd0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c011550 tx=0x7f4d8c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed57e72a96b75760 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.120+0000 7f4d88da4700 1 -- 172.21.15.12:0/2198541055 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00e040 con 0x55f464784890 2024-05-08T23:58:02.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.120+0000 7f4d88da4700 1 -- 172.21.15.12:0/2198541055 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d8c00f040 con 0x55f464784890 2024-05-08T23:58:02.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.120+0000 7f4d88da4700 1 -- 172.21.15.12:0/2198541055 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c004320 con 0x55f464784890 2024-05-08T23:58:02.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.120+0000 7f4da8642b80 1 -- 172.21.15.12:0/2198541055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 msgr2=0x55f464784cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.120+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2198541055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f464784cd0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c011550 tx=0x7f4d8c009920 comp rx=0 tx=0).stop 2024-05-08T23:58:02.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.120+0000 7f4da8642b80 1 -- 172.21.15.12:0/2198541055 shutdown_connections 2024-05-08T23:58:02.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.120+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2198541055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f464784cd0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.120+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2198541055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 0x55f46478a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.120+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2198541055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f46478a720 0x55f464784350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.120+0000 7f4da8642b80 1 -- 172.21.15.12:0/2198541055 >> 172.21.15.12:0/2198541055 conn(0x55f4646e2210 msgr2=0x55f46477edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 1 -- 172.21.15.12:0/2198541055 shutdown_connections 2024-05-08T23:58:02.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 1 -- 172.21.15.12:0/2198541055 wait complete. 2024-05-08T23:58:02.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 1 -- start start 2024-05-08T23:58:02.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 1 Processor -- start 2024-05-08T23:58:02.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f46478dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f46478e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f46478a720 0x55f46478e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f464670560 con 0x55f464784890 2024-05-08T23:58:02.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f464670200 con 0x55f464789e00 2024-05-08T23:58:02.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4645576a0 con 0x55f46478a720 2024-05-08T23:58:02.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4da8642b80 10 client.0 _ll_register_callbacks cb 0x55f464679b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:02.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.121+0000 7f4d9a224700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f46478dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d99a23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f46478e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d9aa25700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f46478a720 0x55f46478e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d99a23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f46478e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48756/0 (socket says 172.21.15.12:48756) 2024-05-08T23:58:02.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d99a23700 1 -- 172.21.15.12:0/1031254563 learned_addr learned my addr 172.21.15.12:0/1031254563 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d9a224700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f46478dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46062/0 (socket says 172.21.15.12:46062) 2024-05-08T23:58:02.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d99a23700 1 -- 172.21.15.12:0/1031254563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f46478a720 msgr2=0x55f46478e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d99a23700 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f46478a720 0x55f46478e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d99a23700 1 -- 172.21.15.12:0/1031254563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 msgr2=0x55f46478dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d99a23700 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f46478dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d99a23700 1 -- 172.21.15.12:0/1031254563 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d941a6370 con 0x55f464789e00 2024-05-08T23:58:02.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d9a224700 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f46478dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d99a23700 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f46478e010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d84099cc0 tx=0x7f4d8409a740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af69468713883924 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:58:02.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d7dffb700 1 -- 172.21.15.12:0/1031254563 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d840996b0 con 0x55f464789e00 2024-05-08T23:58:02.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4da8642b80 1 -- 172.21.15.12:0/1031254563 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f46462d270 con 0x55f464789e00 2024-05-08T23:58:02.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d7dffb700 1 -- 172.21.15.12:0/1031254563 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4d8409ddf0 con 0x55f464789e00 2024-05-08T23:58:02.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4d7dffb700 1 -- 172.21.15.12:0/1031254563 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d840a2440 con 0x55f464789e00 2024-05-08T23:58:02.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4da8642b80 10 client.28412 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:02.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.122+0000 7f4da8642b80 1 -- 172.21.15.12:0/1031254563 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f4d8c011230 con 0x55f464789e00 2024-05-08T23:58:02.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4d7dffb700 1 -- 172.21.15.12:0/1031254563 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f46462d270 con 0x55f464789e00 2024-05-08T23:58:02.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4d7dffb700 1 -- 172.21.15.12:0/1031254563 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4d840a1fb0 con 0x55f464789e00 2024-05-08T23:58:02.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4da8642b80 10 client.28412 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:02.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4da8642b80 10 client.28412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:02.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4da8642b80 20 client.28412 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:02.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4da8642b80 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f46479e170 0x55f4647a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4da8642b80 4 client.28412 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:02.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4da8642b80 1 -- 172.21.15.12:0/1031254563 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f464753810 con 0x55f46479e170 2024-05-08T23:58:02.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4d9a224700 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f46479e170 0x55f4647a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4d9a224700 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f46479e170 0x55f4647a0590 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4be72f4f3e81ef server_cookie=5fd52aba487d324f in_seq=0 out_seq=0 2024-05-08T23:58:02.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.123+0000 7f4d7dffb700 10 client.28412 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:02.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4d7dffb700 1 -- 172.21.15.12:0/1031254563 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f464753810 con 0x55f46479e170 2024-05-08T23:58:02.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4d7dffb700 10 client.28412 handle_command_reply: tid=0 2024-05-08T23:58:02.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 client.28412 shutdown 2024-05-08T23:58:02.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 -- 172.21.15.12:0/1031254563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 msgr2=0x55f46478e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f46478e010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d84099cc0 tx=0x7f4d8409a740 comp rx=0 tx=0).stop 2024-05-08T23:58:02.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 -- 172.21.15.12:0/1031254563 shutdown_connections 2024-05-08T23:58:02.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f46478a720 0x55f46478e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f464784890 0x55f46478dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f46479e170 0x55f4647a0590 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 --2- 172.21.15.12:0/1031254563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f46478e010 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4d7dffb700 0 client.28412 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:02.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 -- 172.21.15.12:0/1031254563 >> 172.21.15.12:0/1031254563 conn(0x55f4646e2210 msgr2=0x55f46477e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 -- 172.21.15.12:0/1031254563 shutdown_connections 2024-05-08T23:58:02.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.124+0000 7f4da8642b80 1 -- 172.21.15.12:0/1031254563 wait complete. 2024-05-08T23:58:02.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.125+0000 7f4da8642b80 20 client.28412 trim_cache size 0 max 16384 2024-05-08T23:58:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4da8642b80 1 Processor -- start 2024-05-08T23:58:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4da8642b80 1 -- start start 2024-05-08T23:58:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464784890 0x55f464784350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 0x55f46477ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464789280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4da8642b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46478f0f0 con 0x55f46478a720 2024-05-08T23:58:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4da8642b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46478a580 con 0x55f464784890 2024-05-08T23:58:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4da8642b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4647a57d0 con 0x55f464789e00 2024-05-08T23:58:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4d9a224700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464784890 0x55f464784350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4d9aa25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464789280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4d9a224700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464784890 0x55f464784350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48772/0 (socket says 172.21.15.12:48772) 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4d9a224700 1 -- 172.21.15.12:0/2306176732 learned_addr learned my addr 172.21.15.12:0/2306176732 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4d9aa25700 1 --2- 172.21.15.12:0/2306176732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464789280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46078/0 (socket says 172.21.15.12:46078) 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.143+0000 7f4d99a23700 1 --2- 172.21.15.12:0/2306176732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 0x55f46477ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.144+0000 7f4d9a224700 1 -- 172.21.15.12:0/2306176732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 msgr2=0x55f46477ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.144+0000 7f4d9a224700 1 --2- 172.21.15.12:0/2306176732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 0x55f46477ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.144+0000 7f4d9a224700 1 -- 172.21.15.12:0/2306176732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 msgr2=0x55f464789280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.144+0000 7f4d9a224700 1 --2- 172.21.15.12:0/2306176732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464789280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.144+0000 7f4d9a224700 1 -- 172.21.15.12:0/2306176732 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f464784f40 con 0x55f464784890 2024-05-08T23:58:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.144+0000 7f4d9aa25700 1 --2- 172.21.15.12:0/2306176732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464789280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.144+0000 7f4d9a224700 1 --2- 172.21.15.12:0/2306176732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464784890 0x55f464784350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d900095c0 tx=0x7f4d90020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7b5a9ad105a633b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.144+0000 7f4d895a5700 1 -- 172.21.15.12:0/2306176732 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9002b040 con 0x55f464784890 2024-05-08T23:58:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.144+0000 7f4d895a5700 1 -- 172.21.15.12:0/2306176732 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d9002c040 con 0x55f464784890 2024-05-08T23:58:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.144+0000 7f4d895a5700 1 -- 172.21.15.12:0/2306176732 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90004760 con 0x55f464784890 2024-05-08T23:58:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 -- 172.21.15.12:0/2306176732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464784890 msgr2=0x55f464784350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2306176732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464784890 0x55f464784350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d900095c0 tx=0x7f4d90020000 comp rx=0 tx=0).stop 2024-05-08T23:58:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 -- 172.21.15.12:0/2306176732 shutdown_connections 2024-05-08T23:58:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2306176732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464789280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2306176732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464784890 0x55f464784350 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2306176732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464789e00 0x55f46477ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 -- 172.21.15.12:0/2306176732 >> 172.21.15.12:0/2306176732 conn(0x55f4646e2210 msgr2=0x55f464788b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 -- 172.21.15.12:0/2306176732 shutdown_connections 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 -- 172.21.15.12:0/2306176732 wait complete. 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 -- start start 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 Processor -- start 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464784890 0x55f464795190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f464791740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464791c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4647921c0 con 0x55f46478a720 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.145+0000 7f4da8642b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f464792330 con 0x55f464789e00 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4da8642b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4647924f0 con 0x55f464784890 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4da8642b80 10 client.0 _ll_register_callbacks cb 0x55f464679b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9aa25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464791c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9a224700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464784890 0x55f464795190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d99a23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f464791740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9aa25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464791c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46090/0 (socket says 172.21.15.12:46090) 2024-05-08T23:58:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9a224700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464784890 0x55f464795190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46146/0 (socket says 172.21.15.12:46146) 2024-05-08T23:58:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9aa25700 1 -- 172.21.15.12:0/2909266604 learned_addr learned my addr 172.21.15.12:0/2909266604 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9aa25700 1 -- 172.21.15.12:0/2909266604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464784890 msgr2=0x55f464795190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9aa25700 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464784890 0x55f464795190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9aa25700 1 -- 172.21.15.12:0/2909266604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 msgr2=0x55f464791740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9aa25700 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f464791740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9aa25700 1 -- 172.21.15.12:0/2909266604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4645553b0 con 0x55f46478a720 2024-05-08T23:58:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d99a23700 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f464791740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d9aa25700 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464791c80 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00b010 tx=0x7f4d8c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e41dfabb4817579 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d7dffb700 1 -- 172.21.15.12:0/2909266604 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00e040 con 0x55f46478a720 2024-05-08T23:58:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4da8642b80 1 -- 172.21.15.12:0/2909266604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d8c00f040 con 0x55f46478a720 2024-05-08T23:58:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.146+0000 7f4d7dffb700 1 -- 172.21.15.12:0/2909266604 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4d8c007910 con 0x55f46478a720 2024-05-08T23:58:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4d7dffb700 1 -- 172.21.15.12:0/2909266604 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d8c007af0 con 0x55f46478a720 2024-05-08T23:58:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4da8642b80 10 client.18687 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4da8642b80 1 -- 172.21.15.12:0/2909266604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f4d94110860 con 0x55f46478a720 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4d7dffb700 1 -- 172.21.15.12:0/2909266604 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d8c00f040 con 0x55f46478a720 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4d7dffb700 1 -- 172.21.15.12:0/2909266604 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4d8c01b020 con 0x55f46478a720 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4da8642b80 10 client.18687 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4da8642b80 10 client.18687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4da8642b80 20 client.18687 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f4647f5a20 0x55f4647f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4da8642b80 4 client.18687 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4da8642b80 1 -- 172.21.15.12:0/2909266604 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f4646ccec0 con 0x55f4647f5a20 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.147+0000 7f4d9a224700 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f4647f5a20 0x55f4647f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4d7dffb700 10 client.18687 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4d9a224700 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f4647f5a20 0x55f4647f7e40 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550b71f570a0ef4c server_cookie=7990c3feb825e9a7 in_seq=0 out_seq=0 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4d7dffb700 1 -- 172.21.15.12:0/2909266604 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f4646ccec0 con 0x55f4647f5a20 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4d7dffb700 10 client.18687 handle_command_reply: tid=0 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 client.18687 shutdown 2024-05-08T23:58:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 -- 172.21.15.12:0/2909266604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 msgr2=0x55f464791c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464791c80 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00b010 tx=0x7f4d8c009300 comp rx=0 tx=0).stop 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 -- 172.21.15.12:0/2909266604 shutdown_connections 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f46478a720 0x55f464791c80 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f464784890 0x55f464795190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f4647f5a20 0x55f4647f7e40 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 --2- 172.21.15.12:0/2909266604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f464789e00 0x55f464791740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4d7dffb700 0 client.18687 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 -- 172.21.15.12:0/2909266604 >> 172.21.15.12:0/2909266604 conn(0x55f4646e2210 msgr2=0x55f464788b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 -- 172.21.15.12:0/2909266604 shutdown_connections 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 1 -- 172.21.15.12:0/2909266604 wait complete. 2024-05-08T23:58:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.148+0000 7f4da8642b80 20 client.18687 trim_cache size 0 max 16384 2024-05-08T23:58:02.153 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:58:02.153 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:58:02.153 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:58:02.153 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:58:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 -- 172.21.15.12:0/2127121581 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4d800a1290 msgr2=0x7f4d800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4d800a1290 0x7f4d800a3750 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000b2e0 tx=0x7f4d90005fd0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 -- 172.21.15.12:0/2127121581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d94068970 msgr2=0x7f4d94199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d94068970 0x7f4d94199b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400e9a0 tx=0x7f4d8400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 -- 172.21.15.12:0/2127121581 shutdown_connections 2024-05-08T23:58:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4d800a1290 0x7f4d800a3750 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d9410d940 0x7f4d9419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d94068050 0x7f4d94199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 --2- 172.21.15.12:0/2127121581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d94068970 0x7f4d94199b40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 -- 172.21.15.12:0/2127121581 >> 172.21.15.12:0/2127121581 conn(0x7f4d94076f40 msgr2=0x7f4d9410ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 -- 172.21.15.12:0/2127121581 shutdown_connections 2024-05-08T23:58:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.149+0000 7f4d9c488700 1 -- 172.21.15.12:0/2127121581 wait complete. 2024-05-08T23:58:02.156 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:58:02.157 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:58:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.300+0000 7fce0b16d700 1 -- 172.21.15.12:0/1632980523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce041051a0 msgr2=0x7fce04105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.300+0000 7fce0b16d700 1 --2- 172.21.15.12:0/1632980523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce041051a0 0x7fce04105650 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009f90 tx=0x7fcdfc00b330 comp rx=0 tx=0).stop 2024-05-08T23:58:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.300+0000 7fce0b16d700 1 -- 172.21.15.12:0/1632980523 shutdown_connections 2024-05-08T23:58:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.300+0000 7fce0b16d700 1 --2- 172.21.15.12:0/1632980523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce041051a0 0x7fce04105650 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.300+0000 7fce0b16d700 1 --2- 172.21.15.12:0/1632980523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce040ffda0 0x7fce04100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.300+0000 7fce0b16d700 1 --2- 172.21.15.12:0/1632980523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce041006c0 0x7fce04104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.300+0000 7fce0b16d700 1 -- 172.21.15.12:0/1632980523 >> 172.21.15.12:0/1632980523 conn(0x7fce040fb810 msgr2=0x7fce040fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.300+0000 7fce0b16d700 1 -- 172.21.15.12:0/1632980523 shutdown_connections 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.300+0000 7fce0b16d700 1 -- 172.21.15.12:0/1632980523 wait complete. 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.300+0000 7fce0b16d700 1 Processor -- start 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce0b16d700 1 -- start start 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce0b16d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce040ffda0 0x7fce041a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce0b16d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce041006c0 0x7fce041a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce0b16d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce041051a0 0x7fce0419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce0b16d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0419de40 con 0x7fce040ffda0 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce0b16d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0419dfb0 con 0x7fce041006c0 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce0b16d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0419e170 con 0x7fce041051a0 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce08f09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce040ffda0 0x7fce041a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce08f09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce040ffda0 0x7fce041a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46120/0 (socket says 172.21.15.12:46120) 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce08f09700 1 -- 172.21.15.12:0/3524352146 learned_addr learned my addr 172.21.15.12:0/3524352146 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce03fff700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce041006c0 0x7fce041a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce0970a700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce041051a0 0x7fce0419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce08f09700 1 -- 172.21.15.12:0/3524352146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce041051a0 msgr2=0x7fce0419d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce08f09700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce041051a0 0x7fce0419d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce08f09700 1 -- 172.21.15.12:0/3524352146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce041006c0 msgr2=0x7fce041a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce08f09700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce041006c0 0x7fce041a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce08f09700 1 -- 172.21.15.12:0/3524352146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf8009520 con 0x7fce040ffda0 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.301+0000 7fce03fff700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce041006c0 0x7fce041a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.302+0000 7fce08f09700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce040ffda0 0x7fce041a36a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800eb30 tx=0x7fcdf800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.302+0000 7fce01ffb700 1 -- 172.21.15.12:0/3524352146 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8009880 con 0x7fce040ffda0 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.302+0000 7fce0b16d700 1 -- 172.21.15.12:0/3524352146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf00095f0 con 0x7fce040ffda0 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.302+0000 7fce01ffb700 1 -- 172.21.15.12:0/3524352146 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdf80044e0 con 0x7fce040ffda0 2024-05-08T23:58:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.302+0000 7fce01ffb700 1 -- 172.21.15.12:0/3524352146 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8003d80 con 0x7fce040ffda0 2024-05-08T23:58:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.302+0000 7fce0b16d700 1 -- 172.21.15.12:0/3524352146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc009c40 con 0x7fce040ffda0 2024-05-08T23:58:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.303+0000 7fce0b16d700 1 -- 172.21.15.12:0/3524352146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce04069b30 con 0x7fce040ffda0 2024-05-08T23:58:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.304+0000 7fce01ffb700 1 -- 172.21.15.12:0/3524352146 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcdf8003f20 con 0x7fce040ffda0 2024-05-08T23:58:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.305+0000 7fce01ffb700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcdf40a1290 0x7fcdf40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.305+0000 7fce03fff700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcdf40a1290 0x7fcdf40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.305+0000 7fce01ffb700 1 -- 172.21.15.12:0/3524352146 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcdf8014070 con 0x7fce040ffda0 2024-05-08T23:58:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.306+0000 7fce03fff700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcdf40a1290 0x7fcdf40a3750 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009fd0 tx=0x7fcdf0009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.310+0000 7fce01ffb700 1 -- 172.21.15.12:0/3524352146 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcdf80966c0 con 0x7fce040ffda0 2024-05-08T23:58:02.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.583+0000 7fce0b16d700 1 -- 172.21.15.12:0/3524352146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce04067380 con 0x7fce040ffda0 2024-05-08T23:58:02.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.583+0000 7fce01ffb700 1 -- 172.21.15.12:0/3524352146 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+10877 (secure 0 0 0) 0x7fcdf8099ce0 con 0x7fce040ffda0 2024-05-08T23:58:02.578 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:58:02.578 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:57:48.591091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_28349":{"gid":28349,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2125160991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2125160991},{"type":"v1","addr":"172.21.15.204:6837","nonce":2125160991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:58:02.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.586+0000 7fce0b16d700 1 -- 172.21.15.12:0/3524352146 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcdf40a1290 msgr2=0x7fcdf40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.586+0000 7fce0b16d700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcdf40a1290 0x7fcdf40a3750 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009fd0 tx=0x7fcdf0009310 comp rx=0 tx=0).stop 2024-05-08T23:58:02.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.586+0000 7fce0b16d700 1 -- 172.21.15.12:0/3524352146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce040ffda0 msgr2=0x7fce041a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.586+0000 7fce0b16d700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce040ffda0 0x7fce041a36a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800eb30 tx=0x7fcdf800ee40 comp rx=0 tx=0).stop 2024-05-08T23:58:02.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.586+0000 7fce0b16d700 1 -- 172.21.15.12:0/3524352146 shutdown_connections 2024-05-08T23:58:02.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.586+0000 7fce0b16d700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcdf40a1290 0x7fcdf40a3750 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.586+0000 7fce0b16d700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce041051a0 0x7fce0419d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.586+0000 7fce0b16d700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce040ffda0 0x7fce041a36a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.586+0000 7fce0b16d700 1 --2- 172.21.15.12:0/3524352146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce041006c0 0x7fce041a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.586+0000 7fce0b16d700 1 -- 172.21.15.12:0/3524352146 >> 172.21.15.12:0/3524352146 conn(0x7fce040fb810 msgr2=0x7fce040fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.587+0000 7fce0b16d700 1 -- 172.21.15.12:0/3524352146 shutdown_connections 2024-05-08T23:58:02.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.587+0000 7fce0b16d700 1 -- 172.21.15.12:0/3524352146 wait complete. 2024-05-08T23:58:02.582 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2024-05-08T23:58:02.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:58:02.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.743+0000 7f5c7aae3700 1 -- 172.21.15.12:0/3155120498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c74101b30 msgr2=0x7f5c74105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.743+0000 7f5c7aae3700 1 --2- 172.21.15.12:0/3155120498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c74101b30 0x7f5c74105b70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009990 tx=0x7f5c60009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.743+0000 7f5c7aae3700 1 -- 172.21.15.12:0/3155120498 shutdown_connections 2024-05-08T23:58:02.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.743+0000 7f5c7aae3700 1 --2- 172.21.15.12:0/3155120498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c741062a0 0x7f5c7410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.743+0000 7f5c7aae3700 1 --2- 172.21.15.12:0/3155120498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c74101180 0x7f5c74101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.743+0000 7f5c7aae3700 1 --2- 172.21.15.12:0/3155120498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c74101b30 0x7f5c74105b70 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.743+0000 7f5c7aae3700 1 -- 172.21.15.12:0/3155120498 >> 172.21.15.12:0/3155120498 conn(0x7f5c740fc9e0 msgr2=0x7f5c740fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.743+0000 7f5c7aae3700 1 -- 172.21.15.12:0/3155120498 shutdown_connections 2024-05-08T23:58:02.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.743+0000 7f5c7aae3700 1 -- 172.21.15.12:0/3155120498 wait complete. 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c7aae3700 1 Processor -- start 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c7aae3700 1 -- start start 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c7aae3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c74101180 0x7f5c741973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c7aae3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c741062a0 0x7f5c74197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c7aae3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c7419bcd0 0x7f5c7419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c7aae3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7419c750 con 0x7f5c741062a0 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c7aae3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7419c8c0 con 0x7f5c7419bcd0 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c7aae3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7419ca80 con 0x7f5c74101180 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c73fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c741062a0 0x7f5c74197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c73fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c741062a0 0x7f5c74197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46138/0 (socket says 172.21.15.12:46138) 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c73fff700 1 -- 172.21.15.12:0/881759177 learned_addr learned my addr 172.21.15.12:0/881759177 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.744+0000 7f5c79080700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c7419bcd0 0x7f5c7419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c7887f700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c74101180 0x7f5c741973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c73fff700 1 -- 172.21.15.12:0/881759177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c74101180 msgr2=0x7f5c741973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c73fff700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c74101180 0x7f5c741973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c73fff700 1 -- 172.21.15.12:0/881759177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c7419bcd0 msgr2=0x7f5c7419c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c73fff700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c7419bcd0 0x7f5c7419c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c73fff700 1 -- 172.21.15.12:0/881759177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c600095f0 con 0x7f5c741062a0 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c7887f700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c74101180 0x7f5c741973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c73fff700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c741062a0 0x7f5c74197930 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5c60005770 tx=0x7f5c600057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c71ffb700 1 -- 172.21.15.12:0/881759177 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6001d070 con 0x7f5c741062a0 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c71ffb700 1 -- 172.21.15.12:0/881759177 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c60021410 con 0x7f5c741062a0 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c7aae3700 1 -- 172.21.15.12:0/881759177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c741a3f80 con 0x7f5c741062a0 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c71ffb700 1 -- 172.21.15.12:0/881759177 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60017650 con 0x7f5c741062a0 2024-05-08T23:58:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.745+0000 7f5c7aae3700 1 -- 172.21.15.12:0/881759177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c741a43a0 con 0x7f5c741062a0 2024-05-08T23:58:02.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.748+0000 7f5c71ffb700 1 -- 172.21.15.12:0/881759177 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5c6000fb40 con 0x7f5c741062a0 2024-05-08T23:58:02.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.748+0000 7f5c71ffb700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5c640a1290 0x7f5c640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.748+0000 7f5c71ffb700 1 -- 172.21.15.12:0/881759177 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5c600cdfa0 con 0x7f5c741062a0 2024-05-08T23:58:02.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.749+0000 7f5c7887f700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5c640a1290 0x7f5c640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.749+0000 7f5c7887f700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5c640a1290 0x7f5c640a3750 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800bc10 tx=0x7f5c6800b5f0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.749+0000 7f5c86c9db80 1 Processor -- start 2024-05-08T23:58:02.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.749+0000 7f5c86c9db80 1 -- start start 2024-05-08T23:58:02.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244ae00 0x55af9244b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af92445350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af92445890 0x55af92445cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9239c150 con 0x55af9244b720 2024-05-08T23:58:02.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af92399b10 con 0x55af92445890 2024-05-08T23:58:02.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af92395400 con 0x55af9244ae00 2024-05-08T23:58:02.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c73fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af92445350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c79080700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af92445890 0x55af92445cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c7887f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244ae00 0x55af9244b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c73fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af92445350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46150/0 (socket says 172.21.15.12:46150) 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c73fff700 1 -- 172.21.15.12:0/2094589432 learned_addr learned my addr 172.21.15.12:0/2094589432 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c79080700 1 -- 172.21.15.12:0/2094589432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244ae00 msgr2=0x55af9244b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c79080700 1 --2- 172.21.15.12:0/2094589432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244ae00 0x55af9244b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c79080700 1 -- 172.21.15.12:0/2094589432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 msgr2=0x55af92445350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c79080700 1 --2- 172.21.15.12:0/2094589432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af92445350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c79080700 1 -- 172.21.15.12:0/2094589432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af922163b0 con 0x55af92445890 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c73fff700 1 --2- 172.21.15.12:0/2094589432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af92445350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.750+0000 7f5c79080700 1 --2- 172.21.15.12:0/2094589432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af92445890 0x55af92445cd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c011580 tx=0x7f5c6c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d6236662077019 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c5effd700 1 -- 172.21.15.12:0/2094589432 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c00e040 con 0x55af92445890 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c5effd700 1 -- 172.21.15.12:0/2094589432 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c6c00f040 con 0x55af92445890 2024-05-08T23:58:02.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c5effd700 1 -- 172.21.15.12:0/2094589432 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c004370 con 0x55af92445890 2024-05-08T23:58:02.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c86c9db80 1 -- 172.21.15.12:0/2094589432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af92445890 msgr2=0x55af92445cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/2094589432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af92445890 0x55af92445cd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c011580 tx=0x7f5c6c009920 comp rx=0 tx=0).stop 2024-05-08T23:58:02.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c86c9db80 1 -- 172.21.15.12:0/2094589432 shutdown_connections 2024-05-08T23:58:02.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/2094589432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af92445890 0x55af92445cd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/2094589432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244ae00 0x55af9244b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/2094589432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af92445350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c86c9db80 1 -- 172.21.15.12:0/2094589432 >> 172.21.15.12:0/2094589432 conn(0x55af923a3210 msgr2=0x55af9243fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.751+0000 7f5c86c9db80 1 -- 172.21.15.12:0/2094589432 shutdown_connections 2024-05-08T23:58:02.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.752+0000 7f5c86c9db80 1 -- 172.21.15.12:0/2094589432 wait complete. 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.752+0000 7f5c86c9db80 1 -- start start 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.752+0000 7f5c86c9db80 1 Processor -- start 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.752+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92451750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.752+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244ae00 0x55af9244ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.752+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244b720 0x55af9244f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.752+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af92331560 con 0x55af92445890 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.752+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af92331200 con 0x55af9244b720 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.752+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af922186a0 con 0x55af9244ae00 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c86c9db80 10 client.0 _ll_register_callbacks cb 0x55af9233ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c7887f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92451750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c7887f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92451750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46156/0 (socket says 172.21.15.12:46156) 2024-05-08T23:58:02.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c7887f700 1 -- 172.21.15.12:0/3156788125 learned_addr learned my addr 172.21.15.12:0/3156788125 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c79080700 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244b720 0x55af9244f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c73fff700 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244ae00 0x55af9244ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c7887f700 1 -- 172.21.15.12:0/3156788125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244ae00 msgr2=0x55af9244ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c7887f700 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244ae00 0x55af9244ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c7887f700 1 -- 172.21.15.12:0/3156788125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244b720 msgr2=0x55af9244f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c7887f700 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244b720 0x55af9244f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c7887f700 1 -- 172.21.15.12:0/3156788125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c741a43a0 con 0x55af92445890 2024-05-08T23:58:02.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c79080700 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244b720 0x55af9244f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c7887f700 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92451750 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800ffc0 tx=0x7f5c68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b882d6ea4adf83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c53fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c53fff700 1 -- 172.21.15.12:0/3156788125 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68028470 con 0x55af92445890 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c86c9db80 1 -- 172.21.15.12:0/3156788125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af922ee270 con 0x55af92445890 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.753+0000 7f5c53fff700 1 -- 172.21.15.12:0/3156788125 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5c680285d0 con 0x55af92445890 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c53fff700 1 -- 172.21.15.12:0/3156788125 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c68028790 con 0x55af92445890 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c86c9db80 10 client.18705 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c86c9db80 1 -- 172.21.15.12:0/3156788125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f5c6c011230 con 0x55af92445890 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c53fff700 1 -- 172.21.15.12:0/3156788125 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af922ee270 con 0x55af92445890 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c53fff700 1 -- 172.21.15.12:0/3156788125 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5c6801e5c0 con 0x55af92445890 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c86c9db80 10 client.18705 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c86c9db80 10 client.18705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c86c9db80 20 client.18705 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:02.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55af9245f1c0 0x55af924615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c86c9db80 4 client.18705 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c86c9db80 1 -- 172.21.15.12:0/3156788125 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af92414810 con 0x55af9245f1c0 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.754+0000 7f5c73fff700 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55af9245f1c0 0x55af924615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.755+0000 7f5c73fff700 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55af9245f1c0 0x55af924615e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3919f15e4803deb0 server_cookie=7f2c422d68aa3458 in_seq=0 out_seq=0 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.755+0000 7f5c53fff700 10 client.18705 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.755+0000 7f5c53fff700 1 -- 172.21.15.12:0/3156788125 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af92414810 con 0x55af9245f1c0 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.755+0000 7f5c53fff700 10 client.18705 handle_command_reply: tid=0 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.755+0000 7f5c86c9db80 1 client.18705 shutdown 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 1 -- 172.21.15.12:0/3156788125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 msgr2=0x55af92451750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92451750 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800ffc0 tx=0x7f5c68004750 comp rx=0 tx=0).stop 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 1 -- 172.21.15.12:0/3156788125 shutdown_connections 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244b720 0x55af9244f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92451750 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55af9245f1c0 0x55af924615e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/3156788125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244ae00 0x55af9244ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c53fff700 0 client.18705 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:58:02.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 1 -- 172.21.15.12:0/3156788125 >> 172.21.15.12:0/3156788125 conn(0x55af923a3210 msgr2=0x55af9243f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 1 -- 172.21.15.12:0/3156788125 shutdown_connections 2024-05-08T23:58:02.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 1 -- 172.21.15.12:0/3156788125 wait complete. 2024-05-08T23:58:02.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.756+0000 7f5c86c9db80 20 client.18705 trim_cache size 0 max 16384 2024-05-08T23:58:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.770+0000 7f5c86c9db80 1 Processor -- start 2024-05-08T23:58:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.770+0000 7f5c86c9db80 1 -- start start 2024-05-08T23:58:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.770+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af92445890 0x55af92445350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.770+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 0x55af92445c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.770+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af9245ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af92414810 con 0x55af9244b720 2024-05-08T23:58:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9245bd10 con 0x55af9244ae00 2024-05-08T23:58:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af92440eb0 con 0x55af92445890 2024-05-08T23:58:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c79080700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af9245ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c79080700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af9245ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46170/0 (socket says 172.21.15.12:46170) 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c79080700 1 -- 172.21.15.12:0/4204772977 learned_addr learned my addr 172.21.15.12:0/4204772977 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c7887f700 1 --2- 172.21.15.12:0/4204772977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af92445890 0x55af92445350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c73fff700 1 --2- 172.21.15.12:0/4204772977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 0x55af92445c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c79080700 1 -- 172.21.15.12:0/4204772977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af92445890 msgr2=0x55af92445350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c79080700 1 --2- 172.21.15.12:0/4204772977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af92445890 0x55af92445350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c79080700 1 -- 172.21.15.12:0/4204772977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 msgr2=0x55af92445c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c79080700 1 --2- 172.21.15.12:0/4204772977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 0x55af92445c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c79080700 1 -- 172.21.15.12:0/4204772977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af921c2b60 con 0x55af9244b720 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c79080700 1 --2- 172.21.15.12:0/4204772977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af9245ac20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c011520 tx=0x7f5c6c004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863a558d26a636fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c5f7fe700 1 -- 172.21.15.12:0/4204772977 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c00e040 con 0x55af9244b720 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c5f7fe700 1 -- 172.21.15.12:0/4204772977 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c6c011e80 con 0x55af9244b720 2024-05-08T23:58:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.771+0000 7f5c5f7fe700 1 -- 172.21.15.12:0/4204772977 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c004940 con 0x55af9244b720 2024-05-08T23:58:02.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.772+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4204772977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 msgr2=0x55af9245ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.772+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/4204772977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af9245ac20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c011520 tx=0x7f5c6c004de0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.772+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4204772977 shutdown_connections 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.772+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/4204772977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9244b720 0x55af9245ac20 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.772+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/4204772977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af92445890 0x55af92445350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.772+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/4204772977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 0x55af92445c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.772+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4204772977 >> 172.21.15.12:0/4204772977 conn(0x55af923a3210 msgr2=0x55af92448bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.772+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4204772977 shutdown_connections 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.772+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4204772977 wait complete. 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.772+0000 7f5c86c9db80 1 -- start start 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c86c9db80 1 Processor -- start 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92452140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 0x55af92452680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c86c9db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244b720 0x55af92452bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af924b46b0 con 0x55af92445890 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af924b47f0 con 0x55af9244ae00 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c86c9db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af924b4930 con 0x55af9244b720 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c86c9db80 10 client.0 _ll_register_callbacks cb 0x55af9233ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c7887f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92452140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c7887f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92452140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46178/0 (socket says 172.21.15.12:46178) 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c7887f700 1 -- 172.21.15.12:0/4242977086 learned_addr learned my addr 172.21.15.12:0/4242977086 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c79080700 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244b720 0x55af92452bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.773+0000 7f5c73fff700 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 0x55af92452680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c7887f700 1 -- 172.21.15.12:0/4242977086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244b720 msgr2=0x55af92452bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c7887f700 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244b720 0x55af92452bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c7887f700 1 -- 172.21.15.12:0/4242977086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 msgr2=0x55af92452680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c7887f700 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 0x55af92452680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c7887f700 1 -- 172.21.15.12:0/4242977086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af922163b0 con 0x55af92445890 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c79080700 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244b720 0x55af92452bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c73fff700 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 0x55af92452680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c7887f700 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92452140 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800ffc0 tx=0x7f5c68011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8d78addba85766 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c53fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c53fff700 1 -- 172.21.15.12:0/4242977086 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6801e430 con 0x55af92445890 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c53fff700 1 -- 172.21.15.12:0/4242977086 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5c6801e590 con 0x55af92445890 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4242977086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c6c00f040 con 0x55af92445890 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c53fff700 1 -- 172.21.15.12:0/4242977086 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c6801e750 con 0x55af92445890 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c86c9db80 10 client.18711 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4242977086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55af9235b560 con 0x55af92445890 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.774+0000 7f5c53fff700 1 -- 172.21.15.12:0/4242977086 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c68047020 con 0x55af92445890 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c53fff700 1 -- 172.21.15.12:0/4242977086 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5c68004bd0 con 0x55af92445890 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c86c9db80 10 client.18711 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c86c9db80 10 client.18711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c86c9db80 20 client.18711 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55af924b60f0 0x55af924b8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c86c9db80 4 client.18711 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4242977086 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55af9238dec0 con 0x55af924b60f0 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c73fff700 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55af924b60f0 0x55af924b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c73fff700 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55af924b60f0 0x55af924b8510 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=376af34abf6a70d3 server_cookie=b9640950c7b1efc5 in_seq=0 out_seq=0 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c53fff700 10 client.18711 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c53fff700 1 -- 172.21.15.12:0/4242977086 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55af9238dec0 con 0x55af924b60f0 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.775+0000 7f5c53fff700 10 client.18711 handle_command_reply: tid=0 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 client.18711 shutdown 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4242977086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 msgr2=0x55af92452140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92452140 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800ffc0 tx=0x7f5c68011a70 comp rx=0 tx=0).stop 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4242977086 shutdown_connections 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55af924b60f0 0x55af924b8510 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af9244b720 0x55af92452bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c53fff700 0 client.18711 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:58:02.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af92445890 0x55af92452140 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 --2- 172.21.15.12:0/4242977086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af9244ae00 0x55af92452680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4242977086 >> 172.21.15.12:0/4242977086 conn(0x55af923a3210 msgr2=0x55af9244d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4242977086 shutdown_connections 2024-05-08T23:58:02.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 1 -- 172.21.15.12:0/4242977086 wait complete. 2024-05-08T23:58:02.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.776+0000 7f5c86c9db80 20 client.18711 trim_cache size 0 max 16384 2024-05-08T23:58:02.776 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:58:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 -- 172.21.15.12:0/881759177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5c640a1290 msgr2=0x7f5c640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5c640a1290 0x7f5c640a3750 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800bc10 tx=0x7f5c6800b5f0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 -- 172.21.15.12:0/881759177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c741062a0 msgr2=0x7f5c74197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c741062a0 0x7f5c74197930 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5c60005770 tx=0x7f5c600057a0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 -- 172.21.15.12:0/881759177 shutdown_connections 2024-05-08T23:58:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5c640a1290 0x7f5c640a3750 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c7419bcd0 0x7f5c7419c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c74101180 0x7f5c741973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 --2- 172.21.15.12:0/881759177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c741062a0 0x7f5c74197930 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 -- 172.21.15.12:0/881759177 >> 172.21.15.12:0/881759177 conn(0x7f5c740fc9e0 msgr2=0x7f5c740fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 -- 172.21.15.12:0/881759177 shutdown_connections 2024-05-08T23:58:02.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.777+0000 7f5c7aae3700 1 -- 172.21.15.12:0/881759177 wait complete. 2024-05-08T23:58:02.785 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:58:02.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 639994 6.17 MB/sec execute 478 sec latency 25.593 ms 2024-05-08T23:58:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:02 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3524352146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:58:02.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.943+0000 7f856ebd4700 1 -- 172.21.15.12:0/3343595246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568105170 msgr2=0x7f856810ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.943+0000 7f856ebd4700 1 --2- 172.21.15.12:0/3343595246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568105170 0x7f856810ccf0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f856000b0a0 tx=0x7f856000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.944+0000 7f856ebd4700 1 -- 172.21.15.12:0/3343595246 shutdown_connections 2024-05-08T23:58:02.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.944+0000 7f856ebd4700 1 --2- 172.21.15.12:0/3343595246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568105170 0x7f856810ccf0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.944+0000 7f856ebd4700 1 --2- 172.21.15.12:0/3343595246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568100060 0x7f8568100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.944+0000 7f856ebd4700 1 --2- 172.21.15.12:0/3343595246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8568100a10 0x7f8568104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.944+0000 7f856ebd4700 1 -- 172.21.15.12:0/3343595246 >> 172.21.15.12:0/3343595246 conn(0x7f8568076f30 msgr2=0x7f8568077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.944+0000 7f856ebd4700 1 -- 172.21.15.12:0/3343595246 shutdown_connections 2024-05-08T23:58:02.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.944+0000 7f856ebd4700 1 -- 172.21.15.12:0/3343595246 wait complete. 2024-05-08T23:58:02.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856ebd4700 1 Processor -- start 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856ebd4700 1 -- start start 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856ebd4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100060 0x7f85681952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856ebd4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8568105170 0x7f8568195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856ebd4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568199bd0 0x7f856819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856ebd4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856819a650 con 0x7f8568100060 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856ebd4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856819a7c0 con 0x7f8568105170 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856ebd4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856819a980 con 0x7f8568199bd0 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856c970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100060 0x7f85681952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856c970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100060 0x7f85681952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46202/0 (socket says 172.21.15.12:46202) 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856c970700 1 -- 172.21.15.12:0/3934088768 learned_addr learned my addr 172.21.15.12:0/3934088768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.945+0000 7f856d171700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568199bd0 0x7f856819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f8567fff700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8568105170 0x7f8568195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f856c970700 1 -- 172.21.15.12:0/3934088768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568199bd0 msgr2=0x7f856819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f856c970700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568199bd0 0x7f856819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f856c970700 1 -- 172.21.15.12:0/3934088768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8568105170 msgr2=0x7f8568195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f856c970700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8568105170 0x7f8568195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f856c970700 1 -- 172.21.15.12:0/3934088768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560009d00 con 0x7f8568100060 2024-05-08T23:58:02.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f856d171700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568199bd0 0x7f856819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f856c970700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100060 0x7f85681952f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f855c0097d0 tx=0x7f855c009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f8565ffb700 1 -- 172.21.15.12:0/3934088768 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c010040 con 0x7f8568100060 2024-05-08T23:58:02.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f8565ffb700 1 -- 172.21.15.12:0/3934088768 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f855c013c30 con 0x7f8568100060 2024-05-08T23:58:02.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f856ebd4700 1 -- 172.21.15.12:0/3934088768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85681a1ee0 con 0x7f8568100060 2024-05-08T23:58:02.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.946+0000 7f8565ffb700 1 -- 172.21.15.12:0/3934088768 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c014d60 con 0x7f8568100060 2024-05-08T23:58:02.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.947+0000 7f856ebd4700 1 -- 172.21.15.12:0/3934088768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85681a23f0 con 0x7f8568100060 2024-05-08T23:58:02.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.949+0000 7f8565ffb700 1 -- 172.21.15.12:0/3934088768 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f855c014400 con 0x7f8568100060 2024-05-08T23:58:02.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.950+0000 7f8565ffb700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85580a1290 0x7f85580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.950+0000 7f8565ffb700 1 -- 172.21.15.12:0/3934088768 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f855c0cc8e0 con 0x7f8568100060 2024-05-08T23:58:02.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.951+0000 7f8567fff700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85580a1290 0x7f85580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.951+0000 7f8567fff700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85580a1290 0x7f85580a3750 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8568196840 tx=0x7f8554009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f857ad8eb80 1 Processor -- start 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f857ad8eb80 1 -- start start 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cb720 0x5643a25c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 0x5643a25c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a251c150 con 0x5643a25cae00 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a2519b10 con 0x5643a25cb720 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a2515400 con 0x5643a25c5890 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f856c970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f8567fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cb720 0x5643a25c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f856c970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46208/0 (socket says 172.21.15.12:46208) 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f8567fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cb720 0x5643a25c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48860/0 (socket says 172.21.15.12:48860) 2024-05-08T23:58:02.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f856c970700 1 -- 172.21.15.12:0/129790482 learned_addr learned my addr 172.21.15.12:0/129790482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f856d171700 1 --2- 172.21.15.12:0/129790482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 0x5643a25c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f856d171700 1 -- 172.21.15.12:0/129790482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cb720 msgr2=0x5643a25c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f856d171700 1 --2- 172.21.15.12:0/129790482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cb720 0x5643a25c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f856d171700 1 -- 172.21.15.12:0/129790482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 msgr2=0x5643a25cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f856d171700 1 --2- 172.21.15.12:0/129790482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f856d171700 1 -- 172.21.15.12:0/129790482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643a23963b0 con 0x5643a25c5890 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f856c970700 1 --2- 172.21.15.12:0/129790482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.952+0000 7f8567fff700 1 --2- 172.21.15.12:0/129790482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cb720 0x5643a25c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.953+0000 7f856d171700 1 --2- 172.21.15.12:0/129790482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 0x5643a25c5cd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f856000b780 tx=0x7f8560009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14bbd36e9b7d005c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.953+0000 7f8552ffd700 1 -- 172.21.15.12:0/129790482 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856000e040 con 0x5643a25c5890 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.953+0000 7f8552ffd700 1 -- 172.21.15.12:0/129790482 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8560006080 con 0x5643a25c5890 2024-05-08T23:58:02.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.953+0000 7f8552ffd700 1 -- 172.21.15.12:0/129790482 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560003cf0 con 0x5643a25c5890 2024-05-08T23:58:02.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 -- 172.21.15.12:0/129790482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 msgr2=0x5643a25c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/129790482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 0x5643a25c5cd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f856000b780 tx=0x7f8560009b80 comp rx=0 tx=0).stop 2024-05-08T23:58:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 -- 172.21.15.12:0/129790482 shutdown_connections 2024-05-08T23:58:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/129790482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 0x5643a25c5cd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/129790482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/129790482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cb720 0x5643a25c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 -- 172.21.15.12:0/129790482 >> 172.21.15.12:0/129790482 conn(0x5643a2523210 msgr2=0x5643a25bfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 -- 172.21.15.12:0/129790482 shutdown_connections 2024-05-08T23:58:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 -- 172.21.15.12:0/129790482 wait complete. 2024-05-08T23:58:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 -- start start 2024-05-08T23:58:02.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.954+0000 7f857ad8eb80 1 Processor -- start 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 0x5643a25cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cb720 0x5643a25cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a24b1560 con 0x5643a25cae00 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a24b1200 con 0x5643a25c5890 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a23986a0 con 0x5643a25cb720 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f857ad8eb80 10 client.0 _ll_register_callbacks cb 0x5643a24bab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f8567fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f856c970700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 0x5643a25cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f8567fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46218/0 (socket says 172.21.15.12:46218) 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f8567fff700 1 -- 172.21.15.12:0/984080132 learned_addr learned my addr 172.21.15.12:0/984080132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.955+0000 7f856d171700 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cb720 0x5643a25cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f8567fff700 1 -- 172.21.15.12:0/984080132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cb720 msgr2=0x5643a25cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f8567fff700 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cb720 0x5643a25cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f8567fff700 1 -- 172.21.15.12:0/984080132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 msgr2=0x5643a25cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f8567fff700 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 0x5643a25cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f8567fff700 1 -- 172.21.15.12:0/984080132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855400f820 con 0x5643a25cae00 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f856c970700 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 0x5643a25cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f856d171700 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cb720 0x5643a25cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f8567fff700 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cf010 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8554011470 tx=0x7f8554004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19ba83ade1849f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f8547fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f8547fff700 1 -- 172.21.15.12:0/984080132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554004a30 con 0x5643a25cae00 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f8547fff700 1 -- 172.21.15.12:0/984080132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8554011e20 con 0x5643a25cae00 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f857ad8eb80 1 -- 172.21.15.12:0/984080132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643a246e270 con 0x5643a25cae00 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f8547fff700 1 -- 172.21.15.12:0/984080132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8554004b90 con 0x5643a25cae00 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f857ad8eb80 10 client.18723 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.956+0000 7f857ad8eb80 1 -- 172.21.15.12:0/984080132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f856810d560 con 0x5643a25cae00 2024-05-08T23:58:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f8547fff700 1 -- 172.21.15.12:0/984080132 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5643a246e270 con 0x5643a25cae00 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f8547fff700 1 -- 172.21.15.12:0/984080132 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f85540306a0 con 0x5643a25cae00 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f857ad8eb80 10 client.18723 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f857ad8eb80 10 client.18723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f857ad8eb80 20 client.18723 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5643a25df2a0 0x5643a25e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f857ad8eb80 4 client.18723 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f857ad8eb80 1 -- 172.21.15.12:0/984080132 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643a2594810 con 0x5643a25df2a0 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f856c970700 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5643a25df2a0 0x5643a25e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f8547fff700 10 client.18723 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.957+0000 7f856c970700 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5643a25df2a0 0x5643a25e16c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc7464b5c931b13b server_cookie=86e696fb7decad1e in_seq=0 out_seq=0 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.958+0000 7f8547fff700 1 -- 172.21.15.12:0/984080132 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5643a2594810 con 0x5643a25df2a0 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.958+0000 7f8547fff700 10 client.18723 handle_command_reply: tid=0 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.958+0000 7f857ad8eb80 1 client.18723 shutdown 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.958+0000 7f857ad8eb80 1 -- 172.21.15.12:0/984080132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 msgr2=0x5643a25cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.958+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cf010 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8554011470 tx=0x7f8554004750 comp rx=0 tx=0).stop 2024-05-08T23:58:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.959+0000 7f857ad8eb80 1 -- 172.21.15.12:0/984080132 shutdown_connections 2024-05-08T23:58:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.959+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5643a25df2a0 0x5643a25e16c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.959+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cb720 0x5643a25cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.959+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 0x5643a25cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.959+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/984080132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cae00 0x5643a25cf010 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.959+0000 7f8547fff700 0 client.18723 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:58:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.959+0000 7f857ad8eb80 1 -- 172.21.15.12:0/984080132 >> 172.21.15.12:0/984080132 conn(0x5643a2523210 msgr2=0x5643a25bf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.959+0000 7f857ad8eb80 1 -- 172.21.15.12:0/984080132 shutdown_connections 2024-05-08T23:58:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.959+0000 7f857ad8eb80 1 -- 172.21.15.12:0/984080132 wait complete. 2024-05-08T23:58:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.959+0000 7f857ad8eb80 20 client.18723 trim_cache size 0 max 16384 2024-05-08T23:58:02.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.973+0000 7f857ad8eb80 1 Processor -- start 2024-05-08T23:58:02.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.973+0000 7f857ad8eb80 1 -- start start 2024-05-08T23:58:02.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 0x5643a25c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cae00 0x5643a25e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a25dbd40 con 0x5643a25cb720 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a25c08e0 con 0x5643a25c5890 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a25bcdf0 con 0x5643a25cae00 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856d171700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856d171700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25dabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46230/0 (socket says 172.21.15.12:46230) 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856c970700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 0x5643a25c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856d171700 1 -- 172.21.15.12:0/3471944341 learned_addr learned my addr 172.21.15.12:0/3471944341 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f8567fff700 1 --2- 172.21.15.12:0/3471944341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cae00 0x5643a25e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856d171700 1 -- 172.21.15.12:0/3471944341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cae00 msgr2=0x5643a25e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856d171700 1 --2- 172.21.15.12:0/3471944341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cae00 0x5643a25e1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856d171700 1 -- 172.21.15.12:0/3471944341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 msgr2=0x5643a25c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856d171700 1 --2- 172.21.15.12:0/3471944341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 0x5643a25c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856d171700 1 -- 172.21.15.12:0/3471944341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643a2342b60 con 0x5643a25cb720 2024-05-08T23:58:02.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856c970700 1 --2- 172.21.15.12:0/3471944341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 0x5643a25c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f856d171700 1 --2- 172.21.15.12:0/3471944341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25dabc0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f8560009fd0 tx=0x7f856000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36c4d4f48e1d34f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.974+0000 7f85537fe700 1 -- 172.21.15.12:0/3471944341 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856000e040 con 0x5643a25cb720 2024-05-08T23:58:02.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f85537fe700 1 -- 172.21.15.12:0/3471944341 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f856000be90 con 0x5643a25cb720 2024-05-08T23:58:02.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f85537fe700 1 -- 172.21.15.12:0/3471944341 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600053b0 con 0x5643a25cb720 2024-05-08T23:58:02.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f857ad8eb80 1 -- 172.21.15.12:0/3471944341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 msgr2=0x5643a25dabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/3471944341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25dabc0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f8560009fd0 tx=0x7f856000bd60 comp rx=0 tx=0).stop 2024-05-08T23:58:02.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f857ad8eb80 1 -- 172.21.15.12:0/3471944341 shutdown_connections 2024-05-08T23:58:02.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/3471944341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25dabc0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/3471944341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25c5890 0x5643a25c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/3471944341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25cae00 0x5643a25e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f857ad8eb80 1 -- 172.21.15.12:0/3471944341 >> 172.21.15.12:0/3471944341 conn(0x5643a2523210 msgr2=0x5643a25cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f857ad8eb80 1 -- 172.21.15.12:0/3471944341 shutdown_connections 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.975+0000 7f857ad8eb80 1 -- 172.21.15.12:0/3471944341 wait complete. 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f857ad8eb80 1 -- start start 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f857ad8eb80 1 Processor -- start 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 0x5643a25d6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cae00 0x5643a25d6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f857ad8eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a25d4630 con 0x5643a25cb720 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a25d47a0 con 0x5643a25cae00 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f857ad8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a25d4960 con 0x5643a25c5890 2024-05-08T23:58:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f857ad8eb80 10 client.0 _ll_register_callbacks cb 0x5643a24bab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f856d171700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f8567fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cae00 0x5643a25d6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f856d171700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25d40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46238/0 (socket says 172.21.15.12:46238) 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f8567fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cae00 0x5643a25d6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48886/0 (socket says 172.21.15.12:48886) 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.976+0000 7f856d171700 1 -- 172.21.15.12:0/2735923697 learned_addr learned my addr 172.21.15.12:0/2735923697 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f856c970700 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 0x5643a25d6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f856d171700 1 -- 172.21.15.12:0/2735923697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 msgr2=0x5643a25d6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f856d171700 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 0x5643a25d6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f856d171700 1 -- 172.21.15.12:0/2735923697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cae00 msgr2=0x5643a25d6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f856d171700 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cae00 0x5643a25d6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f856d171700 1 -- 172.21.15.12:0/2735923697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c023070 con 0x5643a25cb720 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f8567fff700 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cae00 0x5643a25d6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f856d171700 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25d40f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8560009a20 tx=0x7f85600095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c1866fa5bffca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f8547fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f8547fff700 1 -- 172.21.15.12:0/2735923697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856000e030 con 0x5643a25cb720 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f857ad8eb80 1 -- 172.21.15.12:0/2735923697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8560006080 con 0x5643a25cb720 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f8547fff700 1 -- 172.21.15.12:0/2735923697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8560003a00 con 0x5643a25cb720 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.977+0000 7f8547fff700 1 -- 172.21.15.12:0/2735923697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8560003bc0 con 0x5643a25cb720 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f857ad8eb80 10 client.18729 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f857ad8eb80 1 -- 172.21.15.12:0/2735923697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5643a23963b0 con 0x5643a25cb720 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f8547fff700 1 -- 172.21.15.12:0/2735923697 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8560006080 con 0x5643a25cb720 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f8547fff700 1 -- 172.21.15.12:0/2735923697 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8560019070 con 0x5643a25cb720 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f857ad8eb80 10 client.18729 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f857ad8eb80 10 client.18729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f857ad8eb80 20 client.18729 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5643a2635350 0x5643a2637740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f857ad8eb80 4 client.18729 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f857ad8eb80 1 -- 172.21.15.12:0/2735923697 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5643a250dec0 con 0x5643a2635350 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.978+0000 7f856c970700 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5643a2635350 0x5643a2637740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f8547fff700 10 client.18729 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f856c970700 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5643a2635350 0x5643a2637740 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5636bbfa6e04e4f1 server_cookie=2da483c7e0fe21ee in_seq=0 out_seq=0 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f8547fff700 1 -- 172.21.15.12:0/2735923697 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5643a250dec0 con 0x5643a2635350 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f8547fff700 10 client.18729 handle_command_reply: tid=0 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 client.18729 shutdown 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 -- 172.21.15.12:0/2735923697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 msgr2=0x5643a25d40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25d40f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8560009a20 tx=0x7f85600095d0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 -- 172.21.15.12:0/2735923697 shutdown_connections 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5643a2635350 0x5643a2637740 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643a25cb720 0x5643a25d40f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643a25c5890 0x5643a25d6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 --2- 172.21.15.12:0/2735923697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5643a25cae00 0x5643a25d6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 -- 172.21.15.12:0/2735923697 >> 172.21.15.12:0/2735923697 conn(0x5643a2523210 msgr2=0x5643a25c9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f8547fff700 0 client.18729 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:58:02.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 -- 172.21.15.12:0/2735923697 shutdown_connections 2024-05-08T23:58:02.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 1 -- 172.21.15.12:0/2735923697 wait complete. 2024-05-08T23:58:02.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.979+0000 7f857ad8eb80 20 client.18729 trim_cache size 0 max 16384 2024-05-08T23:58:02.980 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 -- 172.21.15.12:0/3934088768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85580a1290 msgr2=0x7f85580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85580a1290 0x7f85580a3750 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8568196840 tx=0x7f8554009260 comp rx=0 tx=0).stop 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 -- 172.21.15.12:0/3934088768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100060 msgr2=0x7f85681952f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100060 0x7f85681952f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f855c0097d0 tx=0x7f855c009ae0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 -- 172.21.15.12:0/3934088768 shutdown_connections 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f85580a1290 0x7f85580a3750 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568199bd0 0x7f856819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8568100060 0x7f85681952f0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 --2- 172.21.15.12:0/3934088768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8568105170 0x7f8568195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 -- 172.21.15.12:0/3934088768 >> 172.21.15.12:0/3934088768 conn(0x7f8568076f30 msgr2=0x7f85680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 -- 172.21.15.12:0/3934088768 shutdown_connections 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:02.980+0000 7f856ebd4700 1 -- 172.21.15.12:0/3934088768 wait complete. 2024-05-08T23:58:02.988 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:58:03.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:02 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3524352146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:58:03.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:02 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3524352146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:58:03.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.137+0000 7f3024f93700 1 -- 172.21.15.12:0/1068882957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3020068c90 msgr2=0x7f3020105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:03.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.137+0000 7f3024f93700 1 --2- 172.21.15.12:0/1068882957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3020068c90 0x7f3020105b60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3008009990 tx=0x7f3008009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.137+0000 7f3024f93700 1 -- 172.21.15.12:0/1068882957 shutdown_connections 2024-05-08T23:58:03.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.137+0000 7f3024f93700 1 --2- 172.21.15.12:0/1068882957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3020106290 0x7f302010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.137+0000 7f3024f93700 1 --2- 172.21.15.12:0/1068882957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30200682e0 0x7f30200686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.137+0000 7f3024f93700 1 --2- 172.21.15.12:0/1068882957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3020068c90 0x7f3020105b60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.137+0000 7f3024f93700 1 -- 172.21.15.12:0/1068882957 >> 172.21.15.12:0/1068882957 conn(0x7f3020076f40 msgr2=0x7f3020077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:03.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.137+0000 7f3024f93700 1 -- 172.21.15.12:0/1068882957 shutdown_connections 2024-05-08T23:58:03.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.137+0000 7f3024f93700 1 -- 172.21.15.12:0/1068882957 wait complete. 2024-05-08T23:58:03.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.138+0000 7f3024f93700 1 Processor -- start 2024-05-08T23:58:03.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.138+0000 7f3024f93700 1 -- start start 2024-05-08T23:58:03.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.138+0000 7f3024f93700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30200682e0 0x7f30201973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.138+0000 7f3024f93700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3020106290 0x7f3020197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.138+0000 7f3024f93700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f302019bcd0 0x7f302019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.138+0000 7f3024f93700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302019c750 con 0x7f302019bcd0 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.138+0000 7f3024f93700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302019c8c0 con 0x7f30200682e0 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.138+0000 7f3024f93700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302019ca80 con 0x7f3020106290 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30200682e0 0x7f30201973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f302019bcd0 0x7f302019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f3015d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3020106290 0x7f3020197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f3015d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3020106290 0x7f3020197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46248/0 (socket says 172.21.15.12:46248) 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30200682e0 0x7f30201973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48912/0 (socket says 172.21.15.12:48912) 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f3015d9b700 1 -- 172.21.15.12:0/1291827817 learned_addr learned my addr 172.21.15.12:0/1291827817 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f302019bcd0 0x7f302019c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46260/0 (socket says 172.21.15.12:46260) 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301ed9d700 1 -- 172.21.15.12:0/1291827817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3020106290 msgr2=0x7f3020197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301ed9d700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3020106290 0x7f3020197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301ed9d700 1 -- 172.21.15.12:0/1291827817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30200682e0 msgr2=0x7f30201973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301ed9d700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30200682e0 0x7f30201973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301ed9d700 1 -- 172.21.15.12:0/1291827817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080095f0 con 0x7f302019bcd0 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301e59c700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30200682e0 0x7f30201973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f301ed9d700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f302019bcd0 0x7f302019c150 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f301800efc0 tx=0x7f301800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f3017fff700 1 -- 172.21.15.12:0/1291827817 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018018070 con 0x7f302019bcd0 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f3024f93700 1 -- 172.21.15.12:0/1291827817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30201a3f80 con 0x7f302019bcd0 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f3017fff700 1 -- 172.21.15.12:0/1291827817 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f301800f040 con 0x7f302019bcd0 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f3017fff700 1 -- 172.21.15.12:0/1291827817 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301800cd40 con 0x7f302019bcd0 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.139+0000 7f3024f93700 1 -- 172.21.15.12:0/1291827817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30201a4410 con 0x7f302019bcd0 2024-05-08T23:58:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.142+0000 7f3017fff700 1 -- 172.21.15.12:0/1291827817 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3018003a00 con 0x7f302019bcd0 2024-05-08T23:58:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.143+0000 7f3017fff700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f300c0a1290 0x7f300c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.143+0000 7f3017fff700 1 -- 172.21.15.12:0/1291827817 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f30180cc860 con 0x7f302019bcd0 2024-05-08T23:58:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.143+0000 7f301e59c700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f300c0a1290 0x7f300c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.143+0000 7f301e59c700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f300c0a1290 0x7f300c0a3750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3010011490 tx=0x7f3010009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:03.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f303114db80 1 Processor -- start 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f303114db80 1 -- start start 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f303114db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558ac9fe81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f303114db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe8720 0x558ac9fe2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f303114db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe2890 0x558ac9fe2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f303114db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac9f39150 con 0x558ac9fe2890 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f303114db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac9f36b10 con 0x558ac9fe7e00 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f303114db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac9f32400 con 0x558ac9fe8720 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f301e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558ac9fe81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f301e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558ac9fe81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48922/0 (socket says 172.21.15.12:48922) 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f301ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe2890 0x558ac9fe2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f301e59c700 1 -- 172.21.15.12:0/1642752432 learned_addr learned my addr 172.21.15.12:0/1642752432 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.144+0000 7f3015d9b700 1 --2- 172.21.15.12:0/1642752432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe8720 0x558ac9fe2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.145+0000 7f301e59c700 1 -- 172.21.15.12:0/1642752432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe8720 msgr2=0x558ac9fe2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.145+0000 7f301e59c700 1 --2- 172.21.15.12:0/1642752432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe8720 0x558ac9fe2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.145+0000 7f301e59c700 1 -- 172.21.15.12:0/1642752432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe2890 msgr2=0x558ac9fe2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.145+0000 7f301e59c700 1 --2- 172.21.15.12:0/1642752432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe2890 0x558ac9fe2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.145+0000 7f301e59c700 1 -- 172.21.15.12:0/1642752432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ac9db33b0 con 0x558ac9fe7e00 2024-05-08T23:58:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.145+0000 7f301e59c700 1 --2- 172.21.15.12:0/1642752432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558ac9fe81e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f30100043c0 tx=0x7f3010014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f20b77fcd5baf08c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.145+0000 7f3014d99700 1 -- 172.21.15.12:0/1642752432 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010003c90 con 0x558ac9fe7e00 2024-05-08T23:58:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.145+0000 7f3014d99700 1 -- 172.21.15.12:0/1642752432 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3010003df0 con 0x558ac9fe7e00 2024-05-08T23:58:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.145+0000 7f3014d99700 1 -- 172.21.15.12:0/1642752432 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301001e810 con 0x558ac9fe7e00 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.146+0000 7f303114db80 1 -- 172.21.15.12:0/1642752432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 msgr2=0x558ac9fe81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.146+0000 7f303114db80 1 --2- 172.21.15.12:0/1642752432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558ac9fe81e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f30100043c0 tx=0x7f3010014000 comp rx=0 tx=0).stop 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.146+0000 7f303114db80 1 -- 172.21.15.12:0/1642752432 shutdown_connections 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.146+0000 7f303114db80 1 --2- 172.21.15.12:0/1642752432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe2890 0x558ac9fe2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.146+0000 7f303114db80 1 --2- 172.21.15.12:0/1642752432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558ac9fe81e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.146+0000 7f303114db80 1 --2- 172.21.15.12:0/1642752432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe8720 0x558ac9fe2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.146+0000 7f303114db80 1 -- 172.21.15.12:0/1642752432 >> 172.21.15.12:0/1642752432 conn(0x558ac9f40210 msgr2=0x558ac9fdcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.146+0000 7f303114db80 1 -- 172.21.15.12:0/1642752432 shutdown_connections 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.146+0000 7f303114db80 1 -- 172.21.15.12:0/1642752432 wait complete. 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.146+0000 7f303114db80 1 -- start start 2024-05-08T23:58:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f303114db80 1 Processor -- start 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f303114db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe2890 0x558ac9febb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f303114db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f303114db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe8720 0x558ac9fec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f303114db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac9ece560 con 0x558ac9fe7e00 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f303114db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac9ece200 con 0x558ac9fe2890 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f303114db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac9db56a0 con 0x558ac9fe8720 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f303114db80 10 client.0 _ll_register_callbacks cb 0x558ac9ed7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f3015d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f301e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe2890 0x558ac9febb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f3015d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fec0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46264/0 (socket says 172.21.15.12:46264) 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f3015d9b700 1 -- 172.21.15.12:0/336776478 learned_addr learned my addr 172.21.15.12:0/336776478 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f301e59c700 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe2890 0x558ac9febb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48930/0 (socket says 172.21.15.12:48930) 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f301ed9d700 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe8720 0x558ac9fec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f3015d9b700 1 -- 172.21.15.12:0/336776478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe8720 msgr2=0x558ac9fec5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f3015d9b700 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe8720 0x558ac9fec5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f3015d9b700 1 -- 172.21.15.12:0/336776478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe2890 msgr2=0x558ac9febb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f3015d9b700 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe2890 0x558ac9febb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f3015d9b700 1 -- 172.21.15.12:0/336776478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3010017070 con 0x558ac9fe7e00 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.147+0000 7f301e59c700 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe2890 0x558ac9febb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.148+0000 7f3015d9b700 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fec0a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f30080098e0 tx=0x7f3008005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf29eb5655a167e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.148+0000 7f3005ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.148+0000 7f3005ffb700 1 -- 172.21.15.12:0/336776478 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300801d030 con 0x558ac9fe7e00 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.148+0000 7f3005ffb700 1 -- 172.21.15.12:0/336776478 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3008004500 con 0x558ac9fe7e00 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.148+0000 7f303114db80 1 -- 172.21.15.12:0/336776478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ac9e8b270 con 0x558ac9fe7e00 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.148+0000 7f3005ffb700 1 -- 172.21.15.12:0/336776478 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30080046c0 con 0x558ac9fe7e00 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.148+0000 7f303114db80 10 client.18741 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.148+0000 7f303114db80 1 -- 172.21.15.12:0/336776478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x558ac9d5fb60 con 0x558ac9fe7e00 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.148+0000 7f3005ffb700 1 -- 172.21.15.12:0/336776478 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ac9e8b270 con 0x558ac9fe7e00 2024-05-08T23:58:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.148+0000 7f3005ffb700 1 -- 172.21.15.12:0/336776478 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f300800f930 con 0x558ac9fe7e00 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.149+0000 7f303114db80 10 client.18741 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.149+0000 7f303114db80 10 client.18741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.149+0000 7f303114db80 20 client.18741 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.149+0000 7f303114db80 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ac9ffc230 0x558ac9ffe650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.149+0000 7f303114db80 4 client.18741 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.149+0000 7f303114db80 1 -- 172.21.15.12:0/336776478 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ac9fb1810 con 0x558ac9ffc230 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.149+0000 7f301e59c700 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ac9ffc230 0x558ac9ffe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.149+0000 7f3005ffb700 10 client.18741 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.149+0000 7f301e59c700 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ac9ffc230 0x558ac9ffe650 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28365b72f082f4ca server_cookie=a2ae3b12ae9a3ee8 in_seq=0 out_seq=0 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f3005ffb700 1 -- 172.21.15.12:0/336776478 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ac9fb1810 con 0x558ac9ffc230 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f3005ffb700 10 client.18741 handle_command_reply: tid=0 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 client.18741 shutdown 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 -- 172.21.15.12:0/336776478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 msgr2=0x558ac9fec0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fec0a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f30080098e0 tx=0x7f3008005c20 comp rx=0 tx=0).stop 2024-05-08T23:58:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 -- 172.21.15.12:0/336776478 shutdown_connections 2024-05-08T23:58:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe8720 0x558ac9fec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ac9ffc230 0x558ac9ffe650 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe2890 0x558ac9febb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 --2- 172.21.15.12:0/336776478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fec0a0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f3005ffb700 0 client.18741 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 -- 172.21.15.12:0/336776478 >> 172.21.15.12:0/336776478 conn(0x558ac9f40210 msgr2=0x558ac9fdc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 -- 172.21.15.12:0/336776478 shutdown_connections 2024-05-08T23:58:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 1 -- 172.21.15.12:0/336776478 wait complete. 2024-05-08T23:58:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.150+0000 7f303114db80 20 client.18741 trim_cache size 0 max 16384 2024-05-08T23:58:03.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f303114db80 1 Processor -- start 2024-05-08T23:58:03.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f303114db80 1 -- start start 2024-05-08T23:58:03.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f303114db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 0x558ac9ff7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f303114db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fdcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f303114db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe8720 0x558ac9ff3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f303114db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac9ff8d80 con 0x558ac9fe7e00 2024-05-08T23:58:03.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f303114db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac9fd5040 con 0x558ac9fe8720 2024-05-08T23:58:03.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f303114db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac9fee6d0 con 0x558ac9fe2890 2024-05-08T23:58:03.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f3015d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fdcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f301ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe8720 0x558ac9ff3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f301ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe8720 0x558ac9ff3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48940/0 (socket says 172.21.15.12:48940) 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f301ed9d700 1 -- 172.21.15.12:0/883245418 learned_addr learned my addr 172.21.15.12:0/883245418 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f301e59c700 1 --2- 172.21.15.12:0/883245418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 0x558ac9ff7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f301ed9d700 1 -- 172.21.15.12:0/883245418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 msgr2=0x558ac9ff7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f301ed9d700 1 --2- 172.21.15.12:0/883245418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 0x558ac9ff7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f301ed9d700 1 -- 172.21.15.12:0/883245418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 msgr2=0x558ac9fdcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f301ed9d700 1 --2- 172.21.15.12:0/883245418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fdcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f301ed9d700 1 -- 172.21.15.12:0/883245418 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ac9ef8560 con 0x558ac9fe8720 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.165+0000 7f3015d9b700 1 --2- 172.21.15.12:0/883245418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fdcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.166+0000 7f301ed9d700 1 --2- 172.21.15.12:0/883245418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe8720 0x558ac9ff3170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3018095e90 tx=0x7f3018009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a7c73ed3783fd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.166+0000 7f301559a700 1 -- 172.21.15.12:0/883245418 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301801b320 con 0x558ac9fe8720 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.166+0000 7f301559a700 1 -- 172.21.15.12:0/883245418 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3018004ad0 con 0x558ac9fe8720 2024-05-08T23:58:03.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.166+0000 7f301559a700 1 -- 172.21.15.12:0/883245418 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018097e10 con 0x558ac9fe8720 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.166+0000 7f303114db80 1 -- 172.21.15.12:0/883245418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe8720 msgr2=0x558ac9ff3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.166+0000 7f303114db80 1 --2- 172.21.15.12:0/883245418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe8720 0x558ac9ff3170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3018095e90 tx=0x7f3018009370 comp rx=0 tx=0).stop 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 -- 172.21.15.12:0/883245418 shutdown_connections 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 --2- 172.21.15.12:0/883245418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe8720 0x558ac9ff3170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 --2- 172.21.15.12:0/883245418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 0x558ac9ff7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 --2- 172.21.15.12:0/883245418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe7e00 0x558ac9fdcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 -- 172.21.15.12:0/883245418 >> 172.21.15.12:0/883245418 conn(0x558ac9f40210 msgr2=0x558ac9fec440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 -- 172.21.15.12:0/883245418 shutdown_connections 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 -- 172.21.15.12:0/883245418 wait complete. 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 -- start start 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 Processor -- start 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 0x558aca0466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558aca046c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe8720 0x558aca051740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aca047510 con 0x558ac9fe8720 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aca051c80 con 0x558ac9fe7e00 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.167+0000 7f303114db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aca051dc0 con 0x558ac9fe2890 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f303114db80 10 client.0 _ll_register_callbacks cb 0x558ac9ed7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe8720 0x558aca051740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 0x558aca0466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f3015d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558aca046c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe8720 0x558aca051740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46282/0 (socket says 172.21.15.12:46282) 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 0x558aca0466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46286/0 (socket says 172.21.15.12:46286) 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f3015d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558aca046c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48944/0 (socket says 172.21.15.12:48944) 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301ed9d700 1 -- 172.21.15.12:0/2960007353 learned_addr learned my addr 172.21.15.12:0/2960007353 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301ed9d700 1 -- 172.21.15.12:0/2960007353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 msgr2=0x558aca0466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301ed9d700 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 0x558aca0466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301ed9d700 1 -- 172.21.15.12:0/2960007353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 msgr2=0x558aca046c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301ed9d700 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558aca046c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301ed9d700 1 -- 172.21.15.12:0/2960007353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ac9d5fb60 con 0x558ac9fe8720 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f301ed9d700 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe8720 0x558aca051740 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f3018099690 tx=0x7f301801b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cebac0bd52b34f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f3005ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f3005ffb700 1 -- 172.21.15.12:0/2960007353 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018065370 con 0x558ac9fe8720 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f3005ffb700 1 -- 172.21.15.12:0/2960007353 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f301809d810 con 0x558ac9fe8720 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f303114db80 1 -- 172.21.15.12:0/2960007353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3010003df0 con 0x558ac9fe8720 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f3005ffb700 1 -- 172.21.15.12:0/2960007353 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f301809a0e0 con 0x558ac9fe8720 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f303114db80 10 client.18747 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.168+0000 7f303114db80 1 -- 172.21.15.12:0/2960007353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f30201a4410 con 0x558ac9fe8720 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f3005ffb700 1 -- 172.21.15.12:0/2960007353 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3010003df0 con 0x558ac9fe8720 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f3005ffb700 1 -- 172.21.15.12:0/2960007353 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f30180a1940 con 0x558ac9fe8720 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f303114db80 10 client.18747 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:03.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f303114db80 10 client.18747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f303114db80 20 client.18747 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f303114db80 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558aca053040 0x558aca055430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f303114db80 4 client.18747 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f303114db80 1 -- 172.21.15.12:0/2960007353 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558ac9f2aec0 con 0x558aca053040 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f301e59c700 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558aca053040 0x558aca055430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f301e59c700 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558aca053040 0x558aca055430 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37126d1a596287e server_cookie=1d64d129962d53f2 in_seq=0 out_seq=0 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.169+0000 7f3005ffb700 10 client.18747 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f3005ffb700 1 -- 172.21.15.12:0/2960007353 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558ac9f2aec0 con 0x558aca053040 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f3005ffb700 10 client.18747 handle_command_reply: tid=0 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 client.18747 shutdown 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 -- 172.21.15.12:0/2960007353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe8720 msgr2=0x558aca051740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe8720 0x558aca051740 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f3018099690 tx=0x7f301801b320 comp rx=0 tx=0).stop 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 -- 172.21.15.12:0/2960007353 shutdown_connections 2024-05-08T23:58:03.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ac9fe8720 0x558aca051740 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9fe2890 0x558aca0466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558aca053040 0x558aca055430 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 --2- 172.21.15.12:0/2960007353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ac9fe7e00 0x558aca046c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 -- 172.21.15.12:0/2960007353 >> 172.21.15.12:0/2960007353 conn(0x558ac9f40210 msgr2=0x558ac9fe6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:03.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f3005ffb700 0 client.18747 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:03.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 -- 172.21.15.12:0/2960007353 shutdown_connections 2024-05-08T23:58:03.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 1 -- 172.21.15.12:0/2960007353 wait complete. 2024-05-08T23:58:03.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f303114db80 20 client.18747 trim_cache size 0 max 16384 2024-05-08T23:58:03.171 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:58:03.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f3024f93700 1 -- 172.21.15.12:0/1291827817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f300c0a1290 msgr2=0x7f300c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:03.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f3024f93700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f300c0a1290 0x7f300c0a3750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3010011490 tx=0x7f3010009260 comp rx=0 tx=0).stop 2024-05-08T23:58:03.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f3024f93700 1 -- 172.21.15.12:0/1291827817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f302019bcd0 msgr2=0x7f302019c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:03.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.170+0000 7f3024f93700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f302019bcd0 0x7f302019c150 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f301800efc0 tx=0x7f301800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.171+0000 7f3024f93700 1 -- 172.21.15.12:0/1291827817 shutdown_connections 2024-05-08T23:58:03.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.171+0000 7f3024f93700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f300c0a1290 0x7f300c0a3750 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.171+0000 7f3024f93700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f302019bcd0 0x7f302019c150 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.171+0000 7f3024f93700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30200682e0 0x7f30201973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.171+0000 7f3024f93700 1 --2- 172.21.15.12:0/1291827817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3020106290 0x7f3020197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:03.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.171+0000 7f3024f93700 1 -- 172.21.15.12:0/1291827817 >> 172.21.15.12:0/1291827817 conn(0x7f3020076f40 msgr2=0x7f30200fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:03.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.171+0000 7f3024f93700 1 -- 172.21.15.12:0/1291827817 shutdown_connections 2024-05-08T23:58:03.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:03.171+0000 7f3024f93700 1 -- 172.21.15.12:0/1291827817 wait complete. 2024-05-08T23:58:03.892 INFO:tasks.workunit.client.0.smithi012.stdout: 1 640948 6.16 MB/sec execute 479 sec latency 427.326 ms 2024-05-08T23:58:03.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:03 smithi012 ceph-mon[42503]: pgmap v369: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-05-08T23:58:04.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:03 smithi204 ceph-mon[47373]: pgmap v369: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-05-08T23:58:04.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:03 smithi160 ceph-mon[46152]: pgmap v369: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-05-08T23:58:04.180 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:58:04.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:58:04.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.335+0000 7f5352f7f700 1 -- 172.21.15.12:0/2614341817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f534c101160 msgr2=0x7f534c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.335+0000 7f5352f7f700 1 --2- 172.21.15.12:0/2614341817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f534c101160 0x7f534c101540 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5340009990 tx=0x7f5340009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.336+0000 7f5352f7f700 1 -- 172.21.15.12:0/2614341817 shutdown_connections 2024-05-08T23:58:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.336+0000 7f5352f7f700 1 --2- 172.21.15.12:0/2614341817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534c106280 0x7f534c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.336+0000 7f5352f7f700 1 --2- 172.21.15.12:0/2614341817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f534c101160 0x7f534c101540 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.336+0000 7f5352f7f700 1 --2- 172.21.15.12:0/2614341817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f534c101b10 0x7f534c105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.336+0000 7f5352f7f700 1 -- 172.21.15.12:0/2614341817 >> 172.21.15.12:0/2614341817 conn(0x7f534c0fc9e0 msgr2=0x7f534c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.336+0000 7f5352f7f700 1 -- 172.21.15.12:0/2614341817 shutdown_connections 2024-05-08T23:58:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.336+0000 7f5352f7f700 1 -- 172.21.15.12:0/2614341817 wait complete. 2024-05-08T23:58:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.336+0000 7f5352f7f700 1 Processor -- start 2024-05-08T23:58:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.336+0000 7f5352f7f700 1 -- start start 2024-05-08T23:58:04.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5352f7f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f534c101160 0x7f534c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5352f7f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534c106280 0x7f534c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5352f7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f534c19bc70 0x7f534c19c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5352f7f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c19c6f0 con 0x7f534c101160 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5352f7f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c19c860 con 0x7f534c19bc70 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5352f7f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c19ca20 con 0x7f534c106280 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5350d1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f534c101160 0x7f534c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5350d1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f534c101160 0x7f534c1973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46300/0 (socket says 172.21.15.12:46300) 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5350d1b700 1 -- 172.21.15.12:0/1397099289 learned_addr learned my addr 172.21.15.12:0/1397099289 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f534bfff700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534c106280 0x7f534c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f535151c700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f534c19bc70 0x7f534c19c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5350d1b700 1 -- 172.21.15.12:0/1397099289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534c106280 msgr2=0x7f534c197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5350d1b700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534c106280 0x7f534c197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5350d1b700 1 -- 172.21.15.12:0/1397099289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f534c19bc70 msgr2=0x7f534c19c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5350d1b700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f534c19bc70 0x7f534c19c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5350d1b700 1 -- 172.21.15.12:0/1397099289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53400095f0 con 0x7f534c101160 2024-05-08T23:58:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5350d1b700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f534c101160 0x7f534c1973e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5340005770 tx=0x7f53400057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5349ffb700 1 -- 172.21.15.12:0/1397099289 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534001d070 con 0x7f534c101160 2024-05-08T23:58:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.337+0000 7f5349ffb700 1 -- 172.21.15.12:0/1397099289 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f534000fc30 con 0x7f534c101160 2024-05-08T23:58:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.338+0000 7f5352f7f700 1 -- 172.21.15.12:0/1397099289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534c1a3f20 con 0x7f534c101160 2024-05-08T23:58:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.338+0000 7f5349ffb700 1 -- 172.21.15.12:0/1397099289 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340022d50 con 0x7f534c101160 2024-05-08T23:58:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.338+0000 7f5352f7f700 1 -- 172.21.15.12:0/1397099289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c1a4340 con 0x7f534c101160 2024-05-08T23:58:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.340+0000 7f5349ffb700 1 -- 172.21.15.12:0/1397099289 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5340022470 con 0x7f534c101160 2024-05-08T23:58:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.341+0000 7f5349ffb700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f533c0a1310 0x7f533c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.341+0000 7f534bfff700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f533c0a1310 0x7f533c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.341+0000 7f5349ffb700 1 -- 172.21.15.12:0/1397099289 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f53400cddf0 con 0x7f534c101160 2024-05-08T23:58:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.341+0000 7f534bfff700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f533c0a1310 0x7f533c0a37d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f534c1988e0 tx=0x7f533800a300 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.342+0000 7f535f139b80 1 Processor -- start 2024-05-08T23:58:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.342+0000 7f535f139b80 1 -- start start 2024-05-08T23:58:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.342+0000 7f535f139b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.342+0000 7f535f139b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f07130a7a0 0x55f0713043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f535f139b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 0x55f071304d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f535f139b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f07125b1d0 con 0x55f071309e80 2024-05-08T23:58:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f535f139b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071258b90 con 0x55f071304910 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f535f139b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071254480 con 0x55f07130a7a0 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5350d1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5350d1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46302/0 (socket says 172.21.15.12:46302) 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5350d1b700 1 -- 172.21.15.12:0/2918264321 learned_addr learned my addr 172.21.15.12:0/2918264321 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f535151c700 1 --2- 172.21.15.12:0/2918264321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 0x55f071304d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f534bfff700 1 --2- 172.21.15.12:0/2918264321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f07130a7a0 0x55f0713043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5350d1b700 1 -- 172.21.15.12:0/2918264321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f07130a7a0 msgr2=0x55f0713043d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5350d1b700 1 --2- 172.21.15.12:0/2918264321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f07130a7a0 0x55f0713043d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5350d1b700 1 -- 172.21.15.12:0/2918264321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 msgr2=0x55f071304d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5350d1b700 1 --2- 172.21.15.12:0/2918264321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 0x55f071304d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5350d1b700 1 -- 172.21.15.12:0/2918264321 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0710d5430 con 0x55f071309e80 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f535151c700 1 --2- 172.21.15.12:0/2918264321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 0x55f071304d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5350d1b700 1 --2- 172.21.15.12:0/2918264321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130a260 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f5340097ab0 tx=0x7f534009af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d066c3f6d408ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5336ffd700 1 -- 172.21.15.12:0/2918264321 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534009ee70 con 0x55f071309e80 2024-05-08T23:58:04.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5336ffd700 1 -- 172.21.15.12:0/2918264321 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53400a3430 con 0x55f071309e80 2024-05-08T23:58:04.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.343+0000 7f5336ffd700 1 -- 172.21.15.12:0/2918264321 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534009a530 con 0x55f071309e80 2024-05-08T23:58:04.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.344+0000 7f535f139b80 1 -- 172.21.15.12:0/2918264321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 msgr2=0x55f07130a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.344+0000 7f535f139b80 1 --2- 172.21.15.12:0/2918264321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130a260 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f5340097ab0 tx=0x7f534009af40 comp rx=0 tx=0).stop 2024-05-08T23:58:04.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.344+0000 7f535f139b80 1 -- 172.21.15.12:0/2918264321 shutdown_connections 2024-05-08T23:58:04.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.344+0000 7f535f139b80 1 --2- 172.21.15.12:0/2918264321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 0x55f071304d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.344+0000 7f535f139b80 1 --2- 172.21.15.12:0/2918264321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130a260 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.344+0000 7f535f139b80 1 --2- 172.21.15.12:0/2918264321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f07130a7a0 0x55f0713043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.344+0000 7f535f139b80 1 -- 172.21.15.12:0/2918264321 >> 172.21.15.12:0/2918264321 conn(0x55f071262290 msgr2=0x55f0712fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.344+0000 7f535f139b80 1 -- 172.21.15.12:0/2918264321 shutdown_connections 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.344+0000 7f535f139b80 1 -- 172.21.15.12:0/2918264321 wait complete. 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f535f139b80 1 -- start start 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f535f139b80 1 Processor -- start 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f535f139b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071304910 0x55f07130daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f535f139b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f535f139b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f07130a7a0 0x55f07130e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f535f139b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0711f05e0 con 0x55f071309e80 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f535f139b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0711f0280 con 0x55f07130a7a0 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f535f139b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0710d7720 con 0x55f071304910 2024-05-08T23:58:04.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f535f139b80 10 client.0 _ll_register_callbacks cb 0x55f0711f9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f534bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f534bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46310/0 (socket says 172.21.15.12:46310) 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f534bfff700 1 -- 172.21.15.12:0/3214898096 learned_addr learned my addr 172.21.15.12:0/3214898096 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f535151c700 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f07130a7a0 0x55f07130e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.345+0000 7f5350d1b700 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071304910 0x55f07130daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f534bfff700 1 -- 172.21.15.12:0/3214898096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071304910 msgr2=0x55f07130daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f534bfff700 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071304910 0x55f07130daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f534bfff700 1 -- 172.21.15.12:0/3214898096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f07130a7a0 msgr2=0x55f07130e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f534bfff700 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f07130a7a0 0x55f07130e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f534bfff700 1 -- 172.21.15.12:0/3214898096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c1a4340 con 0x55f071309e80 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f534bfff700 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130e030 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f533801d960 tx=0x7f53380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30574dd6a63934a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f532bfff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f532bfff700 1 -- 172.21.15.12:0/3214898096 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5338004a30 con 0x55f071309e80 2024-05-08T23:58:04.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f532bfff700 1 -- 172.21.15.12:0/3214898096 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f5338004b90 con 0x55f071309e80 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f535f139b80 1 -- 172.21.15.12:0/3214898096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0711ad2f0 con 0x55f071309e80 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f532bfff700 1 -- 172.21.15.12:0/3214898096 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5338004d50 con 0x55f071309e80 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f535f139b80 10 client.18759 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f535f139b80 1 -- 172.21.15.12:0/3214898096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55f071081be0 con 0x55f071309e80 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.346+0000 7f532bfff700 1 -- 172.21.15.12:0/3214898096 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f0711ad2f0 con 0x55f071309e80 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.347+0000 7f532bfff700 1 -- 172.21.15.12:0/3214898096 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5338010a70 con 0x55f071309e80 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.347+0000 7f535f139b80 10 client.18759 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.347+0000 7f535f139b80 10 client.18759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.347+0000 7f535f139b80 20 client.18759 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.347+0000 7f535f139b80 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f07131e080 0x55f0713204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.347+0000 7f535f139b80 4 client.18759 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.347+0000 7f535f139b80 1 -- 172.21.15.12:0/3214898096 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0712d3890 con 0x55f07131e080 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.347+0000 7f5350d1b700 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f07131e080 0x55f0713204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.347+0000 7f532bfff700 10 client.18759 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.347+0000 7f5350d1b700 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f07131e080 0x55f0713204a0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04daf74b7411dcd server_cookie=329525b9716b8b4e in_seq=0 out_seq=0 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f532bfff700 1 -- 172.21.15.12:0/3214898096 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0712d3890 con 0x55f07131e080 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f532bfff700 10 client.18759 handle_command_reply: tid=0 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 client.18759 shutdown 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 -- 172.21.15.12:0/3214898096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 msgr2=0x55f07130e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130e030 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f533801d960 tx=0x7f53380047c0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 -- 172.21.15.12:0/3214898096 shutdown_connections 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f07131e080 0x55f0713204a0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f07130a7a0 0x55f07130e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071304910 0x55f07130daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f532bfff700 0 client.18759 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 --2- 172.21.15.12:0/3214898096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071309e80 0x55f07130e030 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 -- 172.21.15.12:0/3214898096 >> 172.21.15.12:0/3214898096 conn(0x55f071262290 msgr2=0x55f0712fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 -- 172.21.15.12:0/3214898096 shutdown_connections 2024-05-08T23:58:04.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 1 -- 172.21.15.12:0/3214898096 wait complete. 2024-05-08T23:58:04.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.348+0000 7f535f139b80 20 client.18759 trim_cache size 0 max 16384 2024-05-08T23:58:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.367+0000 7f535f139b80 1 Processor -- start 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.367+0000 7f535f139b80 1 -- start start 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.367+0000 7f535f139b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 0x55f0712feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.367+0000 7f535f139b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071309e80 0x55f07130e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.367+0000 7f535f139b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f07130a7a0 0x55f07130cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.367+0000 7f535f139b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071305320 con 0x55f07130a7a0 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.367+0000 7f535f139b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0712fbe40 con 0x55f071304910 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.367+0000 7f535f139b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0713006c0 con 0x55f071309e80 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.367+0000 7f535151c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f07130a7a0 0x55f07130cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.367+0000 7f535151c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f07130a7a0 0x55f07130cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46320/0 (socket says 172.21.15.12:46320) 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f535151c700 1 -- 172.21.15.12:0/731476566 learned_addr learned my addr 172.21.15.12:0/731476566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f534bfff700 1 --2- 172.21.15.12:0/731476566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071309e80 0x55f07130e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f5350d1b700 1 --2- 172.21.15.12:0/731476566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 0x55f0712feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f535151c700 1 -- 172.21.15.12:0/731476566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071309e80 msgr2=0x55f07130e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f535151c700 1 --2- 172.21.15.12:0/731476566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071309e80 0x55f07130e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f535151c700 1 -- 172.21.15.12:0/731476566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 msgr2=0x55f0712feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f535151c700 1 --2- 172.21.15.12:0/731476566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 0x55f0712feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f535151c700 1 -- 172.21.15.12:0/731476566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f07121a5e0 con 0x55f07130a7a0 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f535151c700 1 --2- 172.21.15.12:0/731476566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f07130a7a0 0x55f07130cad0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5344014600 tx=0x7f5344007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9a506191bfdde5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f53377fe700 1 -- 172.21.15.12:0/731476566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344014ea0 con 0x55f07130a7a0 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f53377fe700 1 -- 172.21.15.12:0/731476566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5344016af0 con 0x55f07130a7a0 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.368+0000 7f53377fe700 1 -- 172.21.15.12:0/731476566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534401fd30 con 0x55f07130a7a0 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 -- 172.21.15.12:0/731476566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f07130a7a0 msgr2=0x55f07130cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 --2- 172.21.15.12:0/731476566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f07130a7a0 0x55f07130cad0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5344014600 tx=0x7f5344007950 comp rx=0 tx=0).stop 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 -- 172.21.15.12:0/731476566 shutdown_connections 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 --2- 172.21.15.12:0/731476566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f07130a7a0 0x55f07130cad0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 --2- 172.21.15.12:0/731476566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f071304910 0x55f0712feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 --2- 172.21.15.12:0/731476566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071309e80 0x55f07130e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 -- 172.21.15.12:0/731476566 >> 172.21.15.12:0/731476566 conn(0x55f071262290 msgr2=0x55f07130c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 -- 172.21.15.12:0/731476566 shutdown_connections 2024-05-08T23:58:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 -- 172.21.15.12:0/731476566 wait complete. 2024-05-08T23:58:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 -- start start 2024-05-08T23:58:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.369+0000 7f535f139b80 1 Processor -- start 2024-05-08T23:58:04.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f535f139b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071304910 0x55f071313d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f535f139b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071309e80 0x55f0713142b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f535f139b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f07130a7a0 0x55f071314810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f535f139b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071374790 con 0x55f071304910 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f535f139b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071374900 con 0x55f07130a7a0 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f535f139b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071374ac0 con 0x55f071309e80 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f535f139b80 10 client.0 _ll_register_callbacks cb 0x55f0711f9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f5350d1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071304910 0x55f071313d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f5350d1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071304910 0x55f071313d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46328/0 (socket says 172.21.15.12:46328) 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f5350d1b700 1 -- 172.21.15.12:0/2650050222 learned_addr learned my addr 172.21.15.12:0/2650050222 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f535151c700 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f07130a7a0 0x55f071314810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f534bfff700 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071309e80 0x55f0713142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f5350d1b700 1 -- 172.21.15.12:0/2650050222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071309e80 msgr2=0x55f0713142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f5350d1b700 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071309e80 0x55f0713142b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f5350d1b700 1 -- 172.21.15.12:0/2650050222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f07130a7a0 msgr2=0x55f071314810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f5350d1b700 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f07130a7a0 0x55f071314810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.370+0000 7f5350d1b700 1 -- 172.21.15.12:0/2650050222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5344014340 con 0x55f071304910 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f5350d1b700 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071304910 0x55f071313d70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5340006010 tx=0x7f53400b0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d98d810a4c03c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f532bfff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f532bfff700 1 -- 172.21.15.12:0/2650050222 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53400b9040 con 0x55f071304910 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f532bfff700 1 -- 172.21.15.12:0/2650050222 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f53400b8040 con 0x55f071304910 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f535f139b80 1 -- 172.21.15.12:0/2650050222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53400a3430 con 0x55f071304910 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f532bfff700 1 -- 172.21.15.12:0/2650050222 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5340097380 con 0x55f071304910 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f535f139b80 10 client.18765 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f535f139b80 1 -- 172.21.15.12:0/2650050222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55f0710d9d50 con 0x55f071304910 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f532bfff700 1 -- 172.21.15.12:0/2650050222 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53400a3430 con 0x55f071304910 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f532bfff700 1 -- 172.21.15.12:0/2650050222 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f534009a630 con 0x55f071304910 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f535f139b80 10 client.18765 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f535f139b80 10 client.18765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f535f139b80 20 client.18765 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f535f139b80 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f071375de0 0x55f071378200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f535f139b80 4 client.18765 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:58:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.371+0000 7f535f139b80 1 -- 172.21.15.12:0/2650050222 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f07124cf40 con 0x55f071375de0 2024-05-08T23:58:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.372+0000 7f534bfff700 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f071375de0 0x55f071378200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.372+0000 7f534bfff700 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f071375de0 0x55f071378200 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec2d63757f42a00 server_cookie=a4503ae78d583327 in_seq=0 out_seq=0 2024-05-08T23:58:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.372+0000 7f532bfff700 10 client.18765 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f532bfff700 1 -- 172.21.15.12:0/2650050222 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f07124cf40 con 0x55f071375de0 2024-05-08T23:58:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f532bfff700 10 client.18765 handle_command_reply: tid=0 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 client.18765 shutdown 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 -- 172.21.15.12:0/2650050222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071304910 msgr2=0x55f071313d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071304910 0x55f071313d70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5340006010 tx=0x7f53400b0040 comp rx=0 tx=0).stop 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 -- 172.21.15.12:0/2650050222 shutdown_connections 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f071375de0 0x55f071378200 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f07130a7a0 0x55f071314810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f071304910 0x55f071313d70 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 --2- 172.21.15.12:0/2650050222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f071309e80 0x55f0713142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 -- 172.21.15.12:0/2650050222 >> 172.21.15.12:0/2650050222 conn(0x55f071262290 msgr2=0x55f07130ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f532bfff700 0 client.18765 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 -- 172.21.15.12:0/2650050222 shutdown_connections 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.373+0000 7f535f139b80 1 -- 172.21.15.12:0/2650050222 wait complete. 2024-05-08T23:58:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f535f139b80 20 client.18765 trim_cache size 0 max 16384 2024-05-08T23:58:04.375 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:58:04.375 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:58:04.375 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "f6178a64-3e75-4ba3-9992-1efd55a36fc7", 2024-05-08T23:58:04.375 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:58:04.375 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 -- 172.21.15.12:0/1397099289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f533c0a1310 msgr2=0x7f533c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f533c0a1310 0x7f533c0a37d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f534c1988e0 tx=0x7f533800a300 comp rx=0 tx=0).stop 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 -- 172.21.15.12:0/1397099289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f534c101160 msgr2=0x7f534c1973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f534c101160 0x7f534c1973e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5340005770 tx=0x7f53400057a0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 -- 172.21.15.12:0/1397099289 shutdown_connections 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f533c0a1310 0x7f533c0a37d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f534c19bc70 0x7f534c19c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f534c101160 0x7f534c1973e0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 --2- 172.21.15.12:0/1397099289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f534c106280 0x7f534c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 -- 172.21.15.12:0/1397099289 >> 172.21.15.12:0/1397099289 conn(0x7f534c0fc9e0 msgr2=0x7f534c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 -- 172.21.15.12:0/1397099289 shutdown_connections 2024-05-08T23:58:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.374+0000 7f5352f7f700 1 -- 172.21.15.12:0/1397099289 wait complete. 2024-05-08T23:58:04.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:58:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 -- 172.21.15.12:0/3664027808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33280682e0 msgr2=0x7f33280686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 --2- 172.21.15.12:0/3664027808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33280682e0 0x7f33280686c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f331c0099c0 tx=0x7f331c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 -- 172.21.15.12:0/3664027808 shutdown_connections 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 --2- 172.21.15.12:0/3664027808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3328106290 0x7f332810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 --2- 172.21.15.12:0/3664027808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33280682e0 0x7f33280686c0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 --2- 172.21.15.12:0/3664027808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3328068c90 0x7f3328105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 -- 172.21.15.12:0/3664027808 >> 172.21.15.12:0/3664027808 conn(0x7f3328076f40 msgr2=0x7f3328077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 -- 172.21.15.12:0/3664027808 shutdown_connections 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 -- 172.21.15.12:0/3664027808 wait complete. 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 Processor -- start 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.532+0000 7f332ee50700 1 -- start start 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f332ee50700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33280682e0 0x7f33281973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f332ee50700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3328106290 0x7f33281978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f332ee50700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f332819bc80 0x7f332819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f332ee50700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332819c700 con 0x7f3328106290 2024-05-08T23:58:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f332ee50700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332819c870 con 0x7f33280682e0 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f332ee50700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332819ca30 con 0x7f332819bc80 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f3327fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3328106290 0x7f33281978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f332cbec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33280682e0 0x7f33281973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f3327fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3328106290 0x7f33281978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46344/0 (socket says 172.21.15.12:46344) 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f3327fff700 1 -- 172.21.15.12:0/2948452219 learned_addr learned my addr 172.21.15.12:0/2948452219 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f332d3ed700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f332819bc80 0x7f332819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f3327fff700 1 -- 172.21.15.12:0/2948452219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f332819bc80 msgr2=0x7f332819c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f3327fff700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f332819bc80 0x7f332819c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f3327fff700 1 -- 172.21.15.12:0/2948452219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33280682e0 msgr2=0x7f33281973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f3327fff700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33280682e0 0x7f33281973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f3327fff700 1 -- 172.21.15.12:0/2948452219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0095f0 con 0x7f3328106290 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f332cbec700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33280682e0 0x7f33281973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.533+0000 7f3327fff700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3328106290 0x7f33281978e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f331400e970 tx=0x7f331400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.534+0000 7f3325ffb700 1 -- 172.21.15.12:0/2948452219 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314009790 con 0x7f3328106290 2024-05-08T23:58:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.534+0000 7f332ee50700 1 -- 172.21.15.12:0/2948452219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33281a3f90 con 0x7f3328106290 2024-05-08T23:58:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.534+0000 7f3325ffb700 1 -- 172.21.15.12:0/2948452219 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f331400c7e0 con 0x7f3328106290 2024-05-08T23:58:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.534+0000 7f3325ffb700 1 -- 172.21.15.12:0/2948452219 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33140054e0 con 0x7f3328106290 2024-05-08T23:58:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.534+0000 7f332ee50700 1 -- 172.21.15.12:0/2948452219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33281a4460 con 0x7f3328106290 2024-05-08T23:58:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.536+0000 7f3325ffb700 1 -- 172.21.15.12:0/2948452219 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f331400c950 con 0x7f3328106290 2024-05-08T23:58:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.537+0000 7f3325ffb700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33180a1290 0x7f33180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.537+0000 7f3325ffb700 1 -- 172.21.15.12:0/2948452219 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f3314014070 con 0x7f3328106290 2024-05-08T23:58:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.537+0000 7f332cbec700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33180a1290 0x7f33180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.537+0000 7f332cbec700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33180a1290 0x7f33180a3750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f331c009fd0 tx=0x7f331c005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f333b00ab80 1 Processor -- start 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f333b00ab80 1 -- start start 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea01350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642bea01cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f333b00ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642be958150 con 0x5642bea06e00 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f333b00ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642be955b10 con 0x5642bea01890 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f333b00ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642be951400 con 0x5642bea07720 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f332cbec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f3327fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea01350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f3327fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea01350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46376/0 (socket says 172.21.15.12:46376) 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f3327fff700 1 -- 172.21.15.12:0/4287666935 learned_addr learned my addr 172.21.15.12:0/4287666935 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.538+0000 7f332d3ed700 1 --2- 172.21.15.12:0/4287666935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642bea01cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.539+0000 7f3327fff700 1 -- 172.21.15.12:0/4287666935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 msgr2=0x5642bea01cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.539+0000 7f3327fff700 1 --2- 172.21.15.12:0/4287666935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642bea01cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.539+0000 7f3327fff700 1 -- 172.21.15.12:0/4287666935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 msgr2=0x5642bea071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.539+0000 7f3327fff700 1 --2- 172.21.15.12:0/4287666935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.539+0000 7f3327fff700 1 -- 172.21.15.12:0/4287666935 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642be7d23b0 con 0x5642bea07720 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.539+0000 7f332d3ed700 1 --2- 172.21.15.12:0/4287666935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642bea01cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.539+0000 7f3327fff700 1 --2- 172.21.15.12:0/4287666935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea01350 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f331400c4e0 tx=0x7f331400ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4987b74d3408b547 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.539+0000 7f3312ffd700 1 -- 172.21.15.12:0/4287666935 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33140b1070 con 0x5642bea07720 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.539+0000 7f3312ffd700 1 -- 172.21.15.12:0/4287666935 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f331409de60 con 0x5642bea07720 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.539+0000 7f3312ffd700 1 -- 172.21.15.12:0/4287666935 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331409a290 con 0x5642bea07720 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 -- 172.21.15.12:0/4287666935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 msgr2=0x5642bea01350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 --2- 172.21.15.12:0/4287666935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea01350 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f331400c4e0 tx=0x7f331400ccd0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 -- 172.21.15.12:0/4287666935 shutdown_connections 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 --2- 172.21.15.12:0/4287666935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642bea01cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 --2- 172.21.15.12:0/4287666935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 --2- 172.21.15.12:0/4287666935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea01350 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 -- 172.21.15.12:0/4287666935 >> 172.21.15.12:0/4287666935 conn(0x5642be95f210 msgr2=0x5642be9fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 -- 172.21.15.12:0/4287666935 shutdown_connections 2024-05-08T23:58:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 -- 172.21.15.12:0/4287666935 wait complete. 2024-05-08T23:58:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 -- start start 2024-05-08T23:58:04.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.540+0000 7f333b00ab80 1 Processor -- start 2024-05-08T23:58:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea01890 0x5642bea0abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea06e00 0x5642bea0b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea07720 0x5642bea0b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f333b00ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642be8ed560 con 0x5642bea01890 2024-05-08T23:58:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f333b00ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642be8ed200 con 0x5642bea07720 2024-05-08T23:58:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f333b00ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642be7d46a0 con 0x5642bea06e00 2024-05-08T23:58:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f333b00ab80 10 client.0 _ll_register_callbacks cb 0x5642be8f6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f3327fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea06e00 0x5642bea0b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f332cbec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea01890 0x5642bea0abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f3327fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea06e00 0x5642bea0b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46378/0 (socket says 172.21.15.12:46378) 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f3327fff700 1 -- 172.21.15.12:0/3953137325 learned_addr learned my addr 172.21.15.12:0/3953137325 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f332d3ed700 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea07720 0x5642bea0b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f332cbec700 1 -- 172.21.15.12:0/3953137325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea06e00 msgr2=0x5642bea0b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f332cbec700 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea06e00 0x5642bea0b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f332cbec700 1 -- 172.21.15.12:0/3953137325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea07720 msgr2=0x5642bea0b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f332cbec700 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea07720 0x5642bea0b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.541+0000 7f332cbec700 1 -- 172.21.15.12:0/3953137325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33281a4460 con 0x5642bea01890 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.542+0000 7f332cbec700 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea01890 0x5642bea0abb0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f331c0058e0 tx=0x7f331c019050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2db4a8b0a8c058b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.542+0000 7f3307fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.542+0000 7f3307fff700 1 -- 172.21.15.12:0/3953137325 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c02a050 con 0x5642bea01890 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.542+0000 7f3307fff700 1 -- 172.21.15.12:0/3953137325 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f331c00f660 con 0x5642bea01890 2024-05-08T23:58:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.542+0000 7f333b00ab80 1 -- 172.21.15.12:0/3953137325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642be8aa270 con 0x5642bea01890 2024-05-08T23:58:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.542+0000 7f3307fff700 1 -- 172.21.15.12:0/3953137325 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f331c00f820 con 0x5642bea01890 2024-05-08T23:58:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.542+0000 7f333b00ab80 10 client.18777 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.542+0000 7f333b00ab80 1 -- 172.21.15.12:0/3953137325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5642be77eb60 con 0x5642bea01890 2024-05-08T23:58:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.542+0000 7f3307fff700 1 -- 172.21.15.12:0/3953137325 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5642be8aa270 con 0x5642bea01890 2024-05-08T23:58:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.542+0000 7f3307fff700 1 -- 172.21.15.12:0/3953137325 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f331c023a50 con 0x5642bea01890 2024-05-08T23:58:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.543+0000 7f333b00ab80 10 client.18777 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.543+0000 7f333b00ab80 10 client.18777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.543+0000 7f333b00ab80 20 client.18777 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.543+0000 7f333b00ab80 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642bea1b330 0x5642bea1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.543+0000 7f333b00ab80 4 client.18777 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.543+0000 7f333b00ab80 1 -- 172.21.15.12:0/3953137325 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642be9d0810 con 0x5642bea1b330 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.543+0000 7f3327fff700 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642bea1b330 0x5642bea1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.543+0000 7f3327fff700 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642bea1b330 0x5642bea1d720 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44141bd97811a5cd server_cookie=da18a060a5336f3e in_seq=0 out_seq=0 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.543+0000 7f3307fff700 10 client.18777 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f3307fff700 1 -- 172.21.15.12:0/3953137325 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642be9d0810 con 0x5642bea1b330 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f3307fff700 10 client.18777 handle_command_reply: tid=0 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 client.18777 shutdown 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 -- 172.21.15.12:0/3953137325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea01890 msgr2=0x5642bea0abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea01890 0x5642bea0abb0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f331c0058e0 tx=0x7f331c019050 comp rx=0 tx=0).stop 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 -- 172.21.15.12:0/3953137325 shutdown_connections 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642bea1b330 0x5642bea1d720 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea07720 0x5642bea0b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea01890 0x5642bea0abb0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 --2- 172.21.15.12:0/3953137325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea06e00 0x5642bea0b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f3307fff700 0 client.18777 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:04.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 -- 172.21.15.12:0/3953137325 >> 172.21.15.12:0/3953137325 conn(0x5642be95f210 msgr2=0x5642be9fb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 -- 172.21.15.12:0/3953137325 shutdown_connections 2024-05-08T23:58:04.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 1 -- 172.21.15.12:0/3953137325 wait complete. 2024-05-08T23:58:04.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.544+0000 7f333b00ab80 20 client.18777 trim_cache size 0 max 16384 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f333b00ab80 1 Processor -- start 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f333b00ab80 1 -- start start 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642be9fbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea0b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea0b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f333b00ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bea0bb80 con 0x5642bea06e00 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f333b00ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bea1b150 con 0x5642bea01890 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f333b00ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bea17e20 con 0x5642bea07720 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f3327fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea0b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f3327fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea0b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46366/0 (socket says 172.21.15.12:46366) 2024-05-08T23:58:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f3327fff700 1 -- 172.21.15.12:0/3396854340 learned_addr learned my addr 172.21.15.12:0/3396854340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f332d3ed700 1 --2- 172.21.15.12:0/3396854340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea0b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.564+0000 7f332cbec700 1 --2- 172.21.15.12:0/3396854340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642be9fbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.565+0000 7f3327fff700 1 -- 172.21.15.12:0/3396854340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 msgr2=0x5642bea0b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.565+0000 7f3327fff700 1 --2- 172.21.15.12:0/3396854340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea0b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.565+0000 7f3327fff700 1 -- 172.21.15.12:0/3396854340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 msgr2=0x5642be9fbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.565+0000 7f3327fff700 1 --2- 172.21.15.12:0/3396854340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642be9fbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.565+0000 7f3327fff700 1 -- 172.21.15.12:0/3396854340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642be917560 con 0x5642bea06e00 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.565+0000 7f3327fff700 1 --2- 172.21.15.12:0/3396854340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea0b100 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f331400efd0 tx=0x7f331400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177f2f4a5ff724ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.565+0000 7f33137fe700 1 -- 172.21.15.12:0/3396854340 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33140b1040 con 0x5642bea06e00 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.565+0000 7f33137fe700 1 -- 172.21.15.12:0/3396854340 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3314005660 con 0x5642bea06e00 2024-05-08T23:58:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.565+0000 7f33137fe700 1 -- 172.21.15.12:0/3396854340 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33140b2e40 con 0x5642bea06e00 2024-05-08T23:58:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 -- 172.21.15.12:0/3396854340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 msgr2=0x5642bea0b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 --2- 172.21.15.12:0/3396854340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea0b100 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f331400efd0 tx=0x7f331400cf50 comp rx=0 tx=0).stop 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 -- 172.21.15.12:0/3396854340 shutdown_connections 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 --2- 172.21.15.12:0/3396854340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea0b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 --2- 172.21.15.12:0/3396854340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642be9fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 --2- 172.21.15.12:0/3396854340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea0b100 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 -- 172.21.15.12:0/3396854340 >> 172.21.15.12:0/3396854340 conn(0x5642be95f210 msgr2=0x5642be9f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 -- 172.21.15.12:0/3396854340 shutdown_connections 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 -- 172.21.15.12:0/3396854340 wait complete. 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 -- start start 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.566+0000 7f333b00ab80 1 Processor -- start 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642bea10340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea10880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f333b00ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea70800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f333b00ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bea70d40 con 0x5642bea06e00 2024-05-08T23:58:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f333b00ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bea70e80 con 0x5642bea01890 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f333b00ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bea70fc0 con 0x5642bea07720 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f333b00ab80 10 client.0 _ll_register_callbacks cb 0x5642be8f6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f3327fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea10880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f332cbec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642bea10340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f332d3ed700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea70800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f3327fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea10880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46372/0 (socket says 172.21.15.12:46372) 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f3327fff700 1 -- 172.21.15.12:0/4156694668 learned_addr learned my addr 172.21.15.12:0/4156694668 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f332cbec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642bea10340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49054/0 (socket says 172.21.15.12:49054) 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f332d3ed700 1 -- 172.21.15.12:0/4156694668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 msgr2=0x5642bea10340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f332d3ed700 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642bea10340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f332d3ed700 1 -- 172.21.15.12:0/4156694668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 msgr2=0x5642bea10880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f332d3ed700 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea10880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f332d3ed700 1 -- 172.21.15.12:0/4156694668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642be7d23b0 con 0x5642bea07720 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f3327fff700 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea10880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f332d3ed700 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea70800 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3320007ca0 tx=0x7f3320007cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5823cdb84ca041d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.567+0000 7f3307fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f3307fff700 1 -- 172.21.15.12:0/4156694668 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332000e040 con 0x5642bea07720 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f333b00ab80 1 -- 172.21.15.12:0/4156694668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f331409de60 con 0x5642bea07720 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f3307fff700 1 -- 172.21.15.12:0/4156694668 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f3320004db0 con 0x5642bea07720 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f3307fff700 1 -- 172.21.15.12:0/4156694668 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3320018aa0 con 0x5642bea07720 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f333b00ab80 10 client.28498 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f333b00ab80 1 -- 172.21.15.12:0/4156694668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5642be7d6cd0 con 0x5642bea07720 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f3307fff700 1 -- 172.21.15.12:0/4156694668 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f332002c080 con 0x5642bea07720 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f3307fff700 1 -- 172.21.15.12:0/4156694668 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3320028ca0 con 0x5642bea07720 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f333b00ab80 10 client.28498 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f333b00ab80 10 client.28498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f333b00ab80 20 client.28498 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f333b00ab80 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642bea72830 0x5642bea74c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f333b00ab80 4 client.28498 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f333b00ab80 1 -- 172.21.15.12:0/4156694668 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5642be949ec0 con 0x5642bea72830 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.568+0000 7f332cbec700 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642bea72830 0x5642bea74c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f3307fff700 10 client.28498 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f332cbec700 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642bea72830 0x5642bea74c50 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2db4820494ded7d server_cookie=5ae2291a8069d4fb in_seq=0 out_seq=0 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f3307fff700 1 -- 172.21.15.12:0/4156694668 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5642be949ec0 con 0x5642bea72830 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f3307fff700 10 client.28498 handle_command_reply: tid=0 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 client.28498 shutdown 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 -- 172.21.15.12:0/4156694668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 msgr2=0x5642bea70800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea70800 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3320007ca0 tx=0x7f3320007cd0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 -- 172.21.15.12:0/4156694668 shutdown_connections 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642bea07720 0x5642bea70800 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642bea72830 0x5642bea74c50 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642bea01890 0x5642bea10340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 --2- 172.21.15.12:0/4156694668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642bea06e00 0x5642bea10880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f3307fff700 0 client.28498 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 -- 172.21.15.12:0/4156694668 >> 172.21.15.12:0/4156694668 conn(0x5642be95f210 msgr2=0x5642bea093c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 -- 172.21.15.12:0/4156694668 shutdown_connections 2024-05-08T23:58:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 1 -- 172.21.15.12:0/4156694668 wait complete. 2024-05-08T23:58:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.569+0000 7f333b00ab80 20 client.28498 trim_cache size 0 max 16384 2024-05-08T23:58:04.571 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:58:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:58:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:58:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: "f6178a64-3e75-4ba3-9992-1efd55a36fc7": { 2024-05-08T23:58:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:58:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "f6178a64-3e75-4ba3-9992-1efd55a36fc7", 2024-05-08T23:58:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:58:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:58:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:58:04.572 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:58:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 -- 172.21.15.12:0/2948452219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33180a1290 msgr2=0x7f33180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33180a1290 0x7f33180a3750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f331c009fd0 tx=0x7f331c005fb0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 -- 172.21.15.12:0/2948452219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3328106290 msgr2=0x7f33281978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3328106290 0x7f33281978e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f331400e970 tx=0x7f331400ec80 comp rx=0 tx=0).stop 2024-05-08T23:58:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 -- 172.21.15.12:0/2948452219 shutdown_connections 2024-05-08T23:58:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33180a1290 0x7f33180a3750 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f332819bc80 0x7f332819c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33280682e0 0x7f33281973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 --2- 172.21.15.12:0/2948452219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3328106290 0x7f33281978e0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 -- 172.21.15.12:0/2948452219 >> 172.21.15.12:0/2948452219 conn(0x7f3328076f40 msgr2=0x7f33280fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 -- 172.21.15.12:0/2948452219 shutdown_connections 2024-05-08T23:58:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:04.570+0000 7f332ee50700 1 -- 172.21.15.12:0/2948452219 wait complete. 2024-05-08T23:58:04.579 INFO:tasks.cephfs.filesystem:scrub status for tag:f6178a64-3e75-4ba3-9992-1efd55a36fc7 - {'path': '/', 'tag': 'f6178a64-3e75-4ba3-9992-1efd55a36fc7', 'options': 'recursive,force'} 2024-05-08T23:58:04.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 642280 6.17 MB/sec execute 480 sec latency 924.394 ms 2024-05-08T23:58:05.023 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:04 smithi160 ceph-mon[46152]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T23:58:05.023 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:04 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:58:05.023 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:04 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:05.024 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:04 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:58:05.024 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:04 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:05.024 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:04 smithi204 ceph-mon[47373]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T23:58:05.024 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:04 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:58:05.024 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:04 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:05.024 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:04 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:58:05.024 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:04 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:05.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:04 smithi012 ceph-mon[42503]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T23:58:05.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:04 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:58:05.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:04 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:05.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:04 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:58:05.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:04 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:05.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 643779 6.17 MB/sec execute 481 sec latency 17.355 ms 2024-05-08T23:58:05.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:05 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:58:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:05 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:58:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:05 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:05 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:58:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:05 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:05 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:05.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:05 smithi012 ceph-mon[42503]: pgmap v371: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.8 MiB/s wr, 167 op/s 2024-05-08T23:58:06.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:05 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:58:06.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:05 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:58:06.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:05 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:06.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:05 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:58:06.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:05 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:06.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:05 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:06.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:05 smithi204 ceph-mon[47373]: pgmap v371: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.8 MiB/s wr, 167 op/s 2024-05-08T23:58:06.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:05 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:58:06.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:05 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:58:06.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:05 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:06.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:05 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:58:06.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:05 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:06.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:05 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:06.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:05 smithi160 ceph-mon[46152]: pgmap v371: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.8 MiB/s wr, 167 op/s 2024-05-08T23:58:06.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 644492 6.16 MB/sec execute 482 sec latency 590.955 ms 2024-05-08T23:58:07.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 644492 6.15 MB/sec execute 483 sec latency 1591.059 ms 2024-05-08T23:58:07.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:07 smithi012 ceph-mon[42503]: pgmap v372: 97 pgs: 97 active+clean; 196 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-08T23:58:08.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:07 smithi204 ceph-mon[47373]: pgmap v372: 97 pgs: 97 active+clean; 196 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-08T23:58:08.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:07 smithi160 ceph-mon[46152]: pgmap v372: 97 pgs: 97 active+clean; 196 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-08T23:58:08.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 644492 6.14 MB/sec execute 484 sec latency 2591.111 ms 2024-05-08T23:58:09.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 644492 6.13 MB/sec execute 485 sec latency 3591.188 ms 2024-05-08T23:58:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:09 smithi012 ceph-mon[42503]: pgmap v373: 97 pgs: 97 active+clean; 196 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-08T23:58:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:09 smithi204 ceph-mon[47373]: pgmap v373: 97 pgs: 97 active+clean; 196 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-08T23:58:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:09 smithi160 ceph-mon[46152]: pgmap v373: 97 pgs: 97 active+clean; 196 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-08T23:58:10.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 644492 6.11 MB/sec execute 486 sec latency 4591.275 ms 2024-05-08T23:58:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.152+0000 7f57ce9d4700 1 -- 172.21.15.12:0/32597400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c80682e0 msgr2=0x7f57c8068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.152+0000 7f57ce9d4700 1 --2- 172.21.15.12:0/32597400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c80682e0 0x7f57c8068760 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009990 tx=0x7f57bc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.152+0000 7f57ce9d4700 1 -- 172.21.15.12:0/32597400 shutdown_connections 2024-05-08T23:58:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.152+0000 7f57ce9d4700 1 --2- 172.21.15.12:0/32597400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57c8068ca0 0x7f57c810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.152+0000 7f57ce9d4700 1 --2- 172.21.15.12:0/32597400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57c8107000 0x7f57c81073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.152+0000 7f57ce9d4700 1 --2- 172.21.15.12:0/32597400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c80682e0 0x7f57c8068760 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.152+0000 7f57ce9d4700 1 -- 172.21.15.12:0/32597400 >> 172.21.15.12:0/32597400 conn(0x7f57c8076f10 msgr2=0x7f57c8077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.152+0000 7f57ce9d4700 1 -- 172.21.15.12:0/32597400 shutdown_connections 2024-05-08T23:58:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.152+0000 7f57ce9d4700 1 -- 172.21.15.12:0/32597400 wait complete. 2024-05-08T23:58:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57ce9d4700 1 Processor -- start 2024-05-08T23:58:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57ce9d4700 1 -- start start 2024-05-08T23:58:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57ce9d4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57c80682e0 0x7f57c80ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57ce9d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57c8068ca0 0x7f57c8100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57ce9d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8107000 0x7f57c8100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57ce9d4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c81a5680 con 0x7f57c8107000 2024-05-08T23:58:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57ce9d4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c81a57f0 con 0x7f57c8068ca0 2024-05-08T23:58:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57ce9d4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c81a59b0 con 0x7f57c80682e0 2024-05-08T23:58:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57c80682e0 0x7f57c80ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57ccf71700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8107000 0x7f57c8100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57c80682e0 0x7f57c80ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59620/0 (socket says 172.21.15.12:59620) 2024-05-08T23:58:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57ccf71700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8107000 0x7f57c8100890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58128/0 (socket says 172.21.15.12:58128) 2024-05-08T23:58:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57c7fff700 1 -- 172.21.15.12:0/898742808 learned_addr learned my addr 172.21.15.12:0/898742808 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.153+0000 7f57c77fe700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57c8068ca0 0x7f57c8100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57c7fff700 1 -- 172.21.15.12:0/898742808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57c8068ca0 msgr2=0x7f57c8100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57c7fff700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57c8068ca0 0x7f57c8100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57c7fff700 1 -- 172.21.15.12:0/898742808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8107000 msgr2=0x7f57c8100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57c7fff700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8107000 0x7f57c8100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57c7fff700 1 -- 172.21.15.12:0/898742808 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc0095f0 con 0x7f57c80682e0 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57c7fff700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57c80682e0 0x7f57c80ffe10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f57b800e9a0 tx=0x7f57b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57c57fa700 1 -- 172.21.15.12:0/898742808 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8009790 con 0x7f57c80682e0 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57ce9d4700 1 -- 172.21.15.12:0/898742808 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c81a5c90 con 0x7f57c80682e0 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57c57fa700 1 -- 172.21.15.12:0/898742808 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57b8010430 con 0x7f57c80682e0 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57c57fa700 1 -- 172.21.15.12:0/898742808 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8018680 con 0x7f57c80682e0 2024-05-08T23:58:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.154+0000 7f57ce9d4700 1 -- 172.21.15.12:0/898742808 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81a61c0 con 0x7f57c80682e0 2024-05-08T23:58:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.157+0000 7f57c57fa700 1 -- 172.21.15.12:0/898742808 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f57b80105a0 con 0x7f57c80682e0 2024-05-08T23:58:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.158+0000 7f57c57fa700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57b00a1310 0x7f57b00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.158+0000 7f57c57fa700 1 -- 172.21.15.12:0/898742808 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f57b8014070 con 0x7f57c80682e0 2024-05-08T23:58:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.158+0000 7f57c77fe700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57b00a1310 0x7f57b00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.159+0000 7f57c77fe700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57b00a1310 0x7f57b00a37d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00b2e0 tx=0x7f57bc005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.159+0000 7f57dab8eb80 1 Processor -- start 2024-05-08T23:58:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.159+0000 7f57dab8eb80 1 -- start start 2024-05-08T23:58:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.159+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031dcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.159+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031d6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.159+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031d78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.159+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b10312e0f0 con 0x55b1031d7480 2024-05-08T23:58:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.159+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b10312bab0 con 0x55b1031dd270 2024-05-08T23:58:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.159+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1031273a0 con 0x55b1031dc950 2024-05-08T23:58:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031dcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031d6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031dcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59630/0 (socket says 172.21.15.12:59630) 2024-05-08T23:58:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57ccf71700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031d78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c7fff700 1 -- 172.21.15.12:0/1605375768 learned_addr learned my addr 172.21.15.12:0/1605375768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c77fe700 1 -- 172.21.15.12:0/1605375768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 msgr2=0x55b1031dcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c77fe700 1 --2- 172.21.15.12:0/1605375768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031dcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c77fe700 1 -- 172.21.15.12:0/1605375768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 msgr2=0x55b1031d78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c77fe700 1 --2- 172.21.15.12:0/1605375768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031d78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c77fe700 1 -- 172.21.15.12:0/1605375768 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b102fa8350 con 0x55b1031dd270 2024-05-08T23:58:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c7fff700 1 --2- 172.21.15.12:0/1605375768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031dcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57c77fe700 1 --2- 172.21.15.12:0/1605375768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031d6f40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0059d0 tx=0x7f57bc020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84e19279d72ad777 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57ae7fc700 1 -- 172.21.15.12:0/1605375768 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00be90 con 0x55b1031dd270 2024-05-08T23:58:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57ae7fc700 1 -- 172.21.15.12:0/1605375768 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57bc01e040 con 0x55b1031dd270 2024-05-08T23:58:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.160+0000 7f57ae7fc700 1 -- 172.21.15.12:0/1605375768 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00f790 con 0x55b1031dd270 2024-05-08T23:58:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.161+0000 7f57dab8eb80 1 -- 172.21.15.12:0/1605375768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 msgr2=0x55b1031d6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.161+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/1605375768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031d6f40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0059d0 tx=0x7f57bc020000 comp rx=0 tx=0).stop 2024-05-08T23:58:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.161+0000 7f57dab8eb80 1 -- 172.21.15.12:0/1605375768 shutdown_connections 2024-05-08T23:58:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.161+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/1605375768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031d78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.161+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/1605375768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031dcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.161+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/1605375768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031d6f40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.161+0000 7f57dab8eb80 1 -- 172.21.15.12:0/1605375768 >> 172.21.15.12:0/1605375768 conn(0x55b1031351b0 msgr2=0x55b1031d1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.161+0000 7f57dab8eb80 1 -- 172.21.15.12:0/1605375768 shutdown_connections 2024-05-08T23:58:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.161+0000 7f57dab8eb80 1 -- 172.21.15.12:0/1605375768 wait complete. 2024-05-08T23:58:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.162+0000 7f57dab8eb80 1 -- start start 2024-05-08T23:58:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.162+0000 7f57dab8eb80 1 Processor -- start 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.162+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031d7480 0x55b1031e3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.162+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031e0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.162+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031dd270 0x55b1031e0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.162+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1030c3500 con 0x55b1031dd270 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.162+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1030c31a0 con 0x55b1031d7480 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.162+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b102faa640 con 0x55b1031dc950 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.162+0000 7f57dab8eb80 10 client.0 _ll_register_callbacks cb 0x55b1030ccad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031d7480 0x55b1031e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031e0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59636/0 (socket says 172.21.15.12:59636) 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031d7480 0x55b1031e3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39412/0 (socket says 172.21.15.12:39412) 2024-05-08T23:58:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57ccf71700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031dd270 0x55b1031e0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c77fe700 1 -- 172.21.15.12:0/3437841561 learned_addr learned my addr 172.21.15.12:0/3437841561 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c77fe700 1 -- 172.21.15.12:0/3437841561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031d7480 msgr2=0x55b1031e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c77fe700 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031d7480 0x55b1031e3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c77fe700 1 -- 172.21.15.12:0/3437841561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031dd270 msgr2=0x55b1031e0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c77fe700 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031dd270 0x55b1031e0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c77fe700 1 -- 172.21.15.12:0/3437841561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81a61c0 con 0x55b1031dc950 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c7fff700 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031d7480 0x55b1031e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57c77fe700 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031e0510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f57bc032040 tx=0x7f57bc00fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67fc115f0bc2555e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57a77fe700 1 -- 172.21.15.12:0/3437841561 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00fea0 con 0x55b1031dc950 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57a77fe700 1 -- 172.21.15.12:0/3437841561 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f57bc004710 con 0x55b1031dc950 2024-05-08T23:58:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.163+0000 7f57dab8eb80 1 -- 172.21.15.12:0/3437841561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b103080210 con 0x55b1031dc950 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57a77fe700 1 -- 172.21.15.12:0/3437841561 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57bc00be90 con 0x55b1031dc950 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57dab8eb80 10 client.28510 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57dab8eb80 1 -- 172.21.15.12:0/3437841561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f57c810e630 con 0x55b1031dc950 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57a77fe700 1 -- 172.21.15.12:0/3437841561 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57bc047020 con 0x55b1031dc950 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57a77fe700 1 -- 172.21.15.12:0/3437841561 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f57bc038930 con 0x55b1031dc950 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57dab8eb80 10 client.28510 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57dab8eb80 10 client.28510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57dab8eb80 20 client.28510 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b1031f8f80 0x55b1031fb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57dab8eb80 4 client.28510 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57dab8eb80 1 -- 172.21.15.12:0/3437841561 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1031a67b0 con 0x55b1031f8f80 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.164+0000 7f57c7fff700 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b1031f8f80 0x55b1031fb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.165+0000 7f57a77fe700 10 client.28510 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.165+0000 7f57c7fff700 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b1031f8f80 0x55b1031fb3a0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee8bd12280aa466 server_cookie=f31c32a83416a9b7 in_seq=0 out_seq=0 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57a77fe700 1 -- 172.21.15.12:0/3437841561 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b1031a67b0 con 0x55b1031f8f80 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57a77fe700 10 client.28510 handle_command_reply: tid=0 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 client.28510 shutdown 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 -- 172.21.15.12:0/3437841561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 msgr2=0x55b1031e0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031e0510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f57bc032040 tx=0x7f57bc00fc40 comp rx=0 tx=0).stop 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 -- 172.21.15.12:0/3437841561 shutdown_connections 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031dd270 0x55b1031e0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b1031f8f80 0x55b1031fb3a0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031d7480 0x55b1031e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/3437841561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031e0510 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57a77fe700 0 client.28510 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 -- 172.21.15.12:0/3437841561 >> 172.21.15.12:0/3437841561 conn(0x55b1031351b0 msgr2=0x55b1031db4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 -- 172.21.15.12:0/3437841561 shutdown_connections 2024-05-08T23:58:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 1 -- 172.21.15.12:0/3437841561 wait complete. 2024-05-08T23:58:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.166+0000 7f57dab8eb80 20 client.28510 trim_cache size 0 max 16384 2024-05-08T23:58:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57dab8eb80 1 Processor -- start 2024-05-08T23:58:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57dab8eb80 1 -- start start 2024-05-08T23:58:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1031c9ab0 con 0x55b1031d7480 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1031c9c20 con 0x55b1031dd270 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1031dcd30 con 0x55b1031dc950 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58150/0 (socket says 172.21.15.12:58150) 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57c7fff700 1 -- 172.21.15.12:0/140968830 learned_addr learned my addr 172.21.15.12:0/140968830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57c77fe700 1 --2- 172.21.15.12:0/140968830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57ccf71700 1 --2- 172.21.15.12:0/140968830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57c7fff700 1 -- 172.21.15.12:0/140968830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 msgr2=0x55b1031d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57c7fff700 1 --2- 172.21.15.12:0/140968830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031d1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57c7fff700 1 -- 172.21.15.12:0/140968830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 msgr2=0x55b1031d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57c7fff700 1 --2- 172.21.15.12:0/140968830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.185+0000 7f57c7fff700 1 -- 172.21.15.12:0/140968830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b102f54b00 con 0x55b1031d7480 2024-05-08T23:58:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.186+0000 7f57c7fff700 1 --2- 172.21.15.12:0/140968830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e04d0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f57b809c010 tx=0x7f57b801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb671f7bf10f2e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.186+0000 7f57aeffd700 1 -- 172.21.15.12:0/140968830 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8099fa0 con 0x55b1031d7480 2024-05-08T23:58:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.186+0000 7f57aeffd700 1 -- 172.21.15.12:0/140968830 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57b809dd70 con 0x55b1031d7480 2024-05-08T23:58:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.186+0000 7f57aeffd700 1 -- 172.21.15.12:0/140968830 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80a1f30 con 0x55b1031d7480 2024-05-08T23:58:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 -- 172.21.15.12:0/140968830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 msgr2=0x55b1031e04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/140968830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e04d0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f57b809c010 tx=0x7f57b801a040 comp rx=0 tx=0).stop 2024-05-08T23:58:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 -- 172.21.15.12:0/140968830 shutdown_connections 2024-05-08T23:58:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/140968830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/140968830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e04d0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/140968830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 -- 172.21.15.12:0/140968830 >> 172.21.15.12:0/140968830 conn(0x55b1031351b0 msgr2=0x55b1031d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 -- 172.21.15.12:0/140968830 shutdown_connections 2024-05-08T23:58:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 -- 172.21.15.12:0/140968830 wait complete. 2024-05-08T23:58:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 -- start start 2024-05-08T23:58:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.187+0000 7f57dab8eb80 1 Processor -- start 2024-05-08T23:58:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57dab8eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031e4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1031e8530 con 0x55b1031d7480 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b10324e460 con 0x55b1031dd270 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57dab8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b10324e620 con 0x55b1031dc950 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57dab8eb80 10 client.0 _ll_register_callbacks cb 0x55b1030ccad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58152/0 (socket says 172.21.15.12:58152) 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57ccf71700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57c7fff700 1 -- 172.21.15.12:0/834807815 learned_addr learned my addr 172.21.15.12:0/834807815 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57c77fe700 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57ccf71700 1 -- 172.21.15.12:0/834807815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 msgr2=0x55b1031e40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57ccf71700 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031e40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57ccf71700 1 -- 172.21.15.12:0/834807815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 msgr2=0x55b1031e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57ccf71700 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57ccf71700 1 -- 172.21.15.12:0/834807815 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81a5c90 con 0x55b1031dd270 2024-05-08T23:58:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57ccf71700 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031e4630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f57c0007bd0 tx=0x7f57c0007ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21b0f59fcad84476 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:58:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.188+0000 7f57a77fe700 1 -- 172.21.15.12:0/834807815 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c001c070 con 0x55b1031dd270 2024-05-08T23:58:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57dab8eb80 1 -- 172.21.15.12:0/834807815 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57bc01e040 con 0x55b1031dd270 2024-05-08T23:58:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57a77fe700 1 -- 172.21.15.12:0/834807815 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f57c0004bb0 con 0x55b1031dd270 2024-05-08T23:58:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57a77fe700 1 -- 172.21.15.12:0/834807815 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57c000e040 con 0x55b1031dd270 2024-05-08T23:58:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57dab8eb80 10 client.28526 fetch_fsmap learned FSMap version 50 2024-05-08T23:58:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57dab8eb80 1 -- 172.21.15.12:0/834807815 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55b1030ed500 con 0x55b1031dd270 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57a77fe700 1 -- 172.21.15.12:0/834807815 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c002d020 con 0x55b1031dd270 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57a77fe700 1 -- 172.21.15.12:0/834807815 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f57c0029b70 con 0x55b1031dd270 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57dab8eb80 10 client.28526 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57dab8eb80 10 client.28526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57dab8eb80 20 client.28526 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b10324fe80 0x55b1032522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57dab8eb80 4 client.28526 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57dab8eb80 1 -- 172.21.15.12:0/834807815 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b10311fe60 con 0x55b10324fe80 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.189+0000 7f57c7fff700 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b10324fe80 0x55b1032522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.190+0000 7f57c7fff700 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b10324fe80 0x55b1032522a0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82122e5d66fd4ed8 server_cookie=b707de16eeeec095 in_seq=0 out_seq=0 2024-05-08T23:58:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.190+0000 7f57a77fe700 10 client.28526 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57a77fe700 1 -- 172.21.15.12:0/834807815 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b10311fe60 con 0x55b10324fe80 2024-05-08T23:58:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57a77fe700 10 client.28526 handle_command_reply: tid=0 2024-05-08T23:58:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 client.28526 shutdown 2024-05-08T23:58:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 -- 172.21.15.12:0/834807815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 msgr2=0x55b1031e4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031e4630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f57c0007bd0 tx=0x7f57c0007ee0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 -- 172.21.15.12:0/834807815 shutdown_connections 2024-05-08T23:58:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b10324fe80 0x55b1032522a0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1031dd270 0x55b1031e4630 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1031d7480 0x55b1031e3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57a77fe700 0 client.28526 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 --2- 172.21.15.12:0/834807815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1031dc950 0x55b1031e40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 -- 172.21.15.12:0/834807815 >> 172.21.15.12:0/834807815 conn(0x55b1031351b0 msgr2=0x55b1031d55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 -- 172.21.15.12:0/834807815 shutdown_connections 2024-05-08T23:58:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 1 -- 172.21.15.12:0/834807815 wait complete. 2024-05-08T23:58:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.228+0000 7f57dab8eb80 20 client.28526 trim_cache size 0 max 16384 2024-05-08T23:58:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:58:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:58:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:58:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:58:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 -- 172.21.15.12:0/898742808 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57b00a1310 msgr2=0x7f57b00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57b00a1310 0x7f57b00a37d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00b2e0 tx=0x7f57bc005960 comp rx=0 tx=0).stop 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 -- 172.21.15.12:0/898742808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57c80682e0 msgr2=0x7f57c80ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57c80682e0 0x7f57c80ffe10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f57b800e9a0 tx=0x7f57b800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 -- 172.21.15.12:0/898742808 shutdown_connections 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f57b00a1310 0x7f57b00a37d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8107000 0x7f57c8100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57c80682e0 0x7f57c80ffe10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 --2- 172.21.15.12:0/898742808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57c8068ca0 0x7f57c8100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 -- 172.21.15.12:0/898742808 >> 172.21.15.12:0/898742808 conn(0x7f57c8076f10 msgr2=0x7f57c80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 -- 172.21.15.12:0/898742808 shutdown_connections 2024-05-08T23:58:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:11.229+0000 7f57ce9d4700 1 -- 172.21.15.12:0/898742808 wait complete. 2024-05-08T23:58:11.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 646549 6.13 MB/sec execute 487 sec latency 4733.145 ms 2024-05-08T23:58:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:11 smithi012 ceph-mon[42503]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T23:58:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:11 smithi012 ceph-mon[42503]: pgmap v375: 97 pgs: 97 active+clean; 191 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-05-08T23:58:12.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:58:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-08T23:58:12.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:58:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-08T23:58:12.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:58:12 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/2369597126 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5642796061a0 con 0x5642795dd000 2024-05-08T23:58:12.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:58:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-08T23:58:12.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:11 smithi204 ceph-mon[47373]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T23:58:12.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:11 smithi204 ceph-mon[47373]: pgmap v375: 97 pgs: 97 active+clean; 191 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-05-08T23:58:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:11 smithi160 ceph-mon[46152]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T23:58:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:11 smithi160 ceph-mon[46152]: pgmap v375: 97 pgs: 97 active+clean; 191 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-05-08T23:58:12.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 647774 6.13 MB/sec execute 488 sec latency 27.505 ms 2024-05-08T23:58:13.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:12 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-08T23:58:13.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:12 smithi012 ceph-mon[42503]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T23:58:13.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:12 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-08T23:58:13.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:12 smithi204 ceph-mon[47373]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T23:58:13.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:12 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-08T23:58:13.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:12 smithi160 ceph-mon[46152]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T23:58:13.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 648042 6.12 MB/sec execute 489 sec latency 597.819 ms 2024-05-08T23:58:14.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:13 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/1281701527,v1:172.21.15.204:6837/1281701527] up:boot 2024-05-08T23:58:14.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:13 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:58:14.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:13 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:58:14.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:13 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:58:14.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:13 smithi012 ceph-mon[42503]: pgmap v377: 97 pgs: 97 active+clean; 191 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-05-08T23:58:14.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:13 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/1281701527,v1:172.21.15.204:6837/1281701527] up:boot 2024-05-08T23:58:14.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:13 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:58:14.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:13 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:58:14.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:13 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:58:14.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:13 smithi204 ceph-mon[47373]: pgmap v377: 97 pgs: 97 active+clean; 191 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-05-08T23:58:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:13 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/1281701527,v1:172.21.15.204:6837/1281701527] up:boot 2024-05-08T23:58:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:13 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:58:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:13 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:58:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:13 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:58:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:13 smithi160 ceph-mon[46152]: pgmap v377: 97 pgs: 97 active+clean; 191 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-05-08T23:58:14.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 648042 6.10 MB/sec execute 490 sec latency 1597.880 ms 2024-05-08T23:58:15.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 648042 6.09 MB/sec execute 491 sec latency 2598.006 ms 2024-05-08T23:58:15.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:15 smithi012 ceph-mon[42503]: pgmap v378: 97 pgs: 97 active+clean; 191 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 569 KiB/s wr, 44 op/s 2024-05-08T23:58:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:15 smithi204 ceph-mon[47373]: pgmap v378: 97 pgs: 97 active+clean; 191 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 569 KiB/s wr, 44 op/s 2024-05-08T23:58:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:15 smithi160 ceph-mon[46152]: pgmap v378: 97 pgs: 97 active+clean; 191 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 569 KiB/s wr, 44 op/s 2024-05-08T23:58:16.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 650000 6.10 MB/sec execute 492 sec latency 2757.181 ms 2024-05-08T23:58:17.893 INFO:tasks.workunit.client.0.smithi012.stdout: 1 651077 6.10 MB/sec execute 493 sec latency 75.035 ms 2024-05-08T23:58:17.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:17 smithi012 ceph-mon[42503]: pgmap v379: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-05-08T23:58:18.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:17 smithi204 ceph-mon[47373]: pgmap v379: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-05-08T23:58:18.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:17 smithi160 ceph-mon[46152]: pgmap v379: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-05-08T23:58:18.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 651583 6.09 MB/sec execute 494 sec latency 521.756 ms 2024-05-08T23:58:18.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:18 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:58:19.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:18 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:58:19.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:18 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:58:19.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 651583 6.08 MB/sec execute 495 sec latency 1521.827 ms 2024-05-08T23:58:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:19 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:58:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:19 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:58:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:19 smithi012 ceph-mon[42503]: pgmap v380: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-05-08T23:58:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:19 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:58:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:19 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:58:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:19 smithi204 ceph-mon[47373]: pgmap v380: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-05-08T23:58:20.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:19 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:58:20.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:19 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:58:20.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:19 smithi160 ceph-mon[46152]: pgmap v380: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-05-08T23:58:20.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 651583 6.07 MB/sec execute 496 sec latency 2521.903 ms 2024-05-08T23:58:21.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 653661 6.08 MB/sec execute 497 sec latency 2669.124 ms 2024-05-08T23:58:21.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:21 smithi012 ceph-mon[42503]: pgmap v381: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-05-08T23:58:22.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:21 smithi204 ceph-mon[47373]: pgmap v381: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-05-08T23:58:22.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:21 smithi160 ceph-mon[46152]: pgmap v381: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-05-08T23:58:22.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 654911 6.08 MB/sec execute 498 sec latency 19.071 ms 2024-05-08T23:58:23.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 655132 6.07 MB/sec execute 499 sec latency 710.933 ms 2024-05-08T23:58:23.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:23 smithi012 ceph-mon[42503]: pgmap v382: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-05-08T23:58:24.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:23 smithi204 ceph-mon[47373]: pgmap v382: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-05-08T23:58:24.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:23 smithi160 ceph-mon[46152]: pgmap v382: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-05-08T23:58:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.356+0000 7fb506559700 1 -- 172.21.15.12:0/3005592597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb500100760 msgr2=0x7fb50010cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.356+0000 7fb506559700 1 --2- 172.21.15.12:0/3005592597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb500100760 0x7fb50010cc00 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800b0a0 tx=0x7fb4f800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.356+0000 7fb506559700 1 -- 172.21.15.12:0/3005592597 shutdown_connections 2024-05-08T23:58:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.356+0000 7fb506559700 1 --2- 172.21.15.12:0/3005592597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb500100760 0x7fb50010cc00 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.356+0000 7fb506559700 1 --2- 172.21.15.12:0/3005592597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb500105e60 0x7fb500106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.356+0000 7fb506559700 1 --2- 172.21.15.12:0/3005592597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5000ffda0 0x7fb500100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.356+0000 7fb506559700 1 -- 172.21.15.12:0/3005592597 >> 172.21.15.12:0/3005592597 conn(0x7fb500076f10 msgr2=0x7fb500077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.356+0000 7fb506559700 1 -- 172.21.15.12:0/3005592597 shutdown_connections 2024-05-08T23:58:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.356+0000 7fb506559700 1 -- 172.21.15.12:0/3005592597 wait complete. 2024-05-08T23:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb506559700 1 Processor -- start 2024-05-08T23:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb506559700 1 -- start start 2024-05-08T23:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb506559700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5000ffda0 0x7fb500196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb506559700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb500100760 0x7fb500196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb506559700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb500105e60 0x7fb50019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb506559700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50019b1e0 con 0x7fb500105e60 2024-05-08T23:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb506559700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50019b350 con 0x7fb500100760 2024-05-08T23:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb506559700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50019b510 con 0x7fb5000ffda0 2024-05-08T23:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb504af6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb500105e60 0x7fb50019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb504af6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb500105e60 0x7fb50019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36966/0 (socket says 172.21.15.12:36966) 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.357+0000 7fb504af6700 1 -- 172.21.15.12:0/2286462668 learned_addr learned my addr 172.21.15.12:0/2286462668 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb4fffff700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5000ffda0 0x7fb500196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb4f7fff700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb500100760 0x7fb500196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb504af6700 1 -- 172.21.15.12:0/2286462668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5000ffda0 msgr2=0x7fb500196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb504af6700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5000ffda0 0x7fb500196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb504af6700 1 -- 172.21.15.12:0/2286462668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb500100760 msgr2=0x7fb500196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb504af6700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb500100760 0x7fb500196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb504af6700 1 -- 172.21.15.12:0/2286462668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f8009d00 con 0x7fb500105e60 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb504af6700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb500105e60 0x7fb50019aba0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800b070 tx=0x7fb4f80078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb4fdffb700 1 -- 172.21.15.12:0/2286462668 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f800e040 con 0x7fb500105e60 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb4fdffb700 1 -- 172.21.15.12:0/2286462668 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f801d470 con 0x7fb500105e60 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb506559700 1 -- 172.21.15.12:0/2286462668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50019b790 con 0x7fb500105e60 2024-05-08T23:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb4fdffb700 1 -- 172.21.15.12:0/2286462668 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80052a0 con 0x7fb500105e60 2024-05-08T23:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.358+0000 7fb506559700 1 -- 172.21.15.12:0/2286462668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5001a3030 con 0x7fb500105e60 2024-05-08T23:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.361+0000 7fb4fdffb700 1 -- 172.21.15.12:0/2286462668 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb4f801d5e0 con 0x7fb500105e60 2024-05-08T23:58:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.361+0000 7fb4fdffb700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4ec0a1290 0x7fb4ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.362+0000 7fb4fffff700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4ec0a1290 0x7fb4ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.362+0000 7fb4fdffb700 1 -- 172.21.15.12:0/2286462668 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb4f80ce1f0 con 0x7fb500105e60 2024-05-08T23:58:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.362+0000 7fb4fffff700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4ec0a1290 0x7fb4ec0a3750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009c60 tx=0x7fb4f0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.362+0000 7fb512713b80 1 Processor -- start 2024-05-08T23:58:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.362+0000 7fb512713b80 1 -- start start 2024-05-08T23:58:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb512713b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb512713b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0bac5270 0x55ca0babef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb512713b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0babf480 0x55ca0babf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb512713b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0ba160f0 con 0x55ca0bac4950 2024-05-08T23:58:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb512713b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0ba13ab0 con 0x55ca0bac5270 2024-05-08T23:58:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb512713b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0ba0f3a0 con 0x55ca0babf480 2024-05-08T23:58:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36978/0 (socket says 172.21.15.12:36978) 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4fffff700 1 -- 172.21.15.12:0/772100986 learned_addr learned my addr 172.21.15.12:0/772100986 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb504af6700 1 --2- 172.21.15.12:0/772100986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0babf480 0x55ca0babf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4f7fff700 1 --2- 172.21.15.12:0/772100986 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0bac5270 0x55ca0babef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4fffff700 1 -- 172.21.15.12:0/772100986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0babf480 msgr2=0x55ca0babf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4fffff700 1 --2- 172.21.15.12:0/772100986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0babf480 0x55ca0babf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4fffff700 1 -- 172.21.15.12:0/772100986 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0bac5270 msgr2=0x55ca0babef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4fffff700 1 --2- 172.21.15.12:0/772100986 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0bac5270 0x55ca0babef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4fffff700 1 -- 172.21.15.12:0/772100986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0b890350 con 0x55ca0bac4950 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4fffff700 1 --2- 172.21.15.12:0/772100986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac4d30 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb4f001c660 tx=0x7fb4f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39404914722de04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.363+0000 7fb4f67fc700 1 -- 172.21.15.12:0/772100986 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0021070 con 0x55ca0bac4950 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb4f67fc700 1 -- 172.21.15.12:0/772100986 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f001cd00 con 0x55ca0bac4950 2024-05-08T23:58:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb4f67fc700 1 -- 172.21.15.12:0/772100986 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000e6a0 con 0x55ca0bac4950 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb512713b80 1 -- 172.21.15.12:0/772100986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 msgr2=0x55ca0bac4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb512713b80 1 --2- 172.21.15.12:0/772100986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac4d30 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb4f001c660 tx=0x7fb4f0014000 comp rx=0 tx=0).stop 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb512713b80 1 -- 172.21.15.12:0/772100986 shutdown_connections 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb512713b80 1 --2- 172.21.15.12:0/772100986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0babf480 0x55ca0babf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb512713b80 1 --2- 172.21.15.12:0/772100986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac4d30 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb512713b80 1 --2- 172.21.15.12:0/772100986 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0bac5270 0x55ca0babef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb512713b80 1 -- 172.21.15.12:0/772100986 >> 172.21.15.12:0/772100986 conn(0x55ca0ba1d1b0 msgr2=0x55ca0bab9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb512713b80 1 -- 172.21.15.12:0/772100986 shutdown_connections 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.364+0000 7fb512713b80 1 -- 172.21.15.12:0/772100986 wait complete. 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.365+0000 7fb512713b80 1 -- start start 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.365+0000 7fb512713b80 1 Processor -- start 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.365+0000 7fb512713b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0babf480 0x55ca0bac8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.365+0000 7fb512713b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.365+0000 7fb512713b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0bac5270 0x55ca0bac90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.365+0000 7fb512713b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0b9ab500 con 0x55ca0bac4950 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.365+0000 7fb512713b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0b9ab1a0 con 0x55ca0bac5270 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.365+0000 7fb512713b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0b892640 con 0x55ca0babf480 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.365+0000 7fb512713b80 10 client.0 _ll_register_callbacks cb 0x55ca0b9b4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36982/0 (socket says 172.21.15.12:36982) 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4f7fff700 1 -- 172.21.15.12:0/3683274474 learned_addr learned my addr 172.21.15.12:0/3683274474 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb504af6700 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0bac5270 0x55ca0bac90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4fffff700 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0babf480 0x55ca0bac8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4f7fff700 1 -- 172.21.15.12:0/3683274474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0babf480 msgr2=0x55ca0bac8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4f7fff700 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0babf480 0x55ca0bac8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4f7fff700 1 -- 172.21.15.12:0/3683274474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0bac5270 msgr2=0x55ca0bac90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4f7fff700 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0bac5270 0x55ca0bac90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4f7fff700 1 -- 172.21.15.12:0/3683274474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f0017070 con 0x55ca0bac4950 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4f7fff700 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac8b60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8013710 tx=0x7fb4e8013a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0e322e1fcb1b08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4df7fe700 1 -- 172.21.15.12:0/3683274474 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80118b0 con 0x55ca0bac4950 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4df7fe700 1 -- 172.21.15.12:0/3683274474 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb4e8011a10 con 0x55ca0bac4950 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb512713b80 1 -- 172.21.15.12:0/3683274474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca0b968210 con 0x55ca0bac4950 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb4df7fe700 1 -- 172.21.15.12:0/3683274474 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4e8011bd0 con 0x55ca0bac4950 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb512713b80 10 client.18819 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.366+0000 7fb512713b80 1 -- 172.21.15.12:0/3683274474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55ca0b83cb00 con 0x55ca0bac4950 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb4df7fe700 1 -- 172.21.15.12:0/3683274474 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca0b968210 con 0x55ca0bac4950 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb4df7fe700 1 -- 172.21.15.12:0/3683274474 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb4e801d3f0 con 0x55ca0bac4950 2024-05-08T23:58:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb512713b80 10 client.18819 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb512713b80 10 client.18819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb512713b80 20 client.18819 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb512713b80 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ca0bad8db0 0x55ca0badb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb512713b80 4 client.18819 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb512713b80 1 -- 172.21.15.12:0/3683274474 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca0ba8e7b0 con 0x55ca0bad8db0 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb4fffff700 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ca0bad8db0 0x55ca0badb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb4df7fe700 10 client.18819 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.367+0000 7fb4fffff700 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ca0bad8db0 0x55ca0badb1d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f93bfe2fbe9726 server_cookie=1e7ef3dfc77e7faf in_seq=0 out_seq=0 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb4df7fe700 1 -- 172.21.15.12:0/3683274474 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca0ba8e7b0 con 0x55ca0bad8db0 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb4df7fe700 10 client.18819 handle_command_reply: tid=0 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb512713b80 1 client.18819 shutdown 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb512713b80 1 -- 172.21.15.12:0/3683274474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 msgr2=0x55ca0bac8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb512713b80 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac8b60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8013710 tx=0x7fb4e8013a20 comp rx=0 tx=0).stop 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb512713b80 1 -- 172.21.15.12:0/3683274474 shutdown_connections 2024-05-08T23:58:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb512713b80 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ca0bad8db0 0x55ca0badb1d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb512713b80 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0bac5270 0x55ca0bac90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb512713b80 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0babf480 0x55ca0bac8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb512713b80 1 --2- 172.21.15.12:0/3683274474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac4950 0x55ca0bac8b60 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb4df7fe700 0 client.18819 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb512713b80 1 -- 172.21.15.12:0/3683274474 >> 172.21.15.12:0/3683274474 conn(0x55ca0ba1d1b0 msgr2=0x55ca0babdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.368+0000 7fb512713b80 1 -- 172.21.15.12:0/3683274474 shutdown_connections 2024-05-08T23:58:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.369+0000 7fb512713b80 1 -- 172.21.15.12:0/3683274474 wait complete. 2024-05-08T23:58:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.369+0000 7fb512713b80 20 client.18819 trim_cache size 0 max 16384 2024-05-08T23:58:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.384+0000 7fb512713b80 1 Processor -- start 2024-05-08T23:58:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.384+0000 7fb512713b80 1 -- start start 2024-05-08T23:58:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb512713b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 0x55ca0bac2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb512713b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0bac4950 0x55ca0bab9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb512713b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0babf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb512713b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0baba780 con 0x55ca0bac5270 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb512713b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0bacb160 con 0x55ca0babf480 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb512713b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0babef40 con 0x55ca0bac4950 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb504af6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0babf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb504af6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0babf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36992/0 (socket says 172.21.15.12:36992) 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb504af6700 1 -- 172.21.15.12:0/3419325846 learned_addr learned my addr 172.21.15.12:0/3419325846 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb4f7fff700 1 --2- 172.21.15.12:0/3419325846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0bac4950 0x55ca0bab9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb4fffff700 1 --2- 172.21.15.12:0/3419325846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 0x55ca0bac2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb504af6700 1 -- 172.21.15.12:0/3419325846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0bac4950 msgr2=0x55ca0bab9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb504af6700 1 --2- 172.21.15.12:0/3419325846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0bac4950 0x55ca0bab9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb504af6700 1 -- 172.21.15.12:0/3419325846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 msgr2=0x55ca0bac2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb504af6700 1 --2- 172.21.15.12:0/3419325846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 0x55ca0bac2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb504af6700 1 -- 172.21.15.12:0/3419325846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0b9d5500 con 0x55ca0bac5270 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb4fffff700 1 --2- 172.21.15.12:0/3419325846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 0x55ca0bac2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.385+0000 7fb504af6700 1 --2- 172.21.15.12:0/3419325846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0babf860 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb4f809b360 tx=0x7fb4f8097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13dca0c5a4496b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb4f6ffd700 1 -- 172.21.15.12:0/3419325846 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f809ad30 con 0x55ca0bac5270 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb4f6ffd700 1 -- 172.21.15.12:0/3419325846 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f80ce040 con 0x55ca0bac5270 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb4f6ffd700 1 -- 172.21.15.12:0/3419325846 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f801d8e0 con 0x55ca0bac5270 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb512713b80 1 -- 172.21.15.12:0/3419325846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 msgr2=0x55ca0babf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb512713b80 1 --2- 172.21.15.12:0/3419325846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0babf860 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb4f809b360 tx=0x7fb4f8097040 comp rx=0 tx=0).stop 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb512713b80 1 -- 172.21.15.12:0/3419325846 shutdown_connections 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb512713b80 1 --2- 172.21.15.12:0/3419325846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0babf860 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb512713b80 1 --2- 172.21.15.12:0/3419325846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 0x55ca0bac2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb512713b80 1 --2- 172.21.15.12:0/3419325846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0bac4950 0x55ca0bab9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb512713b80 1 -- 172.21.15.12:0/3419325846 >> 172.21.15.12:0/3419325846 conn(0x55ca0ba1d1b0 msgr2=0x55ca0bac6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb512713b80 1 -- 172.21.15.12:0/3419325846 shutdown_connections 2024-05-08T23:58:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.386+0000 7fb512713b80 1 -- 172.21.15.12:0/3419325846 wait complete. 2024-05-08T23:58:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.387+0000 7fb512713b80 1 -- start start 2024-05-08T23:58:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.387+0000 7fb512713b80 1 Processor -- start 2024-05-08T23:58:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.387+0000 7fb512713b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 0x55ca0bacdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.387+0000 7fb512713b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0bac4950 0x55ca0bace160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.387+0000 7fb512713b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0bace6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.387+0000 7fb512713b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0bb2e2b0 con 0x55ca0bac5270 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.387+0000 7fb512713b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0bb2e3f0 con 0x55ca0babf480 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.387+0000 7fb512713b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0bb2e530 con 0x55ca0bac4950 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb512713b80 10 client.0 _ll_register_callbacks cb 0x55ca0b9b4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb504af6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0bace6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb504af6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0bace6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37002/0 (socket says 172.21.15.12:37002) 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb504af6700 1 -- 172.21.15.12:0/477932664 learned_addr learned my addr 172.21.15.12:0/477932664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb4f7fff700 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0bac4950 0x55ca0bace160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb4fffff700 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 0x55ca0bacdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb504af6700 1 -- 172.21.15.12:0/477932664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0bac4950 msgr2=0x55ca0bace160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb504af6700 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0bac4950 0x55ca0bace160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb504af6700 1 -- 172.21.15.12:0/477932664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 msgr2=0x55ca0bacdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb504af6700 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 0x55ca0bacdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb504af6700 1 -- 172.21.15.12:0/477932664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5001a3030 con 0x55ca0bac5270 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb504af6700 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0bace6a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009fd0 tx=0x7fb4f801db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf97a17a79792ea0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb4df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb4df7fe700 1 -- 172.21.15.12:0/477932664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f809b3d0 con 0x55ca0bac5270 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb4df7fe700 1 -- 172.21.15.12:0/477932664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb4f809bad0 con 0x55ca0bac5270 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb512713b80 1 -- 172.21.15.12:0/477932664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4f001cd00 con 0x55ca0bac5270 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb4df7fe700 1 -- 172.21.15.12:0/477932664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f809f200 con 0x55ca0bac5270 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb512713b80 10 client.18825 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.388+0000 7fb512713b80 1 -- 172.21.15.12:0/477932664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55ca0b894c70 con 0x55ca0bac5270 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb4df7fe700 1 -- 172.21.15.12:0/477932664 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f001cd00 con 0x55ca0bac5270 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb4df7fe700 1 -- 172.21.15.12:0/477932664 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb4f80b1020 con 0x55ca0bac5270 2024-05-08T23:58:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb512713b80 10 client.18825 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb512713b80 10 client.18825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb512713b80 20 client.18825 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb512713b80 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ca0bb2f8b0 0x55ca0bb31cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb512713b80 4 client.18825 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:58:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb512713b80 1 -- 172.21.15.12:0/477932664 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca0ba07e60 con 0x55ca0bb2f8b0 2024-05-08T23:58:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb4fffff700 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ca0bb2f8b0 0x55ca0bb31cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb4df7fe700 10 client.18825 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.389+0000 7fb4fffff700 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ca0bb2f8b0 0x55ca0bb31cd0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16a3536c9ee6fbbc server_cookie=81fe08df91fb3ffd in_seq=0 out_seq=0 2024-05-08T23:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb4df7fe700 1 -- 172.21.15.12:0/477932664 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca0ba07e60 con 0x55ca0bb2f8b0 2024-05-08T23:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb4df7fe700 10 client.18825 handle_command_reply: tid=0 2024-05-08T23:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 client.18825 shutdown 2024-05-08T23:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 -- 172.21.15.12:0/477932664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 msgr2=0x55ca0bace6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0bace6a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009fd0 tx=0x7fb4f801db90 comp rx=0 tx=0).stop 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 -- 172.21.15.12:0/477932664 shutdown_connections 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca0bac5270 0x55ca0bace6a0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ca0bb2f8b0 0x55ca0bb31cd0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0babf480 0x55ca0bacdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 --2- 172.21.15.12:0/477932664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca0bac4950 0x55ca0bace160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 -- 172.21.15.12:0/477932664 >> 172.21.15.12:0/477932664 conn(0x55ca0ba1d1b0 msgr2=0x55ca0bac9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb4df7fe700 0 client.18825 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 -- 172.21.15.12:0/477932664 shutdown_connections 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 1 -- 172.21.15.12:0/477932664 wait complete. 2024-05-08T23:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.438+0000 7fb512713b80 20 client.18825 trim_cache size 0 max 16384 2024-05-08T23:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 -- 172.21.15.12:0/2286462668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4ec0a1290 msgr2=0x7fb4ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4ec0a1290 0x7fb4ec0a3750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009c60 tx=0x7fb4f0009260 comp rx=0 tx=0).stop 2024-05-08T23:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 -- 172.21.15.12:0/2286462668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb500105e60 msgr2=0x7fb50019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb500105e60 0x7fb50019aba0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800b070 tx=0x7fb4f80078e0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 -- 172.21.15.12:0/2286462668 shutdown_connections 2024-05-08T23:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb4ec0a1290 0x7fb4ec0a3750 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb500105e60 0x7fb50019aba0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5000ffda0 0x7fb500196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 --2- 172.21.15.12:0/2286462668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb500100760 0x7fb500196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 -- 172.21.15.12:0/2286462668 >> 172.21.15.12:0/2286462668 conn(0x7fb500076f10 msgr2=0x7fb5000fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 -- 172.21.15.12:0/2286462668 shutdown_connections 2024-05-08T23:58:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:24.439+0000 7fb506559700 1 -- 172.21.15.12:0/2286462668 wait complete. 2024-05-08T23:58:24.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 655132 6.06 MB/sec execute 500 sec latency 1710.996 ms 2024-05-08T23:58:25.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 655132 6.04 MB/sec execute 501 sec latency 2711.073 ms 2024-05-08T23:58:25.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:25 smithi012 ceph-mon[42503]: pgmap v383: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-05-08T23:58:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:25 smithi204 ceph-mon[47373]: pgmap v383: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-05-08T23:58:26.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:25 smithi160 ceph-mon[46152]: pgmap v383: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-05-08T23:58:26.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 657210 6.06 MB/sec execute 502 sec latency 2851.814 ms 2024-05-08T23:58:27.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 658471 6.06 MB/sec execute 503 sec latency 23.890 ms 2024-05-08T23:58:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:27 smithi012 ceph-mon[42503]: pgmap v384: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-05-08T23:58:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:27 smithi204 ceph-mon[47373]: pgmap v384: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-05-08T23:58:28.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:27 smithi160 ceph-mon[46152]: pgmap v384: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-05-08T23:58:28.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 658675 6.05 MB/sec execute 504 sec latency 729.006 ms 2024-05-08T23:58:29.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 658675 6.03 MB/sec execute 505 sec latency 1729.127 ms 2024-05-08T23:58:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:29 smithi012 ceph-mon[42503]: pgmap v385: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-08T23:58:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:29 smithi204 ceph-mon[47373]: pgmap v385: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-08T23:58:30.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:29 smithi160 ceph-mon[46152]: pgmap v385: 97 pgs: 97 active+clean; 197 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-08T23:58:30.894 INFO:tasks.workunit.client.0.smithi012.stdout: 1 658675 6.02 MB/sec execute 506 sec latency 2729.202 ms 2024-05-08T23:58:31.895 INFO:tasks.workunit.client.0.smithi012.stdout: 1 660751 6.04 MB/sec execute 507 sec latency 2874.463 ms 2024-05-08T23:58:31.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:31 smithi012 ceph-mon[42503]: pgmap v386: 97 pgs: 97 active+clean; 201 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-05-08T23:58:32.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:31 smithi204 ceph-mon[47373]: pgmap v386: 97 pgs: 97 active+clean; 201 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-05-08T23:58:32.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:31 smithi160 ceph-mon[46152]: pgmap v386: 97 pgs: 97 active+clean; 201 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-05-08T23:58:32.895 INFO:tasks.workunit.client.0.smithi012.stdout: 1 662002 6.04 MB/sec execute 508 sec latency 25.397 ms 2024-05-08T23:58:33.895 INFO:tasks.workunit.client.0.smithi012.stdout: 1 662222 6.03 MB/sec execute 509 sec latency 726.351 ms 2024-05-08T23:58:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:33 smithi012 ceph-mon[42503]: pgmap v387: 97 pgs: 97 active+clean; 201 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-05-08T23:58:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:33 smithi204 ceph-mon[47373]: pgmap v387: 97 pgs: 97 active+clean; 201 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-05-08T23:58:34.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:33 smithi160 ceph-mon[46152]: pgmap v387: 97 pgs: 97 active+clean; 201 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-05-08T23:58:34.580 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:58:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.734+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3808903544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0dec073370 msgr2=0x7f0dec073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.734+0000 7f0df2d0d700 1 --2- 172.21.15.12:0/3808903544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0dec073370 0x7f0dec073750 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009990 tx=0x7f0de0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.734+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3808903544 shutdown_connections 2024-05-08T23:58:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.734+0000 7f0df2d0d700 1 --2- 172.21.15.12:0/3808903544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec10d790 0x7f0dec10fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.734+0000 7f0df2d0d700 1 --2- 172.21.15.12:0/3808903544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0dec073370 0x7f0dec073750 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.734+0000 7f0df2d0d700 1 --2- 172.21.15.12:0/3808903544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dec073c90 0x7f0dec10d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.734+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3808903544 >> 172.21.15.12:0/3808903544 conn(0x7f0dec0fc9e0 msgr2=0x7f0dec0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.734+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3808903544 shutdown_connections 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.734+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3808903544 wait complete. 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.734+0000 7f0df2d0d700 1 Processor -- start 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0df2d0d700 1 -- start start 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0df2d0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dec073370 0x7f0dec199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0df2d0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0dec073c90 0x7f0dec199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0df2d0d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec10d790 0x7f0dec19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0df2d0d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec19e530 con 0x7f0dec073c90 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0df2d0d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec19e6a0 con 0x7f0dec073370 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0df2d0d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec19e860 con 0x7f0dec10d790 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0debfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0dec073c90 0x7f0dec199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0debfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0dec073c90 0x7f0dec199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33456/0 (socket says 172.21.15.12:33456) 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0debfff700 1 -- 172.21.15.12:0/3382784498 learned_addr learned my addr 172.21.15.12:0/3382784498 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0df12aa700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec10d790 0x7f0dec19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dec073370 0x7f0dec199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0debfff700 1 -- 172.21.15.12:0/3382784498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec10d790 msgr2=0x7f0dec19def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0debfff700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec10d790 0x7f0dec19def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0debfff700 1 -- 172.21.15.12:0/3382784498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dec073370 msgr2=0x7f0dec199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0debfff700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dec073370 0x7f0dec199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0debfff700 1 -- 172.21.15.12:0/3382784498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de00095f0 con 0x7f0dec073c90 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.735+0000 7f0debfff700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0dec073c90 0x7f0dec199b50 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800d980 tx=0x7f0dd800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.736+0000 7f0de9ffb700 1 -- 172.21.15.12:0/3382784498 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8004220 con 0x7f0dec073c90 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.736+0000 7f0de9ffb700 1 -- 172.21.15.12:0/3382784498 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dd8007640 con 0x7f0dec073c90 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.736+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3382784498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec19eb40 con 0x7f0dec073c90 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.736+0000 7f0de9ffb700 1 -- 172.21.15.12:0/3382784498 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8005690 con 0x7f0dec073c90 2024-05-08T23:58:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.736+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3382784498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec1a6380 con 0x7f0dec073c90 2024-05-08T23:58:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.738+0000 7f0de9ffb700 1 -- 172.21.15.12:0/3382784498 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0dd8004750 con 0x7f0dec073c90 2024-05-08T23:58:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.739+0000 7f0de9ffb700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0ddc0a1290 0x7f0ddc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.739+0000 7f0de9ffb700 1 -- 172.21.15.12:0/3382784498 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0dd80cca20 con 0x7f0dec073c90 2024-05-08T23:58:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.739+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0ddc0a1290 0x7f0ddc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.739+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0ddc0a1290 0x7f0ddc0a3750 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0de000b2e0 tx=0x7f0de0005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0dfeec7b80 1 Processor -- start 2024-05-08T23:58:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0dfeec7b80 1 -- start start 2024-05-08T23:58:34.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 0x562b371b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 0x562b371ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371ac890 0x562b371accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b37103150 con 0x562b371ac890 2024-05-08T23:58:34.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b37100b10 con 0x562b371b2720 2024-05-08T23:58:34.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b370fc400 con 0x562b371b1e00 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0df12aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371ac890 0x562b371accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0df12aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371ac890 0x562b371accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33462/0 (socket says 172.21.15.12:33462) 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0df12aa700 1 -- 172.21.15.12:0/4276175880 learned_addr learned my addr 172.21.15.12:0/4276175880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/4276175880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 0x562b371b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.740+0000 7f0debfff700 1 --2- 172.21.15.12:0/4276175880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 0x562b371ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.741+0000 7f0df12aa700 1 -- 172.21.15.12:0/4276175880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 msgr2=0x562b371b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.741+0000 7f0df12aa700 1 --2- 172.21.15.12:0/4276175880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 0x562b371b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.741+0000 7f0df12aa700 1 -- 172.21.15.12:0/4276175880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 msgr2=0x562b371ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.741+0000 7f0df12aa700 1 --2- 172.21.15.12:0/4276175880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 0x562b371ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.741+0000 7f0df12aa700 1 -- 172.21.15.12:0/4276175880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b36f7d3b0 con 0x562b371ac890 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.741+0000 7f0df12aa700 1 --2- 172.21.15.12:0/4276175880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371ac890 0x562b371accd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f0de4011600 tx=0x7f0de4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5d57c7aae82f9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.741+0000 7f0dd6ffd700 1 -- 172.21.15.12:0/4276175880 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de400e040 con 0x562b371ac890 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.741+0000 7f0dd6ffd700 1 -- 172.21.15.12:0/4276175880 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0de400f040 con 0x562b371ac890 2024-05-08T23:58:34.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.741+0000 7f0dd6ffd700 1 -- 172.21.15.12:0/4276175880 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4004760 con 0x562b371ac890 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/4276175880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371ac890 msgr2=0x562b371accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/4276175880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371ac890 0x562b371accd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f0de4011600 tx=0x7f0de4009920 comp rx=0 tx=0).stop 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/4276175880 shutdown_connections 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/4276175880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371ac890 0x562b371accd0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/4276175880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 0x562b371b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/4276175880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 0x562b371ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/4276175880 >> 172.21.15.12:0/4276175880 conn(0x562b3710a210 msgr2=0x562b371a6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/4276175880 shutdown_connections 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/4276175880 wait complete. 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 -- start start 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.742+0000 7f0dfeec7b80 1 Processor -- start 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 0x562b371b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 0x562b371b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b2720 0x562b371b6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b37098560 con 0x562b371b2720 2024-05-08T23:58:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b37098200 con 0x562b371ac890 2024-05-08T23:58:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b36f7f6a0 con 0x562b371b1e00 2024-05-08T23:58:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0dfeec7b80 10 client.0 _ll_register_callbacks cb 0x562b370a1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0df12aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b2720 0x562b371b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0df0aa9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 0x562b371b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0debfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 0x562b371b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0df12aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b2720 0x562b371b6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33470/0 (socket says 172.21.15.12:33470) 2024-05-08T23:58:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0df12aa700 1 -- 172.21.15.12:0/2750480927 learned_addr learned my addr 172.21.15.12:0/2750480927 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0debfff700 1 -- 172.21.15.12:0/2750480927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 msgr2=0x562b371b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0debfff700 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 0x562b371b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0debfff700 1 -- 172.21.15.12:0/2750480927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b2720 msgr2=0x562b371b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0debfff700 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b2720 0x562b371b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0debfff700 1 -- 172.21.15.12:0/2750480927 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de40112b0 con 0x562b371b1e00 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.743+0000 7f0df12aa700 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b2720 0x562b371b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0debfff700 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 0x562b371b6010 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80ca1b0 tx=0x7f0dd809a3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a177042c93b888 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dcbfff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dcbfff700 1 -- 172.21.15.12:0/2750480927 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80a1ff0 con 0x562b371b1e00 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2750480927 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b37055270 con 0x562b371b1e00 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dcbfff700 1 -- 172.21.15.12:0/2750480927 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0dd8099dd0 con 0x562b371b1e00 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dcbfff700 1 -- 172.21.15.12:0/2750480927 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dd8099f90 con 0x562b371b1e00 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dfeec7b80 10 client.28552 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2750480927 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x562b36f29b60 con 0x562b371b1e00 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dcbfff700 1 -- 172.21.15.12:0/2750480927 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562b37055270 con 0x562b371b1e00 2024-05-08T23:58:34.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dcbfff700 1 -- 172.21.15.12:0/2750480927 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0dd80a1c80 con 0x562b371b1e00 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dfeec7b80 10 client.28552 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dfeec7b80 10 client.28552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.744+0000 7f0dfeec7b80 20 client.28552 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.745+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b371c61a0 0x562b371c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.745+0000 7f0dfeec7b80 4 client.28552 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.745+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2750480927 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b3717b810 con 0x562b371c61a0 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.745+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b371c61a0 0x562b371c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.745+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b371c61a0 0x562b371c85c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca918cc789202871 server_cookie=bacc1bcaf13bb67a in_seq=0 out_seq=0 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.745+0000 7f0dcbfff700 10 client.28552 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dcbfff700 1 -- 172.21.15.12:0/2750480927 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b3717b810 con 0x562b371c61a0 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dcbfff700 10 client.28552 handle_command_reply: tid=0 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 client.28552 shutdown 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2750480927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 msgr2=0x562b371b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 0x562b371b6010 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80ca1b0 tx=0x7f0dd809a3d0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2750480927 shutdown_connections 2024-05-08T23:58:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b2720 0x562b371b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b371c61a0 0x562b371c85c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 0x562b371b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/2750480927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b1e00 0x562b371b6010 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dcbfff700 0 client.28552 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2750480927 >> 172.21.15.12:0/2750480927 conn(0x562b3710a210 msgr2=0x562b371a65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2750480927 shutdown_connections 2024-05-08T23:58:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2750480927 wait complete. 2024-05-08T23:58:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.746+0000 7f0dfeec7b80 20 client.28552 trim_cache size 0 max 16384 2024-05-08T23:58:34.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0dfeec7b80 1 Processor -- start 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0dfeec7b80 1 -- start start 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 0x562b371ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b2720 0x562b371bd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3719f110 con 0x562b371b1e00 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b371a7b30 con 0x562b371ac890 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b371b70f0 con 0x562b371b2720 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0debfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0debfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33482/0 (socket says 172.21.15.12:33482) 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0debfff700 1 -- 172.21.15.12:0/2798775974 learned_addr learned my addr 172.21.15.12:0/2798775974 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/2798775974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 0x562b371ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0df12aa700 1 --2- 172.21.15.12:0/2798775974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b2720 0x562b371bd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0debfff700 1 -- 172.21.15.12:0/2798775974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b2720 msgr2=0x562b371bd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0debfff700 1 --2- 172.21.15.12:0/2798775974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b2720 0x562b371bd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0debfff700 1 -- 172.21.15.12:0/2798775974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 msgr2=0x562b371ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0debfff700 1 --2- 172.21.15.12:0/2798775974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 0x562b371ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0debfff700 1 -- 172.21.15.12:0/2798775974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b370c2560 con 0x562b371b1e00 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.765+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/2798775974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 0x562b371ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.766+0000 7f0debfff700 1 --2- 172.21.15.12:0/2798775974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371a6d20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8007e80 tx=0x7f0dd8098120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e0ba131e2eceb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.766+0000 7f0dd77fe700 1 -- 172.21.15.12:0/2798775974 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8014070 con 0x562b371b1e00 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.766+0000 7f0dd77fe700 1 -- 172.21.15.12:0/2798775974 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dd8099940 con 0x562b371b1e00 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.766+0000 7f0dd77fe700 1 -- 172.21.15.12:0/2798775974 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8099060 con 0x562b371b1e00 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.766+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2798775974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 msgr2=0x562b371a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/2798775974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371a6d20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8007e80 tx=0x7f0dd8098120 comp rx=0 tx=0).stop 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2798775974 shutdown_connections 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/2798775974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371b2720 0x562b371bd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/2798775974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371ac890 0x562b371ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/2798775974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371a6d20 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2798775974 >> 172.21.15.12:0/2798775974 conn(0x562b3710a210 msgr2=0x562b371a8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2798775974 shutdown_connections 2024-05-08T23:58:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/2798775974 wait complete. 2024-05-08T23:58:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 -- start start 2024-05-08T23:58:34.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 Processor -- start 2024-05-08T23:58:34.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371ac890 0x562b371bd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 0x562b371b9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b371b9bc0 con 0x562b371b1e00 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b371b9d30 con 0x562b371b2720 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.767+0000 7f0dfeec7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b371b9ef0 con 0x562b371ac890 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0dfeec7b80 10 client.0 _ll_register_callbacks cb 0x562b370a1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0debfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0debfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371b9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33496/0 (socket says 172.21.15.12:33496) 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0debfff700 1 -- 172.21.15.12:0/16134964 learned_addr learned my addr 172.21.15.12:0/16134964 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0df12aa700 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 0x562b371b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371ac890 0x562b371bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0debfff700 1 -- 172.21.15.12:0/16134964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371ac890 msgr2=0x562b371bd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:34.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0debfff700 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371ac890 0x562b371bd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0debfff700 1 -- 172.21.15.12:0/16134964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 msgr2=0x562b371b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0debfff700 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 0x562b371b9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0debfff700 1 -- 172.21.15.12:0/16134964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b36f29b60 con 0x562b371b1e00 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0df12aa700 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 0x562b371b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0debfff700 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371b9140 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8000c00 tx=0x7f0dd80045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7060d0328687c19d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0dcbfff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0dcbfff700 1 -- 172.21.15.12:0/16134964 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8014040 con 0x562b371b1e00 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0dcbfff700 1 -- 172.21.15.12:0/16134964 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0dd809ad10 con 0x562b371b1e00 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/16134964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0de400f040 con 0x562b371b1e00 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0dcbfff700 1 -- 172.21.15.12:0/16134964 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dd80a1ff0 con 0x562b371b1e00 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0dfeec7b80 10 client.18843 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.768+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/16134964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x562b36f81cd0 con 0x562b371b1e00 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.769+0000 7f0dcbfff700 1 -- 172.21.15.12:0/16134964 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0de400f040 con 0x562b371b1e00 2024-05-08T23:58:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.769+0000 7f0dcbfff700 1 -- 172.21.15.12:0/16134964 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0dd8095960 con 0x562b371b1e00 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.769+0000 7f0dfeec7b80 10 client.18843 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.769+0000 7f0dfeec7b80 10 client.18843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.769+0000 7f0dfeec7b80 20 client.18843 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.769+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3721c6b0 0x562b3721eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.769+0000 7f0dfeec7b80 4 client.18843 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.769+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/16134964 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562b370f4ec0 con 0x562b3721c6b0 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.769+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3721c6b0 0x562b3721eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0df0aa9700 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3721c6b0 0x562b3721eaa0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f391693dd0ef685 server_cookie=87d0c75d9860d062 in_seq=0 out_seq=0 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dcbfff700 10 client.18843 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dcbfff700 1 -- 172.21.15.12:0/16134964 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562b370f4ec0 con 0x562b3721c6b0 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dcbfff700 10 client.18843 handle_command_reply: tid=0 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 client.18843 shutdown 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/16134964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 msgr2=0x562b371b9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371b9140 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8000c00 tx=0x7f0dd80045b0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/16134964 shutdown_connections 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3721c6b0 0x562b3721eaa0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b371b2720 0x562b371b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dcbfff700 0 client.18843 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b371ac890 0x562b371bd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 --2- 172.21.15.12:0/16134964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b371b1e00 0x562b371b9140 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/16134964 >> 172.21.15.12:0/16134964 conn(0x562b3710a210 msgr2=0x562b371b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/16134964 shutdown_connections 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 1 -- 172.21.15.12:0/16134964 wait complete. 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.770+0000 7f0dfeec7b80 20 client.18843 trim_cache size 0 max 16384 2024-05-08T23:58:34.771 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:58:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:58:34.772 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:58:34.772 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:58:34.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3382784498 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0ddc0a1290 msgr2=0x7f0ddc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0ddc0a1290 0x7f0ddc0a3750 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0de000b2e0 tx=0x7f0de0005fd0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3382784498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0dec073c90 msgr2=0x7f0dec199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0dec073c90 0x7f0dec199b50 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800d980 tx=0x7f0dd800dc90 comp rx=0 tx=0).stop 2024-05-08T23:58:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3382784498 shutdown_connections 2024-05-08T23:58:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0ddc0a1290 0x7f0ddc0a3750 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec10d790 0x7f0dec19def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dec073370 0x7f0dec199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 --2- 172.21.15.12:0/3382784498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0dec073c90 0x7f0dec199b50 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3382784498 >> 172.21.15.12:0/3382784498 conn(0x7f0dec0fc9e0 msgr2=0x7f0dec0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3382784498 shutdown_connections 2024-05-08T23:58:34.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.771+0000 7f0df2d0d700 1 -- 172.21.15.12:0/3382784498 wait complete. 2024-05-08T23:58:34.779 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:58:34.779 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:58:34.895 INFO:tasks.workunit.client.0.smithi012.stdout: 1 662222 6.01 MB/sec execute 510 sec latency 1726.405 ms 2024-05-08T23:58:34.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.921+0000 7f47d9eda700 1 -- 172.21.15.12:0/3710742603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d41008f0 msgr2=0x7f47d4104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.921+0000 7f47d9eda700 1 --2- 172.21.15.12:0/3710742603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d41008f0 0x7f47d4104930 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009990 tx=0x7f47c8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.922+0000 7f47d9eda700 1 -- 172.21.15.12:0/3710742603 shutdown_connections 2024-05-08T23:58:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.922+0000 7f47d9eda700 1 --2- 172.21.15.12:0/3710742603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4105060 0x7f47d410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.922+0000 7f47d9eda700 1 --2- 172.21.15.12:0/3710742603 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47d40fff40 0x7f47d4100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.922+0000 7f47d9eda700 1 --2- 172.21.15.12:0/3710742603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d41008f0 0x7f47d4104930 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.922+0000 7f47d9eda700 1 -- 172.21.15.12:0/3710742603 >> 172.21.15.12:0/3710742603 conn(0x7f47d40fb7e0 msgr2=0x7f47d40fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.922+0000 7f47d9eda700 1 -- 172.21.15.12:0/3710742603 shutdown_connections 2024-05-08T23:58:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.922+0000 7f47d9eda700 1 -- 172.21.15.12:0/3710742603 wait complete. 2024-05-08T23:58:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.922+0000 7f47d9eda700 1 Processor -- start 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d9eda700 1 -- start start 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d9eda700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47d40fff40 0x7f47d4197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d9eda700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4105060 0x7f47d41978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d9eda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d419bc50 0x7f47d419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d9eda700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d419c6d0 con 0x7f47d419bc50 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d9eda700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d419c840 con 0x7f47d40fff40 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d9eda700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d419ca00 con 0x7f47d4105060 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d419bc50 0x7f47d419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d419bc50 0x7f47d419c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33516/0 (socket says 172.21.15.12:33516) 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d3fff700 1 -- 172.21.15.12:0/3475876530 learned_addr learned my addr 172.21.15.12:0/3475876530 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d2ffd700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4105060 0x7f47d41978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d37fe700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47d40fff40 0x7f47d4197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d3fff700 1 -- 172.21.15.12:0/3475876530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4105060 msgr2=0x7f47d41978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d3fff700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4105060 0x7f47d41978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d3fff700 1 -- 172.21.15.12:0/3475876530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47d40fff40 msgr2=0x7f47d4197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d3fff700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47d40fff40 0x7f47d4197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d3fff700 1 -- 172.21.15.12:0/3475876530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c80095f0 con 0x7f47d419bc50 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d37fe700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47d40fff40 0x7f47d4197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.923+0000 7f47d3fff700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d419bc50 0x7f47d419c0d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00e940 tx=0x7f47cc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.924+0000 7f47d0ff9700 1 -- 172.21.15.12:0/3475876530 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0097b0 con 0x7f47d419bc50 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.924+0000 7f47d0ff9700 1 -- 172.21.15.12:0/3475876530 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47cc00ee90 con 0x7f47d419bc50 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.924+0000 7f47d9eda700 1 -- 172.21.15.12:0/3475876530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d41a3f60 con 0x7f47d419bc50 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.924+0000 7f47d0ff9700 1 -- 172.21.15.12:0/3475876530 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc005520 con 0x7f47d419bc50 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.924+0000 7f47d9eda700 1 -- 172.21.15.12:0/3475876530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d41a4440 con 0x7f47d419bc50 2024-05-08T23:58:34.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.925+0000 7f47d9eda700 1 -- 172.21.15.12:0/3475876530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d4109340 con 0x7f47d419bc50 2024-05-08T23:58:34.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.926+0000 7f47d0ff9700 1 -- 172.21.15.12:0/3475876530 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f47cc004750 con 0x7f47d419bc50 2024-05-08T23:58:34.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.926+0000 7f47d0ff9700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47bc0a1290 0x7f47bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:34.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.927+0000 7f47d0ff9700 1 -- 172.21.15.12:0/3475876530 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f47cc014070 con 0x7f47d419bc50 2024-05-08T23:58:34.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.927+0000 7f47d37fe700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47bc0a1290 0x7f47bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:34.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.927+0000 7f47d37fe700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47bc0a1290 0x7f47bc0a3750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f47c40093a0 tx=0x7f47c400b040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:34.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:34.931+0000 7f47d0ff9700 1 -- 172.21.15.12:0/3475876530 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f47cc096470 con 0x7f47d419bc50 2024-05-08T23:58:35.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.205+0000 7f47d9eda700 1 -- 172.21.15.12:0/3475876530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47d4101410 con 0x7f47d419bc50 2024-05-08T23:58:35.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.206+0000 7f47d0ff9700 1 -- 172.21.15.12:0/3475876530 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f47cc099a90 con 0x7f47d419bc50 2024-05-08T23:58:35.201 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:58:35.201 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:58:12.840713+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_28532":{"gid":28532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/1281701527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1281701527},{"type":"v1","addr":"172.21.15.204:6837","nonce":1281701527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:58:35.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 -- 172.21.15.12:0/3475876530 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47bc0a1290 msgr2=0x7f47bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47bc0a1290 0x7f47bc0a3750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f47c40093a0 tx=0x7f47c400b040 comp rx=0 tx=0).stop 2024-05-08T23:58:35.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 -- 172.21.15.12:0/3475876530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d419bc50 msgr2=0x7f47d419c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d419bc50 0x7f47d419c0d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00e940 tx=0x7f47cc00ec50 comp rx=0 tx=0).stop 2024-05-08T23:58:35.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 -- 172.21.15.12:0/3475876530 shutdown_connections 2024-05-08T23:58:35.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47bc0a1290 0x7f47bc0a3750 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d419bc50 0x7f47d419c0d0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47d40fff40 0x7f47d4197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 --2- 172.21.15.12:0/3475876530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4105060 0x7f47d41978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 -- 172.21.15.12:0/3475876530 >> 172.21.15.12:0/3475876530 conn(0x7f47d40fb7e0 msgr2=0x7f47d40683f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 -- 172.21.15.12:0/3475876530 shutdown_connections 2024-05-08T23:58:35.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.209+0000 7f47d9eda700 1 -- 172.21.15.12:0/3475876530 wait complete. 2024-05-08T23:58:35.205 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 52 2024-05-08T23:58:35.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:58:35.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.363+0000 7fda1dc84700 1 -- 172.21.15.12:0/255911257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda18105170 msgr2=0x7fda1810ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.363+0000 7fda1dc84700 1 --2- 172.21.15.12:0/255911257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda18105170 0x7fda1810ccf0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fda1000b0a0 tx=0x7fda1000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.363+0000 7fda1dc84700 1 -- 172.21.15.12:0/255911257 shutdown_connections 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.363+0000 7fda1dc84700 1 --2- 172.21.15.12:0/255911257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda18105170 0x7fda1810ccf0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.363+0000 7fda1dc84700 1 --2- 172.21.15.12:0/255911257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda18100050 0x7fda18100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.363+0000 7fda1dc84700 1 --2- 172.21.15.12:0/255911257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda18100a00 0x7fda18104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.363+0000 7fda1dc84700 1 -- 172.21.15.12:0/255911257 >> 172.21.15.12:0/255911257 conn(0x7fda18076fa0 msgr2=0x7fda180773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.363+0000 7fda1dc84700 1 -- 172.21.15.12:0/255911257 shutdown_connections 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.363+0000 7fda1dc84700 1 -- 172.21.15.12:0/255911257 wait complete. 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.364+0000 7fda1dc84700 1 Processor -- start 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.364+0000 7fda1dc84700 1 -- start start 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.364+0000 7fda1dc84700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda18100050 0x7fda18197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.364+0000 7fda1dc84700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda18105170 0x7fda18197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.364+0000 7fda1dc84700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda1819be20 0x7fda1819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.364+0000 7fda1dc84700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1819c8a0 con 0x7fda1819be20 2024-05-08T23:58:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.364+0000 7fda1dc84700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1819ca10 con 0x7fda18100050 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.364+0000 7fda1dc84700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1819cbd0 con 0x7fda18105170 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda17fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda1819be20 0x7fda1819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda17fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda1819be20 0x7fda1819c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33528/0 (socket says 172.21.15.12:33528) 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda177fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda18100050 0x7fda18197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda17fff700 1 -- 172.21.15.12:0/468070334 learned_addr learned my addr 172.21.15.12:0/468070334 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda16ffd700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda18105170 0x7fda18197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda17fff700 1 -- 172.21.15.12:0/468070334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda18105170 msgr2=0x7fda18197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda17fff700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda18105170 0x7fda18197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda17fff700 1 -- 172.21.15.12:0/468070334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda18100050 msgr2=0x7fda18197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda17fff700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda18100050 0x7fda18197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda17fff700 1 -- 172.21.15.12:0/468070334 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda10009d00 con 0x7fda1819be20 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda16ffd700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda18105170 0x7fda18197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda17fff700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda1819be20 0x7fda1819c2a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fda1000b070 tx=0x7fda10007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda14ff9700 1 -- 172.21.15.12:0/468070334 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1000e040 con 0x7fda1819be20 2024-05-08T23:58:35.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda14ff9700 1 -- 172.21.15.12:0/468070334 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1000bdc0 con 0x7fda1819be20 2024-05-08T23:58:35.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda14ff9700 1 -- 172.21.15.12:0/468070334 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda100052a0 con 0x7fda1819be20 2024-05-08T23:58:35.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda1dc84700 1 -- 172.21.15.12:0/468070334 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda181a40d0 con 0x7fda1819be20 2024-05-08T23:58:35.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.365+0000 7fda1dc84700 1 -- 172.21.15.12:0/468070334 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda181a44f0 con 0x7fda1819be20 2024-05-08T23:58:35.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.368+0000 7fda14ff9700 1 -- 172.21.15.12:0/468070334 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fda1001d480 con 0x7fda1819be20 2024-05-08T23:58:35.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.368+0000 7fda14ff9700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fda040a1310 0x7fda040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.368+0000 7fda177fe700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fda040a1310 0x7fda040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.368+0000 7fda14ff9700 1 -- 172.21.15.12:0/468070334 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fda100cdbc0 con 0x7fda1819be20 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.369+0000 7fda177fe700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fda040a1310 0x7fda040a37d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fda08009c60 tx=0x7fda08009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.369+0000 7fda29e3eb80 1 Processor -- start 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.369+0000 7fda29e3eb80 1 -- start start 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.369+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 0x557fec3021e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.369+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec2fc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.369+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec2fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.369+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec253150 con 0x557fec302720 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.369+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec250b10 con 0x557fec301e00 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.369+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec24c400 con 0x557fec2fc890 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec2fc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec2fc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33538/0 (socket says 172.21.15.12:33538) 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda16ffd700 1 -- 172.21.15.12:0/3661358810 learned_addr learned my addr 172.21.15.12:0/3661358810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda17fff700 1 --2- 172.21.15.12:0/3661358810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec2fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda177fe700 1 --2- 172.21.15.12:0/3661358810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 0x557fec3021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda16ffd700 1 -- 172.21.15.12:0/3661358810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 msgr2=0x557fec2fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda16ffd700 1 --2- 172.21.15.12:0/3661358810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec2fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda16ffd700 1 -- 172.21.15.12:0/3661358810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 msgr2=0x557fec3021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda16ffd700 1 --2- 172.21.15.12:0/3661358810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 0x557fec3021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda16ffd700 1 -- 172.21.15.12:0/3661358810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fec0cd3b0 con 0x557fec302720 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda16ffd700 1 --2- 172.21.15.12:0/3661358810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec2fc350 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fda0000dcb0 tx=0x7fda0000f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14db3644b42355fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda0dffb700 1 -- 172.21.15.12:0/3661358810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0000fc90 con 0x557fec302720 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda0dffb700 1 -- 172.21.15.12:0/3661358810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda0000fdf0 con 0x557fec302720 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.370+0000 7fda0dffb700 1 -- 172.21.15.12:0/3661358810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00010660 con 0x557fec302720 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.371+0000 7fda29e3eb80 1 -- 172.21.15.12:0/3661358810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 msgr2=0x557fec2fc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.371+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/3661358810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec2fc350 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fda0000dcb0 tx=0x7fda0000f360 comp rx=0 tx=0).stop 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.371+0000 7fda29e3eb80 1 -- 172.21.15.12:0/3661358810 shutdown_connections 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.371+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/3661358810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec2fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.371+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/3661358810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 0x557fec3021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.371+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/3661358810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec2fc350 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.371+0000 7fda29e3eb80 1 -- 172.21.15.12:0/3661358810 >> 172.21.15.12:0/3661358810 conn(0x557fec25a210 msgr2=0x557fec2f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.371+0000 7fda29e3eb80 1 -- 172.21.15.12:0/3661358810 shutdown_connections 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.371+0000 7fda29e3eb80 1 -- 172.21.15.12:0/3661358810 wait complete. 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.371+0000 7fda29e3eb80 1 -- start start 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda29e3eb80 1 Processor -- start 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec2fc890 0x557fec305b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec3060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec302720 0x557fec3065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec1e8560 con 0x557fec301e00 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec1e8200 con 0x557fec2fc890 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec0cf6a0 con 0x557fec302720 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda29e3eb80 10 client.0 _ll_register_callbacks cb 0x557fec1f1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec3060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec3060b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33540/0 (socket says 172.21.15.12:33540) 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.372+0000 7fda16ffd700 1 -- 172.21.15.12:0/599689439 learned_addr learned my addr 172.21.15.12:0/599689439 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda177fe700 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec2fc890 0x557fec305b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda17fff700 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec302720 0x557fec3065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda16ffd700 1 -- 172.21.15.12:0/599689439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec302720 msgr2=0x557fec3065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda16ffd700 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec302720 0x557fec3065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda16ffd700 1 -- 172.21.15.12:0/599689439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec2fc890 msgr2=0x557fec305b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda16ffd700 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec2fc890 0x557fec305b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda16ffd700 1 -- 172.21.15.12:0/599689439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0000d8e0 con 0x557fec301e00 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda16ffd700 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec3060b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fda0000f760 tx=0x7fda00011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaba710d61f6fbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fd9f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/599689439 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00029070 con 0x557fec301e00 2024-05-08T23:58:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda29e3eb80 1 -- 172.21.15.12:0/599689439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fec1a5270 con 0x557fec301e00 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/599689439 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fda00011650 con 0x557fec301e00 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/599689439 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda00011810 con 0x557fec301e00 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda29e3eb80 10 client.18861 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.373+0000 7fda29e3eb80 1 -- 172.21.15.12:0/599689439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x557fec079b60 con 0x557fec301e00 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.374+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/599689439 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557fec1a5270 con 0x557fec301e00 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.374+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/599689439 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fda0002c030 con 0x557fec301e00 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.374+0000 7fda29e3eb80 10 client.18861 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.374+0000 7fda29e3eb80 10 client.18861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.374+0000 7fda29e3eb80 20 client.18861 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.374+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557fec3160e0 0x557fec318500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.374+0000 7fda29e3eb80 4 client.18861 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.374+0000 7fda29e3eb80 1 -- 172.21.15.12:0/599689439 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fec2cb810 con 0x557fec3160e0 2024-05-08T23:58:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.374+0000 7fda177fe700 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557fec3160e0 0x557fec318500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.375+0000 7fda177fe700 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557fec3160e0 0x557fec318500 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c254a9717f316722 server_cookie=f3e91364464fd4e1 in_seq=0 out_seq=0 2024-05-08T23:58:35.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.375+0000 7fd9f6ffd700 10 client.18861 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.375+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/599689439 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fec2cb810 con 0x557fec3160e0 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.375+0000 7fd9f6ffd700 10 client.18861 handle_command_reply: tid=0 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.375+0000 7fda29e3eb80 1 client.18861 shutdown 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 1 -- 172.21.15.12:0/599689439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 msgr2=0x557fec3060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec3060b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fda0000f760 tx=0x7fda00011bf0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 1 -- 172.21.15.12:0/599689439 shutdown_connections 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557fec3160e0 0x557fec318500 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec302720 0x557fec3065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec2fc890 0x557fec305b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fd9f6ffd700 0 client.18861 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:58:35.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/599689439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec3060b0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 1 -- 172.21.15.12:0/599689439 >> 172.21.15.12:0/599689439 conn(0x557fec25a210 msgr2=0x557fec2f65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 1 -- 172.21.15.12:0/599689439 shutdown_connections 2024-05-08T23:58:35.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 1 -- 172.21.15.12:0/599689439 wait complete. 2024-05-08T23:58:35.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.376+0000 7fda29e3eb80 20 client.18861 trim_cache size 0 max 16384 2024-05-08T23:58:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda29e3eb80 1 Processor -- start 2024-05-08T23:58:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda29e3eb80 1 -- start start 2024-05-08T23:58:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec311c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 0x557fec2f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec3060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec2f3e00 con 0x557fec302720 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec2f8790 con 0x557fec301e00 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec2fc6d0 con 0x557fec2fc890 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda17fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec3060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda17fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec3060e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33552/0 (socket says 172.21.15.12:33552) 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.390+0000 7fda17fff700 1 -- 172.21.15.12:0/2183880079 learned_addr learned my addr 172.21.15.12:0/2183880079 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda177fe700 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec311c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda16ffd700 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 0x557fec2f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda17fff700 1 -- 172.21.15.12:0/2183880079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 msgr2=0x557fec311c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda17fff700 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec311c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda17fff700 1 -- 172.21.15.12:0/2183880079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 msgr2=0x557fec2f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda17fff700 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 0x557fec2f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda17fff700 1 -- 172.21.15.12:0/2183880079 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fec212560 con 0x557fec302720 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda177fe700 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec311c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda16ffd700 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 0x557fec2f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda17fff700 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec3060e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fda100972a0 tx=0x7fda10009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5f3a0c023e600e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda0e7fc700 1 -- 172.21.15.12:0/2183880079 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1009b4a0 con 0x557fec302720 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda0e7fc700 1 -- 172.21.15.12:0/2183880079 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1009ada0 con 0x557fec302720 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.391+0000 7fda0e7fc700 1 -- 172.21.15.12:0/2183880079 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1009b260 con 0x557fec302720 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 -- 172.21.15.12:0/2183880079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 msgr2=0x557fec3060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec3060e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fda100972a0 tx=0x7fda10009040 comp rx=0 tx=0).stop 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 -- 172.21.15.12:0/2183880079 shutdown_connections 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec302720 0x557fec3060e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec311c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/2183880079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec301e00 0x557fec2f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 -- 172.21.15.12:0/2183880079 >> 172.21.15.12:0/2183880079 conn(0x557fec25a210 msgr2=0x557fec304460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 -- 172.21.15.12:0/2183880079 shutdown_connections 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 -- 172.21.15.12:0/2183880079 wait complete. 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 -- start start 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.392+0000 7fda29e3eb80 1 Processor -- start 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec30b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec30b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda29e3eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec302720 0x557fec36f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec30bff0 con 0x557fec301e00 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec36fef0 con 0x557fec302720 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda29e3eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec3700b0 con 0x557fec2fc890 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda29e3eb80 10 client.0 _ll_register_callbacks cb 0x557fec1f1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec30b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec30b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33554/0 (socket says 172.21.15.12:33554) 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda16ffd700 1 -- 172.21.15.12:0/846172526 learned_addr learned my addr 172.21.15.12:0/846172526 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda17fff700 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec302720 0x557fec36f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda177fe700 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec30b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda16ffd700 1 -- 172.21.15.12:0/846172526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 msgr2=0x557fec30b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda16ffd700 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec30b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda16ffd700 1 -- 172.21.15.12:0/846172526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec302720 msgr2=0x557fec36f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda16ffd700 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec302720 0x557fec36f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda16ffd700 1 -- 172.21.15.12:0/846172526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda181a44f0 con 0x557fec301e00 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda177fe700 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec30b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda17fff700 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec302720 0x557fec36f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fda16ffd700 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec30b880 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fda0000fdc0 tx=0x7fda0000d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0095400e8b337d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.393+0000 7fd9f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/846172526 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00029040 con 0x557fec301e00 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/846172526 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fda0000f790 con 0x557fec301e00 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fda29e3eb80 1 -- 172.21.15.12:0/846172526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda0000fdf0 con 0x557fec301e00 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/846172526 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda00011650 con 0x557fec301e00 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fda29e3eb80 10 client.18867 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fda29e3eb80 1 -- 172.21.15.12:0/846172526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x557fec0d1cd0 con 0x557fec301e00 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/846172526 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda0000fdf0 con 0x557fec301e00 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/846172526 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fda0000dce0 con 0x557fec301e00 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fda29e3eb80 10 client.18867 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fda29e3eb80 10 client.18867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fda29e3eb80 20 client.18867 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557fec371560 0x557fec373980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fda29e3eb80 4 client.18867 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:58:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fda29e3eb80 1 -- 172.21.15.12:0/846172526 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557fec244ec0 con 0x557fec371560 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.394+0000 7fda177fe700 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557fec371560 0x557fec373980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fd9f6ffd700 10 client.18867 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda177fe700 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557fec371560 0x557fec373980 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22b1e01b0d1b4cf server_cookie=f2af7bfbf32f95ea in_seq=0 out_seq=0 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fd9f6ffd700 1 -- 172.21.15.12:0/846172526 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557fec244ec0 con 0x557fec371560 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fd9f6ffd700 10 client.18867 handle_command_reply: tid=0 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 client.18867 shutdown 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 -- 172.21.15.12:0/846172526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 msgr2=0x557fec30b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec30b880 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fda0000fdc0 tx=0x7fda0000d760 comp rx=0 tx=0).stop 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 -- 172.21.15.12:0/846172526 shutdown_connections 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fec302720 0x557fec36f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557fec371560 0x557fec373980 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fec2fc890 0x557fec30b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 --2- 172.21.15.12:0/846172526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fec301e00 0x557fec30b880 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 -- 172.21.15.12:0/846172526 >> 172.21.15.12:0/846172526 conn(0x557fec25a210 msgr2=0x557fec300c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fd9f6ffd700 0 client.18867 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:58:35.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 -- 172.21.15.12:0/846172526 shutdown_connections 2024-05-08T23:58:35.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.395+0000 7fda29e3eb80 1 -- 172.21.15.12:0/846172526 wait complete. 2024-05-08T23:58:35.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda29e3eb80 20 client.18867 trim_cache size 0 max 16384 2024-05-08T23:58:35.398 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:58:35.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 -- 172.21.15.12:0/468070334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fda040a1310 msgr2=0x7fda040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fda040a1310 0x7fda040a37d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fda08009c60 tx=0x7fda08009260 comp rx=0 tx=0).stop 2024-05-08T23:58:35.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 -- 172.21.15.12:0/468070334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda1819be20 msgr2=0x7fda1819c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda1819be20 0x7fda1819c2a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fda1000b070 tx=0x7fda10007cf0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 -- 172.21.15.12:0/468070334 shutdown_connections 2024-05-08T23:58:35.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fda040a1310 0x7fda040a37d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda1819be20 0x7fda1819c2a0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda18100050 0x7fda18197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 --2- 172.21.15.12:0/468070334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda18105170 0x7fda18197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 -- 172.21.15.12:0/468070334 >> 172.21.15.12:0/468070334 conn(0x7fda18076fa0 msgr2=0x7fda180697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 -- 172.21.15.12:0/468070334 shutdown_connections 2024-05-08T23:58:35.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.396+0000 7fda1dc84700 1 -- 172.21.15.12:0/468070334 wait complete. 2024-05-08T23:58:35.404 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:58:35.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:35 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3475876530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:58:35.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:35 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3475876530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:58:35.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.549+0000 7f70aaac2700 1 -- 172.21.15.12:0/3584121832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a4100a10 msgr2=0x7f70a4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.549+0000 7f70aaac2700 1 --2- 172.21.15.12:0/3584121832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a4100a10 0x7f70a4104a30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f7090009990 tx=0x7f7090009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.549+0000 7f70aaac2700 1 -- 172.21.15.12:0/3584121832 shutdown_connections 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.549+0000 7f70aaac2700 1 --2- 172.21.15.12:0/3584121832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a4105160 0x7f70a410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.549+0000 7f70aaac2700 1 --2- 172.21.15.12:0/3584121832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a4100060 0x7f70a4100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.549+0000 7f70aaac2700 1 --2- 172.21.15.12:0/3584121832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a4100a10 0x7f70a4104a30 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.549+0000 7f70aaac2700 1 -- 172.21.15.12:0/3584121832 >> 172.21.15.12:0/3584121832 conn(0x7f70a4076f40 msgr2=0x7f70a4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.549+0000 7f70aaac2700 1 -- 172.21.15.12:0/3584121832 shutdown_connections 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.549+0000 7f70aaac2700 1 -- 172.21.15.12:0/3584121832 wait complete. 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70aaac2700 1 Processor -- start 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70aaac2700 1 -- start start 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70aaac2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a4100060 0x7f70a41973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70aaac2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a4105160 0x7f70a4197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70aaac2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a419bc70 0x7f70a419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70aaac2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a419c6f0 con 0x7f70a419bc70 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70aaac2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a419c860 con 0x7f70a4105160 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70aaac2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a419ca20 con 0x7f70a4100060 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70a905f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a419bc70 0x7f70a419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70a905f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a419bc70 0x7f70a419c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33574/0 (socket says 172.21.15.12:33574) 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70a905f700 1 -- 172.21.15.12:0/1357620838 learned_addr learned my addr 172.21.15.12:0/1357620838 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70a885e700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a4100060 0x7f70a41973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70a3fff700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a4105160 0x7f70a4197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70a905f700 1 -- 172.21.15.12:0/1357620838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a4100060 msgr2=0x7f70a41973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70a905f700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a4100060 0x7f70a41973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70a905f700 1 -- 172.21.15.12:0/1357620838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a4105160 msgr2=0x7f70a4197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70a905f700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a4105160 0x7f70a4197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.550+0000 7f70a905f700 1 -- 172.21.15.12:0/1357620838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70900095f0 con 0x7f70a419bc70 2024-05-08T23:58:35.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.551+0000 7f70a905f700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a419bc70 0x7f70a419c0f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f709c007ee0 tx=0x7f709c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.551+0000 7f70a1ffb700 1 -- 172.21.15.12:0/1357620838 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c017070 con 0x7f70a419bc70 2024-05-08T23:58:35.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.551+0000 7f70a1ffb700 1 -- 172.21.15.12:0/1357620838 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c00f040 con 0x7f70a419bc70 2024-05-08T23:58:35.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.551+0000 7f70aaac2700 1 -- 172.21.15.12:0/1357620838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a41a3f80 con 0x7f70a419bc70 2024-05-08T23:58:35.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.551+0000 7f70a1ffb700 1 -- 172.21.15.12:0/1357620838 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c005580 con 0x7f70a419bc70 2024-05-08T23:58:35.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.551+0000 7f70aaac2700 1 -- 172.21.15.12:0/1357620838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a41a43b0 con 0x7f70a419bc70 2024-05-08T23:58:35.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.554+0000 7f70a1ffb700 1 -- 172.21.15.12:0/1357620838 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f709c00db20 con 0x7f70a419bc70 2024-05-08T23:58:35.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.555+0000 7f70a1ffb700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70940a1290 0x7f70940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.555+0000 7f70a1ffb700 1 -- 172.21.15.12:0/1357620838 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f709c0cc370 con 0x7f70a419bc70 2024-05-08T23:58:35.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.555+0000 7f70a885e700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70940a1290 0x7f70940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.555+0000 7f70a885e700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70940a1290 0x7f70940a3750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7098005950 tx=0x7f709800b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70b6c7cb80 1 Processor -- start 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70b6c7cb80 1 -- start start 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 0x55659f4f5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4ef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f442150 con 0x55659f4efcf0 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f43fb10 con 0x55659f4f5b80 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f43b400 con 0x55659f4f5260 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70a905f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70a905f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33590/0 (socket says 172.21.15.12:33590) 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70a885e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 0x55659f4f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70a905f700 1 -- 172.21.15.12:0/2420512152 learned_addr learned my addr 172.21.15.12:0/2420512152 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70a3fff700 1 --2- 172.21.15.12:0/2420512152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70a905f700 1 -- 172.21.15.12:0/2420512152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 msgr2=0x55659f4f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70a905f700 1 --2- 172.21.15.12:0/2420512152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 0x55659f4f5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70a905f700 1 -- 172.21.15.12:0/2420512152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 msgr2=0x55659f4ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70a905f700 1 --2- 172.21.15.12:0/2420512152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.556+0000 7f70a905f700 1 -- 172.21.15.12:0/2420512152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55659f2bc3b0 con 0x55659f4efcf0 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.557+0000 7f70a905f700 1 --2- 172.21.15.12:0/2420512152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f0170 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f709c095ed0 tx=0x7f709c0957a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee32a114c313d3e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.557+0000 7f708effd700 1 -- 172.21.15.12:0/2420512152 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c09d290 con 0x55659f4efcf0 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.557+0000 7f708effd700 1 -- 172.21.15.12:0/2420512152 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c0a18e0 con 0x55659f4efcf0 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.557+0000 7f708effd700 1 -- 172.21.15.12:0/2420512152 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c022250 con 0x55659f4efcf0 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.557+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/2420512152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 msgr2=0x55659f4f0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.557+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/2420512152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f0170 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f709c095ed0 tx=0x7f709c0957a0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.557+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/2420512152 shutdown_connections 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.557+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/2420512152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f0170 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.557+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/2420512152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 0x55659f4f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.557+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/2420512152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.558+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/2420512152 >> 172.21.15.12:0/2420512152 conn(0x55659f449210 msgr2=0x55659f4e5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.558+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/2420512152 shutdown_connections 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.558+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/2420512152 wait complete. 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.558+0000 7f70b6c7cb80 1 -- start start 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.558+0000 7f70b6c7cb80 1 Processor -- start 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.558+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 0x55659f4f9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4f9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f3d7560 con 0x55659f4efcf0 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f3d7200 con 0x55659f4f5b80 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f2be6a0 con 0x55659f4f5260 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70b6c7cb80 10 client.0 _ll_register_callbacks cb 0x55659f3e0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:35.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a885e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 0x55659f4f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a885e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33606/0 (socket says 172.21.15.12:33606) 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a885e700 1 -- 172.21.15.12:0/1955238194 learned_addr learned my addr 172.21.15.12:0/1955238194 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a905f700 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a885e700 1 -- 172.21.15.12:0/1955238194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 msgr2=0x55659f4f9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a885e700 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 0x55659f4f9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a885e700 1 -- 172.21.15.12:0/1955238194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 msgr2=0x55659f4f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a885e700 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a885e700 1 -- 172.21.15.12:0/1955238194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a41a43b0 con 0x55659f4efcf0 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a3fff700 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 0x55659f4f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70a885e700 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f8f10 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f70980116f0 tx=0x7f70980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f162bc2e4cd9faae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f7083fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:35.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f7083fff700 1 -- 172.21.15.12:0/1955238194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098028470 con 0x55659f4efcf0 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f7083fff700 1 -- 172.21.15.12:0/1955238194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7098017070 con 0x55659f4efcf0 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.559+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/1955238194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55659f394270 con 0x55659f4efcf0 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f7083fff700 1 -- 172.21.15.12:0/1955238194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70980285d0 con 0x55659f4efcf0 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f70b6c7cb80 10 client.18879 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/1955238194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55659f268b60 con 0x55659f4efcf0 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f7083fff700 1 -- 172.21.15.12:0/1955238194 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55659f394270 con 0x55659f4efcf0 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f7083fff700 1 -- 172.21.15.12:0/1955238194 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f70980315b0 con 0x55659f4efcf0 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f70b6c7cb80 10 client.18879 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f70b6c7cb80 10 client.18879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f70b6c7cb80 20 client.18879 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55659f509610 0x55659f50ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f70b6c7cb80 4 client.18879 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/1955238194 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55659f4ba810 con 0x55659f509610 2024-05-08T23:58:35.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.560+0000 7f70a3fff700 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55659f509610 0x55659f50ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.561+0000 7f70a3fff700 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55659f509610 0x55659f50ba30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d05cc36ab2d36280 server_cookie=8efb611d775e893c in_seq=0 out_seq=0 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.561+0000 7f7083fff700 10 client.18879 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f7083fff700 1 -- 172.21.15.12:0/1955238194 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55659f4ba810 con 0x55659f509610 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f7083fff700 10 client.18879 handle_command_reply: tid=0 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 client.18879 shutdown 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/1955238194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 msgr2=0x55659f4f8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f8f10 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f70980116f0 tx=0x7f70980047c0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/1955238194 shutdown_connections 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55659f509610 0x55659f50ba30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4f8f10 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/1955238194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5260 0x55659f4f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f7083fff700 0 client.18879 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:58:35.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/1955238194 >> 172.21.15.12:0/1955238194 conn(0x55659f449210 msgr2=0x55659f4e55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/1955238194 shutdown_connections 2024-05-08T23:58:35.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/1955238194 wait complete. 2024-05-08T23:58:35.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.562+0000 7f70b6c7cb80 20 client.18879 trim_cache size 0 max 16384 2024-05-08T23:58:35.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.576+0000 7f70b6c7cb80 1 Processor -- start 2024-05-08T23:58:35.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.576+0000 7f70b6c7cb80 1 -- start start 2024-05-08T23:58:35.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.576+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4efcf0 0x55659f505020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.576+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4f5260 0x55659f4e5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.576+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4f9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.576+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f4de040 con 0x55659f4f5260 2024-05-08T23:58:35.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.576+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f4f5890 con 0x55659f4f5b80 2024-05-08T23:58:35.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.576+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f510be0 con 0x55659f4efcf0 2024-05-08T23:58:35.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4f5260 0x55659f4e5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4f5260 0x55659f4e5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33610/0 (socket says 172.21.15.12:33610) 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a3fff700 1 -- 172.21.15.12:0/2649450094 learned_addr learned my addr 172.21.15.12:0/2649450094 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a885e700 1 --2- 172.21.15.12:0/2649450094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4efcf0 0x55659f505020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a905f700 1 --2- 172.21.15.12:0/2649450094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a3fff700 1 -- 172.21.15.12:0/2649450094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4efcf0 msgr2=0x55659f505020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a3fff700 1 --2- 172.21.15.12:0/2649450094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4efcf0 0x55659f505020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a3fff700 1 -- 172.21.15.12:0/2649450094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 msgr2=0x55659f4f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a3fff700 1 --2- 172.21.15.12:0/2649450094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4f9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a3fff700 1 -- 172.21.15.12:0/2649450094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55659f401560 con 0x55659f4f5260 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f70a3fff700 1 --2- 172.21.15.12:0/2649450094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4f5260 0x55659f4e5d20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f70a406a6d0 tx=0x7f7090005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b22155608c6f368 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f708f7fe700 1 -- 172.21.15.12:0/2649450094 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709001d070 con 0x55659f4f5260 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f708f7fe700 1 -- 172.21.15.12:0/2649450094 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709001e040 con 0x55659f4f5260 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.577+0000 7f708f7fe700 1 -- 172.21.15.12:0/2649450094 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709002a070 con 0x55659f4f5260 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/2649450094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4f5260 msgr2=0x55659f4e5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/2649450094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4f5260 0x55659f4e5d20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f70a406a6d0 tx=0x7f7090005960 comp rx=0 tx=0).stop 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/2649450094 shutdown_connections 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/2649450094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5b80 0x55659f4f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/2649450094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4efcf0 0x55659f505020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/2649450094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4f5260 0x55659f4e5d20 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/2649450094 >> 172.21.15.12:0/2649450094 conn(0x55659f449210 msgr2=0x55659f4f2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/2649450094 shutdown_connections 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/2649450094 wait complete. 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 -- start start 2024-05-08T23:58:35.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.578+0000 7f70b6c7cb80 1 Processor -- start 2024-05-08T23:58:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5260 0x55659f4feca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70b6c7cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5b80 0x55659f55ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f4ff440 con 0x55659f4efcf0 2024-05-08T23:58:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f55f040 con 0x55659f4f5260 2024-05-08T23:58:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70b6c7cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659f55f200 con 0x55659f4f5b80 2024-05-08T23:58:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70b6c7cb80 10 client.0 _ll_register_callbacks cb 0x55659f3e0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70a885e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70a885e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4fe760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33622/0 (socket says 172.21.15.12:33622) 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70a885e700 1 -- 172.21.15.12:0/3247900228 learned_addr learned my addr 172.21.15.12:0/3247900228 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70a905f700 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5b80 0x55659f55ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.579+0000 7f70a3fff700 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5260 0x55659f4feca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70a885e700 1 -- 172.21.15.12:0/3247900228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5b80 msgr2=0x55659f55ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70a885e700 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5b80 0x55659f55ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70a885e700 1 -- 172.21.15.12:0/3247900228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5260 msgr2=0x55659f4feca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70a885e700 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5260 0x55659f4feca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70a885e700 1 -- 172.21.15.12:0/3247900228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a410d550 con 0x55659f4efcf0 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70a3fff700 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5260 0x55659f4feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70a905f700 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5b80 0x55659f55ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70a885e700 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4fe760 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f70980116c0 tx=0x7f7098031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef6d38361982f2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f7083fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f7083fff700 1 -- 172.21.15.12:0/3247900228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098010a70 con 0x55659f4efcf0 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f7083fff700 1 -- 172.21.15.12:0/3247900228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7098017040 con 0x55659f4efcf0 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/3247900228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f709c0a18e0 con 0x55659f4efcf0 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f7083fff700 1 -- 172.21.15.12:0/3247900228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7098031b60 con 0x55659f4efcf0 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70b6c7cb80 10 client.18885 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/3247900228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55659f2c0cd0 con 0x55659f4efcf0 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f7083fff700 1 -- 172.21.15.12:0/3247900228 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c0a18e0 con 0x55659f4efcf0 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f7083fff700 1 -- 172.21.15.12:0/3247900228 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f70980053d0 con 0x55659f4efcf0 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70b6c7cb80 10 client.18885 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.580+0000 7f70b6c7cb80 10 client.18885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.581+0000 7f70b6c7cb80 20 client.18885 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.581+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55659f5608b0 0x55659f562cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.581+0000 7f70b6c7cb80 4 client.18885 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:58:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.581+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/3247900228 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55659f433ec0 con 0x55659f5608b0 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.581+0000 7f70a3fff700 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55659f5608b0 0x55659f562cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.581+0000 7f70a3fff700 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55659f5608b0 0x55659f562cd0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55894f05fd224812 server_cookie=92f83646ff8867d2 in_seq=0 out_seq=0 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.581+0000 7f7083fff700 10 client.18885 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f7083fff700 1 -- 172.21.15.12:0/3247900228 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55659f433ec0 con 0x55659f5608b0 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f7083fff700 10 client.18885 handle_command_reply: tid=0 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 client.18885 shutdown 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/3247900228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 msgr2=0x55659f4fe760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4fe760 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f70980116c0 tx=0x7f7098031880 comp rx=0 tx=0).stop 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/3247900228 shutdown_connections 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55659f5608b0 0x55659f562cd0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55659f4f5b80 0x55659f55ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55659f4efcf0 0x55659f4fe760 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 --2- 172.21.15.12:0/3247900228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55659f4f5260 0x55659f4feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f7083fff700 0 client.18885 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:58:35.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/3247900228 >> 172.21.15.12:0/3247900228 conn(0x55659f449210 msgr2=0x55659f4f7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/3247900228 shutdown_connections 2024-05-08T23:58:35.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 1 -- 172.21.15.12:0/3247900228 wait complete. 2024-05-08T23:58:35.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70b6c7cb80 20 client.18885 trim_cache size 0 max 16384 2024-05-08T23:58:35.583 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:58:35.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70aaac2700 1 -- 172.21.15.12:0/1357620838 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70940a1290 msgr2=0x7f70940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70aaac2700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70940a1290 0x7f70940a3750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7098005950 tx=0x7f709800b410 comp rx=0 tx=0).stop 2024-05-08T23:58:35.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70aaac2700 1 -- 172.21.15.12:0/1357620838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a419bc70 msgr2=0x7f70a419c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.582+0000 7f70aaac2700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a419bc70 0x7f70a419c0f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f709c007ee0 tx=0x7f709c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.583+0000 7f70aaac2700 1 -- 172.21.15.12:0/1357620838 shutdown_connections 2024-05-08T23:58:35.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.583+0000 7f70aaac2700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70940a1290 0x7f70940a3750 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.583+0000 7f70aaac2700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a419bc70 0x7f70a419c0f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.583+0000 7f70aaac2700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70a4100060 0x7f70a41973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.583+0000 7f70aaac2700 1 --2- 172.21.15.12:0/1357620838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a4105160 0x7f70a4197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.583+0000 7f70aaac2700 1 -- 172.21.15.12:0/1357620838 >> 172.21.15.12:0/1357620838 conn(0x7f70a4076f40 msgr2=0x7f70a4068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.583+0000 7f70aaac2700 1 -- 172.21.15.12:0/1357620838 shutdown_connections 2024-05-08T23:58:35.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.583+0000 7f70aaac2700 1 -- 172.21.15.12:0/1357620838 wait complete. 2024-05-08T23:58:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:35 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3475876530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:58:35.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.735+0000 7fc93ed50700 1 -- 172.21.15.12:0/3074066991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc938069dd0 msgr2=0x7fc938105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.735+0000 7fc93ed50700 1 --2- 172.21.15.12:0/3074066991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc938069dd0 0x7fc938105b70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc924009990 tx=0x7fc924009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.735+0000 7fc93ed50700 1 -- 172.21.15.12:0/3074066991 shutdown_connections 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.735+0000 7fc93ed50700 1 --2- 172.21.15.12:0/3074066991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9381062a0 0x7fc93810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.735+0000 7fc93ed50700 1 --2- 172.21.15.12:0/3074066991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc938069420 0x7fc938069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.735+0000 7fc93ed50700 1 --2- 172.21.15.12:0/3074066991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc938069dd0 0x7fc938105b70 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.735+0000 7fc93ed50700 1 -- 172.21.15.12:0/3074066991 >> 172.21.15.12:0/3074066991 conn(0x7fc938076ba0 msgr2=0x7fc938076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.735+0000 7fc93ed50700 1 -- 172.21.15.12:0/3074066991 shutdown_connections 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.735+0000 7fc93ed50700 1 -- 172.21.15.12:0/3074066991 wait complete. 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc93ed50700 1 Processor -- start 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc93ed50700 1 -- start start 2024-05-08T23:58:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc93ed50700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc938069420 0x7fc9381a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc93ed50700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9381062a0 0x7fc9381a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc93ed50700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc93810cea0 0x7fc93810d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc93ed50700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93810d920 con 0x7fc9381062a0 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc93ed50700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93810da60 con 0x7fc938069420 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc93ed50700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93810dc20 con 0x7fc93810cea0 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc937fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9381062a0 0x7fc9381a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc937fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9381062a0 0x7fc9381a3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33640/0 (socket says 172.21.15.12:33640) 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc937fff700 1 -- 172.21.15.12:0/3447133887 learned_addr learned my addr 172.21.15.12:0/3447133887 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc93caec700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc938069420 0x7fc9381a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.736+0000 7fc93d2ed700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc93810cea0 0x7fc93810d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc937fff700 1 -- 172.21.15.12:0/3447133887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc93810cea0 msgr2=0x7fc93810d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc937fff700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc93810cea0 0x7fc93810d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc937fff700 1 -- 172.21.15.12:0/3447133887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc938069420 msgr2=0x7fc9381a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc937fff700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc938069420 0x7fc9381a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc937fff700 1 -- 172.21.15.12:0/3447133887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9240095f0 con 0x7fc9381062a0 2024-05-08T23:58:35.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc937fff700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9381062a0 0x7fc9381a3c20 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc9240059d0 tx=0x7fc924004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc935ffb700 1 -- 172.21.15.12:0/3447133887 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92401d070 con 0x7fc9381062a0 2024-05-08T23:58:35.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc93ed50700 1 -- 172.21.15.12:0/3447133887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9381aa4f0 con 0x7fc9381062a0 2024-05-08T23:58:35.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc935ffb700 1 -- 172.21.15.12:0/3447133887 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc924022470 con 0x7fc9381062a0 2024-05-08T23:58:35.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc935ffb700 1 -- 172.21.15.12:0/3447133887 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924017650 con 0x7fc9381062a0 2024-05-08T23:58:35.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.737+0000 7fc93ed50700 1 -- 172.21.15.12:0/3447133887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9381aa9c0 con 0x7fc9381062a0 2024-05-08T23:58:35.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.740+0000 7fc935ffb700 1 -- 172.21.15.12:0/3447133887 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc9240225e0 con 0x7fc9381062a0 2024-05-08T23:58:35.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.740+0000 7fc935ffb700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9280a1290 0x7fc9280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.740+0000 7fc935ffb700 1 -- 172.21.15.12:0/3447133887 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc9240cdf40 con 0x7fc9381062a0 2024-05-08T23:58:35.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.740+0000 7fc93caec700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9280a1290 0x7fc9280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc94af0ab80 1 Processor -- start 2024-05-08T23:58:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc94af0ab80 1 -- start start 2024-05-08T23:58:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc93caec700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9280a1290 0x7fc9280a3750 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009c60 tx=0x7fc92c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5ce00 0x560a3fb5d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5d720 0x560a3fb57350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb57cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3faae150 con 0x560a3fb5ce00 2024-05-08T23:58:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3faabb10 con 0x560a3fb5d720 2024-05-08T23:58:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3faa7400 con 0x560a3fb57890 2024-05-08T23:58:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc93caec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5ce00 0x560a3fb5d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc93caec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5ce00 0x560a3fb5d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33642/0 (socket says 172.21.15.12:33642) 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.741+0000 7fc93caec700 1 -- 172.21.15.12:0/2172142266 learned_addr learned my addr 172.21.15.12:0/2172142266 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc93d2ed700 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb57cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc937fff700 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5d720 0x560a3fb57350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc93caec700 1 -- 172.21.15.12:0/2172142266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 msgr2=0x560a3fb57cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc93caec700 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb57cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc93caec700 1 -- 172.21.15.12:0/2172142266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5d720 msgr2=0x560a3fb57350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc93caec700 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5d720 0x560a3fb57350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc93caec700 1 -- 172.21.15.12:0/2172142266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a3f9283b0 con 0x560a3fb5ce00 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc937fff700 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5d720 0x560a3fb57350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc93d2ed700 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb57cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc93caec700 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5ce00 0x560a3fb5d1e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e4e0 tx=0x7fc92c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0d70cca89c411b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc922ffd700 1 -- 172.21.15.12:0/2172142266 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c026430 con 0x560a3fb5ce00 2024-05-08T23:58:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc922ffd700 1 -- 172.21.15.12:0/2172142266 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92c021070 con 0x560a3fb5ce00 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.742+0000 7fc922ffd700 1 -- 172.21.15.12:0/2172142266 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c01d6e0 con 0x560a3fb5ce00 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 -- 172.21.15.12:0/2172142266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5ce00 msgr2=0x560a3fb5d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5ce00 0x560a3fb5d1e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e4e0 tx=0x7fc92c014000 comp rx=0 tx=0).stop 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 -- 172.21.15.12:0/2172142266 shutdown_connections 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb57cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5ce00 0x560a3fb5d1e0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/2172142266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5d720 0x560a3fb57350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 -- 172.21.15.12:0/2172142266 >> 172.21.15.12:0/2172142266 conn(0x560a3fab5210 msgr2=0x560a3fb51df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 -- 172.21.15.12:0/2172142266 shutdown_connections 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 -- 172.21.15.12:0/2172142266 wait complete. 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 -- start start 2024-05-08T23:58:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.743+0000 7fc94af0ab80 1 Processor -- start 2024-05-08T23:58:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb60a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb60f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3fa43560 con 0x560a3fb5d720 2024-05-08T23:58:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3fa43200 con 0x560a3fb5ce00 2024-05-08T23:58:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3f92a6a0 con 0x560a3fb57890 2024-05-08T23:58:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc94af0ab80 10 client.0 _ll_register_callbacks cb 0x560a3fa4cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc93d2ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc93d2ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33644/0 (socket says 172.21.15.12:33644) 2024-05-08T23:58:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc93d2ed700 1 -- 172.21.15.12:0/3146369163 learned_addr learned my addr 172.21.15.12:0/3146369163 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc93caec700 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb60a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc937fff700 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb60f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc93d2ed700 1 -- 172.21.15.12:0/3146369163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 msgr2=0x560a3fb60a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc93d2ed700 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb60a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc93d2ed700 1 -- 172.21.15.12:0/3146369163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 msgr2=0x560a3fb60f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc93d2ed700 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb60f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.744+0000 7fc93d2ed700 1 -- 172.21.15.12:0/3146369163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c017070 con 0x560a3fb5d720 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc93d2ed700 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb614b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fc930007c60 tx=0x7fc930007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5635b83868cd88dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc917fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc917fff700 1 -- 172.21.15.12:0/3146369163 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93000e040 con 0x560a3fb5d720 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc917fff700 1 -- 172.21.15.12:0/3146369163 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc930013970 con 0x560a3fb5d720 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3146369163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a3fa00270 con 0x560a3fb5d720 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc917fff700 1 -- 172.21.15.12:0/3146369163 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc930013b30 con 0x560a3fb5d720 2024-05-08T23:58:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc94af0ab80 10 client.18897 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3146369163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x560a3f8d4b60 con 0x560a3fb5d720 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc917fff700 1 -- 172.21.15.12:0/3146369163 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a3fa00270 con 0x560a3fb5d720 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc917fff700 1 -- 172.21.15.12:0/3146369163 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc93001e3b0 con 0x560a3fb5d720 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc94af0ab80 10 client.18897 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc94af0ab80 10 client.18897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc94af0ab80 20 client.18897 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a3fb71180 0x560a3fb735a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc94af0ab80 4 client.18897 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.745+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3146369163 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a3fb26810 con 0x560a3fb71180 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.746+0000 7fc93caec700 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a3fb71180 0x560a3fb735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.746+0000 7fc93caec700 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a3fb71180 0x560a3fb735a0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6199977dceecaf3 server_cookie=2c7840cf63ed9b65 in_seq=0 out_seq=0 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.746+0000 7fc917fff700 10 client.18897 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc917fff700 1 -- 172.21.15.12:0/3146369163 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a3fb26810 con 0x560a3fb71180 2024-05-08T23:58:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc917fff700 10 client.18897 handle_command_reply: tid=0 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 client.18897 shutdown 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3146369163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 msgr2=0x560a3fb614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb614b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fc930007c60 tx=0x7fc930007c90 comp rx=0 tx=0).stop 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3146369163 shutdown_connections 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb614b0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb60a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a3fb71180 0x560a3fb735a0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/3146369163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb60f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3146369163 >> 172.21.15.12:0/3146369163 conn(0x560a3fab5210 msgr2=0x560a3fb515a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc917fff700 0 client.18897 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3146369163 shutdown_connections 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3146369163 wait complete. 2024-05-08T23:58:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.747+0000 7fc94af0ab80 20 client.18897 trim_cache size 0 max 16384 2024-05-08T23:58:35.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.761+0000 7fc94af0ab80 1 Processor -- start 2024-05-08T23:58:35.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.761+0000 7fc94af0ab80 1 -- start start 2024-05-08T23:58:35.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb5d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb57c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb51d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3fb582a0 con 0x560a3fb5d720 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3fb4edc0 con 0x560a3fb5ce00 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3fb4c800 con 0x560a3fb57890 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93d2ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb51d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93d2ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb51d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33658/0 (socket says 172.21.15.12:33658) 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93d2ed700 1 -- 172.21.15.12:0/3477849962 learned_addr learned my addr 172.21.15.12:0/3477849962 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93caec700 1 --2- 172.21.15.12:0/3477849962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb5d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc937fff700 1 --2- 172.21.15.12:0/3477849962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb57c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93d2ed700 1 -- 172.21.15.12:0/3477849962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 msgr2=0x560a3fb5d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93d2ed700 1 --2- 172.21.15.12:0/3477849962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb5d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93d2ed700 1 -- 172.21.15.12:0/3477849962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 msgr2=0x560a3fb57c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93d2ed700 1 --2- 172.21.15.12:0/3477849962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb57c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93d2ed700 1 -- 172.21.15.12:0/3477849962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a3fa6d560 con 0x560a3fb5d720 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93caec700 1 --2- 172.21.15.12:0/3477849962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb5d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc93d2ed700 1 --2- 172.21.15.12:0/3477849962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb51d20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc930000f80 tx=0x7fc930013330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3191cd87f08db2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.762+0000 7fc9237fe700 1 -- 172.21.15.12:0/3477849962 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93000e040 con 0x560a3fb5d720 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc9237fe700 1 -- 172.21.15.12:0/3477849962 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc93000f040 con 0x560a3fb5d720 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc9237fe700 1 -- 172.21.15.12:0/3477849962 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc930016430 con 0x560a3fb5d720 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3477849962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 msgr2=0x560a3fb51d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/3477849962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb51d20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc930000f80 tx=0x7fc930013330 comp rx=0 tx=0).stop 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3477849962 shutdown_connections 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/3477849962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb51d20 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/3477849962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb5d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/3477849962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb57c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3477849962 >> 172.21.15.12:0/3477849962 conn(0x560a3fab5210 msgr2=0x560a3fb77f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3477849962 shutdown_connections 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc94af0ab80 1 -- 172.21.15.12:0/3477849962 wait complete. 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.763+0000 7fc94af0ab80 1 -- start start 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc94af0ab80 1 Processor -- start 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb68180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc94af0ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb64210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3fb648f0 con 0x560a3fb5d720 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3fb64a60 con 0x560a3fb5ce00 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc94af0ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3fb64c20 con 0x560a3fb57890 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc94af0ab80 10 client.0 _ll_register_callbacks cb 0x560a3fa4cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc937fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc93d2ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb64210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc937fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb686c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:59298/0 (socket says 172.21.15.12:59298) 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc937fff700 1 -- 172.21.15.12:0/1083447504 learned_addr learned my addr 172.21.15.12:0/1083447504 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.764+0000 7fc93caec700 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb68180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc937fff700 1 -- 172.21.15.12:0/1083447504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 msgr2=0x560a3fb68180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc937fff700 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb68180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc937fff700 1 -- 172.21.15.12:0/1083447504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 msgr2=0x560a3fb64210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc937fff700 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb64210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc937fff700 1 -- 172.21.15.12:0/1083447504 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc930016e70 con 0x560a3fb5ce00 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc93caec700 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb68180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc937fff700 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb686c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc92400f740 tx=0x7fc924022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a54f26bb49bc44b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc917fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:58:35.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc917fff700 1 -- 172.21.15.12:0/1083447504 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92409d030 con 0x560a3fb5ce00 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc94af0ab80 1 -- 172.21.15.12:0/1083447504 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc92c021070 con 0x560a3fb5ce00 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc917fff700 1 -- 172.21.15.12:0/1083447504 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fc92409e070 con 0x560a3fb5ce00 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc917fff700 1 -- 172.21.15.12:0/1083447504 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc92409b820 con 0x560a3fb5ce00 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc94af0ab80 10 client.28616 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc94af0ab80 1 -- 172.21.15.12:0/1083447504 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x560a3f92ccd0 con 0x560a3fb5ce00 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.765+0000 7fc917fff700 1 -- 172.21.15.12:0/1083447504 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92c021070 con 0x560a3fb5ce00 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc917fff700 1 -- 172.21.15.12:0/1083447504 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc9240b92c0 con 0x560a3fb5ce00 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc94af0ab80 10 client.28616 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc94af0ab80 10 client.28616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc94af0ab80 20 client.28616 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a3fbc76c0 0x560a3fbc9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc94af0ab80 4 client.28616 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc94af0ab80 1 -- 172.21.15.12:0/1083447504 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a3fa9fec0 con 0x560a3fbc76c0 2024-05-08T23:58:35.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc93caec700 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a3fbc76c0 0x560a3fbc9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc93caec700 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a3fbc76c0 0x560a3fbc9ab0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5833aaa56f436569 server_cookie=6420874f96a67a6e in_seq=0 out_seq=0 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc917fff700 10 client.28616 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc917fff700 1 -- 172.21.15.12:0/1083447504 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a3fa9fec0 con 0x560a3fbc76c0 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc917fff700 10 client.28616 handle_command_reply: tid=0 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.766+0000 7fc94af0ab80 1 client.28616 shutdown 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 1 -- 172.21.15.12:0/1083447504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 msgr2=0x560a3fb686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb686c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc92400f740 tx=0x7fc924022f90 comp rx=0 tx=0).stop 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 1 -- 172.21.15.12:0/1083447504 shutdown_connections 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a3fb5d720 0x560a3fb64210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a3fb57890 0x560a3fb68180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a3fbc76c0 0x560a3fbc9ab0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 1 --2- 172.21.15.12:0/1083447504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a3fb5ce00 0x560a3fb686c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc917fff700 0 client.28616 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:35.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 1 -- 172.21.15.12:0/1083447504 >> 172.21.15.12:0/1083447504 conn(0x560a3fab5210 msgr2=0x560a3fb5abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 1 -- 172.21.15.12:0/1083447504 shutdown_connections 2024-05-08T23:58:35.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 1 -- 172.21.15.12:0/1083447504 wait complete. 2024-05-08T23:58:35.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc94af0ab80 20 client.28616 trim_cache size 0 max 16384 2024-05-08T23:58:35.769 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc93ed50700 1 -- 172.21.15.12:0/3447133887 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9280a1290 msgr2=0x7fc9280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc93ed50700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9280a1290 0x7fc9280a3750 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009c60 tx=0x7fc92c009260 comp rx=0 tx=0).stop 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc93ed50700 1 -- 172.21.15.12:0/3447133887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9381062a0 msgr2=0x7fc9381a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc93ed50700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9381062a0 0x7fc9381a3c20 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc9240059d0 tx=0x7fc924004730 comp rx=0 tx=0).stop 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc93ed50700 1 -- 172.21.15.12:0/3447133887 shutdown_connections 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc93ed50700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc9280a1290 0x7fc9280a3750 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc93ed50700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc93810cea0 0x7fc93810d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc93ed50700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc938069420 0x7fc9381a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.767+0000 7fc93ed50700 1 --2- 172.21.15.12:0/3447133887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9381062a0 0x7fc9381a3c20 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.768+0000 7fc93ed50700 1 -- 172.21.15.12:0/3447133887 >> 172.21.15.12:0/3447133887 conn(0x7fc938076ba0 msgr2=0x7fc938101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.768+0000 7fc93ed50700 1 -- 172.21.15.12:0/3447133887 shutdown_connections 2024-05-08T23:58:35.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:35.768+0000 7fc93ed50700 1 -- 172.21.15.12:0/3447133887 wait complete. 2024-05-08T23:58:35.897 INFO:tasks.workunit.client.0.smithi012.stdout: 1 662222 6.00 MB/sec execute 511 sec latency 2728.416 ms 2024-05-08T23:58:36.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:36 smithi204 ceph-mon[47373]: pgmap v388: 97 pgs: 97 active+clean; 201 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-05-08T23:58:36.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:36 smithi160 ceph-mon[46152]: pgmap v388: 97 pgs: 97 active+clean; 201 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-05-08T23:58:36.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:36 smithi012 ceph-mon[42503]: pgmap v388: 97 pgs: 97 active+clean; 201 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-05-08T23:58:36.777 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:58:36.777 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:58:36.897 INFO:tasks.workunit.client.0.smithi012.stdout: 1 664214 6.01 MB/sec execute 512 sec latency 2867.111 ms 2024-05-08T23:58:36.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.955+0000 7f39c69e1700 1 -- 172.21.15.12:0/3983805974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c0105160 msgr2=0x7f39c010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:36.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.955+0000 7f39c69e1700 1 --2- 172.21.15.12:0/3983805974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c0105160 0x7f39c010cce0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f39b800b0a0 tx=0x7f39b800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.955+0000 7f39c69e1700 1 -- 172.21.15.12:0/3983805974 shutdown_connections 2024-05-08T23:58:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.955+0000 7f39c69e1700 1 --2- 172.21.15.12:0/3983805974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c0105160 0x7f39c010cce0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.955+0000 7f39c69e1700 1 --2- 172.21.15.12:0/3983805974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c0100060 0x7f39c0100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.955+0000 7f39c69e1700 1 --2- 172.21.15.12:0/3983805974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c0100a10 0x7f39c0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.955+0000 7f39c69e1700 1 -- 172.21.15.12:0/3983805974 >> 172.21.15.12:0/3983805974 conn(0x7f39c0076f40 msgr2=0x7f39c0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.955+0000 7f39c69e1700 1 -- 172.21.15.12:0/3983805974 shutdown_connections 2024-05-08T23:58:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.955+0000 7f39c69e1700 1 -- 172.21.15.12:0/3983805974 wait complete. 2024-05-08T23:58:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.956+0000 7f39c69e1700 1 Processor -- start 2024-05-08T23:58:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.956+0000 7f39c69e1700 1 -- start start 2024-05-08T23:58:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.956+0000 7f39c69e1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c0100060 0x7f39c01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.956+0000 7f39c69e1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c0105160 0x7f39c0199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39c69e1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c019df90 0x7f39c019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39c69e1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c019ea10 con 0x7f39c0100060 2024-05-08T23:58:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39c69e1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c019eb80 con 0x7f39c019df90 2024-05-08T23:58:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39c69e1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c019ed40 con 0x7f39c0105160 2024-05-08T23:58:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c0100060 0x7f39c01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c0100060 0x7f39c01996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33688/0 (socket says 172.21.15.12:33688) 2024-05-08T23:58:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bffff700 1 -- 172.21.15.12:0/3177211288 learned_addr learned my addr 172.21.15.12:0/3177211288 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bf7fe700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c0105160 0x7f39c0199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c019df90 0x7f39c019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bf7fe700 1 -- 172.21.15.12:0/3177211288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c019df90 msgr2=0x7f39c019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:36.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bf7fe700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c019df90 0x7f39c019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bf7fe700 1 -- 172.21.15.12:0/3177211288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c0100060 msgr2=0x7f39c01996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bf7fe700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c0100060 0x7f39c01996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bf7fe700 1 -- 172.21.15.12:0/3177211288 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b8009d00 con 0x7f39c0105160 2024-05-08T23:58:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bffff700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c0100060 0x7f39c01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bf7fe700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c0105160 0x7f39c0199bf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e9c0 tx=0x7f39a800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bd7fa700 1 -- 172.21.15.12:0/3177211288 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a800c9c0 con 0x7f39c0105160 2024-05-08T23:58:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bd7fa700 1 -- 172.21.15.12:0/3177211288 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a8010430 con 0x7f39c0105160 2024-05-08T23:58:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39bd7fa700 1 -- 172.21.15.12:0/3177211288 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8018680 con 0x7f39c0105160 2024-05-08T23:58:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.957+0000 7f39c69e1700 1 -- 172.21.15.12:0/3177211288 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c01a62a0 con 0x7f39c0105160 2024-05-08T23:58:36.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.958+0000 7f39c69e1700 1 -- 172.21.15.12:0/3177211288 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c01a6730 con 0x7f39c0105160 2024-05-08T23:58:36.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.960+0000 7f39bd7fa700 1 -- 172.21.15.12:0/3177211288 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f39a800cb20 con 0x7f39c0105160 2024-05-08T23:58:36.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.961+0000 7f39bd7fa700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 0x7f39ac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.961+0000 7f39bd7fa700 1 -- 172.21.15.12:0/3177211288 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f39a8014070 con 0x7f39c0105160 2024-05-08T23:58:36.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.961+0000 7f39bffff700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 0x7f39ac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39bffff700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 0x7f39ac0a3750 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f39b0011550 tx=0x7f39b0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:36.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39d2b9bb80 1 Processor -- start 2024-05-08T23:58:36.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39d2b9bb80 1 -- start start 2024-05-08T23:58:36.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f733410e80 0x55f733411260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f73340b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f73340b910 0x55f73340bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7333621d0 con 0x55f73340b910 2024-05-08T23:58:36.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73335fb90 con 0x55f733410e80 2024-05-08T23:58:36.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73335b480 con 0x55f7334117a0 2024-05-08T23:58:36.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39c4f7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f73340b910 0x55f73340bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39bffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f733410e80 0x55f733411260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39c4f7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f73340b910 0x55f73340bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33704/0 (socket says 172.21.15.12:33704) 2024-05-08T23:58:36.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39c4f7e700 1 -- 172.21.15.12:0/3401964204 learned_addr learned my addr 172.21.15.12:0/3401964204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:36.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.962+0000 7f39bffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f733410e80 0x55f733411260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:59324/0 (socket says 172.21.15.12:59324) 2024-05-08T23:58:36.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39bf7fe700 1 --2- 172.21.15.12:0/3401964204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f73340b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39c4f7e700 1 -- 172.21.15.12:0/3401964204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 msgr2=0x55f73340b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:36.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/3401964204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f73340b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39c4f7e700 1 -- 172.21.15.12:0/3401964204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f733410e80 msgr2=0x55f733411260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:36.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/3401964204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f733410e80 0x55f733411260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39c4f7e700 1 -- 172.21.15.12:0/3401964204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7331dc430 con 0x55f73340b910 2024-05-08T23:58:36.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/3401964204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f73340b910 0x55f73340bd50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f39b8015000 tx=0x7f39b80092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c02b937dda74c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39b67fc700 1 -- 172.21.15.12:0/3401964204 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b800e040 con 0x55f73340b910 2024-05-08T23:58:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39b67fc700 1 -- 172.21.15.12:0/3401964204 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b8006000 con 0x55f73340b910 2024-05-08T23:58:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.963+0000 7f39b67fc700 1 -- 172.21.15.12:0/3401964204 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8007910 con 0x55f73340b910 2024-05-08T23:58:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.964+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/3401964204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f73340b910 msgr2=0x55f73340bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.964+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/3401964204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f73340b910 0x55f73340bd50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f39b8015000 tx=0x7f39b80092e0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.964+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/3401964204 shutdown_connections 2024-05-08T23:58:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.964+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/3401964204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f73340b910 0x55f73340bd50 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.964+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/3401964204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f733410e80 0x55f733411260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.964+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/3401964204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f73340b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.964+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/3401964204 >> 172.21.15.12:0/3401964204 conn(0x55f733369290 msgr2=0x55f733405e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.964+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/3401964204 shutdown_connections 2024-05-08T23:58:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.964+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/3401964204 wait complete. 2024-05-08T23:58:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.964+0000 7f39d2b9bb80 1 -- start start 2024-05-08T23:58:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39d2b9bb80 1 Processor -- start 2024-05-08T23:58:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f733417770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733414ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733415130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7332f75e0 con 0x55f733410e80 2024-05-08T23:58:36.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7332f7280 con 0x55f73340b910 2024-05-08T23:58:36.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7331de720 con 0x55f7334117a0 2024-05-08T23:58:36.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39d2b9bb80 10 client.0 _ll_register_callbacks cb 0x55f733300bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:36.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733414ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733414ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33716/0 (socket says 172.21.15.12:33716) 2024-05-08T23:58:36.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39bf7fe700 1 -- 172.21.15.12:0/1939900422 learned_addr learned my addr 172.21.15.12:0/1939900422 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:36.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733415130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.965+0000 7f39bffff700 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f733417770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f39c4f7e700 1 -- 172.21.15.12:0/1939900422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 msgr2=0x55f733417770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f733417770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f39c4f7e700 1 -- 172.21.15.12:0/1939900422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 msgr2=0x55f733414ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733414ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f39c4f7e700 1 -- 172.21.15.12:0/1939900422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c010d550 con 0x55f7334117a0 2024-05-08T23:58:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f39bf7fe700 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733414ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733415130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009350 tx=0x7f39b800bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2f26017a178763 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f399f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:58:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f399f7fe700 1 -- 172.21.15.12:0/1939900422 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b800e040 con 0x55f7334117a0 2024-05-08T23:58:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f399f7fe700 1 -- 172.21.15.12:0/1939900422 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f39b80076b0 con 0x55f7334117a0 2024-05-08T23:58:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1939900422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7332b42f0 con 0x55f7334117a0 2024-05-08T23:58:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f399f7fe700 1 -- 172.21.15.12:0/1939900422 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39b8007870 con 0x55f7334117a0 2024-05-08T23:58:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f39d2b9bb80 10 client.28630 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1939900422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55f733188be0 con 0x55f7334117a0 2024-05-08T23:58:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.966+0000 7f399f7fe700 1 -- 172.21.15.12:0/1939900422 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7332b42f0 con 0x55f7334117a0 2024-05-08T23:58:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.967+0000 7f399f7fe700 1 -- 172.21.15.12:0/1939900422 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f39b8019070 con 0x55f7334117a0 2024-05-08T23:58:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.967+0000 7f39d2b9bb80 10 client.28630 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.967+0000 7f39d2b9bb80 10 client.28630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.967+0000 7f39d2b9bb80 20 client.28630 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.967+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f733424dd0 0x55f7334271c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.967+0000 7f39d2b9bb80 4 client.28630 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.967+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1939900422 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7333da890 con 0x55f733424dd0 2024-05-08T23:58:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.967+0000 7f39bffff700 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f733424dd0 0x55f7334271c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.967+0000 7f399f7fe700 10 client.28630 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.967+0000 7f39bffff700 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f733424dd0 0x55f7334271c0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71b7e7a5fab4a4e0 server_cookie=c1c3dfb0be1951c3 in_seq=0 out_seq=0 2024-05-08T23:58:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.968+0000 7f399f7fe700 1 -- 172.21.15.12:0/1939900422 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f7333da890 con 0x55f733424dd0 2024-05-08T23:58:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.968+0000 7f399f7fe700 10 client.28630 handle_command_reply: tid=0 2024-05-08T23:58:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.968+0000 7f39d2b9bb80 1 client.28630 shutdown 2024-05-08T23:58:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1939900422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 msgr2=0x55f733415130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733415130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009350 tx=0x7f39b800bc60 comp rx=0 tx=0).stop 2024-05-08T23:58:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1939900422 shutdown_connections 2024-05-08T23:58:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733415130 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f733424dd0 0x55f7334271c0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f733417770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/1939900422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733414ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f399f7fe700 0 client.28630 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1939900422 >> 172.21.15.12:0/1939900422 conn(0x55f733369290 msgr2=0x55f733405620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1939900422 shutdown_connections 2024-05-08T23:58:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1939900422 wait complete. 2024-05-08T23:58:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.969+0000 7f39d2b9bb80 20 client.28630 trim_cache size 0 max 16384 2024-05-08T23:58:36.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.987+0000 7f39d2b9bb80 1 Processor -- start 2024-05-08T23:58:36.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.987+0000 7f39d2b9bb80 1 -- start start 2024-05-08T23:58:36.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f73340b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733415000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733405da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7332b4320 con 0x55f733410e80 2024-05-08T23:58:36.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73340c350 con 0x55f73340b910 2024-05-08T23:58:36.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f733402e90 con 0x55f7334117a0 2024-05-08T23:58:36.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733415000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39c4f7e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733405da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39bffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f73340b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39c4f7e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733405da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57558/0 (socket says 172.21.15.12:57558) 2024-05-08T23:58:36.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39c4f7e700 1 -- 172.21.15.12:0/1186620363 learned_addr learned my addr 172.21.15.12:0/1186620363 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:36.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39c4f7e700 1 -- 172.21.15.12:0/1186620363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 msgr2=0x55f73340b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:36.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/1186620363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f73340b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39c4f7e700 1 -- 172.21.15.12:0/1186620363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 msgr2=0x55f733415000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:36.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/1186620363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733415000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39c4f7e700 1 -- 172.21.15.12:0/1186620363 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7333215e0 con 0x55f7334117a0 2024-05-08T23:58:36.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39bf7fe700 1 --2- 172.21.15.12:0/1186620363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733415000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:36.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.988+0000 7f39bffff700 1 --2- 172.21.15.12:0/1186620363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f73340b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:36.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39c4f7e700 1 --2- 172.21.15.12:0/1186620363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733405da0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f39b8025c80 tx=0x7f39b8025e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1788bd1b464c163e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:36.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39b6ffd700 1 -- 172.21.15.12:0/1186620363 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b800e040 con 0x55f7334117a0 2024-05-08T23:58:36.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39b6ffd700 1 -- 172.21.15.12:0/1186620363 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b800bcd0 con 0x55f7334117a0 2024-05-08T23:58:36.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39b6ffd700 1 -- 172.21.15.12:0/1186620363 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80259c0 con 0x55f7334117a0 2024-05-08T23:58:36.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1186620363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 msgr2=0x55f733405da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:36.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/1186620363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733405da0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f39b8025c80 tx=0x7f39b8025e20 comp rx=0 tx=0).stop 2024-05-08T23:58:36.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1186620363 shutdown_connections 2024-05-08T23:58:36.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/1186620363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7334117a0 0x55f733405da0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/1186620363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f73340b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/1186620363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f733410e80 0x55f733415000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.989+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1186620363 >> 172.21.15.12:0/1186620363 conn(0x55f733369290 msgr2=0x55f733413380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:36.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.990+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1186620363 shutdown_connections 2024-05-08T23:58:36.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.990+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/1186620363 wait complete. 2024-05-08T23:58:36.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.990+0000 7f39d2b9bb80 1 -- start start 2024-05-08T23:58:36.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.990+0000 7f39d2b9bb80 1 Processor -- start 2024-05-08T23:58:36.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.990+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f73341c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.990+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f733410e80 0x55f73341c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.990+0000 7f39d2b9bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7334117a0 0x55f7334186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.990+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f733418d80 con 0x55f7334117a0 2024-05-08T23:58:36.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.990+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f733418ef0 con 0x55f73340b910 2024-05-08T23:58:36.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.990+0000 7f39d2b9bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7334190b0 con 0x55f733410e80 2024-05-08T23:58:36.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39d2b9bb80 10 client.0 _ll_register_callbacks cb 0x55f733300bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:36.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39bf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f733410e80 0x55f73341c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39c4f7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7334117a0 0x55f7334186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39bf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f733410e80 0x55f73341c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57574/0 (socket says 172.21.15.12:57574) 2024-05-08T23:58:36.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39bf7fe700 1 -- 172.21.15.12:0/374027984 learned_addr learned my addr 172.21.15.12:0/374027984 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:36.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39c4f7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7334117a0 0x55f7334186b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33734/0 (socket says 172.21.15.12:33734) 2024-05-08T23:58:36.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39bffff700 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f73341c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:36.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39bf7fe700 1 -- 172.21.15.12:0/374027984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 msgr2=0x55f73341c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:36.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39bf7fe700 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f73341c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39bf7fe700 1 -- 172.21.15.12:0/374027984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7334117a0 msgr2=0x55f7334186b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:36.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39bf7fe700 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7334117a0 0x55f7334186b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:36.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39bf7fe700 1 -- 172.21.15.12:0/374027984 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b0017070 con 0x55f733410e80 2024-05-08T23:58:36.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39bf7fe700 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f733410e80 0x55f73341c5e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f39a8099ba0 tx=0x7f39a809a3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1f4f5d06abe328 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:36.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f399f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:58:36.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f399f7fe700 1 -- 172.21.15.12:0/374027984 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8099ea0 con 0x55f733410e80 2024-05-08T23:58:36.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f399f7fe700 1 -- 172.21.15.12:0/374027984 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f39a809dcd0 con 0x55f733410e80 2024-05-08T23:58:36.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/374027984 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39b8006000 con 0x55f733410e80 2024-05-08T23:58:36.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f399f7fe700 1 -- 172.21.15.12:0/374027984 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39a80a2320 con 0x55f733410e80 2024-05-08T23:58:36.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39d2b9bb80 10 client.28636 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:36.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.991+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/374027984 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f39c01a6730 con 0x55f733410e80 2024-05-08T23:58:36.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f399f7fe700 1 -- 172.21.15.12:0/374027984 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b8006000 con 0x55f733410e80 2024-05-08T23:58:36.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f399f7fe700 1 -- 172.21.15.12:0/374027984 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f39a80a1f20 con 0x55f733410e80 2024-05-08T23:58:36.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f39d2b9bb80 10 client.28636 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:36.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f39d2b9bb80 10 client.28636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:36.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f39d2b9bb80 20 client.28636 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:36.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f73347c5e0 0x55f73347ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:36.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f39d2b9bb80 4 client.28636 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:58:36.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/374027984 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f733353f40 con 0x55f73347c5e0 2024-05-08T23:58:37.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f39bffff700 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f73347c5e0 0x55f73347ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f399f7fe700 10 client.28636 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:37.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.992+0000 7f39bffff700 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f73347c5e0 0x55f73347ea00 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570663896cc9821c server_cookie=995b2a45fafe7315 in_seq=0 out_seq=0 2024-05-08T23:58:37.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f399f7fe700 1 -- 172.21.15.12:0/374027984 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f733353f40 con 0x55f73347c5e0 2024-05-08T23:58:37.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f399f7fe700 10 client.28636 handle_command_reply: tid=0 2024-05-08T23:58:37.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f39d2b9bb80 1 client.28636 shutdown 2024-05-08T23:58:37.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/374027984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f733410e80 msgr2=0x55f73341c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f733410e80 0x55f73341c5e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f39a8099ba0 tx=0x7f39a809a3e0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/374027984 shutdown_connections 2024-05-08T23:58:37.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7334117a0 0x55f7334186b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f73347c5e0 0x55f73347ea00 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f73340b910 0x55f73341c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f39d2b9bb80 1 --2- 172.21.15.12:0/374027984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f733410e80 0x55f73341c5e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f399f7fe700 0 client.28636 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:37.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/374027984 >> 172.21.15.12:0/374027984 conn(0x55f733369290 msgr2=0x55f733412410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.994+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/374027984 shutdown_connections 2024-05-08T23:58:37.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39d2b9bb80 1 -- 172.21.15.12:0/374027984 wait complete. 2024-05-08T23:58:37.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39d2b9bb80 20 client.28636 trim_cache size 0 max 16384 2024-05-08T23:58:37.002 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:58:37.003 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:58:37.003 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "62de4a63-0b8e-4d35-92a9-62c00802177c", 2024-05-08T23:58:37.003 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:58:37.003 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:58:37.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 -- 172.21.15.12:0/3177211288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 msgr2=0x7f39ac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 0x7f39ac0a3750 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f39b0011550 tx=0x7f39b0009260 comp rx=0 tx=0).stop 2024-05-08T23:58:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 -- 172.21.15.12:0/3177211288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c0105160 msgr2=0x7f39c0199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c0105160 0x7f39c0199bf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e9c0 tx=0x7f39a800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 -- 172.21.15.12:0/3177211288 shutdown_connections 2024-05-08T23:58:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39ac0a1290 0x7f39ac0a3750 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39c019df90 0x7f39c019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39c0100060 0x7f39c01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 --2- 172.21.15.12:0/3177211288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c0105160 0x7f39c0199bf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 -- 172.21.15.12:0/3177211288 >> 172.21.15.12:0/3177211288 conn(0x7f39c0076f40 msgr2=0x7f39c00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 -- 172.21.15.12:0/3177211288 shutdown_connections 2024-05-08T23:58:37.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:36.995+0000 7f39c69e1700 1 -- 172.21.15.12:0/3177211288 wait complete. 2024-05-08T23:58:37.007 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:58:37.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.162+0000 7f97fda61700 1 -- 172.21.15.12:0/290163926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97f8105ed0 msgr2=0x7f97f81062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.162+0000 7f97fda61700 1 --2- 172.21.15.12:0/290163926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97f8105ed0 0x7f97f81062b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009990 tx=0x7f97e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.162+0000 7f97fda61700 1 -- 172.21.15.12:0/290163926 shutdown_connections 2024-05-08T23:58:37.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.162+0000 7f97fda61700 1 --2- 172.21.15.12:0/290163926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97f80689f0 0x7f97f810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.162+0000 7f97fda61700 1 --2- 172.21.15.12:0/290163926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97f8105ed0 0x7f97f81062b0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.162+0000 7f97fda61700 1 --2- 172.21.15.12:0/290163926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f8068030 0x7f97f80684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.162+0000 7f97fda61700 1 -- 172.21.15.12:0/290163926 >> 172.21.15.12:0/290163926 conn(0x7f97f8076f40 msgr2=0x7f97f8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.162+0000 7f97fda61700 1 -- 172.21.15.12:0/290163926 shutdown_connections 2024-05-08T23:58:37.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.162+0000 7f97fda61700 1 -- 172.21.15.12:0/290163926 wait complete. 2024-05-08T23:58:37.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.163+0000 7f97fda61700 1 Processor -- start 2024-05-08T23:58:37.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.163+0000 7f97fda61700 1 -- start start 2024-05-08T23:58:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97fda61700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97f8068030 0x7f97f80ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97fda61700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97f80689f0 0x7f97f81003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97fda61700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f8105ed0 0x7f97f81008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97fda61700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81a57c0 con 0x7f97f80689f0 2024-05-08T23:58:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97fda61700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81a5930 con 0x7f97f8068030 2024-05-08T23:58:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97fda61700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81a5af0 con 0x7f97f8105ed0 2024-05-08T23:58:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97f80689f0 0x7f97f81003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97f8068030 0x7f97f80ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97f8068030 0x7f97f80ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:59364/0 (socket says 172.21.15.12:59364) 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f6ffd700 1 -- 172.21.15.12:0/2707920728 learned_addr learned my addr 172.21.15.12:0/2707920728 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f67fc700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97f80689f0 0x7f97f81003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33742/0 (socket says 172.21.15.12:33742) 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f77fe700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f8105ed0 0x7f97f81008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f6ffd700 1 -- 172.21.15.12:0/2707920728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f8105ed0 msgr2=0x7f97f81008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f8105ed0 0x7f97f81008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f6ffd700 1 -- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97f80689f0 msgr2=0x7f97f81003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97f80689f0 0x7f97f81003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f6ffd700 1 -- 172.21.15.12:0/2707920728 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e80095f0 con 0x7f97f8068030 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f77fe700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f8105ed0 0x7f97f81008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f67fc700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97f80689f0 0x7f97f81003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.164+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97f8068030 0x7f97f80ffe60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f97e8000c00 tx=0x7f97e8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.165+0000 7f97fca5f700 1 -- 172.21.15.12:0/2707920728 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e801d070 con 0x7f97f8068030 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.165+0000 7f97fda61700 1 -- 172.21.15.12:0/2707920728 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f81a5d70 con 0x7f97f8068030 2024-05-08T23:58:37.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.165+0000 7f97fca5f700 1 -- 172.21.15.12:0/2707920728 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97e8022470 con 0x7f97f8068030 2024-05-08T23:58:37.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.165+0000 7f97fca5f700 1 -- 172.21.15.12:0/2707920728 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e8017650 con 0x7f97f8068030 2024-05-08T23:58:37.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.165+0000 7f97fda61700 1 -- 172.21.15.12:0/2707920728 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f81a61a0 con 0x7f97f8068030 2024-05-08T23:58:37.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.167+0000 7f97fca5f700 1 -- 172.21.15.12:0/2707920728 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f97e80225e0 con 0x7f97f8068030 2024-05-08T23:58:37.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.168+0000 7f97fca5f700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97e40a1290 0x7f97e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.168+0000 7f97fca5f700 1 -- 172.21.15.12:0/2707920728 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f97e80cded0 con 0x7f97f8068030 2024-05-08T23:58:37.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.168+0000 7f97f67fc700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97e40a1290 0x7f97e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.168+0000 7f97f67fc700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97e40a1290 0x7f97e40a3750 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97f8101590 tx=0x7f97e0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f9809c1bb80 1 Processor -- start 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f9809c1bb80 1 -- start start 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b7e00 0x56124e5b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b8720 0x56124e5b2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b2890 0x56124e5b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e509150 con 0x56124e5b7e00 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e506b10 con 0x56124e5b8720 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e502400 con 0x56124e5b2890 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f97f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b7e00 0x56124e5b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f97f67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b8720 0x56124e5b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f97f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b7e00 0x56124e5b81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33746/0 (socket says 172.21.15.12:33746) 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f97f6ffd700 1 -- 172.21.15.12:0/1815631289 learned_addr learned my addr 172.21.15.12:0/1815631289 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.169+0000 7f97f77fe700 1 --2- 172.21.15.12:0/1815631289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b2890 0x56124e5b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.170+0000 7f97f6ffd700 1 -- 172.21.15.12:0/1815631289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b2890 msgr2=0x56124e5b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.170+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/1815631289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b2890 0x56124e5b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.170+0000 7f97f6ffd700 1 -- 172.21.15.12:0/1815631289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b8720 msgr2=0x56124e5b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.170+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/1815631289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b8720 0x56124e5b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.170+0000 7f97f6ffd700 1 -- 172.21.15.12:0/1815631289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56124e3833b0 con 0x56124e5b7e00 2024-05-08T23:58:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.170+0000 7f97f67fc700 1 --2- 172.21.15.12:0/1815631289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b8720 0x56124e5b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.170+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/1815631289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b7e00 0x56124e5b81e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f97e80975b0 tx=0x7f97e809af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce4024a67172519 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.170+0000 7f97edffb700 1 -- 172.21.15.12:0/1815631289 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e809b0b0 con 0x56124e5b7e00 2024-05-08T23:58:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.170+0000 7f97edffb700 1 -- 172.21.15.12:0/1815631289 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97e809eee0 con 0x56124e5b7e00 2024-05-08T23:58:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.170+0000 7f97edffb700 1 -- 172.21.15.12:0/1815631289 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e800fac0 con 0x56124e5b7e00 2024-05-08T23:58:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.171+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1815631289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b7e00 msgr2=0x56124e5b81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.171+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/1815631289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b7e00 0x56124e5b81e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f97e80975b0 tx=0x7f97e809af90 comp rx=0 tx=0).stop 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.171+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1815631289 shutdown_connections 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.171+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/1815631289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b2890 0x56124e5b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.171+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/1815631289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b7e00 0x56124e5b81e0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.171+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/1815631289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b8720 0x56124e5b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.171+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1815631289 >> 172.21.15.12:0/1815631289 conn(0x56124e510210 msgr2=0x56124e5acdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.171+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1815631289 shutdown_connections 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.171+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1815631289 wait complete. 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.171+0000 7f9809c1bb80 1 -- start start 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f9809c1bb80 1 Processor -- start 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b2890 0x56124e5be750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b7e00 0x56124e5bb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b8720 0x56124e5bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e49e560 con 0x56124e5b2890 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e49e200 con 0x56124e5b7e00 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e3856a0 con 0x56124e5b8720 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f9809c1bb80 10 client.0 _ll_register_callbacks cb 0x56124e4a7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b2890 0x56124e5be750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b7e00 0x56124e5bb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b7e00 0x56124e5bb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:59388/0 (socket says 172.21.15.12:59388) 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f67fc700 1 -- 172.21.15.12:0/1146227938 learned_addr learned my addr 172.21.15.12:0/1146227938 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b2890 0x56124e5be750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33760/0 (socket says 172.21.15.12:33760) 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f77fe700 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b8720 0x56124e5bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f67fc700 1 -- 172.21.15.12:0/1146227938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b8720 msgr2=0x56124e5bc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f67fc700 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b8720 0x56124e5bc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f67fc700 1 -- 172.21.15.12:0/1146227938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b2890 msgr2=0x56124e5be750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f67fc700 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b2890 0x56124e5be750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f67fc700 1 -- 172.21.15.12:0/1146227938 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e0017070 con 0x56124e5b7e00 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b2890 0x56124e5be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.172+0000 7f97f77fe700 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b8720 0x56124e5bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.173+0000 7f97f67fc700 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b7e00 0x56124e5bb9d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f97e0003a00 tx=0x7f97e00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41aeceb9f051f817 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.173+0000 7f97d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.173+0000 7f97d6ffd700 1 -- 172.21.15.12:0/1146227938 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e001fc20 con 0x56124e5b7e00 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.173+0000 7f97d6ffd700 1 -- 172.21.15.12:0/1146227938 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f97e001fd80 con 0x56124e5b7e00 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.173+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1146227938 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56124e45b270 con 0x56124e5b7e00 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.173+0000 7f97d6ffd700 1 -- 172.21.15.12:0/1146227938 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97e0003e80 con 0x56124e5b7e00 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.173+0000 7f9809c1bb80 10 client.28646 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.173+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1146227938 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f97f81a61a0 con 0x56124e5b7e00 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.173+0000 7f97d6ffd700 1 -- 172.21.15.12:0/1146227938 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56124e45b270 con 0x56124e5b7e00 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.174+0000 7f97d6ffd700 1 -- 172.21.15.12:0/1146227938 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f97e0027500 con 0x56124e5b7e00 2024-05-08T23:58:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.174+0000 7f9809c1bb80 10 client.28646 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.174+0000 7f9809c1bb80 10 client.28646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.174+0000 7f9809c1bb80 20 client.28646 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.174+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56124e5cbe70 0x56124e5ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.174+0000 7f9809c1bb80 4 client.28646 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.174+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1146227938 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56124e581810 con 0x56124e5cbe70 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.174+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56124e5cbe70 0x56124e5ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.174+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56124e5cbe70 0x56124e5ce290 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb355c5bebb293d0 server_cookie=3d1b2970900f8fa5 in_seq=0 out_seq=0 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.174+0000 7f97d6ffd700 10 client.28646 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f97d6ffd700 1 -- 172.21.15.12:0/1146227938 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56124e581810 con 0x56124e5cbe70 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f97d6ffd700 10 client.28646 handle_command_reply: tid=0 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 client.28646 shutdown 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1146227938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b7e00 msgr2=0x56124e5bb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b7e00 0x56124e5bb9d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f97e0003a00 tx=0x7f97e00047c0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1146227938 shutdown_connections 2024-05-08T23:58:37.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b8720 0x56124e5bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b2890 0x56124e5be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56124e5cbe70 0x56124e5ce290 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/1146227938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b7e00 0x56124e5bb9d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f97d6ffd700 0 client.28646 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:37.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1146227938 >> 172.21.15.12:0/1146227938 conn(0x56124e510210 msgr2=0x56124e5ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1146227938 shutdown_connections 2024-05-08T23:58:37.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.175+0000 7f9809c1bb80 1 -- 172.21.15.12:0/1146227938 wait complete. 2024-05-08T23:58:37.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.176+0000 7f9809c1bb80 20 client.28646 trim_cache size 0 max 16384 2024-05-08T23:58:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.195+0000 7f9809c1bb80 1 Processor -- start 2024-05-08T23:58:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.195+0000 7f9809c1bb80 1 -- start start 2024-05-08T23:58:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 0x56124e5bb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 0x56124e5acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5be280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e5a9dc0 con 0x56124e5b8720 2024-05-08T23:58:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e5a7800 con 0x56124e5b2890 2024-05-08T23:58:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e5d34a0 con 0x56124e5b7e00 2024-05-08T23:58:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5be280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 0x56124e5bb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5be280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33766/0 (socket says 172.21.15.12:33766) 2024-05-08T23:58:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f77fe700 1 -- 172.21.15.12:0/3534399993 learned_addr learned my addr 172.21.15.12:0/3534399993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f67fc700 1 --2- 172.21.15.12:0/3534399993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 0x56124e5acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f67fc700 1 -- 172.21.15.12:0/3534399993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 msgr2=0x56124e5bb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f67fc700 1 --2- 172.21.15.12:0/3534399993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 0x56124e5bb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f67fc700 1 -- 172.21.15.12:0/3534399993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 msgr2=0x56124e5be280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f67fc700 1 --2- 172.21.15.12:0/3534399993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5be280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f67fc700 1 -- 172.21.15.12:0/3534399993 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56124e5b2f40 con 0x56124e5b7e00 2024-05-08T23:58:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f77fe700 1 --2- 172.21.15.12:0/3534399993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5be280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.196+0000 7f97f67fc700 1 --2- 172.21.15.12:0/3534399993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 0x56124e5acd20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f97e00039d0 tx=0x7f97e0004100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5563ef2852463965 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f97ee7fc700 1 -- 172.21.15.12:0/3534399993 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0003e80 con 0x56124e5b7e00 2024-05-08T23:58:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f97ee7fc700 1 -- 172.21.15.12:0/3534399993 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97e00273e0 con 0x56124e5b7e00 2024-05-08T23:58:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f97ee7fc700 1 -- 172.21.15.12:0/3534399993 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0031440 con 0x56124e5b7e00 2024-05-08T23:58:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3534399993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 msgr2=0x56124e5acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/3534399993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 0x56124e5acd20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f97e00039d0 tx=0x7f97e0004100 comp rx=0 tx=0).stop 2024-05-08T23:58:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3534399993 shutdown_connections 2024-05-08T23:58:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/3534399993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5be280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/3534399993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 0x56124e5bb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/3534399993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 0x56124e5acd20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3534399993 >> 172.21.15.12:0/3534399993 conn(0x56124e510210 msgr2=0x56124e5bb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3534399993 shutdown_connections 2024-05-08T23:58:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.197+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3534399993 wait complete. 2024-05-08T23:58:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.198+0000 7f9809c1bb80 1 -- start start 2024-05-08T23:58:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.198+0000 7f9809c1bb80 1 Processor -- start 2024-05-08T23:58:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 0x56124e5c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 0x56124e5c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f9809c1bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e5b4350 con 0x56124e5b8720 2024-05-08T23:58:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e5b44c0 con 0x56124e5b2890 2024-05-08T23:58:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f9809c1bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124e5b4680 con 0x56124e5b7e00 2024-05-08T23:58:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f9809c1bb80 10 client.0 _ll_register_callbacks cb 0x56124e4a7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 0x56124e5c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 0x56124e5c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:59406/0 (socket says 172.21.15.12:59406) 2024-05-08T23:58:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f6ffd700 1 -- 172.21.15.12:0/3075294535 learned_addr learned my addr 172.21.15.12:0/3075294535 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f67fc700 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 0x56124e5c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f6ffd700 1 -- 172.21.15.12:0/3075294535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 msgr2=0x56124e5c33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 0x56124e5c33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f6ffd700 1 -- 172.21.15.12:0/3075294535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 msgr2=0x56124e5b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f6ffd700 1 -- 172.21.15.12:0/3075294535 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f81a61a0 con 0x56124e5b2890 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f77fe700 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97f6ffd700 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 0x56124e5c2e80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f97e80a2f70 tx=0x7f97e8005d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3938539cdd124060 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.199+0000 7f97d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f97d6ffd700 1 -- 172.21.15.12:0/3075294535 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e809b130 con 0x56124e5b2890 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f97d6ffd700 1 -- 172.21.15.12:0/3075294535 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f97e807b070 con 0x56124e5b2890 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3075294535 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97e809eee0 con 0x56124e5b2890 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f97d6ffd700 1 -- 172.21.15.12:0/3075294535 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97e80a10c0 con 0x56124e5b2890 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f9809c1bb80 10 client.28652 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3075294535 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x56124e5a94e0 con 0x56124e5b2890 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f97d6ffd700 1 -- 172.21.15.12:0/3075294535 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97e80a7020 con 0x56124e5b2890 2024-05-08T23:58:37.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f97d6ffd700 1 -- 172.21.15.12:0/3075294535 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f97e800fbc0 con 0x56124e5b2890 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f9809c1bb80 10 client.28652 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f9809c1bb80 10 client.28652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f9809c1bb80 20 client.28652 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56124e6231a0 0x56124e6255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f9809c1bb80 4 client.28652 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3075294535 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56124e4faec0 con 0x56124e6231a0 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.200+0000 7f97f67fc700 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56124e6231a0 0x56124e6255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.201+0000 7f97d6ffd700 10 client.28652 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.201+0000 7f97f67fc700 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56124e6231a0 0x56124e6255c0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=236e792745eea1ea server_cookie=9cbb3e911f2880d in_seq=0 out_seq=0 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.201+0000 7f97d6ffd700 1 -- 172.21.15.12:0/3075294535 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56124e4faec0 con 0x56124e6231a0 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.201+0000 7f97d6ffd700 10 client.28652 handle_command_reply: tid=0 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.201+0000 7f9809c1bb80 1 client.28652 shutdown 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.201+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3075294535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 msgr2=0x56124e5c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.201+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 0x56124e5c2e80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f97e80a2f70 tx=0x7f97e8005d40 comp rx=0 tx=0).stop 2024-05-08T23:58:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3075294535 shutdown_connections 2024-05-08T23:58:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56124e5b8720 0x56124e5b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56124e6231a0 0x56124e6255c0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56124e5b2890 0x56124e5c2e80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f9809c1bb80 1 --2- 172.21.15.12:0/3075294535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124e5b7e00 0x56124e5c33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f97d6ffd700 0 client.28652 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3075294535 >> 172.21.15.12:0/3075294535 conn(0x56124e510210 msgr2=0x56124e5b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3075294535 shutdown_connections 2024-05-08T23:58:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f9809c1bb80 1 -- 172.21.15.12:0/3075294535 wait complete. 2024-05-08T23:58:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f9809c1bb80 20 client.28652 trim_cache size 0 max 16384 2024-05-08T23:58:37.212 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:58:37.212 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:58:37.212 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:58:37.212 INFO:teuthology.orchestra.run.smithi012.stdout: "62de4a63-0b8e-4d35-92a9-62c00802177c": { 2024-05-08T23:58:37.213 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:58:37.213 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "62de4a63-0b8e-4d35-92a9-62c00802177c", 2024-05-08T23:58:37.213 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:58:37.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:58:37.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:58:37.213 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:58:37.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f97fda61700 1 -- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97e40a1290 msgr2=0x7f97e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f97fda61700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97e40a1290 0x7f97e40a3750 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97f8101590 tx=0x7f97e0009260 comp rx=0 tx=0).stop 2024-05-08T23:58:37.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f97fda61700 1 -- 172.21.15.12:0/2707920728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97f8068030 msgr2=0x7f97f80ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f97fda61700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97f8068030 0x7f97f80ffe60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f97e8000c00 tx=0x7f97e8004730 comp rx=0 tx=0).stop 2024-05-08T23:58:37.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f97fda61700 1 -- 172.21.15.12:0/2707920728 shutdown_connections 2024-05-08T23:58:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.202+0000 7f97fda61700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97e40a1290 0x7f97e40a3750 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.203+0000 7f97fda61700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f8105ed0 0x7f97f81008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.203+0000 7f97fda61700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97f8068030 0x7f97f80ffe60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.203+0000 7f97fda61700 1 --2- 172.21.15.12:0/2707920728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97f80689f0 0x7f97f81003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.203+0000 7f97fda61700 1 -- 172.21.15.12:0/2707920728 >> 172.21.15.12:0/2707920728 conn(0x7f97f8076f40 msgr2=0x7f97f80fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.203+0000 7f97fda61700 1 -- 172.21.15.12:0/2707920728 shutdown_connections 2024-05-08T23:58:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:58:37.203+0000 7f97fda61700 1 -- 172.21.15.12:0/2707920728 wait complete. 2024-05-08T23:58:37.216 INFO:tasks.cephfs.filesystem:scrub status for tag:62de4a63-0b8e-4d35-92a9-62c00802177c - {'path': '/', 'tag': '62de4a63-0b8e-4d35-92a9-62c00802177c', 'options': 'recursive,force'} 2024-05-08T23:58:37.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:37 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:58:37.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:37 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:37.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:37 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:58:37.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:37 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.579+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/662499470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a58101aa0 msgr2=0x7f7a58105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.579+0000 7f7a5cc0d700 1 --2- 172.21.15.12:0/662499470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a58101aa0 0x7f7a58105ae0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009990 tx=0x7f7a4c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.579+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/662499470 shutdown_connections 2024-05-08T23:58:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.579+0000 7f7a5cc0d700 1 --2- 172.21.15.12:0/662499470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a58106210 0x7f7a5810dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.579+0000 7f7a5cc0d700 1 --2- 172.21.15.12:0/662499470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a581010f0 0x7f7a581014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.579+0000 7f7a5cc0d700 1 --2- 172.21.15.12:0/662499470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a58101aa0 0x7f7a58105ae0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.579+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/662499470 >> 172.21.15.12:0/662499470 conn(0x7f7a580fc970 msgr2=0x7f7a580fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.579+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/662499470 shutdown_connections 2024-05-08T23:58:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.579+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/662499470 wait complete. 2024-05-08T23:58:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.580+0000 7f7a5cc0d700 1 Processor -- start 2024-05-08T23:58:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.580+0000 7f7a5cc0d700 1 -- start start 2024-05-08T23:58:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a5cc0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a581010f0 0x7f7a58197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a5cc0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a58106210 0x7f7a58197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a5cc0d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a5819bcf0 0x7f7a5819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a5cc0d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5819c770 con 0x7f7a581010f0 2024-05-08T23:58:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a5cc0d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5819c8e0 con 0x7f7a58106210 2024-05-08T23:58:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a5cc0d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5819caa0 con 0x7f7a5819bcf0 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a581010f0 0x7f7a58197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a581010f0 0x7f7a58197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33800/0 (socket says 172.21.15.12:33800) 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a55d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a58106210 0x7f7a58197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a5659c700 1 -- 172.21.15.12:0/1903012054 learned_addr learned my addr 172.21.15.12:0/1903012054 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a5819bcf0 0x7f7a5819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a55d9b700 1 -- 172.21.15.12:0/1903012054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a5819bcf0 msgr2=0x7f7a5819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a55d9b700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a5819bcf0 0x7f7a5819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a55d9b700 1 -- 172.21.15.12:0/1903012054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a581010f0 msgr2=0x7f7a58197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a55d9b700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a581010f0 0x7f7a58197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a55d9b700 1 -- 172.21.15.12:0/1903012054 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48009520 con 0x7f7a58106210 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a5819bcf0 0x7f7a5819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a5659c700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a581010f0 0x7f7a58197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.581+0000 7f7a55d9b700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a58106210 0x7f7a58197950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0098e0 tx=0x7f7a4c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.582+0000 7f7a477fe700 1 -- 172.21.15.12:0/1903012054 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c01d070 con 0x7f7a58106210 2024-05-08T23:58:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.582+0000 7f7a477fe700 1 -- 172.21.15.12:0/1903012054 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a4c022470 con 0x7f7a58106210 2024-05-08T23:58:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.582+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/1903012054 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c0095f0 con 0x7f7a58106210 2024-05-08T23:58:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.582+0000 7f7a477fe700 1 -- 172.21.15.12:0/1903012054 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c017650 con 0x7f7a58106210 2024-05-08T23:58:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.582+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/1903012054 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a581a4230 con 0x7f7a58106210 2024-05-08T23:58:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.584+0000 7f7a477fe700 1 -- 172.21.15.12:0/1903012054 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7a4c022850 con 0x7f7a58106210 2024-05-08T23:58:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.585+0000 7f7a477fe700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a400a1310 0x7f7a400a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.585+0000 7f7a5659c700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a400a1310 0x7f7a400a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.585+0000 7f7a477fe700 1 -- 172.21.15.12:0/1903012054 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7a4c0cdf60 con 0x7f7a58106210 2024-05-08T23:58:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.586+0000 7f7a5659c700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a400a1310 0x7f7a400a37d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009fd0 tx=0x7f7a48009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a68dc7b80 1 Processor -- start 2024-05-08T23:58:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a68dc7b80 1 -- start start 2024-05-08T23:58:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858e60 0x5573f285c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 0x5573f285d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f27ab0f0 con 0x5573f2858e60 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f27a8ab0 con 0x5573f285ce40 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f27a43a0 con 0x5573f2858540 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a56d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 0x5573f285d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57664/0 (socket says 172.21.15.12:57664) 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a56d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 0x5573f285d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:59446/0 (socket says 172.21.15.12:59446) 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.587+0000 7f7a5659c700 1 -- 172.21.15.12:0/380257184 learned_addr learned my addr 172.21.15.12:0/380257184 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.588+0000 7f7a55d9b700 1 --2- 172.21.15.12:0/380257184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858e60 0x5573f285c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.588+0000 7f7a56d9d700 1 -- 172.21.15.12:0/380257184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858540 msgr2=0x5573f2858920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.588+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/380257184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.588+0000 7f7a56d9d700 1 -- 172.21.15.12:0/380257184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858e60 msgr2=0x5573f285c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.588+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/380257184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858e60 0x5573f285c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.588+0000 7f7a56d9d700 1 -- 172.21.15.12:0/380257184 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573f2625350 con 0x5573f285ce40 2024-05-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.588+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/380257184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 0x5573f285d2a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009d10 tx=0x7f7a500077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284b8bf74e768707 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.588+0000 7f7a44ff9700 1 -- 172.21.15.12:0/380257184 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50014610 con 0x5573f285ce40 2024-05-08T23:58:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.588+0000 7f7a44ff9700 1 -- 172.21.15.12:0/380257184 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a5000f050 con 0x5573f285ce40 2024-05-08T23:58:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.588+0000 7f7a44ff9700 1 -- 172.21.15.12:0/380257184 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50013930 con 0x5573f285ce40 2024-05-08T23:58:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.589+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/380257184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 msgr2=0x5573f285d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.589+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/380257184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 0x5573f285d2a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009d10 tx=0x7f7a500077e0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.589+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/380257184 shutdown_connections 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.589+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/380257184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 0x5573f285d2a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.589+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/380257184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.589+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/380257184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858e60 0x5573f285c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.589+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/380257184 >> 172.21.15.12:0/380257184 conn(0x5573f27b21b0 msgr2=0x5573f284e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.590+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/380257184 shutdown_connections 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.590+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/380257184 wait complete. 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.590+0000 7f7a68dc7b80 1 -- start start 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.590+0000 7f7a68dc7b80 1 Processor -- start 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2864610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 0x5573f28618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f2862020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f2740500 con 0x5573f2858540 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f27401a0 con 0x5573f2858e60 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f2627640 con 0x5573f285ce40 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a68dc7b80 10 client.0 _ll_register_callbacks cb 0x5573f2749ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2864610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a56d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f2862020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a55d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 0x5573f28618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a56d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f2862020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57676/0 (socket says 172.21.15.12:57676) 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2864610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33828/0 (socket says 172.21.15.12:33828) 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a56d9d700 1 -- 172.21.15.12:0/976721784 learned_addr learned my addr 172.21.15.12:0/976721784 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a56d9d700 1 -- 172.21.15.12:0/976721784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 msgr2=0x5573f28618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 0x5573f28618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a56d9d700 1 -- 172.21.15.12:0/976721784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 msgr2=0x5573f2864610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2864610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a56d9d700 1 -- 172.21.15.12:0/976721784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a50009940 con 0x5573f285ce40 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a55d9b700 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 0x5573f28618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f2862020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7a50013550 tx=0x7f7a50004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e572d7d797a96849 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.591+0000 7f7a3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a3dffb700 1 -- 172.21.15.12:0/976721784 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50013d70 con 0x5573f285ce40 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a3dffb700 1 -- 172.21.15.12:0/976721784 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7a50003a00 con 0x5573f285ce40 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/976721784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573f26fd210 con 0x5573f285ce40 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a3dffb700 1 -- 172.21.15.12:0/976721784 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a50009e60 con 0x5573f285ce40 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a68dc7b80 10 client.28666 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/976721784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5573f25d1b00 con 0x5573f285ce40 2024-05-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a3dffb700 1 -- 172.21.15.12:0/976721784 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573f26fd210 con 0x5573f285ce40 2024-05-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a3dffb700 1 -- 172.21.15.12:0/976721784 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7a50014410 con 0x5573f285ce40 2024-05-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a68dc7b80 10 client.28666 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a68dc7b80 10 client.28666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a68dc7b80 20 client.28666 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573f2871f10 0x5573f2874330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a68dc7b80 4 client.28666 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/976721784 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573f28237b0 con 0x5573f2871f10 2024-05-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.592+0000 7f7a5659c700 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573f2871f10 0x5573f2874330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.593+0000 7f7a5659c700 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573f2871f10 0x5573f2874330 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70c551289192eb20 server_cookie=9dddb84d17695c61 in_seq=0 out_seq=0 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.593+0000 7f7a3dffb700 10 client.28666 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a3dffb700 1 -- 172.21.15.12:0/976721784 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573f28237b0 con 0x5573f2871f10 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a3dffb700 10 client.28666 handle_command_reply: tid=0 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 client.28666 shutdown 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/976721784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 msgr2=0x5573f2862020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f2862020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7a50013550 tx=0x7f7a50004f90 comp rx=0 tx=0).stop 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/976721784 shutdown_connections 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f2862020 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2864610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573f2871f10 0x5573f2874330 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/976721784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 0x5573f28618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/976721784 >> 172.21.15.12:0/976721784 conn(0x5573f27b21b0 msgr2=0x5573f2852f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a3dffb700 0 client.28666 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/976721784 shutdown_connections 2024-05-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/976721784 wait complete. 2024-05-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.594+0000 7f7a68dc7b80 20 client.28666 trim_cache size 0 max 16384 2024-05-08T23:58:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.611+0000 7f7a68dc7b80 1 Processor -- start 2024-05-08T23:58:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.611+0000 7f7a68dc7b80 1 -- start start 2024-05-08T23:58:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858e60 0x5573f286db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 0x5573f284e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f28794e0 con 0x5573f2858540 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f285fe90 con 0x5573f285ce40 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f285f990 con 0x5573f2858e60 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a55d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858e60 0x5573f286db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a55d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858e60 0x5573f286db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57684/0 (socket says 172.21.15.12:57684) 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33844/0 (socket says 172.21.15.12:33844) 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a55d9b700 1 -- 172.21.15.12:0/1156682649 learned_addr learned my addr 172.21.15.12:0/1156682649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/1156682649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 0x5573f284e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a55d9b700 1 -- 172.21.15.12:0/1156682649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 msgr2=0x5573f284e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a55d9b700 1 --2- 172.21.15.12:0/1156682649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 0x5573f284e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a55d9b700 1 -- 172.21.15.12:0/1156682649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 msgr2=0x5573f2858920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a55d9b700 1 --2- 172.21.15.12:0/1156682649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a55d9b700 1 -- 172.21.15.12:0/1156682649 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573f276a500 con 0x5573f2858e60 2024-05-08T23:58:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.612+0000 7f7a5659c700 1 --2- 172.21.15.12:0/1156682649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.613+0000 7f7a55d9b700 1 --2- 172.21.15.12:0/1156682649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858e60 0x5573f286db20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c09b2a0 tx=0x7f7a4c0cd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f999606ea2b27f40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.613+0000 7f7a457fa700 1 -- 172.21.15.12:0/1156682649 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0b9070 con 0x5573f2858e60 2024-05-08T23:58:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.613+0000 7f7a457fa700 1 -- 172.21.15.12:0/1156682649 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a4c09ac70 con 0x5573f2858e60 2024-05-08T23:58:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.613+0000 7f7a457fa700 1 -- 172.21.15.12:0/1156682649 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c09b7d0 con 0x5573f2858e60 2024-05-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1156682649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858e60 msgr2=0x5573f286db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/1156682649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858e60 0x5573f286db20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c09b2a0 tx=0x7f7a4c0cd040 comp rx=0 tx=0).stop 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1156682649 shutdown_connections 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/1156682649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f285ce40 0x5573f284e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/1156682649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2858920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/1156682649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f2858e60 0x5573f286db20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1156682649 >> 172.21.15.12:0/1156682649 conn(0x5573f27b21b0 msgr2=0x5573f2852f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1156682649 shutdown_connections 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1156682649 wait complete. 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 -- start start 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.614+0000 7f7a68dc7b80 1 Processor -- start 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2863b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 0x5573f28640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a68dc7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f28c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f28649e0 con 0x5573f2858540 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f28c6900 con 0x5573f2858e60 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a68dc7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f28c6a40 con 0x5573f285ce40 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a68dc7b80 10 client.0 _ll_register_callbacks cb 0x5573f2749ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2863b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a56d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f28c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2863b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33846/0 (socket says 172.21.15.12:33846) 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a5659c700 1 -- 172.21.15.12:0/1878192696 learned_addr learned my addr 172.21.15.12:0/1878192696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a55d9b700 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 0x5573f28640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a56d9d700 1 -- 172.21.15.12:0/1878192696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 msgr2=0x5573f28640d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 0x5573f28640d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a56d9d700 1 -- 172.21.15.12:0/1878192696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 msgr2=0x5573f2863b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2863b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a56d9d700 1 -- 172.21.15.12:0/1878192696 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a48017050 con 0x5573f285ce40 2024-05-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a55d9b700 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 0x5573f28640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a5659c700 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2863b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a56d9d700 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f28c63c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a50007820 tx=0x7f7a50013da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3542799c699b398 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.615+0000 7f7a3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a3dffb700 1 -- 172.21.15.12:0/1878192696 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50009620 con 0x5573f285ce40 2024-05-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a3dffb700 1 -- 172.21.15.12:0/1878192696 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7a50009230 con 0x5573f285ce40 2024-05-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1878192696 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a5000f050 con 0x5573f285ce40 2024-05-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a3dffb700 1 -- 172.21.15.12:0/1878192696 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a500137c0 con 0x5573f285ce40 2024-05-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a68dc7b80 10 client.28672 fetch_fsmap learned FSMap version 52 2024-05-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1878192696 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f7a581a4230 con 0x5573f285ce40 2024-05-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a3dffb700 1 -- 172.21.15.12:0/1878192696 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a5000f050 con 0x5573f285ce40 2024-05-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a3dffb700 1 -- 172.21.15.12:0/1878192696 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 52) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7a50026630 con 0x5573f285ce40 2024-05-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a68dc7b80 10 client.28672 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a68dc7b80 10 client.28672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a68dc7b80 20 client.28672 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573f28c7e60 0x5573f28ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a68dc7b80 4 client.28672 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:58:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1878192696 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5573f279ce60 con 0x5573f28c7e60 2024-05-08T23:58:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.616+0000 7f7a5659c700 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573f28c7e60 0x5573f28ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.617+0000 7f7a5659c700 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573f28c7e60 0x5573f28ca280 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73f1d06cc18fc5af server_cookie=61f77b0ef583aa6c in_seq=0 out_seq=0 2024-05-08T23:58:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.617+0000 7f7a3dffb700 10 client.28672 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a3dffb700 1 -- 172.21.15.12:0/1878192696 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5573f279ce60 con 0x5573f28c7e60 2024-05-08T23:58:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a3dffb700 10 client.28672 handle_command_reply: tid=0 2024-05-08T23:58:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 client.28672 shutdown 2024-05-08T23:58:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1878192696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 msgr2=0x5573f28c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f28c63c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a50007820 tx=0x7f7a50013da0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1878192696 shutdown_connections 2024-05-08T23:58:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573f285ce40 0x5573f28c63c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5573f2858540 0x5573f2863b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573f28c7e60 0x5573f28ca280 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 --2- 172.21.15.12:0/1878192696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573f2858e60 0x5573f28640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a3dffb700 0 client.28672 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1878192696 >> 172.21.15.12:0/1878192696 conn(0x5573f27b21b0 msgr2=0x5573f285d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1878192696 shutdown_connections 2024-05-08T23:58:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.662+0000 7f7a68dc7b80 1 -- 172.21.15.12:0/1878192696 wait complete. 2024-05-08T23:58:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a68dc7b80 20 client.28672 trim_cache size 0 max 16384 2024-05-08T23:58:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:58:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:58:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:58:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:58:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/1903012054 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a400a1310 msgr2=0x7f7a400a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a400a1310 0x7f7a400a37d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009fd0 tx=0x7f7a48009260 comp rx=0 tx=0).stop 2024-05-08T23:58:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/1903012054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a58106210 msgr2=0x7f7a58197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a58106210 0x7f7a58197950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0098e0 tx=0x7f7a4c005740 comp rx=0 tx=0).stop 2024-05-08T23:58:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/1903012054 shutdown_connections 2024-05-08T23:58:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a400a1310 0x7f7a400a37d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a5819bcf0 0x7f7a5819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a581010f0 0x7f7a58197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 --2- 172.21.15.12:0/1903012054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a58106210 0x7f7a58197950 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/1903012054 >> 172.21.15.12:0/1903012054 conn(0x7f7a580fc970 msgr2=0x7f7a580fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.663+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/1903012054 shutdown_connections 2024-05-08T23:58:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:37.664+0000 7f7a5cc0d700 1 -- 172.21.15.12:0/1903012054 wait complete. 2024-05-08T23:58:37.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:37 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:58:37.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:37 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:37.897 INFO:tasks.workunit.client.0.smithi012.stdout: 1 665498 6.01 MB/sec execute 513 sec latency 29.769 ms 2024-05-08T23:58:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:38 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:58:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:38 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:38 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:58:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:38 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:58:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:38 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:38 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:58:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:38 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:38 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:38 smithi012 ceph-mon[42503]: pgmap v389: 97 pgs: 97 active+clean; 206 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-08T23:58:38.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:38 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:58:38.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:38 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:38 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:58:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:38 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:58:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:38 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:38 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:58:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:38 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:38 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:38 smithi204 ceph-mon[47373]: pgmap v389: 97 pgs: 97 active+clean; 206 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-08T23:58:38.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:38 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:58:38.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:38 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:38 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:58:38.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:38 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:58:38.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:38 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:38 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:58:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:38 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:38 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:58:38.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:38 smithi160 ceph-mon[46152]: pgmap v389: 97 pgs: 97 active+clean; 206 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-08T23:58:38.897 INFO:tasks.workunit.client.0.smithi012.stdout: 1 667059 6.02 MB/sec execute 514 sec latency 266.054 ms 2024-05-08T23:58:39.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:39 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:58:39.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:39 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:58:39.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:39 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:58:39.897 INFO:tasks.workunit.client.0.smithi012.stdout: 1 668403 6.02 MB/sec execute 515 sec latency 22.765 ms 2024-05-08T23:58:40.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:40 smithi012 ceph-mon[42503]: pgmap v390: 97 pgs: 97 active+clean; 206 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-05-08T23:58:40.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:58:40.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:58:40.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:58:40.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:40 smithi204 ceph-mon[47373]: pgmap v390: 97 pgs: 97 active+clean; 206 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-05-08T23:58:40.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:58:40.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:58:40.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:58:40.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:40 smithi160 ceph-mon[46152]: pgmap v390: 97 pgs: 97 active+clean; 206 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-05-08T23:58:40.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:40 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:58:40.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:40 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:58:40.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:40 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:58:40.897 INFO:tasks.workunit.client.0.smithi012.stdout: 1 669314 6.02 MB/sec execute 516 sec latency 213.289 ms 2024-05-08T23:58:41.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 669314 6.01 MB/sec execute 517 sec latency 1213.336 ms 2024-05-08T23:58:42.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:42 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:58:42.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:42 smithi012 ceph-mon[42503]: pgmap v391: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-05-08T23:58:42.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:42 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:58:42.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:42 smithi204 ceph-mon[47373]: pgmap v391: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-05-08T23:58:42.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:42 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:58:42.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:42 smithi160 ceph-mon[46152]: pgmap v391: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-05-08T23:58:42.897 INFO:tasks.workunit.client.0.smithi012.stdout: 1 669509 5.99 MB/sec execute 518 sec latency 2107.974 ms 2024-05-08T23:58:43.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:43 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:58:43.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:43 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:58:43.897 INFO:tasks.workunit.client.0.smithi012.stdout: 1 671555 6.01 MB/sec execute 519 sec latency 14.274 ms 2024-05-08T23:58:43.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:43 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:58:44.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:44 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:58:44.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:44 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:58:44.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:44 smithi204 ceph-mon[47373]: pgmap v392: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-08T23:58:44.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:44 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:58:44.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:44 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:58:44.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:44 smithi160 ceph-mon[46152]: pgmap v392: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-08T23:58:44.897 INFO:tasks.workunit.client.0.smithi012.stdout: 1 672755 6.01 MB/sec execute 520 sec latency 22.536 ms 2024-05-08T23:58:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:44 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:58:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:44 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:58:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:44 smithi012 ceph-mon[42503]: pgmap v392: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-08T23:58:45.897 INFO:tasks.workunit.client.0.smithi012.stdout: 1 672856 6.00 MB/sec execute 521 sec latency 877.836 ms 2024-05-08T23:58:45.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:45 smithi012 ceph-mon[42503]: pgmap v393: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-08T23:58:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:45 smithi204 ceph-mon[47373]: pgmap v393: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-08T23:58:46.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:45 smithi160 ceph-mon[46152]: pgmap v393: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-08T23:58:46.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 674852 6.01 MB/sec execute 522 sec latency 1015.150 ms 2024-05-08T23:58:47.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 676219 6.01 MB/sec execute 523 sec latency 16.308 ms 2024-05-08T23:58:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:47 smithi012 ceph-mon[42503]: pgmap v394: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 193 op/s 2024-05-08T23:58:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:47 smithi204 ceph-mon[47373]: pgmap v394: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 193 op/s 2024-05-08T23:58:48.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:47 smithi160 ceph-mon[46152]: pgmap v394: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 193 op/s 2024-05-08T23:58:48.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 677973 6.01 MB/sec execute 524 sec latency 307.689 ms 2024-05-08T23:58:49.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 679413 6.02 MB/sec execute 525 sec latency 23.704 ms 2024-05-08T23:58:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:49 smithi012 ceph-mon[42503]: pgmap v395: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-08T23:58:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:49 smithi204 ceph-mon[47373]: pgmap v395: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-08T23:58:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:49 smithi160 ceph-mon[46152]: pgmap v395: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-08T23:58:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.792+0000 7fd220287700 1 -- 172.21.15.12:0/839236427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd218106230 msgr2=0x7fd21810ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.792+0000 7fd220287700 1 --2- 172.21.15.12:0/839236427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd218106230 0x7fd21810ddb0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd21000b0a0 tx=0x7fd21000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.792+0000 7fd220287700 1 -- 172.21.15.12:0/839236427 shutdown_connections 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.792+0000 7fd220287700 1 --2- 172.21.15.12:0/839236427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd218106230 0x7fd21810ddb0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.792+0000 7fd220287700 1 --2- 172.21.15.12:0/839236427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd218069900 0x7fd218069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.792+0000 7fd220287700 1 --2- 172.21.15.12:0/839236427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd21806a2b0 0x7fd218105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.792+0000 7fd220287700 1 -- 172.21.15.12:0/839236427 >> 172.21.15.12:0/839236427 conn(0x7fd218076c10 msgr2=0x7fd218077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.792+0000 7fd220287700 1 -- 172.21.15.12:0/839236427 shutdown_connections 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.792+0000 7fd220287700 1 -- 172.21.15.12:0/839236427 wait complete. 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd220287700 1 Processor -- start 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd220287700 1 -- start start 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd220287700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd218069900 0x7fd2181996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd220287700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd218106230 0x7fd218199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd220287700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd21819df50 0x7fd21819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd220287700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21819e9d0 con 0x7fd218106230 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd220287700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21819eb40 con 0x7fd218069900 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd220287700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21819ed00 con 0x7fd21819df50 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21d822700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd218106230 0x7fd218199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21d822700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd218106230 0x7fd218199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50410/0 (socket says 172.21.15.12:50410) 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21d822700 1 -- 172.21.15.12:0/2716760661 learned_addr learned my addr 172.21.15.12:0/2716760661 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21e023700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd218069900 0x7fd2181996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21e824700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd21819df50 0x7fd21819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21d822700 1 -- 172.21.15.12:0/2716760661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd21819df50 msgr2=0x7fd21819e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21d822700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd21819df50 0x7fd21819e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21d822700 1 -- 172.21.15.12:0/2716760661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd218069900 msgr2=0x7fd2181996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21d822700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd218069900 0x7fd2181996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21d822700 1 -- 172.21.15.12:0/2716760661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210009d00 con 0x7fd218106230 2024-05-08T23:58:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.793+0000 7fd21e023700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd218069900 0x7fd2181996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:58:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.794+0000 7fd21d822700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd218106230 0x7fd218199c00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd20800e9f0 tx=0x7fd20800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.794+0000 7fd20f7fe700 1 -- 172.21.15.12:0/2716760661 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208009750 con 0x7fd218106230 2024-05-08T23:58:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.794+0000 7fd220287700 1 -- 172.21.15.12:0/2716760661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2181a6260 con 0x7fd218106230 2024-05-08T23:58:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.794+0000 7fd20f7fe700 1 -- 172.21.15.12:0/2716760661 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd208004cf0 con 0x7fd218106230 2024-05-08T23:58:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.794+0000 7fd20f7fe700 1 -- 172.21.15.12:0/2716760661 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20800eee0 con 0x7fd218106230 2024-05-08T23:58:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.794+0000 7fd220287700 1 -- 172.21.15.12:0/2716760661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2181a66f0 con 0x7fd218106230 2024-05-08T23:58:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.796+0000 7fd20f7fe700 1 -- 172.21.15.12:0/2716760661 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd208004750 con 0x7fd218106230 2024-05-08T23:58:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.797+0000 7fd20f7fe700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2040a1290 0x7fd2040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.797+0000 7fd20f7fe700 1 -- 172.21.15.12:0/2716760661 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd208014070 con 0x7fd218106230 2024-05-08T23:58:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.797+0000 7fd21e023700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2040a1290 0x7fd2040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.797+0000 7fd21e023700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2040a1290 0x7fd2040a3750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd214009cb0 tx=0x7fd214009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.798+0000 7fd22c441b80 1 Processor -- start 2024-05-08T23:58:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.798+0000 7fd22c441b80 1 -- start start 2024-05-08T23:58:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.798+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f623950 0x55754f623d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.798+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f624270 0x55754f61df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.798+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f61e480 0x55754f61e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.798+0000 7fd22c441b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f5750f0 con 0x55754f623950 2024-05-08T23:58:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.798+0000 7fd22c441b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f572ab0 con 0x55754f61e480 2024-05-08T23:58:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.798+0000 7fd22c441b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f56e3a0 con 0x55754f624270 2024-05-08T23:58:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21e023700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f623950 0x55754f623d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21e023700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f623950 0x55754f623d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50424/0 (socket says 172.21.15.12:50424) 2024-05-08T23:58:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21d822700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f624270 0x55754f61df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21e023700 1 -- 172.21.15.12:0/56244494 learned_addr learned my addr 172.21.15.12:0/56244494 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21e824700 1 --2- 172.21.15.12:0/56244494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f61e480 0x55754f61e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21e023700 1 -- 172.21.15.12:0/56244494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f624270 msgr2=0x55754f61df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21e023700 1 --2- 172.21.15.12:0/56244494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f624270 0x55754f61df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21e023700 1 -- 172.21.15.12:0/56244494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f61e480 msgr2=0x55754f61e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21e023700 1 --2- 172.21.15.12:0/56244494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f61e480 0x55754f61e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21e023700 1 -- 172.21.15.12:0/56244494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55754f3ef350 con 0x55754f623950 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21d822700 1 --2- 172.21.15.12:0/56244494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f624270 0x55754f61df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd21e023700 1 --2- 172.21.15.12:0/56244494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f623950 0x55754f623d30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd214011fb0 tx=0x7fd214014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0be31ee4aa3d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd20cda4700 1 -- 172.21.15.12:0/56244494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214003680 con 0x55754f623950 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd20cda4700 1 -- 172.21.15.12:0/56244494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd214003ca0 con 0x55754f623950 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.799+0000 7fd20cda4700 1 -- 172.21.15.12:0/56244494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214012cd0 con 0x55754f623950 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.800+0000 7fd22c441b80 1 -- 172.21.15.12:0/56244494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f623950 msgr2=0x55754f623d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.800+0000 7fd22c441b80 1 --2- 172.21.15.12:0/56244494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f623950 0x55754f623d30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd214011fb0 tx=0x7fd214014000 comp rx=0 tx=0).stop 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.800+0000 7fd22c441b80 1 -- 172.21.15.12:0/56244494 shutdown_connections 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.800+0000 7fd22c441b80 1 --2- 172.21.15.12:0/56244494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f61e480 0x55754f61e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.800+0000 7fd22c441b80 1 --2- 172.21.15.12:0/56244494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f623950 0x55754f623d30 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.800+0000 7fd22c441b80 1 --2- 172.21.15.12:0/56244494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f624270 0x55754f61df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.800+0000 7fd22c441b80 1 -- 172.21.15.12:0/56244494 >> 172.21.15.12:0/56244494 conn(0x55754f57c1b0 msgr2=0x55754f618840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.800+0000 7fd22c441b80 1 -- 172.21.15.12:0/56244494 shutdown_connections 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.800+0000 7fd22c441b80 1 -- 172.21.15.12:0/56244494 wait complete. 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd22c441b80 1 -- start start 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd22c441b80 1 Processor -- start 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f62a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f6275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f627cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd22c441b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f50a500 con 0x55754f624270 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd22c441b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f50a1a0 con 0x55754f623950 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd22c441b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f3f1640 con 0x55754f61e480 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd22c441b80 10 client.0 _ll_register_callbacks cb 0x55754f513ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd21e824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f627cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd21d822700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f6275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd21d822700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f6275c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:40608/0 (socket says 172.21.15.12:40608) 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd21d822700 1 -- 172.21.15.12:0/352105298 learned_addr learned my addr 172.21.15.12:0/352105298 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd21e824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f627cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50434/0 (socket says 172.21.15.12:50434) 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.801+0000 7fd21e023700 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f62a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd21d822700 1 -- 172.21.15.12:0/352105298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 msgr2=0x55754f62a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd21d822700 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f62a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd21d822700 1 -- 172.21.15.12:0/352105298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 msgr2=0x55754f627cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd21d822700 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f627cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd21d822700 1 -- 172.21.15.12:0/352105298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2181a66f0 con 0x55754f623950 2024-05-08T23:58:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd21d822700 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f6275c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd208099fb0 tx=0x7fd208099a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6bc32f080837cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd201ffb700 1 -- 172.21.15.12:0/352105298 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080b6070 con 0x55754f623950 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd22c441b80 1 -- 172.21.15.12:0/352105298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55754f4c7210 con 0x55754f623950 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd201ffb700 1 -- 172.21.15.12:0/352105298 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd20809de50 con 0x55754f623950 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd201ffb700 1 -- 172.21.15.12:0/352105298 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2080a24a0 con 0x55754f623950 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd22c441b80 10 client.28682 fetch_fsmap learned FSMap version 53 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd22c441b80 1 -- 172.21.15.12:0/352105298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7fd214011c60 con 0x55754f623950 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.802+0000 7fd201ffb700 1 -- 172.21.15.12:0/352105298 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55754f4c7210 con 0x55754f623950 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.803+0000 7fd201ffb700 1 -- 172.21.15.12:0/352105298 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 53) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd20809a280 con 0x55754f623950 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.803+0000 7fd22c441b80 10 client.28682 fetch_fsmap finished waiting for FSMap version 53 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.803+0000 7fd22c441b80 10 client.28682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.803+0000 7fd22c441b80 20 client.28682 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.803+0000 7fd22c441b80 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55754f637ca0 0x55754f63a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.803+0000 7fd22c441b80 4 client.28682 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.803+0000 7fd22c441b80 1 -- 172.21.15.12:0/352105298 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55754f5ed7b0 con 0x55754f637ca0 2024-05-08T23:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.803+0000 7fd21e023700 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55754f637ca0 0x55754f63a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.803+0000 7fd201ffb700 10 client.28682 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.803+0000 7fd21e023700 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55754f637ca0 0x55754f63a0c0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d373ef2cf2ea9365 server_cookie=34a3edd19a15ac5a in_seq=0 out_seq=0 2024-05-08T23:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd201ffb700 1 -- 172.21.15.12:0/352105298 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55754f5ed7b0 con 0x55754f637ca0 2024-05-08T23:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd201ffb700 10 client.28682 handle_command_reply: tid=0 2024-05-08T23:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd22c441b80 1 client.28682 shutdown 2024-05-08T23:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd22c441b80 1 -- 172.21.15.12:0/352105298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 msgr2=0x55754f6275c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd22c441b80 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f6275c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd208099fb0 tx=0x7fd208099a70 comp rx=0 tx=0).stop 2024-05-08T23:58:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd22c441b80 1 -- 172.21.15.12:0/352105298 shutdown_connections 2024-05-08T23:58:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd22c441b80 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f627cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd22c441b80 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f62a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd22c441b80 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55754f637ca0 0x55754f63a0c0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd22c441b80 1 --2- 172.21.15.12:0/352105298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f6275c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd22c441b80 1 -- 172.21.15.12:0/352105298 >> 172.21.15.12:0/352105298 conn(0x55754f57c1b0 msgr2=0x55754f61cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.804+0000 7fd201ffb700 0 client.28682 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.805+0000 7fd22c441b80 1 -- 172.21.15.12:0/352105298 shutdown_connections 2024-05-08T23:58:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.805+0000 7fd22c441b80 1 -- 172.21.15.12:0/352105298 wait complete. 2024-05-08T23:58:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.805+0000 7fd22c441b80 20 client.28682 trim_cache size 0 max 16384 2024-05-08T23:58:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.820+0000 7fd22c441b80 1 Processor -- start 2024-05-08T23:58:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.820+0000 7fd22c441b80 1 -- start start 2024-05-08T23:58:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f618910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f63e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd22c441b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f635390 con 0x55754f624270 2024-05-08T23:58:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd22c441b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f6197b0 con 0x55754f623950 2024-05-08T23:58:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd22c441b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f61a0f0 con 0x55754f61e480 2024-05-08T23:58:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21e824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21e824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50438/0 (socket says 172.21.15.12:50438) 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21e824700 1 -- 172.21.15.12:0/1354326773 learned_addr learned my addr 172.21.15.12:0/1354326773 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21d822700 1 --2- 172.21.15.12:0/1354326773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f63e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21e023700 1 --2- 172.21.15.12:0/1354326773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f618910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21e824700 1 -- 172.21.15.12:0/1354326773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 msgr2=0x55754f618910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21e824700 1 --2- 172.21.15.12:0/1354326773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f618910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21e824700 1 -- 172.21.15.12:0/1354326773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 msgr2=0x55754f63e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21e824700 1 --2- 172.21.15.12:0/1354326773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f63e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21e824700 1 -- 172.21.15.12:0/1354326773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55754f39bb00 con 0x55754f624270 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21d822700 1 --2- 172.21.15.12:0/1354326773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f63e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.821+0000 7fd21e824700 1 --2- 172.21.15.12:0/1354326773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62ebe0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd210015000 tx=0x7fd2100094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f9e9a8aad21aa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd20d5a5700 1 -- 172.21.15.12:0/1354326773 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000e040 con 0x55754f624270 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd20d5a5700 1 -- 172.21.15.12:0/1354326773 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd210007d40 con 0x55754f624270 2024-05-08T23:58:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd20d5a5700 1 -- 172.21.15.12:0/1354326773 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210003e00 con 0x55754f624270 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd22c441b80 1 -- 172.21.15.12:0/1354326773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 msgr2=0x55754f62ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd22c441b80 1 --2- 172.21.15.12:0/1354326773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62ebe0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd210015000 tx=0x7fd2100094f0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd22c441b80 1 -- 172.21.15.12:0/1354326773 shutdown_connections 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd22c441b80 1 --2- 172.21.15.12:0/1354326773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62ebe0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd22c441b80 1 --2- 172.21.15.12:0/1354326773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f618910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd22c441b80 1 --2- 172.21.15.12:0/1354326773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f63e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd22c441b80 1 -- 172.21.15.12:0/1354326773 >> 172.21.15.12:0/1354326773 conn(0x55754f57c1b0 msgr2=0x55754f61d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd22c441b80 1 -- 172.21.15.12:0/1354326773 shutdown_connections 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.822+0000 7fd22c441b80 1 -- 172.21.15.12:0/1354326773 wait complete. 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd22c441b80 1 -- start start 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd22c441b80 1 Processor -- start 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f62cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f62d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd22c441b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd22c441b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f68d160 con 0x55754f624270 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd22c441b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f68d2a0 con 0x55754f623950 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd22c441b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754f68d3e0 con 0x55754f61e480 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd22c441b80 10 client.0 _ll_register_callbacks cb 0x7fd208099e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd21d822700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f62d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd21e824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd21d822700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f62d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:40628/0 (socket says 172.21.15.12:40628) 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd21d822700 1 -- 172.21.15.12:0/244751735 learned_addr learned my addr 172.21.15.12:0/244751735 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd21e824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50450/0 (socket says 172.21.15.12:50450) 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.823+0000 7fd21e023700 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f62cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd21e824700 1 -- 172.21.15.12:0/244751735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 msgr2=0x55754f62cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd21e824700 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f62cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd21e824700 1 -- 172.21.15.12:0/244751735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 msgr2=0x55754f62d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd21e824700 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f62d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd21e824700 1 -- 172.21.15.12:0/244751735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd214011c60 con 0x55754f624270 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd21d822700 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f62d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd21e824700 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62d640 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x55754f613620 tx=0x7fd2100099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28e4f32efacc216 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd201ffb700 1 -- 172.21.15.12:0/244751735 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000e040 con 0x55754f624270 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd201ffb700 1 -- 172.21.15.12:0/244751735 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd210004e40 con 0x55754f624270 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd22c441b80 1 -- 172.21.15.12:0/244751735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd214003ca0 con 0x55754f624270 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd201ffb700 1 -- 172.21.15.12:0/244751735 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2100092e0 con 0x55754f624270 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd22c441b80 10 client.18975 fetch_fsmap learned FSMap version 53 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd22c441b80 1 -- 172.21.15.12:0/244751735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7fd21810e620 con 0x55754f624270 2024-05-08T23:58:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd201ffb700 1 -- 172.21.15.12:0/244751735 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd214003ca0 con 0x55754f624270 2024-05-08T23:58:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.824+0000 7fd201ffb700 1 -- 172.21.15.12:0/244751735 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 53) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd21000b0d0 con 0x55754f624270 2024-05-08T23:58:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.825+0000 7fd22c441b80 10 client.18975 fetch_fsmap finished waiting for FSMap version 53 2024-05-08T23:58:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.825+0000 7fd22c441b80 10 client.18975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:58:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.825+0000 7fd22c441b80 20 client.18975 populate_metadata read hostname 'smithi012' 2024-05-08T23:58:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.825+0000 7fd22c441b80 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55754f68e880 0x55754f690c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:58:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.825+0000 7fd22c441b80 4 client.18975 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:58:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.825+0000 7fd22c441b80 1 -- 172.21.15.12:0/244751735 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55754f566e60 con 0x55754f68e880 2024-05-08T23:58:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.825+0000 7fd21e023700 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55754f68e880 0x55754f690c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:58:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.825+0000 7fd21e023700 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55754f68e880 0x55754f690c70 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d43f6b3cce38a0d6 server_cookie=dbfe013912103945 in_seq=0 out_seq=0 2024-05-08T23:58:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.825+0000 7fd201ffb700 10 client.18975 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.897+0000 7fd201ffb700 1 -- 172.21.15.12:0/244751735 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55754f566e60 con 0x55754f68e880 2024-05-08T23:58:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.897+0000 7fd201ffb700 10 client.18975 handle_command_reply: tid=0 2024-05-08T23:58:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.897+0000 7fd22c441b80 1 client.18975 shutdown 2024-05-08T23:58:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.898+0000 7fd22c441b80 1 -- 172.21.15.12:0/244751735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 msgr2=0x55754f62d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.898+0000 7fd22c441b80 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62d640 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x55754f613620 tx=0x7fd2100099e0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.898+0000 7fd22c441b80 1 -- 172.21.15.12:0/244751735 shutdown_connections 2024-05-08T23:58:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.898+0000 7fd22c441b80 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55754f624270 0x55754f62d640 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.898+0000 7fd22c441b80 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55754f61e480 0x55754f62cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.898+0000 7fd22c441b80 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55754f68e880 0x55754f690c70 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.898+0000 7fd22c441b80 1 --2- 172.21.15.12:0/244751735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55754f623950 0x55754f62d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.898+0000 7fd22c441b80 1 -- 172.21.15.12:0/244751735 >> 172.21.15.12:0/244751735 conn(0x55754f57c1b0 msgr2=0x55754f633690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.898+0000 7fd201ffb700 0 client.18975 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:58:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.901+0000 7fd22c441b80 1 -- 172.21.15.12:0/244751735 shutdown_connections 2024-05-08T23:58:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.901+0000 7fd22c441b80 1 -- 172.21.15.12:0/244751735 wait complete. 2024-05-08T23:58:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.902+0000 7fd22c441b80 20 client.18975 trim_cache size 0 max 16384 2024-05-08T23:58:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:58:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:58:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:58:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:58:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.902+0000 7fd220287700 1 -- 172.21.15.12:0/2716760661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2040a1290 msgr2=0x7fd2040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.902+0000 7fd220287700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2040a1290 0x7fd2040a3750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd214009cb0 tx=0x7fd214009260 comp rx=0 tx=0).stop 2024-05-08T23:58:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.902+0000 7fd220287700 1 -- 172.21.15.12:0/2716760661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd218106230 msgr2=0x7fd218199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:58:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.902+0000 7fd220287700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd218106230 0x7fd218199c00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd20800e9f0 tx=0x7fd20800ed00 comp rx=0 tx=0).stop 2024-05-08T23:58:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.902+0000 7fd220287700 1 -- 172.21.15.12:0/2716760661 shutdown_connections 2024-05-08T23:58:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.902+0000 7fd220287700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2040a1290 0x7fd2040a3750 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.902+0000 7fd220287700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd21819df50 0x7fd21819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.902+0000 7fd220287700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd218069900 0x7fd2181996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.902+0000 7fd220287700 1 --2- 172.21.15.12:0/2716760661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd218106230 0x7fd218199c00 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:58:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.903+0000 7fd220287700 1 -- 172.21.15.12:0/2716760661 >> 172.21.15.12:0/2716760661 conn(0x7fd218076c10 msgr2=0x7fd21810c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:58:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.903+0000 7fd220287700 1 -- 172.21.15.12:0/2716760661 shutdown_connections 2024-05-08T23:58:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:58:50.903+0000 7fd220287700 1 -- 172.21.15.12:0/2716760661 wait complete. 2024-05-08T23:58:50.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 680004 6.01 MB/sec execute 526 sec latency 443.764 ms 2024-05-08T23:58:51.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 681987 6.02 MB/sec execute 527 sec latency 28.611 ms 2024-05-08T23:58:51.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:51 smithi012 ceph-mon[42503]: pgmap v396: 97 pgs: 97 active+clean; 207 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-05-08T23:58:52.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:51 smithi204 ceph-mon[47373]: pgmap v396: 97 pgs: 97 active+clean; 207 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-05-08T23:58:52.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:51 smithi160 ceph-mon[46152]: pgmap v396: 97 pgs: 97 active+clean; 207 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-05-08T23:58:52.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 683350 6.03 MB/sec execute 528 sec latency 22.462 ms 2024-05-08T23:58:53.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 685065 6.03 MB/sec execute 529 sec latency 328.943 ms 2024-05-08T23:58:53.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:53 smithi012 ceph-mon[42503]: pgmap v397: 97 pgs: 97 active+clean; 207 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-05-08T23:58:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:53 smithi204 ceph-mon[47373]: pgmap v397: 97 pgs: 97 active+clean; 207 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-05-08T23:58:54.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:53 smithi160 ceph-mon[46152]: pgmap v397: 97 pgs: 97 active+clean; 207 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-05-08T23:58:54.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 686468 6.03 MB/sec execute 530 sec latency 12.948 ms 2024-05-08T23:58:55.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 687092 6.03 MB/sec execute 531 sec latency 433.812 ms 2024-05-08T23:58:55.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:55 smithi012 ceph-mon[42503]: pgmap v398: 97 pgs: 97 active+clean; 207 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-05-08T23:58:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:55 smithi204 ceph-mon[47373]: pgmap v398: 97 pgs: 97 active+clean; 207 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-05-08T23:58:56.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:55 smithi160 ceph-mon[46152]: pgmap v398: 97 pgs: 97 active+clean; 207 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-05-08T23:58:56.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 688952 6.04 MB/sec execute 532 sec latency 565.672 ms 2024-05-08T23:58:57.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 690166 6.03 MB/sec execute 533 sec latency 26.374 ms 2024-05-08T23:58:57.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:57 smithi012 ceph-mon[42503]: pgmap v399: 97 pgs: 97 active+clean; 213 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 234 op/s 2024-05-08T23:58:58.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:57 smithi204 ceph-mon[47373]: pgmap v399: 97 pgs: 97 active+clean; 213 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 234 op/s 2024-05-08T23:58:58.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:57 smithi160 ceph-mon[46152]: pgmap v399: 97 pgs: 97 active+clean; 213 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 234 op/s 2024-05-08T23:58:58.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 690640 6.03 MB/sec execute 534 sec latency 487.070 ms 2024-05-08T23:58:59.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 690640 6.02 MB/sec execute 535 sec latency 1487.170 ms 2024-05-08T23:58:59.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:58:59 smithi012 ceph-mon[42503]: pgmap v400: 97 pgs: 97 active+clean; 213 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-05-08T23:59:00.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:58:59 smithi204 ceph-mon[47373]: pgmap v400: 97 pgs: 97 active+clean; 213 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-05-08T23:59:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:58:59 smithi160 ceph-mon[46152]: pgmap v400: 97 pgs: 97 active+clean; 213 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-05-08T23:59:00.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 691507 6.01 MB/sec execute 536 sec latency 1737.118 ms 2024-05-08T23:59:01.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 693084 6.02 MB/sec execute 537 sec latency 20.581 ms 2024-05-08T23:59:01.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:01 smithi012 ceph-mon[42503]: pgmap v401: 97 pgs: 97 active+clean; 214 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 277 op/s 2024-05-08T23:59:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:01 smithi204 ceph-mon[47373]: pgmap v401: 97 pgs: 97 active+clean; 214 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 277 op/s 2024-05-08T23:59:02.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:01 smithi160 ceph-mon[46152]: pgmap v401: 97 pgs: 97 active+clean; 214 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 277 op/s 2024-05-08T23:59:02.898 INFO:tasks.workunit.client.0.smithi012.stdout: 1 694470 6.02 MB/sec execute 538 sec latency 126.342 ms 2024-05-08T23:59:03.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 696187 6.03 MB/sec execute 539 sec latency 25.862 ms 2024-05-08T23:59:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.031+0000 7f8a1782d700 1 -- 172.21.15.12:0/3358686999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a100682e0 msgr2=0x7f8a10068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.031+0000 7f8a1782d700 1 --2- 172.21.15.12:0/3358686999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a100682e0 0x7f8a10068760 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800b0a0 tx=0x7f8a0800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.031+0000 7f8a1782d700 1 -- 172.21.15.12:0/3358686999 shutdown_connections 2024-05-08T23:59:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.031+0000 7f8a1782d700 1 --2- 172.21.15.12:0/3358686999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a100682e0 0x7f8a10068760 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.031+0000 7f8a1782d700 1 --2- 172.21.15.12:0/3358686999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a10105e30 0x7f8a101038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.031+0000 7f8a1782d700 1 --2- 172.21.15.12:0/3358686999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a10103df0 0x7f8a10104270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.031+0000 7f8a1782d700 1 -- 172.21.15.12:0/3358686999 >> 172.21.15.12:0/3358686999 conn(0x7f8a10076f10 msgr2=0x7f8a10077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.031+0000 7f8a1782d700 1 -- 172.21.15.12:0/3358686999 shutdown_connections 2024-05-08T23:59:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.031+0000 7f8a1782d700 1 -- 172.21.15.12:0/3358686999 wait complete. 2024-05-08T23:59:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a1782d700 1 Processor -- start 2024-05-08T23:59:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a1782d700 1 -- start start 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a1782d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a100682e0 0x7f8a101a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a1782d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a10103df0 0x7f8a101a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a1782d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a10105e30 0x7f8a1019d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a1782d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1019dee0 con 0x7f8a10105e30 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a1782d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1019e050 con 0x7f8a10103df0 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a1782d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1019e210 con 0x7f8a100682e0 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a15dca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a10105e30 0x7f8a1019d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a15dca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a10105e30 0x7f8a1019d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37324/0 (socket says 172.21.15.12:37324) 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a155c9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a100682e0 0x7f8a101a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a15dca700 1 -- 172.21.15.12:0/4218517691 learned_addr learned my addr 172.21.15.12:0/4218517691 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.032+0000 7f8a14dc8700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a10103df0 0x7f8a101a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a15dca700 1 -- 172.21.15.12:0/4218517691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a100682e0 msgr2=0x7f8a101a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a15dca700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a100682e0 0x7f8a101a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a15dca700 1 -- 172.21.15.12:0/4218517691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a10103df0 msgr2=0x7f8a101a3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a15dca700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a10103df0 0x7f8a101a3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a15dca700 1 -- 172.21.15.12:0/4218517691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a08009d00 con 0x7f8a10105e30 2024-05-08T23:59:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a15dca700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a10105e30 0x7f8a1019d870 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800b070 tx=0x7f8a080078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a067fc700 1 -- 172.21.15.12:0/4218517691 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800e040 con 0x7f8a10105e30 2024-05-08T23:59:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a067fc700 1 -- 172.21.15.12:0/4218517691 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0801d470 con 0x7f8a10105e30 2024-05-08T23:59:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a1782d700 1 -- 172.21.15.12:0/4218517691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1019e490 con 0x7f8a10105e30 2024-05-08T23:59:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a067fc700 1 -- 172.21.15.12:0/4218517691 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a080052a0 con 0x7f8a10105e30 2024-05-08T23:59:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.033+0000 7f8a1782d700 1 -- 172.21.15.12:0/4218517691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a101aa6e0 con 0x7f8a10105e30 2024-05-08T23:59:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.036+0000 7f8a067fc700 1 -- 172.21.15.12:0/4218517691 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8a0801d5e0 con 0x7f8a10105e30 2024-05-08T23:59:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.036+0000 7f8a067fc700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89fc0a1310 0x7f89fc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.037+0000 7f8a155c9700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89fc0a1310 0x7f89fc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.037+0000 7f8a067fc700 1 -- 172.21.15.12:0/4218517691 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8a080cdf30 con 0x7f8a10105e30 2024-05-08T23:59:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.037+0000 7f8a155c9700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89fc0a1310 0x7f89fc0a37d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009be0 tx=0x7f8a0c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a239e7b80 1 Processor -- start 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a239e7b80 1 -- start start 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888d950 0x557c6888dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68887f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c68888480 0x557c688888a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c687df0f0 con 0x557c6888d950 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c687dcab0 con 0x557c6888e270 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c687d83a0 con 0x557c68888480 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a155c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888d950 0x557c6888dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a155c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888d950 0x557c6888dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37330/0 (socket says 172.21.15.12:37330) 2024-05-08T23:59:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a155c9700 1 -- 172.21.15.12:0/339160164 learned_addr learned my addr 172.21.15.12:0/339160164 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a14dc8700 1 --2- 172.21.15.12:0/339160164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68887f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.038+0000 7f8a15dca700 1 --2- 172.21.15.12:0/339160164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c68888480 0x557c688888a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.039+0000 7f8a155c9700 1 -- 172.21.15.12:0/339160164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c68888480 msgr2=0x557c688888a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.039+0000 7f8a155c9700 1 --2- 172.21.15.12:0/339160164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c68888480 0x557c688888a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.039+0000 7f8a155c9700 1 -- 172.21.15.12:0/339160164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 msgr2=0x557c68887f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.039+0000 7f8a155c9700 1 --2- 172.21.15.12:0/339160164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68887f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.039+0000 7f8a155c9700 1 -- 172.21.15.12:0/339160164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c68659350 con 0x557c6888d950 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.039+0000 7f8a15dca700 1 --2- 172.21.15.12:0/339160164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c68888480 0x557c688888a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.039+0000 7f8a155c9700 1 --2- 172.21.15.12:0/339160164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888d950 0x557c6888dd30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c011d70 tx=0x7f8a0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b4e1df37e16d9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.039+0000 7f89fb7fe700 1 -- 172.21.15.12:0/339160164 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c011da0 con 0x557c6888d950 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.039+0000 7f89fb7fe700 1 -- 172.21.15.12:0/339160164 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0c004020 con 0x557c6888d950 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.039+0000 7f89fb7fe700 1 -- 172.21.15.12:0/339160164 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c003bc0 con 0x557c6888d950 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 -- 172.21.15.12:0/339160164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888d950 msgr2=0x557c6888dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/339160164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888d950 0x557c6888dd30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c011d70 tx=0x7f8a0c014000 comp rx=0 tx=0).stop 2024-05-08T23:59:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 -- 172.21.15.12:0/339160164 shutdown_connections 2024-05-08T23:59:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/339160164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c68888480 0x557c688888a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/339160164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888d950 0x557c6888dd30 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/339160164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68887f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 -- 172.21.15.12:0/339160164 >> 172.21.15.12:0/339160164 conn(0x557c687e61b0 msgr2=0x557c68882840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 -- 172.21.15.12:0/339160164 shutdown_connections 2024-05-08T23:59:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 -- 172.21.15.12:0/339160164 wait complete. 2024-05-08T23:59:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 -- start start 2024-05-08T23:59:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.040+0000 7f8a239e7b80 1 Processor -- start 2024-05-08T23:59:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68894190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c688914d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68891b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c68774500 con 0x557c68888480 2024-05-08T23:59:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c687741a0 con 0x557c6888e270 2024-05-08T23:59:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6865b640 con 0x557c6888d950 2024-05-08T23:59:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a239e7b80 10 client.0 _ll_register_callbacks cb 0x557c6877dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a155c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68894190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a155c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68894190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37346/0 (socket says 172.21.15.12:37346) 2024-05-08T23:59:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a155c9700 1 -- 172.21.15.12:0/175601922 learned_addr learned my addr 172.21.15.12:0/175601922 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a15dca700 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68891b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a14dc8700 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c688914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a155c9700 1 -- 172.21.15.12:0/175601922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 msgr2=0x557c688914d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a155c9700 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c688914d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a155c9700 1 -- 172.21.15.12:0/175601922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 msgr2=0x557c68891b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a155c9700 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68891b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.041+0000 7f8a155c9700 1 -- 172.21.15.12:0/175601922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a101aa6e0 con 0x557c68888480 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f8a155c9700 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68894190 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009210 tx=0x7f8a0c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a22581060ffabcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f89f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f89f8ff9700 1 -- 172.21.15.12:0/175601922 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c003860 con 0x557c68888480 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f89f8ff9700 1 -- 172.21.15.12:0/175601922 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8a0c0039c0 con 0x557c68888480 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f8a239e7b80 1 -- 172.21.15.12:0/175601922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c68731210 con 0x557c68888480 2024-05-08T23:59:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f89f8ff9700 1 -- 172.21.15.12:0/175601922 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a0c003b80 con 0x557c68888480 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f8a239e7b80 10 client.18987 fetch_fsmap learned FSMap version 53 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f8a239e7b80 1 -- 172.21.15.12:0/175601922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f8a0c017070 con 0x557c68888480 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f89f8ff9700 1 -- 172.21.15.12:0/175601922 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0c03c020 con 0x557c68888480 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f89f8ff9700 1 -- 172.21.15.12:0/175601922 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 53) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a0c024030 con 0x557c68888480 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f8a239e7b80 10 client.18987 fetch_fsmap finished waiting for FSMap version 53 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f8a239e7b80 10 client.18987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.042+0000 7f8a239e7b80 20 client.18987 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.043+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557c688a1b60 0x557c688a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.043+0000 7f8a239e7b80 4 client.18987 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.043+0000 7f8a239e7b80 1 -- 172.21.15.12:0/175601922 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c688577b0 con 0x557c688a1b60 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.043+0000 7f8a14dc8700 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557c688a1b60 0x557c688a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.043+0000 7f89f8ff9700 10 client.18987 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.043+0000 7f8a14dc8700 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557c688a1b60 0x557c688a3f80 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16f7e9e7005b2afc server_cookie=d60b32819a1849fa in_seq=0 out_seq=0 2024-05-08T23:59:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f89f8ff9700 1 -- 172.21.15.12:0/175601922 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c688577b0 con 0x557c688a1b60 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f89f8ff9700 10 client.18987 handle_command_reply: tid=0 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 client.18987 shutdown 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 -- 172.21.15.12:0/175601922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 msgr2=0x557c68894190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68894190 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009210 tx=0x7f8a0c011b40 comp rx=0 tx=0).stop 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 -- 172.21.15.12:0/175601922 shutdown_connections 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557c688a1b60 0x557c688a3f80 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68891b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68894190 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/175601922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c688914d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f89f8ff9700 0 client.18987 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 -- 172.21.15.12:0/175601922 >> 172.21.15.12:0/175601922 conn(0x557c687e61b0 msgr2=0x557c68886f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 -- 172.21.15.12:0/175601922 shutdown_connections 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 1 -- 172.21.15.12:0/175601922 wait complete. 2024-05-08T23:59:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.044+0000 7f8a239e7b80 20 client.18987 trim_cache size 0 max 16384 2024-05-08T23:59:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.060+0000 7f8a239e7b80 1 Processor -- start 2024-05-08T23:59:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a239e7b80 1 -- start start 2024-05-08T23:59:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68882910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68890dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68891310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c68892430 con 0x557c68888480 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6887aa40 con 0x557c6888e270 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c688882e0 con 0x557c6888d950 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a155c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68882910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a14dc8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68890dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a155c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68882910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37362/0 (socket says 172.21.15.12:37362) 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a14dc8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68890dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47346/0 (socket says 172.21.15.12:47346) 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a155c9700 1 -- 172.21.15.12:0/60146531 learned_addr learned my addr 172.21.15.12:0/60146531 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a15dca700 1 --2- 172.21.15.12:0/60146531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68891310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a155c9700 1 -- 172.21.15.12:0/60146531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 msgr2=0x557c68890dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a155c9700 1 --2- 172.21.15.12:0/60146531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68890dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a155c9700 1 -- 172.21.15.12:0/60146531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 msgr2=0x557c68891310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a155c9700 1 --2- 172.21.15.12:0/60146531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68891310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a155c9700 1 -- 172.21.15.12:0/60146531 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c68605b00 con 0x557c68888480 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a14dc8700 1 --2- 172.21.15.12:0/60146531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68890dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.061+0000 7f8a15dca700 1 --2- 172.21.15.12:0/60146531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68891310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.062+0000 7f8a155c9700 1 --2- 172.21.15.12:0/60146531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68882910 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c016040 tx=0x7f8a0c011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2114ac00a3fe9ee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.062+0000 7f89fbfff700 1 -- 172.21.15.12:0/60146531 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c026500 con 0x557c68888480 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.062+0000 7f89fbfff700 1 -- 172.21.15.12:0/60146531 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0c03c030 con 0x557c68888480 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.062+0000 7f89fbfff700 1 -- 172.21.15.12:0/60146531 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c003b00 con 0x557c68888480 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 -- 172.21.15.12:0/60146531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 msgr2=0x557c68882910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/60146531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68882910 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c016040 tx=0x7f8a0c011e50 comp rx=0 tx=0).stop 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 -- 172.21.15.12:0/60146531 shutdown_connections 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/60146531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c6888e270 0x557c68891310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/60146531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c68888480 0x557c68882910 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/60146531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68890dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 -- 172.21.15.12:0/60146531 >> 172.21.15.12:0/60146531 conn(0x557c687e61b0 msgr2=0x557c6889d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 -- 172.21.15.12:0/60146531 shutdown_connections 2024-05-08T23:59:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 -- 172.21.15.12:0/60146531 wait complete. 2024-05-08T23:59:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 -- start start 2024-05-08T23:59:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.063+0000 7f8a239e7b80 1 Processor -- start 2024-05-08T23:59:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c68888480 0x557c68896bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68897130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a239e7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888e270 0x557c68897670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c688f7020 con 0x557c6888e270 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c688f7160 con 0x557c68888480 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a239e7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c688f72a0 con 0x557c6888d950 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a239e7b80 10 client.0 _ll_register_callbacks cb 0x557c6877dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a155c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c68888480 0x557c68896bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a15dca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888e270 0x557c68897670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a155c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c68888480 0x557c68896bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:36832/0 (socket says 172.21.15.12:36832) 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a155c9700 1 -- 172.21.15.12:0/2388644713 learned_addr learned my addr 172.21.15.12:0/2388644713 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a14dc8700 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68897130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a155c9700 1 -- 172.21.15.12:0/2388644713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 msgr2=0x557c68897130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a155c9700 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68897130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a155c9700 1 -- 172.21.15.12:0/2388644713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888e270 msgr2=0x557c68897670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a155c9700 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888e270 0x557c68897670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a155c9700 1 -- 172.21.15.12:0/2388644713 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a00009b60 con 0x557c68888480 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a15dca700 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888e270 0x557c68897670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.064+0000 7f8a14dc8700 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68897130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f8a155c9700 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c68888480 0x557c68896bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c042040 tx=0x7f8a0c005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15b4c1e9c57df5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f89f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f89f8ff9700 1 -- 172.21.15.12:0/2388644713 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c026740 con 0x557c68888480 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f89f8ff9700 1 -- 172.21.15.12:0/2388644713 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8a0c03d080 con 0x557c68888480 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f8a239e7b80 1 -- 172.21.15.12:0/2388644713 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a0c004020 con 0x557c68888480 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f89f8ff9700 1 -- 172.21.15.12:0/2388644713 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a0c0268a0 con 0x557c68888480 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f8a239e7b80 10 client.28706 fetch_fsmap learned FSMap version 53 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f8a239e7b80 1 -- 172.21.15.12:0/2388644713 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f8a1019e490 con 0x557c68888480 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f89f8ff9700 1 -- 172.21.15.12:0/2388644713 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0c055020 con 0x557c68888480 2024-05-08T23:59:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f89f8ff9700 1 -- 172.21.15.12:0/2388644713 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 53) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a0c00e690 con 0x557c68888480 2024-05-08T23:59:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f8a239e7b80 10 client.28706 fetch_fsmap finished waiting for FSMap version 53 2024-05-08T23:59:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f8a239e7b80 10 client.28706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f8a239e7b80 20 client.28706 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557c688f84f0 0x557c688fa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f8a239e7b80 4 client.28706 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:59:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.065+0000 7f8a239e7b80 1 -- 172.21.15.12:0/2388644713 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c687d0e60 con 0x557c688f84f0 2024-05-08T23:59:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.066+0000 7f8a14dc8700 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557c688f84f0 0x557c688fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.066+0000 7f89f8ff9700 10 client.28706 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.066+0000 7f8a14dc8700 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557c688f84f0 0x557c688fa8e0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622c6c5c1c4c138a server_cookie=25513cf3dbace276 in_seq=0 out_seq=0 2024-05-08T23:59:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f89f8ff9700 1 -- 172.21.15.12:0/2388644713 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c687d0e60 con 0x557c688f84f0 2024-05-08T23:59:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f89f8ff9700 10 client.28706 handle_command_reply: tid=0 2024-05-08T23:59:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 client.28706 shutdown 2024-05-08T23:59:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 -- 172.21.15.12:0/2388644713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c68888480 msgr2=0x557c68896bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c68888480 0x557c68896bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c042040 tx=0x7f8a0c005ce0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 -- 172.21.15.12:0/2388644713 shutdown_connections 2024-05-08T23:59:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557c6888e270 0x557c68897670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557c688f84f0 0x557c688fa8e0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557c68888480 0x557c68896bf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 --2- 172.21.15.12:0/2388644713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c6888d950 0x557c68897130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f89f8ff9700 0 client.28706 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 -- 172.21.15.12:0/2388644713 >> 172.21.15.12:0/2388644713 conn(0x557c687e61b0 msgr2=0x557c6888b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 -- 172.21.15.12:0/2388644713 shutdown_connections 2024-05-08T23:59:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.127+0000 7f8a239e7b80 1 -- 172.21.15.12:0/2388644713 wait complete. 2024-05-08T23:59:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.128+0000 7f8a239e7b80 20 client.28706 trim_cache size 0 max 16384 2024-05-08T23:59:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:59:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:59:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:59:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:59:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.128+0000 7f8a1782d700 1 -- 172.21.15.12:0/4218517691 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89fc0a1310 msgr2=0x7f89fc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.128+0000 7f8a1782d700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89fc0a1310 0x7f89fc0a37d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009be0 tx=0x7f8a0c009260 comp rx=0 tx=0).stop 2024-05-08T23:59:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.128+0000 7f8a1782d700 1 -- 172.21.15.12:0/4218517691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a10105e30 msgr2=0x7f8a1019d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.128+0000 7f8a1782d700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a10105e30 0x7f8a1019d870 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800b070 tx=0x7f8a080078e0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.129+0000 7f8a1782d700 1 -- 172.21.15.12:0/4218517691 shutdown_connections 2024-05-08T23:59:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.129+0000 7f8a1782d700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89fc0a1310 0x7f89fc0a37d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.129+0000 7f8a1782d700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a10105e30 0x7f8a1019d870 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.129+0000 7f8a1782d700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a100682e0 0x7f8a101a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.129+0000 7f8a1782d700 1 --2- 172.21.15.12:0/4218517691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a10103df0 0x7f8a101a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.129+0000 7f8a1782d700 1 -- 172.21.15.12:0/4218517691 >> 172.21.15.12:0/4218517691 conn(0x7f8a10076f10 msgr2=0x7f8a100fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.129+0000 7f8a1782d700 1 -- 172.21.15.12:0/4218517691 shutdown_connections 2024-05-08T23:59:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:04.129+0000 7f8a1782d700 1 -- 172.21.15.12:0/4218517691 wait complete. 2024-05-08T23:59:04.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:04 smithi012 ceph-mon[42503]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T23:59:04.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:04 smithi012 ceph-mon[42503]: pgmap v403: 97 pgs: 97 active+clean; 214 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-05-08T23:59:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:04 smithi204 ceph-mon[47373]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T23:59:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:04 smithi204 ceph-mon[47373]: pgmap v403: 97 pgs: 97 active+clean; 214 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-05-08T23:59:04.762 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:59:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-08T23:59:04.762 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:59:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-08T23:59:04.763 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:59:04 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/2819561749 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5644047ac1a0 con 0x564404782c00 2024-05-08T23:59:04.763 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:59:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-08T23:59:04.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:04 smithi160 ceph-mon[46152]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T23:59:04.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:04 smithi160 ceph-mon[46152]: pgmap v403: 97 pgs: 97 active+clean; 214 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-05-08T23:59:04.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 697475 6.03 MB/sec execute 540 sec latency 18.425 ms 2024-05-08T23:59:05.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:05 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-08T23:59:05.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:05 smithi012 ceph-mon[42503]: osdmap e106: 12 total, 12 up, 12 in 2024-05-08T23:59:05.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:05 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-08T23:59:05.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:05 smithi204 ceph-mon[47373]: osdmap e106: 12 total, 12 up, 12 in 2024-05-08T23:59:05.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:05 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-08T23:59:05.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:05 smithi160 ceph-mon[46152]: osdmap e106: 12 total, 12 up, 12 in 2024-05-08T23:59:05.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 697732 6.02 MB/sec execute 541 sec latency 815.775 ms 2024-05-08T23:59:06.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:06 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/3248496694,v1:172.21.15.204:6837/3248496694] up:boot 2024-05-08T23:59:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:06 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:59:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:06 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:59:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:06 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:06.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:06 smithi204 ceph-mon[47373]: pgmap v405: 97 pgs: 97 active+clean; 214 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.0 MiB/s wr, 135 op/s 2024-05-08T23:59:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:06 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/3248496694,v1:172.21.15.204:6837/3248496694] up:boot 2024-05-08T23:59:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:06 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:59:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:06 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:59:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:06 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:06.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:06 smithi160 ceph-mon[46152]: pgmap v405: 97 pgs: 97 active+clean; 214 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.0 MiB/s wr, 135 op/s 2024-05-08T23:59:06.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 699793 6.04 MB/sec execute 542 sec latency 951.394 ms 2024-05-08T23:59:06.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:06 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/3248496694,v1:172.21.15.204:6837/3248496694] up:boot 2024-05-08T23:59:06.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:06 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:59:06.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:06 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:59:06.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:06 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:06.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:06 smithi012 ceph-mon[42503]: pgmap v405: 97 pgs: 97 active+clean; 214 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.0 MiB/s wr, 135 op/s 2024-05-08T23:59:07.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:59:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.375+0000 7fa51fdf4700 1 -- 172.21.15.12:0/2304821932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa518073370 msgr2=0x7fa518073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.375+0000 7fa51fdf4700 1 --2- 172.21.15.12:0/2304821932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa518073370 0x7fa518073750 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa514009990 tx=0x7fa514009ca0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.375+0000 7fa51fdf4700 1 -- 172.21.15.12:0/2304821932 shutdown_connections 2024-05-08T23:59:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.375+0000 7fa51fdf4700 1 --2- 172.21.15.12:0/2304821932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51810d810 0x7fa51810fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.375+0000 7fa51fdf4700 1 --2- 172.21.15.12:0/2304821932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa518073370 0x7fa518073750 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.375+0000 7fa51fdf4700 1 --2- 172.21.15.12:0/2304821932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa518073c90 0x7fa51810d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.375+0000 7fa51fdf4700 1 -- 172.21.15.12:0/2304821932 >> 172.21.15.12:0/2304821932 conn(0x7fa5180fca80 msgr2=0x7fa5180feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.375+0000 7fa51fdf4700 1 -- 172.21.15.12:0/2304821932 shutdown_connections 2024-05-08T23:59:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.375+0000 7fa51fdf4700 1 -- 172.21.15.12:0/2304821932 wait complete. 2024-05-08T23:59:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51fdf4700 1 Processor -- start 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51fdf4700 1 -- start start 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51fdf4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa518073370 0x7fa5181996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51fdf4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa518073c90 0x7fa518199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51fdf4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51810d810 0x7fa51819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51fdf4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51819e5d0 con 0x7fa518073c90 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51fdf4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51819e740 con 0x7fa518073370 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51fdf4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51819e900 con 0x7fa51810d810 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51d38f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa518073c90 0x7fa518199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51db90700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa518073370 0x7fa5181996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51d38f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa518073c90 0x7fa518199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37400/0 (socket says 172.21.15.12:37400) 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51d38f700 1 -- 172.21.15.12:0/4289398938 learned_addr learned my addr 172.21.15.12:0/4289398938 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.376+0000 7fa51e391700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51810d810 0x7fa51819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa51db90700 1 -- 172.21.15.12:0/4289398938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51810d810 msgr2=0x7fa51819df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa51db90700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51810d810 0x7fa51819df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa51db90700 1 -- 172.21.15.12:0/4289398938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa518073c90 msgr2=0x7fa518199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa51db90700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa518073c90 0x7fa518199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa51db90700 1 -- 172.21.15.12:0/4289398938 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5140095f0 con 0x7fa518073370 2024-05-08T23:59:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa51db90700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa518073370 0x7fa5181996b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa5140098e0 tx=0x7fa514005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa50effd700 1 -- 172.21.15.12:0/4289398938 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51401d070 con 0x7fa518073370 2024-05-08T23:59:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa50effd700 1 -- 172.21.15.12:0/4289398938 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa514022470 con 0x7fa518073370 2024-05-08T23:59:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa51fdf4700 1 -- 172.21.15.12:0/4289398938 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51819eb80 con 0x7fa518073370 2024-05-08T23:59:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa50effd700 1 -- 172.21.15.12:0/4289398938 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514017650 con 0x7fa518073370 2024-05-08T23:59:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.377+0000 7fa51fdf4700 1 -- 172.21.15.12:0/4289398938 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181a6420 con 0x7fa518073370 2024-05-08T23:59:07.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.380+0000 7fa50effd700 1 -- 172.21.15.12:0/4289398938 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa5140225e0 con 0x7fa518073370 2024-05-08T23:59:07.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.380+0000 7fa50effd700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5040a1290 0x7fa5040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.380+0000 7fa51d38f700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5040a1290 0x7fa5040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.380+0000 7fa50effd700 1 -- 172.21.15.12:0/4289398938 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa5140ce030 con 0x7fa518073370 2024-05-08T23:59:07.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.381+0000 7fa51d38f700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5040a1290 0x7fa5040a3750 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa51819ac00 tx=0x7fa508009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:07.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.381+0000 7fa52bfaeb80 1 Processor -- start 2024-05-08T23:59:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.381+0000 7fa52bfaeb80 1 -- start start 2024-05-08T23:59:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.381+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d03e00 0x562c63d041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.381+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 0x562c63cfe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.381+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63cfe890 0x562c63cfecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.381+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63c55150 con 0x562c63cfe890 2024-05-08T23:59:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.381+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63c52b10 con 0x562c63d04720 2024-05-08T23:59:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.381+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63c4e400 con 0x562c63d03e00 2024-05-08T23:59:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51e391700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63cfe890 0x562c63cfecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51db90700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d03e00 0x562c63d041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51db90700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d03e00 0x562c63d041e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47380/0 (socket says 172.21.15.12:47380) 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51e391700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63cfe890 0x562c63cfecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37416/0 (socket says 172.21.15.12:37416) 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51db90700 1 -- 172.21.15.12:0/3353020624 learned_addr learned my addr 172.21.15.12:0/3353020624 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51d38f700 1 --2- 172.21.15.12:0/3353020624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 0x562c63cfe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51db90700 1 -- 172.21.15.12:0/3353020624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 msgr2=0x562c63cfe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51db90700 1 --2- 172.21.15.12:0/3353020624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 0x562c63cfe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51db90700 1 -- 172.21.15.12:0/3353020624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63cfe890 msgr2=0x562c63cfecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51db90700 1 --2- 172.21.15.12:0/3353020624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63cfe890 0x562c63cfecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51db90700 1 -- 172.21.15.12:0/3353020624 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c63acf3b0 con 0x562c63d03e00 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51d38f700 1 --2- 172.21.15.12:0/3353020624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 0x562c63cfe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa51db90700 1 --2- 172.21.15.12:0/3353020624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d03e00 0x562c63d041e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa51409b290 tx=0x7fa514009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f4c0eed97b7e30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa503fff700 1 -- 172.21.15.12:0/3353020624 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51409ba00 con 0x562c63d03e00 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa503fff700 1 -- 172.21.15.12:0/3353020624 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5140b0030 con 0x562c63d03e00 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.382+0000 7fa503fff700 1 -- 172.21.15.12:0/3353020624 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51409b560 con 0x562c63d03e00 2024-05-08T23:59:07.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.383+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/3353020624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d03e00 msgr2=0x562c63d041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.383+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/3353020624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d03e00 0x562c63d041e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa51409b290 tx=0x7fa514009580 comp rx=0 tx=0).stop 2024-05-08T23:59:07.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.383+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/3353020624 shutdown_connections 2024-05-08T23:59:07.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.383+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/3353020624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63cfe890 0x562c63cfecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.383+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/3353020624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d03e00 0x562c63d041e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.383+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/3353020624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 0x562c63cfe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.383+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/3353020624 >> 172.21.15.12:0/3353020624 conn(0x562c63c5c210 msgr2=0x562c63cf8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:07.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.383+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/3353020624 shutdown_connections 2024-05-08T23:59:07.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.383+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/3353020624 wait complete. 2024-05-08T23:59:07.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa52bfaeb80 1 -- start start 2024-05-08T23:59:07.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa52bfaeb80 1 Processor -- start 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63d0a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d03e00 0x562c63d079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 0x562c63d08110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63bea560 con 0x562c63d03e00 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63bea200 con 0x562c63d04720 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63ad16a0 con 0x562c63cfe890 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa52bfaeb80 10 client.0 _ll_register_callbacks cb 0x562c63bf3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa51db90700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63d0a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa51d38f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d03e00 0x562c63d079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa51db90700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63d0a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47392/0 (socket says 172.21.15.12:47392) 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa51db90700 1 -- 172.21.15.12:0/1411531125 learned_addr learned my addr 172.21.15.12:0/1411531125 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:07.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.384+0000 7fa51e391700 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 0x562c63d08110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa51db90700 1 -- 172.21.15.12:0/1411531125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 msgr2=0x562c63d08110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa51db90700 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 0x562c63d08110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa51db90700 1 -- 172.21.15.12:0/1411531125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d03e00 msgr2=0x562c63d079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa51db90700 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d03e00 0x562c63d079d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa51db90700 1 -- 172.21.15.12:0/1411531125 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181a6420 con 0x562c63cfe890 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa51db90700 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63d0a6e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa5140ce310 tx=0x7fa51409aaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2d4cd1416774868 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa5017fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa5017fa700 1 -- 172.21.15.12:0/1411531125 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51409b300 con 0x562c63cfe890 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa5017fa700 1 -- 172.21.15.12:0/1411531125 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa51409f130 con 0x562c63cfe890 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1411531125 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c63ba7270 con 0x562c63cfe890 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa5017fa700 1 -- 172.21.15.12:0/1411531125 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5140a36f0 con 0x562c63cfe890 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa52bfaeb80 10 client.28726 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.385+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1411531125 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x562c63a7bb60 con 0x562c63cfe890 2024-05-08T23:59:07.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa5017fa700 1 -- 172.21.15.12:0/1411531125 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c63ba7270 con 0x562c63cfe890 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa5017fa700 1 -- 172.21.15.12:0/1411531125 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa5140971d0 con 0x562c63cfe890 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa52bfaeb80 10 client.28726 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa52bfaeb80 10 client.28726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa52bfaeb80 20 client.28726 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c63d178e0 0x562c63d19cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa52bfaeb80 4 client.28726 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1411531125 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c63ccd810 con 0x562c63d178e0 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa51d38f700 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c63d178e0 0x562c63d19cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa51d38f700 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c63d178e0 0x562c63d19cd0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b6fd12de68fc31 server_cookie=15caeb81232b6e4b in_seq=0 out_seq=0 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.386+0000 7fa5017fa700 10 client.28726 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa5017fa700 1 -- 172.21.15.12:0/1411531125 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c63ccd810 con 0x562c63d178e0 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa5017fa700 10 client.28726 handle_command_reply: tid=0 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa52bfaeb80 1 client.28726 shutdown 2024-05-08T23:59:07.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1411531125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 msgr2=0x562c63d0a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63d0a6e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa5140ce310 tx=0x7fa51409aaa0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1411531125 shutdown_connections 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c63d178e0 0x562c63d19cd0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d04720 0x562c63d08110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63d0a6e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa5017fa700 0 client.28726 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/1411531125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d03e00 0x562c63d079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.387+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1411531125 >> 172.21.15.12:0/1411531125 conn(0x562c63c5c210 msgr2=0x562c63cf85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.388+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1411531125 shutdown_connections 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.388+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1411531125 wait complete. 2024-05-08T23:59:07.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.388+0000 7fa52bfaeb80 20 client.28726 trim_cache size 0 max 16384 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa52bfaeb80 1 Processor -- start 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa52bfaeb80 1 -- start start 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63cfe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d03e00 0x562c63cf8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d04720 0x562c63d01b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63d0ac90 con 0x562c63d04720 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63cf1040 con 0x562c63d03e00 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63cf3800 con 0x562c63cfe890 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa51e391700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d04720 0x562c63d01b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa51e391700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d04720 0x562c63d01b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37438/0 (socket says 172.21.15.12:37438) 2024-05-08T23:59:07.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa51e391700 1 -- 172.21.15.12:0/1965953017 learned_addr learned my addr 172.21.15.12:0/1965953017 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa51db90700 1 --2- 172.21.15.12:0/1965953017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63cfe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.408+0000 7fa51d38f700 1 --2- 172.21.15.12:0/1965953017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d03e00 0x562c63cf8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.409+0000 7fa51e391700 1 -- 172.21.15.12:0/1965953017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 msgr2=0x562c63cfe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.409+0000 7fa51e391700 1 --2- 172.21.15.12:0/1965953017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63cfe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.409+0000 7fa51e391700 1 -- 172.21.15.12:0/1965953017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d03e00 msgr2=0x562c63cf8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.409+0000 7fa51e391700 1 --2- 172.21.15.12:0/1965953017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d03e00 0x562c63cf8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.409+0000 7fa51e391700 1 -- 172.21.15.12:0/1965953017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c63c14560 con 0x562c63d04720 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.409+0000 7fa51db90700 1 --2- 172.21.15.12:0/1965953017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63cfe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.409+0000 7fa51e391700 1 --2- 172.21.15.12:0/1965953017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d04720 0x562c63d01b20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa5100118b0 tx=0x7fa510009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0d3d62f840c863 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.409+0000 7fa50cff9700 1 -- 172.21.15.12:0/1965953017 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51000e040 con 0x562c63d04720 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.409+0000 7fa50cff9700 1 -- 172.21.15.12:0/1965953017 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa51000f040 con 0x562c63d04720 2024-05-08T23:59:07.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.409+0000 7fa50cff9700 1 -- 172.21.15.12:0/1965953017 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510004730 con 0x562c63d04720 2024-05-08T23:59:07.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1965953017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d04720 msgr2=0x562c63d01b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/1965953017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d04720 0x562c63d01b20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa5100118b0 tx=0x7fa510009920 comp rx=0 tx=0).stop 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1965953017 shutdown_connections 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/1965953017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d04720 0x562c63d01b20 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/1965953017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63cfe890 0x562c63cfe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/1965953017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63d03e00 0x562c63cf8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1965953017 >> 172.21.15.12:0/1965953017 conn(0x562c63c5c210 msgr2=0x562c63cfec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1965953017 shutdown_connections 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/1965953017 wait complete. 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 -- start start 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.410+0000 7fa52bfaeb80 1 Processor -- start 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63cfe890 0x562c63d0eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d03e00 0x562c63d0f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa52bfaeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d04720 0x562c63cff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63d00000 con 0x562c63d03e00 2024-05-08T23:59:07.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63d00170 con 0x562c63cfe890 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa52bfaeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63d00330 con 0x562c63d04720 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa52bfaeb80 10 client.0 _ll_register_callbacks cb 0x562c63bf3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa51db90700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63cfe890 0x562c63d0eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa51e391700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d04720 0x562c63cff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa51db90700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63cfe890 0x562c63d0eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:36880/0 (socket says 172.21.15.12:36880) 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa51db90700 1 -- 172.21.15.12:0/2190086919 learned_addr learned my addr 172.21.15.12:0/2190086919 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa51e391700 1 -- 172.21.15.12:0/2190086919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63cfe890 msgr2=0x562c63d0eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa51e391700 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63cfe890 0x562c63d0eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa51e391700 1 -- 172.21.15.12:0/2190086919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d03e00 msgr2=0x562c63d0f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa51e391700 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d03e00 0x562c63d0f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.411+0000 7fa51e391700 1 -- 172.21.15.12:0/2190086919 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51407e070 con 0x562c63d04720 2024-05-08T23:59:07.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.412+0000 7fa51e391700 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d04720 0x562c63cff920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa51000b010 tx=0x7fa510003cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5884e589980a611 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:07.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.412+0000 7fa5017fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:59:07.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.412+0000 7fa5017fa700 1 -- 172.21.15.12:0/2190086919 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51000e070 con 0x562c63d04720 2024-05-08T23:59:07.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.412+0000 7fa5017fa700 1 -- 172.21.15.12:0/2190086919 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa510004740 con 0x562c63d04720 2024-05-08T23:59:07.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.412+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/2190086919 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5140b0030 con 0x562c63d04720 2024-05-08T23:59:07.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.412+0000 7fa5017fa700 1 -- 172.21.15.12:0/2190086919 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5100093a0 con 0x562c63d04720 2024-05-08T23:59:07.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.412+0000 7fa52bfaeb80 10 client.28732 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:07.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.412+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/2190086919 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fa510011530 con 0x562c63d04720 2024-05-08T23:59:07.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.412+0000 7fa5017fa700 1 -- 172.21.15.12:0/2190086919 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5140b0030 con 0x562c63d04720 2024-05-08T23:59:07.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.413+0000 7fa5017fa700 1 -- 172.21.15.12:0/2190086919 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa510017d10 con 0x562c63d04720 2024-05-08T23:59:07.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.413+0000 7fa52bfaeb80 10 client.28732 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.413+0000 7fa52bfaeb80 10 client.28732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.413+0000 7fa52bfaeb80 20 client.28732 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.413+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c63d6ebf0 0x562c63d71010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.413+0000 7fa52bfaeb80 4 client.28732 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.413+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/2190086919 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562c63c46ec0 con 0x562c63d6ebf0 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.413+0000 7fa51db90700 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c63d6ebf0 0x562c63d71010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.413+0000 7fa5017fa700 10 client.28732 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.413+0000 7fa51db90700 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c63d6ebf0 0x562c63d71010 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69bd113450ad6c3 server_cookie=15667f8ca2e61bb7 in_seq=0 out_seq=0 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa5017fa700 1 -- 172.21.15.12:0/2190086919 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562c63c46ec0 con 0x562c63d6ebf0 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa5017fa700 10 client.28732 handle_command_reply: tid=0 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 client.28732 shutdown 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/2190086919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d04720 msgr2=0x562c63cff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d04720 0x562c63cff920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa51000b010 tx=0x7fa510003cb0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/2190086919 shutdown_connections 2024-05-08T23:59:07.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63d04720 0x562c63cff920 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c63d6ebf0 0x562c63d71010 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c63cfe890 0x562c63d0eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 --2- 172.21.15.12:0/2190086919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c63d03e00 0x562c63d0f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/2190086919 >> 172.21.15.12:0/2190086919 conn(0x562c63c5c210 msgr2=0x562c63d01b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:07.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa5017fa700 0 client.28732 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:07.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/2190086919 shutdown_connections 2024-05-08T23:59:07.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 1 -- 172.21.15.12:0/2190086919 wait complete. 2024-05-08T23:59:07.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.414+0000 7fa52bfaeb80 20 client.28732 trim_cache size 0 max 16384 2024-05-08T23:59:07.415 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:59:07.415 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:59:07.416 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:59:07.416 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:59:07.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 -- 172.21.15.12:0/4289398938 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5040a1290 msgr2=0x7fa5040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5040a1290 0x7fa5040a3750 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa51819ac00 tx=0x7fa508009260 comp rx=0 tx=0).stop 2024-05-08T23:59:07.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 -- 172.21.15.12:0/4289398938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa518073370 msgr2=0x7fa5181996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa518073370 0x7fa5181996b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa5140098e0 tx=0x7fa514005740 comp rx=0 tx=0).stop 2024-05-08T23:59:07.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 -- 172.21.15.12:0/4289398938 shutdown_connections 2024-05-08T23:59:07.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa5040a1290 0x7fa5040a3750 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51810d810 0x7fa51819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa518073370 0x7fa5181996b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 --2- 172.21.15.12:0/4289398938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa518073c90 0x7fa518199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 -- 172.21.15.12:0/4289398938 >> 172.21.15.12:0/4289398938 conn(0x7fa5180fca80 msgr2=0x7fa5180fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:07.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 -- 172.21.15.12:0/4289398938 shutdown_connections 2024-05-08T23:59:07.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.415+0000 7fa51fdf4700 1 -- 172.21.15.12:0/4289398938 wait complete. 2024-05-08T23:59:07.424 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:59:07.424 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:59:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.584+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/2324053323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed01050a0 msgr2=0x7f9ed010cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.584+0000 7f9ed6a3d700 1 --2- 172.21.15.12:0/2324053323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed01050a0 0x7f9ed010cc20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009f90 tx=0x7f9ec800b330 comp rx=0 tx=0).stop 2024-05-08T23:59:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.584+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/2324053323 shutdown_connections 2024-05-08T23:59:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.584+0000 7f9ed6a3d700 1 --2- 172.21.15.12:0/2324053323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed01050a0 0x7f9ed010cc20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.584+0000 7f9ed6a3d700 1 --2- 172.21.15.12:0/2324053323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ed00fff80 0x7f9ed0100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.584+0000 7f9ed6a3d700 1 --2- 172.21.15.12:0/2324053323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed0100930 0x7f9ed0104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.584+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/2324053323 >> 172.21.15.12:0/2324053323 conn(0x7f9ed00fb840 msgr2=0x7f9ed00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.584+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/2324053323 shutdown_connections 2024-05-08T23:59:07.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.584+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/2324053323 wait complete. 2024-05-08T23:59:07.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.585+0000 7f9ed6a3d700 1 Processor -- start 2024-05-08T23:59:07.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.585+0000 7f9ed6a3d700 1 -- start start 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.585+0000 7f9ed6a3d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed00fff80 0x7f9ed0196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.585+0000 7f9ed6a3d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ed01050a0 0x7f9ed0196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.585+0000 7f9ed6a3d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed019a9d0 0x7f9ed019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.585+0000 7f9ed6a3d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019b450 con 0x7f9ed01050a0 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.585+0000 7f9ed6a3d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019b5c0 con 0x7f9ed019a9d0 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.585+0000 7f9ed6a3d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019b780 con 0x7f9ed00fff80 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ed01050a0 0x7f9ed0196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ed01050a0 0x7f9ed0196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37466/0 (socket says 172.21.15.12:37466) 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecf7fe700 1 -- 172.21.15.12:0/3510309788 learned_addr learned my addr 172.21.15.12:0/3510309788 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecffff700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed00fff80 0x7f9ed0196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ed4fda700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed019a9d0 0x7f9ed019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecf7fe700 1 -- 172.21.15.12:0/3510309788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed00fff80 msgr2=0x7f9ed0196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecf7fe700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed00fff80 0x7f9ed0196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecf7fe700 1 -- 172.21.15.12:0/3510309788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed019a9d0 msgr2=0x7f9ed019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecf7fe700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed019a9d0 0x7f9ed019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecf7fe700 1 -- 172.21.15.12:0/3510309788 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec00095f0 con 0x7f9ed01050a0 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ed4fda700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed019a9d0 0x7f9ed019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecffff700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed00fff80 0x7f9ed0196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecf7fe700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ed01050a0 0x7f9ed0196680 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400ea80 tx=0x7f9ec400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecd7fa700 1 -- 172.21.15.12:0/3510309788 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4013070 con 0x7f9ed01050a0 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecd7fa700 1 -- 172.21.15.12:0/3510309788 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ec400be80 con 0x7f9ed01050a0 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ecd7fa700 1 -- 172.21.15.12:0/3510309788 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4005350 con 0x7f9ed01050a0 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.586+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/3510309788 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec8009c40 con 0x7f9ed01050a0 2024-05-08T23:59:07.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.587+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/3510309788 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed00695a0 con 0x7f9ed01050a0 2024-05-08T23:59:07.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.588+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/3510309788 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed0101450 con 0x7f9ed01050a0 2024-05-08T23:59:07.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.589+0000 7f9ecd7fa700 1 -- 172.21.15.12:0/3510309788 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9ec4004930 con 0x7f9ed01050a0 2024-05-08T23:59:07.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.589+0000 7f9ecd7fa700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9eb80a1290 0x7f9eb80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:07.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.590+0000 7f9ecffff700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9eb80a1290 0x7f9eb80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:07.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.590+0000 7f9ecd7fa700 1 -- 172.21.15.12:0/3510309788 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9ec40ce910 con 0x7f9ed01050a0 2024-05-08T23:59:07.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.590+0000 7f9ecffff700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9eb80a1290 0x7f9eb80a3750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0005fd0 tx=0x7f9ec0009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:07.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.594+0000 7f9ecd7fa700 1 -- 172.21.15.12:0/3510309788 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9ec4097d70 con 0x7f9ed01050a0 2024-05-08T23:59:07.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.880+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/3510309788 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ed01973f0 con 0x7f9ed01050a0 2024-05-08T23:59:08.036 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:07 smithi160 ceph-mon[46152]: pgmap v406: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-05-08T23:59:08.037 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:07 smithi204 ceph-mon[47373]: pgmap v406: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-05-08T23:59:08.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.880+0000 7f9ecd7fa700 1 -- 172.21.15.12:0/3510309788 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+10879 (secure 0 0 0) 0x7f9ec4025020 con 0x7f9ed01050a0 2024-05-08T23:59:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:59:08.037 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:59:05.426915+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_28714":{"gid":28714,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3248496694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3248496694},{"type":"v1","addr":"172.21.15.204:6837","nonce":3248496694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:59:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/3510309788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9eb80a1290 msgr2=0x7f9eb80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9eb80a1290 0x7f9eb80a3750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0005fd0 tx=0x7f9ec0009310 comp rx=0 tx=0).stop 2024-05-08T23:59:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/3510309788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ed01050a0 msgr2=0x7f9ed0196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ed01050a0 0x7f9ed0196680 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400ea80 tx=0x7f9ec400ed90 comp rx=0 tx=0).stop 2024-05-08T23:59:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/3510309788 shutdown_connections 2024-05-08T23:59:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9eb80a1290 0x7f9eb80a3750 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed019a9d0 0x7f9ed019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ed00fff80 0x7f9ed0196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 --2- 172.21.15.12:0/3510309788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ed01050a0 0x7f9ed0196680 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/3510309788 >> 172.21.15.12:0/3510309788 conn(0x7f9ed00fb840 msgr2=0x7f9ed00fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/3510309788 shutdown_connections 2024-05-08T23:59:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:07.883+0000 7f9ed6a3d700 1 -- 172.21.15.12:0/3510309788 wait complete. 2024-05-08T23:59:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 55 2024-05-08T23:59:08.039 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:07 smithi012 ceph-mon[42503]: pgmap v406: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-05-08T23:59:08.040 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:59:08.041 INFO:tasks.workunit.client.0.smithi012.stdout: 1 701017 6.03 MB/sec execute 543 sec latency 37.215 ms 2024-05-08T23:59:08.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.195+0000 7fb2db367700 1 -- 172.21.15.12:0/1446807396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d4106280 msgr2=0x7fb2d410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.195+0000 7fb2db367700 1 --2- 172.21.15.12:0/1446807396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d4106280 0x7fb2d410de00 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00b0a0 tx=0x7fb2cc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.195+0000 7fb2db367700 1 -- 172.21.15.12:0/1446807396 shutdown_connections 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.195+0000 7fb2db367700 1 --2- 172.21.15.12:0/1446807396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d4106280 0x7fb2d410de00 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.195+0000 7fb2db367700 1 --2- 172.21.15.12:0/1446807396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d4101160 0x7fb2d4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.195+0000 7fb2db367700 1 --2- 172.21.15.12:0/1446807396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d4101b10 0x7fb2d4105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.195+0000 7fb2db367700 1 -- 172.21.15.12:0/1446807396 >> 172.21.15.12:0/1446807396 conn(0x7fb2d40fc9e0 msgr2=0x7fb2d40fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.195+0000 7fb2db367700 1 -- 172.21.15.12:0/1446807396 shutdown_connections 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.195+0000 7fb2db367700 1 -- 172.21.15.12:0/1446807396 wait complete. 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2db367700 1 Processor -- start 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2db367700 1 -- start start 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2db367700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d4101160 0x7fb2d41975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2db367700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d4106280 0x7fb2d4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2db367700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d419beb0 0x7fb2d419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2db367700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d419c930 con 0x7fb2d4106280 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2db367700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d419caa0 con 0x7fb2d4101160 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2db367700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d419cc60 con 0x7fb2d419beb0 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d8902700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d4106280 0x7fb2d4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d9103700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d4101160 0x7fb2d41975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d8902700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d4106280 0x7fb2d4197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37480/0 (socket says 172.21.15.12:37480) 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d8902700 1 -- 172.21.15.12:0/1091867117 learned_addr learned my addr 172.21.15.12:0/1091867117 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d9904700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d419beb0 0x7fb2d419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d9103700 1 -- 172.21.15.12:0/1091867117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d419beb0 msgr2=0x7fb2d419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d9103700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d419beb0 0x7fb2d419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d9103700 1 -- 172.21.15.12:0/1091867117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d4106280 msgr2=0x7fb2d4197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d9103700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d4106280 0x7fb2d4197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d9103700 1 -- 172.21.15.12:0/1091867117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc009d00 con 0x7fb2d4101160 2024-05-08T23:59:08.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.196+0000 7fb2d8902700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d4106280 0x7fb2d4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.197+0000 7fb2d9103700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d4101160 0x7fb2d41975d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000e9f0 tx=0x7fb2d000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.197+0000 7fb2ca7fc700 1 -- 172.21.15.12:0/1091867117 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000c9c0 con 0x7fb2d4101160 2024-05-08T23:59:08.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.197+0000 7fb2db367700 1 -- 172.21.15.12:0/1091867117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d41a41c0 con 0x7fb2d4101160 2024-05-08T23:59:08.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.197+0000 7fb2ca7fc700 1 -- 172.21.15.12:0/1091867117 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d0010430 con 0x7fb2d4101160 2024-05-08T23:59:08.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.197+0000 7fb2ca7fc700 1 -- 172.21.15.12:0/1091867117 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0018680 con 0x7fb2d4101160 2024-05-08T23:59:08.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.197+0000 7fb2db367700 1 -- 172.21.15.12:0/1091867117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d41a46f0 con 0x7fb2d4101160 2024-05-08T23:59:08.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.200+0000 7fb2ca7fc700 1 -- 172.21.15.12:0/1091867117 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb2d0010860 con 0x7fb2d4101160 2024-05-08T23:59:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.201+0000 7fb2ca7fc700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2c00a1290 0x7fb2c00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.201+0000 7fb2ca7fc700 1 -- 172.21.15.12:0/1091867117 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb2d0014070 con 0x7fb2d4101160 2024-05-08T23:59:08.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.201+0000 7fb2d8902700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2c00a1290 0x7fb2c00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.201+0000 7fb2e7521b80 1 Processor -- start 2024-05-08T23:59:08.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.201+0000 7fb2d8902700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2c00a1290 0x7fb2c00a3750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4198b20 tx=0x7fb2c4009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2e7521b80 1 -- start start 2024-05-08T23:59:08.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73893d150 con 0x55d7389e6890 2024-05-08T23:59:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73893ab10 con 0x55d7389ec720 2024-05-08T23:59:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d738936400 con 0x55d7389ebe00 2024-05-08T23:59:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9904700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9103700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9904700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37490/0 (socket says 172.21.15.12:37490) 2024-05-08T23:59:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9103700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389ec1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47448/0 (socket says 172.21.15.12:47448) 2024-05-08T23:59:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9904700 1 -- 172.21.15.12:0/908590920 learned_addr learned my addr 172.21.15.12:0/908590920 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d8902700 1 --2- 172.21.15.12:0/908590920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9103700 1 -- 172.21.15.12:0/908590920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 msgr2=0x55d7389e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9103700 1 --2- 172.21.15.12:0/908590920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9103700 1 -- 172.21.15.12:0/908590920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 msgr2=0x55d7389e6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9103700 1 --2- 172.21.15.12:0/908590920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9103700 1 -- 172.21.15.12:0/908590920 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7387b73b0 con 0x55d7389ebe00 2024-05-08T23:59:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.202+0000 7fb2d9904700 1 --2- 172.21.15.12:0/908590920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.203+0000 7fb2d9103700 1 --2- 172.21.15.12:0/908590920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389ec1e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00ca550 tx=0x7fb2d0099940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e0e2e2e11bb95d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.203+0000 7fb2bf7fe700 1 -- 172.21.15.12:0/908590920 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d009dd40 con 0x55d7389ebe00 2024-05-08T23:59:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.203+0000 7fb2bf7fe700 1 -- 172.21.15.12:0/908590920 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d00a2390 con 0x55d7389ebe00 2024-05-08T23:59:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.203+0000 7fb2bf7fe700 1 -- 172.21.15.12:0/908590920 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0099400 con 0x55d7389ebe00 2024-05-08T23:59:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.203+0000 7fb2e7521b80 1 -- 172.21.15.12:0/908590920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 msgr2=0x55d7389ec1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.203+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/908590920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389ec1e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00ca550 tx=0x7fb2d0099940 comp rx=0 tx=0).stop 2024-05-08T23:59:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.204+0000 7fb2e7521b80 1 -- 172.21.15.12:0/908590920 shutdown_connections 2024-05-08T23:59:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.204+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/908590920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.204+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/908590920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389ec1e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.204+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/908590920 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.204+0000 7fb2e7521b80 1 -- 172.21.15.12:0/908590920 >> 172.21.15.12:0/908590920 conn(0x55d738944210 msgr2=0x55d7389e0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.204+0000 7fb2e7521b80 1 -- 172.21.15.12:0/908590920 shutdown_connections 2024-05-08T23:59:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.204+0000 7fb2e7521b80 1 -- 172.21.15.12:0/908590920 wait complete. 2024-05-08T23:59:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.204+0000 7fb2e7521b80 1 -- start start 2024-05-08T23:59:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.204+0000 7fb2e7521b80 1 Processor -- start 2024-05-08T23:59:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7388d2560 con 0x55d7389e6890 2024-05-08T23:59:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7388d2200 con 0x55d7389ec720 2024-05-08T23:59:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7387b96a0 con 0x55d7389ebe00 2024-05-08T23:59:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2e7521b80 10 client.0 _ll_register_callbacks cb 0x55d7388dbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d9103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d8902700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d9103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389efb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37494/0 (socket says 172.21.15.12:37494) 2024-05-08T23:59:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d9103700 1 -- 172.21.15.12:0/727719969 learned_addr learned my addr 172.21.15.12:0/727719969 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d9904700 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d9103700 1 -- 172.21.15.12:0/727719969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 msgr2=0x55d7389f0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d9103700 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389f0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d9103700 1 -- 172.21.15.12:0/727719969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 msgr2=0x55d7389f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d9103700 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d9103700 1 -- 172.21.15.12:0/727719969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d410e670 con 0x55d7389e6890 2024-05-08T23:59:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d8902700 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2d9103700 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389efb20 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000e9c0 tx=0x7fb2d0010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d197ac2e32f6b6c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.205+0000 7fb2bcff9700 1 -- 172.21.15.12:0/727719969 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00b6070 con 0x55d7389e6890 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2bcff9700 1 -- 172.21.15.12:0/727719969 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb2d00982b0 con 0x55d7389e6890 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2e7521b80 1 -- 172.21.15.12:0/727719969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d73888f270 con 0x55d7389e6890 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2bcff9700 1 -- 172.21.15.12:0/727719969 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2d00b82b0 con 0x55d7389e6890 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2e7521b80 10 client.19032 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2e7521b80 1 -- 172.21.15.12:0/727719969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fb2d41a46f0 con 0x55d7389e6890 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2bcff9700 1 -- 172.21.15.12:0/727719969 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d73888f270 con 0x55d7389e6890 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2bcff9700 1 -- 172.21.15.12:0/727719969 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb2d000cce0 con 0x55d7389e6890 2024-05-08T23:59:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2e7521b80 10 client.19032 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2e7521b80 10 client.19032 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:59:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2e7521b80 20 client.19032 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d7389ffc90 0x55d738a02080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2e7521b80 4 client.19032 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2e7521b80 1 -- 172.21.15.12:0/727719969 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7389b5810 con 0x55d7389ffc90 2024-05-08T23:59:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.206+0000 7fb2d8902700 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d7389ffc90 0x55d738a02080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.207+0000 7fb2bcff9700 10 client.19032 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:59:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.207+0000 7fb2d8902700 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d7389ffc90 0x55d738a02080 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f04b8ae11deb10a8 server_cookie=8a837ade51e377cb in_seq=0 out_seq=0 2024-05-08T23:59:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.207+0000 7fb2bcff9700 1 -- 172.21.15.12:0/727719969 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7389b5810 con 0x55d7389ffc90 2024-05-08T23:59:08.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.207+0000 7fb2bcff9700 10 client.19032 handle_command_reply: tid=0 2024-05-08T23:59:08.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 client.19032 shutdown 2024-05-08T23:59:08.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 -- 172.21.15.12:0/727719969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 msgr2=0x55d7389efb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389efb20 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000e9c0 tx=0x7fb2d0010bd0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 -- 172.21.15.12:0/727719969 shutdown_connections 2024-05-08T23:59:08.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389efb20 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d7389ffc90 0x55d738a02080 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/727719969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 -- 172.21.15.12:0/727719969 >> 172.21.15.12:0/727719969 conn(0x55d738944210 msgr2=0x55d7389e05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2bcff9700 0 client.19032 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:59:08.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 -- 172.21.15.12:0/727719969 shutdown_connections 2024-05-08T23:59:08.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 1 -- 172.21.15.12:0/727719969 wait complete. 2024-05-08T23:59:08.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.208+0000 7fb2e7521b80 20 client.19032 trim_cache size 0 max 16384 2024-05-08T23:59:08.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2e7521b80 1 Processor -- start 2024-05-08T23:59:08.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2e7521b80 1 -- start start 2024-05-08T23:59:08.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7389e6740 con 0x55d7389e6890 2024-05-08T23:59:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d738a07250 con 0x55d7389ec720 2024-05-08T23:59:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7389d90b0 con 0x55d7389ebe00 2024-05-08T23:59:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2d9103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2d9103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37500/0 (socket says 172.21.15.12:37500) 2024-05-08T23:59:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2d9103700 1 -- 172.21.15.12:0/638223563 learned_addr learned my addr 172.21.15.12:0/638223563 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2d8902700 1 --2- 172.21.15.12:0/638223563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2d9904700 1 --2- 172.21.15.12:0/638223563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2d9103700 1 -- 172.21.15.12:0/638223563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 msgr2=0x55d7389e0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2d9103700 1 --2- 172.21.15.12:0/638223563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389e0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2d9103700 1 -- 172.21.15.12:0/638223563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 msgr2=0x55d7389f0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2d9103700 1 --2- 172.21.15.12:0/638223563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.222+0000 7fb2d9103700 1 -- 172.21.15.12:0/638223563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d738763b60 con 0x55d7389e6890 2024-05-08T23:59:08.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2d9103700 1 --2- 172.21.15.12:0/638223563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6c70 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c5b0 tx=0x7fb2d009de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32385cdbdbd02268 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2bffff700 1 -- 172.21.15.12:0/638223563 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d009a610 con 0x55d7389e6890 2024-05-08T23:59:08.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2bffff700 1 -- 172.21.15.12:0/638223563 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d00b82b0 con 0x55d7389e6890 2024-05-08T23:59:08.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2bffff700 1 -- 172.21.15.12:0/638223563 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00a1f00 con 0x55d7389e6890 2024-05-08T23:59:08.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2e7521b80 1 -- 172.21.15.12:0/638223563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 msgr2=0x55d7389e6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/638223563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6c70 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c5b0 tx=0x7fb2d009de00 comp rx=0 tx=0).stop 2024-05-08T23:59:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2e7521b80 1 -- 172.21.15.12:0/638223563 shutdown_connections 2024-05-08T23:59:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/638223563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/638223563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389e6c70 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/638223563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389e0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.223+0000 7fb2e7521b80 1 -- 172.21.15.12:0/638223563 >> 172.21.15.12:0/638223563 conn(0x55d738944210 msgr2=0x55d7389ee3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.224+0000 7fb2e7521b80 1 -- 172.21.15.12:0/638223563 shutdown_connections 2024-05-08T23:59:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.224+0000 7fb2e7521b80 1 -- 172.21.15.12:0/638223563 wait complete. 2024-05-08T23:59:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.224+0000 7fb2e7521b80 1 -- start start 2024-05-08T23:59:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.224+0000 7fb2e7521b80 1 Processor -- start 2024-05-08T23:59:08.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.224+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.224+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.224+0000 7fb2e7521b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.224+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d738a59500 con 0x55d7389e6890 2024-05-08T23:59:08.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.224+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d738a59640 con 0x55d7389ec720 2024-05-08T23:59:08.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.224+0000 7fb2e7521b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d738a597b0 con 0x55d7389ebe00 2024-05-08T23:59:08.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2e7521b80 10 client.0 _ll_register_callbacks cb 0x55d7388dbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:08.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d9103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d9103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37516/0 (socket says 172.21.15.12:37516) 2024-05-08T23:59:08.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d9103700 1 -- 172.21.15.12:0/1377435834 learned_addr learned my addr 172.21.15.12:0/1377435834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d8902700 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d9904700 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d9103700 1 -- 172.21.15.12:0/1377435834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 msgr2=0x55d7389f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d9103700 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d9103700 1 -- 172.21.15.12:0/1377435834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 msgr2=0x55d7389f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d9103700 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d9103700 1 -- 172.21.15.12:0/1377435834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d41a46f0 con 0x55d7389e6890 2024-05-08T23:59:08.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2d9103700 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389f62d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009800 tx=0x7fb2d0009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f49c36e4a567860 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:08.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2bcff9700 1 -- 172.21.15.12:0/1377435834 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00b6070 con 0x55d7389e6890 2024-05-08T23:59:08.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2bcff9700 1 -- 172.21.15.12:0/1377435834 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb2d00982b0 con 0x55d7389e6890 2024-05-08T23:59:08.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2e7521b80 1 -- 172.21.15.12:0/1377435834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2d00a2390 con 0x55d7389e6890 2024-05-08T23:59:08.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2bcff9700 1 -- 172.21.15.12:0/1377435834 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2d0095dd0 con 0x55d7389e6890 2024-05-08T23:59:08.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2e7521b80 10 client.19038 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:08.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.225+0000 7fb2e7521b80 1 -- 172.21.15.12:0/1377435834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55d7388fc560 con 0x55d7389e6890 2024-05-08T23:59:08.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2bcff9700 1 -- 172.21.15.12:0/1377435834 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d00a2390 con 0x55d7389e6890 2024-05-08T23:59:08.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2bcff9700 1 -- 172.21.15.12:0/1377435834 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb2d00b2ac0 con 0x55d7389e6890 2024-05-08T23:59:08.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2e7521b80 10 client.19038 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:08.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2e7521b80 10 client.19038 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:59:08.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2e7521b80 20 client.19038 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:08.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d738a5aad0 0x55d738a5cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2e7521b80 4 client.19038 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:59:08.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2e7521b80 1 -- 172.21.15.12:0/1377435834 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d73892eec0 con 0x55d738a5aad0 2024-05-08T23:59:08.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2d8902700 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d738a5aad0 0x55d738a5cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2d8902700 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d738a5aad0 0x55d738a5cec0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e77df3c3de294099 server_cookie=4fde8a51f71d4a3a in_seq=0 out_seq=0 2024-05-08T23:59:08.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.226+0000 7fb2bcff9700 10 client.19038 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:59:08.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2bcff9700 1 -- 172.21.15.12:0/1377435834 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d73892eec0 con 0x55d738a5aad0 2024-05-08T23:59:08.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2bcff9700 10 client.19038 handle_command_reply: tid=0 2024-05-08T23:59:08.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 client.19038 shutdown 2024-05-08T23:59:08.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 -- 172.21.15.12:0/1377435834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 msgr2=0x55d7389f62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389f62d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009800 tx=0x7fb2d0009ef0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 -- 172.21.15.12:0/1377435834 shutdown_connections 2024-05-08T23:59:08.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7389ec720 0x55d7389f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7389e6890 0x55d7389f62d0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d738a5aad0 0x55d738a5cec0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 --2- 172.21.15.12:0/1377435834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7389ebe00 0x55d7389f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2bcff9700 0 client.19038 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:59:08.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 -- 172.21.15.12:0/1377435834 >> 172.21.15.12:0/1377435834 conn(0x55d738944210 msgr2=0x55d7389eac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 -- 172.21.15.12:0/1377435834 shutdown_connections 2024-05-08T23:59:08.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 1 -- 172.21.15.12:0/1377435834 wait complete. 2024-05-08T23:59:08.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2e7521b80 20 client.19038 trim_cache size 0 max 16384 2024-05-08T23:59:08.237 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:59:08.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.227+0000 7fb2db367700 1 -- 172.21.15.12:0/1091867117 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2c00a1290 msgr2=0x7fb2c00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2c00a1290 0x7fb2c00a3750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4198b20 tx=0x7fb2c4009260 comp rx=0 tx=0).stop 2024-05-08T23:59:08.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 -- 172.21.15.12:0/1091867117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d4101160 msgr2=0x7fb2d41975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d4101160 0x7fb2d41975d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000e9f0 tx=0x7fb2d000ed00 comp rx=0 tx=0).stop 2024-05-08T23:59:08.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 -- 172.21.15.12:0/1091867117 shutdown_connections 2024-05-08T23:59:08.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2c00a1290 0x7fb2c00a3750 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d419beb0 0x7fb2d419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d4101160 0x7fb2d41975d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 --2- 172.21.15.12:0/1091867117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d4106280 0x7fb2d4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 -- 172.21.15.12:0/1091867117 >> 172.21.15.12:0/1091867117 conn(0x7fb2d40fc9e0 msgr2=0x7fb2d40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 -- 172.21.15.12:0/1091867117 shutdown_connections 2024-05-08T23:59:08.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.228+0000 7fb2db367700 1 -- 172.21.15.12:0/1091867117 wait complete. 2024-05-08T23:59:08.241 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:59:08.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.388+0000 7f455a521700 1 -- 172.21.15.12:0/741461133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4554100060 msgr2=0x7f4554100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.388+0000 7f455a521700 1 --2- 172.21.15.12:0/741461133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4554100060 0x7f4554100440 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f4544009990 tx=0x7f4544009ca0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.388+0000 7f455a521700 1 -- 172.21.15.12:0/741461133 shutdown_connections 2024-05-08T23:59:08.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.388+0000 7f455a521700 1 --2- 172.21.15.12:0/741461133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4554105160 0x7f455410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.388+0000 7f455a521700 1 --2- 172.21.15.12:0/741461133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4554100060 0x7f4554100440 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.388+0000 7f455a521700 1 --2- 172.21.15.12:0/741461133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554100a10 0x7f4554104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.388+0000 7f455a521700 1 -- 172.21.15.12:0/741461133 >> 172.21.15.12:0/741461133 conn(0x7f4554076f40 msgr2=0x7f4554077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.388+0000 7f455a521700 1 -- 172.21.15.12:0/741461133 shutdown_connections 2024-05-08T23:59:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.388+0000 7f455a521700 1 -- 172.21.15.12:0/741461133 wait complete. 2024-05-08T23:59:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.389+0000 7f455a521700 1 Processor -- start 2024-05-08T23:59:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.389+0000 7f455a521700 1 -- start start 2024-05-08T23:59:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.389+0000 7f455a521700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4554100060 0x7f4554101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.389+0000 7f455a521700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4554105160 0x7f4554102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.389+0000 7f455a521700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554102550 0x7f45541a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.389+0000 7f455a521700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541a5be0 con 0x7f4554105160 2024-05-08T23:59:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.389+0000 7f455a521700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541a5d50 con 0x7f4554100060 2024-05-08T23:59:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f455a521700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541a5f10 con 0x7f4554102550 2024-05-08T23:59:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4554105160 0x7f4554102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f4553fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4554100060 0x7f4554101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4554105160 0x7f4554102010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42276/0 (socket says 172.21.15.12:42276) 2024-05-08T23:59:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45537fe700 1 -- 172.21.15.12:0/1636226753 learned_addr learned my addr 172.21.15.12:0/1636226753 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f4558abe700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554102550 0x7f45541a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45537fe700 1 -- 172.21.15.12:0/1636226753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554102550 msgr2=0x7f45541a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45537fe700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554102550 0x7f45541a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45537fe700 1 -- 172.21.15.12:0/1636226753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4554100060 msgr2=0x7f4554101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45537fe700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4554100060 0x7f4554101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45537fe700 1 -- 172.21.15.12:0/1636226753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c009c40 con 0x7f4554105160 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f4553fff700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4554100060 0x7f4554101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45537fe700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4554105160 0x7f4554102010 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f454800e970 tx=0x7f454800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45517fa700 1 -- 172.21.15.12:0/1636226753 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548009790 con 0x7f4554105160 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f45517fa700 1 -- 172.21.15.12:0/1636226753 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f454800c7c0 con 0x7f4554105160 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.390+0000 7f455a521700 1 -- 172.21.15.12:0/1636226753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45440095f0 con 0x7f4554105160 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.391+0000 7f45517fa700 1 -- 172.21.15.12:0/1636226753 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45480054b0 con 0x7f4554105160 2024-05-08T23:59:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.391+0000 7f455a521700 1 -- 172.21.15.12:0/1636226753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45541a65c0 con 0x7f4554105160 2024-05-08T23:59:08.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.394+0000 7f45517fa700 1 -- 172.21.15.12:0/1636226753 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f454800c930 con 0x7f4554105160 2024-05-08T23:59:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.394+0000 7f45517fa700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f453c0a1310 0x7f453c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.394+0000 7f45517fa700 1 -- 172.21.15.12:0/1636226753 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4548014070 con 0x7f4554105160 2024-05-08T23:59:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.394+0000 7f4553fff700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f453c0a1310 0x7f453c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f45666dbb80 1 Processor -- start 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f45666dbb80 1 -- start start 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e0e00 0x55fe471e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e1720 0x55fe471db350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471db890 0x55fe471dbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f45666dbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe47132150 con 0x55fe471e0e00 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f45666dbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4712fb10 con 0x55fe471db890 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f45666dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4712b400 con 0x55fe471e1720 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f4553fff700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f453c0a1310 0x7f453c0a37d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f454400b2e0 tx=0x7f4544005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f4553fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e0e00 0x55fe471e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f4558abe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471db890 0x55fe471dbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f4553fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e0e00 0x55fe471e11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42286/0 (socket says 172.21.15.12:42286) 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f4553fff700 1 -- 172.21.15.12:0/2996892376 learned_addr learned my addr 172.21.15.12:0/2996892376 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f45537fe700 1 --2- 172.21.15.12:0/2996892376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e1720 0x55fe471db350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f4558abe700 1 -- 172.21.15.12:0/2996892376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e1720 msgr2=0x55fe471db350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f4558abe700 1 --2- 172.21.15.12:0/2996892376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e1720 0x55fe471db350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f4558abe700 1 -- 172.21.15.12:0/2996892376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e0e00 msgr2=0x55fe471e11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f4558abe700 1 --2- 172.21.15.12:0/2996892376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e0e00 0x55fe471e11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.395+0000 7f4558abe700 1 -- 172.21.15.12:0/2996892376 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe46fac3b0 con 0x55fe471db890 2024-05-08T23:59:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.396+0000 7f4558abe700 1 --2- 172.21.15.12:0/2996892376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471db890 0x55fe471dbcd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f454c00e650 tx=0x7f454c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9acd15f9318970f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.396+0000 7f453a7fc700 1 -- 172.21.15.12:0/2996892376 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c00e040 con 0x55fe471db890 2024-05-08T23:59:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.396+0000 7f453a7fc700 1 -- 172.21.15.12:0/2996892376 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f454c00f040 con 0x55fe471db890 2024-05-08T23:59:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.396+0000 7f453a7fc700 1 -- 172.21.15.12:0/2996892376 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c007500 con 0x55fe471db890 2024-05-08T23:59:08.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 -- 172.21.15.12:0/2996892376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471db890 msgr2=0x55fe471dbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 --2- 172.21.15.12:0/2996892376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471db890 0x55fe471dbcd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f454c00e650 tx=0x7f454c009920 comp rx=0 tx=0).stop 2024-05-08T23:59:08.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 -- 172.21.15.12:0/2996892376 shutdown_connections 2024-05-08T23:59:08.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 --2- 172.21.15.12:0/2996892376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471db890 0x55fe471dbcd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 --2- 172.21.15.12:0/2996892376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e0e00 0x55fe471e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 --2- 172.21.15.12:0/2996892376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e1720 0x55fe471db350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 -- 172.21.15.12:0/2996892376 >> 172.21.15.12:0/2996892376 conn(0x55fe47139210 msgr2=0x55fe471d5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 -- 172.21.15.12:0/2996892376 shutdown_connections 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 -- 172.21.15.12:0/2996892376 wait complete. 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 -- start start 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.397+0000 7f45666dbb80 1 Processor -- start 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 0x55fe471e5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e1720 0x55fe471e5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f45666dbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe470c7560 con 0x55fe471db890 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f45666dbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe470c7200 con 0x55fe471e0e00 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f45666dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe46fae6a0 con 0x55fe471e1720 2024-05-08T23:59:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f45666dbb80 10 client.0 _ll_register_callbacks cb 0x55fe470d0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f4553fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f4553fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471e4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42290/0 (socket says 172.21.15.12:42290) 2024-05-08T23:59:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f4553fff700 1 -- 172.21.15.12:0/1804118235 learned_addr learned my addr 172.21.15.12:0/1804118235 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f4558abe700 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e1720 0x55fe471e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f45537fe700 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 0x55fe471e5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f4553fff700 1 -- 172.21.15.12:0/1804118235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e1720 msgr2=0x55fe471e5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f4553fff700 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e1720 0x55fe471e5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f4553fff700 1 -- 172.21.15.12:0/1804118235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 msgr2=0x55fe471e5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f4553fff700 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 0x55fe471e5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f4553fff700 1 -- 172.21.15.12:0/1804118235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c00e330 con 0x55fe471db890 2024-05-08T23:59:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f45537fe700 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 0x55fe471e5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f4553fff700 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471e4bf0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f454401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e401487c909510f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.398+0000 7f45337fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45337fe700 1 -- 172.21.15.12:0/1804118235 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544005250 con 0x55fe471db890 2024-05-08T23:59:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45666dbb80 1 -- 172.21.15.12:0/1804118235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe47084270 con 0x55fe471db890 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45337fe700 1 -- 172.21.15.12:0/1804118235 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f454400fe70 con 0x55fe471db890 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45337fe700 1 -- 172.21.15.12:0/1804118235 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45440053b0 con 0x55fe471db890 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45666dbb80 10 client.19050 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45666dbb80 1 -- 172.21.15.12:0/1804118235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55fe46f58b60 con 0x55fe471db890 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45337fe700 1 -- 172.21.15.12:0/1804118235 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f454403c020 con 0x55fe471db890 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45337fe700 1 -- 172.21.15.12:0/1804118235 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f45440047d0 con 0x55fe471db890 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45666dbb80 10 client.19050 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45666dbb80 10 client.19050 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45666dbb80 20 client.19050 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45666dbb80 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55fe471f5230 0x55fe471f7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45666dbb80 4 client.19050 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.399+0000 7f45666dbb80 1 -- 172.21.15.12:0/1804118235 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe471aa810 con 0x55fe471f5230 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.400+0000 7f45537fe700 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55fe471f5230 0x55fe471f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.400+0000 7f45537fe700 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55fe471f5230 0x55fe471f7650 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bec05f1994489c5f server_cookie=d6a45e5c27cf440 in_seq=0 out_seq=0 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.400+0000 7f45337fe700 10 client.19050 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.400+0000 7f45337fe700 1 -- 172.21.15.12:0/1804118235 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe471aa810 con 0x55fe471f5230 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.400+0000 7f45337fe700 10 client.19050 handle_command_reply: tid=0 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.400+0000 7f45666dbb80 1 client.19050 shutdown 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 1 -- 172.21.15.12:0/1804118235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 msgr2=0x55fe471e4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471e4bf0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f454401a050 comp rx=0 tx=0).stop 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 1 -- 172.21.15.12:0/1804118235 shutdown_connections 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55fe471f5230 0x55fe471f7650 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e1720 0x55fe471e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471e4bf0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45337fe700 0 client.19050 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:59:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 1 --2- 172.21.15.12:0/1804118235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 0x55fe471e5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 1 -- 172.21.15.12:0/1804118235 >> 172.21.15.12:0/1804118235 conn(0x55fe47139210 msgr2=0x55fe471d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 1 -- 172.21.15.12:0/1804118235 shutdown_connections 2024-05-08T23:59:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 1 -- 172.21.15.12:0/1804118235 wait complete. 2024-05-08T23:59:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.401+0000 7f45666dbb80 20 client.19050 trim_cache size 0 max 16384 2024-05-08T23:59:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.415+0000 7f45666dbb80 1 Processor -- start 2024-05-08T23:59:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f45666dbb80 1 -- start start 2024-05-08T23:59:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471db350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e0e00 0x55fe471e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e1720 0x55fe471d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f45666dbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe471fc820 con 0x55fe471db890 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f45666dbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe471e3140 con 0x55fe471e1720 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f45666dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe471e1430 con 0x55fe471e0e00 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f4553fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471db350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f4558abe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e1720 0x55fe471d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f4553fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471db350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42298/0 (socket says 172.21.15.12:42298) 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f45537fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e0e00 0x55fe471e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f4553fff700 1 -- 172.21.15.12:0/1554561337 learned_addr learned my addr 172.21.15.12:0/1554561337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f4558abe700 1 -- 172.21.15.12:0/1554561337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e0e00 msgr2=0x55fe471e03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f4558abe700 1 --2- 172.21.15.12:0/1554561337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e0e00 0x55fe471e03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f4558abe700 1 -- 172.21.15.12:0/1554561337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 msgr2=0x55fe471db350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f4558abe700 1 --2- 172.21.15.12:0/1554561337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471db350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f4558abe700 1 -- 172.21.15.12:0/1554561337 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe470f1560 con 0x55fe471e1720 2024-05-08T23:59:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.416+0000 7f45537fe700 1 --2- 172.21.15.12:0/1554561337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e0e00 0x55fe471e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.417+0000 7f4558abe700 1 --2- 172.21.15.12:0/1554561337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e1720 0x55fe471d5d20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f454c00efd0 tx=0x7f454c009a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aef1c6ce4aabe42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.417+0000 7f453affd700 1 -- 172.21.15.12:0/1554561337 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c00e040 con 0x55fe471e1720 2024-05-08T23:59:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.417+0000 7f453affd700 1 -- 172.21.15.12:0/1554561337 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f454c004050 con 0x55fe471e1720 2024-05-08T23:59:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.417+0000 7f453affd700 1 -- 172.21.15.12:0/1554561337 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c004b60 con 0x55fe471e1720 2024-05-08T23:59:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 -- 172.21.15.12:0/1554561337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e1720 msgr2=0x55fe471d5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 --2- 172.21.15.12:0/1554561337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e1720 0x55fe471d5d20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f454c00efd0 tx=0x7f454c009a30 comp rx=0 tx=0).stop 2024-05-08T23:59:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 -- 172.21.15.12:0/1554561337 shutdown_connections 2024-05-08T23:59:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 --2- 172.21.15.12:0/1554561337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e1720 0x55fe471d5d20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 --2- 172.21.15.12:0/1554561337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471db890 0x55fe471db350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 --2- 172.21.15.12:0/1554561337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471e0e00 0x55fe471e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 -- 172.21.15.12:0/1554561337 >> 172.21.15.12:0/1554561337 conn(0x55fe47139210 msgr2=0x55fe471dfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 -- 172.21.15.12:0/1554561337 shutdown_connections 2024-05-08T23:59:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 -- 172.21.15.12:0/1554561337 wait complete. 2024-05-08T23:59:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 -- start start 2024-05-08T23:59:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.418+0000 7f45666dbb80 1 Processor -- start 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471db890 0x55fe471e9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 0x55fe471ea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f45666dbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e1720 0x55fe4724a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f45666dbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe471eabd0 con 0x55fe471e1720 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f45666dbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe471ead40 con 0x55fe471e0e00 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f45666dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4724ac80 con 0x55fe471db890 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f45666dbb80 10 client.0 _ll_register_callbacks cb 0x55fe470d0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f45537fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 0x55fe471ea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f4553fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471db890 0x55fe471e9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f45537fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 0x55fe471ea3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48478/0 (socket says 172.21.15.12:48478) 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f45537fe700 1 -- 172.21.15.12:0/2871941813 learned_addr learned my addr 172.21.15.12:0/2871941813 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f4558abe700 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e1720 0x55fe4724a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f4553fff700 1 -- 172.21.15.12:0/2871941813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 msgr2=0x55fe471ea3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f4553fff700 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 0x55fe471ea3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f4553fff700 1 -- 172.21.15.12:0/2871941813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e1720 msgr2=0x55fe4724a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f4553fff700 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e1720 0x55fe4724a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f4553fff700 1 -- 172.21.15.12:0/2871941813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45541a65c0 con 0x55fe471db890 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.419+0000 7f4558abe700 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e1720 0x55fe4724a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.420+0000 7f4553fff700 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471db890 0x55fe471e9eb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4544009ce0 tx=0x7f45440199a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac537d0c89282f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.420+0000 7f45337fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.420+0000 7f45337fe700 1 -- 172.21.15.12:0/2871941813 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400fe70 con 0x55fe471db890 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.420+0000 7f45666dbb80 1 -- 172.21.15.12:0/2871941813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f454c00f040 con 0x55fe471db890 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.420+0000 7f45337fe700 1 -- 172.21.15.12:0/2871941813 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f454403d080 con 0x55fe471db890 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.420+0000 7f45337fe700 1 -- 172.21.15.12:0/2871941813 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4544004d50 con 0x55fe471db890 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.420+0000 7f45666dbb80 10 client.28774 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.420+0000 7f45666dbb80 1 -- 172.21.15.12:0/2871941813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55fe46fac3b0 con 0x55fe471db890 2024-05-08T23:59:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.420+0000 7f45337fe700 1 -- 172.21.15.12:0/2871941813 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f454404e020 con 0x55fe471db890 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.420+0000 7f45337fe700 1 -- 172.21.15.12:0/2871941813 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f454403f040 con 0x55fe471db890 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45666dbb80 10 client.28774 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45666dbb80 10 client.28774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45666dbb80 20 client.28774 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45666dbb80 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55fe4724bfc0 0x55fe4724e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45666dbb80 4 client.28774 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45666dbb80 1 -- 172.21.15.12:0/2871941813 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe47123ec0 con 0x55fe4724bfc0 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45537fe700 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55fe4724bfc0 0x55fe4724e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45537fe700 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55fe4724bfc0 0x55fe4724e3e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b04c22916bb53496 server_cookie=798bc6fda93d02c in_seq=0 out_seq=0 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45337fe700 10 client.28774 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45337fe700 1 -- 172.21.15.12:0/2871941813 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe47123ec0 con 0x55fe4724bfc0 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45337fe700 10 client.28774 handle_command_reply: tid=0 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.421+0000 7f45666dbb80 1 client.28774 shutdown 2024-05-08T23:59:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 1 -- 172.21.15.12:0/2871941813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471db890 msgr2=0x55fe471e9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471db890 0x55fe471e9eb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4544009ce0 tx=0x7f45440199a0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 1 -- 172.21.15.12:0/2871941813 shutdown_connections 2024-05-08T23:59:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55fe4724bfc0 0x55fe4724e3e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe471e1720 0x55fe4724a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe471db890 0x55fe471e9eb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 1 --2- 172.21.15.12:0/2871941813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fe471e0e00 0x55fe471ea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45337fe700 0 client.28774 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:59:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 1 -- 172.21.15.12:0/2871941813 >> 172.21.15.12:0/2871941813 conn(0x55fe47139210 msgr2=0x55fe471e54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 1 -- 172.21.15.12:0/2871941813 shutdown_connections 2024-05-08T23:59:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 1 -- 172.21.15.12:0/2871941813 wait complete. 2024-05-08T23:59:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f45666dbb80 20 client.28774 trim_cache size 0 max 16384 2024-05-08T23:59:08.423 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:59:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f455a521700 1 -- 172.21.15.12:0/1636226753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f453c0a1310 msgr2=0x7f453c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f455a521700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f453c0a1310 0x7f453c0a37d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f454400b2e0 tx=0x7f4544005960 comp rx=0 tx=0).stop 2024-05-08T23:59:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f455a521700 1 -- 172.21.15.12:0/1636226753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4554105160 msgr2=0x7f4554102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f455a521700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4554105160 0x7f4554102010 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f454800e970 tx=0x7f454800ec80 comp rx=0 tx=0).stop 2024-05-08T23:59:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f455a521700 1 -- 172.21.15.12:0/1636226753 shutdown_connections 2024-05-08T23:59:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.422+0000 7f455a521700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f453c0a1310 0x7f453c0a37d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.423+0000 7f455a521700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554102550 0x7f45541a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.423+0000 7f455a521700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4554100060 0x7f4554101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.423+0000 7f455a521700 1 --2- 172.21.15.12:0/1636226753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4554105160 0x7f4554102010 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.423+0000 7f455a521700 1 -- 172.21.15.12:0/1636226753 >> 172.21.15.12:0/1636226753 conn(0x7f4554076f40 msgr2=0x7f4554068550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.423+0000 7f455a521700 1 -- 172.21.15.12:0/1636226753 shutdown_connections 2024-05-08T23:59:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.423+0000 7f455a521700 1 -- 172.21.15.12:0/1636226753 wait complete. 2024-05-08T23:59:08.431 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:59:08.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.584+0000 7f6059235700 1 -- 172.21.15.12:0/1251498733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6054100060 msgr2=0x7f6054100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.584+0000 7f6059235700 1 --2- 172.21.15.12:0/1251498733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6054100060 0x7f6054100440 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f6044009990 tx=0x7f6044009ca0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.584+0000 7f6059235700 1 -- 172.21.15.12:0/1251498733 shutdown_connections 2024-05-08T23:59:08.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.584+0000 7f6059235700 1 --2- 172.21.15.12:0/1251498733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6054105180 0x7f605410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.584+0000 7f6059235700 1 --2- 172.21.15.12:0/1251498733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6054100060 0x7f6054100440 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.584+0000 7f6059235700 1 --2- 172.21.15.12:0/1251498733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6054100a10 0x7f6054104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.584+0000 7f6059235700 1 -- 172.21.15.12:0/1251498733 >> 172.21.15.12:0/1251498733 conn(0x7f6054076fa0 msgr2=0x7f60540773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.584+0000 7f6059235700 1 -- 172.21.15.12:0/1251498733 shutdown_connections 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.584+0000 7f6059235700 1 -- 172.21.15.12:0/1251498733 wait complete. 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6059235700 1 Processor -- start 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6059235700 1 -- start start 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6059235700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6054100060 0x7f60541973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6059235700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6054105180 0x7f60541978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6059235700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f605419bc80 0x7f605419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6059235700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605419c700 con 0x7f6054105180 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6059235700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605419c870 con 0x7f605419bc80 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6059235700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605419ca30 con 0x7f6054100060 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6052d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6054100060 0x7f60541973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f605359e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f605419bc80 0x7f605419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f605259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6054105180 0x7f60541978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6052d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6054100060 0x7f60541973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:36330/0 (socket says 172.21.15.12:36330) 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.585+0000 7f6052d9d700 1 -- 172.21.15.12:0/1807126675 learned_addr learned my addr 172.21.15.12:0/1807126675 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f6052d9d700 1 -- 172.21.15.12:0/1807126675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f605419bc80 msgr2=0x7f605419c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f6052d9d700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f605419bc80 0x7f605419c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f6052d9d700 1 -- 172.21.15.12:0/1807126675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6054105180 msgr2=0x7f60541978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f6052d9d700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6054105180 0x7f60541978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f6052d9d700 1 -- 172.21.15.12:0/1807126675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440095f0 con 0x7f6054100060 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f605259c700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6054105180 0x7f60541978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f605359e700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f605419bc80 0x7f605419c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f6052d9d700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6054100060 0x7f60541973a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6044004760 tx=0x7f6044004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f603bfff700 1 -- 172.21.15.12:0/1807126675 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604401d070 con 0x7f6054100060 2024-05-08T23:59:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f603bfff700 1 -- 172.21.15.12:0/1807126675 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6044022470 con 0x7f6054100060 2024-05-08T23:59:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f6059235700 1 -- 172.21.15.12:0/1807126675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60541a3f30 con 0x7f6054100060 2024-05-08T23:59:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f603bfff700 1 -- 172.21.15.12:0/1807126675 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044017650 con 0x7f6054100060 2024-05-08T23:59:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.586+0000 7f6059235700 1 -- 172.21.15.12:0/1807126675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60541a4350 con 0x7f6054100060 2024-05-08T23:59:08.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.589+0000 7f603bfff700 1 -- 172.21.15.12:0/1807126675 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f60440177b0 con 0x7f6054100060 2024-05-08T23:59:08.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.589+0000 7f603bfff700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f603c0a1310 0x7f603c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.590+0000 7f605259c700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f603c0a1310 0x7f603c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.590+0000 7f603bfff700 1 -- 172.21.15.12:0/1807126675 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f60440cddb0 con 0x7f6054100060 2024-05-08T23:59:08.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.590+0000 7f605259c700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f603c0a1310 0x7f603c0a37d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f60541988f0 tx=0x7f6048009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.590+0000 7f60653efb80 1 Processor -- start 2024-05-08T23:59:08.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.590+0000 7f60653efb80 1 -- start start 2024-05-08T23:59:08.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f60653efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f60653efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c6f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f60653efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 0x563d1c6f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f60653efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c649150 con 0x563d1c6f7e00 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f60653efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c646b10 con 0x563d1c6f8720 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f60653efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c642400 con 0x563d1c6f2890 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f6052d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f605259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c6f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f6052d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6f81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42330/0 (socket says 172.21.15.12:42330) 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f6052d9d700 1 -- 172.21.15.12:0/2551672091 learned_addr learned my addr 172.21.15.12:0/2551672091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f605359e700 1 --2- 172.21.15.12:0/2551672091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 0x563d1c6f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f605259c700 1 -- 172.21.15.12:0/2551672091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 msgr2=0x563d1c6f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f605259c700 1 --2- 172.21.15.12:0/2551672091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 0x563d1c6f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f605259c700 1 -- 172.21.15.12:0/2551672091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 msgr2=0x563d1c6f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f605259c700 1 --2- 172.21.15.12:0/2551672091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f605259c700 1 -- 172.21.15.12:0/2551672091 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d1c4c33b0 con 0x563d1c6f8720 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f6052d9d700 1 --2- 172.21.15.12:0/2551672091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f605259c700 1 --2- 172.21.15.12:0/2551672091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c6f2350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f60480042f0 tx=0x7f6048014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68984d95abbba13f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.591+0000 7f60397fa700 1 -- 172.21.15.12:0/2551672091 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048023070 con 0x563d1c6f8720 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60397fa700 1 -- 172.21.15.12:0/2551672091 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f604801e0d0 con 0x563d1c6f8720 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60397fa700 1 -- 172.21.15.12:0/2551672091 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604801cca0 con 0x563d1c6f8720 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60653efb80 1 -- 172.21.15.12:0/2551672091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 msgr2=0x563d1c6f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60653efb80 1 --2- 172.21.15.12:0/2551672091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c6f2350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f60480042f0 tx=0x7f6048014000 comp rx=0 tx=0).stop 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60653efb80 1 -- 172.21.15.12:0/2551672091 shutdown_connections 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60653efb80 1 --2- 172.21.15.12:0/2551672091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 0x563d1c6f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60653efb80 1 --2- 172.21.15.12:0/2551672091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60653efb80 1 --2- 172.21.15.12:0/2551672091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c6f2350 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60653efb80 1 -- 172.21.15.12:0/2551672091 >> 172.21.15.12:0/2551672091 conn(0x563d1c650210 msgr2=0x563d1c6ecdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60653efb80 1 -- 172.21.15.12:0/2551672091 shutdown_connections 2024-05-08T23:59:08.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.592+0000 7f60653efb80 1 -- 172.21.15.12:0/2551672091 wait complete. 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.593+0000 7f60653efb80 1 -- start start 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.593+0000 7f60653efb80 1 Processor -- start 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.593+0000 7f60653efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f2890 0x563d1c6fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.593+0000 7f60653efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6fba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.593+0000 7f60653efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f8720 0x563d1c6fc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.593+0000 7f60653efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c5de560 con 0x563d1c6f7e00 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.593+0000 7f60653efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c5de200 con 0x563d1c6f2890 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.593+0000 7f60653efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c4c56a0 con 0x563d1c6f8720 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.593+0000 7f60653efb80 10 client.0 _ll_register_callbacks cb 0x563d1c5e7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f605259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f6052d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f2890 0x563d1c6fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f6052d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f2890 0x563d1c6fe7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48522/0 (socket says 172.21.15.12:48522) 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f605359e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f8720 0x563d1c6fc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f605259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6fba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42344/0 (socket says 172.21.15.12:42344) 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f6052d9d700 1 -- 172.21.15.12:0/1932898336 learned_addr learned my addr 172.21.15.12:0/1932898336 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f605259c700 1 -- 172.21.15.12:0/1932898336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f8720 msgr2=0x563d1c6fc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f605259c700 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f8720 0x563d1c6fc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f605259c700 1 -- 172.21.15.12:0/1932898336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f2890 msgr2=0x563d1c6fe7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f605259c700 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f2890 0x563d1c6fe7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f605259c700 1 -- 172.21.15.12:0/1932898336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60541a4350 con 0x563d1c6f7e00 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f605359e700 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f8720 0x563d1c6fc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f6052d9d700 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f2890 0x563d1c6fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f605259c700 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6fba40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x563d1c6eebd0 tx=0x7f6048005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f186f99c62c3069f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f6036ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f6036ff5700 1 -- 172.21.15.12:0/1932898336 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048023040 con 0x563d1c6f7e00 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f6036ff5700 1 -- 172.21.15.12:0/1932898336 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f604801c8a0 con 0x563d1c6f7e00 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.594+0000 7f60653efb80 1 -- 172.21.15.12:0/1932898336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d1c59b270 con 0x563d1c6f7e00 2024-05-08T23:59:08.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f6036ff5700 1 -- 172.21.15.12:0/1932898336 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f604801ca80 con 0x563d1c6f7e00 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f60653efb80 10 client.19068 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f60653efb80 1 -- 172.21.15.12:0/1932898336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f6048017070 con 0x563d1c6f7e00 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f6036ff5700 1 -- 172.21.15.12:0/1932898336 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f604803c080 con 0x563d1c6f7e00 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f6036ff5700 1 -- 172.21.15.12:0/1932898336 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f604802e030 con 0x563d1c6f7e00 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f60653efb80 10 client.19068 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f60653efb80 10 client.19068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f60653efb80 20 client.19068 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f60653efb80 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563d1c70c110 0x563d1c70e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f60653efb80 4 client.19068 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f60653efb80 1 -- 172.21.15.12:0/1932898336 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d1c6c1810 con 0x563d1c70c110 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.595+0000 7f6052d9d700 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563d1c70c110 0x563d1c70e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.596+0000 7f6052d9d700 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563d1c70c110 0x563d1c70e530 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=375b4eeb39c80ea8 server_cookie=5f8b6f9c786cc157 in_seq=0 out_seq=0 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.596+0000 7f6036ff5700 10 client.19068 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:08.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f6036ff5700 1 -- 172.21.15.12:0/1932898336 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d1c6c1810 con 0x563d1c70c110 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f6036ff5700 10 client.19068 handle_command_reply: tid=0 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 client.19068 shutdown 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 -- 172.21.15.12:0/1932898336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 msgr2=0x563d1c6fba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6fba40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x563d1c6eebd0 tx=0x7f6048005d10 comp rx=0 tx=0).stop 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 -- 172.21.15.12:0/1932898336 shutdown_connections 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f8720 0x563d1c6fc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563d1c70c110 0x563d1c70e530 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f2890 0x563d1c6fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 --2- 172.21.15.12:0/1932898336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c6fba40 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f6036ff5700 0 client.19068 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 -- 172.21.15.12:0/1932898336 >> 172.21.15.12:0/1932898336 conn(0x563d1c650210 msgr2=0x563d1c6ec5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 -- 172.21.15.12:0/1932898336 shutdown_connections 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 1 -- 172.21.15.12:0/1932898336 wait complete. 2024-05-08T23:59:08.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.597+0000 7f60653efb80 20 client.19068 trim_cache size 0 max 16384 2024-05-08T23:59:08.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f60653efb80 1 Processor -- start 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f60653efb80 1 -- start start 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f60653efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f2890 0x563d1c6ecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f60653efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f7e00 0x563d1c6fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f60653efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c6fe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f60653efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c6ee640 con 0x563d1c6f2890 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f60653efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c6e9dc0 con 0x563d1c6f8720 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f60653efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c6fced0 con 0x563d1c6f7e00 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f6052d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f2890 0x563d1c6ecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f605359e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c6fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f6052d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f2890 0x563d1c6ecd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42346/0 (socket says 172.21.15.12:42346) 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f6052d9d700 1 -- 172.21.15.12:0/3504403508 learned_addr learned my addr 172.21.15.12:0/3504403508 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.613+0000 7f605259c700 1 --2- 172.21.15.12:0/3504403508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f7e00 0x563d1c6fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.614+0000 7f605359e700 1 -- 172.21.15.12:0/3504403508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f7e00 msgr2=0x563d1c6fe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.614+0000 7f605359e700 1 --2- 172.21.15.12:0/3504403508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f7e00 0x563d1c6fe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.614+0000 7f605359e700 1 -- 172.21.15.12:0/3504403508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f2890 msgr2=0x563d1c6ecd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.614+0000 7f605359e700 1 --2- 172.21.15.12:0/3504403508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f2890 0x563d1c6ecd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.614+0000 7f605359e700 1 -- 172.21.15.12:0/3504403508 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d1c6f2f40 con 0x563d1c6f8720 2024-05-08T23:59:08.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.614+0000 7f6052d9d700 1 --2- 172.21.15.12:0/3504403508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f2890 0x563d1c6ecd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.614+0000 7f605359e700 1 --2- 172.21.15.12:0/3504403508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c6fe830 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f604c00e9d0 tx=0x7f604c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06dc7178ca04ba3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.614+0000 7f6039ffb700 1 -- 172.21.15.12:0/3504403508 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c004db0 con 0x563d1c6f8720 2024-05-08T23:59:08.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.614+0000 7f6039ffb700 1 -- 172.21.15.12:0/3504403508 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f604c00f040 con 0x563d1c6f8720 2024-05-08T23:59:08.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.614+0000 7f6039ffb700 1 -- 172.21.15.12:0/3504403508 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c007900 con 0x563d1c6f8720 2024-05-08T23:59:08.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 -- 172.21.15.12:0/3504403508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 msgr2=0x563d1c6fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 --2- 172.21.15.12:0/3504403508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c6fe830 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f604c00e9d0 tx=0x7f604c009920 comp rx=0 tx=0).stop 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 -- 172.21.15.12:0/3504403508 shutdown_connections 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 --2- 172.21.15.12:0/3504403508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c6fe830 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 --2- 172.21.15.12:0/3504403508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f2890 0x563d1c6ecd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 --2- 172.21.15.12:0/3504403508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f7e00 0x563d1c6fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 -- 172.21.15.12:0/3504403508 >> 172.21.15.12:0/3504403508 conn(0x563d1c650210 msgr2=0x563d1c707cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 -- 172.21.15.12:0/3504403508 shutdown_connections 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 -- 172.21.15.12:0/3504403508 wait complete. 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 -- start start 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.615+0000 7f60653efb80 1 Processor -- start 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f60653efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 0x563d1c703510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f60653efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c7011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f60653efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c701720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f60653efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c701dc0 con 0x563d1c6f7e00 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f60653efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c701f30 con 0x563d1c6f8720 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f60653efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1c7615f0 con 0x563d1c6f2890 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f60653efb80 10 client.0 _ll_register_callbacks cb 0x563d1c5e7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f6052d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 0x563d1c703510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f605259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c7011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f6052d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 0x563d1c703510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:36376/0 (socket says 172.21.15.12:36376) 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f605259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c7011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42354/0 (socket says 172.21.15.12:42354) 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f6052d9d700 1 -- 172.21.15.12:0/3214629990 learned_addr learned my addr 172.21.15.12:0/3214629990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f605359e700 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c701720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f6052d9d700 1 -- 172.21.15.12:0/3214629990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 msgr2=0x563d1c701720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f6052d9d700 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c701720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f6052d9d700 1 -- 172.21.15.12:0/3214629990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 msgr2=0x563d1c7011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f6052d9d700 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c7011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f6052d9d700 1 -- 172.21.15.12:0/3214629990 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c00e680 con 0x563d1c6f2890 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.616+0000 7f605259c700 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c7011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f6052d9d700 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 0x563d1c703510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f60440cb5c0 tx=0x7f60440a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=437b18327ef821ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f6036ff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f6036ff5700 1 -- 172.21.15.12:0/3214629990 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604409af60 con 0x563d1c6f2890 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f6036ff5700 1 -- 172.21.15.12:0/3214629990 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f604409ed90 con 0x563d1c6f2890 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f60653efb80 1 -- 172.21.15.12:0/3214629990 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f604801e0d0 con 0x563d1c6f2890 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f6036ff5700 1 -- 172.21.15.12:0/3214629990 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60440a3350 con 0x563d1c6f2890 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f60653efb80 10 client.28792 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f60653efb80 1 -- 172.21.15.12:0/3214629990 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x563d1c6e94e0 con 0x563d1c6f2890 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f6036ff5700 1 -- 172.21.15.12:0/3214629990 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f604407e080 con 0x563d1c6f2890 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f6036ff5700 1 -- 172.21.15.12:0/3214629990 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f604409a450 con 0x563d1c6f2890 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f60653efb80 10 client.28792 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f60653efb80 10 client.28792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f60653efb80 20 client.28792 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f60653efb80 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563d1c762900 0x563d1c764d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f60653efb80 4 client.28792 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f60653efb80 1 -- 172.21.15.12:0/3214629990 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d1c63aec0 con 0x563d1c762900 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.617+0000 7f605259c700 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563d1c762900 0x563d1c764d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f605259c700 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563d1c762900 0x563d1c764d20 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee8b6dd84dae4b8 server_cookie=2eaf496fb13ffbea in_seq=0 out_seq=0 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f6036ff5700 10 client.28792 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f6036ff5700 1 -- 172.21.15.12:0/3214629990 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563d1c63aec0 con 0x563d1c762900 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f6036ff5700 10 client.28792 handle_command_reply: tid=0 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 client.28792 shutdown 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 -- 172.21.15.12:0/3214629990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 msgr2=0x563d1c703510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 0x563d1c703510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f60440cb5c0 tx=0x7f60440a1040 comp rx=0 tx=0).stop 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 -- 172.21.15.12:0/3214629990 shutdown_connections 2024-05-08T23:59:08.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563d1c762900 0x563d1c764d20 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d1c6f8720 0x563d1c701720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d1c6f2890 0x563d1c703510 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f6036ff5700 0 client.28792 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 --2- 172.21.15.12:0/3214629990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d1c6f7e00 0x563d1c7011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 -- 172.21.15.12:0/3214629990 >> 172.21.15.12:0/3214629990 conn(0x563d1c650210 msgr2=0x563d1c6f6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 -- 172.21.15.12:0/3214629990 shutdown_connections 2024-05-08T23:59:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.618+0000 7f60653efb80 1 -- 172.21.15.12:0/3214629990 wait complete. 2024-05-08T23:59:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f60653efb80 20 client.28792 trim_cache size 0 max 16384 2024-05-08T23:59:08.619 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:59:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 -- 172.21.15.12:0/1807126675 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f603c0a1310 msgr2=0x7f603c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f603c0a1310 0x7f603c0a37d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f60541988f0 tx=0x7f6048009260 comp rx=0 tx=0).stop 2024-05-08T23:59:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 -- 172.21.15.12:0/1807126675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6054100060 msgr2=0x7f60541973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6054100060 0x7f60541973a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6044004760 tx=0x7f6044004790 comp rx=0 tx=0).stop 2024-05-08T23:59:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 -- 172.21.15.12:0/1807126675 shutdown_connections 2024-05-08T23:59:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f603c0a1310 0x7f603c0a37d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f605419bc80 0x7f605419c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6054100060 0x7f60541973a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 --2- 172.21.15.12:0/1807126675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6054105180 0x7f60541978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:08.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 -- 172.21.15.12:0/1807126675 >> 172.21.15.12:0/1807126675 conn(0x7f6054076fa0 msgr2=0x7f6054103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:08.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 -- 172.21.15.12:0/1807126675 shutdown_connections 2024-05-08T23:59:08.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:08.619+0000 7f6059235700 1 -- 172.21.15.12:0/1807126675 wait complete. 2024-05-08T23:59:08.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 702569 6.04 MB/sec execute 544 sec latency 294.016 ms 2024-05-08T23:59:08.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:08 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3510309788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:59:09.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:08 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3510309788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:59:09.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:08 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3510309788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:59:09.627 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:59:09.628 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:59:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.788+0000 7f335712c700 1 -- 172.21.15.12:0/2409681606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3350069dd0 msgr2=0x7f3350105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.788+0000 7f335712c700 1 --2- 172.21.15.12:0/2409681606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3350069dd0 0x7f3350105b70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f333c009990 tx=0x7f333c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.788+0000 7f335712c700 1 -- 172.21.15.12:0/2409681606 shutdown_connections 2024-05-08T23:59:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.788+0000 7f335712c700 1 --2- 172.21.15.12:0/2409681606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33501062a0 0x7f335010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.788+0000 7f335712c700 1 --2- 172.21.15.12:0/2409681606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3350069420 0x7f3350069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.788+0000 7f335712c700 1 --2- 172.21.15.12:0/2409681606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3350069dd0 0x7f3350105b70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.788+0000 7f335712c700 1 -- 172.21.15.12:0/2409681606 >> 172.21.15.12:0/2409681606 conn(0x7f3350076ba0 msgr2=0x7f3350076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.788+0000 7f335712c700 1 -- 172.21.15.12:0/2409681606 shutdown_connections 2024-05-08T23:59:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.788+0000 7f335712c700 1 -- 172.21.15.12:0/2409681606 wait complete. 2024-05-08T23:59:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f335712c700 1 Processor -- start 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f335712c700 1 -- start start 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f335712c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3350069420 0x7f3350197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f335712c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33501062a0 0x7f3350197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f335712c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f335019bce0 0x7f335019c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f335712c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335019c760 con 0x7f3350069420 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f335712c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335019c8d0 con 0x7f335019bce0 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f335712c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335019ca90 con 0x7f33501062a0 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f3354ec8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3350069420 0x7f3350197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f334ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33501062a0 0x7f3350197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f33556c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f335019bce0 0x7f335019c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f3354ec8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3350069420 0x7f3350197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42374/0 (socket says 172.21.15.12:42374) 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f334ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33501062a0 0x7f3350197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:36386/0 (socket says 172.21.15.12:36386) 2024-05-08T23:59:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f3354ec8700 1 -- 172.21.15.12:0/1670451226 learned_addr learned my addr 172.21.15.12:0/1670451226 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f3354ec8700 1 -- 172.21.15.12:0/1670451226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33501062a0 msgr2=0x7f3350197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f3354ec8700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33501062a0 0x7f3350197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f3354ec8700 1 -- 172.21.15.12:0/1670451226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f335019bce0 msgr2=0x7f335019c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f3354ec8700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f335019bce0 0x7f335019c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f3354ec8700 1 -- 172.21.15.12:0/1670451226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0095f0 con 0x7f3350069420 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.789+0000 7f33556c9700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f335019bce0 0x7f335019c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.790+0000 7f3354ec8700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3350069420 0x7f3350197400 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f334400e9a0 tx=0x7f334400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.790+0000 7f334dffb700 1 -- 172.21.15.12:0/1670451226 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344009790 con 0x7f3350069420 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.790+0000 7f334dffb700 1 -- 172.21.15.12:0/1670451226 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3344010430 con 0x7f3350069420 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.790+0000 7f335712c700 1 -- 172.21.15.12:0/1670451226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33501a3ff0 con 0x7f3350069420 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.790+0000 7f334dffb700 1 -- 172.21.15.12:0/1670451226 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344018680 con 0x7f3350069420 2024-05-08T23:59:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.790+0000 7f335712c700 1 -- 172.21.15.12:0/1670451226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33501a4470 con 0x7f3350069420 2024-05-08T23:59:09.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.792+0000 7f334dffb700 1 -- 172.21.15.12:0/1670451226 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f33440105a0 con 0x7f3350069420 2024-05-08T23:59:09.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.793+0000 7f334dffb700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33400a1290 0x7f33400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.793+0000 7f334dffb700 1 -- 172.21.15.12:0/1670451226 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3344014070 con 0x7f3350069420 2024-05-08T23:59:09.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.793+0000 7f334ffff700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33400a1290 0x7f33400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.794+0000 7f334ffff700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33400a1290 0x7f33400a3750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3350198950 tx=0x7f333c005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:09.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f33632e6b80 1 Processor -- start 2024-05-08T23:59:09.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f33632e6b80 1 -- start start 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb30670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb30bb0 0x55deefb2a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 0x55deefb2b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f33632e6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefa7d1d0 con 0x55deefb2ad20 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f33632e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefa7ab90 con 0x55deefb30bb0 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f33632e6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefa76480 con 0x55deefb30290 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f3354ec8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb30670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f33556c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 0x55deefb2b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f3354ec8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb30670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:36388/0 (socket says 172.21.15.12:36388) 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f3354ec8700 1 -- 172.21.15.12:0/976585394 learned_addr learned my addr 172.21.15.12:0/976585394 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f334ffff700 1 --2- 172.21.15.12:0/976585394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb30bb0 0x55deefb2a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f3354ec8700 1 -- 172.21.15.12:0/976585394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb30bb0 msgr2=0x55deefb2a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f3354ec8700 1 --2- 172.21.15.12:0/976585394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb30bb0 0x55deefb2a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f3354ec8700 1 -- 172.21.15.12:0/976585394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 msgr2=0x55deefb2b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:09.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f3354ec8700 1 --2- 172.21.15.12:0/976585394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 0x55deefb2b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.795+0000 7f3354ec8700 1 -- 172.21.15.12:0/976585394 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deef8f7430 con 0x55deefb30290 2024-05-08T23:59:09.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.796+0000 7f3354ec8700 1 --2- 172.21.15.12:0/976585394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb30670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f33440a1f10 tx=0x7f334409a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfdc5b50d8703e1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:09.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.796+0000 7f333affd700 1 -- 172.21.15.12:0/976585394 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440a2440 con 0x55deefb30290 2024-05-08T23:59:09.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.796+0000 7f333affd700 1 -- 172.21.15.12:0/976585394 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3344010880 con 0x55deefb30290 2024-05-08T23:59:09.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.796+0000 7f333affd700 1 -- 172.21.15.12:0/976585394 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344099740 con 0x55deefb30290 2024-05-08T23:59:09.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.797+0000 7f33632e6b80 1 -- 172.21.15.12:0/976585394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 msgr2=0x55deefb30670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.797+0000 7f33632e6b80 1 --2- 172.21.15.12:0/976585394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb30670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f33440a1f10 tx=0x7f334409a6c0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.797+0000 7f33632e6b80 1 -- 172.21.15.12:0/976585394 shutdown_connections 2024-05-08T23:59:09.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.797+0000 7f33632e6b80 1 --2- 172.21.15.12:0/976585394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 0x55deefb2b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.797+0000 7f33632e6b80 1 --2- 172.21.15.12:0/976585394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb30670 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.797+0000 7f33632e6b80 1 --2- 172.21.15.12:0/976585394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb30bb0 0x55deefb2a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.797+0000 7f33632e6b80 1 -- 172.21.15.12:0/976585394 >> 172.21.15.12:0/976585394 conn(0x55deefa84290 msgr2=0x55deefb20e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.797+0000 7f33632e6b80 1 -- 172.21.15.12:0/976585394 shutdown_connections 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.797+0000 7f33632e6b80 1 -- 172.21.15.12:0/976585394 wait complete. 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.797+0000 7f33632e6b80 1 -- start start 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f33632e6b80 1 Processor -- start 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 0x55deefb33f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb34460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb30bb0 0x55deefb349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f33632e6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefa125e0 con 0x55deefb2ad20 2024-05-08T23:59:09.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f33632e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefa12280 con 0x55deefb30bb0 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f33632e6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deef8f9720 con 0x55deefb30290 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f33632e6b80 10 client.0 _ll_register_callbacks cb 0x55deefa1bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f3354ec8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 0x55deefb33f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f334ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb34460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f334ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb34460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:36396/0 (socket says 172.21.15.12:36396) 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f3354ec8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 0x55deefb33f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42394/0 (socket says 172.21.15.12:42394) 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f3354ec8700 1 -- 172.21.15.12:0/2136952857 learned_addr learned my addr 172.21.15.12:0/2136952857 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.798+0000 7f33556c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb30bb0 0x55deefb349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.799+0000 7f3354ec8700 1 -- 172.21.15.12:0/2136952857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 msgr2=0x55deefb34460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.799+0000 7f3354ec8700 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb34460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.799+0000 7f3354ec8700 1 -- 172.21.15.12:0/2136952857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb30bb0 msgr2=0x55deefb349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.799+0000 7f3354ec8700 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb30bb0 0x55deefb349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.799+0000 7f3354ec8700 1 -- 172.21.15.12:0/2136952857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33501a4470 con 0x55deefb2ad20 2024-05-08T23:59:09.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.799+0000 7f3354ec8700 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 0x55deefb33f20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f33440964c0 tx=0x7f3344099740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4375ecc558408bbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.799+0000 7f332ffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.799+0000 7f332ffff700 1 -- 172.21.15.12:0/2136952857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440a2440 con 0x55deefb2ad20 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.799+0000 7f33632e6b80 1 -- 172.21.15.12:0/2136952857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55deef9cf2f0 con 0x55deefb2ad20 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.799+0000 7f332ffff700 1 -- 172.21.15.12:0/2136952857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3344094de0 con 0x55deefb2ad20 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f332ffff700 1 -- 172.21.15.12:0/2136952857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33440a1fb0 con 0x55deefb2ad20 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f33632e6b80 10 client.19086 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f33632e6b80 1 -- 172.21.15.12:0/2136952857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55deef8a3be0 con 0x55deefb2ad20 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f332ffff700 1 -- 172.21.15.12:0/2136952857 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55deef9cf2f0 con 0x55deefb2ad20 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f332ffff700 1 -- 172.21.15.12:0/2136952857 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3344099150 con 0x55deefb2ad20 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f33632e6b80 10 client.19086 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f33632e6b80 10 client.19086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f33632e6b80 20 client.19086 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55deefb44370 0x55deefb46760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f33632e6b80 4 client.19086 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f33632e6b80 1 -- 172.21.15.12:0/2136952857 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55deefaf5890 con 0x55deefb44370 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.800+0000 7f334ffff700 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55deefb44370 0x55deefb46760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.801+0000 7f332ffff700 10 client.19086 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.801+0000 7f334ffff700 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55deefb44370 0x55deefb46760 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=201421cd931e921b server_cookie=aa9ecb1193d363c3 in_seq=0 out_seq=0 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.801+0000 7f332ffff700 1 -- 172.21.15.12:0/2136952857 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55deefaf5890 con 0x55deefb44370 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f332ffff700 10 client.19086 handle_command_reply: tid=0 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f33632e6b80 1 client.19086 shutdown 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f33632e6b80 1 -- 172.21.15.12:0/2136952857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 msgr2=0x55deefb33f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 0x55deefb33f20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f33440964c0 tx=0x7f3344099740 comp rx=0 tx=0).stop 2024-05-08T23:59:09.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f33632e6b80 1 -- 172.21.15.12:0/2136952857 shutdown_connections 2024-05-08T23:59:09.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55deefb44370 0x55deefb46760 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb30bb0 0x55deefb349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb2ad20 0x55deefb33f20 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2136952857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb34460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f332ffff700 0 client.19086 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:09.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f33632e6b80 1 -- 172.21.15.12:0/2136952857 >> 172.21.15.12:0/2136952857 conn(0x55deefa84290 msgr2=0x55deefb20620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:09.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.802+0000 7f33632e6b80 1 -- 172.21.15.12:0/2136952857 shutdown_connections 2024-05-08T23:59:09.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.803+0000 7f33632e6b80 1 -- 172.21.15.12:0/2136952857 wait complete. 2024-05-08T23:59:09.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.803+0000 7f33632e6b80 20 client.19086 trim_cache size 0 max 16384 2024-05-08T23:59:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.822+0000 7f33632e6b80 1 Processor -- start 2024-05-08T23:59:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.822+0000 7f33632e6b80 1 -- start start 2024-05-08T23:59:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb2f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb20da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 0x55deefb347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f33632e6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefb226c0 con 0x55deefb30bb0 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f33632e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefb4b900 con 0x55deefb2ad20 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f33632e6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefb1b880 con 0x55deefb30290 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f33556c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 0x55deefb347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f3354ec8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb2f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f33556c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 0x55deefb347c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42406/0 (socket says 172.21.15.12:42406) 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f33556c9700 1 -- 172.21.15.12:0/1540654652 learned_addr learned my addr 172.21.15.12:0/1540654652 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f334ffff700 1 --2- 172.21.15.12:0/1540654652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb20da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f3354ec8700 1 -- 172.21.15.12:0/1540654652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 msgr2=0x55deefb20da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f3354ec8700 1 --2- 172.21.15.12:0/1540654652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb20da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f3354ec8700 1 -- 172.21.15.12:0/1540654652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 msgr2=0x55deefb347c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f3354ec8700 1 --2- 172.21.15.12:0/1540654652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 0x55deefb347c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f3354ec8700 1 -- 172.21.15.12:0/1540654652 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deefa3c5e0 con 0x55deefb2ad20 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f33556c9700 1 --2- 172.21.15.12:0/1540654652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 0x55deefb347c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.823+0000 7f3354ec8700 1 --2- 172.21.15.12:0/1540654652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb2f8e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f33440cac00 tx=0x7f334409deb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf31da134703d572 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.824+0000 7f333b7fe700 1 -- 172.21.15.12:0/1540654652 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334409a6c0 con 0x55deefb2ad20 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.824+0000 7f333b7fe700 1 -- 172.21.15.12:0/1540654652 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33440ade90 con 0x55deefb2ad20 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.824+0000 7f333b7fe700 1 -- 172.21.15.12:0/1540654652 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440a1fb0 con 0x55deefb2ad20 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.824+0000 7f33632e6b80 1 -- 172.21.15.12:0/1540654652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 msgr2=0x55deefb2f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.824+0000 7f33632e6b80 1 --2- 172.21.15.12:0/1540654652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb2f8e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f33440cac00 tx=0x7f334409deb0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.824+0000 7f33632e6b80 1 -- 172.21.15.12:0/1540654652 shutdown_connections 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.824+0000 7f33632e6b80 1 --2- 172.21.15.12:0/1540654652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 0x55deefb347c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.824+0000 7f33632e6b80 1 --2- 172.21.15.12:0/1540654652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb2f8e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.824+0000 7f33632e6b80 1 --2- 172.21.15.12:0/1540654652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb20da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.824+0000 7f33632e6b80 1 -- 172.21.15.12:0/1540654652 >> 172.21.15.12:0/1540654652 conn(0x55deefa84290 msgr2=0x55deefb253c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.825+0000 7f33632e6b80 1 -- 172.21.15.12:0/1540654652 shutdown_connections 2024-05-08T23:59:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.825+0000 7f33632e6b80 1 -- 172.21.15.12:0/1540654652 wait complete. 2024-05-08T23:59:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.825+0000 7f33632e6b80 1 -- start start 2024-05-08T23:59:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.825+0000 7f33632e6b80 1 Processor -- start 2024-05-08T23:59:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb3b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f33632e6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 0x55deefb3a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f33632e6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefb3ab00 con 0x55deefb30bb0 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f33632e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefb3ac70 con 0x55deefb2ad20 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f33632e6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deefb9a7e0 con 0x55deefb30290 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f33632e6b80 10 client.0 _ll_register_callbacks cb 0x55deefa1bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f3354ec8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb3b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f334ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f3354ec8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb3b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48600/0 (socket says 172.21.15.12:48600) 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f3354ec8700 1 -- 172.21.15.12:0/2398166795 learned_addr learned my addr 172.21.15.12:0/2398166795 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f334ffff700 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb39e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:36410/0 (socket says 172.21.15.12:36410) 2024-05-08T23:59:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f33556c9700 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 0x55deefb3a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f334ffff700 1 -- 172.21.15.12:0/2398166795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 msgr2=0x55deefb3b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f334ffff700 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb3b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f334ffff700 1 -- 172.21.15.12:0/2398166795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 msgr2=0x55deefb3a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f334ffff700 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 0x55deefb3a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f334ffff700 1 -- 172.21.15.12:0/2398166795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334800e6c0 con 0x55deefb30290 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f3354ec8700 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f334ffff700 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb39e30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f333c000c00 tx=0x7f333c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=190f8132ad98cd1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.826+0000 7f332ffff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f332ffff700 1 -- 172.21.15.12:0/2398166795 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c0360c0 con 0x55deefb30290 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f332ffff700 1 -- 172.21.15.12:0/2398166795 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f333c036220 con 0x55deefb30290 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f33632e6b80 1 -- 172.21.15.12:0/2398166795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3344010880 con 0x55deefb30290 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f332ffff700 1 -- 172.21.15.12:0/2398166795 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f333c0363e0 con 0x55deefb30290 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f33632e6b80 10 client.28810 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f33632e6b80 1 -- 172.21.15.12:0/2398166795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55deef8a3be0 con 0x55deefb30290 2024-05-08T23:59:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f332ffff700 1 -- 172.21.15.12:0/2398166795 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3344010880 con 0x55deefb30290 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f332ffff700 1 -- 172.21.15.12:0/2398166795 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f333c004cd0 con 0x55deefb30290 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f33632e6b80 10 client.28810 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f33632e6b80 10 client.28810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f33632e6b80 20 client.28810 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55deefb9bfe0 0x55deefb9e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f33632e6b80 4 client.28810 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f33632e6b80 1 -- 172.21.15.12:0/2398166795 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55deefa6ef40 con 0x55deefb9bfe0 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.827+0000 7f3354ec8700 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55deefb9bfe0 0x55deefb9e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.828+0000 7f332ffff700 10 client.28810 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.828+0000 7f3354ec8700 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55deefb9bfe0 0x55deefb9e400 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab148a2df6a837f9 server_cookie=e5704f72cb224c2 in_seq=0 out_seq=0 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f332ffff700 1 -- 172.21.15.12:0/2398166795 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55deefa6ef40 con 0x55deefb9bfe0 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f332ffff700 10 client.28810 handle_command_reply: tid=0 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 client.28810 shutdown 2024-05-08T23:59:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 -- 172.21.15.12:0/2398166795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 msgr2=0x55deefb39e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb39e30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f333c000c00 tx=0x7f333c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 -- 172.21.15.12:0/2398166795 shutdown_connections 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55deefb30bb0 0x55deefb3a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55deefb9bfe0 0x55deefb9e400 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55deefb2ad20 0x55deefb3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 --2- 172.21.15.12:0/2398166795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55deefb30290 0x55deefb39e30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f332ffff700 0 client.28810 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 -- 172.21.15.12:0/2398166795 >> 172.21.15.12:0/2398166795 conn(0x55deefa84290 msgr2=0x55deefb25c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 -- 172.21.15.12:0/2398166795 shutdown_connections 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 1 -- 172.21.15.12:0/2398166795 wait complete. 2024-05-08T23:59:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.829+0000 7f33632e6b80 20 client.28810 trim_cache size 0 max 16384 2024-05-08T23:59:09.830 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:59:09.830 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:59:09.830 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "229e1683-12c4-4f32-8653-39fa98c44359", 2024-05-08T23:59:09.830 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:59:09.830 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 -- 172.21.15.12:0/1670451226 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33400a1290 msgr2=0x7f33400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33400a1290 0x7f33400a3750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3350198950 tx=0x7f333c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 -- 172.21.15.12:0/1670451226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3350069420 msgr2=0x7f3350197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3350069420 0x7f3350197400 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f334400e9a0 tx=0x7f334400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 -- 172.21.15.12:0/1670451226 shutdown_connections 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33400a1290 0x7f33400a3750 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f335019bce0 0x7f335019c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3350069420 0x7f3350197400 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 --2- 172.21.15.12:0/1670451226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33501062a0 0x7f3350197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 -- 172.21.15.12:0/1670451226 >> 172.21.15.12:0/1670451226 conn(0x7f3350076ba0 msgr2=0x7f33500ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:09.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 -- 172.21.15.12:0/1670451226 shutdown_connections 2024-05-08T23:59:09.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.830+0000 7f335712c700 1 -- 172.21.15.12:0/1670451226 wait complete. 2024-05-08T23:59:09.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:59:09.871 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:09 smithi012 ceph-mon[42503]: pgmap v407: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-05-08T23:59:09.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 704043 6.04 MB/sec execute 545 sec latency 22.749 ms 2024-05-08T23:59:09.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.995+0000 7f0ce1052700 1 -- 172.21.15.12:0/3314830559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cdc068c90 msgr2=0x7f0cdc105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.995+0000 7f0ce1052700 1 --2- 172.21.15.12:0/3314830559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cdc068c90 0x7f0cdc105b60 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00099c0 tx=0x7f0cd0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.995+0000 7f0ce1052700 1 -- 172.21.15.12:0/3314830559 shutdown_connections 2024-05-08T23:59:09.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.995+0000 7f0ce1052700 1 --2- 172.21.15.12:0/3314830559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc106290 0x7f0cdc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.995+0000 7f0ce1052700 1 --2- 172.21.15.12:0/3314830559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cdc0682e0 0x7f0cdc0686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.995+0000 7f0ce1052700 1 --2- 172.21.15.12:0/3314830559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cdc068c90 0x7f0cdc105b60 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.995+0000 7f0ce1052700 1 -- 172.21.15.12:0/3314830559 >> 172.21.15.12:0/3314830559 conn(0x7f0cdc076f40 msgr2=0x7f0cdc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:09.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.996+0000 7f0ce1052700 1 -- 172.21.15.12:0/3314830559 shutdown_connections 2024-05-08T23:59:09.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.996+0000 7f0ce1052700 1 -- 172.21.15.12:0/3314830559 wait complete. 2024-05-08T23:59:09.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.996+0000 7f0ce1052700 1 Processor -- start 2024-05-08T23:59:09.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0ce1052700 1 -- start start 2024-05-08T23:59:09.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0ce1052700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cdc0682e0 0x7f0cdc197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0ce1052700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cdc106290 0x7f0cdc1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0ce1052700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc19bc70 0x7f0cdc19c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:09.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0ce1052700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc19c6f0 con 0x7f0cdc0682e0 2024-05-08T23:59:09.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0ce1052700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc19c860 con 0x7f0cdc106290 2024-05-08T23:59:09.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0ce1052700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc19ca20 con 0x7f0cdc19bc70 2024-05-08T23:59:09.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cdc0682e0 0x7f0cdc197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cdc0682e0 0x7f0cdc197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42442/0 (socket says 172.21.15.12:42442) 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0cdad9d700 1 -- 172.21.15.12:0/4292999952 learned_addr learned my addr 172.21.15.12:0/4292999952 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0cda59c700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cdc106290 0x7f0cdc1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0cdb59e700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc19bc70 0x7f0cdc19c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0cdad9d700 1 -- 172.21.15.12:0/4292999952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc19bc70 msgr2=0x7f0cdc19c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc19bc70 0x7f0cdc19c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0cdad9d700 1 -- 172.21.15.12:0/4292999952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cdc106290 msgr2=0x7f0cdc1978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.997+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cdc106290 0x7f0cdc1978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.998+0000 7f0cdad9d700 1 -- 172.21.15.12:0/4292999952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc009520 con 0x7f0cdc0682e0 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.998+0000 7f0cda59c700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cdc106290 0x7f0cdc1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.998+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cdc0682e0 0x7f0cdc197390 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00eb30 tx=0x7f0ccc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.998+0000 7f0cc3fff700 1 -- 172.21.15.12:0/4292999952 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc009880 con 0x7f0cdc0682e0 2024-05-08T23:59:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.998+0000 7f0ce1052700 1 -- 172.21.15.12:0/4292999952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd00095f0 con 0x7f0cdc0682e0 2024-05-08T23:59:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.998+0000 7f0cc3fff700 1 -- 172.21.15.12:0/4292999952 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ccc0044e0 con 0x7f0cdc0682e0 2024-05-08T23:59:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.998+0000 7f0cc3fff700 1 -- 172.21.15.12:0/4292999952 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc003d80 con 0x7f0cdc0682e0 2024-05-08T23:59:09.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:09.998+0000 7f0ce1052700 1 -- 172.21.15.12:0/4292999952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc1a4270 con 0x7f0cdc0682e0 2024-05-08T23:59:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.001+0000 7f0cc3fff700 1 -- 172.21.15.12:0/4292999952 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0ccc018430 con 0x7f0cdc0682e0 2024-05-08T23:59:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.002+0000 7f0cc3fff700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0cc40a1290 0x7f0cc40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.002+0000 7f0cc3fff700 1 -- 172.21.15.12:0/4292999952 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0ccc014070 con 0x7f0cdc0682e0 2024-05-08T23:59:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.002+0000 7f0cda59c700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0cc40a1290 0x7f0cc40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.002+0000 7f0ced20cb80 1 Processor -- start 2024-05-08T23:59:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.002+0000 7f0ced20cb80 1 -- start start 2024-05-08T23:59:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cda59c700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0cc40a1290 0x7f0cc40a3750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009990 tx=0x7f0cd0005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12330e00 0x559b123311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12331720 0x559b1232b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1232bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b12282150 con 0x559b1232b890 2024-05-08T23:59:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1227fb10 con 0x559b12331720 2024-05-08T23:59:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1227b400 con 0x559b12330e00 2024-05-08T23:59:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdb59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1232bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cda59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12331720 0x559b1232b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12330e00 0x559b123311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdb59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1232bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42450/0 (socket says 172.21.15.12:42450) 2024-05-08T23:59:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdb59e700 1 -- 172.21.15.12:0/3812903323 learned_addr learned my addr 172.21.15.12:0/3812903323 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cda59c700 1 --2- 172.21.15.12:0/3812903323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12331720 0x559b1232b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48620/0 (socket says 172.21.15.12:48620) 2024-05-08T23:59:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdad9d700 1 -- 172.21.15.12:0/3812903323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12331720 msgr2=0x559b1232b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/3812903323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12331720 0x559b1232b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdad9d700 1 -- 172.21.15.12:0/3812903323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 msgr2=0x559b1232bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/3812903323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1232bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdad9d700 1 -- 172.21.15.12:0/3812903323 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b120fc3b0 con 0x559b12330e00 2024-05-08T23:59:10.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdb59e700 1 --2- 172.21.15.12:0/3812903323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1232bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.003+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/3812903323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12330e00 0x559b123311e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0caa60 tx=0x7f0ccc099e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1df0185103d8dac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0cc17fa700 1 -- 172.21.15.12:0/3812903323 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc0850e0 con 0x559b12330e00 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0cc17fa700 1 -- 172.21.15.12:0/3812903323 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ccc00cdd0 con 0x559b12330e00 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0cc17fa700 1 -- 172.21.15.12:0/3812903323 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc09a8c0 con 0x559b12330e00 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0ced20cb80 1 -- 172.21.15.12:0/3812903323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12330e00 msgr2=0x559b123311e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/3812903323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12330e00 0x559b123311e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0caa60 tx=0x7f0ccc099e00 comp rx=0 tx=0).stop 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0ced20cb80 1 -- 172.21.15.12:0/3812903323 shutdown_connections 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/3812903323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1232bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/3812903323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12330e00 0x559b123311e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/3812903323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12331720 0x559b1232b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0ced20cb80 1 -- 172.21.15.12:0/3812903323 >> 172.21.15.12:0/3812903323 conn(0x559b12289210 msgr2=0x559b12325df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:10.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0ced20cb80 1 -- 172.21.15.12:0/3812903323 shutdown_connections 2024-05-08T23:59:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.004+0000 7f0ced20cb80 1 -- 172.21.15.12:0/3812903323 wait complete. 2024-05-08T23:59:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0ced20cb80 1 -- start start 2024-05-08T23:59:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0ced20cb80 1 Processor -- start 2024-05-08T23:59:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b1232b890 0x559b12334ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b123350e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b12331720 0x559b12335620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b12217560 con 0x559b12331720 2024-05-08T23:59:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b12217200 con 0x559b12330e00 2024-05-08T23:59:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b120fe6a0 con 0x559b1232b890 2024-05-08T23:59:10.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0ced20cb80 10 client.0 _ll_register_callbacks cb 0x559b12220b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0cdb59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b12331720 0x559b12335620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0cda59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b123350e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0cda59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b123350e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48634/0 (socket says 172.21.15.12:48634) 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.005+0000 7f0cdb59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b12331720 0x559b12335620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42462/0 (socket says 172.21.15.12:42462) 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cda59c700 1 -- 172.21.15.12:0/1303762843 learned_addr learned my addr 172.21.15.12:0/1303762843 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b1232b890 0x559b12334ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cda59c700 1 -- 172.21.15.12:0/1303762843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b1232b890 msgr2=0x559b12334ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cda59c700 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b1232b890 0x559b12334ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cda59c700 1 -- 172.21.15.12:0/1303762843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b12331720 msgr2=0x559b12335620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cda59c700 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b12331720 0x559b12335620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cda59c700 1 -- 172.21.15.12:0/1303762843 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd4011330 con 0x559b12330e00 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cda59c700 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b123350e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc06a6d0 tx=0x7f0cd001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68cbd33e982610 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:10.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cbeff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cbeff5700 1 -- 172.21.15.12:0/1303762843 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd002c040 con 0x559b12330e00 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cbeff5700 1 -- 172.21.15.12:0/1303762843 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0cd0004680 con 0x559b12330e00 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1303762843 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b121d4270 con 0x559b12330e00 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0cbeff5700 1 -- 172.21.15.12:0/1303762843 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd000f870 con 0x559b12330e00 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0ced20cb80 10 client.28814 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.006+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1303762843 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f0cdc1a4270 con 0x559b12330e00 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.007+0000 7f0cbeff5700 1 -- 172.21.15.12:0/1303762843 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd003d080 con 0x559b12330e00 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.007+0000 7f0cbeff5700 1 -- 172.21.15.12:0/1303762843 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0cd00319e0 con 0x559b12330e00 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.007+0000 7f0ced20cb80 10 client.28814 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.007+0000 7f0ced20cb80 10 client.28814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.007+0000 7f0ced20cb80 20 client.28814 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.007+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559b12345290 0x559b123476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.007+0000 7f0ced20cb80 4 client.28814 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:10.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.007+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1303762843 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b122fa810 con 0x559b12345290 2024-05-08T23:59:10.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.008+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559b12345290 0x559b123476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.008+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559b12345290 0x559b123476b0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dabf8663bd2575c7 server_cookie=46dbd748fab7e562 in_seq=0 out_seq=0 2024-05-08T23:59:10.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.008+0000 7f0cbeff5700 10 client.28814 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:10.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0cbeff5700 1 -- 172.21.15.12:0/1303762843 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b122fa810 con 0x559b12345290 2024-05-08T23:59:10.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0cbeff5700 10 client.28814 handle_command_reply: tid=0 2024-05-08T23:59:10.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 client.28814 shutdown 2024-05-08T23:59:10.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1303762843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 msgr2=0x559b123350e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:10.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b123350e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc06a6d0 tx=0x7f0cd001f000 comp rx=0 tx=0).stop 2024-05-08T23:59:10.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1303762843 shutdown_connections 2024-05-08T23:59:10.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b12331720 0x559b12335620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b1232b890 0x559b12334ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559b12345290 0x559b123476b0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/1303762843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b123350e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1303762843 >> 172.21.15.12:0/1303762843 conn(0x559b12289210 msgr2=0x559b123255a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:10.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0cbeff5700 0 client.28814 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:10.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1303762843 shutdown_connections 2024-05-08T23:59:10.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1303762843 wait complete. 2024-05-08T23:59:10.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.009+0000 7f0ced20cb80 20 client.28814 trim_cache size 0 max 16384 2024-05-08T23:59:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:09 smithi204 ceph-mon[47373]: pgmap v407: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-05-08T23:59:10.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:09 smithi160 ceph-mon[46152]: pgmap v407: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-05-08T23:59:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.029+0000 7f0ced20cb80 1 Processor -- start 2024-05-08T23:59:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.029+0000 7f0ced20cb80 1 -- start start 2024-05-08T23:59:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.029+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b12340c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.029+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b12325d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.029+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 0x559b12335110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.029+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b123376a0 con 0x559b1232b890 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.029+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1234c8c0 con 0x559b12330e00 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.029+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b12320800 con 0x559b12331720 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b12340c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdb59e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 0x559b12335110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b12340c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42476/0 (socket says 172.21.15.12:42476) 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdad9d700 1 -- 172.21.15.12:0/1062734466 learned_addr learned my addr 172.21.15.12:0/1062734466 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cda59c700 1 --2- 172.21.15.12:0/1062734466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b12325d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdb59e700 1 -- 172.21.15.12:0/1062734466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 msgr2=0x559b12325d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdb59e700 1 --2- 172.21.15.12:0/1062734466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b12325d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdb59e700 1 -- 172.21.15.12:0/1062734466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 msgr2=0x559b12340c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdb59e700 1 --2- 172.21.15.12:0/1062734466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b12340c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdb59e700 1 -- 172.21.15.12:0/1062734466 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b120a8b60 con 0x559b12331720 2024-05-08T23:59:10.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/1062734466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b12340c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cdb59e700 1 --2- 172.21.15.12:0/1062734466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 0x559b12335110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4016b60 tx=0x7f0cd400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a7d537429fc5f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.030+0000 7f0cc1ffb700 1 -- 172.21.15.12:0/1062734466 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd401b050 con 0x559b12331720 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0cc1ffb700 1 -- 172.21.15.12:0/1062734466 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd401c040 con 0x559b12331720 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0cc1ffb700 1 -- 172.21.15.12:0/1062734466 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400e040 con 0x559b12331720 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1062734466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 msgr2=0x559b12335110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/1062734466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 0x559b12335110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4016b60 tx=0x7f0cd400b040 comp rx=0 tx=0).stop 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1062734466 shutdown_connections 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/1062734466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 0x559b12335110 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/1062734466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b12340c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/1062734466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b12325d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1062734466 >> 172.21.15.12:0/1062734466 conn(0x559b12289210 msgr2=0x559b1232bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1062734466 shutdown_connections 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.031+0000 7f0ced20cb80 1 -- 172.21.15.12:0/1062734466 wait complete. 2024-05-08T23:59:10.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.032+0000 7f0ced20cb80 1 -- start start 2024-05-08T23:59:10.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.032+0000 7f0ced20cb80 1 Processor -- start 2024-05-08T23:59:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.032+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1233a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.032+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b1233a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.032+0000 7f0ced20cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 0x559b1239a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.032+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1233af30 con 0x559b1232b890 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.032+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1239acd0 con 0x559b12330e00 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.032+0000 7f0ced20cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1239ae10 con 0x559b12331720 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.032+0000 7f0ced20cb80 10 client.0 _ll_register_callbacks cb 0x559b12220b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.032+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1233a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1233a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42488/0 (socket says 172.21.15.12:42488) 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cdad9d700 1 -- 172.21.15.12:0/2316030934 learned_addr learned my addr 172.21.15.12:0/2316030934 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cda59c700 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b1233a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cdb59e700 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 0x559b1239a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cdad9d700 1 -- 172.21.15.12:0/2316030934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 msgr2=0x559b1239a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 0x559b1239a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cdad9d700 1 -- 172.21.15.12:0/2316030934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 msgr2=0x559b1233a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:10.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b1233a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cdad9d700 1 -- 172.21.15.12:0/2316030934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b120fc3b0 con 0x559b1232b890 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cdad9d700 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1233a150 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc096e90 tx=0x7f0ccc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a10bb23989bedfc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cbeff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cbeff5700 1 -- 172.21.15.12:0/2316030934 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc0850b0 con 0x559b1232b890 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cbeff5700 1 -- 172.21.15.12:0/2316030934 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0ccc09a420 con 0x559b1232b890 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0ced20cb80 1 -- 172.21.15.12:0/2316030934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ccc00cdd0 con 0x559b1232b890 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0cbeff5700 1 -- 172.21.15.12:0/2316030934 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ccc099910 con 0x559b1232b890 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0ced20cb80 10 client.19110 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.033+0000 7f0ced20cb80 1 -- 172.21.15.12:0/2316030934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f0cd4016810 con 0x559b1232b890 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.034+0000 7f0cbeff5700 1 -- 172.21.15.12:0/2316030934 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ccc00cdd0 con 0x559b1232b890 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.034+0000 7f0cbeff5700 1 -- 172.21.15.12:0/2316030934 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0ccc09a6f0 con 0x559b1232b890 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.034+0000 7f0ced20cb80 10 client.19110 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.034+0000 7f0ced20cb80 10 client.19110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:10.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.034+0000 7f0ced20cb80 20 client.19110 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.034+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559b1239bf60 0x559b1239e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.034+0000 7f0ced20cb80 4 client.19110 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.034+0000 7f0ced20cb80 1 -- 172.21.15.12:0/2316030934 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b12273ec0 con 0x559b1239bf60 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.034+0000 7f0cda59c700 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559b1239bf60 0x559b1239e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0cbeff5700 10 client.19110 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.034+0000 7f0cda59c700 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559b1239bf60 0x559b1239e350 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f713ca138c2433b6 server_cookie=29a541339a22e167 in_seq=0 out_seq=0 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0cbeff5700 1 -- 172.21.15.12:0/2316030934 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559b12273ec0 con 0x559b1239bf60 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0cbeff5700 10 client.19110 handle_command_reply: tid=0 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 client.19110 shutdown 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 -- 172.21.15.12:0/2316030934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 msgr2=0x559b1233a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1233a150 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc096e90 tx=0x7f0ccc003820 comp rx=0 tx=0).stop 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 -- 172.21.15.12:0/2316030934 shutdown_connections 2024-05-08T23:59:10.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b12331720 0x559b1239a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b1232b890 0x559b1233a150 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559b1239bf60 0x559b1239e350 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 --2- 172.21.15.12:0/2316030934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b12330e00 0x559b1233a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 -- 172.21.15.12:0/2316030934 >> 172.21.15.12:0/2316030934 conn(0x559b12289210 msgr2=0x559b12333450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:10.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0cbeff5700 0 client.19110 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:10.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 -- 172.21.15.12:0/2316030934 shutdown_connections 2024-05-08T23:59:10.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 1 -- 172.21.15.12:0/2316030934 wait complete. 2024-05-08T23:59:10.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.035+0000 7f0ced20cb80 20 client.19110 trim_cache size 0 max 16384 2024-05-08T23:59:10.036 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:59:10.036 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:59:10.037 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:59:10.037 INFO:teuthology.orchestra.run.smithi012.stdout: "229e1683-12c4-4f32-8653-39fa98c44359": { 2024-05-08T23:59:10.037 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:59:10.037 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "229e1683-12c4-4f32-8653-39fa98c44359", 2024-05-08T23:59:10.037 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:59:10.037 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:59:10.037 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:59:10.037 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:59:10.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 -- 172.21.15.12:0/4292999952 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0cc40a1290 msgr2=0x7f0cc40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:10.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0cc40a1290 0x7f0cc40a3750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009990 tx=0x7f0cd0005fb0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 -- 172.21.15.12:0/4292999952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cdc0682e0 msgr2=0x7f0cdc197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:10.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cdc0682e0 0x7f0cdc197390 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00eb30 tx=0x7f0ccc00ee40 comp rx=0 tx=0).stop 2024-05-08T23:59:10.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 -- 172.21.15.12:0/4292999952 shutdown_connections 2024-05-08T23:59:10.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0cc40a1290 0x7f0cc40a3750 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc19bc70 0x7f0cdc19c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cdc0682e0 0x7f0cdc197390 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 --2- 172.21.15.12:0/4292999952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cdc106290 0x7f0cdc1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:10.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 -- 172.21.15.12:0/4292999952 >> 172.21.15.12:0/4292999952 conn(0x7f0cdc076f40 msgr2=0x7f0cdc0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:10.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 -- 172.21.15.12:0/4292999952 shutdown_connections 2024-05-08T23:59:10.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:10.036+0000 7f0ce1052700 1 -- 172.21.15.12:0/4292999952 wait complete. 2024-05-08T23:59:10.045 INFO:tasks.cephfs.filesystem:scrub status for tag:229e1683-12c4-4f32-8653-39fa98c44359 - {'path': '/', 'tag': '229e1683-12c4-4f32-8653-39fa98c44359', 'options': 'recursive,force'} 2024-05-08T23:59:10.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 704822 6.04 MB/sec execute 546 sec latency 534.453 ms 2024-05-08T23:59:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:10 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:59:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:10 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:10 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:59:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:10 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:59:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:10 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:59:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:10 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:10 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:10.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:10 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:11.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:10 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:59:11.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:10 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:11.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:10 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:59:11.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:10 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:59:11.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:10 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:59:11.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:10 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:11.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:10 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:11.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:10 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:11.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:10 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:59:11.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:10 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:11.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:10 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:59:11.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:10 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:59:11.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:10 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:59:11.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:10 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:11.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:10 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:11.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:10 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:11.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 704822 6.03 MB/sec execute 547 sec latency 1534.508 ms 2024-05-08T23:59:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:11 smithi012 ceph-mon[42503]: osdmap e107: 12 total, 12 up, 12 in 2024-05-08T23:59:12.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:11 smithi012 ceph-mon[42503]: pgmap v409: 97 pgs: 97 active+clean; 195 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-05-08T23:59:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:11 smithi204 ceph-mon[47373]: osdmap e107: 12 total, 12 up, 12 in 2024-05-08T23:59:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:11 smithi204 ceph-mon[47373]: pgmap v409: 97 pgs: 97 active+clean; 195 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-05-08T23:59:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:11 smithi160 ceph-mon[46152]: osdmap e107: 12 total, 12 up, 12 in 2024-05-08T23:59:12.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:11 smithi160 ceph-mon[46152]: pgmap v409: 97 pgs: 97 active+clean; 195 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-05-08T23:59:12.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 704822 6.02 MB/sec execute 548 sec latency 2534.612 ms 2024-05-08T23:59:13.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 706153 6.02 MB/sec execute 549 sec latency 3009.806 ms 2024-05-08T23:59:13.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:13 smithi012 ceph-mon[42503]: pgmap v410: 97 pgs: 97 active+clean; 195 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-05-08T23:59:14.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:13 smithi204 ceph-mon[47373]: pgmap v410: 97 pgs: 97 active+clean; 195 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-05-08T23:59:14.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:13 smithi160 ceph-mon[46152]: pgmap v410: 97 pgs: 97 active+clean; 195 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-05-08T23:59:14.899 INFO:tasks.workunit.client.0.smithi012.stdout: 1 707711 6.02 MB/sec execute 550 sec latency 14.155 ms 2024-05-08T23:59:15.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 708365 6.02 MB/sec execute 551 sec latency 392.869 ms 2024-05-08T23:59:15.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:15 smithi012 ceph-mon[42503]: pgmap v411: 97 pgs: 97 active+clean; 195 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-05-08T23:59:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:15 smithi204 ceph-mon[47373]: pgmap v411: 97 pgs: 97 active+clean; 195 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-05-08T23:59:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:15 smithi160 ceph-mon[46152]: pgmap v411: 97 pgs: 97 active+clean; 195 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-05-08T23:59:16.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 710351 6.03 MB/sec execute 552 sec latency 523.754 ms 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.266+0000 7f22e5c94700 1 -- 172.21.15.12:0/1491162370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0100760 msgr2=0x7f22e010cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.266+0000 7f22e5c94700 1 --2- 172.21.15.12:0/1491162370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0100760 0x7f22e010cc00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f22d800b0a0 tx=0x7f22d800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.266+0000 7f22e5c94700 1 -- 172.21.15.12:0/1491162370 shutdown_connections 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.266+0000 7f22e5c94700 1 --2- 172.21.15.12:0/1491162370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0100760 0x7f22e010cc00 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.266+0000 7f22e5c94700 1 --2- 172.21.15.12:0/1491162370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0105e60 0x7f22e0106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.266+0000 7f22e5c94700 1 --2- 172.21.15.12:0/1491162370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e00ffda0 0x7f22e0100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.266+0000 7f22e5c94700 1 -- 172.21.15.12:0/1491162370 >> 172.21.15.12:0/1491162370 conn(0x7f22e0076f10 msgr2=0x7f22e0077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.266+0000 7f22e5c94700 1 -- 172.21.15.12:0/1491162370 shutdown_connections 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.266+0000 7f22e5c94700 1 -- 172.21.15.12:0/1491162370 wait complete. 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.266+0000 7f22e5c94700 1 Processor -- start 2024-05-08T23:59:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22e5c94700 1 -- start start 2024-05-08T23:59:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22e5c94700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e00ffda0 0x7f22e0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22e5c94700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100760 0x7f22e0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22e5c94700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0105e60 0x7f22e019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22e5c94700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e019c350 con 0x7f22e0105e60 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22e5c94700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e019c4c0 con 0x7f22e0100760 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22e5c94700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e019c680 con 0x7f22e00ffda0 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22dffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0105e60 0x7f22e019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22deffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100760 0x7f22e0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22dffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0105e60 0x7f22e019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42498/0 (socket says 172.21.15.12:42498) 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22deffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100760 0x7f22e0197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48674/0 (socket says 172.21.15.12:48674) 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22dffff700 1 -- 172.21.15.12:0/2686357607 learned_addr learned my addr 172.21.15.12:0/2686357607 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22df7fe700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e00ffda0 0x7f22e0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22dffff700 1 -- 172.21.15.12:0/2686357607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e00ffda0 msgr2=0x7f22e0197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22dffff700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e00ffda0 0x7f22e0197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22dffff700 1 -- 172.21.15.12:0/2686357607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100760 msgr2=0x7f22e0197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22dffff700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100760 0x7f22e0197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22dffff700 1 -- 172.21.15.12:0/2686357607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d8009d00 con 0x7f22e0105e60 2024-05-08T23:59:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.267+0000 7f22deffd700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100760 0x7f22e0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.268+0000 7f22dffff700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0105e60 0x7f22e019bd10 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f22d800b070 tx=0x7f22d8007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.268+0000 7f22dcff9700 1 -- 172.21.15.12:0/2686357607 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800e040 con 0x7f22e0105e60 2024-05-08T23:59:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.268+0000 7f22dcff9700 1 -- 172.21.15.12:0/2686357607 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d801d470 con 0x7f22e0105e60 2024-05-08T23:59:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.268+0000 7f22e5c94700 1 -- 172.21.15.12:0/2686357607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e019c900 con 0x7f22e0105e60 2024-05-08T23:59:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.268+0000 7f22dcff9700 1 -- 172.21.15.12:0/2686357607 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80052a0 con 0x7f22e0105e60 2024-05-08T23:59:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.268+0000 7f22e5c94700 1 -- 172.21.15.12:0/2686357607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e01a41a0 con 0x7f22e0105e60 2024-05-08T23:59:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.271+0000 7f22dcff9700 1 -- 172.21.15.12:0/2686357607 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f22d801d870 con 0x7f22e0105e60 2024-05-08T23:59:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.271+0000 7f22dcff9700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22c80a1290 0x7f22c80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.271+0000 7f22dcff9700 1 -- 172.21.15.12:0/2686357607 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f22d80cdff0 con 0x7f22e0105e60 2024-05-08T23:59:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.271+0000 7f22df7fe700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22c80a1290 0x7f22c80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22f1e4eb80 1 Processor -- start 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22f1e4eb80 1 -- start start 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22df7fe700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22c80a1290 0x7f22c80a3750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f22d0007b60 tx=0x7f22d00073d0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d36d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 0x559261d30f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d31480 0x559261d318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261c880f0 con 0x559261d36950 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261c85ab0 con 0x559261d31480 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261c813a0 con 0x559261d37270 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d36d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d36d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42502/0 (socket says 172.21.15.12:42502) 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22df7fe700 1 -- 172.21.15.12:0/3180145110 learned_addr learned my addr 172.21.15.12:0/3180145110 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22dffff700 1 --2- 172.21.15.12:0/3180145110 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d31480 0x559261d318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.272+0000 7f22deffd700 1 --2- 172.21.15.12:0/3180145110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 0x559261d30f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.273+0000 7f22df7fe700 1 -- 172.21.15.12:0/3180145110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 msgr2=0x559261d30f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.273+0000 7f22df7fe700 1 --2- 172.21.15.12:0/3180145110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 0x559261d30f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.273+0000 7f22df7fe700 1 -- 172.21.15.12:0/3180145110 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d31480 msgr2=0x559261d318a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.273+0000 7f22df7fe700 1 --2- 172.21.15.12:0/3180145110 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d31480 0x559261d318a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.273+0000 7f22df7fe700 1 -- 172.21.15.12:0/3180145110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559261b02350 con 0x559261d36950 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.273+0000 7f22dffff700 1 --2- 172.21.15.12:0/3180145110 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d31480 0x559261d318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.273+0000 7f22df7fe700 1 --2- 172.21.15.12:0/3180145110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d36d30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f22d0010d90 tx=0x7f22d0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e23d4cb8c79b8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.273+0000 7f22c5ffb700 1 -- 172.21.15.12:0/3180145110 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0011bb0 con 0x559261d36950 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.273+0000 7f22c5ffb700 1 -- 172.21.15.12:0/3180145110 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d0015070 con 0x559261d36950 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.273+0000 7f22c5ffb700 1 -- 172.21.15.12:0/3180145110 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d001c610 con 0x559261d36950 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.274+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3180145110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 msgr2=0x559261d36d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.274+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3180145110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d36d30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f22d0010d90 tx=0x7f22d0013000 comp rx=0 tx=0).stop 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.274+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3180145110 shutdown_connections 2024-05-08T23:59:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.274+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3180145110 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d31480 0x559261d318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.274+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3180145110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d36d30 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.274+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3180145110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 0x559261d30f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.274+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3180145110 >> 172.21.15.12:0/3180145110 conn(0x559261c8f1b0 msgr2=0x559261d2b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.274+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3180145110 shutdown_connections 2024-05-08T23:59:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.274+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3180145110 wait complete. 2024-05-08T23:59:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.274+0000 7f22f1e4eb80 1 -- start start 2024-05-08T23:59:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22f1e4eb80 1 Processor -- start 2024-05-08T23:59:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 0x559261d3d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d36950 0x559261d3a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 0x559261d3aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261c1d500 con 0x559261d31480 2024-05-08T23:59:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261c1d1a0 con 0x559261d36950 2024-05-08T23:59:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261b04640 con 0x559261d37270 2024-05-08T23:59:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22f1e4eb80 10 client.0 _ll_register_callbacks cb 0x559261c26ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22dffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 0x559261d3aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 0x559261d3d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22dffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 0x559261d3aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:36512/0 (socket says 172.21.15.12:36512) 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22dffff700 1 -- 172.21.15.12:0/4106968340 learned_addr learned my addr 172.21.15.12:0/4106968340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.275+0000 7f22deffd700 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d36950 0x559261d3a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22dffff700 1 -- 172.21.15.12:0/4106968340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d36950 msgr2=0x559261d3a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22dffff700 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d36950 0x559261d3a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22dffff700 1 -- 172.21.15.12:0/4106968340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 msgr2=0x559261d3d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22dffff700 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 0x559261d3d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22dffff700 1 -- 172.21.15.12:0/4106968340 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e01a41a0 con 0x559261d37270 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22deffd700 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d36950 0x559261d3a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22dffff700 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 0x559261d3aba0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f22d80cbd50 tx=0x7f22d809ab00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a25da0b2d86af5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22beffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22beffd700 1 -- 172.21.15.12:0/4106968340 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80cbea0 con 0x559261d37270 2024-05-08T23:59:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/4106968340 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559261bda210 con 0x559261d37270 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22beffd700 1 -- 172.21.15.12:0/4106968340 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f22d809efd0 con 0x559261d37270 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22beffd700 1 -- 172.21.15.12:0/4106968340 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d80a3590 con 0x559261d37270 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22f1e4eb80 10 client.28840 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.276+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/4106968340 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f22d0010a40 con 0x559261d37270 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22beffd700 1 -- 172.21.15.12:0/4106968340 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559261bda210 con 0x559261d37270 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22beffd700 1 -- 172.21.15.12:0/4106968340 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f22d809a690 con 0x559261d37270 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22f1e4eb80 10 client.28840 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22f1e4eb80 10 client.28840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22f1e4eb80 20 client.28840 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559261d4a970 0x559261d4cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22f1e4eb80 4 client.28840 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/4106968340 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559261d007b0 con 0x559261d4a970 2024-05-08T23:59:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22df7fe700 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559261d4a970 0x559261d4cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22beffd700 10 client.28840 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.277+0000 7f22df7fe700 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559261d4a970 0x559261d4cd90 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e14aa05b4fa4a488 server_cookie=8ac41bbff6e59f62 in_seq=0 out_seq=0 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.279+0000 7f22beffd700 1 -- 172.21.15.12:0/4106968340 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559261d007b0 con 0x559261d4a970 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.279+0000 7f22beffd700 10 client.28840 handle_command_reply: tid=0 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 client.28840 shutdown 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/4106968340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 msgr2=0x559261d3aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 0x559261d3aba0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f22d80cbd50 tx=0x7f22d809ab00 comp rx=0 tx=0).stop 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/4106968340 shutdown_connections 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d37270 0x559261d3aba0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 0x559261d3d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559261d4a970 0x559261d4cd90 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/4106968340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d36950 0x559261d3a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/4106968340 >> 172.21.15.12:0/4106968340 conn(0x559261c8f1b0 msgr2=0x559261d2ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22beffd700 0 client.28840 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/4106968340 shutdown_connections 2024-05-08T23:59:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/4106968340 wait complete. 2024-05-08T23:59:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.280+0000 7f22f1e4eb80 20 client.28840 trim_cache size 0 max 16384 2024-05-08T23:59:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.296+0000 7f22f1e4eb80 1 Processor -- start 2024-05-08T23:59:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.296+0000 7f22f1e4eb80 1 -- start start 2024-05-08T23:59:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 0x559261d2b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d36950 0x559261d3cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 0x559261d41c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261d3bb20 con 0x559261d31480 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261d312e0 con 0x559261d37270 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261d51fd0 con 0x559261d36950 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 0x559261d2b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22dffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 0x559261d41c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 0x559261d2b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42526/0 (socket says 172.21.15.12:42526) 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22dffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 0x559261d41c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48714/0 (socket says 172.21.15.12:48714) 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22df7fe700 1 -- 172.21.15.12:0/3223938862 learned_addr learned my addr 172.21.15.12:0/3223938862 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22deffd700 1 --2- 172.21.15.12:0/3223938862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d36950 0x559261d3cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22dffff700 1 -- 172.21.15.12:0/3223938862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d36950 msgr2=0x559261d3cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22dffff700 1 --2- 172.21.15.12:0/3223938862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d36950 0x559261d3cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22dffff700 1 -- 172.21.15.12:0/3223938862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 msgr2=0x559261d2b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22dffff700 1 --2- 172.21.15.12:0/3223938862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 0x559261d2b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22dffff700 1 -- 172.21.15.12:0/3223938862 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559261aaeb00 con 0x559261d37270 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22df7fe700 1 --2- 172.21.15.12:0/3223938862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 0x559261d2b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.297+0000 7f22dffff700 1 --2- 172.21.15.12:0/3223938862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 0x559261d41c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f22d80ce2d0 tx=0x7f22d809b320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2855192fb47d42e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.298+0000 7f22c67fc700 1 -- 172.21.15.12:0/3223938862 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80cbea0 con 0x559261d37270 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.298+0000 7f22c67fc700 1 -- 172.21.15.12:0/3223938862 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d8005400 con 0x559261d37270 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.298+0000 7f22c67fc700 1 -- 172.21.15.12:0/3223938862 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80a10b0 con 0x559261d37270 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.298+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3223938862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 msgr2=0x559261d41c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.298+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3223938862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 0x559261d41c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f22d80ce2d0 tx=0x7f22d809b320 comp rx=0 tx=0).stop 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.298+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3223938862 shutdown_connections 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.298+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3223938862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 0x559261d41c10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.298+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3223938862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d31480 0x559261d2b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.298+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3223938862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d36950 0x559261d3cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.298+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3223938862 >> 172.21.15.12:0/3223938862 conn(0x559261c8f1b0 msgr2=0x559261d38d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.299+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3223938862 shutdown_connections 2024-05-08T23:59:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.299+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3223938862 wait complete. 2024-05-08T23:59:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.299+0000 7f22f1e4eb80 1 -- start start 2024-05-08T23:59:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.299+0000 7f22f1e4eb80 1 Processor -- start 2024-05-08T23:59:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.299+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d31480 0x559261d41fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.299+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d3fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.299+0000 7f22f1e4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 0x559261d40190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.299+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261d408a0 con 0x559261d36950 2024-05-08T23:59:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.299+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261d40a10 con 0x559261d37270 2024-05-08T23:59:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.299+0000 7f22f1e4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559261d9fe70 con 0x559261d31480 2024-05-08T23:59:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22f1e4eb80 10 client.0 _ll_register_callbacks cb 0x559261c26ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22deffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d3fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22deffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d3fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42540/0 (socket says 172.21.15.12:42540) 2024-05-08T23:59:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22deffd700 1 -- 172.21.15.12:0/3154467969 learned_addr learned my addr 172.21.15.12:0/3154467969 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22df7fe700 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d31480 0x559261d41fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22dffff700 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 0x559261d40190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22deffd700 1 -- 172.21.15.12:0/3154467969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d31480 msgr2=0x559261d41fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22deffd700 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d31480 0x559261d41fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22deffd700 1 -- 172.21.15.12:0/3154467969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 msgr2=0x559261d40190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22deffd700 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 0x559261d40190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22deffd700 1 -- 172.21.15.12:0/3154467969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559261b02350 con 0x559261d36950 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22deffd700 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d3fbf0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x559261d3baf0 tx=0x7f22d4010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a363fe05663d5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22beffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22beffd700 1 -- 172.21.15.12:0/3154467969 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d400fd10 con 0x559261d36950 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22beffd700 1 -- 172.21.15.12:0/3154467969 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f22d400fe70 con 0x559261d36950 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3154467969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22d0015070 con 0x559261d36950 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.300+0000 7f22beffd700 1 -- 172.21.15.12:0/3154467969 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d4004030 con 0x559261d36950 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22f1e4eb80 10 client.19128 fetch_fsmap learned FSMap version 55 2024-05-08T23:59:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3154467969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f22d0010a40 con 0x559261d36950 2024-05-08T23:59:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22beffd700 1 -- 172.21.15.12:0/3154467969 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d4029020 con 0x559261d36950 2024-05-08T23:59:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22beffd700 1 -- 172.21.15.12:0/3154467969 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 55) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f22d4005140 con 0x559261d36950 2024-05-08T23:59:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22f1e4eb80 10 client.19128 fetch_fsmap finished waiting for FSMap version 55 2024-05-08T23:59:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22f1e4eb80 10 client.19128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22f1e4eb80 20 client.19128 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559261da16f0 0x559261da3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22f1e4eb80 4 client.19128 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:59:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3154467969 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559261c79e60 con 0x559261da16f0 2024-05-08T23:59:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22df7fe700 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559261da16f0 0x559261da3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22beffd700 10 client.19128 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.301+0000 7f22df7fe700 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559261da16f0 0x559261da3b10 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab9dc13f55a9618 server_cookie=747d9749f0ba5920 in_seq=0 out_seq=0 2024-05-08T23:59:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22beffd700 1 -- 172.21.15.12:0/3154467969 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559261c79e60 con 0x559261da16f0 2024-05-08T23:59:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22beffd700 10 client.19128 handle_command_reply: tid=0 2024-05-08T23:59:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22f1e4eb80 1 client.19128 shutdown 2024-05-08T23:59:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3154467969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 msgr2=0x559261d3fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d3fbf0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x559261d3baf0 tx=0x7f22d4010f40 comp rx=0 tx=0).stop 2024-05-08T23:59:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3154467969 shutdown_connections 2024-05-08T23:59:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559261da16f0 0x559261da3b10 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559261d37270 0x559261d40190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559261d31480 0x559261d41fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22beffd700 0 client.19128 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22f1e4eb80 1 --2- 172.21.15.12:0/3154467969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559261d36950 0x559261d3fbf0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.365+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3154467969 >> 172.21.15.12:0/3154467969 conn(0x559261c8f1b0 msgr2=0x559261d38cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3154467969 shutdown_connections 2024-05-08T23:59:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22f1e4eb80 1 -- 172.21.15.12:0/3154467969 wait complete. 2024-05-08T23:59:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22f1e4eb80 20 client.19128 trim_cache size 0 max 16384 2024-05-08T23:59:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:59:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:59:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:59:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22e5c94700 1 -- 172.21.15.12:0/2686357607 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22c80a1290 msgr2=0x7f22c80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22e5c94700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22c80a1290 0x7f22c80a3750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f22d0007b60 tx=0x7f22d00073d0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22e5c94700 1 -- 172.21.15.12:0/2686357607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0105e60 msgr2=0x7f22e019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22e5c94700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0105e60 0x7f22e019bd10 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f22d800b070 tx=0x7f22d8007930 comp rx=0 tx=0).stop 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22e5c94700 1 -- 172.21.15.12:0/2686357607 shutdown_connections 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22e5c94700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22c80a1290 0x7f22c80a3750 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22e5c94700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0105e60 0x7f22e019bd10 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22e5c94700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e00ffda0 0x7f22e0197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22e5c94700 1 --2- 172.21.15.12:0/2686357607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100760 0x7f22e0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.366+0000 7f22e5c94700 1 -- 172.21.15.12:0/2686357607 >> 172.21.15.12:0/2686357607 conn(0x7f22e0076f10 msgr2=0x7f22e0068560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.367+0000 7f22e5c94700 1 -- 172.21.15.12:0/2686357607 shutdown_connections 2024-05-08T23:59:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:17.367+0000 7f22e5c94700 1 -- 172.21.15.12:0/2686357607 wait complete. 2024-05-08T23:59:17.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 711474 6.02 MB/sec execute 553 sec latency 25.567 ms 2024-05-08T23:59:17.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:17 smithi012 ceph-mon[42503]: pgmap v412: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-05-08T23:59:18.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:17 smithi204 ceph-mon[47373]: pgmap v412: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-05-08T23:59:18.012 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:59:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-08T23:59:18.012 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:59:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-08T23:59:18.012 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:59:17 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/482848280 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bda716c1a0 con 0x55bda7143000 2024-05-08T23:59:18.013 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 08 23:59:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-08T23:59:18.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:17 smithi160 ceph-mon[46152]: pgmap v412: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-05-08T23:59:18.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 713429 6.03 MB/sec execute 554 sec latency 204.089 ms 2024-05-08T23:59:18.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:18 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-08T23:59:18.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:18 smithi012 ceph-mon[42503]: osdmap e108: 12 total, 12 up, 12 in 2024-05-08T23:59:18.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:18 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:59:19.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:18 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-08T23:59:19.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:18 smithi204 ceph-mon[47373]: osdmap e108: 12 total, 12 up, 12 in 2024-05-08T23:59:19.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:18 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:59:19.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:18 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-08T23:59:19.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:18 smithi160 ceph-mon[46152]: osdmap e108: 12 total, 12 up, 12 in 2024-05-08T23:59:19.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:18 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:59:19.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 714872 6.04 MB/sec execute 555 sec latency 24.157 ms 2024-05-08T23:59:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:19 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:59:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:19 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:59:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:19 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/864772354,v1:172.21.15.204:6837/864772354] up:boot 2024-05-08T23:59:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:19 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:59:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:19 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:59:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:19 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:19 smithi012 ceph-mon[42503]: pgmap v414: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-05-08T23:59:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:19 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:59:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:19 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:59:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:19 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/864772354,v1:172.21.15.204:6837/864772354] up:boot 2024-05-08T23:59:20.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:19 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:59:20.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:19 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:59:20.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:19 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:20.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:19 smithi204 ceph-mon[47373]: pgmap v414: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-05-08T23:59:20.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:19 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:59:20.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:19 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:59:20.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:19 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/864772354,v1:172.21.15.204:6837/864772354] up:boot 2024-05-08T23:59:20.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:19 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-08T23:59:20.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:19 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:59:20.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:19 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:20.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:19 smithi160 ceph-mon[46152]: pgmap v414: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-05-08T23:59:20.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 715457 6.03 MB/sec execute 556 sec latency 430.018 ms 2024-05-08T23:59:21.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 715457 6.02 MB/sec execute 557 sec latency 1430.077 ms 2024-05-08T23:59:21.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:21 smithi012 ceph-mon[42503]: pgmap v415: 97 pgs: 97 active+clean; 196 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-05-08T23:59:22.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:21 smithi204 ceph-mon[47373]: pgmap v415: 97 pgs: 97 active+clean; 196 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-05-08T23:59:22.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:21 smithi160 ceph-mon[46152]: pgmap v415: 97 pgs: 97 active+clean; 196 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-05-08T23:59:22.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 715639 6.01 MB/sec execute 558 sec latency 2331.869 ms 2024-05-08T23:59:23.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 717749 6.02 MB/sec execute 559 sec latency 14.685 ms 2024-05-08T23:59:23.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:23 smithi012 ceph-mon[42503]: pgmap v416: 97 pgs: 97 active+clean; 196 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-05-08T23:59:24.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:23 smithi204 ceph-mon[47373]: pgmap v416: 97 pgs: 97 active+clean; 196 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-05-08T23:59:24.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:23 smithi160 ceph-mon[46152]: pgmap v416: 97 pgs: 97 active+clean; 196 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-05-08T23:59:24.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 719005 6.02 MB/sec execute 560 sec latency 38.529 ms 2024-05-08T23:59:25.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 719005 6.01 MB/sec execute 561 sec latency 1038.647 ms 2024-05-08T23:59:25.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:25 smithi012 ceph-mon[42503]: pgmap v417: 97 pgs: 97 active+clean; 196 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-05-08T23:59:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:25 smithi204 ceph-mon[47373]: pgmap v417: 97 pgs: 97 active+clean; 196 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-05-08T23:59:26.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:25 smithi160 ceph-mon[46152]: pgmap v417: 97 pgs: 97 active+clean; 196 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-05-08T23:59:26.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 721118 6.03 MB/sec execute 562 sec latency 1182.167 ms 2024-05-08T23:59:27.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 722505 6.02 MB/sec execute 563 sec latency 23.656 ms 2024-05-08T23:59:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:27 smithi012 ceph-mon[42503]: pgmap v418: 97 pgs: 97 active+clean; 200 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 198 op/s 2024-05-08T23:59:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:27 smithi204 ceph-mon[47373]: pgmap v418: 97 pgs: 97 active+clean; 200 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 198 op/s 2024-05-08T23:59:28.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:27 smithi160 ceph-mon[46152]: pgmap v418: 97 pgs: 97 active+clean; 200 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 198 op/s 2024-05-08T23:59:28.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 724063 6.03 MB/sec execute 564 sec latency 411.848 ms 2024-05-08T23:59:29.900 INFO:tasks.workunit.client.0.smithi012.stdout: 1 725463 6.03 MB/sec execute 565 sec latency 43.532 ms 2024-05-08T23:59:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:29 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:29 smithi012 ceph-mon[42503]: pgmap v419: 97 pgs: 97 active+clean; 200 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-05-08T23:59:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:29 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:29 smithi204 ceph-mon[47373]: pgmap v419: 97 pgs: 97 active+clean; 200 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-05-08T23:59:30.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:29 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:30.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:29 smithi160 ceph-mon[46152]: pgmap v419: 97 pgs: 97 active+clean; 200 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-05-08T23:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.501+0000 7f4cee7d5700 1 -- 172.21.15.12:0/2165235295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ce8100080 msgr2=0x7f4ce8100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.501+0000 7f4cee7d5700 1 --2- 172.21.15.12:0/2165235295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ce8100080 0x7f4ce8100500 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0099c0 tx=0x7f4cdc009cd0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.502+0000 7f4cee7d5700 1 -- 172.21.15.12:0/2165235295 shutdown_connections 2024-05-08T23:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.502+0000 7f4cee7d5700 1 --2- 172.21.15.12:0/2165235295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ce8100a40 0x7f4ce810cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.502+0000 7f4cee7d5700 1 --2- 172.21.15.12:0/2165235295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce8105f20 0x7f4ce8106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.502+0000 7f4cee7d5700 1 --2- 172.21.15.12:0/2165235295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ce8100080 0x7f4ce8100500 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.502+0000 7f4cee7d5700 1 -- 172.21.15.12:0/2165235295 >> 172.21.15.12:0/2165235295 conn(0x7f4ce8076c10 msgr2=0x7f4ce8077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.502+0000 7f4cee7d5700 1 -- 172.21.15.12:0/2165235295 shutdown_connections 2024-05-08T23:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.502+0000 7f4cee7d5700 1 -- 172.21.15.12:0/2165235295 wait complete. 2024-05-08T23:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.502+0000 7f4cee7d5700 1 Processor -- start 2024-05-08T23:59:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.502+0000 7f4cee7d5700 1 -- start start 2024-05-08T23:59:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4cee7d5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce8100080 0x7f4ce8199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4cee7d5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ce8100a40 0x7f4ce8199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4cee7d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ce8105f20 0x7f4ce819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4cee7d5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce819e400 con 0x7f4ce8105f20 2024-05-08T23:59:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4cee7d5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce819e570 con 0x7f4ce8100080 2024-05-08T23:59:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4cee7d5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce819e730 con 0x7f4ce8100a40 2024-05-08T23:59:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce8100080 0x7f4ce8199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4cecd72700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ce8105f20 0x7f4ce819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ce8100a40 0x7f4ce8199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce8100080 0x7f4ce8199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34148/0 (socket says 172.21.15.12:34148) 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ce8100a40 0x7f4ce8199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:37918/0 (socket says 172.21.15.12:37918) 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce7fff700 1 -- 172.21.15.12:0/944284747 learned_addr learned my addr 172.21.15.12:0/944284747 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce7fff700 1 -- 172.21.15.12:0/944284747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ce8100a40 msgr2=0x7f4ce8199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ce8100a40 0x7f4ce8199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce7fff700 1 -- 172.21.15.12:0/944284747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ce8105f20 msgr2=0x7f4ce819ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ce8105f20 0x7f4ce819ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce7fff700 1 -- 172.21.15.12:0/944284747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc0095f0 con 0x7f4ce8100080 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4cecd72700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ce8105f20 0x7f4ce819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.503+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce8100080 0x7f4ce8199530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800e970 tx=0x7f4cd800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.504+0000 7f4ce57fa700 1 -- 172.21.15.12:0/944284747 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8009790 con 0x7f4ce8100080 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.504+0000 7f4cee7d5700 1 -- 172.21.15.12:0/944284747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce819ea10 con 0x7f4ce8100080 2024-05-08T23:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.504+0000 7f4ce57fa700 1 -- 172.21.15.12:0/944284747 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cd800c7e0 con 0x7f4ce8100080 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.504+0000 7f4ce57fa700 1 -- 172.21.15.12:0/944284747 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8003c70 con 0x7f4ce8100080 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.504+0000 7f4cee7d5700 1 -- 172.21.15.12:0/944284747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce81a6250 con 0x7f4ce8100080 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.506+0000 7f4ce57fa700 1 -- 172.21.15.12:0/944284747 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4cd800cc50 con 0x7f4ce8100080 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.507+0000 7f4ce57fa700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cd00a1290 0x7f4cd00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.507+0000 7f4ce57fa700 1 -- 172.21.15.12:0/944284747 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4cd8014070 con 0x7f4ce8100080 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.507+0000 7f4ce77fe700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cd00a1290 0x7f4cd00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.507+0000 7f4ce77fe700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cd00a1290 0x7f4cd00a3750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009fd0 tx=0x7f4cdc019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cfa98fb80 1 Processor -- start 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cfa98fb80 1 -- start start 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 0x561e189aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189ab270 0x561e189a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e188fc0f0 con 0x561e189a5480 2024-05-08T23:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e188f9ab0 con 0x561e189aa950 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e188f53a0 con 0x561e189ab270 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4ce77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189ab270 0x561e189a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cecd72700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4ce77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189ab270 0x561e189a4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:37928/0 (socket says 172.21.15.12:37928) 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cecd72700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189a58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58426/0 (socket says 172.21.15.12:58426) 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4ce77fe700 1 -- 172.21.15.12:0/2662855850 learned_addr learned my addr 172.21.15.12:0/2662855850 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/2662855850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 0x561e189aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cecd72700 1 -- 172.21.15.12:0/2662855850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189ab270 msgr2=0x561e189a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cecd72700 1 --2- 172.21.15.12:0/2662855850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189ab270 0x561e189a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cecd72700 1 -- 172.21.15.12:0/2662855850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 msgr2=0x561e189aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cecd72700 1 --2- 172.21.15.12:0/2662855850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 0x561e189aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4cecd72700 1 -- 172.21.15.12:0/2662855850 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e18776350 con 0x561e189a5480 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4ce77fe700 1 --2- 172.21.15.12:0/2662855850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189ab270 0x561e189a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.508+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/2662855850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 0x561e189aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.509+0000 7f4cecd72700 1 --2- 172.21.15.12:0/2662855850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189a58a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0003db0 tx=0x7f4ce0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7473a6fcd7e37bd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.509+0000 7f4cce7fc700 1 -- 172.21.15.12:0/2662855850 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0004bb0 con 0x561e189a5480 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.509+0000 7f4cce7fc700 1 -- 172.21.15.12:0/2662855850 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ce000f040 con 0x561e189a5480 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.509+0000 7f4cce7fc700 1 -- 172.21.15.12:0/2662855850 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0007cb0 con 0x561e189a5480 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2662855850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 msgr2=0x561e189a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/2662855850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189a58a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0003db0 tx=0x7f4ce0009920 comp rx=0 tx=0).stop 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2662855850 shutdown_connections 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/2662855850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189a58a0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/2662855850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 0x561e189aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/2662855850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189ab270 0x561e189a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2662855850 >> 172.21.15.12:0/2662855850 conn(0x561e189031b0 msgr2=0x561e1899f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2662855850 shutdown_connections 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2662855850 wait complete. 2024-05-08T23:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 -- start start 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.510+0000 7f4cfa98fb80 1 Processor -- start 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189aa950 0x561e189ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 0x561e189aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e18891500 con 0x561e189a5480 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e188911a0 con 0x561e189ab270 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e18778640 con 0x561e189aa950 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cfa98fb80 10 client.0 _ll_register_callbacks cb 0x561e1889aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4ce7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cecd72700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 0x561e189aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4ce7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58438/0 (socket says 172.21.15.12:58438) 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4ce7fff700 1 -- 172.21.15.12:0/4170683502 learned_addr learned my addr 172.21.15.12:0/4170683502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4ce77fe700 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189aa950 0x561e189ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cecd72700 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 0x561e189aeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:34154/0 (socket says 172.21.15.12:34154) 2024-05-08T23:59:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4ce7fff700 1 -- 172.21.15.12:0/4170683502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189aa950 msgr2=0x561e189ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189aa950 0x561e189ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4ce7fff700 1 -- 172.21.15.12:0/4170683502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 msgr2=0x561e189aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 0x561e189aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4ce7fff700 1 -- 172.21.15.12:0/4170683502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce810d4e0 con 0x561e189a5480 2024-05-08T23:59:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cecd72700 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 0x561e189aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189b1160 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8096a70 tx=0x7f4cd8099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585deec29c55ecf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.511+0000 7f4cc77fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cc77fe700 1 -- 172.21.15.12:0/4170683502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd809a460 con 0x561e189a5480 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cc77fe700 1 -- 172.21.15.12:0/4170683502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4cd809de30 con 0x561e189a5480 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/4170683502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e1884e210 con 0x561e189a5480 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cc77fe700 1 -- 172.21.15.12:0/4170683502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cd80a2480 con 0x561e189a5480 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cfa98fb80 10 client.19146 fetch_fsmap learned FSMap version 58 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/4170683502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7f4ce0003a60 con 0x561e189a5480 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cc77fe700 1 -- 172.21.15.12:0/4170683502 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e1884e210 con 0x561e189a5480 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cc77fe700 1 -- 172.21.15.12:0/4170683502 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 58) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4cd80a1ff0 con 0x561e189a5480 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cfa98fb80 10 client.19146 fetch_fsmap finished waiting for FSMap version 58 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cfa98fb80 10 client.19146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cfa98fb80 20 client.19146 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561e189beb30 0x561e189c0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cfa98fb80 4 client.19146 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/4170683502 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e189747b0 con 0x561e189beb30 2024-05-08T23:59:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.512+0000 7f4ce77fe700 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561e189beb30 0x561e189c0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.513+0000 7f4cc77fe700 10 client.19146 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.513+0000 7f4ce77fe700 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561e189beb30 0x561e189c0f20 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a222a30a1773f264 server_cookie=cfd1802847272ade in_seq=0 out_seq=0 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cc77fe700 1 -- 172.21.15.12:0/4170683502 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e189747b0 con 0x561e189beb30 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cc77fe700 10 client.19146 handle_command_reply: tid=0 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 client.19146 shutdown 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/4170683502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 msgr2=0x561e189b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189b1160 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8096a70 tx=0x7f4cd8099a50 comp rx=0 tx=0).stop 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/4170683502 shutdown_connections 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561e189beb30 0x561e189c0f20 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 0x561e189aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189b1160 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/4170683502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189aa950 0x561e189ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/4170683502 >> 172.21.15.12:0/4170683502 conn(0x561e189031b0 msgr2=0x561e189a94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cc77fe700 0 client.19146 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/4170683502 shutdown_connections 2024-05-08T23:59:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/4170683502 wait complete. 2024-05-08T23:59:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.514+0000 7f4cfa98fb80 20 client.19146 trim_cache size 0 max 16384 2024-05-08T23:59:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.530+0000 7f4cfa98fb80 1 Processor -- start 2024-05-08T23:59:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.530+0000 7f4cfa98fb80 1 -- start start 2024-05-08T23:59:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.530+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189aa950 0x561e1899f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 0x561e189b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e189bc260 con 0x561e189a5480 2024-05-08T23:59:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e189c6110 con 0x561e189ab270 2024-05-08T23:59:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e189a40e0 con 0x561e189aa950 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4ce7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4ce7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58452/0 (socket says 172.21.15.12:58452) 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4ce7fff700 1 -- 172.21.15.12:0/76028538 learned_addr learned my addr 172.21.15.12:0/76028538 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4cecd72700 1 --2- 172.21.15.12:0/76028538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 0x561e189b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4ce77fe700 1 --2- 172.21.15.12:0/76028538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189aa950 0x561e1899f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4ce7fff700 1 -- 172.21.15.12:0/76028538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189aa950 msgr2=0x561e1899f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/76028538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189aa950 0x561e1899f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4ce7fff700 1 -- 172.21.15.12:0/76028538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 msgr2=0x561e189b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/76028538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 0x561e189b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4ce7fff700 1 -- 172.21.15.12:0/76028538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e18722b00 con 0x561e189a5480 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/76028538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189aea50 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800efd0 tx=0x7f4cd809a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0aa262b22b9cbd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4cceffd700 1 -- 172.21.15.12:0/76028538 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd807b080 con 0x561e189a5480 2024-05-08T23:59:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4cceffd700 1 -- 172.21.15.12:0/76028538 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cd809a700 con 0x561e189a5480 2024-05-08T23:59:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.531+0000 7f4cceffd700 1 -- 172.21.15.12:0/76028538 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd809a260 con 0x561e189a5480 2024-05-08T23:59:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.532+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/76028538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 msgr2=0x561e189aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.532+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/76028538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189aea50 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800efd0 tx=0x7f4cd809a180 comp rx=0 tx=0).stop 2024-05-08T23:59:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.532+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/76028538 shutdown_connections 2024-05-08T23:59:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.532+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/76028538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189ab270 0x561e189b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.532+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/76028538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189a5480 0x561e189aea50 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.532+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/76028538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189aa950 0x561e1899f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.532+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/76028538 >> 172.21.15.12:0/76028538 conn(0x561e189031b0 msgr2=0x561e189a35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.532+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/76028538 shutdown_connections 2024-05-08T23:59:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.532+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/76028538 wait complete. 2024-05-08T23:59:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.533+0000 7f4cfa98fb80 1 -- start start 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.533+0000 7f4cfa98fb80 1 Processor -- start 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.533+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189a5480 0x561e189b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.533+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 0x561e189b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.533+0000 7f4cfa98fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189ab270 0x561e189b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.533+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e189b44a0 con 0x561e189ab270 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.533+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e189b4610 con 0x561e189aa950 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.533+0000 7f4cfa98fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e189b47d0 con 0x561e189a5480 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.533+0000 7f4cfa98fb80 10 client.0 _ll_register_callbacks cb 0x561e1889aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4ce7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189a5480 0x561e189b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4cecd72700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189ab270 0x561e189b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4ce7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189a5480 0x561e189b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:37972/0 (socket says 172.21.15.12:37972) 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4cecd72700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189ab270 0x561e189b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58464/0 (socket says 172.21.15.12:58464) 2024-05-08T23:59:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4ce7fff700 1 -- 172.21.15.12:0/2948565532 learned_addr learned my addr 172.21.15.12:0/2948565532 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4ce77fe700 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 0x561e189b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4ce7fff700 1 -- 172.21.15.12:0/2948565532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 msgr2=0x561e189b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 0x561e189b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4ce7fff700 1 -- 172.21.15.12:0/2948565532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189ab270 msgr2=0x561e189b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189ab270 0x561e189b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4ce7fff700 1 -- 172.21.15.12:0/2948565532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce0003a60 con 0x561e189a5480 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4cecd72700 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189ab270 0x561e189b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4ce7fff700 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189a5480 0x561e189b5c70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8099d00 tx=0x7f4cd8003710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=193a89013f07ba77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4cc77fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4cc77fe700 1 -- 172.21.15.12:0/2948565532 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd807b070 con 0x561e189a5480 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4cc77fe700 1 -- 172.21.15.12:0/2948565532 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4cd809de30 con 0x561e189a5480 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2948565532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e188ede60 con 0x561e189a5480 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4cc77fe700 1 -- 172.21.15.12:0/2948565532 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cd809a260 con 0x561e189a5480 2024-05-08T23:59:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4cfa98fb80 10 client.28870 fetch_fsmap learned FSMap version 58 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.534+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2948565532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x561e188bb500 con 0x561e189a5480 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4cc77fe700 1 -- 172.21.15.12:0/2948565532 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e188ede60 con 0x561e189a5480 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4cc77fe700 1 -- 172.21.15.12:0/2948565532 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 58) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4cd8098f60 con 0x561e189a5480 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4cfa98fb80 10 client.28870 fetch_fsmap finished waiting for FSMap version 58 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4cfa98fb80 10 client.28870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4cfa98fb80 20 client.28870 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561e18a14e70 0x561e18a17260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4cfa98fb80 4 client.28870 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2948565532 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561e18743c90 con 0x561e18a14e70 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4ce77fe700 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561e18a14e70 0x561e18a17260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4ce77fe700 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561e18a14e70 0x561e18a17260 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aac9e080ac42353 server_cookie=e987bb78f62edda8 in_seq=0 out_seq=0 2024-05-08T23:59:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.535+0000 7f4cc77fe700 10 client.28870 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.596+0000 7f4cc77fe700 1 -- 172.21.15.12:0/2948565532 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561e18743c90 con 0x561e18a14e70 2024-05-08T23:59:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.596+0000 7f4cc77fe700 10 client.28870 handle_command_reply: tid=0 2024-05-08T23:59:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.596+0000 7f4cfa98fb80 1 client.28870 shutdown 2024-05-08T23:59:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.600+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2948565532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189a5480 msgr2=0x561e189b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.600+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189a5480 0x561e189b5c70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8099d00 tx=0x7f4cd8003710 comp rx=0 tx=0).stop 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.600+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2948565532 shutdown_connections 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e189ab270 0x561e189b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e189a5480 0x561e189b5c70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561e18a14e70 0x561e18a17260 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cfa98fb80 1 --2- 172.21.15.12:0/2948565532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e189aa950 0x561e189b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cc77fe700 0 client.28870 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2948565532 >> 172.21.15.12:0/2948565532 conn(0x561e189031b0 msgr2=0x561e189ba720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2948565532 shutdown_connections 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cfa98fb80 1 -- 172.21.15.12:0/2948565532 wait complete. 2024-05-08T23:59:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cfa98fb80 20 client.28870 trim_cache size 0 max 16384 2024-05-08T23:59:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:59:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:59:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:59:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cee7d5700 1 -- 172.21.15.12:0/944284747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cd00a1290 msgr2=0x7f4cd00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cee7d5700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cd00a1290 0x7f4cd00a3750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009fd0 tx=0x7f4cdc019040 comp rx=0 tx=0).stop 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cee7d5700 1 -- 172.21.15.12:0/944284747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce8100080 msgr2=0x7f4ce8199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.601+0000 7f4cee7d5700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce8100080 0x7f4ce8199530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800e970 tx=0x7f4cd800ec80 comp rx=0 tx=0).stop 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.602+0000 7f4cee7d5700 1 -- 172.21.15.12:0/944284747 shutdown_connections 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.602+0000 7f4cee7d5700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cd00a1290 0x7f4cd00a3750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.602+0000 7f4cee7d5700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ce8105f20 0x7f4ce819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.602+0000 7f4cee7d5700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce8100080 0x7f4ce8199530 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.602+0000 7f4cee7d5700 1 --2- 172.21.15.12:0/944284747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ce8100a40 0x7f4ce8199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.602+0000 7f4cee7d5700 1 -- 172.21.15.12:0/944284747 >> 172.21.15.12:0/944284747 conn(0x7f4ce8076c10 msgr2=0x7f4ce8069730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.602+0000 7f4cee7d5700 1 -- 172.21.15.12:0/944284747 shutdown_connections 2024-05-08T23:59:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:30.602+0000 7f4cee7d5700 1 -- 172.21.15.12:0/944284747 wait complete. 2024-05-08T23:59:30.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 726822 6.03 MB/sec execute 566 sec latency 57.152 ms 2024-05-08T23:59:31.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 728516 6.04 MB/sec execute 567 sec latency 12.357 ms 2024-05-08T23:59:31.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:31 smithi012 ceph-mon[42503]: pgmap v420: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-05-08T23:59:32.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:31 smithi204 ceph-mon[47373]: pgmap v420: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-05-08T23:59:32.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:31 smithi160 ceph-mon[46152]: pgmap v420: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-05-08T23:59:32.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 730364 6.04 MB/sec execute 568 sec latency 23.229 ms 2024-05-08T23:59:33.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 732172 6.05 MB/sec execute 569 sec latency 23.311 ms 2024-05-08T23:59:34.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:34 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:34.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:34 smithi204 ceph-mon[47373]: pgmap v421: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-05-08T23:59:34.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:34 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:34.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:34 smithi160 ceph-mon[46152]: pgmap v421: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-05-08T23:59:34.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 733368 6.05 MB/sec execute 570 sec latency 17.961 ms 2024-05-08T23:59:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:34 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:59:34.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:34 smithi012 ceph-mon[42503]: pgmap v421: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-05-08T23:59:35.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 735416 6.06 MB/sec execute 571 sec latency 13.387 ms 2024-05-08T23:59:35.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:35 smithi012 ceph-mon[42503]: pgmap v422: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-05-08T23:59:36.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:35 smithi204 ceph-mon[47373]: pgmap v422: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-05-08T23:59:36.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:35 smithi160 ceph-mon[46152]: pgmap v422: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-05-08T23:59:36.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 736628 6.06 MB/sec execute 572 sec latency 18.609 ms 2024-05-08T23:59:37.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 738525 6.07 MB/sec execute 573 sec latency 43.655 ms 2024-05-08T23:59:37.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:37 smithi012 ceph-mon[42503]: pgmap v423: 97 pgs: 97 active+clean; 206 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 267 op/s 2024-05-08T23:59:38.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:37 smithi204 ceph-mon[47373]: pgmap v423: 97 pgs: 97 active+clean; 206 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 267 op/s 2024-05-08T23:59:38.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:37 smithi160 ceph-mon[46152]: pgmap v423: 97 pgs: 97 active+clean; 206 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 267 op/s 2024-05-08T23:59:38.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 739794 6.07 MB/sec execute 574 sec latency 13.131 ms 2024-05-08T23:59:39.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 741156 6.07 MB/sec execute 575 sec latency 19.608 ms 2024-05-08T23:59:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:39 smithi012 ceph-mon[42503]: pgmap v424: 97 pgs: 97 active+clean; 206 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-08T23:59:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:39 smithi204 ceph-mon[47373]: pgmap v424: 97 pgs: 97 active+clean; 206 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-08T23:59:40.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:39 smithi160 ceph-mon[46152]: pgmap v424: 97 pgs: 97 active+clean; 206 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-08T23:59:40.046 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:59:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.227+0000 7fe261463700 1 -- 172.21.15.12:0/4079687714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe25c0682e0 msgr2=0x7fe25c0686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.227+0000 7fe261463700 1 --2- 172.21.15.12:0/4079687714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe25c0682e0 0x7fe25c0686c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0099c0 tx=0x7fe24c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.227+0000 7fe261463700 1 -- 172.21.15.12:0/4079687714 shutdown_connections 2024-05-08T23:59:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.227+0000 7fe261463700 1 --2- 172.21.15.12:0/4079687714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe25c106290 0x7fe25c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.227+0000 7fe261463700 1 --2- 172.21.15.12:0/4079687714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe25c0682e0 0x7fe25c0686c0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.227+0000 7fe261463700 1 --2- 172.21.15.12:0/4079687714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe25c068c90 0x7fe25c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.227+0000 7fe261463700 1 -- 172.21.15.12:0/4079687714 >> 172.21.15.12:0/4079687714 conn(0x7fe25c076f40 msgr2=0x7fe25c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.227+0000 7fe261463700 1 -- 172.21.15.12:0/4079687714 shutdown_connections 2024-05-08T23:59:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.227+0000 7fe261463700 1 -- 172.21.15.12:0/4079687714 wait complete. 2024-05-08T23:59:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.227+0000 7fe261463700 1 Processor -- start 2024-05-08T23:59:40.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe261463700 1 -- start start 2024-05-08T23:59:40.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe261463700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe25c0682e0 0x7fe25c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe261463700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe25c106290 0x7fe25c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe261463700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe25c19bcb0 0x7fe25c19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe261463700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c19c730 con 0x7fe25c106290 2024-05-08T23:59:40.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe261463700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c19c8a0 con 0x7fe25c19bcb0 2024-05-08T23:59:40.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe261463700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c19ca60 con 0x7fe25c0682e0 2024-05-08T23:59:40.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe25a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe25c106290 0x7fe25c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe25affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe25c0682e0 0x7fe25c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe25a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe25c106290 0x7fe25c197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49226/0 (socket says 172.21.15.12:49226) 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe25affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe25c0682e0 0x7fe25c197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54516/0 (socket says 172.21.15.12:54516) 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe25a7fc700 1 -- 172.21.15.12:0/402099330 learned_addr learned my addr 172.21.15.12:0/402099330 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe25b7fe700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe25c19bcb0 0x7fe25c19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe25affd700 1 -- 172.21.15.12:0/402099330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe25c19bcb0 msgr2=0x7fe25c19c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe25affd700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe25c19bcb0 0x7fe25c19c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe25affd700 1 -- 172.21.15.12:0/402099330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe25c106290 msgr2=0x7fe25c197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.228+0000 7fe25affd700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe25c106290 0x7fe25c197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.229+0000 7fe25affd700 1 -- 172.21.15.12:0/402099330 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c0095f0 con 0x7fe25c0682e0 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.229+0000 7fe25b7fe700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe25c19bcb0 0x7fe25c19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.229+0000 7fe25affd700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe25c0682e0 0x7fe25c197420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0098e0 tx=0x7fe24c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.229+0000 7fe243fff700 1 -- 172.21.15.12:0/402099330 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c01c070 con 0x7fe25c0682e0 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.229+0000 7fe243fff700 1 -- 172.21.15.12:0/402099330 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24c00fe30 con 0x7fe25c0682e0 2024-05-08T23:59:40.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.229+0000 7fe261463700 1 -- 172.21.15.12:0/402099330 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25c1a3f60 con 0x7fe25c0682e0 2024-05-08T23:59:40.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.229+0000 7fe243fff700 1 -- 172.21.15.12:0/402099330 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c022d20 con 0x7fe25c0682e0 2024-05-08T23:59:40.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.229+0000 7fe261463700 1 -- 172.21.15.12:0/402099330 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25c1a4380 con 0x7fe25c0682e0 2024-05-08T23:59:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.232+0000 7fe243fff700 1 -- 172.21.15.12:0/402099330 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe24c020070 con 0x7fe25c0682e0 2024-05-08T23:59:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.232+0000 7fe243fff700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2440a1310 0x7fe2440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.233+0000 7fe25a7fc700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2440a1310 0x7fe2440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.233+0000 7fe243fff700 1 -- 172.21.15.12:0/402099330 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe24c0cd7c0 con 0x7fe25c0682e0 2024-05-08T23:59:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.233+0000 7fe25a7fc700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2440a1310 0x7fe2440a37d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe25c198920 tx=0x7fe250009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe26d61db80 1 Processor -- start 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe26d61db80 1 -- start start 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aefec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aefe6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefe6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe26d61db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642aef3d150 con 0x5642aefe6890 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe26d61db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642aef3ab10 con 0x5642aefec720 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe26d61db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642aef36400 con 0x5642aefebe00 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe25b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefe6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe25b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefe6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49240/0 (socket says 172.21.15.12:49240) 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.234+0000 7fe25b7fe700 1 -- 172.21.15.12:0/954988864 learned_addr learned my addr 172.21.15.12:0/954988864 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe25affd700 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aefec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe25a7fc700 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aefe6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe25affd700 1 -- 172.21.15.12:0/954988864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 msgr2=0x5642aefe6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe25affd700 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aefe6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe25affd700 1 -- 172.21.15.12:0/954988864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 msgr2=0x5642aefe6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe25affd700 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefe6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe25affd700 1 -- 172.21.15.12:0/954988864 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642aedb73b0 con 0x5642aefebe00 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe25b7fe700 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefe6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe25a7fc700 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aefe6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe25affd700 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aefec1e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0caf50 tx=0x7fe24c09a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d721033e145b813 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe2417fa700 1 -- 172.21.15.12:0/954988864 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c09a9a0 con 0x5642aefebe00 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe2417fa700 1 -- 172.21.15.12:0/954988864 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24c09e7d0 con 0x5642aefebe00 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.235+0000 7fe2417fa700 1 -- 172.21.15.12:0/954988864 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c099e90 con 0x5642aefebe00 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.236+0000 7fe26d61db80 1 -- 172.21.15.12:0/954988864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 msgr2=0x5642aefec1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.236+0000 7fe26d61db80 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aefec1e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0caf50 tx=0x7fe24c09a560 comp rx=0 tx=0).stop 2024-05-08T23:59:40.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.236+0000 7fe26d61db80 1 -- 172.21.15.12:0/954988864 shutdown_connections 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.236+0000 7fe26d61db80 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefe6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.236+0000 7fe26d61db80 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aefec1e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.236+0000 7fe26d61db80 1 --2- 172.21.15.12:0/954988864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aefe6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.236+0000 7fe26d61db80 1 -- 172.21.15.12:0/954988864 >> 172.21.15.12:0/954988864 conn(0x5642aef44210 msgr2=0x5642aefe0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.236+0000 7fe26d61db80 1 -- 172.21.15.12:0/954988864 shutdown_connections 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.236+0000 7fe26d61db80 1 -- 172.21.15.12:0/954988864 wait complete. 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe26d61db80 1 -- start start 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe26d61db80 1 Processor -- start 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefefb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefebe00 0x5642aeff0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefec720 0x5642aeff05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe26d61db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642aeed2560 con 0x5642aefe6890 2024-05-08T23:59:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe26d61db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642aeed2200 con 0x5642aefebe00 2024-05-08T23:59:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe26d61db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642aedb96a0 con 0x5642aefec720 2024-05-08T23:59:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe26d61db80 10 client.0 _ll_register_callbacks cb 0x5642aeedbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe25affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefefb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe25a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefebe00 0x5642aeff0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe25a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefebe00 0x5642aeff0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49708/0 (socket says 172.21.15.12:49708) 2024-05-08T23:59:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe25affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefefb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49252/0 (socket says 172.21.15.12:49252) 2024-05-08T23:59:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.237+0000 7fe25a7fc700 1 -- 172.21.15.12:0/3740394328 learned_addr learned my addr 172.21.15.12:0/3740394328 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe25b7fe700 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefec720 0x5642aeff05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe25a7fc700 1 -- 172.21.15.12:0/3740394328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefec720 msgr2=0x5642aeff05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe25a7fc700 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefec720 0x5642aeff05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe25a7fc700 1 -- 172.21.15.12:0/3740394328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 msgr2=0x5642aefefb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe25a7fc700 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefefb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe25a7fc700 1 -- 172.21.15.12:0/3740394328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe250017070 con 0x5642aefebe00 2024-05-08T23:59:40.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe25affd700 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefefb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe25a7fc700 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefebe00 0x5642aeff0060 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe2500038e0 tx=0x7fe2500047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71221555a4c6a781 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe23eff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe23eff5700 1 -- 172.21.15.12:0/3740394328 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25001e800 con 0x5642aefebe00 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe23eff5700 1 -- 172.21.15.12:0/3740394328 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe25001e960 con 0x5642aefebe00 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.238+0000 7fe26d61db80 1 -- 172.21.15.12:0/3740394328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642aee8f270 con 0x5642aefebe00 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe23eff5700 1 -- 172.21.15.12:0/3740394328 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe25001eb20 con 0x5642aefebe00 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe26d61db80 10 client.28874 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe26d61db80 1 -- 172.21.15.12:0/3740394328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7fe25c1a4380 con 0x5642aefebe00 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe23eff5700 1 -- 172.21.15.12:0/3740394328 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5642aee8f270 con 0x5642aefebe00 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe23eff5700 1 -- 172.21.15.12:0/3740394328 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe25000e3b0 con 0x5642aefebe00 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe26d61db80 10 client.28874 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe26d61db80 10 client.28874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe26d61db80 20 client.28874 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe26d61db80 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642aeffff00 0x5642af002320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe26d61db80 4 client.28874 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe26d61db80 1 -- 172.21.15.12:0/3740394328 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642aefb5810 con 0x5642aeffff00 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.239+0000 7fe25affd700 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642aeffff00 0x5642af002320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.240+0000 7fe23eff5700 10 client.28874 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.240+0000 7fe25affd700 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642aeffff00 0x5642af002320 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b25055a2ab0c19 server_cookie=8de9543e6b90b2ad in_seq=0 out_seq=0 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.240+0000 7fe23eff5700 1 -- 172.21.15.12:0/3740394328 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642aefb5810 con 0x5642aeffff00 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.240+0000 7fe23eff5700 10 client.28874 handle_command_reply: tid=0 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.240+0000 7fe26d61db80 1 client.28874 shutdown 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 1 -- 172.21.15.12:0/3740394328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefebe00 msgr2=0x5642aeff0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefebe00 0x5642aeff0060 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe2500038e0 tx=0x7fe2500047c0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 1 -- 172.21.15.12:0/3740394328 shutdown_connections 2024-05-08T23:59:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefec720 0x5642aeff05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefefb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642aeffff00 0x5642af002320 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 1 --2- 172.21.15.12:0/3740394328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefebe00 0x5642aeff0060 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe23eff5700 0 client.28874 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 1 -- 172.21.15.12:0/3740394328 >> 172.21.15.12:0/3740394328 conn(0x5642aef44210 msgr2=0x5642aefe05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 1 -- 172.21.15.12:0/3740394328 shutdown_connections 2024-05-08T23:59:40.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 1 -- 172.21.15.12:0/3740394328 wait complete. 2024-05-08T23:59:40.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.241+0000 7fe26d61db80 20 client.28874 trim_cache size 0 max 16384 2024-05-08T23:59:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe26d61db80 1 Processor -- start 2024-05-08T23:59:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe26d61db80 1 -- start start 2024-05-08T23:59:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefeb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aefe0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aeff03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe26d61db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642aefe6710 con 0x5642aefe6890 2024-05-08T23:59:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe26d61db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642af0074f0 con 0x5642aefec720 2024-05-08T23:59:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe26d61db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642aefdb800 con 0x5642aefebe00 2024-05-08T23:59:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe25affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefeb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe25affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefeb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49258/0 (socket says 172.21.15.12:49258) 2024-05-08T23:59:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe25affd700 1 -- 172.21.15.12:0/644400188 learned_addr learned my addr 172.21.15.12:0/644400188 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe25b7fe700 1 --2- 172.21.15.12:0/644400188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aeff03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.261+0000 7fe25a7fc700 1 --2- 172.21.15.12:0/644400188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aefe0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.262+0000 7fe25affd700 1 -- 172.21.15.12:0/644400188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 msgr2=0x5642aefe0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.262+0000 7fe25affd700 1 --2- 172.21.15.12:0/644400188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aefe0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.262+0000 7fe25affd700 1 -- 172.21.15.12:0/644400188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 msgr2=0x5642aeff03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.262+0000 7fe25affd700 1 --2- 172.21.15.12:0/644400188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aeff03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.262+0000 7fe25affd700 1 -- 172.21.15.12:0/644400188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642aefe6f40 con 0x5642aefe6890 2024-05-08T23:59:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.262+0000 7fe25a7fc700 1 --2- 172.21.15.12:0/644400188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aefe0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:59:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.262+0000 7fe25affd700 1 --2- 172.21.15.12:0/644400188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefeb4a0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0cb550 tx=0x7fe24c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d42b4cab836bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.262+0000 7fe241ffb700 1 -- 172.21.15.12:0/644400188 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c098d40 con 0x5642aefe6890 2024-05-08T23:59:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.262+0000 7fe241ffb700 1 -- 172.21.15.12:0/644400188 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24c099e90 con 0x5642aefe6890 2024-05-08T23:59:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.262+0000 7fe241ffb700 1 -- 172.21.15.12:0/644400188 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c0a2900 con 0x5642aefe6890 2024-05-08T23:59:40.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.263+0000 7fe26d61db80 1 -- 172.21.15.12:0/644400188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 msgr2=0x5642aefeb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.263+0000 7fe26d61db80 1 --2- 172.21.15.12:0/644400188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefeb4a0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0cb550 tx=0x7fe24c005c00 comp rx=0 tx=0).stop 2024-05-08T23:59:40.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.263+0000 7fe26d61db80 1 -- 172.21.15.12:0/644400188 shutdown_connections 2024-05-08T23:59:40.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.263+0000 7fe26d61db80 1 --2- 172.21.15.12:0/644400188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aeff03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.263+0000 7fe26d61db80 1 --2- 172.21.15.12:0/644400188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aefeb4a0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.263+0000 7fe26d61db80 1 --2- 172.21.15.12:0/644400188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aefe0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.263+0000 7fe26d61db80 1 -- 172.21.15.12:0/644400188 >> 172.21.15.12:0/644400188 conn(0x5642aef44210 msgr2=0x5642aefee3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.263+0000 7fe26d61db80 1 -- 172.21.15.12:0/644400188 shutdown_connections 2024-05-08T23:59:40.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.263+0000 7fe26d61db80 1 -- 172.21.15.12:0/644400188 wait complete. 2024-05-08T23:59:40.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.263+0000 7fe26d61db80 1 -- start start 2024-05-08T23:59:40.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe26d61db80 1 Processor -- start 2024-05-08T23:59:40.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aeff4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aeff4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe26d61db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aeff54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe26d61db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642af0597d0 con 0x5642aefe6890 2024-05-08T23:59:40.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe26d61db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642af059910 con 0x5642aefec720 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe26d61db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642af059a50 con 0x5642aefebe00 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe26d61db80 10 client.0 _ll_register_callbacks cb 0x5642aeedbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aeff4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25a7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aeff4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aeff54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aeff54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49724/0 (socket says 172.21.15.12:49724) 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25b7fe700 1 -- 172.21.15.12:0/4126223829 learned_addr learned my addr 172.21.15.12:0/4126223829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25a7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aeff4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54558/0 (socket says 172.21.15.12:54558) 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25b7fe700 1 -- 172.21.15.12:0/4126223829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 msgr2=0x5642aeff4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25b7fe700 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aeff4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25b7fe700 1 -- 172.21.15.12:0/4126223829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 msgr2=0x5642aeff4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25b7fe700 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aeff4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.264+0000 7fe25b7fe700 1 -- 172.21.15.12:0/4126223829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642aedb73b0 con 0x5642aefec720 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe25b7fe700 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aeff54b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe254007f70 tx=0x7fe254018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797c04cd14500f1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe23eff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe23eff5700 1 -- 172.21.15.12:0/4126223829 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25400e040 con 0x5642aefec720 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe23eff5700 1 -- 172.21.15.12:0/4126223829 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe254018dd0 con 0x5642aefec720 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe26d61db80 1 -- 172.21.15.12:0/4126223829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe24c09e7d0 con 0x5642aefec720 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe23eff5700 1 -- 172.21.15.12:0/4126223829 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe254004bb0 con 0x5642aefec720 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe26d61db80 10 client.28880 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe26d61db80 1 -- 172.21.15.12:0/4126223829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5642aed63b60 con 0x5642aefec720 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe23eff5700 1 -- 172.21.15.12:0/4126223829 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24c09e7d0 con 0x5642aefec720 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe23eff5700 1 -- 172.21.15.12:0/4126223829 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe2540203f0 con 0x5642aefec720 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe26d61db80 10 client.28880 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:40.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe26d61db80 10 client.28880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:40.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe26d61db80 20 client.28880 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:40.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe26d61db80 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642af05b390 0x5642af05d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe26d61db80 4 client.28880 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:59:40.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.265+0000 7fe26d61db80 1 -- 172.21.15.12:0/4126223829 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5642aef2eec0 con 0x5642af05b390 2024-05-08T23:59:40.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe25affd700 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642af05b390 0x5642af05d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe25affd700 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642af05b390 0x5642af05d7b0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f55c4b6ea74b2741 server_cookie=75cd6659a8c12373 in_seq=0 out_seq=0 2024-05-08T23:59:40.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe23eff5700 10 client.28880 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe23eff5700 1 -- 172.21.15.12:0/4126223829 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5642aef2eec0 con 0x5642af05b390 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe23eff5700 10 client.28880 handle_command_reply: tid=0 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 client.28880 shutdown 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 -- 172.21.15.12:0/4126223829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 msgr2=0x5642aeff54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aeff54b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe254007f70 tx=0x7fe254018420 comp rx=0 tx=0).stop 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 -- 172.21.15.12:0/4126223829 shutdown_connections 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642af05b390 0x5642af05d7b0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5642aefec720 0x5642aeff54b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5642aefe6890 0x5642aeff4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 --2- 172.21.15.12:0/4126223829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642aefebe00 0x5642aeff4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe23eff5700 0 client.28880 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 -- 172.21.15.12:0/4126223829 >> 172.21.15.12:0/4126223829 conn(0x5642aef44210 msgr2=0x5642aeff0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 -- 172.21.15.12:0/4126223829 shutdown_connections 2024-05-08T23:59:40.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.266+0000 7fe26d61db80 1 -- 172.21.15.12:0/4126223829 wait complete. 2024-05-08T23:59:40.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe26d61db80 20 client.28880 trim_cache size 0 max 16384 2024-05-08T23:59:40.270 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:59:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-08T23:59:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-08T23:59:40.270 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:59:40.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 -- 172.21.15.12:0/402099330 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2440a1310 msgr2=0x7fe2440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2440a1310 0x7fe2440a37d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe25c198920 tx=0x7fe250009260 comp rx=0 tx=0).stop 2024-05-08T23:59:40.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 -- 172.21.15.12:0/402099330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe25c0682e0 msgr2=0x7fe25c197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe25c0682e0 0x7fe25c197420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0098e0 tx=0x7fe24c00f690 comp rx=0 tx=0).stop 2024-05-08T23:59:40.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 -- 172.21.15.12:0/402099330 shutdown_connections 2024-05-08T23:59:40.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2440a1310 0x7fe2440a37d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe25c19bcb0 0x7fe25c19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe25c0682e0 0x7fe25c197420 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 --2- 172.21.15.12:0/402099330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe25c106290 0x7fe25c197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 -- 172.21.15.12:0/402099330 >> 172.21.15.12:0/402099330 conn(0x7fe25c076f40 msgr2=0x7fe25c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 -- 172.21.15.12:0/402099330 shutdown_connections 2024-05-08T23:59:40.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.267+0000 7fe261463700 1 -- 172.21.15.12:0/402099330 wait complete. 2024-05-08T23:59:40.276 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:59:40.276 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:59:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.428+0000 7f4d11bfb700 1 -- 172.21.15.12:0/1288121719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d0c100530 msgr2=0x7f4d0c10d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.428+0000 7f4d11bfb700 1 --2- 172.21.15.12:0/1288121719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d0c100530 0x7f4d0c10d1f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009990 tx=0x7f4d00009ca0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.429+0000 7f4d11bfb700 1 -- 172.21.15.12:0/1288121719 shutdown_connections 2024-05-08T23:59:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.429+0000 7f4d11bfb700 1 --2- 172.21.15.12:0/1288121719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0c10d730 0x7f4d0c10fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.429+0000 7f4d11bfb700 1 --2- 172.21.15.12:0/1288121719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c0ffb80 0x7f4d0c0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.429+0000 7f4d11bfb700 1 --2- 172.21.15.12:0/1288121719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d0c100530 0x7f4d0c10d1f0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.429+0000 7f4d11bfb700 1 -- 172.21.15.12:0/1288121719 >> 172.21.15.12:0/1288121719 conn(0x7f4d0c0fb7e0 msgr2=0x7f4d0c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.429+0000 7f4d11bfb700 1 -- 172.21.15.12:0/1288121719 shutdown_connections 2024-05-08T23:59:40.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.429+0000 7f4d11bfb700 1 -- 172.21.15.12:0/1288121719 wait complete. 2024-05-08T23:59:40.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.429+0000 7f4d11bfb700 1 Processor -- start 2024-05-08T23:59:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d11bfb700 1 -- start start 2024-05-08T23:59:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d11bfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d0c0ffb80 0x7f4d0c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d11bfb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0c100530 0x7f4d0c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d11bfb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c10d730 0x7f4d0c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d11bfb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c19e450 con 0x7f4d0c0ffb80 2024-05-08T23:59:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d11bfb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c19e5c0 con 0x7f4d0c10d730 2024-05-08T23:59:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d11bfb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c19e780 con 0x7f4d0c100530 2024-05-08T23:59:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d0c0ffb80 0x7f4d0c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0c100530 0x7f4d0c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d0c0ffb80 0x7f4d0c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49288/0 (socket says 172.21.15.12:49288) 2024-05-08T23:59:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0b7fe700 1 -- 172.21.15.12:0/479969482 learned_addr learned my addr 172.21.15.12:0/479969482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0bfff700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c10d730 0x7f4d0c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0affd700 1 -- 172.21.15.12:0/479969482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c10d730 msgr2=0x7f4d0c19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0affd700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c10d730 0x7f4d0c19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0affd700 1 -- 172.21.15.12:0/479969482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d0c0ffb80 msgr2=0x7f4d0c199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0affd700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d0c0ffb80 0x7f4d0c199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0affd700 1 -- 172.21.15.12:0/479969482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc009520 con 0x7f4d0c100530 2024-05-08T23:59:40.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.430+0000 7f4d0b7fe700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d0c0ffb80 0x7f4d0c199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.431+0000 7f4d0affd700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0c100530 0x7f4d0c199ac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d00000c00 tx=0x7f4d00004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.431+0000 7f4d08ff9700 1 -- 172.21.15.12:0/479969482 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0001d070 con 0x7f4d0c100530 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.431+0000 7f4d08ff9700 1 -- 172.21.15.12:0/479969482 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d00021410 con 0x7f4d0c100530 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.431+0000 7f4d11bfb700 1 -- 172.21.15.12:0/479969482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d000095f0 con 0x7f4d0c100530 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.431+0000 7f4d08ff9700 1 -- 172.21.15.12:0/479969482 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00017650 con 0x7f4d0c100530 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.431+0000 7f4d11bfb700 1 -- 172.21.15.12:0/479969482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c1a60c0 con 0x7f4d0c100530 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.432+0000 7f4d11bfb700 1 -- 172.21.15.12:0/479969482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0c108950 con 0x7f4d0c100530 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.433+0000 7f4d08ff9700 1 -- 172.21.15.12:0/479969482 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4d00021580 con 0x7f4d0c100530 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.434+0000 7f4d08ff9700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cf40a1310 0x7f4cf40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.434+0000 7f4d0b7fe700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cf40a1310 0x7f4cf40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.434+0000 7f4d08ff9700 1 -- 172.21.15.12:0/479969482 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4d000cdfe0 con 0x7f4d0c100530 2024-05-08T23:59:40.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.434+0000 7f4d0b7fe700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cf40a1310 0x7f4cf40a37d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009fd0 tx=0x7f4cfc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.437+0000 7f4d08ff9700 1 -- 172.21.15.12:0/479969482 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4d000974a0 con 0x7f4d0c100530 2024-05-08T23:59:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.715+0000 7f4d11bfb700 1 -- 172.21.15.12:0/479969482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d0c19a830 con 0x7f4d0c100530 2024-05-08T23:59:40.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.716+0000 7f4d08ff9700 1 -- 172.21.15.12:0/479969482 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+10879 (secure 0 0 0) 0x7f4d0009aac0 con 0x7f4d0c100530 2024-05-08T23:59:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-08T23:59:40.710 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-08T23:59:33.411617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_28844":{"gid":28844,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/864772354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":864772354},{"type":"v1","addr":"172.21.15.204:6837","nonce":864772354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:59:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 -- 172.21.15.12:0/479969482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cf40a1310 msgr2=0x7f4cf40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cf40a1310 0x7f4cf40a37d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009fd0 tx=0x7f4cfc009260 comp rx=0 tx=0).stop 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 -- 172.21.15.12:0/479969482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0c100530 msgr2=0x7f4d0c199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0c100530 0x7f4d0c199ac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d00000c00 tx=0x7f4d00004730 comp rx=0 tx=0).stop 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 -- 172.21.15.12:0/479969482 shutdown_connections 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4cf40a1310 0x7f4cf40a37d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c10d730 0x7f4d0c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d0c0ffb80 0x7f4d0c199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 --2- 172.21.15.12:0/479969482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d0c100530 0x7f4d0c199ac0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 -- 172.21.15.12:0/479969482 >> 172.21.15.12:0/479969482 conn(0x7f4d0c0fb7e0 msgr2=0x7f4d0c0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 -- 172.21.15.12:0/479969482 shutdown_connections 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.718+0000 7f4d11bfb700 1 -- 172.21.15.12:0/479969482 wait complete. 2024-05-08T23:59:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 59 2024-05-08T23:59:40.727 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:59:40.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.878+0000 7f995bf27700 1 -- 172.21.15.12:0/2492758036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9954068050 msgr2=0x7f9954068430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.878+0000 7f995bf27700 1 --2- 172.21.15.12:0/2492758036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9954068050 0x7f9954068430 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f9950009990 tx=0x7f9950009ca0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.878+0000 7f995bf27700 1 -- 172.21.15.12:0/2492758036 shutdown_connections 2024-05-08T23:59:40.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.878+0000 7f995bf27700 1 --2- 172.21.15.12:0/2492758036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f995410d940 0x7f995410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.878+0000 7f995bf27700 1 --2- 172.21.15.12:0/2492758036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9954068050 0x7f9954068430 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.878+0000 7f995bf27700 1 --2- 172.21.15.12:0/2492758036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9954068970 0x7f995410d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.878+0000 7f995bf27700 1 -- 172.21.15.12:0/2492758036 >> 172.21.15.12:0/2492758036 conn(0x7f9954076f40 msgr2=0x7f9954077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.878+0000 7f995bf27700 1 -- 172.21.15.12:0/2492758036 shutdown_connections 2024-05-08T23:59:40.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.878+0000 7f995bf27700 1 -- 172.21.15.12:0/2492758036 wait complete. 2024-05-08T23:59:40.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f995bf27700 1 Processor -- start 2024-05-08T23:59:40.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f995bf27700 1 -- start start 2024-05-08T23:59:40.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f995bf27700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9954068050 0x7f99541995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f995bf27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9954068970 0x7f9954199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f995bf27700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f995410d940 0x7f995419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f995bf27700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995419e4d0 con 0x7f9954068970 2024-05-08T23:59:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f995bf27700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995419e640 con 0x7f995410d940 2024-05-08T23:59:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f995bf27700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995419e800 con 0x7f9954068050 2024-05-08T23:59:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f99594c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9954068970 0x7f9954199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f99594c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9954068970 0x7f9954199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49302/0 (socket says 172.21.15.12:49302) 2024-05-08T23:59:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f99594c2700 1 -- 172.21.15.12:0/3986487896 learned_addr learned my addr 172.21.15.12:0/3986487896 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f995a4c4700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f995410d940 0x7f995419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.879+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9954068050 0x7f99541995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f99594c2700 1 -- 172.21.15.12:0/3986487896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9954068050 msgr2=0x7f99541995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f99594c2700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9954068050 0x7f99541995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f99594c2700 1 -- 172.21.15.12:0/3986487896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f995410d940 msgr2=0x7f995419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f99594c2700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f995410d940 0x7f995419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f99594c2700 1 -- 172.21.15.12:0/3986487896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500095f0 con 0x7f9954068970 2024-05-08T23:59:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f995a4c4700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f995410d940 0x7f995419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f99594c2700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9954068970 0x7f9954199af0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f994400d950 tx=0x7f994400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f994affd700 1 -- 172.21.15.12:0/3986487896 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994400f890 con 0x7f9954068970 2024-05-08T23:59:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f994affd700 1 -- 172.21.15.12:0/3986487896 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9944010460 con 0x7f9954068970 2024-05-08T23:59:40.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f995bf27700 1 -- 172.21.15.12:0/3986487896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995419eae0 con 0x7f9954068970 2024-05-08T23:59:40.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f994affd700 1 -- 172.21.15.12:0/3986487896 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994400e610 con 0x7f9954068970 2024-05-08T23:59:40.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.880+0000 7f995bf27700 1 -- 172.21.15.12:0/3986487896 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99541a6320 con 0x7f9954068970 2024-05-08T23:59:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.883+0000 7f994affd700 1 -- 172.21.15.12:0/3986487896 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f99440108d0 con 0x7f9954068970 2024-05-08T23:59:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.883+0000 7f994affd700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99400a1310 0x7f99400a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.884+0000 7f994affd700 1 -- 172.21.15.12:0/3986487896 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f99440cc9e0 con 0x7f9954068970 2024-05-08T23:59:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.884+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99400a1310 0x7f99400a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.884+0000 7f99680e1b80 1 Processor -- start 2024-05-08T23:59:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.884+0000 7f99680e1b80 1 -- start start 2024-05-08T23:59:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.884+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99400a1310 0x7f99400a37d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f995000b2e0 tx=0x7f9950005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.884+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed1b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.884+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1b720 0x55e01ed15350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.884+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed15890 0x55e01ed15cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.884+0000 7f99680e1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ec6c150 con 0x55e01ed1ae00 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f99680e1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ec69b10 con 0x55e01ed15890 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f99680e1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ec65400 con 0x55e01ed1b720 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f9959cc3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed1b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f9959cc3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed1b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49312/0 (socket says 172.21.15.12:49312) 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f9959cc3700 1 -- 172.21.15.12:0/3022784866 learned_addr learned my addr 172.21.15.12:0/3022784866 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f995a4c4700 1 --2- 172.21.15.12:0/3022784866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed15890 0x55e01ed15cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f99594c2700 1 --2- 172.21.15.12:0/3022784866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1b720 0x55e01ed15350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f9959cc3700 1 -- 172.21.15.12:0/3022784866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1b720 msgr2=0x55e01ed15350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3022784866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1b720 0x55e01ed15350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f9959cc3700 1 -- 172.21.15.12:0/3022784866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed15890 msgr2=0x55e01ed15cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3022784866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed15890 0x55e01ed15cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f9959cc3700 1 -- 172.21.15.12:0/3022784866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e01eae63b0 con 0x55e01ed1ae00 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f995a4c4700 1 --2- 172.21.15.12:0/3022784866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed15890 0x55e01ed15cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3022784866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed1b1e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f9950009990 tx=0x7f9950020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60496823e583efc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.885+0000 7f993ffff700 1 -- 172.21.15.12:0/3022784866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950004700 con 0x55e01ed1ae00 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.886+0000 7f993ffff700 1 -- 172.21.15.12:0/3022784866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f995001e040 con 0x55e01ed1ae00 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.886+0000 7f993ffff700 1 -- 172.21.15.12:0/3022784866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000f9e0 con 0x55e01ed1ae00 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.886+0000 7f99680e1b80 1 -- 172.21.15.12:0/3022784866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 msgr2=0x55e01ed1b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.886+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3022784866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed1b1e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f9950009990 tx=0x7f9950020000 comp rx=0 tx=0).stop 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.886+0000 7f99680e1b80 1 -- 172.21.15.12:0/3022784866 shutdown_connections 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.886+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3022784866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed15890 0x55e01ed15cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.886+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3022784866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed1b1e0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.886+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3022784866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1b720 0x55e01ed15350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.886+0000 7f99680e1b80 1 -- 172.21.15.12:0/3022784866 >> 172.21.15.12:0/3022784866 conn(0x55e01ec73210 msgr2=0x55e01ed0fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.886+0000 7f99680e1b80 1 -- 172.21.15.12:0/3022784866 shutdown_connections 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.887+0000 7f99680e1b80 1 -- 172.21.15.12:0/3022784866 wait complete. 2024-05-08T23:59:40.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.887+0000 7f99680e1b80 1 -- start start 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.887+0000 7f99680e1b80 1 Processor -- start 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.887+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 0x55e01ed1eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1ae00 0x55e01ed1f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1b720 0x55e01ed1f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f99680e1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ec01560 con 0x55e01ed15890 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f99680e1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ec01200 con 0x55e01ed1ae00 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f99680e1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01eae86a0 con 0x55e01ed1b720 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f99680e1b80 10 client.0 _ll_register_callbacks cb 0x55e01ec0ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f9959cc3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 0x55e01ed1eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f9959cc3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 0x55e01ed1eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49314/0 (socket says 172.21.15.12:49314) 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f9959cc3700 1 -- 172.21.15.12:0/3243806664 learned_addr learned my addr 172.21.15.12:0/3243806664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f99594c2700 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1ae00 0x55e01ed1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f995a4c4700 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1b720 0x55e01ed1f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f9959cc3700 1 -- 172.21.15.12:0/3243806664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1b720 msgr2=0x55e01ed1f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1b720 0x55e01ed1f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f9959cc3700 1 -- 172.21.15.12:0/3243806664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1ae00 msgr2=0x55e01ed1f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1ae00 0x55e01ed1f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f9959cc3700 1 -- 172.21.15.12:0/3243806664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c003a60 con 0x55e01ed15890 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f99594c2700 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1ae00 0x55e01ed1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 0x55e01ed1eb40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9950032040 tx=0x7f995000f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a02f079a5e9649b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.888+0000 7f993d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f993d7fa700 1 -- 172.21.15.12:0/3243806664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950004700 con 0x55e01ed15890 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f993d7fa700 1 -- 172.21.15.12:0/3243806664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9950004860 con 0x55e01ed15890 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f99680e1b80 1 -- 172.21.15.12:0/3243806664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e01ebbe270 con 0x55e01ed15890 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f993d7fa700 1 -- 172.21.15.12:0/3243806664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f995000be90 con 0x55e01ed15890 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f99680e1b80 10 client.19188 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f99680e1b80 1 -- 172.21.15.12:0/3243806664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f9954110860 con 0x55e01ed15890 2024-05-08T23:59:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f993d7fa700 1 -- 172.21.15.12:0/3243806664 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e01ebbe270 con 0x55e01ed15890 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f993d7fa700 1 -- 172.21.15.12:0/3243806664 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9950024410 con 0x55e01ed15890 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f99680e1b80 10 client.19188 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f99680e1b80 10 client.19188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f99680e1b80 20 client.19188 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e01ed2ed10 0x55e01ed31100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f99680e1b80 4 client.19188 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.889+0000 7f99680e1b80 1 -- 172.21.15.12:0/3243806664 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e01ece4810 con 0x55e01ed2ed10 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.890+0000 7f99594c2700 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e01ed2ed10 0x55e01ed31100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.890+0000 7f99594c2700 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e01ed2ed10 0x55e01ed31100 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7a4ed5b1b258a94 server_cookie=16f934f6aa240d9e in_seq=0 out_seq=0 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.890+0000 7f993d7fa700 10 client.19188 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f993d7fa700 1 -- 172.21.15.12:0/3243806664 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e01ece4810 con 0x55e01ed2ed10 2024-05-08T23:59:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f993d7fa700 10 client.19188 handle_command_reply: tid=0 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 client.19188 shutdown 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 -- 172.21.15.12:0/3243806664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 msgr2=0x55e01ed1eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 0x55e01ed1eb40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9950032040 tx=0x7f995000f870 comp rx=0 tx=0).stop 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 -- 172.21.15.12:0/3243806664 shutdown_connections 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e01ed2ed10 0x55e01ed31100 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1b720 0x55e01ed1f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 0x55e01ed1eb40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3243806664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1ae00 0x55e01ed1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f993d7fa700 0 client.19188 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 -- 172.21.15.12:0/3243806664 >> 172.21.15.12:0/3243806664 conn(0x55e01ec73210 msgr2=0x55e01ed0f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 -- 172.21.15.12:0/3243806664 shutdown_connections 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 1 -- 172.21.15.12:0/3243806664 wait complete. 2024-05-08T23:59:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.891+0000 7f99680e1b80 20 client.19188 trim_cache size 0 max 16384 2024-05-08T23:59:40.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f99680e1b80 1 Processor -- start 2024-05-08T23:59:40.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f99680e1b80 1 -- start start 2024-05-08T23:59:40.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed15890 0x55e01ed2abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed0fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1b720 0x55e01ed1f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f99680e1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ed11e90 con 0x55e01ed1ae00 2024-05-08T23:59:40.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f99680e1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ed20140 con 0x55e01ed1b720 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f99680e1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ed36270 con 0x55e01ed15890 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f99594c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed0fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f9959cc3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed15890 0x55e01ed2abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f9959cc3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed15890 0x55e01ed2abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54626/0 (socket says 172.21.15.12:54626) 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f9959cc3700 1 -- 172.21.15.12:0/749673607 learned_addr learned my addr 172.21.15.12:0/749673607 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.906+0000 7f995a4c4700 1 --2- 172.21.15.12:0/749673607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1b720 0x55e01ed1f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.907+0000 7f9959cc3700 1 -- 172.21.15.12:0/749673607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1b720 msgr2=0x55e01ed1f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.907+0000 7f9959cc3700 1 --2- 172.21.15.12:0/749673607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1b720 0x55e01ed1f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.907+0000 7f9959cc3700 1 -- 172.21.15.12:0/749673607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 msgr2=0x55e01ed0fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.907+0000 7f9959cc3700 1 --2- 172.21.15.12:0/749673607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed0fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.907+0000 7f9959cc3700 1 -- 172.21.15.12:0/749673607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e01ea92b60 con 0x55e01ed15890 2024-05-08T23:59:40.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.907+0000 7f99594c2700 1 --2- 172.21.15.12:0/749673607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed0fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:40.903 INFO:tasks.workunit.client.0.smithi012.stdout: 1 742730 6.08 MB/sec execute 576 sec latency 20.453 ms 2024-05-08T23:59:40.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.907+0000 7f9959cc3700 1 --2- 172.21.15.12:0/749673607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed15890 0x55e01ed2abc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f995000c010 tx=0x7f9950009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d33cacc71c1b168 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.907+0000 7f9948ff9700 1 -- 172.21.15.12:0/749673607 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950024a60 con 0x55e01ed15890 2024-05-08T23:59:40.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.907+0000 7f9948ff9700 1 -- 172.21.15.12:0/749673607 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9950024410 con 0x55e01ed15890 2024-05-08T23:59:40.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.907+0000 7f9948ff9700 1 -- 172.21.15.12:0/749673607 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995002f6a0 con 0x55e01ed15890 2024-05-08T23:59:40.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 -- 172.21.15.12:0/749673607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed15890 msgr2=0x55e01ed2abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 --2- 172.21.15.12:0/749673607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed15890 0x55e01ed2abc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f995000c010 tx=0x7f9950009310 comp rx=0 tx=0).stop 2024-05-08T23:59:40.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 -- 172.21.15.12:0/749673607 shutdown_connections 2024-05-08T23:59:40.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 --2- 172.21.15.12:0/749673607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1b720 0x55e01ed1f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 --2- 172.21.15.12:0/749673607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed15890 0x55e01ed2abc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 --2- 172.21.15.12:0/749673607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed1ae00 0x55e01ed0fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 -- 172.21.15.12:0/749673607 >> 172.21.15.12:0/749673607 conn(0x55e01ec73210 msgr2=0x55e01ed0c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 -- 172.21.15.12:0/749673607 shutdown_connections 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 -- 172.21.15.12:0/749673607 wait complete. 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 -- start start 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.908+0000 7f99680e1b80 1 Processor -- start 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 0x55e01ed260e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1ae00 0x55e01ed26620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99680e1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1b720 0x55e01ed23670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99680e1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ed23d40 con 0x55e01ed15890 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99680e1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ed23eb0 con 0x55e01ed1b720 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99680e1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01ed24070 con 0x55e01ed1ae00 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99680e1b80 10 client.0 _ll_register_callbacks cb 0x55e01ec0ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99594c2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1ae00 0x55e01ed26620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f9959cc3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 0x55e01ed260e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99594c2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1ae00 0x55e01ed26620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54642/0 (socket says 172.21.15.12:54642) 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f995a4c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1b720 0x55e01ed23670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99594c2700 1 -- 172.21.15.12:0/3822248607 learned_addr learned my addr 172.21.15.12:0/3822248607 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99594c2700 1 -- 172.21.15.12:0/3822248607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1b720 msgr2=0x55e01ed23670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99594c2700 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1b720 0x55e01ed23670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99594c2700 1 -- 172.21.15.12:0/3822248607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 msgr2=0x55e01ed260e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99594c2700 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 0x55e01ed260e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99594c2700 1 -- 172.21.15.12:0/3822248607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99541a6320 con 0x55e01ed1ae00 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f99594c2700 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1ae00 0x55e01ed26620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9944099830 tx=0x7f9944099980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217cdc6e12e4a772 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.909+0000 7f993d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f993d7fa700 1 -- 172.21.15.12:0/3822248607 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99440a1fb0 con 0x55e01ed1ae00 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f993d7fa700 1 -- 172.21.15.12:0/3822248607 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9944099d90 con 0x55e01ed1ae00 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f99680e1b80 1 -- 172.21.15.12:0/3822248607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f995001e040 con 0x55e01ed1ae00 2024-05-08T23:59:40.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f993d7fa700 1 -- 172.21.15.12:0/3822248607 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9944099f50 con 0x55e01ed1ae00 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f99680e1b80 10 client.28912 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f99680e1b80 1 -- 172.21.15.12:0/3822248607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f9954110860 con 0x55e01ed1ae00 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f993d7fa700 1 -- 172.21.15.12:0/3822248607 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f995001e040 con 0x55e01ed1ae00 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f993d7fa700 1 -- 172.21.15.12:0/3822248607 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99440a1bb0 con 0x55e01ed1ae00 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f99680e1b80 10 client.28912 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f99680e1b80 10 client.28912 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f99680e1b80 20 client.28912 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e01ed85400 0x55e01ed87820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f99680e1b80 4 client.28912 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f99680e1b80 1 -- 172.21.15.12:0/3822248607 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e01ec5dec0 con 0x55e01ed85400 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.910+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e01ed85400 0x55e01ed87820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f993d7fa700 10 client.28912 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-08T23:59:40.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f9959cc3700 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e01ed85400 0x55e01ed87820 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44225fe6c903f89d server_cookie=671c8ed16b582f12 in_seq=0 out_seq=0 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f993d7fa700 1 -- 172.21.15.12:0/3822248607 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e01ec5dec0 con 0x55e01ed85400 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f993d7fa700 10 client.28912 handle_command_reply: tid=0 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 client.28912 shutdown 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 -- 172.21.15.12:0/3822248607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1ae00 msgr2=0x55e01ed26620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1ae00 0x55e01ed26620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9944099830 tx=0x7f9944099980 comp rx=0 tx=0).stop 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 -- 172.21.15.12:0/3822248607 shutdown_connections 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e01ed1b720 0x55e01ed23670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e01ed15890 0x55e01ed260e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55e01ed85400 0x55e01ed87820 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 --2- 172.21.15.12:0/3822248607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e01ed1ae00 0x55e01ed26620 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f993d7fa700 0 client.28912 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-08T23:59:40.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 -- 172.21.15.12:0/3822248607 >> 172.21.15.12:0/3822248607 conn(0x55e01ec73210 msgr2=0x55e01ed1f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 -- 172.21.15.12:0/3822248607 shutdown_connections 2024-05-08T23:59:40.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.911+0000 7f99680e1b80 1 -- 172.21.15.12:0/3822248607 wait complete. 2024-05-08T23:59:40.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f99680e1b80 20 client.28912 trim_cache size 0 max 16384 2024-05-08T23:59:40.912 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:59:40.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 -- 172.21.15.12:0/3986487896 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99400a1310 msgr2=0x7f99400a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99400a1310 0x7f99400a37d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f995000b2e0 tx=0x7f9950005960 comp rx=0 tx=0).stop 2024-05-08T23:59:40.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 -- 172.21.15.12:0/3986487896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9954068970 msgr2=0x7f9954199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:40.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9954068970 0x7f9954199af0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f994400d950 tx=0x7f994400dc60 comp rx=0 tx=0).stop 2024-05-08T23:59:40.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 -- 172.21.15.12:0/3986487896 shutdown_connections 2024-05-08T23:59:40.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99400a1310 0x7f99400a37d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f995410d940 0x7f995419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9954068050 0x7f99541995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 --2- 172.21.15.12:0/3986487896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9954068970 0x7f9954199af0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:40.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 -- 172.21.15.12:0/3986487896 >> 172.21.15.12:0/3986487896 conn(0x7f9954076f40 msgr2=0x7f995410ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:40.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 -- 172.21.15.12:0/3986487896 shutdown_connections 2024-05-08T23:59:40.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:40.912+0000 7f995bf27700 1 -- 172.21.15.12:0/3986487896 wait complete. 2024-05-08T23:59:40.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:59:40.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:59:40.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:59:40.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:59:40.926 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:59:41.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:59:41.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:59:41.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:59:41.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:59:41.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:40 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:59:41.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:40 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:59:41.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:40 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:59:41.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:40 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-08T23:59:41.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.083+0000 7f63260a5700 1 -- 172.21.15.12:0/3369635402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6320100060 msgr2=0x7f6320100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.083+0000 7f63260a5700 1 --2- 172.21.15.12:0/3369635402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6320100060 0x7f6320100440 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6310009990 tx=0x7f6310009ca0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.083+0000 7f63260a5700 1 -- 172.21.15.12:0/3369635402 shutdown_connections 2024-05-08T23:59:41.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.083+0000 7f63260a5700 1 --2- 172.21.15.12:0/3369635402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6320105160 0x7f632010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.083+0000 7f63260a5700 1 --2- 172.21.15.12:0/3369635402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6320100060 0x7f6320100440 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.083+0000 7f63260a5700 1 --2- 172.21.15.12:0/3369635402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6320100a10 0x7f6320104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.083+0000 7f63260a5700 1 -- 172.21.15.12:0/3369635402 >> 172.21.15.12:0/3369635402 conn(0x7f6320076f40 msgr2=0x7f6320077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.083+0000 7f63260a5700 1 -- 172.21.15.12:0/3369635402 shutdown_connections 2024-05-08T23:59:41.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.083+0000 7f63260a5700 1 -- 172.21.15.12:0/3369635402 wait complete. 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.084+0000 7f63260a5700 1 Processor -- start 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.084+0000 7f63260a5700 1 -- start start 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f63260a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6320100060 0x7f63201962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f63260a5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6320105160 0x7f63201967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f63260a5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f632019ab90 0x7f632019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f63260a5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632019b610 con 0x7f6320100060 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f63260a5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632019b780 con 0x7f632019ab90 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f63260a5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632019b940 con 0x7f6320105160 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6320100060 0x7f63201962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6320100060 0x7f63201962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49352/0 (socket says 172.21.15.12:49352) 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631f7fe700 1 -- 172.21.15.12:0/636389099 learned_addr learned my addr 172.21.15.12:0/636389099 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631ffff700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f632019ab90 0x7f632019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631effd700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6320105160 0x7f63201967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631f7fe700 1 -- 172.21.15.12:0/636389099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6320105160 msgr2=0x7f63201967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631f7fe700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6320105160 0x7f63201967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631f7fe700 1 -- 172.21.15.12:0/636389099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f632019ab90 msgr2=0x7f632019b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631f7fe700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f632019ab90 0x7f632019b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631f7fe700 1 -- 172.21.15.12:0/636389099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63100095f0 con 0x7f6320100060 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.085+0000 7f631ffff700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f632019ab90 0x7f632019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.086+0000 7f631f7fe700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6320100060 0x7f63201962b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6310004760 tx=0x7f6310004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.086+0000 7f631cff9700 1 -- 172.21.15.12:0/636389099 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631001d070 con 0x7f6320100060 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.086+0000 7f631cff9700 1 -- 172.21.15.12:0/636389099 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6310022470 con 0x7f6320100060 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.086+0000 7f63260a5700 1 -- 172.21.15.12:0/636389099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63201a2e40 con 0x7f6320100060 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.086+0000 7f631cff9700 1 -- 172.21.15.12:0/636389099 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310017650 con 0x7f6320100060 2024-05-08T23:59:41.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.086+0000 7f63260a5700 1 -- 172.21.15.12:0/636389099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63201a3200 con 0x7f6320100060 2024-05-08T23:59:41.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.088+0000 7f631cff9700 1 -- 172.21.15.12:0/636389099 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f63100177b0 con 0x7f6320100060 2024-05-08T23:59:41.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.089+0000 7f631cff9700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63080a1290 0x7f63080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.089+0000 7f631cff9700 1 -- 172.21.15.12:0/636389099 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f63100cdea0 con 0x7f6320100060 2024-05-08T23:59:41.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.089+0000 7f631effd700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63080a1290 0x7f63080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.090+0000 7f631effd700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63080a1290 0x7f63080a3750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6320197800 tx=0x7f631400b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.090+0000 7f633225fb80 1 Processor -- start 2024-05-08T23:59:41.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.090+0000 7f633225fb80 1 -- start start 2024-05-08T23:59:41.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.090+0000 7f633225fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 0x56335d4f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.090+0000 7f633225fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4f3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.090+0000 7f633225fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f3890 0x56335d4f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.090+0000 7f633225fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d44a150 con 0x56335d4f3890 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.090+0000 7f633225fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d447b10 con 0x56335d4f8e00 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.090+0000 7f633225fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d443400 con 0x56335d4f9720 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 0x56335d4f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f3890 0x56335d4f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 0x56335d4f91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49830/0 (socket says 172.21.15.12:49830) 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631f7fe700 1 -- 172.21.15.12:0/2419816083 learned_addr learned my addr 172.21.15.12:0/2419816083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631effd700 1 --2- 172.21.15.12:0/2419816083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631ffff700 1 -- 172.21.15.12:0/2419816083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 msgr2=0x56335d4f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631ffff700 1 --2- 172.21.15.12:0/2419816083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631ffff700 1 -- 172.21.15.12:0/2419816083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 msgr2=0x56335d4f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631ffff700 1 --2- 172.21.15.12:0/2419816083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 0x56335d4f91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631ffff700 1 -- 172.21.15.12:0/2419816083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56335d2c43b0 con 0x56335d4f3890 2024-05-08T23:59:41.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631f7fe700 1 --2- 172.21.15.12:0/2419816083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 0x56335d4f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:41.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f631ffff700 1 --2- 172.21.15.12:0/2419816083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f3890 0x56335d4f3cd0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6318003db0 tx=0x7f6318009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8858cd72dacf51ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f6305ffb700 1 -- 172.21.15.12:0/2419816083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318004bb0 con 0x56335d4f3890 2024-05-08T23:59:41.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f6305ffb700 1 -- 172.21.15.12:0/2419816083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f631800f040 con 0x56335d4f3890 2024-05-08T23:59:41.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.091+0000 7f6305ffb700 1 -- 172.21.15.12:0/2419816083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318007c20 con 0x56335d4f3890 2024-05-08T23:59:41.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.092+0000 7f633225fb80 1 -- 172.21.15.12:0/2419816083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f3890 msgr2=0x56335d4f3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.092+0000 7f633225fb80 1 --2- 172.21.15.12:0/2419816083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f3890 0x56335d4f3cd0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6318003db0 tx=0x7f6318009920 comp rx=0 tx=0).stop 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.092+0000 7f633225fb80 1 -- 172.21.15.12:0/2419816083 shutdown_connections 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.092+0000 7f633225fb80 1 --2- 172.21.15.12:0/2419816083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f3890 0x56335d4f3cd0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.092+0000 7f633225fb80 1 --2- 172.21.15.12:0/2419816083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 0x56335d4f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.092+0000 7f633225fb80 1 --2- 172.21.15.12:0/2419816083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.092+0000 7f633225fb80 1 -- 172.21.15.12:0/2419816083 >> 172.21.15.12:0/2419816083 conn(0x56335d451210 msgr2=0x56335d4eddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.092+0000 7f633225fb80 1 -- 172.21.15.12:0/2419816083 shutdown_connections 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.092+0000 7f633225fb80 1 -- 172.21.15.12:0/2419816083 wait complete. 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.093+0000 7f633225fb80 1 -- start start 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.093+0000 7f633225fb80 1 Processor -- start 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f633225fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d4fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f633225fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f633225fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f633225fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d3df560 con 0x56335d4f8e00 2024-05-08T23:59:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f633225fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d3df200 con 0x56335d4f3890 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f633225fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d2c66a0 con 0x56335d4f9720 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f633225fb80 10 client.0 _ll_register_callbacks cb 0x56335d3e8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4fd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49378/0 (socket says 172.21.15.12:49378) 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d4fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631effd700 1 -- 172.21.15.12:0/1532999314 learned_addr learned my addr 172.21.15.12:0/1532999314 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631ffff700 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631f7fe700 1 -- 172.21.15.12:0/1532999314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 msgr2=0x56335d4fd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631f7fe700 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4fd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631f7fe700 1 -- 172.21.15.12:0/1532999314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 msgr2=0x56335d4fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631f7fe700 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4fd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631f7fe700 1 -- 172.21.15.12:0/1532999314 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63201a3200 con 0x56335d4f3890 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631effd700 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:41.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f631f7fe700 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d4fcb60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6310097580 tx=0x7f631009ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32a12bd11c0354a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.094+0000 7f62feffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f62feffd700 1 -- 172.21.15.12:0/1532999314 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63100b2070 con 0x56335d4f3890 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f62feffd700 1 -- 172.21.15.12:0/1532999314 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f631009b080 con 0x56335d4f3890 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f633225fb80 1 -- 172.21.15.12:0/1532999314 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56335d39c270 con 0x56335d4f3890 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f62feffd700 1 -- 172.21.15.12:0/1532999314 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f631009e030 con 0x56335d4f3890 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f633225fb80 10 client.28916 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f633225fb80 1 -- 172.21.15.12:0/1532999314 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f6318003a60 con 0x56335d4f3890 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f62feffd700 1 -- 172.21.15.12:0/1532999314 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63100a7080 con 0x56335d4f3890 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f62feffd700 1 -- 172.21.15.12:0/1532999314 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f63100b9170 con 0x56335d4f3890 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f633225fb80 10 client.28916 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f633225fb80 10 client.28916 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f633225fb80 20 client.28916 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f633225fb80 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56335d50ab20 0x56335d513b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f633225fb80 4 client.28916 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f633225fb80 1 -- 172.21.15.12:0/1532999314 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56335d4c2810 con 0x56335d50ab20 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.095+0000 7f631effd700 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56335d50ab20 0x56335d513b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.096+0000 7f631effd700 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56335d50ab20 0x56335d513b80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2780dc0b5bd9e7ff server_cookie=8e06ee2e99e8160 in_seq=0 out_seq=0 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.096+0000 7f62feffd700 10 client.28916 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f62feffd700 1 -- 172.21.15.12:0/1532999314 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56335d4c2810 con 0x56335d50ab20 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f62feffd700 10 client.28916 handle_command_reply: tid=0 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 client.28916 shutdown 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 -- 172.21.15.12:0/1532999314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 msgr2=0x56335d4fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d4fcb60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6310097580 tx=0x7f631009ac40 comp rx=0 tx=0).stop 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 -- 172.21.15.12:0/1532999314 shutdown_connections 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56335d50ab20 0x56335d513b80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4fd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d4fcb60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f62feffd700 0 client.28916 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:59:41.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 --2- 172.21.15.12:0/1532999314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 -- 172.21.15.12:0/1532999314 >> 172.21.15.12:0/1532999314 conn(0x56335d451210 msgr2=0x56335d4ed5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 -- 172.21.15.12:0/1532999314 shutdown_connections 2024-05-08T23:59:41.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 1 -- 172.21.15.12:0/1532999314 wait complete. 2024-05-08T23:59:41.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.097+0000 7f633225fb80 20 client.28916 trim_cache size 0 max 16384 2024-05-08T23:59:41.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.112+0000 7f633225fb80 1 Processor -- start 2024-05-08T23:59:41.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.112+0000 7f633225fb80 1 -- start start 2024-05-08T23:59:41.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.112+0000 7f633225fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f3890 0x56335d4f3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.112+0000 7f633225fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 0x56335d4edd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.112+0000 7f633225fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f9720 0x56335d4fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.112+0000 7f633225fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d4e6070 con 0x56335d4f9720 2024-05-08T23:59:41.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.112+0000 7f633225fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d4f9560 con 0x56335d4f8e00 2024-05-08T23:59:41.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.112+0000 7f633225fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d4ef640 con 0x56335d4f3890 2024-05-08T23:59:41.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f9720 0x56335d4fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 0x56335d4edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f9720 0x56335d4fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49384/0 (socket says 172.21.15.12:49384) 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631ffff700 1 -- 172.21.15.12:0/4029170756 learned_addr learned my addr 172.21.15.12:0/4029170756 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631f7fe700 1 --2- 172.21.15.12:0/4029170756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f3890 0x56335d4f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631ffff700 1 -- 172.21.15.12:0/4029170756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f3890 msgr2=0x56335d4f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631ffff700 1 --2- 172.21.15.12:0/4029170756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f3890 0x56335d4f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631ffff700 1 -- 172.21.15.12:0/4029170756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 msgr2=0x56335d4edd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631ffff700 1 --2- 172.21.15.12:0/4029170756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 0x56335d4edd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631ffff700 1 -- 172.21.15.12:0/4029170756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56335d270b60 con 0x56335d4f9720 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f631ffff700 1 --2- 172.21.15.12:0/4029170756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f9720 0x56335d4fd400 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f63180060b0 tx=0x7f631800afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49724d5ab8d405b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f63067fc700 1 -- 172.21.15.12:0/4029170756 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318009210 con 0x56335d4f9720 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f63067fc700 1 -- 172.21.15.12:0/4029170756 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6318009370 con 0x56335d4f9720 2024-05-08T23:59:41.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.113+0000 7f63067fc700 1 -- 172.21.15.12:0/4029170756 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631800e4f0 con 0x56335d4f9720 2024-05-08T23:59:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.114+0000 7f633225fb80 1 -- 172.21.15.12:0/4029170756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f9720 msgr2=0x56335d4fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.114+0000 7f633225fb80 1 --2- 172.21.15.12:0/4029170756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f9720 0x56335d4fd400 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f63180060b0 tx=0x7f631800afd0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.114+0000 7f633225fb80 1 -- 172.21.15.12:0/4029170756 shutdown_connections 2024-05-08T23:59:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.114+0000 7f633225fb80 1 --2- 172.21.15.12:0/4029170756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f9720 0x56335d4fd400 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.114+0000 7f633225fb80 1 --2- 172.21.15.12:0/4029170756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f3890 0x56335d4f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.114+0000 7f633225fb80 1 --2- 172.21.15.12:0/4029170756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f8e00 0x56335d4edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.114+0000 7f633225fb80 1 -- 172.21.15.12:0/4029170756 >> 172.21.15.12:0/4029170756 conn(0x56335d451210 msgr2=0x56335d4f3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.114+0000 7f633225fb80 1 -- 172.21.15.12:0/4029170756 shutdown_connections 2024-05-08T23:59:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.114+0000 7f633225fb80 1 -- 172.21.15.12:0/4029170756 wait complete. 2024-05-08T23:59:41.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.115+0000 7f633225fb80 1 -- start start 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.115+0000 7f633225fb80 1 Processor -- start 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.115+0000 7f633225fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d516030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.115+0000 7f633225fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.115+0000 7f633225fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.115+0000 7f633225fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d4f6690 con 0x56335d4f8e00 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.115+0000 7f633225fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d4f6800 con 0x56335d4f3890 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.115+0000 7f633225fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335d4f4f60 con 0x56335d4f9720 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.115+0000 7f633225fb80 10 client.0 _ll_register_callbacks cb 0x56335d3e8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d516030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49394/0 (socket says 172.21.15.12:49394) 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d516030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49870/0 (socket says 172.21.15.12:49870) 2024-05-08T23:59:41.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631effd700 1 -- 172.21.15.12:0/475330543 learned_addr learned my addr 172.21.15.12:0/475330543 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631f7fe700 1 -- 172.21.15.12:0/475330543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 msgr2=0x56335d4f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631f7fe700 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631f7fe700 1 -- 172.21.15.12:0/475330543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 msgr2=0x56335d4f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631f7fe700 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631f7fe700 1 -- 172.21.15.12:0/475330543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56335d2c43b0 con 0x56335d4f3890 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631ffff700 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631effd700 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f631f7fe700 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d516030 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6310097af0 tx=0x7f63100b93a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b2d5780eee4a898 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f62feffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f62feffd700 1 -- 172.21.15.12:0/475330543 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63100b2020 con 0x56335d4f3890 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f62feffd700 1 -- 172.21.15.12:0/475330543 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f631009b080 con 0x56335d4f3890 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.116+0000 7f633225fb80 1 -- 172.21.15.12:0/475330543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f631800f040 con 0x56335d4f3890 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f62feffd700 1 -- 172.21.15.12:0/475330543 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f631009e040 con 0x56335d4f3890 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f633225fb80 10 client.28922 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:41.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f633225fb80 1 -- 172.21.15.12:0/475330543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x56335d409560 con 0x56335d4f3890 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f62feffd700 1 -- 172.21.15.12:0/475330543 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63100a7020 con 0x56335d4f3890 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f62feffd700 1 -- 172.21.15.12:0/475330543 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f63100b3ba0 con 0x56335d4f3890 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f633225fb80 10 client.28922 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f633225fb80 10 client.28922 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f633225fb80 20 client.28922 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f633225fb80 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56335d568260 0x56335d56a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f633225fb80 4 client.28922 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f633225fb80 1 -- 172.21.15.12:0/475330543 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56335d43bec0 con 0x56335d568260 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.117+0000 7f631effd700 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56335d568260 0x56335d56a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f62feffd700 10 client.28922 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f631effd700 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56335d568260 0x56335d56a680 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1596c2da637e478 server_cookie=ac1468d82f8f9423 in_seq=0 out_seq=0 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f62feffd700 1 -- 172.21.15.12:0/475330543 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56335d43bec0 con 0x56335d568260 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f62feffd700 10 client.28922 handle_command_reply: tid=0 2024-05-08T23:59:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 client.28922 shutdown 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 -- 172.21.15.12:0/475330543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 msgr2=0x56335d516030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d516030 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6310097af0 tx=0x7f63100b93a0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 -- 172.21.15.12:0/475330543 shutdown_connections 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56335d568260 0x56335d56a680 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56335d4f9720 0x56335d4f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56335d4f3890 0x56335d516030 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 --2- 172.21.15.12:0/475330543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56335d4f8e00 0x56335d4f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f62feffd700 0 client.28922 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 -- 172.21.15.12:0/475330543 >> 172.21.15.12:0/475330543 conn(0x56335d451210 msgr2=0x56335d5081b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 -- 172.21.15.12:0/475330543 shutdown_connections 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 1 -- 172.21.15.12:0/475330543 wait complete. 2024-05-08T23:59:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.118+0000 7f633225fb80 20 client.28922 trim_cache size 0 max 16384 2024-05-08T23:59:41.119 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:59:41.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 -- 172.21.15.12:0/636389099 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63080a1290 msgr2=0x7f63080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63080a1290 0x7f63080a3750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6320197800 tx=0x7f631400b410 comp rx=0 tx=0).stop 2024-05-08T23:59:41.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 -- 172.21.15.12:0/636389099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6320100060 msgr2=0x7f63201962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6320100060 0x7f63201962b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6310004760 tx=0x7f6310004790 comp rx=0 tx=0).stop 2024-05-08T23:59:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 -- 172.21.15.12:0/636389099 shutdown_connections 2024-05-08T23:59:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f63080a1290 0x7f63080a3750 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f632019ab90 0x7f632019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6320100060 0x7f63201962b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 --2- 172.21.15.12:0/636389099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6320105160 0x7f63201967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 -- 172.21.15.12:0/636389099 >> 172.21.15.12:0/636389099 conn(0x7f6320076f40 msgr2=0x7f63200fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 -- 172.21.15.12:0/636389099 shutdown_connections 2024-05-08T23:59:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.119+0000 7f63260a5700 1 -- 172.21.15.12:0/636389099 wait complete. 2024-05-08T23:59:41.127 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:59:41.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.276+0000 7fd506782700 1 -- 172.21.15.12:0/1795241085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd500105090 msgr2=0x7fd50010cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.276+0000 7fd506782700 1 --2- 172.21.15.12:0/1795241085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd500105090 0x7fd50010cc10 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800b0a0 tx=0x7fd4f800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.276+0000 7fd506782700 1 -- 172.21.15.12:0/1795241085 shutdown_connections 2024-05-08T23:59:41.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.276+0000 7fd506782700 1 --2- 172.21.15.12:0/1795241085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd500105090 0x7fd50010cc10 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.276+0000 7fd506782700 1 --2- 172.21.15.12:0/1795241085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5000fff70 0x7fd500100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.276+0000 7fd506782700 1 --2- 172.21.15.12:0/1795241085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd500100920 0x7fd500104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.276+0000 7fd506782700 1 -- 172.21.15.12:0/1795241085 >> 172.21.15.12:0/1795241085 conn(0x7fd5000fb7b0 msgr2=0x7fd5000fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.276+0000 7fd506782700 1 -- 172.21.15.12:0/1795241085 shutdown_connections 2024-05-08T23:59:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.276+0000 7fd506782700 1 -- 172.21.15.12:0/1795241085 wait complete. 2024-05-08T23:59:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd506782700 1 Processor -- start 2024-05-08T23:59:41.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd506782700 1 -- start start 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd506782700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5000fff70 0x7fd500197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd506782700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd500105090 0x7fd500197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd506782700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50019bd20 0x7fd50019c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd506782700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50019c7a0 con 0x7fd50019bd20 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd506782700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50019c910 con 0x7fd500105090 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd506782700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50019cad0 con 0x7fd5000fff70 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd504d1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50019bd20 0x7fd50019c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd504d1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50019bd20 0x7fd50019c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49426/0 (socket says 172.21.15.12:49426) 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd504d1f700 1 -- 172.21.15.12:0/1441303883 learned_addr learned my addr 172.21.15.12:0/1441303883 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.277+0000 7fd4ff7fe700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd500105090 0x7fd500197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd4fffff700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5000fff70 0x7fd500197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd504d1f700 1 -- 172.21.15.12:0/1441303883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5000fff70 msgr2=0x7fd500197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd504d1f700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5000fff70 0x7fd500197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd504d1f700 1 -- 172.21.15.12:0/1441303883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd500105090 msgr2=0x7fd500197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd504d1f700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd500105090 0x7fd500197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd504d1f700 1 -- 172.21.15.12:0/1441303883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8009d00 con 0x7fd50019bd20 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd504d1f700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50019bd20 0x7fd50019c1a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8006130 tx=0x7fd4f800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd4fd7fa700 1 -- 172.21.15.12:0/1441303883 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800e040 con 0x7fd50019bd20 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd4fd7fa700 1 -- 172.21.15.12:0/1441303883 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4f8007e50 con 0x7fd50019bd20 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd506782700 1 -- 172.21.15.12:0/1441303883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5001a3fd0 con 0x7fd50019bd20 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd4fd7fa700 1 -- 172.21.15.12:0/1441303883 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f801b6b0 con 0x7fd50019bd20 2024-05-08T23:59:41.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.278+0000 7fd506782700 1 -- 172.21.15.12:0/1441303883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5001a4450 con 0x7fd50019bd20 2024-05-08T23:59:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.281+0000 7fd4fd7fa700 1 -- 172.21.15.12:0/1441303883 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd4f801d480 con 0x7fd50019bd20 2024-05-08T23:59:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.282+0000 7fd4fd7fa700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd4e80a1280 0x7fd4e80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.282+0000 7fd4fd7fa700 1 -- 172.21.15.12:0/1441303883 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd4f80ce4d0 con 0x7fd50019bd20 2024-05-08T23:59:41.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.282+0000 7fd4fffff700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd4e80a1280 0x7fd4e80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd4fffff700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd4e80a1280 0x7fd4e80a3740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009bf0 tx=0x7fd4f0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd51293cb80 1 Processor -- start 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd51293cb80 1 -- start start 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9f21f0 0x5567cc9f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9ecca0 0x5567cc9ed100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd51293cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cc93f150 con 0x5567cc9f21f0 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd51293cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cc93cb10 con 0x5567cc9f2b10 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd51293cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cc938400 con 0x5567cc9ecca0 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd4ff7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd4ff7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9ec760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49908/0 (socket says 172.21.15.12:49908) 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd4ff7fe700 1 -- 172.21.15.12:0/1731878912 learned_addr learned my addr 172.21.15.12:0/1731878912 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.283+0000 7fd504d1f700 1 --2- 172.21.15.12:0/1731878912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9ecca0 0x5567cc9ed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.284+0000 7fd4ff7fe700 1 -- 172.21.15.12:0/1731878912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9ecca0 msgr2=0x5567cc9ed100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.284+0000 7fd4ff7fe700 1 --2- 172.21.15.12:0/1731878912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9ecca0 0x5567cc9ed100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.284+0000 7fd4ff7fe700 1 -- 172.21.15.12:0/1731878912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9f21f0 msgr2=0x5567cc9f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.284+0000 7fd4ff7fe700 1 --2- 172.21.15.12:0/1731878912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9f21f0 0x5567cc9f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.284+0000 7fd4ff7fe700 1 -- 172.21.15.12:0/1731878912 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567cc7b93b0 con 0x5567cc9f2b10 2024-05-08T23:59:41.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.284+0000 7fd504d1f700 1 --2- 172.21.15.12:0/1731878912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9ecca0 0x5567cc9ed100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:59:41.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.284+0000 7fd4ff7fe700 1 --2- 172.21.15.12:0/1731878912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9ec760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009cb0 tx=0x7fd4f4011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f14d037c2c6a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.284+0000 7fd4e67fc700 1 -- 172.21.15.12:0/1731878912 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4011dd0 con 0x5567cc9f2b10 2024-05-08T23:59:41.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.284+0000 7fd4e67fc700 1 -- 172.21.15.12:0/1731878912 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4f4004510 con 0x5567cc9f2b10 2024-05-08T23:59:41.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.284+0000 7fd4e67fc700 1 -- 172.21.15.12:0/1731878912 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40052f0 con 0x5567cc9f2b10 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.285+0000 7fd51293cb80 1 -- 172.21.15.12:0/1731878912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 msgr2=0x5567cc9ec760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.285+0000 7fd51293cb80 1 --2- 172.21.15.12:0/1731878912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9ec760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009cb0 tx=0x7fd4f4011440 comp rx=0 tx=0).stop 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.285+0000 7fd51293cb80 1 -- 172.21.15.12:0/1731878912 shutdown_connections 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.285+0000 7fd51293cb80 1 --2- 172.21.15.12:0/1731878912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9ecca0 0x5567cc9ed100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.285+0000 7fd51293cb80 1 --2- 172.21.15.12:0/1731878912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9f21f0 0x5567cc9f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.285+0000 7fd51293cb80 1 --2- 172.21.15.12:0/1731878912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9ec760 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.285+0000 7fd51293cb80 1 -- 172.21.15.12:0/1731878912 >> 172.21.15.12:0/1731878912 conn(0x5567cc946210 msgr2=0x5567cc9e2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.285+0000 7fd51293cb80 1 -- 172.21.15.12:0/1731878912 shutdown_connections 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.285+0000 7fd51293cb80 1 -- 172.21.15.12:0/1731878912 wait complete. 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.285+0000 7fd51293cb80 1 -- start start 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd51293cb80 1 Processor -- start 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 0x5567cc9f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f21f0 0x5567cc9f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f2b10 0x5567cc9f6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd51293cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cc8d4560 con 0x5567cc9ecca0 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd51293cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cc8d4200 con 0x5567cc9f21f0 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd51293cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cc7bb6a0 con 0x5567cc9f2b10 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd51293cb80 10 client.0 _ll_register_callbacks cb 0x5567cc8ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd4fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 0x5567cc9f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd504d1f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f2b10 0x5567cc9f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd4fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 0x5567cc9f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49438/0 (socket says 172.21.15.12:49438) 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd4fffff700 1 -- 172.21.15.12:0/488576192 learned_addr learned my addr 172.21.15.12:0/488576192 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd4ff7fe700 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f21f0 0x5567cc9f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd504d1f700 1 -- 172.21.15.12:0/488576192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f21f0 msgr2=0x5567cc9f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd504d1f700 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f21f0 0x5567cc9f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd504d1f700 1 -- 172.21.15.12:0/488576192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 msgr2=0x5567cc9f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd504d1f700 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 0x5567cc9f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd504d1f700 1 -- 172.21.15.12:0/488576192 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f40098e0 con 0x5567cc9f2b10 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.286+0000 7fd4ff7fe700 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f21f0 0x5567cc9f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.287+0000 7fd504d1f700 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f2b10 0x5567cc9f6920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80cbc00 tx=0x7fd4f809b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c5afc52c17e802 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.287+0000 7fd4df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.287+0000 7fd4df7fe700 1 -- 172.21.15.12:0/488576192 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f809f230 con 0x5567cc9f2b10 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.287+0000 7fd51293cb80 1 -- 172.21.15.12:0/488576192 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567cc891270 con 0x5567cc9f2b10 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.287+0000 7fd4df7fe700 1 -- 172.21.15.12:0/488576192 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd4f801d940 con 0x5567cc9f2b10 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.287+0000 7fd4df7fe700 1 -- 172.21.15.12:0/488576192 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4f80bb070 con 0x5567cc9f2b10 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.287+0000 7fd51293cb80 10 client.28942 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.287+0000 7fd51293cb80 1 -- 172.21.15.12:0/488576192 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5567cc765b60 con 0x5567cc9f2b10 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.287+0000 7fd4df7fe700 1 -- 172.21.15.12:0/488576192 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567cc891270 con 0x5567cc9f2b10 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.287+0000 7fd4df7fe700 1 -- 172.21.15.12:0/488576192 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd4f80860b0 con 0x5567cc9f2b10 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.288+0000 7fd51293cb80 10 client.28942 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.288+0000 7fd51293cb80 10 client.28942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.288+0000 7fd51293cb80 20 client.28942 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.288+0000 7fd51293cb80 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567cca06480 0x5567cca088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.288+0000 7fd51293cb80 4 client.28942 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.288+0000 7fd51293cb80 1 -- 172.21.15.12:0/488576192 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567cc9b7810 con 0x5567cca06480 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.288+0000 7fd4fffff700 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567cca06480 0x5567cca088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.288+0000 7fd4df7fe700 10 client.28942 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.288+0000 7fd4fffff700 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567cca06480 0x5567cca088a0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eef1522e8edb88f server_cookie=60a8ab7ac082ec4d in_seq=0 out_seq=0 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd4df7fe700 1 -- 172.21.15.12:0/488576192 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567cc9b7810 con 0x5567cca06480 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd4df7fe700 10 client.28942 handle_command_reply: tid=0 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 client.28942 shutdown 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 -- 172.21.15.12:0/488576192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f2b10 msgr2=0x5567cc9f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f2b10 0x5567cc9f6920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80cbc00 tx=0x7fd4f809b4a0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 -- 172.21.15.12:0/488576192 shutdown_connections 2024-05-08T23:59:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f2b10 0x5567cc9f6920 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 0x5567cc9f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567cca06480 0x5567cca088a0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 --2- 172.21.15.12:0/488576192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f21f0 0x5567cc9f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 -- 172.21.15.12:0/488576192 >> 172.21.15.12:0/488576192 conn(0x5567cc946210 msgr2=0x5567cc9e25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd4df7fe700 0 client.28942 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 -- 172.21.15.12:0/488576192 shutdown_connections 2024-05-08T23:59:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 1 -- 172.21.15.12:0/488576192 wait complete. 2024-05-08T23:59:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.289+0000 7fd51293cb80 20 client.28942 trim_cache size 0 max 16384 2024-05-08T23:59:41.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd51293cb80 1 Processor -- start 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd51293cb80 1 -- start start 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9ecca0 0x5567cca02010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cca026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9f2b10 0x5567cc9e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd51293cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cc9ed660 con 0x5567cc9f2b10 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd51293cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cc9dfdc0 con 0x5567cc9ecca0 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd51293cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cc9e4640 con 0x5567cc9f21f0 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd4ff7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cca026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd504d1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9f2b10 0x5567cc9e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd4ff7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cca026b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54726/0 (socket says 172.21.15.12:54726) 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd4ff7fe700 1 -- 172.21.15.12:0/847199280 learned_addr learned my addr 172.21.15.12:0/847199280 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:41.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.304+0000 7fd4fffff700 1 --2- 172.21.15.12:0/847199280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9ecca0 0x5567cca02010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.305+0000 7fd4ff7fe700 1 -- 172.21.15.12:0/847199280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9ecca0 msgr2=0x5567cca02010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.305+0000 7fd4ff7fe700 1 --2- 172.21.15.12:0/847199280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9ecca0 0x5567cca02010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.305+0000 7fd4ff7fe700 1 -- 172.21.15.12:0/847199280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9f2b10 msgr2=0x5567cc9e2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.305+0000 7fd4ff7fe700 1 --2- 172.21.15.12:0/847199280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9f2b10 0x5567cc9e2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.305+0000 7fd4ff7fe700 1 -- 172.21.15.12:0/847199280 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567cc8fe560 con 0x5567cc9f21f0 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.305+0000 7fd504d1f700 1 --2- 172.21.15.12:0/847199280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9f2b10 0x5567cc9e2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.305+0000 7fd4ff7fe700 1 --2- 172.21.15.12:0/847199280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cca026b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009cb0 tx=0x7fd4f4005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0894728471b0d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.305+0000 7fd4e6ffd700 1 -- 172.21.15.12:0/847199280 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4004d60 con 0x5567cc9f21f0 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.305+0000 7fd4e6ffd700 1 -- 172.21.15.12:0/847199280 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4f4011bd0 con 0x5567cc9f21f0 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.305+0000 7fd4e6ffd700 1 -- 172.21.15.12:0/847199280 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4005650 con 0x5567cc9f21f0 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.306+0000 7fd51293cb80 1 -- 172.21.15.12:0/847199280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 msgr2=0x5567cca026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.306+0000 7fd51293cb80 1 --2- 172.21.15.12:0/847199280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cca026b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009cb0 tx=0x7fd4f4005ce0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.306+0000 7fd51293cb80 1 -- 172.21.15.12:0/847199280 shutdown_connections 2024-05-08T23:59:41.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.306+0000 7fd51293cb80 1 --2- 172.21.15.12:0/847199280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9f2b10 0x5567cc9e2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.306+0000 7fd51293cb80 1 --2- 172.21.15.12:0/847199280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9ecca0 0x5567cca02010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.306+0000 7fd51293cb80 1 --2- 172.21.15.12:0/847199280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cca026b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.306+0000 7fd51293cb80 1 -- 172.21.15.12:0/847199280 >> 172.21.15.12:0/847199280 conn(0x5567cc946210 msgr2=0x5567cc9df210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.306+0000 7fd51293cb80 1 -- 172.21.15.12:0/847199280 shutdown_connections 2024-05-08T23:59:41.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.306+0000 7fd51293cb80 1 -- 172.21.15.12:0/847199280 wait complete. 2024-05-08T23:59:41.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.306+0000 7fd51293cb80 1 -- start start 2024-05-08T23:59:41.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd51293cb80 1 Processor -- start 2024-05-08T23:59:41.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 0x5567cc9fb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cc9fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd51293cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd51293cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cca5b980 con 0x5567cc9ecca0 2024-05-08T23:59:41.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd51293cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cca5bac0 con 0x5567cc9f2b10 2024-05-08T23:59:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd51293cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567cca5bc00 con 0x5567cc9f21f0 2024-05-08T23:59:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd51293cb80 10 client.0 _ll_register_callbacks cb 0x5567cc8ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd4fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 0x5567cc9fb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd4ff7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cc9fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd504d1f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd504d1f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9fbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49934/0 (socket says 172.21.15.12:49934) 2024-05-08T23:59:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd4ff7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cc9fba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54740/0 (socket says 172.21.15.12:54740) 2024-05-08T23:59:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd4fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 0x5567cc9fb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49450/0 (socket says 172.21.15.12:49450) 2024-05-08T23:59:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd504d1f700 1 -- 172.21.15.12:0/2993953500 learned_addr learned my addr 172.21.15.12:0/2993953500 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd504d1f700 1 -- 172.21.15.12:0/2993953500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 msgr2=0x5567cc9fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd504d1f700 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cc9fba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd504d1f700 1 -- 172.21.15.12:0/2993953500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 msgr2=0x5567cc9fb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd504d1f700 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 0x5567cc9fb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd504d1f700 1 -- 172.21.15.12:0/2993953500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567cc765b60 con 0x5567cc9f2b10 2024-05-08T23:59:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.307+0000 7fd4ff7fe700 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cc9fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd504d1f700 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9fbfc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80a3510 tx=0x7fd4f800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166aa78e0e9271b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd4df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:59:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd4df7fe700 1 -- 172.21.15.12:0/2993953500 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f80bb0b0 con 0x5567cc9f2b10 2024-05-08T23:59:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd4df7fe700 1 -- 172.21.15.12:0/2993953500 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd4f801d940 con 0x5567cc9f2b10 2024-05-08T23:59:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd51293cb80 1 -- 172.21.15.12:0/2993953500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4f4004510 con 0x5567cc9f2b10 2024-05-08T23:59:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd4df7fe700 1 -- 172.21.15.12:0/2993953500 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4f809bb40 con 0x5567cc9f2b10 2024-05-08T23:59:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd51293cb80 10 client.28940 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd51293cb80 1 -- 172.21.15.12:0/2993953500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7fd4f401d040 con 0x5567cc9f2b10 2024-05-08T23:59:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd4df7fe700 1 -- 172.21.15.12:0/2993953500 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4f4004510 con 0x5567cc9f2b10 2024-05-08T23:59:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd4df7fe700 1 -- 172.21.15.12:0/2993953500 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd4f809ab40 con 0x5567cc9f2b10 2024-05-08T23:59:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd51293cb80 10 client.28940 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:41.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd51293cb80 10 client.28940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:41.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd51293cb80 20 client.28940 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:41.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd51293cb80 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567cca5d070 0x5567cca5f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:41.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.308+0000 7fd51293cb80 4 client.28940 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:59:41.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.309+0000 7fd51293cb80 1 -- 172.21.15.12:0/2993953500 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5567cc930ec0 con 0x5567cca5d070 2024-05-08T23:59:41.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.309+0000 7fd4fffff700 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567cca5d070 0x5567cca5f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.309+0000 7fd4df7fe700 10 client.28940 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.309+0000 7fd4fffff700 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567cca5d070 0x5567cca5f490 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b681f1d14050e375 server_cookie=c1958d387c8212e4 in_seq=0 out_seq=0 2024-05-08T23:59:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.309+0000 7fd4df7fe700 1 -- 172.21.15.12:0/2993953500 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5567cc930ec0 con 0x5567cca5d070 2024-05-08T23:59:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.309+0000 7fd4df7fe700 10 client.28940 handle_command_reply: tid=0 2024-05-08T23:59:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.309+0000 7fd51293cb80 1 client.28940 shutdown 2024-05-08T23:59:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.309+0000 7fd51293cb80 1 -- 172.21.15.12:0/2993953500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 msgr2=0x5567cc9fbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.309+0000 7fd51293cb80 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9fbfc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80a3510 tx=0x7fd4f800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd51293cb80 1 -- 172.21.15.12:0/2993953500 shutdown_connections 2024-05-08T23:59:41.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd51293cb80 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567cca5d070 0x5567cca5f490 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd51293cb80 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567cc9f2b10 0x5567cc9fbfc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd51293cb80 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567cc9ecca0 0x5567cc9fb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd51293cb80 1 --2- 172.21.15.12:0/2993953500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567cc9f21f0 0x5567cc9fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd51293cb80 1 -- 172.21.15.12:0/2993953500 >> 172.21.15.12:0/2993953500 conn(0x5567cc946210 msgr2=0x5567cc9f4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd4df7fe700 0 client.28940 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:41.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd51293cb80 1 -- 172.21.15.12:0/2993953500 shutdown_connections 2024-05-08T23:59:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd51293cb80 1 -- 172.21.15.12:0/2993953500 wait complete. 2024-05-08T23:59:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd51293cb80 20 client.28940 trim_cache size 0 max 16384 2024-05-08T23:59:41.318 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-08T23:59:41.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 -- 172.21.15.12:0/1441303883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd4e80a1280 msgr2=0x7fd4e80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd4e80a1280 0x7fd4e80a3740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009bf0 tx=0x7fd4f0009260 comp rx=0 tx=0).stop 2024-05-08T23:59:41.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 -- 172.21.15.12:0/1441303883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50019bd20 msgr2=0x7fd50019c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:41.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50019bd20 0x7fd50019c1a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8006130 tx=0x7fd4f800bdc0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 -- 172.21.15.12:0/1441303883 shutdown_connections 2024-05-08T23:59:41.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd4e80a1280 0x7fd4e80a3740 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50019bd20 0x7fd50019c1a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5000fff70 0x7fd500197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 --2- 172.21.15.12:0/1441303883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd500105090 0x7fd500197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:41.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 -- 172.21.15.12:0/1441303883 >> 172.21.15.12:0/1441303883 conn(0x7fd5000fb7b0 msgr2=0x7fd500068330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:41.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 -- 172.21.15.12:0/1441303883 shutdown_connections 2024-05-08T23:59:41.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:41.310+0000 7fd506782700 1 -- 172.21.15.12:0/1441303883 wait complete. 2024-05-08T23:59:41.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 744271 6.08 MB/sec execute 577 sec latency 19.718 ms 2024-05-08T23:59:41.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:41 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/479969482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:59:41.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:41 smithi012 ceph-mon[42503]: pgmap v425: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-05-08T23:59:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:41 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/479969482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:59:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:41 smithi204 ceph-mon[47373]: pgmap v425: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-05-08T23:59:42.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:41 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/479969482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:59:42.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:41 smithi160 ceph-mon[46152]: pgmap v425: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-05-08T23:59:42.324 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:59:42.324 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:59:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.492+0000 7f7a7e061700 1 -- 172.21.15.12:0/3250172744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a78100a10 msgr2=0x7f7a78104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.492+0000 7f7a7e061700 1 --2- 172.21.15.12:0/3250172744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a78100a10 0x7f7a78104a30 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009990 tx=0x7f7a60009ca0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.492+0000 7f7a7e061700 1 -- 172.21.15.12:0/3250172744 shutdown_connections 2024-05-08T23:59:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.492+0000 7f7a7e061700 1 --2- 172.21.15.12:0/3250172744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a78105160 0x7f7a7810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.492+0000 7f7a7e061700 1 --2- 172.21.15.12:0/3250172744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a78100060 0x7f7a78100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.492+0000 7f7a7e061700 1 --2- 172.21.15.12:0/3250172744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a78100a10 0x7f7a78104a30 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.492+0000 7f7a7e061700 1 -- 172.21.15.12:0/3250172744 >> 172.21.15.12:0/3250172744 conn(0x7f7a78076f40 msgr2=0x7f7a78077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.492+0000 7f7a7e061700 1 -- 172.21.15.12:0/3250172744 shutdown_connections 2024-05-08T23:59:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.492+0000 7f7a7e061700 1 -- 172.21.15.12:0/3250172744 wait complete. 2024-05-08T23:59:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a7e061700 1 Processor -- start 2024-05-08T23:59:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a7e061700 1 -- start start 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a7e061700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a78100060 0x7f7a78197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a7e061700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a78105160 0x7f7a78197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a7e061700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a7819bcc0 0x7f7a7819c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a7e061700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7819c740 con 0x7f7a78100060 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a7e061700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7819c8b0 con 0x7f7a7819bcc0 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a7e061700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7819ca70 con 0x7f7a78105160 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a777fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a78100060 0x7f7a78197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a777fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a78100060 0x7f7a78197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49464/0 (socket says 172.21.15.12:49464) 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a78105160 0x7f7a78197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a7819bcc0 0x7f7a7819c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.493+0000 7f7a777fe700 1 -- 172.21.15.12:0/2952344783 learned_addr learned my addr 172.21.15.12:0/2952344783 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a777fe700 1 -- 172.21.15.12:0/2952344783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a78105160 msgr2=0x7f7a78197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a777fe700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a78105160 0x7f7a78197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a777fe700 1 -- 172.21.15.12:0/2952344783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a7819bcc0 msgr2=0x7f7a7819c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a777fe700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a7819bcc0 0x7f7a7819c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a777fe700 1 -- 172.21.15.12:0/2952344783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a600095f0 con 0x7f7a78100060 2024-05-08T23:59:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a77fff700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a7819bcc0 0x7f7a7819c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a777fe700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a78100060 0x7f7a78197430 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800e9a0 tx=0x7f7a6800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a757fa700 1 -- 172.21.15.12:0/2952344783 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68009790 con 0x7f7a78100060 2024-05-08T23:59:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a7e061700 1 -- 172.21.15.12:0/2952344783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a781a3fd0 con 0x7f7a78100060 2024-05-08T23:59:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a757fa700 1 -- 172.21.15.12:0/2952344783 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a68010430 con 0x7f7a78100060 2024-05-08T23:59:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a757fa700 1 -- 172.21.15.12:0/2952344783 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68018680 con 0x7f7a78100060 2024-05-08T23:59:42.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.494+0000 7f7a7e061700 1 -- 172.21.15.12:0/2952344783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a781a43f0 con 0x7f7a78100060 2024-05-08T23:59:42.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.497+0000 7f7a757fa700 1 -- 172.21.15.12:0/2952344783 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7a680105a0 con 0x7f7a78100060 2024-05-08T23:59:42.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.497+0000 7f7a757fa700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a640a1290 0x7f7a640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.497+0000 7f7a757fa700 1 -- 172.21.15.12:0/2952344783 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7a68014070 con 0x7f7a78100060 2024-05-08T23:59:42.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.497+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a640a1290 0x7f7a640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.498+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a640a1290 0x7f7a640a3750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000b2e0 tx=0x7f7a60005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.498+0000 7f7a8a21bb80 1 Processor -- start 2024-05-08T23:59:42.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.498+0000 7f7a8a21bb80 1 -- start start 2024-05-08T23:59:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.498+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2ee80 0x5604bcb2f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.498+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 0x5604bcb293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.498+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 0x5604bcb29d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.498+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bca801d0 con 0x5604bcb29910 2024-05-08T23:59:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.498+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bca7db90 con 0x5604bcb2f7a0 2024-05-08T23:59:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.498+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bca79480 con 0x5604bcb2ee80 2024-05-08T23:59:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2ee80 0x5604bcb2f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 0x5604bcb293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 0x5604bcb29d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2ee80 0x5604bcb2f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54764/0 (socket says 172.21.15.12:54764) 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a777fe700 1 -- 172.21.15.12:0/3775780848 learned_addr learned my addr 172.21.15.12:0/3775780848 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a6ffff700 1 -- 172.21.15.12:0/3775780848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2ee80 msgr2=0x5604bcb2f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/3775780848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2ee80 0x5604bcb2f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a6ffff700 1 -- 172.21.15.12:0/3775780848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 msgr2=0x5604bcb29d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/3775780848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 0x5604bcb29d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a6ffff700 1 -- 172.21.15.12:0/3775780848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604bc8fa430 con 0x5604bcb2f7a0 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/3775780848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 0x5604bcb293d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006010 tx=0x7f7a60020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c2700ba98a852c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a6dffb700 1 -- 172.21.15.12:0/3775780848 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a600056c0 con 0x5604bcb2f7a0 2024-05-08T23:59:42.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a6dffb700 1 -- 172.21.15.12:0/3775780848 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a6001e040 con 0x5604bcb2f7a0 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.499+0000 7f7a6dffb700 1 -- 172.21.15.12:0/3775780848 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000f840 con 0x5604bcb2f7a0 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.500+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/3775780848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 msgr2=0x5604bcb293d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.500+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/3775780848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 0x5604bcb293d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006010 tx=0x7f7a60020000 comp rx=0 tx=0).stop 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.500+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/3775780848 shutdown_connections 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.500+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/3775780848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 0x5604bcb29d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.500+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/3775780848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2ee80 0x5604bcb2f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.500+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/3775780848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 0x5604bcb293d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.500+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/3775780848 >> 172.21.15.12:0/3775780848 conn(0x5604bca87290 msgr2=0x5604bcb23e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.500+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/3775780848 shutdown_connections 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.500+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/3775780848 wait complete. 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a8a21bb80 1 -- start start 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a8a21bb80 1 Processor -- start 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 0x5604bcb32cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 0x5604bcb33200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2f7a0 0x5604bcb33740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bca155e0 con 0x5604bcb29910 2024-05-08T23:59:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bca15280 con 0x5604bcb2ee80 2024-05-08T23:59:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bc8fc720 con 0x5604bcb2f7a0 2024-05-08T23:59:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a8a21bb80 10 client.0 _ll_register_callbacks cb 0x5604bca1ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a777fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 0x5604bcb32cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 0x5604bcb33200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a777fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 0x5604bcb32cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49478/0 (socket says 172.21.15.12:49478) 2024-05-08T23:59:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2f7a0 0x5604bcb33740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.501+0000 7f7a777fe700 1 -- 172.21.15.12:0/4003845641 learned_addr learned my addr 172.21.15.12:0/4003845641 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:42.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a6ffff700 1 -- 172.21.15.12:0/4003845641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2f7a0 msgr2=0x5604bcb33740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2f7a0 0x5604bcb33740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a6ffff700 1 -- 172.21.15.12:0/4003845641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 msgr2=0x5604bcb32cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 0x5604bcb32cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a6ffff700 1 -- 172.21.15.12:0/4003845641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7810d550 con 0x5604bcb2ee80 2024-05-08T23:59:42.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a777fe700 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 0x5604bcb32cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:42.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 0x5604bcb33200 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7a60032040 tx=0x7f7a6000b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1238df6c5eedf3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:59:42.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a56ffd700 1 -- 172.21.15.12:0/4003845641 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000f870 con 0x5604bcb2ee80 2024-05-08T23:59:42.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a56ffd700 1 -- 172.21.15.12:0/4003845641 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7a60004d10 con 0x5604bcb2ee80 2024-05-08T23:59:42.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/4003845641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604bc9d22f0 con 0x5604bcb2ee80 2024-05-08T23:59:42.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a56ffd700 1 -- 172.21.15.12:0/4003845641 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a60004500 con 0x5604bcb2ee80 2024-05-08T23:59:42.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a8a21bb80 10 client.28952 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:42.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/4003845641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5604bc8a6be0 con 0x5604bcb2ee80 2024-05-08T23:59:42.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.502+0000 7f7a56ffd700 1 -- 172.21.15.12:0/4003845641 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604bc9d22f0 con 0x5604bcb2ee80 2024-05-08T23:59:42.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.503+0000 7f7a56ffd700 1 -- 172.21.15.12:0/4003845641 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7a60039c20 con 0x5604bcb2ee80 2024-05-08T23:59:42.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.503+0000 7f7a8a21bb80 10 client.28952 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:42.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.503+0000 7f7a8a21bb80 10 client.28952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:42.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.503+0000 7f7a8a21bb80 20 client.28952 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:42.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.503+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bcb42e10 0x5604bcb45200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.503+0000 7f7a8a21bb80 4 client.28952 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:42.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.503+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/4003845641 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604bcaf8890 con 0x5604bcb42e10 2024-05-08T23:59:42.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.503+0000 7f7a777fe700 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bcb42e10 0x5604bcb45200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.503+0000 7f7a56ffd700 10 client.28952 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:42.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.503+0000 7f7a777fe700 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bcb42e10 0x5604bcb45200 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0cf3351032b0126 server_cookie=abe67e44a3404ad1 in_seq=0 out_seq=0 2024-05-08T23:59:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a56ffd700 1 -- 172.21.15.12:0/4003845641 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604bcaf8890 con 0x5604bcb42e10 2024-05-08T23:59:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a56ffd700 10 client.28952 handle_command_reply: tid=0 2024-05-08T23:59:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a8a21bb80 1 client.28952 shutdown 2024-05-08T23:59:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/4003845641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 msgr2=0x5604bcb33200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 0x5604bcb33200 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7a60032040 tx=0x7f7a6000b5d0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/4003845641 shutdown_connections 2024-05-08T23:59:42.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb2f7a0 0x5604bcb33740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb29910 0x5604bcb32cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bcb42e10 0x5604bcb45200 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/4003845641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 0x5604bcb33200 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/4003845641 >> 172.21.15.12:0/4003845641 conn(0x5604bca87290 msgr2=0x5604bcb23620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.504+0000 7f7a56ffd700 0 client.28952 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.505+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/4003845641 shutdown_connections 2024-05-08T23:59:42.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.505+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/4003845641 wait complete. 2024-05-08T23:59:42.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.505+0000 7f7a8a21bb80 20 client.28952 trim_cache size 0 max 16384 2024-05-08T23:59:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a8a21bb80 1 Processor -- start 2024-05-08T23:59:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a8a21bb80 1 -- start start 2024-05-08T23:59:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 0x5604bcb23da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2f7a0 0x5604bcb33560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bcb1c0c0 con 0x5604bcb2f7a0 2024-05-08T23:59:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bcb2f4b0 con 0x5604bcb2ee80 2024-05-08T23:59:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bcb2c8f0 con 0x5604bcb29910 2024-05-08T23:59:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2f7a0 0x5604bcb33560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2f7a0 0x5604bcb33560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49484/0 (socket says 172.21.15.12:49484) 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.523+0000 7f7a77fff700 1 -- 172.21.15.12:0/639406993 learned_addr learned my addr 172.21.15.12:0/639406993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/639406993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 0x5604bcb23da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a777fe700 1 -- 172.21.15.12:0/639406993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 msgr2=0x5604bcb23da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a777fe700 1 --2- 172.21.15.12:0/639406993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 0x5604bcb23da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a777fe700 1 -- 172.21.15.12:0/639406993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2f7a0 msgr2=0x5604bcb33560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a777fe700 1 --2- 172.21.15.12:0/639406993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2f7a0 0x5604bcb33560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a777fe700 1 -- 172.21.15.12:0/639406993 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604bcb20560 con 0x5604bcb29910 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a77fff700 1 --2- 172.21.15.12:0/639406993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2f7a0 0x5604bcb33560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a777fe700 1 --2- 172.21.15.12:0/639406993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb293d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a680cab80 tx=0x7f7a6801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed821c4a3c203a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a6e7fc700 1 -- 172.21.15.12:0/639406993 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6809dd70 con 0x5604bcb29910 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a6e7fc700 1 -- 172.21.15.12:0/639406993 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a680a23c0 con 0x5604bcb29910 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.524+0000 7f7a6e7fc700 1 -- 172.21.15.12:0/639406993 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680a1f30 con 0x5604bcb29910 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.525+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/639406993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 msgr2=0x5604bcb293d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.525+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/639406993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb293d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a680cab80 tx=0x7f7a6801a040 comp rx=0 tx=0).stop 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.525+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/639406993 shutdown_connections 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.525+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/639406993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2f7a0 0x5604bcb33560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.525+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/639406993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb293d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.525+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/639406993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2ee80 0x5604bcb23da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.525+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/639406993 >> 172.21.15.12:0/639406993 conn(0x5604bca87290 msgr2=0x5604bcb31590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.525+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/639406993 shutdown_connections 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.525+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/639406993 wait complete. 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.525+0000 7f7a8a21bb80 1 -- start start 2024-05-08T23:59:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a8a21bb80 1 Processor -- start 2024-05-08T23:59:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb3a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2ee80 0x5604bcb388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a8a21bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 0x5604bcb38de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bcb39490 con 0x5604bcb2ee80 2024-05-08T23:59:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bcb39600 con 0x5604bcb2f7a0 2024-05-08T23:59:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a8a21bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bcb99280 con 0x5604bcb29910 2024-05-08T23:59:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a8a21bb80 10 client.0 _ll_register_callbacks cb 0x5604bca1ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb3a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2ee80 0x5604bcb388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb3a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54774/0 (socket says 172.21.15.12:54774) 2024-05-08T23:59:42.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a777fe700 1 -- 172.21.15.12:0/437908501 learned_addr learned my addr 172.21.15.12:0/437908501 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:42.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a77fff700 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 0x5604bcb38de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a777fe700 1 -- 172.21.15.12:0/437908501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 msgr2=0x5604bcb38de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a777fe700 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 0x5604bcb38de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a777fe700 1 -- 172.21.15.12:0/437908501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2ee80 msgr2=0x5604bcb388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a777fe700 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2ee80 0x5604bcb388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a777fe700 1 -- 172.21.15.12:0/437908501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7000e6c0 con 0x5604bcb29910 2024-05-08T23:59:42.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.526+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2ee80 0x5604bcb388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:42.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a777fe700 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb3a560 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a680ca5e0 tx=0x7f7a68096390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=547f5adf98520060 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-08T23:59:42.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a56ffd700 1 -- 172.21.15.12:0/437908501 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68010880 con 0x5604bcb29910 2024-05-08T23:59:42.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a56ffd700 1 -- 172.21.15.12:0/437908501 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7a680b81d0 con 0x5604bcb29910 2024-05-08T23:59:42.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/437908501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a6001e040 con 0x5604bcb29910 2024-05-08T23:59:42.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a56ffd700 1 -- 172.21.15.12:0/437908501 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a680109e0 con 0x5604bcb29910 2024-05-08T23:59:42.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a8a21bb80 10 client.28966 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:42.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/437908501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f7a781a43f0 con 0x5604bcb29910 2024-05-08T23:59:42.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a56ffd700 1 -- 172.21.15.12:0/437908501 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a6001e040 con 0x5604bcb29910 2024-05-08T23:59:42.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a56ffd700 1 -- 172.21.15.12:0/437908501 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7a68094b50 con 0x5604bcb29910 2024-05-08T23:59:42.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a8a21bb80 10 client.28966 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:42.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a8a21bb80 10 client.28966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:42.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.527+0000 7f7a8a21bb80 20 client.28966 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:42.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.528+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bcb9a6c0 0x5604bcb9cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.528+0000 7f7a8a21bb80 4 client.28966 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.528+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/437908501 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5604bca71f40 con 0x5604bcb9a6c0 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.528+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bcb9a6c0 0x5604bcb9cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.528+0000 7f7a6ffff700 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bcb9a6c0 0x5604bcb9cae0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d5a4d9971d3d0f server_cookie=18c7ac15d6c75b6e in_seq=0 out_seq=0 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.528+0000 7f7a56ffd700 10 client.28966 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.529+0000 7f7a56ffd700 1 -- 172.21.15.12:0/437908501 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5604bca71f40 con 0x5604bcb9a6c0 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.529+0000 7f7a56ffd700 10 client.28966 handle_command_reply: tid=0 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.529+0000 7f7a8a21bb80 1 client.28966 shutdown 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/437908501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 msgr2=0x5604bcb3a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb3a560 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a680ca5e0 tx=0x7f7a68096390 comp rx=0 tx=0).stop 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/437908501 shutdown_connections 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604bcb9a6c0 0x5604bcb9cae0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604bcb2f7a0 0x5604bcb38de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a56ffd700 0 client.28966 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:42.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604bcb29910 0x5604bcb3a560 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 1 --2- 172.21.15.12:0/437908501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604bcb2ee80 0x5604bcb388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/437908501 >> 172.21.15.12:0/437908501 conn(0x5604bca87290 msgr2=0x5604bcb2ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/437908501 shutdown_connections 2024-05-08T23:59:42.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 1 -- 172.21.15.12:0/437908501 wait complete. 2024-05-08T23:59:42.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a8a21bb80 20 client.28966 trim_cache size 0 max 16384 2024-05-08T23:59:42.532 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:59:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-08T23:59:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "ad2d8e7c-fcb4-432b-a6d5-b06fb0d7268d", 2024-05-08T23:59:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-08T23:59:42.533 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:59:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a7e061700 1 -- 172.21.15.12:0/2952344783 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a640a1290 msgr2=0x7f7a640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a7e061700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a640a1290 0x7f7a640a3750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000b2e0 tx=0x7f7a60005960 comp rx=0 tx=0).stop 2024-05-08T23:59:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a7e061700 1 -- 172.21.15.12:0/2952344783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a78100060 msgr2=0x7f7a78197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a7e061700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a78100060 0x7f7a78197430 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800e9a0 tx=0x7f7a6800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a7e061700 1 -- 172.21.15.12:0/2952344783 shutdown_connections 2024-05-08T23:59:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a7e061700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a640a1290 0x7f7a640a3750 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a7e061700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a7819bcc0 0x7f7a7819c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a7e061700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a78100060 0x7f7a78197430 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a7e061700 1 --2- 172.21.15.12:0/2952344783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a78105160 0x7f7a78197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.530+0000 7f7a7e061700 1 -- 172.21.15.12:0/2952344783 >> 172.21.15.12:0/2952344783 conn(0x7f7a78076f40 msgr2=0x7f7a780683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.531+0000 7f7a7e061700 1 -- 172.21.15.12:0/2952344783 shutdown_connections 2024-05-08T23:59:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.531+0000 7f7a7e061700 1 -- 172.21.15.12:0/2952344783 wait complete. 2024-05-08T23:59:42.541 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:59:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.694+0000 7f86900da700 1 -- 172.21.15.12:0/2190556578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f868810d940 msgr2=0x7f868810fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.694+0000 7f86900da700 1 --2- 172.21.15.12:0/2190556578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f868810d940 0x7f868810fd30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f868000b0a0 tx=0x7f868000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.694+0000 7f86900da700 1 -- 172.21.15.12:0/2190556578 shutdown_connections 2024-05-08T23:59:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.694+0000 7f86900da700 1 --2- 172.21.15.12:0/2190556578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f868810d940 0x7f868810fd30 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.694+0000 7f86900da700 1 --2- 172.21.15.12:0/2190556578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688068050 0x7f8688068430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.694+0000 7f86900da700 1 --2- 172.21.15.12:0/2190556578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8688068970 0x7f868810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.694+0000 7f86900da700 1 -- 172.21.15.12:0/2190556578 >> 172.21.15.12:0/2190556578 conn(0x7f8688076f40 msgr2=0x7f8688077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.695+0000 7f86900da700 1 -- 172.21.15.12:0/2190556578 shutdown_connections 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.695+0000 7f86900da700 1 -- 172.21.15.12:0/2190556578 wait complete. 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.695+0000 7f86900da700 1 Processor -- start 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.695+0000 7f86900da700 1 -- start start 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.695+0000 7f86900da700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688068050 0x7f8688199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.695+0000 7f86900da700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688068970 0x7f8688199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f86900da700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 0x7f868819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f86900da700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868819e5f0 con 0x7f8688068050 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f86900da700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868819e760 con 0x7f8688068970 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f86900da700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868819e920 con 0x7f868810d940 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868de76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688068050 0x7f8688199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868de76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688068050 0x7f8688199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49496/0 (socket says 172.21.15.12:49496) 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868de76700 1 -- 172.21.15.12:0/2899288066 learned_addr learned my addr 172.21.15.12:0/2899288066 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:42.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868e677700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 0x7f868819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868d675700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688068970 0x7f8688199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868de76700 1 -- 172.21.15.12:0/2899288066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 msgr2=0x7f868819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868de76700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 0x7f868819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868de76700 1 -- 172.21.15.12:0/2899288066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688068970 msgr2=0x7f8688199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868de76700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688068970 0x7f8688199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868de76700 1 -- 172.21.15.12:0/2899288066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684009520 con 0x7f8688068050 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f868de76700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688068050 0x7f8688199720 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f868400eb30 tx=0x7f868400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f867effd700 1 -- 172.21.15.12:0/2899288066 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684009880 con 0x7f8688068050 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f86900da700 1 -- 172.21.15.12:0/2899288066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8680009d00 con 0x7f8688068050 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f867effd700 1 -- 172.21.15.12:0/2899288066 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86840044e0 con 0x7f8688068050 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f867effd700 1 -- 172.21.15.12:0/2899288066 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684003d80 con 0x7f8688068050 2024-05-08T23:59:42.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.696+0000 7f86900da700 1 -- 172.21.15.12:0/2899288066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86881a6260 con 0x7f8688068050 2024-05-08T23:59:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.699+0000 7f867effd700 1 -- 172.21.15.12:0/2899288066 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8684003f20 con 0x7f8688068050 2024-05-08T23:59:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.700+0000 7f867effd700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86740a1290 0x7f86740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.700+0000 7f868d675700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86740a1290 0x7f86740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.700+0000 7f867effd700 1 -- 172.21.15.12:0/2899288066 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8684014070 con 0x7f8688068050 2024-05-08T23:59:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.700+0000 7f869c294b80 1 Processor -- start 2024-05-08T23:59:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.700+0000 7f868d675700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86740a1290 0x7f86740a3750 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f868819ac20 tx=0x7f8678009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f869c294b80 1 -- start start 2024-05-08T23:59:42.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f869c294b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb5b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f869c294b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb55350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f869c294b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 0x55863fb55cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f869c294b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863faac150 con 0x55863fb5b720 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f869c294b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863faa9b10 con 0x55863fb5ae00 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f869c294b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863faa5400 con 0x55863fb55890 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f868d675700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb55350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f868d675700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb55350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49512/0 (socket says 172.21.15.12:49512) 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f868d675700 1 -- 172.21.15.12:0/2049130486 learned_addr learned my addr 172.21.15.12:0/2049130486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f868de76700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb5b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f868e677700 1 --2- 172.21.15.12:0/2049130486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 0x55863fb55cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f868d675700 1 -- 172.21.15.12:0/2049130486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 msgr2=0x55863fb55cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f868d675700 1 --2- 172.21.15.12:0/2049130486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 0x55863fb55cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f868d675700 1 -- 172.21.15.12:0/2049130486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 msgr2=0x55863fb5b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f868d675700 1 --2- 172.21.15.12:0/2049130486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb5b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.701+0000 7f868d675700 1 -- 172.21.15.12:0/2049130486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55863f9263b0 con 0x55863fb5b720 2024-05-08T23:59:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.702+0000 7f868d675700 1 --2- 172.21.15.12:0/2049130486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb55350 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8678011d40 tx=0x7f8678014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4047b78d47e686a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.702+0000 7f8673fff700 1 -- 172.21.15.12:0/2049130486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678011d70 con 0x55863fb5b720 2024-05-08T23:59:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.702+0000 7f8673fff700 1 -- 172.21.15.12:0/2049130486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8678004020 con 0x55863fb5b720 2024-05-08T23:59:42.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.702+0000 7f8673fff700 1 -- 172.21.15.12:0/2049130486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678003b60 con 0x55863fb5b720 2024-05-08T23:59:42.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 -- 172.21.15.12:0/2049130486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 msgr2=0x55863fb55350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 --2- 172.21.15.12:0/2049130486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb55350 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8678011d40 tx=0x7f8678014000 comp rx=0 tx=0).stop 2024-05-08T23:59:42.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 -- 172.21.15.12:0/2049130486 shutdown_connections 2024-05-08T23:59:42.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 --2- 172.21.15.12:0/2049130486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 0x55863fb55cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 --2- 172.21.15.12:0/2049130486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb5b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 --2- 172.21.15.12:0/2049130486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb55350 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 -- 172.21.15.12:0/2049130486 >> 172.21.15.12:0/2049130486 conn(0x55863fab3210 msgr2=0x55863fb4fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 -- 172.21.15.12:0/2049130486 shutdown_connections 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 -- 172.21.15.12:0/2049130486 wait complete. 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 -- start start 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.703+0000 7f869c294b80 1 Processor -- start 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f869c294b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb55890 0x55863fb5eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f869c294b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5ae00 0x55863fb5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f869c294b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb5f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f869c294b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fa41560 con 0x55863fb5b720 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f869c294b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fa41200 con 0x55863fb55890 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f869c294b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863f9286a0 con 0x55863fb5ae00 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f869c294b80 10 client.0 _ll_register_callbacks cb 0x55863fa4ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:42.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868e677700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb5f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868de76700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb55890 0x55863fb5eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868e677700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb5f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49514/0 (socket says 172.21.15.12:49514) 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868d675700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5ae00 0x55863fb5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868e677700 1 -- 172.21.15.12:0/3875043561 learned_addr learned my addr 172.21.15.12:0/3875043561 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868de76700 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb55890 0x55863fb5eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50048/0 (socket says 172.21.15.12:50048) 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868e677700 1 -- 172.21.15.12:0/3875043561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5ae00 msgr2=0x55863fb5f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868e677700 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5ae00 0x55863fb5f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868e677700 1 -- 172.21.15.12:0/3875043561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb55890 msgr2=0x55863fb5eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868e677700 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb55890 0x55863fb5eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868e677700 1 -- 172.21.15.12:0/3875043561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8678017070 con 0x55863fb5b720 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.704+0000 7f868de76700 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb55890 0x55863fb5eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f868e677700 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb5f5d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f8680009610 tx=0x7f8680007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb033b85569bf48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f86717fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:42.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f86717fa700 1 -- 172.21.15.12:0/3875043561 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000e040 con 0x55863fb5b720 2024-05-08T23:59:42.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f86717fa700 1 -- 172.21.15.12:0/3875043561 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8680003a00 con 0x55863fb5b720 2024-05-08T23:59:42.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f869c294b80 1 -- 172.21.15.12:0/3875043561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55863f9fe270 con 0x55863fb5b720 2024-05-08T23:59:42.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f86717fa700 1 -- 172.21.15.12:0/3875043561 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8680003bc0 con 0x55863fb5b720 2024-05-08T23:59:42.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f869c294b80 10 client.19257 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:42.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f869c294b80 1 -- 172.21.15.12:0/3875043561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55863f8d2b60 con 0x55863fb5b720 2024-05-08T23:59:42.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f86717fa700 1 -- 172.21.15.12:0/3875043561 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55863f9fe270 con 0x55863fb5b720 2024-05-08T23:59:42.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f86717fa700 1 -- 172.21.15.12:0/3875043561 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8680022020 con 0x55863fb5b720 2024-05-08T23:59:42.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f869c294b80 10 client.19257 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:42.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f869c294b80 10 client.19257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:42.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f869c294b80 20 client.19257 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f869c294b80 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55863fb6f2b0 0x55863fb716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.705+0000 7f869c294b80 4 client.19257 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.706+0000 7f869c294b80 1 -- 172.21.15.12:0/3875043561 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55863fb24810 con 0x55863fb6f2b0 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.706+0000 7f868de76700 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55863fb6f2b0 0x55863fb716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.706+0000 7f868de76700 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55863fb6f2b0 0x55863fb716d0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af508ee1e669b459 server_cookie=6faecd0ebb776c7e in_seq=0 out_seq=0 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.706+0000 7f86717fa700 10 client.19257 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f86717fa700 1 -- 172.21.15.12:0/3875043561 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55863fb24810 con 0x55863fb6f2b0 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f86717fa700 10 client.19257 handle_command_reply: tid=0 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 client.19257 shutdown 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 -- 172.21.15.12:0/3875043561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 msgr2=0x55863fb5f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb5f5d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f8680009610 tx=0x7f8680007ea0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 -- 172.21.15.12:0/3875043561 shutdown_connections 2024-05-08T23:59:42.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb5f5d0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55863fb6f2b0 0x55863fb716d0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb55890 0x55863fb5eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 --2- 172.21.15.12:0/3875043561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5ae00 0x55863fb5f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f86717fa700 0 client.19257 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:42.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 -- 172.21.15.12:0/3875043561 >> 172.21.15.12:0/3875043561 conn(0x55863fab3210 msgr2=0x55863fb4f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 -- 172.21.15.12:0/3875043561 shutdown_connections 2024-05-08T23:59:42.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 1 -- 172.21.15.12:0/3875043561 wait complete. 2024-05-08T23:59:42.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.707+0000 7f869c294b80 20 client.19257 trim_cache size 0 max 16384 2024-05-08T23:59:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.726+0000 7f869c294b80 1 Processor -- start 2024-05-08T23:59:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.726+0000 7f869c294b80 1 -- start start 2024-05-08T23:59:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f869c294b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 0x55863fb55350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f869c294b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb6ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f869c294b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb4fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f869c294b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fb50e80 con 0x55863fb5b720 2024-05-08T23:59:42.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f869c294b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fb6c930 con 0x55863fb5ae00 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f869c294b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fb51070 con 0x55863fb55890 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868e677700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb4fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868e677700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb4fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49516/0 (socket says 172.21.15.12:49516) 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868de76700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 0x55863fb55350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868e677700 1 -- 172.21.15.12:0/777238819 learned_addr learned my addr 172.21.15.12:0/777238819 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868d675700 1 --2- 172.21.15.12:0/777238819 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb6ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868e677700 1 -- 172.21.15.12:0/777238819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 msgr2=0x55863fb55350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868e677700 1 --2- 172.21.15.12:0/777238819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 0x55863fb55350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868e677700 1 -- 172.21.15.12:0/777238819 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 msgr2=0x55863fb6ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868e677700 1 --2- 172.21.15.12:0/777238819 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb6ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868e677700 1 -- 172.21.15.12:0/777238819 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55863fa6b560 con 0x55863fb5b720 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868de76700 1 --2- 172.21.15.12:0/777238819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 0x55863fb55350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f868e677700 1 --2- 172.21.15.12:0/777238819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb4fd20 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f86800054b0 tx=0x7f8680008f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fa4c79917da355 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f867cff9700 1 -- 172.21.15.12:0/777238819 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000e030 con 0x55863fb5b720 2024-05-08T23:59:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.727+0000 7f867cff9700 1 -- 172.21.15.12:0/777238819 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8680006000 con 0x55863fb5b720 2024-05-08T23:59:42.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.728+0000 7f867cff9700 1 -- 172.21.15.12:0/777238819 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680007700 con 0x55863fb5b720 2024-05-08T23:59:42.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.728+0000 7f869c294b80 1 -- 172.21.15.12:0/777238819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 msgr2=0x55863fb4fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.728+0000 7f869c294b80 1 --2- 172.21.15.12:0/777238819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb4fd20 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f86800054b0 tx=0x7f8680008f20 comp rx=0 tx=0).stop 2024-05-08T23:59:42.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.728+0000 7f869c294b80 1 -- 172.21.15.12:0/777238819 shutdown_connections 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.728+0000 7f869c294b80 1 --2- 172.21.15.12:0/777238819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb5b720 0x55863fb4fd20 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.728+0000 7f869c294b80 1 --2- 172.21.15.12:0/777238819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb55890 0x55863fb55350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.728+0000 7f869c294b80 1 --2- 172.21.15.12:0/777238819 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb6ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.728+0000 7f869c294b80 1 -- 172.21.15.12:0/777238819 >> 172.21.15.12:0/777238819 conn(0x55863fab3210 msgr2=0x55863fb5d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.728+0000 7f869c294b80 1 -- 172.21.15.12:0/777238819 shutdown_connections 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.728+0000 7f869c294b80 1 -- 172.21.15.12:0/777238819 wait complete. 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.729+0000 7f869c294b80 1 -- start start 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.729+0000 7f869c294b80 1 Processor -- start 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.729+0000 7f869c294b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb55890 0x55863fb66670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.729+0000 7f869c294b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb64150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.729+0000 7f869c294b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5b720 0x55863fb64890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.729+0000 7f869c294b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fb64fa0 con 0x55863fb55890 2024-05-08T23:59:42.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.729+0000 7f869c294b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fbc4770 con 0x55863fb5ae00 2024-05-08T23:59:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.729+0000 7f869c294b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fbc48b0 con 0x55863fb5b720 2024-05-08T23:59:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.729+0000 7f869c294b80 10 client.0 _ll_register_callbacks cb 0x55863fa4ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868de76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb55890 0x55863fb66670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868d675700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb64150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868d675700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb64150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50070/0 (socket says 172.21.15.12:50070) 2024-05-08T23:59:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868d675700 1 -- 172.21.15.12:0/3167442829 learned_addr learned my addr 172.21.15.12:0/3167442829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868de76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb55890 0x55863fb66670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49520/0 (socket says 172.21.15.12:49520) 2024-05-08T23:59:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868e677700 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5b720 0x55863fb64890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868d675700 1 -- 172.21.15.12:0/3167442829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5b720 msgr2=0x55863fb64890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868d675700 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5b720 0x55863fb64890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868d675700 1 -- 172.21.15.12:0/3167442829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb55890 msgr2=0x55863fb66670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868d675700 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb55890 0x55863fb66670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868d675700 1 -- 172.21.15.12:0/3167442829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86881a6260 con 0x55863fb5ae00 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868e677700 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5b720 0x55863fb64890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868de76700 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb55890 0x55863fb66670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f868d675700 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb64150 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8678011d10 tx=0x7f867800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8daea92cd5114ae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f86717fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f86717fa700 1 -- 172.21.15.12:0/3167442829 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867802c040 con 0x55863fb5ae00 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f86717fa700 1 -- 172.21.15.12:0/3167442829 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8678005450 con 0x55863fb5ae00 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.730+0000 7f869c294b80 1 -- 172.21.15.12:0/3167442829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8678004020 con 0x55863fb5ae00 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f86717fa700 1 -- 172.21.15.12:0/3167442829 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8678005630 con 0x55863fb5ae00 2024-05-08T23:59:42.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f869c294b80 10 client.28976 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f869c294b80 1 -- 172.21.15.12:0/3167442829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55863f8d2b60 con 0x55863fb5ae00 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f86717fa700 1 -- 172.21.15.12:0/3167442829 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8678004020 con 0x55863fb5ae00 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f86717fa700 1 -- 172.21.15.12:0/3167442829 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f867802f070 con 0x55863fb5ae00 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f869c294b80 10 client.28976 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f869c294b80 10 client.28976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f869c294b80 20 client.28976 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f869c294b80 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55863fbc60e0 0x55863fbc8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f869c294b80 4 client.28976 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f869c294b80 1 -- 172.21.15.12:0/3167442829 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55863fa9dec0 con 0x55863fbc60e0 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.731+0000 7f868de76700 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55863fbc60e0 0x55863fbc8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.732+0000 7f86717fa700 10 client.28976 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.732+0000 7f868de76700 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55863fbc60e0 0x55863fbc8500 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4279bbd782856f9 server_cookie=287d9bee7b1acb46 in_seq=0 out_seq=0 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f86717fa700 1 -- 172.21.15.12:0/3167442829 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55863fa9dec0 con 0x55863fbc60e0 2024-05-08T23:59:42.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f86717fa700 10 client.28976 handle_command_reply: tid=0 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 client.28976 shutdown 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 -- 172.21.15.12:0/3167442829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 msgr2=0x55863fb64150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb64150 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8678011d10 tx=0x7f867800ed30 comp rx=0 tx=0).stop 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 -- 172.21.15.12:0/3167442829 shutdown_connections 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55863fb5b720 0x55863fb64890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55863fb55890 0x55863fb66670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55863fbc60e0 0x55863fbc8500 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 --2- 172.21.15.12:0/3167442829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55863fb5ae00 0x55863fb64150 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 -- 172.21.15.12:0/3167442829 >> 172.21.15.12:0/3167442829 conn(0x55863fab3210 msgr2=0x55863fb59c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f86717fa700 0 client.28976 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 -- 172.21.15.12:0/3167442829 shutdown_connections 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 1 -- 172.21.15.12:0/3167442829 wait complete. 2024-05-08T23:59:42.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.733+0000 7f869c294b80 20 client.28976 trim_cache size 0 max 16384 2024-05-08T23:59:42.734 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-08T23:59:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:59:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-08T23:59:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "ad2d8e7c-fcb4-432b-a6d5-b06fb0d7268d": { 2024-05-08T23:59:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-08T23:59:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "ad2d8e7c-fcb4-432b-a6d5-b06fb0d7268d", 2024-05-08T23:59:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-08T23:59:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:59:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-08T23:59:42.734 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-08T23:59:42.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 -- 172.21.15.12:0/2899288066 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86740a1290 msgr2=0x7f86740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86740a1290 0x7f86740a3750 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f868819ac20 tx=0x7f8678009260 comp rx=0 tx=0).stop 2024-05-08T23:59:42.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 -- 172.21.15.12:0/2899288066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688068050 msgr2=0x7f8688199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:42.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688068050 0x7f8688199720 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f868400eb30 tx=0x7f868400ee40 comp rx=0 tx=0).stop 2024-05-08T23:59:42.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 -- 172.21.15.12:0/2899288066 shutdown_connections 2024-05-08T23:59:42.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86740a1290 0x7f86740a3750 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 0x7f868819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688068050 0x7f8688199720 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 --2- 172.21.15.12:0/2899288066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688068970 0x7f8688199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:42.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 -- 172.21.15.12:0/2899288066 >> 172.21.15.12:0/2899288066 conn(0x7f8688076f40 msgr2=0x7f868810ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:42.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 -- 172.21.15.12:0/2899288066 shutdown_connections 2024-05-08T23:59:42.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-08T23:59:42.734+0000 7f86900da700 1 -- 172.21.15.12:0/2899288066 wait complete. 2024-05-08T23:59:42.742 INFO:tasks.cephfs.filesystem:scrub status for tag:ad2d8e7c-fcb4-432b-a6d5-b06fb0d7268d - {'path': '/', 'tag': 'ad2d8e7c-fcb4-432b-a6d5-b06fb0d7268d', 'options': 'recursive,force'} 2024-05-08T23:59:42.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 746001 6.09 MB/sec execute 578 sec latency 65.195 ms 2024-05-08T23:59:43.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:42 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:59:43.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:42 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:43.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:42 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:59:43.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:42 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:43.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:42 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:59:43.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:42 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.733+0000 7f3a03dd1700 1 -- 172.21.15.12:0/379921421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39fc100880 msgr2=0x7f39fc100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.733+0000 7f3a03dd1700 1 --2- 172.21.15.12:0/379921421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39fc100880 0x7f39fc100c60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b0a0 tx=0x7f39f800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.734+0000 7f3a03dd1700 1 -- 172.21.15.12:0/379921421 shutdown_connections 2024-05-08T23:59:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.734+0000 7f3a03dd1700 1 --2- 172.21.15.12:0/379921421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39fc10d8f0 0x7f39fc10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.734+0000 7f3a03dd1700 1 --2- 172.21.15.12:0/379921421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39fc100880 0x7f39fc100c60 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.734+0000 7f3a03dd1700 1 --2- 172.21.15.12:0/379921421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39fc1011a0 0x7f39fc10d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.734+0000 7f3a03dd1700 1 -- 172.21.15.12:0/379921421 >> 172.21.15.12:0/379921421 conn(0x7f39fc076f10 msgr2=0x7f39fc077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.734+0000 7f3a03dd1700 1 -- 172.21.15.12:0/379921421 shutdown_connections 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.734+0000 7f3a03dd1700 1 -- 172.21.15.12:0/379921421 wait complete. 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.734+0000 7f3a03dd1700 1 Processor -- start 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.734+0000 7f3a03dd1700 1 -- start start 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a03dd1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39fc100880 0x7f39fc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a03dd1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39fc1011a0 0x7f39fc199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a03dd1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39fc10d8f0 0x7f39fc19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a03dd1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc19e620 con 0x7f39fc1011a0 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a03dd1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc19e790 con 0x7f39fc10d8f0 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a03dd1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc19e950 con 0x7f39fc100880 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a0136c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39fc1011a0 0x7f39fc199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a01b6d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39fc100880 0x7f39fc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a0136c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39fc1011a0 0x7f39fc199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49528/0 (socket says 172.21.15.12:49528) 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a0136c700 1 -- 172.21.15.12:0/455887634 learned_addr learned my addr 172.21.15.12:0/455887634 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a0236e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39fc10d8f0 0x7f39fc19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a01b6d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39fc100880 0x7f39fc199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54860/0 (socket says 172.21.15.12:54860) 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a01b6d700 1 -- 172.21.15.12:0/455887634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39fc10d8f0 msgr2=0x7f39fc19dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39fc10d8f0 0x7f39fc19dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a01b6d700 1 -- 172.21.15.12:0/455887634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39fc1011a0 msgr2=0x7f39fc199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39fc1011a0 0x7f39fc199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a01b6d700 1 -- 172.21.15.12:0/455887634 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f4009520 con 0x7f39fc100880 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a0136c700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39fc1011a0 0x7f39fc199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39fc100880 0x7f39fc199700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b070 tx=0x7f39f80078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.735+0000 7f39f2ffd700 1 -- 172.21.15.12:0/455887634 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f800e040 con 0x7f39fc100880 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.736+0000 7f39f2ffd700 1 -- 172.21.15.12:0/455887634 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f801d470 con 0x7f39fc100880 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.736+0000 7f39f2ffd700 1 -- 172.21.15.12:0/455887634 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80052a0 con 0x7f39fc100880 2024-05-08T23:59:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.736+0000 7f3a03dd1700 1 -- 172.21.15.12:0/455887634 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f8009d00 con 0x7f39fc100880 2024-05-08T23:59:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.736+0000 7f3a03dd1700 1 -- 172.21.15.12:0/455887634 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc1a6290 con 0x7f39fc100880 2024-05-08T23:59:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.738+0000 7f39f2ffd700 1 -- 172.21.15.12:0/455887634 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f39f801d5e0 con 0x7f39fc100880 2024-05-08T23:59:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.738+0000 7f39f2ffd700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39e80a1310 0x7f39e80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.739+0000 7f39f2ffd700 1 -- 172.21.15.12:0/455887634 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f39f80cde80 con 0x7f39fc100880 2024-05-08T23:59:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.739+0000 7f3a0136c700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39e80a1310 0x7f39e80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.739+0000 7f3a0136c700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39e80a1310 0x7f39e80a37d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f39fc19ac50 tx=0x7f39ec009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.739+0000 7f3a0ff8bb80 1 Processor -- start 2024-05-08T23:59:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0ff8bb80 1 -- start start 2024-05-08T23:59:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 0x5606a300cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3006f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 0x5606a30078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2f5e0f0 con 0x5606a300d270 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2f5bab0 con 0x5606a3007480 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2f573a0 con 0x5606a300c950 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0136c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3006f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0136c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3006f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49532/0 (socket says 172.21.15.12:49532) 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0136c700 1 -- 172.21.15.12:0/2210537265 learned_addr learned my addr 172.21.15.12:0/2210537265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a0236e700 1 --2- 172.21.15.12:0/2210537265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 0x5606a30078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.740+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/2210537265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 0x5606a300cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.741+0000 7f3a0136c700 1 -- 172.21.15.12:0/2210537265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 msgr2=0x5606a300cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.741+0000 7f3a0136c700 1 --2- 172.21.15.12:0/2210537265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 0x5606a300cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.741+0000 7f3a0136c700 1 -- 172.21.15.12:0/2210537265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 msgr2=0x5606a30078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.741+0000 7f3a0136c700 1 --2- 172.21.15.12:0/2210537265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 0x5606a30078a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.741+0000 7f3a0136c700 1 -- 172.21.15.12:0/2210537265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606a2dd8350 con 0x5606a300d270 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.741+0000 7f3a0236e700 1 --2- 172.21.15.12:0/2210537265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 0x5606a30078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.741+0000 7f3a0136c700 1 --2- 172.21.15.12:0/2210537265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3006f40 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009880 tx=0x7f39ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0423a5d771f707 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.741+0000 7f39e7fff700 1 -- 172.21.15.12:0/2210537265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec004a30 con 0x5606a300d270 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.741+0000 7f39e7fff700 1 -- 172.21.15.12:0/2210537265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39ec021070 con 0x5606a300d270 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.741+0000 7f39e7fff700 1 -- 172.21.15.12:0/2210537265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec01c610 con 0x5606a300d270 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.742+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/2210537265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 msgr2=0x5606a3006f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.742+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/2210537265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3006f40 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009880 tx=0x7f39ec014000 comp rx=0 tx=0).stop 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.742+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/2210537265 shutdown_connections 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.742+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/2210537265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 0x5606a30078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.742+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/2210537265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 0x5606a300cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.742+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/2210537265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3006f40 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.742+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/2210537265 >> 172.21.15.12:0/2210537265 conn(0x5606a2f651b0 msgr2=0x5606a3001840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.742+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/2210537265 shutdown_connections 2024-05-08T23:59:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.742+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/2210537265 wait complete. 2024-05-08T23:59:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.742+0000 7f3a0ff8bb80 1 -- start start 2024-05-08T23:59:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0ff8bb80 1 Processor -- start 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a3007480 0x5606a3010640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 0x5606a3010b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a30110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2ef3500 con 0x5606a300d270 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2ef31a0 con 0x5606a300c950 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2dda640 con 0x5606a3007480 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0ff8bb80 10 client.0 _ll_register_callbacks cb 0x5606a2efcad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0236e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a30110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0136c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 0x5606a3010b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0236e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a30110c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49538/0 (socket says 172.21.15.12:49538) 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a01b6d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a3007480 0x5606a3010640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0136c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 0x5606a3010b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50104/0 (socket says 172.21.15.12:50104) 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.743+0000 7f3a0236e700 1 -- 172.21.15.12:0/1447616164 learned_addr learned my addr 172.21.15.12:0/1447616164 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a0236e700 1 -- 172.21.15.12:0/1447616164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a3007480 msgr2=0x5606a3010640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a0236e700 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a3007480 0x5606a3010640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a0236e700 1 -- 172.21.15.12:0/1447616164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 msgr2=0x5606a3010b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a0236e700 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 0x5606a3010b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a0236e700 1 -- 172.21.15.12:0/1447616164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f4009a60 con 0x5606a300d270 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a3007480 0x5606a3010640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a0136c700 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 0x5606a3010b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a0236e700 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a30110c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f39f4011d10 tx=0x7f39f400f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e02ab052a49b0f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f39e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f39e57fa700 1 -- 172.21.15.12:0/1447616164 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4016070 con 0x5606a300d270 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f39e57fa700 1 -- 172.21.15.12:0/1447616164 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f39f400fdc0 con 0x5606a300d270 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/1447616164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606a2eb0210 con 0x5606a300d270 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f39e57fa700 1 -- 172.21.15.12:0/1447616164 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f4023410 con 0x5606a300d270 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a0ff8bb80 10 client.19275 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/1447616164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f39ec017070 con 0x5606a300d270 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.744+0000 7f39e57fa700 1 -- 172.21.15.12:0/1447616164 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5606a2eb0210 con 0x5606a300d270 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.745+0000 7f39e57fa700 1 -- 172.21.15.12:0/1447616164 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f39f401a4f0 con 0x5606a300d270 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.745+0000 7f3a0ff8bb80 10 client.19275 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.745+0000 7f3a0ff8bb80 10 client.19275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.745+0000 7f3a0ff8bb80 20 client.19275 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.745+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5606a3020d70 0x5606a3023190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.745+0000 7f3a0ff8bb80 4 client.19275 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.745+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/1447616164 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606a2fd67b0 con 0x5606a3020d70 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.745+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5606a3020d70 0x5606a3023190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.745+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5606a3020d70 0x5606a3023190 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2b61024d2f1237 server_cookie=de0ae732c37961fe in_seq=0 out_seq=0 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.745+0000 7f39e57fa700 10 client.19275 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f39e57fa700 1 -- 172.21.15.12:0/1447616164 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f39f807fa90 con 0x5606a3020d70 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f39e57fa700 10 client.19275 handle_command_reply: tid=0 2024-05-08T23:59:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 client.19275 shutdown 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/1447616164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 msgr2=0x5606a30110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a30110c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f39f4011d10 tx=0x7f39f400f6a0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/1447616164 shutdown_connections 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a30110c0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a3007480 0x5606a3010640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5606a3020d70 0x5606a3023190 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/1447616164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 0x5606a3010b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/1447616164 >> 172.21.15.12:0/1447616164 conn(0x5606a2f651b0 msgr2=0x5606a300b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f39e57fa700 0 client.19275 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/1447616164 shutdown_connections 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/1447616164 wait complete. 2024-05-08T23:59:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.746+0000 7f3a0ff8bb80 20 client.19275 trim_cache size 0 max 16384 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.762+0000 7f3a0ff8bb80 1 Processor -- start 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.762+0000 7f3a0ff8bb80 1 -- start start 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a3007480 0x5606a301cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 0x5606a3001910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300d270 0x5606a3010f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a301e420 con 0x5606a3007480 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a30061e0 con 0x5606a300c950 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a3013920 con 0x5606a300d270 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a01b6d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a3007480 0x5606a301cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a0136c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 0x5606a3001910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a01b6d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a3007480 0x5606a301cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49540/0 (socket says 172.21.15.12:49540) 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a01b6d700 1 -- 172.21.15.12:0/839280515 learned_addr learned my addr 172.21.15.12:0/839280515 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a0236e700 1 --2- 172.21.15.12:0/839280515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300d270 0x5606a3010f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a01b6d700 1 -- 172.21.15.12:0/839280515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300d270 msgr2=0x5606a3010f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/839280515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300d270 0x5606a3010f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a01b6d700 1 -- 172.21.15.12:0/839280515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 msgr2=0x5606a3001910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/839280515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 0x5606a3001910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a01b6d700 1 -- 172.21.15.12:0/839280515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606a2d84b00 con 0x5606a3007480 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.763+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/839280515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a3007480 0x5606a301cd60 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f39f809aff0 tx=0x7f39f801dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8a6e18f45e1403 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f39f0ff9700 1 -- 172.21.15.12:0/839280515 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80aeea0 con 0x5606a3007480 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f39f0ff9700 1 -- 172.21.15.12:0/839280515 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5606a2fd67b0 con 0x5606a3007480 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f39f0ff9700 1 -- 172.21.15.12:0/839280515 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f809b2c0 con 0x5606a3007480 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/839280515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a3007480 msgr2=0x5606a301cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/839280515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a3007480 0x5606a301cd60 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f39f809aff0 tx=0x7f39f801dca0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/839280515 shutdown_connections 2024-05-08T23:59:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/839280515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300d270 0x5606a3010f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/839280515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a3007480 0x5606a301cd60 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/839280515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a300c950 0x5606a3001910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/839280515 >> 172.21.15.12:0/839280515 conn(0x5606a2f651b0 msgr2=0x5606a2ffde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.764+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/839280515 shutdown_connections 2024-05-08T23:59:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.765+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/839280515 wait complete. 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.765+0000 7f3a0ff8bb80 1 -- start start 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.765+0000 7f3a0ff8bb80 1 Processor -- start 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.765+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 0x5606a3018160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.765+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 0x5606a3015bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.765+0000 7f3a0ff8bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3016330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a3016a10 con 0x5606a300d270 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a3076270 con 0x5606a3007480 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0ff8bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a30763b0 con 0x5606a300c950 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0ff8bb80 10 client.0 _ll_register_callbacks cb 0x7f39f40117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a01b6d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 0x5606a3018160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0236e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3016330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a01b6d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 0x5606a3018160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50126/0 (socket says 172.21.15.12:50126) 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a01b6d700 1 -- 172.21.15.12:0/414428046 learned_addr learned my addr 172.21.15.12:0/414428046 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0136c700 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 0x5606a3015bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0236e700 1 -- 172.21.15.12:0/414428046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 msgr2=0x5606a3015bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0236e700 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 0x5606a3015bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0236e700 1 -- 172.21.15.12:0/414428046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 msgr2=0x5606a3018160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0236e700 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 0x5606a3018160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0236e700 1 -- 172.21.15.12:0/414428046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc1a6290 con 0x5606a300d270 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0136c700 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 0x5606a3015bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f3a0236e700 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3016330 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f39f401a950 tx=0x7f39f401a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820c1183c475ab73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f39e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f39e57fa700 1 -- 172.21.15.12:0/414428046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4030070 con 0x5606a300d270 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.766+0000 7f39e57fa700 1 -- 172.21.15.12:0/414428046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f39f4000e40 con 0x5606a300d270 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/414428046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39ec021070 con 0x5606a300d270 2024-05-08T23:59:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f39e57fa700 1 -- 172.21.15.12:0/414428046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f4016070 con 0x5606a300d270 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f3a0ff8bb80 10 client.19281 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/414428046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5606a2f1d500 con 0x5606a300d270 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f39e57fa700 1 -- 172.21.15.12:0/414428046 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39ec021070 con 0x5606a300d270 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f39e57fa700 1 -- 172.21.15.12:0/414428046 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f39f4033030 con 0x5606a300d270 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f3a0ff8bb80 10 client.19281 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f3a0ff8bb80 10 client.19281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f3a0ff8bb80 20 client.19281 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5606a3077a80 0x5606a3079ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f3a0ff8bb80 4 client.19281 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/414428046 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5606a2f4fe60 con 0x5606a3077a80 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.767+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5606a3077a80 0x5606a3079ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.768+0000 7f39e57fa700 10 client.19281 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.768+0000 7f3a01b6d700 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5606a3077a80 0x5606a3079ea0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15cf50dce44ecee8 server_cookie=fb666b49148573c2 in_seq=0 out_seq=0 2024-05-08T23:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f39e57fa700 1 -- 172.21.15.12:0/414428046 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5606a2f4fe60 con 0x5606a3077a80 2024-05-08T23:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f39e57fa700 10 client.19281 handle_command_reply: tid=0 2024-05-08T23:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 client.19281 shutdown 2024-05-08T23:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/414428046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 msgr2=0x5606a3016330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3016330 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f39f401a950 tx=0x7f39f401a620 comp rx=0 tx=0).stop 2024-05-08T23:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/414428046 shutdown_connections 2024-05-08T23:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606a300d270 0x5606a3016330 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5606a3077a80 0x5606a3079ea0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5606a3007480 0x5606a3018160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 --2- 172.21.15.12:0/414428046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a300c950 0x5606a3015bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f39e57fa700 0 client.19281 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/414428046 >> 172.21.15.12:0/414428046 conn(0x5606a2f651b0 msgr2=0x5606a3005670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/414428046 shutdown_connections 2024-05-08T23:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 1 -- 172.21.15.12:0/414428046 wait complete. 2024-05-08T23:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.830+0000 7f3a0ff8bb80 20 client.19281 trim_cache size 0 max 16384 2024-05-08T23:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 -- 172.21.15.12:0/455887634 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39e80a1310 msgr2=0x7f39e80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39e80a1310 0x7f39e80a37d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f39fc19ac50 tx=0x7f39ec009260 comp rx=0 tx=0).stop 2024-05-08T23:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 -- 172.21.15.12:0/455887634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39fc100880 msgr2=0x7f39fc199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39fc100880 0x7f39fc199700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b070 tx=0x7f39f80078e0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 -- 172.21.15.12:0/455887634 shutdown_connections 2024-05-08T23:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f39e80a1310 0x7f39e80a37d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39fc10d8f0 0x7f39fc19dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39fc100880 0x7f39fc199700 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 --2- 172.21.15.12:0/455887634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39fc1011a0 0x7f39fc199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 -- 172.21.15.12:0/455887634 >> 172.21.15.12:0/455887634 conn(0x7f39fc076f10 msgr2=0x7f39fc10f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 -- 172.21.15.12:0/455887634 shutdown_connections 2024-05-08T23:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:43.831+0000 7f3a03dd1700 1 -- 172.21.15.12:0/455887634 wait complete. 2024-05-08T23:59:43.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 747335 6.09 MB/sec execute 579 sec latency 35.446 ms 2024-05-08T23:59:44.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:44 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-08T23:59:44.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:44 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:44 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:59:44.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:44 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:59:44.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:44 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:44 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-08T23:59:44.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:44 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:44 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:44 smithi204 ceph-mon[47373]: pgmap v426: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-05-08T23:59:44.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:44 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-08T23:59:44.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:44 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:44 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:59:44.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:44 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:59:44.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:44 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:44 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-08T23:59:44.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:44 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:44 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:44 smithi160 ceph-mon[46152]: pgmap v426: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-05-08T23:59:44.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:44 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-08T23:59:44.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:44 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:44 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:59:44.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:44 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:59:44.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:44 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:44 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-08T23:59:44.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:44 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:44 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-08T23:59:44.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:44 smithi012 ceph-mon[42503]: pgmap v426: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-05-08T23:59:44.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 749547 6.10 MB/sec execute 580 sec latency 17.467 ms 2024-05-08T23:59:45.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 750837 6.10 MB/sec execute 581 sec latency 17.286 ms 2024-05-08T23:59:45.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:45 smithi012 ceph-mon[42503]: pgmap v427: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-05-08T23:59:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:45 smithi204 ceph-mon[47373]: pgmap v427: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-05-08T23:59:46.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:45 smithi160 ceph-mon[46152]: pgmap v427: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-05-08T23:59:46.901 INFO:tasks.workunit.client.0.smithi012.stdout: 1 752852 6.11 MB/sec execute 582 sec latency 15.435 ms 2024-05-08T23:59:47.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 753956 6.11 MB/sec execute 583 sec latency 33.984 ms 2024-05-08T23:59:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:47 smithi012 ceph-mon[42503]: pgmap v428: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-05-08T23:59:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:47 smithi204 ceph-mon[47373]: pgmap v428: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-05-08T23:59:48.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:47 smithi160 ceph-mon[46152]: pgmap v428: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-05-08T23:59:48.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 756301 6.12 MB/sec execute 584 sec latency 18.265 ms 2024-05-08T23:59:48.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:48 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:59:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:48 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:59:49.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:48 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:59:49.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 757555 6.12 MB/sec execute 585 sec latency 29.107 ms 2024-05-08T23:59:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:49 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:59:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:49 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-08T23:59:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:49 smithi012 ceph-mon[42503]: pgmap v429: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-05-08T23:59:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:49 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:59:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:49 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-08T23:59:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:49 smithi204 ceph-mon[47373]: pgmap v429: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-05-08T23:59:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:49 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:59:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:49 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-08T23:59:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:49 smithi160 ceph-mon[46152]: pgmap v429: 97 pgs: 97 active+clean; 212 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-05-08T23:59:50.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 758914 6.12 MB/sec execute 586 sec latency 17.405 ms 2024-05-08T23:59:51.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 760533 6.13 MB/sec execute 587 sec latency 10.745 ms 2024-05-08T23:59:51.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:51 smithi012 ceph-mon[42503]: pgmap v430: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2024-05-08T23:59:52.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:51 smithi204 ceph-mon[47373]: pgmap v430: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2024-05-08T23:59:52.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:51 smithi160 ceph-mon[46152]: pgmap v430: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2024-05-08T23:59:52.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 762066 6.13 MB/sec execute 588 sec latency 47.353 ms 2024-05-08T23:59:53.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 763835 6.14 MB/sec execute 589 sec latency 23.751 ms 2024-05-08T23:59:53.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:53 smithi012 ceph-mon[42503]: pgmap v431: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-05-08T23:59:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:53 smithi204 ceph-mon[47373]: pgmap v431: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-05-08T23:59:54.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:53 smithi160 ceph-mon[46152]: pgmap v431: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-05-08T23:59:54.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 765030 6.14 MB/sec execute 590 sec latency 31.042 ms 2024-05-08T23:59:55.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 767026 6.15 MB/sec execute 591 sec latency 16.462 ms 2024-05-08T23:59:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:55 smithi012 ceph-mon[42503]: pgmap v432: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-05-08T23:59:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:55 smithi204 ceph-mon[47373]: pgmap v432: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-05-08T23:59:56.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:55 smithi160 ceph-mon[46152]: pgmap v432: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-05-08T23:59:56.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 768178 6.14 MB/sec execute 592 sec latency 25.837 ms 2024-05-08T23:59:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.960+0000 7ff1e2394700 1 -- 172.21.15.12:0/3469406153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1dc105e60 msgr2=0x7ff1dc106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.960+0000 7ff1e2394700 1 --2- 172.21.15.12:0/3469406153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1dc105e60 0x7ff1dc106240 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009990 tx=0x7ff1cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.960+0000 7ff1e2394700 1 -- 172.21.15.12:0/3469406153 shutdown_connections 2024-05-08T23:59:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.960+0000 7ff1e2394700 1 --2- 172.21.15.12:0/3469406153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1dc100760 0x7ff1dc10cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.960+0000 7ff1e2394700 1 --2- 172.21.15.12:0/3469406153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1dc105e60 0x7ff1dc106240 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.960+0000 7ff1e2394700 1 --2- 172.21.15.12:0/3469406153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1dc0ffda0 0x7ff1dc100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.960+0000 7ff1e2394700 1 -- 172.21.15.12:0/3469406153 >> 172.21.15.12:0/3469406153 conn(0x7ff1dc076f10 msgr2=0x7ff1dc077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.960+0000 7ff1e2394700 1 -- 172.21.15.12:0/3469406153 shutdown_connections 2024-05-08T23:59:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.960+0000 7ff1e2394700 1 -- 172.21.15.12:0/3469406153 wait complete. 2024-05-08T23:59:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.961+0000 7ff1e2394700 1 Processor -- start 2024-05-08T23:59:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.961+0000 7ff1e2394700 1 -- start start 2024-05-08T23:59:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e2394700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1dc0ffda0 0x7ff1dc10d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e2394700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1dc100760 0x7ff1dc10db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e2394700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1dc105e60 0x7ff1dc10e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e2394700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc1a5680 con 0x7ff1dc0ffda0 2024-05-08T23:59:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e2394700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc1a57f0 con 0x7ff1dc100760 2024-05-08T23:59:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e2394700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc1a59b0 con 0x7ff1dc105e60 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1dc0ffda0 0x7ff1dc10d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1dc0ffda0 0x7ff1dc10d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35910/0 (socket says 172.21.15.12:35910) 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1dbfff700 1 -- 172.21.15.12:0/1709783067 learned_addr learned my addr 172.21.15.12:0/1709783067 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e0931700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1dc105e60 0x7ff1dc10e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1db7fe700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1dc100760 0x7ff1dc10db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e0931700 1 -- 172.21.15.12:0/1709783067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1dc100760 msgr2=0x7ff1dc10db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e0931700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1dc100760 0x7ff1dc10db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e0931700 1 -- 172.21.15.12:0/1709783067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1dc0ffda0 msgr2=0x7ff1dc10d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e0931700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1dc0ffda0 0x7ff1dc10d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e0931700 1 -- 172.21.15.12:0/1709783067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0095f0 con 0x7ff1dc105e60 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1dc0ffda0 0x7ff1dc10d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.962+0000 7ff1e0931700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1dc105e60 0x7ff1dc10e040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400efc0 tx=0x7ff1d400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.963+0000 7ff1d97fa700 1 -- 172.21.15.12:0/1709783067 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4018070 con 0x7ff1dc105e60 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.963+0000 7ff1e2394700 1 -- 172.21.15.12:0/1709783067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc1a5c30 con 0x7ff1dc105e60 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.963+0000 7ff1d97fa700 1 -- 172.21.15.12:0/1709783067 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1d400f040 con 0x7ff1dc105e60 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.963+0000 7ff1d97fa700 1 -- 172.21.15.12:0/1709783067 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400cd40 con 0x7ff1dc105e60 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.963+0000 7ff1e2394700 1 -- 172.21.15.12:0/1709783067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc1a60f0 con 0x7ff1dc105e60 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.965+0000 7ff1d97fa700 1 -- 172.21.15.12:0/1709783067 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff1d4003a00 con 0x7ff1dc105e60 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.966+0000 7ff1d97fa700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1c40a1290 0x7ff1c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.966+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1c40a1290 0x7ff1c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.966+0000 7ff1d97fa700 1 -- 172.21.15.12:0/1709783067 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff1d40cc8d0 con 0x7ff1dc105e60 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.966+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1c40a1290 0x7ff1c40a3750 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00b2e0 tx=0x7ff1cc005bc0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.967+0000 7ff1ee54eb80 1 Processor -- start 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.967+0000 7ff1ee54eb80 1 -- start start 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.967+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 0x55e15433dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.967+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433e270 0x55e154337f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.967+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 0x55e1543388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.967+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15428f0f0 con 0x55e15433d950 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.967+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15428cab0 con 0x55e154338480 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.967+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1542883a0 con 0x55e15433e270 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.967+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 0x55e15433dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 0x55e15433dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35914/0 (socket says 172.21.15.12:35914) 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1dbfff700 1 -- 172.21.15.12:0/3737517772 learned_addr learned my addr 172.21.15.12:0/3737517772 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1db7fe700 1 --2- 172.21.15.12:0/3737517772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433e270 0x55e154337f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1e0931700 1 --2- 172.21.15.12:0/3737517772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 0x55e1543388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1e0931700 1 -- 172.21.15.12:0/3737517772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433e270 msgr2=0x55e154337f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1e0931700 1 --2- 172.21.15.12:0/3737517772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433e270 0x55e154337f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1e0931700 1 -- 172.21.15.12:0/3737517772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 msgr2=0x55e15433dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1e0931700 1 --2- 172.21.15.12:0/3737517772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 0x55e15433dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1e0931700 1 -- 172.21.15.12:0/3737517772 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e154109350 con 0x55e154338480 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/3737517772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 0x55e15433dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1e0931700 1 --2- 172.21.15.12:0/3737517772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 0x55e1543388a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40ca000 tx=0x7ff1d40998a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d2a5384d711ced server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1c27fc700 1 -- 172.21.15.12:0/3737517772 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4004ad0 con 0x55e154338480 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1c27fc700 1 -- 172.21.15.12:0/3737517772 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1d409a150 con 0x55e154338480 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.968+0000 7ff1c27fc700 1 -- 172.21.15.12:0/3737517772 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d401b580 con 0x55e154338480 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.969+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3737517772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 msgr2=0x55e1543388a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.969+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/3737517772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 0x55e1543388a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40ca000 tx=0x7ff1d40998a0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.969+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3737517772 shutdown_connections 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.969+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/3737517772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 0x55e1543388a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.969+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/3737517772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 0x55e15433dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.969+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/3737517772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433e270 0x55e154337f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.969+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3737517772 >> 172.21.15.12:0/3737517772 conn(0x55e1542961b0 msgr2=0x55e154332840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.969+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3737517772 shutdown_connections 2024-05-08T23:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.969+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3737517772 wait complete. 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.969+0000 7ff1ee54eb80 1 -- start start 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.970+0000 7ff1ee54eb80 1 Processor -- start 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.970+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e154338480 0x55e1543415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.970+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433d950 0x55e154341af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.970+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433e270 0x55e154342030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.970+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e154224500 con 0x55e154338480 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.970+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1542241a0 con 0x55e15433d950 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.970+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15410b640 con 0x55e15433e270 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.970+0000 7ff1ee54eb80 10 client.0 _ll_register_callbacks cb 0x55e15422dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.970+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e154338480 0x55e1543415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433d950 0x55e154341af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e154338480 0x55e1543415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35922/0 (socket says 172.21.15.12:35922) 2024-05-08T23:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1dbfff700 1 -- 172.21.15.12:0/80374767 learned_addr learned my addr 172.21.15.12:0/80374767 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1e0931700 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433e270 0x55e154342030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1dbfff700 1 -- 172.21.15.12:0/80374767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433e270 msgr2=0x55e154342030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433e270 0x55e154342030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1dbfff700 1 -- 172.21.15.12:0/80374767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433d950 msgr2=0x55e154341af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433d950 0x55e154341af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1dbfff700 1 -- 172.21.15.12:0/80374767 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc068280 con 0x55e154338480 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1db7fe700 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433d950 0x55e154341af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e154338480 0x55e1543415b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00fbb0 tx=0x7ff1cc01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448d88a8bdd9071f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/80374767 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc02b050 con 0x55e154338480 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/80374767 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff1cc026270 con 0x55e154338480 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.971+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/80374767 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1541e1210 con 0x55e154338480 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/80374767 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1cc026840 con 0x55e154338480 2024-05-08T23:59:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1ee54eb80 10 client.19293 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/80374767 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7ff1dc1a60f0 con 0x55e154338480 2024-05-08T23:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/80374767 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1541e1210 con 0x55e154338480 2024-05-08T23:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/80374767 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff1cc024be0 con 0x55e154338480 2024-05-08T23:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1ee54eb80 10 client.19293 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1ee54eb80 10 client.19293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1ee54eb80 20 client.19293 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e154351cb0 0x55e1543540d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1ee54eb80 4 client.19293 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/80374767 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1543077b0 con 0x55e154351cb0 2024-05-08T23:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.972+0000 7ff1db7fe700 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e154351cb0 0x55e1543540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.973+0000 7ff1bb7fe700 10 client.19293 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.973+0000 7ff1db7fe700 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e154351cb0 0x55e1543540d0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85055ebcb30a2bfa server_cookie=bd589a57129a3335 in_seq=0 out_seq=0 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.973+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/80374767 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e1543077b0 con 0x55e154351cb0 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.973+0000 7ff1bb7fe700 10 client.19293 handle_command_reply: tid=0 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.973+0000 7ff1ee54eb80 1 client.19293 shutdown 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/80374767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e154338480 msgr2=0x55e1543415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e154338480 0x55e1543415b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00fbb0 tx=0x7ff1cc01a050 comp rx=0 tx=0).stop 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/80374767 shutdown_connections 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433e270 0x55e154342030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e154338480 0x55e1543415b0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e154351cb0 0x55e1543540d0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/80374767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433d950 0x55e154341af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1bb7fe700 0 client.19293 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/80374767 >> 172.21.15.12:0/80374767 conn(0x55e1542961b0 msgr2=0x55e15433c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/80374767 shutdown_connections 2024-05-08T23:59:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/80374767 wait complete. 2024-05-08T23:59:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.974+0000 7ff1ee54eb80 20 client.19293 trim_cache size 0 max 16384 2024-05-08T23:59:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1ee54eb80 1 Processor -- start 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1ee54eb80 1 -- start start 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e154338480 0x55e154338860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 0x55e154332910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433e270 0x55e154341e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e154333330 con 0x55e15433d950 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e154333930 con 0x55e15433e270 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1543592e0 con 0x55e154338480 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e154338480 0x55e154338860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e154338480 0x55e154338860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:35952/0 (socket says 172.21.15.12:35952) 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1dbfff700 1 -- 172.21.15.12:0/294014601 learned_addr learned my addr 172.21.15.12:0/294014601 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1db7fe700 1 --2- 172.21.15.12:0/294014601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 0x55e154332910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.990+0000 7ff1e0931700 1 --2- 172.21.15.12:0/294014601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433e270 0x55e154341e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.991+0000 7ff1dbfff700 1 -- 172.21.15.12:0/294014601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433e270 msgr2=0x55e154341e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.991+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/294014601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433e270 0x55e154341e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.991+0000 7ff1dbfff700 1 -- 172.21.15.12:0/294014601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 msgr2=0x55e154332910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.991+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/294014601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 0x55e154332910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.991+0000 7ff1dbfff700 1 -- 172.21.15.12:0/294014601 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1540b5b00 con 0x55e154338480 2024-05-08T23:59:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.991+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/294014601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e154338480 0x55e154338860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000c00 tx=0x7ff1cc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7df597e415b99f00 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.991+0000 7ff1c2ffd700 1 -- 172.21.15.12:0/294014601 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc02b040 con 0x55e154338480 2024-05-08T23:59:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.991+0000 7ff1c2ffd700 1 -- 172.21.15.12:0/294014601 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1cc02c040 con 0x55e154338480 2024-05-08T23:59:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.991+0000 7ff1c2ffd700 1 -- 172.21.15.12:0/294014601 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc026970 con 0x55e154338480 2024-05-08T23:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.992+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/294014601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e154338480 msgr2=0x55e154338860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.992+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/294014601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e154338480 0x55e154338860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000c00 tx=0x7ff1cc009310 comp rx=0 tx=0).stop 2024-05-08T23:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.992+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/294014601 shutdown_connections 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.992+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/294014601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e15433e270 0x55e154341e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.992+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/294014601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e154338480 0x55e154338860 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.992+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/294014601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433d950 0x55e154332910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.992+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/294014601 >> 172.21.15.12:0/294014601 conn(0x55e1542961b0 msgr2=0x55e15433fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.992+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/294014601 shutdown_connections 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.992+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/294014601 wait complete. 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.992+0000 7ff1ee54eb80 1 -- start start 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1ee54eb80 1 Processor -- start 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 0x55e154348be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433d950 0x55e154349120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1ee54eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433e270 0x55e15433b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15433ba90 con 0x55e15433e270 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15433bc00 con 0x55e154338480 2024-05-08T23:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1ee54eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15433bdc0 con 0x55e15433d950 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1ee54eb80 10 client.0 _ll_register_callbacks cb 0x55e15422dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1e0931700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433e270 0x55e15433b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1db7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433d950 0x55e154349120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 0x55e154348be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1dbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 0x55e154348be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48792/0 (socket says 172.21.15.12:48792) 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1db7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433d950 0x55e154349120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:35956/0 (socket says 172.21.15.12:35956) 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1e0931700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433e270 0x55e15433b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35948/0 (socket says 172.21.15.12:35948) 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1dbfff700 1 -- 172.21.15.12:0/3260312340 learned_addr learned my addr 172.21.15.12:0/3260312340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1e0931700 1 -- 172.21.15.12:0/3260312340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433d950 msgr2=0x55e154349120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1e0931700 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433d950 0x55e154349120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1e0931700 1 -- 172.21.15.12:0/3260312340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 msgr2=0x55e154348be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1e0931700 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 0x55e154348be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1e0931700 1 -- 172.21.15.12:0/3260312340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc1a60f0 con 0x55e15433e270 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1e0931700 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433e270 0x55e15433b380 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4099c10 tx=0x7ff1d4004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aedf49dacf5fd5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.993+0000 7ff1bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/3260312340 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d401b520 con 0x55e15433e270 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/3260312340 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff1d4099a50 con 0x55e15433e270 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3260312340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1d409a150 con 0x55e15433e270 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/3260312340 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d40b2380 con 0x55e15433e270 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1ee54eb80 10 client.19299 fetch_fsmap learned FSMap version 59 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3260312340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55e15424e500 con 0x55e15433e270 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/3260312340 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1d4065020 con 0x55e15433e270 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/3260312340 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 59) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff1d4098fd0 con 0x55e15433e270 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1ee54eb80 10 client.19299 fetch_fsmap finished waiting for FSMap version 59 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1ee54eb80 10 client.19299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1ee54eb80 20 client.19299 populate_metadata read hostname 'smithi012' 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e1543a81b0 0x55e1543aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1ee54eb80 4 client.19299 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T23:59:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3260312340 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e154280e60 con 0x55e1543a81b0 2024-05-08T23:59:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.994+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e1543a81b0 0x55e1543aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:59:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.995+0000 7ff1dbfff700 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e1543a81b0 0x55e1543aa5a0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed293b71a75a08b6 server_cookie=fe4af9aa5f7241c3 in_seq=0 out_seq=0 2024-05-08T23:59:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:56.995+0000 7ff1bb7fe700 10 client.19299 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.061+0000 7ff1bb7fe700 1 -- 172.21.15.12:0/3260312340 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e154280e60 con 0x55e1543a81b0 2024-05-08T23:59:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.061+0000 7ff1bb7fe700 10 client.19299 handle_command_reply: tid=0 2024-05-08T23:59:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.061+0000 7ff1ee54eb80 1 client.19299 shutdown 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.061+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3260312340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433e270 msgr2=0x55e15433b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.061+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433e270 0x55e15433b380 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4099c10 tx=0x7ff1d4004d30 comp rx=0 tx=0).stop 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3260312340 shutdown_connections 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e15433e270 0x55e15433b380 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e1543a81b0 0x55e1543aa5a0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e154338480 0x55e154348be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1ee54eb80 1 --2- 172.21.15.12:0/3260312340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e15433d950 0x55e154349120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3260312340 >> 172.21.15.12:0/3260312340 conn(0x55e1542961b0 msgr2=0x55e1543375b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1bb7fe700 0 client.19299 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3260312340 shutdown_connections 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1ee54eb80 1 -- 172.21.15.12:0/3260312340 wait complete. 2024-05-08T23:59:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1ee54eb80 20 client.19299 trim_cache size 0 max 16384 2024-05-08T23:59:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-08T23:59:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-08T23:59:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-08T23:59:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1e2394700 1 -- 172.21.15.12:0/1709783067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1c40a1290 msgr2=0x7ff1c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1e2394700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1c40a1290 0x7ff1c40a3750 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00b2e0 tx=0x7ff1cc005bc0 comp rx=0 tx=0).stop 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1e2394700 1 -- 172.21.15.12:0/1709783067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1dc105e60 msgr2=0x7ff1dc10e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.062+0000 7ff1e2394700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1dc105e60 0x7ff1dc10e040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400efc0 tx=0x7ff1d400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.063+0000 7ff1e2394700 1 -- 172.21.15.12:0/1709783067 shutdown_connections 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.063+0000 7ff1e2394700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff1c40a1290 0x7ff1c40a3750 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.063+0000 7ff1e2394700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1dc105e60 0x7ff1dc10e040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.063+0000 7ff1e2394700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1dc0ffda0 0x7ff1dc10d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.063+0000 7ff1e2394700 1 --2- 172.21.15.12:0/1709783067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1dc100760 0x7ff1dc10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.063+0000 7ff1e2394700 1 -- 172.21.15.12:0/1709783067 >> 172.21.15.12:0/1709783067 conn(0x7ff1dc076f10 msgr2=0x7ff1dc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.063+0000 7ff1e2394700 1 -- 172.21.15.12:0/1709783067 shutdown_connections 2024-05-08T23:59:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-08T23:59:57.063+0000 7ff1e2394700 1 -- 172.21.15.12:0/1709783067 wait complete. 2024-05-08T23:59:57.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 770124 6.15 MB/sec execute 593 sec latency 56.327 ms 2024-05-08T23:59:57.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:57 smithi012 ceph-mon[42503]: pgmap v433: 97 pgs: 97 active+clean; 225 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-05-08T23:59:58.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:57 smithi204 ceph-mon[47373]: pgmap v433: 97 pgs: 97 active+clean; 225 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-05-08T23:59:58.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:57 smithi160 ceph-mon[46152]: pgmap v433: 97 pgs: 97 active+clean; 225 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-05-08T23:59:58.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 771386 6.15 MB/sec execute 594 sec latency 17.087 ms 2024-05-08T23:59:59.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 773373 6.16 MB/sec execute 595 sec latency 19.524 ms 2024-05-08T23:59:59.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 08 23:59:59 smithi012 ceph-mon[42503]: pgmap v434: 97 pgs: 97 active+clean; 225 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-05-09T00:00:00.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 08 23:59:59 smithi204 ceph-mon[47373]: pgmap v434: 97 pgs: 97 active+clean; 225 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-05-09T00:00:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 08 23:59:59 smithi160 ceph-mon[46152]: pgmap v434: 97 pgs: 97 active+clean; 225 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-05-09T00:00:00.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 774152 6.16 MB/sec execute 596 sec latency 552.698 ms 2024-05-09T00:00:00.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:00 smithi012 ceph-mon[42503]: overall HEALTH_OK 2024-05-09T00:00:01.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:00 smithi204 ceph-mon[47373]: overall HEALTH_OK 2024-05-09T00:00:01.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:00 smithi160 ceph-mon[46152]: overall HEALTH_OK 2024-05-09T00:00:01.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 774152 6.15 MB/sec execute 597 sec latency 1552.785 ms 2024-05-09T00:00:01.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:01 smithi012 ceph-mon[42503]: pgmap v435: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 296 op/s 2024-05-09T00:00:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:01 smithi204 ceph-mon[47373]: pgmap v435: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 296 op/s 2024-05-09T00:00:02.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:01 smithi160 ceph-mon[46152]: pgmap v435: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 296 op/s 2024-05-09T00:00:02.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 774216 6.14 MB/sec execute 598 sec latency 2434.190 ms 2024-05-09T00:00:03.903 INFO:tasks.workunit.client.0.smithi012.stdout: 1 775086 6.14 MB/sec execute 599 sec latency 46.875 ms 2024-05-09T00:00:04.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:04 smithi012 ceph-mon[42503]: osdmap e109: 12 total, 12 up, 12 in 2024-05-09T00:00:04.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:04 smithi012 ceph-mon[42503]: pgmap v437: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-05-09T00:00:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:04 smithi204 ceph-mon[47373]: osdmap e109: 12 total, 12 up, 12 in 2024-05-09T00:00:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:04 smithi204 ceph-mon[47373]: pgmap v437: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-05-09T00:00:04.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:04 smithi160 ceph-mon[46152]: osdmap e109: 12 total, 12 up, 12 in 2024-05-09T00:00:04.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:04 smithi160 ceph-mon[46152]: pgmap v437: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-05-09T00:00:04.902 INFO:tasks.workunit.client.0.smithi012.stdout: 1 cleanup 600 sec 2024-05-09T00:00:05.110 INFO:tasks.workunit.client.0.smithi012.stdout: 0 cleanup 600 sec 2024-05-09T00:00:05.110 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-05-09T00:00:05.111 INFO:tasks.workunit.client.0.smithi012.stdout: Operation Count AvgLat MaxLat 2024-05-09T00:00:05.111 INFO:tasks.workunit.client.0.smithi012.stdout: ---------------------------------------- 2024-05-09T00:00:05.111 INFO:tasks.workunit.client.0.smithi012.stdout: NTCreateX 117652 1.563 9981.779 2024-05-09T00:00:05.111 INFO:tasks.workunit.client.0.smithi012.stdout: Close 86430 0.003 4.464 2024-05-09T00:00:05.111 INFO:tasks.workunit.client.0.smithi012.stdout: Rename 4981 39.752 4733.132 2024-05-09T00:00:05.111 INFO:tasks.workunit.client.0.smithi012.stdout: Unlink 23761 1.728 2602.256 2024-05-09T00:00:05.111 INFO:tasks.workunit.client.0.smithi012.stdout: Deltree 2 78.616 91.589 2024-05-09T00:00:05.111 INFO:tasks.workunit.client.0.smithi012.stdout: Mkdir 1 0.016 0.016 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: Qpathinfo 106692 1.177 13724.185 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: Qfileinfo 18660 0.005 1.903 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: Qfsinfo 19536 0.209 9.001 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: Sfileinfo 9598 0.015 4.179 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: Find 41221 0.338 6190.181 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: WriteX 58484 0.068 88.872 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: ReadX 184255 0.009 21.235 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: LockX 382 1.074 11.990 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: UnlockX 382 0.786 5.526 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: Flush 8252 2.979 33.974 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-05-09T00:00:05.112 INFO:tasks.workunit.client.0.smithi012.stdout:Throughput 6.13569 MB/sec 1 clients 1 procs max_latency=13724.200 ms 2024-05-09T00:00:05.113 INFO:tasks.workunit.client.0.smithi012.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-09T00:00:05.113 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-05-09T00:00:05.113 INFO:tasks.workunit.client.0.smithi012.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-09T00:00:05.113 INFO:tasks.workunit.client.0.smithi012.stdout:9 of 10 processes prepared for launch 0 sec 2024-05-09T00:00:05.213 INFO:tasks.workunit.client.0.smithi012.stdout:10 of 10 processes prepared for launch 0 sec 2024-05-09T00:00:05.213 INFO:tasks.workunit.client.0.smithi012.stdout:releasing clients 2024-05-09T00:00:05.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:05 smithi012 ceph-mon[42503]: pgmap v438: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-05-09T00:00:06.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:05 smithi204 ceph-mon[47373]: pgmap v438: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-05-09T00:00:06.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:05 smithi160 ceph-mon[46152]: pgmap v438: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-05-09T00:00:06.213 INFO:tasks.workunit.client.0.smithi012.stdout: 10 396 140.86 MB/sec warmup 1 sec latency 607.543 ms 2024-05-09T00:00:07.213 INFO:tasks.workunit.client.0.smithi012.stdout: 10 861 101.25 MB/sec warmup 2 sec latency 1607.606 ms 2024-05-09T00:00:07.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:07 smithi012 ceph-mon[42503]: pgmap v439: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-05-09T00:00:08.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:07 smithi204 ceph-mon[47373]: pgmap v439: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-05-09T00:00:08.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:07 smithi160 ceph-mon[46152]: pgmap v439: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-05-09T00:00:08.213 INFO:tasks.workunit.client.0.smithi012.stdout: 10 1492 80.75 MB/sec warmup 3 sec latency 2607.675 ms 2024-05-09T00:00:09.213 INFO:tasks.workunit.client.0.smithi012.stdout: 10 2273 66.24 MB/sec warmup 4 sec latency 3607.762 ms 2024-05-09T00:00:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:09 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:00:09.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:09 smithi012 ceph-mon[42503]: pgmap v440: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-05-09T00:00:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:09 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:00:10.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:09 smithi204 ceph-mon[47373]: pgmap v440: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-05-09T00:00:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:09 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:00:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:09 smithi160 ceph-mon[46152]: pgmap v440: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-05-09T00:00:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.192+0000 7fddf572e700 1 -- 172.21.15.12:0/2248025021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0105f20 msgr2=0x7fddf0106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.192+0000 7fddf572e700 1 --2- 172.21.15.12:0/2248025021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0105f20 0x7fddf0106300 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fdde000b0d0 tx=0x7fdde000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.192+0000 7fddf572e700 1 -- 172.21.15.12:0/2248025021 shutdown_connections 2024-05-09T00:00:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.192+0000 7fddf572e700 1 --2- 172.21.15.12:0/2248025021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf0100a40 0x7fddf010cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.192+0000 7fddf572e700 1 --2- 172.21.15.12:0/2248025021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0105f20 0x7fddf0106300 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.192+0000 7fddf572e700 1 --2- 172.21.15.12:0/2248025021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddf0100080 0x7fddf0100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.192+0000 7fddf572e700 1 -- 172.21.15.12:0/2248025021 >> 172.21.15.12:0/2248025021 conn(0x7fddf0076c10 msgr2=0x7fddf0077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.192+0000 7fddf572e700 1 -- 172.21.15.12:0/2248025021 shutdown_connections 2024-05-09T00:00:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.192+0000 7fddf572e700 1 -- 172.21.15.12:0/2248025021 wait complete. 2024-05-09T00:00:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.193+0000 7fddf572e700 1 Processor -- start 2024-05-09T00:00:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.193+0000 7fddf572e700 1 -- start start 2024-05-09T00:00:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.193+0000 7fddf572e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0100080 0x7fddf0196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.193+0000 7fddf572e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddf0100a40 0x7fddf0196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.193+0000 7fddf572e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf0105f20 0x7fddf019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.193+0000 7fddf572e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf019b240 con 0x7fddf0100080 2024-05-09T00:00:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.193+0000 7fddf572e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf019b3b0 con 0x7fddf0105f20 2024-05-09T00:00:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.193+0000 7fddf572e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf019b570 con 0x7fddf0100a40 2024-05-09T00:00:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.193+0000 7fddeeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0100080 0x7fddf0196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddeeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0100080 0x7fddf0196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60348/0 (socket says 172.21.15.12:60348) 2024-05-09T00:00:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddeeffd700 1 -- 172.21.15.12:0/60957120 learned_addr learned my addr 172.21.15.12:0/60957120 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddef7fe700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf0105f20 0x7fddf019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddee7fc700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddf0100a40 0x7fddf0196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddeeffd700 1 -- 172.21.15.12:0/60957120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddf0100a40 msgr2=0x7fddf0196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddeeffd700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddf0100a40 0x7fddf0196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddeeffd700 1 -- 172.21.15.12:0/60957120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf0105f20 msgr2=0x7fddf019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddeeffd700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf0105f20 0x7fddf019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddeeffd700 1 -- 172.21.15.12:0/60957120 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0009d00 con 0x7fddf0100080 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddeeffd700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0100080 0x7fddf0196320 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fdde000bd50 tx=0x7fdde000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fdde7fff700 1 -- 172.21.15.12:0/60957120 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde000e040 con 0x7fddf0100080 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fdde7fff700 1 -- 172.21.15.12:0/60957120 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde001b8e0 con 0x7fddf0100080 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddf572e700 1 -- 172.21.15.12:0/60957120 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf019b7f0 con 0x7fddf0100080 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fdde7fff700 1 -- 172.21.15.12:0/60957120 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde00127f0 con 0x7fddf0100080 2024-05-09T00:00:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.194+0000 7fddf572e700 1 -- 172.21.15.12:0/60957120 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf01a3090 con 0x7fddf0100080 2024-05-09T00:00:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.197+0000 7fdde7fff700 1 -- 172.21.15.12:0/60957120 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdde001ba50 con 0x7fddf0100080 2024-05-09T00:00:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.197+0000 7fdde7fff700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdddc0a1310 0x7fdddc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.197+0000 7fdde7fff700 1 -- 172.21.15.12:0/60957120 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdde00cdfb0 con 0x7fddf0100080 2024-05-09T00:00:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.197+0000 7fddee7fc700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdddc0a1310 0x7fdddc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.198+0000 7fddee7fc700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdddc0a1310 0x7fdddc0a37d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fddf0197870 tx=0x7fddd80073d0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.198+0000 7fde018e8b80 1 Processor -- start 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.198+0000 7fde018e8b80 1 -- start start 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.198+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 0x556571c36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c3cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fde018e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571b830f0 con 0x556571c30350 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fde018e8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571b80ab0 con 0x556571c35db0 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fde018e8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571b7c3a0 con 0x556571c30d10 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddee7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddeeffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 0x556571c36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddef7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c3cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddee7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c307d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60356/0 (socket says 172.21.15.12:60356) 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddee7fc700 1 -- 172.21.15.12:0/3992119912 learned_addr learned my addr 172.21.15.12:0/3992119912 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddeeffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 0x556571c36190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44148/0 (socket says 172.21.15.12:44148) 2024-05-09T00:00:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddee7fc700 1 -- 172.21.15.12:0/3992119912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 msgr2=0x556571c3cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddee7fc700 1 --2- 172.21.15.12:0/3992119912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c3cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddee7fc700 1 -- 172.21.15.12:0/3992119912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 msgr2=0x556571c36190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddee7fc700 1 --2- 172.21.15.12:0/3992119912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 0x556571c36190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddee7fc700 1 -- 172.21.15.12:0/3992119912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565719fd350 con 0x556571c30350 2024-05-09T00:00:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddeeffd700 1 --2- 172.21.15.12:0/3992119912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 0x556571c36190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fddee7fc700 1 --2- 172.21.15.12:0/3992119912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c307d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fddd8010e60 tx=0x7fddd8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5797bc697747d371 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fdde57fa700 1 -- 172.21.15.12:0/3992119912 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8011db0 con 0x556571c30350 2024-05-09T00:00:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fdde57fa700 1 -- 172.21.15.12:0/3992119912 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddd8015070 con 0x556571c30350 2024-05-09T00:00:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.199+0000 7fdde57fa700 1 -- 172.21.15.12:0/3992119912 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8003e60 con 0x556571c30350 2024-05-09T00:00:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.200+0000 7fde018e8b80 1 -- 172.21.15.12:0/3992119912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 msgr2=0x556571c307d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.200+0000 7fde018e8b80 1 --2- 172.21.15.12:0/3992119912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c307d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fddd8010e60 tx=0x7fddd8013000 comp rx=0 tx=0).stop 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.200+0000 7fde018e8b80 1 -- 172.21.15.12:0/3992119912 shutdown_connections 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.200+0000 7fde018e8b80 1 --2- 172.21.15.12:0/3992119912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c3cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.200+0000 7fde018e8b80 1 --2- 172.21.15.12:0/3992119912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 0x556571c36190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.200+0000 7fde018e8b80 1 --2- 172.21.15.12:0/3992119912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c307d0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.200+0000 7fde018e8b80 1 -- 172.21.15.12:0/3992119912 >> 172.21.15.12:0/3992119912 conn(0x556571b8a1b0 msgr2=0x556571c26710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.200+0000 7fde018e8b80 1 -- 172.21.15.12:0/3992119912 shutdown_connections 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.200+0000 7fde018e8b80 1 -- 172.21.15.12:0/3992119912 wait complete. 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fde018e8b80 1 -- start start 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fde018e8b80 1 Processor -- start 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c3c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30d10 0x556571c39c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c35db0 0x556571c3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fde018e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571b18500 con 0x556571c30d10 2024-05-09T00:00:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fde018e8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571b181a0 con 0x556571c30350 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fde018e8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565719ff640 con 0x556571c35db0 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fde018e8b80 10 client.0 _ll_register_callbacks cb 0x556571b21ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fddee7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30d10 0x556571c39c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fddeeffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c3c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fddeeffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c3c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44152/0 (socket says 172.21.15.12:44152) 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fddef7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c35db0 0x556571c3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.201+0000 7fddeeffd700 1 -- 172.21.15.12:0/1240628645 learned_addr learned my addr 172.21.15.12:0/1240628645 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fddeeffd700 1 -- 172.21.15.12:0/1240628645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c35db0 msgr2=0x556571c3a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fddeeffd700 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c35db0 0x556571c3a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fddeeffd700 1 -- 172.21.15.12:0/1240628645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30d10 msgr2=0x556571c39c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fddeeffd700 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30d10 0x556571c39c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fddeeffd700 1 -- 172.21.15.12:0/1240628645 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf01a3090 con 0x556571c30350 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fddeeffd700 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c3c990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009fd0 tx=0x7fdde009ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd25372957ff916a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fddd67fc700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:00:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fddd67fc700 1 -- 172.21.15.12:0/1240628645 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde009b140 con 0x556571c30350 2024-05-09T00:00:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fddd67fc700 1 -- 172.21.15.12:0/1240628645 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdde00cbe40 con 0x556571c30350 2024-05-09T00:00:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fde018e8b80 1 -- 172.21.15.12:0/1240628645 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556571ad5210 con 0x556571c30350 2024-05-09T00:00:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fddd67fc700 1 -- 172.21.15.12:0/1240628645 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdde009ef70 con 0x556571c30350 2024-05-09T00:00:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fde018e8b80 10 client.29024 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.202+0000 7fde018e8b80 1 -- 172.21.15.12:0/1240628645 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7fddd8010b40 con 0x556571c30350 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fddd67fc700 1 -- 172.21.15.12:0/1240628645 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556571ad5210 con 0x556571c30350 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fddd67fc700 1 -- 172.21.15.12:0/1240628645 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fdde009b3a0 con 0x556571c30350 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fde018e8b80 10 client.29024 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fde018e8b80 10 client.29024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fde018e8b80 20 client.29024 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fde018e8b80 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556571c49d00 0x556571c4c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fde018e8b80 4 client.29024 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fde018e8b80 1 -- 172.21.15.12:0/1240628645 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556571bfb7b0 con 0x556571c49d00 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fddee7fc700 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556571c49d00 0x556571c4c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fddee7fc700 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556571c49d00 0x556571c4c120 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22cce90e6a7e2f0a server_cookie=8cba3546f719176b in_seq=0 out_seq=0 2024-05-09T00:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.203+0000 7fddd67fc700 10 client.29024 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fddd67fc700 1 -- 172.21.15.12:0/1240628645 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556571bfb7b0 con 0x556571c49d00 2024-05-09T00:00:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fddd67fc700 10 client.29024 handle_command_reply: tid=0 2024-05-09T00:00:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 client.29024 shutdown 2024-05-09T00:00:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 -- 172.21.15.12:0/1240628645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 msgr2=0x556571c3c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c3c990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009fd0 tx=0x7fdde009ad00 comp rx=0 tx=0).stop 2024-05-09T00:00:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 -- 172.21.15.12:0/1240628645 shutdown_connections 2024-05-09T00:00:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c35db0 0x556571c3a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556571c49d00 0x556571c4c120 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c3c990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 --2- 172.21.15.12:0/1240628645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30d10 0x556571c39c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fddd67fc700 0 client.29024 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 -- 172.21.15.12:0/1240628645 >> 172.21.15.12:0/1240628645 conn(0x556571b8a1b0 msgr2=0x556571c2f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 -- 172.21.15.12:0/1240628645 shutdown_connections 2024-05-09T00:00:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 1 -- 172.21.15.12:0/1240628645 wait complete. 2024-05-09T00:00:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.209+0000 7fde018e8b80 20 client.29024 trim_cache size 0 max 16384 2024-05-09T00:00:10.213 INFO:tasks.workunit.client.0.smithi012.stdout: 10 2877 62.97 MB/sec warmup 5 sec latency 4607.812 ms 2024-05-09T00:00:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.226+0000 7fde018e8b80 1 Processor -- start 2024-05-09T00:00:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.226+0000 7fde018e8b80 1 -- start start 2024-05-09T00:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.226+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c37990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.226+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.226+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c35db0 0x556571c410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.226+0000 7fde018e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571c28670 con 0x556571c35db0 2024-05-09T00:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.226+0000 7fde018e8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571c512d0 con 0x556571c30350 2024-05-09T00:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.226+0000 7fde018e8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571c281b0 con 0x556571c30d10 2024-05-09T00:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddeeffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c37990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c35db0 0x556571c410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddee7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddee7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c36190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40460/0 (socket says 172.21.15.12:40460) 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddeeffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c37990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44162/0 (socket says 172.21.15.12:44162) 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c35db0 0x556571c410f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60382/0 (socket says 172.21.15.12:60382) 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddee7fc700 1 -- 172.21.15.12:0/588425940 learned_addr learned my addr 172.21.15.12:0/588425940 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddee7fc700 1 -- 172.21.15.12:0/588425940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 msgr2=0x556571c37990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddee7fc700 1 --2- 172.21.15.12:0/588425940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c37990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddee7fc700 1 -- 172.21.15.12:0/588425940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c35db0 msgr2=0x556571c410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddee7fc700 1 --2- 172.21.15.12:0/588425940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c35db0 0x556571c410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddee7fc700 1 -- 172.21.15.12:0/588425940 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565719a9b00 con 0x556571c30d10 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddef7fe700 1 --2- 172.21.15.12:0/588425940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c35db0 0x556571c410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fddee7fc700 1 --2- 172.21.15.12:0/588425940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c36190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fddd80074d0 tx=0x7fddd8007460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d78bb5c1cc683daa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fdde5ffb700 1 -- 172.21.15.12:0/588425940 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8010e90 con 0x556571c30d10 2024-05-09T00:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fdde5ffb700 1 -- 172.21.15.12:0/588425940 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddd8003e50 con 0x556571c30d10 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.227+0000 7fdde5ffb700 1 -- 172.21.15.12:0/588425940 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd80054f0 con 0x556571c30d10 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.228+0000 7fde018e8b80 1 -- 172.21.15.12:0/588425940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 msgr2=0x556571c36190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.228+0000 7fde018e8b80 1 --2- 172.21.15.12:0/588425940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c36190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fddd80074d0 tx=0x7fddd8007460 comp rx=0 tx=0).stop 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.228+0000 7fde018e8b80 1 -- 172.21.15.12:0/588425940 shutdown_connections 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.228+0000 7fde018e8b80 1 --2- 172.21.15.12:0/588425940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c35db0 0x556571c410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.228+0000 7fde018e8b80 1 --2- 172.21.15.12:0/588425940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c30350 0x556571c37990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.228+0000 7fde018e8b80 1 --2- 172.21.15.12:0/588425940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c36190 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.228+0000 7fde018e8b80 1 -- 172.21.15.12:0/588425940 >> 172.21.15.12:0/588425940 conn(0x556571b8a1b0 msgr2=0x556571c33d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.228+0000 7fde018e8b80 1 -- 172.21.15.12:0/588425940 shutdown_connections 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.228+0000 7fde018e8b80 1 -- 172.21.15.12:0/588425940 wait complete. 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fde018e8b80 1 -- start start 2024-05-09T00:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fde018e8b80 1 Processor -- start 2024-05-09T00:00:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c3f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c3f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fde018e8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 0x556571c3fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fde018e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571c9f1c0 con 0x556571c30350 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fde018e8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571c9f300 con 0x556571c35db0 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fde018e8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556571c9f440 con 0x556571c30d10 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fde018e8b80 10 client.0 _ll_register_callbacks cb 0x556571b21ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fddeeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c3f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fddeeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c3f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60394/0 (socket says 172.21.15.12:60394) 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fddeeffd700 1 -- 172.21.15.12:0/913887592 learned_addr learned my addr 172.21.15.12:0/913887592 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fddee7fc700 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c3f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.229+0000 7fddef7fe700 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 0x556571c3fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fddeeffd700 1 -- 172.21.15.12:0/913887592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 msgr2=0x556571c3f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fddeeffd700 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c3f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fddeeffd700 1 -- 172.21.15.12:0/913887592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 msgr2=0x556571c3fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fddeeffd700 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 0x556571c3fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fddeeffd700 1 -- 172.21.15.12:0/913887592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde8009c00 con 0x556571c30350 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fddeeffd700 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c3f0d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009cd0 tx=0x7fdde0007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915d3c20babaa531 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fddd67fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fddd67fc700 1 -- 172.21.15.12:0/913887592 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde009a6d0 con 0x556571c30350 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fddd67fc700 1 -- 172.21.15.12:0/913887592 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdde00cbe40 con 0x556571c30350 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fde018e8b80 1 -- 172.21.15.12:0/913887592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddd8015070 con 0x556571c30350 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fddd67fc700 1 -- 172.21.15.12:0/913887592 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdde009a890 con 0x556571c30350 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fde018e8b80 10 client.19317 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.230+0000 7fde018e8b80 1 -- 172.21.15.12:0/913887592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x5565719fd350 con 0x556571c30350 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fddd67fc700 1 -- 172.21.15.12:0/913887592 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddd8015070 con 0x556571c30350 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fddd67fc700 1 -- 172.21.15.12:0/913887592 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fdde00a30a0 con 0x556571c30350 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fde018e8b80 10 client.19317 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fde018e8b80 10 client.19317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fde018e8b80 20 client.19317 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fde018e8b80 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556571ca0780 0x556571ca2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fde018e8b80 4 client.19317 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:00:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fde018e8b80 1 -- 172.21.15.12:0/913887592 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556571b74e60 con 0x556571ca0780 2024-05-09T00:00:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fddee7fc700 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556571ca0780 0x556571ca2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fddee7fc700 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556571ca0780 0x556571ca2ba0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3cd942c6c43041c server_cookie=d83c52595f9708aa in_seq=0 out_seq=0 2024-05-09T00:00:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.231+0000 7fddd67fc700 10 client.19317 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.650+0000 7fddd67fc700 1 -- 172.21.15.12:0/913887592 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556571b74e60 con 0x556571ca0780 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.650+0000 7fddd67fc700 10 client.19317 handle_command_reply: tid=0 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.650+0000 7fde018e8b80 1 client.19317 shutdown 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.650+0000 7fde018e8b80 1 -- 172.21.15.12:0/913887592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 msgr2=0x556571c3f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.650+0000 7fde018e8b80 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c3f0d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009cd0 tx=0x7fdde0007f10 comp rx=0 tx=0).stop 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.651+0000 7fde018e8b80 1 -- 172.21.15.12:0/913887592 shutdown_connections 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.651+0000 7fde018e8b80 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556571ca0780 0x556571ca2ba0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.651+0000 7fde018e8b80 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556571c35db0 0x556571c3fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.651+0000 7fde018e8b80 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556571c30350 0x556571c3f0d0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.651+0000 7fde018e8b80 1 --2- 172.21.15.12:0/913887592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556571c30d10 0x556571c3f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.651+0000 7fddd67fc700 0 client.19317 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.651+0000 7fde018e8b80 1 -- 172.21.15.12:0/913887592 >> 172.21.15.12:0/913887592 conn(0x556571b8a1b0 msgr2=0x556571c45ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.651+0000 7fde018e8b80 1 -- 172.21.15.12:0/913887592 shutdown_connections 2024-05-09T00:00:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.651+0000 7fde018e8b80 1 -- 172.21.15.12:0/913887592 wait complete. 2024-05-09T00:00:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.651+0000 7fde018e8b80 20 client.19317 trim_cache size 0 max 16384 2024-05-09T00:00:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:00:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:00:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:00:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:00:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 -- 172.21.15.12:0/60957120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdddc0a1310 msgr2=0x7fdddc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdddc0a1310 0x7fdddc0a37d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fddf0197870 tx=0x7fddd80073d0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 -- 172.21.15.12:0/60957120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0100080 msgr2=0x7fddf0196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0100080 0x7fddf0196320 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fdde000bd50 tx=0x7fdde000bd80 comp rx=0 tx=0).stop 2024-05-09T00:00:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 -- 172.21.15.12:0/60957120 shutdown_connections 2024-05-09T00:00:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdddc0a1310 0x7fdddc0a37d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf0105f20 0x7fddf019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0100080 0x7fddf0196320 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 --2- 172.21.15.12:0/60957120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddf0100a40 0x7fddf0196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 -- 172.21.15.12:0/60957120 >> 172.21.15.12:0/60957120 conn(0x7fddf0076c10 msgr2=0x7fddf00feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 -- 172.21.15.12:0/60957120 shutdown_connections 2024-05-09T00:00:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:10.654+0000 7fddf572e700 1 -- 172.21.15.12:0/60957120 wait complete. 2024-05-09T00:00:11.213 INFO:tasks.workunit.client.0.smithi012.stdout: 10 3307 59.10 MB/sec warmup 6 sec latency 5607.853 ms 2024-05-09T00:00:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:11 smithi012 ceph-mon[42503]: osdmap e110: 12 total, 12 up, 12 in 2024-05-09T00:00:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:11 smithi012 ceph-mon[42503]: pgmap v442: 97 pgs: 97 active+clean; 226 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 634 op/s 2024-05-09T00:00:12.213 INFO:tasks.workunit.client.0.smithi012.stdout: 10 3542 52.36 MB/sec warmup 7 sec latency 6607.904 ms 2024-05-09T00:00:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:11 smithi204 ceph-mon[47373]: osdmap e110: 12 total, 12 up, 12 in 2024-05-09T00:00:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:11 smithi204 ceph-mon[47373]: pgmap v442: 97 pgs: 97 active+clean; 226 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 634 op/s 2024-05-09T00:00:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:11 smithi160 ceph-mon[46152]: osdmap e110: 12 total, 12 up, 12 in 2024-05-09T00:00:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:11 smithi160 ceph-mon[46152]: pgmap v442: 97 pgs: 97 active+clean; 226 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 634 op/s 2024-05-09T00:00:12.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:00:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.923+0000 7f14a1965700 1 -- 172.21.15.12:0/4294032316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f149c068c90 msgr2=0x7f149c105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.923+0000 7f14a1965700 1 --2- 172.21.15.12:0/4294032316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f149c068c90 0x7f149c105b60 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f14900099c0 tx=0x7f1490009cd0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.924+0000 7f14a1965700 1 -- 172.21.15.12:0/4294032316 shutdown_connections 2024-05-09T00:00:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.924+0000 7f14a1965700 1 --2- 172.21.15.12:0/4294032316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149c106290 0x7f149c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.924+0000 7f14a1965700 1 --2- 172.21.15.12:0/4294032316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f149c0682e0 0x7f149c0686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.924+0000 7f14a1965700 1 --2- 172.21.15.12:0/4294032316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f149c068c90 0x7f149c105b60 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.924+0000 7f14a1965700 1 -- 172.21.15.12:0/4294032316 >> 172.21.15.12:0/4294032316 conn(0x7f149c076f40 msgr2=0x7f149c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.924+0000 7f14a1965700 1 -- 172.21.15.12:0/4294032316 shutdown_connections 2024-05-09T00:00:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.924+0000 7f14a1965700 1 -- 172.21.15.12:0/4294032316 wait complete. 2024-05-09T00:00:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.924+0000 7f14a1965700 1 Processor -- start 2024-05-09T00:00:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f14a1965700 1 -- start start 2024-05-09T00:00:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f14a1965700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149c0682e0 0x7f149c0ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f14a1965700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f149c106290 0x7f149c1003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f14a1965700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f149c1008e0 0x7f149c1a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f14a1965700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c1a5b80 con 0x7f149c1008e0 2024-05-09T00:00:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f14a1965700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c1a5cf0 con 0x7f149c0682e0 2024-05-09T00:00:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f14a1965700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c1a5eb0 con 0x7f149c106290 2024-05-09T00:00:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f149affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149c0682e0 0x7f149c0ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f149affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149c0682e0 0x7f149c0ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44174/0 (socket says 172.21.15.12:44174) 2024-05-09T00:00:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f149affd700 1 -- 172.21.15.12:0/1341411595 learned_addr learned my addr 172.21.15.12:0/1341411595 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f149b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f149c1008e0 0x7f149c1a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f149a7fc700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f149c106290 0x7f149c1003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f149affd700 1 -- 172.21.15.12:0/1341411595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f149c106290 msgr2=0x7f149c1003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f149affd700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f149c106290 0x7f149c1003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f149affd700 1 -- 172.21.15.12:0/1341411595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f149c1008e0 msgr2=0x7f149c1a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f149affd700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f149c1008e0 0x7f149c1a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.925+0000 7f149affd700 1 -- 172.21.15.12:0/1341411595 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c009520 con 0x7f149c0682e0 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.926+0000 7f149affd700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149c0682e0 0x7f149c0ffe60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f148c00eb30 tx=0x7f148c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.926+0000 7f14a0963700 1 -- 172.21.15.12:0/1341411595 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c009880 con 0x7f149c0682e0 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.926+0000 7f14a0963700 1 -- 172.21.15.12:0/1341411595 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f148c0044e0 con 0x7f149c0682e0 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.926+0000 7f14a1965700 1 -- 172.21.15.12:0/1341411595 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14900095f0 con 0x7f149c0682e0 2024-05-09T00:00:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.926+0000 7f14a0963700 1 -- 172.21.15.12:0/1341411595 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c003d80 con 0x7f149c0682e0 2024-05-09T00:00:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.926+0000 7f14a1965700 1 -- 172.21.15.12:0/1341411595 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c1a64e0 con 0x7f149c0682e0 2024-05-09T00:00:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.928+0000 7f14a0963700 1 -- 172.21.15.12:0/1341411595 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f148c018430 con 0x7f149c0682e0 2024-05-09T00:00:12.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.929+0000 7f14a0963700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14840a1290 0x7f14840a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.929+0000 7f149a7fc700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14840a1290 0x7f14840a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.929+0000 7f14a0963700 1 -- 172.21.15.12:0/1341411595 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f148c014070 con 0x7f149c0682e0 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f14adb1fb80 1 Processor -- start 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f14adb1fb80 1 -- start start 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149a7fc700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14840a1290 0x7f14840a3750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1490009fd0 tx=0x7f1490019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a95053d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a95053d720 0x55a950537350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a950537890 0x55a950537cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95048e150 con 0x55a950537890 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95048bb10 con 0x55a95053d720 2024-05-09T00:00:12.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950487400 con 0x55a95053ce00 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a95053d720 0x55a950537350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a950537890 0x55a950537cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a95053d720 0x55a950537350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44178/0 (socket says 172.21.15.12:44178) 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a950537890 0x55a950537cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60412/0 (socket says 172.21.15.12:60412) 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149a7fc700 1 -- 172.21.15.12:0/3200290584 learned_addr learned my addr 172.21.15.12:0/3200290584 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149affd700 1 --2- 172.21.15.12:0/3200290584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a95053d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149a7fc700 1 -- 172.21.15.12:0/3200290584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 msgr2=0x55a95053d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149a7fc700 1 --2- 172.21.15.12:0/3200290584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a95053d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149a7fc700 1 -- 172.21.15.12:0/3200290584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a950537890 msgr2=0x55a950537cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149a7fc700 1 --2- 172.21.15.12:0/3200290584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a950537890 0x55a950537cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.930+0000 7f149a7fc700 1 -- 172.21.15.12:0/3200290584 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9503083b0 con 0x55a95053d720 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.931+0000 7f149a7fc700 1 --2- 172.21.15.12:0/3200290584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a95053d720 0x55a950537350 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1490015000 tx=0x7f149001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a766af8a6f855670 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.931+0000 7f1481ffb700 1 -- 172.21.15.12:0/3200290584 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14900054e0 con 0x55a95053d720 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.931+0000 7f1481ffb700 1 -- 172.21.15.12:0/3200290584 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f149001d040 con 0x55a95053d720 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.931+0000 7f1481ffb700 1 -- 172.21.15.12:0/3200290584 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149000f820 con 0x55a95053d720 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3200290584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a95053d720 msgr2=0x55a950537350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/3200290584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a95053d720 0x55a950537350 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1490015000 tx=0x7f149001f000 comp rx=0 tx=0).stop 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3200290584 shutdown_connections 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/3200290584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a950537890 0x55a950537cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/3200290584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a95053d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/3200290584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a95053d720 0x55a950537350 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3200290584 >> 172.21.15.12:0/3200290584 conn(0x55a950495210 msgr2=0x55a950531df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:12.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3200290584 shutdown_connections 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3200290584 wait complete. 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 -- start start 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.932+0000 7f14adb1fb80 1 Processor -- start 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950540d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a950541270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 0x55a9505417b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950423560 con 0x55a95053d720 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950423200 con 0x55a950537890 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95030a6a0 con 0x55a95053ce00 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f14adb1fb80 10 client.0 _ll_register_callbacks cb 0x55a95042cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950540d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 0x55a9505417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950540d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44182/0 (socket says 172.21.15.12:44182) 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149affd700 1 -- 172.21.15.12:0/3380644101 learned_addr learned my addr 172.21.15.12:0/3380644101 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149a7fc700 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a950541270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149affd700 1 -- 172.21.15.12:0/3380644101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 msgr2=0x55a950541270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149affd700 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a950541270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149affd700 1 -- 172.21.15.12:0/3380644101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 msgr2=0x55a9505417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149affd700 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 0x55a9505417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149affd700 1 -- 172.21.15.12:0/3380644101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c10e650 con 0x55a950537890 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149a7fc700 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a950541270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f149affd700 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950540d30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f148c00c5b0 tx=0x7f148c099df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b6ab02f05005ef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.933+0000 7f147affd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f147affd700 1 -- 172.21.15.12:0/3380644101 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0cc040 con 0x55a950537890 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f147affd700 1 -- 172.21.15.12:0/3380644101 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f148c00cdd0 con 0x55a950537890 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3380644101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9503e0270 con 0x55a950537890 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f147affd700 1 -- 172.21.15.12:0/3380644101 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f148c09a3a0 con 0x55a950537890 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f14adb1fb80 10 client.29042 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3380644101 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55a9502b4b60 con 0x55a950537890 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f147affd700 1 -- 172.21.15.12:0/3380644101 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9503e0270 con 0x55a950537890 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f147affd700 1 -- 172.21.15.12:0/3380644101 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f148c0a2390 con 0x55a950537890 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f14adb1fb80 10 client.29042 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f14adb1fb80 10 client.29042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f14adb1fb80 20 client.29042 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a950551250 0x55a950553670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f14adb1fb80 4 client.29042 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3380644101 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a950506810 con 0x55a950551250 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.934+0000 7f149a7fc700 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a950551250 0x55a950553670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.935+0000 7f147affd700 10 client.29042 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.935+0000 7f149a7fc700 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a950551250 0x55a950553670 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b3a9e817e5e87 server_cookie=876a581b8fd5e0e5 in_seq=0 out_seq=0 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.936+0000 7f147affd700 1 -- 172.21.15.12:0/3380644101 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a950506810 con 0x55a950551250 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.936+0000 7f147affd700 10 client.29042 handle_command_reply: tid=0 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.936+0000 7f14adb1fb80 1 client.29042 shutdown 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3380644101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 msgr2=0x55a950540d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950540d30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f148c00c5b0 tx=0x7f148c099df0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3380644101 shutdown_connections 2024-05-09T00:00:12.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 0x55a9505417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a950551250 0x55a950553670 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950540d30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f147affd700 0 client.29042 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:12.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/3380644101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a950541270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3380644101 >> 172.21.15.12:0/3380644101 conn(0x55a950495210 msgr2=0x55a9505315a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:12.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3380644101 shutdown_connections 2024-05-09T00:00:12.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 1 -- 172.21.15.12:0/3380644101 wait complete. 2024-05-09T00:00:12.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.937+0000 7f14adb1fb80 20 client.29042 trim_cache size 0 max 16384 2024-05-09T00:00:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f14adb1fb80 1 Processor -- start 2024-05-09T00:00:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f14adb1fb80 1 -- start start 2024-05-09T00:00:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950537350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a95054d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 0x55a950531d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950533ea0 con 0x55a95053d720 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950532c00 con 0x55a950537890 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9505588d0 con 0x55a95053ce00 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f149b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 0x55a950531d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f149affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950537350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f149b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 0x55a950531d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60434/0 (socket says 172.21.15.12:60434) 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f149affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950537350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44188/0 (socket says 172.21.15.12:44188) 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.956+0000 7f149b7fe700 1 -- 172.21.15.12:0/1351221458 learned_addr learned my addr 172.21.15.12:0/1351221458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f149a7fc700 1 --2- 172.21.15.12:0/1351221458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a95054d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f149affd700 1 -- 172.21.15.12:0/1351221458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 msgr2=0x55a95054d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f149affd700 1 --2- 172.21.15.12:0/1351221458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a95054d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f149affd700 1 -- 172.21.15.12:0/1351221458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 msgr2=0x55a950531d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f149affd700 1 --2- 172.21.15.12:0/1351221458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 0x55a950531d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f149affd700 1 -- 172.21.15.12:0/1351221458 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a95044d560 con 0x55a950537890 2024-05-09T00:00:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f149a7fc700 1 --2- 172.21.15.12:0/1351221458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a95054d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:00:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f149affd700 1 --2- 172.21.15.12:0/1351221458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950537350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f148c005bf0 tx=0x7f148c09a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968d9c0efbcd2f4f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f14827fc700 1 -- 172.21.15.12:0/1351221458 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0cc070 con 0x55a950537890 2024-05-09T00:00:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f14827fc700 1 -- 172.21.15.12:0/1351221458 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f148c0b8250 con 0x55a950537890 2024-05-09T00:00:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.957+0000 7f14827fc700 1 -- 172.21.15.12:0/1351221458 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c099890 con 0x55a950537890 2024-05-09T00:00:12.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.958+0000 7f14adb1fb80 1 -- 172.21.15.12:0/1351221458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 msgr2=0x55a950537350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:12.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.958+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/1351221458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950537350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f148c005bf0 tx=0x7f148c09a2a0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.958+0000 7f14adb1fb80 1 -- 172.21.15.12:0/1351221458 shutdown_connections 2024-05-09T00:00:12.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.958+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/1351221458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053d720 0x55a950531d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.958+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/1351221458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a950537350 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.958+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/1351221458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053ce00 0x55a95054d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.958+0000 7f14adb1fb80 1 -- 172.21.15.12:0/1351221458 >> 172.21.15.12:0/1351221458 conn(0x55a950495210 msgr2=0x55a95053a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.958+0000 7f14adb1fb80 1 -- 172.21.15.12:0/1351221458 shutdown_connections 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.958+0000 7f14adb1fb80 1 -- 172.21.15.12:0/1351221458 wait complete. 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.958+0000 7f14adb1fb80 1 -- start start 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f14adb1fb80 1 Processor -- start 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a9505440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053ce00 0x55a950544620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f14adb1fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053d720 0x55a950544b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9505a6740 con 0x55a95053ce00 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9505a6880 con 0x55a950537890 2024-05-09T00:00:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f14adb1fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9505a69f0 con 0x55a95053d720 2024-05-09T00:00:12.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f14adb1fb80 10 client.0 _ll_register_callbacks cb 0x55a95042cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:12.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f149a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053ce00 0x55a950544620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f149b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053d720 0x55a950544b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f149affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a9505440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f149b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053d720 0x55a950544b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40520/0 (socket says 172.21.15.12:40520) 2024-05-09T00:00:12.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.959+0000 7f149b7fe700 1 -- 172.21.15.12:0/657266094 learned_addr learned my addr 172.21.15.12:0/657266094 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:12.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f149affd700 1 -- 172.21.15.12:0/657266094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053d720 msgr2=0x55a950544b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:12.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f149affd700 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053d720 0x55a950544b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f149affd700 1 -- 172.21.15.12:0/657266094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053ce00 msgr2=0x55a950544620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:12.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f149affd700 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053ce00 0x55a950544620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f149affd700 1 -- 172.21.15.12:0/657266094 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9502b4b60 con 0x55a950537890 2024-05-09T00:00:12.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f149b7fe700 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053d720 0x55a950544b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:12.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f149a7fc700 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053ce00 0x55a950544620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:12.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f149affd700 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a9505440e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f148c00eb00 tx=0x7f148c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4efb98f994058b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:12.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f147affd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:00:12.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f147affd700 1 -- 172.21.15.12:0/657266094 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0cc070 con 0x55a950537890 2024-05-09T00:00:12.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f147affd700 1 -- 172.21.15.12:0/657266094 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f148c00cdd0 con 0x55a950537890 2024-05-09T00:00:12.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f14adb1fb80 1 -- 172.21.15.12:0/657266094 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f149001d040 con 0x55a950537890 2024-05-09T00:00:12.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f147affd700 1 -- 172.21.15.12:0/657266094 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f148c00cbd0 con 0x55a950537890 2024-05-09T00:00:12.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f14adb1fb80 10 client.29048 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:12.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.960+0000 7f14adb1fb80 1 -- 172.21.15.12:0/657266094 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55a95030ccd0 con 0x55a950537890 2024-05-09T00:00:12.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.961+0000 7f147affd700 1 -- 172.21.15.12:0/657266094 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f149001d040 con 0x55a950537890 2024-05-09T00:00:12.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.961+0000 7f147affd700 1 -- 172.21.15.12:0/657266094 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f148c0b85f0 con 0x55a950537890 2024-05-09T00:00:12.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.961+0000 7f14adb1fb80 10 client.29048 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:12.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.961+0000 7f14adb1fb80 10 client.29048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:12.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.961+0000 7f14adb1fb80 20 client.29048 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:12.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.961+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a9505a7cd0 0x55a9505aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:12.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.961+0000 7f14adb1fb80 4 client.29048 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:00:12.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.961+0000 7f14adb1fb80 1 -- 172.21.15.12:0/657266094 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a95047fec0 con 0x55a9505a7cd0 2024-05-09T00:00:12.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.961+0000 7f149a7fc700 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a9505a7cd0 0x55a9505aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:12.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.962+0000 7f147affd700 10 client.29048 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:12.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.962+0000 7f149a7fc700 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a9505a7cd0 0x55a9505aa0c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70bd76fb8aa8d316 server_cookie=2b1300672cb38241 in_seq=0 out_seq=0 2024-05-09T00:00:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.964+0000 7f147affd700 1 -- 172.21.15.12:0/657266094 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a95047fec0 con 0x55a9505a7cd0 2024-05-09T00:00:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.964+0000 7f147affd700 10 client.29048 handle_command_reply: tid=0 2024-05-09T00:00:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.964+0000 7f14adb1fb80 1 client.29048 shutdown 2024-05-09T00:00:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 1 -- 172.21.15.12:0/657266094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 msgr2=0x55a9505440e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a9505440e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f148c00eb00 tx=0x7f148c003820 comp rx=0 tx=0).stop 2024-05-09T00:00:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 1 -- 172.21.15.12:0/657266094 shutdown_connections 2024-05-09T00:00:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a95053d720 0x55a950544b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a9505a7cd0 0x55a9505aa0c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a950537890 0x55a9505440e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 1 --2- 172.21.15.12:0/657266094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a95053ce00 0x55a950544620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f147affd700 0 client.29048 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:12.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 1 -- 172.21.15.12:0/657266094 >> 172.21.15.12:0/657266094 conn(0x55a950495210 msgr2=0x55a95053bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:12.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 1 -- 172.21.15.12:0/657266094 shutdown_connections 2024-05-09T00:00:12.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 1 -- 172.21.15.12:0/657266094 wait complete. 2024-05-09T00:00:12.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14adb1fb80 20 client.29048 trim_cache size 0 max 16384 2024-05-09T00:00:12.967 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:00:12.967 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:00:12.967 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:00:12.968 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:00:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.965+0000 7f14a1965700 1 -- 172.21.15.12:0/1341411595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14840a1290 msgr2=0x7f14840a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.966+0000 7f14a1965700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14840a1290 0x7f14840a3750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1490009fd0 tx=0x7f1490019040 comp rx=0 tx=0).stop 2024-05-09T00:00:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.966+0000 7f14a1965700 1 -- 172.21.15.12:0/1341411595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149c0682e0 msgr2=0x7f149c0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.966+0000 7f14a1965700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149c0682e0 0x7f149c0ffe60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f148c00eb30 tx=0x7f148c00ee40 comp rx=0 tx=0).stop 2024-05-09T00:00:12.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.966+0000 7f14a1965700 1 -- 172.21.15.12:0/1341411595 shutdown_connections 2024-05-09T00:00:12.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.966+0000 7f14a1965700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14840a1290 0x7f14840a3750 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1490009fd0 tx=0x7f1490019040 comp rx=0 tx=0).stop 2024-05-09T00:00:12.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.966+0000 7f14a1965700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f149c1008e0 0x7f149c1a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.966+0000 7f14a1965700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149c0682e0 0x7f149c0ffe60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.966+0000 7f14a1965700 1 --2- 172.21.15.12:0/1341411595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f149c106290 0x7f149c1003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:12.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.966+0000 7f14a1965700 1 -- 172.21.15.12:0/1341411595 >> 172.21.15.12:0/1341411595 conn(0x7f149c076f40 msgr2=0x7f149c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:12.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.967+0000 7f14a1965700 1 -- 172.21.15.12:0/1341411595 shutdown_connections 2024-05-09T00:00:12.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:12.967+0000 7f14a1965700 1 -- 172.21.15.12:0/1341411595 wait complete. 2024-05-09T00:00:12.979 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:00:12.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:00:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.131+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/663617193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8ac101c60 msgr2=0x7fa8ac102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.131+0000 7fa8b0b4d700 1 --2- 172.21.15.12:0/663617193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8ac101c60 0x7fa8ac102040 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009990 tx=0x7fa89c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.131+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/663617193 shutdown_connections 2024-05-09T00:00:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.131+0000 7fa8b0b4d700 1 --2- 172.21.15.12:0/663617193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ac1000f0 0x7fa8ac10fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.131+0000 7fa8b0b4d700 1 --2- 172.21.15.12:0/663617193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8ac101c60 0x7fa8ac102040 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.131+0000 7fa8b0b4d700 1 --2- 172.21.15.12:0/663617193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ac0ff730 0x7fa8ac0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.131+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/663617193 >> 172.21.15.12:0/663617193 conn(0x7fa8ac0750b0 msgr2=0x7fa8ac0754c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.131+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/663617193 shutdown_connections 2024-05-09T00:00:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.131+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/663617193 wait complete. 2024-05-09T00:00:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.132+0000 7fa8b0b4d700 1 Processor -- start 2024-05-09T00:00:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.132+0000 7fa8b0b4d700 1 -- start start 2024-05-09T00:00:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.132+0000 7fa8b0b4d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ac0ff730 0x7fa8ac199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.132+0000 7fa8b0b4d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ac1000f0 0x7fa8ac199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.132+0000 7fa8b0b4d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8ac101c60 0x7fa8ac19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.132+0000 7fa8b0b4d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac19e450 con 0x7fa8ac101c60 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.132+0000 7fa8b0b4d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac19e5c0 con 0x7fa8ac0ff730 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.132+0000 7fa8b0b4d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac19e780 con 0x7fa8ac1000f0 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.132+0000 7fa8aad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8ac101c60 0x7fa8ac19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.132+0000 7fa8a9d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ac1000f0 0x7fa8ac199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8a9d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ac1000f0 0x7fa8ac199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40530/0 (socket says 172.21.15.12:40530) 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8aad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8ac101c60 0x7fa8ac19de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60458/0 (socket says 172.21.15.12:60458) 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8a9d9b700 1 -- 172.21.15.12:0/3429206519 learned_addr learned my addr 172.21.15.12:0/3429206519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8aa59c700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ac0ff730 0x7fa8ac199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8a9d9b700 1 -- 172.21.15.12:0/3429206519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ac0ff730 msgr2=0x7fa8ac199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8a9d9b700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ac0ff730 0x7fa8ac199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8a9d9b700 1 -- 172.21.15.12:0/3429206519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8ac101c60 msgr2=0x7fa8ac19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8a9d9b700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8ac101c60 0x7fa8ac19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8a9d9b700 1 -- 172.21.15.12:0/3429206519 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a0009520 con 0x7fa8ac1000f0 2024-05-09T00:00:13.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8a9d9b700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ac1000f0 0x7fa8ac199a70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000eb30 tx=0x7fa8a000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa89b7fe700 1 -- 172.21.15.12:0/3429206519 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0009880 con 0x7fa8ac1000f0 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa89b7fe700 1 -- 172.21.15.12:0/3429206519 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8a00044e0 con 0x7fa8ac1000f0 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa89b7fe700 1 -- 172.21.15.12:0/3429206519 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0003d80 con 0x7fa8ac1000f0 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/3429206519 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89c0095f0 con 0x7fa8ac1000f0 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.133+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/3429206519 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac1a60c0 con 0x7fa8ac1000f0 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.135+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/3429206519 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ac10b620 con 0x7fa8ac1000f0 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.136+0000 7fa89b7fe700 1 -- 172.21.15.12:0/3429206519 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa8a0003f20 con 0x7fa8ac1000f0 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.136+0000 7fa89b7fe700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8940a1290 0x7fa8940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.137+0000 7fa89b7fe700 1 -- 172.21.15.12:0/3429206519 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa8a0014070 con 0x7fa8ac1000f0 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.137+0000 7fa8aa59c700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8940a1290 0x7fa8940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.137+0000 7fa8aa59c700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8940a1290 0x7fa8940a3750 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa89c006010 tx=0x7fa89c00b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.140+0000 7fa89b7fe700 1 -- 172.21.15.12:0/3429206519 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa8a0096a90 con 0x7fa8ac1000f0 2024-05-09T00:00:13.215 INFO:tasks.workunit.client.0.smithi012.stdout: 10 3750 45.93 MB/sec warmup 8 sec latency 7607.944 ms 2024-05-09T00:00:13.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.464+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/3429206519 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8ac068ff0 con 0x7fa8ac1000f0 2024-05-09T00:00:13.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.465+0000 7fa89b7fe700 1 -- 172.21.15.12:0/3429206519 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+10880 (secure 0 0 0) 0x7fa8a009a0b0 con 0x7fa8ac1000f0 2024-05-09T00:00:13.461 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:00:13.461 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:00:08.606803+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_28844":{"gid":28844,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/864772354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":864772354},{"type":"v1","addr":"172.21.15.204:6837","nonce":864772354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:00:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.467+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/3429206519 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8940a1290 msgr2=0x7fa8940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.467+0000 7fa8b0b4d700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8940a1290 0x7fa8940a3750 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa89c006010 tx=0x7fa89c00b260 comp rx=0 tx=0).stop 2024-05-09T00:00:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.467+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/3429206519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ac1000f0 msgr2=0x7fa8ac199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.467+0000 7fa8b0b4d700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ac1000f0 0x7fa8ac199a70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000eb30 tx=0x7fa8a000ee40 comp rx=0 tx=0).stop 2024-05-09T00:00:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.468+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/3429206519 shutdown_connections 2024-05-09T00:00:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.468+0000 7fa8b0b4d700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa8940a1290 0x7fa8940a3750 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.468+0000 7fa8b0b4d700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8ac101c60 0x7fa8ac19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.468+0000 7fa8b0b4d700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ac0ff730 0x7fa8ac199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.468+0000 7fa8b0b4d700 1 --2- 172.21.15.12:0/3429206519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8ac1000f0 0x7fa8ac199a70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.468+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/3429206519 >> 172.21.15.12:0/3429206519 conn(0x7fa8ac0750b0 msgr2=0x7fa8ac0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.468+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/3429206519 shutdown_connections 2024-05-09T00:00:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.468+0000 7fa8b0b4d700 1 -- 172.21.15.12:0/3429206519 wait complete. 2024-05-09T00:00:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 60 2024-05-09T00:00:13.478 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:00:13.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.650+0000 7fcfae1a4700 1 -- 172.21.15.12:0/1236274655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa80ff7b0 msgr2=0x7fcfa80ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.650+0000 7fcfae1a4700 1 --2- 172.21.15.12:0/1236274655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa80ff7b0 0x7fcfa80ffc30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009990 tx=0x7fcf9c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.651+0000 7fcfae1a4700 1 -- 172.21.15.12:0/1236274655 shutdown_connections 2024-05-09T00:00:13.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.651+0000 7fcfae1a4700 1 --2- 172.21.15.12:0/1236274655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfa8100170 0x7fcfa810c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.651+0000 7fcfae1a4700 1 --2- 172.21.15.12:0/1236274655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa8105600 0x7fcfa81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.651+0000 7fcfae1a4700 1 --2- 172.21.15.12:0/1236274655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa80ff7b0 0x7fcfa80ffc30 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.651+0000 7fcfae1a4700 1 -- 172.21.15.12:0/1236274655 >> 172.21.15.12:0/1236274655 conn(0x7fcfa8074fe0 msgr2=0x7fcfa80753f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.651+0000 7fcfae1a4700 1 -- 172.21.15.12:0/1236274655 shutdown_connections 2024-05-09T00:00:13.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.651+0000 7fcfae1a4700 1 -- 172.21.15.12:0/1236274655 wait complete. 2024-05-09T00:00:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.651+0000 7fcfae1a4700 1 Processor -- start 2024-05-09T00:00:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfae1a4700 1 -- start start 2024-05-09T00:00:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfae1a4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfa80ff7b0 0x7fcfa81972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfae1a4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa8100170 0x7fcfa8197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfae1a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa8105600 0x7fcfa819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfae1a4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa819c1c0 con 0x7fcfa8105600 2024-05-09T00:00:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfae1a4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa819c330 con 0x7fcfa80ff7b0 2024-05-09T00:00:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfae1a4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa819c4f0 con 0x7fcfa8100170 2024-05-09T00:00:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa8105600 0x7fcfa819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfa80ff7b0 0x7fcfa81972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa8105600 0x7fcfa819bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60476/0 (socket says 172.21.15.12:60476) 2024-05-09T00:00:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa7fff700 1 -- 172.21.15.12:0/3653234767 learned_addr learned my addr 172.21.15.12:0/3653234767 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa8100170 0x7fcfa8197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa77fe700 1 -- 172.21.15.12:0/3653234767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa8100170 msgr2=0x7fcfa8197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa8100170 0x7fcfa8197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa77fe700 1 -- 172.21.15.12:0/3653234767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa8105600 msgr2=0x7fcfa819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa8105600 0x7fcfa819bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa77fe700 1 -- 172.21.15.12:0/3653234767 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c0095f0 con 0x7fcfa80ff7b0 2024-05-09T00:00:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.652+0000 7fcfa7fff700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa8105600 0x7fcfa819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.653+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfa80ff7b0 0x7fcfa81972f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800e9f0 tx=0x7fcf9800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.653+0000 7fcfa4ff9700 1 -- 172.21.15.12:0/3653234767 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9800c9c0 con 0x7fcfa80ff7b0 2024-05-09T00:00:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.653+0000 7fcfa4ff9700 1 -- 172.21.15.12:0/3653234767 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf98010430 con 0x7fcfa80ff7b0 2024-05-09T00:00:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.653+0000 7fcfae1a4700 1 -- 172.21.15.12:0/3653234767 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa819c7d0 con 0x7fcfa80ff7b0 2024-05-09T00:00:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.653+0000 7fcfa4ff9700 1 -- 172.21.15.12:0/3653234767 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98018680 con 0x7fcfa80ff7b0 2024-05-09T00:00:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.653+0000 7fcfae1a4700 1 -- 172.21.15.12:0/3653234767 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa81a4010 con 0x7fcfa80ff7b0 2024-05-09T00:00:13.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.656+0000 7fcfa4ff9700 1 -- 172.21.15.12:0/3653234767 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcf98010860 con 0x7fcfa80ff7b0 2024-05-09T00:00:13.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.656+0000 7fcfa4ff9700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf900a1280 0x7fcf900a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.656+0000 7fcfa4ff9700 1 -- 172.21.15.12:0/3653234767 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcf98014070 con 0x7fcfa80ff7b0 2024-05-09T00:00:13.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.656+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf900a1280 0x7fcf900a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.657+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf900a1280 0x7fcf900a3740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcfa81987f0 tx=0x7fcf9c005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.657+0000 7fcfba35eb80 1 Processor -- start 2024-05-09T00:00:13.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.657+0000 7fcfba35eb80 1 -- start start 2024-05-09T00:00:13.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.657+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.657+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70ca5720 0x55dc70c9f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.657+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70c9f890 0x55dc70c9fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.657+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70bf6150 con 0x55dc70c9f890 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.657+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70bf3b10 con 0x55dc70ca5720 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.657+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70bef400 con 0x55dc70ca4e00 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70c9f890 0x55dc70c9fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70ca5720 0x55dc70c9f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40548/0 (socket says 172.21.15.12:40548) 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa77fe700 1 -- 172.21.15.12:0/1175873676 learned_addr learned my addr 172.21.15.12:0/1175873676 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa77fe700 1 -- 172.21.15.12:0/1175873676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70ca5720 msgr2=0x55dc70c9f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/1175873676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70ca5720 0x55dc70c9f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa77fe700 1 -- 172.21.15.12:0/1175873676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70c9f890 msgr2=0x55dc70c9fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/1175873676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70c9f890 0x55dc70c9fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa77fe700 1 -- 172.21.15.12:0/1175873676 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc70a703b0 con 0x55dc70ca4e00 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/1175873676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70ca5720 0x55dc70c9f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/1175873676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca51e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcf980a1e60 tx=0x7fcf98099940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adbacdaecf42c0fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcf8dffb700 1 -- 172.21.15.12:0/1175873676 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9809dd40 con 0x55dc70ca4e00 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcf8dffb700 1 -- 172.21.15.12:0/1175873676 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf980a2390 con 0x55dc70ca4e00 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.658+0000 7fcf8dffb700 1 -- 172.21.15.12:0/1175873676 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98099640 con 0x55dc70ca4e00 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.659+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1175873676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 msgr2=0x55dc70ca51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.659+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/1175873676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca51e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcf980a1e60 tx=0x7fcf98099940 comp rx=0 tx=0).stop 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.659+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1175873676 shutdown_connections 2024-05-09T00:00:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.659+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/1175873676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70c9f890 0x55dc70c9fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.659+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/1175873676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca51e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.659+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/1175873676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70ca5720 0x55dc70c9f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.659+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1175873676 >> 172.21.15.12:0/1175873676 conn(0x55dc70bfd210 msgr2=0x55dc70c99df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.659+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1175873676 shutdown_connections 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.659+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1175873676 wait complete. 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfba35eb80 1 -- start start 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfba35eb80 1 Processor -- start 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70ca8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca5720 0x55dc70ca9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70b8b560 con 0x55dc70ca5720 2024-05-09T00:00:13.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70b8b200 con 0x55dc70c9f890 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70a726a0 con 0x55dc70ca4e00 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfba35eb80 10 client.0 _ll_register_callbacks cb 0x55dc70b94b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfa6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfa77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70ca8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfa7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca5720 0x55dc70ca9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfa6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40558/0 (socket says 172.21.15.12:40558) 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.660+0000 7fcfa6ffd700 1 -- 172.21.15.12:0/3957120183 learned_addr learned my addr 172.21.15.12:0/3957120183 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcfa6ffd700 1 -- 172.21.15.12:0/3957120183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 msgr2=0x55dc70ca8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70ca8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcfa6ffd700 1 -- 172.21.15.12:0/3957120183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca5720 msgr2=0x55dc70ca9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca5720 0x55dc70ca9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcfa6ffd700 1 -- 172.21.15.12:0/3957120183 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa8068b60 con 0x55dc70ca4e00 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca8fc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009990 tx=0x7fcf9c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa89f41263c8c0f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcf86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcf86ffd700 1 -- 172.21.15.12:0/3957120183 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c004bc0 con 0x55dc70ca4e00 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcf86ffd700 1 -- 172.21.15.12:0/3957120183 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcf9c004d20 con 0x55dc70ca4e00 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3957120183 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc70b48270 con 0x55dc70ca4e00 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcf86ffd700 1 -- 172.21.15.12:0/3957120183 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf9c004ee0 con 0x55dc70ca4e00 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcfba35eb80 10 client.29074 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.661+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3957120183 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7fcfa81a4010 con 0x55dc70ca4e00 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcf86ffd700 1 -- 172.21.15.12:0/3957120183 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc70b48270 con 0x55dc70ca4e00 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcf86ffd700 1 -- 172.21.15.12:0/3957120183 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fcf9c02f070 con 0x55dc70ca4e00 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcfba35eb80 10 client.29074 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcfba35eb80 10 client.29074 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:00:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcfba35eb80 20 client.29074 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:13.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc70cb91d0 0x55dc70cbb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcfba35eb80 4 client.29074 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:13.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3957120183 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc70c6e810 con 0x55dc70cb91d0 2024-05-09T00:00:13.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc70cb91d0 0x55dc70cbb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc70cb91d0 0x55dc70cbb5f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19574c89ff671989 server_cookie=4bf963ba1420564b in_seq=0 out_seq=0 2024-05-09T00:00:13.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.662+0000 7fcf86ffd700 10 client.29074 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:00:13.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcf86ffd700 1 -- 172.21.15.12:0/3957120183 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc70c6e810 con 0x55dc70cb91d0 2024-05-09T00:00:13.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcf86ffd700 10 client.29074 handle_command_reply: tid=0 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 client.29074 shutdown 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3957120183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 msgr2=0x55dc70ca8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca8fc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009990 tx=0x7fcf9c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3957120183 shutdown_connections 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca5720 0x55dc70ca9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70ca8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc70cb91d0 0x55dc70cbb5f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/3957120183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca4e00 0x55dc70ca8fc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcf86ffd700 0 client.29074 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3957120183 >> 172.21.15.12:0/3957120183 conn(0x55dc70bfd210 msgr2=0x55dc70c995a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3957120183 shutdown_connections 2024-05-09T00:00:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.663+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3957120183 wait complete. 2024-05-09T00:00:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.664+0000 7fcfba35eb80 20 client.29074 trim_cache size 0 max 16384 2024-05-09T00:00:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.679+0000 7fcfba35eb80 1 Processor -- start 2024-05-09T00:00:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.679+0000 7fcfba35eb80 1 -- start start 2024-05-09T00:00:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.679+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70c9f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.679+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 0x55dc70cbb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.679+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca5720 0x55dc70c99d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.679+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70c96e30 con 0x55dc70ca4e00 2024-05-09T00:00:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.679+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70ca5590 con 0x55dc70c9f890 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.679+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70cc0830 con 0x55dc70ca5720 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 0x55dc70cbb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70c9f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70c9f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44284/0 (socket says 172.21.15.12:44284) 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 0x55dc70cbb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60494/0 (socket says 172.21.15.12:60494) 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa77fe700 1 -- 172.21.15.12:0/3807316665 learned_addr learned my addr 172.21.15.12:0/3807316665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa7fff700 1 --2- 172.21.15.12:0/3807316665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca5720 0x55dc70c99d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa77fe700 1 -- 172.21.15.12:0/3807316665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca5720 msgr2=0x55dc70c99d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/3807316665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca5720 0x55dc70c99d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa77fe700 1 -- 172.21.15.12:0/3807316665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 msgr2=0x55dc70cbb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/3807316665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 0x55dc70cbb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa77fe700 1 -- 172.21.15.12:0/3807316665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc70a1cb60 con 0x55dc70c9f890 2024-05-09T00:00:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/3807316665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 0x55dc70cbb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/3807316665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70c9f350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800e9c0 tx=0x7fcf98009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a2d315931516ce0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcf8e7fc700 1 -- 172.21.15.12:0/3807316665 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf980b82c0 con 0x55dc70c9f890 2024-05-09T00:00:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcf8e7fc700 1 -- 172.21.15.12:0/3807316665 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf980105a0 con 0x55dc70c9f890 2024-05-09T00:00:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.680+0000 7fcf8e7fc700 1 -- 172.21.15.12:0/3807316665 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9809a3b0 con 0x55dc70c9f890 2024-05-09T00:00:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.681+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3807316665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 msgr2=0x55dc70c9f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.681+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/3807316665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70c9f350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800e9c0 tx=0x7fcf98009790 comp rx=0 tx=0).stop 2024-05-09T00:00:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.681+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3807316665 shutdown_connections 2024-05-09T00:00:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.681+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/3807316665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca5720 0x55dc70c99d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.681+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/3807316665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70c9f350 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.681+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/3807316665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 0x55dc70cbb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.681+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3807316665 >> 172.21.15.12:0/3807316665 conn(0x55dc70bfd210 msgr2=0x55dc70c9fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.681+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3807316665 shutdown_connections 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.681+0000 7fcfba35eb80 1 -- 172.21.15.12:0/3807316665 wait complete. 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.682+0000 7fcfba35eb80 1 -- start start 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.682+0000 7fcfba35eb80 1 Processor -- start 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70cb00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 0x55dc70cb05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfba35eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca5720 0x55dc70cae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70cae860 con 0x55dc70ca4e00 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70cae9d0 con 0x55dc70c9f890 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfba35eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc70caeb90 con 0x55dc70ca5720 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfba35eb80 10 client.0 _ll_register_callbacks cb 0x55dc70b94b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfa77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70cb00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfa77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70cb00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44288/0 (socket says 172.21.15.12:44288) 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfa77fe700 1 -- 172.21.15.12:0/1813837151 learned_addr learned my addr 172.21.15.12:0/1813837151 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfa7fff700 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca5720 0x55dc70cae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 0x55dc70cb05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfa77fe700 1 -- 172.21.15.12:0/1813837151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca5720 msgr2=0x55dc70cae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca5720 0x55dc70cae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfa77fe700 1 -- 172.21.15.12:0/1813837151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 msgr2=0x55dc70cb05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 0x55dc70cb05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.683+0000 7fcfa77fe700 1 -- 172.21.15.12:0/1813837151 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa81a4010 con 0x55dc70c9f890 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.684+0000 7fcfa77fe700 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70cb00b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcf98099c10 tx=0x7fcf98010710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=352c7bf6a24bb298 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.684+0000 7fcf86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.684+0000 7fcf86ffd700 1 -- 172.21.15.12:0/1813837151 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98099870 con 0x55dc70c9f890 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.684+0000 7fcf86ffd700 1 -- 172.21.15.12:0/1813837151 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcf980b82c0 con 0x55dc70c9f890 2024-05-09T00:00:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.684+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1813837151 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf980a2390 con 0x55dc70c9f890 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.684+0000 7fcf86ffd700 1 -- 172.21.15.12:0/1813837151 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf98099660 con 0x55dc70c9f890 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.684+0000 7fcfba35eb80 10 client.29072 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.684+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1813837151 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55dc70a703b0 con 0x55dc70c9f890 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.684+0000 7fcf86ffd700 1 -- 172.21.15.12:0/1813837151 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf980a2390 con 0x55dc70c9f890 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.684+0000 7fcf86ffd700 1 -- 172.21.15.12:0/1813837151 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fcf9800cb20 con 0x55dc70c9f890 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcfba35eb80 10 client.29072 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcfba35eb80 10 client.29072 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcfba35eb80 20 client.29072 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc70d0f510 0x55dc70d11900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcfba35eb80 4 client.29072 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1813837151 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc70be7ec0 con 0x55dc70d0f510 2024-05-09T00:00:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc70d0f510 0x55dc70d11900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcfa6ffd700 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc70d0f510 0x55dc70d11900 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea1f3439a4c24768 server_cookie=fa33ea4663e79ea9 in_seq=0 out_seq=0 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcf86ffd700 10 client.29072 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcf86ffd700 1 -- 172.21.15.12:0/1813837151 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc70be7ec0 con 0x55dc70d0f510 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcf86ffd700 10 client.29072 handle_command_reply: tid=0 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.685+0000 7fcfba35eb80 1 client.29072 shutdown 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1813837151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 msgr2=0x55dc70cb00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70cb00b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcf98099c10 tx=0x7fcf98010710 comp rx=0 tx=0).stop 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1813837151 shutdown_connections 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55dc70d0f510 0x55dc70d11900 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc70ca5720 0x55dc70cae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc70c9f890 0x55dc70cb00b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcf86ffd700 0 client.29072 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:00:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 1 --2- 172.21.15.12:0/1813837151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dc70ca4e00 0x55dc70cb05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1813837151 >> 172.21.15.12:0/1813837151 conn(0x55dc70bfd210 msgr2=0x55dc70ca9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1813837151 shutdown_connections 2024-05-09T00:00:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 1 -- 172.21.15.12:0/1813837151 wait complete. 2024-05-09T00:00:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfba35eb80 20 client.29072 trim_cache size 0 max 16384 2024-05-09T00:00:13.687 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:00:13.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfae1a4700 1 -- 172.21.15.12:0/3653234767 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf900a1280 msgr2=0x7fcf900a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfae1a4700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf900a1280 0x7fcf900a3740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcfa81987f0 tx=0x7fcf9c005fd0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.686+0000 7fcfae1a4700 1 -- 172.21.15.12:0/3653234767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfa80ff7b0 msgr2=0x7fcfa81972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.687+0000 7fcfae1a4700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfa80ff7b0 0x7fcfa81972f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800e9f0 tx=0x7fcf9800ed00 comp rx=0 tx=0).stop 2024-05-09T00:00:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.687+0000 7fcfae1a4700 1 -- 172.21.15.12:0/3653234767 shutdown_connections 2024-05-09T00:00:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.687+0000 7fcfae1a4700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcf900a1280 0x7fcf900a3740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.687+0000 7fcfae1a4700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa8105600 0x7fcfa819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.687+0000 7fcfae1a4700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfa80ff7b0 0x7fcfa81972f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.687+0000 7fcfae1a4700 1 --2- 172.21.15.12:0/3653234767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa8100170 0x7fcfa8197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.687+0000 7fcfae1a4700 1 -- 172.21.15.12:0/3653234767 >> 172.21.15.12:0/3653234767 conn(0x7fcfa8074fe0 msgr2=0x7fcfa810ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.687+0000 7fcfae1a4700 1 -- 172.21.15.12:0/3653234767 shutdown_connections 2024-05-09T00:00:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.687+0000 7fcfae1a4700 1 -- 172.21.15.12:0/3653234767 wait complete. 2024-05-09T00:00:13.696 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:00:13.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:13 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3429206519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:00:13.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:13 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3429206519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:00:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.864+0000 7fd7edd94700 1 -- 172.21.15.12:0/3051305055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7e8101160 msgr2=0x7fd7e8101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.864+0000 7fd7edd94700 1 --2- 172.21.15.12:0/3051305055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7e8101160 0x7fd7e8101540 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009990 tx=0x7fd7d8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.864+0000 7fd7edd94700 1 -- 172.21.15.12:0/3051305055 shutdown_connections 2024-05-09T00:00:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.864+0000 7fd7edd94700 1 --2- 172.21.15.12:0/3051305055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7e8106280 0x7fd7e810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.864+0000 7fd7edd94700 1 --2- 172.21.15.12:0/3051305055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7e8101160 0x7fd7e8101540 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.864+0000 7fd7edd94700 1 --2- 172.21.15.12:0/3051305055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e8101b10 0x7fd7e8105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.864+0000 7fd7edd94700 1 -- 172.21.15.12:0/3051305055 >> 172.21.15.12:0/3051305055 conn(0x7fd7e80fc9c0 msgr2=0x7fd7e80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.864+0000 7fd7edd94700 1 -- 172.21.15.12:0/3051305055 shutdown_connections 2024-05-09T00:00:13.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.865+0000 7fd7edd94700 1 -- 172.21.15.12:0/3051305055 wait complete. 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.865+0000 7fd7edd94700 1 Processor -- start 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7edd94700 1 -- start start 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7edd94700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7e8101160 0x7fd7e80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7edd94700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e8106280 0x7fd7e80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7edd94700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7e8073d20 0x7fd7e81a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7edd94700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81a5bd0 con 0x7fd7e8101160 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7edd94700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81a5d40 con 0x7fd7e8073d20 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7edd94700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81a5f00 con 0x7fd7e8106280 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7e8101160 0x7fd7e80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e8106280 0x7fd7e80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7e8101160 0x7fd7e80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60518/0 (socket says 172.21.15.12:60518) 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e77fe700 1 -- 172.21.15.12:0/4148493067 learned_addr learned my addr 172.21.15.12:0/4148493067 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e7fff700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7e8073d20 0x7fd7e81a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e77fe700 1 -- 172.21.15.12:0/4148493067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e8106280 msgr2=0x7fd7e80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e8106280 0x7fd7e80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e77fe700 1 -- 172.21.15.12:0/4148493067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7e8073d20 msgr2=0x7fd7e81a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7e8073d20 0x7fd7e81a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e77fe700 1 -- 172.21.15.12:0/4148493067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80095f0 con 0x7fd7e8101160 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.866+0000 7fd7e6ffd700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e8106280 0x7fd7e80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.867+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7e8101160 0x7fd7e80732a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8004760 tx=0x7fd7d8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.867+0000 7fd7e4ff9700 1 -- 172.21.15.12:0/4148493067 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d801d070 con 0x7fd7e8101160 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.867+0000 7fd7e4ff9700 1 -- 172.21.15.12:0/4148493067 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d8022470 con 0x7fd7e8101160 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.867+0000 7fd7edd94700 1 -- 172.21.15.12:0/4148493067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e81a6180 con 0x7fd7e8101160 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.867+0000 7fd7e4ff9700 1 -- 172.21.15.12:0/4148493067 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8017650 con 0x7fd7e8101160 2024-05-09T00:00:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.867+0000 7fd7edd94700 1 -- 172.21.15.12:0/4148493067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e81a65a0 con 0x7fd7e8101160 2024-05-09T00:00:13.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.869+0000 7fd7e4ff9700 1 -- 172.21.15.12:0/4148493067 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd7d80225e0 con 0x7fd7e8101160 2024-05-09T00:00:13.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.870+0000 7fd7e4ff9700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd7d00a1290 0x7fd7d00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.870+0000 7fd7e6ffd700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd7d00a1290 0x7fd7d00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.870+0000 7fd7e4ff9700 1 -- 172.21.15.12:0/4148493067 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd7d80cde20 con 0x7fd7e8101160 2024-05-09T00:00:13.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.871+0000 7fd7e6ffd700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd7d00a1290 0x7fd7d00a3750 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80749d0 tx=0x7fd7dc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.872+0000 7fd7f9f4eb80 1 Processor -- start 2024-05-09T00:00:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.872+0000 7fd7f9f4eb80 1 -- start start 2024-05-09T00:00:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e4e00 0x5647b77e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e5720 0x5647b77df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b7736150 con 0x5647b77df890 2024-05-09T00:00:13.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b7733b10 con 0x5647b77e4e00 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b772f400 con 0x5647b77e5720 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e4e00 0x5647b77e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e5720 0x5647b77df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e5720 0x5647b77df350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40600/0 (socket says 172.21.15.12:40600) 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77dfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60532/0 (socket says 172.21.15.12:60532) 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e4e00 0x5647b77e51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44320/0 (socket says 172.21.15.12:44320) 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e6ffd700 1 -- 172.21.15.12:0/539103880 learned_addr learned my addr 172.21.15.12:0/539103880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e77fe700 1 -- 172.21.15.12:0/539103880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e5720 msgr2=0x5647b77df350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/539103880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e5720 0x5647b77df350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e77fe700 1 -- 172.21.15.12:0/539103880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 msgr2=0x5647b77dfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/539103880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77dfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e77fe700 1 -- 172.21.15.12:0/539103880 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647b75b03b0 con 0x5647b77e4e00 2024-05-09T00:00:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e7fff700 1 --2- 172.21.15.12:0/539103880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77dfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e6ffd700 1 --2- 172.21.15.12:0/539103880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e5720 0x5647b77df350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.873+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/539103880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e4e00 0x5647b77e51e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80cb580 tx=0x7fd7d809ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=351eebf7adb0d77a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.874+0000 7fd7cdffb700 1 -- 172.21.15.12:0/539103880 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d809afd0 con 0x5647b77e4e00 2024-05-09T00:00:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.874+0000 7fd7cdffb700 1 -- 172.21.15.12:0/539103880 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d809ee00 con 0x5647b77e4e00 2024-05-09T00:00:13.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.874+0000 7fd7cdffb700 1 -- 172.21.15.12:0/539103880 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8095e40 con 0x5647b77e4e00 2024-05-09T00:00:13.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.875+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/539103880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e4e00 msgr2=0x5647b77e51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.875+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/539103880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e4e00 0x5647b77e51e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80cb580 tx=0x7fd7d809ae20 comp rx=0 tx=0).stop 2024-05-09T00:00:13.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.875+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/539103880 shutdown_connections 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.875+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/539103880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77dfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.875+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/539103880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e4e00 0x5647b77e51e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.875+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/539103880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e5720 0x5647b77df350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.875+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/539103880 >> 172.21.15.12:0/539103880 conn(0x5647b773d210 msgr2=0x5647b77d9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.875+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/539103880 shutdown_connections 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.875+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/539103880 wait complete. 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.875+0000 7fd7f9f4eb80 1 -- start start 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7f9f4eb80 1 Processor -- start 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 0x5647b77e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b76cb560 con 0x5647b77df890 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b76cb200 con 0x5647b77e5720 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b75b26a0 con 0x5647b77e4e00 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7f9f4eb80 10 client.0 _ll_register_callbacks cb 0x5647b76d4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7e77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7e7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7e7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77e95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44328/0 (socket says 172.21.15.12:44328) 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7e77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77e8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60548/0 (socket says 172.21.15.12:60548) 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.876+0000 7fd7e7fff700 1 -- 172.21.15.12:0/260110408 learned_addr learned my addr 172.21.15.12:0/260110408 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7e6ffd700 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 0x5647b77e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7e7fff700 1 -- 172.21.15.12:0/260110408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 msgr2=0x5647b77e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7e7fff700 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 0x5647b77e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7e7fff700 1 -- 172.21.15.12:0/260110408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 msgr2=0x5647b77e8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7e7fff700 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77e8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7e7fff700 1 -- 172.21.15.12:0/260110408 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc017070 con 0x5647b77e5720 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7e6ffd700 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 0x5647b77e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7e7fff700 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77e95e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0012ea0 tx=0x7fd7e0014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4684a99e455d94b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/260110408 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000e040 con 0x5647b77e5720 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/260110408 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd7e001a070 con 0x5647b77e5720 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/260110408 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647b7688270 con 0x5647b77e5720 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/260110408 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7e00093a0 con 0x5647b77e5720 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7f9f4eb80 10 client.29084 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.877+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/260110408 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7fd7e81a65a0 con 0x5647b77e5720 2024-05-09T00:00:13.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.878+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/260110408 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647b7688270 con 0x5647b77e5720 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.878+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/260110408 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd7e0026070 con 0x5647b77e5720 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.878+0000 7fd7f9f4eb80 10 client.29084 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.878+0000 7fd7f9f4eb80 10 client.29084 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.878+0000 7fd7f9f4eb80 20 client.29084 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.878+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5647b77f8fa0 0x5647b77fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.878+0000 7fd7f9f4eb80 4 client.29084 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.878+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/260110408 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647b77ae810 con 0x5647b77f8fa0 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.878+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5647b77f8fa0 0x5647b77fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.879+0000 7fd7c6ffd700 10 client.29084 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.879+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5647b77f8fa0 0x5647b77fb3c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eac6b60a0c9b7fd9 server_cookie=8b0aeeaac94c96f3 in_seq=0 out_seq=0 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.879+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/260110408 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5647b77ae810 con 0x5647b77f8fa0 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.879+0000 7fd7c6ffd700 10 client.29084 handle_command_reply: tid=0 2024-05-09T00:00:13.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.879+0000 7fd7f9f4eb80 1 client.29084 shutdown 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/260110408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 msgr2=0x5647b77e95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77e95e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0012ea0 tx=0x7fd7e0014600 comp rx=0 tx=0).stop 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/260110408 shutdown_connections 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5647b77f8fa0 0x5647b77fb3c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77e95e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77df890 0x5647b77e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/260110408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 0x5647b77e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7c6ffd700 0 client.29084 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/260110408 >> 172.21.15.12:0/260110408 conn(0x5647b773d210 msgr2=0x5647b77d95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/260110408 shutdown_connections 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/260110408 wait complete. 2024-05-09T00:00:13.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.880+0000 7fd7f9f4eb80 20 client.29084 trim_cache size 0 max 16384 2024-05-09T00:00:13.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.896+0000 7fd7f9f4eb80 1 Processor -- start 2024-05-09T00:00:13.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7f9f4eb80 1 -- start start 2024-05-09T00:00:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77df890 0x5647b77f4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 0x5647b77d9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e5720 0x5647b77efe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b77ea110 con 0x5647b77e5720 2024-05-09T00:00:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b78005d0 con 0x5647b77df890 2024-05-09T00:00:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b77e0260 con 0x5647b77e4e00 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e5720 0x5647b77efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e5720 0x5647b77efe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60556/0 (socket says 172.21.15.12:60556) 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7e77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77df890 0x5647b77f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7e7fff700 1 -- 172.21.15.12:0/2219576570 learned_addr learned my addr 172.21.15.12:0/2219576570 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.897+0000 7fd7e6ffd700 1 --2- 172.21.15.12:0/2219576570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 0x5647b77d9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.898+0000 7fd7e77fe700 1 -- 172.21.15.12:0/2219576570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 msgr2=0x5647b77d9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.898+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/2219576570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 0x5647b77d9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.898+0000 7fd7e77fe700 1 -- 172.21.15.12:0/2219576570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e5720 msgr2=0x5647b77efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.898+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/2219576570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e5720 0x5647b77efe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.898+0000 7fd7e77fe700 1 -- 172.21.15.12:0/2219576570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647b755cb60 con 0x5647b77df890 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.898+0000 7fd7e7fff700 1 --2- 172.21.15.12:0/2219576570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e5720 0x5647b77efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.898+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/2219576570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77df890 0x5647b77f4930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5647b77ebd40 tx=0x7fd7d8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dce4b092e07ed79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.898+0000 7fd7ce7fc700 1 -- 172.21.15.12:0/2219576570 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d809afd0 con 0x5647b77df890 2024-05-09T00:00:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.898+0000 7fd7ce7fc700 1 -- 172.21.15.12:0/2219576570 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d8099370 con 0x5647b77df890 2024-05-09T00:00:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.898+0000 7fd7ce7fc700 1 -- 172.21.15.12:0/2219576570 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d809b230 con 0x5647b77df890 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.899+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2219576570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77df890 msgr2=0x5647b77f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.899+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/2219576570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77df890 0x5647b77f4930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5647b77ebd40 tx=0x7fd7d8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.899+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2219576570 shutdown_connections 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.899+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/2219576570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e5720 0x5647b77efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.899+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/2219576570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77df890 0x5647b77f4930 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.899+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/2219576570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77e4e00 0x5647b77d9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.899+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2219576570 >> 172.21.15.12:0/2219576570 conn(0x5647b773d210 msgr2=0x5647b77d6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.899+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2219576570 shutdown_connections 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.899+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2219576570 wait complete. 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.900+0000 7fd7f9f4eb80 1 -- start start 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.900+0000 7fd7f9f4eb80 1 Processor -- start 2024-05-09T00:00:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.900+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77df890 0x5647b77eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.900+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e4e00 0x5647b77ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.900+0000 7fd7f9f4eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77ee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.900+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b784e4a0 con 0x5647b77e4e00 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.900+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b784e5e0 con 0x5647b77e5720 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.900+0000 7fd7f9f4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b784e720 con 0x5647b77df890 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.900+0000 7fd7f9f4eb80 10 client.0 _ll_register_callbacks cb 0x5647b76d4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77df890 0x5647b77eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77df890 0x5647b77eddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40634/0 (socket says 172.21.15.12:40634) 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e77fe700 1 -- 172.21.15.12:0/2098772039 learned_addr learned my addr 172.21.15.12:0/2098772039 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e6ffd700 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e4e00 0x5647b77ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e7fff700 1 -- 172.21.15.12:0/2098772039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77df890 msgr2=0x5647b77eddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e7fff700 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77df890 0x5647b77eddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e7fff700 1 -- 172.21.15.12:0/2098772039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e4e00 msgr2=0x5647b77ee310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e7fff700 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e4e00 0x5647b77ee310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e7fff700 1 -- 172.21.15.12:0/2098772039 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e81a65a0 con 0x5647b77e5720 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7e7fff700 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77ee850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0002230 tx=0x7fd7e001fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4713bd1225762cb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/2098772039 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000e040 con 0x5647b77e5720 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/2098772039 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd7e001a030 con 0x5647b77e5720 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.901+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2098772039 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7d809ee00 con 0x5647b77e5720 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/2098772039 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7e00093a0 con 0x5647b77e5720 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7f9f4eb80 10 client.29090 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2098772039 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x5647b75b03b0 con 0x5647b77e5720 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/2098772039 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7e003c080 con 0x5647b77e5720 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/2098772039 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd7e002e070 con 0x5647b77e5720 2024-05-09T00:00:13.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7f9f4eb80 10 client.29090 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7f9f4eb80 10 client.29090 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7f9f4eb80 20 client.29090 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5647b784fd90 0x5647b78521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7f9f4eb80 4 client.29090 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2098772039 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5647b7727ec0 con 0x5647b784fd90 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.902+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5647b784fd90 0x5647b78521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7c6ffd700 10 client.29090 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7e77fe700 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5647b784fd90 0x5647b78521b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5d3dcd0b940333d server_cookie=7c12f1e18f421732 in_seq=0 out_seq=0 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7c6ffd700 1 -- 172.21.15.12:0/2098772039 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5647b7727ec0 con 0x5647b784fd90 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7c6ffd700 10 client.29090 handle_command_reply: tid=0 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 client.29090 shutdown 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2098772039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 msgr2=0x5647b77ee850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77ee850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0002230 tx=0x7fd7e001fc30 comp rx=0 tx=0).stop 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2098772039 shutdown_connections 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5647b784fd90 0x5647b78521b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5647b77e5720 0x5647b77ee850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647b77df890 0x5647b77eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 --2- 172.21.15.12:0/2098772039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647b77e4e00 0x5647b77ee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7c6ffd700 0 client.29090 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2098772039 >> 172.21.15.12:0/2098772039 conn(0x5647b773d210 msgr2=0x5647b77e3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2098772039 shutdown_connections 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 1 -- 172.21.15.12:0/2098772039 wait complete. 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.903+0000 7fd7f9f4eb80 20 client.29090 trim_cache size 0 max 16384 2024-05-09T00:00:13.902 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:00:13.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 -- 172.21.15.12:0/4148493067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd7d00a1290 msgr2=0x7fd7d00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd7d00a1290 0x7fd7d00a3750 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80749d0 tx=0x7fd7dc009260 comp rx=0 tx=0).stop 2024-05-09T00:00:13.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 -- 172.21.15.12:0/4148493067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7e8101160 msgr2=0x7fd7e80732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:13.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7e8101160 0x7fd7e80732a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8004760 tx=0x7fd7d8004790 comp rx=0 tx=0).stop 2024-05-09T00:00:13.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 -- 172.21.15.12:0/4148493067 shutdown_connections 2024-05-09T00:00:13.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd7d00a1290 0x7fd7d00a3750 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7e8073d20 0x7fd7e81a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7e8101160 0x7fd7e80732a0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 --2- 172.21.15.12:0/4148493067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7e8106280 0x7fd7e80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:13.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 -- 172.21.15.12:0/4148493067 >> 172.21.15.12:0/4148493067 conn(0x7fd7e80fc9c0 msgr2=0x7fd7e80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:13.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 -- 172.21.15.12:0/4148493067 shutdown_connections 2024-05-09T00:00:13.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:13.904+0000 7fd7edd94700 1 -- 172.21.15.12:0/4148493067 wait complete. 2024-05-09T00:00:13.917 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:00:14.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:13 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3429206519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:00:14.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.124+0000 7fc497a57700 1 -- 172.21.15.12:0/2300045965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc490105160 msgr2=0x7fc49010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:14.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.124+0000 7fc497a57700 1 --2- 172.21.15.12:0/2300045965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc490105160 0x7fc49010cce0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fc48800b0a0 tx=0x7fc48800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.124+0000 7fc497a57700 1 -- 172.21.15.12:0/2300045965 shutdown_connections 2024-05-09T00:00:14.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.124+0000 7fc497a57700 1 --2- 172.21.15.12:0/2300045965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc490105160 0x7fc49010cce0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.125+0000 7fc497a57700 1 --2- 172.21.15.12:0/2300045965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc490100060 0x7fc490100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.125+0000 7fc497a57700 1 --2- 172.21.15.12:0/2300045965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc490100a10 0x7fc490104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.125+0000 7fc497a57700 1 -- 172.21.15.12:0/2300045965 >> 172.21.15.12:0/2300045965 conn(0x7fc490076f40 msgr2=0x7fc490077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.125+0000 7fc497a57700 1 -- 172.21.15.12:0/2300045965 shutdown_connections 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.125+0000 7fc497a57700 1 -- 172.21.15.12:0/2300045965 wait complete. 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.125+0000 7fc497a57700 1 Processor -- start 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.125+0000 7fc497a57700 1 -- start start 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc497a57700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc490100060 0x7fc4901996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc497a57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc490105160 0x7fc490199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc497a57700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc49019dfd0 0x7fc49019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc497a57700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49019ea50 con 0x7fc490105160 2024-05-09T00:00:14.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc497a57700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49019ebc0 con 0x7fc490100060 2024-05-09T00:00:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc497a57700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49019ed80 con 0x7fc49019dfd0 2024-05-09T00:00:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc494ff2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc490105160 0x7fc490199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc4957f3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc490100060 0x7fc4901996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc494ff2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc490105160 0x7fc490199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60570/0 (socket says 172.21.15.12:60570) 2024-05-09T00:00:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc494ff2700 1 -- 172.21.15.12:0/1913747022 learned_addr learned my addr 172.21.15.12:0/1913747022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc495ff4700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc49019dfd0 0x7fc49019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc4957f3700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc490100060 0x7fc4901996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44370/0 (socket says 172.21.15.12:44370) 2024-05-09T00:00:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc494ff2700 1 -- 172.21.15.12:0/1913747022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc49019dfd0 msgr2=0x7fc49019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:14.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc494ff2700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc49019dfd0 0x7fc49019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc494ff2700 1 -- 172.21.15.12:0/1913747022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc490100060 msgr2=0x7fc4901996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc494ff2700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc490100060 0x7fc4901996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc494ff2700 1 -- 172.21.15.12:0/1913747022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc488009d00 con 0x7fc490105160 2024-05-09T00:00:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc4957f3700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc490100060 0x7fc4901996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.126+0000 7fc494ff2700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc490105160 0x7fc490199c30 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc48000e970 tx=0x7fc48000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.127+0000 7fc4867fc700 1 -- 172.21.15.12:0/1913747022 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480009790 con 0x7fc490105160 2024-05-09T00:00:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.127+0000 7fc4867fc700 1 -- 172.21.15.12:0/1913747022 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc48000c7e0 con 0x7fc490105160 2024-05-09T00:00:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.127+0000 7fc497a57700 1 -- 172.21.15.12:0/1913747022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4901a62e0 con 0x7fc490105160 2024-05-09T00:00:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.127+0000 7fc4867fc700 1 -- 172.21.15.12:0/1913747022 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800054e0 con 0x7fc490105160 2024-05-09T00:00:14.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.127+0000 7fc497a57700 1 -- 172.21.15.12:0/1913747022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4901a6810 con 0x7fc490105160 2024-05-09T00:00:14.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.129+0000 7fc4867fc700 1 -- 172.21.15.12:0/1913747022 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc48000c950 con 0x7fc490105160 2024-05-09T00:00:14.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.130+0000 7fc4867fc700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc47c0a1310 0x7fc47c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.131+0000 7fc4867fc700 1 -- 172.21.15.12:0/1913747022 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc480014070 con 0x7fc490105160 2024-05-09T00:00:14.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.131+0000 7fc4a3c11b80 1 Processor -- start 2024-05-09T00:00:14.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.131+0000 7fc4a3c11b80 1 -- start start 2024-05-09T00:00:14.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.131+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d5509e00 0x55c3d550a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.131+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d550a720 0x55c3d5504350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.131+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.131+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d545b150 con 0x55c3d5504890 2024-05-09T00:00:14.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.131+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d5458b10 con 0x55c3d5509e00 2024-05-09T00:00:14.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.131+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d5454400 con 0x55c3d550a720 2024-05-09T00:00:14.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.132+0000 7fc495ff4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.132+0000 7fc495ff4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60576/0 (socket says 172.21.15.12:60576) 2024-05-09T00:00:14.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.132+0000 7fc495ff4700 1 -- 172.21.15.12:0/1866299870 learned_addr learned my addr 172.21.15.12:0/1866299870 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.132+0000 7fc494ff2700 1 --2- 172.21.15.12:0/1866299870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d550a720 0x55c3d5504350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.132+0000 7fc495ff4700 1 -- 172.21.15.12:0/1866299870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d550a720 msgr2=0x55c3d5504350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.132+0000 7fc495ff4700 1 --2- 172.21.15.12:0/1866299870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d550a720 0x55c3d5504350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.133+0000 7fc495ff4700 1 -- 172.21.15.12:0/1866299870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d5509e00 msgr2=0x55c3d550a1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:00:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.133+0000 7fc495ff4700 1 --2- 172.21.15.12:0/1866299870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d5509e00 0x55c3d550a1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.133+0000 7fc495ff4700 1 -- 172.21.15.12:0/1866299870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d52d53b0 con 0x55c3d5504890 2024-05-09T00:00:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.133+0000 7fc4957f3700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc47c0a1310 0x7fc47c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.133+0000 7fc495ff4700 1 --2- 172.21.15.12:0/1866299870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504cd0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc488015000 tx=0x7fc4880092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc48cf6e2f6ec7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:14.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.133+0000 7fc47b7fe700 1 -- 172.21.15.12:0/1866299870 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48800e040 con 0x55c3d5504890 2024-05-09T00:00:14.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.134+0000 7fc47b7fe700 1 -- 172.21.15.12:0/1866299870 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc488026070 con 0x55c3d5504890 2024-05-09T00:00:14.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.134+0000 7fc47b7fe700 1 -- 172.21.15.12:0/1866299870 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488007740 con 0x55c3d5504890 2024-05-09T00:00:14.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.134+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/1866299870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 msgr2=0x55c3d5504cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:14.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.134+0000 7fc4957f3700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc47c0a1310 0x7fc47c0a37d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc48c011550 tx=0x7fc48c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:14.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.134+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/1866299870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504cd0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc488015000 tx=0x7fc4880092e0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.135+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/1866299870 shutdown_connections 2024-05-09T00:00:14.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.135+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/1866299870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504cd0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.135+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/1866299870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d5509e00 0x55c3d550a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.135+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/1866299870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d550a720 0x55c3d5504350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.135+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/1866299870 >> 172.21.15.12:0/1866299870 conn(0x55c3d5462210 msgr2=0x55c3d54fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:14.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.136+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/1866299870 shutdown_connections 2024-05-09T00:00:14.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.137+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/1866299870 wait complete. 2024-05-09T00:00:14.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.137+0000 7fc4a3c11b80 1 -- start start 2024-05-09T00:00:14.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.137+0000 7fc4a3c11b80 1 Processor -- start 2024-05-09T00:00:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.139+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 0x55c3d550dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.139+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5509e00 0x55c3d550e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d550a720 0x55c3d550e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d53f0560 con 0x55c3d5509e00 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d53f0200 con 0x55c3d550a720 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d52d76a0 con 0x55c3d5504890 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc4a3c11b80 10 client.0 _ll_register_callbacks cb 0x55c3d53f9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc494ff2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5509e00 0x55c3d550e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc4957f3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 0x55c3d550dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc494ff2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5509e00 0x55c3d550e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60586/0 (socket says 172.21.15.12:60586) 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc494ff2700 1 -- 172.21.15.12:0/3829931275 learned_addr learned my addr 172.21.15.12:0/3829931275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc494ff2700 1 -- 172.21.15.12:0/3829931275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 msgr2=0x55c3d550dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc494ff2700 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 0x55c3d550dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc494ff2700 1 -- 172.21.15.12:0/3829931275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d550a720 msgr2=0x55c3d550e670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc494ff2700 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d550a720 0x55c3d550e670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.140+0000 7fc494ff2700 1 -- 172.21.15.12:0/3829931275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49010d550 con 0x55c3d5509e00 2024-05-09T00:00:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.141+0000 7fc494ff2700 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5509e00 0x55c3d550e130 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc48000efd0 tx=0x7fc48009a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2133bfef145eaa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:14.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.141+0000 7fc478ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:14.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.141+0000 7fc478ff9700 1 -- 172.21.15.12:0/3829931275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800a23f0 con 0x55c3d5509e00 2024-05-09T00:00:14.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.141+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/3829931275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3d53ad270 con 0x55c3d5509e00 2024-05-09T00:00:14.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.141+0000 7fc478ff9700 1 -- 172.21.15.12:0/3829931275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc4800ade40 con 0x55c3d5509e00 2024-05-09T00:00:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.141+0000 7fc478ff9700 1 -- 172.21.15.12:0/3829931275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4800b8250 con 0x55c3d5509e00 2024-05-09T00:00:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.141+0000 7fc4a3c11b80 10 client.19389 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.142+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/3829931275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55c3d5281b60 con 0x55c3d5509e00 2024-05-09T00:00:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.142+0000 7fc478ff9700 1 -- 172.21.15.12:0/3829931275 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3d53ad270 con 0x55c3d5509e00 2024-05-09T00:00:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.142+0000 7fc478ff9700 1 -- 172.21.15.12:0/3829931275 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc480099460 con 0x55c3d5509e00 2024-05-09T00:00:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.143+0000 7fc4a3c11b80 10 client.19389 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.143+0000 7fc4a3c11b80 10 client.19389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.143+0000 7fc4a3c11b80 20 client.19389 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.143+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3d552f2a0 0x55c3d55316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.143+0000 7fc4a3c11b80 4 client.19389 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:14.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.143+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/3829931275 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3d54d3810 con 0x55c3d552f2a0 2024-05-09T00:00:14.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.143+0000 7fc4957f3700 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3d552f2a0 0x55c3d55316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.145+0000 7fc478ff9700 10 client.19389 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:14.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.145+0000 7fc4957f3700 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3d552f2a0 0x55c3d55316c0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b884d3e14ee61ff server_cookie=bfcb600882f12f87 in_seq=0 out_seq=0 2024-05-09T00:00:14.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc478ff9700 1 -- 172.21.15.12:0/3829931275 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3d54d3810 con 0x55c3d552f2a0 2024-05-09T00:00:14.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc478ff9700 10 client.19389 handle_command_reply: tid=0 2024-05-09T00:00:14.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc4a3c11b80 1 client.19389 shutdown 2024-05-09T00:00:14.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/3829931275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5509e00 msgr2=0x55c3d550e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:14.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5509e00 0x55c3d550e130 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc48000efd0 tx=0x7fc48009a720 comp rx=0 tx=0).stop 2024-05-09T00:00:14.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/3829931275 shutdown_connections 2024-05-09T00:00:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3d552f2a0 0x55c3d55316c0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d550a720 0x55c3d550e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc478ff9700 0 client.19389 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 0x55c3d550dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/3829931275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5509e00 0x55c3d550e130 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.146+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/3829931275 >> 172.21.15.12:0/3829931275 conn(0x55c3d5462210 msgr2=0x55c3d54fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.147+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/3829931275 shutdown_connections 2024-05-09T00:00:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.147+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/3829931275 wait complete. 2024-05-09T00:00:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.147+0000 7fc4a3c11b80 20 client.19389 trim_cache size 0 max 16384 2024-05-09T00:00:14.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4a3c11b80 1 Processor -- start 2024-05-09T00:00:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4a3c11b80 1 -- start start 2024-05-09T00:00:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5509e00 0x55c3d5500a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d550a720 0x55c3d5519bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d552bd50 con 0x55c3d5504890 2024-05-09T00:00:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d550f1a0 con 0x55c3d550a720 2024-05-09T00:00:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d552c8e0 con 0x55c3d5509e00 2024-05-09T00:00:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4957f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc495ff4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d550a720 0x55c3d5519bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4957f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60602/0 (socket says 172.21.15.12:60602) 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc4957f3700 1 -- 172.21.15.12:0/1565833447 learned_addr learned my addr 172.21.15.12:0/1565833447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.172+0000 7fc494ff2700 1 --2- 172.21.15.12:0/1565833447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5509e00 0x55c3d5500a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.173+0000 7fc4957f3700 1 -- 172.21.15.12:0/1565833447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5509e00 msgr2=0x55c3d5500a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.173+0000 7fc4957f3700 1 --2- 172.21.15.12:0/1565833447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5509e00 0x55c3d5500a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.173+0000 7fc4957f3700 1 -- 172.21.15.12:0/1565833447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d550a720 msgr2=0x55c3d5519bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.173+0000 7fc4957f3700 1 --2- 172.21.15.12:0/1565833447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d550a720 0x55c3d5519bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.173+0000 7fc4957f3700 1 -- 172.21.15.12:0/1565833447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d541a560 con 0x55c3d5504890 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.173+0000 7fc495ff4700 1 --2- 172.21.15.12:0/1565833447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d550a720 0x55c3d5519bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.173+0000 7fc4957f3700 1 --2- 172.21.15.12:0/1565833447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504350 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x55c3d5531fd0 tx=0x7fc48c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897aa1227de99409 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.173+0000 7fc47bfff700 1 -- 172.21.15.12:0/1565833447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c024040 con 0x55c3d5504890 2024-05-09T00:00:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.173+0000 7fc47bfff700 1 -- 172.21.15.12:0/1565833447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc48c004030 con 0x55c3d5504890 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.173+0000 7fc47bfff700 1 -- 172.21.15.12:0/1565833447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c027e40 con 0x55c3d5504890 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.174+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/1565833447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 msgr2=0x55c3d5504350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.174+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/1565833447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504350 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x55c3d5531fd0 tx=0x7fc48c013040 comp rx=0 tx=0).stop 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.175+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/1565833447 shutdown_connections 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.175+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/1565833447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d550a720 0x55c3d5519bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.175+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/1565833447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d5504890 0x55c3d5504350 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.175+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/1565833447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5509e00 0x55c3d5500a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.175+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/1565833447 >> 172.21.15.12:0/1565833447 conn(0x55c3d5462210 msgr2=0x55c3d54fb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.175+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/1565833447 shutdown_connections 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.175+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/1565833447 wait complete. 2024-05-09T00:00:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.175+0000 7fc4a3c11b80 1 -- start start 2024-05-09T00:00:14.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc4a3c11b80 1 Processor -- start 2024-05-09T00:00:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 0x55c3d5580780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d5509e00 0x55c3d5580cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc4a3c11b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d550a720 0x55c3d5581200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d55130f0 con 0x55c3d550a720 2024-05-09T00:00:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d5513260 con 0x55c3d5509e00 2024-05-09T00:00:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc4a3c11b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d5513420 con 0x55c3d5504890 2024-05-09T00:00:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc4a3c11b80 10 client.0 _ll_register_callbacks cb 0x55c3d53f9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc4957f3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 0x55c3d5580780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc494ff2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d5509e00 0x55c3d5580cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc494ff2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d5509e00 0x55c3d5580cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44394/0 (socket says 172.21.15.12:44394) 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc4957f3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 0x55c3d5580780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40706/0 (socket says 172.21.15.12:40706) 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.176+0000 7fc494ff2700 1 -- 172.21.15.12:0/4107139659 learned_addr learned my addr 172.21.15.12:0/4107139659 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.177+0000 7fc4957f3700 1 -- 172.21.15.12:0/4107139659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d5509e00 msgr2=0x55c3d5580cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.177+0000 7fc4957f3700 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d5509e00 0x55c3d5580cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.177+0000 7fc4957f3700 1 -- 172.21.15.12:0/4107139659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d550a720 msgr2=0x55c3d5581200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.177+0000 7fc4957f3700 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d550a720 0x55c3d5581200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.177+0000 7fc4957f3700 1 -- 172.21.15.12:0/4107139659 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3d52d53b0 con 0x55c3d5504890 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.180+0000 7fc4957f3700 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 0x55c3d5580780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc48c013010 tx=0x7fc48c003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e20671a4a207c108 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.180+0000 7fc478ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.180+0000 7fc478ff9700 1 -- 172.21.15.12:0/4107139659 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c027de0 con 0x55c3d5504890 2024-05-09T00:00:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.181+0000 7fc478ff9700 1 -- 172.21.15.12:0/4107139659 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc48c027290 con 0x55c3d5504890 2024-05-09T00:00:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.181+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/4107139659 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc488026070 con 0x55c3d5504890 2024-05-09T00:00:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.183+0000 7fc478ff9700 1 -- 172.21.15.12:0/4107139659 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc48c027450 con 0x55c3d5504890 2024-05-09T00:00:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.183+0000 7fc4a3c11b80 10 client.29116 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.183+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/4107139659 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7fc48c017070 con 0x55c3d5504890 2024-05-09T00:00:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.184+0000 7fc478ff9700 1 -- 172.21.15.12:0/4107139659 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc488026070 con 0x55c3d5504890 2024-05-09T00:00:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.184+0000 7fc478ff9700 1 -- 172.21.15.12:0/4107139659 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc48c004c00 con 0x55c3d5504890 2024-05-09T00:00:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.184+0000 7fc4a3c11b80 10 client.29116 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.184+0000 7fc4a3c11b80 10 client.29116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.184+0000 7fc4a3c11b80 20 client.29116 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.184+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3d55971d0 0x55c3d55995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.184+0000 7fc4a3c11b80 4 client.29116 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:00:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.184+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/4107139659 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3d544cec0 con 0x55c3d55971d0 2024-05-09T00:00:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.184+0000 7fc494ff2700 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3d55971d0 0x55c3d55995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:14.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.185+0000 7fc478ff9700 10 client.29116 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:14.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.185+0000 7fc494ff2700 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3d55971d0 0x55c3d55995f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b577769898f24e server_cookie=5223e6cf1d7dffb0 in_seq=0 out_seq=0 2024-05-09T00:00:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc478ff9700 1 -- 172.21.15.12:0/4107139659 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3d544cec0 con 0x55c3d55971d0 2024-05-09T00:00:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc478ff9700 10 client.29116 handle_command_reply: tid=0 2024-05-09T00:00:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 client.29116 shutdown 2024-05-09T00:00:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/4107139659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 msgr2=0x55c3d5580780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 0x55c3d5580780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc48c013010 tx=0x7fc48c003f90 comp rx=0 tx=0).stop 2024-05-09T00:00:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/4107139659 shutdown_connections 2024-05-09T00:00:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3d550a720 0x55c3d5581200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3d5504890 0x55c3d5580780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3d55971d0 0x55c3d55995f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 --2- 172.21.15.12:0/4107139659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3d5509e00 0x55c3d5580cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc478ff9700 0 client.29116 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/4107139659 >> 172.21.15.12:0/4107139659 conn(0x55c3d5462210 msgr2=0x55c3d550c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/4107139659 shutdown_connections 2024-05-09T00:00:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.191+0000 7fc4a3c11b80 1 -- 172.21.15.12:0/4107139659 wait complete. 2024-05-09T00:00:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc4a3c11b80 20 client.29116 trim_cache size 0 max 16384 2024-05-09T00:00:14.188 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:00:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc497a57700 1 -- 172.21.15.12:0/1913747022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc47c0a1310 msgr2=0x7fc47c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc497a57700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc47c0a1310 0x7fc47c0a37d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc48c011550 tx=0x7fc48c009260 comp rx=0 tx=0).stop 2024-05-09T00:00:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc497a57700 1 -- 172.21.15.12:0/1913747022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc490105160 msgr2=0x7fc490199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc497a57700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc490105160 0x7fc490199c30 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc48000e970 tx=0x7fc48000ec80 comp rx=0 tx=0).stop 2024-05-09T00:00:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc497a57700 1 -- 172.21.15.12:0/1913747022 shutdown_connections 2024-05-09T00:00:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc497a57700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc47c0a1310 0x7fc47c0a37d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc497a57700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc49019dfd0 0x7fc49019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc497a57700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc490100060 0x7fc4901996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc497a57700 1 --2- 172.21.15.12:0/1913747022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc490105160 0x7fc490199c30 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.192+0000 7fc497a57700 1 -- 172.21.15.12:0/1913747022 >> 172.21.15.12:0/1913747022 conn(0x7fc490076f40 msgr2=0x7fc49010c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.193+0000 7fc497a57700 1 -- 172.21.15.12:0/1913747022 shutdown_connections 2024-05-09T00:00:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:14.193+0000 7fc497a57700 1 -- 172.21.15.12:0/1913747022 wait complete. 2024-05-09T00:00:14.213 INFO:tasks.workunit.client.0.smithi012.stdout: 10 4092 41.77 MB/sec warmup 9 sec latency 8607.996 ms 2024-05-09T00:00:14.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:14 smithi012 ceph-mon[42503]: pgmap v443: 97 pgs: 97 active+clean; 226 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 515 op/s 2024-05-09T00:00:15.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:14 smithi204 ceph-mon[47373]: pgmap v443: 97 pgs: 97 active+clean; 226 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 515 op/s 2024-05-09T00:00:15.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:14 smithi160 ceph-mon[46152]: pgmap v443: 97 pgs: 97 active+clean; 226 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 515 op/s 2024-05-09T00:00:15.206 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:00:15.207 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:00:15.213 INFO:tasks.workunit.client.0.smithi012.stdout: 10 4710 41.44 MB/sec warmup 10 sec latency 9608.034 ms 2024-05-09T00:00:15.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.374+0000 7ff83d1e3700 1 -- 172.21.15.12:0/1460408132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838069770 msgr2=0x7ff838069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.374+0000 7ff83d1e3700 1 --2- 172.21.15.12:0/1460408132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838069770 0x7ff838069b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff828009990 tx=0x7ff828009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.374+0000 7ff83d1e3700 1 -- 172.21.15.12:0/1460408132 shutdown_connections 2024-05-09T00:00:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.374+0000 7ff83d1e3700 1 --2- 172.21.15.12:0/1460408132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff838106280 0x7ff83810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.374+0000 7ff83d1e3700 1 --2- 172.21.15.12:0/1460408132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838069770 0x7ff838069b50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.374+0000 7ff83d1e3700 1 --2- 172.21.15.12:0/1460408132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff83806a120 0x7ff838105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.374+0000 7ff83d1e3700 1 -- 172.21.15.12:0/1460408132 >> 172.21.15.12:0/1460408132 conn(0x7ff838076fa0 msgr2=0x7ff8380773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.374+0000 7ff83d1e3700 1 -- 172.21.15.12:0/1460408132 shutdown_connections 2024-05-09T00:00:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.375+0000 7ff83d1e3700 1 -- 172.21.15.12:0/1460408132 wait complete. 2024-05-09T00:00:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83d1e3700 1 Processor -- start 2024-05-09T00:00:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83d1e3700 1 -- start start 2024-05-09T00:00:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83d1e3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff838069770 0x7ff8380ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83d1e3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838106280 0x7ff838100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83d1e3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff838100890 0x7ff8381a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83d1e3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8381a5bd0 con 0x7ff838100890 2024-05-09T00:00:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83d1e3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8381a5d40 con 0x7ff838069770 2024-05-09T00:00:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83d1e3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8381a5f00 con 0x7ff838106280 2024-05-09T00:00:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff836d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff838069770 0x7ff8380ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838106280 0x7ff838100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838106280 0x7ff838100350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40714/0 (socket says 172.21.15.12:40714) 2024-05-09T00:00:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff836d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff838069770 0x7ff8380ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44420/0 (socket says 172.21.15.12:44420) 2024-05-09T00:00:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83659c700 1 -- 172.21.15.12:0/3446487855 learned_addr learned my addr 172.21.15.12:0/3446487855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.376+0000 7ff83759e700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff838100890 0x7ff8381a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff83659c700 1 -- 172.21.15.12:0/3446487855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff838069770 msgr2=0x7ff8380ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff83659c700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff838069770 0x7ff8380ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff83659c700 1 -- 172.21.15.12:0/3446487855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff838100890 msgr2=0x7ff8381a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff83659c700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff838100890 0x7ff8381a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff83659c700 1 -- 172.21.15.12:0/3446487855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8280095f0 con 0x7ff838106280 2024-05-09T00:00:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff836d9d700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff838069770 0x7ff8380ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff83659c700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838106280 0x7ff838100350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00e970 tx=0x7ff82c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff81ffff700 1 -- 172.21.15.12:0/3446487855 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c009790 con 0x7ff838106280 2024-05-09T00:00:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff81ffff700 1 -- 172.21.15.12:0/3446487855 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff82c00c7e0 con 0x7ff838106280 2024-05-09T00:00:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff83d1e3700 1 -- 172.21.15.12:0/3446487855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8381a61e0 con 0x7ff838106280 2024-05-09T00:00:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff81ffff700 1 -- 172.21.15.12:0/3446487855 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0054e0 con 0x7ff838106280 2024-05-09T00:00:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.377+0000 7ff83d1e3700 1 -- 172.21.15.12:0/3446487855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8381a6740 con 0x7ff838106280 2024-05-09T00:00:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.380+0000 7ff81ffff700 1 -- 172.21.15.12:0/3446487855 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff82c021440 con 0x7ff838106280 2024-05-09T00:00:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.380+0000 7ff81ffff700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8200a1310 0x7ff8200a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.380+0000 7ff836d9d700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8200a1310 0x7ff8200a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.381+0000 7ff81ffff700 1 -- 172.21.15.12:0/3446487855 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff82c014070 con 0x7ff838106280 2024-05-09T00:00:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.381+0000 7ff836d9d700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8200a1310 0x7ff8200a37d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff82800b2e0 tx=0x7ff828005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff84939db80 1 Processor -- start 2024-05-09T00:00:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff84939db80 1 -- start start 2024-05-09T00:00:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff84939db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff84939db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 0x564d287ed3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff84939db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff84939db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d287441d0 con 0x564d287f37a0 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff84939db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d28741b90 con 0x564d287f2e80 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff84939db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2873d480 con 0x564d287ed910 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff83659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 0x564d287ed3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff836d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff83759e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff836d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44436/0 (socket says 172.21.15.12:44436) 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.382+0000 7ff836d9d700 1 -- 172.21.15.12:0/2465634845 learned_addr learned my addr 172.21.15.12:0/2465634845 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.383+0000 7ff836d9d700 1 -- 172.21.15.12:0/2465634845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 msgr2=0x564d287edd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.383+0000 7ff836d9d700 1 --2- 172.21.15.12:0/2465634845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287edd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.383+0000 7ff836d9d700 1 -- 172.21.15.12:0/2465634845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 msgr2=0x564d287ed3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.383+0000 7ff836d9d700 1 --2- 172.21.15.12:0/2465634845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 0x564d287ed3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.383+0000 7ff836d9d700 1 -- 172.21.15.12:0/2465634845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d285be430 con 0x564d287f2e80 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.383+0000 7ff836d9d700 1 --2- 172.21.15.12:0/2465634845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f3260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff828005610 tx=0x7ff828020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=301d4e17e76f8f97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.383+0000 7ff81d7fa700 1 -- 172.21.15.12:0/2465634845 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82802b070 con 0x564d287f2e80 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.383+0000 7ff81d7fa700 1 -- 172.21.15.12:0/2465634845 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff82801e040 con 0x564d287f2e80 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.383+0000 7ff81d7fa700 1 -- 172.21.15.12:0/2465634845 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828024d40 con 0x564d287f2e80 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.384+0000 7ff84939db80 1 -- 172.21.15.12:0/2465634845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 msgr2=0x564d287f3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.384+0000 7ff84939db80 1 --2- 172.21.15.12:0/2465634845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f3260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff828005610 tx=0x7ff828020000 comp rx=0 tx=0).stop 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.384+0000 7ff84939db80 1 -- 172.21.15.12:0/2465634845 shutdown_connections 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.384+0000 7ff84939db80 1 --2- 172.21.15.12:0/2465634845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287edd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.384+0000 7ff84939db80 1 --2- 172.21.15.12:0/2465634845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f3260 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.384+0000 7ff84939db80 1 --2- 172.21.15.12:0/2465634845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 0x564d287ed3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.384+0000 7ff84939db80 1 -- 172.21.15.12:0/2465634845 >> 172.21.15.12:0/2465634845 conn(0x564d2874b290 msgr2=0x564d287e7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.384+0000 7ff84939db80 1 -- 172.21.15.12:0/2465634845 shutdown_connections 2024-05-09T00:00:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.384+0000 7ff84939db80 1 -- 172.21.15.12:0/2465634845 wait complete. 2024-05-09T00:00:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.384+0000 7ff84939db80 1 -- start start 2024-05-09T00:00:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff84939db80 1 Processor -- start 2024-05-09T00:00:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff84939db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287ed910 0x564d287f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff84939db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff84939db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287f37a0 0x564d287f7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff84939db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d286d95e0 con 0x564d287ed910 2024-05-09T00:00:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff84939db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d286d9280 con 0x564d287f2e80 2024-05-09T00:00:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff84939db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d285c0720 con 0x564d287f37a0 2024-05-09T00:00:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff84939db80 10 client.0 _ll_register_callbacks cb 0x564d286e2bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff836d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287ed910 0x564d287f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff83659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff836d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287ed910 0x564d287f6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60638/0 (socket says 172.21.15.12:60638) 2024-05-09T00:00:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff836d9d700 1 -- 172.21.15.12:0/2961574809 learned_addr learned my addr 172.21.15.12:0/2961574809 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.385+0000 7ff83759e700 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287f37a0 0x564d287f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff836d9d700 1 -- 172.21.15.12:0/2961574809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287f37a0 msgr2=0x564d287f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff836d9d700 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287f37a0 0x564d287f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff836d9d700 1 -- 172.21.15.12:0/2961574809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 msgr2=0x564d287f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff836d9d700 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff836d9d700 1 -- 172.21.15.12:0/2961574809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83810e670 con 0x564d287ed910 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff83659c700 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff836d9d700 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287ed910 0x564d287f6ad0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff8280098e0 tx=0x7ff828024640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66854a3757834a4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff81aff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff81aff5700 1 -- 172.21.15.12:0/2961574809 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82802b040 con 0x564d287ed910 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff81aff5700 1 -- 172.21.15.12:0/2961574809 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff828005640 con 0x564d287ed910 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff84939db80 1 -- 172.21.15.12:0/2961574809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d286962f0 con 0x564d287ed910 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff81aff5700 1 -- 172.21.15.12:0/2961574809 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff828024cf0 con 0x564d287ed910 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff84939db80 10 client.19404 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.386+0000 7ff84939db80 1 -- 172.21.15.12:0/2961574809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x564d2856abe0 con 0x564d287ed910 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.387+0000 7ff81aff5700 1 -- 172.21.15.12:0/2961574809 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff828044020 con 0x564d287ed910 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.387+0000 7ff81aff5700 1 -- 172.21.15.12:0/2961574809 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff8280173f0 con 0x564d287ed910 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.387+0000 7ff84939db80 10 client.19404 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.387+0000 7ff84939db80 10 client.19404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.387+0000 7ff84939db80 20 client.19404 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.387+0000 7ff84939db80 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d28806cb0 0x564d288090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.387+0000 7ff84939db80 4 client.19404 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.387+0000 7ff84939db80 1 -- 172.21.15.12:0/2961574809 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d287bc890 con 0x564d28806cb0 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.387+0000 7ff83659c700 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d28806cb0 0x564d288090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.388+0000 7ff81aff5700 10 client.19404 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.388+0000 7ff83659c700 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d28806cb0 0x564d288090a0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474844e196249d66 server_cookie=1cfade84c062c0d3 in_seq=0 out_seq=0 2024-05-09T00:00:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff81aff5700 1 -- 172.21.15.12:0/2961574809 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564d287bc890 con 0x564d28806cb0 2024-05-09T00:00:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff81aff5700 10 client.19404 handle_command_reply: tid=0 2024-05-09T00:00:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 client.19404 shutdown 2024-05-09T00:00:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 -- 172.21.15.12:0/2961574809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287ed910 msgr2=0x564d287f6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287ed910 0x564d287f6ad0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff8280098e0 tx=0x7ff828024640 comp rx=0 tx=0).stop 2024-05-09T00:00:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 -- 172.21.15.12:0/2961574809 shutdown_connections 2024-05-09T00:00:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287f37a0 0x564d287f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287ed910 0x564d287f6ad0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d28806cb0 0x564d288090a0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 --2- 172.21.15.12:0/2961574809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff81aff5700 0 client.19404 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 -- 172.21.15.12:0/2961574809 >> 172.21.15.12:0/2961574809 conn(0x564d2874b290 msgr2=0x564d287e7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 -- 172.21.15.12:0/2961574809 shutdown_connections 2024-05-09T00:00:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 1 -- 172.21.15.12:0/2961574809 wait complete. 2024-05-09T00:00:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.393+0000 7ff84939db80 20 client.19404 trim_cache size 0 max 16384 2024-05-09T00:00:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.413+0000 7ff84939db80 1 Processor -- start 2024-05-09T00:00:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.413+0000 7ff84939db80 1 -- start start 2024-05-09T00:00:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff84939db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287ed3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff84939db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f2e80 0x564d287e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff84939db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f37a0 0x564d287fe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff84939db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d287f9640 con 0x564d287f2e80 2024-05-09T00:00:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff84939db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d287e4e40 con 0x564d287f37a0 2024-05-09T00:00:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff84939db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d287e2b10 con 0x564d287ed910 2024-05-09T00:00:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff836d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287ed3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff836d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287ed3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40748/0 (socket says 172.21.15.12:40748) 2024-05-09T00:00:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff836d9d700 1 -- 172.21.15.12:0/1116612262 learned_addr learned my addr 172.21.15.12:0/1116612262 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:15.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff83759e700 1 --2- 172.21.15.12:0/1116612262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f37a0 0x564d287fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff83659c700 1 --2- 172.21.15.12:0/1116612262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f2e80 0x564d287e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff836d9d700 1 -- 172.21.15.12:0/1116612262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f37a0 msgr2=0x564d287fe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff836d9d700 1 --2- 172.21.15.12:0/1116612262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f37a0 0x564d287fe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff836d9d700 1 -- 172.21.15.12:0/1116612262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f2e80 msgr2=0x564d287e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff836d9d700 1 --2- 172.21.15.12:0/1116612262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f2e80 0x564d287e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff836d9d700 1 -- 172.21.15.12:0/1116612262 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d287e4560 con 0x564d287ed910 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.414+0000 7ff836d9d700 1 --2- 172.21.15.12:0/1116612262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287ed3d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff828009520 tx=0x7ff8280094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=372299a1a49f0aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.415+0000 7ff81dffb700 1 -- 172.21.15.12:0/1116612262 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82802b030 con 0x564d287ed910 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.415+0000 7ff81dffb700 1 -- 172.21.15.12:0/1116612262 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff828044030 con 0x564d287ed910 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.415+0000 7ff81dffb700 1 -- 172.21.15.12:0/1116612262 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828024d50 con 0x564d287ed910 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.415+0000 7ff84939db80 1 -- 172.21.15.12:0/1116612262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 msgr2=0x564d287ed3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.415+0000 7ff84939db80 1 --2- 172.21.15.12:0/1116612262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287ed3d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff828009520 tx=0x7ff8280094b0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.415+0000 7ff84939db80 1 -- 172.21.15.12:0/1116612262 shutdown_connections 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.415+0000 7ff84939db80 1 --2- 172.21.15.12:0/1116612262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f37a0 0x564d287fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.415+0000 7ff84939db80 1 --2- 172.21.15.12:0/1116612262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287ed3d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.415+0000 7ff84939db80 1 --2- 172.21.15.12:0/1116612262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f2e80 0x564d287e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.415+0000 7ff84939db80 1 -- 172.21.15.12:0/1116612262 >> 172.21.15.12:0/1116612262 conn(0x564d2874b290 msgr2=0x564d287f53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 1 -- 172.21.15.12:0/1116612262 shutdown_connections 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 1 -- 172.21.15.12:0/1116612262 wait complete. 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 1 -- start start 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 1 Processor -- start 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287fe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287fa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 0x564d287faf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d287fb610 con 0x564d287f37a0 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d287fb780 con 0x564d287f2e80 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2885d150 con 0x564d287ed910 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff84939db80 10 client.0 _ll_register_callbacks cb 0x564d286e2bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff836d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287fe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff83759e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 0x564d287faf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.416+0000 7ff836d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287fe520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40760/0 (socket says 172.21.15.12:40760) 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff83759e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 0x564d287faf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60662/0 (socket says 172.21.15.12:60662) 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff836d9d700 1 -- 172.21.15.12:0/3712180814 learned_addr learned my addr 172.21.15.12:0/3712180814 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff83659c700 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287fa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff836d9d700 1 -- 172.21.15.12:0/3712180814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 msgr2=0x564d287fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff836d9d700 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287fa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff836d9d700 1 -- 172.21.15.12:0/3712180814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 msgr2=0x564d287faf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff836d9d700 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 0x564d287faf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff836d9d700 1 -- 172.21.15.12:0/3712180814 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d2856abe0 con 0x564d287ed910 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff83759e700 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 0x564d287faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff836d9d700 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287fe520 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff82800b690 tx=0x7ff82800b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43c15acd875cb400 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff81aff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff81aff5700 1 -- 172.21.15.12:0/3712180814 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82802b040 con 0x564d287ed910 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff81aff5700 1 -- 172.21.15.12:0/3712180814 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff828045080 con 0x564d287ed910 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.417+0000 7ff84939db80 1 -- 172.21.15.12:0/3712180814 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff82801e040 con 0x564d287ed910 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff81aff5700 1 -- 172.21.15.12:0/3712180814 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff828024cf0 con 0x564d287ed910 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff84939db80 10 client.29134 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff84939db80 1 -- 172.21.15.12:0/3712180814 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x564d287edfc0 con 0x564d287ed910 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff81aff5700 1 -- 172.21.15.12:0/3712180814 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff82801e040 con 0x564d287ed910 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff81aff5700 1 -- 172.21.15.12:0/3712180814 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff828059070 con 0x564d287ed910 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff84939db80 10 client.29134 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff84939db80 10 client.29134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff84939db80 20 client.29134 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff84939db80 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d2885e410 0x564d28860800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff84939db80 4 client.29134 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:00:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff84939db80 1 -- 172.21.15.12:0/3712180814 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564d28735f40 con 0x564d2885e410 2024-05-09T00:00:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.418+0000 7ff83659c700 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d2885e410 0x564d28860800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.419+0000 7ff83659c700 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d2885e410 0x564d28860800 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dee47d735a47b8f server_cookie=a974761214181236 in_seq=0 out_seq=0 2024-05-09T00:00:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.419+0000 7ff81aff5700 10 client.29134 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:15.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff81aff5700 1 -- 172.21.15.12:0/3712180814 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564d28735f40 con 0x564d2885e410 2024-05-09T00:00:15.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff81aff5700 10 client.29134 handle_command_reply: tid=0 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 client.29134 shutdown 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 -- 172.21.15.12:0/3712180814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 msgr2=0x564d287fe520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287fe520 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff82800b690 tx=0x7ff82800b390 comp rx=0 tx=0).stop 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 -- 172.21.15.12:0/3712180814 shutdown_connections 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d287f37a0 0x564d287faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d287ed910 0x564d287fe520 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d2885e410 0x564d28860800 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 --2- 172.21.15.12:0/3712180814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d287f2e80 0x564d287fa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff81aff5700 0 client.29134 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 -- 172.21.15.12:0/3712180814 >> 172.21.15.12:0/3712180814 conn(0x564d2874b290 msgr2=0x564d287f1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 -- 172.21.15.12:0/3712180814 shutdown_connections 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 1 -- 172.21.15.12:0/3712180814 wait complete. 2024-05-09T00:00:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.424+0000 7ff84939db80 20 client.29134 trim_cache size 0 max 16384 2024-05-09T00:00:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:00:15.421 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:00:15.421 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "d356106f-f5f3-4192-a713-396dcacfc0dc", 2024-05-09T00:00:15.421 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:00:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:00:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 -- 172.21.15.12:0/3446487855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8200a1310 msgr2=0x7ff8200a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8200a1310 0x7ff8200a37d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff82800b2e0 tx=0x7ff828005fd0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 -- 172.21.15.12:0/3446487855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838106280 msgr2=0x7ff838100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838106280 0x7ff838100350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00e970 tx=0x7ff82c00ec80 comp rx=0 tx=0).stop 2024-05-09T00:00:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 -- 172.21.15.12:0/3446487855 shutdown_connections 2024-05-09T00:00:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8200a1310 0x7ff8200a37d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff838100890 0x7ff8381a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff838069770 0x7ff8380ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 --2- 172.21.15.12:0/3446487855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838106280 0x7ff838100350 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 -- 172.21.15.12:0/3446487855 >> 172.21.15.12:0/3446487855 conn(0x7ff838076fa0 msgr2=0x7ff8380fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 -- 172.21.15.12:0/3446487855 shutdown_connections 2024-05-09T00:00:15.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.425+0000 7ff83d1e3700 1 -- 172.21.15.12:0/3446487855 wait complete. 2024-05-09T00:00:15.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:00:15.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.635+0000 7efce7bb9700 1 -- 172.21.15.12:0/3002990564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efce0101180 msgr2=0x7efce0101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.635+0000 7efce7bb9700 1 --2- 172.21.15.12:0/3002990564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efce0101180 0x7efce0101560 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009990 tx=0x7efcdc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.635+0000 7efce7bb9700 1 -- 172.21.15.12:0/3002990564 shutdown_connections 2024-05-09T00:00:15.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.635+0000 7efce7bb9700 1 --2- 172.21.15.12:0/3002990564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efce01062a0 0x7efce010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.635+0000 7efce7bb9700 1 --2- 172.21.15.12:0/3002990564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efce0101180 0x7efce0101560 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.635+0000 7efce7bb9700 1 --2- 172.21.15.12:0/3002990564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efce0101b30 0x7efce0105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.635+0000 7efce7bb9700 1 -- 172.21.15.12:0/3002990564 >> 172.21.15.12:0/3002990564 conn(0x7efce00fc9e0 msgr2=0x7efce00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.635+0000 7efce7bb9700 1 -- 172.21.15.12:0/3002990564 shutdown_connections 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.635+0000 7efce7bb9700 1 -- 172.21.15.12:0/3002990564 wait complete. 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce7bb9700 1 Processor -- start 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce7bb9700 1 -- start start 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce7bb9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efce0101180 0x7efce00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce7bb9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efce01062a0 0x7efce00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce7bb9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efce0073d20 0x7efce01a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce7bb9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce01a5cc0 con 0x7efce0101180 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce7bb9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce01a5e30 con 0x7efce01062a0 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce7bb9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce01a5ff0 con 0x7efce0073d20 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce5955700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efce0101180 0x7efce00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce5955700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efce0101180 0x7efce00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60680/0 (socket says 172.21.15.12:60680) 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce5955700 1 -- 172.21.15.12:0/3516382626 learned_addr learned my addr 172.21.15.12:0/3516382626 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.636+0000 7efce5154700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efce01062a0 0x7efce00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efce6156700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efce0073d20 0x7efce01a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efce5955700 1 -- 172.21.15.12:0/3516382626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efce0073d20 msgr2=0x7efce01a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efce5955700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efce0073d20 0x7efce01a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efce5955700 1 -- 172.21.15.12:0/3516382626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efce01062a0 msgr2=0x7efce00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efce5955700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efce01062a0 0x7efce00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efce5955700 1 -- 172.21.15.12:0/3516382626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc0095f0 con 0x7efce0101180 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efce5154700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efce01062a0 0x7efce00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efce5955700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efce0101180 0x7efce00732a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7efcdc000c00 tx=0x7efcdc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efcd6ffd700 1 -- 172.21.15.12:0/3516382626 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc01d070 con 0x7efce0101180 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efcd6ffd700 1 -- 172.21.15.12:0/3516382626 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcdc022470 con 0x7efce0101180 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efce7bb9700 1 -- 172.21.15.12:0/3516382626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce01a6270 con 0x7efce0101180 2024-05-09T00:00:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efcd6ffd700 1 -- 172.21.15.12:0/3516382626 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc017650 con 0x7efce0101180 2024-05-09T00:00:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.637+0000 7efce7bb9700 1 -- 172.21.15.12:0/3516382626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce01a66f0 con 0x7efce0101180 2024-05-09T00:00:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.640+0000 7efcd6ffd700 1 -- 172.21.15.12:0/3516382626 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efcdc0225e0 con 0x7efce0101180 2024-05-09T00:00:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.641+0000 7efcd6ffd700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efccc0a1310 0x7efccc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.641+0000 7efce5154700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efccc0a1310 0x7efccc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.641+0000 7efcd6ffd700 1 -- 172.21.15.12:0/3516382626 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efcdc0cde70 con 0x7efce0101180 2024-05-09T00:00:15.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.641+0000 7efcf3d73b80 1 Processor -- start 2024-05-09T00:00:15.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efcf3d73b80 1 -- start start 2024-05-09T00:00:15.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5154700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efccc0a1310 0x7efccc0a37d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efce00749d0 tx=0x7efcd0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fd51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fcf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fcfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0f26150 con 0x55b3f0fd5720 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0f23b10 con 0x55b3f0fd4e00 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0f1f400 con 0x55b3f0fcf890 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fcf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fcf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60690/0 (socket says 172.21.15.12:60690) 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5955700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fd51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5154700 1 -- 172.21.15.12:0/156490339 learned_addr learned my addr 172.21.15.12:0/156490339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce6156700 1 --2- 172.21.15.12:0/156490339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fcfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5154700 1 -- 172.21.15.12:0/156490339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 msgr2=0x55b3f0fcfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5154700 1 --2- 172.21.15.12:0/156490339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fcfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5154700 1 -- 172.21.15.12:0/156490339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 msgr2=0x55b3f0fd51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5154700 1 --2- 172.21.15.12:0/156490339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fd51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5154700 1 -- 172.21.15.12:0/156490339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3f0da03b0 con 0x55b3f0fd5720 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.642+0000 7efce5955700 1 --2- 172.21.15.12:0/156490339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fd51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.643+0000 7efce5154700 1 --2- 172.21.15.12:0/156490339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fcf350 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7efcd0004390 tx=0x7efcd0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af78629853434ac3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.643+0000 7efccbfff700 1 -- 172.21.15.12:0/156490339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0003d10 con 0x55b3f0fd5720 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.643+0000 7efccbfff700 1 -- 172.21.15.12:0/156490339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcd0003e70 con 0x55b3f0fd5720 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.643+0000 7efccbfff700 1 -- 172.21.15.12:0/156490339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd001e770 con 0x55b3f0fd5720 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 -- 172.21.15.12:0/156490339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 msgr2=0x55b3f0fcf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/156490339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fcf350 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7efcd0004390 tx=0x7efcd0014000 comp rx=0 tx=0).stop 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 -- 172.21.15.12:0/156490339 shutdown_connections 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/156490339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fcfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/156490339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fd51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/156490339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fcf350 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 -- 172.21.15.12:0/156490339 >> 172.21.15.12:0/156490339 conn(0x55b3f0f2d210 msgr2=0x55b3f0fc9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 -- 172.21.15.12:0/156490339 shutdown_connections 2024-05-09T00:00:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 -- 172.21.15.12:0/156490339 wait complete. 2024-05-09T00:00:15.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 -- start start 2024-05-09T00:00:15.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.644+0000 7efcf3d73b80 1 Processor -- start 2024-05-09T00:00:15.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fcf890 0x55b3f0fdb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fd8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fd5720 0x55b3f0fd9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0ebb560 con 0x55b3f0fcf890 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0ebb200 con 0x55b3f0fd4e00 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0da26a0 con 0x55b3f0fd5720 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efcf3d73b80 10 client.0 _ll_register_callbacks cb 0x55b3f0ec4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efce5955700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fcf890 0x55b3f0fdb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efce5955700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fcf890 0x55b3f0fdb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60694/0 (socket says 172.21.15.12:60694) 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efce5955700 1 -- 172.21.15.12:0/2571903267 learned_addr learned my addr 172.21.15.12:0/2571903267 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.645+0000 7efce5154700 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fd8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efce6156700 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fd5720 0x55b3f0fd9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efce5955700 1 -- 172.21.15.12:0/2571903267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fd5720 msgr2=0x55b3f0fd9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efce5955700 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fd5720 0x55b3f0fd9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efce5955700 1 -- 172.21.15.12:0/2571903267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 msgr2=0x55b3f0fd8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efce5955700 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fd8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efce5955700 1 -- 172.21.15.12:0/2571903267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce01a66f0 con 0x55b3f0fcf890 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efce5154700 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fd8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efce5955700 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fcf890 0x55b3f0fdb750 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0cbbd0 tx=0x7efcdc09a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d41e02d8ac47217 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efcc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efcc97fa700 1 -- 172.21.15.12:0/2571903267 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc09b020 con 0x55b3f0fcf890 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efcc97fa700 1 -- 172.21.15.12:0/2571903267 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efcdc09ee50 con 0x55b3f0fcf890 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efcf3d73b80 1 -- 172.21.15.12:0/2571903267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3f0e78270 con 0x55b3f0fcf890 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efcc97fa700 1 -- 172.21.15.12:0/2571903267 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcdc0a3410 con 0x55b3f0fcf890 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efcf3d73b80 10 client.19422 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efcf3d73b80 1 -- 172.21.15.12:0/2571903267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7efcd0017070 con 0x55b3f0fcf890 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.646+0000 7efcc97fa700 1 -- 172.21.15.12:0/2571903267 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3f0e78270 con 0x55b3f0fcf890 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.647+0000 7efcc97fa700 1 -- 172.21.15.12:0/2571903267 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7efcdc0cbd20 con 0x55b3f0fcf890 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.647+0000 7efcf3d73b80 10 client.19422 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.647+0000 7efcf3d73b80 10 client.19422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:15.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.647+0000 7efcf3d73b80 20 client.19422 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:15.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.647+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b3f0fe8d30 0x55b3f0feb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.647+0000 7efcf3d73b80 4 client.19422 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:15.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.647+0000 7efcf3d73b80 1 -- 172.21.15.12:0/2571903267 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3f0f9e810 con 0x55b3f0fe8d30 2024-05-09T00:00:15.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.647+0000 7efce5154700 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b3f0fe8d30 0x55b3f0feb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.647+0000 7efce5154700 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b3f0fe8d30 0x55b3f0feb150 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f884dedd54779595 server_cookie=4cd3628960882da6 in_seq=0 out_seq=0 2024-05-09T00:00:15.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.647+0000 7efcc97fa700 10 client.19422 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:15.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcc97fa700 1 -- 172.21.15.12:0/2571903267 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b3f0f9e810 con 0x55b3f0fe8d30 2024-05-09T00:00:15.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcc97fa700 10 client.19422 handle_command_reply: tid=0 2024-05-09T00:00:15.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcf3d73b80 1 client.19422 shutdown 2024-05-09T00:00:15.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcf3d73b80 1 -- 172.21.15.12:0/2571903267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fcf890 msgr2=0x55b3f0fdb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fcf890 0x55b3f0fdb750 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0cbbd0 tx=0x7efcdc09a980 comp rx=0 tx=0).stop 2024-05-09T00:00:15.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcf3d73b80 1 -- 172.21.15.12:0/2571903267 shutdown_connections 2024-05-09T00:00:15.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fd5720 0x55b3f0fd9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fcf890 0x55b3f0fdb750 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b3f0fe8d30 0x55b3f0feb150 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/2571903267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fd8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcf3d73b80 1 -- 172.21.15.12:0/2571903267 >> 172.21.15.12:0/2571903267 conn(0x55b3f0f2d210 msgr2=0x55b3f0fc95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcc97fa700 0 client.19422 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:15.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.650+0000 7efcf3d73b80 1 -- 172.21.15.12:0/2571903267 shutdown_connections 2024-05-09T00:00:15.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.651+0000 7efcf3d73b80 1 -- 172.21.15.12:0/2571903267 wait complete. 2024-05-09T00:00:15.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.651+0000 7efcf3d73b80 20 client.19422 trim_cache size 0 max 16384 2024-05-09T00:00:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.669+0000 7efcf3d73b80 1 Processor -- start 2024-05-09T00:00:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efcf3d73b80 1 -- start start 2024-05-09T00:00:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fcf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fc9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fe0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0fc4a90 con 0x55b3f0fd5720 2024-05-09T00:00:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0fc4800 con 0x55b3f0fd4e00 2024-05-09T00:00:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0ff0340 con 0x55b3f0fcf890 2024-05-09T00:00:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce6156700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fe0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce5955700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fcf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce5154700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fc9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce5955700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fcf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40810/0 (socket says 172.21.15.12:40810) 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce5154700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fc9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44498/0 (socket says 172.21.15.12:44498) 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce5955700 1 -- 172.21.15.12:0/3112085205 learned_addr learned my addr 172.21.15.12:0/3112085205 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce6156700 1 --2- 172.21.15.12:0/3112085205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fe0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60706/0 (socket says 172.21.15.12:60706) 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce5955700 1 -- 172.21.15.12:0/3112085205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 msgr2=0x55b3f0fc9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce5955700 1 --2- 172.21.15.12:0/3112085205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fc9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce5955700 1 -- 172.21.15.12:0/3112085205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 msgr2=0x55b3f0fe0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce5955700 1 --2- 172.21.15.12:0/3112085205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fe0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.670+0000 7efce5955700 1 -- 172.21.15.12:0/3112085205 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3f0d4cb60 con 0x55b3f0fcf890 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.671+0000 7efce5955700 1 --2- 172.21.15.12:0/3112085205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fcf350 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efcdc09afb0 tx=0x7efcdc09af20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19f82f2101dbded4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.671+0000 7efcd4da4700 1 -- 172.21.15.12:0/3112085205 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc09ee50 con 0x55b3f0fcf890 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.671+0000 7efcd4da4700 1 -- 172.21.15.12:0/3112085205 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcdc017e90 con 0x55b3f0fcf890 2024-05-09T00:00:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.671+0000 7efcd4da4700 1 -- 172.21.15.12:0/3112085205 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0a3000 con 0x55b3f0fcf890 2024-05-09T00:00:15.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.671+0000 7efcf3d73b80 1 -- 172.21.15.12:0/3112085205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 msgr2=0x55b3f0fcf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.671+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/3112085205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fcf350 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efcdc09afb0 tx=0x7efcdc09af20 comp rx=0 tx=0).stop 2024-05-09T00:00:15.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.671+0000 7efcf3d73b80 1 -- 172.21.15.12:0/3112085205 shutdown_connections 2024-05-09T00:00:15.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.671+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/3112085205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd5720 0x55b3f0fe0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.672+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/3112085205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fcf350 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.672+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/3112085205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fc9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.672+0000 7efcf3d73b80 1 -- 172.21.15.12:0/3112085205 >> 172.21.15.12:0/3112085205 conn(0x55b3f0f2d210 msgr2=0x55b3f0fd2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.672+0000 7efcf3d73b80 1 -- 172.21.15.12:0/3112085205 shutdown_connections 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.672+0000 7efcf3d73b80 1 -- 172.21.15.12:0/3112085205 wait complete. 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.672+0000 7efcf3d73b80 1 -- start start 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.672+0000 7efcf3d73b80 1 Processor -- start 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.672+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fe0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.672+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fde710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efcf3d73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd5720 0x55b3f0fdec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0fdf190 con 0x55b3f0fd4e00 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0fdf300 con 0x55b3f0fd5720 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efcf3d73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f0fdf4c0 con 0x55b3f0fcf890 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efcf3d73b80 10 client.0 _ll_register_callbacks cb 0x55b3f0ec4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5955700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fe0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fde710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60722/0 (socket says 172.21.15.12:60722) 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5154700 1 -- 172.21.15.12:0/4294966743 learned_addr learned my addr 172.21.15.12:0/4294966743 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:15.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce6156700 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd5720 0x55b3f0fdec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5955700 1 -- 172.21.15.12:0/4294966743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd5720 msgr2=0x55b3f0fdec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5955700 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd5720 0x55b3f0fdec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5955700 1 -- 172.21.15.12:0/4294966743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd4e00 msgr2=0x55b3f0fde710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5955700 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fde710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5955700 1 -- 172.21.15.12:0/4294966743 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd0017070 con 0x55b3f0fcf890 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5154700 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efce5955700 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fe0220 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efcdc097a90 tx=0x7efcdc005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c80077d60d91a51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efcc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efcc97fa700 1 -- 172.21.15.12:0/4294966743 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc09ee50 con 0x55b3f0fcf890 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.673+0000 7efcc97fa700 1 -- 172.21.15.12:0/4294966743 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efcdc0b9020 con 0x55b3f0fcf890 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcf3d73b80 1 -- 172.21.15.12:0/4294966743 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcd0003e70 con 0x55b3f0fcf890 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcc97fa700 1 -- 172.21.15.12:0/4294966743 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcdc0a3190 con 0x55b3f0fcf890 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcf3d73b80 10 client.29152 fetch_fsmap learned FSMap version 60 2024-05-09T00:00:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcf3d73b80 1 -- 172.21.15.12:0/4294966743 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55b3f0ee5560 con 0x55b3f0fcf890 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcc97fa700 1 -- 172.21.15.12:0/4294966743 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcd0003e70 con 0x55b3f0fcf890 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcc97fa700 1 -- 172.21.15.12:0/4294966743 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 60) v1 ==== 7482+0+0 (secure 0 0 0) 0x7efcdc097730 con 0x55b3f0fcf890 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcf3d73b80 10 client.29152 fetch_fsmap finished waiting for FSMap version 60 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcf3d73b80 10 client.29152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcf3d73b80 20 client.29152 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b3f1040150 0x55b3f1042540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcf3d73b80 4 client.29152 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efcf3d73b80 1 -- 172.21.15.12:0/4294966743 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b3f0f17ec0 con 0x55b3f1040150 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.674+0000 7efce5154700 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b3f1040150 0x55b3f1042540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.675+0000 7efce5154700 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b3f1040150 0x55b3f1042540 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bfc702b645f5f7e server_cookie=a1298a18d7af2715 in_seq=0 out_seq=0 2024-05-09T00:00:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.675+0000 7efcc97fa700 10 client.29152 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcc97fa700 1 -- 172.21.15.12:0/4294966743 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55b3f0f17ec0 con 0x55b3f1040150 2024-05-09T00:00:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcc97fa700 10 client.29152 handle_command_reply: tid=0 2024-05-09T00:00:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 client.29152 shutdown 2024-05-09T00:00:15.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 -- 172.21.15.12:0/4294966743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 msgr2=0x55b3f0fe0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fe0220 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efcdc097a90 tx=0x7efcdc005fd0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 -- 172.21.15.12:0/4294966743 shutdown_connections 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b3f1040150 0x55b3f1042540 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b3f0fd5720 0x55b3f0fdec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3f0fcf890 0x55b3f0fe0220 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 --2- 172.21.15.12:0/4294966743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3f0fd4e00 0x55b3f0fde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcc97fa700 0 client.29152 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 -- 172.21.15.12:0/4294966743 >> 172.21.15.12:0/4294966743 conn(0x55b3f0f2d210 msgr2=0x55b3f0fd7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 -- 172.21.15.12:0/4294966743 shutdown_connections 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 1 -- 172.21.15.12:0/4294966743 wait complete. 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.678+0000 7efcf3d73b80 20 client.29152 trim_cache size 0 max 16384 2024-05-09T00:00:15.675 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:00:15.676 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (461 inodes in the stack)", 2024-05-09T00:00:15.676 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:00:15.676 INFO:teuthology.orchestra.run.smithi012.stdout: "d356106f-f5f3-4192-a713-396dcacfc0dc": { 2024-05-09T00:00:15.676 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:00:15.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "d356106f-f5f3-4192-a713-396dcacfc0dc", 2024-05-09T00:00:15.676 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:00:15.676 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:00:15.676 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:00:15.676 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 -- 172.21.15.12:0/3516382626 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efccc0a1310 msgr2=0x7efccc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efccc0a1310 0x7efccc0a37d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efce00749d0 tx=0x7efcd0009260 comp rx=0 tx=0).stop 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 -- 172.21.15.12:0/3516382626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efce0101180 msgr2=0x7efce00732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efce0101180 0x7efce00732a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7efcdc000c00 tx=0x7efcdc005740 comp rx=0 tx=0).stop 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 -- 172.21.15.12:0/3516382626 shutdown_connections 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efccc0a1310 0x7efccc0a37d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efce0073d20 0x7efce01a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efce0101180 0x7efce00732a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 --2- 172.21.15.12:0/3516382626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efce01062a0 0x7efce00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 -- 172.21.15.12:0/3516382626 >> 172.21.15.12:0/3516382626 conn(0x7efce00fc9e0 msgr2=0x7efce00fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 -- 172.21.15.12:0/3516382626 shutdown_connections 2024-05-09T00:00:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:15.679+0000 7efce7bb9700 1 -- 172.21.15.12:0/3516382626 wait complete. 2024-05-09T00:00:15.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:15 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:00:15.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:15 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:15.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:15 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:00:15.688 INFO:tasks.cephfs.filesystem:scrub status for tag:d356106f-f5f3-4192-a713-396dcacfc0dc - {'path': '/', 'tag': 'd356106f-f5f3-4192-a713-396dcacfc0dc', 'options': 'recursive,force'} 2024-05-09T00:00:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:15 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:00:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:15 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:15 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:00:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:15 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:00:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:15 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:15 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:00:16.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 5448 39.55 MB/sec warmup 11 sec latency 10445.470 ms 2024-05-09T00:00:16.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:16 smithi012 ceph-mon[42503]: pgmap v444: 97 pgs: 97 active+clean; 226 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 515 op/s 2024-05-09T00:00:16.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:16 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:00:16.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:16.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:16 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:00:16.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:17.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:16 smithi204 ceph-mon[47373]: pgmap v444: 97 pgs: 97 active+clean; 226 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 515 op/s 2024-05-09T00:00:17.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:16 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:00:17.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:17.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:16 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:00:17.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:17.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:16 smithi160 ceph-mon[46152]: pgmap v444: 97 pgs: 97 active+clean; 226 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 515 op/s 2024-05-09T00:00:17.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:16 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:00:17.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:17.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:16 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:00:17.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:17.213 INFO:tasks.workunit.client.0.smithi012.stdout: 10 6147 41.05 MB/sec warmup 12 sec latency 592.604 ms 2024-05-09T00:00:17.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:17 smithi012 ceph-mon[42503]: pgmap v445: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 637 op/s 2024-05-09T00:00:18.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:17 smithi204 ceph-mon[47373]: pgmap v445: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 637 op/s 2024-05-09T00:00:18.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:17 smithi160 ceph-mon[46152]: pgmap v445: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 637 op/s 2024-05-09T00:00:18.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 6614 41.31 MB/sec warmup 13 sec latency 1592.781 ms 2024-05-09T00:00:18.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:18 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:19.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:18 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:19.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:18 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:19.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 6769 38.55 MB/sec warmup 14 sec latency 2592.818 ms 2024-05-09T00:00:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:19 smithi012 ceph-mon[42503]: pgmap v446: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 637 op/s 2024-05-09T00:00:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:19 smithi204 ceph-mon[47373]: pgmap v446: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 637 op/s 2024-05-09T00:00:20.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:19 smithi160 ceph-mon[46152]: pgmap v446: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 637 op/s 2024-05-09T00:00:20.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 6988 36.04 MB/sec warmup 15 sec latency 3592.855 ms 2024-05-09T00:00:21.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 7389 34.87 MB/sec warmup 16 sec latency 4495.181 ms 2024-05-09T00:00:21.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:21 smithi012 ceph-mon[42503]: pgmap v447: 97 pgs: 97 active+clean; 251 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 769 op/s 2024-05-09T00:00:22.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:21 smithi204 ceph-mon[47373]: pgmap v447: 97 pgs: 97 active+clean; 251 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 769 op/s 2024-05-09T00:00:22.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:21 smithi160 ceph-mon[46152]: pgmap v447: 97 pgs: 97 active+clean; 251 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 769 op/s 2024-05-09T00:00:22.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 8119 35.01 MB/sec warmup 17 sec latency 98.940 ms 2024-05-09T00:00:23.215 INFO:tasks.workunit.client.0.smithi012.stdout: 10 8788 34.68 MB/sec warmup 18 sec latency 133.416 ms 2024-05-09T00:00:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.808+0000 7f94d062a700 1 -- 172.21.15.12:0/3376914607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c8069ad0 msgr2=0x7f94c810d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.808+0000 7f94d062a700 1 --2- 172.21.15.12:0/3376914607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c8069ad0 0x7f94c810d3b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009990 tx=0x7f94b8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.808+0000 7f94d062a700 1 -- 172.21.15.12:0/3376914607 shutdown_connections 2024-05-09T00:00:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.808+0000 7f94d062a700 1 --2- 172.21.15.12:0/3376914607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c810d8f0 0x7f94c810fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.808+0000 7f94d062a700 1 --2- 172.21.15.12:0/3376914607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c80691b0 0x7f94c8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.808+0000 7f94d062a700 1 --2- 172.21.15.12:0/3376914607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c8069ad0 0x7f94c810d3b0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.808+0000 7f94d062a700 1 -- 172.21.15.12:0/3376914607 >> 172.21.15.12:0/3376914607 conn(0x7f94c8076f40 msgr2=0x7f94c8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.808+0000 7f94d062a700 1 -- 172.21.15.12:0/3376914607 shutdown_connections 2024-05-09T00:00:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.808+0000 7f94d062a700 1 -- 172.21.15.12:0/3376914607 wait complete. 2024-05-09T00:00:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.809+0000 7f94d062a700 1 Processor -- start 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.809+0000 7f94d062a700 1 -- start start 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.809+0000 7f94d062a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c80691b0 0x7f94c8199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.809+0000 7f94d062a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c8069ad0 0x7f94c8199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.809+0000 7f94d062a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c810d8f0 0x7f94c819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.809+0000 7f94d062a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c819e490 con 0x7f94c810d8f0 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.809+0000 7f94d062a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c819e600 con 0x7f94c8069ad0 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.809+0000 7f94d062a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c819e7c0 con 0x7f94c80691b0 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94cebc7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c810d8f0 0x7f94c819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94ce3c6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c80691b0 0x7f94c8199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94cdbc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c8069ad0 0x7f94c8199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94cebc7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c810d8f0 0x7f94c819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50584/0 (socket says 172.21.15.12:50584) 2024-05-09T00:00:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94cebc7700 1 -- 172.21.15.12:0/2868373442 learned_addr learned my addr 172.21.15.12:0/2868373442 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94cebc7700 1 -- 172.21.15.12:0/2868373442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c80691b0 msgr2=0x7f94c8199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94cebc7700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c80691b0 0x7f94c8199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94cebc7700 1 -- 172.21.15.12:0/2868373442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c8069ad0 msgr2=0x7f94c8199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94cebc7700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c8069ad0 0x7f94c8199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94cebc7700 1 -- 172.21.15.12:0/2868373442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b80095f0 con 0x7f94c810d8f0 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c80691b0 0x7f94c8199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94cebc7700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c810d8f0 0x7f94c819de50 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f94c0007ec0 tx=0x7f94c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94bf7fe700 1 -- 172.21.15.12:0/2868373442 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0017070 con 0x7f94c810d8f0 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94bf7fe700 1 -- 172.21.15.12:0/2868373442 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94c000f040 con 0x7f94c810d8f0 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94d062a700 1 -- 172.21.15.12:0/2868373442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c819eaa0 con 0x7f94c810d8f0 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.810+0000 7f94bf7fe700 1 -- 172.21.15.12:0/2868373442 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0004020 con 0x7f94c810d8f0 2024-05-09T00:00:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.811+0000 7f94d062a700 1 -- 172.21.15.12:0/2868373442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c81a62e0 con 0x7f94c810d8f0 2024-05-09T00:00:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.813+0000 7f94bf7fe700 1 -- 172.21.15.12:0/2868373442 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f94c000db20 con 0x7f94c810d8f0 2024-05-09T00:00:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.814+0000 7f94bf7fe700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f94b40a1290 0x7f94b40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.814+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f94b40a1290 0x7f94b40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.814+0000 7f94bf7fe700 1 -- 172.21.15.12:0/2868373442 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f94c00cc210 con 0x7f94c810d8f0 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.815+0000 7f94dc7e4b80 1 Processor -- start 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.815+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f94b40a1290 0x7f94b40a3750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f94c4011490 tx=0x7f94c4009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.815+0000 7f94dc7e4b80 1 -- start start 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.815+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb2950 0x562b3dbb2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.815+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 0x562b3dbacf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.815+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbad480 0x562b3dbad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.815+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3db040f0 con 0x562b3dbb2950 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.815+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3db01ab0 con 0x562b3dbad480 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.815+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3dafd3a0 con 0x562b3dbb3270 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94cdbc5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 0x562b3dbacf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94ce3c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb2950 0x562b3dbb2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94cdbc5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 0x562b3dbacf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:58886/0 (socket says 172.21.15.12:58886) 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94ce3c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb2950 0x562b3dbb2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50588/0 (socket says 172.21.15.12:50588) 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94cdbc5700 1 -- 172.21.15.12:0/3877606143 learned_addr learned my addr 172.21.15.12:0/3877606143 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94cebc7700 1 --2- 172.21.15.12:0/3877606143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbad480 0x562b3dbad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94ce3c6700 1 -- 172.21.15.12:0/3877606143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 msgr2=0x562b3dbacf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/3877606143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 0x562b3dbacf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94ce3c6700 1 -- 172.21.15.12:0/3877606143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbad480 msgr2=0x562b3dbad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/3877606143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbad480 0x562b3dbad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94ce3c6700 1 -- 172.21.15.12:0/3877606143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b3d97e350 con 0x562b3dbb2950 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/3877606143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb2950 0x562b3dbb2d30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f94c40042f0 tx=0x7f94c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c12935c3eff448 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.816+0000 7f94bcff9700 1 -- 172.21.15.12:0/3877606143 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4003d10 con 0x562b3dbb2950 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.817+0000 7f94bcff9700 1 -- 172.21.15.12:0/3877606143 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94c4003e70 con 0x562b3dbb2950 2024-05-09T00:00:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.817+0000 7f94bcff9700 1 -- 172.21.15.12:0/3877606143 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c401e650 con 0x562b3dbb2950 2024-05-09T00:00:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.817+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/3877606143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb2950 msgr2=0x562b3dbb2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.817+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/3877606143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb2950 0x562b3dbb2d30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f94c40042f0 tx=0x7f94c4014000 comp rx=0 tx=0).stop 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.817+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/3877606143 shutdown_connections 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.817+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/3877606143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbad480 0x562b3dbad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.817+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/3877606143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb2950 0x562b3dbb2d30 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.817+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/3877606143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 0x562b3dbacf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.817+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/3877606143 >> 172.21.15.12:0/3877606143 conn(0x562b3db0b1b0 msgr2=0x562b3dba7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.818+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/3877606143 shutdown_connections 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.818+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/3877606143 wait complete. 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.818+0000 7f94dc7e4b80 1 -- start start 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.818+0000 7f94dc7e4b80 1 Processor -- start 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.818+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbb91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbb64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb3270 0x562b3dbb6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3da99500 con 0x562b3dbb3270 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3da991a0 con 0x562b3dbb2950 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3d980640 con 0x562b3dbad480 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94dc7e4b80 10 client.0 _ll_register_callbacks cb 0x562b3daa2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94ce3c6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbb91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94cdbc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbb64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94ce3c6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbb91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:58900/0 (socket says 172.21.15.12:58900) 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94ce3c6700 1 -- 172.21.15.12:0/4160255507 learned_addr learned my addr 172.21.15.12:0/4160255507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94cebc7700 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb3270 0x562b3dbb6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94ce3c6700 1 -- 172.21.15.12:0/4160255507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 msgr2=0x562b3dbb64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbb64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94ce3c6700 1 -- 172.21.15.12:0/4160255507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb3270 msgr2=0x562b3dbb6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb3270 0x562b3dbb6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94ce3c6700 1 -- 172.21.15.12:0/4160255507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c4017070 con 0x562b3dbad480 2024-05-09T00:00:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbb91d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f94c4003960 tx=0x7f94c40266a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd8bfe5feda5775e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.819+0000 7f94b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.820+0000 7f94b1ffb700 1 -- 172.21.15.12:0/4160255507 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c402a040 con 0x562b3dbad480 2024-05-09T00:00:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.820+0000 7f94b1ffb700 1 -- 172.21.15.12:0/4160255507 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f94c402f480 con 0x562b3dbad480 2024-05-09T00:00:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.820+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/4160255507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b3da56210 con 0x562b3dbad480 2024-05-09T00:00:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.823+0000 7f94b1ffb700 1 -- 172.21.15.12:0/4160255507 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94c4026ea0 con 0x562b3dbad480 2024-05-09T00:00:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94dc7e4b80 10 client.29164 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/4160255507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x562b3d92ab00 con 0x562b3dbad480 2024-05-09T00:00:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94b1ffb700 1 -- 172.21.15.12:0/4160255507 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94c403c020 con 0x562b3dbad480 2024-05-09T00:00:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94b1ffb700 1 -- 172.21.15.12:0/4160255507 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f94c4024020 con 0x562b3dbad480 2024-05-09T00:00:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94dc7e4b80 10 client.29164 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94dc7e4b80 10 client.29164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94dc7e4b80 20 client.29164 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3dbc6b70 0x562b3dbc8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94dc7e4b80 4 client.29164 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/4160255507 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b3db7c7b0 con 0x562b3dbc6b70 2024-05-09T00:00:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.824+0000 7f94cdbc5700 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3dbc6b70 0x562b3dbc8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.825+0000 7f94cdbc5700 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3dbc6b70 0x562b3dbc8f90 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e955b523ce7b483d server_cookie=e92722bbec19ffa3 in_seq=0 out_seq=0 2024-05-09T00:00:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.825+0000 7f94b1ffb700 10 client.29164 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94b1ffb700 1 -- 172.21.15.12:0/4160255507 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b3db7c7b0 con 0x562b3dbc6b70 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94b1ffb700 10 client.29164 handle_command_reply: tid=0 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 client.29164 shutdown 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/4160255507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 msgr2=0x562b3dbb91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbb91d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f94c4003960 tx=0x7f94c40266a0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/4160255507 shutdown_connections 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb3270 0x562b3dbb6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbb91d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3dbc6b70 0x562b3dbc8f90 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/4160255507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbb64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94b1ffb700 0 client.29164 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/4160255507 >> 172.21.15.12:0/4160255507 conn(0x562b3db0b1b0 msgr2=0x562b3dbabf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/4160255507 shutdown_connections 2024-05-09T00:00:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/4160255507 wait complete. 2024-05-09T00:00:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.827+0000 7f94dc7e4b80 20 client.29164 trim_cache size 0 max 16384 2024-05-09T00:00:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94dc7e4b80 1 Processor -- start 2024-05-09T00:00:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94dc7e4b80 1 -- start start 2024-05-09T00:00:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbad480 0x562b3dba7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 0x562b3dbb8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3dbadde0 con 0x562b3dbad480 2024-05-09T00:00:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3dbb73d0 con 0x562b3dbb2950 2024-05-09T00:00:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3dbb70f0 con 0x562b3dbb3270 2024-05-09T00:00:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94ce3c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbad480 0x562b3dba7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94ce3c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbad480 0x562b3dba7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50618/0 (socket says 172.21.15.12:50618) 2024-05-09T00:00:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94ce3c6700 1 -- 172.21.15.12:0/1143994220 learned_addr learned my addr 172.21.15.12:0/1143994220 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94cdbc5700 1 --2- 172.21.15.12:0/1143994220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94cebc7700 1 --2- 172.21.15.12:0/1143994220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 0x562b3dbb8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94cdbc5700 1 -- 172.21.15.12:0/1143994220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 msgr2=0x562b3dbb8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94cdbc5700 1 --2- 172.21.15.12:0/1143994220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 0x562b3dbb8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.846+0000 7f94cdbc5700 1 -- 172.21.15.12:0/1143994220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbad480 msgr2=0x562b3dba7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.847+0000 7f94cdbc5700 1 --2- 172.21.15.12:0/1143994220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbad480 0x562b3dba7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.847+0000 7f94cdbc5700 1 -- 172.21.15.12:0/1143994220 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b3dac3500 con 0x562b3dbb2950 2024-05-09T00:00:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.847+0000 7f94cdbc5700 1 --2- 172.21.15.12:0/1143994220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbad860 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94b80098e0 tx=0x7f94b8005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcf8cd6271089b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.847+0000 7f94bd7fa700 1 -- 172.21.15.12:0/1143994220 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b802a070 con 0x562b3dbb2950 2024-05-09T00:00:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.847+0000 7f94bd7fa700 1 -- 172.21.15.12:0/1143994220 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94b801e040 con 0x562b3dbb2950 2024-05-09T00:00:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.847+0000 7f94bd7fa700 1 -- 172.21.15.12:0/1143994220 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b801d040 con 0x562b3dbb2950 2024-05-09T00:00:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.848+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/1143994220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 msgr2=0x562b3dbad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.848+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/1143994220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbad860 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94b80098e0 tx=0x7f94b8005960 comp rx=0 tx=0).stop 2024-05-09T00:00:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.848+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/1143994220 shutdown_connections 2024-05-09T00:00:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.848+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/1143994220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbb3270 0x562b3dbb8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.848+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/1143994220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbad480 0x562b3dba7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.848+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/1143994220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbad860 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.848+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/1143994220 >> 172.21.15.12:0/1143994220 conn(0x562b3db0b1b0 msgr2=0x562b3dbb4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.848+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/1143994220 shutdown_connections 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.848+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/1143994220 wait complete. 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.848+0000 7f94dc7e4b80 1 -- start start 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94dc7e4b80 1 Processor -- start 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbbbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbbc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94dc7e4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb3270 0x562b3dc24840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3dc24d80 con 0x562b3dbb3270 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3dc24ec0 con 0x562b3dbb2950 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94dc7e4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3dc25030 con 0x562b3dbad480 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94dc7e4b80 10 client.0 _ll_register_callbacks cb 0x562b3daa2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94cdbc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbbc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94ce3c6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbbbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94cdbc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbbc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39476/0 (socket says 172.21.15.12:39476) 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94ce3c6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbbbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:58914/0 (socket says 172.21.15.12:58914) 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.849+0000 7f94cdbc5700 1 -- 172.21.15.12:0/2989099299 learned_addr learned my addr 172.21.15.12:0/2989099299 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94cebc7700 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb3270 0x562b3dc24840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94ce3c6700 1 -- 172.21.15.12:0/2989099299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 msgr2=0x562b3dbbc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbbc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94ce3c6700 1 -- 172.21.15.12:0/2989099299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb3270 msgr2=0x562b3dc24840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb3270 0x562b3dc24840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94ce3c6700 1 -- 172.21.15.12:0/2989099299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c8110810 con 0x562b3dbad480 2024-05-09T00:00:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94ce3c6700 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbbbee0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f94c4003960 tx=0x7f94c4026cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=737378c676303b8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94b1ffb700 1 -- 172.21.15.12:0/2989099299 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c402a040 con 0x562b3dbad480 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94b1ffb700 1 -- 172.21.15.12:0/2989099299 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f94c403d080 con 0x562b3dbad480 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.850+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/2989099299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94c4003e70 con 0x562b3dbad480 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94b1ffb700 1 -- 172.21.15.12:0/2989099299 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94c4026710 con 0x562b3dbad480 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94dc7e4b80 10 client.29170 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/2989099299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x562b3d982c70 con 0x562b3dbad480 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94b1ffb700 1 -- 172.21.15.12:0/2989099299 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94c4003e70 con 0x562b3dbad480 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94b1ffb700 1 -- 172.21.15.12:0/2989099299 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f94c4003680 con 0x562b3dbad480 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94dc7e4b80 10 client.29170 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94dc7e4b80 10 client.29170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94dc7e4b80 20 client.29170 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3dc26400 0x562b3dc287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94dc7e4b80 4 client.29170 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:00:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/2989099299 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b3daf5e60 con 0x562b3dc26400 2024-05-09T00:00:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.851+0000 7f94cdbc5700 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3dc26400 0x562b3dc287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.852+0000 7f94b1ffb700 10 client.29170 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:23.852+0000 7f94cdbc5700 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3dc26400 0x562b3dc287f0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=291f023ac11b41e3 server_cookie=8604e2df3d840679 in_seq=0 out_seq=0 2024-05-09T00:00:24.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:23 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:00:24.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:23 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:00:24.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:23 smithi012 ceph-mon[42503]: pgmap v448: 97 pgs: 97 active+clean; 251 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-05-09T00:00:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94b1ffb700 1 -- 172.21.15.12:0/2989099299 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b3daf5e60 con 0x562b3dc26400 2024-05-09T00:00:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94b1ffb700 10 client.29170 handle_command_reply: tid=0 2024-05-09T00:00:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 client.29170 shutdown 2024-05-09T00:00:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/2989099299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 msgr2=0x562b3dbbbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbbbee0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f94c4003960 tx=0x7f94c4026cd0 comp rx=0 tx=0).stop 2024-05-09T00:00:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/2989099299 shutdown_connections 2024-05-09T00:00:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b3dbb3270 0x562b3dc24840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b3dbad480 0x562b3dbbbee0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562b3dc26400 0x562b3dc287f0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 --2- 172.21.15.12:0/2989099299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3dbb2950 0x562b3dbbc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94b1ffb700 0 client.29170 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/2989099299 >> 172.21.15.12:0/2989099299 conn(0x562b3db0b1b0 msgr2=0x562b3dbb0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/2989099299 shutdown_connections 2024-05-09T00:00:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 1 -- 172.21.15.12:0/2989099299 wait complete. 2024-05-09T00:00:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.205+0000 7f94dc7e4b80 20 client.29170 trim_cache size 0 max 16384 2024-05-09T00:00:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:00:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:00:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:00:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:00:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 -- 172.21.15.12:0/2868373442 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f94b40a1290 msgr2=0x7f94b40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f94b40a1290 0x7f94b40a3750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f94c4011490 tx=0x7f94c4009260 comp rx=0 tx=0).stop 2024-05-09T00:00:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 -- 172.21.15.12:0/2868373442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c810d8f0 msgr2=0x7f94c819de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c810d8f0 0x7f94c819de50 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f94c0007ec0 tx=0x7f94c000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:00:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 -- 172.21.15.12:0/2868373442 shutdown_connections 2024-05-09T00:00:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f94b40a1290 0x7f94b40a3750 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94c810d8f0 0x7f94c819de50 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94c80691b0 0x7f94c8199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 --2- 172.21.15.12:0/2868373442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94c8069ad0 0x7f94c8199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 -- 172.21.15.12:0/2868373442 >> 172.21.15.12:0/2868373442 conn(0x7f94c8076f40 msgr2=0x7f94c810f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 -- 172.21.15.12:0/2868373442 shutdown_connections 2024-05-09T00:00:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:24.206+0000 7f94d062a700 1 -- 172.21.15.12:0/2868373442 wait complete. 2024-05-09T00:00:24.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 9463 36.22 MB/sec warmup 19 sec latency 211.758 ms 2024-05-09T00:00:24.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:23 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:00:24.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:23 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:00:24.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:23 smithi204 ceph-mon[47373]: pgmap v448: 97 pgs: 97 active+clean; 251 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-05-09T00:00:24.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:23 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:00:24.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:23 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:00:24.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:23 smithi160 ceph-mon[46152]: pgmap v448: 97 pgs: 97 active+clean; 251 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-05-09T00:00:25.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:00:25.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:00:25.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10173 36.44 MB/sec warmup 20 sec latency 52.257 ms 2024-05-09T00:00:25.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:00:25.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:00:25.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:00:25.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:00:26.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:25 smithi012 ceph-mon[42503]: pgmap v449: 97 pgs: 97 active+clean; 251 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-05-09T00:00:26.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 11079 36.50 MB/sec warmup 21 sec latency 88.899 ms 2024-05-09T00:00:26.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:25 smithi204 ceph-mon[47373]: pgmap v449: 97 pgs: 97 active+clean; 251 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-05-09T00:00:26.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:25 smithi160 ceph-mon[46152]: pgmap v449: 97 pgs: 97 active+clean; 251 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-05-09T00:00:27.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 12410 37.96 MB/sec warmup 22 sec latency 82.968 ms 2024-05-09T00:00:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:27 smithi012 ceph-mon[42503]: pgmap v450: 97 pgs: 97 active+clean; 255 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 734 op/s 2024-05-09T00:00:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:27 smithi204 ceph-mon[47373]: pgmap v450: 97 pgs: 97 active+clean; 255 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 734 op/s 2024-05-09T00:00:28.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:27 smithi160 ceph-mon[46152]: pgmap v450: 97 pgs: 97 active+clean; 255 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 734 op/s 2024-05-09T00:00:28.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 13482 39.72 MB/sec warmup 23 sec latency 42.199 ms 2024-05-09T00:00:29.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 14104 38.89 MB/sec warmup 24 sec latency 90.354 ms 2024-05-09T00:00:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:29 smithi012 ceph-mon[42503]: pgmap v451: 97 pgs: 97 active+clean; 255 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 557 op/s 2024-05-09T00:00:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:29 smithi204 ceph-mon[47373]: pgmap v451: 97 pgs: 97 active+clean; 255 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 557 op/s 2024-05-09T00:00:30.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:29 smithi160 ceph-mon[46152]: pgmap v451: 97 pgs: 97 active+clean; 255 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 557 op/s 2024-05-09T00:00:30.215 INFO:tasks.workunit.client.0.smithi012.stdout: 10 15181 39.53 MB/sec warmup 25 sec latency 55.612 ms 2024-05-09T00:00:31.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 16328 40.50 MB/sec warmup 26 sec latency 36.971 ms 2024-05-09T00:00:31.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:31 smithi012 ceph-mon[42503]: pgmap v452: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.1 MiB/s wr, 805 op/s 2024-05-09T00:00:32.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:31 smithi204 ceph-mon[47373]: pgmap v452: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.1 MiB/s wr, 805 op/s 2024-05-09T00:00:32.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:31 smithi160 ceph-mon[46152]: pgmap v452: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.1 MiB/s wr, 805 op/s 2024-05-09T00:00:32.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 17181 40.75 MB/sec warmup 27 sec latency 44.598 ms 2024-05-09T00:00:33.214 INFO:tasks.workunit.client.0.smithi012.stdout: 10 18172 41.20 MB/sec warmup 28 sec latency 51.209 ms 2024-05-09T00:00:33.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:33 smithi012 ceph-mon[42503]: pgmap v453: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-05-09T00:00:34.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:33 smithi204 ceph-mon[47373]: pgmap v453: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-05-09T00:00:34.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:33 smithi160 ceph-mon[46152]: pgmap v453: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-05-09T00:00:34.215 INFO:tasks.workunit.client.0.smithi012.stdout: 10 18579 40.52 MB/sec warmup 29 sec latency 560.106 ms 2024-05-09T00:00:35.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 18676 39.28 MB/sec warmup 30 sec latency 1026.912 ms 2024-05-09T00:00:35.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:35 smithi012 ceph-mon[42503]: pgmap v454: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-05-09T00:00:36.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:35 smithi204 ceph-mon[47373]: pgmap v454: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-05-09T00:00:36.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:35 smithi160 ceph-mon[46152]: pgmap v454: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-05-09T00:00:36.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 19151 38.81 MB/sec warmup 31 sec latency 1475.866 ms 2024-05-09T00:00:37.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 19946 38.95 MB/sec warmup 32 sec latency 68.890 ms 2024-05-09T00:00:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.375+0000 7f18ca745700 1 -- 172.21.15.12:0/1256665318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c4106230 msgr2=0x7f18c410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.375+0000 7f18ca745700 1 --2- 172.21.15.12:0/1256665318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c4106230 0x7f18c410ddb0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00b0a0 tx=0x7f18bc00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.375+0000 7f18ca745700 1 -- 172.21.15.12:0/1256665318 shutdown_connections 2024-05-09T00:00:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.375+0000 7f18ca745700 1 --2- 172.21.15.12:0/1256665318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c4106230 0x7f18c410ddb0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.375+0000 7f18ca745700 1 --2- 172.21.15.12:0/1256665318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18c4101110 0x7f18c41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.375+0000 7f18ca745700 1 --2- 172.21.15.12:0/1256665318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18c4101ac0 0x7f18c4105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.375+0000 7f18ca745700 1 -- 172.21.15.12:0/1256665318 >> 172.21.15.12:0/1256665318 conn(0x7f18c40fc950 msgr2=0x7f18c40fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.375+0000 7f18ca745700 1 -- 172.21.15.12:0/1256665318 shutdown_connections 2024-05-09T00:00:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.375+0000 7f18ca745700 1 -- 172.21.15.12:0/1256665318 wait complete. 2024-05-09T00:00:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.376+0000 7f18ca745700 1 Processor -- start 2024-05-09T00:00:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.376+0000 7f18ca745700 1 -- start start 2024-05-09T00:00:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18ca745700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18c4101110 0x7f18c4197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18ca745700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c4106230 0x7f18c4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18ca745700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18c419be50 0x7f18c419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18ca745700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419c8d0 con 0x7f18c4106230 2024-05-09T00:00:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18ca745700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419ca40 con 0x7f18c4101110 2024-05-09T00:00:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18ca745700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419cc00 con 0x7f18c419be50 2024-05-09T00:00:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18c37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c4106230 0x7f18c4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18c3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18c4101110 0x7f18c4197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18c8ce2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18c419be50 0x7f18c419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18c37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c4106230 0x7f18c4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55636/0 (socket says 172.21.15.12:55636) 2024-05-09T00:00:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.377+0000 7f18c37fe700 1 -- 172.21.15.12:0/340824875 learned_addr learned my addr 172.21.15.12:0/340824875 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18c37fe700 1 -- 172.21.15.12:0/340824875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18c419be50 msgr2=0x7f18c419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18c37fe700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18c419be50 0x7f18c419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18c37fe700 1 -- 172.21.15.12:0/340824875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18c4101110 msgr2=0x7f18c4197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18c37fe700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18c4101110 0x7f18c4197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18c37fe700 1 -- 172.21.15.12:0/340824875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc009d00 con 0x7f18c4106230 2024-05-09T00:00:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18c3fff700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18c4101110 0x7f18c4197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18c37fe700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c4106230 0x7f18c4197ab0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f18b400b500 tx=0x7f18b400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18c17fa700 1 -- 172.21.15.12:0/340824875 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4011830 con 0x7f18c4106230 2024-05-09T00:00:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18c17fa700 1 -- 172.21.15.12:0/340824875 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18b4011e50 con 0x7f18c4106230 2024-05-09T00:00:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18ca745700 1 -- 172.21.15.12:0/340824875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c4073230 con 0x7f18c4106230 2024-05-09T00:00:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18c17fa700 1 -- 172.21.15.12:0/340824875 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4010de0 con 0x7f18c4106230 2024-05-09T00:00:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.378+0000 7f18ca745700 1 -- 172.21.15.12:0/340824875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c4073790 con 0x7f18c4106230 2024-05-09T00:00:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.381+0000 7f18c17fa700 1 -- 172.21.15.12:0/340824875 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f18b4010480 con 0x7f18c4106230 2024-05-09T00:00:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.382+0000 7f18c17fa700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18ac0a1310 0x7f18ac0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.382+0000 7f18c17fa700 1 -- 172.21.15.12:0/340824875 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f18b40cc590 con 0x7f18c4106230 2024-05-09T00:00:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.382+0000 7f18c3fff700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18ac0a1310 0x7f18ac0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.383+0000 7f18c3fff700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18ac0a1310 0x7f18ac0a37d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f18b8011490 tx=0x7f18b8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.383+0000 7f18d68ffb80 1 Processor -- start 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.383+0000 7f18d68ffb80 1 -- start start 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcb950 0x55a485bcbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bcc270 0x55a485bc5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bc6480 0x55a485bc68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485b1d0f0 con 0x55a485bcb950 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485b1aab0 con 0x55a485bcc270 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485b163a0 con 0x55a485bc6480 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcb950 0x55a485bcbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcb950 0x55a485bcbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55638/0 (socket says 172.21.15.12:55638) 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18c3fff700 1 -- 172.21.15.12:0/560706328 learned_addr learned my addr 172.21.15.12:0/560706328 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18c8ce2700 1 --2- 172.21.15.12:0/560706328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bc6480 0x55a485bc68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18c3fff700 1 -- 172.21.15.12:0/560706328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bc6480 msgr2=0x55a485bc68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18c3fff700 1 --2- 172.21.15.12:0/560706328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bc6480 0x55a485bc68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18c3fff700 1 -- 172.21.15.12:0/560706328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bcc270 msgr2=0x55a485bc5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18c3fff700 1 --2- 172.21.15.12:0/560706328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bcc270 0x55a485bc5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.384+0000 7f18c3fff700 1 -- 172.21.15.12:0/560706328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a485997350 con 0x55a485bcb950 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.385+0000 7f18c3fff700 1 --2- 172.21.15.12:0/560706328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcb950 0x55a485bcbd30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f18b80043c0 tx=0x7f18b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93dfd48648aaf5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.385+0000 7f18aa7fc700 1 -- 172.21.15.12:0/560706328 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8003d10 con 0x55a485bcb950 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.385+0000 7f18aa7fc700 1 -- 172.21.15.12:0/560706328 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18b8003e70 con 0x55a485bcb950 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.385+0000 7f18aa7fc700 1 -- 172.21.15.12:0/560706328 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b800e7a0 con 0x55a485bcb950 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.386+0000 7f18d68ffb80 1 -- 172.21.15.12:0/560706328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcb950 msgr2=0x55a485bcbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.386+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/560706328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcb950 0x55a485bcbd30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f18b80043c0 tx=0x7f18b8014000 comp rx=0 tx=0).stop 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.386+0000 7f18d68ffb80 1 -- 172.21.15.12:0/560706328 shutdown_connections 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.386+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/560706328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bc6480 0x55a485bc68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.386+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/560706328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcb950 0x55a485bcbd30 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.386+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/560706328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bcc270 0x55a485bc5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.386+0000 7f18d68ffb80 1 -- 172.21.15.12:0/560706328 >> 172.21.15.12:0/560706328 conn(0x55a485b241b0 msgr2=0x55a485bc0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.386+0000 7f18d68ffb80 1 -- 172.21.15.12:0/560706328 shutdown_connections 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.386+0000 7f18d68ffb80 1 -- 172.21.15.12:0/560706328 wait complete. 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.387+0000 7f18d68ffb80 1 -- start start 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.387+0000 7f18d68ffb80 1 Processor -- start 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.388+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 0x55a485bcf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.388+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bcfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.388+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bd0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.388+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485ab2500 con 0x55a485bcc270 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.388+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485ab21a0 con 0x55a485bc6480 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.388+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485999640 con 0x55a485bcb950 2024-05-09T00:00:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.388+0000 7f18d68ffb80 10 client.0 _ll_register_callbacks cb 0x55a485abbad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.388+0000 7f18c8ce2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bd0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.388+0000 7f18c8ce2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bd0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55640/0 (socket says 172.21.15.12:55640) 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.388+0000 7f18c8ce2700 1 -- 172.21.15.12:0/4014996977 learned_addr learned my addr 172.21.15.12:0/4014996977 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.389+0000 7f18c8ce2700 1 -- 172.21.15.12:0/4014996977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 msgr2=0x55a485bcfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.389+0000 7f18c8ce2700 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bcfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.389+0000 7f18c8ce2700 1 -- 172.21.15.12:0/4014996977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 msgr2=0x55a485bcf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.389+0000 7f18c8ce2700 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 0x55a485bcf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.389+0000 7f18c8ce2700 1 -- 172.21.15.12:0/4014996977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b8017070 con 0x55a485bcc270 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.389+0000 7f18c8ce2700 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bd0050 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00b0a0 tx=0x7f18bc00bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a825dcc5fc73a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.389+0000 7f18a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.389+0000 7f18a37fe700 1 -- 172.21.15.12:0/4014996977 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00e040 con 0x55a485bcc270 2024-05-09T00:00:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.389+0000 7f18a37fe700 1 -- 172.21.15.12:0/4014996977 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f18bc003a00 con 0x55a485bcc270 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.389+0000 7f18d68ffb80 1 -- 172.21.15.12:0/4014996977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a485a6f210 con 0x55a485bcc270 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.390+0000 7f18a37fe700 1 -- 172.21.15.12:0/4014996977 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18bc0078f0 con 0x55a485bcc270 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.390+0000 7f18d68ffb80 10 client.19458 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.390+0000 7f18d68ffb80 1 -- 172.21.15.12:0/4014996977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55a485943b00 con 0x55a485bcc270 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.390+0000 7f18a37fe700 1 -- 172.21.15.12:0/4014996977 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a485a6f210 con 0x55a485bcc270 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.391+0000 7f18a37fe700 1 -- 172.21.15.12:0/4014996977 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f18bc00b0d0 con 0x55a485bcc270 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.391+0000 7f18d68ffb80 10 client.19458 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.391+0000 7f18d68ffb80 10 client.19458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.391+0000 7f18d68ffb80 20 client.19458 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.391+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a485be84b0 0x55a485bea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.391+0000 7f18d68ffb80 4 client.19458 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.391+0000 7f18d68ffb80 1 -- 172.21.15.12:0/4014996977 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a485b957b0 con 0x55a485be84b0 2024-05-09T00:00:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.392+0000 7f18c3fff700 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a485be84b0 0x55a485bea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.394+0000 7f18c3fff700 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a485be84b0 0x55a485bea8d0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6318d060580a467 server_cookie=e7c1fa18411052cc in_seq=0 out_seq=0 2024-05-09T00:00:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.394+0000 7f18a37fe700 10 client.19458 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18a37fe700 1 -- 172.21.15.12:0/4014996977 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a485b957b0 con 0x55a485be84b0 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18a37fe700 10 client.19458 handle_command_reply: tid=0 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 client.19458 shutdown 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 -- 172.21.15.12:0/4014996977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 msgr2=0x55a485bd0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bd0050 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00b0a0 tx=0x7f18bc00bd10 comp rx=0 tx=0).stop 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 -- 172.21.15.12:0/4014996977 shutdown_connections 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bd0050 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a485be84b0 0x55a485bea8d0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 0x55a485bcf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/4014996977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bcfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 -- 172.21.15.12:0/4014996977 >> 172.21.15.12:0/4014996977 conn(0x55a485b241b0 msgr2=0x55a485bca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18a37fe700 0 client.19458 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 -- 172.21.15.12:0/4014996977 shutdown_connections 2024-05-09T00:00:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.398+0000 7f18d68ffb80 1 -- 172.21.15.12:0/4014996977 wait complete. 2024-05-09T00:00:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.399+0000 7f18d68ffb80 20 client.19458 trim_cache size 0 max 16384 2024-05-09T00:00:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.421+0000 7f18d68ffb80 1 Processor -- start 2024-05-09T00:00:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18d68ffb80 1 -- start start 2024-05-09T00:00:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 0x55a485be3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bc0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bc6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485bd2180 con 0x55a485bcc270 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485bc6da0 con 0x55a485bc6480 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485bd25b0 con 0x55a485bcb950 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18c37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bc0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18c3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 0x55a485be3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18c37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bc0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50152/0 (socket says 172.21.15.12:50152) 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18c3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 0x55a485be3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:55094/0 (socket says 172.21.15.12:55094) 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.422+0000 7f18c37fe700 1 -- 172.21.15.12:0/2424363354 learned_addr learned my addr 172.21.15.12:0/2424363354 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18c8ce2700 1 --2- 172.21.15.12:0/2424363354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bc6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18c37fe700 1 -- 172.21.15.12:0/2424363354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 msgr2=0x55a485be3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18c37fe700 1 --2- 172.21.15.12:0/2424363354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 0x55a485be3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18c37fe700 1 -- 172.21.15.12:0/2424363354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 msgr2=0x55a485bc6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18c37fe700 1 --2- 172.21.15.12:0/2424363354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bc6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18c37fe700 1 -- 172.21.15.12:0/2424363354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a485adc500 con 0x55a485bcb950 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18c3fff700 1 --2- 172.21.15.12:0/2424363354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 0x55a485be3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18c8ce2700 1 --2- 172.21.15.12:0/2424363354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bc6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18c37fe700 1 --2- 172.21.15.12:0/2424363354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bc0910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f18b40c9eb0 tx=0x7f18b4095b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=500e635180fcc2c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18aaffd700 1 -- 172.21.15.12:0/2424363354 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4099120 con 0x55a485bcb950 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18aaffd700 1 -- 172.21.15.12:0/2424363354 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18b4099ec0 con 0x55a485bcb950 2024-05-09T00:00:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.423+0000 7f18aaffd700 1 -- 172.21.15.12:0/2424363354 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4099c80 con 0x55a485bcb950 2024-05-09T00:00:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.424+0000 7f18d68ffb80 1 -- 172.21.15.12:0/2424363354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 msgr2=0x55a485bc0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.424+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/2424363354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bc0910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f18b40c9eb0 tx=0x7f18b4095b80 comp rx=0 tx=0).stop 2024-05-09T00:00:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.424+0000 7f18d68ffb80 1 -- 172.21.15.12:0/2424363354 shutdown_connections 2024-05-09T00:00:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.424+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/2424363354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bcc270 0x55a485bc6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.424+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/2424363354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bc6480 0x55a485be3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.424+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/2424363354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bc0910 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.424+0000 7f18d68ffb80 1 -- 172.21.15.12:0/2424363354 >> 172.21.15.12:0/2424363354 conn(0x55a485b241b0 msgr2=0x55a485bc4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.424+0000 7f18d68ffb80 1 -- 172.21.15.12:0/2424363354 shutdown_connections 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.424+0000 7f18d68ffb80 1 -- 172.21.15.12:0/2424363354 wait complete. 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.425+0000 7f18d68ffb80 1 -- start start 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.425+0000 7f18d68ffb80 1 Processor -- start 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.425+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bc6480 0x55a485bd00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.425+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.425+0000 7f18d68ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bcc270 0x55a485bcf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.425+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485bcf750 con 0x55a485bc6480 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.425+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485bcf8c0 con 0x55a485bcc270 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.425+0000 7f18d68ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a485bdec60 con 0x55a485bcb950 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.425+0000 7f18d68ffb80 10 client.0 _ll_register_callbacks cb 0x55a485abbad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.426+0000 7f18c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bc6480 0x55a485bd00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.426+0000 7f18c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bc6480 0x55a485bd00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55666/0 (socket says 172.21.15.12:55666) 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.426+0000 7f18c3fff700 1 -- 172.21.15.12:0/1419375403 learned_addr learned my addr 172.21.15.12:0/1419375403 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.426+0000 7f18c37fe700 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.426+0000 7f18c8ce2700 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bcc270 0x55a485bcf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.426+0000 7f18c3fff700 1 -- 172.21.15.12:0/1419375403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 msgr2=0x55a485bceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.426+0000 7f18c3fff700 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.426+0000 7f18c3fff700 1 -- 172.21.15.12:0/1419375403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bcc270 msgr2=0x55a485bcf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.426+0000 7f18c3fff700 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bcc270 0x55a485bcf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.426+0000 7f18c3fff700 1 -- 172.21.15.12:0/1419375403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a485997350 con 0x55a485bc6480 2024-05-09T00:00:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.427+0000 7f18c3fff700 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bc6480 0x55a485bd00b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f18b8005950 tx=0x7f18b8009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc57dc0b6729887 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.427+0000 7f18a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18a37fe700 1 -- 172.21.15.12:0/1419375403 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b802c040 con 0x55a485bc6480 2024-05-09T00:00:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18a37fe700 1 -- 172.21.15.12:0/1419375403 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f18b8030530 con 0x55a485bc6480 2024-05-09T00:00:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18d68ffb80 1 -- 172.21.15.12:0/1419375403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18b8003e70 con 0x55a485bc6480 2024-05-09T00:00:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18a37fe700 1 -- 172.21.15.12:0/1419375403 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18b8030a90 con 0x55a485bc6480 2024-05-09T00:00:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18d68ffb80 10 client.19461 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18d68ffb80 1 -- 172.21.15.12:0/1419375403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f18c4073790 con 0x55a485bc6480 2024-05-09T00:00:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18a37fe700 1 -- 172.21.15.12:0/1419375403 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18b803f080 con 0x55a485bc6480 2024-05-09T00:00:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18a37fe700 1 -- 172.21.15.12:0/1419375403 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f18b8024020 con 0x55a485bc6480 2024-05-09T00:00:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18d68ffb80 10 client.19461 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18d68ffb80 10 client.19461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18d68ffb80 20 client.19461 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a485c47c90 0x55a485c4a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18d68ffb80 4 client.19461 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:00:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.428+0000 7f18d68ffb80 1 -- 172.21.15.12:0/1419375403 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a485b0ee60 con 0x55a485c47c90 2024-05-09T00:00:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.429+0000 7f18c37fe700 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a485c47c90 0x55a485c4a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.429+0000 7f18c37fe700 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a485c47c90 0x55a485c4a0b0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=134f23781406fe25 server_cookie=d1ad60db54b1b984 in_seq=0 out_seq=0 2024-05-09T00:00:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.429+0000 7f18a37fe700 10 client.19461 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18a37fe700 1 -- 172.21.15.12:0/1419375403 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a485b0ee60 con 0x55a485c47c90 2024-05-09T00:00:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18a37fe700 10 client.19461 handle_command_reply: tid=0 2024-05-09T00:00:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18d68ffb80 1 client.19461 shutdown 2024-05-09T00:00:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18d68ffb80 1 -- 172.21.15.12:0/1419375403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bc6480 msgr2=0x55a485bd00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bc6480 0x55a485bd00b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f18b8005950 tx=0x7f18b8009770 comp rx=0 tx=0).stop 2024-05-09T00:00:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18d68ffb80 1 -- 172.21.15.12:0/1419375403 shutdown_connections 2024-05-09T00:00:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a485c47c90 0x55a485c4a0b0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a485bcc270 0x55a485bcf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a485bc6480 0x55a485bd00b0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18a37fe700 0 client.19461 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18d68ffb80 1 --2- 172.21.15.12:0/1419375403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a485bcb950 0x55a485bceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.618+0000 7f18d68ffb80 1 -- 172.21.15.12:0/1419375403 >> 172.21.15.12:0/1419375403 conn(0x55a485b241b0 msgr2=0x55a485bcdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:00:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:00:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:00:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:00:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.619+0000 7f18d68ffb80 1 -- 172.21.15.12:0/1419375403 shutdown_connections 2024-05-09T00:00:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.619+0000 7f18d68ffb80 1 -- 172.21.15.12:0/1419375403 wait complete. 2024-05-09T00:00:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.619+0000 7f18d68ffb80 20 client.19461 trim_cache size 0 max 16384 2024-05-09T00:00:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.619+0000 7f18ca745700 1 -- 172.21.15.12:0/340824875 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18ac0a1310 msgr2=0x7f18ac0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.619+0000 7f18ca745700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18ac0a1310 0x7f18ac0a37d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f18b8011490 tx=0x7f18b8009260 comp rx=0 tx=0).stop 2024-05-09T00:00:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.619+0000 7f18ca745700 1 -- 172.21.15.12:0/340824875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c4106230 msgr2=0x7f18c4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.619+0000 7f18ca745700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c4106230 0x7f18c4197ab0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f18b400b500 tx=0x7f18b400b810 comp rx=0 tx=0).stop 2024-05-09T00:00:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.620+0000 7f18ca745700 1 -- 172.21.15.12:0/340824875 shutdown_connections 2024-05-09T00:00:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.620+0000 7f18ca745700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f18ac0a1310 0x7f18ac0a37d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.620+0000 7f18ca745700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18c419be50 0x7f18c419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.620+0000 7f18ca745700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18c4101110 0x7f18c4197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.620+0000 7f18ca745700 1 --2- 172.21.15.12:0/340824875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18c4106230 0x7f18c4197ab0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.620+0000 7f18ca745700 1 -- 172.21.15.12:0/340824875 >> 172.21.15.12:0/340824875 conn(0x7f18c40fc950 msgr2=0x7f18c40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.621+0000 7f18ca745700 1 -- 172.21.15.12:0/340824875 shutdown_connections 2024-05-09T00:00:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:37.621+0000 7f18ca745700 1 -- 172.21.15.12:0/340824875 wait complete. 2024-05-09T00:00:37.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:37 smithi012 ceph-mon[42503]: pgmap v455: 97 pgs: 97 active+clean; 576 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 37 MiB/s wr, 1.16k op/s 2024-05-09T00:00:38.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:37 smithi204 ceph-mon[47373]: pgmap v455: 97 pgs: 97 active+clean; 576 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 37 MiB/s wr, 1.16k op/s 2024-05-09T00:00:38.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:37 smithi160 ceph-mon[46152]: pgmap v455: 97 pgs: 97 active+clean; 576 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 37 MiB/s wr, 1.16k op/s 2024-05-09T00:00:38.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 20767 39.00 MB/sec warmup 33 sec latency 81.647 ms 2024-05-09T00:00:39.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 21774 39.58 MB/sec warmup 34 sec latency 50.427 ms 2024-05-09T00:00:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:39 smithi012 ceph-mon[42503]: pgmap v456: 97 pgs: 97 active+clean; 576 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 34 MiB/s wr, 758 op/s 2024-05-09T00:00:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:39 smithi204 ceph-mon[47373]: pgmap v456: 97 pgs: 97 active+clean; 576 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 34 MiB/s wr, 758 op/s 2024-05-09T00:00:40.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:39 smithi160 ceph-mon[46152]: pgmap v456: 97 pgs: 97 active+clean; 576 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 34 MiB/s wr, 758 op/s 2024-05-09T00:00:40.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 22687 40.03 MB/sec warmup 35 sec latency 35.738 ms 2024-05-09T00:00:40.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:00:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:00:41.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:40 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:00:41.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 23339 39.80 MB/sec warmup 36 sec latency 67.205 ms 2024-05-09T00:00:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:41 smithi012 ceph-mon[42503]: pgmap v457: 97 pgs: 97 active+clean; 490 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 36 MiB/s wr, 1.16k op/s 2024-05-09T00:00:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:00:42.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:00:42.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.221 INFO:tasks.workunit.client.0.smithi012.stdout: 10 24276 39.82 MB/sec warmup 37 sec latency 43.325 ms 2024-05-09T00:00:42.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:41 smithi204 ceph-mon[47373]: pgmap v457: 97 pgs: 97 active+clean; 490 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 36 MiB/s wr, 1.16k op/s 2024-05-09T00:00:42.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:00:42.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:00:42.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:42.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:41 smithi160 ceph-mon[46152]: pgmap v457: 97 pgs: 97 active+clean; 490 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 36 MiB/s wr, 1.16k op/s 2024-05-09T00:00:42.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:00:42.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:00:42.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:00:43.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 25514 40.67 MB/sec warmup 38 sec latency 63.664 ms 2024-05-09T00:00:43.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:43 smithi012 ceph-mon[42503]: pgmap v458: 97 pgs: 97 active+clean; 490 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 33 MiB/s wr, 916 op/s 2024-05-09T00:00:44.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:43 smithi204 ceph-mon[47373]: pgmap v458: 97 pgs: 97 active+clean; 490 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 33 MiB/s wr, 916 op/s 2024-05-09T00:00:44.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:43 smithi160 ceph-mon[46152]: pgmap v458: 97 pgs: 97 active+clean; 490 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 33 MiB/s wr, 916 op/s 2024-05-09T00:00:44.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 26442 41.06 MB/sec warmup 39 sec latency 63.307 ms 2024-05-09T00:00:45.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27348 41.09 MB/sec warmup 40 sec latency 42.080 ms 2024-05-09T00:00:45.690 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:00:45.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.866+0000 7f3a98999700 1 -- 172.21.15.12:0/27960275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a94100060 msgr2=0x7f3a94100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.866+0000 7f3a98999700 1 --2- 172.21.15.12:0/27960275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a94100060 0x7f3a94100440 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009990 tx=0x7f3a84009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.866+0000 7f3a98999700 1 -- 172.21.15.12:0/27960275 shutdown_connections 2024-05-09T00:00:45.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.866+0000 7f3a98999700 1 --2- 172.21.15.12:0/27960275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a94105180 0x7f3a9410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.866+0000 7f3a98999700 1 --2- 172.21.15.12:0/27960275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a94100060 0x7f3a94100440 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.866+0000 7f3a98999700 1 --2- 172.21.15.12:0/27960275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a94100a10 0x7f3a94104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.866+0000 7f3a98999700 1 -- 172.21.15.12:0/27960275 >> 172.21.15.12:0/27960275 conn(0x7f3a94076f10 msgr2=0x7f3a94077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:45.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.867+0000 7f3a98999700 1 -- 172.21.15.12:0/27960275 shutdown_connections 2024-05-09T00:00:45.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.867+0000 7f3a98999700 1 -- 172.21.15.12:0/27960275 wait complete. 2024-05-09T00:00:45.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a98999700 1 Processor -- start 2024-05-09T00:00:45.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a98999700 1 -- start start 2024-05-09T00:00:45.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a98999700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a94100060 0x7f3a94106d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a98999700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a94105180 0x7f3a94107260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a98999700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a941077a0 0x7f3a941a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a98999700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9410af10 con 0x7f3a94100060 2024-05-09T00:00:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a98999700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a941aa120 con 0x7f3a941077a0 2024-05-09T00:00:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a98999700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a941aa2e0 con 0x7f3a94105180 2024-05-09T00:00:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a91d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a94105180 0x7f3a94107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a91d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a94105180 0x7f3a94107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33406/0 (socket says 172.21.15.12:33406) 2024-05-09T00:00:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a91d9b700 1 -- 172.21.15.12:0/3960886299 learned_addr learned my addr 172.21.15.12:0/3960886299 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.868+0000 7f3a9259c700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a94100060 0x7f3a94106d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a91d9b700 1 -- 172.21.15.12:0/3960886299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a941077a0 msgr2=0x7f3a941a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a941077a0 0x7f3a941a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a91d9b700 1 -- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a94100060 msgr2=0x7f3a94106d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a94100060 0x7f3a94106d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a91d9b700 1 -- 172.21.15.12:0/3960886299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a840095f0 con 0x7f3a94105180 2024-05-09T00:00:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a9259c700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a94100060 0x7f3a94106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:00:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a94105180 0x7f3a94107260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a880099f0 tx=0x7f3a88009db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a837fe700 1 -- 172.21.15.12:0/3960886299 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88012070 con 0x7f3a94105180 2024-05-09T00:00:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a837fe700 1 -- 172.21.15.12:0/3960886299 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a8800de90 con 0x7f3a94105180 2024-05-09T00:00:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a837fe700 1 -- 172.21.15.12:0/3960886299 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88005280 con 0x7f3a94105180 2024-05-09T00:00:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a98999700 1 -- 172.21.15.12:0/3960886299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a941aa5c0 con 0x7f3a94105180 2024-05-09T00:00:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.869+0000 7f3a98999700 1 -- 172.21.15.12:0/3960886299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a941aaa80 con 0x7f3a94105180 2024-05-09T00:00:45.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.872+0000 7f3a837fe700 1 -- 172.21.15.12:0/3960886299 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3a88004750 con 0x7f3a94105180 2024-05-09T00:00:45.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.873+0000 7f3a837fe700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3a7c0a1310 0x7f3a7c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.873+0000 7f3a837fe700 1 -- 172.21.15.12:0/3960886299 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3a880ce1e0 con 0x7f3a94105180 2024-05-09T00:00:45.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.873+0000 7f3a9259c700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3a7c0a1310 0x7f3a7c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.874+0000 7f3a9259c700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3a7c0a1310 0x7f3a7c0a37d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400b2e0 tx=0x7f3a84005940 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:45.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3aa4b53b80 1 Processor -- start 2024-05-09T00:00:45.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3aa4b53b80 1 -- start start 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 0x5636941951e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694195720 0x56369418f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369418f890 0x56369418fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636940e6150 con 0x563694195720 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636940e3b10 con 0x56369418f890 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636940df400 con 0x563694194e00 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3a9259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 0x5636941951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3a9259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 0x5636941951e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33408/0 (socket says 172.21.15.12:33408) 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.875+0000 7f3a9259c700 1 -- 172.21.15.12:0/2490747597 learned_addr learned my addr 172.21.15.12:0/2490747597 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/2490747597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694195720 0x56369418f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a92d9d700 1 --2- 172.21.15.12:0/2490747597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369418f890 0x56369418fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a9259c700 1 -- 172.21.15.12:0/2490747597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369418f890 msgr2=0x56369418fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a9259c700 1 --2- 172.21.15.12:0/2490747597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369418f890 0x56369418fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a9259c700 1 -- 172.21.15.12:0/2490747597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694195720 msgr2=0x56369418f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a9259c700 1 --2- 172.21.15.12:0/2490747597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694195720 0x56369418f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a9259c700 1 -- 172.21.15.12:0/2490747597 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563693f603b0 con 0x563694194e00 2024-05-09T00:00:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/2490747597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694195720 0x56369418f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:00:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a9259c700 1 --2- 172.21.15.12:0/2490747597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 0x5636941951e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3a84015000 tx=0x7f3a8401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f769fb36c1c150 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a80ff9700 1 -- 172.21.15.12:0/2490747597 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8402b070 con 0x563694194e00 2024-05-09T00:00:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a80ff9700 1 -- 172.21.15.12:0/2490747597 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a8401e040 con 0x563694194e00 2024-05-09T00:00:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.876+0000 7f3a80ff9700 1 -- 172.21.15.12:0/2490747597 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84004e60 con 0x563694194e00 2024-05-09T00:00:45.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.877+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/2490747597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 msgr2=0x5636941951e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.877+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/2490747597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 0x5636941951e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3a84015000 tx=0x7f3a8401f000 comp rx=0 tx=0).stop 2024-05-09T00:00:45.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.877+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/2490747597 shutdown_connections 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.877+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/2490747597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369418f890 0x56369418fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.877+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/2490747597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 0x5636941951e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.877+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/2490747597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694195720 0x56369418f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.877+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/2490747597 >> 172.21.15.12:0/2490747597 conn(0x5636940ed210 msgr2=0x563694189df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.877+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/2490747597 shutdown_connections 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.878+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/2490747597 wait complete. 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.878+0000 7f3aa4b53b80 1 -- start start 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.878+0000 7f3aa4b53b80 1 Processor -- start 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.878+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369418f890 0x56369419b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x5636941989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 0x5636941990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369407b560 con 0x563694194e00 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369407b200 con 0x563694195720 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563693f626a0 con 0x56369418f890 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3aa4b53b80 10 client.0 _ll_register_callbacks cb 0x563694084b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a9259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369418f890 0x56369419b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a9259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369418f890 0x56369419b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33412/0 (socket says 172.21.15.12:33412) 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a9259c700 1 -- 172.21.15.12:0/437249029 learned_addr learned my addr 172.21.15.12:0/437249029 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x5636941989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a92d9d700 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 0x5636941990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a91d9b700 1 -- 172.21.15.12:0/437249029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369418f890 msgr2=0x56369419b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369418f890 0x56369419b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a91d9b700 1 -- 172.21.15.12:0/437249029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 msgr2=0x5636941990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 0x5636941990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a91d9b700 1 -- 172.21.15.12:0/437249029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9410d570 con 0x563694194e00 2024-05-09T00:00:45.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a9259c700 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369418f890 0x56369419b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a92d9d700 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 0x5636941990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x5636941989f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3a880c96f0 tx=0x7f3a88095520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940baa8d43a818ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.879+0000 7f3a79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.880+0000 7f3a79ffb700 1 -- 172.21.15.12:0/437249029 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8809c070 con 0x563694194e00 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.880+0000 7f3a79ffb700 1 -- 172.21.15.12:0/437249029 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3a880990b0 con 0x563694194e00 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.880+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/437249029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563694038270 con 0x563694194e00 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.880+0000 7f3a79ffb700 1 -- 172.21.15.12:0/437249029 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a8809cee0 con 0x563694194e00 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.880+0000 7f3aa4b53b80 10 client.19473 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.880+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/437249029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x563693f0cb60 con 0x563694194e00 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.880+0000 7f3a79ffb700 1 -- 172.21.15.12:0/437249029 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563694038270 con 0x563694194e00 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.880+0000 7f3a79ffb700 1 -- 172.21.15.12:0/437249029 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3a880985a0 con 0x563694194e00 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.880+0000 7f3aa4b53b80 10 client.19473 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.880+0000 7f3aa4b53b80 10 client.19473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:45.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.881+0000 7f3aa4b53b80 20 client.19473 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.881+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5636941a91c0 0x5636941ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.881+0000 7f3aa4b53b80 4 client.19473 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.881+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/437249029 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56369415e810 con 0x5636941a91c0 2024-05-09T00:00:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.881+0000 7f3a9259c700 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5636941a91c0 0x5636941ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.881+0000 7f3a9259c700 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5636941a91c0 0x5636941ab5e0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2027e49835828038 server_cookie=7f85dfa65e2a3bd in_seq=0 out_seq=0 2024-05-09T00:00:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.881+0000 7f3a79ffb700 10 client.19473 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:45.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3a79ffb700 1 -- 172.21.15.12:0/437249029 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56369415e810 con 0x5636941a91c0 2024-05-09T00:00:45.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3a79ffb700 10 client.19473 handle_command_reply: tid=0 2024-05-09T00:00:45.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3aa4b53b80 1 client.19473 shutdown 2024-05-09T00:00:45.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/437249029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 msgr2=0x5636941989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x5636941989f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3a880c96f0 tx=0x7f3a88095520 comp rx=0 tx=0).stop 2024-05-09T00:00:45.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/437249029 shutdown_connections 2024-05-09T00:00:45.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5636941a91c0 0x5636941ab5e0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 0x5636941990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369418f890 0x56369419b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/437249029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x5636941989f0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.886+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/437249029 >> 172.21.15.12:0/437249029 conn(0x5636940ed210 msgr2=0x5636941895a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:45.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.887+0000 7f3a79ffb700 0 client.19473 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:45.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.887+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/437249029 shutdown_connections 2024-05-09T00:00:45.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.887+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/437249029 wait complete. 2024-05-09T00:00:45.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.887+0000 7f3aa4b53b80 20 client.19473 trim_cache size 0 max 16384 2024-05-09T00:00:45.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.912+0000 7f3aa4b53b80 1 Processor -- start 2024-05-09T00:00:45.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.912+0000 7f3aa4b53b80 1 -- start start 2024-05-09T00:00:45.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369418f890 0x56369418f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x563694189d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694195720 0x5636941972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563694184a90 con 0x563694194e00 2024-05-09T00:00:45.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563694184800 con 0x56369418f890 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563694182040 con 0x563694195720 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a91d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x563694189d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a91d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x563694189d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38978/0 (socket says 172.21.15.12:38978) 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a91d9b700 1 -- 172.21.15.12:0/4087474194 learned_addr learned my addr 172.21.15.12:0/4087474194 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a9259c700 1 --2- 172.21.15.12:0/4087474194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369418f890 0x56369418f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a92d9d700 1 --2- 172.21.15.12:0/4087474194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694195720 0x5636941972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a91d9b700 1 -- 172.21.15.12:0/4087474194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694195720 msgr2=0x5636941972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/4087474194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694195720 0x5636941972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a91d9b700 1 -- 172.21.15.12:0/4087474194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369418f890 msgr2=0x56369418f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/4087474194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369418f890 0x56369418f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a91d9b700 1 -- 172.21.15.12:0/4087474194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56369418ff40 con 0x563694194e00 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.913+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/4087474194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x563694189d20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f3a88099040 tx=0x7f3a88004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac813e060046d2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.914+0000 7f3a817fa700 1 -- 172.21.15.12:0/4087474194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8809c070 con 0x563694194e00 2024-05-09T00:00:45.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.914+0000 7f3a817fa700 1 -- 172.21.15.12:0/4087474194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a88098a10 con 0x563694194e00 2024-05-09T00:00:45.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.914+0000 7f3a817fa700 1 -- 172.21.15.12:0/4087474194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88082070 con 0x563694194e00 2024-05-09T00:00:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.915+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/4087474194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 msgr2=0x563694189d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.915+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/4087474194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x563694189d20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f3a88099040 tx=0x7f3a88004f70 comp rx=0 tx=0).stop 2024-05-09T00:00:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.915+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/4087474194 shutdown_connections 2024-05-09T00:00:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.915+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/4087474194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694195720 0x5636941972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.915+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/4087474194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369418f890 0x56369418f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.915+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/4087474194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563694194e00 0x563694189d20 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.915+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/4087474194 >> 172.21.15.12:0/4087474194 conn(0x5636940ed210 msgr2=0x5636941a4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.915+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/4087474194 shutdown_connections 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.915+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/4087474194 wait complete. 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.915+0000 7f3aa4b53b80 1 -- start start 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3aa4b53b80 1 Processor -- start 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56369418f890 0x56369419c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 0x56369419c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3aa4b53b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 0x5636941f8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636941f8bc0 con 0x56369418f890 2024-05-09T00:00:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636941f8d30 con 0x563694195720 2024-05-09T00:00:45.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3aa4b53b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636941f8ef0 con 0x563694194e00 2024-05-09T00:00:45.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3aa4b53b80 10 client.0 _ll_register_callbacks cb 0x7f3a88098f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:45.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3a9259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56369418f890 0x56369419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3a9259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56369418f890 0x56369419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38994/0 (socket says 172.21.15.12:38994) 2024-05-09T00:00:45.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3a9259c700 1 -- 172.21.15.12:0/641737835 learned_addr learned my addr 172.21.15.12:0/641737835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:45.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3a92d9d700 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 0x5636941f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 0x56369419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.916+0000 7f3a9259c700 1 -- 172.21.15.12:0/641737835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 msgr2=0x56369419c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:45.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3a9259c700 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 0x56369419c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3a9259c700 1 -- 172.21.15.12:0/641737835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 msgr2=0x5636941f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:45.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3a9259c700 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 0x5636941f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3a9259c700 1 -- 172.21.15.12:0/641737835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563693f0cb60 con 0x56369418f890 2024-05-09T00:00:45.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3a92d9d700 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 0x5636941f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:00:45.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3a9259c700 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56369418f890 0x56369419c450 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005780 tx=0x7f3a84004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596aa22069cae136 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:45.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3a79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:45.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3a79ffb700 1 -- 172.21.15.12:0/641737835 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84035040 con 0x56369418f890 2024-05-09T00:00:45.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3a79ffb700 1 -- 172.21.15.12:0/641737835 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3a84037d70 con 0x56369418f890 2024-05-09T00:00:45.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/641737835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a8401e040 con 0x56369418f890 2024-05-09T00:00:45.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.917+0000 7f3a79ffb700 1 -- 172.21.15.12:0/641737835 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a8402b040 con 0x56369418f890 2024-05-09T00:00:45.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3aa4b53b80 10 client.19479 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/641737835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5636940a5560 con 0x56369418f890 2024-05-09T00:00:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3a79ffb700 1 -- 172.21.15.12:0/641737835 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a84044020 con 0x56369418f890 2024-05-09T00:00:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3a79ffb700 1 -- 172.21.15.12:0/641737835 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3a84030500 con 0x56369418f890 2024-05-09T00:00:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3aa4b53b80 10 client.19479 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3aa4b53b80 10 client.19479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3aa4b53b80 20 client.19479 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563694200270 0x563694202690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3aa4b53b80 4 client.19479 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:00:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/641737835 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5636940d7ec0 con 0x563694200270 2024-05-09T00:00:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.918+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563694200270 0x563694202690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.919+0000 7f3a79ffb700 10 client.19479 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:45.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.919+0000 7f3a91d9b700 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563694200270 0x563694202690 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b80e47e882f77fd server_cookie=351f04173a0fef15 in_seq=0 out_seq=0 2024-05-09T00:00:45.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:45 smithi012 ceph-mon[42503]: pgmap v459: 97 pgs: 97 active+clean; 490 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 33 MiB/s wr, 916 op/s 2024-05-09T00:00:45.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3a79ffb700 1 -- 172.21.15.12:0/641737835 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5636940d7ec0 con 0x563694200270 2024-05-09T00:00:45.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3a79ffb700 10 client.19479 handle_command_reply: tid=0 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 client.19479 shutdown 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/641737835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56369418f890 msgr2=0x56369419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56369418f890 0x56369419c450 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005780 tx=0x7f3a84004620 comp rx=0 tx=0).stop 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/641737835 shutdown_connections 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563694200270 0x563694202690 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563694195720 0x5636941f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56369418f890 0x56369419c450 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 --2- 172.21.15.12:0/641737835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563694194e00 0x56369419c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/641737835 >> 172.21.15.12:0/641737835 conn(0x5636940ed210 msgr2=0x5636941a4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3a79ffb700 0 client.19479 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/641737835 shutdown_connections 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.921+0000 7f3aa4b53b80 1 -- 172.21.15.12:0/641737835 wait complete. 2024-05-09T00:00:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.922+0000 7f3aa4b53b80 20 client.19479 trim_cache size 0 max 16384 2024-05-09T00:00:45.921 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:00:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:00:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:00:45.921 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:00:45.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.922+0000 7f3a98999700 1 -- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3a7c0a1310 msgr2=0x7f3a7c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.922+0000 7f3a98999700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3a7c0a1310 0x7f3a7c0a37d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400b2e0 tx=0x7f3a84005940 comp rx=0 tx=0).stop 2024-05-09T00:00:45.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.922+0000 7f3a98999700 1 -- 172.21.15.12:0/3960886299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a94105180 msgr2=0x7f3a94107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.922+0000 7f3a98999700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a94105180 0x7f3a94107260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a880099f0 tx=0x7f3a88009db0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.923+0000 7f3a98999700 1 -- 172.21.15.12:0/3960886299 shutdown_connections 2024-05-09T00:00:45.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.923+0000 7f3a98999700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3a7c0a1310 0x7f3a7c0a37d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.923+0000 7f3a98999700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a941077a0 0x7f3a941a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.923+0000 7f3a98999700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a94100060 0x7f3a94106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.923+0000 7f3a98999700 1 --2- 172.21.15.12:0/3960886299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a94105180 0x7f3a94107260 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.923+0000 7f3a98999700 1 -- 172.21.15.12:0/3960886299 >> 172.21.15.12:0/3960886299 conn(0x7f3a94076f10 msgr2=0x7f3a940fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:45.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.923+0000 7f3a98999700 1 -- 172.21.15.12:0/3960886299 shutdown_connections 2024-05-09T00:00:45.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:45.923+0000 7f3a98999700 1 -- 172.21.15.12:0/3960886299 wait complete. 2024-05-09T00:00:45.935 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:00:45.935 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:00:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:45 smithi204 ceph-mon[47373]: pgmap v459: 97 pgs: 97 active+clean; 490 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 33 MiB/s wr, 916 op/s 2024-05-09T00:00:46.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:45 smithi160 ceph-mon[46152]: pgmap v459: 97 pgs: 97 active+clean; 490 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 33 MiB/s wr, 916 op/s 2024-05-09T00:00:46.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.099+0000 7f214c61e700 1 -- 172.21.15.12:0/3900245416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2144101050 msgr2=0x7f2144101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.099+0000 7f214c61e700 1 --2- 172.21.15.12:0/3900245416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2144101050 0x7f2144101430 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f2140009990 tx=0x7f2140009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.099+0000 7f214c61e700 1 -- 172.21.15.12:0/3900245416 shutdown_connections 2024-05-09T00:00:46.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.099+0000 7f214c61e700 1 --2- 172.21.15.12:0/3900245416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2144106170 0x7f214410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.099+0000 7f214c61e700 1 --2- 172.21.15.12:0/3900245416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2144101050 0x7f2144101430 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.099+0000 7f214c61e700 1 --2- 172.21.15.12:0/3900245416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144101a00 0x7f2144105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.099+0000 7f214c61e700 1 -- 172.21.15.12:0/3900245416 >> 172.21.15.12:0/3900245416 conn(0x7f21440fc8f0 msgr2=0x7f21440fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.099+0000 7f214c61e700 1 -- 172.21.15.12:0/3900245416 shutdown_connections 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.099+0000 7f214c61e700 1 -- 172.21.15.12:0/3900245416 wait complete. 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.100+0000 7f214c61e700 1 Processor -- start 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.100+0000 7f214c61e700 1 -- start start 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.100+0000 7f214c61e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2144101050 0x7f2144197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.100+0000 7f214c61e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144106170 0x7f2144197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.100+0000 7f214c61e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214419bbd0 0x7f214419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.100+0000 7f214c61e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214419c650 con 0x7f214419bbd0 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.100+0000 7f214c61e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214419c7c0 con 0x7f2144101050 2024-05-09T00:00:46.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.100+0000 7f214c61e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214419c980 con 0x7f2144106170 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.100+0000 7f214abbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214419bbd0 0x7f214419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f214abbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214419bbd0 0x7f214419c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39014/0 (socket says 172.21.15.12:39014) 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f214abbb700 1 -- 172.21.15.12:0/373576501 learned_addr learned my addr 172.21.15.12:0/373576501 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f2149bb9700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144106170 0x7f2144197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f214a3ba700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2144101050 0x7f2144197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f2149bb9700 1 -- 172.21.15.12:0/373576501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2144101050 msgr2=0x7f2144197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f2149bb9700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2144101050 0x7f2144197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f2149bb9700 1 -- 172.21.15.12:0/373576501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214419bbd0 msgr2=0x7f214419c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f2149bb9700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214419bbd0 0x7f214419c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f2149bb9700 1 -- 172.21.15.12:0/373576501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400095f0 con 0x7f2144106170 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f214abbb700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214419bbd0 0x7f214419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:46.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f2149bb9700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144106170 0x7f2144197880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f213400e9c0 tx=0x7f213400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f213b7fe700 1 -- 172.21.15.12:0/373576501 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134004220 con 0x7f2144106170 2024-05-09T00:00:46.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f213b7fe700 1 -- 172.21.15.12:0/373576501 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2134004cf0 con 0x7f2144106170 2024-05-09T00:00:46.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f214c61e700 1 -- 172.21.15.12:0/373576501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21441a3ee0 con 0x7f2144106170 2024-05-09T00:00:46.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f213b7fe700 1 -- 172.21.15.12:0/373576501 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213400ce40 con 0x7f2144106170 2024-05-09T00:00:46.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.101+0000 7f214c61e700 1 -- 172.21.15.12:0/373576501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21441a4310 con 0x7f2144106170 2024-05-09T00:00:46.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.102+0000 7f214c61e700 1 -- 172.21.15.12:0/373576501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21441093e0 con 0x7f2144106170 2024-05-09T00:00:46.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.104+0000 7f213b7fe700 1 -- 172.21.15.12:0/373576501 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2134004750 con 0x7f2144106170 2024-05-09T00:00:46.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.104+0000 7f213b7fe700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21300a1290 0x7f21300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.104+0000 7f214a3ba700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21300a1290 0x7f21300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.104+0000 7f213b7fe700 1 -- 172.21.15.12:0/373576501 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2134014070 con 0x7f2144106170 2024-05-09T00:00:46.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.105+0000 7f214a3ba700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21300a1290 0x7f21300a3750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f214000b2e0 tx=0x7f2140005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.109+0000 7f213b7fe700 1 -- 172.21.15.12:0/373576501 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f21340963a0 con 0x7f2144106170 2024-05-09T00:00:46.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 28404 41.63 MB/sec warmup 41 sec latency 74.896 ms 2024-05-09T00:00:46.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.428+0000 7f214c61e700 1 -- 172.21.15.12:0/373576501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2144102520 con 0x7f2144106170 2024-05-09T00:00:46.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.428+0000 7f213b7fe700 1 -- 172.21.15.12:0/373576501 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+10882 (secure 0 0 0) 0x7f2134004e60 con 0x7f2144106170 2024-05-09T00:00:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:00:46.423 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:00:22.819142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_28844":{"gid":28844,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/864772354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":864772354},{"type":"v1","addr":"172.21.15.204:6837","nonce":864772354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:00:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 -- 172.21.15.12:0/373576501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21300a1290 msgr2=0x7f21300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21300a1290 0x7f21300a3750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f214000b2e0 tx=0x7f2140005960 comp rx=0 tx=0).stop 2024-05-09T00:00:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 -- 172.21.15.12:0/373576501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144106170 msgr2=0x7f2144197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144106170 0x7f2144197880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f213400e9c0 tx=0x7f213400ecd0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 -- 172.21.15.12:0/373576501 shutdown_connections 2024-05-09T00:00:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f21300a1290 0x7f21300a3750 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214419bbd0 0x7f214419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2144101050 0x7f2144197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 --2- 172.21.15.12:0/373576501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144106170 0x7f2144197880 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 -- 172.21.15.12:0/373576501 >> 172.21.15.12:0/373576501 conn(0x7f21440fc8f0 msgr2=0x7f21440fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 -- 172.21.15.12:0/373576501 shutdown_connections 2024-05-09T00:00:46.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.431+0000 7f214c61e700 1 -- 172.21.15.12:0/373576501 wait complete. 2024-05-09T00:00:46.427 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 61 2024-05-09T00:00:46.441 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:00:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.606+0000 7f256f34a700 1 -- 172.21.15.12:0/3438720264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25681062a0 msgr2=0x7f256810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.606+0000 7f256f34a700 1 --2- 172.21.15.12:0/3438720264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25681062a0 0x7f256810de20 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f256000b0a0 tx=0x7f256000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.606+0000 7f256f34a700 1 -- 172.21.15.12:0/3438720264 shutdown_connections 2024-05-09T00:00:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.606+0000 7f256f34a700 1 --2- 172.21.15.12:0/3438720264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25681062a0 0x7f256810de20 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.606+0000 7f256f34a700 1 --2- 172.21.15.12:0/3438720264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25680693a0 0x7f2568069780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.606+0000 7f256f34a700 1 --2- 172.21.15.12:0/3438720264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2568069d50 0x7f2568105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.606+0000 7f256f34a700 1 -- 172.21.15.12:0/3438720264 >> 172.21.15.12:0/3438720264 conn(0x7f2568076ba0 msgr2=0x7f2568076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.606+0000 7f256f34a700 1 -- 172.21.15.12:0/3438720264 shutdown_connections 2024-05-09T00:00:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.606+0000 7f256f34a700 1 -- 172.21.15.12:0/3438720264 wait complete. 2024-05-09T00:00:46.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256f34a700 1 Processor -- start 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256f34a700 1 -- start start 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256f34a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25680693a0 0x7f25681001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256f34a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25681062a0 0x7f2568100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256f34a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2568100c40 0x7f25681a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256f34a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25681a5cb0 con 0x7f2568100c40 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256f34a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25681a5e20 con 0x7f25681062a0 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256f34a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25681a5fe0 con 0x7f25680693a0 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256d0e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25680693a0 0x7f25681001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256d8e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2568100c40 0x7f25681a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256d0e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25680693a0 0x7f25681001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33460/0 (socket says 172.21.15.12:33460) 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256d0e6700 1 -- 172.21.15.12:0/1469372501 learned_addr learned my addr 172.21.15.12:0/1469372501 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.607+0000 7f256c8e5700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25681062a0 0x7f2568100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f256d0e6700 1 -- 172.21.15.12:0/1469372501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25681062a0 msgr2=0x7f2568100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f256d0e6700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25681062a0 0x7f2568100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f256d0e6700 1 -- 172.21.15.12:0/1469372501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2568100c40 msgr2=0x7f25681a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f256d0e6700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2568100c40 0x7f25681a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f256d0e6700 1 -- 172.21.15.12:0/1469372501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2560009d00 con 0x7f25680693a0 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f256d8e7700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2568100c40 0x7f25681a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f256d0e6700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25680693a0 0x7f25681001c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f256400e9a0 tx=0x7f256400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f255e7fc700 1 -- 172.21.15.12:0/1469372501 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564009790 con 0x7f25680693a0 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f255e7fc700 1 -- 172.21.15.12:0/1469372501 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2564010430 con 0x7f25680693a0 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f256f34a700 1 -- 172.21.15.12:0/1469372501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25681a62c0 con 0x7f25680693a0 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f255e7fc700 1 -- 172.21.15.12:0/1469372501 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564018680 con 0x7f25680693a0 2024-05-09T00:00:46.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.608+0000 7f256f34a700 1 -- 172.21.15.12:0/1469372501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25681a67d0 con 0x7f25680693a0 2024-05-09T00:00:46.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.611+0000 7f255e7fc700 1 -- 172.21.15.12:0/1469372501 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f25640105a0 con 0x7f25680693a0 2024-05-09T00:00:46.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.612+0000 7f255e7fc700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25540a1290 0x7f25540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.612+0000 7f256c8e5700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25540a1290 0x7f25540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.612+0000 7f255e7fc700 1 -- 172.21.15.12:0/1469372501 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2564014070 con 0x7f25680693a0 2024-05-09T00:00:46.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.612+0000 7f256c8e5700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25540a1290 0x7f25540a3750 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f25681018f0 tx=0x7f255800b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f257b504b80 1 Processor -- start 2024-05-09T00:00:46.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f257b504b80 1 -- start start 2024-05-09T00:00:46.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f257b504b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 0x5561a7f001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f257b504b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7f00720 0x5561a7efa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f257b504b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 0x5561a7efacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f257b504b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7e51150 con 0x5561a7effe00 2024-05-09T00:00:46.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f257b504b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7e4eb10 con 0x5561a7f00720 2024-05-09T00:00:46.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f257b504b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7e4a400 con 0x5561a7efa890 2024-05-09T00:00:46.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f256d0e6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 0x5561a7f001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f256d0e6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 0x5561a7f001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39048/0 (socket says 172.21.15.12:39048) 2024-05-09T00:00:46.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f256d0e6700 1 -- 172.21.15.12:0/2058748276 learned_addr learned my addr 172.21.15.12:0/2058748276 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f256d8e7700 1 --2- 172.21.15.12:0/2058748276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 0x5561a7efacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f256c8e5700 1 --2- 172.21.15.12:0/2058748276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7f00720 0x5561a7efa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f256d0e6700 1 -- 172.21.15.12:0/2058748276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 msgr2=0x5561a7efacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f256d0e6700 1 --2- 172.21.15.12:0/2058748276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 0x5561a7efacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.613+0000 7f256d0e6700 1 -- 172.21.15.12:0/2058748276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7f00720 msgr2=0x5561a7efa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.614+0000 7f256d0e6700 1 --2- 172.21.15.12:0/2058748276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7f00720 0x5561a7efa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.614+0000 7f256d0e6700 1 -- 172.21.15.12:0/2058748276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a7ccb3b0 con 0x5561a7effe00 2024-05-09T00:00:46.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.614+0000 7f256d0e6700 1 --2- 172.21.15.12:0/2058748276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 0x5561a7f001e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f25640964c0 tx=0x7f256409a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3c63b2c47ae26a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.614+0000 7f25537fe700 1 -- 172.21.15.12:0/2058748276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25640a2440 con 0x5561a7effe00 2024-05-09T00:00:46.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.614+0000 7f25537fe700 1 -- 172.21.15.12:0/2058748276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2564010880 con 0x5561a7effe00 2024-05-09T00:00:46.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.614+0000 7f25537fe700 1 -- 172.21.15.12:0/2058748276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564099780 con 0x5561a7effe00 2024-05-09T00:00:46.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 -- 172.21.15.12:0/2058748276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 msgr2=0x5561a7f001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 --2- 172.21.15.12:0/2058748276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 0x5561a7f001e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f25640964c0 tx=0x7f256409a6c0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 -- 172.21.15.12:0/2058748276 shutdown_connections 2024-05-09T00:00:46.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 --2- 172.21.15.12:0/2058748276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 0x5561a7efacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 --2- 172.21.15.12:0/2058748276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 0x5561a7f001e0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 --2- 172.21.15.12:0/2058748276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7f00720 0x5561a7efa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 -- 172.21.15.12:0/2058748276 >> 172.21.15.12:0/2058748276 conn(0x5561a7e58210 msgr2=0x5561a7ef4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 -- 172.21.15.12:0/2058748276 shutdown_connections 2024-05-09T00:00:46.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 -- 172.21.15.12:0/2058748276 wait complete. 2024-05-09T00:00:46.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 -- start start 2024-05-09T00:00:46.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.615+0000 7f257b504b80 1 Processor -- start 2024-05-09T00:00:46.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f257b504b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7efa890 0x5561a7f03b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f257b504b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7effe00 0x5561a7f040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f257b504b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f257b504b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7de6560 con 0x5561a7f00720 2024-05-09T00:00:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f257b504b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7de6200 con 0x5561a7efa890 2024-05-09T00:00:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f257b504b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7ccd6a0 con 0x5561a7effe00 2024-05-09T00:00:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f257b504b80 10 client.0 _ll_register_callbacks cb 0x5561a7defb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f256d8e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f256d8e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39054/0 (socket says 172.21.15.12:39054) 2024-05-09T00:00:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f256d8e7700 1 -- 172.21.15.12:0/1920362673 learned_addr learned my addr 172.21.15.12:0/1920362673 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f256c8e5700 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7effe00 0x5561a7f040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.616+0000 7f256d0e6700 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7efa890 0x5561a7f03b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.617+0000 7f256c8e5700 1 -- 172.21.15.12:0/1920362673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7efa890 msgr2=0x5561a7f03b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.617+0000 7f256c8e5700 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7efa890 0x5561a7f03b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.617+0000 7f256c8e5700 1 -- 172.21.15.12:0/1920362673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 msgr2=0x5561a7f045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.617+0000 7f256c8e5700 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.617+0000 7f256c8e5700 1 -- 172.21.15.12:0/1920362673 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25681a67d0 con 0x5561a7effe00 2024-05-09T00:00:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.617+0000 7f256d8e7700 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.617+0000 7f256c8e5700 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7effe00 0x5561a7f040a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f255800ba00 tx=0x7f25580047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=522715683f420493 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.617+0000 7f2550ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.619+0000 7f2550ff9700 1 -- 172.21.15.12:0/1920362673 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558010e80 con 0x5561a7effe00 2024-05-09T00:00:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.619+0000 7f2550ff9700 1 -- 172.21.15.12:0/1920362673 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2558017070 con 0x5561a7effe00 2024-05-09T00:00:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.619+0000 7f257b504b80 1 -- 172.21.15.12:0/1920362673 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561a7da3270 con 0x5561a7effe00 2024-05-09T00:00:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f2550ff9700 1 -- 172.21.15.12:0/1920362673 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2558004830 con 0x5561a7effe00 2024-05-09T00:00:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f257b504b80 10 client.29218 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f257b504b80 1 -- 172.21.15.12:0/1920362673 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5561a7c77b60 con 0x5561a7effe00 2024-05-09T00:00:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f2550ff9700 1 -- 172.21.15.12:0/1920362673 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561a7da3270 con 0x5561a7effe00 2024-05-09T00:00:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f2550ff9700 1 -- 172.21.15.12:0/1920362673 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2558041470 con 0x5561a7effe00 2024-05-09T00:00:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f257b504b80 10 client.29218 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f257b504b80 10 client.29218 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:00:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f257b504b80 20 client.29218 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f257b504b80 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5561a7f14260 0x5561a7f16680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f257b504b80 4 client.29218 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.621+0000 7f257b504b80 1 -- 172.21.15.12:0/1920362673 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561a7ec9810 con 0x5561a7f14260 2024-05-09T00:00:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.622+0000 7f256d0e6700 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5561a7f14260 0x5561a7f16680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.622+0000 7f2550ff9700 10 client.29218 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:00:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.622+0000 7f256d0e6700 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5561a7f14260 0x5561a7f16680 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67724775619b7ae server_cookie=30cea23bd8c04eb5 in_seq=0 out_seq=0 2024-05-09T00:00:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.623+0000 7f2550ff9700 1 -- 172.21.15.12:0/1920362673 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5561a7ec9810 con 0x5561a7f14260 2024-05-09T00:00:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.623+0000 7f2550ff9700 10 client.29218 handle_command_reply: tid=0 2024-05-09T00:00:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.623+0000 7f257b504b80 1 client.29218 shutdown 2024-05-09T00:00:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.623+0000 7f257b504b80 1 -- 172.21.15.12:0/1920362673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7effe00 msgr2=0x5561a7f040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f257b504b80 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7effe00 0x5561a7f040a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f255800ba00 tx=0x7f25580047c0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f257b504b80 1 -- 172.21.15.12:0/1920362673 shutdown_connections 2024-05-09T00:00:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f257b504b80 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f257b504b80 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7efa890 0x5561a7f03b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f257b504b80 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5561a7f14260 0x5561a7f16680 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f257b504b80 1 --2- 172.21.15.12:0/1920362673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7effe00 0x5561a7f040a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f2550ff9700 0 client.29218 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:00:46.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f257b504b80 1 -- 172.21.15.12:0/1920362673 >> 172.21.15.12:0/1920362673 conn(0x5561a7e58210 msgr2=0x5561a7ef45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f257b504b80 1 -- 172.21.15.12:0/1920362673 shutdown_connections 2024-05-09T00:00:46.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f257b504b80 1 -- 172.21.15.12:0/1920362673 wait complete. 2024-05-09T00:00:46.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.624+0000 7f257b504b80 20 client.29218 trim_cache size 0 max 16384 2024-05-09T00:00:46.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.639+0000 7f257b504b80 1 Processor -- start 2024-05-09T00:00:46.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.639+0000 7f257b504b80 1 -- start start 2024-05-09T00:00:46.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.639+0000 7f257b504b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 0x5561a7efa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.639+0000 7f257b504b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7effe00 0x5561a7eff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f257b504b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f0fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f257b504b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7f10d80 con 0x5561a7f00720 2024-05-09T00:00:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f257b504b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7f05160 con 0x5561a7effe00 2024-05-09T00:00:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f257b504b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7f06b60 con 0x5561a7efa890 2024-05-09T00:00:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d8e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f0fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d0e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 0x5561a7efa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d8e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f0fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39060/0 (socket says 172.21.15.12:39060) 2024-05-09T00:00:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d8e7700 1 -- 172.21.15.12:0/4143355824 learned_addr learned my addr 172.21.15.12:0/4143355824 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256c8e5700 1 --2- 172.21.15.12:0/4143355824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7effe00 0x5561a7eff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d8e7700 1 -- 172.21.15.12:0/4143355824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 msgr2=0x5561a7efa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d8e7700 1 --2- 172.21.15.12:0/4143355824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 0x5561a7efa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d8e7700 1 -- 172.21.15.12:0/4143355824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7effe00 msgr2=0x5561a7eff4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d8e7700 1 --2- 172.21.15.12:0/4143355824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7effe00 0x5561a7eff4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d8e7700 1 -- 172.21.15.12:0/4143355824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a7ef14e0 con 0x5561a7f00720 2024-05-09T00:00:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d0e6700 1 --2- 172.21.15.12:0/4143355824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 0x5561a7efa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.640+0000 7f256d8e7700 1 --2- 172.21.15.12:0/4143355824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f0fc20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f2560007bd0 tx=0x7f2560007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91992685f4fe15a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f2553fff700 1 -- 172.21.15.12:0/4143355824 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256000e040 con 0x5561a7f00720 2024-05-09T00:00:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f2553fff700 1 -- 172.21.15.12:0/4143355824 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2560022070 con 0x5561a7f00720 2024-05-09T00:00:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f2553fff700 1 -- 172.21.15.12:0/4143355824 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25600054a0 con 0x5561a7f00720 2024-05-09T00:00:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f257b504b80 1 -- 172.21.15.12:0/4143355824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 msgr2=0x5561a7f0fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f257b504b80 1 --2- 172.21.15.12:0/4143355824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f0fc20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f2560007bd0 tx=0x7f2560007c00 comp rx=0 tx=0).stop 2024-05-09T00:00:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f257b504b80 1 -- 172.21.15.12:0/4143355824 shutdown_connections 2024-05-09T00:00:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f257b504b80 1 --2- 172.21.15.12:0/4143355824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7f00720 0x5561a7f0fc20 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f257b504b80 1 --2- 172.21.15.12:0/4143355824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7efa890 0x5561a7efa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f257b504b80 1 --2- 172.21.15.12:0/4143355824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7effe00 0x5561a7eff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f257b504b80 1 -- 172.21.15.12:0/4143355824 >> 172.21.15.12:0/4143355824 conn(0x5561a7e58210 msgr2=0x5561a7f02430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.641+0000 7f257b504b80 1 -- 172.21.15.12:0/4143355824 shutdown_connections 2024-05-09T00:00:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.642+0000 7f257b504b80 1 -- 172.21.15.12:0/4143355824 wait complete. 2024-05-09T00:00:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.642+0000 7f257b504b80 1 -- start start 2024-05-09T00:00:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.642+0000 7f257b504b80 1 Processor -- start 2024-05-09T00:00:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.642+0000 7f257b504b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7efa890 0x5561a7f08ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.642+0000 7f257b504b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 0x5561a7f09410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.642+0000 7f257b504b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7f00720 0x5561a7f09950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.642+0000 7f257b504b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7f69890 con 0x5561a7effe00 2024-05-09T00:00:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f257b504b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7f69a00 con 0x5561a7efa890 2024-05-09T00:00:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f257b504b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7f69bc0 con 0x5561a7f00720 2024-05-09T00:00:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f257b504b80 10 client.0 _ll_register_callbacks cb 0x5561a7defb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256d8e7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7f00720 0x5561a7f09950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256d8e7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7f00720 0x5561a7f09950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33486/0 (socket says 172.21.15.12:33486) 2024-05-09T00:00:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256d8e7700 1 -- 172.21.15.12:0/2255993523 learned_addr learned my addr 172.21.15.12:0/2255993523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256d0e6700 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7efa890 0x5561a7f08ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256c8e5700 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 0x5561a7f09410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256d8e7700 1 -- 172.21.15.12:0/2255993523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7efa890 msgr2=0x5561a7f08ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256d8e7700 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7efa890 0x5561a7f08ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256d8e7700 1 -- 172.21.15.12:0/2255993523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 msgr2=0x5561a7f09410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256d8e7700 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 0x5561a7f09410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256d8e7700 1 -- 172.21.15.12:0/2255993523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256405fe70 con 0x5561a7f00720 2024-05-09T00:00:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f256d8e7700 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7f00720 0x5561a7f09950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2560009a20 tx=0x7f2560003ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb917edeabfd970d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.643+0000 7f2550ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f2550ff9700 1 -- 172.21.15.12:0/2255993523 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256000e070 con 0x5561a7f00720 2024-05-09T00:00:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f2550ff9700 1 -- 172.21.15.12:0/2255993523 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f256000bc60 con 0x5561a7f00720 2024-05-09T00:00:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f257b504b80 1 -- 172.21.15.12:0/2255993523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2564010880 con 0x5561a7f00720 2024-05-09T00:00:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f2550ff9700 1 -- 172.21.15.12:0/2255993523 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2560007ea0 con 0x5561a7f00720 2024-05-09T00:00:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f257b504b80 10 client.29224 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f257b504b80 1 -- 172.21.15.12:0/2255993523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f256810e690 con 0x5561a7f00720 2024-05-09T00:00:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f2550ff9700 1 -- 172.21.15.12:0/2255993523 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2564010880 con 0x5561a7f00720 2024-05-09T00:00:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f2550ff9700 1 -- 172.21.15.12:0/2255993523 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2560036020 con 0x5561a7f00720 2024-05-09T00:00:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f257b504b80 10 client.29224 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f257b504b80 10 client.29224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:00:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f257b504b80 20 client.29224 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f257b504b80 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5561a7f6b050 0x5561a7f6d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f257b504b80 4 client.29224 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:00:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.644+0000 7f257b504b80 1 -- 172.21.15.12:0/2255993523 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5561a7e42ec0 con 0x5561a7f6b050 2024-05-09T00:00:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.645+0000 7f256d0e6700 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5561a7f6b050 0x5561a7f6d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.645+0000 7f2550ff9700 10 client.29224 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:00:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.645+0000 7f256d0e6700 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5561a7f6b050 0x5561a7f6d470 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97b350d49c348213 server_cookie=5a95b93acd121f92 in_seq=0 out_seq=0 2024-05-09T00:00:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f2550ff9700 1 -- 172.21.15.12:0/2255993523 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5561a7e42ec0 con 0x5561a7f6b050 2024-05-09T00:00:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f2550ff9700 10 client.29224 handle_command_reply: tid=0 2024-05-09T00:00:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 client.29224 shutdown 2024-05-09T00:00:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 -- 172.21.15.12:0/2255993523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7f00720 msgr2=0x5561a7f09950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7f00720 0x5561a7f09950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2560009a20 tx=0x7f2560003ef0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 -- 172.21.15.12:0/2255993523 shutdown_connections 2024-05-09T00:00:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5561a7f6b050 0x5561a7f6d470 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5561a7f00720 0x5561a7f09950 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5561a7efa890 0x5561a7f08ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 --2- 172.21.15.12:0/2255993523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561a7effe00 0x5561a7f09410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f2550ff9700 0 client.29224 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:00:46.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 -- 172.21.15.12:0/2255993523 >> 172.21.15.12:0/2255993523 conn(0x5561a7e58210 msgr2=0x5561a7eff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 -- 172.21.15.12:0/2255993523 shutdown_connections 2024-05-09T00:00:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 1 -- 172.21.15.12:0/2255993523 wait complete. 2024-05-09T00:00:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.647+0000 7f257b504b80 20 client.29224 trim_cache size 0 max 16384 2024-05-09T00:00:46.654 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:00:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 -- 172.21.15.12:0/1469372501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25540a1290 msgr2=0x7f25540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25540a1290 0x7f25540a3750 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f25681018f0 tx=0x7f255800b410 comp rx=0 tx=0).stop 2024-05-09T00:00:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 -- 172.21.15.12:0/1469372501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25680693a0 msgr2=0x7f25681001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25680693a0 0x7f25681001c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f256400e9a0 tx=0x7f256400ecb0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 -- 172.21.15.12:0/1469372501 shutdown_connections 2024-05-09T00:00:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25540a1290 0x7f25540a3750 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2568100c40 0x7f25681a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25680693a0 0x7f25681001c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 --2- 172.21.15.12:0/1469372501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25681062a0 0x7f2568100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 -- 172.21.15.12:0/1469372501 >> 172.21.15.12:0/1469372501 conn(0x7f2568076ba0 msgr2=0x7f25680fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 -- 172.21.15.12:0/1469372501 shutdown_connections 2024-05-09T00:00:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.648+0000 7f256f34a700 1 -- 172.21.15.12:0/1469372501 wait complete. 2024-05-09T00:00:46.658 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:00:46.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:46 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/373576501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:00:46.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.838+0000 7fdcdde7f700 1 -- 172.21.15.12:0/770000721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd8105160 msgr2=0x7fdcd810cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.838+0000 7fdcdde7f700 1 --2- 172.21.15.12:0/770000721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd8105160 0x7fdcd810cce0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009f90 tx=0x7fdcd000b330 comp rx=0 tx=0).stop 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.838+0000 7fdcdde7f700 1 -- 172.21.15.12:0/770000721 shutdown_connections 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.838+0000 7fdcdde7f700 1 --2- 172.21.15.12:0/770000721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd8105160 0x7fdcd810cce0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.838+0000 7fdcdde7f700 1 --2- 172.21.15.12:0/770000721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd8100060 0x7fdcd8100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.838+0000 7fdcdde7f700 1 --2- 172.21.15.12:0/770000721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcd8100a10 0x7fdcd8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.838+0000 7fdcdde7f700 1 -- 172.21.15.12:0/770000721 >> 172.21.15.12:0/770000721 conn(0x7fdcd8076f40 msgr2=0x7fdcd8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.838+0000 7fdcdde7f700 1 -- 172.21.15.12:0/770000721 shutdown_connections 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.838+0000 7fdcdde7f700 1 -- 172.21.15.12:0/770000721 wait complete. 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcdde7f700 1 Processor -- start 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcdde7f700 1 -- start start 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcdde7f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd8100060 0x7fdcd81974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcdde7f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd8105160 0x7fdcd81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcdde7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcd819bd90 0x7fdcd819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcdde7f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd819c810 con 0x7fdcd8100060 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcdde7f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd819c980 con 0x7fdcd819bd90 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcdde7f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd819cb40 con 0x7fdcd8105160 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcd77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd8100060 0x7fdcd81974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcd77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd8100060 0x7fdcd81974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39082/0 (socket says 172.21.15.12:39082) 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcd6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd8105160 0x7fdcd81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcd77fe700 1 -- 172.21.15.12:0/795432775 learned_addr learned my addr 172.21.15.12:0/795432775 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.839+0000 7fdcd7fff700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcd819bd90 0x7fdcd819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcd77fe700 1 -- 172.21.15.12:0/795432775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd8105160 msgr2=0x7fdcd81979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcd77fe700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd8105160 0x7fdcd81979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcd77fe700 1 -- 172.21.15.12:0/795432775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcd819bd90 msgr2=0x7fdcd819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcd77fe700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcd819bd90 0x7fdcd819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcd77fe700 1 -- 172.21.15.12:0/795432775 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0009c40 con 0x7fdcd8100060 2024-05-09T00:00:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcd6ffd700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd8105160 0x7fdcd81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcd77fe700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd8100060 0x7fdcd81974b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800e9c0 tx=0x7fdcc800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcd4ff9700 1 -- 172.21.15.12:0/795432775 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8004a30 con 0x7fdcd8100060 2024-05-09T00:00:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcd4ff9700 1 -- 172.21.15.12:0/795432775 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcc800cc80 con 0x7fdcd8100060 2024-05-09T00:00:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcdde7f700 1 -- 172.21.15.12:0/795432775 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd81a40a0 con 0x7fdcd8100060 2024-05-09T00:00:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcd4ff9700 1 -- 172.21.15.12:0/795432775 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80054e0 con 0x7fdcd8100060 2024-05-09T00:00:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.840+0000 7fdcdde7f700 1 -- 172.21.15.12:0/795432775 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd81a4580 con 0x7fdcd8100060 2024-05-09T00:00:46.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.842+0000 7fdcd4ff9700 1 -- 172.21.15.12:0/795432775 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdcc8004b90 con 0x7fdcd8100060 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.843+0000 7fdcd4ff9700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdcc00a1290 0x7fdcc00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.843+0000 7fdcd4ff9700 1 -- 172.21.15.12:0/795432775 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdcc8014070 con 0x7fdcd8100060 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.843+0000 7fdcd6ffd700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdcc00a1290 0x7fdcc00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.844+0000 7fdcd6ffd700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdcc00a1290 0x7fdcc00a3750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8198a00 tx=0x7fdccc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.844+0000 7fdcea039b80 1 Processor -- start 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.844+0000 7fdcea039b80 1 -- start start 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9210 0x55d64b8f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9b30 0x55d64b8f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f3ca0 0x55d64b8f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcea039b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b846150 con 0x55d64b8f3ca0 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcea039b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b843b10 con 0x55d64b8f9210 2024-05-09T00:00:46.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcea039b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b83f400 con 0x55d64b8f9b30 2024-05-09T00:00:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9b30 0x55d64b8f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9b30 0x55d64b8f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33514/0 (socket says 172.21.15.12:33514) 2024-05-09T00:00:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd6ffd700 1 -- 172.21.15.12:0/489817340 learned_addr learned my addr 172.21.15.12:0/489817340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd7fff700 1 --2- 172.21.15.12:0/489817340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f3ca0 0x55d64b8f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd77fe700 1 --2- 172.21.15.12:0/489817340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9210 0x55d64b8f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd6ffd700 1 -- 172.21.15.12:0/489817340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9210 msgr2=0x55d64b8f95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd6ffd700 1 --2- 172.21.15.12:0/489817340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9210 0x55d64b8f95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd6ffd700 1 -- 172.21.15.12:0/489817340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f3ca0 msgr2=0x55d64b8f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd6ffd700 1 --2- 172.21.15.12:0/489817340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f3ca0 0x55d64b8f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd6ffd700 1 -- 172.21.15.12:0/489817340 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d64b6c03b0 con 0x55d64b8f9b30 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.845+0000 7fdcd7fff700 1 --2- 172.21.15.12:0/489817340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f3ca0 0x55d64b8f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.846+0000 7fdcd6ffd700 1 --2- 172.21.15.12:0/489817340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9b30 0x55d64b8f3760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00e700 tx=0x7fdccc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45039dbcccc45b60 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.846+0000 7fdcbdffb700 1 -- 172.21.15.12:0/489817340 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004230 con 0x55d64b8f9b30 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.846+0000 7fdcbdffb700 1 -- 172.21.15.12:0/489817340 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdccc004390 con 0x55d64b8f9b30 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.846+0000 7fdcbdffb700 1 -- 172.21.15.12:0/489817340 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc01e850 con 0x55d64b8f9b30 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.847+0000 7fdcea039b80 1 -- 172.21.15.12:0/489817340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9b30 msgr2=0x55d64b8f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.847+0000 7fdcea039b80 1 --2- 172.21.15.12:0/489817340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9b30 0x55d64b8f3760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00e700 tx=0x7fdccc014000 comp rx=0 tx=0).stop 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.847+0000 7fdcea039b80 1 -- 172.21.15.12:0/489817340 shutdown_connections 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.847+0000 7fdcea039b80 1 --2- 172.21.15.12:0/489817340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f3ca0 0x55d64b8f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.847+0000 7fdcea039b80 1 --2- 172.21.15.12:0/489817340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9210 0x55d64b8f95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.847+0000 7fdcea039b80 1 --2- 172.21.15.12:0/489817340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9b30 0x55d64b8f3760 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.847+0000 7fdcea039b80 1 -- 172.21.15.12:0/489817340 >> 172.21.15.12:0/489817340 conn(0x55d64b84d210 msgr2=0x55d64b8e9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.847+0000 7fdcea039b80 1 -- 172.21.15.12:0/489817340 shutdown_connections 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.847+0000 7fdcea039b80 1 -- 172.21.15.12:0/489817340 wait complete. 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.847+0000 7fdcea039b80 1 -- start start 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcea039b80 1 Processor -- start 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f3ca0 0x55d64b8fcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9210 0x55d64b8fd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8fda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcea039b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b7db560 con 0x55d64b8f9b30 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcea039b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b7db200 con 0x55d64b8f3ca0 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcea039b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b6c26a0 con 0x55d64b8f9210 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcea039b80 10 client.0 _ll_register_callbacks cb 0x55d64b7e4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcd7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcd7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8fda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39094/0 (socket says 172.21.15.12:39094) 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcd7fff700 1 -- 172.21.15.12:0/2688238973 learned_addr learned my addr 172.21.15.12:0/2688238973 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcd6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9210 0x55d64b8fd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcd7fff700 1 -- 172.21.15.12:0/2688238973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9210 msgr2=0x55d64b8fd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcd7fff700 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9210 0x55d64b8fd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcd7fff700 1 -- 172.21.15.12:0/2688238973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f3ca0 msgr2=0x55d64b8fcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcd7fff700 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f3ca0 0x55d64b8fcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcd7fff700 1 -- 172.21.15.12:0/2688238973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc017070 con 0x55d64b8f9b30 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.848+0000 7fdcd6ffd700 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9210 0x55d64b8fd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.849+0000 7fdcd7fff700 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8fda40 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0007cb0 tx=0x7fdcd0007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796243a1df053795 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.849+0000 7fdcb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.849+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/2688238973 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000e040 con 0x55d64b8f9b30 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.849+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/2688238973 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdcd0006000 con 0x55d64b8f9b30 2024-05-09T00:00:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.849+0000 7fdcea039b80 1 -- 172.21.15.12:0/2688238973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d64b798270 con 0x55d64b8f9b30 2024-05-09T00:00:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.849+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/2688238973 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcd0009ac0 con 0x55d64b8f9b30 2024-05-09T00:00:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.849+0000 7fdcea039b80 10 client.19515 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.849+0000 7fdcea039b80 1 -- 172.21.15.12:0/2688238973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55d64b66cb60 con 0x55d64b8f9b30 2024-05-09T00:00:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.849+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/2688238973 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d64b798270 con 0x55d64b8f9b30 2024-05-09T00:00:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.850+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/2688238973 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdcd002b070 con 0x55d64b8f9b30 2024-05-09T00:00:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.850+0000 7fdcea039b80 10 client.19515 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:46.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.850+0000 7fdcea039b80 10 client.19515 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:00:46.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.850+0000 7fdcea039b80 20 client.19515 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:46.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.850+0000 7fdcea039b80 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55d64b90d5d0 0x55d64b90f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.850+0000 7fdcea039b80 4 client.19515 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:46.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.850+0000 7fdcea039b80 1 -- 172.21.15.12:0/2688238973 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d64b8be810 con 0x55d64b90d5d0 2024-05-09T00:00:46.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.850+0000 7fdcd77fe700 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55d64b90d5d0 0x55d64b90f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.850+0000 7fdcd77fe700 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55d64b90d5d0 0x55d64b90f9f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0e3d1ff3652f358 server_cookie=ae507a4c11dafdf6 in_seq=0 out_seq=0 2024-05-09T00:00:46.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.850+0000 7fdcb6ffd700 10 client.19515 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:00:46.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.852+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/2688238973 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d64b8be810 con 0x55d64b90d5d0 2024-05-09T00:00:46.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.852+0000 7fdcb6ffd700 10 client.19515 handle_command_reply: tid=0 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.852+0000 7fdcea039b80 1 client.19515 shutdown 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.852+0000 7fdcea039b80 1 -- 172.21.15.12:0/2688238973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 msgr2=0x55d64b8fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcea039b80 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8fda40 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0007cb0 tx=0x7fdcd0007ce0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcea039b80 1 -- 172.21.15.12:0/2688238973 shutdown_connections 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcea039b80 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55d64b90d5d0 0x55d64b90f9f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcea039b80 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8fda40 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcea039b80 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f3ca0 0x55d64b8fcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcb6ffd700 0 client.19515 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcea039b80 1 --2- 172.21.15.12:0/2688238973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f9210 0x55d64b8fd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcea039b80 1 -- 172.21.15.12:0/2688238973 >> 172.21.15.12:0/2688238973 conn(0x55d64b84d210 msgr2=0x55d64b8e95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcea039b80 1 -- 172.21.15.12:0/2688238973 shutdown_connections 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcea039b80 1 -- 172.21.15.12:0/2688238973 wait complete. 2024-05-09T00:00:46.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.853+0000 7fdcea039b80 20 client.19515 trim_cache size 0 max 16384 2024-05-09T00:00:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.868+0000 7fdcea039b80 1 Processor -- start 2024-05-09T00:00:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.868+0000 7fdcea039b80 1 -- start start 2024-05-09T00:00:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b909030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9210 0x55d64b8f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8e9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcea039b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b90acb0 con 0x55d64b8f9b30 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcea039b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b8e6dc0 con 0x55d64b8f9210 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcea039b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b90a1b0 con 0x55d64b8f3ca0 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8e9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b909030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8e9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39110/0 (socket says 172.21.15.12:39110) 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd7fff700 1 -- 172.21.15.12:0/4288160624 learned_addr learned my addr 172.21.15.12:0/4288160624 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd6ffd700 1 --2- 172.21.15.12:0/4288160624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9210 0x55d64b8f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd7fff700 1 -- 172.21.15.12:0/4288160624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 msgr2=0x55d64b909030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd7fff700 1 --2- 172.21.15.12:0/4288160624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b909030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd7fff700 1 -- 172.21.15.12:0/4288160624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9210 msgr2=0x55d64b8f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd7fff700 1 --2- 172.21.15.12:0/4288160624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9210 0x55d64b8f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd7fff700 1 -- 172.21.15.12:0/4288160624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d64b805560 con 0x55d64b8f9b30 2024-05-09T00:00:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd77fe700 1 --2- 172.21.15.12:0/4288160624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b909030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.869+0000 7fdcd7fff700 1 --2- 172.21.15.12:0/4288160624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8e9d20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009960 tx=0x7fdcd000afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cdec62fb7db2fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.870+0000 7fdcbe7fc700 1 -- 172.21.15.12:0/4288160624 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000e030 con 0x55d64b8f9b30 2024-05-09T00:00:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.870+0000 7fdcbe7fc700 1 -- 172.21.15.12:0/4288160624 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcd0020070 con 0x55d64b8f9b30 2024-05-09T00:00:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.870+0000 7fdcbe7fc700 1 -- 172.21.15.12:0/4288160624 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000b990 con 0x55d64b8f9b30 2024-05-09T00:00:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.871+0000 7fdcea039b80 1 -- 172.21.15.12:0/4288160624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 msgr2=0x55d64b8e9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.871+0000 7fdcea039b80 1 --2- 172.21.15.12:0/4288160624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8e9d20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009960 tx=0x7fdcd000afc0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.871+0000 7fdcea039b80 1 -- 172.21.15.12:0/4288160624 shutdown_connections 2024-05-09T00:00:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.871+0000 7fdcea039b80 1 --2- 172.21.15.12:0/4288160624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9b30 0x55d64b8e9d20 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.871+0000 7fdcea039b80 1 --2- 172.21.15.12:0/4288160624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b909030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.871+0000 7fdcea039b80 1 --2- 172.21.15.12:0/4288160624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9210 0x55d64b8f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.871+0000 7fdcea039b80 1 -- 172.21.15.12:0/4288160624 >> 172.21.15.12:0/4288160624 conn(0x55d64b84d210 msgr2=0x55d64b8f80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.871+0000 7fdcea039b80 1 -- 172.21.15.12:0/4288160624 shutdown_connections 2024-05-09T00:00:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.871+0000 7fdcea039b80 1 -- 172.21.15.12:0/4288160624 wait complete. 2024-05-09T00:00:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.871+0000 7fdcea039b80 1 -- start start 2024-05-09T00:00:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcea039b80 1 Processor -- start 2024-05-09T00:00:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b9037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9210 0x55d64b903ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcea039b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9b30 0x55d64b904220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcea039b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b962ad0 con 0x55d64b8f9210 2024-05-09T00:00:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcea039b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b962c10 con 0x55d64b8f9b30 2024-05-09T00:00:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcea039b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64b8ec330 con 0x55d64b8f3ca0 2024-05-09T00:00:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcea039b80 10 client.0 _ll_register_callbacks cb 0x55d64b7e4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcd77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b9037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcd7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9b30 0x55d64b904220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcd6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9210 0x55d64b903ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcd77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b9037a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33530/0 (socket says 172.21.15.12:33530) 2024-05-09T00:00:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.872+0000 7fdcd77fe700 1 -- 172.21.15.12:0/1930917328 learned_addr learned my addr 172.21.15.12:0/1930917328 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcd77fe700 1 -- 172.21.15.12:0/1930917328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9b30 msgr2=0x55d64b904220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcd77fe700 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9b30 0x55d64b904220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcd77fe700 1 -- 172.21.15.12:0/1930917328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9210 msgr2=0x55d64b903ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcd77fe700 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9210 0x55d64b903ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcd77fe700 1 -- 172.21.15.12:0/1930917328 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd81a4580 con 0x55d64b8f3ca0 2024-05-09T00:00:46.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcd77fe700 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b9037a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80ca560 tx=0x7fdcc8099b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f0eea333c2985e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:46.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:46.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/1930917328 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc809dcf0 con 0x55d64b8f3ca0 2024-05-09T00:00:46.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/1930917328 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdcc80a2340 con 0x55d64b8f3ca0 2024-05-09T00:00:46.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcea039b80 1 -- 172.21.15.12:0/1930917328 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdccc004390 con 0x55d64b8f3ca0 2024-05-09T00:00:46.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/1930917328 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcc80ca780 con 0x55d64b8f3ca0 2024-05-09T00:00:46.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcea039b80 10 client.29242 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:46.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.873+0000 7fdcea039b80 1 -- 172.21.15.12:0/1930917328 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55d64b66cb60 con 0x55d64b8f3ca0 2024-05-09T00:00:46.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.874+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/1930917328 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdccc004390 con 0x55d64b8f3ca0 2024-05-09T00:00:46.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.874+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/1930917328 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdcc809a120 con 0x55d64b8f3ca0 2024-05-09T00:00:46.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.875+0000 7fdcea039b80 10 client.29242 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:46.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.876+0000 7fdcea039b80 10 client.29242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:00:46.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.876+0000 7fdcea039b80 20 client.29242 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:46.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.876+0000 7fdcea039b80 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55d64b8edaf0 0x55d64b8eff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:46.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.876+0000 7fdcea039b80 4 client.29242 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:00:46.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.876+0000 7fdcea039b80 1 -- 172.21.15.12:0/1930917328 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d64b837ec0 con 0x55d64b8edaf0 2024-05-09T00:00:46.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.876+0000 7fdcd6ffd700 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55d64b8edaf0 0x55d64b8eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:46.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.876+0000 7fdcb6ffd700 10 client.29242 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:00:46.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.876+0000 7fdcd6ffd700 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55d64b8edaf0 0x55d64b8eff10 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cde4082fb41e0fe9 server_cookie=c74bebf8f5b5fde5 in_seq=0 out_seq=0 2024-05-09T00:00:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcb6ffd700 1 -- 172.21.15.12:0/1930917328 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d64b837ec0 con 0x55d64b8edaf0 2024-05-09T00:00:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcb6ffd700 10 client.29242 handle_command_reply: tid=0 2024-05-09T00:00:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 client.29242 shutdown 2024-05-09T00:00:46.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 -- 172.21.15.12:0/1930917328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 msgr2=0x55d64b9037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b9037a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80ca560 tx=0x7fdcc8099b20 comp rx=0 tx=0).stop 2024-05-09T00:00:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 -- 172.21.15.12:0/1930917328 shutdown_connections 2024-05-09T00:00:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55d64b8edaf0 0x55d64b8eff10 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d64b8f9b30 0x55d64b904220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d64b8f3ca0 0x55d64b9037a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 --2- 172.21.15.12:0/1930917328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d64b8f9210 0x55d64b903ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcb6ffd700 0 client.29242 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:00:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 -- 172.21.15.12:0/1930917328 >> 172.21.15.12:0/1930917328 conn(0x55d64b84d210 msgr2=0x55d64b8fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 -- 172.21.15.12:0/1930917328 shutdown_connections 2024-05-09T00:00:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 1 -- 172.21.15.12:0/1930917328 wait complete. 2024-05-09T00:00:46.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.880+0000 7fdcea039b80 20 client.29242 trim_cache size 0 max 16384 2024-05-09T00:00:46.885 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:00:46.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 -- 172.21.15.12:0/795432775 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdcc00a1290 msgr2=0x7fdcc00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdcc00a1290 0x7fdcc00a3750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8198a00 tx=0x7fdccc009260 comp rx=0 tx=0).stop 2024-05-09T00:00:46.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 -- 172.21.15.12:0/795432775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd8100060 msgr2=0x7fdcd81974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:46.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd8100060 0x7fdcd81974b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800e9c0 tx=0x7fdcc800ecd0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 -- 172.21.15.12:0/795432775 shutdown_connections 2024-05-09T00:00:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdcc00a1290 0x7fdcc00a3750 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcd819bd90 0x7fdcd819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcd8100060 0x7fdcd81974b0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 --2- 172.21.15.12:0/795432775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd8105160 0x7fdcd81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:46.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 -- 172.21.15.12:0/795432775 >> 172.21.15.12:0/795432775 conn(0x7fdcd8076f40 msgr2=0x7fdcd8068310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 -- 172.21.15.12:0/795432775 shutdown_connections 2024-05-09T00:00:46.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:46.881+0000 7fdcdde7f700 1 -- 172.21.15.12:0/795432775 wait complete. 2024-05-09T00:00:46.891 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:00:47.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:46 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/373576501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:00:47.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:46 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/373576501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:00:47.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.111+0000 7f13b9842700 1 -- 172.21.15.12:0/842069738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b4100220 msgr2=0x7f13b410ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.111+0000 7f13b9842700 1 --2- 172.21.15.12:0/842069738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b4100220 0x7f13b410ddf0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00b0a0 tx=0x7f13ac00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.111+0000 7f13b9842700 1 -- 172.21.15.12:0/842069738 shutdown_connections 2024-05-09T00:00:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.111+0000 7f13b9842700 1 --2- 172.21.15.12:0/842069738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b4100220 0x7f13b410ddf0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.111+0000 7f13b9842700 1 --2- 172.21.15.12:0/842069738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13b4105f40 0x7f13b4106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.111+0000 7f13b9842700 1 --2- 172.21.15.12:0/842069738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b40ff860 0x7f13b40ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.111+0000 7f13b9842700 1 -- 172.21.15.12:0/842069738 >> 172.21.15.12:0/842069738 conn(0x7f13b40751c0 msgr2=0x7f13b40755d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.111+0000 7f13b9842700 1 -- 172.21.15.12:0/842069738 shutdown_connections 2024-05-09T00:00:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.111+0000 7f13b9842700 1 -- 172.21.15.12:0/842069738 wait complete. 2024-05-09T00:00:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b9842700 1 Processor -- start 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b9842700 1 -- start start 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b9842700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b40ff860 0x7f13b4199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b9842700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13b4100220 0x7f13b4199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b9842700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b4105f40 0x7f13b419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b9842700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b419d5a0 con 0x7f13b4105f40 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b9842700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b419d710 con 0x7f13b4100220 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b9842700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b419d8d0 con 0x7f13b40ff860 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b4105f40 0x7f13b419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b4105f40 0x7f13b419cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39134/0 (socket says 172.21.15.12:39134) 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b37fe700 1 -- 172.21.15.12:0/2536106412 learned_addr learned my addr 172.21.15.12:0/2536106412 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b27fc700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13b4100220 0x7f13b4199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.112+0000 7f13b2ffd700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b40ff860 0x7f13b4199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b37fe700 1 -- 172.21.15.12:0/2536106412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b40ff860 msgr2=0x7f13b4199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b37fe700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b40ff860 0x7f13b4199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b37fe700 1 -- 172.21.15.12:0/2536106412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13b4100220 msgr2=0x7f13b4199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b37fe700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13b4100220 0x7f13b4199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b37fe700 1 -- 172.21.15.12:0/2536106412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac009d00 con 0x7f13b4105f40 2024-05-09T00:00:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b37fe700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b4105f40 0x7f13b419cfa0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00b070 tx=0x7f13ac0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b8840700 1 -- 172.21.15.12:0/2536106412 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac00e040 con 0x7f13b4105f40 2024-05-09T00:00:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b8840700 1 -- 172.21.15.12:0/2536106412 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ac01d470 con 0x7f13b4105f40 2024-05-09T00:00:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b9842700 1 -- 172.21.15.12:0/2536106412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b419db50 con 0x7f13b4105f40 2024-05-09T00:00:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b8840700 1 -- 172.21.15.12:0/2536106412 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0052a0 con 0x7f13b4105f40 2024-05-09T00:00:47.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.113+0000 7f13b9842700 1 -- 172.21.15.12:0/2536106412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b41a6510 con 0x7f13b4105f40 2024-05-09T00:00:47.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.116+0000 7f13b8840700 1 -- 172.21.15.12:0/2536106412 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f13ac01d5e0 con 0x7f13b4105f40 2024-05-09T00:00:47.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.117+0000 7f13b8840700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f139c0a1290 0x7f139c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.117+0000 7f13b2ffd700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f139c0a1290 0x7f139c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.117+0000 7f13b8840700 1 -- 172.21.15.12:0/2536106412 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f13ac0cdf80 con 0x7f13b4105f40 2024-05-09T00:00:47.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.117+0000 7f13b2ffd700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f139c0a1290 0x7f139c0a3750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009c60 tx=0x7f13a4009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:47.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.118+0000 7f13c59fcb80 1 Processor -- start 2024-05-09T00:00:47.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.118+0000 7f13c59fcb80 1 -- start start 2024-05-09T00:00:47.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.118+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac112e00 0x55a3ac1131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.118+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac113720 0x55a3ac10d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.118+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.118+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ac064150 con 0x55a3ac10d890 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.118+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ac061b10 con 0x55a3ac112e00 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.118+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ac05d400 con 0x55a3ac113720 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac113720 0x55a3ac10d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac112e00 0x55a3ac1131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac113720 0x55a3ac10d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33564/0 (socket says 172.21.15.12:33564) 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b27fc700 1 -- 172.21.15.12:0/963305912 learned_addr learned my addr 172.21.15.12:0/963305912 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b27fc700 1 -- 172.21.15.12:0/963305912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac112e00 msgr2=0x55a3ac1131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b27fc700 1 --2- 172.21.15.12:0/963305912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac112e00 0x55a3ac1131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b27fc700 1 -- 172.21.15.12:0/963305912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 msgr2=0x55a3ac10dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:47.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b27fc700 1 --2- 172.21.15.12:0/963305912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b27fc700 1 -- 172.21.15.12:0/963305912 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3abede3b0 con 0x55a3ac113720 2024-05-09T00:00:47.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b37fe700 1 --2- 172.21.15.12:0/963305912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:47.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f13b27fc700 1 --2- 172.21.15.12:0/963305912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac113720 0x55a3ac10d350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009cf0 tx=0x7f13a8011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9434430a956c2419 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:47.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.119+0000 7f1399ffb700 1 -- 172.21.15.12:0/963305912 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8011d80 con 0x55a3ac113720 2024-05-09T00:00:47.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.120+0000 7f1399ffb700 1 -- 172.21.15.12:0/963305912 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13a8004510 con 0x55a3ac113720 2024-05-09T00:00:47.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.120+0000 7f1399ffb700 1 -- 172.21.15.12:0/963305912 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8005220 con 0x55a3ac113720 2024-05-09T00:00:47.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.120+0000 7f13c59fcb80 1 -- 172.21.15.12:0/963305912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac113720 msgr2=0x55a3ac10d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.120+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/963305912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac113720 0x55a3ac10d350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009cf0 tx=0x7f13a8011440 comp rx=0 tx=0).stop 2024-05-09T00:00:47.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.122+0000 7f13c59fcb80 1 -- 172.21.15.12:0/963305912 shutdown_connections 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.122+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/963305912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.122+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/963305912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac112e00 0x55a3ac1131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.122+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/963305912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac113720 0x55a3ac10d350 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.122+0000 7f13c59fcb80 1 -- 172.21.15.12:0/963305912 >> 172.21.15.12:0/963305912 conn(0x55a3ac06b210 msgr2=0x55a3ac107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.122+0000 7f13c59fcb80 1 -- 172.21.15.12:0/963305912 shutdown_connections 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.122+0000 7f13c59fcb80 1 -- 172.21.15.12:0/963305912 wait complete. 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.122+0000 7f13c59fcb80 1 -- start start 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13c59fcb80 1 Processor -- start 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac116bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac117110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac113720 0x55a3ac117650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3abff9560 con 0x55a3ac10d890 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3abff9200 con 0x55a3ac113720 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3abee06a0 con 0x55a3ac112e00 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13c59fcb80 10 client.0 _ll_register_callbacks cb 0x55a3ac002b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13b27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac117110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13b27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac117110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33578/0 (socket says 172.21.15.12:33578) 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13b27fc700 1 -- 172.21.15.12:0/677757737 learned_addr learned my addr 172.21.15.12:0/677757737 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13b37fe700 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac113720 0x55a3ac117650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13b2ffd700 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac116bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13b27fc700 1 -- 172.21.15.12:0/677757737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac113720 msgr2=0x55a3ac117650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13b27fc700 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac113720 0x55a3ac117650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13b27fc700 1 -- 172.21.15.12:0/677757737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 msgr2=0x55a3ac116bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13b27fc700 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac116bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.123+0000 7f13b27fc700 1 -- 172.21.15.12:0/677757737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b41a6510 con 0x55a3ac112e00 2024-05-09T00:00:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.124+0000 7f13b27fc700 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac117110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009cc0 tx=0x7f13a8005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a642e179c8b9115a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.124+0000 7f1392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.124+0000 7f1392ffd700 1 -- 172.21.15.12:0/677757737 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8004260 con 0x55a3ac112e00 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.124+0000 7f1392ffd700 1 -- 172.21.15.12:0/677757737 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f13a8003680 con 0x55a3ac112e00 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.124+0000 7f13c59fcb80 1 -- 172.21.15.12:0/677757737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3abfb6270 con 0x55a3ac112e00 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.124+0000 7f1392ffd700 1 -- 172.21.15.12:0/677757737 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13a8003860 con 0x55a3ac112e00 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.124+0000 7f13c59fcb80 10 client.29254 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.124+0000 7f13c59fcb80 1 -- 172.21.15.12:0/677757737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f13a8009920 con 0x55a3ac112e00 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.124+0000 7f1392ffd700 1 -- 172.21.15.12:0/677757737 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3abfb6270 con 0x55a3ac112e00 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.124+0000 7f1392ffd700 1 -- 172.21.15.12:0/677757737 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f13a80053b0 con 0x55a3ac112e00 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.125+0000 7f13c59fcb80 10 client.29254 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.125+0000 7f13c59fcb80 10 client.29254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.125+0000 7f13c59fcb80 20 client.29254 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.125+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a3ac127150 0x55a3ac129570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.125+0000 7f13c59fcb80 4 client.29254 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.125+0000 7f13c59fcb80 1 -- 172.21.15.12:0/677757737 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3ac0dc810 con 0x55a3ac127150 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.125+0000 7f13b2ffd700 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a3ac127150 0x55a3ac129570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.125+0000 7f1392ffd700 10 client.29254 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.125+0000 7f13b2ffd700 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a3ac127150 0x55a3ac129570 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60dfb44ef9eff5cd server_cookie=23b1ebe9dd2755e8 in_seq=0 out_seq=0 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.127+0000 7f1392ffd700 1 -- 172.21.15.12:0/677757737 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a3ac0dc810 con 0x55a3ac127150 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.127+0000 7f1392ffd700 10 client.29254 handle_command_reply: tid=0 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.127+0000 7f13c59fcb80 1 client.29254 shutdown 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.127+0000 7f13c59fcb80 1 -- 172.21.15.12:0/677757737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 msgr2=0x55a3ac117110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.127+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac117110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009cc0 tx=0x7f13a8005e10 comp rx=0 tx=0).stop 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.128+0000 7f13c59fcb80 1 -- 172.21.15.12:0/677757737 shutdown_connections 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.128+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a3ac127150 0x55a3ac129570 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.128+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac113720 0x55a3ac117650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.128+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.128+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/677757737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac117110 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.128+0000 7f1392ffd700 0 client.29254 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.128+0000 7f13c59fcb80 1 -- 172.21.15.12:0/677757737 >> 172.21.15.12:0/677757737 conn(0x55a3ac06b210 msgr2=0x55a3ac1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.129+0000 7f13c59fcb80 1 -- 172.21.15.12:0/677757737 shutdown_connections 2024-05-09T00:00:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.129+0000 7f13c59fcb80 1 -- 172.21.15.12:0/677757737 wait complete. 2024-05-09T00:00:47.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.129+0000 7f13c59fcb80 20 client.29254 trim_cache size 0 max 16384 2024-05-09T00:00:47.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.144+0000 7f13c59fcb80 1 Processor -- start 2024-05-09T00:00:47.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.144+0000 7f13c59fcb80 1 -- start start 2024-05-09T00:00:47.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac107d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac113720 0x55a3ac117470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ac12e7b0 con 0x55a3ac10d890 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ac1089e0 con 0x55a3ac113720 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ac119df0 con 0x55a3ac112e00 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39170/0 (socket says 172.21.15.12:39170) 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b2ffd700 1 -- 172.21.15.12:0/887323937 learned_addr learned my addr 172.21.15.12:0/887323937 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b27fc700 1 --2- 172.21.15.12:0/887323937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b37fe700 1 --2- 172.21.15.12:0/887323937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac113720 0x55a3ac117470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b27fc700 1 -- 172.21.15.12:0/887323937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac113720 msgr2=0x55a3ac117470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b27fc700 1 --2- 172.21.15.12:0/887323937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac113720 0x55a3ac117470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b27fc700 1 -- 172.21.15.12:0/887323937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 msgr2=0x55a3ac10d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b27fc700 1 --2- 172.21.15.12:0/887323937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b27fc700 1 -- 172.21.15.12:0/887323937 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3abe8ab60 con 0x55a3ac112e00 2024-05-09T00:00:47.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b2ffd700 1 --2- 172.21.15.12:0/887323937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.145+0000 7f13b27fc700 1 --2- 172.21.15.12:0/887323937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac107d20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009c10 tx=0x7f13a8011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f9d91ac05192c83 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.146+0000 7f139a7fc700 1 -- 172.21.15.12:0/887323937 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8011840 con 0x55a3ac112e00 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.146+0000 7f139a7fc700 1 -- 172.21.15.12:0/887323937 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13a8011c30 con 0x55a3ac112e00 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.146+0000 7f139a7fc700 1 -- 172.21.15.12:0/887323937 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8005060 con 0x55a3ac112e00 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.146+0000 7f13c59fcb80 1 -- 172.21.15.12:0/887323937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 msgr2=0x55a3ac107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.147+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/887323937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac107d20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009c10 tx=0x7f13a8011440 comp rx=0 tx=0).stop 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.147+0000 7f13c59fcb80 1 -- 172.21.15.12:0/887323937 shutdown_connections 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.147+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/887323937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac113720 0x55a3ac117470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.147+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/887323937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac10d890 0x55a3ac10d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.147+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/887323937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac112e00 0x55a3ac107d20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.147+0000 7f13c59fcb80 1 -- 172.21.15.12:0/887323937 >> 172.21.15.12:0/887323937 conn(0x55a3ac06b210 msgr2=0x55a3ac1154a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.147+0000 7f13c59fcb80 1 -- 172.21.15.12:0/887323937 shutdown_connections 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.147+0000 7f13c59fcb80 1 -- 172.21.15.12:0/887323937 wait complete. 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.147+0000 7f13c59fcb80 1 -- start start 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.147+0000 7f13c59fcb80 1 Processor -- start 2024-05-09T00:00:47.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac10d890 0x55a3ac11e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac112e00 0x55a3ac11c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13c59fcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac113720 0x55a3ac11c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ac11cf70 con 0x55a3ac113720 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ac17c650 con 0x55a3ac112e00 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13c59fcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ac17c790 con 0x55a3ac10d890 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13c59fcb80 10 client.0 _ll_register_callbacks cb 0x55a3ac002b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac10d890 0x55a3ac11e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac113720 0x55a3ac11c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac10d890 0x55a3ac11e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33598/0 (socket says 172.21.15.12:33598) 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b2ffd700 1 -- 172.21.15.12:0/3810964077 learned_addr learned my addr 172.21.15.12:0/3810964077 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b27fc700 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac112e00 0x55a3ac11c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b2ffd700 1 -- 172.21.15.12:0/3810964077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac112e00 msgr2=0x55a3ac11c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b2ffd700 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac112e00 0x55a3ac11c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b2ffd700 1 -- 172.21.15.12:0/3810964077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac113720 msgr2=0x55a3ac11c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:47.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b2ffd700 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac113720 0x55a3ac11c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b2ffd700 1 -- 172.21.15.12:0/3810964077 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a4017070 con 0x55a3ac10d890 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f13b2ffd700 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac10d890 0x55a3ac11e6c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f13a402b1c0 tx=0x7f13a40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcffe507142ee052 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.148+0000 7f1392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f1392ffd700 1 -- 172.21.15.12:0/3810964077 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4022040 con 0x55a3ac10d890 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f1392ffd700 1 -- 172.21.15.12:0/3810964077 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f13a4028820 con 0x55a3ac10d890 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f13c59fcb80 1 -- 172.21.15.12:0/3810964077 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3ac055ec0 con 0x55a3ac10d890 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f1392ffd700 1 -- 172.21.15.12:0/3810964077 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13a40289e0 con 0x55a3ac10d890 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f13c59fcb80 10 client.29260 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f13c59fcb80 1 -- 172.21.15.12:0/3810964077 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f13a8009920 con 0x55a3ac10d890 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f1392ffd700 1 -- 172.21.15.12:0/3810964077 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13a4040020 con 0x55a3ac10d890 2024-05-09T00:00:47.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f1392ffd700 1 -- 172.21.15.12:0/3810964077 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f13a401db80 con 0x55a3ac10d890 2024-05-09T00:00:47.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f13c59fcb80 10 client.29260 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:47.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f13c59fcb80 10 client.29260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:47.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f13c59fcb80 20 client.29260 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:47.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a3ac17e060 0x55a3ac180480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f13c59fcb80 4 client.29260 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:00:47.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f13c59fcb80 1 -- 172.21.15.12:0/3810964077 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a3abeabcf0 con 0x55a3ac17e060 2024-05-09T00:00:47.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.149+0000 7f13b27fc700 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a3ac17e060 0x55a3ac180480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.150+0000 7f13b27fc700 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a3ac17e060 0x55a3ac180480 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b3424166b8b45be server_cookie=3becb147f123d794 in_seq=0 out_seq=0 2024-05-09T00:00:47.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.150+0000 7f1392ffd700 10 client.29260 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.155+0000 7f1392ffd700 1 -- 172.21.15.12:0/3810964077 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a3abeabcf0 con 0x55a3ac17e060 2024-05-09T00:00:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.155+0000 7f1392ffd700 10 client.29260 handle_command_reply: tid=0 2024-05-09T00:00:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.155+0000 7f13c59fcb80 1 client.29260 shutdown 2024-05-09T00:00:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.155+0000 7f13c59fcb80 1 -- 172.21.15.12:0/3810964077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac10d890 msgr2=0x55a3ac11e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.155+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac10d890 0x55a3ac11e6c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f13a402b1c0 tx=0x7f13a40047c0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.155+0000 7f13c59fcb80 1 -- 172.21.15.12:0/3810964077 shutdown_connections 2024-05-09T00:00:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.155+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3ac113720 0x55a3ac11c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.155+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a3ac10d890 0x55a3ac11e6c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.155+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a3ac17e060 0x55a3ac180480 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.155+0000 7f13c59fcb80 1 --2- 172.21.15.12:0/3810964077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a3ac112e00 0x55a3ac11c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.156+0000 7f13c59fcb80 1 -- 172.21.15.12:0/3810964077 >> 172.21.15.12:0/3810964077 conn(0x55a3ac06b210 msgr2=0x55a3ac111ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:47.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.156+0000 7f1392ffd700 0 client.29260 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:47.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.156+0000 7f13c59fcb80 1 -- 172.21.15.12:0/3810964077 shutdown_connections 2024-05-09T00:00:47.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.156+0000 7f13c59fcb80 1 -- 172.21.15.12:0/3810964077 wait complete. 2024-05-09T00:00:47.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.156+0000 7f13c59fcb80 20 client.29260 trim_cache size 0 max 16384 2024-05-09T00:00:47.153 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:00:47.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.156+0000 7f13b9842700 1 -- 172.21.15.12:0/2536106412 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f139c0a1290 msgr2=0x7f139c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.156+0000 7f13b9842700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f139c0a1290 0x7f139c0a3750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009c60 tx=0x7f13a4009260 comp rx=0 tx=0).stop 2024-05-09T00:00:47.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.156+0000 7f13b9842700 1 -- 172.21.15.12:0/2536106412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b4105f40 msgr2=0x7f13b419cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.156+0000 7f13b9842700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b4105f40 0x7f13b419cfa0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00b070 tx=0x7f13ac0078e0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.156+0000 7f13b9842700 1 -- 172.21.15.12:0/2536106412 shutdown_connections 2024-05-09T00:00:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.157+0000 7f13b9842700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f139c0a1290 0x7f139c0a3750 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.157+0000 7f13b9842700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b4105f40 0x7f13b419cfa0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.157+0000 7f13b9842700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b40ff860 0x7f13b4199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.157+0000 7f13b9842700 1 --2- 172.21.15.12:0/2536106412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13b4100220 0x7f13b4199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.157+0000 7f13b9842700 1 -- 172.21.15.12:0/2536106412 >> 172.21.15.12:0/2536106412 conn(0x7f13b40751c0 msgr2=0x7f13b40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.157+0000 7f13b9842700 1 -- 172.21.15.12:0/2536106412 shutdown_connections 2024-05-09T00:00:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:47.157+0000 7f13b9842700 1 -- 172.21.15.12:0/2536106412 wait complete. 2024-05-09T00:00:47.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 29250 41.77 MB/sec warmup 42 sec latency 39.145 ms 2024-05-09T00:00:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:47 smithi012 ceph-mon[42503]: pgmap v460: 97 pgs: 97 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 36 MiB/s wr, 1.19k op/s 2024-05-09T00:00:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:47 smithi204 ceph-mon[47373]: pgmap v460: 97 pgs: 97 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 36 MiB/s wr, 1.19k op/s 2024-05-09T00:00:48.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:47 smithi160 ceph-mon[46152]: pgmap v460: 97 pgs: 97 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 36 MiB/s wr, 1.19k op/s 2024-05-09T00:00:48.166 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:00:48.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:00:48.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 30092 41.86 MB/sec warmup 43 sec latency 48.937 ms 2024-05-09T00:00:48.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.354+0000 7f7116927700 1 -- 172.21.15.12:0/3184906609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7110073340 msgr2=0x7f7110073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.354+0000 7f7116927700 1 --2- 172.21.15.12:0/3184906609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7110073340 0x7f7110073720 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f7100009990 tx=0x7f7100009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.354+0000 7f7116927700 1 -- 172.21.15.12:0/3184906609 shutdown_connections 2024-05-09T00:00:48.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.354+0000 7f7116927700 1 --2- 172.21.15.12:0/3184906609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7110111f00 0x7f71101142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.354+0000 7f7116927700 1 --2- 172.21.15.12:0/3184906609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7110073340 0x7f7110073720 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.354+0000 7f7116927700 1 --2- 172.21.15.12:0/3184906609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7110073c60 0x7f71101119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.354+0000 7f7116927700 1 -- 172.21.15.12:0/3184906609 >> 172.21.15.12:0/3184906609 conn(0x7f71100fc9e0 msgr2=0x7f71100fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.354+0000 7f7116927700 1 -- 172.21.15.12:0/3184906609 shutdown_connections 2024-05-09T00:00:48.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.354+0000 7f7116927700 1 -- 172.21.15.12:0/3184906609 wait complete. 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.355+0000 7f7116927700 1 Processor -- start 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.355+0000 7f7116927700 1 -- start start 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.355+0000 7f7116927700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7110073340 0x7f71101a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.355+0000 7f7116927700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7110073c60 0x7f71101a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.355+0000 7f7116927700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7110111f00 0x7f711019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.355+0000 7f7116927700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711019dec0 con 0x7f7110111f00 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.355+0000 7f7116927700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711019e030 con 0x7f7110073c60 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f7116927700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711019e1f0 con 0x7f7110073340 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f7114ec4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7110111f00 0x7f711019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f710f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7110073c60 0x7f71101a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f7114ec4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7110111f00 0x7f711019d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39192/0 (socket says 172.21.15.12:39192) 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f710f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7110073c60 0x7f71101a3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33058/0 (socket says 172.21.15.12:33058) 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f7114ec4700 1 -- 172.21.15.12:0/403125089 learned_addr learned my addr 172.21.15.12:0/403125089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f710ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7110073340 0x7f71101a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f7114ec4700 1 -- 172.21.15.12:0/403125089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7110073340 msgr2=0x7f71101a3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f7114ec4700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7110073340 0x7f71101a3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f7114ec4700 1 -- 172.21.15.12:0/403125089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7110073c60 msgr2=0x7f71101a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f7114ec4700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7110073c60 0x7f71101a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f7114ec4700 1 -- 172.21.15.12:0/403125089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71000095f0 con 0x7f7110111f00 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.356+0000 7f7114ec4700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7110111f00 0x7f711019d850 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f710800ef90 tx=0x7f710800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.357+0000 7f710d7fa700 1 -- 172.21.15.12:0/403125089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108018070 con 0x7f7110111f00 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.357+0000 7f7116927700 1 -- 172.21.15.12:0/403125089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711019e470 con 0x7f7110111f00 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.357+0000 7f710d7fa700 1 -- 172.21.15.12:0/403125089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f710800f040 con 0x7f7110111f00 2024-05-09T00:00:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.357+0000 7f710d7fa700 1 -- 172.21.15.12:0/403125089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710800cda0 con 0x7f7110111f00 2024-05-09T00:00:48.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.357+0000 7f7116927700 1 -- 172.21.15.12:0/403125089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71101aa710 con 0x7f7110111f00 2024-05-09T00:00:48.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.359+0000 7f710d7fa700 1 -- 172.21.15.12:0/403125089 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7108003a00 con 0x7f7110111f00 2024-05-09T00:00:48.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.360+0000 7f710d7fa700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70f80a1290 0x7f70f80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.360+0000 7f710d7fa700 1 -- 172.21.15.12:0/403125089 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f71080cc820 con 0x7f7110111f00 2024-05-09T00:00:48.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.361+0000 7f710ffff700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70f80a1290 0x7f70f80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.361+0000 7f710ffff700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70f80a1290 0x7f70f80a3750 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f710000b2e0 tx=0x7f7100005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.361+0000 7f7122ae1b80 1 Processor -- start 2024-05-09T00:00:48.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.361+0000 7f7122ae1b80 1 -- start start 2024-05-09T00:00:48.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 0x55b09e67f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e67f7a0 0x55b09e6793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e679910 0x55b09e679d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e5d01d0 con 0x55b09e67f7a0 2024-05-09T00:00:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e5cdb90 con 0x55b09e67ee80 2024-05-09T00:00:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e5c9480 con 0x55b09e679910 2024-05-09T00:00:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f710f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e67f7a0 0x55b09e6793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7114ec4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e679910 0x55b09e679d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f710ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 0x55b09e67f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f710f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e67f7a0 0x55b09e6793d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39202/0 (socket says 172.21.15.12:39202) 2024-05-09T00:00:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f710f7fe700 1 -- 172.21.15.12:0/3562074686 learned_addr learned my addr 172.21.15.12:0/3562074686 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7114ec4700 1 -- 172.21.15.12:0/3562074686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 msgr2=0x55b09e67f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7114ec4700 1 --2- 172.21.15.12:0/3562074686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 0x55b09e67f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7114ec4700 1 -- 172.21.15.12:0/3562074686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e67f7a0 msgr2=0x55b09e6793d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7114ec4700 1 --2- 172.21.15.12:0/3562074686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e67f7a0 0x55b09e6793d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f7114ec4700 1 -- 172.21.15.12:0/3562074686 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b09e44a430 con 0x55b09e679910 2024-05-09T00:00:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.362+0000 7f710f7fe700 1 --2- 172.21.15.12:0/3562074686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e67f7a0 0x55b09e6793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.363+0000 7f7114ec4700 1 --2- 172.21.15.12:0/3562074686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e679910 0x55b09e679d50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f71080c9f80 tx=0x7f7108095710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c93344fb6527e80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.363+0000 7f70f67fc700 1 -- 172.21.15.12:0/3562074686 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108099330 con 0x55b09e679910 2024-05-09T00:00:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.363+0000 7f70f67fc700 1 -- 172.21.15.12:0/3562074686 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f710801b320 con 0x55b09e679910 2024-05-09T00:00:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.363+0000 7f70f67fc700 1 -- 172.21.15.12:0/3562074686 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71080960e0 con 0x55b09e679910 2024-05-09T00:00:48.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.364+0000 7f7122ae1b80 1 -- 172.21.15.12:0/3562074686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e679910 msgr2=0x55b09e679d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.364+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/3562074686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e679910 0x55b09e679d50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f71080c9f80 tx=0x7f7108095710 comp rx=0 tx=0).stop 2024-05-09T00:00:48.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.364+0000 7f7122ae1b80 1 -- 172.21.15.12:0/3562074686 shutdown_connections 2024-05-09T00:00:48.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.364+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/3562074686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e679910 0x55b09e679d50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.364+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/3562074686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 0x55b09e67f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.364+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/3562074686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e67f7a0 0x55b09e6793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.364+0000 7f7122ae1b80 1 -- 172.21.15.12:0/3562074686 >> 172.21.15.12:0/3562074686 conn(0x55b09e5d7290 msgr2=0x55b09e673e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.364+0000 7f7122ae1b80 1 -- 172.21.15.12:0/3562074686 shutdown_connections 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.364+0000 7f7122ae1b80 1 -- 172.21.15.12:0/3562074686 wait complete. 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.364+0000 7f7122ae1b80 1 -- start start 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f7122ae1b80 1 Processor -- start 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 0x55b09e682c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 0x55b09e6831b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67f7a0 0x55b09e6836f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e5655e0 con 0x55b09e679910 2024-05-09T00:00:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e565280 con 0x55b09e67ee80 2024-05-09T00:00:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e44c720 con 0x55b09e67f7a0 2024-05-09T00:00:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f710ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 0x55b09e682c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f710ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 0x55b09e682c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39204/0 (socket says 172.21.15.12:39204) 2024-05-09T00:00:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f710ffff700 1 -- 172.21.15.12:0/1520668947 learned_addr learned my addr 172.21.15.12:0/1520668947 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f7122ae1b80 10 client.0 _ll_register_callbacks cb 0x55b09e56ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f7114ec4700 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67f7a0 0x55b09e6836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.365+0000 7f710f7fe700 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 0x55b09e6831b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f710ffff700 1 -- 172.21.15.12:0/1520668947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67f7a0 msgr2=0x55b09e6836f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f710ffff700 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67f7a0 0x55b09e6836f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f710ffff700 1 -- 172.21.15.12:0/1520668947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 msgr2=0x55b09e6831b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f710ffff700 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 0x55b09e6831b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f710ffff700 1 -- 172.21.15.12:0/1520668947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710400b960 con 0x55b09e679910 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f7114ec4700 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67f7a0 0x55b09e6836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f710f7fe700 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 0x55b09e6831b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f710ffff700 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 0x55b09e682c70 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f7100009990 tx=0x7f7100009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0c53ca3e72a193 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f70ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1520668947 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100004e80 con 0x55b09e679910 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1520668947 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f710000f460 con 0x55b09e679910 2024-05-09T00:00:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1520668947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b09e5222f0 con 0x55b09e679910 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1520668947 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f710000f620 con 0x55b09e679910 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f7122ae1b80 10 client.19551 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1520668947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f71101aa710 con 0x55b09e679910 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.366+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1520668947 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b09e5222f0 con 0x55b09e679910 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.367+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1520668947 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f710002f070 con 0x55b09e679910 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.367+0000 7f7122ae1b80 10 client.19551 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.367+0000 7f7122ae1b80 10 client.19551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.367+0000 7f7122ae1b80 20 client.19551 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.367+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b09e693450 0x55b09e695870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.367+0000 7f7122ae1b80 4 client.19551 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.367+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1520668947 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b09e648890 con 0x55b09e693450 2024-05-09T00:00:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.367+0000 7f710f7fe700 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b09e693450 0x55b09e695870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.367+0000 7f710f7fe700 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b09e693450 0x55b09e695870 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b59ef578d79a58f server_cookie=5b35d52669a3b4c9 in_seq=0 out_seq=0 2024-05-09T00:00:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.367+0000 7f70ef7fe700 10 client.19551 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.370+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1520668947 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b09e648890 con 0x55b09e693450 2024-05-09T00:00:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.370+0000 7f70ef7fe700 10 client.19551 handle_command_reply: tid=0 2024-05-09T00:00:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.371+0000 7f7122ae1b80 1 client.19551 shutdown 2024-05-09T00:00:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1520668947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 msgr2=0x55b09e682c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 0x55b09e682c70 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f7100009990 tx=0x7f7100009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1520668947 shutdown_connections 2024-05-09T00:00:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67f7a0 0x55b09e6836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 0x55b09e682c70 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b09e693450 0x55b09e695870 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1520668947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e67ee80 0x55b09e6831b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f70ef7fe700 0 client.19551 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1520668947 >> 172.21.15.12:0/1520668947 conn(0x55b09e5d7290 msgr2=0x55b09e673620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1520668947 shutdown_connections 2024-05-09T00:00:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1520668947 wait complete. 2024-05-09T00:00:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.372+0000 7f7122ae1b80 20 client.19551 trim_cache size 0 max 16384 2024-05-09T00:00:48.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.392+0000 7f7122ae1b80 1 Processor -- start 2024-05-09T00:00:48.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.392+0000 7f7122ae1b80 1 -- start start 2024-05-09T00:00:48.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e5d7290 0x55b09e6793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e679910 0x55b09e67e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 0x55b09e673da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e6842a0 con 0x55b09e5d7290 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e675ae0 con 0x55b09e679910 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e670e90 con 0x55b09e67ee80 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f710ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e5d7290 0x55b09e6793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f710ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e5d7290 0x55b09e6793d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52788/0 (socket says 172.21.15.12:52788) 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f710ffff700 1 -- 172.21.15.12:0/822703013 learned_addr learned my addr 172.21.15.12:0/822703013 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f7114ec4700 1 --2- 172.21.15.12:0/822703013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 0x55b09e673da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f710f7fe700 1 --2- 172.21.15.12:0/822703013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e679910 0x55b09e67e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f710ffff700 1 -- 172.21.15.12:0/822703013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 msgr2=0x55b09e673da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f710ffff700 1 --2- 172.21.15.12:0/822703013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 0x55b09e673da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f710ffff700 1 -- 172.21.15.12:0/822703013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e679910 msgr2=0x55b09e67e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f710ffff700 1 --2- 172.21.15.12:0/822703013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e679910 0x55b09e67e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f710ffff700 1 -- 172.21.15.12:0/822703013 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b09e670560 con 0x55b09e5d7290 2024-05-09T00:00:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.393+0000 7f710ffff700 1 --2- 172.21.15.12:0/822703013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e5d7290 0x55b09e6793d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7100005f50 tx=0x7f7100004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77657c7f720cfe12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.394+0000 7f70f6ffd700 1 -- 172.21.15.12:0/822703013 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100004e80 con 0x55b09e5d7290 2024-05-09T00:00:48.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.394+0000 7f70f6ffd700 1 -- 172.21.15.12:0/822703013 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7100028040 con 0x55b09e5d7290 2024-05-09T00:00:48.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.394+0000 7f70f6ffd700 1 -- 172.21.15.12:0/822703013 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710000fb80 con 0x55b09e5d7290 2024-05-09T00:00:48.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.394+0000 7f7122ae1b80 1 -- 172.21.15.12:0/822703013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e5d7290 msgr2=0x55b09e6793d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.394+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/822703013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e5d7290 0x55b09e6793d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7100005f50 tx=0x7f7100004bf0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.394+0000 7f7122ae1b80 1 -- 172.21.15.12:0/822703013 shutdown_connections 2024-05-09T00:00:48.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.394+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/822703013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 0x55b09e673da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.395+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/822703013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e5d7290 0x55b09e6793d0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.395+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/822703013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e679910 0x55b09e67e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.395+0000 7f7122ae1b80 1 -- 172.21.15.12:0/822703013 >> 172.21.15.12:0/822703013 conn(0x55b09e68ef70 msgr2=0x55b09e67dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.395+0000 7f7122ae1b80 1 -- 172.21.15.12:0/822703013 shutdown_connections 2024-05-09T00:00:48.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.395+0000 7f7122ae1b80 1 -- 172.21.15.12:0/822703013 wait complete. 2024-05-09T00:00:48.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.395+0000 7f7122ae1b80 1 -- start start 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.395+0000 7f7122ae1b80 1 Processor -- start 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e5d7290 0x55b09e689ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 0x55b09e68a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7122ae1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 0x55b09e6e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e6e8e90 con 0x55b09e679910 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e6e9000 con 0x55b09e5d7290 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7122ae1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09e6e91c0 con 0x55b09e67ee80 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7122ae1b80 10 client.0 _ll_register_callbacks cb 0x55b09e56ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f710ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e5d7290 0x55b09e689ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7114ec4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 0x55b09e6e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f710ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e5d7290 0x55b09e689ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45154/0 (socket says 172.21.15.12:45154) 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f710ffff700 1 -- 172.21.15.12:0/1401525972 learned_addr learned my addr 172.21.15.12:0/1401525972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f710f7fe700 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 0x55b09e68a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7114ec4700 1 -- 172.21.15.12:0/1401525972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e5d7290 msgr2=0x55b09e689ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7114ec4700 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e5d7290 0x55b09e689ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7114ec4700 1 -- 172.21.15.12:0/1401525972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 msgr2=0x55b09e68a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7114ec4700 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 0x55b09e68a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.396+0000 7f7114ec4700 1 -- 172.21.15.12:0/1401525972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71101aa710 con 0x55b09e67ee80 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f7114ec4700 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 0x55b09e6e8950 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7108004bd0 tx=0x7f71080092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c48674df9cc8e46 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f70ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1401525972 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108099330 con 0x55b09e67ee80 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1401525972 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f710809a0d0 con 0x55b09e67ee80 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1401525972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f710801b320 con 0x55b09e67ee80 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1401525972 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71080999d0 con 0x55b09e67ee80 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f7122ae1b80 10 client.29278 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1401525972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55b09e3f6be0 con 0x55b09e67ee80 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1401525972 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f710801b320 con 0x55b09e67ee80 2024-05-09T00:00:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1401525972 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7108098ec0 con 0x55b09e67ee80 2024-05-09T00:00:48.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f7122ae1b80 10 client.29278 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:48.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f7122ae1b80 10 client.29278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:48.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.397+0000 7f7122ae1b80 20 client.29278 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:48.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.398+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b09e67f7a0 0x55b09e6ec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.398+0000 7f7122ae1b80 4 client.29278 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:00:48.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.398+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1401525972 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b09e5c1f40 con 0x55b09e67f7a0 2024-05-09T00:00:48.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.398+0000 7f710ffff700 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b09e67f7a0 0x55b09e6ec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.398+0000 7f70ef7fe700 10 client.29278 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:48.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.398+0000 7f710ffff700 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b09e67f7a0 0x55b09e6ec4b0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=308091a04fafeef2 server_cookie=50b7a7524fb6eebe in_seq=0 out_seq=0 2024-05-09T00:00:48.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.400+0000 7f70ef7fe700 1 -- 172.21.15.12:0/1401525972 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b09e5c1f40 con 0x55b09e67f7a0 2024-05-09T00:00:48.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.400+0000 7f70ef7fe700 10 client.29278 handle_command_reply: tid=0 2024-05-09T00:00:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.400+0000 7f7122ae1b80 1 client.29278 shutdown 2024-05-09T00:00:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.400+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1401525972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 msgr2=0x55b09e6e8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.400+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 0x55b09e6e8950 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7108004bd0 tx=0x7f71080092e0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1401525972 shutdown_connections 2024-05-09T00:00:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b09e67ee80 0x55b09e6e8950 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b09e67f7a0 0x55b09e6ec4b0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b09e5d7290 0x55b09e689ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f70ef7fe700 0 client.29278 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7122ae1b80 1 --2- 172.21.15.12:0/1401525972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b09e679910 0x55b09e68a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1401525972 >> 172.21.15.12:0/1401525972 conn(0x55b09e68ef70 msgr2=0x55b09e67fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1401525972 shutdown_connections 2024-05-09T00:00:48.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7122ae1b80 1 -- 172.21.15.12:0/1401525972 wait complete. 2024-05-09T00:00:48.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7122ae1b80 20 client.29278 trim_cache size 0 max 16384 2024-05-09T00:00:48.400 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:00:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:00:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "172fa6d2-7a27-4c91-9254-631d3527541f", 2024-05-09T00:00:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:00:48.401 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:00:48.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7116927700 1 -- 172.21.15.12:0/403125089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70f80a1290 msgr2=0x7f70f80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7116927700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70f80a1290 0x7f70f80a3750 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f710000b2e0 tx=0x7f7100005960 comp rx=0 tx=0).stop 2024-05-09T00:00:48.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7116927700 1 -- 172.21.15.12:0/403125089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7110111f00 msgr2=0x7f711019d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.401+0000 7f7116927700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7110111f00 0x7f711019d850 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f710800ef90 tx=0x7f710800c3a0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.402+0000 7f7116927700 1 -- 172.21.15.12:0/403125089 shutdown_connections 2024-05-09T00:00:48.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.402+0000 7f7116927700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70f80a1290 0x7f70f80a3750 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.402+0000 7f7116927700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7110111f00 0x7f711019d850 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.402+0000 7f7116927700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7110073340 0x7f71101a3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.402+0000 7f7116927700 1 --2- 172.21.15.12:0/403125089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7110073c60 0x7f71101a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.402+0000 7f7116927700 1 -- 172.21.15.12:0/403125089 >> 172.21.15.12:0/403125089 conn(0x7f71100fc9e0 msgr2=0x7f71100fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.402+0000 7f7116927700 1 -- 172.21.15.12:0/403125089 shutdown_connections 2024-05-09T00:00:48.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.402+0000 7f7116927700 1 -- 172.21.15.12:0/403125089 wait complete. 2024-05-09T00:00:48.411 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:00:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.570+0000 7fdf5f88c700 1 -- 172.21.15.12:0/2997058195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58101170 msgr2=0x7fdf58101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.570+0000 7fdf5f88c700 1 --2- 172.21.15.12:0/2997058195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58101170 0x7fdf58101550 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009990 tx=0x7fdf54009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.570+0000 7fdf5f88c700 1 -- 172.21.15.12:0/2997058195 shutdown_connections 2024-05-09T00:00:48.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.570+0000 7fdf5f88c700 1 --2- 172.21.15.12:0/2997058195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58106290 0x7fdf5810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.570+0000 7fdf5f88c700 1 --2- 172.21.15.12:0/2997058195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58101170 0x7fdf58101550 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.570+0000 7fdf5f88c700 1 --2- 172.21.15.12:0/2997058195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf58101b20 0x7fdf58105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.571+0000 7fdf5f88c700 1 -- 172.21.15.12:0/2997058195 >> 172.21.15.12:0/2997058195 conn(0x7fdf580fc9d0 msgr2=0x7fdf580fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.571+0000 7fdf5f88c700 1 -- 172.21.15.12:0/2997058195 shutdown_connections 2024-05-09T00:00:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.571+0000 7fdf5f88c700 1 -- 172.21.15.12:0/2997058195 wait complete. 2024-05-09T00:00:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.571+0000 7fdf5f88c700 1 Processor -- start 2024-05-09T00:00:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.571+0000 7fdf5f88c700 1 -- start start 2024-05-09T00:00:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5f88c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101170 0x7fdf581973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5f88c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58106290 0x7fdf58197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5f88c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5819bcb0 0x7fdf5819c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5f88c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5819c730 con 0x7fdf58106290 2024-05-09T00:00:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5f88c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5819c8a0 con 0x7fdf58101170 2024-05-09T00:00:48.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5f88c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5819ca60 con 0x7fdf5819bcb0 2024-05-09T00:00:48.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5ce27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58106290 0x7fdf58197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5ce27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58106290 0x7fdf58197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52812/0 (socket says 172.21.15.12:52812) 2024-05-09T00:00:48.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5ce27700 1 -- 172.21.15.12:0/1104207106 learned_addr learned my addr 172.21.15.12:0/1104207106 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:48.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5d628700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101170 0x7fdf581973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5de29700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5819bcb0 0x7fdf5819c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5ce27700 1 -- 172.21.15.12:0/1104207106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5819bcb0 msgr2=0x7fdf5819c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5ce27700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5819bcb0 0x7fdf5819c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5ce27700 1 -- 172.21.15.12:0/1104207106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101170 msgr2=0x7fdf581973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5ce27700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101170 0x7fdf581973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.572+0000 7fdf5ce27700 1 -- 172.21.15.12:0/1104207106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540095f0 con 0x7fdf58106290 2024-05-09T00:00:48.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.573+0000 7fdf5ce27700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58106290 0x7fdf58197910 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800e970 tx=0x7fdf4800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.573+0000 7fdf4e7fc700 1 -- 172.21.15.12:0/1104207106 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48009790 con 0x7fdf58106290 2024-05-09T00:00:48.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.573+0000 7fdf4e7fc700 1 -- 172.21.15.12:0/1104207106 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf4800c7e0 con 0x7fdf58106290 2024-05-09T00:00:48.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.573+0000 7fdf5f88c700 1 -- 172.21.15.12:0/1104207106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf581a3fc0 con 0x7fdf58106290 2024-05-09T00:00:48.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.573+0000 7fdf5f88c700 1 -- 172.21.15.12:0/1104207106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf581a4440 con 0x7fdf58106290 2024-05-09T00:00:48.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.575+0000 7fdf4e7fc700 1 -- 172.21.15.12:0/1104207106 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480054e0 con 0x7fdf58106290 2024-05-09T00:00:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.576+0000 7fdf4e7fc700 1 -- 172.21.15.12:0/1104207106 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdf48021440 con 0x7fdf58106290 2024-05-09T00:00:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.576+0000 7fdf4e7fc700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf440a56a0 0x7fdf440a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.577+0000 7fdf4e7fc700 1 -- 172.21.15.12:0/1104207106 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdf48014070 con 0x7fdf58106290 2024-05-09T00:00:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.577+0000 7fdf5d628700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf440a56a0 0x7fdf440a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.577+0000 7fdf6ba46b80 1 Processor -- start 2024-05-09T00:00:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf6ba46b80 1 -- start start 2024-05-09T00:00:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5d628700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf440a56a0 0x7fdf440a7b60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400b2e0 tx=0x7fdf54005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8ccfb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccf5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8ccf5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8cc4c150 con 0x555a8ccfb720 2024-05-09T00:00:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8cc49b10 con 0x555a8ccfae00 2024-05-09T00:00:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8cc45400 con 0x555a8ccf5890 2024-05-09T00:00:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5de29700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8ccf5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5d628700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8ccfb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5ce27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccf5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5de29700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8ccf5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51288/0 (socket says 172.21.15.12:51288) 2024-05-09T00:00:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5de29700 1 -- 172.21.15.12:0/2887733695 learned_addr learned my addr 172.21.15.12:0/2887733695 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5d628700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8ccfb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45180/0 (socket says 172.21.15.12:45180) 2024-05-09T00:00:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5ce27700 1 -- 172.21.15.12:0/2887733695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 msgr2=0x555a8ccf5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5ce27700 1 --2- 172.21.15.12:0/2887733695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8ccf5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5ce27700 1 -- 172.21.15.12:0/2887733695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 msgr2=0x555a8ccfb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.578+0000 7fdf5ce27700 1 --2- 172.21.15.12:0/2887733695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8ccfb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.579+0000 7fdf5ce27700 1 -- 172.21.15.12:0/2887733695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a8cac63b0 con 0x555a8ccfb720 2024-05-09T00:00:48.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.579+0000 7fdf5de29700 1 --2- 172.21.15.12:0/2887733695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8ccf5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.579+0000 7fdf5ce27700 1 --2- 172.21.15.12:0/2887733695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccf5350 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdf480ca600 tx=0x7fdf4809ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7032b90a638e5e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.579+0000 7fdf437fe700 1 -- 172.21.15.12:0/2887733695 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4809ed80 con 0x555a8ccfb720 2024-05-09T00:00:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.579+0000 7fdf437fe700 1 -- 172.21.15.12:0/2887733695 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf480a33d0 con 0x555a8ccfb720 2024-05-09T00:00:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.579+0000 7fdf437fe700 1 -- 172.21.15.12:0/2887733695 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4809a440 con 0x555a8ccfb720 2024-05-09T00:00:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2887733695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 msgr2=0x555a8ccf5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/2887733695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccf5350 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdf480ca600 tx=0x7fdf4809ab60 comp rx=0 tx=0).stop 2024-05-09T00:00:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2887733695 shutdown_connections 2024-05-09T00:00:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/2887733695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8ccf5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/2887733695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8ccfb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/2887733695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccf5350 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2887733695 >> 172.21.15.12:0/2887733695 conn(0x555a8cc53210 msgr2=0x555a8ccefdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2887733695 shutdown_connections 2024-05-09T00:00:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2887733695 wait complete. 2024-05-09T00:00:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 -- start start 2024-05-09T00:00:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.580+0000 7fdf6ba46b80 1 Processor -- start 2024-05-09T00:00:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccf5890 0x555a8ccfeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccfae00 0x555a8ccfeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8cbe1560 con 0x555a8ccfb720 2024-05-09T00:00:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8cbe1200 con 0x555a8ccf5890 2024-05-09T00:00:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8cac86a0 con 0x555a8ccfae00 2024-05-09T00:00:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf6ba46b80 10 client.0 _ll_register_callbacks cb 0x555a8cbeab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf5de29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf5ce27700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccfae00 0x555a8ccfeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf5de29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52828/0 (socket says 172.21.15.12:52828) 2024-05-09T00:00:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf5d628700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccf5890 0x555a8ccfeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf5de29700 1 -- 172.21.15.12:0/1625887469 learned_addr learned my addr 172.21.15.12:0/1625887469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf5de29700 1 -- 172.21.15.12:0/1625887469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccfae00 msgr2=0x555a8ccfeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf5de29700 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccfae00 0x555a8ccfeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf5de29700 1 -- 172.21.15.12:0/1625887469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccf5890 msgr2=0x555a8ccfeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf5de29700 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccf5890 0x555a8ccfeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.581+0000 7fdf5de29700 1 -- 172.21.15.12:0/1625887469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf50011330 con 0x555a8ccfb720 2024-05-09T00:00:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf5de29700 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff530 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdf50012010 tx=0x7fdf50014650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4e879abe199d9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf40ff9700 1 -- 172.21.15.12:0/1625887469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000e040 con 0x555a8ccfb720 2024-05-09T00:00:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf40ff9700 1 -- 172.21.15.12:0/1625887469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdf5001a070 con 0x555a8ccfb720 2024-05-09T00:00:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1625887469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a8cb9e270 con 0x555a8ccfb720 2024-05-09T00:00:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf40ff9700 1 -- 172.21.15.12:0/1625887469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf50014de0 con 0x555a8ccfb720 2024-05-09T00:00:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf6ba46b80 10 client.19569 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1625887469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7fdf581a4440 con 0x555a8ccfb720 2024-05-09T00:00:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf40ff9700 1 -- 172.21.15.12:0/1625887469 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a8cb9e270 con 0x555a8ccfb720 2024-05-09T00:00:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf40ff9700 1 -- 172.21.15.12:0/1625887469 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdf50025070 con 0x555a8ccfb720 2024-05-09T00:00:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf6ba46b80 10 client.19569 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf6ba46b80 10 client.19569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf6ba46b80 20 client.19569 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555a8cd0f320 0x555a8cd11740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf6ba46b80 4 client.19569 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.582+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1625887469 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a8ccc4810 con 0x555a8cd0f320 2024-05-09T00:00:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.583+0000 7fdf5d628700 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555a8cd0f320 0x555a8cd11740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.583+0000 7fdf5d628700 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555a8cd0f320 0x555a8cd11740 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ef3b5950885495 server_cookie=1a7eadb07b7fe359 in_seq=0 out_seq=0 2024-05-09T00:00:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.583+0000 7fdf40ff9700 10 client.19569 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:48.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf40ff9700 1 -- 172.21.15.12:0/1625887469 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a8ccc4810 con 0x555a8cd0f320 2024-05-09T00:00:48.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf40ff9700 10 client.19569 handle_command_reply: tid=0 2024-05-09T00:00:48.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 client.19569 shutdown 2024-05-09T00:00:48.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1625887469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 msgr2=0x555a8ccff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff530 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdf50012010 tx=0x7fdf50014650 comp rx=0 tx=0).stop 2024-05-09T00:00:48.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1625887469 shutdown_connections 2024-05-09T00:00:48.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff530 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555a8cd0f320 0x555a8cd11740 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccf5890 0x555a8ccfeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/1625887469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccfae00 0x555a8ccfeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf40ff9700 0 client.19569 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:48.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1625887469 >> 172.21.15.12:0/1625887469 conn(0x555a8cc53210 msgr2=0x555a8ccef5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1625887469 shutdown_connections 2024-05-09T00:00:48.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1625887469 wait complete. 2024-05-09T00:00:48.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.586+0000 7fdf6ba46b80 20 client.19569 trim_cache size 0 max 16384 2024-05-09T00:00:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.607+0000 7fdf6ba46b80 1 Processor -- start 2024-05-09T00:00:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.607+0000 7fdf6ba46b80 1 -- start start 2024-05-09T00:00:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.607+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8ccfa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.607+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8ccefd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.607+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.607+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8ccecdc0 con 0x555a8ccfb720 2024-05-09T00:00:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.607+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8cd16950 con 0x555a8ccfae00 2024-05-09T00:00:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.607+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8cce8040 con 0x555a8ccf5890 2024-05-09T00:00:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5de29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5d628700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8ccfa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5de29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52844/0 (socket says 172.21.15.12:52844) 2024-05-09T00:00:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5de29700 1 -- 172.21.15.12:0/1143604341 learned_addr learned my addr 172.21.15.12:0/1143604341 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5ce27700 1 --2- 172.21.15.12:0/1143604341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8ccefd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5de29700 1 -- 172.21.15.12:0/1143604341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 msgr2=0x555a8ccfa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5de29700 1 --2- 172.21.15.12:0/1143604341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8ccfa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5de29700 1 -- 172.21.15.12:0/1143604341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 msgr2=0x555a8ccefd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5de29700 1 --2- 172.21.15.12:0/1143604341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8ccefd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5de29700 1 -- 172.21.15.12:0/1143604341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a8ccf5f40 con 0x555a8ccfb720 2024-05-09T00:00:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf5de29700 1 --2- 172.21.15.12:0/1143604341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff350 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006000 tx=0x7fdf50006030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914330955f4aec43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf43fff700 1 -- 172.21.15.12:0/1143604341 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000e070 con 0x555a8ccfb720 2024-05-09T00:00:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf43fff700 1 -- 172.21.15.12:0/1143604341 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf5000f040 con 0x555a8ccfb720 2024-05-09T00:00:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.608+0000 7fdf43fff700 1 -- 172.21.15.12:0/1143604341 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5001e7e0 con 0x555a8ccfb720 2024-05-09T00:00:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.609+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1143604341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 msgr2=0x555a8ccff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.609+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/1143604341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff350 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006000 tx=0x7fdf50006030 comp rx=0 tx=0).stop 2024-05-09T00:00:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.609+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1143604341 shutdown_connections 2024-05-09T00:00:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.609+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/1143604341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccff350 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.609+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/1143604341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8ccfa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.609+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/1143604341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8ccefd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.609+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1143604341 >> 172.21.15.12:0/1143604341 conn(0x555a8cc53210 msgr2=0x555a8ccfd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.610+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1143604341 shutdown_connections 2024-05-09T00:00:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.610+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/1143604341 wait complete. 2024-05-09T00:00:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.610+0000 7fdf6ba46b80 1 -- start start 2024-05-09T00:00:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.610+0000 7fdf6ba46b80 1 Processor -- start 2024-05-09T00:00:48.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8cd06170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8cd066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf6ba46b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccf6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8ccf76d0 con 0x555a8ccfb720 2024-05-09T00:00:48.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8ccf7840 con 0x555a8ccfae00 2024-05-09T00:00:48.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf6ba46b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8ccf7a00 con 0x555a8ccf5890 2024-05-09T00:00:48.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf6ba46b80 10 client.0 _ll_register_callbacks cb 0x555a8cbeab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:48.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5d628700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8cd06170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5de29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccf6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5ce27700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8cd066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5d628700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8cd06170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51328/0 (socket says 172.21.15.12:51328) 2024-05-09T00:00:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5de29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccf6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52848/0 (socket says 172.21.15.12:52848) 2024-05-09T00:00:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5de29700 1 -- 172.21.15.12:0/2123626031 learned_addr learned my addr 172.21.15.12:0/2123626031 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5ce27700 1 -- 172.21.15.12:0/2123626031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 msgr2=0x555a8cd06170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5ce27700 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8cd06170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5ce27700 1 -- 172.21.15.12:0/2123626031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 msgr2=0x555a8ccf6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5ce27700 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccf6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.611+0000 7fdf5ce27700 1 -- 172.21.15.12:0/2123626031 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf581a4440 con 0x555a8ccfae00 2024-05-09T00:00:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.612+0000 7fdf5ce27700 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8cd066b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x555a8cd01cc0 tx=0x7fdf4800ce10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e708b85c3ee01db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.612+0000 7fdf40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:00:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.612+0000 7fdf40ff9700 1 -- 172.21.15.12:0/2123626031 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4809ee00 con 0x555a8ccfae00 2024-05-09T00:00:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.612+0000 7fdf40ff9700 1 -- 172.21.15.12:0/2123626031 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdf4809a440 con 0x555a8ccfae00 2024-05-09T00:00:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.612+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2123626031 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf480a33d0 con 0x555a8ccfae00 2024-05-09T00:00:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.612+0000 7fdf40ff9700 1 -- 172.21.15.12:0/2123626031 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf4809a600 con 0x555a8ccfae00 2024-05-09T00:00:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.612+0000 7fdf6ba46b80 10 client.29276 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.612+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2123626031 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x555a8ca72b60 con 0x555a8ccfae00 2024-05-09T00:00:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.613+0000 7fdf40ff9700 1 -- 172.21.15.12:0/2123626031 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf480a33d0 con 0x555a8ccfae00 2024-05-09T00:00:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.613+0000 7fdf40ff9700 1 -- 172.21.15.12:0/2123626031 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdf48095b60 con 0x555a8ccfae00 2024-05-09T00:00:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.613+0000 7fdf6ba46b80 10 client.29276 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.613+0000 7fdf6ba46b80 10 client.29276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.613+0000 7fdf6ba46b80 20 client.29276 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.613+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555a8cd668c0 0x555a8cd68ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.613+0000 7fdf6ba46b80 4 client.29276 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:00:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.613+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2123626031 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555a8cc3dec0 con 0x555a8cd668c0 2024-05-09T00:00:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.613+0000 7fdf5d628700 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555a8cd668c0 0x555a8cd68ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.613+0000 7fdf40ff9700 10 client.29276 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.614+0000 7fdf5d628700 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555a8cd668c0 0x555a8cd68ce0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7dda1d30fe289c8 server_cookie=efa02e2a3bf09e12 in_seq=0 out_seq=0 2024-05-09T00:00:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf40ff9700 1 -- 172.21.15.12:0/2123626031 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x555a8cc3dec0 con 0x555a8cd668c0 2024-05-09T00:00:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf40ff9700 10 client.29276 handle_command_reply: tid=0 2024-05-09T00:00:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 client.29276 shutdown 2024-05-09T00:00:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2123626031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 msgr2=0x555a8cd066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8cd066b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x555a8cd01cc0 tx=0x7fdf4800ce10 comp rx=0 tx=0).stop 2024-05-09T00:00:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2123626031 shutdown_connections 2024-05-09T00:00:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555a8ccfb720 0x555a8ccf6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a8ccf5890 0x555a8cd06170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555a8cd668c0 0x555a8cd68ce0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 --2- 172.21.15.12:0/2123626031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555a8ccfae00 0x555a8cd066b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2123626031 >> 172.21.15.12:0/2123626031 conn(0x555a8cc53210 msgr2=0x555a8ccfa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf40ff9700 0 client.29276 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2123626031 shutdown_connections 2024-05-09T00:00:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 1 -- 172.21.15.12:0/2123626031 wait complete. 2024-05-09T00:00:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.616+0000 7fdf6ba46b80 20 client.29276 trim_cache size 0 max 16384 2024-05-09T00:00:48.620 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:00:48.620 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (204 inodes in the stack)", 2024-05-09T00:00:48.620 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:00:48.620 INFO:teuthology.orchestra.run.smithi012.stdout: "172fa6d2-7a27-4c91-9254-631d3527541f": { 2024-05-09T00:00:48.621 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:00:48.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "172fa6d2-7a27-4c91-9254-631d3527541f", 2024-05-09T00:00:48.621 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:00:48.621 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:00:48.621 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:00:48.621 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:00:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 -- 172.21.15.12:0/1104207106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf440a56a0 msgr2=0x7fdf440a7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf440a56a0 0x7fdf440a7b60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400b2e0 tx=0x7fdf54005960 comp rx=0 tx=0).stop 2024-05-09T00:00:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 -- 172.21.15.12:0/1104207106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58106290 msgr2=0x7fdf58197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58106290 0x7fdf58197910 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800e970 tx=0x7fdf4800ec80 comp rx=0 tx=0).stop 2024-05-09T00:00:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 -- 172.21.15.12:0/1104207106 shutdown_connections 2024-05-09T00:00:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf440a56a0 0x7fdf440a7b60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5819bcb0 0x7fdf5819c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf58101170 0x7fdf581973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 --2- 172.21.15.12:0/1104207106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf58106290 0x7fdf58197910 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 -- 172.21.15.12:0/1104207106 >> 172.21.15.12:0/1104207106 conn(0x7fdf580fc9d0 msgr2=0x7fdf580fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 -- 172.21.15.12:0/1104207106 shutdown_connections 2024-05-09T00:00:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:00:48.617+0000 7fdf5f88c700 1 -- 172.21.15.12:0/1104207106 wait complete. 2024-05-09T00:00:48.627 INFO:tasks.cephfs.filesystem:scrub status for tag:172fa6d2-7a27-4c91-9254-631d3527541f - {'path': '/', 'tag': '172fa6d2-7a27-4c91-9254-631d3527541f', 'options': 'recursive,force'} 2024-05-09T00:00:48.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:48 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:00:48.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:48 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:48.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:48 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:00:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:48 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:00:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:48 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:48 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:00:49.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:48 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:00:49.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:48 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:49.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:48 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:00:49.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 30852 41.66 MB/sec warmup 44 sec latency 104.270 ms 2024-05-09T00:00:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:49 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:00:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:49 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:00:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:49 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:49 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:00:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:49 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:49 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:49 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:49.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:49 smithi012 ceph-mon[42503]: pgmap v461: 97 pgs: 97 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 682 op/s 2024-05-09T00:00:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:49 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:00:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:49 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:00:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:49 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:49 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:00:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:49 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:49 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:49 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:49 smithi204 ceph-mon[47373]: pgmap v461: 97 pgs: 97 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 682 op/s 2024-05-09T00:00:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:49 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:00:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:49 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:00:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:49 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:50.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:49 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:00:50.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:49 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:50.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:49 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:50.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:49 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:00:50.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:49 smithi160 ceph-mon[46152]: pgmap v461: 97 pgs: 97 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 682 op/s 2024-05-09T00:00:50.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 31769 41.98 MB/sec warmup 45 sec latency 65.844 ms 2024-05-09T00:00:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.792+0000 7fe2cef17700 1 -- 172.21.15.12:0/3767504738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2c8105e60 msgr2=0x7fe2c8106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.792+0000 7fe2cef17700 1 --2- 172.21.15.12:0/3767504738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2c8105e60 0x7fe2c8106240 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009990 tx=0x7fe2bc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.793+0000 7fe2cef17700 1 -- 172.21.15.12:0/3767504738 shutdown_connections 2024-05-09T00:00:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.793+0000 7fe2cef17700 1 --2- 172.21.15.12:0/3767504738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8100760 0x7fe2c810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.793+0000 7fe2cef17700 1 --2- 172.21.15.12:0/3767504738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2c8105e60 0x7fe2c8106240 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.793+0000 7fe2cef17700 1 --2- 172.21.15.12:0/3767504738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2c80ffda0 0x7fe2c8100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.793+0000 7fe2cef17700 1 -- 172.21.15.12:0/3767504738 >> 172.21.15.12:0/3767504738 conn(0x7fe2c8076f10 msgr2=0x7fe2c8077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.793+0000 7fe2cef17700 1 -- 172.21.15.12:0/3767504738 shutdown_connections 2024-05-09T00:00:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.793+0000 7fe2cef17700 1 -- 172.21.15.12:0/3767504738 wait complete. 2024-05-09T00:00:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cef17700 1 Processor -- start 2024-05-09T00:00:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cef17700 1 -- start start 2024-05-09T00:00:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cef17700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2c80ffda0 0x7fe2c810cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cef17700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2c8100760 0x7fe2c8107b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cef17700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8105e60 0x7fe2c8108130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cef17700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c8108900 con 0x7fe2c8105e60 2024-05-09T00:00:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cef17700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c81a1540 con 0x7fe2c8100760 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cef17700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c81a1700 con 0x7fe2c80ffda0 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cccb3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2c80ffda0 0x7fe2c810cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cd4b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8105e60 0x7fe2c8108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cccb3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2c80ffda0 0x7fe2c810cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51346/0 (socket says 172.21.15.12:51346) 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2cccb3700 1 -- 172.21.15.12:0/4125171583 learned_addr learned my addr 172.21.15.12:0/4125171583 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.794+0000 7fe2c7fff700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2c8100760 0x7fe2c8107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2cd4b4700 1 -- 172.21.15.12:0/4125171583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2c80ffda0 msgr2=0x7fe2c810cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2cd4b4700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2c80ffda0 0x7fe2c810cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2cd4b4700 1 -- 172.21.15.12:0/4125171583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2c8100760 msgr2=0x7fe2c8107b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2cd4b4700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2c8100760 0x7fe2c8107b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2cd4b4700 1 -- 172.21.15.12:0/4125171583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0095f0 con 0x7fe2c8105e60 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2c80ffda0 0x7fe2c810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:00:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2cd4b4700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8105e60 0x7fe2c8108130 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000efc0 tx=0x7fe2c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2c5ffb700 1 -- 172.21.15.12:0/4125171583 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0018070 con 0x7fe2c8105e60 2024-05-09T00:00:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2c5ffb700 1 -- 172.21.15.12:0/4125171583 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2c000f040 con 0x7fe2c8105e60 2024-05-09T00:00:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2cef17700 1 -- 172.21.15.12:0/4125171583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c81a1980 con 0x7fe2c8105e60 2024-05-09T00:00:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.795+0000 7fe2c5ffb700 1 -- 172.21.15.12:0/4125171583 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000cd40 con 0x7fe2c8105e60 2024-05-09T00:00:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.796+0000 7fe2cef17700 1 -- 172.21.15.12:0/4125171583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c81a1e40 con 0x7fe2c8105e60 2024-05-09T00:00:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.798+0000 7fe2c5ffb700 1 -- 172.21.15.12:0/4125171583 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe2c0003a00 con 0x7fe2c8105e60 2024-05-09T00:00:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.799+0000 7fe2c5ffb700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2b80a1290 0x7fe2b80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.799+0000 7fe2c5ffb700 1 -- 172.21.15.12:0/4125171583 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe2c00cc920 con 0x7fe2c8105e60 2024-05-09T00:00:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.800+0000 7fe2db0d1b80 1 Processor -- start 2024-05-09T00:00:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.800+0000 7fe2db0d1b80 1 -- start start 2024-05-09T00:00:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.800+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2b80a1290 0x7fe2b80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.800+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1ba950 0x56464b1bad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.800+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1bb270 0x56464b1b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.800+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1b5480 0x56464b1b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.800+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b10c0f0 con 0x56464b1ba950 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.800+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b109ab0 con 0x56464b1b5480 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.800+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b1053a0 con 0x56464b1bb270 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cd4b4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1b5480 0x56464b1b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1bb270 0x56464b1b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cd4b4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1b5480 0x56464b1b58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45246/0 (socket says 172.21.15.12:45246) 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cd4b4700 1 -- 172.21.15.12:0/549388690 learned_addr learned my addr 172.21.15.12:0/549388690 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/549388690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1ba950 0x56464b1bad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cd4b4700 1 -- 172.21.15.12:0/549388690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1bb270 msgr2=0x56464b1b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cd4b4700 1 --2- 172.21.15.12:0/549388690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1bb270 0x56464b1b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cd4b4700 1 -- 172.21.15.12:0/549388690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1ba950 msgr2=0x56464b1bad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cd4b4700 1 --2- 172.21.15.12:0/549388690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1ba950 0x56464b1bad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cd4b4700 1 -- 172.21.15.12:0/549388690 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56464af86350 con 0x56464b1b5480 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/549388690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1ba950 0x56464b1bad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:00:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2b80a1290 0x7fe2b80a3750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc006010 tx=0x7fe2bc00b610 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.801+0000 7fe2cd4b4700 1 --2- 172.21.15.12:0/549388690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1b5480 0x56464b1b58a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00b7fd0 tx=0x7fe2c0099940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94d340d49f3fdf20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.803+0000 7fe2b2ffd700 1 -- 172.21.15.12:0/549388690 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0004ad0 con 0x56464b1b5480 2024-05-09T00:00:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.803+0000 7fe2b2ffd700 1 -- 172.21.15.12:0/549388690 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2c009a170 con 0x56464b1b5480 2024-05-09T00:00:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.803+0000 7fe2b2ffd700 1 -- 172.21.15.12:0/549388690 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c001b580 con 0x56464b1b5480 2024-05-09T00:00:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.804+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/549388690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1b5480 msgr2=0x56464b1b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.804+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/549388690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1b5480 0x56464b1b58a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00b7fd0 tx=0x7fe2c0099940 comp rx=0 tx=0).stop 2024-05-09T00:00:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.806+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/549388690 shutdown_connections 2024-05-09T00:00:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.806+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/549388690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1b5480 0x56464b1b58a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.806+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/549388690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1ba950 0x56464b1bad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.806+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/549388690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1bb270 0x56464b1b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.806+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/549388690 >> 172.21.15.12:0/549388690 conn(0x56464b1131b0 msgr2=0x56464b1af840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.806+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/549388690 shutdown_connections 2024-05-09T00:00:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.806+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/549388690 wait complete. 2024-05-09T00:00:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.807+0000 7fe2db0d1b80 1 -- start start 2024-05-09T00:00:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.807+0000 7fe2db0d1b80 1 Processor -- start 2024-05-09T00:00:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1be6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1bebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1bf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b0a1500 con 0x56464b1b5480 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b0a11a0 con 0x56464b1bb270 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464af88640 con 0x56464b1ba950 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2db0d1b80 10 client.0 _ll_register_callbacks cb 0x56464b0aaad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1bebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1bebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51364/0 (socket says 172.21.15.12:51364) 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2cccb3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1be6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2c7fff700 1 -- 172.21.15.12:0/1158100829 learned_addr learned my addr 172.21.15.12:0/1158100829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2cd4b4700 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1bf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2cccb3700 1 -- 172.21.15.12:0/1158100829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 msgr2=0x56464b1bebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1bebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2cccb3700 1 -- 172.21.15.12:0/1158100829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 msgr2=0x56464b1bf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1bf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2cccb3700 1 -- 172.21.15.12:0/1158100829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c81a1e40 con 0x56464b1b5480 2024-05-09T00:00:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.808+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1be6b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc005910 tx=0x7fe2bc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ddc7a957488437 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2a7fff700 1 -- 172.21.15.12:0/1158100829 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc01d030 con 0x56464b1b5480 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1158100829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56464b05e210 con 0x56464b1b5480 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2a7fff700 1 -- 172.21.15.12:0/1158100829 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe2bc02f650 con 0x56464b1b5480 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2a7fff700 1 -- 172.21.15.12:0/1158100829 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2bc02f810 con 0x56464b1b5480 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2db0d1b80 10 client.19587 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1158100829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56464af32b00 con 0x56464b1b5480 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2a7fff700 1 -- 172.21.15.12:0/1158100829 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56464b05e210 con 0x56464b1b5480 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2a7fff700 1 -- 172.21.15.12:0/1158100829 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe2bc0269d0 con 0x56464b1b5480 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2db0d1b80 10 client.19587 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2db0d1b80 10 client.19587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2db0d1b80 20 client.19587 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56464b1ced40 0x56464b1d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2db0d1b80 4 client.19587 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:00:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.809+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1158100829 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56464b1847b0 con 0x56464b1ced40 2024-05-09T00:00:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.810+0000 7fe2c7fff700 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56464b1ced40 0x56464b1d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.810+0000 7fe2c7fff700 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56464b1ced40 0x56464b1d1160 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e7d57bf7423c360 server_cookie=46d38d7dba8f6e06 in_seq=0 out_seq=0 2024-05-09T00:00:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.810+0000 7fe2a7fff700 10 client.19587 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2a7fff700 1 -- 172.21.15.12:0/1158100829 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56464b1847b0 con 0x56464b1ced40 2024-05-09T00:00:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2a7fff700 10 client.19587 handle_command_reply: tid=0 2024-05-09T00:00:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2db0d1b80 1 client.19587 shutdown 2024-05-09T00:00:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1158100829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 msgr2=0x56464b1be6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1be6b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc005910 tx=0x7fe2bc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1158100829 shutdown_connections 2024-05-09T00:00:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56464b1ced40 0x56464b1d1160 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1bf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1be6b0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2a7fff700 0 client.19587 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/1158100829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1bebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1158100829 >> 172.21.15.12:0/1158100829 conn(0x56464b1131b0 msgr2=0x56464b1b94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.813+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1158100829 shutdown_connections 2024-05-09T00:00:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.814+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1158100829 wait complete. 2024-05-09T00:00:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.814+0000 7fe2db0d1b80 20 client.19587 trim_cache size 0 max 16384 2024-05-09T00:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.836+0000 7fe2db0d1b80 1 Processor -- start 2024-05-09T00:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.836+0000 7fe2db0d1b80 1 -- start start 2024-05-09T00:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b1cac00 con 0x56464b1b5480 2024-05-09T00:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b1b0330 con 0x56464b1bb270 2024-05-09T00:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b1cb8f0 con 0x56464b1ba950 2024-05-09T00:00:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2cccb3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2cccb3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52902/0 (socket says 172.21.15.12:52902) 2024-05-09T00:00:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2cccb3700 1 -- 172.21.15.12:0/1019682927 learned_addr learned my addr 172.21.15.12:0/1019682927 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2cd4b4700 1 --2- 172.21.15.12:0/1019682927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2cccb3700 1 -- 172.21.15.12:0/1019682927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 msgr2=0x56464b1b14b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:00:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/1019682927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1b14b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2cccb3700 1 -- 172.21.15.12:0/1019682927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 msgr2=0x56464b1ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/1019682927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.837+0000 7fe2cccb3700 1 -- 172.21.15.12:0/1019682927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56464b0cb500 con 0x56464b1b5480 2024-05-09T00:00:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.838+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/1019682927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1af910 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc000c00 tx=0x7fe2bc00b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437a13e9797b89cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.838+0000 7fe2b37fe700 1 -- 172.21.15.12:0/1019682927 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc01d040 con 0x56464b1b5480 2024-05-09T00:00:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.838+0000 7fe2b37fe700 1 -- 172.21.15.12:0/1019682927 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2bc027400 con 0x56464b1b5480 2024-05-09T00:00:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.838+0000 7fe2b37fe700 1 -- 172.21.15.12:0/1019682927 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc02f8b0 con 0x56464b1b5480 2024-05-09T00:00:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1019682927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 msgr2=0x56464b1af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/1019682927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1af910 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc000c00 tx=0x7fe2bc00b310 comp rx=0 tx=0).stop 2024-05-09T00:00:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1019682927 shutdown_connections 2024-05-09T00:00:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/1019682927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/1019682927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1af910 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/1019682927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1019682927 >> 172.21.15.12:0/1019682927 conn(0x56464b1131b0 msgr2=0x56464b1bcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1019682927 shutdown_connections 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/1019682927 wait complete. 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 -- start start 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.839+0000 7fe2db0d1b80 1 Processor -- start 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2db0d1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b1c2770 con 0x56464b1b5480 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b1c28e0 con 0x56464b1bb270 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2db0d1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464b1c2aa0 con 0x56464b1ba950 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2db0d1b80 10 client.0 _ll_register_callbacks cb 0x56464b0aaad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2cccb3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1c1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51390/0 (socket says 172.21.15.12:51390) 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2c7fff700 1 -- 172.21.15.12:0/3426102495 learned_addr learned my addr 172.21.15.12:0/3426102495 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.840+0000 7fe2cd4b4700 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.841+0000 7fe2cccb3700 1 -- 172.21.15.12:0/3426102495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 msgr2=0x56464b1c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.841+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.841+0000 7fe2cccb3700 1 -- 172.21.15.12:0/3426102495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 msgr2=0x56464b1c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.841+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1c2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.841+0000 7fe2cccb3700 1 -- 172.21.15.12:0/3426102495 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56464af32b00 con 0x56464b1b5480 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.841+0000 7fe2cccb3700 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1c5f10 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc026e30 tx=0x7fe2bc00ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a16c6451217bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.841+0000 7fe2a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.841+0000 7fe2a7fff700 1 -- 172.21.15.12:0/3426102495 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc01d070 con 0x56464b1b5480 2024-05-09T00:00:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.841+0000 7fe2a7fff700 1 -- 172.21.15.12:0/3426102495 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe2bc026610 con 0x56464b1b5480 2024-05-09T00:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.842+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/3426102495 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2c009a170 con 0x56464b1b5480 2024-05-09T00:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2a7fff700 1 -- 172.21.15.12:0/3426102495 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2bc026810 con 0x56464b1b5480 2024-05-09T00:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2db0d1b80 10 client.19593 fetch_fsmap learned FSMap version 61 2024-05-09T00:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/3426102495 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56464af8ac70 con 0x56464b1b5480 2024-05-09T00:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2a7fff700 1 -- 172.21.15.12:0/3426102495 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2bc053030 con 0x56464b1b5480 2024-05-09T00:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2a7fff700 1 -- 172.21.15.12:0/3426102495 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe2bc04a020 con 0x56464b1b5480 2024-05-09T00:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2db0d1b80 10 client.19593 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2db0d1b80 10 client.19593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2db0d1b80 20 client.19593 populate_metadata read hostname 'smithi012' 2024-05-09T00:00:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56464b2255a0 0x56464b2279c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2db0d1b80 4 client.19593 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.843+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/3426102495 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56464b0fde60 con 0x56464b2255a0 2024-05-09T00:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.844+0000 7fe2c7fff700 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56464b2255a0 0x56464b2279c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.844+0000 7fe2c7fff700 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56464b2255a0 0x56464b2279c0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29afaa1170cecb9e server_cookie=980d67474fbbd8ae in_seq=0 out_seq=0 2024-05-09T00:00:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:50.844+0000 7fe2a7fff700 10 client.19593 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2a7fff700 1 -- 172.21.15.12:0/3426102495 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56464b0fde60 con 0x56464b2255a0 2024-05-09T00:00:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2a7fff700 10 client.19593 handle_command_reply: tid=0 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2db0d1b80 1 client.19593 shutdown 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/3426102495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 msgr2=0x56464b1c5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1c5f10 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc026e30 tx=0x7fe2bc00ffd0 comp rx=0 tx=0).stop 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/3426102495 shutdown_connections 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56464b2255a0 0x56464b2279c0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56464b1bb270 0x56464b1c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56464b1b5480 0x56464b1c5f10 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2db0d1b80 1 --2- 172.21.15.12:0/3426102495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56464b1ba950 0x56464b1c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/3426102495 >> 172.21.15.12:0/3426102495 conn(0x56464b1131b0 msgr2=0x56464b1b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.165+0000 7fe2a7fff700 0 client.19593 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.166+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/3426102495 shutdown_connections 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.166+0000 7fe2db0d1b80 1 -- 172.21.15.12:0/3426102495 wait complete. 2024-05-09T00:00:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.166+0000 7fe2db0d1b80 20 client.19593 trim_cache size 0 max 16384 2024-05-09T00:00:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:00:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:00:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:00:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 -- 172.21.15.12:0/4125171583 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2b80a1290 msgr2=0x7fe2b80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2b80a1290 0x7fe2b80a3750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc006010 tx=0x7fe2bc00b610 comp rx=0 tx=0).stop 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 -- 172.21.15.12:0/4125171583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8105e60 msgr2=0x7fe2c8108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8105e60 0x7fe2c8108130 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000efc0 tx=0x7fe2c000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 -- 172.21.15.12:0/4125171583 shutdown_connections 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2b80a1290 0x7fe2b80a3750 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8105e60 0x7fe2c8108130 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2c80ffda0 0x7fe2c810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 --2- 172.21.15.12:0/4125171583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2c8100760 0x7fe2c8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 -- 172.21.15.12:0/4125171583 >> 172.21.15.12:0/4125171583 conn(0x7fe2c8076f10 msgr2=0x7fe2c80fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 -- 172.21.15.12:0/4125171583 shutdown_connections 2024-05-09T00:00:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:00:51.170+0000 7fe2cef17700 1 -- 172.21.15.12:0/4125171583 wait complete. 2024-05-09T00:00:51.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 32411 41.61 MB/sec warmup 46 sec latency 207.602 ms 2024-05-09T00:00:51.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:51 smithi012 ceph-mon[42503]: pgmap v462: 97 pgs: 97 active+clean; 530 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.0 MiB/s wr, 1.13k op/s 2024-05-09T00:00:52.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:51 smithi204 ceph-mon[47373]: pgmap v462: 97 pgs: 97 active+clean; 530 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.0 MiB/s wr, 1.13k op/s 2024-05-09T00:00:52.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:51 smithi160 ceph-mon[46152]: pgmap v462: 97 pgs: 97 active+clean; 530 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.0 MiB/s wr, 1.13k op/s 2024-05-09T00:00:52.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33627 42.21 MB/sec warmup 47 sec latency 51.541 ms 2024-05-09T00:00:53.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 34603 42.63 MB/sec warmup 48 sec latency 44.123 ms 2024-05-09T00:00:53.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:53 smithi012 ceph-mon[42503]: pgmap v463: 97 pgs: 97 active+clean; 530 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 719 op/s 2024-05-09T00:00:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:53 smithi204 ceph-mon[47373]: pgmap v463: 97 pgs: 97 active+clean; 530 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 719 op/s 2024-05-09T00:00:54.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:53 smithi160 ceph-mon[46152]: pgmap v463: 97 pgs: 97 active+clean; 530 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 719 op/s 2024-05-09T00:00:54.216 INFO:tasks.workunit.client.0.smithi012.stdout: 10 35248 42.34 MB/sec warmup 49 sec latency 148.742 ms 2024-05-09T00:00:55.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 36247 42.50 MB/sec warmup 50 sec latency 46.015 ms 2024-05-09T00:00:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:55 smithi012 ceph-mon[42503]: pgmap v464: 97 pgs: 97 active+clean; 530 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 719 op/s 2024-05-09T00:00:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:55 smithi204 ceph-mon[47373]: pgmap v464: 97 pgs: 97 active+clean; 530 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 719 op/s 2024-05-09T00:00:56.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:55 smithi160 ceph-mon[46152]: pgmap v464: 97 pgs: 97 active+clean; 530 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 719 op/s 2024-05-09T00:00:56.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 37338 42.89 MB/sec warmup 51 sec latency 67.555 ms 2024-05-09T00:00:57.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 38223 43.08 MB/sec warmup 52 sec latency 62.824 ms 2024-05-09T00:00:57.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:57 smithi012 ceph-mon[42503]: pgmap v465: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.3 MiB/s wr, 987 op/s 2024-05-09T00:00:58.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:57 smithi204 ceph-mon[47373]: pgmap v465: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.3 MiB/s wr, 987 op/s 2024-05-09T00:00:58.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:57 smithi160 ceph-mon[46152]: pgmap v465: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.3 MiB/s wr, 987 op/s 2024-05-09T00:00:58.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 39031 42.93 MB/sec warmup 53 sec latency 73.156 ms 2024-05-09T00:00:58.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:58 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:00:59.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:58 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:00:59.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:58 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:00:59.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 39800 42.98 MB/sec warmup 54 sec latency 42.302 ms 2024-05-09T00:00:59.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:59 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:00:59.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:59 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:00:59.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:00:59 smithi012 ceph-mon[42503]: pgmap v466: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 711 op/s 2024-05-09T00:01:00.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:59 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:01:00.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:59 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:01:00.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:00:59 smithi204 ceph-mon[47373]: pgmap v466: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 711 op/s 2024-05-09T00:01:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:59 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:01:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:59 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:01:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:00:59 smithi160 ceph-mon[46152]: pgmap v466: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 711 op/s 2024-05-09T00:01:00.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 40765 43.00 MB/sec warmup 55 sec latency 90.609 ms 2024-05-09T00:01:01.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 41072 42.47 MB/sec warmup 56 sec latency 679.488 ms 2024-05-09T00:01:01.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:01 smithi012 ceph-mon[42503]: pgmap v467: 97 pgs: 97 active+clean; 582 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 962 op/s 2024-05-09T00:01:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:01 smithi204 ceph-mon[47373]: pgmap v467: 97 pgs: 97 active+clean; 582 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 962 op/s 2024-05-09T00:01:02.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:01 smithi160 ceph-mon[46152]: pgmap v467: 97 pgs: 97 active+clean; 582 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 962 op/s 2024-05-09T00:01:02.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 41072 41.72 MB/sec warmup 57 sec latency 1679.572 ms 2024-05-09T00:01:03.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 41439 41.33 MB/sec warmup 58 sec latency 2246.738 ms 2024-05-09T00:01:04.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 41528 40.73 MB/sec warmup 59 sec latency 372.759 ms 2024-05-09T00:01:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.352+0000 7f15698cd700 1 -- 172.21.15.12:0/2968210350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1564105130 msgr2=0x7f156410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.352+0000 7f15698cd700 1 --2- 172.21.15.12:0/2968210350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1564105130 0x7f156410ccc0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b0a0 tx=0x7f155c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.354+0000 7f15698cd700 1 -- 172.21.15.12:0/2968210350 shutdown_connections 2024-05-09T00:01:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.354+0000 7f15698cd700 1 --2- 172.21.15.12:0/2968210350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1564105130 0x7f156410ccc0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.354+0000 7f15698cd700 1 --2- 172.21.15.12:0/2968210350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15641002f0 0x7f15641006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.354+0000 7f15698cd700 1 --2- 172.21.15.12:0/2968210350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1564100ca0 0x7f1564104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.354+0000 7f15698cd700 1 -- 172.21.15.12:0/2968210350 >> 172.21.15.12:0/2968210350 conn(0x7f1564076c00 msgr2=0x7f1564077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.354+0000 7f15698cd700 1 -- 172.21.15.12:0/2968210350 shutdown_connections 2024-05-09T00:01:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.354+0000 7f15698cd700 1 -- 172.21.15.12:0/2968210350 wait complete. 2024-05-09T00:01:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.354+0000 7f15698cd700 1 Processor -- start 2024-05-09T00:01:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f15698cd700 1 -- start start 2024-05-09T00:01:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f15698cd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15641002f0 0x7f15641972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f15698cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1564105130 0x7f1564197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f15698cd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f156419bbc0 0x7f156419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f15698cd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156419c640 con 0x7f1564105130 2024-05-09T00:01:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f15698cd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156419c7b0 con 0x7f156419bbc0 2024-05-09T00:01:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f15698cd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156419c8f0 con 0x7f15641002f0 2024-05-09T00:01:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f15627fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1564105130 0x7f1564197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f1562ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15641002f0 0x7f15641972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f15627fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1564105130 0x7f1564197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37240/0 (socket says 172.21.15.12:37240) 2024-05-09T00:01:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.355+0000 7f15627fc700 1 -- 172.21.15.12:0/3361123490 learned_addr learned my addr 172.21.15.12:0/3361123490 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f1562ffd700 1 -- 172.21.15.12:0/3361123490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f156419bbc0 msgr2=0x7f156419c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f1562ffd700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f156419bbc0 0x7f156419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f1562ffd700 1 -- 172.21.15.12:0/3361123490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1564105130 msgr2=0x7f1564197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f1562ffd700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1564105130 0x7f1564197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f1562ffd700 1 -- 172.21.15.12:0/3361123490 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c009d00 con 0x7f15641002f0 2024-05-09T00:01:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f1562ffd700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15641002f0 0x7f15641972e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f155400e9f0 tx=0x7f155400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f15688cb700 1 -- 172.21.15.12:0/3361123490 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155400eee0 con 0x7f15641002f0 2024-05-09T00:01:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f15698cd700 1 -- 172.21.15.12:0/3361123490 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15641019b0 con 0x7f15641002f0 2024-05-09T00:01:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f15688cb700 1 -- 172.21.15.12:0/3361123490 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f155400cbd0 con 0x7f15641002f0 2024-05-09T00:01:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f15688cb700 1 -- 172.21.15.12:0/3361123490 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554018680 con 0x7f15641002f0 2024-05-09T00:01:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.356+0000 7f15698cd700 1 -- 172.21.15.12:0/3361123490 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1564101f10 con 0x7f15641002f0 2024-05-09T00:01:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.361+0000 7f1575a87b80 1 Processor -- start 2024-05-09T00:01:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.362+0000 7f15688cb700 1 -- 172.21.15.12:0/3361123490 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f155400cd40 con 0x7f15641002f0 2024-05-09T00:01:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.362+0000 7f1575a87b80 1 -- start start 2024-05-09T00:01:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.363+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3c160 0x55c07db3c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.363+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3ca80 0x55c07db36750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.363+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 0x55c07db370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.363+0000 7f1575a87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07da850f0 con 0x55c07db3ca80 2024-05-09T00:01:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.363+0000 7f1575a87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07da82ab0 con 0x55c07db3c160 2024-05-09T00:01:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.363+0000 7f1575a87b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07da7e3a0 con 0x55c07db36c90 2024-05-09T00:01:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.363+0000 7f15637fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3c160 0x55c07db3c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15637fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3c160 0x55c07db3c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50818/0 (socket says 172.21.15.12:50818) 2024-05-09T00:01:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15637fe700 1 -- 172.21.15.12:0/2405714235 learned_addr learned my addr 172.21.15.12:0/2405714235 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15627fc700 1 --2- 172.21.15.12:0/2405714235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 0x55c07db370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15688cb700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f154c0a1310 0x7f154c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15688cb700 1 -- 172.21.15.12:0/3361123490 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1554014070 con 0x7f15641002f0 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15627fc700 1 -- 172.21.15.12:0/2405714235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3c160 msgr2=0x55c07db3c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15627fc700 1 --2- 172.21.15.12:0/2405714235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3c160 0x55c07db3c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15627fc700 1 -- 172.21.15.12:0/2405714235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3ca80 msgr2=0x55c07db36750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15627fc700 1 --2- 172.21.15.12:0/2405714235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3ca80 0x55c07db36750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15627fc700 1 -- 172.21.15.12:0/2405714235 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07d8ff350 con 0x55c07db36c90 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.364+0000 7f15637fe700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f154c0a1310 0x7f154c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.365+0000 7f15627fc700 1 --2- 172.21.15.12:0/2405714235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 0x55c07db370b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1558009a50 tx=0x7f1558009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c430cb71ae393d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.365+0000 7f1549ffb700 1 -- 172.21.15.12:0/2405714235 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558009550 con 0x55c07db36c90 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.365+0000 7f15637fe700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f154c0a1310 0x7f154c0a37d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b780 tx=0x7f155c0099e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.365+0000 7f1549ffb700 1 -- 172.21.15.12:0/2405714235 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f155800ed90 con 0x55c07db36c90 2024-05-09T00:01:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.365+0000 7f1549ffb700 1 -- 172.21.15.12:0/2405714235 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580123d0 con 0x55c07db36c90 2024-05-09T00:01:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.366+0000 7f1575a87b80 1 -- 172.21.15.12:0/2405714235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 msgr2=0x55c07db370b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.366+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2405714235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 0x55c07db370b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1558009a50 tx=0x7f1558009d60 comp rx=0 tx=0).stop 2024-05-09T00:01:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.366+0000 7f1575a87b80 1 -- 172.21.15.12:0/2405714235 shutdown_connections 2024-05-09T00:01:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.366+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2405714235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 0x55c07db370b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.366+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2405714235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3c160 0x55c07db3c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.366+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2405714235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3ca80 0x55c07db36750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.366+0000 7f1575a87b80 1 -- 172.21.15.12:0/2405714235 >> 172.21.15.12:0/2405714235 conn(0x55c07da8c1b0 msgr2=0x55c07db28840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.367+0000 7f1575a87b80 1 -- 172.21.15.12:0/2405714235 shutdown_connections 2024-05-09T00:01:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.368+0000 7f1575a87b80 1 -- 172.21.15.12:0/2405714235 wait complete. 2024-05-09T00:01:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.368+0000 7f1575a87b80 1 -- start start 2024-05-09T00:01:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.368+0000 7f1575a87b80 1 Processor -- start 2024-05-09T00:01:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 0x55c07db42a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3c160 0x55c07db3fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db40440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f1575a87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07da1a500 con 0x55c07db3c160 2024-05-09T00:01:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f1575a87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07da1a1a0 con 0x55c07db3ca80 2024-05-09T00:01:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f1575a87b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07d901640 con 0x55c07db36c90 2024-05-09T00:01:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f1575a87b80 10 client.0 _ll_register_callbacks cb 0x55c07da23ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f15637fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db40440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f15637fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db40440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50822/0 (socket says 172.21.15.12:50822) 2024-05-09T00:01:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f15637fe700 1 -- 172.21.15.12:0/2437699405 learned_addr learned my addr 172.21.15.12:0/2437699405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.369+0000 7f1562ffd700 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 0x55c07db42a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.370+0000 7f15637fe700 1 -- 172.21.15.12:0/2437699405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 msgr2=0x55c07db42a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.370+0000 7f15637fe700 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 0x55c07db42a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.370+0000 7f15637fe700 1 -- 172.21.15.12:0/2437699405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3c160 msgr2=0x55c07db3fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.370+0000 7f15637fe700 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3c160 0x55c07db3fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.370+0000 7f15637fe700 1 -- 172.21.15.12:0/2437699405 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15580096b0 con 0x55c07db3ca80 2024-05-09T00:01:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.370+0000 7f15637fe700 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db40440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f155c009b80 tx=0x7f155c006210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbdb6de016ab78e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.370+0000 7f1542ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:01:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.371+0000 7f1542ffd700 1 -- 172.21.15.12:0/2437699405 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00e040 con 0x55c07db3ca80 2024-05-09T00:01:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.371+0000 7f1542ffd700 1 -- 172.21.15.12:0/2437699405 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f155c01d650 con 0x55c07db3ca80 2024-05-09T00:01:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.371+0000 7f1575a87b80 1 -- 172.21.15.12:0/2437699405 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c07d9d7210 con 0x55c07db3ca80 2024-05-09T00:01:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.373+0000 7f1542ffd700 1 -- 172.21.15.12:0/2437699405 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f155c01d810 con 0x55c07db3ca80 2024-05-09T00:01:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.374+0000 7f1575a87b80 10 client.29300 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.374+0000 7f1575a87b80 1 -- 172.21.15.12:0/2437699405 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55c07d8abb00 con 0x55c07db3ca80 2024-05-09T00:01:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.377+0000 7f1542ffd700 1 -- 172.21.15.12:0/2437699405 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c07d9d7210 con 0x55c07db3ca80 2024-05-09T00:01:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.377+0000 7f1542ffd700 1 -- 172.21.15.12:0/2437699405 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f155c028bb0 con 0x55c07db3ca80 2024-05-09T00:01:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.378+0000 7f1575a87b80 10 client.29300 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.378+0000 7f1575a87b80 10 client.29300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.378+0000 7f1575a87b80 20 client.29300 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.378+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c07db50470 0x55c07db52890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.378+0000 7f1575a87b80 4 client.29300 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.378+0000 7f1575a87b80 1 -- 172.21.15.12:0/2437699405 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c07dafd7b0 con 0x55c07db50470 2024-05-09T00:01:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.379+0000 7f1562ffd700 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c07db50470 0x55c07db52890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.379+0000 7f1562ffd700 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c07db50470 0x55c07db52890 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a069e6b20fac7730 server_cookie=e443089048c72e3 in_seq=0 out_seq=0 2024-05-09T00:01:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.379+0000 7f1542ffd700 10 client.29300 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.380+0000 7f1542ffd700 1 -- 172.21.15.12:0/2437699405 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c07dafd7b0 con 0x55c07db50470 2024-05-09T00:01:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.380+0000 7f1542ffd700 10 client.29300 handle_command_reply: tid=0 2024-05-09T00:01:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.380+0000 7f1575a87b80 1 client.29300 shutdown 2024-05-09T00:01:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.380+0000 7f1575a87b80 1 -- 172.21.15.12:0/2437699405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 msgr2=0x55c07db40440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db40440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f155c009b80 tx=0x7f155c006210 comp rx=0 tx=0).stop 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1575a87b80 1 -- 172.21.15.12:0/2437699405 shutdown_connections 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c07db50470 0x55c07db52890 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db40440 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db36c90 0x55c07db42a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1542ffd700 0 client.29300 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2437699405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3c160 0x55c07db3fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1575a87b80 1 -- 172.21.15.12:0/2437699405 >> 172.21.15.12:0/2437699405 conn(0x55c07da8c1b0 msgr2=0x55c07db3acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1575a87b80 1 -- 172.21.15.12:0/2437699405 shutdown_connections 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1575a87b80 1 -- 172.21.15.12:0/2437699405 wait complete. 2024-05-09T00:01:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.381+0000 7f1575a87b80 20 client.29300 trim_cache size 0 max 16384 2024-05-09T00:01:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.419+0000 7f1575a87b80 1 Processor -- start 2024-05-09T00:01:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.419+0000 7f1575a87b80 1 -- start start 2024-05-09T00:01:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.419+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db36c90 0x55c07db3fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.422+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 0x55c07db403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.422+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db37070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.422+0000 7f1575a87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07db41190 con 0x55c07db36c90 2024-05-09T00:01:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.422+0000 7f1575a87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07db40c30 con 0x55c07db3ca80 2024-05-09T00:01:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.422+0000 7f1575a87b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07db57aa0 con 0x55c07db3c160 2024-05-09T00:01:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f1562ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db36c90 0x55c07db3fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f15637fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db37070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f1562ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db36c90 0x55c07db3fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37260/0 (socket says 172.21.15.12:37260) 2024-05-09T00:01:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f15637fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db37070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50836/0 (socket says 172.21.15.12:50836) 2024-05-09T00:01:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f1562ffd700 1 -- 172.21.15.12:0/4285488225 learned_addr learned my addr 172.21.15.12:0/4285488225 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f15627fc700 1 --2- 172.21.15.12:0/4285488225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 0x55c07db403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f1562ffd700 1 -- 172.21.15.12:0/4285488225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 msgr2=0x55c07db403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f1562ffd700 1 --2- 172.21.15.12:0/4285488225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 0x55c07db403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f1562ffd700 1 -- 172.21.15.12:0/4285488225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 msgr2=0x55c07db37070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f1562ffd700 1 --2- 172.21.15.12:0/4285488225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db37070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.423+0000 7f1562ffd700 1 -- 172.21.15.12:0/4285488225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07da44500 con 0x55c07db36c90 2024-05-09T00:01:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.424+0000 7f1562ffd700 1 --2- 172.21.15.12:0/4285488225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db36c90 0x55c07db3fca0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f155400c5b0 tx=0x7f1554005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e37f8f2a0b6f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.424+0000 7f154a7fc700 1 -- 172.21.15.12:0/4285488225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155409dd90 con 0x55c07db36c90 2024-05-09T00:01:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.424+0000 7f154a7fc700 1 -- 172.21.15.12:0/4285488225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15540a23e0 con 0x55c07db36c90 2024-05-09T00:01:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.424+0000 7f154a7fc700 1 -- 172.21.15.12:0/4285488225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15540996b0 con 0x55c07db36c90 2024-05-09T00:01:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.426+0000 7f1575a87b80 1 -- 172.21.15.12:0/4285488225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db36c90 msgr2=0x55c07db3fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.426+0000 7f1575a87b80 1 --2- 172.21.15.12:0/4285488225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db36c90 0x55c07db3fca0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f155400c5b0 tx=0x7f1554005760 comp rx=0 tx=0).stop 2024-05-09T00:01:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.426+0000 7f1575a87b80 1 -- 172.21.15.12:0/4285488225 shutdown_connections 2024-05-09T00:01:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.426+0000 7f1575a87b80 1 --2- 172.21.15.12:0/4285488225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db3ca80 0x55c07db37070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.426+0000 7f1575a87b80 1 --2- 172.21.15.12:0/4285488225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db36c90 0x55c07db3fca0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.426+0000 7f1575a87b80 1 --2- 172.21.15.12:0/4285488225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 0x55c07db403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.426+0000 7f1575a87b80 1 -- 172.21.15.12:0/4285488225 >> 172.21.15.12:0/4285488225 conn(0x55c07da8c1b0 msgr2=0x55c07db3f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.426+0000 7f1575a87b80 1 -- 172.21.15.12:0/4285488225 shutdown_connections 2024-05-09T00:01:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.426+0000 7f1575a87b80 1 -- 172.21.15.12:0/4285488225 wait complete. 2024-05-09T00:01:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.427+0000 7f1575a87b80 1 -- start start 2024-05-09T00:01:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.427+0000 7f1575a87b80 1 Processor -- start 2024-05-09T00:01:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.428+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db36c90 0x55c07db43420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.428+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 0x55c07db43960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.428+0000 7f1575a87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3ca80 0x55c07dba5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.428+0000 7f1575a87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07db44100 con 0x55c07db3ca80 2024-05-09T00:01:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.428+0000 7f1575a87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07db44270 con 0x55c07db36c90 2024-05-09T00:01:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.428+0000 7f1575a87b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07dbb6eb0 con 0x55c07db3c160 2024-05-09T00:01:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.428+0000 7f1575a87b80 10 client.0 _ll_register_callbacks cb 0x55c07da23ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.428+0000 7f1562ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db36c90 0x55c07db43420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.429+0000 7f15627fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 0x55c07db43960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.429+0000 7f15627fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 0x55c07db43960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:49362/0 (socket says 172.21.15.12:49362) 2024-05-09T00:01:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.429+0000 7f15627fc700 1 -- 172.21.15.12:0/2033414005 learned_addr learned my addr 172.21.15.12:0/2033414005 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.431+0000 7f15637fe700 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3ca80 0x55c07dba5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.431+0000 7f1562ffd700 1 -- 172.21.15.12:0/2033414005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 msgr2=0x55c07db43960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.431+0000 7f1562ffd700 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 0x55c07db43960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.431+0000 7f1562ffd700 1 -- 172.21.15.12:0/2033414005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3ca80 msgr2=0x55c07dba5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.431+0000 7f1562ffd700 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3ca80 0x55c07dba5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.432+0000 7f1562ffd700 1 -- 172.21.15.12:0/2033414005 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1564101f10 con 0x55c07db36c90 2024-05-09T00:01:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.438+0000 7f1562ffd700 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db36c90 0x55c07db43420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f15540ca650 tx=0x7f1554005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82f1ac7b8548ddf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.438+0000 7f1542ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:01:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.449+0000 7f1542ffd700 1 -- 172.21.15.12:0/2033414005 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554099650 con 0x55c07db36c90 2024-05-09T00:01:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.449+0000 7f1542ffd700 1 -- 172.21.15.12:0/2033414005 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f155409a1c0 con 0x55c07db36c90 2024-05-09T00:01:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.449+0000 7f1575a87b80 1 -- 172.21.15.12:0/2033414005 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f155800ed90 con 0x55c07db36c90 2024-05-09T00:01:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.508+0000 7f1542ffd700 1 -- 172.21.15.12:0/2033414005 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f155409a3c0 con 0x55c07db36c90 2024-05-09T00:01:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.512+0000 7f1575a87b80 10 client.29306 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.512+0000 7f1575a87b80 1 -- 172.21.15.12:0/2033414005 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55c07d903c70 con 0x55c07db36c90 2024-05-09T00:01:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.520+0000 7f1542ffd700 1 -- 172.21.15.12:0/2033414005 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f155800ed90 con 0x55c07db36c90 2024-05-09T00:01:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.520+0000 7f1542ffd700 1 -- 172.21.15.12:0/2033414005 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1554094b70 con 0x55c07db36c90 2024-05-09T00:01:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.521+0000 7f1575a87b80 10 client.29306 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.521+0000 7f1575a87b80 10 client.29306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.521+0000 7f1575a87b80 20 client.29306 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.521+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c07dbb8300 0x55c07dbba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.521+0000 7f1575a87b80 4 client.29306 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:01:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.521+0000 7f1575a87b80 1 -- 172.21.15.12:0/2033414005 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c07da76e60 con 0x55c07dbb8300 2024-05-09T00:01:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.527+0000 7f15627fc700 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c07dbb8300 0x55c07dbba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.546+0000 7f15627fc700 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c07dbb8300 0x55c07dbba720 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=791e0bb1e2ad3bef server_cookie=757ba3b23a8369da in_seq=0 out_seq=0 2024-05-09T00:01:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:04.547+0000 7f1542ffd700 10 client.29306 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:04 smithi204 ceph-mon[47373]: osdmap e111: 12 total, 12 up, 12 in 2024-05-09T00:01:04.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:04 smithi204 ceph-mon[47373]: pgmap v469: 97 pgs: 97 active+clean; 582 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 622 op/s 2024-05-09T00:01:04.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:04 smithi160 ceph-mon[46152]: osdmap e111: 12 total, 12 up, 12 in 2024-05-09T00:01:04.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:04 smithi160 ceph-mon[46152]: pgmap v469: 97 pgs: 97 active+clean; 582 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 622 op/s 2024-05-09T00:01:04.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:04 smithi012 ceph-mon[42503]: osdmap e111: 12 total, 12 up, 12 in 2024-05-09T00:01:04.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:04 smithi012 ceph-mon[42503]: pgmap v469: 97 pgs: 97 active+clean; 582 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 622 op/s 2024-05-09T00:01:05.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 41567 40.12 MB/sec warmup 60 sec latency 824.081 ms 2024-05-09T00:01:05.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:05 smithi012 ceph-mon[42503]: pgmap v470: 97 pgs: 97 active+clean; 582 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 622 op/s 2024-05-09T00:01:06.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:05 smithi204 ceph-mon[47373]: pgmap v470: 97 pgs: 97 active+clean; 582 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 622 op/s 2024-05-09T00:01:06.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:05 smithi160 ceph-mon[46152]: pgmap v470: 97 pgs: 97 active+clean; 582 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 622 op/s 2024-05-09T00:01:06.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 41607 39.47 MB/sec warmup 61 sec latency 1112.979 ms 2024-05-09T00:01:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1542ffd700 1 -- 172.21.15.12:0/2033414005 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c07da76e60 con 0x55c07dbb8300 2024-05-09T00:01:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1542ffd700 10 client.29306 handle_command_reply: tid=0 2024-05-09T00:01:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 client.29306 shutdown 2024-05-09T00:01:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 -- 172.21.15.12:0/2033414005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db36c90 msgr2=0x55c07db43420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db36c90 0x55c07db43420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f15540ca650 tx=0x7f1554005ce0 comp rx=0 tx=0).stop 2024-05-09T00:01:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 -- 172.21.15.12:0/2033414005 shutdown_connections 2024-05-09T00:01:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c07db3ca80 0x55c07dba5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c07dbb8300 0x55c07dbba720 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c07db36c90 0x55c07db43420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 --2- 172.21.15.12:0/2033414005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c07db3c160 0x55c07db43960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1542ffd700 0 client.29306 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 -- 172.21.15.12:0/2033414005 >> 172.21.15.12:0/2033414005 conn(0x55c07da8c1b0 msgr2=0x55c07db3e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 -- 172.21.15.12:0/2033414005 shutdown_connections 2024-05-09T00:01:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.418+0000 7f1575a87b80 1 -- 172.21.15.12:0/2033414005 wait complete. 2024-05-09T00:01:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f1575a87b80 20 client.29306 trim_cache size 0 max 16384 2024-05-09T00:01:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:01:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:01:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:01:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:01:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f15698cd700 1 -- 172.21.15.12:0/3361123490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f154c0a1310 msgr2=0x7f154c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f15698cd700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f154c0a1310 0x7f154c0a37d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b780 tx=0x7f155c0099e0 comp rx=0 tx=0).stop 2024-05-09T00:01:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f15698cd700 1 -- 172.21.15.12:0/3361123490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15641002f0 msgr2=0x7f15641972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f15698cd700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15641002f0 0x7f15641972e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f155400e9f0 tx=0x7f155400ed00 comp rx=0 tx=0).stop 2024-05-09T00:01:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f15698cd700 1 -- 172.21.15.12:0/3361123490 shutdown_connections 2024-05-09T00:01:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f15698cd700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f154c0a1310 0x7f154c0a37d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f15698cd700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f156419bbc0 0x7f156419c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f15698cd700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15641002f0 0x7f15641972e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f15698cd700 1 --2- 172.21.15.12:0/3361123490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1564105130 0x7f1564197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.419+0000 7f15698cd700 1 -- 172.21.15.12:0/3361123490 >> 172.21.15.12:0/3361123490 conn(0x7f1564076c00 msgr2=0x7f1564069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.421+0000 7f15698cd700 1 -- 172.21.15.12:0/3361123490 shutdown_connections 2024-05-09T00:01:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:06.422+0000 7f15698cd700 1 -- 172.21.15.12:0/3361123490 wait complete. 2024-05-09T00:01:07.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 42141 39.50 MB/sec warmup 62 sec latency 343.321 ms 2024-05-09T00:01:07.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:07 smithi012 ceph-mon[42503]: pgmap v471: 97 pgs: 97 active+clean; 753 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 24 MiB/s wr, 1.03k op/s 2024-05-09T00:01:08.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:07 smithi204 ceph-mon[47373]: pgmap v471: 97 pgs: 97 active+clean; 753 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 24 MiB/s wr, 1.03k op/s 2024-05-09T00:01:08.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:07 smithi160 ceph-mon[46152]: pgmap v471: 97 pgs: 97 active+clean; 753 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 24 MiB/s wr, 1.03k op/s 2024-05-09T00:01:08.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 42655 39.38 MB/sec warmup 63 sec latency 76.075 ms 2024-05-09T00:01:09.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 43413 39.22 MB/sec warmup 64 sec latency 90.081 ms 2024-05-09T00:01:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:09 smithi012 ceph-mon[42503]: pgmap v472: 97 pgs: 97 active+clean; 753 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 24 MiB/s wr, 1.03k op/s 2024-05-09T00:01:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:09 smithi204 ceph-mon[47373]: pgmap v472: 97 pgs: 97 active+clean; 753 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 24 MiB/s wr, 1.03k op/s 2024-05-09T00:01:10.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:09 smithi160 ceph-mon[46152]: pgmap v472: 97 pgs: 97 active+clean; 753 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 24 MiB/s wr, 1.03k op/s 2024-05-09T00:01:10.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 44564 39.47 MB/sec warmup 65 sec latency 69.462 ms 2024-05-09T00:01:11.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 45546 39.88 MB/sec warmup 66 sec latency 41.693 ms 2024-05-09T00:01:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:11 smithi012 ceph-mon[42503]: osdmap e112: 12 total, 12 up, 12 in 2024-05-09T00:01:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:11 smithi012 ceph-mon[42503]: pgmap v474: 97 pgs: 97 active+clean; 601 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 31 MiB/s wr, 1.58k op/s 2024-05-09T00:01:12.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 46238 39.84 MB/sec warmup 67 sec latency 35.300 ms 2024-05-09T00:01:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:11 smithi204 ceph-mon[47373]: osdmap e112: 12 total, 12 up, 12 in 2024-05-09T00:01:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:11 smithi204 ceph-mon[47373]: pgmap v474: 97 pgs: 97 active+clean; 601 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 31 MiB/s wr, 1.58k op/s 2024-05-09T00:01:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:11 smithi160 ceph-mon[46152]: osdmap e112: 12 total, 12 up, 12 in 2024-05-09T00:01:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:11 smithi160 ceph-mon[46152]: pgmap v474: 97 pgs: 97 active+clean; 601 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 31 MiB/s wr, 1.58k op/s 2024-05-09T00:01:13.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 47098 39.84 MB/sec warmup 68 sec latency 37.728 ms 2024-05-09T00:01:13.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:13 smithi012 ceph-mon[42503]: pgmap v475: 97 pgs: 97 active+clean; 601 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 25 MiB/s wr, 1.28k op/s 2024-05-09T00:01:14.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:13 smithi204 ceph-mon[47373]: pgmap v475: 97 pgs: 97 active+clean; 601 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 25 MiB/s wr, 1.28k op/s 2024-05-09T00:01:14.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:13 smithi160 ceph-mon[46152]: pgmap v475: 97 pgs: 97 active+clean; 601 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 25 MiB/s wr, 1.28k op/s 2024-05-09T00:01:14.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 48027 40.05 MB/sec warmup 69 sec latency 60.137 ms 2024-05-09T00:01:15.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 48928 40.13 MB/sec warmup 70 sec latency 62.475 ms 2024-05-09T00:01:15.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:15 smithi012 ceph-mon[42503]: pgmap v476: 97 pgs: 97 active+clean; 601 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 25 MiB/s wr, 1.28k op/s 2024-05-09T00:01:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:15 smithi204 ceph-mon[47373]: pgmap v476: 97 pgs: 97 active+clean; 601 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 25 MiB/s wr, 1.28k op/s 2024-05-09T00:01:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:15 smithi160 ceph-mon[46152]: pgmap v476: 97 pgs: 97 active+clean; 601 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 25 MiB/s wr, 1.28k op/s 2024-05-09T00:01:16.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 49682 40.10 MB/sec warmup 71 sec latency 140.492 ms 2024-05-09T00:01:17.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 50651 40.41 MB/sec warmup 72 sec latency 42.950 ms 2024-05-09T00:01:17.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:17 smithi012 ceph-mon[42503]: pgmap v477: 97 pgs: 97 active+clean; 632 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.4 MiB/s wr, 865 op/s 2024-05-09T00:01:18.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:17 smithi204 ceph-mon[47373]: pgmap v477: 97 pgs: 97 active+clean; 632 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.4 MiB/s wr, 865 op/s 2024-05-09T00:01:18.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:17 smithi160 ceph-mon[46152]: pgmap v477: 97 pgs: 97 active+clean; 632 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.4 MiB/s wr, 865 op/s 2024-05-09T00:01:18.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51481 40.36 MB/sec warmup 73 sec latency 68.644 ms 2024-05-09T00:01:18.629 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:01:18.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.803+0000 7f5d61c56700 1 -- 172.21.15.12:0/724810499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d5c107ee0 msgr2=0x7f5d5c1143f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.803+0000 7f5d61c56700 1 --2- 172.21.15.12:0/724810499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d5c107ee0 0x7f5d5c1143f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400b0a0 tx=0x7f5d5400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.803+0000 7f5d61c56700 1 -- 172.21.15.12:0/724810499 shutdown_connections 2024-05-09T00:01:18.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.803+0000 7f5d61c56700 1 --2- 172.21.15.12:0/724810499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d5c107ee0 0x7f5d5c1143f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.803+0000 7f5d61c56700 1 --2- 172.21.15.12:0/724810499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d5c10c540 0x7f5d5c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.803+0000 7f5d61c56700 1 --2- 172.21.15.12:0/724810499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c107520 0x7f5d5c1079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.803+0000 7f5d61c56700 1 -- 172.21.15.12:0/724810499 >> 172.21.15.12:0/724810499 conn(0x7f5d5c076dc0 msgr2=0x7f5d5c105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:18.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.804+0000 7f5d61c56700 1 -- 172.21.15.12:0/724810499 shutdown_connections 2024-05-09T00:01:18.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.804+0000 7f5d61c56700 1 -- 172.21.15.12:0/724810499 wait complete. 2024-05-09T00:01:18.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.804+0000 7f5d61c56700 1 Processor -- start 2024-05-09T00:01:18.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d61c56700 1 -- start start 2024-05-09T00:01:18.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d61c56700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c107520 0x7f5d5c1a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d61c56700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d5c107ee0 0x7f5d5c1a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d61c56700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d5c10c540 0x7f5d5c19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d61c56700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c19dea0 con 0x7f5d5c10c540 2024-05-09T00:01:18.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d61c56700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c19e010 con 0x7f5d5c107ee0 2024-05-09T00:01:18.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d61c56700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c19e1d0 con 0x7f5d5c107520 2024-05-09T00:01:18.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c107520 0x7f5d5c1a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c107520 0x7f5d5c1a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53546/0 (socket says 172.21.15.12:53546) 2024-05-09T00:01:18.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/249166447 learned_addr learned my addr 172.21.15.12:0/249166447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:18.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d5affd700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d5c107ee0 0x7f5d5c1a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d5bfff700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d5c10c540 0x7f5d5c19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/249166447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d5c107ee0 msgr2=0x7f5d5c1a3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d5c107ee0 0x7f5d5c1a3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/249166447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d5c10c540 msgr2=0x7f5d5c19d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d5c10c540 0x7f5d5c19d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.805+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/249166447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d54009d00 con 0x7f5d5c107520 2024-05-09T00:01:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.806+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c107520 0x7f5d5c1a3700 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00d980 tx=0x7f5d4c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.806+0000 7f5d58ff9700 1 -- 172.21.15.12:0/249166447 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c004020 con 0x7f5d5c107520 2024-05-09T00:01:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.806+0000 7f5d58ff9700 1 -- 172.21.15.12:0/249166447 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d4c00f690 con 0x7f5d5c107520 2024-05-09T00:01:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.806+0000 7f5d61c56700 1 -- 172.21.15.12:0/249166447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c19e4b0 con 0x7f5d5c107520 2024-05-09T00:01:18.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.806+0000 7f5d58ff9700 1 -- 172.21.15.12:0/249166447 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c003a60 con 0x7f5d5c107520 2024-05-09T00:01:18.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.806+0000 7f5d61c56700 1 -- 172.21.15.12:0/249166447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1aa6f0 con 0x7f5d5c107520 2024-05-09T00:01:18.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.809+0000 7f5d58ff9700 1 -- 172.21.15.12:0/249166447 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5d4c00fb00 con 0x7f5d5c107520 2024-05-09T00:01:18.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.809+0000 7f5d58ff9700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d440a1310 0x7f5d440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.810+0000 7f5d58ff9700 1 -- 172.21.15.12:0/249166447 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5d4c0cc810 con 0x7f5d5c107520 2024-05-09T00:01:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.810+0000 7f5d5affd700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d440a1310 0x7f5d440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.810+0000 7f5d5affd700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d440a1310 0x7f5d440a37d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c19ec50 tx=0x7f5d50009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.810+0000 7f5d6de10b80 1 Processor -- start 2024-05-09T00:01:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.810+0000 7f5d6de10b80 1 -- start start 2024-05-09T00:01:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cee00 0x562ecb0cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cf720 0x562ecb0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecb020150 con 0x562ecb0cf720 2024-05-09T00:01:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecb01db10 con 0x562ecb0cee00 2024-05-09T00:01:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecb019400 con 0x562ecb0c9890 2024-05-09T00:01:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cf720 0x562ecb0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cf720 0x562ecb0c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60360/0 (socket says 172.21.15.12:60360) 2024-05-09T00:01:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53560/0 (socket says 172.21.15.12:53560) 2024-05-09T00:01:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5affd700 1 -- 172.21.15.12:0/4176084683 learned_addr learned my addr 172.21.15.12:0/4176084683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/4176084683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cee00 0x562ecb0cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5bfff700 1 -- 172.21.15.12:0/4176084683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cee00 msgr2=0x562ecb0cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5bfff700 1 --2- 172.21.15.12:0/4176084683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cee00 0x562ecb0cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5bfff700 1 -- 172.21.15.12:0/4176084683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cf720 msgr2=0x562ecb0c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5bfff700 1 --2- 172.21.15.12:0/4176084683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cf720 0x562ecb0c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5bfff700 1 -- 172.21.15.12:0/4176084683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ecae9a3b0 con 0x562ecb0c9890 2024-05-09T00:01:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5affd700 1 --2- 172.21.15.12:0/4176084683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cf720 0x562ecb0c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.811+0000 7f5d5bfff700 1 --2- 172.21.15.12:0/4176084683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0c9cd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c1a4400 tx=0x7f5d54009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78b774204575f6f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.812+0000 7f5d41ffb700 1 -- 172.21.15.12:0/4176084683 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400e040 con 0x562ecb0c9890 2024-05-09T00:01:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.812+0000 7f5d41ffb700 1 -- 172.21.15.12:0/4176084683 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d54006000 con 0x562ecb0c9890 2024-05-09T00:01:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.812+0000 7f5d41ffb700 1 -- 172.21.15.12:0/4176084683 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54007740 con 0x562ecb0c9890 2024-05-09T00:01:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 -- 172.21.15.12:0/4176084683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 msgr2=0x562ecb0c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/4176084683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0c9cd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c1a4400 tx=0x7f5d54009360 comp rx=0 tx=0).stop 2024-05-09T00:01:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 -- 172.21.15.12:0/4176084683 shutdown_connections 2024-05-09T00:01:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/4176084683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0c9cd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/4176084683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cee00 0x562ecb0cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/4176084683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cf720 0x562ecb0c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 -- 172.21.15.12:0/4176084683 >> 172.21.15.12:0/4176084683 conn(0x562ecb027210 msgr2=0x562ecb0c3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:18.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 -- 172.21.15.12:0/4176084683 shutdown_connections 2024-05-09T00:01:18.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 -- 172.21.15.12:0/4176084683 wait complete. 2024-05-09T00:01:18.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 -- start start 2024-05-09T00:01:18.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.813+0000 7f5d6de10b80 1 Processor -- start 2024-05-09T00:01:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 0x562ecb0d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cf720 0x562ecb0d3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecafb5560 con 0x562ecb0cee00 2024-05-09T00:01:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecafb5200 con 0x562ecb0cf720 2024-05-09T00:01:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecae9c6a0 con 0x562ecb0c9890 2024-05-09T00:01:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d6de10b80 10 client.0 _ll_register_callbacks cb 0x562ecafbeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 0x562ecb0d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0d5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53576/0 (socket says 172.21.15.12:53576) 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 0x562ecb0d2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60372/0 (socket says 172.21.15.12:60372) 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/2984007795 learned_addr learned my addr 172.21.15.12:0/2984007795 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5bfff700 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cf720 0x562ecb0d3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/2984007795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cf720 msgr2=0x562ecb0d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cf720 0x562ecb0d3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/2984007795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 msgr2=0x562ecb0d2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 0x562ecb0d2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/2984007795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d50017070 con 0x562ecb0c9890 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0d5740 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00d950 tx=0x7f5d4c09a030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81c046cc65d12b9a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.814+0000 7f5d3affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d3affd700 1 -- 172.21.15.12:0/2984007795 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c084040 con 0x562ecb0c9890 2024-05-09T00:01:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d3affd700 1 -- 172.21.15.12:0/2984007795 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5d4c0a1db0 con 0x562ecb0c9890 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d6de10b80 1 -- 172.21.15.12:0/2984007795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ecaf72270 con 0x562ecb0c9890 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d3affd700 1 -- 172.21.15.12:0/2984007795 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d4c099b90 con 0x562ecb0c9890 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d6de10b80 10 client.29341 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d6de10b80 1 -- 172.21.15.12:0/2984007795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f5d5c114cf0 con 0x562ecb0c9890 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d3affd700 1 -- 172.21.15.12:0/2984007795 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ecaf72270 con 0x562ecb0c9890 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d3affd700 1 -- 172.21.15.12:0/2984007795 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5d4c0a1920 con 0x562ecb0c9890 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d6de10b80 10 client.29341 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d6de10b80 10 client.29341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d6de10b80 20 client.29341 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562ecb0e32e0 0x562ecb0e5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d6de10b80 4 client.29341 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.815+0000 7f5d6de10b80 1 -- 172.21.15.12:0/2984007795 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ecb098810 con 0x562ecb0e32e0 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.816+0000 7f5d5affd700 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562ecb0e32e0 0x562ecb0e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.816+0000 7f5d3affd700 10 client.29341 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.816+0000 7f5d5affd700 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562ecb0e32e0 0x562ecb0e5700 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da2396814451362e server_cookie=66f3b09b75d2514c in_seq=0 out_seq=0 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.819+0000 7f5d3affd700 1 -- 172.21.15.12:0/2984007795 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ecb098810 con 0x562ecb0e32e0 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.819+0000 7f5d3affd700 10 client.29341 handle_command_reply: tid=0 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.819+0000 7f5d6de10b80 1 client.29341 shutdown 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.819+0000 7f5d6de10b80 1 -- 172.21.15.12:0/2984007795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 msgr2=0x562ecb0d5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.819+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0d5740 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00d950 tx=0x7f5d4c09a030 comp rx=0 tx=0).stop 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.820+0000 7f5d6de10b80 1 -- 172.21.15.12:0/2984007795 shutdown_connections 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.820+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562ecb0e32e0 0x562ecb0e5700 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.820+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cf720 0x562ecb0d3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.820+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0c9890 0x562ecb0d5740 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.820+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/2984007795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 0x562ecb0d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.820+0000 7f5d3affd700 0 client.29341 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.820+0000 7f5d6de10b80 1 -- 172.21.15.12:0/2984007795 >> 172.21.15.12:0/2984007795 conn(0x562ecb027210 msgr2=0x562ecb0c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.820+0000 7f5d6de10b80 1 -- 172.21.15.12:0/2984007795 shutdown_connections 2024-05-09T00:01:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.820+0000 7f5d6de10b80 1 -- 172.21.15.12:0/2984007795 wait complete. 2024-05-09T00:01:18.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.820+0000 7f5d6de10b80 20 client.29341 trim_cache size 0 max 16384 2024-05-09T00:01:18.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d6de10b80 1 Processor -- start 2024-05-09T00:01:18.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d6de10b80 1 -- start start 2024-05-09T00:01:18.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0c9890 0x562ecb0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cee00 0x562ecb0dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 0x562ecb0c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecb0be800 con 0x562ecb0c9890 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecb0e0860 con 0x562ecb0cee00 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecb0ea8b0 con 0x562ecb0cf720 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0c9890 0x562ecb0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0c9890 0x562ecb0c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60376/0 (socket says 172.21.15.12:60376) 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/753862057 learned_addr learned my addr 172.21.15.12:0/753862057 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d5affd700 1 --2- 172.21.15.12:0/753862057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cee00 0x562ecb0dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d5bfff700 1 --2- 172.21.15.12:0/753862057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 0x562ecb0c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/753862057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 msgr2=0x562ecb0c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.842+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/753862057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 0x562ecb0c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.843+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/753862057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cee00 msgr2=0x562ecb0dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.843+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/753862057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cee00 0x562ecb0dec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.843+0000 7f5d5b7fe700 1 -- 172.21.15.12:0/753862057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ecae46b60 con 0x562ecb0c9890 2024-05-09T00:01:18.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.843+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/753862057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0c9890 0x562ecb0c9350 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0ca020 tx=0x7f5d4c099a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ca0762862ddddf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:18.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.843+0000 7f5d427fc700 1 -- 172.21.15.12:0/753862057 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c084040 con 0x562ecb0c9890 2024-05-09T00:01:18.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.843+0000 7f5d427fc700 1 -- 172.21.15.12:0/753862057 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d4c09a0a0 con 0x562ecb0c9890 2024-05-09T00:01:18.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.843+0000 7f5d427fc700 1 -- 172.21.15.12:0/753862057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c003d10 con 0x562ecb0c9890 2024-05-09T00:01:18.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.844+0000 7f5d6de10b80 1 -- 172.21.15.12:0/753862057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0c9890 msgr2=0x562ecb0c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.844+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/753862057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0c9890 0x562ecb0c9350 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0ca020 tx=0x7f5d4c099a90 comp rx=0 tx=0).stop 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.844+0000 7f5d6de10b80 1 -- 172.21.15.12:0/753862057 shutdown_connections 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.844+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/753862057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 0x562ecb0c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.844+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/753862057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0c9890 0x562ecb0c9350 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.844+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/753862057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0cee00 0x562ecb0dec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.844+0000 7f5d6de10b80 1 -- 172.21.15.12:0/753862057 >> 172.21.15.12:0/753862057 conn(0x562ecb027210 msgr2=0x562ecb0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.844+0000 7f5d6de10b80 1 -- 172.21.15.12:0/753862057 shutdown_connections 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.844+0000 7f5d6de10b80 1 -- 172.21.15.12:0/753862057 wait complete. 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.844+0000 7f5d6de10b80 1 -- start start 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d6de10b80 1 Processor -- start 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0c9890 0x562ecb0da410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 0x562ecb0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d6de10b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 0x562ecb0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecb0cb4c0 con 0x562ecb0cee00 2024-05-09T00:01:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecb0cb630 con 0x562ecb0c9890 2024-05-09T00:01:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d6de10b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecb0cb7f0 con 0x562ecb0cf720 2024-05-09T00:01:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d6de10b80 10 client.0 _ll_register_callbacks cb 0x562ecafbeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d5affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 0x562ecb0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d5affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 0x562ecb0ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60378/0 (socket says 172.21.15.12:60378) 2024-05-09T00:01:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d5affd700 1 -- 172.21.15.12:0/1962890863 learned_addr learned my addr 172.21.15.12:0/1962890863 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.845+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0c9890 0x562ecb0da410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d5bfff700 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 0x562ecb0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d5bfff700 1 -- 172.21.15.12:0/1962890863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0c9890 msgr2=0x562ecb0da410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d5bfff700 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0c9890 0x562ecb0da410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d5bfff700 1 -- 172.21.15.12:0/1962890863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 msgr2=0x562ecb0ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d5bfff700 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 0x562ecb0ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d5bfff700 1 -- 172.21.15.12:0/1962890863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c114cf0 con 0x562ecb0cf720 2024-05-09T00:01:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d5bfff700 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 0x562ecb0caee0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400b070 tx=0x7f5d54007dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac0d7fd7a454f877 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d3affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d3affd700 1 -- 172.21.15.12:0/1962890863 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54029070 con 0x562ecb0cf720 2024-05-09T00:01:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d3affd700 1 -- 172.21.15.12:0/1962890863 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5d5400bcc0 con 0x562ecb0cf720 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.846+0000 7f5d6de10b80 1 -- 172.21.15.12:0/1962890863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d54006000 con 0x562ecb0cf720 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d3affd700 1 -- 172.21.15.12:0/1962890863 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d5400e040 con 0x562ecb0cf720 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d6de10b80 10 client.29347 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d6de10b80 1 -- 172.21.15.12:0/1962890863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x562ecafdf560 con 0x562ecb0cf720 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d3affd700 1 -- 172.21.15.12:0/1962890863 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d5403a020 con 0x562ecb0cf720 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d3affd700 1 -- 172.21.15.12:0/1962890863 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5d54019030 con 0x562ecb0cf720 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d6de10b80 10 client.29347 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d6de10b80 10 client.29347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d6de10b80 20 client.29347 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562ecb14aa90 0x562ecb14ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d6de10b80 4 client.29347 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d6de10b80 1 -- 172.21.15.12:0/1962890863 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ecb011ec0 con 0x562ecb14aa90 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.847+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562ecb14aa90 0x562ecb14ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.848+0000 7f5d3affd700 10 client.29347 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.848+0000 7f5d5b7fe700 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562ecb14aa90 0x562ecb14ceb0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a586103f731c5e4 server_cookie=cdf75dcb1e836170 in_seq=0 out_seq=0 2024-05-09T00:01:18.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.854+0000 7f5d3affd700 1 -- 172.21.15.12:0/1962890863 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562ecb011ec0 con 0x562ecb14aa90 2024-05-09T00:01:18.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.854+0000 7f5d3affd700 10 client.29347 handle_command_reply: tid=0 2024-05-09T00:01:18.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.854+0000 7f5d6de10b80 1 client.29347 shutdown 2024-05-09T00:01:18.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 1 -- 172.21.15.12:0/1962890863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 msgr2=0x562ecb0caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 0x562ecb0caee0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400b070 tx=0x7f5d54007dd0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 1 -- 172.21.15.12:0/1962890863 shutdown_connections 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ecb0cf720 0x562ecb0caee0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562ecb14aa90 0x562ecb14ceb0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ecb0c9890 0x562ecb0da410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 1 --2- 172.21.15.12:0/1962890863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562ecb0cee00 0x562ecb0ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d3affd700 0 client.29347 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 1 -- 172.21.15.12:0/1962890863 >> 172.21.15.12:0/1962890863 conn(0x562ecb027210 msgr2=0x562ecb0ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 1 -- 172.21.15.12:0/1962890863 shutdown_connections 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 1 -- 172.21.15.12:0/1962890863 wait complete. 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.855+0000 7f5d6de10b80 20 client.29347 trim_cache size 0 max 16384 2024-05-09T00:01:18.850 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:01:18.851 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:01:18.851 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:01:18.851 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:01:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 -- 172.21.15.12:0/249166447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d440a1310 msgr2=0x7f5d440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d440a1310 0x7f5d440a37d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c19ec50 tx=0x7f5d50009260 comp rx=0 tx=0).stop 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 -- 172.21.15.12:0/249166447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c107520 msgr2=0x7f5d5c1a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c107520 0x7f5d5c1a3700 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00d980 tx=0x7f5d4c00dc90 comp rx=0 tx=0).stop 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 -- 172.21.15.12:0/249166447 shutdown_connections 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5d440a1310 0x7f5d440a37d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d5c10c540 0x7f5d5c19d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c107520 0x7f5d5c1a3700 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 --2- 172.21.15.12:0/249166447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d5c107ee0 0x7f5d5c1a3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 -- 172.21.15.12:0/249166447 >> 172.21.15.12:0/249166447 conn(0x7f5d5c076dc0 msgr2=0x7f5d5c077300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 -- 172.21.15.12:0/249166447 shutdown_connections 2024-05-09T00:01:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:18.856+0000 7f5d61c56700 1 -- 172.21.15.12:0/249166447 wait complete. 2024-05-09T00:01:18.869 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:01:18.869 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:01:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.049+0000 7fef93864700 1 -- 172.21.15.12:0/1139877049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef8c0fff40 msgr2=0x7fef8c100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.049+0000 7fef93864700 1 --2- 172.21.15.12:0/1139877049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef8c0fff40 0x7fef8c100320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fef88009990 tx=0x7fef88009ca0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.049+0000 7fef93864700 1 -- 172.21.15.12:0/1139877049 shutdown_connections 2024-05-09T00:01:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.049+0000 7fef93864700 1 --2- 172.21.15.12:0/1139877049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef8c105060 0x7fef8c10cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.049+0000 7fef93864700 1 --2- 172.21.15.12:0/1139877049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef8c0fff40 0x7fef8c100320 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.049+0000 7fef93864700 1 --2- 172.21.15.12:0/1139877049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef8c1008f0 0x7fef8c104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.049+0000 7fef93864700 1 -- 172.21.15.12:0/1139877049 >> 172.21.15.12:0/1139877049 conn(0x7fef8c0fb7e0 msgr2=0x7fef8c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.049+0000 7fef93864700 1 -- 172.21.15.12:0/1139877049 shutdown_connections 2024-05-09T00:01:19.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.049+0000 7fef93864700 1 -- 172.21.15.12:0/1139877049 wait complete. 2024-05-09T00:01:19.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.050+0000 7fef93864700 1 Processor -- start 2024-05-09T00:01:19.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.050+0000 7fef93864700 1 -- start start 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.050+0000 7fef93864700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef8c0fff40 0x7fef8c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.050+0000 7fef93864700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef8c105060 0x7fef8c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.050+0000 7fef93864700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef8c19dea0 0x7fef8c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.050+0000 7fef93864700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c19e920 con 0x7fef8c105060 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.050+0000 7fef93864700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c19ea90 con 0x7fef8c0fff40 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.050+0000 7fef93864700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c19ec50 con 0x7fef8c19dea0 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef90dff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef8c105060 0x7fef8c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef90dff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef8c105060 0x7fef8c199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60392/0 (socket says 172.21.15.12:60392) 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef90dff700 1 -- 172.21.15.12:0/2834101544 learned_addr learned my addr 172.21.15.12:0/2834101544 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef91600700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef8c0fff40 0x7fef8c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef91e01700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef8c19dea0 0x7fef8c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef90dff700 1 -- 172.21.15.12:0/2834101544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef8c19dea0 msgr2=0x7fef8c19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef90dff700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef8c19dea0 0x7fef8c19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef90dff700 1 -- 172.21.15.12:0/2834101544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef8c0fff40 msgr2=0x7fef8c1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef90dff700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef8c0fff40 0x7fef8c1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef90dff700 1 -- 172.21.15.12:0/2834101544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef880095f0 con 0x7fef8c105060 2024-05-09T00:01:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef90dff700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef8c105060 0x7fef8c199b00 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e970 tx=0x7fef7c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.051+0000 7fef827fc700 1 -- 172.21.15.12:0/2834101544 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c009790 con 0x7fef8c105060 2024-05-09T00:01:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.052+0000 7fef93864700 1 -- 172.21.15.12:0/2834101544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c1a61b0 con 0x7fef8c105060 2024-05-09T00:01:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.052+0000 7fef827fc700 1 -- 172.21.15.12:0/2834101544 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef7c00c7e0 con 0x7fef8c105060 2024-05-09T00:01:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.052+0000 7fef827fc700 1 -- 172.21.15.12:0/2834101544 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0054e0 con 0x7fef8c105060 2024-05-09T00:01:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.052+0000 7fef93864700 1 -- 172.21.15.12:0/2834101544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c1a6690 con 0x7fef8c105060 2024-05-09T00:01:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.053+0000 7fef93864700 1 -- 172.21.15.12:0/2834101544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c101410 con 0x7fef8c105060 2024-05-09T00:01:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.054+0000 7fef827fc700 1 -- 172.21.15.12:0/2834101544 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fef7c00c950 con 0x7fef8c105060 2024-05-09T00:01:19.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.055+0000 7fef827fc700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fef780a1290 0x7fef780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.055+0000 7fef827fc700 1 -- 172.21.15.12:0/2834101544 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fef7c014070 con 0x7fef8c105060 2024-05-09T00:01:19.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.057+0000 7fef91600700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fef780a1290 0x7fef780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.057+0000 7fef91600700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fef780a1290 0x7fef780a3750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fef88006010 tx=0x7fef88005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.062+0000 7fef827fc700 1 -- 172.21.15.12:0/2834101544 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fef7c0d0030 con 0x7fef8c105060 2024-05-09T00:01:19.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 52466 40.48 MB/sec warmup 74 sec latency 69.854 ms 2024-05-09T00:01:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.380+0000 7fef93864700 1 -- 172.21.15.12:0/2834101544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef8c19a8c0 con 0x7fef8c105060 2024-05-09T00:01:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.381+0000 7fef827fc700 1 -- 172.21.15.12:0/2834101544 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+10882 (secure 0 0 0) 0x7fef7c096370 con 0x7fef8c105060 2024-05-09T00:01:19.375 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:01:19.376 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:00:22.819142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_28844":{"gid":28844,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/864772354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":864772354},{"type":"v1","addr":"172.21.15.204:6837","nonce":864772354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:01:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 -- 172.21.15.12:0/2834101544 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fef780a1290 msgr2=0x7fef780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fef780a1290 0x7fef780a3750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fef88006010 tx=0x7fef88005960 comp rx=0 tx=0).stop 2024-05-09T00:01:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 -- 172.21.15.12:0/2834101544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef8c105060 msgr2=0x7fef8c199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef8c105060 0x7fef8c199b00 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e970 tx=0x7fef7c00ec80 comp rx=0 tx=0).stop 2024-05-09T00:01:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 -- 172.21.15.12:0/2834101544 shutdown_connections 2024-05-09T00:01:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fef780a1290 0x7fef780a3750 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef8c19dea0 0x7fef8c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef8c0fff40 0x7fef8c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 --2- 172.21.15.12:0/2834101544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef8c105060 0x7fef8c199b00 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 -- 172.21.15.12:0/2834101544 >> 172.21.15.12:0/2834101544 conn(0x7fef8c0fb7e0 msgr2=0x7fef8c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 -- 172.21.15.12:0/2834101544 shutdown_connections 2024-05-09T00:01:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.384+0000 7fef93864700 1 -- 172.21.15.12:0/2834101544 wait complete. 2024-05-09T00:01:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 61 2024-05-09T00:01:19.396 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:01:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.571+0000 7ff858db3700 1 -- 172.21.15.12:0/3940375340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8540ffdd0 msgr2=0x7ff854100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.571+0000 7ff858db3700 1 --2- 172.21.15.12:0/3940375340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8540ffdd0 0x7ff854100250 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7ff848009990 tx=0x7ff848009ca0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.572+0000 7ff858db3700 1 -- 172.21.15.12:0/3940375340 shutdown_connections 2024-05-09T00:01:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.572+0000 7ff858db3700 1 --2- 172.21.15.12:0/3940375340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff854100790 0x7ff85410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.572+0000 7ff858db3700 1 --2- 172.21.15.12:0/3940375340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff854105e90 0x7ff854106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.572+0000 7ff858db3700 1 --2- 172.21.15.12:0/3940375340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8540ffdd0 0x7ff854100250 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.572+0000 7ff858db3700 1 -- 172.21.15.12:0/3940375340 >> 172.21.15.12:0/3940375340 conn(0x7ff854076fd0 msgr2=0x7ff8540773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.572+0000 7ff858db3700 1 -- 172.21.15.12:0/3940375340 shutdown_connections 2024-05-09T00:01:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.572+0000 7ff858db3700 1 -- 172.21.15.12:0/3940375340 wait complete. 2024-05-09T00:01:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff858db3700 1 Processor -- start 2024-05-09T00:01:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff858db3700 1 -- start start 2024-05-09T00:01:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff858db3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8540ffdd0 0x7ff854198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff858db3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff854100790 0x7ff8541985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff858db3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff854105e90 0x7ff8541a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff858db3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff854198e80 con 0x7ff8540ffdd0 2024-05-09T00:01:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff858db3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8541a3640 con 0x7ff854100790 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff858db3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8541a3780 con 0x7ff854105e90 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff85259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8540ffdd0 0x7ff854198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff85259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8540ffdd0 0x7ff854198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60408/0 (socket says 172.21.15.12:60408) 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff85259c700 1 -- 172.21.15.12:0/607307815 learned_addr learned my addr 172.21.15.12:0/607307815 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff851d9b700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff854100790 0x7ff8541985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff852d9d700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff854105e90 0x7ff8541a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff85259c700 1 -- 172.21.15.12:0/607307815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff854105e90 msgr2=0x7ff8541a3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff85259c700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff854105e90 0x7ff8541a3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff85259c700 1 -- 172.21.15.12:0/607307815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff854100790 msgr2=0x7ff8541985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff85259c700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff854100790 0x7ff8541985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.573+0000 7ff85259c700 1 -- 172.21.15.12:0/607307815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8480095f0 con 0x7ff8540ffdd0 2024-05-09T00:01:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.574+0000 7ff852d9d700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff854105e90 0x7ff8541a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.574+0000 7ff851d9b700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff854100790 0x7ff8541985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.574+0000 7ff85259c700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8540ffdd0 0x7ff854198080 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7ff84400e9a0 tx=0x7ff84400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.574+0000 7ff8437fe700 1 -- 172.21.15.12:0/607307815 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844009790 con 0x7ff8540ffdd0 2024-05-09T00:01:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.574+0000 7ff8437fe700 1 -- 172.21.15.12:0/607307815 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff844010430 con 0x7ff8540ffdd0 2024-05-09T00:01:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.574+0000 7ff858db3700 1 -- 172.21.15.12:0/607307815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff854195ed0 con 0x7ff8540ffdd0 2024-05-09T00:01:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.574+0000 7ff8437fe700 1 -- 172.21.15.12:0/607307815 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844018680 con 0x7ff8540ffdd0 2024-05-09T00:01:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.574+0000 7ff858db3700 1 -- 172.21.15.12:0/607307815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff854196400 con 0x7ff8540ffdd0 2024-05-09T00:01:19.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.578+0000 7fc781ce5700 1 -- 172.21.15.12:0/4004547380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc77c1000f0 msgr2=0x7fc77c100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.578+0000 7fc781ce5700 1 --2- 172.21.15.12:0/4004547380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc77c1000f0 0x7fc77c100570 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc77400f430 tx=0x7fc77400f740 comp rx=0 tx=0).stop 2024-05-09T00:01:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.577+0000 7ff8437fe700 1 -- 172.21.15.12:0/607307815 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff8440105a0 con 0x7ff8540ffdd0 2024-05-09T00:01:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.578+0000 7ff8437fe700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff83c0a1310 0x7ff83c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.578+0000 7fc781ce5700 1 -- 172.21.15.12:0/4004547380 shutdown_connections 2024-05-09T00:01:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.578+0000 7fc781ce5700 1 --2- 172.21.15.12:0/4004547380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc77c100ab0 0x7fc77c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.578+0000 7fc781ce5700 1 --2- 172.21.15.12:0/4004547380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c105f90 0x7fc77c106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.578+0000 7fc781ce5700 1 --2- 172.21.15.12:0/4004547380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc77c1000f0 0x7fc77c100570 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.578+0000 7fc781ce5700 1 -- 172.21.15.12:0/4004547380 >> 172.21.15.12:0/4004547380 conn(0x7fc77c076ba0 msgr2=0x7fc77c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.578+0000 7ff851d9b700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff83c0a1310 0x7ff83c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.578+0000 7ff8437fe700 1 -- 172.21.15.12:0/607307815 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff844014070 con 0x7ff8540ffdd0 2024-05-09T00:01:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.579+0000 7fc781ce5700 1 -- 172.21.15.12:0/4004547380 shutdown_connections 2024-05-09T00:01:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.579+0000 7fc781ce5700 1 -- 172.21.15.12:0/4004547380 wait complete. 2024-05-09T00:01:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.579+0000 7ff851d9b700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff83c0a1310 0x7ff83c0a37d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff84800b2e0 tx=0x7ff848020040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.579+0000 7ff864f6db80 1 Processor -- start 2024-05-09T00:01:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.579+0000 7ff864f6db80 1 -- start start 2024-05-09T00:01:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.580+0000 7fc781ce5700 1 Processor -- start 2024-05-09T00:01:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5095950 0x55bed5095d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5096270 0x55bed508ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5090480 0x55bed50908a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff864f6db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed4fe70f0 con 0x55bed5095950 2024-05-09T00:01:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff864f6db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed4fe4ab0 con 0x55bed5090480 2024-05-09T00:01:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff864f6db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed4fe03a0 con 0x55bed5096270 2024-05-09T00:01:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.580+0000 7fc781ce5700 1 -- start start 2024-05-09T00:01:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff852d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5090480 0x55bed50908a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff852d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5090480 0x55bed50908a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43592/0 (socket says 172.21.15.12:43592) 2024-05-09T00:01:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff852d9d700 1 -- 172.21.15.12:0/1629807236 learned_addr learned my addr 172.21.15.12:0/1629807236 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff851d9b700 1 --2- 172.21.15.12:0/1629807236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5096270 0x55bed508ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff85259c700 1 --2- 172.21.15.12:0/1629807236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5095950 0x55bed5095d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.580+0000 7fc781ce5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc77c1000f0 0x7fc77c195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.580+0000 7fc781ce5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc77c100ab0 0x7fc77c1957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.580+0000 7fc781ce5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c105f90 0x7fc77c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.580+0000 7fc781ce5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19a1a0 con 0x7fc77c1000f0 2024-05-09T00:01:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.580+0000 7fc781ce5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19a310 con 0x7fc77c100ab0 2024-05-09T00:01:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.580+0000 7fc781ce5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19a4d0 con 0x7fc77c105f90 2024-05-09T00:01:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff852d9d700 1 -- 172.21.15.12:0/1629807236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5096270 msgr2=0x55bed508ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff852d9d700 1 --2- 172.21.15.12:0/1629807236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5096270 0x55bed508ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff852d9d700 1 -- 172.21.15.12:0/1629807236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5095950 msgr2=0x55bed5095d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff852d9d700 1 --2- 172.21.15.12:0/1629807236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5095950 0x55bed5095d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.580+0000 7ff852d9d700 1 -- 172.21.15.12:0/1629807236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bed4e61350 con 0x55bed5090480 2024-05-09T00:01:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc77c1000f0 0x7fc77c195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c105f90 0x7fc77c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c105f90 0x7fc77c199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53664/0 (socket says 172.21.15.12:53664) 2024-05-09T00:01:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77bfff700 1 -- 172.21.15.12:0/521650190 learned_addr learned my addr 172.21.15.12:0/521650190 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc77c1000f0 0x7fc77c195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60434/0 (socket says 172.21.15.12:60434) 2024-05-09T00:01:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77affd700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc77c100ab0 0x7fc77c1957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.581+0000 7ff852d9d700 1 --2- 172.21.15.12:0/1629807236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5090480 0x55bed50908a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff84c003e00 tx=0x7ff84c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=414ee1b5be4e688d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77bfff700 1 -- 172.21.15.12:0/521650190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc77c100ab0 msgr2=0x7fc77c1957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77bfff700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc77c100ab0 0x7fc77c1957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77bfff700 1 -- 172.21.15.12:0/521650190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc77c1000f0 msgr2=0x7fc77c195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77bfff700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc77c1000f0 0x7fc77c195280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77bfff700 1 -- 172.21.15.12:0/521650190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc770009520 con 0x7fc77c105f90 2024-05-09T00:01:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.581+0000 7ff840ff9700 1 -- 172.21.15.12:0/1629807236 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0092d0 con 0x55bed5090480 2024-05-09T00:01:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.581+0000 7ff840ff9700 1 -- 172.21.15.12:0/1629807236 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff84c00f040 con 0x55bed5090480 2024-05-09T00:01:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.581+0000 7ff840ff9700 1 -- 172.21.15.12:0/1629807236 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c007900 con 0x55bed5090480 2024-05-09T00:01:19.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc77bfff700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c105f90 0x7fc77c199b60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc77000ea80 tx=0x7fc77000ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc778ff9700 1 -- 172.21.15.12:0/521650190 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700098a0 con 0x7fc77c105f90 2024-05-09T00:01:19.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc778ff9700 1 -- 172.21.15.12:0/521650190 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc77000ce10 con 0x7fc77c105f90 2024-05-09T00:01:19.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc778ff9700 1 -- 172.21.15.12:0/521650190 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770003d00 con 0x7fc77c105f90 2024-05-09T00:01:19.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.581+0000 7fc781ce5700 1 -- 172.21.15.12:0/521650190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77400f0b0 con 0x7fc77c105f90 2024-05-09T00:01:19.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.582+0000 7fc781ce5700 1 -- 172.21.15.12:0/521650190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c1a1e10 con 0x7fc77c105f90 2024-05-09T00:01:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.582+0000 7ff864f6db80 1 -- 172.21.15.12:0/1629807236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5090480 msgr2=0x55bed50908a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.582+0000 7ff864f6db80 1 --2- 172.21.15.12:0/1629807236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5090480 0x55bed50908a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff84c003e00 tx=0x7ff84c009920 comp rx=0 tx=0).stop 2024-05-09T00:01:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.583+0000 7ff864f6db80 1 -- 172.21.15.12:0/1629807236 shutdown_connections 2024-05-09T00:01:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.583+0000 7ff864f6db80 1 --2- 172.21.15.12:0/1629807236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5090480 0x55bed50908a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.583+0000 7ff864f6db80 1 --2- 172.21.15.12:0/1629807236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5095950 0x55bed5095d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.583+0000 7ff864f6db80 1 --2- 172.21.15.12:0/1629807236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5096270 0x55bed508ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.583+0000 7ff864f6db80 1 -- 172.21.15.12:0/1629807236 >> 172.21.15.12:0/1629807236 conn(0x55bed4fee1b0 msgr2=0x55bed508a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.584+0000 7fc778ff9700 1 -- 172.21.15.12:0/521650190 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc7700043b0 con 0x7fc77c105f90 2024-05-09T00:01:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.585+0000 7ff864f6db80 1 -- 172.21.15.12:0/1629807236 shutdown_connections 2024-05-09T00:01:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.585+0000 7ff864f6db80 1 -- 172.21.15.12:0/1629807236 wait complete. 2024-05-09T00:01:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.585+0000 7fc778ff9700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7640a1350 0x7fc7640a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.585+0000 7ff864f6db80 1 -- start start 2024-05-09T00:01:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.585+0000 7ff864f6db80 1 Processor -- start 2024-05-09T00:01:19.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.585+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7640a1350 0x7fc7640a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.585+0000 7fc778ff9700 1 -- 172.21.15.12:0/521650190 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc770014070 con 0x7fc77c105f90 2024-05-09T00:01:19.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc78de9fb80 1 Processor -- start 2024-05-09T00:01:19.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc78de9fb80 1 -- start start 2024-05-09T00:01:19.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7640a1350 0x7fc7640a3810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc77c067bb0 tx=0x7fc76c00f040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6dc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 0x55bed509c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed5099530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5096270 0x55bed5099c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff864f6db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed4f7c500 con 0x55bed5090480 2024-05-09T00:01:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff864f6db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed4f7c1a0 con 0x55bed5095950 2024-05-09T00:01:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff864f6db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed4e63640 con 0x55bed5096270 2024-05-09T00:01:19.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6d6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6d6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff864f6db80 10 client.0 _ll_register_callbacks cb 0x55bed4f85ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff851d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed5099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff851d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed5099530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43616/0 (socket says 172.21.15.12:43616) 2024-05-09T00:01:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff851d9b700 1 -- 172.21.15.12:0/462875440 learned_addr learned my addr 172.21.15.12:0/462875440 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c62d150 con 0x55d18c6dc720 2024-05-09T00:01:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c62ab10 con 0x55d18c6d6890 2024-05-09T00:01:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c626400 con 0x55d18c6dbe00 2024-05-09T00:01:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc77b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6dc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc77bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6d6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc77b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6dc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53672/0 (socket says 172.21.15.12:53672) 2024-05-09T00:01:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc77bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6d6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43620/0 (socket says 172.21.15.12:43620) 2024-05-09T00:01:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff852d9d700 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5096270 0x55bed5099c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff85259c700 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 0x55bed509c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc77b7fe700 1 -- 172.21.15.12:0/648007171 learned_addr learned my addr 172.21.15.12:0/648007171 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.586+0000 7fc77affd700 1 --2- 172.21.15.12:0/648007171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6d6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff851d9b700 1 -- 172.21.15.12:0/462875440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5096270 msgr2=0x55bed5099c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff851d9b700 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5096270 0x55bed5099c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff851d9b700 1 -- 172.21.15.12:0/462875440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 msgr2=0x55bed509c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff851d9b700 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 0x55bed509c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.586+0000 7ff851d9b700 1 -- 172.21.15.12:0/462875440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84c003ae0 con 0x55bed5095950 2024-05-09T00:01:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.587+0000 7fc77b7fe700 1 -- 172.21.15.12:0/648007171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 msgr2=0x55d18c6d6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.587+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/648007171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6d6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.587+0000 7fc77b7fe700 1 -- 172.21.15.12:0/648007171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 msgr2=0x55d18c6d6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.587+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/648007171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6d6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.587+0000 7fc77b7fe700 1 -- 172.21.15.12:0/648007171 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d18c4a73b0 con 0x55d18c6dbe00 2024-05-09T00:01:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.587+0000 7fc77bfff700 1 --2- 172.21.15.12:0/648007171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6d6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.587+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/648007171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6dc1e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0092e0 tx=0x7fc76c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a53ecd4ce7433b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.587+0000 7ff851d9b700 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed5099530 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff848009990 tx=0x7ff848009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55936b772c9c956c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.587+0000 7ff839ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:01:19.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.587+0000 7fc761ffb700 1 -- 172.21.15.12:0/648007171 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c004220 con 0x55d18c6dbe00 2024-05-09T00:01:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.587+0000 7ff839ffb700 1 -- 172.21.15.12:0/462875440 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84801d040 con 0x55bed5095950 2024-05-09T00:01:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.587+0000 7ff839ffb700 1 -- 172.21.15.12:0/462875440 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff848041070 con 0x55bed5095950 2024-05-09T00:01:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.587+0000 7ff864f6db80 1 -- 172.21.15.12:0/462875440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bed4f39210 con 0x55bed5095950 2024-05-09T00:01:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.587+0000 7ff839ffb700 1 -- 172.21.15.12:0/462875440 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff848023040 con 0x55bed5095950 2024-05-09T00:01:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.587+0000 7ff864f6db80 10 client.29348 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.587+0000 7ff864f6db80 1 -- 172.21.15.12:0/462875440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55bed4e0db00 con 0x55bed5095950 2024-05-09T00:01:19.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.588+0000 7fc761ffb700 1 -- 172.21.15.12:0/648007171 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc76c013040 con 0x55d18c6dbe00 2024-05-09T00:01:19.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.588+0000 7fc761ffb700 1 -- 172.21.15.12:0/648007171 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c003b00 con 0x55d18c6dbe00 2024-05-09T00:01:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.588+0000 7ff839ffb700 1 -- 172.21.15.12:0/462875440 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bed4f39210 con 0x55bed5095950 2024-05-09T00:01:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.588+0000 7ff839ffb700 1 -- 172.21.15.12:0/462875440 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff8480293f0 con 0x55bed5095950 2024-05-09T00:01:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.588+0000 7ff864f6db80 10 client.29348 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.588+0000 7ff864f6db80 10 client.29348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.588+0000 7ff864f6db80 20 client.29348 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.588+0000 7ff864f6db80 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bed50a84f0 0x55bed50aa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.588+0000 7ff864f6db80 4 client.29348 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.588+0000 7ff864f6db80 1 -- 172.21.15.12:0/462875440 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bed505f7b0 con 0x55bed50a84f0 2024-05-09T00:01:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.588+0000 7ff85259c700 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bed50a84f0 0x55bed50aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.588+0000 7fc78de9fb80 1 -- 172.21.15.12:0/648007171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 msgr2=0x55d18c6dc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.588+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/648007171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6dc1e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0092e0 tx=0x7fc76c015000 comp rx=0 tx=0).stop 2024-05-09T00:01:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.589+0000 7fc78de9fb80 1 -- 172.21.15.12:0/648007171 shutdown_connections 2024-05-09T00:01:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.589+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/648007171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6d6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.589+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/648007171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6dc1e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.589+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/648007171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6d6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.589+0000 7fc78de9fb80 1 -- 172.21.15.12:0/648007171 >> 172.21.15.12:0/648007171 conn(0x55d18c634210 msgr2=0x55d18c6d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.589+0000 7fc78de9fb80 1 -- 172.21.15.12:0/648007171 shutdown_connections 2024-05-09T00:01:19.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.589+0000 7fc78de9fb80 1 -- 172.21.15.12:0/648007171 wait complete. 2024-05-09T00:01:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.589+0000 7ff85259c700 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bed50a84f0 0x55bed50aa910 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed19fb83d53b0580 server_cookie=fd811161d338027f in_seq=0 out_seq=0 2024-05-09T00:01:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.589+0000 7ff839ffb700 10 client.29348 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:19.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.589+0000 7fc78de9fb80 1 -- start start 2024-05-09T00:01:19.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.589+0000 7fc78de9fb80 1 Processor -- start 2024-05-09T00:01:19.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6d6890 0x55d18c6dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dbe00 0x55d18c6e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c5c2560 con 0x55d18c6dc720 2024-05-09T00:01:19.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c5c2200 con 0x55d18c6dbe00 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c4a96a0 con 0x55d18c6d6890 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc78de9fb80 10 client.0 _ll_register_callbacks cb 0x55d18c5cbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc77b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6d6890 0x55d18c6dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc77b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6d6890 0x55d18c6dfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53690/0 (socket says 172.21.15.12:53690) 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc77b7fe700 1 -- 172.21.15.12:0/3162259167 learned_addr learned my addr 172.21.15.12:0/3162259167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc77affd700 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dbe00 0x55d18c6e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc77bfff700 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc77b7fe700 1 -- 172.21.15.12:0/3162259167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dbe00 msgr2=0x55d18c6e00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dbe00 0x55d18c6e00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc77b7fe700 1 -- 172.21.15.12:0/3162259167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 msgr2=0x55d18c6e05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6e05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.590+0000 7fc77b7fe700 1 -- 172.21.15.12:0/3162259167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c1a1e10 con 0x55d18c6d6890 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.591+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6d6890 0x55d18c6dfb60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009350 tx=0x7fc76c019f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb08be99aa13059f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.591+0000 7fc75affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.591+0000 7fc75affd700 1 -- 172.21.15.12:0/3162259167 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c005460 con 0x55d18c6d6890 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.591+0000 7fc75affd700 1 -- 172.21.15.12:0/3162259167 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc76c0055c0 con 0x55d18c6d6890 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.591+0000 7fc78de9fb80 1 -- 172.21.15.12:0/3162259167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d18c57f270 con 0x55d18c6d6890 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.591+0000 7fc75affd700 1 -- 172.21.15.12:0/3162259167 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc76c0046f0 con 0x55d18c6d6890 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.591+0000 7fc78de9fb80 10 client.29377 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.591+0000 7fc78de9fb80 1 -- 172.21.15.12:0/3162259167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7fc76c00d990 con 0x55d18c6d6890 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.591+0000 7fc75affd700 1 -- 172.21.15.12:0/3162259167 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d18c57f270 con 0x55d18c6d6890 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.592+0000 7fc75affd700 1 -- 172.21.15.12:0/3162259167 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc76c02c040 con 0x55d18c6d6890 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.592+0000 7fc78de9fb80 10 client.29377 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.592+0000 7fc78de9fb80 10 client.29377 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:01:19.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.592+0000 7fc78de9fb80 20 client.29377 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:19.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.592+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d18c6f0200 0x55d18c6f2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.592+0000 7fc78de9fb80 4 client.29377 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:19.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.592+0000 7fc78de9fb80 1 -- 172.21.15.12:0/3162259167 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d18c6a5810 con 0x55d18c6f0200 2024-05-09T00:01:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.592+0000 7ff839ffb700 1 -- 172.21.15.12:0/462875440 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bed505f7b0 con 0x55bed50a84f0 2024-05-09T00:01:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.592+0000 7ff839ffb700 10 client.29348 handle_command_reply: tid=0 2024-05-09T00:01:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.592+0000 7ff864f6db80 1 client.29348 shutdown 2024-05-09T00:01:19.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.592+0000 7fc77affd700 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d18c6f0200 0x55d18c6f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.592+0000 7fc75affd700 10 client.29377 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:01:19.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.593+0000 7fc77affd700 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d18c6f0200 0x55d18c6f2620 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb45112199be792d server_cookie=f06c47fd68275087 in_seq=0 out_seq=0 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 1 -- 172.21.15.12:0/462875440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 msgr2=0x55bed5099530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed5099530 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff848009990 tx=0x7ff848009ca0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 1 -- 172.21.15.12:0/462875440 shutdown_connections 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5096270 0x55bed5099c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 0x55bed509c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bed50a84f0 0x55bed50aa910 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 1 --2- 172.21.15.12:0/462875440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed5099530 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff839ffb700 0 client.29348 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 1 -- 172.21.15.12:0/462875440 >> 172.21.15.12:0/462875440 conn(0x55bed4fee1b0 msgr2=0x55bed50944a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 1 -- 172.21.15.12:0/462875440 shutdown_connections 2024-05-09T00:01:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 1 -- 172.21.15.12:0/462875440 wait complete. 2024-05-09T00:01:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.593+0000 7ff864f6db80 20 client.29348 trim_cache size 0 max 16384 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.595+0000 7fc75affd700 1 -- 172.21.15.12:0/3162259167 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d18c6a5810 con 0x55d18c6f0200 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.595+0000 7fc75affd700 10 client.29377 handle_command_reply: tid=0 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.595+0000 7fc78de9fb80 1 client.29377 shutdown 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 1 -- 172.21.15.12:0/3162259167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6d6890 msgr2=0x55d18c6dfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6d6890 0x55d18c6dfb60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009350 tx=0x7fc76c019f60 comp rx=0 tx=0).stop 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 1 -- 172.21.15.12:0/3162259167 shutdown_connections 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6e05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d18c6f0200 0x55d18c6f2620 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6d6890 0x55d18c6dfb60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/3162259167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dbe00 0x55d18c6e00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc75affd700 0 client.29377 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 1 -- 172.21.15.12:0/3162259167 >> 172.21.15.12:0/3162259167 conn(0x55d18c634210 msgr2=0x55d18c6d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 1 -- 172.21.15.12:0/3162259167 shutdown_connections 2024-05-09T00:01:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 1 -- 172.21.15.12:0/3162259167 wait complete. 2024-05-09T00:01:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.596+0000 7fc78de9fb80 20 client.29377 trim_cache size 0 max 16384 2024-05-09T00:01:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.614+0000 7ff864f6db80 1 Processor -- start 2024-05-09T00:01:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.614+0000 7ff864f6db80 1 -- start start 2024-05-09T00:01:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.614+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 0x55bed508a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.614+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5095950 0x55bed508fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.614+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5096270 0x55bed508c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.614+0000 7ff864f6db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed508c9f0 con 0x55bed5090480 2024-05-09T00:01:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.614+0000 7ff864f6db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed509a250 con 0x55bed5096270 2024-05-09T00:01:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.614+0000 7ff864f6db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed508f110 con 0x55bed5095950 2024-05-09T00:01:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff85259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 0x55bed508a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff852d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5096270 0x55bed508c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff85259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 0x55bed508a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60472/0 (socket says 172.21.15.12:60472) 2024-05-09T00:01:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff85259c700 1 -- 172.21.15.12:0/4084460337 learned_addr learned my addr 172.21.15.12:0/4084460337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff851d9b700 1 --2- 172.21.15.12:0/4084460337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5095950 0x55bed508fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff852d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5096270 0x55bed508c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43642/0 (socket says 172.21.15.12:43642) 2024-05-09T00:01:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff85259c700 1 -- 172.21.15.12:0/4084460337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5095950 msgr2=0x55bed508fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff85259c700 1 --2- 172.21.15.12:0/4084460337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5095950 0x55bed508fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff85259c700 1 -- 172.21.15.12:0/4084460337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5096270 msgr2=0x55bed508c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff85259c700 1 --2- 172.21.15.12:0/4084460337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5096270 0x55bed508c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.615+0000 7ff85259c700 1 -- 172.21.15.12:0/4084460337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bed4fa6500 con 0x55bed5090480 2024-05-09T00:01:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc78de9fb80 1 Processor -- start 2024-05-09T00:01:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc78de9fb80 1 -- start start 2024-05-09T00:01:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6ebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c6d1e80 con 0x55d18c6dc720 2024-05-09T00:01:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c6cb800 con 0x55d18c6d6890 2024-05-09T00:01:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c6cddc0 con 0x55d18c6dbe00 2024-05-09T00:01:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6ebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43648/0 (socket says 172.21.15.12:43648) 2024-05-09T00:01:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77b7fe700 1 -- 172.21.15.12:0/1455926316 learned_addr learned my addr 172.21.15.12:0/1455926316 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77affd700 1 --2- 172.21.15.12:0/1455926316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6ebbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53706/0 (socket says 172.21.15.12:53706) 2024-05-09T00:01:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77bfff700 1 --2- 172.21.15.12:0/1455926316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.616+0000 7ff85259c700 1 --2- 172.21.15.12:0/4084460337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 0x55bed508a910 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff844099c40 tx=0x7ff844010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa2fbe74c07192e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.616+0000 7ff8417fa700 1 -- 172.21.15.12:0/4084460337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8440b1070 con 0x55bed5090480 2024-05-09T00:01:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77b7fe700 1 -- 172.21.15.12:0/1455926316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 msgr2=0x55d18c6ebbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/1455926316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6ebbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77b7fe700 1 -- 172.21.15.12:0/1455926316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 msgr2=0x55d18c6d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/1455926316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.616+0000 7fc77b7fe700 1 -- 172.21.15.12:0/1455926316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d18c453b60 con 0x55d18c6d6890 2024-05-09T00:01:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.617+0000 7fc77affd700 1 --2- 172.21.15.12:0/1455926316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6ebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.617+0000 7ff8417fa700 1 -- 172.21.15.12:0/4084460337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff844098080 con 0x55bed5090480 2024-05-09T00:01:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.617+0000 7ff8417fa700 1 -- 172.21.15.12:0/4084460337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844099430 con 0x55bed5090480 2024-05-09T00:01:19.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.617+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/1455926316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6f25f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc76c000c00 tx=0x7fc76c02fee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dfe6150b756aff6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.617+0000 7fc7627fc700 1 -- 172.21.15.12:0/1455926316 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c02e660 con 0x55d18c6d6890 2024-05-09T00:01:19.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.617+0000 7fc7627fc700 1 -- 172.21.15.12:0/1455926316 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc76c0196f0 con 0x55d18c6d6890 2024-05-09T00:01:19.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.617+0000 7fc7627fc700 1 -- 172.21.15.12:0/1455926316 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c004bf0 con 0x55d18c6d6890 2024-05-09T00:01:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.617+0000 7ff864f6db80 1 -- 172.21.15.12:0/4084460337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 msgr2=0x55bed508a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.617+0000 7ff864f6db80 1 --2- 172.21.15.12:0/4084460337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 0x55bed508a910 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff844099c40 tx=0x7ff844010b80 comp rx=0 tx=0).stop 2024-05-09T00:01:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.618+0000 7ff864f6db80 1 -- 172.21.15.12:0/4084460337 shutdown_connections 2024-05-09T00:01:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.618+0000 7ff864f6db80 1 --2- 172.21.15.12:0/4084460337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5096270 0x55bed508c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.618+0000 7ff864f6db80 1 --2- 172.21.15.12:0/4084460337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5090480 0x55bed508a910 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.618+0000 7ff864f6db80 1 --2- 172.21.15.12:0/4084460337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5095950 0x55bed508fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.618+0000 7ff864f6db80 1 -- 172.21.15.12:0/4084460337 >> 172.21.15.12:0/4084460337 conn(0x55bed4fee1b0 msgr2=0x55bed5098dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.618+0000 7ff864f6db80 1 -- 172.21.15.12:0/4084460337 shutdown_connections 2024-05-09T00:01:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.618+0000 7ff864f6db80 1 -- 172.21.15.12:0/4084460337 wait complete. 2024-05-09T00:01:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.618+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1455926316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 msgr2=0x55d18c6f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.618+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/1455926316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6f25f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc76c000c00 tx=0x7fc76c02fee0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.618+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1455926316 shutdown_connections 2024-05-09T00:01:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.618+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/1455926316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6dc720 0x55d18c6d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.618+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/1455926316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6d6890 0x55d18c6f25f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.618+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/1455926316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6ebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.618+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1455926316 >> 172.21.15.12:0/1455926316 conn(0x55d18c634210 msgr2=0x55d18c6d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.618+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1455926316 shutdown_connections 2024-05-09T00:01:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.618+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1455926316 wait complete. 2024-05-09T00:01:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.618+0000 7ff864f6db80 1 -- start start 2024-05-09T00:01:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.618+0000 7ff864f6db80 1 Processor -- start 2024-05-09T00:01:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.618+0000 7fc78de9fb80 1 -- start start 2024-05-09T00:01:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5090480 0x55bed50b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed50b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc78de9fb80 1 Processor -- start 2024-05-09T00:01:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff864f6db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5096270 0x55bed509b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff864f6db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed5095120 con 0x55bed5096270 2024-05-09T00:01:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff864f6db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed509bc00 con 0x55bed5095950 2024-05-09T00:01:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff864f6db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed509bdc0 con 0x55bed5090480 2024-05-09T00:01:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff864f6db80 10 client.0 _ll_register_callbacks cb 0x55bed4f85ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6d6890 0x55d18c6e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6e7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc78de9fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dc720 0x55d18c6e50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c6e5790 con 0x55d18c6d6890 2024-05-09T00:01:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c6e5900 con 0x55d18c6dc720 2024-05-09T00:01:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc78de9fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18c6e5ac0 con 0x55d18c6dbe00 2024-05-09T00:01:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc78de9fb80 10 client.0 _ll_register_callbacks cb 0x55d18c5cbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff851d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed50b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff851d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed50b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43652/0 (socket says 172.21.15.12:43652) 2024-05-09T00:01:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff851d9b700 1 -- 172.21.15.12:0/3304530453 learned_addr learned my addr 172.21.15.12:0/3304530453 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc77affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc77affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6e7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53720/0 (socket says 172.21.15.12:53720) 2024-05-09T00:01:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc77affd700 1 -- 172.21.15.12:0/1153960089 learned_addr learned my addr 172.21.15.12:0/1153960089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.619+0000 7ff85259c700 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5090480 0x55bed50b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.619+0000 7fc77bfff700 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dc720 0x55d18c6e50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.620+0000 7ff851d9b700 1 -- 172.21.15.12:0/3304530453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5090480 msgr2=0x55bed50b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.620+0000 7ff851d9b700 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5090480 0x55bed50b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.620+0000 7ff851d9b700 1 -- 172.21.15.12:0/3304530453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5096270 msgr2=0x55bed509b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.620+0000 7ff851d9b700 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5096270 0x55bed509b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.620+0000 7ff851d9b700 1 -- 172.21.15.12:0/3304530453 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff854196400 con 0x55bed5095950 2024-05-09T00:01:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc77affd700 1 -- 172.21.15.12:0/1153960089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dc720 msgr2=0x55d18c6e50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc77affd700 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dc720 0x55d18c6e50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc77affd700 1 -- 172.21.15.12:0/1153960089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6d6890 msgr2=0x55d18c6e70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc77affd700 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6d6890 0x55d18c6e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc77affd700 1 -- 172.21.15.12:0/1153960089 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c10d550 con 0x55d18c6dbe00 2024-05-09T00:01:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc77bfff700 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dc720 0x55d18c6e50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:01:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc77affd700 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6e7620 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc77400dd70 tx=0x7fc774006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1d9a63fa90046a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc75affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.620+0000 7ff851d9b700 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed50b4000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55bed509aad0 tx=0x7ff848029970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c690f947959a8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.620+0000 7ff839ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:01:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc75affd700 1 -- 172.21.15.12:0/1153960089 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774026040 con 0x55d18c6dbe00 2024-05-09T00:01:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc75affd700 1 -- 172.21.15.12:0/1153960089 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc774003e10 con 0x55d18c6dbe00 2024-05-09T00:01:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1153960089 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc76c013040 con 0x55d18c6dbe00 2024-05-09T00:01:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.620+0000 7ff839ffb700 1 -- 172.21.15.12:0/3304530453 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84801d070 con 0x55bed5095950 2024-05-09T00:01:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.620+0000 7ff839ffb700 1 -- 172.21.15.12:0/3304530453 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff848041040 con 0x55bed5095950 2024-05-09T00:01:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.621+0000 7ff864f6db80 1 -- 172.21.15.12:0/3304530453 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff84c00f040 con 0x55bed5095950 2024-05-09T00:01:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.620+0000 7fc75affd700 1 -- 172.21.15.12:0/1153960089 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc774012040 con 0x55d18c6dbe00 2024-05-09T00:01:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc78de9fb80 10 client.29386 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1153960089 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7fc76c00d990 con 0x55d18c6dbe00 2024-05-09T00:01:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.621+0000 7ff839ffb700 1 -- 172.21.15.12:0/3304530453 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff848023040 con 0x55bed5095950 2024-05-09T00:01:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.621+0000 7ff864f6db80 10 client.29363 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.621+0000 7ff864f6db80 1 -- 172.21.15.12:0/3304530453 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55bed4e65c70 con 0x55bed5095950 2024-05-09T00:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc75affd700 1 -- 172.21.15.12:0/1153960089 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc774037020 con 0x55d18c6dbe00 2024-05-09T00:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc75affd700 1 -- 172.21.15.12:0/1153960089 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc7740058e0 con 0x55d18c6dbe00 2024-05-09T00:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc78de9fb80 10 client.29386 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc78de9fb80 10 client.29386 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc78de9fb80 20 client.29386 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d18c7469e0 0x55d18c748e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc78de9fb80 4 client.29386 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1153960089 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d18c61eec0 con 0x55d18c7469e0 2024-05-09T00:01:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.621+0000 7ff839ffb700 1 -- 172.21.15.12:0/3304530453 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff84c00f040 con 0x55bed5095950 2024-05-09T00:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.621+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d18c7469e0 0x55d18c748e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.621+0000 7ff839ffb700 1 -- 172.21.15.12:0/3304530453 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff84804f590 con 0x55bed5095950 2024-05-09T00:01:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.621+0000 7ff864f6db80 10 client.29363 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.621+0000 7ff864f6db80 10 client.29363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.621+0000 7ff864f6db80 20 client.29363 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.622+0000 7ff864f6db80 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bed5100ea0 0x55bed51032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.622+0000 7ff864f6db80 4 client.29363 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:01:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.622+0000 7ff864f6db80 1 -- 172.21.15.12:0/3304530453 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bed4fd8e60 con 0x55bed5100ea0 2024-05-09T00:01:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.622+0000 7fc75affd700 10 client.29386 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:01:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.622+0000 7fc77b7fe700 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d18c7469e0 0x55d18c748e00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f7ac34d0b6e95da server_cookie=9bfa4401a7271a1e in_seq=0 out_seq=0 2024-05-09T00:01:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.622+0000 7ff85259c700 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bed5100ea0 0x55bed51032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.622+0000 7ff839ffb700 10 client.29363 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.622+0000 7ff85259c700 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bed5100ea0 0x55bed51032c0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf05cb092e955e61 server_cookie=1aa25ca3c8571061 in_seq=0 out_seq=0 2024-05-09T00:01:19.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc75affd700 1 -- 172.21.15.12:0/1153960089 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d18c61eec0 con 0x55d18c7469e0 2024-05-09T00:01:19.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc75affd700 10 client.29386 handle_command_reply: tid=0 2024-05-09T00:01:19.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 client.29386 shutdown 2024-05-09T00:01:19.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1153960089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 msgr2=0x55d18c6e7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6e7620 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc77400dd70 tx=0x7fc774006000 comp rx=0 tx=0).stop 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1153960089 shutdown_connections 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d18c6dc720 0x55d18c6e50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d18c6d6890 0x55d18c6e70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55d18c7469e0 0x55d18c748e00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 --2- 172.21.15.12:0/1153960089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d18c6dbe00 0x55d18c6e7620 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc75affd700 0 client.29386 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1153960089 >> 172.21.15.12:0/1153960089 conn(0x55d18c634210 msgr2=0x55d18c6de340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1153960089 shutdown_connections 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.623+0000 7fc78de9fb80 1 -- 172.21.15.12:0/1153960089 wait complete. 2024-05-09T00:01:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc78de9fb80 20 client.29386 trim_cache size 0 max 16384 2024-05-09T00:01:19.636 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:01:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 -- 172.21.15.12:0/521650190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7640a1350 msgr2=0x7fc7640a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7640a1350 0x7fc7640a3810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc77c067bb0 tx=0x7fc76c00f040 comp rx=0 tx=0).stop 2024-05-09T00:01:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 -- 172.21.15.12:0/521650190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c105f90 msgr2=0x7fc77c199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c105f90 0x7fc77c199b60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc77000ea80 tx=0x7fc77000ed90 comp rx=0 tx=0).stop 2024-05-09T00:01:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 -- 172.21.15.12:0/521650190 shutdown_connections 2024-05-09T00:01:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7640a1350 0x7fc7640a3810 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c105f90 0x7fc77c199b60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc77c1000f0 0x7fc77c195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 --2- 172.21.15.12:0/521650190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc77c100ab0 0x7fc77c1957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 -- 172.21.15.12:0/521650190 >> 172.21.15.12:0/521650190 conn(0x7fc77c076ba0 msgr2=0x7fc77c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 -- 172.21.15.12:0/521650190 shutdown_connections 2024-05-09T00:01:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.624+0000 7fc781ce5700 1 -- 172.21.15.12:0/521650190 wait complete. 2024-05-09T00:01:19.638 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:01:19.663 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:19 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2834101544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:01:19.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:19 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2834101544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:01:19.770 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:19 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2834101544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:01:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.852+0000 7f6b0adac700 1 -- 172.21.15.12:0/766991053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b041052b0 msgr2=0x7f6b04105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.852+0000 7f6b0adac700 1 --2- 172.21.15.12:0/766991053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b041052b0 0x7f6b04105c40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009990 tx=0x7f6af4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.854+0000 7f6b0adac700 1 -- 172.21.15.12:0/766991053 shutdown_connections 2024-05-09T00:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.854+0000 7f6b0adac700 1 --2- 172.21.15.12:0/766991053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b041052b0 0x7f6b04105c40 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.854+0000 7f6b0adac700 1 --2- 172.21.15.12:0/766991053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b040ff630 0x7f6b040ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.854+0000 7f6b0adac700 1 --2- 172.21.15.12:0/766991053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b040fffe0 0x7f6b04104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.854+0000 7f6b0adac700 1 -- 172.21.15.12:0/766991053 >> 172.21.15.12:0/766991053 conn(0x7f6b04074d40 msgr2=0x7f6b04075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.856+0000 7f6b0adac700 1 -- 172.21.15.12:0/766991053 shutdown_connections 2024-05-09T00:01:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.858+0000 7f6b0adac700 1 -- 172.21.15.12:0/766991053 wait complete. 2024-05-09T00:01:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.859+0000 7f6b0adac700 1 Processor -- start 2024-05-09T00:01:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b0adac700 1 -- start start 2024-05-09T00:01:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b0adac700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b040ff630 0x7f6b04115010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b0adac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b041052b0 0x7f6b04115590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b0adac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b0410e180 0x7f6b0410e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b0adac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0410edd0 con 0x7f6b0410e180 2024-05-09T00:01:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b0adac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0410ef40 con 0x7f6b041052b0 2024-05-09T00:01:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b0adac700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04112bd0 con 0x7f6b040ff630 2024-05-09T00:01:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b03fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b041052b0 0x7f6b04115590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b03fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b041052b0 0x7f6b04115590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43670/0 (socket says 172.21.15.12:43670) 2024-05-09T00:01:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b03fff700 1 -- 172.21.15.12:0/2505511243 learned_addr learned my addr 172.21.15.12:0/2505511243 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.860+0000 7f6b09349700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b0410e180 0x7f6b0410e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.861+0000 7f6b03fff700 1 -- 172.21.15.12:0/2505511243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b040ff630 msgr2=0x7f6b04115010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:01:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.861+0000 7f6b03fff700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b040ff630 0x7f6b04115010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.861+0000 7f6b03fff700 1 -- 172.21.15.12:0/2505511243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b0410e180 msgr2=0x7f6b0410e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.861+0000 7f6b03fff700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b0410e180 0x7f6b0410e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.861+0000 7f6b03fff700 1 -- 172.21.15.12:0/2505511243 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af40095f0 con 0x7f6b041052b0 2024-05-09T00:01:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.861+0000 7f6b03fff700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b041052b0 0x7f6b04115590 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00b9e0 tx=0x7f6afc00bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.862+0000 7f6b01ffb700 1 -- 172.21.15.12:0/2505511243 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00c9b0 con 0x7f6b041052b0 2024-05-09T00:01:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.862+0000 7f6b0adac700 1 -- 172.21.15.12:0/2505511243 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b04112eb0 con 0x7f6b041052b0 2024-05-09T00:01:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.862+0000 7f6b01ffb700 1 -- 172.21.15.12:0/2505511243 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6afc014440 con 0x7f6b041052b0 2024-05-09T00:01:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.862+0000 7f6b01ffb700 1 -- 172.21.15.12:0/2505511243 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0125a0 con 0x7f6b041052b0 2024-05-09T00:01:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.862+0000 7f6b0adac700 1 -- 172.21.15.12:0/2505511243 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b04113410 con 0x7f6b041052b0 2024-05-09T00:01:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.865+0000 7f6b01ffb700 1 -- 172.21.15.12:0/2505511243 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6afc0145b0 con 0x7f6b041052b0 2024-05-09T00:01:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.866+0000 7f6b01ffb700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6aec0a1310 0x7f6aec0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.866+0000 7f6b08b48700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6aec0a1310 0x7f6aec0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.866+0000 7f6b01ffb700 1 -- 172.21.15.12:0/2505511243 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6afc0cc260 con 0x7f6b041052b0 2024-05-09T00:01:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.867+0000 7f6b08b48700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6aec0a1310 0x7f6aec0a37d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6af800ba10 tx=0x7f6af800b3f0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.867+0000 7f6b16f66b80 1 Processor -- start 2024-05-09T00:01:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.867+0000 7f6b16f66b80 1 -- start start 2024-05-09T00:01:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.867+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c6e00 0x55f6c92c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.867+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92c1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.867+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c1890 0x55f6c92c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.867+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c9218150 con 0x55f6c92c7720 2024-05-09T00:01:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.867+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c9215b10 con 0x55f6c92c1890 2024-05-09T00:01:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.867+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c9211400 con 0x55f6c92c6e00 2024-05-09T00:01:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b08b48700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c6e00 0x55f6c92c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b03fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92c1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b03fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92c1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60530/0 (socket says 172.21.15.12:60530) 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b08b48700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c6e00 0x55f6c92c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53742/0 (socket says 172.21.15.12:53742) 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b03fff700 1 -- 172.21.15.12:0/408704595 learned_addr learned my addr 172.21.15.12:0/408704595 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b09349700 1 --2- 172.21.15.12:0/408704595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c1890 0x55f6c92c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b03fff700 1 -- 172.21.15.12:0/408704595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c6e00 msgr2=0x55f6c92c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b03fff700 1 --2- 172.21.15.12:0/408704595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c6e00 0x55f6c92c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b03fff700 1 -- 172.21.15.12:0/408704595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c1890 msgr2=0x55f6c92c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b03fff700 1 --2- 172.21.15.12:0/408704595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c1890 0x55f6c92c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b03fff700 1 -- 172.21.15.12:0/408704595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6c90923b0 con 0x55f6c92c7720 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b08b48700 1 --2- 172.21.15.12:0/408704595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c6e00 0x55f6c92c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6b03fff700 1 --2- 172.21.15.12:0/408704595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92c1350 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00b9b0 tx=0x7f6afc0990a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9021f8038bbddae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.868+0000 7f6aeaffd700 1 -- 172.21.15.12:0/408704595 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0840e0 con 0x55f6c92c7720 2024-05-09T00:01:19.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.869+0000 7f6aeaffd700 1 -- 172.21.15.12:0/408704595 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6afc099430 con 0x55f6c92c7720 2024-05-09T00:01:19.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.869+0000 7f6aeaffd700 1 -- 172.21.15.12:0/408704595 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc098920 con 0x55f6c92c7720 2024-05-09T00:01:19.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.869+0000 7f6b16f66b80 1 -- 172.21.15.12:0/408704595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 msgr2=0x55f6c92c1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.869+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/408704595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92c1350 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00b9b0 tx=0x7f6afc0990a0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.869+0000 7f6b16f66b80 1 -- 172.21.15.12:0/408704595 shutdown_connections 2024-05-09T00:01:19.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.869+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/408704595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c1890 0x55f6c92c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.869+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/408704595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c6e00 0x55f6c92c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.869+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/408704595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92c1350 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.869+0000 7f6b16f66b80 1 -- 172.21.15.12:0/408704595 >> 172.21.15.12:0/408704595 conn(0x55f6c921f210 msgr2=0x55f6c92bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.870+0000 7f6b16f66b80 1 -- 172.21.15.12:0/408704595 shutdown_connections 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.870+0000 7f6b16f66b80 1 -- 172.21.15.12:0/408704595 wait complete. 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.870+0000 7f6b16f66b80 1 -- start start 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.870+0000 7f6b16f66b80 1 Processor -- start 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 0x55f6c92caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c92cb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c7720 0x55f6c92cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c91ad560 con 0x55f6c92c6e00 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c91ad200 con 0x55f6c92c7720 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c90946a0 con 0x55f6c92c1890 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b16f66b80 10 client.0 _ll_register_callbacks cb 0x55f6c91b6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b03fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c92cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b03fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c92cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60538/0 (socket says 172.21.15.12:60538) 2024-05-09T00:01:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b03fff700 1 -- 172.21.15.12:0/3075111218 learned_addr learned my addr 172.21.15.12:0/3075111218 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b08b48700 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 0x55f6c92caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b09349700 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c7720 0x55f6c92cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b09349700 1 -- 172.21.15.12:0/3075111218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 msgr2=0x55f6c92caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b09349700 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 0x55f6c92caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b09349700 1 -- 172.21.15.12:0/3075111218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 msgr2=0x55f6c92cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b09349700 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c92cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b09349700 1 -- 172.21.15.12:0/3075111218 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af800d7d0 con 0x55f6c92c7720 2024-05-09T00:01:19.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.871+0000 7f6b03fff700 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c92cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:19.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.872+0000 7f6b09349700 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c7720 0x55f6c92cb570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005b40 tx=0x7f6af4003790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7453109e21b1412 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.872+0000 7f6ae3fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:01:19.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.872+0000 7f6ae3fff700 1 -- 172.21.15.12:0/3075111218 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af401d070 con 0x55f6c92c7720 2024-05-09T00:01:19.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.872+0000 7f6ae3fff700 1 -- 172.21.15.12:0/3075111218 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6af40054d0 con 0x55f6c92c7720 2024-05-09T00:01:19.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.872+0000 7f6b16f66b80 1 -- 172.21.15.12:0/3075111218 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6c916a270 con 0x55f6c92c7720 2024-05-09T00:01:19.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.872+0000 7f6ae3fff700 1 -- 172.21.15.12:0/3075111218 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af4003860 con 0x55f6c92c7720 2024-05-09T00:01:19.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.872+0000 7f6b16f66b80 10 client.29378 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.872+0000 7f6b16f66b80 1 -- 172.21.15.12:0/3075111218 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f6b04113410 con 0x55f6c92c7720 2024-05-09T00:01:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.873+0000 7f6ae3fff700 1 -- 172.21.15.12:0/3075111218 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6c916a270 con 0x55f6c92c7720 2024-05-09T00:01:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.873+0000 7f6ae3fff700 1 -- 172.21.15.12:0/3075111218 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6af402b070 con 0x55f6c92c7720 2024-05-09T00:01:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.873+0000 7f6b16f66b80 10 client.29378 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.873+0000 7f6b16f66b80 10 client.29378 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:01:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.873+0000 7f6b16f66b80 20 client.29378 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.873+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f6c92db260 0x55f6c92dd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.873+0000 7f6b16f66b80 4 client.29378 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.873+0000 7f6b16f66b80 1 -- 172.21.15.12:0/3075111218 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6c9290810 con 0x55f6c92db260 2024-05-09T00:01:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.874+0000 7f6b08b48700 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f6c92db260 0x55f6c92dd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.874+0000 7f6b08b48700 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f6c92db260 0x55f6c92dd680 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7034c47f6fcfc6b2 server_cookie=7b129ef51abb59f4 in_seq=0 out_seq=0 2024-05-09T00:01:19.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.874+0000 7f6ae3fff700 10 client.29378 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:01:19.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6ae3fff700 1 -- 172.21.15.12:0/3075111218 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f6c9290810 con 0x55f6c92db260 2024-05-09T00:01:19.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6ae3fff700 10 client.29378 handle_command_reply: tid=0 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 client.29378 shutdown 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 -- 172.21.15.12:0/3075111218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c7720 msgr2=0x55f6c92cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c7720 0x55f6c92cb570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005b40 tx=0x7f6af4003790 comp rx=0 tx=0).stop 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 -- 172.21.15.12:0/3075111218 shutdown_connections 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f6c92db260 0x55f6c92dd680 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c7720 0x55f6c92cb570 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 0x55f6c92caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/3075111218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c92cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6ae3fff700 0 client.29378 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 -- 172.21.15.12:0/3075111218 >> 172.21.15.12:0/3075111218 conn(0x55f6c921f210 msgr2=0x55f6c92bb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 -- 172.21.15.12:0/3075111218 shutdown_connections 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 1 -- 172.21.15.12:0/3075111218 wait complete. 2024-05-09T00:01:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.877+0000 7f6b16f66b80 20 client.29378 trim_cache size 0 max 16384 2024-05-09T00:01:19.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.894+0000 7f6b16f66b80 1 Processor -- start 2024-05-09T00:01:19.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.894+0000 7f6b16f66b80 1 -- start start 2024-05-09T00:01:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.894+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 0x55f6c92c1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.894+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c6e00 0x55f6c92c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.894+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92bbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c92e2850 con 0x55f6c92c7720 2024-05-09T00:01:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c92b6800 con 0x55f6c92c6e00 2024-05-09T00:01:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c92ddce0 con 0x55f6c92c1890 2024-05-09T00:01:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b09349700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92bbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b08b48700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 0x55f6c92c1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b09349700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92bbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60552/0 (socket says 172.21.15.12:60552) 2024-05-09T00:01:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b09349700 1 -- 172.21.15.12:0/753321207 learned_addr learned my addr 172.21.15.12:0/753321207 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b03fff700 1 --2- 172.21.15.12:0/753321207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c6e00 0x55f6c92c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b09349700 1 -- 172.21.15.12:0/753321207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 msgr2=0x55f6c92c1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b09349700 1 --2- 172.21.15.12:0/753321207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 0x55f6c92c1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b09349700 1 -- 172.21.15.12:0/753321207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c6e00 msgr2=0x55f6c92c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b09349700 1 --2- 172.21.15.12:0/753321207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c6e00 0x55f6c92c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b09349700 1 -- 172.21.15.12:0/753321207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6c92c1f40 con 0x55f6c92c7720 2024-05-09T00:01:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b08b48700 1 --2- 172.21.15.12:0/753321207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 0x55f6c92c1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.895+0000 7f6b03fff700 1 --2- 172.21.15.12:0/753321207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c6e00 0x55f6c92c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:01:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.896+0000 7f6b09349700 1 --2- 172.21.15.12:0/753321207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92bbd20 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6af40098e0 tx=0x7f6af4005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3049322869d3ee09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.896+0000 7f6aeb7fe700 1 -- 172.21.15.12:0/753321207 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af401d070 con 0x55f6c92c7720 2024-05-09T00:01:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.896+0000 7f6aeb7fe700 1 -- 172.21.15.12:0/753321207 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af401e040 con 0x55f6c92c7720 2024-05-09T00:01:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.896+0000 7f6aeb7fe700 1 -- 172.21.15.12:0/753321207 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4003ac0 con 0x55f6c92c7720 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.897+0000 7f6b16f66b80 1 -- 172.21.15.12:0/753321207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 msgr2=0x55f6c92bbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.897+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/753321207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92bbd20 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6af40098e0 tx=0x7f6af4005da0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.897+0000 7f6b16f66b80 1 -- 172.21.15.12:0/753321207 shutdown_connections 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.897+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/753321207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c7720 0x55f6c92bbd20 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.897+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/753321207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c1890 0x55f6c92c1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.897+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/753321207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c6e00 0x55f6c92c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.897+0000 7f6b16f66b80 1 -- 172.21.15.12:0/753321207 >> 172.21.15.12:0/753321207 conn(0x55f6c921f210 msgr2=0x55f6c92c5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.898+0000 7f6b16f66b80 1 -- 172.21.15.12:0/753321207 shutdown_connections 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.899+0000 7f6b16f66b80 1 -- 172.21.15.12:0/753321207 wait complete. 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.899+0000 7f6b16f66b80 1 -- start start 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.899+0000 7f6b16f66b80 1 Processor -- start 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c1890 0x55f6c9329b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c932a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b16f66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c7720 0x55f6c93256b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c9325d90 con 0x55f6c92c6e00 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c9325f00 con 0x55f6c92c1890 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b16f66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c93260c0 con 0x55f6c92c7720 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b16f66b80 10 client.0 _ll_register_callbacks cb 0x55f6c91b6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b09349700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c7720 0x55f6c93256b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b09349700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c7720 0x55f6c93256b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53760/0 (socket says 172.21.15.12:53760) 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b09349700 1 -- 172.21.15.12:0/1617295310 learned_addr learned my addr 172.21.15.12:0/1617295310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.900+0000 7f6b03fff700 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c932a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.901+0000 7f6b09349700 1 -- 172.21.15.12:0/1617295310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c1890 msgr2=0x55f6c9329b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.901+0000 7f6b09349700 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c1890 0x55f6c9329b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.901+0000 7f6b09349700 1 -- 172.21.15.12:0/1617295310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 msgr2=0x55f6c932a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:19.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.901+0000 7f6b09349700 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c932a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.901+0000 7f6b09349700 1 -- 172.21.15.12:0/1617295310 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af800d960 con 0x55f6c92c7720 2024-05-09T00:01:19.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.901+0000 7f6b03fff700 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c932a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:19.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.907+0000 7f6b09349700 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c7720 0x55f6c93256b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005b40 tx=0x7f6af40052d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f722058dd8c3cd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:19.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.907+0000 7f6ae3fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:19.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.908+0000 7f6ae3fff700 1 -- 172.21.15.12:0/1617295310 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af401d040 con 0x55f6c92c7720 2024-05-09T00:01:19.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.908+0000 7f6ae3fff700 1 -- 172.21.15.12:0/1617295310 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6af4000e40 con 0x55f6c92c7720 2024-05-09T00:01:19.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.908+0000 7f6b16f66b80 1 -- 172.21.15.12:0/1617295310 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6afc099430 con 0x55f6c92c7720 2024-05-09T00:01:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.909+0000 7f6ae3fff700 1 -- 172.21.15.12:0/1617295310 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af402c460 con 0x55f6c92c7720 2024-05-09T00:01:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.910+0000 7f6b16f66b80 10 client.29401 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.910+0000 7f6b16f66b80 1 -- 172.21.15.12:0/1617295310 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f6b04113410 con 0x55f6c92c7720 2024-05-09T00:01:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.910+0000 7f6ae3fff700 1 -- 172.21.15.12:0/1617295310 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6afc099430 con 0x55f6c92c7720 2024-05-09T00:01:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.910+0000 7f6ae3fff700 1 -- 172.21.15.12:0/1617295310 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6af400b260 con 0x55f6c92c7720 2024-05-09T00:01:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.910+0000 7f6b16f66b80 10 client.29401 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.910+0000 7f6b16f66b80 10 client.29401 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:01:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.910+0000 7f6b16f66b80 20 client.29401 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.910+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f6c9332510 0x55f6c9334930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.910+0000 7f6b16f66b80 4 client.29401 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:01:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.910+0000 7f6b16f66b80 1 -- 172.21.15.12:0/1617295310 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f6c9209ec0 con 0x55f6c9332510 2024-05-09T00:01:19.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.911+0000 7f6b08b48700 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f6c9332510 0x55f6c9334930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:19.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.912+0000 7f6b08b48700 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f6c9332510 0x55f6c9334930 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=442dc6cca1f76925 server_cookie=15ddf242e520a873 in_seq=0 out_seq=0 2024-05-09T00:01:19.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.912+0000 7f6ae3fff700 10 client.29401 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:01:19.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.912+0000 7f6ae3fff700 1 -- 172.21.15.12:0/1617295310 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f6c9209ec0 con 0x55f6c9332510 2024-05-09T00:01:19.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.912+0000 7f6ae3fff700 10 client.29401 handle_command_reply: tid=0 2024-05-09T00:01:19.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.912+0000 7f6b16f66b80 1 client.29401 shutdown 2024-05-09T00:01:19.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.914+0000 7f6b16f66b80 1 -- 172.21.15.12:0/1617295310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c7720 msgr2=0x55f6c93256b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.914+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c7720 0x55f6c93256b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005b40 tx=0x7f6af40052d0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.914+0000 7f6b16f66b80 1 -- 172.21.15.12:0/1617295310 shutdown_connections 2024-05-09T00:01:19.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.914+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55f6c9332510 0x55f6c9334930 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.914+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6c92c7720 0x55f6c93256b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.914+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6c92c1890 0x55f6c9329b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.914+0000 7f6b16f66b80 1 --2- 172.21.15.12:0/1617295310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6c92c6e00 0x55f6c932a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.914+0000 7f6ae3fff700 0 client.29401 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:01:19.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.914+0000 7f6b16f66b80 1 -- 172.21.15.12:0/1617295310 >> 172.21.15.12:0/1617295310 conn(0x55f6c921f210 msgr2=0x55f6c92cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.915+0000 7f6b16f66b80 1 -- 172.21.15.12:0/1617295310 shutdown_connections 2024-05-09T00:01:19.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.915+0000 7f6b16f66b80 1 -- 172.21.15.12:0/1617295310 wait complete. 2024-05-09T00:01:19.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.915+0000 7f6b16f66b80 20 client.29401 trim_cache size 0 max 16384 2024-05-09T00:01:19.911 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.916+0000 7f6b0adac700 1 -- 172.21.15.12:0/2505511243 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6aec0a1310 msgr2=0x7f6aec0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.916+0000 7f6b0adac700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6aec0a1310 0x7f6aec0a37d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6af800ba10 tx=0x7f6af800b3f0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.916+0000 7f6b0adac700 1 -- 172.21.15.12:0/2505511243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b041052b0 msgr2=0x7f6b04115590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.916+0000 7f6b0adac700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b041052b0 0x7f6b04115590 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00b9e0 tx=0x7f6afc00bcf0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.916+0000 7f6b0adac700 1 -- 172.21.15.12:0/2505511243 shutdown_connections 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.916+0000 7f6b0adac700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6aec0a1310 0x7f6aec0a37d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.916+0000 7f6b0adac700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b0410e180 0x7f6b0410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.916+0000 7f6b0adac700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b040ff630 0x7f6b04115010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.916+0000 7f6b0adac700 1 --2- 172.21.15.12:0/2505511243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b041052b0 0x7f6b04115590 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.916+0000 7f6b0adac700 1 -- 172.21.15.12:0/2505511243 >> 172.21.15.12:0/2505511243 conn(0x7f6b04074d40 msgr2=0x7f6b04102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.917+0000 7f6b0adac700 1 -- 172.21.15.12:0/2505511243 shutdown_connections 2024-05-09T00:01:19.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:19.917+0000 7f6b0adac700 1 -- 172.21.15.12:0/2505511243 wait complete. 2024-05-09T00:01:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff839ffb700 1 -- 172.21.15.12:0/3304530453 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bed4fd8e60 con 0x55bed5100ea0 2024-05-09T00:01:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff839ffb700 10 client.29363 handle_command_reply: tid=0 2024-05-09T00:01:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 client.29363 shutdown 2024-05-09T00:01:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 -- 172.21.15.12:0/3304530453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 msgr2=0x55bed50b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed50b4000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55bed509aad0 tx=0x7ff848029970 comp rx=0 tx=0).stop 2024-05-09T00:01:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 -- 172.21.15.12:0/3304530453 shutdown_connections 2024-05-09T00:01:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bed5096270 0x55bed509b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bed5090480 0x55bed50b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bed5100ea0 0x55bed51032c0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 --2- 172.21.15.12:0/3304530453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bed5095950 0x55bed50b4000 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff839ffb700 0 client.29363 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 -- 172.21.15.12:0/3304530453 >> 172.21.15.12:0/3304530453 conn(0x55bed4fee1b0 msgr2=0x55bed50a54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 -- 172.21.15.12:0/3304530453 shutdown_connections 2024-05-09T00:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 1 -- 172.21.15.12:0/3304530453 wait complete. 2024-05-09T00:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.920+0000 7ff864f6db80 20 client.29363 trim_cache size 0 max 16384 2024-05-09T00:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.924+0000 7ff858db3700 1 -- 172.21.15.12:0/607307815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff83c0a1310 msgr2=0x7ff83c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.924+0000 7ff858db3700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff83c0a1310 0x7ff83c0a37d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff84800b2e0 tx=0x7ff848020040 comp rx=0 tx=0).stop 2024-05-09T00:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.924+0000 7ff858db3700 1 -- 172.21.15.12:0/607307815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8540ffdd0 msgr2=0x7ff854198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.924+0000 7ff858db3700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8540ffdd0 0x7ff854198080 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7ff84400e9a0 tx=0x7ff84400ecb0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.928+0000 7ff858db3700 1 -- 172.21.15.12:0/607307815 shutdown_connections 2024-05-09T00:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.928+0000 7ff858db3700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff83c0a1310 0x7ff83c0a37d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.928+0000 7ff858db3700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff854105e90 0x7ff8541a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.928+0000 7ff858db3700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8540ffdd0 0x7ff854198080 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.928+0000 7ff858db3700 1 --2- 172.21.15.12:0/607307815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff854100790 0x7ff8541985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.928+0000 7ff858db3700 1 -- 172.21.15.12:0/607307815 >> 172.21.15.12:0/607307815 conn(0x7ff854076fd0 msgr2=0x7ff8540fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.928+0000 7ff858db3700 1 -- 172.21.15.12:0/607307815 shutdown_connections 2024-05-09T00:01:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:19.928+0000 7ff858db3700 1 -- 172.21.15.12:0/607307815 wait complete. 2024-05-09T00:01:19.931 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:01:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.165+0000 7f53a20f7700 1 -- 172.21.15.12:0/2859935473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f539c101090 msgr2=0x7f539c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:20.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.165+0000 7f53a20f7700 1 --2- 172.21.15.12:0/2859935473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f539c101090 0x7f539c101470 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f538c009990 tx=0x7f538c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.165+0000 7f53a20f7700 1 -- 172.21.15.12:0/2859935473 shutdown_connections 2024-05-09T00:01:20.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.165+0000 7f53a20f7700 1 --2- 172.21.15.12:0/2859935473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f539c1061b0 0x7f539c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.165+0000 7f53a20f7700 1 --2- 172.21.15.12:0/2859935473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f539c101090 0x7f539c101470 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.165+0000 7f53a20f7700 1 --2- 172.21.15.12:0/2859935473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f539c101a40 0x7f539c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.165+0000 7f53a20f7700 1 -- 172.21.15.12:0/2859935473 >> 172.21.15.12:0/2859935473 conn(0x7f539c0fc910 msgr2=0x7f539c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:20.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.165+0000 7f53a20f7700 1 -- 172.21.15.12:0/2859935473 shutdown_connections 2024-05-09T00:01:20.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.165+0000 7f53a20f7700 1 -- 172.21.15.12:0/2859935473 wait complete. 2024-05-09T00:01:20.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.166+0000 7f53a20f7700 1 Processor -- start 2024-05-09T00:01:20.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.166+0000 7f53a20f7700 1 -- start start 2024-05-09T00:01:20.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.167+0000 7f53a20f7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f539c101090 0x7f539c1a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.167+0000 7f53a20f7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f539c1061b0 0x7f539c1a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.167+0000 7f53a20f7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f539c10cdf0 0x7f539c10d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.167+0000 7f53a20f7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c10d870 con 0x7f539c10cdf0 2024-05-09T00:01:20.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.167+0000 7f53a20f7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c10d9b0 con 0x7f539c1061b0 2024-05-09T00:01:20.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.167+0000 7f53a20f7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c10db70 con 0x7f539c101090 2024-05-09T00:01:20.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.167+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f539c101090 0x7f539c1a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.167+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f539c101090 0x7f539c1a3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53770/0 (socket says 172.21.15.12:53770) 2024-05-09T00:01:20.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.167+0000 7f539b7fe700 1 -- 172.21.15.12:0/47156823 learned_addr learned my addr 172.21.15.12:0/47156823 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:20.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f539b7fe700 1 -- 172.21.15.12:0/47156823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f539c1061b0 msgr2=0x7f539c1a3ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:01:20.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f539b7fe700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f539c1061b0 0x7f539c1a3ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f539b7fe700 1 -- 172.21.15.12:0/47156823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f539c10cdf0 msgr2=0x7f539c10d2a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:01:20.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f539b7fe700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f539c10cdf0 0x7f539c10d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f539b7fe700 1 -- 172.21.15.12:0/47156823 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c0095f0 con 0x7f539c101090 2024-05-09T00:01:20.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f539b7fe700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f539c101090 0x7f539c1a3660 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f538c004760 tx=0x7f538c009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:20.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f5398ff9700 1 -- 172.21.15.12:0/47156823 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c01d070 con 0x7f539c101090 2024-05-09T00:01:20.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f5398ff9700 1 -- 172.21.15.12:0/47156823 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f538c022470 con 0x7f539c101090 2024-05-09T00:01:20.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f53a20f7700 1 -- 172.21.15.12:0/47156823 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539c1aa470 con 0x7f539c101090 2024-05-09T00:01:20.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f5398ff9700 1 -- 172.21.15.12:0/47156823 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c017650 con 0x7f539c101090 2024-05-09T00:01:20.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.168+0000 7f53a20f7700 1 -- 172.21.15.12:0/47156823 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c1aa940 con 0x7f539c101090 2024-05-09T00:01:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.171+0000 7f5398ff9700 1 -- 172.21.15.12:0/47156823 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f538c0177b0 con 0x7f539c101090 2024-05-09T00:01:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.172+0000 7f5398ff9700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53840a1300 0x7f53840a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.172+0000 7f5398ff9700 1 -- 172.21.15.12:0/47156823 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f538c0cdff0 con 0x7f539c101090 2024-05-09T00:01:20.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.173+0000 7f53ae2b1b80 1 Processor -- start 2024-05-09T00:01:20.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.173+0000 7f53ae2b1b80 1 -- start start 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.173+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 0x55c824be21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.173+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 0x55c824bdc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.173+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bdccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.173+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824b33150 con 0x55c824be2720 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.173+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824b30b10 con 0x55c824bdc890 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.173+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824b2c400 con 0x55c824be1e00 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.173+0000 7f539affd700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53840a1300 0x7f53840a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 0x55c824be21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 0x55c824be21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53772/0 (socket says 172.21.15.12:53772) 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539b7fe700 1 -- 172.21.15.12:0/231612201 learned_addr learned my addr 172.21.15.12:0/231612201 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:20.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539affd700 1 --2- 172.21.15.12:0/231612201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 0x55c824bdc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539bfff700 1 --2- 172.21.15.12:0/231612201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bdccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539b7fe700 1 -- 172.21.15.12:0/231612201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 msgr2=0x55c824bdccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:20.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539b7fe700 1 --2- 172.21.15.12:0/231612201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bdccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539b7fe700 1 -- 172.21.15.12:0/231612201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 msgr2=0x55c824bdc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539b7fe700 1 --2- 172.21.15.12:0/231612201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 0x55c824bdc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539b7fe700 1 -- 172.21.15.12:0/231612201 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8249ad3b0 con 0x55c824be1e00 2024-05-09T00:01:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f539b7fe700 1 --2- 172.21.15.12:0/231612201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 0x55c824be21e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f538c0a3060 tx=0x7f538c09b010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce56d6ec151ce61 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.174+0000 7f5381ffb700 1 -- 172.21.15.12:0/231612201 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c0cbea0 con 0x55c824be1e00 2024-05-09T00:01:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.175+0000 7f5381ffb700 1 -- 172.21.15.12:0/231612201 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f538c09b1a0 con 0x55c824be1e00 2024-05-09T00:01:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.175+0000 7f5381ffb700 1 -- 172.21.15.12:0/231612201 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c0978b0 con 0x55c824be1e00 2024-05-09T00:01:20.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.176+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/231612201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 msgr2=0x55c824be21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:20.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.176+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/231612201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 0x55c824be21e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f538c0a3060 tx=0x7f538c09b010 comp rx=0 tx=0).stop 2024-05-09T00:01:20.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.176+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/231612201 shutdown_connections 2024-05-09T00:01:20.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.176+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/231612201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bdccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.176+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/231612201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 0x55c824be21e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.176+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/231612201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 0x55c824bdc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.176+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/231612201 >> 172.21.15.12:0/231612201 conn(0x55c824b3a210 msgr2=0x55c824bd6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.176+0000 7f539affd700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53840a1300 0x7f53840a37c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55c824bd8bd0 tx=0x7f5390009670 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.176+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/231612201 shutdown_connections 2024-05-09T00:01:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.176+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/231612201 wait complete. 2024-05-09T00:01:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.177+0000 7f53ae2b1b80 1 -- start start 2024-05-09T00:01:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.177+0000 7f53ae2b1b80 1 Processor -- start 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.177+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824bdc890 0x55c824be8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.177+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824be2720 0x55c824be60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824ac8560 con 0x55c824be1e00 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824ac8200 con 0x55c824be2720 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8249af6a0 con 0x55c824bdc890 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f53ae2b1b80 10 client.0 _ll_register_callbacks cb 0x55c824ad1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60592/0 (socket says 172.21.15.12:60592) 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539affd700 1 -- 172.21.15.12:0/2014418288 learned_addr learned my addr 172.21.15.12:0/2014418288 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539b7fe700 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824bdc890 0x55c824be8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539affd700 1 -- 172.21.15.12:0/2014418288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824bdc890 msgr2=0x55c824be8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539affd700 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824bdc890 0x55c824be8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539affd700 1 -- 172.21.15.12:0/2014418288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824be2720 msgr2=0x55c824be60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539bfff700 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824be2720 0x55c824be60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539affd700 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824be2720 0x55c824be60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539affd700 1 -- 172.21.15.12:0/2014418288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c1aa940 con 0x55c824be1e00 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f539affd700 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be59b0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f53900126d0 tx=0x7f5390009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed602b7ad43c221 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.178+0000 7f537affd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.179+0000 7f537affd700 1 -- 172.21.15.12:0/2014418288 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390009d40 con 0x55c824be1e00 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.179+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2014418288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c824a85270 con 0x55c824be1e00 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.179+0000 7f537affd700 1 -- 172.21.15.12:0/2014418288 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f539001f690 con 0x55c824be1e00 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.179+0000 7f537affd700 1 -- 172.21.15.12:0/2014418288 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f539001f850 con 0x55c824be1e00 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.179+0000 7f53ae2b1b80 10 client.19686 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.179+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2014418288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55c824959b60 con 0x55c824be1e00 2024-05-09T00:01:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.179+0000 7f537affd700 1 -- 172.21.15.12:0/2014418288 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c824a85270 con 0x55c824be1e00 2024-05-09T00:01:20.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.180+0000 7f537affd700 1 -- 172.21.15.12:0/2014418288 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5390031b60 con 0x55c824be1e00 2024-05-09T00:01:20.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.181+0000 7f53ae2b1b80 10 client.19686 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:20.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.181+0000 7f53ae2b1b80 10 client.19686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:20.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.181+0000 7f53ae2b1b80 20 client.19686 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:20.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.181+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c824bf5ef0 0x55c824bf8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.181+0000 7f53ae2b1b80 4 client.19686 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:20.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.181+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2014418288 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c824bab810 con 0x55c824bf5ef0 2024-05-09T00:01:20.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.181+0000 7f539b7fe700 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c824bf5ef0 0x55c824bf8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.182+0000 7f539b7fe700 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c824bf5ef0 0x55c824bf8310 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848ab69acb8c2ba8 server_cookie=67968b007aa72225 in_seq=0 out_seq=0 2024-05-09T00:01:20.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.182+0000 7f537affd700 10 client.19686 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.185+0000 7f537affd700 1 -- 172.21.15.12:0/2014418288 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c824bab810 con 0x55c824bf5ef0 2024-05-09T00:01:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.185+0000 7f537affd700 10 client.19686 handle_command_reply: tid=0 2024-05-09T00:01:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.185+0000 7f53ae2b1b80 1 client.19686 shutdown 2024-05-09T00:01:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2014418288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 msgr2=0x55c824be59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be59b0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f53900126d0 tx=0x7f5390009260 comp rx=0 tx=0).stop 2024-05-09T00:01:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2014418288 shutdown_connections 2024-05-09T00:01:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c824bf5ef0 0x55c824bf8310 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824be2720 0x55c824be60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824bdc890 0x55c824be8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2014418288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be59b0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2014418288 >> 172.21.15.12:0/2014418288 conn(0x55c824b3a210 msgr2=0x55c824bd65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f537affd700 0 client.19686 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2014418288 shutdown_connections 2024-05-09T00:01:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2014418288 wait complete. 2024-05-09T00:01:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.186+0000 7f53ae2b1b80 20 client.19686 trim_cache size 0 max 16384 2024-05-09T00:01:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f53ae2b1b80 1 Processor -- start 2024-05-09T00:01:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f53ae2b1b80 1 -- start start 2024-05-09T00:01:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bd6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be2720 0x55c824be87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824bd3dc0 con 0x55c824be1e00 2024-05-09T00:01:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824be6e40 con 0x55c824bdc890 2024-05-09T00:01:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824bcf0d0 con 0x55c824be2720 2024-05-09T00:01:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f539affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f539affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60600/0 (socket says 172.21.15.12:60600) 2024-05-09T00:01:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f539affd700 1 -- 172.21.15.12:0/489945591 learned_addr learned my addr 172.21.15.12:0/489945591 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f539b7fe700 1 --2- 172.21.15.12:0/489945591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bd6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.203+0000 7f539bfff700 1 --2- 172.21.15.12:0/489945591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be2720 0x55c824be87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.204+0000 7f539affd700 1 -- 172.21.15.12:0/489945591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be2720 msgr2=0x55c824be87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:20.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.204+0000 7f539affd700 1 --2- 172.21.15.12:0/489945591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be2720 0x55c824be87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.204+0000 7f539affd700 1 -- 172.21.15.12:0/489945591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 msgr2=0x55c824bd6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:20.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.204+0000 7f539affd700 1 --2- 172.21.15.12:0/489945591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bd6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.204+0000 7f539affd700 1 -- 172.21.15.12:0/489945591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c824af2560 con 0x55c824be1e00 2024-05-09T00:01:20.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.204+0000 7f539affd700 1 --2- 172.21.15.12:0/489945591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be8260 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5390031f10 tx=0x7f5390005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb743af729cc16a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:20.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.204+0000 7f53827fc700 1 -- 172.21.15.12:0/489945591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390012ea0 con 0x55c824be1e00 2024-05-09T00:01:20.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.204+0000 7f53827fc700 1 -- 172.21.15.12:0/489945591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5390031470 con 0x55c824be1e00 2024-05-09T00:01:20.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.204+0000 7f53827fc700 1 -- 172.21.15.12:0/489945591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539001fd00 con 0x55c824be1e00 2024-05-09T00:01:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.205+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/489945591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 msgr2=0x55c824be8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.205+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/489945591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be8260 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5390031f10 tx=0x7f5390005e40 comp rx=0 tx=0).stop 2024-05-09T00:01:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.206+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/489945591 shutdown_connections 2024-05-09T00:01:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.206+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/489945591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be2720 0x55c824be87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.206+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/489945591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bd6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.206+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/489945591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be1e00 0x55c824be8260 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.206+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/489945591 >> 172.21.15.12:0/489945591 conn(0x55c824b3a210 msgr2=0x55c824bdfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.206+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/489945591 shutdown_connections 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.206+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/489945591 wait complete. 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.206+0000 7f53ae2b1b80 1 -- start start 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.206+0000 7f53ae2b1b80 1 Processor -- start 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bed360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 0x55c824bdeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f53ae2b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 0x55c824bdf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824bddb00 con 0x55c824be2720 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824bddc70 con 0x55c824bdc890 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f53ae2b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c824bdde30 con 0x55c824be1e00 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f53ae2b1b80 10 client.0 _ll_register_callbacks cb 0x55c824ad1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 0x55c824bdf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 0x55c824bdf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60608/0 (socket says 172.21.15.12:60608) 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539bfff700 1 -- 172.21.15.12:0/2727243893 learned_addr learned my addr 172.21.15.12:0/2727243893 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539affd700 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 0x55c824bdeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539bfff700 1 -- 172.21.15.12:0/2727243893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 msgr2=0x55c824bdeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539bfff700 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 0x55c824bdeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539bfff700 1 -- 172.21.15.12:0/2727243893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 msgr2=0x55c824bed360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539bfff700 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bed360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539bfff700 1 -- 172.21.15.12:0/2727243893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c824959b60 con 0x55c824be2720 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.207+0000 7f539b7fe700 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bed360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.208+0000 7f539bfff700 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 0x55c824bdf220 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5394007de0 tx=0x7f5394007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e5c70aff487e0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.208+0000 7f537affd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.208+0000 7f537affd700 1 -- 172.21.15.12:0/2727243893 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394010050 con 0x55c824be2720 2024-05-09T00:01:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.208+0000 7f537affd700 1 -- 172.21.15.12:0/2727243893 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5394015b30 con 0x55c824be2720 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.208+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2727243893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f538c09b1a0 con 0x55c824be2720 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.208+0000 7f537affd700 1 -- 172.21.15.12:0/2727243893 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5394015cf0 con 0x55c824be2720 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.208+0000 7f53ae2b1b80 10 client.19692 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.208+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2727243893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55c8249b1cd0 con 0x55c824be2720 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.208+0000 7f537affd700 1 -- 172.21.15.12:0/2727243893 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f538c09b1a0 con 0x55c824be2720 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.209+0000 7f537affd700 1 -- 172.21.15.12:0/2727243893 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f53940223b0 con 0x55c824be2720 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.209+0000 7f53ae2b1b80 10 client.19692 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.209+0000 7f53ae2b1b80 10 client.19692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.209+0000 7f53ae2b1b80 20 client.19692 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.209+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c824c4d010 0x55c824c4f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.209+0000 7f53ae2b1b80 4 client.19692 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:01:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.209+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2727243893 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c824b24ec0 con 0x55c824c4d010 2024-05-09T00:01:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.209+0000 7f539b7fe700 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c824c4d010 0x55c824c4f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.209+0000 7f537affd700 10 client.19692 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.209+0000 7f539b7fe700 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c824c4d010 0x55c824c4f430 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd7d04caffd21b7 server_cookie=416c66e861ffc0c4 in_seq=0 out_seq=0 2024-05-09T00:01:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.211+0000 7f537affd700 1 -- 172.21.15.12:0/2727243893 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c824b24ec0 con 0x55c824c4d010 2024-05-09T00:01:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.211+0000 7f537affd700 10 client.19692 handle_command_reply: tid=0 2024-05-09T00:01:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.211+0000 7f53ae2b1b80 1 client.19692 shutdown 2024-05-09T00:01:20.208 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:01:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.211+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2727243893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 msgr2=0x55c824bdf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.211+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 0x55c824bdf220 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5394007de0 tx=0x7f5394007e10 comp rx=0 tx=0).stop 2024-05-09T00:01:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.212+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2727243893 shutdown_connections 2024-05-09T00:01:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.212+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c824be2720 0x55c824bdf220 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.212+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c824c4d010 0x55c824c4f430 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.212+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c824bdc890 0x55c824bed360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.212+0000 7f53ae2b1b80 1 --2- 172.21.15.12:0/2727243893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c824be1e00 0x55c824bdeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.212+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2727243893 >> 172.21.15.12:0/2727243893 conn(0x55c824b3a210 msgr2=0x55c824be4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.212+0000 7f537affd700 0 client.19692 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.212+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2727243893 shutdown_connections 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.212+0000 7f53ae2b1b80 1 -- 172.21.15.12:0/2727243893 wait complete. 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.212+0000 7f53ae2b1b80 20 client.19692 trim_cache size 0 max 16384 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 -- 172.21.15.12:0/47156823 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53840a1300 msgr2=0x7f53840a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53840a1300 0x7f53840a37c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55c824bd8bd0 tx=0x7f5390009670 comp rx=0 tx=0).stop 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 -- 172.21.15.12:0/47156823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f539c101090 msgr2=0x7f539c1a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f539c101090 0x7f539c1a3660 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f538c004760 tx=0x7f538c009570 comp rx=0 tx=0).stop 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 -- 172.21.15.12:0/47156823 shutdown_connections 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f53840a1300 0x7f53840a37c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f539c10cdf0 0x7f539c10d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f539c101090 0x7f539c1a3660 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f538c004760 tx=0x7f538c009570 comp rx=0 tx=0).stop 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 --2- 172.21.15.12:0/47156823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f539c1061b0 0x7f539c1a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 -- 172.21.15.12:0/47156823 >> 172.21.15.12:0/47156823 conn(0x7f539c0fc910 msgr2=0x7f539c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 -- 172.21.15.12:0/47156823 shutdown_connections 2024-05-09T00:01:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:20.213+0000 7f53a20f7700 1 -- 172.21.15.12:0/47156823 wait complete. 2024-05-09T00:01:20.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 53237 40.64 MB/sec warmup 75 sec latency 164.988 ms 2024-05-09T00:01:20.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:20 smithi012 ceph-mon[42503]: pgmap v478: 97 pgs: 97 active+clean; 632 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.4 MiB/s wr, 865 op/s 2024-05-09T00:01:20.761 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:20 smithi204 ceph-mon[47373]: pgmap v478: 97 pgs: 97 active+clean; 632 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.4 MiB/s wr, 865 op/s 2024-05-09T00:01:20.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:20 smithi160 ceph-mon[46152]: pgmap v478: 97 pgs: 97 active+clean; 632 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.4 MiB/s wr, 865 op/s 2024-05-09T00:01:21.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 54039 40.61 MB/sec warmup 76 sec latency 47.030 ms 2024-05-09T00:01:21.226 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:01:21.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:01:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.412+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/504389127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a780ff580 msgr2=0x7f0a780ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.412+0000 7f0a7e5ce700 1 --2- 172.21.15.12:0/504389127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a780ff580 0x7f0a780ffa00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009990 tx=0x7f0a6c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.412+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/504389127 shutdown_connections 2024-05-09T00:01:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.412+0000 7f0a7e5ce700 1 --2- 172.21.15.12:0/504389127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a780fff40 0x7f0a7810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.412+0000 7f0a7e5ce700 1 --2- 172.21.15.12:0/504389127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a78105640 0x7f0a78105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.412+0000 7f0a7e5ce700 1 --2- 172.21.15.12:0/504389127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a780ff580 0x7f0a780ffa00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.412+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/504389127 >> 172.21.15.12:0/504389127 conn(0x7f0a780763a0 msgr2=0x7f0a780fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.412+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/504389127 shutdown_connections 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.413+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/504389127 wait complete. 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.413+0000 7f0a7e5ce700 1 Processor -- start 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a7e5ce700 1 -- start start 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a7e5ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a780ff580 0x7f0a78072540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a7e5ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a780fff40 0x7f0a78072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a7e5ce700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a78105640 0x7f0a7806b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a7e5ce700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7806be40 con 0x7f0a780ff580 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a7e5ce700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7806bfb0 con 0x7f0a780fff40 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a7e5ce700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7806c170 con 0x7f0a78105640 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a77fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a780ff580 0x7f0a78072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a77fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a780ff580 0x7f0a78072540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60620/0 (socket says 172.21.15.12:60620) 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a77fff700 1 -- 172.21.15.12:0/2521626975 learned_addr learned my addr 172.21.15.12:0/2521626975 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a7cb6b700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a78105640 0x7f0a7806b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.414+0000 7f0a777fe700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a780fff40 0x7f0a78072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.415+0000 7f0a77fff700 1 -- 172.21.15.12:0/2521626975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a78105640 msgr2=0x7f0a7806b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.415+0000 7f0a77fff700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a78105640 0x7f0a7806b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.415+0000 7f0a77fff700 1 -- 172.21.15.12:0/2521626975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a780fff40 msgr2=0x7f0a78072ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.415+0000 7f0a77fff700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a780fff40 0x7f0a78072ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.415+0000 7f0a77fff700 1 -- 172.21.15.12:0/2521626975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0095f0 con 0x7f0a780ff580 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.415+0000 7f0a77fff700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a780ff580 0x7f0a78072540 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000ba50 tx=0x7f0a7000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.415+0000 7f0a757fa700 1 -- 172.21.15.12:0/2521626975 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000c930 con 0x7f0a780ff580 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.416+0000 7f0a757fa700 1 -- 172.21.15.12:0/2521626975 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a70014440 con 0x7f0a780ff580 2024-05-09T00:01:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.416+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/2521626975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7806c450 con 0x7f0a780ff580 2024-05-09T00:01:21.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.416+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/2521626975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a78070450 con 0x7f0a780ff580 2024-05-09T00:01:21.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.416+0000 7f0a757fa700 1 -- 172.21.15.12:0/2521626975 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a700075d0 con 0x7f0a780ff580 2024-05-09T00:01:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.422+0000 7f0a757fa700 1 -- 172.21.15.12:0/2521626975 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0a7000ca90 con 0x7f0a780ff580 2024-05-09T00:01:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.423+0000 7f0a8a788b80 1 Processor -- start 2024-05-09T00:01:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.423+0000 7f0a8a788b80 1 -- start start 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.423+0000 7f0a757fa700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0a600a1340 0x7f0a600a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 0x560a909cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909cd7a0 0x560a909c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909c7910 0x560a909c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9091e1d0 con 0x560a909cd7a0 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9091bb90 con 0x560a909cce80 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a90917480 con 0x560a909c7910 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a757fa700 1 -- 172.21.15.12:0/2521626975 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0a70006e40 con 0x7f0a780ff580 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a7cb6b700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0a600a1340 0x7f0a600a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a7cb6b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909c7910 0x560a909c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a7cb6b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909c7910 0x560a909c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53826/0 (socket says 172.21.15.12:53826) 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a7cb6b700 1 -- 172.21.15.12:0/4002633521 learned_addr learned my addr 172.21.15.12:0/4002633521 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a77fff700 1 --2- 172.21.15.12:0/4002633521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 0x560a909cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.424+0000 7f0a7cb6b700 1 -- 172.21.15.12:0/4002633521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 msgr2=0x560a909cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.425+0000 7f0a7cb6b700 1 --2- 172.21.15.12:0/4002633521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 0x560a909cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.425+0000 7f0a7cb6b700 1 -- 172.21.15.12:0/4002633521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909cd7a0 msgr2=0x560a909c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.425+0000 7f0a7cb6b700 1 --2- 172.21.15.12:0/4002633521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909cd7a0 0x560a909c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.425+0000 7f0a7cb6b700 1 -- 172.21.15.12:0/4002633521 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a90798430 con 0x560a909c7910 2024-05-09T00:01:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.426+0000 7f0a77fff700 1 --2- 172.21.15.12:0/4002633521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 0x560a909cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:01:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.426+0000 7f0a7cb6b700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0a600a1340 0x7f0a600a3800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009fd0 tx=0x7f0a68009ec0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.426+0000 7f0a7cb6b700 1 --2- 172.21.15.12:0/4002633521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909c7910 0x560a909c7d50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0a68015fd0 tx=0x7f0a6801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551b6c15a23dde5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.426+0000 7f0a5e7fc700 1 -- 172.21.15.12:0/4002633521 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68015390 con 0x560a909c7910 2024-05-09T00:01:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.426+0000 7f0a5e7fc700 1 -- 172.21.15.12:0/4002633521 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a68022b10 con 0x560a909c7910 2024-05-09T00:01:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.426+0000 7f0a5e7fc700 1 -- 172.21.15.12:0/4002633521 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68017950 con 0x560a909c7910 2024-05-09T00:01:21.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.427+0000 7f0a8a788b80 1 -- 172.21.15.12:0/4002633521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909c7910 msgr2=0x560a909c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.427+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/4002633521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909c7910 0x560a909c7d50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0a68015fd0 tx=0x7f0a6801a000 comp rx=0 tx=0).stop 2024-05-09T00:01:21.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.427+0000 7f0a8a788b80 1 -- 172.21.15.12:0/4002633521 shutdown_connections 2024-05-09T00:01:21.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.427+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/4002633521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909c7910 0x560a909c7d50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.427+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/4002633521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 0x560a909cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.427+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/4002633521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909cd7a0 0x560a909c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.427+0000 7f0a8a788b80 1 -- 172.21.15.12:0/4002633521 >> 172.21.15.12:0/4002633521 conn(0x560a90925290 msgr2=0x560a909c1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.428+0000 7f0a8a788b80 1 -- 172.21.15.12:0/4002633521 shutdown_connections 2024-05-09T00:01:21.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.428+0000 7f0a8a788b80 1 -- 172.21.15.12:0/4002633521 wait complete. 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.429+0000 7f0a8a788b80 1 -- start start 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.429+0000 7f0a8a788b80 1 Processor -- start 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.430+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.430+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 0x560a909d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.430+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cd7a0 0x560a909d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.430+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a908b35e0 con 0x560a909c7910 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.430+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a908b3280 con 0x560a909cce80 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.430+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9079a720 con 0x560a909cd7a0 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.430+0000 7f0a8a788b80 10 client.0 _ll_register_callbacks cb 0x560a908bcbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.431+0000 7f0a77fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.431+0000 7f0a77fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60644/0 (socket says 172.21.15.12:60644) 2024-05-09T00:01:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.431+0000 7f0a77fff700 1 -- 172.21.15.12:0/2569639066 learned_addr learned my addr 172.21.15.12:0/2569639066 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.431+0000 7f0a777fe700 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 0x560a909d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.431+0000 7f0a77fff700 1 -- 172.21.15.12:0/2569639066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cd7a0 msgr2=0x560a909d1250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.431+0000 7f0a77fff700 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cd7a0 0x560a909d1250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.431+0000 7f0a77fff700 1 -- 172.21.15.12:0/2569639066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 msgr2=0x560a909d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.431+0000 7f0a77fff700 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 0x560a909d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.431+0000 7f0a77fff700 1 -- 172.21.15.12:0/2569639066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a68018450 con 0x560a909c7910 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.432+0000 7f0a77fff700 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d3820 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0a70099500 tx=0x7f0a700c9f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b2e8ba1cd0d9e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.432+0000 7f0a577fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.432+0000 7f0a577fe700 1 -- 172.21.15.12:0/2569639066 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a700a1960 con 0x560a909c7910 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.432+0000 7f0a577fe700 1 -- 172.21.15.12:0/2569639066 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0a700840b0 con 0x560a909c7910 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.432+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2569639066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a908702f0 con 0x560a909c7910 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.432+0000 7f0a577fe700 1 -- 172.21.15.12:0/2569639066 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a700b4020 con 0x560a909c7910 2024-05-09T00:01:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.433+0000 7f0a8a788b80 10 client.19698 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.433+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2569639066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x560a90744be0 con 0x560a909c7910 2024-05-09T00:01:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.434+0000 7f0a577fe700 1 -- 172.21.15.12:0/2569639066 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a908702f0 con 0x560a909c7910 2024-05-09T00:01:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.434+0000 7f0a577fe700 1 -- 172.21.15.12:0/2569639066 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0a700a14d0 con 0x560a909c7910 2024-05-09T00:01:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.434+0000 7f0a8a788b80 10 client.19698 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.434+0000 7f0a8a788b80 10 client.19698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.434+0000 7f0a8a788b80 20 client.19698 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.434+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a909e1150 0x560a909e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.434+0000 7f0a8a788b80 4 client.19698 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.434+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2569639066 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a90996890 con 0x560a909e1150 2024-05-09T00:01:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.435+0000 7f0a777fe700 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a909e1150 0x560a909e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.436+0000 7f0a577fe700 10 client.19698 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.436+0000 7f0a777fe700 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a909e1150 0x560a909e3570 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc92f5eaf65bf14 server_cookie=4cbda2db3928799e in_seq=0 out_seq=0 2024-05-09T00:01:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.445+0000 7f0a577fe700 1 -- 172.21.15.12:0/2569639066 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a90996890 con 0x560a909e1150 2024-05-09T00:01:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.445+0000 7f0a577fe700 10 client.19698 handle_command_reply: tid=0 2024-05-09T00:01:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.445+0000 7f0a8a788b80 1 client.19698 shutdown 2024-05-09T00:01:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.445+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2569639066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 msgr2=0x560a909d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.445+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d3820 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0a70099500 tx=0x7f0a700c9f70 comp rx=0 tx=0).stop 2024-05-09T00:01:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.446+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2569639066 shutdown_connections 2024-05-09T00:01:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.446+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cd7a0 0x560a909d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.446+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d3820 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.446+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a909e1150 0x560a909e3570 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.446+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/2569639066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cce80 0x560a909d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.446+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2569639066 >> 172.21.15.12:0/2569639066 conn(0x560a90925290 msgr2=0x560a909c1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.446+0000 7f0a577fe700 0 client.19698 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:21.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.448+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2569639066 shutdown_connections 2024-05-09T00:01:21.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.448+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2569639066 wait complete. 2024-05-09T00:01:21.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.448+0000 7f0a8a788b80 20 client.19698 trim_cache size 0 max 16384 2024-05-09T00:01:21.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a8a788b80 1 Processor -- start 2024-05-09T00:01:21.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a8a788b80 1 -- start start 2024-05-09T00:01:21.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 0x560a909c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 0x560a909cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a909bc8b0 con 0x560a909c7910 2024-05-09T00:01:21.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a909de7f0 con 0x560a909cd7a0 2024-05-09T00:01:21.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a909d1ac0 con 0x560a909cce80 2024-05-09T00:01:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a7cb6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 0x560a909cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 0x560a909c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a7cb6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 0x560a909cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43764/0 (socket says 172.21.15.12:43764) 2024-05-09T00:01:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.477+0000 7f0a7cb6b700 1 -- 172.21.15.12:0/2792134830 learned_addr learned my addr 172.21.15.12:0/2792134830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.478+0000 7f0a77fff700 1 --2- 172.21.15.12:0/2792134830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.478+0000 7f0a7cb6b700 1 -- 172.21.15.12:0/2792134830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 msgr2=0x560a909c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.478+0000 7f0a7cb6b700 1 --2- 172.21.15.12:0/2792134830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 0x560a909c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.478+0000 7f0a7cb6b700 1 -- 172.21.15.12:0/2792134830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 msgr2=0x560a909c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.478+0000 7f0a7cb6b700 1 --2- 172.21.15.12:0/2792134830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.478+0000 7f0a7cb6b700 1 -- 172.21.15.12:0/2792134830 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a908dd5e0 con 0x560a909cd7a0 2024-05-09T00:01:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.478+0000 7f0a7cb6b700 1 --2- 172.21.15.12:0/2792134830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 0x560a909cbbd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a680093c0 tx=0x7f0a680090e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=312ab711120753a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.478+0000 7f0a5effd700 1 -- 172.21.15.12:0/2792134830 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68015390 con 0x560a909cd7a0 2024-05-09T00:01:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.479+0000 7f0a5effd700 1 -- 172.21.15.12:0/2792134830 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a68030070 con 0x560a909cd7a0 2024-05-09T00:01:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.479+0000 7f0a5effd700 1 -- 172.21.15.12:0/2792134830 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6802ca60 con 0x560a909cd7a0 2024-05-09T00:01:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.479+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2792134830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 msgr2=0x560a909cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.479+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/2792134830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 0x560a909cbbd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a680093c0 tx=0x7f0a680090e0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.479+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2792134830 shutdown_connections 2024-05-09T00:01:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.479+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/2792134830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 0x560a909cbbd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.479+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/2792134830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.479+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/2792134830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 0x560a909c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.479+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2792134830 >> 172.21.15.12:0/2792134830 conn(0x560a90925290 msgr2=0x560a909cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.480+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2792134830 shutdown_connections 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.480+0000 7f0a8a788b80 1 -- 172.21.15.12:0/2792134830 wait complete. 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.480+0000 7f0a8a788b80 1 -- start start 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.480+0000 7f0a8a788b80 1 Processor -- start 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 0x560a909d5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a8a788b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 0x560a90a37610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a909d5c40 con 0x560a909c7910 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a909d5db0 con 0x560a909cd7a0 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a8a788b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a90a37b80 con 0x560a909cce80 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a8a788b80 10 client.0 _ll_register_callbacks cb 0x560a908bcbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 0x560a909d5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 0x560a909d5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53864/0 (socket says 172.21.15.12:53864) 2024-05-09T00:01:21.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a777fe700 1 -- 172.21.15.12:0/283523859 learned_addr learned my addr 172.21.15.12:0/283523859 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:21.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.481+0000 7f0a77fff700 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a777fe700 1 -- 172.21.15.12:0/283523859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 msgr2=0x560a90a37610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a777fe700 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 0x560a90a37610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a777fe700 1 -- 172.21.15.12:0/283523859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 msgr2=0x560a909d4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a777fe700 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a777fe700 1 -- 172.21.15.12:0/283523859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a90798430 con 0x560a909cce80 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a77fff700 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a777fe700 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 0x560a909d5480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00a010 tx=0x7f0a6c0038c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=571a9727c90a8982 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a577fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a577fe700 1 -- 172.21.15.12:0/283523859 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c003960 con 0x560a909cce80 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a577fe700 1 -- 172.21.15.12:0/283523859 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0a6c012070 con 0x560a909cce80 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a8a788b80 1 -- 172.21.15.12:0/283523859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a68022b10 con 0x560a909cce80 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.482+0000 7f0a577fe700 1 -- 172.21.15.12:0/283523859 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a6c003ac0 con 0x560a909cce80 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a8a788b80 10 client.29434 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a8a788b80 1 -- 172.21.15.12:0/283523859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x560a9079cd50 con 0x560a909cce80 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a577fe700 1 -- 172.21.15.12:0/283523859 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a68022b10 con 0x560a909cce80 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a577fe700 1 -- 172.21.15.12:0/283523859 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0a6c02d060 con 0x560a909cce80 2024-05-09T00:01:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a8a788b80 10 client.29434 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a8a788b80 10 client.29434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a8a788b80 20 client.29434 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a90a39510 0x560a90a3b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a8a788b80 4 client.29434 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:01:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a8a788b80 1 -- 172.21.15.12:0/283523859 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560a9090ff40 con 0x560a90a39510 2024-05-09T00:01:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.483+0000 7f0a77fff700 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a90a39510 0x560a90a3b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.484+0000 7f0a77fff700 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a90a39510 0x560a90a3b930 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44ccb7c0a535ec50 server_cookie=50e83c5aba051b3f in_seq=0 out_seq=0 2024-05-09T00:01:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.484+0000 7f0a577fe700 10 client.29434 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.490+0000 7f0a577fe700 1 -- 172.21.15.12:0/283523859 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560a9090ff40 con 0x560a90a39510 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.490+0000 7f0a577fe700 10 client.29434 handle_command_reply: tid=0 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.490+0000 7f0a8a788b80 1 client.29434 shutdown 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 1 -- 172.21.15.12:0/283523859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 msgr2=0x560a909d5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 0x560a909d5480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00a010 tx=0x7f0a6c0038c0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 1 -- 172.21.15.12:0/283523859 shutdown_connections 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a90a39510 0x560a90a3b930 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a909cd7a0 0x560a90a37610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a909c7910 0x560a909d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a577fe700 0 client.29434 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 1 --2- 172.21.15.12:0/283523859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a909cce80 0x560a909d5480 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 1 -- 172.21.15.12:0/283523859 >> 172.21.15.12:0/283523859 conn(0x560a90925290 msgr2=0x560a909cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 1 -- 172.21.15.12:0/283523859 shutdown_connections 2024-05-09T00:01:21.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 1 -- 172.21.15.12:0/283523859 wait complete. 2024-05-09T00:01:21.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.491+0000 7f0a8a788b80 20 client.29434 trim_cache size 0 max 16384 2024-05-09T00:01:21.487 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:01:21.487 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:01:21.487 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "38b5e8dc-e8a1-42a6-ab70-4c68a19f1f39", 2024-05-09T00:01:21.487 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:01:21.487 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:01:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/2521626975 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0a600a1340 msgr2=0x7f0a600a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0a600a1340 0x7f0a600a3800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009fd0 tx=0x7f0a68009ec0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/2521626975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a780ff580 msgr2=0x7f0a78072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a780ff580 0x7f0a78072540 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000ba50 tx=0x7f0a7000bd60 comp rx=0 tx=0).stop 2024-05-09T00:01:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/2521626975 shutdown_connections 2024-05-09T00:01:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0a600a1340 0x7f0a600a3800 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a78105640 0x7f0a7806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a780ff580 0x7f0a78072540 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 --2- 172.21.15.12:0/2521626975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a780fff40 0x7f0a78072ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/2521626975 >> 172.21.15.12:0/2521626975 conn(0x7f0a780763a0 msgr2=0x7f0a780fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/2521626975 shutdown_connections 2024-05-09T00:01:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.493+0000 7f0a7e5ce700 1 -- 172.21.15.12:0/2521626975 wait complete. 2024-05-09T00:01:21.507 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:01:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.698+0000 7fdf91f99700 1 -- 172.21.15.12:0/2068525115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf8c0ff410 msgr2=0x7fdf8c0ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.698+0000 7fdf91f99700 1 --2- 172.21.15.12:0/2068525115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf8c0ff410 0x7fdf8c0ff890 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fdf800098f0 tx=0x7fdf80009c00 comp rx=0 tx=0).stop 2024-05-09T00:01:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.699+0000 7fdf91f99700 1 -- 172.21.15.12:0/2068525115 shutdown_connections 2024-05-09T00:01:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.699+0000 7fdf91f99700 1 --2- 172.21.15.12:0/2068525115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf8c0ffeb0 0x7fdf8c103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.699+0000 7fdf91f99700 1 --2- 172.21.15.12:0/2068525115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf8c106050 0x7fdf8c106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.699+0000 7fdf91f99700 1 --2- 172.21.15.12:0/2068525115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf8c0ff410 0x7fdf8c0ff890 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.699+0000 7fdf91f99700 1 -- 172.21.15.12:0/2068525115 >> 172.21.15.12:0/2068525115 conn(0x7fdf8c0763a0 msgr2=0x7fdf8c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.699+0000 7fdf91f99700 1 -- 172.21.15.12:0/2068525115 shutdown_connections 2024-05-09T00:01:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.699+0000 7fdf91f99700 1 -- 172.21.15.12:0/2068525115 wait complete. 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.700+0000 7fdf91f99700 1 Processor -- start 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.700+0000 7fdf91f99700 1 -- start start 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.700+0000 7fdf91f99700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf8c0ff410 0x7fdf8c1a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.700+0000 7fdf91f99700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf8c0ffeb0 0x7fdf8c1a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.700+0000 7fdf91f99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf8c106050 0x7fdf8c19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.700+0000 7fdf91f99700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c19e080 con 0x7fdf8c106050 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.700+0000 7fdf91f99700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c19e1f0 con 0x7fdf8c0ffeb0 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.700+0000 7fdf91f99700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c19e3b0 con 0x7fdf8c0ff410 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.700+0000 7fdf8b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf8c0ff410 0x7fdf8c1a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf8bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf8c106050 0x7fdf8c19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf8b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf8c0ff410 0x7fdf8c1a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53886/0 (socket says 172.21.15.12:53886) 2024-05-09T00:01:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/879649351 learned_addr learned my addr 172.21.15.12:0/879649351 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf8affd700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf8c0ffeb0 0x7fdf8c1a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/879649351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf8c0ffeb0 msgr2=0x7fdf8c1a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf8c0ffeb0 0x7fdf8c1a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/879649351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf8c106050 msgr2=0x7fdf8c19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf8c106050 0x7fdf8c19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/879649351 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c0095f0 con 0x7fdf8c0ff410 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf8c0ff410 0x7fdf8c1a38f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00ec10 tx=0x7fdf7c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf88ff9700 1 -- 172.21.15.12:0/879649351 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00cc90 con 0x7fdf8c0ff410 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf91f99700 1 -- 172.21.15.12:0/879649351 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf80009520 con 0x7fdf8c0ff410 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf88ff9700 1 -- 172.21.15.12:0/879649351 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7c0044e0 con 0x7fdf8c0ff410 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.701+0000 7fdf88ff9700 1 -- 172.21.15.12:0/879649351 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c010430 con 0x7fdf8c0ff410 2024-05-09T00:01:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.702+0000 7fdf91f99700 1 -- 172.21.15.12:0/879649351 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8c1aa700 con 0x7fdf8c0ff410 2024-05-09T00:01:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.704+0000 7fdf88ff9700 1 -- 172.21.15.12:0/879649351 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdf7c003fb0 con 0x7fdf8c0ff410 2024-05-09T00:01:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.705+0000 7fdf88ff9700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf740a1290 0x7fdf740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.705+0000 7fdf88ff9700 1 -- 172.21.15.12:0/879649351 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdf7c014070 con 0x7fdf8c0ff410 2024-05-09T00:01:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.706+0000 7fdf8affd700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf740a1290 0x7fdf740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.706+0000 7fdf9e153b80 1 Processor -- start 2024-05-09T00:01:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.706+0000 7fdf8affd700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf740a1290 0x7fdf740a3750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009fd0 tx=0x7fdf80019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.706+0000 7fdf9e153b80 1 -- start start 2024-05-09T00:01:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 0x55ffc9b1e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b1ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc9a64150 con 0x55ffc9b23df0 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc9a61b10 con 0x55ffc9b24710 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc9a5d400 con 0x55ffc9b1e8a0 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf8b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b241d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60686/0 (socket says 172.21.15.12:60686) 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/3028738028 learned_addr learned my addr 172.21.15.12:0/3028738028 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf8bfff700 1 --2- 172.21.15.12:0/3028738028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b1ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/3028738028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 msgr2=0x55ffc9b1ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/3028738028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b1ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/3028738028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 msgr2=0x55ffc9b1e360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/3028738028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 0x55ffc9b1e360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/3028738028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffc98de3b0 con 0x55ffc9b23df0 2024-05-09T00:01:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.707+0000 7fdf8bfff700 1 --2- 172.21.15.12:0/3028738028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b1ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.708+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/3028738028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b241d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c096d50 tx=0x7fdf7c09a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894a8363d0c83da1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.708+0000 7fdf71ffb700 1 -- 172.21.15.12:0/3028738028 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c016330 con 0x55ffc9b23df0 2024-05-09T00:01:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.708+0000 7fdf71ffb700 1 -- 172.21.15.12:0/3028738028 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7c0b2e90 con 0x55ffc9b23df0 2024-05-09T00:01:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.708+0000 7fdf71ffb700 1 -- 172.21.15.12:0/3028738028 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c098680 con 0x55ffc9b23df0 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.709+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3028738028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 msgr2=0x55ffc9b241d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.709+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3028738028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b241d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c096d50 tx=0x7fdf7c09a130 comp rx=0 tx=0).stop 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.709+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3028738028 shutdown_connections 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.709+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3028738028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b1ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.709+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3028738028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b241d0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.709+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3028738028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 0x55ffc9b1e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.709+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3028738028 >> 172.21.15.12:0/3028738028 conn(0x55ffc9a6b210 msgr2=0x55ffc9b07df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.709+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3028738028 shutdown_connections 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.709+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3028738028 wait complete. 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.709+0000 7fdf9e153b80 1 -- start start 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf9e153b80 1 Processor -- start 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b28820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b24710 0x55ffc9b166c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc99f9560 con 0x55ffc9b23df0 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc99f9200 con 0x55ffc9b1e8a0 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc98e06a0 con 0x55ffc9b24710 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf9e153b80 10 client.0 _ll_register_callbacks cb 0x55ffc9a02b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf8bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b24710 0x55ffc9b166c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf8b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf8bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b24710 0x55ffc9b166c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53902/0 (socket says 172.21.15.12:53902) 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.710+0000 7fdf8b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b282e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43808/0 (socket says 172.21.15.12:43808) 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf8bfff700 1 -- 172.21.15.12:0/3058071910 learned_addr learned my addr 172.21.15.12:0/3058071910 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf8bfff700 1 -- 172.21.15.12:0/3058071910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b1e8a0 msgr2=0x55ffc9b282e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf8bfff700 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b282e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf8bfff700 1 -- 172.21.15.12:0/3058071910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 msgr2=0x55ffc9b28820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf8bfff700 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b28820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf8bfff700 1 -- 172.21.15.12:0/3058071910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf84011230 con 0x55ffc9b24710 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf8bfff700 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b24710 0x55ffc9b166c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdf84007c30 tx=0x7fdf84007f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=127096c2f1b1b1ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf6affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf6affd700 1 -- 172.21.15.12:0/3058071910 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400e040 con 0x55ffc9b24710 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf6affd700 1 -- 172.21.15.12:0/3058071910 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdf84011d30 con 0x55ffc9b24710 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.711+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3058071910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffc99b6270 con 0x55ffc9b24710 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf6affd700 1 -- 172.21.15.12:0/3058071910 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf8401f460 con 0x55ffc9b24710 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf9e153b80 10 client.29446 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3058071910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7fdf8c1aa700 con 0x55ffc9b24710 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf6affd700 1 -- 172.21.15.12:0/3058071910 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffc99b6270 con 0x55ffc9b24710 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf6affd700 1 -- 172.21.15.12:0/3058071910 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdf84026070 con 0x55ffc9b24710 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf9e153b80 10 client.29446 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf9e153b80 10 client.29446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf9e153b80 20 client.29446 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ffc9b1ad10 0x55ffc9b1d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf9e153b80 4 client.29446 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.712+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3058071910 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffc9adc810 con 0x55ffc9b1ad10 2024-05-09T00:01:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.713+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ffc9b1ad10 0x55ffc9b1d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.713+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ffc9b1ad10 0x55ffc9b1d130 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e3b97bb9c180cf server_cookie=70ab16f29e347569 in_seq=0 out_seq=0 2024-05-09T00:01:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.714+0000 7fdf6affd700 10 client.29446 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf6affd700 1 -- 172.21.15.12:0/3058071910 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffc9adc810 con 0x55ffc9b1ad10 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf6affd700 10 client.29446 handle_command_reply: tid=0 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf9e153b80 1 client.29446 shutdown 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3058071910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b24710 msgr2=0x55ffc9b166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b24710 0x55ffc9b166c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdf84007c30 tx=0x7fdf84007f40 comp rx=0 tx=0).stop 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3058071910 shutdown_connections 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b24710 0x55ffc9b166c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ffc9b1ad10 0x55ffc9b1d130 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3058071910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b28820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.716+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3058071910 >> 172.21.15.12:0/3058071910 conn(0x55ffc9a6b210 msgr2=0x55ffc9b075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.717+0000 7fdf6affd700 0 client.29446 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.717+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3058071910 shutdown_connections 2024-05-09T00:01:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.717+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3058071910 wait complete. 2024-05-09T00:01:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.717+0000 7fdf9e153b80 20 client.29446 trim_cache size 0 max 16384 2024-05-09T00:01:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.740+0000 7fdf9e153b80 1 Processor -- start 2024-05-09T00:01:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.740+0000 7fdf9e153b80 1 -- start start 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.740+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b25bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.740+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b23df0 0x55ffc9b07d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.740+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b24710 0x55ffc9b28640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.740+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc9b29100 con 0x55ffc9b24710 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.740+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc9b17800 con 0x55ffc9b23df0 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.740+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc9b260f0 con 0x55ffc9b1e8a0 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b25bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b23df0 0x55ffc9b07d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b25bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53906/0 (socket says 172.21.15.12:53906) 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b23df0 0x55ffc9b07d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43824/0 (socket says 172.21.15.12:43824) 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/3651053076 learned_addr learned my addr 172.21.15.12:0/3651053076 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8bfff700 1 --2- 172.21.15.12:0/3651053076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b24710 0x55ffc9b28640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/3651053076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b23df0 msgr2=0x55ffc9b07d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/3651053076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b23df0 0x55ffc9b07d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/3651053076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b24710 msgr2=0x55ffc9b28640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/3651053076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b24710 0x55ffc9b28640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8b7fe700 1 -- 172.21.15.12:0/3651053076 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffc988ab60 con 0x55ffc9b1e8a0 2024-05-09T00:01:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8bfff700 1 --2- 172.21.15.12:0/3651053076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b24710 0x55ffc9b28640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:01:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/3651053076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b25bb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c5b0 tx=0x7fdf7c003a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26434b977f58202 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.741+0000 7fdf727fc700 1 -- 172.21.15.12:0/3651053076 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c09a9e0 con 0x55ffc9b1e8a0 2024-05-09T00:01:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.742+0000 7fdf727fc700 1 -- 172.21.15.12:0/3651053076 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7c09a2e0 con 0x55ffc9b1e8a0 2024-05-09T00:01:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.742+0000 7fdf727fc700 1 -- 172.21.15.12:0/3651053076 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c098790 con 0x55ffc9b1e8a0 2024-05-09T00:01:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.742+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3651053076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 msgr2=0x55ffc9b25bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.742+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3651053076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b25bb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c5b0 tx=0x7fdf7c003a10 comp rx=0 tx=0).stop 2024-05-09T00:01:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.742+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3651053076 shutdown_connections 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.742+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3651053076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b24710 0x55ffc9b28640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.742+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3651053076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b25bb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.742+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3651053076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b23df0 0x55ffc9b07d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.742+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3651053076 >> 172.21.15.12:0/3651053076 conn(0x55ffc9a6b210 msgr2=0x55ffc9b23170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.742+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3651053076 shutdown_connections 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.743+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3651053076 wait complete. 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.743+0000 7fdf9e153b80 1 -- start start 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.743+0000 7fdf9e153b80 1 Processor -- start 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.744+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b11af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.744+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b27750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.744+0000 7fdf9e153b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 0x55ffc9b27e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.744+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc9b28580 con 0x55ffc9b23df0 2024-05-09T00:01:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.744+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc9b8d640 con 0x55ffc9b24710 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.744+0000 7fdf9e153b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc9b8d780 con 0x55ffc9b1e8a0 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.744+0000 7fdf9e153b80 10 client.0 _ll_register_callbacks cb 0x55ffc9a02b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b27750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf8bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 0x55ffc9b27e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf8bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 0x55ffc9b27e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43834/0 (socket says 172.21.15.12:43834) 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf8bfff700 1 -- 172.21.15.12:0/3756205079 learned_addr learned my addr 172.21.15.12:0/3756205079 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf8bfff700 1 -- 172.21.15.12:0/3756205079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 msgr2=0x55ffc9b11af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf8bfff700 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b11af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf8bfff700 1 -- 172.21.15.12:0/3756205079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 msgr2=0x55ffc9b27750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf8bfff700 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b27750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf8bfff700 1 -- 172.21.15.12:0/3756205079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffc98de3b0 con 0x55ffc9b24710 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf8bfff700 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 0x55ffc9b27e70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf84007980 tx=0x7fdf840169e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f6e17ab8c2ee04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf6affd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf6affd700 1 -- 172.21.15.12:0/3756205079 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400e040 con 0x55ffc9b24710 2024-05-09T00:01:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf6affd700 1 -- 172.21.15.12:0/3756205079 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdf84003a00 con 0x55ffc9b24710 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.745+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3756205079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf7c0b2e90 con 0x55ffc9b24710 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf6affd700 1 -- 172.21.15.12:0/3756205079 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf84003c00 con 0x55ffc9b24710 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf9e153b80 10 client.29417 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3756205079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55ffc9a23560 con 0x55ffc9b24710 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf6affd700 1 -- 172.21.15.12:0/3756205079 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7c0b2e90 con 0x55ffc9b24710 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf6affd700 1 -- 172.21.15.12:0/3756205079 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdf84037070 con 0x55ffc9b24710 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf9e153b80 10 client.29417 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf9e153b80 10 client.29417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf9e153b80 20 client.29417 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ffc9b8ee60 0x55ffc9b91280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf9e153b80 4 client.29417 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3756205079 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ffc9a55ec0 con 0x55ffc9b8ee60 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.746+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ffc9b8ee60 0x55ffc9b91280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.747+0000 7fdf6affd700 10 client.29417 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:21.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.747+0000 7fdf8b7fe700 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ffc9b8ee60 0x55ffc9b91280 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51704ccb81c3af75 server_cookie=92b80fa0993607ef in_seq=0 out_seq=0 2024-05-09T00:01:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf6affd700 1 -- 172.21.15.12:0/3756205079 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55ffc9a55ec0 con 0x55ffc9b8ee60 2024-05-09T00:01:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf6affd700 10 client.29417 handle_command_reply: tid=0 2024-05-09T00:01:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 client.29417 shutdown 2024-05-09T00:01:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3756205079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 msgr2=0x55ffc9b27e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 0x55ffc9b27e70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf84007980 tx=0x7fdf840169e0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3756205079 shutdown_connections 2024-05-09T00:01:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ffc9b8ee60 0x55ffc9b91280 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffc9b24710 0x55ffc9b27e70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffc9b1e8a0 0x55ffc9b11af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf6affd700 0 client.29417 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:21.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 --2- 172.21.15.12:0/3756205079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ffc9b23df0 0x55ffc9b27750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3756205079 >> 172.21.15.12:0/3756205079 conn(0x55ffc9a6b210 msgr2=0x55ffc9b221d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3756205079 shutdown_connections 2024-05-09T00:01:21.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 1 -- 172.21.15.12:0/3756205079 wait complete. 2024-05-09T00:01:21.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.752+0000 7fdf9e153b80 20 client.29417 trim_cache size 0 max 16384 2024-05-09T00:01:21.748 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:01:21.748 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (114 inodes in the stack)", 2024-05-09T00:01:21.748 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:01:21.748 INFO:teuthology.orchestra.run.smithi012.stdout: "38b5e8dc-e8a1-42a6-ab70-4c68a19f1f39": { 2024-05-09T00:01:21.749 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:01:21.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "38b5e8dc-e8a1-42a6-ab70-4c68a19f1f39", 2024-05-09T00:01:21.749 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:01:21.749 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:01:21.749 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:01:21.749 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 -- 172.21.15.12:0/879649351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf740a1290 msgr2=0x7fdf740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf740a1290 0x7fdf740a3750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009fd0 tx=0x7fdf80019040 comp rx=0 tx=0).stop 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 -- 172.21.15.12:0/879649351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf8c0ff410 msgr2=0x7fdf8c1a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf8c0ff410 0x7fdf8c1a38f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00ec10 tx=0x7fdf7c00ef20 comp rx=0 tx=0).stop 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 -- 172.21.15.12:0/879649351 shutdown_connections 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdf740a1290 0x7fdf740a3750 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf8c106050 0x7fdf8c19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf8c0ff410 0x7fdf8c1a38f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 --2- 172.21.15.12:0/879649351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf8c0ffeb0 0x7fdf8c1a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 -- 172.21.15.12:0/879649351 >> 172.21.15.12:0/879649351 conn(0x7fdf8c0763a0 msgr2=0x7fdf8c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 -- 172.21.15.12:0/879649351 shutdown_connections 2024-05-09T00:01:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:21.753+0000 7fdf91f99700 1 -- 172.21.15.12:0/879649351 wait complete. 2024-05-09T00:01:21.764 INFO:tasks.cephfs.filesystem:scrub status for tag:38b5e8dc-e8a1-42a6-ab70-4c68a19f1f39 - {'path': '/', 'tag': '38b5e8dc-e8a1-42a6-ab70-4c68a19f1f39', 'options': 'recursive,force'} 2024-05-09T00:01:21.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:21 smithi012 ceph-mon[42503]: pgmap v479: 97 pgs: 97 active+clean; 648 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-05-09T00:01:22.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:21 smithi204 ceph-mon[47373]: pgmap v479: 97 pgs: 97 active+clean; 648 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-05-09T00:01:22.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:21 smithi160 ceph-mon[46152]: pgmap v479: 97 pgs: 97 active+clean; 648 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-05-09T00:01:22.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 54915 40.70 MB/sec warmup 77 sec latency 70.182 ms 2024-05-09T00:01:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:22 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:01:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:22 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:01:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:22 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:01:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:22 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:01:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:22.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:23.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:22 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:01:23.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:23.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:22 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:01:23.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:22 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:01:23.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:22 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:01:23.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:23.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:23.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:23.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:22 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:01:23.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:23.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:22 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:01:23.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:22 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:01:23.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:22 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:01:23.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:23.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:23.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:23.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 55923 40.91 MB/sec warmup 78 sec latency 69.680 ms 2024-05-09T00:01:23.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:23 smithi012 ceph-mon[42503]: pgmap v480: 97 pgs: 97 active+clean; 648 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-05-09T00:01:24.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:23 smithi204 ceph-mon[47373]: pgmap v480: 97 pgs: 97 active+clean; 648 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-05-09T00:01:24.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:23 smithi160 ceph-mon[46152]: pgmap v480: 97 pgs: 97 active+clean; 648 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-05-09T00:01:24.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 56761 40.94 MB/sec warmup 79 sec latency 76.035 ms 2024-05-09T00:01:25.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 57731 41.06 MB/sec warmup 80 sec latency 94.497 ms 2024-05-09T00:01:25.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:25 smithi012 ceph-mon[42503]: pgmap v481: 97 pgs: 97 active+clean; 648 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 639 op/s 2024-05-09T00:01:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:25 smithi204 ceph-mon[47373]: pgmap v481: 97 pgs: 97 active+clean; 648 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 639 op/s 2024-05-09T00:01:26.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:25 smithi160 ceph-mon[46152]: pgmap v481: 97 pgs: 97 active+clean; 648 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 639 op/s 2024-05-09T00:01:26.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 58795 41.34 MB/sec warmup 81 sec latency 34.659 ms 2024-05-09T00:01:27.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 59400 41.23 MB/sec warmup 82 sec latency 41.271 ms 2024-05-09T00:01:27.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:27 smithi012 ceph-mon[42503]: pgmap v482: 97 pgs: 97 active+clean; 678 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.4 MiB/s wr, 962 op/s 2024-05-09T00:01:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:27 smithi204 ceph-mon[47373]: pgmap v482: 97 pgs: 97 active+clean; 678 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.4 MiB/s wr, 962 op/s 2024-05-09T00:01:28.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:27 smithi160 ceph-mon[46152]: pgmap v482: 97 pgs: 97 active+clean; 678 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.4 MiB/s wr, 962 op/s 2024-05-09T00:01:28.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 60080 41.07 MB/sec warmup 83 sec latency 93.440 ms 2024-05-09T00:01:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:28 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:01:29.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:28 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:01:29.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:28 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:01:29.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 61176 41.22 MB/sec warmup 84 sec latency 56.131 ms 2024-05-09T00:01:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:01:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:01:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:29 smithi012 ceph-mon[42503]: pgmap v483: 97 pgs: 97 active+clean; 678 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.2 MiB/s wr, 700 op/s 2024-05-09T00:01:30.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:01:30.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:01:30.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:29 smithi204 ceph-mon[47373]: pgmap v483: 97 pgs: 97 active+clean; 678 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.2 MiB/s wr, 700 op/s 2024-05-09T00:01:30.019 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:01:30.020 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:01:30.020 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:29 smithi160 ceph-mon[46152]: pgmap v483: 97 pgs: 97 active+clean; 678 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.2 MiB/s wr, 700 op/s 2024-05-09T00:01:30.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 62019 41.45 MB/sec warmup 85 sec latency 242.063 ms 2024-05-09T00:01:31.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 62045 40.97 MB/sec warmup 86 sec latency 945.703 ms 2024-05-09T00:01:31.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:31 smithi012 ceph-mon[42503]: pgmap v484: 97 pgs: 97 active+clean; 864 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 24 MiB/s wr, 1.16k op/s 2024-05-09T00:01:32.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:31 smithi204 ceph-mon[47373]: pgmap v484: 97 pgs: 97 active+clean; 864 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 24 MiB/s wr, 1.16k op/s 2024-05-09T00:01:32.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:31 smithi160 ceph-mon[46152]: pgmap v484: 97 pgs: 97 active+clean; 864 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 24 MiB/s wr, 1.16k op/s 2024-05-09T00:01:32.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 62413 40.72 MB/sec warmup 87 sec latency 625.966 ms 2024-05-09T00:01:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.104+0000 7fbf2e647700 1 -- 172.21.15.12:0/1305530560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf28105150 msgr2=0x7fbf2810ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.104+0000 7fbf2e647700 1 --2- 172.21.15.12:0/1305530560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf28105150 0x7fbf2810ccd0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000b0a0 tx=0x7fbf2000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.104+0000 7fbf2e647700 1 -- 172.21.15.12:0/1305530560 shutdown_connections 2024-05-09T00:01:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.104+0000 7fbf2e647700 1 --2- 172.21.15.12:0/1305530560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf28105150 0x7fbf2810ccd0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.104+0000 7fbf2e647700 1 --2- 172.21.15.12:0/1305530560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf28100030 0x7fbf28100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.104+0000 7fbf2e647700 1 --2- 172.21.15.12:0/1305530560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf281009e0 0x7fbf28104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.104+0000 7fbf2e647700 1 -- 172.21.15.12:0/1305530560 >> 172.21.15.12:0/1305530560 conn(0x7fbf28076f70 msgr2=0x7fbf28077380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.104+0000 7fbf2e647700 1 -- 172.21.15.12:0/1305530560 shutdown_connections 2024-05-09T00:01:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.104+0000 7fbf2e647700 1 -- 172.21.15.12:0/1305530560 wait complete. 2024-05-09T00:01:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.105+0000 7fbf2e647700 1 Processor -- start 2024-05-09T00:01:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.105+0000 7fbf2e647700 1 -- start start 2024-05-09T00:01:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf2e647700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf28100030 0x7fbf28197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf2e647700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf28105150 0x7fbf28197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf2e647700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2819be10 0x7fbf2819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf2e647700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2819c890 con 0x7fbf28105150 2024-05-09T00:01:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf2e647700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2819ca00 con 0x7fbf2819be10 2024-05-09T00:01:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf2e647700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2819cbc0 con 0x7fbf28100030 2024-05-09T00:01:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf27fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf28100030 0x7fbf28197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf2cbe4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2819be10 0x7fbf2819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf2cbe4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2819be10 0x7fbf2819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47088/0 (socket says 172.21.15.12:47088) 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf27fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf28100030 0x7fbf28197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48520/0 (socket says 172.21.15.12:48520) 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/4108246850 learned_addr learned my addr 172.21.15.12:0/4108246850 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf1ffff700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf28105150 0x7fbf28197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf27fff700 1 -- 172.21.15.12:0/4108246850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2819be10 msgr2=0x7fbf2819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf27fff700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2819be10 0x7fbf2819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf27fff700 1 -- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf28105150 msgr2=0x7fbf28197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf27fff700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf28105150 0x7fbf28197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf27fff700 1 -- 172.21.15.12:0/4108246850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf20009d00 con 0x7fbf28100030 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2819be10 0x7fbf2819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.106+0000 7fbf1ffff700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf28105150 0x7fbf28197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.107+0000 7fbf27fff700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf28100030 0x7fbf28197530 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800e9f0 tx=0x7fbf1800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.107+0000 7fbf25ffb700 1 -- 172.21.15.12:0/4108246850 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800eee0 con 0x7fbf28100030 2024-05-09T00:01:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.107+0000 7fbf2e647700 1 -- 172.21.15.12:0/4108246850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf281a4120 con 0x7fbf28100030 2024-05-09T00:01:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.107+0000 7fbf2e647700 1 -- 172.21.15.12:0/4108246850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf281a4650 con 0x7fbf28100030 2024-05-09T00:01:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.107+0000 7fbf25ffb700 1 -- 172.21.15.12:0/4108246850 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf1800cbd0 con 0x7fbf28100030 2024-05-09T00:01:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.107+0000 7fbf25ffb700 1 -- 172.21.15.12:0/4108246850 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18018680 con 0x7fbf28100030 2024-05-09T00:01:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.110+0000 7fbf25ffb700 1 -- 172.21.15.12:0/4108246850 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbf18010430 con 0x7fbf28100030 2024-05-09T00:01:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.111+0000 7fbf25ffb700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf140a56a0 0x7fbf140a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.111+0000 7fbf3a801b80 1 Processor -- start 2024-05-09T00:01:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.111+0000 7fbf25ffb700 1 -- 172.21.15.12:0/4108246850 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbf18014070 con 0x7fbf28100030 2024-05-09T00:01:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.112+0000 7fbf1ffff700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf140a56a0 0x7fbf140a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.112+0000 7fbf3a801b80 1 -- start start 2024-05-09T00:01:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.112+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997d950 0x55889997dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.112+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55889997e270 0x558899977f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.112+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558899978480 0x5588999788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.112+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588998cf0f0 con 0x558899978480 2024-05-09T00:01:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.112+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588998ccab0 con 0x55889997d950 2024-05-09T00:01:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.112+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588998c83a0 con 0x55889997e270 2024-05-09T00:01:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf27fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997d950 0x55889997dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf2cbe4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558899978480 0x5588999788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf2cbe4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558899978480 0x5588999788a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55364/0 (socket says 172.21.15.12:55364) 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/978151092 learned_addr learned my addr 172.21.15.12:0/978151092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf27fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997d950 0x55889997dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47096/0 (socket says 172.21.15.12:47096) 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf1ffff700 1 --2- 172.21.15.12:0/978151092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55889997e270 0x558899977f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/978151092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55889997e270 msgr2=0x558899977f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/978151092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55889997e270 0x558899977f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/978151092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997d950 msgr2=0x55889997dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/978151092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997d950 0x55889997dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/978151092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558899749350 con 0x558899978480 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf27fff700 1 --2- 172.21.15.12:0/978151092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997d950 0x55889997dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.113+0000 7fbf1ffff700 1 --2- 172.21.15.12:0/978151092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55889997e270 0x558899977f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.114+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/978151092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558899978480 0x5588999788a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fbf20007990 tx=0x7fbf200079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0824524ef17940 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.114+0000 7fbf1e7fc700 1 -- 172.21.15.12:0/978151092 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2000e040 con 0x558899978480 2024-05-09T00:01:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.114+0000 7fbf1ffff700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf140a56a0 0x7fbf140a7b60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x558899973790 tx=0x7fbf100096d0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.114+0000 7fbf1e7fc700 1 -- 172.21.15.12:0/978151092 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2000ba50 con 0x558899978480 2024-05-09T00:01:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.114+0000 7fbf1e7fc700 1 -- 172.21.15.12:0/978151092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2001c440 con 0x558899978480 2024-05-09T00:01:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.115+0000 7fbf3a801b80 1 -- 172.21.15.12:0/978151092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558899978480 msgr2=0x5588999788a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.115+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/978151092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558899978480 0x5588999788a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fbf20007990 tx=0x7fbf200079c0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.115+0000 7fbf3a801b80 1 -- 172.21.15.12:0/978151092 shutdown_connections 2024-05-09T00:01:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.115+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/978151092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558899978480 0x5588999788a0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.115+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/978151092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997d950 0x55889997dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.115+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/978151092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55889997e270 0x558899977f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.115+0000 7fbf3a801b80 1 -- 172.21.15.12:0/978151092 >> 172.21.15.12:0/978151092 conn(0x5588998d61b0 msgr2=0x558899972840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.115+0000 7fbf3a801b80 1 -- 172.21.15.12:0/978151092 shutdown_connections 2024-05-09T00:01:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.115+0000 7fbf3a801b80 1 -- 172.21.15.12:0/978151092 wait complete. 2024-05-09T00:01:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.116+0000 7fbf3a801b80 1 -- start start 2024-05-09T00:01:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.116+0000 7fbf3a801b80 1 Processor -- start 2024-05-09T00:01:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558899978480 0x5588999841a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55889997d950 0x558899981490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997e270 0x558899981aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558899864500 con 0x55889997e270 2024-05-09T00:01:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588998641a0 con 0x558899978480 2024-05-09T00:01:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55889974b640 con 0x55889997d950 2024-05-09T00:01:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf3a801b80 10 client.0 _ll_register_callbacks cb 0x55889986dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf2cbe4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997e270 0x558899981aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf2cbe4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997e270 0x558899981aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55380/0 (socket says 172.21.15.12:55380) 2024-05-09T00:01:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/1534216114 learned_addr learned my addr 172.21.15.12:0/1534216114 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.118+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/1534216114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55889997d950 msgr2=0x558899981490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:01:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.119+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55889997d950 0x558899981490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.119+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/1534216114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558899978480 msgr2=0x5588999841a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.119+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558899978480 0x5588999841a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.119+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/1534216114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf10014410 con 0x55889997e270 2024-05-09T00:01:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.119+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997e270 0x558899981aa0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fbf20004ba0 tx=0x7fbf20007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669d88de4694dbac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.119+0000 7fbf077fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:01:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.119+0000 7fbf077fe700 1 -- 172.21.15.12:0/1534216114 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2000e040 con 0x55889997e270 2024-05-09T00:01:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.119+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1534216114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558899821210 con 0x55889997e270 2024-05-09T00:01:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.119+0000 7fbf077fe700 1 -- 172.21.15.12:0/1534216114 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbf20003c10 con 0x55889997e270 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf077fe700 1 -- 172.21.15.12:0/1534216114 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf20003dd0 con 0x55889997e270 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf3a801b80 10 client.19728 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1534216114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7fbf2810d540 con 0x55889997e270 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf077fe700 1 -- 172.21.15.12:0/1534216114 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558899821210 con 0x55889997e270 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf077fe700 1 -- 172.21.15.12:0/1534216114 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbf20019050 con 0x55889997e270 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf3a801b80 10 client.19728 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf3a801b80 10 client.19728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf3a801b80 20 client.19728 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588999a2760 0x5588999a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf3a801b80 4 client.19728 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1534216114 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588999477b0 con 0x5588999a2760 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.120+0000 7fbf27fff700 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588999a2760 0x5588999a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.121+0000 7fbf27fff700 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588999a2760 0x5588999a4b80 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1bd8ce03fe40bd8 server_cookie=f134f25e5158345f in_seq=0 out_seq=0 2024-05-09T00:01:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.121+0000 7fbf077fe700 10 client.19728 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.124+0000 7fbf077fe700 1 -- 172.21.15.12:0/1534216114 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588999477b0 con 0x5588999a2760 2024-05-09T00:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.124+0000 7fbf077fe700 10 client.19728 handle_command_reply: tid=0 2024-05-09T00:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.124+0000 7fbf3a801b80 1 client.19728 shutdown 2024-05-09T00:01:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1534216114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997e270 msgr2=0x558899981aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997e270 0x558899981aa0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fbf20004ba0 tx=0x7fbf20007da0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1534216114 shutdown_connections 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997e270 0x558899981aa0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5588999a2760 0x5588999a4b80 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558899978480 0x5588999841a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1534216114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55889997d950 0x558899981490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1534216114 >> 172.21.15.12:0/1534216114 conn(0x5588998d61b0 msgr2=0x55889997c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf077fe700 0 client.19728 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1534216114 shutdown_connections 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1534216114 wait complete. 2024-05-09T00:01:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.125+0000 7fbf3a801b80 20 client.19728 trim_cache size 0 max 16384 2024-05-09T00:01:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.145+0000 7fbf3a801b80 1 Processor -- start 2024-05-09T00:01:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.145+0000 7fbf3a801b80 1 -- start start 2024-05-09T00:01:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.145+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 0x5588999819e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.145+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 0x558899972910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.145+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 0x558899978860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.145+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55889996f9b0 con 0x55889997d950 2024-05-09T00:01:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.145+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55889999f690 con 0x55889997e270 2024-05-09T00:01:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.145+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588999735f0 con 0x558899978480 2024-05-09T00:01:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf1ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 0x558899972910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf27fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 0x5588999819e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf1ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 0x558899972910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55394/0 (socket says 172.21.15.12:55394) 2024-05-09T00:01:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf1ffff700 1 -- 172.21.15.12:0/1721641934 learned_addr learned my addr 172.21.15.12:0/1721641934 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf27fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 0x5588999819e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48554/0 (socket says 172.21.15.12:48554) 2024-05-09T00:01:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/1721641934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 0x558899978860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf1ffff700 1 -- 172.21.15.12:0/1721641934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 msgr2=0x5588999819e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf1ffff700 1 --2- 172.21.15.12:0/1721641934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 0x5588999819e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf1ffff700 1 -- 172.21.15.12:0/1721641934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 msgr2=0x558899978860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf1ffff700 1 --2- 172.21.15.12:0/1721641934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 0x558899978860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf1ffff700 1 -- 172.21.15.12:0/1721641934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588996f5b00 con 0x55889997d950 2024-05-09T00:01:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.146+0000 7fbf1ffff700 1 --2- 172.21.15.12:0/1721641934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 0x558899972910 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005e90 tx=0x7fbf100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab59aec66c89a92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.147+0000 7fbf1effd700 1 -- 172.21.15.12:0/1721641934 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10003880 con 0x55889997d950 2024-05-09T00:01:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.147+0000 7fbf1effd700 1 -- 172.21.15.12:0/1721641934 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf10020420 con 0x55889997d950 2024-05-09T00:01:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.147+0000 7fbf1effd700 1 -- 172.21.15.12:0/1721641934 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10003ea0 con 0x55889997d950 2024-05-09T00:01:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.148+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1721641934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 msgr2=0x558899972910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.148+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1721641934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 0x558899972910 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005e90 tx=0x7fbf100047c0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.149+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1721641934 shutdown_connections 2024-05-09T00:01:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.149+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1721641934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 0x558899978860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.149+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1721641934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 0x5588999819e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.149+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1721641934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 0x558899972910 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.149+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1721641934 >> 172.21.15.12:0/1721641934 conn(0x5588998d61b0 msgr2=0x5588999a94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.149+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1721641934 shutdown_connections 2024-05-09T00:01:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.149+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1721641934 wait complete. 2024-05-09T00:01:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.149+0000 7fbf3a801b80 1 -- start start 2024-05-09T00:01:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.150+0000 7fbf3a801b80 1 Processor -- start 2024-05-09T00:01:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 0x558899988e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 0x558899986bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf3a801b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 0x558899987130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558899987670 con 0x55889997d950 2024-05-09T00:01:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588999877e0 con 0x55889997e270 2024-05-09T00:01:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf3a801b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588999879a0 con 0x558899978480 2024-05-09T00:01:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf3a801b80 10 client.0 _ll_register_callbacks cb 0x55889986dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf2cbe4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 0x558899987130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf27fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 0x558899988e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf2cbe4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 0x558899987130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47134/0 (socket says 172.21.15.12:47134) 2024-05-09T00:01:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/1478535867 learned_addr learned my addr 172.21.15.12:0/1478535867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf1ffff700 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 0x558899986bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/1478535867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 msgr2=0x558899988e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.152+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 0x558899988e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/1478535867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 msgr2=0x558899986bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 0x558899986bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf2cbe4700 1 -- 172.21.15.12:0/1478535867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf100140b0 con 0x55889997e270 2024-05-09T00:01:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf2cbe4700 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 0x558899987130 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009fd0 tx=0x7fbf20009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5d111b28a87295 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf077fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:01:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf077fe700 1 -- 172.21.15.12:0/1478535867 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2000e070 con 0x55889997e270 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1478535867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf2000ba50 con 0x55889997e270 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf077fe700 1 -- 172.21.15.12:0/1478535867 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbf20038990 con 0x55889997e270 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf077fe700 1 -- 172.21.15.12:0/1478535867 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf20004bd0 con 0x55889997e270 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf3a801b80 10 client.29432 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1478535867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55889988e500 con 0x55889997e270 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.153+0000 7fbf077fe700 1 -- 172.21.15.12:0/1478535867 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2000ba50 con 0x55889997e270 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.154+0000 7fbf077fe700 1 -- 172.21.15.12:0/1478535867 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbf2000b0d0 con 0x55889997e270 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.154+0000 7fbf3a801b80 10 client.29432 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.154+0000 7fbf3a801b80 10 client.29432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.154+0000 7fbf3a801b80 20 client.29432 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.154+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558899a015c0 0x558899a039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.154+0000 7fbf3a801b80 4 client.29432 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.154+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1478535867 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588998c0e60 con 0x558899a015c0 2024-05-09T00:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.154+0000 7fbf27fff700 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558899a015c0 0x558899a039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.155+0000 7fbf27fff700 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558899a015c0 0x558899a039e0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17469d75dd2a202 server_cookie=447a7bf878d94c2c in_seq=0 out_seq=0 2024-05-09T00:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.155+0000 7fbf077fe700 10 client.29432 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:33.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 62807 40.47 MB/sec warmup 88 sec latency 81.992 ms 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.390+0000 7fbf077fe700 1 -- 172.21.15.12:0/1478535867 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588998c0e60 con 0x558899a015c0 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.390+0000 7fbf077fe700 10 client.29432 handle_command_reply: tid=0 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.390+0000 7fbf3a801b80 1 client.29432 shutdown 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1478535867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 msgr2=0x558899987130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 0x558899987130 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009fd0 tx=0x7fbf20009380 comp rx=0 tx=0).stop 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1478535867 shutdown_connections 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558899a015c0 0x558899a039e0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55889997e270 0x558899987130 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558899978480 0x558899988e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 1 --2- 172.21.15.12:0/1478535867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55889997d950 0x558899986bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1478535867 >> 172.21.15.12:0/1478535867 conn(0x5588998d61b0 msgr2=0x558899976510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf077fe700 0 client.29432 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1478535867 shutdown_connections 2024-05-09T00:01:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 1 -- 172.21.15.12:0/1478535867 wait complete. 2024-05-09T00:01:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.391+0000 7fbf3a801b80 20 client.29432 trim_cache size 0 max 16384 2024-05-09T00:01:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:01:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:01:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:01:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 -- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf140a56a0 msgr2=0x7fbf140a7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf140a56a0 0x7fbf140a7b60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x558899973790 tx=0x7fbf100096d0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 -- 172.21.15.12:0/4108246850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf28100030 msgr2=0x7fbf28197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf28100030 0x7fbf28197530 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800e9f0 tx=0x7fbf1800ed00 comp rx=0 tx=0).stop 2024-05-09T00:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 -- 172.21.15.12:0/4108246850 shutdown_connections 2024-05-09T00:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf140a56a0 0x7fbf140a7b60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2819be10 0x7fbf2819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf28100030 0x7fbf28197530 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 --2- 172.21.15.12:0/4108246850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf28105150 0x7fbf28197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 -- 172.21.15.12:0/4108246850 >> 172.21.15.12:0/4108246850 conn(0x7fbf28076f70 msgr2=0x7fbf28069410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 -- 172.21.15.12:0/4108246850 shutdown_connections 2024-05-09T00:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:33.392+0000 7fbf2e647700 1 -- 172.21.15.12:0/4108246850 wait complete. 2024-05-09T00:01:33.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:33 smithi012 ceph-mon[42503]: pgmap v485: 97 pgs: 97 active+clean; 864 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 20 MiB/s wr, 780 op/s 2024-05-09T00:01:34.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:33 smithi204 ceph-mon[47373]: pgmap v485: 97 pgs: 97 active+clean; 864 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 20 MiB/s wr, 780 op/s 2024-05-09T00:01:34.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:33 smithi160 ceph-mon[46152]: pgmap v485: 97 pgs: 97 active+clean; 864 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 20 MiB/s wr, 780 op/s 2024-05-09T00:01:34.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 63557 40.44 MB/sec warmup 89 sec latency 112.234 ms 2024-05-09T00:01:35.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 64737 40.64 MB/sec warmup 90 sec latency 44.916 ms 2024-05-09T00:01:35.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:35 smithi012 ceph-mon[42503]: pgmap v486: 97 pgs: 97 active+clean; 864 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 20 MiB/s wr, 780 op/s 2024-05-09T00:01:36.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:35 smithi204 ceph-mon[47373]: pgmap v486: 97 pgs: 97 active+clean; 864 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 20 MiB/s wr, 780 op/s 2024-05-09T00:01:36.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:35 smithi160 ceph-mon[46152]: pgmap v486: 97 pgs: 97 active+clean; 864 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 20 MiB/s wr, 780 op/s 2024-05-09T00:01:36.222 INFO:tasks.workunit.client.0.smithi012.stdout: 10 65617 40.84 MB/sec warmup 91 sec latency 39.296 ms 2024-05-09T00:01:37.223 INFO:tasks.workunit.client.0.smithi012.stdout: 10 66348 40.80 MB/sec warmup 92 sec latency 51.525 ms 2024-05-09T00:01:37.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:37 smithi012 ceph-mon[42503]: pgmap v487: 97 pgs: 97 active+clean; 696 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 23 MiB/s wr, 1.29k op/s 2024-05-09T00:01:38.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:37 smithi204 ceph-mon[47373]: pgmap v487: 97 pgs: 97 active+clean; 696 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 23 MiB/s wr, 1.29k op/s 2024-05-09T00:01:38.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:37 smithi160 ceph-mon[46152]: pgmap v487: 97 pgs: 97 active+clean; 696 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 23 MiB/s wr, 1.29k op/s 2024-05-09T00:01:38.223 INFO:tasks.workunit.client.0.smithi012.stdout: 10 67272 40.85 MB/sec warmup 93 sec latency 77.551 ms 2024-05-09T00:01:39.223 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68285 40.94 MB/sec warmup 94 sec latency 66.039 ms 2024-05-09T00:01:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:39 smithi012 ceph-mon[42503]: pgmap v488: 97 pgs: 97 active+clean; 696 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 20 MiB/s wr, 965 op/s 2024-05-09T00:01:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:39 smithi160 ceph-mon[46152]: pgmap v488: 97 pgs: 97 active+clean; 696 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 20 MiB/s wr, 965 op/s 2024-05-09T00:01:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:39 smithi204 ceph-mon[47373]: pgmap v488: 97 pgs: 97 active+clean; 696 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 20 MiB/s wr, 965 op/s 2024-05-09T00:01:40.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69256 41.15 MB/sec warmup 95 sec latency 39.301 ms 2024-05-09T00:01:41.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 70147 41.23 MB/sec warmup 96 sec latency 37.546 ms 2024-05-09T00:01:41.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:41 smithi012 ceph-mon[42503]: pgmap v489: 97 pgs: 97 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 23 MiB/s wr, 1.22k op/s 2024-05-09T00:01:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:41 smithi204 ceph-mon[47373]: pgmap v489: 97 pgs: 97 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 23 MiB/s wr, 1.22k op/s 2024-05-09T00:01:42.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:41 smithi160 ceph-mon[46152]: pgmap v489: 97 pgs: 97 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 23 MiB/s wr, 1.22k op/s 2024-05-09T00:01:42.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 70908 41.24 MB/sec warmup 97 sec latency 52.421 ms 2024-05-09T00:01:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:42 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:01:43.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:42 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:01:43.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:42 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:01:43.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 71755 41.20 MB/sec warmup 98 sec latency 34.739 ms 2024-05-09T00:01:43.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:43 smithi012 ceph-mon[42503]: pgmap v490: 97 pgs: 97 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 765 op/s 2024-05-09T00:01:44.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:43 smithi204 ceph-mon[47373]: pgmap v490: 97 pgs: 97 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 765 op/s 2024-05-09T00:01:44.075 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:43 smithi160 ceph-mon[46152]: pgmap v490: 97 pgs: 97 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 765 op/s 2024-05-09T00:01:44.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 72712 41.27 MB/sec warmup 99 sec latency 41.186 ms 2024-05-09T00:01:45.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 73655 41.47 MB/sec warmup 100 sec latency 71.151 ms 2024-05-09T00:01:45.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:45 smithi012 ceph-mon[42503]: pgmap v491: 97 pgs: 97 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 765 op/s 2024-05-09T00:01:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:45 smithi204 ceph-mon[47373]: pgmap v491: 97 pgs: 97 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 765 op/s 2024-05-09T00:01:46.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:45 smithi160 ceph-mon[46152]: pgmap v491: 97 pgs: 97 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 765 op/s 2024-05-09T00:01:46.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 74406 41.40 MB/sec warmup 101 sec latency 92.293 ms 2024-05-09T00:01:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.557+0000 7fd8e0506700 1 -- 172.21.15.12:0/81362213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8d8104d90 msgr2=0x7fd8d8105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.557+0000 7fd8e0506700 1 --2- 172.21.15.12:0/81362213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8d8104d90 0x7fd8d8105170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40099c0 tx=0x7fd8d4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.557+0000 7fd8e0506700 1 -- 172.21.15.12:0/81362213 shutdown_connections 2024-05-09T00:01:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.557+0000 7fd8e0506700 1 --2- 172.21.15.12:0/81362213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8d8100ef0 0x7fd8d810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.557+0000 7fd8e0506700 1 --2- 172.21.15.12:0/81362213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8d8104d90 0x7fd8d8105170 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.557+0000 7fd8e0506700 1 --2- 172.21.15.12:0/81362213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d8100530 0x7fd8d81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.557+0000 7fd8e0506700 1 -- 172.21.15.12:0/81362213 >> 172.21.15.12:0/81362213 conn(0x7fd8d8076380 msgr2=0x7fd8d80fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.557+0000 7fd8e0506700 1 -- 172.21.15.12:0/81362213 shutdown_connections 2024-05-09T00:01:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.557+0000 7fd8e0506700 1 -- 172.21.15.12:0/81362213 wait complete. 2024-05-09T00:01:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8e0506700 1 Processor -- start 2024-05-09T00:01:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8e0506700 1 -- start start 2024-05-09T00:01:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8e0506700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8d8100530 0x7fd8d8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8e0506700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8d8100ef0 0x7fd8d8199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8e0506700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d8104d90 0x7fd8d819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8e0506700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d819e4a0 con 0x7fd8d8100ef0 2024-05-09T00:01:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8e0506700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d819e610 con 0x7fd8d8104d90 2024-05-09T00:01:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8e0506700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d819e7d0 con 0x7fd8d8100530 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8de2a2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8d8100530 0x7fd8d8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8deaa3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d8104d90 0x7fd8d819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8ddaa1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8d8100ef0 0x7fd8d8199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8deaa3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d8104d90 0x7fd8d819de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:36058/0 (socket says 172.21.15.12:36058) 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.558+0000 7fd8deaa3700 1 -- 172.21.15.12:0/2409644392 learned_addr learned my addr 172.21.15.12:0/2409644392 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/2409644392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8d8100530 msgr2=0x7fd8d8199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8d8100530 0x7fd8d8199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/2409644392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d8104d90 msgr2=0x7fd8d819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d8104d90 0x7fd8d819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/2409644392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d40095f0 con 0x7fd8d8100ef0 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8deaa3700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d8104d90 0x7fd8d819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8d8100ef0 0x7fd8d8199ac0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800c970 tx=0x7fd8c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8cf7fe700 1 -- 172.21.15.12:0/2409644392 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8004440 con 0x7fd8d8100ef0 2024-05-09T00:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8cf7fe700 1 -- 172.21.15.12:0/2409644392 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8c8003b40 con 0x7fd8d8100ef0 2024-05-09T00:01:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8cf7fe700 1 -- 172.21.15.12:0/2409644392 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8007bc0 con 0x7fd8d8100ef0 2024-05-09T00:01:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8e0506700 1 -- 172.21.15.12:0/2409644392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d819eab0 con 0x7fd8d8100ef0 2024-05-09T00:01:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.559+0000 7fd8e0506700 1 -- 172.21.15.12:0/2409644392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d81a62f0 con 0x7fd8d8100ef0 2024-05-09T00:01:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.563+0000 7fd8cf7fe700 1 -- 172.21.15.12:0/2409644392 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd8c8003cb0 con 0x7fd8d8100ef0 2024-05-09T00:01:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.564+0000 7fd8cf7fe700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8c40a1310 0x7fd8c40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.565+0000 7fd8cf7fe700 1 -- 172.21.15.12:0/2409644392 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd8c80ccc50 con 0x7fd8d8100ef0 2024-05-09T00:01:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.565+0000 7fd8ec6c0b80 1 Processor -- start 2024-05-09T00:01:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.565+0000 7fd8ec6c0b80 1 -- start start 2024-05-09T00:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x56427388fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273889f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 0x56427388a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642737e10f0 con 0x56427388a480 2024-05-09T00:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642737deab0 con 0x564273890270 2024-05-09T00:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642737da3a0 con 0x56427388f950 2024-05-09T00:01:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8de2a2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x56427388fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ddaa1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273889f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8deaa3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 0x56427388a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ddaa1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273889f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:36062/0 (socket says 172.21.15.12:36062) 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/1990360262 learned_addr learned my addr 172.21.15.12:0/1990360262 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8de2a2700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8c40a1310 0x7fd8c40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/1990360262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 msgr2=0x56427388fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/1990360262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x56427388fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/1990360262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 msgr2=0x56427388a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/1990360262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 0x56427388a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/1990360262 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56427365b350 con 0x564273890270 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.566+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/1990360262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273889f40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8096780 tx=0x7fd8c8099670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3717f5122f560aa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.567+0000 7fd8de2a2700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8c40a1310 0x7fd8c40a37d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009990 tx=0x7fd8d4000bc0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.567+0000 7fd8ccff9700 1 -- 172.21.15.12:0/1990360262 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8099d10 con 0x564273890270 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.567+0000 7fd8ccff9700 1 -- 172.21.15.12:0/1990360262 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8c809db40 con 0x564273890270 2024-05-09T00:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.567+0000 7fd8ccff9700 1 -- 172.21.15.12:0/1990360262 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c809a1d0 con 0x564273890270 2024-05-09T00:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.568+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1990360262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 msgr2=0x564273889f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.568+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/1990360262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273889f40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8096780 tx=0x7fd8c8099670 comp rx=0 tx=0).stop 2024-05-09T00:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.568+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1990360262 shutdown_connections 2024-05-09T00:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.568+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/1990360262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 0x56427388a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.568+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/1990360262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x56427388fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.568+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/1990360262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273889f40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.568+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1990360262 >> 172.21.15.12:0/1990360262 conn(0x5642737e81b0 msgr2=0x564273884840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.568+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1990360262 shutdown_connections 2024-05-09T00:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.568+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1990360262 wait complete. 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.568+0000 7fd8ec6c0b80 1 -- start start 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ec6c0b80 1 Processor -- start 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388a480 0x564273896260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273893550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 0x564273893c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564273776500 con 0x564273890270 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642737761a0 con 0x56427388a480 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427365d640 con 0x56427388f950 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ec6c0b80 10 client.0 _ll_register_callbacks cb 0x56427377fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ddaa1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273893550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ddaa1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273893550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54130/0 (socket says 172.21.15.12:54130) 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/1162841151 learned_addr learned my addr 172.21.15.12:0/1162841151 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8de2a2700 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388a480 0x564273896260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.569+0000 7fd8deaa3700 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 0x564273893c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/1162841151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388a480 msgr2=0x564273896260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388a480 0x564273896260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/1162841151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 msgr2=0x564273893c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 0x564273893c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/1162841151 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d81a62f0 con 0x56427388f950 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8de2a2700 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388a480 0x564273896260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273893550 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8099430 tx=0x7fd8c8099510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5def431f96842e34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/1162841151 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8099d10 con 0x56427388f950 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1162841151 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564273733210 con 0x56427388f950 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/1162841151 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd8c80a2190 con 0x56427388f950 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/1162841151 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c80addf0 con 0x56427388f950 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8ec6c0b80 10 client.29476 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.570+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1162841151 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x564273607b00 con 0x56427388f950 2024-05-09T00:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.571+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/1162841151 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8c807c080 con 0x56427388f950 2024-05-09T00:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.571+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/1162841151 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd8c8065020 con 0x56427388f950 2024-05-09T00:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.571+0000 7fd8ec6c0b80 10 client.29476 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.571+0000 7fd8ec6c0b80 10 client.29476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.571+0000 7fd8ec6c0b80 20 client.29476 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.571+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642738a3970 0x5642738a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.571+0000 7fd8ec6c0b80 4 client.29476 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.571+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1162841151 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642738597b0 con 0x5642738a3970 2024-05-09T00:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.572+0000 7fd8de2a2700 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642738a3970 0x5642738a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.572+0000 7fd8de2a2700 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642738a3970 0x5642738a5d90 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969d94e0acbe9214 server_cookie=1aef05e7991dca58 in_seq=0 out_seq=0 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.572+0000 7fd8c1ffb700 10 client.29476 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/1162841151 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642738597b0 con 0x5642738a3970 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8c1ffb700 10 client.29476 handle_command_reply: tid=0 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8ec6c0b80 1 client.29476 shutdown 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1162841151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 msgr2=0x564273893550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273893550 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8099430 tx=0x7fd8c8099510 comp rx=0 tx=0).stop 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1162841151 shutdown_connections 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 0x564273893c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642738a3970 0x5642738a5d90 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388a480 0x564273896260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/1162841151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273893550 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1162841151 >> 172.21.15.12:0/1162841151 conn(0x5642737e81b0 msgr2=0x56427388e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.575+0000 7fd8c1ffb700 0 client.29476 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.576+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1162841151 shutdown_connections 2024-05-09T00:01:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.576+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/1162841151 wait complete. 2024-05-09T00:01:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.576+0000 7fd8ec6c0b80 20 client.29476 trim_cache size 0 max 16384 2024-05-09T00:01:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.598+0000 7fd8ec6c0b80 1 Processor -- start 2024-05-09T00:01:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.598+0000 7fd8ec6c0b80 1 -- start start 2024-05-09T00:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.598+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388a480 0x564273893c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.598+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388f950 0x564273893510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.598+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 0x56427389f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.598+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564273889140 con 0x564273890270 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.598+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427388ff80 con 0x56427388f950 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.598+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564273887f30 con 0x56427388a480 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8deaa3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 0x56427389f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8deaa3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 0x56427389f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40034/0 (socket says 172.21.15.12:40034) 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8ddaa1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388f950 0x564273893510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8deaa3700 1 -- 172.21.15.12:0/2210547420 learned_addr learned my addr 172.21.15.12:0/2210547420 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/2210547420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388a480 msgr2=0x564273893c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/2210547420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388a480 0x564273893c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/2210547420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 msgr2=0x56427389f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/2210547420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 0x56427389f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8ddaa1700 1 -- 172.21.15.12:0/2210547420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642737a0500 con 0x56427388f950 2024-05-09T00:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8deaa3700 1 --2- 172.21.15.12:0/2210547420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 0x56427389f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.599+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/2210547420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388f950 0x564273893510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80a1fd0 tx=0x7fd8c800ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4174c502ac9283f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.601+0000 7fd8cd7fa700 1 -- 172.21.15.12:0/2210547420 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8099670 con 0x56427388f950 2024-05-09T00:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.601+0000 7fd8cd7fa700 1 -- 172.21.15.12:0/2210547420 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564273733210 con 0x56427388f950 2024-05-09T00:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.601+0000 7fd8cd7fa700 1 -- 172.21.15.12:0/2210547420 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8095b00 con 0x56427388f950 2024-05-09T00:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.602+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/2210547420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388f950 msgr2=0x564273893510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.602+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/2210547420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388f950 0x564273893510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80a1fd0 tx=0x7fd8c800ec30 comp rx=0 tx=0).stop 2024-05-09T00:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.602+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/2210547420 shutdown_connections 2024-05-09T00:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.602+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/2210547420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564273890270 0x56427389f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.602+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/2210547420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388a480 0x564273893c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.602+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/2210547420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56427388f950 0x564273893510 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.602+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/2210547420 >> 172.21.15.12:0/2210547420 conn(0x5642737e81b0 msgr2=0x564273886320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.602+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/2210547420 shutdown_connections 2024-05-09T00:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.602+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/2210547420 wait complete. 2024-05-09T00:01:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8ec6c0b80 1 -- start start 2024-05-09T00:01:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8ec6c0b80 1 Processor -- start 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 0x5642738926e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273892c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8ec6c0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273893160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642738f7e50 con 0x56427388a480 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642738f7fc0 con 0x564273890270 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8ec6c0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642738f8180 con 0x56427388f950 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8ec6c0b80 10 client.0 _ll_register_callbacks cb 0x56427377fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8deaa3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273893160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.604+0000 7fd8de2a2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 0x5642738926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.603+0000 7fd8ddaa1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273892c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.604+0000 7fd8deaa3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273893160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:36082/0 (socket says 172.21.15.12:36082) 2024-05-09T00:01:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.604+0000 7fd8deaa3700 1 -- 172.21.15.12:0/4161876844 learned_addr learned my addr 172.21.15.12:0/4161876844 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.604+0000 7fd8ddaa1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273892c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54150/0 (socket says 172.21.15.12:54150) 2024-05-09T00:01:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.604+0000 7fd8de2a2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 0x5642738926e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40048/0 (socket says 172.21.15.12:40048) 2024-05-09T00:01:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.604+0000 7fd8deaa3700 1 -- 172.21.15.12:0/4161876844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 msgr2=0x564273892c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.604+0000 7fd8deaa3700 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273892c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.604+0000 7fd8deaa3700 1 -- 172.21.15.12:0/4161876844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 msgr2=0x5642738926e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.604+0000 7fd8deaa3700 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 0x5642738926e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.604+0000 7fd8deaa3700 1 -- 172.21.15.12:0/4161876844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564273607b00 con 0x564273890270 2024-05-09T00:01:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.605+0000 7fd8deaa3700 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273893160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0007f20 tx=0x7fd8d0016620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b48df458d514f31f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.606+0000 7fd8c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:01:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.606+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/4161876844 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0004bb0 con 0x564273890270 2024-05-09T00:01:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.606+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/4161876844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8c809db40 con 0x564273890270 2024-05-09T00:01:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.606+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/4161876844 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd8d0004d10 con 0x564273890270 2024-05-09T00:01:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.606+0000 7fd8ddaa1700 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273892c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:01:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.607+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/4161876844 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8d0005020 con 0x564273890270 2024-05-09T00:01:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.607+0000 7fd8ec6c0b80 10 client.29447 fetch_fsmap learned FSMap version 61 2024-05-09T00:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.607+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/4161876844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56427365fc70 con 0x564273890270 2024-05-09T00:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.607+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/4161876844 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8c809db40 con 0x564273890270 2024-05-09T00:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.608+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/4161876844 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 61) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd8d0014050 con 0x564273890270 2024-05-09T00:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.608+0000 7fd8ec6c0b80 10 client.29447 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T00:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.608+0000 7fd8ec6c0b80 10 client.29447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.608+0000 7fd8ec6c0b80 20 client.29447 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.608+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642738f9980 0x5642738fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.608+0000 7fd8ec6c0b80 4 client.29447 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.608+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/4161876844 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5642737d2e60 con 0x5642738f9980 2024-05-09T00:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.608+0000 7fd8de2a2700 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642738f9980 0x5642738fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.608+0000 7fd8de2a2700 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642738f9980 0x5642738fbda0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83c70e32f3fd66f server_cookie=3c7424d0686a1714 in_seq=0 out_seq=0 2024-05-09T00:01:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.608+0000 7fd8c1ffb700 10 client.29447 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8c1ffb700 1 -- 172.21.15.12:0/4161876844 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5642737d2e60 con 0x5642738f9980 2024-05-09T00:01:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8c1ffb700 10 client.29447 handle_command_reply: tid=0 2024-05-09T00:01:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8ec6c0b80 1 client.29447 shutdown 2024-05-09T00:01:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/4161876844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 msgr2=0x564273893160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273893160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0007f20 tx=0x7fd8d0016620 comp rx=0 tx=0).stop 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/4161876844 shutdown_connections 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5642738f9980 0x5642738fbda0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564273890270 0x564273893160 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56427388a480 0x5642738926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8ec6c0b80 1 --2- 172.21.15.12:0/4161876844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56427388f950 0x564273892c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/4161876844 >> 172.21.15.12:0/4161876844 conn(0x5642737e81b0 msgr2=0x5642738885f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.885+0000 7fd8c1ffb700 0 client.29447 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.886+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/4161876844 shutdown_connections 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.886+0000 7fd8ec6c0b80 1 -- 172.21.15.12:0/4161876844 wait complete. 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.886+0000 7fd8ec6c0b80 20 client.29447 trim_cache size 0 max 16384 2024-05-09T00:01:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:01:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:01:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:01:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 -- 172.21.15.12:0/2409644392 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8c40a1310 msgr2=0x7fd8c40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8c40a1310 0x7fd8c40a37d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009990 tx=0x7fd8d4000bc0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 -- 172.21.15.12:0/2409644392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8d8100ef0 msgr2=0x7fd8d8199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8d8100ef0 0x7fd8d8199ac0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800c970 tx=0x7fd8c800cc80 comp rx=0 tx=0).stop 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 -- 172.21.15.12:0/2409644392 shutdown_connections 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8c40a1310 0x7fd8c40a37d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d8104d90 0x7fd8d819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8d8100530 0x7fd8d8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 --2- 172.21.15.12:0/2409644392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8d8100ef0 0x7fd8d8199ac0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 -- 172.21.15.12:0/2409644392 >> 172.21.15.12:0/2409644392 conn(0x7fd8d8076380 msgr2=0x7fd8d80ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 -- 172.21.15.12:0/2409644392 shutdown_connections 2024-05-09T00:01:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:01:46.887+0000 7fd8e0506700 1 -- 172.21.15.12:0/2409644392 wait complete. 2024-05-09T00:01:47.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 75184 41.39 MB/sec warmup 102 sec latency 160.220 ms 2024-05-09T00:01:47.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:01:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-09T00:01:47.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:01:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-09T00:01:47.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:01:47 smithi204 ceph-mds[88022]: -- 172.21.15.204:0/3131612754 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bc64ae21a0 con 0x55bc64ab8c00 2024-05-09T00:01:47.262 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:01:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-09T00:01:47.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:47 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-09T00:01:47.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:47 smithi204 ceph-mon[47373]: osdmap e113: 12 total, 12 up, 12 in 2024-05-09T00:01:47.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:47 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-09T00:01:47.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:47 smithi012 ceph-mon[42503]: osdmap e113: 12 total, 12 up, 12 in 2024-05-09T00:01:47.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:47 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-09T00:01:47.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:47 smithi160 ceph-mon[46152]: osdmap e113: 12 total, 12 up, 12 in 2024-05-09T00:01:48.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 76216 41.58 MB/sec warmup 103 sec latency 49.949 ms 2024-05-09T00:01:48.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:48 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/3146138040,v1:172.21.15.204:6837/3146138040] up:boot 2024-05-09T00:01:48.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:48 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:01:48.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:48 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:01:48.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:48 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:01:48.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:48 smithi012 ceph-mon[42503]: pgmap v493: 97 pgs: 97 active+clean; 757 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 611 op/s 2024-05-09T00:01:48.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:48 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/3146138040,v1:172.21.15.204:6837/3146138040] up:boot 2024-05-09T00:01:48.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:48 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:01:48.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:48 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:01:48.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:48 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:01:48.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:48 smithi204 ceph-mon[47373]: pgmap v493: 97 pgs: 97 active+clean; 757 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 611 op/s 2024-05-09T00:01:48.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:48 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/3146138040,v1:172.21.15.204:6837/3146138040] up:boot 2024-05-09T00:01:48.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:48 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:01:48.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:48 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:01:48.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:48 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:01:48.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:48 smithi160 ceph-mon[46152]: pgmap v493: 97 pgs: 97 active+clean; 757 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 611 op/s 2024-05-09T00:01:49.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 76972 41.58 MB/sec warmup 104 sec latency 38.353 ms 2024-05-09T00:01:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:49 smithi012 ceph-mon[42503]: pgmap v494: 97 pgs: 97 active+clean; 757 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 611 op/s 2024-05-09T00:01:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:49 smithi204 ceph-mon[47373]: pgmap v494: 97 pgs: 97 active+clean; 757 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 611 op/s 2024-05-09T00:01:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:49 smithi160 ceph-mon[46152]: pgmap v494: 97 pgs: 97 active+clean; 757 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 611 op/s 2024-05-09T00:01:50.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 77461 41.44 MB/sec warmup 105 sec latency 320.201 ms 2024-05-09T00:01:50.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:50 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:01:50.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:50 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:01:50.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:50 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:01:51.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:50 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:01:51.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:50 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:01:51.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:50 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:01:51.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:50 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:01:51.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:50 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:01:51.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:50 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:01:51.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 78216 41.34 MB/sec warmup 106 sec latency 66.416 ms 2024-05-09T00:01:51.765 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:01:51.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:51 smithi012 ceph-mon[42503]: pgmap v495: 97 pgs: 97 active+clean; 770 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-09T00:01:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.946+0000 7ff8f955b700 1 -- 172.21.15.12:0/3629455387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f4100050 msgr2=0x7ff8f4100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.946+0000 7ff8f955b700 1 --2- 172.21.15.12:0/3629455387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f4100050 0x7ff8f4100430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40098f0 tx=0x7ff8e4009c00 comp rx=0 tx=0).stop 2024-05-09T00:01:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.946+0000 7ff8f955b700 1 -- 172.21.15.12:0/3629455387 shutdown_connections 2024-05-09T00:01:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.946+0000 7ff8f955b700 1 --2- 172.21.15.12:0/3629455387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8f4105170 0x7ff8f410ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.946+0000 7ff8f955b700 1 --2- 172.21.15.12:0/3629455387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f4100050 0x7ff8f4100430 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.946+0000 7ff8f955b700 1 --2- 172.21.15.12:0/3629455387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f4100a00 0x7ff8f4104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.946+0000 7ff8f955b700 1 -- 172.21.15.12:0/3629455387 >> 172.21.15.12:0/3629455387 conn(0x7ff8f4076fa0 msgr2=0x7ff8f40773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.946+0000 7ff8f955b700 1 -- 172.21.15.12:0/3629455387 shutdown_connections 2024-05-09T00:01:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.946+0000 7ff8f955b700 1 -- 172.21.15.12:0/3629455387 wait complete. 2024-05-09T00:01:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.947+0000 7ff8f955b700 1 Processor -- start 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f955b700 1 -- start start 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f955b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8f4100050 0x7ff8f4196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f955b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f4105170 0x7ff8f41968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f955b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f419ac60 0x7ff8f419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f955b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f419b6e0 con 0x7ff8f4100050 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f955b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f419b850 con 0x7ff8f419ac60 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f955b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f419ba10 con 0x7ff8f4105170 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8f4100050 0x7ff8f4196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f419ac60 0x7ff8f419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8f4100050 0x7ff8f4196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53126/0 (socket says 172.21.15.12:53126) 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f4105170 0x7ff8f41968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/1664247157 learned_addr learned my addr 172.21.15.12:0/1664247157 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:51.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/1664247157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f4105170 msgr2=0x7ff8f41968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:51.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f4105170 0x7ff8f41968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/1664247157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f419ac60 msgr2=0x7ff8f419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:51.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f419ac60 0x7ff8f419b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.948+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/1664247157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e4009520 con 0x7ff8f4100050 2024-05-09T00:01:51.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.949+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8f4100050 0x7ff8f4196380 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400fa00 tx=0x7ff8e400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:51.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.949+0000 7ff8dbfff700 1 -- 172.21.15.12:0/1664247157 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e401c070 con 0x7ff8f4100050 2024-05-09T00:01:51.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.949+0000 7ff8dbfff700 1 -- 172.21.15.12:0/1664247157 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8e400ba40 con 0x7ff8f4100050 2024-05-09T00:01:51.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.949+0000 7ff8f955b700 1 -- 172.21.15.12:0/1664247157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f4069450 con 0x7ff8f4100050 2024-05-09T00:01:51.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.949+0000 7ff8dbfff700 1 -- 172.21.15.12:0/1664247157 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4004930 con 0x7ff8f4100050 2024-05-09T00:01:51.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.949+0000 7ff8f955b700 1 -- 172.21.15.12:0/1664247157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f4069900 con 0x7ff8f4100050 2024-05-09T00:01:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.953+0000 7ff8dbfff700 1 -- 172.21.15.12:0/1664247157 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff8e4020070 con 0x7ff8f4100050 2024-05-09T00:01:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.953+0000 7ff8dbfff700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8dc0a1290 0x7ff8dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.953+0000 7ff905715b80 1 Processor -- start 2024-05-09T00:01:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.953+0000 7ff905715b80 1 -- start start 2024-05-09T00:01:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f27fc700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8dc0a1290 0x7ff8dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8dbfff700 1 -- 172.21.15.12:0/1664247157 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff8e40cd8b0 con 0x7ff8f4100050 2024-05-09T00:01:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff905715b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 0x55ebd48ee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff905715b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff905715b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff905715b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd483b150 con 0x55ebd48e8ca0 2024-05-09T00:01:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff905715b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd4838b10 con 0x55ebd48eeb30 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff905715b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd4834400 con 0x55ebd48ee210 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 0x55ebd48ee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 0x55ebd48ee5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43122/0 (socket says 172.21.15.12:43122) 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/510263562 learned_addr learned my addr 172.21.15.12:0/510263562 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f27fc700 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f37fe700 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f27fc700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8dc0a1290 0x7ff8dc0a3750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff8f41978d0 tx=0x7ff8e8009e10 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/510263562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 msgr2=0x55ebd48e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48e8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/510263562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 msgr2=0x55ebd48e90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48e90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/510263562 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebd46b53b0 con 0x55ebd48ee210 2024-05-09T00:01:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f37fe700 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48e90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:01:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.954+0000 7ff8f27fc700 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.955+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 0x55ebd48ee5f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40098c0 tx=0x7ff8e4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31963e92b773c402 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.955+0000 7ff8d97fa700 1 -- 172.21.15.12:0/510263562 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e409b190 con 0x55ebd48ee210 2024-05-09T00:01:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.955+0000 7ff8d97fa700 1 -- 172.21.15.12:0/510263562 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8e409aa90 con 0x55ebd48ee210 2024-05-09T00:01:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.955+0000 7ff8d97fa700 1 -- 172.21.15.12:0/510263562 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e409acf0 con 0x55ebd48ee210 2024-05-09T00:01:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 -- 172.21.15.12:0/510263562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 msgr2=0x55ebd48ee5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 0x55ebd48ee5f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40098c0 tx=0x7ff8e4009c00 comp rx=0 tx=0).stop 2024-05-09T00:01:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 -- 172.21.15.12:0/510263562 shutdown_connections 2024-05-09T00:01:51.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48e90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 0x55ebd48ee5f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 --2- 172.21.15.12:0/510263562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 -- 172.21.15.12:0/510263562 >> 172.21.15.12:0/510263562 conn(0x55ebd4842210 msgr2=0x55ebd48dedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 -- 172.21.15.12:0/510263562 shutdown_connections 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 -- 172.21.15.12:0/510263562 wait complete. 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 -- start start 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.956+0000 7ff905715b80 1 Processor -- start 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff905715b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd48f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff905715b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48ee210 0x55ebd48f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff905715b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48f2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff905715b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd47d0560 con 0x55ebd48ee210 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff905715b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd47d0200 con 0x55ebd48eeb30 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff905715b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd46b76a0 con 0x55ebd48e8ca0 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff905715b80 10 client.0 _ll_register_callbacks cb 0x55ebd47d9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:51.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd48f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff8f27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48ee210 0x55ebd48f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd48f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43126/0 (socket says 172.21.15.12:43126) 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/3464625765 learned_addr learned my addr 172.21.15.12:0/3464625765 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff8f37fe700 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/3464625765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 msgr2=0x55ebd48f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/3464625765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48ee210 msgr2=0x55ebd48f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48ee210 0x55ebd48f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.957+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/3464625765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e800e740 con 0x55ebd48e8ca0 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.958+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd48f1e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40cb640 tx=0x7ff8e40053f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52fb03b5ef769053 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.958+0000 7ff8d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.958+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/3464625765 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40b33b0 con 0x55ebd48e8ca0 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.958+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/3464625765 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff8e40a2e80 con 0x55ebd48e8ca0 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.958+0000 7ff905715b80 1 -- 172.21.15.12:0/3464625765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebd478d270 con 0x55ebd48e8ca0 2024-05-09T00:01:51.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.958+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/3464625765 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8e4099f80 con 0x55ebd48e8ca0 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.958+0000 7ff905715b80 10 client.29497 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.958+0000 7ff905715b80 1 -- 172.21.15.12:0/3464625765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7ff8f4069900 con 0x55ebd48e8ca0 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.958+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/3464625765 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ebd478d270 con 0x55ebd48e8ca0 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.959+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/3464625765 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff8e40a29f0 con 0x55ebd48e8ca0 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.960+0000 7ff905715b80 10 client.29497 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.960+0000 7ff905715b80 10 client.29497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.960+0000 7ff905715b80 20 client.29497 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.960+0000 7ff905715b80 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ebd4912e30 0x55ebd4915250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.960+0000 7ff905715b80 4 client.29497 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.960+0000 7ff905715b80 1 -- 172.21.15.12:0/3464625765 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebd48b3810 con 0x55ebd4912e30 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.960+0000 7ff8f27fc700 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ebd4912e30 0x55ebd4915250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.960+0000 7ff8d6ff5700 10 client.29497 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:51.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.960+0000 7ff8f27fc700 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ebd4912e30 0x55ebd4915250 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5057b1ba994d286b server_cookie=96a052dd94c1b293 in_seq=0 out_seq=0 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/3464625765 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ebd48b3810 con 0x55ebd4912e30 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff8d6ff5700 10 client.29497 handle_command_reply: tid=0 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 client.29497 shutdown 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 -- 172.21.15.12:0/3464625765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 msgr2=0x55ebd48f1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd48f1e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40cb640 tx=0x7ff8e40053f0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 -- 172.21.15.12:0/3464625765 shutdown_connections 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ebd4912e30 0x55ebd4915250 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd48f1e10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 --2- 172.21.15.12:0/3464625765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48ee210 0x55ebd48f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 -- 172.21.15.12:0/3464625765 >> 172.21.15.12:0/3464625765 conn(0x55ebd4842210 msgr2=0x55ebd48de5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:51.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff8d6ff5700 0 client.29497 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:51.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 -- 172.21.15.12:0/3464625765 shutdown_connections 2024-05-09T00:01:51.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.962+0000 7ff905715b80 1 -- 172.21.15.12:0/3464625765 wait complete. 2024-05-09T00:01:51.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.963+0000 7ff905715b80 20 client.29497 trim_cache size 0 max 16384 2024-05-09T00:01:51.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.987+0000 7ff905715b80 1 Processor -- start 2024-05-09T00:01:51.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.987+0000 7ff905715b80 1 -- start start 2024-05-09T00:01:51.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.987+0000 7ff905715b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48ded20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.987+0000 7ff905715b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 0x55ebd48ecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.987+0000 7ff905715b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.987+0000 7ff905715b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd490fde0 con 0x55ebd48e8ca0 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.987+0000 7ff905715b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd48dbdc0 con 0x55ebd48eeb30 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.987+0000 7ff905715b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd48d9800 con 0x55ebd48ee210 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48ded20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48ded20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53142/0 (socket says 172.21.15.12:53142) 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/2488506671 learned_addr learned my addr 172.21.15.12:0/2488506671 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f37fe700 1 --2- 172.21.15.12:0/2488506671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f27fc700 1 --2- 172.21.15.12:0/2488506671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 0x55ebd48ecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/2488506671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 msgr2=0x55ebd48ecf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/2488506671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 0x55ebd48ecf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/2488506671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 msgr2=0x55ebd48ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/2488506671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/2488506671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebd4661b60 con 0x55ebd48e8ca0 2024-05-09T00:01:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.988+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/2488506671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48ded20 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff8e409aa20 tx=0x7ff8e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc0f226d5b15b0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.989+0000 7ff8d9ffb700 1 -- 172.21.15.12:0/2488506671 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40a2e80 con 0x55ebd48e8ca0 2024-05-09T00:01:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.989+0000 7ff8d9ffb700 1 -- 172.21.15.12:0/2488506671 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8e409e8c0 con 0x55ebd48e8ca0 2024-05-09T00:01:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.989+0000 7ff8d9ffb700 1 -- 172.21.15.12:0/2488506671 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4099f80 con 0x55ebd48e8ca0 2024-05-09T00:01:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.990+0000 7ff905715b80 1 -- 172.21.15.12:0/2488506671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 msgr2=0x55ebd48ded20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.990+0000 7ff905715b80 1 --2- 172.21.15.12:0/2488506671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48ded20 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff8e409aa20 tx=0x7ff8e4009c00 comp rx=0 tx=0).stop 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.991+0000 7ff905715b80 1 -- 172.21.15.12:0/2488506671 shutdown_connections 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.991+0000 7ff905715b80 1 --2- 172.21.15.12:0/2488506671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48eeb30 0x55ebd48ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.991+0000 7ff905715b80 1 --2- 172.21.15.12:0/2488506671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48e8ca0 0x55ebd48ded20 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.991+0000 7ff905715b80 1 --2- 172.21.15.12:0/2488506671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48ee210 0x55ebd48ecf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.991+0000 7ff905715b80 1 -- 172.21.15.12:0/2488506671 >> 172.21.15.12:0/2488506671 conn(0x55ebd4842210 msgr2=0x55ebd48f06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.991+0000 7ff905715b80 1 -- 172.21.15.12:0/2488506671 shutdown_connections 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.991+0000 7ff905715b80 1 -- 172.21.15.12:0/2488506671 wait complete. 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.992+0000 7ff905715b80 1 -- start start 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.992+0000 7ff905715b80 1 Processor -- start 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.992+0000 7ff905715b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd4908380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.992+0000 7ff905715b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48ee210 0x55ebd49088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.992+0000 7ff905715b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48eeb30 0x55ebd4968330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.992+0000 7ff905715b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd4968870 con 0x55ebd48eeb30 2024-05-09T00:01:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.992+0000 7ff905715b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd49689b0 con 0x55ebd48ee210 2024-05-09T00:01:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.992+0000 7ff905715b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd4968b70 con 0x55ebd48e8ca0 2024-05-09T00:01:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.992+0000 7ff905715b80 10 client.0 _ll_register_callbacks cb 0x55ebd47d9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd4908380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd4908380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43152/0 (socket says 172.21.15.12:43152) 2024-05-09T00:01:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/1939227690 learned_addr learned my addr 172.21.15.12:0/1939227690 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f27fc700 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48ee210 0x55ebd49088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f37fe700 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48eeb30 0x55ebd4968330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/1939227690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48ee210 msgr2=0x55ebd49088c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48ee210 0x55ebd49088c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/1939227690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48eeb30 msgr2=0x55ebd4968330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48eeb30 0x55ebd4968330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f2ffd700 1 -- 172.21.15.12:0/1939227690 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f4069900 con 0x55ebd48e8ca0 2024-05-09T00:01:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8f2ffd700 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd4908380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff8e409a9d0 tx=0x7ff8e4005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72461ae16806e725 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.993+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/1939227690 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e409a180 con 0x55ebd48e8ca0 2024-05-09T00:01:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.994+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/1939227690 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff8e40a2e80 con 0x55ebd48e8ca0 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.994+0000 7ff905715b80 1 -- 172.21.15.12:0/1939227690 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8e409aa90 con 0x55ebd48e8ca0 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.994+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/1939227690 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8e40664a0 con 0x55ebd48e8ca0 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.994+0000 7ff905715b80 10 client.29500 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.994+0000 7ff905715b80 1 -- 172.21.15.12:0/1939227690 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55ebd47fa560 con 0x55ebd48e8ca0 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.994+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/1939227690 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8e409aa90 con 0x55ebd48e8ca0 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.995+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/1939227690 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff8e4099b10 con 0x55ebd48e8ca0 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.995+0000 7ff905715b80 10 client.29500 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.995+0000 7ff905715b80 10 client.29500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.995+0000 7ff905715b80 20 client.29500 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.995+0000 7ff905715b80 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ebd4969e70 0x55ebd496c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.995+0000 7ff905715b80 4 client.29500 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.995+0000 7ff905715b80 1 -- 172.21.15.12:0/1939227690 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ebd482cec0 con 0x55ebd4969e70 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.995+0000 7ff8f27fc700 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ebd4969e70 0x55ebd496c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.996+0000 7ff8f27fc700 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ebd4969e70 0x55ebd496c260 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536b2818f27fbe64 server_cookie=363ccf7e4bc27100 in_seq=0 out_seq=0 2024-05-09T00:01:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:51.996+0000 7ff8d6ff5700 10 client.29500 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff8d6ff5700 1 -- 172.21.15.12:0/1939227690 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ebd482cec0 con 0x55ebd4969e70 2024-05-09T00:01:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff8d6ff5700 10 client.29500 handle_command_reply: tid=0 2024-05-09T00:01:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff905715b80 1 client.29500 shutdown 2024-05-09T00:01:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff905715b80 1 -- 172.21.15.12:0/1939227690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 msgr2=0x55ebd4908380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff905715b80 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd4908380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff8e409a9d0 tx=0x7ff8e4005960 comp rx=0 tx=0).stop 2024-05-09T00:01:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff905715b80 1 -- 172.21.15.12:0/1939227690 shutdown_connections 2024-05-09T00:01:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff905715b80 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ebd48eeb30 0x55ebd4968330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff905715b80 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebd48e8ca0 0x55ebd4908380 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff905715b80 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ebd4969e70 0x55ebd496c260 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff905715b80 1 --2- 172.21.15.12:0/1939227690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ebd48ee210 0x55ebd49088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff8d6ff5700 0 client.29500 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:51.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.001+0000 7ff905715b80 1 -- 172.21.15.12:0/1939227690 >> 172.21.15.12:0/1939227690 conn(0x55ebd4842210 msgr2=0x55ebd48eefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:51.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.002+0000 7ff905715b80 1 -- 172.21.15.12:0/1939227690 shutdown_connections 2024-05-09T00:01:51.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.002+0000 7ff905715b80 1 -- 172.21.15.12:0/1939227690 wait complete. 2024-05-09T00:01:51.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.002+0000 7ff905715b80 20 client.29500 trim_cache size 0 max 16384 2024-05-09T00:01:51.997 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:01:51.997 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:01:51.997 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:01:51.997 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:01:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.002+0000 7ff8f955b700 1 -- 172.21.15.12:0/1664247157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8dc0a1290 msgr2=0x7ff8dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.002+0000 7ff8f955b700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8dc0a1290 0x7ff8dc0a3750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff8f41978d0 tx=0x7ff8e8009e10 comp rx=0 tx=0).stop 2024-05-09T00:01:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.002+0000 7ff8f955b700 1 -- 172.21.15.12:0/1664247157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8f4100050 msgr2=0x7ff8f4196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.002+0000 7ff8f955b700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8f4100050 0x7ff8f4196380 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400fa00 tx=0x7ff8e400fa30 comp rx=0 tx=0).stop 2024-05-09T00:01:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.003+0000 7ff8f955b700 1 -- 172.21.15.12:0/1664247157 shutdown_connections 2024-05-09T00:01:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.003+0000 7ff8f955b700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8dc0a1290 0x7ff8dc0a3750 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.003+0000 7ff8f955b700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f419ac60 0x7ff8f419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.003+0000 7ff8f955b700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8f4100050 0x7ff8f4196380 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.003+0000 7ff8f955b700 1 --2- 172.21.15.12:0/1664247157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8f4105170 0x7ff8f41968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:51.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.003+0000 7ff8f955b700 1 -- 172.21.15.12:0/1664247157 >> 172.21.15.12:0/1664247157 conn(0x7ff8f4076fa0 msgr2=0x7ff8f40fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:51.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.003+0000 7ff8f955b700 1 -- 172.21.15.12:0/1664247157 shutdown_connections 2024-05-09T00:01:51.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.003+0000 7ff8f955b700 1 -- 172.21.15.12:0/1664247157 wait complete. 2024-05-09T00:01:52.012 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:01:52.012 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:01:52.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:51 smithi204 ceph-mon[47373]: pgmap v495: 97 pgs: 97 active+clean; 770 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-09T00:01:52.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:51 smithi160 ceph-mon[46152]: pgmap v495: 97 pgs: 97 active+clean; 770 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-09T00:01:52.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.196+0000 7f6d48bbb700 1 -- 172.21.15.12:0/515522177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d44105060 msgr2=0x7f6d4410cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.196+0000 7f6d48bbb700 1 --2- 172.21.15.12:0/515522177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d44105060 0x7f6d4410cbe0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00b0a0 tx=0x7f6d3c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.196+0000 7f6d48bbb700 1 -- 172.21.15.12:0/515522177 shutdown_connections 2024-05-09T00:01:52.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.196+0000 7f6d48bbb700 1 --2- 172.21.15.12:0/515522177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d44105060 0x7f6d4410cbe0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.196+0000 7f6d48bbb700 1 --2- 172.21.15.12:0/515522177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d440fff40 0x7f6d44100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.196+0000 7f6d48bbb700 1 --2- 172.21.15.12:0/515522177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d441008f0 0x7f6d44104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.196+0000 7f6d48bbb700 1 -- 172.21.15.12:0/515522177 >> 172.21.15.12:0/515522177 conn(0x7f6d440fb7e0 msgr2=0x7f6d440fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.196+0000 7f6d48bbb700 1 -- 172.21.15.12:0/515522177 shutdown_connections 2024-05-09T00:01:52.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.196+0000 7f6d48bbb700 1 -- 172.21.15.12:0/515522177 wait complete. 2024-05-09T00:01:52.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.197+0000 7f6d48bbb700 1 Processor -- start 2024-05-09T00:01:52.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.197+0000 7f6d48bbb700 1 -- start start 2024-05-09T00:01:52.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.197+0000 7f6d48bbb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d440fff40 0x7f6d441982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.197+0000 7f6d48bbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d44105060 0x7f6d44198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.197+0000 7f6d48bbb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d44198d40 0x7f6d441a3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.197+0000 7f6d48bbb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d441a39e0 con 0x7f6d44105060 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.197+0000 7f6d48bbb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d441a3b50 con 0x7f6d44198d40 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.197+0000 7f6d48bbb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d441a3d10 con 0x7f6d440fff40 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d41d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d44105060 0x7f6d44198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d42d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d44198d40 0x7f6d441a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d4259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d440fff40 0x7f6d441982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d41d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d44105060 0x7f6d44198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53176/0 (socket says 172.21.15.12:53176) 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d41d9b700 1 -- 172.21.15.12:0/2447184971 learned_addr learned my addr 172.21.15.12:0/2447184971 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d41d9b700 1 -- 172.21.15.12:0/2447184971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d440fff40 msgr2=0x7f6d441982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d41d9b700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d440fff40 0x7f6d441982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d41d9b700 1 -- 172.21.15.12:0/2447184971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d44198d40 msgr2=0x7f6d441a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d41d9b700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d44198d40 0x7f6d441a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d41d9b700 1 -- 172.21.15.12:0/2447184971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c009d00 con 0x7f6d44105060 2024-05-09T00:01:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d42d9d700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d44198d40 0x7f6d441a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d41d9b700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d44105060 0x7f6d44198800 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800e970 tx=0x7f6d3800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.198+0000 7f6d337fe700 1 -- 172.21.15.12:0/2447184971 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38009790 con 0x7f6d44105060 2024-05-09T00:01:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.199+0000 7f6d48bbb700 1 -- 172.21.15.12:0/2447184971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d44196110 con 0x7f6d44105060 2024-05-09T00:01:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.199+0000 7f6d337fe700 1 -- 172.21.15.12:0/2447184971 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d3800c7e0 con 0x7f6d44105060 2024-05-09T00:01:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.199+0000 7f6d337fe700 1 -- 172.21.15.12:0/2447184971 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d380054e0 con 0x7f6d44105060 2024-05-09T00:01:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.199+0000 7f6d48bbb700 1 -- 172.21.15.12:0/2447184971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d44196670 con 0x7f6d44105060 2024-05-09T00:01:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.200+0000 7f6d48bbb700 1 -- 172.21.15.12:0/2447184971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d44108330 con 0x7f6d44105060 2024-05-09T00:01:52.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.202+0000 7f6d337fe700 1 -- 172.21.15.12:0/2447184971 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6d3800c950 con 0x7f6d44105060 2024-05-09T00:01:52.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.202+0000 7f6d337fe700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d2c0a1290 0x7f6d2c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.203+0000 7f6d337fe700 1 -- 172.21.15.12:0/2447184971 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6d38014070 con 0x7f6d44105060 2024-05-09T00:01:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.203+0000 7f6d4259c700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d2c0a1290 0x7f6d2c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.203+0000 7f6d4259c700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d2c0a1290 0x7f6d2c0a3750 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6d34010d80 tx=0x7f6d34012040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.207+0000 7f6d337fe700 1 -- 172.21.15.12:0/2447184971 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6d38096560 con 0x7f6d44105060 2024-05-09T00:01:52.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 79130 41.51 MB/sec warmup 107 sec latency 100.888 ms 2024-05-09T00:01:52.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.511+0000 7f6d48bbb700 1 -- 172.21.15.12:0/2447184971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d44068590 con 0x7f6d44105060 2024-05-09T00:01:52.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.512+0000 7f6d337fe700 1 -- 172.21.15.12:0/2447184971 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+10885 (secure 0 0 0) 0x7f6d38099b80 con 0x7f6d44105060 2024-05-09T00:01:52.507 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:01:52.507 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:01:47.089722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_29453":{"gid":29453,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3146138040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3146138040},{"type":"v1","addr":"172.21.15.204:6837","nonce":3146138040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 -- 172.21.15.12:0/2447184971 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d2c0a1290 msgr2=0x7f6d2c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d2c0a1290 0x7f6d2c0a3750 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6d34010d80 tx=0x7f6d34012040 comp rx=0 tx=0).stop 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 -- 172.21.15.12:0/2447184971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d44105060 msgr2=0x7f6d44198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d44105060 0x7f6d44198800 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800e970 tx=0x7f6d3800ec80 comp rx=0 tx=0).stop 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 -- 172.21.15.12:0/2447184971 shutdown_connections 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6d2c0a1290 0x7f6d2c0a3750 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d44198d40 0x7f6d441a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d440fff40 0x7f6d441982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 --2- 172.21.15.12:0/2447184971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d44105060 0x7f6d44198800 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 -- 172.21.15.12:0/2447184971 >> 172.21.15.12:0/2447184971 conn(0x7f6d440fb7e0 msgr2=0x7f6d440fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 -- 172.21.15.12:0/2447184971 shutdown_connections 2024-05-09T00:01:52.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.515+0000 7f6d48bbb700 1 -- 172.21.15.12:0/2447184971 wait complete. 2024-05-09T00:01:52.511 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 63 2024-05-09T00:01:52.524 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:01:52.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.687+0000 7f54a63e8700 1 -- 172.21.15.12:0/598324635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a00682e0 msgr2=0x7f54a00686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.687+0000 7f54a63e8700 1 --2- 172.21.15.12:0/598324635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a00682e0 0x7f54a00686c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f54900099c0 tx=0x7f5490009cd0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.687+0000 7f54a63e8700 1 -- 172.21.15.12:0/598324635 shutdown_connections 2024-05-09T00:01:52.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.687+0000 7f54a63e8700 1 --2- 172.21.15.12:0/598324635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a0106290 0x7f54a010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.687+0000 7f54a63e8700 1 --2- 172.21.15.12:0/598324635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a00682e0 0x7f54a00686c0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.687+0000 7f54a63e8700 1 --2- 172.21.15.12:0/598324635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54a0068c90 0x7f54a0105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.687+0000 7f54a63e8700 1 -- 172.21.15.12:0/598324635 >> 172.21.15.12:0/598324635 conn(0x7f54a0076f40 msgr2=0x7f54a0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.687+0000 7f54a63e8700 1 -- 172.21.15.12:0/598324635 shutdown_connections 2024-05-09T00:01:52.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.687+0000 7f54a63e8700 1 -- 172.21.15.12:0/598324635 wait complete. 2024-05-09T00:01:52.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.688+0000 7f54a63e8700 1 Processor -- start 2024-05-09T00:01:52.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.688+0000 7f54a63e8700 1 -- start start 2024-05-09T00:01:52.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a63e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a00682e0 0x7f54a01973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a63e8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54a0106290 0x7f54a0197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a63e8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a019bcd0 0x7f54a019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a63e8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a019c750 con 0x7f54a00682e0 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a63e8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a019c8c0 con 0x7f54a0106290 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a63e8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a019ca80 con 0x7f54a019bcd0 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a4985700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a019bcd0 0x7f54a019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a4985700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a019bcd0 0x7f54a019c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43198/0 (socket says 172.21.15.12:43198) 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a4985700 1 -- 172.21.15.12:0/2303753289 learned_addr learned my addr 172.21.15.12:0/2303753289 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a4985700 1 -- 172.21.15.12:0/2303753289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54a0106290 msgr2=0x7f54a0197930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a4985700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54a0106290 0x7f54a0197930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a4985700 1 -- 172.21.15.12:0/2303753289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a00682e0 msgr2=0x7f54a01973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a4985700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a00682e0 0x7f54a01973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.689+0000 7f54a4985700 1 -- 172.21.15.12:0/2303753289 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5498009c40 con 0x7f54a019bcd0 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.690+0000 7f54a4985700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a019bcd0 0x7f54a019c150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f549800c370 tx=0x7f549800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.690+0000 7f549d7fa700 1 -- 172.21.15.12:0/2303753289 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5498017070 con 0x7f54a019bcd0 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.690+0000 7f549d7fa700 1 -- 172.21.15.12:0/2303753289 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f549800f040 con 0x7f54a019bcd0 2024-05-09T00:01:52.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.690+0000 7f54a63e8700 1 -- 172.21.15.12:0/2303753289 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54900095f0 con 0x7f54a019bcd0 2024-05-09T00:01:52.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.690+0000 7f549d7fa700 1 -- 172.21.15.12:0/2303753289 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549800e050 con 0x7f54a019bcd0 2024-05-09T00:01:52.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.690+0000 7f54a63e8700 1 -- 172.21.15.12:0/2303753289 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a01000b0 con 0x7f54a019bcd0 2024-05-09T00:01:52.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.693+0000 7f549d7fa700 1 -- 172.21.15.12:0/2303753289 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5498004500 con 0x7f54a019bcd0 2024-05-09T00:01:52.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.693+0000 7f549d7fa700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f548c0a1310 0x7f548c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.694+0000 7f549d7fa700 1 -- 172.21.15.12:0/2303753289 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f54980cc6a0 con 0x7f54a019bcd0 2024-05-09T00:01:52.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.694+0000 7f549ffff700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f548c0a1310 0x7f548c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.694+0000 7f549ffff700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f548c0a1310 0x7f548c0a37d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f54900058e0 tx=0x7f5490000bc0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f54b25a2b80 1 Processor -- start 2024-05-09T00:01:52.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f54b25a2b80 1 -- start start 2024-05-09T00:01:52.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d7fe00 0x564f54d801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d7acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54cd1150 con 0x564f54d80720 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54cceb10 con 0x564f54d7fe00 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54cca400 con 0x564f54d7a890 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f549ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d7fe00 0x564f54d801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53192/0 (socket says 172.21.15.12:53192) 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f549f7fe700 1 -- 172.21.15.12:0/2666861854 learned_addr learned my addr 172.21.15.12:0/2666861854 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.695+0000 7f54a4985700 1 --2- 172.21.15.12:0/2666861854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d7acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.696+0000 7f549ffff700 1 -- 172.21.15.12:0/2666861854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 msgr2=0x564f54d7acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.696+0000 7f549ffff700 1 --2- 172.21.15.12:0/2666861854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d7acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.696+0000 7f549ffff700 1 -- 172.21.15.12:0/2666861854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 msgr2=0x564f54d7a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.696+0000 7f549ffff700 1 --2- 172.21.15.12:0/2666861854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.696+0000 7f549ffff700 1 -- 172.21.15.12:0/2666861854 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f54b4b3b0 con 0x564f54d7fe00 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.696+0000 7f549f7fe700 1 --2- 172.21.15.12:0/2666861854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.696+0000 7f549ffff700 1 --2- 172.21.15.12:0/2666861854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d7fe00 0x564f54d801e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5490009fd0 tx=0x7f549001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883a26c622a8de83 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.696+0000 7f54967fc700 1 -- 172.21.15.12:0/2666861854 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490029070 con 0x564f54d7fe00 2024-05-09T00:01:52.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.696+0000 7f54967fc700 1 -- 172.21.15.12:0/2666861854 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5490005bc0 con 0x564f54d7fe00 2024-05-09T00:01:52.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.696+0000 7f54967fc700 1 -- 172.21.15.12:0/2666861854 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549001d070 con 0x564f54d7fe00 2024-05-09T00:01:52.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.697+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2666861854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d7fe00 msgr2=0x564f54d801e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.697+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/2666861854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d7fe00 0x564f54d801e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5490009fd0 tx=0x7f549001d000 comp rx=0 tx=0).stop 2024-05-09T00:01:52.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.697+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2666861854 shutdown_connections 2024-05-09T00:01:52.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.697+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/2666861854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d7acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.697+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/2666861854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d7fe00 0x564f54d801e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.697+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/2666861854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.697+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2666861854 >> 172.21.15.12:0/2666861854 conn(0x564f54cd8210 msgr2=0x564f54d74df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.697+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2666861854 shutdown_connections 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.697+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2666861854 wait complete. 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.697+0000 7f54b25a2b80 1 -- start start 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f54b25a2b80 1 Processor -- start 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7a890 0x564f54d83b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7fe00 0x564f54d840a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 0x564f54d845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54c66560 con 0x564f54d7a890 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54c66200 con 0x564f54d80720 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54b4d6a0 con 0x564f54d7fe00 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f54b25a2b80 10 client.0 _ll_register_callbacks cb 0x564f54c6fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:52.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f549ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7a890 0x564f54d83b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7fe00 0x564f54d840a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f549ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7a890 0x564f54d83b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53202/0 (socket says 172.21.15.12:53202) 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f549ffff700 1 -- 172.21.15.12:0/2902600118 learned_addr learned my addr 172.21.15.12:0/2902600118 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.698+0000 7f54a4985700 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 0x564f54d845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f549f7fe700 1 -- 172.21.15.12:0/2902600118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 msgr2=0x564f54d845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f549f7fe700 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 0x564f54d845e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f549f7fe700 1 -- 172.21.15.12:0/2902600118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7a890 msgr2=0x564f54d83b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f549f7fe700 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7a890 0x564f54d83b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f549f7fe700 1 -- 172.21.15.12:0/2902600118 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54880098b0 con 0x564f54d7fe00 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f54a4985700 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 0x564f54d845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f549ffff700 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7a890 0x564f54d83b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f549f7fe700 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7fe00 0x564f54d840a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f54880046b0 tx=0x7f5488005c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3e36e474131bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f547f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f547f7fe700 1 -- 172.21.15.12:0/2902600118 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800e7c0 con 0x564f54d7fe00 2024-05-09T00:01:52.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f547f7fe700 1 -- 172.21.15.12:0/2902600118 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f548800e920 con 0x564f54d7fe00 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2902600118 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f54d49810 con 0x564f54d7fe00 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f547f7fe700 1 -- 172.21.15.12:0/2902600118 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f548800eae0 con 0x564f54d7fe00 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f54b25a2b80 10 client.29518 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.699+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2902600118 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7f54a010e650 con 0x564f54d7fe00 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f547f7fe700 1 -- 172.21.15.12:0/2902600118 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f54d49810 con 0x564f54d7fe00 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f547f7fe700 1 -- 172.21.15.12:0/2902600118 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f54880223d0 con 0x564f54d7fe00 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f54b25a2b80 10 client.29518 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f54b25a2b80 10 client.29518 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f54b25a2b80 20 client.29518 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f54d94270 0x564f54d96690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f54b25a2b80 4 client.29518 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2902600118 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f54cc2ec0 con 0x564f54d94270 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f549ffff700 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f54d94270 0x564f54d96690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f547f7fe700 10 client.29518 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:01:52.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.700+0000 7f549ffff700 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f54d94270 0x564f54d96690 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5036fb9f52e36d0 server_cookie=42a1601a56d46075 in_seq=0 out_seq=0 2024-05-09T00:01:52.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f547f7fe700 1 -- 172.21.15.12:0/2902600118 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564f54cc2ec0 con 0x564f54d94270 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f547f7fe700 10 client.29518 handle_command_reply: tid=0 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f54b25a2b80 1 client.29518 shutdown 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2902600118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7fe00 msgr2=0x564f54d840a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7fe00 0x564f54d840a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f54880046b0 tx=0x7f5488005c10 comp rx=0 tx=0).stop 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2902600118 shutdown_connections 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 0x564f54d845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7a890 0x564f54d83b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f54d94270 0x564f54d96690 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/2902600118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7fe00 0x564f54d840a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f547f7fe700 0 client.29518 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.703+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2902600118 >> 172.21.15.12:0/2902600118 conn(0x564f54cd8210 msgr2=0x564f54d745a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.704+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2902600118 shutdown_connections 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.704+0000 7f54b25a2b80 1 -- 172.21.15.12:0/2902600118 wait complete. 2024-05-09T00:01:52.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.704+0000 7f54b25a2b80 20 client.29518 trim_cache size 0 max 16384 2024-05-09T00:01:52.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54b25a2b80 1 Processor -- start 2024-05-09T00:01:52.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54b25a2b80 1 -- start start 2024-05-09T00:01:52.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d74d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d7fe00 0x564f54d84400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54d6f800 con 0x564f54d80720 2024-05-09T00:01:52.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54d9b8a0 con 0x564f54d7fe00 2024-05-09T00:01:52.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54d75ac0 con 0x564f54d7a890 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54a4985700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54a4985700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53204/0 (socket says 172.21.15.12:53204) 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54a4985700 1 -- 172.21.15.12:0/210271936 learned_addr learned my addr 172.21.15.12:0/210271936 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f549ffff700 1 --2- 172.21.15.12:0/210271936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d74d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54a4985700 1 -- 172.21.15.12:0/210271936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 msgr2=0x564f54d74d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54a4985700 1 --2- 172.21.15.12:0/210271936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d74d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54a4985700 1 -- 172.21.15.12:0/210271936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d7fe00 msgr2=0x564f54d84400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54a4985700 1 --2- 172.21.15.12:0/210271936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d7fe00 0x564f54d84400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.721+0000 7f54a4985700 1 -- 172.21.15.12:0/210271936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f54af7b60 con 0x564f54d80720 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.722+0000 7f54a4985700 1 --2- 172.21.15.12:0/210271936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7ec50 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5498099780 tx=0x7f5498098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a540551e3029ab5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.722+0000 7f5496ffd700 1 -- 172.21.15.12:0/210271936 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54980b92b0 con 0x564f54d80720 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.722+0000 7f5496ffd700 1 -- 172.21.15.12:0/210271936 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54980997f0 con 0x564f54d80720 2024-05-09T00:01:52.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.722+0000 7f5496ffd700 1 -- 172.21.15.12:0/210271936 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5498098f60 con 0x564f54d80720 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 -- 172.21.15.12:0/210271936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 msgr2=0x564f54d7ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/210271936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7ec50 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5498099780 tx=0x7f5498098040 comp rx=0 tx=0).stop 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 -- 172.21.15.12:0/210271936 shutdown_connections 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/210271936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d80720 0x564f54d7ec50 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/210271936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d74d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/210271936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d7fe00 0x564f54d84400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 -- 172.21.15.12:0/210271936 >> 172.21.15.12:0/210271936 conn(0x564f54cd8210 msgr2=0x564f54d82430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 -- 172.21.15.12:0/210271936 shutdown_connections 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 -- 172.21.15.12:0/210271936 wait complete. 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 -- start start 2024-05-09T00:01:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.723+0000 7f54b25a2b80 1 Processor -- start 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7fe00 0x564f54d89630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f54b25a2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 0x564f54de9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54d89ed0 con 0x564f54d7fe00 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54de9cb0 con 0x564f54d80720 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f54b25a2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f54de9df0 con 0x564f54d7a890 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f54b25a2b80 10 client.0 _ll_register_callbacks cb 0x564f54c6fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7fe00 0x564f54d89630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7fe00 0x564f54d89630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53216/0 (socket says 172.21.15.12:53216) 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f549f7fe700 1 -- 172.21.15.12:0/893828822 learned_addr learned my addr 172.21.15.12:0/893828822 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f549ffff700 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f54a4985700 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 0x564f54de9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f549f7fe700 1 -- 172.21.15.12:0/893828822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 msgr2=0x564f54d890f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f549f7fe700 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d890f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f549f7fe700 1 -- 172.21.15.12:0/893828822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 msgr2=0x564f54de9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f549f7fe700 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 0x564f54de9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.724+0000 7f549f7fe700 1 -- 172.21.15.12:0/893828822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a01000b0 con 0x564f54d7fe00 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.725+0000 7f549f7fe700 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7fe00 0x564f54d89630 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f5488022760 tx=0x7f5488009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b08bfe4aacdb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.725+0000 7f547f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.725+0000 7f547f7fe700 1 -- 172.21.15.12:0/893828822 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488005240 con 0x564f54d7fe00 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.725+0000 7f547f7fe700 1 -- 172.21.15.12:0/893828822 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5488000e40 con 0x564f54d7fe00 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.725+0000 7f54b25a2b80 1 -- 172.21.15.12:0/893828822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f54c23270 con 0x564f54d7fe00 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.725+0000 7f547f7fe700 1 -- 172.21.15.12:0/893828822 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5488022ec0 con 0x564f54d7fe00 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.725+0000 7f54b25a2b80 10 client.19794 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.725+0000 7f54b25a2b80 1 -- 172.21.15.12:0/893828822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x564f54c90560 con 0x564f54d7fe00 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.725+0000 7f547f7fe700 1 -- 172.21.15.12:0/893828822 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f54c23270 con 0x564f54d7fe00 2024-05-09T00:01:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.725+0000 7f547f7fe700 1 -- 172.21.15.12:0/893828822 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5488004bb0 con 0x564f54d7fe00 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.726+0000 7f54b25a2b80 10 client.19794 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.726+0000 7f54b25a2b80 10 client.19794 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.726+0000 7f54b25a2b80 20 client.19794 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.726+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f54deb4f0 0x564f54ded910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.726+0000 7f54b25a2b80 4 client.19794 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.726+0000 7f54b25a2b80 1 -- 172.21.15.12:0/893828822 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f54b18cf0 con 0x564f54deb4f0 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.726+0000 7f549ffff700 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f54deb4f0 0x564f54ded910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.726+0000 7f547f7fe700 10 client.19794 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.726+0000 7f549ffff700 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f54deb4f0 0x564f54ded910 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e2ccc122562e2a4 server_cookie=cdb9ec7c1bc5335b in_seq=0 out_seq=0 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f547f7fe700 1 -- 172.21.15.12:0/893828822 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f54b18cf0 con 0x564f54deb4f0 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f547f7fe700 10 client.19794 handle_command_reply: tid=0 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 client.19794 shutdown 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 -- 172.21.15.12:0/893828822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7fe00 msgr2=0x564f54d89630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7fe00 0x564f54d89630 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f5488022760 tx=0x7f5488009350 comp rx=0 tx=0).stop 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 -- 172.21.15.12:0/893828822 shutdown_connections 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564f54d80720 0x564f54de9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x564f54deb4f0 0x564f54ded910 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f54d7a890 0x564f54d890f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 --2- 172.21.15.12:0/893828822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f54d7fe00 0x564f54d89630 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 -- 172.21.15.12:0/893828822 >> 172.21.15.12:0/893828822 conn(0x564f54cd8210 msgr2=0x564f54d7ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f547f7fe700 0 client.19794 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 -- 172.21.15.12:0/893828822 shutdown_connections 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.727+0000 7f54b25a2b80 1 -- 172.21.15.12:0/893828822 wait complete. 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.728+0000 7f54b25a2b80 20 client.19794 trim_cache size 0 max 16384 2024-05-09T00:01:52.726 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:01:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.728+0000 7f54a63e8700 1 -- 172.21.15.12:0/2303753289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f548c0a1310 msgr2=0x7f548c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.728+0000 7f54a63e8700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f548c0a1310 0x7f548c0a37d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f54900058e0 tx=0x7f5490000bc0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.728+0000 7f54a63e8700 1 -- 172.21.15.12:0/2303753289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a019bcd0 msgr2=0x7f54a019c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.728+0000 7f54a63e8700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a019bcd0 0x7f54a019c150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f549800c370 tx=0x7f549800c730 comp rx=0 tx=0).stop 2024-05-09T00:01:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.729+0000 7f54a63e8700 1 -- 172.21.15.12:0/2303753289 shutdown_connections 2024-05-09T00:01:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.729+0000 7f54a63e8700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f548c0a1310 0x7f548c0a37d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.729+0000 7f54a63e8700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a019bcd0 0x7f54a019c150 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.729+0000 7f54a63e8700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a00682e0 0x7f54a01973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.729+0000 7f54a63e8700 1 --2- 172.21.15.12:0/2303753289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54a0106290 0x7f54a0197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.729+0000 7f54a63e8700 1 -- 172.21.15.12:0/2303753289 >> 172.21.15.12:0/2303753289 conn(0x7f54a0076f40 msgr2=0x7f54a00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.729+0000 7f54a63e8700 1 -- 172.21.15.12:0/2303753289 shutdown_connections 2024-05-09T00:01:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.729+0000 7f54a63e8700 1 -- 172.21.15.12:0/2303753289 wait complete. 2024-05-09T00:01:52.738 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:01:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.905+0000 7f2b0cab9700 1 -- 172.21.15.12:0/2455003866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b08100060 msgr2=0x7f2b08100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.905+0000 7f2b0cab9700 1 --2- 172.21.15.12:0/2455003866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b08100060 0x7f2b08100440 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009990 tx=0x7f2af8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.905+0000 7f2b0cab9700 1 -- 172.21.15.12:0/2455003866 shutdown_connections 2024-05-09T00:01:52.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.905+0000 7f2b0cab9700 1 --2- 172.21.15.12:0/2455003866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b08105160 0x7f2b0810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.905+0000 7f2b0cab9700 1 --2- 172.21.15.12:0/2455003866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b08100060 0x7f2b08100440 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.905+0000 7f2b0cab9700 1 --2- 172.21.15.12:0/2455003866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b08100a10 0x7f2b08104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.905+0000 7f2b0cab9700 1 -- 172.21.15.12:0/2455003866 >> 172.21.15.12:0/2455003866 conn(0x7f2b08076f40 msgr2=0x7f2b08077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.905+0000 7f2b0cab9700 1 -- 172.21.15.12:0/2455003866 shutdown_connections 2024-05-09T00:01:52.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.905+0000 7f2b0cab9700 1 -- 172.21.15.12:0/2455003866 wait complete. 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.906+0000 7f2b0cab9700 1 Processor -- start 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.906+0000 7f2b0cab9700 1 -- start start 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0cab9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b08100060 0x7f2b081962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0cab9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b08105160 0x7f2b08196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0cab9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b0819abb0 0x7f2b0819b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0cab9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0819b630 con 0x7f2b0819abb0 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0cab9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0819b7a0 con 0x7f2b08105160 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0cab9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0819b960 con 0x7f2b08100060 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b08100060 0x7f2b081962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b05d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b08105160 0x7f2b08196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b05d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b08105160 0x7f2b08196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:46500/0 (socket says 172.21.15.12:46500) 2024-05-09T00:01:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b08100060 0x7f2b081962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43252/0 (socket says 172.21.15.12:43252) 2024-05-09T00:01:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b05d9b700 1 -- 172.21.15.12:0/3967097308 learned_addr learned my addr 172.21.15.12:0/3967097308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0659c700 1 -- 172.21.15.12:0/3967097308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b08105160 msgr2=0x7f2b08196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0659c700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b08105160 0x7f2b08196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0659c700 1 -- 172.21.15.12:0/3967097308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b0819abb0 msgr2=0x7f2b0819b030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:01:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0659c700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b0819abb0 0x7f2b0819b030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.907+0000 7f2b0659c700 1 -- 172.21.15.12:0/3967097308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80095f0 con 0x7f2b08100060 2024-05-09T00:01:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.908+0000 7f2b0659c700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b08100060 0x7f2b081962d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2af80059d0 tx=0x7f2af8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.908+0000 7f2af77fe700 1 -- 172.21.15.12:0/3967097308 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af801d070 con 0x7f2b08100060 2024-05-09T00:01:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.908+0000 7f2af77fe700 1 -- 172.21.15.12:0/3967097308 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2af8022470 con 0x7f2b08100060 2024-05-09T00:01:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.908+0000 7f2af77fe700 1 -- 172.21.15.12:0/3967097308 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8017650 con 0x7f2b08100060 2024-05-09T00:01:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.908+0000 7f2b0cab9700 1 -- 172.21.15.12:0/3967097308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b08067ff0 con 0x7f2b08100060 2024-05-09T00:01:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.908+0000 7f2b0cab9700 1 -- 172.21.15.12:0/3967097308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b08068450 con 0x7f2b08100060 2024-05-09T00:01:52.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.912+0000 7f2af77fe700 1 -- 172.21.15.12:0/3967097308 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2af8022850 con 0x7f2b08100060 2024-05-09T00:01:52.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.913+0000 7f2af77fe700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2af00a1310 0x7f2af00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.913+0000 7f2b18c73b80 1 Processor -- start 2024-05-09T00:01:52.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.913+0000 7f2af77fe700 1 -- 172.21.15.12:0/3967097308 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2af80cdda0 con 0x7f2b08100060 2024-05-09T00:01:52.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.913+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2af00a1310 0x7f2af00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.913+0000 7f2b18c73b80 1 -- start start 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 0x562cfa12e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa128b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 0x562cfa1294e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa077150 con 0x562cfa12e610 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2af00a1310 0x7f2af00a37d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2b08197820 tx=0x7f2afc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa074b10 con 0x562cfa12ef30 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa070400 con 0x562cfa1290a0 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 0x562cfa1294e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 0x562cfa1294e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43266/0 (socket says 172.21.15.12:43266) 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b06d9d700 1 -- 172.21.15.12:0/3554304878 learned_addr learned my addr 172.21.15.12:0/3554304878 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.914+0000 7f2b0659c700 1 --2- 172.21.15.12:0/3554304878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 0x562cfa12e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.915+0000 7f2b0659c700 1 -- 172.21.15.12:0/3554304878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 msgr2=0x562cfa1294e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.915+0000 7f2b0659c700 1 --2- 172.21.15.12:0/3554304878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 0x562cfa1294e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.915+0000 7f2b0659c700 1 -- 172.21.15.12:0/3554304878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 msgr2=0x562cfa128b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:01:52.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.915+0000 7f2b0659c700 1 --2- 172.21.15.12:0/3554304878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa128b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.915+0000 7f2b0659c700 1 -- 172.21.15.12:0/3554304878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cf9ef13b0 con 0x562cfa12e610 2024-05-09T00:01:52.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.915+0000 7f2b0659c700 1 --2- 172.21.15.12:0/3554304878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 0x562cfa12e9f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f2af80cb5c0 tx=0x7f2af8097480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3581e53cf381ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.915+0000 7f2af4ff9700 1 -- 172.21.15.12:0/3554304878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af80a10f0 con 0x562cfa12e610 2024-05-09T00:01:52.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.915+0000 7f2af4ff9700 1 -- 172.21.15.12:0/3554304878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2af809b010 con 0x562cfa12e610 2024-05-09T00:01:52.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.915+0000 7f2af4ff9700 1 -- 172.21.15.12:0/3554304878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af809a500 con 0x562cfa12e610 2024-05-09T00:01:52.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.916+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3554304878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 msgr2=0x562cfa12e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.916+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/3554304878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 0x562cfa12e9f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f2af80cb5c0 tx=0x7f2af8097480 comp rx=0 tx=0).stop 2024-05-09T00:01:52.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.917+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3554304878 shutdown_connections 2024-05-09T00:01:52.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.917+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/3554304878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 0x562cfa1294e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.917+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/3554304878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 0x562cfa12e9f0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.917+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/3554304878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa128b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.917+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3554304878 >> 172.21.15.12:0/3554304878 conn(0x562cfa07e210 msgr2=0x562cfa11adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.918+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3554304878 shutdown_connections 2024-05-09T00:01:52.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.918+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3554304878 wait complete. 2024-05-09T00:01:52.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.918+0000 7f2b18c73b80 1 -- start start 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.918+0000 7f2b18c73b80 1 Processor -- start 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 0x562cfa134f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 0x562cfa132260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa1329a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa00c560 con 0x562cfa12e610 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa00c200 con 0x562cfa12ef30 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf9ef36a0 con 0x562cfa1290a0 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b18c73b80 10 client.0 _ll_register_callbacks cb 0x562cfa015b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa1329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa1329a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:46526/0 (socket says 172.21.15.12:46526) 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b06d9d700 1 -- 172.21.15.12:0/2596113866 learned_addr learned my addr 172.21.15.12:0/2596113866 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 0x562cfa132260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b0659c700 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 0x562cfa134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b05d9b700 1 -- 172.21.15.12:0/2596113866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 msgr2=0x562cfa134f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 0x562cfa134f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b05d9b700 1 -- 172.21.15.12:0/2596113866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 msgr2=0x562cfa1329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa1329a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:52 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2447184971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.919+0000 7f2b05d9b700 1 -- 172.21.15.12:0/2596113866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b08068450 con 0x562cfa12e610 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.920+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 0x562cfa132260 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009be0 tx=0x7f2afc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4af793805c47f4f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.920+0000 7f2aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.920+0000 7f2aedffb700 1 -- 172.21.15.12:0/2596113866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc026600 con 0x562cfa12e610 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.920+0000 7f2aedffb700 1 -- 172.21.15.12:0/2596113866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2afc026760 con 0x562cfa12e610 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.920+0000 7f2b18c73b80 1 -- 172.21.15.12:0/2596113866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cf9fc9270 con 0x562cfa12e610 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2aedffb700 1 -- 172.21.15.12:0/2596113866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2afc026920 con 0x562cfa12e610 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2b18c73b80 10 client.19803 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2b18c73b80 1 -- 172.21.15.12:0/2596113866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x562cf9e9db60 con 0x562cfa12e610 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2aedffb700 1 -- 172.21.15.12:0/2596113866 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562cf9fc9270 con 0x562cfa12e610 2024-05-09T00:01:52.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2aedffb700 1 -- 172.21.15.12:0/2596113866 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2afc024020 con 0x562cfa12e610 2024-05-09T00:01:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2b18c73b80 10 client.19803 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2b18c73b80 10 client.19803 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:01:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2b18c73b80 20 client.19803 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562cfa1536b0 0x562cfa155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2b18c73b80 4 client.19803 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.921+0000 7f2b18c73b80 1 -- 172.21.15.12:0/2596113866 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cfa0ef810 con 0x562cfa1536b0 2024-05-09T00:01:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.922+0000 7f2b0659c700 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562cfa1536b0 0x562cfa155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.922+0000 7f2aedffb700 10 client.19803 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:01:52.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.922+0000 7f2b0659c700 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562cfa1536b0 0x562cfa155ad0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9016a644bda37ca4 server_cookie=b6ae0bbc5e53a290 in_seq=0 out_seq=0 2024-05-09T00:01:52.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.928+0000 7f2aedffb700 1 -- 172.21.15.12:0/2596113866 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562cfa0ef810 con 0x562cfa1536b0 2024-05-09T00:01:52.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.928+0000 7f2aedffb700 10 client.19803 handle_command_reply: tid=0 2024-05-09T00:01:52.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.928+0000 7f2b18c73b80 1 client.19803 shutdown 2024-05-09T00:01:52.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.928+0000 7f2b18c73b80 1 -- 172.21.15.12:0/2596113866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 msgr2=0x562cfa132260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.928+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 0x562cfa132260 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009be0 tx=0x7f2afc0047c0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.929+0000 7f2b18c73b80 1 -- 172.21.15.12:0/2596113866 shutdown_connections 2024-05-09T00:01:52.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.929+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562cfa1536b0 0x562cfa155ad0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.929+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa1329a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.929+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa1290a0 0x562cfa134f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.929+0000 7f2aedffb700 0 client.19803 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:01:52.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.929+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/2596113866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12e610 0x562cfa132260 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.929+0000 7f2b18c73b80 1 -- 172.21.15.12:0/2596113866 >> 172.21.15.12:0/2596113866 conn(0x562cfa07e210 msgr2=0x562cfa11a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.930+0000 7f2b18c73b80 1 -- 172.21.15.12:0/2596113866 shutdown_connections 2024-05-09T00:01:52.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.930+0000 7f2b18c73b80 1 -- 172.21.15.12:0/2596113866 wait complete. 2024-05-09T00:01:52.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.931+0000 7f2b18c73b80 20 client.19803 trim_cache size 0 max 16384 2024-05-09T00:01:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.946+0000 7f2b18c73b80 1 Processor -- start 2024-05-09T00:01:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.946+0000 7f2b18c73b80 1 -- start start 2024-05-09T00:01:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa1290a0 0x562cfa128b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12ef30 0x562cfa131b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa1501f0 con 0x562cfa12ef30 2024-05-09T00:01:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa115a90 con 0x562cfa1290a0 2024-05-09T00:01:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa15ac80 con 0x562cfa12e610 2024-05-09T00:01:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b05d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa1290a0 0x562cfa128b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b05d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa14f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43280/0 (socket says 172.21.15.12:43280) 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa1290a0 0x562cfa128b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:46532/0 (socket says 172.21.15.12:46532) 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b05d9b700 1 -- 172.21.15.12:0/1375015291 learned_addr learned my addr 172.21.15.12:0/1375015291 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b06d9d700 1 --2- 172.21.15.12:0/1375015291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12ef30 0x562cfa131b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b05d9b700 1 -- 172.21.15.12:0/1375015291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa1290a0 msgr2=0x562cfa128b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/1375015291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa1290a0 0x562cfa128b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b05d9b700 1 -- 172.21.15.12:0/1375015291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12ef30 msgr2=0x562cfa131b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/1375015291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12ef30 0x562cfa131b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b05d9b700 1 -- 172.21.15.12:0/1375015291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cfa036560 con 0x562cfa12e610 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.947+0000 7f2b0659c700 1 --2- 172.21.15.12:0/1375015291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa1290a0 0x562cfa128b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:52.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.948+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/1375015291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa14f0c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009340 tx=0x7f2afc003d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b47caefef999d909 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.948+0000 7f2af57fa700 1 -- 172.21.15.12:0/1375015291 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc01e950 con 0x562cfa12e610 2024-05-09T00:01:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.948+0000 7f2af57fa700 1 -- 172.21.15.12:0/1375015291 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2afc01e3f0 con 0x562cfa12e610 2024-05-09T00:01:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.948+0000 7f2af57fa700 1 -- 172.21.15.12:0/1375015291 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc01c5e0 con 0x562cfa12e610 2024-05-09T00:01:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 -- 172.21.15.12:0/1375015291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 msgr2=0x562cfa14f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/1375015291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa14f0c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009340 tx=0x7f2afc003d10 comp rx=0 tx=0).stop 2024-05-09T00:01:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 -- 172.21.15.12:0/1375015291 shutdown_connections 2024-05-09T00:01:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/1375015291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa12ef30 0x562cfa131b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/1375015291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa1290a0 0x562cfa128b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/1375015291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa14f0c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 -- 172.21.15.12:0/1375015291 >> 172.21.15.12:0/1375015291 conn(0x562cfa07e210 msgr2=0x562cfa129480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 -- 172.21.15.12:0/1375015291 shutdown_connections 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 -- 172.21.15.12:0/1375015291 wait complete. 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 -- start start 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.949+0000 7f2b18c73b80 1 Processor -- start 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa1290a0 0x562cfa14aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa12b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b18c73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa12b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa12a140 con 0x562cfa1290a0 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa12a2b0 con 0x562cfa12ef30 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b18c73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cfa12a470 con 0x562cfa12e610 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b18c73b80 10 client.0 _ll_register_callbacks cb 0x562cfa015b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa1290a0 0x562cfa14aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa1290a0 0x562cfa14aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53260/0 (socket says 172.21.15.12:53260) 2024-05-09T00:01:52.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b0659c700 1 -- 172.21.15.12:0/3007932811 learned_addr learned my addr 172.21.15.12:0/3007932811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa12b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b06d9d700 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa12b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b0659c700 1 -- 172.21.15.12:0/3007932811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 msgr2=0x562cfa12b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b0659c700 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa12b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b0659c700 1 -- 172.21.15.12:0/3007932811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 msgr2=0x562cfa12b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b0659c700 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa12b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b0659c700 1 -- 172.21.15.12:0/3007932811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cf9ef13b0 con 0x562cfa1290a0 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa12b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2b0659c700 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa1290a0 0x562cfa14aaa0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2af80cb670 tx=0x7f2af8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46adb4c251b6d52c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.950+0000 7f2aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2aedffb700 1 -- 172.21.15.12:0/3007932811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af80a3400 con 0x562cfa1290a0 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2aedffb700 1 -- 172.21.15.12:0/3007932811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2af809a500 con 0x562cfa1290a0 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3007932811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2af809b010 con 0x562cfa1290a0 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2aedffb700 1 -- 172.21.15.12:0/3007932811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2af809a6c0 con 0x562cfa1290a0 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2b18c73b80 10 client.19809 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:52.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3007932811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x562cf9e9db60 con 0x562cfa1290a0 2024-05-09T00:01:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2aedffb700 1 -- 172.21.15.12:0/3007932811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2af809b010 con 0x562cfa1290a0 2024-05-09T00:01:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2aedffb700 1 -- 172.21.15.12:0/3007932811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2af807c020 con 0x562cfa1290a0 2024-05-09T00:01:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2b18c73b80 10 client.19809 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2b18c73b80 10 client.19809 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:01:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2b18c73b80 20 client.19809 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562cfa1aa0f0 0x562cfa1ac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2b18c73b80 4 client.19809 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:01:52.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.951+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3007932811 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562cfa068ec0 con 0x562cfa1aa0f0 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.954+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562cfa1aa0f0 0x562cfa1ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.954+0000 7f2b05d9b700 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562cfa1aa0f0 0x562cfa1ac510 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f9e481682a5b245 server_cookie=aa11787d19b786c5 in_seq=0 out_seq=0 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.954+0000 7f2aedffb700 10 client.19809 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.956+0000 7f2aedffb700 1 -- 172.21.15.12:0/3007932811 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562cfa068ec0 con 0x562cfa1aa0f0 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.956+0000 7f2aedffb700 10 client.19809 handle_command_reply: tid=0 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.956+0000 7f2b18c73b80 1 client.19809 shutdown 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.956+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3007932811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa1290a0 msgr2=0x562cfa14aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.956+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa1290a0 0x562cfa14aaa0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2af80cb670 tx=0x7f2af8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.957+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3007932811 shutdown_connections 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.957+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x562cfa1aa0f0 0x562cfa1ac510 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.957+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562cfa12ef30 0x562cfa12b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.957+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562cfa1290a0 0x562cfa14aaa0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.957+0000 7f2aedffb700 0 client.19809 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:01:52.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.957+0000 7f2b18c73b80 1 --2- 172.21.15.12:0/3007932811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562cfa12e610 0x562cfa12b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.957+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3007932811 >> 172.21.15.12:0/3007932811 conn(0x562cfa07e210 msgr2=0x562cfa11c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.957+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3007932811 shutdown_connections 2024-05-09T00:01:52.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.957+0000 7f2b18c73b80 1 -- 172.21.15.12:0/3007932811 wait complete. 2024-05-09T00:01:52.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.957+0000 7f2b18c73b80 20 client.19809 trim_cache size 0 max 16384 2024-05-09T00:01:52.953 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:01:52.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.958+0000 7f2b0cab9700 1 -- 172.21.15.12:0/3967097308 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2af00a1310 msgr2=0x7f2af00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.958+0000 7f2b0cab9700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2af00a1310 0x7f2af00a37d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2b08197820 tx=0x7f2afc009260 comp rx=0 tx=0).stop 2024-05-09T00:01:52.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.958+0000 7f2b0cab9700 1 -- 172.21.15.12:0/3967097308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b08100060 msgr2=0x7f2b081962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:52.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.958+0000 7f2b0cab9700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b08100060 0x7f2b081962d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2af80059d0 tx=0x7f2af8004730 comp rx=0 tx=0).stop 2024-05-09T00:01:52.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.958+0000 7f2b0cab9700 1 -- 172.21.15.12:0/3967097308 shutdown_connections 2024-05-09T00:01:52.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.958+0000 7f2b0cab9700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2af00a1310 0x7f2af00a37d0 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2b08197820 tx=0x7f2afc009260 comp rx=0 tx=0).stop 2024-05-09T00:01:52.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.958+0000 7f2b0cab9700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b0819abb0 0x7f2b0819b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.958+0000 7f2b0cab9700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b08100060 0x7f2b081962d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.958+0000 7f2b0cab9700 1 --2- 172.21.15.12:0/3967097308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b08105160 0x7f2b08196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:52.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.958+0000 7f2b0cab9700 1 -- 172.21.15.12:0/3967097308 >> 172.21.15.12:0/3967097308 conn(0x7f2b08076f40 msgr2=0x7f2b080fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:52.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.960+0000 7f2b0cab9700 1 -- 172.21.15.12:0/3967097308 shutdown_connections 2024-05-09T00:01:52.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:52.960+0000 7f2b0cab9700 1 -- 172.21.15.12:0/3967097308 wait complete. 2024-05-09T00:01:52.969 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:01:53.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:52 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2447184971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:01:53.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:52 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2447184971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:01:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.158+0000 7f611ffdd700 1 -- 172.21.15.12:0/226120301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6118111d80 msgr2=0x7f6118114170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.158+0000 7f611ffdd700 1 --2- 172.21.15.12:0/226120301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6118111d80 0x7f6118114170 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f611000b0a0 tx=0x7f611000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.158+0000 7f611ffdd700 1 -- 172.21.15.12:0/226120301 shutdown_connections 2024-05-09T00:01:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.158+0000 7f611ffdd700 1 --2- 172.21.15.12:0/226120301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6118111d80 0x7f6118114170 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.158+0000 7f611ffdd700 1 --2- 172.21.15.12:0/226120301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61180754d0 0x7f61180758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.158+0000 7f611ffdd700 1 --2- 172.21.15.12:0/226120301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6118075df0 0x7f6118111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.158+0000 7f611ffdd700 1 -- 172.21.15.12:0/226120301 >> 172.21.15.12:0/226120301 conn(0x7f61180febb0 msgr2=0x7f6118100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.158+0000 7f611ffdd700 1 -- 172.21.15.12:0/226120301 shutdown_connections 2024-05-09T00:01:53.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.158+0000 7f611ffdd700 1 -- 172.21.15.12:0/226120301 wait complete. 2024-05-09T00:01:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.159+0000 7f611ffdd700 1 Processor -- start 2024-05-09T00:01:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.159+0000 7f611ffdd700 1 -- start start 2024-05-09T00:01:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.159+0000 7f611ffdd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61180754d0 0x7f611819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.159+0000 7f611ffdd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6118075df0 0x7f611819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611ffdd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6118111d80 0x7f61181a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611ffdd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61181a2900 con 0x7f6118075df0 2024-05-09T00:01:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611ffdd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61181a2a70 con 0x7f61180754d0 2024-05-09T00:01:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611ffdd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61181a2c30 con 0x7f6118111d80 2024-05-09T00:01:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611d578700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6118075df0 0x7f611819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611d578700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6118075df0 0x7f611819dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53282/0 (socket says 172.21.15.12:53282) 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611d578700 1 -- 172.21.15.12:0/196484018 learned_addr learned my addr 172.21.15.12:0/196484018 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611dd79700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61180754d0 0x7f611819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611e57a700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6118111d80 0x7f61181a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611d578700 1 -- 172.21.15.12:0/196484018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6118111d80 msgr2=0x7f61181a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611d578700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6118111d80 0x7f61181a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611d578700 1 -- 172.21.15.12:0/196484018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61180754d0 msgr2=0x7f611819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611d578700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61180754d0 0x7f611819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611d578700 1 -- 172.21.15.12:0/196484018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6110009d00 con 0x7f6118075df0 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f611d578700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6118075df0 0x7f611819dfb0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f610800e9a0 tx=0x7f610800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.160+0000 7f610effd700 1 -- 172.21.15.12:0/196484018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108009790 con 0x7f6118075df0 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.161+0000 7f611ffdd700 1 -- 172.21.15.12:0/196484018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61181a2f10 con 0x7f6118075df0 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.161+0000 7f610effd700 1 -- 172.21.15.12:0/196484018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6108010430 con 0x7f6118075df0 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.161+0000 7f610effd700 1 -- 172.21.15.12:0/196484018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108018680 con 0x7f6118075df0 2024-05-09T00:01:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.161+0000 7f611ffdd700 1 -- 172.21.15.12:0/196484018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61181aa7c0 con 0x7f6118075df0 2024-05-09T00:01:53.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.163+0000 7f610effd700 1 -- 172.21.15.12:0/196484018 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f61080105a0 con 0x7f6118075df0 2024-05-09T00:01:53.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.164+0000 7f610effd700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61040a1290 0x7f61040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.164+0000 7f610effd700 1 -- 172.21.15.12:0/196484018 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6108014070 con 0x7f6118075df0 2024-05-09T00:01:53.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.165+0000 7f612c197b80 1 Processor -- start 2024-05-09T00:01:53.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.165+0000 7f611dd79700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61040a1290 0x7f61040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.165+0000 7f612c197b80 1 -- start start 2024-05-09T00:01:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.165+0000 7f612c197b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x5624014621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.165+0000 7f612c197b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562401462720 0x56240145c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.165+0000 7f612c197b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145c890 0x56240145ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.165+0000 7f612c197b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624013b3150 con 0x562401462720 2024-05-09T00:01:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.165+0000 7f612c197b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624013b0b10 con 0x562401461e00 2024-05-09T00:01:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.165+0000 7f612c197b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624013ac400 con 0x56240145c890 2024-05-09T00:01:53.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.165+0000 7f611e57a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145c890 0x56240145ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611dd79700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x5624014621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611e57a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145c890 0x56240145ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43312/0 (socket says 172.21.15.12:43312) 2024-05-09T00:01:53.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611dd79700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x5624014621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:46568/0 (socket says 172.21.15.12:46568) 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611e57a700 1 -- 172.21.15.12:0/2808229450 learned_addr learned my addr 172.21.15.12:0/2808229450 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611dd79700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61040a1290 0x7f61040a3750 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6114006fd0 tx=0x7f6114009190 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611dd79700 1 -- 172.21.15.12:0/2808229450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145c890 msgr2=0x56240145ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611dd79700 1 --2- 172.21.15.12:0/2808229450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145c890 0x56240145ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611dd79700 1 -- 172.21.15.12:0/2808229450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562401462720 msgr2=0x56240145c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611dd79700 1 --2- 172.21.15.12:0/2808229450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562401462720 0x56240145c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611dd79700 1 -- 172.21.15.12:0/2808229450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56240122d3b0 con 0x562401461e00 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f611dd79700 1 --2- 172.21.15.12:0/2808229450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x5624014621e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f611400e930 tx=0x7f6114011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66121a8444cee719 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.166+0000 7f6103fff700 1 -- 172.21.15.12:0/2808229450 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114015480 con 0x562401461e00 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.167+0000 7f6103fff700 1 -- 172.21.15.12:0/2808229450 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6114015aa0 con 0x562401461e00 2024-05-09T00:01:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.167+0000 7f6103fff700 1 -- 172.21.15.12:0/2808229450 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114013b00 con 0x562401461e00 2024-05-09T00:01:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.168+0000 7f612c197b80 1 -- 172.21.15.12:0/2808229450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 msgr2=0x5624014621e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.168+0000 7f612c197b80 1 --2- 172.21.15.12:0/2808229450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x5624014621e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f611400e930 tx=0x7f6114011000 comp rx=0 tx=0).stop 2024-05-09T00:01:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.168+0000 7f612c197b80 1 -- 172.21.15.12:0/2808229450 shutdown_connections 2024-05-09T00:01:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.168+0000 7f612c197b80 1 --2- 172.21.15.12:0/2808229450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145c890 0x56240145ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.168+0000 7f612c197b80 1 --2- 172.21.15.12:0/2808229450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x5624014621e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.168+0000 7f612c197b80 1 --2- 172.21.15.12:0/2808229450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562401462720 0x56240145c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.168+0000 7f612c197b80 1 -- 172.21.15.12:0/2808229450 >> 172.21.15.12:0/2808229450 conn(0x5624013ba210 msgr2=0x562401456df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.169+0000 7f612c197b80 1 -- 172.21.15.12:0/2808229450 shutdown_connections 2024-05-09T00:01:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.169+0000 7f612c197b80 1 -- 172.21.15.12:0/2808229450 wait complete. 2024-05-09T00:01:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.169+0000 7f612c197b80 1 -- start start 2024-05-09T00:01:53.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.169+0000 7f612c197b80 1 Processor -- start 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f612c197b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x562401465b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f612c197b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401461e00 0x5624014660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f612c197b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401462720 0x562401466600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f612c197b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401348560 con 0x56240145c890 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f612c197b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401348200 con 0x562401462720 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f612c197b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56240122f6a0 con 0x562401461e00 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f612c197b80 10 client.0 _ll_register_callbacks cb 0x562401351b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f611dd79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x562401465b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f611dd79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x562401465b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53302/0 (socket says 172.21.15.12:53302) 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f611dd79700 1 -- 172.21.15.12:0/114186886 learned_addr learned my addr 172.21.15.12:0/114186886 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f611d578700 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401461e00 0x5624014660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f611e57a700 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401462720 0x562401466600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f611d578700 1 -- 172.21.15.12:0/114186886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401462720 msgr2=0x562401466600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f611d578700 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401462720 0x562401466600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f611d578700 1 -- 172.21.15.12:0/114186886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 msgr2=0x562401465b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f611d578700 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x562401465b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.170+0000 7f611d578700 1 -- 172.21.15.12:0/114186886 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611400e6c0 con 0x562401461e00 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f611d578700 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401461e00 0x5624014660c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f61080ca600 tx=0x7f6108099a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ed1ebb2544be0f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f61017fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f61017fa700 1 -- 172.21.15.12:0/114186886 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61080996b0 con 0x562401461e00 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f612c197b80 1 -- 172.21.15.12:0/114186886 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562401305270 con 0x562401461e00 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f61017fa700 1 -- 172.21.15.12:0/114186886 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f610809ddf0 con 0x562401461e00 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f61017fa700 1 -- 172.21.15.12:0/114186886 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61080a2440 con 0x562401461e00 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f612c197b80 10 client.29554 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:53.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f612c197b80 1 -- 172.21.15.12:0/114186886 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5624011d9b60 con 0x562401461e00 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f61017fa700 1 -- 172.21.15.12:0/114186886 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562401305270 con 0x562401461e00 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f61017fa700 1 -- 172.21.15.12:0/114186886 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f61080b2b70 con 0x562401461e00 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f612c197b80 10 client.29554 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f612c197b80 10 client.29554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.171+0000 7f612c197b80 20 client.29554 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.172+0000 7f612c197b80 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5624014761c0 0x5624014785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.172+0000 7f612c197b80 4 client.29554 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.172+0000 7f612c197b80 1 -- 172.21.15.12:0/114186886 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56240142b810 con 0x5624014761c0 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.172+0000 7f611dd79700 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5624014761c0 0x5624014785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.172+0000 7f611dd79700 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5624014761c0 0x5624014785e0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143baf3063e6cbed server_cookie=22157ce0d91da0a7 in_seq=0 out_seq=0 2024-05-09T00:01:53.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.172+0000 7f61017fa700 10 client.29554 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:53.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f61017fa700 1 -- 172.21.15.12:0/114186886 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56240142b810 con 0x5624014761c0 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f61017fa700 10 client.29554 handle_command_reply: tid=0 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 client.29554 shutdown 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 -- 172.21.15.12:0/114186886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401461e00 msgr2=0x5624014660c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401461e00 0x5624014660c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f61080ca600 tx=0x7f6108099a10 comp rx=0 tx=0).stop 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 -- 172.21.15.12:0/114186886 shutdown_connections 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5624014761c0 0x5624014785e0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401462720 0x562401466600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x562401465b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 --2- 172.21.15.12:0/114186886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401461e00 0x5624014660c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 -- 172.21.15.12:0/114186886 >> 172.21.15.12:0/114186886 conn(0x5624013ba210 msgr2=0x5624014565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f61017fa700 0 client.29554 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 -- 172.21.15.12:0/114186886 shutdown_connections 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 1 -- 172.21.15.12:0/114186886 wait complete. 2024-05-09T00:01:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.175+0000 7f612c197b80 20 client.29554 trim_cache size 0 max 16384 2024-05-09T00:01:53.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f612c197b80 1 Processor -- start 2024-05-09T00:01:53.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f612c197b80 1 -- start start 2024-05-09T00:01:53.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f612c197b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240145c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f612c197b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x562401461350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f612c197b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401462720 0x562401456d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f612c197b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624014625a0 con 0x56240145c890 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f612c197b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401461890 con 0x562401461e00 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f612c197b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401472d70 con 0x562401462720 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f611dd79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240145c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f611d578700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x562401461350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f611dd79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240145c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53308/0 (socket says 172.21.15.12:53308) 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f611dd79700 1 -- 172.21.15.12:0/4264966159 learned_addr learned my addr 172.21.15.12:0/4264966159 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.192+0000 7f611e57a700 1 --2- 172.21.15.12:0/4264966159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401462720 0x562401456d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.193+0000 7f611dd79700 1 -- 172.21.15.12:0/4264966159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401462720 msgr2=0x562401456d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.193+0000 7f611dd79700 1 --2- 172.21.15.12:0/4264966159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401462720 0x562401456d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.193+0000 7f611dd79700 1 -- 172.21.15.12:0/4264966159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 msgr2=0x562401461350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:53.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.193+0000 7f611dd79700 1 --2- 172.21.15.12:0/4264966159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x562401461350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.193+0000 7f611dd79700 1 -- 172.21.15.12:0/4264966159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562401372560 con 0x56240145c890 2024-05-09T00:01:53.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.193+0000 7f611dd79700 1 --2- 172.21.15.12:0/4264966159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240145c350 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f611400e4b0 tx=0x7f6114009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d64bbfb0b506901 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:53.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.193+0000 7f610cff9700 1 -- 172.21.15.12:0/4264966159 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114015480 con 0x56240145c890 2024-05-09T00:01:53.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.193+0000 7f610cff9700 1 -- 172.21.15.12:0/4264966159 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6114031850 con 0x56240145c890 2024-05-09T00:01:53.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.193+0000 7f610cff9700 1 -- 172.21.15.12:0/4264966159 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61140043c0 con 0x56240145c890 2024-05-09T00:01:53.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.194+0000 7f612c197b80 1 -- 172.21.15.12:0/4264966159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 msgr2=0x56240145c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.194+0000 7f612c197b80 1 --2- 172.21.15.12:0/4264966159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240145c350 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f611400e4b0 tx=0x7f6114009200 comp rx=0 tx=0).stop 2024-05-09T00:01:53.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.194+0000 7f612c197b80 1 -- 172.21.15.12:0/4264966159 shutdown_connections 2024-05-09T00:01:53.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.194+0000 7f612c197b80 1 --2- 172.21.15.12:0/4264966159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401462720 0x562401456d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.194+0000 7f612c197b80 1 --2- 172.21.15.12:0/4264966159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240145c350 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.194+0000 7f612c197b80 1 --2- 172.21.15.12:0/4264966159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x562401461350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.194+0000 7f612c197b80 1 -- 172.21.15.12:0/4264966159 >> 172.21.15.12:0/4264966159 conn(0x5624013ba210 msgr2=0x562401460c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:53.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.194+0000 7f612c197b80 1 -- 172.21.15.12:0/4264966159 shutdown_connections 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.194+0000 7f612c197b80 1 -- 172.21.15.12:0/4264966159 wait complete. 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.195+0000 7f612c197b80 1 -- start start 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.195+0000 7f612c197b80 1 Processor -- start 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.195+0000 7f612c197b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240146b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.195+0000 7f612c197b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x56240146b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.195+0000 7f612c197b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401462720 0x5624014cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.195+0000 7f612c197b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56240146bfa0 con 0x56240145c890 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.195+0000 7f612c197b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624014cbbc0 con 0x562401461e00 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.195+0000 7f612c197b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624014cbd30 con 0x562401462720 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.195+0000 7f612c197b80 10 client.0 _ll_register_callbacks cb 0x562401351b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.195+0000 7f611dd79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240146b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611dd79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240146b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53322/0 (socket says 172.21.15.12:53322) 2024-05-09T00:01:53.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611dd79700 1 -- 172.21.15.12:0/1454417099 learned_addr learned my addr 172.21.15.12:0/1454417099 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611d578700 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x56240146b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611e57a700 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401462720 0x5624014cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611dd79700 1 -- 172.21.15.12:0/1454417099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401462720 msgr2=0x5624014cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611dd79700 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401462720 0x5624014cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611dd79700 1 -- 172.21.15.12:0/1454417099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 msgr2=0x56240146b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611dd79700 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x56240146b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611dd79700 1 -- 172.21.15.12:0/1454417099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56240122d3b0 con 0x56240145c890 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611d578700 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x56240146b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f611dd79700 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240146b150 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6114006f00 tx=0x7f6114006da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1f10ed90a88493 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f61017fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f61017fa700 1 -- 172.21.15.12:0/1454417099 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114031590 con 0x56240145c890 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f61017fa700 1 -- 172.21.15.12:0/1454417099 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f611402e040 con 0x56240145c890 2024-05-09T00:01:53.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.196+0000 7f612c197b80 1 -- 172.21.15.12:0/1454417099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6114015aa0 con 0x56240145c890 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f61017fa700 1 -- 172.21.15.12:0/1454417099 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61140316f0 con 0x56240145c890 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f612c197b80 10 client.19824 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f612c197b80 1 -- 172.21.15.12:0/1454417099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x562401231cd0 con 0x56240145c890 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f61017fa700 1 -- 172.21.15.12:0/1454417099 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6114015aa0 con 0x56240145c890 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f61017fa700 1 -- 172.21.15.12:0/1454417099 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f611403a070 con 0x56240145c890 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f612c197b80 10 client.19824 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f612c197b80 10 client.19824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f612c197b80 20 client.19824 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f612c197b80 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5624014cd050 0x5624014cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f612c197b80 4 client.19824 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:01:53.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.197+0000 7f612c197b80 1 -- 172.21.15.12:0/1454417099 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5624013a4ec0 con 0x5624014cd050 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.198+0000 7f611d578700 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5624014cd050 0x5624014cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.199+0000 7f61017fa700 10 client.19824 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.199+0000 7f611d578700 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5624014cd050 0x5624014cf470 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f738849d62caf7 server_cookie=a80c4716dbf29bc in_seq=0 out_seq=0 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.201+0000 7f61017fa700 1 -- 172.21.15.12:0/1454417099 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5624013a4ec0 con 0x5624014cd050 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.201+0000 7f61017fa700 10 client.19824 handle_command_reply: tid=0 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.201+0000 7f612c197b80 1 client.19824 shutdown 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.201+0000 7f612c197b80 1 -- 172.21.15.12:0/1454417099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 msgr2=0x56240146b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.201+0000 7f612c197b80 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240146b150 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6114006f00 tx=0x7f6114006da0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.202+0000 7f612c197b80 1 -- 172.21.15.12:0/1454417099 shutdown_connections 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.202+0000 7f612c197b80 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562401462720 0x5624014cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.202+0000 7f612c197b80 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56240145c890 0x56240146b150 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.202+0000 7f612c197b80 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5624014cd050 0x5624014cf470 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.202+0000 7f612c197b80 1 --2- 172.21.15.12:0/1454417099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562401461e00 0x56240146b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.202+0000 7f612c197b80 1 -- 172.21.15.12:0/1454417099 >> 172.21.15.12:0/1454417099 conn(0x5624013ba210 msgr2=0x562401464450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:53.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.202+0000 7f61017fa700 0 client.19824 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:53.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.202+0000 7f612c197b80 1 -- 172.21.15.12:0/1454417099 shutdown_connections 2024-05-09T00:01:53.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.202+0000 7f612c197b80 1 -- 172.21.15.12:0/1454417099 wait complete. 2024-05-09T00:01:53.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.202+0000 7f612c197b80 20 client.19824 trim_cache size 0 max 16384 2024-05-09T00:01:53.198 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:01:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 -- 172.21.15.12:0/196484018 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61040a1290 msgr2=0x7f61040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61040a1290 0x7f61040a3750 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6114006fd0 tx=0x7f6114009190 comp rx=0 tx=0).stop 2024-05-09T00:01:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 -- 172.21.15.12:0/196484018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6118075df0 msgr2=0x7f611819dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6118075df0 0x7f611819dfb0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f610800e9a0 tx=0x7f610800ecb0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 -- 172.21.15.12:0/196484018 shutdown_connections 2024-05-09T00:01:53.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61040a1290 0x7f61040a3750 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6118111d80 0x7f61181a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61180754d0 0x7f611819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 --2- 172.21.15.12:0/196484018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6118075df0 0x7f611819dfb0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 -- 172.21.15.12:0/196484018 >> 172.21.15.12:0/196484018 conn(0x7f61180febb0 msgr2=0x7f6118100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 -- 172.21.15.12:0/196484018 shutdown_connections 2024-05-09T00:01:53.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:53.203+0000 7f611ffdd700 1 -- 172.21.15.12:0/196484018 wait complete. 2024-05-09T00:01:53.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 79989 41.50 MB/sec warmup 108 sec latency 53.625 ms 2024-05-09T00:01:53.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:53 smithi012 ceph-mon[42503]: pgmap v496: 97 pgs: 97 active+clean; 770 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-09T00:01:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:53 smithi204 ceph-mon[47373]: pgmap v496: 97 pgs: 97 active+clean; 770 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-09T00:01:54.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:53 smithi160 ceph-mon[46152]: pgmap v496: 97 pgs: 97 active+clean; 770 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-09T00:01:54.214 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:01:54.215 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:01:54.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 80834 41.58 MB/sec warmup 109 sec latency 38.700 ms 2024-05-09T00:01:54.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.405+0000 7feab5c93700 1 -- 172.21.15.12:0/2336033458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab0101080 msgr2=0x7feab0101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.405+0000 7feab5c93700 1 --2- 172.21.15.12:0/2336033458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab0101080 0x7feab0101460 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009990 tx=0x7feaa0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.406+0000 7feab5c93700 1 -- 172.21.15.12:0/2336033458 shutdown_connections 2024-05-09T00:01:54.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.406+0000 7feab5c93700 1 --2- 172.21.15.12:0/2336033458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab01061a0 0x7feab010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.406+0000 7feab5c93700 1 --2- 172.21.15.12:0/2336033458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab0101080 0x7feab0101460 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.406+0000 7feab5c93700 1 --2- 172.21.15.12:0/2336033458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feab0101a30 0x7feab0105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.406+0000 7feab5c93700 1 -- 172.21.15.12:0/2336033458 >> 172.21.15.12:0/2336033458 conn(0x7feab00fc940 msgr2=0x7feab00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.406+0000 7feab5c93700 1 -- 172.21.15.12:0/2336033458 shutdown_connections 2024-05-09T00:01:54.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.406+0000 7feab5c93700 1 -- 172.21.15.12:0/2336033458 wait complete. 2024-05-09T00:01:54.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.406+0000 7feab5c93700 1 Processor -- start 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feab5c93700 1 -- start start 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feab5c93700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab0101080 0x7feab00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feab5c93700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab01061a0 0x7feab00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feab5c93700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feab0073d20 0x7feab01a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feab5c93700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a5b60 con 0x7feab0101080 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feab5c93700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a5cd0 con 0x7feab01061a0 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feab5c93700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a5e90 con 0x7feab0073d20 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feaaf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab0101080 0x7feab00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feaaf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab0101080 0x7feab00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53338/0 (socket says 172.21.15.12:53338) 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feaaf7fe700 1 -- 172.21.15.12:0/3437654181 learned_addr learned my addr 172.21.15.12:0/3437654181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab01061a0 0x7feab00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.407+0000 7feaaffff700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feab0073d20 0x7feab01a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feaaf7fe700 1 -- 172.21.15.12:0/3437654181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feab0073d20 msgr2=0x7feab01a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feab0073d20 0x7feab01a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feaaf7fe700 1 -- 172.21.15.12:0/3437654181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab01061a0 msgr2=0x7feab00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab01061a0 0x7feab00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feaaf7fe700 1 -- 172.21.15.12:0/3437654181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00095f0 con 0x7feab0101080 2024-05-09T00:01:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab0101080 0x7feab00732a0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7feaa0005910 tx=0x7feaa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feaad7fa700 1 -- 172.21.15.12:0/3437654181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa001d070 con 0x7feab0101080 2024-05-09T00:01:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feab5c93700 1 -- 172.21.15.12:0/3437654181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab01a6110 con 0x7feab0101080 2024-05-09T00:01:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feaad7fa700 1 -- 172.21.15.12:0/3437654181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa0022470 con 0x7feab0101080 2024-05-09T00:01:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feaad7fa700 1 -- 172.21.15.12:0/3437654181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0017650 con 0x7feab0101080 2024-05-09T00:01:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.408+0000 7feab5c93700 1 -- 172.21.15.12:0/3437654181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab01a6530 con 0x7feab0101080 2024-05-09T00:01:54.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.412+0000 7feaad7fa700 1 -- 172.21.15.12:0/3437654181 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7feaa00225e0 con 0x7feab0101080 2024-05-09T00:01:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.413+0000 7feaad7fa700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fea9c0a1280 0x7fea9c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.413+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fea9c0a1280 0x7fea9c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.413+0000 7feaad7fa700 1 -- 172.21.15.12:0/3437654181 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7feaa00cdec0 con 0x7feab0101080 2024-05-09T00:01:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.413+0000 7feac1e4db80 1 Processor -- start 2024-05-09T00:01:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.413+0000 7feac1e4db80 1 -- start start 2024-05-09T00:01:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.413+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267fa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.413+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 0x560d267f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.413+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.413+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fea9c0a1280 0x7fea9c0a3740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feab00749d0 tx=0x7fea98009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feac1e4db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d2674b1d0 con 0x560d267f4910 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feac1e4db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d26748b90 con 0x560d267f9e80 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feac1e4db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d26744480 con 0x560d267fa7a0 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaf7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaf7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267fa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:46632/0 (socket says 172.21.15.12:46632) 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaf7fe700 1 -- 172.21.15.12:0/986459949 learned_addr learned my addr 172.21.15.12:0/986459949 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/986459949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 0x560d267f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaf7fe700 1 -- 172.21.15.12:0/986459949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 msgr2=0x560d267f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/986459949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 0x560d267f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaf7fe700 1 -- 172.21.15.12:0/986459949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 msgr2=0x560d267f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/986459949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaf7fe700 1 -- 172.21.15.12:0/986459949 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d265c5430 con 0x560d267f9e80 2024-05-09T00:01:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaffff700 1 --2- 172.21.15.12:0/986459949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/986459949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 0x560d267f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/986459949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267fa260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feaa00975a0 tx=0x7feaa0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8532f5691451ff86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.414+0000 7feaa5ffb700 1 -- 172.21.15.12:0/986459949 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa009b7a0 con 0x560d267f9e80 2024-05-09T00:01:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.415+0000 7feaa5ffb700 1 -- 172.21.15.12:0/986459949 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa009b0a0 con 0x560d267f9e80 2024-05-09T00:01:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.415+0000 7feaa5ffb700 1 -- 172.21.15.12:0/986459949 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa00977b0 con 0x560d267f9e80 2024-05-09T00:01:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.415+0000 7feac1e4db80 1 -- 172.21.15.12:0/986459949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 msgr2=0x560d267fa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.415+0000 7feac1e4db80 1 --2- 172.21.15.12:0/986459949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267fa260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feaa00975a0 tx=0x7feaa0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.415+0000 7feac1e4db80 1 -- 172.21.15.12:0/986459949 shutdown_connections 2024-05-09T00:01:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.415+0000 7feac1e4db80 1 --2- 172.21.15.12:0/986459949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.415+0000 7feac1e4db80 1 --2- 172.21.15.12:0/986459949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267fa260 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.415+0000 7feac1e4db80 1 --2- 172.21.15.12:0/986459949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 0x560d267f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.415+0000 7feac1e4db80 1 -- 172.21.15.12:0/986459949 >> 172.21.15.12:0/986459949 conn(0x560d26752290 msgr2=0x560d267eee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.415+0000 7feac1e4db80 1 -- 172.21.15.12:0/986459949 shutdown_connections 2024-05-09T00:01:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.416+0000 7feac1e4db80 1 -- 172.21.15.12:0/986459949 wait complete. 2024-05-09T00:01:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.416+0000 7feac1e4db80 1 -- start start 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.416+0000 7feac1e4db80 1 Processor -- start 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f9e80 0x560d267fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267fa7a0 0x560d267fe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feac1e4db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d266e05e0 con 0x560d267f4910 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feac1e4db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d266e0280 con 0x560d267fa7a0 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feac1e4db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d265c7720 con 0x560d267f9e80 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feac1e4db80 10 client.0 _ll_register_callbacks cb 0x560d266e9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaaf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaa6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f9e80 0x560d267fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaaf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267fdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53350/0 (socket says 172.21.15.12:53350) 2024-05-09T00:01:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaaf7fe700 1 -- 172.21.15.12:0/1806110475 learned_addr learned my addr 172.21.15.12:0/1806110475 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaaf7fe700 1 -- 172.21.15.12:0/1806110475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f9e80 msgr2=0x560d267fe0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f9e80 0x560d267fe0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaaf7fe700 1 -- 172.21.15.12:0/1806110475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267fa7a0 msgr2=0x560d267fe630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267fa7a0 0x560d267fe630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaaf7fe700 1 -- 172.21.15.12:0/1806110475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea98017070 con 0x560d267f4910 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f9e80 0x560d267fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.417+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267fdbb0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7feaa0096e20 tx=0x7feaa0022bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b083697cc4fa3762 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7fea8effd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7fea8effd700 1 -- 172.21.15.12:0/1806110475 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa009b7a0 con 0x560d267f4910 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7fea8effd700 1 -- 172.21.15.12:0/1806110475 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7feaa009eed0 con 0x560d267f4910 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7feac1e4db80 1 -- 172.21.15.12:0/1806110475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d2669d2f0 con 0x560d267f4910 2024-05-09T00:01:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7fea8effd700 1 -- 172.21.15.12:0/1806110475 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaa00a3490 con 0x560d267f4910 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7feac1e4db80 10 client.19836 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7feac1e4db80 1 -- 172.21.15.12:0/1806110475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7feab01a6530 con 0x560d267f4910 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7fea8effd700 1 -- 172.21.15.12:0/1806110475 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d2669d2f0 con 0x560d267f4910 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7fea8effd700 1 -- 172.21.15.12:0/1806110475 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feaa00a3000 con 0x560d267f4910 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7feac1e4db80 10 client.19836 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7feac1e4db80 10 client.19836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7feac1e4db80 20 client.19836 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.418+0000 7feac1e4db80 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560d2680dac0 0x560d2680fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.419+0000 7feac1e4db80 4 client.19836 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.419+0000 7feac1e4db80 1 -- 172.21.15.12:0/1806110475 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d267c3890 con 0x560d2680dac0 2024-05-09T00:01:54.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.419+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560d2680dac0 0x560d2680fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.419+0000 7fea8effd700 10 client.19836 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:54.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.419+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560d2680dac0 0x560d2680fee0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f51e9bb984a3d21 server_cookie=e090c2387d3e84ab in_seq=0 out_seq=0 2024-05-09T00:01:54.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7fea8effd700 1 -- 172.21.15.12:0/1806110475 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d267c3890 con 0x560d2680dac0 2024-05-09T00:01:54.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7fea8effd700 10 client.19836 handle_command_reply: tid=0 2024-05-09T00:01:54.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 client.19836 shutdown 2024-05-09T00:01:54.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 -- 172.21.15.12:0/1806110475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 msgr2=0x560d267fdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267fdbb0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7feaa0096e20 tx=0x7feaa0022bd0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 -- 172.21.15.12:0/1806110475 shutdown_connections 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560d2680dac0 0x560d2680fee0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267fa7a0 0x560d267fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267fdbb0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 --2- 172.21.15.12:0/1806110475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f9e80 0x560d267fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 -- 172.21.15.12:0/1806110475 >> 172.21.15.12:0/1806110475 conn(0x560d26752290 msgr2=0x560d267ee620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7fea8effd700 0 client.19836 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 -- 172.21.15.12:0/1806110475 shutdown_connections 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.421+0000 7feac1e4db80 1 -- 172.21.15.12:0/1806110475 wait complete. 2024-05-09T00:01:54.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.422+0000 7feac1e4db80 20 client.19836 trim_cache size 0 max 16384 2024-05-09T00:01:54.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.442+0000 7feac1e4db80 1 Processor -- start 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.442+0000 7feac1e4db80 1 -- start start 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.442+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.442+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.442+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 0x560d267fe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feac1e4db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d26800960 con 0x560d267f4910 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feac1e4db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d267c3890 con 0x560d267f9e80 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feac1e4db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d2669d2f0 con 0x560d267fa7a0 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaaf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaa6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaaffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 0x560d267fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaaf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53360/0 (socket says 172.21.15.12:53360) 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaa6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267eeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:46662/0 (socket says 172.21.15.12:46662) 2024-05-09T00:01:54.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaaf7fe700 1 -- 172.21.15.12:0/3348636357 learned_addr learned my addr 172.21.15.12:0/3348636357 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:54.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaa6ffd700 1 -- 172.21.15.12:0/3348636357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 msgr2=0x560d267fe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/3348636357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 0x560d267fe450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaa6ffd700 1 -- 172.21.15.12:0/3348636357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 msgr2=0x560d267f9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/3348636357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaa6ffd700 1 -- 172.21.15.12:0/3348636357 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d26571be0 con 0x560d267f9e80 2024-05-09T00:01:54.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/3348636357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:54.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.443+0000 7feaaffff700 1 --2- 172.21.15.12:0/3348636357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 0x560d267fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:54.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.444+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/3348636357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267eeda0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x560d267f0c50 tx=0x7fea98014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13de49ad7df56aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.445+0000 7feaa67fc700 1 -- 172.21.15.12:0/3348636357 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98022070 con 0x560d267f9e80 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.445+0000 7feaa67fc700 1 -- 172.21.15.12:0/3348636357 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea98023040 con 0x560d267f9e80 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.445+0000 7feaa67fc700 1 -- 172.21.15.12:0/3348636357 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea980038c0 con 0x560d267f9e80 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.446+0000 7feac1e4db80 1 -- 172.21.15.12:0/3348636357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 msgr2=0x560d267eeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.446+0000 7feac1e4db80 1 --2- 172.21.15.12:0/3348636357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267eeda0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x560d267f0c50 tx=0x7fea98014000 comp rx=0 tx=0).stop 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.446+0000 7feac1e4db80 1 -- 172.21.15.12:0/3348636357 shutdown_connections 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.446+0000 7feac1e4db80 1 --2- 172.21.15.12:0/3348636357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267fa7a0 0x560d267fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.446+0000 7feac1e4db80 1 --2- 172.21.15.12:0/3348636357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f4910 0x560d267f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.446+0000 7feac1e4db80 1 --2- 172.21.15.12:0/3348636357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267f9e80 0x560d267eeda0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.446+0000 7feac1e4db80 1 -- 172.21.15.12:0/3348636357 >> 172.21.15.12:0/3348636357 conn(0x560d26752290 msgr2=0x560d267fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.446+0000 7feac1e4db80 1 -- 172.21.15.12:0/3348636357 shutdown_connections 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.446+0000 7feac1e4db80 1 -- 172.21.15.12:0/3348636357 wait complete. 2024-05-09T00:01:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.446+0000 7feac1e4db80 1 -- start start 2024-05-09T00:01:54.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feac1e4db80 1 Processor -- start 2024-05-09T00:01:54.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f4910 0x560d26805350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f9e80 0x560d26803790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feac1e4db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267fa7a0 0x560d26803cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feac1e4db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d26804400 con 0x560d267f9e80 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feac1e4db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d26804570 con 0x560d267fa7a0 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feac1e4db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d26863fd0 con 0x560d267f4910 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feac1e4db80 10 client.0 _ll_register_callbacks cb 0x560d266e9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feaa6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f9e80 0x560d26803790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feaaf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f4910 0x560d26805350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feaa6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f9e80 0x560d26803790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53364/0 (socket says 172.21.15.12:53364) 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feaa6ffd700 1 -- 172.21.15.12:0/382620055 learned_addr learned my addr 172.21.15.12:0/382620055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.447+0000 7feaaffff700 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267fa7a0 0x560d26803cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7feaaf7fe700 1 -- 172.21.15.12:0/382620055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267fa7a0 msgr2=0x560d26803cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267fa7a0 0x560d26803cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7feaaf7fe700 1 -- 172.21.15.12:0/382620055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f9e80 msgr2=0x560d26803790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f9e80 0x560d26803790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7feaaf7fe700 1 -- 172.21.15.12:0/382620055 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab01a6530 con 0x560d267f4910 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7feaaf7fe700 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f4910 0x560d26805350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feaa000f740 tx=0x7feaa0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61df14717226f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7fea8effd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7fea8effd700 1 -- 172.21.15.12:0/382620055 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa009b500 con 0x560d267f4910 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7feac1e4db80 1 -- 172.21.15.12:0/382620055 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaa009b0a0 con 0x560d267f4910 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7fea8effd700 1 -- 172.21.15.12:0/382620055 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7feaa00b93a0 con 0x560d267f4910 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7fea8effd700 1 -- 172.21.15.12:0/382620055 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaa00b9230 con 0x560d267f4910 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7feac1e4db80 10 client.29578 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.448+0000 7feac1e4db80 1 -- 172.21.15.12:0/382620055 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7feaa8011420 con 0x560d267f4910 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7fea8effd700 1 -- 172.21.15.12:0/382620055 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa009b0a0 con 0x560d267f4910 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7fea8effd700 1 -- 172.21.15.12:0/382620055 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feaa009a590 con 0x560d267f4910 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7feac1e4db80 10 client.29578 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7feac1e4db80 10 client.29578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7feac1e4db80 20 client.29578 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7feac1e4db80 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560d268653b0 0x560d268677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7feac1e4db80 4 client.29578 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:01:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7feac1e4db80 1 -- 172.21.15.12:0/382620055 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560d2673cf40 con 0x560d268653b0 2024-05-09T00:01:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560d268653b0 0x560d268677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7fea8effd700 10 client.29578 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.449+0000 7feaa6ffd700 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560d268653b0 0x560d268677d0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d56c6a502e3c8c server_cookie=30db4804553dd437 in_seq=0 out_seq=0 2024-05-09T00:01:54.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.455+0000 7fea8effd700 1 -- 172.21.15.12:0/382620055 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560d2673cf40 con 0x560d268653b0 2024-05-09T00:01:54.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.455+0000 7fea8effd700 10 client.29578 handle_command_reply: tid=0 2024-05-09T00:01:54.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.455+0000 7feac1e4db80 1 client.29578 shutdown 2024-05-09T00:01:54.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 1 -- 172.21.15.12:0/382620055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f4910 msgr2=0x560d26805350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f4910 0x560d26805350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feaa000f740 tx=0x7feaa0009310 comp rx=0 tx=0).stop 2024-05-09T00:01:54.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 1 -- 172.21.15.12:0/382620055 shutdown_connections 2024-05-09T00:01:54.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560d268653b0 0x560d268677d0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560d267fa7a0 0x560d26803cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d267f4910 0x560d26805350 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7fea8effd700 0 client.29578 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:54.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 1 --2- 172.21.15.12:0/382620055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d267f9e80 0x560d26803790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 1 -- 172.21.15.12:0/382620055 >> 172.21.15.12:0/382620055 conn(0x560d26752290 msgr2=0x560d267f93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 1 -- 172.21.15.12:0/382620055 shutdown_connections 2024-05-09T00:01:54.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 1 -- 172.21.15.12:0/382620055 wait complete. 2024-05-09T00:01:54.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.456+0000 7feac1e4db80 20 client.29578 trim_cache size 0 max 16384 2024-05-09T00:01:54.451 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:01:54.452 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:01:54.452 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "1ac37b97-8f3c-46a1-bfa2-0bb2fb3a7f87", 2024-05-09T00:01:54.452 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:01:54.452 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:01:54.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 -- 172.21.15.12:0/3437654181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fea9c0a1280 msgr2=0x7fea9c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fea9c0a1280 0x7fea9c0a3740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feab00749d0 tx=0x7fea98009260 comp rx=0 tx=0).stop 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 -- 172.21.15.12:0/3437654181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab0101080 msgr2=0x7feab00732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab0101080 0x7feab00732a0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7feaa0005910 tx=0x7feaa0004730 comp rx=0 tx=0).stop 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 -- 172.21.15.12:0/3437654181 shutdown_connections 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fea9c0a1280 0x7fea9c0a3740 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feab0073d20 0x7feab01a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab0101080 0x7feab00732a0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 --2- 172.21.15.12:0/3437654181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab01061a0 0x7feab00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 -- 172.21.15.12:0/3437654181 >> 172.21.15.12:0/3437654181 conn(0x7feab00fc940 msgr2=0x7feab00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 -- 172.21.15.12:0/3437654181 shutdown_connections 2024-05-09T00:01:54.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.457+0000 7feab5c93700 1 -- 172.21.15.12:0/3437654181 wait complete. 2024-05-09T00:01:54.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:01:54.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.680+0000 7fd89e240700 1 -- 172.21.15.12:0/409125240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd898106280 msgr2=0x7fd89810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.680+0000 7fd89e240700 1 --2- 172.21.15.12:0/409125240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd898106280 0x7fd89810de00 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b0a0 tx=0x7fd89000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.680+0000 7fd89e240700 1 -- 172.21.15.12:0/409125240 shutdown_connections 2024-05-09T00:01:54.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.680+0000 7fd89e240700 1 --2- 172.21.15.12:0/409125240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd898106280 0x7fd89810de00 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.680+0000 7fd89e240700 1 --2- 172.21.15.12:0/409125240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd898101160 0x7fd898101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.680+0000 7fd89e240700 1 --2- 172.21.15.12:0/409125240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd898101b10 0x7fd898105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.680+0000 7fd89e240700 1 -- 172.21.15.12:0/409125240 >> 172.21.15.12:0/409125240 conn(0x7fd8980fc9e0 msgr2=0x7fd8980fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.680+0000 7fd89e240700 1 -- 172.21.15.12:0/409125240 shutdown_connections 2024-05-09T00:01:54.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.680+0000 7fd89e240700 1 -- 172.21.15.12:0/409125240 wait complete. 2024-05-09T00:01:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.681+0000 7fd89e240700 1 Processor -- start 2024-05-09T00:01:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd89e240700 1 -- start start 2024-05-09T00:01:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd89e240700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd898101160 0x7fd898197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd89e240700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd898106280 0x7fd898197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd89e240700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd89819be30 0x7fd89819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd89e240700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89819c8b0 con 0x7fd898106280 2024-05-09T00:01:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd89e240700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89819c9f0 con 0x7fd89819be30 2024-05-09T00:01:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd89e240700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89819cb60 con 0x7fd898101160 2024-05-09T00:01:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd898101160 0x7fd898197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd898101160 0x7fd898197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43448/0 (socket says 172.21.15.12:43448) 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd8977fe700 1 -- 172.21.15.12:0/2598124049 learned_addr learned my addr 172.21.15.12:0/2598124049 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd896ffd700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd898106280 0x7fd898197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.682+0000 7fd897fff700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd89819be30 0x7fd89819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd896ffd700 1 -- 172.21.15.12:0/2598124049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd898101160 msgr2=0x7fd898197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd896ffd700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd898101160 0x7fd898197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd896ffd700 1 -- 172.21.15.12:0/2598124049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd89819be30 msgr2=0x7fd89819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd896ffd700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd89819be30 0x7fd89819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd896ffd700 1 -- 172.21.15.12:0/2598124049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd890009d00 con 0x7fd898106280 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd8977fe700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd898101160 0x7fd898197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd896ffd700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd898106280 0x7fd898197a90 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fd898067dd0 tx=0x7fd88c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd894ff9700 1 -- 172.21.15.12:0/2598124049 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c011830 con 0x7fd898106280 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd89e240700 1 -- 172.21.15.12:0/2598124049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89806a9c0 con 0x7fd898106280 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd894ff9700 1 -- 172.21.15.12:0/2598124049 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd88c011e50 con 0x7fd898106280 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd894ff9700 1 -- 172.21.15.12:0/2598124049 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c010e50 con 0x7fd898106280 2024-05-09T00:01:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.683+0000 7fd89e240700 1 -- 172.21.15.12:0/2598124049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89806aed0 con 0x7fd898106280 2024-05-09T00:01:54.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.687+0000 7fd894ff9700 1 -- 172.21.15.12:0/2598124049 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd88c0104f0 con 0x7fd898106280 2024-05-09T00:01:54.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd8aa3fab80 1 Processor -- start 2024-05-09T00:01:54.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd8aa3fab80 1 -- start start 2024-05-09T00:01:54.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c910bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902350 0x55a40c9027d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c902d10 0x55a40c9174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c859150 con 0x55a40c9107e0 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c856b10 con 0x55a40c902d10 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c852400 con 0x55a40c902350 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd894ff9700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8800a1310 0x7fd8800a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd897fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c910bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd897fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c910bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53380/0 (socket says 172.21.15.12:53380) 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd897fff700 1 -- 172.21.15.12:0/1466351198 learned_addr learned my addr 172.21.15.12:0/1466351198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd894ff9700 1 -- 172.21.15.12:0/2598124049 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd88c0cc730 con 0x7fd898106280 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.689+0000 7fd8977fe700 1 --2- 172.21.15.12:0/1466351198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902350 0x55a40c9027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd897fff700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8800a1310 0x7fd8800a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd897fff700 1 -- 172.21.15.12:0/1466351198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902350 msgr2=0x55a40c9027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd897fff700 1 --2- 172.21.15.12:0/1466351198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902350 0x55a40c9027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd897fff700 1 -- 172.21.15.12:0/1466351198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c902d10 msgr2=0x55a40c9174e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd897fff700 1 --2- 172.21.15.12:0/1466351198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c902d10 0x55a40c9174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd897fff700 1 -- 172.21.15.12:0/1466351198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a40c6d33b0 con 0x55a40c9107e0 2024-05-09T00:01:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd897fff700 1 --2- 172.21.15.12:0/1466351198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c910bc0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fd890007db0 tx=0x7fd890007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2914dc6d0cd42bf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd897fff700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8800a1310 0x7fd8800a37d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd890009cd0 tx=0x7fd890025000 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd87dffb700 1 -- 172.21.15.12:0/1466351198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89000e040 con 0x55a40c9107e0 2024-05-09T00:01:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd87dffb700 1 -- 172.21.15.12:0/1466351198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd890007ac0 con 0x55a40c9107e0 2024-05-09T00:01:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.690+0000 7fd87dffb700 1 -- 172.21.15.12:0/1466351198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890027400 con 0x55a40c9107e0 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.691+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/1466351198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 msgr2=0x55a40c910bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.691+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/1466351198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c910bc0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fd890007db0 tx=0x7fd890007de0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.691+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/1466351198 shutdown_connections 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.691+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/1466351198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c902d10 0x55a40c9174e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.691+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/1466351198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c910bc0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.691+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/1466351198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902350 0x55a40c9027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.691+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/1466351198 >> 172.21.15.12:0/1466351198 conn(0x55a40c860210 msgr2=0x55a40c8fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.691+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/1466351198 shutdown_connections 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.691+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/1466351198 wait complete. 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.692+0000 7fd8aa3fab80 1 -- start start 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.692+0000 7fd8aa3fab80 1 Processor -- start 2024-05-09T00:01:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.692+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 0x55a40c91a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.692+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c91aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.692+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c9107e0 0x55a40c9147b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.692+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c7ee560 con 0x55a40c902350 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.692+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c7ee200 con 0x55a40c9107e0 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.692+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c6d56a0 con 0x55a40c902d10 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd8aa3fab80 10 client.0 _ll_register_callbacks cb 0x55a40c7f7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 0x55a40c91a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd896ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c91aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd896ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c91aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43480/0 (socket says 172.21.15.12:43480) 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd896ffd700 1 -- 172.21.15.12:0/3517323382 learned_addr learned my addr 172.21.15.12:0/3517323382 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd896ffd700 1 -- 172.21.15.12:0/3517323382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c9107e0 msgr2=0x55a40c9147b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd896ffd700 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c9107e0 0x55a40c9147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd896ffd700 1 -- 172.21.15.12:0/3517323382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 msgr2=0x55a40c91a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd896ffd700 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 0x55a40c91a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd896ffd700 1 -- 172.21.15.12:0/3517323382 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89810e670 con 0x55a40c902d10 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd8977fe700 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 0x55a40c91a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd896ffd700 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c91aa30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0998c0 tx=0x7fd88c099470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=119ea63946eee0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.693+0000 7fd876ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd876ffd700 1 -- 172.21.15.12:0/3517323382 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c084050 con 0x55a40c902d10 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd876ffd700 1 -- 172.21.15.12:0/3517323382 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd88c099930 con 0x55a40c902d10 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3517323382 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a40c7ab270 con 0x55a40c902d10 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd876ffd700 1 -- 172.21.15.12:0/3517323382 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd88c09d760 con 0x55a40c902d10 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd8aa3fab80 10 client.29590 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3517323382 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55a40c67fb60 con 0x55a40c902d10 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd876ffd700 1 -- 172.21.15.12:0/3517323382 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a40c7ab270 con 0x55a40c902d10 2024-05-09T00:01:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd876ffd700 1 -- 172.21.15.12:0/3517323382 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd88c0a1920 con 0x55a40c902d10 2024-05-09T00:01:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd8aa3fab80 10 client.29590 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd8aa3fab80 10 client.29590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd8aa3fab80 20 client.29590 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a40c924ab0 0x55a40c926ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd8aa3fab80 4 client.29590 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:01:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.694+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3517323382 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a40c8d1810 con 0x55a40c924ab0 2024-05-09T00:01:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.695+0000 7fd8977fe700 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a40c924ab0 0x55a40c926ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.696+0000 7fd8977fe700 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a40c924ab0 0x55a40c926ed0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=885328c2f5e7e8bf server_cookie=28eb7989d3432b84 in_seq=0 out_seq=0 2024-05-09T00:01:54.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.696+0000 7fd876ffd700 10 client.29590 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:54.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd876ffd700 1 -- 172.21.15.12:0/3517323382 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a40c8d1810 con 0x55a40c924ab0 2024-05-09T00:01:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd876ffd700 10 client.29590 handle_command_reply: tid=0 2024-05-09T00:01:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 client.29590 shutdown 2024-05-09T00:01:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3517323382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 msgr2=0x55a40c91aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c91aa30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0998c0 tx=0x7fd88c099470 comp rx=0 tx=0).stop 2024-05-09T00:01:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3517323382 shutdown_connections 2024-05-09T00:01:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a40c924ab0 0x55a40c926ed0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c9107e0 0x55a40c9147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd876ffd700 0 client.29590 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 0x55a40c91a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3517323382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c91aa30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3517323382 >> 172.21.15.12:0/3517323382 conn(0x55a40c860210 msgr2=0x55a40c8fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3517323382 shutdown_connections 2024-05-09T00:01:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3517323382 wait complete. 2024-05-09T00:01:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.698+0000 7fd8aa3fab80 20 client.29590 trim_cache size 0 max 16384 2024-05-09T00:01:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.718+0000 7fd8aa3fab80 1 Processor -- start 2024-05-09T00:01:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.718+0000 7fd8aa3fab80 1 -- start start 2024-05-09T00:01:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.718+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c902350 0x55a40c910080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.718+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c8fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.718+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c91a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.718+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c912d00 con 0x55a40c9107e0 2024-05-09T00:01:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.718+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c9033b0 con 0x55a40c902350 2024-05-09T00:01:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.718+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c8f5040 con 0x55a40c902d10 2024-05-09T00:01:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd897fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c91a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd897fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c91a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53398/0 (socket says 172.21.15.12:53398) 2024-05-09T00:01:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd897fff700 1 -- 172.21.15.12:0/1689214944 learned_addr learned my addr 172.21.15.12:0/1689214944 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd896ffd700 1 --2- 172.21.15.12:0/1689214944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c8fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd8977fe700 1 --2- 172.21.15.12:0/1689214944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c902350 0x55a40c910080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd897fff700 1 -- 172.21.15.12:0/1689214944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 msgr2=0x55a40c8fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd897fff700 1 --2- 172.21.15.12:0/1689214944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c8fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd897fff700 1 -- 172.21.15.12:0/1689214944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c902350 msgr2=0x55a40c910080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd897fff700 1 --2- 172.21.15.12:0/1689214944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c902350 0x55a40c910080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd897fff700 1 -- 172.21.15.12:0/1689214944 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a40c818560 con 0x55a40c9107e0 2024-05-09T00:01:54.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd8977fe700 1 --2- 172.21.15.12:0/1689214944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c902350 0x55a40c910080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:01:54.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd897fff700 1 --2- 172.21.15.12:0/1689214944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c91a2e0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd8900060c0 tx=0x7fd89000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1794a5596194f236 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd87e7fc700 1 -- 172.21.15.12:0/1689214944 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890035070 con 0x55a40c9107e0 2024-05-09T00:01:54.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd87e7fc700 1 -- 172.21.15.12:0/1689214944 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89000f040 con 0x55a40c9107e0 2024-05-09T00:01:54.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.719+0000 7fd87e7fc700 1 -- 172.21.15.12:0/1689214944 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89000e030 con 0x55a40c9107e0 2024-05-09T00:01:54.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.720+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/1689214944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 msgr2=0x55a40c91a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.720+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/1689214944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c91a2e0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd8900060c0 tx=0x7fd89000b700 comp rx=0 tx=0).stop 2024-05-09T00:01:54.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.720+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/1689214944 shutdown_connections 2024-05-09T00:01:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.720+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/1689214944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c9107e0 0x55a40c91a2e0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.720+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/1689214944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c902350 0x55a40c910080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.720+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/1689214944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c8fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.720+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/1689214944 >> 172.21.15.12:0/1689214944 conn(0x55a40c860210 msgr2=0x55a40c90f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.721+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/1689214944 shutdown_connections 2024-05-09T00:01:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.721+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/1689214944 wait complete. 2024-05-09T00:01:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.721+0000 7fd8aa3fab80 1 -- start start 2024-05-09T00:01:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.721+0000 7fd8aa3fab80 1 Processor -- start 2024-05-09T00:01:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.721+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 0x55a40c976510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c974a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd8aa3fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c9107e0 0x55a40c974f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c976a50 con 0x55a40c902350 2024-05-09T00:01:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c976bc0 con 0x55a40c9107e0 2024-05-09T00:01:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd8aa3fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40c976d80 con 0x55a40c902d10 2024-05-09T00:01:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd8aa3fab80 10 client.0 _ll_register_callbacks cb 0x55a40c7f7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:01:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd896ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c974a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd896ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c974a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43496/0 (socket says 172.21.15.12:43496) 2024-05-09T00:01:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd896ffd700 1 -- 172.21.15.12:0/3828017944 learned_addr learned my addr 172.21.15.12:0/3828017944 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:01:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd8977fe700 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 0x55a40c976510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd896ffd700 1 -- 172.21.15.12:0/3828017944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c9107e0 msgr2=0x55a40c974f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd896ffd700 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c9107e0 0x55a40c974f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd896ffd700 1 -- 172.21.15.12:0/3828017944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 msgr2=0x55a40c976510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:01:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd896ffd700 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 0x55a40c976510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd896ffd700 1 -- 172.21.15.12:0/3828017944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a40c6d33b0 con 0x55a40c902d10 2024-05-09T00:01:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd896ffd700 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c974a50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55a40c915cf0 tx=0x7fd88c00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217c35413d284d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd876ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:01:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.722+0000 7fd876ffd700 1 -- 172.21.15.12:0/3828017944 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c084040 con 0x55a40c902d10 2024-05-09T00:01:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd876ffd700 1 -- 172.21.15.12:0/3828017944 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd88c099930 con 0x55a40c902d10 2024-05-09T00:01:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3828017944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd890007ac0 con 0x55a40c902d10 2024-05-09T00:01:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd876ffd700 1 -- 172.21.15.12:0/3828017944 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd88c09d760 con 0x55a40c902d10 2024-05-09T00:01:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd8aa3fab80 10 client.29596 fetch_fsmap learned FSMap version 63 2024-05-09T00:01:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3828017944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55a40c6d7cd0 con 0x55a40c902d10 2024-05-09T00:01:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd876ffd700 1 -- 172.21.15.12:0/3828017944 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd88c07c020 con 0x55a40c902d10 2024-05-09T00:01:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd876ffd700 1 -- 172.21.15.12:0/3828017944 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd88c0b90b0 con 0x55a40c902d10 2024-05-09T00:01:54.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd8aa3fab80 10 client.29596 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:01:54.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd8aa3fab80 10 client.29596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:01:54.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd8aa3fab80 20 client.29596 populate_metadata read hostname 'smithi012' 2024-05-09T00:01:54.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a40c908460 0x55a40c90a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:54.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd8aa3fab80 4 client.29596 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:01:54.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3828017944 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a40c84aec0 con 0x55a40c908460 2024-05-09T00:01:54.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.723+0000 7fd8977fe700 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a40c908460 0x55a40c90a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:54.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.724+0000 7fd876ffd700 10 client.29596 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:54.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.724+0000 7fd8977fe700 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a40c908460 0x55a40c90a880 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40dbe3f14560117 server_cookie=962fe7736d4b677c in_seq=0 out_seq=0 2024-05-09T00:01:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.728+0000 7fd876ffd700 1 -- 172.21.15.12:0/3828017944 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55a40c84aec0 con 0x55a40c908460 2024-05-09T00:01:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.728+0000 7fd876ffd700 10 client.29596 handle_command_reply: tid=0 2024-05-09T00:01:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.728+0000 7fd8aa3fab80 1 client.29596 shutdown 2024-05-09T00:01:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3828017944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 msgr2=0x55a40c974a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c974a50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55a40c915cf0 tx=0x7fd88c00bf90 comp rx=0 tx=0).stop 2024-05-09T00:01:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3828017944 shutdown_connections 2024-05-09T00:01:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a40c908460 0x55a40c90a880 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a40c9107e0 0x55a40c974f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a40c902350 0x55a40c976510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd876ffd700 0 client.29596 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:01:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 1 --2- 172.21.15.12:0/3828017944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a40c902d10 0x55a40c974a50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3828017944 >> 172.21.15.12:0/3828017944 conn(0x55a40c860210 msgr2=0x55a40c91a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3828017944 shutdown_connections 2024-05-09T00:01:54.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 1 -- 172.21.15.12:0/3828017944 wait complete. 2024-05-09T00:01:54.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd8aa3fab80 20 client.29596 trim_cache size 0 max 16384 2024-05-09T00:01:54.732 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:01:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (200 inodes in the stack)", 2024-05-09T00:01:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:01:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "1ac37b97-8f3c-46a1-bfa2-0bb2fb3a7f87": { 2024-05-09T00:01:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:01:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "1ac37b97-8f3c-46a1-bfa2-0bb2fb3a7f87", 2024-05-09T00:01:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:01:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:01:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:01:54.733 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:01:54.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd89e240700 1 -- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8800a1310 msgr2=0x7fd8800a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd89e240700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8800a1310 0x7fd8800a37d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd890009cd0 tx=0x7fd890025000 comp rx=0 tx=0).stop 2024-05-09T00:01:54.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd89e240700 1 -- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd898106280 msgr2=0x7fd898197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:54.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd89e240700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd898106280 0x7fd898197a90 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fd898067dd0 tx=0x7fd88c00b7e0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd89e240700 1 -- 172.21.15.12:0/2598124049 shutdown_connections 2024-05-09T00:01:54.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd89e240700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd8800a1310 0x7fd8800a37d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd89e240700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd89819be30 0x7fd89819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd89e240700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd898101160 0x7fd898197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd89e240700 1 --2- 172.21.15.12:0/2598124049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd898106280 0x7fd898197a90 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:54.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.729+0000 7fd89e240700 1 -- 172.21.15.12:0/2598124049 >> 172.21.15.12:0/2598124049 conn(0x7fd8980fc9e0 msgr2=0x7fd8980fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:54.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.730+0000 7fd89e240700 1 -- 172.21.15.12:0/2598124049 shutdown_connections 2024-05-09T00:01:54.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:01:54.730+0000 7fd89e240700 1 -- 172.21.15.12:0/2598124049 wait complete. 2024-05-09T00:01:54.739 INFO:tasks.cephfs.filesystem:scrub status for tag:1ac37b97-8f3c-46a1-bfa2-0bb2fb3a7f87 - {'path': '/', 'tag': '1ac37b97-8f3c-46a1-bfa2-0bb2fb3a7f87', 'options': 'recursive,force'} 2024-05-09T00:01:54.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:54 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:01:54.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:54.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:01:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:54 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:01:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:01:55.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:54 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:01:55.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:55.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:01:55.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 81563 41.58 MB/sec warmup 110 sec latency 85.954 ms 2024-05-09T00:01:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:55 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:01:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:55 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:01:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:55 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:55 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:55 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:55 smithi012 ceph-mon[42503]: pgmap v497: 97 pgs: 97 active+clean; 770 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-09T00:01:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:55 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:01:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:55 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:01:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:55 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:55 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:55 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:55 smithi204 ceph-mon[47373]: pgmap v497: 97 pgs: 97 active+clean; 770 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-09T00:01:56.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:55 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:01:56.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:55 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:01:56.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:55 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:56.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:55 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:56.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:55 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:01:56.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:55 smithi160 ceph-mon[46152]: pgmap v497: 97 pgs: 97 active+clean; 770 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 733 op/s 2024-05-09T00:01:56.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 82345 41.56 MB/sec warmup 111 sec latency 75.793 ms 2024-05-09T00:01:57.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 83091 41.53 MB/sec warmup 112 sec latency 66.762 ms 2024-05-09T00:01:57.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:57 smithi012 ceph-mon[42503]: pgmap v498: 97 pgs: 97 active+clean; 797 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 733 op/s 2024-05-09T00:01:58.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:57 smithi204 ceph-mon[47373]: pgmap v498: 97 pgs: 97 active+clean; 797 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 733 op/s 2024-05-09T00:01:58.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:57 smithi160 ceph-mon[46152]: pgmap v498: 97 pgs: 97 active+clean; 797 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 733 op/s 2024-05-09T00:01:58.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 84029 41.57 MB/sec warmup 113 sec latency 50.424 ms 2024-05-09T00:01:59.224 INFO:tasks.workunit.client.0.smithi012.stdout: 10 84975 41.75 MB/sec warmup 114 sec latency 51.141 ms 2024-05-09T00:01:59.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:01:59 smithi012 ceph-mon[42503]: pgmap v499: 97 pgs: 97 active+clean; 797 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.6 MiB/s wr, 643 op/s 2024-05-09T00:02:00.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:01:59 smithi204 ceph-mon[47373]: pgmap v499: 97 pgs: 97 active+clean; 797 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.6 MiB/s wr, 643 op/s 2024-05-09T00:02:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:01:59 smithi160 ceph-mon[46152]: pgmap v499: 97 pgs: 97 active+clean; 797 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.6 MiB/s wr, 643 op/s 2024-05-09T00:02:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.037+0000 7f80a9d08700 1 -- 172.21.15.12:0/2623383452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a40ffda0 msgr2=0x7f80a4100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.037+0000 7f80a9d08700 1 --2- 172.21.15.12:0/2623383452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a40ffda0 0x7f80a4100220 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f809800faf0 tx=0x7f809800fe00 comp rx=0 tx=0).stop 2024-05-09T00:02:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.037+0000 7f80a9d08700 1 -- 172.21.15.12:0/2623383452 shutdown_connections 2024-05-09T00:02:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.037+0000 7f80a9d08700 1 --2- 172.21.15.12:0/2623383452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80a4100760 0x7f80a410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.037+0000 7f80a9d08700 1 --2- 172.21.15.12:0/2623383452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80a4105e60 0x7f80a4106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.037+0000 7f80a9d08700 1 --2- 172.21.15.12:0/2623383452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a40ffda0 0x7f80a4100220 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.037+0000 7f80a9d08700 1 -- 172.21.15.12:0/2623383452 >> 172.21.15.12:0/2623383452 conn(0x7f80a4076f10 msgr2=0x7f80a4077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.037+0000 7f80a9d08700 1 -- 172.21.15.12:0/2623383452 shutdown_connections 2024-05-09T00:02:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.037+0000 7f80a9d08700 1 -- 172.21.15.12:0/2623383452 wait complete. 2024-05-09T00:02:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.038+0000 7f80a9d08700 1 Processor -- start 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a9d08700 1 -- start start 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a9d08700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80a40ffda0 0x7f80a4195120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a9d08700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a4100760 0x7f80a4195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a9d08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80a4105e60 0x7f80a4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a9d08700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a419a040 con 0x7f80a4105e60 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a9d08700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a419a1b0 con 0x7f80a40ffda0 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a9d08700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a419a370 con 0x7f80a4100760 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a4100760 0x7f80a4195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a4100760 0x7f80a4195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51940/0 (socket says 172.21.15.12:51940) 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a2ffd700 1 -- 172.21.15.12:0/3295099827 learned_addr learned my addr 172.21.15.12:0/3295099827 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a37fe700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80a40ffda0 0x7f80a4195120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a3fff700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80a4105e60 0x7f80a4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a2ffd700 1 -- 172.21.15.12:0/3295099827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80a40ffda0 msgr2=0x7f80a4195120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a2ffd700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80a40ffda0 0x7f80a4195120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a2ffd700 1 -- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80a4105e60 msgr2=0x7f80a4199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.039+0000 7f80a2ffd700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80a4105e60 0x7f80a4199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.040+0000 7f80a2ffd700 1 -- 172.21.15.12:0/3295099827 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809800f750 con 0x7f80a4100760 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.040+0000 7f80a3fff700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80a4105e60 0x7f80a4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.040+0000 7f80a2ffd700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a4100760 0x7f80a4195660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8098009fd0 tx=0x7f8098005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.040+0000 7f80a0ff9700 1 -- 172.21.15.12:0/3295099827 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098003680 con 0x7f80a4100760 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.040+0000 7f80a0ff9700 1 -- 172.21.15.12:0/3295099827 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8098003ca0 con 0x7f80a4100760 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.040+0000 7f80a9d08700 1 -- 172.21.15.12:0/3295099827 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a419a5f0 con 0x7f80a4100760 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.040+0000 7f80a0ff9700 1 -- 172.21.15.12:0/3295099827 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809800ed00 con 0x7f80a4100760 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.040+0000 7f80a9d08700 1 -- 172.21.15.12:0/3295099827 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a41a1e90 con 0x7f80a4100760 2024-05-09T00:02:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.043+0000 7f80a0ff9700 1 -- 172.21.15.12:0/3295099827 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f80980037e0 con 0x7f80a4100760 2024-05-09T00:02:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.043+0000 7f80a0ff9700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f808c0a1310 0x7f808c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.044+0000 7f80a0ff9700 1 -- 172.21.15.12:0/3295099827 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f80980d1fc0 con 0x7f80a4100760 2024-05-09T00:02:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.044+0000 7f80a37fe700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f808c0a1310 0x7f808c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.044+0000 7f80a37fe700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f808c0a1310 0x7f808c0a37d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8094009a10 tx=0x7f8094009190 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80b5ec2b80 1 Processor -- start 2024-05-09T00:02:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80b5ec2b80 1 -- start start 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c2568d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2562f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c25638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c24ba0f0 con 0x55b8c2569270 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c24b7ab0 con 0x55b8c2563480 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c24b33a0 con 0x55b8c2568950 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2562f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2562f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51450/0 (socket says 172.21.15.12:51450) 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.045+0000 7f80a2ffd700 1 -- 172.21.15.12:0/873385410 learned_addr learned my addr 172.21.15.12:0/873385410 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f80a37fe700 1 --2- 172.21.15.12:0/873385410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c2568d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f80a3fff700 1 --2- 172.21.15.12:0/873385410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c25638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f80a37fe700 1 -- 172.21.15.12:0/873385410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 msgr2=0x55b8c25638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f80a37fe700 1 --2- 172.21.15.12:0/873385410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c25638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f80a37fe700 1 -- 172.21.15.12:0/873385410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 msgr2=0x55b8c2562f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f80a37fe700 1 --2- 172.21.15.12:0/873385410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2562f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f80a37fe700 1 -- 172.21.15.12:0/873385410 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8c2334350 con 0x55b8c2568950 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f80a37fe700 1 --2- 172.21.15.12:0/873385410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c2568d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f809400e9e0 tx=0x7f8094011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5f40612b7a8e672 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f8089ffb700 1 -- 172.21.15.12:0/873385410 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400ee20 con 0x55b8c2568950 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f8089ffb700 1 -- 172.21.15.12:0/873385410 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8094015940 con 0x55b8c2568950 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.046+0000 7f8089ffb700 1 -- 172.21.15.12:0/873385410 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094013950 con 0x55b8c2568950 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.047+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/873385410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 msgr2=0x55b8c2568d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.047+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/873385410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c2568d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f809400e9e0 tx=0x7f8094011000 comp rx=0 tx=0).stop 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.047+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/873385410 shutdown_connections 2024-05-09T00:02:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.047+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/873385410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c25638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.047+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/873385410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c2568d30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.047+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/873385410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2562f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.047+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/873385410 >> 172.21.15.12:0/873385410 conn(0x55b8c24c11b0 msgr2=0x55b8c255d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.047+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/873385410 shutdown_connections 2024-05-09T00:02:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.047+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/873385410 wait complete. 2024-05-09T00:02:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.048+0000 7f80b5ec2b80 1 -- start start 2024-05-09T00:02:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.048+0000 7f80b5ec2b80 1 Processor -- start 2024-05-09T00:02:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c256f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2568950 0x55b8c256c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2569270 0x55b8c256cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c244f500 con 0x55b8c2568950 2024-05-09T00:02:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c244f1a0 con 0x55b8c2563480 2024-05-09T00:02:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c2336640 con 0x55b8c2569270 2024-05-09T00:02:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80b5ec2b80 10 client.0 _ll_register_callbacks cb 0x55b8c2458ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2568950 0x55b8c256c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2568950 0x55b8c256c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51462/0 (socket says 172.21.15.12:51462) 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a2ffd700 1 -- 172.21.15.12:0/2889363202 learned_addr learned my addr 172.21.15.12:0/2889363202 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a3fff700 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2569270 0x55b8c256cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a37fe700 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c256f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a2ffd700 1 -- 172.21.15.12:0/2889363202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2569270 msgr2=0x55b8c256cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a2ffd700 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2569270 0x55b8c256cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a2ffd700 1 -- 172.21.15.12:0/2889363202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 msgr2=0x55b8c256f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a2ffd700 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c256f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a2ffd700 1 -- 172.21.15.12:0/2889363202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809400e6c0 con 0x55b8c2568950 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.049+0000 7f80a37fe700 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c256f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.050+0000 7f80a2ffd700 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2568950 0x55b8c256c550 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f80980cf800 tx=0x7f809809ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad3155b64f3b4c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.050+0000 7f8082ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.050+0000 7f8082ffd700 1 -- 172.21.15.12:0/2889363202 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980cfea0 con 0x55b8c2568950 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.050+0000 7f8082ffd700 1 -- 172.21.15.12:0/2889363202 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f809809f1a0 con 0x55b8c2568950 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.050+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/2889363202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8c240c210 con 0x55b8c2568950 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.050+0000 7f8082ffd700 1 -- 172.21.15.12:0/2889363202 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80980a52c0 con 0x55b8c2568950 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.050+0000 7f80b5ec2b80 10 client.19872 fetch_fsmap learned FSMap version 63 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.050+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/2889363202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55b8c22e0b00 con 0x55b8c2568950 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.050+0000 7f8082ffd700 1 -- 172.21.15.12:0/2889363202 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8c240c210 con 0x55b8c2568950 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.051+0000 7f8082ffd700 1 -- 172.21.15.12:0/2889363202 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f809800c070 con 0x55b8c2568950 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.051+0000 7f80b5ec2b80 10 client.19872 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.051+0000 7f80b5ec2b80 10 client.19872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.051+0000 7f80b5ec2b80 20 client.19872 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.051+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8c25852e0 0x55b8c2587700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.051+0000 7f80b5ec2b80 4 client.19872 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.051+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/2889363202 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8c25327b0 con 0x55b8c25852e0 2024-05-09T00:02:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.051+0000 7f80a37fe700 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8c25852e0 0x55b8c2587700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.051+0000 7f8082ffd700 10 client.19872 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.051+0000 7f80a37fe700 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8c25852e0 0x55b8c2587700 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673e093496a638d2 server_cookie=3c94640bf217d0e8 in_seq=0 out_seq=0 2024-05-09T00:02:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f8082ffd700 1 -- 172.21.15.12:0/2889363202 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8c25327b0 con 0x55b8c25852e0 2024-05-09T00:02:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f8082ffd700 10 client.19872 handle_command_reply: tid=0 2024-05-09T00:02:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 client.19872 shutdown 2024-05-09T00:02:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/2889363202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2568950 msgr2=0x55b8c256c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2568950 0x55b8c256c550 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f80980cf800 tx=0x7f809809ebf0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/2889363202 shutdown_connections 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2569270 0x55b8c256cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8c25852e0 0x55b8c2587700 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c256f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/2889363202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2568950 0x55b8c256c550 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f8082ffd700 0 client.19872 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/2889363202 >> 172.21.15.12:0/2889363202 conn(0x55b8c24c11b0 msgr2=0x55b8c2561f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/2889363202 shutdown_connections 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/2889363202 wait complete. 2024-05-09T00:02:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.055+0000 7f80b5ec2b80 20 client.19872 trim_cache size 0 max 16384 2024-05-09T00:02:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.075+0000 7f80b5ec2b80 1 Processor -- start 2024-05-09T00:02:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.076+0000 7f80b5ec2b80 1 -- start start 2024-05-09T00:02:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.076+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c2580ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.076+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c255d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.076+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2561f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.076+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c258c8f0 con 0x55b8c2569270 2024-05-09T00:02:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.076+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c2567190 con 0x55b8c2563480 2024-05-09T00:02:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.076+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c2568d30 con 0x55b8c2568950 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.076+0000 7f80a3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2561f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.076+0000 7f80a3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2561f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51470/0 (socket says 172.21.15.12:51470) 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.076+0000 7f80a3fff700 1 -- 172.21.15.12:0/3499603044 learned_addr learned my addr 172.21.15.12:0/3499603044 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.077+0000 7f80a3fff700 1 -- 172.21.15.12:0/3499603044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 msgr2=0x55b8c255d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.077+0000 7f80a3fff700 1 --2- 172.21.15.12:0/3499603044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c255d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.077+0000 7f80a3fff700 1 -- 172.21.15.12:0/3499603044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 msgr2=0x55b8c2580ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.077+0000 7f80a3fff700 1 --2- 172.21.15.12:0/3499603044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c2580ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.077+0000 7f80a3fff700 1 -- 172.21.15.12:0/3499603044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8c2479500 con 0x55b8c2569270 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.077+0000 7f80a3fff700 1 --2- 172.21.15.12:0/3499603044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2561f30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f809c00b610 tx=0x7f809c00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ca3d411fcfa76b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.077+0000 7f808a7fc700 1 -- 172.21.15.12:0/3499603044 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c010040 con 0x55b8c2569270 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.077+0000 7f808a7fc700 1 -- 172.21.15.12:0/3499603044 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f809c018cd0 con 0x55b8c2569270 2024-05-09T00:02:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.077+0000 7f808a7fc700 1 -- 172.21.15.12:0/3499603044 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c020ce0 con 0x55b8c2569270 2024-05-09T00:02:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.078+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3499603044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 msgr2=0x55b8c2561f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.078+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/3499603044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2561f30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f809c00b610 tx=0x7f809c00d040 comp rx=0 tx=0).stop 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.078+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3499603044 shutdown_connections 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.078+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/3499603044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2569270 0x55b8c2561f30 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.078+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/3499603044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2563480 0x55b8c2580ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.078+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/3499603044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c255d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.078+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3499603044 >> 172.21.15.12:0/3499603044 conn(0x55b8c24c11b0 msgr2=0x55b8c2559e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.079+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3499603044 shutdown_connections 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.079+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3499603044 wait complete. 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.079+0000 7f80b5ec2b80 1 -- start start 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.079+0000 7f80b5ec2b80 1 Processor -- start 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2563480 0x55b8c256ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c256e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80b5ec2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2569270 0x55b8c256e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c25da9b0 con 0x55b8c2563480 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c25dab20 con 0x55b8c2569270 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80b5ec2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c25dace0 con 0x55b8c2568950 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80b5ec2b80 10 client.0 _ll_register_callbacks cb 0x55b8c2458ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c256e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c256e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:51976/0 (socket says 172.21.15.12:51976) 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a2ffd700 1 -- 172.21.15.12:0/3278464275 learned_addr learned my addr 172.21.15.12:0/3278464275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a3fff700 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2569270 0x55b8c256e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a37fe700 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2563480 0x55b8c256ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a2ffd700 1 -- 172.21.15.12:0/3278464275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2569270 msgr2=0x55b8c256e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a2ffd700 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2569270 0x55b8c256e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a2ffd700 1 -- 172.21.15.12:0/3278464275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2563480 msgr2=0x55b8c256ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a2ffd700 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2563480 0x55b8c256ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a2ffd700 1 -- 172.21.15.12:0/3278464275 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c00b2c0 con 0x55b8c2568950 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f80a2ffd700 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c256e330 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f809809eea0 tx=0x7f809800fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4b2a29164d6dd39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.080+0000 7f8082ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.081+0000 7f8082ffd700 1 -- 172.21.15.12:0/3278464275 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980cfea0 con 0x55b8c2568950 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.081+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3278464275 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8094015940 con 0x55b8c2568950 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.081+0000 7f8082ffd700 1 -- 172.21.15.12:0/3278464275 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f809809f1a0 con 0x55b8c2568950 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.081+0000 7f8082ffd700 1 -- 172.21.15.12:0/3278464275 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f809809eb00 con 0x55b8c2568950 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.081+0000 7f80b5ec2b80 10 client.29611 fetch_fsmap learned FSMap version 63 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.081+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3278464275 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55b8c2338c70 con 0x55b8c2568950 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.081+0000 7f8082ffd700 1 -- 172.21.15.12:0/3278464275 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8094015940 con 0x55b8c2568950 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.081+0000 7f8082ffd700 1 -- 172.21.15.12:0/3278464275 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f809800c070 con 0x55b8c2568950 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.082+0000 7f80b5ec2b80 10 client.29611 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:02:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.082+0000 7f80b5ec2b80 10 client.29611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.082+0000 7f80b5ec2b80 20 client.29611 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.082+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8c25dbfe0 0x55b8c25de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.082+0000 7f80b5ec2b80 4 client.29611 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:02:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.082+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3278464275 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b8c24abe60 con 0x55b8c25dbfe0 2024-05-09T00:02:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.082+0000 7f80a37fe700 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8c25dbfe0 0x55b8c25de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.086+0000 7f80a37fe700 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8c25dbfe0 0x55b8c25de3d0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f61a709cab48b2f6 server_cookie=a012c3ff5fbad618 in_seq=0 out_seq=0 2024-05-09T00:02:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.086+0000 7f8082ffd700 10 client.29611 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:00.229 INFO:tasks.workunit.client.0.smithi012.stdout: 10 85616 41.66 MB/sec warmup 115 sec latency 152.324 ms 2024-05-09T00:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f8082ffd700 1 -- 172.21.15.12:0/3278464275 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b8c24abe60 con 0x55b8c25dbfe0 2024-05-09T00:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f8082ffd700 10 client.29611 handle_command_reply: tid=0 2024-05-09T00:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 client.29611 shutdown 2024-05-09T00:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3278464275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 msgr2=0x55b8c256e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c256e330 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f809809eea0 tx=0x7f809800fe60 comp rx=0 tx=0).stop 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3278464275 shutdown_connections 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8c25dbfe0 0x55b8c25de3d0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8c2569270 0x55b8c256e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8c2563480 0x55b8c256ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 --2- 172.21.15.12:0/3278464275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8c2568950 0x55b8c256e330 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f8082ffd700 0 client.29611 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3278464275 >> 172.21.15.12:0/3278464275 conn(0x55b8c24c11b0 msgr2=0x55b8c2562640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3278464275 shutdown_connections 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 1 -- 172.21.15.12:0/3278464275 wait complete. 2024-05-09T00:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.396+0000 7f80b5ec2b80 20 client.29611 trim_cache size 0 max 16384 2024-05-09T00:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:02:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 -- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f808c0a1310 msgr2=0x7f808c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f808c0a1310 0x7f808c0a37d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8094009a10 tx=0x7f8094009190 comp rx=0 tx=0).stop 2024-05-09T00:02:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 -- 172.21.15.12:0/3295099827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a4100760 msgr2=0x7f80a4195660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a4100760 0x7f80a4195660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8098009fd0 tx=0x7f8098005f10 comp rx=0 tx=0).stop 2024-05-09T00:02:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 -- 172.21.15.12:0/3295099827 shutdown_connections 2024-05-09T00:02:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f808c0a1310 0x7f808c0a37d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80a4105e60 0x7f80a4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80a40ffda0 0x7f80a4195120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 --2- 172.21.15.12:0/3295099827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a4100760 0x7f80a4195660 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 -- 172.21.15.12:0/3295099827 >> 172.21.15.12:0/3295099827 conn(0x7f80a4076f10 msgr2=0x7f80a40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 -- 172.21.15.12:0/3295099827 shutdown_connections 2024-05-09T00:02:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:00.397+0000 7f80a9d08700 1 -- 172.21.15.12:0/3295099827 wait complete. 2024-05-09T00:02:01.229 INFO:tasks.workunit.client.0.smithi012.stdout: 10 85867 41.38 MB/sec warmup 116 sec latency 542.758 ms 2024-05-09T00:02:01.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:01 smithi012 ceph-mon[42503]: pgmap v500: 97 pgs: 97 active+clean; 886 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 960 op/s 2024-05-09T00:02:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:01 smithi204 ceph-mon[47373]: pgmap v500: 97 pgs: 97 active+clean; 886 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 960 op/s 2024-05-09T00:02:02.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:01 smithi160 ceph-mon[46152]: pgmap v500: 97 pgs: 97 active+clean; 886 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 960 op/s 2024-05-09T00:02:02.229 INFO:tasks.workunit.client.0.smithi012.stdout: 10 85874 41.03 MB/sec warmup 117 sec latency 1519.264 ms 2024-05-09T00:02:03.229 INFO:tasks.workunit.client.0.smithi012.stdout: 10 86216 40.81 MB/sec warmup 118 sec latency 1830.737 ms 2024-05-09T00:02:04.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 86533 40.58 MB/sec warmup 119 sec latency 378.510 ms 2024-05-09T00:02:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:02:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:02:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:02:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:04 smithi204 ceph-mon[47373]: osdmap e114: 12 total, 12 up, 12 in 2024-05-09T00:02:04.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:04 smithi204 ceph-mon[47373]: pgmap v502: 97 pgs: 97 active+clean; 960 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-05-09T00:02:04.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:02:04.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:02:04.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:02:04.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:04 smithi160 ceph-mon[46152]: osdmap e114: 12 total, 12 up, 12 in 2024-05-09T00:02:04.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:04 smithi160 ceph-mon[46152]: pgmap v502: 97 pgs: 97 active+clean; 960 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-05-09T00:02:04.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:02:04.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:02:04.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:02:04.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:04 smithi012 ceph-mon[42503]: osdmap e114: 12 total, 12 up, 12 in 2024-05-09T00:02:04.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:04 smithi012 ceph-mon[42503]: pgmap v502: 97 pgs: 97 active+clean; 960 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-05-09T00:02:05.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:05 smithi012 ceph-mon[42503]: pgmap v503: 97 pgs: 97 active+clean; 960 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-05-09T00:02:06.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:05 smithi204 ceph-mon[47373]: pgmap v503: 97 pgs: 97 active+clean; 960 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-05-09T00:02:06.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:05 smithi160 ceph-mon[46152]: pgmap v503: 97 pgs: 97 active+clean; 960 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-05-09T00:02:06.231 INFO:tasks.workunit.client.0.smithi012.stdout: 10 87528 37.38 MB/sec execute 1 sec latency 298.767 ms 2024-05-09T00:02:07.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 88252 37.81 MB/sec execute 2 sec latency 98.150 ms 2024-05-09T00:02:07.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:07 smithi012 ceph-mon[42503]: pgmap v504: 97 pgs: 97 active+clean; 820 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.28k op/s 2024-05-09T00:02:08.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:07 smithi204 ceph-mon[47373]: pgmap v504: 97 pgs: 97 active+clean; 820 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.28k op/s 2024-05-09T00:02:08.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:07 smithi160 ceph-mon[46152]: pgmap v504: 97 pgs: 97 active+clean; 820 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.28k op/s 2024-05-09T00:02:08.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 89003 36.87 MB/sec execute 3 sec latency 84.599 ms 2024-05-09T00:02:09.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 89943 37.94 MB/sec execute 4 sec latency 41.074 ms 2024-05-09T00:02:09.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:09 smithi012 ceph-mon[42503]: pgmap v505: 97 pgs: 97 active+clean; 820 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.28k op/s 2024-05-09T00:02:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:09 smithi204 ceph-mon[47373]: pgmap v505: 97 pgs: 97 active+clean; 820 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.28k op/s 2024-05-09T00:02:10.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:09 smithi160 ceph-mon[46152]: pgmap v505: 97 pgs: 97 active+clean; 820 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.28k op/s 2024-05-09T00:02:10.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90949 44.26 MB/sec execute 5 sec latency 70.476 ms 2024-05-09T00:02:11.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 91680 42.54 MB/sec execute 6 sec latency 58.042 ms 2024-05-09T00:02:12.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:11 smithi012 ceph-mon[42503]: osdmap e115: 12 total, 12 up, 12 in 2024-05-09T00:02:12.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:11 smithi012 ceph-mon[42503]: pgmap v507: 97 pgs: 97 active+clean; 842 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 951 op/s 2024-05-09T00:02:12.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 92555 42.72 MB/sec execute 7 sec latency 51.691 ms 2024-05-09T00:02:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:11 smithi204 ceph-mon[47373]: osdmap e115: 12 total, 12 up, 12 in 2024-05-09T00:02:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:11 smithi204 ceph-mon[47373]: pgmap v507: 97 pgs: 97 active+clean; 842 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 951 op/s 2024-05-09T00:02:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:11 smithi160 ceph-mon[46152]: osdmap e115: 12 total, 12 up, 12 in 2024-05-09T00:02:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:11 smithi160 ceph-mon[46152]: pgmap v507: 97 pgs: 97 active+clean; 842 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 951 op/s 2024-05-09T00:02:13.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 93527 44.51 MB/sec execute 8 sec latency 41.129 ms 2024-05-09T00:02:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.536+0000 7f8f89e87700 1 -- 172.21.15.12:0/772200821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f84105e60 msgr2=0x7f8f84106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.536+0000 7f8f89e87700 1 --2- 172.21.15.12:0/772200821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f84105e60 0x7f8f84106240 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8f740098f0 tx=0x7f8f74009c00 comp rx=0 tx=0).stop 2024-05-09T00:02:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.537+0000 7f8f89e87700 1 -- 172.21.15.12:0/772200821 shutdown_connections 2024-05-09T00:02:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.537+0000 7f8f89e87700 1 --2- 172.21.15.12:0/772200821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f84100760 0x7f8f8410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.537+0000 7f8f89e87700 1 --2- 172.21.15.12:0/772200821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f84105e60 0x7f8f84106240 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.537+0000 7f8f89e87700 1 --2- 172.21.15.12:0/772200821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f840ffda0 0x7f8f84100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.537+0000 7f8f89e87700 1 -- 172.21.15.12:0/772200821 >> 172.21.15.12:0/772200821 conn(0x7f8f84076f10 msgr2=0x7f8f84077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.537+0000 7f8f89e87700 1 -- 172.21.15.12:0/772200821 shutdown_connections 2024-05-09T00:02:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.537+0000 7f8f89e87700 1 -- 172.21.15.12:0/772200821 wait complete. 2024-05-09T00:02:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.537+0000 7f8f89e87700 1 Processor -- start 2024-05-09T00:02:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f89e87700 1 -- start start 2024-05-09T00:02:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f89e87700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f840ffda0 0x7f8f84199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f89e87700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f84100760 0x7f8f84199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f89e87700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f84105e60 0x7f8f8419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f89e87700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8419e570 con 0x7f8f840ffda0 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f89e87700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8419e6e0 con 0x7f8f84105e60 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f89e87700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8419e8a0 con 0x7f8f84100760 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f840ffda0 0x7f8f84199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f84100760 0x7f8f84199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f840ffda0 0x7f8f84199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43432/0 (socket says 172.21.15.12:43432) 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f837fe700 1 -- 172.21.15.12:0/1229433688 learned_addr learned my addr 172.21.15.12:0/1229433688 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f84100760 0x7f8f84199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39524/0 (socket says 172.21.15.12:39524) 2024-05-09T00:02:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f83fff700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f84105e60 0x7f8f8419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f82ffd700 1 -- 172.21.15.12:0/1229433688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f84105e60 msgr2=0x7f8f8419df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f84105e60 0x7f8f8419df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f82ffd700 1 -- 172.21.15.12:0/1229433688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f840ffda0 msgr2=0x7f8f84199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f840ffda0 0x7f8f84199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f82ffd700 1 -- 172.21.15.12:0/1229433688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74009520 con 0x7f8f84100760 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.538+0000 7f8f837fe700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f840ffda0 0x7f8f84199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.539+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f84100760 0x7f8f84199b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800e9c0 tx=0x7f8f7800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.539+0000 7f8f80ff9700 1 -- 172.21.15.12:0/1229433688 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800c9c0 con 0x7f8f84100760 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.539+0000 7f8f89e87700 1 -- 172.21.15.12:0/1229433688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8419eb80 con 0x7f8f84100760 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.539+0000 7f8f80ff9700 1 -- 172.21.15.12:0/1229433688 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f78010430 con 0x7f8f84100760 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.539+0000 7f8f80ff9700 1 -- 172.21.15.12:0/1229433688 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78018680 con 0x7f8f84100760 2024-05-09T00:02:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.539+0000 7f8f89e87700 1 -- 172.21.15.12:0/1229433688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f841a63c0 con 0x7f8f84100760 2024-05-09T00:02:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.541+0000 7f8f80ff9700 1 -- 172.21.15.12:0/1229433688 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8f78010860 con 0x7f8f84100760 2024-05-09T00:02:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.542+0000 7f8f80ff9700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f6c0a1310 0x7f8f6c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.542+0000 7f8f837fe700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f6c0a1310 0x7f8f6c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.542+0000 7f8f80ff9700 1 -- 172.21.15.12:0/1229433688 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8f78014070 con 0x7f8f84100760 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.543+0000 7f8f837fe700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f6c0a1310 0x7f8f6c0a37d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009fd0 tx=0x7f8f74019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.543+0000 7f8f96041b80 1 Processor -- start 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f96041b80 1 -- start start 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 0x556c92ecbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 0x556c92ecfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92ed06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f96041b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92e1a0f0 con 0x556c92ecc260 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f96041b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92e17ab0 con 0x556c92ecb940 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f96041b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92e133a0 con 0x556c92ed0240 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f837fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 0x556c92ecbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f83fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92ed06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 0x556c92ecfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f837fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 0x556c92ecbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:44634/0 (socket says 172.21.15.12:44634) 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f837fe700 1 -- 172.21.15.12:0/2106552214 learned_addr learned my addr 172.21.15.12:0/2106552214 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f83fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92ed06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39540/0 (socket says 172.21.15.12:39540) 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f82ffd700 1 -- 172.21.15.12:0/2106552214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 msgr2=0x556c92ed06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/2106552214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92ed06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f82ffd700 1 -- 172.21.15.12:0/2106552214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 msgr2=0x556c92ecbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/2106552214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 0x556c92ecbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.544+0000 7f8f82ffd700 1 -- 172.21.15.12:0/2106552214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c92c94350 con 0x556c92ecc260 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.545+0000 7f8f83fff700 1 --2- 172.21.15.12:0/2106552214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92ed06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.545+0000 7f8f837fe700 1 --2- 172.21.15.12:0/2106552214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 0x556c92ecbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.545+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/2106552214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 0x556c92ecfd00 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8f780ca3e0 tx=0x7f8f78099ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4271688d3d49aa82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.545+0000 7f8f69ffb700 1 -- 172.21.15.12:0/2106552214 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7809dbd0 con 0x556c92ecc260 2024-05-09T00:02:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.545+0000 7f8f69ffb700 1 -- 172.21.15.12:0/2106552214 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f780a2220 con 0x556c92ecc260 2024-05-09T00:02:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.545+0000 7f8f69ffb700 1 -- 172.21.15.12:0/2106552214 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f780994f0 con 0x556c92ecc260 2024-05-09T00:02:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.546+0000 7f8f96041b80 1 -- 172.21.15.12:0/2106552214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 msgr2=0x556c92ecfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.546+0000 7f8f96041b80 1 --2- 172.21.15.12:0/2106552214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 0x556c92ecfd00 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8f780ca3e0 tx=0x7f8f78099ca0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.546+0000 7f8f96041b80 1 -- 172.21.15.12:0/2106552214 shutdown_connections 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.546+0000 7f8f96041b80 1 --2- 172.21.15.12:0/2106552214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92ed06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.546+0000 7f8f96041b80 1 --2- 172.21.15.12:0/2106552214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 0x556c92ecbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.546+0000 7f8f96041b80 1 --2- 172.21.15.12:0/2106552214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 0x556c92ecfd00 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.546+0000 7f8f96041b80 1 -- 172.21.15.12:0/2106552214 >> 172.21.15.12:0/2106552214 conn(0x556c92e211b0 msgr2=0x556c92ebd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.546+0000 7f8f96041b80 1 -- 172.21.15.12:0/2106552214 shutdown_connections 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.546+0000 7f8f96041b80 1 -- 172.21.15.12:0/2106552214 wait complete. 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.546+0000 7f8f96041b80 1 -- start start 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f96041b80 1 Processor -- start 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 0x556c92ed4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecc260 0x556c92ed52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ed0240 0x556c92ed5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f96041b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92daf500 con 0x556c92ed0240 2024-05-09T00:02:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f96041b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92daf1a0 con 0x556c92ecb940 2024-05-09T00:02:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f96041b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92c96640 con 0x556c92ecc260 2024-05-09T00:02:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f96041b80 10 client.0 _ll_register_callbacks cb 0x556c92db8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f83fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ed0240 0x556c92ed5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f83fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ed0240 0x556c92ed5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43454/0 (socket says 172.21.15.12:43454) 2024-05-09T00:02:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f83fff700 1 -- 172.21.15.12:0/688623446 learned_addr learned my addr 172.21.15.12:0/688623446 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.547+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecc260 0x556c92ed52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f82ffd700 1 -- 172.21.15.12:0/688623446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 msgr2=0x556c92ed4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 0x556c92ed4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f82ffd700 1 -- 172.21.15.12:0/688623446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ed0240 msgr2=0x556c92ed5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ed0240 0x556c92ed5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f82ffd700 1 -- 172.21.15.12:0/688623446 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7c011230 con 0x556c92ecc260 2024-05-09T00:02:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f83fff700 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ed0240 0x556c92ed5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecc260 0x556c92ed52d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x556c92ecc730 tx=0x7f8f780107d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e189d372390e2e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:02:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f62ffd700 1 -- 172.21.15.12:0/688623446 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7809dbd0 con 0x556c92ecc260 2024-05-09T00:02:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f62ffd700 1 -- 172.21.15.12:0/688623446 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8f7809a210 con 0x556c92ecc260 2024-05-09T00:02:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f96041b80 1 -- 172.21.15.12:0/688623446 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c92d6c210 con 0x556c92ecc260 2024-05-09T00:02:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f62ffd700 1 -- 172.21.15.12:0/688623446 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f780a1d90 con 0x556c92ecc260 2024-05-09T00:02:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f96041b80 10 client.29623 fetch_fsmap learned FSMap version 63 2024-05-09T00:02:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.548+0000 7f8f96041b80 1 -- 172.21.15.12:0/688623446 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7f8f841a63c0 con 0x556c92ecc260 2024-05-09T00:02:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.549+0000 7f8f62ffd700 1 -- 172.21.15.12:0/688623446 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c92d6c210 con 0x556c92ecc260 2024-05-09T00:02:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.549+0000 7f8f62ffd700 1 -- 172.21.15.12:0/688623446 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8f78095b90 con 0x556c92ecc260 2024-05-09T00:02:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.549+0000 7f8f96041b80 10 client.29623 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:02:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.549+0000 7f8f96041b80 10 client.29623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.549+0000 7f8f96041b80 20 client.29623 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.549+0000 7f8f96041b80 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556c92ee5190 0x556c92ee75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.549+0000 7f8f96041b80 4 client.29623 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.549+0000 7f8f96041b80 1 -- 172.21.15.12:0/688623446 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c92e927b0 con 0x556c92ee5190 2024-05-09T00:02:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.549+0000 7f8f837fe700 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556c92ee5190 0x556c92ee75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.550+0000 7f8f62ffd700 10 client.29623 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.550+0000 7f8f837fe700 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556c92ee5190 0x556c92ee75b0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183d228aca5977b2 server_cookie=42b2f088d8d0abf8 in_seq=0 out_seq=0 2024-05-09T00:02:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f62ffd700 1 -- 172.21.15.12:0/688623446 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c92e927b0 con 0x556c92ee5190 2024-05-09T00:02:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f62ffd700 10 client.29623 handle_command_reply: tid=0 2024-05-09T00:02:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 client.29623 shutdown 2024-05-09T00:02:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 -- 172.21.15.12:0/688623446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecc260 msgr2=0x556c92ed52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecc260 0x556c92ed52d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x556c92ecc730 tx=0x7f8f780107d0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 -- 172.21.15.12:0/688623446 shutdown_connections 2024-05-09T00:02:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ed0240 0x556c92ed5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556c92ee5190 0x556c92ee75b0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecb940 0x556c92ed4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 --2- 172.21.15.12:0/688623446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecc260 0x556c92ed52d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f62ffd700 0 client.29623 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 -- 172.21.15.12:0/688623446 >> 172.21.15.12:0/688623446 conn(0x556c92e211b0 msgr2=0x556c92ec1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 -- 172.21.15.12:0/688623446 shutdown_connections 2024-05-09T00:02:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.555+0000 7f8f96041b80 1 -- 172.21.15.12:0/688623446 wait complete. 2024-05-09T00:02:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.556+0000 7f8f96041b80 20 client.29623 trim_cache size 0 max 16384 2024-05-09T00:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.573+0000 7f8f96041b80 1 Processor -- start 2024-05-09T00:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.573+0000 7f8f96041b80 1 -- start start 2024-05-09T00:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecb940 0x556c92ebd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 0x556c92ebf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ed0240 0x556c92ebf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f96041b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92ebe330 con 0x556c92ecc260 2024-05-09T00:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f96041b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92eec760 con 0x556c92ed0240 2024-05-09T00:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f96041b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92ecef20 con 0x556c92ecb940 2024-05-09T00:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 0x556c92ebf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f837fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecb940 0x556c92ebd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 0x556c92ebf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43466/0 (socket says 172.21.15.12:43466) 2024-05-09T00:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f82ffd700 1 -- 172.21.15.12:0/2689190983 learned_addr learned my addr 172.21.15.12:0/2689190983 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f83fff700 1 --2- 172.21.15.12:0/2689190983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ed0240 0x556c92ebf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f837fe700 1 -- 172.21.15.12:0/2689190983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ed0240 msgr2=0x556c92ebf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f837fe700 1 --2- 172.21.15.12:0/2689190983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ed0240 0x556c92ebf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f837fe700 1 -- 172.21.15.12:0/2689190983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 msgr2=0x556c92ebf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f837fe700 1 --2- 172.21.15.12:0/2689190983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 0x556c92ebf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f837fe700 1 -- 172.21.15.12:0/2689190983 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c92c40b00 con 0x556c92ecb940 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.574+0000 7f8f837fe700 1 --2- 172.21.15.12:0/2689190983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecb940 0x556c92ebd910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c010 tx=0x7f8f7401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471d206075fe0cf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.575+0000 7f8f6a7fc700 1 -- 172.21.15.12:0/2689190983 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400b560 con 0x556c92ecb940 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.575+0000 7f8f6a7fc700 1 -- 172.21.15.12:0/2689190983 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f7401d040 con 0x556c92ecb940 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.575+0000 7f8f6a7fc700 1 -- 172.21.15.12:0/2689190983 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74003e70 con 0x556c92ecb940 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.575+0000 7f8f96041b80 1 -- 172.21.15.12:0/2689190983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecb940 msgr2=0x556c92ebd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.575+0000 7f8f96041b80 1 --2- 172.21.15.12:0/2689190983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecb940 0x556c92ebd910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c010 tx=0x7f8f7401f000 comp rx=0 tx=0).stop 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.576+0000 7f8f96041b80 1 -- 172.21.15.12:0/2689190983 shutdown_connections 2024-05-09T00:02:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.576+0000 7f8f96041b80 1 --2- 172.21.15.12:0/2689190983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ed0240 0x556c92ebf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.576+0000 7f8f96041b80 1 --2- 172.21.15.12:0/2689190983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ecb940 0x556c92ebd910 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.576+0000 7f8f96041b80 1 --2- 172.21.15.12:0/2689190983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecc260 0x556c92ebf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.576+0000 7f8f96041b80 1 -- 172.21.15.12:0/2689190983 >> 172.21.15.12:0/2689190983 conn(0x556c92e211b0 msgr2=0x556c92ec1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.576+0000 7f8f96041b80 1 -- 172.21.15.12:0/2689190983 shutdown_connections 2024-05-09T00:02:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.576+0000 7f8f96041b80 1 -- 172.21.15.12:0/2689190983 wait complete. 2024-05-09T00:02:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.576+0000 7f8f96041b80 1 -- start start 2024-05-09T00:02:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.576+0000 7f8f96041b80 1 Processor -- start 2024-05-09T00:02:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecb940 0x556c92edc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecc260 0x556c92eda390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f96041b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92eda8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f96041b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92edaed0 con 0x556c92ecb940 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f96041b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92edb040 con 0x556c92ecc260 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f96041b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c92edb200 con 0x556c92ed0240 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecb940 0x556c92edc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecb940 0x556c92edc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43474/0 (socket says 172.21.15.12:43474) 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f837fe700 1 -- 172.21.15.12:0/233754777 learned_addr learned my addr 172.21.15.12:0/233754777 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecc260 0x556c92eda390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f96041b80 10 client.0 _ll_register_callbacks cb 0x556c92db8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f83fff700 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92eda8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.577+0000 7f8f837fe700 1 -- 172.21.15.12:0/233754777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 msgr2=0x556c92eda8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f837fe700 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92eda8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f837fe700 1 -- 172.21.15.12:0/233754777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecc260 msgr2=0x556c92eda390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f837fe700 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecc260 0x556c92eda390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f837fe700 1 -- 172.21.15.12:0/233754777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7400f5e0 con 0x556c92ecb940 2024-05-09T00:02:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecc260 0x556c92eda390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f83fff700 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92eda8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:02:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f837fe700 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecb940 0x556c92edc650 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009c40 tx=0x7f8f7400b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4365bec68c46480 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f62ffd700 1 -- 172.21.15.12:0/233754777 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74003880 con 0x556c92ecb940 2024-05-09T00:02:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f62ffd700 1 -- 172.21.15.12:0/233754777 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8f74003af0 con 0x556c92ecb940 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f96041b80 1 -- 172.21.15.12:0/233754777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f780a2220 con 0x556c92ecb940 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f62ffd700 1 -- 172.21.15.12:0/233754777 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f74003cd0 con 0x556c92ecb940 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f96041b80 10 client.19896 fetch_fsmap learned FSMap version 63 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f96041b80 1 -- 172.21.15.12:0/233754777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x556c92dd9500 con 0x556c92ecb940 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.578+0000 7f8f62ffd700 1 -- 172.21.15.12:0/233754777 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f780a2220 con 0x556c92ecb940 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.579+0000 7f8f62ffd700 1 -- 172.21.15.12:0/233754777 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 63) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8f74031280 con 0x556c92ecb940 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.579+0000 7f8f96041b80 10 client.19896 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.579+0000 7f8f96041b80 10 client.19896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.579+0000 7f8f96041b80 20 client.19896 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.579+0000 7f8f96041b80 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556c92f3b920 0x556c92f3dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.579+0000 7f8f96041b80 4 client.19896 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.579+0000 7f8f96041b80 1 -- 172.21.15.12:0/233754777 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556c92e0be60 con 0x556c92f3b920 2024-05-09T00:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.579+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556c92f3b920 0x556c92f3dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.579+0000 7f8f62ffd700 10 client.19896 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.579+0000 7f8f82ffd700 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556c92f3b920 0x556c92f3dd40 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9111ae477b4d5784 server_cookie=58b25db169824b63 in_seq=0 out_seq=0 2024-05-09T00:02:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.871+0000 7f8f62ffd700 1 -- 172.21.15.12:0/233754777 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556c92e0be60 con 0x556c92f3b920 2024-05-09T00:02:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.871+0000 7f8f62ffd700 10 client.19896 handle_command_reply: tid=0 2024-05-09T00:02:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.871+0000 7f8f96041b80 1 client.19896 shutdown 2024-05-09T00:02:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.871+0000 7f8f96041b80 1 -- 172.21.15.12:0/233754777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecb940 msgr2=0x556c92edc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.871+0000 7f8f96041b80 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecb940 0x556c92edc650 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009c40 tx=0x7f8f7400b340 comp rx=0 tx=0).stop 2024-05-09T00:02:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.872+0000 7f8f96041b80 1 -- 172.21.15.12:0/233754777 shutdown_connections 2024-05-09T00:02:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.872+0000 7f8f96041b80 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c92ed0240 0x556c92eda8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.872+0000 7f8f96041b80 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c92ecb940 0x556c92edc650 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.872+0000 7f8f96041b80 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556c92f3b920 0x556c92f3dd40 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.872+0000 7f8f96041b80 1 --2- 172.21.15.12:0/233754777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c92ecc260 0x556c92eda390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.872+0000 7f8f96041b80 1 -- 172.21.15.12:0/233754777 >> 172.21.15.12:0/233754777 conn(0x556c92e211b0 msgr2=0x556c92ebf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.872+0000 7f8f62ffd700 0 client.19896 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.872+0000 7f8f96041b80 1 -- 172.21.15.12:0/233754777 shutdown_connections 2024-05-09T00:02:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.872+0000 7f8f96041b80 1 -- 172.21.15.12:0/233754777 wait complete. 2024-05-09T00:02:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.873+0000 7f8f96041b80 20 client.19896 trim_cache size 0 max 16384 2024-05-09T00:02:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:02:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:02:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:02:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:02:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.873+0000 7f8f89e87700 1 -- 172.21.15.12:0/1229433688 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f6c0a1310 msgr2=0x7f8f6c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.873+0000 7f8f89e87700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f6c0a1310 0x7f8f6c0a37d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009fd0 tx=0x7f8f74019040 comp rx=0 tx=0).stop 2024-05-09T00:02:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.874+0000 7f8f89e87700 1 -- 172.21.15.12:0/1229433688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f84100760 msgr2=0x7f8f84199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.874+0000 7f8f89e87700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f84100760 0x7f8f84199b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800e9c0 tx=0x7f8f7800ecd0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.874+0000 7f8f89e87700 1 -- 172.21.15.12:0/1229433688 shutdown_connections 2024-05-09T00:02:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.874+0000 7f8f89e87700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8f6c0a1310 0x7f8f6c0a37d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.874+0000 7f8f89e87700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f84105e60 0x7f8f8419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.874+0000 7f8f89e87700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f840ffda0 0x7f8f84199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.874+0000 7f8f89e87700 1 --2- 172.21.15.12:0/1229433688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f84100760 0x7f8f84199b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.874+0000 7f8f89e87700 1 -- 172.21.15.12:0/1229433688 >> 172.21.15.12:0/1229433688 conn(0x7f8f84076f10 msgr2=0x7f8f84068320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.874+0000 7f8f89e87700 1 -- 172.21.15.12:0/1229433688 shutdown_connections 2024-05-09T00:02:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:13.874+0000 7f8f89e87700 1 -- 172.21.15.12:0/1229433688 wait complete. 2024-05-09T00:02:13.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:13 smithi012 ceph-mon[42503]: pgmap v508: 97 pgs: 97 active+clean; 851 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 872 op/s 2024-05-09T00:02:14.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:13 smithi204 ceph-mon[47373]: pgmap v508: 97 pgs: 97 active+clean; 851 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 872 op/s 2024-05-09T00:02:14.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:13 smithi160 ceph-mon[46152]: pgmap v508: 97 pgs: 97 active+clean; 851 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 872 op/s 2024-05-09T00:02:14.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 94145 42.83 MB/sec execute 9 sec latency 202.543 ms 2024-05-09T00:02:15.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 94988 43.15 MB/sec execute 10 sec latency 32.517 ms 2024-05-09T00:02:15.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:15 smithi012 ceph-mon[42503]: pgmap v509: 97 pgs: 97 active+clean; 851 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 872 op/s 2024-05-09T00:02:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:15 smithi204 ceph-mon[47373]: pgmap v509: 97 pgs: 97 active+clean; 851 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 872 op/s 2024-05-09T00:02:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:15 smithi160 ceph-mon[46152]: pgmap v509: 97 pgs: 97 active+clean; 851 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 872 op/s 2024-05-09T00:02:16.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 95905 43.86 MB/sec execute 11 sec latency 60.019 ms 2024-05-09T00:02:17.231 INFO:tasks.workunit.client.0.smithi012.stdout: 10 96606 43.60 MB/sec execute 12 sec latency 65.171 ms 2024-05-09T00:02:17.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:17 smithi012 ceph-mon[42503]: pgmap v510: 97 pgs: 97 active+clean; 861 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-05-09T00:02:18.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:17 smithi204 ceph-mon[47373]: pgmap v510: 97 pgs: 97 active+clean; 861 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-05-09T00:02:18.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:17 smithi160 ceph-mon[46152]: pgmap v510: 97 pgs: 97 active+clean; 861 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-05-09T00:02:18.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 97374 43.09 MB/sec execute 13 sec latency 57.304 ms 2024-05-09T00:02:19.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98279 43.18 MB/sec execute 14 sec latency 70.314 ms 2024-05-09T00:02:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:19 smithi012 ceph-mon[42503]: pgmap v511: 97 pgs: 97 active+clean; 861 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-05-09T00:02:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:19 smithi204 ceph-mon[47373]: pgmap v511: 97 pgs: 97 active+clean; 861 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-05-09T00:02:20.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:19 smithi160 ceph-mon[46152]: pgmap v511: 97 pgs: 97 active+clean; 861 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-05-09T00:02:20.231 INFO:tasks.workunit.client.0.smithi012.stdout: 10 99247 44.20 MB/sec execute 15 sec latency 30.424 ms 2024-05-09T00:02:21.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 100022 44.12 MB/sec execute 16 sec latency 36.907 ms 2024-05-09T00:02:22.230 INFO:tasks.workunit.client.0.smithi012.stdout: 10 100720 43.76 MB/sec execute 17 sec latency 144.490 ms 2024-05-09T00:02:22.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:22 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:22.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:22 smithi012 ceph-mon[42503]: pgmap v512: 97 pgs: 97 active+clean; 874 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.6 MiB/s wr, 801 op/s 2024-05-09T00:02:22.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:22 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:22.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:22 smithi204 ceph-mon[47373]: pgmap v512: 97 pgs: 97 active+clean; 874 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.6 MiB/s wr, 801 op/s 2024-05-09T00:02:22.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:22 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:22.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:22 smithi160 ceph-mon[46152]: pgmap v512: 97 pgs: 97 active+clean; 874 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.6 MiB/s wr, 801 op/s 2024-05-09T00:02:23.231 INFO:tasks.workunit.client.0.smithi012.stdout: 10 101540 43.76 MB/sec execute 18 sec latency 41.937 ms 2024-05-09T00:02:23.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:23 smithi012 ceph-mon[42503]: pgmap v513: 97 pgs: 97 active+clean; 893 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.4 MiB/s wr, 689 op/s 2024-05-09T00:02:24.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:23 smithi204 ceph-mon[47373]: pgmap v513: 97 pgs: 97 active+clean; 893 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.4 MiB/s wr, 689 op/s 2024-05-09T00:02:24.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:23 smithi160 ceph-mon[46152]: pgmap v513: 97 pgs: 97 active+clean; 893 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.4 MiB/s wr, 689 op/s 2024-05-09T00:02:24.231 INFO:tasks.workunit.client.0.smithi012.stdout: 10 102383 43.87 MB/sec execute 19 sec latency 54.633 ms 2024-05-09T00:02:24.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:02:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.930+0000 7f1f47618700 1 -- 172.21.15.12:0/396094232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40106280 msgr2=0x7f1f4010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.930+0000 7f1f47618700 1 --2- 172.21.15.12:0/396094232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40106280 0x7f1f4010de00 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800b0a0 tx=0x7f1f3800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.930+0000 7f1f47618700 1 -- 172.21.15.12:0/396094232 shutdown_connections 2024-05-09T00:02:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.930+0000 7f1f47618700 1 --2- 172.21.15.12:0/396094232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40106280 0x7f1f4010de00 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.930+0000 7f1f47618700 1 --2- 172.21.15.12:0/396094232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f40101160 0x7f1f40101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.930+0000 7f1f47618700 1 --2- 172.21.15.12:0/396094232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f40101b10 0x7f1f40105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.930+0000 7f1f47618700 1 -- 172.21.15.12:0/396094232 >> 172.21.15.12:0/396094232 conn(0x7f1f400fc9e0 msgr2=0x7f1f400fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.930+0000 7f1f47618700 1 -- 172.21.15.12:0/396094232 shutdown_connections 2024-05-09T00:02:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.930+0000 7f1f47618700 1 -- 172.21.15.12:0/396094232 wait complete. 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f47618700 1 Processor -- start 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f47618700 1 -- start start 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f47618700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101160 0x7f1f400732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f47618700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f40106280 0x7f1f400737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f47618700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f40074f60 0x7f1f40073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f47618700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f401aa200 con 0x7f1f40101160 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f47618700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f401aa370 con 0x7f1f40074f60 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f47618700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f401aa530 con 0x7f1f40106280 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f44bb3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f40106280 0x7f1f400737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f44bb3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f40106280 0x7f1f400737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40250/0 (socket says 172.21.15.12:40250) 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f45bb5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f40074f60 0x7f1f40073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.931+0000 7f1f44bb3700 1 -- 172.21.15.12:0/502298615 learned_addr learned my addr 172.21.15.12:0/502298615 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f44bb3700 1 -- 172.21.15.12:0/502298615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f40074f60 msgr2=0x7f1f40073d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f40074f60 0x7f1f40073d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f44bb3700 1 -- 172.21.15.12:0/502298615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101160 msgr2=0x7f1f400732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101160 0x7f1f400732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f44bb3700 1 -- 172.21.15.12:0/502298615 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30009520 con 0x7f1f40106280 2024-05-09T00:02:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f40106280 0x7f1f400737e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000ea30 tx=0x7f1f3000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f367fc700 1 -- 172.21.15.12:0/502298615 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30009820 con 0x7f1f40106280 2024-05-09T00:02:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f47618700 1 -- 172.21.15.12:0/502298615 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f38009d00 con 0x7f1f40106280 2024-05-09T00:02:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f367fc700 1 -- 172.21.15.12:0/502298615 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f300044e0 con 0x7f1f40106280 2024-05-09T00:02:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f367fc700 1 -- 172.21.15.12:0/502298615 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30003dc0 con 0x7f1f40106280 2024-05-09T00:02:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.932+0000 7f1f47618700 1 -- 172.21.15.12:0/502298615 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f401aabb0 con 0x7f1f40106280 2024-05-09T00:02:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.935+0000 7f1f367fc700 1 -- 172.21.15.12:0/502298615 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1f30003f80 con 0x7f1f40106280 2024-05-09T00:02:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.936+0000 7f1f367fc700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f2c0a1290 0x7f1f2c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.936+0000 7f1f453b4700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f2c0a1290 0x7f1f2c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.936+0000 7f1f367fc700 1 -- 172.21.15.12:0/502298615 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1f30014070 con 0x7f1f40106280 2024-05-09T00:02:24.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.937+0000 7f1f453b4700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f2c0a1290 0x7f1f2c0a3750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c005950 tx=0x7f1f3c0058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.937+0000 7f1f537d2b80 1 Processor -- start 2024-05-09T00:02:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.937+0000 7f1f537d2b80 1 -- start start 2024-05-09T00:02:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1acdf0 0x56156a1ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1ad710 0x56156a1a7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1a78a0 0x56156a1a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a0ed150 con 0x56156a1acdf0 2024-05-09T00:02:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a0eab10 con 0x56156a1a78a0 2024-05-09T00:02:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a0e6400 con 0x56156a1ad710 2024-05-09T00:02:24.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f453b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1acdf0 0x56156a1ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f453b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1acdf0 0x56156a1ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56454/0 (socket says 172.21.15.12:56454) 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f453b4700 1 -- 172.21.15.12:0/73666428 learned_addr learned my addr 172.21.15.12:0/73666428 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/73666428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1ad710 0x56156a1a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f45bb5700 1 --2- 172.21.15.12:0/73666428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1a78a0 0x56156a1a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f453b4700 1 -- 172.21.15.12:0/73666428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1ad710 msgr2=0x56156a1a7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f453b4700 1 --2- 172.21.15.12:0/73666428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1ad710 0x56156a1a7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f453b4700 1 -- 172.21.15.12:0/73666428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1a78a0 msgr2=0x56156a1a7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f453b4700 1 --2- 172.21.15.12:0/73666428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1a78a0 0x56156a1a7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.938+0000 7f1f453b4700 1 -- 172.21.15.12:0/73666428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561569f673b0 con 0x56156a1acdf0 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.939+0000 7f1f453b4700 1 --2- 172.21.15.12:0/73666428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1acdf0 0x56156a1ad1d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00ebc0 tx=0x7f1f3c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eace8a1d1064865f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.939+0000 7f1f2b7fe700 1 -- 172.21.15.12:0/73666428 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c010ac0 con 0x56156a1acdf0 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.939+0000 7f1f2b7fe700 1 -- 172.21.15.12:0/73666428 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f3c009820 con 0x56156a1acdf0 2024-05-09T00:02:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.939+0000 7f1f2b7fe700 1 -- 172.21.15.12:0/73666428 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c01c680 con 0x56156a1acdf0 2024-05-09T00:02:24.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.940+0000 7f1f537d2b80 1 -- 172.21.15.12:0/73666428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1acdf0 msgr2=0x56156a1ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:24.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.940+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/73666428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1acdf0 0x56156a1ad1d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00ebc0 tx=0x7f1f3c011000 comp rx=0 tx=0).stop 2024-05-09T00:02:24.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.940+0000 7f1f537d2b80 1 -- 172.21.15.12:0/73666428 shutdown_connections 2024-05-09T00:02:24.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.940+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/73666428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1a78a0 0x56156a1a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.940+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/73666428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1acdf0 0x56156a1ad1d0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.940+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/73666428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1ad710 0x56156a1a7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.940+0000 7f1f537d2b80 1 -- 172.21.15.12:0/73666428 >> 172.21.15.12:0/73666428 conn(0x56156a0f4210 msgr2=0x56156a190df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:24.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.940+0000 7f1f537d2b80 1 -- 172.21.15.12:0/73666428 shutdown_connections 2024-05-09T00:02:24.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.940+0000 7f1f537d2b80 1 -- 172.21.15.12:0/73666428 wait complete. 2024-05-09T00:02:24.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.941+0000 7f1f537d2b80 1 -- start start 2024-05-09T00:02:24.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.941+0000 7f1f537d2b80 1 Processor -- start 2024-05-09T00:02:24.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.941+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.941+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1b1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.941+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a1b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.941+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a082560 con 0x56156a1a78a0 2024-05-09T00:02:24.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.941+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a082200 con 0x56156a1ad710 2024-05-09T00:02:24.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.941+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561569f696a0 con 0x56156a1acdf0 2024-05-09T00:02:24.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.941+0000 7f1f537d2b80 10 client.0 _ll_register_callbacks cb 0x56156a08bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:24.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f453b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f44bb3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f453b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56468/0 (socket says 172.21.15.12:56468) 2024-05-09T00:02:24.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f453b4700 1 -- 172.21.15.12:0/2761386109 learned_addr learned my addr 172.21.15.12:0/2761386109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f453b4700 1 -- 172.21.15.12:0/2761386109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 msgr2=0x56156a1b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f453b4700 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f453b4700 1 -- 172.21.15.12:0/2761386109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 msgr2=0x56156a1b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f453b4700 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a1b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f453b4700 1 -- 172.21.15.12:0/2761386109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c00e870 con 0x56156a1a78a0 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f453b4700 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b0b50 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c010f30 tx=0x7f1f3c010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a0596cb81f16f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.942+0000 7f1f28ff9700 1 -- 172.21.15.12:0/2761386109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c025a30 con 0x56156a1a78a0 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2761386109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56156a165810 con 0x56156a1a78a0 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f28ff9700 1 -- 172.21.15.12:0/2761386109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1f3c01c420 con 0x56156a1a78a0 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f28ff9700 1 -- 172.21.15.12:0/2761386109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f3c01c5e0 con 0x56156a1a78a0 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f537d2b80 10 client.19905 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2761386109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x561569f13b60 con 0x56156a1a78a0 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f28ff9700 1 -- 172.21.15.12:0/2761386109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56156a165810 con 0x56156a1a78a0 2024-05-09T00:02:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f28ff9700 1 -- 172.21.15.12:0/2761386109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1f3c02d070 con 0x56156a1a78a0 2024-05-09T00:02:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f537d2b80 10 client.19905 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f537d2b80 10 client.19905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f537d2b80 20 client.19905 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156a1c10c0 0x56156a1c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f537d2b80 4 client.19905 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.943+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2761386109 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56156a0deec0 con 0x56156a1c10c0 2024-05-09T00:02:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.944+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156a1c10c0 0x56156a1c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.944+0000 7f1f28ff9700 10 client.19905 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:24.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.944+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156a1c10c0 0x56156a1c34e0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266fe1195c8786f3 server_cookie=147b94362890d045 in_seq=0 out_seq=0 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f28ff9700 1 -- 172.21.15.12:0/2761386109 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56156a0deec0 con 0x56156a1c10c0 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f28ff9700 10 client.19905 handle_command_reply: tid=0 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 client.19905 shutdown 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2761386109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 msgr2=0x56156a1b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b0b50 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c010f30 tx=0x7f1f3c010f60 comp rx=0 tx=0).stop 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2761386109 shutdown_connections 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156a1c10c0 0x56156a1c34e0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a1b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b0b50 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/2761386109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2761386109 >> 172.21.15.12:0/2761386109 conn(0x56156a0f4210 msgr2=0x56156a1905a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f28ff9700 0 client.19905 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2761386109 shutdown_connections 2024-05-09T00:02:24.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2761386109 wait complete. 2024-05-09T00:02:24.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.952+0000 7f1f537d2b80 20 client.19905 trim_cache size 0 max 16384 2024-05-09T00:02:24.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f537d2b80 1 Processor -- start 2024-05-09T00:02:24.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f537d2b80 1 -- start start 2024-05-09T00:02:24.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a190d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a1ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a1bdcd0 con 0x56156a1a78a0 2024-05-09T00:02:24.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a1b36c0 con 0x56156a1ad710 2024-05-09T00:02:24.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a1aeba0 con 0x56156a1acdf0 2024-05-09T00:02:24.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f453b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a190d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f453b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a190d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56478/0 (socket says 172.21.15.12:56478) 2024-05-09T00:02:24.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f453b4700 1 -- 172.21.15.12:0/2232912538 learned_addr learned my addr 172.21.15.12:0/2232912538 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:24.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.974+0000 7f1f45bb5700 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a1ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f45bb5700 1 -- 172.21.15.12:0/2232912538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 msgr2=0x56156a1aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f45bb5700 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f45bb5700 1 -- 172.21.15.12:0/2232912538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 msgr2=0x56156a190d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f45bb5700 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a190d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f45bb5700 1 -- 172.21.15.12:0/2232912538 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56156a0ac560 con 0x56156a1ad710 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f453b4700 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a190d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f45bb5700 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a1ab210 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1f380062a0 tx=0x7f1f38009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c24434ad0eb8542 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f2bfff700 1 -- 172.21.15.12:0/2232912538 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3800e040 con 0x56156a1ad710 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f2bfff700 1 -- 172.21.15.12:0/2232912538 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f38007cc0 con 0x56156a1ad710 2024-05-09T00:02:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.975+0000 7f1f2bfff700 1 -- 172.21.15.12:0/2232912538 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f38003c60 con 0x56156a1ad710 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.976+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2232912538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 msgr2=0x56156a1ab210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.976+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a1ab210 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1f380062a0 tx=0x7f1f38009450 comp rx=0 tx=0).stop 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.976+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2232912538 shutdown_connections 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.976+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a1ab210 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.976+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a190d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.976+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/2232912538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.976+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2232912538 >> 172.21.15.12:0/2232912538 conn(0x56156a0f4210 msgr2=0x56156a1c7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.976+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2232912538 shutdown_connections 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.976+0000 7f1f537d2b80 1 -- 172.21.15.12:0/2232912538 wait complete. 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.976+0000 7f1f537d2b80 1 -- start start 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f537d2b80 1 Processor -- start 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f537d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a2165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a1b6e80 con 0x56156a1a78a0 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a216b00 con 0x56156a1ad710 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f537d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156a216c40 con 0x56156a1acdf0 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f537d2b80 10 client.0 _ll_register_callbacks cb 0x56156a08bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f453b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f453b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56480/0 (socket says 172.21.15.12:56480) 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f453b4700 1 -- 172.21.15.12:0/804747044 learned_addr learned my addr 172.21.15.12:0/804747044 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.977+0000 7f1f45bb5700 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a2165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f453b4700 1 -- 172.21.15.12:0/804747044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 msgr2=0x56156a1b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f453b4700 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f453b4700 1 -- 172.21.15.12:0/804747044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 msgr2=0x56156a2165c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f453b4700 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a2165c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f453b4700 1 -- 172.21.15.12:0/804747044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561569f13b60 con 0x56156a1a78a0 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f453b4700 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b6250 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c010f30 tx=0x7f1f3c030c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c61b33a32f623c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f28ff9700 1 -- 172.21.15.12:0/804747044 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c025a30 con 0x56156a1a78a0 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f537d2b80 1 -- 172.21.15.12:0/804747044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56156a03f270 con 0x56156a1a78a0 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f28ff9700 1 -- 172.21.15.12:0/804747044 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1f3c004830 con 0x56156a1a78a0 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f28ff9700 1 -- 172.21.15.12:0/804747044 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f3c025b90 con 0x56156a1a78a0 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f537d2b80 10 client.19911 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.978+0000 7f1f537d2b80 1 -- 172.21.15.12:0/804747044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x7f1f401aabb0 con 0x56156a1a78a0 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.979+0000 7f1f28ff9700 1 -- 172.21.15.12:0/804747044 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f3c052020 con 0x56156a1a78a0 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.979+0000 7f1f28ff9700 1 -- 172.21.15.12:0/804747044 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1f3c010430 con 0x56156a1a78a0 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.979+0000 7f1f537d2b80 10 client.19911 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.979+0000 7f1f537d2b80 10 client.19911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.979+0000 7f1f537d2b80 20 client.19911 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.979+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156a218160 0x56156a21a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:24.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.979+0000 7f1f537d2b80 4 client.19911 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:02:24.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.979+0000 7f1f537d2b80 1 -- 172.21.15.12:0/804747044 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561569f34cf0 con 0x56156a218160 2024-05-09T00:02:24.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.979+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156a218160 0x56156a21a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:24.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f44bb3700 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156a218160 0x56156a21a580 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b65e1a18de93af0 server_cookie=56c4d3af4d395ed5 in_seq=0 out_seq=0 2024-05-09T00:02:24.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f28ff9700 10 client.19911 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f28ff9700 1 -- 172.21.15.12:0/804747044 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561569f34cf0 con 0x56156a218160 2024-05-09T00:02:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f28ff9700 10 client.19911 handle_command_reply: tid=0 2024-05-09T00:02:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f537d2b80 1 client.19911 shutdown 2024-05-09T00:02:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f537d2b80 1 -- 172.21.15.12:0/804747044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 msgr2=0x56156a1b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b6250 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c010f30 tx=0x7f1f3c030c00 comp rx=0 tx=0).stop 2024-05-09T00:02:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f537d2b80 1 -- 172.21.15.12:0/804747044 shutdown_connections 2024-05-09T00:02:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156a218160 0x56156a21a580 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156a1ad710 0x56156a2165c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156a1a78a0 0x56156a1b6250 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f28ff9700 0 client.19911 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:24.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f537d2b80 1 --2- 172.21.15.12:0/804747044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156a1acdf0 0x56156a1b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.980+0000 7f1f537d2b80 1 -- 172.21.15.12:0/804747044 >> 172.21.15.12:0/804747044 conn(0x56156a0f4210 msgr2=0x56156a1af400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:24.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f537d2b80 1 -- 172.21.15.12:0/804747044 shutdown_connections 2024-05-09T00:02:24.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f537d2b80 1 -- 172.21.15.12:0/804747044 wait complete. 2024-05-09T00:02:24.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f537d2b80 20 client.19911 trim_cache size 0 max 16384 2024-05-09T00:02:24.980 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:02:24.980 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:02:24.980 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:02:24.981 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:02:24.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f47618700 1 -- 172.21.15.12:0/502298615 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f2c0a1290 msgr2=0x7f1f2c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:24.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f47618700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f2c0a1290 0x7f1f2c0a3750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c005950 tx=0x7f1f3c0058e0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f47618700 1 -- 172.21.15.12:0/502298615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f40106280 msgr2=0x7f1f400737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:24.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f47618700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f40106280 0x7f1f400737e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000ea30 tx=0x7f1f3000ed40 comp rx=0 tx=0).stop 2024-05-09T00:02:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f47618700 1 -- 172.21.15.12:0/502298615 shutdown_connections 2024-05-09T00:02:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f47618700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1f2c0a1290 0x7f1f2c0a3750 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f47618700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f40074f60 0x7f1f40073d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f47618700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101160 0x7f1f400732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.981+0000 7f1f47618700 1 --2- 172.21.15.12:0/502298615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f40106280 0x7f1f400737e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.982+0000 7f1f47618700 1 -- 172.21.15.12:0/502298615 >> 172.21.15.12:0/502298615 conn(0x7f1f400fc9e0 msgr2=0x7f1f400fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.982+0000 7f1f47618700 1 -- 172.21.15.12:0/502298615 shutdown_connections 2024-05-09T00:02:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:24.982+0000 7f1f47618700 1 -- 172.21.15.12:0/502298615 wait complete. 2024-05-09T00:02:24.991 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:02:24.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:02:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.159+0000 7f74f5caa700 1 -- 172.21.15.12:0/1568702616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74f010d770 msgr2=0x7f74f010fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.159+0000 7f74f5caa700 1 --2- 172.21.15.12:0/1568702616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74f010d770 0x7f74f010fb60 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b0a0 tx=0x7f74e800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.159+0000 7f74f5caa700 1 -- 172.21.15.12:0/1568702616 shutdown_connections 2024-05-09T00:02:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.159+0000 7f74f5caa700 1 --2- 172.21.15.12:0/1568702616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74f010d770 0x7f74f010fb60 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.159+0000 7f74f5caa700 1 --2- 172.21.15.12:0/1568702616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74f00ffbc0 0x7f74f00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.159+0000 7f74f5caa700 1 --2- 172.21.15.12:0/1568702616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74f0100570 0x7f74f010d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.159+0000 7f74f5caa700 1 -- 172.21.15.12:0/1568702616 >> 172.21.15.12:0/1568702616 conn(0x7f74f00fb810 msgr2=0x7f74f00fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.160+0000 7f74f5caa700 1 -- 172.21.15.12:0/1568702616 shutdown_connections 2024-05-09T00:02:25.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.160+0000 7f74f5caa700 1 -- 172.21.15.12:0/1568702616 wait complete. 2024-05-09T00:02:25.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.161+0000 7f74f5caa700 1 Processor -- start 2024-05-09T00:02:25.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.161+0000 7f74f5caa700 1 -- start start 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74f5caa700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74f00ffbc0 0x7f74f0199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74f5caa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74f0100570 0x7f74f0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74f5caa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74f010d770 0x7f74f019de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74f5caa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f019e480 con 0x7f74f0100570 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74f5caa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f019e5f0 con 0x7f74f010d770 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74f5caa700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f019e7b0 con 0x7f74f00ffbc0 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74ef7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74f00ffbc0 0x7f74f0199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74f0100570 0x7f74f0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74ef7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74f00ffbc0 0x7f74f0199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40318/0 (socket says 172.21.15.12:40318) 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74ef7fe700 1 -- 172.21.15.12:0/3796756023 learned_addr learned my addr 172.21.15.12:0/3796756023 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74effff700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74f010d770 0x7f74f019de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74eeffd700 1 -- 172.21.15.12:0/3796756023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74f00ffbc0 msgr2=0x7f74f0199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74eeffd700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74f00ffbc0 0x7f74f0199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74eeffd700 1 -- 172.21.15.12:0/3796756023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74f010d770 msgr2=0x7f74f019de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74eeffd700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74f010d770 0x7f74f019de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74eeffd700 1 -- 172.21.15.12:0/3796756023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e8009d00 con 0x7f74f0100570 2024-05-09T00:02:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.162+0000 7f74effff700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74f010d770 0x7f74f019de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.163+0000 7f74eeffd700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74f0100570 0x7f74f0199aa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f74e4006fd0 tx=0x7f74e400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.163+0000 7f74ecff9700 1 -- 172.21.15.12:0/3796756023 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4011830 con 0x7f74f0100570 2024-05-09T00:02:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.163+0000 7f74ecff9700 1 -- 172.21.15.12:0/3796756023 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74e4011e50 con 0x7f74f0100570 2024-05-09T00:02:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.163+0000 7f74ecff9700 1 -- 172.21.15.12:0/3796756023 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4010d70 con 0x7f74f0100570 2024-05-09T00:02:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.163+0000 7f74f5caa700 1 -- 172.21.15.12:0/3796756023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f019ea90 con 0x7f74f0100570 2024-05-09T00:02:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.163+0000 7f74f5caa700 1 -- 172.21.15.12:0/3796756023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f01a62d0 con 0x7f74f0100570 2024-05-09T00:02:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.164+0000 7f74f5caa700 1 -- 172.21.15.12:0/3796756023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f0108900 con 0x7f74f0100570 2024-05-09T00:02:25.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.166+0000 7f74ecff9700 1 -- 172.21.15.12:0/3796756023 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f74e4010410 con 0x7f74f0100570 2024-05-09T00:02:25.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.166+0000 7f74ecff9700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f74d80a1390 0x7f74d80a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.167+0000 7f74ecff9700 1 -- 172.21.15.12:0/3796756023 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f74e40cca40 con 0x7f74f0100570 2024-05-09T00:02:25.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.167+0000 7f74ef7fe700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f74d80a1390 0x7f74d80a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.167+0000 7f74ef7fe700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f74d80a1390 0x7f74d80a3850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009d20 tx=0x7f74e0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.171+0000 7f74ecff9700 1 -- 172.21.15.12:0/3796756023 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f74e4095f50 con 0x7f74f0100570 2024-05-09T00:02:25.231 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103241 43.63 MB/sec execute 20 sec latency 103.043 ms 2024-05-09T00:02:25.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.537+0000 7f74f5caa700 1 -- 172.21.15.12:0/3796756023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74f019a860 con 0x7f74f0100570 2024-05-09T00:02:25.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.538+0000 7f74ecff9700 1 -- 172.21.15.12:0/3796756023 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+10883 (secure 0 0 0) 0x7f74e4015020 con 0x7f74f0100570 2024-05-09T00:02:25.533 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:02:25.533 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":64,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:02:21.116693+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_29453":{"gid":29453,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3146138040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3146138040},{"type":"v1","addr":"172.21.15.204:6837","nonce":3146138040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:02:25.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.541+0000 7f74f5caa700 1 -- 172.21.15.12:0/3796756023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f74d80a1390 msgr2=0x7f74d80a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.541+0000 7f74f5caa700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f74d80a1390 0x7f74d80a3850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009d20 tx=0x7f74e0009260 comp rx=0 tx=0).stop 2024-05-09T00:02:25.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.541+0000 7f74f5caa700 1 -- 172.21.15.12:0/3796756023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74f0100570 msgr2=0x7f74f0199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.541+0000 7f74f5caa700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74f0100570 0x7f74f0199aa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f74e4006fd0 tx=0x7f74e400b730 comp rx=0 tx=0).stop 2024-05-09T00:02:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.541+0000 7f74f5caa700 1 -- 172.21.15.12:0/3796756023 shutdown_connections 2024-05-09T00:02:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.541+0000 7f74f5caa700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f74d80a1390 0x7f74d80a3850 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.541+0000 7f74f5caa700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74f010d770 0x7f74f019de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.541+0000 7f74f5caa700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74f00ffbc0 0x7f74f0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.541+0000 7f74f5caa700 1 --2- 172.21.15.12:0/3796756023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74f0100570 0x7f74f0199aa0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.541+0000 7f74f5caa700 1 -- 172.21.15.12:0/3796756023 >> 172.21.15.12:0/3796756023 conn(0x7f74f00fb810 msgr2=0x7f74f00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.542+0000 7f74f5caa700 1 -- 172.21.15.12:0/3796756023 shutdown_connections 2024-05-09T00:02:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.542+0000 7f74f5caa700 1 -- 172.21.15.12:0/3796756023 wait complete. 2024-05-09T00:02:25.537 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2024-05-09T00:02:25.552 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:02:25.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.734+0000 7f532a67b700 1 -- 172.21.15.12:0/1396147510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5324101b10 msgr2=0x7f5324105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.734+0000 7f532a67b700 1 --2- 172.21.15.12:0/1396147510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5324101b10 0x7f5324105b50 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f5318009990 tx=0x7f5318009ca0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.734+0000 7f532a67b700 1 -- 172.21.15.12:0/1396147510 shutdown_connections 2024-05-09T00:02:25.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.734+0000 7f532a67b700 1 --2- 172.21.15.12:0/1396147510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5324106280 0x7f532410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.734+0000 7f532a67b700 1 --2- 172.21.15.12:0/1396147510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5324101160 0x7f5324101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.734+0000 7f532a67b700 1 --2- 172.21.15.12:0/1396147510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5324101b10 0x7f5324105b50 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.734+0000 7f532a67b700 1 -- 172.21.15.12:0/1396147510 >> 172.21.15.12:0/1396147510 conn(0x7f53240fc9e0 msgr2=0x7f53240fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.734+0000 7f532a67b700 1 -- 172.21.15.12:0/1396147510 shutdown_connections 2024-05-09T00:02:25.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.734+0000 7f532a67b700 1 -- 172.21.15.12:0/1396147510 wait complete. 2024-05-09T00:02:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.735+0000 7f532a67b700 1 Processor -- start 2024-05-09T00:02:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.735+0000 7f532a67b700 1 -- start start 2024-05-09T00:02:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.735+0000 7f532a67b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5324101160 0x7f53241973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.735+0000 7f532a67b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5324106280 0x7f5324197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.735+0000 7f532a67b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f532419bcb0 0x7f532419c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.735+0000 7f532a67b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532419c730 con 0x7f532419bcb0 2024-05-09T00:02:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.735+0000 7f532a67b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532419c8a0 con 0x7f5324101160 2024-05-09T00:02:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.735+0000 7f532a67b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532419ca60 con 0x7f5324106280 2024-05-09T00:02:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f5323fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5324101160 0x7f53241973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f5323fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5324101160 0x7f53241973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45454/0 (socket says 172.21.15.12:45454) 2024-05-09T00:02:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f5323fff700 1 -- 172.21.15.12:0/1222541295 learned_addr learned my addr 172.21.15.12:0/1222541295 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f53237fe700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5324106280 0x7f5324197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f5328c18700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f532419bcb0 0x7f532419c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f5323fff700 1 -- 172.21.15.12:0/1222541295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5324106280 msgr2=0x7f5324197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f5323fff700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5324106280 0x7f5324197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f5323fff700 1 -- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f532419bcb0 msgr2=0x7f532419c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f5323fff700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f532419bcb0 0x7f532419c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f5323fff700 1 -- 172.21.15.12:0/1222541295 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53180095f0 con 0x7f5324101160 2024-05-09T00:02:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.736+0000 7f5328c18700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f532419bcb0 0x7f532419c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.737+0000 7f5323fff700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5324101160 0x7f53241973d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f531400e9a0 tx=0x7f531400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.737+0000 7f53217fa700 1 -- 172.21.15.12:0/1222541295 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314009790 con 0x7f5324101160 2024-05-09T00:02:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.737+0000 7f532a67b700 1 -- 172.21.15.12:0/1222541295 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53241a3fc0 con 0x7f5324101160 2024-05-09T00:02:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.737+0000 7f53217fa700 1 -- 172.21.15.12:0/1222541295 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531400cbd0 con 0x7f5324101160 2024-05-09T00:02:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.737+0000 7f53217fa700 1 -- 172.21.15.12:0/1222541295 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314018680 con 0x7f5324101160 2024-05-09T00:02:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.737+0000 7f532a67b700 1 -- 172.21.15.12:0/1222541295 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53241a4440 con 0x7f5324101160 2024-05-09T00:02:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.740+0000 7f53217fa700 1 -- 172.21.15.12:0/1222541295 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f531400cd40 con 0x7f5324101160 2024-05-09T00:02:25.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.741+0000 7f53217fa700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f530c0a1290 0x7f530c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.741+0000 7f53237fe700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f530c0a1290 0x7f530c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.741+0000 7f53217fa700 1 -- 172.21.15.12:0/1222541295 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5314014070 con 0x7f5324101160 2024-05-09T00:02:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.742+0000 7f53237fe700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f530c0a1290 0x7f530c0a3750 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5318006010 tx=0x7f531800b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5336835b80 1 Processor -- start 2024-05-09T00:02:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5336835b80 1 -- start start 2024-05-09T00:02:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5336835b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5e1f0 0x56147ac5e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5336835b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5eb10 0x56147ac58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5336835b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 0x56147ac59100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5336835b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147abab150 con 0x56147ac5e1f0 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5336835b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147aba8b10 con 0x56147ac58ca0 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5336835b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147aba4400 con 0x56147ac5eb10 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5323fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5e1f0 0x56147ac5e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5323fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5e1f0 0x56147ac5e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56520/0 (socket says 172.21.15.12:56520) 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5323fff700 1 -- 172.21.15.12:0/2467787217 learned_addr learned my addr 172.21.15.12:0/2467787217 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f5328c18700 1 --2- 172.21.15.12:0/2467787217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 0x56147ac59100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.743+0000 7f53237fe700 1 --2- 172.21.15.12:0/2467787217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5eb10 0x56147ac58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.744+0000 7f5328c18700 1 -- 172.21.15.12:0/2467787217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5eb10 msgr2=0x56147ac58760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.744+0000 7f5328c18700 1 --2- 172.21.15.12:0/2467787217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5eb10 0x56147ac58760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.744+0000 7f5328c18700 1 -- 172.21.15.12:0/2467787217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5e1f0 msgr2=0x56147ac5e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.744+0000 7f5328c18700 1 --2- 172.21.15.12:0/2467787217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5e1f0 0x56147ac5e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.744+0000 7f5328c18700 1 -- 172.21.15.12:0/2467787217 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56147aa253b0 con 0x56147ac58ca0 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.744+0000 7f53237fe700 1 --2- 172.21.15.12:0/2467787217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5eb10 0x56147ac58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.744+0000 7f5328c18700 1 --2- 172.21.15.12:0/2467787217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 0x56147ac59100 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f531c003db0 tx=0x7f531c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcaf9dbda87f45c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.744+0000 7f530a7fc700 1 -- 172.21.15.12:0/2467787217 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c004bb0 con 0x56147ac58ca0 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.745+0000 7f530a7fc700 1 -- 172.21.15.12:0/2467787217 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531c00f040 con 0x56147ac58ca0 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.745+0000 7f530a7fc700 1 -- 172.21.15.12:0/2467787217 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c007cb0 con 0x56147ac58ca0 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.745+0000 7f5336835b80 1 -- 172.21.15.12:0/2467787217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 msgr2=0x56147ac59100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.745+0000 7f5336835b80 1 --2- 172.21.15.12:0/2467787217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 0x56147ac59100 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f531c003db0 tx=0x7f531c009920 comp rx=0 tx=0).stop 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.745+0000 7f5336835b80 1 -- 172.21.15.12:0/2467787217 shutdown_connections 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.745+0000 7f5336835b80 1 --2- 172.21.15.12:0/2467787217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 0x56147ac59100 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.745+0000 7f5336835b80 1 --2- 172.21.15.12:0/2467787217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5e1f0 0x56147ac5e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.745+0000 7f5336835b80 1 --2- 172.21.15.12:0/2467787217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5eb10 0x56147ac58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.745+0000 7f5336835b80 1 -- 172.21.15.12:0/2467787217 >> 172.21.15.12:0/2467787217 conn(0x56147abb2210 msgr2=0x56147ac4edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.746+0000 7f5336835b80 1 -- 172.21.15.12:0/2467787217 shutdown_connections 2024-05-09T00:02:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.746+0000 7f5336835b80 1 -- 172.21.15.12:0/2467787217 wait complete. 2024-05-09T00:02:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.746+0000 7f5336835b80 1 -- start start 2024-05-09T00:02:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.746+0000 7f5336835b80 1 Processor -- start 2024-05-09T00:02:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f5336835b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 0x56147ac61f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f5336835b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 0x56147ac62450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f5336835b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 0x56147ac62990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f5336835b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147ab40560 con 0x56147ac5eb10 2024-05-09T00:02:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f5336835b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147ab40200 con 0x56147ac58ca0 2024-05-09T00:02:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f5336835b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147aa276a0 con 0x56147ac5e1f0 2024-05-09T00:02:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f5336835b80 10 client.0 _ll_register_callbacks cb 0x56147ab49b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f53237fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 0x56147ac62450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f53237fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 0x56147ac62450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40358/0 (socket says 172.21.15.12:40358) 2024-05-09T00:02:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f53237fe700 1 -- 172.21.15.12:0/1500787715 learned_addr learned my addr 172.21.15.12:0/1500787715 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f5323fff700 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 0x56147ac61f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f5328c18700 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 0x56147ac62990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f53237fe700 1 -- 172.21.15.12:0/1500787715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 msgr2=0x56147ac61f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f53237fe700 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 0x56147ac61f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f53237fe700 1 -- 172.21.15.12:0/1500787715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 msgr2=0x56147ac62990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f53237fe700 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 0x56147ac62990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.747+0000 7f53237fe700 1 -- 172.21.15.12:0/1500787715 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532410e670 con 0x56147ac5e1f0 2024-05-09T00:02:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.748+0000 7f53237fe700 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 0x56147ac62450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5318009990 tx=0x7f531801a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca4c5db78e2dfc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.748+0000 7f53037fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:02:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.748+0000 7f53037fe700 1 -- 172.21.15.12:0/1500787715 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531802b050 con 0x56147ac5e1f0 2024-05-09T00:02:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.748+0000 7f53037fe700 1 -- 172.21.15.12:0/1500787715 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5318004af0 con 0x56147ac5e1f0 2024-05-09T00:02:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.748+0000 7f5336835b80 1 -- 172.21.15.12:0/1500787715 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56147aafd270 con 0x56147ac5e1f0 2024-05-09T00:02:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.748+0000 7f53037fe700 1 -- 172.21.15.12:0/1500787715 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f531800bed0 con 0x56147ac5e1f0 2024-05-09T00:02:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.748+0000 7f5336835b80 10 client.29665 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.748+0000 7f5336835b80 1 -- 172.21.15.12:0/1500787715 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x7f531c003a60 con 0x56147ac5e1f0 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.748+0000 7f53037fe700 1 -- 172.21.15.12:0/1500787715 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56147aafd270 con 0x56147ac5e1f0 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.749+0000 7f53037fe700 1 -- 172.21.15.12:0/1500787715 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5318024410 con 0x56147ac5e1f0 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.749+0000 7f5336835b80 10 client.29665 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.749+0000 7f5336835b80 10 client.29665 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.749+0000 7f5336835b80 20 client.29665 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.749+0000 7f5336835b80 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56147ac726e0 0x56147ac74b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.749+0000 7f5336835b80 4 client.29665 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.749+0000 7f5336835b80 1 -- 172.21.15.12:0/1500787715 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56147ac23810 con 0x56147ac726e0 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.749+0000 7f5323fff700 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56147ac726e0 0x56147ac74b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.749+0000 7f53037fe700 10 client.29665 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:02:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.749+0000 7f5323fff700 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56147ac726e0 0x56147ac74b00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d158118814711089 server_cookie=d929cc52a9a6f83d in_seq=0 out_seq=0 2024-05-09T00:02:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f53037fe700 1 -- 172.21.15.12:0/1500787715 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56147ac23810 con 0x56147ac726e0 2024-05-09T00:02:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f53037fe700 10 client.29665 handle_command_reply: tid=0 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f5336835b80 1 client.29665 shutdown 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f5336835b80 1 -- 172.21.15.12:0/1500787715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 msgr2=0x56147ac62450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f5336835b80 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 0x56147ac62450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5318009990 tx=0x7f531801a050 comp rx=0 tx=0).stop 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f5336835b80 1 -- 172.21.15.12:0/1500787715 shutdown_connections 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f5336835b80 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 0x56147ac62990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f5336835b80 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac58ca0 0x56147ac61f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f5336835b80 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56147ac726e0 0x56147ac74b00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f5336835b80 1 --2- 172.21.15.12:0/1500787715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 0x56147ac62450 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f53037fe700 0 client.29665 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.753+0000 7f5336835b80 1 -- 172.21.15.12:0/1500787715 >> 172.21.15.12:0/1500787715 conn(0x56147abb2210 msgr2=0x56147ac4e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.754+0000 7f5336835b80 1 -- 172.21.15.12:0/1500787715 shutdown_connections 2024-05-09T00:02:25.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.754+0000 7f5336835b80 1 -- 172.21.15.12:0/1500787715 wait complete. 2024-05-09T00:02:25.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.754+0000 7f5336835b80 20 client.29665 trim_cache size 0 max 16384 2024-05-09T00:02:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.772+0000 7f5336835b80 1 Processor -- start 2024-05-09T00:02:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.772+0000 7f5336835b80 1 -- start start 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5336835b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac58ca0 0x56147ac50a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5336835b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 0x56147ac4ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5336835b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac5eb10 0x56147ac5c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5336835b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147ac63510 con 0x56147ac58ca0 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5336835b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147ac6fcf0 con 0x56147ac5eb10 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5336835b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147ac61d00 con 0x56147ac5e1f0 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5323fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac58ca0 0x56147ac50a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5323fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac58ca0 0x56147ac50a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56536/0 (socket says 172.21.15.12:56536) 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5323fff700 1 -- 172.21.15.12:0/1347039527 learned_addr learned my addr 172.21.15.12:0/1347039527 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f53237fe700 1 --2- 172.21.15.12:0/1347039527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 0x56147ac4ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5328c18700 1 --2- 172.21.15.12:0/1347039527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac5eb10 0x56147ac5c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5323fff700 1 -- 172.21.15.12:0/1347039527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 msgr2=0x56147ac4ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5323fff700 1 --2- 172.21.15.12:0/1347039527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 0x56147ac4ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5323fff700 1 -- 172.21.15.12:0/1347039527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac5eb10 msgr2=0x56147ac5c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5323fff700 1 --2- 172.21.15.12:0/1347039527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac5eb10 0x56147ac5c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.773+0000 7f5323fff700 1 -- 172.21.15.12:0/1347039527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56147a9d1b60 con 0x56147ac58ca0 2024-05-09T00:02:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.774+0000 7f5323fff700 1 --2- 172.21.15.12:0/1347039527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac58ca0 0x56147ac50a00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f53140caca0 tx=0x7f531401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee441f09bb971f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.774+0000 7f530affd700 1 -- 172.21.15.12:0/1347039527 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531409a060 con 0x56147ac58ca0 2024-05-09T00:02:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.774+0000 7f530affd700 1 -- 172.21.15.12:0/1347039527 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531409de90 con 0x56147ac58ca0 2024-05-09T00:02:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.774+0000 7f530affd700 1 -- 172.21.15.12:0/1347039527 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53140997b0 con 0x56147ac58ca0 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.775+0000 7f5336835b80 1 -- 172.21.15.12:0/1347039527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac58ca0 msgr2=0x56147ac50a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.775+0000 7f5336835b80 1 --2- 172.21.15.12:0/1347039527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac58ca0 0x56147ac50a00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f53140caca0 tx=0x7f531401a040 comp rx=0 tx=0).stop 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.775+0000 7f5336835b80 1 -- 172.21.15.12:0/1347039527 shutdown_connections 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.775+0000 7f5336835b80 1 --2- 172.21.15.12:0/1347039527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac5eb10 0x56147ac5c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.775+0000 7f5336835b80 1 --2- 172.21.15.12:0/1347039527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac58ca0 0x56147ac50a00 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.775+0000 7f5336835b80 1 --2- 172.21.15.12:0/1347039527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac5e1f0 0x56147ac4ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.775+0000 7f5336835b80 1 -- 172.21.15.12:0/1347039527 >> 172.21.15.12:0/1347039527 conn(0x56147abb2210 msgr2=0x56147ac59080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.775+0000 7f5336835b80 1 -- 172.21.15.12:0/1347039527 shutdown_connections 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.775+0000 7f5336835b80 1 -- 172.21.15.12:0/1347039527 wait complete. 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.775+0000 7f5336835b80 1 -- start start 2024-05-09T00:02:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5336835b80 1 Processor -- start 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5336835b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac58ca0 0x56147acc3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5336835b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac5e1f0 0x56147acc4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5336835b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 0x56147acc4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5336835b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147ac676a0 con 0x56147ac5eb10 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5336835b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147ac677e0 con 0x56147ac5e1f0 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5336835b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147ac679a0 con 0x56147ac58ca0 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5336835b80 10 client.0 _ll_register_callbacks cb 0x56147ab49b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5328c18700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 0x56147acc4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5328c18700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 0x56147acc4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56544/0 (socket says 172.21.15.12:56544) 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5328c18700 1 -- 172.21.15.12:0/1114636161 learned_addr learned my addr 172.21.15.12:0/1114636161 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f5323fff700 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac58ca0 0x56147acc3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.776+0000 7f53237fe700 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac5e1f0 0x56147acc4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f5328c18700 1 -- 172.21.15.12:0/1114636161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac58ca0 msgr2=0x56147acc3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f5328c18700 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac58ca0 0x56147acc3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f5328c18700 1 -- 172.21.15.12:0/1114636161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac5e1f0 msgr2=0x56147acc4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f5328c18700 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac5e1f0 0x56147acc4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f5328c18700 1 -- 172.21.15.12:0/1114636161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53241a4440 con 0x56147ac5eb10 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f5323fff700 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac58ca0 0x56147acc3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f5328c18700 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 0x56147acc4640 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f531c009b30 tx=0x7f531c004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e44357f0733b7b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f53037fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f53037fe700 1 -- 172.21.15.12:0/1114636161 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c007710 con 0x56147ac5eb10 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f53037fe700 1 -- 172.21.15.12:0/1114636161 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f531c007870 con 0x56147ac5eb10 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.777+0000 7f5336835b80 1 -- 172.21.15.12:0/1114636161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f531c00f040 con 0x56147ac5eb10 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f53037fe700 1 -- 172.21.15.12:0/1114636161 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f531c007a30 con 0x56147ac5eb10 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f5336835b80 10 client.19935 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f5336835b80 1 -- 172.21.15.12:0/1114636161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x56147ab6a560 con 0x56147ac5eb10 2024-05-09T00:02:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f53037fe700 1 -- 172.21.15.12:0/1114636161 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531c035080 con 0x56147ac5eb10 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f53037fe700 1 -- 172.21.15.12:0/1114636161 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f531c01f450 con 0x56147ac5eb10 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f5336835b80 10 client.19935 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f5336835b80 10 client.19935 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f5336835b80 20 client.19935 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f5336835b80 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56147acda710 0x56147acdcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f5336835b80 4 client.19935 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f5336835b80 1 -- 172.21.15.12:0/1114636161 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56147ab9cec0 con 0x56147acda710 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.778+0000 7f5323fff700 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56147acda710 0x56147acdcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.779+0000 7f5323fff700 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56147acda710 0x56147acdcb30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad6378d68599708c server_cookie=60b84edac5a2461d in_seq=0 out_seq=0 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.779+0000 7f53037fe700 10 client.19935 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.779+0000 7f53037fe700 1 -- 172.21.15.12:0/1114636161 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56147ab9cec0 con 0x56147acda710 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.779+0000 7f53037fe700 10 client.19935 handle_command_reply: tid=0 2024-05-09T00:02:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.779+0000 7f5336835b80 1 client.19935 shutdown 2024-05-09T00:02:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.779+0000 7f5336835b80 1 -- 172.21.15.12:0/1114636161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 msgr2=0x56147acc4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.779+0000 7f5336835b80 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 0x56147acc4640 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f531c009b30 tx=0x7f531c004bd0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.780+0000 7f5336835b80 1 -- 172.21.15.12:0/1114636161 shutdown_connections 2024-05-09T00:02:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.780+0000 7f5336835b80 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56147ac5eb10 0x56147acc4640 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.780+0000 7f5336835b80 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56147acda710 0x56147acdcb30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.780+0000 7f5336835b80 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56147ac58ca0 0x56147acc3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.780+0000 7f5336835b80 1 --2- 172.21.15.12:0/1114636161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56147ac5e1f0 0x56147acc4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.780+0000 7f5336835b80 1 -- 172.21.15.12:0/1114636161 >> 172.21.15.12:0/1114636161 conn(0x56147abb2210 msgr2=0x56147ac5c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.780+0000 7f53037fe700 0 client.19935 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:02:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.781+0000 7f5336835b80 1 -- 172.21.15.12:0/1114636161 shutdown_connections 2024-05-09T00:02:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.782+0000 7f5336835b80 1 -- 172.21.15.12:0/1114636161 wait complete. 2024-05-09T00:02:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.782+0000 7f5336835b80 20 client.19935 trim_cache size 0 max 16384 2024-05-09T00:02:25.780 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:02:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.782+0000 7f532a67b700 1 -- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f530c0a1290 msgr2=0x7f530c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.782+0000 7f532a67b700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f530c0a1290 0x7f530c0a3750 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5318006010 tx=0x7f531800b260 comp rx=0 tx=0).stop 2024-05-09T00:02:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.782+0000 7f532a67b700 1 -- 172.21.15.12:0/1222541295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5324101160 msgr2=0x7f53241973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.782+0000 7f532a67b700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5324101160 0x7f53241973d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f531400e9a0 tx=0x7f531400ecb0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.783+0000 7f532a67b700 1 -- 172.21.15.12:0/1222541295 shutdown_connections 2024-05-09T00:02:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.783+0000 7f532a67b700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f530c0a1290 0x7f530c0a3750 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.783+0000 7f532a67b700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f532419bcb0 0x7f532419c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.783+0000 7f532a67b700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5324101160 0x7f53241973d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.783+0000 7f532a67b700 1 --2- 172.21.15.12:0/1222541295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5324106280 0x7f5324197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.783+0000 7f532a67b700 1 -- 172.21.15.12:0/1222541295 >> 172.21.15.12:0/1222541295 conn(0x7f53240fc9e0 msgr2=0x7f53240fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.783+0000 7f532a67b700 1 -- 172.21.15.12:0/1222541295 shutdown_connections 2024-05-09T00:02:25.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.783+0000 7f532a67b700 1 -- 172.21.15.12:0/1222541295 wait complete. 2024-05-09T00:02:25.791 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:02:25.827 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:25 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3796756023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:02:25.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.979+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/3113630870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdf4105180 msgr2=0x7fcdf410cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.979+0000 7fcdfa9a7700 1 --2- 172.21.15.12:0/3113630870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdf4105180 0x7fcdf410cd00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00d960 tx=0x7fcdec00dc70 comp rx=0 tx=0).stop 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.979+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/3113630870 shutdown_connections 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.979+0000 7fcdfa9a7700 1 --2- 172.21.15.12:0/3113630870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdf4105180 0x7fcdf410cd00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.979+0000 7fcdfa9a7700 1 --2- 172.21.15.12:0/3113630870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100060 0x7fcdf4100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.979+0000 7fcdfa9a7700 1 --2- 172.21.15.12:0/3113630870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf4100a10 0x7fcdf4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.979+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/3113630870 >> 172.21.15.12:0/3113630870 conn(0x7fcdf4076fa0 msgr2=0x7fcdf40773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.979+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/3113630870 shutdown_connections 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.979+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/3113630870 wait complete. 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.979+0000 7fcdfa9a7700 1 Processor -- start 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdfa9a7700 1 -- start start 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdfa9a7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100060 0x7fcdf4195170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdfa9a7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf4105180 0x7fcdf41956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdfa9a7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdf4199a50 0x7fcdf4199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdfa9a7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf419a4d0 con 0x7fcdf4105180 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdfa9a7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf419a640 con 0x7fcdf4199a50 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdfa9a7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf419a800 con 0x7fcdf4100060 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdf3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100060 0x7fcdf4195170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdf3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100060 0x7fcdf4195170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40382/0 (socket says 172.21.15.12:40382) 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdf3fff700 1 -- 172.21.15.12:0/1407649227 learned_addr learned my addr 172.21.15.12:0/1407649227 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdeb7fe700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf4105180 0x7fcdf41956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdf8f44700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdf4199a50 0x7fcdf4199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdf3fff700 1 -- 172.21.15.12:0/1407649227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdf4199a50 msgr2=0x7fcdf4199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdf4199a50 0x7fcdf4199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdf3fff700 1 -- 172.21.15.12:0/1407649227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf4105180 msgr2=0x7fcdf41956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf4105180 0x7fcdf41956b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.980+0000 7fcdf3fff700 1 -- 172.21.15.12:0/1407649227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec00d590 con 0x7fcdf4100060 2024-05-09T00:02:25.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.981+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100060 0x7fcdf4195170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c0c0 tx=0x7fcde400c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.981+0000 7fcdf1ffb700 1 -- 172.21.15.12:0/1407649227 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4018070 con 0x7fcdf4100060 2024-05-09T00:02:25.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.981+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/1407649227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf41a1d00 con 0x7fcdf4100060 2024-05-09T00:02:25.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.981+0000 7fcdf1ffb700 1 -- 172.21.15.12:0/1407649227 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcde400f040 con 0x7fcdf4100060 2024-05-09T00:02:25.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.981+0000 7fcdf1ffb700 1 -- 172.21.15.12:0/1407649227 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde400e020 con 0x7fcdf4100060 2024-05-09T00:02:25.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.981+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/1407649227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf41a2190 con 0x7fcdf4100060 2024-05-09T00:02:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.984+0000 7fcdf1ffb700 1 -- 172.21.15.12:0/1407649227 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcde400ca50 con 0x7fcdf4100060 2024-05-09T00:02:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.985+0000 7fcdf1ffb700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcde00a12d0 0x7fcde00a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.985+0000 7fcdf1ffb700 1 -- 172.21.15.12:0/1407649227 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcde40cca30 con 0x7fcdf4100060 2024-05-09T00:02:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.985+0000 7fcdeb7fe700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcde00a12d0 0x7fcde00a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.985+0000 7fce06b61b80 1 Processor -- start 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.985+0000 7fce06b61b80 1 -- start start 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fde00 0x555cc02fe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fe720 0x555cc02f8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 0x555cc02f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fce06b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc024f150 con 0x555cc02fe720 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fce06b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc024cb10 con 0x555cc02f8890 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fce06b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc0248400 con 0x555cc02fde00 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdeb7fe700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcde00a12d0 0x7fcde00a3790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcdf41966c0 tx=0x7fcddc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdeb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fe720 0x555cc02f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdeb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fe720 0x555cc02f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56552/0 (socket says 172.21.15.12:56552) 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdeb7fe700 1 -- 172.21.15.12:0/3647364433 learned_addr learned my addr 172.21.15.12:0/3647364433 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/3647364433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fde00 0x555cc02fe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdeb7fe700 1 -- 172.21.15.12:0/3647364433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fde00 msgr2=0x555cc02fe1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdeb7fe700 1 --2- 172.21.15.12:0/3647364433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fde00 0x555cc02fe1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdeb7fe700 1 -- 172.21.15.12:0/3647364433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 msgr2=0x555cc02f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdeb7fe700 1 --2- 172.21.15.12:0/3647364433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 0x555cc02f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdeb7fe700 1 -- 172.21.15.12:0/3647364433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cc00c93b0 con 0x555cc02fe720 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.986+0000 7fcdeb7fe700 1 --2- 172.21.15.12:0/3647364433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fe720 0x555cc02f8350 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011cd0 tx=0x7fcddc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e148585f242edb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.987+0000 7fcdea7fc700 1 -- 172.21.15.12:0/3647364433 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc004830 con 0x555cc02fe720 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.987+0000 7fcdea7fc700 1 -- 172.21.15.12:0/3647364433 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcddc004e50 con 0x555cc02fe720 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.987+0000 7fcdea7fc700 1 -- 172.21.15.12:0/3647364433 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc005350 con 0x555cc02fe720 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.987+0000 7fce06b61b80 1 -- 172.21.15.12:0/3647364433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fe720 msgr2=0x555cc02f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.987+0000 7fce06b61b80 1 --2- 172.21.15.12:0/3647364433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fe720 0x555cc02f8350 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011cd0 tx=0x7fcddc014000 comp rx=0 tx=0).stop 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.988+0000 7fce06b61b80 1 -- 172.21.15.12:0/3647364433 shutdown_connections 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.988+0000 7fce06b61b80 1 --2- 172.21.15.12:0/3647364433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 0x555cc02f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.988+0000 7fce06b61b80 1 --2- 172.21.15.12:0/3647364433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fde00 0x555cc02fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.988+0000 7fce06b61b80 1 --2- 172.21.15.12:0/3647364433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fe720 0x555cc02f8350 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.988+0000 7fce06b61b80 1 -- 172.21.15.12:0/3647364433 >> 172.21.15.12:0/3647364433 conn(0x555cc0256210 msgr2=0x555cc02f2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.988+0000 7fce06b61b80 1 -- 172.21.15.12:0/3647364433 shutdown_connections 2024-05-09T00:02:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.988+0000 7fce06b61b80 1 -- 172.21.15.12:0/3647364433 wait complete. 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.988+0000 7fce06b61b80 1 -- start start 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fce06b61b80 1 Processor -- start 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 0x555cc03045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 0x555cc0301b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fe720 0x555cc03020b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fce06b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc01e4560 con 0x555cc02fde00 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fce06b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc01e4200 con 0x555cc02f8890 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fce06b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc00cb6a0 con 0x555cc02fe720 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fce06b61b80 10 client.0 _ll_register_callbacks cb 0x555cc01edb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fcdf3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 0x555cc03045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fcdf3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 0x555cc03045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45558/0 (socket says 172.21.15.12:45558) 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fcdf3fff700 1 -- 172.21.15.12:0/2728232864 learned_addr learned my addr 172.21.15.12:0/2728232864 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fcdf8f44700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fe720 0x555cc03020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.989+0000 7fcdeb7fe700 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 0x555cc0301b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fcdf8f44700 1 -- 172.21.15.12:0/2728232864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 msgr2=0x555cc03045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fcdf8f44700 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 0x555cc03045d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fcdf8f44700 1 -- 172.21.15.12:0/2728232864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 msgr2=0x555cc0301b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fcdf8f44700 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 0x555cc0301b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fcdf8f44700 1 -- 172.21.15.12:0/2728232864 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc017070 con 0x555cc02fe720 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 0x555cc03045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fcdf8f44700 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fe720 0x555cc03020b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00d880 tx=0x7fcdec00d270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79fe88ff03afdc17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fcdd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fcdd37fe700 1 -- 172.21.15.12:0/2728232864 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00d410 con 0x555cc02fe720 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fcdd37fe700 1 -- 172.21.15.12:0/2728232864 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcdec009990 con 0x555cc02fe720 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.990+0000 7fce06b61b80 1 -- 172.21.15.12:0/2728232864 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cc01a1270 con 0x555cc02fe720 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fcdd37fe700 1 -- 172.21.15.12:0/2728232864 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdec009b50 con 0x555cc02fe720 2024-05-09T00:02:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fce06b61b80 10 client.29683 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fce06b61b80 1 -- 172.21.15.12:0/2728232864 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x555cc0075b60 con 0x555cc02fe720 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fcdd37fe700 1 -- 172.21.15.12:0/2728232864 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555cc01a1270 con 0x555cc02fe720 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fcdd37fe700 1 -- 172.21.15.12:0/2728232864 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fcdec01c030 con 0x555cc02fe720 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fce06b61b80 10 client.29683 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fce06b61b80 10 client.29683 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fce06b61b80 20 client.29683 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fce06b61b80 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555cc030c220 0x555cc030e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fce06b61b80 4 client.29683 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fce06b61b80 1 -- 172.21.15.12:0/2728232864 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cc02c7810 con 0x555cc030c220 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.991+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555cc030c220 0x555cc030e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.992+0000 7fcdd37fe700 10 client.29683 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:02:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.992+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555cc030c220 0x555cc030e640 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43d8c357889f89fc server_cookie=ea7df655bd7ab7e1 in_seq=0 out_seq=0 2024-05-09T00:02:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fcdd37fe700 1 -- 172.21.15.12:0/2728232864 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555cc02c7810 con 0x555cc030c220 2024-05-09T00:02:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fcdd37fe700 10 client.29683 handle_command_reply: tid=0 2024-05-09T00:02:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 client.29683 shutdown 2024-05-09T00:02:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 -- 172.21.15.12:0/2728232864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fe720 msgr2=0x555cc03020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fe720 0x555cc03020b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00d880 tx=0x7fcdec00d270 comp rx=0 tx=0).stop 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 -- 172.21.15.12:0/2728232864 shutdown_connections 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555cc030c220 0x555cc030e640 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fe720 0x555cc03020b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02f8890 0x555cc03045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 --2- 172.21.15.12:0/2728232864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 0x555cc0301b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 -- 172.21.15.12:0/2728232864 >> 172.21.15.12:0/2728232864 conn(0x555cc0256210 msgr2=0x555cc02f25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fcdd37fe700 0 client.29683 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 -- 172.21.15.12:0/2728232864 shutdown_connections 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 1 -- 172.21.15.12:0/2728232864 wait complete. 2024-05-09T00:02:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:25.995+0000 7fce06b61b80 20 client.29683 trim_cache size 0 max 16384 2024-05-09T00:02:26.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fce06b61b80 1 Processor -- start 2024-05-09T00:02:26.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fce06b61b80 1 -- start start 2024-05-09T00:02:26.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02f8890 0x555cc02f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fde00 0x555cc0304360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02fe720 0x555cc0312f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fce06b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc03092c0 con 0x555cc02f8890 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fce06b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc0317c70 con 0x555cc02fe720 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fce06b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc02efdc0 con 0x555cc02fde00 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fcdf3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02f8890 0x555cc02f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fcdeb7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fde00 0x555cc0304360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fcdf3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02f8890 0x555cc02f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56572/0 (socket says 172.21.15.12:56572) 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fcdf3fff700 1 -- 172.21.15.12:0/2059398817 learned_addr learned my addr 172.21.15.12:0/2059398817 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.012+0000 7fcdf8f44700 1 --2- 172.21.15.12:0/2059398817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02fe720 0x555cc0312f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.013+0000 7fcdf3fff700 1 -- 172.21.15.12:0/2059398817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fde00 msgr2=0x555cc0304360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.013+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/2059398817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fde00 0x555cc0304360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.013+0000 7fcdf3fff700 1 -- 172.21.15.12:0/2059398817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02fe720 msgr2=0x555cc0312f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.013+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/2059398817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02fe720 0x555cc0312f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.013+0000 7fcdf3fff700 1 -- 172.21.15.12:0/2059398817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cc020e560 con 0x555cc02f8890 2024-05-09T00:02:26.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.013+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/2059398817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02f8890 0x555cc02f2d20 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fcde40965c0 tx=0x7fcde40cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0e829fbd0d26ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:26.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.013+0000 7fcdeaffd700 1 -- 172.21.15.12:0/2059398817 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde409a250 con 0x555cc02f8890 2024-05-09T00:02:26.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.013+0000 7fcdeaffd700 1 -- 172.21.15.12:0/2059398817 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcde40b9040 con 0x555cc02f8890 2024-05-09T00:02:26.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.013+0000 7fcdeaffd700 1 -- 172.21.15.12:0/2059398817 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde400cd00 con 0x555cc02f8890 2024-05-09T00:02:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.014+0000 7fce06b61b80 1 -- 172.21.15.12:0/2059398817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02f8890 msgr2=0x555cc02f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.014+0000 7fce06b61b80 1 --2- 172.21.15.12:0/2059398817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02f8890 0x555cc02f2d20 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fcde40965c0 tx=0x7fcde40cc040 comp rx=0 tx=0).stop 2024-05-09T00:02:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.014+0000 7fce06b61b80 1 -- 172.21.15.12:0/2059398817 shutdown_connections 2024-05-09T00:02:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.014+0000 7fce06b61b80 1 --2- 172.21.15.12:0/2059398817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02fe720 0x555cc0312f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.014+0000 7fce06b61b80 1 --2- 172.21.15.12:0/2059398817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02f8890 0x555cc02f2d20 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.014+0000 7fce06b61b80 1 --2- 172.21.15.12:0/2059398817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02fde00 0x555cc0304360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.014+0000 7fce06b61b80 1 -- 172.21.15.12:0/2059398817 >> 172.21.15.12:0/2059398817 conn(0x555cc0256210 msgr2=0x555cc0301b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.014+0000 7fce06b61b80 1 -- 172.21.15.12:0/2059398817 shutdown_connections 2024-05-09T00:02:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.014+0000 7fce06b61b80 1 -- 172.21.15.12:0/2059398817 wait complete. 2024-05-09T00:02:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.015+0000 7fce06b61b80 1 -- start start 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.015+0000 7fce06b61b80 1 Processor -- start 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.015+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02f8890 0x555cc0303760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.015+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 0x555cc0303ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.015+0000 7fce06b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02fe720 0x555cc03041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.015+0000 7fce06b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc03138c0 con 0x555cc02fde00 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.015+0000 7fce06b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc03785a0 con 0x555cc02fe720 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.015+0000 7fce06b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc0378730 con 0x555cc02f8890 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fce06b61b80 10 client.0 _ll_register_callbacks cb 0x555cc01edb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdf3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02f8890 0x555cc0303760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdeb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 0x555cc0303ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdf3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02f8890 0x555cc0303760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40414/0 (socket says 172.21.15.12:40414) 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdf3fff700 1 -- 172.21.15.12:0/1754915409 learned_addr learned my addr 172.21.15.12:0/1754915409 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdeb7fe700 1 -- 172.21.15.12:0/1754915409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02f8890 msgr2=0x555cc0303760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdeb7fe700 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02f8890 0x555cc0303760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdeb7fe700 1 -- 172.21.15.12:0/1754915409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02fe720 msgr2=0x555cc03041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdeb7fe700 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02fe720 0x555cc03041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdeb7fe700 1 -- 172.21.15.12:0/1754915409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf41a2190 con 0x555cc02fde00 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdeb7fe700 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 0x555cc0303ca0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x555cc02f4bd0 tx=0x7fcddc00e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71ce4f33786154 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdd37fe700 1 -- 172.21.15.12:0/1754915409 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc02c030 con 0x555cc02fde00 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fcdd37fe700 1 -- 172.21.15.12:0/1754915409 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcddc004c70 con 0x555cc02fde00 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.016+0000 7fce06b61b80 1 -- 172.21.15.12:0/1754915409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcddc004e50 con 0x555cc02fde00 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fcdd37fe700 1 -- 172.21.15.12:0/1754915409 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcddc00ee20 con 0x555cc02fde00 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fce06b61b80 10 client.19953 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fce06b61b80 1 -- 172.21.15.12:0/1754915409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x555cc00cdcd0 con 0x555cc02fde00 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fcdd37fe700 1 -- 172.21.15.12:0/1754915409 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcddc004e50 con 0x555cc02fde00 2024-05-09T00:02:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fcdd37fe700 1 -- 172.21.15.12:0/1754915409 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fcddc024030 con 0x555cc02fde00 2024-05-09T00:02:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fce06b61b80 10 client.19953 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fce06b61b80 10 client.19953 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:02:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fce06b61b80 20 client.19953 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fce06b61b80 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555cc0379eb0 0x555cc037c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fce06b61b80 4 client.19953 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:02:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.017+0000 7fce06b61b80 1 -- 172.21.15.12:0/1754915409 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555cc0240ec0 con 0x555cc0379eb0 2024-05-09T00:02:26.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:25 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3796756023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:02:26.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:25 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3796756023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:02:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.019+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555cc0379eb0 0x555cc037c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.019+0000 7fcdd37fe700 10 client.19953 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:02:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.019+0000 7fcdf3fff700 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555cc0379eb0 0x555cc037c2d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=66947f0c23f18eef server_cookie=1286eca4e6192b5a in_seq=0 out_seq=0 2024-05-09T00:02:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.021+0000 7fcdd37fe700 1 -- 172.21.15.12:0/1754915409 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555cc0240ec0 con 0x555cc0379eb0 2024-05-09T00:02:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.021+0000 7fcdd37fe700 10 client.19953 handle_command_reply: tid=0 2024-05-09T00:02:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.021+0000 7fce06b61b80 1 client.19953 shutdown 2024-05-09T00:02:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 1 -- 172.21.15.12:0/1754915409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 msgr2=0x555cc0303ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 0x555cc0303ca0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x555cc02f4bd0 tx=0x7fcddc00e9f0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 1 -- 172.21.15.12:0/1754915409 shutdown_connections 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555cc0379eb0 0x555cc037c2d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cc02fe720 0x555cc03041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555cc02f8890 0x555cc0303760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 1 --2- 172.21.15.12:0/1754915409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc02fde00 0x555cc0303ca0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fcdd37fe700 0 client.19953 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 1 -- 172.21.15.12:0/1754915409 >> 172.21.15.12:0/1754915409 conn(0x555cc0256210 msgr2=0x555cc02fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 1 -- 172.21.15.12:0/1754915409 shutdown_connections 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 1 -- 172.21.15.12:0/1754915409 wait complete. 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.022+0000 7fce06b61b80 20 client.19953 trim_cache size 0 max 16384 2024-05-09T00:02:26.018 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:02:26.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/1407649227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcde00a12d0 msgr2=0x7fcde00a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcde00a12d0 0x7fcde00a3790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcdf41966c0 tx=0x7fcddc009260 comp rx=0 tx=0).stop 2024-05-09T00:02:26.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/1407649227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100060 msgr2=0x7fcdf4195170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100060 0x7fcdf4195170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c0c0 tx=0x7fcde400c3d0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/1407649227 shutdown_connections 2024-05-09T00:02:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcde00a12d0 0x7fcde00a3790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdf4199a50 0x7fcdf4199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdf4100060 0x7fcdf4195170 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 --2- 172.21.15.12:0/1407649227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdf4105180 0x7fcdf41956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/1407649227 >> 172.21.15.12:0/1407649227 conn(0x7fcdf4076fa0 msgr2=0x7fcdf40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/1407649227 shutdown_connections 2024-05-09T00:02:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.023+0000 7fcdfa9a7700 1 -- 172.21.15.12:0/1407649227 wait complete. 2024-05-09T00:02:26.034 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:02:26.231 INFO:tasks.workunit.client.0.smithi012.stdout: 10 104269 44.38 MB/sec execute 21 sec latency 52.905 ms 2024-05-09T00:02:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.253+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2127584513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b0100060 msgr2=0x7fc1b0100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.253+0000 7fc1b6d52700 1 --2- 172.21.15.12:0/2127584513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b0100060 0x7fc1b0100440 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009990 tx=0x7fc1a4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.253+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2127584513 shutdown_connections 2024-05-09T00:02:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.253+0000 7fc1b6d52700 1 --2- 172.21.15.12:0/2127584513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b0105180 0x7fc1b010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.253+0000 7fc1b6d52700 1 --2- 172.21.15.12:0/2127584513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b0100060 0x7fc1b0100440 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.253+0000 7fc1b6d52700 1 --2- 172.21.15.12:0/2127584513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1b0100a10 0x7fc1b0104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.253+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2127584513 >> 172.21.15.12:0/2127584513 conn(0x7fc1b0076fa0 msgr2=0x7fc1b00773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.253+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2127584513 shutdown_connections 2024-05-09T00:02:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.253+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2127584513 wait complete. 2024-05-09T00:02:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.254+0000 7fc1b6d52700 1 Processor -- start 2024-05-09T00:02:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.254+0000 7fc1b6d52700 1 -- start start 2024-05-09T00:02:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.254+0000 7fc1b6d52700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b0100060 0x7fc1b0101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.254+0000 7fc1b6d52700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1b0105180 0x7fc1b0102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.254+0000 7fc1b6d52700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b0102570 0x7fc1b01a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.254+0000 7fc1b6d52700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b01a5c50 con 0x7fc1b0102570 2024-05-09T00:02:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.254+0000 7fc1b6d52700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b01a5dc0 con 0x7fc1b0105180 2024-05-09T00:02:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.254+0000 7fc1b6d52700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b01a5f80 con 0x7fc1b0100060 2024-05-09T00:02:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.254+0000 7fc1b4aee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b0100060 0x7fc1b0101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.254+0000 7fc1affff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1b0105180 0x7fc1b0102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b4aee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b0100060 0x7fc1b0101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40430/0 (socket says 172.21.15.12:40430) 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b4aee700 1 -- 172.21.15.12:0/2380374022 learned_addr learned my addr 172.21.15.12:0/2380374022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b52ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b0102570 0x7fc1b01a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1affff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1b0105180 0x7fc1b0102030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45600/0 (socket says 172.21.15.12:45600) 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b4aee700 1 -- 172.21.15.12:0/2380374022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1b0105180 msgr2=0x7fc1b0102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b4aee700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1b0105180 0x7fc1b0102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b4aee700 1 -- 172.21.15.12:0/2380374022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b0102570 msgr2=0x7fc1b01a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b4aee700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b0102570 0x7fc1b01a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b4aee700 1 -- 172.21.15.12:0/2380374022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40095f0 con 0x7fc1b0100060 2024-05-09T00:02:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b52ef700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b0102570 0x7fc1b01a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:26.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b4aee700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b0100060 0x7fc1b0101af0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4005910 tx=0x7fc1a4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:26.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1adffb700 1 -- 172.21.15.12:0/2380374022 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a401d070 con 0x7fc1b0100060 2024-05-09T00:02:26.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1adffb700 1 -- 172.21.15.12:0/2380374022 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1a4022470 con 0x7fc1b0100060 2024-05-09T00:02:26.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2380374022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b01a6200 con 0x7fc1b0100060 2024-05-09T00:02:26.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.255+0000 7fc1adffb700 1 -- 172.21.15.12:0/2380374022 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4017650 con 0x7fc1b0100060 2024-05-09T00:02:26.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.256+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2380374022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b01a6620 con 0x7fc1b0100060 2024-05-09T00:02:26.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.259+0000 7fc1adffb700 1 -- 172.21.15.12:0/2380374022 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc1a40225e0 con 0x7fc1b0100060 2024-05-09T00:02:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.259+0000 7fc1adffb700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1a00a1290 0x7fc1a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.260+0000 7fc1adffb700 1 -- 172.21.15.12:0/2380374022 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc1a40cdf50 con 0x7fc1b0100060 2024-05-09T00:02:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.260+0000 7fc1affff700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1a00a1290 0x7fc1a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.260+0000 7fc1affff700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1a00a1290 0x7fc1a00a3750 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0103220 tx=0x7fc19c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.260+0000 7fc1c2f0cb80 1 Processor -- start 2024-05-09T00:02:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.260+0000 7fc1c2f0cb80 1 -- start start 2024-05-09T00:02:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 0x55b62a01b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01b720 0x55b62a015350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a015890 0x55b62a015cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b629f6c150 con 0x55b62a01b720 2024-05-09T00:02:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b629f69b10 con 0x55b62a015890 2024-05-09T00:02:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b629f65400 con 0x55b62a01ae00 2024-05-09T00:02:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1affff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01b720 0x55b62a015350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1affff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01b720 0x55b62a015350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56610/0 (socket says 172.21.15.12:56610) 2024-05-09T00:02:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1affff700 1 -- 172.21.15.12:0/1370375634 learned_addr learned my addr 172.21.15.12:0/1370375634 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1b4aee700 1 --2- 172.21.15.12:0/1370375634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 0x55b62a01b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1b52ef700 1 --2- 172.21.15.12:0/1370375634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a015890 0x55b62a015cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1b52ef700 1 -- 172.21.15.12:0/1370375634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 msgr2=0x55b62a01b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1b52ef700 1 --2- 172.21.15.12:0/1370375634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 0x55b62a01b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1b52ef700 1 -- 172.21.15.12:0/1370375634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01b720 msgr2=0x55b62a015350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1b52ef700 1 --2- 172.21.15.12:0/1370375634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01b720 0x55b62a015350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1b52ef700 1 -- 172.21.15.12:0/1370375634 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b629de63b0 con 0x55b62a015890 2024-05-09T00:02:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.261+0000 7fc1affff700 1 --2- 172.21.15.12:0/1370375634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01b720 0x55b62a015350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.262+0000 7fc1b52ef700 1 --2- 172.21.15.12:0/1370375634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a015890 0x55b62a015cd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8003db0 tx=0x7fc1a8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c736c0299a2c2554 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.262+0000 7fc19affd700 1 -- 172.21.15.12:0/1370375634 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8004bb0 con 0x55b62a015890 2024-05-09T00:02:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.262+0000 7fc19affd700 1 -- 172.21.15.12:0/1370375634 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1a800f040 con 0x55b62a015890 2024-05-09T00:02:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.262+0000 7fc19affd700 1 -- 172.21.15.12:0/1370375634 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8007c50 con 0x55b62a015890 2024-05-09T00:02:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.262+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/1370375634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a015890 msgr2=0x55b62a015cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.262+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/1370375634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a015890 0x55b62a015cd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8003db0 tx=0x7fc1a8009920 comp rx=0 tx=0).stop 2024-05-09T00:02:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.263+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/1370375634 shutdown_connections 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.263+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/1370375634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a015890 0x55b62a015cd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.263+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/1370375634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 0x55b62a01b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.263+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/1370375634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01b720 0x55b62a015350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.263+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/1370375634 >> 172.21.15.12:0/1370375634 conn(0x55b629f73210 msgr2=0x55b62a00fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.263+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/1370375634 shutdown_connections 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.263+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/1370375634 wait complete. 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.263+0000 7fc1c2f0cb80 1 -- start start 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.263+0000 7fc1c2f0cb80 1 Processor -- start 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 0x55b62a01ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a01efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a01f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b629f01560 con 0x55b62a01ae00 2024-05-09T00:02:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b629f01200 con 0x55b62a01b720 2024-05-09T00:02:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b629de86a0 con 0x55b62a015890 2024-05-09T00:02:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1c2f0cb80 10 client.0 _ll_register_callbacks cb 0x55b629f0ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b4aee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 0x55b62a01ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b4aee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 0x55b62a01ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40440/0 (socket says 172.21.15.12:40440) 2024-05-09T00:02:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b4aee700 1 -- 172.21.15.12:0/2555456085 learned_addr learned my addr 172.21.15.12:0/2555456085 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1affff700 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a01efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b52ef700 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a01f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b4aee700 1 -- 172.21.15.12:0/2555456085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 msgr2=0x55b62a01f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b4aee700 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a01f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b4aee700 1 -- 172.21.15.12:0/2555456085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 msgr2=0x55b62a01efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b4aee700 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a01efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b4aee700 1 -- 172.21.15.12:0/2555456085 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c017070 con 0x55b62a015890 2024-05-09T00:02:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b52ef700 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a01f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:02:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1affff700 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a01efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc1b4aee700 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 0x55b62a01ea70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc1a409ada0 tx=0x7fc1a4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753924161ffd2f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.264+0000 7fc18ffff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:02:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc18ffff700 1 -- 172.21.15.12:0/2555456085 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a409b7a0 con 0x55b62a015890 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2555456085 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b629ebe270 con 0x55b62a015890 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc18ffff700 1 -- 172.21.15.12:0/2555456085 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc1a409b0a0 con 0x55b62a015890 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc18ffff700 1 -- 172.21.15.12:0/2555456085 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a409eed0 con 0x55b62a015890 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc1c2f0cb80 10 client.29701 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2555456085 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x7fc1a8003a60 con 0x55b62a015890 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc18ffff700 1 -- 172.21.15.12:0/2555456085 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b629ebe270 con 0x55b62a015890 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc18ffff700 1 -- 172.21.15.12:0/2555456085 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc1a409a590 con 0x55b62a015890 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc1c2f0cb80 10 client.29701 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc1c2f0cb80 10 client.29701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc1c2f0cb80 20 client.29701 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b62a02eea0 0x55b62a0312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc1c2f0cb80 4 client.29701 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:26.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.265+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2555456085 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b629fe4810 con 0x55b62a02eea0 2024-05-09T00:02:26.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.266+0000 7fc1affff700 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b62a02eea0 0x55b62a0312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.266+0000 7fc18ffff700 10 client.29701 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:26.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.266+0000 7fc1affff700 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b62a02eea0 0x55b62a0312c0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=687ccdf39178d64f server_cookie=b08554c7c0ef9c12 in_seq=0 out_seq=0 2024-05-09T00:02:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.274+0000 7fc18ffff700 1 -- 172.21.15.12:0/2555456085 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b629fe4810 con 0x55b62a02eea0 2024-05-09T00:02:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.274+0000 7fc18ffff700 10 client.29701 handle_command_reply: tid=0 2024-05-09T00:02:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 client.29701 shutdown 2024-05-09T00:02:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2555456085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 msgr2=0x55b62a01ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 0x55b62a01ea70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc1a409ada0 tx=0x7fc1a4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2555456085 shutdown_connections 2024-05-09T00:02:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b62a02eea0 0x55b62a0312c0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a01f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 0x55b62a01ea70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc18ffff700 0 client.29701 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:26.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/2555456085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a01efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2555456085 >> 172.21.15.12:0/2555456085 conn(0x55b629f73210 msgr2=0x55b62a00f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:26.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2555456085 shutdown_connections 2024-05-09T00:02:26.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2555456085 wait complete. 2024-05-09T00:02:26.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.275+0000 7fc1c2f0cb80 20 client.29701 trim_cache size 0 max 16384 2024-05-09T00:02:26.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.291+0000 7fc1c2f0cb80 1 Processor -- start 2024-05-09T00:02:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.291+0000 7fc1c2f0cb80 1 -- start start 2024-05-09T00:02:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a015890 0x55b62a015350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 0x55b62a015c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a00fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62a011e50 con 0x55b62a015890 2024-05-09T00:02:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62a0364d0 con 0x55b62a01b720 2024-05-09T00:02:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62a00a800 con 0x55b62a01ae00 2024-05-09T00:02:26.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1affff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 0x55b62a015c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1b52ef700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a00fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1b52ef700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a00fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45630/0 (socket says 172.21.15.12:45630) 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1affff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 0x55b62a015c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40454/0 (socket says 172.21.15.12:40454) 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1b52ef700 1 -- 172.21.15.12:0/2421509838 learned_addr learned my addr 172.21.15.12:0/2421509838 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1affff700 1 -- 172.21.15.12:0/2421509838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 msgr2=0x55b62a00fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1affff700 1 --2- 172.21.15.12:0/2421509838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a00fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.292+0000 7fc1affff700 1 -- 172.21.15.12:0/2421509838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a015890 msgr2=0x55b62a015350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.293+0000 7fc1affff700 1 --2- 172.21.15.12:0/2421509838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a015890 0x55b62a015350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.293+0000 7fc1affff700 1 -- 172.21.15.12:0/2421509838 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b629d92b60 con 0x55b62a01ae00 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.293+0000 7fc1b52ef700 1 --2- 172.21.15.12:0/2421509838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a00fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.293+0000 7fc1affff700 1 --2- 172.21.15.12:0/2421509838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 0x55b62a015c70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55b62a01ffb0 tx=0x7fc19c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f70ed1deffb19704 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.293+0000 7fc19b7fe700 1 -- 172.21.15.12:0/2421509838 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c022030 con 0x55b62a01ae00 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.293+0000 7fc19b7fe700 1 -- 172.21.15.12:0/2421509838 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19c023040 con 0x55b62a01ae00 2024-05-09T00:02:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.293+0000 7fc19b7fe700 1 -- 172.21.15.12:0/2421509838 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c01d670 con 0x55b62a01ae00 2024-05-09T00:02:26.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.294+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2421509838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 msgr2=0x55b62a015c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.294+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/2421509838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 0x55b62a015c70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55b62a01ffb0 tx=0x7fc19c014000 comp rx=0 tx=0).stop 2024-05-09T00:02:26.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.294+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2421509838 shutdown_connections 2024-05-09T00:02:26.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.294+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/2421509838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a00fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.294+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/2421509838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a015890 0x55b62a015350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.294+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/2421509838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a01ae00 0x55b62a015c70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.294+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2421509838 >> 172.21.15.12:0/2421509838 conn(0x55b629f73210 msgr2=0x55b62a01d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:26.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.294+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2421509838 shutdown_connections 2024-05-09T00:02:26.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.294+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/2421509838 wait complete. 2024-05-09T00:02:26.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.295+0000 7fc1c2f0cb80 1 -- start start 2024-05-09T00:02:26.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.295+0000 7fc1c2f0cb80 1 Processor -- start 2024-05-09T00:02:26.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.295+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 0x55b62a080a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.295+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a026170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1c2f0cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a0266b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62a07d350 con 0x55b62a01ae00 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62a07d4c0 con 0x55b62a01b720 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1c2f0cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62a07d680 con 0x55b62a015890 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1c2f0cb80 10 client.0 _ll_register_callbacks cb 0x55b629f0ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1affff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a026170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1affff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a026170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56624/0 (socket says 172.21.15.12:56624) 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1b52ef700 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a0266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1affff700 1 -- 172.21.15.12:0/923368695 learned_addr learned my addr 172.21.15.12:0/923368695 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1b4aee700 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 0x55b62a080a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1affff700 1 -- 172.21.15.12:0/923368695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 msgr2=0x55b62a080a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1affff700 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 0x55b62a080a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1affff700 1 -- 172.21.15.12:0/923368695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 msgr2=0x55b62a0266b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1affff700 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a0266b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.296+0000 7fc1affff700 1 -- 172.21.15.12:0/923368695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b629de63b0 con 0x55b62a01ae00 2024-05-09T00:02:26.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.297+0000 7fc1affff700 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a026170 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x55b62a031bd0 tx=0x7fc19c0090c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c4086152f76648 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:26.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.297+0000 7fc18ffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:26.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.297+0000 7fc18ffff700 1 -- 172.21.15.12:0/923368695 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c022050 con 0x55b62a01ae00 2024-05-09T00:02:26.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.297+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/923368695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1a800f040 con 0x55b62a01ae00 2024-05-09T00:02:26.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.297+0000 7fc18ffff700 1 -- 172.21.15.12:0/923368695 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc19c003f40 con 0x55b62a01ae00 2024-05-09T00:02:26.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.297+0000 7fc18ffff700 1 -- 172.21.15.12:0/923368695 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19c02b0b0 con 0x55b62a01ae00 2024-05-09T00:02:26.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.297+0000 7fc1c2f0cb80 10 client.19968 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.297+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/923368695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x7fc19c004360 con 0x55b62a01ae00 2024-05-09T00:02:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.297+0000 7fc18ffff700 1 -- 172.21.15.12:0/923368695 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19c040020 con 0x55b62a01ae00 2024-05-09T00:02:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.298+0000 7fc18ffff700 1 -- 172.21.15.12:0/923368695 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc19c032070 con 0x55b62a01ae00 2024-05-09T00:02:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.298+0000 7fc1c2f0cb80 10 client.19968 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.298+0000 7fc1c2f0cb80 10 client.19968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.298+0000 7fc1c2f0cb80 20 client.19968 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.298+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b62a092d90 0x55b62a0951b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.298+0000 7fc1c2f0cb80 4 client.19968 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:02:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.298+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/923368695 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b629f5dec0 con 0x55b62a092d90 2024-05-09T00:02:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.298+0000 7fc1b4aee700 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b62a092d90 0x55b62a0951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.298+0000 7fc18ffff700 10 client.19968 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.299+0000 7fc1b4aee700 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b62a092d90 0x55b62a0951b0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf9b69568a016bf server_cookie=99333d7f014a2c66 in_seq=0 out_seq=0 2024-05-09T00:02:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc18ffff700 1 -- 172.21.15.12:0/923368695 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b629f5dec0 con 0x55b62a092d90 2024-05-09T00:02:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc18ffff700 10 client.19968 handle_command_reply: tid=0 2024-05-09T00:02:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc1c2f0cb80 1 client.19968 shutdown 2024-05-09T00:02:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/923368695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 msgr2=0x55b62a026170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a026170 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x55b62a031bd0 tx=0x7fc19c0090c0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/923368695 shutdown_connections 2024-05-09T00:02:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b62a092d90 0x55b62a0951b0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b62a01b720 0x55b62a0266b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b62a015890 0x55b62a080a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc18ffff700 0 client.19968 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc1c2f0cb80 1 --2- 172.21.15.12:0/923368695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b62a01ae00 0x55b62a026170 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/923368695 >> 172.21.15.12:0/923368695 conn(0x55b629f73210 msgr2=0x55b62a019b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.300+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/923368695 shutdown_connections 2024-05-09T00:02:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.301+0000 7fc1c2f0cb80 1 -- 172.21.15.12:0/923368695 wait complete. 2024-05-09T00:02:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.301+0000 7fc1c2f0cb80 20 client.19968 trim_cache size 0 max 16384 2024-05-09T00:02:26.299 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:02:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2380374022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1a00a1290 msgr2=0x7fc1a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1a00a1290 0x7fc1a00a3750 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0103220 tx=0x7fc19c009260 comp rx=0 tx=0).stop 2024-05-09T00:02:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2380374022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b0100060 msgr2=0x7fc1b0101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b0100060 0x7fc1b0101af0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4005910 tx=0x7fc1a4004730 comp rx=0 tx=0).stop 2024-05-09T00:02:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2380374022 shutdown_connections 2024-05-09T00:02:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc1a00a1290 0x7fc1a00a3750 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b0102570 0x7fc1b01a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b0100060 0x7fc1b0101af0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 --2- 172.21.15.12:0/2380374022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1b0105180 0x7fc1b0102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2380374022 >> 172.21.15.12:0/2380374022 conn(0x7fc1b0076fa0 msgr2=0x7fc1b0069930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:26.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2380374022 shutdown_connections 2024-05-09T00:02:26.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:26.302+0000 7fc1b6d52700 1 -- 172.21.15.12:0/2380374022 wait complete. 2024-05-09T00:02:26.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:26 smithi012 ceph-mon[42503]: pgmap v514: 97 pgs: 97 active+clean; 893 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.2 MiB/s wr, 607 op/s 2024-05-09T00:02:27.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:26 smithi160 ceph-mon[46152]: pgmap v514: 97 pgs: 97 active+clean; 893 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.2 MiB/s wr, 607 op/s 2024-05-09T00:02:27.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:26 smithi204 ceph-mon[47373]: pgmap v514: 97 pgs: 97 active+clean; 893 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.2 MiB/s wr, 607 op/s 2024-05-09T00:02:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.081+0000 7f72f0b03700 1 -- 172.21.15.12:0/4095358647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72ec10c4b0 msgr2=0x7f72ec10c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.081+0000 7f72f0b03700 1 --2- 172.21.15.12:0/4095358647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72ec10c4b0 0x7f72ec10c890 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f72e0007570 tx=0x7f72e0007880 comp rx=0 tx=0).stop 2024-05-09T00:02:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.084+0000 7f72f0b03700 1 -- 172.21.15.12:0/4095358647 shutdown_connections 2024-05-09T00:02:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.084+0000 7f72f0b03700 1 --2- 172.21.15.12:0/4095358647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec108960 0x7f72ec079200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.084+0000 7f72f0b03700 1 --2- 172.21.15.12:0/4095358647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72ec10c4b0 0x7f72ec10c890 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.084+0000 7f72f0b03700 1 --2- 172.21.15.12:0/4095358647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec107fa0 0x7f72ec108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.084+0000 7f72f0b03700 1 -- 172.21.15.12:0/4095358647 >> 172.21.15.12:0/4095358647 conn(0x7f72ec06bf80 msgr2=0x7f72ec06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.084+0000 7f72f0b03700 1 -- 172.21.15.12:0/4095358647 shutdown_connections 2024-05-09T00:02:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.084+0000 7f72f0b03700 1 -- 172.21.15.12:0/4095358647 wait complete. 2024-05-09T00:02:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.085+0000 7f72f0b03700 1 Processor -- start 2024-05-09T00:02:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.085+0000 7f72f0b03700 1 -- start start 2024-05-09T00:02:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.085+0000 7f72f0b03700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec107fa0 0x7f72ec0839d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.085+0000 7f72f0b03700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec108960 0x7f72ec07da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.085+0000 7f72f0b03700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72ec10c4b0 0x7f72ec07dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.085+0000 7f72f0b03700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec07e6e0 con 0x7f72ec10c4b0 2024-05-09T00:02:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.085+0000 7f72f0b03700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec07e850 con 0x7f72ec108960 2024-05-09T00:02:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.085+0000 7f72f0b03700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1bb0e0 con 0x7f72ec107fa0 2024-05-09T00:02:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.086+0000 7f72ea59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec107fa0 0x7f72ec0839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.086+0000 7f72ea59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec107fa0 0x7f72ec0839d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40472/0 (socket says 172.21.15.12:40472) 2024-05-09T00:02:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.086+0000 7f72ea59c700 1 -- 172.21.15.12:0/3658956482 learned_addr learned my addr 172.21.15.12:0/3658956482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.086+0000 7f72ea59c700 1 -- 172.21.15.12:0/3658956482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec108960 msgr2=0x7f72ec07da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.086+0000 7f72ea59c700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec108960 0x7f72ec07da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.086+0000 7f72ea59c700 1 -- 172.21.15.12:0/3658956482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72ec10c4b0 msgr2=0x7f72ec07dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.086+0000 7f72ea59c700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72ec10c4b0 0x7f72ec07dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.086+0000 7f72ea59c700 1 -- 172.21.15.12:0/3658956482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e0007220 con 0x7f72ec107fa0 2024-05-09T00:02:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.087+0000 7f72ea59c700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec107fa0 0x7f72ec0839d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f72e0005950 tx=0x7f72e0004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.089+0000 7f72db7fe700 1 -- 172.21.15.12:0/3658956482 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0021070 con 0x7f72ec107fa0 2024-05-09T00:02:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.090+0000 7f72db7fe700 1 -- 172.21.15.12:0/3658956482 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e0007e60 con 0x7f72ec107fa0 2024-05-09T00:02:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.090+0000 7f72db7fe700 1 -- 172.21.15.12:0/3658956482 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e000f070 con 0x7f72ec107fa0 2024-05-09T00:02:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.091+0000 7f72f0b03700 1 -- 172.21.15.12:0/3658956482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec1bb360 con 0x7f72ec107fa0 2024-05-09T00:02:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.091+0000 7f72f0b03700 1 -- 172.21.15.12:0/3658956482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec1bb790 con 0x7f72ec107fa0 2024-05-09T00:02:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.094+0000 7f72db7fe700 1 -- 172.21.15.12:0/3658956482 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f72e000a2a0 con 0x7f72ec107fa0 2024-05-09T00:02:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.096+0000 7f72db7fe700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72d40a13d0 0x7f72d40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.096+0000 7f72db7fe700 1 -- 172.21.15.12:0/3658956482 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f72e00ccb00 con 0x7f72ec107fa0 2024-05-09T00:02:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.097+0000 7f72fccbdb80 1 Processor -- start 2024-05-09T00:02:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.099+0000 7f72fccbdb80 1 -- start start 2024-05-09T00:02:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.099+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4759950 0x5602e4759d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.099+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e475a270 0x5602e4753f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.099+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e4754480 0x5602e47548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.099+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e46ab0f0 con 0x5602e475a270 2024-05-09T00:02:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.099+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e46a8ab0 con 0x5602e4759950 2024-05-09T00:02:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.099+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e46a43a0 con 0x5602e4754480 2024-05-09T00:02:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.100+0000 7f72ea59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4759950 0x5602e4759d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.100+0000 7f72ea59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4759950 0x5602e4759d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45666/0 (socket says 172.21.15.12:45666) 2024-05-09T00:02:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.100+0000 7f72ea59c700 1 -- 172.21.15.12:0/2227303221 learned_addr learned my addr 172.21.15.12:0/2227303221 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.101+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72d40a13d0 0x7f72d40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.101+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/2227303221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e475a270 0x5602e4753f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.102+0000 7f72e9d9b700 1 -- 172.21.15.12:0/2227303221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e4754480 msgr2=0x5602e47548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.102+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/2227303221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e4754480 0x5602e47548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.102+0000 7f72e9d9b700 1 -- 172.21.15.12:0/2227303221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4759950 msgr2=0x5602e4759d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.102+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/2227303221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4759950 0x5602e4759d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.102+0000 7f72e9d9b700 1 -- 172.21.15.12:0/2227303221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602e4525350 con 0x5602e475a270 2024-05-09T00:02:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.102+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/2227303221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e475a270 0x5602e4753f40 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f72ec07f1a0 tx=0x7f72dc0165c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7236665c5a1ca9a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.103+0000 7f72d8ff9700 1 -- 172.21.15.12:0/2227303221 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0099c0 con 0x5602e475a270 2024-05-09T00:02:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.103+0000 7f72d8ff9700 1 -- 172.21.15.12:0/2227303221 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72dc00f070 con 0x5602e475a270 2024-05-09T00:02:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.103+0000 7f72d8ff9700 1 -- 172.21.15.12:0/2227303221 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc007500 con 0x5602e475a270 2024-05-09T00:02:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.103+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72d40a13d0 0x7f72d40a3890 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f72ec10b9a0 tx=0x7f72dc00f000 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.109+0000 7f72fccbdb80 1 -- 172.21.15.12:0/2227303221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e475a270 msgr2=0x5602e4753f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.109+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/2227303221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e475a270 0x5602e4753f40 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f72ec07f1a0 tx=0x7f72dc0165c0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.109+0000 7f72fccbdb80 1 -- 172.21.15.12:0/2227303221 shutdown_connections 2024-05-09T00:02:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.109+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/2227303221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e4754480 0x5602e47548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.109+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/2227303221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4759950 0x5602e4759d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.109+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/2227303221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e475a270 0x5602e4753f40 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.109+0000 7f72fccbdb80 1 -- 172.21.15.12:0/2227303221 >> 172.21.15.12:0/2227303221 conn(0x5602e46b21b0 msgr2=0x5602e474e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.109+0000 7f72fccbdb80 1 -- 172.21.15.12:0/2227303221 shutdown_connections 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.109+0000 7f72fccbdb80 1 -- 172.21.15.12:0/2227303221 wait complete. 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.110+0000 7f72fccbdb80 1 -- start start 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.110+0000 7f72fccbdb80 1 Processor -- start 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.110+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4754480 0x5602e4760210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.110+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e475d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.110+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e475a270 0x5602e475dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.110+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e4640500 con 0x5602e4759950 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.110+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e46401a0 con 0x5602e4754480 2024-05-09T00:02:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.110+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e4527640 con 0x5602e475a270 2024-05-09T00:02:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.110+0000 7f72fccbdb80 10 client.0 _ll_register_callbacks cb 0x5602e4649ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.111+0000 7f72e9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e475d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.111+0000 7f72e9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e475d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56660/0 (socket says 172.21.15.12:56660) 2024-05-09T00:02:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.111+0000 7f72e9d9b700 1 -- 172.21.15.12:0/1423619173 learned_addr learned my addr 172.21.15.12:0/1423619173 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.111+0000 7f72e9d9b700 1 -- 172.21.15.12:0/1423619173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e475a270 msgr2=0x5602e475dc90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:02:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.111+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e475a270 0x5602e475dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.111+0000 7f72e9d9b700 1 -- 172.21.15.12:0/1423619173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4754480 msgr2=0x5602e4760210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.111+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4754480 0x5602e4760210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.111+0000 7f72e9d9b700 1 -- 172.21.15.12:0/1423619173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc012af0 con 0x5602e4759950 2024-05-09T00:02:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.112+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e475d550 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f72dc004b10 tx=0x7f72dc018f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616d447713a3b11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.112+0000 7f72d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.112+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1423619173 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc018cf0 con 0x5602e4759950 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.112+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1423619173 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f72dc004bb0 con 0x5602e4759950 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.112+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1423619173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602e45fd210 con 0x5602e4759950 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.112+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1423619173 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72dc0188b0 con 0x5602e4759950 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.112+0000 7f72fccbdb80 10 client.19977 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.112+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1423619173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x5602e44d1b00 con 0x5602e4759950 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.113+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1423619173 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602e45fd210 con 0x5602e4759950 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.113+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1423619173 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f72dc033050 con 0x5602e4759950 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.113+0000 7f72fccbdb80 10 client.19977 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.113+0000 7f72fccbdb80 10 client.19977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.113+0000 7f72fccbdb80 20 client.19977 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.113+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5602e476dbd0 0x5602e476fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.113+0000 7f72fccbdb80 4 client.19977 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.113+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1423619173 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602e47237b0 con 0x5602e476dbd0 2024-05-09T00:02:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.120+0000 7f72ea59c700 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5602e476dbd0 0x5602e476fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.151+0000 7f72ea59c700 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5602e476dbd0 0x5602e476fff0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70b86d39f535fca2 server_cookie=ae96bfae1bac3292 in_seq=0 out_seq=0 2024-05-09T00:02:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.153+0000 7f72d1ffb700 10 client.19977 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.162+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1423619173 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602e47237b0 con 0x5602e476dbd0 2024-05-09T00:02:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.162+0000 7f72d1ffb700 10 client.19977 handle_command_reply: tid=0 2024-05-09T00:02:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.162+0000 7f72fccbdb80 1 client.19977 shutdown 2024-05-09T00:02:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.162+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1423619173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 msgr2=0x5602e475d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.162+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e475d550 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f72dc004b10 tx=0x7f72dc018f60 comp rx=0 tx=0).stop 2024-05-09T00:02:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.163+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1423619173 shutdown_connections 2024-05-09T00:02:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.163+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e475a270 0x5602e475dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.163+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5602e476dbd0 0x5602e476fff0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.163+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4754480 0x5602e4760210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.163+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1423619173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e475d550 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.163+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1423619173 >> 172.21.15.12:0/1423619173 conn(0x5602e46b21b0 msgr2=0x5602e4752f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.163+0000 7f72d1ffb700 0 client.19977 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.163+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1423619173 shutdown_connections 2024-05-09T00:02:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.163+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1423619173 wait complete. 2024-05-09T00:02:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.165+0000 7f72fccbdb80 20 client.19977 trim_cache size 0 max 16384 2024-05-09T00:02:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.203+0000 7f72fccbdb80 1 Processor -- start 2024-05-09T00:02:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72fccbdb80 1 -- start start 2024-05-09T00:02:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4754480 0x5602e474e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e4769690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e475a270 0x5602e4774950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e47542e0 con 0x5602e4759950 2024-05-09T00:02:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e4746ba0 con 0x5602e4754480 2024-05-09T00:02:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e4750290 con 0x5602e475a270 2024-05-09T00:02:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72ead9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e475a270 0x5602e4774950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72ead9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e475a270 0x5602e4774950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40490/0 (socket says 172.21.15.12:40490) 2024-05-09T00:02:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72ead9d700 1 -- 172.21.15.12:0/2556857888 learned_addr learned my addr 172.21.15.12:0/2556857888 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.205+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/2556857888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e4769690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.207+0000 7f72e9d9b700 1 -- 172.21.15.12:0/2556857888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e475a270 msgr2=0x5602e4774950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.207+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/2556857888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e475a270 0x5602e4774950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.207+0000 7f72e9d9b700 1 -- 172.21.15.12:0/2556857888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4754480 msgr2=0x5602e474e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.207+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/2556857888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4754480 0x5602e474e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.207+0000 7f72e9d9b700 1 -- 172.21.15.12:0/2556857888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602e466a500 con 0x5602e4759950 2024-05-09T00:02:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.207+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/2556857888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e4769690 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009690 tx=0x7f72dc0165c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda07a238ee6dc2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.208+0000 7f72d97fa700 1 -- 172.21.15.12:0/2556857888 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc012940 con 0x5602e4759950 2024-05-09T00:02:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.208+0000 7f72d97fa700 1 -- 172.21.15.12:0/2556857888 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72dc0188b0 con 0x5602e4759950 2024-05-09T00:02:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.208+0000 7f72d97fa700 1 -- 172.21.15.12:0/2556857888 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc007960 con 0x5602e4759950 2024-05-09T00:02:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.210+0000 7f72fccbdb80 1 -- 172.21.15.12:0/2556857888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 msgr2=0x5602e4769690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.210+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/2556857888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e4769690 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009690 tx=0x7f72dc0165c0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.210+0000 7f72fccbdb80 1 -- 172.21.15.12:0/2556857888 shutdown_connections 2024-05-09T00:02:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.210+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/2556857888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e475a270 0x5602e4774950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.210+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/2556857888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e4754480 0x5602e474e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.210+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/2556857888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4759950 0x5602e4769690 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.210+0000 7f72fccbdb80 1 -- 172.21.15.12:0/2556857888 >> 172.21.15.12:0/2556857888 conn(0x5602e46b21b0 msgr2=0x5602e47504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.212+0000 7f72fccbdb80 1 -- 172.21.15.12:0/2556857888 shutdown_connections 2024-05-09T00:02:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.212+0000 7f72fccbdb80 1 -- 172.21.15.12:0/2556857888 wait complete. 2024-05-09T00:02:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.212+0000 7f72fccbdb80 1 -- start start 2024-05-09T00:02:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.213+0000 7f72fccbdb80 1 Processor -- start 2024-05-09T00:02:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.213+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4754480 0x5602e4764dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.214+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e4759950 0x5602e4762bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.214+0000 7f72fccbdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e475a270 0x5602e4763100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.214+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e47636d0 con 0x5602e4754480 2024-05-09T00:02:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.214+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e4763840 con 0x5602e475a270 2024-05-09T00:02:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.214+0000 7f72fccbdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e4763a00 con 0x5602e4759950 2024-05-09T00:02:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.214+0000 7f72fccbdb80 10 client.0 _ll_register_callbacks cb 0x5602e4649ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.214+0000 7f72ea59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4754480 0x5602e4764dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.214+0000 7f72ea59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4754480 0x5602e4764dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56690/0 (socket says 172.21.15.12:56690) 2024-05-09T00:02:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.214+0000 7f72ea59c700 1 -- 172.21.15.12:0/1627667070 learned_addr learned my addr 172.21.15.12:0/1627667070 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.216+0000 7f72ea59c700 1 -- 172.21.15.12:0/1627667070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e4759950 msgr2=0x5602e4762bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.216+0000 7f72ea59c700 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e4759950 0x5602e4762bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.216+0000 7f72ea59c700 1 -- 172.21.15.12:0/1627667070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e475a270 msgr2=0x5602e4763100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.216+0000 7f72ea59c700 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e475a270 0x5602e4763100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.216+0000 7f72ea59c700 1 -- 172.21.15.12:0/1627667070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602e44d1b00 con 0x5602e4754480 2024-05-09T00:02:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.216+0000 7f72ea59c700 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4754480 0x5602e4764dd0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f72e0099950 tx=0x7f72e00ca1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23343d77e88e9273 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.216+0000 7f72d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.217+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1627667070 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e00b1070 con 0x5602e4754480 2024-05-09T00:02:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.217+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1627667070 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f72e0099c50 con 0x5602e4754480 2024-05-09T00:02:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.217+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1627667070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72dc00f070 con 0x5602e4754480 2024-05-09T00:02:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.217+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1627667070 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72e009da80 con 0x5602e4754480 2024-05-09T00:02:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.217+0000 7f72fccbdb80 10 client.19983 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.217+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1627667070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x5602e4529c70 con 0x5602e4754480 2024-05-09T00:02:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.219+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1627667070 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72dc00f070 con 0x5602e4754480 2024-05-09T00:02:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.219+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1627667070 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f72e0099fa0 con 0x5602e4754480 2024-05-09T00:02:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.219+0000 7f72fccbdb80 10 client.19983 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.219+0000 7f72fccbdb80 10 client.19983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.219+0000 7f72fccbdb80 20 client.19983 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.219+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5602e47c44e0 0x5602e47c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.219+0000 7f72fccbdb80 4 client.19983 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:02:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.219+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1627667070 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5602e469ce60 con 0x5602e47c44e0 2024-05-09T00:02:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.220+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5602e47c44e0 0x5602e47c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.233+0000 7f72e9d9b700 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5602e47c44e0 0x5602e47c6900 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd6daf07624944c2 server_cookie=81ab7c393c256407 in_seq=0 out_seq=0 2024-05-09T00:02:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:27.233+0000 7f72d1ffb700 10 client.19983 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:27.231 INFO:tasks.workunit.client.0.smithi012.stdout: 10 104363 42.72 MB/sec execute 22 sec latency 689.625 ms 2024-05-09T00:02:27.346 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:02:27.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:02:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.514+0000 7f45106bc700 1 -- 172.21.15.12:0/9222132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4508100060 msgr2=0x7f4508100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.514+0000 7f45106bc700 1 --2- 172.21.15.12:0/9222132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4508100060 0x7f4508100440 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f4504009990 tx=0x7f4504009ca0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.514+0000 7f45106bc700 1 -- 172.21.15.12:0/9222132 shutdown_connections 2024-05-09T00:02:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.515+0000 7f45106bc700 1 --2- 172.21.15.12:0/9222132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4508105160 0x7f450810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.515+0000 7f45106bc700 1 --2- 172.21.15.12:0/9222132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4508100060 0x7f4508100440 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.515+0000 7f45106bc700 1 --2- 172.21.15.12:0/9222132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4508100a10 0x7f4508104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.515+0000 7f45106bc700 1 -- 172.21.15.12:0/9222132 >> 172.21.15.12:0/9222132 conn(0x7f4508076f40 msgr2=0x7f4508077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.515+0000 7f45106bc700 1 -- 172.21.15.12:0/9222132 shutdown_connections 2024-05-09T00:02:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.515+0000 7f45106bc700 1 -- 172.21.15.12:0/9222132 wait complete. 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.515+0000 7f45106bc700 1 Processor -- start 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f45106bc700 1 -- start start 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f45106bc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4508100060 0x7f4508198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f45106bc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4508105160 0x7f4508198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f45106bc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4508198c90 0x7f45081a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f45106bc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45081a3970 con 0x7f4508100060 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f45106bc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45081a3ae0 con 0x7f4508105160 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f45106bc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45081a3ca0 con 0x7f4508198c90 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450e458700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4508100060 0x7f4508198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450e458700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4508100060 0x7f4508198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56712/0 (socket says 172.21.15.12:56712) 2024-05-09T00:02:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450e458700 1 -- 172.21.15.12:0/2669662585 learned_addr learned my addr 172.21.15.12:0/2669662585 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450ec59700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4508198c90 0x7f45081a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450dc57700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4508105160 0x7f4508198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450e458700 1 -- 172.21.15.12:0/2669662585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4508198c90 msgr2=0x7f45081a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450e458700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4508198c90 0x7f45081a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450e458700 1 -- 172.21.15.12:0/2669662585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4508105160 msgr2=0x7f4508198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450e458700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4508105160 0x7f4508198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450e458700 1 -- 172.21.15.12:0/2669662585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040095f0 con 0x7f4508100060 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.516+0000 7f450e458700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4508100060 0x7f4508198210 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4504004760 tx=0x7f4504004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.517+0000 7f44ff7fe700 1 -- 172.21.15.12:0/2669662585 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450401d070 con 0x7f4508100060 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.517+0000 7f44ff7fe700 1 -- 172.21.15.12:0/2669662585 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4504022470 con 0x7f4508100060 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.517+0000 7f45106bc700 1 -- 172.21.15.12:0/2669662585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45081a3f20 con 0x7f4508100060 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.517+0000 7f44ff7fe700 1 -- 172.21.15.12:0/2669662585 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504017650 con 0x7f4508100060 2024-05-09T00:02:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.517+0000 7f45106bc700 1 -- 172.21.15.12:0/2669662585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45081a4340 con 0x7f4508100060 2024-05-09T00:02:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.520+0000 7f44ff7fe700 1 -- 172.21.15.12:0/2669662585 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f45040225e0 con 0x7f4508100060 2024-05-09T00:02:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.520+0000 7f44ff7fe700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44f40a1310 0x7f44f40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.521+0000 7f44ff7fe700 1 -- 172.21.15.12:0/2669662585 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f45040cdd60 con 0x7f4508100060 2024-05-09T00:02:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.521+0000 7f450dc57700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44f40a1310 0x7f44f40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.521+0000 7f450dc57700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44f40a1310 0x7f44f40a37d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4508068800 tx=0x7f44f8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.522+0000 7f451c876b80 1 Processor -- start 2024-05-09T00:02:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.522+0000 7f451c876b80 1 -- start start 2024-05-09T00:02:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.522+0000 7f451c876b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8290 0x560e08ba8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.522+0000 7f451c876b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8bb0 0x560e08ba27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.522+0000 7f451c876b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba2d20 0x560e08ba3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.522+0000 7f451c876b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08af51d0 con 0x560e08ba8bb0 2024-05-09T00:02:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.522+0000 7f451c876b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08af2b90 con 0x560e08ba2d20 2024-05-09T00:02:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.522+0000 7f451c876b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08aee480 con 0x560e08ba8290 2024-05-09T00:02:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.522+0000 7f450dc57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8bb0 0x560e08ba27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450ec59700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba2d20 0x560e08ba3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450dc57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8bb0 0x560e08ba27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56728/0 (socket says 172.21.15.12:56728) 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450e458700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8290 0x560e08ba8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450dc57700 1 -- 172.21.15.12:0/2329177092 learned_addr learned my addr 172.21.15.12:0/2329177092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450ec59700 1 --2- 172.21.15.12:0/2329177092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba2d20 0x560e08ba3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:45700/0 (socket says 172.21.15.12:45700) 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450e458700 1 -- 172.21.15.12:0/2329177092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba2d20 msgr2=0x560e08ba3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450e458700 1 --2- 172.21.15.12:0/2329177092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba2d20 0x560e08ba3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450e458700 1 -- 172.21.15.12:0/2329177092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8bb0 msgr2=0x560e08ba27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450e458700 1 --2- 172.21.15.12:0/2329177092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8bb0 0x560e08ba27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450e458700 1 -- 172.21.15.12:0/2329177092 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e0896f430 con 0x560e08ba8290 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450dc57700 1 --2- 172.21.15.12:0/2329177092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8bb0 0x560e08ba27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f450e458700 1 --2- 172.21.15.12:0/2329177092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8290 0x560e08ba8670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f45040cb4c0 tx=0x7f450409ad60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a825e8d9011f1365 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f44fcff9700 1 -- 172.21.15.12:0/2329177092 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45040a1070 con 0x560e08ba8290 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f44fcff9700 1 -- 172.21.15.12:0/2329177092 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f450409b610 con 0x560e08ba8290 2024-05-09T00:02:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.523+0000 7f44fcff9700 1 -- 172.21.15.12:0/2329177092 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450409b3d0 con 0x560e08ba8290 2024-05-09T00:02:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.524+0000 7f451c876b80 1 -- 172.21.15.12:0/2329177092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8290 msgr2=0x560e08ba8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.524+0000 7f451c876b80 1 --2- 172.21.15.12:0/2329177092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8290 0x560e08ba8670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f45040cb4c0 tx=0x7f450409ad60 comp rx=0 tx=0).stop 2024-05-09T00:02:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.524+0000 7f451c876b80 1 -- 172.21.15.12:0/2329177092 shutdown_connections 2024-05-09T00:02:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.524+0000 7f451c876b80 1 --2- 172.21.15.12:0/2329177092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba2d20 0x560e08ba3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.524+0000 7f451c876b80 1 --2- 172.21.15.12:0/2329177092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8290 0x560e08ba8670 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.524+0000 7f451c876b80 1 --2- 172.21.15.12:0/2329177092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8bb0 0x560e08ba27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.524+0000 7f451c876b80 1 -- 172.21.15.12:0/2329177092 >> 172.21.15.12:0/2329177092 conn(0x560e08afc290 msgr2=0x560e08b98e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.524+0000 7f451c876b80 1 -- 172.21.15.12:0/2329177092 shutdown_connections 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.524+0000 7f451c876b80 1 -- 172.21.15.12:0/2329177092 wait complete. 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f451c876b80 1 -- start start 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f451c876b80 1 Processor -- start 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f451c876b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba2d20 0x560e08babf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f451c876b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8290 0x560e08bac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f451c876b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8bb0 0x560e08bac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f451c876b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08a8a5e0 con 0x560e08ba2d20 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f451c876b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08a8a280 con 0x560e08ba8290 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f451c876b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08971720 con 0x560e08ba8bb0 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f451c876b80 10 client.0 _ll_register_callbacks cb 0x560e08a93bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f450e458700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba2d20 0x560e08babf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f450e458700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba2d20 0x560e08babf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56738/0 (socket says 172.21.15.12:56738) 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f450e458700 1 -- 172.21.15.12:0/2158452935 learned_addr learned my addr 172.21.15.12:0/2158452935 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f450dc57700 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8290 0x560e08bac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f450ec59700 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8bb0 0x560e08bac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f450e458700 1 -- 172.21.15.12:0/2158452935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8bb0 msgr2=0x560e08bac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f450e458700 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8bb0 0x560e08bac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f450e458700 1 -- 172.21.15.12:0/2158452935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8290 msgr2=0x560e08bac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f450e458700 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8290 0x560e08bac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.525+0000 7f450e458700 1 -- 172.21.15.12:0/2158452935 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f8017070 con 0x560e08ba2d20 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f450e458700 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba2d20 0x560e08babf50 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f45040cba40 tx=0x7f4504022b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe96ac1a31c9df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f44f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f44f1ffb700 1 -- 172.21.15.12:0/2158452935 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450409af10 con 0x560e08ba2d20 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f44f1ffb700 1 -- 172.21.15.12:0/2158452935 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f450409ed40 con 0x560e08ba2d20 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f451c876b80 1 -- 172.21.15.12:0/2158452935 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e08a472f0 con 0x560e08ba2d20 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f44f1ffb700 1 -- 172.21.15.12:0/2158452935 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45040a3300 con 0x560e08ba2d20 2024-05-09T00:02:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f451c876b80 10 client.19995 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f451c876b80 1 -- 172.21.15.12:0/2158452935 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x7f45081a4340 con 0x560e08ba2d20 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f44f1ffb700 1 -- 172.21.15.12:0/2158452935 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e08a472f0 con 0x560e08ba2d20 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f44f1ffb700 1 -- 172.21.15.12:0/2158452935 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f4504096d00 con 0x560e08ba2d20 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f451c876b80 10 client.19995 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f451c876b80 10 client.19995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.526+0000 7f451c876b80 20 client.19995 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.527+0000 7f451c876b80 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560e08bbbf80 0x560e08bbe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.527+0000 7f451c876b80 4 client.19995 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.527+0000 7f451c876b80 1 -- 172.21.15.12:0/2158452935 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e08b6d890 con 0x560e08bbbf80 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.527+0000 7f450dc57700 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560e08bbbf80 0x560e08bbe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.527+0000 7f450dc57700 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560e08bbbf80 0x560e08bbe3a0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752542fa6e0d4c51 server_cookie=6a57dfc8b4ebfec8 in_seq=0 out_seq=0 2024-05-09T00:02:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:27.527+0000 7f44f1ffb700 10 client.19995 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:27.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:27 smithi012 ceph-mon[42503]: pgmap v515: 97 pgs: 97 active+clean; 1018 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-05-09T00:02:28.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:27 smithi204 ceph-mon[47373]: pgmap v515: 97 pgs: 97 active+clean; 1018 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-05-09T00:02:28.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:27 smithi160 ceph-mon[46152]: pgmap v515: 97 pgs: 97 active+clean; 1018 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-05-09T00:02:28.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.166+0000 7f44f1ffb700 1 -- 172.21.15.12:0/2158452935 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e08b6d890 con 0x560e08bbbf80 2024-05-09T00:02:28.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.166+0000 7f44f1ffb700 10 client.19995 handle_command_reply: tid=0 2024-05-09T00:02:28.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.166+0000 7f451c876b80 1 client.19995 shutdown 2024-05-09T00:02:28.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.166+0000 7f451c876b80 1 -- 172.21.15.12:0/2158452935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba2d20 msgr2=0x560e08babf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.166+0000 7f451c876b80 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba2d20 0x560e08babf50 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f45040cba40 tx=0x7f4504022b10 comp rx=0 tx=0).stop 2024-05-09T00:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.167+0000 7f72d1ffb700 1 -- 172.21.15.12:0/1627667070 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5602e469ce60 con 0x5602e47c44e0 2024-05-09T00:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.167+0000 7f72d1ffb700 10 client.19983 handle_command_reply: tid=0 2024-05-09T00:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.167+0000 7f72fccbdb80 1 client.19983 shutdown 2024-05-09T00:02:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.168+0000 7f451c876b80 1 -- 172.21.15.12:0/2158452935 shutdown_connections 2024-05-09T00:02:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.168+0000 7f451c876b80 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba8bb0 0x560e08bac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.168+0000 7f451c876b80 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba2d20 0x560e08babf50 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.168+0000 7f451c876b80 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560e08bbbf80 0x560e08bbe3a0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.168+0000 7f451c876b80 1 --2- 172.21.15.12:0/2158452935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8290 0x560e08bac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.168+0000 7f451c876b80 1 -- 172.21.15.12:0/2158452935 >> 172.21.15.12:0/2158452935 conn(0x560e08afc290 msgr2=0x560e08b98620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.168+0000 7f44f1ffb700 0 client.19995 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.168+0000 7f451c876b80 1 -- 172.21.15.12:0/2158452935 shutdown_connections 2024-05-09T00:02:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.168+0000 7f451c876b80 1 -- 172.21.15.12:0/2158452935 wait complete. 2024-05-09T00:02:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.168+0000 7f451c876b80 20 client.19995 trim_cache size 0 max 16384 2024-05-09T00:02:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.169+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1627667070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4754480 msgr2=0x5602e4764dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.169+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4754480 0x5602e4764dd0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f72e0099950 tx=0x7f72e00ca1d0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.170+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1627667070 shutdown_connections 2024-05-09T00:02:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.170+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5602e47c44e0 0x5602e47c6900 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.170+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5602e475a270 0x5602e4763100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.170+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602e4754480 0x5602e4764dd0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.170+0000 7f72fccbdb80 1 --2- 172.21.15.12:0/1627667070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5602e4759950 0x5602e4762bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.170+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1627667070 >> 172.21.15.12:0/1627667070 conn(0x5602e46b21b0 msgr2=0x5602e4774950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.177+0000 7f72d1ffb700 0 client.19983 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.177+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1627667070 shutdown_connections 2024-05-09T00:02:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.177+0000 7f72fccbdb80 1 -- 172.21.15.12:0/1627667070 wait complete. 2024-05-09T00:02:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.177+0000 7f72fccbdb80 20 client.19983 trim_cache size 0 max 16384 2024-05-09T00:02:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:02:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:02:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:02:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:02:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.177+0000 7f72f0b03700 1 -- 172.21.15.12:0/3658956482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72d40a13d0 msgr2=0x7f72d40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.177+0000 7f72f0b03700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72d40a13d0 0x7f72d40a3890 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f72ec10b9a0 tx=0x7f72dc00f000 comp rx=0 tx=0).stop 2024-05-09T00:02:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.178+0000 7f72f0b03700 1 -- 172.21.15.12:0/3658956482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec107fa0 msgr2=0x7f72ec0839d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.178+0000 7f72f0b03700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec107fa0 0x7f72ec0839d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f72e0005950 tx=0x7f72e0004a30 comp rx=0 tx=0).stop 2024-05-09T00:02:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.178+0000 7f72f0b03700 1 -- 172.21.15.12:0/3658956482 shutdown_connections 2024-05-09T00:02:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.178+0000 7f72f0b03700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f72d40a13d0 0x7f72d40a3890 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f72ec10b9a0 tx=0x7f72dc00f000 comp rx=0 tx=0).stop 2024-05-09T00:02:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.178+0000 7f72f0b03700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72ec10c4b0 0x7f72ec07dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.178+0000 7f72f0b03700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72ec107fa0 0x7f72ec0839d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.178+0000 7f72f0b03700 1 --2- 172.21.15.12:0/3658956482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec108960 0x7f72ec07da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.178+0000 7f72f0b03700 1 -- 172.21.15.12:0/3658956482 >> 172.21.15.12:0/3658956482 conn(0x7f72ec06bf80 msgr2=0x7f72ec109bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.178+0000 7f72f0b03700 1 -- 172.21.15.12:0/3658956482 shutdown_connections 2024-05-09T00:02:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:28.178+0000 7f72f0b03700 1 -- 172.21.15.12:0/3658956482 wait complete. 2024-05-09T00:02:28.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.201+0000 7f451c876b80 1 Processor -- start 2024-05-09T00:02:28.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.201+0000 7f451c876b80 1 -- start start 2024-05-09T00:02:28.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f451c876b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08ba78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f451c876b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 0x560e08b98da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f451c876b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8bb0 0x560e08baaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f451c876b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08bad3e0 con 0x560e08ba8290 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f451c876b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08bad5c0 con 0x560e08ba8bb0 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f451c876b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08b9aee0 con 0x560e08ba2d20 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f450dc57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 0x560e08b98da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f450dc57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 0x560e08b98da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56750/0 (socket says 172.21.15.12:56750) 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f450dc57700 1 -- 172.21.15.12:0/2419514316 learned_addr learned my addr 172.21.15.12:0/2419514316 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f450e458700 1 --2- 172.21.15.12:0/2419514316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08ba78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f450e458700 1 -- 172.21.15.12:0/2419514316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8bb0 msgr2=0x560e08baaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f450e458700 1 --2- 172.21.15.12:0/2419514316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8bb0 0x560e08baaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f450e458700 1 -- 172.21.15.12:0/2419514316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 msgr2=0x560e08b98da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f450e458700 1 --2- 172.21.15.12:0/2419514316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 0x560e08b98da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.202+0000 7f450e458700 1 -- 172.21.15.12:0/2419514316 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e08b95560 con 0x560e08ba2d20 2024-05-09T00:02:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.203+0000 7f450e458700 1 --2- 172.21.15.12:0/2419514316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08ba78b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4504097980 tx=0x7f450409ad60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d38cdd67bc07a18d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.203+0000 7f44fd7fa700 1 -- 172.21.15.12:0/2419514316 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504099e70 con 0x560e08ba2d20 2024-05-09T00:02:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.203+0000 7f44fd7fa700 1 -- 172.21.15.12:0/2419514316 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f450409ed40 con 0x560e08ba2d20 2024-05-09T00:02:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.203+0000 7f44fd7fa700 1 -- 172.21.15.12:0/2419514316 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450409af10 con 0x560e08ba2d20 2024-05-09T00:02:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.204+0000 7f451c876b80 1 -- 172.21.15.12:0/2419514316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 msgr2=0x560e08ba78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.204+0000 7f451c876b80 1 --2- 172.21.15.12:0/2419514316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08ba78b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4504097980 tx=0x7f450409ad60 comp rx=0 tx=0).stop 2024-05-09T00:02:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.204+0000 7f451c876b80 1 -- 172.21.15.12:0/2419514316 shutdown_connections 2024-05-09T00:02:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.204+0000 7f451c876b80 1 --2- 172.21.15.12:0/2419514316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8bb0 0x560e08baaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.204+0000 7f451c876b80 1 --2- 172.21.15.12:0/2419514316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08ba78b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.204+0000 7f451c876b80 1 --2- 172.21.15.12:0/2419514316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 0x560e08b98da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.204+0000 7f451c876b80 1 -- 172.21.15.12:0/2419514316 >> 172.21.15.12:0/2419514316 conn(0x560e08afc290 msgr2=0x560e08baa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.207+0000 7f451c876b80 1 -- 172.21.15.12:0/2419514316 shutdown_connections 2024-05-09T00:02:28.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.207+0000 7f451c876b80 1 -- 172.21.15.12:0/2419514316 wait complete. 2024-05-09T00:02:28.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.207+0000 7f451c876b80 1 -- start start 2024-05-09T00:02:28.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.207+0000 7f451c876b80 1 Processor -- start 2024-05-09T00:02:28.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.208+0000 7f451c876b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08bb3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.208+0000 7f451c876b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 0x560e08ba4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.208+0000 7f451c876b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8bb0 0x560e08ba48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.208+0000 7f451c876b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08ba2400 con 0x560e08ba8290 2024-05-09T00:02:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.208+0000 7f451c876b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08ba2540 con 0x560e08ba8bb0 2024-05-09T00:02:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.208+0000 7f451c876b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e08ba2680 con 0x560e08ba2d20 2024-05-09T00:02:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.208+0000 7f451c876b80 10 client.0 _ll_register_callbacks cb 0x560e08a93bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.208+0000 7f450e458700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08bb3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.208+0000 7f450e458700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08bb3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40534/0 (socket says 172.21.15.12:40534) 2024-05-09T00:02:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.208+0000 7f450e458700 1 -- 172.21.15.12:0/1542324751 learned_addr learned my addr 172.21.15.12:0/1542324751 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.209+0000 7f450ec59700 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8bb0 0x560e08ba48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.209+0000 7f450dc57700 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 0x560e08ba4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.209+0000 7f450e458700 1 -- 172.21.15.12:0/1542324751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8bb0 msgr2=0x560e08ba48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.209+0000 7f450e458700 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8bb0 0x560e08ba48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.209+0000 7f450e458700 1 -- 172.21.15.12:0/1542324751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 msgr2=0x560e08ba4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.209+0000 7f450e458700 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 0x560e08ba4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.209+0000 7f450e458700 1 -- 172.21.15.12:0/1542324751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45081a4340 con 0x560e08ba2d20 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.209+0000 7f450e458700 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08bb3550 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f45040a2dd0 tx=0x7f4504097190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857bf3c4839458c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.209+0000 7f44f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.210+0000 7f44f1ffb700 1 -- 172.21.15.12:0/1542324751 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450409aa50 con 0x560e08ba2d20 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.210+0000 7f44f1ffb700 1 -- 172.21.15.12:0/1542324751 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f450409af10 con 0x560e08ba2d20 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.210+0000 7f451c876b80 1 -- 172.21.15.12:0/1542324751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f450409b610 con 0x560e08ba2d20 2024-05-09T00:02:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.210+0000 7f44f1ffb700 1 -- 172.21.15.12:0/1542324751 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45040a3300 con 0x560e08ba2d20 2024-05-09T00:02:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.210+0000 7f451c876b80 10 client.29734 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.210+0000 7f451c876b80 1 -- 172.21.15.12:0/1542324751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x560e0891bbe0 con 0x560e08ba2d20 2024-05-09T00:02:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.210+0000 7f44f1ffb700 1 -- 172.21.15.12:0/1542324751 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f450409b610 con 0x560e08ba2d20 2024-05-09T00:02:28.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.212+0000 7f44f1ffb700 1 -- 172.21.15.12:0/1542324751 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f4504099e70 con 0x560e08ba2d20 2024-05-09T00:02:28.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.212+0000 7f451c876b80 10 client.29734 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:28.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.212+0000 7f451c876b80 10 client.29734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:28.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.212+0000 7f451c876b80 20 client.29734 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:28.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.212+0000 7f451c876b80 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560e08c0b030 0x560e08c0d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.212+0000 7f451c876b80 4 client.29734 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:02:28.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.212+0000 7f451c876b80 1 -- 172.21.15.12:0/1542324751 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560e08ae6f40 con 0x560e08c0b030 2024-05-09T00:02:28.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.214+0000 7f450dc57700 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560e08c0b030 0x560e08c0d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.215+0000 7f450dc57700 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560e08c0b030 0x560e08c0d450 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97bc34442b28907d server_cookie=830f24b8222d172a in_seq=0 out_seq=0 2024-05-09T00:02:28.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.215+0000 7f44f1ffb700 10 client.29734 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.218+0000 7f44f1ffb700 1 -- 172.21.15.12:0/1542324751 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560e08ae6f40 con 0x560e08c0b030 2024-05-09T00:02:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.218+0000 7f44f1ffb700 10 client.29734 handle_command_reply: tid=0 2024-05-09T00:02:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.218+0000 7f451c876b80 1 client.29734 shutdown 2024-05-09T00:02:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.218+0000 7f451c876b80 1 -- 172.21.15.12:0/1542324751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 msgr2=0x560e08bb3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.218+0000 7f451c876b80 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08bb3550 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f45040a2dd0 tx=0x7f4504097190 comp rx=0 tx=0).stop 2024-05-09T00:02:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.219+0000 7f451c876b80 1 -- 172.21.15.12:0/1542324751 shutdown_connections 2024-05-09T00:02:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.219+0000 7f451c876b80 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560e08c0b030 0x560e08c0d450 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.219+0000 7f451c876b80 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e08ba8bb0 0x560e08ba48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.219+0000 7f451c876b80 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e08ba2d20 0x560e08bb3550 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.219+0000 7f451c876b80 1 --2- 172.21.15.12:0/1542324751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e08ba8290 0x560e08ba4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.219+0000 7f451c876b80 1 -- 172.21.15.12:0/1542324751 >> 172.21.15.12:0/1542324751 conn(0x560e08afc290 msgr2=0x560e08ba7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.219+0000 7f44f1ffb700 0 client.29734 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.220+0000 7f451c876b80 1 -- 172.21.15.12:0/1542324751 shutdown_connections 2024-05-09T00:02:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.220+0000 7f451c876b80 1 -- 172.21.15.12:0/1542324751 wait complete. 2024-05-09T00:02:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.220+0000 7f451c876b80 20 client.29734 trim_cache size 0 max 16384 2024-05-09T00:02:28.215 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:02:28.215 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:02:28.215 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "12118ea9-4c38-40cf-8ac4-6f088b899bb2", 2024-05-09T00:02:28.215 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:02:28.215 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:02:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.221+0000 7f45106bc700 1 -- 172.21.15.12:0/2669662585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44f40a1310 msgr2=0x7f44f40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.221+0000 7f45106bc700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44f40a1310 0x7f44f40a37d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4508068800 tx=0x7f44f8009260 comp rx=0 tx=0).stop 2024-05-09T00:02:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.221+0000 7f45106bc700 1 -- 172.21.15.12:0/2669662585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4508100060 msgr2=0x7f4508198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.221+0000 7f45106bc700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4508100060 0x7f4508198210 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4504004760 tx=0x7f4504004790 comp rx=0 tx=0).stop 2024-05-09T00:02:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.221+0000 7f45106bc700 1 -- 172.21.15.12:0/2669662585 shutdown_connections 2024-05-09T00:02:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.221+0000 7f45106bc700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f44f40a1310 0x7f44f40a37d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.221+0000 7f45106bc700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4508198c90 0x7f45081a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.221+0000 7f45106bc700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4508100060 0x7f4508198210 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.221+0000 7f45106bc700 1 --2- 172.21.15.12:0/2669662585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4508105160 0x7f4508198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.221+0000 7f45106bc700 1 -- 172.21.15.12:0/2669662585 >> 172.21.15.12:0/2669662585 conn(0x7f4508076f40 msgr2=0x7f45080fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.226+0000 7f45106bc700 1 -- 172.21.15.12:0/2669662585 shutdown_connections 2024-05-09T00:02:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.226+0000 7f45106bc700 1 -- 172.21.15.12:0/2669662585 wait complete. 2024-05-09T00:02:28.231 INFO:tasks.workunit.client.0.smithi012.stdout: 10 104595 41.60 MB/sec execute 23 sec latency 755.275 ms 2024-05-09T00:02:28.245 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:02:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.478+0000 7fe5cf019700 1 -- 172.21.15.12:0/2910058426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c810d810 msgr2=0x7fe5c810fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.478+0000 7fe5cf019700 1 --2- 172.21.15.12:0/2910058426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c810d810 0x7fe5c810fc40 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000b0a0 tx=0x7fe5c000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.479+0000 7fe5cf019700 1 -- 172.21.15.12:0/2910058426 shutdown_connections 2024-05-09T00:02:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.479+0000 7fe5cf019700 1 --2- 172.21.15.12:0/2910058426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c810d810 0x7fe5c810fc40 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.479+0000 7fe5cf019700 1 --2- 172.21.15.12:0/2910058426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8073370 0x7fe5c8073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.479+0000 7fe5cf019700 1 --2- 172.21.15.12:0/2910058426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c8073c90 0x7fe5c810d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.479+0000 7fe5cf019700 1 -- 172.21.15.12:0/2910058426 >> 172.21.15.12:0/2910058426 conn(0x7fe5c80fca80 msgr2=0x7fe5c80feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.479+0000 7fe5cf019700 1 -- 172.21.15.12:0/2910058426 shutdown_connections 2024-05-09T00:02:28.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.479+0000 7fe5cf019700 1 -- 172.21.15.12:0/2910058426 wait complete. 2024-05-09T00:02:28.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.480+0000 7fe5cf019700 1 Processor -- start 2024-05-09T00:02:28.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.480+0000 7fe5cf019700 1 -- start start 2024-05-09T00:02:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.480+0000 7fe5cf019700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8073370 0x7fe5c810d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.480+0000 7fe5cf019700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c8073c90 0x7fe5c810a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.480+0000 7fe5cf019700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c810d810 0x7fe5c810a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.480+0000 7fe5cf019700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c810b0e0 con 0x7fe5c810d810 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.480+0000 7fe5cf019700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c81a9f90 con 0x7fe5c8073c90 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.480+0000 7fe5cf019700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c81aa0d0 con 0x7fe5c8073370 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.481+0000 7fe5cd5b6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c810d810 0x7fe5c810a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.481+0000 7fe5cd5b6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c810d810 0x7fe5c810a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41036/0 (socket says 172.21.15.12:41036) 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.481+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/2255865699 learned_addr learned my addr 172.21.15.12:0/2255865699 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.481+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8073370 0x7fe5c810d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.481+0000 7fe5c7fff700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c8073c90 0x7fe5c810a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.481+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/2255865699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8073370 msgr2=0x7fe5c810d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.481+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8073370 0x7fe5c810d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.481+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/2255865699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c8073c90 msgr2=0x7fe5c810a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.481+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c8073c90 0x7fe5c810a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.481+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/2255865699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c0009d00 con 0x7fe5c810d810 2024-05-09T00:02:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.482+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c810d810 0x7fe5c810a910 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000b070 tx=0x7fe5c00079e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.482+0000 7fe5c5ffb700 1 -- 172.21.15.12:0/2255865699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000e040 con 0x7fe5c810d810 2024-05-09T00:02:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.482+0000 7fe5c5ffb700 1 -- 172.21.15.12:0/2255865699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5c000be80 con 0x7fe5c810d810 2024-05-09T00:02:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.482+0000 7fe5c5ffb700 1 -- 172.21.15.12:0/2255865699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00052a0 con 0x7fe5c810d810 2024-05-09T00:02:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.482+0000 7fe5cf019700 1 -- 172.21.15.12:0/2255865699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c81aa2d0 con 0x7fe5c810d810 2024-05-09T00:02:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.483+0000 7fe5cf019700 1 -- 172.21.15.12:0/2255865699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c81aa790 con 0x7fe5c810d810 2024-05-09T00:02:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.488+0000 7fe5db1d3b80 1 Processor -- start 2024-05-09T00:02:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.495+0000 7fe5db1d3b80 1 -- start start 2024-05-09T00:02:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.495+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 0x55b9e67c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.495+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.495+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.495+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e6715150 con 0x55b9e67c3e00 2024-05-09T00:02:28.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.495+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e6712b10 con 0x55b9e67be890 2024-05-09T00:02:28.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.495+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e670e400 con 0x55b9e67c4720 2024-05-09T00:02:28.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5ccdb5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5ccdb5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67be350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34490/0 (socket says 172.21.15.12:34490) 2024-05-09T00:02:28.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5ccdb5700 1 -- 172.21.15.12:0/3600363295 learned_addr learned my addr 172.21.15.12:0/3600363295 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:28.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5c7fff700 1 --2- 172.21.15.12:0/3600363295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 0x55b9e67c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5c7fff700 1 --2- 172.21.15.12:0/3600363295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5c5ffb700 1 -- 172.21.15.12:0/2255865699 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe5c001d470 con 0x7fe5c810d810 2024-05-09T00:02:28.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5ccdb5700 1 -- 172.21.15.12:0/3600363295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 msgr2=0x55b9e67becd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/3600363295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67becd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5ccdb5700 1 -- 172.21.15.12:0/3600363295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 msgr2=0x55b9e67c41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/3600363295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 0x55b9e67c41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.496+0000 7fe5ccdb5700 1 -- 172.21.15.12:0/3600363295 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9e658f3b0 con 0x55b9e67c4720 2024-05-09T00:02:28.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.497+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/3600363295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67be350 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009cb0 tx=0x7fe5bc0113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2167f6c3db8c2da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:28.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.497+0000 7fe5c5ffb700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe5b80a1310 0x7fe5b80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.497+0000 7fe5c5ffb700 1 -- 172.21.15.12:0/2255865699 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe5c00cdc40 con 0x7fe5c810d810 2024-05-09T00:02:28.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.497+0000 7fe5c7fff700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe5b80a1310 0x7fe5b80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.498+0000 7fe5c7fff700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe5b80a1310 0x7fe5b80a37d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400d010 tx=0x7fe5b4016040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:28.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.498+0000 7fe5b2ffd700 1 -- 172.21.15.12:0/3600363295 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc011c60 con 0x55b9e67c4720 2024-05-09T00:02:28.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.498+0000 7fe5b2ffd700 1 -- 172.21.15.12:0/3600363295 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5bc011dc0 con 0x55b9e67c4720 2024-05-09T00:02:28.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.498+0000 7fe5b2ffd700 1 -- 172.21.15.12:0/3600363295 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc00f730 con 0x55b9e67c4720 2024-05-09T00:02:28.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.499+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3600363295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 msgr2=0x55b9e67be350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.499+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/3600363295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67be350 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009cb0 tx=0x7fe5bc0113e0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.502+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3600363295 shutdown_connections 2024-05-09T00:02:28.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.502+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/3600363295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67becd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.502+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/3600363295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 0x55b9e67c41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.502+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/3600363295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67be350 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.502+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3600363295 >> 172.21.15.12:0/3600363295 conn(0x55b9e671c210 msgr2=0x55b9e67b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.502+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3600363295 shutdown_connections 2024-05-09T00:02:28.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.502+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3600363295 wait complete. 2024-05-09T00:02:28.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.502+0000 7fe5db1d3b80 1 -- start start 2024-05-09T00:02:28.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5db1d3b80 1 Processor -- start 2024-05-09T00:02:28.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 0x55b9e67c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e66aa560 con 0x55b9e67c3e00 2024-05-09T00:02:28.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e66aa200 con 0x55b9e67be890 2024-05-09T00:02:28.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e65916a0 con 0x55b9e67c4720 2024-05-09T00:02:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5db1d3b80 10 client.0 _ll_register_callbacks cb 0x55b9e66b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5cd5b6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5cd5b6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34500/0 (socket says 172.21.15.12:34500) 2024-05-09T00:02:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.503+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/3253921296 learned_addr learned my addr 172.21.15.12:0/3253921296 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.504+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.504+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/3253921296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 msgr2=0x55b9e67c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.504+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.504+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/3253921296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 msgr2=0x55b9e67c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.504+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 0x55b9e67c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.504+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/3253921296 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5bc009910 con 0x55b9e67c4720 2024-05-09T00:02:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.504+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8610 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00972c0 tx=0x7fe5c009a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95728c92d13b20d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.504+0000 7fe5a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:02:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.504+0000 7fe5a7fff700 1 -- 172.21.15.12:0/3253921296 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c009b220 con 0x55b9e67c4720 2024-05-09T00:02:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.504+0000 7fe5a7fff700 1 -- 172.21.15.12:0/3253921296 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe5c009ebf0 con 0x55b9e67c4720 2024-05-09T00:02:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.505+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3253921296 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9e6667270 con 0x55b9e67c4720 2024-05-09T00:02:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.505+0000 7fe5a7fff700 1 -- 172.21.15.12:0/3253921296 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5c00a31b0 con 0x55b9e67c4720 2024-05-09T00:02:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.505+0000 7fe5db1d3b80 10 client.29749 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.505+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3253921296 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x55b9e653bb60 con 0x55b9e67c4720 2024-05-09T00:02:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.507+0000 7fe5a7fff700 1 -- 172.21.15.12:0/3253921296 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9e6667270 con 0x55b9e67c4720 2024-05-09T00:02:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.508+0000 7fe5a7fff700 1 -- 172.21.15.12:0/3253921296 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe5c00b8f70 con 0x55b9e67c4720 2024-05-09T00:02:28.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.508+0000 7fe5db1d3b80 10 client.29749 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:28.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.508+0000 7fe5db1d3b80 10 client.29749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:28.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.508+0000 7fe5db1d3b80 20 client.29749 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:28.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.508+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9e67d7fe0 0x55b9e67da400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.508+0000 7fe5db1d3b80 4 client.29749 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:28.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.508+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3253921296 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9e678d810 con 0x55b9e67d7fe0 2024-05-09T00:02:28.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.509+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9e67d7fe0 0x55b9e67da400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.511+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9e67d7fe0 0x55b9e67da400 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecf3fa5d57a7fcb8 server_cookie=7994091a6574f61d in_seq=0 out_seq=0 2024-05-09T00:02:28.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.511+0000 7fe5a7fff700 10 client.29749 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:28.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5a7fff700 1 -- 172.21.15.12:0/3253921296 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9e678d810 con 0x55b9e67d7fe0 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5a7fff700 10 client.29749 handle_command_reply: tid=0 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 client.29749 shutdown 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3253921296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 msgr2=0x55b9e67c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8610 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00972c0 tx=0x7fe5c009a7a0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3253921296 shutdown_connections 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8610 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9e67d7fe0 0x55b9e67da400 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/3253921296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 0x55b9e67c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3253921296 >> 172.21.15.12:0/3253921296 conn(0x55b9e671c210 msgr2=0x55b9e67b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5a7fff700 0 client.29749 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3253921296 shutdown_connections 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/3253921296 wait complete. 2024-05-09T00:02:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.516+0000 7fe5db1d3b80 20 client.29749 trim_cache size 0 max 16384 2024-05-09T00:02:28.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.554+0000 7fe5db1d3b80 1 Processor -- start 2024-05-09T00:02:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.554+0000 7fe5db1d3b80 1 -- start start 2024-05-09T00:02:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.554+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 0x55b9e67b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e67c9190 con 0x55b9e67c3e00 2024-05-09T00:02:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e67c1870 con 0x55b9e67be890 2024-05-09T00:02:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e67d40d0 con 0x55b9e67c4720 2024-05-09T00:02:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5cd5b6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5cd5b6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:34502/0 (socket says 172.21.15.12:34502) 2024-05-09T00:02:28.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/1466485602 learned_addr learned my addr 172.21.15.12:0/1466485602 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:28.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/1466485602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/1466485602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 msgr2=0x55b9e67be350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/1466485602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67be350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/1466485602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 msgr2=0x55b9e67b8d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:02:28.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/1466485602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 0x55b9e67b8d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.555+0000 7fe5cd5b6700 1 -- 172.21.15.12:0/1466485602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9e67bef40 con 0x55b9e67c4720 2024-05-09T00:02:28.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.556+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/1466485602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00a3120 tx=0x7fe5c0099370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb8f69311a4b8a64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:28.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.557+0000 7fe5b37fe700 1 -- 172.21.15.12:0/1466485602 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0079ea0 con 0x55b9e67c4720 2024-05-09T00:02:28.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.557+0000 7fe5b37fe700 1 -- 172.21.15.12:0/1466485602 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5c009b4c0 con 0x55b9e67c4720 2024-05-09T00:02:28.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.557+0000 7fe5b37fe700 1 -- 172.21.15.12:0/1466485602 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c009a2b0 con 0x55b9e67c4720 2024-05-09T00:02:28.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.558+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/1466485602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 msgr2=0x55b9e67c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.558+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/1466485602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00a3120 tx=0x7fe5c0099370 comp rx=0 tx=0).stop 2024-05-09T00:02:28.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.558+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/1466485602 shutdown_connections 2024-05-09T00:02:28.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.558+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/1466485602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67c8430 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.558+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/1466485602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.558+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/1466485602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67c3e00 0x55b9e67b8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.558+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/1466485602 >> 172.21.15.12:0/1466485602 conn(0x55b9e671c210 msgr2=0x55b9e67c6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.559+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/1466485602 shutdown_connections 2024-05-09T00:02:28.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.570+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/1466485602 wait complete. 2024-05-09T00:02:28.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.570+0000 7fe5db1d3b80 1 -- start start 2024-05-09T00:02:28.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.570+0000 7fe5db1d3b80 1 Processor -- start 2024-05-09T00:02:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5db1d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67cdbd0 0x55b9e682d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e682da50 con 0x55b9e67cdbd0 2024-05-09T00:02:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e682dbc0 con 0x55b9e67be890 2024-05-09T00:02:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5db1d3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e682dd80 con 0x55b9e67c4720 2024-05-09T00:02:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5db1d3b80 10 client.0 _ll_register_callbacks cb 0x55b9e66b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5ccdb5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5ccdb5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:38120/0 (socket says 172.21.15.12:38120) 2024-05-09T00:02:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5ccdb5700 1 -- 172.21.15.12:0/606399174 learned_addr learned my addr 172.21.15.12:0/606399174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.571+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67cdbd0 0x55b9e682d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.572+0000 7fe5c7fff700 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.572+0000 7fe5ccdb5700 1 -- 172.21.15.12:0/606399174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 msgr2=0x55b9e67cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.572+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.572+0000 7fe5ccdb5700 1 -- 172.21.15.12:0/606399174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67cdbd0 msgr2=0x55b9e682d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.572+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67cdbd0 0x55b9e682d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.572+0000 7fe5ccdb5700 1 -- 172.21.15.12:0/606399174 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c00b2050 con 0x55b9e67be890 2024-05-09T00:02:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.572+0000 7fe5cd5b6700 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67cdbd0 0x55b9e682d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.572+0000 7fe5ccdb5700 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67cd150 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc000c00 tx=0x7fe5bc0113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fe8ad07c9cf8be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.572+0000 7fe5a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:02:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.573+0000 7fe5a7fff700 1 -- 172.21.15.12:0/606399174 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0050b0 con 0x55b9e67be890 2024-05-09T00:02:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.573+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/606399174 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9e6706ec0 con 0x55b9e67be890 2024-05-09T00:02:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.573+0000 7fe5a7fff700 1 -- 172.21.15.12:0/606399174 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe5bc005210 con 0x55b9e67be890 2024-05-09T00:02:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.573+0000 7fe5a7fff700 1 -- 172.21.15.12:0/606399174 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5bc0053d0 con 0x55b9e67be890 2024-05-09T00:02:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.573+0000 7fe5db1d3b80 10 client.29663 fetch_fsmap learned FSMap version 64 2024-05-09T00:02:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.573+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/606399174 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x55b9e66d4560 con 0x55b9e67be890 2024-05-09T00:02:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.574+0000 7fe5a7fff700 1 -- 172.21.15.12:0/606399174 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9e6706ec0 con 0x55b9e67be890 2024-05-09T00:02:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.575+0000 7fe5a7fff700 1 -- 172.21.15.12:0/606399174 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 64) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe5bc003680 con 0x55b9e67be890 2024-05-09T00:02:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.575+0000 7fe5db1d3b80 10 client.29663 fetch_fsmap finished waiting for FSMap version 64 2024-05-09T00:02:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.575+0000 7fe5db1d3b80 10 client.29663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.575+0000 7fe5db1d3b80 20 client.29663 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.575+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9e682f4c0 0x55b9e68318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.575+0000 7fe5db1d3b80 4 client.29663 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:02:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.575+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/606399174 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b9e655ccf0 con 0x55b9e682f4c0 2024-05-09T00:02:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.575+0000 7fe5c7fff700 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9e682f4c0 0x55b9e68318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:28.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.576+0000 7fe5c7fff700 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9e682f4c0 0x55b9e68318e0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb89bbae30e79b7 server_cookie=83033986056084aa in_seq=0 out_seq=0 2024-05-09T00:02:28.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.579+0000 7fe5a7fff700 10 client.29663 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:28.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.579+0000 7fe5a7fff700 1 -- 172.21.15.12:0/606399174 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55b9e655ccf0 con 0x55b9e682f4c0 2024-05-09T00:02:28.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.579+0000 7fe5a7fff700 10 client.29663 handle_command_reply: tid=0 2024-05-09T00:02:28.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.579+0000 7fe5db1d3b80 1 client.29663 shutdown 2024-05-09T00:02:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.585+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/606399174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 msgr2=0x55b9e67cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.585+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67cd150 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc000c00 tx=0x7fe5bc0113e0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.585+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/606399174 shutdown_connections 2024-05-09T00:02:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.585+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b9e67cdbd0 0x55b9e682d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.585+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b9e682f4c0 0x55b9e68318e0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.585+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b9e67be890 0x55b9e67cd150 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc000c00 tx=0x7fe5bc0113e0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.585+0000 7fe5db1d3b80 1 --2- 172.21.15.12:0/606399174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9e67c4720 0x55b9e67cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.585+0000 7fe5a7fff700 0 client.29663 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.585+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/606399174 >> 172.21.15.12:0/606399174 conn(0x55b9e671c210 msgr2=0x55b9e67c2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.587+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/606399174 shutdown_connections 2024-05-09T00:02:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.589+0000 7fe5db1d3b80 1 -- 172.21.15.12:0/606399174 wait complete. 2024-05-09T00:02:28.584 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:02:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (161 inodes in the stack)", 2024-05-09T00:02:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:02:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "12118ea9-4c38-40cf-8ac4-6f088b899bb2": { 2024-05-09T00:02:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:02:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "12118ea9-4c38-40cf-8ac4-6f088b899bb2", 2024-05-09T00:02:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:02:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:02:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:02:28.585 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:02:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.590+0000 7fe5db1d3b80 20 client.29663 trim_cache size 0 max 16384 2024-05-09T00:02:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.592+0000 7fe5cf019700 1 -- 172.21.15.12:0/2255865699 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe5b80a1310 msgr2=0x7fe5b80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.592+0000 7fe5cf019700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe5b80a1310 0x7fe5b80a37d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400d010 tx=0x7fe5b4016040 comp rx=0 tx=0).stop 2024-05-09T00:02:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.592+0000 7fe5cf019700 1 -- 172.21.15.12:0/2255865699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c810d810 msgr2=0x7fe5c810a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.592+0000 7fe5cf019700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c810d810 0x7fe5c810a910 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000b070 tx=0x7fe5c00079e0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.593+0000 7fe5cf019700 1 -- 172.21.15.12:0/2255865699 shutdown_connections 2024-05-09T00:02:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.593+0000 7fe5cf019700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe5b80a1310 0x7fe5b80a37d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.593+0000 7fe5cf019700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c810d810 0x7fe5c810a910 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.593+0000 7fe5cf019700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c8073370 0x7fe5c810d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.593+0000 7fe5cf019700 1 --2- 172.21.15.12:0/2255865699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c8073c90 0x7fe5c810a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.593+0000 7fe5cf019700 1 -- 172.21.15.12:0/2255865699 >> 172.21.15.12:0/2255865699 conn(0x7fe5c80fca80 msgr2=0x7fe5c80fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.593+0000 7fe5cf019700 1 -- 172.21.15.12:0/2255865699 shutdown_connections 2024-05-09T00:02:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:28.593+0000 7fe5cf019700 1 -- 172.21.15.12:0/2255865699 wait complete. 2024-05-09T00:02:28.611 INFO:tasks.cephfs.filesystem:scrub status for tag:12118ea9-4c38-40cf-8ac4-6f088b899bb2 - {'path': '/', 'tag': '12118ea9-4c38-40cf-8ac4-6f088b899bb2', 'options': 'recursive,force'} 2024-05-09T00:02:28.621 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:02:28 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setuser ceph since I am not root 2024-05-09T00:02:28.621 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:02:28 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: ignoring --setgroup ceph since I am not root 2024-05-09T00:02:28.621 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:02:28 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: starting mds.c at 2024-05-09T00:02:28.621 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:28 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:02:28.621 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:28 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:28.621 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:28 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:02:28.621 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:28 smithi204 ceph-mon[47373]: daemon mds.c restarted 2024-05-09T00:02:28.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:28 smithi204 ceph-mon[47373]: osdmap e116: 12 total, 12 up, 12 in 2024-05-09T00:02:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:28 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:02:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:28 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:28 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:02:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:28 smithi012 ceph-mon[42503]: daemon mds.c restarted 2024-05-09T00:02:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:28 smithi012 ceph-mon[42503]: osdmap e116: 12 total, 12 up, 12 in 2024-05-09T00:02:29.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:28 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:02:29.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:28 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:29.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:28 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:02:29.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:28 smithi160 ceph-mon[46152]: daemon mds.c restarted 2024-05-09T00:02:29.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:28 smithi160 ceph-mon[46152]: osdmap e116: 12 total, 12 up, 12 in 2024-05-09T00:02:29.232 INFO:tasks.workunit.client.0.smithi012.stdout: 10 104937 40.67 MB/sec execute 24 sec latency 84.039 ms 2024-05-09T00:02:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:29 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:02:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:29 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:02:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:29 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:29 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6836/761168719,v1:172.21.15.204:6837/761168719] up:boot 2024-05-09T00:02:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:29 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:02:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:02:29.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:29 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:29 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:29 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:29 smithi012 ceph-mon[42503]: pgmap v517: 97 pgs: 97 active+clean; 1018 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 18 MiB/s wr, 775 op/s 2024-05-09T00:02:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:29 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:02:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:29 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:02:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:29 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:29 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6836/761168719,v1:172.21.15.204:6837/761168719] up:boot 2024-05-09T00:02:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:29 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:02:30.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:02:30.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:29 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:30.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:29 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:30.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:29 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:30.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:29 smithi204 ceph-mon[47373]: pgmap v517: 97 pgs: 97 active+clean; 1018 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 18 MiB/s wr, 775 op/s 2024-05-09T00:02:30.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:29 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:02:30.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:29 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:02:30.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:29 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:30.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:29 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6836/761168719,v1:172.21.15.204:6837/761168719] up:boot 2024-05-09T00:02:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:29 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:02:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:02:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:29 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:29 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:29 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:02:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:29 smithi160 ceph-mon[46152]: pgmap v517: 97 pgs: 97 active+clean; 1018 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 18 MiB/s wr, 775 op/s 2024-05-09T00:02:30.232 INFO:tasks.workunit.client.0.smithi012.stdout: 10 105528 40.17 MB/sec execute 25 sec latency 52.049 ms 2024-05-09T00:02:31.232 INFO:tasks.workunit.client.0.smithi012.stdout: 10 106436 39.93 MB/sec execute 26 sec latency 40.449 ms 2024-05-09T00:02:31.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:31 smithi012 ceph-mon[42503]: pgmap v518: 97 pgs: 97 active+clean; 1018 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 16 MiB/s wr, 646 op/s 2024-05-09T00:02:32.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:31 smithi204 ceph-mon[47373]: pgmap v518: 97 pgs: 97 active+clean; 1018 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 16 MiB/s wr, 646 op/s 2024-05-09T00:02:32.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:31 smithi160 ceph-mon[46152]: pgmap v518: 97 pgs: 97 active+clean; 1018 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 16 MiB/s wr, 646 op/s 2024-05-09T00:02:32.232 INFO:tasks.workunit.client.0.smithi012.stdout: 10 107634 41.17 MB/sec execute 27 sec latency 45.630 ms 2024-05-09T00:02:33.232 INFO:tasks.workunit.client.0.smithi012.stdout: 10 108296 41.48 MB/sec execute 28 sec latency 64.303 ms 2024-05-09T00:02:34.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:33 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:34.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:33 smithi012 ceph-mon[42503]: pgmap v519: 97 pgs: 97 active+clean; 916 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 23 MiB/s wr, 1.19k op/s 2024-05-09T00:02:34.232 INFO:tasks.workunit.client.0.smithi012.stdout: 10 108696 40.43 MB/sec execute 29 sec latency 239.742 ms 2024-05-09T00:02:34.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:33 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:34.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:33 smithi204 ceph-mon[47373]: pgmap v519: 97 pgs: 97 active+clean; 916 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 23 MiB/s wr, 1.19k op/s 2024-05-09T00:02:34.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:33 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:34.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:33 smithi160 ceph-mon[46152]: pgmap v519: 97 pgs: 97 active+clean; 916 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 23 MiB/s wr, 1.19k op/s 2024-05-09T00:02:35.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 109563 40.27 MB/sec execute 30 sec latency 50.345 ms 2024-05-09T00:02:35.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:35 smithi012 ceph-mon[42503]: pgmap v520: 97 pgs: 97 active+clean; 916 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 23 MiB/s wr, 1.19k op/s 2024-05-09T00:02:36.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:35 smithi204 ceph-mon[47373]: pgmap v520: 97 pgs: 97 active+clean; 916 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 23 MiB/s wr, 1.19k op/s 2024-05-09T00:02:36.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:35 smithi160 ceph-mon[46152]: pgmap v520: 97 pgs: 97 active+clean; 916 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 23 MiB/s wr, 1.19k op/s 2024-05-09T00:02:36.232 INFO:tasks.workunit.client.0.smithi012.stdout: 10 110695 41.14 MB/sec execute 31 sec latency 37.308 ms 2024-05-09T00:02:37.232 INFO:tasks.workunit.client.0.smithi012.stdout: 10 111514 41.77 MB/sec execute 32 sec latency 56.167 ms 2024-05-09T00:02:37.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:37 smithi012 ceph-mon[42503]: pgmap v521: 97 pgs: 97 active+clean; 915 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-05-09T00:02:38.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:37 smithi204 ceph-mon[47373]: pgmap v521: 97 pgs: 97 active+clean; 915 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-05-09T00:02:38.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:37 smithi160 ceph-mon[46152]: pgmap v521: 97 pgs: 97 active+clean; 915 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-05-09T00:02:38.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 112027 41.14 MB/sec execute 33 sec latency 56.984 ms 2024-05-09T00:02:38.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:38 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:02:39.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:38 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:02:39.052 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:38 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:02:39.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 112606 40.52 MB/sec execute 34 sec latency 125.994 ms 2024-05-09T00:02:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:39 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:02:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:39 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:02:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:39 smithi012 ceph-mon[42503]: pgmap v522: 97 pgs: 97 active+clean; 915 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 899 op/s 2024-05-09T00:02:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:39 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:02:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:39 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:02:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:39 smithi204 ceph-mon[47373]: pgmap v522: 97 pgs: 97 active+clean; 915 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 899 op/s 2024-05-09T00:02:40.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:39 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:02:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:39 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:02:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:39 smithi160 ceph-mon[46152]: pgmap v522: 97 pgs: 97 active+clean; 915 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 899 op/s 2024-05-09T00:02:40.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 113609 41.01 MB/sec execute 35 sec latency 81.466 ms 2024-05-09T00:02:41.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114582 41.42 MB/sec execute 36 sec latency 63.860 ms 2024-05-09T00:02:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.342+0000 7ffa22831700 1 -- 172.21.15.12:0/2693184520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0ffdd0 msgr2=0x7ffa1c10dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.342+0000 7ffa22831700 1 --2- 172.21.15.12:0/2693184520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0ffdd0 0x7ffa1c10dde0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400b0d0 tx=0x7ffa1400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.343+0000 7ffa22831700 1 -- 172.21.15.12:0/2693184520 shutdown_connections 2024-05-09T00:02:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.343+0000 7ffa22831700 1 --2- 172.21.15.12:0/2693184520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0ffdd0 0x7ffa1c10dde0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.343+0000 7ffa22831700 1 --2- 172.21.15.12:0/2693184520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa1c107020 0x7ffa1c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.343+0000 7ffa22831700 1 --2- 172.21.15.12:0/2693184520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa1c0ff410 0x7ffa1c0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.343+0000 7ffa22831700 1 -- 172.21.15.12:0/2693184520 >> 172.21.15.12:0/2693184520 conn(0x7ffa1c0763a0 msgr2=0x7ffa1c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.344+0000 7ffa22831700 1 -- 172.21.15.12:0/2693184520 shutdown_connections 2024-05-09T00:02:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.344+0000 7ffa22831700 1 -- 172.21.15.12:0/2693184520 wait complete. 2024-05-09T00:02:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.344+0000 7ffa22831700 1 Processor -- start 2024-05-09T00:02:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.345+0000 7ffa22831700 1 -- start start 2024-05-09T00:02:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.345+0000 7ffa22831700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa1c0ff410 0x7ffa1c10cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.345+0000 7ffa22831700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0ffdd0 0x7ffa1c10d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.345+0000 7ffa22831700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa1c107020 0x7ffa1c19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.345+0000 7ffa22831700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19b340 con 0x7ffa1c0ffdd0 2024-05-09T00:02:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.345+0000 7ffa22831700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19b4b0 con 0x7ffa1c107020 2024-05-09T00:02:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.345+0000 7ffa22831700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19b670 con 0x7ffa1c0ff410 2024-05-09T00:02:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0ffdd0 0x7ffa1c10d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0ffdd0 0x7ffa1c10d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52034/0 (socket says 172.21.15.12:52034) 2024-05-09T00:02:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1b7fe700 1 -- 172.21.15.12:0/878799037 learned_addr learned my addr 172.21.15.12:0/878799037 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa20dce700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa1c107020 0x7ffa1c19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa1c0ff410 0x7ffa1c10cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1b7fe700 1 -- 172.21.15.12:0/878799037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa1c0ff410 msgr2=0x7ffa1c10cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1b7fe700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa1c0ff410 0x7ffa1c10cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1b7fe700 1 -- 172.21.15.12:0/878799037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa1c107020 msgr2=0x7ffa1c19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1b7fe700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa1c107020 0x7ffa1c19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1b7fe700 1 -- 172.21.15.12:0/878799037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa14009d00 con 0x7ffa1c0ffdd0 2024-05-09T00:02:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa1c0ff410 0x7ffa1c10cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:02:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.346+0000 7ffa1b7fe700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0ffdd0 0x7ffa1c10d430 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00e9a0 tx=0x7ffa0c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.347+0000 7ffa197fa700 1 -- 172.21.15.12:0/878799037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c009790 con 0x7ffa1c0ffdd0 2024-05-09T00:02:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.347+0000 7ffa197fa700 1 -- 172.21.15.12:0/878799037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0c00cbd0 con 0x7ffa1c0ffdd0 2024-05-09T00:02:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.347+0000 7ffa197fa700 1 -- 172.21.15.12:0/878799037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c018680 con 0x7ffa1c0ffdd0 2024-05-09T00:02:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.348+0000 7ffa22831700 1 -- 172.21.15.12:0/878799037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c19b8f0 con 0x7ffa1c0ffdd0 2024-05-09T00:02:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.348+0000 7ffa22831700 1 -- 172.21.15.12:0/878799037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c19de40 con 0x7ffa1c0ffdd0 2024-05-09T00:02:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.352+0000 7ffa197fa700 1 -- 172.21.15.12:0/878799037 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ffa0c00cd40 con 0x7ffa1c0ffdd0 2024-05-09T00:02:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.353+0000 7ffa197fa700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1390 0x7ffa040a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.353+0000 7ffa197fa700 1 -- 172.21.15.12:0/878799037 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ffa0c014070 con 0x7ffa1c0ffdd0 2024-05-09T00:02:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.353+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1390 0x7ffa040a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa2e9ebb80 1 Processor -- start 2024-05-09T00:02:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa2e9ebb80 1 -- start start 2024-05-09T00:02:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a100950 0x55df0a100d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a0faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0a0520f0 con 0x55df0a100950 2024-05-09T00:02:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0a04fab0 con 0x55df0a0fb480 2024-05-09T00:02:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0a04b3a0 con 0x55df0a101270 2024-05-09T00:02:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa1b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a0faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa1b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a0faf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:56434/0 (socket says 172.21.15.12:56434) 2024-05-09T00:02:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.354+0000 7ffa1b7fe700 1 -- 172.21.15.12:0/1607897605 learned_addr learned my addr 172.21.15.12:0/1607897605 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa20dce700 1 --2- 172.21.15.12:0/1607897605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa1b7fe700 1 -- 172.21.15.12:0/1607897605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 msgr2=0x55df0a0fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa1b7fe700 1 --2- 172.21.15.12:0/1607897605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a0fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/1607897605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a100950 0x55df0a100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa1b7fe700 1 -- 172.21.15.12:0/1607897605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a100950 msgr2=0x55df0a100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa1b7fe700 1 --2- 172.21.15.12:0/1607897605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a100950 0x55df0a100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa1b7fe700 1 -- 172.21.15.12:0/1607897605 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df09ecc350 con 0x55df0a101270 2024-05-09T00:02:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/1607897605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a100950 0x55df0a100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:02:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa1b7fe700 1 --2- 172.21.15.12:0/1607897605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a0faf40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00f010 tx=0x7ffa0c09a990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b87bc144a7cf472 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1390 0x7ffa040a3850 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005e10 tx=0x7ffa10005d60 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.355+0000 7ffa0a7fc700 1 -- 172.21.15.12:0/1607897605 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0a2670 con 0x55df0a101270 2024-05-09T00:02:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.356+0000 7ffa0a7fc700 1 -- 172.21.15.12:0/1607897605 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0c098330 con 0x55df0a101270 2024-05-09T00:02:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.356+0000 7ffa0a7fc700 1 -- 172.21.15.12:0/1607897605 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c09a6b0 con 0x55df0a101270 2024-05-09T00:02:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.356+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/1607897605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 msgr2=0x55df0a0faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.356+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/1607897605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a0faf40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00f010 tx=0x7ffa0c09a990 comp rx=0 tx=0).stop 2024-05-09T00:02:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.357+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/1607897605 shutdown_connections 2024-05-09T00:02:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.357+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/1607897605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a0fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.357+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/1607897605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a100950 0x55df0a100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.357+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/1607897605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a0faf40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.357+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/1607897605 >> 172.21.15.12:0/1607897605 conn(0x55df0a0591b0 msgr2=0x55df0a0f5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.358+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/1607897605 shutdown_connections 2024-05-09T00:02:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.359+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/1607897605 wait complete. 2024-05-09T00:02:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.360+0000 7ffa2e9ebb80 1 -- start start 2024-05-09T00:02:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.360+0000 7ffa2e9ebb80 1 Processor -- start 2024-05-09T00:02:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.360+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a107220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.360+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a100950 0x55df0a104510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a104c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df09fe7500 con 0x55df0a100950 2024-05-09T00:02:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df09fe71a0 con 0x55df0a0fb480 2024-05-09T00:02:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df09ece640 con 0x55df0a101270 2024-05-09T00:02:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa2e9ebb80 10 client.0 _ll_register_callbacks cb 0x55df09ff0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa20dce700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a104c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa20dce700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a104c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:56438/0 (socket says 172.21.15.12:56438) 2024-05-09T00:02:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa20dce700 1 -- 172.21.15.12:0/653206037 learned_addr learned my addr 172.21.15.12:0/653206037 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a107220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa20dce700 1 -- 172.21.15.12:0/653206037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 msgr2=0x55df0a107220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa20dce700 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a107220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa20dce700 1 -- 172.21.15.12:0/653206037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a100950 msgr2=0x55df0a104510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:02:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa20dce700 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a100950 0x55df0a104510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.361+0000 7ffa20dce700 1 -- 172.21.15.12:0/653206037 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c19de40 con 0x55df0a101270 2024-05-09T00:02:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.362+0000 7ffa20dce700 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a104c50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffa14004cb0 tx=0x7ffa14004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=885356d08d862126 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.363+0000 7ff9ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:02:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.363+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/653206037 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1400e040 con 0x55df0a101270 2024-05-09T00:02:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.363+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/653206037 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ffa14009450 con 0x55df0a101270 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.363+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/653206037 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df09fa4210 con 0x55df0a101270 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.363+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/653206037 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa140078f0 con 0x55df0a101270 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.363+0000 7ffa2e9ebb80 10 client.29767 fetch_fsmap learned FSMap version 67 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.363+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/653206037 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55df09e78b00 con 0x55df0a101270 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.364+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/653206037 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df09fa4210 con 0x55df0a101270 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.364+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/653206037 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 67) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ffa1400b100 con 0x55df0a101270 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.364+0000 7ffa2e9ebb80 10 client.29767 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.364+0000 7ffa2e9ebb80 10 client.29767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.364+0000 7ffa2e9ebb80 20 client.29767 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.364+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55df0a125bc0 0x55df0a127fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.364+0000 7ffa2e9ebb80 4 client.29767 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.364+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/653206037 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df0a0ca7b0 con 0x55df0a125bc0 2024-05-09T00:02:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.364+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55df0a125bc0 0x55df0a127fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.365+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55df0a125bc0 0x55df0a127fe0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9703fb76b100c89 server_cookie=4cad1239ba5c065e in_seq=0 out_seq=0 2024-05-09T00:02:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.365+0000 7ff9ff7fe700 10 client.29767 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.369+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/653206037 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df0a0ca7b0 con 0x55df0a125bc0 2024-05-09T00:02:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.369+0000 7ff9ff7fe700 10 client.29767 handle_command_reply: tid=0 2024-05-09T00:02:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.369+0000 7ffa2e9ebb80 1 client.29767 shutdown 2024-05-09T00:02:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.369+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/653206037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 msgr2=0x55df0a104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.369+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a104c50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffa14004cb0 tx=0x7ffa14004ce0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.370+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/653206037 shutdown_connections 2024-05-09T00:02:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.370+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a101270 0x55df0a104c50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.370+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55df0a125bc0 0x55df0a127fe0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.370+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.370+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/653206037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a100950 0x55df0a104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.370+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/653206037 >> 172.21.15.12:0/653206037 conn(0x55df0a0591b0 msgr2=0x55df0a0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.370+0000 7ff9ff7fe700 0 client.29767 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.371+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/653206037 shutdown_connections 2024-05-09T00:02:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.371+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/653206037 wait complete. 2024-05-09T00:02:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.371+0000 7ffa2e9ebb80 20 client.29767 trim_cache size 0 max 16384 2024-05-09T00:02:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.391+0000 7ffa2e9ebb80 1 Processor -- start 2024-05-09T00:02:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa2e9ebb80 1 -- start start 2024-05-09T00:02:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a104b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 0x55df0a0f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a101270 0x55df0a0ff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0a0f6330 con 0x55df0a101270 2024-05-09T00:02:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0a0f7290 con 0x55df0a0fb480 2024-05-09T00:02:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0a0fa140 con 0x55df0a100950 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa20dce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a101270 0x55df0a0ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa1bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a104b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa1bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a104b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:59676/0 (socket says 172.21.15.12:59676) 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa1b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 0x55df0a0f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa1bfff700 1 -- 172.21.15.12:0/4010355074 learned_addr learned my addr 172.21.15.12:0/4010355074 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa20dce700 1 -- 172.21.15.12:0/4010355074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 msgr2=0x55df0a0f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa20dce700 1 --2- 172.21.15.12:0/4010355074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 0x55df0a0f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa20dce700 1 -- 172.21.15.12:0/4010355074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 msgr2=0x55df0a104b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa20dce700 1 --2- 172.21.15.12:0/4010355074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a104b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.392+0000 7ffa20dce700 1 -- 172.21.15.12:0/4010355074 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df0a011500 con 0x55df0a101270 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.393+0000 7ffa20dce700 1 --2- 172.21.15.12:0/4010355074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a101270 0x55df0a0ff390 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ffa14004c80 tx=0x7ffa1402e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdde5892299c687 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.393+0000 7ffa0affd700 1 -- 172.21.15.12:0/4010355074 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1400e020 con 0x55df0a101270 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.393+0000 7ffa0affd700 1 -- 172.21.15.12:0/4010355074 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa14006120 con 0x55df0a101270 2024-05-09T00:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.393+0000 7ffa0affd700 1 -- 172.21.15.12:0/4010355074 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1402ea60 con 0x55df0a101270 2024-05-09T00:02:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/4010355074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a101270 msgr2=0x55df0a0ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/4010355074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a101270 0x55df0a0ff390 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ffa14004c80 tx=0x7ffa1402e820 comp rx=0 tx=0).stop 2024-05-09T00:02:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/4010355074 shutdown_connections 2024-05-09T00:02:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/4010355074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a101270 0x55df0a0ff390 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/4010355074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a0fb480 0x55df0a104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/4010355074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 0x55df0a0f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/4010355074 >> 172.21.15.12:0/4010355074 conn(0x55df0a0591b0 msgr2=0x55df0a0f1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/4010355074 shutdown_connections 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/4010355074 wait complete. 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 -- start start 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.394+0000 7ffa2e9ebb80 1 Processor -- start 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a0fb480 0x55df0a11cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 0x55df0a11d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa2e9ebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a101270 0x55df0a118da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0a1194b0 con 0x55df0a0fb480 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0a119620 con 0x55df0a101270 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa2e9ebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0a1197e0 con 0x55df0a100950 2024-05-09T00:02:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa2e9ebb80 10 client.0 _ll_register_callbacks cb 0x55df09ff0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a0fb480 0x55df0a11cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 0x55df0a11d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a0fb480 0x55df0a11cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52054/0 (socket says 172.21.15.12:52054) 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1bfff700 1 -- 172.21.15.12:0/3305922992 learned_addr learned my addr 172.21.15.12:0/3305922992 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa20dce700 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a101270 0x55df0a118da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1bfff700 1 -- 172.21.15.12:0/3305922992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 msgr2=0x55df0a11d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 0x55df0a11d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1bfff700 1 -- 172.21.15.12:0/3305922992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a101270 msgr2=0x55df0a118da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a101270 0x55df0a118da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1bfff700 1 -- 172.21.15.12:0/3305922992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df09e78b00 con 0x55df0a0fb480 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1b7fe700 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 0x55df0a11d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ffa1bfff700 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a0fb480 0x55df0a11cc30 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000fc20 tx=0x7ffa100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2327ef0d18df681e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.395+0000 7ff9ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3305922992 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10024040 con 0x55df0a0fb480 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3305922992 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ffa10026750 con 0x55df0a0fb480 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/3305922992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa0c098330 con 0x55df0a0fb480 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3305922992 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa10026910 con 0x55df0a0fb480 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ffa2e9ebb80 10 client.20031 fetch_fsmap learned FSMap version 67 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/3305922992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55df09ed0c70 con 0x55df0a0fb480 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3305922992 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0c098330 con 0x55df0a0fb480 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3305922992 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 67) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ffa1001e590 con 0x55df0a0fb480 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ffa2e9ebb80 10 client.20031 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ffa2e9ebb80 10 client.20031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ffa2e9ebb80 20 client.20031 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55df0a17c4b0 0x55df0a17e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ffa2e9ebb80 4 client.20031 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:02:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.396+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/3305922992 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df0a043e60 con 0x55df0a17c4b0 2024-05-09T00:02:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.397+0000 7ffa1b7fe700 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55df0a17c4b0 0x55df0a17e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.397+0000 7ff9ff7fe700 10 client.20031 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.397+0000 7ffa1b7fe700 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55df0a17c4b0 0x55df0a17e8d0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d399ff0cfd999 server_cookie=996cc312814543c4 in_seq=0 out_seq=0 2024-05-09T00:02:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.744+0000 7ff9ff7fe700 1 -- 172.21.15.12:0/3305922992 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df0a043e60 con 0x55df0a17c4b0 2024-05-09T00:02:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.744+0000 7ff9ff7fe700 10 client.20031 handle_command_reply: tid=0 2024-05-09T00:02:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.744+0000 7ffa2e9ebb80 1 client.20031 shutdown 2024-05-09T00:02:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.744+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/3305922992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a0fb480 msgr2=0x55df0a11cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.744+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a0fb480 0x55df0a11cc30 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000fc20 tx=0x7ffa100047c0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.746+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/3305922992 shutdown_connections 2024-05-09T00:02:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.746+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55df0a17c4b0 0x55df0a17e8d0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.746+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55df0a101270 0x55df0a118da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.746+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df0a0fb480 0x55df0a11cc30 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.746+0000 7ffa2e9ebb80 1 --2- 172.21.15.12:0/3305922992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df0a100950 0x55df0a11d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.746+0000 7ff9ff7fe700 0 client.20031 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.746+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/3305922992 >> 172.21.15.12:0/3305922992 conn(0x55df0a0591b0 msgr2=0x55df0a12c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.746+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/3305922992 shutdown_connections 2024-05-09T00:02:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.746+0000 7ffa2e9ebb80 1 -- 172.21.15.12:0/3305922992 wait complete. 2024-05-09T00:02:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.748+0000 7ffa2e9ebb80 20 client.20031 trim_cache size 0 max 16384 2024-05-09T00:02:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:02:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:02:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:02:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:02:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.749+0000 7ffa22831700 1 -- 172.21.15.12:0/878799037 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1390 msgr2=0x7ffa040a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.749+0000 7ffa22831700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1390 0x7ffa040a3850 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005e10 tx=0x7ffa10005d60 comp rx=0 tx=0).stop 2024-05-09T00:02:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.749+0000 7ffa22831700 1 -- 172.21.15.12:0/878799037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0ffdd0 msgr2=0x7ffa1c10d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.749+0000 7ffa22831700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0ffdd0 0x7ffa1c10d430 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00e9a0 tx=0x7ffa0c00ed60 comp rx=0 tx=0).stop 2024-05-09T00:02:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.749+0000 7ffa22831700 1 -- 172.21.15.12:0/878799037 shutdown_connections 2024-05-09T00:02:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.749+0000 7ffa22831700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1390 0x7ffa040a3850 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.749+0000 7ffa22831700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa1c107020 0x7ffa1c19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.749+0000 7ffa22831700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa1c0ff410 0x7ffa1c10cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.749+0000 7ffa22831700 1 --2- 172.21.15.12:0/878799037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0ffdd0 0x7ffa1c10d430 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.749+0000 7ffa22831700 1 -- 172.21.15.12:0/878799037 >> 172.21.15.12:0/878799037 conn(0x7ffa1c0763a0 msgr2=0x7ffa1c1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.751+0000 7ffa22831700 1 -- 172.21.15.12:0/878799037 shutdown_connections 2024-05-09T00:02:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:41.754+0000 7ffa22831700 1 -- 172.21.15.12:0/878799037 wait complete. 2024-05-09T00:02:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:41 smithi204 ceph-mon[47373]: pgmap v523: 97 pgs: 97 active+clean; 915 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-05-09T00:02:42.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:41 smithi160 ceph-mon[46152]: pgmap v523: 97 pgs: 97 active+clean; 915 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-05-09T00:02:42.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:41 smithi012 ceph-mon[42503]: pgmap v523: 97 pgs: 97 active+clean; 915 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-05-09T00:02:42.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 115257 41.40 MB/sec execute 37 sec latency 228.025 ms 2024-05-09T00:02:43.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 115934 41.17 MB/sec execute 38 sec latency 69.770 ms 2024-05-09T00:02:43.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:43 smithi012 ceph-mon[42503]: pgmap v524: 97 pgs: 97 active+clean; 927 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-09T00:02:44.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:43 smithi204 ceph-mon[47373]: pgmap v524: 97 pgs: 97 active+clean; 927 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-09T00:02:44.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:43 smithi160 ceph-mon[46152]: pgmap v524: 97 pgs: 97 active+clean; 927 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-09T00:02:44.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 116653 41.02 MB/sec execute 39 sec latency 85.151 ms 2024-05-09T00:02:45.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 117527 41.09 MB/sec execute 40 sec latency 42.220 ms 2024-05-09T00:02:45.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:45 smithi012 ceph-mon[42503]: pgmap v525: 97 pgs: 97 active+clean; 927 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.1 MiB/s wr, 591 op/s 2024-05-09T00:02:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:45 smithi204 ceph-mon[47373]: pgmap v525: 97 pgs: 97 active+clean; 927 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.1 MiB/s wr, 591 op/s 2024-05-09T00:02:46.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:45 smithi160 ceph-mon[46152]: pgmap v525: 97 pgs: 97 active+clean; 927 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.1 MiB/s wr, 591 op/s 2024-05-09T00:02:46.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 118492 41.47 MB/sec execute 41 sec latency 60.211 ms 2024-05-09T00:02:47.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 119300 41.71 MB/sec execute 42 sec latency 56.184 ms 2024-05-09T00:02:47.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:47 smithi012 ceph-mon[42503]: pgmap v526: 97 pgs: 97 active+clean; 954 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-05-09T00:02:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:47 smithi204 ceph-mon[47373]: pgmap v526: 97 pgs: 97 active+clean; 954 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-05-09T00:02:48.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:47 smithi160 ceph-mon[46152]: pgmap v526: 97 pgs: 97 active+clean; 954 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-05-09T00:02:48.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 119907 41.29 MB/sec execute 43 sec latency 67.755 ms 2024-05-09T00:02:49.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 120769 41.22 MB/sec execute 44 sec latency 59.387 ms 2024-05-09T00:02:49.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:49 smithi012 ceph-mon[42503]: pgmap v527: 97 pgs: 97 active+clean; 954 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 574 op/s 2024-05-09T00:02:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:49 smithi204 ceph-mon[47373]: pgmap v527: 97 pgs: 97 active+clean; 954 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 574 op/s 2024-05-09T00:02:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:49 smithi160 ceph-mon[46152]: pgmap v527: 97 pgs: 97 active+clean; 954 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 574 op/s 2024-05-09T00:02:50.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 121928 41.77 MB/sec execute 45 sec latency 35.294 ms 2024-05-09T00:02:51.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:50 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:51.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 122730 42.19 MB/sec execute 46 sec latency 63.937 ms 2024-05-09T00:02:51.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:50 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:51.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:50 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:52.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:51 smithi012 ceph-mon[42503]: pgmap v528: 97 pgs: 97 active+clean; 954 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 574 op/s 2024-05-09T00:02:52.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 123328 41.79 MB/sec execute 47 sec latency 40.693 ms 2024-05-09T00:02:52.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:51 smithi204 ceph-mon[47373]: pgmap v528: 97 pgs: 97 active+clean; 954 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 574 op/s 2024-05-09T00:02:52.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:51 smithi160 ceph-mon[46152]: pgmap v528: 97 pgs: 97 active+clean; 954 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 574 op/s 2024-05-09T00:02:53.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:52 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:53.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:52 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:53.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:52 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:53.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:52 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:02:53.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 124226 41.64 MB/sec execute 48 sec latency 43.515 ms 2024-05-09T00:02:53.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:52 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:53.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:52 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:53.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:52 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:53.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:52 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:02:53.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:52 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:53.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:52 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:53.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:52 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:53.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:52 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:02:54.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:53 smithi012 ceph-mon[42503]: pgmap v529: 97 pgs: 97 active+clean; 982 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.9 MiB/s wr, 818 op/s 2024-05-09T00:02:54.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 125278 42.22 MB/sec execute 49 sec latency 37.925 ms 2024-05-09T00:02:54.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:53 smithi204 ceph-mon[47373]: pgmap v529: 97 pgs: 97 active+clean; 982 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.9 MiB/s wr, 818 op/s 2024-05-09T00:02:54.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:53 smithi160 ceph-mon[46152]: pgmap v529: 97 pgs: 97 active+clean; 982 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.9 MiB/s wr, 818 op/s 2024-05-09T00:02:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.892+0000 7f2d2888f700 1 -- 172.21.15.12:0/2373404851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d24100760 msgr2=0x7f2d2410cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.892+0000 7f2d2888f700 1 --2- 172.21.15.12:0/2373404851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d24100760 0x7f2d2410cc00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b0a0 tx=0x7f2d1c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.892+0000 7f2d2888f700 1 -- 172.21.15.12:0/2373404851 shutdown_connections 2024-05-09T00:02:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.892+0000 7f2d2888f700 1 --2- 172.21.15.12:0/2373404851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d24100760 0x7f2d2410cc00 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.892+0000 7f2d2888f700 1 --2- 172.21.15.12:0/2373404851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24105e60 0x7f2d24106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.892+0000 7f2d2888f700 1 --2- 172.21.15.12:0/2373404851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d240ffda0 0x7f2d24100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.892+0000 7f2d2888f700 1 -- 172.21.15.12:0/2373404851 >> 172.21.15.12:0/2373404851 conn(0x7f2d24076f10 msgr2=0x7f2d24077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.892+0000 7f2d2888f700 1 -- 172.21.15.12:0/2373404851 shutdown_connections 2024-05-09T00:02:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.892+0000 7f2d2888f700 1 -- 172.21.15.12:0/2373404851 wait complete. 2024-05-09T00:02:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2888f700 1 Processor -- start 2024-05-09T00:02:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2888f700 1 -- start start 2024-05-09T00:02:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2888f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d240ffda0 0x7f2d241973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2888f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d24100760 0x7f2d24197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2888f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24105e60 0x7f2d2419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2888f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2419c2f0 con 0x7f2d240ffda0 2024-05-09T00:02:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2888f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2419c460 con 0x7f2d24100760 2024-05-09T00:02:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2888f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2419c620 con 0x7f2d24105e60 2024-05-09T00:02:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d240ffda0 0x7f2d241973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d21d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d24100760 0x7f2d24197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d240ffda0 0x7f2d241973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48628/0 (socket says 172.21.15.12:48628) 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d22d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24105e60 0x7f2d2419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.893+0000 7f2d2259c700 1 -- 172.21.15.12:0/3006989813 learned_addr learned my addr 172.21.15.12:0/3006989813 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d22d9d700 1 -- 172.21.15.12:0/3006989813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d24100760 msgr2=0x7f2d24197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d22d9d700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d24100760 0x7f2d24197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d22d9d700 1 -- 172.21.15.12:0/3006989813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d240ffda0 msgr2=0x7f2d241973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d22d9d700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d240ffda0 0x7f2d241973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d22d9d700 1 -- 172.21.15.12:0/3006989813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c009d00 con 0x7f2d24105e60 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d21d9b700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d24100760 0x7f2d24197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d22d9d700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24105e60 0x7f2d2419bcb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b0a0 tx=0x7f2d1c0078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d137fe700 1 -- 172.21.15.12:0/3006989813 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c00e040 con 0x7f2d24105e60 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d2888f700 1 -- 172.21.15.12:0/3006989813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2419c8a0 con 0x7f2d24105e60 2024-05-09T00:02:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d137fe700 1 -- 172.21.15.12:0/3006989813 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d1c01d470 con 0x7f2d24105e60 2024-05-09T00:02:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d137fe700 1 -- 172.21.15.12:0/3006989813 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c005260 con 0x7f2d24105e60 2024-05-09T00:02:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.894+0000 7f2d2888f700 1 -- 172.21.15.12:0/3006989813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d241a4140 con 0x7f2d24105e60 2024-05-09T00:02:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.897+0000 7f2d137fe700 1 -- 172.21.15.12:0/3006989813 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2d1c01d5e0 con 0x7f2d24105e60 2024-05-09T00:02:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.897+0000 7f2d137fe700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d0c0a1290 0x7f2d0c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.897+0000 7f2d2259c700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d0c0a1290 0x7f2d0c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.897+0000 7f2d137fe700 1 -- 172.21.15.12:0/3006989813 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2d1c0cde40 con 0x7f2d24105e60 2024-05-09T00:02:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.898+0000 7f2d2259c700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d0c0a1290 0x7f2d0c0a3750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009d20 tx=0x7f2d14009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d34a49b80 1 Processor -- start 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d34a49b80 1 -- start start 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc57d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc51f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc528a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbcba90f0 con 0x559fbcc58270 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbcba6ab0 con 0x559fbcc52480 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbcba23a0 con 0x559fbcc57950 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d2259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc57d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d21d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc51f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d2259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc57d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41796/0 (socket says 172.21.15.12:41796) 2024-05-09T00:02:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d21d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc51f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48634/0 (socket says 172.21.15.12:48634) 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d2259c700 1 -- 172.21.15.12:0/2542436588 learned_addr learned my addr 172.21.15.12:0/2542436588 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.899+0000 7f2d22d9d700 1 --2- 172.21.15.12:0/2542436588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.900+0000 7f2d2259c700 1 -- 172.21.15.12:0/2542436588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 msgr2=0x559fbcc528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.900+0000 7f2d2259c700 1 --2- 172.21.15.12:0/2542436588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.900+0000 7f2d2259c700 1 -- 172.21.15.12:0/2542436588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 msgr2=0x559fbcc51f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.900+0000 7f2d2259c700 1 --2- 172.21.15.12:0/2542436588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc51f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.900+0000 7f2d2259c700 1 -- 172.21.15.12:0/2542436588 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fbca23350 con 0x559fbcc57950 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.900+0000 7f2d21d9b700 1 --2- 172.21.15.12:0/2542436588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc51f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.900+0000 7f2d2259c700 1 --2- 172.21.15.12:0/2542436588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc57d30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e540 tx=0x7f2d14014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb62a3e20463c8e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.900+0000 7f2d10ff9700 1 -- 172.21.15.12:0/2542436588 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14026430 con 0x559fbcc57950 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.900+0000 7f2d10ff9700 1 -- 172.21.15.12:0/2542436588 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d14021070 con 0x559fbcc57950 2024-05-09T00:02:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.901+0000 7f2d10ff9700 1 -- 172.21.15.12:0/2542436588 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1401d790 con 0x559fbcc57950 2024-05-09T00:02:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.901+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2542436588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 msgr2=0x559fbcc57d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.901+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2542436588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc57d30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e540 tx=0x7f2d14014000 comp rx=0 tx=0).stop 2024-05-09T00:02:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.901+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2542436588 shutdown_connections 2024-05-09T00:02:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.901+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2542436588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.901+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2542436588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc57d30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.901+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2542436588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc51f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.901+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2542436588 >> 172.21.15.12:0/2542436588 conn(0x559fbcbb01b0 msgr2=0x559fbcc4c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.902+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2542436588 shutdown_connections 2024-05-09T00:02:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.902+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2542436588 wait complete. 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.902+0000 7f2d34a49b80 1 -- start start 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.902+0000 7f2d34a49b80 1 Processor -- start 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.902+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc5b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.902+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc57950 0x559fbcc5ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.902+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc58270 0x559fbcc5bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.902+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbcb3e500 con 0x559fbcc57950 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.902+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbcb3e1a0 con 0x559fbcc52480 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.902+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbca25640 con 0x559fbcc58270 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d34a49b80 10 client.0 _ll_register_callbacks cb 0x559fbcb47ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d21d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc57950 0x559fbcc5ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d2259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc5b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d2259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc5b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:55828/0 (socket says 172.21.15.12:55828) 2024-05-09T00:02:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d21d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc57950 0x559fbcc5ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48636/0 (socket says 172.21.15.12:48636) 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d2259c700 1 -- 172.21.15.12:0/2081695449 learned_addr learned my addr 172.21.15.12:0/2081695449 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d22d9d700 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc58270 0x559fbcc5bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d21d9b700 1 -- 172.21.15.12:0/2081695449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc58270 msgr2=0x559fbcc5bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d21d9b700 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc58270 0x559fbcc5bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d21d9b700 1 -- 172.21.15.12:0/2081695449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 msgr2=0x559fbcc5b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d21d9b700 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc5b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d21d9b700 1 -- 172.21.15.12:0/2081695449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d18009c90 con 0x559fbcc57950 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d2259c700 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc5b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d21d9b700 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc57950 0x559fbcc5ba50 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x559fbcc4d790 tx=0x7f2d1800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25ca27ab5ef0f2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2081695449 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18017070 con 0x559fbcc57950 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2081695449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fbcafb210 con 0x559fbcc57950 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.903+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2081695449 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2d18004030 con 0x559fbcc57950 2024-05-09T00:02:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2081695449 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d180041f0 con 0x559fbcc57950 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d34a49b80 10 client.20043 fetch_fsmap learned FSMap version 68 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2081695449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7f2d14017070 con 0x559fbcc57950 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2081695449 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559fbcafb210 con 0x559fbcc57950 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2081695449 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f2d180039e0 con 0x559fbcc57950 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d34a49b80 10 client.20043 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d34a49b80 10 client.20043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d34a49b80 20 client.20043 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559fbcc6bd10 0x559fbcc6e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d34a49b80 4 client.20043 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.904+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2081695449 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fbcc217b0 con 0x559fbcc6bd10 2024-05-09T00:02:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.906+0000 7f2d2259c700 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559fbcc6bd10 0x559fbcc6e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.907+0000 7f2d2259c700 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559fbcc6bd10 0x559fbcc6e130 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819a5ea675109e77 server_cookie=88ebe0a04823263d in_seq=0 out_seq=0 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.907+0000 7f2d09ffb700 10 client.20043 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2081695449 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559fbcc217b0 con 0x559fbcc6bd10 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d09ffb700 10 client.20043 handle_command_reply: tid=0 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d34a49b80 1 client.20043 shutdown 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2081695449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc57950 msgr2=0x559fbcc5ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc57950 0x559fbcc5ba50 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x559fbcc4d790 tx=0x7f2d1800f3c0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2081695449 shutdown_connections 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc58270 0x559fbcc5bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559fbcc6bd10 0x559fbcc6e130 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc5b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2081695449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc57950 0x559fbcc5ba50 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d09ffb700 0 client.20043 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2081695449 >> 172.21.15.12:0/2081695449 conn(0x559fbcbb01b0 msgr2=0x559fbcc50f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.908+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2081695449 shutdown_connections 2024-05-09T00:02:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.909+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2081695449 wait complete. 2024-05-09T00:02:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.909+0000 7f2d34a49b80 20 client.20043 trim_cache size 0 max 16384 2024-05-09T00:02:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d34a49b80 1 Processor -- start 2024-05-09T00:02:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d34a49b80 1 -- start start 2024-05-09T00:02:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc4c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc56c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc5bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbcc477c0 con 0x559fbcc58270 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbcc47330 con 0x559fbcc52480 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbcc5cb80 con 0x559fbcc57950 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d22d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc5bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d22d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc5bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48642/0 (socket says 172.21.15.12:48642) 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d2259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc4c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d22d9d700 1 -- 172.21.15.12:0/2023721408 learned_addr learned my addr 172.21.15.12:0/2023721408 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.926+0000 7f2d21d9b700 1 --2- 172.21.15.12:0/2023721408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc56c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.927+0000 7f2d22d9d700 1 -- 172.21.15.12:0/2023721408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 msgr2=0x559fbcc56c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.927+0000 7f2d22d9d700 1 --2- 172.21.15.12:0/2023721408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc56c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.927+0000 7f2d22d9d700 1 -- 172.21.15.12:0/2023721408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 msgr2=0x559fbcc4c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.927+0000 7f2d22d9d700 1 --2- 172.21.15.12:0/2023721408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc4c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.927+0000 7f2d22d9d700 1 -- 172.21.15.12:0/2023721408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fbc9cfb00 con 0x559fbcc58270 2024-05-09T00:02:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.927+0000 7f2d21d9b700 1 --2- 172.21.15.12:0/2023721408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc56c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.927+0000 7f2d22d9d700 1 --2- 172.21.15.12:0/2023721408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc5bdb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b070 tx=0x7f2d1c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65db8a4e493d9545 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.927+0000 7f2d117fa700 1 -- 172.21.15.12:0/2023721408 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c09a980 con 0x559fbcc58270 2024-05-09T00:02:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.927+0000 7f2d117fa700 1 -- 172.21.15.12:0/2023721408 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d1c09b720 con 0x559fbcc58270 2024-05-09T00:02:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.927+0000 7f2d117fa700 1 -- 172.21.15.12:0/2023721408 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c097730 con 0x559fbcc58270 2024-05-09T00:02:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2023721408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 msgr2=0x559fbcc5bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2023721408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc5bdb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b070 tx=0x7f2d1c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2023721408 shutdown_connections 2024-05-09T00:02:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2023721408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc5bdb0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2023721408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc52480 0x559fbcc4c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2023721408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc57950 0x559fbcc56c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2023721408 >> 172.21.15.12:0/2023721408 conn(0x559fbcbb01b0 msgr2=0x559fbcc56560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2023721408 shutdown_connections 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2023721408 wait complete. 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 -- start start 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.928+0000 7f2d34a49b80 1 Processor -- start 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc52480 0x559fbcc63100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc57950 0x559fbcc605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d34a49b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc60d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbcc61420 con 0x559fbcc58270 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbccc11d0 con 0x559fbcc57950 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d34a49b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbccc1310 con 0x559fbcc52480 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d34a49b80 10 client.0 _ll_register_callbacks cb 0x559fbcb47ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d22d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc60d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d21d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc57950 0x559fbcc605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d2259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc52480 0x559fbcc63100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d2259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc52480 0x559fbcc63100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41818/0 (socket says 172.21.15.12:41818) 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d21d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc57950 0x559fbcc605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:55840/0 (socket says 172.21.15.12:55840) 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d22d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc60d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48650/0 (socket says 172.21.15.12:48650) 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d2259c700 1 -- 172.21.15.12:0/2701556615 learned_addr learned my addr 172.21.15.12:0/2701556615 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d2259c700 1 -- 172.21.15.12:0/2701556615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc57950 msgr2=0x559fbcc605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d2259c700 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc57950 0x559fbcc605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d2259c700 1 -- 172.21.15.12:0/2701556615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 msgr2=0x559fbcc60d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d2259c700 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc60d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d2259c700 1 -- 172.21.15.12:0/2701556615 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d14017070 con 0x559fbcc52480 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.929+0000 7f2d22d9d700 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc60d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d2259c700 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc52480 0x559fbcc63100 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d1401d010 tx=0x7f2d14005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a1d1e78731cfcf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2701556615 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400ede0 con 0x559fbcc52480 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2701556615 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2d14005040 con 0x559fbcc52480 2024-05-09T00:02:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2701556615 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d14021070 con 0x559fbcc52480 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2701556615 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d14005200 con 0x559fbcc52480 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d34a49b80 10 client.29791 fetch_fsmap learned FSMap version 68 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2701556615 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7f2d241a4140 con 0x559fbcc52480 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2701556615 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d14021070 con 0x559fbcc52480 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2701556615 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f2d140330f0 con 0x559fbcc52480 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.930+0000 7f2d34a49b80 10 client.29791 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.931+0000 7f2d34a49b80 10 client.29791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.931+0000 7f2d34a49b80 20 client.29791 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.931+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559fbccc2ac0 0x559fbccc4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.931+0000 7f2d34a49b80 4 client.29791 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.931+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2701556615 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559fbcb9ae60 con 0x559fbccc2ac0 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.931+0000 7f2d21d9b700 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559fbccc2ac0 0x559fbccc4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.931+0000 7f2d09ffb700 10 client.29791 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:54.931+0000 7f2d21d9b700 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559fbccc2ac0 0x559fbccc4ee0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969dc89847446b9f server_cookie=d735272c686c859a in_seq=0 out_seq=0 2024-05-09T00:02:55.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 125853 41.97 MB/sec execute 50 sec latency 207.984 ms 2024-05-09T00:02:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d09ffb700 1 -- 172.21.15.12:0/2701556615 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559fbcb9ae60 con 0x559fbccc2ac0 2024-05-09T00:02:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d09ffb700 10 client.29791 handle_command_reply: tid=0 2024-05-09T00:02:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 client.29791 shutdown 2024-05-09T00:02:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2701556615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc52480 msgr2=0x559fbcc63100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc52480 0x559fbcc63100 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d1401d010 tx=0x7f2d14005c80 comp rx=0 tx=0).stop 2024-05-09T00:02:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2701556615 shutdown_connections 2024-05-09T00:02:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fbcc58270 0x559fbcc60d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fbcc52480 0x559fbcc63100 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559fbccc2ac0 0x559fbccc4ee0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 --2- 172.21.15.12:0/2701556615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559fbcc57950 0x559fbcc605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d09ffb700 0 client.29791 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2701556615 >> 172.21.15.12:0/2701556615 conn(0x559fbcbb01b0 msgr2=0x559fbcc5bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2701556615 shutdown_connections 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 1 -- 172.21.15.12:0/2701556615 wait complete. 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.271+0000 7f2d34a49b80 20 client.29791 trim_cache size 0 max 16384 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:02:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:02:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 -- 172.21.15.12:0/3006989813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d0c0a1290 msgr2=0x7f2d0c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d0c0a1290 0x7f2d0c0a3750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009d20 tx=0x7f2d14009260 comp rx=0 tx=0).stop 2024-05-09T00:02:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 -- 172.21.15.12:0/3006989813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24105e60 msgr2=0x7f2d2419bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24105e60 0x7f2d2419bcb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b0a0 tx=0x7f2d1c0078e0 comp rx=0 tx=0).stop 2024-05-09T00:02:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 -- 172.21.15.12:0/3006989813 shutdown_connections 2024-05-09T00:02:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2d0c0a1290 0x7f2d0c0a3750 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24105e60 0x7f2d2419bcb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d240ffda0 0x7f2d241973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 --2- 172.21.15.12:0/3006989813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d24100760 0x7f2d24197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 -- 172.21.15.12:0/3006989813 >> 172.21.15.12:0/3006989813 conn(0x7f2d24076f10 msgr2=0x7f2d24068500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 -- 172.21.15.12:0/3006989813 shutdown_connections 2024-05-09T00:02:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:02:55.272+0000 7f2d2888f700 1 -- 172.21.15.12:0/3006989813 wait complete. 2024-05-09T00:02:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:55 smithi012 ceph-mon[42503]: pgmap v530: 97 pgs: 97 active+clean; 982 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.8 MiB/s wr, 479 op/s 2024-05-09T00:02:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:55 smithi204 ceph-mon[47373]: pgmap v530: 97 pgs: 97 active+clean; 982 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.8 MiB/s wr, 479 op/s 2024-05-09T00:02:56.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:55 smithi160 ceph-mon[46152]: pgmap v530: 97 pgs: 97 active+clean; 982 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.8 MiB/s wr, 479 op/s 2024-05-09T00:02:56.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 126574 41.77 MB/sec execute 51 sec latency 154.866 ms 2024-05-09T00:02:57.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 127376 41.85 MB/sec execute 52 sec latency 220.038 ms 2024-05-09T00:02:57.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:57 smithi012 ceph-mon[42503]: pgmap v531: 97 pgs: 97 active+clean; 987 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 724 op/s 2024-05-09T00:02:58.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:57 smithi204 ceph-mon[47373]: pgmap v531: 97 pgs: 97 active+clean; 987 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 724 op/s 2024-05-09T00:02:58.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:57 smithi160 ceph-mon[46152]: pgmap v531: 97 pgs: 97 active+clean; 987 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 724 op/s 2024-05-09T00:02:58.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 127409 41.14 MB/sec execute 53 sec latency 954.003 ms 2024-05-09T00:02:58.613 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:02:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.821+0000 7f951dda3700 1 -- 172.21.15.12:0/3326861277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9518100a10 msgr2=0x7f9518104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.821+0000 7f951dda3700 1 --2- 172.21.15.12:0/3326861277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9518100a10 0x7f9518104a30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f9508009990 tx=0x7f9508009ca0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.821+0000 7f951dda3700 1 -- 172.21.15.12:0/3326861277 shutdown_connections 2024-05-09T00:02:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.821+0000 7f951dda3700 1 --2- 172.21.15.12:0/3326861277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9518105160 0x7f951810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.821+0000 7f951dda3700 1 --2- 172.21.15.12:0/3326861277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9518100060 0x7f9518100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.821+0000 7f951dda3700 1 --2- 172.21.15.12:0/3326861277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9518100a10 0x7f9518104a30 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.821+0000 7f951dda3700 1 -- 172.21.15.12:0/3326861277 >> 172.21.15.12:0/3326861277 conn(0x7f9518076f40 msgr2=0x7f9518077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:58.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.822+0000 7f951dda3700 1 -- 172.21.15.12:0/3326861277 shutdown_connections 2024-05-09T00:02:58.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.822+0000 7f951dda3700 1 -- 172.21.15.12:0/3326861277 wait complete. 2024-05-09T00:02:58.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.822+0000 7f951dda3700 1 Processor -- start 2024-05-09T00:02:58.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f951dda3700 1 -- start start 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f951dda3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9518100060 0x7f9518195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f951dda3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9518105160 0x7f95181957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f951dda3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9518199b40 0x7f9518199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f951dda3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951819a5c0 con 0x7f9518105160 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f951dda3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951819a730 con 0x7f9518199b40 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f951dda3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951819a8f0 con 0x7f9518100060 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f9517fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9518199b40 0x7f9518199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f95177fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9518100060 0x7f9518195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f9517fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9518199b40 0x7f9518199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52860/0 (socket says 172.21.15.12:52860) 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.824+0000 7f9517fff700 1 -- 172.21.15.12:0/3376129391 learned_addr learned my addr 172.21.15.12:0/3376129391 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:58.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.825+0000 7f9516ffd700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9518105160 0x7f95181957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.825+0000 7f9516ffd700 1 -- 172.21.15.12:0/3376129391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9518100060 msgr2=0x7f9518195260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:58.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.825+0000 7f9516ffd700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9518100060 0x7f9518195260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.825+0000 7f9516ffd700 1 -- 172.21.15.12:0/3376129391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9518199b40 msgr2=0x7f9518199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:58.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.825+0000 7f9516ffd700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9518199b40 0x7f9518199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.825+0000 7f9516ffd700 1 -- 172.21.15.12:0/3376129391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95080095f0 con 0x7f9518105160 2024-05-09T00:02:58.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.826+0000 7f9516ffd700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9518105160 0x7f95181957a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f951806a6d0 tx=0x7f9508003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:58.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.827+0000 7f9514ff9700 1 -- 172.21.15.12:0/3376129391 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950801d070 con 0x7f9518105160 2024-05-09T00:02:58.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.827+0000 7f9514ff9700 1 -- 172.21.15.12:0/3376129391 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9508003e90 con 0x7f9518105160 2024-05-09T00:02:58.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.827+0000 7f9514ff9700 1 -- 172.21.15.12:0/3376129391 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950800fb90 con 0x7f9518105160 2024-05-09T00:02:58.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.828+0000 7f951dda3700 1 -- 172.21.15.12:0/3376129391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95181a1df0 con 0x7f9518105160 2024-05-09T00:02:58.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.829+0000 7f951dda3700 1 -- 172.21.15.12:0/3376129391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95181a21c0 con 0x7f9518105160 2024-05-09T00:02:58.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.834+0000 7f9529f5db80 1 Processor -- start 2024-05-09T00:02:58.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.834+0000 7f9529f5db80 1 -- start start 2024-05-09T00:02:58.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.834+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.834+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 0x5625b2e3d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.834+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e3dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.834+0000 7f9529f5db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2d94150 con 0x5625b2e43720 2024-05-09T00:02:58.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.834+0000 7f9529f5db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2d91b10 con 0x5625b2e3d890 2024-05-09T00:02:58.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.834+0000 7f9529f5db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2d8d400 con 0x5625b2e42e00 2024-05-09T00:02:58.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.835+0000 7f9514ff9700 1 -- 172.21.15.12:0/3376129391 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f950800fcf0 con 0x7f9518105160 2024-05-09T00:02:58.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.835+0000 7f95177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 0x5625b2e3d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.835+0000 7f95177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 0x5625b2e3d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47726/0 (socket says 172.21.15.12:47726) 2024-05-09T00:02:58.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.835+0000 7f95177fe700 1 -- 172.21.15.12:0/3682955548 learned_addr learned my addr 172.21.15.12:0/3682955548 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:58.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.835+0000 7f95177fe700 1 -- 172.21.15.12:0/3682955548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 msgr2=0x5625b2e431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:58.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.835+0000 7f95177fe700 1 --2- 172.21.15.12:0/3682955548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.835+0000 7f95177fe700 1 -- 172.21.15.12:0/3682955548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 msgr2=0x5625b2e3dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:58.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.836+0000 7f9516ffd700 1 --2- 172.21.15.12:0/3682955548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e3dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.836+0000 7f95177fe700 1 --2- 172.21.15.12:0/3682955548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e3dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.836+0000 7f95177fe700 1 -- 172.21.15.12:0/3682955548 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b2c0e3b0 con 0x5625b2e43720 2024-05-09T00:02:58.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.836+0000 7f9516ffd700 1 --2- 172.21.15.12:0/3682955548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e3dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:02:58.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.836+0000 7f95177fe700 1 --2- 172.21.15.12:0/3682955548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 0x5625b2e3d350 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f9510011800 tx=0x7f9510011310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a9234563f329b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:58.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.836+0000 7f9514ff9700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95000b23a0 0x7f95000b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.836+0000 7f9514ff9700 1 -- 172.21.15.12:0/3376129391 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f95080cde90 con 0x7f9518105160 2024-05-09T00:02:58.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.837+0000 7f9517fff700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95000b23a0 0x7f95000b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.837+0000 7f9517fff700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95000b23a0 0x7f95000b4860 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f950c0093d0 tx=0x7f950c009360 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:58.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.838+0000 7f94fdffb700 1 -- 172.21.15.12:0/3682955548 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951000f040 con 0x5625b2e43720 2024-05-09T00:02:58.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.838+0000 7f94fdffb700 1 -- 172.21.15.12:0/3682955548 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9510004730 con 0x5625b2e43720 2024-05-09T00:02:58.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.838+0000 7f94fdffb700 1 -- 172.21.15.12:0/3682955548 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100087e0 con 0x5625b2e43720 2024-05-09T00:02:58.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.839+0000 7f9529f5db80 1 -- 172.21.15.12:0/3682955548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 msgr2=0x5625b2e3d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:58.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.839+0000 7f9529f5db80 1 --2- 172.21.15.12:0/3682955548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 0x5625b2e3d350 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f9510011800 tx=0x7f9510011310 comp rx=0 tx=0).stop 2024-05-09T00:02:58.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.839+0000 7f9529f5db80 1 -- 172.21.15.12:0/3682955548 shutdown_connections 2024-05-09T00:02:58.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.839+0000 7f9529f5db80 1 --2- 172.21.15.12:0/3682955548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e3dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.839+0000 7f9529f5db80 1 --2- 172.21.15.12:0/3682955548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.839+0000 7f9529f5db80 1 --2- 172.21.15.12:0/3682955548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 0x5625b2e3d350 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.839+0000 7f9529f5db80 1 -- 172.21.15.12:0/3682955548 >> 172.21.15.12:0/3682955548 conn(0x5625b2d9b210 msgr2=0x5625b2e37df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:58.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.839+0000 7f9529f5db80 1 -- 172.21.15.12:0/3682955548 shutdown_connections 2024-05-09T00:02:58.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.839+0000 7f9529f5db80 1 -- 172.21.15.12:0/3682955548 wait complete. 2024-05-09T00:02:58.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.839+0000 7f9529f5db80 1 -- start start 2024-05-09T00:02:58.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.840+0000 7f9529f5db80 1 Processor -- start 2024-05-09T00:02:58.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.840+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e46b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.840+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.840+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 0x5625b2e475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.840+0000 7f9529f5db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2d29560 con 0x5625b2e43720 2024-05-09T00:02:58.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.840+0000 7f9529f5db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2d29200 con 0x5625b2e3d890 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.840+0000 7f9529f5db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2c106a0 con 0x5625b2e42e00 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f95177fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e46b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f95177fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e46b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52874/0 (socket says 172.21.15.12:52874) 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f95177fe700 1 -- 172.21.15.12:0/4124182102 learned_addr learned my addr 172.21.15.12:0/4124182102 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f9529f5db80 10 client.0 _ll_register_callbacks cb 0x5625b2d32b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f9517fff700 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 0x5625b2e475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f9516ffd700 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f95177fe700 1 -- 172.21.15.12:0/4124182102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 msgr2=0x5625b2e470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f95177fe700 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f95177fe700 1 -- 172.21.15.12:0/4124182102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 msgr2=0x5625b2e475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f95177fe700 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 0x5625b2e475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f95177fe700 1 -- 172.21.15.12:0/4124182102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9510003d50 con 0x5625b2e3d890 2024-05-09T00:02:58.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f9516ffd700 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:02:58.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f95177fe700 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e46b60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9510004010 tx=0x7f951000f3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9f2558b01d64b3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:58.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.841+0000 7f94f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:02:58.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.843+0000 7f94f6ffd700 1 -- 172.21.15.12:0/4124182102 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951000f040 con 0x5625b2e3d890 2024-05-09T00:02:58.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.843+0000 7f94f6ffd700 1 -- 172.21.15.12:0/4124182102 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f951000f5f0 con 0x5625b2e3d890 2024-05-09T00:02:58.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.843+0000 7f9529f5db80 1 -- 172.21.15.12:0/4124182102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625b2ce6270 con 0x5625b2e3d890 2024-05-09T00:02:58.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.846+0000 7f94f6ffd700 1 -- 172.21.15.12:0/4124182102 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9510003bf0 con 0x5625b2e3d890 2024-05-09T00:02:58.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.847+0000 7f9529f5db80 10 client.29708 fetch_fsmap learned FSMap version 69 2024-05-09T00:02:58.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.847+0000 7f9529f5db80 1 -- 172.21.15.12:0/4124182102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x5625b2bbab60 con 0x5625b2e3d890 2024-05-09T00:02:58.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.848+0000 7f94f6ffd700 1 -- 172.21.15.12:0/4124182102 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625b2ce6270 con 0x5625b2e3d890 2024-05-09T00:02:58.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.849+0000 7f94f6ffd700 1 -- 172.21.15.12:0/4124182102 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9510026020 con 0x5625b2e3d890 2024-05-09T00:02:58.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.849+0000 7f9529f5db80 10 client.29708 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:02:58.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.849+0000 7f9529f5db80 10 client.29708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:58.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.849+0000 7f9529f5db80 20 client.29708 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:58.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.849+0000 7f9529f5db80 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b2e570c0 0x5625b2e594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.849+0000 7f9529f5db80 4 client.29708 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:58.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.849+0000 7f9529f5db80 1 -- 172.21.15.12:0/4124182102 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625b2e0c810 con 0x5625b2e570c0 2024-05-09T00:02:58.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.849+0000 7f9516ffd700 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b2e570c0 0x5625b2e594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.850+0000 7f9516ffd700 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b2e570c0 0x5625b2e594e0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78159ea9c69b64b server_cookie=9962a8f152fad330 in_seq=0 out_seq=0 2024-05-09T00:02:58.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.850+0000 7f94f6ffd700 10 client.29708 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:58.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.852+0000 7f94f6ffd700 1 -- 172.21.15.12:0/4124182102 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625b2e0c810 con 0x5625b2e570c0 2024-05-09T00:02:58.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.852+0000 7f94f6ffd700 10 client.29708 handle_command_reply: tid=0 2024-05-09T00:02:58.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.852+0000 7f9529f5db80 1 client.29708 shutdown 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 1 -- 172.21.15.12:0/4124182102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 msgr2=0x5625b2e46b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e46b60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9510004010 tx=0x7f951000f3d0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 1 -- 172.21.15.12:0/4124182102 shutdown_connections 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e43720 0x5625b2e475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b2e570c0 0x5625b2e594e0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e46b60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 1 --2- 172.21.15.12:0/4124182102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f94f6ffd700 0 client.29708 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 1 -- 172.21.15.12:0/4124182102 >> 172.21.15.12:0/4124182102 conn(0x5625b2d9b210 msgr2=0x5625b2e375a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 1 -- 172.21.15.12:0/4124182102 shutdown_connections 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 1 -- 172.21.15.12:0/4124182102 wait complete. 2024-05-09T00:02:58.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.853+0000 7f9529f5db80 20 client.29708 trim_cache size 0 max 16384 2024-05-09T00:02:58.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f9529f5db80 1 Processor -- start 2024-05-09T00:02:58.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f9529f5db80 1 -- start start 2024-05-09T00:02:58.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e3d890 0x5625b2e52c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e37d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e43720 0x5625b2e470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f9529f5db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2e39070 con 0x5625b2e3d890 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f9529f5db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2e53df0 con 0x5625b2e43720 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f9529f5db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2e34dc0 con 0x5625b2e42e00 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f95177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e3d890 0x5625b2e52c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f95177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e3d890 0x5625b2e52c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47740/0 (socket says 172.21.15.12:47740) 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f95177fe700 1 -- 172.21.15.12:0/1896258633 learned_addr learned my addr 172.21.15.12:0/1896258633 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.878+0000 7f9517fff700 1 --2- 172.21.15.12:0/1896258633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e43720 0x5625b2e470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.879+0000 7f95177fe700 1 -- 172.21.15.12:0/1896258633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 msgr2=0x5625b2e37d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.879+0000 7f95177fe700 1 --2- 172.21.15.12:0/1896258633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e37d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.879+0000 7f95177fe700 1 -- 172.21.15.12:0/1896258633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e43720 msgr2=0x5625b2e470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.879+0000 7f95177fe700 1 --2- 172.21.15.12:0/1896258633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e43720 0x5625b2e470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.879+0000 7f95177fe700 1 -- 172.21.15.12:0/1896258633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b2d53560 con 0x5625b2e3d890 2024-05-09T00:02:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.879+0000 7f95177fe700 1 --2- 172.21.15.12:0/1896258633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e3d890 0x5625b2e52c20 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f9510004390 tx=0x7f9510011530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8137c2ea62de65ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.879+0000 7f94fe7fc700 1 -- 172.21.15.12:0/1896258633 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510006000 con 0x5625b2e3d890 2024-05-09T00:02:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.879+0000 7f94fe7fc700 1 -- 172.21.15.12:0/1896258633 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9510015e80 con 0x5625b2e3d890 2024-05-09T00:02:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.879+0000 7f94fe7fc700 1 -- 172.21.15.12:0/1896258633 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100048a0 con 0x5625b2e3d890 2024-05-09T00:02:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.880+0000 7f9529f5db80 1 -- 172.21.15.12:0/1896258633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e3d890 msgr2=0x5625b2e52c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.880+0000 7f9529f5db80 1 --2- 172.21.15.12:0/1896258633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e3d890 0x5625b2e52c20 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f9510004390 tx=0x7f9510011530 comp rx=0 tx=0).stop 2024-05-09T00:02:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.880+0000 7f9529f5db80 1 -- 172.21.15.12:0/1896258633 shutdown_connections 2024-05-09T00:02:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.880+0000 7f9529f5db80 1 --2- 172.21.15.12:0/1896258633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e43720 0x5625b2e470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.880+0000 7f9529f5db80 1 --2- 172.21.15.12:0/1896258633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e3d890 0x5625b2e52c20 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.880+0000 7f9529f5db80 1 --2- 172.21.15.12:0/1896258633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e42e00 0x5625b2e37d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.880+0000 7f9529f5db80 1 -- 172.21.15.12:0/1896258633 >> 172.21.15.12:0/1896258633 conn(0x5625b2d9b210 msgr2=0x5625b2e34210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.881+0000 7f9529f5db80 1 -- 172.21.15.12:0/1896258633 shutdown_connections 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.881+0000 7f9529f5db80 1 -- 172.21.15.12:0/1896258633 wait complete. 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.881+0000 7f9529f5db80 1 -- start start 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.881+0000 7f9529f5db80 1 Processor -- start 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e4e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e42e00 0x5625b2e4c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f9529f5db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e43720 0x5625b2e4c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f9529f5db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2e4cf60 con 0x5625b2e42e00 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f9529f5db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2eac5c0 con 0x5625b2e3d890 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f9529f5db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b2eac700 con 0x5625b2e43720 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f9529f5db80 10 client.0 _ll_register_callbacks cb 0x5625b2d32b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f95177fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e4e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f95177fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e4e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52892/0 (socket says 172.21.15.12:52892) 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f95177fe700 1 -- 172.21.15.12:0/3907231255 learned_addr learned my addr 172.21.15.12:0/3907231255 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f9516ffd700 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e42e00 0x5625b2e4c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f95177fe700 1 -- 172.21.15.12:0/3907231255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e43720 msgr2=0x5625b2e4c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f95177fe700 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e43720 0x5625b2e4c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f95177fe700 1 -- 172.21.15.12:0/3907231255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e42e00 msgr2=0x5625b2e4c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f95177fe700 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e42e00 0x5625b2e4c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f95177fe700 1 -- 172.21.15.12:0/3907231255 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625b2bbab60 con 0x5625b2e3d890 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f95177fe700 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e4e690 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9510003b10 tx=0x7f9510004e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=976aa6a5581d4333 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.882+0000 7f94f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.883+0000 7f94f6ffd700 1 -- 172.21.15.12:0/3907231255 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510006000 con 0x5625b2e3d890 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.883+0000 7f94f6ffd700 1 -- 172.21.15.12:0/3907231255 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9510015760 con 0x5625b2e3d890 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.883+0000 7f9529f5db80 1 -- 172.21.15.12:0/3907231255 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625b2d85ec0 con 0x5625b2e3d890 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.883+0000 7f94f6ffd700 1 -- 172.21.15.12:0/3907231255 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9510003bf0 con 0x5625b2e3d890 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.883+0000 7f9529f5db80 10 client.29714 fetch_fsmap learned FSMap version 69 2024-05-09T00:02:58.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.883+0000 7f9529f5db80 1 -- 172.21.15.12:0/3907231255 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x5625b2c12cd0 con 0x5625b2e3d890 2024-05-09T00:02:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.883+0000 7f94f6ffd700 1 -- 172.21.15.12:0/3907231255 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625b2d85ec0 con 0x5625b2e3d890 2024-05-09T00:02:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.883+0000 7f94f6ffd700 1 -- 172.21.15.12:0/3907231255 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9510038020 con 0x5625b2e3d890 2024-05-09T00:02:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.884+0000 7f9529f5db80 10 client.29714 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:02:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.884+0000 7f9529f5db80 10 client.29714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:02:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.884+0000 7f9529f5db80 20 client.29714 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.884+0000 7f9529f5db80 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b2eada00 0x5625b2eafe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.884+0000 7f9529f5db80 4 client.29714 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:02:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.884+0000 7f9529f5db80 1 -- 172.21.15.12:0/3907231255 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5625b2bdbcf0 con 0x5625b2eada00 2024-05-09T00:02:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.884+0000 7f9516ffd700 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b2eada00 0x5625b2eafe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:58.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.885+0000 7f9516ffd700 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b2eada00 0x5625b2eafe20 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9e79f5803818771 server_cookie=a92eec8e6aa0076c in_seq=0 out_seq=0 2024-05-09T00:02:58.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.885+0000 7f94f6ffd700 10 client.29714 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:58.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f94f6ffd700 1 -- 172.21.15.12:0/3907231255 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5625b2bdbcf0 con 0x5625b2eada00 2024-05-09T00:02:58.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f94f6ffd700 10 client.29714 handle_command_reply: tid=0 2024-05-09T00:02:58.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f9529f5db80 1 client.29714 shutdown 2024-05-09T00:02:58.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f9529f5db80 1 -- 172.21.15.12:0/3907231255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 msgr2=0x5625b2e4e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:58.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f9529f5db80 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e4e690 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9510003b10 tx=0x7f9510004e20 comp rx=0 tx=0).stop 2024-05-09T00:02:58.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f9529f5db80 1 -- 172.21.15.12:0/3907231255 shutdown_connections 2024-05-09T00:02:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f9529f5db80 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b2e43720 0x5625b2e4c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f9529f5db80 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5625b2eada00 0x5625b2eafe20 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f9529f5db80 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625b2e3d890 0x5625b2e4e690 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f9529f5db80 1 --2- 172.21.15.12:0/3907231255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625b2e42e00 0x5625b2e4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f94f6ffd700 0 client.29714 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:02:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.889+0000 7f9529f5db80 1 -- 172.21.15.12:0/3907231255 >> 172.21.15.12:0/3907231255 conn(0x5625b2d9b210 msgr2=0x5625b2e453d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f9529f5db80 1 -- 172.21.15.12:0/3907231255 shutdown_connections 2024-05-09T00:02:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f9529f5db80 1 -- 172.21.15.12:0/3907231255 wait complete. 2024-05-09T00:02:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f9529f5db80 20 client.29714 trim_cache size 0 max 16384 2024-05-09T00:02:58.886 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:02:58.886 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:02:58.886 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:02:58.886 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:02:58.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 -- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95000b23a0 msgr2=0x7f95000b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:58.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95000b23a0 0x7f95000b4860 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f950c0093d0 tx=0x7f950c009360 comp rx=0 tx=0).stop 2024-05-09T00:02:58.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 -- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9518105160 msgr2=0x7f95181957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:58.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9518105160 0x7f95181957a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f951806a6d0 tx=0x7f9508003a90 comp rx=0 tx=0).stop 2024-05-09T00:02:58.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 -- 172.21.15.12:0/3376129391 shutdown_connections 2024-05-09T00:02:58.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95000b23a0 0x7f95000b4860 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9518199b40 0x7f9518199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9518100060 0x7f9518195260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 --2- 172.21.15.12:0/3376129391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9518105160 0x7f95181957a0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:58.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 -- 172.21.15.12:0/3376129391 >> 172.21.15.12:0/3376129391 conn(0x7f9518076f40 msgr2=0x7f95180fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:58.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.890+0000 7f951dda3700 1 -- 172.21.15.12:0/3376129391 shutdown_connections 2024-05-09T00:02:58.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:58.891+0000 7f951dda3700 1 -- 172.21.15.12:0/3376129391 wait complete. 2024-05-09T00:02:58.906 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:02:58.906 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:02:59.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.126+0000 7fb3478f3700 1 -- 172.21.15.12:0/3931598156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb340101a10 msgr2=0x7fb340105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.126+0000 7fb3478f3700 1 --2- 172.21.15.12:0/3931598156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb340101a10 0x7fb340105a50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb334009990 tx=0x7fb334009ca0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.126+0000 7fb3478f3700 1 -- 172.21.15.12:0/3931598156 shutdown_connections 2024-05-09T00:02:59.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.126+0000 7fb3478f3700 1 --2- 172.21.15.12:0/3931598156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb340106180 0x7fb34010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.126+0000 7fb3478f3700 1 --2- 172.21.15.12:0/3931598156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb340101060 0x7fb340101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.126+0000 7fb3478f3700 1 --2- 172.21.15.12:0/3931598156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb340101a10 0x7fb340105a50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.126+0000 7fb3478f3700 1 -- 172.21.15.12:0/3931598156 >> 172.21.15.12:0/3931598156 conn(0x7fb340078360 msgr2=0x7fb340078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:59.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.126+0000 7fb3478f3700 1 -- 172.21.15.12:0/3931598156 shutdown_connections 2024-05-09T00:02:59.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.126+0000 7fb3478f3700 1 -- 172.21.15.12:0/3931598156 wait complete. 2024-05-09T00:02:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.127+0000 7fb3478f3700 1 Processor -- start 2024-05-09T00:02:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.127+0000 7fb3478f3700 1 -- start start 2024-05-09T00:02:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.127+0000 7fb3478f3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb340101060 0x7fb340197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.127+0000 7fb3478f3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb340106180 0x7fb340197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.127+0000 7fb3478f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb34019bd30 0x7fb34019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.127+0000 7fb3478f3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019c7b0 con 0x7fb34019bd30 2024-05-09T00:02:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.127+0000 7fb3478f3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019c8f0 con 0x7fb340106180 2024-05-09T00:02:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.127+0000 7fb3478f3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019ca60 con 0x7fb340101060 2024-05-09T00:02:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.128+0000 7fb344e8e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb340106180 0x7fb340197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.128+0000 7fb344e8e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb340106180 0x7fb340197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52914/0 (socket says 172.21.15.12:52914) 2024-05-09T00:02:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.128+0000 7fb344e8e700 1 -- 172.21.15.12:0/775104011 learned_addr learned my addr 172.21.15.12:0/775104011 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:59.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.129+0000 7fb345e90700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb34019bd30 0x7fb34019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.129+0000 7fb344e8e700 1 -- 172.21.15.12:0/775104011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb340101060 msgr2=0x7fb340197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.129+0000 7fb344e8e700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb340101060 0x7fb340197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.129+0000 7fb344e8e700 1 -- 172.21.15.12:0/775104011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb34019bd30 msgr2=0x7fb34019c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.129+0000 7fb344e8e700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb34019bd30 0x7fb34019c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.129+0000 7fb344e8e700 1 -- 172.21.15.12:0/775104011 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340095f0 con 0x7fb340106180 2024-05-09T00:02:59.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.132+0000 7fb344e8e700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb340106180 0x7fb340197990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb334003b60 tx=0x7fb334003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:59.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.133+0000 7fb33e7fc700 1 -- 172.21.15.12:0/775104011 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33401d070 con 0x7fb340106180 2024-05-09T00:02:59.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.133+0000 7fb33e7fc700 1 -- 172.21.15.12:0/775104011 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb334003e10 con 0x7fb340106180 2024-05-09T00:02:59.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.133+0000 7fb33e7fc700 1 -- 172.21.15.12:0/775104011 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33400fe20 con 0x7fb340106180 2024-05-09T00:02:59.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.133+0000 7fb3478f3700 1 -- 172.21.15.12:0/775104011 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34006a9c0 con 0x7fb340106180 2024-05-09T00:02:59.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.133+0000 7fb3478f3700 1 -- 172.21.15.12:0/775104011 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34006ae20 con 0x7fb340106180 2024-05-09T00:02:59.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.135+0000 7fb3478f3700 1 -- 172.21.15.12:0/775104011 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c0052f0 con 0x7fb340106180 2024-05-09T00:02:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.139+0000 7fb33e7fc700 1 -- 172.21.15.12:0/775104011 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb33400f460 con 0x7fb340106180 2024-05-09T00:02:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.139+0000 7fb33e7fc700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3280a58b0 0x7fb3280a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.140+0000 7fb33e7fc700 1 -- 172.21.15.12:0/775104011 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb3340cdee0 con 0x7fb340106180 2024-05-09T00:02:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.140+0000 7fb34568f700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3280a58b0 0x7fb3280a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.142+0000 7fb34568f700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3280a58b0 0x7fb3280a7d70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb338009770 tx=0x7fb338006cd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:59.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.144+0000 7fb33e7fc700 1 -- 172.21.15.12:0/775104011 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb334097420 con 0x7fb340106180 2024-05-09T00:02:59.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 127618 40.57 MB/sec execute 54 sec latency 557.306 ms 2024-05-09T00:02:59.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.541+0000 7fb3478f3700 1 -- 172.21.15.12:0/775104011 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb32c005870 con 0x7fb340106180 2024-05-09T00:02:59.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.543+0000 7fb33e7fc700 1 -- 172.21.15.12:0/775104011 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+10877 (secure 0 0 0) 0x7fb33409aa40 con 0x7fb340106180 2024-05-09T00:02:59.538 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:02:59.538 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:02:58.668216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6837/4284773137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":4284773137},{"type":"v1","addr":"172.21.15.12:6837","nonce":4284773137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:02:59.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 -- 172.21.15.12:0/775104011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3280a58b0 msgr2=0x7fb3280a7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3280a58b0 0x7fb3280a7d70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb338009770 tx=0x7fb338006cd0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 -- 172.21.15.12:0/775104011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb340106180 msgr2=0x7fb340197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb340106180 0x7fb340197990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb334003b60 tx=0x7fb334003c40 comp rx=0 tx=0).stop 2024-05-09T00:02:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 -- 172.21.15.12:0/775104011 shutdown_connections 2024-05-09T00:02:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb3280a58b0 0x7fb3280a7d70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb34019bd30 0x7fb34019c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb340101060 0x7fb340197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 --2- 172.21.15.12:0/775104011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb340106180 0x7fb340197990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 -- 172.21.15.12:0/775104011 >> 172.21.15.12:0/775104011 conn(0x7fb340078360 msgr2=0x7fb3400ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:59.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 -- 172.21.15.12:0/775104011 shutdown_connections 2024-05-09T00:02:59.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.548+0000 7fb3478f3700 1 -- 172.21.15.12:0/775104011 wait complete. 2024-05-09T00:02:59.544 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 69 2024-05-09T00:02:59.559 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:02:59.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.800+0000 7f8b91636700 1 -- 172.21.15.12:0/3960042859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c0ff840 msgr2=0x7f8b8c0ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.800+0000 7f8b91636700 1 --2- 172.21.15.12:0/3960042859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c0ff840 0x7f8b8c0ffc20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009990 tx=0x7f8b7c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.803+0000 7f8b91636700 1 -- 172.21.15.12:0/3960042859 shutdown_connections 2024-05-09T00:02:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.803+0000 7f8b91636700 1 --2- 172.21.15.12:0/3960042859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b8c111cd0 0x7f8b8c1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.803+0000 7f8b91636700 1 --2- 172.21.15.12:0/3960042859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c0ff840 0x7f8b8c0ffc20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.803+0000 7f8b91636700 1 --2- 172.21.15.12:0/3960042859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b8c100160 0x7f8b8c111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.803+0000 7f8b91636700 1 -- 172.21.15.12:0/3960042859 >> 172.21.15.12:0/3960042859 conn(0x7f8b8c075120 msgr2=0x7f8b8c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:59.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.804+0000 7f8b91636700 1 -- 172.21.15.12:0/3960042859 shutdown_connections 2024-05-09T00:02:59.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.807+0000 7f8b91636700 1 -- 172.21.15.12:0/3960042859 wait complete. 2024-05-09T00:02:59.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.808+0000 7f8b91636700 1 Processor -- start 2024-05-09T00:02:59.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.808+0000 7f8b91636700 1 -- start start 2024-05-09T00:02:59.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.808+0000 7f8b91636700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b8c0ff840 0x7f8b8c072450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.808+0000 7f8b91636700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b8c100160 0x7f8b8c072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.808+0000 7f8b91636700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c111cd0 0x7f8b8c06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.808+0000 7f8b91636700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c06dd50 con 0x7f8b8c100160 2024-05-09T00:02:59.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.808+0000 7f8b91636700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c06dec0 con 0x7f8b8c0ff840 2024-05-09T00:02:59.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.808+0000 7f8b91636700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c06e080 con 0x7f8b8c111cd0 2024-05-09T00:02:59.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.809+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b8c100160 0x7f8b8c072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.809+0000 7f8b8b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c111cd0 0x7f8b8c06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.809+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b8c100160 0x7f8b8c072990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47790/0 (socket says 172.21.15.12:47790) 2024-05-09T00:02:59.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.809+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/2820873683 learned_addr learned my addr 172.21.15.12:0/2820873683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:59.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.809+0000 7f8b8affd700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b8c0ff840 0x7f8b8c072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.809+0000 7f8b8b7fe700 1 -- 172.21.15.12:0/2820873683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b8c0ff840 msgr2=0x7f8b8c072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.809+0000 7f8b8b7fe700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b8c0ff840 0x7f8b8c072450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.809+0000 7f8b8b7fe700 1 -- 172.21.15.12:0/2820873683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b8c100160 msgr2=0x7f8b8c072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.809+0000 7f8b8b7fe700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b8c100160 0x7f8b8c072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.809+0000 7f8b8b7fe700 1 -- 172.21.15.12:0/2820873683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0095f0 con 0x7f8b8c111cd0 2024-05-09T00:02:59.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.810+0000 7f8b8b7fe700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c111cd0 0x7f8b8c06d5b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400bdf0 tx=0x7f8b84009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.811+0000 7f8b73fff700 1 -- 172.21.15.12:0/2820873683 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400cb30 con 0x7f8b8c111cd0 2024-05-09T00:02:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.811+0000 7f8b73fff700 1 -- 172.21.15.12:0/2820873683 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8400cc90 con 0x7f8b8c111cd0 2024-05-09T00:02:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.811+0000 7f8b73fff700 1 -- 172.21.15.12:0/2820873683 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b840125e0 con 0x7f8b8c111cd0 2024-05-09T00:02:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.811+0000 7f8b91636700 1 -- 172.21.15.12:0/2820873683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c06e360 con 0x7f8b8c111cd0 2024-05-09T00:02:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.811+0000 7f8b91636700 1 -- 172.21.15.12:0/2820873683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c1b3040 con 0x7f8b8c111cd0 2024-05-09T00:02:59.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.817+0000 7f8b9d7f0b80 1 Processor -- start 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.818+0000 7f8b9d7f0b80 1 -- start start 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.818+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699de00 0x55a38699e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.818+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699e720 0x55a386998350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.818+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a386998890 0x55a386998cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.818+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3868ef150 con 0x55a38699e720 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.818+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3868ecb10 con 0x55a386998890 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.818+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3868e8400 con 0x55a38699de00 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.819+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699de00 0x55a38699e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.819+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699de00 0x55a38699e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39640/0 (socket says 172.21.15.12:39640) 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.819+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/338764953 learned_addr learned my addr 172.21.15.12:0/338764953 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.819+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/338764953 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a386998890 0x55a386998cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.819+0000 7f8b8affd700 1 --2- 172.21.15.12:0/338764953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699e720 0x55a386998350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.820+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/338764953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699de00 msgr2=0x55a38699e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.820+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/338764953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699de00 0x55a38699e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.820+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/338764953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699e720 msgr2=0x55a386998350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.820+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/338764953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699e720 0x55a386998350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.820+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/338764953 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3867693b0 con 0x55a386998890 2024-05-09T00:02:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.821+0000 7f8b73fff700 1 -- 172.21.15.12:0/2820873683 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8b84014440 con 0x7f8b8c111cd0 2024-05-09T00:02:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.822+0000 7f8b73fff700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b740a1460 0x7f8b740a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.822+0000 7f8b73fff700 1 -- 172.21.15.12:0/2820873683 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8b84006e40 con 0x7f8b8c111cd0 2024-05-09T00:02:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.822+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/338764953 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a386998890 0x55a386998cd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b80006e10 tx=0x7f8b80009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=349a8a0d495189a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.825+0000 7f8b717fa700 1 -- 172.21.15.12:0/338764953 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80017e90 con 0x55a386998890 2024-05-09T00:02:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.825+0000 7f8b717fa700 1 -- 172.21.15.12:0/338764953 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b80016aa0 con 0x55a386998890 2024-05-09T00:02:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.825+0000 7f8b717fa700 1 -- 172.21.15.12:0/338764953 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8003aa50 con 0x55a386998890 2024-05-09T00:02:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.825+0000 7f8b8affd700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b740a1460 0x7f8b740a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.827+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/338764953 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a386998890 msgr2=0x55a386998cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.827+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/338764953 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a386998890 0x55a386998cd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b80006e10 tx=0x7f8b80009290 comp rx=0 tx=0).stop 2024-05-09T00:02:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.827+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/338764953 shutdown_connections 2024-05-09T00:02:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.827+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/338764953 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a386998890 0x55a386998cd0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.827+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/338764953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699de00 0x55a38699e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.827+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/338764953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699e720 0x55a386998350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.827+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/338764953 >> 172.21.15.12:0/338764953 conn(0x55a3868f6210 msgr2=0x55a386992df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:59.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.828+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/338764953 shutdown_connections 2024-05-09T00:02:59.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.828+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/338764953 wait complete. 2024-05-09T00:02:59.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.828+0000 7f8b9d7f0b80 1 -- start start 2024-05-09T00:02:59.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.828+0000 7f8b9d7f0b80 1 Processor -- start 2024-05-09T00:02:59.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.828+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a386998890 0x55a3869a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.828+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699de00 0x55a3869a2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.828+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699e720 0x55a3869a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a386884560 con 0x55a38699de00 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a386884200 con 0x55a386998890 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a38676b6a0 con 0x55a38699e720 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b9d7f0b80 10 client.0 _ll_register_callbacks cb 0x55a38688db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b8affd700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b740a1460 0x7f8b740a3920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55a386994bd0 tx=0x7f8b7c00b580 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699de00 0x55a3869a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699de00 0x55a3869a2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47802/0 (socket says 172.21.15.12:47802) 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/3515604897 learned_addr learned my addr 172.21.15.12:0/3515604897 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/3515604897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699e720 msgr2=0x55a3869a25c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699e720 0x55a3869a25c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/3515604897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a386998890 msgr2=0x55a3869a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a386998890 0x55a3869a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.829+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/3515604897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b800174c0 con 0x55a38699de00 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.830+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699de00 0x55a3869a2080 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8b80017490 tx=0x7f8b80016810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5027d3ac8cd5ce2e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:59.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.830+0000 7f8b6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:59.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.833+0000 7f8b6eff5700 1 -- 172.21.15.12:0/3515604897 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8003aae0 con 0x55a38699de00 2024-05-09T00:02:59.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.833+0000 7f8b6eff5700 1 -- 172.21.15.12:0/3515604897 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8b80000e40 con 0x55a38699de00 2024-05-09T00:02:59.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.833+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3515604897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a386841270 con 0x55a38699de00 2024-05-09T00:02:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.835+0000 7f8b6eff5700 1 -- 172.21.15.12:0/3515604897 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b80017e90 con 0x55a38699de00 2024-05-09T00:02:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.836+0000 7f8b9d7f0b80 10 client.20079 fetch_fsmap learned FSMap version 69 2024-05-09T00:02:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.836+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3515604897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55a386715b60 con 0x55a38699de00 2024-05-09T00:02:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.837+0000 7f8b6eff5700 1 -- 172.21.15.12:0/3515604897 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a386841270 con 0x55a38699de00 2024-05-09T00:02:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.838+0000 7f8b6eff5700 1 -- 172.21.15.12:0/3515604897 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b80049020 con 0x55a38699de00 2024-05-09T00:02:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.838+0000 7f8b9d7f0b80 10 client.20079 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:02:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.838+0000 7f8b9d7f0b80 10 client.20079 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:02:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.838+0000 7f8b9d7f0b80 20 client.20079 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.838+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55a3869a7ea0 0x55a3869aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.838+0000 7f8b9d7f0b80 4 client.20079 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:02:59.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.838+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3515604897 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a386967810 con 0x55a3869a7ea0 2024-05-09T00:02:59.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.838+0000 7f8b8affd700 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55a3869a7ea0 0x55a3869aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.844+0000 7f8b8affd700 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55a3869a7ea0 0x55a3869aa2c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88e659e272973af1 server_cookie=5a475a949bfbd146 in_seq=0 out_seq=0 2024-05-09T00:02:59.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b6eff5700 10 client.20079 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:02:59.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b6eff5700 1 -- 172.21.15.12:0/3515604897 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a386967810 con 0x55a3869a7ea0 2024-05-09T00:02:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b6eff5700 10 client.20079 handle_command_reply: tid=0 2024-05-09T00:02:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 client.20079 shutdown 2024-05-09T00:02:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3515604897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699de00 msgr2=0x55a3869a2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699de00 0x55a3869a2080 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8b80017490 tx=0x7f8b80016810 comp rx=0 tx=0).stop 2024-05-09T00:02:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3515604897 shutdown_connections 2024-05-09T00:02:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55a3869a7ea0 0x55a3869aa2c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699e720 0x55a3869a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a386998890 0x55a3869a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/3515604897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a38699de00 0x55a3869a2080 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3515604897 >> 172.21.15.12:0/3515604897 conn(0x55a3868f6210 msgr2=0x55a3869925a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:59.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b6eff5700 0 client.20079 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:02:59.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3515604897 shutdown_connections 2024-05-09T00:02:59.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.846+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3515604897 wait complete. 2024-05-09T00:02:59.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.850+0000 7f8b9d7f0b80 20 client.20079 trim_cache size 0 max 16384 2024-05-09T00:02:59.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.875+0000 7f8b9d7f0b80 1 Processor -- start 2024-05-09T00:02:59.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.876+0000 7f8b9d7f0b80 1 -- start start 2024-05-09T00:02:59.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.876+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 0x55a386998350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.876+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699de00 0x55a386992d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.876+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699e720 0x55a3869a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.876+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3869a3140 con 0x55a386998890 2024-05-09T00:02:59.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.876+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a386994a30 con 0x55a38699de00 2024-05-09T00:02:59.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.876+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a38698d800 con 0x55a38699e720 2024-05-09T00:02:59.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.876+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699de00 0x55a386992d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.876+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699de00 0x55a386992d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52960/0 (socket says 172.21.15.12:52960) 2024-05-09T00:02:59.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.876+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/3476527665 learned_addr learned my addr 172.21.15.12:0/3476527665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.877+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/3476527665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699e720 msgr2=0x55a3869a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.877+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/3476527665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699e720 0x55a3869a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.877+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/3476527665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 msgr2=0x55a386998350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.877+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/3476527665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 0x55a386998350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.877+0000 7f8b8a7fc700 1 -- 172.21.15.12:0/3476527665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3868ae560 con 0x55a38699de00 2024-05-09T00:02:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.877+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/3476527665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699de00 0x55a386992d20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8b800048a0 tx=0x7f8b80000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7f21fe6926da17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.878+0000 7f8b71ffb700 1 -- 172.21.15.12:0/3476527665 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80016930 con 0x55a38699de00 2024-05-09T00:02:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.878+0000 7f8b71ffb700 1 -- 172.21.15.12:0/3476527665 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b80017e90 con 0x55a38699de00 2024-05-09T00:02:59.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.878+0000 7f8b71ffb700 1 -- 172.21.15.12:0/3476527665 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b800156c0 con 0x55a38699de00 2024-05-09T00:02:59.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.879+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3476527665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699de00 msgr2=0x55a386992d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.879+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/3476527665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699de00 0x55a386992d20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8b800048a0 tx=0x7f8b80000bc0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.881+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3476527665 shutdown_connections 2024-05-09T00:02:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.881+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/3476527665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699e720 0x55a3869a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.881+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/3476527665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 0x55a386998350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.881+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/3476527665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699de00 0x55a386992d20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.881+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3476527665 >> 172.21.15.12:0/3476527665 conn(0x55a3868f6210 msgr2=0x55a3869a0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:59.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.882+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3476527665 shutdown_connections 2024-05-09T00:02:59.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.893+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/3476527665 wait complete. 2024-05-09T00:02:59.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.893+0000 7f8b9d7f0b80 1 -- start start 2024-05-09T00:02:59.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.893+0000 7f8b9d7f0b80 1 Processor -- start 2024-05-09T00:02:59.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.900+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 0x55a3869b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.900+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699de00 0x55a3869b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.900+0000 7f8b9d7f0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699e720 0x55a3869b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.900+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a386a29530 con 0x55a386998890 2024-05-09T00:02:59.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.900+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a386a29670 con 0x55a38699e720 2024-05-09T00:02:59.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.900+0000 7f8b9d7f0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a386a29830 con 0x55a38699de00 2024-05-09T00:02:59.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.900+0000 7f8b9d7f0b80 10 client.0 _ll_register_callbacks cb 0x55a38688db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:02:59.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.900+0000 7f8b8affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 0x55a3869b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.900+0000 7f8b8affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 0x55a3869b7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47820/0 (socket says 172.21.15.12:47820) 2024-05-09T00:02:59.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.900+0000 7f8b8affd700 1 -- 172.21.15.12:0/2728866735 learned_addr learned my addr 172.21.15.12:0/2728866735 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:02:59.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.901+0000 7f8b8affd700 1 -- 172.21.15.12:0/2728866735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699de00 msgr2=0x55a3869b78d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:02:59.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.901+0000 7f8b8affd700 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699de00 0x55a3869b78d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.901+0000 7f8b8affd700 1 -- 172.21.15.12:0/2728866735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699e720 msgr2=0x55a3869b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:02:59.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.901+0000 7f8b8affd700 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699e720 0x55a3869b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.901+0000 7f8b8affd700 1 -- 172.21.15.12:0/2728866735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a386715b60 con 0x55a386998890 2024-05-09T00:02:59.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.901+0000 7f8b8affd700 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 0x55a3869b7390 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00c010 tx=0x7f8b7c003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de1d02370c91789 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:59.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.902+0000 7f8b6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:02:59.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.902+0000 7f8b6eff5700 1 -- 172.21.15.12:0/2728866735 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c030770 con 0x55a386998890 2024-05-09T00:02:59.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.902+0000 7f8b6eff5700 1 -- 172.21.15.12:0/2728866735 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8b7c0308d0 con 0x55a386998890 2024-05-09T00:02:59.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.902+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/2728866735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b80016aa0 con 0x55a386998890 2024-05-09T00:02:59.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.903+0000 7f8b6eff5700 1 -- 172.21.15.12:0/2728866735 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b7c0050c0 con 0x55a386998890 2024-05-09T00:02:59.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.903+0000 7f8b9d7f0b80 10 client.20082 fetch_fsmap learned FSMap version 69 2024-05-09T00:02:59.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.903+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/2728866735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55a38676dcd0 con 0x55a386998890 2024-05-09T00:02:59.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.904+0000 7f8b6eff5700 1 -- 172.21.15.12:0/2728866735 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b80016aa0 con 0x55a386998890 2024-05-09T00:02:59.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.904+0000 7f8b6eff5700 1 -- 172.21.15.12:0/2728866735 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b7c038070 con 0x55a386998890 2024-05-09T00:02:59.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.905+0000 7f8b9d7f0b80 10 client.20082 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:02:59.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.905+0000 7f8b9d7f0b80 10 client.20082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:02:59.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.905+0000 7f8b9d7f0b80 20 client.20082 populate_metadata read hostname 'smithi012' 2024-05-09T00:02:59.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.905+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55a386a2afb0 0x55a386a2d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:59.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.905+0000 7f8b9d7f0b80 4 client.20082 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:02:59.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.905+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/2728866735 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a3868e0ec0 con 0x55a386a2afb0 2024-05-09T00:02:59.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.910+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55a386a2afb0 0x55a386a2d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.910+0000 7f8b8a7fc700 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55a386a2afb0 0x55a386a2d3f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ca9cc438c9df1d7 server_cookie=9d814dd83238b730 in_seq=0 out_seq=0 2024-05-09T00:02:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.910+0000 7f8b6eff5700 10 client.20082 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:02:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.911+0000 7f8b6eff5700 1 -- 172.21.15.12:0/2728866735 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a3868e0ec0 con 0x55a386a2afb0 2024-05-09T00:02:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.911+0000 7f8b6eff5700 10 client.20082 handle_command_reply: tid=0 2024-05-09T00:02:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.911+0000 7f8b9d7f0b80 1 client.20082 shutdown 2024-05-09T00:02:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.912+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/2728866735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 msgr2=0x55a3869b7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.912+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 0x55a3869b7390 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00c010 tx=0x7f8b7c003b40 comp rx=0 tx=0).stop 2024-05-09T00:02:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.912+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/2728866735 shutdown_connections 2024-05-09T00:02:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.912+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a38699e720 0x55a3869b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.912+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a386998890 0x55a3869b7390 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.912+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55a386a2afb0 0x55a386a2d3f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.912+0000 7f8b9d7f0b80 1 --2- 172.21.15.12:0/2728866735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a38699de00 0x55a3869b78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.912+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/2728866735 >> 172.21.15.12:0/2728866735 conn(0x55a3868f6210 msgr2=0x55a38699cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:59.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.912+0000 7f8b6eff5700 0 client.20082 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:02:59.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.915+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/2728866735 shutdown_connections 2024-05-09T00:02:59.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.917+0000 7f8b9d7f0b80 1 -- 172.21.15.12:0/2728866735 wait complete. 2024-05-09T00:02:59.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.917+0000 7f8b9d7f0b80 20 client.20082 trim_cache size 0 max 16384 2024-05-09T00:02:59.913 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:02:59.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:59 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:02:59.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:59 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/775104011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:02:59.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:02:59 smithi012 ceph-mon[42503]: pgmap v532: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 690 op/s 2024-05-09T00:02:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.925+0000 7f8b91636700 1 -- 172.21.15.12:0/2820873683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b740a1460 msgr2=0x7f8b740a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.925+0000 7f8b91636700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b740a1460 0x7f8b740a3920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55a386994bd0 tx=0x7f8b7c00b580 comp rx=0 tx=0).stop 2024-05-09T00:02:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.925+0000 7f8b91636700 1 -- 172.21.15.12:0/2820873683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c111cd0 msgr2=0x7f8b8c06d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.925+0000 7f8b91636700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c111cd0 0x7f8b8c06d5b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400bdf0 tx=0x7f8b84009ed0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.927+0000 7f8b91636700 1 -- 172.21.15.12:0/2820873683 shutdown_connections 2024-05-09T00:02:59.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.927+0000 7f8b91636700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b740a1460 0x7f8b740a3920 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.927+0000 7f8b91636700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c111cd0 0x7f8b8c06d5b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.927+0000 7f8b91636700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b8c0ff840 0x7f8b8c072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.927+0000 7f8b91636700 1 --2- 172.21.15.12:0/2820873683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b8c100160 0x7f8b8c072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:59.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.927+0000 7f8b91636700 1 -- 172.21.15.12:0/2820873683 >> 172.21.15.12:0/2820873683 conn(0x7f8b8c075120 msgr2=0x7f8b8c10f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:59.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.929+0000 7f8b91636700 1 -- 172.21.15.12:0/2820873683 shutdown_connections 2024-05-09T00:02:59.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:02:59.929+0000 7f8b91636700 1 -- 172.21.15.12:0/2820873683 wait complete. 2024-05-09T00:02:59.946 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:03:00.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:59 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:00.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:59 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/775104011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:03:00.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:02:59 smithi204 ceph-mon[47373]: pgmap v532: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 690 op/s 2024-05-09T00:03:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:59 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:00.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:59 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/775104011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:03:00.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:02:59 smithi160 ceph-mon[46152]: pgmap v532: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 690 op/s 2024-05-09T00:03:00.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.192+0000 7f0129f33700 1 -- 172.21.15.12:0/2110575331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f012411a780 msgr2=0x7f012411cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.192+0000 7f0129f33700 1 --2- 172.21.15.12:0/2110575331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f012411a780 0x7f012411cb70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f011401a9a0 tx=0x7f011401acb0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.192+0000 7f0129f33700 1 -- 172.21.15.12:0/2110575331 shutdown_connections 2024-05-09T00:03:00.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.192+0000 7f0129f33700 1 --2- 172.21.15.12:0/2110575331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f012411a780 0x7f012411cb70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.192+0000 7f0129f33700 1 --2- 172.21.15.12:0/2110575331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f012410f6a0 0x7f0124107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.192+0000 7f0129f33700 1 --2- 172.21.15.12:0/2110575331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f012410fb10 0x7f01241084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.192+0000 7f0129f33700 1 -- 172.21.15.12:0/2110575331 >> 172.21.15.12:0/2110575331 conn(0x7f012406bff0 msgr2=0x7f012406c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.193+0000 7f0129f33700 1 -- 172.21.15.12:0/2110575331 shutdown_connections 2024-05-09T00:03:00.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.193+0000 7f0129f33700 1 -- 172.21.15.12:0/2110575331 wait complete. 2024-05-09T00:03:00.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.193+0000 7f0129f33700 1 Processor -- start 2024-05-09T00:03:00.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.194+0000 7f0129f33700 1 -- start start 2024-05-09T00:03:00.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.194+0000 7f0129f33700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f012410f6a0 0x7f0124119cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.194+0000 7f0129f33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f012410fb10 0x7f012411a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.194+0000 7f0129f33700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f012411a780 0x7f0124115f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.194+0000 7f0129f33700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124116710 con 0x7f012410fb10 2024-05-09T00:03:00.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.194+0000 7f0129f33700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124116880 con 0x7f012411a780 2024-05-09T00:03:00.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.194+0000 7f0129f33700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124116a40 con 0x7f012410f6a0 2024-05-09T00:03:00.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.195+0000 7f0122ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f012410fb10 0x7f012411a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.195+0000 7f0122ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f012410fb10 0x7f012411a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47840/0 (socket says 172.21.15.12:47840) 2024-05-09T00:03:00.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.195+0000 7f0122ffd700 1 -- 172.21.15.12:0/995384380 learned_addr learned my addr 172.21.15.12:0/995384380 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:00.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.195+0000 7f0122ffd700 1 -- 172.21.15.12:0/995384380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f012410f6a0 msgr2=0x7f0124119cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:03:00.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.195+0000 7f0122ffd700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f012410f6a0 0x7f0124119cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.195+0000 7f0122ffd700 1 -- 172.21.15.12:0/995384380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f012411a780 msgr2=0x7f0124115f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.195+0000 7f0122ffd700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f012411a780 0x7f0124115f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.195+0000 7f0122ffd700 1 -- 172.21.15.12:0/995384380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011401a600 con 0x7f012410fb10 2024-05-09T00:03:00.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.195+0000 7f0122ffd700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f012410fb10 0x7f012411a230 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f011c00b9c0 tx=0x7f011c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.196+0000 7f0120ff9700 1 -- 172.21.15.12:0/995384380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c00ca30 con 0x7f012410fb10 2024-05-09T00:03:00.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.196+0000 7f0129f33700 1 -- 172.21.15.12:0/995384380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01241aa410 con 0x7f012410fb10 2024-05-09T00:03:00.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.196+0000 7f0129f33700 1 -- 172.21.15.12:0/995384380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01241aa8c0 con 0x7f012410fb10 2024-05-09T00:03:00.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.198+0000 7f0120ff9700 1 -- 172.21.15.12:0/995384380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f011c00cb90 con 0x7f012410fb10 2024-05-09T00:03:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.198+0000 7f0120ff9700 1 -- 172.21.15.12:0/995384380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c01d610 con 0x7f012410fb10 2024-05-09T00:03:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.203+0000 7f0120ff9700 1 -- 172.21.15.12:0/995384380 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f011c01d770 con 0x7f012410fb10 2024-05-09T00:03:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.206+0000 7f0120ff9700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f010c0a1350 0x7f010c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.207+0000 7f0120ff9700 1 -- 172.21.15.12:0/995384380 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f011c006e40 con 0x7f012410fb10 2024-05-09T00:03:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.208+0000 7f01360edb80 1 Processor -- start 2024-05-09T00:03:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.208+0000 7f01360edb80 1 -- start start 2024-05-09T00:03:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.208+0000 7f01360edb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 0x5628114a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.208+0000 7f01360edb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 0x56281149c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.208+0000 7f01360edb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56281149c890 0x56281149ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.208+0000 7f01360edb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628113f3150 con 0x5628114a1e00 2024-05-09T00:03:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.208+0000 7f01360edb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628113f0b10 con 0x56281149c890 2024-05-09T00:03:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.208+0000 7f01360edb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628113ec400 con 0x5628114a2720 2024-05-09T00:03:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.209+0000 7f0122ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 0x56281149c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.209+0000 7f0122ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 0x56281149c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39702/0 (socket says 172.21.15.12:39702) 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.209+0000 7f0122ffd700 1 -- 172.21.15.12:0/1705409466 learned_addr learned my addr 172.21.15.12:0/1705409466 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.209+0000 7f0123fff700 1 --2- 172.21.15.12:0/1705409466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56281149c890 0x56281149ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.209+0000 7f0123fff700 1 -- 172.21.15.12:0/1705409466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 msgr2=0x56281149c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.209+0000 7f0123fff700 1 --2- 172.21.15.12:0/1705409466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 0x56281149c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.209+0000 7f0123fff700 1 -- 172.21.15.12:0/1705409466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 msgr2=0x5628114a21e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.209+0000 7f0123fff700 1 --2- 172.21.15.12:0/1705409466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 0x5628114a21e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.209+0000 7f0123fff700 1 -- 172.21.15.12:0/1705409466 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56281126d3b0 con 0x56281149c890 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.209+0000 7f0123fff700 1 --2- 172.21.15.12:0/1705409466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56281149c890 0x56281149ccd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0114005b40 tx=0x7f01140036c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba984109d97ffaa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f0109ffb700 1 -- 172.21.15.12:0/1705409466 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011401c9f0 con 0x56281149c890 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f0109ffb700 1 -- 172.21.15.12:0/1705409466 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0114005030 con 0x56281149c890 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f0109ffb700 1 -- 172.21.15.12:0/1705409466 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114004a40 con 0x56281149c890 2024-05-09T00:03:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f01237fe700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f010c0a1350 0x7f010c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f01360edb80 1 -- 172.21.15.12:0/1705409466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56281149c890 msgr2=0x56281149ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f01360edb80 1 --2- 172.21.15.12:0/1705409466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56281149c890 0x56281149ccd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0114005b40 tx=0x7f01140036c0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f01360edb80 1 -- 172.21.15.12:0/1705409466 shutdown_connections 2024-05-09T00:03:00.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f01360edb80 1 --2- 172.21.15.12:0/1705409466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56281149c890 0x56281149ccd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f01360edb80 1 --2- 172.21.15.12:0/1705409466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 0x5628114a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f01360edb80 1 --2- 172.21.15.12:0/1705409466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 0x56281149c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f01360edb80 1 -- 172.21.15.12:0/1705409466 >> 172.21.15.12:0/1705409466 conn(0x5628113fa210 msgr2=0x562811496df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f01360edb80 1 -- 172.21.15.12:0/1705409466 shutdown_connections 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.210+0000 7f01237fe700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f010c0a1350 0x7f010c0a3810 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0118009750 tx=0x7f0118006cb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.211+0000 7f01360edb80 1 -- 172.21.15.12:0/1705409466 wait complete. 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.211+0000 7f01360edb80 1 -- start start 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.211+0000 7f01360edb80 1 Processor -- start 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.211+0000 7f01360edb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56281149c890 0x5628114a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.211+0000 7f01360edb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a1e00 0x5628114a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.211+0000 7f01360edb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a2720 0x5628114a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.211+0000 7f01360edb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562811388560 con 0x5628114a2720 2024-05-09T00:03:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.211+0000 7f01360edb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562811388200 con 0x5628114a1e00 2024-05-09T00:03:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.211+0000 7f01360edb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281126f6a0 con 0x56281149c890 2024-05-09T00:03:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.211+0000 7f01360edb80 10 client.0 _ll_register_callbacks cb 0x562811391b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f01237fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56281149c890 0x5628114a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f01237fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56281149c890 0x5628114a8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39712/0 (socket says 172.21.15.12:39712) 2024-05-09T00:03:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f01237fe700 1 -- 172.21.15.12:0/1593759960 learned_addr learned my addr 172.21.15.12:0/1593759960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f0123fff700 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a2720 0x5628114a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f01237fe700 1 -- 172.21.15.12:0/1593759960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a1e00 msgr2=0x5628114a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f01237fe700 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a1e00 0x5628114a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f01237fe700 1 -- 172.21.15.12:0/1593759960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a2720 msgr2=0x5628114a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f01237fe700 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a2720 0x5628114a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f01237fe700 1 -- 172.21.15.12:0/1593759960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012411d460 con 0x56281149c890 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f01237fe700 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56281149c890 0x5628114a8760 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f01180123b0 tx=0x7f0118004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85998e643242a97d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.212+0000 7f0102ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f0102ffd700 1 -- 172.21.15.12:0/1593759960 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011801a070 con 0x56281149c890 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f0102ffd700 1 -- 172.21.15.12:0/1593759960 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0118012c40 con 0x56281149c890 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f01360edb80 1 -- 172.21.15.12:0/1593759960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562811345270 con 0x56281149c890 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f0102ffd700 1 -- 172.21.15.12:0/1593759960 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01180152e0 con 0x56281149c890 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f01360edb80 10 client.29812 fetch_fsmap learned FSMap version 69 2024-05-09T00:03:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f01360edb80 1 -- 172.21.15.12:0/1593759960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x562811219b60 con 0x56281149c890 2024-05-09T00:03:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f0102ffd700 1 -- 172.21.15.12:0/1593759960 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562811345270 con 0x56281149c890 2024-05-09T00:03:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f0102ffd700 1 -- 172.21.15.12:0/1593759960 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01180273d0 con 0x56281149c890 2024-05-09T00:03:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f01360edb80 10 client.29812 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:03:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f01360edb80 10 client.29812 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:03:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f01360edb80 20 client.29812 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f01360edb80 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5628114b61b0 0x5628114b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f01360edb80 4 client.29812 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.213+0000 7f01360edb80 1 -- 172.21.15.12:0/1593759960 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56281146b810 con 0x5628114b61b0 2024-05-09T00:03:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.222+0000 7f0122ffd700 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5628114b61b0 0x5628114b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.223+0000 7f0122ffd700 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5628114b61b0 0x5628114b85d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5be799d894ade899 server_cookie=e3ac11a9dcca932a in_seq=0 out_seq=0 2024-05-09T00:03:00.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.223+0000 7f0102ffd700 10 client.29812 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:03:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.231+0000 7f0102ffd700 1 -- 172.21.15.12:0/1593759960 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56281146b810 con 0x5628114b61b0 2024-05-09T00:03:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.231+0000 7f0102ffd700 10 client.29812 handle_command_reply: tid=0 2024-05-09T00:03:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.231+0000 7f01360edb80 1 client.29812 shutdown 2024-05-09T00:03:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.232+0000 7f01360edb80 1 -- 172.21.15.12:0/1593759960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56281149c890 msgr2=0x5628114a8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.232+0000 7f01360edb80 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56281149c890 0x5628114a8760 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f01180123b0 tx=0x7f0118004750 comp rx=0 tx=0).stop 2024-05-09T00:03:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.232+0000 7f01360edb80 1 -- 172.21.15.12:0/1593759960 shutdown_connections 2024-05-09T00:03:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.232+0000 7f01360edb80 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5628114b61b0 0x5628114b85d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.232+0000 7f01360edb80 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a2720 0x5628114a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.232+0000 7f01360edb80 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56281149c890 0x5628114a8760 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.232+0000 7f01360edb80 1 --2- 172.21.15.12:0/1593759960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a1e00 0x5628114a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.232+0000 7f01360edb80 1 -- 172.21.15.12:0/1593759960 >> 172.21.15.12:0/1593759960 conn(0x5628113fa210 msgr2=0x5628114965a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.232+0000 7f0102ffd700 0 client.29812 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:03:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.232+0000 7f01360edb80 1 -- 172.21.15.12:0/1593759960 shutdown_connections 2024-05-09T00:03:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.233+0000 7f01360edb80 1 -- 172.21.15.12:0/1593759960 wait complete. 2024-05-09T00:03:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.235+0000 7f01360edb80 20 client.29812 trim_cache size 0 max 16384 2024-05-09T00:03:00.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 128059 40.29 MB/sec execute 55 sec latency 79.861 ms 2024-05-09T00:03:00.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f01360edb80 1 Processor -- start 2024-05-09T00:03:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f01360edb80 1 -- start start 2024-05-09T00:03:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f01360edb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56281149c890 0x56281149c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f01360edb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 0x562811496d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f01360edb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 0x5628114ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f01360edb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562811497e80 con 0x5628114a1e00 2024-05-09T00:03:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f01360edb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628114a3fa0 con 0x56281149c890 2024-05-09T00:03:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f01360edb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562811497260 con 0x5628114a2720 2024-05-09T00:03:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f0122ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 0x562811496d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f0122ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 0x562811496d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47860/0 (socket says 172.21.15.12:47860) 2024-05-09T00:03:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.257+0000 7f0122ffd700 1 -- 172.21.15.12:0/3218796161 learned_addr learned my addr 172.21.15.12:0/3218796161 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.258+0000 7f0123fff700 1 --2- 172.21.15.12:0/3218796161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 0x5628114ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.258+0000 7f0122ffd700 1 -- 172.21.15.12:0/3218796161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 msgr2=0x5628114ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.258+0000 7f0122ffd700 1 --2- 172.21.15.12:0/3218796161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 0x5628114ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.258+0000 7f0122ffd700 1 -- 172.21.15.12:0/3218796161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56281149c890 msgr2=0x56281149c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.258+0000 7f0122ffd700 1 --2- 172.21.15.12:0/3218796161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56281149c890 0x56281149c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.258+0000 7f0122ffd700 1 -- 172.21.15.12:0/3218796161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628113b2560 con 0x5628114a1e00 2024-05-09T00:03:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.258+0000 7f0122ffd700 1 --2- 172.21.15.12:0/3218796161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 0x562811496d20 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f011c09a480 tx=0x7f011c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293646b02fdd2c72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.259+0000 7f010a7fc700 1 -- 172.21.15.12:0/3218796161 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c09e320 con 0x5628114a1e00 2024-05-09T00:03:00.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.259+0000 7f010a7fc700 1 -- 172.21.15.12:0/3218796161 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f011c0a2970 con 0x5628114a1e00 2024-05-09T00:03:00.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.259+0000 7f010a7fc700 1 -- 172.21.15.12:0/3218796161 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c0caee0 con 0x5628114a1e00 2024-05-09T00:03:00.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.260+0000 7f01360edb80 1 -- 172.21.15.12:0/3218796161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 msgr2=0x562811496d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.260+0000 7f01360edb80 1 --2- 172.21.15.12:0/3218796161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 0x562811496d20 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f011c09a480 tx=0x7f011c000f40 comp rx=0 tx=0).stop 2024-05-09T00:03:00.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.260+0000 7f01360edb80 1 -- 172.21.15.12:0/3218796161 shutdown_connections 2024-05-09T00:03:00.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.260+0000 7f01360edb80 1 --2- 172.21.15.12:0/3218796161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a2720 0x5628114ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.260+0000 7f01360edb80 1 --2- 172.21.15.12:0/3218796161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56281149c890 0x56281149c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.261+0000 7f01360edb80 1 --2- 172.21.15.12:0/3218796161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5628114a1e00 0x562811496d20 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.261+0000 7f01360edb80 1 -- 172.21.15.12:0/3218796161 >> 172.21.15.12:0/3218796161 conn(0x5628113fa210 msgr2=0x56281149fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.263+0000 7f01360edb80 1 -- 172.21.15.12:0/3218796161 shutdown_connections 2024-05-09T00:03:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.266+0000 7f01360edb80 1 -- 172.21.15.12:0/3218796161 wait complete. 2024-05-09T00:03:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.266+0000 7f01360edb80 1 -- start start 2024-05-09T00:03:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.266+0000 7f01360edb80 1 Processor -- start 2024-05-09T00:03:00.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.268+0000 7f01360edb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56281149c890 0x5628114ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.268+0000 7f01360edb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a1e00 0x5628114ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.268+0000 7f01360edb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a2720 0x5628114ab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.268+0000 7f01360edb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628114abe10 con 0x56281149c890 2024-05-09T00:03:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.268+0000 7f01360edb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628114abf80 con 0x5628114a2720 2024-05-09T00:03:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.268+0000 7f01360edb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628114ac140 con 0x5628114a1e00 2024-05-09T00:03:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.268+0000 7f01360edb80 10 client.0 _ll_register_callbacks cb 0x7f0118015990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.269+0000 7f0123fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a2720 0x5628114ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.269+0000 7f0123fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a2720 0x5628114ab700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:53048/0 (socket says 172.21.15.12:53048) 2024-05-09T00:03:00.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.269+0000 7f0123fff700 1 -- 172.21.15.12:0/2101229455 learned_addr learned my addr 172.21.15.12:0/2101229455 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:00.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.269+0000 7f0122ffd700 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a1e00 0x5628114ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.269+0000 7f01237fe700 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56281149c890 0x5628114ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.269+0000 7f0123fff700 1 -- 172.21.15.12:0/2101229455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a1e00 msgr2=0x5628114ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.269+0000 7f0123fff700 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a1e00 0x5628114ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.269+0000 7f0123fff700 1 -- 172.21.15.12:0/2101229455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56281149c890 msgr2=0x5628114ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.269+0000 7f0123fff700 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56281149c890 0x5628114ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.269+0000 7f0123fff700 1 -- 172.21.15.12:0/2101229455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01241aa8c0 con 0x5628114a2720 2024-05-09T00:03:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.270+0000 7f0123fff700 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a2720 0x5628114ab700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0114010040 tx=0x7f011401ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1809b6d0eb5c25b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.270+0000 7f0102ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:03:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.270+0000 7f0102ffd700 1 -- 172.21.15.12:0/2101229455 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011402d040 con 0x5628114a2720 2024-05-09T00:03:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.270+0000 7f01360edb80 1 -- 172.21.15.12:0/2101229455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0114005030 con 0x5628114a2720 2024-05-09T00:03:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.270+0000 7f0102ffd700 1 -- 172.21.15.12:0/2101229455 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0114005670 con 0x5628114a2720 2024-05-09T00:03:00.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f0102ffd700 1 -- 172.21.15.12:0/2101229455 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0114004660 con 0x5628114a2720 2024-05-09T00:03:00.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f01360edb80 10 client.29738 fetch_fsmap learned FSMap version 69 2024-05-09T00:03:00.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f01360edb80 1 -- 172.21.15.12:0/2101229455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x562811271cd0 con 0x5628114a2720 2024-05-09T00:03:00.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f0102ffd700 1 -- 172.21.15.12:0/2101229455 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0114005030 con 0x5628114a2720 2024-05-09T00:03:00.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f0102ffd700 1 -- 172.21.15.12:0/2101229455 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f011400e9f0 con 0x5628114a2720 2024-05-09T00:03:00.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f01360edb80 10 client.29738 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:03:00.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f01360edb80 10 client.29738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:03:00.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f01360edb80 20 client.29738 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:00.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f01360edb80 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56281150d5b0 0x56281150f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f01360edb80 4 client.29738 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:03:00.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.273+0000 7f01360edb80 1 -- 172.21.15.12:0/2101229455 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5628113e4ec0 con 0x56281150d5b0 2024-05-09T00:03:00.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.274+0000 7f01237fe700 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56281150d5b0 0x56281150f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.275+0000 7f01237fe700 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56281150d5b0 0x56281150f9a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abc6a0774b8a04c9 server_cookie=c8dd5207d0f6eb31 in_seq=0 out_seq=0 2024-05-09T00:03:00.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f0102ffd700 10 client.29738 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:03:00.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f0102ffd700 1 -- 172.21.15.12:0/2101229455 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5628113e4ec0 con 0x56281150d5b0 2024-05-09T00:03:00.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f0102ffd700 10 client.29738 handle_command_reply: tid=0 2024-05-09T00:03:00.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f01360edb80 1 client.29738 shutdown 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f01360edb80 1 -- 172.21.15.12:0/2101229455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a2720 msgr2=0x5628114ab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f01360edb80 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a2720 0x5628114ab700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0114010040 tx=0x7f011401ce30 comp rx=0 tx=0).stop 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f01360edb80 1 -- 172.21.15.12:0/2101229455 shutdown_connections 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f01360edb80 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56281150d5b0 0x56281150f9a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f01360edb80 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5628114a2720 0x5628114ab700 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f01360edb80 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56281149c890 0x5628114ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f01360edb80 1 --2- 172.21.15.12:0/2101229455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628114a1e00 0x5628114ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f01360edb80 1 -- 172.21.15.12:0/2101229455 >> 172.21.15.12:0/2101229455 conn(0x5628113fa210 msgr2=0x5628114a42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.276+0000 7f0102ffd700 0 client.29738 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.277+0000 7f01360edb80 1 -- 172.21.15.12:0/2101229455 shutdown_connections 2024-05-09T00:03:00.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.277+0000 7f01360edb80 1 -- 172.21.15.12:0/2101229455 wait complete. 2024-05-09T00:03:00.274 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:03:00.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.277+0000 7f01360edb80 20 client.29738 trim_cache size 0 max 16384 2024-05-09T00:03:00.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.278+0000 7f0129f33700 1 -- 172.21.15.12:0/995384380 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f010c0a1350 msgr2=0x7f010c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.278+0000 7f0129f33700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f010c0a1350 0x7f010c0a3810 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0118009750 tx=0x7f0118006cb0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.278+0000 7f0129f33700 1 -- 172.21.15.12:0/995384380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f012410fb10 msgr2=0x7f012411a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.278+0000 7f0129f33700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f012410fb10 0x7f012411a230 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f011c00b9c0 tx=0x7f011c00bd80 comp rx=0 tx=0).stop 2024-05-09T00:03:00.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.280+0000 7f0129f33700 1 -- 172.21.15.12:0/995384380 shutdown_connections 2024-05-09T00:03:00.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.280+0000 7f0129f33700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f010c0a1350 0x7f010c0a3810 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.280+0000 7f0129f33700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f012411a780 0x7f0124115f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.280+0000 7f0129f33700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f012410f6a0 0x7f0124119cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.280+0000 7f0129f33700 1 --2- 172.21.15.12:0/995384380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f012410fb10 0x7f012411a230 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.280+0000 7f0129f33700 1 -- 172.21.15.12:0/995384380 >> 172.21.15.12:0/995384380 conn(0x7f012406bff0 msgr2=0x7f012411bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.281+0000 7f0129f33700 1 -- 172.21.15.12:0/995384380 shutdown_connections 2024-05-09T00:03:00.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.281+0000 7f0129f33700 1 -- 172.21.15.12:0/995384380 wait complete. 2024-05-09T00:03:00.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:03:00.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.550+0000 7f505f9c8700 1 -- 172.21.15.12:0/1434961077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5058068590 msgr2=0x7f5058068a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.550+0000 7f505f9c8700 1 --2- 172.21.15.12:0/1434961077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5058068590 0x7f5058068a10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f504801a9a0 tx=0x7f504801acb0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.554+0000 7f505f9c8700 1 -- 172.21.15.12:0/1434961077 shutdown_connections 2024-05-09T00:03:00.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.554+0000 7f505f9c8700 1 --2- 172.21.15.12:0/1434961077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5058103930 0x7f5058103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.554+0000 7f505f9c8700 1 --2- 172.21.15.12:0/1434961077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5058105e30 0x7f5058068050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.554+0000 7f505f9c8700 1 --2- 172.21.15.12:0/1434961077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5058068590 0x7f5058068a10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.554+0000 7f505f9c8700 1 -- 172.21.15.12:0/1434961077 >> 172.21.15.12:0/1434961077 conn(0x7f5058076f40 msgr2=0x7f5058077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.555+0000 7f505f9c8700 1 -- 172.21.15.12:0/1434961077 shutdown_connections 2024-05-09T00:03:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.555+0000 7f505f9c8700 1 -- 172.21.15.12:0/1434961077 wait complete. 2024-05-09T00:03:00.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.556+0000 7f505f9c8700 1 Processor -- start 2024-05-09T00:03:00.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.557+0000 7f505f9c8700 1 -- start start 2024-05-09T00:03:00.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.558+0000 7f505f9c8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5058068590 0x7f5058113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.558+0000 7f505f9c8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5058103930 0x7f50581143c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.558+0000 7f505f9c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5058105e30 0x7f5058110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.558+0000 7f505f9c8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50581107d0 con 0x7f5058105e30 2024-05-09T00:03:00.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.558+0000 7f505f9c8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5058110940 con 0x7f5058068590 2024-05-09T00:03:00.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.558+0000 7f505f9c8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5058110b00 con 0x7f5058103930 2024-05-09T00:03:00.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.558+0000 7f505cf63700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5058103930 0x7f50581143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.558+0000 7f505cf63700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5058103930 0x7f50581143c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39754/0 (socket says 172.21.15.12:39754) 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.558+0000 7f505cf63700 1 -- 172.21.15.12:0/2460780629 learned_addr learned my addr 172.21.15.12:0/2460780629 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.558+0000 7f505df65700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5058105e30 0x7f5058110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.559+0000 7f505df65700 1 -- 172.21.15.12:0/2460780629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5058103930 msgr2=0x7f50581143c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.559+0000 7f505df65700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5058103930 0x7f50581143c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.559+0000 7f505df65700 1 -- 172.21.15.12:0/2460780629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5058068590 msgr2=0x7f5058113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.559+0000 7f505df65700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5058068590 0x7f5058113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.559+0000 7f505df65700 1 -- 172.21.15.12:0/2460780629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504801a600 con 0x7f5058105e30 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.559+0000 7f505df65700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5058105e30 0x7f5058110040 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5050009ed0 tx=0x7f505000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.560+0000 7f504e7fc700 1 -- 172.21.15.12:0/2460780629 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050004220 con 0x7f5058105e30 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.560+0000 7f505f9c8700 1 -- 172.21.15.12:0/2460780629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5058110de0 con 0x7f5058105e30 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.560+0000 7f505f9c8700 1 -- 172.21.15.12:0/2460780629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505806acb0 con 0x7f5058105e30 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.560+0000 7f504e7fc700 1 -- 172.21.15.12:0/2460780629 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5050004cf0 con 0x7f5058105e30 2024-05-09T00:03:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.560+0000 7f504e7fc700 1 -- 172.21.15.12:0/2460780629 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000cdd0 con 0x7f5058105e30 2024-05-09T00:03:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.565+0000 7f506bb82b80 1 Processor -- start 2024-05-09T00:03:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.565+0000 7f504e7fc700 1 -- 172.21.15.12:0/2460780629 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5050004380 con 0x7f5058105e30 2024-05-09T00:03:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.565+0000 7f506bb82b80 1 -- start start 2024-05-09T00:03:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bee00 0x564d3c2bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bf720 0x564d3c2b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2b9890 0x564d3c2b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f506bb82b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c210150 con 0x564d3c2b9890 2024-05-09T00:03:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f506bb82b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c20db10 con 0x564d3c2bf720 2024-05-09T00:03:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f506bb82b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c209400 con 0x564d3c2bee00 2024-05-09T00:03:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f505cf63700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bee00 0x564d3c2bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f505cf63700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bee00 0x564d3c2bf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39760/0 (socket says 172.21.15.12:39760) 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f505cf63700 1 -- 172.21.15.12:0/3104682489 learned_addr learned my addr 172.21.15.12:0/3104682489 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f504e7fc700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50440a1310 0x7f50440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f504e7fc700 1 -- 172.21.15.12:0/2460780629 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5050036070 con 0x7f5058105e30 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.566+0000 7f505cf63700 1 --2- 172.21.15.12:0/3104682489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2b9890 0x564d3c2b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.567+0000 7f505cf63700 1 -- 172.21.15.12:0/3104682489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bf720 msgr2=0x564d3c2b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.567+0000 7f505cf63700 1 --2- 172.21.15.12:0/3104682489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bf720 0x564d3c2b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.567+0000 7f505cf63700 1 -- 172.21.15.12:0/3104682489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2b9890 msgr2=0x564d3c2b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.567+0000 7f505cf63700 1 --2- 172.21.15.12:0/3104682489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2b9890 0x564d3c2b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.567+0000 7f505cf63700 1 -- 172.21.15.12:0/3104682489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d3c08a3b0 con 0x564d3c2bee00 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.567+0000 7f505cf63700 1 --2- 172.21.15.12:0/3104682489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bee00 0x564d3c2bf1e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f504801cab0 tx=0x7f5048005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=364a18409bd4bf21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.567+0000 7f50437fe700 1 -- 172.21.15.12:0/3104682489 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5048005a00 con 0x564d3c2bee00 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.567+0000 7f50437fe700 1 -- 172.21.15.12:0/3104682489 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5048003d20 con 0x564d3c2bee00 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.567+0000 7f50437fe700 1 -- 172.21.15.12:0/3104682489 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50480055e0 con 0x564d3c2bee00 2024-05-09T00:03:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.568+0000 7f505df65700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50440a1310 0x7f50440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.568+0000 7f506bb82b80 1 -- 172.21.15.12:0/3104682489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bee00 msgr2=0x564d3c2bf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.568+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3104682489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bee00 0x564d3c2bf1e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f504801cab0 tx=0x7f5048005e80 comp rx=0 tx=0).stop 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.569+0000 7f506bb82b80 1 -- 172.21.15.12:0/3104682489 shutdown_connections 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.569+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3104682489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2b9890 0x564d3c2b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.569+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3104682489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bee00 0x564d3c2bf1e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.569+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3104682489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bf720 0x564d3c2b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.569+0000 7f506bb82b80 1 -- 172.21.15.12:0/3104682489 >> 172.21.15.12:0/3104682489 conn(0x564d3c217210 msgr2=0x564d3c2b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.569+0000 7f506bb82b80 1 -- 172.21.15.12:0/3104682489 shutdown_connections 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.569+0000 7f506bb82b80 1 -- 172.21.15.12:0/3104682489 wait complete. 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.569+0000 7f506bb82b80 1 -- start start 2024-05-09T00:03:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.569+0000 7f506bb82b80 1 Processor -- start 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.570+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.570+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bee00 0x564d3c2c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.570+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bf720 0x564d3c2c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.570+0000 7f506bb82b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c1a5560 con 0x564d3c2bee00 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.570+0000 7f506bb82b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c1a5200 con 0x564d3c2bf720 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.570+0000 7f506bb82b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c08c6a0 con 0x564d3c2b9890 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.570+0000 7f506bb82b80 10 client.0 _ll_register_callbacks cb 0x564d3c1aeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.572+0000 7f505cf63700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.572+0000 7f505cf63700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2c2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39772/0 (socket says 172.21.15.12:39772) 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.572+0000 7f505cf63700 1 -- 172.21.15.12:0/3942024479 learned_addr learned my addr 172.21.15.12:0/3942024479 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.572+0000 7f505d764700 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bee00 0x564d3c2c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f505cf63700 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bf720 0x564d3c2c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f505cf63700 1 -- 172.21.15.12:0/3942024479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bf720 msgr2=0x564d3c2c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f505cf63700 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bf720 0x564d3c2c3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f505cf63700 1 -- 172.21.15.12:0/3942024479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bee00 msgr2=0x564d3c2c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f505cf63700 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bee00 0x564d3c2c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f505cf63700 1 -- 172.21.15.12:0/3942024479 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5048030040 con 0x564d3c2b9890 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f505d764700 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bee00 0x564d3c2c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f505cf63700 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2c2b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f504801c2f0 tx=0x7f50480053c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7be5cca2e8265773 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f5040ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f5040ff9700 1 -- 172.21.15.12:0/3942024479 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504801ccc0 con 0x564d3c2b9890 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f5040ff9700 1 -- 172.21.15.12:0/3942024479 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f504801ce20 con 0x564d3c2b9890 2024-05-09T00:03:00.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.573+0000 7f506bb82b80 1 -- 172.21.15.12:0/3942024479 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d3c162270 con 0x564d3c2b9890 2024-05-09T00:03:00.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.578+0000 7f5040ff9700 1 -- 172.21.15.12:0/3942024479 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5048003860 con 0x564d3c2b9890 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.580+0000 7f506bb82b80 10 client.29830 fetch_fsmap learned FSMap version 69 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.580+0000 7f506bb82b80 1 -- 172.21.15.12:0/3942024479 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x564d3c036b60 con 0x564d3c2b9890 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.580+0000 7f5040ff9700 1 -- 172.21.15.12:0/3942024479 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d3c162270 con 0x564d3c2b9890 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.580+0000 7f5040ff9700 1 -- 172.21.15.12:0/3942024479 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5048059420 con 0x564d3c2b9890 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.580+0000 7f506bb82b80 10 client.29830 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.580+0000 7f506bb82b80 10 client.29830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.580+0000 7f506bb82b80 20 client.29830 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.580+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d3c2d2e10 0x564d3c2d5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.580+0000 7f506bb82b80 4 client.29830 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.580+0000 7f506bb82b80 1 -- 172.21.15.12:0/3942024479 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d3c288810 con 0x564d3c2d2e10 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.582+0000 7f505d764700 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d3c2d2e10 0x564d3c2d5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.583+0000 7f505d764700 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d3c2d2e10 0x564d3c2d5230 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4626fb0639e0eae server_cookie=fc486d21fccbc6be in_seq=0 out_seq=0 2024-05-09T00:03:00.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.583+0000 7f5040ff9700 10 client.29830 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:00.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.594+0000 7f505df65700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50440a1310 0x7f50440a37d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f50500b7fe0 tx=0x7f5050009f00 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.594+0000 7f5040ff9700 1 -- 172.21.15.12:0/3942024479 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564d3c288810 con 0x564d3c2d2e10 2024-05-09T00:03:00.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.594+0000 7f5040ff9700 10 client.29830 handle_command_reply: tid=0 2024-05-09T00:03:00.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.594+0000 7f506bb82b80 1 client.29830 shutdown 2024-05-09T00:03:00.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.594+0000 7f506bb82b80 1 -- 172.21.15.12:0/3942024479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 msgr2=0x564d3c2c2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.594+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2c2b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f504801c2f0 tx=0x7f50480053c0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.597+0000 7f506bb82b80 1 -- 172.21.15.12:0/3942024479 shutdown_connections 2024-05-09T00:03:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.597+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d3c2d2e10 0x564d3c2d5230 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.597+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bf720 0x564d3c2c3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.597+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2c2b00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.597+0000 7f5040ff9700 0 client.29830 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.597+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3942024479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bee00 0x564d3c2c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.597+0000 7f506bb82b80 1 -- 172.21.15.12:0/3942024479 >> 172.21.15.12:0/3942024479 conn(0x564d3c217210 msgr2=0x564d3c2b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.601+0000 7f506bb82b80 1 -- 172.21.15.12:0/3942024479 shutdown_connections 2024-05-09T00:03:00.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.602+0000 7f506bb82b80 1 -- 172.21.15.12:0/3942024479 wait complete. 2024-05-09T00:03:00.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.602+0000 7f506bb82b80 20 client.29830 trim_cache size 0 max 16384 2024-05-09T00:03:00.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.625+0000 7f506bb82b80 1 Processor -- start 2024-05-09T00:03:00.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.625+0000 7f506bb82b80 1 -- start start 2024-05-09T00:03:00.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2d5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bee00 0x564d3c2b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bf720 0x564d3c2ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f506bb82b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c2b0e00 con 0x564d3c2bf720 2024-05-09T00:03:00.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f506bb82b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c2b5e50 con 0x564d3c2bee00 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f506bb82b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c2ac070 con 0x564d3c2b9890 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f505cf63700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2d5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f505cf63700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2d5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39784/0 (socket says 172.21.15.12:39784) 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f505cf63700 1 -- 172.21.15.12:0/3292267894 learned_addr learned my addr 172.21.15.12:0/3292267894 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f505cf63700 1 --2- 172.21.15.12:0/3292267894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bf720 0x564d3c2ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f505cf63700 1 -- 172.21.15.12:0/3292267894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bee00 msgr2=0x564d3c2b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f505cf63700 1 --2- 172.21.15.12:0/3292267894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bee00 0x564d3c2b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f505cf63700 1 -- 172.21.15.12:0/3292267894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bf720 msgr2=0x564d3c2ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f505cf63700 1 --2- 172.21.15.12:0/3292267894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bf720 0x564d3c2ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.626+0000 7f505cf63700 1 -- 172.21.15.12:0/3292267894 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d3c1cf560 con 0x564d3c2b9890 2024-05-09T00:03:00.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.627+0000 7f505cf63700 1 --2- 172.21.15.12:0/3292267894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2d5200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5048005170 tx=0x7f5048005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75309dcb40ce7fc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.627+0000 7f5043fff700 1 -- 172.21.15.12:0/3292267894 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5048059aa0 con 0x564d3c2b9890 2024-05-09T00:03:00.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.627+0000 7f5043fff700 1 -- 172.21.15.12:0/3292267894 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5048059420 con 0x564d3c2b9890 2024-05-09T00:03:00.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.627+0000 7f5043fff700 1 -- 172.21.15.12:0/3292267894 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5048034710 con 0x564d3c2b9890 2024-05-09T00:03:00.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.628+0000 7f506bb82b80 1 -- 172.21.15.12:0/3292267894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 msgr2=0x564d3c2d5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.628+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3292267894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2d5200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5048005170 tx=0x7f5048005e80 comp rx=0 tx=0).stop 2024-05-09T00:03:00.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.628+0000 7f506bb82b80 1 -- 172.21.15.12:0/3292267894 shutdown_connections 2024-05-09T00:03:00.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.628+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3292267894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2bf720 0x564d3c2ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.628+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3292267894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2b9890 0x564d3c2d5200 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.628+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3292267894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bee00 0x564d3c2b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.628+0000 7f506bb82b80 1 -- 172.21.15.12:0/3292267894 >> 172.21.15.12:0/3292267894 conn(0x564d3c217210 msgr2=0x564d3c2b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.629+0000 7f506bb82b80 1 -- 172.21.15.12:0/3292267894 shutdown_connections 2024-05-09T00:03:00.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.629+0000 7f506bb82b80 1 -- 172.21.15.12:0/3292267894 wait complete. 2024-05-09T00:03:00.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.629+0000 7f506bb82b80 1 -- start start 2024-05-09T00:03:00.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.629+0000 7f506bb82b80 1 Processor -- start 2024-05-09T00:03:00.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.631+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2b9890 0x564d3c320610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.631+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bee00 0x564d3c320b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.631+0000 7f506bb82b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bf720 0x564d3c2c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.631+0000 7f506bb82b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c2c9ad0 con 0x564d3c2b9890 2024-05-09T00:03:00.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.631+0000 7f506bb82b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c2c9c40 con 0x564d3c2bee00 2024-05-09T00:03:00.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.631+0000 7f506bb82b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c2c9e00 con 0x564d3c2bf720 2024-05-09T00:03:00.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.631+0000 7f506bb82b80 10 client.0 _ll_register_callbacks cb 0x564d3c1aeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:00.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.631+0000 7f505d764700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bee00 0x564d3c320b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.631+0000 7f505d764700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bee00 0x564d3c320b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:53082/0 (socket says 172.21.15.12:53082) 2024-05-09T00:03:00.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.631+0000 7f505d764700 1 -- 172.21.15.12:0/3288694126 learned_addr learned my addr 172.21.15.12:0/3288694126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:00.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.633+0000 7f505cf63700 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bf720 0x564d3c2c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.633+0000 7f505cf63700 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2b9890 0x564d3c320610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.633+0000 7f505cf63700 1 -- 172.21.15.12:0/3288694126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bee00 msgr2=0x564d3c320b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.633+0000 7f505cf63700 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bee00 0x564d3c320b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.633+0000 7f505cf63700 1 -- 172.21.15.12:0/3288694126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2b9890 msgr2=0x564d3c320610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:00.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.633+0000 7f505cf63700 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2b9890 0x564d3c320610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.633+0000 7f505cf63700 1 -- 172.21.15.12:0/3288694126 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d3c036b60 con 0x564d3c2bf720 2024-05-09T00:03:00.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.634+0000 7f505cf63700 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bf720 0x564d3c2c9500 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5048059920 tx=0x7f504801c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d99e03eac717ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:00.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.634+0000 7f5040ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:03:00.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.634+0000 7f5040ff9700 1 -- 172.21.15.12:0/3288694126 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5048059610 con 0x564d3c2bf720 2024-05-09T00:03:00.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.634+0000 7f5040ff9700 1 -- 172.21.15.12:0/3288694126 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5048000e40 con 0x564d3c2bf720 2024-05-09T00:03:00.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.634+0000 7f506bb82b80 1 -- 172.21.15.12:0/3288694126 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5048003d20 con 0x564d3c2bf720 2024-05-09T00:03:00.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.634+0000 7f5040ff9700 1 -- 172.21.15.12:0/3288694126 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5048003a70 con 0x564d3c2bf720 2024-05-09T00:03:00.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.641+0000 7f506bb82b80 10 client.29836 fetch_fsmap learned FSMap version 69 2024-05-09T00:03:00.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.641+0000 7f506bb82b80 1 -- 172.21.15.12:0/3288694126 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x564d3c08ecd0 con 0x564d3c2bf720 2024-05-09T00:03:00.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.642+0000 7f5040ff9700 1 -- 172.21.15.12:0/3288694126 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5048003d20 con 0x564d3c2bf720 2024-05-09T00:03:00.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.642+0000 7f5040ff9700 1 -- 172.21.15.12:0/3288694126 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5048063450 con 0x564d3c2bf720 2024-05-09T00:03:00.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.642+0000 7f506bb82b80 10 client.29836 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:03:00.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.642+0000 7f506bb82b80 10 client.29836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:00.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.642+0000 7f506bb82b80 20 client.29836 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:00.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.642+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d3c3290c0 0x564d3c32b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:00.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.642+0000 7f506bb82b80 4 client.29836 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:03:00.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.642+0000 7f506bb82b80 1 -- 172.21.15.12:0/3288694126 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d3c201ec0 con 0x564d3c3290c0 2024-05-09T00:03:00.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.643+0000 7f505d764700 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d3c3290c0 0x564d3c32b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:00.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.649+0000 7f505d764700 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d3c3290c0 0x564d3c32b4b0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=105f5612049a69a6 server_cookie=1897e692d6e63a5f in_seq=0 out_seq=0 2024-05-09T00:03:00.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.652+0000 7f5040ff9700 10 client.29836 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:00.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.658+0000 7f5040ff9700 1 -- 172.21.15.12:0/3288694126 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d3c201ec0 con 0x564d3c3290c0 2024-05-09T00:03:00.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.658+0000 7f5040ff9700 10 client.29836 handle_command_reply: tid=0 2024-05-09T00:03:00.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.658+0000 7f506bb82b80 1 client.29836 shutdown 2024-05-09T00:03:00.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.659+0000 7f506bb82b80 1 -- 172.21.15.12:0/3288694126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bf720 msgr2=0x564d3c2c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.659+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bf720 0x564d3c2c9500 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5048059920 tx=0x7f504801c3a0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.659+0000 7f506bb82b80 1 -- 172.21.15.12:0/3288694126 shutdown_connections 2024-05-09T00:03:00.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.659+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3c2bf720 0x564d3c2c9500 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.659+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d3c2b9890 0x564d3c320610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.659+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564d3c3290c0 0x564d3c32b4b0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.659+0000 7f506bb82b80 1 --2- 172.21.15.12:0/3288694126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564d3c2bee00 0x564d3c320b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.659+0000 7f506bb82b80 1 -- 172.21.15.12:0/3288694126 >> 172.21.15.12:0/3288694126 conn(0x564d3c217210 msgr2=0x564d3c2bcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.659+0000 7f5040ff9700 0 client.29836 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:00.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.659+0000 7f506bb82b80 1 -- 172.21.15.12:0/3288694126 shutdown_connections 2024-05-09T00:03:00.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.661+0000 7f506bb82b80 1 -- 172.21.15.12:0/3288694126 wait complete. 2024-05-09T00:03:00.656 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:03:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.662+0000 7f506bb82b80 20 client.29836 trim_cache size 0 max 16384 2024-05-09T00:03:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.663+0000 7f505f9c8700 1 -- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50440a1310 msgr2=0x7f50440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.663+0000 7f505f9c8700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50440a1310 0x7f50440a37d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f50500b7fe0 tx=0x7f5050009f00 comp rx=0 tx=0).stop 2024-05-09T00:03:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.663+0000 7f505f9c8700 1 -- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5058105e30 msgr2=0x7f5058110040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.663+0000 7f505f9c8700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5058105e30 0x7f5058110040 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5050009ed0 tx=0x7f505000ec50 comp rx=0 tx=0).stop 2024-05-09T00:03:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.663+0000 7f505f9c8700 1 -- 172.21.15.12:0/2460780629 shutdown_connections 2024-05-09T00:03:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.663+0000 7f505f9c8700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50440a1310 0x7f50440a37d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.663+0000 7f505f9c8700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5058105e30 0x7f5058110040 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.663+0000 7f505f9c8700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5058068590 0x7f5058113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.663+0000 7f505f9c8700 1 --2- 172.21.15.12:0/2460780629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5058103930 0x7f50581143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.663+0000 7f505f9c8700 1 -- 172.21.15.12:0/2460780629 >> 172.21.15.12:0/2460780629 conn(0x7f5058076f40 msgr2=0x7f50580fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.666+0000 7f505f9c8700 1 -- 172.21.15.12:0/2460780629 shutdown_connections 2024-05-09T00:03:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:00.666+0000 7f505f9c8700 1 -- 172.21.15.12:0/2460780629 wait complete. 2024-05-09T00:03:01.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 128483 40.04 MB/sec execute 56 sec latency 113.553 ms 2024-05-09T00:03:01.687 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:03:01.688 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:03:01.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.906+0000 7f8c0efdd700 1 -- 172.21.15.12:0/4070554667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c081052b0 msgr2=0x7f8c08105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:01.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.906+0000 7f8c0efdd700 1 --2- 172.21.15.12:0/4070554667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c081052b0 0x7f8c08105c40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009f90 tx=0x7f8c0000b330 comp rx=0 tx=0).stop 2024-05-09T00:03:01.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.907+0000 7f8c0efdd700 1 -- 172.21.15.12:0/4070554667 shutdown_connections 2024-05-09T00:03:01.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.907+0000 7f8c0efdd700 1 --2- 172.21.15.12:0/4070554667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c081052b0 0x7f8c08105c40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.907+0000 7f8c0efdd700 1 --2- 172.21.15.12:0/4070554667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c080ff630 0x7f8c080ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.907+0000 7f8c0efdd700 1 --2- 172.21.15.12:0/4070554667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c080fffe0 0x7f8c08104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.907+0000 7f8c0efdd700 1 -- 172.21.15.12:0/4070554667 >> 172.21.15.12:0/4070554667 conn(0x7f8c08074d40 msgr2=0x7f8c08075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:01.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.907+0000 7f8c0efdd700 1 -- 172.21.15.12:0/4070554667 shutdown_connections 2024-05-09T00:03:01.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.907+0000 7f8c0efdd700 1 -- 172.21.15.12:0/4070554667 wait complete. 2024-05-09T00:03:01.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.908+0000 7f8c0efdd700 1 Processor -- start 2024-05-09T00:03:01.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.908+0000 7f8c0efdd700 1 -- start start 2024-05-09T00:03:01.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.908+0000 7f8c0efdd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c080ff630 0x7f8c080726d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.908+0000 7f8c0efdd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081052b0 0x7f8c08072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.908+0000 7f8c0efdd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c0806b900 0x7f8c0806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.908+0000 7f8c0efdd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0806c540 con 0x7f8c081052b0 2024-05-09T00:03:01.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.908+0000 7f8c0efdd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08070260 con 0x7f8c0806b900 2024-05-09T00:03:01.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.908+0000 7f8c0efdd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08070420 con 0x7f8c080ff630 2024-05-09T00:03:01.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c0cd79700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c080ff630 0x7f8c080726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c0cd79700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c080ff630 0x7f8c080726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39812/0 (socket says 172.21.15.12:39812) 2024-05-09T00:03:01.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c0cd79700 1 -- 172.21.15.12:0/295062661 learned_addr learned my addr 172.21.15.12:0/295062661 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:01.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c0d57a700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c0806b900 0x7f8c0806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c07fff700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081052b0 0x7f8c08072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c0d57a700 1 -- 172.21.15.12:0/295062661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c080ff630 msgr2=0x7f8c080726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c0d57a700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c080ff630 0x7f8c080726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c0d57a700 1 -- 172.21.15.12:0/295062661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081052b0 msgr2=0x7f8c08072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c0d57a700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081052b0 0x7f8c08072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c0d57a700 1 -- 172.21.15.12:0/295062661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf4009520 con 0x7f8c0806b900 2024-05-09T00:03:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.909+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c080ff630 0x7f8c080726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:03:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.910+0000 7f8c0d57a700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c0806b900 0x7f8c0806bdb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000b6c0 tx=0x7f8c000079d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.910+0000 7f8c05ffb700 1 -- 172.21.15.12:0/295062661 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0000e040 con 0x7f8c0806b900 2024-05-09T00:03:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.910+0000 7f8c05ffb700 1 -- 172.21.15.12:0/295062661 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c00007e80 con 0x7f8c0806b900 2024-05-09T00:03:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.910+0000 7f8c05ffb700 1 -- 172.21.15.12:0/295062661 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c000052a0 con 0x7f8c0806b900 2024-05-09T00:03:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.910+0000 7f8c0efdd700 1 -- 172.21.15.12:0/295062661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c00009c40 con 0x7f8c0806b900 2024-05-09T00:03:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.911+0000 7f8c0efdd700 1 -- 172.21.15.12:0/295062661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c08070880 con 0x7f8c0806b900 2024-05-09T00:03:01.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.915+0000 7f8c05ffb700 1 -- 172.21.15.12:0/295062661 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8c0001b450 con 0x7f8c0806b900 2024-05-09T00:03:01.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.916+0000 7f8c05ffb700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf80a1390 0x7f8bf80a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.916+0000 7f8c05ffb700 1 -- 172.21.15.12:0/295062661 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8c000cdfa0 con 0x7f8c0806b900 2024-05-09T00:03:01.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.916+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf80a1390 0x7f8bf80a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c1b197b80 1 Processor -- start 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c1b197b80 1 -- start start 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf80a1390 0x7f8bf80a3850 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc005950 tx=0x7f8bfc009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e96260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e967a0 0x55e0d9e903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e90910 0x55e0d9e90d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9de71d0 con 0x55e0d9e90910 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9de4b90 con 0x55e0d9e95e80 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9de0480 con 0x55e0d9e967a0 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c0cd79700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e96260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c0cd79700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e96260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:53102/0 (socket says 172.21.15.12:53102) 2024-05-09T00:03:01.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c0cd79700 1 -- 172.21.15.12:0/1277883410 learned_addr learned my addr 172.21.15.12:0/1277883410 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:01.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.917+0000 7f8c07fff700 1 --2- 172.21.15.12:0/1277883410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e967a0 0x55e0d9e903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.918+0000 7f8c0cd79700 1 -- 172.21.15.12:0/1277883410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e967a0 msgr2=0x55e0d9e903d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:01.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.918+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/1277883410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e967a0 0x55e0d9e903d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.918+0000 7f8c0cd79700 1 -- 172.21.15.12:0/1277883410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e90910 msgr2=0x55e0d9e90d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:01.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.918+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/1277883410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e90910 0x55e0d9e90d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.918+0000 7f8c0cd79700 1 -- 172.21.15.12:0/1277883410 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0d9c61430 con 0x55e0d9e95e80 2024-05-09T00:03:01.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.918+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/1277883410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e96260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc012000 tx=0x7f8bfc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b012e3ed901b2cf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:01.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:01 smithi012 ceph-mon[42503]: pgmap v533: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 690 op/s 2024-05-09T00:03:01.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.920+0000 7f8bf2ffd700 1 -- 172.21.15.12:0/1277883410 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc01f620 con 0x55e0d9e95e80 2024-05-09T00:03:01.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.920+0000 7f8bf2ffd700 1 -- 172.21.15.12:0/1277883410 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bfc016070 con 0x55e0d9e95e80 2024-05-09T00:03:01.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.920+0000 7f8bf2ffd700 1 -- 172.21.15.12:0/1277883410 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc01daa0 con 0x55e0d9e95e80 2024-05-09T00:03:01.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.921+0000 7f8c1b197b80 1 -- 172.21.15.12:0/1277883410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 msgr2=0x55e0d9e96260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:01.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.921+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/1277883410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e96260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc012000 tx=0x7f8bfc014000 comp rx=0 tx=0).stop 2024-05-09T00:03:01.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.921+0000 7f8c1b197b80 1 -- 172.21.15.12:0/1277883410 shutdown_connections 2024-05-09T00:03:01.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.921+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/1277883410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e90910 0x55e0d9e90d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.921+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/1277883410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e96260 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.921+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/1277883410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e967a0 0x55e0d9e903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.921+0000 7f8c1b197b80 1 -- 172.21.15.12:0/1277883410 >> 172.21.15.12:0/1277883410 conn(0x55e0d9dee290 msgr2=0x55e0d9e8ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:01.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.923+0000 7f8c1b197b80 1 -- 172.21.15.12:0/1277883410 shutdown_connections 2024-05-09T00:03:01.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.923+0000 7f8c1b197b80 1 -- 172.21.15.12:0/1277883410 wait complete. 2024-05-09T00:03:01.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.924+0000 7f8c1b197b80 1 -- start start 2024-05-09T00:03:01.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.924+0000 7f8c1b197b80 1 Processor -- start 2024-05-09T00:03:01.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e90910 0x55e0d9e99d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9d7c5e0 con 0x55e0d9e90910 2024-05-09T00:03:01.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9d7c280 con 0x55e0d9e967a0 2024-05-09T00:03:01.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9c63720 con 0x55e0d9e95e80 2024-05-09T00:03:01.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c1b197b80 10 client.0 _ll_register_callbacks cb 0x55e0d9d85bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:01.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c07fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c07fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39834/0 (socket says 172.21.15.12:39834) 2024-05-09T00:03:01.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c07fff700 1 -- 172.21.15.12:0/2592240007 learned_addr learned my addr 172.21.15.12:0/2592240007 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:01.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.925+0000 7f8c0d57a700 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.926+0000 7f8c07fff700 1 -- 172.21.15.12:0/2592240007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 msgr2=0x55e0d9e9a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:01.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.926+0000 7f8c07fff700 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.926+0000 7f8c07fff700 1 -- 172.21.15.12:0/2592240007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e90910 msgr2=0x55e0d9e99d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:01.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.926+0000 7f8c07fff700 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e90910 0x55e0d9e99d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.926+0000 7f8c07fff700 1 -- 172.21.15.12:0/2592240007 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc011ce0 con 0x55e0d9e95e80 2024-05-09T00:03:01.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.926+0000 7f8c07fff700 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9a290 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40117e0 tx=0x7f8bf4011ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f06fd2ccbcb5e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:01.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.926+0000 7f8be7fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:03:01.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.929+0000 7f8be7fff700 1 -- 172.21.15.12:0/2592240007 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4016070 con 0x55e0d9e95e80 2024-05-09T00:03:01.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.929+0000 7f8be7fff700 1 -- 172.21.15.12:0/2592240007 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8bf4004020 con 0x55e0d9e95e80 2024-05-09T00:03:01.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.929+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2592240007 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0d9d392f0 con 0x55e0d9e95e80 2024-05-09T00:03:01.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.929+0000 7f8be7fff700 1 -- 172.21.15.12:0/2592240007 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf40041e0 con 0x55e0d9e95e80 2024-05-09T00:03:01.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.929+0000 7f8c1b197b80 10 client.29848 fetch_fsmap learned FSMap version 69 2024-05-09T00:03:01.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.929+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2592240007 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55e0d9c0dbe0 con 0x55e0d9e95e80 2024-05-09T00:03:01.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.930+0000 7f8be7fff700 1 -- 172.21.15.12:0/2592240007 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf4027080 con 0x55e0d9e95e80 2024-05-09T00:03:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.930+0000 7f8be7fff700 1 -- 172.21.15.12:0/2592240007 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8bf4003930 con 0x55e0d9e95e80 2024-05-09T00:03:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.930+0000 7f8c1b197b80 10 client.29848 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:03:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.930+0000 7f8c1b197b80 10 client.29848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.930+0000 7f8c1b197b80 20 client.29848 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.930+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d9eaa440 0x55e0d9eac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.930+0000 7f8c1b197b80 4 client.29848 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.930+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2592240007 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0d9e5f890 con 0x55e0d9eaa440 2024-05-09T00:03:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.930+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d9eaa440 0x55e0d9eac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.931+0000 7f8be7fff700 10 client.29848 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.931+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d9eaa440 0x55e0d9eac860 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d6cd700f386ef3 server_cookie=1db34456d1caa644 in_seq=0 out_seq=0 2024-05-09T00:03:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.932+0000 7f8be7fff700 1 -- 172.21.15.12:0/2592240007 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0d9e5f890 con 0x55e0d9eaa440 2024-05-09T00:03:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.932+0000 7f8be7fff700 10 client.29848 handle_command_reply: tid=0 2024-05-09T00:03:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.932+0000 7f8c1b197b80 1 client.29848 shutdown 2024-05-09T00:03:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.933+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2592240007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e95e80 msgr2=0x55e0d9e9a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.933+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9a290 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40117e0 tx=0x7f8bf4011ba0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.933+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2592240007 shutdown_connections 2024-05-09T00:03:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.933+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d9eaa440 0x55e0d9eac860 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.933+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.933+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e90910 0x55e0d9e99d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.933+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/2592240007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9a290 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.933+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2592240007 >> 172.21.15.12:0/2592240007 conn(0x55e0d9dee290 msgr2=0x55e0d9e8a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:01.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.933+0000 7f8be7fff700 0 client.29848 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:01.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.933+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2592240007 shutdown_connections 2024-05-09T00:03:01.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.936+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2592240007 wait complete. 2024-05-09T00:03:01.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.937+0000 7f8c1b197b80 20 client.29848 trim_cache size 0 max 16384 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c1b197b80 1 Processor -- start 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c1b197b80 1 -- start start 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e90910 0x55e0d9e903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e95e80 0x55e0d9e8ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9e87e40 con 0x55e0d9e95e80 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9e912b0 con 0x55e0d9e967a0 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9e982a0 con 0x55e0d9e90910 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c0d57a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c0d57a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:53118/0 (socket says 172.21.15.12:53118) 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.971+0000 7f8c0d57a700 1 -- 172.21.15.12:0/2924894972 learned_addr learned my addr 172.21.15.12:0/2924894972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.972+0000 7f8c0d57a700 1 -- 172.21.15.12:0/2924894972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e90910 msgr2=0x55e0d9e903d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:03:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.972+0000 7f8c0d57a700 1 --2- 172.21.15.12:0/2924894972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e90910 0x55e0d9e903d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.972+0000 7f8c0d57a700 1 -- 172.21.15.12:0/2924894972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e95e80 msgr2=0x55e0d9e8ada0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:03:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.972+0000 7f8c0d57a700 1 --2- 172.21.15.12:0/2924894972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e95e80 0x55e0d9e8ada0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.972+0000 7f8c0d57a700 1 -- 172.21.15.12:0/2924894972 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0d9e87560 con 0x55e0d9e967a0 2024-05-09T00:03:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.972+0000 7f8c0d57a700 1 --2- 172.21.15.12:0/2924894972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9a5f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8c00097620 tx=0x7f8c00006020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c74af76478474d77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.972+0000 7f8bf37fe700 1 -- 172.21.15.12:0/2924894972 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0009d070 con 0x55e0d9e967a0 2024-05-09T00:03:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.973+0000 7f8bf37fe700 1 -- 172.21.15.12:0/2924894972 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0009b820 con 0x55e0d9e967a0 2024-05-09T00:03:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.973+0000 7f8bf37fe700 1 -- 172.21.15.12:0/2924894972 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00097830 con 0x55e0d9e967a0 2024-05-09T00:03:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2924894972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 msgr2=0x55e0d9e9a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/2924894972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9a5f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8c00097620 tx=0x7f8c00006020 comp rx=0 tx=0).stop 2024-05-09T00:03:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2924894972 shutdown_connections 2024-05-09T00:03:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/2924894972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9a5f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/2924894972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e90910 0x55e0d9e903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/2924894972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e95e80 0x55e0d9e8ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2924894972 >> 172.21.15.12:0/2924894972 conn(0x55e0d9dee290 msgr2=0x55e0d9e98620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2924894972 shutdown_connections 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 -- 172.21.15.12:0/2924894972 wait complete. 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 -- start start 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.974+0000 7f8c1b197b80 1 Processor -- start 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.975+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e90910 0x55e0d9ea12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.975+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.975+0000 7f8c1b197b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.975+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9ea0270 con 0x55e0d9e967a0 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.975+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9ea03e0 con 0x55e0d9e95e80 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.975+0000 7f8c1b197b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d9ea0520 con 0x55e0d9e90910 2024-05-09T00:03:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.975+0000 7f8c1b197b80 10 client.0 _ll_register_callbacks cb 0x55e0d9d85bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:01.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.975+0000 7f8c07fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.975+0000 7f8c07fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:53128/0 (socket says 172.21.15.12:53128) 2024-05-09T00:03:01.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.975+0000 7f8c07fff700 1 -- 172.21.15.12:0/892183765 learned_addr learned my addr 172.21.15.12:0/892183765 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:01.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.976+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e90910 0x55e0d9ea12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8c07fff700 1 -- 172.21.15.12:0/892183765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e90910 msgr2=0x55e0d9ea12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:01.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8c07fff700 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e90910 0x55e0d9ea12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8c07fff700 1 -- 172.21.15.12:0/892183765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e967a0 msgr2=0x55e0d9e9fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:01.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8c07fff700 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8c07fff700 1 -- 172.21.15.12:0/892183765 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0009fe70 con 0x55e0d9e95e80 2024-05-09T00:03:01.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8c07fff700 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9f7f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4003de0 tx=0x7f8bf4005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58a1f1be7dbbd9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:01.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8be7fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:03:01.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8be7fff700 1 -- 172.21.15.12:0/892183765 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4016040 con 0x55e0d9e95e80 2024-05-09T00:03:01.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8c1b197b80 1 -- 172.21.15.12:0/892183765 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bfc016070 con 0x55e0d9e95e80 2024-05-09T00:03:01.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8be7fff700 1 -- 172.21.15.12:0/892183765 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f8bf4000e40 con 0x55e0d9e95e80 2024-05-09T00:03:01.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.978+0000 7f8be7fff700 1 -- 172.21.15.12:0/892183765 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf4003740 con 0x55e0d9e95e80 2024-05-09T00:03:01.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.979+0000 7f8c1b197b80 10 client.29765 fetch_fsmap learned FSMap version 69 2024-05-09T00:03:01.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.979+0000 7f8c1b197b80 1 -- 172.21.15.12:0/892183765 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55e0d9da65e0 con 0x55e0d9e95e80 2024-05-09T00:03:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.983+0000 7f8be7fff700 1 -- 172.21.15.12:0/892183765 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bfc016070 con 0x55e0d9e95e80 2024-05-09T00:03:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.983+0000 7f8be7fff700 1 -- 172.21.15.12:0/892183765 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8bf4003f00 con 0x55e0d9e95e80 2024-05-09T00:03:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.983+0000 7f8c1b197b80 10 client.29765 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:03:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.983+0000 7f8c1b197b80 10 client.29765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.983+0000 7f8c1b197b80 20 client.29765 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.983+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d9f12e10 0x55e0d9f15230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.983+0000 7f8c1b197b80 4 client.29765 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:03:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.983+0000 7f8c1b197b80 1 -- 172.21.15.12:0/892183765 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e0d9dd8f40 con 0x55e0d9f12e10 2024-05-09T00:03:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.984+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d9f12e10 0x55e0d9f15230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.984+0000 7f8be7fff700 10 client.29765 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.984+0000 7f8c0cd79700 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d9f12e10 0x55e0d9f15230 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a2b4b74f843477 server_cookie=dfdd49cd72bef405 in_seq=0 out_seq=0 2024-05-09T00:03:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.985+0000 7f8be7fff700 1 -- 172.21.15.12:0/892183765 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e0d9dd8f40 con 0x55e0d9f12e10 2024-05-09T00:03:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.985+0000 7f8be7fff700 10 client.29765 handle_command_reply: tid=0 2024-05-09T00:03:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.985+0000 7f8c1b197b80 1 client.29765 shutdown 2024-05-09T00:03:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.985+0000 7f8c1b197b80 1 -- 172.21.15.12:0/892183765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 msgr2=0x55e0d9e9f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.985+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9f7f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4003de0 tx=0x7f8bf4005c60 comp rx=0 tx=0).stop 2024-05-09T00:03:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.986+0000 7f8c1b197b80 1 -- 172.21.15.12:0/892183765 shutdown_connections 2024-05-09T00:03:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.986+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0d9e967a0 0x55e0d9e9fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.986+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0d9e90910 0x55e0d9ea12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.986+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e0d9f12e10 0x55e0d9f15230 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.986+0000 7f8c1b197b80 1 --2- 172.21.15.12:0/892183765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e0d9e95e80 0x55e0d9e9f7f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.986+0000 7f8c1b197b80 1 -- 172.21.15.12:0/892183765 >> 172.21.15.12:0/892183765 conn(0x55e0d9dee290 msgr2=0x55e0d9e94e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.986+0000 7f8be7fff700 0 client.29765 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.986+0000 7f8c1b197b80 1 -- 172.21.15.12:0/892183765 shutdown_connections 2024-05-09T00:03:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.986+0000 7f8c1b197b80 1 -- 172.21.15.12:0/892183765 wait complete. 2024-05-09T00:03:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.987+0000 7f8c1b197b80 20 client.29765 trim_cache size 0 max 16384 2024-05-09T00:03:01.982 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:03:01.982 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:03:01.982 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "52d03b32-465a-46dc-821d-c9f28565796d", 2024-05-09T00:03:01.982 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:03:01.982 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:03:01.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 -- 172.21.15.12:0/295062661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf80a1390 msgr2=0x7f8bf80a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:01.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf80a1390 0x7f8bf80a3850 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc005950 tx=0x7f8bfc009310 comp rx=0 tx=0).stop 2024-05-09T00:03:01.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 -- 172.21.15.12:0/295062661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c0806b900 msgr2=0x7f8c0806bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c0806b900 0x7f8c0806bdb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000b6c0 tx=0x7f8c000079d0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 -- 172.21.15.12:0/295062661 shutdown_connections 2024-05-09T00:03:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf80a1390 0x7f8bf80a3850 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c0806b900 0x7f8c0806bdb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c080ff630 0x7f8c080726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 --2- 172.21.15.12:0/295062661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081052b0 0x7f8c08072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 -- 172.21.15.12:0/295062661 >> 172.21.15.12:0/295062661 conn(0x7f8c08074d40 msgr2=0x7f8c08102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 -- 172.21.15.12:0/295062661 shutdown_connections 2024-05-09T00:03:01.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:01.988+0000 7f8c0efdd700 1 -- 172.21.15.12:0/295062661 wait complete. 2024-05-09T00:03:02.006 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:03:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:01 smithi204 ceph-mon[47373]: pgmap v533: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 690 op/s 2024-05-09T00:03:02.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:01 smithi160 ceph-mon[46152]: pgmap v533: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 690 op/s 2024-05-09T00:03:02.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 128850 39.58 MB/sec execute 57 sec latency 159.663 ms 2024-05-09T00:03:02.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.248+0000 7f680d28c700 1 -- 172.21.15.12:0/1726705303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68081062c0 msgr2=0x7f680810de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.248+0000 7f680d28c700 1 --2- 172.21.15.12:0/1726705303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68081062c0 0x7f680810de50 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f6800009f90 tx=0x7f680000b330 comp rx=0 tx=0).stop 2024-05-09T00:03:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.248+0000 7f680d28c700 1 -- 172.21.15.12:0/1726705303 shutdown_connections 2024-05-09T00:03:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.248+0000 7f680d28c700 1 --2- 172.21.15.12:0/1726705303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68081062c0 0x7f680810de50 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.248+0000 7f680d28c700 1 --2- 172.21.15.12:0/1726705303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6808069720 0x7f6808069b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.248+0000 7f680d28c700 1 --2- 172.21.15.12:0/1726705303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f680806a0d0 0x7f6808105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.248+0000 7f680d28c700 1 -- 172.21.15.12:0/1726705303 >> 172.21.15.12:0/1726705303 conn(0x7f6808077c90 msgr2=0x7f68080780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.249+0000 7f680d28c700 1 -- 172.21.15.12:0/1726705303 shutdown_connections 2024-05-09T00:03:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.250+0000 7f680d28c700 1 -- 172.21.15.12:0/1726705303 wait complete. 2024-05-09T00:03:02.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.251+0000 7f680d28c700 1 Processor -- start 2024-05-09T00:03:02.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.251+0000 7f680d28c700 1 -- start start 2024-05-09T00:03:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.251+0000 7f680d28c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6808069720 0x7f680810ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.251+0000 7f680d28c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68081062c0 0x7f680810b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.251+0000 7f680d28c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f680819ac70 0x7f680819b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.251+0000 7f680d28c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680819b720 con 0x7f68081062c0 2024-05-09T00:03:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.251+0000 7f680d28c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680819b890 con 0x7f680819ac70 2024-05-09T00:03:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.251+0000 7f680d28c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680819ba50 con 0x7f6808069720 2024-05-09T00:03:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.252+0000 7f68067fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68081062c0 0x7f680810b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.252+0000 7f68067fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68081062c0 0x7f680810b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47984/0 (socket says 172.21.15.12:47984) 2024-05-09T00:03:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.252+0000 7f68067fc700 1 -- 172.21.15.12:0/591751019 learned_addr learned my addr 172.21.15.12:0/591751019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.253+0000 7f68067fc700 1 -- 172.21.15.12:0/591751019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6808069720 msgr2=0x7f680810ad60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:03:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.253+0000 7f68067fc700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6808069720 0x7f680810ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.253+0000 7f68067fc700 1 -- 172.21.15.12:0/591751019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f680819ac70 msgr2=0x7f680819b120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:03:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.253+0000 7f68067fc700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f680819ac70 0x7f680819b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.253+0000 7f68067fc700 1 -- 172.21.15.12:0/591751019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800009c40 con 0x7f68081062c0 2024-05-09T00:03:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.254+0000 7f68067fc700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68081062c0 0x7f680810b2a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00b520 tx=0x7f67fc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.254+0000 7f67effff700 1 -- 172.21.15.12:0/591751019 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc011830 con 0x7f68081062c0 2024-05-09T00:03:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.254+0000 7f67effff700 1 -- 172.21.15.12:0/591751019 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67fc011e50 con 0x7f68081062c0 2024-05-09T00:03:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.254+0000 7f680d28c700 1 -- 172.21.15.12:0/591751019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680819dda0 con 0x7f68081062c0 2024-05-09T00:03:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.254+0000 7f67effff700 1 -- 172.21.15.12:0/591751019 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc00f2e0 con 0x7f68081062c0 2024-05-09T00:03:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.254+0000 7f680d28c700 1 -- 172.21.15.12:0/591751019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680819e300 con 0x7f68081062c0 2024-05-09T00:03:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.260+0000 7f6819446b80 1 Processor -- start 2024-05-09T00:03:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.261+0000 7f67effff700 1 -- 172.21.15.12:0/591751019 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f67fc011990 con 0x7f68081062c0 2024-05-09T00:03:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.261+0000 7f6819446b80 1 -- start start 2024-05-09T00:03:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f67effff700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67f00a13d0 0x7f67f00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f6819446b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 0x55f9b09ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f6819446b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09ce720 0x55f9b09c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f6819446b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b09c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f6819446b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b091f150 con 0x55f9b09ce720 2024-05-09T00:03:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f6819446b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b091cb10 con 0x55f9b09cde00 2024-05-09T00:03:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f6819446b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b0918400 con 0x55f9b09c8890 2024-05-09T00:03:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f67effff700 1 -- 172.21.15.12:0/591751019 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f67fc0cc4e0 con 0x7f68081062c0 2024-05-09T00:03:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f68077fe700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67f00a13d0 0x7f67f00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f68077fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b09c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f68077fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b09c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39848/0 (socket says 172.21.15.12:39848) 2024-05-09T00:03:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f68077fe700 1 -- 172.21.15.12:0/1174776961 learned_addr learned my addr 172.21.15.12:0/1174776961 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.262+0000 7f68067fc700 1 --2- 172.21.15.12:0/1174776961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09ce720 0x55f9b09c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.263+0000 7f68077fe700 1 -- 172.21.15.12:0/1174776961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 msgr2=0x55f9b09ce1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:03:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.263+0000 7f68077fe700 1 --2- 172.21.15.12:0/1174776961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 0x55f9b09ce1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.263+0000 7f68077fe700 1 -- 172.21.15.12:0/1174776961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09ce720 msgr2=0x55f9b09c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.263+0000 7f68077fe700 1 --2- 172.21.15.12:0/1174776961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09ce720 0x55f9b09c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.263+0000 7f68077fe700 1 -- 172.21.15.12:0/1174776961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9b07993b0 con 0x55f9b09c8890 2024-05-09T00:03:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.263+0000 7f68067fc700 1 --2- 172.21.15.12:0/1174776961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09ce720 0x55f9b09c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:03:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.263+0000 7f68077fe700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67f00a13d0 0x7f67f00a3890 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6800009650 tx=0x7f6800007b30 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.264+0000 7f68077fe700 1 --2- 172.21.15.12:0/1174776961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b09c8cd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f680000cc90 tx=0x7f6800022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b3f96abea7cf5ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:02.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.266+0000 7f67ed7fa700 1 -- 172.21.15.12:0/1174776961 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000e070 con 0x55f9b09c8890 2024-05-09T00:03:02.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.266+0000 7f67ed7fa700 1 -- 172.21.15.12:0/1174776961 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68000073e0 con 0x55f9b09c8890 2024-05-09T00:03:02.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.266+0000 7f67ed7fa700 1 -- 172.21.15.12:0/1174776961 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000c340 con 0x55f9b09c8890 2024-05-09T00:03:02.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.267+0000 7f6819446b80 1 -- 172.21.15.12:0/1174776961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 msgr2=0x55f9b09c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:02.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.267+0000 7f6819446b80 1 --2- 172.21.15.12:0/1174776961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b09c8cd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f680000cc90 tx=0x7f6800022000 comp rx=0 tx=0).stop 2024-05-09T00:03:02.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.267+0000 7f6819446b80 1 -- 172.21.15.12:0/1174776961 shutdown_connections 2024-05-09T00:03:02.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.267+0000 7f6819446b80 1 --2- 172.21.15.12:0/1174776961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b09c8cd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.267+0000 7f6819446b80 1 --2- 172.21.15.12:0/1174776961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 0x55f9b09ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.267+0000 7f6819446b80 1 --2- 172.21.15.12:0/1174776961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09ce720 0x55f9b09c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.267+0000 7f6819446b80 1 -- 172.21.15.12:0/1174776961 >> 172.21.15.12:0/1174776961 conn(0x55f9b0926210 msgr2=0x55f9b09c2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:02.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.268+0000 7f6819446b80 1 -- 172.21.15.12:0/1174776961 shutdown_connections 2024-05-09T00:03:02.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.269+0000 7f6819446b80 1 -- 172.21.15.12:0/1174776961 wait complete. 2024-05-09T00:03:02.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.269+0000 7f6819446b80 1 -- start start 2024-05-09T00:03:02.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.269+0000 7f6819446b80 1 Processor -- start 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f6819446b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09c8890 0x55f9b09d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f6819446b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 0x55f9b09d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f6819446b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09ce720 0x55f9b09d25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f6819446b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b08b4560 con 0x55f9b09c8890 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f6819446b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b08b4200 con 0x55f9b09cde00 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f6819446b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b079b6a0 con 0x55f9b09ce720 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f6819446b80 10 client.0 _ll_register_callbacks cb 0x55f9b08bdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f68077fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09ce720 0x55f9b09d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f68067fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 0x55f9b09d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f68077fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09ce720 0x55f9b09d25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39860/0 (socket says 172.21.15.12:39860) 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f68077fe700 1 -- 172.21.15.12:0/790492300 learned_addr learned my addr 172.21.15.12:0/790492300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f68067fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 0x55f9b09d20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:53148/0 (socket says 172.21.15.12:53148) 2024-05-09T00:03:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.270+0000 7f6806ffd700 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09c8890 0x55f9b09d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f68067fc700 1 -- 172.21.15.12:0/790492300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09ce720 msgr2=0x55f9b09d25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f68067fc700 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09ce720 0x55f9b09d25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f68067fc700 1 -- 172.21.15.12:0/790492300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09c8890 msgr2=0x55f9b09d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f68067fc700 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09c8890 0x55f9b09d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f68067fc700 1 -- 172.21.15.12:0/790492300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6800025070 con 0x55f9b09cde00 2024-05-09T00:03:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f68077fe700 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09ce720 0x55f9b09d25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:02.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f68067fc700 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 0x55f9b09d20b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0c9d20 tx=0x7f67fc010f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a77bda2cccbf0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:02.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f67eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:03:02.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f67eaff5700 1 -- 172.21.15.12:0/790492300 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc09d510 con 0x55f9b09cde00 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f67eaff5700 1 -- 172.21.15.12:0/790492300 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f67fc0a1b60 con 0x55f9b09cde00 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.271+0000 7f6819446b80 1 -- 172.21.15.12:0/790492300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9b0871270 con 0x55f9b09cde00 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.272+0000 7f67eaff5700 1 -- 172.21.15.12:0/790492300 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67fc09d170 con 0x55f9b09cde00 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.272+0000 7f6819446b80 10 client.29771 fetch_fsmap learned FSMap version 69 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.272+0000 7f6819446b80 1 -- 172.21.15.12:0/790492300 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55f9b0745b60 con 0x55f9b09cde00 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.272+0000 7f67eaff5700 1 -- 172.21.15.12:0/790492300 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9b0871270 con 0x55f9b09cde00 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.272+0000 7f67eaff5700 1 -- 172.21.15.12:0/790492300 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67fc099940 con 0x55f9b09cde00 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.273+0000 7f6819446b80 10 client.29771 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.273+0000 7f6819446b80 10 client.29771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.273+0000 7f6819446b80 20 client.29771 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.273+0000 7f6819446b80 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f9b09e2180 0x55f9b09e45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.273+0000 7f6819446b80 4 client.29771 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.273+0000 7f6819446b80 1 -- 172.21.15.12:0/790492300 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9b0997810 con 0x55f9b09e2180 2024-05-09T00:03:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.273+0000 7f6806ffd700 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f9b09e2180 0x55f9b09e45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.274+0000 7f67eaff5700 10 client.29771 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.274+0000 7f6806ffd700 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f9b09e2180 0x55f9b09e45a0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd2e8d5b01ce4324 server_cookie=8ecdb5758b53f27d in_seq=0 out_seq=0 2024-05-09T00:03:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.276+0000 7f67eaff5700 1 -- 172.21.15.12:0/790492300 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9b0997810 con 0x55f9b09e2180 2024-05-09T00:03:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.276+0000 7f67eaff5700 10 client.29771 handle_command_reply: tid=0 2024-05-09T00:03:02.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.277+0000 7f6819446b80 1 client.29771 shutdown 2024-05-09T00:03:02.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.278+0000 7f6819446b80 1 -- 172.21.15.12:0/790492300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 msgr2=0x55f9b09d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:02.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.278+0000 7f6819446b80 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 0x55f9b09d20b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0c9d20 tx=0x7f67fc010f30 comp rx=0 tx=0).stop 2024-05-09T00:03:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.279+0000 7f6819446b80 1 -- 172.21.15.12:0/790492300 shutdown_connections 2024-05-09T00:03:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.279+0000 7f6819446b80 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09ce720 0x55f9b09d25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.279+0000 7f6819446b80 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09c8890 0x55f9b09d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.279+0000 7f6819446b80 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f9b09e2180 0x55f9b09e45a0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.279+0000 7f6819446b80 1 --2- 172.21.15.12:0/790492300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09cde00 0x55f9b09d20b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.279+0000 7f6819446b80 1 -- 172.21.15.12:0/790492300 >> 172.21.15.12:0/790492300 conn(0x55f9b0926210 msgr2=0x55f9b09c25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.279+0000 7f67eaff5700 0 client.29771 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.280+0000 7f6819446b80 1 -- 172.21.15.12:0/790492300 shutdown_connections 2024-05-09T00:03:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.280+0000 7f6819446b80 1 -- 172.21.15.12:0/790492300 wait complete. 2024-05-09T00:03:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.281+0000 7f6819446b80 20 client.29771 trim_cache size 0 max 16384 2024-05-09T00:03:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.311+0000 7f6819446b80 1 Processor -- start 2024-05-09T00:03:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.311+0000 7f6819446b80 1 -- start start 2024-05-09T00:03:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.311+0000 7f6819446b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b09c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.311+0000 7f6819446b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09cde00 0x55f9b09c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.311+0000 7f6819446b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.311+0000 7f6819446b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b09bd870 con 0x55f9b09cde00 2024-05-09T00:03:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.311+0000 7f6819446b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b0997810 con 0x55f9b09ce720 2024-05-09T00:03:02.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.311+0000 7f6819446b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b09bfdc0 con 0x55f9b09c8890 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.312+0000 7f68077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.312+0000 7f68077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:53160/0 (socket says 172.21.15.12:53160) 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.312+0000 7f68077fe700 1 -- 172.21.15.12:0/598262204 learned_addr learned my addr 172.21.15.12:0/598262204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.312+0000 7f68077fe700 1 -- 172.21.15.12:0/598262204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 msgr2=0x55f9b09c8350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.312+0000 7f68077fe700 1 --2- 172.21.15.12:0/598262204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b09c8350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.312+0000 7f68077fe700 1 -- 172.21.15.12:0/598262204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09cde00 msgr2=0x55f9b09c2d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.312+0000 7f68077fe700 1 --2- 172.21.15.12:0/598262204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09cde00 0x55f9b09c2d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.312+0000 7f68077fe700 1 -- 172.21.15.12:0/598262204 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9b08de560 con 0x55f9b09ce720 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.312+0000 7f68077fe700 1 --2- 172.21.15.12:0/598262204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d9170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f680000b820 tx=0x7f680000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=774773040a84d2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.313+0000 7f67edffb700 1 -- 172.21.15.12:0/598262204 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000e040 con 0x55f9b09ce720 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.313+0000 7f67edffb700 1 -- 172.21.15.12:0/598262204 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6800007920 con 0x55f9b09ce720 2024-05-09T00:03:02.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.313+0000 7f67edffb700 1 -- 172.21.15.12:0/598262204 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000be50 con 0x55f9b09ce720 2024-05-09T00:03:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 -- 172.21.15.12:0/598262204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 msgr2=0x55f9b09d9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 --2- 172.21.15.12:0/598262204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d9170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f680000b820 tx=0x7f680000b330 comp rx=0 tx=0).stop 2024-05-09T00:03:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 -- 172.21.15.12:0/598262204 shutdown_connections 2024-05-09T00:03:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 --2- 172.21.15.12:0/598262204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d9170 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 --2- 172.21.15.12:0/598262204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b09c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 --2- 172.21.15.12:0/598262204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09cde00 0x55f9b09c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 -- 172.21.15.12:0/598262204 >> 172.21.15.12:0/598262204 conn(0x55f9b0926210 msgr2=0x55f9b09cbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 -- 172.21.15.12:0/598262204 shutdown_connections 2024-05-09T00:03:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 -- 172.21.15.12:0/598262204 wait complete. 2024-05-09T00:03:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 -- start start 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 Processor -- start 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b0a40cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09cde00 0x55f9b0a41230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b09d8f10 con 0x55f9b09cde00 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b09d9050 con 0x55f9b09ce720 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b09d9210 con 0x55f9b09c8890 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.314+0000 7f6819446b80 10 client.0 _ll_register_callbacks cb 0x55f9b08bdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.315+0000 7f68077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.315+0000 7f68077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:53168/0 (socket says 172.21.15.12:53168) 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.315+0000 7f68077fe700 1 -- 172.21.15.12:0/1261993493 learned_addr learned my addr 172.21.15.12:0/1261993493 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.315+0000 7f68077fe700 1 -- 172.21.15.12:0/1261993493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 msgr2=0x55f9b0a40cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.315+0000 7f68077fe700 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b0a40cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.315+0000 7f68077fe700 1 -- 172.21.15.12:0/1261993493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09cde00 msgr2=0x55f9b0a41230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:02.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.315+0000 7f68077fe700 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09cde00 0x55f9b0a41230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.315+0000 7f68077fe700 1 -- 172.21.15.12:0/1261993493 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680002d040 con 0x55f9b09ce720 2024-05-09T00:03:02.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.315+0000 7f68077fe700 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d8940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6800009550 tx=0x7f6800004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b414b294a56541b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:02.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.315+0000 7f67eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:03:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.316+0000 7f67eaff5700 1 -- 172.21.15.12:0/1261993493 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000e040 con 0x55f9b09ce720 2024-05-09T00:03:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.317+0000 7f67eaff5700 1 -- 172.21.15.12:0/1261993493 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6800003ef0 con 0x55f9b09ce720 2024-05-09T00:03:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.317+0000 7f6819446b80 1 -- 172.21.15.12:0/1261993493 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68000073e0 con 0x55f9b09ce720 2024-05-09T00:03:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.317+0000 7f67eaff5700 1 -- 172.21.15.12:0/1261993493 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68000040b0 con 0x55f9b09ce720 2024-05-09T00:03:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.317+0000 7f6819446b80 10 client.29777 fetch_fsmap learned FSMap version 69 2024-05-09T00:03:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.317+0000 7f6819446b80 1 -- 172.21.15.12:0/1261993493 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55f9b079dcd0 con 0x55f9b09ce720 2024-05-09T00:03:02.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.317+0000 7f67eaff5700 1 -- 172.21.15.12:0/1261993493 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68000073e0 con 0x55f9b09ce720 2024-05-09T00:03:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.317+0000 7f67eaff5700 1 -- 172.21.15.12:0/1261993493 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f680003aa30 con 0x55f9b09ce720 2024-05-09T00:03:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.318+0000 7f6819446b80 10 client.29777 fetch_fsmap finished waiting for FSMap version 69 2024-05-09T00:03:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.318+0000 7f6819446b80 10 client.29777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.318+0000 7f6819446b80 20 client.29777 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.318+0000 7f6819446b80 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f9b0a49920 0x55f9b0a4bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.318+0000 7f6819446b80 4 client.29777 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:03:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.318+0000 7f6819446b80 1 -- 172.21.15.12:0/1261993493 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f9b0910ec0 con 0x55f9b0a49920 2024-05-09T00:03:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.319+0000 7f6806ffd700 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f9b0a49920 0x55f9b0a4bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.319+0000 7f6806ffd700 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f9b0a49920 0x55f9b0a4bd40 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e7d68bcbcfef1 server_cookie=87b25297f9d1feac in_seq=0 out_seq=0 2024-05-09T00:03:02.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.319+0000 7f67eaff5700 10 client.29777 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:02.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f67eaff5700 1 -- 172.21.15.12:0/1261993493 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f9b0910ec0 con 0x55f9b0a49920 2024-05-09T00:03:02.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f67eaff5700 10 client.29777 handle_command_reply: tid=0 2024-05-09T00:03:02.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 client.29777 shutdown 2024-05-09T00:03:02.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 -- 172.21.15.12:0/1261993493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 msgr2=0x55f9b09d8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:02.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d8940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6800009550 tx=0x7f6800004f60 comp rx=0 tx=0).stop 2024-05-09T00:03:02.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 -- 172.21.15.12:0/1261993493 shutdown_connections 2024-05-09T00:03:02.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f9b0a49920 0x55f9b0a4bd40 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9b09ce720 0x55f9b09d8940 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9b09c8890 0x55f9b0a40cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 --2- 172.21.15.12:0/1261993493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9b09cde00 0x55f9b0a41230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 -- 172.21.15.12:0/1261993493 >> 172.21.15.12:0/1261993493 conn(0x55f9b0926210 msgr2=0x55f9b09ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:02.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f67eaff5700 0 client.29777 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:02.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 -- 172.21.15.12:0/1261993493 shutdown_connections 2024-05-09T00:03:02.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.320+0000 7f6819446b80 1 -- 172.21.15.12:0/1261993493 wait complete. 2024-05-09T00:03:02.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.321+0000 7f6819446b80 20 client.29777 trim_cache size 0 max 16384 2024-05-09T00:03:02.318 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:03:02.318 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:03:02.318 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:03:02.319 INFO:teuthology.orchestra.run.smithi012.stdout: "52d03b32-465a-46dc-821d-c9f28565796d": { 2024-05-09T00:03:02.319 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:03:02.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "52d03b32-465a-46dc-821d-c9f28565796d", 2024-05-09T00:03:02.319 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:03:02.319 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:03:02.319 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:03:02.319 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:03:02.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.323+0000 7f680d28c700 1 -- 172.21.15.12:0/591751019 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67f00a13d0 msgr2=0x7f67f00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:02.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.323+0000 7f680d28c700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67f00a13d0 0x7f67f00a3890 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6800009650 tx=0x7f6800007b30 comp rx=0 tx=0).stop 2024-05-09T00:03:02.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.323+0000 7f680d28c700 1 -- 172.21.15.12:0/591751019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68081062c0 msgr2=0x7f680810b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:02.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.323+0000 7f680d28c700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68081062c0 0x7f680810b2a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00b520 tx=0x7f67fc00b8e0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.323+0000 7f680d28c700 1 -- 172.21.15.12:0/591751019 shutdown_connections 2024-05-09T00:03:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.323+0000 7f680d28c700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f67f00a13d0 0x7f67f00a3890 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.323+0000 7f680d28c700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f680819ac70 0x7f680819b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.323+0000 7f680d28c700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6808069720 0x7f680810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.323+0000 7f680d28c700 1 --2- 172.21.15.12:0/591751019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68081062c0 0x7f680810b2a0 secure :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00b520 tx=0x7f67fc00b8e0 comp rx=0 tx=0).stop 2024-05-09T00:03:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.323+0000 7f680d28c700 1 -- 172.21.15.12:0/591751019 >> 172.21.15.12:0/591751019 conn(0x7f6808077c90 msgr2=0x7f680810c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.325+0000 7f680d28c700 1 -- 172.21.15.12:0/591751019 shutdown_connections 2024-05-09T00:03:02.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:02.325+0000 7f680d28c700 1 -- 172.21.15.12:0/591751019 wait complete. 2024-05-09T00:03:02.339 INFO:tasks.cephfs.filesystem:scrub status for tag:52d03b32-465a-46dc-821d-c9f28565796d - {'path': '/', 'tag': '52d03b32-465a-46dc-821d-c9f28565796d', 'options': 'recursive,force'} 2024-05-09T00:03:02.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:02 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:03:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:02 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:02 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:03:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:02 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:03:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:02 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:02.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:02 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:03.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:02 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:03:03.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:02 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:03.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:02 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:03:03.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:02 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:03:03.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:02 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:03.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:02 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:03.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:02 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:03:03.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:02 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:03.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:02 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:03:03.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:02 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:03:03.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:02 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:03.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:02 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:03.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129269 39.39 MB/sec execute 58 sec latency 120.762 ms 2024-05-09T00:03:04.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:03 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:03:04.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:03 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:04.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:03 smithi204 ceph-mon[47373]: pgmap v534: 97 pgs: 97 active+clean; 990 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 25 MiB/s wr, 1.39k op/s 2024-05-09T00:03:04.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:03 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:03:04.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:03 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:04.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:03 smithi160 ceph-mon[46152]: pgmap v534: 97 pgs: 97 active+clean; 990 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 25 MiB/s wr, 1.39k op/s 2024-05-09T00:03:04.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:03 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:03:04.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:03 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:04.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:03 smithi012 ceph-mon[42503]: pgmap v534: 97 pgs: 97 active+clean; 990 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 25 MiB/s wr, 1.39k op/s 2024-05-09T00:03:04.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129808 39.10 MB/sec execute 59 sec latency 86.889 ms 2024-05-09T00:03:05.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:04 smithi204 ceph-mon[47373]: osdmap e117: 12 total, 12 up, 12 in 2024-05-09T00:03:05.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:04 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:05.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:04 smithi160 ceph-mon[46152]: osdmap e117: 12 total, 12 up, 12 in 2024-05-09T00:03:05.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:04 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:05.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:04 smithi012 ceph-mon[42503]: osdmap e117: 12 total, 12 up, 12 in 2024-05-09T00:03:05.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:04 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:05.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 130228 38.75 MB/sec execute 60 sec latency 82.992 ms 2024-05-09T00:03:06.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:05 smithi204 ceph-mon[47373]: pgmap v536: 97 pgs: 97 active+clean; 990 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 26 MiB/s wr, 1.38k op/s 2024-05-09T00:03:06.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:05 smithi160 ceph-mon[46152]: pgmap v536: 97 pgs: 97 active+clean; 990 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 26 MiB/s wr, 1.38k op/s 2024-05-09T00:03:06.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:05 smithi012 ceph-mon[42503]: pgmap v536: 97 pgs: 97 active+clean; 990 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 26 MiB/s wr, 1.38k op/s 2024-05-09T00:03:06.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 130654 38.62 MB/sec execute 61 sec latency 155.106 ms 2024-05-09T00:03:07.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131379 38.60 MB/sec execute 62 sec latency 70.179 ms 2024-05-09T00:03:07.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:07 smithi012 ceph-mon[42503]: pgmap v537: 97 pgs: 97 active+clean; 1003 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 26 MiB/s wr, 1.39k op/s 2024-05-09T00:03:08.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:07 smithi204 ceph-mon[47373]: pgmap v537: 97 pgs: 97 active+clean; 1003 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 26 MiB/s wr, 1.39k op/s 2024-05-09T00:03:08.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:07 smithi160 ceph-mon[46152]: pgmap v537: 97 pgs: 97 active+clean; 1003 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 26 MiB/s wr, 1.39k op/s 2024-05-09T00:03:08.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 132134 38.65 MB/sec execute 63 sec latency 56.814 ms 2024-05-09T00:03:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.425+0000 7f331a96b700 1 -- 172.21.15.12:0/3318411287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33141009c0 msgr2=0x7f3314104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.425+0000 7f331a96b700 1 --2- 172.21.15.12:0/3318411287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33141009c0 0x7f3314104a00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3308009990 tx=0x7f3308009ca0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.425+0000 7f331a96b700 1 -- 172.21.15.12:0/3318411287 shutdown_connections 2024-05-09T00:03:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.425+0000 7f331a96b700 1 --2- 172.21.15.12:0/3318411287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314105130 0x7f331410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.425+0000 7f331a96b700 1 --2- 172.21.15.12:0/3318411287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3314100010 0x7f33141003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.425+0000 7f331a96b700 1 --2- 172.21.15.12:0/3318411287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33141009c0 0x7f3314104a00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.425+0000 7f331a96b700 1 -- 172.21.15.12:0/3318411287 >> 172.21.15.12:0/3318411287 conn(0x7f33140fb870 msgr2=0x7f33140fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.425+0000 7f331a96b700 1 -- 172.21.15.12:0/3318411287 shutdown_connections 2024-05-09T00:03:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.425+0000 7f331a96b700 1 -- 172.21.15.12:0/3318411287 wait complete. 2024-05-09T00:03:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.426+0000 7f331a96b700 1 Processor -- start 2024-05-09T00:03:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.426+0000 7f331a96b700 1 -- start start 2024-05-09T00:03:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.426+0000 7f331a96b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3314100010 0x7f3314195220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.426+0000 7f331a96b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314105130 0x7f3314195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.426+0000 7f331a96b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3314199ab0 0x7f3314199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.426+0000 7f331a96b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331419a530 con 0x7f3314199ab0 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.426+0000 7f331a96b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331419a6a0 con 0x7f3314105130 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.426+0000 7f331a96b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331419a860 con 0x7f3314100010 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3313fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3314100010 0x7f3314195220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3318f08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3314199ab0 0x7f3314199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f33137fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314105130 0x7f3314195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3318f08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3314199ab0 0x7f3314199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54406/0 (socket says 172.21.15.12:54406) 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3313fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3314100010 0x7f3314195220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33096/0 (socket says 172.21.15.12:33096) 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3318f08700 1 -- 172.21.15.12:0/382379302 learned_addr learned my addr 172.21.15.12:0/382379302 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3318f08700 1 -- 172.21.15.12:0/382379302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3314100010 msgr2=0x7f3314195220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3318f08700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3314100010 0x7f3314195220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3318f08700 1 -- 172.21.15.12:0/382379302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314105130 msgr2=0x7f3314195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3318f08700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314105130 0x7f3314195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3318f08700 1 -- 172.21.15.12:0/382379302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33080095f0 con 0x7f3314199ab0 2024-05-09T00:03:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3313fff700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3314100010 0x7f3314195220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f3318f08700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3314199ab0 0x7f3314199f30 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f330c00ef30 tx=0x7f330c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f33117fa700 1 -- 172.21.15.12:0/382379302 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c018070 con 0x7f3314199ab0 2024-05-09T00:03:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f33117fa700 1 -- 172.21.15.12:0/382379302 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f330c00f040 con 0x7f3314199ab0 2024-05-09T00:03:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f331a96b700 1 -- 172.21.15.12:0/382379302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33141a1d60 con 0x7f3314199ab0 2024-05-09T00:03:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.427+0000 7f33117fa700 1 -- 172.21.15.12:0/382379302 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c00cd00 con 0x7f3314199ab0 2024-05-09T00:03:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.428+0000 7f331a96b700 1 -- 172.21.15.12:0/382379302 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33141a2290 con 0x7f3314199ab0 2024-05-09T00:03:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.430+0000 7f33117fa700 1 -- 172.21.15.12:0/382379302 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f330c003a00 con 0x7f3314199ab0 2024-05-09T00:03:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.431+0000 7f33117fa700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f32fc0a1310 0x7f32fc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.431+0000 7f3313fff700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f32fc0a1310 0x7f32fc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.431+0000 7f33117fa700 1 -- 172.21.15.12:0/382379302 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f330c0cc730 con 0x7f3314199ab0 2024-05-09T00:03:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.432+0000 7f3313fff700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f32fc0a1310 0x7f32fc0a37d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3304011490 tx=0x7f3304009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.433+0000 7f3326b25b80 1 Processor -- start 2024-05-09T00:03:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.433+0000 7f3326b25b80 1 -- start start 2024-05-09T00:03:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef27950 0x564b3ef27d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 0x564b3ef21f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f3326b25b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ee790f0 con 0x564b3ef28270 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f3326b25b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ee76ab0 con 0x564b3ef22480 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f3326b25b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ee723a0 con 0x564b3ef27950 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f33137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 0x564b3ef21f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f33137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 0x564b3ef21f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54412/0 (socket says 172.21.15.12:54412) 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f33137fe700 1 -- 172.21.15.12:0/439149117 learned_addr learned my addr 172.21.15.12:0/439149117 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f3313fff700 1 --2- 172.21.15.12:0/439149117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef27950 0x564b3ef27d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f3318f08700 1 --2- 172.21.15.12:0/439149117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f33137fe700 1 -- 172.21.15.12:0/439149117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef27950 msgr2=0x564b3ef27d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f33137fe700 1 --2- 172.21.15.12:0/439149117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef27950 0x564b3ef27d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f33137fe700 1 -- 172.21.15.12:0/439149117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 msgr2=0x564b3ef228a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f33137fe700 1 --2- 172.21.15.12:0/439149117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef228a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f33137fe700 1 -- 172.21.15.12:0/439149117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b3ecf3350 con 0x564b3ef28270 2024-05-09T00:03:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.434+0000 7f3313fff700 1 --2- 172.21.15.12:0/439149117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef27950 0x564b3ef27d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.435+0000 7f33137fe700 1 --2- 172.21.15.12:0/439149117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 0x564b3ef21f40 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3308009990 tx=0x7f330800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976953342a085188 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.436+0000 7f32fa7fc700 1 -- 172.21.15.12:0/439149117 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308003680 con 0x564b3ef28270 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.436+0000 7f32fa7fc700 1 -- 172.21.15.12:0/439149117 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f330800d930 con 0x564b3ef28270 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.436+0000 7f32fa7fc700 1 -- 172.21.15.12:0/439149117 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308005690 con 0x564b3ef28270 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.437+0000 7f3326b25b80 1 -- 172.21.15.12:0/439149117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 msgr2=0x564b3ef21f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.437+0000 7f3326b25b80 1 --2- 172.21.15.12:0/439149117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 0x564b3ef21f40 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3308009990 tx=0x7f330800d040 comp rx=0 tx=0).stop 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.437+0000 7f3326b25b80 1 -- 172.21.15.12:0/439149117 shutdown_connections 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.437+0000 7f3326b25b80 1 --2- 172.21.15.12:0/439149117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef228a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.437+0000 7f3326b25b80 1 --2- 172.21.15.12:0/439149117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef27950 0x564b3ef27d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.437+0000 7f3326b25b80 1 --2- 172.21.15.12:0/439149117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 0x564b3ef21f40 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.437+0000 7f3326b25b80 1 -- 172.21.15.12:0/439149117 >> 172.21.15.12:0/439149117 conn(0x564b3ee801b0 msgr2=0x564b3ef1c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.437+0000 7f3326b25b80 1 -- 172.21.15.12:0/439149117 shutdown_connections 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.437+0000 7f3326b25b80 1 -- 172.21.15.12:0/439149117 wait complete. 2024-05-09T00:03:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.438+0000 7f3326b25b80 1 -- start start 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.438+0000 7f3326b25b80 1 Processor -- start 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.438+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef22480 0x564b3ef2e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef27950 0x564b3ef2b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 0x564b3ef2bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f3326b25b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ee0e500 con 0x564b3ef28270 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f3326b25b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ee0e1a0 con 0x564b3ef27950 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f3326b25b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ecf5640 con 0x564b3ef22480 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f3326b25b80 10 client.0 _ll_register_callbacks cb 0x564b3ee17ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f33137fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef27950 0x564b3ef2b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f33137fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef27950 0x564b3ef2b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:38350/0 (socket says 172.21.15.12:38350) 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f33137fe700 1 -- 172.21.15.12:0/896235461 learned_addr learned my addr 172.21.15.12:0/896235461 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f3313fff700 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef22480 0x564b3ef2e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f33137fe700 1 -- 172.21.15.12:0/896235461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef22480 msgr2=0x564b3ef2e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f33137fe700 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef22480 0x564b3ef2e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f33137fe700 1 -- 172.21.15.12:0/896235461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 msgr2=0x564b3ef2bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f33137fe700 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 0x564b3ef2bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.439+0000 7f33137fe700 1 -- 172.21.15.12:0/896235461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3304017070 con 0x564b3ef27950 2024-05-09T00:03:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.440+0000 7f33137fe700 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef27950 0x564b3ef2b570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3308009350 tx=0x7f3308005550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f9dd9af0a062dba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.440+0000 7f32f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:03:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.442+0000 7f32f37fe700 1 -- 172.21.15.12:0/896235461 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308003680 con 0x564b3ef27950 2024-05-09T00:03:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.442+0000 7f32f37fe700 1 -- 172.21.15.12:0/896235461 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f33080037e0 con 0x564b3ef27950 2024-05-09T00:03:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.443+0000 7f3326b25b80 1 -- 172.21.15.12:0/896235461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b3edcb210 con 0x564b3ef27950 2024-05-09T00:03:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.443+0000 7f32f37fe700 1 -- 172.21.15.12:0/896235461 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33080039a0 con 0x564b3ef27950 2024-05-09T00:03:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.443+0000 7f3326b25b80 10 client.29789 fetch_fsmap learned FSMap version 70 2024-05-09T00:03:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.443+0000 7f3326b25b80 1 -- 172.21.15.12:0/896235461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7f331410d520 con 0x564b3ef27950 2024-05-09T00:03:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.444+0000 7f32f37fe700 1 -- 172.21.15.12:0/896235461 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b3edcb210 con 0x564b3ef27950 2024-05-09T00:03:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.445+0000 7f32f37fe700 1 -- 172.21.15.12:0/896235461 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 70) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f330801f470 con 0x564b3ef27950 2024-05-09T00:03:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.446+0000 7f3326b25b80 10 client.29789 fetch_fsmap finished waiting for FSMap version 70 2024-05-09T00:03:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.446+0000 7f3326b25b80 10 client.29789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.446+0000 7f3326b25b80 20 client.29789 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.446+0000 7f3326b25b80 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564b3ef43720 0x564b3ef45b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.446+0000 7f3326b25b80 4 client.29789 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.446+0000 7f3326b25b80 1 -- 172.21.15.12:0/896235461 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b3eef17b0 con 0x564b3ef43720 2024-05-09T00:03:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.446+0000 7f3313fff700 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564b3ef43720 0x564b3ef45b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.447+0000 7f32f37fe700 10 client.29789 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.447+0000 7f3313fff700 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564b3ef43720 0x564b3ef45b10 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30408eeefbff8f7 server_cookie=7aa0c283b604513 in_seq=0 out_seq=0 2024-05-09T00:03:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.449+0000 7f32f37fe700 1 -- 172.21.15.12:0/896235461 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b3eef17b0 con 0x564b3ef43720 2024-05-09T00:03:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.449+0000 7f32f37fe700 10 client.29789 handle_command_reply: tid=0 2024-05-09T00:03:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.450+0000 7f3326b25b80 1 client.29789 shutdown 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.450+0000 7f3326b25b80 1 -- 172.21.15.12:0/896235461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef27950 msgr2=0x564b3ef2b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.450+0000 7f3326b25b80 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef27950 0x564b3ef2b570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3308009350 tx=0x7f3308005550 comp rx=0 tx=0).stop 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.450+0000 7f3326b25b80 1 -- 172.21.15.12:0/896235461 shutdown_connections 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.450+0000 7f3326b25b80 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef28270 0x564b3ef2bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.450+0000 7f3326b25b80 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef22480 0x564b3ef2e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.450+0000 7f3326b25b80 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564b3ef43720 0x564b3ef45b10 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.450+0000 7f3326b25b80 1 --2- 172.21.15.12:0/896235461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef27950 0x564b3ef2b570 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.450+0000 7f3326b25b80 1 -- 172.21.15.12:0/896235461 >> 172.21.15.12:0/896235461 conn(0x564b3ee801b0 msgr2=0x564b3ef264a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.450+0000 7f32f37fe700 0 client.29789 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.451+0000 7f3326b25b80 1 -- 172.21.15.12:0/896235461 shutdown_connections 2024-05-09T00:03:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.451+0000 7f3326b25b80 1 -- 172.21.15.12:0/896235461 wait complete. 2024-05-09T00:03:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.451+0000 7f3326b25b80 20 client.29789 trim_cache size 0 max 16384 2024-05-09T00:03:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.473+0000 7f3326b25b80 1 Processor -- start 2024-05-09T00:03:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.473+0000 7f3326b25b80 1 -- start start 2024-05-09T00:03:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef50f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 0x564b3ef22860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef28270 0x564b3ef2bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3326b25b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ef1e340 con 0x564b3ef27950 2024-05-09T00:03:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3326b25b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ef14a40 con 0x564b3ef22480 2024-05-09T00:03:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3326b25b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ef66d90 con 0x564b3ef28270 2024-05-09T00:03:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f33137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 0x564b3ef22860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f33137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 0x564b3ef22860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54416/0 (socket says 172.21.15.12:54416) 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3313fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef50f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f33137fe700 1 -- 172.21.15.12:0/3159101431 learned_addr learned my addr 172.21.15.12:0/3159101431 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3318f08700 1 --2- 172.21.15.12:0/3159101431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef28270 0x564b3ef2bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3313fff700 1 -- 172.21.15.12:0/3159101431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef28270 msgr2=0x564b3ef2bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3313fff700 1 --2- 172.21.15.12:0/3159101431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef28270 0x564b3ef2bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3313fff700 1 -- 172.21.15.12:0/3159101431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 msgr2=0x564b3ef22860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3313fff700 1 --2- 172.21.15.12:0/3159101431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 0x564b3ef22860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3313fff700 1 -- 172.21.15.12:0/3159101431 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b3ec9fb00 con 0x564b3ef22480 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f33137fe700 1 --2- 172.21.15.12:0/3159101431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 0x564b3ef22860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.474+0000 7f3313fff700 1 --2- 172.21.15.12:0/3159101431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef50f40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x564b3ef460f0 tx=0x7f3304014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b20234749592d3d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.475+0000 7f32faffd700 1 -- 172.21.15.12:0/3159101431 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304024040 con 0x564b3ef22480 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.475+0000 7f32faffd700 1 -- 172.21.15.12:0/3159101431 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3304029870 con 0x564b3ef22480 2024-05-09T00:03:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.475+0000 7f32faffd700 1 -- 172.21.15.12:0/3159101431 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330401ed30 con 0x564b3ef22480 2024-05-09T00:03:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.476+0000 7f3326b25b80 1 -- 172.21.15.12:0/3159101431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 msgr2=0x564b3ef50f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.476+0000 7f3326b25b80 1 --2- 172.21.15.12:0/3159101431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef50f40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x564b3ef460f0 tx=0x7f3304014000 comp rx=0 tx=0).stop 2024-05-09T00:03:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.476+0000 7f3326b25b80 1 -- 172.21.15.12:0/3159101431 shutdown_connections 2024-05-09T00:03:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.476+0000 7f3326b25b80 1 --2- 172.21.15.12:0/3159101431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef28270 0x564b3ef2bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.476+0000 7f3326b25b80 1 --2- 172.21.15.12:0/3159101431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef50f40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.476+0000 7f3326b25b80 1 --2- 172.21.15.12:0/3159101431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 0x564b3ef22860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.476+0000 7f3326b25b80 1 -- 172.21.15.12:0/3159101431 >> 172.21.15.12:0/3159101431 conn(0x564b3ee801b0 msgr2=0x564b3ef29e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.476+0000 7f3326b25b80 1 -- 172.21.15.12:0/3159101431 shutdown_connections 2024-05-09T00:03:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.476+0000 7f3326b25b80 1 -- 172.21.15.12:0/3159101431 wait complete. 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.476+0000 7f3326b25b80 1 -- start start 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3326b25b80 1 Processor -- start 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef4a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 0x564b3ef25380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3326b25b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef28270 0x564b3ef258c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3326b25b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ef25fb0 con 0x564b3ef27950 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3326b25b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ef26120 con 0x564b3ef22480 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3326b25b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3ef262e0 con 0x564b3ef28270 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3326b25b80 10 client.0 _ll_register_callbacks cb 0x564b3ee17ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f33137fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 0x564b3ef25380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3313fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef4a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3313fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef4a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:38370/0 (socket says 172.21.15.12:38370) 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3313fff700 1 -- 172.21.15.12:0/705036652 learned_addr learned my addr 172.21.15.12:0/705036652 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3318f08700 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef28270 0x564b3ef258c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3313fff700 1 -- 172.21.15.12:0/705036652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef28270 msgr2=0x564b3ef258c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3313fff700 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef28270 0x564b3ef258c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3313fff700 1 -- 172.21.15.12:0/705036652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 msgr2=0x564b3ef25380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.477+0000 7f3313fff700 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 0x564b3ef25380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.478+0000 7f3313fff700 1 -- 172.21.15.12:0/705036652 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331410d520 con 0x564b3ef22480 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.478+0000 7f3313fff700 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef4a7d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f330401e950 tx=0x7f3304003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3fa682149ae560d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.478+0000 7f32f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.478+0000 7f32f37fe700 1 -- 172.21.15.12:0/705036652 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304024050 con 0x564b3ef22480 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.478+0000 7f32f37fe700 1 -- 172.21.15.12:0/705036652 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f330401e5f0 con 0x564b3ef22480 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.478+0000 7f3326b25b80 1 -- 172.21.15.12:0/705036652 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f330800d930 con 0x564b3ef22480 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.478+0000 7f32f37fe700 1 -- 172.21.15.12:0/705036652 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3304029700 con 0x564b3ef22480 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.478+0000 7f3326b25b80 10 client.29795 fetch_fsmap learned FSMap version 70 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.478+0000 7f3326b25b80 1 -- 172.21.15.12:0/705036652 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7f3304004480 con 0x564b3ef22480 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.479+0000 7f32f37fe700 1 -- 172.21.15.12:0/705036652 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f330800d930 con 0x564b3ef22480 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.479+0000 7f32f37fe700 1 -- 172.21.15.12:0/705036652 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 70) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3304004750 con 0x564b3ef22480 2024-05-09T00:03:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.479+0000 7f3326b25b80 10 client.29795 fetch_fsmap finished waiting for FSMap version 70 2024-05-09T00:03:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.479+0000 7f3326b25b80 10 client.29795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.479+0000 7f3326b25b80 20 client.29795 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.479+0000 7f3326b25b80 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564b3efb4340 0x564b3efb6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.479+0000 7f3326b25b80 4 client.29795 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:03:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.479+0000 7f3326b25b80 1 -- 172.21.15.12:0/705036652 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564b3ee6ae60 con 0x564b3efb4340 2024-05-09T00:03:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.479+0000 7f33137fe700 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564b3efb4340 0x564b3efb6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.480+0000 7f33137fe700 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564b3efb4340 0x564b3efb6760 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13afce922786c20c server_cookie=e60d8ce26f065f16 in_seq=0 out_seq=0 2024-05-09T00:03:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.480+0000 7f32f37fe700 10 client.29795 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.776+0000 7f32f37fe700 1 -- 172.21.15.12:0/705036652 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564b3ee6ae60 con 0x564b3efb4340 2024-05-09T00:03:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.776+0000 7f32f37fe700 10 client.29795 handle_command_reply: tid=0 2024-05-09T00:03:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.776+0000 7f3326b25b80 1 client.29795 shutdown 2024-05-09T00:03:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.777+0000 7f3326b25b80 1 -- 172.21.15.12:0/705036652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 msgr2=0x564b3ef4a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.777+0000 7f3326b25b80 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef4a7d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f330401e950 tx=0x7f3304003e30 comp rx=0 tx=0).stop 2024-05-09T00:03:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.778+0000 7f3326b25b80 1 -- 172.21.15.12:0/705036652 shutdown_connections 2024-05-09T00:03:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.778+0000 7f3326b25b80 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564b3ef28270 0x564b3ef258c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.778+0000 7f3326b25b80 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564b3efb4340 0x564b3efb6760 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.778+0000 7f3326b25b80 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564b3ef22480 0x564b3ef4a7d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.778+0000 7f3326b25b80 1 --2- 172.21.15.12:0/705036652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564b3ef27950 0x564b3ef25380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.778+0000 7f32f37fe700 0 client.29795 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.778+0000 7f3326b25b80 1 -- 172.21.15.12:0/705036652 >> 172.21.15.12:0/705036652 conn(0x564b3ee801b0 msgr2=0x564b3ef29e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.778+0000 7f3326b25b80 1 -- 172.21.15.12:0/705036652 shutdown_connections 2024-05-09T00:03:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.778+0000 7f3326b25b80 1 -- 172.21.15.12:0/705036652 wait complete. 2024-05-09T00:03:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.778+0000 7f3326b25b80 20 client.29795 trim_cache size 0 max 16384 2024-05-09T00:03:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:03:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:03:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:03:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:03:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 -- 172.21.15.12:0/382379302 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f32fc0a1310 msgr2=0x7f32fc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f32fc0a1310 0x7f32fc0a37d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3304011490 tx=0x7f3304009260 comp rx=0 tx=0).stop 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 -- 172.21.15.12:0/382379302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3314199ab0 msgr2=0x7f3314199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3314199ab0 0x7f3314199f30 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f330c00ef30 tx=0x7f330c00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 -- 172.21.15.12:0/382379302 shutdown_connections 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f32fc0a1310 0x7f32fc0a37d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3314199ab0 0x7f3314199f30 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3314100010 0x7f3314195220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 --2- 172.21.15.12:0/382379302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314105130 0x7f3314195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 -- 172.21.15.12:0/382379302 >> 172.21.15.12:0/382379302 conn(0x7f33140fb870 msgr2=0x7f33140fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 -- 172.21.15.12:0/382379302 shutdown_connections 2024-05-09T00:03:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:08.779+0000 7f331a96b700 1 -- 172.21.15.12:0/382379302 wait complete. 2024-05-09T00:03:08.914 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 00:03:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-d[93495]: ignoring --setuser ceph since I am not root 2024-05-09T00:03:08.914 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 00:03:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-d[93495]: ignoring --setgroup ceph since I am not root 2024-05-09T00:03:08.914 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 00:03:08 smithi012 ceph-mds[93518]: -- 172.21.15.12:0/935380607 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f86f28c1a0 con 0x55f86f262800 2024-05-09T00:03:08.914 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 00:03:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-d[93495]: starting mds.d at 2024-05-09T00:03:08.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:08 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:03:09.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:08 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:03:09.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:08 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:03:09.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 132867 38.64 MB/sec execute 64 sec latency 137.792 ms 2024-05-09T00:03:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:09 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:03:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:09 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:03:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:09 smithi204 ceph-mon[47373]: daemon mds.d restarted 2024-05-09T00:03:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:09 smithi204 ceph-mon[47373]: osdmap e118: 12 total, 12 up, 12 in 2024-05-09T00:03:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:09 smithi204 ceph-mon[47373]: pgmap v539: 97 pgs: 97 active+clean; 1001 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 26 MiB/s wr, 1.48k op/s 2024-05-09T00:03:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:09 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:03:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:09 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:03:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:09 smithi160 ceph-mon[46152]: daemon mds.d restarted 2024-05-09T00:03:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:09 smithi160 ceph-mon[46152]: osdmap e118: 12 total, 12 up, 12 in 2024-05-09T00:03:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:09 smithi160 ceph-mon[46152]: pgmap v539: 97 pgs: 97 active+clean; 1001 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 26 MiB/s wr, 1.48k op/s 2024-05-09T00:03:10.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:09 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:03:10.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:09 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:03:10.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:09 smithi012 ceph-mon[42503]: daemon mds.d restarted 2024-05-09T00:03:10.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:09 smithi012 ceph-mon[42503]: osdmap e118: 12 total, 12 up, 12 in 2024-05-09T00:03:10.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:09 smithi012 ceph-mon[42503]: pgmap v539: 97 pgs: 97 active+clean; 1001 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 26 MiB/s wr, 1.48k op/s 2024-05-09T00:03:10.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 133764 38.83 MB/sec execute 65 sec latency 60.282 ms 2024-05-09T00:03:11.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:10 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.12:6836/1370059141,v1:172.21.15.12:6837/1370059141] up:boot 2024-05-09T00:03:11.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:10 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:03:11.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:10 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T00:03:11.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:10 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:11.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:10 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.12:6836/1370059141,v1:172.21.15.12:6837/1370059141] up:boot 2024-05-09T00:03:11.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:10 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:03:11.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:10 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T00:03:11.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:10 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:11.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:10 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.12:6836/1370059141,v1:172.21.15.12:6837/1370059141] up:boot 2024-05-09T00:03:11.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:10 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:03:11.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:10 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T00:03:11.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:10 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:11.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 134535 38.91 MB/sec execute 66 sec latency 46.910 ms 2024-05-09T00:03:12.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:11 smithi012 ceph-mon[42503]: osdmap e119: 12 total, 12 up, 12 in 2024-05-09T00:03:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:11 smithi012 ceph-mon[42503]: pgmap v541: 97 pgs: 97 active+clean; 1001 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 431 op/s 2024-05-09T00:03:12.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135343 38.91 MB/sec execute 67 sec latency 51.755 ms 2024-05-09T00:03:12.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:11 smithi204 ceph-mon[47373]: osdmap e119: 12 total, 12 up, 12 in 2024-05-09T00:03:12.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:11 smithi204 ceph-mon[47373]: pgmap v541: 97 pgs: 97 active+clean; 1001 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 431 op/s 2024-05-09T00:03:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:11 smithi160 ceph-mon[46152]: osdmap e119: 12 total, 12 up, 12 in 2024-05-09T00:03:12.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:11 smithi160 ceph-mon[46152]: pgmap v541: 97 pgs: 97 active+clean; 1001 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 431 op/s 2024-05-09T00:03:13.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 136195 39.06 MB/sec execute 68 sec latency 52.012 ms 2024-05-09T00:03:13.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:13 smithi012 ceph-mon[42503]: pgmap v542: 97 pgs: 97 active+clean; 1012 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.6 MiB/s wr, 904 op/s 2024-05-09T00:03:14.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:13 smithi204 ceph-mon[47373]: pgmap v542: 97 pgs: 97 active+clean; 1012 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.6 MiB/s wr, 904 op/s 2024-05-09T00:03:14.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:13 smithi160 ceph-mon[46152]: pgmap v542: 97 pgs: 97 active+clean; 1012 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.6 MiB/s wr, 904 op/s 2024-05-09T00:03:14.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 136929 39.09 MB/sec execute 69 sec latency 84.388 ms 2024-05-09T00:03:15.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 137706 39.05 MB/sec execute 70 sec latency 66.198 ms 2024-05-09T00:03:15.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:15 smithi012 ceph-mon[42503]: pgmap v543: 97 pgs: 97 active+clean; 1012 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 526 op/s 2024-05-09T00:03:16.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:15 smithi204 ceph-mon[47373]: pgmap v543: 97 pgs: 97 active+clean; 1012 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 526 op/s 2024-05-09T00:03:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:15 smithi160 ceph-mon[46152]: pgmap v543: 97 pgs: 97 active+clean; 1012 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 526 op/s 2024-05-09T00:03:16.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 138563 39.09 MB/sec execute 71 sec latency 63.749 ms 2024-05-09T00:03:17.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 139516 39.37 MB/sec execute 72 sec latency 47.712 ms 2024-05-09T00:03:17.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:17 smithi012 ceph-mon[42503]: pgmap v544: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.5 MiB/s wr, 694 op/s 2024-05-09T00:03:18.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:17 smithi204 ceph-mon[47373]: pgmap v544: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.5 MiB/s wr, 694 op/s 2024-05-09T00:03:18.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:17 smithi160 ceph-mon[46152]: pgmap v544: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.5 MiB/s wr, 694 op/s 2024-05-09T00:03:18.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 140229 39.39 MB/sec execute 73 sec latency 69.302 ms 2024-05-09T00:03:19.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 140985 39.35 MB/sec execute 74 sec latency 58.212 ms 2024-05-09T00:03:19.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:19 smithi012 ceph-mon[42503]: pgmap v545: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.2 MiB/s wr, 664 op/s 2024-05-09T00:03:20.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:19 smithi204 ceph-mon[47373]: pgmap v545: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.2 MiB/s wr, 664 op/s 2024-05-09T00:03:20.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:19 smithi160 ceph-mon[46152]: pgmap v545: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.2 MiB/s wr, 664 op/s 2024-05-09T00:03:20.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 141950 39.44 MB/sec execute 75 sec latency 53.750 ms 2024-05-09T00:03:21.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 142820 39.64 MB/sec execute 76 sec latency 72.490 ms 2024-05-09T00:03:21.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:21 smithi012 ceph-mon[42503]: pgmap v546: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.7 MiB/s wr, 621 op/s 2024-05-09T00:03:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.929+0000 7fa2ec22e700 1 -- 172.21.15.12:0/1395602070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4100010 msgr2=0x7fa2e41003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.929+0000 7fa2ec22e700 1 --2- 172.21.15.12:0/1395602070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4100010 0x7fa2e41003f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009990 tx=0x7fa2e0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.929+0000 7fa2ec22e700 1 -- 172.21.15.12:0/1395602070 shutdown_connections 2024-05-09T00:03:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.929+0000 7fa2ec22e700 1 --2- 172.21.15.12:0/1395602070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e4105130 0x7fa2e410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.929+0000 7fa2ec22e700 1 --2- 172.21.15.12:0/1395602070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4100010 0x7fa2e41003f0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.929+0000 7fa2ec22e700 1 --2- 172.21.15.12:0/1395602070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e41009c0 0x7fa2e4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.929+0000 7fa2ec22e700 1 -- 172.21.15.12:0/1395602070 >> 172.21.15.12:0/1395602070 conn(0x7fa2e4076f40 msgr2=0x7fa2e4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.929+0000 7fa2ec22e700 1 -- 172.21.15.12:0/1395602070 shutdown_connections 2024-05-09T00:03:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.929+0000 7fa2ec22e700 1 -- 172.21.15.12:0/1395602070 wait complete. 2024-05-09T00:03:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2ec22e700 1 Processor -- start 2024-05-09T00:03:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2ec22e700 1 -- start start 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2ec22e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4100010 0x7fa2e4101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2ec22e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e4105130 0x7fa2e4101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2ec22e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e4102520 0x7fa2e41a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2ec22e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e41a5bf0 con 0x7fa2e4100010 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2ec22e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e41a5d60 con 0x7fa2e4105130 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2ec22e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e41a5f20 con 0x7fa2e4102520 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2e9fca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4100010 0x7fa2e4101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2e9fca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4100010 0x7fa2e4101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52966/0 (socket says 172.21.15.12:52966) 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2e9fca700 1 -- 172.21.15.12:0/3221580240 learned_addr learned my addr 172.21.15.12:0/3221580240 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e4105130 0x7fa2e4101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.930+0000 7fa2ea7cb700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e4102520 0x7fa2e41a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2e9fca700 1 -- 172.21.15.12:0/3221580240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e4102520 msgr2=0x7fa2e41a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e4102520 0x7fa2e41a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2e9fca700 1 -- 172.21.15.12:0/3221580240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e4105130 msgr2=0x7fa2e4101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e4105130 0x7fa2e4101fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2e9fca700 1 -- 172.21.15.12:0/3221580240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00095f0 con 0x7fa2e4100010 2024-05-09T00:03:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4100010 0x7fa2e4101aa0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0005910 tx=0x7fa2e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2daffd700 1 -- 172.21.15.12:0/3221580240 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e001d070 con 0x7fa2e4100010 2024-05-09T00:03:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2ec22e700 1 -- 172.21.15.12:0/3221580240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e41a61a0 con 0x7fa2e4100010 2024-05-09T00:03:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2daffd700 1 -- 172.21.15.12:0/3221580240 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e0022470 con 0x7fa2e4100010 2024-05-09T00:03:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2daffd700 1 -- 172.21.15.12:0/3221580240 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0017650 con 0x7fa2e4100010 2024-05-09T00:03:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.931+0000 7fa2ec22e700 1 -- 172.21.15.12:0/3221580240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e41a65d0 con 0x7fa2e4100010 2024-05-09T00:03:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.935+0000 7fa2daffd700 1 -- 172.21.15.12:0/3221580240 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa2e00225e0 con 0x7fa2e4100010 2024-05-09T00:03:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.936+0000 7fa2daffd700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2d00a1290 0x7fa2d00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.936+0000 7fa2daffd700 1 -- 172.21.15.12:0/3221580240 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa2e00cdf30 con 0x7fa2e4100010 2024-05-09T00:03:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.936+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2d00a1290 0x7fa2d00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.936+0000 7fa2f83e8b80 1 Processor -- start 2024-05-09T00:03:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.936+0000 7fa2f83e8b80 1 -- start start 2024-05-09T00:03:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130202950 0x55b130202d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 0x55b1301fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 0x55b1301fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1301540f0 con 0x55b130203270 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130151ab0 con 0x55b1301fd480 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13014d3a0 con 0x55b130202950 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2d00a1290 0x7fa2d00a3750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4103180 tx=0x7fa2d4009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2e97c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 0x55b1301fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2e97c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 0x55b1301fcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52976/0 (socket says 172.21.15.12:52976) 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2e97c9700 1 -- 172.21.15.12:0/4118688658 learned_addr learned my addr 172.21.15.12:0/4118688658 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2e9fca700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130202950 0x55b130202d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2ea7cb700 1 --2- 172.21.15.12:0/4118688658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 0x55b1301fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2e9fca700 1 -- 172.21.15.12:0/4118688658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 msgr2=0x55b1301fd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/4118688658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 0x55b1301fd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2e9fca700 1 -- 172.21.15.12:0/4118688658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 msgr2=0x55b1301fcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.937+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/4118688658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 0x55b1301fcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.938+0000 7fa2e9fca700 1 -- 172.21.15.12:0/4118688658 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b12ffce350 con 0x55b130202950 2024-05-09T00:03:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.938+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/4118688658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 0x55b1301fcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.938+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/4118688658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130202950 0x55b130202d30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00cb690 tx=0x7fa2e0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74e89904d18bcc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.938+0000 7fa2cffff700 1 -- 172.21.15.12:0/4118688658 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e009ab60 con 0x55b130202950 2024-05-09T00:03:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.938+0000 7fa2cffff700 1 -- 172.21.15.12:0/4118688658 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e009b900 con 0x55b130202950 2024-05-09T00:03:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.938+0000 7fa2cffff700 1 -- 172.21.15.12:0/4118688658 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00a10c0 con 0x55b130202950 2024-05-09T00:03:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.939+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/4118688658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130202950 msgr2=0x55b130202d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.939+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/4118688658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130202950 0x55b130202d30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00cb690 tx=0x7fa2e0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.939+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/4118688658 shutdown_connections 2024-05-09T00:03:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.939+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/4118688658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 0x55b1301fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.939+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/4118688658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130202950 0x55b130202d30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.939+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/4118688658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 0x55b1301fcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.939+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/4118688658 >> 172.21.15.12:0/4118688658 conn(0x55b13015b1b0 msgr2=0x55b1301f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.939+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/4118688658 shutdown_connections 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.939+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/4118688658 wait complete. 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2f83e8b80 1 -- start start 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2f83e8b80 1 Processor -- start 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1301fd480 0x55b1302091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 0x55b130206490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 0x55b130206af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1300e9500 con 0x55b130203270 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1300e91a0 con 0x55b130202950 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b12ffd0640 con 0x55b1301fd480 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2f83e8b80 10 client.0 _ll_register_callbacks cb 0x55b1300f2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2e9fca700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1301fd480 0x55b1302091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2e9fca700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1301fd480 0x55b1302091a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45150/0 (socket says 172.21.15.12:45150) 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2e9fca700 1 -- 172.21.15.12:0/820027693 learned_addr learned my addr 172.21.15.12:0/820027693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 0x55b130206490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.940+0000 7fa2ea7cb700 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 0x55b130206af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2e9fca700 1 -- 172.21.15.12:0/820027693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 msgr2=0x55b130206490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 0x55b130206490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2e9fca700 1 -- 172.21.15.12:0/820027693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 msgr2=0x55b130206af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 0x55b130206af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2e9fca700 1 -- 172.21.15.12:0/820027693 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d4017070 con 0x55b1301fd480 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1301fd480 0x55b1302091a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00ce210 tx=0x7fa2e0022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60fae95d6df3b79a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/820027693 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e009f030 con 0x55b1301fd480 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/820027693 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa2e009b200 con 0x55b1301fd480 2024-05-09T00:03:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/820027693 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1300a6210 con 0x55b1301fd480 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/820027693 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e009ab60 con 0x55b1301fd480 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.941+0000 7fa2f83e8b80 10 client.29893 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.942+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/820027693 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7fa2e41a65d0 con 0x55b1301fd480 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.942+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/820027693 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b1300a6210 con 0x55b1301fd480 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.942+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/820027693 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2e0081070 con 0x55b1301fd480 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.942+0000 7fa2f83e8b80 10 client.29893 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.942+0000 7fa2f83e8b80 10 client.29893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.942+0000 7fa2f83e8b80 20 client.29893 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.942+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b13021eb70 0x55b130220f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.942+0000 7fa2f83e8b80 4 client.29893 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.942+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/820027693 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1301cc7b0 con 0x55b13021eb70 2024-05-09T00:03:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.942+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b13021eb70 0x55b130220f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.943+0000 7fa2cd7fa700 10 client.29893 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.943+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b13021eb70 0x55b130220f90 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac71df101e1475f server_cookie=43d4985ce56cac69 in_seq=0 out_seq=0 2024-05-09T00:03:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/820027693 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b1301cc7b0 con 0x55b13021eb70 2024-05-09T00:03:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2cd7fa700 10 client.29893 handle_command_reply: tid=0 2024-05-09T00:03:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 client.29893 shutdown 2024-05-09T00:03:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/820027693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1301fd480 msgr2=0x55b1302091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1301fd480 0x55b1302091a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00ce210 tx=0x7fa2e0022c90 comp rx=0 tx=0).stop 2024-05-09T00:03:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/820027693 shutdown_connections 2024-05-09T00:03:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130203270 0x55b130206af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b1301fd480 0x55b1302091a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b13021eb70 0x55b130220f90 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/820027693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 0x55b130206490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2cd7fa700 0 client.29893 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/820027693 >> 172.21.15.12:0/820027693 conn(0x55b13015b1b0 msgr2=0x55b1301fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/820027693 shutdown_connections 2024-05-09T00:03:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/820027693 wait complete. 2024-05-09T00:03:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.945+0000 7fa2f83e8b80 20 client.29893 trim_cache size 0 max 16384 2024-05-09T00:03:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.964+0000 7fa2f83e8b80 1 Processor -- start 2024-05-09T00:03:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2f83e8b80 1 -- start start 2024-05-09T00:03:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1301fd480 0x55b130206a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 0x55b1301f7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 0x55b130216160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1302025f0 con 0x55b1301fd480 2024-05-09T00:03:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1301f27c0 con 0x55b130202950 2024-05-09T00:03:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130207400 con 0x55b130203270 2024-05-09T00:03:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2e9fca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1301fd480 0x55b130206a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2e9fca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1301fd480 0x55b130206a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52992/0 (socket says 172.21.15.12:52992) 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2e9fca700 1 -- 172.21.15.12:0/1432165760 learned_addr learned my addr 172.21.15.12:0/1432165760 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 0x55b1301f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2ea7cb700 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 0x55b130216160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2e97c9700 1 -- 172.21.15.12:0/1432165760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 msgr2=0x55b130216160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 0x55b130216160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2e97c9700 1 -- 172.21.15.12:0/1432165760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1301fd480 msgr2=0x55b130206a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1301fd480 0x55b130206a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2e97c9700 1 -- 172.21.15.12:0/1432165760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1301fdb30 con 0x55b130202950 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2ea7cb700 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 0x55b130216160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.965+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1301fd480 0x55b130206a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.966+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 0x55b1301f7910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55b1302079d0 tx=0x7fa2d4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f5c44d5e7ecc6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.966+0000 7fa2d8ff9700 1 -- 172.21.15.12:0/1432165760 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4022040 con 0x55b130202950 2024-05-09T00:03:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.966+0000 7fa2d8ff9700 1 -- 172.21.15.12:0/1432165760 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2d4023040 con 0x55b130202950 2024-05-09T00:03:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.966+0000 7fa2d8ff9700 1 -- 172.21.15.12:0/1432165760 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4005020 con 0x55b130202950 2024-05-09T00:03:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/1432165760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 msgr2=0x55b1301f7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 0x55b1301f7910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55b1302079d0 tx=0x7fa2d4014000 comp rx=0 tx=0).stop 2024-05-09T00:03:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/1432165760 shutdown_connections 2024-05-09T00:03:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 0x55b130216160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1301fd480 0x55b130206a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/1432165760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b130202950 0x55b1301f7910 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/1432165760 >> 172.21.15.12:0/1432165760 conn(0x55b13015b1b0 msgr2=0x55b1301fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/1432165760 shutdown_connections 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/1432165760 wait complete. 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 -- start start 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.967+0000 7fa2f83e8b80 1 Processor -- start 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 0x55b130214140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130202950 0x55b130214680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2f83e8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 0x55b130214bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b130274070 con 0x55b130202950 2024-05-09T00:03:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1302741b0 con 0x55b1301fd480 2024-05-09T00:03:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2f83e8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1302742f0 con 0x55b130203270 2024-05-09T00:03:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2f83e8b80 10 client.0 _ll_register_callbacks cb 0x55b1300f2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2e9fca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 0x55b130214140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2e9fca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 0x55b130214140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:43026/0 (socket says 172.21.15.12:43026) 2024-05-09T00:03:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2ea7cb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 0x55b130214bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2e9fca700 1 -- 172.21.15.12:0/2121418993 learned_addr learned my addr 172.21.15.12:0/2121418993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130202950 0x55b130214680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2e9fca700 1 -- 172.21.15.12:0/2121418993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 msgr2=0x55b130214bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 0x55b130214bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2e9fca700 1 -- 172.21.15.12:0/2121418993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130202950 msgr2=0x55b130214680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130202950 0x55b130214680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2e9fca700 1 -- 172.21.15.12:0/2121418993 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc0115b0 con 0x55b1301fd480 2024-05-09T00:03:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.968+0000 7fa2ea7cb700 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 0x55b130214bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2e9fca700 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 0x55b130214140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0017a40 tx=0x7fa2e0097640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90e824d42df58176 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:03:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/2121418993 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00a3160 con 0x55b1301fd480 2024-05-09T00:03:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/2121418993 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa2e009b200 con 0x55b1301fd480 2024-05-09T00:03:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/2121418993 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2e009b900 con 0x55b1301fd480 2024-05-09T00:03:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/2121418993 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e00177b0 con 0x55b1301fd480 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2f83e8b80 10 client.29819 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/2121418993 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7fa2d4011ae0 con 0x55b1301fd480 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/2121418993 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e009b900 con 0x55b1301fd480 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/2121418993 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2e0081070 con 0x55b1301fd480 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2f83e8b80 10 client.29819 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2f83e8b80 10 client.29819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2f83e8b80 20 client.29819 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b1302755d0 0x55b1302779c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2f83e8b80 4 client.29819 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.969+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/2121418993 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b130145e60 con 0x55b1302755d0 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.970+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b1302755d0 0x55b1302779c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.970+0000 7fa2cd7fa700 10 client.29819 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:21.970+0000 7fa2e97c9700 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b1302755d0 0x55b1302779c0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a76b25703a32a9 server_cookie=6462ede92458af91 in_seq=0 out_seq=0 2024-05-09T00:03:22.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:21 smithi204 ceph-mon[47373]: pgmap v546: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.7 MiB/s wr, 621 op/s 2024-05-09T00:03:22.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:21 smithi160 ceph-mon[46152]: pgmap v546: 97 pgs: 97 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.7 MiB/s wr, 621 op/s 2024-05-09T00:03:22.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 143391 39.56 MB/sec execute 77 sec latency 262.046 ms 2024-05-09T00:03:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2cd7fa700 1 -- 172.21.15.12:0/2121418993 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b130145e60 con 0x55b1302755d0 2024-05-09T00:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2cd7fa700 10 client.29819 handle_command_reply: tid=0 2024-05-09T00:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2f83e8b80 1 client.29819 shutdown 2024-05-09T00:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/2121418993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 msgr2=0x55b130214140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 0x55b130214140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0017a40 tx=0x7fa2e0097640 comp rx=0 tx=0).stop 2024-05-09T00:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/2121418993 shutdown_connections 2024-05-09T00:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b130203270 0x55b130214bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b1302755d0 0x55b1302779c0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b1301fd480 0x55b130214140 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2f83e8b80 1 --2- 172.21.15.12:0/2121418993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b130202950 0x55b130214680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2cd7fa700 0 client.29819 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.341+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/2121418993 >> 172.21.15.12:0/2121418993 conn(0x55b13015b1b0 msgr2=0x55b1301f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.345+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/2121418993 shutdown_connections 2024-05-09T00:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.345+0000 7fa2f83e8b80 1 -- 172.21.15.12:0/2121418993 wait complete. 2024-05-09T00:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.345+0000 7fa2f83e8b80 20 client.29819 trim_cache size 0 max 16384 2024-05-09T00:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 -- 172.21.15.12:0/3221580240 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2d00a1290 msgr2=0x7fa2d00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2d00a1290 0x7fa2d00a3750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4103180 tx=0x7fa2d4009260 comp rx=0 tx=0).stop 2024-05-09T00:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 -- 172.21.15.12:0/3221580240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4100010 msgr2=0x7fa2e4101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4100010 0x7fa2e4101aa0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0005910 tx=0x7fa2e0004730 comp rx=0 tx=0).stop 2024-05-09T00:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 -- 172.21.15.12:0/3221580240 shutdown_connections 2024-05-09T00:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2d00a1290 0x7fa2d00a3750 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e4102520 0x7fa2e41a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e4100010 0x7fa2e4101aa0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 --2- 172.21.15.12:0/3221580240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e4105130 0x7fa2e4101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 -- 172.21.15.12:0/3221580240 >> 172.21.15.12:0/3221580240 conn(0x7fa2e4076f40 msgr2=0x7fa2e4069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 -- 172.21.15.12:0/3221580240 shutdown_connections 2024-05-09T00:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:22.346+0000 7fa2ec22e700 1 -- 172.21.15.12:0/3221580240 wait complete. 2024-05-09T00:03:23.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 144081 39.42 MB/sec execute 78 sec latency 367.512 ms 2024-05-09T00:03:23.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:23 smithi012 ceph-mon[42503]: pgmap v547: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 871 op/s 2024-05-09T00:03:24.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:23 smithi204 ceph-mon[47373]: pgmap v547: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 871 op/s 2024-05-09T00:03:24.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:23 smithi160 ceph-mon[46152]: pgmap v547: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 871 op/s 2024-05-09T00:03:24.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 144954 39.55 MB/sec execute 79 sec latency 71.890 ms 2024-05-09T00:03:25.239 INFO:tasks.workunit.client.0.smithi012.stdout: 10 145788 39.66 MB/sec execute 80 sec latency 51.754 ms 2024-05-09T00:03:25.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:25 smithi012 ceph-mon[42503]: pgmap v548: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 552 op/s 2024-05-09T00:03:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:25 smithi204 ceph-mon[47373]: pgmap v548: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 552 op/s 2024-05-09T00:03:26.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:25 smithi160 ceph-mon[46152]: pgmap v548: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 552 op/s 2024-05-09T00:03:26.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 146453 39.60 MB/sec execute 81 sec latency 74.774 ms 2024-05-09T00:03:27.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 147319 39.69 MB/sec execute 82 sec latency 41.068 ms 2024-05-09T00:03:27.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:27 smithi012 ceph-mon[42503]: pgmap v549: 97 pgs: 97 active+clean; 1.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 720 op/s 2024-05-09T00:03:28.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:27 smithi204 ceph-mon[47373]: pgmap v549: 97 pgs: 97 active+clean; 1.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 720 op/s 2024-05-09T00:03:28.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:27 smithi160 ceph-mon[46152]: pgmap v549: 97 pgs: 97 active+clean; 1.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 720 op/s 2024-05-09T00:03:28.240 INFO:tasks.workunit.client.0.smithi012.stdout: 10 147781 39.48 MB/sec execute 83 sec latency 505.305 ms 2024-05-09T00:03:29.240 INFO:tasks.workunit.client.0.smithi012.stdout: 10 147818 39.06 MB/sec execute 84 sec latency 1087.293 ms 2024-05-09T00:03:29.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:29 smithi012 ceph-mon[42503]: pgmap v550: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 736 op/s 2024-05-09T00:03:30.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:29 smithi204 ceph-mon[47373]: pgmap v550: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 736 op/s 2024-05-09T00:03:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:29 smithi160 ceph-mon[46152]: pgmap v550: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 736 op/s 2024-05-09T00:03:30.244 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148287 38.91 MB/sec execute 85 sec latency 845.194 ms 2024-05-09T00:03:31.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148626 38.69 MB/sec execute 86 sec latency 138.311 ms 2024-05-09T00:03:32.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:31 smithi204 ceph-mon[47373]: pgmap v551: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 687 op/s 2024-05-09T00:03:32.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:31 smithi160 ceph-mon[46152]: pgmap v551: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 687 op/s 2024-05-09T00:03:32.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:31 smithi012 ceph-mon[42503]: pgmap v551: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 687 op/s 2024-05-09T00:03:32.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 149256 38.56 MB/sec execute 87 sec latency 79.107 ms 2024-05-09T00:03:32.340 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:03:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.519+0000 7f26f8c4e700 1 -- 172.21.15.12:0/2286379758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26f4105f90 msgr2=0x7f26f4106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.519+0000 7f26f8c4e700 1 --2- 172.21.15.12:0/2286379758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26f4105f90 0x7f26f4106370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f26e40099c0 tx=0x7f26e4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.520+0000 7f26f8c4e700 1 -- 172.21.15.12:0/2286379758 shutdown_connections 2024-05-09T00:03:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.520+0000 7f26f8c4e700 1 --2- 172.21.15.12:0/2286379758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26f4100ab0 0x7f26f410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.520+0000 7f26f8c4e700 1 --2- 172.21.15.12:0/2286379758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26f4105f90 0x7f26f4106370 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.520+0000 7f26f8c4e700 1 --2- 172.21.15.12:0/2286379758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f41000f0 0x7f26f4100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.520+0000 7f26f8c4e700 1 -- 172.21.15.12:0/2286379758 >> 172.21.15.12:0/2286379758 conn(0x7f26f4076ba0 msgr2=0x7f26f4076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.520+0000 7f26f8c4e700 1 -- 172.21.15.12:0/2286379758 shutdown_connections 2024-05-09T00:03:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.520+0000 7f26f8c4e700 1 -- 172.21.15.12:0/2286379758 wait complete. 2024-05-09T00:03:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f8c4e700 1 Processor -- start 2024-05-09T00:03:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f8c4e700 1 -- start start 2024-05-09T00:03:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f8c4e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26f41000f0 0x7f26f410d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f8c4e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f4100ab0 0x7f26f410db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f8c4e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26f4105f90 0x7f26f410e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f8c4e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f41a5740 con 0x7f26f41000f0 2024-05-09T00:03:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f8c4e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f41a58b0 con 0x7f26f4100ab0 2024-05-09T00:03:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f8c4e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f41a5a70 con 0x7f26f4105f90 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26f41000f0 0x7f26f410d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f2d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26f4105f90 0x7f26f410e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26f41000f0 0x7f26f410d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58196/0 (socket says 172.21.15.12:58196) 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f259c700 1 -- 172.21.15.12:0/3469030807 learned_addr learned my addr 172.21.15.12:0/3469030807 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.521+0000 7f26f1d9b700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f4100ab0 0x7f26f410db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26f259c700 1 -- 172.21.15.12:0/3469030807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26f4105f90 msgr2=0x7f26f410e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26f259c700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26f4105f90 0x7f26f410e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26f259c700 1 -- 172.21.15.12:0/3469030807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f4100ab0 msgr2=0x7f26f410db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26f259c700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f4100ab0 0x7f26f410db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26f259c700 1 -- 172.21.15.12:0/3469030807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40095f0 con 0x7f26f41000f0 2024-05-09T00:03:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26f259c700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26f41000f0 0x7f26f410d620 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f26e40059d0 tx=0x7f26e400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:32.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26e37fe700 1 -- 172.21.15.12:0/3469030807 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e401d070 con 0x7f26f41000f0 2024-05-09T00:03:32.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26e37fe700 1 -- 172.21.15.12:0/3469030807 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26e400be00 con 0x7f26f41000f0 2024-05-09T00:03:32.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26f8c4e700 1 -- 172.21.15.12:0/3469030807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f41a5cf0 con 0x7f26f41000f0 2024-05-09T00:03:32.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26e37fe700 1 -- 172.21.15.12:0/3469030807 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e400faa0 con 0x7f26f41000f0 2024-05-09T00:03:32.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.522+0000 7f26f8c4e700 1 -- 172.21.15.12:0/3469030807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f41a61a0 con 0x7f26f41000f0 2024-05-09T00:03:32.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.525+0000 7f26e37fe700 1 -- 172.21.15.12:0/3469030807 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f26e402b440 con 0x7f26f41000f0 2024-05-09T00:03:32.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.525+0000 7f26e37fe700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26dc0a1290 0x7f26dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.526+0000 7f26f1d9b700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26dc0a1290 0x7f26dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.526+0000 7f26e37fe700 1 -- 172.21.15.12:0/3469030807 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f26e40cd8f0 con 0x7f26f41000f0 2024-05-09T00:03:32.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.526+0000 7f26f1d9b700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26dc0a1290 0x7f26dc0a3750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f26f410ed50 tx=0x7f26e8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:32.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.526+0000 7f2704e08b80 1 Processor -- start 2024-05-09T00:03:32.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f2704e08b80 1 -- start start 2024-05-09T00:03:32.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c509fe00 0x5558c50a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c50a0720 0x5558c509a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509a890 0x5558c509acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f2704e08b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c4ff1150 con 0x5558c50a0720 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f2704e08b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c4feeb10 con 0x5558c509a890 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f2704e08b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c4fea400 con 0x5558c509fe00 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f26f1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c50a0720 0x5558c509a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f26f259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c509fe00 0x5558c50a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f26f259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c509fe00 0x5558c50a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43364/0 (socket says 172.21.15.12:43364) 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f26f2d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509a890 0x5558c509acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.527+0000 7f26f259c700 1 -- 172.21.15.12:0/3637349356 learned_addr learned my addr 172.21.15.12:0/3637349356 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.528+0000 7f26f259c700 1 -- 172.21.15.12:0/3637349356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509a890 msgr2=0x5558c509acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:32.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.528+0000 7f26f259c700 1 --2- 172.21.15.12:0/3637349356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509a890 0x5558c509acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.528+0000 7f26f259c700 1 -- 172.21.15.12:0/3637349356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c50a0720 msgr2=0x5558c509a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.528+0000 7f26f259c700 1 --2- 172.21.15.12:0/3637349356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c50a0720 0x5558c509a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.528+0000 7f26f259c700 1 -- 172.21.15.12:0/3637349356 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c4e6b3b0 con 0x5558c509fe00 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.528+0000 7f26f259c700 1 --2- 172.21.15.12:0/3637349356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c509fe00 0x5558c50a01e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f26e4097510 tx=0x7f26e409a9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7b2f2552bbf9d6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.528+0000 7f26e0ff9700 1 -- 172.21.15.12:0/3637349356 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e409e8d0 con 0x5558c509fe00 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.528+0000 7f26e0ff9700 1 -- 172.21.15.12:0/3637349356 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26e40a2e90 con 0x5558c509fe00 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.528+0000 7f26e0ff9700 1 -- 172.21.15.12:0/3637349356 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e409a1f0 con 0x5558c509fe00 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.529+0000 7f2704e08b80 1 -- 172.21.15.12:0/3637349356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c509fe00 msgr2=0x5558c50a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.529+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3637349356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c509fe00 0x5558c50a01e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f26e4097510 tx=0x7f26e409a9a0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.529+0000 7f2704e08b80 1 -- 172.21.15.12:0/3637349356 shutdown_connections 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.529+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3637349356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509a890 0x5558c509acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.529+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3637349356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c509fe00 0x5558c50a01e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.529+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3637349356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c50a0720 0x5558c509a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.529+0000 7f2704e08b80 1 -- 172.21.15.12:0/3637349356 >> 172.21.15.12:0/3637349356 conn(0x5558c4ff8210 msgr2=0x5558c5094df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.529+0000 7f2704e08b80 1 -- 172.21.15.12:0/3637349356 shutdown_connections 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.529+0000 7f2704e08b80 1 -- 172.21.15.12:0/3637349356 wait complete. 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.529+0000 7f2704e08b80 1 -- start start 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f2704e08b80 1 Processor -- start 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c50a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c50a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c50a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f2704e08b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c4f86560 con 0x5558c509a890 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f2704e08b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c4f86200 con 0x5558c509fe00 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f2704e08b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c4e6d6a0 con 0x5558c50a0720 2024-05-09T00:03:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f2704e08b80 10 client.0 _ll_register_callbacks cb 0x5558c4f8fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f26f259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c50a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f26f1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c50a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f26f2d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c50a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f26f1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c50a3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33370/0 (socket says 172.21.15.12:33370) 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f26f1d9b700 1 -- 172.21.15.12:0/3789488455 learned_addr learned my addr 172.21.15.12:0/3789488455 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f26f259c700 1 -- 172.21.15.12:0/3789488455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 msgr2=0x5558c50a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.530+0000 7f26f259c700 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c50a4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f26f259c700 1 -- 172.21.15.12:0/3789488455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 msgr2=0x5558c50a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f26f259c700 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c50a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f26f259c700 1 -- 172.21.15.12:0/3789488455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f41a61a0 con 0x5558c509a890 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f26f259c700 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c50a6740 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f26e4097510 tx=0x7f26e4004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8d3ddddb7d3fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f26d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:03:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3789488455 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e409e8d0 con 0x5558c509a890 2024-05-09T00:03:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f2704e08b80 1 -- 172.21.15.12:0/3789488455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558c4f43270 con 0x5558c509a890 2024-05-09T00:03:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3789488455 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f26e409af10 con 0x5558c509a890 2024-05-09T00:03:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3789488455 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26e40b3210 con 0x5558c509a890 2024-05-09T00:03:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f2704e08b80 10 client.20178 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f2704e08b80 1 -- 172.21.15.12:0/3789488455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5558c4e17b60 con 0x5558c509a890 2024-05-09T00:03:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.531+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3789488455 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558c4f43270 con 0x5558c509a890 2024-05-09T00:03:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.532+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3789488455 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f26e40ba0e0 con 0x5558c509a890 2024-05-09T00:03:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.532+0000 7f2704e08b80 10 client.20178 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.532+0000 7f2704e08b80 10 client.20178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.532+0000 7f2704e08b80 20 client.20178 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.532+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558c50b39d0 0x5558c50b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.532+0000 7f2704e08b80 4 client.20178 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.532+0000 7f2704e08b80 1 -- 172.21.15.12:0/3789488455 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558c5069810 con 0x5558c50b39d0 2024-05-09T00:03:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.532+0000 7f26f1d9b700 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558c50b39d0 0x5558c50b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.533+0000 7f26f1d9b700 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558c50b39d0 0x5558c50b5dc0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da06f4fc957942c server_cookie=7ee8a60c3c3a2f89 in_seq=0 out_seq=0 2024-05-09T00:03:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.533+0000 7f26d9ffb700 10 client.20178 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.535+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3789488455 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558c5069810 con 0x5558c50b39d0 2024-05-09T00:03:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.535+0000 7f26d9ffb700 10 client.20178 handle_command_reply: tid=0 2024-05-09T00:03:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.535+0000 7f2704e08b80 1 client.20178 shutdown 2024-05-09T00:03:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 1 -- 172.21.15.12:0/3789488455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 msgr2=0x5558c50a6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c50a6740 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f26e4097510 tx=0x7f26e4004f50 comp rx=0 tx=0).stop 2024-05-09T00:03:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 1 -- 172.21.15.12:0/3789488455 shutdown_connections 2024-05-09T00:03:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c50a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c50a6740 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558c50b39d0 0x5558c50b5dc0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3789488455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c50a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f26d9ffb700 0 client.20178 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 1 -- 172.21.15.12:0/3789488455 >> 172.21.15.12:0/3789488455 conn(0x5558c4ff8210 msgr2=0x5558c50945a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 1 -- 172.21.15.12:0/3789488455 shutdown_connections 2024-05-09T00:03:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 1 -- 172.21.15.12:0/3789488455 wait complete. 2024-05-09T00:03:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.536+0000 7f2704e08b80 20 client.20178 trim_cache size 0 max 16384 2024-05-09T00:03:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.560+0000 7f2704e08b80 1 Processor -- start 2024-05-09T00:03:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.560+0000 7f2704e08b80 1 -- start start 2024-05-09T00:03:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.560+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c509a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.560+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c5094d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.560+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c509db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.560+0000 7f2704e08b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c50a6c00 con 0x5558c509a890 2024-05-09T00:03:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.560+0000 7f2704e08b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c508f800 con 0x5558c509fe00 2024-05-09T00:03:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.560+0000 7f2704e08b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c50baef0 con 0x5558c50a0720 2024-05-09T00:03:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.561+0000 7f26f2d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c509db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.561+0000 7f26f2d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c509db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43388/0 (socket says 172.21.15.12:43388) 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.561+0000 7f26f2d9d700 1 -- 172.21.15.12:0/2087810198 learned_addr learned my addr 172.21.15.12:0/2087810198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.561+0000 7f26f1d9b700 1 --2- 172.21.15.12:0/2087810198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c5094d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.561+0000 7f26f2d9d700 1 -- 172.21.15.12:0/2087810198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 msgr2=0x5558c5094d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.561+0000 7f26f2d9d700 1 --2- 172.21.15.12:0/2087810198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c5094d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.561+0000 7f26f2d9d700 1 -- 172.21.15.12:0/2087810198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 msgr2=0x5558c509a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.561+0000 7f26f2d9d700 1 --2- 172.21.15.12:0/2087810198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c509a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.561+0000 7f26f2d9d700 1 -- 172.21.15.12:0/2087810198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c4fb0560 con 0x5558c50a0720 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.561+0000 7f26f2d9d700 1 --2- 172.21.15.12:0/2087810198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c509db80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0119d0 tx=0x7f26ec009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c775e6c8fd4da6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.562+0000 7f26e17fa700 1 -- 172.21.15.12:0/2087810198 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00e040 con 0x5558c50a0720 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.562+0000 7f26e17fa700 1 -- 172.21.15.12:0/2087810198 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26ec00f040 con 0x5558c50a0720 2024-05-09T00:03:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.562+0000 7f26e17fa700 1 -- 172.21.15.12:0/2087810198 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec007ba0 con 0x5558c50a0720 2024-05-09T00:03:32.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.563+0000 7f2704e08b80 1 -- 172.21.15.12:0/2087810198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 msgr2=0x5558c509db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:32.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.563+0000 7f2704e08b80 1 --2- 172.21.15.12:0/2087810198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c509db80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0119d0 tx=0x7f26ec009920 comp rx=0 tx=0).stop 2024-05-09T00:03:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.565+0000 7f2704e08b80 1 -- 172.21.15.12:0/2087810198 shutdown_connections 2024-05-09T00:03:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.565+0000 7f2704e08b80 1 --2- 172.21.15.12:0/2087810198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c509db80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.565+0000 7f2704e08b80 1 --2- 172.21.15.12:0/2087810198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c509a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.565+0000 7f2704e08b80 1 --2- 172.21.15.12:0/2087810198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c5094d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.565+0000 7f2704e08b80 1 -- 172.21.15.12:0/2087810198 >> 172.21.15.12:0/2087810198 conn(0x5558c4ff8210 msgr2=0x5558c509ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.566+0000 7f2704e08b80 1 -- 172.21.15.12:0/2087810198 shutdown_connections 2024-05-09T00:03:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.566+0000 7f2704e08b80 1 -- 172.21.15.12:0/2087810198 wait complete. 2024-05-09T00:03:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.566+0000 7f2704e08b80 1 -- start start 2024-05-09T00:03:32.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.566+0000 7f2704e08b80 1 Processor -- start 2024-05-09T00:03:32.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.566+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c50a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c50a7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f2704e08b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c5102e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f2704e08b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c51033c0 con 0x5558c509a890 2024-05-09T00:03:32.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f2704e08b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c5103500 con 0x5558c509fe00 2024-05-09T00:03:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f2704e08b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c51036c0 con 0x5558c50a0720 2024-05-09T00:03:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f2704e08b80 10 client.0 _ll_register_callbacks cb 0x5558c4f8fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f2d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c5102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f2d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c5102e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43394/0 (socket says 172.21.15.12:43394) 2024-05-09T00:03:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f2d9d700 1 -- 172.21.15.12:0/3931302693 learned_addr learned my addr 172.21.15.12:0/3931302693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f259c700 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c50a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f1d9b700 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c50a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f259c700 1 -- 172.21.15.12:0/3931302693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 msgr2=0x5558c5102e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f259c700 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c5102e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f259c700 1 -- 172.21.15.12:0/3931302693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 msgr2=0x5558c50a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f259c700 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c50a7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f259c700 1 -- 172.21.15.12:0/3931302693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec0116b0 con 0x5558c509a890 2024-05-09T00:03:32.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.567+0000 7f26f1d9b700 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c50a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:03:32.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.568+0000 7f26f2d9d700 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c5102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:32.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.568+0000 7f26f259c700 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c50a71c0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009fd0 tx=0x7f26e4096de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f47b437bc28ba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:32.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.568+0000 7f26d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:03:32.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.568+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3931302693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e409a400 con 0x5558c509a890 2024-05-09T00:03:32.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.568+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3931302693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f26e40ba4a0 con 0x5558c509a890 2024-05-09T00:03:32.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.568+0000 7f2704e08b80 1 -- 172.21.15.12:0/3931302693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26e40a2e90 con 0x5558c509a890 2024-05-09T00:03:32.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.573+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3931302693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26e409e8d0 con 0x5558c509a890 2024-05-09T00:03:32.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.573+0000 7f2704e08b80 10 client.20181 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:32.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.573+0000 7f2704e08b80 1 -- 172.21.15.12:0/3931302693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5558c4e6fcd0 con 0x5558c509a890 2024-05-09T00:03:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.575+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3931302693 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26e40a2e90 con 0x5558c509a890 2024-05-09T00:03:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.575+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3931302693 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f26e4066280 con 0x5558c509a890 2024-05-09T00:03:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.575+0000 7f2704e08b80 10 client.20181 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.575+0000 7f2704e08b80 10 client.20181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.575+0000 7f2704e08b80 20 client.20181 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.575+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558c510aa60 0x5558c510ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.575+0000 7f2704e08b80 4 client.20181 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:03:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.575+0000 7f2704e08b80 1 -- 172.21.15.12:0/3931302693 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5558c4fe2ec0 con 0x5558c510aa60 2024-05-09T00:03:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.576+0000 7f26f1d9b700 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558c510aa60 0x5558c510ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.577+0000 7f26f1d9b700 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558c510aa60 0x5558c510ce50 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f00fc348758054d6 server_cookie=954efd72948a45f in_seq=0 out_seq=0 2024-05-09T00:03:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.577+0000 7f26d9ffb700 10 client.20181 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.578+0000 7f26d9ffb700 1 -- 172.21.15.12:0/3931302693 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5558c4fe2ec0 con 0x5558c510aa60 2024-05-09T00:03:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.578+0000 7f26d9ffb700 10 client.20181 handle_command_reply: tid=0 2024-05-09T00:03:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.578+0000 7f2704e08b80 1 client.20181 shutdown 2024-05-09T00:03:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f2704e08b80 1 -- 172.21.15.12:0/3931302693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 msgr2=0x5558c50a71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c50a71c0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009fd0 tx=0x7f26e4096de0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f2704e08b80 1 -- 172.21.15.12:0/3931302693 shutdown_connections 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c50a0720 0x5558c5102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558c509a890 0x5558c50a71c0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5558c510aa60 0x5558c510ce50 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f2704e08b80 1 --2- 172.21.15.12:0/3931302693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5558c509fe00 0x5558c50a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f2704e08b80 1 -- 172.21.15.12:0/3931302693 >> 172.21.15.12:0/3931302693 conn(0x5558c4ff8210 msgr2=0x5558c509db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f26d9ffb700 0 client.20181 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f2704e08b80 1 -- 172.21.15.12:0/3931302693 shutdown_connections 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.579+0000 7f2704e08b80 1 -- 172.21.15.12:0/3931302693 wait complete. 2024-05-09T00:03:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.580+0000 7f2704e08b80 20 client.20181 trim_cache size 0 max 16384 2024-05-09T00:03:32.575 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:03:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:03:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:03:32.575 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:03:32.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.580+0000 7f26f8c4e700 1 -- 172.21.15.12:0/3469030807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26dc0a1290 msgr2=0x7f26dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:32.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.580+0000 7f26f8c4e700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26dc0a1290 0x7f26dc0a3750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f26f410ed50 tx=0x7f26e8009260 comp rx=0 tx=0).stop 2024-05-09T00:03:32.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.580+0000 7f26f8c4e700 1 -- 172.21.15.12:0/3469030807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26f41000f0 msgr2=0x7f26f410d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:32.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.580+0000 7f26f8c4e700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26f41000f0 0x7f26f410d620 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f26e40059d0 tx=0x7f26e400b9c0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.580+0000 7f26f8c4e700 1 -- 172.21.15.12:0/3469030807 shutdown_connections 2024-05-09T00:03:32.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.581+0000 7f26f8c4e700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26dc0a1290 0x7f26dc0a3750 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.581+0000 7f26f8c4e700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26f4105f90 0x7f26f410e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.581+0000 7f26f8c4e700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26f41000f0 0x7f26f410d620 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.581+0000 7f26f8c4e700 1 --2- 172.21.15.12:0/3469030807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f4100ab0 0x7f26f410db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.581+0000 7f26f8c4e700 1 -- 172.21.15.12:0/3469030807 >> 172.21.15.12:0/3469030807 conn(0x7f26f4076ba0 msgr2=0x7f26f40feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:32.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.581+0000 7f26f8c4e700 1 -- 172.21.15.12:0/3469030807 shutdown_connections 2024-05-09T00:03:32.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.581+0000 7f26f8c4e700 1 -- 172.21.15.12:0/3469030807 wait complete. 2024-05-09T00:03:32.596 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:03:32.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:03:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.798+0000 7f95ed460700 1 -- 172.21.15.12:0/4211469677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e8106190 msgr2=0x7f95e810dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.798+0000 7f95ed460700 1 --2- 172.21.15.12:0/4211469677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e8106190 0x7f95e810dd10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009990 tx=0x7f95dc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.799+0000 7f95ed460700 1 -- 172.21.15.12:0/4211469677 shutdown_connections 2024-05-09T00:03:32.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.799+0000 7f95ed460700 1 --2- 172.21.15.12:0/4211469677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e8106190 0x7f95e810dd10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.799+0000 7f95ed460700 1 --2- 172.21.15.12:0/4211469677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e8101070 0x7f95e8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.799+0000 7f95ed460700 1 --2- 172.21.15.12:0/4211469677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8101a20 0x7f95e8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.799+0000 7f95ed460700 1 -- 172.21.15.12:0/4211469677 >> 172.21.15.12:0/4211469677 conn(0x7f95e80fc950 msgr2=0x7f95e80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:32.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.800+0000 7f95ed460700 1 -- 172.21.15.12:0/4211469677 shutdown_connections 2024-05-09T00:03:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.800+0000 7f95ed460700 1 -- 172.21.15.12:0/4211469677 wait complete. 2024-05-09T00:03:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.801+0000 7f95ed460700 1 Processor -- start 2024-05-09T00:03:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.801+0000 7f95ed460700 1 -- start start 2024-05-09T00:03:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95ed460700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e8101070 0x7f95e8070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95ed460700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8106190 0x7f95e8070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95ed460700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e806b6d0 0x7f95e806bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95ed460700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e806c290 con 0x7f95e8106190 2024-05-09T00:03:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95ed460700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e806c400 con 0x7f95e8101070 2024-05-09T00:03:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95ed460700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e8074290 con 0x7f95e806b6d0 2024-05-09T00:03:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95e77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e806b6d0 0x7f95e806bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95e77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e806b6d0 0x7f95e806bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43402/0 (socket says 172.21.15.12:43402) 2024-05-09T00:03:32.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95e77fe700 1 -- 172.21.15.12:0/2013011555 learned_addr learned my addr 172.21.15.12:0/2013011555 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95e77fe700 1 -- 172.21.15.12:0/2013011555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e8101070 msgr2=0x7f95e8070590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:03:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95e77fe700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e8101070 0x7f95e8070590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95e77fe700 1 -- 172.21.15.12:0/2013011555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8106190 msgr2=0x7f95e8070ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:03:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95e77fe700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8106190 0x7f95e8070ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.802+0000 7f95e77fe700 1 -- 172.21.15.12:0/2013011555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0095f0 con 0x7f95e806b6d0 2024-05-09T00:03:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.803+0000 7f95e77fe700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e806b6d0 0x7f95e806bb80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f95dc004730 tx=0x7f95dc005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.803+0000 7f95cffff700 1 -- 172.21.15.12:0/2013011555 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc01d070 con 0x7f95e806b6d0 2024-05-09T00:03:32.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.803+0000 7f95ed460700 1 -- 172.21.15.12:0/2013011555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e8074510 con 0x7f95e806b6d0 2024-05-09T00:03:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.803+0000 7f95ed460700 1 -- 172.21.15.12:0/2013011555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e8074a10 con 0x7f95e806b6d0 2024-05-09T00:03:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.804+0000 7f95cffff700 1 -- 172.21.15.12:0/2013011555 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95dc004810 con 0x7f95e806b6d0 2024-05-09T00:03:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.804+0000 7f95cffff700 1 -- 172.21.15.12:0/2013011555 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc005340 con 0x7f95e806b6d0 2024-05-09T00:03:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.805+0000 7f95ed460700 1 -- 172.21.15.12:0/2013011555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d40052f0 con 0x7f95e806b6d0 2024-05-09T00:03:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.808+0000 7f95cffff700 1 -- 172.21.15.12:0/2013011555 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f95dc0054a0 con 0x7f95e806b6d0 2024-05-09T00:03:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.809+0000 7f95cffff700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95d00a1290 0x7f95d00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.809+0000 7f95cffff700 1 -- 172.21.15.12:0/2013011555 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f95dc0ce0d0 con 0x7f95e806b6d0 2024-05-09T00:03:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.810+0000 7f95e6ffd700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95d00a1290 0x7f95d00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.811+0000 7f95e6ffd700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95d00a1290 0x7f95d00a3750 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009730 tx=0x7f95e0006cb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:32.813+0000 7f95cffff700 1 -- 172.21.15.12:0/2013011555 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f95dc097660 con 0x7f95e806b6d0 2024-05-09T00:03:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.146+0000 7f95ed460700 1 -- 172.21.15.12:0/2013011555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95d4005160 con 0x7f95e806b6d0 2024-05-09T00:03:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.147+0000 7f95cffff700 1 -- 172.21.15.12:0/2013011555 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+10875 (secure 0 0 0) 0x7f95dc09ac80 con 0x7f95e806b6d0 2024-05-09T00:03:33.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:03:33.142 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":72,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:03:09.677071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24452":{"gid":24452,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6833/1870735591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":1870735591},{"type":"v1","addr":"172.21.15.160:6833","nonce":1870735591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:03:33.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 -- 172.21.15.12:0/2013011555 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95d00a1290 msgr2=0x7f95d00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95d00a1290 0x7f95d00a3750 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009730 tx=0x7f95e0006cb0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 -- 172.21.15.12:0/2013011555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e806b6d0 msgr2=0x7f95e806bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e806b6d0 0x7f95e806bb80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f95dc004730 tx=0x7f95dc005e80 comp rx=0 tx=0).stop 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 -- 172.21.15.12:0/2013011555 shutdown_connections 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f95d00a1290 0x7f95d00a3750 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e806b6d0 0x7f95e806bb80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e8101070 0x7f95e8070590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 --2- 172.21.15.12:0/2013011555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8106190 0x7f95e8070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 -- 172.21.15.12:0/2013011555 >> 172.21.15.12:0/2013011555 conn(0x7f95e80fc950 msgr2=0x7f95e80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 -- 172.21.15.12:0/2013011555 shutdown_connections 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.150+0000 7f95ed460700 1 -- 172.21.15.12:0/2013011555 wait complete. 2024-05-09T00:03:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2024-05-09T00:03:33.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:03:33.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 150150 38.63 MB/sec execute 88 sec latency 74.660 ms 2024-05-09T00:03:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.329+0000 7fab950be700 1 -- 172.21.15.12:0/3362297722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab90109940 msgr2=0x7fab90109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.329+0000 7fab950be700 1 --2- 172.21.15.12:0/3362297722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab90109940 0x7fab90109d20 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fab80009990 tx=0x7fab80009ca0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.330+0000 7fab950be700 1 -- 172.21.15.12:0/3362297722 shutdown_connections 2024-05-09T00:03:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.330+0000 7fab950be700 1 --2- 172.21.15.12:0/3362297722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab90112ef0 0x7fab901152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.330+0000 7fab950be700 1 --2- 172.21.15.12:0/3362297722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab90109940 0x7fab90109d20 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.330+0000 7fab950be700 1 --2- 172.21.15.12:0/3362297722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab9010a260 0x7fab901129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.330+0000 7fab950be700 1 -- 172.21.15.12:0/3362297722 >> 172.21.15.12:0/3362297722 conn(0x7fab9006bff0 msgr2=0x7fab9006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.330+0000 7fab950be700 1 -- 172.21.15.12:0/3362297722 shutdown_connections 2024-05-09T00:03:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.330+0000 7fab950be700 1 -- 172.21.15.12:0/3362297722 wait complete. 2024-05-09T00:03:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.331+0000 7fab950be700 1 Processor -- start 2024-05-09T00:03:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.331+0000 7fab950be700 1 -- start start 2024-05-09T00:03:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.331+0000 7fab950be700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab90109940 0x7fab901a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.331+0000 7fab950be700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab9010a260 0x7fab901a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.331+0000 7fab950be700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab90112ef0 0x7fab901a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.331+0000 7fab950be700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab901a6ec0 con 0x7fab9010a260 2024-05-09T00:03:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.331+0000 7fab950be700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab901a7030 con 0x7fab90112ef0 2024-05-09T00:03:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.331+0000 7fab950be700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab901a71f0 con 0x7fab90109940 2024-05-09T00:03:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab90109940 0x7fab901a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab90109940 0x7fab901a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43424/0 (socket says 172.21.15.12:43424) 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8ed9d700 1 -- 172.21.15.12:0/2225277573 learned_addr learned my addr 172.21.15.12:0/2225277573 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8f59e700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab90112ef0 0x7fab901a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8e59c700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab9010a260 0x7fab901a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8ed9d700 1 -- 172.21.15.12:0/2225277573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab90112ef0 msgr2=0x7fab901a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab90112ef0 0x7fab901a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8ed9d700 1 -- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab9010a260 msgr2=0x7fab901a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab9010a260 0x7fab901a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8ed9d700 1 -- 172.21.15.12:0/2225277573 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab800095f0 con 0x7fab90109940 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8e59c700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab9010a260 0x7fab901a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8f59e700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab90112ef0 0x7fab901a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.332+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab90109940 0x7fab901a1fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fab80005b40 tx=0x7fab80003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.333+0000 7fab77fff700 1 -- 172.21.15.12:0/2225277573 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8001d070 con 0x7fab90109940 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.333+0000 7fab77fff700 1 -- 172.21.15.12:0/2225277573 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab80003e90 con 0x7fab90109940 2024-05-09T00:03:33.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.333+0000 7fab77fff700 1 -- 172.21.15.12:0/2225277573 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8000fd20 con 0x7fab90109940 2024-05-09T00:03:33.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.333+0000 7fab950be700 1 -- 172.21.15.12:0/2225277573 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab901a7470 con 0x7fab90109940 2024-05-09T00:03:33.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.333+0000 7fab950be700 1 -- 172.21.15.12:0/2225277573 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab901a76f0 con 0x7fab90109940 2024-05-09T00:03:33.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.336+0000 7fab77fff700 1 -- 172.21.15.12:0/2225277573 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fab8002b440 con 0x7fab90109940 2024-05-09T00:03:33.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.337+0000 7fab77fff700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab780a1310 0x7fab780a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.337+0000 7fab77fff700 1 -- 172.21.15.12:0/2225277573 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fab800cdc10 con 0x7fab90109940 2024-05-09T00:03:33.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.337+0000 7fab8e59c700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab780a1310 0x7fab780a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.338+0000 7faba1278b80 1 Processor -- start 2024-05-09T00:03:33.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.338+0000 7faba1278b80 1 -- start start 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.338+0000 7faba1278b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047f0610 0x557a047f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.338+0000 7faba1278b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 0x557a047eab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.338+0000 7faba1278b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047eb0a0 0x557a047eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.338+0000 7faba1278b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a04739150 con 0x557a047f0610 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.338+0000 7faba1278b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a04736b10 con 0x557a047eb0a0 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.338+0000 7faba1278b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a04732400 con 0x557a047f0f30 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8f59e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047eb0a0 0x557a047eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047f0610 0x557a047f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8f59e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047eb0a0 0x557a047eb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33420/0 (socket says 172.21.15.12:33420) 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8f59e700 1 -- 172.21.15.12:0/3640468907 learned_addr learned my addr 172.21.15.12:0/3640468907 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8e59c700 1 --2- 172.21.15.12:0/3640468907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 0x557a047eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8e59c700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab780a1310 0x7fab780a37d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fab901a34f0 tx=0x7fab84009190 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8ed9d700 1 -- 172.21.15.12:0/3640468907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 msgr2=0x557a047eab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/3640468907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 0x557a047eab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8ed9d700 1 -- 172.21.15.12:0/3640468907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047eb0a0 msgr2=0x557a047eb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/3640468907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047eb0a0 0x557a047eb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8ed9d700 1 -- 172.21.15.12:0/3640468907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a045b33b0 con 0x557a047f0610 2024-05-09T00:03:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8f59e700 1 --2- 172.21.15.12:0/3640468907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047eb0a0 0x557a047eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/3640468907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047f0610 0x557a047f09f0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fab800cb310 tx=0x7fab8009ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b850c9879e7d61eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab757fa700 1 -- 172.21.15.12:0/3640468907 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab800cb790 con 0x557a047f0610 2024-05-09T00:03:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab757fa700 1 -- 172.21.15.12:0/3640468907 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab80020580 con 0x557a047f0610 2024-05-09T00:03:33.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.339+0000 7fab757fa700 1 -- 172.21.15.12:0/3640468907 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab800a2ee0 con 0x557a047f0610 2024-05-09T00:03:33.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.340+0000 7faba1278b80 1 -- 172.21.15.12:0/3640468907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047f0610 msgr2=0x557a047f09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.340+0000 7faba1278b80 1 --2- 172.21.15.12:0/3640468907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047f0610 0x557a047f09f0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fab800cb310 tx=0x7fab8009ab70 comp rx=0 tx=0).stop 2024-05-09T00:03:33.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.340+0000 7faba1278b80 1 -- 172.21.15.12:0/3640468907 shutdown_connections 2024-05-09T00:03:33.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.340+0000 7faba1278b80 1 --2- 172.21.15.12:0/3640468907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047eb0a0 0x557a047eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.340+0000 7faba1278b80 1 --2- 172.21.15.12:0/3640468907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047f0610 0x557a047f09f0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.340+0000 7faba1278b80 1 --2- 172.21.15.12:0/3640468907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 0x557a047eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.340+0000 7faba1278b80 1 -- 172.21.15.12:0/3640468907 >> 172.21.15.12:0/3640468907 conn(0x557a04740210 msgr2=0x557a047dcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 1 -- 172.21.15.12:0/3640468907 shutdown_connections 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 1 -- 172.21.15.12:0/3640468907 wait complete. 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 1 -- start start 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 1 Processor -- start 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 0x557a047f4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 0x557a047f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a046ce560 con 0x557a047eb0a0 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a046ce200 con 0x557a047f0610 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a045b56a0 con 0x557a047f0f30 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7faba1278b80 10 client.0 _ll_register_callbacks cb 0x557a046d7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:33.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.341+0000 7fab8e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 0x557a047f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.342+0000 7fab8e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 0x557a047f4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33430/0 (socket says 172.21.15.12:33430) 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.342+0000 7fab8e59c700 1 -- 172.21.15.12:0/1844006053 learned_addr learned my addr 172.21.15.12:0/1844006053 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.342+0000 7fab8f59e700 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 0x557a047f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.342+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.342+0000 7fab8f59e700 1 -- 172.21.15.12:0/1844006053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 msgr2=0x557a047f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.342+0000 7fab8f59e700 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 0x557a047f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.342+0000 7fab8f59e700 1 -- 172.21.15.12:0/1844006053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 msgr2=0x557a047f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.342+0000 7fab8f59e700 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.342+0000 7fab8f59e700 1 -- 172.21.15.12:0/1844006053 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8800a420 con 0x557a047f0f30 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.343+0000 7fab8f59e700 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 0x557a047f49a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fab8800afd0 tx=0x7fab88011540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7b9a0ef39062f71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.343+0000 7fab72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.343+0000 7fab72ff5700 1 -- 172.21.15.12:0/1844006053 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88010040 con 0x557a047f0f30 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.343+0000 7fab72ff5700 1 -- 172.21.15.12:0/1844006053 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fab88006e40 con 0x557a047f0f30 2024-05-09T00:03:33.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.343+0000 7faba1278b80 1 -- 172.21.15.12:0/1844006053 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a0468b270 con 0x557a047f0f30 2024-05-09T00:03:33.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.343+0000 7fab72ff5700 1 -- 172.21.15.12:0/1844006053 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab88016450 con 0x557a047f0f30 2024-05-09T00:03:33.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.343+0000 7faba1278b80 10 client.29932 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:33.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.343+0000 7faba1278b80 1 -- 172.21.15.12:0/1844006053 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7fab901a7470 con 0x557a047f0f30 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.343+0000 7fab72ff5700 1 -- 172.21.15.12:0/1844006053 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a0468b270 con 0x557a047f0f30 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.344+0000 7fab72ff5700 1 -- 172.21.15.12:0/1844006053 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fab88008030 con 0x557a047f0f30 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.344+0000 7faba1278b80 10 client.29932 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.344+0000 7faba1278b80 10 client.29932 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.344+0000 7faba1278b80 20 client.29932 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.344+0000 7faba1278b80 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557a047e3350 0x557a047e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.344+0000 7faba1278b80 4 client.29932 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.344+0000 7faba1278b80 1 -- 172.21.15.12:0/1844006053 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a047b1810 con 0x557a047e3350 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.345+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557a047e3350 0x557a047e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.345+0000 7fab72ff5700 10 client.29932 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:03:33.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.345+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557a047e3350 0x557a047e5770 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1e224c9243440c1 server_cookie=5557ac61888f16d2 in_seq=0 out_seq=0 2024-05-09T00:03:33.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7fab72ff5700 1 -- 172.21.15.12:0/1844006053 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a047b1810 con 0x557a047e3350 2024-05-09T00:03:33.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7fab72ff5700 10 client.29932 handle_command_reply: tid=0 2024-05-09T00:03:33.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7faba1278b80 1 client.29932 shutdown 2024-05-09T00:03:33.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7faba1278b80 1 -- 172.21.15.12:0/1844006053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 msgr2=0x557a047f49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7faba1278b80 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 0x557a047f49a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fab8800afd0 tx=0x7fab88011540 comp rx=0 tx=0).stop 2024-05-09T00:03:33.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7faba1278b80 1 -- 172.21.15.12:0/1844006053 shutdown_connections 2024-05-09T00:03:33.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7faba1278b80 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557a047e3350 0x557a047e5770 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7faba1278b80 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0f30 0x557a047f49a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7faba1278b80 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7fab72ff5700 0 client.29932 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:03:33.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7faba1278b80 1 --2- 172.21.15.12:0/1844006053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 0x557a047f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7faba1278b80 1 -- 172.21.15.12:0/1844006053 >> 172.21.15.12:0/1844006053 conn(0x557a04740210 msgr2=0x557a047dc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.348+0000 7faba1278b80 1 -- 172.21.15.12:0/1844006053 shutdown_connections 2024-05-09T00:03:33.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.349+0000 7faba1278b80 1 -- 172.21.15.12:0/1844006053 wait complete. 2024-05-09T00:03:33.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.349+0000 7faba1278b80 20 client.29932 trim_cache size 0 max 16384 2024-05-09T00:03:33.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7faba1278b80 1 Processor -- start 2024-05-09T00:03:33.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7faba1278b80 1 -- start start 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7faba1278b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047e3350 0x557a047f4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7faba1278b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047eb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7faba1278b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0610 0x557a047dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7faba1278b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a047f2320 con 0x557a047eb0a0 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7faba1278b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a047eb9c0 con 0x557a047e3350 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7faba1278b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a047dfe80 con 0x557a047f0610 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7fab8f59e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0610 0x557a047dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7fab8ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047e3350 0x557a047f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7fab8f59e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0610 0x557a047dcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43452/0 (socket says 172.21.15.12:43452) 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7fab8ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047e3350 0x557a047f4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33444/0 (socket says 172.21.15.12:33444) 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7fab8f59e700 1 -- 172.21.15.12:0/3306782405 learned_addr learned my addr 172.21.15.12:0/3306782405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.367+0000 7fab8e59c700 1 --2- 172.21.15.12:0/3306782405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.368+0000 7fab8f59e700 1 -- 172.21.15.12:0/3306782405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047e3350 msgr2=0x557a047f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.368+0000 7fab8f59e700 1 --2- 172.21.15.12:0/3306782405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047e3350 0x557a047f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.368+0000 7fab8f59e700 1 -- 172.21.15.12:0/3306782405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 msgr2=0x557a047eb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.368+0000 7fab8f59e700 1 --2- 172.21.15.12:0/3306782405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047eb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.368+0000 7fab8f59e700 1 -- 172.21.15.12:0/3306782405 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a0455fb60 con 0x557a047f0610 2024-05-09T00:03:33.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.368+0000 7fab8f59e700 1 --2- 172.21.15.12:0/3306782405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0610 0x557a047dcd20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fab8800a010 tx=0x7fab88009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab05fe91812b0a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.368+0000 7fab75ffb700 1 -- 172.21.15.12:0/3306782405 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88010040 con 0x557a047f0610 2024-05-09T00:03:33.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.368+0000 7fab75ffb700 1 -- 172.21.15.12:0/3306782405 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab88006e40 con 0x557a047f0610 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.368+0000 7fab75ffb700 1 -- 172.21.15.12:0/3306782405 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88016b70 con 0x557a047f0610 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.369+0000 7faba1278b80 1 -- 172.21.15.12:0/3306782405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0610 msgr2=0x557a047dcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.369+0000 7faba1278b80 1 --2- 172.21.15.12:0/3306782405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0610 0x557a047dcd20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fab8800a010 tx=0x7fab88009760 comp rx=0 tx=0).stop 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.369+0000 7faba1278b80 1 -- 172.21.15.12:0/3306782405 shutdown_connections 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.369+0000 7faba1278b80 1 --2- 172.21.15.12:0/3306782405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047f0610 0x557a047dcd20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.369+0000 7faba1278b80 1 --2- 172.21.15.12:0/3306782405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047e3350 0x557a047f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.369+0000 7faba1278b80 1 --2- 172.21.15.12:0/3306782405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047eb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.369+0000 7faba1278b80 1 -- 172.21.15.12:0/3306782405 >> 172.21.15.12:0/3306782405 conn(0x557a04740210 msgr2=0x557a047d9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.370+0000 7faba1278b80 1 -- 172.21.15.12:0/3306782405 shutdown_connections 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.370+0000 7faba1278b80 1 -- 172.21.15.12:0/3306782405 wait complete. 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.370+0000 7faba1278b80 1 -- start start 2024-05-09T00:03:33.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.370+0000 7faba1278b80 1 Processor -- start 2024-05-09T00:03:33.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7faba1278b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047e3350 0x557a047e6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7faba1278b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7faba1278b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 0x557a047e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7faba1278b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a04862060 con 0x557a047eb0a0 2024-05-09T00:03:33.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7faba1278b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a048621d0 con 0x557a047f0610 2024-05-09T00:03:33.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7faba1278b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a04862390 con 0x557a047e3350 2024-05-09T00:03:33.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7faba1278b80 10 client.0 _ll_register_callbacks cb 0x557a046d7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:33.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047e3350 0x557a047e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8f59e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 0x557a047e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047e3350 0x557a047e6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43464/0 (socket says 172.21.15.12:43464) 2024-05-09T00:03:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8ed9d700 1 -- 172.21.15.12:0/78849599 learned_addr learned my addr 172.21.15.12:0/78849599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047e66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58310/0 (socket says 172.21.15.12:58310) 2024-05-09T00:03:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8ed9d700 1 -- 172.21.15.12:0/78849599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 msgr2=0x557a047e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 0x557a047e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8ed9d700 1 -- 172.21.15.12:0/78849599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 msgr2=0x557a047e66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047e66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8ed9d700 1 -- 172.21.15.12:0/78849599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab901a7470 con 0x557a047e3350 2024-05-09T00:03:33.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab8ed9d700 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047e3350 0x557a047e6170 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fab80011040 tx=0x7fab8009b5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ef58aba78f8e100 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.371+0000 7fab72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:03:33.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7fab72ff5700 1 -- 172.21.15.12:0/78849599 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8009b1c0 con 0x557a047e3350 2024-05-09T00:03:33.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7fab72ff5700 1 -- 172.21.15.12:0/78849599 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fab8000f460 con 0x557a047e3350 2024-05-09T00:03:33.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7faba1278b80 1 -- 172.21.15.12:0/78849599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab80020580 con 0x557a047e3350 2024-05-09T00:03:33.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7fab72ff5700 1 -- 172.21.15.12:0/78849599 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab800a31d0 con 0x557a047e3350 2024-05-09T00:03:33.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7faba1278b80 10 client.29938 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:33.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7faba1278b80 1 -- 172.21.15.12:0/78849599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x557a046f8560 con 0x557a047e3350 2024-05-09T00:03:33.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7fab72ff5700 1 -- 172.21.15.12:0/78849599 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab80020580 con 0x557a047e3350 2024-05-09T00:03:33.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7fab72ff5700 1 -- 172.21.15.12:0/78849599 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fab800aeba0 con 0x557a047e3350 2024-05-09T00:03:33.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7faba1278b80 10 client.29938 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:33.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7faba1278b80 10 client.29938 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:03:33.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7faba1278b80 20 client.29938 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:33.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7faba1278b80 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557a047f0f30 0x557a048657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7faba1278b80 4 client.29938 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:03:33.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.372+0000 7faba1278b80 1 -- 172.21.15.12:0/78849599 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557a0472aec0 con 0x557a047f0f30 2024-05-09T00:03:33.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7fab8e59c700 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557a047f0f30 0x557a048657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7fab72ff5700 10 client.29938 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:03:33.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7fab8e59c700 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557a047f0f30 0x557a048657d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1e97b3ac78f41ae server_cookie=4ac0b23736e6fc2c in_seq=0 out_seq=0 2024-05-09T00:03:33.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7fab72ff5700 1 -- 172.21.15.12:0/78849599 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557a0472aec0 con 0x557a047f0f30 2024-05-09T00:03:33.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7fab72ff5700 10 client.29938 handle_command_reply: tid=0 2024-05-09T00:03:33.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7faba1278b80 1 client.29938 shutdown 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7faba1278b80 1 -- 172.21.15.12:0/78849599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047e3350 msgr2=0x557a047e6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7faba1278b80 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047e3350 0x557a047e6170 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fab80011040 tx=0x7fab8009b5a0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7faba1278b80 1 -- 172.21.15.12:0/78849599 shutdown_connections 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7faba1278b80 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a047f0610 0x557a047e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7faba1278b80 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x557a047f0f30 0x557a048657d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7faba1278b80 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a047e3350 0x557a047e6170 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7faba1278b80 1 --2- 172.21.15.12:0/78849599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a047eb0a0 0x557a047e66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.373+0000 7faba1278b80 1 -- 172.21.15.12:0/78849599 >> 172.21.15.12:0/78849599 conn(0x557a04740210 msgr2=0x557a04815750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.374+0000 7fab72ff5700 0 client.29938 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.374+0000 7faba1278b80 1 -- 172.21.15.12:0/78849599 shutdown_connections 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.374+0000 7faba1278b80 1 -- 172.21.15.12:0/78849599 wait complete. 2024-05-09T00:03:33.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.374+0000 7faba1278b80 20 client.29938 trim_cache size 0 max 16384 2024-05-09T00:03:33.378 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:03:33.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.374+0000 7fab950be700 1 -- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab780a1310 msgr2=0x7fab780a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.374+0000 7fab950be700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab780a1310 0x7fab780a37d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fab901a34f0 tx=0x7fab84009190 comp rx=0 tx=0).stop 2024-05-09T00:03:33.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.375+0000 7fab950be700 1 -- 172.21.15.12:0/2225277573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab90109940 msgr2=0x7fab901a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.375+0000 7fab950be700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab90109940 0x7fab901a1fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fab80005b40 tx=0x7fab80003a90 comp rx=0 tx=0).stop 2024-05-09T00:03:33.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.375+0000 7fab950be700 1 -- 172.21.15.12:0/2225277573 shutdown_connections 2024-05-09T00:03:33.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.375+0000 7fab950be700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab780a1310 0x7fab780a37d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.375+0000 7fab950be700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab90112ef0 0x7fab901a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.375+0000 7fab950be700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab90109940 0x7fab901a1fa0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.375+0000 7fab950be700 1 --2- 172.21.15.12:0/2225277573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab9010a260 0x7fab901a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.375+0000 7fab950be700 1 -- 172.21.15.12:0/2225277573 >> 172.21.15.12:0/2225277573 conn(0x7fab9006bff0 msgr2=0x7fab90071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.375+0000 7fab950be700 1 -- 172.21.15.12:0/2225277573 shutdown_connections 2024-05-09T00:03:33.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.375+0000 7fab950be700 1 -- 172.21.15.12:0/2225277573 wait complete. 2024-05-09T00:03:33.387 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:03:33.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:33 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2013011555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:03:33.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:33 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2013011555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:03:33.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:33 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2013011555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:03:33.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.563+0000 7fc73aaac700 1 -- 172.21.15.12:0/873331040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7340ffdd0 msgr2=0x7fc73410dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.563+0000 7fc73aaac700 1 --2- 172.21.15.12:0/873331040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7340ffdd0 0x7fc73410dda0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00b0a0 tx=0x7fc72c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.563+0000 7fc73aaac700 1 -- 172.21.15.12:0/873331040 shutdown_connections 2024-05-09T00:03:33.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.563+0000 7fc73aaac700 1 --2- 172.21.15.12:0/873331040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7340ffdd0 0x7fc73410dda0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.563+0000 7fc73aaac700 1 --2- 172.21.15.12:0/873331040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc734105ef0 0x7fc7341062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.563+0000 7fc73aaac700 1 --2- 172.21.15.12:0/873331040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7340ff410 0x7fc7340ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.563+0000 7fc73aaac700 1 -- 172.21.15.12:0/873331040 >> 172.21.15.12:0/873331040 conn(0x7fc7340763a0 msgr2=0x7fc7340fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.563+0000 7fc73aaac700 1 -- 172.21.15.12:0/873331040 shutdown_connections 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.563+0000 7fc73aaac700 1 -- 172.21.15.12:0/873331040 wait complete. 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc73aaac700 1 Processor -- start 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc73aaac700 1 -- start start 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc73aaac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7340ff410 0x7fc7341975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc73aaac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7340ffdd0 0x7fc734197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc73aaac700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc734105ef0 0x7fc73419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc73aaac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73419c4d0 con 0x7fc7340ff410 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc73aaac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73419c640 con 0x7fc7340ffdd0 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc73aaac700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73419c800 con 0x7fc734105ef0 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc738848700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7340ff410 0x7fc7341975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc739049700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc734105ef0 0x7fc73419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc738848700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7340ff410 0x7fc7341975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58324/0 (socket says 172.21.15.12:58324) 2024-05-09T00:03:33.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc738848700 1 -- 172.21.15.12:0/2231781545 learned_addr learned my addr 172.21.15.12:0/2231781545 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.564+0000 7fc733fff700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7340ffdd0 0x7fc734197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.565+0000 7fc739049700 1 -- 172.21.15.12:0/2231781545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7340ffdd0 msgr2=0x7fc734197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.565+0000 7fc739049700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7340ffdd0 0x7fc734197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.565+0000 7fc739049700 1 -- 172.21.15.12:0/2231781545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7340ff410 msgr2=0x7fc7341975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.565+0000 7fc739049700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7340ff410 0x7fc7341975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.565+0000 7fc739049700 1 -- 172.21.15.12:0/2231781545 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728009520 con 0x7fc734105ef0 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.565+0000 7fc739049700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc734105ef0 0x7fc73419be90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00b0a0 tx=0x7fc72c0078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.565+0000 7fc731ffb700 1 -- 172.21.15.12:0/2231781545 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00e040 con 0x7fc734105ef0 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.565+0000 7fc731ffb700 1 -- 172.21.15.12:0/2231781545 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc72c020070 con 0x7fc734105ef0 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.565+0000 7fc73aaac700 1 -- 172.21.15.12:0/2231781545 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72c009d00 con 0x7fc734105ef0 2024-05-09T00:03:33.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.565+0000 7fc731ffb700 1 -- 172.21.15.12:0/2231781545 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c027480 con 0x7fc734105ef0 2024-05-09T00:03:33.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.566+0000 7fc73aaac700 1 -- 172.21.15.12:0/2231781545 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc734100520 con 0x7fc734105ef0 2024-05-09T00:03:33.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.568+0000 7fc731ffb700 1 -- 172.21.15.12:0/2231781545 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc72c01c6c0 con 0x7fc734105ef0 2024-05-09T00:03:33.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.569+0000 7fc731ffb700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7240a1290 0x7fc7240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.569+0000 7fc738848700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7240a1290 0x7fc7240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.569+0000 7fc731ffb700 1 -- 172.21.15.12:0/2231781545 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc72c022070 con 0x7fc734105ef0 2024-05-09T00:03:33.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.570+0000 7fc738848700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7240a1290 0x7fc7240a3750 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc728009e70 tx=0x7fc728009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.570+0000 7fc746c66b80 1 Processor -- start 2024-05-09T00:03:33.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.570+0000 7fc746c66b80 1 -- start start 2024-05-09T00:03:33.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.570+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2610 0x563e441f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.570+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2f30 0x563e441ecb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.570+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e441ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.570+0000 7fc746c66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e4413b150 con 0x563e441f2f30 2024-05-09T00:03:33.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.570+0000 7fc746c66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e44138b10 con 0x563e441ed0a0 2024-05-09T00:03:33.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.570+0000 7fc746c66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e44134400 con 0x563e441f2610 2024-05-09T00:03:33.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc733fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2f30 0x563e441ecb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc733fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2f30 0x563e441ecb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58338/0 (socket says 172.21.15.12:58338) 2024-05-09T00:03:33.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc733fff700 1 -- 172.21.15.12:0/2437152977 learned_addr learned my addr 172.21.15.12:0/2437152977 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc738848700 1 --2- 172.21.15.12:0/2437152977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2610 0x563e441f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc739049700 1 --2- 172.21.15.12:0/2437152977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e441ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc733fff700 1 -- 172.21.15.12:0/2437152977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2610 msgr2=0x563e441f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc733fff700 1 --2- 172.21.15.12:0/2437152977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2610 0x563e441f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc733fff700 1 -- 172.21.15.12:0/2437152977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 msgr2=0x563e441ed4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc733fff700 1 --2- 172.21.15.12:0/2437152977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e441ed4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc733fff700 1 -- 172.21.15.12:0/2437152977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e43fb53b0 con 0x563e441f2f30 2024-05-09T00:03:33.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc733fff700 1 --2- 172.21.15.12:0/2437152977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2f30 0x563e441ecb60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc720009d20 tx=0x7fc720011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5dbd51e8fd14e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc71effd700 1 -- 172.21.15.12:0/2437152977 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720019630 con 0x563e441f2f30 2024-05-09T00:03:33.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.571+0000 7fc71effd700 1 -- 172.21.15.12:0/2437152977 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc720019c50 con 0x563e441f2f30 2024-05-09T00:03:33.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.572+0000 7fc71effd700 1 -- 172.21.15.12:0/2437152977 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000ec40 con 0x563e441f2f30 2024-05-09T00:03:33.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.572+0000 7fc746c66b80 1 -- 172.21.15.12:0/2437152977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2f30 msgr2=0x563e441ecb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.572+0000 7fc746c66b80 1 --2- 172.21.15.12:0/2437152977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2f30 0x563e441ecb60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc720009d20 tx=0x7fc720011720 comp rx=0 tx=0).stop 2024-05-09T00:03:33.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.572+0000 7fc746c66b80 1 -- 172.21.15.12:0/2437152977 shutdown_connections 2024-05-09T00:03:33.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.572+0000 7fc746c66b80 1 --2- 172.21.15.12:0/2437152977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e441ed4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.572+0000 7fc746c66b80 1 --2- 172.21.15.12:0/2437152977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2610 0x563e441f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.572+0000 7fc746c66b80 1 --2- 172.21.15.12:0/2437152977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2f30 0x563e441ecb60 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.572+0000 7fc746c66b80 1 -- 172.21.15.12:0/2437152977 >> 172.21.15.12:0/2437152977 conn(0x563e44142210 msgr2=0x563e441dedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.573+0000 7fc746c66b80 1 -- 172.21.15.12:0/2437152977 shutdown_connections 2024-05-09T00:03:33.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.573+0000 7fc746c66b80 1 -- 172.21.15.12:0/2437152977 wait complete. 2024-05-09T00:03:33.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.573+0000 7fc746c66b80 1 -- start start 2024-05-09T00:03:33.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.573+0000 7fc746c66b80 1 Processor -- start 2024-05-09T00:03:33.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441ed0a0 0x563e441f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441f6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441f2f30 0x563e441f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc746c66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e440d0560 con 0x563e441f2610 2024-05-09T00:03:33.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc746c66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e440d0200 con 0x563e441f2f30 2024-05-09T00:03:33.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc746c66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e43fb76a0 con 0x563e441ed0a0 2024-05-09T00:03:33.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc746c66b80 10 client.0 _ll_register_callbacks cb 0x563e440d9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:33.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc739049700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441f2f30 0x563e441f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc738848700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441ed0a0 0x563e441f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc739049700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441f2f30 0x563e441f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33514/0 (socket says 172.21.15.12:33514) 2024-05-09T00:03:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc739049700 1 -- 172.21.15.12:0/2915006198 learned_addr learned my addr 172.21.15.12:0/2915006198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc733fff700 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc738848700 1 -- 172.21.15.12:0/2915006198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441f2f30 msgr2=0x563e441f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc738848700 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441f2f30 0x563e441f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc738848700 1 -- 172.21.15.12:0/2915006198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 msgr2=0x563e441f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc738848700 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc738848700 1 -- 172.21.15.12:0/2915006198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc720009980 con 0x563e441ed0a0 2024-05-09T00:03:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc739049700 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441f2f30 0x563e441f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.574+0000 7fc733fff700 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc738848700 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441ed0a0 0x563e441f62c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc728009c50 tx=0x7fc7280047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=befcb85087618b83 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc713fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc713fff700 1 -- 172.21.15.12:0/2915006198 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72801fc70 con 0x563e441ed0a0 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc746c66b80 1 -- 172.21.15.12:0/2915006198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e4408d270 con 0x563e441ed0a0 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc713fff700 1 -- 172.21.15.12:0/2915006198 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc72801fdd0 con 0x563e441ed0a0 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc713fff700 1 -- 172.21.15.12:0/2915006198 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc728028440 con 0x563e441ed0a0 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc746c66b80 10 client.29950 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc746c66b80 1 -- 172.21.15.12:0/2915006198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x563e43f61b60 con 0x563e441ed0a0 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc713fff700 1 -- 172.21.15.12:0/2915006198 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc728034080 con 0x563e441ed0a0 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc713fff700 1 -- 172.21.15.12:0/2915006198 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc728030650 con 0x563e441ed0a0 2024-05-09T00:03:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc746c66b80 10 client.29950 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.575+0000 7fc746c66b80 10 client.29950 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:03:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.576+0000 7fc746c66b80 20 client.29950 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.576+0000 7fc746c66b80 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563e44206ac0 0x563e44208ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.576+0000 7fc746c66b80 4 client.29950 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.576+0000 7fc746c66b80 1 -- 172.21.15.12:0/2915006198 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e441b3810 con 0x563e44206ac0 2024-05-09T00:03:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.576+0000 7fc733fff700 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563e44206ac0 0x563e44208ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.576+0000 7fc733fff700 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563e44206ac0 0x563e44208ee0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75e99e34a17e59de server_cookie=2772b221184f0c4d in_seq=0 out_seq=0 2024-05-09T00:03:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.576+0000 7fc713fff700 10 client.29950 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.581+0000 7fc713fff700 1 -- 172.21.15.12:0/2915006198 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e441b3810 con 0x563e44206ac0 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.581+0000 7fc713fff700 10 client.29950 handle_command_reply: tid=0 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.581+0000 7fc746c66b80 1 client.29950 shutdown 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 1 -- 172.21.15.12:0/2915006198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441ed0a0 msgr2=0x563e441f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441ed0a0 0x563e441f62c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc728009c50 tx=0x7fc7280047c0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 1 -- 172.21.15.12:0/2915006198 shutdown_connections 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563e44206ac0 0x563e44208ee0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441f2f30 0x563e441f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc713fff700 0 client.29950 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441ed0a0 0x563e441f62c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 1 --2- 172.21.15.12:0/2915006198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 1 -- 172.21.15.12:0/2915006198 >> 172.21.15.12:0/2915006198 conn(0x563e44142210 msgr2=0x563e441de5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 1 -- 172.21.15.12:0/2915006198 shutdown_connections 2024-05-09T00:03:33.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 1 -- 172.21.15.12:0/2915006198 wait complete. 2024-05-09T00:03:33.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.582+0000 7fc746c66b80 20 client.29950 trim_cache size 0 max 16384 2024-05-09T00:03:33.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc746c66b80 1 Processor -- start 2024-05-09T00:03:33.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc746c66b80 1 -- start start 2024-05-09T00:03:33.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e44202430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441ded20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 0x563e441f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc746c66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e441f4830 con 0x563e441f2610 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc746c66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e441f7890 con 0x563e441ed0a0 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc746c66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e441f9350 con 0x563e441f2f30 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc733fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441ded20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc733fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441ded20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58360/0 (socket says 172.21.15.12:58360) 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc733fff700 1 -- 172.21.15.12:0/372573079 learned_addr learned my addr 172.21.15.12:0/372573079 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc738848700 1 --2- 172.21.15.12:0/372573079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e44202430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.600+0000 7fc739049700 1 --2- 172.21.15.12:0/372573079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 0x563e441f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.601+0000 7fc733fff700 1 -- 172.21.15.12:0/372573079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 msgr2=0x563e441f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.601+0000 7fc733fff700 1 --2- 172.21.15.12:0/372573079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 0x563e441f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.601+0000 7fc733fff700 1 -- 172.21.15.12:0/372573079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 msgr2=0x563e44202430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.601+0000 7fc733fff700 1 --2- 172.21.15.12:0/372573079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e44202430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.601+0000 7fc733fff700 1 -- 172.21.15.12:0/372573079 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e440fa560 con 0x563e441f2610 2024-05-09T00:03:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.601+0000 7fc733fff700 1 --2- 172.21.15.12:0/372573079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441ded20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fc72001c040 tx=0x7fc72000e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2259dc10e996b4cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.601+0000 7fc71f7fe700 1 -- 172.21.15.12:0/372573079 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720029070 con 0x563e441f2610 2024-05-09T00:03:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.601+0000 7fc71f7fe700 1 -- 172.21.15.12:0/372573079 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc72000e4d0 con 0x563e441f2610 2024-05-09T00:03:33.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.601+0000 7fc71f7fe700 1 -- 172.21.15.12:0/372573079 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200197f0 con 0x563e441f2610 2024-05-09T00:03:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.602+0000 7fc746c66b80 1 -- 172.21.15.12:0/372573079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 msgr2=0x563e441ded20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.602+0000 7fc746c66b80 1 --2- 172.21.15.12:0/372573079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441ded20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fc72001c040 tx=0x7fc72000e740 comp rx=0 tx=0).stop 2024-05-09T00:03:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.602+0000 7fc746c66b80 1 -- 172.21.15.12:0/372573079 shutdown_connections 2024-05-09T00:03:33.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.602+0000 7fc746c66b80 1 --2- 172.21.15.12:0/372573079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 0x563e441f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.602+0000 7fc746c66b80 1 --2- 172.21.15.12:0/372573079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e44202430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.602+0000 7fc746c66b80 1 --2- 172.21.15.12:0/372573079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441ded20 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.602+0000 7fc746c66b80 1 -- 172.21.15.12:0/372573079 >> 172.21.15.12:0/372573079 conn(0x563e44142210 msgr2=0x563e441db210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.602+0000 7fc746c66b80 1 -- 172.21.15.12:0/372573079 shutdown_connections 2024-05-09T00:03:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.603+0000 7fc746c66b80 1 -- 172.21.15.12:0/372573079 wait complete. 2024-05-09T00:03:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.603+0000 7fc746c66b80 1 -- start start 2024-05-09T00:03:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.603+0000 7fc746c66b80 1 Processor -- start 2024-05-09T00:03:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e441fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc746c66b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 0x563e441fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc746c66b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e4425bf80 con 0x563e441f2610 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc746c66b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e4425c0c0 con 0x563e441ed0a0 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc746c66b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e4425c230 con 0x563e441f2f30 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc746c66b80 10 client.0 _ll_register_callbacks cb 0x563e440d9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc738848700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e441fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc738848700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e441fcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33530/0 (socket says 172.21.15.12:33530) 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc738848700 1 -- 172.21.15.12:0/3432844242 learned_addr learned my addr 172.21.15.12:0/3432844242 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc739049700 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 0x563e441fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc738848700 1 -- 172.21.15.12:0/3432844242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 msgr2=0x563e441fd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc738848700 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 0x563e441fd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc738848700 1 -- 172.21.15.12:0/3432844242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 msgr2=0x563e441fd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc738848700 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc738848700 1 -- 172.21.15.12:0/3432844242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e43fb53b0 con 0x563e441ed0a0 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.604+0000 7fc739049700 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 0x563e441fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.605+0000 7fc738848700 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e441fcba0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc728030a30 tx=0x7fc72800ff80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a677f60b81ae8c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.605+0000 7fc713fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.605+0000 7fc713fff700 1 -- 172.21.15.12:0/3432844242 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72801fc70 con 0x563e441ed0a0 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.605+0000 7fc746c66b80 1 -- 172.21.15.12:0/3432844242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc720019c50 con 0x563e441ed0a0 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.605+0000 7fc713fff700 1 -- 172.21.15.12:0/3432844242 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc72801fdd0 con 0x563e441ed0a0 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc713fff700 1 -- 172.21.15.12:0/3432844242 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc728004040 con 0x563e441ed0a0 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc746c66b80 10 client.29867 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc746c66b80 1 -- 172.21.15.12:0/3432844242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x563e43fb9cd0 con 0x563e441ed0a0 2024-05-09T00:03:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc713fff700 1 -- 172.21.15.12:0/3432844242 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc720019c50 con 0x563e441ed0a0 2024-05-09T00:03:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc713fff700 1 -- 172.21.15.12:0/3432844242 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc728003bb0 con 0x563e441ed0a0 2024-05-09T00:03:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc746c66b80 10 client.29867 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc746c66b80 10 client.29867 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:03:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc746c66b80 20 client.29867 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc746c66b80 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563e4425da00 0x563e4425fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc746c66b80 4 client.29867 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:03:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc746c66b80 1 -- 172.21.15.12:0/3432844242 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563e4412cec0 con 0x563e4425da00 2024-05-09T00:03:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.606+0000 7fc733fff700 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563e4425da00 0x563e4425fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.607+0000 7fc733fff700 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563e4425da00 0x563e4425fe20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=387e7cbd0e6582e1 server_cookie=f40477a3d2874dd8 in_seq=0 out_seq=0 2024-05-09T00:03:33.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.607+0000 7fc713fff700 10 client.29867 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:03:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.610+0000 7fc713fff700 1 -- 172.21.15.12:0/3432844242 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563e4412cec0 con 0x563e4425da00 2024-05-09T00:03:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.610+0000 7fc713fff700 10 client.29867 handle_command_reply: tid=0 2024-05-09T00:03:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.610+0000 7fc746c66b80 1 client.29867 shutdown 2024-05-09T00:03:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.611+0000 7fc746c66b80 1 -- 172.21.15.12:0/3432844242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 msgr2=0x563e441fcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.611+0000 7fc746c66b80 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e441fcba0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc728030a30 tx=0x7fc72800ff80 comp rx=0 tx=0).stop 2024-05-09T00:03:33.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.611+0000 7fc746c66b80 1 -- 172.21.15.12:0/3432844242 shutdown_connections 2024-05-09T00:03:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.611+0000 7fc746c66b80 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563e4425da00 0x563e4425fe20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.611+0000 7fc746c66b80 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e441f2f30 0x563e441fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.611+0000 7fc746c66b80 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e441ed0a0 0x563e441fcba0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.611+0000 7fc713fff700 0 client.29867 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:03:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.611+0000 7fc746c66b80 1 --2- 172.21.15.12:0/3432844242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e441f2610 0x563e441fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.611+0000 7fc746c66b80 1 -- 172.21.15.12:0/3432844242 >> 172.21.15.12:0/3432844242 conn(0x563e44142210 msgr2=0x563e441f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.611+0000 7fc746c66b80 1 -- 172.21.15.12:0/3432844242 shutdown_connections 2024-05-09T00:03:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.612+0000 7fc746c66b80 1 -- 172.21.15.12:0/3432844242 wait complete. 2024-05-09T00:03:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.612+0000 7fc746c66b80 20 client.29867 trim_cache size 0 max 16384 2024-05-09T00:03:33.609 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:03:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 -- 172.21.15.12:0/2231781545 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7240a1290 msgr2=0x7fc7240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7240a1290 0x7fc7240a3750 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc728009e70 tx=0x7fc728009260 comp rx=0 tx=0).stop 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 -- 172.21.15.12:0/2231781545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc734105ef0 msgr2=0x7fc73419be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc734105ef0 0x7fc73419be90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00b0a0 tx=0x7fc72c0078e0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 -- 172.21.15.12:0/2231781545 shutdown_connections 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc7240a1290 0x7fc7240a3750 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc734105ef0 0x7fc73419be90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7340ff410 0x7fc7341975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 --2- 172.21.15.12:0/2231781545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7340ffdd0 0x7fc734197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 -- 172.21.15.12:0/2231781545 >> 172.21.15.12:0/2231781545 conn(0x7fc7340763a0 msgr2=0x7fc734104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 -- 172.21.15.12:0/2231781545 shutdown_connections 2024-05-09T00:03:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.613+0000 7fc73aaac700 1 -- 172.21.15.12:0/2231781545 wait complete. 2024-05-09T00:03:33.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:03:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.846+0000 7fa0436bf700 1 -- 172.21.15.12:0/275648018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa03c100060 msgr2=0x7fa03c100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.846+0000 7fa0436bf700 1 --2- 172.21.15.12:0/275648018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa03c100060 0x7fa03c100440 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa038009990 tx=0x7fa038009ca0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.846+0000 7fa0436bf700 1 -- 172.21.15.12:0/275648018 shutdown_connections 2024-05-09T00:03:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.846+0000 7fa0436bf700 1 --2- 172.21.15.12:0/275648018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa03c105180 0x7fa03c10cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.846+0000 7fa0436bf700 1 --2- 172.21.15.12:0/275648018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa03c100060 0x7fa03c100440 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.846+0000 7fa0436bf700 1 --2- 172.21.15.12:0/275648018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03c100a10 0x7fa03c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.846+0000 7fa0436bf700 1 -- 172.21.15.12:0/275648018 >> 172.21.15.12:0/275648018 conn(0x7fa03c076f70 msgr2=0x7fa03c077380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.846+0000 7fa0436bf700 1 -- 172.21.15.12:0/275648018 shutdown_connections 2024-05-09T00:03:33.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.846+0000 7fa0436bf700 1 -- 172.21.15.12:0/275648018 wait complete. 2024-05-09T00:03:33.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.847+0000 7fa0436bf700 1 Processor -- start 2024-05-09T00:03:33.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.847+0000 7fa0436bf700 1 -- start start 2024-05-09T00:03:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.847+0000 7fa0436bf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03c100060 0x7fa03c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.847+0000 7fa0436bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa03c105180 0x7fa03c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.847+0000 7fa0436bf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa03c198d10 0x7fa03c1a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.847+0000 7fa0436bf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c1a39f0 con 0x7fa03c105180 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.847+0000 7fa0436bf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c1a3b60 con 0x7fa03c100060 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.847+0000 7fa0436bf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c1a3d20 con 0x7fa03c198d10 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.847+0000 7fa040c5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa03c105180 0x7fa03c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa040c5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa03c105180 0x7fa03c1987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58378/0 (socket says 172.21.15.12:58378) 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa04145b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03c100060 0x7fa03c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa040c5a700 1 -- 172.21.15.12:0/1353866314 learned_addr learned my addr 172.21.15.12:0/1353866314 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa041c5c700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa03c198d10 0x7fa03c1a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa04145b700 1 -- 172.21.15.12:0/1353866314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa03c198d10 msgr2=0x7fa03c1a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa04145b700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa03c198d10 0x7fa03c1a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa04145b700 1 -- 172.21.15.12:0/1353866314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa03c105180 msgr2=0x7fa03c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa04145b700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa03c105180 0x7fa03c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa04145b700 1 -- 172.21.15.12:0/1353866314 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0380095f0 con 0x7fa03c100060 2024-05-09T00:03:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa040c5a700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa03c105180 0x7fa03c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa04145b700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03c100060 0x7fa03c198290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa038000c00 tx=0x7fa038004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa0327fc700 1 -- 172.21.15.12:0/1353866314 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03801d070 con 0x7fa03c100060 2024-05-09T00:03:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa0327fc700 1 -- 172.21.15.12:0/1353866314 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa038021410 con 0x7fa03c100060 2024-05-09T00:03:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa0436bf700 1 -- 172.21.15.12:0/1353866314 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03c196080 con 0x7fa03c100060 2024-05-09T00:03:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.848+0000 7fa0327fc700 1 -- 172.21.15.12:0/1353866314 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa038017650 con 0x7fa03c100060 2024-05-09T00:03:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.849+0000 7fa0436bf700 1 -- 172.21.15.12:0/1353866314 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03c196580 con 0x7fa03c100060 2024-05-09T00:03:33.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.852+0000 7fa0327fc700 1 -- 172.21.15.12:0/1353866314 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa038021580 con 0x7fa03c100060 2024-05-09T00:03:33.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa0327fc700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa0280a1310 0x7fa0280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa04f879b80 1 Processor -- start 2024-05-09T00:03:33.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa040c5a700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa0280a1310 0x7fa0280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa04f879b80 1 -- start start 2024-05-09T00:03:33.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa0327fc700 1 -- 172.21.15.12:0/1353866314 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa0380cde30 con 0x7fa03c100060 2024-05-09T00:03:33.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb0140b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140bf30 0x55eb01405b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 0x55eb014064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa04f879b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb01354150 con 0x55eb0140b610 2024-05-09T00:03:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa04f879b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb01351b10 con 0x55eb014060a0 2024-05-09T00:03:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.853+0000 7fa04f879b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb0134d400 con 0x55eb0140bf30 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa04145b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb0140b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa04145b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb0140b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58384/0 (socket says 172.21.15.12:58384) 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa04145b700 1 -- 172.21.15.12:0/3772434908 learned_addr learned my addr 172.21.15.12:0/3772434908 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa040c5a700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa0280a1310 0x7fa0280a37d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa03c069c60 tx=0x7fa02c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa041c5c700 1 --2- 172.21.15.12:0/3772434908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 0x55eb014064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa040c5a700 1 --2- 172.21.15.12:0/3772434908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140bf30 0x55eb01405b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa04145b700 1 -- 172.21.15.12:0/3772434908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140bf30 msgr2=0x55eb01405b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa04145b700 1 --2- 172.21.15.12:0/3772434908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140bf30 0x55eb01405b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa04145b700 1 -- 172.21.15.12:0/3772434908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 msgr2=0x55eb014064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa04145b700 1 --2- 172.21.15.12:0/3772434908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 0x55eb014064e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa04145b700 1 -- 172.21.15.12:0/3772434908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb011ce3b0 con 0x55eb0140b610 2024-05-09T00:03:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa041c5c700 1 --2- 172.21.15.12:0/3772434908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 0x55eb014064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:03:33.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa04145b700 1 --2- 172.21.15.12:0/3772434908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb0140b9f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa0380974e0 tx=0x7fa03809adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e850cd1e8055a6c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.854+0000 7fa0277fe700 1 -- 172.21.15.12:0/3772434908 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03809b6e0 con 0x55eb0140b610 2024-05-09T00:03:33.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.855+0000 7fa0277fe700 1 -- 172.21.15.12:0/3772434908 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0380a10c0 con 0x55eb0140b610 2024-05-09T00:03:33.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.855+0000 7fa0277fe700 1 -- 172.21.15.12:0/3772434908 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03809b240 con 0x55eb0140b610 2024-05-09T00:03:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.856+0000 7fa04f879b80 1 -- 172.21.15.12:0/3772434908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 msgr2=0x55eb0140b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.856+0000 7fa04f879b80 1 --2- 172.21.15.12:0/3772434908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb0140b9f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa0380974e0 tx=0x7fa03809adb0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.856+0000 7fa04f879b80 1 -- 172.21.15.12:0/3772434908 shutdown_connections 2024-05-09T00:03:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.856+0000 7fa04f879b80 1 --2- 172.21.15.12:0/3772434908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 0x55eb014064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.856+0000 7fa04f879b80 1 --2- 172.21.15.12:0/3772434908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb0140b9f0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.856+0000 7fa04f879b80 1 --2- 172.21.15.12:0/3772434908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140bf30 0x55eb01405b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.856+0000 7fa04f879b80 1 -- 172.21.15.12:0/3772434908 >> 172.21.15.12:0/3772434908 conn(0x55eb0135b210 msgr2=0x55eb013f7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.856+0000 7fa04f879b80 1 -- 172.21.15.12:0/3772434908 shutdown_connections 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.856+0000 7fa04f879b80 1 -- 172.21.15.12:0/3772434908 wait complete. 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.856+0000 7fa04f879b80 1 -- start start 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.857+0000 7fa04f879b80 1 Processor -- start 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.857+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb014060a0 0x55eb01411ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.857+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140b610 0x55eb0140f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.857+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140bf30 0x55eb0140f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.857+0000 7fa04f879b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb012e9560 con 0x55eb0140bf30 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.857+0000 7fa04f879b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb012e9200 con 0x55eb0140b610 2024-05-09T00:03:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.857+0000 7fa04f879b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb011d06a0 con 0x55eb014060a0 2024-05-09T00:03:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.857+0000 7fa04f879b80 10 client.0 _ll_register_callbacks cb 0x55eb012f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.857+0000 7fa04145b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb014060a0 0x55eb01411ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa04145b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb014060a0 0x55eb01411ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43556/0 (socket says 172.21.15.12:43556) 2024-05-09T00:03:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa04145b700 1 -- 172.21.15.12:0/3977815243 learned_addr learned my addr 172.21.15.12:0/3977815243 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa040c5a700 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140b610 0x55eb0140f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa041c5c700 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140bf30 0x55eb0140f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa04145b700 1 -- 172.21.15.12:0/3977815243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140b610 msgr2=0x55eb0140f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa04145b700 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140b610 0x55eb0140f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa04145b700 1 -- 172.21.15.12:0/3977815243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140bf30 msgr2=0x55eb0140f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa04145b700 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140bf30 0x55eb0140f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa04145b700 1 -- 172.21.15.12:0/3977815243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03c196580 con 0x55eb014060a0 2024-05-09T00:03:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa040c5a700 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140b610 0x55eb0140f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:03:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa041c5c700 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140bf30 0x55eb0140f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:03:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa04145b700 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb014060a0 0x55eb01411ec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa038097490 tx=0x7fa03809a7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8709dcf715b558 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa024ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:03:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa024ff9700 1 -- 172.21.15.12:0/3977815243 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03809b6e0 con 0x55eb014060a0 2024-05-09T00:03:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa024ff9700 1 -- 172.21.15.12:0/3977815243 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa03809afe0 con 0x55eb014060a0 2024-05-09T00:03:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.858+0000 7fa04f879b80 1 -- 172.21.15.12:0/3977815243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb012a6270 con 0x55eb014060a0 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa024ff9700 1 -- 172.21.15.12:0/3977815243 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa03809ee10 con 0x55eb014060a0 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa04f879b80 10 client.29968 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa04f879b80 1 -- 172.21.15.12:0/3977815243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55eb0117ab60 con 0x55eb014060a0 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa024ff9700 1 -- 172.21.15.12:0/3977815243 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eb012a6270 con 0x55eb014060a0 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa024ff9700 1 -- 172.21.15.12:0/3977815243 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa0380a3090 con 0x55eb014060a0 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa04f879b80 10 client.29968 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa04f879b80 10 client.29968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa04f879b80 20 client.29968 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa04f879b80 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55eb0141f190 0x55eb014215b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa04f879b80 4 client.29968 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.859+0000 7fa04f879b80 1 -- 172.21.15.12:0/3977815243 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb013cc810 con 0x55eb0141f190 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.860+0000 7fa040c5a700 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55eb0141f190 0x55eb014215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.860+0000 7fa024ff9700 10 client.29968 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.860+0000 7fa040c5a700 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55eb0141f190 0x55eb014215b0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5139f9f7a5ad41f5 server_cookie=dfcd1737242d9299 in_seq=0 out_seq=0 2024-05-09T00:03:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa024ff9700 1 -- 172.21.15.12:0/3977815243 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb013cc810 con 0x55eb0141f190 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa024ff9700 10 client.29968 handle_command_reply: tid=0 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa04f879b80 1 client.29968 shutdown 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa04f879b80 1 -- 172.21.15.12:0/3977815243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb014060a0 msgr2=0x55eb01411ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa04f879b80 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb014060a0 0x55eb01411ec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa038097490 tx=0x7fa03809a7b0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa04f879b80 1 -- 172.21.15.12:0/3977815243 shutdown_connections 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa04f879b80 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140bf30 0x55eb0140f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa04f879b80 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb014060a0 0x55eb01411ec0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa04f879b80 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55eb0141f190 0x55eb014215b0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa04f879b80 1 --2- 172.21.15.12:0/3977815243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140b610 0x55eb0140f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa04f879b80 1 -- 172.21.15.12:0/3977815243 >> 172.21.15.12:0/3977815243 conn(0x55eb0135b210 msgr2=0x55eb013f75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.862+0000 7fa024ff9700 0 client.29968 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.863+0000 7fa04f879b80 1 -- 172.21.15.12:0/3977815243 shutdown_connections 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.863+0000 7fa04f879b80 1 -- 172.21.15.12:0/3977815243 wait complete. 2024-05-09T00:03:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.863+0000 7fa04f879b80 20 client.29968 trim_cache size 0 max 16384 2024-05-09T00:03:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa04f879b80 1 Processor -- start 2024-05-09T00:03:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa04f879b80 1 -- start start 2024-05-09T00:03:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 0x55eb013f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb014165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140bf30 0x55eb01416b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa04f879b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb0141c170 con 0x55eb0140b610 2024-05-09T00:03:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa04f879b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb013f4dc0 con 0x55eb014060a0 2024-05-09T00:03:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa04f879b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb013f9e50 con 0x55eb0140bf30 2024-05-09T00:03:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa040c5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb014165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa040c5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb014165f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58392/0 (socket says 172.21.15.12:58392) 2024-05-09T00:03:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa040c5a700 1 -- 172.21.15.12:0/2736939599 learned_addr learned my addr 172.21.15.12:0/2736939599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa041c5c700 1 --2- 172.21.15.12:0/2736939599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140bf30 0x55eb01416b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa04145b700 1 --2- 172.21.15.12:0/2736939599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 0x55eb013f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.879+0000 7fa04145b700 1 -- 172.21.15.12:0/2736939599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140bf30 msgr2=0x55eb01416b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.880+0000 7fa04145b700 1 --2- 172.21.15.12:0/2736939599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140bf30 0x55eb01416b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.880+0000 7fa04145b700 1 -- 172.21.15.12:0/2736939599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 msgr2=0x55eb014165f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.880+0000 7fa04145b700 1 --2- 172.21.15.12:0/2736939599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb014165f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.880+0000 7fa04145b700 1 -- 172.21.15.12:0/2736939599 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb01313560 con 0x55eb014060a0 2024-05-09T00:03:33.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.880+0000 7fa040c5a700 1 --2- 172.21.15.12:0/2736939599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb014165f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.880+0000 7fa04145b700 1 --2- 172.21.15.12:0/2736939599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 0x55eb013f7d20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa038005b40 tx=0x7fa0380a1050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3103b2810f6a51dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.880+0000 7fa027fff700 1 -- 172.21.15.12:0/2736939599 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03809b6e0 con 0x55eb014060a0 2024-05-09T00:03:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.880+0000 7fa027fff700 1 -- 172.21.15.12:0/2736939599 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0380a33d0 con 0x55eb014060a0 2024-05-09T00:03:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.880+0000 7fa027fff700 1 -- 172.21.15.12:0/2736939599 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0380a2f40 con 0x55eb014060a0 2024-05-09T00:03:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 -- 172.21.15.12:0/2736939599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 msgr2=0x55eb013f7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 --2- 172.21.15.12:0/2736939599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 0x55eb013f7d20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa038005b40 tx=0x7fa0380a1050 comp rx=0 tx=0).stop 2024-05-09T00:03:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 -- 172.21.15.12:0/2736939599 shutdown_connections 2024-05-09T00:03:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 --2- 172.21.15.12:0/2736939599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140bf30 0x55eb01416b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 --2- 172.21.15.12:0/2736939599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb014060a0 0x55eb013f7d20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 --2- 172.21.15.12:0/2736939599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb0140b610 0x55eb014165f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 -- 172.21.15.12:0/2736939599 >> 172.21.15.12:0/2736939599 conn(0x55eb0135b210 msgr2=0x55eb01409350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 -- 172.21.15.12:0/2736939599 shutdown_connections 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 -- 172.21.15.12:0/2736939599 wait complete. 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 -- start start 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.881+0000 7fa04f879b80 1 Processor -- start 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb014060a0 0x55eb01416ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140b610 0x55eb01408160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04f879b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140bf30 0x55eb014088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04f879b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb01407070 con 0x55eb014060a0 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04f879b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb014071e0 con 0x55eb0140bf30 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04f879b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb014073a0 con 0x55eb0140b610 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04f879b80 10 client.0 _ll_register_callbacks cb 0x55eb012f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:33.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04145b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb014060a0 0x55eb01416ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa040c5a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140b610 0x55eb01408160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04145b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb014060a0 0x55eb01416ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58396/0 (socket says 172.21.15.12:58396) 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04145b700 1 -- 172.21.15.12:0/2711757776 learned_addr learned my addr 172.21.15.12:0/2711757776 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa041c5c700 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140bf30 0x55eb014088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04145b700 1 -- 172.21.15.12:0/2711757776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140b610 msgr2=0x55eb01408160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04145b700 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140b610 0x55eb01408160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04145b700 1 -- 172.21.15.12:0/2711757776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140bf30 msgr2=0x55eb014088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04145b700 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140bf30 0x55eb014088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04145b700 1 -- 172.21.15.12:0/2711757776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02c017070 con 0x55eb014060a0 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa04145b700 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb014060a0 0x55eb01416ae0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa038005b40 tx=0x7fa03809adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ad988cdb3fab93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.882+0000 7fa024ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:03:33.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa024ff9700 1 -- 172.21.15.12:0/2711757776 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03809b6e0 con 0x55eb014060a0 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa024ff9700 1 -- 172.21.15.12:0/2711757776 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa03809afe0 con 0x55eb014060a0 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa04f879b80 1 -- 172.21.15.12:0/2711757776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0380a10c0 con 0x55eb014060a0 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa024ff9700 1 -- 172.21.15.12:0/2711757776 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa03800fb10 con 0x55eb014060a0 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa04f879b80 10 client.20235 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa04f879b80 1 -- 172.21.15.12:0/2711757776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55eb0117ab60 con 0x55eb014060a0 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa024ff9700 1 -- 172.21.15.12:0/2711757776 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0380a10c0 con 0x55eb014060a0 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa024ff9700 1 -- 172.21.15.12:0/2711757776 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa038084170 con 0x55eb014060a0 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa04f879b80 10 client.20235 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa04f879b80 10 client.20235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa04f879b80 20 client.20235 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa04f879b80 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55eb01475b30 0x55eb01477f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa04f879b80 4 client.20235 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa04f879b80 1 -- 172.21.15.12:0/2711757776 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eb01345ec0 con 0x55eb01475b30 2024-05-09T00:03:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.883+0000 7fa040c5a700 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55eb01475b30 0x55eb01477f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.884+0000 7fa024ff9700 10 client.20235 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.884+0000 7fa040c5a700 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55eb01475b30 0x55eb01477f20 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc6e9580ce7bace3 server_cookie=bd926de9ba955349 in_seq=0 out_seq=0 2024-05-09T00:03:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa024ff9700 1 -- 172.21.15.12:0/2711757776 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eb01345ec0 con 0x55eb01475b30 2024-05-09T00:03:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa024ff9700 10 client.20235 handle_command_reply: tid=0 2024-05-09T00:03:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa04f879b80 1 client.20235 shutdown 2024-05-09T00:03:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa04f879b80 1 -- 172.21.15.12:0/2711757776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb014060a0 msgr2=0x55eb01416ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa04f879b80 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb014060a0 0x55eb01416ae0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa038005b40 tx=0x7fa03809adb0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa04f879b80 1 -- 172.21.15.12:0/2711757776 shutdown_connections 2024-05-09T00:03:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa04f879b80 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55eb01475b30 0x55eb01477f20 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa04f879b80 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb0140bf30 0x55eb014088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa04f879b80 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb014060a0 0x55eb01416ae0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa04f879b80 1 --2- 172.21.15.12:0/2711757776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb0140b610 0x55eb01408160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa024ff9700 0 client.20235 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.888+0000 7fa04f879b80 1 -- 172.21.15.12:0/2711757776 >> 172.21.15.12:0/2711757776 conn(0x55eb0135b210 msgr2=0x55eb0140da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.889+0000 7fa04f879b80 1 -- 172.21.15.12:0/2711757776 shutdown_connections 2024-05-09T00:03:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.889+0000 7fa04f879b80 1 -- 172.21.15.12:0/2711757776 wait complete. 2024-05-09T00:03:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.889+0000 7fa04f879b80 20 client.20235 trim_cache size 0 max 16384 2024-05-09T00:03:33.886 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:03:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.890+0000 7fa0436bf700 1 -- 172.21.15.12:0/1353866314 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa0280a1310 msgr2=0x7fa0280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.890+0000 7fa0436bf700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa0280a1310 0x7fa0280a37d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa03c069c60 tx=0x7fa02c009260 comp rx=0 tx=0).stop 2024-05-09T00:03:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.890+0000 7fa0436bf700 1 -- 172.21.15.12:0/1353866314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03c100060 msgr2=0x7fa03c198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.890+0000 7fa0436bf700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03c100060 0x7fa03c198290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa038000c00 tx=0x7fa038004730 comp rx=0 tx=0).stop 2024-05-09T00:03:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.890+0000 7fa0436bf700 1 -- 172.21.15.12:0/1353866314 shutdown_connections 2024-05-09T00:03:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.890+0000 7fa0436bf700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa0280a1310 0x7fa0280a37d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.890+0000 7fa0436bf700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa03c198d10 0x7fa03c1a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.890+0000 7fa0436bf700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03c100060 0x7fa03c198290 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa038000c00 tx=0x7fa038004730 comp rx=0 tx=0).stop 2024-05-09T00:03:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.890+0000 7fa0436bf700 1 --2- 172.21.15.12:0/1353866314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa03c105180 0x7fa03c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.890+0000 7fa0436bf700 1 -- 172.21.15.12:0/1353866314 >> 172.21.15.12:0/1353866314 conn(0x7fa03c076f70 msgr2=0x7fa03c106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.892+0000 7fa0436bf700 1 -- 172.21.15.12:0/1353866314 shutdown_connections 2024-05-09T00:03:33.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:33.892+0000 7fa0436bf700 1 -- 172.21.15.12:0/1353866314 wait complete. 2024-05-09T00:03:34.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 151118 38.89 MB/sec execute 89 sec latency 36.447 ms 2024-05-09T00:03:34.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:34 smithi204 ceph-mon[47373]: pgmap v552: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 1.14k op/s 2024-05-09T00:03:34.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:34 smithi160 ceph-mon[46152]: pgmap v552: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 1.14k op/s 2024-05-09T00:03:34.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:34 smithi012 ceph-mon[42503]: pgmap v552: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 1.14k op/s 2024-05-09T00:03:34.902 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:03:34.902 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:03:35.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.066+0000 7f0d98b08700 1 -- 172.21.15.12:0/1328611611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d941062a0 msgr2=0x7f0d9410de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.066+0000 7f0d98b08700 1 --2- 172.21.15.12:0/1328611611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d941062a0 0x7f0d9410de20 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00b0d0 tx=0x7f0d8c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.066+0000 7f0d98b08700 1 -- 172.21.15.12:0/1328611611 shutdown_connections 2024-05-09T00:03:35.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.066+0000 7f0d98b08700 1 --2- 172.21.15.12:0/1328611611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d941062a0 0x7f0d9410de20 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.066+0000 7f0d98b08700 1 --2- 172.21.15.12:0/1328611611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d940697a0 0x7f0d94069b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.066+0000 7f0d98b08700 1 --2- 172.21.15.12:0/1328611611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d9406a150 0x7f0d94105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.066+0000 7f0d98b08700 1 -- 172.21.15.12:0/1328611611 >> 172.21.15.12:0/1328611611 conn(0x7f0d94076fa0 msgr2=0x7f0d940773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.066+0000 7f0d98b08700 1 -- 172.21.15.12:0/1328611611 shutdown_connections 2024-05-09T00:03:35.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.066+0000 7f0d98b08700 1 -- 172.21.15.12:0/1328611611 wait complete. 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.067+0000 7f0d98b08700 1 Processor -- start 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.067+0000 7f0d98b08700 1 -- start start 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d98b08700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d940697a0 0x7f0d94197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d98b08700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d941062a0 0x7f0d94197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d98b08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d9419bdf0 0x7f0d9419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d98b08700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9419c870 con 0x7f0d9419bdf0 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d98b08700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9419c9e0 con 0x7f0d940697a0 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d98b08700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9419cba0 con 0x7f0d941062a0 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d9259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d940697a0 0x7f0d94197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d92d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d9419bdf0 0x7f0d9419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d91d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d941062a0 0x7f0d94197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d9259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d940697a0 0x7f0d94197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33604/0 (socket says 172.21.15.12:33604) 2024-05-09T00:03:35.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d92d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d9419bdf0 0x7f0d9419c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58410/0 (socket says 172.21.15.12:58410) 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d9259c700 1 -- 172.21.15.12:0/3846779907 learned_addr learned my addr 172.21.15.12:0/3846779907 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d92d9d700 1 -- 172.21.15.12:0/3846779907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d941062a0 msgr2=0x7f0d94197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d92d9d700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d941062a0 0x7f0d94197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d92d9d700 1 -- 172.21.15.12:0/3846779907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d940697a0 msgr2=0x7f0d94197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d92d9d700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d940697a0 0x7f0d94197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d92d9d700 1 -- 172.21.15.12:0/3846779907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c009d00 con 0x7f0d9419bdf0 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d9259c700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d940697a0 0x7f0d94197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.068+0000 7f0d92d9d700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d9419bdf0 0x7f0d9419c270 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00b770 tx=0x7f0d8c00bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.069+0000 7f0d837fe700 1 -- 172.21.15.12:0/3846779907 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00e040 con 0x7f0d9419bdf0 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.069+0000 7f0d837fe700 1 -- 172.21.15.12:0/3846779907 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8c005030 con 0x7f0d9419bdf0 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.069+0000 7f0d837fe700 1 -- 172.21.15.12:0/3846779907 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c01b6a0 con 0x7f0d9419bdf0 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.069+0000 7f0d98b08700 1 -- 172.21.15.12:0/3846779907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d941a40a0 con 0x7f0d9419bdf0 2024-05-09T00:03:35.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.069+0000 7f0d98b08700 1 -- 172.21.15.12:0/3846779907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d941a44d0 con 0x7f0d9419bdf0 2024-05-09T00:03:35.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.072+0000 7f0d837fe700 1 -- 172.21.15.12:0/3846779907 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0d8c0054a0 con 0x7f0d9419bdf0 2024-05-09T00:03:35.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.072+0000 7f0d837fe700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0d7c0a1310 0x7f0d7c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.072+0000 7f0d837fe700 1 -- 172.21.15.12:0/3846779907 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0d8c0cda60 con 0x7f0d9419bdf0 2024-05-09T00:03:35.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.072+0000 7f0d9259c700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0d7c0a1310 0x7f0d7c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.073+0000 7f0d9259c700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0d7c0a1310 0x7f0d7c0a37d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0d84011550 tx=0x7f0d84009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.073+0000 7f0da4cc2b80 1 Processor -- start 2024-05-09T00:03:35.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0da4cc2b80 1 -- start start 2024-05-09T00:03:35.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x563019856260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x5630198503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019850d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630197a71d0 con 0x563019850910 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630197a4b90 con 0x5630198567a0 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630197a0480 con 0x563019855e80 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0d92d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019850d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0d92d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019850d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58416/0 (socket says 172.21.15.12:58416) 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0d91d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x5630198503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0d9259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x563019856260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0d92d9d700 1 -- 172.21.15.12:0/893350128 learned_addr learned my addr 172.21.15.12:0/893350128 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0d92d9d700 1 -- 172.21.15.12:0/893350128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 msgr2=0x563019856260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0d92d9d700 1 --2- 172.21.15.12:0/893350128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x563019856260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0d92d9d700 1 -- 172.21.15.12:0/893350128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 msgr2=0x5630198503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.074+0000 7f0d92d9d700 1 --2- 172.21.15.12:0/893350128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x5630198503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.075+0000 7f0d92d9d700 1 -- 172.21.15.12:0/893350128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563019621430 con 0x563019850910 2024-05-09T00:03:35.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.075+0000 7f0d92d9d700 1 --2- 172.21.15.12:0/893350128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019850d50 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c097680 tx=0x7f0d8c096950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f39d2403d830de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.075+0000 7f0d80ff9700 1 -- 172.21.15.12:0/893350128 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c09a570 con 0x563019850910 2024-05-09T00:03:35.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.075+0000 7f0d80ff9700 1 -- 172.21.15.12:0/893350128 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8c09b310 con 0x563019850910 2024-05-09T00:03:35.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.075+0000 7f0d80ff9700 1 -- 172.21.15.12:0/893350128 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c09ae70 con 0x563019850910 2024-05-09T00:03:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.076+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/893350128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 msgr2=0x563019850d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.076+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/893350128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019850d50 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c097680 tx=0x7f0d8c096950 comp rx=0 tx=0).stop 2024-05-09T00:03:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.076+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/893350128 shutdown_connections 2024-05-09T00:03:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.076+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/893350128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019850d50 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.076+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/893350128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x563019856260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.076+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/893350128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x5630198503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.076+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/893350128 >> 172.21.15.12:0/893350128 conn(0x5630197ae290 msgr2=0x56301984ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.076+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/893350128 shutdown_connections 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.076+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/893350128 wait complete. 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.076+0000 7f0da4cc2b80 1 -- start start 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0da4cc2b80 1 Processor -- start 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019859bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x56301985a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x56301985a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301973c5e0 con 0x563019850910 2024-05-09T00:03:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301973c280 con 0x5630198567a0 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019623720 con 0x563019855e80 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0da4cc2b80 10 client.0 _ll_register_callbacks cb 0x563019745bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0d9259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019859bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0d9259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019859bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58426/0 (socket says 172.21.15.12:58426) 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0d9259c700 1 -- 172.21.15.12:0/1598502988 learned_addr learned my addr 172.21.15.12:0/1598502988 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.077+0000 7f0d91d9b700 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x56301985a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0d9259c700 1 -- 172.21.15.12:0/1598502988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 msgr2=0x56301985a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0d9259c700 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x56301985a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0d9259c700 1 -- 172.21.15.12:0/1598502988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 msgr2=0x56301985a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0d9259c700 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x56301985a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0d9259c700 1 -- 172.21.15.12:0/1598502988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d941a44d0 con 0x563019850910 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0d9259c700 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019859bf0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ed60 tx=0x7f0d840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8481f36d4d2377f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0d79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0d79ffb700 1 -- 172.21.15.12:0/1598502988 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84022070 con 0x563019850910 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0d79ffb700 1 -- 172.21.15.12:0/1598502988 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0d8401dbb0 con 0x563019850910 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/1598502988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630196f92f0 con 0x563019850910 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0d79ffb700 1 -- 172.21.15.12:0/1598502988 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d8401dd70 con 0x563019850910 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0da4cc2b80 10 client.20247 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.078+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/1598502988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5630195cdbe0 con 0x563019850910 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.079+0000 7f0d79ffb700 1 -- 172.21.15.12:0/1598502988 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5630196f92f0 con 0x563019850910 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.079+0000 7f0d79ffb700 1 -- 172.21.15.12:0/1598502988 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0d8402e070 con 0x563019850910 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.080+0000 7f0da4cc2b80 10 client.20247 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.080+0000 7f0da4cc2b80 10 client.20247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.080+0000 7f0da4cc2b80 20 client.20247 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.080+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56301986a050 0x56301986c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.080+0000 7f0da4cc2b80 4 client.20247 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:35.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.080+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/1598502988 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56301981f890 con 0x56301986a050 2024-05-09T00:03:35.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.081+0000 7f0d91d9b700 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56301986a050 0x56301986c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.081+0000 7f0d79ffb700 10 client.20247 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.081+0000 7f0d91d9b700 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56301986a050 0x56301986c470 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641afdde3fb7ab19 server_cookie=375af087895f99bf in_seq=0 out_seq=0 2024-05-09T00:03:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0d79ffb700 1 -- 172.21.15.12:0/1598502988 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56301981f890 con 0x56301986a050 2024-05-09T00:03:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0d79ffb700 10 client.20247 handle_command_reply: tid=0 2024-05-09T00:03:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 client.20247 shutdown 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/1598502988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 msgr2=0x563019859bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019859bf0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ed60 tx=0x7f0d840047c0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/1598502988 shutdown_connections 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56301986a050 0x56301986c470 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x56301985a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x563019859bf0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/1598502988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x56301985a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/1598502988 >> 172.21.15.12:0/1598502988 conn(0x5630197ae290 msgr2=0x56301984a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0d79ffb700 0 client.20247 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/1598502988 shutdown_connections 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/1598502988 wait complete. 2024-05-09T00:03:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.086+0000 7f0da4cc2b80 20 client.20247 trim_cache size 0 max 16384 2024-05-09T00:03:35.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.106+0000 7f0da4cc2b80 1 Processor -- start 2024-05-09T00:03:35.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.106+0000 7f0da4cc2b80 1 -- start start 2024-05-09T00:03:35.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x5630198553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x56301984ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019843190 con 0x563019850910 2024-05-09T00:03:35.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630198677d0 con 0x5630198567a0 2024-05-09T00:03:35.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019866bc0 con 0x563019855e80 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0d9259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0d9259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58430/0 (socket says 172.21.15.12:58430) 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0d9259c700 1 -- 172.21.15.12:0/4173384292 learned_addr learned my addr 172.21.15.12:0/4173384292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0d91d9b700 1 --2- 172.21.15.12:0/4173384292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x5630198553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0d92d9d700 1 --2- 172.21.15.12:0/4173384292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x56301984ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0d9259c700 1 -- 172.21.15.12:0/4173384292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 msgr2=0x5630198553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0d9259c700 1 --2- 172.21.15.12:0/4173384292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x5630198553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0d9259c700 1 -- 172.21.15.12:0/4173384292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 msgr2=0x56301984ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0d9259c700 1 --2- 172.21.15.12:0/4173384292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x56301984ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.107+0000 7f0d9259c700 1 -- 172.21.15.12:0/4173384292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630197665e0 con 0x563019850910 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.108+0000 7f0d9259c700 1 --2- 172.21.15.12:0/4173384292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198503d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ed30 tx=0x7f0d84009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7a82b1e04e41bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.108+0000 7f0d817fa700 1 -- 172.21.15.12:0/4173384292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84022030 con 0x563019850910 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.108+0000 7f0d817fa700 1 -- 172.21.15.12:0/4173384292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8401a390 con 0x563019850910 2024-05-09T00:03:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.108+0000 7f0d817fa700 1 -- 172.21.15.12:0/4173384292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8401abf0 con 0x563019850910 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/4173384292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 msgr2=0x5630198503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/4173384292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198503d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ed30 tx=0x7f0d84009350 comp rx=0 tx=0).stop 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/4173384292 shutdown_connections 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/4173384292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x56301984ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/4173384292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198503d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/4173384292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x5630198553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/4173384292 >> 172.21.15.12:0/4173384292 conn(0x5630197ae290 msgr2=0x563019854ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/4173384292 shutdown_connections 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/4173384292 wait complete. 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 -- start start 2024-05-09T00:03:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.109+0000 7f0da4cc2b80 1 Processor -- start 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x563019860710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0da4cc2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x5630198c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019860e20 con 0x563019850910 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630198c0ac0 con 0x5630198567a0 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0da4cc2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630198c0c30 con 0x563019855e80 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0da4cc2b80 10 client.0 _ll_register_callbacks cb 0x563019745bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d9259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d9259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58438/0 (socket says 172.21.15.12:58438) 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d9259c700 1 -- 172.21.15.12:0/2815594547 learned_addr learned my addr 172.21.15.12:0/2815594547 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d91d9b700 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x563019860710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d9259c700 1 -- 172.21.15.12:0/2815594547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 msgr2=0x563019860710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d92d9d700 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x5630198c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d9259c700 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x563019860710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d9259c700 1 -- 172.21.15.12:0/2815594547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 msgr2=0x5630198c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d9259c700 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x5630198c0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d9259c700 1 -- 172.21.15.12:0/2815594547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630195cdbe0 con 0x563019850910 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d91d9b700 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x563019860710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.110+0000 7f0d92d9d700 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x5630198c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.111+0000 7f0d9259c700 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198601d0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f0d84016040 tx=0x7f0d84026640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6004e7af43fb3052 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.111+0000 7f0d79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.111+0000 7f0d79ffb700 1 -- 172.21.15.12:0/2815594547 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84022040 con 0x563019850910 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.111+0000 7f0d79ffb700 1 -- 172.21.15.12:0/2815594547 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0d84023070 con 0x563019850910 2024-05-09T00:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.111+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/2815594547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d8c09b310 con 0x563019850910 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.111+0000 7f0d79ffb700 1 -- 172.21.15.12:0/2815594547 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d840269c0 con 0x563019850910 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.111+0000 7f0da4cc2b80 10 client.20253 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.111+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/2815594547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x563019625d50 con 0x563019850910 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.111+0000 7f0d79ffb700 1 -- 172.21.15.12:0/2815594547 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8c09b310 con 0x563019850910 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.112+0000 7f0d79ffb700 1 -- 172.21.15.12:0/2815594547 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0d8401d410 con 0x563019850910 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.112+0000 7f0da4cc2b80 10 client.20253 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.112+0000 7f0da4cc2b80 10 client.20253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.112+0000 7f0da4cc2b80 20 client.20253 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.112+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5630198c2080 0x5630198c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.112+0000 7f0da4cc2b80 4 client.20253 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.112+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/2815594547 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563019798f40 con 0x5630198c2080 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.112+0000 7f0d91d9b700 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5630198c2080 0x5630198c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.112+0000 7f0d91d9b700 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5630198c2080 0x5630198c4470 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c67c067ed888e2 server_cookie=5118bcdc99ccc705 in_seq=0 out_seq=0 2024-05-09T00:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.112+0000 7f0d79ffb700 10 client.20253 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.117+0000 7f0d79ffb700 1 -- 172.21.15.12:0/2815594547 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563019798f40 con 0x5630198c2080 2024-05-09T00:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.117+0000 7f0d79ffb700 10 client.20253 handle_command_reply: tid=0 2024-05-09T00:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.117+0000 7f0da4cc2b80 1 client.20253 shutdown 2024-05-09T00:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.117+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/2815594547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 msgr2=0x5630198601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.117+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198601d0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f0d84016040 tx=0x7f0d84026640 comp rx=0 tx=0).stop 2024-05-09T00:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/2815594547 shutdown_connections 2024-05-09T00:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5630198c2080 0x5630198c4470 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630198567a0 0x5630198c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563019850910 0x5630198601d0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0d79ffb700 0 client.20253 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0da4cc2b80 1 --2- 172.21.15.12:0/2815594547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563019855e80 0x563019860710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/2815594547 >> 172.21.15.12:0/2815594547 conn(0x5630197ae290 msgr2=0x563019856c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/2815594547 shutdown_connections 2024-05-09T00:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0da4cc2b80 1 -- 172.21.15.12:0/2815594547 wait complete. 2024-05-09T00:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0da4cc2b80 20 client.20253 trim_cache size 0 max 16384 2024-05-09T00:03:35.116 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:03:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:03:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "d1fffd96-f6a9-4e9d-923b-169802f536e1", 2024-05-09T00:03:35.117 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:03:35.117 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:03:35.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0d98b08700 1 -- 172.21.15.12:0/3846779907 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0d7c0a1310 msgr2=0x7f0d7c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0d98b08700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0d7c0a1310 0x7f0d7c0a37d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0d84011550 tx=0x7f0d84009260 comp rx=0 tx=0).stop 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.118+0000 7f0d98b08700 1 -- 172.21.15.12:0/3846779907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d9419bdf0 msgr2=0x7f0d9419c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.119+0000 7f0d98b08700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d9419bdf0 0x7f0d9419c270 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00b770 tx=0x7f0d8c00bcd0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.119+0000 7f0d98b08700 1 -- 172.21.15.12:0/3846779907 shutdown_connections 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.119+0000 7f0d98b08700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0d7c0a1310 0x7f0d7c0a37d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.119+0000 7f0d98b08700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d9419bdf0 0x7f0d9419c270 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.119+0000 7f0d98b08700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d940697a0 0x7f0d94197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.119+0000 7f0d98b08700 1 --2- 172.21.15.12:0/3846779907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d941062a0 0x7f0d94197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.119+0000 7f0d98b08700 1 -- 172.21.15.12:0/3846779907 >> 172.21.15.12:0/3846779907 conn(0x7f0d94076fa0 msgr2=0x7f0d940fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.119+0000 7f0d98b08700 1 -- 172.21.15.12:0/3846779907 shutdown_connections 2024-05-09T00:03:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.119+0000 7f0d98b08700 1 -- 172.21.15.12:0/3846779907 wait complete. 2024-05-09T00:03:35.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:03:35.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 151819 38.92 MB/sec execute 90 sec latency 52.356 ms 2024-05-09T00:03:35.295 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:35 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:03:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.307+0000 7f95495cd700 1 -- 172.21.15.12:0/774392348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544107f20 msgr2=0x7f9544114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.307+0000 7f95495cd700 1 --2- 172.21.15.12:0/774392348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544107f20 0x7f9544114430 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f953c00b0a0 tx=0x7f953c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.307+0000 7f95495cd700 1 -- 172.21.15.12:0/774392348 shutdown_connections 2024-05-09T00:03:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.307+0000 7f95495cd700 1 --2- 172.21.15.12:0/774392348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544107f20 0x7f9544114430 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.307+0000 7f95495cd700 1 --2- 172.21.15.12:0/774392348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954410c580 0x7f954410c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.307+0000 7f95495cd700 1 --2- 172.21.15.12:0/774392348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9544107560 0x7f95441079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.307+0000 7f95495cd700 1 -- 172.21.15.12:0/774392348 >> 172.21.15.12:0/774392348 conn(0x7f9544076d90 msgr2=0x7f9544105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.307+0000 7f95495cd700 1 -- 172.21.15.12:0/774392348 shutdown_connections 2024-05-09T00:03:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.307+0000 7f95495cd700 1 -- 172.21.15.12:0/774392348 wait complete. 2024-05-09T00:03:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.308+0000 7f95495cd700 1 Processor -- start 2024-05-09T00:03:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.308+0000 7f95495cd700 1 -- start start 2024-05-09T00:03:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.308+0000 7f95495cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544107560 0x7f954419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.308+0000 7f95495cd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9544107f20 0x7f954419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.308+0000 7f95495cd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954410c580 0x7f95441a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.308+0000 7f95495cd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95441a2940 con 0x7f9544107560 2024-05-09T00:03:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.308+0000 7f95495cd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95441a2ab0 con 0x7f9544107f20 2024-05-09T00:03:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.308+0000 7f95495cd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95441a2c70 con 0x7f954410c580 2024-05-09T00:03:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544107560 0x7f954419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544107560 0x7f954419dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58458/0 (socket says 172.21.15.12:58458) 2024-05-09T00:03:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f9542ffd700 1 -- 172.21.15.12:0/176687154 learned_addr learned my addr 172.21.15.12:0/176687154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f95437fe700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954410c580 0x7f95441a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f95427fc700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9544107f20 0x7f954419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f9542ffd700 1 -- 172.21.15.12:0/176687154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954410c580 msgr2=0x7f95441a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f9542ffd700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954410c580 0x7f95441a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f9542ffd700 1 -- 172.21.15.12:0/176687154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9544107f20 msgr2=0x7f954419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f9542ffd700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9544107f20 0x7f954419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f9542ffd700 1 -- 172.21.15.12:0/176687154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c009d00 con 0x7f9544107560 2024-05-09T00:03:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f9542ffd700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544107560 0x7f954419dab0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f953400e970 tx=0x7f953400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f952bfff700 1 -- 172.21.15.12:0/176687154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534009790 con 0x7f9544107560 2024-05-09T00:03:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f952bfff700 1 -- 172.21.15.12:0/176687154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f953400eec0 con 0x7f9544107560 2024-05-09T00:03:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f95495cd700 1 -- 172.21.15.12:0/176687154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95441a2f50 con 0x7f9544107560 2024-05-09T00:03:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f952bfff700 1 -- 172.21.15.12:0/176687154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534003cf0 con 0x7f9544107560 2024-05-09T00:03:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.309+0000 7f95495cd700 1 -- 172.21.15.12:0/176687154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95441aa800 con 0x7f9544107560 2024-05-09T00:03:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.312+0000 7f952bfff700 1 -- 172.21.15.12:0/176687154 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f953400ccd0 con 0x7f9544107560 2024-05-09T00:03:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.313+0000 7f952bfff700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f952c0a1290 0x7f952c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.313+0000 7f952bfff700 1 -- 172.21.15.12:0/176687154 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9534014070 con 0x7f9544107560 2024-05-09T00:03:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.314+0000 7f95427fc700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f952c0a1290 0x7f952c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.314+0000 7f9555787b80 1 Processor -- start 2024-05-09T00:03:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.314+0000 7f9555787b80 1 -- start start 2024-05-09T00:03:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.314+0000 7f9555787b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83abe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.314+0000 7f9555787b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 0x562e83ab8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.314+0000 7f9555787b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83ab8890 0x562e83ab8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.314+0000 7f9555787b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e83a0f150 con 0x562e83abe720 2024-05-09T00:03:35.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9555787b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e83a0cb10 con 0x562e83ab8890 2024-05-09T00:03:35.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9555787b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e83a08400 con 0x562e83abde00 2024-05-09T00:03:35.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f95427fc700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f952c0a1290 0x7f952c0a3750 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9538005950 tx=0x7f95380058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83abe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83abe1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43662/0 (socket says 172.21.15.12:43662) 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9542ffd700 1 -- 172.21.15.12:0/2585038186 learned_addr learned my addr 172.21.15.12:0/2585038186 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f95427fc700 1 --2- 172.21.15.12:0/2585038186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 0x562e83ab8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f95437fe700 1 --2- 172.21.15.12:0/2585038186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83ab8890 0x562e83ab8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9542ffd700 1 -- 172.21.15.12:0/2585038186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83ab8890 msgr2=0x562e83ab8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9542ffd700 1 --2- 172.21.15.12:0/2585038186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83ab8890 0x562e83ab8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9542ffd700 1 -- 172.21.15.12:0/2585038186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 msgr2=0x562e83ab8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9542ffd700 1 --2- 172.21.15.12:0/2585038186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 0x562e83ab8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9542ffd700 1 -- 172.21.15.12:0/2585038186 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e838893b0 con 0x562e83abde00 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f95437fe700 1 --2- 172.21.15.12:0/2585038186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83ab8890 0x562e83ab8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.315+0000 7f9542ffd700 1 --2- 172.21.15.12:0/2585038186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83abe1e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9534096580 tx=0x7f9534099f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679b50dded48acb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.316+0000 7f95297fa700 1 -- 172.21.15.12:0/2585038186 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953409a080 con 0x562e83abde00 2024-05-09T00:03:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.316+0000 7f95297fa700 1 -- 172.21.15.12:0/2585038186 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f953409deb0 con 0x562e83abde00 2024-05-09T00:03:35.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.316+0000 7f95297fa700 1 -- 172.21.15.12:0/2585038186 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953409a520 con 0x562e83abde00 2024-05-09T00:03:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.316+0000 7f9555787b80 1 -- 172.21.15.12:0/2585038186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 msgr2=0x562e83abe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.316+0000 7f9555787b80 1 --2- 172.21.15.12:0/2585038186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83abe1e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9534096580 tx=0x7f9534099f80 comp rx=0 tx=0).stop 2024-05-09T00:03:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.317+0000 7f9555787b80 1 -- 172.21.15.12:0/2585038186 shutdown_connections 2024-05-09T00:03:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.317+0000 7f9555787b80 1 --2- 172.21.15.12:0/2585038186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83ab8890 0x562e83ab8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.317+0000 7f9555787b80 1 --2- 172.21.15.12:0/2585038186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83abe1e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.317+0000 7f9555787b80 1 --2- 172.21.15.12:0/2585038186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 0x562e83ab8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.317+0000 7f9555787b80 1 -- 172.21.15.12:0/2585038186 >> 172.21.15.12:0/2585038186 conn(0x562e83a16210 msgr2=0x562e83ab2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.317+0000 7f9555787b80 1 -- 172.21.15.12:0/2585038186 shutdown_connections 2024-05-09T00:03:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.317+0000 7f9555787b80 1 -- 172.21.15.12:0/2585038186 wait complete. 2024-05-09T00:03:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.317+0000 7f9555787b80 1 -- start start 2024-05-09T00:03:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.317+0000 7f9555787b80 1 Processor -- start 2024-05-09T00:03:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f9555787b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83ab8890 0x562e83ac1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f9555787b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abde00 0x562e83ac2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f9555787b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 0x562e83ac2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f9555787b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e839a4560 con 0x562e83abe720 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f9555787b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e839a4200 con 0x562e83abde00 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f9555787b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8388b6a0 con 0x562e83ab8890 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f9555787b80 10 client.0 _ll_register_callbacks cb 0x562e839adb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f95437fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 0x562e83ac2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f95437fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 0x562e83ac2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58478/0 (socket says 172.21.15.12:58478) 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f95437fe700 1 -- 172.21.15.12:0/3952486006 learned_addr learned my addr 172.21.15.12:0/3952486006 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f95427fc700 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abde00 0x562e83ac2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f9542ffd700 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83ab8890 0x562e83ac1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f95437fe700 1 -- 172.21.15.12:0/3952486006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83ab8890 msgr2=0x562e83ac1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f95437fe700 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83ab8890 0x562e83ac1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f95437fe700 1 -- 172.21.15.12:0/3952486006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abde00 msgr2=0x562e83ac2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f95437fe700 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abde00 0x562e83ac2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f95437fe700 1 -- 172.21.15.12:0/3952486006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95441aa800 con 0x562e83abe720 2024-05-09T00:03:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.318+0000 7f95427fc700 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abde00 0x562e83ac2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.319+0000 7f95437fe700 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 0x562e83ac2550 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f953c007ed0 tx=0x7f953c007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b583035882980875 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.319+0000 7f9526ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:03:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.319+0000 7f9526ff5700 1 -- 172.21.15.12:0/3952486006 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00e040 con 0x562e83abe720 2024-05-09T00:03:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.319+0000 7f9526ff5700 1 -- 172.21.15.12:0/3952486006 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f953c003c00 con 0x562e83abe720 2024-05-09T00:03:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.319+0000 7f9555787b80 1 -- 172.21.15.12:0/3952486006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e83961270 con 0x562e83abe720 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9526ff5700 1 -- 172.21.15.12:0/3952486006 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f953c006000 con 0x562e83abe720 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9555787b80 10 client.20265 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9555787b80 1 -- 172.21.15.12:0/3952486006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x562e83835b60 con 0x562e83abe720 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9526ff5700 1 -- 172.21.15.12:0/3952486006 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e83961270 con 0x562e83abe720 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9526ff5700 1 -- 172.21.15.12:0/3952486006 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f953c022020 con 0x562e83abe720 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9555787b80 10 client.20265 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9555787b80 10 client.20265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9555787b80 20 client.20265 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9555787b80 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e83ad2210 0x562e83ad4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9555787b80 4 client.20265 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.320+0000 7f9555787b80 1 -- 172.21.15.12:0/3952486006 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e83a87810 con 0x562e83ad2210 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.321+0000 7f9542ffd700 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e83ad2210 0x562e83ad4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.321+0000 7f9542ffd700 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e83ad2210 0x562e83ad4630 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc1a6184ff8aa14b server_cookie=d320321e80f9d04 in_seq=0 out_seq=0 2024-05-09T00:03:35.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.321+0000 7f9526ff5700 10 client.20265 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.324+0000 7f9526ff5700 1 -- 172.21.15.12:0/3952486006 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e83a87810 con 0x562e83ad2210 2024-05-09T00:03:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.324+0000 7f9526ff5700 10 client.20265 handle_command_reply: tid=0 2024-05-09T00:03:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.325+0000 7f9555787b80 1 client.20265 shutdown 2024-05-09T00:03:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 1 -- 172.21.15.12:0/3952486006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 msgr2=0x562e83ac2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 0x562e83ac2550 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f953c007ed0 tx=0x7f953c007f00 comp rx=0 tx=0).stop 2024-05-09T00:03:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 1 -- 172.21.15.12:0/3952486006 shutdown_connections 2024-05-09T00:03:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83abe720 0x562e83ac2550 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83ab8890 0x562e83ac1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e83ad2210 0x562e83ad4630 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 1 --2- 172.21.15.12:0/3952486006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abde00 0x562e83ac2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 1 -- 172.21.15.12:0/3952486006 >> 172.21.15.12:0/3952486006 conn(0x562e83a16210 msgr2=0x562e83ab25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9526ff5700 0 client.20265 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 1 -- 172.21.15.12:0/3952486006 shutdown_connections 2024-05-09T00:03:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 1 -- 172.21.15.12:0/3952486006 wait complete. 2024-05-09T00:03:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.326+0000 7f9555787b80 20 client.20265 trim_cache size 0 max 16384 2024-05-09T00:03:35.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9555787b80 1 Processor -- start 2024-05-09T00:03:35.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9555787b80 1 -- start start 2024-05-09T00:03:35.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9555787b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ab2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9555787b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83ac2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9555787b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abe720 0x562e83ac2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9555787b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e83ab4a20 con 0x562e83ab8890 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9555787b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e83ab4e50 con 0x562e83abe720 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9555787b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e83ad9840 con 0x562e83abde00 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ab2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ab2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58482/0 (socket says 172.21.15.12:58482) 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f9542ffd700 1 -- 172.21.15.12:0/1649641253 learned_addr learned my addr 172.21.15.12:0/1649641253 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f95437fe700 1 --2- 172.21.15.12:0/1649641253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abe720 0x562e83ac2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.346+0000 7f95427fc700 1 --2- 172.21.15.12:0/1649641253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83ac2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.347+0000 7f9542ffd700 1 -- 172.21.15.12:0/1649641253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 msgr2=0x562e83ac2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.347+0000 7f9542ffd700 1 --2- 172.21.15.12:0/1649641253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83ac2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.347+0000 7f9542ffd700 1 -- 172.21.15.12:0/1649641253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abe720 msgr2=0x562e83ac2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.347+0000 7f9542ffd700 1 --2- 172.21.15.12:0/1649641253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abe720 0x562e83ac2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.347+0000 7f9542ffd700 1 -- 172.21.15.12:0/1649641253 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e839ce560 con 0x562e83ab8890 2024-05-09T00:03:35.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.347+0000 7f9542ffd700 1 --2- 172.21.15.12:0/1649641253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ab2d20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f953409a010 tx=0x7f953400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39986b3d24f036a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.347+0000 7f9529ffb700 1 -- 172.21.15.12:0/1649641253 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953409a080 con 0x562e83ab8890 2024-05-09T00:03:35.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.347+0000 7f9529ffb700 1 -- 172.21.15.12:0/1649641253 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9534098420 con 0x562e83ab8890 2024-05-09T00:03:35.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.347+0000 7f9529ffb700 1 -- 172.21.15.12:0/1649641253 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534099570 con 0x562e83ab8890 2024-05-09T00:03:35.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.348+0000 7f9555787b80 1 -- 172.21.15.12:0/1649641253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 msgr2=0x562e83ab2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.348+0000 7f9555787b80 1 --2- 172.21.15.12:0/1649641253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ab2d20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f953409a010 tx=0x7f953400cbd0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.348+0000 7f9555787b80 1 -- 172.21.15.12:0/1649641253 shutdown_connections 2024-05-09T00:03:35.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.348+0000 7f9555787b80 1 --2- 172.21.15.12:0/1649641253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abe720 0x562e83ac2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.348+0000 7f9555787b80 1 --2- 172.21.15.12:0/1649641253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ab2d20 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.348+0000 7f9555787b80 1 --2- 172.21.15.12:0/1649641253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83ac2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.348+0000 7f9555787b80 1 -- 172.21.15.12:0/1649641253 >> 172.21.15.12:0/1649641253 conn(0x562e83a16210 msgr2=0x562e83ac03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.348+0000 7f9555787b80 1 -- 172.21.15.12:0/1649641253 shutdown_connections 2024-05-09T00:03:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.348+0000 7f9555787b80 1 -- 172.21.15.12:0/1649641253 wait complete. 2024-05-09T00:03:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.348+0000 7f9555787b80 1 -- start start 2024-05-09T00:03:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9555787b80 1 Processor -- start 2024-05-09T00:03:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9555787b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ac7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9555787b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83ac7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9555787b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abe720 0x562e83b27710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9555787b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e83b27c50 con 0x562e83ab8890 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9555787b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e83b27d90 con 0x562e83abe720 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9555787b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e83b27ed0 con 0x562e83abde00 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9555787b80 10 client.0 _ll_register_callbacks cb 0x562e839adb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ac7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f95427fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83ac7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f95437fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abe720 0x562e83b27710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ac7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58488/0 (socket says 172.21.15.12:58488) 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f95427fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83ac7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43682/0 (socket says 172.21.15.12:43682) 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9542ffd700 1 -- 172.21.15.12:0/408120825 learned_addr learned my addr 172.21.15.12:0/408120825 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9542ffd700 1 -- 172.21.15.12:0/408120825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 msgr2=0x562e83ac7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9542ffd700 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83ac7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9542ffd700 1 -- 172.21.15.12:0/408120825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abe720 msgr2=0x562e83b27710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9542ffd700 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abe720 0x562e83b27710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.349+0000 7f9542ffd700 1 -- 172.21.15.12:0/408120825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95340b6040 con 0x562e83ab8890 2024-05-09T00:03:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.350+0000 7f9542ffd700 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ac7340 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9534096580 tx=0x7f95340985b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be4bcb8bcb1a727 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.350+0000 7f9526ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:03:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.350+0000 7f9526ff5700 1 -- 172.21.15.12:0/408120825 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953409a080 con 0x562e83ab8890 2024-05-09T00:03:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.350+0000 7f9526ff5700 1 -- 172.21.15.12:0/408120825 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f95340b82b0 con 0x562e83ab8890 2024-05-09T00:03:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.350+0000 7f9555787b80 1 -- 172.21.15.12:0/408120825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f953409deb0 con 0x562e83ab8890 2024-05-09T00:03:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.350+0000 7f9526ff5700 1 -- 172.21.15.12:0/408120825 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95340b8470 con 0x562e83ab8890 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.350+0000 7f9555787b80 10 client.20271 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.350+0000 7f9555787b80 1 -- 172.21.15.12:0/408120825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x562e8388dcd0 con 0x562e83ab8890 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.350+0000 7f9526ff5700 1 -- 172.21.15.12:0/408120825 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f953409deb0 con 0x562e83ab8890 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.350+0000 7f9526ff5700 1 -- 172.21.15.12:0/408120825 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f953407d070 con 0x562e83ab8890 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.351+0000 7f9555787b80 10 client.20271 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.351+0000 7f9555787b80 10 client.20271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.351+0000 7f9555787b80 20 client.20271 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.351+0000 7f9555787b80 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e83b29150 0x562e83b2b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.351+0000 7f9555787b80 4 client.20271 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.351+0000 7f9555787b80 1 -- 172.21.15.12:0/408120825 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562e83a00ec0 con 0x562e83b29150 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.351+0000 7f95427fc700 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e83b29150 0x562e83b2b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.351+0000 7f9526ff5700 10 client.20271 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.351+0000 7f95427fc700 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e83b29150 0x562e83b2b540 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed28e889fcd59b4b server_cookie=121c7ea1d10ca74 in_seq=0 out_seq=0 2024-05-09T00:03:35.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.353+0000 7f9526ff5700 1 -- 172.21.15.12:0/408120825 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x562e83a00ec0 con 0x562e83b29150 2024-05-09T00:03:35.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.353+0000 7f9526ff5700 10 client.20271 handle_command_reply: tid=0 2024-05-09T00:03:35.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.353+0000 7f9555787b80 1 client.20271 shutdown 2024-05-09T00:03:35.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.353+0000 7f9555787b80 1 -- 172.21.15.12:0/408120825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 msgr2=0x562e83ac7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.353+0000 7f9555787b80 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ac7340 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9534096580 tx=0x7f95340985b0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.353+0000 7f9555787b80 1 -- 172.21.15.12:0/408120825 shutdown_connections 2024-05-09T00:03:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.353+0000 7f9555787b80 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e83b29150 0x562e83b2b540 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f9555787b80 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e83abe720 0x562e83b27710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f9555787b80 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e83ab8890 0x562e83ac7340 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f9555787b80 1 --2- 172.21.15.12:0/408120825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e83abde00 0x562e83ac7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f9555787b80 1 -- 172.21.15.12:0/408120825 >> 172.21.15.12:0/408120825 conn(0x562e83a16210 msgr2=0x562e83abcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f9526ff5700 0 client.20271 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f9555787b80 1 -- 172.21.15.12:0/408120825 shutdown_connections 2024-05-09T00:03:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f9555787b80 1 -- 172.21.15.12:0/408120825 wait complete. 2024-05-09T00:03:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f9555787b80 20 client.20271 trim_cache size 0 max 16384 2024-05-09T00:03:35.353 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:03:35.354 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (281 inodes in the stack)", 2024-05-09T00:03:35.354 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:03:35.354 INFO:teuthology.orchestra.run.smithi012.stdout: "d1fffd96-f6a9-4e9d-923b-169802f536e1": { 2024-05-09T00:03:35.354 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:03:35.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "d1fffd96-f6a9-4e9d-923b-169802f536e1", 2024-05-09T00:03:35.354 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:03:35.354 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:03:35.354 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:03:35.354 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:03:35.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 -- 172.21.15.12:0/176687154 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f952c0a1290 msgr2=0x7f952c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f952c0a1290 0x7f952c0a3750 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9538005950 tx=0x7f95380058e0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 -- 172.21.15.12:0/176687154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544107560 msgr2=0x7f954419dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544107560 0x7f954419dab0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f953400e970 tx=0x7f953400ec80 comp rx=0 tx=0).stop 2024-05-09T00:03:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 -- 172.21.15.12:0/176687154 shutdown_connections 2024-05-09T00:03:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f952c0a1290 0x7f952c0a3750 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954410c580 0x7f95441a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544107560 0x7f954419dab0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 --2- 172.21.15.12:0/176687154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9544107f20 0x7f954419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 -- 172.21.15.12:0/176687154 >> 172.21.15.12:0/176687154 conn(0x7f9544076d90 msgr2=0x7f9544077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.354+0000 7f95495cd700 1 -- 172.21.15.12:0/176687154 shutdown_connections 2024-05-09T00:03:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:03:35.355+0000 7f95495cd700 1 -- 172.21.15.12:0/176687154 wait complete. 2024-05-09T00:03:35.364 INFO:tasks.cephfs.filesystem:scrub status for tag:d1fffd96-f6a9-4e9d-923b-169802f536e1 - {'path': '/', 'tag': 'd1fffd96-f6a9-4e9d-923b-169802f536e1', 'options': 'recursive,force'} 2024-05-09T00:03:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.487+0000 7fdb65f7f700 1 -- 172.21.15.12:0/1676261688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb60105e60 msgr2=0x7fdb60106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.487+0000 7fdb65f7f700 1 --2- 172.21.15.12:0/1676261688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb60105e60 0x7fdb60106240 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009990 tx=0x7fdb50009ca0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.487+0000 7fdb65f7f700 1 -- 172.21.15.12:0/1676261688 shutdown_connections 2024-05-09T00:03:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.487+0000 7fdb65f7f700 1 --2- 172.21.15.12:0/1676261688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb60100760 0x7fdb6010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.487+0000 7fdb65f7f700 1 --2- 172.21.15.12:0/1676261688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb60105e60 0x7fdb60106240 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.487+0000 7fdb65f7f700 1 --2- 172.21.15.12:0/1676261688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb600ffda0 0x7fdb60100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.487+0000 7fdb65f7f700 1 -- 172.21.15.12:0/1676261688 >> 172.21.15.12:0/1676261688 conn(0x7fdb60076f10 msgr2=0x7fdb60077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.487+0000 7fdb65f7f700 1 -- 172.21.15.12:0/1676261688 shutdown_connections 2024-05-09T00:03:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.487+0000 7fdb65f7f700 1 -- 172.21.15.12:0/1676261688 wait complete. 2024-05-09T00:03:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.488+0000 7fdb65f7f700 1 Processor -- start 2024-05-09T00:03:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.488+0000 7fdb65f7f700 1 -- start start 2024-05-09T00:03:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.488+0000 7fdb65f7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb600ffda0 0x7fdb60101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.488+0000 7fdb65f7f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb60100760 0x7fdb60101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.488+0000 7fdb65f7f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60105e60 0x7fdb601024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.488+0000 7fdb65f7f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601a5790 con 0x7fdb60100760 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.488+0000 7fdb65f7f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601a5900 con 0x7fdb600ffda0 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.488+0000 7fdb65f7f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601a5ac0 con 0x7fdb60105e60 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.488+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb600ffda0 0x7fdb60101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.488+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60105e60 0x7fdb601024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb600ffda0 0x7fdb60101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33710/0 (socket says 172.21.15.12:33710) 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60105e60 0x7fdb601024d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43690/0 (socket says 172.21.15.12:43690) 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5f7fe700 1 -- 172.21.15.12:0/3387900905 learned_addr learned my addr 172.21.15.12:0/3387900905 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5effd700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb60100760 0x7fdb60101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5ffff700 1 -- 172.21.15.12:0/3387900905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb600ffda0 msgr2=0x7fdb60101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5ffff700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb600ffda0 0x7fdb60101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5ffff700 1 -- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb60100760 msgr2=0x7fdb60101f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5ffff700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb60100760 0x7fdb60101f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5ffff700 1 -- 172.21.15.12:0/3387900905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb500095f0 con 0x7fdb60105e60 2024-05-09T00:03:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5f7fe700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb600ffda0 0x7fdb60101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5effd700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb60100760 0x7fdb60101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5ffff700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60105e60 0x7fdb601024d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800efc0 tx=0x7fdb5800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5cff9700 1 -- 172.21.15.12:0/3387900905 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58018070 con 0x7fdb60105e60 2024-05-09T00:03:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5cff9700 1 -- 172.21.15.12:0/3387900905 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb5800f040 con 0x7fdb60105e60 2024-05-09T00:03:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb5cff9700 1 -- 172.21.15.12:0/3387900905 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800cd40 con 0x7fdb60105e60 2024-05-09T00:03:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.489+0000 7fdb65f7f700 1 -- 172.21.15.12:0/3387900905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb601a5d40 con 0x7fdb60105e60 2024-05-09T00:03:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.490+0000 7fdb65f7f700 1 -- 172.21.15.12:0/3387900905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb601a6200 con 0x7fdb60105e60 2024-05-09T00:03:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.492+0000 7fdb5cff9700 1 -- 172.21.15.12:0/3387900905 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdb58003a00 con 0x7fdb60105e60 2024-05-09T00:03:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.493+0000 7fdb5cff9700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb480a1310 0x7fdb480a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.493+0000 7fdb5f7fe700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb480a1310 0x7fdb480a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.493+0000 7fdb5cff9700 1 -- 172.21.15.12:0/3387900905 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdb580cc840 con 0x7fdb60105e60 2024-05-09T00:03:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.494+0000 7fdb5f7fe700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb480a1310 0x7fdb480a37d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb50006010 tx=0x7fdb5000b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.494+0000 7fdb72139b80 1 Processor -- start 2024-05-09T00:03:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.494+0000 7fdb72139b80 1 -- start start 2024-05-09T00:03:35.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb29ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a0480 0x55b8cb2a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb72139b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb1f70f0 con 0x55b8cb2a6270 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb72139b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb1f4ab0 con 0x55b8cb2a5950 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb72139b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb1f03a0 con 0x55b8cb2a0480 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb29ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb29ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58514/0 (socket says 172.21.15.12:58514) 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a0480 0x55b8cb2a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5effd700 1 -- 172.21.15.12:0/3562021159 learned_addr learned my addr 172.21.15.12:0/3562021159 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5effd700 1 -- 172.21.15.12:0/3562021159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a0480 msgr2=0x55b8cb2a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5effd700 1 --2- 172.21.15.12:0/3562021159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a0480 0x55b8cb2a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5effd700 1 -- 172.21.15.12:0/3562021159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a5950 msgr2=0x55b8cb2a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5effd700 1 --2- 172.21.15.12:0/3562021159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5effd700 1 -- 172.21.15.12:0/3562021159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8cb071350 con 0x55b8cb2a6270 2024-05-09T00:03:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.495+0000 7fdb5ffff700 1 --2- 172.21.15.12:0/3562021159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a0480 0x55b8cb2a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.496+0000 7fdb5effd700 1 --2- 172.21.15.12:0/3562021159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb29ff40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fdb540115f0 tx=0x7fdb54011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b7f0f33f7b254 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.496+0000 7fdb45ffb700 1 -- 172.21.15.12:0/3562021159 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54004230 con 0x55b8cb2a6270 2024-05-09T00:03:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.496+0000 7fdb45ffb700 1 -- 172.21.15.12:0/3562021159 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb54004390 con 0x55b8cb2a6270 2024-05-09T00:03:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.496+0000 7fdb45ffb700 1 -- 172.21.15.12:0/3562021159 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54019830 con 0x55b8cb2a6270 2024-05-09T00:03:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.497+0000 7fdb72139b80 1 -- 172.21.15.12:0/3562021159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 msgr2=0x55b8cb29ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.497+0000 7fdb72139b80 1 --2- 172.21.15.12:0/3562021159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb29ff40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fdb540115f0 tx=0x7fdb54011900 comp rx=0 tx=0).stop 2024-05-09T00:03:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.497+0000 7fdb72139b80 1 -- 172.21.15.12:0/3562021159 shutdown_connections 2024-05-09T00:03:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.497+0000 7fdb72139b80 1 --2- 172.21.15.12:0/3562021159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a0480 0x55b8cb2a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.497+0000 7fdb72139b80 1 --2- 172.21.15.12:0/3562021159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.497+0000 7fdb72139b80 1 --2- 172.21.15.12:0/3562021159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb29ff40 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.497+0000 7fdb72139b80 1 -- 172.21.15.12:0/3562021159 >> 172.21.15.12:0/3562021159 conn(0x55b8cb1fe1b0 msgr2=0x55b8cb29a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.499+0000 7fdb72139b80 1 -- 172.21.15.12:0/3562021159 shutdown_connections 2024-05-09T00:03:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.499+0000 7fdb72139b80 1 -- 172.21.15.12:0/3562021159 wait complete. 2024-05-09T00:03:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.499+0000 7fdb72139b80 1 -- start start 2024-05-09T00:03:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.499+0000 7fdb72139b80 1 Processor -- start 2024-05-09T00:03:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a6270 0x55b8cb2a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb72139b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb18c500 con 0x55b8cb2a0480 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb72139b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb18c1a0 con 0x55b8cb2a5950 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb72139b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb073640 con 0x55b8cb2a6270 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb72139b80 10 client.0 _ll_register_callbacks cb 0x55b8cb195ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2ac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58518/0 (socket says 172.21.15.12:58518) 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb5f7fe700 1 -- 172.21.15.12:0/2148578947 learned_addr learned my addr 172.21.15.12:0/2148578947 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb5effd700 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb5ffff700 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a6270 0x55b8cb2a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb5f7fe700 1 -- 172.21.15.12:0/2148578947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a6270 msgr2=0x55b8cb2a9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb5f7fe700 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a6270 0x55b8cb2a9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb5f7fe700 1 -- 172.21.15.12:0/2148578947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a5950 msgr2=0x55b8cb2a9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb5f7fe700 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.500+0000 7fdb5f7fe700 1 -- 172.21.15.12:0/2148578947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb601a6200 con 0x55b8cb2a0480 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.501+0000 7fdb5f7fe700 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2ac1c0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000fb70 tx=0x7fdb5001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac1da86eedc9d1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.501+0000 7fdb3effd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.501+0000 7fdb3effd700 1 -- 172.21.15.12:0/2148578947 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5002b050 con 0x55b8cb2a0480 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.501+0000 7fdb3effd700 1 -- 172.21.15.12:0/2148578947 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdb500056a0 con 0x55b8cb2a0480 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.501+0000 7fdb72139b80 1 -- 172.21.15.12:0/2148578947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8cb149210 con 0x55b8cb2a0480 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.501+0000 7fdb3effd700 1 -- 172.21.15.12:0/2148578947 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb50026a40 con 0x55b8cb2a0480 2024-05-09T00:03:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.501+0000 7fdb72139b80 10 client.20283 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.501+0000 7fdb72139b80 1 -- 172.21.15.12:0/2148578947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7fdb54009d50 con 0x55b8cb2a0480 2024-05-09T00:03:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.502+0000 7fdb3effd700 1 -- 172.21.15.12:0/2148578947 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8cb149210 con 0x55b8cb2a0480 2024-05-09T00:03:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.502+0000 7fdb3effd700 1 -- 172.21.15.12:0/2148578947 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb50024a90 con 0x55b8cb2a0480 2024-05-09T00:03:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.502+0000 7fdb72139b80 10 client.20283 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.502+0000 7fdb72139b80 10 client.20283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.502+0000 7fdb72139b80 20 client.20283 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.502+0000 7fdb72139b80 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8cb2b9bf0 0x55b8cb2bc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.502+0000 7fdb72139b80 4 client.20283 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.502+0000 7fdb72139b80 1 -- 172.21.15.12:0/2148578947 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8cb26f7b0 con 0x55b8cb2b9bf0 2024-05-09T00:03:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.504+0000 7fdb5effd700 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8cb2b9bf0 0x55b8cb2bc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.505+0000 7fdb5effd700 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8cb2b9bf0 0x55b8cb2bc010 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24cbe22577a93862 server_cookie=cae7b8b73a846a84 in_seq=0 out_seq=0 2024-05-09T00:03:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.505+0000 7fdb3effd700 10 client.20283 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.510+0000 7fdb3effd700 1 -- 172.21.15.12:0/2148578947 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8cb26f7b0 con 0x55b8cb2b9bf0 2024-05-09T00:03:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.510+0000 7fdb3effd700 10 client.20283 handle_command_reply: tid=0 2024-05-09T00:03:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.510+0000 7fdb72139b80 1 client.20283 shutdown 2024-05-09T00:03:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.511+0000 7fdb72139b80 1 -- 172.21.15.12:0/2148578947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 msgr2=0x55b8cb2ac1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.511+0000 7fdb72139b80 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2ac1c0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000fb70 tx=0x7fdb5001a050 comp rx=0 tx=0).stop 2024-05-09T00:03:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.512+0000 7fdb72139b80 1 -- 172.21.15.12:0/2148578947 shutdown_connections 2024-05-09T00:03:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.512+0000 7fdb72139b80 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a6270 0x55b8cb2a9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.512+0000 7fdb72139b80 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2ac1c0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.512+0000 7fdb72139b80 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8cb2b9bf0 0x55b8cb2bc010 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.512+0000 7fdb72139b80 1 --2- 172.21.15.12:0/2148578947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.512+0000 7fdb3effd700 0 client.20283 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.512+0000 7fdb72139b80 1 -- 172.21.15.12:0/2148578947 >> 172.21.15.12:0/2148578947 conn(0x55b8cb1fe1b0 msgr2=0x55b8cb2a44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.512+0000 7fdb72139b80 1 -- 172.21.15.12:0/2148578947 shutdown_connections 2024-05-09T00:03:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.512+0000 7fdb72139b80 1 -- 172.21.15.12:0/2148578947 wait complete. 2024-05-09T00:03:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.512+0000 7fdb72139b80 20 client.20283 trim_cache size 0 max 16384 2024-05-09T00:03:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb72139b80 1 Processor -- start 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb72139b80 1 -- start start 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 0x55b8cb29a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a6270 0x55b8cb2abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb72139b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb2aaab0 con 0x55b8cb2a0480 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb72139b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb29b330 con 0x55b8cb2a6270 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb72139b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb2c1250 con 0x55b8cb2a5950 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58520/0 (socket says 172.21.15.12:58520) 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.533+0000 7fdb5f7fe700 1 -- 172.21.15.12:0/2620280019 learned_addr learned my addr 172.21.15.12:0/2620280019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.534+0000 7fdb5effd700 1 --2- 172.21.15.12:0/2620280019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 0x55b8cb29a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.534+0000 7fdb5ffff700 1 --2- 172.21.15.12:0/2620280019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a6270 0x55b8cb2abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.534+0000 7fdb5effd700 1 -- 172.21.15.12:0/2620280019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a6270 msgr2=0x55b8cb2abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.534+0000 7fdb5effd700 1 --2- 172.21.15.12:0/2620280019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a6270 0x55b8cb2abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.534+0000 7fdb5effd700 1 -- 172.21.15.12:0/2620280019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 msgr2=0x55b8cb2a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.534+0000 7fdb5effd700 1 --2- 172.21.15.12:0/2620280019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.534+0000 7fdb5effd700 1 -- 172.21.15.12:0/2620280019 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8cb01db00 con 0x55b8cb2a5950 2024-05-09T00:03:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.534+0000 7fdb5effd700 1 --2- 172.21.15.12:0/2620280019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 0x55b8cb29a910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdb5401b040 tx=0x7fdb54011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28d308feca9b823d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.534+0000 7fdb467fc700 1 -- 172.21.15.12:0/2620280019 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54029040 con 0x55b8cb2a5950 2024-05-09T00:03:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.535+0000 7fdb467fc700 1 -- 172.21.15.12:0/2620280019 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb5402ad00 con 0x55b8cb2a5950 2024-05-09T00:03:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.535+0000 7fdb467fc700 1 -- 172.21.15.12:0/2620280019 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5402a700 con 0x55b8cb2a5950 2024-05-09T00:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.535+0000 7fdb72139b80 1 -- 172.21.15.12:0/2620280019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 msgr2=0x55b8cb29a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.535+0000 7fdb72139b80 1 --2- 172.21.15.12:0/2620280019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 0x55b8cb29a910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdb5401b040 tx=0x7fdb54011900 comp rx=0 tx=0).stop 2024-05-09T00:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.535+0000 7fdb72139b80 1 -- 172.21.15.12:0/2620280019 shutdown_connections 2024-05-09T00:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.535+0000 7fdb72139b80 1 --2- 172.21.15.12:0/2620280019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a6270 0x55b8cb2abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.535+0000 7fdb72139b80 1 --2- 172.21.15.12:0/2620280019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a0480 0x55b8cb2a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.535+0000 7fdb72139b80 1 --2- 172.21.15.12:0/2620280019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 0x55b8cb29a910 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.535+0000 7fdb72139b80 1 -- 172.21.15.12:0/2620280019 >> 172.21.15.12:0/2620280019 conn(0x55b8cb1fe1b0 msgr2=0x55b8cb296e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.536+0000 7fdb72139b80 1 -- 172.21.15.12:0/2620280019 shutdown_connections 2024-05-09T00:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.536+0000 7fdb72139b80 1 -- 172.21.15.12:0/2620280019 wait complete. 2024-05-09T00:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.536+0000 7fdb72139b80 1 -- start start 2024-05-09T00:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.536+0000 7fdb72139b80 1 Processor -- start 2024-05-09T00:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a0480 0x55b8cb2b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb72139b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb2a3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb72139b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb2a4010 con 0x55b8cb2a6270 2024-05-09T00:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb72139b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb2a4180 con 0x55b8cb2a0480 2024-05-09T00:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb72139b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8cb2a4340 con 0x55b8cb2a5950 2024-05-09T00:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb72139b80 10 client.0 _ll_register_callbacks cb 0x55b8cb195ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb2a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a0480 0x55b8cb2b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb2a3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58526/0 (socket says 172.21.15.12:58526) 2024-05-09T00:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5ffff700 1 -- 172.21.15.12:0/996584570 learned_addr learned my addr 172.21.15.12:0/996584570 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5effd700 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5effd700 1 -- 172.21.15.12:0/996584570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a0480 msgr2=0x55b8cb2b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5effd700 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a0480 0x55b8cb2b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5effd700 1 -- 172.21.15.12:0/996584570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 msgr2=0x55b8cb2a3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5effd700 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb2a3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5effd700 1 -- 172.21.15.12:0/996584570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8cb071350 con 0x55b8cb2a5950 2024-05-09T00:03:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5ffff700 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb2a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb5effd700 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a3380 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdb54005950 tx=0x7fdb54005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b75f43f2c15249e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.537+0000 7fdb3effd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb3effd700 1 -- 172.21.15.12:0/996584570 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54029040 con 0x55b8cb2a5950 2024-05-09T00:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb3effd700 1 -- 172.21.15.12:0/996584570 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdb54000e40 con 0x55b8cb2a5950 2024-05-09T00:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb72139b80 1 -- 172.21.15.12:0/996584570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb54004390 con 0x55b8cb2a5950 2024-05-09T00:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb3effd700 1 -- 172.21.15.12:0/996584570 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb540046e0 con 0x55b8cb2a5950 2024-05-09T00:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb72139b80 10 client.30028 fetch_fsmap learned FSMap version 72 2024-05-09T00:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb72139b80 1 -- 172.21.15.12:0/996584570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55b8cb1b6500 con 0x55b8cb2a5950 2024-05-09T00:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb3effd700 1 -- 172.21.15.12:0/996584570 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb54004390 con 0x55b8cb2a5950 2024-05-09T00:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb3effd700 1 -- 172.21.15.12:0/996584570 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb54011620 con 0x55b8cb2a5950 2024-05-09T00:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb72139b80 10 client.30028 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T00:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb72139b80 10 client.30028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb72139b80 20 client.30028 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb72139b80 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8cb3104a0 0x55b8cb312890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb72139b80 4 client.30028 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.538+0000 7fdb72139b80 1 -- 172.21.15.12:0/996584570 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b8cb1e8e60 con 0x55b8cb3104a0 2024-05-09T00:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.539+0000 7fdb5f7fe700 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8cb3104a0 0x55b8cb312890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.539+0000 7fdb3effd700 10 client.30028 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.539+0000 7fdb5f7fe700 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8cb3104a0 0x55b8cb312890 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3292f37a5112c816 server_cookie=163edb331856ce36 in_seq=0 out_seq=0 2024-05-09T00:03:35.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:35 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:03:35.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:35 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:03:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.884+0000 7fdb3effd700 1 -- 172.21.15.12:0/996584570 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b8cb1e8e60 con 0x55b8cb3104a0 2024-05-09T00:03:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.884+0000 7fdb3effd700 10 client.30028 handle_command_reply: tid=0 2024-05-09T00:03:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.884+0000 7fdb72139b80 1 client.30028 shutdown 2024-05-09T00:03:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 1 -- 172.21.15.12:0/996584570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 msgr2=0x55b8cb2a3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a3380 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdb54005950 tx=0x7fdb54005e40 comp rx=0 tx=0).stop 2024-05-09T00:03:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 1 -- 172.21.15.12:0/996584570 shutdown_connections 2024-05-09T00:03:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8cb2a6270 0x55b8cb2a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b8cb3104a0 0x55b8cb312890 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b8cb2a0480 0x55b8cb2b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 1 --2- 172.21.15.12:0/996584570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8cb2a5950 0x55b8cb2a3380 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 1 -- 172.21.15.12:0/996584570 >> 172.21.15.12:0/996584570 conn(0x55b8cb1fe1b0 msgr2=0x55b8cb2b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb3effd700 0 client.30028 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 1 -- 172.21.15.12:0/996584570 shutdown_connections 2024-05-09T00:03:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 1 -- 172.21.15.12:0/996584570 wait complete. 2024-05-09T00:03:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.886+0000 7fdb72139b80 20 client.30028 trim_cache size 0 max 16384 2024-05-09T00:03:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:03:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:03:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:03:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:03:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 -- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb480a1310 msgr2=0x7fdb480a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb480a1310 0x7fdb480a37d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdb50006010 tx=0x7fdb5000b260 comp rx=0 tx=0).stop 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 -- 172.21.15.12:0/3387900905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60105e60 msgr2=0x7fdb601024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60105e60 0x7fdb601024d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800efc0 tx=0x7fdb5800c3a0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 -- 172.21.15.12:0/3387900905 shutdown_connections 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb480a1310 0x7fdb480a37d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60105e60 0x7fdb601024d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb600ffda0 0x7fdb60101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 --2- 172.21.15.12:0/3387900905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb60100760 0x7fdb60101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 -- 172.21.15.12:0/3387900905 >> 172.21.15.12:0/3387900905 conn(0x7fdb60076f10 msgr2=0x7fdb60068460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 -- 172.21.15.12:0/3387900905 shutdown_connections 2024-05-09T00:03:35.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:35.888+0000 7fdb65f7f700 1 -- 172.21.15.12:0/3387900905 wait complete. 2024-05-09T00:03:36.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 152334 38.71 MB/sec execute 91 sec latency 189.056 ms 2024-05-09T00:03:36.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:36 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:03:36.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:36 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:36.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:36 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:03:36.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:36 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:03:36.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:36 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:36.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:36 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:36.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:36 smithi012 ceph-mon[42503]: pgmap v553: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 20 MiB/s wr, 932 op/s 2024-05-09T00:03:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:36 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:03:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:36 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:36 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:03:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:36 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:03:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:36 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:36 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:36.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:36 smithi204 ceph-mon[47373]: pgmap v553: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 20 MiB/s wr, 932 op/s 2024-05-09T00:03:36.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:36 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:03:36.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:36 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:36.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:36 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:03:36.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:36 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:03:36.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:36 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:36.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:36 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:36.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:36 smithi160 ceph-mon[46152]: pgmap v553: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 20 MiB/s wr, 932 op/s 2024-05-09T00:03:36.765 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:03:36 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: ignoring --setuser ceph since I am not root 2024-05-09T00:03:36.765 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:03:36 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: ignoring --setgroup ceph since I am not root 2024-05-09T00:03:36.765 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:03:36 smithi160 ceph-mds[81732]: -- 172.21.15.160:0/3936935343 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557f0a3be1a0 con 0x557f0a394c00 2024-05-09T00:03:36.765 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:03:36 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: starting mds.k at 2024-05-09T00:03:37.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 153312 38.85 MB/sec execute 92 sec latency 43.995 ms 2024-05-09T00:03:37.663 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:37 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:37.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:37 smithi012 ceph-mon[42503]: daemon mds.k restarted 2024-05-09T00:03:37.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:37 smithi012 ceph-mon[42503]: osdmap e120: 12 total, 12 up, 12 in 2024-05-09T00:03:37.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:37 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:37.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:37 smithi204 ceph-mon[47373]: daemon mds.k restarted 2024-05-09T00:03:37.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:37 smithi204 ceph-mon[47373]: osdmap e120: 12 total, 12 up, 12 in 2024-05-09T00:03:37.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:37 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:03:37.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:37 smithi160 ceph-mon[46152]: daemon mds.k restarted 2024-05-09T00:03:37.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:37 smithi160 ceph-mon[46152]: osdmap e120: 12 total, 12 up, 12 in 2024-05-09T00:03:38.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 154113 38.95 MB/sec execute 93 sec latency 63.769 ms 2024-05-09T00:03:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:38 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6832/181468454,v1:172.21.15.160:6833/181468454] up:boot 2024-05-09T00:03:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:38 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:03:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:38 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:03:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:38 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:38.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:38 smithi012 ceph-mon[42503]: pgmap v555: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 1.17k op/s 2024-05-09T00:03:38.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:38 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6832/181468454,v1:172.21.15.160:6833/181468454] up:boot 2024-05-09T00:03:38.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:38 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:03:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:38 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:03:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:38 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:38.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:38 smithi204 ceph-mon[47373]: pgmap v555: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 1.17k op/s 2024-05-09T00:03:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:38 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6832/181468454,v1:172.21.15.160:6833/181468454] up:boot 2024-05-09T00:03:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:38 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:03:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:38 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:03:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:38 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:03:38.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:38 smithi160 ceph-mon[46152]: pgmap v555: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 25 MiB/s wr, 1.17k op/s 2024-05-09T00:03:39.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 154686 38.87 MB/sec execute 94 sec latency 142.349 ms 2024-05-09T00:03:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:39 smithi204 ceph-mon[47373]: pgmap v556: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-05-09T00:03:40.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:39 smithi160 ceph-mon[46152]: pgmap v556: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-05-09T00:03:40.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:39 smithi012 ceph-mon[42503]: pgmap v556: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-05-09T00:03:40.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 155247 38.75 MB/sec execute 95 sec latency 256.953 ms 2024-05-09T00:03:41.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 156077 38.74 MB/sec execute 96 sec latency 75.240 ms 2024-05-09T00:03:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:41 smithi204 ceph-mon[47373]: pgmap v557: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-05-09T00:03:42.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:41 smithi160 ceph-mon[46152]: pgmap v557: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-05-09T00:03:42.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:41 smithi012 ceph-mon[42503]: pgmap v557: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-05-09T00:03:42.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 157007 38.98 MB/sec execute 97 sec latency 40.937 ms 2024-05-09T00:03:43.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 157715 39.00 MB/sec execute 98 sec latency 74.424 ms 2024-05-09T00:03:43.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:43 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:03:44.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:43 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:03:44.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:43 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:03:44.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 158391 38.88 MB/sec execute 99 sec latency 49.612 ms 2024-05-09T00:03:44.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:44 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:03:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:44 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:03:44.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:44 smithi012 ceph-mon[42503]: pgmap v558: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.7 MiB/s wr, 768 op/s 2024-05-09T00:03:45.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:44 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:03:45.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:44 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:03:45.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:44 smithi204 ceph-mon[47373]: pgmap v558: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.7 MiB/s wr, 768 op/s 2024-05-09T00:03:45.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:44 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:03:45.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:44 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:03:45.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:44 smithi160 ceph-mon[46152]: pgmap v558: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.7 MiB/s wr, 768 op/s 2024-05-09T00:03:45.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 159223 38.93 MB/sec execute 100 sec latency 88.512 ms 2024-05-09T00:03:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:45 smithi204 ceph-mon[47373]: pgmap v559: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.2 MiB/s wr, 694 op/s 2024-05-09T00:03:46.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:45 smithi160 ceph-mon[46152]: pgmap v559: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.2 MiB/s wr, 694 op/s 2024-05-09T00:03:46.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:45 smithi012 ceph-mon[42503]: pgmap v559: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.2 MiB/s wr, 694 op/s 2024-05-09T00:03:46.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 160106 39.06 MB/sec execute 101 sec latency 61.854 ms 2024-05-09T00:03:47.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 160846 39.10 MB/sec execute 102 sec latency 64.885 ms 2024-05-09T00:03:48.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:47 smithi204 ceph-mon[47373]: pgmap v560: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 828 op/s 2024-05-09T00:03:48.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:47 smithi160 ceph-mon[46152]: pgmap v560: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 828 op/s 2024-05-09T00:03:48.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:47 smithi012 ceph-mon[42503]: pgmap v560: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 828 op/s 2024-05-09T00:03:48.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 161482 39.00 MB/sec execute 103 sec latency 106.990 ms 2024-05-09T00:03:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.042+0000 7f25a1740700 1 -- 172.21.15.12:0/1748804619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f259c107020 msgr2=0x7f259c107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.042+0000 7f25a1740700 1 --2- 172.21.15.12:0/1748804619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f259c107020 0x7f259c107400 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f258c0099c0 tx=0x7f258c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.042+0000 7f25a1740700 1 -- 172.21.15.12:0/1748804619 shutdown_connections 2024-05-09T00:03:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.042+0000 7f25a1740700 1 --2- 172.21.15.12:0/1748804619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c0ffdd0 0x7f259c10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.042+0000 7f25a1740700 1 --2- 172.21.15.12:0/1748804619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f259c107020 0x7f259c107400 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.042+0000 7f25a1740700 1 --2- 172.21.15.12:0/1748804619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c0ff410 0x7f259c0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.042+0000 7f25a1740700 1 -- 172.21.15.12:0/1748804619 >> 172.21.15.12:0/1748804619 conn(0x7f259c0763a0 msgr2=0x7f259c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.042+0000 7f25a1740700 1 -- 172.21.15.12:0/1748804619 shutdown_connections 2024-05-09T00:03:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.042+0000 7f25a1740700 1 -- 172.21.15.12:0/1748804619 wait complete. 2024-05-09T00:03:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.043+0000 7f25a1740700 1 Processor -- start 2024-05-09T00:03:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.043+0000 7f25a1740700 1 -- start start 2024-05-09T00:03:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f25a1740700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f259c0ff410 0x7f259c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f25a1740700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c0ffdd0 0x7f259c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f25a1740700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c107020 0x7f259c19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f25a1740700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c19e4e0 con 0x7f259c0ff410 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f25a1740700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c19e650 con 0x7f259c0ffdd0 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f25a1740700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c19e810 con 0x7f259c107020 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f259affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f259c0ff410 0x7f259c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f259a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c0ffdd0 0x7f259c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f259affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f259c0ff410 0x7f259c1995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38964/0 (socket says 172.21.15.12:38964) 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f259affd700 1 -- 172.21.15.12:0/561733404 learned_addr learned my addr 172.21.15.12:0/561733404 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f259b7fe700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c107020 0x7f259c19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f259a7fc700 1 -- 172.21.15.12:0/561733404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c107020 msgr2=0x7f259c19dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f259a7fc700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c107020 0x7f259c19dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f259a7fc700 1 -- 172.21.15.12:0/561733404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f259c0ff410 msgr2=0x7f259c1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f259a7fc700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f259c0ff410 0x7f259c1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.044+0000 7f259a7fc700 1 -- 172.21.15.12:0/561733404 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c0095f0 con 0x7f259c0ffdd0 2024-05-09T00:03:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.045+0000 7f259affd700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f259c0ff410 0x7f259c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:03:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.045+0000 7f259a7fc700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c0ffdd0 0x7f259c199b00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f259000d750 tx=0x7f259000da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.045+0000 7f2583fff700 1 -- 172.21.15.12:0/561733404 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590004020 con 0x7f259c0ffdd0 2024-05-09T00:03:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.045+0000 7f2583fff700 1 -- 172.21.15.12:0/561733404 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2590004830 con 0x7f259c0ffdd0 2024-05-09T00:03:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.045+0000 7f2583fff700 1 -- 172.21.15.12:0/561733404 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900054c0 con 0x7f259c0ffdd0 2024-05-09T00:03:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.046+0000 7f25a1740700 1 -- 172.21.15.12:0/561733404 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c19eaf0 con 0x7f259c0ffdd0 2024-05-09T00:03:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.046+0000 7f25a1740700 1 -- 172.21.15.12:0/561733404 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c1a6330 con 0x7f259c0ffdd0 2024-05-09T00:03:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.049+0000 7f2583fff700 1 -- 172.21.15.12:0/561733404 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f25900049a0 con 0x7f259c0ffdd0 2024-05-09T00:03:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.050+0000 7f2583fff700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25840a1390 0x7f25840a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.050+0000 7f2583fff700 1 -- 172.21.15.12:0/561733404 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f25900cc5a0 con 0x7f259c0ffdd0 2024-05-09T00:03:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.050+0000 7f259affd700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25840a1390 0x7f25840a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.051+0000 7f259affd700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25840a1390 0x7f25840a3850 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f258c000c00 tx=0x7f258c005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.052+0000 7f25ad8fab80 1 Processor -- start 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.052+0000 7f25ad8fab80 1 -- start start 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510ec110 0x562d510ec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510eca30 0x562d510ee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d510eee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d510460f0 con 0x562d510eca30 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d51043ab0 con 0x562d510ee9f0 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5103f3a0 con 0x562d510ec110 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f259affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510ec110 0x562d510ec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f259affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510ec110 0x562d510ec4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44656/0 (socket says 172.21.15.12:44656) 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f259affd700 1 -- 172.21.15.12:0/3853535635 learned_addr learned my addr 172.21.15.12:0/3853535635 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f259b7fe700 1 --2- 172.21.15.12:0/3853535635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d510eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.053+0000 7f259a7fc700 1 --2- 172.21.15.12:0/3853535635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510eca30 0x562d510ee4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.054+0000 7f259affd700 1 -- 172.21.15.12:0/3853535635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 msgr2=0x562d510eee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.054+0000 7f259affd700 1 --2- 172.21.15.12:0/3853535635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d510eee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.054+0000 7f259affd700 1 -- 172.21.15.12:0/3853535635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510eca30 msgr2=0x562d510ee4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.054+0000 7f259affd700 1 --2- 172.21.15.12:0/3853535635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510eca30 0x562d510ee4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.054+0000 7f259affd700 1 -- 172.21.15.12:0/3853535635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d50ec0350 con 0x562d510ec110 2024-05-09T00:03:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.054+0000 7f259b7fe700 1 --2- 172.21.15.12:0/3853535635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d510eee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:03:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.054+0000 7f259affd700 1 --2- 172.21.15.12:0/3853535635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510ec110 0x562d510ec4f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f258c004140 tx=0x7f258c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a49818abe8999a18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.054+0000 7f25817fa700 1 -- 172.21.15.12:0/3853535635 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c02a070 con 0x562d510ec110 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.054+0000 7f25817fa700 1 -- 172.21.15.12:0/3853535635 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258c01d040 con 0x562d510ec110 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.054+0000 7f25817fa700 1 -- 172.21.15.12:0/3853535635 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00fb20 con 0x562d510ec110 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.055+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3853535635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510ec110 msgr2=0x562d510ec4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.055+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3853535635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510ec110 0x562d510ec4f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f258c004140 tx=0x7f258c01f000 comp rx=0 tx=0).stop 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.055+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3853535635 shutdown_connections 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.055+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3853535635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d510eee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.055+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3853535635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510ec110 0x562d510ec4f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.055+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3853535635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510eca30 0x562d510ee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.055+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3853535635 >> 172.21.15.12:0/3853535635 conn(0x562d5104d1b0 msgr2=0x562d510e9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.056+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3853535635 shutdown_connections 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.056+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3853535635 wait complete. 2024-05-09T00:03:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.056+0000 7f25ad8fab80 1 -- start start 2024-05-09T00:03:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.056+0000 7f25ad8fab80 1 Processor -- start 2024-05-09T00:03:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.056+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ec110 0x562d5110f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.056+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d5110fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.056+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ee9f0 0x562d51109890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.056+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d50fdb500 con 0x562d510ee9f0 2024-05-09T00:03:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.056+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d50fdb1a0 con 0x562d510ec110 2024-05-09T00:03:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.056+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d50ec2640 con 0x562d510eca30 2024-05-09T00:03:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.057+0000 7f25ad8fab80 10 client.0 _ll_register_callbacks cb 0x562d50fe4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.057+0000 7f259affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ec110 0x562d5110f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.057+0000 7f259affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ec110 0x562d5110f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:38402/0 (socket says 172.21.15.12:38402) 2024-05-09T00:03:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.057+0000 7f259affd700 1 -- 172.21.15.12:0/1637581741 learned_addr learned my addr 172.21.15.12:0/1637581741 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.057+0000 7f259a7fc700 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d5110fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.057+0000 7f259b7fe700 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ee9f0 0x562d51109890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.057+0000 7f259affd700 1 -- 172.21.15.12:0/1637581741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 msgr2=0x562d5110fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.057+0000 7f259affd700 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d5110fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.058+0000 7f259affd700 1 -- 172.21.15.12:0/1637581741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ee9f0 msgr2=0x562d51109890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.058+0000 7f259affd700 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ee9f0 0x562d51109890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.058+0000 7f259affd700 1 -- 172.21.15.12:0/1637581741 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c10e6e0 con 0x562d510ec110 2024-05-09T00:03:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.058+0000 7f259affd700 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ec110 0x562d5110f590 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f258c030040 tx=0x7f258c009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a180df282174ac6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.058+0000 7f257eff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:03:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.058+0000 7f257eff5700 1 -- 172.21.15.12:0/1637581741 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c02a040 con 0x562d510ec110 2024-05-09T00:03:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.059+0000 7f257eff5700 1 -- 172.21.15.12:0/1637581741 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f258c003b10 con 0x562d510ec110 2024-05-09T00:03:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.059+0000 7f25ad8fab80 1 -- 172.21.15.12:0/1637581741 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d50f98210 con 0x562d510ec110 2024-05-09T00:03:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f257eff5700 1 -- 172.21.15.12:0/1637581741 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f258c003cd0 con 0x562d510ec110 2024-05-09T00:03:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f25ad8fab80 10 client.29948 fetch_fsmap learned FSMap version 74 2024-05-09T00:03:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f25ad8fab80 1 -- 172.21.15.12:0/1637581741 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x562d50e6cb00 con 0x562d510ec110 2024-05-09T00:03:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f257eff5700 1 -- 172.21.15.12:0/1637581741 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d50f98210 con 0x562d510ec110 2024-05-09T00:03:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f257eff5700 1 -- 172.21.15.12:0/1637581741 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f258c040420 con 0x562d510ec110 2024-05-09T00:03:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f25ad8fab80 10 client.29948 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:03:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f25ad8fab80 10 client.29948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f25ad8fab80 20 client.29948 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562d51119650 0x562d5111ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f25ad8fab80 4 client.29948 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:03:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.061+0000 7f25ad8fab80 1 -- 172.21.15.12:0/1637581741 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d510be7b0 con 0x562d51119650 2024-05-09T00:03:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.062+0000 7f259a7fc700 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562d51119650 0x562d5111ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.062+0000 7f259a7fc700 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562d51119650 0x562d5111ba40 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e41a058342163c server_cookie=e92df8ee6dafc1c3 in_seq=0 out_seq=0 2024-05-09T00:03:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.063+0000 7f257eff5700 10 client.29948 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.066+0000 7f257eff5700 1 -- 172.21.15.12:0/1637581741 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d510be7b0 con 0x562d51119650 2024-05-09T00:03:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.066+0000 7f257eff5700 10 client.29948 handle_command_reply: tid=0 2024-05-09T00:03:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.066+0000 7f25ad8fab80 1 client.29948 shutdown 2024-05-09T00:03:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 1 -- 172.21.15.12:0/1637581741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ec110 msgr2=0x562d5110f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ec110 0x562d5110f590 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f258c030040 tx=0x7f258c009550 comp rx=0 tx=0).stop 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 1 -- 172.21.15.12:0/1637581741 shutdown_connections 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ee9f0 0x562d51109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562d51119650 0x562d5111ba40 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ec110 0x562d5110f590 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/1637581741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d5110fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f257eff5700 0 client.29948 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 1 -- 172.21.15.12:0/1637581741 >> 172.21.15.12:0/1637581741 conn(0x562d5104d1b0 msgr2=0x562d510f15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 1 -- 172.21.15.12:0/1637581741 shutdown_connections 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 1 -- 172.21.15.12:0/1637581741 wait complete. 2024-05-09T00:03:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.067+0000 7f25ad8fab80 20 client.29948 trim_cache size 0 max 16384 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f25ad8fab80 1 Processor -- start 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f25ad8fab80 1 -- start start 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d510ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d51108760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d510f1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d51120c00 con 0x562d510ec110 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d510f23b0 con 0x562d510ee9f0 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d510f6d30 con 0x562d510eca30 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f259affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d510ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f259affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d510ee3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38994/0 (socket says 172.21.15.12:38994) 2024-05-09T00:03:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f259affd700 1 -- 172.21.15.12:0/3687980854 learned_addr learned my addr 172.21.15.12:0/3687980854 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.091+0000 7f259a7fc700 1 --2- 172.21.15.12:0/3687980854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d51108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.092+0000 7f259affd700 1 -- 172.21.15.12:0/3687980854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 msgr2=0x562d51108760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.092+0000 7f259affd700 1 --2- 172.21.15.12:0/3687980854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d51108760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.092+0000 7f259affd700 1 -- 172.21.15.12:0/3687980854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 msgr2=0x562d510f1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.092+0000 7f259affd700 1 --2- 172.21.15.12:0/3687980854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d510f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.092+0000 7f259affd700 1 -- 172.21.15.12:0/3687980854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d51005500 con 0x562d510ec110 2024-05-09T00:03:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.092+0000 7f259affd700 1 --2- 172.21.15.12:0/3687980854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d510ee3d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f258c009990 tx=0x7f258c00bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991ea943737ce09e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.092+0000 7f2581ffb700 1 -- 172.21.15.12:0/3687980854 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c040aa0 con 0x562d510ec110 2024-05-09T00:03:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.092+0000 7f2581ffb700 1 -- 172.21.15.12:0/3687980854 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258c003dd0 con 0x562d510ec110 2024-05-09T00:03:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.092+0000 7f2581ffb700 1 -- 172.21.15.12:0/3687980854 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c037460 con 0x562d510ec110 2024-05-09T00:03:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.093+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3687980854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 msgr2=0x562d510ee3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.093+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3687980854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d510ee3d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f258c009990 tx=0x7f258c00bfb0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.093+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3687980854 shutdown_connections 2024-05-09T00:03:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.093+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3687980854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d510f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.093+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3687980854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d510ee3d0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.093+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3687980854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d51108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.093+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3687980854 >> 172.21.15.12:0/3687980854 conn(0x562d5104d1b0 msgr2=0x562d510eb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.094+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3687980854 shutdown_connections 2024-05-09T00:03:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.095+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3687980854 wait complete. 2024-05-09T00:03:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.095+0000 7f25ad8fab80 1 -- start start 2024-05-09T00:03:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.095+0000 7f25ad8fab80 1 Processor -- start 2024-05-09T00:03:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.095+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d511122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.095+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d51112800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.095+0000 7f25ad8fab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d51112d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.095+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5116eab0 con 0x562d510ec110 2024-05-09T00:03:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.095+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5116ec20 con 0x562d510ee9f0 2024-05-09T00:03:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.095+0000 7f25ad8fab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5117fdf0 con 0x562d510eca30 2024-05-09T00:03:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.095+0000 7f25ad8fab80 10 client.0 _ll_register_callbacks cb 0x562d50fe4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d511122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d511122c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39004/0 (socket says 172.21.15.12:39004) 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259affd700 1 -- 172.21.15.12:0/3189403910 learned_addr learned my addr 172.21.15.12:0/3189403910 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259b7fe700 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d51112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259a7fc700 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d51112800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259affd700 1 -- 172.21.15.12:0/3189403910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 msgr2=0x562d51112800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259affd700 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d51112800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259affd700 1 -- 172.21.15.12:0/3189403910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 msgr2=0x562d51112d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259affd700 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d51112d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259affd700 1 -- 172.21.15.12:0/3189403910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d50e6cb00 con 0x562d510ec110 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f259affd700 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d511122c0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f258c00fce0 tx=0x7f258c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194b9da68188213a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f257eff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:03:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.096+0000 7f257eff5700 1 -- 172.21.15.12:0/3189403910 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c02a030 con 0x562d510ec110 2024-05-09T00:03:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.097+0000 7f257eff5700 1 -- 172.21.15.12:0/3189403910 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f258c037460 con 0x562d510ec110 2024-05-09T00:03:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.097+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3189403910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f258c01d040 con 0x562d510ec110 2024-05-09T00:03:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.097+0000 7f257eff5700 1 -- 172.21.15.12:0/3189403910 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f258c037620 con 0x562d510ec110 2024-05-09T00:03:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.097+0000 7f25ad8fab80 10 client.20310 fetch_fsmap learned FSMap version 74 2024-05-09T00:03:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.097+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3189403910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x562d50ec4c70 con 0x562d510ec110 2024-05-09T00:03:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.097+0000 7f257eff5700 1 -- 172.21.15.12:0/3189403910 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258c01d040 con 0x562d510ec110 2024-05-09T00:03:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.097+0000 7f257eff5700 1 -- 172.21.15.12:0/3189403910 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f258c046030 con 0x562d510ec110 2024-05-09T00:03:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.098+0000 7f25ad8fab80 10 client.20310 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:03:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.098+0000 7f25ad8fab80 10 client.20310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:03:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.098+0000 7f25ad8fab80 20 client.20310 populate_metadata read hostname 'smithi012' 2024-05-09T00:03:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.098+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562d511810b0 0x562d511834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.098+0000 7f25ad8fab80 4 client.20310 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:03:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.098+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3189403910 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562d51037e60 con 0x562d511810b0 2024-05-09T00:03:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.098+0000 7f259a7fc700 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562d511810b0 0x562d511834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.098+0000 7f259a7fc700 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562d511810b0 0x562d511834a0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26274e6d247f1cf1 server_cookie=f1d90c1ff9f5ab76 in_seq=0 out_seq=0 2024-05-09T00:03:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.098+0000 7f257eff5700 10 client.20310 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:49.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 162320 39.01 MB/sec execute 104 sec latency 105.318 ms 2024-05-09T00:03:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.494+0000 7f257eff5700 1 -- 172.21.15.12:0/3189403910 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562d51037e60 con 0x562d511810b0 2024-05-09T00:03:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.494+0000 7f257eff5700 10 client.20310 handle_command_reply: tid=0 2024-05-09T00:03:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 client.20310 shutdown 2024-05-09T00:03:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3189403910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 msgr2=0x562d511122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d511122c0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f258c00fce0 tx=0x7f258c009360 comp rx=0 tx=0).stop 2024-05-09T00:03:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3189403910 shutdown_connections 2024-05-09T00:03:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562d511810b0 0x562d511834a0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562d510ee9f0 0x562d51112d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562d510ec110 0x562d511122c0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 --2- 172.21.15.12:0/3189403910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d510eca30 0x562d51112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3189403910 >> 172.21.15.12:0/3189403910 conn(0x562d5104d1b0 msgr2=0x562d510f1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f257eff5700 0 client.20310 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:03:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3189403910 shutdown_connections 2024-05-09T00:03:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.495+0000 7f25ad8fab80 1 -- 172.21.15.12:0/3189403910 wait complete. 2024-05-09T00:03:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.496+0000 7f25ad8fab80 20 client.20310 trim_cache size 0 max 16384 2024-05-09T00:03:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:03:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:03:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:03:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:03:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.496+0000 7f25a1740700 1 -- 172.21.15.12:0/561733404 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25840a1390 msgr2=0x7f25840a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.496+0000 7f25a1740700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25840a1390 0x7f25840a3850 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f258c000c00 tx=0x7f258c005fb0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.496+0000 7f25a1740700 1 -- 172.21.15.12:0/561733404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c0ffdd0 msgr2=0x7f259c199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.496+0000 7f25a1740700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c0ffdd0 0x7f259c199b00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f259000d750 tx=0x7f259000da60 comp rx=0 tx=0).stop 2024-05-09T00:03:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.498+0000 7f25a1740700 1 -- 172.21.15.12:0/561733404 shutdown_connections 2024-05-09T00:03:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.498+0000 7f25a1740700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25840a1390 0x7f25840a3850 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.498+0000 7f25a1740700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259c107020 0x7f259c19dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.498+0000 7f25a1740700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f259c0ff410 0x7f259c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.498+0000 7f25a1740700 1 --2- 172.21.15.12:0/561733404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c0ffdd0 0x7f259c199b00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.498+0000 7f25a1740700 1 -- 172.21.15.12:0/561733404 >> 172.21.15.12:0/561733404 conn(0x7f259c0763a0 msgr2=0x7f259c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.498+0000 7f25a1740700 1 -- 172.21.15.12:0/561733404 shutdown_connections 2024-05-09T00:03:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:03:49.498+0000 7f25a1740700 1 -- 172.21.15.12:0/561733404 wait complete. 2024-05-09T00:03:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:49 smithi204 ceph-mon[47373]: pgmap v561: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 591 op/s 2024-05-09T00:03:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:49 smithi160 ceph-mon[46152]: pgmap v561: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 591 op/s 2024-05-09T00:03:50.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:49 smithi012 ceph-mon[42503]: pgmap v561: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 591 op/s 2024-05-09T00:03:50.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 163237 39.17 MB/sec execute 105 sec latency 258.956 ms 2024-05-09T00:03:51.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 163974 39.23 MB/sec execute 106 sec latency 40.151 ms 2024-05-09T00:03:52.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:51 smithi204 ceph-mon[47373]: pgmap v562: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.1 MiB/s wr, 455 op/s 2024-05-09T00:03:52.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:51 smithi160 ceph-mon[46152]: pgmap v562: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.1 MiB/s wr, 455 op/s 2024-05-09T00:03:52.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:51 smithi012 ceph-mon[42503]: pgmap v562: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.1 MiB/s wr, 455 op/s 2024-05-09T00:03:52.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164646 39.16 MB/sec execute 107 sec latency 43.975 ms 2024-05-09T00:03:53.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:52 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:53.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:52 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:53.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:52 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:53.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 165399 39.10 MB/sec execute 108 sec latency 42.481 ms 2024-05-09T00:03:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:53 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:53 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:53 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:03:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:53 smithi204 ceph-mon[47373]: pgmap v563: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 10 MiB/s wr, 772 op/s 2024-05-09T00:03:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:53 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:53 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:53 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:03:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:53 smithi160 ceph-mon[46152]: pgmap v563: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 10 MiB/s wr, 772 op/s 2024-05-09T00:03:54.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:53 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:54.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:53 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:54.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:53 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:03:54.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:53 smithi012 ceph-mon[42503]: pgmap v563: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 10 MiB/s wr, 772 op/s 2024-05-09T00:03:54.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 165535 38.80 MB/sec execute 109 sec latency 794.692 ms 2024-05-09T00:03:55.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 165724 38.55 MB/sec execute 110 sec latency 895.797 ms 2024-05-09T00:03:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:55 smithi204 ceph-mon[47373]: pgmap v564: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.6 MiB/s wr, 664 op/s 2024-05-09T00:03:56.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:55 smithi160 ceph-mon[46152]: pgmap v564: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.6 MiB/s wr, 664 op/s 2024-05-09T00:03:56.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:55 smithi012 ceph-mon[42503]: pgmap v564: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.6 MiB/s wr, 664 op/s 2024-05-09T00:03:56.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 166160 38.43 MB/sec execute 111 sec latency 119.474 ms 2024-05-09T00:03:57.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 167009 38.48 MB/sec execute 112 sec latency 72.651 ms 2024-05-09T00:03:58.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:57 smithi204 ceph-mon[47373]: pgmap v565: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 953 op/s 2024-05-09T00:03:58.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:57 smithi160 ceph-mon[46152]: pgmap v565: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 953 op/s 2024-05-09T00:03:58.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:57 smithi012 ceph-mon[42503]: pgmap v565: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 953 op/s 2024-05-09T00:03:58.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 167733 38.56 MB/sec execute 113 sec latency 114.705 ms 2024-05-09T00:03:59.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 168453 38.57 MB/sec execute 114 sec latency 49.362 ms 2024-05-09T00:04:00.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:03:59 smithi204 ceph-mon[47373]: pgmap v566: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 969 op/s 2024-05-09T00:04:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:03:59 smithi160 ceph-mon[46152]: pgmap v566: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 969 op/s 2024-05-09T00:04:00.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:03:59 smithi012 ceph-mon[42503]: pgmap v566: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 969 op/s 2024-05-09T00:04:00.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 169201 38.54 MB/sec execute 115 sec latency 50.591 ms 2024-05-09T00:04:01.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 169964 38.55 MB/sec execute 116 sec latency 78.266 ms 2024-05-09T00:04:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:01 smithi204 ceph-mon[47373]: pgmap v567: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 938 op/s 2024-05-09T00:04:02.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:01 smithi160 ceph-mon[46152]: pgmap v567: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 938 op/s 2024-05-09T00:04:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:01 smithi012 ceph-mon[42503]: pgmap v567: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 938 op/s 2024-05-09T00:04:02.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170346 38.31 MB/sec execute 117 sec latency 141.433 ms 2024-05-09T00:04:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.671+0000 7f5931ebf700 1 -- 172.21.15.12:0/2291456432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f592c073340 msgr2=0x7f592c073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.671+0000 7f5931ebf700 1 --2- 172.21.15.12:0/2291456432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f592c073340 0x7f592c073720 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5920009990 tx=0x7f5920009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.671+0000 7f5931ebf700 1 -- 172.21.15.12:0/2291456432 shutdown_connections 2024-05-09T00:04:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.671+0000 7f5931ebf700 1 --2- 172.21.15.12:0/2291456432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c111eb0 0x7f592c1142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.671+0000 7f5931ebf700 1 --2- 172.21.15.12:0/2291456432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f592c073340 0x7f592c073720 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.671+0000 7f5931ebf700 1 --2- 172.21.15.12:0/2291456432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c073c60 0x7f592c111970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.671+0000 7f5931ebf700 1 -- 172.21.15.12:0/2291456432 >> 172.21.15.12:0/2291456432 conn(0x7f592c0fc970 msgr2=0x7f592c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.671+0000 7f5931ebf700 1 -- 172.21.15.12:0/2291456432 shutdown_connections 2024-05-09T00:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.671+0000 7f5931ebf700 1 -- 172.21.15.12:0/2291456432 wait complete. 2024-05-09T00:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.672+0000 7f5931ebf700 1 Processor -- start 2024-05-09T00:04:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.672+0000 7f5931ebf700 1 -- start start 2024-05-09T00:04:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f5931ebf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c073340 0x7f592c1a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f5931ebf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c073c60 0x7f592c1a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f5931ebf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f592c111eb0 0x7f592c19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f5931ebf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c19dea0 con 0x7f592c073340 2024-05-09T00:04:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f5931ebf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c19e010 con 0x7f592c111eb0 2024-05-09T00:04:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f5931ebf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c19e1d0 con 0x7f592c073c60 2024-05-09T00:04:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f592b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c073340 0x7f592c1a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f592bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f592c111eb0 0x7f592c19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f592affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c073c60 0x7f592c1a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f592b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c073340 0x7f592c1a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51744/0 (socket says 172.21.15.12:51744) 2024-05-09T00:04:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f592b7fe700 1 -- 172.21.15.12:0/184749318 learned_addr learned my addr 172.21.15.12:0/184749318 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f592b7fe700 1 -- 172.21.15.12:0/184749318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c073c60 msgr2=0x7f592c1a3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f592b7fe700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c073c60 0x7f592c1a3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f592b7fe700 1 -- 172.21.15.12:0/184749318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f592c111eb0 msgr2=0x7f592c19d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f592b7fe700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f592c111eb0 0x7f592c19d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.673+0000 7f592b7fe700 1 -- 172.21.15.12:0/184749318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59200095f0 con 0x7f592c073340 2024-05-09T00:04:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.674+0000 7f592b7fe700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c073340 0x7f592c1a3700 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5920005910 tx=0x7f5920004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.674+0000 7f5928ff9700 1 -- 172.21.15.12:0/184749318 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592001d070 con 0x7f592c073340 2024-05-09T00:04:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.674+0000 7f5928ff9700 1 -- 172.21.15.12:0/184749318 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5920022470 con 0x7f592c073340 2024-05-09T00:04:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.674+0000 7f5928ff9700 1 -- 172.21.15.12:0/184749318 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920017650 con 0x7f592c073340 2024-05-09T00:04:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.674+0000 7f5931ebf700 1 -- 172.21.15.12:0/184749318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c19e450 con 0x7f592c073340 2024-05-09T00:04:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.674+0000 7f5931ebf700 1 -- 172.21.15.12:0/184749318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c1aa6f0 con 0x7f592c073340 2024-05-09T00:04:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.678+0000 7f593e079b80 1 Processor -- start 2024-05-09T00:04:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.679+0000 7f593e079b80 1 -- start start 2024-05-09T00:04:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.679+0000 7f593e079b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f043950 0x55ad5f043d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.679+0000 7f593e079b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f03df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.679+0000 7f593e079b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f03e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.679+0000 7f593e079b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5ef950f0 con 0x55ad5f043950 2024-05-09T00:04:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.679+0000 7f593e079b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5ef92ab0 con 0x55ad5f03e480 2024-05-09T00:04:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.679+0000 7f593e079b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5ef8e3a0 con 0x55ad5f044270 2024-05-09T00:04:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.680+0000 7f592affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f03e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.680+0000 7f592affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f03e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41382/0 (socket says 172.21.15.12:41382) 2024-05-09T00:04:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.680+0000 7f592affd700 1 -- 172.21.15.12:0/3643596923 learned_addr learned my addr 172.21.15.12:0/3643596923 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.680+0000 7f5928ff9700 1 -- 172.21.15.12:0/184749318 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f59200225e0 con 0x7f592c073340 2024-05-09T00:04:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.680+0000 7f592affd700 1 -- 172.21.15.12:0/3643596923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 msgr2=0x55ad5f03df40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:04:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.680+0000 7f592affd700 1 --2- 172.21.15.12:0/3643596923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f03df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.680+0000 7f592affd700 1 -- 172.21.15.12:0/3643596923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f043950 msgr2=0x55ad5f043d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.680+0000 7f592affd700 1 --2- 172.21.15.12:0/3643596923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f043950 0x55ad5f043d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.680+0000 7f592affd700 1 -- 172.21.15.12:0/3643596923 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad5ee0f350 con 0x55ad5f03e480 2024-05-09T00:04:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.681+0000 7f592affd700 1 --2- 172.21.15.12:0/3643596923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f03e8a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f591c009d20 tx=0x7f591c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82aad93eb2901924 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.681+0000 7f5911ffb700 1 -- 172.21.15.12:0/3643596923 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c019630 con 0x55ad5f03e480 2024-05-09T00:04:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.681+0000 7f5928ff9700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59140a1290 0x7f59140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.681+0000 7f5911ffb700 1 -- 172.21.15.12:0/3643596923 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f591c019c50 con 0x55ad5f03e480 2024-05-09T00:04:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.681+0000 7f5911ffb700 1 -- 172.21.15.12:0/3643596923 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c00ebc0 con 0x55ad5f03e480 2024-05-09T00:04:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.681+0000 7f592bfff700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59140a1290 0x7f59140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.681+0000 7f5928ff9700 1 -- 172.21.15.12:0/184749318 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f59200cdf00 con 0x7f592c073340 2024-05-09T00:04:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.681+0000 7f592bfff700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59140a1290 0x7f59140a3750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f592c1a4400 tx=0x7f5924009920 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.682+0000 7f593e079b80 1 -- 172.21.15.12:0/3643596923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 msgr2=0x55ad5f03e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.682+0000 7f593e079b80 1 --2- 172.21.15.12:0/3643596923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f03e8a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f591c009d20 tx=0x7f591c011720 comp rx=0 tx=0).stop 2024-05-09T00:04:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.683+0000 7f593e079b80 1 -- 172.21.15.12:0/3643596923 shutdown_connections 2024-05-09T00:04:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.683+0000 7f593e079b80 1 --2- 172.21.15.12:0/3643596923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f03e8a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.683+0000 7f593e079b80 1 --2- 172.21.15.12:0/3643596923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f043950 0x55ad5f043d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.683+0000 7f593e079b80 1 --2- 172.21.15.12:0/3643596923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f03df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.683+0000 7f593e079b80 1 -- 172.21.15.12:0/3643596923 >> 172.21.15.12:0/3643596923 conn(0x55ad5ef9c1b0 msgr2=0x55ad5f038840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.683+0000 7f593e079b80 1 -- 172.21.15.12:0/3643596923 shutdown_connections 2024-05-09T00:04:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.683+0000 7f593e079b80 1 -- 172.21.15.12:0/3643596923 wait complete. 2024-05-09T00:04:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.683+0000 7f593e079b80 1 -- start start 2024-05-09T00:04:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.684+0000 7f593e079b80 1 Processor -- start 2024-05-09T00:04:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.684+0000 7f593e079b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 0x55ad5f04a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.684+0000 7f593e079b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f043950 0x55ad5f0474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.684+0000 7f593e079b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f047ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.684+0000 7f593e079b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5ef2a500 con 0x55ad5f03e480 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.684+0000 7f593e079b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5ef2a1a0 con 0x55ad5f043950 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.684+0000 7f593e079b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5ee11640 con 0x55ad5f044270 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.684+0000 7f593e079b80 10 client.0 _ll_register_callbacks cb 0x55ad5ef33ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f043950 0x55ad5f0474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f047ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f047ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54158/0 (socket says 172.21.15.12:54158) 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592bfff700 1 -- 172.21.15.12:0/1327503194 learned_addr learned my addr 172.21.15.12:0/1327503194 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592b7fe700 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 0x55ad5f04a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592bfff700 1 -- 172.21.15.12:0/1327503194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f043950 msgr2=0x55ad5f0474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592bfff700 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f043950 0x55ad5f0474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592bfff700 1 -- 172.21.15.12:0/1327503194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 msgr2=0x55ad5f04a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592bfff700 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 0x55ad5f04a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592bfff700 1 -- 172.21.15.12:0/1327503194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c009980 con 0x55ad5f044270 2024-05-09T00:04:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.685+0000 7f592affd700 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f043950 0x55ad5f0474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:04:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.687+0000 7f592bfff700 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f047ba0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5924017e40 tx=0x7f5924004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b64e2538afb8afb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.687+0000 7f590affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.688+0000 7f590affd700 1 -- 172.21.15.12:0/1327503194 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592400e040 con 0x55ad5f044270 2024-05-09T00:04:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.688+0000 7f590affd700 1 -- 172.21.15.12:0/1327503194 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f592401c040 con 0x55ad5f044270 2024-05-09T00:04:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.689+0000 7f593e079b80 1 -- 172.21.15.12:0/1327503194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad5eee7210 con 0x55ad5f044270 2024-05-09T00:04:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.690+0000 7f590affd700 1 -- 172.21.15.12:0/1327503194 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5924015e30 con 0x55ad5f044270 2024-05-09T00:04:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.690+0000 7f593e079b80 10 client.30061 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.690+0000 7f593e079b80 1 -- 172.21.15.12:0/1327503194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55ad5edbbb00 con 0x55ad5f044270 2024-05-09T00:04:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.690+0000 7f590affd700 1 -- 172.21.15.12:0/1327503194 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad5eee7210 con 0x55ad5f044270 2024-05-09T00:04:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.690+0000 7f590affd700 1 -- 172.21.15.12:0/1327503194 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5924021650 con 0x55ad5f044270 2024-05-09T00:04:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.690+0000 7f593e079b80 10 client.30061 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.690+0000 7f593e079b80 10 client.30061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.691+0000 7f593e079b80 20 client.30061 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.691+0000 7f593e079b80 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad5f068d10 0x55ad5f06b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.691+0000 7f593e079b80 4 client.30061 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.691+0000 7f593e079b80 1 -- 172.21.15.12:0/1327503194 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad5f00d7b0 con 0x55ad5f068d10 2024-05-09T00:04:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.691+0000 7f592b7fe700 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad5f068d10 0x55ad5f06b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.692+0000 7f590affd700 10 client.30061 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.692+0000 7f592b7fe700 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad5f068d10 0x55ad5f06b130 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b88bd275e5200299 server_cookie=81edb7cc33c6d5c9 in_seq=0 out_seq=0 2024-05-09T00:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.693+0000 7f590affd700 1 -- 172.21.15.12:0/1327503194 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad5f00d7b0 con 0x55ad5f068d10 2024-05-09T00:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.693+0000 7f590affd700 10 client.30061 handle_command_reply: tid=0 2024-05-09T00:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.693+0000 7f593e079b80 1 client.30061 shutdown 2024-05-09T00:04:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 1 -- 172.21.15.12:0/1327503194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 msgr2=0x55ad5f047ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f047ba0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5924017e40 tx=0x7f5924004b40 comp rx=0 tx=0).stop 2024-05-09T00:04:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 1 -- 172.21.15.12:0/1327503194 shutdown_connections 2024-05-09T00:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f047ba0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 0x55ad5f04a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad5f068d10 0x55ad5f06b130 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 1 --2- 172.21.15.12:0/1327503194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f043950 0x55ad5f0474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 1 -- 172.21.15.12:0/1327503194 >> 172.21.15.12:0/1327503194 conn(0x55ad5ef9c1b0 msgr2=0x55ad5f0424a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f590affd700 0 client.30061 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 1 -- 172.21.15.12:0/1327503194 shutdown_connections 2024-05-09T00:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 1 -- 172.21.15.12:0/1327503194 wait complete. 2024-05-09T00:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.694+0000 7f593e079b80 20 client.30061 trim_cache size 0 max 16384 2024-05-09T00:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.714+0000 7f593e079b80 1 Processor -- start 2024-05-09T00:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f593e079b80 1 -- start start 2024-05-09T00:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f593e079b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 0x55ad5f038910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f593e079b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f043950 0x55ad5f049d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f593e079b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f044270 0x55ad5f064730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f593e079b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f039930 con 0x55ad5f03e480 2024-05-09T00:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f593e079b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f03a4b0 con 0x55ad5f044270 2024-05-09T00:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f593e079b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f03a290 con 0x55ad5f043950 2024-05-09T00:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f592b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 0x55ad5f038910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f592b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 0x55ad5f038910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51764/0 (socket says 172.21.15.12:51764) 2024-05-09T00:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f592b7fe700 1 -- 172.21.15.12:0/2755706579 learned_addr learned my addr 172.21.15.12:0/2755706579 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.715+0000 7f592affd700 1 --2- 172.21.15.12:0/2755706579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f043950 0x55ad5f049d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.716+0000 7f592b7fe700 1 -- 172.21.15.12:0/2755706579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f043950 msgr2=0x55ad5f049d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.716+0000 7f592b7fe700 1 --2- 172.21.15.12:0/2755706579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f043950 0x55ad5f049d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.716+0000 7f592b7fe700 1 -- 172.21.15.12:0/2755706579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f044270 msgr2=0x55ad5f064730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.716+0000 7f592b7fe700 1 --2- 172.21.15.12:0/2755706579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f044270 0x55ad5f064730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.716+0000 7f592b7fe700 1 -- 172.21.15.12:0/2755706579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad5ef54500 con 0x55ad5f03e480 2024-05-09T00:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.716+0000 7f592b7fe700 1 --2- 172.21.15.12:0/2755706579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 0x55ad5f038910 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f592009b190 tx=0x7f59200a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b490560a0f1c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.716+0000 7f59127fc700 1 -- 172.21.15.12:0/2755706579 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592009ad40 con 0x55ad5f03e480 2024-05-09T00:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.716+0000 7f59127fc700 1 -- 172.21.15.12:0/2755706579 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59200b2070 con 0x55ad5f03e480 2024-05-09T00:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.716+0000 7f59127fc700 1 -- 172.21.15.12:0/2755706579 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920066c90 con 0x55ad5f03e480 2024-05-09T00:04:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.717+0000 7f593e079b80 1 -- 172.21.15.12:0/2755706579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 msgr2=0x55ad5f038910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.717+0000 7f593e079b80 1 --2- 172.21.15.12:0/2755706579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 0x55ad5f038910 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f592009b190 tx=0x7f59200a1040 comp rx=0 tx=0).stop 2024-05-09T00:04:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.718+0000 7f593e079b80 1 -- 172.21.15.12:0/2755706579 shutdown_connections 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.718+0000 7f593e079b80 1 --2- 172.21.15.12:0/2755706579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f044270 0x55ad5f064730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.718+0000 7f593e079b80 1 --2- 172.21.15.12:0/2755706579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f03e480 0x55ad5f038910 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.718+0000 7f593e079b80 1 --2- 172.21.15.12:0/2755706579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f043950 0x55ad5f049d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.718+0000 7f593e079b80 1 -- 172.21.15.12:0/2755706579 >> 172.21.15.12:0/2755706579 conn(0x55ad5ef9c1b0 msgr2=0x55ad5f03c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.720+0000 7f593e079b80 1 -- 172.21.15.12:0/2755706579 shutdown_connections 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.720+0000 7f593e079b80 1 -- 172.21.15.12:0/2755706579 wait complete. 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.720+0000 7f593e079b80 1 -- start start 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.720+0000 7f593e079b80 1 Processor -- start 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f593e079b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f05fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f593e079b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f043950 0x55ad5f0601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f593e079b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f046960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f593e079b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f047000 con 0x55ad5f043950 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f593e079b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f047170 con 0x55ad5f03e480 2024-05-09T00:04:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f593e079b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f047330 con 0x55ad5f044270 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f593e079b80 10 client.0 _ll_register_callbacks cb 0x55ad5ef33ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f592b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f05fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f592b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f05fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41402/0 (socket says 172.21.15.12:41402) 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f592b7fe700 1 -- 172.21.15.12:0/4127019646 learned_addr learned my addr 172.21.15.12:0/4127019646 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f592affd700 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f043950 0x55ad5f0601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f592b7fe700 1 -- 172.21.15.12:0/4127019646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 msgr2=0x55ad5f046960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f592b7fe700 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f046960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f592b7fe700 1 -- 172.21.15.12:0/4127019646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f043950 msgr2=0x55ad5f0601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f592b7fe700 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f043950 0x55ad5f0601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.721+0000 7f592b7fe700 1 -- 172.21.15.12:0/4127019646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c1aa6f0 con 0x55ad5f03e480 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.722+0000 7f592b7fe700 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f05fc80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5920022c40 tx=0x7f592009adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42577f44736be04d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.722+0000 7f590affd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:04:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.722+0000 7f590affd700 1 -- 172.21.15.12:0/4127019646 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592007e070 con 0x55ad5f03e480 2024-05-09T00:04:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.722+0000 7f590affd700 1 -- 172.21.15.12:0/4127019646 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f592009b900 con 0x55ad5f03e480 2024-05-09T00:04:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.722+0000 7f593e079b80 1 -- 172.21.15.12:0/4127019646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f591c019c50 con 0x55ad5f03e480 2024-05-09T00:04:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.722+0000 7f590affd700 1 -- 172.21.15.12:0/4127019646 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59200a35f0 con 0x55ad5f03e480 2024-05-09T00:04:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.722+0000 7f593e079b80 10 client.29966 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.722+0000 7f593e079b80 1 -- 172.21.15.12:0/4127019646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55ad5ee13c70 con 0x55ad5f03e480 2024-05-09T00:04:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.723+0000 7f590affd700 1 -- 172.21.15.12:0/4127019646 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f591c019c50 con 0x55ad5f03e480 2024-05-09T00:04:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.723+0000 7f590affd700 1 -- 172.21.15.12:0/4127019646 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5920081030 con 0x55ad5f03e480 2024-05-09T00:04:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.723+0000 7f593e079b80 10 client.29966 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.723+0000 7f593e079b80 10 client.29966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.723+0000 7f593e079b80 20 client.29966 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.723+0000 7f593e079b80 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad5f0cfeb0 0x55ad5f0d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.723+0000 7f593e079b80 4 client.29966 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:04:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.723+0000 7f593e079b80 1 -- 172.21.15.12:0/4127019646 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ad5ef86e60 con 0x55ad5f0cfeb0 2024-05-09T00:04:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.723+0000 7f592affd700 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad5f0cfeb0 0x55ad5f0d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.726+0000 7f590affd700 10 client.29966 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.726+0000 7f592affd700 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad5f0cfeb0 0x55ad5f0d22a0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea44e6a2fa1fb07 server_cookie=b181740b0dcfef72 in_seq=0 out_seq=0 2024-05-09T00:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.949+0000 7f590affd700 1 -- 172.21.15.12:0/4127019646 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ad5ef86e60 con 0x55ad5f0cfeb0 2024-05-09T00:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.950+0000 7f590affd700 10 client.29966 handle_command_reply: tid=0 2024-05-09T00:04:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.950+0000 7f593e079b80 1 client.29966 shutdown 2024-05-09T00:04:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.951+0000 7f593e079b80 1 -- 172.21.15.12:0/4127019646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 msgr2=0x55ad5f05fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.951+0000 7f593e079b80 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f05fc80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5920022c40 tx=0x7f592009adf0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.951+0000 7f593e079b80 1 -- 172.21.15.12:0/4127019646 shutdown_connections 2024-05-09T00:04:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.951+0000 7f593e079b80 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad5f044270 0x55ad5f046960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.951+0000 7f593e079b80 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad5f0cfeb0 0x55ad5f0d22a0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.951+0000 7f593e079b80 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad5f03e480 0x55ad5f05fc80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.951+0000 7f593e079b80 1 --2- 172.21.15.12:0/4127019646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad5f043950 0x55ad5f0601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.951+0000 7f593e079b80 1 -- 172.21.15.12:0/4127019646 >> 172.21.15.12:0/4127019646 conn(0x55ad5ef9c1b0 msgr2=0x55ad5f064730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.951+0000 7f590affd700 0 client.29966 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.953+0000 7f593e079b80 1 -- 172.21.15.12:0/4127019646 shutdown_connections 2024-05-09T00:04:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.953+0000 7f593e079b80 1 -- 172.21.15.12:0/4127019646 wait complete. 2024-05-09T00:04:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.954+0000 7f593e079b80 20 client.29966 trim_cache size 0 max 16384 2024-05-09T00:04:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:04:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:04:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:04:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:04:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.958+0000 7f5931ebf700 1 -- 172.21.15.12:0/184749318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59140a1290 msgr2=0x7f59140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.958+0000 7f5931ebf700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59140a1290 0x7f59140a3750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f592c1a4400 tx=0x7f5924009920 comp rx=0 tx=0).stop 2024-05-09T00:04:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.958+0000 7f5931ebf700 1 -- 172.21.15.12:0/184749318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c073340 msgr2=0x7f592c1a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.958+0000 7f5931ebf700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c073340 0x7f592c1a3700 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5920005910 tx=0x7f5920004730 comp rx=0 tx=0).stop 2024-05-09T00:04:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.959+0000 7f5931ebf700 1 -- 172.21.15.12:0/184749318 shutdown_connections 2024-05-09T00:04:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.959+0000 7f5931ebf700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59140a1290 0x7f59140a3750 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.959+0000 7f5931ebf700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f592c111eb0 0x7f592c19d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.959+0000 7f5931ebf700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c073340 0x7f592c1a3700 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.959+0000 7f5931ebf700 1 --2- 172.21.15.12:0/184749318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c073c60 0x7f592c1a3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.959+0000 7f5931ebf700 1 -- 172.21.15.12:0/184749318 >> 172.21.15.12:0/184749318 conn(0x7f592c0fc970 msgr2=0x7f592c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.959+0000 7f5931ebf700 1 -- 172.21.15.12:0/184749318 shutdown_connections 2024-05-09T00:04:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:02.962+0000 7f5931ebf700 1 -- 172.21.15.12:0/184749318 wait complete. 2024-05-09T00:04:03.249 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170470 38.10 MB/sec execute 118 sec latency 259.997 ms 2024-05-09T00:04:04.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170781 37.94 MB/sec execute 119 sec latency 296.656 ms 2024-05-09T00:04:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:04 smithi204 ceph-mon[47373]: osdmap e121: 12 total, 12 up, 12 in 2024-05-09T00:04:04.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:04 smithi204 ceph-mon[47373]: pgmap v569: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-05-09T00:04:04.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:04 smithi160 ceph-mon[46152]: osdmap e121: 12 total, 12 up, 12 in 2024-05-09T00:04:04.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:04 smithi160 ceph-mon[46152]: pgmap v569: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-05-09T00:04:04.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:04 smithi012 ceph-mon[42503]: osdmap e121: 12 total, 12 up, 12 in 2024-05-09T00:04:04.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:04 smithi012 ceph-mon[42503]: pgmap v569: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-05-09T00:04:05.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 171188 37.85 MB/sec execute 120 sec latency 322.488 ms 2024-05-09T00:04:05.365 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:04:05.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.596+0000 7f845fc41700 1 -- 172.21.15.12:0/850569061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8458107ef0 msgr2=0x7f84581143b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.596+0000 7f845fc41700 1 --2- 172.21.15.12:0/850569061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8458107ef0 0x7f84581143b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f8454009920 tx=0x7f8454009c30 comp rx=0 tx=0).stop 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.598+0000 7f845fc41700 1 -- 172.21.15.12:0/850569061 shutdown_connections 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.598+0000 7f845fc41700 1 --2- 172.21.15.12:0/850569061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8458107ef0 0x7f84581143b0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.598+0000 7f845fc41700 1 --2- 172.21.15.12:0/850569061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f845810d5f0 0x7f845810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.598+0000 7f845fc41700 1 --2- 172.21.15.12:0/850569061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8458107530 0x7f84581079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.598+0000 7f845fc41700 1 -- 172.21.15.12:0/850569061 >> 172.21.15.12:0/850569061 conn(0x7f8458075490 msgr2=0x7f84580778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.598+0000 7f845fc41700 1 -- 172.21.15.12:0/850569061 shutdown_connections 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.598+0000 7f845fc41700 1 -- 172.21.15.12:0/850569061 wait complete. 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.599+0000 7f845fc41700 1 Processor -- start 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845fc41700 1 -- start start 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845fc41700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8458107530 0x7f84580725e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845fc41700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8458107ef0 0x7f8458072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845fc41700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845810d5f0 0x7f845806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845fc41700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845806bf80 con 0x7f8458107530 2024-05-09T00:04:05.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845fc41700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845806c0f0 con 0x7f8458107ef0 2024-05-09T00:04:05.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845fc41700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845806c2b0 con 0x7f845810d5f0 2024-05-09T00:04:05.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845d9dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8458107530 0x7f84580725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845d9dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8458107530 0x7f84580725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51796/0 (socket says 172.21.15.12:51796) 2024-05-09T00:04:05.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845d9dd700 1 -- 172.21.15.12:0/3501314358 learned_addr learned my addr 172.21.15.12:0/3501314358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:05.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.600+0000 7f845e1de700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845810d5f0 0x7f845806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.601+0000 7f845d1dc700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8458107ef0 0x7f8458072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.601+0000 7f845d9dd700 1 -- 172.21.15.12:0/3501314358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845810d5f0 msgr2=0x7f845806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.601+0000 7f845d9dd700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845810d5f0 0x7f845806b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.601+0000 7f845d9dd700 1 -- 172.21.15.12:0/3501314358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8458107ef0 msgr2=0x7f8458072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.601+0000 7f845d9dd700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8458107ef0 0x7f8458072b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.601+0000 7f845d9dd700 1 -- 172.21.15.12:0/3501314358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8450009d00 con 0x7f8458107530 2024-05-09T00:04:05.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.601+0000 7f845d9dd700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8458107530 0x7f84580725e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f844800b550 tx=0x7f844800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:05.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.602+0000 7f844effd700 1 -- 172.21.15.12:0/3501314358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448011830 con 0x7f8458107530 2024-05-09T00:04:05.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.602+0000 7f844effd700 1 -- 172.21.15.12:0/3501314358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8448011e50 con 0x7f8458107530 2024-05-09T00:04:05.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.602+0000 7f844effd700 1 -- 172.21.15.12:0/3501314358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844800f2e0 con 0x7f8458107530 2024-05-09T00:04:05.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.602+0000 7f845fc41700 1 -- 172.21.15.12:0/3501314358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8454009520 con 0x7f8458107530 2024-05-09T00:04:05.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.602+0000 7f845fc41700 1 -- 172.21.15.12:0/3501314358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84580703b0 con 0x7f8458107530 2024-05-09T00:04:05.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.607+0000 7f846bdfbb80 1 Processor -- start 2024-05-09T00:04:05.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.607+0000 7f844effd700 1 -- 172.21.15.12:0/3501314358 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f844800f440 con 0x7f8458107530 2024-05-09T00:04:05.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.608+0000 7f846bdfbb80 1 -- start start 2024-05-09T00:04:05.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.608+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc0e00 0x5623e8fc11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.608+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 0x5623e8fbb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.608+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.608+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e8f12150 con 0x5623e8fbb890 2024-05-09T00:04:05.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.608+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e8f0fb10 con 0x5623e8fc0e00 2024-05-09T00:04:05.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.608+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e8f0b400 con 0x5623e8fc1720 2024-05-09T00:04:05.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.609+0000 7f845d1dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.609+0000 7f845d1dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51804/0 (socket says 172.21.15.12:51804) 2024-05-09T00:04:05.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.609+0000 7f845d1dc700 1 -- 172.21.15.12:0/3583330691 learned_addr learned my addr 172.21.15.12:0/3583330691 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:05.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.609+0000 7f844effd700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f84440a13d0 0x7f84440a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.609+0000 7f844effd700 1 -- 172.21.15.12:0/3501314358 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f84480cc9c0 con 0x7f8458107530 2024-05-09T00:04:05.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.609+0000 7f845d1dc700 1 -- 172.21.15.12:0/3583330691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 msgr2=0x5623e8fbb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:05.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.609+0000 7f845d1dc700 1 --2- 172.21.15.12:0/3583330691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 0x5623e8fbb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.609+0000 7f845d1dc700 1 -- 172.21.15.12:0/3583330691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc0e00 msgr2=0x5623e8fc11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:05.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.609+0000 7f845d1dc700 1 --2- 172.21.15.12:0/3583330691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc0e00 0x5623e8fc11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.609+0000 7f845d1dc700 1 -- 172.21.15.12:0/3583330691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e8d8c3b0 con 0x5623e8fbb890 2024-05-09T00:04:05.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.610+0000 7f845d1dc700 1 --2- 172.21.15.12:0/3583330691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbbcd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8450009520 tx=0x7f84500099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86577a97993c1f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:05.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.612+0000 7f8443fff700 1 -- 172.21.15.12:0/3583330691 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845000e070 con 0x5623e8fbb890 2024-05-09T00:04:05.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.612+0000 7f8443fff700 1 -- 172.21.15.12:0/3583330691 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f845000f040 con 0x5623e8fbb890 2024-05-09T00:04:05.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.612+0000 7f8443fff700 1 -- 172.21.15.12:0/3583330691 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450014b50 con 0x5623e8fbb890 2024-05-09T00:04:05.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.612+0000 7f845e1de700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f84440a13d0 0x7f84440a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.613+0000 7f846bdfbb80 1 -- 172.21.15.12:0/3583330691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 msgr2=0x5623e8fbbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.613+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/3583330691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbbcd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8450009520 tx=0x7f84500099e0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.613+0000 7f845e1de700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f84440a13d0 0x7f84440a3890 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f84540098f0 tx=0x7f8454019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:05.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.615+0000 7f846bdfbb80 1 -- 172.21.15.12:0/3583330691 shutdown_connections 2024-05-09T00:04:05.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.615+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/3583330691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbbcd0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.615+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/3583330691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc0e00 0x5623e8fc11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.615+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/3583330691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 0x5623e8fbb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.615+0000 7f846bdfbb80 1 -- 172.21.15.12:0/3583330691 >> 172.21.15.12:0/3583330691 conn(0x5623e8f19210 msgr2=0x5623e8fb5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:05.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.619+0000 7f846bdfbb80 1 -- 172.21.15.12:0/3583330691 shutdown_connections 2024-05-09T00:04:05.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.619+0000 7f846bdfbb80 1 -- 172.21.15.12:0/3583330691 wait complete. 2024-05-09T00:04:05.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.619+0000 7f846bdfbb80 1 -- start start 2024-05-09T00:04:05.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.619+0000 7f846bdfbb80 1 Processor -- start 2024-05-09T00:04:05.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.622+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fbb890 0x5623e8fc77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.622+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 0x5623e8fc4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.622+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc1720 0x5623e8fc5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.622+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e8ea7560 con 0x5623e8fc0e00 2024-05-09T00:04:05.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.622+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e8ea7200 con 0x5623e8fc1720 2024-05-09T00:04:05.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.622+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e8d8e6a0 con 0x5623e8fbb890 2024-05-09T00:04:05.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.622+0000 7f846bdfbb80 10 client.0 _ll_register_callbacks cb 0x5623e8eb0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:05.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.622+0000 7f845d1dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 0x5623e8fc4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.622+0000 7f845d1dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 0x5623e8fc4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51820/0 (socket says 172.21.15.12:51820) 2024-05-09T00:04:05.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.622+0000 7f845d1dc700 1 -- 172.21.15.12:0/1209353230 learned_addr learned my addr 172.21.15.12:0/1209353230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:05.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.623+0000 7f845d1dc700 1 -- 172.21.15.12:0/1209353230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fbb890 msgr2=0x5623e8fc77e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:04:05.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.623+0000 7f845d1dc700 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fbb890 0x5623e8fc77e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.623+0000 7f845d1dc700 1 -- 172.21.15.12:0/1209353230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc1720 msgr2=0x5623e8fc5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:05.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.623+0000 7f845d1dc700 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc1720 0x5623e8fc5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.623+0000 7f845d1dc700 1 -- 172.21.15.12:0/1209353230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8450009200 con 0x5623e8fc0e00 2024-05-09T00:04:05.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.625+0000 7f845d1dc700 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 0x5623e8fc4ad0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f84500145d0 tx=0x7f84500060c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa8e21e80895217 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:05.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.628+0000 7f84417fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:05.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.628+0000 7f84417fa700 1 -- 172.21.15.12:0/1209353230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845000e040 con 0x5623e8fc0e00 2024-05-09T00:04:05.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.628+0000 7f84417fa700 1 -- 172.21.15.12:0/1209353230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8450014a90 con 0x5623e8fc0e00 2024-05-09T00:04:05.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.628+0000 7f846bdfbb80 1 -- 172.21.15.12:0/1209353230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623e8e64270 con 0x5623e8fc0e00 2024-05-09T00:04:05.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.629+0000 7f84417fa700 1 -- 172.21.15.12:0/1209353230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8450014c50 con 0x5623e8fc0e00 2024-05-09T00:04:05.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.629+0000 7f846bdfbb80 10 client.20337 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:05.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.629+0000 7f846bdfbb80 1 -- 172.21.15.12:0/1209353230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5623e8d38b60 con 0x5623e8fc0e00 2024-05-09T00:04:05.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.631+0000 7f84417fa700 1 -- 172.21.15.12:0/1209353230 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623e8e64270 con 0x5623e8fc0e00 2024-05-09T00:04:05.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.631+0000 7f84417fa700 1 -- 172.21.15.12:0/1209353230 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8450004240 con 0x5623e8fc0e00 2024-05-09T00:04:05.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.631+0000 7f846bdfbb80 10 client.20337 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:05.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.631+0000 7f846bdfbb80 10 client.20337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:05.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.631+0000 7f846bdfbb80 20 client.20337 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:05.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.631+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5623e8fd50e0 0x5623e8fd7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.631+0000 7f846bdfbb80 4 client.20337 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:05.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.631+0000 7f846bdfbb80 1 -- 172.21.15.12:0/1209353230 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623e8f8a810 con 0x5623e8fd50e0 2024-05-09T00:04:05.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.632+0000 7f845d9dd700 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5623e8fd50e0 0x5623e8fd7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.632+0000 7f845d9dd700 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5623e8fd50e0 0x5623e8fd7500 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d797e7728d5bec3b server_cookie=fa3f053715e679b9 in_seq=0 out_seq=0 2024-05-09T00:04:05.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.637+0000 7f84417fa700 10 client.20337 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:05.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.637+0000 7f84417fa700 1 -- 172.21.15.12:0/1209353230 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623e8f8a810 con 0x5623e8fd50e0 2024-05-09T00:04:05.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.637+0000 7f84417fa700 10 client.20337 handle_command_reply: tid=0 2024-05-09T00:04:05.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.637+0000 7f846bdfbb80 1 client.20337 shutdown 2024-05-09T00:04:05.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.648+0000 7f846bdfbb80 1 -- 172.21.15.12:0/1209353230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 msgr2=0x5623e8fc4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.648+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 0x5623e8fc4ad0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f84500145d0 tx=0x7f84500060c0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.654+0000 7f846bdfbb80 1 -- 172.21.15.12:0/1209353230 shutdown_connections 2024-05-09T00:04:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.654+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5623e8fd50e0 0x5623e8fd7500 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.654+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc1720 0x5623e8fc5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.654+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fbb890 0x5623e8fc77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.654+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/1209353230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 0x5623e8fc4ad0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.654+0000 7f846bdfbb80 1 -- 172.21.15.12:0/1209353230 >> 172.21.15.12:0/1209353230 conn(0x5623e8f19210 msgr2=0x5623e8fb55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:05.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.662+0000 7f84417fa700 0 client.20337 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:05.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.662+0000 7f846bdfbb80 1 -- 172.21.15.12:0/1209353230 shutdown_connections 2024-05-09T00:04:05.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.663+0000 7f846bdfbb80 1 -- 172.21.15.12:0/1209353230 wait complete. 2024-05-09T00:04:05.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.664+0000 7f846bdfbb80 20 client.20337 trim_cache size 0 max 16384 2024-05-09T00:04:05.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.693+0000 7f846bdfbb80 1 Processor -- start 2024-05-09T00:04:05.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.693+0000 7f846bdfbb80 1 -- start start 2024-05-09T00:04:05.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.694+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.694+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc0e00 0x5623e8fb5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.694+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 0x5623e8fbec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.694+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e8fae070 con 0x5623e8fbb890 2024-05-09T00:04:05.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.694+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e8fb2df0 con 0x5623e8fc0e00 2024-05-09T00:04:05.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.694+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e8fc5aa0 con 0x5623e8fc1720 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.694+0000 7f845d9dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.694+0000 7f845d9dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51830/0 (socket says 172.21.15.12:51830) 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.695+0000 7f845d9dd700 1 -- 172.21.15.12:0/3152768397 learned_addr learned my addr 172.21.15.12:0/3152768397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.695+0000 7f845d9dd700 1 -- 172.21.15.12:0/3152768397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 msgr2=0x5623e8fbec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.695+0000 7f845d9dd700 1 --2- 172.21.15.12:0/3152768397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 0x5623e8fbec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.695+0000 7f845d9dd700 1 -- 172.21.15.12:0/3152768397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc0e00 msgr2=0x5623e8fb5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.695+0000 7f845d9dd700 1 --2- 172.21.15.12:0/3152768397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc0e00 0x5623e8fb5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.695+0000 7f845d9dd700 1 -- 172.21.15.12:0/3152768397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e8ed1560 con 0x5623e8fbb890 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.695+0000 7f845d9dd700 1 --2- 172.21.15.12:0/3152768397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbb350 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f8448010b20 tx=0x7f8448004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ca5e992ee9a7fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.696+0000 7f844cda4700 1 -- 172.21.15.12:0/3152768397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84480a1ff0 con 0x5623e8fbb890 2024-05-09T00:04:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.696+0000 7f844cda4700 1 -- 172.21.15.12:0/3152768397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8448097f10 con 0x5623e8fbb890 2024-05-09T00:04:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.696+0000 7f844cda4700 1 -- 172.21.15.12:0/3152768397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448099dd0 con 0x5623e8fbb890 2024-05-09T00:04:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 -- 172.21.15.12:0/3152768397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 msgr2=0x5623e8fbb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/3152768397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbb350 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f8448010b20 tx=0x7f8448004790 comp rx=0 tx=0).stop 2024-05-09T00:04:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 -- 172.21.15.12:0/3152768397 shutdown_connections 2024-05-09T00:04:05.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/3152768397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 0x5623e8fbec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/3152768397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fbb890 0x5623e8fbb350 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/3152768397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fc0e00 0x5623e8fb5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 -- 172.21.15.12:0/3152768397 >> 172.21.15.12:0/3152768397 conn(0x5623e8f19210 msgr2=0x5623e8fdbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:05.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 -- 172.21.15.12:0/3152768397 shutdown_connections 2024-05-09T00:04:05.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 -- 172.21.15.12:0/3152768397 wait complete. 2024-05-09T00:04:05.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 -- start start 2024-05-09T00:04:05.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.697+0000 7f846bdfbb80 1 Processor -- start 2024-05-09T00:04:05.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.698+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fbb890 0x5623e8fcc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.698+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 0x5623e8fca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.698+0000 7f846bdfbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 0x5623e8fca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.698+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e8fcafb0 con 0x5623e8fc0e00 2024-05-09T00:04:05.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.698+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e902a5c0 con 0x5623e8fbb890 2024-05-09T00:04:05.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.698+0000 7f846bdfbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e902a700 con 0x5623e8fc1720 2024-05-09T00:04:05.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.698+0000 7f846bdfbb80 10 client.0 _ll_register_callbacks cb 0x5623e8eb0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:05.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.698+0000 7f845d9dd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fbb890 0x5623e8fcc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.698+0000 7f845d9dd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fbb890 0x5623e8fcc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41436/0 (socket says 172.21.15.12:41436) 2024-05-09T00:04:05.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.698+0000 7f845d9dd700 1 -- 172.21.15.12:0/4102322524 learned_addr learned my addr 172.21.15.12:0/4102322524 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:05.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.699+0000 7f845d9dd700 1 -- 172.21.15.12:0/4102322524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 msgr2=0x5623e8fca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:05.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.699+0000 7f845d9dd700 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 0x5623e8fca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.699+0000 7f845d9dd700 1 -- 172.21.15.12:0/4102322524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 msgr2=0x5623e8fca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:05.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.699+0000 7f845d9dd700 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 0x5623e8fca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.699+0000 7f845d9dd700 1 -- 172.21.15.12:0/4102322524 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84480b9040 con 0x5623e8fbb890 2024-05-09T00:04:05.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.699+0000 7f845d9dd700 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fbb890 0x5623e8fcc6d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8448099760 tx=0x7f8448099580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8d70ce019c5413 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:05.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.699+0000 7f84417fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:04:05.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.700+0000 7f84417fa700 1 -- 172.21.15.12:0/4102322524 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448011c30 con 0x5623e8fbb890 2024-05-09T00:04:05.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.700+0000 7f84417fa700 1 -- 172.21.15.12:0/4102322524 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8448099060 con 0x5623e8fbb890 2024-05-09T00:04:05.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.700+0000 7f846bdfbb80 1 -- 172.21.15.12:0/4102322524 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f845000f040 con 0x5623e8fbb890 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.700+0000 7f84417fa700 1 -- 172.21.15.12:0/4102322524 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8448099220 con 0x5623e8fbb890 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.700+0000 7f846bdfbb80 10 client.29972 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.700+0000 7f846bdfbb80 1 -- 172.21.15.12:0/4102322524 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5623e8d90cd0 con 0x5623e8fbb890 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.700+0000 7f84417fa700 1 -- 172.21.15.12:0/4102322524 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8448088020 con 0x5623e8fbb890 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.700+0000 7f84417fa700 1 -- 172.21.15.12:0/4102322524 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f84480b2260 con 0x5623e8fbb890 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.701+0000 7f846bdfbb80 10 client.29972 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.701+0000 7f846bdfbb80 10 client.29972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.701+0000 7f846bdfbb80 20 client.29972 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.701+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5623e902b970 0x5623e902dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.701+0000 7f846bdfbb80 4 client.29972 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:04:05.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.701+0000 7f846bdfbb80 1 -- 172.21.15.12:0/4102322524 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5623e8f03ec0 con 0x5623e902b970 2024-05-09T00:04:05.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.703+0000 7f845d1dc700 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5623e902b970 0x5623e902dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.710+0000 7f84417fa700 10 client.29972 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:05.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.710+0000 7f845d1dc700 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5623e902b970 0x5623e902dd90 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102a291d3f3a0b9d server_cookie=a245f238b32da179 in_seq=0 out_seq=0 2024-05-09T00:04:05.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.716+0000 7f84417fa700 1 -- 172.21.15.12:0/4102322524 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5623e8f03ec0 con 0x5623e902b970 2024-05-09T00:04:05.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.716+0000 7f84417fa700 10 client.29972 handle_command_reply: tid=0 2024-05-09T00:04:05.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.716+0000 7f846bdfbb80 1 client.29972 shutdown 2024-05-09T00:04:05.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.717+0000 7f846bdfbb80 1 -- 172.21.15.12:0/4102322524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fbb890 msgr2=0x5623e8fcc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.717+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fbb890 0x5623e8fcc6d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8448099760 tx=0x7f8448099580 comp rx=0 tx=0).stop 2024-05-09T00:04:05.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.717+0000 7f846bdfbb80 1 -- 172.21.15.12:0/4102322524 shutdown_connections 2024-05-09T00:04:05.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.717+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e8fc1720 0x5623e8fca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.717+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5623e902b970 0x5623e902dd90 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.717+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5623e8fbb890 0x5623e8fcc6d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.717+0000 7f846bdfbb80 1 --2- 172.21.15.12:0/4102322524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623e8fc0e00 0x5623e8fca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.717+0000 7f846bdfbb80 1 -- 172.21.15.12:0/4102322524 >> 172.21.15.12:0/4102322524 conn(0x5623e8f19210 msgr2=0x5623e8fbec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:05.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.717+0000 7f84417fa700 0 client.29972 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:05.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.717+0000 7f846bdfbb80 1 -- 172.21.15.12:0/4102322524 shutdown_connections 2024-05-09T00:04:05.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.718+0000 7f846bdfbb80 1 -- 172.21.15.12:0/4102322524 wait complete. 2024-05-09T00:04:05.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.718+0000 7f846bdfbb80 20 client.29972 trim_cache size 0 max 16384 2024-05-09T00:04:05.713 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:04:05.713 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:04:05.713 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:04:05.713 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:04:05.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.719+0000 7f845fc41700 1 -- 172.21.15.12:0/3501314358 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f84440a13d0 msgr2=0x7f84440a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.719+0000 7f845fc41700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f84440a13d0 0x7f84440a3890 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f84540098f0 tx=0x7f8454019040 comp rx=0 tx=0).stop 2024-05-09T00:04:05.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.719+0000 7f845fc41700 1 -- 172.21.15.12:0/3501314358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8458107530 msgr2=0x7f84580725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.719+0000 7f845fc41700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8458107530 0x7f84580725e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f844800b550 tx=0x7f844800b860 comp rx=0 tx=0).stop 2024-05-09T00:04:05.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.720+0000 7f845fc41700 1 -- 172.21.15.12:0/3501314358 shutdown_connections 2024-05-09T00:04:05.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.720+0000 7f845fc41700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f84440a13d0 0x7f84440a3890 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.720+0000 7f845fc41700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845810d5f0 0x7f845806b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.720+0000 7f845fc41700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8458107530 0x7f84580725e0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.720+0000 7f845fc41700 1 --2- 172.21.15.12:0/3501314358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8458107ef0 0x7f8458072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.720+0000 7f845fc41700 1 -- 172.21.15.12:0/3501314358 >> 172.21.15.12:0/3501314358 conn(0x7f8458075490 msgr2=0x7f8458076c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:05.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.720+0000 7f845fc41700 1 -- 172.21.15.12:0/3501314358 shutdown_connections 2024-05-09T00:04:05.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.720+0000 7f845fc41700 1 -- 172.21.15.12:0/3501314358 wait complete. 2024-05-09T00:04:05.736 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:04:05.736 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:04:05.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.964+0000 7f7efb75a700 1 -- 172.21.15.12:0/1149502623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef41082e0 msgr2=0x7f7ef410c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.964+0000 7f7efb75a700 1 --2- 172.21.15.12:0/1149502623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef41082e0 0x7f7ef410c0b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009990 tx=0x7f7ee4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.966+0000 7f7efb75a700 1 -- 172.21.15.12:0/1149502623 shutdown_connections 2024-05-09T00:04:05.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.966+0000 7f7efb75a700 1 --2- 172.21.15.12:0/1149502623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ef410c790 0x7f7ef4114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.966+0000 7f7efb75a700 1 --2- 172.21.15.12:0/1149502623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ef4107930 0x7f7ef4107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.966+0000 7f7efb75a700 1 --2- 172.21.15.12:0/1149502623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef41082e0 0x7f7ef410c0b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.966+0000 7f7efb75a700 1 -- 172.21.15.12:0/1149502623 >> 172.21.15.12:0/1149502623 conn(0x7f7ef4076e40 msgr2=0x7f7ef4105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:05.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.968+0000 7f7efb75a700 1 -- 172.21.15.12:0/1149502623 shutdown_connections 2024-05-09T00:04:05.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.968+0000 7f7efb75a700 1 -- 172.21.15.12:0/1149502623 wait complete. 2024-05-09T00:04:05.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.969+0000 7f7efb75a700 1 Processor -- start 2024-05-09T00:04:05.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.970+0000 7f7efb75a700 1 -- start start 2024-05-09T00:04:05.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.971+0000 7f7efb75a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef4107930 0x7f7ef40814a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.971+0000 7f7efb75a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ef41082e0 0x7f7ef40819e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.971+0000 7f7efb75a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ef410c790 0x7f7ef407b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.971+0000 7f7efb75a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef407be40 con 0x7f7ef410c790 2024-05-09T00:04:05.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.971+0000 7f7efb75a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef407bfb0 con 0x7f7ef41082e0 2024-05-09T00:04:05.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.971+0000 7f7efb75a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef407c170 con 0x7f7ef4107930 2024-05-09T00:04:05.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.971+0000 7f7ef9cf7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ef410c790 0x7f7ef407b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.971+0000 7f7ef9cf7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ef410c790 0x7f7ef407b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51866/0 (socket says 172.21.15.12:51866) 2024-05-09T00:04:05.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.972+0000 7f7ef9cf7700 1 -- 172.21.15.12:0/2133122661 learned_addr learned my addr 172.21.15.12:0/2133122661 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:05.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.972+0000 7f7ef9cf7700 1 -- 172.21.15.12:0/2133122661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef4107930 msgr2=0x7f7ef40814a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:04:05.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.972+0000 7f7ef9cf7700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef4107930 0x7f7ef40814a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.972+0000 7f7ef9cf7700 1 -- 172.21.15.12:0/2133122661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ef41082e0 msgr2=0x7f7ef40819e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:05.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.972+0000 7f7ef9cf7700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ef41082e0 0x7f7ef40819e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:05.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.972+0000 7f7ef9cf7700 1 -- 172.21.15.12:0/2133122661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee40095f0 con 0x7f7ef410c790 2024-05-09T00:04:05.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.972+0000 7f7ef9cf7700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ef410c790 0x7f7ef407b670 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00b9e0 tx=0x7f7eec00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:05.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.973+0000 7f7eea7fc700 1 -- 172.21.15.12:0/2133122661 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00c950 con 0x7f7ef410c790 2024-05-09T00:04:05.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.973+0000 7f7eea7fc700 1 -- 172.21.15.12:0/2133122661 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7eec014440 con 0x7f7ef410c790 2024-05-09T00:04:05.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.973+0000 7f7eea7fc700 1 -- 172.21.15.12:0/2133122661 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0075d0 con 0x7f7ef410c790 2024-05-09T00:04:05.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.976+0000 7f7efb75a700 1 -- 172.21.15.12:0/2133122661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef40709f0 con 0x7f7ef410c790 2024-05-09T00:04:05.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.976+0000 7f7efb75a700 1 -- 172.21.15.12:0/2133122661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef4070e80 con 0x7f7ef410c790 2024-05-09T00:04:05.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.978+0000 7f7efb75a700 1 -- 172.21.15.12:0/2133122661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef407c390 con 0x7f7ef410c790 2024-05-09T00:04:05.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.980+0000 7f7eea7fc700 1 -- 172.21.15.12:0/2133122661 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7eec0145b0 con 0x7f7ef410c790 2024-05-09T00:04:05.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.981+0000 7f7eea7fc700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ee00a1420 0x7f7ee00a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:05.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.981+0000 7f7eea7fc700 1 -- 172.21.15.12:0/2133122661 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7eec006e40 con 0x7f7ef410c790 2024-05-09T00:04:05.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.981+0000 7f7ef94f6700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ee00a1420 0x7f7ee00a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:05.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.983+0000 7f7ef94f6700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ee00a1420 0x7f7ee00a38e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009730 tx=0x7f7ef0006cb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:05.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:05.991+0000 7f7eea7fc700 1 -- 172.21.15.12:0/2133122661 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ef407c390 con 0x7f7ef410c790 2024-05-09T00:04:06.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:05 smithi204 ceph-mon[47373]: pgmap v570: 97 pgs: 4 active+clean+snaptrim, 82 active+clean, 11 active+clean+snaptrim_wait; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-05-09T00:04:06.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:05 smithi160 ceph-mon[46152]: pgmap v570: 97 pgs: 4 active+clean+snaptrim, 82 active+clean, 11 active+clean+snaptrim_wait; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-05-09T00:04:06.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:05 smithi012 ceph-mon[42503]: pgmap v570: 97 pgs: 4 active+clean+snaptrim, 82 active+clean, 11 active+clean+snaptrim_wait; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-05-09T00:04:06.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 171530 37.67 MB/sec execute 121 sec latency 103.807 ms 2024-05-09T00:04:06.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.411+0000 7f7efb75a700 1 -- 172.21.15.12:0/2133122661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ef407c330 con 0x7f7ef410c790 2024-05-09T00:04:06.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.416+0000 7f7eea7fc700 1 -- 172.21.15.12:0/2133122661 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+10872 (secure 0 0 0) 0x7f7eec019090 con 0x7f7ef410c790 2024-05-09T00:04:06.413 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:04:06.413 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":74,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:03:37.323291+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_30034":{"gid":30034,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6833/181468454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":181468454},{"type":"v1","addr":"172.21.15.160:6833","nonce":181468454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:04:06.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.420+0000 7f7efb75a700 1 -- 172.21.15.12:0/2133122661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ee00a1420 msgr2=0x7f7ee00a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.420+0000 7f7efb75a700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ee00a1420 0x7f7ee00a38e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009730 tx=0x7f7ef0006cb0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.420+0000 7f7efb75a700 1 -- 172.21.15.12:0/2133122661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ef410c790 msgr2=0x7f7ef407b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.420+0000 7f7efb75a700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ef410c790 0x7f7ef407b670 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00b9e0 tx=0x7f7eec00bda0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.420+0000 7f7efb75a700 1 -- 172.21.15.12:0/2133122661 shutdown_connections 2024-05-09T00:04:06.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.420+0000 7f7efb75a700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ee00a1420 0x7f7ee00a38e0 secure :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009730 tx=0x7f7ef0006cb0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.420+0000 7f7efb75a700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ef410c790 0x7f7ef407b670 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.420+0000 7f7efb75a700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef4107930 0x7f7ef40814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.420+0000 7f7efb75a700 1 --2- 172.21.15.12:0/2133122661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ef41082e0 0x7f7ef40819e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.420+0000 7f7efb75a700 1 -- 172.21.15.12:0/2133122661 >> 172.21.15.12:0/2133122661 conn(0x7f7ef4076e40 msgr2=0x7f7ef4078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:06.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.424+0000 7f7efb75a700 1 -- 172.21.15.12:0/2133122661 shutdown_connections 2024-05-09T00:04:06.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.424+0000 7f7efb75a700 1 -- 172.21.15.12:0/2133122661 wait complete. 2024-05-09T00:04:06.419 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 74 2024-05-09T00:04:06.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:04:06.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.662+0000 7f1acc46e700 1 -- 172.21.15.12:0/2527005035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ac4107530 msgr2=0x7f1ac41079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.662+0000 7f1acc46e700 1 --2- 172.21.15.12:0/2527005035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ac4107530 0x7f1ac41079b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00099c0 tx=0x7f1ac0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.662+0000 7f1acc46e700 1 -- 172.21.15.12:0/2527005035 shutdown_connections 2024-05-09T00:04:06.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.662+0000 7f1acc46e700 1 --2- 172.21.15.12:0/2527005035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ac4107ef0 0x7f1ac41143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.662+0000 7f1acc46e700 1 --2- 172.21.15.12:0/2527005035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ac410d5f0 0x7f1ac410d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.662+0000 7f1acc46e700 1 --2- 172.21.15.12:0/2527005035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ac4107530 0x7f1ac41079b0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.662+0000 7f1acc46e700 1 -- 172.21.15.12:0/2527005035 >> 172.21.15.12:0/2527005035 conn(0x7f1ac4075490 msgr2=0x7f1ac40778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:06.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.662+0000 7f1acc46e700 1 -- 172.21.15.12:0/2527005035 shutdown_connections 2024-05-09T00:04:06.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.662+0000 7f1acc46e700 1 -- 172.21.15.12:0/2527005035 wait complete. 2024-05-09T00:04:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.663+0000 7f1acc46e700 1 Processor -- start 2024-05-09T00:04:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.663+0000 7f1acc46e700 1 -- start start 2024-05-09T00:04:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.663+0000 7f1acc46e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ac4107530 0x7f1ac41a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.663+0000 7f1acc46e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ac4107ef0 0x7f1ac41a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.663+0000 7f1acc46e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ac419d9e0 0x7f1ac419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.663+0000 7f1acc46e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac419e460 con 0x7f1ac419d9e0 2024-05-09T00:04:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.663+0000 7f1acc46e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac419e5a0 con 0x7f1ac4107530 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.663+0000 7f1acc46e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac419e760 con 0x7f1ac4107ef0 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1ac9a09700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ac4107ef0 0x7f1ac41a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1acaa0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ac419d9e0 0x7f1ac419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1acaa0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ac419d9e0 0x7f1ac419de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51874/0 (socket says 172.21.15.12:51874) 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1acaa0b700 1 -- 172.21.15.12:0/843596554 learned_addr learned my addr 172.21.15.12:0/843596554 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ac4107ef0 0x7f1ac41a3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54220/0 (socket says 172.21.15.12:54220) 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1aca20a700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ac4107530 0x7f1ac41a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1ac9a09700 1 -- 172.21.15.12:0/843596554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ac4107530 msgr2=0x7f1ac41a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ac4107530 0x7f1ac41a38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1ac9a09700 1 -- 172.21.15.12:0/843596554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ac419d9e0 msgr2=0x7f1ac419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ac419d9e0 0x7f1ac419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.664+0000 7f1ac9a09700 1 -- 172.21.15.12:0/843596554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00095f0 con 0x7f1ac4107ef0 2024-05-09T00:04:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.665+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ac4107ef0 0x7f1ac41a3df0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000b580 tx=0x7f1ac0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:06.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.665+0000 7f1abb7fe700 1 -- 172.21.15.12:0/843596554 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac001d070 con 0x7f1ac4107ef0 2024-05-09T00:04:06.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.665+0000 7f1acc46e700 1 -- 172.21.15.12:0/843596554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac406a9c0 con 0x7f1ac4107ef0 2024-05-09T00:04:06.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.665+0000 7f1abb7fe700 1 -- 172.21.15.12:0/843596554 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac000fbf0 con 0x7f1ac4107ef0 2024-05-09T00:04:06.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.665+0000 7f1abb7fe700 1 -- 172.21.15.12:0/843596554 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0022c00 con 0x7f1ac4107ef0 2024-05-09T00:04:06.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.665+0000 7f1acc46e700 1 -- 172.21.15.12:0/843596554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac406af20 con 0x7f1ac4107ef0 2024-05-09T00:04:06.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.669+0000 7f1ad8628b80 1 Processor -- start 2024-05-09T00:04:06.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.669+0000 7f1abb7fe700 1 -- 172.21.15.12:0/843596554 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1ac000f730 con 0x7f1ac4107ef0 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.670+0000 7f1abb7fe700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1ab00a1390 0x7f1ab00a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.670+0000 7f1abb7fe700 1 -- 172.21.15.12:0/843596554 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1ac00ce150 con 0x7f1ac4107ef0 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.671+0000 7f1ad8628b80 1 -- start start 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.671+0000 7f1acaa0b700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1ab00a1390 0x7f1ab00a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.672+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x5605965781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.672+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 0x560596572350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.672+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596572890 0x560596572cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.672+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605964c9150 con 0x560596572890 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.672+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605964c6b10 con 0x560596577e00 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.672+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605964c2400 con 0x560596578720 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.672+0000 7f1acaa0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596572890 0x560596572cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.672+0000 7f1acaa0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596572890 0x560596572cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51890/0 (socket says 172.21.15.12:51890) 2024-05-09T00:04:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.672+0000 7f1acaa0b700 1 -- 172.21.15.12:0/577701258 learned_addr learned my addr 172.21.15.12:0/577701258 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:06.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.672+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/577701258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 0x560596572350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.673+0000 7f1ac9a09700 1 -- 172.21.15.12:0/577701258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 msgr2=0x5605965781e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:06.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.673+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/577701258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x5605965781e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.673+0000 7f1ac9a09700 1 -- 172.21.15.12:0/577701258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596572890 msgr2=0x560596572cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:06.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.673+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/577701258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596572890 0x560596572cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.673+0000 7f1acaa0b700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1ab00a1390 0x7f1ab00a3850 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00e250 tx=0x7f1abc009710 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:06.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.673+0000 7f1ac9a09700 1 -- 172.21.15.12:0/577701258 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605963433b0 con 0x560596578720 2024-05-09T00:04:06.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.673+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/577701258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 0x560596572350 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00cb830 tx=0x7f1ac0003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe91a7ce1f0f7b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:06.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.674+0000 7f1ab8da4700 1 -- 172.21.15.12:0/577701258 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac009b280 con 0x560596578720 2024-05-09T00:04:06.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.674+0000 7f1ab8da4700 1 -- 172.21.15.12:0/577701258 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac009f0b0 con 0x560596578720 2024-05-09T00:04:06.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.674+0000 7f1ab8da4700 1 -- 172.21.15.12:0/577701258 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac009b740 con 0x560596578720 2024-05-09T00:04:06.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.675+0000 7f1ad8628b80 1 -- 172.21.15.12:0/577701258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 msgr2=0x560596572350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.675+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/577701258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 0x560596572350 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00cb830 tx=0x7f1ac0003680 comp rx=0 tx=0).stop 2024-05-09T00:04:06.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.675+0000 7f1ad8628b80 1 -- 172.21.15.12:0/577701258 shutdown_connections 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.675+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/577701258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596572890 0x560596572cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.675+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/577701258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x5605965781e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.675+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/577701258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 0x560596572350 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.675+0000 7f1ad8628b80 1 -- 172.21.15.12:0/577701258 >> 172.21.15.12:0/577701258 conn(0x5605964d0210 msgr2=0x56059656cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.675+0000 7f1ad8628b80 1 -- 172.21.15.12:0/577701258 shutdown_connections 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.675+0000 7f1ad8628b80 1 -- 172.21.15.12:0/577701258 wait complete. 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.676+0000 7f1ad8628b80 1 -- start start 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.676+0000 7f1ad8628b80 1 Processor -- start 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.676+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596572890 0x56059657e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.676+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x56059657b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.676+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 0x56059657c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.676+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059645e560 con 0x560596578720 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.676+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059645e200 con 0x560596577e00 2024-05-09T00:04:06.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.676+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605963456a0 con 0x560596572890 2024-05-09T00:04:06.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.676+0000 7f1ad8628b80 10 client.0 _ll_register_callbacks cb 0x560596467b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:06.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.677+0000 7f1ac9a09700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x56059657b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.677+0000 7f1ac9a09700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x56059657b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41474/0 (socket says 172.21.15.12:41474) 2024-05-09T00:04:06.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.677+0000 7f1ac9a09700 1 -- 172.21.15.12:0/1443872177 learned_addr learned my addr 172.21.15.12:0/1443872177 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:06.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.677+0000 7f1acaa0b700 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 0x56059657c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.677+0000 7f1aca20a700 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596572890 0x56059657e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.678+0000 7f1aca20a700 1 -- 172.21.15.12:0/1443872177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 msgr2=0x56059657b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.678+0000 7f1aca20a700 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x56059657b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.678+0000 7f1aca20a700 1 -- 172.21.15.12:0/1443872177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 msgr2=0x56059657c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.678+0000 7f1aca20a700 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 0x56059657c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.678+0000 7f1aca20a700 1 -- 172.21.15.12:0/1443872177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac406af20 con 0x560596572890 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.678+0000 7f1aca20a700 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596572890 0x56059657e770 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4108650 tx=0x7f1ab400c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7520262a4ed4882c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.678+0000 7f1aadffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.678+0000 7f1aadffb700 1 -- 172.21.15.12:0/1443872177 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400ce60 con 0x560596572890 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.678+0000 7f1ad8628b80 1 -- 172.21.15.12:0/1443872177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56059641b270 con 0x560596572890 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.678+0000 7f1aadffb700 1 -- 172.21.15.12:0/1443872177 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1ab40113f0 con 0x560596572890 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.679+0000 7f1aadffb700 1 -- 172.21.15.12:0/1443872177 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ab40115b0 con 0x560596572890 2024-05-09T00:04:06.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.679+0000 7f1ad8628b80 10 client.30082 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:06.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.679+0000 7f1ad8628b80 1 -- 172.21.15.12:0/1443872177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5605962efb60 con 0x560596572890 2024-05-09T00:04:06.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.679+0000 7f1aadffb700 1 -- 172.21.15.12:0/1443872177 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56059641b270 con 0x560596572890 2024-05-09T00:04:06.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.680+0000 7f1aadffb700 1 -- 172.21.15.12:0/1443872177 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1ab401a9d0 con 0x560596572890 2024-05-09T00:04:06.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.680+0000 7f1ad8628b80 10 client.30082 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:06.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.680+0000 7f1ad8628b80 10 client.30082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:04:06.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.680+0000 7f1ad8628b80 20 client.30082 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:06.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.680+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56059658be10 0x56059658e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.680+0000 7f1ad8628b80 4 client.30082 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:06.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.680+0000 7f1ad8628b80 1 -- 172.21.15.12:0/1443872177 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560596541810 con 0x56059658be10 2024-05-09T00:04:06.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.681+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56059658be10 0x56059658e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.681+0000 7f1aadffb700 10 client.30082 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:04:06.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.681+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56059658be10 0x56059658e230 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e78f48ac419ec560 server_cookie=94502867e3db49bd in_seq=0 out_seq=0 2024-05-09T00:04:06.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1aadffb700 1 -- 172.21.15.12:0/1443872177 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560596541810 con 0x56059658be10 2024-05-09T00:04:06.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1aadffb700 10 client.30082 handle_command_reply: tid=0 2024-05-09T00:04:06.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1ad8628b80 1 client.30082 shutdown 2024-05-09T00:04:06.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1ad8628b80 1 -- 172.21.15.12:0/1443872177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596572890 msgr2=0x56059657e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596572890 0x56059657e770 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4108650 tx=0x7f1ab400c430 comp rx=0 tx=0).stop 2024-05-09T00:04:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1ad8628b80 1 -- 172.21.15.12:0/1443872177 shutdown_connections 2024-05-09T00:04:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 0x56059657c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56059658be10 0x56059658e230 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596572890 0x56059657e770 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1aadffb700 0 client.30082 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:04:06.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/1443872177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x56059657b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.682+0000 7f1ad8628b80 1 -- 172.21.15.12:0/1443872177 >> 172.21.15.12:0/1443872177 conn(0x5605964d0210 msgr2=0x56059656c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:06.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.684+0000 7f1ad8628b80 1 -- 172.21.15.12:0/1443872177 shutdown_connections 2024-05-09T00:04:06.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.685+0000 7f1ad8628b80 1 -- 172.21.15.12:0/1443872177 wait complete. 2024-05-09T00:04:06.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.685+0000 7f1ad8628b80 20 client.30082 trim_cache size 0 max 16384 2024-05-09T00:04:06.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:06 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2133122661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:04:06.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.704+0000 7f1ad8628b80 1 Processor -- start 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.705+0000 7f1ad8628b80 1 -- start start 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.705+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596572890 0x56059656cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.705+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x560596582ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.705+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 0x560596583410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.705+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560596588830 con 0x560596578720 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.705+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560596588220 con 0x560596577e00 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.705+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059656d260 con 0x560596572890 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.705+0000 7f1acaa0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 0x560596583410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.705+0000 7f1acaa0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 0x560596583410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51902/0 (socket says 172.21.15.12:51902) 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.705+0000 7f1acaa0b700 1 -- 172.21.15.12:0/4207550585 learned_addr learned my addr 172.21.15.12:0/4207550585 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.706+0000 7f1acaa0b700 1 -- 172.21.15.12:0/4207550585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596572890 msgr2=0x56059656cd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.706+0000 7f1acaa0b700 1 --2- 172.21.15.12:0/4207550585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596572890 0x56059656cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.706+0000 7f1acaa0b700 1 -- 172.21.15.12:0/4207550585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 msgr2=0x560596582ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.706+0000 7f1acaa0b700 1 --2- 172.21.15.12:0/4207550585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x560596582ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.706+0000 7f1acaa0b700 1 -- 172.21.15.12:0/4207550585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560596572f40 con 0x560596578720 2024-05-09T00:04:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.706+0000 7f1acaa0b700 1 --2- 172.21.15.12:0/4207550585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 0x560596583410 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0175d0 tx=0x7f1abc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bab983e2e87ef8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.706+0000 7f1ab95a5700 1 -- 172.21.15.12:0/4207550585 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc017e20 con 0x560596578720 2024-05-09T00:04:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.706+0000 7f1ab95a5700 1 -- 172.21.15.12:0/4207550585 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1abc00f040 con 0x560596578720 2024-05-09T00:04:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.706+0000 7f1ab95a5700 1 -- 172.21.15.12:0/4207550585 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0297c0 con 0x560596578720 2024-05-09T00:04:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.707+0000 7f1ad8628b80 1 -- 172.21.15.12:0/4207550585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 msgr2=0x560596583410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.707+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/4207550585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 0x560596583410 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0175d0 tx=0x7f1abc019000 comp rx=0 tx=0).stop 2024-05-09T00:04:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.709+0000 7f1ad8628b80 1 -- 172.21.15.12:0/4207550585 shutdown_connections 2024-05-09T00:04:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.709+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/4207550585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596578720 0x560596583410 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.709+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/4207550585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596572890 0x56059656cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.709+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/4207550585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596577e00 0x560596582ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.709+0000 7f1ad8628b80 1 -- 172.21.15.12:0/4207550585 >> 172.21.15.12:0/4207550585 conn(0x5605964d0210 msgr2=0x560596575b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:06.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.709+0000 7f1ad8628b80 1 -- 172.21.15.12:0/4207550585 shutdown_connections 2024-05-09T00:04:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.709+0000 7f1ad8628b80 1 -- 172.21.15.12:0/4207550585 wait complete. 2024-05-09T00:04:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.710+0000 7f1ad8628b80 1 -- start start 2024-05-09T00:04:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.710+0000 7f1ad8628b80 1 Processor -- start 2024-05-09T00:04:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.710+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596572890 0x560596582ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.710+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596577e00 0x560596573b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.710+0000 7f1ad8628b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 0x560596574040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.710+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560596583670 con 0x560596577e00 2024-05-09T00:04:06.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.710+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605965837e0 con 0x560596572890 2024-05-09T00:04:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ad8628b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605965745d0 con 0x560596578720 2024-05-09T00:04:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ad8628b80 10 client.0 _ll_register_callbacks cb 0x560596467b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ac9a09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596577e00 0x560596573b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ac9a09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596577e00 0x560596573b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51916/0 (socket says 172.21.15.12:51916) 2024-05-09T00:04:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ac9a09700 1 -- 172.21.15.12:0/2451542182 learned_addr learned my addr 172.21.15.12:0/2451542182 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:06.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1acaa0b700 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 0x560596574040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ac9a09700 1 -- 172.21.15.12:0/2451542182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 msgr2=0x560596574040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 0x560596574040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ac9a09700 1 -- 172.21.15.12:0/2451542182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596572890 msgr2=0x560596582ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596572890 0x560596582ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ac9a09700 1 -- 172.21.15.12:0/2451542182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc00ee90 con 0x560596577e00 2024-05-09T00:04:06.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1acaa0b700 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 0x560596574040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:04:06.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1ac9a09700 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596577e00 0x560596573b00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1ac009ffd0 tx=0x7f1ac000fd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb4b01102556013 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:06.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.711+0000 7f1aadffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:06.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.712+0000 7f1aadffb700 1 -- 172.21.15.12:0/2451542182 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0066c60 con 0x560596577e00 2024-05-09T00:04:06.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.712+0000 7f1aadffb700 1 -- 172.21.15.12:0/2451542182 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1ac0022470 con 0x560596577e00 2024-05-09T00:04:06.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.712+0000 7f1ad8628b80 1 -- 172.21.15.12:0/2451542182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ac009f0b0 con 0x560596577e00 2024-05-09T00:04:06.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.719+0000 7f1aadffb700 1 -- 172.21.15.12:0/2451542182 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ac009a770 con 0x560596577e00 2024-05-09T00:04:06.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.719+0000 7f1ad8628b80 10 client.20361 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:06.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.719+0000 7f1ad8628b80 1 -- 172.21.15.12:0/2451542182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x560596488560 con 0x560596577e00 2024-05-09T00:04:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.720+0000 7f1aadffb700 1 -- 172.21.15.12:0/2451542182 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac009f0b0 con 0x560596577e00 2024-05-09T00:04:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.720+0000 7f1aadffb700 1 -- 172.21.15.12:0/2451542182 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1ac00a32f0 con 0x560596577e00 2024-05-09T00:04:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.720+0000 7f1ad8628b80 10 client.20361 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.720+0000 7f1ad8628b80 10 client.20361 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:04:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.720+0000 7f1ad8628b80 20 client.20361 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.720+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56059658be10 0x5605965e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.720+0000 7f1ad8628b80 4 client.20361 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:04:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.720+0000 7f1ad8628b80 1 -- 172.21.15.12:0/2451542182 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5605964baec0 con 0x56059658be10 2024-05-09T00:04:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.720+0000 7f1aca20a700 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56059658be10 0x5605965e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.721+0000 7f1aca20a700 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56059658be10 0x5605965e52c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6491c6e4aaec1af4 server_cookie=b803ee4c3416d18f in_seq=0 out_seq=0 2024-05-09T00:04:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.721+0000 7f1aadffb700 10 client.20361 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:04:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1aadffb700 1 -- 172.21.15.12:0/2451542182 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5605964baec0 con 0x56059658be10 2024-05-09T00:04:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1aadffb700 10 client.20361 handle_command_reply: tid=0 2024-05-09T00:04:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1ad8628b80 1 client.20361 shutdown 2024-05-09T00:04:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1ad8628b80 1 -- 172.21.15.12:0/2451542182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596577e00 msgr2=0x560596573b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596577e00 0x560596573b00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1ac009ffd0 tx=0x7f1ac000fd80 comp rx=0 tx=0).stop 2024-05-09T00:04:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1ad8628b80 1 -- 172.21.15.12:0/2451542182 shutdown_connections 2024-05-09T00:04:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56059658be10 0x5605965e52c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560596578720 0x560596574040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560596572890 0x560596582ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1ad8628b80 1 --2- 172.21.15.12:0/2451542182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560596577e00 0x560596573b00 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1ad8628b80 1 -- 172.21.15.12:0/2451542182 >> 172.21.15.12:0/2451542182 conn(0x5605964d0210 msgr2=0x56059657b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.726+0000 7f1aadffb700 0 client.20361 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:04:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.727+0000 7f1ad8628b80 1 -- 172.21.15.12:0/2451542182 shutdown_connections 2024-05-09T00:04:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.727+0000 7f1ad8628b80 1 -- 172.21.15.12:0/2451542182 wait complete. 2024-05-09T00:04:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.727+0000 7f1ad8628b80 20 client.20361 trim_cache size 0 max 16384 2024-05-09T00:04:06.722 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:04:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.728+0000 7f1acc46e700 1 -- 172.21.15.12:0/843596554 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1ab00a1390 msgr2=0x7f1ab00a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.728+0000 7f1acc46e700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1ab00a1390 0x7f1ab00a3850 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00e250 tx=0x7f1abc009710 comp rx=0 tx=0).stop 2024-05-09T00:04:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.728+0000 7f1acc46e700 1 -- 172.21.15.12:0/843596554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ac4107ef0 msgr2=0x7f1ac41a3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.728+0000 7f1acc46e700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ac4107ef0 0x7f1ac41a3df0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000b580 tx=0x7f1ac0004730 comp rx=0 tx=0).stop 2024-05-09T00:04:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.728+0000 7f1acc46e700 1 -- 172.21.15.12:0/843596554 shutdown_connections 2024-05-09T00:04:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.728+0000 7f1acc46e700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1ab00a1390 0x7f1ab00a3850 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.728+0000 7f1acc46e700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ac419d9e0 0x7f1ac419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.728+0000 7f1acc46e700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ac4107530 0x7f1ac41a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.728+0000 7f1acc46e700 1 --2- 172.21.15.12:0/843596554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ac4107ef0 0x7f1ac41a3df0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.728+0000 7f1acc46e700 1 -- 172.21.15.12:0/843596554 >> 172.21.15.12:0/843596554 conn(0x7f1ac4075490 msgr2=0x7f1ac4076d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:06.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.730+0000 7f1acc46e700 1 -- 172.21.15.12:0/843596554 shutdown_connections 2024-05-09T00:04:06.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.734+0000 7f1acc46e700 1 -- 172.21.15.12:0/843596554 wait complete. 2024-05-09T00:04:06.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:04:06.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.959+0000 7ff9d2735700 1 -- 172.21.15.12:0/359922623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc100060 msgr2=0x7ff9cc100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.959+0000 7ff9d2735700 1 --2- 172.21.15.12:0/359922623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc100060 0x7ff9cc100440 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009990 tx=0x7ff9bc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.959+0000 7ff9d2735700 1 -- 172.21.15.12:0/359922623 shutdown_connections 2024-05-09T00:04:06.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.959+0000 7ff9d2735700 1 --2- 172.21.15.12:0/359922623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9cc105160 0x7ff9cc10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.959+0000 7ff9d2735700 1 --2- 172.21.15.12:0/359922623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc100060 0x7ff9cc100440 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.959+0000 7ff9d2735700 1 --2- 172.21.15.12:0/359922623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc100a10 0x7ff9cc104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.959+0000 7ff9d2735700 1 -- 172.21.15.12:0/359922623 >> 172.21.15.12:0/359922623 conn(0x7ff9cc076f40 msgr2=0x7ff9cc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:06.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.959+0000 7ff9d2735700 1 -- 172.21.15.12:0/359922623 shutdown_connections 2024-05-09T00:04:06.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.959+0000 7ff9d2735700 1 -- 172.21.15.12:0/359922623 wait complete. 2024-05-09T00:04:06.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.960+0000 7ff9d2735700 1 Processor -- start 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.960+0000 7ff9d2735700 1 -- start start 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.960+0000 7ff9d2735700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc100060 0x7ff9cc1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.960+0000 7ff9d2735700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc105160 0x7ff9cc197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.960+0000 7ff9d2735700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9cc19bca0 0x7ff9cc19c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.960+0000 7ff9d2735700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc19c720 con 0x7ff9cc105160 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.960+0000 7ff9d2735700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc19c890 con 0x7ff9cc19bca0 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.960+0000 7ff9d2735700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc19ca50 con 0x7ff9cc100060 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.960+0000 7ff9cb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc105160 0x7ff9cc197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.960+0000 7ff9cbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc100060 0x7ff9cc1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9cbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc100060 0x7ff9cc1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54248/0 (socket says 172.21.15.12:54248) 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9cbfff700 1 -- 172.21.15.12:0/1587355210 learned_addr learned my addr 172.21.15.12:0/1587355210 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9cb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc105160 0x7ff9cc197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51944/0 (socket says 172.21.15.12:51944) 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9cc19bca0 0x7ff9cc19c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9cbfff700 1 -- 172.21.15.12:0/1587355210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9cc19bca0 msgr2=0x7ff9cc19c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9cbfff700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9cc19bca0 0x7ff9cc19c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9cbfff700 1 -- 172.21.15.12:0/1587355210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc105160 msgr2=0x7ff9cc197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9cbfff700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc105160 0x7ff9cc197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9cbfff700 1 -- 172.21.15.12:0/1587355210 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc0095f0 con 0x7ff9cc100060 2024-05-09T00:04:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9cbfff700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc100060 0x7ff9cc1973c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc004760 tx=0x7ff9bc004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9c97fa700 1 -- 172.21.15.12:0/1587355210 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc01d070 con 0x7ff9cc100060 2024-05-09T00:04:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9c97fa700 1 -- 172.21.15.12:0/1587355210 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9bc022470 con 0x7ff9cc100060 2024-05-09T00:04:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.961+0000 7ff9c97fa700 1 -- 172.21.15.12:0/1587355210 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc017650 con 0x7ff9cc100060 2024-05-09T00:04:06.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.962+0000 7ff9d2735700 1 -- 172.21.15.12:0/1587355210 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc101a00 con 0x7ff9cc100060 2024-05-09T00:04:06.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.963+0000 7ff9d2735700 1 -- 172.21.15.12:0/1587355210 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc101e30 con 0x7ff9cc100060 2024-05-09T00:04:06.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.966+0000 7ff9c97fa700 1 -- 172.21.15.12:0/1587355210 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff9bc0225e0 con 0x7ff9cc100060 2024-05-09T00:04:06.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.967+0000 7ff9de8efb80 1 Processor -- start 2024-05-09T00:04:06.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9de8efb80 1 -- start start 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b61350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b61cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9ab8150 con 0x55ede9b66e00 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9ab5b10 con 0x55ede9b61890 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9ab1400 con 0x55ede9b67720 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9c97fa700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9b40a1310 0x7ff9b40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9cbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b61350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9d0cd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9d0cd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b671e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51948/0 (socket says 172.21.15.12:51948) 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/724864189 learned_addr learned my addr 172.21.15.12:0/724864189 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9cb7fe700 1 --2- 172.21.15.12:0/724864189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b61cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9c97fa700 1 -- 172.21.15.12:0/1587355210 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff9bc0cdac0 con 0x7ff9cc100060 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.968+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9b40a1310 0x7ff9b40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9cbfff700 1 -- 172.21.15.12:0/724864189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 msgr2=0x55ede9b61cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9cbfff700 1 --2- 172.21.15.12:0/724864189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b61cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9cbfff700 1 -- 172.21.15.12:0/724864189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 msgr2=0x55ede9b671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9cbfff700 1 --2- 172.21.15.12:0/724864189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9cbfff700 1 -- 172.21.15.12:0/724864189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ede99323b0 con 0x55ede9b67720 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/724864189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9cbfff700 1 --2- 172.21.15.12:0/724864189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b61350 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0cb200 tx=0x7ff9bc06dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1948022b2816467 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9b40a1310 0x7ff9b40a37d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400fcb0 tx=0x7ff9c4004b40 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9b27fc700 1 -- 172.21.15.12:0/724864189 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc088040 con 0x55ede9b67720 2024-05-09T00:04:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9b27fc700 1 -- 172.21.15.12:0/724864189 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9bc09a5b0 con 0x55ede9b67720 2024-05-09T00:04:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.969+0000 7ff9b27fc700 1 -- 172.21.15.12:0/724864189 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0a3040 con 0x55ede9b67720 2024-05-09T00:04:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.970+0000 7ff9de8efb80 1 -- 172.21.15.12:0/724864189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 msgr2=0x55ede9b61350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.970+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/724864189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b61350 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0cb200 tx=0x7ff9bc06dde0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.970+0000 7ff9de8efb80 1 -- 172.21.15.12:0/724864189 shutdown_connections 2024-05-09T00:04:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.970+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/724864189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b61cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.970+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/724864189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.970+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/724864189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b61350 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.970+0000 7ff9de8efb80 1 -- 172.21.15.12:0/724864189 >> 172.21.15.12:0/724864189 conn(0x55ede9abf210 msgr2=0x55ede9b5bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:06.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.970+0000 7ff9de8efb80 1 -- 172.21.15.12:0/724864189 shutdown_connections 2024-05-09T00:04:06.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.970+0000 7ff9de8efb80 1 -- 172.21.15.12:0/724864189 wait complete. 2024-05-09T00:04:06.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.970+0000 7ff9de8efb80 1 -- start start 2024-05-09T00:04:06.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.971+0000 7ff9de8efb80 1 Processor -- start 2024-05-09T00:04:06.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.971+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b6aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.971+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.971+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.971+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9a4d560 con 0x55ede9b66e00 2024-05-09T00:04:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.971+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9a4d200 con 0x55ede9b61890 2024-05-09T00:04:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.971+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede99346a0 con 0x55ede9b67720 2024-05-09T00:04:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.971+0000 7ff9de8efb80 10 client.0 _ll_register_callbacks cb 0x55ede9a56b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.972+0000 7ff9d0cd2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.972+0000 7ff9d0cd2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b6b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54258/0 (socket says 172.21.15.12:54258) 2024-05-09T00:04:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.972+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/2789495050 learned_addr learned my addr 172.21.15.12:0/2789495050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.972+0000 7ff9cb7fe700 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.972+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/2789495050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 msgr2=0x55ede9b6aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.972+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b6aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.972+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/2789495050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 msgr2=0x55ede9b6b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.972+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b6b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.972+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/2789495050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c400f820 con 0x55ede9b67720 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.972+0000 7ff9cb7fe700 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b6b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.973+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b6b570 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40094a0 tx=0x7ff9c4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e517292292c24899 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.973+0000 7ff9ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.973+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/2789495050 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c400e040 con 0x55ede9b67720 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.973+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/2789495050 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff9c4004db0 con 0x55ede9b67720 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.973+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2789495050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ede9a0a270 con 0x55ede9b67720 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.973+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/2789495050 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c4007e80 con 0x55ede9b67720 2024-05-09T00:04:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.973+0000 7ff9de8efb80 10 client.30097 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.973+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2789495050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7ff9cc101e30 con 0x55ede9b67720 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.973+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/2789495050 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ede9a0a270 con 0x55ede9b67720 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.973+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/2789495050 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff9c40254d0 con 0x55ede9b67720 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.974+0000 7ff9de8efb80 10 client.30097 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.974+0000 7ff9de8efb80 10 client.30097 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.974+0000 7ff9de8efb80 20 client.30097 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.974+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ede9b7afc0 0x55ede9b7d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.974+0000 7ff9de8efb80 4 client.30097 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.974+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2789495050 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ede9b30810 con 0x55ede9b7afc0 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.974+0000 7ff9cbfff700 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ede9b7afc0 0x55ede9b7d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.974+0000 7ff9ab7fe700 10 client.30097 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:04:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.974+0000 7ff9cbfff700 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ede9b7afc0 0x55ede9b7d3e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82973b3ca309667c server_cookie=cb4195abd08f670d in_seq=0 out_seq=0 2024-05-09T00:04:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.978+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/2789495050 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ede9b30810 con 0x55ede9b7afc0 2024-05-09T00:04:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.978+0000 7ff9ab7fe700 10 client.30097 handle_command_reply: tid=0 2024-05-09T00:04:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.979+0000 7ff9de8efb80 1 client.30097 shutdown 2024-05-09T00:04:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.979+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2789495050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 msgr2=0x55ede9b6b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.979+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b6b570 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40094a0 tx=0x7ff9c4004ad0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.979+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2789495050 shutdown_connections 2024-05-09T00:04:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.979+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ede9b7afc0 0x55ede9b7d3e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.979+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b6b570 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.979+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b6aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.979+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/2789495050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b6b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.979+0000 7ff9ab7fe700 0 client.30097 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:04:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.979+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2789495050 >> 172.21.15.12:0/2789495050 conn(0x55ede9abf210 msgr2=0x55ede9b5b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.981+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2789495050 shutdown_connections 2024-05-09T00:04:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.981+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2789495050 wait complete. 2024-05-09T00:04:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:06.981+0000 7ff9de8efb80 20 client.30097 trim_cache size 0 max 16384 2024-05-09T00:04:06.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.003+0000 7ff9de8efb80 1 Processor -- start 2024-05-09T00:04:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9de8efb80 1 -- start start 2024-05-09T00:04:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b61350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b5bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b662d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9b58dc0 con 0x55ede9b66e00 2024-05-09T00:04:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9b67560 con 0x55ede9b61890 2024-05-09T00:04:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9b825f0 con 0x55ede9b67720 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9cb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b5bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9d0cd2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9cb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b5bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51966/0 (socket says 172.21.15.12:51966) 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.004+0000 7ff9cb7fe700 1 -- 172.21.15.12:0/2231547012 learned_addr learned my addr 172.21.15.12:0/2231547012 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.005+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/2231547012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 msgr2=0x55ede9b61350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.005+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/2231547012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b61350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.005+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/2231547012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 msgr2=0x55ede9b5bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.005+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/2231547012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b5bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.005+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/2231547012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ede98deb60 con 0x55ede9b67720 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.005+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/2231547012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b662d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40258d0 tx=0x7ff9c4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3d8ce2e8e5b0e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.005+0000 7ff9b2ffd700 1 -- 172.21.15.12:0/2231547012 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c400e030 con 0x55ede9b67720 2024-05-09T00:04:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.005+0000 7ff9b2ffd700 1 -- 172.21.15.12:0/2231547012 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9c400f040 con 0x55ede9b67720 2024-05-09T00:04:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.005+0000 7ff9b2ffd700 1 -- 172.21.15.12:0/2231547012 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c401ac90 con 0x55ede9b67720 2024-05-09T00:04:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.006+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2231547012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 msgr2=0x55ede9b662d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.006+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/2231547012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b662d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40258d0 tx=0x7ff9c4009e30 comp rx=0 tx=0).stop 2024-05-09T00:04:07.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.007+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2231547012 shutdown_connections 2024-05-09T00:04:07.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.007+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/2231547012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b662d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.007+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/2231547012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b61350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.007+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/2231547012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b5bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.007+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2231547012 >> 172.21.15.12:0/2231547012 conn(0x55ede9abf210 msgr2=0x55ede9b65bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.010+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2231547012 shutdown_connections 2024-05-09T00:04:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.010+0000 7ff9de8efb80 1 -- 172.21.15.12:0/2231547012 wait complete. 2024-05-09T00:04:07.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.011+0000 7ff9de8efb80 1 -- start start 2024-05-09T00:04:07.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.011+0000 7ff9de8efb80 1 Processor -- start 2024-05-09T00:04:07.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.011+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.011+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b71ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.011+0000 7ff9de8efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b62930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.011+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9b727f0 con 0x55ede9b66e00 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.011+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9b62e70 con 0x55ede9b61890 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.011+0000 7ff9de8efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede9b63030 con 0x55ede9b67720 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.011+0000 7ff9de8efb80 10 client.0 _ll_register_callbacks cb 0x55ede9a56b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.012+0000 7ff9d0cd2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b62930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.012+0000 7ff9d0cd2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b62930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54278/0 (socket says 172.21.15.12:54278) 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.012+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/3139603370 learned_addr learned my addr 172.21.15.12:0/3139603370 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.012+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/3139603370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 msgr2=0x55ede9b643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.012+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.012+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/3139603370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 msgr2=0x55ede9b71ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.012+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b71ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.012+0000 7ff9d0cd2700 1 -- 172.21.15.12:0/3139603370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc101e30 con 0x55ede9b67720 2024-05-09T00:04:07.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.012+0000 7ff9d0cd2700 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b62930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4025920 tx=0x7ff9c400fc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3731ba786701e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:07.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.013+0000 7ff9ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:07.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.013+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/3139603370 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c400e040 con 0x55ede9b67720 2024-05-09T00:04:07.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.013+0000 7ff9de8efb80 1 -- 172.21.15.12:0/3139603370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9bc09a5b0 con 0x55ede9b67720 2024-05-09T00:04:07.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.013+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/3139603370 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff9c402e9c0 con 0x55ede9b67720 2024-05-09T00:04:07.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.013+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/3139603370 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c402eba0 con 0x55ede9b67720 2024-05-09T00:04:07.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.013+0000 7ff9de8efb80 10 client.30103 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:07.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.013+0000 7ff9de8efb80 1 -- 172.21.15.12:0/3139603370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55ede9a77560 con 0x55ede9b67720 2024-05-09T00:04:07.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.014+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/3139603370 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9bc09a5b0 con 0x55ede9b67720 2024-05-09T00:04:07.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.014+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/3139603370 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff9c4007500 con 0x55ede9b67720 2024-05-09T00:04:07.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.014+0000 7ff9de8efb80 10 client.30103 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:07.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.014+0000 7ff9de8efb80 10 client.30103 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:04:07.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.014+0000 7ff9de8efb80 20 client.30103 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:07.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.014+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ede9bd1f40 0x55ede9bd4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.014+0000 7ff9de8efb80 4 client.30103 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:04:07.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.014+0000 7ff9de8efb80 1 -- 172.21.15.12:0/3139603370 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ede9aa9ec0 con 0x55ede9bd1f40 2024-05-09T00:04:07.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.015+0000 7ff9cbfff700 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ede9bd1f40 0x55ede9bd4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:06 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2133122661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:04:07.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.018+0000 7ff9ab7fe700 10 client.30103 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:04:07.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.018+0000 7ff9cbfff700 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ede9bd1f40 0x55ede9bd4360 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6d0a3f99c3c23a server_cookie=8e617924892d0fa4 in_seq=0 out_seq=0 2024-05-09T00:04:07.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:06 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2133122661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:04:07.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9ab7fe700 1 -- 172.21.15.12:0/3139603370 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ede9aa9ec0 con 0x55ede9bd1f40 2024-05-09T00:04:07.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9ab7fe700 10 client.30103 handle_command_reply: tid=0 2024-05-09T00:04:07.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9de8efb80 1 client.30103 shutdown 2024-05-09T00:04:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9de8efb80 1 -- 172.21.15.12:0/3139603370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 msgr2=0x55ede9b62930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b62930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4025920 tx=0x7ff9c400fc20 comp rx=0 tx=0).stop 2024-05-09T00:04:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9de8efb80 1 -- 172.21.15.12:0/3139603370 shutdown_connections 2024-05-09T00:04:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ede9bd1f40 0x55ede9bd4360 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ede9b67720 0x55ede9b62930 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9ab7fe700 0 client.30103 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:04:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ede9b61890 0x55ede9b643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9de8efb80 1 --2- 172.21.15.12:0/3139603370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ede9b66e00 0x55ede9b71ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.022+0000 7ff9de8efb80 1 -- 172.21.15.12:0/3139603370 >> 172.21.15.12:0/3139603370 conn(0x55ede9abf210 msgr2=0x55ede9b66450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.023+0000 7ff9de8efb80 1 -- 172.21.15.12:0/3139603370 shutdown_connections 2024-05-09T00:04:07.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.027+0000 7ff9de8efb80 1 -- 172.21.15.12:0/3139603370 wait complete. 2024-05-09T00:04:07.026 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:04:07.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.030+0000 7ff9de8efb80 20 client.30103 trim_cache size 0 max 16384 2024-05-09T00:04:07.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.030+0000 7ff9d2735700 1 -- 172.21.15.12:0/1587355210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9b40a1310 msgr2=0x7ff9b40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.030+0000 7ff9d2735700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9b40a1310 0x7ff9b40a37d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400fcb0 tx=0x7ff9c4004b40 comp rx=0 tx=0).stop 2024-05-09T00:04:07.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.030+0000 7ff9d2735700 1 -- 172.21.15.12:0/1587355210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc100060 msgr2=0x7ff9cc1973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.030+0000 7ff9d2735700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc100060 0x7ff9cc1973c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc004760 tx=0x7ff9bc004790 comp rx=0 tx=0).stop 2024-05-09T00:04:07.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.031+0000 7ff9d2735700 1 -- 172.21.15.12:0/1587355210 shutdown_connections 2024-05-09T00:04:07.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.031+0000 7ff9d2735700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff9b40a1310 0x7ff9b40a37d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.031+0000 7ff9d2735700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9cc19bca0 0x7ff9cc19c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.031+0000 7ff9d2735700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc100060 0x7ff9cc1973c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.031+0000 7ff9d2735700 1 --2- 172.21.15.12:0/1587355210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc105160 0x7ff9cc197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.031+0000 7ff9d2735700 1 -- 172.21.15.12:0/1587355210 >> 172.21.15.12:0/1587355210 conn(0x7ff9cc076f40 msgr2=0x7ff9cc068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:07.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.031+0000 7ff9d2735700 1 -- 172.21.15.12:0/1587355210 shutdown_connections 2024-05-09T00:04:07.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.031+0000 7ff9d2735700 1 -- 172.21.15.12:0/1587355210 wait complete. 2024-05-09T00:04:07.048 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:04:07.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.242+0000 7f22aa01d700 1 -- 172.21.15.12:0/104756645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22a40693e0 msgr2=0x7f22a40697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.242+0000 7f22aa01d700 1 --2- 172.21.15.12:0/104756645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22a40693e0 0x7f22a40697c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2294009990 tx=0x7f2294009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.242+0000 7f22aa01d700 1 -- 172.21.15.12:0/104756645 shutdown_connections 2024-05-09T00:04:07.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.242+0000 7f22aa01d700 1 --2- 172.21.15.12:0/104756645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22a41062a0 0x7f22a410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.242+0000 7f22aa01d700 1 --2- 172.21.15.12:0/104756645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22a40693e0 0x7f22a40697c0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.242+0000 7f22aa01d700 1 --2- 172.21.15.12:0/104756645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22a4069d90 0x7f22a4105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.242+0000 7f22aa01d700 1 -- 172.21.15.12:0/104756645 >> 172.21.15.12:0/104756645 conn(0x7f22a4076ba0 msgr2=0x7f22a4076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:07.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.244+0000 7f22aa01d700 1 -- 172.21.15.12:0/104756645 shutdown_connections 2024-05-09T00:04:07.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.244+0000 7f22aa01d700 1 -- 172.21.15.12:0/104756645 wait complete. 2024-05-09T00:04:07.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.245+0000 7f22aa01d700 1 Processor -- start 2024-05-09T00:04:07.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.245+0000 7f22aa01d700 1 -- start start 2024-05-09T00:04:07.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22aa01d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22a40693e0 0x7f22a4197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22aa01d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22a41062a0 0x7f22a41979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22aa01d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22a419bd60 0x7f22a419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22aa01d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419c7e0 con 0x7f22a41062a0 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22aa01d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419c950 con 0x7f22a40693e0 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22aa01d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419cb10 con 0x7f22a419bd60 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22a41062a0 0x7f22a41979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22a41062a0 0x7f22a41979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51994/0 (socket says 172.21.15.12:51994) 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a2ffd700 1 -- 172.21.15.12:0/4120637172 learned_addr learned my addr 172.21.15.12:0/4120637172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a37fe700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22a40693e0 0x7f22a4197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a3fff700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22a419bd60 0x7f22a419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a2ffd700 1 -- 172.21.15.12:0/4120637172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22a419bd60 msgr2=0x7f22a419c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22a419bd60 0x7f22a419c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a2ffd700 1 -- 172.21.15.12:0/4120637172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22a40693e0 msgr2=0x7f22a4197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22a40693e0 0x7f22a4197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a2ffd700 1 -- 172.21.15.12:0/4120637172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22940095f0 con 0x7f22a41062a0 2024-05-09T00:04:07.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a37fe700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22a40693e0 0x7f22a4197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:04:07.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.246+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22a41062a0 0x7f22a41979c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f229800b500 tx=0x7f229800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:07.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.247+0000 7f22a0ff9700 1 -- 172.21.15.12:0/4120637172 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298011830 con 0x7f22a41062a0 2024-05-09T00:04:07.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.247+0000 7f22aa01d700 1 -- 172.21.15.12:0/4120637172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a41a4070 con 0x7f22a41062a0 2024-05-09T00:04:07.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.247+0000 7f22aa01d700 1 -- 172.21.15.12:0/4120637172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a41a45a0 con 0x7f22a41062a0 2024-05-09T00:04:07.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.247+0000 7f22a0ff9700 1 -- 172.21.15.12:0/4120637172 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2298009330 con 0x7f22a41062a0 2024-05-09T00:04:07.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.247+0000 7f22a0ff9700 1 -- 172.21.15.12:0/4120637172 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229801ae90 con 0x7f22a41062a0 2024-05-09T00:04:07.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.249+0000 7f22a0ff9700 1 -- 172.21.15.12:0/4120637172 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f22980193b0 con 0x7f22a41062a0 2024-05-09T00:04:07.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.252+0000 7f22b61d7b80 1 Processor -- start 2024-05-09T00:04:07.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.252+0000 7f22a0ff9700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f228c0a1290 0x7f228c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.253+0000 7f22a0ff9700 1 -- 172.21.15.12:0/4120637172 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f22980cc660 con 0x7f22a41062a0 2024-05-09T00:04:07.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.253+0000 7f22b61d7b80 1 -- start start 2024-05-09T00:04:07.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.253+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb16e00 0x5614efb171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.253+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 0x5614efb11350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.253+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 0x5614efb11cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.253+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614efa68150 con 0x5614efb11890 2024-05-09T00:04:07.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.253+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614efa65b10 con 0x5614efb16e00 2024-05-09T00:04:07.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.253+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614efa61400 con 0x5614efb17720 2024-05-09T00:04:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.254+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 0x5614efb11350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.254+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 0x5614efb11350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54296/0 (socket says 172.21.15.12:54296) 2024-05-09T00:04:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.254+0000 7f22a2ffd700 1 -- 172.21.15.12:0/170209561 learned_addr learned my addr 172.21.15.12:0/170209561 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.254+0000 7f22a37fe700 1 --2- 172.21.15.12:0/170209561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb16e00 0x5614efb171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.254+0000 7f22a2ffd700 1 -- 172.21.15.12:0/170209561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb16e00 msgr2=0x5614efb171e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.254+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/170209561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb16e00 0x5614efb171e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.254+0000 7f22a2ffd700 1 -- 172.21.15.12:0/170209561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 msgr2=0x5614efb11cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.254+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/170209561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 0x5614efb11cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.254+0000 7f22a2ffd700 1 -- 172.21.15.12:0/170209561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614ef8e23b0 con 0x5614efb17720 2024-05-09T00:04:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.254+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/170209561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 0x5614efb11350 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2298097330 tx=0x7f2298098ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a0b08b94ca7dee6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.255+0000 7f22a3fff700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f228c0a1290 0x7f228c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.255+0000 7f2289ffb700 1 -- 172.21.15.12:0/170209561 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298011ba0 con 0x5614efb17720 2024-05-09T00:04:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.255+0000 7f2289ffb700 1 -- 172.21.15.12:0/170209561 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22980966b0 con 0x5614efb17720 2024-05-09T00:04:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.255+0000 7f2289ffb700 1 -- 172.21.15.12:0/170209561 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298005a50 con 0x5614efb17720 2024-05-09T00:04:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.256+0000 7f22b61d7b80 1 -- 172.21.15.12:0/170209561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 msgr2=0x5614efb11350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.256+0000 7f22a3fff700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f228c0a1290 0x7f228c0a3750 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f229c0093c0 tx=0x7f229c009920 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.256+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/170209561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 0x5614efb11350 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2298097330 tx=0x7f2298098ad0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.256+0000 7f22b61d7b80 1 -- 172.21.15.12:0/170209561 shutdown_connections 2024-05-09T00:04:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.256+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/170209561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 0x5614efb11cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.256+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/170209561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb16e00 0x5614efb171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.256+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/170209561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 0x5614efb11350 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.256+0000 7f22b61d7b80 1 -- 172.21.15.12:0/170209561 >> 172.21.15.12:0/170209561 conn(0x5614efa6f210 msgr2=0x5614efb0bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.256+0000 7f22b61d7b80 1 -- 172.21.15.12:0/170209561 shutdown_connections 2024-05-09T00:04:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.256+0000 7f22b61d7b80 1 -- 172.21.15.12:0/170209561 wait complete. 2024-05-09T00:04:07.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 171998 37.56 MB/sec execute 122 sec latency 78.515 ms 2024-05-09T00:04:07.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.257+0000 7f22b61d7b80 1 -- start start 2024-05-09T00:04:07.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.257+0000 7f22b61d7b80 1 Processor -- start 2024-05-09T00:04:07.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb11890 0x5614efb1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb16e00 0x5614efb1b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb17720 0x5614efb1b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614ef9fd560 con 0x5614efb17720 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614ef9fd200 con 0x5614efb16e00 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614ef8e46a0 con 0x5614efb11890 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22b61d7b80 10 client.0 _ll_register_callbacks cb 0x5614efa06b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb11890 0x5614efb1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb11890 0x5614efb1ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54308/0 (socket says 172.21.15.12:54308) 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22a37fe700 1 -- 172.21.15.12:0/2281359635 learned_addr learned my addr 172.21.15.12:0/2281359635 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22a37fe700 1 -- 172.21.15.12:0/2281359635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb16e00 msgr2=0x5614efb1b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22a37fe700 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb16e00 0x5614efb1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22a37fe700 1 -- 172.21.15.12:0/2281359635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb17720 msgr2=0x5614efb1b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:07.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22a37fe700 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb17720 0x5614efb1b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22a37fe700 1 -- 172.21.15.12:0/2281359635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a41a45a0 con 0x5614efb11890 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.258+0000 7f22a37fe700 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb11890 0x5614efb1ab40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2294009990 tx=0x7f229400bb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1a8d69e60a43ace server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.259+0000 7f2282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.259+0000 7f2282ffd700 1 -- 172.21.15.12:0/2281359635 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229400fa30 con 0x5614efb11890 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.259+0000 7f2282ffd700 1 -- 172.21.15.12:0/2281359635 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f229400fb90 con 0x5614efb11890 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.259+0000 7f22b61d7b80 1 -- 172.21.15.12:0/2281359635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614ef9ba270 con 0x5614efb11890 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.259+0000 7f2282ffd700 1 -- 172.21.15.12:0/2281359635 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f229400fd50 con 0x5614efb11890 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.259+0000 7f22b61d7b80 10 client.30115 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.259+0000 7f22b61d7b80 1 -- 172.21.15.12:0/2281359635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5614ef88eb60 con 0x5614efb11890 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.259+0000 7f2282ffd700 1 -- 172.21.15.12:0/2281359635 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614ef9ba270 con 0x5614efb11890 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.259+0000 7f2282ffd700 1 -- 172.21.15.12:0/2281359635 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2294044070 con 0x5614efb11890 2024-05-09T00:04:07.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.260+0000 7f22b61d7b80 10 client.30115 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:07.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.260+0000 7f22b61d7b80 10 client.30115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:07.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.260+0000 7f22b61d7b80 20 client.30115 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:07.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.260+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5614efb222a0 0x5614efb246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.260+0000 7f22b61d7b80 4 client.30115 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:07.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.260+0000 7f22b61d7b80 1 -- 172.21.15.12:0/2281359635 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614efae0810 con 0x5614efb222a0 2024-05-09T00:04:07.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.262+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5614efb222a0 0x5614efb246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.264+0000 7f2282ffd700 10 client.30115 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:07.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.264+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5614efb222a0 0x5614efb246c0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f1f8c0f23d591d server_cookie=edb6f24dd4e3ba85 in_seq=0 out_seq=0 2024-05-09T00:04:07.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f2282ffd700 1 -- 172.21.15.12:0/2281359635 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614efae0810 con 0x5614efb222a0 2024-05-09T00:04:07.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f2282ffd700 10 client.30115 handle_command_reply: tid=0 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 client.30115 shutdown 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 -- 172.21.15.12:0/2281359635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb11890 msgr2=0x5614efb1ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb11890 0x5614efb1ab40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2294009990 tx=0x7f229400bb80 comp rx=0 tx=0).stop 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 -- 172.21.15.12:0/2281359635 shutdown_connections 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb17720 0x5614efb1b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb11890 0x5614efb1ab40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5614efb222a0 0x5614efb246c0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/2281359635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb16e00 0x5614efb1b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 -- 172.21.15.12:0/2281359635 >> 172.21.15.12:0/2281359635 conn(0x5614efa6f210 msgr2=0x5614efb0b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f2282ffd700 0 client.30115 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 -- 172.21.15.12:0/2281359635 shutdown_connections 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 1 -- 172.21.15.12:0/2281359635 wait complete. 2024-05-09T00:04:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.272+0000 7f22b61d7b80 20 client.30115 trim_cache size 0 max 16384 2024-05-09T00:04:07.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22b61d7b80 1 Processor -- start 2024-05-09T00:04:07.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22b61d7b80 1 -- start start 2024-05-09T00:04:07.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb11890 0x5614efb11350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb16e00 0x5614efb0bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 0x5614efb1b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614efb08df0 con 0x5614efb16e00 2024-05-09T00:04:07.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614efb1c180 con 0x5614efb11890 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614efb06800 con 0x5614efb17720 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb16e00 0x5614efb0bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb16e00 0x5614efb0bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52030/0 (socket says 172.21.15.12:52030) 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22a2ffd700 1 -- 172.21.15.12:0/1627107882 learned_addr learned my addr 172.21.15.12:0/1627107882 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.290+0000 7f22a3fff700 1 --2- 172.21.15.12:0/1627107882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 0x5614efb1b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.291+0000 7f22a2ffd700 1 -- 172.21.15.12:0/1627107882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 msgr2=0x5614efb1b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.291+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/1627107882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 0x5614efb1b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.291+0000 7f22a2ffd700 1 -- 172.21.15.12:0/1627107882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb11890 msgr2=0x5614efb11350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.291+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/1627107882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb11890 0x5614efb11350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.291+0000 7f22a2ffd700 1 -- 172.21.15.12:0/1627107882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614efb11f40 con 0x5614efb16e00 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.291+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/1627107882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb16e00 0x5614efb0bd20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f229800bcc0 tx=0x7f22980a1eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289ab90269666f2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.291+0000 7f228a7fc700 1 -- 172.21.15.12:0/1627107882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980b2120 con 0x5614efb16e00 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.291+0000 7f228a7fc700 1 -- 172.21.15.12:0/1627107882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22980b2280 con 0x5614efb16e00 2024-05-09T00:04:07.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.291+0000 7f228a7fc700 1 -- 172.21.15.12:0/1627107882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980b93a0 con 0x5614efb16e00 2024-05-09T00:04:07.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.292+0000 7f22b61d7b80 1 -- 172.21.15.12:0/1627107882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb16e00 msgr2=0x5614efb0bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.292+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/1627107882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb16e00 0x5614efb0bd20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f229800bcc0 tx=0x7f22980a1eb0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.292+0000 7f22b61d7b80 1 -- 172.21.15.12:0/1627107882 shutdown_connections 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.292+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/1627107882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb17720 0x5614efb1b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.292+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/1627107882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb11890 0x5614efb11350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.292+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/1627107882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb16e00 0x5614efb0bd20 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.292+0000 7f22b61d7b80 1 -- 172.21.15.12:0/1627107882 >> 172.21.15.12:0/1627107882 conn(0x5614efa6f210 msgr2=0x5614efb19450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.292+0000 7f22b61d7b80 1 -- 172.21.15.12:0/1627107882 shutdown_connections 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.292+0000 7f22b61d7b80 1 -- 172.21.15.12:0/1627107882 wait complete. 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.293+0000 7f22b61d7b80 1 -- start start 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.293+0000 7f22b61d7b80 1 Processor -- start 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.293+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 0x5614efb7c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.293+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb16e00 0x5614efb7c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.293+0000 7f22b61d7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb17720 0x5614efb27020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.293+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614efb7d050 con 0x5614efb11890 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.293+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614efb27560 con 0x5614efb17720 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.293+0000 7f22b61d7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614efb276f0 con 0x5614efb16e00 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.293+0000 7f22b61d7b80 10 client.0 _ll_register_callbacks cb 0x5614efa06b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 0x5614efb7c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 0x5614efb7c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52042/0 (socket says 172.21.15.12:52042) 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a37fe700 1 -- 172.21.15.12:0/4012862861 learned_addr learned my addr 172.21.15.12:0/4012862861 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a3fff700 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb17720 0x5614efb27020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb16e00 0x5614efb7c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a2ffd700 1 -- 172.21.15.12:0/4012862861 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb17720 msgr2=0x5614efb27020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb17720 0x5614efb27020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a2ffd700 1 -- 172.21.15.12:0/4012862861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 msgr2=0x5614efb7c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 0x5614efb7c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a2ffd700 1 -- 172.21.15.12:0/4012862861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614ef8e23b0 con 0x5614efb16e00 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a37fe700 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 0x5614efb7c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f22a2ffd700 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb16e00 0x5614efb7c740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x5614efad6d40 tx=0x7f229801a880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb668ad6a4b2f491 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.294+0000 7f2282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f2282ffd700 1 -- 172.21.15.12:0/4012862861 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980b92b0 con 0x5614efb16e00 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f2282ffd700 1 -- 172.21.15.12:0/4012862861 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f2298005a50 con 0x5614efb16e00 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f22b61d7b80 1 -- 172.21.15.12:0/4012862861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22980966b0 con 0x5614efb16e00 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f2282ffd700 1 -- 172.21.15.12:0/4012862861 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2298011ca0 con 0x5614efb16e00 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f22b61d7b80 10 client.30121 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f22b61d7b80 1 -- 172.21.15.12:0/4012862861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5614efb084e0 con 0x5614efb16e00 2024-05-09T00:04:07.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f2282ffd700 1 -- 172.21.15.12:0/4012862861 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22980966b0 con 0x5614efb16e00 2024-05-09T00:04:07.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f2282ffd700 1 -- 172.21.15.12:0/4012862861 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f22980960d0 con 0x5614efb16e00 2024-05-09T00:04:07.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f22b61d7b80 10 client.30121 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:07.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f22b61d7b80 10 client.30121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:07.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f22b61d7b80 20 client.30121 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:07.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5614efb92b20 0x5614efb94f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:07.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f22b61d7b80 4 client.30121 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:04:07.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.295+0000 7f22b61d7b80 1 -- 172.21.15.12:0/4012862861 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614efa59ec0 con 0x5614efb92b20 2024-05-09T00:04:07.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.297+0000 7f22a37fe700 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5614efb92b20 0x5614efb94f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.297+0000 7f22a37fe700 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5614efb92b20 0x5614efb94f10 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e7cb354ffe7fc2 server_cookie=a832551160174547 in_seq=0 out_seq=0 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.297+0000 7f2282ffd700 10 client.30121 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f2282ffd700 1 -- 172.21.15.12:0/4012862861 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614efa59ec0 con 0x5614efb92b20 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f2282ffd700 10 client.30121 handle_command_reply: tid=0 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 client.30121 shutdown 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 -- 172.21.15.12:0/4012862861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb16e00 msgr2=0x5614efb7c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb16e00 0x5614efb7c740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x5614efad6d40 tx=0x7f229801a880 comp rx=0 tx=0).stop 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 -- 172.21.15.12:0/4012862861 shutdown_connections 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5614efb92b20 0x5614efb94f10 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5614efb17720 0x5614efb27020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efb11890 0x5614efb7c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f2282ffd700 0 client.30121 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 --2- 172.21.15.12:0/4012862861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5614efb16e00 0x5614efb7c740 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 -- 172.21.15.12:0/4012862861 >> 172.21.15.12:0/4012862861 conn(0x5614efa6f210 msgr2=0x5614efb19040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:07.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 -- 172.21.15.12:0/4012862861 shutdown_connections 2024-05-09T00:04:07.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 1 -- 172.21.15.12:0/4012862861 wait complete. 2024-05-09T00:04:07.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.300+0000 7f22b61d7b80 20 client.30121 trim_cache size 0 max 16384 2024-05-09T00:04:07.296 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:04:07.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 -- 172.21.15.12:0/4120637172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f228c0a1290 msgr2=0x7f228c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f228c0a1290 0x7f228c0a3750 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f229c0093c0 tx=0x7f229c009920 comp rx=0 tx=0).stop 2024-05-09T00:04:07.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 -- 172.21.15.12:0/4120637172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22a41062a0 msgr2=0x7f22a41979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:07.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22a41062a0 0x7f22a41979c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f229800b500 tx=0x7f229800b8c0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 -- 172.21.15.12:0/4120637172 shutdown_connections 2024-05-09T00:04:07.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f228c0a1290 0x7f228c0a3750 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22a419bd60 0x7f22a419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22a40693e0 0x7f22a4197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 --2- 172.21.15.12:0/4120637172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22a41062a0 0x7f22a41979c0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:07.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 -- 172.21.15.12:0/4120637172 >> 172.21.15.12:0/4120637172 conn(0x7f22a4076ba0 msgr2=0x7f22a40ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:07.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 -- 172.21.15.12:0/4120637172 shutdown_connections 2024-05-09T00:04:07.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:07.301+0000 7f22aa01d700 1 -- 172.21.15.12:0/4120637172 wait complete. 2024-05-09T00:04:08.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:07 smithi204 ceph-mon[47373]: pgmap v571: 97 pgs: 5 active+clean+snaptrim, 81 active+clean, 11 active+clean+snaptrim_wait; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 883 op/s 2024-05-09T00:04:08.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:07 smithi160 ceph-mon[46152]: pgmap v571: 97 pgs: 5 active+clean+snaptrim, 81 active+clean, 11 active+clean+snaptrim_wait; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 883 op/s 2024-05-09T00:04:08.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:07 smithi012 ceph-mon[42503]: pgmap v571: 97 pgs: 5 active+clean+snaptrim, 81 active+clean, 11 active+clean+snaptrim_wait; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 883 op/s 2024-05-09T00:04:08.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 172677 37.56 MB/sec execute 123 sec latency 781.579 ms 2024-05-09T00:04:08.314 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:04:08.315 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:04:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.489+0000 7f482183d700 1 -- 172.21.15.12:0/4050217757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f481c069d90 msgr2=0x7f481c105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.489+0000 7f482183d700 1 --2- 172.21.15.12:0/4050217757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f481c069d90 0x7f481c105b70 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f4810009990 tx=0x7f4810009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.489+0000 7f482183d700 1 -- 172.21.15.12:0/4050217757 shutdown_connections 2024-05-09T00:04:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.489+0000 7f482183d700 1 --2- 172.21.15.12:0/4050217757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481c1062a0 0x7f481c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.489+0000 7f482183d700 1 --2- 172.21.15.12:0/4050217757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f481c0693e0 0x7f481c0697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.489+0000 7f482183d700 1 --2- 172.21.15.12:0/4050217757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f481c069d90 0x7f481c105b70 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.489+0000 7f482183d700 1 -- 172.21.15.12:0/4050217757 >> 172.21.15.12:0/4050217757 conn(0x7f481c076ba0 msgr2=0x7f481c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.489+0000 7f482183d700 1 -- 172.21.15.12:0/4050217757 shutdown_connections 2024-05-09T00:04:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.489+0000 7f482183d700 1 -- 172.21.15.12:0/4050217757 wait complete. 2024-05-09T00:04:08.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.490+0000 7f482183d700 1 Processor -- start 2024-05-09T00:04:08.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.490+0000 7f482183d700 1 -- start start 2024-05-09T00:04:08.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.490+0000 7f482183d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f481c0693e0 0x7f481c1001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.490+0000 7f482183d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f481c1062a0 0x7f481c100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.490+0000 7f482183d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481c100c40 0x7f481c1a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.490+0000 7f482183d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c1a5be0 con 0x7f481c1062a0 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.490+0000 7f482183d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c1a5d50 con 0x7f481c0693e0 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.490+0000 7f482183d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c1a5f10 con 0x7f481c100c40 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f481c1062a0 0x7f481c100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f481c0693e0 0x7f481c1001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f481c1062a0 0x7f481c100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57572/0 (socket says 172.21.15.12:57572) 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481a7fc700 1 -- 172.21.15.12:0/2037516476 learned_addr learned my addr 172.21.15.12:0/2037516476 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481b7fe700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481c100c40 0x7f481c1a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481affd700 1 -- 172.21.15.12:0/2037516476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481c100c40 msgr2=0x7f481c1a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481affd700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481c100c40 0x7f481c1a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481affd700 1 -- 172.21.15.12:0/2037516476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f481c1062a0 msgr2=0x7f481c100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481affd700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f481c1062a0 0x7f481c100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481affd700 1 -- 172.21.15.12:0/2037516476 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100095f0 con 0x7f481c0693e0 2024-05-09T00:04:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481a7fc700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f481c1062a0 0x7f481c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:08.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.491+0000 7f481affd700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f481c0693e0 0x7f481c1001c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f480c00e9f0 tx=0x7f480c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.493+0000 7f482083b700 1 -- 172.21.15.12:0/2037516476 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c00c9c0 con 0x7f481c0693e0 2024-05-09T00:04:08.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.493+0000 7f482083b700 1 -- 172.21.15.12:0/2037516476 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f480c010430 con 0x7f481c0693e0 2024-05-09T00:04:08.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.493+0000 7f482183d700 1 -- 172.21.15.12:0/2037516476 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c1a61f0 con 0x7f481c0693e0 2024-05-09T00:04:08.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.493+0000 7f482083b700 1 -- 172.21.15.12:0/2037516476 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c018680 con 0x7f481c0693e0 2024-05-09T00:04:08.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.493+0000 7f482183d700 1 -- 172.21.15.12:0/2037516476 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c1a6750 con 0x7f481c0693e0 2024-05-09T00:04:08.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.496+0000 7f482083b700 1 -- 172.21.15.12:0/2037516476 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f480c010860 con 0x7f481c0693e0 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.497+0000 7f482d9f7b80 1 Processor -- start 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.497+0000 7f482083b700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48040a1310 0x7f48040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.497+0000 7f482d9f7b80 1 -- start start 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.497+0000 7f481b7fe700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48040a1310 0x7f48040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.497+0000 7f482083b700 1 -- 172.21.15.12:0/2037516476 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f480c014070 con 0x7f481c0693e0 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.497+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 0x561500c75260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.497+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c757a0 0x561500c6f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c6f910 0x561500c6fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500bc61d0 con 0x561500c757a0 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500bc3b90 con 0x561500c74e80 2024-05-09T00:04:08.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500bbf480 con 0x561500c6f910 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 0x561500c75260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c6f910 0x561500c6fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 0x561500c75260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50806/0 (socket says 172.21.15.12:50806) 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481affd700 1 -- 172.21.15.12:0/1679245417 learned_addr learned my addr 172.21.15.12:0/1679245417 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481a7fc700 1 --2- 172.21.15.12:0/1679245417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c757a0 0x561500c6f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481b7fe700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48040a1310 0x7f48040a37d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f481400f570 tx=0x7f4814009920 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481affd700 1 -- 172.21.15.12:0/1679245417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c6f910 msgr2=0x561500c6fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481affd700 1 --2- 172.21.15.12:0/1679245417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c6f910 0x561500c6fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481affd700 1 -- 172.21.15.12:0/1679245417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c757a0 msgr2=0x561500c6f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481affd700 1 --2- 172.21.15.12:0/1679245417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c757a0 0x561500c6f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481affd700 1 -- 172.21.15.12:0/1679245417 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561500a40430 con 0x561500c74e80 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481b7fe700 1 --2- 172.21.15.12:0/1679245417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c6f910 0x561500c6fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.498+0000 7f481affd700 1 --2- 172.21.15.12:0/1679245417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 0x561500c75260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f480c0965d0 tx=0x7f480c099a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaa2705cf27aa6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.499+0000 7f4801ffb700 1 -- 172.21.15.12:0/1679245417 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c0b1070 con 0x561500c74e80 2024-05-09T00:04:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.499+0000 7f4801ffb700 1 -- 172.21.15.12:0/1679245417 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f480c09a0d0 con 0x561500c74e80 2024-05-09T00:04:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.499+0000 7f4801ffb700 1 -- 172.21.15.12:0/1679245417 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c09a330 con 0x561500c74e80 2024-05-09T00:04:08.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.499+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1679245417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 msgr2=0x561500c75260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.499+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/1679245417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 0x561500c75260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f480c0965d0 tx=0x7f480c099a30 comp rx=0 tx=0).stop 2024-05-09T00:04:08.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.499+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1679245417 shutdown_connections 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.499+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/1679245417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c6f910 0x561500c6fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.499+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/1679245417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 0x561500c75260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.499+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/1679245417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c757a0 0x561500c6f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.499+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1679245417 >> 172.21.15.12:0/1679245417 conn(0x561500bcd290 msgr2=0x561500c69e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.500+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1679245417 shutdown_connections 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.500+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1679245417 wait complete. 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.500+0000 7f482d9f7b80 1 -- start start 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.500+0000 7f482d9f7b80 1 Processor -- start 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c6f910 0x561500c78b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c74e80 0x561500c79050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c757a0 0x561500c79590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500b5b5e0 con 0x561500c757a0 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500b5b280 con 0x561500c6f910 2024-05-09T00:04:08.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500a42720 con 0x561500c74e80 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f482d9f7b80 10 client.0 _ll_register_callbacks cb 0x561500b64bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f481affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c6f910 0x561500c78b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f481affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c6f910 0x561500c78b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50818/0 (socket says 172.21.15.12:50818) 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f481affd700 1 -- 172.21.15.12:0/630060505 learned_addr learned my addr 172.21.15.12:0/630060505 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f481b7fe700 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c757a0 0x561500c79590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f481affd700 1 -- 172.21.15.12:0/630060505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c74e80 msgr2=0x561500c79050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f481affd700 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c74e80 0x561500c79050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f481affd700 1 -- 172.21.15.12:0/630060505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c757a0 msgr2=0x561500c79590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f481affd700 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c757a0 0x561500c79590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f481affd700 1 -- 172.21.15.12:0/630060505 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4814019460 con 0x561500c6f910 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f481affd700 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c6f910 0x561500c78b10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c5b0 tx=0x7f480c0092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d8707a461f32c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.501+0000 7f47faffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:04:08.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f47faffd700 1 -- 172.21.15.12:0/630060505 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c0b1070 con 0x561500c6f910 2024-05-09T00:04:08.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f47faffd700 1 -- 172.21.15.12:0/630060505 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f480c09df00 con 0x561500c6f910 2024-05-09T00:04:08.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f482d9f7b80 1 -- 172.21.15.12:0/630060505 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561500b182f0 con 0x561500c6f910 2024-05-09T00:04:08.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f47faffd700 1 -- 172.21.15.12:0/630060505 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f480c0a2550 con 0x561500c6f910 2024-05-09T00:04:08.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f482d9f7b80 10 client.30005 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:08.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f482d9f7b80 1 -- 172.21.15.12:0/630060505 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7f481c1a6750 con 0x561500c6f910 2024-05-09T00:04:08.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f47faffd700 1 -- 172.21.15.12:0/630060505 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561500b182f0 con 0x561500c6f910 2024-05-09T00:04:08.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f47faffd700 1 -- 172.21.15.12:0/630060505 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f480c07c060 con 0x561500c6f910 2024-05-09T00:04:08.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f482d9f7b80 10 client.30005 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:08.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f482d9f7b80 10 client.30005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:08.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f482d9f7b80 20 client.30005 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561500c88ec0 0x561500c8b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f482d9f7b80 4 client.30005 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f482d9f7b80 1 -- 172.21.15.12:0/630060505 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561500c3e890 con 0x561500c88ec0 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.502+0000 7f481a7fc700 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561500c88ec0 0x561500c8b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.503+0000 7f47faffd700 10 client.30005 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.503+0000 7f481a7fc700 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561500c88ec0 0x561500c8b2e0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e329ed2a1e8429f server_cookie=54a2c207c05a9f9 in_seq=0 out_seq=0 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.503+0000 7f47faffd700 1 -- 172.21.15.12:0/630060505 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561500c3e890 con 0x561500c88ec0 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.503+0000 7f47faffd700 10 client.30005 handle_command_reply: tid=0 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.503+0000 7f482d9f7b80 1 client.30005 shutdown 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 1 -- 172.21.15.12:0/630060505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c6f910 msgr2=0x561500c78b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c6f910 0x561500c78b10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c5b0 tx=0x7f480c0092d0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 1 -- 172.21.15.12:0/630060505 shutdown_connections 2024-05-09T00:04:08.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c757a0 0x561500c79590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561500c88ec0 0x561500c8b2e0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c6f910 0x561500c78b10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/630060505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c74e80 0x561500c79050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 1 -- 172.21.15.12:0/630060505 >> 172.21.15.12:0/630060505 conn(0x561500bcd290 msgr2=0x561500c69620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f47faffd700 0 client.30005 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:08.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 1 -- 172.21.15.12:0/630060505 shutdown_connections 2024-05-09T00:04:08.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 1 -- 172.21.15.12:0/630060505 wait complete. 2024-05-09T00:04:08.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.504+0000 7f482d9f7b80 20 client.30005 trim_cache size 0 max 16384 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.525+0000 7f482d9f7b80 1 Processor -- start 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.525+0000 7f482d9f7b80 1 -- start start 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.525+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c74490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c74e80 0x561500c69da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c757a0 0x561500c77af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500c904f0 con 0x561500c6f910 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500c64880 con 0x561500c757a0 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500c77060 con 0x561500c74e80 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c74490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c74490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57608/0 (socket says 172.21.15.12:57608) 2024-05-09T00:04:08.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481affd700 1 -- 172.21.15.12:0/2276378099 learned_addr learned my addr 172.21.15.12:0/2276378099 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481b7fe700 1 --2- 172.21.15.12:0/2276378099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c757a0 0x561500c77af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481a7fc700 1 --2- 172.21.15.12:0/2276378099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c74e80 0x561500c69da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481affd700 1 -- 172.21.15.12:0/2276378099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c74e80 msgr2=0x561500c69da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481affd700 1 --2- 172.21.15.12:0/2276378099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c74e80 0x561500c69da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481affd700 1 -- 172.21.15.12:0/2276378099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c757a0 msgr2=0x561500c77af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481affd700 1 --2- 172.21.15.12:0/2276378099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c757a0 0x561500c77af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481affd700 1 -- 172.21.15.12:0/2276378099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561500c6ffc0 con 0x561500c6f910 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481a7fc700 1 --2- 172.21.15.12:0/2276378099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c74e80 0x561500c69da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f481affd700 1 --2- 172.21.15.12:0/2276378099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c74490 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f480c098420 tx=0x7f480c005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6873025f0df12d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f48027fc700 1 -- 172.21.15.12:0/2276378099 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c0b1070 con 0x561500c6f910 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f48027fc700 1 -- 172.21.15.12:0/2276378099 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f480c09df00 con 0x561500c6f910 2024-05-09T00:04:08.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.526+0000 7f48027fc700 1 -- 172.21.15.12:0/2276378099 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c09a330 con 0x561500c6f910 2024-05-09T00:04:08.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.527+0000 7f482d9f7b80 1 -- 172.21.15.12:0/2276378099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 msgr2=0x561500c74490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.527+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/2276378099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c74490 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f480c098420 tx=0x7f480c005bf0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.527+0000 7f482d9f7b80 1 -- 172.21.15.12:0/2276378099 shutdown_connections 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.527+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/2276378099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c757a0 0x561500c77af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.527+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/2276378099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c74490 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.527+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/2276378099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c74e80 0x561500c69da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.527+0000 7f482d9f7b80 1 -- 172.21.15.12:0/2276378099 >> 172.21.15.12:0/2276378099 conn(0x561500bcd290 msgr2=0x561500c773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.527+0000 7f482d9f7b80 1 -- 172.21.15.12:0/2276378099 shutdown_connections 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.527+0000 7f482d9f7b80 1 -- 172.21.15.12:0/2276378099 wait complete. 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f482d9f7b80 1 -- start start 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f482d9f7b80 1 Processor -- start 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c7cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 0x561500c7d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f482d9f7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c757a0 0x561500cdf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500c7dc90 con 0x561500c6f910 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500c7ddd0 con 0x561500c74e80 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f482d9f7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561500cdf900 con 0x561500c757a0 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f482d9f7b80 10 client.0 _ll_register_callbacks cb 0x561500b64bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f481affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c7cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f481affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c7cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57618/0 (socket says 172.21.15.12:57618) 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.528+0000 7f481affd700 1 -- 172.21.15.12:0/1429786623 learned_addr learned my addr 172.21.15.12:0/1429786623 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f481b7fe700 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c757a0 0x561500cdf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f481a7fc700 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 0x561500c7d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f481affd700 1 -- 172.21.15.12:0/1429786623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c757a0 msgr2=0x561500cdf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f481affd700 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c757a0 0x561500cdf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f481affd700 1 -- 172.21.15.12:0/1429786623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 msgr2=0x561500c7d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f481affd700 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 0x561500c7d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f481affd700 1 -- 172.21.15.12:0/1429786623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c1a6750 con 0x561500c6f910 2024-05-09T00:04:08.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f481affd700 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c7cfa0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f480c099ba0 tx=0x7f480c09a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b8245f15d77a71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f47faffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f47faffd700 1 -- 172.21.15.12:0/1429786623 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c0b1070 con 0x561500c6f910 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f47faffd700 1 -- 172.21.15.12:0/1429786623 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f480c096100 con 0x561500c6f910 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.529+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1429786623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f480c09a0d0 con 0x561500c6f910 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f47faffd700 1 -- 172.21.15.12:0/1429786623 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f480c095f10 con 0x561500c6f910 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f482d9f7b80 10 client.20406 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1429786623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x561500c66560 con 0x561500c6f910 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f47faffd700 1 -- 172.21.15.12:0/1429786623 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f480c09a0d0 con 0x561500c6f910 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f47faffd700 1 -- 172.21.15.12:0/1429786623 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f480c0a9070 con 0x561500c6f910 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f482d9f7b80 10 client.20406 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f482d9f7b80 10 client.20406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f482d9f7b80 20 client.20406 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561500ce0c60 0x561500ce3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f482d9f7b80 4 client.20406 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:04:08.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.530+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1429786623 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561500bb7f40 con 0x561500ce0c60 2024-05-09T00:04:08.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.531+0000 7f481a7fc700 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561500ce0c60 0x561500ce3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.531+0000 7f47faffd700 10 client.20406 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:08.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.531+0000 7f481a7fc700 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561500ce0c60 0x561500ce3050 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d45de0f06d418158 server_cookie=2e58752ca9885733 in_seq=0 out_seq=0 2024-05-09T00:04:08.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.532+0000 7f47faffd700 1 -- 172.21.15.12:0/1429786623 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561500bb7f40 con 0x561500ce0c60 2024-05-09T00:04:08.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.532+0000 7f47faffd700 10 client.20406 handle_command_reply: tid=0 2024-05-09T00:04:08.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.532+0000 7f482d9f7b80 1 client.20406 shutdown 2024-05-09T00:04:08.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1429786623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 msgr2=0x561500c7cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c7cfa0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f480c099ba0 tx=0x7f480c09a560 comp rx=0 tx=0).stop 2024-05-09T00:04:08.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1429786623 shutdown_connections 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561500c757a0 0x561500cdf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561500c6f910 0x561500c7cfa0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561500ce0c60 0x561500ce3050 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 1 --2- 172.21.15.12:0/1429786623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561500c74e80 0x561500c7d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1429786623 >> 172.21.15.12:0/1429786623 conn(0x561500bcd290 msgr2=0x561500c764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f47faffd700 0 client.20406 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1429786623 shutdown_connections 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 1 -- 172.21.15.12:0/1429786623 wait complete. 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.533+0000 7f482d9f7b80 20 client.20406 trim_cache size 0 max 16384 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:04:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:04:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "28a9d5e6-d01b-4e79-8e51-d34d4a9646b9", 2024-05-09T00:04:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:04:08.532 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:04:08.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 -- 172.21.15.12:0/2037516476 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48040a1310 msgr2=0x7f48040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48040a1310 0x7f48040a37d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f481400f570 tx=0x7f4814009920 comp rx=0 tx=0).stop 2024-05-09T00:04:08.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 -- 172.21.15.12:0/2037516476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f481c0693e0 msgr2=0x7f481c1001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f481c0693e0 0x7f481c1001c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f480c00e9f0 tx=0x7f480c00ed00 comp rx=0 tx=0).stop 2024-05-09T00:04:08.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 -- 172.21.15.12:0/2037516476 shutdown_connections 2024-05-09T00:04:08.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48040a1310 0x7f48040a37d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f481c100c40 0x7f481c1a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f481c0693e0 0x7f481c1001c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 --2- 172.21.15.12:0/2037516476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f481c1062a0 0x7f481c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 -- 172.21.15.12:0/2037516476 >> 172.21.15.12:0/2037516476 conn(0x7f481c076ba0 msgr2=0x7f481c107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 -- 172.21.15.12:0/2037516476 shutdown_connections 2024-05-09T00:04:08.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.534+0000 7f482183d700 1 -- 172.21.15.12:0/2037516476 wait complete. 2024-05-09T00:04:08.543 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:04:08.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.727+0000 7f404da78700 1 -- 172.21.15.12:0/95274229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4048103d20 msgr2=0x7f4048107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.727+0000 7f404da78700 1 --2- 172.21.15.12:0/95274229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4048103d20 0x7f4048107d60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f403c009990 tx=0x7f403c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.727+0000 7f404da78700 1 -- 172.21.15.12:0/95274229 shutdown_connections 2024-05-09T00:04:08.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.727+0000 7f404da78700 1 --2- 172.21.15.12:0/95274229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4048108490 0x7f4048110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.727+0000 7f404da78700 1 --2- 172.21.15.12:0/95274229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4048103370 0x7f4048103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.727+0000 7f404da78700 1 --2- 172.21.15.12:0/95274229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4048103d20 0x7f4048107d60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.727+0000 7f404da78700 1 -- 172.21.15.12:0/95274229 >> 172.21.15.12:0/95274229 conn(0x7f40480febb0 msgr2=0x7f4048100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.727+0000 7f404da78700 1 -- 172.21.15.12:0/95274229 shutdown_connections 2024-05-09T00:04:08.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.727+0000 7f404da78700 1 -- 172.21.15.12:0/95274229 wait complete. 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.728+0000 7f404da78700 1 Processor -- start 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.728+0000 7f404da78700 1 -- start start 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f404da78700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4048103370 0x7f4048199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f404da78700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4048108490 0x7f4048199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f404da78700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404819deb0 0x7f404819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f404da78700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404819e930 con 0x7f4048108490 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f404da78700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404819eaa0 con 0x7f4048103370 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f404da78700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404819ec60 con 0x7f404819deb0 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f40477fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4048103370 0x7f4048199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f4047fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404819deb0 0x7f404819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f4047fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404819deb0 0x7f404819e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43128/0 (socket says 172.21.15.12:43128) 2024-05-09T00:04:08.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f40477fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4048103370 0x7f4048199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50852/0 (socket says 172.21.15.12:50852) 2024-05-09T00:04:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f4047fff700 1 -- 172.21.15.12:0/4175513859 learned_addr learned my addr 172.21.15.12:0/4175513859 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f4046ffd700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4048108490 0x7f4048199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f4047fff700 1 -- 172.21.15.12:0/4175513859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4048103370 msgr2=0x7f4048199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f4047fff700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4048103370 0x7f4048199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f4047fff700 1 -- 172.21.15.12:0/4175513859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4048108490 msgr2=0x7f4048199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f4047fff700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4048108490 0x7f4048199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f4047fff700 1 -- 172.21.15.12:0/4175513859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0095f0 con 0x7f404819deb0 2024-05-09T00:04:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.729+0000 7f40477fe700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4048103370 0x7f4048199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.730+0000 7f4047fff700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404819deb0 0x7f404819e330 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f404000ef30 tx=0x7f404000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.730+0000 7f4044ff9700 1 -- 172.21.15.12:0/4175513859 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040018070 con 0x7f404819deb0 2024-05-09T00:04:08.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.730+0000 7f4044ff9700 1 -- 172.21.15.12:0/4175513859 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404000f040 con 0x7f404819deb0 2024-05-09T00:04:08.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.730+0000 7f404da78700 1 -- 172.21.15.12:0/4175513859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40481a6160 con 0x7f404819deb0 2024-05-09T00:04:08.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.730+0000 7f4044ff9700 1 -- 172.21.15.12:0/4175513859 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404000cd00 con 0x7f404819deb0 2024-05-09T00:04:08.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.730+0000 7f404da78700 1 -- 172.21.15.12:0/4175513859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40481a6640 con 0x7f404819deb0 2024-05-09T00:04:08.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.732+0000 7f4044ff9700 1 -- 172.21.15.12:0/4175513859 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4040003a00 con 0x7f404819deb0 2024-05-09T00:04:08.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.733+0000 7f4044ff9700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f40300a1310 0x7f40300a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.733+0000 7f4044ff9700 1 -- 172.21.15.12:0/4175513859 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f40400cc780 con 0x7f404819deb0 2024-05-09T00:04:08.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.733+0000 7f40477fe700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f40300a1310 0x7f40300a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.734+0000 7f40477fe700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f40300a1310 0x7f40300a37d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4038011490 tx=0x7f4038009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.735+0000 7f4059c32b80 1 Processor -- start 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.735+0000 7f4059c32b80 1 -- start start 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.735+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.735+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c4720 0x55a41e3be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.735+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 0x55a41e3becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f4059c32b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e315150 con 0x55a41e3c4720 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f4059c32b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e312b10 con 0x55a41e3c3e00 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f4059c32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e30e400 con 0x55a41e3be890 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f4046ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c4720 0x55a41e3be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f40477fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f4047fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 0x55a41e3becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f40477fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50862/0 (socket says 172.21.15.12:50862) 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f40477fe700 1 -- 172.21.15.12:0/1307240609 learned_addr learned my addr 172.21.15.12:0/1307240609 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f4046ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c4720 0x55a41e3be350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57646/0 (socket says 172.21.15.12:57646) 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f40477fe700 1 -- 172.21.15.12:0/1307240609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 msgr2=0x55a41e3becd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f40477fe700 1 --2- 172.21.15.12:0/1307240609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 0x55a41e3becd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f40477fe700 1 -- 172.21.15.12:0/1307240609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c4720 msgr2=0x55a41e3be350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f40477fe700 1 --2- 172.21.15.12:0/1307240609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c4720 0x55a41e3be350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f40477fe700 1 -- 172.21.15.12:0/1307240609 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a41e18f3b0 con 0x55a41e3c3e00 2024-05-09T00:04:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f4047fff700 1 --2- 172.21.15.12:0/1307240609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 0x55a41e3becd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.736+0000 7f40477fe700 1 --2- 172.21.15.12:0/1307240609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3c41e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f40380043c0 tx=0x7f4038014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa530bee5b205159 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.737+0000 7f402dffb700 1 -- 172.21.15.12:0/1307240609 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038003d10 con 0x55a41e3c3e00 2024-05-09T00:04:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.737+0000 7f402dffb700 1 -- 172.21.15.12:0/1307240609 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4038003e70 con 0x55a41e3c3e00 2024-05-09T00:04:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.737+0000 7f402dffb700 1 -- 172.21.15.12:0/1307240609 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403801e780 con 0x55a41e3c3e00 2024-05-09T00:04:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.738+0000 7f4059c32b80 1 -- 172.21.15.12:0/1307240609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 msgr2=0x55a41e3c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.738+0000 7f4059c32b80 1 --2- 172.21.15.12:0/1307240609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3c41e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f40380043c0 tx=0x7f4038014000 comp rx=0 tx=0).stop 2024-05-09T00:04:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.738+0000 7f4059c32b80 1 -- 172.21.15.12:0/1307240609 shutdown_connections 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.738+0000 7f4059c32b80 1 --2- 172.21.15.12:0/1307240609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 0x55a41e3becd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.738+0000 7f4059c32b80 1 --2- 172.21.15.12:0/1307240609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3c41e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.738+0000 7f4059c32b80 1 --2- 172.21.15.12:0/1307240609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c4720 0x55a41e3be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.738+0000 7f4059c32b80 1 -- 172.21.15.12:0/1307240609 >> 172.21.15.12:0/1307240609 conn(0x55a41e31c210 msgr2=0x55a41e3b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.738+0000 7f4059c32b80 1 -- 172.21.15.12:0/1307240609 shutdown_connections 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.738+0000 7f4059c32b80 1 -- 172.21.15.12:0/1307240609 wait complete. 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.738+0000 7f4059c32b80 1 -- start start 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f4059c32b80 1 Processor -- start 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 0x55a41e3c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 0x55a41e3c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c4720 0x55a41e3c8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f4059c32b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e2aa560 con 0x55a41e3c3e00 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f4059c32b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e2aa200 con 0x55a41e3c4720 2024-05-09T00:04:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f4059c32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e1916a0 con 0x55a41e3be890 2024-05-09T00:04:08.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f4059c32b80 10 client.0 _ll_register_callbacks cb 0x55a41e2b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:08.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f40477fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 0x55a41e3c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f4046ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 0x55a41e3c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f40477fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 0x55a41e3c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43154/0 (socket says 172.21.15.12:43154) 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.739+0000 7f40477fe700 1 -- 172.21.15.12:0/2755351414 learned_addr learned my addr 172.21.15.12:0/2755351414 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.740+0000 7f4046ffd700 1 -- 172.21.15.12:0/2755351414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 msgr2=0x55a41e3c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.740+0000 7f4046ffd700 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 0x55a41e3c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.740+0000 7f4046ffd700 1 -- 172.21.15.12:0/2755351414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c4720 msgr2=0x55a41e3c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.740+0000 7f4046ffd700 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c4720 0x55a41e3c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.740+0000 7f4046ffd700 1 -- 172.21.15.12:0/2755351414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40400b7e70 con 0x55a41e3c3e00 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.740+0000 7f4046ffd700 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 0x55a41e3c80c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f403c006010 tx=0x7f403c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aaf1ce07265e97a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.741+0000 7f4026ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.741+0000 7f4026ffd700 1 -- 172.21.15.12:0/2755351414 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c01d030 con 0x55a41e3c3e00 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.741+0000 7f4026ffd700 1 -- 172.21.15.12:0/2755351414 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f403c004500 con 0x55a41e3c3e00 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.741+0000 7f4059c32b80 1 -- 172.21.15.12:0/2755351414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a41e267270 con 0x55a41e3c3e00 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.741+0000 7f4026ffd700 1 -- 172.21.15.12:0/2755351414 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f403c00bee0 con 0x55a41e3c3e00 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.741+0000 7f4059c32b80 10 client.20418 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.741+0000 7f4059c32b80 1 -- 172.21.15.12:0/2755351414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7f4038017070 con 0x55a41e3c3e00 2024-05-09T00:04:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.741+0000 7f4026ffd700 1 -- 172.21.15.12:0/2755351414 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a41e267270 con 0x55a41e3c3e00 2024-05-09T00:04:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.742+0000 7f4026ffd700 1 -- 172.21.15.12:0/2755351414 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f403c0357e0 con 0x55a41e3c3e00 2024-05-09T00:04:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.742+0000 7f4059c32b80 10 client.20418 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.742+0000 7f4059c32b80 10 client.20418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.742+0000 7f4059c32b80 20 client.20418 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.742+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a41e3d8330 0x55a41e3da750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.742+0000 7f4059c32b80 4 client.20418 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.742+0000 7f4059c32b80 1 -- 172.21.15.12:0/2755351414 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a41e38d810 con 0x55a41e3d8330 2024-05-09T00:04:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.743+0000 7f40477fe700 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a41e3d8330 0x55a41e3da750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.743+0000 7f4026ffd700 10 client.20418 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.743+0000 7f40477fe700 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a41e3d8330 0x55a41e3da750 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b796b0f92e219e21 server_cookie=8b3c40e814a6245 in_seq=0 out_seq=0 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.746+0000 7f4026ffd700 1 -- 172.21.15.12:0/2755351414 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a41e38d810 con 0x55a41e3d8330 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.746+0000 7f4026ffd700 10 client.20418 handle_command_reply: tid=0 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.746+0000 7f4059c32b80 1 client.20418 shutdown 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.746+0000 7f4059c32b80 1 -- 172.21.15.12:0/2755351414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 msgr2=0x55a41e3c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.746+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 0x55a41e3c80c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f403c006010 tx=0x7f403c005800 comp rx=0 tx=0).stop 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.747+0000 7f4059c32b80 1 -- 172.21.15.12:0/2755351414 shutdown_connections 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.747+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a41e3d8330 0x55a41e3da750 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.747+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c4720 0x55a41e3c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.747+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3be890 0x55a41e3c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.747+0000 7f4026ffd700 0 client.20418 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.747+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2755351414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 0x55a41e3c80c0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.747+0000 7f4059c32b80 1 -- 172.21.15.12:0/2755351414 >> 172.21.15.12:0/2755351414 conn(0x55a41e31c210 msgr2=0x55a41e3b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.747+0000 7f4059c32b80 1 -- 172.21.15.12:0/2755351414 shutdown_connections 2024-05-09T00:04:08.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.747+0000 7f4059c32b80 1 -- 172.21.15.12:0/2755351414 wait complete. 2024-05-09T00:04:08.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.747+0000 7f4059c32b80 20 client.20418 trim_cache size 0 max 16384 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4059c32b80 1 Processor -- start 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4059c32b80 1 -- start start 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3be890 0x55a41e3be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 0x55a41e3c3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 0x55a41e3b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4059c32b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e3c9180 con 0x55a41e3c3e00 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4059c32b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e3c1870 con 0x55a41e3be890 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4059c32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e3b9e80 con 0x55a41e3c4720 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4047fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 0x55a41e3b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4047fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 0x55a41e3b8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43160/0 (socket says 172.21.15.12:43160) 2024-05-09T00:04:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f4047fff700 1 -- 172.21.15.12:0/912250514 learned_addr learned my addr 172.21.15.12:0/912250514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:08.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.771+0000 7f40477fe700 1 --2- 172.21.15.12:0/912250514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3be890 0x55a41e3be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.772+0000 7f4046ffd700 1 --2- 172.21.15.12:0/912250514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 0x55a41e3c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.772+0000 7f40477fe700 1 -- 172.21.15.12:0/912250514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 msgr2=0x55a41e3b8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.772+0000 7f40477fe700 1 --2- 172.21.15.12:0/912250514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 0x55a41e3b8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.772+0000 7f40477fe700 1 -- 172.21.15.12:0/912250514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 msgr2=0x55a41e3c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.772+0000 7f40477fe700 1 --2- 172.21.15.12:0/912250514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 0x55a41e3c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.772+0000 7f40477fe700 1 -- 172.21.15.12:0/912250514 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a41e13bb60 con 0x55a41e3be890 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.772+0000 7f40477fe700 1 --2- 172.21.15.12:0/912250514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3be890 0x55a41e3be350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55a41e3db030 tx=0x7f4038017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895f09f3579356b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.772+0000 7f402e7fc700 1 -- 172.21.15.12:0/912250514 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403802b040 con 0x55a41e3be890 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.772+0000 7f402e7fc700 1 -- 172.21.15.12:0/912250514 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40380371c0 con 0x55a41e3be890 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.772+0000 7f402e7fc700 1 -- 172.21.15.12:0/912250514 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038003b10 con 0x55a41e3be890 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.773+0000 7f4059c32b80 1 -- 172.21.15.12:0/912250514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3be890 msgr2=0x55a41e3be350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.773+0000 7f4059c32b80 1 --2- 172.21.15.12:0/912250514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3be890 0x55a41e3be350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55a41e3db030 tx=0x7f4038017000 comp rx=0 tx=0).stop 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.773+0000 7f4059c32b80 1 -- 172.21.15.12:0/912250514 shutdown_connections 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.773+0000 7f4059c32b80 1 --2- 172.21.15.12:0/912250514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 0x55a41e3b8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.773+0000 7f4059c32b80 1 --2- 172.21.15.12:0/912250514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3be890 0x55a41e3be350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.773+0000 7f4059c32b80 1 --2- 172.21.15.12:0/912250514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3c3e00 0x55a41e3c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.773+0000 7f4059c32b80 1 -- 172.21.15.12:0/912250514 >> 172.21.15.12:0/912250514 conn(0x55a41e31c210 msgr2=0x55a41e3c2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.773+0000 7f4059c32b80 1 -- 172.21.15.12:0/912250514 shutdown_connections 2024-05-09T00:04:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.773+0000 7f4059c32b80 1 -- 172.21.15.12:0/912250514 wait complete. 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.774+0000 7f4059c32b80 1 -- start start 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.774+0000 7f4059c32b80 1 Processor -- start 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.774+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3be890 0x55a41e3cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.774+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.774+0000 7f4059c32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 0x55a41e3cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.774+0000 7f4059c32b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e42d960 con 0x55a41e3be890 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.774+0000 7f4059c32b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e42dad0 con 0x55a41e3c3e00 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4059c32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41e42dc90 con 0x55a41e3c4720 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4059c32b80 10 client.0 _ll_register_callbacks cb 0x55a41e2b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f40477fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3be890 0x55a41e3cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4046ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4046ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50888/0 (socket says 172.21.15.12:50888) 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4047fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 0x55a41e3cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f40477fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3be890 0x55a41e3cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57676/0 (socket says 172.21.15.12:57676) 2024-05-09T00:04:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4046ffd700 1 -- 172.21.15.12:0/2534251553 learned_addr learned my addr 172.21.15.12:0/2534251553 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4047fff700 1 -- 172.21.15.12:0/2534251553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 msgr2=0x55a41e3cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4047fff700 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4047fff700 1 -- 172.21.15.12:0/2534251553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3be890 msgr2=0x55a41e3cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4047fff700 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3be890 0x55a41e3cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4047fff700 1 -- 172.21.15.12:0/2534251553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a41e18f3b0 con 0x55a41e3c4720 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4046ffd700 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4047fff700 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 0x55a41e3cdbd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4040095e10 tx=0x7f4040095640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39b7e327c865e011 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4026ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.775+0000 7f4026ffd700 1 -- 172.21.15.12:0/2534251553 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404009a010 con 0x55a41e3c4720 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4026ffd700 1 -- 172.21.15.12:0/2534251553 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f404000ce60 con 0x55a41e3c4720 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4059c32b80 1 -- 172.21.15.12:0/2534251553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4038003e70 con 0x55a41e3c4720 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4026ffd700 1 -- 172.21.15.12:0/2534251553 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4040099910 con 0x55a41e3c4720 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4059c32b80 10 client.30154 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4059c32b80 1 -- 172.21.15.12:0/2534251553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55a41e2d4560 con 0x55a41e3c4720 2024-05-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4026ffd700 1 -- 172.21.15.12:0/2534251553 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4038003e70 con 0x55a41e3c4720 2024-05-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4026ffd700 1 -- 172.21.15.12:0/2534251553 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4040099c00 con 0x55a41e3c4720 2024-05-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4059c32b80 10 client.30154 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4059c32b80 10 client.30154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4059c32b80 20 client.30154 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a41e42f330 0x55a41e431750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4059c32b80 4 client.30154 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f4059c32b80 1 -- 172.21.15.12:0/2534251553 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a41e306ec0 con 0x55a41e42f330 2024-05-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.776+0000 7f40477fe700 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a41e42f330 0x55a41e431750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.777+0000 7f4026ffd700 10 client.30154 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.777+0000 7f40477fe700 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a41e42f330 0x55a41e431750 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3bbf453025dce22 server_cookie=3567948370f0f297 in_seq=0 out_seq=0 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4026ffd700 1 -- 172.21.15.12:0/2534251553 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55a41e306ec0 con 0x55a41e42f330 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4026ffd700 10 client.30154 handle_command_reply: tid=0 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4059c32b80 1 client.30154 shutdown 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4059c32b80 1 -- 172.21.15.12:0/2534251553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 msgr2=0x55a41e3cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 0x55a41e3cdbd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4040095e10 tx=0x7f4040095640 comp rx=0 tx=0).stop 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4059c32b80 1 -- 172.21.15.12:0/2534251553 shutdown_connections 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a41e3c4720 0x55a41e3cdbd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a41e3be890 0x55a41e3cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a41e42f330 0x55a41e431750 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4059c32b80 1 --2- 172.21.15.12:0/2534251553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a41e3c3e00 0x55a41e3cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.781+0000 7f4059c32b80 1 -- 172.21.15.12:0/2534251553 >> 172.21.15.12:0/2534251553 conn(0x55a41e31c210 msgr2=0x55a41e3c2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.782+0000 7f4026ffd700 0 client.30154 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.782+0000 7f4059c32b80 1 -- 172.21.15.12:0/2534251553 shutdown_connections 2024-05-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.782+0000 7f4059c32b80 1 -- 172.21.15.12:0/2534251553 wait complete. 2024-05-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.782+0000 7f4059c32b80 20 client.30154 trim_cache size 0 max 16384 2024-05-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (55 inodes in the stack)", 2024-05-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "28a9d5e6-d01b-4e79-8e51-d34d4a9646b9": { 2024-05-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "28a9d5e6-d01b-4e79-8e51-d34d4a9646b9", 2024-05-09T00:04:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:04:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:04:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:04:08.778 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:04:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.782+0000 7f404da78700 1 -- 172.21.15.12:0/4175513859 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f40300a1310 msgr2=0x7f40300a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.782+0000 7f404da78700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f40300a1310 0x7f40300a37d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4038011490 tx=0x7f4038009260 comp rx=0 tx=0).stop 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.782+0000 7f404da78700 1 -- 172.21.15.12:0/4175513859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404819deb0 msgr2=0x7f404819e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.782+0000 7f404da78700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404819deb0 0x7f404819e330 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f404000ef30 tx=0x7f404000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.783+0000 7f404da78700 1 -- 172.21.15.12:0/4175513859 shutdown_connections 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.783+0000 7f404da78700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f40300a1310 0x7f40300a37d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.783+0000 7f404da78700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404819deb0 0x7f404819e330 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.783+0000 7f404da78700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4048103370 0x7f4048199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.783+0000 7f404da78700 1 --2- 172.21.15.12:0/4175513859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4048108490 0x7f4048199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.783+0000 7f404da78700 1 -- 172.21.15.12:0/4175513859 >> 172.21.15.12:0/4175513859 conn(0x7f40480febb0 msgr2=0x7f40480ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.783+0000 7f404da78700 1 -- 172.21.15.12:0/4175513859 shutdown_connections 2024-05-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:08.783+0000 7f404da78700 1 -- 172.21.15.12:0/4175513859 wait complete. 2024-05-09T00:04:08.793 INFO:tasks.cephfs.filesystem:scrub status for tag:28a9d5e6-d01b-4e79-8e51-d34d4a9646b9 - {'path': '/', 'tag': '28a9d5e6-d01b-4e79-8e51-d34d4a9646b9', 'options': 'recursive,force'} 2024-05-09T00:04:09.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:08 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:04:09.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:08 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:09.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:08 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:04:09.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:08 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:04:09.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:08 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:09.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:08 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:04:09.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:08 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:04:09.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:08 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:09.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:08 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:04:09.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 173311 37.58 MB/sec execute 124 sec latency 1781.618 ms 2024-05-09T00:04:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:09 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:04:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:09 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:04:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:09 smithi204 ceph-mon[47373]: pgmap v572: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.5 MiB/s wr, 738 op/s 2024-05-09T00:04:10.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:09 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:04:10.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:09 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:04:10.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:09 smithi160 ceph-mon[46152]: pgmap v572: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.5 MiB/s wr, 738 op/s 2024-05-09T00:04:10.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:09 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:04:10.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:09 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:04:10.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:09 smithi012 ceph-mon[42503]: pgmap v572: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.5 MiB/s wr, 738 op/s 2024-05-09T00:04:10.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 173884 37.48 MB/sec execute 125 sec latency 2781.651 ms 2024-05-09T00:04:11.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 174516 37.45 MB/sec execute 126 sec latency 3781.698 ms 2024-05-09T00:04:12.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:11 smithi204 ceph-mon[47373]: pgmap v573: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.5 MiB/s wr, 738 op/s 2024-05-09T00:04:12.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:11 smithi160 ceph-mon[46152]: pgmap v573: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.5 MiB/s wr, 738 op/s 2024-05-09T00:04:12.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:11 smithi012 ceph-mon[42503]: pgmap v573: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.5 MiB/s wr, 738 op/s 2024-05-09T00:04:12.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 175271 37.46 MB/sec execute 127 sec latency 4781.747 ms 2024-05-09T00:04:13.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 175932 37.46 MB/sec execute 128 sec latency 5534.774 ms 2024-05-09T00:04:13.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:13 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:13.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:13 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:13.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:13 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:14.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 176667 37.53 MB/sec execute 129 sec latency 6308.587 ms 2024-05-09T00:04:14.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:14 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:14.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:14 smithi012 ceph-mon[42503]: pgmap v574: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 694 op/s 2024-05-09T00:04:14.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:14 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:14.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:14 smithi204 ceph-mon[47373]: pgmap v574: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 694 op/s 2024-05-09T00:04:14.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:14 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:14.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:14 smithi160 ceph-mon[46152]: pgmap v574: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 694 op/s 2024-05-09T00:04:15.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 177353 37.48 MB/sec execute 130 sec latency 7308.624 ms 2024-05-09T00:04:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:15 smithi204 ceph-mon[47373]: pgmap v575: 97 pgs: 97 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.1 MiB/s wr, 602 op/s 2024-05-09T00:04:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:15 smithi160 ceph-mon[46152]: pgmap v575: 97 pgs: 97 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.1 MiB/s wr, 602 op/s 2024-05-09T00:04:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.118+0000 7fc06633e700 1 -- 172.21.15.12:0/968112469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc060101aa0 msgr2=0x7fc060105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.118+0000 7fc06633e700 1 --2- 172.21.15.12:0/968112469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc060101aa0 0x7fc060105ae0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc054009990 tx=0x7fc054009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.120+0000 7fc06633e700 1 -- 172.21.15.12:0/968112469 shutdown_connections 2024-05-09T00:04:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.120+0000 7fc06633e700 1 --2- 172.21.15.12:0/968112469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc060106210 0x7fc06010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.120+0000 7fc06633e700 1 --2- 172.21.15.12:0/968112469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0601010f0 0x7fc0601014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.120+0000 7fc06633e700 1 --2- 172.21.15.12:0/968112469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc060101aa0 0x7fc060105ae0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.120+0000 7fc06633e700 1 -- 172.21.15.12:0/968112469 >> 172.21.15.12:0/968112469 conn(0x7fc0600fc9d0 msgr2=0x7fc0600fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.121+0000 7fc06633e700 1 -- 172.21.15.12:0/968112469 shutdown_connections 2024-05-09T00:04:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.121+0000 7fc06633e700 1 -- 172.21.15.12:0/968112469 wait complete. 2024-05-09T00:04:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.121+0000 7fc06633e700 1 Processor -- start 2024-05-09T00:04:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc06633e700 1 -- start start 2024-05-09T00:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc06633e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0601010f0 0x7fc0601a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc06633e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc060106210 0x7fc0601a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc06633e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06010ce90 0x7fc06010d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc06633e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06010d920 con 0x7fc06010ce90 2024-05-09T00:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc06633e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06010da90 con 0x7fc0601010f0 2024-05-09T00:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc06633e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06010dc50 con 0x7fc060106210 2024-05-09T00:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc05f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc060106210 0x7fc0601a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc05ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0601010f0 0x7fc0601a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc05f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc060106210 0x7fc0601a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43180/0 (socket says 172.21.15.12:43180) 2024-05-09T00:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.122+0000 7fc05f7fe700 1 -- 172.21.15.12:0/2761088320 learned_addr learned my addr 172.21.15.12:0/2761088320 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc0648db700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06010ce90 0x7fc06010d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc05f7fe700 1 -- 172.21.15.12:0/2761088320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0601010f0 msgr2=0x7fc0601a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0601010f0 0x7fc0601a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc05f7fe700 1 -- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06010ce90 msgr2=0x7fc06010d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06010ce90 0x7fc06010d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc05f7fe700 1 -- 172.21.15.12:0/2761088320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc058009c40 con 0x7fc060106210 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc0648db700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06010ce90 0x7fc06010d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc060106210 0x7fc0601a3bf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc054005950 tx=0x7fc054005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc05d7fa700 1 -- 172.21.15.12:0/2761088320 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05401d070 con 0x7fc060106210 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc05d7fa700 1 -- 172.21.15.12:0/2761088320 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc05400beb0 con 0x7fc060106210 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc06633e700 1 -- 172.21.15.12:0/2761088320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0540095f0 con 0x7fc060106210 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.123+0000 7fc05d7fa700 1 -- 172.21.15.12:0/2761088320 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054021dc0 con 0x7fc060106210 2024-05-09T00:04:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.124+0000 7fc06633e700 1 -- 172.21.15.12:0/2761088320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0601aa7f0 con 0x7fc060106210 2024-05-09T00:04:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.126+0000 7fc05d7fa700 1 -- 172.21.15.12:0/2761088320 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc054021410 con 0x7fc060106210 2024-05-09T00:04:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.127+0000 7fc05d7fa700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0480a1290 0x7fc0480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.127+0000 7fc05d7fa700 1 -- 172.21.15.12:0/2761088320 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc0540cdec0 con 0x7fc060106210 2024-05-09T00:04:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.128+0000 7fc05ffff700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0480a1290 0x7fc0480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.129+0000 7fc05ffff700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0480a1290 0x7fc0480a3750 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc050009c60 tx=0x7fc050009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.129+0000 7fc0724f8b80 1 Processor -- start 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.129+0000 7fc0724f8b80 1 -- start start 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.129+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd608520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.129+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608a60 0x557fdd60c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.129+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 0x557fdd60cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.129+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd54e0f0 con 0x557fdd608140 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.129+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd54bab0 con 0x557fdd608a60 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.129+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd5473a0 con 0x557fdd60ca40 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc05ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd608520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc05f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608a60 0x557fdd60c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc05ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd608520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57708/0 (socket says 172.21.15.12:57708) 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc05ffff700 1 -- 172.21.15.12:0/3697186148 learned_addr learned my addr 172.21.15.12:0/3697186148 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc0648db700 1 --2- 172.21.15.12:0/3697186148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 0x557fdd60cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc05f7fe700 1 -- 172.21.15.12:0/3697186148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 msgr2=0x557fdd60cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/3697186148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 0x557fdd60cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc05f7fe700 1 -- 172.21.15.12:0/3697186148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 msgr2=0x557fdd608520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/3697186148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd608520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc05f7fe700 1 -- 172.21.15.12:0/3697186148 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fdd3c8350 con 0x557fdd608a60 2024-05-09T00:04:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/3697186148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608a60 0x557fdd60c500 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc0540cb620 tx=0x7fc05409b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f11e9e84605bb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.130+0000 7fc0467fc700 1 -- 172.21.15.12:0/3697186148 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0540a3460 con 0x557fdd608a60 2024-05-09T00:04:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.131+0000 7fc0467fc700 1 -- 172.21.15.12:0/3697186148 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc054099410 con 0x557fdd608a60 2024-05-09T00:04:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.131+0000 7fc0467fc700 1 -- 172.21.15.12:0/3697186148 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05409a7c0 con 0x557fdd608a60 2024-05-09T00:04:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.132+0000 7fc0724f8b80 1 -- 172.21.15.12:0/3697186148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608a60 msgr2=0x557fdd60c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.132+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/3697186148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608a60 0x557fdd60c500 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc0540cb620 tx=0x7fc05409b770 comp rx=0 tx=0).stop 2024-05-09T00:04:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.132+0000 7fc0724f8b80 1 -- 172.21.15.12:0/3697186148 shutdown_connections 2024-05-09T00:04:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.132+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/3697186148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 0x557fdd60cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.132+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/3697186148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd608520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.132+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/3697186148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608a60 0x557fdd60c500 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.132+0000 7fc0724f8b80 1 -- 172.21.15.12:0/3697186148 >> 172.21.15.12:0/3697186148 conn(0x557fdd5551b0 msgr2=0x557fdd5f1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.132+0000 7fc0724f8b80 1 -- 172.21.15.12:0/3697186148 shutdown_connections 2024-05-09T00:04:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.132+0000 7fc0724f8b80 1 -- 172.21.15.12:0/3697186148 wait complete. 2024-05-09T00:04:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.132+0000 7fc0724f8b80 1 -- start start 2024-05-09T00:04:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc0724f8b80 1 Processor -- start 2024-05-09T00:04:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608140 0x557fdd614210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 0x557fdd611550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 0x557fdd611c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd4e3500 con 0x557fdd608a60 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd4e31a0 con 0x557fdd60ca40 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd3ca640 con 0x557fdd608140 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc0724f8b80 10 client.0 _ll_register_callbacks cb 0x557fdd4ecad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc05ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608140 0x557fdd614210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc05f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 0x557fdd611550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc05ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608140 0x557fdd614210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:43198/0 (socket says 172.21.15.12:43198) 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.133+0000 7fc05ffff700 1 -- 172.21.15.12:0/516413917 learned_addr learned my addr 172.21.15.12:0/516413917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.134+0000 7fc0648db700 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 0x557fdd611c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.134+0000 7fc05f7fe700 1 -- 172.21.15.12:0/516413917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608140 msgr2=0x557fdd614210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.134+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608140 0x557fdd614210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.134+0000 7fc05f7fe700 1 -- 172.21.15.12:0/516413917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 msgr2=0x557fdd611c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.134+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 0x557fdd611c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.134+0000 7fc05f7fe700 1 -- 172.21.15.12:0/516413917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0601aa7f0 con 0x557fdd608a60 2024-05-09T00:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.134+0000 7fc05ffff700 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608140 0x557fdd614210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.134+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 0x557fdd611550 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc0540993c0 tx=0x7fc054005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b22e797d4a5706 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.134+0000 7fc03f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.135+0000 7fc03f7fe700 1 -- 172.21.15.12:0/516413917 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054064070 con 0x557fdd608a60 2024-05-09T00:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.135+0000 7fc03f7fe700 1 -- 172.21.15.12:0/516413917 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc05409a560 con 0x557fdd608a60 2024-05-09T00:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.135+0000 7fc0724f8b80 1 -- 172.21.15.12:0/516413917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fdd4a0210 con 0x557fdd608a60 2024-05-09T00:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.135+0000 7fc03f7fe700 1 -- 172.21.15.12:0/516413917 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc05409a720 con 0x557fdd608a60 2024-05-09T00:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.135+0000 7fc0724f8b80 10 client.20436 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.135+0000 7fc0724f8b80 1 -- 172.21.15.12:0/516413917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x557fdd374b00 con 0x557fdd608a60 2024-05-09T00:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.136+0000 7fc03f7fe700 1 -- 172.21.15.12:0/516413917 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557fdd4a0210 con 0x557fdd608a60 2024-05-09T00:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.136+0000 7fc03f7fe700 1 -- 172.21.15.12:0/516413917 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc0540a2fd0 con 0x557fdd608a60 2024-05-09T00:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.136+0000 7fc0724f8b80 10 client.20436 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.136+0000 7fc0724f8b80 10 client.20436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.136+0000 7fc0724f8b80 20 client.20436 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.136+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557fdd6213e0 0x557fdd6237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.136+0000 7fc0724f8b80 4 client.20436 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.136+0000 7fc0724f8b80 1 -- 172.21.15.12:0/516413917 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fdd5c67b0 con 0x557fdd6213e0 2024-05-09T00:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.136+0000 7fc05ffff700 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557fdd6213e0 0x557fdd6237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.139+0000 7fc05ffff700 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557fdd6213e0 0x557fdd6237d0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c68fdcba838c9993 server_cookie=c7e07b60a60968e8 in_seq=0 out_seq=0 2024-05-09T00:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.139+0000 7fc03f7fe700 10 client.20436 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc03f7fe700 1 -- 172.21.15.12:0/516413917 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fdd5c67b0 con 0x557fdd6213e0 2024-05-09T00:04:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc03f7fe700 10 client.20436 handle_command_reply: tid=0 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc0724f8b80 1 client.20436 shutdown 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc0724f8b80 1 -- 172.21.15.12:0/516413917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 msgr2=0x557fdd611550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 0x557fdd611550 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc0540993c0 tx=0x7fc054005be0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc0724f8b80 1 -- 172.21.15.12:0/516413917 shutdown_connections 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557fdd6213e0 0x557fdd6237d0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 0x557fdd611c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608140 0x557fdd614210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/516413917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 0x557fdd611550 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.146+0000 7fc0724f8b80 1 -- 172.21.15.12:0/516413917 >> 172.21.15.12:0/516413917 conn(0x557fdd5551b0 msgr2=0x557fdd5f5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.147+0000 7fc03f7fe700 0 client.20436 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.147+0000 7fc0724f8b80 1 -- 172.21.15.12:0/516413917 shutdown_connections 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.147+0000 7fc0724f8b80 1 -- 172.21.15.12:0/516413917 wait complete. 2024-05-09T00:04:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.147+0000 7fc0724f8b80 20 client.20436 trim_cache size 0 max 16384 2024-05-09T00:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.167+0000 7fc0724f8b80 1 Processor -- start 2024-05-09T00:04:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.167+0000 7fc0724f8b80 1 -- start start 2024-05-09T00:04:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.167+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608140 0x557fdd611c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.167+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 0x557fdd61d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.167+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 0x557fdd5f1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.167+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd61e410 con 0x557fdd608a60 2024-05-09T00:04:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.167+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd611320 con 0x557fdd608140 2024-05-09T00:04:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.167+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd5f34e0 con 0x557fdd60ca40 2024-05-09T00:04:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608140 0x557fdd611c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608140 0x557fdd611c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50938/0 (socket says 172.21.15.12:50938) 2024-05-09T00:04:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05ffff700 1 -- 172.21.15.12:0/2974648885 learned_addr learned my addr 172.21.15.12:0/2974648885 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 0x557fdd61d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc0648db700 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 0x557fdd5f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05ffff700 1 -- 172.21.15.12:0/2974648885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 msgr2=0x557fdd5f1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05ffff700 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 0x557fdd5f1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05ffff700 1 -- 172.21.15.12:0/2974648885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 msgr2=0x557fdd61d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05ffff700 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 0x557fdd61d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05ffff700 1 -- 172.21.15.12:0/2974648885 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fdd50d500 con 0x557fdd608140 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc0648db700 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 0x557fdd5f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 0x557fdd61d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.168+0000 7fc05ffff700 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608140 0x557fdd611c50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc050013010 tx=0x7fc050014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e49936ac7b47cffc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.169+0000 7fc046ffd700 1 -- 172.21.15.12:0/2974648885 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050004580 con 0x557fdd608140 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.169+0000 7fc046ffd700 1 -- 172.21.15.12:0/2974648885 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc050023040 con 0x557fdd608140 2024-05-09T00:04:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.169+0000 7fc046ffd700 1 -- 172.21.15.12:0/2974648885 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05001d970 con 0x557fdd608140 2024-05-09T00:04:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.169+0000 7fc0724f8b80 1 -- 172.21.15.12:0/2974648885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608140 msgr2=0x557fdd611c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.169+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608140 0x557fdd611c50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc050013010 tx=0x7fc050014000 comp rx=0 tx=0).stop 2024-05-09T00:04:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.169+0000 7fc0724f8b80 1 -- 172.21.15.12:0/2974648885 shutdown_connections 2024-05-09T00:04:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.169+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd60ca40 0x557fdd5f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.169+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd608140 0x557fdd611c50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.169+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/2974648885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608a60 0x557fdd61d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.169+0000 7fc0724f8b80 1 -- 172.21.15.12:0/2974648885 >> 172.21.15.12:0/2974648885 conn(0x557fdd5551b0 msgr2=0x557fdd5f5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:15 smithi012 ceph-mon[42503]: pgmap v575: 97 pgs: 97 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.1 MiB/s wr, 602 op/s 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.171+0000 7fc0724f8b80 1 -- 172.21.15.12:0/2974648885 shutdown_connections 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.171+0000 7fc0724f8b80 1 -- 172.21.15.12:0/2974648885 wait complete. 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.171+0000 7fc0724f8b80 1 -- start start 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.171+0000 7fc0724f8b80 1 Processor -- start 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.172+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd618950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.172+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608a60 0x557fdd618e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.172+0000 7fc0724f8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 0x557fdd6169b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.172+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd6170c0 con 0x557fdd608140 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.172+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd617230 con 0x557fdd60ca40 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.172+0000 7fc0724f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fdd6173f0 con 0x557fdd608a60 2024-05-09T00:04:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc0724f8b80 10 client.0 _ll_register_callbacks cb 0x557fdd4ecad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc05ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd618950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc05ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd618950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57742/0 (socket says 172.21.15.12:57742) 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc05ffff700 1 -- 172.21.15.12:0/1837951365 learned_addr learned my addr 172.21.15.12:0/1837951365 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc0648db700 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 0x557fdd6169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608a60 0x557fdd618e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc05ffff700 1 -- 172.21.15.12:0/1837951365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608a60 msgr2=0x557fdd618e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc05ffff700 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608a60 0x557fdd618e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc05ffff700 1 -- 172.21.15.12:0/1837951365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 msgr2=0x557fdd6169b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc05ffff700 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 0x557fdd6169b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc05ffff700 1 -- 172.21.15.12:0/1837951365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fdd374b00 con 0x557fdd608140 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc0648db700 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 0x557fdd6169b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc05ffff700 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd618950 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc05001d450 tx=0x7fc0500096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428010edc1afb7f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc03f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc03f7fe700 1 -- 172.21.15.12:0/1837951365 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050005450 con 0x557fdd608140 2024-05-09T00:04:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc03f7fe700 1 -- 172.21.15.12:0/1837951365 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc0500055b0 con 0x557fdd608140 2024-05-09T00:04:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.173+0000 7fc0724f8b80 1 -- 172.21.15.12:0/1837951365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc054099410 con 0x557fdd608140 2024-05-09T00:04:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.174+0000 7fc03f7fe700 1 -- 172.21.15.12:0/1837951365 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc050004560 con 0x557fdd608140 2024-05-09T00:04:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.174+0000 7fc0724f8b80 10 client.20442 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.174+0000 7fc0724f8b80 1 -- 172.21.15.12:0/1837951365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7fc050017070 con 0x557fdd608140 2024-05-09T00:04:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.174+0000 7fc03f7fe700 1 -- 172.21.15.12:0/1837951365 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc054099410 con 0x557fdd608140 2024-05-09T00:04:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.175+0000 7fc03f7fe700 1 -- 172.21.15.12:0/1837951365 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc050029380 con 0x557fdd608140 2024-05-09T00:04:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.175+0000 7fc0724f8b80 10 client.20442 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.175+0000 7fc0724f8b80 10 client.20442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.175+0000 7fc0724f8b80 20 client.20442 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.175+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557fdd673a40 0x557fdd673e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.175+0000 7fc0724f8b80 4 client.20442 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:04:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.175+0000 7fc0724f8b80 1 -- 172.21.15.12:0/1837951365 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557fdd53fe60 con 0x557fdd673a40 2024-05-09T00:04:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.175+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557fdd673a40 0x557fdd673e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.176+0000 7fc03f7fe700 10 client.20442 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:16.176+0000 7fc05f7fe700 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557fdd673a40 0x557fdd673e50 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f199c93908701f6f server_cookie=9cc6786ae0fffc49 in_seq=0 out_seq=0 2024-05-09T00:04:16.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 178050 37.49 MB/sec execute 131 sec latency 8308.664 ms 2024-05-09T00:04:17.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:16 smithi012 ceph-mon[42503]: osdmap e122: 12 total, 12 up, 12 in 2024-05-09T00:04:17.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:17.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 178570 37.44 MB/sec execute 132 sec latency 8495.934 ms 2024-05-09T00:04:17.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:16 smithi204 ceph-mon[47373]: osdmap e122: 12 total, 12 up, 12 in 2024-05-09T00:04:17.261 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:17.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:16 smithi160 ceph-mon[46152]: osdmap e122: 12 total, 12 up, 12 in 2024-05-09T00:04:17.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:18.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 179215 37.35 MB/sec execute 133 sec latency 3814.955 ms 2024-05-09T00:04:18.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:17 smithi204 ceph-mon[47373]: pgmap v577: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.2 MiB/s wr, 685 op/s 2024-05-09T00:04:18.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:17 smithi160 ceph-mon[46152]: pgmap v577: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.2 MiB/s wr, 685 op/s 2024-05-09T00:04:18.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:17 smithi012 ceph-mon[42503]: pgmap v577: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.2 MiB/s wr, 685 op/s 2024-05-09T00:04:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.989+0000 7fc03f7fe700 1 -- 172.21.15.12:0/1837951365 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557fdd53fe60 con 0x557fdd673a40 2024-05-09T00:04:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.989+0000 7fc03f7fe700 10 client.20442 handle_command_reply: tid=0 2024-05-09T00:04:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.989+0000 7fc0724f8b80 1 client.20442 shutdown 2024-05-09T00:04:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 1 -- 172.21.15.12:0/1837951365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 msgr2=0x557fdd618950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd618950 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc05001d450 tx=0x7fc0500096d0 comp rx=0 tx=0).stop 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 1 -- 172.21.15.12:0/1837951365 shutdown_connections 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557fdd673a40 0x557fdd673e50 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fdd60ca40 0x557fdd6169b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557fdd608140 0x557fdd618950 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 1 --2- 172.21.15.12:0/1837951365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557fdd608a60 0x557fdd618e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 1 -- 172.21.15.12:0/1837951365 >> 172.21.15.12:0/1837951365 conn(0x557fdd5551b0 msgr2=0x557fdd6280c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc03f7fe700 0 client.20442 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 1 -- 172.21.15.12:0/1837951365 shutdown_connections 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 1 -- 172.21.15.12:0/1837951365 wait complete. 2024-05-09T00:04:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.996+0000 7fc0724f8b80 20 client.20442 trim_cache size 0 max 16384 2024-05-09T00:04:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:04:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:04:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:04:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:04:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.997+0000 7fc06633e700 1 -- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0480a1290 msgr2=0x7fc0480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.997+0000 7fc06633e700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0480a1290 0x7fc0480a3750 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc050009c60 tx=0x7fc050009260 comp rx=0 tx=0).stop 2024-05-09T00:04:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.997+0000 7fc06633e700 1 -- 172.21.15.12:0/2761088320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc060106210 msgr2=0x7fc0601a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.997+0000 7fc06633e700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc060106210 0x7fc0601a3bf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc054005950 tx=0x7fc054005740 comp rx=0 tx=0).stop 2024-05-09T00:04:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.998+0000 7fc06633e700 1 -- 172.21.15.12:0/2761088320 shutdown_connections 2024-05-09T00:04:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.998+0000 7fc06633e700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0480a1290 0x7fc0480a3750 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.998+0000 7fc06633e700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06010ce90 0x7fc06010d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.998+0000 7fc06633e700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0601010f0 0x7fc0601a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.998+0000 7fc06633e700 1 --2- 172.21.15.12:0/2761088320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc060106210 0x7fc0601a3bf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:18.998+0000 7fc06633e700 1 -- 172.21.15.12:0/2761088320 >> 172.21.15.12:0/2761088320 conn(0x7fc0600fc9d0 msgr2=0x7fc0600fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:19.002+0000 7fc06633e700 1 -- 172.21.15.12:0/2761088320 shutdown_connections 2024-05-09T00:04:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:19.002+0000 7fc06633e700 1 -- 172.21.15.12:0/2761088320 wait complete. 2024-05-09T00:04:19.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 179723 37.26 MB/sec execute 134 sec latency 4562.499 ms 2024-05-09T00:04:20.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:19 smithi204 ceph-mon[47373]: pgmap v578: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-05-09T00:04:20.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:19 smithi160 ceph-mon[46152]: pgmap v578: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-05-09T00:04:20.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:19 smithi012 ceph-mon[42503]: pgmap v578: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-05-09T00:04:20.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 180571 37.42 MB/sec execute 135 sec latency 73.761 ms 2024-05-09T00:04:21.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 181071 37.30 MB/sec execute 136 sec latency 127.364 ms 2024-05-09T00:04:22.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:21 smithi204 ceph-mon[47373]: pgmap v579: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-05-09T00:04:22.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:21 smithi160 ceph-mon[46152]: pgmap v579: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-05-09T00:04:22.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:21 smithi012 ceph-mon[42503]: pgmap v579: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-05-09T00:04:22.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 181834 37.29 MB/sec execute 137 sec latency 112.465 ms 2024-05-09T00:04:23.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 182593 37.35 MB/sec execute 138 sec latency 96.309 ms 2024-05-09T00:04:23.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:23 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:04:24.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:23 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:04:24.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:23 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:04:24.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 183056 37.29 MB/sec execute 139 sec latency 204.243 ms 2024-05-09T00:04:24.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:04:24.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:04:24.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:24 smithi012 ceph-mon[42503]: pgmap v580: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.0 MiB/s wr, 673 op/s 2024-05-09T00:04:25.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:04:25.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:04:25.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:24 smithi204 ceph-mon[47373]: pgmap v580: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.0 MiB/s wr, 673 op/s 2024-05-09T00:04:25.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:04:25.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:04:25.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:24 smithi160 ceph-mon[46152]: pgmap v580: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.0 MiB/s wr, 673 op/s 2024-05-09T00:04:25.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 183071 37.03 MB/sec execute 140 sec latency 1074.042 ms 2024-05-09T00:04:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:25 smithi204 ceph-mon[47373]: pgmap v581: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-05-09T00:04:26.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:25 smithi160 ceph-mon[46152]: pgmap v581: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-05-09T00:04:26.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:25 smithi012 ceph-mon[42503]: pgmap v581: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-05-09T00:04:26.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 183393 36.87 MB/sec execute 141 sec latency 1303.036 ms 2024-05-09T00:04:27.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 183742 36.75 MB/sec execute 142 sec latency 136.417 ms 2024-05-09T00:04:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:27 smithi204 ceph-mon[47373]: pgmap v582: 97 pgs: 97 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 19 MiB/s wr, 936 op/s 2024-05-09T00:04:28.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:27 smithi160 ceph-mon[46152]: pgmap v582: 97 pgs: 97 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 19 MiB/s wr, 936 op/s 2024-05-09T00:04:28.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:27 smithi012 ceph-mon[42503]: pgmap v582: 97 pgs: 97 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 19 MiB/s wr, 936 op/s 2024-05-09T00:04:28.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184341 36.72 MB/sec execute 143 sec latency 133.784 ms 2024-05-09T00:04:29.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184932 36.67 MB/sec execute 144 sec latency 554.734 ms 2024-05-09T00:04:30.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:29 smithi204 ceph-mon[47373]: pgmap v583: 97 pgs: 97 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 760 op/s 2024-05-09T00:04:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:29 smithi160 ceph-mon[46152]: pgmap v583: 97 pgs: 97 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 760 op/s 2024-05-09T00:04:30.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:29 smithi012 ceph-mon[42503]: pgmap v583: 97 pgs: 97 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 760 op/s 2024-05-09T00:04:30.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 185563 36.64 MB/sec execute 145 sec latency 1554.768 ms 2024-05-09T00:04:31.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 186125 36.62 MB/sec execute 146 sec latency 2554.800 ms 2024-05-09T00:04:32.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:31 smithi204 ceph-mon[47373]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 734 op/s 2024-05-09T00:04:32.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:31 smithi160 ceph-mon[46152]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 734 op/s 2024-05-09T00:04:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.167+0000 7efebf553700 1 -- 172.21.15.12:0/731506880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeb8105130 msgr2=0x7efeb810ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.167+0000 7efebf553700 1 --2- 172.21.15.12:0/731506880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeb8105130 0x7efeb810ccb0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7efeb000b0a0 tx=0x7efeb000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.167+0000 7efebf553700 1 -- 172.21.15.12:0/731506880 shutdown_connections 2024-05-09T00:04:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.167+0000 7efebf553700 1 --2- 172.21.15.12:0/731506880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeb8105130 0x7efeb810ccb0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.167+0000 7efebf553700 1 --2- 172.21.15.12:0/731506880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb8100010 0x7efeb81003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.167+0000 7efebf553700 1 --2- 172.21.15.12:0/731506880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb81009c0 0x7efeb8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.167+0000 7efebf553700 1 -- 172.21.15.12:0/731506880 >> 172.21.15.12:0/731506880 conn(0x7efeb80fb870 msgr2=0x7efeb80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.167+0000 7efebf553700 1 -- 172.21.15.12:0/731506880 shutdown_connections 2024-05-09T00:04:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.168+0000 7efebf553700 1 -- 172.21.15.12:0/731506880 wait complete. 2024-05-09T00:04:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.168+0000 7efebf553700 1 Processor -- start 2024-05-09T00:04:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.168+0000 7efebf553700 1 -- start start 2024-05-09T00:04:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.168+0000 7efebf553700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb8100010 0x7efeb8101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.168+0000 7efebf553700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeb8105130 0x7efeb8101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.168+0000 7efebf553700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb8102520 0x7efeb81a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.168+0000 7efebf553700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb81a5c90 con 0x7efeb8105130 2024-05-09T00:04:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.168+0000 7efebf553700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb81a5e00 con 0x7efeb8102520 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.168+0000 7efebf553700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb81a5fc0 con 0x7efeb8100010 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebd2ef700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb8100010 0x7efeb8101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebdaf0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb8102520 0x7efeb81a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebd2ef700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb8100010 0x7efeb8101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50744/0 (socket says 172.21.15.12:50744) 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebd2ef700 1 -- 172.21.15.12:0/1459475200 learned_addr learned my addr 172.21.15.12:0/1459475200 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebcaee700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeb8105130 0x7efeb8101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebd2ef700 1 -- 172.21.15.12:0/1459475200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb8102520 msgr2=0x7efeb81a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebd2ef700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb8102520 0x7efeb81a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebd2ef700 1 -- 172.21.15.12:0/1459475200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeb8105130 msgr2=0x7efeb8101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebd2ef700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeb8105130 0x7efeb8101fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebd2ef700 1 -- 172.21.15.12:0/1459475200 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0009d00 con 0x7efeb8100010 2024-05-09T00:04:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efebd2ef700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb8100010 0x7efeb8101aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efeb400d950 tx=0x7efeb400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:32.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:31 smithi012 ceph-mon[42503]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 734 op/s 2024-05-09T00:04:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.169+0000 7efeae7fc700 1 -- 172.21.15.12:0/1459475200 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb400f890 con 0x7efeb8100010 2024-05-09T00:04:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.170+0000 7efeae7fc700 1 -- 172.21.15.12:0/1459475200 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb4010460 con 0x7efeb8100010 2024-05-09T00:04:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.170+0000 7efeae7fc700 1 -- 172.21.15.12:0/1459475200 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb400e610 con 0x7efeb8100010 2024-05-09T00:04:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.170+0000 7efebf553700 1 -- 172.21.15.12:0/1459475200 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb81a62a0 con 0x7efeb8100010 2024-05-09T00:04:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.170+0000 7efebf553700 1 -- 172.21.15.12:0/1459475200 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb81a67b0 con 0x7efeb8100010 2024-05-09T00:04:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.172+0000 7efeae7fc700 1 -- 172.21.15.12:0/1459475200 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efeb40108d0 con 0x7efeb8100010 2024-05-09T00:04:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.172+0000 7efeae7fc700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efea40a1390 0x7efea40a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.173+0000 7efeae7fc700 1 -- 172.21.15.12:0/1459475200 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efeb40ccac0 con 0x7efeb8100010 2024-05-09T00:04:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.173+0000 7efebcaee700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efea40a1390 0x7efea40a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.174+0000 7efebcaee700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efea40a1390 0x7efea40a3850 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efeb81031d0 tx=0x7efea8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.174+0000 7efecb70db80 1 Processor -- start 2024-05-09T00:04:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.174+0000 7efecb70db80 1 -- start start 2024-05-09T00:04:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efecb70db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f57c950 0x560a1f57cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efecb70db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f576f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efecb70db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f577480 0x560a1f5778a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efecb70db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f4ce0f0 con 0x560a1f577480 2024-05-09T00:04:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efecb70db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f4cbab0 con 0x560a1f57c950 2024-05-09T00:04:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efecb70db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f4c73a0 con 0x560a1f57d270 2024-05-09T00:04:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efebdaf0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f577480 0x560a1f5778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efebcaee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f576f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efebdaf0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f577480 0x560a1f5778a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53310/0 (socket says 172.21.15.12:53310) 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efebdaf0700 1 -- 172.21.15.12:0/574821697 learned_addr learned my addr 172.21.15.12:0/574821697 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efebd2ef700 1 --2- 172.21.15.12:0/574821697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f57c950 0x560a1f57cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efebdaf0700 1 -- 172.21.15.12:0/574821697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 msgr2=0x560a1f576f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efebdaf0700 1 --2- 172.21.15.12:0/574821697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f576f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efebdaf0700 1 -- 172.21.15.12:0/574821697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f57c950 msgr2=0x560a1f57cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efebdaf0700 1 --2- 172.21.15.12:0/574821697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f57c950 0x560a1f57cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.175+0000 7efebdaf0700 1 -- 172.21.15.12:0/574821697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a1f348350 con 0x560a1f577480 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.176+0000 7efebdaf0700 1 --2- 172.21.15.12:0/574821697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f577480 0x560a1f5778a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7efeb0015000 tx=0x7efeb0009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d465724d50a694 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.176+0000 7efea37fe700 1 -- 172.21.15.12:0/574821697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb000e040 con 0x560a1f577480 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.176+0000 7efea37fe700 1 -- 172.21.15.12:0/574821697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb0009b80 con 0x560a1f577480 2024-05-09T00:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.176+0000 7efea37fe700 1 -- 172.21.15.12:0/574821697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0003a00 con 0x560a1f577480 2024-05-09T00:04:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.177+0000 7efecb70db80 1 -- 172.21.15.12:0/574821697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f577480 msgr2=0x560a1f5778a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.177+0000 7efecb70db80 1 --2- 172.21.15.12:0/574821697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f577480 0x560a1f5778a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7efeb0015000 tx=0x7efeb0009410 comp rx=0 tx=0).stop 2024-05-09T00:04:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.177+0000 7efecb70db80 1 -- 172.21.15.12:0/574821697 shutdown_connections 2024-05-09T00:04:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.177+0000 7efecb70db80 1 --2- 172.21.15.12:0/574821697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f577480 0x560a1f5778a0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.177+0000 7efecb70db80 1 --2- 172.21.15.12:0/574821697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f57c950 0x560a1f57cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.177+0000 7efecb70db80 1 --2- 172.21.15.12:0/574821697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f576f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.177+0000 7efecb70db80 1 -- 172.21.15.12:0/574821697 >> 172.21.15.12:0/574821697 conn(0x560a1f4d51b0 msgr2=0x560a1f571840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.177+0000 7efecb70db80 1 -- 172.21.15.12:0/574821697 shutdown_connections 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.177+0000 7efecb70db80 1 -- 172.21.15.12:0/574821697 wait complete. 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.177+0000 7efecb70db80 1 -- start start 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.178+0000 7efecb70db80 1 Processor -- start 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.178+0000 7efecb70db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 0x560a1f5831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.178+0000 7efecb70db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f580490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.178+0000 7efecb70db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f580aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.178+0000 7efecb70db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f463500 con 0x560a1f57c950 2024-05-09T00:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.178+0000 7efecb70db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f4631a0 con 0x560a1f577480 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.178+0000 7efecb70db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f34a640 con 0x560a1f57d270 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.178+0000 7efecb70db80 10 client.0 _ll_register_callbacks cb 0x560a1f46cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebdaf0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f580aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebdaf0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f580aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50766/0 (socket says 172.21.15.12:50766) 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebdaf0700 1 -- 172.21.15.12:0/965788950 learned_addr learned my addr 172.21.15.12:0/965788950 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebcaee700 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f580490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebd2ef700 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 0x560a1f5831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebdaf0700 1 -- 172.21.15.12:0/965788950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 msgr2=0x560a1f5831a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebdaf0700 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 0x560a1f5831a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebdaf0700 1 -- 172.21.15.12:0/965788950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 msgr2=0x560a1f580490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebdaf0700 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f580490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebdaf0700 1 -- 172.21.15.12:0/965788950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb810d520 con 0x560a1f57d270 2024-05-09T00:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efebdaf0700 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f580aa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efeb000b780 tx=0x7efeb0007cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89c76663dd0b1dc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efea0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.179+0000 7efea0ff9700 1 -- 172.21.15.12:0/965788950 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb000e040 con 0x560a1f57d270 2024-05-09T00:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.180+0000 7efecb70db80 1 -- 172.21.15.12:0/965788950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a1f5467b0 con 0x560a1f57d270 2024-05-09T00:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.180+0000 7efea0ff9700 1 -- 172.21.15.12:0/965788950 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efeb0007810 con 0x560a1f57d270 2024-05-09T00:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.180+0000 7efea0ff9700 1 -- 172.21.15.12:0/965788950 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeb0003a00 con 0x560a1f57d270 2024-05-09T00:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.180+0000 7efecb70db80 10 client.30184 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.180+0000 7efecb70db80 1 -- 172.21.15.12:0/965788950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x560a1f2f4b00 con 0x560a1f57d270 2024-05-09T00:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.180+0000 7efea0ff9700 1 -- 172.21.15.12:0/965788950 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a1f5467b0 con 0x560a1f57d270 2024-05-09T00:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.180+0000 7efea0ff9700 1 -- 172.21.15.12:0/965788950 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efeb0019070 con 0x560a1f57d270 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.181+0000 7efecb70db80 10 client.30184 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.181+0000 7efecb70db80 10 client.30184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.181+0000 7efecb70db80 20 client.30184 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.181+0000 7efecb70db80 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a1f590830 0x560a1f592c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.181+0000 7efecb70db80 4 client.30184 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.181+0000 7efecb70db80 1 -- 172.21.15.12:0/965788950 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a1f4bfe60 con 0x560a1f590830 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.181+0000 7efebd2ef700 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a1f590830 0x560a1f592c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.181+0000 7efebd2ef700 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a1f590830 0x560a1f592c20 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8279fd76a3f93922 server_cookie=16cb8495bbad6e65 in_seq=0 out_seq=0 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.181+0000 7efea0ff9700 10 client.30184 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efea0ff9700 1 -- 172.21.15.12:0/965788950 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a1f4bfe60 con 0x560a1f590830 2024-05-09T00:04:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efea0ff9700 10 client.30184 handle_command_reply: tid=0 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 client.30184 shutdown 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 -- 172.21.15.12:0/965788950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 msgr2=0x560a1f580aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f580aa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efeb000b780 tx=0x7efeb0007cf0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 -- 172.21.15.12:0/965788950 shutdown_connections 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f580aa0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a1f590830 0x560a1f592c20 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 0x560a1f5831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 --2- 172.21.15.12:0/965788950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f580490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efea0ff9700 0 client.30184 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 -- 172.21.15.12:0/965788950 >> 172.21.15.12:0/965788950 conn(0x560a1f4d51b0 msgr2=0x560a1f57b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 -- 172.21.15.12:0/965788950 shutdown_connections 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.183+0000 7efecb70db80 1 -- 172.21.15.12:0/965788950 wait complete. 2024-05-09T00:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.184+0000 7efecb70db80 20 client.30184 trim_cache size 0 max 16384 2024-05-09T00:04:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efecb70db80 1 Processor -- start 2024-05-09T00:04:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efecb70db80 1 -- start start 2024-05-09T00:04:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efecb70db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 0x560a1f571910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efecb70db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f5809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efecb70db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f57ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efecb70db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f57c400 con 0x560a1f57c950 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efecb70db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f58d680 con 0x560a1f577480 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efecb70db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f5725f0 con 0x560a1f57d270 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efebcaee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f5809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efebcaee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f5809e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53330/0 (socket says 172.21.15.12:53330) 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efebcaee700 1 -- 172.21.15.12:0/485207174 learned_addr learned my addr 172.21.15.12:0/485207174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.202+0000 7efebdaf0700 1 --2- 172.21.15.12:0/485207174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f57ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efebd2ef700 1 --2- 172.21.15.12:0/485207174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 0x560a1f571910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efebcaee700 1 -- 172.21.15.12:0/485207174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 msgr2=0x560a1f57ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efebcaee700 1 --2- 172.21.15.12:0/485207174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f57ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efebcaee700 1 -- 172.21.15.12:0/485207174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 msgr2=0x560a1f571910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efebcaee700 1 --2- 172.21.15.12:0/485207174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 0x560a1f571910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efebcaee700 1 -- 172.21.15.12:0/485207174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a1f48d500 con 0x560a1f57c950 2024-05-09T00:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efebd2ef700 1 --2- 172.21.15.12:0/485207174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 0x560a1f571910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efebcaee700 1 --2- 172.21.15.12:0/485207174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f5809e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7efeb8103550 tx=0x7efea8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872f52544ae5e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efea3fff700 1 -- 172.21.15.12:0/485207174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8024070 con 0x560a1f57c950 2024-05-09T00:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efea3fff700 1 -- 172.21.15.12:0/485207174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea801f810 con 0x560a1f57c950 2024-05-09T00:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.203+0000 7efea3fff700 1 -- 172.21.15.12:0/485207174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8003870 con 0x560a1f57c950 2024-05-09T00:04:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.204+0000 7efecb70db80 1 -- 172.21.15.12:0/485207174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 msgr2=0x560a1f5809e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.204+0000 7efecb70db80 1 --2- 172.21.15.12:0/485207174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f5809e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7efeb8103550 tx=0x7efea8014000 comp rx=0 tx=0).stop 2024-05-09T00:04:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.204+0000 7efecb70db80 1 -- 172.21.15.12:0/485207174 shutdown_connections 2024-05-09T00:04:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.204+0000 7efecb70db80 1 --2- 172.21.15.12:0/485207174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f57d270 0x560a1f57ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.204+0000 7efecb70db80 1 --2- 172.21.15.12:0/485207174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f577480 0x560a1f571910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.204+0000 7efecb70db80 1 --2- 172.21.15.12:0/485207174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f5809e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.204+0000 7efecb70db80 1 -- 172.21.15.12:0/485207174 >> 172.21.15.12:0/485207174 conn(0x560a1f4d51b0 msgr2=0x560a1f575510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.204+0000 7efecb70db80 1 -- 172.21.15.12:0/485207174 shutdown_connections 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.204+0000 7efecb70db80 1 -- 172.21.15.12:0/485207174 wait complete. 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.205+0000 7efecb70db80 1 -- start start 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.205+0000 7efecb70db80 1 Processor -- start 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.205+0000 7efecb70db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f577480 0x560a1f585bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.205+0000 7efecb70db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f586130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.205+0000 7efecb70db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f57d270 0x560a1f586670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efecb70db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f5ee510 con 0x560a1f57c950 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efecb70db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f5ee650 con 0x560a1f57d270 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efecb70db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1f5ee810 con 0x560a1f577480 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efecb70db80 10 client.0 _ll_register_callbacks cb 0x560a1f46cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efebcaee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f586130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efebcaee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f586130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53344/0 (socket says 172.21.15.12:53344) 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efebcaee700 1 -- 172.21.15.12:0/2617229292 learned_addr learned my addr 172.21.15.12:0/2617229292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efebcaee700 1 -- 172.21.15.12:0/2617229292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f577480 msgr2=0x560a1f585bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efebcaee700 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f577480 0x560a1f585bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efebcaee700 1 -- 172.21.15.12:0/2617229292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f57d270 msgr2=0x560a1f586670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efebcaee700 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f57d270 0x560a1f586670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efebcaee700 1 -- 172.21.15.12:0/2617229292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea8017070 con 0x560a1f57c950 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efebcaee700 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f586130 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7efea801fdc0 tx=0x7efea80094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f5d305ec0dcd43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efea0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.206+0000 7efea0ff9700 1 -- 172.21.15.12:0/2617229292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8024040 con 0x560a1f57c950 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efea0ff9700 1 -- 172.21.15.12:0/2617229292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efea8005510 con 0x560a1f57c950 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efecb70db80 1 -- 172.21.15.12:0/2617229292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a1f420210 con 0x560a1f57c950 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efea0ff9700 1 -- 172.21.15.12:0/2617229292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7efea80056d0 con 0x560a1f57c950 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efecb70db80 10 client.20460 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efecb70db80 1 -- 172.21.15.12:0/2617229292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x560a1f34cc70 con 0x560a1f57c950 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efea0ff9700 1 -- 172.21.15.12:0/2617229292 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea803e020 con 0x560a1f57c950 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efea0ff9700 1 -- 172.21.15.12:0/2617229292 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efea8003c30 con 0x560a1f57c950 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efecb70db80 10 client.20460 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efecb70db80 10 client.20460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efecb70db80 20 client.20460 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efecb70db80 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a1f5effc0 0x560a1f5f2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efecb70db80 4 client.20460 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.207+0000 7efecb70db80 1 -- 172.21.15.12:0/2617229292 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a1f315c90 con 0x560a1f5effc0 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.208+0000 7efebd2ef700 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a1f5effc0 0x560a1f5f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.208+0000 7efea0ff9700 10 client.20460 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:32.208+0000 7efebd2ef700 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a1f5effc0 0x560a1f5f2480 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64828d1cdbf40f14 server_cookie=58675ac071521e6c in_seq=0 out_seq=0 2024-05-09T00:04:32.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 186671 36.56 MB/sec execute 147 sec latency 3554.837 ms 2024-05-09T00:04:33.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 187204 36.52 MB/sec execute 148 sec latency 4554.871 ms 2024-05-09T00:04:34.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:33 smithi204 ceph-mon[47373]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 956 op/s 2024-05-09T00:04:34.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:33 smithi160 ceph-mon[46152]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 956 op/s 2024-05-09T00:04:34.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:33 smithi012 ceph-mon[42503]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 956 op/s 2024-05-09T00:04:34.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 187843 36.46 MB/sec execute 149 sec latency 5554.905 ms 2024-05-09T00:04:35.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188532 36.52 MB/sec execute 150 sec latency 6554.943 ms 2024-05-09T00:04:36.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:35 smithi204 ceph-mon[47373]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 18 MiB/s wr, 795 op/s 2024-05-09T00:04:36.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:35 smithi160 ceph-mon[46152]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 18 MiB/s wr, 795 op/s 2024-05-09T00:04:36.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:35 smithi012 ceph-mon[42503]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 18 MiB/s wr, 795 op/s 2024-05-09T00:04:36.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 189117 36.47 MB/sec execute 151 sec latency 7554.986 ms 2024-05-09T00:04:37.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 189838 36.48 MB/sec execute 152 sec latency 8555.050 ms 2024-05-09T00:04:38.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:37 smithi204 ceph-mon[47373]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 926 op/s 2024-05-09T00:04:38.019 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:37 smithi160 ceph-mon[46152]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 926 op/s 2024-05-09T00:04:38.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:37 smithi012 ceph-mon[42503]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 926 op/s 2024-05-09T00:04:38.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 190356 36.43 MB/sec execute 153 sec latency 9555.100 ms 2024-05-09T00:04:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.633+0000 7efea0ff9700 1 -- 172.21.15.12:0/2617229292 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7efeb4014040 con 0x560a1f5effc0 2024-05-09T00:04:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.633+0000 7efea0ff9700 10 client.20460 handle_command_reply: tid=0 2024-05-09T00:04:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.633+0000 7efecb70db80 1 client.20460 shutdown 2024-05-09T00:04:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.633+0000 7efecb70db80 1 -- 172.21.15.12:0/2617229292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 msgr2=0x560a1f586130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.633+0000 7efecb70db80 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f586130 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7efea801fdc0 tx=0x7efea80094b0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.635+0000 7efecb70db80 1 -- 172.21.15.12:0/2617229292 shutdown_connections 2024-05-09T00:04:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.635+0000 7efecb70db80 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560a1f5effc0 0x560a1f5f2480 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.635+0000 7efecb70db80 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a1f57d270 0x560a1f586670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.635+0000 7efecb70db80 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a1f577480 0x560a1f585bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.635+0000 7efecb70db80 1 --2- 172.21.15.12:0/2617229292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560a1f57c950 0x560a1f586130 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.635+0000 7efecb70db80 1 -- 172.21.15.12:0/2617229292 >> 172.21.15.12:0/2617229292 conn(0x560a1f4d51b0 msgr2=0x560a1f57ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.635+0000 7efea0ff9700 0 client.20460 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.635+0000 7efecb70db80 1 -- 172.21.15.12:0/2617229292 shutdown_connections 2024-05-09T00:04:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.635+0000 7efecb70db80 1 -- 172.21.15.12:0/2617229292 wait complete. 2024-05-09T00:04:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:04:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:04:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:04:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:04:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.637+0000 7efecb70db80 20 client.20460 trim_cache size 0 max 16384 2024-05-09T00:04:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.638+0000 7efebf553700 1 -- 172.21.15.12:0/1459475200 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efea40a1390 msgr2=0x7efea40a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.638+0000 7efebf553700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efea40a1390 0x7efea40a3850 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efeb81031d0 tx=0x7efea8009260 comp rx=0 tx=0).stop 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.638+0000 7efebf553700 1 -- 172.21.15.12:0/1459475200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb8100010 msgr2=0x7efeb8101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.638+0000 7efebf553700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb8100010 0x7efeb8101aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efeb400d950 tx=0x7efeb400dc60 comp rx=0 tx=0).stop 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.638+0000 7efebf553700 1 -- 172.21.15.12:0/1459475200 shutdown_connections 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.638+0000 7efebf553700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efea40a1390 0x7efea40a3850 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efeb81031d0 tx=0x7efea8009260 comp rx=0 tx=0).stop 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.638+0000 7efebf553700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb8102520 0x7efeb81a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.638+0000 7efebf553700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb8100010 0x7efeb8101aa0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.638+0000 7efebf553700 1 --2- 172.21.15.12:0/1459475200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeb8105130 0x7efeb8101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.638+0000 7efebf553700 1 -- 172.21.15.12:0/1459475200 >> 172.21.15.12:0/1459475200 conn(0x7efeb80fb870 msgr2=0x7efeb8069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.639+0000 7efebf553700 1 -- 172.21.15.12:0/1459475200 shutdown_connections 2024-05-09T00:04:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:38.639+0000 7efebf553700 1 -- 172.21.15.12:0/1459475200 wait complete. 2024-05-09T00:04:38.794 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:04:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.971+0000 7f4fdfce6700 1 -- 172.21.15.12:0/439818190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fd8104e70 msgr2=0x7f4fd8105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:38.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.971+0000 7f4fdfce6700 1 --2- 172.21.15.12:0/439818190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fd8104e70 0x7f4fd8105250 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400ddd0 tx=0x7f4fd40074a0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.971+0000 7f4fdfce6700 1 -- 172.21.15.12:0/439818190 shutdown_connections 2024-05-09T00:04:38.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.971+0000 7f4fdfce6700 1 --2- 172.21.15.12:0/439818190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8101330 0x7f4fd810cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.971+0000 7f4fdfce6700 1 --2- 172.21.15.12:0/439818190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fd8104e70 0x7f4fd8105250 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.972+0000 7f4fdfce6700 1 --2- 172.21.15.12:0/439818190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fd8100970 0x7f4fd8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.972+0000 7f4fdfce6700 1 -- 172.21.15.12:0/439818190 >> 172.21.15.12:0/439818190 conn(0x7f4fd80751c0 msgr2=0x7f4fd80755d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:38.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.972+0000 7f4fdfce6700 1 -- 172.21.15.12:0/439818190 shutdown_connections 2024-05-09T00:04:38.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.972+0000 7f4fdfce6700 1 -- 172.21.15.12:0/439818190 wait complete. 2024-05-09T00:04:38.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.972+0000 7f4fdfce6700 1 Processor -- start 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdfce6700 1 -- start start 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdfce6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fd8100970 0x7f4fd81130c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdfce6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8101330 0x7f4fd8113620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdfce6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fd8104e70 0x7f4fd810e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdfce6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd810ea60 con 0x7f4fd8100970 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdfce6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd810ebd0 con 0x7f4fd8104e70 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdfce6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd810ed90 con 0x7f4fd8101330 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdda82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fd8100970 0x7f4fd81130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdda82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fd8100970 0x7f4fd81130c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57624/0 (socket says 172.21.15.12:57624) 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdda82700 1 -- 172.21.15.12:0/3400995272 learned_addr learned my addr 172.21.15.12:0/3400995272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:38.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdda82700 1 -- 172.21.15.12:0/3400995272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8101330 msgr2=0x7f4fd8113620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:38.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdda82700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8101330 0x7f4fd8113620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdda82700 1 -- 172.21.15.12:0/3400995272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fd8104e70 msgr2=0x7f4fd810e290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:04:38.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdda82700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fd8104e70 0x7f4fd810e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.973+0000 7f4fdda82700 1 -- 172.21.15.12:0/3400995272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd400da00 con 0x7f4fd8100970 2024-05-09T00:04:38.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.974+0000 7f4fdda82700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fd8100970 0x7f4fd81130c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400dda0 tx=0x7f4fd4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:38.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.974+0000 7f4fceffd700 1 -- 172.21.15.12:0/3400995272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4004460 con 0x7f4fd8100970 2024-05-09T00:04:38.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.974+0000 7f4fceffd700 1 -- 172.21.15.12:0/3400995272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd4013040 con 0x7f4fd8100970 2024-05-09T00:04:38.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.974+0000 7f4fdfce6700 1 -- 172.21.15.12:0/3400995272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd810f010 con 0x7f4fd8100970 2024-05-09T00:04:38.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.974+0000 7f4fdfce6700 1 -- 172.21.15.12:0/3400995272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd811b080 con 0x7f4fd8100970 2024-05-09T00:04:38.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.974+0000 7f4fceffd700 1 -- 172.21.15.12:0/3400995272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40176b0 con 0x7f4fd8100970 2024-05-09T00:04:38.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.978+0000 7f4fceffd700 1 -- 172.21.15.12:0/3400995272 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4fd4017810 con 0x7f4fd8100970 2024-05-09T00:04:38.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.979+0000 7f4fceffd700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fc40a1390 0x7f4fc40a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.979+0000 7f4fceffd700 1 -- 172.21.15.12:0/3400995272 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4fd40cf330 con 0x7f4fd8100970 2024-05-09T00:04:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.979+0000 7f4fdd281700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fc40a1390 0x7f4fc40a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.980+0000 7f4fdd281700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fc40a1390 0x7f4fc40a3850 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4fd810f950 tx=0x7f4fc8006d20 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.981+0000 7f4febea0b80 1 Processor -- start 2024-05-09T00:04:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.981+0000 7f4febea0b80 1 -- start start 2024-05-09T00:04:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fdee00 0x55a047fdf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fd9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fd9890 0x55a047fd9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4febea0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047f30150 con 0x55a047fd9890 2024-05-09T00:04:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4febea0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047f2db10 con 0x55a047fdee00 2024-05-09T00:04:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4febea0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047f29400 con 0x55a047fdf720 2024-05-09T00:04:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fdd281700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fd9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fdd281700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fd9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39180/0 (socket says 172.21.15.12:39180) 2024-05-09T00:04:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fdd281700 1 -- 172.21.15.12:0/1377291423 learned_addr learned my addr 172.21.15.12:0/1377291423 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fde283700 1 --2- 172.21.15.12:0/1377291423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fd9890 0x55a047fd9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fdda82700 1 --2- 172.21.15.12:0/1377291423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fdee00 0x55a047fdf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fdd281700 1 -- 172.21.15.12:0/1377291423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fdee00 msgr2=0x55a047fdf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fdd281700 1 --2- 172.21.15.12:0/1377291423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fdee00 0x55a047fdf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fdd281700 1 -- 172.21.15.12:0/1377291423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fd9890 msgr2=0x55a047fd9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fdd281700 1 --2- 172.21.15.12:0/1377291423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fd9890 0x55a047fd9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fdd281700 1 -- 172.21.15.12:0/1377291423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a047daa3b0 con 0x55a047fdf720 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.982+0000 7f4fde283700 1 --2- 172.21.15.12:0/1377291423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fd9890 0x55a047fd9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.983+0000 7f4fdd281700 1 --2- 172.21.15.12:0/1377291423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fd9350 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80158f0 tx=0x7f4fc8006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=583752b0cf4fdb40 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.983+0000 7f4fc3fff700 1 -- 172.21.15.12:0/1377291423 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc801a070 con 0x55a047fdf720 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.983+0000 7f4fc3fff700 1 -- 172.21.15.12:0/1377291423 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc80128f0 con 0x55a047fdf720 2024-05-09T00:04:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.983+0000 7f4fc3fff700 1 -- 172.21.15.12:0/1377291423 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc80137a0 con 0x55a047fdf720 2024-05-09T00:04:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.984+0000 7f4febea0b80 1 -- 172.21.15.12:0/1377291423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 msgr2=0x55a047fd9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.984+0000 7f4febea0b80 1 --2- 172.21.15.12:0/1377291423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fd9350 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80158f0 tx=0x7f4fc8006f90 comp rx=0 tx=0).stop 2024-05-09T00:04:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.986+0000 7f4febea0b80 1 -- 172.21.15.12:0/1377291423 shutdown_connections 2024-05-09T00:04:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.986+0000 7f4febea0b80 1 --2- 172.21.15.12:0/1377291423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fd9890 0x55a047fd9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.987+0000 7f4febea0b80 1 --2- 172.21.15.12:0/1377291423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fdee00 0x55a047fdf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.987+0000 7f4febea0b80 1 --2- 172.21.15.12:0/1377291423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fd9350 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.987+0000 7f4febea0b80 1 -- 172.21.15.12:0/1377291423 >> 172.21.15.12:0/1377291423 conn(0x55a047f37210 msgr2=0x55a047fd3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.987+0000 7f4febea0b80 1 -- 172.21.15.12:0/1377291423 shutdown_connections 2024-05-09T00:04:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.987+0000 7f4febea0b80 1 -- 172.21.15.12:0/1377291423 wait complete. 2024-05-09T00:04:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.987+0000 7f4febea0b80 1 -- start start 2024-05-09T00:04:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.987+0000 7f4febea0b80 1 Processor -- start 2024-05-09T00:04:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fd9890 0x55a047fe2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdee00 0x55a047fe30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fdf720 0x55a047fe3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4febea0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047ec5560 con 0x55a047fdee00 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4febea0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047ec5200 con 0x55a047fdf720 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4febea0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047dac6a0 con 0x55a047fd9890 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4febea0b80 10 client.0 _ll_register_callbacks cb 0x55a047eceb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4fdda82700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fd9890 0x55a047fe2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4fdda82700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fd9890 0x55a047fe2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39194/0 (socket says 172.21.15.12:39194) 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4fdda82700 1 -- 172.21.15.12:0/3981211972 learned_addr learned my addr 172.21.15.12:0/3981211972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.988+0000 7f4fdd281700 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdee00 0x55a047fe30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4fde283700 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fdf720 0x55a047fe3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:38.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4fdd281700 1 -- 172.21.15.12:0/3981211972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fd9890 msgr2=0x55a047fe2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:38.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4fdd281700 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fd9890 0x55a047fe2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4fdd281700 1 -- 172.21.15.12:0/3981211972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fdf720 msgr2=0x55a047fe3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4fdd281700 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fdf720 0x55a047fe3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4fdd281700 1 -- 172.21.15.12:0/3981211972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd00098c0 con 0x55a047fdee00 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4fdd281700 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdee00 0x55a047fe30c0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8015fd0 tx=0x7f4fc8012660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df27dd4867c585d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4fc17fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4fc17fa700 1 -- 172.21.15.12:0/3981211972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc802d070 con 0x55a047fdee00 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4fc17fa700 1 -- 172.21.15.12:0/3981211972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4fc8027630 con 0x55a047fdee00 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.989+0000 7f4febea0b80 1 -- 172.21.15.12:0/3981211972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a047e82270 con 0x55a047fdee00 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.990+0000 7f4fc17fa700 1 -- 172.21.15.12:0/3981211972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc801a050 con 0x55a047fdee00 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.990+0000 7f4febea0b80 10 client.20472 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.990+0000 7f4febea0b80 1 -- 172.21.15.12:0/3981211972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7f4fc80155d0 con 0x55a047fdee00 2024-05-09T00:04:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.990+0000 7f4fc17fa700 1 -- 172.21.15.12:0/3981211972 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a047e82270 con 0x55a047fdee00 2024-05-09T00:04:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.991+0000 7f4fc17fa700 1 -- 172.21.15.12:0/3981211972 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4fc8030070 con 0x55a047fdee00 2024-05-09T00:04:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.991+0000 7f4febea0b80 10 client.20472 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.991+0000 7f4febea0b80 10 client.20472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.991+0000 7f4febea0b80 20 client.20472 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.991+0000 7f4febea0b80 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a047ff3190 0x55a047ff55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.991+0000 7f4febea0b80 4 client.20472 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.991+0000 7f4febea0b80 1 -- 172.21.15.12:0/3981211972 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a047fa8810 con 0x55a047ff3190 2024-05-09T00:04:38.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.991+0000 7f4fdda82700 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a047ff3190 0x55a047ff55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:38.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.992+0000 7f4fc17fa700 10 client.20472 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:38.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.992+0000 7f4fdda82700 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a047ff3190 0x55a047ff55b0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be05a28fcfd37dbb server_cookie=4481a60c25737056 in_seq=0 out_seq=0 2024-05-09T00:04:38.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.996+0000 7f4fc17fa700 1 -- 172.21.15.12:0/3981211972 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a047fa8810 con 0x55a047ff3190 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.996+0000 7f4fc17fa700 10 client.20472 handle_command_reply: tid=0 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.996+0000 7f4febea0b80 1 client.20472 shutdown 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 1 -- 172.21.15.12:0/3981211972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdee00 msgr2=0x55a047fe30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdee00 0x55a047fe30c0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8015fd0 tx=0x7f4fc8012660 comp rx=0 tx=0).stop 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 1 -- 172.21.15.12:0/3981211972 shutdown_connections 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a047ff3190 0x55a047ff55b0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fdf720 0x55a047fe3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fd9890 0x55a047fe2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 1 --2- 172.21.15.12:0/3981211972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdee00 0x55a047fe30c0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 1 -- 172.21.15.12:0/3981211972 >> 172.21.15.12:0/3981211972 conn(0x55a047f37210 msgr2=0x55a047fd35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4fc17fa700 0 client.20472 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 1 -- 172.21.15.12:0/3981211972 shutdown_connections 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 1 -- 172.21.15.12:0/3981211972 wait complete. 2024-05-09T00:04:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:38.997+0000 7f4febea0b80 20 client.20472 trim_cache size 0 max 16384 2024-05-09T00:04:39.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4febea0b80 1 Processor -- start 2024-05-09T00:04:39.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4febea0b80 1 -- start start 2024-05-09T00:04:39.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fd9890 0x55a047fd3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdee00 0x55a047fe30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdf720 0x55a047fe3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4febea0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047fe5700 con 0x55a047fdf720 2024-05-09T00:04:39.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4febea0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047fd5640 con 0x55a047fd9890 2024-05-09T00:04:39.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4febea0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047fd0df0 con 0x55a047fdee00 2024-05-09T00:04:39.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4fdd281700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdee00 0x55a047fe30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4fdd281700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdee00 0x55a047fe30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39210/0 (socket says 172.21.15.12:39210) 2024-05-09T00:04:39.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.024+0000 7f4fdd281700 1 -- 172.21.15.12:0/539682177 learned_addr learned my addr 172.21.15.12:0/539682177 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.025+0000 7f4fdd281700 1 -- 172.21.15.12:0/539682177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fd9890 msgr2=0x55a047fd3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.025+0000 7f4fdd281700 1 --2- 172.21.15.12:0/539682177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fd9890 0x55a047fd3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.025+0000 7f4fdd281700 1 -- 172.21.15.12:0/539682177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdf720 msgr2=0x55a047fe3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.025+0000 7f4fdd281700 1 --2- 172.21.15.12:0/539682177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdf720 0x55a047fe3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.025+0000 7f4fdd281700 1 -- 172.21.15.12:0/539682177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a047d56b60 con 0x55a047fdee00 2024-05-09T00:04:39.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.025+0000 7f4fdd281700 1 --2- 172.21.15.12:0/539682177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdee00 0x55a047fe30d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8000c00 tx=0x7f4fc800a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6726e3d0b7d33dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.026+0000 7f4fccff9700 1 -- 172.21.15.12:0/539682177 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc802d030 con 0x55a047fdee00 2024-05-09T00:04:39.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.026+0000 7f4fccff9700 1 -- 172.21.15.12:0/539682177 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc8035020 con 0x55a047fdee00 2024-05-09T00:04:39.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.026+0000 7f4fccff9700 1 -- 172.21.15.12:0/539682177 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc801a040 con 0x55a047fdee00 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 -- 172.21.15.12:0/539682177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdee00 msgr2=0x55a047fe30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 --2- 172.21.15.12:0/539682177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdee00 0x55a047fe30d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8000c00 tx=0x7f4fc800a040 comp rx=0 tx=0).stop 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 -- 172.21.15.12:0/539682177 shutdown_connections 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 --2- 172.21.15.12:0/539682177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdf720 0x55a047fe3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 --2- 172.21.15.12:0/539682177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fd9890 0x55a047fd3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 --2- 172.21.15.12:0/539682177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdee00 0x55a047fe30d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 -- 172.21.15.12:0/539682177 >> 172.21.15.12:0/539682177 conn(0x55a047f37210 msgr2=0x55a047fdcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 -- 172.21.15.12:0/539682177 shutdown_connections 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 -- 172.21.15.12:0/539682177 wait complete. 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 -- start start 2024-05-09T00:04:39.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.027+0000 7f4febea0b80 1 Processor -- start 2024-05-09T00:04:39.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.030+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fd9890 0x55a047fea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.030+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdee00 0x55a047fe81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.030+0000 7f4febea0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fe8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.030+0000 7f4febea0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047fe8e30 con 0x55a047fdee00 2024-05-09T00:04:39.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.030+0000 7f4febea0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a047fe8fa0 con 0x55a047fd9890 2024-05-09T00:04:39.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.030+0000 7f4febea0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a048048690 con 0x55a047fdf720 2024-05-09T00:04:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.030+0000 7f4febea0b80 10 client.0 _ll_register_callbacks cb 0x55a047eceb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.030+0000 7f4fde283700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fe8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.030+0000 7f4fde283700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fe8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39220/0 (socket says 172.21.15.12:39220) 2024-05-09T00:04:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.030+0000 7f4fde283700 1 -- 172.21.15.12:0/1418274919 learned_addr learned my addr 172.21.15.12:0/1418274919 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.031+0000 7f4fde283700 1 -- 172.21.15.12:0/1418274919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fd9890 msgr2=0x55a047fea5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:04:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.031+0000 7f4fde283700 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fd9890 0x55a047fea5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.031+0000 7f4fde283700 1 -- 172.21.15.12:0/1418274919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdee00 msgr2=0x55a047fe81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.031+0000 7f4fde283700 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdee00 0x55a047fe81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.031+0000 7f4fde283700 1 -- 172.21.15.12:0/1418274919 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc80155d0 con 0x55a047fdf720 2024-05-09T00:04:39.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.032+0000 7f4fde283700 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fe8720 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00042c0 tx=0x7f4fd00043a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b51e0e7bbc1c6ad1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.032+0000 7f4fc17fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:39.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.032+0000 7f4fc17fa700 1 -- 172.21.15.12:0/1418274919 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0004930 con 0x55a047fdf720 2024-05-09T00:04:39.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.032+0000 7f4fc17fa700 1 -- 172.21.15.12:0/1418274919 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4fd0004d40 con 0x55a047fdf720 2024-05-09T00:04:39.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.032+0000 7f4febea0b80 1 -- 172.21.15.12:0/1418274919 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fc80128f0 con 0x55a047fdf720 2024-05-09T00:04:39.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4fc17fa700 1 -- 172.21.15.12:0/1418274919 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fd0007710 con 0x55a047fdf720 2024-05-09T00:04:39.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4febea0b80 10 client.30202 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:39.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4febea0b80 1 -- 172.21.15.12:0/1418274919 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55a047eef560 con 0x55a047fdf720 2024-05-09T00:04:39.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4fc17fa700 1 -- 172.21.15.12:0/1418274919 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc80128f0 con 0x55a047fdf720 2024-05-09T00:04:39.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4fc17fa700 1 -- 172.21.15.12:0/1418274919 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4fd0016b20 con 0x55a047fdf720 2024-05-09T00:04:39.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4febea0b80 10 client.30202 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:39.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4febea0b80 10 client.30202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:39.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4febea0b80 20 client.30202 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4febea0b80 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a048049e70 0x55a04804c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4febea0b80 4 client.30202 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:04:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.033+0000 7f4febea0b80 1 -- 172.21.15.12:0/1418274919 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a047f21ec0 con 0x55a048049e70 2024-05-09T00:04:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.034+0000 7f4fdda82700 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a048049e70 0x55a04804c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.034+0000 7f4fdda82700 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a048049e70 0x55a04804c290 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3632604f72147d3 server_cookie=553cb443b3cc45b0 in_seq=0 out_seq=0 2024-05-09T00:04:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.034+0000 7f4fc17fa700 10 client.30202 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4fc17fa700 1 -- 172.21.15.12:0/1418274919 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a047f21ec0 con 0x55a048049e70 2024-05-09T00:04:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4fc17fa700 10 client.30202 handle_command_reply: tid=0 2024-05-09T00:04:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4febea0b80 1 client.30202 shutdown 2024-05-09T00:04:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4febea0b80 1 -- 172.21.15.12:0/1418274919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 msgr2=0x55a047fe8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4febea0b80 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fe8720 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00042c0 tx=0x7f4fd00043a0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4febea0b80 1 -- 172.21.15.12:0/1418274919 shutdown_connections 2024-05-09T00:04:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4febea0b80 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a047fdf720 0x55a047fe8720 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4febea0b80 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a048049e70 0x55a04804c290 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4febea0b80 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a047fd9890 0x55a047fea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4febea0b80 1 --2- 172.21.15.12:0/1418274919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a047fdee00 0x55a047fe81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4febea0b80 1 -- 172.21.15.12:0/1418274919 >> 172.21.15.12:0/1418274919 conn(0x55a047f37210 msgr2=0x55a047fe1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.038+0000 7f4fc17fa700 0 client.30202 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:39.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.041+0000 7f4febea0b80 1 -- 172.21.15.12:0/1418274919 shutdown_connections 2024-05-09T00:04:39.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.042+0000 7f4febea0b80 1 -- 172.21.15.12:0/1418274919 wait complete. 2024-05-09T00:04:39.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.042+0000 7f4febea0b80 20 client.30202 trim_cache size 0 max 16384 2024-05-09T00:04:39.036 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:04:39.036 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:04:39.036 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:04:39.036 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:04:39.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.042+0000 7f4fdfce6700 1 -- 172.21.15.12:0/3400995272 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fc40a1390 msgr2=0x7f4fc40a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.042+0000 7f4fdfce6700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fc40a1390 0x7f4fc40a3850 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4fd810f950 tx=0x7f4fc8006d20 comp rx=0 tx=0).stop 2024-05-09T00:04:39.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.042+0000 7f4fdfce6700 1 -- 172.21.15.12:0/3400995272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fd8100970 msgr2=0x7f4fd81130c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.042+0000 7f4fdfce6700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fd8100970 0x7f4fd81130c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400dda0 tx=0x7f4fd4007dd0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.043+0000 7f4fdfce6700 1 -- 172.21.15.12:0/3400995272 shutdown_connections 2024-05-09T00:04:39.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.043+0000 7f4fdfce6700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fc40a1390 0x7f4fc40a3850 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.043+0000 7f4fdfce6700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fd8104e70 0x7f4fd810e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.043+0000 7f4fdfce6700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fd8100970 0x7f4fd81130c0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.043+0000 7f4fdfce6700 1 --2- 172.21.15.12:0/3400995272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8101330 0x7f4fd8113620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.043+0000 7f4fdfce6700 1 -- 172.21.15.12:0/3400995272 >> 172.21.15.12:0/3400995272 conn(0x7f4fd80751c0 msgr2=0x7f4fd80fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.043+0000 7f4fdfce6700 1 -- 172.21.15.12:0/3400995272 shutdown_connections 2024-05-09T00:04:39.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.043+0000 7f4fdfce6700 1 -- 172.21.15.12:0/3400995272 wait complete. 2024-05-09T00:04:39.054 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:04:39.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:04:39.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.209+0000 7f7b2e533700 1 -- 172.21.15.12:0/1909004140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b280fff40 msgr2=0x7f7b28100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.209+0000 7f7b2e533700 1 --2- 172.21.15.12:0/1909004140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b280fff40 0x7f7b28100320 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009990 tx=0x7f7b18009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.209+0000 7f7b2e533700 1 -- 172.21.15.12:0/1909004140 shutdown_connections 2024-05-09T00:04:39.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.209+0000 7f7b2e533700 1 --2- 172.21.15.12:0/1909004140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b28105060 0x7f7b2810cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.209+0000 7f7b2e533700 1 --2- 172.21.15.12:0/1909004140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b280fff40 0x7f7b28100320 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.209+0000 7f7b2e533700 1 --2- 172.21.15.12:0/1909004140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b281008f0 0x7f7b28104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.209+0000 7f7b2e533700 1 -- 172.21.15.12:0/1909004140 >> 172.21.15.12:0/1909004140 conn(0x7f7b280fb7e0 msgr2=0x7f7b280fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.209+0000 7f7b2e533700 1 -- 172.21.15.12:0/1909004140 shutdown_connections 2024-05-09T00:04:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.209+0000 7f7b2e533700 1 -- 172.21.15.12:0/1909004140 wait complete. 2024-05-09T00:04:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.210+0000 7f7b2e533700 1 Processor -- start 2024-05-09T00:04:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.210+0000 7f7b2e533700 1 -- start start 2024-05-09T00:04:39.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b2e533700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280fff40 0x7f7b28199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b2e533700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b28105060 0x7f7b28199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b2e533700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b2819ddf0 0x7f7b2819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b2e533700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2819e870 con 0x7f7b2819ddf0 2024-05-09T00:04:39.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b2e533700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2819e9e0 con 0x7f7b280fff40 2024-05-09T00:04:39.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b2e533700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2819eba0 con 0x7f7b28105060 2024-05-09T00:04:39.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b277fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b28105060 0x7f7b28199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b2cad0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b2819ddf0 0x7f7b2819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b277fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b28105060 0x7f7b28199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39250/0 (socket says 172.21.15.12:39250) 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b277fe700 1 -- 172.21.15.12:0/2071664107 learned_addr learned my addr 172.21.15.12:0/2071664107 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b27fff700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280fff40 0x7f7b28199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b277fe700 1 -- 172.21.15.12:0/2071664107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280fff40 msgr2=0x7f7b28199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b277fe700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280fff40 0x7f7b28199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b277fe700 1 -- 172.21.15.12:0/2071664107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b2819ddf0 msgr2=0x7f7b2819e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b277fe700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b2819ddf0 0x7f7b2819e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.211+0000 7f7b277fe700 1 -- 172.21.15.12:0/2071664107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c009520 con 0x7f7b28105060 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.212+0000 7f7b277fe700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b28105060 0x7f7b28199a50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00ea30 tx=0x7f7b1c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.212+0000 7f7b257fa700 1 -- 172.21.15.12:0/2071664107 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c00cbd0 con 0x7f7b28105060 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.212+0000 7f7b2e533700 1 -- 172.21.15.12:0/2071664107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b180095f0 con 0x7f7b28105060 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.212+0000 7f7b257fa700 1 -- 172.21.15.12:0/2071664107 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b1c00eed0 con 0x7f7b28105060 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.212+0000 7f7b257fa700 1 -- 172.21.15.12:0/2071664107 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c003d20 con 0x7f7b28105060 2024-05-09T00:04:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.212+0000 7f7b2e533700 1 -- 172.21.15.12:0/2071664107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b281a6450 con 0x7f7b28105060 2024-05-09T00:04:39.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.213+0000 7f7b2e533700 1 -- 172.21.15.12:0/2071664107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b281030a0 con 0x7f7b28105060 2024-05-09T00:04:39.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.214+0000 7f7b257fa700 1 -- 172.21.15.12:0/2071664107 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7b1c003ee0 con 0x7f7b28105060 2024-05-09T00:04:39.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.215+0000 7f7b257fa700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b100a1290 0x7f7b100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.215+0000 7f7b257fa700 1 -- 172.21.15.12:0/2071664107 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7b1c014070 con 0x7f7b28105060 2024-05-09T00:04:39.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.215+0000 7f7b27fff700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b100a1290 0x7f7b100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.216+0000 7f7b27fff700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b100a1290 0x7f7b100a3750 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800b2e0 tx=0x7f7b18005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.220+0000 7f7b257fa700 1 -- 172.21.15.12:0/2071664107 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7b1c096460 con 0x7f7b28105060 2024-05-09T00:04:39.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 190921 36.38 MB/sec execute 154 sec latency 9906.504 ms 2024-05-09T00:04:39.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.527+0000 7f7b2e533700 1 -- 172.21.15.12:0/2071664107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b28101410 con 0x7f7b28105060 2024-05-09T00:04:39.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.529+0000 7f7b257fa700 1 -- 172.21.15.12:0/2071664107 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+10872 (secure 0 0 0) 0x7f7b1c099a80 con 0x7f7b28105060 2024-05-09T00:04:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:04:39.527 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":74,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:03:37.323291+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_30034":{"gid":30034,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6833/181468454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":181468454},{"type":"v1","addr":"172.21.15.160:6833","nonce":181468454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:04:39.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 -- 172.21.15.12:0/2071664107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b100a1290 msgr2=0x7f7b100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b100a1290 0x7f7b100a3750 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800b2e0 tx=0x7f7b18005fd0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 -- 172.21.15.12:0/2071664107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b28105060 msgr2=0x7f7b28199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b28105060 0x7f7b28199a50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00ea30 tx=0x7f7b1c00ed40 comp rx=0 tx=0).stop 2024-05-09T00:04:39.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 -- 172.21.15.12:0/2071664107 shutdown_connections 2024-05-09T00:04:39.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7b100a1290 0x7f7b100a3750 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b2819ddf0 0x7f7b2819e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280fff40 0x7f7b28199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 --2- 172.21.15.12:0/2071664107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b28105060 0x7f7b28199a50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 -- 172.21.15.12:0/2071664107 >> 172.21.15.12:0/2071664107 conn(0x7f7b280fb7e0 msgr2=0x7f7b280fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 -- 172.21.15.12:0/2071664107 shutdown_connections 2024-05-09T00:04:39.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.534+0000 7f7b2e533700 1 -- 172.21.15.12:0/2071664107 wait complete. 2024-05-09T00:04:39.529 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 74 2024-05-09T00:04:39.544 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:04:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.709+0000 7f0214bc4700 1 -- 172.21.15.12:0/1530521146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0210100060 msgr2=0x7f0210100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.709+0000 7f0214bc4700 1 --2- 172.21.15.12:0/1530521146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0210100060 0x7f0210100440 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f0200009990 tx=0x7f0200009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.709+0000 7f0214bc4700 1 -- 172.21.15.12:0/1530521146 shutdown_connections 2024-05-09T00:04:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.709+0000 7f0214bc4700 1 --2- 172.21.15.12:0/1530521146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0210105160 0x7f021010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.709+0000 7f0214bc4700 1 --2- 172.21.15.12:0/1530521146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0210100060 0x7f0210100440 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.709+0000 7f0214bc4700 1 --2- 172.21.15.12:0/1530521146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210100a10 0x7f0210104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.709+0000 7f0214bc4700 1 -- 172.21.15.12:0/1530521146 >> 172.21.15.12:0/1530521146 conn(0x7f0210076f40 msgr2=0x7f0210077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.709+0000 7f0214bc4700 1 -- 172.21.15.12:0/1530521146 shutdown_connections 2024-05-09T00:04:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.709+0000 7f0214bc4700 1 -- 172.21.15.12:0/1530521146 wait complete. 2024-05-09T00:04:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f0214bc4700 1 Processor -- start 2024-05-09T00:04:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f0214bc4700 1 -- start start 2024-05-09T00:04:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f0214bc4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0210100060 0x7f0210197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f0214bc4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210105160 0x7f02101979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f0214bc4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f021019bcf0 0x7f021019c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f0214bc4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021019c770 con 0x7f0210100060 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f0214bc4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021019c8e0 con 0x7f021019bcf0 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f0214bc4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021019caa0 con 0x7f0210105160 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f020e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0210100060 0x7f0210197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f020ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f021019bcf0 0x7f021019c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f020dd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210105160 0x7f02101979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f020ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f021019bcf0 0x7f021019c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58208/0 (socket says 172.21.15.12:58208) 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.711+0000 7f020ed9d700 1 -- 172.21.15.12:0/2300313587 learned_addr learned my addr 172.21.15.12:0/2300313587 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.710+0000 7f020e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0210100060 0x7f0210197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57692/0 (socket says 172.21.15.12:57692) 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.711+0000 7f020ed9d700 1 -- 172.21.15.12:0/2300313587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210105160 msgr2=0x7f02101979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.711+0000 7f020ed9d700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210105160 0x7f02101979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.711+0000 7f020ed9d700 1 -- 172.21.15.12:0/2300313587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0210100060 msgr2=0x7f0210197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.711+0000 7f020ed9d700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0210100060 0x7f0210197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.711+0000 7f020ed9d700 1 -- 172.21.15.12:0/2300313587 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02000095f0 con 0x7f021019bcf0 2024-05-09T00:04:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.711+0000 7f020dd9b700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210105160 0x7f02101979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.711+0000 7f020ed9d700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f021019bcf0 0x7f021019c170 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f020800ef30 tx=0x7f020800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.711+0000 7f01ff7fe700 1 -- 172.21.15.12:0/2300313587 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208018070 con 0x7f021019bcf0 2024-05-09T00:04:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.711+0000 7f0214bc4700 1 -- 172.21.15.12:0/2300313587 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02101a3fa0 con 0x7f021019bcf0 2024-05-09T00:04:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.712+0000 7f01ff7fe700 1 -- 172.21.15.12:0/2300313587 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f020800f040 con 0x7f021019bcf0 2024-05-09T00:04:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.712+0000 7f01ff7fe700 1 -- 172.21.15.12:0/2300313587 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020800cd00 con 0x7f021019bcf0 2024-05-09T00:04:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.712+0000 7f0214bc4700 1 -- 172.21.15.12:0/2300313587 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02101a4480 con 0x7f021019bcf0 2024-05-09T00:04:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.714+0000 7f01ff7fe700 1 -- 172.21.15.12:0/2300313587 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0208004000 con 0x7f021019bcf0 2024-05-09T00:04:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.715+0000 7f01ff7fe700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01f80a1290 0x7f01f80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.715+0000 7f020e59c700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01f80a1290 0x7f01f80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.715+0000 7f01ff7fe700 1 -- 172.21.15.12:0/2300313587 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f02080cc810 con 0x7f021019bcf0 2024-05-09T00:04:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.715+0000 7f020e59c700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01f80a1290 0x7f01f80a3750 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f020000b2e0 tx=0x7f0200005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.715+0000 7f0220d7eb80 1 Processor -- start 2024-05-09T00:04:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f0220d7eb80 1 -- start start 2024-05-09T00:04:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb862e00 0x55c9cb8631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb85d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb7b4150 con 0x55c9cb85d890 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb7b1b10 con 0x55c9cb863720 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb7ad400 con 0x55c9cb862e00 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f020ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f020e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb862e00 0x55c9cb8631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f020ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57696/0 (socket says 172.21.15.12:57696) 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f020dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb85d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.716+0000 7f020ed9d700 1 -- 172.21.15.12:0/4016787220 learned_addr learned my addr 172.21.15.12:0/4016787220 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.717+0000 7f020ed9d700 1 -- 172.21.15.12:0/4016787220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb862e00 msgr2=0x55c9cb8631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.717+0000 7f020ed9d700 1 --2- 172.21.15.12:0/4016787220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb862e00 0x55c9cb8631e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.717+0000 7f020ed9d700 1 -- 172.21.15.12:0/4016787220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 msgr2=0x55c9cb85d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.717+0000 7f020ed9d700 1 --2- 172.21.15.12:0/4016787220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb85d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.717+0000 7f020ed9d700 1 -- 172.21.15.12:0/4016787220 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9cb62e3b0 con 0x55c9cb85d890 2024-05-09T00:04:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.717+0000 7f020ed9d700 1 --2- 172.21.15.12:0/4016787220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85dcd0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f02080963d0 tx=0x7f02080994a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a009a3ef35220c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.717+0000 7f01fcff9700 1 -- 172.21.15.12:0/4016787220 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020800ce80 con 0x55c9cb85d890 2024-05-09T00:04:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.717+0000 7f01fcff9700 1 -- 172.21.15.12:0/4016787220 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f020809a060 con 0x55c9cb85d890 2024-05-09T00:04:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.717+0000 7f01fcff9700 1 -- 172.21.15.12:0/4016787220 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208097e20 con 0x55c9cb85d890 2024-05-09T00:04:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.718+0000 7f0220d7eb80 1 -- 172.21.15.12:0/4016787220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 msgr2=0x55c9cb85dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.718+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/4016787220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85dcd0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f02080963d0 tx=0x7f02080994a0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.718+0000 7f0220d7eb80 1 -- 172.21.15.12:0/4016787220 shutdown_connections 2024-05-09T00:04:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.718+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/4016787220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85dcd0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.718+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/4016787220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb862e00 0x55c9cb8631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.718+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/4016787220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb85d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.718+0000 7f0220d7eb80 1 -- 172.21.15.12:0/4016787220 >> 172.21.15.12:0/4016787220 conn(0x55c9cb7bb210 msgr2=0x55c9cb857df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.718+0000 7f0220d7eb80 1 -- 172.21.15.12:0/4016787220 shutdown_connections 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.718+0000 7f0220d7eb80 1 -- 172.21.15.12:0/4016787220 wait complete. 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f0220d7eb80 1 -- start start 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f0220d7eb80 1 Processor -- start 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb85d890 0x55c9cb866ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 0x55c9cb866ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb863720 0x55c9cb867530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb749560 con 0x55c9cb862e00 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb749200 con 0x55c9cb85d890 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb6306a0 con 0x55c9cb863720 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f0220d7eb80 10 client.0 _ll_register_callbacks cb 0x55c9cb752b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f020e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb85d890 0x55c9cb866ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f020ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb863720 0x55c9cb867530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f020e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb85d890 0x55c9cb866ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58228/0 (socket says 172.21.15.12:58228) 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f020dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 0x55c9cb866ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.719+0000 7f020e59c700 1 -- 172.21.15.12:0/2674332835 learned_addr learned my addr 172.21.15.12:0/2674332835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f020e59c700 1 -- 172.21.15.12:0/2674332835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb863720 msgr2=0x55c9cb867530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f020e59c700 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb863720 0x55c9cb867530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f020e59c700 1 -- 172.21.15.12:0/2674332835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 msgr2=0x55c9cb866ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f020e59c700 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 0x55c9cb866ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f020e59c700 1 -- 172.21.15.12:0/2674332835 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02101a4480 con 0x55c9cb85d890 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f020e59c700 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb85d890 0x55c9cb866ab0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0200015000 tx=0x7f0200009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c765ec08ccecea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f01f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2674332835 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200004c70 con 0x55c9cb85d890 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2674332835 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0200004dd0 con 0x55c9cb85d890 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2674332835 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9cb706270 con 0x55c9cb85d890 2024-05-09T00:04:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2674332835 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f020000f460 con 0x55c9cb85d890 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f0220d7eb80 10 client.30083 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.720+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2674332835 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55c9cb5dab60 con 0x55c9cb85d890 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.721+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2674332835 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9cb706270 con 0x55c9cb85d890 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.721+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2674332835 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f020002f070 con 0x55c9cb85d890 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.721+0000 7f0220d7eb80 10 client.30083 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.721+0000 7f0220d7eb80 10 client.30083 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.721+0000 7f0220d7eb80 20 client.30083 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.721+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c9cb877190 0x55c9cb8795b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.721+0000 7f0220d7eb80 4 client.30083 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.721+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2674332835 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9cb82c810 con 0x55c9cb877190 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.721+0000 7f020dd9b700 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c9cb877190 0x55c9cb8795b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.722+0000 7f020dd9b700 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c9cb877190 0x55c9cb8795b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=298176173eda9585 server_cookie=e2cca047452e644c in_seq=0 out_seq=0 2024-05-09T00:04:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.722+0000 7f01f5ffb700 10 client.30083 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.722+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2674332835 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9cb82c810 con 0x55c9cb877190 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.722+0000 7f01f5ffb700 10 client.30083 handle_command_reply: tid=0 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.722+0000 7f0220d7eb80 1 client.30083 shutdown 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2674332835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb85d890 msgr2=0x55c9cb866ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb85d890 0x55c9cb866ab0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0200015000 tx=0x7f0200009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2674332835 shutdown_connections 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c9cb877190 0x55c9cb8795b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb863720 0x55c9cb867530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb85d890 0x55c9cb866ab0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f01f5ffb700 0 client.30083 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2674332835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 0x55c9cb866ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2674332835 >> 172.21.15.12:0/2674332835 conn(0x55c9cb7bb210 msgr2=0x55c9cb8575a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2674332835 shutdown_connections 2024-05-09T00:04:39.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2674332835 wait complete. 2024-05-09T00:04:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.723+0000 7f0220d7eb80 20 client.30083 trim_cache size 0 max 16384 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.739+0000 7f0220d7eb80 1 Processor -- start 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.739+0000 7f0220d7eb80 1 -- start start 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb862e00 0x55c9cb862430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb857d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb867f80 con 0x55c9cb85d890 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb854dc0 con 0x55c9cb863720 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb859e80 con 0x55c9cb862e00 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57706/0 (socket says 172.21.15.12:57706) 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020e59c700 1 -- 172.21.15.12:0/4156929180 learned_addr learned my addr 172.21.15.12:0/4156929180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020dd9b700 1 --2- 172.21.15.12:0/4156929180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb862e00 0x55c9cb862430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020ed9d700 1 --2- 172.21.15.12:0/4156929180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb857d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020e59c700 1 -- 172.21.15.12:0/4156929180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb862e00 msgr2=0x55c9cb862430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020e59c700 1 --2- 172.21.15.12:0/4156929180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb862e00 0x55c9cb862430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020e59c700 1 -- 172.21.15.12:0/4156929180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 msgr2=0x55c9cb857d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020e59c700 1 --2- 172.21.15.12:0/4156929180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb857d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020e59c700 1 -- 172.21.15.12:0/4156929180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9cb773560 con 0x55c9cb85d890 2024-05-09T00:04:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.740+0000 7f020e59c700 1 --2- 172.21.15.12:0/4156929180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85d350 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f0200009990 tx=0x7f0200024a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e4e8574511abd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.741+0000 7f01fd7fa700 1 -- 172.21.15.12:0/4156929180 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020000b310 con 0x55c9cb85d890 2024-05-09T00:04:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.741+0000 7f01fd7fa700 1 -- 172.21.15.12:0/4156929180 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0200028070 con 0x55c9cb85d890 2024-05-09T00:04:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.741+0000 7f01fd7fa700 1 -- 172.21.15.12:0/4156929180 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020000fad0 con 0x55c9cb85d890 2024-05-09T00:04:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.741+0000 7f0220d7eb80 1 -- 172.21.15.12:0/4156929180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 msgr2=0x55c9cb85d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.741+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/4156929180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85d350 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f0200009990 tx=0x7f0200024a50 comp rx=0 tx=0).stop 2024-05-09T00:04:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 -- 172.21.15.12:0/4156929180 shutdown_connections 2024-05-09T00:04:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/4156929180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb857d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/4156929180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb85d890 0x55c9cb85d350 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/4156929180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb862e00 0x55c9cb862430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 -- 172.21.15.12:0/4156929180 >> 172.21.15.12:0/4156929180 conn(0x55c9cb7bb210 msgr2=0x55c9cb8653a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 -- 172.21.15.12:0/4156929180 shutdown_connections 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 -- 172.21.15.12:0/4156929180 wait complete. 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 -- start start 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 Processor -- start 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb85d890 0x55c9cb86e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 0x55c9cb86a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb86a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb86ae70 con 0x55c9cb862e00 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb8cc680 con 0x55c9cb863720 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb8cc7c0 con 0x55c9cb85d890 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.742+0000 7f0220d7eb80 10 client.0 _ll_register_callbacks cb 0x55c9cb752b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 0x55c9cb86a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 0x55c9cb86a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57712/0 (socket says 172.21.15.12:57712) 2024-05-09T00:04:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb85d890 0x55c9cb86e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020dd9b700 1 -- 172.21.15.12:0/2328251799 learned_addr learned my addr 172.21.15.12:0/2328251799 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020ed9d700 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb86a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020dd9b700 1 -- 172.21.15.12:0/2328251799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb85d890 msgr2=0x55c9cb86e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020dd9b700 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb85d890 0x55c9cb86e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020dd9b700 1 -- 172.21.15.12:0/2328251799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 msgr2=0x55c9cb86a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020dd9b700 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb86a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020dd9b700 1 -- 172.21.15.12:0/2328251799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9cb5dab60 con 0x55c9cb862e00 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020e59c700 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb85d890 0x55c9cb86e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f020dd9b700 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 0x55c9cb86a020 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f021006a6d0 tx=0x7f0204012a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a26b8d5c045a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f01f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2328251799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204017030 con 0x55c9cb862e00 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2328251799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0204020520 con 0x55c9cb862e00 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.743+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2328251799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f020809a060 con 0x55c9cb862e00 2024-05-09T00:04:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2328251799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02040206e0 con 0x55c9cb862e00 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f0220d7eb80 10 client.20496 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2328251799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55c9cb632cd0 con 0x55c9cb862e00 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2328251799 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f020809a060 con 0x55c9cb862e00 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2328251799 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0204027070 con 0x55c9cb862e00 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f0220d7eb80 10 client.20496 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f0220d7eb80 10 client.20496 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f0220d7eb80 20 client.20496 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c9cb8cdff0 0x55c9cb8d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f0220d7eb80 4 client.20496 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2328251799 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c9cb7a5ec0 con 0x55c9cb8cdff0 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.744+0000 7f020e59c700 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c9cb8cdff0 0x55c9cb8d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.745+0000 7f020e59c700 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c9cb8cdff0 0x55c9cb8d0410 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=438648d2bc161035 server_cookie=2d35b1e28b8d04a5 in_seq=0 out_seq=0 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.745+0000 7f01f5ffb700 10 client.20496 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:04:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f01f5ffb700 1 -- 172.21.15.12:0/2328251799 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c9cb7a5ec0 con 0x55c9cb8cdff0 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f01f5ffb700 10 client.20496 handle_command_reply: tid=0 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 client.20496 shutdown 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2328251799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 msgr2=0x55c9cb86a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 0x55c9cb86a020 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f021006a6d0 tx=0x7f0204012a70 comp rx=0 tx=0).stop 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2328251799 shutdown_connections 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c9cb863720 0x55c9cb86a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c9cb8cdff0 0x55c9cb8d0410 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9cb85d890 0x55c9cb86e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 --2- 172.21.15.12:0/2328251799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9cb862e00 0x55c9cb86a020 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f01f5ffb700 0 client.20496 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2328251799 >> 172.21.15.12:0/2328251799 conn(0x55c9cb7bb210 msgr2=0x55c9cb864440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2328251799 shutdown_connections 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 1 -- 172.21.15.12:0/2328251799 wait complete. 2024-05-09T00:04:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.746+0000 7f0220d7eb80 20 client.20496 trim_cache size 0 max 16384 2024-05-09T00:04:39.745 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:04:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 -- 172.21.15.12:0/2300313587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01f80a1290 msgr2=0x7f01f80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01f80a1290 0x7f01f80a3750 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f020000b2e0 tx=0x7f0200005fd0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 -- 172.21.15.12:0/2300313587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f021019bcf0 msgr2=0x7f021019c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f021019bcf0 0x7f021019c170 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f020800ef30 tx=0x7f020800c3a0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 -- 172.21.15.12:0/2300313587 shutdown_connections 2024-05-09T00:04:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f01f80a1290 0x7f01f80a3750 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f021019bcf0 0x7f021019c170 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0210100060 0x7f0210197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 --2- 172.21.15.12:0/2300313587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210105160 0x7f02101979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 -- 172.21.15.12:0/2300313587 >> 172.21.15.12:0/2300313587 conn(0x7f0210076f40 msgr2=0x7f0210103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 -- 172.21.15.12:0/2300313587 shutdown_connections 2024-05-09T00:04:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.747+0000 7f0214bc4700 1 -- 172.21.15.12:0/2300313587 wait complete. 2024-05-09T00:04:39.755 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:04:39.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:39 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2071664107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:04:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.922+0000 7f0529c20700 1 -- 172.21.15.12:0/2361982431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05241000f0 msgr2=0x7f0524100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.922+0000 7f0529c20700 1 --2- 172.21.15.12:0/2361982431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05241000f0 0x7f0524100570 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f05180099c0 tx=0x7f0518009cd0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.922+0000 7f0529c20700 1 -- 172.21.15.12:0/2361982431 shutdown_connections 2024-05-09T00:04:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.922+0000 7f0529c20700 1 --2- 172.21.15.12:0/2361982431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0524100ab0 0x7f052410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.922+0000 7f0529c20700 1 --2- 172.21.15.12:0/2361982431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0524105f90 0x7f0524106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.922+0000 7f0529c20700 1 --2- 172.21.15.12:0/2361982431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05241000f0 0x7f0524100570 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.922+0000 7f0529c20700 1 -- 172.21.15.12:0/2361982431 >> 172.21.15.12:0/2361982431 conn(0x7f0524076ba0 msgr2=0x7f0524076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.922+0000 7f0529c20700 1 -- 172.21.15.12:0/2361982431 shutdown_connections 2024-05-09T00:04:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.922+0000 7f0529c20700 1 -- 172.21.15.12:0/2361982431 wait complete. 2024-05-09T00:04:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.923+0000 7f0529c20700 1 Processor -- start 2024-05-09T00:04:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.923+0000 7f0529c20700 1 -- start start 2024-05-09T00:04:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.923+0000 7f0529c20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05241000f0 0x7f0524198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.923+0000 7f0529c20700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0524100ab0 0x7f0524198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.923+0000 7f0529c20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0524105f90 0x7f05241a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.924+0000 7f0529c20700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0524198f90 con 0x7f0524105f90 2024-05-09T00:04:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.924+0000 7f0529c20700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0524195ff0 con 0x7f05241000f0 2024-05-09T00:04:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.924+0000 7f0529c20700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05241961b0 con 0x7f0524100ab0 2024-05-09T00:04:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.926+0000 7f0522ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0524100ab0 0x7f0524198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.926+0000 7f0522ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0524100ab0 0x7f0524198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39330/0 (socket says 172.21.15.12:39330) 2024-05-09T00:04:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.926+0000 7f0522ffd700 1 -- 172.21.15.12:0/1085607935 learned_addr learned my addr 172.21.15.12:0/1085607935 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.926+0000 7f05237fe700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05241000f0 0x7f0524198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.926+0000 7f0523fff700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0524105f90 0x7f05241a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.926+0000 7f0522ffd700 1 -- 172.21.15.12:0/1085607935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05241000f0 msgr2=0x7f0524198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.926+0000 7f0522ffd700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05241000f0 0x7f0524198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.926+0000 7f0522ffd700 1 -- 172.21.15.12:0/1085607935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0524105f90 msgr2=0x7f05241a3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.926+0000 7f0522ffd700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0524105f90 0x7f05241a3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.926+0000 7f0522ffd700 1 -- 172.21.15.12:0/1085607935 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05180095f0 con 0x7f0524100ab0 2024-05-09T00:04:39.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.927+0000 7f0522ffd700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0524100ab0 0x7f0524198740 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f051800b580 tx=0x7f051800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.930+0000 7f0520ff9700 1 -- 172.21.15.12:0/1085607935 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051801c070 con 0x7f0524100ab0 2024-05-09T00:04:39.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.930+0000 7f0529c20700 1 -- 172.21.15.12:0/1085607935 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0524196430 con 0x7f0524100ab0 2024-05-09T00:04:39.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.930+0000 7f0520ff9700 1 -- 172.21.15.12:0/1085607935 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0518022420 con 0x7f0524100ab0 2024-05-09T00:04:39.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.930+0000 7f0520ff9700 1 -- 172.21.15.12:0/1085607935 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051800fea0 con 0x7f0524100ab0 2024-05-09T00:04:39.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.931+0000 7f0529c20700 1 -- 172.21.15.12:0/1085607935 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0524196930 con 0x7f0524100ab0 2024-05-09T00:04:39.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.936+0000 7f0520ff9700 1 -- 172.21.15.12:0/1085607935 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0518020070 con 0x7f0524100ab0 2024-05-09T00:04:39.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.936+0000 7f0535ddab80 1 Processor -- start 2024-05-09T00:04:39.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.936+0000 7f0535ddab80 1 -- start start 2024-05-09T00:04:39.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.937+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f857f10 0x555c0f8582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.937+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f852360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.937+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 0x555c0f852d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.937+0000 7f0535ddab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f798150 con 0x555c0f858830 2024-05-09T00:04:39.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.937+0000 7f0535ddab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f795b10 con 0x555c0f857f10 2024-05-09T00:04:39.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.937+0000 7f0535ddab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f791400 con 0x555c0f8528a0 2024-05-09T00:04:39.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.937+0000 7f0520ff9700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f050c0a1310 0x7f050c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.937+0000 7f0520ff9700 1 -- 172.21.15.12:0/1085607935 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f05180cdb70 con 0x7f0524100ab0 2024-05-09T00:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.938+0000 7f05237fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f852360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.938+0000 7f05237fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f852360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57732/0 (socket says 172.21.15.12:57732) 2024-05-09T00:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.938+0000 7f05237fe700 1 -- 172.21.15.12:0/3357029716 learned_addr learned my addr 172.21.15.12:0/3357029716 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.938+0000 7f05237fe700 1 -- 172.21.15.12:0/3357029716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 msgr2=0x555c0f852d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.938+0000 7f05237fe700 1 --2- 172.21.15.12:0/3357029716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 0x555c0f852d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.938+0000 7f05237fe700 1 -- 172.21.15.12:0/3357029716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f857f10 msgr2=0x555c0f8582f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.938+0000 7f05237fe700 1 --2- 172.21.15.12:0/3357029716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f857f10 0x555c0f8582f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.938+0000 7f05237fe700 1 -- 172.21.15.12:0/3357029716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c0f6123b0 con 0x555c0f858830 2024-05-09T00:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.939+0000 7f05237fe700 1 --2- 172.21.15.12:0/3357029716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f852360 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f0514009cb0 tx=0x7f05140113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4761a1c198001640 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.939+0000 7f0509ffb700 1 -- 172.21.15.12:0/3357029716 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514011d10 con 0x555c0f858830 2024-05-09T00:04:39.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.939+0000 7f0509ffb700 1 -- 172.21.15.12:0/3357029716 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0514011e70 con 0x555c0f858830 2024-05-09T00:04:39.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.939+0000 7f0509ffb700 1 -- 172.21.15.12:0/3357029716 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051400f7c0 con 0x555c0f858830 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0523fff700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f050c0a1310 0x7f050c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0535ddab80 1 -- 172.21.15.12:0/3357029716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 msgr2=0x555c0f852360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0535ddab80 1 --2- 172.21.15.12:0/3357029716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f852360 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f0514009cb0 tx=0x7f05140113e0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0535ddab80 1 -- 172.21.15.12:0/3357029716 shutdown_connections 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0535ddab80 1 --2- 172.21.15.12:0/3357029716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 0x555c0f852d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0535ddab80 1 --2- 172.21.15.12:0/3357029716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f857f10 0x555c0f8582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0535ddab80 1 --2- 172.21.15.12:0/3357029716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f852360 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0535ddab80 1 -- 172.21.15.12:0/3357029716 >> 172.21.15.12:0/3357029716 conn(0x555c0f79f210 msgr2=0x555c0f83bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0523fff700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f050c0a1310 0x7f050c0a37d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f051c009300 tx=0x7f051c00b040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0535ddab80 1 -- 172.21.15.12:0/3357029716 shutdown_connections 2024-05-09T00:04:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.940+0000 7f0535ddab80 1 -- 172.21.15.12:0/3357029716 wait complete. 2024-05-09T00:04:39.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.941+0000 7f0535ddab80 1 -- start start 2024-05-09T00:04:39.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.941+0000 7f0535ddab80 1 Processor -- start 2024-05-09T00:04:39.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f8528a0 0x555c0f85e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f857f10 0x555c0f85ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f85c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0535ddab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f72d560 con 0x555c0f858830 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0535ddab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f72d200 con 0x555c0f8528a0 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0535ddab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f6146a0 con 0x555c0f857f10 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0535ddab80 10 client.0 _ll_register_callbacks cb 0x555c0f736b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0522ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f857f10 0x555c0f85ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0522ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f857f10 0x555c0f85ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39344/0 (socket says 172.21.15.12:39344) 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0522ffd700 1 -- 172.21.15.12:0/1857191855 learned_addr learned my addr 172.21.15.12:0/1857191855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0523fff700 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f85c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0522ffd700 1 -- 172.21.15.12:0/1857191855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f8528a0 msgr2=0x555c0f85e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0522ffd700 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f8528a0 0x555c0f85e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0522ffd700 1 -- 172.21.15.12:0/1857191855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 msgr2=0x555c0f85c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0522ffd700 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f85c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.942+0000 7f0522ffd700 1 -- 172.21.15.12:0/1857191855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0514009910 con 0x555c0f857f10 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.943+0000 7f0522ffd700 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f857f10 0x555c0f85ba50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f051809add0 tx=0x7f051809aa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25700921270f303d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.943+0000 7f0502ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.943+0000 7f0502ffd700 1 -- 172.21.15.12:0/1857191855 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051809ae40 con 0x555c0f857f10 2024-05-09T00:04:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.943+0000 7f0502ffd700 1 -- 172.21.15.12:0/1857191855 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f051809ec70 con 0x555c0f857f10 2024-05-09T00:04:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.943+0000 7f0535ddab80 1 -- 172.21.15.12:0/1857191855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c0f6ea270 con 0x555c0f857f10 2024-05-09T00:04:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.943+0000 7f0502ffd700 1 -- 172.21.15.12:0/1857191855 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05180a3230 con 0x555c0f857f10 2024-05-09T00:04:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.943+0000 7f0535ddab80 10 client.30235 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.943+0000 7f0535ddab80 1 -- 172.21.15.12:0/1857191855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x555c0f5beb60 con 0x555c0f857f10 2024-05-09T00:04:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.944+0000 7f0502ffd700 1 -- 172.21.15.12:0/1857191855 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c0f6ea270 con 0x555c0f857f10 2024-05-09T00:04:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.945+0000 7f0502ffd700 1 -- 172.21.15.12:0/1857191855 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f05180a2da0 con 0x555c0f857f10 2024-05-09T00:04:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.945+0000 7f0535ddab80 10 client.30235 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.945+0000 7f0535ddab80 10 client.30235 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:04:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.945+0000 7f0535ddab80 20 client.30235 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.945+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555c0f861770 0x555c0f863b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.945+0000 7f0535ddab80 4 client.30235 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.945+0000 7f0535ddab80 1 -- 172.21.15.12:0/1857191855 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c0f810810 con 0x555c0f861770 2024-05-09T00:04:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.945+0000 7f05237fe700 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555c0f861770 0x555c0f863b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.945+0000 7f05237fe700 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555c0f861770 0x555c0f863b90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=321a0039c9a5060 server_cookie=d79e3417486682f4 in_seq=0 out_seq=0 2024-05-09T00:04:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.945+0000 7f0502ffd700 10 client.30235 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:04:39.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.953+0000 7f0502ffd700 1 -- 172.21.15.12:0/1857191855 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c0f810810 con 0x555c0f861770 2024-05-09T00:04:39.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.953+0000 7f0502ffd700 10 client.30235 handle_command_reply: tid=0 2024-05-09T00:04:39.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.953+0000 7f0535ddab80 1 client.30235 shutdown 2024-05-09T00:04:39.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.954+0000 7f0535ddab80 1 -- 172.21.15.12:0/1857191855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f857f10 msgr2=0x555c0f85ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.954+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f857f10 0x555c0f85ba50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f051809add0 tx=0x7f051809aa70 comp rx=0 tx=0).stop 2024-05-09T00:04:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.957+0000 7f0535ddab80 1 -- 172.21.15.12:0/1857191855 shutdown_connections 2024-05-09T00:04:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.957+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555c0f861770 0x555c0f863b90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.957+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f85c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.957+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f8528a0 0x555c0f85e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.957+0000 7f0502ffd700 0 client.30235 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:04:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.957+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1857191855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f857f10 0x555c0f85ba50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.957+0000 7f0535ddab80 1 -- 172.21.15.12:0/1857191855 >> 172.21.15.12:0/1857191855 conn(0x555c0f79f210 msgr2=0x555c0f83b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.957+0000 7f0535ddab80 1 -- 172.21.15.12:0/1857191855 shutdown_connections 2024-05-09T00:04:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.957+0000 7f0535ddab80 1 -- 172.21.15.12:0/1857191855 wait complete. 2024-05-09T00:04:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.957+0000 7f0535ddab80 20 client.30235 trim_cache size 0 max 16384 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0535ddab80 1 Processor -- start 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0535ddab80 1 -- start start 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 0x555c0f83bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f857f10 0x555c0f8719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f871f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0535ddab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f859fe0 con 0x555c0f858830 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0535ddab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f852660 con 0x555c0f857f10 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0535ddab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f834040 con 0x555c0f8528a0 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0523fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f871f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0523fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f871f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57756/0 (socket says 172.21.15.12:57756) 2024-05-09T00:04:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0523fff700 1 -- 172.21.15.12:0/1473236442 learned_addr learned my addr 172.21.15.12:0/1473236442 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.980+0000 7f0522ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f857f10 0x555c0f8719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.981+0000 7f0523fff700 1 -- 172.21.15.12:0/1473236442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 msgr2=0x555c0f83bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.981+0000 7f0523fff700 1 --2- 172.21.15.12:0/1473236442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 0x555c0f83bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.981+0000 7f0523fff700 1 -- 172.21.15.12:0/1473236442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f857f10 msgr2=0x555c0f8719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.981+0000 7f0523fff700 1 --2- 172.21.15.12:0/1473236442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f857f10 0x555c0f8719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.981+0000 7f0523fff700 1 -- 172.21.15.12:0/1473236442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c0f757560 con 0x555c0f858830 2024-05-09T00:04:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.981+0000 7f0523fff700 1 --2- 172.21.15.12:0/1473236442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f871f30 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f051c0155d0 tx=0x7f051c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0777c4227227347 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.981+0000 7f050a7fc700 1 -- 172.21.15.12:0/1473236442 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00e050 con 0x555c0f858830 2024-05-09T00:04:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.981+0000 7f050a7fc700 1 -- 172.21.15.12:0/1473236442 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f051c01a070 con 0x555c0f858830 2024-05-09T00:04:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.981+0000 7f050a7fc700 1 -- 172.21.15.12:0/1473236442 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c0169c0 con 0x555c0f858830 2024-05-09T00:04:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.982+0000 7f0535ddab80 1 -- 172.21.15.12:0/1473236442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 msgr2=0x555c0f871f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.982+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1473236442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f871f30 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f051c0155d0 tx=0x7f051c013000 comp rx=0 tx=0).stop 2024-05-09T00:04:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.982+0000 7f0535ddab80 1 -- 172.21.15.12:0/1473236442 shutdown_connections 2024-05-09T00:04:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.983+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1473236442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f858830 0x555c0f871f30 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.983+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1473236442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 0x555c0f83bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.983+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1473236442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f857f10 0x555c0f8719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.983+0000 7f0535ddab80 1 -- 172.21.15.12:0/1473236442 >> 172.21.15.12:0/1473236442 conn(0x555c0f79f210 msgr2=0x555c0f855c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.983+0000 7f0535ddab80 1 -- 172.21.15.12:0/1473236442 shutdown_connections 2024-05-09T00:04:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.983+0000 7f0535ddab80 1 -- 172.21.15.12:0/1473236442 wait complete. 2024-05-09T00:04:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.983+0000 7f0535ddab80 1 -- start start 2024-05-09T00:04:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.983+0000 7f0535ddab80 1 Processor -- start 2024-05-09T00:04:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 0x555c0f871ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f857f10 0x555c0f854c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0535ddab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f858830 0x555c0f855360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0535ddab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f853bb0 con 0x555c0f857f10 2024-05-09T00:04:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0535ddab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f853d20 con 0x555c0f858830 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0535ddab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0f853ee0 con 0x555c0f8528a0 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0535ddab80 10 client.0 _ll_register_callbacks cb 0x7f0518097250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0522ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f857f10 0x555c0f854c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0522ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f857f10 0x555c0f854c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57760/0 (socket says 172.21.15.12:57760) 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0522ffd700 1 -- 172.21.15.12:0/1918223566 learned_addr learned my addr 172.21.15.12:0/1918223566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0522ffd700 1 -- 172.21.15.12:0/1918223566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 msgr2=0x555c0f871ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0522ffd700 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 0x555c0f871ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0522ffd700 1 -- 172.21.15.12:0/1918223566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f858830 msgr2=0x555c0f855360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0522ffd700 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f858830 0x555c0f855360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.984+0000 7f0522ffd700 1 -- 172.21.15.12:0/1918223566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051c007e90 con 0x555c0f857f10 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.985+0000 7f0522ffd700 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f857f10 0x555c0f854c20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f05180cb8d0 tx=0x7f0518022690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e74eb97c25b533e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.985+0000 7f0502ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.985+0000 7f0502ffd700 1 -- 172.21.15.12:0/1918223566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051800fbe0 con 0x555c0f857f10 2024-05-09T00:04:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.985+0000 7f0502ffd700 1 -- 172.21.15.12:0/1918223566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f051809ec70 con 0x555c0f857f10 2024-05-09T00:04:39.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.985+0000 7f0535ddab80 1 -- 172.21.15.12:0/1918223566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0514011e70 con 0x555c0f857f10 2024-05-09T00:04:39.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.985+0000 7f0502ffd700 1 -- 172.21.15.12:0/1918223566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f051800fd40 con 0x555c0f857f10 2024-05-09T00:04:39.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.985+0000 7f0535ddab80 10 client.20514 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:39.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.985+0000 7f0535ddab80 1 -- 172.21.15.12:0/1918223566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x555c0f616cd0 con 0x555c0f857f10 2024-05-09T00:04:39.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.985+0000 7f0502ffd700 1 -- 172.21.15.12:0/1918223566 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0514011e70 con 0x555c0f857f10 2024-05-09T00:04:39.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.985+0000 7f0502ffd700 1 -- 172.21.15.12:0/1918223566 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0518099da0 con 0x555c0f857f10 2024-05-09T00:04:39.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.988+0000 7f0535ddab80 10 client.20514 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:39.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.988+0000 7f0535ddab80 10 client.20514 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:04:39.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.988+0000 7f0535ddab80 20 client.20514 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:39.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.988+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555c0f8e5190 0x555c0f8e7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:39.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.988+0000 7f0535ddab80 4 client.20514 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:04:39.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.988+0000 7f0535ddab80 1 -- 172.21.15.12:0/1918223566 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c0f789ec0 con 0x555c0f8e5190 2024-05-09T00:04:39.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.988+0000 7f05237fe700 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555c0f8e5190 0x555c0f8e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:39.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.989+0000 7f05237fe700 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555c0f8e5190 0x555c0f8e7580 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=643d09b1359c0f3a server_cookie=2368e096e73e969f in_seq=0 out_seq=0 2024-05-09T00:04:39.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.989+0000 7f0502ffd700 10 client.20514 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:04:39.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.992+0000 7f0502ffd700 1 -- 172.21.15.12:0/1918223566 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c0f789ec0 con 0x555c0f8e5190 2024-05-09T00:04:39.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.992+0000 7f0502ffd700 10 client.20514 handle_command_reply: tid=0 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.992+0000 7f0535ddab80 1 client.20514 shutdown 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.992+0000 7f0535ddab80 1 -- 172.21.15.12:0/1918223566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f857f10 msgr2=0x555c0f854c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.992+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f857f10 0x555c0f854c20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f05180cb8d0 tx=0x7f0518022690 comp rx=0 tx=0).stop 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.993+0000 7f0535ddab80 1 -- 172.21.15.12:0/1918223566 shutdown_connections 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.993+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x555c0f8e5190 0x555c0f8e7580 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.993+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0f858830 0x555c0f855360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.993+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0f8528a0 0x555c0f871ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.993+0000 7f0535ddab80 1 --2- 172.21.15.12:0/1918223566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c0f857f10 0x555c0f854c20 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.993+0000 7f0535ddab80 1 -- 172.21.15.12:0/1918223566 >> 172.21.15.12:0/1918223566 conn(0x555c0f79f210 msgr2=0x555c0f85a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.993+0000 7f0502ffd700 0 client.20514 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.993+0000 7f0535ddab80 1 -- 172.21.15.12:0/1918223566 shutdown_connections 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.993+0000 7f0535ddab80 1 -- 172.21.15.12:0/1918223566 wait complete. 2024-05-09T00:04:39.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.993+0000 7f0535ddab80 20 client.20514 trim_cache size 0 max 16384 2024-05-09T00:04:39.989 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:04:39.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 -- 172.21.15.12:0/1085607935 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f050c0a1310 msgr2=0x7f050c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f050c0a1310 0x7f050c0a37d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f051c009300 tx=0x7f051c00b040 comp rx=0 tx=0).stop 2024-05-09T00:04:39.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 -- 172.21.15.12:0/1085607935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0524100ab0 msgr2=0x7f0524198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:39.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0524100ab0 0x7f0524198740 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f051800b580 tx=0x7f051800bfd0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 -- 172.21.15.12:0/1085607935 shutdown_connections 2024-05-09T00:04:39.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f050c0a1310 0x7f050c0a37d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0524105f90 0x7f05241a3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05241000f0 0x7f0524198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 --2- 172.21.15.12:0/1085607935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0524100ab0 0x7f0524198740 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:39.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 -- 172.21.15.12:0/1085607935 >> 172.21.15.12:0/1085607935 conn(0x7f0524076ba0 msgr2=0x7f05240fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:39.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 -- 172.21.15.12:0/1085607935 shutdown_connections 2024-05-09T00:04:39.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:39.994+0000 7f0529c20700 1 -- 172.21.15.12:0/1085607935 wait complete. 2024-05-09T00:04:40.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:04:40.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:39 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2071664107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:04:40.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:39 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2071664107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:04:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.176+0000 7ff361ccb700 1 -- 172.21.15.12:0/3736909523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff35c1000f0 msgr2=0x7ff35c100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.176+0000 7ff361ccb700 1 --2- 172.21.15.12:0/3736909523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff35c1000f0 0x7ff35c100570 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff3500099c0 tx=0x7ff350009cd0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.176+0000 7ff361ccb700 1 -- 172.21.15.12:0/3736909523 shutdown_connections 2024-05-09T00:04:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.176+0000 7ff361ccb700 1 --2- 172.21.15.12:0/3736909523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff35c100ab0 0x7ff35c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.176+0000 7ff361ccb700 1 --2- 172.21.15.12:0/3736909523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff35c105f90 0x7ff35c106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.176+0000 7ff361ccb700 1 --2- 172.21.15.12:0/3736909523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff35c1000f0 0x7ff35c100570 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.176+0000 7ff361ccb700 1 -- 172.21.15.12:0/3736909523 >> 172.21.15.12:0/3736909523 conn(0x7ff35c076ba0 msgr2=0x7ff35c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.176+0000 7ff361ccb700 1 -- 172.21.15.12:0/3736909523 shutdown_connections 2024-05-09T00:04:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.176+0000 7ff361ccb700 1 -- 172.21.15.12:0/3736909523 wait complete. 2024-05-09T00:04:40.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.177+0000 7ff361ccb700 1 Processor -- start 2024-05-09T00:04:40.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.177+0000 7ff361ccb700 1 -- start start 2024-05-09T00:04:40.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.177+0000 7ff361ccb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff35c1000f0 0x7ff35c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.177+0000 7ff361ccb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff35c100ab0 0x7ff35c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.177+0000 7ff361ccb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff35c105f90 0x7ff35c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.177+0000 7ff361ccb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c19c1e0 con 0x7ff35c100ab0 2024-05-09T00:04:40.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.177+0000 7ff361ccb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c19c350 con 0x7ff35c105f90 2024-05-09T00:04:40.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.177+0000 7ff361ccb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c19c510 con 0x7ff35c1000f0 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff35c1000f0 0x7ff35c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff35c1000f0 0x7ff35c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39380/0 (socket says 172.21.15.12:39380) 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35b7fe700 1 -- 172.21.15.12:0/1350535905 learned_addr learned my addr 172.21.15.12:0/1350535905 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35affd700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff35c100ab0 0x7ff35c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35bfff700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff35c105f90 0x7ff35c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35b7fe700 1 -- 172.21.15.12:0/1350535905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff35c105f90 msgr2=0x7ff35c19bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff35c105f90 0x7ff35c19bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35b7fe700 1 -- 172.21.15.12:0/1350535905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff35c100ab0 msgr2=0x7ff35c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff35c100ab0 0x7ff35c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35b7fe700 1 -- 172.21.15.12:0/1350535905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3500095f0 con 0x7ff35c1000f0 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff35c1000f0 0x7ff35c1972c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00e970 tx=0x7ff34c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.178+0000 7ff358ff9700 1 -- 172.21.15.12:0/1350535905 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c009790 con 0x7ff35c1000f0 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.179+0000 7ff361ccb700 1 -- 172.21.15.12:0/1350535905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35c19c7f0 con 0x7ff35c1000f0 2024-05-09T00:04:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.179+0000 7ff358ff9700 1 -- 172.21.15.12:0/1350535905 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff34c00eec0 con 0x7ff35c1000f0 2024-05-09T00:04:40.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.179+0000 7ff358ff9700 1 -- 172.21.15.12:0/1350535905 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c003cf0 con 0x7ff35c1000f0 2024-05-09T00:04:40.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.179+0000 7ff361ccb700 1 -- 172.21.15.12:0/1350535905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c1a4030 con 0x7ff35c1000f0 2024-05-09T00:04:40.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.181+0000 7ff358ff9700 1 -- 172.21.15.12:0/1350535905 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff34c00ccd0 con 0x7ff35c1000f0 2024-05-09T00:04:40.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.182+0000 7ff358ff9700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3440a1290 0x7ff3440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.182+0000 7ff358ff9700 1 -- 172.21.15.12:0/1350535905 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff34c014070 con 0x7ff35c1000f0 2024-05-09T00:04:40.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.182+0000 7ff35affd700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3440a1290 0x7ff3440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.183+0000 7ff35affd700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3440a1290 0x7ff3440a3750 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff350009fd0 tx=0x7ff350019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:40.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.184+0000 7ff36de85b80 1 Processor -- start 2024-05-09T00:04:40.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.184+0000 7ff36de85b80 1 -- start start 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.184+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c7411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 0x563e7c73b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 0x563e7c73bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff36de85b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c692150 con 0x563e7c73b890 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff36de85b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c68fb10 con 0x563e7c740e00 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff36de85b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c68b400 con 0x563e7c741720 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c7411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c7411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58298/0 (socket says 172.21.15.12:58298) 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2200548403 learned_addr learned my addr 172.21.15.12:0/2200548403 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35bfff700 1 --2- 172.21.15.12:0/2200548403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 0x563e7c73bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35affd700 1 --2- 172.21.15.12:0/2200548403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 0x563e7c73b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2200548403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 msgr2=0x563e7c73b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:40.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/2200548403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 0x563e7c73b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2200548403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 msgr2=0x563e7c73bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/2200548403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 0x563e7c73bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2200548403 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e7c50c3b0 con 0x563e7c740e00 2024-05-09T00:04:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.185+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/2200548403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c7411e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff34c096570 tx=0x7ff34c099f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b14fe831251057c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.186+0000 7ff341ffb700 1 -- 172.21.15.12:0/2200548403 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c09a070 con 0x563e7c740e00 2024-05-09T00:04:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.186+0000 7ff341ffb700 1 -- 172.21.15.12:0/2200548403 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff34c09dea0 con 0x563e7c740e00 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.186+0000 7ff341ffb700 1 -- 172.21.15.12:0/2200548403 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c09a2d0 con 0x563e7c740e00 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.186+0000 7ff36de85b80 1 -- 172.21.15.12:0/2200548403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 msgr2=0x563e7c7411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.187+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2200548403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c7411e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff34c096570 tx=0x7ff34c099f70 comp rx=0 tx=0).stop 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.187+0000 7ff36de85b80 1 -- 172.21.15.12:0/2200548403 shutdown_connections 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.187+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2200548403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 0x563e7c73bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.187+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2200548403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c7411e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.187+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2200548403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 0x563e7c73b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.187+0000 7ff36de85b80 1 -- 172.21.15.12:0/2200548403 >> 172.21.15.12:0/2200548403 conn(0x563e7c699210 msgr2=0x563e7c735df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.187+0000 7ff36de85b80 1 -- 172.21.15.12:0/2200548403 shutdown_connections 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.187+0000 7ff36de85b80 1 -- 172.21.15.12:0/2200548403 wait complete. 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.187+0000 7ff36de85b80 1 -- start start 2024-05-09T00:04:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.187+0000 7ff36de85b80 1 Processor -- start 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 0x563e7c744ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c745010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 0x563e7c745550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff36de85b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c627560 con 0x563e7c73b890 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff36de85b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c627200 con 0x563e7c740e00 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff36de85b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c50e6a0 con 0x563e7c741720 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff36de85b80 10 client.0 _ll_register_callbacks cb 0x563e7c630b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff35affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c745010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff35b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 0x563e7c744ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff35affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c745010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58308/0 (socket says 172.21.15.12:58308) 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.188+0000 7ff35affd700 1 -- 172.21.15.12:0/3742897102 learned_addr learned my addr 172.21.15.12:0/3742897102 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff35b7fe700 1 -- 172.21.15.12:0/3742897102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 msgr2=0x563e7c745550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 0x563e7c745550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff35b7fe700 1 -- 172.21.15.12:0/3742897102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 msgr2=0x563e7c745010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:40.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c745010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff35b7fe700 1 -- 172.21.15.12:0/3742897102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c1a4030 con 0x563e7c73b890 2024-05-09T00:04:40.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 0x563e7c744ad0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff34c096a90 tx=0x7ff34c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184147ceea9e095e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:40.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff33affd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:40.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff33affd700 1 -- 172.21.15.12:0/3742897102 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c0b81f0 con 0x563e7c73b890 2024-05-09T00:04:40.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff36de85b80 1 -- 172.21.15.12:0/3742897102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e7c5e4270 con 0x563e7c73b890 2024-05-09T00:04:40.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff33affd700 1 -- 172.21.15.12:0/3742897102 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff34c0b8350 con 0x563e7c73b890 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff33affd700 1 -- 172.21.15.12:0/3742897102 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff34c099560 con 0x563e7c73b890 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff36de85b80 10 client.20526 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.189+0000 7ff36de85b80 1 -- 172.21.15.12:0/3742897102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x563e7c4b8b60 con 0x563e7c73b890 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.190+0000 7ff33affd700 1 -- 172.21.15.12:0/3742897102 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff34c083020 con 0x563e7c73b890 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.190+0000 7ff33affd700 1 -- 172.21.15.12:0/3742897102 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff34c0a2060 con 0x563e7c73b890 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.190+0000 7ff36de85b80 10 client.20526 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.190+0000 7ff36de85b80 10 client.20526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.190+0000 7ff36de85b80 20 client.20526 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.190+0000 7ff36de85b80 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e7c759260 0x563e7c75b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.190+0000 7ff36de85b80 4 client.20526 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.190+0000 7ff36de85b80 1 -- 172.21.15.12:0/3742897102 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e7c70a810 con 0x563e7c759260 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.190+0000 7ff35affd700 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e7c759260 0x563e7c75b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.191+0000 7ff35affd700 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e7c759260 0x563e7c75b6a0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced632d4e5f11539 server_cookie=a805ae6979e72089 in_seq=0 out_seq=0 2024-05-09T00:04:40.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.191+0000 7ff33affd700 10 client.20526 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.194+0000 7ff33affd700 1 -- 172.21.15.12:0/3742897102 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e7c70a810 con 0x563e7c759260 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.194+0000 7ff33affd700 10 client.20526 handle_command_reply: tid=0 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.194+0000 7ff36de85b80 1 client.20526 shutdown 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.194+0000 7ff36de85b80 1 -- 172.21.15.12:0/3742897102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 msgr2=0x563e7c744ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.194+0000 7ff36de85b80 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 0x563e7c744ad0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff34c096a90 tx=0x7ff34c0036f0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.195+0000 7ff36de85b80 1 -- 172.21.15.12:0/3742897102 shutdown_connections 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.195+0000 7ff36de85b80 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 0x563e7c745550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.195+0000 7ff36de85b80 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c73b890 0x563e7c744ad0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.195+0000 7ff36de85b80 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e7c759260 0x563e7c75b6a0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.195+0000 7ff36de85b80 1 --2- 172.21.15.12:0/3742897102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c740e00 0x563e7c745010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.195+0000 7ff33affd700 0 client.20526 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:40.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.195+0000 7ff36de85b80 1 -- 172.21.15.12:0/3742897102 >> 172.21.15.12:0/3742897102 conn(0x563e7c699210 msgr2=0x563e7c7355a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:40.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.195+0000 7ff36de85b80 1 -- 172.21.15.12:0/3742897102 shutdown_connections 2024-05-09T00:04:40.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.195+0000 7ff36de85b80 1 -- 172.21.15.12:0/3742897102 wait complete. 2024-05-09T00:04:40.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.195+0000 7ff36de85b80 20 client.20526 trim_cache size 0 max 16384 2024-05-09T00:04:40.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff36de85b80 1 Processor -- start 2024-05-09T00:04:40.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff36de85b80 1 -- start start 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c73b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c740e00 0x563e7c7556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c741720 0x563e7c735d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff36de85b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c7608b0 con 0x563e7c741720 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff36de85b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c730800 con 0x563e7c73b890 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff36de85b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c741540 con 0x563e7c740e00 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff35b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c73b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff35b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c73b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58312/0 (socket says 172.21.15.12:58312) 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2341003167 learned_addr learned my addr 172.21.15.12:0/2341003167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff35bfff700 1 --2- 172.21.15.12:0/2341003167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c741720 0x563e7c735d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.214+0000 7ff35affd700 1 --2- 172.21.15.12:0/2341003167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c740e00 0x563e7c7556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.215+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2341003167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c740e00 msgr2=0x563e7c7556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.215+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/2341003167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c740e00 0x563e7c7556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.215+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2341003167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c741720 msgr2=0x563e7c735d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.215+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/2341003167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c741720 0x563e7c735d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.215+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2341003167 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e7c651560 con 0x563e7c73b890 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.215+0000 7ff35bfff700 1 --2- 172.21.15.12:0/2341003167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c741720 0x563e7c735d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.215+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/2341003167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c73b350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0983c0 tx=0x7ff34c099f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0536150a2e914d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.215+0000 7ff3427fc700 1 -- 172.21.15.12:0/2341003167 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c099170 con 0x563e7c73b890 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.215+0000 7ff3427fc700 1 -- 172.21.15.12:0/2341003167 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff34c083050 con 0x563e7c73b890 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.215+0000 7ff3427fc700 1 -- 172.21.15.12:0/2341003167 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c0b8450 con 0x563e7c73b890 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.216+0000 7ff36de85b80 1 -- 172.21.15.12:0/2341003167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 msgr2=0x563e7c73b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.216+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2341003167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c73b350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0983c0 tx=0x7ff34c099f70 comp rx=0 tx=0).stop 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.216+0000 7ff36de85b80 1 -- 172.21.15.12:0/2341003167 shutdown_connections 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.216+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2341003167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c741720 0x563e7c735d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.216+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2341003167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c73b350 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.216+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2341003167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c740e00 0x563e7c7556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.216+0000 7ff36de85b80 1 -- 172.21.15.12:0/2341003167 >> 172.21.15.12:0/2341003167 conn(0x563e7c699210 msgr2=0x563e7c73bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.216+0000 7ff36de85b80 1 -- 172.21.15.12:0/2341003167 shutdown_connections 2024-05-09T00:04:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.216+0000 7ff36de85b80 1 -- 172.21.15.12:0/2341003167 wait complete. 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.217+0000 7ff36de85b80 1 -- start start 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.217+0000 7ff36de85b80 1 Processor -- start 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c74c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c740e00 0x563e7c74cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff36de85b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 0x563e7c74d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff36de85b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c7ae770 con 0x563e7c740e00 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff36de85b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c7ae8b0 con 0x563e7c73b890 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff36de85b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e7c7ae9f0 con 0x563e7c741720 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff36de85b80 10 client.0 _ll_register_callbacks cb 0x563e7c630b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff35affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c740e00 0x563e7c74cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff35b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c74c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff35affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c740e00 0x563e7c74cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57818/0 (socket says 172.21.15.12:57818) 2024-05-09T00:04:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff35b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c74c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58320/0 (socket says 172.21.15.12:58320) 2024-05-09T00:04:40.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff35affd700 1 -- 172.21.15.12:0/2037896774 learned_addr learned my addr 172.21.15.12:0/2037896774 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:40.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2037896774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 msgr2=0x563e7c74d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:40.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 0x563e7c74d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2037896774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c740e00 msgr2=0x563e7c74cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:40.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c740e00 0x563e7c74cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.218+0000 7ff35b7fe700 1 -- 172.21.15.12:0/2037896774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e7c4b8b60 con 0x563e7c73b890 2024-05-09T00:04:40.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.219+0000 7ff35b7fe700 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c74c5d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff34c09a000 tx=0x7ff34c0b8560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e9cad95a61f676b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:40.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.219+0000 7ff33affd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:04:40.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.219+0000 7ff33affd700 1 -- 172.21.15.12:0/2037896774 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c099170 con 0x563e7c73b890 2024-05-09T00:04:40.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.219+0000 7ff33affd700 1 -- 172.21.15.12:0/2037896774 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff34c099360 con 0x563e7c73b890 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.219+0000 7ff36de85b80 1 -- 172.21.15.12:0/2037896774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff34c09dea0 con 0x563e7c73b890 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.219+0000 7ff33affd700 1 -- 172.21.15.12:0/2037896774 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff34c0963b0 con 0x563e7c73b890 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.219+0000 7ff36de85b80 10 client.30116 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.219+0000 7ff36de85b80 1 -- 172.21.15.12:0/2037896774 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x563e7c510cd0 con 0x563e7c73b890 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.219+0000 7ff33affd700 1 -- 172.21.15.12:0/2037896774 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff34c09dea0 con 0x563e7c73b890 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.220+0000 7ff33affd700 1 -- 172.21.15.12:0/2037896774 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff34c09eea0 con 0x563e7c73b890 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.220+0000 7ff36de85b80 10 client.30116 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.220+0000 7ff36de85b80 10 client.30116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.220+0000 7ff36de85b80 20 client.30116 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.220+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e7c7afba0 0x563e7c7b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.220+0000 7ff36de85b80 4 client.30116 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.220+0000 7ff36de85b80 1 -- 172.21.15.12:0/2037896774 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563e7c683ec0 con 0x563e7c7afba0 2024-05-09T00:04:40.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.220+0000 7ff35affd700 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e7c7afba0 0x563e7c7b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:40.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.220+0000 7ff33affd700 10 client.30116 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:40.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.220+0000 7ff35affd700 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e7c7afba0 0x563e7c7b1f90 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c8eaea9c4c4c78 server_cookie=350ca1d3cc76fcff in_seq=0 out_seq=0 2024-05-09T00:04:40.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.222+0000 7ff33affd700 1 -- 172.21.15.12:0/2037896774 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563e7c683ec0 con 0x563e7c7afba0 2024-05-09T00:04:40.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.222+0000 7ff33affd700 10 client.30116 handle_command_reply: tid=0 2024-05-09T00:04:40.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.222+0000 7ff36de85b80 1 client.30116 shutdown 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 1 -- 172.21.15.12:0/2037896774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 msgr2=0x563e7c74c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c74c5d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff34c09a000 tx=0x7ff34c0b8560 comp rx=0 tx=0).stop 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 1 -- 172.21.15.12:0/2037896774 shutdown_connections 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e7c741720 0x563e7c74d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563e7c7afba0 0x563e7c7b1f90 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e7c73b890 0x563e7c74c5d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 1 --2- 172.21.15.12:0/2037896774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e7c740e00 0x563e7c74cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 1 -- 172.21.15.12:0/2037896774 >> 172.21.15.12:0/2037896774 conn(0x563e7c699210 msgr2=0x563e7c743380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff33affd700 0 client.30116 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 1 -- 172.21.15.12:0/2037896774 shutdown_connections 2024-05-09T00:04:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 1 -- 172.21.15.12:0/2037896774 wait complete. 2024-05-09T00:04:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.224+0000 7ff36de85b80 20 client.30116 trim_cache size 0 max 16384 2024-05-09T00:04:40.221 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:04:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 -- 172.21.15.12:0/1350535905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3440a1290 msgr2=0x7ff3440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3440a1290 0x7ff3440a3750 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff350009fd0 tx=0x7ff350019040 comp rx=0 tx=0).stop 2024-05-09T00:04:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 -- 172.21.15.12:0/1350535905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff35c1000f0 msgr2=0x7ff35c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff35c1000f0 0x7ff35c1972c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00e970 tx=0x7ff34c00ec80 comp rx=0 tx=0).stop 2024-05-09T00:04:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 -- 172.21.15.12:0/1350535905 shutdown_connections 2024-05-09T00:04:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff3440a1290 0x7ff3440a3750 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff35c105f90 0x7ff35c19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff35c1000f0 0x7ff35c1972c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 --2- 172.21.15.12:0/1350535905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff35c100ab0 0x7ff35c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 -- 172.21.15.12:0/1350535905 >> 172.21.15.12:0/1350535905 conn(0x7ff35c076ba0 msgr2=0x7ff35c069300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 -- 172.21.15.12:0/1350535905 shutdown_connections 2024-05-09T00:04:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:40.225+0000 7ff361ccb700 1 -- 172.21.15.12:0/1350535905 wait complete. 2024-05-09T00:04:40.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 191575 36.37 MB/sec execute 155 sec latency 88.128 ms 2024-05-09T00:04:40.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:40 smithi012 ceph-mon[42503]: pgmap v588: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.7 MiB/s wr, 661 op/s 2024-05-09T00:04:41.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:40 smithi204 ceph-mon[47373]: pgmap v588: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.7 MiB/s wr, 661 op/s 2024-05-09T00:04:41.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:40 smithi160 ceph-mon[46152]: pgmap v588: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.7 MiB/s wr, 661 op/s 2024-05-09T00:04:41.236 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:04:41.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:04:41.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 192261 36.38 MB/sec execute 156 sec latency 156.455 ms 2024-05-09T00:04:41.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.434+0000 7f251ad6a700 1 -- 172.21.15.12:0/325408976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2514105160 msgr2=0x7f251410cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.434+0000 7f251ad6a700 1 --2- 172.21.15.12:0/325408976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2514105160 0x7f251410cce0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f250c00b0a0 tx=0x7f250c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.435+0000 7f251ad6a700 1 -- 172.21.15.12:0/325408976 shutdown_connections 2024-05-09T00:04:41.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.435+0000 7f251ad6a700 1 --2- 172.21.15.12:0/325408976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2514105160 0x7f251410cce0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.435+0000 7f251ad6a700 1 --2- 172.21.15.12:0/325408976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2514100060 0x7f2514100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.435+0000 7f251ad6a700 1 --2- 172.21.15.12:0/325408976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2514100a10 0x7f2514104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.435+0000 7f251ad6a700 1 -- 172.21.15.12:0/325408976 >> 172.21.15.12:0/325408976 conn(0x7f2514076f40 msgr2=0x7f2514077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.435+0000 7f251ad6a700 1 -- 172.21.15.12:0/325408976 shutdown_connections 2024-05-09T00:04:41.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.435+0000 7f251ad6a700 1 -- 172.21.15.12:0/325408976 wait complete. 2024-05-09T00:04:41.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.435+0000 7f251ad6a700 1 Processor -- start 2024-05-09T00:04:41.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f251ad6a700 1 -- start start 2024-05-09T00:04:41.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f251ad6a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2514100060 0x7f25141a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f251ad6a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2514105160 0x7f25141a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f251ad6a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f251419dc10 0x7f251419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f251ad6a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251419e710 con 0x7f2514105160 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f251ad6a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251419e880 con 0x7f251419dc10 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f251ad6a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251419ea40 con 0x7f2514100060 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2513fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2514105160 0x7f25141a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2518b06700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2514100060 0x7f25141a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2513fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2514105160 0x7f25141a3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57846/0 (socket says 172.21.15.12:57846) 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2513fff700 1 -- 172.21.15.12:0/1287593862 learned_addr learned my addr 172.21.15.12:0/1287593862 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2519307700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f251419dc10 0x7f251419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2513fff700 1 -- 172.21.15.12:0/1287593862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2514100060 msgr2=0x7f25141a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2513fff700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2514100060 0x7f25141a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2513fff700 1 -- 172.21.15.12:0/1287593862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f251419dc10 msgr2=0x7f251419e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2513fff700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f251419dc10 0x7f251419e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2513fff700 1 -- 172.21.15.12:0/1287593862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c009d00 con 0x7f2514105160 2024-05-09T00:04:41.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.436+0000 7f2518b06700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2514100060 0x7f25141a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:41.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.437+0000 7f2513fff700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2514105160 0x7f25141a3e80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f250000e970 tx=0x7f250000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.437+0000 7f2511ffb700 1 -- 172.21.15.12:0/1287593862 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500009790 con 0x7f2514105160 2024-05-09T00:04:41.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.437+0000 7f2511ffb700 1 -- 172.21.15.12:0/1287593862 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f250000c7e0 con 0x7f2514105160 2024-05-09T00:04:41.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.437+0000 7f2511ffb700 1 -- 172.21.15.12:0/1287593862 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000054e0 con 0x7f2514105160 2024-05-09T00:04:41.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.437+0000 7f251ad6a700 1 -- 172.21.15.12:0/1287593862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25141aa800 con 0x7f2514105160 2024-05-09T00:04:41.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.437+0000 7f251ad6a700 1 -- 172.21.15.12:0/1287593862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25141aacc0 con 0x7f2514105160 2024-05-09T00:04:41.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.440+0000 7f2511ffb700 1 -- 172.21.15.12:0/1287593862 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f250000c950 con 0x7f2514105160 2024-05-09T00:04:41.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.441+0000 7f2511ffb700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25040a1290 0x7f25040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.441+0000 7f2518b06700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25040a1290 0x7f25040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.441+0000 7f2511ffb700 1 -- 172.21.15.12:0/1287593862 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2500014070 con 0x7f2514105160 2024-05-09T00:04:41.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.442+0000 7f2518b06700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25040a1290 0x7f25040a3750 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2508011490 tx=0x7f2508009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.442+0000 7f2526f24b80 1 Processor -- start 2024-05-09T00:04:41.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.442+0000 7f2526f24b80 1 -- start start 2024-05-09T00:04:41.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a989a7a0 0x55f5a98943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a9894910 0x55f5a9894d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2526f24b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a97eb1d0 con 0x55f5a9899e80 2024-05-09T00:04:41.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2526f24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a97e8b90 con 0x55f5a9894910 2024-05-09T00:04:41.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2526f24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a97e4480 con 0x55f5a989a7a0 2024-05-09T00:04:41.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2518b06700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2513fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a989a7a0 0x55f5a98943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2518b06700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57858/0 (socket says 172.21.15.12:57858) 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2519307700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a9894910 0x55f5a9894d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2513fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a989a7a0 0x55f5a98943d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39436/0 (socket says 172.21.15.12:39436) 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2518b06700 1 -- 172.21.15.12:0/666267784 learned_addr learned my addr 172.21.15.12:0/666267784 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2513fff700 1 -- 172.21.15.12:0/666267784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a9894910 msgr2=0x55f5a9894d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2513fff700 1 --2- 172.21.15.12:0/666267784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a9894910 0x55f5a9894d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2513fff700 1 -- 172.21.15.12:0/666267784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 msgr2=0x55f5a989a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2513fff700 1 --2- 172.21.15.12:0/666267784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2513fff700 1 -- 172.21.15.12:0/666267784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5a9665430 con 0x55f5a989a7a0 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.443+0000 7f2518b06700 1 --2- 172.21.15.12:0/666267784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.444+0000 7f2513fff700 1 --2- 172.21.15.12:0/666267784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a989a7a0 0x55f5a98943d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f25000ca600 tx=0x7f2500099c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a94568dd72d7507 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.444+0000 7f24feffd700 1 -- 172.21.15.12:0/666267784 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500099fc0 con 0x55f5a989a7a0 2024-05-09T00:04:41.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.444+0000 7f24feffd700 1 -- 172.21.15.12:0/666267784 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25000ade90 con 0x55f5a989a7a0 2024-05-09T00:04:41.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.444+0000 7f24feffd700 1 -- 172.21.15.12:0/666267784 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500016590 con 0x55f5a989a7a0 2024-05-09T00:04:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 -- 172.21.15.12:0/666267784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a989a7a0 msgr2=0x55f5a98943d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 --2- 172.21.15.12:0/666267784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a989a7a0 0x55f5a98943d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f25000ca600 tx=0x7f2500099c30 comp rx=0 tx=0).stop 2024-05-09T00:04:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 -- 172.21.15.12:0/666267784 shutdown_connections 2024-05-09T00:04:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 --2- 172.21.15.12:0/666267784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a9894910 0x55f5a9894d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 --2- 172.21.15.12:0/666267784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 --2- 172.21.15.12:0/666267784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a989a7a0 0x55f5a98943d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 -- 172.21.15.12:0/666267784 >> 172.21.15.12:0/666267784 conn(0x55f5a97f2290 msgr2=0x55f5a988ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 -- 172.21.15.12:0/666267784 shutdown_connections 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 -- 172.21.15.12:0/666267784 wait complete. 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 -- start start 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.445+0000 7f2526f24b80 1 Processor -- start 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a989db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a989e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2526f24b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a97805e0 con 0x55f5a9899e80 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2526f24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a9780280 con 0x55f5a989a7a0 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2526f24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a9667720 con 0x55f5a9894910 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2526f24b80 10 client.0 _ll_register_callbacks cb 0x55f5a9789bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2518b06700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a989db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2513fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2518b06700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a989db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39452/0 (socket says 172.21.15.12:39452) 2024-05-09T00:04:41.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2519307700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a989e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.446+0000 7f2518b06700 1 -- 172.21.15.12:0/3810970911 learned_addr learned my addr 172.21.15.12:0/3810970911 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:41.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f2513fff700 1 -- 172.21.15.12:0/3810970911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 msgr2=0x55f5a989db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f2513fff700 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a989db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f2513fff700 1 -- 172.21.15.12:0/3810970911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 msgr2=0x55f5a989e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f2513fff700 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a989e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f2513fff700 1 -- 172.21.15.12:0/3810970911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2508017070 con 0x55f5a9899e80 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f2513fff700 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989e070 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f25000a2600 tx=0x7f25000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829d461abc5824ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f24f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f24f3fff700 1 -- 172.21.15.12:0/3810970911 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500016540 con 0x55f5a9899e80 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f24f3fff700 1 -- 172.21.15.12:0/3810970911 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2500099fc0 con 0x55f5a9899e80 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f2526f24b80 1 -- 172.21.15.12:0/3810970911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5a973d2f0 con 0x55f5a9899e80 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f24f3fff700 1 -- 172.21.15.12:0/3810970911 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f250009a220 con 0x55f5a9899e80 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f2526f24b80 10 client.20544 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.447+0000 7f2526f24b80 1 -- 172.21.15.12:0/3810970911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7f25141aacc0 con 0x55f5a9899e80 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.448+0000 7f24f3fff700 1 -- 172.21.15.12:0/3810970911 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5a973d2f0 con 0x55f5a9899e80 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.449+0000 7f24f3fff700 1 -- 172.21.15.12:0/3810970911 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f25000a1fb0 con 0x55f5a9899e80 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.449+0000 7f2526f24b80 10 client.20544 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.449+0000 7f2526f24b80 10 client.20544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.449+0000 7f2526f24b80 20 client.20544 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.449+0000 7f2526f24b80 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f5a98ade50 0x55f5a98b0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.449+0000 7f2526f24b80 4 client.20544 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.449+0000 7f2526f24b80 1 -- 172.21.15.12:0/3810970911 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5a9863890 con 0x55f5a98ade50 2024-05-09T00:04:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.449+0000 7f2518b06700 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f5a98ade50 0x55f5a98b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.449+0000 7f24f3fff700 10 client.20544 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:41.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.449+0000 7f2518b06700 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f5a98ade50 0x55f5a98b0240 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eabc7940cc8ad10f server_cookie=ebc8e297f4090bee in_seq=0 out_seq=0 2024-05-09T00:04:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f24f3fff700 1 -- 172.21.15.12:0/3810970911 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f5a9863890 con 0x55f5a98ade50 2024-05-09T00:04:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f24f3fff700 10 client.20544 handle_command_reply: tid=0 2024-05-09T00:04:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 client.20544 shutdown 2024-05-09T00:04:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 -- 172.21.15.12:0/3810970911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 msgr2=0x55f5a989e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989e070 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f25000a2600 tx=0x7f25000047c0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 -- 172.21.15.12:0/3810970911 shutdown_connections 2024-05-09T00:04:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f5a98ade50 0x55f5a98b0240 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a989e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a989db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f24f3fff700 0 client.20544 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 --2- 172.21.15.12:0/3810970911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a989e070 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 -- 172.21.15.12:0/3810970911 >> 172.21.15.12:0/3810970911 conn(0x55f5a97f2290 msgr2=0x55f5a988e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 -- 172.21.15.12:0/3810970911 shutdown_connections 2024-05-09T00:04:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 1 -- 172.21.15.12:0/3810970911 wait complete. 2024-05-09T00:04:41.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.454+0000 7f2526f24b80 20 client.20544 trim_cache size 0 max 16384 2024-05-09T00:04:41.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2526f24b80 1 Processor -- start 2024-05-09T00:04:41.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2526f24b80 1 -- start start 2024-05-09T00:04:41.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a98943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9899310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a988eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2526f24b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a988be90 con 0x55f5a9899e80 2024-05-09T00:04:41.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2526f24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a9887180 con 0x55f5a989a7a0 2024-05-09T00:04:41.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2526f24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a98b53b0 con 0x55f5a9894910 2024-05-09T00:04:41.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2513fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9899310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2513fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9899310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57880/0 (socket says 172.21.15.12:57880) 2024-05-09T00:04:41.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2518b06700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a98943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.475+0000 7f2513fff700 1 -- 172.21.15.12:0/2179245859 learned_addr learned my addr 172.21.15.12:0/2179245859 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:41.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.476+0000 7f2519307700 1 --2- 172.21.15.12:0/2179245859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a988eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.476+0000 7f2513fff700 1 -- 172.21.15.12:0/2179245859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 msgr2=0x55f5a98943d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.476+0000 7f2513fff700 1 --2- 172.21.15.12:0/2179245859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a98943d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.476+0000 7f2513fff700 1 -- 172.21.15.12:0/2179245859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 msgr2=0x55f5a988eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.476+0000 7f2513fff700 1 --2- 172.21.15.12:0/2179245859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a988eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.476+0000 7f2513fff700 1 -- 172.21.15.12:0/2179245859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5a9894fc0 con 0x55f5a9899e80 2024-05-09T00:04:41.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.476+0000 7f2513fff700 1 --2- 172.21.15.12:0/2179245859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9899310 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2500009230 tx=0x7f2500099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e1b95f8d1ecb78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.476+0000 7f24ff7fe700 1 -- 172.21.15.12:0/2179245859 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500099fc0 con 0x55f5a9899e80 2024-05-09T00:04:41.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.476+0000 7f24ff7fe700 1 -- 172.21.15.12:0/2179245859 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f250009a950 con 0x55f5a9899e80 2024-05-09T00:04:41.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.476+0000 7f24ff7fe700 1 -- 172.21.15.12:0/2179245859 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500099710 con 0x55f5a9899e80 2024-05-09T00:04:41.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.477+0000 7f2526f24b80 1 -- 172.21.15.12:0/2179245859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 msgr2=0x55f5a9899310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.477+0000 7f2526f24b80 1 --2- 172.21.15.12:0/2179245859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9899310 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2500009230 tx=0x7f2500099bd0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.477+0000 7f2526f24b80 1 -- 172.21.15.12:0/2179245859 shutdown_connections 2024-05-09T00:04:41.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.477+0000 7f2526f24b80 1 --2- 172.21.15.12:0/2179245859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a988eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.477+0000 7f2526f24b80 1 --2- 172.21.15.12:0/2179245859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a98943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.477+0000 7f2526f24b80 1 --2- 172.21.15.12:0/2179245859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9899310 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.477+0000 7f2526f24b80 1 -- 172.21.15.12:0/2179245859 >> 172.21.15.12:0/2179245859 conn(0x55f5a97f2290 msgr2=0x55f5a9898c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.479+0000 7f2526f24b80 1 -- 172.21.15.12:0/2179245859 shutdown_connections 2024-05-09T00:04:41.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.480+0000 7f2526f24b80 1 -- 172.21.15.12:0/2179245859 wait complete. 2024-05-09T00:04:41.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.480+0000 7f2526f24b80 1 -- start start 2024-05-09T00:04:41.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.480+0000 7f2526f24b80 1 Processor -- start 2024-05-09T00:04:41.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a98965d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9896b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2526f24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a9897050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2526f24b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a99042f0 con 0x55f5a9899e80 2024-05-09T00:04:41.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2526f24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a9904460 con 0x55f5a989a7a0 2024-05-09T00:04:41.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2526f24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a9904620 con 0x55f5a9894910 2024-05-09T00:04:41.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2526f24b80 10 client.0 _ll_register_callbacks cb 0x55f5a9789bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:41.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2518b06700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a98965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2518b06700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a98965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39462/0 (socket says 172.21.15.12:39462) 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2518b06700 1 -- 172.21.15.12:0/354487252 learned_addr learned my addr 172.21.15.12:0/354487252 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2519307700 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a9897050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2513fff700 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9896b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2518b06700 1 -- 172.21.15.12:0/354487252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 msgr2=0x55f5a9897050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2518b06700 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a9897050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2518b06700 1 -- 172.21.15.12:0/354487252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 msgr2=0x55f5a9896b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2518b06700 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9896b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2518b06700 1 -- 172.21.15.12:0/354487252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25141aacc0 con 0x55f5a9894910 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.481+0000 7f2513fff700 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9896b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.482+0000 7f2518b06700 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a98965d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2508013010 tx=0x7f25080047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f3283371debf29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.482+0000 7f24f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.482+0000 7f24f3fff700 1 -- 172.21.15.12:0/354487252 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508024040 con 0x55f5a9894910 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.482+0000 7f24f3fff700 1 -- 172.21.15.12:0/354487252 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f25080039d0 con 0x55f5a9894910 2024-05-09T00:04:41.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.482+0000 7f2526f24b80 1 -- 172.21.15.12:0/354487252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25000ade90 con 0x55f5a9894910 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.482+0000 7f24f3fff700 1 -- 172.21.15.12:0/354487252 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2508003b90 con 0x55f5a9894910 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.482+0000 7f2526f24b80 10 client.30265 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.482+0000 7f2526f24b80 1 -- 172.21.15.12:0/354487252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55f5a989d690 con 0x55f5a9894910 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.483+0000 7f24f3fff700 1 -- 172.21.15.12:0/354487252 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25000ade90 con 0x55f5a9894910 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.483+0000 7f24f3fff700 1 -- 172.21.15.12:0/354487252 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2508039070 con 0x55f5a9894910 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.483+0000 7f2526f24b80 10 client.30265 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.483+0000 7f2526f24b80 10 client.30265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.483+0000 7f2526f24b80 20 client.30265 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.483+0000 7f2526f24b80 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f5a9905e20 0x55f5a9908240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.483+0000 7f2526f24b80 4 client.30265 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.483+0000 7f2526f24b80 1 -- 172.21.15.12:0/354487252 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f5a97dcf40 con 0x55f5a9905e20 2024-05-09T00:04:41.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.483+0000 7f2513fff700 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f5a9905e20 0x55f5a9908240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.484+0000 7f24f3fff700 10 client.30265 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:41.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.484+0000 7f2513fff700 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f5a9905e20 0x55f5a9908240 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3cadc38275912a server_cookie=96496b0e030381bf in_seq=0 out_seq=0 2024-05-09T00:04:41.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f24f3fff700 1 -- 172.21.15.12:0/354487252 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f5a97dcf40 con 0x55f5a9905e20 2024-05-09T00:04:41.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f24f3fff700 10 client.30265 handle_command_reply: tid=0 2024-05-09T00:04:41.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 client.30265 shutdown 2024-05-09T00:04:41.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 -- 172.21.15.12:0/354487252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 msgr2=0x55f5a98965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a98965d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2508013010 tx=0x7f25080047c0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 -- 172.21.15.12:0/354487252 shutdown_connections 2024-05-09T00:04:41.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55f5a9905e20 0x55f5a9908240 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f5a989a7a0 0x55f5a9897050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5a9894910 0x55f5a98965d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 --2- 172.21.15.12:0/354487252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f5a9899e80 0x55f5a9896b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 -- 172.21.15.12:0/354487252 >> 172.21.15.12:0/354487252 conn(0x55f5a97f2290 msgr2=0x55f5a9899490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f24f3fff700 0 client.30265 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 -- 172.21.15.12:0/354487252 shutdown_connections 2024-05-09T00:04:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.488+0000 7f2526f24b80 1 -- 172.21.15.12:0/354487252 wait complete. 2024-05-09T00:04:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.489+0000 7f2526f24b80 20 client.30265 trim_cache size 0 max 16384 2024-05-09T00:04:41.484 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:04:41.484 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:04:41.485 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "fc0b7a5e-d547-4e41-a7a0-b3452b6049a0", 2024-05-09T00:04:41.485 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:04:41.485 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:04:41.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 -- 172.21.15.12:0/1287593862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25040a1290 msgr2=0x7f25040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25040a1290 0x7f25040a3750 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2508011490 tx=0x7f2508009260 comp rx=0 tx=0).stop 2024-05-09T00:04:41.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 -- 172.21.15.12:0/1287593862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2514105160 msgr2=0x7f25141a3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2514105160 0x7f25141a3e80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f250000e970 tx=0x7f250000ec80 comp rx=0 tx=0).stop 2024-05-09T00:04:41.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 -- 172.21.15.12:0/1287593862 shutdown_connections 2024-05-09T00:04:41.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f25040a1290 0x7f25040a3750 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f251419dc10 0x7f251419e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2514100060 0x7f25141a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 --2- 172.21.15.12:0/1287593862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2514105160 0x7f25141a3e80 secure :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f250000e970 tx=0x7f250000ec80 comp rx=0 tx=0).stop 2024-05-09T00:04:41.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 -- 172.21.15.12:0/1287593862 >> 172.21.15.12:0/1287593862 conn(0x7f2514076f40 msgr2=0x7f2514068130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 -- 172.21.15.12:0/1287593862 shutdown_connections 2024-05-09T00:04:41.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.490+0000 7f251ad6a700 1 -- 172.21.15.12:0/1287593862 wait complete. 2024-05-09T00:04:41.500 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:04:41.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.708+0000 7f9b382d1700 1 -- 172.21.15.12:0/97667565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30101140 msgr2=0x7f9b30101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.708+0000 7f9b382d1700 1 --2- 172.21.15.12:0/97667565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30101140 0x7f9b30101520 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009990 tx=0x7f9b2c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.708+0000 7f9b382d1700 1 -- 172.21.15.12:0/97667565 shutdown_connections 2024-05-09T00:04:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.708+0000 7f9b382d1700 1 --2- 172.21.15.12:0/97667565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30106260 0x7f9b3010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.708+0000 7f9b382d1700 1 --2- 172.21.15.12:0/97667565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30101140 0x7f9b30101520 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.708+0000 7f9b382d1700 1 --2- 172.21.15.12:0/97667565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b30101af0 0x7f9b30105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.708+0000 7f9b382d1700 1 -- 172.21.15.12:0/97667565 >> 172.21.15.12:0/97667565 conn(0x7f9b300fc9e0 msgr2=0x7f9b300fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.708+0000 7f9b382d1700 1 -- 172.21.15.12:0/97667565 shutdown_connections 2024-05-09T00:04:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.708+0000 7f9b382d1700 1 -- 172.21.15.12:0/97667565 wait complete. 2024-05-09T00:04:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.709+0000 7f9b382d1700 1 Processor -- start 2024-05-09T00:04:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.709+0000 7f9b382d1700 1 -- start start 2024-05-09T00:04:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.709+0000 7f9b382d1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b30101140 0x7f9b300732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.709+0000 7f9b382d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30106260 0x7f9b300737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.709+0000 7f9b382d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30073d20 0x7f9b301a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.709+0000 7f9b382d1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b301a5c70 con 0x7f9b30106260 2024-05-09T00:04:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.709+0000 7f9b382d1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b301a5de0 con 0x7f9b30073d20 2024-05-09T00:04:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.709+0000 7f9b382d1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b301a5fa0 con 0x7f9b30101140 2024-05-09T00:04:41.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.709+0000 7f9b3606d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b30101140 0x7f9b300732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.709+0000 7f9b3606d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b30101140 0x7f9b300732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39480/0 (socket says 172.21.15.12:39480) 2024-05-09T00:04:41.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.710+0000 7f9b3606d700 1 -- 172.21.15.12:0/877501995 learned_addr learned my addr 172.21.15.12:0/877501995 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:41.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.710+0000 7f9b3606d700 1 -- 172.21.15.12:0/877501995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30073d20 msgr2=0x7f9b301a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.710+0000 7f9b3606d700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30073d20 0x7f9b301a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.710+0000 7f9b3606d700 1 -- 172.21.15.12:0/877501995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30106260 msgr2=0x7f9b300737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.710+0000 7f9b3606d700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30106260 0x7f9b300737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.710+0000 7f9b3606d700 1 -- 172.21.15.12:0/877501995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0095f0 con 0x7f9b30101140 2024-05-09T00:04:41.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.710+0000 7f9b3606d700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b30101140 0x7f9b300732a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00c010 tx=0x7f9b2c009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.710+0000 7f9b277fe700 1 -- 172.21.15.12:0/877501995 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c00fb30 con 0x7f9b30101140 2024-05-09T00:04:41.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.710+0000 7f9b277fe700 1 -- 172.21.15.12:0/877501995 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2c027070 con 0x7f9b30101140 2024-05-09T00:04:41.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.710+0000 7f9b382d1700 1 -- 172.21.15.12:0/877501995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b301971f0 con 0x7f9b30101140 2024-05-09T00:04:41.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.711+0000 7f9b277fe700 1 -- 172.21.15.12:0/877501995 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c017a50 con 0x7f9b30101140 2024-05-09T00:04:41.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.711+0000 7f9b382d1700 1 -- 172.21.15.12:0/877501995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b301976a0 con 0x7f9b30101140 2024-05-09T00:04:41.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.714+0000 7f9b277fe700 1 -- 172.21.15.12:0/877501995 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9b2c029030 con 0x7f9b30101140 2024-05-09T00:04:41.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.715+0000 7f9b277fe700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b1c0b22a0 0x7f9b1c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.715+0000 7f9b277fe700 1 -- 172.21.15.12:0/877501995 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9b2c0cd960 con 0x7f9b30101140 2024-05-09T00:04:41.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.716+0000 7f9b4448bb80 1 Processor -- start 2024-05-09T00:04:41.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.716+0000 7f9b4448bb80 1 -- start start 2024-05-09T00:04:41.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6ae00 0x55db21e6b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6b720 0x55db21e65350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e65cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21dbc150 con 0x55db21e6b720 2024-05-09T00:04:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21db9b10 con 0x55db21e65890 2024-05-09T00:04:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21db5400 con 0x55db21e6ae00 2024-05-09T00:04:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b3606d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6ae00 0x55db21e6b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b3686e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e65cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b3686e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e65cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58386/0 (socket says 172.21.15.12:58386) 2024-05-09T00:04:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b3686e700 1 -- 172.21.15.12:0/726969935 learned_addr learned my addr 172.21.15.12:0/726969935 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:41.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b3686e700 1 -- 172.21.15.12:0/726969935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6ae00 msgr2=0x55db21e6b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b3686e700 1 --2- 172.21.15.12:0/726969935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6ae00 0x55db21e6b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b3686e700 1 -- 172.21.15.12:0/726969935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6b720 msgr2=0x55db21e65350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:41.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b3686e700 1 --2- 172.21.15.12:0/726969935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6b720 0x55db21e65350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.717+0000 7f9b3686e700 1 -- 172.21.15.12:0/726969935 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db21c363b0 con 0x55db21e65890 2024-05-09T00:04:41.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.718+0000 7f9b3586c700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b1c0b22a0 0x7f9b1c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.719+0000 7f9b3586c700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b1c0b22a0 0x7f9b1c0b4760 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55db21e61bd0 tx=0x7f9b200058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.719+0000 7f9b3686e700 1 --2- 172.21.15.12:0/726969935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e65cd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009d00 tx=0x7f9b2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35f6ebcbf40a479f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.719+0000 7f9b24da4700 1 -- 172.21.15.12:0/726969935 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2800e070 con 0x55db21e65890 2024-05-09T00:04:41.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.720+0000 7f9b24da4700 1 -- 172.21.15.12:0/726969935 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b28009830 con 0x55db21e65890 2024-05-09T00:04:41.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.720+0000 7f9b24da4700 1 -- 172.21.15.12:0/726969935 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28011390 con 0x55db21e65890 2024-05-09T00:04:41.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.721+0000 7f9b4448bb80 1 -- 172.21.15.12:0/726969935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 msgr2=0x55db21e65cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.721+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/726969935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e65cd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009d00 tx=0x7f9b2800b330 comp rx=0 tx=0).stop 2024-05-09T00:04:41.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.721+0000 7f9b4448bb80 1 -- 172.21.15.12:0/726969935 shutdown_connections 2024-05-09T00:04:41.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.721+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/726969935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e65cd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.721+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/726969935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6ae00 0x55db21e6b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.721+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/726969935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6b720 0x55db21e65350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.721+0000 7f9b4448bb80 1 -- 172.21.15.12:0/726969935 >> 172.21.15.12:0/726969935 conn(0x55db21dc3210 msgr2=0x55db21e5fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.721+0000 7f9b4448bb80 1 -- 172.21.15.12:0/726969935 shutdown_connections 2024-05-09T00:04:41.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.721+0000 7f9b4448bb80 1 -- 172.21.15.12:0/726969935 wait complete. 2024-05-09T00:04:41.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.721+0000 7f9b4448bb80 1 -- start start 2024-05-09T00:04:41.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.722+0000 7f9b4448bb80 1 Processor -- start 2024-05-09T00:04:41.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.722+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e65890 0x55db21e6ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.722+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6ae00 0x55db21e6f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.722+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e6b720 0x55db21e6f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.722+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21d51560 con 0x55db21e65890 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.722+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21d51200 con 0x55db21e6b720 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.722+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21c386a0 con 0x55db21e6ae00 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.722+0000 7f9b4448bb80 10 client.0 _ll_register_callbacks cb 0x55db21d5ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.723+0000 7f9b3606d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e65890 0x55db21e6ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.723+0000 7f9b3606d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e65890 0x55db21e6ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57918/0 (socket says 172.21.15.12:57918) 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.723+0000 7f9b3606d700 1 -- 172.21.15.12:0/899552968 learned_addr learned my addr 172.21.15.12:0/899552968 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.723+0000 7f9b3606d700 1 -- 172.21.15.12:0/899552968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6ae00 msgr2=0x55db21e6f150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.723+0000 7f9b3606d700 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6ae00 0x55db21e6f150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.723+0000 7f9b3606d700 1 -- 172.21.15.12:0/899552968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e6b720 msgr2=0x55db21e6f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.723+0000 7f9b3606d700 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e6b720 0x55db21e6f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.723+0000 7f9b3606d700 1 -- 172.21.15.12:0/899552968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b280099b0 con 0x55db21e65890 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.723+0000 7f9b3606d700 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e65890 0x55db21e6ec10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0cb6c0 tx=0x7f9b2c09b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5bf93dcd249bf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.723+0000 7f9b19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:04:41.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.724+0000 7f9b19ffb700 1 -- 172.21.15.12:0/899552968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0a3010 con 0x55db21e65890 2024-05-09T00:04:41.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.724+0000 7f9b19ffb700 1 -- 172.21.15.12:0/899552968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9b2c00fd20 con 0x55db21e65890 2024-05-09T00:04:41.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.724+0000 7f9b4448bb80 1 -- 172.21.15.12:0/899552968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db21e34810 con 0x55db21e65890 2024-05-09T00:04:41.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.724+0000 7f9b19ffb700 1 -- 172.21.15.12:0/899552968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b2c097330 con 0x55db21e65890 2024-05-09T00:04:41.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.724+0000 7f9b4448bb80 10 client.20556 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:41.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.724+0000 7f9b4448bb80 1 -- 172.21.15.12:0/899552968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55db21be2b60 con 0x55db21e65890 2024-05-09T00:04:41.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.725+0000 7f9b19ffb700 1 -- 172.21.15.12:0/899552968 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db21e34810 con 0x55db21e65890 2024-05-09T00:04:41.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.725+0000 7f9b19ffb700 1 -- 172.21.15.12:0/899552968 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9b2c09a110 con 0x55db21e65890 2024-05-09T00:04:41.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.725+0000 7f9b4448bb80 10 client.20556 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:41.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.725+0000 7f9b4448bb80 10 client.20556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:41.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.725+0000 7f9b4448bb80 20 client.20556 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:41.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.725+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db21e7ef70 0x55db21e81360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.725+0000 7f9b4448bb80 4 client.20556 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:41.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.725+0000 7f9b4448bb80 1 -- 172.21.15.12:0/899552968 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db21dadec0 con 0x55db21e7ef70 2024-05-09T00:04:41.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.726+0000 7f9b3586c700 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db21e7ef70 0x55db21e81360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.726+0000 7f9b19ffb700 10 client.20556 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:41.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.726+0000 7f9b3586c700 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db21e7ef70 0x55db21e81360 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7586221b9c3baf3e server_cookie=33cbc0b83c882cc in_seq=0 out_seq=0 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.728+0000 7f9b19ffb700 1 -- 172.21.15.12:0/899552968 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db21dadec0 con 0x55db21e7ef70 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.728+0000 7f9b19ffb700 10 client.20556 handle_command_reply: tid=0 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.728+0000 7f9b4448bb80 1 client.20556 shutdown 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.728+0000 7f9b4448bb80 1 -- 172.21.15.12:0/899552968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e65890 msgr2=0x55db21e6ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.728+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e65890 0x55db21e6ec10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0cb6c0 tx=0x7f9b2c09b3d0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.729+0000 7f9b4448bb80 1 -- 172.21.15.12:0/899552968 shutdown_connections 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.729+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db21e7ef70 0x55db21e81360 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.729+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e6b720 0x55db21e6f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.729+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e65890 0x55db21e6ec10 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.729+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/899552968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6ae00 0x55db21e6f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.729+0000 7f9b4448bb80 1 -- 172.21.15.12:0/899552968 >> 172.21.15.12:0/899552968 conn(0x55db21dc3210 msgr2=0x55db21e5f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.729+0000 7f9b19ffb700 0 client.20556 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:41.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.732+0000 7f9b4448bb80 1 -- 172.21.15.12:0/899552968 shutdown_connections 2024-05-09T00:04:41.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.732+0000 7f9b4448bb80 1 -- 172.21.15.12:0/899552968 wait complete. 2024-05-09T00:04:41.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.732+0000 7f9b4448bb80 20 client.20556 trim_cache size 0 max 16384 2024-05-09T00:04:41.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.752+0000 7f9b4448bb80 1 Processor -- start 2024-05-09T00:04:41.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.752+0000 7f9b4448bb80 1 -- start start 2024-05-09T00:04:41.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e65350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 0x55db21e5fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6b720 0x55db21e6f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21e61a30 con 0x55db21e6ae00 2024-05-09T00:04:41.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21e5cdc0 con 0x55db21e65890 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21e61e50 con 0x55db21e6b720 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b3586c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 0x55db21e5fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b3586c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 0x55db21e5fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57928/0 (socket says 172.21.15.12:57928) 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b3586c700 1 -- 172.21.15.12:0/894836069 learned_addr learned my addr 172.21.15.12:0/894836069 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b3606d700 1 --2- 172.21.15.12:0/894836069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e65350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b3686e700 1 --2- 172.21.15.12:0/894836069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6b720 0x55db21e6f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b3586c700 1 -- 172.21.15.12:0/894836069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6b720 msgr2=0x55db21e6f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b3586c700 1 --2- 172.21.15.12:0/894836069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6b720 0x55db21e6f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b3586c700 1 -- 172.21.15.12:0/894836069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 msgr2=0x55db21e65350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b3586c700 1 --2- 172.21.15.12:0/894836069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e65350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.753+0000 7f9b3586c700 1 -- 172.21.15.12:0/894836069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db21d7b560 con 0x55db21e6ae00 2024-05-09T00:04:41.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.754+0000 7f9b3586c700 1 --2- 172.21.15.12:0/894836069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 0x55db21e5fd20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9b30074d00 tx=0x7f9b20015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf20685d75039eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.754+0000 7f9b255a5700 1 -- 172.21.15.12:0/894836069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20023070 con 0x55db21e6ae00 2024-05-09T00:04:41.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.754+0000 7f9b255a5700 1 -- 172.21.15.12:0/894836069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2000ee00 con 0x55db21e6ae00 2024-05-09T00:04:41.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.754+0000 7f9b255a5700 1 -- 172.21.15.12:0/894836069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2001da20 con 0x55db21e6ae00 2024-05-09T00:04:41.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 -- 172.21.15.12:0/894836069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 msgr2=0x55db21e5fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/894836069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 0x55db21e5fd20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9b30074d00 tx=0x7f9b20015000 comp rx=0 tx=0).stop 2024-05-09T00:04:41.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 -- 172.21.15.12:0/894836069 shutdown_connections 2024-05-09T00:04:41.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/894836069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6b720 0x55db21e6f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/894836069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e65350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/894836069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 0x55db21e5fd20 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 -- 172.21.15.12:0/894836069 >> 172.21.15.12:0/894836069 conn(0x55db21dc3210 msgr2=0x55db21e6d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 -- 172.21.15.12:0/894836069 shutdown_connections 2024-05-09T00:04:41.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 -- 172.21.15.12:0/894836069 wait complete. 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 -- start start 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.755+0000 7f9b4448bb80 1 Processor -- start 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e74340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 0x55db21e74880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b4448bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6b720 0x55db21ed4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21e74f90 con 0x55db21e6ae00 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21ed49a0 con 0x55db21e65890 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b4448bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db21ed4b10 con 0x55db21e6b720 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b4448bb80 10 client.0 _ll_register_callbacks cb 0x55db21d5ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b3606d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e74340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b3606d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e74340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58418/0 (socket says 172.21.15.12:58418) 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b3606d700 1 -- 172.21.15.12:0/2065162061 learned_addr learned my addr 172.21.15.12:0/2065162061 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b3586c700 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 0x55db21e74880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b3686e700 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6b720 0x55db21ed4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b3606d700 1 -- 172.21.15.12:0/2065162061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6b720 msgr2=0x55db21ed4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b3606d700 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6b720 0x55db21ed4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b3606d700 1 -- 172.21.15.12:0/2065162061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 msgr2=0x55db21e74880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b3606d700 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 0x55db21e74880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.756+0000 7f9b3606d700 1 -- 172.21.15.12:0/2065162061 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2000e820 con 0x55db21e65890 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.757+0000 7f9b3606d700 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e74340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0a2ae0 tx=0x7f9b2c09a870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60346c5aa2d43a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.757+0000 7f9b19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.757+0000 7f9b19ffb700 1 -- 172.21.15.12:0/2065162061 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c09aca0 con 0x55db21e65890 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.757+0000 7f9b19ffb700 1 -- 172.21.15.12:0/2065162061 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9b2c09ae80 con 0x55db21e65890 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.757+0000 7f9b4448bb80 1 -- 172.21.15.12:0/2065162061 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db21d0e270 con 0x55db21e65890 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.757+0000 7f9b19ffb700 1 -- 172.21.15.12:0/2065162061 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b2c09b040 con 0x55db21e65890 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.758+0000 7f9b4448bb80 10 client.30143 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.758+0000 7f9b4448bb80 1 -- 172.21.15.12:0/2065162061 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55db21c3acd0 con 0x55db21e65890 2024-05-09T00:04:41.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.758+0000 7f9b19ffb700 1 -- 172.21.15.12:0/2065162061 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db21d0e270 con 0x55db21e65890 2024-05-09T00:04:41.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.758+0000 7f9b19ffb700 1 -- 172.21.15.12:0/2065162061 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9b2c060ad0 con 0x55db21e65890 2024-05-09T00:04:41.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.758+0000 7f9b4448bb80 10 client.30143 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:41.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.758+0000 7f9b4448bb80 10 client.30143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:41.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.758+0000 7f9b4448bb80 20 client.30143 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:41.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.758+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db21ed5e10 0x55db21ed8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:41.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.758+0000 7f9b4448bb80 4 client.30143 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:04:41.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.758+0000 7f9b4448bb80 1 -- 172.21.15.12:0/2065162061 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55db21c03cf0 con 0x55db21ed5e10 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.760+0000 7f9b3586c700 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db21ed5e10 0x55db21ed8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.761+0000 7f9b19ffb700 10 client.30143 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.761+0000 7f9b3586c700 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db21ed5e10 0x55db21ed8200 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3017dc981cf3d8d server_cookie=3bd9a8672e18651c in_seq=0 out_seq=0 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.762+0000 7f9b19ffb700 1 -- 172.21.15.12:0/2065162061 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55db21c03cf0 con 0x55db21ed5e10 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.762+0000 7f9b19ffb700 10 client.30143 handle_command_reply: tid=0 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 client.30143 shutdown 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 -- 172.21.15.12:0/2065162061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 msgr2=0x55db21e74340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e74340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0a2ae0 tx=0x7f9b2c09a870 comp rx=0 tx=0).stop 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 -- 172.21.15.12:0/2065162061 shutdown_connections 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db21e6b720 0x55db21ed4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55db21ed5e10 0x55db21ed8200 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db21e65890 0x55db21e74340 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 --2- 172.21.15.12:0/2065162061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db21e6ae00 0x55db21e74880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b19ffb700 0 client.30143 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 -- 172.21.15.12:0/2065162061 >> 172.21.15.12:0/2065162061 conn(0x55db21dc3210 msgr2=0x55db21e69c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 -- 172.21.15.12:0/2065162061 shutdown_connections 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 1 -- 172.21.15.12:0/2065162061 wait complete. 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.763+0000 7f9b4448bb80 20 client.30143 trim_cache size 0 max 16384 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (21 inodes in the stack)", 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "fc0b7a5e-d547-4e41-a7a0-b3452b6049a0": { 2024-05-09T00:04:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:04:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "fc0b7a5e-d547-4e41-a7a0-b3452b6049a0", 2024-05-09T00:04:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:04:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:04:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:04:41.760 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 -- 172.21.15.12:0/877501995 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b1c0b22a0 msgr2=0x7f9b1c0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b1c0b22a0 0x7f9b1c0b4760 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55db21e61bd0 tx=0x7f9b200058e0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 -- 172.21.15.12:0/877501995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b30101140 msgr2=0x7f9b300732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b30101140 0x7f9b300732a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00c010 tx=0x7f9b2c009570 comp rx=0 tx=0).stop 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 -- 172.21.15.12:0/877501995 shutdown_connections 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b1c0b22a0 0x7f9b1c0b4760 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30073d20 0x7f9b301a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b30101140 0x7f9b300732a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 --2- 172.21.15.12:0/877501995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30106260 0x7f9b300737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 -- 172.21.15.12:0/877501995 >> 172.21.15.12:0/877501995 conn(0x7f9b300fc9e0 msgr2=0x7f9b300fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 -- 172.21.15.12:0/877501995 shutdown_connections 2024-05-09T00:04:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:04:41.764+0000 7f9b382d1700 1 -- 172.21.15.12:0/877501995 wait complete. 2024-05-09T00:04:41.776 INFO:tasks.cephfs.filesystem:scrub status for tag:fc0b7a5e-d547-4e41-a7a0-b3452b6049a0 - {'path': '/', 'tag': 'fc0b7a5e-d547-4e41-a7a0-b3452b6049a0', 'options': 'recursive,force'} 2024-05-09T00:04:41.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:41 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:04:41.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:41 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:41.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:41 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:04:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:41 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:04:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:41 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:41 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:04:42.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:41 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:04:42.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:41 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:42.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:41 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:04:42.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 192844 36.35 MB/sec execute 157 sec latency 153.070 ms 2024-05-09T00:04:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:42 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:04:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:42 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:04:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:42 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:42 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:04:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:42 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:42 smithi012 ceph-mon[42503]: pgmap v589: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 571 op/s 2024-05-09T00:04:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:42 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:42.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:42 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:43.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:42 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:04:43.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:42 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:04:43.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:42 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:43.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:42 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:04:43.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:42 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:43.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:42 smithi204 ceph-mon[47373]: pgmap v589: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 571 op/s 2024-05-09T00:04:43.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:42 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:43.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:42 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:42 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:04:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:42 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:04:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:42 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:42 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:04:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:42 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:42 smithi160 ceph-mon[46152]: pgmap v589: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 571 op/s 2024-05-09T00:04:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:42 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:42 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:04:43.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 193444 36.33 MB/sec execute 158 sec latency 109.978 ms 2024-05-09T00:04:44.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:43 smithi204 ceph-mon[47373]: pgmap v590: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.7 MiB/s wr, 750 op/s 2024-05-09T00:04:44.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:43 smithi160 ceph-mon[46152]: pgmap v590: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.7 MiB/s wr, 750 op/s 2024-05-09T00:04:44.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:43 smithi012 ceph-mon[42503]: pgmap v590: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.7 MiB/s wr, 750 op/s 2024-05-09T00:04:44.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 194060 36.32 MB/sec execute 159 sec latency 105.835 ms 2024-05-09T00:04:45.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 194693 36.31 MB/sec execute 160 sec latency 122.217 ms 2024-05-09T00:04:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:45 smithi204 ceph-mon[47373]: pgmap v591: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 550 op/s 2024-05-09T00:04:46.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:45 smithi160 ceph-mon[46152]: pgmap v591: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 550 op/s 2024-05-09T00:04:46.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:45 smithi012 ceph-mon[42503]: pgmap v591: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 550 op/s 2024-05-09T00:04:46.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 195265 36.25 MB/sec execute 161 sec latency 186.763 ms 2024-05-09T00:04:47.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 195986 36.26 MB/sec execute 162 sec latency 98.534 ms 2024-05-09T00:04:48.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:47 smithi160 ceph-mon[46152]: pgmap v592: 97 pgs: 97 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.0 MiB/s wr, 680 op/s 2024-05-09T00:04:48.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:47 smithi204 ceph-mon[47373]: pgmap v592: 97 pgs: 97 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.0 MiB/s wr, 680 op/s 2024-05-09T00:04:48.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:47 smithi012 ceph-mon[42503]: pgmap v592: 97 pgs: 97 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.0 MiB/s wr, 680 op/s 2024-05-09T00:04:48.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 196696 36.27 MB/sec execute 163 sec latency 93.554 ms 2024-05-09T00:04:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:48 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:04:49.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:48 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:04:49.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:48 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:04:49.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 197444 36.34 MB/sec execute 164 sec latency 88.777 ms 2024-05-09T00:04:50.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:49 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:04:50.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:49 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:04:50.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:49 smithi012 ceph-mon[42503]: pgmap v593: 97 pgs: 97 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 476 op/s 2024-05-09T00:04:50.258 INFO:tasks.workunit.client.0.smithi012.stdout: 10 197704 36.23 MB/sec execute 165 sec latency 601.078 ms 2024-05-09T00:04:50.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:49 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:04:50.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:49 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:04:50.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:49 smithi160 ceph-mon[46152]: pgmap v593: 97 pgs: 97 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 476 op/s 2024-05-09T00:04:50.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:49 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:04:50.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:49 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:04:50.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:49 smithi204 ceph-mon[47373]: pgmap v593: 97 pgs: 97 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 476 op/s 2024-05-09T00:04:51.258 INFO:tasks.workunit.client.0.smithi012.stdout: 10 197777 36.02 MB/sec execute 166 sec latency 1209.448 ms 2024-05-09T00:04:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.802+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1507312714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bd4069ad0 msgr2=0x7f0bd410dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.802+0000 7f0bda2ce700 1 --2- 172.21.15.12:0/1507312714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bd4069ad0 0x7f0bd410dd90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00b0d0 tx=0x7f0bcc00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.802+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1507312714 shutdown_connections 2024-05-09T00:04:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.802+0000 7f0bda2ce700 1 --2- 172.21.15.12:0/1507312714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bd4069ad0 0x7f0bd410dd90 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.802+0000 7f0bda2ce700 1 --2- 172.21.15.12:0/1507312714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bd4106ed0 0x7f0bd41072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.802+0000 7f0bda2ce700 1 --2- 172.21.15.12:0/1507312714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd4069110 0x7f0bd4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.802+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1507312714 >> 172.21.15.12:0/1507312714 conn(0x7f0bd4076bc0 msgr2=0x7f0bd4076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.802+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1507312714 shutdown_connections 2024-05-09T00:04:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.802+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1507312714 wait complete. 2024-05-09T00:04:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bda2ce700 1 Processor -- start 2024-05-09T00:04:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bda2ce700 1 -- start start 2024-05-09T00:04:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bda2ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bd4069110 0x7f0bd41974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bda2ce700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bd4069ad0 0x7f0bd4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bda2ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd4106ed0 0x7f0bd419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bda2ce700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd419c3b0 con 0x7f0bd4069110 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bda2ce700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd419c520 con 0x7f0bd4106ed0 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bda2ce700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd419c6e0 con 0x7f0bd4069ad0 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bd4069110 0x7f0bd41974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bd4069110 0x7f0bd41974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58206/0 (socket says 172.21.15.12:58206) 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.803+0000 7f0bd3fff700 1 -- 172.21.15.12:0/1382601532 learned_addr learned my addr 172.21.15.12:0/1382601532 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd37fe700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bd4069ad0 0x7f0bd4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd3fff700 1 -- 172.21.15.12:0/1382601532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bd4069ad0 msgr2=0x7f0bd4197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd3fff700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bd4069ad0 0x7f0bd4197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd3fff700 1 -- 172.21.15.12:0/1382601532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd4106ed0 msgr2=0x7f0bd419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd3fff700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd4106ed0 0x7f0bd419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd3fff700 1 -- 172.21.15.12:0/1382601532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc009d00 con 0x7f0bd4069110 2024-05-09T00:04:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd37fe700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bd4069ad0 0x7f0bd4197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:04:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd3fff700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bd4069110 0x7f0bd41974e0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400e9c0 tx=0x7f0bc400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd17fa700 1 -- 172.21.15.12:0/1382601532 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40097b0 con 0x7f0bd4069110 2024-05-09T00:04:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1382601532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd419c9c0 con 0x7f0bd4069110 2024-05-09T00:04:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd17fa700 1 -- 172.21.15.12:0/1382601532 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc4004cf0 con 0x7f0bd4069110 2024-05-09T00:04:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bd17fa700 1 -- 172.21.15.12:0/1382601532 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4005600 con 0x7f0bd4069110 2024-05-09T00:04:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.804+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1382601532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd4100280 con 0x7f0bd4069110 2024-05-09T00:04:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.807+0000 7f0bd17fa700 1 -- 172.21.15.12:0/1382601532 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0bc400cc60 con 0x7f0bd4069110 2024-05-09T00:04:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.808+0000 7f0bd17fa700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bbc0a56a0 0x7f0bbc0a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.808+0000 7f0bd37fe700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bbc0a56a0 0x7f0bbc0a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.808+0000 7f0bd17fa700 1 -- 172.21.15.12:0/1382601532 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0bc4014070 con 0x7f0bd4069110 2024-05-09T00:04:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.809+0000 7f0bd37fe700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bbc0a56a0 0x7f0bbc0a7b60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0bd41989e0 tx=0x7f0bc8009190 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.809+0000 7f0be6488b80 1 Processor -- start 2024-05-09T00:04:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.809+0000 7f0be6488b80 1 -- start start 2024-05-09T00:04:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.809+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eabfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eaba480 0x55645eaba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0be6488b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645ea110f0 con 0x55645eac0270 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0be6488b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645ea0eab0 con 0x55645eaba480 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0be6488b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645ea0a3a0 con 0x55645eabf950 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58216/0 (socket says 172.21.15.12:58216) 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bd37fe700 1 -- 172.21.15.12:0/3624743142 learned_addr learned my addr 172.21.15.12:0/3624743142 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bd3fff700 1 --2- 172.21.15.12:0/3624743142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eabfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bd37fe700 1 -- 172.21.15.12:0/3624743142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 msgr2=0x55645eabfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bd37fe700 1 --2- 172.21.15.12:0/3624743142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eabfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bd37fe700 1 -- 172.21.15.12:0/3624743142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eaba480 msgr2=0x55645eaba8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bd37fe700 1 --2- 172.21.15.12:0/3624743142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eaba480 0x55645eaba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bd37fe700 1 -- 172.21.15.12:0/3624743142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645e88b350 con 0x55645eac0270 2024-05-09T00:04:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bd37fe700 1 --2- 172.21.15.12:0/3624743142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab9f40 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800ee20 tx=0x7f0bc8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb342023e9871755 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.810+0000 7f0bba7fc700 1 -- 172.21.15.12:0/3624743142 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8015910 con 0x55645eac0270 2024-05-09T00:04:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.811+0000 7f0bba7fc700 1 -- 172.21.15.12:0/3624743142 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc8013450 con 0x55645eac0270 2024-05-09T00:04:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.811+0000 7f0bba7fc700 1 -- 172.21.15.12:0/3624743142 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc801f650 con 0x55645eac0270 2024-05-09T00:04:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.812+0000 7f0be6488b80 1 -- 172.21.15.12:0/3624743142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 msgr2=0x55645eab9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.812+0000 7f0be6488b80 1 --2- 172.21.15.12:0/3624743142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab9f40 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800ee20 tx=0x7f0bc8011000 comp rx=0 tx=0).stop 2024-05-09T00:04:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.812+0000 7f0be6488b80 1 -- 172.21.15.12:0/3624743142 shutdown_connections 2024-05-09T00:04:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.812+0000 7f0be6488b80 1 --2- 172.21.15.12:0/3624743142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eaba480 0x55645eaba8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.812+0000 7f0be6488b80 1 --2- 172.21.15.12:0/3624743142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eabfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.812+0000 7f0be6488b80 1 --2- 172.21.15.12:0/3624743142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab9f40 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.812+0000 7f0be6488b80 1 -- 172.21.15.12:0/3624743142 >> 172.21.15.12:0/3624743142 conn(0x55645ea181b0 msgr2=0x55645eab4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.812+0000 7f0be6488b80 1 -- 172.21.15.12:0/3624743142 shutdown_connections 2024-05-09T00:04:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.812+0000 7f0be6488b80 1 -- 172.21.15.12:0/3624743142 wait complete. 2024-05-09T00:04:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.812+0000 7f0be6488b80 1 -- start start 2024-05-09T00:04:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.813+0000 7f0be6488b80 1 Processor -- start 2024-05-09T00:04:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eaba480 0x55645eac6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eac3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 0x55645eac3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0be6488b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645e9a6500 con 0x55645eaba480 2024-05-09T00:04:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0be6488b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645e9a61a0 con 0x55645eac0270 2024-05-09T00:04:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0be6488b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645e88d640 con 0x55645eabf950 2024-05-09T00:04:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0be6488b80 10 client.0 _ll_register_callbacks cb 0x55645e9afad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eac3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eaba480 0x55645eac6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eac3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42406/0 (socket says 172.21.15.12:42406) 2024-05-09T00:04:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.815+0000 7f0bd37fe700 1 -- 172.21.15.12:0/753911180 learned_addr learned my addr 172.21.15.12:0/753911180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.816+0000 7f0bd886b700 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 0x55645eac3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.816+0000 7f0bd37fe700 1 -- 172.21.15.12:0/753911180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 msgr2=0x55645eac3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.816+0000 7f0bd37fe700 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 0x55645eac3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.816+0000 7f0bd37fe700 1 -- 172.21.15.12:0/753911180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eaba480 msgr2=0x55645eac6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.816+0000 7f0bd37fe700 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eaba480 0x55645eac6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.816+0000 7f0bd37fe700 1 -- 172.21.15.12:0/753911180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc800ead0 con 0x55645eabf950 2024-05-09T00:04:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.816+0000 7f0bd37fe700 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eac3620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800edf0 tx=0x7f0bc80046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd9052cec61afe89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.816+0000 7f0bb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:04:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.816+0000 7f0bb37fe700 1 -- 172.21.15.12:0/753911180 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8015d20 con 0x55645eabf950 2024-05-09T00:04:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.816+0000 7f0be6488b80 1 -- 172.21.15.12:0/753911180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55645e963210 con 0x55645eabf950 2024-05-09T00:04:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.817+0000 7f0bb37fe700 1 -- 172.21.15.12:0/753911180 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0bc801f3f0 con 0x55645eabf950 2024-05-09T00:04:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.817+0000 7f0bb37fe700 1 -- 172.21.15.12:0/753911180 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bc8015e80 con 0x55645eabf950 2024-05-09T00:04:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.818+0000 7f0be6488b80 10 client.30292 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.818+0000 7f0be6488b80 1 -- 172.21.15.12:0/753911180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55645e837b00 con 0x55645eabf950 2024-05-09T00:04:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.819+0000 7f0bb37fe700 1 -- 172.21.15.12:0/753911180 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55645e963210 con 0x55645eabf950 2024-05-09T00:04:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.819+0000 7f0bb37fe700 1 -- 172.21.15.12:0/753911180 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0bc803a670 con 0x55645eabf950 2024-05-09T00:04:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.819+0000 7f0be6488b80 10 client.30292 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.819+0000 7f0be6488b80 10 client.30292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.819+0000 7f0be6488b80 20 client.30292 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.820+0000 7f0be6488b80 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55645eae4b30 0x55645eae6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.820+0000 7f0be6488b80 4 client.30292 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:04:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.820+0000 7f0be6488b80 1 -- 172.21.15.12:0/753911180 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55645ea897b0 con 0x55645eae4b30 2024-05-09T00:04:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.820+0000 7f0bd3fff700 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55645eae4b30 0x55645eae6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.820+0000 7f0bb37fe700 10 client.30292 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.820+0000 7f0bd3fff700 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55645eae4b30 0x55645eae6f50 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b71025122dc78fd server_cookie=99cd99242ace5b5c in_seq=0 out_seq=0 2024-05-09T00:04:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0bb37fe700 1 -- 172.21.15.12:0/753911180 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55645ea897b0 con 0x55645eae4b30 2024-05-09T00:04:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0bb37fe700 10 client.30292 handle_command_reply: tid=0 2024-05-09T00:04:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0be6488b80 1 client.30292 shutdown 2024-05-09T00:04:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0be6488b80 1 -- 172.21.15.12:0/753911180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 msgr2=0x55645eac3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0be6488b80 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eac3620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800edf0 tx=0x7f0bc80046d0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0be6488b80 1 -- 172.21.15.12:0/753911180 shutdown_connections 2024-05-09T00:04:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0be6488b80 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55645eae4b30 0x55645eae6f50 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0be6488b80 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 0x55645eac3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0be6488b80 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eaba480 0x55645eac6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0be6488b80 1 --2- 172.21.15.12:0/753911180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eac3620 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0be6488b80 1 -- 172.21.15.12:0/753911180 >> 172.21.15.12:0/753911180 conn(0x55645ea181b0 msgr2=0x55645eabd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.830+0000 7f0bb37fe700 0 client.30292 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.831+0000 7f0be6488b80 1 -- 172.21.15.12:0/753911180 shutdown_connections 2024-05-09T00:04:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.831+0000 7f0be6488b80 1 -- 172.21.15.12:0/753911180 wait complete. 2024-05-09T00:04:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.831+0000 7f0be6488b80 20 client.30292 trim_cache size 0 max 16384 2024-05-09T00:04:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.859+0000 7f0be6488b80 1 Processor -- start 2024-05-09T00:04:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.859+0000 7f0be6488b80 1 -- start start 2024-05-09T00:04:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.859+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eaba480 0x55645eacf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.859+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eabf950 0x55645eab4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.859+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.859+0000 7f0be6488b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645eab1340 con 0x55645eac0270 2024-05-09T00:04:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.859+0000 7f0be6488b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645eac4b80 con 0x55645eabf950 2024-05-09T00:04:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.859+0000 7f0be6488b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645eab5680 con 0x55645eaba480 2024-05-09T00:04:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.859+0000 7f0bd886b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.860+0000 7f0bd886b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58234/0 (socket says 172.21.15.12:58234) 2024-05-09T00:04:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.860+0000 7f0bd886b700 1 -- 172.21.15.12:0/3728585367 learned_addr learned my addr 172.21.15.12:0/3728585367 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.860+0000 7f0bd886b700 1 -- 172.21.15.12:0/3728585367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eaba480 msgr2=0x55645eacf6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.860+0000 7f0bd886b700 1 --2- 172.21.15.12:0/3728585367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eaba480 0x55645eacf6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.860+0000 7f0bd886b700 1 -- 172.21.15.12:0/3728585367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eabf950 msgr2=0x55645eab4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.860+0000 7f0bd886b700 1 --2- 172.21.15.12:0/3728585367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eabf950 0x55645eab4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.860+0000 7f0bd886b700 1 -- 172.21.15.12:0/3728585367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645e9d0500 con 0x55645eac0270 2024-05-09T00:04:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.861+0000 7f0bd886b700 1 --2- 172.21.15.12:0/3728585367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab0e00 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009330 tx=0x7f0bcc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec82c9454ad041ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.861+0000 7f0bbaffd700 1 -- 172.21.15.12:0/3728585367 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00e070 con 0x55645eac0270 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.861+0000 7f0bbaffd700 1 -- 172.21.15.12:0/3728585367 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bcc007d00 con 0x55645eac0270 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.861+0000 7f0bbaffd700 1 -- 172.21.15.12:0/3728585367 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc003c60 con 0x55645eac0270 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.862+0000 7f0be6488b80 1 -- 172.21.15.12:0/3728585367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 msgr2=0x55645eab0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.862+0000 7f0be6488b80 1 --2- 172.21.15.12:0/3728585367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab0e00 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009330 tx=0x7f0bcc009360 comp rx=0 tx=0).stop 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.862+0000 7f0be6488b80 1 -- 172.21.15.12:0/3728585367 shutdown_connections 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.862+0000 7f0be6488b80 1 --2- 172.21.15.12:0/3728585367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eac0270 0x55645eab0e00 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.862+0000 7f0be6488b80 1 --2- 172.21.15.12:0/3728585367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eaba480 0x55645eacf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.862+0000 7f0be6488b80 1 --2- 172.21.15.12:0/3728585367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eabf950 0x55645eab4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.862+0000 7f0be6488b80 1 -- 172.21.15.12:0/3728585367 >> 172.21.15.12:0/3728585367 conn(0x55645ea181b0 msgr2=0x55645eac2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.862+0000 7f0be6488b80 1 -- 172.21.15.12:0/3728585367 shutdown_connections 2024-05-09T00:04:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.862+0000 7f0be6488b80 1 -- 172.21.15.12:0/3728585367 wait complete. 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.863+0000 7f0be6488b80 1 -- start start 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.863+0000 7f0be6488b80 1 Processor -- start 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.864+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eaba480 0x55645eac6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.864+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eac7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.864+0000 7f0be6488b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 0x55645eb427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.864+0000 7f0be6488b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645eac7920 con 0x55645eaba480 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.864+0000 7f0be6488b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645eac7a90 con 0x55645eac0270 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.864+0000 7f0be6488b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645eb33020 con 0x55645eabf950 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.864+0000 7f0be6488b80 10 client.0 _ll_register_callbacks cb 0x55645e9afad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.865+0000 7f0bd886b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 0x55645eb427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.865+0000 7f0bd886b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 0x55645eb427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:36756/0 (socket says 172.21.15.12:36756) 2024-05-09T00:04:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.865+0000 7f0bd886b700 1 -- 172.21.15.12:0/2688784113 learned_addr learned my addr 172.21.15.12:0/2688784113 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:04:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.866+0000 7f0bd886b700 1 -- 172.21.15.12:0/2688784113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 msgr2=0x55645eac7120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:04:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.866+0000 7f0bd886b700 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eac7120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.866+0000 7f0bd886b700 1 -- 172.21.15.12:0/2688784113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eaba480 msgr2=0x55645eac6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.866+0000 7f0bd886b700 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eaba480 0x55645eac6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.866+0000 7f0bd886b700 1 -- 172.21.15.12:0/2688784113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd410e690 con 0x55645eac0270 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.866+0000 7f0bd886b700 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 0x55645eb427f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009a20 tx=0x7f0bcc00bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f71ef996c8b03a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.868+0000 7f0bb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.868+0000 7f0bb37fe700 1 -- 172.21.15.12:0/2688784113 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00e040 con 0x55645eac0270 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.868+0000 7f0bb37fe700 1 -- 172.21.15.12:0/2688784113 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0bcc003c10 con 0x55645eac0270 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.868+0000 7f0be6488b80 1 -- 172.21.15.12:0/2688784113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bc8013450 con 0x55645eac0270 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.868+0000 7f0bb37fe700 1 -- 172.21.15.12:0/2688784113 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bcc003dd0 con 0x55645eac0270 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.868+0000 7f0be6488b80 10 client.30149 fetch_fsmap learned FSMap version 74 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.868+0000 7f0be6488b80 1 -- 172.21.15.12:0/2688784113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55645e88fc70 con 0x55645eac0270 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.868+0000 7f0bb37fe700 1 -- 172.21.15.12:0/2688784113 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc8013450 con 0x55645eac0270 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.868+0000 7f0bb37fe700 1 -- 172.21.15.12:0/2688784113 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0bcc019030 con 0x55645eac0270 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.869+0000 7f0be6488b80 10 client.30149 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.869+0000 7f0be6488b80 10 client.30149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:04:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.869+0000 7f0be6488b80 20 client.30149 populate_metadata read hostname 'smithi012' 2024-05-09T00:04:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.869+0000 7f0be6488b80 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55645eb34470 0x55645eb368d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.869+0000 7f0be6488b80 4 client.30149 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:04:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.869+0000 7f0be6488b80 1 -- 172.21.15.12:0/2688784113 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55645ea02e60 con 0x55645eb34470 2024-05-09T00:04:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.869+0000 7f0bd3fff700 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55645eb34470 0x55645eb368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.886+0000 7f0bb37fe700 10 client.30149 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:51.886+0000 7f0bd3fff700 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55645eb34470 0x55645eb368d0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42019375e322b08 server_cookie=4571ba97455c827a in_seq=0 out_seq=0 2024-05-09T00:04:52.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:51 smithi204 ceph-mon[47373]: pgmap v594: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.0 MiB/s wr, 533 op/s 2024-05-09T00:04:52.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:51 smithi160 ceph-mon[46152]: pgmap v594: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.0 MiB/s wr, 533 op/s 2024-05-09T00:04:52.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:51 smithi012 ceph-mon[42503]: pgmap v594: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.0 MiB/s wr, 533 op/s 2024-05-09T00:04:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.190+0000 7f0bb37fe700 1 -- 172.21.15.12:0/2688784113 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55645ea02e60 con 0x55645eb34470 2024-05-09T00:04:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.190+0000 7f0bb37fe700 10 client.30149 handle_command_reply: tid=0 2024-05-09T00:04:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.193+0000 7f0be6488b80 1 client.30149 shutdown 2024-05-09T00:04:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.199+0000 7f0be6488b80 1 -- 172.21.15.12:0/2688784113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 msgr2=0x55645eb427f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.199+0000 7f0be6488b80 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 0x55645eb427f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009a20 tx=0x7f0bcc00bd80 comp rx=0 tx=0).stop 2024-05-09T00:04:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.199+0000 7f0be6488b80 1 -- 172.21.15.12:0/2688784113 shutdown_connections 2024-05-09T00:04:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.199+0000 7f0be6488b80 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55645eb34470 0x55645eb368d0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.199+0000 7f0be6488b80 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55645eac0270 0x55645eb427f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.199+0000 7f0be6488b80 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55645eaba480 0x55645eac6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.199+0000 7f0be6488b80 1 --2- 172.21.15.12:0/2688784113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55645eabf950 0x55645eac7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.199+0000 7f0be6488b80 1 -- 172.21.15.12:0/2688784113 >> 172.21.15.12:0/2688784113 conn(0x55645ea181b0 msgr2=0x55645eab86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.199+0000 7f0bb37fe700 0 client.30149 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:04:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.199+0000 7f0be6488b80 1 -- 172.21.15.12:0/2688784113 shutdown_connections 2024-05-09T00:04:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.200+0000 7f0be6488b80 1 -- 172.21.15.12:0/2688784113 wait complete. 2024-05-09T00:04:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.200+0000 7f0be6488b80 20 client.30149 trim_cache size 0 max 16384 2024-05-09T00:04:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:04:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:04:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:04:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:04:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.204+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1382601532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bbc0a56a0 msgr2=0x7f0bbc0a7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.204+0000 7f0bda2ce700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bbc0a56a0 0x7f0bbc0a7b60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0bd41989e0 tx=0x7f0bc8009190 comp rx=0 tx=0).stop 2024-05-09T00:04:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.204+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1382601532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bd4069110 msgr2=0x7f0bd41974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.204+0000 7f0bda2ce700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bd4069110 0x7f0bd41974e0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400e9c0 tx=0x7f0bc400ecd0 comp rx=0 tx=0).stop 2024-05-09T00:04:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.206+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1382601532 shutdown_connections 2024-05-09T00:04:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.206+0000 7f0bda2ce700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bbc0a56a0 0x7f0bbc0a7b60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.206+0000 7f0bda2ce700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bd4106ed0 0x7f0bd419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.206+0000 7f0bda2ce700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bd4069110 0x7f0bd41974e0 secure :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400e9c0 tx=0x7f0bc400ecd0 comp rx=0 tx=0).stop 2024-05-09T00:04:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.206+0000 7f0bda2ce700 1 --2- 172.21.15.12:0/1382601532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bd4069ad0 0x7f0bd4197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.206+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1382601532 >> 172.21.15.12:0/1382601532 conn(0x7f0bd4076bc0 msgr2=0x7f0bd40fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.209+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1382601532 shutdown_connections 2024-05-09T00:04:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:04:52.212+0000 7f0bda2ce700 1 -- 172.21.15.12:0/1382601532 wait complete. 2024-05-09T00:04:52.259 INFO:tasks.workunit.client.0.smithi012.stdout: 10 198072 35.88 MB/sec execute 167 sec latency 198.871 ms 2024-05-09T00:04:53.259 INFO:tasks.workunit.client.0.smithi012.stdout: 10 198498 35.78 MB/sec execute 168 sec latency 94.690 ms 2024-05-09T00:04:54.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:53 smithi204 ceph-mon[47373]: pgmap v595: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 19 MiB/s wr, 844 op/s 2024-05-09T00:04:54.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:53 smithi160 ceph-mon[46152]: pgmap v595: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 19 MiB/s wr, 844 op/s 2024-05-09T00:04:54.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:53 smithi012 ceph-mon[42503]: pgmap v595: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 19 MiB/s wr, 844 op/s 2024-05-09T00:04:54.259 INFO:tasks.workunit.client.0.smithi012.stdout: 10 199086 35.79 MB/sec execute 169 sec latency 137.398 ms 2024-05-09T00:04:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:54 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:55.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:54 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:55.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:54 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:55.259 INFO:tasks.workunit.client.0.smithi012.stdout: 10 199626 35.67 MB/sec execute 170 sec latency 105.002 ms 2024-05-09T00:04:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:55 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:56.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:55 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:56.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:55 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:04:56.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:55 smithi204 ceph-mon[47373]: pgmap v596: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 17 MiB/s wr, 659 op/s 2024-05-09T00:04:56.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:55 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:56.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:55 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:56.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:55 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:04:56.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:55 smithi160 ceph-mon[46152]: pgmap v596: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 17 MiB/s wr, 659 op/s 2024-05-09T00:04:56.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:55 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:56.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:55 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:56.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:55 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:04:56.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:55 smithi012 ceph-mon[42503]: pgmap v596: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 17 MiB/s wr, 659 op/s 2024-05-09T00:04:56.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 200417 35.70 MB/sec execute 171 sec latency 75.224 ms 2024-05-09T00:04:57.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 201273 35.88 MB/sec execute 172 sec latency 94.376 ms 2024-05-09T00:04:58.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:57 smithi204 ceph-mon[47373]: pgmap v597: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 18 MiB/s wr, 863 op/s 2024-05-09T00:04:58.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:57 smithi160 ceph-mon[46152]: pgmap v597: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 18 MiB/s wr, 863 op/s 2024-05-09T00:04:58.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:57 smithi012 ceph-mon[42503]: pgmap v597: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 18 MiB/s wr, 863 op/s 2024-05-09T00:04:58.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 201801 35.82 MB/sec execute 173 sec latency 88.318 ms 2024-05-09T00:04:59.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 202338 35.75 MB/sec execute 174 sec latency 102.060 ms 2024-05-09T00:05:00.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:04:59 smithi204 ceph-mon[47373]: pgmap v598: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 17 MiB/s wr, 762 op/s 2024-05-09T00:05:00.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:04:59 smithi160 ceph-mon[46152]: pgmap v598: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 17 MiB/s wr, 762 op/s 2024-05-09T00:05:00.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:04:59 smithi012 ceph-mon[42503]: pgmap v598: 97 pgs: 97 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 17 MiB/s wr, 762 op/s 2024-05-09T00:05:00.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 202867 35.69 MB/sec execute 175 sec latency 130.802 ms 2024-05-09T00:05:01.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 202867 35.49 MB/sec execute 176 sec latency 1130.926 ms 2024-05-09T00:05:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:01 smithi204 ceph-mon[47373]: pgmap v599: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 758 op/s 2024-05-09T00:05:02.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:01 smithi160 ceph-mon[46152]: pgmap v599: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 758 op/s 2024-05-09T00:05:02.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:01 smithi012 ceph-mon[42503]: pgmap v599: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 758 op/s 2024-05-09T00:05:02.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 202867 35.29 MB/sec execute 177 sec latency 2131.039 ms 2024-05-09T00:05:03.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 203112 35.15 MB/sec execute 178 sec latency 2570.574 ms 2024-05-09T00:05:04.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:03 smithi204 ceph-mon[47373]: pgmap v600: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 804 op/s 2024-05-09T00:05:04.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:03 smithi160 ceph-mon[46152]: pgmap v600: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 804 op/s 2024-05-09T00:05:04.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:03 smithi012 ceph-mon[42503]: pgmap v600: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 804 op/s 2024-05-09T00:05:04.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 203597 35.02 MB/sec execute 179 sec latency 125.769 ms 2024-05-09T00:05:05.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 203879 34.93 MB/sec execute 180 sec latency 171.048 ms 2024-05-09T00:05:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.372+0000 7f939d432700 1 -- 172.21.15.12:0/974505628 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400fe90 con 0x7f9398100780 2024-05-09T00:05:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.374+0000 7f93a0698700 1 -- 172.21.15.12:0/974505628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9398100780 msgr2=0x7f9398104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.374+0000 7f93a0698700 1 --2- 172.21.15.12:0/974505628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9398100780 0x7f9398104c40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f93940098f0 tx=0x7f9394009c00 comp rx=0 tx=0).stop 2024-05-09T00:05:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.374+0000 7f93a0698700 1 -- 172.21.15.12:0/974505628 shutdown_connections 2024-05-09T00:05:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.374+0000 7f93a0698700 1 --2- 172.21.15.12:0/974505628 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9398105260 0x7f9398105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.374+0000 7f93a0698700 1 --2- 172.21.15.12:0/974505628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93980ffe60 0x7f9398100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.374+0000 7f93a0698700 1 --2- 172.21.15.12:0/974505628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9398100780 0x7f9398104c40 secure :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f93940098f0 tx=0x7f9394009c00 comp rx=0 tx=0).stop 2024-05-09T00:05:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.374+0000 7f93a0698700 1 -- 172.21.15.12:0/974505628 >> 172.21.15.12:0/974505628 conn(0x7f93980fb870 msgr2=0x7f93980fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.378+0000 7f93a0698700 1 -- 172.21.15.12:0/974505628 shutdown_connections 2024-05-09T00:05:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.379+0000 7f93a0698700 1 -- 172.21.15.12:0/974505628 wait complete. 2024-05-09T00:05:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.379+0000 7f93a0698700 1 Processor -- start 2024-05-09T00:05:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.379+0000 7f93a0698700 1 -- start start 2024-05-09T00:05:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.380+0000 7f93a0698700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93980ffe60 0x7f93981a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.380+0000 7f93a0698700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9398105260 0x7f93981a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.380+0000 7f93a0698700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939819d9e0 0x7f939819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.380+0000 7f93a0698700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939819e460 con 0x7f9398105260 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.380+0000 7f93a0698700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939819e5d0 con 0x7f939819d9e0 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.380+0000 7f93a0698700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939819e790 con 0x7f93980ffe60 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.380+0000 7f939ec35700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939819d9e0 0x7f939819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.380+0000 7f939ec35700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939819d9e0 0x7f939819de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33906/0 (socket says 172.21.15.12:33906) 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.380+0000 7f939ec35700 1 -- 172.21.15.12:0/3156285507 learned_addr learned my addr 172.21.15.12:0/3156285507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.380+0000 7f939dc33700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9398105260 0x7f93981a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.381+0000 7f939dc33700 1 -- 172.21.15.12:0/3156285507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93980ffe60 msgr2=0x7f93981a38b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.381+0000 7f939dc33700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93980ffe60 0x7f93981a38b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.381+0000 7f939dc33700 1 -- 172.21.15.12:0/3156285507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939819d9e0 msgr2=0x7f939819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.381+0000 7f939dc33700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939819d9e0 0x7f939819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.381+0000 7f939dc33700 1 -- 172.21.15.12:0/3156285507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880095f0 con 0x7f9398105260 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.381+0000 7f939dc33700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9398105260 0x7f93981a3df0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f9394008000 tx=0x7f9394010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.382+0000 7f938f7fe700 1 -- 172.21.15.12:0/3156285507 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400b7e0 con 0x7f9398105260 2024-05-09T00:05:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.382+0000 7f938f7fe700 1 -- 172.21.15.12:0/3156285507 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f939403e070 con 0x7f9398105260 2024-05-09T00:05:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.382+0000 7f938f7fe700 1 -- 172.21.15.12:0/3156285507 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394003940 con 0x7f9398105260 2024-05-09T00:05:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.383+0000 7f93a0698700 1 -- 172.21.15.12:0/3156285507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9394009520 con 0x7f9398105260 2024-05-09T00:05:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.384+0000 7f93a0698700 1 -- 172.21.15.12:0/3156285507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939806ccd0 con 0x7f9398105260 2024-05-09T00:05:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.388+0000 7f938f7fe700 1 -- 172.21.15.12:0/3156285507 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9394040070 con 0x7f9398105260 2024-05-09T00:05:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.390+0000 7f93ac852b80 1 Processor -- start 2024-05-09T00:05:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.390+0000 7f93ac852b80 1 -- start start 2024-05-09T00:05:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.390+0000 7f938f7fe700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93840a1310 0x7f93840a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.390+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e735952d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.390+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e73594cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.390+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e73594d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.390+0000 7f93ac852b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7358a40f0 con 0x55e73594d480 2024-05-09T00:05:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.390+0000 7f93ac852b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7358a1ab0 con 0x55e735953270 2024-05-09T00:05:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.390+0000 7f93ac852b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73589d3a0 con 0x55e735952950 2024-05-09T00:05:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.390+0000 7f938f7fe700 1 -- 172.21.15.12:0/3156285507 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f93940ef540 con 0x7f9398105260 2024-05-09T00:05:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939ec35700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93840a1310 0x7f93840a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939dc33700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e73594cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939ec35700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e73594d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939ec35700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e73594d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53626/0 (socket says 172.21.15.12:53626) 2024-05-09T00:05:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939dc33700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e73594cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33916/0 (socket says 172.21.15.12:33916) 2024-05-09T00:05:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939ec35700 1 -- 172.21.15.12:0/2813701804 learned_addr learned my addr 172.21.15.12:0/2813701804 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939e434700 1 --2- 172.21.15.12:0/2813701804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e735952d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939dc33700 1 -- 172.21.15.12:0/2813701804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 msgr2=0x55e735952d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939dc33700 1 --2- 172.21.15.12:0/2813701804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e735952d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939dc33700 1 -- 172.21.15.12:0/2813701804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 msgr2=0x55e73594d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939dc33700 1 --2- 172.21.15.12:0/2813701804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e73594d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.391+0000 7f939dc33700 1 -- 172.21.15.12:0/2813701804 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e73571e350 con 0x55e735953270 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.392+0000 7f939e434700 1 --2- 172.21.15.12:0/2813701804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e735952d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.392+0000 7f939ec35700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93840a1310 0x7f93840a37d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9390007910 tx=0x7f9390009d50 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.392+0000 7f939dc33700 1 --2- 172.21.15.12:0/2813701804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e73594cf40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f93940ecca0 tx=0x7f93940bc4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40ac58a7e3309fda server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.392+0000 7f938cff9700 1 -- 172.21.15.12:0/2813701804 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940dc110 con 0x55e735953270 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.392+0000 7f938cff9700 1 -- 172.21.15.12:0/2813701804 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93940bcdf0 con 0x55e735953270 2024-05-09T00:05:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.392+0000 7f938cff9700 1 -- 172.21.15.12:0/2813701804 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940bcbb0 con 0x55e735953270 2024-05-09T00:05:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.396+0000 7f93ac852b80 1 -- 172.21.15.12:0/2813701804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 msgr2=0x55e73594cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.396+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2813701804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e73594cf40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f93940ecca0 tx=0x7f93940bc4c0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.397+0000 7f93ac852b80 1 -- 172.21.15.12:0/2813701804 shutdown_connections 2024-05-09T00:05:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.397+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2813701804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e73594d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.397+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2813701804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e735952d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.397+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2813701804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e73594cf40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.397+0000 7f93ac852b80 1 -- 172.21.15.12:0/2813701804 >> 172.21.15.12:0/2813701804 conn(0x55e7358ab1b0 msgr2=0x55e735947840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.397+0000 7f93ac852b80 1 -- 172.21.15.12:0/2813701804 shutdown_connections 2024-05-09T00:05:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.399+0000 7f93ac852b80 1 -- 172.21.15.12:0/2813701804 wait complete. 2024-05-09T00:05:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.399+0000 7f93ac852b80 1 -- start start 2024-05-09T00:05:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.399+0000 7f93ac852b80 1 Processor -- start 2024-05-09T00:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.400+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e73594d480 0x55e7359592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.400+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e7359565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.400+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e735953270 0x55e735956d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.400+0000 7f93ac852b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e735839500 con 0x55e735953270 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.400+0000 7f93ac852b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7358391a0 con 0x55e73594d480 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.400+0000 7f93ac852b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e735720640 con 0x55e735952950 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.400+0000 7f93ac852b80 10 client.0 _ll_register_callbacks cb 0x55e735842ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.402+0000 7f939dc33700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e7359565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.402+0000 7f939dc33700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e7359565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33708/0 (socket says 172.21.15.12:33708) 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.402+0000 7f939dc33700 1 -- 172.21.15.12:0/2420909122 learned_addr learned my addr 172.21.15.12:0/2420909122 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.402+0000 7f939e434700 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e73594d480 0x55e7359592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.402+0000 7f939ec35700 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e735953270 0x55e735956d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.402+0000 7f939dc33700 1 -- 172.21.15.12:0/2420909122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e73594d480 msgr2=0x55e7359592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.402+0000 7f939dc33700 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e73594d480 0x55e7359592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.402+0000 7f939dc33700 1 -- 172.21.15.12:0/2420909122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e735953270 msgr2=0x55e735956d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.402+0000 7f939dc33700 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e735953270 0x55e735956d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.402+0000 7f939dc33700 1 -- 172.21.15.12:0/2420909122 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939806ccd0 con 0x55e735952950 2024-05-09T00:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.405+0000 7f939dc33700 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e7359565c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f939400bd50 tx=0x7f93940c03b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f647c5686ac75877 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.405+0000 7f9381ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:05:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.410+0000 7f9381ffb700 1 -- 172.21.15.12:0/2420909122 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940bc6f0 con 0x55e735952950 2024-05-09T00:05:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.410+0000 7f9381ffb700 1 -- 172.21.15.12:0/2420909122 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f93940c0520 con 0x55e735952950 2024-05-09T00:05:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.410+0000 7f93ac852b80 1 -- 172.21.15.12:0/2420909122 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7357f6210 con 0x55e735952950 2024-05-09T00:05:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.410+0000 7f9381ffb700 1 -- 172.21.15.12:0/2420909122 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93940bbdf0 con 0x55e735952950 2024-05-09T00:05:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.410+0000 7f93ac852b80 10 client.30298 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.410+0000 7f93ac852b80 1 -- 172.21.15.12:0/2420909122 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55e7356cab00 con 0x55e735952950 2024-05-09T00:05:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.411+0000 7f9381ffb700 1 -- 172.21.15.12:0/2420909122 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e7357f6210 con 0x55e735952950 2024-05-09T00:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.412+0000 7f9381ffb700 1 -- 172.21.15.12:0/2420909122 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f93940ef0d0 con 0x55e735952950 2024-05-09T00:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.412+0000 7f93ac852b80 10 client.30298 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.412+0000 7f93ac852b80 10 client.30298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.412+0000 7f93ac852b80 20 client.30298 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.412+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e735966430 0x55e735968820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.412+0000 7f93ac852b80 4 client.30298 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.412+0000 7f93ac852b80 1 -- 172.21.15.12:0/2420909122 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e73591c7b0 con 0x55e735966430 2024-05-09T00:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.412+0000 7f939e434700 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e735966430 0x55e735968820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.413+0000 7f9381ffb700 10 client.30298 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.413+0000 7f939e434700 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e735966430 0x55e735968820 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6db381269132a3 server_cookie=d06c84e54763411f in_seq=0 out_seq=0 2024-05-09T00:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f9381ffb700 1 -- 172.21.15.12:0/2420909122 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e73591c7b0 con 0x55e735966430 2024-05-09T00:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f9381ffb700 10 client.30298 handle_command_reply: tid=0 2024-05-09T00:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f93ac852b80 1 client.30298 shutdown 2024-05-09T00:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f93ac852b80 1 -- 172.21.15.12:0/2420909122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 msgr2=0x55e7359565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e7359565c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f939400bd50 tx=0x7f93940c03b0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f93ac852b80 1 -- 172.21.15.12:0/2420909122 shutdown_connections 2024-05-09T00:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e735953270 0x55e735956d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e735966430 0x55e735968820 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e73594d480 0x55e7359592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2420909122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e735952950 0x55e7359565c0 secure :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f939400bd50 tx=0x7f93940c03b0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f9381ffb700 0 client.30298 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.414+0000 7f93ac852b80 1 -- 172.21.15.12:0/2420909122 >> 172.21.15.12:0/2420909122 conn(0x55e7358ab1b0 msgr2=0x55e7359514a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.416+0000 7f93ac852b80 1 -- 172.21.15.12:0/2420909122 shutdown_connections 2024-05-09T00:05:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.416+0000 7f93ac852b80 1 -- 172.21.15.12:0/2420909122 wait complete. 2024-05-09T00:05:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.416+0000 7f93ac852b80 20 client.30298 trim_cache size 0 max 16384 2024-05-09T00:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.439+0000 7f93ac852b80 1 Processor -- start 2024-05-09T00:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f93ac852b80 1 -- start start 2024-05-09T00:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d480 0x55e735949540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e735947910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d860 0x55e735951390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f93ac852b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73593fa40 con 0x55e73594d860 2024-05-09T00:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f93ac852b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e735957bc0 con 0x55e735953270 2024-05-09T00:05:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f93ac852b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73596da30 con 0x55e73594d480 2024-05-09T00:05:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f939e434700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d480 0x55e735949540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f939e434700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d480 0x55e735949540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:33724/0 (socket says 172.21.15.12:33724) 2024-05-09T00:05:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f939ec35700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d860 0x55e735951390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f939e434700 1 -- 172.21.15.12:0/2348102718 learned_addr learned my addr 172.21.15.12:0/2348102718 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.440+0000 7f939dc33700 1 --2- 172.21.15.12:0/2348102718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e735947910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.441+0000 7f939e434700 1 -- 172.21.15.12:0/2348102718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 msgr2=0x55e735947910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.441+0000 7f939e434700 1 --2- 172.21.15.12:0/2348102718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e735947910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.441+0000 7f939e434700 1 -- 172.21.15.12:0/2348102718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d860 msgr2=0x55e735951390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.441+0000 7f939e434700 1 --2- 172.21.15.12:0/2348102718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d860 0x55e735951390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.441+0000 7f939e434700 1 -- 172.21.15.12:0/2348102718 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e735863500 con 0x55e73594d480 2024-05-09T00:05:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.444+0000 7f939e434700 1 --2- 172.21.15.12:0/2348102718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d480 0x55e735949540 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x55e735969130 tx=0x7f9388009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf1385a4d90b7598 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.444+0000 7f938d7fa700 1 -- 172.21.15.12:0/2348102718 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388017040 con 0x55e73594d480 2024-05-09T00:05:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.444+0000 7f938d7fa700 1 -- 172.21.15.12:0/2348102718 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9388012e80 con 0x55e73594d480 2024-05-09T00:05:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.445+0000 7f938d7fa700 1 -- 172.21.15.12:0/2348102718 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93880196a0 con 0x55e73594d480 2024-05-09T00:05:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.445+0000 7f93ac852b80 1 -- 172.21.15.12:0/2348102718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d480 msgr2=0x55e735949540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.445+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2348102718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d480 0x55e735949540 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x55e735969130 tx=0x7f9388009310 comp rx=0 tx=0).stop 2024-05-09T00:05:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.446+0000 7f93ac852b80 1 -- 172.21.15.12:0/2348102718 shutdown_connections 2024-05-09T00:05:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.446+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2348102718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d860 0x55e735951390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.446+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2348102718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d480 0x55e735949540 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.446+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2348102718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e735947910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.446+0000 7f93ac852b80 1 -- 172.21.15.12:0/2348102718 >> 172.21.15.12:0/2348102718 conn(0x55e7358ab1b0 msgr2=0x55e735951d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.450+0000 7f93ac852b80 1 -- 172.21.15.12:0/2348102718 shutdown_connections 2024-05-09T00:05:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.450+0000 7f93ac852b80 1 -- 172.21.15.12:0/2348102718 wait complete. 2024-05-09T00:05:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.450+0000 7f93ac852b80 1 -- start start 2024-05-09T00:05:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f93ac852b80 1 Processor -- start 2024-05-09T00:05:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e735959b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d860 0x55e73595a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f93ac852b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e7359b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f93ac852b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7359b9e20 con 0x55e73594d480 2024-05-09T00:05:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f93ac852b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7359b9f60 con 0x55e735953270 2024-05-09T00:05:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f93ac852b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7359ba120 con 0x55e73594d860 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f93ac852b80 10 client.0 _ll_register_callbacks cb 0x55e735842ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f939e434700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e735959b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f939e434700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e735959b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53640/0 (socket says 172.21.15.12:53640) 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.451+0000 7f939e434700 1 -- 172.21.15.12:0/2583312741 learned_addr learned my addr 172.21.15.12:0/2583312741 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.452+0000 7f939e434700 1 -- 172.21.15.12:0/2583312741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d860 msgr2=0x55e73595a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.452+0000 7f939e434700 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d860 0x55e73595a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.452+0000 7f939e434700 1 -- 172.21.15.12:0/2583312741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 msgr2=0x55e7359b98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.452+0000 7f939e434700 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e7359b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.452+0000 7f939e434700 1 -- 172.21.15.12:0/2583312741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9388009bb0 con 0x55e73594d480 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.452+0000 7f939e434700 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e735959b50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f9388012fb0 tx=0x7f93880048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e668ce823bfc37c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.452+0000 7f9381ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.452+0000 7f9381ffb700 1 -- 172.21.15.12:0/2583312741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388027070 con 0x55e73594d480 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.452+0000 7f9381ffb700 1 -- 172.21.15.12:0/2583312741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9388004cc0 con 0x55e73594d480 2024-05-09T00:05:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.452+0000 7f93ac852b80 1 -- 172.21.15.12:0/2583312741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e735895e60 con 0x55e73594d480 2024-05-09T00:05:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.453+0000 7f9381ffb700 1 -- 172.21.15.12:0/2583312741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9388005020 con 0x55e73594d480 2024-05-09T00:05:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.455+0000 7f93ac852b80 10 client.20595 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.455+0000 7f93ac852b80 1 -- 172.21.15.12:0/2583312741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55e735722c70 con 0x55e73594d480 2024-05-09T00:05:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.456+0000 7f9381ffb700 1 -- 172.21.15.12:0/2583312741 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e735895e60 con 0x55e73594d480 2024-05-09T00:05:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.456+0000 7f9381ffb700 1 -- 172.21.15.12:0/2583312741 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f93880369e0 con 0x55e73594d480 2024-05-09T00:05:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.456+0000 7f93ac852b80 10 client.20595 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.456+0000 7f93ac852b80 10 client.20595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.456+0000 7f93ac852b80 20 client.20595 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.456+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e7359ce770 0x55e7359d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.456+0000 7f93ac852b80 4 client.20595 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:05:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.456+0000 7f93ac852b80 1 -- 172.21.15.12:0/2583312741 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e7356ebc90 con 0x55e7359ce770 2024-05-09T00:05:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.457+0000 7f939dc33700 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e7359ce770 0x55e7359d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.457+0000 7f9381ffb700 10 client.20595 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.457+0000 7f939dc33700 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e7359ce770 0x55e7359d0bb0 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b2cd6b9ad019b4 server_cookie=e66831096b39c7b9 in_seq=0 out_seq=0 2024-05-09T00:05:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f9381ffb700 1 -- 172.21.15.12:0/2583312741 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e7356ebc90 con 0x55e7359ce770 2024-05-09T00:05:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f9381ffb700 10 client.20595 handle_command_reply: tid=0 2024-05-09T00:05:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f93ac852b80 1 client.20595 shutdown 2024-05-09T00:05:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f93ac852b80 1 -- 172.21.15.12:0/2583312741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 msgr2=0x55e735959b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e735959b50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f9388012fb0 tx=0x7f93880048e0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f93ac852b80 1 -- 172.21.15.12:0/2583312741 shutdown_connections 2024-05-09T00:05:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e7359ce770 0x55e7359d0bb0 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e735953270 0x55e7359b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e73594d480 0x55e735959b50 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f93ac852b80 1 --2- 172.21.15.12:0/2583312741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e73594d860 0x55e73595a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f9381ffb700 0 client.20595 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.715+0000 7f93ac852b80 1 -- 172.21.15.12:0/2583312741 >> 172.21.15.12:0/2583312741 conn(0x55e7358ab1b0 msgr2=0x55e735953690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.717+0000 7f93ac852b80 1 -- 172.21.15.12:0/2583312741 shutdown_connections 2024-05-09T00:05:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.717+0000 7f93ac852b80 1 -- 172.21.15.12:0/2583312741 wait complete. 2024-05-09T00:05:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.717+0000 7f93ac852b80 20 client.20595 trim_cache size 0 max 16384 2024-05-09T00:05:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:05:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:05:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:05:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 -- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93840a1310 msgr2=0x7f93840a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93840a1310 0x7f93840a37d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9390007910 tx=0x7f9390009d50 comp rx=0 tx=0).stop 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 -- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9398105260 msgr2=0x7f93981a3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9398105260 0x7f93981a3df0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f9394008000 tx=0x7f9394010fd0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 -- 172.21.15.12:0/3156285507 shutdown_connections 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93840a1310 0x7f93840a37d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939819d9e0 0x7f939819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93980ffe60 0x7f93981a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 --2- 172.21.15.12:0/3156285507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9398105260 0x7f93981a3df0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 -- 172.21.15.12:0/3156285507 >> 172.21.15.12:0/3156285507 conn(0x7f93980fb870 msgr2=0x7f939810ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 -- 172.21.15.12:0/3156285507 shutdown_connections 2024-05-09T00:05:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:05.718+0000 7f93a0698700 1 -- 172.21.15.12:0/3156285507 wait complete. 2024-05-09T00:05:06.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:05 smithi204 ceph-mon[47373]: pgmap v601: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-05-09T00:05:06.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:05 smithi160 ceph-mon[46152]: pgmap v601: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-05-09T00:05:06.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:05 smithi012 ceph-mon[42503]: pgmap v601: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-05-09T00:05:06.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 204349 34.93 MB/sec execute 181 sec latency 261.802 ms 2024-05-09T00:05:07.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205105 35.01 MB/sec execute 182 sec latency 90.082 ms 2024-05-09T00:05:08.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:07 smithi204 ceph-mon[47373]: pgmap v602: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.8 MiB/s wr, 740 op/s 2024-05-09T00:05:08.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:07 smithi160 ceph-mon[46152]: pgmap v602: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.8 MiB/s wr, 740 op/s 2024-05-09T00:05:08.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:07 smithi012 ceph-mon[42503]: pgmap v602: 97 pgs: 97 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.8 MiB/s wr, 740 op/s 2024-05-09T00:05:08.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205618 34.99 MB/sec execute 183 sec latency 123.488 ms 2024-05-09T00:05:09.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 206191 34.94 MB/sec execute 184 sec latency 96.039 ms 2024-05-09T00:05:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:09 smithi204 ceph-mon[47373]: pgmap v603: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 570 op/s 2024-05-09T00:05:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:09 smithi160 ceph-mon[46152]: pgmap v603: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 570 op/s 2024-05-09T00:05:10.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:09 smithi012 ceph-mon[42503]: pgmap v603: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 570 op/s 2024-05-09T00:05:10.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 206744 34.87 MB/sec execute 185 sec latency 107.632 ms 2024-05-09T00:05:11.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 207517 34.91 MB/sec execute 186 sec latency 78.009 ms 2024-05-09T00:05:11.777 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:05:11.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.945+0000 7fde1c97f700 1 -- 172.21.15.12:0/1511351697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde18101b10 msgr2=0x7fde18105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:11.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.945+0000 7fde1c97f700 1 --2- 172.21.15.12:0/1511351697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde18101b10 0x7fde18105b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009990 tx=0x7fde0c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.945+0000 7fde1c97f700 1 -- 172.21.15.12:0/1511351697 shutdown_connections 2024-05-09T00:05:11.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.945+0000 7fde1c97f700 1 --2- 172.21.15.12:0/1511351697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18106280 0x7fde1810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.945+0000 7fde1c97f700 1 --2- 172.21.15.12:0/1511351697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde18101160 0x7fde18101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.945+0000 7fde1c97f700 1 --2- 172.21.15.12:0/1511351697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde18101b10 0x7fde18105b50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.945+0000 7fde1c97f700 1 -- 172.21.15.12:0/1511351697 >> 172.21.15.12:0/1511351697 conn(0x7fde180fc9e0 msgr2=0x7fde180fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:11.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.945+0000 7fde1c97f700 1 -- 172.21.15.12:0/1511351697 shutdown_connections 2024-05-09T00:05:11.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.945+0000 7fde1c97f700 1 -- 172.21.15.12:0/1511351697 wait complete. 2024-05-09T00:05:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1c97f700 1 Processor -- start 2024-05-09T00:05:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1c97f700 1 -- start start 2024-05-09T00:05:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1c97f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde18101160 0x7fde181973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1c97f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18106280 0x7fde18197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1c97f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde1819bc80 0x7fde1819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1c97f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1819c700 con 0x7fde18106280 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1c97f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1819c870 con 0x7fde18101160 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1c97f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1819ca30 con 0x7fde1819bc80 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde15d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18106280 0x7fde18197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde18101160 0x7fde181973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde18101160 0x7fde181973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:38966/0 (socket says 172.21.15.12:38966) 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde1659c700 1 -- 172.21.15.12:0/967408296 learned_addr learned my addr 172.21.15.12:0/967408296 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.946+0000 7fde15d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18106280 0x7fde18197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56848/0 (socket says 172.21.15.12:56848) 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde16d9d700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde1819bc80 0x7fde1819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde1659c700 1 -- 172.21.15.12:0/967408296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde1819bc80 msgr2=0x7fde1819c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde1659c700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde1819bc80 0x7fde1819c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde1659c700 1 -- 172.21.15.12:0/967408296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18106280 msgr2=0x7fde18197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde1659c700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18106280 0x7fde18197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde1659c700 1 -- 172.21.15.12:0/967408296 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0095f0 con 0x7fde18101160 2024-05-09T00:05:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde15d9b700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18106280 0x7fde18197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde1659c700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde18101160 0x7fde181973f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fde0800e9a0 tx=0x7fde0800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde077fe700 1 -- 172.21.15.12:0/967408296 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08009790 con 0x7fde18101160 2024-05-09T00:05:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde077fe700 1 -- 172.21.15.12:0/967408296 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde08010430 con 0x7fde18101160 2024-05-09T00:05:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde1c97f700 1 -- 172.21.15.12:0/967408296 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde181a3f90 con 0x7fde18101160 2024-05-09T00:05:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde077fe700 1 -- 172.21.15.12:0/967408296 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08018680 con 0x7fde18101160 2024-05-09T00:05:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.947+0000 7fde1c97f700 1 -- 172.21.15.12:0/967408296 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde181a43b0 con 0x7fde18101160 2024-05-09T00:05:11.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.950+0000 7fde077fe700 1 -- 172.21.15.12:0/967408296 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fde080105a0 con 0x7fde18101160 2024-05-09T00:05:11.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.950+0000 7fde077fe700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde000a1290 0x7fde000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.951+0000 7fde077fe700 1 -- 172.21.15.12:0/967408296 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fde08014070 con 0x7fde18101160 2024-05-09T00:05:11.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.951+0000 7fde15d9b700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde000a1290 0x7fde000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.951+0000 7fde15d9b700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde000a1290 0x7fde000a3750 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00b2e0 tx=0x7fde0c005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:11.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.952+0000 7fde28b39b80 1 Processor -- start 2024-05-09T00:05:11.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.952+0000 7fde28b39b80 1 -- start start 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.952+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 0x562e66cbc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.952+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 0x562e66cb6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.952+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cb6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.952+0000 7fde28b39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66c0d150 con 0x562e66cbbe00 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.952+0000 7fde28b39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66c0ab10 con 0x562e66cb6890 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.952+0000 7fde28b39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66c06400 con 0x562e66cbc720 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde16d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cb6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde1659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 0x562e66cbc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde15d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 0x562e66cb6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde16d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cb6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:38968/0 (socket says 172.21.15.12:38968) 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde1659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 0x562e66cbc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56854/0 (socket says 172.21.15.12:56854) 2024-05-09T00:05:11.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde16d9d700 1 -- 172.21.15.12:0/3908080283 learned_addr learned my addr 172.21.15.12:0/3908080283 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde15d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 0x562e66cb6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48158/0 (socket says 172.21.15.12:48158) 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde16d9d700 1 -- 172.21.15.12:0/3908080283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 msgr2=0x562e66cb6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde16d9d700 1 --2- 172.21.15.12:0/3908080283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 0x562e66cb6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde16d9d700 1 -- 172.21.15.12:0/3908080283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 msgr2=0x562e66cbc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde16d9d700 1 --2- 172.21.15.12:0/3908080283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 0x562e66cbc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde16d9d700 1 -- 172.21.15.12:0/3908080283 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e66a873b0 con 0x562e66cb6890 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde15d9b700 1 --2- 172.21.15.12:0/3908080283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 0x562e66cb6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde16d9d700 1 --2- 172.21.15.12:0/3908080283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cb6cd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fde10011680 tx=0x7fde10009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5605e920640263bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde04ff9700 1 -- 172.21.15.12:0/3908080283 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1000e040 con 0x562e66cb6890 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde04ff9700 1 -- 172.21.15.12:0/3908080283 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde1000f040 con 0x562e66cb6890 2024-05-09T00:05:11.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.953+0000 7fde04ff9700 1 -- 172.21.15.12:0/3908080283 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10004740 con 0x562e66cb6890 2024-05-09T00:05:11.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.954+0000 7fde28b39b80 1 -- 172.21.15.12:0/3908080283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 msgr2=0x562e66cb6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:11.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.954+0000 7fde28b39b80 1 --2- 172.21.15.12:0/3908080283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cb6cd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fde10011680 tx=0x7fde10009920 comp rx=0 tx=0).stop 2024-05-09T00:05:11.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.954+0000 7fde28b39b80 1 -- 172.21.15.12:0/3908080283 shutdown_connections 2024-05-09T00:05:11.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.954+0000 7fde28b39b80 1 --2- 172.21.15.12:0/3908080283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cb6cd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.954+0000 7fde28b39b80 1 --2- 172.21.15.12:0/3908080283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 0x562e66cbc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.954+0000 7fde28b39b80 1 --2- 172.21.15.12:0/3908080283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 0x562e66cb6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.954+0000 7fde28b39b80 1 -- 172.21.15.12:0/3908080283 >> 172.21.15.12:0/3908080283 conn(0x562e66c14210 msgr2=0x562e66cb0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.954+0000 7fde28b39b80 1 -- 172.21.15.12:0/3908080283 shutdown_connections 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.954+0000 7fde28b39b80 1 -- 172.21.15.12:0/3908080283 wait complete. 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde28b39b80 1 -- start start 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde28b39b80 1 Processor -- start 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 0x562e66cbfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cbbe00 0x562e66cc0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cc05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde28b39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66ba2560 con 0x562e66cbc720 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde28b39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66ba2200 con 0x562e66cbbe00 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde28b39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66a896a0 con 0x562e66cb6890 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde28b39b80 10 client.0 _ll_register_callbacks cb 0x562e66babb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:11.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde1659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 0x562e66cbfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde16d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cc05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.955+0000 7fde15d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cbbe00 0x562e66cc0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fde1659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 0x562e66cbfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48174/0 (socket says 172.21.15.12:48174) 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fde1659c700 1 -- 172.21.15.12:0/2958911102 learned_addr learned my addr 172.21.15.12:0/2958911102 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fde16d9d700 1 -- 172.21.15.12:0/2958911102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 msgr2=0x562e66cbfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fde16d9d700 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 0x562e66cbfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fde16d9d700 1 -- 172.21.15.12:0/2958911102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cbbe00 msgr2=0x562e66cc0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fde16d9d700 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cbbe00 0x562e66cc0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fde16d9d700 1 -- 172.21.15.12:0/2958911102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1810e670 con 0x562e66cbc720 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fde1659c700 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 0x562e66cbfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fde16d9d700 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cc05b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fde10004280 tx=0x7fde10006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cca6a3ec6e30d86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fddfdffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:11.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fddfdffb700 1 -- 172.21.15.12:0/2958911102 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1000e070 con 0x562e66cbc720 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fddfdffb700 1 -- 172.21.15.12:0/2958911102 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fde10003ec0 con 0x562e66cbc720 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.956+0000 7fde28b39b80 1 -- 172.21.15.12:0/2958911102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e66b5f270 con 0x562e66cbc720 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fddfdffb700 1 -- 172.21.15.12:0/2958911102 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde10004590 con 0x562e66cbc720 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fde28b39b80 10 client.20607 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fde28b39b80 1 -- 172.21.15.12:0/2958911102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7fde10011330 con 0x562e66cbc720 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fddfdffb700 1 -- 172.21.15.12:0/2958911102 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e66b5f270 con 0x562e66cbc720 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fddfdffb700 1 -- 172.21.15.12:0/2958911102 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fde1002d070 con 0x562e66cbc720 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fde28b39b80 10 client.20607 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fde28b39b80 10 client.20607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fde28b39b80 20 client.20607 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fde28b39b80 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e66cd0140 0x562e66cd2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fde28b39b80 4 client.20607 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:11.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fde28b39b80 1 -- 172.21.15.12:0/2958911102 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e66c85810 con 0x562e66cd0140 2024-05-09T00:05:11.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.957+0000 7fde1659c700 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e66cd0140 0x562e66cd2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.958+0000 7fde1659c700 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e66cd0140 0x562e66cd2530 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547183fe7f5c7eb server_cookie=6f196e9920f3fd6e in_seq=0 out_seq=0 2024-05-09T00:05:11.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.958+0000 7fddfdffb700 10 client.20607 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:11.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fddfdffb700 1 -- 172.21.15.12:0/2958911102 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e66c85810 con 0x562e66cd0140 2024-05-09T00:05:11.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fddfdffb700 10 client.20607 handle_command_reply: tid=0 2024-05-09T00:05:11.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 client.20607 shutdown 2024-05-09T00:05:11.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 -- 172.21.15.12:0/2958911102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 msgr2=0x562e66cc05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:11.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cc05b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fde10004280 tx=0x7fde10006000 comp rx=0 tx=0).stop 2024-05-09T00:05:11.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 -- 172.21.15.12:0/2958911102 shutdown_connections 2024-05-09T00:05:11.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cc05b0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 0x562e66cbfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e66cd0140 0x562e66cd2530 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 --2- 172.21.15.12:0/2958911102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cbbe00 0x562e66cc0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fddfdffb700 0 client.20607 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:11.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 -- 172.21.15.12:0/2958911102 >> 172.21.15.12:0/2958911102 conn(0x562e66c14210 msgr2=0x562e66cb05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:11.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 -- 172.21.15.12:0/2958911102 shutdown_connections 2024-05-09T00:05:11.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 1 -- 172.21.15.12:0/2958911102 wait complete. 2024-05-09T00:05:11.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.960+0000 7fde28b39b80 20 client.20607 trim_cache size 0 max 16384 2024-05-09T00:05:11.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.980+0000 7fde28b39b80 1 Processor -- start 2024-05-09T00:05:11.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.980+0000 7fde28b39b80 1 -- start start 2024-05-09T00:05:11.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.980+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 0x562e66cb6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.980+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cbbe00 0x562e66cb0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.980+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cbe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.980+0000 7fde28b39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66cbc580 con 0x562e66cbc720 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.980+0000 7fde28b39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66cb1ca0 con 0x562e66cbbe00 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.980+0000 7fde28b39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66cbf920 con 0x562e66cb6890 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde16d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cbe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde1659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 0x562e66cb6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde16d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cbe400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56882/0 (socket says 172.21.15.12:56882) 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde16d9d700 1 -- 172.21.15.12:0/4283577728 learned_addr learned my addr 172.21.15.12:0/4283577728 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde15d9b700 1 --2- 172.21.15.12:0/4283577728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cbbe00 0x562e66cb0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde16d9d700 1 -- 172.21.15.12:0/4283577728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 msgr2=0x562e66cb6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde16d9d700 1 --2- 172.21.15.12:0/4283577728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 0x562e66cb6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde16d9d700 1 -- 172.21.15.12:0/4283577728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cbbe00 msgr2=0x562e66cb0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde16d9d700 1 --2- 172.21.15.12:0/4283577728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cbbe00 0x562e66cb0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde16d9d700 1 -- 172.21.15.12:0/4283577728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e66a33b60 con 0x562e66cbc720 2024-05-09T00:05:11.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde16d9d700 1 --2- 172.21.15.12:0/4283577728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cbe400 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fde10011680 tx=0x7fde100094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ff08eb2b4b97ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:11.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde057fa700 1 -- 172.21.15.12:0/4283577728 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1000e070 con 0x562e66cbc720 2024-05-09T00:05:11.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde057fa700 1 -- 172.21.15.12:0/4283577728 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde10015520 con 0x562e66cbc720 2024-05-09T00:05:11.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.981+0000 7fde057fa700 1 -- 172.21.15.12:0/4283577728 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1001de10 con 0x562e66cbc720 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.982+0000 7fde28b39b80 1 -- 172.21.15.12:0/4283577728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 msgr2=0x562e66cbe400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.982+0000 7fde28b39b80 1 --2- 172.21.15.12:0/4283577728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cbe400 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fde10011680 tx=0x7fde100094f0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.982+0000 7fde28b39b80 1 -- 172.21.15.12:0/4283577728 shutdown_connections 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.982+0000 7fde28b39b80 1 --2- 172.21.15.12:0/4283577728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbc720 0x562e66cbe400 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.982+0000 7fde28b39b80 1 --2- 172.21.15.12:0/4283577728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cb6890 0x562e66cb6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.982+0000 7fde28b39b80 1 --2- 172.21.15.12:0/4283577728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cbbe00 0x562e66cb0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.982+0000 7fde28b39b80 1 -- 172.21.15.12:0/4283577728 >> 172.21.15.12:0/4283577728 conn(0x562e66c14210 msgr2=0x562e66cb9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.983+0000 7fde28b39b80 1 -- 172.21.15.12:0/4283577728 shutdown_connections 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.983+0000 7fde28b39b80 1 -- 172.21.15.12:0/4283577728 wait complete. 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.983+0000 7fde28b39b80 1 -- start start 2024-05-09T00:05:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.983+0000 7fde28b39b80 1 Processor -- start 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.983+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cc7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.983+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 0x562e66cc5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.983+0000 7fde28b39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 0x562e66cc5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.983+0000 7fde28b39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66cc5d70 con 0x562e66cbbe00 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.983+0000 7fde28b39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66cc5ee0 con 0x562e66cb6890 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.983+0000 7fde28b39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e66d25610 con 0x562e66cbc720 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde28b39b80 10 client.0 _ll_register_callbacks cb 0x562e66babb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde1659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cc7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde1659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cc7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39002/0 (socket says 172.21.15.12:39002) 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde1659c700 1 -- 172.21.15.12:0/4281329879 learned_addr learned my addr 172.21.15.12:0/4281329879 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde15d9b700 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 0x562e66cc5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde16d9d700 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 0x562e66cc5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde1659c700 1 -- 172.21.15.12:0/4281329879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 msgr2=0x562e66cc5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde1659c700 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 0x562e66cc5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde1659c700 1 -- 172.21.15.12:0/4281329879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 msgr2=0x562e66cc5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde1659c700 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 0x562e66cc5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde1659c700 1 -- 172.21.15.12:0/4281329879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde10011330 con 0x562e66cb6890 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde15d9b700 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 0x562e66cc5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fde1659c700 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cc7570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde0800c2d0 tx=0x7fde080cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f764e3ee23d0be58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.984+0000 7fddfdffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.985+0000 7fddfdffb700 1 -- 172.21.15.12:0/4281329879 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde080a23c0 con 0x562e66cb6890 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.985+0000 7fddfdffb700 1 -- 172.21.15.12:0/4281329879 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fde08010880 con 0x562e66cb6890 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.985+0000 7fde28b39b80 1 -- 172.21.15.12:0/4281329879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde1000f040 con 0x562e66cb6890 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.985+0000 7fddfdffb700 1 -- 172.21.15.12:0/4281329879 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde080b82f0 con 0x562e66cb6890 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.985+0000 7fde28b39b80 10 client.30182 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.985+0000 7fde28b39b80 1 -- 172.21.15.12:0/4281329879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x562e66bcc560 con 0x562e66cb6890 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.985+0000 7fddfdffb700 1 -- 172.21.15.12:0/4281329879 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde1000f040 con 0x562e66cb6890 2024-05-09T00:05:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.985+0000 7fddfdffb700 1 -- 172.21.15.12:0/4281329879 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fde080a2030 con 0x562e66cb6890 2024-05-09T00:05:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.985+0000 7fde28b39b80 10 client.30182 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.985+0000 7fde28b39b80 10 client.30182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.986+0000 7fde28b39b80 20 client.30182 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.986+0000 7fde28b39b80 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e66d26cd0 0x562e66d290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.986+0000 7fde28b39b80 4 client.30182 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:05:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.986+0000 7fde28b39b80 1 -- 172.21.15.12:0/4281329879 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562e66bfeec0 con 0x562e66d26cd0 2024-05-09T00:05:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.986+0000 7fde15d9b700 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e66d26cd0 0x562e66d290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.986+0000 7fde15d9b700 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e66d26cd0 0x562e66d290c0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85defffc4f7647b4 server_cookie=77fa3b23ba9a47b8 in_seq=0 out_seq=0 2024-05-09T00:05:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.986+0000 7fddfdffb700 10 client.30182 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fddfdffb700 1 -- 172.21.15.12:0/4281329879 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562e66bfeec0 con 0x562e66d26cd0 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fddfdffb700 10 client.30182 handle_command_reply: tid=0 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 client.30182 shutdown 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 -- 172.21.15.12:0/4281329879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 msgr2=0x562e66cc7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cc7570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde0800c2d0 tx=0x7fde080cc040 comp rx=0 tx=0).stop 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 -- 172.21.15.12:0/4281329879 shutdown_connections 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e66cbc720 0x562e66cc5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562e66d26cd0 0x562e66d290c0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e66cb6890 0x562e66cc7570 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 --2- 172.21.15.12:0/4281329879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e66cbbe00 0x562e66cc5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fddfdffb700 0 client.30182 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 -- 172.21.15.12:0/4281329879 >> 172.21.15.12:0/4281329879 conn(0x562e66c14210 msgr2=0x562e66cbe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 -- 172.21.15.12:0/4281329879 shutdown_connections 2024-05-09T00:05:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 1 -- 172.21.15.12:0/4281329879 wait complete. 2024-05-09T00:05:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.990+0000 7fde28b39b80 20 client.30182 trim_cache size 0 max 16384 2024-05-09T00:05:11.986 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:05:11.986 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:05:11.986 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:05:11.986 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 -- 172.21.15.12:0/967408296 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde000a1290 msgr2=0x7fde000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde000a1290 0x7fde000a3750 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00b2e0 tx=0x7fde0c005960 comp rx=0 tx=0).stop 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 -- 172.21.15.12:0/967408296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde18101160 msgr2=0x7fde181973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde18101160 0x7fde181973f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fde0800e9a0 tx=0x7fde0800ecb0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 -- 172.21.15.12:0/967408296 shutdown_connections 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fde000a1290 0x7fde000a3750 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde1819bc80 0x7fde1819c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde18101160 0x7fde181973f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 --2- 172.21.15.12:0/967408296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18106280 0x7fde18197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 -- 172.21.15.12:0/967408296 >> 172.21.15.12:0/967408296 conn(0x7fde180fc9e0 msgr2=0x7fde180fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 -- 172.21.15.12:0/967408296 shutdown_connections 2024-05-09T00:05:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:11.991+0000 7fde1c97f700 1 -- 172.21.15.12:0/967408296 wait complete. 2024-05-09T00:05:12.002 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:11 smithi012 ceph-mon[42503]: pgmap v604: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 542 op/s 2024-05-09T00:05:12.003 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:05:12.003 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:05:12.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:11 smithi204 ceph-mon[47373]: pgmap v604: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 542 op/s 2024-05-09T00:05:12.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:11 smithi160 ceph-mon[46152]: pgmap v604: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 542 op/s 2024-05-09T00:05:12.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.182+0000 7faa99cb1700 1 -- 172.21.15.12:0/509829997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa94105090 msgr2=0x7faa9410cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.182+0000 7faa99cb1700 1 --2- 172.21.15.12:0/509829997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa94105090 0x7faa9410cc10 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00b0a0 tx=0x7faa8c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.182+0000 7faa99cb1700 1 -- 172.21.15.12:0/509829997 shutdown_connections 2024-05-09T00:05:12.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.182+0000 7faa99cb1700 1 --2- 172.21.15.12:0/509829997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa94105090 0x7faa9410cc10 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.182+0000 7faa99cb1700 1 --2- 172.21.15.12:0/509829997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa940fff70 0x7faa94100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.182+0000 7faa99cb1700 1 --2- 172.21.15.12:0/509829997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa94100920 0x7faa94104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.182+0000 7faa99cb1700 1 -- 172.21.15.12:0/509829997 >> 172.21.15.12:0/509829997 conn(0x7faa940fb7d0 msgr2=0x7faa940fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.182+0000 7faa99cb1700 1 -- 172.21.15.12:0/509829997 shutdown_connections 2024-05-09T00:05:12.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.182+0000 7faa99cb1700 1 -- 172.21.15.12:0/509829997 wait complete. 2024-05-09T00:05:12.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.182+0000 7faa99cb1700 1 Processor -- start 2024-05-09T00:05:12.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa99cb1700 1 -- start start 2024-05-09T00:05:12.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa99cb1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa940fff70 0x7faa94195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa99cb1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa94105090 0x7faa94195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa99cb1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa94199c20 0x7faa9419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa99cb1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9419a6a0 con 0x7faa940fff70 2024-05-09T00:05:12.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa99cb1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9419a810 con 0x7faa94105090 2024-05-09T00:05:12.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa99cb1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9419a9d0 con 0x7faa94199c20 2024-05-09T00:05:12.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa940fff70 0x7faa94195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa940fff70 0x7faa94195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56918/0 (socket says 172.21.15.12:56918) 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa937fe700 1 -- 172.21.15.12:0/2989882444 learned_addr learned my addr 172.21.15.12:0/2989882444 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa8affd700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa94105090 0x7faa94195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.183+0000 7faa93fff700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa94199c20 0x7faa9419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa937fe700 1 -- 172.21.15.12:0/2989882444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa94199c20 msgr2=0x7faa9419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa937fe700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa94199c20 0x7faa9419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa937fe700 1 -- 172.21.15.12:0/2989882444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa94105090 msgr2=0x7faa94195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa937fe700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa94105090 0x7faa94195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa937fe700 1 -- 172.21.15.12:0/2989882444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c009d00 con 0x7faa940fff70 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa937fe700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa940fff70 0x7faa94195340 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7faa8400d950 tx=0x7faa8400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa917fa700 1 -- 172.21.15.12:0/2989882444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8400f890 con 0x7faa940fff70 2024-05-09T00:05:12.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa917fa700 1 -- 172.21.15.12:0/2989882444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa84010460 con 0x7faa940fff70 2024-05-09T00:05:12.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa99cb1700 1 -- 172.21.15.12:0/2989882444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa941a1f30 con 0x7faa940fff70 2024-05-09T00:05:12.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa917fa700 1 -- 172.21.15.12:0/2989882444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8400e610 con 0x7faa940fff70 2024-05-09T00:05:12.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.184+0000 7faa99cb1700 1 -- 172.21.15.12:0/2989882444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa941a23b0 con 0x7faa940fff70 2024-05-09T00:05:12.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.185+0000 7faa99cb1700 1 -- 172.21.15.12:0/2989882444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa94108360 con 0x7faa940fff70 2024-05-09T00:05:12.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.187+0000 7faa917fa700 1 -- 172.21.15.12:0/2989882444 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faa840108d0 con 0x7faa940fff70 2024-05-09T00:05:12.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.188+0000 7faa917fa700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa800a34a0 0x7faa800a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.188+0000 7faa917fa700 1 -- 172.21.15.12:0/2989882444 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7faa840cca60 con 0x7faa940fff70 2024-05-09T00:05:12.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.188+0000 7faa8affd700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa800a34a0 0x7faa800a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.188+0000 7faa8affd700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa800a34a0 0x7faa800a5960 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faa94196890 tx=0x7faa7c009190 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.192+0000 7faa917fa700 1 -- 172.21.15.12:0/2989882444 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faa84095ec0 con 0x7faa940fff70 2024-05-09T00:05:12.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 208244 34.94 MB/sec execute 187 sec latency 103.751 ms 2024-05-09T00:05:12.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.484+0000 7faa99cb1700 1 -- 172.21.15.12:0/2989882444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa94101440 con 0x7faa940fff70 2024-05-09T00:05:12.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.485+0000 7faa917fa700 1 -- 172.21.15.12:0/2989882444 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+10872 (secure 0 0 0) 0x7faa84016090 con 0x7faa940fff70 2024-05-09T00:05:12.479 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:05:12.479 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":74,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:03:37.323291+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_30034":{"gid":30034,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6833/181468454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":181468454},{"type":"v1","addr":"172.21.15.160:6833","nonce":181468454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:05:12.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.487+0000 7faa99cb1700 1 -- 172.21.15.12:0/2989882444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa800a34a0 msgr2=0x7faa800a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.487+0000 7faa99cb1700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa800a34a0 0x7faa800a5960 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faa94196890 tx=0x7faa7c009190 comp rx=0 tx=0).stop 2024-05-09T00:05:12.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.487+0000 7faa99cb1700 1 -- 172.21.15.12:0/2989882444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa940fff70 msgr2=0x7faa94195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.487+0000 7faa99cb1700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa940fff70 0x7faa94195340 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7faa8400d950 tx=0x7faa8400dc60 comp rx=0 tx=0).stop 2024-05-09T00:05:12.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.488+0000 7faa99cb1700 1 -- 172.21.15.12:0/2989882444 shutdown_connections 2024-05-09T00:05:12.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.488+0000 7faa99cb1700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7faa800a34a0 0x7faa800a5960 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.488+0000 7faa99cb1700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa94199c20 0x7faa9419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.488+0000 7faa99cb1700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa940fff70 0x7faa94195340 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.488+0000 7faa99cb1700 1 --2- 172.21.15.12:0/2989882444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa94105090 0x7faa94195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.488+0000 7faa99cb1700 1 -- 172.21.15.12:0/2989882444 >> 172.21.15.12:0/2989882444 conn(0x7faa940fb7d0 msgr2=0x7faa940fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.488+0000 7faa99cb1700 1 -- 172.21.15.12:0/2989882444 shutdown_connections 2024-05-09T00:05:12.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.488+0000 7faa99cb1700 1 -- 172.21.15.12:0/2989882444 wait complete. 2024-05-09T00:05:12.483 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 74 2024-05-09T00:05:12.498 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:05:12.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.666+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/1012998005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e0106290 msgr2=0x7fd2e010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.666+0000 7fd2e7c4f700 1 --2- 172.21.15.12:0/1012998005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e0106290 0x7fd2e010de10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800b0d0 tx=0x7fd2d800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.666+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/1012998005 shutdown_connections 2024-05-09T00:05:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.666+0000 7fd2e7c4f700 1 --2- 172.21.15.12:0/1012998005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e0106290 0x7fd2e010de10 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.666+0000 7fd2e7c4f700 1 --2- 172.21.15.12:0/1012998005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2e00682e0 0x7fd2e00686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.666+0000 7fd2e7c4f700 1 --2- 172.21.15.12:0/1012998005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e0068c90 0x7fd2e0105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.666+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/1012998005 >> 172.21.15.12:0/1012998005 conn(0x7fd2e0076f40 msgr2=0x7fd2e0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.666+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/1012998005 shutdown_connections 2024-05-09T00:05:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.666+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/1012998005 wait complete. 2024-05-09T00:05:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.667+0000 7fd2e7c4f700 1 Processor -- start 2024-05-09T00:05:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.667+0000 7fd2e7c4f700 1 -- start start 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e7c4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e00682e0 0x7fd2e0197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e7c4f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2e0106290 0x7fd2e01979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e7c4f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e019bd50 0x7fd2e019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e7c4f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e019c7d0 con 0x7fd2e00682e0 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e59eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e00682e0 0x7fd2e0197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e7c4f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e019c940 con 0x7fd2e019bd50 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e7c4f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e019cb00 con 0x7fd2e0106290 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e59eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e00682e0 0x7fd2e0197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56946/0 (socket says 172.21.15.12:56946) 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e59eb700 1 -- 172.21.15.12:0/2566985049 learned_addr learned my addr 172.21.15.12:0/2566985049 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e51ea700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2e0106290 0x7fd2e01979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e61ec700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e019bd50 0x7fd2e019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e59eb700 1 -- 172.21.15.12:0/2566985049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2e0106290 msgr2=0x7fd2e01979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e59eb700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2e0106290 0x7fd2e01979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e59eb700 1 -- 172.21.15.12:0/2566985049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e019bd50 msgr2=0x7fd2e019c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e59eb700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e019bd50 0x7fd2e019c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e59eb700 1 -- 172.21.15.12:0/2566985049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8009d00 con 0x7fd2e00682e0 2024-05-09T00:05:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.668+0000 7fd2e59eb700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e00682e0 0x7fd2e0197470 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00c9b0 tx=0x7fd2dc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.669+0000 7fd2d6ffd700 1 -- 172.21.15.12:0/2566985049 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc007b00 con 0x7fd2e00682e0 2024-05-09T00:05:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.669+0000 7fd2d6ffd700 1 -- 172.21.15.12:0/2566985049 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2dc00ce50 con 0x7fd2e00682e0 2024-05-09T00:05:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.669+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/2566985049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e01a4060 con 0x7fd2e00682e0 2024-05-09T00:05:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.669+0000 7fd2d6ffd700 1 -- 172.21.15.12:0/2566985049 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc018660 con 0x7fd2e00682e0 2024-05-09T00:05:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.669+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/2566985049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e01a4540 con 0x7fd2e00682e0 2024-05-09T00:05:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.671+0000 7fd2d6ffd700 1 -- 172.21.15.12:0/2566985049 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd2dc00f450 con 0x7fd2e00682e0 2024-05-09T00:05:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.672+0000 7fd2d6ffd700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2cc0a1290 0x7fd2cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.672+0000 7fd2d6ffd700 1 -- 172.21.15.12:0/2566985049 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd2dc0ccc00 con 0x7fd2e00682e0 2024-05-09T00:05:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.672+0000 7fd2e51ea700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2cc0a1290 0x7fd2cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.673+0000 7fd2e51ea700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2cc0a1290 0x7fd2cc0a3750 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2e01989c0 tx=0x7fd2d0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.673+0000 7fd2f3e09b80 1 Processor -- start 2024-05-09T00:05:12.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2f3e09b80 1 -- start start 2024-05-09T00:05:12.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac4e00 0x559bd5ac51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac5720 0x559bd5abf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 0x559bd5abfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5a16150 con 0x559bd5ac5720 2024-05-09T00:05:12.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5a13b10 con 0x559bd5ac4e00 2024-05-09T00:05:12.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5a0f400 con 0x559bd5abf890 2024-05-09T00:05:12.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2e51ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac5720 0x559bd5abf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2e51ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac5720 0x559bd5abf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56954/0 (socket says 172.21.15.12:56954) 2024-05-09T00:05:12.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2e51ea700 1 -- 172.21.15.12:0/4065239987 learned_addr learned my addr 172.21.15.12:0/4065239987 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.674+0000 7fd2e61ec700 1 --2- 172.21.15.12:0/4065239987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 0x559bd5abfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.675+0000 7fd2e61ec700 1 -- 172.21.15.12:0/4065239987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac4e00 msgr2=0x559bd5ac51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.675+0000 7fd2e61ec700 1 --2- 172.21.15.12:0/4065239987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac4e00 0x559bd5ac51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.675+0000 7fd2e61ec700 1 -- 172.21.15.12:0/4065239987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac5720 msgr2=0x559bd5abf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.675+0000 7fd2e61ec700 1 --2- 172.21.15.12:0/4065239987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac5720 0x559bd5abf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.675+0000 7fd2e61ec700 1 -- 172.21.15.12:0/4065239987 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd58903b0 con 0x559bd5abf890 2024-05-09T00:05:12.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.675+0000 7fd2e61ec700 1 --2- 172.21.15.12:0/4065239987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 0x559bd5abfcd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800b770 tx=0x7fd2d8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8fd056972fd4b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.676+0000 7fd2cbfff700 1 -- 172.21.15.12:0/4065239987 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d800e040 con 0x559bd5abf890 2024-05-09T00:05:12.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.676+0000 7fd2cbfff700 1 -- 172.21.15.12:0/4065239987 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2d8009460 con 0x559bd5abf890 2024-05-09T00:05:12.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.676+0000 7fd2cbfff700 1 -- 172.21.15.12:0/4065239987 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8003c60 con 0x559bd5abf890 2024-05-09T00:05:12.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.677+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4065239987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 msgr2=0x559bd5abfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.677+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/4065239987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 0x559bd5abfcd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800b770 tx=0x7fd2d8009c30 comp rx=0 tx=0).stop 2024-05-09T00:05:12.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.678+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4065239987 shutdown_connections 2024-05-09T00:05:12.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.678+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/4065239987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 0x559bd5abfcd0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.678+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/4065239987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac4e00 0x559bd5ac51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.678+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/4065239987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac5720 0x559bd5abf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.678+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4065239987 >> 172.21.15.12:0/4065239987 conn(0x559bd5a1d210 msgr2=0x559bd5ab9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.678+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4065239987 shutdown_connections 2024-05-09T00:05:12.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.678+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4065239987 wait complete. 2024-05-09T00:05:12.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.678+0000 7fd2f3e09b80 1 -- start start 2024-05-09T00:05:12.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.678+0000 7fd2f3e09b80 1 Processor -- start 2024-05-09T00:05:12.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 0x559bd5ac8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5ac9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac5720 0x559bd5ac9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd59ab560 con 0x559bd5ac4e00 2024-05-09T00:05:12.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd59ab200 con 0x559bd5ac5720 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd58926a0 con 0x559bd5abf890 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2f3e09b80 10 client.0 _ll_register_callbacks cb 0x559bd59b4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2e51ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5ac9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2e51ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5ac9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56958/0 (socket says 172.21.15.12:56958) 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2e51ea700 1 -- 172.21.15.12:0/4006365132 learned_addr learned my addr 172.21.15.12:0/4006365132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2e59eb700 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 0x559bd5ac8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.679+0000 7fd2e61ec700 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac5720 0x559bd5ac9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2e51ea700 1 -- 172.21.15.12:0/4006365132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 msgr2=0x559bd5ac8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2e51ea700 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 0x559bd5ac8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2e51ea700 1 -- 172.21.15.12:0/4006365132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac5720 msgr2=0x559bd5ac9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2e51ea700 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac5720 0x559bd5ac9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2e51ea700 1 -- 172.21.15.12:0/4006365132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e010e650 con 0x559bd5ac4e00 2024-05-09T00:05:12.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2e51ea700 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5ac9210 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd2d001f880 tx=0x7fd2d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2178eace8faefc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:12.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2c97fa700 1 -- 172.21.15.12:0/4006365132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d001d740 con 0x559bd5ac4e00 2024-05-09T00:05:12.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2c97fa700 1 -- 172.21.15.12:0/4006365132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd2d001d8a0 con 0x559bd5ac4e00 2024-05-09T00:05:12.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4006365132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bd5968270 con 0x559bd5ac4e00 2024-05-09T00:05:12.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2c97fa700 1 -- 172.21.15.12:0/4006365132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2d001da60 con 0x559bd5ac4e00 2024-05-09T00:05:12.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.680+0000 7fd2f3e09b80 10 client.20631 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:12.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.681+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4006365132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x559bd583cb60 con 0x559bd5ac4e00 2024-05-09T00:05:12.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.681+0000 7fd2c97fa700 1 -- 172.21.15.12:0/4006365132 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2d0034020 con 0x559bd5ac4e00 2024-05-09T00:05:12.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.681+0000 7fd2c97fa700 1 -- 172.21.15.12:0/4006365132 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd2d0027d20 con 0x559bd5ac4e00 2024-05-09T00:05:12.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.681+0000 7fd2f3e09b80 10 client.20631 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:12.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.681+0000 7fd2f3e09b80 10 client.20631 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:05:12.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.681+0000 7fd2f3e09b80 20 client.20631 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:12.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.681+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x559bd5ad9430 0x559bd5adb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.681+0000 7fd2f3e09b80 4 client.20631 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:12.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.681+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4006365132 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bd5a8e810 con 0x559bd5ad9430 2024-05-09T00:05:12.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.681+0000 7fd2e59eb700 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x559bd5ad9430 0x559bd5adb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.682+0000 7fd2c97fa700 10 client.20631 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:05:12.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.682+0000 7fd2e59eb700 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x559bd5ad9430 0x559bd5adb850 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13c9a615e8485f5 server_cookie=c1d2490d222e1c0f in_seq=0 out_seq=0 2024-05-09T00:05:12.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.684+0000 7fd2c97fa700 1 -- 172.21.15.12:0/4006365132 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559bd5a8e810 con 0x559bd5ad9430 2024-05-09T00:05:12.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.684+0000 7fd2c97fa700 10 client.20631 handle_command_reply: tid=0 2024-05-09T00:05:12.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.684+0000 7fd2f3e09b80 1 client.20631 shutdown 2024-05-09T00:05:12.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.686+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4006365132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 msgr2=0x559bd5ac9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.686+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5ac9210 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd2d001f880 tx=0x7fd2d00047c0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.687+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4006365132 shutdown_connections 2024-05-09T00:05:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.687+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac5720 0x559bd5ac9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.687+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x559bd5ad9430 0x559bd5adb850 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.687+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5abf890 0x559bd5ac8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.687+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/4006365132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5ac9210 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.687+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4006365132 >> 172.21.15.12:0/4006365132 conn(0x559bd5a1d210 msgr2=0x559bd5ab95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.687+0000 7fd2c97fa700 0 client.20631 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:05:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.687+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4006365132 shutdown_connections 2024-05-09T00:05:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.688+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/4006365132 wait complete. 2024-05-09T00:05:12.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.688+0000 7fd2f3e09b80 20 client.20631 trim_cache size 0 max 16384 2024-05-09T00:05:12.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2f3e09b80 1 Processor -- start 2024-05-09T00:05:12.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2f3e09b80 1 -- start start 2024-05-09T00:05:12.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5abf890 0x559bd5abf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac4e00 0x559bd5ad53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 0x559bd5ab9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5ab2040 con 0x559bd5abf890 2024-05-09T00:05:12.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5aba8e0 con 0x559bd5ac4e00 2024-05-09T00:05:12.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5ae0a00 con 0x559bd5ac5720 2024-05-09T00:05:12.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2e61ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 0x559bd5ab9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2e61ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 0x559bd5ab9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48250/0 (socket says 172.21.15.12:48250) 2024-05-09T00:05:12.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.710+0000 7fd2e61ec700 1 -- 172.21.15.12:0/801839347 learned_addr learned my addr 172.21.15.12:0/801839347 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.711+0000 7fd2e61ec700 1 -- 172.21.15.12:0/801839347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac4e00 msgr2=0x559bd5ad53d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:05:12.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.711+0000 7fd2e61ec700 1 --2- 172.21.15.12:0/801839347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac4e00 0x559bd5ad53d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.711+0000 7fd2e61ec700 1 -- 172.21.15.12:0/801839347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5abf890 msgr2=0x559bd5abf350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:05:12.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.711+0000 7fd2e61ec700 1 --2- 172.21.15.12:0/801839347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5abf890 0x559bd5abf350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.711+0000 7fd2e61ec700 1 -- 172.21.15.12:0/801839347 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd59d5560 con 0x559bd5ac5720 2024-05-09T00:05:12.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.714+0000 7fd2e61ec700 1 --2- 172.21.15.12:0/801839347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 0x559bd5ab9d20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009fd0 tx=0x7fd2d8006160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f3151c9211cd6ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.715+0000 7fd2d4da4700 1 -- 172.21.15.12:0/801839347 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d800e040 con 0x559bd5ac5720 2024-05-09T00:05:12.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.715+0000 7fd2d4da4700 1 -- 172.21.15.12:0/801839347 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2d8007e00 con 0x559bd5ac5720 2024-05-09T00:05:12.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.715+0000 7fd2d4da4700 1 -- 172.21.15.12:0/801839347 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8007960 con 0x559bd5ac5720 2024-05-09T00:05:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/801839347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 msgr2=0x559bd5ab9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/801839347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 0x559bd5ab9d20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009fd0 tx=0x7fd2d8006160 comp rx=0 tx=0).stop 2024-05-09T00:05:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/801839347 shutdown_connections 2024-05-09T00:05:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/801839347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 0x559bd5ab9d20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/801839347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5abf890 0x559bd5abf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/801839347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5ac4e00 0x559bd5ad53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/801839347 >> 172.21.15.12:0/801839347 conn(0x559bd5a1d210 msgr2=0x559bd5ac2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/801839347 shutdown_connections 2024-05-09T00:05:12.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/801839347 wait complete. 2024-05-09T00:05:12.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 -- start start 2024-05-09T00:05:12.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.716+0000 7fd2f3e09b80 1 Processor -- start 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5abf890 0x559bd5ad0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5acc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2f3e09b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 0x559bd5acc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5acccb0 con 0x559bd5ac4e00 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5acce20 con 0x559bd5abf890 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2f3e09b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5b2c8d0 con 0x559bd5ac5720 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2f3e09b80 10 client.0 _ll_register_callbacks cb 0x559bd59b4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2e51ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5acc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2e51ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5acc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56974/0 (socket says 172.21.15.12:56974) 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2e51ea700 1 -- 172.21.15.12:0/624544978 learned_addr learned my addr 172.21.15.12:0/624544978 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2e61ec700 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 0x559bd5acc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2e51ea700 1 -- 172.21.15.12:0/624544978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 msgr2=0x559bd5acc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2e51ea700 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 0x559bd5acc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2e51ea700 1 -- 172.21.15.12:0/624544978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5abf890 msgr2=0x559bd5ad0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2e51ea700 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5abf890 0x559bd5ad0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.717+0000 7fd2e51ea700 1 -- 172.21.15.12:0/624544978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bd58903b0 con 0x559bd5ac4e00 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.718+0000 7fd2e51ea700 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5acc0e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009fd0 tx=0x7fd2d001dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf9021d421389a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.718+0000 7fd2c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.718+0000 7fd2c97fa700 1 -- 172.21.15.12:0/624544978 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d001da10 con 0x559bd5ac4e00 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.718+0000 7fd2c97fa700 1 -- 172.21.15.12:0/624544978 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd2d0035080 con 0x559bd5ac4e00 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.718+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/624544978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bd5ac0190 con 0x559bd5ac4e00 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.718+0000 7fd2c97fa700 1 -- 172.21.15.12:0/624544978 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2d001db70 con 0x559bd5ac4e00 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.718+0000 7fd2f3e09b80 10 client.20634 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:12.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.718+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/624544978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x559bd5894cd0 con 0x559bd5ac4e00 2024-05-09T00:05:12.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.718+0000 7fd2c97fa700 1 -- 172.21.15.12:0/624544978 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559bd5ac0190 con 0x559bd5ac4e00 2024-05-09T00:05:12.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.719+0000 7fd2c97fa700 1 -- 172.21.15.12:0/624544978 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd2d000e980 con 0x559bd5ac4e00 2024-05-09T00:05:12.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.720+0000 7fd2f3e09b80 10 client.20634 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:12.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.720+0000 7fd2f3e09b80 10 client.20634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:05:12.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.720+0000 7fd2f3e09b80 20 client.20634 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:12.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.720+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x559bd5b2def0 0x559bd5b302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.720+0000 7fd2f3e09b80 4 client.20634 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:05:12.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.720+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/624544978 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fd2d8009460 con 0x559bd5b2def0 2024-05-09T00:05:12.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.720+0000 7fd2e59eb700 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x559bd5b2def0 0x559bd5b302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.720+0000 7fd2c97fa700 10 client.20634 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.720+0000 7fd2e59eb700 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x559bd5b2def0 0x559bd5b302e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30ba18bd5074d48f server_cookie=e2a9df69727109fc in_seq=0 out_seq=0 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.722+0000 7fd2c97fa700 1 -- 172.21.15.12:0/624544978 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fd2d8009460 con 0x559bd5b2def0 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.723+0000 7fd2c97fa700 10 client.20634 handle_command_reply: tid=0 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.723+0000 7fd2f3e09b80 1 client.20634 shutdown 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.723+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/624544978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 msgr2=0x559bd5acc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.723+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5acc0e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009fd0 tx=0x7fd2d001dfd0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.723+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/624544978 shutdown_connections 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.723+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x559bd5b2def0 0x559bd5b302e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.723+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bd5ac5720 0x559bd5acc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.723+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559bd5abf890 0x559bd5ad0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.723+0000 7fd2f3e09b80 1 --2- 172.21.15.12:0/624544978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559bd5ac4e00 0x559bd5acc0e0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.723+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/624544978 >> 172.21.15.12:0/624544978 conn(0x559bd5a1d210 msgr2=0x559bd5ac2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.725+0000 7fd2c97fa700 0 client.20634 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:05:12.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.725+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/624544978 shutdown_connections 2024-05-09T00:05:12.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.725+0000 7fd2f3e09b80 1 -- 172.21.15.12:0/624544978 wait complete. 2024-05-09T00:05:12.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.727+0000 7fd2f3e09b80 20 client.20634 trim_cache size 0 max 16384 2024-05-09T00:05:12.721 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:05:12.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.728+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/2566985049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2cc0a1290 msgr2=0x7fd2cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.728+0000 7fd2e7c4f700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2cc0a1290 0x7fd2cc0a3750 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2e01989c0 tx=0x7fd2d0009260 comp rx=0 tx=0).stop 2024-05-09T00:05:12.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.728+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/2566985049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e00682e0 msgr2=0x7fd2e0197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.728+0000 7fd2e7c4f700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e00682e0 0x7fd2e0197470 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00c9b0 tx=0x7fd2dc00ccc0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.728+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/2566985049 shutdown_connections 2024-05-09T00:05:12.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.728+0000 7fd2e7c4f700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2cc0a1290 0x7fd2cc0a3750 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.728+0000 7fd2e7c4f700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e019bd50 0x7fd2e019c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.728+0000 7fd2e7c4f700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2e00682e0 0x7fd2e0197470 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.728+0000 7fd2e7c4f700 1 --2- 172.21.15.12:0/2566985049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2e0106290 0x7fd2e01979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.728+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/2566985049 >> 172.21.15.12:0/2566985049 conn(0x7fd2e0076f40 msgr2=0x7fd2e00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.729+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/2566985049 shutdown_connections 2024-05-09T00:05:12.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.729+0000 7fd2e7c4f700 1 -- 172.21.15.12:0/2566985049 wait complete. 2024-05-09T00:05:12.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:05:12.767 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:12 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2989882444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:05:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.906+0000 7f22e7ebe700 1 -- 172.21.15.12:0/3797480401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0101540 msgr2=0x7f22e0104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.906+0000 7f22e7ebe700 1 --2- 172.21.15.12:0/3797480401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0101540 0x7f22e0104a70 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f22d00098f0 tx=0x7f22d0009c00 comp rx=0 tx=0).stop 2024-05-09T00:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.906+0000 7f22e7ebe700 1 -- 172.21.15.12:0/3797480401 shutdown_connections 2024-05-09T00:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.906+0000 7f22e7ebe700 1 --2- 172.21.15.12:0/3797480401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e01051a0 0x7f22e010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.906+0000 7f22e7ebe700 1 --2- 172.21.15.12:0/3797480401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e0100b90 0x7f22e0100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.906+0000 7f22e7ebe700 1 --2- 172.21.15.12:0/3797480401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0101540 0x7f22e0104a70 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.906+0000 7f22e7ebe700 1 -- 172.21.15.12:0/3797480401 >> 172.21.15.12:0/3797480401 conn(0x7f22e0076110 msgr2=0x7f22e0076520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.906+0000 7f22e7ebe700 1 -- 172.21.15.12:0/3797480401 shutdown_connections 2024-05-09T00:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.906+0000 7f22e7ebe700 1 -- 172.21.15.12:0/3797480401 wait complete. 2024-05-09T00:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.907+0000 7f22e7ebe700 1 Processor -- start 2024-05-09T00:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.907+0000 7f22e7ebe700 1 -- start start 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.907+0000 7f22e7ebe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100b90 0x7f22e01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.907+0000 7f22e7ebe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e01051a0 0x7f22e0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.907+0000 7f22e7ebe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0198c50 0x7f22e01a3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.907+0000 7f22e7ebe700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e01a38f0 con 0x7f22e0198c50 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.907+0000 7f22e7ebe700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e01a3a60 con 0x7f22e0100b90 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.907+0000 7f22e7ebe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e01a3c20 con 0x7f22e01051a0 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5c5a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100b90 0x7f22e01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5c5a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100b90 0x7f22e01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39100/0 (socket says 172.21.15.12:39100) 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5c5a700 1 -- 172.21.15.12:0/1942415489 learned_addr learned my addr 172.21.15.12:0/1942415489 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e645b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0198c50 0x7f22e01a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5459700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e01051a0 0x7f22e0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5c5a700 1 -- 172.21.15.12:0/1942415489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e01051a0 msgr2=0x7f22e0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e01051a0 0x7f22e0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5c5a700 1 -- 172.21.15.12:0/1942415489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0198c50 msgr2=0x7f22e01a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0198c50 0x7f22e01a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5c5a700 1 -- 172.21.15.12:0/1942415489 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc0095f0 con 0x7f22e0100b90 2024-05-09T00:05:12.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5459700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e01051a0 0x7f22e0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:12.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100b90 0x7f22e01981d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00ec10 tx=0x7f22dc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.908+0000 7f22d6ffd700 1 -- 172.21.15.12:0/1942415489 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00cc90 con 0x7f22e0100b90 2024-05-09T00:05:12.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.909+0000 7f22d6ffd700 1 -- 172.21.15.12:0/1942415489 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22dc0044e0 con 0x7f22e0100b90 2024-05-09T00:05:12.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.909+0000 7f22d6ffd700 1 -- 172.21.15.12:0/1942415489 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc010430 con 0x7f22e0100b90 2024-05-09T00:05:12.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.909+0000 7f22e7ebe700 1 -- 172.21.15.12:0/1942415489 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d0009520 con 0x7f22e0100b90 2024-05-09T00:05:12.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.909+0000 7f22e7ebe700 1 -- 172.21.15.12:0/1942415489 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e01a41a0 con 0x7f22e0100b90 2024-05-09T00:05:12.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.912+0000 7f22d6ffd700 1 -- 172.21.15.12:0/1942415489 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f22dc003f70 con 0x7f22e0100b90 2024-05-09T00:05:12.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.912+0000 7f22d6ffd700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22cc0a1310 0x7f22cc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.913+0000 7f22e5459700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22cc0a1310 0x7f22cc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.913+0000 7f22d6ffd700 1 -- 172.21.15.12:0/1942415489 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f22dc014070 con 0x7f22e0100b90 2024-05-09T00:05:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.913+0000 7f22e5459700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22cc0a1310 0x7f22cc0a37d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009810 tx=0x7f22d0019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22f4078b80 1 Processor -- start 2024-05-09T00:05:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22f4078b80 1 -- start start 2024-05-09T00:05:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17e1f0 0x55984d17e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d17eb10 0x55984d178760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d178ca0 0x55984d179100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22f4078b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d0cb150 con 0x55984d17e1f0 2024-05-09T00:05:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22f4078b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d0c8b10 con 0x55984d17eb10 2024-05-09T00:05:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22f4078b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d0c4400 con 0x55984d178ca0 2024-05-09T00:05:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22e5c5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17e1f0 0x55984d17e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22e5c5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17e1f0 0x55984d17e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56992/0 (socket says 172.21.15.12:56992) 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22e5c5a700 1 -- 172.21.15.12:0/946372062 learned_addr learned my addr 172.21.15.12:0/946372062 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22e5459700 1 --2- 172.21.15.12:0/946372062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d17eb10 0x55984d178760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.914+0000 7f22e645b700 1 --2- 172.21.15.12:0/946372062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d178ca0 0x55984d179100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.915+0000 7f22e5c5a700 1 -- 172.21.15.12:0/946372062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d178ca0 msgr2=0x55984d179100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.915+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/946372062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d178ca0 0x55984d179100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.915+0000 7f22e5c5a700 1 -- 172.21.15.12:0/946372062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d17eb10 msgr2=0x55984d178760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.915+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/946372062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d17eb10 0x55984d178760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.915+0000 7f22e5c5a700 1 -- 172.21.15.12:0/946372062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55984cf453b0 con 0x55984d17e1f0 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.915+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/946372062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17e1f0 0x55984d17e5d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0cb250 tx=0x7f22dc09a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3514fe82164ed153 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.915+0000 7f22cbfff700 1 -- 172.21.15.12:0/946372062 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0ba0d0 con 0x55984d17e1f0 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.915+0000 7f22cbfff700 1 -- 172.21.15.12:0/946372062 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22dc09ad10 con 0x55984d17e1f0 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.915+0000 7f22cbfff700 1 -- 172.21.15.12:0/946372062 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc004220 con 0x55984d17e1f0 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.916+0000 7f22f4078b80 1 -- 172.21.15.12:0/946372062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17e1f0 msgr2=0x55984d17e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.916+0000 7f22f4078b80 1 --2- 172.21.15.12:0/946372062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17e1f0 0x55984d17e5d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0cb250 tx=0x7f22dc09a1d0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.916+0000 7f22f4078b80 1 -- 172.21.15.12:0/946372062 shutdown_connections 2024-05-09T00:05:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.916+0000 7f22f4078b80 1 --2- 172.21.15.12:0/946372062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d178ca0 0x55984d179100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.916+0000 7f22f4078b80 1 --2- 172.21.15.12:0/946372062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17e1f0 0x55984d17e5d0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.916+0000 7f22f4078b80 1 --2- 172.21.15.12:0/946372062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d17eb10 0x55984d178760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.916+0000 7f22f4078b80 1 -- 172.21.15.12:0/946372062 >> 172.21.15.12:0/946372062 conn(0x55984d0d2210 msgr2=0x55984d16edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.916+0000 7f22f4078b80 1 -- 172.21.15.12:0/946372062 shutdown_connections 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.916+0000 7f22f4078b80 1 -- 172.21.15.12:0/946372062 wait complete. 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22f4078b80 1 -- start start 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22f4078b80 1 Processor -- start 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 0x55984d181f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d182490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d1829d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22f4078b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d060560 con 0x55984d17eb10 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22f4078b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d060200 con 0x55984d178ca0 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22f4078b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984cf476a0 con 0x55984d17e1f0 2024-05-09T00:05:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22f4078b80 10 client.0 _ll_register_callbacks cb 0x55984d069b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22e645b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d1829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22e645b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d1829d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56998/0 (socket says 172.21.15.12:56998) 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22e645b700 1 -- 172.21.15.12:0/2674505021 learned_addr learned my addr 172.21.15.12:0/2674505021 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.917+0000 7f22e5459700 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d182490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 0x55984d181f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22e645b700 1 -- 172.21.15.12:0/2674505021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 msgr2=0x55984d182490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22e645b700 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d182490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22e645b700 1 -- 172.21.15.12:0/2674505021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 msgr2=0x55984d181f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22e645b700 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 0x55984d181f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22e645b700 1 -- 172.21.15.12:0/2674505021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e01a41a0 con 0x55984d17eb10 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 0x55984d181f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22e645b700 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d1829d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f22d8012e80 tx=0x7f22d8014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3030012c7f0694 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22c97fa700 1 -- 172.21.15.12:0/2674505021 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800e040 con 0x55984d17eb10 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22f4078b80 1 -- 172.21.15.12:0/2674505021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55984d01d270 con 0x55984d17eb10 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22c97fa700 1 -- 172.21.15.12:0/2674505021 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f22d8014b90 con 0x55984d17eb10 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.918+0000 7f22c97fa700 1 -- 172.21.15.12:0/2674505021 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d8014d50 con 0x55984d17eb10 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22f4078b80 10 client.20646 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22f4078b80 1 -- 172.21.15.12:0/2674505021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55984cef1b60 con 0x55984d17eb10 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22c97fa700 1 -- 172.21.15.12:0/2674505021 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55984d01d270 con 0x55984d17eb10 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22c97fa700 1 -- 172.21.15.12:0/2674505021 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f22d801c020 con 0x55984d17eb10 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22f4078b80 10 client.20646 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22f4078b80 10 client.20646 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22f4078b80 20 client.20646 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22f4078b80 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55984d1a3680 0x55984d1a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22f4078b80 4 client.20646 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22f4078b80 1 -- 172.21.15.12:0/2674505021 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55984d143810 con 0x55984d1a3680 2024-05-09T00:05:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.919+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55984d1a3680 0x55984d1a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.920+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55984d1a3680 0x55984d1a5aa0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d18cab990e7c4ca9 server_cookie=71da229c08c5921f in_seq=0 out_seq=0 2024-05-09T00:05:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.920+0000 7f22c97fa700 10 client.20646 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:05:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22c97fa700 1 -- 172.21.15.12:0/2674505021 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55984d143810 con 0x55984d1a3680 2024-05-09T00:05:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22c97fa700 10 client.20646 handle_command_reply: tid=0 2024-05-09T00:05:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 client.20646 shutdown 2024-05-09T00:05:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 -- 172.21.15.12:0/2674505021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 msgr2=0x55984d1829d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d1829d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f22d8012e80 tx=0x7f22d8014600 comp rx=0 tx=0).stop 2024-05-09T00:05:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 -- 172.21.15.12:0/2674505021 shutdown_connections 2024-05-09T00:05:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55984d1a3680 0x55984d1a5aa0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d1829d0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 0x55984d181f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22c97fa700 0 client.20646 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:05:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 --2- 172.21.15.12:0/2674505021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d182490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 -- 172.21.15.12:0/2674505021 >> 172.21.15.12:0/2674505021 conn(0x55984d0d2210 msgr2=0x55984d16e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 -- 172.21.15.12:0/2674505021 shutdown_connections 2024-05-09T00:05:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 1 -- 172.21.15.12:0/2674505021 wait complete. 2024-05-09T00:05:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.924+0000 7f22f4078b80 20 client.20646 trim_cache size 0 max 16384 2024-05-09T00:05:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.940+0000 7f22f4078b80 1 Processor -- start 2024-05-09T00:05:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22f4078b80 1 -- start start 2024-05-09T00:05:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d178ca0 0x55984d16ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d17c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d17eb10 0x55984d17c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22f4078b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d16f8e0 con 0x55984d178ca0 2024-05-09T00:05:12.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22f4078b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d1a0220 con 0x55984d17eb10 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22f4078b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d1835c0 con 0x55984d17e1f0 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22e5c5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d178ca0 0x55984d16ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22e5c5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d178ca0 0x55984d16ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57014/0 (socket says 172.21.15.12:57014) 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22e5c5a700 1 -- 172.21.15.12:0/1275022171 learned_addr learned my addr 172.21.15.12:0/1275022171 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22e5459700 1 --2- 172.21.15.12:0/1275022171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d17c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22e645b700 1 --2- 172.21.15.12:0/1275022171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d17eb10 0x55984d17c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22e5c5a700 1 -- 172.21.15.12:0/1275022171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 msgr2=0x55984d17c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/1275022171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d17c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22e5c5a700 1 -- 172.21.15.12:0/1275022171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d17eb10 msgr2=0x55984d17c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/1275022171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d17eb10 0x55984d17c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.941+0000 7f22e5c5a700 1 -- 172.21.15.12:0/1275022171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55984d08a560 con 0x55984d178ca0 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.942+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/1275022171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d178ca0 0x55984d16ed20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c5b0 tx=0x7f22dc085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2324e13e081e3d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.942+0000 7f22d4ff9700 1 -- 172.21.15.12:0/1275022171 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc009a10 con 0x55984d178ca0 2024-05-09T00:05:12.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.942+0000 7f22d4ff9700 1 -- 172.21.15.12:0/1275022171 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22dc0a2a90 con 0x55984d178ca0 2024-05-09T00:05:12.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.942+0000 7f22d4ff9700 1 -- 172.21.15.12:0/1275022171 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0850b0 con 0x55984d178ca0 2024-05-09T00:05:12.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 -- 172.21.15.12:0/1275022171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d178ca0 msgr2=0x55984d16ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 --2- 172.21.15.12:0/1275022171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d178ca0 0x55984d16ed20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c5b0 tx=0x7f22dc085040 comp rx=0 tx=0).stop 2024-05-09T00:05:12.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 -- 172.21.15.12:0/1275022171 shutdown_connections 2024-05-09T00:05:12.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 --2- 172.21.15.12:0/1275022171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d17eb10 0x55984d17c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 --2- 172.21.15.12:0/1275022171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d178ca0 0x55984d16ed20 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 --2- 172.21.15.12:0/1275022171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d17c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 -- 172.21.15.12:0/1275022171 >> 172.21.15.12:0/1275022171 conn(0x55984d0d2210 msgr2=0x55984d1aa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 -- 172.21.15.12:0/1275022171 shutdown_connections 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 -- 172.21.15.12:0/1275022171 wait complete. 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 -- start start 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.943+0000 7f22f4078b80 1 Processor -- start 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 0x55984d189a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d187540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22f4078b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d187c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22f4078b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d1883c0 con 0x55984d17eb10 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22f4078b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d177380 con 0x55984d178ca0 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22f4078b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d1774f0 con 0x55984d17e1f0 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22f4078b80 10 client.0 _ll_register_callbacks cb 0x55984d069b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e645b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d187c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e5c5a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 0x55984d189a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e645b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d187c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57026/0 (socket says 172.21.15.12:57026) 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e5459700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d187540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e645b700 1 -- 172.21.15.12:0/3191875816 learned_addr learned my addr 172.21.15.12:0/3191875816 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e645b700 1 -- 172.21.15.12:0/3191875816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 msgr2=0x55984d187540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e645b700 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d187540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e645b700 1 -- 172.21.15.12:0/3191875816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 msgr2=0x55984d189a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e645b700 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 0x55984d189a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e645b700 1 -- 172.21.15.12:0/3191875816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55984cf453b0 con 0x55984d17eb10 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22e645b700 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d187c80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f22d8012ba0 tx=0x7f22d8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55828fc4075038f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.944+0000 7f22c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22c97fa700 1 -- 172.21.15.12:0/3191875816 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800e040 con 0x55984d17eb10 2024-05-09T00:05:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22c97fa700 1 -- 172.21.15.12:0/3191875816 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f22d8004b40 con 0x55984d17eb10 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22f4078b80 1 -- 172.21.15.12:0/3191875816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22dc09ad10 con 0x55984d17eb10 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22c97fa700 1 -- 172.21.15.12:0/3191875816 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d8004d00 con 0x55984d17eb10 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22f4078b80 10 client.20652 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22f4078b80 1 -- 172.21.15.12:0/3191875816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55984cf49cd0 con 0x55984d17eb10 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22c97fa700 1 -- 172.21.15.12:0/3191875816 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22dc09ad10 con 0x55984d17eb10 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22c97fa700 1 -- 172.21.15.12:0/3191875816 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f22d8038070 con 0x55984d17eb10 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22f4078b80 10 client.20652 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22f4078b80 10 client.20652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22f4078b80 20 client.20652 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22f4078b80 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55984d1f0350 0x55984d1f2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22f4078b80 4 client.20652 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.945+0000 7f22f4078b80 1 -- 172.21.15.12:0/3191875816 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55984d0bcec0 con 0x55984d1f0350 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.946+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55984d1f0350 0x55984d1f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.946+0000 7f22c97fa700 10 client.20652 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:05:12.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.946+0000 7f22e5c5a700 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55984d1f0350 0x55984d1f2740 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fae6477add286e0d server_cookie=167e4fed2369ba3b in_seq=0 out_seq=0 2024-05-09T00:05:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22c97fa700 1 -- 172.21.15.12:0/3191875816 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55984d0bcec0 con 0x55984d1f0350 2024-05-09T00:05:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22c97fa700 10 client.20652 handle_command_reply: tid=0 2024-05-09T00:05:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22f4078b80 1 client.20652 shutdown 2024-05-09T00:05:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22f4078b80 1 -- 172.21.15.12:0/3191875816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 msgr2=0x55984d187c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22f4078b80 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d187c80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f22d8012ba0 tx=0x7f22d8009510 comp rx=0 tx=0).stop 2024-05-09T00:05:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22f4078b80 1 -- 172.21.15.12:0/3191875816 shutdown_connections 2024-05-09T00:05:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22f4078b80 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55984d1f0350 0x55984d1f2740 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22f4078b80 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55984d17eb10 0x55984d187c80 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22f4078b80 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55984d178ca0 0x55984d189a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22f4078b80 1 --2- 172.21.15.12:0/3191875816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55984d17e1f0 0x55984d187540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22c97fa700 0 client.20652 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:05:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.951+0000 7f22f4078b80 1 -- 172.21.15.12:0/3191875816 >> 172.21.15.12:0/3191875816 conn(0x55984d0d2210 msgr2=0x55984d17d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22f4078b80 1 -- 172.21.15.12:0/3191875816 shutdown_connections 2024-05-09T00:05:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22f4078b80 1 -- 172.21.15.12:0/3191875816 wait complete. 2024-05-09T00:05:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22f4078b80 20 client.20652 trim_cache size 0 max 16384 2024-05-09T00:05:12.947 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:05:12.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22e7ebe700 1 -- 172.21.15.12:0/1942415489 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22cc0a1310 msgr2=0x7f22cc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22e7ebe700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22cc0a1310 0x7f22cc0a37d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009810 tx=0x7f22d0019040 comp rx=0 tx=0).stop 2024-05-09T00:05:12.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22e7ebe700 1 -- 172.21.15.12:0/1942415489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100b90 msgr2=0x7f22e01981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:12.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22e7ebe700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100b90 0x7f22e01981d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00ec10 tx=0x7f22dc00ef20 comp rx=0 tx=0).stop 2024-05-09T00:05:12.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22e7ebe700 1 -- 172.21.15.12:0/1942415489 shutdown_connections 2024-05-09T00:05:12.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22e7ebe700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f22cc0a1310 0x7f22cc0a37d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22e7ebe700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22e0198c50 0x7f22e01a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22e7ebe700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22e0100b90 0x7f22e01981d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22e7ebe700 1 --2- 172.21.15.12:0/1942415489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e01051a0 0x7f22e0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:12.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.952+0000 7f22e7ebe700 1 -- 172.21.15.12:0/1942415489 >> 172.21.15.12:0/1942415489 conn(0x7f22e0076110 msgr2=0x7f22e00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:12.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.953+0000 7f22e7ebe700 1 -- 172.21.15.12:0/1942415489 shutdown_connections 2024-05-09T00:05:12.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:12.953+0000 7f22e7ebe700 1 -- 172.21.15.12:0/1942415489 wait complete. 2024-05-09T00:05:12.961 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:05:13.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:12 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2989882444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:05:13.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:12 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2989882444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:05:13.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.189+0000 7f7c6e862700 1 -- 172.21.15.12:0/1779961833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c68108480 msgr2=0x7f7c68110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.189+0000 7f7c6e862700 1 --2- 172.21.15.12:0/1779961833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c68108480 0x7f7c68110000 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000b0a0 tx=0x7f7c6000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.190+0000 7f7c6e862700 1 -- 172.21.15.12:0/1779961833 shutdown_connections 2024-05-09T00:05:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.190+0000 7f7c6e862700 1 --2- 172.21.15.12:0/1779961833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c68108480 0x7f7c68110000 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.190+0000 7f7c6e862700 1 --2- 172.21.15.12:0/1779961833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68103360 0x7f7c68103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.190+0000 7f7c6e862700 1 --2- 172.21.15.12:0/1779961833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c68103d10 0x7f7c68107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.190+0000 7f7c6e862700 1 -- 172.21.15.12:0/1779961833 >> 172.21.15.12:0/1779961833 conn(0x7f7c680feb80 msgr2=0x7f7c68100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.190+0000 7f7c6e862700 1 -- 172.21.15.12:0/1779961833 shutdown_connections 2024-05-09T00:05:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.190+0000 7f7c6e862700 1 -- 172.21.15.12:0/1779961833 wait complete. 2024-05-09T00:05:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c6e862700 1 Processor -- start 2024-05-09T00:05:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c6e862700 1 -- start start 2024-05-09T00:05:13.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c6e862700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68103360 0x7f7c68199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c6e862700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c68108480 0x7f7c68199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c6e862700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c6819e000 0x7f7c6819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c6e862700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6819ea80 con 0x7f7c6819e000 2024-05-09T00:05:13.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c6e862700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6819ebf0 con 0x7f7c68108480 2024-05-09T00:05:13.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c6e862700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6819edb0 con 0x7f7c68103360 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c68108480 0x7f7c68199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c6cdff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c6819e000 0x7f7c6819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c68108480 0x7f7c68199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39152/0 (socket says 172.21.15.12:39152) 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c677fe700 1 -- 172.21.15.12:0/3833677059 learned_addr learned my addr 172.21.15.12:0/3833677059 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.191+0000 7f7c67fff700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68103360 0x7f7c68199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c677fe700 1 -- 172.21.15.12:0/3833677059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68103360 msgr2=0x7f7c68199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c677fe700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68103360 0x7f7c68199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c677fe700 1 -- 172.21.15.12:0/3833677059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c6819e000 msgr2=0x7f7c6819e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c677fe700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c6819e000 0x7f7c6819e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c677fe700 1 -- 172.21.15.12:0/3833677059 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c60009d00 con 0x7f7c68108480 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c67fff700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68103360 0x7f7c68199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c677fe700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c68108480 0x7f7c68199c60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00e970 tx=0x7f7c5c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c657fa700 1 -- 172.21.15.12:0/3833677059 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c009790 con 0x7f7c68108480 2024-05-09T00:05:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c6e862700 1 -- 172.21.15.12:0/3833677059 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c681a6310 con 0x7f7c68108480 2024-05-09T00:05:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c657fa700 1 -- 172.21.15.12:0/3833677059 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c5c00c7e0 con 0x7f7c68108480 2024-05-09T00:05:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.192+0000 7f7c657fa700 1 -- 172.21.15.12:0/3833677059 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c0054e0 con 0x7f7c68108480 2024-05-09T00:05:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.193+0000 7f7c6e862700 1 -- 172.21.15.12:0/3833677059 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c681a6840 con 0x7f7c68108480 2024-05-09T00:05:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.195+0000 7f7c657fa700 1 -- 172.21.15.12:0/3833677059 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7c5c00c950 con 0x7f7c68108480 2024-05-09T00:05:13.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.196+0000 7f7c657fa700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7c500a1310 0x7f7c500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.196+0000 7f7c657fa700 1 -- 172.21.15.12:0/3833677059 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7c5c014070 con 0x7f7c68108480 2024-05-09T00:05:13.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.196+0000 7f7c67fff700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7c500a1310 0x7f7c500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.197+0000 7f7c67fff700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7c500a1310 0x7f7c500a37d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7c58011490 tx=0x7f7c58009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:13.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c7aa1cb80 1 Processor -- start 2024-05-09T00:05:13.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c7aa1cb80 1 -- start start 2024-05-09T00:05:13.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45660 0x55862cf45a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 0x55862cf3fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf40570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ce8e150 con 0x55862cf45f80 2024-05-09T00:05:13.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ce8bb10 con 0x55862cf45660 2024-05-09T00:05:13.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ce87400 con 0x55862cf400f0 2024-05-09T00:05:13.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c67fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45660 0x55862cf45a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c67fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45660 0x55862cf45a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39164/0 (socket says 172.21.15.12:39164) 2024-05-09T00:05:13.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c67fff700 1 -- 172.21.15.12:0/2234264648 learned_addr learned my addr 172.21.15.12:0/2234264648 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:13.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.198+0000 7f7c677fe700 1 --2- 172.21.15.12:0/2234264648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 0x55862cf3fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.199+0000 7f7c67fff700 1 -- 172.21.15.12:0/2234264648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 msgr2=0x55862cf40570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:13.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.199+0000 7f7c67fff700 1 --2- 172.21.15.12:0/2234264648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf40570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.199+0000 7f7c67fff700 1 -- 172.21.15.12:0/2234264648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 msgr2=0x55862cf3fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:13.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.199+0000 7f7c67fff700 1 --2- 172.21.15.12:0/2234264648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 0x55862cf3fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.199+0000 7f7c67fff700 1 -- 172.21.15.12:0/2234264648 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55862cd083b0 con 0x55862cf45660 2024-05-09T00:05:13.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.199+0000 7f7c67fff700 1 --2- 172.21.15.12:0/2234264648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45660 0x55862cf45a40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7c580043c0 tx=0x7f7c58014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2855fbe5a8c1ed83 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:13.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.199+0000 7f7c567fc700 1 -- 172.21.15.12:0/2234264648 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c58003d10 con 0x55862cf45660 2024-05-09T00:05:13.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.199+0000 7f7c567fc700 1 -- 172.21.15.12:0/2234264648 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c58003e70 con 0x55862cf45660 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.199+0000 7f7c567fc700 1 -- 172.21.15.12:0/2234264648 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5801e7e0 con 0x55862cf45660 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.200+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/2234264648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45660 msgr2=0x55862cf45a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.200+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/2234264648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45660 0x55862cf45a40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7c580043c0 tx=0x7f7c58014000 comp rx=0 tx=0).stop 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.200+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/2234264648 shutdown_connections 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.200+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/2234264648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf40570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.200+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/2234264648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45660 0x55862cf45a40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.200+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/2234264648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 0x55862cf3fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.200+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/2234264648 >> 172.21.15.12:0/2234264648 conn(0x55862ce95210 msgr2=0x55862cf31df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.201+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/2234264648 shutdown_connections 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.201+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/2234264648 wait complete. 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.201+0000 7f7c7aa1cb80 1 -- start start 2024-05-09T00:05:13.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c7aa1cb80 1 Processor -- start 2024-05-09T00:05:13.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf49a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf49f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 0x55862cf3e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ce23560 con 0x55862cf45660 2024-05-09T00:05:13.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ce23200 con 0x55862cf45f80 2024-05-09T00:05:13.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862cd0a6a0 con 0x55862cf400f0 2024-05-09T00:05:13.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c7aa1cb80 10 client.0 _ll_register_callbacks cb 0x55862ce2cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf49f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf49f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57058/0 (socket says 172.21.15.12:57058) 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.202+0000 7f7c677fe700 1 -- 172.21.15.12:0/3359375214 learned_addr learned my addr 172.21.15.12:0/3359375214 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.203+0000 7f7c67fff700 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf49a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.203+0000 7f7c6cdff700 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 0x55862cf3e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.203+0000 7f7c6cdff700 1 -- 172.21.15.12:0/3359375214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 msgr2=0x55862cf49a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.203+0000 7f7c6cdff700 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf49a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.203+0000 7f7c6cdff700 1 -- 172.21.15.12:0/3359375214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 msgr2=0x55862cf49f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.203+0000 7f7c6cdff700 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf49f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.203+0000 7f7c6cdff700 1 -- 172.21.15.12:0/3359375214 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c58017070 con 0x55862cf45f80 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.203+0000 7f7c6cdff700 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 0x55862cf3e870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000b780 tx=0x7f7c60007e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d077e0f79ad20e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.203+0000 7f7c4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.204+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/3359375214 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000e040 con 0x55862cf45f80 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.204+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/3359375214 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7c600074e0 con 0x55862cf45f80 2024-05-09T00:05:13.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.204+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3359375214 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55862cde0270 con 0x55862cf45f80 2024-05-09T00:05:13.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.204+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/3359375214 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c6000be60 con 0x55862cf45f80 2024-05-09T00:05:13.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.204+0000 7f7c7aa1cb80 10 client.30227 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:13.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.204+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3359375214 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55862ccb4b60 con 0x55862cf45f80 2024-05-09T00:05:13.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.204+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/3359375214 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55862cde0270 con 0x55862cf45f80 2024-05-09T00:05:13.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.204+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/3359375214 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7c60019030 con 0x55862cf45f80 2024-05-09T00:05:13.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.204+0000 7f7c7aa1cb80 10 client.30227 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:13.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.204+0000 7f7c7aa1cb80 10 client.30227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:13.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.205+0000 7f7c7aa1cb80 20 client.30227 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:13.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.205+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55862cf731f0 0x55862cf75610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.205+0000 7f7c7aa1cb80 4 client.30227 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:13.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.205+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3359375214 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55862cf06810 con 0x55862cf731f0 2024-05-09T00:05:13.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.205+0000 7f7c67fff700 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55862cf731f0 0x55862cf75610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.205+0000 7f7c4b7fe700 10 client.30227 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.205+0000 7f7c67fff700 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55862cf731f0 0x55862cf75610 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6abd7b69d0ee5eba server_cookie=660757c14008d1b0 in_seq=0 out_seq=0 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/3359375214 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55862cf06810 con 0x55862cf731f0 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c4b7fe700 10 client.30227 handle_command_reply: tid=0 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c7aa1cb80 1 client.30227 shutdown 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3359375214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 msgr2=0x55862cf3e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 0x55862cf3e870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000b780 tx=0x7f7c60007e30 comp rx=0 tx=0).stop 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3359375214 shutdown_connections 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55862cf731f0 0x55862cf75610 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 0x55862cf3e870 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf49a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c4b7fe700 0 client.30227 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/3359375214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf49f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.206+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3359375214 >> 172.21.15.12:0/3359375214 conn(0x55862ce95210 msgr2=0x55862cf315a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.207+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3359375214 shutdown_connections 2024-05-09T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.207+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3359375214 wait complete. 2024-05-09T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.207+0000 7f7c7aa1cb80 20 client.30227 trim_cache size 0 max 16384 2024-05-09T00:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.222+0000 7f7c7aa1cb80 1 Processor -- start 2024-05-09T00:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.222+0000 7f7c7aa1cb80 1 -- start start 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.222+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf43070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.222+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf47370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.222+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 0x55862cf31d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.222+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862cf495f0 con 0x55862cf45660 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.222+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862cf47d70 con 0x55862cf45f80 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.222+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862cf32260 con 0x55862cf400f0 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf47370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf47370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57068/0 (socket says 172.21.15.12:57068) 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c677fe700 1 -- 172.21.15.12:0/3515044519 learned_addr learned my addr 172.21.15.12:0/3515044519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c6cdff700 1 --2- 172.21.15.12:0/3515044519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 0x55862cf31d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c67fff700 1 --2- 172.21.15.12:0/3515044519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf43070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c677fe700 1 -- 172.21.15.12:0/3515044519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 msgr2=0x55862cf43070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c677fe700 1 --2- 172.21.15.12:0/3515044519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf43070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c677fe700 1 -- 172.21.15.12:0/3515044519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 msgr2=0x55862cf31d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c677fe700 1 --2- 172.21.15.12:0/3515044519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 0x55862cf31d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c677fe700 1 -- 172.21.15.12:0/3515044519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55862ce4d560 con 0x55862cf45660 2024-05-09T00:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c677fe700 1 --2- 172.21.15.12:0/3515044519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf47370 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c096a20 tx=0x7f7c5c005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b00d3a03a89b1d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c56ffd700 1 -- 172.21.15.12:0/3515044519 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c09a6b0 con 0x55862cf45660 2024-05-09T00:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c56ffd700 1 -- 172.21.15.12:0/3515044519 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c5c0b1070 con 0x55862cf45660 2024-05-09T00:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.223+0000 7f7c56ffd700 1 -- 172.21.15.12:0/3515044519 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c09a210 con 0x55862cf45660 2024-05-09T00:05:13.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.224+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3515044519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 msgr2=0x55862cf47370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:13.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.224+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/3515044519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf47370 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c096a20 tx=0x7f7c5c005910 comp rx=0 tx=0).stop 2024-05-09T00:05:13.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.224+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3515044519 shutdown_connections 2024-05-09T00:05:13.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.224+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/3515044519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf45f80 0x55862cf31d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.224+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/3515044519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf400f0 0x55862cf43070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.224+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/3515044519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45660 0x55862cf47370 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.224+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3515044519 >> 172.21.15.12:0/3515044519 conn(0x55862ce95210 msgr2=0x55862cf2e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:13.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.224+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3515044519 shutdown_connections 2024-05-09T00:05:13.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.224+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/3515044519 wait complete. 2024-05-09T00:05:13.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c7aa1cb80 1 -- start start 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c7aa1cb80 1 Processor -- start 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf400f0 0x55862cf3b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf45660 0x55862cf3bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c7aa1cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 0x55862cf39bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862cf3a2d0 con 0x55862cf45f80 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862cf3a440 con 0x55862cf400f0 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c7aa1cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862cf3a600 con 0x55862cf45660 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c7aa1cb80 10 client.0 _ll_register_callbacks cb 0x55862ce2cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c67fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf400f0 0x55862cf3b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c6cdff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 0x55862cf39bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.225+0000 7f7c677fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf45660 0x55862cf3bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c67fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf400f0 0x55862cf3b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39188/0 (socket says 172.21.15.12:39188) 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c67fff700 1 -- 172.21.15.12:0/42183751 learned_addr learned my addr 172.21.15.12:0/42183751 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:13.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c677fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf45660 0x55862cf3bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48338/0 (socket says 172.21.15.12:48338) 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c6cdff700 1 -- 172.21.15.12:0/42183751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf45660 msgr2=0x55862cf3bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c6cdff700 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf45660 0x55862cf3bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c6cdff700 1 -- 172.21.15.12:0/42183751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf400f0 msgr2=0x55862cf3b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c6cdff700 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf400f0 0x55862cf3b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c6cdff700 1 -- 172.21.15.12:0/42183751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c681a6840 con 0x55862cf45f80 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c67fff700 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf400f0 0x55862cf3b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c6cdff700 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 0x55862cf39bc0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009fd0 tx=0x7f7c6001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849f4a1558f67ca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/42183751 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000e040 con 0x55862cf45f80 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/42183751 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7c600076b0 con 0x55862cf45f80 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/42183751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c58003e70 con 0x55862cf45f80 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/42183751 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c600078b0 con 0x55862cf45f80 2024-05-09T00:05:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c7aa1cb80 10 client.20670 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.226+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/42183751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55862cd0ccd0 con 0x55862cf45f80 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.227+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/42183751 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c6003d020 con 0x55862cf45f80 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.227+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/42183751 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7c6000b420 con 0x55862cf45f80 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.227+0000 7f7c7aa1cb80 10 client.20670 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.227+0000 7f7c7aa1cb80 10 client.20670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.227+0000 7f7c7aa1cb80 20 client.20670 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.227+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55862cfca470 0x55862cfcc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.227+0000 7f7c7aa1cb80 4 client.20670 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.227+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/42183751 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55862ce7fec0 con 0x55862cfca470 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.227+0000 7f7c67fff700 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55862cfca470 0x55862cfcc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.228+0000 7f7c67fff700 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55862cfca470 0x55862cfcc860 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31463b51aea88fe6 server_cookie=850928cca488c49e in_seq=0 out_seq=0 2024-05-09T00:05:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.228+0000 7f7c4b7fe700 10 client.20670 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c4b7fe700 1 -- 172.21.15.12:0/42183751 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55862ce7fec0 con 0x55862cfca470 2024-05-09T00:05:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c4b7fe700 10 client.20670 handle_command_reply: tid=0 2024-05-09T00:05:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 client.20670 shutdown 2024-05-09T00:05:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/42183751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 msgr2=0x55862cf39bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 0x55862cf39bc0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009fd0 tx=0x7f7c6001ca90 comp rx=0 tx=0).stop 2024-05-09T00:05:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/42183751 shutdown_connections 2024-05-09T00:05:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55862cf45f80 0x55862cf39bc0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55862cfca470 0x55862cfcc860 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55862cf400f0 0x55862cf3b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 --2- 172.21.15.12:0/42183751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862cf45660 0x55862cf3bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/42183751 >> 172.21.15.12:0/42183751 conn(0x55862ce95210 msgr2=0x55862cf79f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c4b7fe700 0 client.20670 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/42183751 shutdown_connections 2024-05-09T00:05:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.229+0000 7f7c7aa1cb80 1 -- 172.21.15.12:0/42183751 wait complete. 2024-05-09T00:05:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.230+0000 7f7c7aa1cb80 20 client.20670 trim_cache size 0 max 16384 2024-05-09T00:05:13.228 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:05:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 -- 172.21.15.12:0/3833677059 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7c500a1310 msgr2=0x7f7c500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7c500a1310 0x7f7c500a37d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7c58011490 tx=0x7f7c58009260 comp rx=0 tx=0).stop 2024-05-09T00:05:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 -- 172.21.15.12:0/3833677059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c68108480 msgr2=0x7f7c68199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c68108480 0x7f7c68199c60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00e970 tx=0x7f7c5c00ec80 comp rx=0 tx=0).stop 2024-05-09T00:05:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 -- 172.21.15.12:0/3833677059 shutdown_connections 2024-05-09T00:05:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7c500a1310 0x7f7c500a37d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c6819e000 0x7f7c6819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68103360 0x7f7c68199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 --2- 172.21.15.12:0/3833677059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c68108480 0x7f7c68199c60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 -- 172.21.15.12:0/3833677059 >> 172.21.15.12:0/3833677059 conn(0x7f7c680feb80 msgr2=0x7f7c68100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 -- 172.21.15.12:0/3833677059 shutdown_connections 2024-05-09T00:05:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:13.231+0000 7f7c6e862700 1 -- 172.21.15.12:0/3833677059 wait complete. 2024-05-09T00:05:13.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 208856 34.94 MB/sec execute 188 sec latency 192.017 ms 2024-05-09T00:05:14.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:13 smithi204 ceph-mon[47373]: pgmap v605: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-05-09T00:05:14.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:13 smithi160 ceph-mon[46152]: pgmap v605: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-05-09T00:05:14.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:13 smithi012 ceph-mon[42503]: pgmap v605: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-05-09T00:05:14.241 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:05:14.242 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:05:14.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 209539 34.97 MB/sec execute 189 sec latency 108.252 ms 2024-05-09T00:05:14.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.437+0000 7f08d76bd700 1 -- 172.21.15.12:0/2405632638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08d0107560 msgr2=0x7f08d01079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.437+0000 7f08d76bd700 1 --2- 172.21.15.12:0/2405632638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08d0107560 0x7f08d01079e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009990 tx=0x7f08c0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.437+0000 7f08d76bd700 1 -- 172.21.15.12:0/2405632638 shutdown_connections 2024-05-09T00:05:14.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.437+0000 7f08d76bd700 1 --2- 172.21.15.12:0/2405632638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08d0107f20 0x7f08d0114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.437+0000 7f08d76bd700 1 --2- 172.21.15.12:0/2405632638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08d010c580 0x7f08d010c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.437+0000 7f08d76bd700 1 --2- 172.21.15.12:0/2405632638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08d0107560 0x7f08d01079e0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.437+0000 7f08d76bd700 1 -- 172.21.15.12:0/2405632638 >> 172.21.15.12:0/2405632638 conn(0x7f08d0076d90 msgr2=0x7f08d0105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.437+0000 7f08d76bd700 1 -- 172.21.15.12:0/2405632638 shutdown_connections 2024-05-09T00:05:14.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.437+0000 7f08d76bd700 1 -- 172.21.15.12:0/2405632638 wait complete. 2024-05-09T00:05:14.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.438+0000 7f08d76bd700 1 Processor -- start 2024-05-09T00:05:14.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.438+0000 7f08d76bd700 1 -- start start 2024-05-09T00:05:14.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.438+0000 7f08d76bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08d0107560 0x7f08d01a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.438+0000 7f08d76bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08d0107f20 0x7f08d01a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.438+0000 7f08d76bd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08d010c580 0x7f08d019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.438+0000 7f08d76bd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019de20 con 0x7f08d0107f20 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.438+0000 7f08d76bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019df90 con 0x7f08d0107560 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.438+0000 7f08d76bd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019e150 con 0x7f08d010c580 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.438+0000 7f08d4c58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08d0107f20 0x7f08d01a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.438+0000 7f08d5459700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08d0107560 0x7f08d01a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d4c58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08d0107f20 0x7f08d01a3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57094/0 (socket says 172.21.15.12:57094) 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d5459700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08d0107560 0x7f08d01a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39198/0 (socket says 172.21.15.12:39198) 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d4c58700 1 -- 172.21.15.12:0/2544334118 learned_addr learned my addr 172.21.15.12:0/2544334118 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d5c5a700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08d010c580 0x7f08d019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d4c58700 1 -- 172.21.15.12:0/2544334118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08d010c580 msgr2=0x7f08d019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d4c58700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08d010c580 0x7f08d019d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d4c58700 1 -- 172.21.15.12:0/2544334118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08d0107560 msgr2=0x7f08d01a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d4c58700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08d0107560 0x7f08d01a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d4c58700 1 -- 172.21.15.12:0/2544334118 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00095f0 con 0x7f08d0107f20 2024-05-09T00:05:14.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d5459700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08d0107560 0x7f08d01a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d5c5a700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08d010c580 0x7f08d019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d4c58700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08d0107f20 0x7f08d01a3bc0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f08c0000c00 tx=0x7f08c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08c67fc700 1 -- 172.21.15.12:0/2544334118 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c001d070 con 0x7f08d0107f20 2024-05-09T00:05:14.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08c67fc700 1 -- 172.21.15.12:0/2544334118 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08c0022470 con 0x7f08d0107f20 2024-05-09T00:05:14.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d76bd700 1 -- 172.21.15.12:0/2544334118 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d019e3d0 con 0x7f08d0107f20 2024-05-09T00:05:14.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08c67fc700 1 -- 172.21.15.12:0/2544334118 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0017650 con 0x7f08d0107f20 2024-05-09T00:05:14.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.439+0000 7f08d76bd700 1 -- 172.21.15.12:0/2544334118 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d01aa670 con 0x7f08d0107f20 2024-05-09T00:05:14.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.442+0000 7f08c67fc700 1 -- 172.21.15.12:0/2544334118 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f08c00225e0 con 0x7f08d0107f20 2024-05-09T00:05:14.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.442+0000 7f08c67fc700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08bc0a1290 0x7f08bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.443+0000 7f08c67fc700 1 -- 172.21.15.12:0/2544334118 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f08c00cde50 con 0x7f08d0107f20 2024-05-09T00:05:14.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.443+0000 7f08d5459700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08bc0a1290 0x7f08bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.443+0000 7f08d5459700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08bc0a1290 0x7f08bc0a3750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f08d006a400 tx=0x7f08cc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.444+0000 7f08e3877b80 1 Processor -- start 2024-05-09T00:05:14.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.444+0000 7f08e3877b80 1 -- start start 2024-05-09T00:05:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.444+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.444+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x55811629e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.444+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x55811629ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.444+0000 7f08e3877b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581161f51d0 con 0x5581162a3e80 2024-05-09T00:05:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.444+0000 7f08e3877b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581161f2b90 con 0x55811629e910 2024-05-09T00:05:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.444+0000 7f08e3877b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581161ee480 con 0x5581162a47a0 2024-05-09T00:05:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5c5a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x55811629ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5459700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d4c58700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x55811629e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5c5a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x55811629ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39212/0 (socket says 172.21.15.12:39212) 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d4c58700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x55811629e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48368/0 (socket says 172.21.15.12:48368) 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5459700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57104/0 (socket says 172.21.15.12:57104) 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5c5a700 1 -- 172.21.15.12:0/1893712977 learned_addr learned my addr 172.21.15.12:0/1893712977 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5c5a700 1 -- 172.21.15.12:0/1893712977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 msgr2=0x55811629e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5c5a700 1 --2- 172.21.15.12:0/1893712977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x55811629e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5c5a700 1 -- 172.21.15.12:0/1893712977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 msgr2=0x5581162a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5c5a700 1 --2- 172.21.15.12:0/1893712977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5c5a700 1 -- 172.21.15.12:0/1893712977 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55811606f430 con 0x55811629e910 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5459700 1 --2- 172.21.15.12:0/1893712977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d4c58700 1 --2- 172.21.15.12:0/1893712977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x55811629e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08d5c5a700 1 --2- 172.21.15.12:0/1893712977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x55811629ed50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f08c8003e80 tx=0x7f08c8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=711b597ab0b3492c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08bb7fe700 1 -- 172.21.15.12:0/1893712977 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c80092d0 con 0x55811629e910 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08bb7fe700 1 -- 172.21.15.12:0/1893712977 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08c800f040 con 0x55811629e910 2024-05-09T00:05:14.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.445+0000 7f08bb7fe700 1 -- 172.21.15.12:0/1893712977 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800e7e0 con 0x55811629e910 2024-05-09T00:05:14.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.446+0000 7f08e3877b80 1 -- 172.21.15.12:0/1893712977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 msgr2=0x55811629ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.446+0000 7f08e3877b80 1 --2- 172.21.15.12:0/1893712977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x55811629ed50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f08c8003e80 tx=0x7f08c8009920 comp rx=0 tx=0).stop 2024-05-09T00:05:14.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.446+0000 7f08e3877b80 1 -- 172.21.15.12:0/1893712977 shutdown_connections 2024-05-09T00:05:14.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.446+0000 7f08e3877b80 1 --2- 172.21.15.12:0/1893712977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x55811629ed50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.446+0000 7f08e3877b80 1 --2- 172.21.15.12:0/1893712977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.446+0000 7f08e3877b80 1 --2- 172.21.15.12:0/1893712977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x55811629e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.446+0000 7f08e3877b80 1 -- 172.21.15.12:0/1893712977 >> 172.21.15.12:0/1893712977 conn(0x5581161fc290 msgr2=0x558116298e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.448+0000 7f08e3877b80 1 -- 172.21.15.12:0/1893712977 shutdown_connections 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.448+0000 7f08e3877b80 1 -- 172.21.15.12:0/1893712977 wait complete. 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.448+0000 7f08e3877b80 1 -- start start 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.448+0000 7f08e3877b80 1 Processor -- start 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.448+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x5581162aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.448+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x5581162a8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08e3877b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811618a5e0 con 0x5581162a3e80 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08e3877b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811618a280 con 0x55811629e910 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08e3877b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558116071720 con 0x5581162a47a0 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08e3877b80 10 client.0 _ll_register_callbacks cb 0x558116193bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d4c58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d5459700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x5581162aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d4c58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57116/0 (socket says 172.21.15.12:57116) 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d4c58700 1 -- 172.21.15.12:0/1711374217 learned_addr learned my addr 172.21.15.12:0/1711374217 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d5c5a700 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x5581162a8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d5459700 1 -- 172.21.15.12:0/1711374217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 msgr2=0x5581162a8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d5459700 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x5581162a8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d5459700 1 -- 172.21.15.12:0/1711374217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 msgr2=0x5581162a7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d5459700 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d5459700 1 -- 172.21.15.12:0/1711374217 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d01aa670 con 0x55811629e910 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d5c5a700 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x5581162a8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d4c58700 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08d5459700 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x5581162aa7d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0039f0 tx=0x7f08cc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ead67ae3a2eb579 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.449+0000 7f08b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.450+0000 7f08b8ff9700 1 -- 172.21.15.12:0/1711374217 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc01fab0 con 0x55811629e910 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.450+0000 7f08b8ff9700 1 -- 172.21.15.12:0/1711374217 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f08cc01fc10 con 0x55811629e910 2024-05-09T00:05:14.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.450+0000 7f08e3877b80 1 -- 172.21.15.12:0/1711374217 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581161472f0 con 0x55811629e910 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.450+0000 7f08b8ff9700 1 -- 172.21.15.12:0/1711374217 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08cc01fdd0 con 0x55811629e910 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.450+0000 7f08e3877b80 10 client.30245 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.450+0000 7f08e3877b80 1 -- 172.21.15.12:0/1711374217 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7f08c8003b60 con 0x55811629e910 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.450+0000 7f08b8ff9700 1 -- 172.21.15.12:0/1711374217 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581161472f0 con 0x55811629e910 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.451+0000 7f08b8ff9700 1 -- 172.21.15.12:0/1711374217 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f08cc005540 con 0x55811629e910 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.451+0000 7f08e3877b80 10 client.30245 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.451+0000 7f08e3877b80 10 client.30245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.451+0000 7f08e3877b80 20 client.30245 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.451+0000 7f08e3877b80 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581162b8310 0x5581162ba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.451+0000 7f08e3877b80 4 client.30245 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.451+0000 7f08e3877b80 1 -- 172.21.15.12:0/1711374217 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55811626d890 con 0x5581162b8310 2024-05-09T00:05:14.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.451+0000 7f08d4c58700 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581162b8310 0x5581162ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.452+0000 7f08d4c58700 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581162b8310 0x5581162ba730 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb443cc8acfaae5 server_cookie=3d651da97d6911c5 in_seq=0 out_seq=0 2024-05-09T00:05:14.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.452+0000 7f08b8ff9700 10 client.30245 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.456+0000 7f08b8ff9700 1 -- 172.21.15.12:0/1711374217 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55811626d890 con 0x5581162b8310 2024-05-09T00:05:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.456+0000 7f08b8ff9700 10 client.30245 handle_command_reply: tid=0 2024-05-09T00:05:14.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.456+0000 7f08e3877b80 1 client.30245 shutdown 2024-05-09T00:05:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.456+0000 7f08e3877b80 1 -- 172.21.15.12:0/1711374217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 msgr2=0x5581162aa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.456+0000 7f08e3877b80 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x5581162aa7d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0039f0 tx=0x7f08cc0047c0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.457+0000 7f08e3877b80 1 -- 172.21.15.12:0/1711374217 shutdown_connections 2024-05-09T00:05:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.457+0000 7f08e3877b80 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a47a0 0x5581162a8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.457+0000 7f08e3877b80 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581162b8310 0x5581162ba730 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.457+0000 7f08e3877b80 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55811629e910 0x5581162aa7d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.457+0000 7f08e3877b80 1 --2- 172.21.15.12:0/1711374217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581162a3e80 0x5581162a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.457+0000 7f08e3877b80 1 -- 172.21.15.12:0/1711374217 >> 172.21.15.12:0/1711374217 conn(0x5581161fc290 msgr2=0x558116298620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.457+0000 7f08b8ff9700 0 client.30245 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.457+0000 7f08e3877b80 1 -- 172.21.15.12:0/1711374217 shutdown_connections 2024-05-09T00:05:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.457+0000 7f08e3877b80 1 -- 172.21.15.12:0/1711374217 wait complete. 2024-05-09T00:05:14.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.457+0000 7f08e3877b80 20 client.30245 trim_cache size 0 max 16384 2024-05-09T00:05:14.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.481+0000 7f08e3877b80 1 Processor -- start 2024-05-09T00:05:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08e3877b80 1 -- start start 2024-05-09T00:05:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x55811629e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 0x558116298da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 0x5581162af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08e3877b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558116295e40 con 0x55811629e910 2024-05-09T00:05:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08e3877b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558116293880 con 0x5581162a47a0 2024-05-09T00:05:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08e3877b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581162b59a0 con 0x5581162a3e80 2024-05-09T00:05:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08d5459700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x55811629e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08d5459700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x55811629e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57132/0 (socket says 172.21.15.12:57132) 2024-05-09T00:05:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08d5459700 1 -- 172.21.15.12:0/640976375 learned_addr learned my addr 172.21.15.12:0/640976375 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08d4c58700 1 --2- 172.21.15.12:0/640976375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 0x558116298da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.482+0000 7f08d5c5a700 1 --2- 172.21.15.12:0/640976375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 0x5581162af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.483+0000 7f08d5459700 1 -- 172.21.15.12:0/640976375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 msgr2=0x558116298da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.483+0000 7f08d5459700 1 --2- 172.21.15.12:0/640976375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 0x558116298da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.483+0000 7f08d5459700 1 -- 172.21.15.12:0/640976375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 msgr2=0x5581162af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.483+0000 7f08d5459700 1 --2- 172.21.15.12:0/640976375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 0x5581162af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.483+0000 7f08d5459700 1 -- 172.21.15.12:0/640976375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55811601bbe0 con 0x55811629e910 2024-05-09T00:05:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.483+0000 7f08d5c5a700 1 --2- 172.21.15.12:0/640976375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 0x5581162af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:05:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.483+0000 7f08d5459700 1 --2- 172.21.15.12:0/640976375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x55811629e3d0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f08cc011c30 tx=0x7f08cc009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e16754bf085958c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.483+0000 7f08bbfff700 1 -- 172.21.15.12:0/640976375 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc028a40 con 0x55811629e910 2024-05-09T00:05:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.483+0000 7f08bbfff700 1 -- 172.21.15.12:0/640976375 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08cc028ba0 con 0x55811629e910 2024-05-09T00:05:14.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.483+0000 7f08bbfff700 1 -- 172.21.15.12:0/640976375 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc005040 con 0x55811629e910 2024-05-09T00:05:14.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.484+0000 7f08e3877b80 1 -- 172.21.15.12:0/640976375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 msgr2=0x55811629e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.484+0000 7f08e3877b80 1 --2- 172.21.15.12:0/640976375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x55811629e3d0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f08cc011c30 tx=0x7f08cc009370 comp rx=0 tx=0).stop 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.484+0000 7f08e3877b80 1 -- 172.21.15.12:0/640976375 shutdown_connections 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.484+0000 7f08e3877b80 1 --2- 172.21.15.12:0/640976375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 0x5581162af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.484+0000 7f08e3877b80 1 --2- 172.21.15.12:0/640976375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x55811629e3d0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.484+0000 7f08e3877b80 1 --2- 172.21.15.12:0/640976375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 0x558116298da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.484+0000 7f08e3877b80 1 -- 172.21.15.12:0/640976375 >> 172.21.15.12:0/640976375 conn(0x5581161fc290 msgr2=0x5581162a6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.484+0000 7f08e3877b80 1 -- 172.21.15.12:0/640976375 shutdown_connections 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.484+0000 7f08e3877b80 1 -- 172.21.15.12:0/640976375 wait complete. 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.484+0000 7f08e3877b80 1 -- start start 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08e3877b80 1 Processor -- start 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x5581162af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 0x5581162adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08e3877b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 0x5581162ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08e3877b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581162ae980 con 0x55811629e910 2024-05-09T00:05:14.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08e3877b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581163087b0 con 0x5581162a47a0 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08e3877b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558116308970 con 0x5581162a3e80 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08e3877b80 10 client.0 _ll_register_callbacks cb 0x558116193bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08d5459700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x5581162af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08d5459700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x5581162af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57146/0 (socket says 172.21.15.12:57146) 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08d4c58700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 0x5581162adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08d5459700 1 -- 172.21.15.12:0/3807069609 learned_addr learned my addr 172.21.15.12:0/3807069609 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.485+0000 7f08d5c5a700 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 0x5581162ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08d5459700 1 -- 172.21.15.12:0/3807069609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 msgr2=0x5581162adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08d5459700 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 0x5581162adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08d5459700 1 -- 172.21.15.12:0/3807069609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 msgr2=0x5581162ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08d5459700 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 0x5581162ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08d5459700 1 -- 172.21.15.12:0/3807069609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c8003b60 con 0x55811629e910 2024-05-09T00:05:14.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08d4c58700 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 0x5581162adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08d5459700 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x5581162af6c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f08cc016040 tx=0x7f08cc028a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e32887b36a7679 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08b8ff9700 1 -- 172.21.15.12:0/3807069609 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc01fc00 con 0x55811629e910 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08b8ff9700 1 -- 172.21.15.12:0/3807069609 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f08cc01fd60 con 0x55811629e910 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08e3877b80 1 -- 172.21.15.12:0/3807069609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08c800f040 con 0x55811629e910 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.486+0000 7f08b8ff9700 1 -- 172.21.15.12:0/3807069609 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08cc042410 con 0x55811629e910 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08e3877b80 10 client.20688 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08e3877b80 1 -- 172.21.15.12:0/3807069609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5581161b45e0 con 0x55811629e910 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08b8ff9700 1 -- 172.21.15.12:0/3807069609 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08cc04f020 con 0x55811629e910 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08b8ff9700 1 -- 172.21.15.12:0/3807069609 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f08cc036030 con 0x55811629e910 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08e3877b80 10 client.20688 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08e3877b80 10 client.20688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08e3877b80 20 client.20688 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:14.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08e3877b80 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558116309d50 0x55811630a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08e3877b80 4 client.20688 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:05:14.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08e3877b80 1 -- 172.21.15.12:0/3807069609 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5581161e6f40 con 0x558116309d50 2024-05-09T00:05:14.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.487+0000 7f08d4c58700 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558116309d50 0x55811630a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.488+0000 7f08d4c58700 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558116309d50 0x55811630a160 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3446f3a56b6e214 server_cookie=cdea233855b77b8a in_seq=0 out_seq=0 2024-05-09T00:05:14.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.488+0000 7f08b8ff9700 10 client.20688 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:14.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08b8ff9700 1 -- 172.21.15.12:0/3807069609 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5581161e6f40 con 0x558116309d50 2024-05-09T00:05:14.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08b8ff9700 10 client.20688 handle_command_reply: tid=0 2024-05-09T00:05:14.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08e3877b80 1 client.20688 shutdown 2024-05-09T00:05:14.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08e3877b80 1 -- 172.21.15.12:0/3807069609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 msgr2=0x5581162af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08e3877b80 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x5581162af6c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f08cc016040 tx=0x7f08cc028a20 comp rx=0 tx=0).stop 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08e3877b80 1 -- 172.21.15.12:0/3807069609 shutdown_connections 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08e3877b80 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558116309d50 0x55811630a160 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08e3877b80 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581162a47a0 0x5581162ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08e3877b80 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55811629e910 0x5581162af6c0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08e3877b80 1 --2- 172.21.15.12:0/3807069609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581162a3e80 0x5581162adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08e3877b80 1 -- 172.21.15.12:0/3807069609 >> 172.21.15.12:0/3807069609 conn(0x5581161fc290 msgr2=0x5581162a2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08b8ff9700 0 client.20688 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.491+0000 7f08e3877b80 1 -- 172.21.15.12:0/3807069609 shutdown_connections 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.492+0000 7f08e3877b80 1 -- 172.21.15.12:0/3807069609 wait complete. 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.492+0000 7f08e3877b80 20 client.20688 trim_cache size 0 max 16384 2024-05-09T00:05:14.488 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:05:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:05:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "3bbf984a-f4a4-403c-b94a-170e5c3f860d", 2024-05-09T00:05:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:05:14.489 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:05:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.492+0000 7f08d76bd700 1 -- 172.21.15.12:0/2544334118 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08bc0a1290 msgr2=0x7f08bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.492+0000 7f08d76bd700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08bc0a1290 0x7f08bc0a3750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f08d006a400 tx=0x7f08cc009260 comp rx=0 tx=0).stop 2024-05-09T00:05:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.492+0000 7f08d76bd700 1 -- 172.21.15.12:0/2544334118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08d0107f20 msgr2=0x7f08d01a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.492+0000 7f08d76bd700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08d0107f20 0x7f08d01a3bc0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f08c0000c00 tx=0x7f08c0004730 comp rx=0 tx=0).stop 2024-05-09T00:05:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.493+0000 7f08d76bd700 1 -- 172.21.15.12:0/2544334118 shutdown_connections 2024-05-09T00:05:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.493+0000 7f08d76bd700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f08bc0a1290 0x7f08bc0a3750 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.493+0000 7f08d76bd700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08d010c580 0x7f08d019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.493+0000 7f08d76bd700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08d0107560 0x7f08d01a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.493+0000 7f08d76bd700 1 --2- 172.21.15.12:0/2544334118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08d0107f20 0x7f08d01a3bc0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.493+0000 7f08d76bd700 1 -- 172.21.15.12:0/2544334118 >> 172.21.15.12:0/2544334118 conn(0x7f08d0076d90 msgr2=0x7f08d0077370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.493+0000 7f08d76bd700 1 -- 172.21.15.12:0/2544334118 shutdown_connections 2024-05-09T00:05:14.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.493+0000 7f08d76bd700 1 -- 172.21.15.12:0/2544334118 wait complete. 2024-05-09T00:05:14.503 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:05:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.728+0000 7f547012d700 1 -- 172.21.15.12:0/3063530614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5468105640 msgr2=0x7f5468105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.728+0000 7f547012d700 1 --2- 172.21.15.12:0/3063530614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5468105640 0x7f5468105a20 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f5458009920 tx=0x7f5458009c30 comp rx=0 tx=0).stop 2024-05-09T00:05:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.728+0000 7f547012d700 1 -- 172.21.15.12:0/3063530614 shutdown_connections 2024-05-09T00:05:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.728+0000 7f547012d700 1 --2- 172.21.15.12:0/3063530614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54680fff40 0x7f546810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.728+0000 7f547012d700 1 --2- 172.21.15.12:0/3063530614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5468105640 0x7f5468105a20 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.728+0000 7f547012d700 1 --2- 172.21.15.12:0/3063530614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54680ff580 0x7f54680ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.728+0000 7f547012d700 1 -- 172.21.15.12:0/3063530614 >> 172.21.15.12:0/3063530614 conn(0x7f54680763a0 msgr2=0x7f54680fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.728+0000 7f547012d700 1 -- 172.21.15.12:0/3063530614 shutdown_connections 2024-05-09T00:05:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.728+0000 7f547012d700 1 -- 172.21.15.12:0/3063530614 wait complete. 2024-05-09T00:05:14.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.729+0000 7f547012d700 1 Processor -- start 2024-05-09T00:05:14.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.729+0000 7f547012d700 1 -- start start 2024-05-09T00:05:14.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.730+0000 7f547012d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54680ff580 0x7f5468107580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.730+0000 7f547012d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54680fff40 0x7f5468107ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.730+0000 7f547012d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5468105640 0x7f5468108000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.730+0000 7f547012d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468113f90 con 0x7f5468105640 2024-05-09T00:05:14.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.730+0000 7f547012d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468114100 con 0x7f54680fff40 2024-05-09T00:05:14.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.730+0000 7f547012d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681142c0 con 0x7f54680ff580 2024-05-09T00:05:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.730+0000 7f546e6ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5468105640 0x7f5468108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.730+0000 7f546e6ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5468105640 0x7f5468108000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57154/0 (socket says 172.21.15.12:57154) 2024-05-09T00:05:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.730+0000 7f546e6ca700 1 -- 172.21.15.12:0/2913234908 learned_addr learned my addr 172.21.15.12:0/2913234908 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.730+0000 7f546dec9700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54680ff580 0x7f5468107580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.731+0000 7f546e6ca700 1 -- 172.21.15.12:0/2913234908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54680ff580 msgr2=0x7f5468107580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.731+0000 7f546e6ca700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54680ff580 0x7f5468107580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.731+0000 7f546e6ca700 1 -- 172.21.15.12:0/2913234908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54680fff40 msgr2=0x7f5468107ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.731+0000 7f546e6ca700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54680fff40 0x7f5468107ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.731+0000 7f546e6ca700 1 -- 172.21.15.12:0/2913234908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5458009520 con 0x7f5468105640 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.731+0000 7f546e6ca700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5468105640 0x7f5468108000 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f546000b970 tx=0x7f546000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.731+0000 7f545effd700 1 -- 172.21.15.12:0/2913234908 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000c9b0 con 0x7f5468105640 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.731+0000 7f545effd700 1 -- 172.21.15.12:0/2913234908 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5460014440 con 0x7f5468105640 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.731+0000 7f547012d700 1 -- 172.21.15.12:0/2913234908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54681145a0 con 0x7f5468105640 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.731+0000 7f545effd700 1 -- 172.21.15.12:0/2913234908 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460007570 con 0x7f5468105640 2024-05-09T00:05:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.732+0000 7f547012d700 1 -- 172.21.15.12:0/2913234908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5468114b00 con 0x7f5468105640 2024-05-09T00:05:14.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.735+0000 7f545effd700 1 -- 172.21.15.12:0/2913234908 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f54600076d0 con 0x7f5468105640 2024-05-09T00:05:14.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.736+0000 7f545effd700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54540a1350 0x7f54540a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.736+0000 7f547c2e7b80 1 Processor -- start 2024-05-09T00:05:14.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.737+0000 7f545effd700 1 -- 172.21.15.12:0/2913234908 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f54600cc2f0 con 0x7f5468105640 2024-05-09T00:05:14.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.737+0000 7f546dec9700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54540a1350 0x7f54540a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.737+0000 7f547c2e7b80 1 -- start start 2024-05-09T00:05:14.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.737+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41be1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.737+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41be720 0x5653b41b8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.737+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 0x5653b41b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.737+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b410f150 con 0x5653b41bde00 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.737+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b410cb10 con 0x5653b41b8890 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.737+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b4108400 con 0x5653b41be720 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546e6ca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 0x5653b41b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546e6ca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 0x5653b41b8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39252/0 (socket says 172.21.15.12:39252) 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546e6ca700 1 -- 172.21.15.12:0/3546283612 learned_addr learned my addr 172.21.15.12:0/3546283612 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546dec9700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54540a1350 0x7f54540a3810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5458009230 tx=0x7f545800f500 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546dec9700 1 --2- 172.21.15.12:0/3546283612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41be1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546d6c8700 1 --2- 172.21.15.12:0/3546283612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41be720 0x5653b41b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546e6ca700 1 -- 172.21.15.12:0/3546283612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41be720 msgr2=0x5653b41b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546e6ca700 1 --2- 172.21.15.12:0/3546283612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41be720 0x5653b41b8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546e6ca700 1 -- 172.21.15.12:0/3546283612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 msgr2=0x5653b41be1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546e6ca700 1 --2- 172.21.15.12:0/3546283612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41be1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.738+0000 7f546e6ca700 1 -- 172.21.15.12:0/3546283612 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653b3f893b0 con 0x5653b41b8890 2024-05-09T00:05:14.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.739+0000 7f546d6c8700 1 --2- 172.21.15.12:0/3546283612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41be720 0x5653b41b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:05:14.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.740+0000 7f546e6ca700 1 --2- 172.21.15.12:0/3546283612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 0x5653b41b8cd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f54600a1410 tx=0x7f54600993c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=308d6f5268da1a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.742+0000 7f5453fff700 1 -- 172.21.15.12:0/3546283612 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546009d2f0 con 0x5653b41b8890 2024-05-09T00:05:14.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.742+0000 7f5453fff700 1 -- 172.21.15.12:0/3546283612 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54600cb070 con 0x5653b41b8890 2024-05-09T00:05:14.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.742+0000 7f5453fff700 1 -- 172.21.15.12:0/3546283612 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546005e830 con 0x5653b41b8890 2024-05-09T00:05:14.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.743+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3546283612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 msgr2=0x5653b41b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.743+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3546283612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 0x5653b41b8cd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f54600a1410 tx=0x7f54600993c0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.744+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3546283612 shutdown_connections 2024-05-09T00:05:14.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.744+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3546283612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 0x5653b41b8cd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.744+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3546283612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41be1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.744+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3546283612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41be720 0x5653b41b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.744+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3546283612 >> 172.21.15.12:0/3546283612 conn(0x5653b4116210 msgr2=0x5653b41b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.745+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3546283612 shutdown_connections 2024-05-09T00:05:14.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.747+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3546283612 wait complete. 2024-05-09T00:05:14.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.748+0000 7f547c2e7b80 1 -- start start 2024-05-09T00:05:14.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.748+0000 7f547c2e7b80 1 Processor -- start 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.750+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 0x5653b41c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.750+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41c2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.750+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41be720 0x5653b41c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.750+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b40a4560 con 0x5653b41bde00 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.750+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b40a4200 con 0x5653b41b8890 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.750+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b3f8b6a0 con 0x5653b41be720 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.750+0000 7f547c2e7b80 10 client.0 _ll_register_callbacks cb 0x5653b40adb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.751+0000 7f546dec9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 0x5653b41c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.751+0000 7f546d6c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.751+0000 7f546d6c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41c2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57162/0 (socket says 172.21.15.12:57162) 2024-05-09T00:05:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.751+0000 7f546d6c8700 1 -- 172.21.15.12:0/323284444 learned_addr learned my addr 172.21.15.12:0/323284444 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.751+0000 7f546d6c8700 1 -- 172.21.15.12:0/323284444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41be720 msgr2=0x5653b41c25a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.751+0000 7f546d6c8700 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41be720 0x5653b41c25a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.751+0000 7f546d6c8700 1 -- 172.21.15.12:0/323284444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 msgr2=0x5653b41c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.751+0000 7f546d6c8700 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 0x5653b41c1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.751+0000 7f546d6c8700 1 -- 172.21.15.12:0/323284444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5468114b00 con 0x5653b41bde00 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.751+0000 7f546d6c8700 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41c2060 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f54640206b0 tx=0x7f5464020a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce32a97b87687a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.752+0000 7f54517fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.752+0000 7f54517fa700 1 -- 172.21.15.12:0/323284444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464004830 con 0x5653b41bde00 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.752+0000 7f54517fa700 1 -- 172.21.15.12:0/323284444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5464004990 con 0x5653b41bde00 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.752+0000 7f547c2e7b80 1 -- 172.21.15.12:0/323284444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653b4061270 con 0x5653b41bde00 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.752+0000 7f54517fa700 1 -- 172.21.15.12:0/323284444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5464004b50 con 0x5653b41bde00 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.753+0000 7f547c2e7b80 10 client.20700 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:14.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.753+0000 7f547c2e7b80 1 -- 172.21.15.12:0/323284444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5653b3f35b60 con 0x5653b41bde00 2024-05-09T00:05:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.753+0000 7f54517fa700 1 -- 172.21.15.12:0/323284444 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653b4061270 con 0x5653b41bde00 2024-05-09T00:05:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.754+0000 7f54517fa700 1 -- 172.21.15.12:0/323284444 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5464021930 con 0x5653b41bde00 2024-05-09T00:05:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.754+0000 7f547c2e7b80 10 client.20700 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.754+0000 7f547c2e7b80 10 client.20700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.754+0000 7f547c2e7b80 20 client.20700 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.754+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5653b41d2320 0x5653b41d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.754+0000 7f547c2e7b80 4 client.20700 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.755+0000 7f547c2e7b80 1 -- 172.21.15.12:0/323284444 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653b4187810 con 0x5653b41d2320 2024-05-09T00:05:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.755+0000 7f546dec9700 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5653b41d2320 0x5653b41d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.757+0000 7f546dec9700 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5653b41d2320 0x5653b41d4740 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68662cc4767331a2 server_cookie=77b2c65b952febf9 in_seq=0 out_seq=0 2024-05-09T00:05:14.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.757+0000 7f54517fa700 10 client.20700 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:14.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.759+0000 7f54517fa700 1 -- 172.21.15.12:0/323284444 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653b4187810 con 0x5653b41d2320 2024-05-09T00:05:14.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.759+0000 7f54517fa700 10 client.20700 handle_command_reply: tid=0 2024-05-09T00:05:14.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.759+0000 7f547c2e7b80 1 client.20700 shutdown 2024-05-09T00:05:14.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.759+0000 7f547c2e7b80 1 -- 172.21.15.12:0/323284444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 msgr2=0x5653b41c2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.759+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41c2060 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f54640206b0 tx=0x7f5464020a70 comp rx=0 tx=0).stop 2024-05-09T00:05:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.760+0000 7f547c2e7b80 1 -- 172.21.15.12:0/323284444 shutdown_connections 2024-05-09T00:05:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.760+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41be720 0x5653b41c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.760+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5653b41d2320 0x5653b41d4740 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.760+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41b8890 0x5653b41c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.760+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/323284444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41c2060 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.760+0000 7f547c2e7b80 1 -- 172.21.15.12:0/323284444 >> 172.21.15.12:0/323284444 conn(0x5653b4116210 msgr2=0x5653b41b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.760+0000 7f54517fa700 0 client.20700 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.760+0000 7f547c2e7b80 1 -- 172.21.15.12:0/323284444 shutdown_connections 2024-05-09T00:05:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.760+0000 7f547c2e7b80 1 -- 172.21.15.12:0/323284444 wait complete. 2024-05-09T00:05:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.761+0000 7f547c2e7b80 20 client.20700 trim_cache size 0 max 16384 2024-05-09T00:05:14.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f547c2e7b80 1 Processor -- start 2024-05-09T00:05:14.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f547c2e7b80 1 -- start start 2024-05-09T00:05:14.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41b8890 0x5653b41b8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 0x5653b41c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b41c3140 con 0x5653b41bde00 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b41ad800 con 0x5653b41be720 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b41d9930 con 0x5653b41b8890 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f546d6c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f546dec9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41b8890 0x5653b41b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f546d6c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57164/0 (socket says 172.21.15.12:57164) 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f546d6c8700 1 -- 172.21.15.12:0/3583751373 learned_addr learned my addr 172.21.15.12:0/3583751373 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.788+0000 7f546e6ca700 1 --2- 172.21.15.12:0/3583751373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 0x5653b41c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f546dec9700 1 -- 172.21.15.12:0/3583751373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 msgr2=0x5653b41c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f546dec9700 1 --2- 172.21.15.12:0/3583751373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 0x5653b41c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f546dec9700 1 -- 172.21.15.12:0/3583751373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 msgr2=0x5653b41b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f546dec9700 1 --2- 172.21.15.12:0/3583751373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f546dec9700 1 -- 172.21.15.12:0/3583751373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653b40ce560 con 0x5653b41b8890 2024-05-09T00:05:14.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f546e6ca700 1 --2- 172.21.15.12:0/3583751373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 0x5653b41c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f546d6c8700 1 --2- 172.21.15.12:0/3583751373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f546dec9700 1 --2- 172.21.15.12:0/3583751373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41b8890 0x5653b41b8350 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f54580098f0 tx=0x7f545801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd5e177fe34f97c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f545cff9700 1 -- 172.21.15.12:0/3583751373 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545802a040 con 0x5653b41b8890 2024-05-09T00:05:14.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f545cff9700 1 -- 172.21.15.12:0/3583751373 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f545800fab0 con 0x5653b41b8890 2024-05-09T00:05:14.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.789+0000 7f545cff9700 1 -- 172.21.15.12:0/3583751373 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545801c070 con 0x5653b41b8890 2024-05-09T00:05:14.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.790+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3583751373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41b8890 msgr2=0x5653b41b8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.790+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3583751373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41b8890 0x5653b41b8350 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f54580098f0 tx=0x7f545801c000 comp rx=0 tx=0).stop 2024-05-09T00:05:14.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.790+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3583751373 shutdown_connections 2024-05-09T00:05:14.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.790+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3583751373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 0x5653b41c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.790+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3583751373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41b8890 0x5653b41b8350 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.790+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3583751373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41bde00 0x5653b41b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.790+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3583751373 >> 172.21.15.12:0/3583751373 conn(0x5653b4116210 msgr2=0x5653b41bbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.790+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3583751373 shutdown_connections 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.790+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3583751373 wait complete. 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.791+0000 7f547c2e7b80 1 -- start start 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.791+0000 7f547c2e7b80 1 Processor -- start 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.791+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41b8890 0x5653b41c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.791+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41bde00 0x5653b41bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.791+0000 7f547c2e7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 0x5653b41bb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.791+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b41b9b70 con 0x5653b41b8890 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.791+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b41b9ce0 con 0x5653b41be720 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.791+0000 7f547c2e7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b41b9ea0 con 0x5653b41bde00 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.791+0000 7f547c2e7b80 10 client.0 _ll_register_callbacks cb 0x5653b40adb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546dec9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41b8890 0x5653b41c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546dec9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41b8890 0x5653b41c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57176/0 (socket says 172.21.15.12:57176) 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546dec9700 1 -- 172.21.15.12:0/3079941367 learned_addr learned my addr 172.21.15.12:0/3079941367 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546d6c8700 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41bde00 0x5653b41bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546e6ca700 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 0x5653b41bb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546d6c8700 1 -- 172.21.15.12:0/3079941367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 msgr2=0x5653b41bb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546d6c8700 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 0x5653b41bb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546d6c8700 1 -- 172.21.15.12:0/3079941367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41b8890 msgr2=0x5653b41c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546d6c8700 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41b8890 0x5653b41c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546d6c8700 1 -- 172.21.15.12:0/3079941367 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653b3f35b60 con 0x5653b41bde00 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546dec9700 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41b8890 0x5653b41c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546e6ca700 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 0x5653b41bb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f546d6c8700 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41bde00 0x5653b41bac80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5464005e00 tx=0x7f5464020650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2fdb9639338460d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.792+0000 7f54517fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:05:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f54517fa700 1 -- 172.21.15.12:0/3079941367 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464043070 con 0x5653b41bde00 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3079941367 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54600cb070 con 0x5653b41bde00 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f54517fa700 1 -- 172.21.15.12:0/3079941367 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5464000e40 con 0x5653b41bde00 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f54517fa700 1 -- 172.21.15.12:0/3079941367 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5464021e80 con 0x5653b41bde00 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f547c2e7b80 10 client.30400 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3079941367 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5653b3f893b0 con 0x5653b41bde00 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f54517fa700 1 -- 172.21.15.12:0/3079941367 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54600cb070 con 0x5653b41bde00 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f54517fa700 1 -- 172.21.15.12:0/3079941367 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5464046020 con 0x5653b41bde00 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f547c2e7b80 10 client.30400 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f547c2e7b80 10 client.30400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f547c2e7b80 20 client.30400 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5653b4229360 0x5653b422b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f547c2e7b80 4 client.30400 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.793+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3079941367 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5653b4100ec0 con 0x5653b4229360 2024-05-09T00:05:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.794+0000 7f546dec9700 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5653b4229360 0x5653b422b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:14.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.794+0000 7f546dec9700 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5653b4229360 0x5653b422b780 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1611bfa07630de2f server_cookie=fb7525a82225f047 in_seq=0 out_seq=0 2024-05-09T00:05:14.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.794+0000 7f54517fa700 10 client.30400 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f54517fa700 1 -- 172.21.15.12:0/3079941367 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x5653b4100ec0 con 0x5653b4229360 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f54517fa700 10 client.30400 handle_command_reply: tid=0 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 client.30400 shutdown 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3079941367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41bde00 msgr2=0x5653b41bac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41bde00 0x5653b41bac80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5464005e00 tx=0x7f5464020650 comp rx=0 tx=0).stop 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3079941367 shutdown_connections 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5653b4229360 0x5653b422b780 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653b41be720 0x5653b41bb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653b41b8890 0x5653b41c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 --2- 172.21.15.12:0/3079941367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5653b41bde00 0x5653b41bac80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f54517fa700 0 client.30400 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:14.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3079941367 >> 172.21.15.12:0/3079941367 conn(0x5653b4116210 msgr2=0x5653b41c03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3079941367 shutdown_connections 2024-05-09T00:05:14.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 1 -- 172.21.15.12:0/3079941367 wait complete. 2024-05-09T00:05:14.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.797+0000 7f547c2e7b80 20 client.30400 trim_cache size 0 max 16384 2024-05-09T00:05:14.794 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:05:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (85 inodes in the stack)", 2024-05-09T00:05:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:05:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "3bbf984a-f4a4-403c-b94a-170e5c3f860d": { 2024-05-09T00:05:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:05:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "3bbf984a-f4a4-403c-b94a-170e5c3f860d", 2024-05-09T00:05:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:05:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:05:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:05:14.795 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 -- 172.21.15.12:0/2913234908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54540a1350 msgr2=0x7f54540a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54540a1350 0x7f54540a3810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5458009230 tx=0x7f545800f500 comp rx=0 tx=0).stop 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 -- 172.21.15.12:0/2913234908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5468105640 msgr2=0x7f5468108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5468105640 0x7f5468108000 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f546000b970 tx=0x7f546000bd30 comp rx=0 tx=0).stop 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 -- 172.21.15.12:0/2913234908 shutdown_connections 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f54540a1350 0x7f54540a3810 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5468105640 0x7f5468108000 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54680ff580 0x7f5468107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 --2- 172.21.15.12:0/2913234908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54680fff40 0x7f5468107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 -- 172.21.15.12:0/2913234908 >> 172.21.15.12:0/2913234908 conn(0x7f54680763a0 msgr2=0x7f54680fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 -- 172.21.15.12:0/2913234908 shutdown_connections 2024-05-09T00:05:14.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:14.798+0000 7f547012d700 1 -- 172.21.15.12:0/2913234908 wait complete. 2024-05-09T00:05:14.807 INFO:tasks.cephfs.filesystem:scrub status for tag:3bbf984a-f4a4-403c-b94a-170e5c3f860d - {'path': '/', 'tag': '3bbf984a-f4a4-403c-b94a-170e5c3f860d', 'options': 'recursive,force'} 2024-05-09T00:05:14.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:14 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:05:14.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:14 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:14.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:14 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:05:15.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:14 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:05:15.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:14 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:15.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:14 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:05:15.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:14 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:05:15.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:14 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:15.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:14 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:05:15.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 210106 34.95 MB/sec execute 190 sec latency 114.780 ms 2024-05-09T00:05:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:15 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:05:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:15 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:05:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:15 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:15 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:15 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:15 smithi204 ceph-mon[47373]: pgmap v606: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 602 op/s 2024-05-09T00:05:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:15 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:05:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:15 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:05:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:15 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:15 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:15 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:16.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:15 smithi160 ceph-mon[46152]: pgmap v606: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 602 op/s 2024-05-09T00:05:16.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:15 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:05:16.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:15 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:05:16.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:15 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:16.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:15 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:16.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:15 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:16.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:15 smithi012 ceph-mon[42503]: pgmap v606: 97 pgs: 97 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 602 op/s 2024-05-09T00:05:16.260 INFO:tasks.workunit.client.0.smithi012.stdout: 10 210668 34.91 MB/sec execute 191 sec latency 104.946 ms 2024-05-09T00:05:17.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 211313 34.89 MB/sec execute 192 sec latency 87.573 ms 2024-05-09T00:05:18.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:17 smithi204 ceph-mon[47373]: pgmap v607: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.6 MiB/s wr, 782 op/s 2024-05-09T00:05:18.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:17 smithi160 ceph-mon[46152]: pgmap v607: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.6 MiB/s wr, 782 op/s 2024-05-09T00:05:18.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:17 smithi012 ceph-mon[42503]: pgmap v607: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.6 MiB/s wr, 782 op/s 2024-05-09T00:05:18.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 212026 34.93 MB/sec execute 193 sec latency 126.007 ms 2024-05-09T00:05:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.879+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3146173832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f4101aa0 msgr2=0x7f42f4105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.879+0000 7f42fc6fe700 1 --2- 172.21.15.12:0/3146173832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f4101aa0 0x7f42f4105ae0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009990 tx=0x7f42e4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.879+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3146173832 shutdown_connections 2024-05-09T00:05:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.879+0000 7f42fc6fe700 1 --2- 172.21.15.12:0/3146173832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f4106210 0x7f42f410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.879+0000 7f42fc6fe700 1 --2- 172.21.15.12:0/3146173832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f41010f0 0x7f42f41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.879+0000 7f42fc6fe700 1 --2- 172.21.15.12:0/3146173832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f4101aa0 0x7f42f4105ae0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.879+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3146173832 >> 172.21.15.12:0/3146173832 conn(0x7f42f40fc9d0 msgr2=0x7f42f40fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.879+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3146173832 shutdown_connections 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.880+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3146173832 wait complete. 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.880+0000 7f42fc6fe700 1 Processor -- start 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.880+0000 7f42fc6fe700 1 -- start start 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.880+0000 7f42fc6fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f41010f0 0x7f42f4197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.880+0000 7f42fc6fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f4106210 0x7f42f4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.880+0000 7f42fc6fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f419bce0 0x7f42f419c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.880+0000 7f42fc6fe700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f419c760 con 0x7f42f41010f0 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.880+0000 7f42fc6fe700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f419c8d0 con 0x7f42f4106210 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.880+0000 7f42fc6fe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f419ca90 con 0x7f42f419bce0 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fa49a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f41010f0 0x7f42f4197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fa49a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f41010f0 0x7f42f4197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49250/0 (socket says 172.21.15.12:49250) 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fa49a700 1 -- 172.21.15.12:0/3598887336 learned_addr learned my addr 172.21.15.12:0/3598887336 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42f9c99700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f4106210 0x7f42f4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fac9b700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f419bce0 0x7f42f419c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fa49a700 1 -- 172.21.15.12:0/3598887336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f419bce0 msgr2=0x7f42f419c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fa49a700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f419bce0 0x7f42f419c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fa49a700 1 -- 172.21.15.12:0/3598887336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f4106210 msgr2=0x7f42f4197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fa49a700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f4106210 0x7f42f4197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fa49a700 1 -- 172.21.15.12:0/3598887336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e40095f0 con 0x7f42f41010f0 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fa49a700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f41010f0 0x7f42f4197400 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f42f000e9c0 tx=0x7f42f000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42eb7fe700 1 -- 172.21.15.12:0/3598887336 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f00097b0 con 0x7f42f41010f0 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.881+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3598887336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f41a3ff0 con 0x7f42f41010f0 2024-05-09T00:05:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.882+0000 7f42eb7fe700 1 -- 172.21.15.12:0/3598887336 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42f0004cf0 con 0x7f42f41010f0 2024-05-09T00:05:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.882+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3598887336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f41a4470 con 0x7f42f41010f0 2024-05-09T00:05:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.882+0000 7f42eb7fe700 1 -- 172.21.15.12:0/3598887336 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0005600 con 0x7f42f41010f0 2024-05-09T00:05:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.885+0000 7f42eb7fe700 1 -- 172.21.15.12:0/3598887336 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f42f000cbd0 con 0x7f42f41010f0 2024-05-09T00:05:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.885+0000 7f42eb7fe700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42e00a1290 0x7f42e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.886+0000 7f42eb7fe700 1 -- 172.21.15.12:0/3598887336 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f42f0014070 con 0x7f42f41010f0 2024-05-09T00:05:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.886+0000 7f42f9c99700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42e00a1290 0x7f42e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.886+0000 7f42f9c99700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42e00a1290 0x7f42e00a3750 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f42e4006010 tx=0x7f42e400b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f43088b8b80 1 Processor -- start 2024-05-09T00:05:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f43088b8b80 1 -- start start 2024-05-09T00:05:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf00e950 0x563adf00ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 0x563adf008f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf009480 0x563adf0098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f43088b8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adef600f0 con 0x563adf00f270 2024-05-09T00:05:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f43088b8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adef5dab0 con 0x563adf00e950 2024-05-09T00:05:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f43088b8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adef593a0 con 0x563adf009480 2024-05-09T00:05:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f42fac9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf009480 0x563adf0098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f42fac9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf009480 0x563adf0098a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:58346/0 (socket says 172.21.15.12:58346) 2024-05-09T00:05:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.887+0000 7f42fac9b700 1 -- 172.21.15.12:0/1917212826 learned_addr learned my addr 172.21.15.12:0/1917212826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42f9c99700 1 --2- 172.21.15.12:0/1917212826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 0x563adf008f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42fac9b700 1 -- 172.21.15.12:0/1917212826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf00e950 msgr2=0x563adf00ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42fac9b700 1 --2- 172.21.15.12:0/1917212826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf00e950 0x563adf00ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42fac9b700 1 -- 172.21.15.12:0/1917212826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 msgr2=0x563adf008f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42fac9b700 1 --2- 172.21.15.12:0/1917212826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 0x563adf008f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42fac9b700 1 -- 172.21.15.12:0/1917212826 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563adedda350 con 0x563adf009480 2024-05-09T00:05:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42f9c99700 1 --2- 172.21.15.12:0/1917212826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 0x563adf008f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:05:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42fac9b700 1 --2- 172.21.15.12:0/1917212826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf009480 0x563adf0098a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f42ec011680 tx=0x7f42ec009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18edc584e0f6c58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42e8ff9700 1 -- 172.21.15.12:0/1917212826 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec00e040 con 0x563adf009480 2024-05-09T00:05:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42e8ff9700 1 -- 172.21.15.12:0/1917212826 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42ec00f040 con 0x563adf009480 2024-05-09T00:05:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.888+0000 7f42e8ff9700 1 -- 172.21.15.12:0/1917212826 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0048c0 con 0x563adf009480 2024-05-09T00:05:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.889+0000 7f43088b8b80 1 -- 172.21.15.12:0/1917212826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf009480 msgr2=0x563adf0098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.889+0000 7f43088b8b80 1 --2- 172.21.15.12:0/1917212826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf009480 0x563adf0098a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f42ec011680 tx=0x7f42ec009920 comp rx=0 tx=0).stop 2024-05-09T00:05:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.889+0000 7f43088b8b80 1 -- 172.21.15.12:0/1917212826 shutdown_connections 2024-05-09T00:05:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.889+0000 7f43088b8b80 1 --2- 172.21.15.12:0/1917212826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf009480 0x563adf0098a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.889+0000 7f43088b8b80 1 --2- 172.21.15.12:0/1917212826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf00e950 0x563adf00ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.889+0000 7f43088b8b80 1 --2- 172.21.15.12:0/1917212826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 0x563adf008f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.889+0000 7f43088b8b80 1 -- 172.21.15.12:0/1917212826 >> 172.21.15.12:0/1917212826 conn(0x563adef671b0 msgr2=0x563adf003840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.889+0000 7f43088b8b80 1 -- 172.21.15.12:0/1917212826 shutdown_connections 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.889+0000 7f43088b8b80 1 -- 172.21.15.12:0/1917212826 wait complete. 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.890+0000 7f43088b8b80 1 -- start start 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.890+0000 7f43088b8b80 1 Processor -- start 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf015280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00e950 0x563adf012570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 0x563adf012cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f43088b8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adeef5500 con 0x563adf00f270 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f43088b8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adeef51a0 con 0x563adf009480 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f43088b8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adeddc640 con 0x563adf00e950 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f43088b8b80 10 client.0 _ll_register_callbacks cb 0x563adeefead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42f9c99700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00e950 0x563adf012570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42fac9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 0x563adf012cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42f9c99700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00e950 0x563adf012570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:58354/0 (socket says 172.21.15.12:58354) 2024-05-09T00:05:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42f9c99700 1 -- 172.21.15.12:0/1379961095 learned_addr learned my addr 172.21.15.12:0/1379961095 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42fa49a700 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf015280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42fac9b700 1 -- 172.21.15.12:0/1379961095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00e950 msgr2=0x563adf012570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42fac9b700 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00e950 0x563adf012570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42fac9b700 1 -- 172.21.15.12:0/1379961095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 msgr2=0x563adf015280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42fac9b700 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf015280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42fac9b700 1 -- 172.21.15.12:0/1379961095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec011330 con 0x563adf00f270 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42fa49a700 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf015280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.891+0000 7f42f9c99700 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00e950 0x563adf012570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.892+0000 7f42fac9b700 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 0x563adf012cb0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00b010 tx=0x7f42ec003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576f188f1fb33dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.892+0000 7f42ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.892+0000 7f42ddffb700 1 -- 172.21.15.12:0/1379961095 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec00e030 con 0x563adf00f270 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.892+0000 7f42ddffb700 1 -- 172.21.15.12:0/1379961095 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f42ec0093a0 con 0x563adf00f270 2024-05-09T00:05:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.892+0000 7f43088b8b80 1 -- 172.21.15.12:0/1379961095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563adeeb2210 con 0x563adf00f270 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.892+0000 7f42ddffb700 1 -- 172.21.15.12:0/1379961095 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42ec0041e0 con 0x563adf00f270 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.892+0000 7f43088b8b80 10 client.20718 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.892+0000 7f43088b8b80 1 -- 172.21.15.12:0/1379961095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x563aded86b00 con 0x563adf00f270 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.892+0000 7f42ddffb700 1 -- 172.21.15.12:0/1379961095 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563adeeb2210 con 0x563adf00f270 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.892+0000 7f42ddffb700 1 -- 172.21.15.12:0/1379961095 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f42ec02d070 con 0x563adf00f270 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.893+0000 7f43088b8b80 10 client.20718 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.893+0000 7f43088b8b80 10 client.20718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.893+0000 7f43088b8b80 20 client.20718 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.893+0000 7f43088b8b80 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563adf022c90 0x563adf0250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.893+0000 7f43088b8b80 4 client.20718 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.893+0000 7f43088b8b80 1 -- 172.21.15.12:0/1379961095 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563adefd87b0 con 0x563adf022c90 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.893+0000 7f42fa49a700 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563adf022c90 0x563adf0250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.894+0000 7f42fa49a700 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563adf022c90 0x563adf0250b0 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9543a85d44b8e82e server_cookie=90089cce0ac0725 in_seq=0 out_seq=0 2024-05-09T00:05:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.894+0000 7f42ddffb700 10 client.20718 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f42ddffb700 1 -- 172.21.15.12:0/1379961095 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563adefd87b0 con 0x563adf022c90 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f42ddffb700 10 client.20718 handle_command_reply: tid=0 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 client.20718 shutdown 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 -- 172.21.15.12:0/1379961095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 msgr2=0x563adf012cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 0x563adf012cb0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00b010 tx=0x7f42ec003c30 comp rx=0 tx=0).stop 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 -- 172.21.15.12:0/1379961095 shutdown_connections 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00f270 0x563adf012cb0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563adf022c90 0x563adf0250b0 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf015280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 --2- 172.21.15.12:0/1379961095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00e950 0x563adf012570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 -- 172.21.15.12:0/1379961095 >> 172.21.15.12:0/1379961095 conn(0x563adef671b0 msgr2=0x563adf00d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f42ddffb700 0 client.20718 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 -- 172.21.15.12:0/1379961095 shutdown_connections 2024-05-09T00:05:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 1 -- 172.21.15.12:0/1379961095 wait complete. 2024-05-09T00:05:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.895+0000 7f43088b8b80 20 client.20718 trim_cache size 0 max 16384 2024-05-09T00:05:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.913+0000 7f43088b8b80 1 Processor -- start 2024-05-09T00:05:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.913+0000 7f43088b8b80 1 -- start start 2024-05-09T00:05:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf0054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 0x563adf007630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f43088b8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adf01f810 con 0x563adf00e950 2024-05-09T00:05:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f43088b8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adf0059f0 con 0x563adf009480 2024-05-09T00:05:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f43088b8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adf005290 con 0x563adf00f270 2024-05-09T00:05:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42f9c99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf0054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42fa49a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42fac9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 0x563adf007630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42f9c99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf0054b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49264/0 (socket says 172.21.15.12:49264) 2024-05-09T00:05:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42fa49a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf003910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:55066/0 (socket says 172.21.15.12:55066) 2024-05-09T00:05:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42f9c99700 1 -- 172.21.15.12:0/2631980324 learned_addr learned my addr 172.21.15.12:0/2631980324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42f9c99700 1 -- 172.21.15.12:0/2631980324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 msgr2=0x563adf007630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42f9c99700 1 --2- 172.21.15.12:0/2631980324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 0x563adf007630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42f9c99700 1 -- 172.21.15.12:0/2631980324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 msgr2=0x563adf003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42f9c99700 1 --2- 172.21.15.12:0/2631980324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42f9c99700 1 -- 172.21.15.12:0/2631980324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563adef1f500 con 0x563adf00e950 2024-05-09T00:05:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42fac9b700 1 --2- 172.21.15.12:0/2631980324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 0x563adf007630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.914+0000 7f42fa49a700 1 --2- 172.21.15.12:0/2631980324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf003910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.915+0000 7f42f9c99700 1 --2- 172.21.15.12:0/2631980324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf0054b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f42e4015000 tx=0x7f42e401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6bfdcfe61c3ea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.915+0000 7f42e97fa700 1 -- 172.21.15.12:0/2631980324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4004700 con 0x563adf00e950 2024-05-09T00:05:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.915+0000 7f42e97fa700 1 -- 172.21.15.12:0/2631980324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42e402c040 con 0x563adf00e950 2024-05-09T00:05:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.915+0000 7f42e97fa700 1 -- 172.21.15.12:0/2631980324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e400f9a0 con 0x563adf00e950 2024-05-09T00:05:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 -- 172.21.15.12:0/2631980324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 msgr2=0x563adf0054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 --2- 172.21.15.12:0/2631980324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf0054b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f42e4015000 tx=0x7f42e401f000 comp rx=0 tx=0).stop 2024-05-09T00:05:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 -- 172.21.15.12:0/2631980324 shutdown_connections 2024-05-09T00:05:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 --2- 172.21.15.12:0/2631980324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 0x563adf007630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 --2- 172.21.15.12:0/2631980324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf003910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 --2- 172.21.15.12:0/2631980324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf0054b0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 -- 172.21.15.12:0/2631980324 >> 172.21.15.12:0/2631980324 conn(0x563adef671b0 msgr2=0x563adefffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 -- 172.21.15.12:0/2631980324 shutdown_connections 2024-05-09T00:05:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 -- 172.21.15.12:0/2631980324 wait complete. 2024-05-09T00:05:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 -- start start 2024-05-09T00:05:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.916+0000 7f43088b8b80 1 Processor -- start 2024-05-09T00:05:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf019e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf00c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f43088b8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 0x563adf00c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f43088b8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adf00ceb0 con 0x563adf00e950 2024-05-09T00:05:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f43088b8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adf00d020 con 0x563adf009480 2024-05-09T00:05:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f43088b8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563adf00d1e0 con 0x563adf00f270 2024-05-09T00:05:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f43088b8b80 10 client.0 _ll_register_callbacks cb 0x563adeefead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42f9c99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf00c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42f9c99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf00c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49276/0 (socket says 172.21.15.12:49276) 2024-05-09T00:05:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42f9c99700 1 -- 172.21.15.12:0/2331147186 learned_addr learned my addr 172.21.15.12:0/2331147186 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42fa49a700 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf019e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42fac9b700 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 0x563adf00c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42f9c99700 1 -- 172.21.15.12:0/2331147186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 msgr2=0x563adf00c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42f9c99700 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 0x563adf00c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42f9c99700 1 -- 172.21.15.12:0/2331147186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 msgr2=0x563adf019e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42f9c99700 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf019e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42f9c99700 1 -- 172.21.15.12:0/2331147186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f41a4470 con 0x563adf00e950 2024-05-09T00:05:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42fa49a700 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf019e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42f9c99700 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf00c3f0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f42e400f540 tx=0x7f42e400fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e2bb1bcf4d376c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.917+0000 7f42ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.918+0000 7f42ddffb700 1 -- 172.21.15.12:0/2331147186 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4029030 con 0x563adf00e950 2024-05-09T00:05:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.918+0000 7f42ddffb700 1 -- 172.21.15.12:0/2331147186 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f42e4004910 con 0x563adf00e950 2024-05-09T00:05:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.918+0000 7f43088b8b80 1 -- 172.21.15.12:0/2331147186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42ec00f040 con 0x563adf00e950 2024-05-09T00:05:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.918+0000 7f42ddffb700 1 -- 172.21.15.12:0/2331147186 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42e400be90 con 0x563adf00e950 2024-05-09T00:05:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.920+0000 7f43088b8b80 10 client.20724 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.920+0000 7f43088b8b80 1 -- 172.21.15.12:0/2331147186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x563aded86b00 con 0x563adf00e950 2024-05-09T00:05:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.920+0000 7f42ddffb700 1 -- 172.21.15.12:0/2331147186 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42ec00f040 con 0x563adf00e950 2024-05-09T00:05:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.920+0000 7f42ddffb700 1 -- 172.21.15.12:0/2331147186 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f42e4032c60 con 0x563adf00e950 2024-05-09T00:05:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.920+0000 7f43088b8b80 10 client.20724 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.921+0000 7f43088b8b80 10 client.20724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.921+0000 7f43088b8b80 20 client.20724 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.921+0000 7f43088b8b80 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563adf0793e0 0x563adf07b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.921+0000 7f43088b8b80 4 client.20724 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:05:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.921+0000 7f43088b8b80 1 -- 172.21.15.12:0/2331147186 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563adef51e60 con 0x563adf0793e0 2024-05-09T00:05:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.921+0000 7f42fa49a700 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563adf0793e0 0x563adf07b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.921+0000 7f42fa49a700 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563adf0793e0 0x563adf07b800 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=341c7745214381b2 server_cookie=96beb373a8f6a1b4 in_seq=0 out_seq=0 2024-05-09T00:05:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:18.921+0000 7f42ddffb700 10 client.20724 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:19.263 INFO:tasks.workunit.client.0.smithi012.stdout: 10 212469 34.86 MB/sec execute 194 sec latency 303.839 ms 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.365+0000 7f42ddffb700 1 -- 172.21.15.12:0/2331147186 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563adef51e60 con 0x563adf0793e0 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.365+0000 7f42ddffb700 10 client.20724 handle_command_reply: tid=0 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.365+0000 7f43088b8b80 1 client.20724 shutdown 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.365+0000 7f43088b8b80 1 -- 172.21.15.12:0/2331147186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 msgr2=0x563adf00c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.365+0000 7f43088b8b80 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf00c3f0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f42e400f540 tx=0x7f42e400fc70 comp rx=0 tx=0).stop 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.366+0000 7f43088b8b80 1 -- 172.21.15.12:0/2331147186 shutdown_connections 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.366+0000 7f43088b8b80 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563adf00f270 0x563adf00c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.366+0000 7f43088b8b80 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x563adf0793e0 0x563adf07b800 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.366+0000 7f43088b8b80 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563adf009480 0x563adf019e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.366+0000 7f43088b8b80 1 --2- 172.21.15.12:0/2331147186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563adf00e950 0x563adf00c3f0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.366+0000 7f42ddffb700 0 client.20724 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.366+0000 7f43088b8b80 1 -- 172.21.15.12:0/2331147186 >> 172.21.15.12:0/2331147186 conn(0x563adef671b0 msgr2=0x563adf007630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.367+0000 7f43088b8b80 1 -- 172.21.15.12:0/2331147186 shutdown_connections 2024-05-09T00:05:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.367+0000 7f43088b8b80 1 -- 172.21.15.12:0/2331147186 wait complete. 2024-05-09T00:05:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.367+0000 7f43088b8b80 20 client.20724 trim_cache size 0 max 16384 2024-05-09T00:05:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:05:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:05:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:05:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3598887336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42e00a1290 msgr2=0x7f42e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42e00a1290 0x7f42e00a3750 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f42e4006010 tx=0x7f42e400b260 comp rx=0 tx=0).stop 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3598887336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f41010f0 msgr2=0x7f42f4197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f41010f0 0x7f42f4197400 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f42f000e9c0 tx=0x7f42f000ecd0 comp rx=0 tx=0).stop 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3598887336 shutdown_connections 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42e00a1290 0x7f42e00a3750 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f419bce0 0x7f42f419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f41010f0 0x7f42f4197400 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 --2- 172.21.15.12:0/3598887336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f4106210 0x7f42f4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3598887336 >> 172.21.15.12:0/3598887336 conn(0x7f42f40fc9d0 msgr2=0x7f42f40fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3598887336 shutdown_connections 2024-05-09T00:05:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:19.368+0000 7f42fc6fe700 1 -- 172.21.15.12:0/3598887336 wait complete. 2024-05-09T00:05:20.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:19 smithi204 ceph-mon[47373]: pgmap v608: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-05-09T00:05:20.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:19 smithi160 ceph-mon[46152]: pgmap v608: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-05-09T00:05:20.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:19 smithi012 ceph-mon[42503]: pgmap v608: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-05-09T00:05:20.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 213173 34.89 MB/sec execute 195 sec latency 173.486 ms 2024-05-09T00:05:21.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 213799 34.91 MB/sec execute 196 sec latency 108.597 ms 2024-05-09T00:05:22.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:21 smithi204 ceph-mon[47373]: pgmap v609: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 503 op/s 2024-05-09T00:05:22.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:21 smithi160 ceph-mon[46152]: pgmap v609: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 503 op/s 2024-05-09T00:05:22.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:21 smithi012 ceph-mon[42503]: pgmap v609: 97 pgs: 97 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 503 op/s 2024-05-09T00:05:22.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 214310 34.84 MB/sec execute 197 sec latency 114.253 ms 2024-05-09T00:05:23.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 214975 34.83 MB/sec execute 198 sec latency 98.221 ms 2024-05-09T00:05:23.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:23 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:05:24.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:23 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:05:24.020 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:23 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:05:24.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 215557 34.82 MB/sec execute 199 sec latency 120.900 ms 2024-05-09T00:05:24.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:05:24.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:05:24.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:24 smithi012 ceph-mon[42503]: pgmap v610: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.2 MiB/s wr, 742 op/s 2024-05-09T00:05:25.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:05:25.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:05:25.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:24 smithi204 ceph-mon[47373]: pgmap v610: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.2 MiB/s wr, 742 op/s 2024-05-09T00:05:25.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:05:25.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:05:25.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:24 smithi160 ceph-mon[46152]: pgmap v610: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.2 MiB/s wr, 742 op/s 2024-05-09T00:05:25.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 216144 34.80 MB/sec execute 200 sec latency 115.985 ms 2024-05-09T00:05:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:25 smithi204 ceph-mon[47373]: pgmap v611: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 572 op/s 2024-05-09T00:05:26.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:25 smithi160 ceph-mon[46152]: pgmap v611: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 572 op/s 2024-05-09T00:05:26.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:25 smithi012 ceph-mon[42503]: pgmap v611: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 572 op/s 2024-05-09T00:05:26.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 216583 34.76 MB/sec execute 201 sec latency 241.074 ms 2024-05-09T00:05:27.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 217186 34.77 MB/sec execute 202 sec latency 119.941 ms 2024-05-09T00:05:28.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:27 smithi160 ceph-mon[46152]: pgmap v612: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.7 MiB/s wr, 748 op/s 2024-05-09T00:05:28.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:27 smithi204 ceph-mon[47373]: pgmap v612: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.7 MiB/s wr, 748 op/s 2024-05-09T00:05:28.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:27 smithi012 ceph-mon[42503]: pgmap v612: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.7 MiB/s wr, 748 op/s 2024-05-09T00:05:28.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 217724 34.70 MB/sec execute 203 sec latency 133.107 ms 2024-05-09T00:05:29.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 218395 34.71 MB/sec execute 204 sec latency 152.373 ms 2024-05-09T00:05:30.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:29 smithi204 ceph-mon[47373]: pgmap v613: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 571 op/s 2024-05-09T00:05:30.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:29 smithi160 ceph-mon[46152]: pgmap v613: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 571 op/s 2024-05-09T00:05:30.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:29 smithi012 ceph-mon[42503]: pgmap v613: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 571 op/s 2024-05-09T00:05:30.261 INFO:tasks.workunit.client.0.smithi012.stdout: 10 218971 34.71 MB/sec execute 205 sec latency 126.971 ms 2024-05-09T00:05:31.263 INFO:tasks.workunit.client.0.smithi012.stdout: 10 219375 34.62 MB/sec execute 206 sec latency 256.390 ms 2024-05-09T00:05:32.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:31 smithi204 ceph-mon[47373]: pgmap v614: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 555 op/s 2024-05-09T00:05:32.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:31 smithi160 ceph-mon[46152]: pgmap v614: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 555 op/s 2024-05-09T00:05:32.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:31 smithi012 ceph-mon[42503]: pgmap v614: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 555 op/s 2024-05-09T00:05:32.263 INFO:tasks.workunit.client.0.smithi012.stdout: 10 220078 34.66 MB/sec execute 207 sec latency 122.910 ms 2024-05-09T00:05:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.512+0000 7fdb43663700 1 -- 172.21.15.12:0/2867940722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3c101110 msgr2=0x7fdb3c1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.512+0000 7fdb43663700 1 --2- 172.21.15.12:0/2867940722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3c101110 0x7fdb3c1014f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009990 tx=0x7fdb38009ca0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.513+0000 7fdb43663700 1 -- 172.21.15.12:0/2867940722 shutdown_connections 2024-05-09T00:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.513+0000 7fdb43663700 1 --2- 172.21.15.12:0/2867940722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb3c106230 0x7fdb3c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.513+0000 7fdb43663700 1 --2- 172.21.15.12:0/2867940722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3c101110 0x7fdb3c1014f0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.513+0000 7fdb43663700 1 --2- 172.21.15.12:0/2867940722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3c101ac0 0x7fdb3c105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.513+0000 7fdb43663700 1 -- 172.21.15.12:0/2867940722 >> 172.21.15.12:0/2867940722 conn(0x7fdb3c0fc9b0 msgr2=0x7fdb3c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.513+0000 7fdb43663700 1 -- 172.21.15.12:0/2867940722 shutdown_connections 2024-05-09T00:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.513+0000 7fdb43663700 1 -- 172.21.15.12:0/2867940722 wait complete. 2024-05-09T00:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb43663700 1 Processor -- start 2024-05-09T00:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb43663700 1 -- start start 2024-05-09T00:05:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb43663700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3c101110 0x7fdb3c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb43663700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb3c106230 0x7fdb3c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb43663700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3c19bc50 0x7fdb3c19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb43663700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c19c6d0 con 0x7fdb3c101110 2024-05-09T00:05:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb43663700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c19c840 con 0x7fdb3c106230 2024-05-09T00:05:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb43663700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c19ca00 con 0x7fdb3c19bc50 2024-05-09T00:05:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb413ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3c101110 0x7fdb3c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb41c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3c19bc50 0x7fdb3c19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb413ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3c101110 0x7fdb3c197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59196/0 (socket says 172.21.15.12:59196) 2024-05-09T00:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb41c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3c19bc50 0x7fdb3c19c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44674/0 (socket says 172.21.15.12:44674) 2024-05-09T00:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb413ff700 1 -- 172.21.15.12:0/4185416406 learned_addr learned my addr 172.21.15.12:0/4185416406 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.514+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb3c106230 0x7fdb3c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb413ff700 1 -- 172.21.15.12:0/4185416406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3c19bc50 msgr2=0x7fdb3c19c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb413ff700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3c19bc50 0x7fdb3c19c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb413ff700 1 -- 172.21.15.12:0/4185416406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb3c106230 msgr2=0x7fdb3c1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb413ff700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb3c106230 0x7fdb3c1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb413ff700 1 -- 172.21.15.12:0/4185416406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb380095f0 con 0x7fdb3c101110 2024-05-09T00:05:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb41c00700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3c19bc50 0x7fdb3c19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb413ff700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3c101110 0x7fdb3c197370 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fdb38004760 tx=0x7fdb38004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb327fc700 1 -- 172.21.15.12:0/4185416406 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3801d070 con 0x7fdb3c101110 2024-05-09T00:05:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb43663700 1 -- 172.21.15.12:0/4185416406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c1a3f00 con 0x7fdb3c101110 2024-05-09T00:05:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb327fc700 1 -- 172.21.15.12:0/4185416406 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb3800fb30 con 0x7fdb3c101110 2024-05-09T00:05:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb327fc700 1 -- 172.21.15.12:0/4185416406 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38022db0 con 0x7fdb3c101110 2024-05-09T00:05:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.515+0000 7fdb43663700 1 -- 172.21.15.12:0/4185416406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c0731d0 con 0x7fdb3c101110 2024-05-09T00:05:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.518+0000 7fdb327fc700 1 -- 172.21.15.12:0/4185416406 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdb3800fca0 con 0x7fdb3c101110 2024-05-09T00:05:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.519+0000 7fdb327fc700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb280a56a0 0x7fdb280a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.519+0000 7fdb327fc700 1 -- 172.21.15.12:0/4185416406 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdb380cdfa0 con 0x7fdb3c101110 2024-05-09T00:05:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.519+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb280a56a0 0x7fdb280a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.520+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb280a56a0 0x7fdb280a7b60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c1988c0 tx=0x7fdb2c00b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.520+0000 7fdb4f81db80 1 Processor -- start 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.520+0000 7fdb4f81db80 1 -- start start 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.520+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7e7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.520+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.520+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e1c90 0x55d48f7e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.520+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f7300f0 con 0x55d48f7e7a80 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.520+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f72dab0 con 0x55d48f7e7160 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.520+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f7293a0 con 0x55d48f7e1c90 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb40bfe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb40bfe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59202/0 (socket says 172.21.15.12:59202) 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb40bfe700 1 -- 172.21.15.12:0/2881427784 learned_addr learned my addr 172.21.15.12:0/2881427784 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb41c00700 1 --2- 172.21.15.12:0/2881427784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e1c90 0x55d48f7e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb413ff700 1 --2- 172.21.15.12:0/2881427784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb40bfe700 1 -- 172.21.15.12:0/2881427784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e1c90 msgr2=0x55d48f7e20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/2881427784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e1c90 0x55d48f7e20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb40bfe700 1 -- 172.21.15.12:0/2881427784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 msgr2=0x55d48f7e7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/2881427784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7e7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb40bfe700 1 -- 172.21.15.12:0/2881427784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d48f5aa350 con 0x55d48f7e7a80 2024-05-09T00:05:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/2881427784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e1750 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c010f40 tx=0x7fdb2c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46d7d32aebb169a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.521+0000 7fdb277fe700 1 -- 172.21.15.12:0/2881427784 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c00b950 con 0x55d48f7e7a80 2024-05-09T00:05:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.522+0000 7fdb277fe700 1 -- 172.21.15.12:0/2881427784 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb2c015070 con 0x55d48f7e7a80 2024-05-09T00:05:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.522+0000 7fdb277fe700 1 -- 172.21.15.12:0/2881427784 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c01c610 con 0x55d48f7e7a80 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 -- 172.21.15.12:0/2881427784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 msgr2=0x55d48f7e1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/2881427784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e1750 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c010f40 tx=0x7fdb2c013000 comp rx=0 tx=0).stop 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 -- 172.21.15.12:0/2881427784 shutdown_connections 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/2881427784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e1c90 0x55d48f7e20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/2881427784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7e7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/2881427784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e1750 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 -- 172.21.15.12:0/2881427784 >> 172.21.15.12:0/2881427784 conn(0x55d48f7371b0 msgr2=0x55d48f7d3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 -- 172.21.15.12:0/2881427784 shutdown_connections 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 -- 172.21.15.12:0/2881427784 wait complete. 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 -- start start 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.523+0000 7fdb4f81db80 1 Processor -- start 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.524+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.524+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7ead60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.524+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e7a80 0x55d48f7eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.524+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f6c5500 con 0x55d48f7e1c90 2024-05-09T00:05:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.524+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f6c51a0 con 0x55d48f7e7160 2024-05-09T00:05:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.524+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f5ac640 con 0x55d48f7e7a80 2024-05-09T00:05:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.524+0000 7fdb4f81db80 10 client.0 _ll_register_callbacks cb 0x55d48f6cead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.524+0000 7fdb40bfe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7ead60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.524+0000 7fdb40bfe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7ead60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:59046/0 (socket says 172.21.15.12:59046) 2024-05-09T00:05:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.524+0000 7fdb40bfe700 1 -- 172.21.15.12:0/1143676990 learned_addr learned my addr 172.21.15.12:0/1143676990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb413ff700 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb40bfe700 1 -- 172.21.15.12:0/1143676990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e7a80 msgr2=0x55d48f7eb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e7a80 0x55d48f7eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb40bfe700 1 -- 172.21.15.12:0/1143676990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 msgr2=0x55d48f7eda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7eda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb40bfe700 1 -- 172.21.15.12:0/1143676990 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2c010c20 con 0x55d48f7e7160 2024-05-09T00:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb413ff700 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7ead60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0059d0 tx=0x7fdb2c00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=282a9a89696886f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1143676990 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c02b070 con 0x55d48f7e7160 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1143676990 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d48f682210 con 0x55d48f7e7160 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.525+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1143676990 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdb2c017070 con 0x55d48f7e7160 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1143676990 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb2c012e30 con 0x55d48f7e7160 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb4f81db80 10 client.30290 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1143676990 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55d48f556b00 con 0x55d48f7e7160 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1143676990 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb2c03e020 con 0x55d48f7e7160 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1143676990 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb2c011440 con 0x55d48f7e7160 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb4f81db80 10 client.30290 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb4f81db80 10 client.30290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb4f81db80 20 client.30290 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d48f7d9e20 0x55d48f7dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb4f81db80 4 client.30290 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1143676990 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d48f7a87b0 con 0x55d48f7d9e20 2024-05-09T00:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.526+0000 7fdb413ff700 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d48f7d9e20 0x55d48f7dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.527+0000 7fdb24ff9700 10 client.30290 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.527+0000 7fdb413ff700 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d48f7d9e20 0x55d48f7dc240 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a72fe62fd6d1ef server_cookie=f48e4112fc6f557b in_seq=0 out_seq=0 2024-05-09T00:05:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1143676990 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d48f7a87b0 con 0x55d48f7d9e20 2024-05-09T00:05:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb24ff9700 10 client.30290 handle_command_reply: tid=0 2024-05-09T00:05:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 client.30290 shutdown 2024-05-09T00:05:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1143676990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 msgr2=0x55d48f7ead60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7ead60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0059d0 tx=0x7fdb2c00bf10 comp rx=0 tx=0).stop 2024-05-09T00:05:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1143676990 shutdown_connections 2024-05-09T00:05:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e7a80 0x55d48f7eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d48f7d9e20 0x55d48f7dc240 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1143676990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7ead60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1143676990 >> 172.21.15.12:0/1143676990 conn(0x55d48f7371b0 msgr2=0x55d48f7e4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb24ff9700 0 client.30290 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1143676990 shutdown_connections 2024-05-09T00:05:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.529+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1143676990 wait complete. 2024-05-09T00:05:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.530+0000 7fdb4f81db80 20 client.30290 trim_cache size 0 max 16384 2024-05-09T00:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.548+0000 7fdb4f81db80 1 Processor -- start 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.548+0000 7fdb4f81db80 1 -- start start 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e1c90 0x55d48f7d50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f7e25b0 con 0x55d48f7e7a80 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f7e0940 con 0x55d48f7e7160 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f7dc210 con 0x55d48f7e1c90 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb41c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb413ff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e1c90 0x55d48f7d50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb41c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59216/0 (socket says 172.21.15.12:59216) 2024-05-09T00:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb41c00700 1 -- 172.21.15.12:0/4257093978 learned_addr learned my addr 172.21.15.12:0/4257093978 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/4257093978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb41c00700 1 -- 172.21.15.12:0/4257093978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e1c90 msgr2=0x55d48f7d50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb41c00700 1 --2- 172.21.15.12:0/4257093978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e1c90 0x55d48f7d50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb41c00700 1 -- 172.21.15.12:0/4257093978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 msgr2=0x55d48f7cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb41c00700 1 --2- 172.21.15.12:0/4257093978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.549+0000 7fdb41c00700 1 -- 172.21.15.12:0/4257093978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d48f6ef500 con 0x55d48f7e7a80 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.550+0000 7fdb41c00700 1 --2- 172.21.15.12:0/4257093978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e2070 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fdb34003d20 tx=0x7fdb34003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14de3abdc083ae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.550+0000 7fdb27fff700 1 -- 172.21.15.12:0/4257093978 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb340048c0 con 0x55d48f7e7a80 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.550+0000 7fdb27fff700 1 -- 172.21.15.12:0/4257093978 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb3400f040 con 0x55d48f7e7a80 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.550+0000 7fdb27fff700 1 -- 172.21.15.12:0/4257093978 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb340167a0 con 0x55d48f7e7a80 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 -- 172.21.15.12:0/4257093978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 msgr2=0x55d48f7e2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/4257093978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e2070 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fdb34003d20 tx=0x7fdb34003d50 comp rx=0 tx=0).stop 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 -- 172.21.15.12:0/4257093978 shutdown_connections 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/4257093978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e7a80 0x55d48f7e2070 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/4257093978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e1c90 0x55d48f7d50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/4257093978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7160 0x55d48f7cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 -- 172.21.15.12:0/4257093978 >> 172.21.15.12:0/4257093978 conn(0x55d48f7371b0 msgr2=0x55d48f7dfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 -- 172.21.15.12:0/4257093978 shutdown_connections 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 -- 172.21.15.12:0/4257093978 wait complete. 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 -- start start 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.551+0000 7fdb4f81db80 1 Processor -- start 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e7160 0x55d48f7ed610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb4f81db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7a80 0x55d48f7ec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f7edf20 con 0x55d48f7e1c90 2024-05-09T00:05:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f7ecad0 con 0x55d48f7e7a80 2024-05-09T00:05:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb4f81db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d48f7ecc90 con 0x55d48f7e7160 2024-05-09T00:05:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb4f81db80 10 client.0 _ll_register_callbacks cb 0x55d48f6cead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb413ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb41c00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7a80 0x55d48f7ec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb413ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59228/0 (socket says 172.21.15.12:59228) 2024-05-09T00:05:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb413ff700 1 -- 172.21.15.12:0/1801318751 learned_addr learned my addr 172.21.15.12:0/1801318751 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e7160 0x55d48f7ed610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb413ff700 1 -- 172.21.15.12:0/1801318751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e7160 msgr2=0x55d48f7ed610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb413ff700 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e7160 0x55d48f7ed610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb413ff700 1 -- 172.21.15.12:0/1801318751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7a80 msgr2=0x55d48f7ec590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb413ff700 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7a80 0x55d48f7ec590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.552+0000 7fdb413ff700 1 -- 172.21.15.12:0/1801318751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb34003a00 con 0x55d48f7e1c90 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.553+0000 7fdb413ff700 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7f28f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fdb3809ace0 tx=0x7fdb380a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172f6c7cf839ac91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.553+0000 7fdb24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.553+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1801318751 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380b2040 con 0x55d48f7e1c90 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.553+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1801318751 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdb380b7030 con 0x55d48f7e1c90 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.553+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1801318751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb2c015070 con 0x55d48f7e1c90 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.553+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1801318751 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb380a3600 con 0x55d48f7e1c90 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.553+0000 7fdb4f81db80 10 client.20739 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.553+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1801318751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55d48f5aec70 con 0x55d48f7e1c90 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.553+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1801318751 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb2c015070 con 0x55d48f7e1c90 2024-05-09T00:05:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.553+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1801318751 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb380b92e0 con 0x55d48f7e1c90 2024-05-09T00:05:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.554+0000 7fdb4f81db80 10 client.20739 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.554+0000 7fdb4f81db80 10 client.20739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.554+0000 7fdb4f81db80 20 client.20739 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.554+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d48f85a6d0 0x55d48f85cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.554+0000 7fdb4f81db80 4 client.20739 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:05:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.554+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1801318751 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d48f721e60 con 0x55d48f85a6d0 2024-05-09T00:05:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.554+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d48f85a6d0 0x55d48f85cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.554+0000 7fdb40bfe700 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d48f85a6d0 0x55d48f85cac0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b877fd2e3940c55a server_cookie=119ec23655465afb in_seq=0 out_seq=0 2024-05-09T00:05:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.554+0000 7fdb24ff9700 10 client.20739 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb24ff9700 1 -- 172.21.15.12:0/1801318751 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d48f721e60 con 0x55d48f85a6d0 2024-05-09T00:05:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb24ff9700 10 client.20739 handle_command_reply: tid=0 2024-05-09T00:05:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb4f81db80 1 client.20739 shutdown 2024-05-09T00:05:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1801318751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 msgr2=0x55d48f7f28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7f28f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fdb3809ace0 tx=0x7fdb380a1040 comp rx=0 tx=0).stop 2024-05-09T00:05:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1801318751 shutdown_connections 2024-05-09T00:05:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d48f85a6d0 0x55d48f85cac0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d48f7e7a80 0x55d48f7ec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d48f7e1c90 0x55d48f7f28f0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb24ff9700 0 client.20739 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb4f81db80 1 --2- 172.21.15.12:0/1801318751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d48f7e7160 0x55d48f7ed610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.930+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1801318751 >> 172.21.15.12:0/1801318751 conn(0x55d48f7371b0 msgr2=0x55d48f7dfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.931+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1801318751 shutdown_connections 2024-05-09T00:05:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.931+0000 7fdb4f81db80 1 -- 172.21.15.12:0/1801318751 wait complete. 2024-05-09T00:05:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.931+0000 7fdb4f81db80 20 client.20739 trim_cache size 0 max 16384 2024-05-09T00:05:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:05:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:05:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:05:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:05:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.931+0000 7fdb43663700 1 -- 172.21.15.12:0/4185416406 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb280a56a0 msgr2=0x7fdb280a7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.931+0000 7fdb43663700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb280a56a0 0x7fdb280a7b60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c1988c0 tx=0x7fdb2c00b410 comp rx=0 tx=0).stop 2024-05-09T00:05:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.931+0000 7fdb43663700 1 -- 172.21.15.12:0/4185416406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3c101110 msgr2=0x7fdb3c197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.931+0000 7fdb43663700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3c101110 0x7fdb3c197370 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fdb38004760 tx=0x7fdb38004790 comp rx=0 tx=0).stop 2024-05-09T00:05:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.932+0000 7fdb43663700 1 -- 172.21.15.12:0/4185416406 shutdown_connections 2024-05-09T00:05:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.932+0000 7fdb43663700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb280a56a0 0x7fdb280a7b60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.932+0000 7fdb43663700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3c19bc50 0x7fdb3c19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.932+0000 7fdb43663700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3c101110 0x7fdb3c197370 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.932+0000 7fdb43663700 1 --2- 172.21.15.12:0/4185416406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb3c106230 0x7fdb3c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.932+0000 7fdb43663700 1 -- 172.21.15.12:0/4185416406 >> 172.21.15.12:0/4185416406 conn(0x7fdb3c0fc9b0 msgr2=0x7fdb3c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.932+0000 7fdb43663700 1 -- 172.21.15.12:0/4185416406 shutdown_connections 2024-05-09T00:05:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:32.932+0000 7fdb43663700 1 -- 172.21.15.12:0/4185416406 wait complete. 2024-05-09T00:05:33.263 INFO:tasks.workunit.client.0.smithi012.stdout: 10 220541 34.62 MB/sec execute 208 sec latency 332.398 ms 2024-05-09T00:05:34.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:33 smithi204 ceph-mon[47373]: pgmap v615: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-05-09T00:05:34.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:33 smithi160 ceph-mon[46152]: pgmap v615: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-05-09T00:05:34.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:33 smithi012 ceph-mon[42503]: pgmap v615: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-05-09T00:05:34.263 INFO:tasks.workunit.client.0.smithi012.stdout: 10 221209 34.62 MB/sec execute 209 sec latency 85.105 ms 2024-05-09T00:05:35.263 INFO:tasks.workunit.client.0.smithi012.stdout: 10 221867 34.63 MB/sec execute 210 sec latency 95.674 ms 2024-05-09T00:05:36.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:35 smithi204 ceph-mon[47373]: pgmap v616: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.4 MiB/s wr, 534 op/s 2024-05-09T00:05:36.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:35 smithi160 ceph-mon[46152]: pgmap v616: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.4 MiB/s wr, 534 op/s 2024-05-09T00:05:36.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:35 smithi012 ceph-mon[42503]: pgmap v616: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.4 MiB/s wr, 534 op/s 2024-05-09T00:05:36.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 222352 34.56 MB/sec execute 211 sec latency 178.089 ms 2024-05-09T00:05:37.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 223073 34.61 MB/sec execute 212 sec latency 130.368 ms 2024-05-09T00:05:38.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:37 smithi204 ceph-mon[47373]: pgmap v617: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 782 op/s 2024-05-09T00:05:38.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:37 smithi160 ceph-mon[46152]: pgmap v617: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 782 op/s 2024-05-09T00:05:38.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:37 smithi012 ceph-mon[42503]: pgmap v617: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 782 op/s 2024-05-09T00:05:38.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 223662 34.61 MB/sec execute 213 sec latency 100.762 ms 2024-05-09T00:05:39.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 224290 34.58 MB/sec execute 214 sec latency 85.758 ms 2024-05-09T00:05:40.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:39 smithi204 ceph-mon[47373]: pgmap v618: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.1 MiB/s wr, 629 op/s 2024-05-09T00:05:40.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:39 smithi160 ceph-mon[46152]: pgmap v618: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.1 MiB/s wr, 629 op/s 2024-05-09T00:05:40.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:39 smithi012 ceph-mon[42503]: pgmap v618: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.1 MiB/s wr, 629 op/s 2024-05-09T00:05:40.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 224926 34.60 MB/sec execute 215 sec latency 108.115 ms 2024-05-09T00:05:41.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 225404 34.56 MB/sec execute 216 sec latency 315.461 ms 2024-05-09T00:05:42.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 225901 34.50 MB/sec execute 217 sec latency 120.595 ms 2024-05-09T00:05:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:41 smithi204 ceph-mon[47373]: pgmap v619: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.8 MiB/s wr, 615 op/s 2024-05-09T00:05:42.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:41 smithi160 ceph-mon[46152]: pgmap v619: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.8 MiB/s wr, 615 op/s 2024-05-09T00:05:42.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:41 smithi012 ceph-mon[42503]: pgmap v619: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.8 MiB/s wr, 615 op/s 2024-05-09T00:05:43.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 226575 34.50 MB/sec execute 218 sec latency 105.198 ms 2024-05-09T00:05:44.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:43 smithi204 ceph-mon[47373]: pgmap v620: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 803 op/s 2024-05-09T00:05:44.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:43 smithi160 ceph-mon[46152]: pgmap v620: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 803 op/s 2024-05-09T00:05:44.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:43 smithi012 ceph-mon[42503]: pgmap v620: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 803 op/s 2024-05-09T00:05:44.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 227273 34.53 MB/sec execute 219 sec latency 110.829 ms 2024-05-09T00:05:44.808 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:05:44.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.978+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3723237096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa180ffad0 msgr2=0x7ffa180ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:44.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.978+0000 7ffa1fc1d700 1 --2- 172.21.15.12:0/3723237096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa180ffad0 0x7ffa180ffeb0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400b0a0 tx=0x7ffa1400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.978+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3723237096 shutdown_connections 2024-05-09T00:05:44.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.978+0000 7ffa1fc1d700 1 --2- 172.21.15.12:0/3723237096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa181062e0 0x7ffa1810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.978+0000 7ffa1fc1d700 1 --2- 172.21.15.12:0/3723237096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa180ffad0 0x7ffa180ffeb0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.978+0000 7ffa1fc1d700 1 --2- 172.21.15.12:0/3723237096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa18100480 0x7ffa18105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.978+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3723237096 >> 172.21.15.12:0/3723237096 conn(0x7ffa180750c0 msgr2=0x7ffa180754d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:44.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.978+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3723237096 shutdown_connections 2024-05-09T00:05:44.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.978+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3723237096 wait complete. 2024-05-09T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.979+0000 7ffa1fc1d700 1 Processor -- start 2024-05-09T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.979+0000 7ffa1fc1d700 1 -- start start 2024-05-09T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1fc1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa180ffad0 0x7ffa18100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1fc1d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa181062e0 0x7ffa18100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1fc1d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa181014c0 0x7ffa181a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1fc1d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa181a5df0 con 0x7ffa180ffad0 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1fc1d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa181a5f60 con 0x7ffa181014c0 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1fc1d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa181a6120 con 0x7ffa181062e0 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1d9b9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa180ffad0 0x7ffa18100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1d9b9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa180ffad0 0x7ffa18100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58878/0 (socket says 172.21.15.12:58878) 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1d9b9700 1 -- 172.21.15.12:0/3096445307 learned_addr learned my addr 172.21.15.12:0/3096445307 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1d1b8700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa181062e0 0x7ffa18100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa181014c0 0x7ffa181a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1d1b8700 1 -- 172.21.15.12:0/3096445307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa181014c0 msgr2=0x7ffa181a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1d1b8700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa181014c0 0x7ffa181a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1d1b8700 1 -- 172.21.15.12:0/3096445307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa180ffad0 msgr2=0x7ffa18100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1d1b8700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa180ffad0 0x7ffa18100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1d1b8700 1 -- 172.21.15.12:0/3096445307 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa14009d00 con 0x7ffa181062e0 2024-05-09T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.980+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa181014c0 0x7ffa181a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.981+0000 7ffa1d1b8700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa181062e0 0x7ffa18100f80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800e9c0 tx=0x7ffa0800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.981+0000 7ffa0effd700 1 -- 172.21.15.12:0/3096445307 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800c9c0 con 0x7ffa181062e0 2024-05-09T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.981+0000 7ffa0effd700 1 -- 172.21.15.12:0/3096445307 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa08010430 con 0x7ffa181062e0 2024-05-09T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.981+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3096445307 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa181a6400 con 0x7ffa181062e0 2024-05-09T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.981+0000 7ffa0effd700 1 -- 172.21.15.12:0/3096445307 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08018680 con 0x7ffa181062e0 2024-05-09T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.981+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3096445307 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa181a6910 con 0x7ffa181062e0 2024-05-09T00:05:44.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.984+0000 7ffa0effd700 1 -- 172.21.15.12:0/3096445307 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ffa0800cb20 con 0x7ffa181062e0 2024-05-09T00:05:44.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.985+0000 7ffa0effd700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1290 0x7ffa040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.985+0000 7ffa0effd700 1 -- 172.21.15.12:0/3096445307 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ffa08014070 con 0x7ffa181062e0 2024-05-09T00:05:44.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.985+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1290 0x7ffa040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:44.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.986+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1290 0x7ffa040a3750 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400b780 tx=0x7ffa14006230 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:44.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.988+0000 7ffa2bdd7b80 1 Processor -- start 2024-05-09T00:05:44.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa2bdd7b80 1 -- start start 2024-05-09T00:05:44.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 0x55e53f32f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f32f720 0x55e53f329350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 0x55e53f329cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f280150 con 0x55e53f329890 2024-05-09T00:05:44.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f27db10 con 0x55e53f32f720 2024-05-09T00:05:44.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f279400 con 0x55e53f32ee00 2024-05-09T00:05:44.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa1d9b9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 0x55e53f32f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:44.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa1d9b9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 0x55e53f32f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45062/0 (socket says 172.21.15.12:45062) 2024-05-09T00:05:44.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa1d9b9700 1 -- 172.21.15.12:0/2010637154 learned_addr learned my addr 172.21.15.12:0/2010637154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:44.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.989+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/2010637154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 0x55e53f329cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:44.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.990+0000 7ffa1d9b9700 1 -- 172.21.15.12:0/2010637154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f32f720 msgr2=0x55e53f329350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:05:44.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.990+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/2010637154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f32f720 0x55e53f329350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.990+0000 7ffa1d9b9700 1 -- 172.21.15.12:0/2010637154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 msgr2=0x55e53f329cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:44.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.990+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/2010637154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 0x55e53f329cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.990+0000 7ffa1d9b9700 1 -- 172.21.15.12:0/2010637154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e53f0fa3b0 con 0x55e53f32ee00 2024-05-09T00:05:44.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.990+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/2010637154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 0x55e53f32f1e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ffa14007dc0 tx=0x7ffa1401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa0692b7a327ad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:44.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.990+0000 7ffa03fff700 1 -- 172.21.15.12:0/2010637154 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa14023040 con 0x55e53f32ee00 2024-05-09T00:05:44.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.990+0000 7ffa03fff700 1 -- 172.21.15.12:0/2010637154 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa1400bd30 con 0x55e53f32ee00 2024-05-09T00:05:44.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.990+0000 7ffa03fff700 1 -- 172.21.15.12:0/2010637154 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1400e040 con 0x55e53f32ee00 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.991+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/2010637154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 msgr2=0x55e53f32f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.991+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/2010637154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 0x55e53f32f1e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ffa14007dc0 tx=0x7ffa1401c000 comp rx=0 tx=0).stop 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.991+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/2010637154 shutdown_connections 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.991+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/2010637154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 0x55e53f329cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.991+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/2010637154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 0x55e53f32f1e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.991+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/2010637154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f32f720 0x55e53f329350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.991+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/2010637154 >> 172.21.15.12:0/2010637154 conn(0x55e53f287210 msgr2=0x55e53f323df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.992+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/2010637154 shutdown_connections 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.992+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/2010637154 wait complete. 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.992+0000 7ffa2bdd7b80 1 -- start start 2024-05-09T00:05:44.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.992+0000 7ffa2bdd7b80 1 Processor -- start 2024-05-09T00:05:44.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 0x55e53f332c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f32ee00 0x55e53f333180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f3336c0 0x55e53f350150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f215560 con 0x55e53f3336c0 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f215200 con 0x55e53f32ee00 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f0fc6a0 con 0x55e53f329890 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa2bdd7b80 10 client.0 _ll_register_callbacks cb 0x55e53f21eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa1e1ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f3336c0 0x55e53f350150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa1e1ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f3336c0 0x55e53f350150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58896/0 (socket says 172.21.15.12:58896) 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa1e1ba700 1 -- 172.21.15.12:0/545065927 learned_addr learned my addr 172.21.15.12:0/545065927 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 0x55e53f332c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.993+0000 7ffa1d1b8700 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f32ee00 0x55e53f333180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa1e1ba700 1 -- 172.21.15.12:0/545065927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 msgr2=0x55e53f332c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 0x55e53f332c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa1e1ba700 1 -- 172.21.15.12:0/545065927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f32ee00 msgr2=0x55e53f333180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f32ee00 0x55e53f333180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa1e1ba700 1 -- 172.21.15.12:0/545065927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1810e710 con 0x55e53f3336c0 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 0x55e53f332c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f3336c0 0x55e53f350150 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ffa10012cf0 tx=0x7ffa10009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90794346d07d2743 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa017fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa017fa700 1 -- 172.21.15.12:0/545065927 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10017070 con 0x55e53f3336c0 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/545065927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e53f1d2270 con 0x55e53f3336c0 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa017fa700 1 -- 172.21.15.12:0/545065927 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ffa1000f870 con 0x55e53f3336c0 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa017fa700 1 -- 172.21.15.12:0/545065927 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa100122e0 con 0x55e53f3336c0 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa2bdd7b80 10 client.20751 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.994+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/545065927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55e53f0a6b60 con 0x55e53f3336c0 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.995+0000 7ffa017fa700 1 -- 172.21.15.12:0/545065927 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa10028080 con 0x55e53f3336c0 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.995+0000 7ffa017fa700 1 -- 172.21.15.12:0/545065927 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffa10010b50 con 0x55e53f3336c0 2024-05-09T00:05:44.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.995+0000 7ffa2bdd7b80 10 client.20751 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:44.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.995+0000 7ffa2bdd7b80 10 client.20751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:44.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.995+0000 7ffa2bdd7b80 20 client.20751 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:44.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.995+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e53f354750 0x55e53f356b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:44.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.995+0000 7ffa2bdd7b80 4 client.20751 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:44.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.995+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/545065927 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e53f2f8810 con 0x55e53f354750 2024-05-09T00:05:44.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.995+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e53f354750 0x55e53f356b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:44.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.996+0000 7ffa017fa700 10 client.20751 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:44.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.996+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e53f354750 0x55e53f356b70 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936f333b055e62d4 server_cookie=9cb0ef085578de3a in_seq=0 out_seq=0 2024-05-09T00:05:44.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa017fa700 1 -- 172.21.15.12:0/545065927 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e53f2f8810 con 0x55e53f354750 2024-05-09T00:05:44.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa017fa700 10 client.20751 handle_command_reply: tid=0 2024-05-09T00:05:44.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 client.20751 shutdown 2024-05-09T00:05:44.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/545065927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f3336c0 msgr2=0x55e53f350150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:44.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f3336c0 0x55e53f350150 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ffa10012cf0 tx=0x7ffa10009e00 comp rx=0 tx=0).stop 2024-05-09T00:05:44.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/545065927 shutdown_connections 2024-05-09T00:05:44.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f3336c0 0x55e53f350150 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 0x55e53f332c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e53f354750 0x55e53f356b70 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/545065927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f32ee00 0x55e53f333180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:44.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/545065927 >> 172.21.15.12:0/545065927 conn(0x55e53f287210 msgr2=0x55e53f3235a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:44.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa017fa700 0 client.20751 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:44.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/545065927 shutdown_connections 2024-05-09T00:05:44.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/545065927 wait complete. 2024-05-09T00:05:44.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:44.999+0000 7ffa2bdd7b80 20 client.20751 trim_cache size 0 max 16384 2024-05-09T00:05:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.021+0000 7ffa2bdd7b80 1 Processor -- start 2024-05-09T00:05:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.021+0000 7ffa2bdd7b80 1 -- start start 2024-05-09T00:05:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.021+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 0x55e53f329350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.021+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f32ee00 0x55e53f323d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.021+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 0x55e53f32dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.021+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f3256b0 con 0x55e53f32ee00 2024-05-09T00:05:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.021+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f35bdb0 con 0x55e53f3336c0 2024-05-09T00:05:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.021+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f31e830 con 0x55e53f329890 2024-05-09T00:05:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1d1b8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f32ee00 0x55e53f323d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1d1b8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f32ee00 0x55e53f323d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58910/0 (socket says 172.21.15.12:58910) 2024-05-09T00:05:45.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1d9b9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 0x55e53f329350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1d1b8700 1 -- 172.21.15.12:0/1787385514 learned_addr learned my addr 172.21.15.12:0/1787385514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:45.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/1787385514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 0x55e53f32dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1d1b8700 1 -- 172.21.15.12:0/1787385514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 msgr2=0x55e53f329350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1d1b8700 1 --2- 172.21.15.12:0/1787385514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 0x55e53f329350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1d1b8700 1 -- 172.21.15.12:0/1787385514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 msgr2=0x55e53f32dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1d1b8700 1 --2- 172.21.15.12:0/1787385514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 0x55e53f32dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1d1b8700 1 -- 172.21.15.12:0/1787385514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e53f329f40 con 0x55e53f32ee00 2024-05-09T00:05:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/1787385514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 0x55e53f32dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.022+0000 7ffa1d1b8700 1 --2- 172.21.15.12:0/1787385514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f32ee00 0x55e53f323d20 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ffa08099ec0 tx=0x7ffa08099af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8814b0c36c717a24 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.023+0000 7ffa0cda4700 1 -- 172.21.15.12:0/1787385514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08099f30 con 0x55e53f32ee00 2024-05-09T00:05:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.023+0000 7ffa0cda4700 1 -- 172.21.15.12:0/1787385514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0809dd60 con 0x55e53f32ee00 2024-05-09T00:05:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.023+0000 7ffa0cda4700 1 -- 172.21.15.12:0/1787385514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08010860 con 0x55e53f32ee00 2024-05-09T00:05:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.024+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/1787385514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f32ee00 msgr2=0x55e53f323d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.024+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/1787385514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f32ee00 0x55e53f323d20 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ffa08099ec0 tx=0x7ffa08099af0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.024+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/1787385514 shutdown_connections 2024-05-09T00:05:45.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.024+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/1787385514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 0x55e53f32dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.024+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/1787385514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f329890 0x55e53f329350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.024+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/1787385514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f32ee00 0x55e53f323d20 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.024+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/1787385514 >> 172.21.15.12:0/1787385514 conn(0x55e53f287210 msgr2=0x55e53f331510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.024+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/1787385514 shutdown_connections 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.024+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/1787385514 wait complete. 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.025+0000 7ffa2bdd7b80 1 -- start start 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.025+0000 7ffa2bdd7b80 1 Processor -- start 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.025+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 0x55e53f349310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.025+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 0x55e53f349850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.025+0000 7ffa2bdd7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 0x55e53f3a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.025+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f3aa190 con 0x55e53f329890 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.025+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f3aa2d0 con 0x55e53f3336c0 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.025+0000 7ffa2bdd7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53f3aa460 con 0x55e53f32ee00 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa2bdd7b80 10 client.0 _ll_register_callbacks cb 0x55e53f21eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1d9b9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 0x55e53f349310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1d9b9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 0x55e53f349310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58916/0 (socket says 172.21.15.12:58916) 2024-05-09T00:05:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1d1b8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 0x55e53f349850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1d9b9700 1 -- 172.21.15.12:0/3610981795 learned_addr learned my addr 172.21.15.12:0/3610981795 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 0x55e53f3a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1e1ba700 1 -- 172.21.15.12:0/3610981795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 msgr2=0x55e53f349850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 0x55e53f349850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1e1ba700 1 -- 172.21.15.12:0/3610981795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 msgr2=0x55e53f349310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 0x55e53f349310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1e1ba700 1 -- 172.21.15.12:0/3610981795 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e53f0a6b60 con 0x55e53f3336c0 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.026+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 0x55e53f349310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa1e1ba700 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 0x55e53f3a9c50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55e53f331420 tx=0x7ffa1000faa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bace89fb30526202 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa017fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa017fa700 1 -- 172.21.15.12:0/3610981795 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10017070 con 0x55e53f3336c0 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/3610981795 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa1400bd30 con 0x55e53f3336c0 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa017fa700 1 -- 172.21.15.12:0/3610981795 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ffa10000e40 con 0x55e53f3336c0 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa017fa700 1 -- 172.21.15.12:0/3610981795 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa100122e0 con 0x55e53f3336c0 2024-05-09T00:05:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa2bdd7b80 10 client.30311 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/3610981795 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7ffa181a6910 con 0x55e53f3336c0 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa017fa700 1 -- 172.21.15.12:0/3610981795 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa1400bd30 con 0x55e53f3336c0 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa017fa700 1 -- 172.21.15.12:0/3610981795 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffa10010430 con 0x55e53f3336c0 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa2bdd7b80 10 client.30311 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa2bdd7b80 10 client.30311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa2bdd7b80 20 client.30311 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e53f3abc30 0x55e53f3ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa2bdd7b80 4 client.30311 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.027+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/3610981795 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e53f271ec0 con 0x55e53f3abc30 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.028+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e53f3abc30 0x55e53f3ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.028+0000 7ffa017fa700 10 client.30311 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.028+0000 7ffa1d9b9700 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e53f3abc30 0x55e53f3ae050 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7fd34cee13a8e77 server_cookie=a903d025cea36b6 in_seq=0 out_seq=0 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa017fa700 1 -- 172.21.15.12:0/3610981795 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e53f271ec0 con 0x55e53f3abc30 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa017fa700 10 client.30311 handle_command_reply: tid=0 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa2bdd7b80 1 client.30311 shutdown 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/3610981795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 msgr2=0x55e53f3a9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 0x55e53f3a9c50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55e53f331420 tx=0x7ffa1000faa0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/3610981795 shutdown_connections 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e53f3abc30 0x55e53f3ae050 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e53f3336c0 0x55e53f3a9c50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e53f329890 0x55e53f349310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa2bdd7b80 1 --2- 172.21.15.12:0/3610981795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e53f32ee00 0x55e53f349850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/3610981795 >> 172.21.15.12:0/3610981795 conn(0x55e53f287210 msgr2=0x55e53f32dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa017fa700 0 client.30311 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:45.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.029+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/3610981795 shutdown_connections 2024-05-09T00:05:45.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa2bdd7b80 1 -- 172.21.15.12:0/3610981795 wait complete. 2024-05-09T00:05:45.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa2bdd7b80 20 client.30311 trim_cache size 0 max 16384 2024-05-09T00:05:45.028 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:05:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:05:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:05:45.028 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:05:45.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3096445307 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1290 msgr2=0x7ffa040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa1fc1d700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1290 0x7ffa040a3750 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400b780 tx=0x7ffa14006230 comp rx=0 tx=0).stop 2024-05-09T00:05:45.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3096445307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa181062e0 msgr2=0x7ffa18100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa1fc1d700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa181062e0 0x7ffa18100f80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800e9c0 tx=0x7ffa0800ecd0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3096445307 shutdown_connections 2024-05-09T00:05:45.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa1fc1d700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffa040a1290 0x7ffa040a3750 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa1fc1d700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa181014c0 0x7ffa181a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa1fc1d700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa180ffad0 0x7ffa18100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa1fc1d700 1 --2- 172.21.15.12:0/3096445307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa181062e0 0x7ffa18100f80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.030+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3096445307 >> 172.21.15.12:0/3096445307 conn(0x7ffa180750c0 msgr2=0x7ffa180fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.031+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3096445307 shutdown_connections 2024-05-09T00:05:45.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.031+0000 7ffa1fc1d700 1 -- 172.21.15.12:0/3096445307 wait complete. 2024-05-09T00:05:45.041 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:05:45.041 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:05:45.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.227+0000 7f8a6c837700 1 -- 172.21.15.12:0/2640123089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a681008f0 msgr2=0x7f8a68104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.227+0000 7f8a6c837700 1 --2- 172.21.15.12:0/2640123089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a681008f0 0x7f8a68104930 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009990 tx=0x7f8a5c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.227+0000 7f8a6c837700 1 -- 172.21.15.12:0/2640123089 shutdown_connections 2024-05-09T00:05:45.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.227+0000 7f8a6c837700 1 --2- 172.21.15.12:0/2640123089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a68105060 0x7f8a6810cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.227+0000 7f8a6c837700 1 --2- 172.21.15.12:0/2640123089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a680fff40 0x7f8a68100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.227+0000 7f8a6c837700 1 --2- 172.21.15.12:0/2640123089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a681008f0 0x7f8a68104930 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.227+0000 7f8a6c837700 1 -- 172.21.15.12:0/2640123089 >> 172.21.15.12:0/2640123089 conn(0x7f8a680fb7e0 msgr2=0x7f8a680fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.227+0000 7f8a6c837700 1 -- 172.21.15.12:0/2640123089 shutdown_connections 2024-05-09T00:05:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.227+0000 7f8a6c837700 1 -- 172.21.15.12:0/2640123089 wait complete. 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6c837700 1 Processor -- start 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6c837700 1 -- start start 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6c837700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a680fff40 0x7f8a68197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6c837700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a68105060 0x7f8a681978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6c837700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a6819bc40 0x7f8a6819c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6c837700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6819c6c0 con 0x7f8a680fff40 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6c837700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6819c830 con 0x7f8a6819bc40 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6c837700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6819c9f0 con 0x7f8a68105060 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a680fff40 0x7f8a68197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a680fff40 0x7f8a68197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58934/0 (socket says 172.21.15.12:58934) 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6659c700 1 -- 172.21.15.12:0/1050287215 learned_addr learned my addr 172.21.15.12:0/1050287215 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:45.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a65d9b700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a68105060 0x7f8a681978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a66d9d700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a6819bc40 0x7f8a6819c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6659c700 1 -- 172.21.15.12:0/1050287215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a68105060 msgr2=0x7f8a681978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6659c700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a68105060 0x7f8a681978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6659c700 1 -- 172.21.15.12:0/1050287215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a6819bc40 msgr2=0x7f8a6819c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6659c700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a6819bc40 0x7f8a6819c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.228+0000 7f8a6659c700 1 -- 172.21.15.12:0/1050287215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c0095f0 con 0x7f8a680fff40 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.229+0000 7f8a6659c700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a680fff40 0x7f8a68197360 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800e9a0 tx=0x7f8a5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.229+0000 7f8a577fe700 1 -- 172.21.15.12:0/1050287215 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58009790 con 0x7f8a680fff40 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.229+0000 7f8a6c837700 1 -- 172.21.15.12:0/1050287215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a681a3f50 con 0x7f8a680fff40 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.229+0000 7f8a577fe700 1 -- 172.21.15.12:0/1050287215 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a58010430 con 0x7f8a680fff40 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.229+0000 7f8a577fe700 1 -- 172.21.15.12:0/1050287215 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58018680 con 0x7f8a680fff40 2024-05-09T00:05:45.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.229+0000 7f8a6c837700 1 -- 172.21.15.12:0/1050287215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a681a4420 con 0x7f8a680fff40 2024-05-09T00:05:45.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.230+0000 7f8a6c837700 1 -- 172.21.15.12:0/1050287215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a68108240 con 0x7f8a680fff40 2024-05-09T00:05:45.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.232+0000 7f8a577fe700 1 -- 172.21.15.12:0/1050287215 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8a580105a0 con 0x7f8a680fff40 2024-05-09T00:05:45.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.232+0000 7f8a577fe700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a500a1310 0x7f8a500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.233+0000 7f8a577fe700 1 -- 172.21.15.12:0/1050287215 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8a58014070 con 0x7f8a680fff40 2024-05-09T00:05:45.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.233+0000 7f8a65d9b700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a500a1310 0x7f8a500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.234+0000 7f8a65d9b700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a500a1310 0x7f8a500a37d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8a681988b0 tx=0x7f8a5c005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:45.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.239+0000 7f8a577fe700 1 -- 172.21.15.12:0/1050287215 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8a58052d60 con 0x7f8a680fff40 2024-05-09T00:05:45.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 227944 34.55 MB/sec execute 220 sec latency 96.967 ms 2024-05-09T00:05:45.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.583+0000 7f8a6c837700 1 -- 172.21.15.12:0/1050287215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a68101410 con 0x7f8a680fff40 2024-05-09T00:05:45.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.584+0000 7f8a577fe700 1 -- 172.21.15.12:0/1050287215 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+10872 (secure 0 0 0) 0x7f8a580d3080 con 0x7f8a680fff40 2024-05-09T00:05:45.578 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:05:45.578 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":74,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:03:37.323291+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_30034":{"gid":30034,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6833/181468454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":181468454},{"type":"v1","addr":"172.21.15.160:6833","nonce":181468454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:05:45.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 -- 172.21.15.12:0/1050287215 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a500a1310 msgr2=0x7f8a500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a500a1310 0x7f8a500a37d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8a681988b0 tx=0x7f8a5c005fd0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 -- 172.21.15.12:0/1050287215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a680fff40 msgr2=0x7f8a68197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a680fff40 0x7f8a68197360 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800e9a0 tx=0x7f8a5800ecb0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 -- 172.21.15.12:0/1050287215 shutdown_connections 2024-05-09T00:05:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a500a1310 0x7f8a500a37d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a6819bc40 0x7f8a6819c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a680fff40 0x7f8a68197360 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 --2- 172.21.15.12:0/1050287215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a68105060 0x7f8a681978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 -- 172.21.15.12:0/1050287215 >> 172.21.15.12:0/1050287215 conn(0x7f8a680fb7e0 msgr2=0x7f8a68103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 -- 172.21.15.12:0/1050287215 shutdown_connections 2024-05-09T00:05:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.586+0000 7f8a6c837700 1 -- 172.21.15.12:0/1050287215 wait complete. 2024-05-09T00:05:45.582 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 74 2024-05-09T00:05:45.595 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:05:45.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.767+0000 7f786c4c0700 1 -- 172.21.15.12:0/1281042123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786410db90 msgr2=0x7f786410ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.767+0000 7f786c4c0700 1 --2- 172.21.15.12:0/1281042123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786410db90 0x7f786410ff80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f785c009f90 tx=0x7f785c00b330 comp rx=0 tx=0).stop 2024-05-09T00:05:45.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.767+0000 7f786c4c0700 1 -- 172.21.15.12:0/1281042123 shutdown_connections 2024-05-09T00:05:45.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.767+0000 7f786c4c0700 1 --2- 172.21.15.12:0/1281042123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786410db90 0x7f786410ff80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.767+0000 7f786c4c0700 1 --2- 172.21.15.12:0/1281042123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7864102010 0x7f78640ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.767+0000 7f786c4c0700 1 --2- 172.21.15.12:0/1281042123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78640ff950 0x7f78640ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.767+0000 7f786c4c0700 1 -- 172.21.15.12:0/1281042123 >> 172.21.15.12:0/1281042123 conn(0x7f78640763a0 msgr2=0x7f78640fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.767+0000 7f786c4c0700 1 -- 172.21.15.12:0/1281042123 shutdown_connections 2024-05-09T00:05:45.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.767+0000 7f786c4c0700 1 -- 172.21.15.12:0/1281042123 wait complete. 2024-05-09T00:05:45.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.768+0000 7f786c4c0700 1 Processor -- start 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.768+0000 7f786c4c0700 1 -- start start 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786c4c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78640ff950 0x7f786419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786c4c0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7864102010 0x7f786419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786c4c0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786410db90 0x7f7864199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786c4c0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864199dc0 con 0x7f78640ff950 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786c4c0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864199f30 con 0x7f7864102010 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786c4c0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786419a0f0 con 0x7f786410db90 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786a25c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78640ff950 0x7f786419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786a25c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78640ff950 0x7f786419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58962/0 (socket says 172.21.15.12:58962) 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786a25c700 1 -- 172.21.15.12:0/1681835843 learned_addr learned my addr 172.21.15.12:0/1681835843 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786aa5d700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786410db90 0x7f7864199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f7869a5b700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7864102010 0x7f786419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786aa5d700 1 -- 172.21.15.12:0/1681835843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7864102010 msgr2=0x7f786419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786aa5d700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7864102010 0x7f786419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786aa5d700 1 -- 172.21.15.12:0/1681835843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78640ff950 msgr2=0x7f786419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786aa5d700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78640ff950 0x7f786419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786aa5d700 1 -- 172.21.15.12:0/1681835843 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854009520 con 0x7f786410db90 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786a25c700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78640ff950 0x7f786419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.769+0000 7f786aa5d700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786410db90 0x7f7864199760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f785c00b6c0 tx=0x7f785c0094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.770+0000 7f785b7fe700 1 -- 172.21.15.12:0/1681835843 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00e040 con 0x7f786410db90 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.770+0000 7f785b7fe700 1 -- 172.21.15.12:0/1681835843 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f785c007e10 con 0x7f786410db90 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.770+0000 7f786c4c0700 1 -- 172.21.15.12:0/1681835843 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78600095f0 con 0x7f786410db90 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.770+0000 7f785b7fe700 1 -- 172.21.15.12:0/1681835843 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c005040 con 0x7f786410db90 2024-05-09T00:05:45.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.770+0000 7f786c4c0700 1 -- 172.21.15.12:0/1681835843 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c009c40 con 0x7f786410db90 2024-05-09T00:05:45.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.772+0000 7f785b7fe700 1 -- 172.21.15.12:0/1681835843 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f785c01b450 con 0x7f786410db90 2024-05-09T00:05:45.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.773+0000 7f785b7fe700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78500a1310 0x7f78500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.773+0000 7f785b7fe700 1 -- 172.21.15.12:0/1681835843 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f785c0cdf10 con 0x7f786410db90 2024-05-09T00:05:45.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.773+0000 7f786a25c700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78500a1310 0x7f78500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.774+0000 7f786a25c700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78500a1310 0x7f78500a37d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7860005d10 tx=0x7f7860009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:45.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.774+0000 7f787867ab80 1 Processor -- start 2024-05-09T00:05:45.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.774+0000 7f787867ab80 1 -- start start 2024-05-09T00:05:45.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f787867ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2685e00 0x55f6d26861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f787867ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d2680350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f787867ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2680cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f787867ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d25d7150 con 0x55f6d2685e00 2024-05-09T00:05:45.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f787867ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d25d4b10 con 0x55f6d2680890 2024-05-09T00:05:45.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f787867ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d25d0400 con 0x55f6d2686720 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f786aa5d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2680cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f7869a5b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d2680350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f786aa5d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2680cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41144/0 (socket says 172.21.15.12:41144) 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f786aa5d700 1 -- 172.21.15.12:0/3445418429 learned_addr learned my addr 172.21.15.12:0/3445418429 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f786a25c700 1 --2- 172.21.15.12:0/3445418429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2685e00 0x55f6d26861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f7869a5b700 1 -- 172.21.15.12:0/3445418429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 msgr2=0x55f6d2680cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f7869a5b700 1 --2- 172.21.15.12:0/3445418429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2680cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f7869a5b700 1 -- 172.21.15.12:0/3445418429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2685e00 msgr2=0x55f6d26861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f7869a5b700 1 --2- 172.21.15.12:0/3445418429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2685e00 0x55f6d26861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.775+0000 7f7869a5b700 1 -- 172.21.15.12:0/3445418429 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6d24513b0 con 0x55f6d2686720 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.776+0000 7f7869a5b700 1 --2- 172.21.15.12:0/3445418429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d2680350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7854009d60 tx=0x7f78540116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5843bb9e2c97b874 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.776+0000 7f7858ff9700 1 -- 172.21.15.12:0/3445418429 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854011c20 con 0x55f6d2686720 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.776+0000 7f7858ff9700 1 -- 172.21.15.12:0/3445418429 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7854011d80 con 0x55f6d2686720 2024-05-09T00:05:45.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.776+0000 7f7858ff9700 1 -- 172.21.15.12:0/3445418429 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785400f800 con 0x55f6d2686720 2024-05-09T00:05:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.777+0000 7f787867ab80 1 -- 172.21.15.12:0/3445418429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 msgr2=0x55f6d2680350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.777+0000 7f787867ab80 1 --2- 172.21.15.12:0/3445418429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d2680350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7854009d60 tx=0x7f78540116c0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.777+0000 7f787867ab80 1 -- 172.21.15.12:0/3445418429 shutdown_connections 2024-05-09T00:05:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.777+0000 7f787867ab80 1 --2- 172.21.15.12:0/3445418429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2680cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.777+0000 7f787867ab80 1 --2- 172.21.15.12:0/3445418429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2685e00 0x55f6d26861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.777+0000 7f787867ab80 1 --2- 172.21.15.12:0/3445418429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d2680350 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.777+0000 7f787867ab80 1 -- 172.21.15.12:0/3445418429 >> 172.21.15.12:0/3445418429 conn(0x55f6d25de210 msgr2=0x55f6d267adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.778+0000 7f787867ab80 1 -- 172.21.15.12:0/3445418429 shutdown_connections 2024-05-09T00:05:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.778+0000 7f787867ab80 1 -- 172.21.15.12:0/3445418429 wait complete. 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.778+0000 7f787867ab80 1 -- start start 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.778+0000 7f787867ab80 1 Processor -- start 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f787867ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2689ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f787867ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2685e00 0x55f6d268a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f787867ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d268a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f787867ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d256c560 con 0x55f6d2685e00 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f787867ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d256c200 con 0x55f6d2680890 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f787867ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d24536a0 con 0x55f6d2686720 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f787867ab80 10 client.0 _ll_register_callbacks cb 0x55f6d2575b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f786a25c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2689ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f786a25c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2689ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41152/0 (socket says 172.21.15.12:41152) 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f786a25c700 1 -- 172.21.15.12:0/1829683909 learned_addr learned my addr 172.21.15.12:0/1829683909 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f7869a5b700 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2685e00 0x55f6d268a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f786aa5d700 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d268a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f786a25c700 1 -- 172.21.15.12:0/1829683909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 msgr2=0x55f6d268a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f786a25c700 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d268a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f786a25c700 1 -- 172.21.15.12:0/1829683909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2685e00 msgr2=0x55f6d268a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f786a25c700 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2685e00 0x55f6d268a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f786a25c700 1 -- 172.21.15.12:0/1829683909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78540099c0 con 0x55f6d2680890 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f786a25c700 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2689ae0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7860011fb0 tx=0x7f78600047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26e825108a12d37a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.779+0000 7f784dffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.780+0000 7f784dffb700 1 -- 172.21.15.12:0/1829683909 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860022070 con 0x55f6d2680890 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.780+0000 7f784dffb700 1 -- 172.21.15.12:0/1829683909 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f786001ddc0 con 0x55f6d2680890 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.780+0000 7f787867ab80 1 -- 172.21.15.12:0/1829683909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6d2529270 con 0x55f6d2680890 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.780+0000 7f784dffb700 1 -- 172.21.15.12:0/1829683909 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7860026410 con 0x55f6d2680890 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.780+0000 7f787867ab80 10 client.30329 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:45.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.780+0000 7f787867ab80 1 -- 172.21.15.12:0/1829683909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55f6d23fdb60 con 0x55f6d2680890 2024-05-09T00:05:45.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.780+0000 7f784dffb700 1 -- 172.21.15.12:0/1829683909 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6d2529270 con 0x55f6d2680890 2024-05-09T00:05:45.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.781+0000 7f784dffb700 1 -- 172.21.15.12:0/1829683909 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f786002e070 con 0x55f6d2680890 2024-05-09T00:05:45.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.781+0000 7f787867ab80 10 client.30329 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:45.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.781+0000 7f787867ab80 10 client.30329 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:05:45.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.781+0000 7f787867ab80 20 client.30329 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:45.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.781+0000 7f787867ab80 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f6d269a1e0 0x55f6d269c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.781+0000 7f787867ab80 4 client.30329 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:45.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.781+0000 7f787867ab80 1 -- 172.21.15.12:0/1829683909 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6d264f810 con 0x55f6d269a1e0 2024-05-09T00:05:45.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.781+0000 7f7869a5b700 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f6d269a1e0 0x55f6d269c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.781+0000 7f784dffb700 10 client.30329 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.781+0000 7f7869a5b700 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f6d269a1e0 0x55f6d269c600 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c20037443065e26 server_cookie=d75ecb29c360a58c in_seq=0 out_seq=0 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.783+0000 7f784dffb700 1 -- 172.21.15.12:0/1829683909 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f6d264f810 con 0x55f6d269a1e0 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.783+0000 7f784dffb700 10 client.30329 handle_command_reply: tid=0 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.783+0000 7f787867ab80 1 client.30329 shutdown 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 1 -- 172.21.15.12:0/1829683909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 msgr2=0x55f6d2689ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2689ae0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7860011fb0 tx=0x7f78600047c0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 1 -- 172.21.15.12:0/1829683909 shutdown_connections 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f6d269a1e0 0x55f6d269c600 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d268a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2680890 0x55f6d2689ae0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f784dffb700 0 client.30329 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 1 --2- 172.21.15.12:0/1829683909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2685e00 0x55f6d268a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 1 -- 172.21.15.12:0/1829683909 >> 172.21.15.12:0/1829683909 conn(0x55f6d25de210 msgr2=0x55f6d267a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 1 -- 172.21.15.12:0/1829683909 shutdown_connections 2024-05-09T00:05:45.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 1 -- 172.21.15.12:0/1829683909 wait complete. 2024-05-09T00:05:45.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.784+0000 7f787867ab80 20 client.30329 trim_cache size 0 max 16384 2024-05-09T00:05:45.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f787867ab80 1 Processor -- start 2024-05-09T00:05:45.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f787867ab80 1 -- start start 2024-05-09T00:05:45.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f787867ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d26861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f787867ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2685e00 0x55f6d2695c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f787867ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d267ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f787867ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d26806d0 con 0x55f6d2680890 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f787867ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d26a1810 con 0x55f6d2685e00 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f787867ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d2675800 con 0x55f6d2686720 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f786a25c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d26861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f786aa5d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d267ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f786a25c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d26861e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58980/0 (socket says 172.21.15.12:58980) 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.802+0000 7f786a25c700 1 -- 172.21.15.12:0/517399735 learned_addr learned my addr 172.21.15.12:0/517399735 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f7869a5b700 1 --2- 172.21.15.12:0/517399735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2685e00 0x55f6d2695c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f786a25c700 1 -- 172.21.15.12:0/517399735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 msgr2=0x55f6d267ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f786a25c700 1 --2- 172.21.15.12:0/517399735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d267ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f786a25c700 1 -- 172.21.15.12:0/517399735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2685e00 msgr2=0x55f6d2695c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f786a25c700 1 --2- 172.21.15.12:0/517399735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2685e00 0x55f6d2695c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f786a25c700 1 -- 172.21.15.12:0/517399735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6d2596560 con 0x55f6d2680890 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f786aa5d700 1 --2- 172.21.15.12:0/517399735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d267ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f786a25c700 1 --2- 172.21.15.12:0/517399735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d26861e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f7860009930 tx=0x7f786000fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dbba28e1c73356 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f78597fa700 1 -- 172.21.15.12:0/517399735 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860022070 con 0x55f6d2680890 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f78597fa700 1 -- 172.21.15.12:0/517399735 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78600114a0 con 0x55f6d2680890 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.803+0000 7f78597fa700 1 -- 172.21.15.12:0/517399735 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786003a7f0 con 0x55f6d2680890 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.804+0000 7f787867ab80 1 -- 172.21.15.12:0/517399735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 msgr2=0x55f6d26861e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.804+0000 7f787867ab80 1 --2- 172.21.15.12:0/517399735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d26861e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f7860009930 tx=0x7f786000fe10 comp rx=0 tx=0).stop 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.804+0000 7f787867ab80 1 -- 172.21.15.12:0/517399735 shutdown_connections 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.804+0000 7f787867ab80 1 --2- 172.21.15.12:0/517399735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d267ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.804+0000 7f787867ab80 1 --2- 172.21.15.12:0/517399735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d26861e0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.804+0000 7f787867ab80 1 --2- 172.21.15.12:0/517399735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2685e00 0x55f6d2695c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.804+0000 7f787867ab80 1 -- 172.21.15.12:0/517399735 >> 172.21.15.12:0/517399735 conn(0x55f6d25de210 msgr2=0x55f6d2677210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.804+0000 7f787867ab80 1 -- 172.21.15.12:0/517399735 shutdown_connections 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.804+0000 7f787867ab80 1 -- 172.21.15.12:0/517399735 wait complete. 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f787867ab80 1 -- start start 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f787867ab80 1 Processor -- start 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f787867ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d268f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f787867ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2685e00 0x55f6d268f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f787867ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d268fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f787867ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d2691a60 con 0x55f6d2680890 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f787867ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d26ef6e0 con 0x55f6d2685e00 2024-05-09T00:05:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f787867ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d26ef820 con 0x55f6d2686720 2024-05-09T00:05:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f787867ab80 10 client.0 _ll_register_callbacks cb 0x55f6d2575b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f786a25c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d268f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f786a25c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d268f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58988/0 (socket says 172.21.15.12:58988) 2024-05-09T00:05:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f786a25c700 1 -- 172.21.15.12:0/1414833761 learned_addr learned my addr 172.21.15.12:0/1414833761 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f786aa5d700 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d268fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f7869a5b700 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2685e00 0x55f6d268f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f786a25c700 1 -- 172.21.15.12:0/1414833761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 msgr2=0x55f6d268fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f786a25c700 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d268fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f786a25c700 1 -- 172.21.15.12:0/1414833761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2685e00 msgr2=0x55f6d268f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f786a25c700 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2685e00 0x55f6d268f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.805+0000 7f786a25c700 1 -- 172.21.15.12:0/1414833761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c009c40 con 0x55f6d2680890 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.806+0000 7f786a25c700 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d268f150 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f786001cfc0 tx=0x7f7860011700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f73df2662af968b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.806+0000 7f784dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.806+0000 7f784dffb700 1 -- 172.21.15.12:0/1414833761 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860022070 con 0x55f6d2680890 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.806+0000 7f784dffb700 1 -- 172.21.15.12:0/1414833761 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7860004020 con 0x55f6d2680890 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.806+0000 7f787867ab80 1 -- 172.21.15.12:0/1414833761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7854011d80 con 0x55f6d2680890 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.806+0000 7f784dffb700 1 -- 172.21.15.12:0/1414833761 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7860005a00 con 0x55f6d2680890 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.806+0000 7f787867ab80 10 client.20781 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:45.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.806+0000 7f787867ab80 1 -- 172.21.15.12:0/1414833761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55f6d23fdb60 con 0x55f6d2680890 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.806+0000 7f784dffb700 1 -- 172.21.15.12:0/1414833761 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7854011d80 con 0x55f6d2680890 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.806+0000 7f784dffb700 1 -- 172.21.15.12:0/1414833761 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f786001d410 con 0x55f6d2680890 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.807+0000 7f787867ab80 10 client.20781 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.807+0000 7f787867ab80 10 client.20781 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.807+0000 7f787867ab80 20 client.20781 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.807+0000 7f787867ab80 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f6d26f0c90 0x55f6d26f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.807+0000 7f787867ab80 4 client.20781 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.807+0000 7f787867ab80 1 -- 172.21.15.12:0/1414833761 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f6d25c8ec0 con 0x55f6d26f0c90 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.807+0000 7f7869a5b700 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f6d26f0c90 0x55f6d26f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.807+0000 7f7869a5b700 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f6d26f0c90 0x55f6d26f30b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=763b78d29c07efd server_cookie=8132f3926b606e45 in_seq=0 out_seq=0 2024-05-09T00:05:45.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.807+0000 7f784dffb700 10 client.20781 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:05:45.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f784dffb700 1 -- 172.21.15.12:0/1414833761 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f6d25c8ec0 con 0x55f6d26f0c90 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f784dffb700 10 client.20781 handle_command_reply: tid=0 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 client.20781 shutdown 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 -- 172.21.15.12:0/1414833761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 msgr2=0x55f6d268f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d268f150 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f786001cfc0 tx=0x7f7860011700 comp rx=0 tx=0).stop 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 -- 172.21.15.12:0/1414833761 shutdown_connections 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55f6d26f0c90 0x55f6d26f30b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6d2686720 0x55f6d268fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6d2680890 0x55f6d268f150 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 --2- 172.21.15.12:0/1414833761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6d2685e00 0x55f6d268f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 -- 172.21.15.12:0/1414833761 >> 172.21.15.12:0/1414833761 conn(0x55f6d25de210 msgr2=0x55f6d2688310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f784dffb700 0 client.20781 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:05:45.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 -- 172.21.15.12:0/1414833761 shutdown_connections 2024-05-09T00:05:45.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.809+0000 7f787867ab80 1 -- 172.21.15.12:0/1414833761 wait complete. 2024-05-09T00:05:45.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f787867ab80 20 client.20781 trim_cache size 0 max 16384 2024-05-09T00:05:45.810 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:05:45.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 -- 172.21.15.12:0/1681835843 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78500a1310 msgr2=0x7f78500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78500a1310 0x7f78500a37d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7860005d10 tx=0x7f7860009310 comp rx=0 tx=0).stop 2024-05-09T00:05:45.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 -- 172.21.15.12:0/1681835843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786410db90 msgr2=0x7f7864199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:45.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786410db90 0x7f7864199760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f785c00b6c0 tx=0x7f785c0094b0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 -- 172.21.15.12:0/1681835843 shutdown_connections 2024-05-09T00:05:45.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f78500a1310 0x7f78500a37d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786410db90 0x7f7864199760 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78640ff950 0x7f786419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 --2- 172.21.15.12:0/1681835843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7864102010 0x7f786419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:45.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 -- 172.21.15.12:0/1681835843 >> 172.21.15.12:0/1681835843 conn(0x7f78640763a0 msgr2=0x7f786410f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:45.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 -- 172.21.15.12:0/1681835843 shutdown_connections 2024-05-09T00:05:45.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:45.810+0000 7f786c4c0700 1 -- 172.21.15.12:0/1681835843 wait complete. 2024-05-09T00:05:45.820 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:05:45.854 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:45 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1050287215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:05:46.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:45 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1050287215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:05:46.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:45 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1050287215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:05:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.048+0000 7fb305ae8700 1 -- 172.21.15.12:0/1323974111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb30007fc20 msgr2=0x7fb3000800a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.048+0000 7fb305ae8700 1 --2- 172.21.15.12:0/1323974111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb30007fc20 0x7fb3000800a0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009990 tx=0x7fb2f4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.054+0000 7fb305ae8700 1 -- 172.21.15.12:0/1323974111 shutdown_connections 2024-05-09T00:05:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.054+0000 7fb305ae8700 1 --2- 172.21.15.12:0/1323974111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb300079850 0x7fb300079ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.054+0000 7fb305ae8700 1 --2- 172.21.15.12:0/1323974111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300081940 0x7fb30007f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.054+0000 7fb305ae8700 1 --2- 172.21.15.12:0/1323974111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb30007fc20 0x7fb3000800a0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.054+0000 7fb305ae8700 1 -- 172.21.15.12:0/1323974111 >> 172.21.15.12:0/1323974111 conn(0x7fb300076dc0 msgr2=0x7fb300105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.066+0000 7fb305ae8700 1 -- 172.21.15.12:0/1323974111 shutdown_connections 2024-05-09T00:05:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.066+0000 7fb305ae8700 1 -- 172.21.15.12:0/1323974111 wait complete. 2024-05-09T00:05:46.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.067+0000 7fb305ae8700 1 Processor -- start 2024-05-09T00:05:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.067+0000 7fb305ae8700 1 -- start start 2024-05-09T00:05:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.068+0000 7fb305ae8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300079850 0x7fb300072890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.068+0000 7fb305ae8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30007fc20 0x7fb30006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.068+0000 7fb305ae8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb300081940 0x7fb30006de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.068+0000 7fb305ae8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30006e480 con 0x7fb300081940 2024-05-09T00:05:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.068+0000 7fb305ae8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30006e5f0 con 0x7fb300079850 2024-05-09T00:05:46.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.068+0000 7fb305ae8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30006e7b0 con 0x7fb30007fc20 2024-05-09T00:05:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.072+0000 7fb2fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb300081940 0x7fb30006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.072+0000 7fb2fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb300081940 0x7fb30006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58998/0 (socket says 172.21.15.12:58998) 2024-05-09T00:05:46.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.072+0000 7fb2fffff700 1 -- 172.21.15.12:0/3851151208 learned_addr learned my addr 172.21.15.12:0/3851151208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.077+0000 7fb2feffd700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30007fc20 0x7fb30006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.079+0000 7fb2fffff700 1 -- 172.21.15.12:0/3851151208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30007fc20 msgr2=0x7fb30006d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.079+0000 7fb2fffff700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30007fc20 0x7fb30006d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.079+0000 7fb2fffff700 1 -- 172.21.15.12:0/3851151208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300079850 msgr2=0x7fb300072890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.079+0000 7fb2fffff700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300079850 0x7fb300072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.079+0000 7fb2fffff700 1 -- 172.21.15.12:0/3851151208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f40095f0 con 0x7fb300081940 2024-05-09T00:05:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.079+0000 7fb2fffff700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb300081940 0x7fb30006de80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b4a0 tx=0x7fb2f000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.082+0000 7fb2fcff9700 1 -- 172.21.15.12:0/3851151208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0011830 con 0x7fb300081940 2024-05-09T00:05:46.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.082+0000 7fb2fcff9700 1 -- 172.21.15.12:0/3851151208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f0011e50 con 0x7fb300081940 2024-05-09T00:05:46.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.082+0000 7fb2fcff9700 1 -- 172.21.15.12:0/3851151208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000f2e0 con 0x7fb300081940 2024-05-09T00:05:46.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.082+0000 7fb305ae8700 1 -- 172.21.15.12:0/3851151208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3001bb3a0 con 0x7fb300081940 2024-05-09T00:05:46.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.082+0000 7fb305ae8700 1 -- 172.21.15.12:0/3851151208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3001bb830 con 0x7fb300081940 2024-05-09T00:05:46.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.091+0000 7fb2fcff9700 1 -- 172.21.15.12:0/3851151208 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb2f000f440 con 0x7fb300081940 2024-05-09T00:05:46.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.093+0000 7fb2fcff9700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2e80a14a0 0x7fb2e80a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.093+0000 7fb2fcff9700 1 -- 172.21.15.12:0/3851151208 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb2f00ccab0 con 0x7fb300081940 2024-05-09T00:05:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.098+0000 7fb311ca2b80 1 Processor -- start 2024-05-09T00:05:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.098+0000 7fb311ca2b80 1 -- start start 2024-05-09T00:05:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.098+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce3e00 0x560785ce41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.098+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce4720 0x560785cde350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.098+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785cde890 0x560785cdecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.098+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785c35150 con 0x560785cde890 2024-05-09T00:05:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.098+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785c32b10 con 0x560785ce4720 2024-05-09T00:05:46.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.098+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785c2e400 con 0x560785ce3e00 2024-05-09T00:05:46.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.111+0000 7fb2ff7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce3e00 0x560785ce41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.111+0000 7fb2ff7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce3e00 0x560785ce41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45204/0 (socket says 172.21.15.12:45204) 2024-05-09T00:05:46.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.111+0000 7fb2ff7fe700 1 -- 172.21.15.12:0/1791779108 learned_addr learned my addr 172.21.15.12:0/1791779108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.111+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2e80a14a0 0x7fb2e80a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.111+0000 7fb2feffd700 1 --2- 172.21.15.12:0/1791779108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce4720 0x560785cde350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.112+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2e80a14a0 0x7fb2e80a3960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8013470 tx=0x7fb2f80097a0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.116+0000 7fb2ff7fe700 1 -- 172.21.15.12:0/1791779108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce4720 msgr2=0x560785cde350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.116+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/1791779108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce4720 0x560785cde350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.116+0000 7fb2ff7fe700 1 -- 172.21.15.12:0/1791779108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785cde890 msgr2=0x560785cdecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.116+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/1791779108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785cde890 0x560785cdecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.116+0000 7fb2ff7fe700 1 -- 172.21.15.12:0/1791779108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560785aaf3b0 con 0x560785ce3e00 2024-05-09T00:05:46.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.120+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/1791779108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce3e00 0x560785ce41e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009f00 tx=0x7fb2f8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b3d8498daa50a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.129+0000 7fb2edffb700 1 -- 172.21.15.12:0/1791779108 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8016d20 con 0x560785ce3e00 2024-05-09T00:05:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.129+0000 7fb2edffb700 1 -- 172.21.15.12:0/1791779108 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f801a070 con 0x560785ce3e00 2024-05-09T00:05:46.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.129+0000 7fb2edffb700 1 -- 172.21.15.12:0/1791779108 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8004720 con 0x560785ce3e00 2024-05-09T00:05:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.130+0000 7fb311ca2b80 1 -- 172.21.15.12:0/1791779108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce3e00 msgr2=0x560785ce41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.130+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/1791779108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce3e00 0x560785ce41e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009f00 tx=0x7fb2f8019000 comp rx=0 tx=0).stop 2024-05-09T00:05:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.130+0000 7fb311ca2b80 1 -- 172.21.15.12:0/1791779108 shutdown_connections 2024-05-09T00:05:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.130+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/1791779108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785cde890 0x560785cdecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.130+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/1791779108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce3e00 0x560785ce41e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.130+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/1791779108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce4720 0x560785cde350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.130+0000 7fb311ca2b80 1 -- 172.21.15.12:0/1791779108 >> 172.21.15.12:0/1791779108 conn(0x560785c3c210 msgr2=0x560785cd8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.139+0000 7fb311ca2b80 1 -- 172.21.15.12:0/1791779108 shutdown_connections 2024-05-09T00:05:46.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.140+0000 7fb311ca2b80 1 -- 172.21.15.12:0/1791779108 wait complete. 2024-05-09T00:05:46.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.140+0000 7fb311ca2b80 1 -- start start 2024-05-09T00:05:46.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb311ca2b80 1 Processor -- start 2024-05-09T00:05:46.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 0x560785ce7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce3e00 0x560785ce80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce4720 0x560785ce85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785bca560 con 0x560785ce3e00 2024-05-09T00:05:46.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785bca200 con 0x560785ce4720 2024-05-09T00:05:46.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785ab16a0 con 0x560785cde890 2024-05-09T00:05:46.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb311ca2b80 10 client.0 _ll_register_callbacks cb 0x560785bd3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:46.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb2feffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce3e00 0x560785ce80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb2feffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce3e00 0x560785ce80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59006/0 (socket says 172.21.15.12:59006) 2024-05-09T00:05:46.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.141+0000 7fb2feffd700 1 -- 172.21.15.12:0/765505599 learned_addr learned my addr 172.21.15.12:0/765505599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.142+0000 7fb2feffd700 1 -- 172.21.15.12:0/765505599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 msgr2=0x560785ce7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.142+0000 7fb2feffd700 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 0x560785ce7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.142+0000 7fb2feffd700 1 -- 172.21.15.12:0/765505599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce4720 msgr2=0x560785ce85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.142+0000 7fb2feffd700 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce4720 0x560785ce85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.142+0000 7fb2feffd700 1 -- 172.21.15.12:0/765505599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f800e070 con 0x560785ce3e00 2024-05-09T00:05:46.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.142+0000 7fb2feffd700 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce3e00 0x560785ce80a0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4015000 tx=0x7fb2f40037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15741b7c9dd9ed2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.142+0000 7fb2e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:46.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.143+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/765505599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f401f070 con 0x560785ce3e00 2024-05-09T00:05:46.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.143+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/765505599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb2f4003b50 con 0x560785ce3e00 2024-05-09T00:05:46.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.143+0000 7fb311ca2b80 1 -- 172.21.15.12:0/765505599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560785b87270 con 0x560785ce3e00 2024-05-09T00:05:46.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.143+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/765505599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2f4003d10 con 0x560785ce3e00 2024-05-09T00:05:46.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.143+0000 7fb311ca2b80 10 client.20790 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:46.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.143+0000 7fb311ca2b80 1 -- 172.21.15.12:0/765505599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x560785a5bb60 con 0x560785ce3e00 2024-05-09T00:05:46.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.143+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/765505599 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560785b87270 con 0x560785ce3e00 2024-05-09T00:05:46.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.144+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/765505599 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb2f402b070 con 0x560785ce3e00 2024-05-09T00:05:46.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.144+0000 7fb311ca2b80 10 client.20790 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:46.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.144+0000 7fb311ca2b80 10 client.20790 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:05:46.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.144+0000 7fb311ca2b80 20 client.20790 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:46.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.144+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560785cf83e0 0x560785cfa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.144+0000 7fb311ca2b80 4 client.20790 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:46.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.144+0000 7fb311ca2b80 1 -- 172.21.15.12:0/765505599 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560785cad810 con 0x560785cf83e0 2024-05-09T00:05:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.156+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560785cf83e0 0x560785cfa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.164+0000 7fb2e2ffd700 10 client.20790 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:05:46.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.164+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560785cf83e0 0x560785cfa800 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d6af5b33ca64037 server_cookie=2c9b932b01483c7c in_seq=0 out_seq=0 2024-05-09T00:05:46.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.167+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/765505599 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560785cad810 con 0x560785cf83e0 2024-05-09T00:05:46.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.167+0000 7fb2e2ffd700 10 client.20790 handle_command_reply: tid=0 2024-05-09T00:05:46.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.167+0000 7fb311ca2b80 1 client.20790 shutdown 2024-05-09T00:05:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.169+0000 7f346db63700 1 -- 172.21.15.12:0/1975182768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34680ff410 msgr2=0x7f34680ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.169+0000 7f346db63700 1 --2- 172.21.15.12:0/1975182768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34680ff410 0x7f34680ff890 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3450009990 tx=0x7f3450009ca0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.172+0000 7f346db63700 1 -- 172.21.15.12:0/1975182768 shutdown_connections 2024-05-09T00:05:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.172+0000 7f346db63700 1 --2- 172.21.15.12:0/1975182768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34680ffdd0 0x7f34681038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.172+0000 7f346db63700 1 --2- 172.21.15.12:0/1975182768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3468105dd0 0x7f34681061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.172+0000 7f346db63700 1 --2- 172.21.15.12:0/1975182768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34680ff410 0x7f34680ff890 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.172+0000 7f346db63700 1 -- 172.21.15.12:0/1975182768 >> 172.21.15.12:0/1975182768 conn(0x7f34680763a0 msgr2=0x7f34680fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.171+0000 7fb311ca2b80 1 -- 172.21.15.12:0/765505599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce3e00 msgr2=0x560785ce80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.171+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce3e00 0x560785ce80a0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4015000 tx=0x7fb2f40037a0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.173+0000 7fb311ca2b80 1 -- 172.21.15.12:0/765505599 shutdown_connections 2024-05-09T00:05:46.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.173+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560785cf83e0 0x560785cfa800 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.173+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce4720 0x560785ce85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.173+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 0x560785ce7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.173+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/765505599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce3e00 0x560785ce80a0 secure :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4015000 tx=0x7fb2f40037a0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.173+0000 7fb311ca2b80 1 -- 172.21.15.12:0/765505599 >> 172.21.15.12:0/765505599 conn(0x560785c3c210 msgr2=0x560785cd85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.173+0000 7fb2e2ffd700 0 client.20790 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:05:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.174+0000 7fb311ca2b80 1 -- 172.21.15.12:0/765505599 shutdown_connections 2024-05-09T00:05:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.174+0000 7fb311ca2b80 1 -- 172.21.15.12:0/765505599 wait complete. 2024-05-09T00:05:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.172+0000 7f346db63700 1 -- 172.21.15.12:0/1975182768 shutdown_connections 2024-05-09T00:05:46.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.174+0000 7fb311ca2b80 20 client.20790 trim_cache size 0 max 16384 2024-05-09T00:05:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.182+0000 7f346db63700 1 -- 172.21.15.12:0/1975182768 wait complete. 2024-05-09T00:05:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.183+0000 7f346db63700 1 Processor -- start 2024-05-09T00:05:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.185+0000 7f346db63700 1 -- start start 2024-05-09T00:05:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.185+0000 7f346db63700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34680ff410 0x7f346806d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.185+0000 7f346db63700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34680ffdd0 0x7f346806dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.185+0000 7f346db63700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3468105dd0 0x7f34681abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.185+0000 7f346db63700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34681ac3a0 con 0x7f34680ff410 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.185+0000 7f346db63700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34681ac510 con 0x7f34680ffdd0 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.185+0000 7f346db63700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34681ac6d0 con 0x7f3468105dd0 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.186+0000 7f34677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34680ff410 0x7f346806d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.186+0000 7f34677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34680ff410 0x7f346806d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59012/0 (socket says 172.21.15.12:59012) 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.186+0000 7f34677fe700 1 -- 172.21.15.12:0/2690076203 learned_addr learned my addr 172.21.15.12:0/2690076203 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.186+0000 7f34677fe700 1 -- 172.21.15.12:0/2690076203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3468105dd0 msgr2=0x7f34681abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.186+0000 7f34677fe700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3468105dd0 0x7f34681abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.186+0000 7f34677fe700 1 -- 172.21.15.12:0/2690076203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34680ffdd0 msgr2=0x7f346806dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.186+0000 7f34677fe700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34680ffdd0 0x7f346806dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.186+0000 7f34677fe700 1 -- 172.21.15.12:0/2690076203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500095f0 con 0x7f34680ff410 2024-05-09T00:05:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.186+0000 7f34677fe700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34680ff410 0x7f346806d730 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f345800b530 tx=0x7f345800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.191+0000 7f3464ff9700 1 -- 172.21.15.12:0/2690076203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458011830 con 0x7f34680ff410 2024-05-09T00:05:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.191+0000 7f3464ff9700 1 -- 172.21.15.12:0/2690076203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3458011e50 con 0x7f34680ff410 2024-05-09T00:05:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.191+0000 7f346db63700 1 -- 172.21.15.12:0/2690076203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34681ac9b0 con 0x7f34680ff410 2024-05-09T00:05:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.191+0000 7f346db63700 1 -- 172.21.15.12:0/2690076203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34681b2fa0 con 0x7f34680ff410 2024-05-09T00:05:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.195+0000 7f3464ff9700 1 -- 172.21.15.12:0/2690076203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345800f2e0 con 0x7f34680ff410 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.197+0000 7f3479d1db80 1 Processor -- start 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.197+0000 7f3479d1db80 1 -- start start 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.197+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 0x55c06e6c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.197+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 0x55c06e6c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.197+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.197+0000 7f3479d1db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e5f90f0 con 0x55c06e6c9950 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.197+0000 7f3479d1db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e5f6ab0 con 0x55c06e6c3f50 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.197+0000 7f3479d1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e5f23a0 con 0x55c06e6c4910 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.198+0000 7f3464ff9700 1 -- 172.21.15.12:0/2690076203 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f345800f480 con 0x7f34680ff410 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.199+0000 7f3464ff9700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34540a1460 0x7f34540a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.199+0000 7f3464ff9700 1 -- 172.21.15.12:0/2690076203 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f34580ccad0 con 0x7f34680ff410 2024-05-09T00:05:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.199+0000 7f3466ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.199+0000 7f3467fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 0x55c06e6c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.199+0000 7f3467fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 0x55c06e6c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59022/0 (socket says 172.21.15.12:59022) 2024-05-09T00:05:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.199+0000 7f3467fff700 1 -- 172.21.15.12:0/3440281480 learned_addr learned my addr 172.21.15.12:0/3440281480 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.200+0000 7f34677fe700 1 --2- 172.21.15.12:0/3440281480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 0x55c06e6c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.200+0000 7f3467fff700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34540a1460 0x7f34540a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.207+0000 7f3466ffd700 1 -- 172.21.15.12:0/3440281480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 msgr2=0x55c06e6c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.207+0000 7f3466ffd700 1 --2- 172.21.15.12:0/3440281480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 0x55c06e6c43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.207+0000 7f3466ffd700 1 -- 172.21.15.12:0/3440281480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 msgr2=0x55c06e6c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.207+0000 7f3466ffd700 1 --2- 172.21.15.12:0/3440281480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 0x55c06e6c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.207+0000 7f3466ffd700 1 -- 172.21.15.12:0/3440281480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c06e473350 con 0x55c06e6c4910 2024-05-09T00:05:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.207+0000 7f34677fe700 1 --2- 172.21.15.12:0/3440281480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 0x55c06e6c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:05:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.207+0000 7f3466ffd700 1 --2- 172.21.15.12:0/3440281480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6d0700 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3450006010 tx=0x7f34500036c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=994ba1d103ad2233 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.208+0000 7f345dffb700 1 -- 172.21.15.12:0/3440281480 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450061070 con 0x55c06e6c4910 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.208+0000 7f345dffb700 1 -- 172.21.15.12:0/3440281480 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3450005da0 con 0x55c06e6c4910 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.208+0000 7f345dffb700 1 -- 172.21.15.12:0/3440281480 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450005320 con 0x55c06e6c4910 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.209+0000 7f3479d1db80 1 -- 172.21.15.12:0/3440281480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 msgr2=0x55c06e6d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.209+0000 7f3479d1db80 1 --2- 172.21.15.12:0/3440281480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6d0700 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3450006010 tx=0x7f34500036c0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.209+0000 7f3467fff700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34540a1460 0x7f34540a3920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3460009fd0 tx=0x7f3460004c50 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.209+0000 7f3479d1db80 1 -- 172.21.15.12:0/3440281480 shutdown_connections 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.209+0000 7f3479d1db80 1 --2- 172.21.15.12:0/3440281480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6d0700 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.209+0000 7f3479d1db80 1 --2- 172.21.15.12:0/3440281480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 0x55c06e6c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.209+0000 7f3479d1db80 1 --2- 172.21.15.12:0/3440281480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 0x55c06e6c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.209+0000 7f3479d1db80 1 -- 172.21.15.12:0/3440281480 >> 172.21.15.12:0/3440281480 conn(0x55c06e6001b0 msgr2=0x55c06e69c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.211+0000 7f3479d1db80 1 -- 172.21.15.12:0/3440281480 shutdown_connections 2024-05-09T00:05:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.211+0000 7fb311ca2b80 1 Processor -- start 2024-05-09T00:05:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.211+0000 7f3479d1db80 1 -- 172.21.15.12:0/3440281480 wait complete. 2024-05-09T00:05:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.211+0000 7f3479d1db80 1 -- start start 2024-05-09T00:05:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.212+0000 7f3479d1db80 1 Processor -- start 2024-05-09T00:05:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.211+0000 7fb311ca2b80 1 -- start start 2024-05-09T00:05:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.212+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 0x560785cde350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.212+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce4720 0x560785cd8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.212+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce23d0 0x560785ce6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.212+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785ce91d0 con 0x560785ce4720 2024-05-09T00:05:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.212+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785cd3800 con 0x560785ce23d0 2024-05-09T00:05:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.212+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785cff9f0 con 0x560785cde890 2024-05-09T00:05:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.212+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c3f50 0x55c06e6a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.212+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c4910 0x55c06e6aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.212+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c9950 0x55c06e6a5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.212+0000 7f3479d1db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e58e500 con 0x55c06e6c4910 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.212+0000 7f3479d1db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e58e1a0 con 0x55c06e6c9950 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.212+0000 7f3479d1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e475640 con 0x55c06e6c3f50 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.212+0000 7f3479d1db80 10 client.0 _ll_register_callbacks cb 0x55c06e597ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.212+0000 7f34677fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c3f50 0x55c06e6a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.213+0000 7f3466ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c4910 0x55c06e6aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.213+0000 7f3466ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c4910 0x55c06e6aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59030/0 (socket says 172.21.15.12:59030) 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.213+0000 7f3466ffd700 1 -- 172.21.15.12:0/3871753960 learned_addr learned my addr 172.21.15.12:0/3871753960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.213+0000 7f3466ffd700 1 -- 172.21.15.12:0/3871753960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c3f50 msgr2=0x55c06e6a9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.213+0000 7f3466ffd700 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c3f50 0x55c06e6a9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.213+0000 7f3466ffd700 1 -- 172.21.15.12:0/3871753960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c9950 msgr2=0x55c06e6a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.213+0000 7f3466ffd700 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c9950 0x55c06e6a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.213+0000 7f3466ffd700 1 -- 172.21.15.12:0/3871753960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3468104190 con 0x55c06e6c4910 2024-05-09T00:05:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.213+0000 7f3466ffd700 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c4910 0x55c06e6aa4a0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f34500098e0 tx=0x7f345000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57123f4181dabf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.213+0000 7f3446ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.213+0000 7fb2ff7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 0x560785cde350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.213+0000 7fb2ff7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 0x560785cde350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45264/0 (socket says 172.21.15.12:45264) 2024-05-09T00:05:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.213+0000 7fb2ff7fe700 1 -- 172.21.15.12:0/1016634122 learned_addr learned my addr 172.21.15.12:0/1016634122 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2feffd700 1 --2- 172.21.15.12:0/1016634122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce4720 0x560785cd8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2ff7fe700 1 -- 172.21.15.12:0/1016634122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce23d0 msgr2=0x560785ce6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/1016634122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce23d0 0x560785ce6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2ff7fe700 1 -- 172.21.15.12:0/1016634122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce4720 msgr2=0x560785cd8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/1016634122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce4720 0x560785cd8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2ff7fe700 1 -- 172.21.15.12:0/1016634122 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560785bf4560 con 0x560785cde890 2024-05-09T00:05:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/1016634122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 0x560785cde350 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009f00 tx=0x7fb2f8004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=497435a8b30e0e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.214+0000 7f3446ffd700 1 -- 172.21.15.12:0/3871753960 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450061070 con 0x55c06e6c4910 2024-05-09T00:05:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.214+0000 7f3446ffd700 1 -- 172.21.15.12:0/3871753960 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3450005250 con 0x55c06e6c4910 2024-05-09T00:05:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.214+0000 7f3479d1db80 1 -- 172.21.15.12:0/3871753960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c06e54b210 con 0x55c06e6c4910 2024-05-09T00:05:46.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2feffd700 1 --2- 172.21.15.12:0/1016634122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce4720 0x560785cd8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:05:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.214+0000 7f3446ffd700 1 -- 172.21.15.12:0/3871753960 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f345000bb10 con 0x55c06e6c4910 2024-05-09T00:05:46.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2ee7fc700 1 -- 172.21.15.12:0/1016634122 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8016b20 con 0x560785cde890 2024-05-09T00:05:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2ee7fc700 1 -- 172.21.15.12:0/1016634122 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f8003ec0 con 0x560785cde890 2024-05-09T00:05:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.214+0000 7fb2ee7fc700 1 -- 172.21.15.12:0/1016634122 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f802e150 con 0x560785cde890 2024-05-09T00:05:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.215+0000 7f3479d1db80 10 client.20799 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.215+0000 7f3479d1db80 1 -- 172.21.15.12:0/3871753960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55c06e41fb00 con 0x55c06e6c4910 2024-05-09T00:05:46.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.216+0000 7fb311ca2b80 1 -- 172.21.15.12:0/1016634122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 msgr2=0x560785cde350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.216+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/1016634122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 0x560785cde350 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009f00 tx=0x7fb2f8004260 comp rx=0 tx=0).stop 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.216+0000 7fb311ca2b80 1 -- 172.21.15.12:0/1016634122 shutdown_connections 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.216+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/1016634122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785ce23d0 0x560785ce6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.216+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/1016634122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785cde890 0x560785cde350 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.216+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/1016634122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce4720 0x560785cd8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.216+0000 7fb311ca2b80 1 -- 172.21.15.12:0/1016634122 >> 172.21.15.12:0/1016634122 conn(0x560785c3c210 msgr2=0x560785ce1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.216+0000 7fb311ca2b80 1 -- 172.21.15.12:0/1016634122 shutdown_connections 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.217+0000 7fb311ca2b80 1 -- 172.21.15.12:0/1016634122 wait complete. 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.217+0000 7fb311ca2b80 1 -- start start 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.217+0000 7fb311ca2b80 1 Processor -- start 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.218+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785cde890 0x560785cef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.218+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce23d0 0x560785ced810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.218+0000 7fb311ca2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce4720 0x560785cedd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.218+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785cee290 con 0x560785ce23d0 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.218+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785cee400 con 0x560785cde890 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.218+0000 7fb311ca2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560785cee5c0 con 0x560785ce4720 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.218+0000 7fb311ca2b80 10 client.0 _ll_register_callbacks cb 0x560785bd3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.218+0000 7fb2fffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce4720 0x560785cedd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.218+0000 7fb2fffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce4720 0x560785cedd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45266/0 (socket says 172.21.15.12:45266) 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.218+0000 7fb2fffff700 1 -- 172.21.15.12:0/2092067595 learned_addr learned my addr 172.21.15.12:0/2092067595 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.219+0000 7fb2fffff700 1 -- 172.21.15.12:0/2092067595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785cde890 msgr2=0x560785cef370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.219+0000 7fb2feffd700 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce23d0 0x560785ced810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.220+0000 7fb2fffff700 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785cde890 0x560785cef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.220+0000 7fb2fffff700 1 -- 172.21.15.12:0/2092067595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce23d0 msgr2=0x560785ced810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.220+0000 7fb2fffff700 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce23d0 0x560785ced810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.220+0000 7fb2fffff700 1 -- 172.21.15.12:0/2092067595 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560785aaf3b0 con 0x560785ce4720 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.220+0000 7fb2feffd700 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce23d0 0x560785ced810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:05:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.220+0000 7fb2fffff700 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce4720 0x560785cedd50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000bfd0 tx=0x7fb2f00995e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ef3f873aac73dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.220+0000 7fb2e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:05:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.220+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/2092067595 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f009a0e0 con 0x560785ce4720 2024-05-09T00:05:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.220+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/2092067595 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb2f0018520 con 0x560785ce4720 2024-05-09T00:05:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.221+0000 7fb311ca2b80 1 -- 172.21.15.12:0/2092067595 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2f801a070 con 0x560785ce4720 2024-05-09T00:05:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.221+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/2092067595 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2f009dab0 con 0x560785ce4720 2024-05-09T00:05:46.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.221+0000 7fb311ca2b80 10 client.30496 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:46.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.221+0000 7fb311ca2b80 1 -- 172.21.15.12:0/2092067595 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x560785ab3cd0 con 0x560785ce4720 2024-05-09T00:05:46.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.222+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/2092067595 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f801a070 con 0x560785ce4720 2024-05-09T00:05:46.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.222+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/2092067595 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb2f0095a70 con 0x560785ce4720 2024-05-09T00:05:46.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.223+0000 7fb311ca2b80 10 client.30496 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:46.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.223+0000 7fb311ca2b80 10 client.30496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:05:46.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.223+0000 7fb311ca2b80 20 client.30496 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:46.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.223+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560785d4fe70 0x560785d52290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.223+0000 7fb311ca2b80 4 client.30496 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:05:46.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.223+0000 7fb311ca2b80 1 -- 172.21.15.12:0/2092067595 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560785c26ec0 con 0x560785d4fe70 2024-05-09T00:05:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.229+0000 7f3446ffd700 1 -- 172.21.15.12:0/3871753960 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c06e54b210 con 0x55c06e6c4910 2024-05-09T00:05:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.229+0000 7f3446ffd700 1 -- 172.21.15.12:0/3871753960 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3450076070 con 0x55c06e6c4910 2024-05-09T00:05:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.229+0000 7f3479d1db80 10 client.20799 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.229+0000 7f3479d1db80 10 client.20799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.229+0000 7f3479d1db80 20 client.20799 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.229+0000 7f3479d1db80 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c06e6b4fe0 0x55c06e6b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.229+0000 7f3479d1db80 4 client.20799 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.229+0000 7f3479d1db80 1 -- 172.21.15.12:0/3871753960 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c06e6717b0 con 0x55c06e6b4fe0 2024-05-09T00:05:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.230+0000 7f34677fe700 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c06e6b4fe0 0x55c06e6b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.231+0000 7f34677fe700 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c06e6b4fe0 0x55c06e6b7400 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f061ade9783e8ab4 server_cookie=626211c51f729f1b in_seq=0 out_seq=0 2024-05-09T00:05:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.231+0000 7f3446ffd700 10 client.20799 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.228+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560785d4fe70 0x560785d52290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.230+0000 7fb2ff7fe700 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560785d4fe70 0x560785d52290 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bf21b1b77db46c4 server_cookie=2cea01553a5392f0 in_seq=0 out_seq=0 2024-05-09T00:05:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.231+0000 7fb2e2ffd700 10 client.30496 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:05:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.237+0000 7f3446ffd700 1 -- 172.21.15.12:0/3871753960 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c06e6717b0 con 0x55c06e6b4fe0 2024-05-09T00:05:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.237+0000 7f3446ffd700 10 client.20799 handle_command_reply: tid=0 2024-05-09T00:05:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.238+0000 7f3479d1db80 1 client.20799 shutdown 2024-05-09T00:05:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.239+0000 7f3479d1db80 1 -- 172.21.15.12:0/3871753960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c4910 msgr2=0x55c06e6aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.239+0000 7f3479d1db80 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c4910 0x55c06e6aa4a0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f34500098e0 tx=0x7f345000bf40 comp rx=0 tx=0).stop 2024-05-09T00:05:46.239 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.241+0000 7fb2e2ffd700 1 -- 172.21.15.12:0/2092067595 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560785c26ec0 con 0x560785d4fe70 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.241+0000 7fb2e2ffd700 10 client.30496 handle_command_reply: tid=0 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.241+0000 7fb311ca2b80 1 client.30496 shutdown 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 1 -- 172.21.15.12:0/2092067595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce4720 msgr2=0x560785cedd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce4720 0x560785cedd50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000bfd0 tx=0x7fb2f00995e0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 1 -- 172.21.15.12:0/2092067595 shutdown_connections 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x560785d4fe70 0x560785d52290 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560785ce4720 0x560785cedd50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560785cde890 0x560785cef370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 1 --2- 172.21.15.12:0/2092067595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560785ce23d0 0x560785ced810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 1 -- 172.21.15.12:0/2092067595 >> 172.21.15.12:0/2092067595 conn(0x560785c3c210 msgr2=0x560785ce62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb2e2ffd700 0 client.30496 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 1 -- 172.21.15.12:0/2092067595 shutdown_connections 2024-05-09T00:05:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 1 -- 172.21.15.12:0/2092067595 wait complete. 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.242+0000 7fb311ca2b80 20 client.30496 trim_cache size 0 max 16384 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.243+0000 7fb305ae8700 1 -- 172.21.15.12:0/3851151208 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2e80a14a0 msgr2=0x7fb2e80a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.243+0000 7fb305ae8700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2e80a14a0 0x7fb2e80a3960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8013470 tx=0x7fb2f80097a0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.243+0000 7fb305ae8700 1 -- 172.21.15.12:0/3851151208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb300081940 msgr2=0x7fb30006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.243+0000 7fb305ae8700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb300081940 0x7fb30006de80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b4a0 tx=0x7fb2f000b860 comp rx=0 tx=0).stop 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.244+0000 7fb305ae8700 1 -- 172.21.15.12:0/3851151208 shutdown_connections 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.244+0000 7fb305ae8700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2e80a14a0 0x7fb2e80a3960 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.244+0000 7fb305ae8700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb300081940 0x7fb30006de80 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.244+0000 7fb305ae8700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300079850 0x7fb300072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.244+0000 7fb305ae8700 1 --2- 172.21.15.12:0/3851151208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30007fc20 0x7fb30006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.244+0000 7fb305ae8700 1 -- 172.21.15.12:0/3851151208 >> 172.21.15.12:0/3851151208 conn(0x7fb300076dc0 msgr2=0x7fb30007d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.244+0000 7fb305ae8700 1 -- 172.21.15.12:0/3851151208 shutdown_connections 2024-05-09T00:05:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.244+0000 7fb305ae8700 1 -- 172.21.15.12:0/3851151208 wait complete. 2024-05-09T00:05:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.240+0000 7f3479d1db80 1 -- 172.21.15.12:0/3871753960 shutdown_connections 2024-05-09T00:05:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.240+0000 7f3479d1db80 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c06e6b4fe0 0x55c06e6b7400 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.240+0000 7f3479d1db80 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c9950 0x55c06e6a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.240+0000 7f3479d1db80 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c3f50 0x55c06e6a9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.240+0000 7f3479d1db80 1 --2- 172.21.15.12:0/3871753960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c4910 0x55c06e6aa4a0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.240+0000 7f3479d1db80 1 -- 172.21.15.12:0/3871753960 >> 172.21.15.12:0/3871753960 conn(0x55c06e6001b0 msgr2=0x55c06e69c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.240+0000 7f3446ffd700 0 client.20799 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.245+0000 7f3479d1db80 1 -- 172.21.15.12:0/3871753960 shutdown_connections 2024-05-09T00:05:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.246+0000 7f3479d1db80 1 -- 172.21.15.12:0/3871753960 wait complete. 2024-05-09T00:05:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.250+0000 7f3479d1db80 20 client.20799 trim_cache size 0 max 16384 2024-05-09T00:05:46.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 228390 34.47 MB/sec execute 221 sec latency 268.420 ms 2024-05-09T00:05:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.282+0000 7f3479d1db80 1 Processor -- start 2024-05-09T00:05:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.282+0000 7f3479d1db80 1 -- start start 2024-05-09T00:05:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c3f50 0x55c06e6c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c4910 0x55c06e6a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c9950 0x55c06e6a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f3479d1db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e69e340 con 0x55c06e6c3f50 2024-05-09T00:05:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f3479d1db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e69d930 con 0x55c06e6c4910 2024-05-09T00:05:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f3479d1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6bc5d0 con 0x55c06e6c9950 2024-05-09T00:05:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f34677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c3f50 0x55c06e6c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f34677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c3f50 0x55c06e6c3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59070/0 (socket says 172.21.15.12:59070) 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f34677fe700 1 -- 172.21.15.12:0/2826111332 learned_addr learned my addr 172.21.15.12:0/2826111332 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f3466ffd700 1 --2- 172.21.15.12:0/2826111332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c4910 0x55c06e6a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f3467fff700 1 --2- 172.21.15.12:0/2826111332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c9950 0x55c06e6a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f34677fe700 1 -- 172.21.15.12:0/2826111332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c9950 msgr2=0x55c06e6a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f34677fe700 1 --2- 172.21.15.12:0/2826111332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c9950 0x55c06e6a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f34677fe700 1 -- 172.21.15.12:0/2826111332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c4910 msgr2=0x55c06e6a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f34677fe700 1 --2- 172.21.15.12:0/2826111332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c4910 0x55c06e6a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f34677fe700 1 -- 172.21.15.12:0/2826111332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c06e5b8500 con 0x55c06e6c3f50 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.283+0000 7f34677fe700 1 --2- 172.21.15.12:0/2826111332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c3f50 0x55c06e6c3960 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f34580969f0 tx=0x7f34580ccce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a9a62f7d1816f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.284+0000 7f345e7fc700 1 -- 172.21.15.12:0/2826111332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580a1040 con 0x55c06e6c3f50 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.284+0000 7f345e7fc700 1 -- 172.21.15.12:0/2826111332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34580960c0 con 0x55c06e6c3f50 2024-05-09T00:05:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.284+0000 7f345e7fc700 1 -- 172.21.15.12:0/2826111332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458099690 con 0x55c06e6c3f50 2024-05-09T00:05:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.284+0000 7f3479d1db80 1 -- 172.21.15.12:0/2826111332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c3f50 msgr2=0x55c06e6c3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.284+0000 7f3479d1db80 1 --2- 172.21.15.12:0/2826111332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c3f50 0x55c06e6c3960 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f34580969f0 tx=0x7f34580ccce0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.285+0000 7f3479d1db80 1 -- 172.21.15.12:0/2826111332 shutdown_connections 2024-05-09T00:05:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.285+0000 7f3479d1db80 1 --2- 172.21.15.12:0/2826111332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c9950 0x55c06e6a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.285+0000 7f3479d1db80 1 --2- 172.21.15.12:0/2826111332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c3f50 0x55c06e6c3960 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.285+0000 7f3479d1db80 1 --2- 172.21.15.12:0/2826111332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c4910 0x55c06e6a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.285+0000 7f3479d1db80 1 -- 172.21.15.12:0/2826111332 >> 172.21.15.12:0/2826111332 conn(0x55c06e6001b0 msgr2=0x55c06e6c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.285+0000 7f3479d1db80 1 -- 172.21.15.12:0/2826111332 shutdown_connections 2024-05-09T00:05:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.285+0000 7f3479d1db80 1 -- 172.21.15.12:0/2826111332 wait complete. 2024-05-09T00:05:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.285+0000 7f3479d1db80 1 -- start start 2024-05-09T00:05:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.285+0000 7f3479d1db80 1 Processor -- start 2024-05-09T00:05:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 0x55c06e6aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6acb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3479d1db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 0x55c06e6ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3479d1db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6ad610 con 0x55c06e6c9950 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3479d1db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6ad780 con 0x55c06e6c3f50 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3479d1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6ad940 con 0x55c06e6c4910 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3479d1db80 10 client.0 _ll_register_callbacks cb 0x55c06e597ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3466ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3467fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 0x55c06e6ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f34677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 0x55c06e6aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3466ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6acb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45278/0 (socket says 172.21.15.12:45278) 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3466ffd700 1 -- 172.21.15.12:0/2664629258 learned_addr learned my addr 172.21.15.12:0/2664629258 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f34677fe700 1 -- 172.21.15.12:0/2664629258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 msgr2=0x55c06e6acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f34677fe700 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6acb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f34677fe700 1 -- 172.21.15.12:0/2664629258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 msgr2=0x55c06e6ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f34677fe700 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 0x55c06e6ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f34677fe700 1 -- 172.21.15.12:0/2664629258 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345809ae70 con 0x55c06e6c3f50 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3467fff700 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 0x55c06e6ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3466ffd700 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f34677fe700 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 0x55c06e6aeda0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3458010fd0 tx=0x7f3458009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2d1b8d7a273435 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.286+0000 7f3446ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3446ffd700 1 -- 172.21.15.12:0/2664629258 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580a1070 con 0x55c06e6c3f50 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3446ffd700 1 -- 172.21.15.12:0/2664629258 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3458099430 con 0x55c06e6c3f50 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3479d1db80 1 -- 172.21.15.12:0/2664629258 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3450005da0 con 0x55c06e6c3f50 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3446ffd700 1 -- 172.21.15.12:0/2664629258 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34580995f0 con 0x55c06e6c3f50 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3479d1db80 10 client.30344 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3479d1db80 1 -- 172.21.15.12:0/2664629258 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55c06e477c70 con 0x55c06e6c3f50 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3446ffd700 1 -- 172.21.15.12:0/2664629258 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34580ab070 con 0x55c06e6c3f50 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3446ffd700 1 -- 172.21.15.12:0/2664629258 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3458098ea0 con 0x55c06e6c3f50 2024-05-09T00:05:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3479d1db80 10 client.30344 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3479d1db80 10 client.30344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3479d1db80 20 client.30344 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3479d1db80 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c06e712910 0x55c06e714d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3479d1db80 4 client.30344 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:05:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.287+0000 7f3479d1db80 1 -- 172.21.15.12:0/2664629258 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c06e5eae60 con 0x55c06e712910 2024-05-09T00:05:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.288+0000 7f3466ffd700 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c06e712910 0x55c06e714d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.288+0000 7f3446ffd700 10 client.30344 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.288+0000 7f3466ffd700 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c06e712910 0x55c06e714d30 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aca318275c8d6d3 server_cookie=ea1c73d13b812fb0 in_seq=0 out_seq=0 2024-05-09T00:05:46.292 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:05:46.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.612+0000 7f7cfec58700 1 -- 172.21.15.12:0/1170952223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cf810fb10 msgr2=0x7f7cf81084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.612+0000 7f7cfec58700 1 --2- 172.21.15.12:0/1170952223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cf810fb10 0x7f7cf81084e0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009990 tx=0x7f7cec009ca0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.621+0000 7f7cfec58700 1 -- 172.21.15.12:0/1170952223 shutdown_connections 2024-05-09T00:05:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.622+0000 7f7cfec58700 1 --2- 172.21.15.12:0/1170952223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf811a780 0x7f7cf811cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.622+0000 7f7cfec58700 1 --2- 172.21.15.12:0/1170952223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf810f6a0 0x7f7cf8107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.622+0000 7f7cfec58700 1 --2- 172.21.15.12:0/1170952223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cf810fb10 0x7f7cf81084e0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.622+0000 7f7cfec58700 1 -- 172.21.15.12:0/1170952223 >> 172.21.15.12:0/1170952223 conn(0x7f7cf806bff0 msgr2=0x7f7cf806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.632+0000 7f7cfec58700 1 -- 172.21.15.12:0/1170952223 shutdown_connections 2024-05-09T00:05:46.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.636+0000 7f7cfec58700 1 -- 172.21.15.12:0/1170952223 wait complete. 2024-05-09T00:05:46.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.637+0000 7f7cfec58700 1 Processor -- start 2024-05-09T00:05:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.639+0000 7f7cfec58700 1 -- start start 2024-05-09T00:05:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.639+0000 7f7cfec58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cf810f6a0 0x7f7cf8118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.639+0000 7f7cfec58700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf810fb10 0x7f7cf8119170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.639+0000 7f7cfec58700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf811a780 0x7f7cf8111db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.639+0000 7f7cfec58700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf8112580 con 0x7f7cf810f6a0 2024-05-09T00:05:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.639+0000 7f7cfec58700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf81126f0 con 0x7f7cf810fb10 2024-05-09T00:05:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.639+0000 7f7cfec58700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf81128b0 con 0x7f7cf811a780 2024-05-09T00:05:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.639+0000 7f7cfc9f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cf810f6a0 0x7f7cf8118bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.639+0000 7f7cfc9f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cf810f6a0 0x7f7cf8118bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59102/0 (socket says 172.21.15.12:59102) 2024-05-09T00:05:46.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.639+0000 7f7cfc9f4700 1 -- 172.21.15.12:0/2627864485 learned_addr learned my addr 172.21.15.12:0/2627864485 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.640+0000 7f7cfc9f4700 1 -- 172.21.15.12:0/2627864485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf811a780 msgr2=0x7f7cf8111db0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:05:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.640+0000 7f7cfc9f4700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf811a780 0x7f7cf8111db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.640+0000 7f7cfc9f4700 1 -- 172.21.15.12:0/2627864485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf810fb10 msgr2=0x7f7cf8119170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.640+0000 7f7cfc9f4700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf810fb10 0x7f7cf8119170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.640+0000 7f7cfc9f4700 1 -- 172.21.15.12:0/2627864485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec0095f0 con 0x7f7cf810f6a0 2024-05-09T00:05:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.641+0000 7f7cfc9f4700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cf810f6a0 0x7f7cf8118bf0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000b840 tx=0x7f7cf000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.648+0000 7f7cf5ffb700 1 -- 172.21.15.12:0/2627864485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf000d890 con 0x7f7cf810f6a0 2024-05-09T00:05:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.648+0000 7f7cf5ffb700 1 -- 172.21.15.12:0/2627864485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cf000e430 con 0x7f7cf810f6a0 2024-05-09T00:05:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.648+0000 7f7cf5ffb700 1 -- 172.21.15.12:0/2627864485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf000c5e0 con 0x7f7cf810f6a0 2024-05-09T00:05:46.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.648+0000 7f7cfec58700 1 -- 172.21.15.12:0/2627864485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf8116780 con 0x7f7cf810f6a0 2024-05-09T00:05:46.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.649+0000 7f7cfec58700 1 -- 172.21.15.12:0/2627864485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf8116c40 con 0x7f7cf810f6a0 2024-05-09T00:05:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.657+0000 7f7d0ae12b80 1 Processor -- start 2024-05-09T00:05:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.657+0000 7f7d0ae12b80 1 -- start start 2024-05-09T00:05:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.657+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f67de00 0x562c6f67e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.657+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67e720 0x562c6f678350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.657+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f678890 0x562c6f678cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.658+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f5cf150 con 0x562c6f67de00 2024-05-09T00:05:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.658+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f5ccb10 con 0x562c6f678890 2024-05-09T00:05:46.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.658+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f5c8400 con 0x562c6f67e720 2024-05-09T00:05:46.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.660+0000 7f7cfc9f4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67e720 0x562c6f678350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.660+0000 7f7cfc9f4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67e720 0x562c6f678350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45304/0 (socket says 172.21.15.12:45304) 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.660+0000 7f7cfc9f4700 1 -- 172.21.15.12:0/1332880577 learned_addr learned my addr 172.21.15.12:0/1332880577 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.660+0000 7f7cf5ffb700 1 -- 172.21.15.12:0/2627864485 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7cf000d9f0 con 0x7f7cf810f6a0 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.661+0000 7f7cf5ffb700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ce00a14a0 0x7f7ce00a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.662+0000 7f7cf5ffb700 1 -- 172.21.15.12:0/2627864485 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7cf00ce910 con 0x7f7cf810f6a0 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.662+0000 7f7cfc9f4700 1 -- 172.21.15.12:0/1332880577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f678890 msgr2=0x562c6f678cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.662+0000 7f7cfc9f4700 1 --2- 172.21.15.12:0/1332880577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f678890 0x562c6f678cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.662+0000 7f7cfc9f4700 1 -- 172.21.15.12:0/1332880577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f67de00 msgr2=0x562c6f67e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.662+0000 7f7cfc9f4700 1 --2- 172.21.15.12:0/1332880577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f67de00 0x562c6f67e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.662+0000 7f7cfc9f4700 1 -- 172.21.15.12:0/1332880577 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c6f4493b0 con 0x562c6f67e720 2024-05-09T00:05:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.663+0000 7f7cfc9f4700 1 --2- 172.21.15.12:0/1332880577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67e720 0x562c6f678350 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0097ed0 tx=0x7f7cf009b510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2e1961a651f2eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.664+0000 7f7cdeffd700 1 -- 172.21.15.12:0/1332880577 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf009b9d0 con 0x562c6f67e720 2024-05-09T00:05:46.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.664+0000 7f7cdeffd700 1 -- 172.21.15.12:0/1332880577 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cf009f800 con 0x562c6f67e720 2024-05-09T00:05:46.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.664+0000 7f7cdeffd700 1 -- 172.21.15.12:0/1332880577 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf000dca0 con 0x562c6f67e720 2024-05-09T00:05:46.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.664+0000 7f7cfd1f5700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ce00a14a0 0x7f7ce00a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.666+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/1332880577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67e720 msgr2=0x562c6f678350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.666+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/1332880577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67e720 0x562c6f678350 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0097ed0 tx=0x7f7cf009b510 comp rx=0 tx=0).stop 2024-05-09T00:05:46.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.666+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/1332880577 shutdown_connections 2024-05-09T00:05:46.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.666+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/1332880577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f678890 0x562c6f678cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.666+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/1332880577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f67de00 0x562c6f67e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.666+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/1332880577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67e720 0x562c6f678350 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.666+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/1332880577 >> 172.21.15.12:0/1332880577 conn(0x562c6f5d6210 msgr2=0x562c6f672df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.666+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/1332880577 shutdown_connections 2024-05-09T00:05:46.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.666+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/1332880577 wait complete. 2024-05-09T00:05:46.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.667+0000 7f7d0ae12b80 1 -- start start 2024-05-09T00:05:46.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.667+0000 7f7d0ae12b80 1 Processor -- start 2024-05-09T00:05:46.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.667+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f678890 0x562c6f681ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.667+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f682010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.667+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 0x562c6f682550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.667+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f564560 con 0x562c6f678890 2024-05-09T00:05:46.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.668+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f564200 con 0x562c6f67e720 2024-05-09T00:05:46.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.668+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f44b6a0 con 0x562c6f67de00 2024-05-09T00:05:46.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.668+0000 7f7d0ae12b80 10 client.0 _ll_register_callbacks cb 0x562c6f56db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:46.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.668+0000 7f7cfd1f5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 0x562c6f682550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.668+0000 7f7cfd1f5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 0x562c6f682550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41294/0 (socket says 172.21.15.12:41294) 2024-05-09T00:05:46.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.668+0000 7f7cfd1f5700 1 -- 172.21.15.12:0/2166204570 learned_addr learned my addr 172.21.15.12:0/2166204570 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.669+0000 7f7cf7fff700 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f682010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.669+0000 7f7cfd1f5700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ce00a14a0 0x7f7ce00a3960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800ad50 tx=0x7f7ce80136e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.669+0000 7f7cf7fff700 1 -- 172.21.15.12:0/2166204570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 msgr2=0x562c6f682550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.669+0000 7f7cf7fff700 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 0x562c6f682550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.669+0000 7f7cf7fff700 1 -- 172.21.15.12:0/2166204570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f678890 msgr2=0x562c6f681ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.669+0000 7f7cf7fff700 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f678890 0x562c6f681ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.669+0000 7f7cf7fff700 1 -- 172.21.15.12:0/2166204570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf8116c40 con 0x562c6f67de00 2024-05-09T00:05:46.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.670+0000 7f7cf7fff700 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f682010 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7cf810ff30 tx=0x7f7cec003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f767dfb5ab8d12f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.671+0000 7f7cd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:05:46.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.677+0000 7f7cd7fff700 1 -- 172.21.15.12:0/2166204570 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec01d070 con 0x562c6f67de00 2024-05-09T00:05:46.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.677+0000 7f7cd7fff700 1 -- 172.21.15.12:0/2166204570 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7cec003ba0 con 0x562c6f67de00 2024-05-09T00:05:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.684+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/2166204570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c6f521270 con 0x562c6f67de00 2024-05-09T00:05:46.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.709+0000 7f7cd7fff700 1 -- 172.21.15.12:0/2166204570 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cec003d60 con 0x562c6f67de00 2024-05-09T00:05:46.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.709+0000 7f7d0ae12b80 10 client.30508 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:46.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.709+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/2166204570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x562c6f3f5b60 con 0x562c6f67de00 2024-05-09T00:05:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.720+0000 7f7cd7fff700 1 -- 172.21.15.12:0/2166204570 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c6f521270 con 0x562c6f67de00 2024-05-09T00:05:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.720+0000 7f7cd7fff700 1 -- 172.21.15.12:0/2166204570 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7cec02c870 con 0x562c6f67de00 2024-05-09T00:05:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.720+0000 7f7d0ae12b80 10 client.30508 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.720+0000 7f7d0ae12b80 10 client.30508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.720+0000 7f7d0ae12b80 20 client.30508 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.720+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6f6922a0 0x562c6f6946c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.720+0000 7f7d0ae12b80 4 client.30508 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:46.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.720+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/2166204570 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c6f647810 con 0x562c6f6922a0 2024-05-09T00:05:46.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.730+0000 7f7cfc9f4700 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6f6922a0 0x562c6f6946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.743+0000 7f7cfc9f4700 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6f6922a0 0x562c6f6946c0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1df4254898865b server_cookie=595098258f288d8d in_seq=0 out_seq=0 2024-05-09T00:05:46.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.744+0000 7f7cd7fff700 10 client.30508 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.818+0000 7f3446ffd700 1 -- 172.21.15.12:0/2664629258 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c06e5eae60 con 0x55c06e712910 2024-05-09T00:05:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.818+0000 7f3446ffd700 10 client.30344 handle_command_reply: tid=0 2024-05-09T00:05:46.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7cd7fff700 1 -- 172.21.15.12:0/2166204570 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c6f647810 con 0x562c6f6922a0 2024-05-09T00:05:46.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7cd7fff700 10 client.30508 handle_command_reply: tid=0 2024-05-09T00:05:46.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7d0ae12b80 1 client.30508 shutdown 2024-05-09T00:05:46.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/2166204570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 msgr2=0x562c6f682010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f682010 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7cf810ff30 tx=0x7f7cec003730 comp rx=0 tx=0).stop 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/2166204570 shutdown_connections 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6f6922a0 0x562c6f6946c0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 0x562c6f682550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f678890 0x562c6f681ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/2166204570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f682010 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/2166204570 >> 172.21.15.12:0/2166204570 conn(0x562c6f5d6210 msgr2=0x562c6f6725a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7cd7fff700 0 client.30508 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.819+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/2166204570 shutdown_connections 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.820+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/2166204570 wait complete. 2024-05-09T00:05:46.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.820+0000 7f7d0ae12b80 20 client.30508 trim_cache size 0 max 16384 2024-05-09T00:05:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.821+0000 7f3479d1db80 1 client.30344 shutdown 2024-05-09T00:05:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.821+0000 7f3479d1db80 1 -- 172.21.15.12:0/2664629258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 msgr2=0x55c06e6aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.821+0000 7f3479d1db80 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 0x55c06e6aeda0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3458010fd0 tx=0x7f3458009470 comp rx=0 tx=0).stop 2024-05-09T00:05:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.822+0000 7f3479d1db80 1 -- 172.21.15.12:0/2664629258 shutdown_connections 2024-05-09T00:05:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.822+0000 7f3479d1db80 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c06e6c9950 0x55c06e6ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.822+0000 7f3479d1db80 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c06e712910 0x55c06e714d30 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.822+0000 7f3479d1db80 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c06e6c3f50 0x55c06e6aeda0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.822+0000 7f3479d1db80 1 --2- 172.21.15.12:0/2664629258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c06e6c4910 0x55c06e6acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.822+0000 7f3479d1db80 1 -- 172.21.15.12:0/2664629258 >> 172.21.15.12:0/2664629258 conn(0x55c06e6001b0 msgr2=0x55c06e69ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.825+0000 7f3446ffd700 0 client.30344 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.825+0000 7f3479d1db80 1 -- 172.21.15.12:0/2664629258 shutdown_connections 2024-05-09T00:05:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.825+0000 7f3479d1db80 1 -- 172.21.15.12:0/2664629258 wait complete. 2024-05-09T00:05:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.825+0000 7f3479d1db80 20 client.30344 trim_cache size 0 max 16384 2024-05-09T00:05:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:05:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:05:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:05:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:05:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.830+0000 7f346db63700 1 -- 172.21.15.12:0/2690076203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34540a1460 msgr2=0x7f34540a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.830+0000 7f346db63700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34540a1460 0x7f34540a3920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3460009fd0 tx=0x7f3460004c50 comp rx=0 tx=0).stop 2024-05-09T00:05:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.830+0000 7f346db63700 1 -- 172.21.15.12:0/2690076203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34680ff410 msgr2=0x7f346806d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.830+0000 7f346db63700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34680ff410 0x7f346806d730 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f345800b530 tx=0x7f345800b8f0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.836+0000 7f346db63700 1 -- 172.21.15.12:0/2690076203 shutdown_connections 2024-05-09T00:05:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.836+0000 7f346db63700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f34540a1460 0x7f34540a3920 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.836+0000 7f346db63700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3468105dd0 0x7f34681abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.836+0000 7f346db63700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34680ff410 0x7f346806d730 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.836+0000 7f346db63700 1 --2- 172.21.15.12:0/2690076203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34680ffdd0 0x7f346806dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.836+0000 7f346db63700 1 -- 172.21.15.12:0/2690076203 >> 172.21.15.12:0/2690076203 conn(0x7f34680763a0 msgr2=0x7f34681013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.836+0000 7f346db63700 1 -- 172.21.15.12:0/2690076203 shutdown_connections 2024-05-09T00:05:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:46.836+0000 7f346db63700 1 -- 172.21.15.12:0/2690076203 wait complete. 2024-05-09T00:05:46.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7d0ae12b80 1 Processor -- start 2024-05-09T00:05:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7d0ae12b80 1 -- start start 2024-05-09T00:05:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f678890 0x562c6f678350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f672d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 0x562c6f689200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f684b10 con 0x562c6f678890 2024-05-09T00:05:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f6998b0 con 0x562c6f67e720 2024-05-09T00:05:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f674010 con 0x562c6f67de00 2024-05-09T00:05:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7cf7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f672d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7cf7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f672d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45322/0 (socket says 172.21.15.12:45322) 2024-05-09T00:05:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.843+0000 7f7cf7fff700 1 -- 172.21.15.12:0/1644899789 learned_addr learned my addr 172.21.15.12:0/1644899789 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.844+0000 7f7cfd1f5700 1 --2- 172.21.15.12:0/1644899789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 0x562c6f689200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.844+0000 7f7cfd1f5700 1 -- 172.21.15.12:0/1644899789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 msgr2=0x562c6f672d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.844+0000 7f7cfd1f5700 1 --2- 172.21.15.12:0/1644899789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f672d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.844+0000 7f7cfd1f5700 1 -- 172.21.15.12:0/1644899789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f678890 msgr2=0x562c6f678350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.844+0000 7f7cfd1f5700 1 --2- 172.21.15.12:0/1644899789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f678890 0x562c6f678350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.844+0000 7f7cfd1f5700 1 -- 172.21.15.12:0/1644899789 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c6f678f40 con 0x562c6f67e720 2024-05-09T00:05:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.845+0000 7f7cfd1f5700 1 --2- 172.21.15.12:0/1644899789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 0x562c6f689200 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800a510 tx=0x7f7ce8016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d44e804e0a5c6acf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.845+0000 7f7cdf7fe700 1 -- 172.21.15.12:0/1644899789 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8013d70 con 0x562c6f67e720 2024-05-09T00:05:46.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.845+0000 7f7cdf7fe700 1 -- 172.21.15.12:0/1644899789 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ce801e410 con 0x562c6f67e720 2024-05-09T00:05:46.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.845+0000 7f7cdf7fe700 1 -- 172.21.15.12:0/1644899789 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800f6a0 con 0x562c6f67e720 2024-05-09T00:05:46.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.846+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/1644899789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 msgr2=0x562c6f689200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.846+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/1644899789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 0x562c6f689200 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800a510 tx=0x7f7ce8016000 comp rx=0 tx=0).stop 2024-05-09T00:05:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.849+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/1644899789 shutdown_connections 2024-05-09T00:05:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.849+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/1644899789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f67e720 0x562c6f689200 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.849+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/1644899789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f678890 0x562c6f678350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.849+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/1644899789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f672d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.849+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/1644899789 >> 172.21.15.12:0/1644899789 conn(0x562c6f5d6210 msgr2=0x562c6f67bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.849+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/1644899789 shutdown_connections 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.849+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/1644899789 wait complete. 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.850+0000 7f7d0ae12b80 1 -- start start 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.850+0000 7f7d0ae12b80 1 Processor -- start 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.851+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f678890 0x562c6f6e0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.851+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f6e0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.851+0000 7f7d0ae12b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f67e720 0x562c6f6e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.851+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f6e1dc0 con 0x562c6f67e720 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.851+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f6e1f30 con 0x562c6f678890 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.851+0000 7f7d0ae12b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f6e20f0 con 0x562c6f67de00 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.851+0000 7f7d0ae12b80 10 client.0 _ll_register_callbacks cb 0x562c6f56db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.851+0000 7f7cf7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f6e0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.851+0000 7f7cf7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f6e0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45332/0 (socket says 172.21.15.12:45332) 2024-05-09T00:05:46.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.851+0000 7f7cf7fff700 1 -- 172.21.15.12:0/3295692761 learned_addr learned my addr 172.21.15.12:0/3295692761 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.852+0000 7f7cf7fff700 1 -- 172.21.15.12:0/3295692761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f678890 msgr2=0x562c6f6e0a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:05:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.852+0000 7f7cf7fff700 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f678890 0x562c6f6e0a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.852+0000 7f7cf7fff700 1 -- 172.21.15.12:0/3295692761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f67e720 msgr2=0x562c6f6e17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.852+0000 7f7cf7fff700 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f67e720 0x562c6f6e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.852+0000 7f7cf7fff700 1 -- 172.21.15.12:0/3295692761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce8019050 con 0x562c6f67de00 2024-05-09T00:05:46.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.852+0000 7f7cf7fff700 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f6e0f80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00bb40 tx=0x7f7cec009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f70f9dad68372c36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.852+0000 7f7cd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:05:46.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.852+0000 7f7cd7fff700 1 -- 172.21.15.12:0/3295692761 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec031070 con 0x562c6f67de00 2024-05-09T00:05:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.852+0000 7f7cd7fff700 1 -- 172.21.15.12:0/3295692761 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7cec000e40 con 0x562c6f67de00 2024-05-09T00:05:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.854+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/3295692761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cf009f800 con 0x562c6f67de00 2024-05-09T00:05:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.854+0000 7f7cd7fff700 1 -- 172.21.15.12:0/3295692761 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cec02cea0 con 0x562c6f67de00 2024-05-09T00:05:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.854+0000 7f7d0ae12b80 10 client.30514 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.854+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/3295692761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x562c6f58e560 con 0x562c6f67de00 2024-05-09T00:05:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.855+0000 7f7cd7fff700 1 -- 172.21.15.12:0/3295692761 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cf009f800 con 0x562c6f67de00 2024-05-09T00:05:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.855+0000 7f7cd7fff700 1 -- 172.21.15.12:0/3295692761 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7cec034020 con 0x562c6f67de00 2024-05-09T00:05:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.855+0000 7f7d0ae12b80 10 client.30514 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.855+0000 7f7d0ae12b80 10 client.30514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:46.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.855+0000 7f7d0ae12b80 20 client.30514 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:46.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.855+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6f6e95c0 0x562c6f6eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:46.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.855+0000 7f7d0ae12b80 4 client.30514 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:05:46.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.855+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/3295692761 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c6f5c0ec0 con 0x562c6f6e95c0 2024-05-09T00:05:46.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.855+0000 7f7cfc9f4700 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6f6e95c0 0x562c6f6eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:46.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.865+0000 7f7cd7fff700 10 client.30514 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:46.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.865+0000 7f7cfc9f4700 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6f6e95c0 0x562c6f6eb9e0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de2ec72e297e49c server_cookie=ff2d167dd58ed24e in_seq=0 out_seq=0 2024-05-09T00:05:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.869+0000 7f7cd7fff700 1 -- 172.21.15.12:0/3295692761 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c6f5c0ec0 con 0x562c6f6e95c0 2024-05-09T00:05:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.869+0000 7f7cd7fff700 10 client.30514 handle_command_reply: tid=0 2024-05-09T00:05:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.869+0000 7f7d0ae12b80 1 client.30514 shutdown 2024-05-09T00:05:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.869+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/3295692761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 msgr2=0x562c6f6e0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.869+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f6e0f80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00bb40 tx=0x7f7cec009fd0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.870+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/3295692761 shutdown_connections 2024-05-09T00:05:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.870+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c6f67e720 0x562c6f6e17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.870+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562c6f6e95c0 0x562c6f6eb9e0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.870+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562c6f678890 0x562c6f6e0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.870+0000 7f7d0ae12b80 1 --2- 172.21.15.12:0/3295692761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6f67de00 0x562c6f6e0f80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.870+0000 7f7cd7fff700 0 client.30514 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.870+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/3295692761 >> 172.21.15.12:0/3295692761 conn(0x562c6f5d6210 msgr2=0x562c6f67cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.870+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/3295692761 shutdown_connections 2024-05-09T00:05:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.870+0000 7f7d0ae12b80 1 -- 172.21.15.12:0/3295692761 wait complete. 2024-05-09T00:05:46.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.870+0000 7f7d0ae12b80 20 client.30514 trim_cache size 0 max 16384 2024-05-09T00:05:46.865 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:05:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 -- 172.21.15.12:0/2627864485 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ce00a14a0 msgr2=0x7f7ce00a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ce00a14a0 0x7f7ce00a3960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800ad50 tx=0x7f7ce80136e0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 -- 172.21.15.12:0/2627864485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cf810f6a0 msgr2=0x7f7cf8118bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cf810f6a0 0x7f7cf8118bf0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000b840 tx=0x7f7cf000bc00 comp rx=0 tx=0).stop 2024-05-09T00:05:46.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 -- 172.21.15.12:0/2627864485 shutdown_connections 2024-05-09T00:05:46.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7ce00a14a0 0x7f7ce00a3960 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf811a780 0x7f7cf8111db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cf810f6a0 0x7f7cf8118bf0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 --2- 172.21.15.12:0/2627864485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf810fb10 0x7f7cf8119170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:46.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 -- 172.21.15.12:0/2627864485 >> 172.21.15.12:0/2627864485 conn(0x7f7cf806bff0 msgr2=0x7f7cf806f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:46.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 -- 172.21.15.12:0/2627864485 shutdown_connections 2024-05-09T00:05:46.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:46.871+0000 7f7cfec58700 1 -- 172.21.15.12:0/2627864485 wait complete. 2024-05-09T00:05:47.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:46 smithi204 ceph-mon[47373]: pgmap v621: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.8 MiB/s wr, 561 op/s 2024-05-09T00:05:47.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:46 smithi160 ceph-mon[46152]: pgmap v621: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.8 MiB/s wr, 561 op/s 2024-05-09T00:05:47.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:46 smithi012 ceph-mon[42503]: pgmap v621: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.8 MiB/s wr, 561 op/s 2024-05-09T00:05:47.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 228715 34.40 MB/sec execute 222 sec latency 376.349 ms 2024-05-09T00:05:47.886 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:05:47.886 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:05:48.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:47 smithi204 ceph-mon[47373]: pgmap v622: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 814 op/s 2024-05-09T00:05:48.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:47 smithi160 ceph-mon[46152]: pgmap v622: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 814 op/s 2024-05-09T00:05:48.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.058+0000 7fdeeda0e700 1 -- 172.21.15.12:0/2033212178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8100a00 msgr2=0x7fdee8104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.058+0000 7fdeeda0e700 1 --2- 172.21.15.12:0/2033212178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8100a00 0x7fdee8104a40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00d960 tx=0x7fdedc00dc70 comp rx=0 tx=0).stop 2024-05-09T00:05:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.060+0000 7fdeeda0e700 1 -- 172.21.15.12:0/2033212178 shutdown_connections 2024-05-09T00:05:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.060+0000 7fdeeda0e700 1 --2- 172.21.15.12:0/2033212178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee8105170 0x7fdee810ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.060+0000 7fdeeda0e700 1 --2- 172.21.15.12:0/2033212178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee8100050 0x7fdee8100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.060+0000 7fdeeda0e700 1 --2- 172.21.15.12:0/2033212178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8100a00 0x7fdee8104a40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.060+0000 7fdeeda0e700 1 -- 172.21.15.12:0/2033212178 >> 172.21.15.12:0/2033212178 conn(0x7fdee8076fa0 msgr2=0x7fdee80773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.060+0000 7fdeeda0e700 1 -- 172.21.15.12:0/2033212178 shutdown_connections 2024-05-09T00:05:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.060+0000 7fdeeda0e700 1 -- 172.21.15.12:0/2033212178 wait complete. 2024-05-09T00:05:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.061+0000 7fdeeda0e700 1 Processor -- start 2024-05-09T00:05:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.061+0000 7fdeeda0e700 1 -- start start 2024-05-09T00:05:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.061+0000 7fdeeda0e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee8100050 0x7fdee8195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.061+0000 7fdeeda0e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8105170 0x7fdee81957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.061+0000 7fdeeda0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee8199b60 0x7fdee8199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.061+0000 7fdeeda0e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee819a5e0 con 0x7fdee8199b60 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.061+0000 7fdeeda0e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee819a720 con 0x7fdee8100050 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdeeda0e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee819a890 con 0x7fdee8105170 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee8199b60 0x7fdee8199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee67fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8105170 0x7fdee81957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee67fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8105170 0x7fdee81957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45360/0 (socket says 172.21.15.12:45360) 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee67fc700 1 -- 172.21.15.12:0/3348609895 learned_addr learned my addr 172.21.15.12:0/3348609895 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee8100050 0x7fdee8195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee67fc700 1 -- 172.21.15.12:0/3348609895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee8100050 msgr2=0x7fdee8195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee67fc700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee8100050 0x7fdee8195280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee67fc700 1 -- 172.21.15.12:0/3348609895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee8199b60 msgr2=0x7fdee8199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee67fc700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee8199b60 0x7fdee8199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee67fc700 1 -- 172.21.15.12:0/3348609895 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc00d590 con 0x7fdee8105170 2024-05-09T00:05:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee77fe700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee8199b60 0x7fdee8199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.062+0000 7fdee67fc700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8105170 0x7fdee81957c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00d880 tx=0x7fdedc00c580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.063+0000 7fdeeca0c700 1 -- 172.21.15.12:0/3348609895 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc005390 con 0x7fdee8105170 2024-05-09T00:05:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.063+0000 7fdeeca0c700 1 -- 172.21.15.12:0/3348609895 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdedc0054f0 con 0x7fdee8105170 2024-05-09T00:05:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.063+0000 7fdeeda0e700 1 -- 172.21.15.12:0/3348609895 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee806a9c0 con 0x7fdee8105170 2024-05-09T00:05:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.063+0000 7fdeeca0c700 1 -- 172.21.15.12:0/3348609895 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc003b10 con 0x7fdee8105170 2024-05-09T00:05:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.063+0000 7fdeeda0e700 1 -- 172.21.15.12:0/3348609895 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee806adb0 con 0x7fdee8105170 2024-05-09T00:05:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.066+0000 7fdeeca0c700 1 -- 172.21.15.12:0/3348609895 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdedc01c040 con 0x7fdee8105170 2024-05-09T00:05:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.067+0000 7fdeeca0c700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fded00a1290 0x7fded00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.067+0000 7fdeeca0c700 1 -- 172.21.15.12:0/3348609895 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdedc0cf340 con 0x7fdee8105170 2024-05-09T00:05:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.067+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fded00a1290 0x7fded00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdef9bc8b80 1 Processor -- start 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fded00a1290 0x7fded00a3750 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fded8005950 tx=0x7fded800b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdef9bc8b80 1 -- start start 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6fffe80 0x5591a7000260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a6ffa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6ffa910 0x5591a6ffad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a6f511d0 con 0x5591a70007a0 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a6f4eb90 con 0x5591a6ffa910 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a6f4a480 con 0x5591a6fffe80 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdee67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a6ffa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdee67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a6ffa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59170/0 (socket says 172.21.15.12:59170) 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.068+0000 7fdee67fc700 1 -- 172.21.15.12:0/902024580 learned_addr learned my addr 172.21.15.12:0/902024580 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.069+0000 7fdee67fc700 1 -- 172.21.15.12:0/902024580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6fffe80 msgr2=0x5591a7000260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.069+0000 7fdee67fc700 1 --2- 172.21.15.12:0/902024580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6fffe80 0x5591a7000260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.069+0000 7fdee67fc700 1 -- 172.21.15.12:0/902024580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6ffa910 msgr2=0x5591a6ffad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.069+0000 7fdee67fc700 1 --2- 172.21.15.12:0/902024580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6ffa910 0x5591a6ffad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.069+0000 7fdee67fc700 1 -- 172.21.15.12:0/902024580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591a6dcb430 con 0x5591a70007a0 2024-05-09T00:05:48.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.069+0000 7fdee67fc700 1 --2- 172.21.15.12:0/902024580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a6ffa3d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fdedc098f60 tx=0x7fdedc09c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecfb5b252069702 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.069+0000 7fdecdffb700 1 -- 172.21.15.12:0/902024580 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc00c350 con 0x5591a70007a0 2024-05-09T00:05:48.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.069+0000 7fdecdffb700 1 -- 172.21.15.12:0/902024580 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdedc09cbf0 con 0x5591a70007a0 2024-05-09T00:05:48.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.069+0000 7fdecdffb700 1 -- 172.21.15.12:0/902024580 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc003c70 con 0x5591a70007a0 2024-05-09T00:05:48.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.070+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/902024580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 msgr2=0x5591a6ffa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.070+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/902024580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a6ffa3d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fdedc098f60 tx=0x7fdedc09c0b0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.070+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/902024580 shutdown_connections 2024-05-09T00:05:48.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.070+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/902024580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6ffa910 0x5591a6ffad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.070+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/902024580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6fffe80 0x5591a7000260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.070+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/902024580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a6ffa3d0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.070+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/902024580 >> 172.21.15.12:0/902024580 conn(0x5591a6f58290 msgr2=0x5591a6ff4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.070+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/902024580 shutdown_connections 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.070+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/902024580 wait complete. 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.071+0000 7fdef9bc8b80 1 -- start start 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.071+0000 7fdef9bc8b80 1 Processor -- start 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.071+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.071+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6fffe80 0x5591a7004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.071+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a70045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.071+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a6ee65e0 con 0x5591a70007a0 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.071+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a6ee6280 con 0x5591a6fffe80 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.071+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a6dcd720 con 0x5591a6ffa910 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdef9bc8b80 10 client.0 _ll_register_callbacks cb 0x5591a6eefbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdee6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdee6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7003b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45386/0 (socket says 172.21.15.12:45386) 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdee6ffd700 1 -- 172.21.15.12:0/4053437419 learned_addr learned my addr 172.21.15.12:0/4053437419 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdee77fe700 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a70045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdee77fe700 1 -- 172.21.15.12:0/4053437419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 msgr2=0x5591a7003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdee77fe700 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdee77fe700 1 -- 172.21.15.12:0/4053437419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6fffe80 msgr2=0x5591a7004070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdee77fe700 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6fffe80 0x5591a7004070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdee77fe700 1 -- 172.21.15.12:0/4053437419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee806adb0 con 0x5591a70007a0 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdee77fe700 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a70045b0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fdee0004630 tx=0x7fdee00049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823f492788bcad10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.072+0000 7fdec6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.073+0000 7fdec6ffd700 1 -- 172.21.15.12:0/4053437419 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee000ebf0 con 0x5591a70007a0 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.073+0000 7fdec6ffd700 1 -- 172.21.15.12:0/4053437419 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdee00093a0 con 0x5591a70007a0 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.073+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/4053437419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591a6ea32f0 con 0x5591a70007a0 2024-05-09T00:05:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.073+0000 7fdec6ffd700 1 -- 172.21.15.12:0/4053437419 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdee000ed50 con 0x5591a70007a0 2024-05-09T00:05:48.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.073+0000 7fdef9bc8b80 10 client.20826 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:48.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.073+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/4053437419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5591a6d77be0 con 0x5591a70007a0 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.073+0000 7fdec6ffd700 1 -- 172.21.15.12:0/4053437419 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5591a6ea32f0 con 0x5591a70007a0 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.074+0000 7fdec6ffd700 1 -- 172.21.15.12:0/4053437419 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdee0015530 con 0x5591a70007a0 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.074+0000 7fdef9bc8b80 10 client.20826 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.074+0000 7fdef9bc8b80 10 client.20826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.074+0000 7fdef9bc8b80 20 client.20826 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.074+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591a700a1e0 0x5591a700a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.074+0000 7fdef9bc8b80 4 client.20826 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.074+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/4053437419 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591a6fc9890 con 0x5591a700a1e0 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.074+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591a700a1e0 0x5591a700a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.074+0000 7fdec6ffd700 10 client.20826 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.074+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591a700a1e0 0x5591a700a5f0 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a4d6855b6e141 server_cookie=8128eb62c649c200 in_seq=0 out_seq=0 2024-05-09T00:05:48.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.078+0000 7fdec6ffd700 1 -- 172.21.15.12:0/4053437419 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5591a6fc9890 con 0x5591a700a1e0 2024-05-09T00:05:48.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.078+0000 7fdec6ffd700 10 client.20826 handle_command_reply: tid=0 2024-05-09T00:05:48.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.078+0000 7fdef9bc8b80 1 client.20826 shutdown 2024-05-09T00:05:48.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.079+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/4053437419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 msgr2=0x5591a70045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.079+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a70045b0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fdee0004630 tx=0x7fdee00049f0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.079+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/4053437419 shutdown_connections 2024-05-09T00:05:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.079+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a70007a0 0x5591a70045b0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.079+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.079+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591a700a1e0 0x5591a700a5f0 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.079+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/4053437419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6fffe80 0x5591a7004070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.079+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/4053437419 >> 172.21.15.12:0/4053437419 conn(0x5591a6f58290 msgr2=0x5591a6ff4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.079+0000 7fdec6ffd700 0 client.20826 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.079+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/4053437419 shutdown_connections 2024-05-09T00:05:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.080+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/4053437419 wait complete. 2024-05-09T00:05:48.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.082+0000 7fdef9bc8b80 20 client.20826 trim_cache size 0 max 16384 2024-05-09T00:05:48.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdef9bc8b80 1 Processor -- start 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdef9bc8b80 1 -- start start 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6ffa910 0x5591a6ff4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6fffe80 0x5591a70238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a70007a0 0x5591a6ffec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a6ff1e40 con 0x5591a6ffa910 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a6fed0c0 con 0x5591a6fffe80 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a6ff5b20 con 0x5591a70007a0 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdee6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6ffa910 0x5591a6ff4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdee77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a70007a0 0x5591a6ffec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdee6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6ffa910 0x5591a6ff4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59188/0 (socket says 172.21.15.12:59188) 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdee77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a70007a0 0x5591a6ffec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45398/0 (socket says 172.21.15.12:45398) 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.109+0000 7fdee6ffd700 1 -- 172.21.15.12:0/3536517028 learned_addr learned my addr 172.21.15.12:0/3536517028 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdee67fc700 1 --2- 172.21.15.12:0/3536517028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6fffe80 0x5591a70238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdee77fe700 1 -- 172.21.15.12:0/3536517028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6fffe80 msgr2=0x5591a70238c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdee77fe700 1 --2- 172.21.15.12:0/3536517028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6fffe80 0x5591a70238c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdee77fe700 1 -- 172.21.15.12:0/3536517028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6ffa910 msgr2=0x5591a6ff4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdee77fe700 1 --2- 172.21.15.12:0/3536517028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6ffa910 0x5591a6ff4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdee77fe700 1 -- 172.21.15.12:0/3536517028 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591a6ff1560 con 0x5591a70007a0 2024-05-09T00:05:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/3536517028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6ffa910 0x5591a6ff4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdee77fe700 1 --2- 172.21.15.12:0/3536517028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a70007a0 0x5591a6ffec00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdee0000f80 tx=0x7fdee0003f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d46992d36cc0657 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdece7fc700 1 -- 172.21.15.12:0/3536517028 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0009990 con 0x5591a70007a0 2024-05-09T00:05:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdece7fc700 1 -- 172.21.15.12:0/3536517028 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdee000f040 con 0x5591a70007a0 2024-05-09T00:05:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.110+0000 7fdece7fc700 1 -- 172.21.15.12:0/3536517028 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee00154d0 con 0x5591a70007a0 2024-05-09T00:05:48.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.111+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3536517028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a70007a0 msgr2=0x5591a6ffec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.111+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/3536517028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a70007a0 0x5591a6ffec00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdee0000f80 tx=0x7fdee0003f70 comp rx=0 tx=0).stop 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.111+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3536517028 shutdown_connections 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.111+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/3536517028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a70007a0 0x5591a6ffec00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.111+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/3536517028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6ffa910 0x5591a6ff4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.111+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/3536517028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a6fffe80 0x5591a70238c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.111+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3536517028 >> 172.21.15.12:0/3536517028 conn(0x5591a6f58290 msgr2=0x5591a7002400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.111+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3536517028 shutdown_connections 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.111+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3536517028 wait complete. 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.112+0000 7fdef9bc8b80 1 -- start start 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.112+0000 7fdef9bc8b80 1 Processor -- start 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.112+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7023640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.112+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6fffe80 0x5591a7023b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.112+0000 7fdef9bc8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a70007a0 0x5591a701da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.112+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a701e180 con 0x5591a6fffe80 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.112+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a701e2f0 con 0x5591a70007a0 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.112+0000 7fdef9bc8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591a701e4b0 con 0x5591a6ffa910 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.112+0000 7fdef9bc8b80 10 client.0 _ll_register_callbacks cb 0x5591a6eefbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7023640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7023640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45412/0 (socket says 172.21.15.12:45412) 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee6ffd700 1 -- 172.21.15.12:0/3890567862 learned_addr learned my addr 172.21.15.12:0/3890567862 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a70007a0 0x5591a701da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee6ffd700 1 -- 172.21.15.12:0/3890567862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a70007a0 msgr2=0x5591a701da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a70007a0 0x5591a701da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee67fc700 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6fffe80 0x5591a7023b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee6ffd700 1 -- 172.21.15.12:0/3890567862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6fffe80 msgr2=0x5591a7023b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6fffe80 0x5591a7023b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee6ffd700 1 -- 172.21.15.12:0/3890567862 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded80038a0 con 0x5591a6ffa910 2024-05-09T00:05:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee67fc700 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6fffe80 0x5591a7023b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7023640 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fded8026ce0 tx=0x7fded8013070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b0895feb53e3cb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdec6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdec6ffd700 1 -- 172.21.15.12:0/3890567862 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8026040 con 0x5591a6ffa910 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdec6ffd700 1 -- 172.21.15.12:0/3890567862 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fded8017070 con 0x5591a6ffa910 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3890567862 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdedc09cbf0 con 0x5591a6ffa910 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.113+0000 7fdec6ffd700 1 -- 172.21.15.12:0/3890567862 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fded8026610 con 0x5591a6ffa910 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdef9bc8b80 10 client.30529 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3890567862 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5591a6d77be0 con 0x5591a6ffa910 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdec6ffd700 1 -- 172.21.15.12:0/3890567862 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdedc09cbf0 con 0x5591a6ffa910 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdec6ffd700 1 -- 172.21.15.12:0/3890567862 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fded8035070 con 0x5591a6ffa910 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdef9bc8b80 10 client.30529 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdef9bc8b80 10 client.30529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdef9bc8b80 20 client.30529 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591a7074210 0x5591a7076630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdef9bc8b80 4 client.30529 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:05:48.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3890567862 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5591a6f42f40 con 0x5591a7074210 2024-05-09T00:05:48.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.114+0000 7fdee67fc700 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591a7074210 0x5591a7076630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.115+0000 7fdee67fc700 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591a7074210 0x5591a7076630 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=598376482588ace0 server_cookie=615a18eabeccdf63 in_seq=0 out_seq=0 2024-05-09T00:05:48.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.115+0000 7fdec6ffd700 10 client.30529 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdec6ffd700 1 -- 172.21.15.12:0/3890567862 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5591a6f42f40 con 0x5591a7074210 2024-05-09T00:05:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdec6ffd700 10 client.30529 handle_command_reply: tid=0 2024-05-09T00:05:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdef9bc8b80 1 client.30529 shutdown 2024-05-09T00:05:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3890567862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 msgr2=0x5591a7023640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7023640 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fded8026ce0 tx=0x7fded8013070 comp rx=0 tx=0).stop 2024-05-09T00:05:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3890567862 shutdown_connections 2024-05-09T00:05:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591a7074210 0x5591a7076630 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5591a70007a0 0x5591a701da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591a6ffa910 0x5591a7023640 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdef9bc8b80 1 --2- 172.21.15.12:0/3890567862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5591a6fffe80 0x5591a7023b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdec6ffd700 0 client.30529 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3890567862 >> 172.21.15.12:0/3890567862 conn(0x5591a6f58290 msgr2=0x5591a6ffec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.119+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3890567862 shutdown_connections 2024-05-09T00:05:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdef9bc8b80 1 -- 172.21.15.12:0/3890567862 wait complete. 2024-05-09T00:05:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdef9bc8b80 20 client.30529 trim_cache size 0 max 16384 2024-05-09T00:05:48.115 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:05:48.115 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:05:48.115 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "d8802d9c-97a0-4647-80e5-f3dc1d0b22f3", 2024-05-09T00:05:48.115 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:05:48.116 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:05:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 -- 172.21.15.12:0/3348609895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fded00a1290 msgr2=0x7fded00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fded00a1290 0x7fded00a3750 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fded8005950 tx=0x7fded800b410 comp rx=0 tx=0).stop 2024-05-09T00:05:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 -- 172.21.15.12:0/3348609895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8105170 msgr2=0x7fdee81957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8105170 0x7fdee81957c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00d880 tx=0x7fdedc00c580 comp rx=0 tx=0).stop 2024-05-09T00:05:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 -- 172.21.15.12:0/3348609895 shutdown_connections 2024-05-09T00:05:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fded00a1290 0x7fded00a3750 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee8199b60 0x7fdee8199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee8100050 0x7fdee8195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 --2- 172.21.15.12:0/3348609895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8105170 0x7fdee81957c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 -- 172.21.15.12:0/3348609895 >> 172.21.15.12:0/3348609895 conn(0x7fdee8076fa0 msgr2=0x7fdee80fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 -- 172.21.15.12:0/3348609895 shutdown_connections 2024-05-09T00:05:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.120+0000 7fdeeda0e700 1 -- 172.21.15.12:0/3348609895 wait complete. 2024-05-09T00:05:48.129 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:47 smithi012 ceph-mon[42503]: pgmap v622: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 814 op/s 2024-05-09T00:05:48.130 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:05:48.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 229319 34.42 MB/sec execute 223 sec latency 133.615 ms 2024-05-09T00:05:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.322+0000 7fa3d3518700 1 -- 172.21.15.12:0/4251605252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3cc106050 msgr2=0x7fa3cc106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.322+0000 7fa3d3518700 1 --2- 172.21.15.12:0/4251605252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3cc106050 0x7fa3cc106430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400b0a0 tx=0x7fa3c400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.322+0000 7fa3d3518700 1 -- 172.21.15.12:0/4251605252 shutdown_connections 2024-05-09T00:05:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.322+0000 7fa3d3518700 1 --2- 172.21.15.12:0/4251605252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0ffeb0 0x7fa3cc103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.322+0000 7fa3d3518700 1 --2- 172.21.15.12:0/4251605252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3cc106050 0x7fa3cc106430 secure :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400b0a0 tx=0x7fa3c400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.322+0000 7fa3d3518700 1 --2- 172.21.15.12:0/4251605252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3cc0ff410 0x7fa3cc0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.322+0000 7fa3d3518700 1 -- 172.21.15.12:0/4251605252 >> 172.21.15.12:0/4251605252 conn(0x7fa3cc0763a0 msgr2=0x7fa3cc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.323+0000 7fa3d3518700 1 -- 172.21.15.12:0/4251605252 shutdown_connections 2024-05-09T00:05:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.323+0000 7fa3d3518700 1 -- 172.21.15.12:0/4251605252 wait complete. 2024-05-09T00:05:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.323+0000 7fa3d3518700 1 Processor -- start 2024-05-09T00:05:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d3518700 1 -- start start 2024-05-09T00:05:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d3518700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3cc0ff410 0x7fa3cc114b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d3518700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0ffeb0 0x7fa3cc10dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d3518700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3cc10e180 0x7fa3cc10e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d3518700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc1127c0 con 0x7fa3cc10e180 2024-05-09T00:05:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d3518700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc112930 con 0x7fa3cc0ffeb0 2024-05-09T00:05:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d3518700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc112af0 con 0x7fa3cc0ff410 2024-05-09T00:05:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d0ab3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0ffeb0 0x7fa3cc10dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d0ab3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0ffeb0 0x7fa3cc10dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41372/0 (socket says 172.21.15.12:41372) 2024-05-09T00:05:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d0ab3700 1 -- 172.21.15.12:0/2563816921 learned_addr learned my addr 172.21.15.12:0/2563816921 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d0ab3700 1 -- 172.21.15.12:0/2563816921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3cc0ff410 msgr2=0x7fa3cc114b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:05:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3cc0ff410 0x7fa3cc114b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d0ab3700 1 -- 172.21.15.12:0/2563816921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3cc10e180 msgr2=0x7fa3cc10e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3cc10e180 0x7fa3cc10e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.324+0000 7fa3d0ab3700 1 -- 172.21.15.12:0/2563816921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c4009d00 con 0x7fa3cc0ffeb0 2024-05-09T00:05:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.325+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0ffeb0 0x7fa3cc10dc40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b530 tx=0x7fa3bc00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.325+0000 7fa3c27fc700 1 -- 172.21.15.12:0/2563816921 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc011830 con 0x7fa3cc0ffeb0 2024-05-09T00:05:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.325+0000 7fa3c27fc700 1 -- 172.21.15.12:0/2563816921 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3bc011e50 con 0x7fa3cc0ffeb0 2024-05-09T00:05:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.325+0000 7fa3c27fc700 1 -- 172.21.15.12:0/2563816921 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00f2e0 con 0x7fa3cc0ffeb0 2024-05-09T00:05:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.326+0000 7fa3d3518700 1 -- 172.21.15.12:0/2563816921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc112dd0 con 0x7fa3cc0ffeb0 2024-05-09T00:05:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.326+0000 7fa3d3518700 1 -- 172.21.15.12:0/2563816921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc113260 con 0x7fa3cc0ffeb0 2024-05-09T00:05:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.331+0000 7fa3df6d2b80 1 Processor -- start 2024-05-09T00:05:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.331+0000 7fa3df6d2b80 1 -- start start 2024-05-09T00:05:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.331+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68defe10 0x558f68df01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.331+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68dea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.331+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68deace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.331+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68d30150 con 0x558f68df0730 2024-05-09T00:05:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.331+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68d2db10 con 0x558f68defe10 2024-05-09T00:05:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.331+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68d29400 con 0x558f68dea8a0 2024-05-09T00:05:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.332+0000 7fa3c27fc700 1 -- 172.21.15.12:0/2563816921 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa3bc010580 con 0x7fa3cc0ffeb0 2024-05-09T00:05:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.333+0000 7fa3d0ab3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68deace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.333+0000 7fa3d0ab3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68deace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45432/0 (socket says 172.21.15.12:45432) 2024-05-09T00:05:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.333+0000 7fa3d0ab3700 1 -- 172.21.15.12:0/4151895830 learned_addr learned my addr 172.21.15.12:0/4151895830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.333+0000 7fa3d0ab3700 1 -- 172.21.15.12:0/4151895830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68defe10 msgr2=0x558f68df01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.333+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/4151895830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68defe10 0x558f68df01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.333+0000 7fa3d0ab3700 1 -- 172.21.15.12:0/4151895830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 msgr2=0x558f68dea360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.333+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/4151895830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68dea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.333+0000 7fa3d0ab3700 1 -- 172.21.15.12:0/4151895830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f68baa3b0 con 0x558f68dea8a0 2024-05-09T00:05:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.333+0000 7fa3c27fc700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3b80a1290 0x7fa3b80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.334+0000 7fa3c27fc700 1 -- 172.21.15.12:0/2563816921 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa3bc0cc720 con 0x7fa3cc0ffeb0 2024-05-09T00:05:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.334+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/4151895830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68deace0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0c9f00 tx=0x7fa3bc0d2e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5926f250b4beec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.334+0000 7fa3b77fe700 1 -- 172.21.15.12:0/4151895830 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc010ee0 con 0x558f68dea8a0 2024-05-09T00:05:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.334+0000 7fa3b77fe700 1 -- 172.21.15.12:0/4151895830 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3bc099720 con 0x558f68dea8a0 2024-05-09T00:05:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.334+0000 7fa3b77fe700 1 -- 172.21.15.12:0/4151895830 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0992b0 con 0x558f68dea8a0 2024-05-09T00:05:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.334+0000 7fa3d1ab5700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3b80a1290 0x7fa3b80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.335+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/4151895830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 msgr2=0x558f68deace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.335+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/4151895830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68deace0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0c9f00 tx=0x7fa3bc0d2e40 comp rx=0 tx=0).stop 2024-05-09T00:05:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.335+0000 7fa3d1ab5700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3b80a1290 0x7fa3b80a3750 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009730 tx=0x7fa3c8006d70 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.335+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/4151895830 shutdown_connections 2024-05-09T00:05:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.335+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/4151895830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68deace0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.335+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/4151895830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68defe10 0x558f68df01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.335+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/4151895830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68dea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.335+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/4151895830 >> 172.21.15.12:0/4151895830 conn(0x558f68d37210 msgr2=0x558f68dd3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.336+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/4151895830 shutdown_connections 2024-05-09T00:05:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.336+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/4151895830 wait complete. 2024-05-09T00:05:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.337+0000 7fa3df6d2b80 1 -- start start 2024-05-09T00:05:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.337+0000 7fa3df6d2b80 1 Processor -- start 2024-05-09T00:05:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.337+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68df3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68defe10 0x558f68df4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68df0730 0x558f68df45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68cc5560 con 0x558f68defe10 2024-05-09T00:05:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68cc5200 con 0x558f68dea8a0 2024-05-09T00:05:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68bac6a0 con 0x558f68df0730 2024-05-09T00:05:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3df6d2b80 10 client.0 _ll_register_callbacks cb 0x558f68cceb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3d12b4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68df3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3d12b4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68df3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41394/0 (socket says 172.21.15.12:41394) 2024-05-09T00:05:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2224162474 learned_addr learned my addr 172.21.15.12:0/2224162474 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68defe10 0x558f68df4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2224162474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68df0730 msgr2=0x558f68df45d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:05:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3d12b4700 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68df0730 0x558f68df45d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2224162474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68defe10 msgr2=0x558f68df4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3d12b4700 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68defe10 0x558f68df4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2224162474 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc113260 con 0x558f68dea8a0 2024-05-09T00:05:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.338+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68defe10 0x558f68df4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.339+0000 7fa3d12b4700 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68df3b50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009040 tx=0x7fa3c4009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=955b0c24438a94cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.339+0000 7fa3b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:05:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.339+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2224162474 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4021070 con 0x558f68dea8a0 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.339+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2224162474 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa3c4004c40 con 0x558f68dea8a0 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.339+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2224162474 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f68c82270 con 0x558f68dea8a0 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.339+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2224162474 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c400e040 con 0x558f68dea8a0 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.339+0000 7fa3df6d2b80 10 client.30368 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.339+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2224162474 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x558f68b56b60 con 0x558f68dea8a0 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.339+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2224162474 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f68c82270 con 0x558f68dea8a0 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.340+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2224162474 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa3c4024020 con 0x558f68dea8a0 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.340+0000 7fa3df6d2b80 10 client.30368 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.340+0000 7fa3df6d2b80 10 client.30368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.340+0000 7fa3df6d2b80 20 client.30368 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.340+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558f68e041c0 0x558f68e065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.340+0000 7fa3df6d2b80 4 client.30368 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.340+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2224162474 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f68da8810 con 0x558f68e041c0 2024-05-09T00:05:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.340+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558f68e041c0 0x558f68e065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.341+0000 7fa3b4ff9700 10 client.30368 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.341+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558f68e041c0 0x558f68e065e0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3ae3f355ae4e2e server_cookie=918a1dcbbf2ea38 in_seq=0 out_seq=0 2024-05-09T00:05:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2224162474 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f68da8810 con 0x558f68e041c0 2024-05-09T00:05:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3b4ff9700 10 client.30368 handle_command_reply: tid=0 2024-05-09T00:05:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3df6d2b80 1 client.30368 shutdown 2024-05-09T00:05:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2224162474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 msgr2=0x558f68df3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68df3b50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009040 tx=0x7fa3c4009300 comp rx=0 tx=0).stop 2024-05-09T00:05:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2224162474 shutdown_connections 2024-05-09T00:05:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68df0730 0x558f68df45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558f68e041c0 0x558f68e065e0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68df3b50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2224162474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68defe10 0x558f68df4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3b4ff9700 0 client.30368 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2224162474 >> 172.21.15.12:0/2224162474 conn(0x558f68d37210 msgr2=0x558f68dd35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.343+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2224162474 shutdown_connections 2024-05-09T00:05:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.344+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2224162474 wait complete. 2024-05-09T00:05:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.344+0000 7fa3df6d2b80 20 client.30368 trim_cache size 0 max 16384 2024-05-09T00:05:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.367+0000 7fa3df6d2b80 1 Processor -- start 2024-05-09T00:05:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.367+0000 7fa3df6d2b80 1 -- start start 2024-05-09T00:05:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.367+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68def300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.367+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68defe10 0x558f68dd3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.367+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68df43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.367+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68dd0dc0 con 0x558f68df0730 2024-05-09T00:05:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68def840 con 0x558f68dea8a0 2024-05-09T00:05:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68dd5640 con 0x558f68defe10 2024-05-09T00:05:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d12b4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68def300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d1ab5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68df43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d12b4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68def300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:41398/0 (socket says 172.21.15.12:41398) 2024-05-09T00:05:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2678428111 learned_addr learned my addr 172.21.15.12:0/2678428111 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d0ab3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68defe10 0x558f68dd3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2678428111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68defe10 msgr2=0x558f68dd3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d12b4700 1 --2- 172.21.15.12:0/2678428111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68defe10 0x558f68dd3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2678428111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 msgr2=0x558f68df43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d12b4700 1 --2- 172.21.15.12:0/2678428111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68df43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2678428111 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f68deaf50 con 0x558f68dea8a0 2024-05-09T00:05:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.368+0000 7fa3d12b4700 1 --2- 172.21.15.12:0/2678428111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68def300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009c00 tx=0x7fa3c401da90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5cd4efc3cc7243a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.369+0000 7fa3b7fff700 1 -- 172.21.15.12:0/2678428111 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4021070 con 0x558f68dea8a0 2024-05-09T00:05:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.372+0000 7fa3b7fff700 1 -- 172.21.15.12:0/2678428111 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3c401dbe0 con 0x558f68dea8a0 2024-05-09T00:05:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.372+0000 7fa3b7fff700 1 -- 172.21.15.12:0/2678428111 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c400e070 con 0x558f68dea8a0 2024-05-09T00:05:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2678428111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 msgr2=0x558f68def300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2678428111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68def300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009c00 tx=0x7fa3c401da90 comp rx=0 tx=0).stop 2024-05-09T00:05:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2678428111 shutdown_connections 2024-05-09T00:05:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2678428111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68df43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2678428111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68dea8a0 0x558f68def300 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2678428111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68defe10 0x558f68dd3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2678428111 >> 172.21.15.12:0/2678428111 conn(0x558f68d37210 msgr2=0x558f68deebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2678428111 shutdown_connections 2024-05-09T00:05:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2678428111 wait complete. 2024-05-09T00:05:48.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 -- start start 2024-05-09T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.373+0000 7fa3df6d2b80 1 Processor -- start 2024-05-09T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68dfaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68defe10 0x558f68dfb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3df6d2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68df9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68df9bf0 con 0x558f68df0730 2024-05-09T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68df9d60 con 0x558f68defe10 2024-05-09T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3df6d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68df9f20 con 0x558f68dea8a0 2024-05-09T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3df6d2b80 10 client.0 _ll_register_callbacks cb 0x558f68cceb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d1ab5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68df9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d12b4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68dfaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d1ab5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68df9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59274/0 (socket says 172.21.15.12:59274) 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d1ab5700 1 -- 172.21.15.12:0/2694570758 learned_addr learned my addr 172.21.15.12:0/2694570758 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68defe10 0x558f68dfb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2694570758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68defe10 msgr2=0x558f68dfb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d12b4700 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68defe10 0x558f68dfb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2694570758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 msgr2=0x558f68df9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d12b4700 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68df9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d12b4700 1 -- 172.21.15.12:0/2694570758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f68b56b60 con 0x558f68dea8a0 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.374+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68defe10 0x558f68dfb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.375+0000 7fa3d12b4700 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68dfaf50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa3c401db60 tx=0x7fa3c40092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f0213c546b88b85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.375+0000 7fa3b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.375+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2694570758 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c400e070 con 0x558f68dea8a0 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.375+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2694570758 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa3c40125f0 con 0x558f68dea8a0 2024-05-09T00:05:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.375+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2694570758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3bc099720 con 0x558f68dea8a0 2024-05-09T00:05:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.375+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2694570758 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c4021040 con 0x558f68dea8a0 2024-05-09T00:05:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.375+0000 7fa3df6d2b80 10 client.30541 fetch_fsmap learned FSMap version 74 2024-05-09T00:05:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.375+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2694570758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x558f68cef560 con 0x558f68dea8a0 2024-05-09T00:05:48.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.375+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2694570758 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3bc099720 con 0x558f68dea8a0 2024-05-09T00:05:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.376+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2694570758 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa3c403c020 con 0x558f68dea8a0 2024-05-09T00:05:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.377+0000 7fa3df6d2b80 10 client.30541 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:05:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.377+0000 7fa3df6d2b80 10 client.30541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:05:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.377+0000 7fa3df6d2b80 20 client.30541 populate_metadata read hostname 'smithi012' 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.377+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558f68e5b460 0x558f68e5d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.377+0000 7fa3df6d2b80 4 client.30541 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.377+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2694570758 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558f68d21ec0 con 0x558f68e5b460 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.377+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558f68e5b460 0x558f68e5d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.377+0000 7fa3b4ff9700 10 client.30541 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.377+0000 7fa3d0ab3700 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558f68e5b460 0x558f68e5d880 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44d8e58ae4524ce server_cookie=1e7c3739e02b05b0 in_seq=0 out_seq=0 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3b4ff9700 1 -- 172.21.15.12:0/2694570758 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558f68d21ec0 con 0x558f68e5b460 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3b4ff9700 10 client.30541 handle_command_reply: tid=0 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 client.30541 shutdown 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2694570758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 msgr2=0x558f68dfaf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68dfaf50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa3c401db60 tx=0x7fa3c40092e0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2694570758 shutdown_connections 2024-05-09T00:05:48.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f68df0730 0x558f68df9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68dea8a0 0x558f68dfaf50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558f68e5b460 0x558f68e5d880 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 --2- 172.21.15.12:0/2694570758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f68defe10 0x558f68dfb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3b4ff9700 0 client.30541 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:05:48.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2694570758 >> 172.21.15.12:0/2694570758 conn(0x558f68d37210 msgr2=0x558f68dd8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2694570758 shutdown_connections 2024-05-09T00:05:48.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.379+0000 7fa3df6d2b80 1 -- 172.21.15.12:0/2694570758 wait complete. 2024-05-09T00:05:48.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3df6d2b80 20 client.30541 trim_cache size 0 max 16384 2024-05-09T00:05:48.376 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:05:48.376 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:05:48.376 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:05:48.376 INFO:teuthology.orchestra.run.smithi012.stdout: "d8802d9c-97a0-4647-80e5-f3dc1d0b22f3": { 2024-05-09T00:05:48.376 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:05:48.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "d8802d9c-97a0-4647-80e5-f3dc1d0b22f3", 2024-05-09T00:05:48.376 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:05:48.376 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:05:48.376 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:05:48.376 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:05:48.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 -- 172.21.15.12:0/2563816921 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3b80a1290 msgr2=0x7fa3b80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3b80a1290 0x7fa3b80a3750 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009730 tx=0x7fa3c8006d70 comp rx=0 tx=0).stop 2024-05-09T00:05:48.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 -- 172.21.15.12:0/2563816921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0ffeb0 msgr2=0x7fa3cc10dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:48.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0ffeb0 0x7fa3cc10dc40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b530 tx=0x7fa3bc00b8f0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 -- 172.21.15.12:0/2563816921 shutdown_connections 2024-05-09T00:05:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3b80a1290 0x7fa3b80a3750 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3cc10e180 0x7fa3cc10e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3cc0ff410 0x7fa3cc114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 --2- 172.21.15.12:0/2563816921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0ffeb0 0x7fa3cc10dc40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 -- 172.21.15.12:0/2563816921 >> 172.21.15.12:0/2563816921 conn(0x7fa3cc0763a0 msgr2=0x7fa3cc101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 -- 172.21.15.12:0/2563816921 shutdown_connections 2024-05-09T00:05:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:05:48.380+0000 7fa3d3518700 1 -- 172.21.15.12:0/2563816921 wait complete. 2024-05-09T00:05:48.390 INFO:tasks.cephfs.filesystem:scrub status for tag:d8802d9c-97a0-4647-80e5-f3dc1d0b22f3 - {'path': '/', 'tag': 'd8802d9c-97a0-4647-80e5-f3dc1d0b22f3', 'options': 'recursive,force'} 2024-05-09T00:05:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:48 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:05:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:48 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:48 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:05:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:48 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:05:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:48 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:49.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:48 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:49.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:48 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:05:49.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:48 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:49.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:48 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:05:49.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:48 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:05:49.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:48 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:49.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:48 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:49.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:48 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:05:49.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:48 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:49.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:48 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:05:49.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:48 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:05:49.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:48 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:49.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:48 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:49.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 229926 34.40 MB/sec execute 224 sec latency 119.214 ms 2024-05-09T00:05:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:49 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:05:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:49 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:49 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:05:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:49 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:49 smithi204 ceph-mon[47373]: pgmap v623: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 617 op/s 2024-05-09T00:05:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:49 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:05:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:49 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:49 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:05:50.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:49 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:50.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:49 smithi160 ceph-mon[46152]: pgmap v623: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 617 op/s 2024-05-09T00:05:50.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:49 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:05:50.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:49 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:50.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:49 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:05:50.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:49 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:05:50.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:49 smithi012 ceph-mon[42503]: pgmap v623: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 617 op/s 2024-05-09T00:05:50.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 230622 34.41 MB/sec execute 225 sec latency 91.995 ms 2024-05-09T00:05:51.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 231183 34.40 MB/sec execute 226 sec latency 172.919 ms 2024-05-09T00:05:52.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:51 smithi204 ceph-mon[47373]: pgmap v624: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 643 op/s 2024-05-09T00:05:52.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:51 smithi160 ceph-mon[46152]: pgmap v624: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 643 op/s 2024-05-09T00:05:52.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:51 smithi012 ceph-mon[42503]: pgmap v624: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 643 op/s 2024-05-09T00:05:52.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 231619 34.34 MB/sec execute 227 sec latency 403.187 ms 2024-05-09T00:05:53.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 232196 34.35 MB/sec execute 228 sec latency 133.561 ms 2024-05-09T00:05:53.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:53 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:05:54.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:53 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:05:54.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:53 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:05:54.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 232751 34.29 MB/sec execute 229 sec latency 88.551 ms 2024-05-09T00:05:54.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:54 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:05:54.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:54 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:05:54.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:54 smithi012 ceph-mon[42503]: pgmap v625: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 15 MiB/s wr, 891 op/s 2024-05-09T00:05:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:54 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:05:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:54 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:05:55.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:54 smithi204 ceph-mon[47373]: pgmap v625: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 15 MiB/s wr, 891 op/s 2024-05-09T00:05:55.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:54 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:05:55.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:54 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:05:55.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:54 smithi160 ceph-mon[46152]: pgmap v625: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 15 MiB/s wr, 891 op/s 2024-05-09T00:05:55.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 233525 34.37 MB/sec execute 230 sec latency 103.362 ms 2024-05-09T00:05:55.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:55 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:56.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:55 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:56.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:55 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:56.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 234031 34.33 MB/sec execute 231 sec latency 106.145 ms 2024-05-09T00:05:56.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:56 smithi012 ceph-mon[42503]: pgmap v626: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 692 op/s 2024-05-09T00:05:56.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:56 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:56.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:56 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:56.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:56 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:05:57.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:56 smithi204 ceph-mon[47373]: pgmap v626: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 692 op/s 2024-05-09T00:05:57.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:56 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:57.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:56 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:57.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:56 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:05:57.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:56 smithi160 ceph-mon[46152]: pgmap v626: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 692 op/s 2024-05-09T00:05:57.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:56 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:57.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:56 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:57.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:56 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:05:57.266 INFO:tasks.workunit.client.0.smithi012.stdout: 10 234429 34.26 MB/sec execute 232 sec latency 290.262 ms 2024-05-09T00:05:58.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:57 smithi204 ceph-mon[47373]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 867 op/s 2024-05-09T00:05:58.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:57 smithi160 ceph-mon[46152]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 867 op/s 2024-05-09T00:05:58.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:57 smithi012 ceph-mon[42503]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 867 op/s 2024-05-09T00:05:58.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 235007 34.23 MB/sec execute 233 sec latency 108.633 ms 2024-05-09T00:05:59.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 235565 34.24 MB/sec execute 234 sec latency 135.486 ms 2024-05-09T00:05:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.997+0000 7f38c7c59700 1 -- 172.21.15.12:0/13022352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c00ffda0 msgr2=0x7f38c0100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.997+0000 7f38c7c59700 1 --2- 172.21.15.12:0/13022352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c00ffda0 0x7f38c0100220 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009990 tx=0x7f38b0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:05:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.997+0000 7f38c7c59700 1 -- 172.21.15.12:0/13022352 shutdown_connections 2024-05-09T00:05:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.997+0000 7f38c7c59700 1 --2- 172.21.15.12:0/13022352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38c0100760 0x7f38c010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.997+0000 7f38c7c59700 1 --2- 172.21.15.12:0/13022352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38c0105e60 0x7f38c0106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.997+0000 7f38c7c59700 1 --2- 172.21.15.12:0/13022352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c00ffda0 0x7f38c0100220 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.997+0000 7f38c7c59700 1 -- 172.21.15.12:0/13022352 >> 172.21.15.12:0/13022352 conn(0x7f38c0076f10 msgr2=0x7f38c0077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.997+0000 7f38c7c59700 1 -- 172.21.15.12:0/13022352 shutdown_connections 2024-05-09T00:05:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.997+0000 7f38c7c59700 1 -- 172.21.15.12:0/13022352 wait complete. 2024-05-09T00:05:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.997+0000 7f38c7c59700 1 Processor -- start 2024-05-09T00:05:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c7c59700 1 -- start start 2024-05-09T00:05:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c7c59700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38c00ffda0 0x7f38c010d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c7c59700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38c0100760 0x7f38c010ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c7c59700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0105e60 0x7f38c010e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c7c59700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c01a56d0 con 0x7f38c0105e60 2024-05-09T00:05:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c7c59700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c01a5840 con 0x7f38c00ffda0 2024-05-09T00:05:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c7c59700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c01a5a00 con 0x7f38c0100760 2024-05-09T00:05:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c59f5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38c00ffda0 0x7f38c010d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c61f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0105e60 0x7f38c010e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c51f4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38c0100760 0x7f38c010ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c59f5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38c00ffda0 0x7f38c010d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52594/0 (socket says 172.21.15.12:52594) 2024-05-09T00:05:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c51f4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38c0100760 0x7f38c010ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:49684/0 (socket says 172.21.15.12:49684) 2024-05-09T00:05:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c61f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0105e60 0x7f38c010e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57632/0 (socket says 172.21.15.12:57632) 2024-05-09T00:05:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c59f5700 1 -- 172.21.15.12:0/1933213368 learned_addr learned my addr 172.21.15.12:0/1933213368 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:05:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c51f4700 1 -- 172.21.15.12:0/1933213368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38c00ffda0 msgr2=0x7f38c010d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c51f4700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38c00ffda0 0x7f38c010d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c51f4700 1 -- 172.21.15.12:0/1933213368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0105e60 msgr2=0x7f38c010e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c51f4700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0105e60 0x7f38c010e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c51f4700 1 -- 172.21.15.12:0/1933213368 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b00095f0 con 0x7f38c0100760 2024-05-09T00:05:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.998+0000 7f38c61f6700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0105e60 0x7f38c010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:05:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.999+0000 7f38c51f4700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38c0100760 0x7f38c010ddb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f38b00059d0 tx=0x7f38b0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.999+0000 7f38b6ffd700 1 -- 172.21.15.12:0/1933213368 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b001d070 con 0x7f38c0100760 2024-05-09T00:05:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.999+0000 7f38c7c59700 1 -- 172.21.15.12:0/1933213368 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c01a5c80 con 0x7f38c0100760 2024-05-09T00:05:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.999+0000 7f38b6ffd700 1 -- 172.21.15.12:0/1933213368 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38b0022470 con 0x7f38c0100760 2024-05-09T00:05:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.999+0000 7f38b6ffd700 1 -- 172.21.15.12:0/1933213368 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b0017650 con 0x7f38c0100760 2024-05-09T00:05:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:05:59.999+0000 7f38c7c59700 1 -- 172.21.15.12:0/1933213368 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c01a6100 con 0x7f38c0100760 2024-05-09T00:06:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.001+0000 7f38b6ffd700 1 -- 172.21.15.12:0/1933213368 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f38b00225e0 con 0x7f38c0100760 2024-05-09T00:06:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.002+0000 7f38b6ffd700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38ac0a1290 0x7f38ac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.003+0000 7f38c59f5700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38ac0a1290 0x7f38ac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.003+0000 7f38b6ffd700 1 -- 172.21.15.12:0/1933213368 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f38b00cdda0 con 0x7f38c0100760 2024-05-09T00:06:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.003+0000 7f38c59f5700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38ac0a1290 0x7f38ac0a3750 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009d20 tx=0x7f38bc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.003+0000 7f38d3e13b80 1 Processor -- start 2024-05-09T00:06:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38d3e13b80 1 -- start start 2024-05-09T00:06:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec700e950 0x558ec700ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700f270 0x558ec7008f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec7009480 0x558ec70098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec6f600f0 con 0x558ec700e950 2024-05-09T00:06:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec6f5dab0 con 0x558ec7009480 2024-05-09T00:06:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec6f593a0 con 0x558ec700f270 2024-05-09T00:06:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c59f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec700e950 0x558ec700ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c59f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec700e950 0x558ec700ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57636/0 (socket says 172.21.15.12:57636) 2024-05-09T00:06:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c59f5700 1 -- 172.21.15.12:0/4165323404 learned_addr learned my addr 172.21.15.12:0/4165323404 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c51f4700 1 --2- 172.21.15.12:0/4165323404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700f270 0x558ec7008f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c61f6700 1 --2- 172.21.15.12:0/4165323404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec7009480 0x558ec70098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c51f4700 1 -- 172.21.15.12:0/4165323404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec7009480 msgr2=0x558ec70098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c51f4700 1 --2- 172.21.15.12:0/4165323404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec7009480 0x558ec70098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c51f4700 1 -- 172.21.15.12:0/4165323404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec700e950 msgr2=0x558ec700ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c51f4700 1 --2- 172.21.15.12:0/4165323404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec700e950 0x558ec700ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c51f4700 1 -- 172.21.15.12:0/4165323404 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ec6dda350 con 0x558ec700f270 2024-05-09T00:06:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c59f5700 1 --2- 172.21.15.12:0/4165323404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec700e950 0x558ec700ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:06:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.004+0000 7f38c51f4700 1 --2- 172.21.15.12:0/4165323404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700f270 0x558ec7008f40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f38b00cb4a0 tx=0x7f38b0097230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52709d8ff52f96c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.005+0000 7f38abfff700 1 -- 172.21.15.12:0/4165323404 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b009a850 con 0x558ec700f270 2024-05-09T00:06:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.005+0000 7f38abfff700 1 -- 172.21.15.12:0/4165323404 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38b009b5f0 con 0x558ec700f270 2024-05-09T00:06:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.005+0000 7f38abfff700 1 -- 172.21.15.12:0/4165323404 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b009b350 con 0x558ec700f270 2024-05-09T00:06:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.005+0000 7f38d3e13b80 1 -- 172.21.15.12:0/4165323404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700f270 msgr2=0x558ec7008f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.005+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/4165323404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700f270 0x558ec7008f40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f38b00cb4a0 tx=0x7f38b0097230 comp rx=0 tx=0).stop 2024-05-09T00:06:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.006+0000 7f38d3e13b80 1 -- 172.21.15.12:0/4165323404 shutdown_connections 2024-05-09T00:06:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.006+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/4165323404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec7009480 0x558ec70098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.006+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/4165323404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec700e950 0x558ec700ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.006+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/4165323404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700f270 0x558ec7008f40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.006+0000 7f38d3e13b80 1 -- 172.21.15.12:0/4165323404 >> 172.21.15.12:0/4165323404 conn(0x558ec6f671b0 msgr2=0x558ec7003840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.006+0000 7f38d3e13b80 1 -- 172.21.15.12:0/4165323404 shutdown_connections 2024-05-09T00:06:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.006+0000 7f38d3e13b80 1 -- 172.21.15.12:0/4165323404 wait complete. 2024-05-09T00:06:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.006+0000 7f38d3e13b80 1 -- start start 2024-05-09T00:06:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.006+0000 7f38d3e13b80 1 Processor -- start 2024-05-09T00:06:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7012550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec7012c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec6ef5500 con 0x558ec7009480 2024-05-09T00:06:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec6ef51a0 con 0x558ec700f270 2024-05-09T00:06:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec6ddc640 con 0x558ec700e950 2024-05-09T00:06:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38d3e13b80 10 client.0 _ll_register_callbacks cb 0x558ec6efead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c51f4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7012550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c59f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c51f4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7012550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:49700/0 (socket says 172.21.15.12:49700) 2024-05-09T00:06:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c51f4700 1 -- 172.21.15.12:0/3610113244 learned_addr learned my addr 172.21.15.12:0/3610113244 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c61f6700 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec7012c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c51f4700 1 -- 172.21.15.12:0/3610113244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 msgr2=0x558ec7012c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c51f4700 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec7012c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c51f4700 1 -- 172.21.15.12:0/3610113244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 msgr2=0x558ec7015260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c51f4700 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c51f4700 1 -- 172.21.15.12:0/3610113244 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c01a6100 con 0x558ec700e950 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.007+0000 7f38c59f5700 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38c51f4700 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7012550 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f38b0097960 tx=0x7f38b0022c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38d7a6a2489c1014 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38a97fa700 1 -- 172.21.15.12:0/3610113244 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b009a850 con 0x558ec700e950 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38d3e13b80 1 -- 172.21.15.12:0/3610113244 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ec6eb2210 con 0x558ec700e950 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38a97fa700 1 -- 172.21.15.12:0/3610113244 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f38b009aef0 con 0x558ec700e950 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38a97fa700 1 -- 172.21.15.12:0/3610113244 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38b009ed20 con 0x558ec700e950 2024-05-09T00:06:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38d3e13b80 10 client.30553 fetch_fsmap learned FSMap version 74 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38d3e13b80 1 -- 172.21.15.12:0/3610113244 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x558ec6d86b00 con 0x558ec700e950 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38a97fa700 1 -- 172.21.15.12:0/3610113244 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38b00a7080 con 0x558ec700e950 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38a97fa700 1 -- 172.21.15.12:0/3610113244 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38b0095b00 con 0x558ec700e950 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38d3e13b80 10 client.30553 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38d3e13b80 10 client.30553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38d3e13b80 20 client.30553 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ec7022460 0x558ec7024880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38d3e13b80 4 client.30553 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.008+0000 7f38d3e13b80 1 -- 172.21.15.12:0/3610113244 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ec6fd87b0 con 0x558ec7022460 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.009+0000 7f38c59f5700 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ec7022460 0x558ec7024880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.009+0000 7f38a97fa700 10 client.30553 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.009+0000 7f38c59f5700 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ec7022460 0x558ec7024880 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f6fd2dc8f7339f server_cookie=ce4086bda39d4f54 in_seq=0 out_seq=0 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38a97fa700 1 -- 172.21.15.12:0/3610113244 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ec6fd87b0 con 0x558ec7022460 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38a97fa700 10 client.30553 handle_command_reply: tid=0 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 client.30553 shutdown 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 -- 172.21.15.12:0/3610113244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 msgr2=0x558ec7012550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7012550 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f38b0097960 tx=0x7f38b0022c80 comp rx=0 tx=0).stop 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 -- 172.21.15.12:0/3610113244 shutdown_connections 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ec7022460 0x558ec7024880 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec7012c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38a97fa700 0 client.30553 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/3610113244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7012550 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 -- 172.21.15.12:0/3610113244 >> 172.21.15.12:0/3610113244 conn(0x558ec6f671b0 msgr2=0x558ec700d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 -- 172.21.15.12:0/3610113244 shutdown_connections 2024-05-09T00:06:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 1 -- 172.21.15.12:0/3610113244 wait complete. 2024-05-09T00:06:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.012+0000 7f38d3e13b80 20 client.30553 trim_cache size 0 max 16384 2024-05-09T00:06:00.020 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:05:59 smithi160 ceph-mon[46152]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 649 op/s 2024-05-09T00:06:00.020 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:05:59 smithi204 ceph-mon[47373]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 649 op/s 2024-05-09T00:06:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.030+0000 7f38d3e13b80 1 Processor -- start 2024-05-09T00:06:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.030+0000 7f38d3e13b80 1 -- start start 2024-05-09T00:06:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.030+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7012510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec7011e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec7024ee0 con 0x558ec7009480 2024-05-09T00:06:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec7010ae0 con 0x558ec700f270 2024-05-09T00:06:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec70081a0 con 0x558ec700e950 2024-05-09T00:06:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38c59f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7012510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38c59f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7012510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57664/0 (socket says 172.21.15.12:57664) 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38c59f5700 1 -- 172.21.15.12:0/4059871050 learned_addr learned my addr 172.21.15.12:0/4059871050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38c51f4700 1 --2- 172.21.15.12:0/4059871050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38c59f5700 1 -- 172.21.15.12:0/4059871050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 msgr2=0x558ec7003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38c59f5700 1 --2- 172.21.15.12:0/4059871050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38c59f5700 1 -- 172.21.15.12:0/4059871050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 msgr2=0x558ec7011e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38c59f5700 1 --2- 172.21.15.12:0/4059871050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec7011e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38c59f5700 1 -- 172.21.15.12:0/4059871050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ec6f1f500 con 0x558ec7009480 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38c59f5700 1 --2- 172.21.15.12:0/4059871050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7012510 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x558ec7025120 tx=0x7f38bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61af7a710862e41c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38b4da4700 1 -- 172.21.15.12:0/4059871050 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc023040 con 0x558ec7009480 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.031+0000 7f38b4da4700 1 -- 172.21.15.12:0/4059871050 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38bc0298e0 con 0x558ec7009480 2024-05-09T00:06:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.032+0000 7f38b4da4700 1 -- 172.21.15.12:0/4059871050 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc032ea0 con 0x558ec7009480 2024-05-09T00:06:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.032+0000 7f38d3e13b80 1 -- 172.21.15.12:0/4059871050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 msgr2=0x558ec7012510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.032+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/4059871050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7012510 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x558ec7025120 tx=0x7f38bc017000 comp rx=0 tx=0).stop 2024-05-09T00:06:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.032+0000 7f38d3e13b80 1 -- 172.21.15.12:0/4059871050 shutdown_connections 2024-05-09T00:06:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.032+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/4059871050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec7011e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.032+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/4059871050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7012510 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.032+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/4059871050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7003910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.032+0000 7f38d3e13b80 1 -- 172.21.15.12:0/4059871050 >> 172.21.15.12:0/4059871050 conn(0x558ec6f671b0 msgr2=0x558ec70291a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.032+0000 7f38d3e13b80 1 -- 172.21.15.12:0/4059871050 shutdown_connections 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.032+0000 7f38d3e13b80 1 -- 172.21.15.12:0/4059871050 wait complete. 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.033+0000 7f38d3e13b80 1 -- start start 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.033+0000 7f38d3e13b80 1 Processor -- start 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.033+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.033+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7015e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.033+0000 7f38d3e13b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec70163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.033+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec7077910 con 0x558ec7009480 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.033+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec7077a50 con 0x558ec700f270 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.033+0000 7f38d3e13b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec7077b90 con 0x558ec700e950 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.033+0000 7f38d3e13b80 10 client.0 _ll_register_callbacks cb 0x558ec6efead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.033+0000 7f38c59f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38c59f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57672/0 (socket says 172.21.15.12:57672) 2024-05-09T00:06:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38c59f5700 1 -- 172.21.15.12:0/1681730021 learned_addr learned my addr 172.21.15.12:0/1681730021 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38c51f4700 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7015e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38c61f6700 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec70163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38c59f5700 1 -- 172.21.15.12:0/1681730021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 msgr2=0x558ec7015e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38c59f5700 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7015e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38c59f5700 1 -- 172.21.15.12:0/1681730021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 msgr2=0x558ec70163a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38c59f5700 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec70163a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38c59f5700 1 -- 172.21.15.12:0/1681730021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc017070 con 0x558ec7009480 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38c59f5700 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015920 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f38bc01efd0 tx=0x7f38bc0323c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9576a331e9d0274f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38a97fa700 1 -- 172.21.15.12:0/1681730021 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc023070 con 0x558ec7009480 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38a97fa700 1 -- 172.21.15.12:0/1681730021 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f38bc029630 con 0x558ec7009480 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.034+0000 7f38d3e13b80 1 -- 172.21.15.12:0/1681730021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38b009b5f0 con 0x558ec7009480 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38a97fa700 1 -- 172.21.15.12:0/1681730021 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38bc032660 con 0x558ec7009480 2024-05-09T00:06:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38d3e13b80 10 client.20856 fetch_fsmap learned FSMap version 74 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38d3e13b80 1 -- 172.21.15.12:0/1681730021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x558ec6ddec70 con 0x558ec7009480 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38a97fa700 1 -- 172.21.15.12:0/1681730021 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38b009b5f0 con 0x558ec7009480 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38a97fa700 1 -- 172.21.15.12:0/1681730021 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 74) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38bc004750 con 0x558ec7009480 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38d3e13b80 10 client.20856 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38d3e13b80 10 client.20856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38d3e13b80 20 client.20856 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ec7079340 0x558ec707b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38d3e13b80 4 client.20856 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38d3e13b80 1 -- 172.21.15.12:0/1681730021 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ec6f51e60 con 0x558ec7079340 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.035+0000 7f38c51f4700 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ec7079340 0x558ec707b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.036+0000 7f38a97fa700 10 client.20856 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.036+0000 7f38c51f4700 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ec7079340 0x558ec707b760 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deab2fe877034799 server_cookie=d01ec4afe78e50d8 in_seq=0 out_seq=0 2024-05-09T00:06:00.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:05:59 smithi012 ceph-mon[42503]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 649 op/s 2024-05-09T00:06:00.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 236022 34.19 MB/sec execute 235 sec latency 231.589 ms 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.446+0000 7f38a97fa700 1 -- 172.21.15.12:0/1681730021 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ec6f51e60 con 0x558ec7079340 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.446+0000 7f38a97fa700 10 client.20856 handle_command_reply: tid=0 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.446+0000 7f38d3e13b80 1 client.20856 shutdown 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 1 -- 172.21.15.12:0/1681730021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 msgr2=0x558ec7015920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015920 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f38bc01efd0 tx=0x7f38bc0323c0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 1 -- 172.21.15.12:0/1681730021 shutdown_connections 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ec7079340 0x558ec707b760 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ec700f270 0x558ec70163a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ec7009480 0x558ec7015920 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 1 --2- 172.21.15.12:0/1681730021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ec700e950 0x558ec7015e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 1 -- 172.21.15.12:0/1681730021 >> 172.21.15.12:0/1681730021 conn(0x558ec6f671b0 msgr2=0x558ec7007630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38a97fa700 0 client.20856 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 1 -- 172.21.15.12:0/1681730021 shutdown_connections 2024-05-09T00:06:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 1 -- 172.21.15.12:0/1681730021 wait complete. 2024-05-09T00:06:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.447+0000 7f38d3e13b80 20 client.20856 trim_cache size 0 max 16384 2024-05-09T00:06:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:06:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:06:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:06:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:06:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 -- 172.21.15.12:0/1933213368 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38ac0a1290 msgr2=0x7f38ac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38ac0a1290 0x7f38ac0a3750 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009d20 tx=0x7f38bc009260 comp rx=0 tx=0).stop 2024-05-09T00:06:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 -- 172.21.15.12:0/1933213368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38c0100760 msgr2=0x7f38c010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38c0100760 0x7f38c010ddb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f38b00059d0 tx=0x7f38b0004730 comp rx=0 tx=0).stop 2024-05-09T00:06:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 -- 172.21.15.12:0/1933213368 shutdown_connections 2024-05-09T00:06:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f38ac0a1290 0x7f38ac0a3750 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0105e60 0x7f38c010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38c00ffda0 0x7f38c010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 --2- 172.21.15.12:0/1933213368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38c0100760 0x7f38c010ddb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 -- 172.21.15.12:0/1933213368 >> 172.21.15.12:0/1933213368 conn(0x7f38c0076f10 msgr2=0x7f38c00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.448+0000 7f38c7c59700 1 -- 172.21.15.12:0/1933213368 shutdown_connections 2024-05-09T00:06:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:00.449+0000 7f38c7c59700 1 -- 172.21.15.12:0/1933213368 wait complete. 2024-05-09T00:06:01.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 236372 34.09 MB/sec execute 236 sec latency 469.931 ms 2024-05-09T00:06:02.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:01 smithi204 ceph-mon[47373]: pgmap v629: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 636 op/s 2024-05-09T00:06:02.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:01 smithi160 ceph-mon[46152]: pgmap v629: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 636 op/s 2024-05-09T00:06:02.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:01 smithi012 ceph-mon[42503]: pgmap v629: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 636 op/s 2024-05-09T00:06:02.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 236372 33.95 MB/sec execute 237 sec latency 1469.993 ms 2024-05-09T00:06:02.515 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:06:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: ignoring --setuser ceph since I am not root 2024-05-09T00:06:02.515 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:06:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: ignoring --setgroup ceph since I am not root 2024-05-09T00:06:02.515 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:06:02 smithi160 ceph-mds[81732]: -- 172.21.15.160:0/87319625 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55844b1841a0 con 0x55844b15ac00 2024-05-09T00:06:02.515 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:06:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: starting mds.k at 2024-05-09T00:06:03.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:02 smithi204 ceph-mon[47373]: daemon mds.k restarted 2024-05-09T00:06:03.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:02 smithi204 ceph-mon[47373]: osdmap e123: 12 total, 12 up, 12 in 2024-05-09T00:06:03.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:02 smithi160 ceph-mon[46152]: daemon mds.k restarted 2024-05-09T00:06:03.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:02 smithi160 ceph-mon[46152]: osdmap e123: 12 total, 12 up, 12 in 2024-05-09T00:06:03.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:02 smithi012 ceph-mon[42503]: daemon mds.k restarted 2024-05-09T00:06:03.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:02 smithi012 ceph-mon[42503]: osdmap e123: 12 total, 12 up, 12 in 2024-05-09T00:06:03.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 236576 33.87 MB/sec execute 238 sec latency 2167.428 ms 2024-05-09T00:06:04.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:03 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6832/3969500142,v1:172.21.15.160:6833/3969500142] up:boot 2024-05-09T00:06:04.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:03 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:06:04.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:03 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:06:04.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:03 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:06:04.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:03 smithi204 ceph-mon[47373]: pgmap v631: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 690 op/s 2024-05-09T00:06:04.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:03 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6832/3969500142,v1:172.21.15.160:6833/3969500142] up:boot 2024-05-09T00:06:04.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:03 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:06:04.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:03 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:06:04.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:03 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:06:04.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:03 smithi160 ceph-mon[46152]: pgmap v631: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 690 op/s 2024-05-09T00:06:04.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:03 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6832/3969500142,v1:172.21.15.160:6833/3969500142] up:boot 2024-05-09T00:06:04.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:03 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:06:04.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:03 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:06:04.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:03 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:06:04.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:03 smithi012 ceph-mon[42503]: pgmap v631: 97 pgs: 97 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 690 op/s 2024-05-09T00:06:04.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 236964 33.82 MB/sec execute 239 sec latency 307.711 ms 2024-05-09T00:06:05.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:04 smithi204 ceph-mon[47373]: osdmap e124: 12 total, 12 up, 12 in 2024-05-09T00:06:05.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:04 smithi160 ceph-mon[46152]: osdmap e124: 12 total, 12 up, 12 in 2024-05-09T00:06:05.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:04 smithi012 ceph-mon[42503]: osdmap e124: 12 total, 12 up, 12 in 2024-05-09T00:06:05.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 237159 33.73 MB/sec execute 240 sec latency 804.197 ms 2024-05-09T00:06:06.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:05 smithi204 ceph-mon[47373]: pgmap v633: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.2 MiB/s wr, 576 op/s 2024-05-09T00:06:06.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:05 smithi160 ceph-mon[46152]: pgmap v633: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.2 MiB/s wr, 576 op/s 2024-05-09T00:06:06.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:05 smithi012 ceph-mon[42503]: pgmap v633: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.2 MiB/s wr, 576 op/s 2024-05-09T00:06:06.265 INFO:tasks.workunit.client.0.smithi012.stdout: 10 237381 33.63 MB/sec execute 241 sec latency 863.991 ms 2024-05-09T00:06:07.266 INFO:tasks.workunit.client.0.smithi012.stdout: 10 237475 33.51 MB/sec execute 242 sec latency 673.687 ms 2024-05-09T00:06:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:07 smithi204 ceph-mon[47373]: pgmap v634: 97 pgs: 14 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 60 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 722 op/s 2024-05-09T00:06:08.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:07 smithi160 ceph-mon[46152]: pgmap v634: 97 pgs: 14 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 60 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 722 op/s 2024-05-09T00:06:08.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:07 smithi012 ceph-mon[42503]: pgmap v634: 97 pgs: 14 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 60 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 722 op/s 2024-05-09T00:06:08.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 237622 33.39 MB/sec execute 243 sec latency 629.570 ms 2024-05-09T00:06:09.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 237832 33.32 MB/sec execute 244 sec latency 451.301 ms 2024-05-09T00:06:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:09 smithi204 ceph-mon[47373]: pgmap v635: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 711 op/s 2024-05-09T00:06:10.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:09 smithi160 ceph-mon[46152]: pgmap v635: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 711 op/s 2024-05-09T00:06:10.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:09 smithi012 ceph-mon[42503]: pgmap v635: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 711 op/s 2024-05-09T00:06:10.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 238393 33.30 MB/sec execute 245 sec latency 171.533 ms 2024-05-09T00:06:11.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 239040 33.34 MB/sec execute 246 sec latency 91.767 ms 2024-05-09T00:06:12.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:11 smithi204 ceph-mon[47373]: pgmap v636: 97 pgs: 13 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 66 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 399 op/s 2024-05-09T00:06:12.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:11 smithi160 ceph-mon[46152]: pgmap v636: 97 pgs: 13 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 66 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 399 op/s 2024-05-09T00:06:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:11 smithi012 ceph-mon[42503]: pgmap v636: 97 pgs: 13 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 66 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 399 op/s 2024-05-09T00:06:12.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 239447 33.27 MB/sec execute 247 sec latency 422.824 ms 2024-05-09T00:06:13.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 239950 33.23 MB/sec execute 248 sec latency 120.745 ms 2024-05-09T00:06:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.582+0000 7f027cc66700 1 -- 172.21.15.12:0/3253574913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780ffdd0 msgr2=0x7f0278100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.582+0000 7f027cc66700 1 --2- 172.21.15.12:0/3253574913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780ffdd0 0x7f0278100250 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f026c0098f0 tx=0x7f026c009c00 comp rx=0 tx=0).stop 2024-05-09T00:06:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.582+0000 7f027cc66700 1 -- 172.21.15.12:0/3253574913 shutdown_connections 2024-05-09T00:06:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.582+0000 7f027cc66700 1 --2- 172.21.15.12:0/3253574913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278100790 0x7f027810cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.582+0000 7f027cc66700 1 --2- 172.21.15.12:0/3253574913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0278105e90 0x7f0278106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.582+0000 7f027cc66700 1 --2- 172.21.15.12:0/3253574913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780ffdd0 0x7f0278100250 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.582+0000 7f027cc66700 1 -- 172.21.15.12:0/3253574913 >> 172.21.15.12:0/3253574913 conn(0x7f0278076fd0 msgr2=0x7f02780773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.582+0000 7f027cc66700 1 -- 172.21.15.12:0/3253574913 shutdown_connections 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.582+0000 7f027cc66700 1 -- 172.21.15.12:0/3253574913 wait complete. 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f027cc66700 1 Processor -- start 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f027cc66700 1 -- start start 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f027cc66700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02780ffdd0 0x7f0278197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f027cc66700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278100790 0x7f0278197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f027cc66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278105e90 0x7f027819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f027cc66700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027819c240 con 0x7f0278105e90 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f027cc66700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027819c3b0 con 0x7f0278100790 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f027cc66700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027819c570 con 0x7f02780ffdd0 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278105e90 0x7f027819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f027659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02780ffdd0 0x7f0278197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f0275d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278100790 0x7f0278197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278105e90 0x7f027819bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45194/0 (socket says 172.21.15.12:45194) 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.583+0000 7f0276d9d700 1 -- 172.21.15.12:0/4009328318 learned_addr learned my addr 172.21.15.12:0/4009328318 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f0276d9d700 1 -- 172.21.15.12:0/4009328318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02780ffdd0 msgr2=0x7f0278197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f0276d9d700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02780ffdd0 0x7f0278197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f0276d9d700 1 -- 172.21.15.12:0/4009328318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278100790 msgr2=0x7f0278197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f0276d9d700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278100790 0x7f0278197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f0276d9d700 1 -- 172.21.15.12:0/4009328318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0270009c40 con 0x7f0278105e90 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f027659c700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02780ffdd0 0x7f0278197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f0276d9d700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278105e90 0x7f027819bc00 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f027000c0f0 tx=0x7f027000c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f02677fe700 1 -- 172.21.15.12:0/4009328318 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270003a00 con 0x7f0278105e90 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f027cc66700 1 -- 172.21.15.12:0/4009328318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02680095f0 con 0x7f0278105e90 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f02677fe700 1 -- 172.21.15.12:0/4009328318 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f027000e030 con 0x7f0278105e90 2024-05-09T00:06:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f02677fe700 1 -- 172.21.15.12:0/4009328318 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270004650 con 0x7f0278105e90 2024-05-09T00:06:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.584+0000 7f027cc66700 1 -- 172.21.15.12:0/4009328318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c009520 con 0x7f0278105e90 2024-05-09T00:06:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.587+0000 7f02677fe700 1 -- 172.21.15.12:0/4009328318 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0270003cf0 con 0x7f0278105e90 2024-05-09T00:06:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.588+0000 7f02677fe700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02600a1290 0x7f02600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.588+0000 7f027659c700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02600a1290 0x7f02600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.588+0000 7f02677fe700 1 -- 172.21.15.12:0/4009328318 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f02700ccbf0 con 0x7f0278105e90 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.588+0000 7f0288e20b80 1 Processor -- start 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f027659c700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02600a1290 0x7f02600a3750 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0268005fd0 tx=0x7f0268009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0288e20b80 1 -- start start 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 0x5650812d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812cc480 0x5650812cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0288e20b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650812230f0 con 0x5650812d2270 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0288e20b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x565081220ab0 con 0x5650812d1950 2024-05-09T00:06:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0288e20b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56508121c3a0 con 0x5650812cc480 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0275d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0275d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45210/0 (socket says 172.21.15.12:45210) 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0275d9b700 1 -- 172.21.15.12:0/747306018 learned_addr learned my addr 172.21.15.12:0/747306018 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f027659c700 1 --2- 172.21.15.12:0/747306018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 0x5650812d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.589+0000 7f0276d9d700 1 --2- 172.21.15.12:0/747306018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812cc480 0x5650812cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.590+0000 7f0275d9b700 1 -- 172.21.15.12:0/747306018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812cc480 msgr2=0x5650812cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.590+0000 7f0275d9b700 1 --2- 172.21.15.12:0/747306018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812cc480 0x5650812cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.590+0000 7f0275d9b700 1 -- 172.21.15.12:0/747306018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 msgr2=0x5650812d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.590+0000 7f0275d9b700 1 --2- 172.21.15.12:0/747306018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 0x5650812d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.590+0000 7f0275d9b700 1 -- 172.21.15.12:0/747306018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56508109d350 con 0x5650812d2270 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.590+0000 7f027659c700 1 --2- 172.21.15.12:0/747306018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 0x5650812d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.590+0000 7f0275d9b700 1 --2- 172.21.15.12:0/747306018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812cbf40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f026c009230 tx=0x7f026c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a859b9d0940af3d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.590+0000 7f0264ff9700 1 -- 172.21.15.12:0/747306018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c00fdd0 con 0x5650812d2270 2024-05-09T00:06:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.590+0000 7f0264ff9700 1 -- 172.21.15.12:0/747306018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f026c01d040 con 0x5650812d2270 2024-05-09T00:06:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.590+0000 7f0264ff9700 1 -- 172.21.15.12:0/747306018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c0052a0 con 0x5650812d2270 2024-05-09T00:06:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.591+0000 7f0288e20b80 1 -- 172.21.15.12:0/747306018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 msgr2=0x5650812cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.591+0000 7f0288e20b80 1 --2- 172.21.15.12:0/747306018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812cbf40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f026c009230 tx=0x7f026c005dc0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.591+0000 7f0288e20b80 1 -- 172.21.15.12:0/747306018 shutdown_connections 2024-05-09T00:06:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.591+0000 7f0288e20b80 1 --2- 172.21.15.12:0/747306018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812cc480 0x5650812cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.591+0000 7f0288e20b80 1 --2- 172.21.15.12:0/747306018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 0x5650812d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.591+0000 7f0288e20b80 1 --2- 172.21.15.12:0/747306018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812cbf40 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.591+0000 7f0288e20b80 1 -- 172.21.15.12:0/747306018 >> 172.21.15.12:0/747306018 conn(0x56508122a1b0 msgr2=0x5650812c6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.591+0000 7f0288e20b80 1 -- 172.21.15.12:0/747306018 shutdown_connections 2024-05-09T00:06:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.591+0000 7f0288e20b80 1 -- 172.21.15.12:0/747306018 wait complete. 2024-05-09T00:06:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.591+0000 7f0288e20b80 1 -- start start 2024-05-09T00:06:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0288e20b80 1 Processor -- start 2024-05-09T00:06:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812cc480 0x5650812d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d1950 0x5650812d5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812d5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0288e20b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650811b8500 con 0x5650812d2270 2024-05-09T00:06:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0288e20b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650811b81a0 con 0x5650812cc480 2024-05-09T00:06:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0288e20b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56508109f640 con 0x5650812d1950 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0288e20b80 10 client.0 _ll_register_callbacks cb 0x5650811c1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812d5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0275d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d1950 0x5650812d5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812d5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45222/0 (socket says 172.21.15.12:45222) 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0275d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d1950 0x5650812d5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40248/0 (socket says 172.21.15.12:40248) 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0276d9d700 1 -- 172.21.15.12:0/1998669035 learned_addr learned my addr 172.21.15.12:0/1998669035 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f027659c700 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812cc480 0x5650812d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0276d9d700 1 -- 172.21.15.12:0/1998669035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d1950 msgr2=0x5650812d5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.592+0000 7f0276d9d700 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d1950 0x5650812d5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f0276d9d700 1 -- 172.21.15.12:0/1998669035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812cc480 msgr2=0x5650812d82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f0276d9d700 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812cc480 0x5650812d82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f0276d9d700 1 -- 172.21.15.12:0/1998669035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0268017070 con 0x5650812d2270 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f0275d9b700 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d1950 0x5650812d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f0276d9d700 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812d5cd0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f0270096790 tx=0x7f027009a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6c94a05d726bdc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f025dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f025dffb700 1 -- 172.21.15.12:0/1998669035 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027009a180 con 0x5650812d2270 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f025dffb700 1 -- 172.21.15.12:0/1998669035 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f02700a2110 con 0x5650812d2270 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f0288e20b80 1 -- 172.21.15.12:0/1998669035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565081175210 con 0x5650812d2270 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f025dffb700 1 -- 172.21.15.12:0/1998669035 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0270099210 con 0x5650812d2270 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f0288e20b80 10 client.20874 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f0288e20b80 1 -- 172.21.15.12:0/1998669035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7f027810d520 con 0x5650812d2270 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f025dffb700 1 -- 172.21.15.12:0/1998669035 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565081175210 con 0x5650812d2270 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.593+0000 7f025dffb700 1 -- 172.21.15.12:0/1998669035 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f02700adb60 con 0x5650812d2270 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.594+0000 7f0288e20b80 10 client.20874 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.594+0000 7f0288e20b80 10 client.20874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.594+0000 7f0288e20b80 20 client.20874 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.594+0000 7f0288e20b80 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5650812e5b80 0x5650812e7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.594+0000 7f0288e20b80 4 client.20874 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.594+0000 7f0288e20b80 1 -- 172.21.15.12:0/1998669035 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56508129b7b0 con 0x5650812e5b80 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.594+0000 7f027659c700 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5650812e5b80 0x5650812e7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.594+0000 7f027659c700 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5650812e5b80 0x5650812e7f70 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d6623f5a41597b server_cookie=30e311fbd3d7dc4a in_seq=0 out_seq=0 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.594+0000 7f025dffb700 10 client.20874 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f025dffb700 1 -- 172.21.15.12:0/1998669035 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56508129b7b0 con 0x5650812e5b80 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f025dffb700 10 client.20874 handle_command_reply: tid=0 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f0288e20b80 1 client.20874 shutdown 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f0288e20b80 1 -- 172.21.15.12:0/1998669035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 msgr2=0x5650812d5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f0288e20b80 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812d5cd0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f0270096790 tx=0x7f027009a510 comp rx=0 tx=0).stop 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f0288e20b80 1 -- 172.21.15.12:0/1998669035 shutdown_connections 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f0288e20b80 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d2270 0x5650812d5cd0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f0288e20b80 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5650812e5b80 0x5650812e7f70 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f0288e20b80 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812cc480 0x5650812d82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f0288e20b80 1 --2- 172.21.15.12:0/1998669035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d1950 0x5650812d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f025dffb700 0 client.20874 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.595+0000 7f0288e20b80 1 -- 172.21.15.12:0/1998669035 >> 172.21.15.12:0/1998669035 conn(0x56508122a1b0 msgr2=0x5650812d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.596+0000 7f0288e20b80 1 -- 172.21.15.12:0/1998669035 shutdown_connections 2024-05-09T00:06:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.596+0000 7f0288e20b80 1 -- 172.21.15.12:0/1998669035 wait complete. 2024-05-09T00:06:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.596+0000 7f0288e20b80 20 client.20874 trim_cache size 0 max 16384 2024-05-09T00:06:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.612+0000 7f0288e20b80 1 Processor -- start 2024-05-09T00:06:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.612+0000 7f0288e20b80 1 -- start start 2024-05-09T00:06:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.612+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812cc480 0x5650812d5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.612+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d1950 0x5650812c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.612+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d2270 0x5650812e1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.612+0000 7f0288e20b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650812d15e0 con 0x5650812d1950 2024-05-09T00:06:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.612+0000 7f0288e20b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650812d6b90 con 0x5650812d2270 2024-05-09T00:06:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.612+0000 7f0288e20b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650812c75f0 con 0x5650812cc480 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.612+0000 7f0275d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d1950 0x5650812c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f027659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812cc480 0x5650812d5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f0275d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d1950 0x5650812c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45232/0 (socket says 172.21.15.12:45232) 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f0275d9b700 1 -- 172.21.15.12:0/3380003187 learned_addr learned my addr 172.21.15.12:0/3380003187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f0276d9d700 1 --2- 172.21.15.12:0/3380003187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d2270 0x5650812e1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f0275d9b700 1 -- 172.21.15.12:0/3380003187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812cc480 msgr2=0x5650812d5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f0275d9b700 1 --2- 172.21.15.12:0/3380003187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812cc480 0x5650812d5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f0275d9b700 1 -- 172.21.15.12:0/3380003187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d2270 msgr2=0x5650812e1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f0275d9b700 1 --2- 172.21.15.12:0/3380003187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d2270 0x5650812e1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f0275d9b700 1 -- 172.21.15.12:0/3380003187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565081049b00 con 0x5650812d1950 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f0275d9b700 1 --2- 172.21.15.12:0/3380003187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d1950 0x5650812c6910 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f026c00c010 tx=0x7f026c005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0e4caf7963bb73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f02657fa700 1 -- 172.21.15.12:0/3380003187 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c00fe90 con 0x5650812d1950 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f02657fa700 1 -- 172.21.15.12:0/3380003187 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f026c00f460 con 0x5650812d1950 2024-05-09T00:06:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.613+0000 7f02657fa700 1 -- 172.21.15.12:0/3380003187 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c005600 con 0x5650812d1950 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.614+0000 7f0288e20b80 1 -- 172.21.15.12:0/3380003187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d1950 msgr2=0x5650812c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.614+0000 7f0288e20b80 1 --2- 172.21.15.12:0/3380003187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d1950 0x5650812c6910 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f026c00c010 tx=0x7f026c005fa0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.614+0000 7f0288e20b80 1 -- 172.21.15.12:0/3380003187 shutdown_connections 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.614+0000 7f0288e20b80 1 --2- 172.21.15.12:0/3380003187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d2270 0x5650812e1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.614+0000 7f0288e20b80 1 --2- 172.21.15.12:0/3380003187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812cc480 0x5650812d5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.614+0000 7f0288e20b80 1 --2- 172.21.15.12:0/3380003187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812d1950 0x5650812c6910 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.614+0000 7f0288e20b80 1 -- 172.21.15.12:0/3380003187 >> 172.21.15.12:0/3380003187 conn(0x56508122a1b0 msgr2=0x5650812d4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.614+0000 7f0288e20b80 1 -- 172.21.15.12:0/3380003187 shutdown_connections 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.614+0000 7f0288e20b80 1 -- 172.21.15.12:0/3380003187 wait complete. 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.615+0000 7f0288e20b80 1 -- start start 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.615+0000 7f0288e20b80 1 Processor -- start 2024-05-09T00:06:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.615+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812cc480 0x5650812dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.615+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 0x5650812dab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.615+0000 7f0288e20b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d2270 0x5650812db0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.615+0000 7f0288e20b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650812db610 con 0x5650812cc480 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.615+0000 7f0288e20b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650812db780 con 0x5650812d1950 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.615+0000 7f0288e20b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650812db940 con 0x5650812d2270 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.615+0000 7f0288e20b80 10 client.0 _ll_register_callbacks cb 0x5650811c1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d2270 0x5650812db0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d2270 0x5650812db0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40268/0 (socket says 172.21.15.12:40268) 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0276d9d700 1 -- 172.21.15.12:0/2718088994 learned_addr learned my addr 172.21.15.12:0/2718088994 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0275d9b700 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 0x5650812dab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0276d9d700 1 -- 172.21.15.12:0/2718088994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 msgr2=0x5650812dab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0276d9d700 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 0x5650812dab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0276d9d700 1 -- 172.21.15.12:0/2718088994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812cc480 msgr2=0x5650812dcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0276d9d700 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812cc480 0x5650812dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0276d9d700 1 -- 172.21.15.12:0/2718088994 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56508109d350 con 0x5650812d2270 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0276d9d700 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d2270 0x5650812db0d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f02700adf10 tx=0x7f0270096110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd307bda1f67bdfa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f025dffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f025dffb700 1 -- 172.21.15.12:0/2718088994 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02700b8060 con 0x5650812d2270 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f0288e20b80 1 -- 172.21.15.12:0/2718088994 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f026c01d040 con 0x5650812d2270 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f025dffb700 1 -- 172.21.15.12:0/2718088994 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f02700a2110 con 0x5650812d2270 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.616+0000 7f025dffb700 1 -- 172.21.15.12:0/2718088994 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0270099af0 con 0x5650812d2270 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.617+0000 7f0288e20b80 10 client.30583 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.617+0000 7f0288e20b80 1 -- 172.21.15.12:0/2718088994 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5650811e2500 con 0x5650812d2270 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.617+0000 7f025dffb700 1 -- 172.21.15.12:0/2718088994 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f026c01d040 con 0x5650812d2270 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.617+0000 7f025dffb700 1 -- 172.21.15.12:0/2718088994 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0270099210 con 0x5650812d2270 2024-05-09T00:06:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.617+0000 7f0288e20b80 10 client.30583 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.617+0000 7f0288e20b80 10 client.30583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.617+0000 7f0288e20b80 20 client.30583 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.617+0000 7f0288e20b80 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56508133b260 0x56508133d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.617+0000 7f0288e20b80 4 client.30583 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:06:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.617+0000 7f0288e20b80 1 -- 172.21.15.12:0/2718088994 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565081214e60 con 0x56508133b260 2024-05-09T00:06:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.618+0000 7f027659c700 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56508133b260 0x56508133d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.619+0000 7f025dffb700 10 client.30583 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.619+0000 7f027659c700 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56508133b260 0x56508133d680 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd38b9b159a6074 server_cookie=615baf978f5cf467 in_seq=0 out_seq=0 2024-05-09T00:06:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.979+0000 7f025dffb700 1 -- 172.21.15.12:0/2718088994 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565081214e60 con 0x56508133b260 2024-05-09T00:06:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.979+0000 7f025dffb700 10 client.30583 handle_command_reply: tid=0 2024-05-09T00:06:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.979+0000 7f0288e20b80 1 client.30583 shutdown 2024-05-09T00:06:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.979+0000 7f0288e20b80 1 -- 172.21.15.12:0/2718088994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d2270 msgr2=0x5650812db0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.979+0000 7f0288e20b80 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d2270 0x5650812db0d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f02700adf10 tx=0x7f0270096110 comp rx=0 tx=0).stop 2024-05-09T00:06:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.983+0000 7f0288e20b80 1 -- 172.21.15.12:0/2718088994 shutdown_connections 2024-05-09T00:06:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.983+0000 7f0288e20b80 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650812d2270 0x5650812db0d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.983+0000 7f0288e20b80 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650812cc480 0x5650812dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.983+0000 7f0288e20b80 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56508133b260 0x56508133d680 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.983+0000 7f0288e20b80 1 --2- 172.21.15.12:0/2718088994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5650812d1950 0x5650812dab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.983+0000 7f0288e20b80 1 -- 172.21.15.12:0/2718088994 >> 172.21.15.12:0/2718088994 conn(0x56508122a1b0 msgr2=0x5650812cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.983+0000 7f025dffb700 0 client.30583 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.984+0000 7f0288e20b80 1 -- 172.21.15.12:0/2718088994 shutdown_connections 2024-05-09T00:06:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.984+0000 7f0288e20b80 1 -- 172.21.15.12:0/2718088994 wait complete. 2024-05-09T00:06:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.984+0000 7f0288e20b80 20 client.30583 trim_cache size 0 max 16384 2024-05-09T00:06:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:06:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:06:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:06:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:06:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.985+0000 7f027cc66700 1 -- 172.21.15.12:0/4009328318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02600a1290 msgr2=0x7f02600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.985+0000 7f027cc66700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02600a1290 0x7f02600a3750 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0268005fd0 tx=0x7f0268009310 comp rx=0 tx=0).stop 2024-05-09T00:06:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.985+0000 7f027cc66700 1 -- 172.21.15.12:0/4009328318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278105e90 msgr2=0x7f027819bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.985+0000 7f027cc66700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278105e90 0x7f027819bc00 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f027000c0f0 tx=0x7f027000c400 comp rx=0 tx=0).stop 2024-05-09T00:06:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.985+0000 7f027cc66700 1 -- 172.21.15.12:0/4009328318 shutdown_connections 2024-05-09T00:06:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.985+0000 7f027cc66700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f02600a1290 0x7f02600a3750 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.985+0000 7f027cc66700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278105e90 0x7f027819bc00 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.985+0000 7f027cc66700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02780ffdd0 0x7f0278197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.985+0000 7f027cc66700 1 --2- 172.21.15.12:0/4009328318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278100790 0x7f0278197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.985+0000 7f027cc66700 1 -- 172.21.15.12:0/4009328318 >> 172.21.15.12:0/4009328318 conn(0x7f0278076fd0 msgr2=0x7f02780698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.986+0000 7f027cc66700 1 -- 172.21.15.12:0/4009328318 shutdown_connections 2024-05-09T00:06:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:13.986+0000 7f027cc66700 1 -- 172.21.15.12:0/4009328318 wait complete. 2024-05-09T00:06:14.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:13 smithi012 ceph-mon[42503]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.6 MiB/s wr, 575 op/s 2024-05-09T00:06:14.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:13 smithi204 ceph-mon[47373]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.6 MiB/s wr, 575 op/s 2024-05-09T00:06:14.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:13 smithi160 ceph-mon[46152]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.6 MiB/s wr, 575 op/s 2024-05-09T00:06:14.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 240409 33.19 MB/sec execute 249 sec latency 316.152 ms 2024-05-09T00:06:15.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 241099 33.24 MB/sec execute 250 sec latency 153.057 ms 2024-05-09T00:06:16.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:15 smithi204 ceph-mon[47373]: pgmap v638: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.3 MiB/s wr, 506 op/s 2024-05-09T00:06:16.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:15 smithi160 ceph-mon[46152]: pgmap v638: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.3 MiB/s wr, 506 op/s 2024-05-09T00:06:16.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:15 smithi012 ceph-mon[42503]: pgmap v638: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.3 MiB/s wr, 506 op/s 2024-05-09T00:06:16.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 241704 33.24 MB/sec execute 251 sec latency 76.844 ms 2024-05-09T00:06:17.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:16 smithi204 ceph-mon[47373]: osdmap e125: 12 total, 12 up, 12 in 2024-05-09T00:06:17.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:16 smithi160 ceph-mon[46152]: osdmap e125: 12 total, 12 up, 12 in 2024-05-09T00:06:17.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 242144 33.20 MB/sec execute 252 sec latency 222.899 ms 2024-05-09T00:06:17.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:16 smithi012 ceph-mon[42503]: osdmap e125: 12 total, 12 up, 12 in 2024-05-09T00:06:18.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:17 smithi204 ceph-mon[47373]: pgmap v640: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 718 op/s 2024-05-09T00:06:18.264 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:17 smithi160 ceph-mon[46152]: pgmap v640: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 718 op/s 2024-05-09T00:06:18.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 242617 33.18 MB/sec execute 253 sec latency 80.829 ms 2024-05-09T00:06:18.391 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:06:18.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:17 smithi012 ceph-mon[42503]: pgmap v640: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 718 op/s 2024-05-09T00:06:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.558+0000 7fd1795ce700 1 -- 172.21.15.12:0/907599684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1741000f0 msgr2=0x7fd174100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.558+0000 7fd1795ce700 1 --2- 172.21.15.12:0/907599684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1741000f0 0x7fd174100570 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0099c0 tx=0x7fd15c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.558+0000 7fd1795ce700 1 -- 172.21.15.12:0/907599684 shutdown_connections 2024-05-09T00:06:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.558+0000 7fd1795ce700 1 --2- 172.21.15.12:0/907599684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd174100ab0 0x7fd17410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.558+0000 7fd1795ce700 1 --2- 172.21.15.12:0/907599684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd174105f90 0x7fd174106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.558+0000 7fd1795ce700 1 --2- 172.21.15.12:0/907599684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1741000f0 0x7fd174100570 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.558+0000 7fd1795ce700 1 -- 172.21.15.12:0/907599684 >> 172.21.15.12:0/907599684 conn(0x7fd174076ba0 msgr2=0x7fd174076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.559+0000 7fd1795ce700 1 -- 172.21.15.12:0/907599684 shutdown_connections 2024-05-09T00:06:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.559+0000 7fd1795ce700 1 -- 172.21.15.12:0/907599684 wait complete. 2024-05-09T00:06:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.559+0000 7fd1795ce700 1 Processor -- start 2024-05-09T00:06:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.559+0000 7fd1795ce700 1 -- start start 2024-05-09T00:06:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd1795ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1741000f0 0x7fd1741981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd1795ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd174100ab0 0x7fd174198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd1795ce700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd174105f90 0x7fd1741a3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd1795ce700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd174198fb0 con 0x7fd174100ab0 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd1795ce700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1741a37b0 con 0x7fd1741000f0 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd1795ce700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1741a38f0 con 0x7fd174105f90 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd16a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd174100ab0 0x7fd174198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd16a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd174100ab0 0x7fd174198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32996/0 (socket says 172.21.15.12:32996) 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd172ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1741000f0 0x7fd1741981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd16a7fc700 1 -- 172.21.15.12:0/2132806445 learned_addr learned my addr 172.21.15.12:0/2132806445 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd1737fe700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd174105f90 0x7fd1741a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd16a7fc700 1 -- 172.21.15.12:0/2132806445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd174105f90 msgr2=0x7fd1741a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd174105f90 0x7fd1741a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd16a7fc700 1 -- 172.21.15.12:0/2132806445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1741000f0 msgr2=0x7fd1741981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1741000f0 0x7fd1741981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd16a7fc700 1 -- 172.21.15.12:0/2132806445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd164009520 con 0x7fd174100ab0 2024-05-09T00:06:18.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.560+0000 7fd172ffd700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1741000f0 0x7fd1741981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:18.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.561+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd174100ab0 0x7fd174198730 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00f690 tx=0x7fd15c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:18.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.561+0000 7fd170ff9700 1 -- 172.21.15.12:0/2132806445 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c01c070 con 0x7fd174100ab0 2024-05-09T00:06:18.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.561+0000 7fd1795ce700 1 -- 172.21.15.12:0/2132806445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c0095f0 con 0x7fd174100ab0 2024-05-09T00:06:18.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.561+0000 7fd170ff9700 1 -- 172.21.15.12:0/2132806445 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd15c00fcf0 con 0x7fd174100ab0 2024-05-09T00:06:18.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.561+0000 7fd170ff9700 1 -- 172.21.15.12:0/2132806445 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c022c60 con 0x7fd174100ab0 2024-05-09T00:06:18.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.561+0000 7fd1795ce700 1 -- 172.21.15.12:0/2132806445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1741a3de0 con 0x7fd174100ab0 2024-05-09T00:06:18.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.564+0000 7fd170ff9700 1 -- 172.21.15.12:0/2132806445 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd15c020070 con 0x7fd174100ab0 2024-05-09T00:06:18.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.565+0000 7fd170ff9700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1600a1290 0x7fd1600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.565+0000 7fd170ff9700 1 -- 172.21.15.12:0/2132806445 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd15c0cdb60 con 0x7fd174100ab0 2024-05-09T00:06:18.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.565+0000 7fd172ffd700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1600a1290 0x7fd1600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.565+0000 7fd172ffd700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1600a1290 0x7fd1600a3750 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd174067bb0 tx=0x7fd164009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:18.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.566+0000 7fd185788b80 1 Processor -- start 2024-05-09T00:06:18.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.566+0000 7fd185788b80 1 -- start start 2024-05-09T00:06:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.566+0000 7fd185788b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f0de00 0x565191f0e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.566+0000 7fd185788b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f08350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.566+0000 7fd185788b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f08890 0x565191f08cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.566+0000 7fd185788b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191e5f150 con 0x565191f0e720 2024-05-09T00:06:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.566+0000 7fd185788b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191e5cb10 con 0x565191f0de00 2024-05-09T00:06:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.566+0000 7fd185788b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191e58400 con 0x565191f08890 2024-05-09T00:06:18.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.566+0000 7fd16a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f08350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd172ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f0de00 0x565191f0e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd16a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f08350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33002/0 (socket says 172.21.15.12:33002) 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f08890 0x565191f08cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd16a7fc700 1 -- 172.21.15.12:0/2922360488 learned_addr learned my addr 172.21.15.12:0/2922360488 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd172ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f0de00 0x565191f0e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58126/0 (socket says 172.21.15.12:58126) 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd16a7fc700 1 -- 172.21.15.12:0/2922360488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f08890 msgr2=0x565191f08cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/2922360488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f08890 0x565191f08cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd16a7fc700 1 -- 172.21.15.12:0/2922360488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f0de00 msgr2=0x565191f0e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/2922360488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f0de00 0x565191f0e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd16a7fc700 1 -- 172.21.15.12:0/2922360488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565191cd93b0 con 0x565191f0e720 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/2922360488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f08350 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fd15c09aca0 tx=0x7fd15c09a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67ec0a0fd39d01b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd1697fa700 1 -- 172.21.15.12:0/2922360488 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c09ad10 con 0x565191f0e720 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd1697fa700 1 -- 172.21.15.12:0/2922360488 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd15c09eb40 con 0x565191f0e720 2024-05-09T00:06:18.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.567+0000 7fd1697fa700 1 -- 172.21.15.12:0/2922360488 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c09a200 con 0x565191f0e720 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.568+0000 7fd185788b80 1 -- 172.21.15.12:0/2922360488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 msgr2=0x565191f08350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.568+0000 7fd185788b80 1 --2- 172.21.15.12:0/2922360488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f08350 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fd15c09aca0 tx=0x7fd15c09a850 comp rx=0 tx=0).stop 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.568+0000 7fd185788b80 1 -- 172.21.15.12:0/2922360488 shutdown_connections 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.568+0000 7fd185788b80 1 --2- 172.21.15.12:0/2922360488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f08890 0x565191f08cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.568+0000 7fd185788b80 1 --2- 172.21.15.12:0/2922360488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f0de00 0x565191f0e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.568+0000 7fd185788b80 1 --2- 172.21.15.12:0/2922360488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f08350 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.568+0000 7fd185788b80 1 -- 172.21.15.12:0/2922360488 >> 172.21.15.12:0/2922360488 conn(0x565191e66210 msgr2=0x565191f02df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.568+0000 7fd185788b80 1 -- 172.21.15.12:0/2922360488 shutdown_connections 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.568+0000 7fd185788b80 1 -- 172.21.15.12:0/2922360488 wait complete. 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd185788b80 1 -- start start 2024-05-09T00:06:18.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd185788b80 1 Processor -- start 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd185788b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f14750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd185788b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0de00 0x565191f11a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd185788b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd185788b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191df4560 con 0x565191f0e720 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd185788b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191df4200 con 0x565191f08890 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd185788b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191cdb6a0 con 0x565191f0de00 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd185788b80 10 client.0 _ll_register_callbacks cb 0x565191dfdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd172ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f14750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33008/0 (socket says 172.21.15.12:33008) 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd1737fe700 1 -- 172.21.15.12:0/129776646 learned_addr learned my addr 172.21.15.12:0/129776646 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:18.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.569+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0de00 0x565191f11a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd1737fe700 1 -- 172.21.15.12:0/129776646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0de00 msgr2=0x565191f11a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd1737fe700 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0de00 0x565191f11a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd1737fe700 1 -- 172.21.15.12:0/129776646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 msgr2=0x565191f14750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd1737fe700 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f14750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd1737fe700 1 -- 172.21.15.12:0/129776646 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1741a3de0 con 0x565191f0e720 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd1737fe700 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f120e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd16c011f60 tx=0x7fd16c014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544da1ee984f3783 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd15a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd15a7fc700 1 -- 172.21.15.12:0/129776646 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00e040 con 0x565191f0e720 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd15a7fc700 1 -- 172.21.15.12:0/129776646 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd16c014b40 con 0x565191f0e720 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd185788b80 1 -- 172.21.15.12:0/129776646 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565191db1270 con 0x565191f0e720 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd15a7fc700 1 -- 172.21.15.12:0/129776646 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd16c014d00 con 0x565191f0e720 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd185788b80 10 client.20889 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:18.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd185788b80 1 -- 172.21.15.12:0/129776646 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x565191c85b60 con 0x565191f0e720 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.570+0000 7fd15a7fc700 1 -- 172.21.15.12:0/129776646 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565191db1270 con 0x565191f0e720 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.571+0000 7fd15a7fc700 1 -- 172.21.15.12:0/129776646 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd16c025070 con 0x565191f0e720 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.571+0000 7fd185788b80 10 client.20889 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.571+0000 7fd185788b80 10 client.20889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.571+0000 7fd185788b80 20 client.20889 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.571+0000 7fd185788b80 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x565191f21e90 0x565191f242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.571+0000 7fd185788b80 4 client.20889 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.571+0000 7fd185788b80 1 -- 172.21.15.12:0/129776646 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565191ed7810 con 0x565191f21e90 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.571+0000 7fd172ffd700 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x565191f21e90 0x565191f242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.571+0000 7fd15a7fc700 10 client.20889 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:18.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.571+0000 7fd172ffd700 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x565191f21e90 0x565191f242b0 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b4360527bd7b91 server_cookie=b159c9c636376857 in_seq=0 out_seq=0 2024-05-09T00:06:18.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd15a7fc700 1 -- 172.21.15.12:0/129776646 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565191ed7810 con 0x565191f21e90 2024-05-09T00:06:18.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd15a7fc700 10 client.20889 handle_command_reply: tid=0 2024-05-09T00:06:18.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 client.20889 shutdown 2024-05-09T00:06:18.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 -- 172.21.15.12:0/129776646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 msgr2=0x565191f120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f120e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd16c011f60 tx=0x7fd16c014600 comp rx=0 tx=0).stop 2024-05-09T00:06:18.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 -- 172.21.15.12:0/129776646 shutdown_connections 2024-05-09T00:06:18.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f120e0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x565191f21e90 0x565191f242b0 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f14750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 --2- 172.21.15.12:0/129776646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0de00 0x565191f11a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd15a7fc700 0 client.20889 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:18.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 -- 172.21.15.12:0/129776646 >> 172.21.15.12:0/129776646 conn(0x565191e66210 msgr2=0x565191f025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:18.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 -- 172.21.15.12:0/129776646 shutdown_connections 2024-05-09T00:06:18.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 1 -- 172.21.15.12:0/129776646 wait complete. 2024-05-09T00:06:18.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.577+0000 7fd185788b80 20 client.20889 trim_cache size 0 max 16384 2024-05-09T00:06:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.598+0000 7fd185788b80 1 Processor -- start 2024-05-09T00:06:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.598+0000 7fd185788b80 1 -- start start 2024-05-09T00:06:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.598+0000 7fd185788b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f08350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.598+0000 7fd185788b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0de00 0x565191f02d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.598+0000 7fd185788b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f10310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.598+0000 7fd185788b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191efd800 con 0x565191f0e720 2024-05-09T00:06:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.598+0000 7fd185788b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191f126c0 con 0x565191f08890 2024-05-09T00:06:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.598+0000 7fd185788b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191f0ff90 con 0x565191f0de00 2024-05-09T00:06:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.598+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f10310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f10310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33012/0 (socket says 172.21.15.12:33012) 2024-05-09T00:06:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd1737fe700 1 -- 172.21.15.12:0/3990384435 learned_addr learned my addr 172.21.15.12:0/3990384435 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:18.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/3990384435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0de00 0x565191f02d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd172ffd700 1 --2- 172.21.15.12:0/3990384435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f08350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd1737fe700 1 -- 172.21.15.12:0/3990384435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0de00 msgr2=0x565191f02d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd1737fe700 1 --2- 172.21.15.12:0/3990384435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0de00 0x565191f02d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd1737fe700 1 -- 172.21.15.12:0/3990384435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 msgr2=0x565191f08350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd1737fe700 1 --2- 172.21.15.12:0/3990384435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f08350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd1737fe700 1 -- 172.21.15.12:0/3990384435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565191f08f40 con 0x565191f0e720 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd172ffd700 1 --2- 172.21.15.12:0/3990384435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f08350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd1737fe700 1 --2- 172.21.15.12:0/3990384435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f10310 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fd16c014950 tx=0x7fd16c014980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a5d006f1646d1a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd169ffb700 1 -- 172.21.15.12:0/3990384435 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00e070 con 0x565191f0e720 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd169ffb700 1 -- 172.21.15.12:0/3990384435 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd16c00f040 con 0x565191f0e720 2024-05-09T00:06:18.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.599+0000 7fd169ffb700 1 -- 172.21.15.12:0/3990384435 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c007a80 con 0x565191f0e720 2024-05-09T00:06:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.600+0000 7fd185788b80 1 -- 172.21.15.12:0/3990384435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 msgr2=0x565191f10310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.600+0000 7fd185788b80 1 --2- 172.21.15.12:0/3990384435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f10310 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fd16c014950 tx=0x7fd16c014980 comp rx=0 tx=0).stop 2024-05-09T00:06:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.600+0000 7fd185788b80 1 -- 172.21.15.12:0/3990384435 shutdown_connections 2024-05-09T00:06:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.600+0000 7fd185788b80 1 --2- 172.21.15.12:0/3990384435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0e720 0x565191f10310 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.600+0000 7fd185788b80 1 --2- 172.21.15.12:0/3990384435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f08350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.600+0000 7fd185788b80 1 --2- 172.21.15.12:0/3990384435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0de00 0x565191f02d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.600+0000 7fd185788b80 1 -- 172.21.15.12:0/3990384435 >> 172.21.15.12:0/3990384435 conn(0x565191e66210 msgr2=0x565191f1d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.600+0000 7fd185788b80 1 -- 172.21.15.12:0/3990384435 shutdown_connections 2024-05-09T00:06:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.600+0000 7fd185788b80 1 -- 172.21.15.12:0/3990384435 wait complete. 2024-05-09T00:06:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd185788b80 1 -- start start 2024-05-09T00:06:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd185788b80 1 Processor -- start 2024-05-09T00:06:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd185788b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd185788b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0de00 0x565191f71b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd185788b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0e720 0x565191f15240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd185788b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191f15780 con 0x565191f0de00 2024-05-09T00:06:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd185788b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191f158f0 con 0x565191f08890 2024-05-09T00:06:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd185788b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565191f15ab0 con 0x565191f0e720 2024-05-09T00:06:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd185788b80 10 client.0 _ll_register_callbacks cb 0x565191dfdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd16a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0de00 0x565191f71b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd16a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0de00 0x565191f71b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33020/0 (socket says 172.21.15.12:33020) 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd16a7fc700 1 -- 172.21.15.12:0/3485273112 learned_addr learned my addr 172.21.15.12:0/3485273112 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd172ffd700 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.601+0000 7fd1737fe700 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0e720 0x565191f15240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd16a7fc700 1 -- 172.21.15.12:0/3485273112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0e720 msgr2=0x565191f15240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0e720 0x565191f15240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd16a7fc700 1 -- 172.21.15.12:0/3485273112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 msgr2=0x565191f715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd16a7fc700 1 -- 172.21.15.12:0/3485273112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565191c85b60 con 0x565191f0de00 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd1737fe700 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0e720 0x565191f15240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd16a7fc700 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0de00 0x565191f71b10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0cb370 tx=0x7fd15c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb77970ae2df17aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd15a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd15a7fc700 1 -- 172.21.15.12:0/3485273112 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0a3100 con 0x565191f0de00 2024-05-09T00:06:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd185788b80 1 -- 172.21.15.12:0/3485273112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd15c09eb40 con 0x565191f0de00 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd15a7fc700 1 -- 172.21.15.12:0/3485273112 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd15c09ad10 con 0x565191f0de00 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd15a7fc700 1 -- 172.21.15.12:0/3485273112 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd15c09af70 con 0x565191f0de00 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd185788b80 10 client.20895 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd185788b80 1 -- 172.21.15.12:0/3485273112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x565191e1e560 con 0x565191f0de00 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd15a7fc700 1 -- 172.21.15.12:0/3485273112 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd15c067080 con 0x565191f0de00 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.602+0000 7fd15a7fc700 1 -- 172.21.15.12:0/3485273112 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd15c09bd20 con 0x565191f0de00 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd185788b80 10 client.20895 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd185788b80 10 client.20895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd185788b80 20 client.20895 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd185788b80 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x565191f79070 0x565191f7b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd185788b80 4 client.20895 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd185788b80 1 -- 172.21.15.12:0/3485273112 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565191e50ec0 con 0x565191f79070 2024-05-09T00:06:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd172ffd700 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x565191f79070 0x565191f7b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd172ffd700 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x565191f79070 0x565191f7b490 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a71609309327673 server_cookie=64fe7466e71fc8f7 in_seq=0 out_seq=0 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd15a7fc700 10 client.20895 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd15a7fc700 1 -- 172.21.15.12:0/3485273112 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x565191e50ec0 con 0x565191f79070 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd15a7fc700 10 client.20895 handle_command_reply: tid=0 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.603+0000 7fd185788b80 1 client.20895 shutdown 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 1 -- 172.21.15.12:0/3485273112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0de00 msgr2=0x565191f71b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0de00 0x565191f71b10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0cb370 tx=0x7fd15c009310 comp rx=0 tx=0).stop 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 1 -- 172.21.15.12:0/3485273112 shutdown_connections 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565191f0e720 0x565191f15240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x565191f79070 0x565191f7b490 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565191f08890 0x565191f715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 1 --2- 172.21.15.12:0/3485273112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565191f0de00 0x565191f71b10 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd15a7fc700 0 client.20895 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 1 -- 172.21.15.12:0/3485273112 >> 172.21.15.12:0/3485273112 conn(0x565191e66210 msgr2=0x565191f100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:18.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 1 -- 172.21.15.12:0/3485273112 shutdown_connections 2024-05-09T00:06:18.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 1 -- 172.21.15.12:0/3485273112 wait complete. 2024-05-09T00:06:18.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd185788b80 20 client.20895 trim_cache size 0 max 16384 2024-05-09T00:06:18.606 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:06:18.606 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:06:18.606 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:06:18.606 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:06:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd1795ce700 1 -- 172.21.15.12:0/2132806445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1600a1290 msgr2=0x7fd1600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.604+0000 7fd1795ce700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1600a1290 0x7fd1600a3750 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd174067bb0 tx=0x7fd164009260 comp rx=0 tx=0).stop 2024-05-09T00:06:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.605+0000 7fd1795ce700 1 -- 172.21.15.12:0/2132806445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd174100ab0 msgr2=0x7fd174198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.605+0000 7fd1795ce700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd174100ab0 0x7fd174198730 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00f690 tx=0x7fd15c00bfd0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.605+0000 7fd1795ce700 1 -- 172.21.15.12:0/2132806445 shutdown_connections 2024-05-09T00:06:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.605+0000 7fd1795ce700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd1600a1290 0x7fd1600a3750 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.605+0000 7fd1795ce700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd174105f90 0x7fd1741a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.605+0000 7fd1795ce700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1741000f0 0x7fd1741981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.605+0000 7fd1795ce700 1 --2- 172.21.15.12:0/2132806445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd174100ab0 0x7fd174198730 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.605+0000 7fd1795ce700 1 -- 172.21.15.12:0/2132806445 >> 172.21.15.12:0/2132806445 conn(0x7fd174076ba0 msgr2=0x7fd1740fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.605+0000 7fd1795ce700 1 -- 172.21.15.12:0/2132806445 shutdown_connections 2024-05-09T00:06:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.605+0000 7fd1795ce700 1 -- 172.21.15.12:0/2132806445 wait complete. 2024-05-09T00:06:18.614 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:06:18.614 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.811+0000 7f806a50a700 1 -- 172.21.15.12:0/3316900445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8064101fa0 msgr2=0x7f8064102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.811+0000 7f806a50a700 1 --2- 172.21.15.12:0/3316900445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8064101fa0 0x7f8064102380 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8054009990 tx=0x7f8054009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.812+0000 7f806a50a700 1 -- 172.21.15.12:0/3316900445 shutdown_connections 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.812+0000 7f806a50a700 1 --2- 172.21.15.12:0/3316900445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806410b3c0 0x7f806410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.812+0000 7f806a50a700 1 --2- 172.21.15.12:0/3316900445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8064101fa0 0x7f8064102380 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.812+0000 7f806a50a700 1 --2- 172.21.15.12:0/3316900445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80641028c0 0x7f806410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.812+0000 7f806a50a700 1 -- 172.21.15.12:0/3316900445 >> 172.21.15.12:0/3316900445 conn(0x7f80640fb7e0 msgr2=0x7f80640fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.812+0000 7f806a50a700 1 -- 172.21.15.12:0/3316900445 shutdown_connections 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.812+0000 7f806a50a700 1 -- 172.21.15.12:0/3316900445 wait complete. 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.812+0000 7f806a50a700 1 Processor -- start 2024-05-09T00:06:18.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.812+0000 7f806a50a700 1 -- start start 2024-05-09T00:06:18.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f806a50a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8064101fa0 0x7f80641972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f806a50a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80641028c0 0x7f8064197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f806a50a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f806410b3c0 0x7f806419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f806a50a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806419c210 con 0x7f80641028c0 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f806a50a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806419c380 con 0x7f8064101fa0 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f806a50a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806419c540 con 0x7f806410b3c0 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f80637fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80641028c0 0x7f8064197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f80637fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80641028c0 0x7f8064197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33034/0 (socket says 172.21.15.12:33034) 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f80637fe700 1 -- 172.21.15.12:0/308887260 learned_addr learned my addr 172.21.15.12:0/308887260 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f8063fff700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8064101fa0 0x7f80641972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f8068aa7700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f806410b3c0 0x7f806419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f80637fe700 1 -- 172.21.15.12:0/308887260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f806410b3c0 msgr2=0x7f806419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f80637fe700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f806410b3c0 0x7f806419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f80637fe700 1 -- 172.21.15.12:0/308887260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8064101fa0 msgr2=0x7f80641972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f80637fe700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8064101fa0 0x7f80641972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.813+0000 7f80637fe700 1 -- 172.21.15.12:0/308887260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c009c40 con 0x7f80641028c0 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.814+0000 7f80637fe700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80641028c0 0x7f8064197830 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f805800e9f0 tx=0x7f805800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.814+0000 7f80617fa700 1 -- 172.21.15.12:0/308887260 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805800c9c0 con 0x7f80641028c0 2024-05-09T00:06:18.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.814+0000 7f80617fa700 1 -- 172.21.15.12:0/308887260 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8058010430 con 0x7f80641028c0 2024-05-09T00:06:18.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.814+0000 7f806a50a700 1 -- 172.21.15.12:0/308887260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80540095f0 con 0x7f80641028c0 2024-05-09T00:06:18.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.814+0000 7f80617fa700 1 -- 172.21.15.12:0/308887260 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058018680 con 0x7f80641028c0 2024-05-09T00:06:18.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.814+0000 7f806a50a700 1 -- 172.21.15.12:0/308887260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80641a3e80 con 0x7f80641028c0 2024-05-09T00:06:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.815+0000 7f806a50a700 1 -- 172.21.15.12:0/308887260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80641064f0 con 0x7f80641028c0 2024-05-09T00:06:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.817+0000 7f80617fa700 1 -- 172.21.15.12:0/308887260 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8058010860 con 0x7f80641028c0 2024-05-09T00:06:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.817+0000 7f80617fa700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f804c0a1290 0x7f804c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.817+0000 7f80617fa700 1 -- 172.21.15.12:0/308887260 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8058014070 con 0x7f80641028c0 2024-05-09T00:06:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.817+0000 7f8063fff700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f804c0a1290 0x7f804c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:18.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.818+0000 7f8063fff700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f804c0a1290 0x7f804c0a3750 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f805400b2e0 tx=0x7f8054005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:18.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:18.821+0000 7f80617fa700 1 -- 172.21.15.12:0/308887260 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8058096700 con 0x7f80641028c0 2024-05-09T00:06:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.150+0000 7f806a50a700 1 -- 172.21.15.12:0/308887260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80640688d0 con 0x7f80641028c0 2024-05-09T00:06:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.151+0000 7f80617fa700 1 -- 172.21.15.12:0/308887260 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+10875 (secure 0 0 0) 0x7f8058099d20 con 0x7f80641028c0 2024-05-09T00:06:19.145 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:06:19.145 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:06:02.725626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_30395":{"gid":30395,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6833/3969500142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3969500142},{"type":"v1","addr":"172.21.15.160:6833","nonce":3969500142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 -- 172.21.15.12:0/308887260 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f804c0a1290 msgr2=0x7f804c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f804c0a1290 0x7f804c0a3750 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f805400b2e0 tx=0x7f8054005fd0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 -- 172.21.15.12:0/308887260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80641028c0 msgr2=0x7f8064197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80641028c0 0x7f8064197830 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f805800e9f0 tx=0x7f805800edb0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 -- 172.21.15.12:0/308887260 shutdown_connections 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f804c0a1290 0x7f804c0a3750 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f806410b3c0 0x7f806419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8064101fa0 0x7f80641972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 --2- 172.21.15.12:0/308887260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80641028c0 0x7f8064197830 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 -- 172.21.15.12:0/308887260 >> 172.21.15.12:0/308887260 conn(0x7f80640fb7e0 msgr2=0x7f80640fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 -- 172.21.15.12:0/308887260 shutdown_connections 2024-05-09T00:06:19.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.153+0000 7f806a50a700 1 -- 172.21.15.12:0/308887260 wait complete. 2024-05-09T00:06:19.149 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 76 2024-05-09T00:06:19.163 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:06:19.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 243026 33.12 MB/sec execute 254 sec latency 152.881 ms 2024-05-09T00:06:19.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.326+0000 7f673920f700 1 -- 172.21.15.12:0/541454187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6734069400 msgr2=0x7f67340697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.326+0000 7f673920f700 1 --2- 172.21.15.12:0/541454187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6734069400 0x7f67340697e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6724009990 tx=0x7f6724009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.331+0000 7f673920f700 1 -- 172.21.15.12:0/541454187 shutdown_connections 2024-05-09T00:06:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.331+0000 7f673920f700 1 --2- 172.21.15.12:0/541454187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67341062a0 0x7f673410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.331+0000 7f673920f700 1 --2- 172.21.15.12:0/541454187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6734069400 0x7f67340697e0 secure :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6724009990 tx=0x7f6724009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.331+0000 7f673920f700 1 --2- 172.21.15.12:0/541454187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6734069db0 0x7f6734105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.331+0000 7f673920f700 1 -- 172.21.15.12:0/541454187 >> 172.21.15.12:0/541454187 conn(0x7f6734076ba0 msgr2=0x7f6734076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.332+0000 7f673920f700 1 -- 172.21.15.12:0/541454187 shutdown_connections 2024-05-09T00:06:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.332+0000 7f673920f700 1 -- 172.21.15.12:0/541454187 wait complete. 2024-05-09T00:06:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.333+0000 7f673920f700 1 Processor -- start 2024-05-09T00:06:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.333+0000 7f673920f700 1 -- start start 2024-05-09T00:06:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673920f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67341062a0 0x7f673419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673920f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673419fa60 0x7f67341996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673920f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6734199be0 0x7f673419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673920f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67341a0060 con 0x7f6734199be0 2024-05-09T00:06:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673920f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67341a6330 con 0x7f67341062a0 2024-05-09T00:06:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673920f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67341a6470 con 0x7f673419fa60 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673419fa60 0x7f67341996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673419fa60 0x7f67341996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:45964/0 (socket says 172.21.15.12:45964) 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673259c700 1 -- 172.21.15.12:0/4055206848 learned_addr learned my addr 172.21.15.12:0/4055206848 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673359e700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6734199be0 0x7f673419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673259c700 1 -- 172.21.15.12:0/4055206848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67341062a0 msgr2=0x7f673419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673259c700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67341062a0 0x7f673419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673259c700 1 -- 172.21.15.12:0/4055206848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6734199be0 msgr2=0x7f673419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673259c700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6734199be0 0x7f673419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.334+0000 7f673259c700 1 -- 172.21.15.12:0/4055206848 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67240095f0 con 0x7f673419fa60 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.335+0000 7f673259c700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673419fa60 0x7f67341996a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f672800e970 tx=0x7f672800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.335+0000 7f671bfff700 1 -- 172.21.15.12:0/4055206848 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800cbd0 con 0x7f673419fa60 2024-05-09T00:06:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.335+0000 7f673920f700 1 -- 172.21.15.12:0/4055206848 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67341a6750 con 0x7f673419fa60 2024-05-09T00:06:19.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.335+0000 7f671bfff700 1 -- 172.21.15.12:0/4055206848 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f672800eec0 con 0x7f673419fa60 2024-05-09T00:06:19.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.335+0000 7f671bfff700 1 -- 172.21.15.12:0/4055206848 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728005610 con 0x7f673419fa60 2024-05-09T00:06:19.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.335+0000 7f673920f700 1 -- 172.21.15.12:0/4055206848 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67341a6c60 con 0x7f673419fa60 2024-05-09T00:06:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.338+0000 7f671bfff700 1 -- 172.21.15.12:0/4055206848 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6728021440 con 0x7f673419fa60 2024-05-09T00:06:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.339+0000 7f671bfff700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f671c0a1290 0x7f671c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.339+0000 7f671bfff700 1 -- 172.21.15.12:0/4055206848 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6728014070 con 0x7f673419fa60 2024-05-09T00:06:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.339+0000 7f6732d9d700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f671c0a1290 0x7f671c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.340+0000 7f6732d9d700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f671c0a1290 0x7f671c0a3750 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6724006010 tx=0x7f6724005e80 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.340+0000 7f67453c9b80 1 Processor -- start 2024-05-09T00:06:19.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.340+0000 7f67453c9b80 1 -- start start 2024-05-09T00:06:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 0x5633093691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309369720 0x563309363350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309363890 0x563309363cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f67453c9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633092ba150 con 0x563309368e00 2024-05-09T00:06:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f67453c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633092b7b10 con 0x563309369720 2024-05-09T00:06:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f67453c9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633092b3400 con 0x563309363890 2024-05-09T00:06:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f6732d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 0x5633093691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f6732d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 0x5633093691e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33074/0 (socket says 172.21.15.12:33074) 2024-05-09T00:06:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f6732d9d700 1 -- 172.21.15.12:0/2227859980 learned_addr learned my addr 172.21.15.12:0/2227859980 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f6732d9d700 1 -- 172.21.15.12:0/2227859980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309363890 msgr2=0x563309363cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f6732d9d700 1 --2- 172.21.15.12:0/2227859980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309363890 0x563309363cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f6732d9d700 1 -- 172.21.15.12:0/2227859980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309369720 msgr2=0x563309363350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f6732d9d700 1 --2- 172.21.15.12:0/2227859980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309369720 0x563309363350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.341+0000 7f6732d9d700 1 -- 172.21.15.12:0/2227859980 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633091343b0 con 0x563309368e00 2024-05-09T00:06:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.343+0000 7f6732d9d700 1 --2- 172.21.15.12:0/2227859980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 0x5633093691e0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f6724015000 tx=0x7f672401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62556d1aa387c645 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.343+0000 7f67197fa700 1 -- 172.21.15.12:0/2227859980 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724029070 con 0x563309368e00 2024-05-09T00:06:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.343+0000 7f67197fa700 1 -- 172.21.15.12:0/2227859980 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f672400bda0 con 0x563309368e00 2024-05-09T00:06:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.343+0000 7f67197fa700 1 -- 172.21.15.12:0/2227859980 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672401d070 con 0x563309368e00 2024-05-09T00:06:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.345+0000 7f67453c9b80 1 -- 172.21.15.12:0/2227859980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 msgr2=0x5633093691e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.345+0000 7f67453c9b80 1 --2- 172.21.15.12:0/2227859980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 0x5633093691e0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f6724015000 tx=0x7f672401d000 comp rx=0 tx=0).stop 2024-05-09T00:06:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.346+0000 7f67453c9b80 1 -- 172.21.15.12:0/2227859980 shutdown_connections 2024-05-09T00:06:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.346+0000 7f67453c9b80 1 --2- 172.21.15.12:0/2227859980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309363890 0x563309363cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.346+0000 7f67453c9b80 1 --2- 172.21.15.12:0/2227859980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 0x5633093691e0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.346+0000 7f67453c9b80 1 --2- 172.21.15.12:0/2227859980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309369720 0x563309363350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.346+0000 7f67453c9b80 1 -- 172.21.15.12:0/2227859980 >> 172.21.15.12:0/2227859980 conn(0x5633092c1210 msgr2=0x56330935ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.346+0000 7f67453c9b80 1 -- 172.21.15.12:0/2227859980 shutdown_connections 2024-05-09T00:06:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.346+0000 7f67453c9b80 1 -- 172.21.15.12:0/2227859980 wait complete. 2024-05-09T00:06:19.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.346+0000 7f67453c9b80 1 -- start start 2024-05-09T00:06:19.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.346+0000 7f67453c9b80 1 Processor -- start 2024-05-09T00:06:19.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309363890 0x56330936ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309368e00 0x56330936cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 0x56330936d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f67453c9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330924f560 con 0x563309363890 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f67453c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330924f200 con 0x563309368e00 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f67453c9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633091366a0 con 0x563309369720 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f67453c9b80 10 client.0 _ll_register_callbacks cb 0x563309258b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f6732d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309363890 0x56330936ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f6732d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309363890 0x56330936ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33084/0 (socket says 172.21.15.12:33084) 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f6732d9d700 1 -- 172.21.15.12:0/1035322568 learned_addr learned my addr 172.21.15.12:0/1035322568 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.347+0000 7f673359e700 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 0x56330936d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.348+0000 7f6732d9d700 1 -- 172.21.15.12:0/1035322568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 msgr2=0x56330936d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.348+0000 7f6732d9d700 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 0x56330936d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.348+0000 7f6732d9d700 1 -- 172.21.15.12:0/1035322568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309368e00 msgr2=0x56330936cfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.348+0000 7f6732d9d700 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309368e00 0x56330936cfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.348+0000 7f6732d9d700 1 -- 172.21.15.12:0/1035322568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673410e690 con 0x563309363890 2024-05-09T00:06:19.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.348+0000 7f6732d9d700 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309363890 0x56330936ca70 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6724009990 tx=0x7f67240057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dde499d61f46e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.348+0000 7f6716ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:19.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.348+0000 7f6716ff5700 1 -- 172.21.15.12:0/1035322568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724029070 con 0x563309363890 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.348+0000 7f6716ff5700 1 -- 172.21.15.12:0/1035322568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f67240049b0 con 0x563309363890 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.348+0000 7f67453c9b80 1 -- 172.21.15.12:0/1035322568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56330920c270 con 0x563309363890 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f6716ff5700 1 -- 172.21.15.12:0/1035322568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6724038430 con 0x563309363890 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f67453c9b80 10 client.20913 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f67453c9b80 1 -- 172.21.15.12:0/1035322568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5633090e0b60 con 0x563309363890 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f6716ff5700 1 -- 172.21.15.12:0/1035322568 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56330920c270 con 0x563309363890 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f6716ff5700 1 -- 172.21.15.12:0/1035322568 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6724036070 con 0x563309363890 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f67453c9b80 10 client.20913 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f67453c9b80 10 client.20913 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f67453c9b80 20 client.20913 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5633093854c0 0x5633093878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f67453c9b80 4 client.20913 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:19.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.349+0000 7f67453c9b80 1 -- 172.21.15.12:0/1035322568 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563309332810 con 0x5633093854c0 2024-05-09T00:06:19.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.351+0000 7f673259c700 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5633093854c0 0x5633093878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.353+0000 7f6716ff5700 10 client.20913 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:06:19.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.353+0000 7f673259c700 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5633093854c0 0x5633093878e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8f2328c85e3e0b9 server_cookie=e27250877444166f in_seq=0 out_seq=0 2024-05-09T00:06:19.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f6716ff5700 1 -- 172.21.15.12:0/1035322568 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563309332810 con 0x5633093854c0 2024-05-09T00:06:19.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f6716ff5700 10 client.20913 handle_command_reply: tid=0 2024-05-09T00:06:19.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 client.20913 shutdown 2024-05-09T00:06:19.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 -- 172.21.15.12:0/1035322568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309363890 msgr2=0x56330936ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309363890 0x56330936ca70 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6724009990 tx=0x7f67240057c0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 -- 172.21.15.12:0/1035322568 shutdown_connections 2024-05-09T00:06:19.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5633093854c0 0x5633093878e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 0x56330936d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309363890 0x56330936ca70 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f6716ff5700 0 client.20913 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:06:19.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1035322568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309368e00 0x56330936cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 -- 172.21.15.12:0/1035322568 >> 172.21.15.12:0/1035322568 conn(0x5633092c1210 msgr2=0x56330935d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 -- 172.21.15.12:0/1035322568 shutdown_connections 2024-05-09T00:06:19.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.355+0000 7f67453c9b80 1 -- 172.21.15.12:0/1035322568 wait complete. 2024-05-09T00:06:19.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.356+0000 7f67453c9b80 20 client.20913 trim_cache size 0 max 16384 2024-05-09T00:06:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.376+0000 7f67453c9b80 1 Processor -- start 2024-05-09T00:06:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.377+0000 7f67453c9b80 1 -- start start 2024-05-09T00:06:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.377+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330935dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.378+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 0x563309368290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.378+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 0x56330936d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.378+0000 7f67453c9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330938cab0 con 0x563309368e00 2024-05-09T00:06:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.378+0000 7f67453c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330935ea20 con 0x563309363890 2024-05-09T00:06:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.378+0000 7f67453c9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563309358a90 con 0x563309369720 2024-05-09T00:06:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.379+0000 7f6732d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330935dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.379+0000 7f6732d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330935dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58222/0 (socket says 172.21.15.12:58222) 2024-05-09T00:06:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.379+0000 7f6732d9d700 1 -- 172.21.15.12:0/76874050 learned_addr learned my addr 172.21.15.12:0/76874050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.379+0000 7f673359e700 1 --2- 172.21.15.12:0/76874050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 0x56330936d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.380+0000 7f673259c700 1 --2- 172.21.15.12:0/76874050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 0x563309368290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.380+0000 7f673359e700 1 -- 172.21.15.12:0/76874050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 msgr2=0x56330935dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.380+0000 7f673359e700 1 --2- 172.21.15.12:0/76874050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330935dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.380+0000 7f673359e700 1 -- 172.21.15.12:0/76874050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 msgr2=0x563309368290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.380+0000 7f673359e700 1 --2- 172.21.15.12:0/76874050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 0x563309368290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.380+0000 7f673359e700 1 -- 172.21.15.12:0/76874050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563309279560 con 0x563309369720 2024-05-09T00:06:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.383+0000 7f673359e700 1 --2- 172.21.15.12:0/76874050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 0x56330936d310 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f672c004b10 tx=0x7f672c0118e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1286753f28dd75dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.383+0000 7f6719ffb700 1 -- 172.21.15.12:0/76874050 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00e040 con 0x563309369720 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.383+0000 7f6719ffb700 1 -- 172.21.15.12:0/76874050 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f672c00f040 con 0x563309369720 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.383+0000 7f6719ffb700 1 -- 172.21.15.12:0/76874050 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c01e7f0 con 0x563309369720 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.384+0000 7f67453c9b80 1 -- 172.21.15.12:0/76874050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 msgr2=0x56330936d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.384+0000 7f67453c9b80 1 --2- 172.21.15.12:0/76874050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 0x56330936d310 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f672c004b10 tx=0x7f672c0118e0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.384+0000 7f67453c9b80 1 -- 172.21.15.12:0/76874050 shutdown_connections 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.384+0000 7f67453c9b80 1 --2- 172.21.15.12:0/76874050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309369720 0x56330936d310 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.384+0000 7f67453c9b80 1 --2- 172.21.15.12:0/76874050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330935dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.384+0000 7f67453c9b80 1 --2- 172.21.15.12:0/76874050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309368e00 0x563309368290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.384+0000 7f67453c9b80 1 -- 172.21.15.12:0/76874050 >> 172.21.15.12:0/76874050 conn(0x5633092c1210 msgr2=0x563309367b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.384+0000 7f67453c9b80 1 -- 172.21.15.12:0/76874050 shutdown_connections 2024-05-09T00:06:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.384+0000 7f67453c9b80 1 -- 172.21.15.12:0/76874050 wait complete. 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.385+0000 7f67453c9b80 1 -- start start 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.385+0000 7f67453c9b80 1 Processor -- start 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.385+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330937ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.385+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309368e00 0x56330937b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.385+0000 7f67453c9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309369720 0x5633093d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.385+0000 7f67453c9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633093d4f00 con 0x563309369720 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.385+0000 7f67453c9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633093d5070 con 0x563309363890 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.385+0000 7f67453c9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633093d5230 con 0x563309368e00 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f67453c9b80 10 client.0 _ll_register_callbacks cb 0x563309258b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f6732d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330937ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f6732d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330937ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58238/0 (socket says 172.21.15.12:58238) 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f673359e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309369720 0x5633093d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f6732d9d700 1 -- 172.21.15.12:0/1637743776 learned_addr learned my addr 172.21.15.12:0/1637743776 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f6732d9d700 1 -- 172.21.15.12:0/1637743776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309368e00 msgr2=0x56330937b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f6732d9d700 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309368e00 0x56330937b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f6732d9d700 1 -- 172.21.15.12:0/1637743776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309369720 msgr2=0x5633093d49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f6732d9d700 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309369720 0x5633093d49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f6732d9d700 1 -- 172.21.15.12:0/1637743776 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c011370 con 0x563309363890 2024-05-09T00:06:19.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.386+0000 7f673359e700 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309369720 0x5633093d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:19.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.387+0000 7f6732d9d700 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330937ac50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f672400f910 tx=0x7f672400fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b41f43bc133bc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.387+0000 7f6716ff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:06:19.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.391+0000 7f6716ff5700 1 -- 172.21.15.12:0/1637743776 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724029020 con 0x563309363890 2024-05-09T00:06:19.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.391+0000 7f6716ff5700 1 -- 172.21.15.12:0/1637743776 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f672400fb80 con 0x563309363890 2024-05-09T00:06:19.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.391+0000 7f67453c9b80 1 -- 172.21.15.12:0/1637743776 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f672400bda0 con 0x563309363890 2024-05-09T00:06:19.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.391+0000 7f6716ff5700 1 -- 172.21.15.12:0/1637743776 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f672401d070 con 0x563309363890 2024-05-09T00:06:19.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.391+0000 7f67453c9b80 10 client.30446 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:19.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.391+0000 7f67453c9b80 1 -- 172.21.15.12:0/1637743776 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x563309138cd0 con 0x563309363890 2024-05-09T00:06:19.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.395+0000 7f6716ff5700 1 -- 172.21.15.12:0/1637743776 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f672404d080 con 0x563309363890 2024-05-09T00:06:19.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.395+0000 7f6716ff5700 1 -- 172.21.15.12:0/1637743776 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6724005050 con 0x563309363890 2024-05-09T00:06:19.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.395+0000 7f67453c9b80 10 client.30446 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:19.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.395+0000 7f67453c9b80 10 client.30446 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:06:19.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.395+0000 7f67453c9b80 20 client.30446 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:19.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.395+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5633093d65b0 0x5633093d69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.395+0000 7f67453c9b80 4 client.30446 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:06:19.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.395+0000 7f67453c9b80 1 -- 172.21.15.12:0/1637743776 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5633092abec0 con 0x5633093d65b0 2024-05-09T00:06:19.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.397+0000 7f673259c700 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5633093d65b0 0x5633093d69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.397+0000 7f6716ff5700 10 client.30446 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:06:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.397+0000 7f673259c700 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5633093d65b0 0x5633093d69c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d1f1ee157ccb135 server_cookie=708edab8a3395e88 in_seq=0 out_seq=0 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.401+0000 7f6716ff5700 1 -- 172.21.15.12:0/1637743776 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5633092abec0 con 0x5633093d65b0 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.401+0000 7f6716ff5700 10 client.30446 handle_command_reply: tid=0 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.401+0000 7f67453c9b80 1 client.30446 shutdown 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 1 -- 172.21.15.12:0/1637743776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 msgr2=0x56330937ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330937ac50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f672400f910 tx=0x7f672400fa30 comp rx=0 tx=0).stop 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 1 -- 172.21.15.12:0/1637743776 shutdown_connections 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563309369720 0x5633093d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563309363890 0x56330937ac50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5633093d65b0 0x5633093d69c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 1 --2- 172.21.15.12:0/1637743776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563309368e00 0x56330937b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 1 -- 172.21.15.12:0/1637743776 >> 172.21.15.12:0/1637743776 conn(0x5633092c1210 msgr2=0x563309368410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f6716ff5700 0 client.30446 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:06:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 1 -- 172.21.15.12:0/1637743776 shutdown_connections 2024-05-09T00:06:19.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 1 -- 172.21.15.12:0/1637743776 wait complete. 2024-05-09T00:06:19.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.402+0000 7f67453c9b80 20 client.30446 trim_cache size 0 max 16384 2024-05-09T00:06:19.399 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:06:19.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.405+0000 7f673920f700 1 -- 172.21.15.12:0/4055206848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f671c0a1290 msgr2=0x7f671c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.405+0000 7f673920f700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f671c0a1290 0x7f671c0a3750 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6724006010 tx=0x7f6724005e80 comp rx=0 tx=0).stop 2024-05-09T00:06:19.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.405+0000 7f673920f700 1 -- 172.21.15.12:0/4055206848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673419fa60 msgr2=0x7f67341996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.405+0000 7f673920f700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673419fa60 0x7f67341996a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f672800e970 tx=0x7f672800ec80 comp rx=0 tx=0).stop 2024-05-09T00:06:19.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.405+0000 7f673920f700 1 -- 172.21.15.12:0/4055206848 shutdown_connections 2024-05-09T00:06:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.405+0000 7f673920f700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f671c0a1290 0x7f671c0a3750 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.405+0000 7f673920f700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6734199be0 0x7f673419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.405+0000 7f673920f700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67341062a0 0x7f673419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.405+0000 7f673920f700 1 --2- 172.21.15.12:0/4055206848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673419fa60 0x7f67341996a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.405+0000 7f673920f700 1 -- 172.21.15.12:0/4055206848 >> 172.21.15.12:0/4055206848 conn(0x7f6734076ba0 msgr2=0x7f67340fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.406+0000 7f673920f700 1 -- 172.21.15.12:0/4055206848 shutdown_connections 2024-05-09T00:06:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.406+0000 7f673920f700 1 -- 172.21.15.12:0/4055206848 wait complete. 2024-05-09T00:06:19.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:19 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/308887260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:06:19.427 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:06:19.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:19 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/308887260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:06:19.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:19 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/308887260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:06:19.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.594+0000 7fc5edd3a700 1 -- 172.21.15.12:0/746675620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5e8105160 msgr2=0x7fc5e810cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.594+0000 7fc5edd3a700 1 --2- 172.21.15.12:0/746675620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5e8105160 0x7fc5e810cce0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000b0a0 tx=0x7fc5e000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.594+0000 7fc5edd3a700 1 -- 172.21.15.12:0/746675620 shutdown_connections 2024-05-09T00:06:19.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.594+0000 7fc5edd3a700 1 --2- 172.21.15.12:0/746675620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5e8105160 0x7fc5e810cce0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.594+0000 7fc5edd3a700 1 --2- 172.21.15.12:0/746675620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8100060 0x7fc5e8100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.594+0000 7fc5edd3a700 1 --2- 172.21.15.12:0/746675620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5e8100a10 0x7fc5e8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.594+0000 7fc5edd3a700 1 -- 172.21.15.12:0/746675620 >> 172.21.15.12:0/746675620 conn(0x7fc5e8076f40 msgr2=0x7fc5e8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.594+0000 7fc5edd3a700 1 -- 172.21.15.12:0/746675620 shutdown_connections 2024-05-09T00:06:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.594+0000 7fc5edd3a700 1 -- 172.21.15.12:0/746675620 wait complete. 2024-05-09T00:06:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.594+0000 7fc5edd3a700 1 Processor -- start 2024-05-09T00:06:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5edd3a700 1 -- start start 2024-05-09T00:06:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5edd3a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5e8100060 0x7fc5e8197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5edd3a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8105160 0x7fc5e8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5edd3a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5e819be60 0x7fc5e819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5edd3a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e819c8e0 con 0x7fc5e8100060 2024-05-09T00:06:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5edd3a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e819ca50 con 0x7fc5e819be60 2024-05-09T00:06:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5edd3a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e819cc10 con 0x7fc5e8105160 2024-05-09T00:06:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5e77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5e8100060 0x7fc5e8197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5e77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5e8100060 0x7fc5e8197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33110/0 (socket says 172.21.15.12:33110) 2024-05-09T00:06:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5e77fe700 1 -- 172.21.15.12:0/340290753 learned_addr learned my addr 172.21.15.12:0/340290753 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5e7fff700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5e819be60 0x7fc5e819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8105160 0x7fc5e8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5e77fe700 1 -- 172.21.15.12:0/340290753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8105160 msgr2=0x7fc5e8197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5e77fe700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8105160 0x7fc5e8197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5e77fe700 1 -- 172.21.15.12:0/340290753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5e819be60 msgr2=0x7fc5e819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5e77fe700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5e819be60 0x7fc5e819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.595+0000 7fc5e77fe700 1 -- 172.21.15.12:0/340290753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e0009d00 con 0x7fc5e8100060 2024-05-09T00:06:19.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.596+0000 7fc5e77fe700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5e8100060 0x7fc5e8197580 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800e9a0 tx=0x7fc5d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.596+0000 7fc5e4ff9700 1 -- 172.21.15.12:0/340290753 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8009790 con 0x7fc5e8100060 2024-05-09T00:06:19.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.596+0000 7fc5edd3a700 1 -- 172.21.15.12:0/340290753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e81a4170 con 0x7fc5e8100060 2024-05-09T00:06:19.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.596+0000 7fc5e4ff9700 1 -- 172.21.15.12:0/340290753 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5d8010430 con 0x7fc5e8100060 2024-05-09T00:06:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.596+0000 7fc5e4ff9700 1 -- 172.21.15.12:0/340290753 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8018680 con 0x7fc5e8100060 2024-05-09T00:06:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.596+0000 7fc5edd3a700 1 -- 172.21.15.12:0/340290753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e81a46a0 con 0x7fc5e8100060 2024-05-09T00:06:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.598+0000 7fc5e4ff9700 1 -- 172.21.15.12:0/340290753 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc5d80105a0 con 0x7fc5e8100060 2024-05-09T00:06:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.599+0000 7fc5e4ff9700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5d00a1290 0x7fc5d00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.599+0000 7fc5e4ff9700 1 -- 172.21.15.12:0/340290753 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc5d8014070 con 0x7fc5e8100060 2024-05-09T00:06:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.599+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5d00a1290 0x7fc5d00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.600+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5d00a1290 0x7fc5d00a3750 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8198ad0 tx=0x7fc5dc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5f9ef4b80 1 Processor -- start 2024-05-09T00:06:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5f9ef4b80 1 -- start start 2024-05-09T00:06:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fa51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8f9f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 0x55ccc8f9fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc8ef6150 con 0x55ccc8fa5720 2024-05-09T00:06:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc8ef3b10 con 0x55ccc8fa4e00 2024-05-09T00:06:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc8eef400 con 0x55ccc8f9f890 2024-05-09T00:06:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5e6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8f9f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5e77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fa51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5e6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8f9f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33116/0 (socket says 172.21.15.12:33116) 2024-05-09T00:06:19.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5e7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 0x55ccc8f9fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/4133596246 learned_addr learned my addr 172.21.15.12:0/4133596246 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/4133596246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 msgr2=0x55ccc8f9fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/4133596246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 0x55ccc8f9fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/4133596246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8fa4e00 msgr2=0x55ccc8fa51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.601+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/4133596246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fa51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.602+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/4133596246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccc8d703b0 con 0x55ccc8fa5720 2024-05-09T00:06:19.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.602+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/4133596246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8f9f350 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc011d10 tx=0x7fc5dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed6effec6f33ac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.602+0000 7fc5cdffb700 1 -- 172.21.15.12:0/4133596246 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc011d40 con 0x55ccc8fa5720 2024-05-09T00:06:19.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.602+0000 7fc5cdffb700 1 -- 172.21.15.12:0/4133596246 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5dc004020 con 0x55ccc8fa5720 2024-05-09T00:06:19.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.602+0000 7fc5cdffb700 1 -- 172.21.15.12:0/4133596246 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc003bc0 con 0x55ccc8fa5720 2024-05-09T00:06:19.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.603+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/4133596246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 msgr2=0x55ccc8f9f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.603+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/4133596246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8f9f350 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc011d10 tx=0x7fc5dc014000 comp rx=0 tx=0).stop 2024-05-09T00:06:19.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.603+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/4133596246 shutdown_connections 2024-05-09T00:06:19.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.603+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/4133596246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 0x55ccc8f9fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.603+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/4133596246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fa51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.603+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/4133596246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8f9f350 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.603+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/4133596246 >> 172.21.15.12:0/4133596246 conn(0x55ccc8efd210 msgr2=0x55ccc8f99df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.603+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/4133596246 shutdown_connections 2024-05-09T00:06:19.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.603+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/4133596246 wait complete. 2024-05-09T00:06:19.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.603+0000 7fc5f9ef4b80 1 -- start start 2024-05-09T00:06:19.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5f9ef4b80 1 Processor -- start 2024-05-09T00:06:19.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 0x55ccc8fab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fa8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8fa9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc8e8b560 con 0x55ccc8fa5720 2024-05-09T00:06:19.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc8e8b200 con 0x55ccc8fa4e00 2024-05-09T00:06:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc8d726a0 con 0x55ccc8f9f890 2024-05-09T00:06:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5f9ef4b80 10 client.0 _ll_register_callbacks cb 0x55ccc8e94b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5e77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 0x55ccc8fab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8fa9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5e77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 0x55ccc8fab750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46038/0 (socket says 172.21.15.12:46038) 2024-05-09T00:06:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5e77fe700 1 -- 172.21.15.12:0/463310934 learned_addr learned my addr 172.21.15.12:0/463310934 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fa8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5e77fe700 1 -- 172.21.15.12:0/463310934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8fa4e00 msgr2=0x55ccc8fa8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5e77fe700 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fa8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5e77fe700 1 -- 172.21.15.12:0/463310934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 msgr2=0x55ccc8fa9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5e77fe700 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8fa9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.604+0000 7fc5e77fe700 1 -- 172.21.15.12:0/463310934 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5dc017070 con 0x55ccc8f9f890 2024-05-09T00:06:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.605+0000 7fc5e77fe700 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 0x55ccc8fab750 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8099cd0 tx=0x7fc5d809a7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0739095c467a19c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.605+0000 7fc5c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.605+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/463310934 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80adea0 con 0x55ccc8f9f890 2024-05-09T00:06:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.605+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/463310934 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccc8e48270 con 0x55ccc8f9f890 2024-05-09T00:06:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.605+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/463310934 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc5d809de00 con 0x55ccc8f9f890 2024-05-09T00:06:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.605+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/463310934 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5d80a2450 con 0x55ccc8f9f890 2024-05-09T00:06:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.605+0000 7fc5f9ef4b80 10 client.30631 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.605+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/463310934 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55ccc8d1cb60 con 0x55ccc8f9f890 2024-05-09T00:06:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.605+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/463310934 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ccc8e48270 con 0x55ccc8f9f890 2024-05-09T00:06:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.606+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/463310934 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc5d80a1fc0 con 0x55ccc8f9f890 2024-05-09T00:06:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.606+0000 7fc5f9ef4b80 10 client.30631 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.606+0000 7fc5f9ef4b80 10 client.30631 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:06:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.606+0000 7fc5f9ef4b80 20 client.30631 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.606+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ccc8fb9120 0x55ccc8fbb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.606+0000 7fc5f9ef4b80 4 client.30631 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.606+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/463310934 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccc8f6e810 con 0x55ccc8fb9120 2024-05-09T00:06:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.606+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ccc8fb9120 0x55ccc8fbb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.606+0000 7fc5c6ffd700 10 client.30631 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:06:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.606+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ccc8fb9120 0x55ccc8fbb510 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65a34da9f1f421a4 server_cookie=61aade182b0c275d in_seq=0 out_seq=0 2024-05-09T00:06:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/463310934 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ccc8f6e810 con 0x55ccc8fb9120 2024-05-09T00:06:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5c6ffd700 10 client.30631 handle_command_reply: tid=0 2024-05-09T00:06:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 client.30631 shutdown 2024-05-09T00:06:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/463310934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 msgr2=0x55ccc8fab750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 0x55ccc8fab750 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8099cd0 tx=0x7fc5d809a7c0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/463310934 shutdown_connections 2024-05-09T00:06:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ccc8fb9120 0x55ccc8fbb510 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8fa9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8f9f890 0x55ccc8fab750 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/463310934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fa8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5c6ffd700 0 client.30631 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:06:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/463310934 >> 172.21.15.12:0/463310934 conn(0x55ccc8efd210 msgr2=0x55ccc8f995a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/463310934 shutdown_connections 2024-05-09T00:06:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/463310934 wait complete. 2024-05-09T00:06:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.610+0000 7fc5f9ef4b80 20 client.30631 trim_cache size 0 max 16384 2024-05-09T00:06:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5f9ef4b80 1 Processor -- start 2024-05-09T00:06:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5f9ef4b80 1 -- start start 2024-05-09T00:06:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8f9f890 0x55ccc8f9f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8f99d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8fa7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc8f94a90 con 0x55ccc8fa5720 2024-05-09T00:06:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc8f96dc0 con 0x55ccc8f9f890 2024-05-09T00:06:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc8f9ba00 con 0x55ccc8fa4e00 2024-05-09T00:06:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8fa7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8fa7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33138/0 (socket says 172.21.15.12:33138) 2024-05-09T00:06:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.625+0000 7fc5e7fff700 1 -- 172.21.15.12:0/90966315 learned_addr learned my addr 172.21.15.12:0/90966315 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/90966315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8f99d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5e77fe700 1 --2- 172.21.15.12:0/90966315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8f9f890 0x55ccc8f9f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/90966315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8f9f890 msgr2=0x55ccc8f9f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/90966315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8f9f890 0x55ccc8f9f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/90966315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 msgr2=0x55ccc8fa7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/90966315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8fa7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/90966315 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccc8eb5560 con 0x55ccc8fa4e00 2024-05-09T00:06:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/90966315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8f99d20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc005950 tx=0x7fc5dc029040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91037e8eb4fc315 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5ce7fc700 1 -- 172.21.15.12:0/90966315 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc02c040 con 0x55ccc8fa4e00 2024-05-09T00:06:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5ce7fc700 1 -- 172.21.15.12:0/90966315 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5dc005640 con 0x55ccc8fa4e00 2024-05-09T00:06:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.626+0000 7fc5ce7fc700 1 -- 172.21.15.12:0/90966315 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc003830 con 0x55ccc8fa4e00 2024-05-09T00:06:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.627+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/90966315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 msgr2=0x55ccc8f99d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.627+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/90966315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8f99d20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc005950 tx=0x7fc5dc029040 comp rx=0 tx=0).stop 2024-05-09T00:06:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.627+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/90966315 shutdown_connections 2024-05-09T00:06:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.627+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/90966315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc8fa7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.627+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/90966315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8f9f890 0x55ccc8f9f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.627+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/90966315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8f99d20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.627+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/90966315 >> 172.21.15.12:0/90966315 conn(0x55ccc8efd210 msgr2=0x55ccc8fbfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.627+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/90966315 shutdown_connections 2024-05-09T00:06:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.627+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/90966315 wait complete. 2024-05-09T00:06:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5f9ef4b80 1 -- start start 2024-05-09T00:06:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5f9ef4b80 1 Processor -- start 2024-05-09T00:06:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8f9f890 0x55ccc8fac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5f9ef4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc900e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc8facf90 con 0x55ccc8fa5720 2024-05-09T00:06:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc900eaf0 con 0x55ccc8f9f890 2024-05-09T00:06:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5f9ef4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc900ec30 con 0x55ccc8fa4e00 2024-05-09T00:06:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5f9ef4b80 10 client.0 _ll_register_callbacks cb 0x55ccc8e94b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5e6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5e77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8f9f890 0x55ccc8fac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5e6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46054/0 (socket says 172.21.15.12:46054) 2024-05-09T00:06:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/589657567 learned_addr learned my addr 172.21.15.12:0/589657567 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.628+0000 7fc5e7fff700 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc900e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/589657567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8f9f890 msgr2=0x55ccc8fac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8f9f890 0x55ccc8fac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/589657567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 msgr2=0x55ccc900e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc900e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5e6ffd700 1 -- 172.21.15.12:0/589657567 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccc8d703b0 con 0x55ccc8fa4e00 2024-05-09T00:06:19.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5e6ffd700 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fac680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009fb0 tx=0x7fc5dc005440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e6ae979406aa85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:19.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/589657567 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc02c050 con 0x55ccc8fa4e00 2024-05-09T00:06:19.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/589657567 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc5dc0313b0 con 0x55ccc8fa4e00 2024-05-09T00:06:19.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/589657567 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5dc004020 con 0x55ccc8fa4e00 2024-05-09T00:06:19.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/589657567 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5dc031570 con 0x55ccc8fa4e00 2024-05-09T00:06:19.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5f9ef4b80 10 client.30637 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:19.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/589657567 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55ccc8d74cd0 con 0x55ccc8fa4e00 2024-05-09T00:06:19.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.629+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/589657567 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5dc04f080 con 0x55ccc8fa4e00 2024-05-09T00:06:19.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.630+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/589657567 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc5dc024030 con 0x55ccc8fa4e00 2024-05-09T00:06:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.630+0000 7fc5f9ef4b80 10 client.30637 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.630+0000 7fc5f9ef4b80 10 client.30637 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:06:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.630+0000 7fc5f9ef4b80 20 client.30637 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.630+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ccc900fec0 0x55ccc90122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.630+0000 7fc5f9ef4b80 4 client.30637 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:06:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.630+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/589657567 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ccc8ee7ec0 con 0x55ccc900fec0 2024-05-09T00:06:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.630+0000 7fc5e77fe700 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ccc900fec0 0x55ccc90122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.630+0000 7fc5e77fe700 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ccc900fec0 0x55ccc90122e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a1518e775e4a8 server_cookie=15cddf3726bc02ea in_seq=0 out_seq=0 2024-05-09T00:06:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.630+0000 7fc5c6ffd700 10 client.30637 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:06:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5c6ffd700 1 -- 172.21.15.12:0/589657567 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ccc8ee7ec0 con 0x55ccc900fec0 2024-05-09T00:06:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5c6ffd700 10 client.30637 handle_command_reply: tid=0 2024-05-09T00:06:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 client.30637 shutdown 2024-05-09T00:06:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/589657567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 msgr2=0x55ccc8fac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fac680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009fb0 tx=0x7fc5dc005440 comp rx=0 tx=0).stop 2024-05-09T00:06:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/589657567 shutdown_connections 2024-05-09T00:06:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55ccc900fec0 0x55ccc90122e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccc8fa5720 0x55ccc900e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ccc8f9f890 0x55ccc8fac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5c6ffd700 0 client.30637 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:06:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 --2- 172.21.15.12:0/589657567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccc8fa4e00 0x55ccc8fac680 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/589657567 >> 172.21.15.12:0/589657567 conn(0x55ccc8efd210 msgr2=0x55ccc8fa72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/589657567 shutdown_connections 2024-05-09T00:06:19.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 1 -- 172.21.15.12:0/589657567 wait complete. 2024-05-09T00:06:19.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.636+0000 7fc5f9ef4b80 20 client.30637 trim_cache size 0 max 16384 2024-05-09T00:06:19.656 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:06:19.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 -- 172.21.15.12:0/340290753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5d00a1290 msgr2=0x7fc5d00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5d00a1290 0x7fc5d00a3750 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8198ad0 tx=0x7fc5dc009260 comp rx=0 tx=0).stop 2024-05-09T00:06:19.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 -- 172.21.15.12:0/340290753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5e8100060 msgr2=0x7fc5e8197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5e8100060 0x7fc5e8197580 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800e9a0 tx=0x7fc5d800ecb0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 -- 172.21.15.12:0/340290753 shutdown_connections 2024-05-09T00:06:19.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc5d00a1290 0x7fc5d00a3750 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5e819be60 0x7fc5e819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5e8100060 0x7fc5e8197580 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 --2- 172.21.15.12:0/340290753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8105160 0x7fc5e8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 -- 172.21.15.12:0/340290753 >> 172.21.15.12:0/340290753 conn(0x7fc5e8076f40 msgr2=0x7fc5e8068420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 -- 172.21.15.12:0/340290753 shutdown_connections 2024-05-09T00:06:19.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.637+0000 7fc5edd3a700 1 -- 172.21.15.12:0/340290753 wait complete. 2024-05-09T00:06:19.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:06:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.821+0000 7f4ff1222700 1 -- 172.21.15.12:0/3432762662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fec10d940 msgr2=0x7f4fec10fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.821+0000 7f4ff1222700 1 --2- 172.21.15.12:0/3432762662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fec10d940 0x7f4fec10fd30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400b0a0 tx=0x7f4fe400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.821+0000 7f4ff1222700 1 -- 172.21.15.12:0/3432762662 shutdown_connections 2024-05-09T00:06:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.821+0000 7f4ff1222700 1 --2- 172.21.15.12:0/3432762662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fec10d940 0x7f4fec10fd30 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.821+0000 7f4ff1222700 1 --2- 172.21.15.12:0/3432762662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fec068050 0x7f4fec068430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.821+0000 7f4ff1222700 1 --2- 172.21.15.12:0/3432762662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fec068970 0x7f4fec10d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.821+0000 7f4ff1222700 1 -- 172.21.15.12:0/3432762662 >> 172.21.15.12:0/3432762662 conn(0x7f4fec076f40 msgr2=0x7f4fec077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.821+0000 7f4ff1222700 1 -- 172.21.15.12:0/3432762662 shutdown_connections 2024-05-09T00:06:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.822+0000 7f4ff1222700 1 -- 172.21.15.12:0/3432762662 wait complete. 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4ff1222700 1 Processor -- start 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4ff1222700 1 -- start start 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4ff1222700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fec068050 0x7f4fec199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4ff1222700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fec068970 0x7f4fec199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4ff1222700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fec10d940 0x7f4fec19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4ff1222700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec19d5b0 con 0x7f4fec10d940 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4ff1222700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec19d720 con 0x7f4fec068050 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4ff1222700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec19d8e0 con 0x7f4fec068970 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4fead9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fec068050 0x7f4fec199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4feb59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fec10d940 0x7f4fec19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4fea59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fec068970 0x7f4fec199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4fead9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fec068050 0x7f4fec199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58308/0 (socket says 172.21.15.12:58308) 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4fea59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fec068970 0x7f4fec199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46070/0 (socket says 172.21.15.12:46070) 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4fead9d700 1 -- 172.21.15.12:0/321918096 learned_addr learned my addr 172.21.15.12:0/321918096 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.823+0000 7f4feb59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fec10d940 0x7f4fec19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33160/0 (socket says 172.21.15.12:33160) 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4fea59c700 1 -- 172.21.15.12:0/321918096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fec068050 msgr2=0x7f4fec199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4fea59c700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fec068050 0x7f4fec199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4fea59c700 1 -- 172.21.15.12:0/321918096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fec10d940 msgr2=0x7f4fec19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4fea59c700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fec10d940 0x7f4fec19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4fea59c700 1 -- 172.21.15.12:0/321918096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4009d00 con 0x7f4fec068970 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4feb59e700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fec10d940 0x7f4fec19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4fea59c700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fec068970 0x7f4fec199cc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9f0 tx=0x7f4fe000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4fd3fff700 1 -- 172.21.15.12:0/321918096 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe000c9c0 con 0x7f4fec068970 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4fd3fff700 1 -- 172.21.15.12:0/321918096 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fe0010430 con 0x7f4fec068970 2024-05-09T00:06:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4ff1222700 1 -- 172.21.15.12:0/321918096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec19dbc0 con 0x7f4fec068970 2024-05-09T00:06:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4fd3fff700 1 -- 172.21.15.12:0/321918096 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0018680 con 0x7f4fec068970 2024-05-09T00:06:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.824+0000 7f4ff1222700 1 -- 172.21.15.12:0/321918096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec1a6520 con 0x7f4fec068970 2024-05-09T00:06:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.827+0000 7f4fd3fff700 1 -- 172.21.15.12:0/321918096 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4fe0010860 con 0x7f4fec068970 2024-05-09T00:06:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.827+0000 7f4fd3fff700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fd40a1310 0x7f4fd40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.827+0000 7f4fead9d700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fd40a1310 0x7f4fd40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.827+0000 7f4fd3fff700 1 -- 172.21.15.12:0/321918096 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4fe0014070 con 0x7f4fec068970 2024-05-09T00:06:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.828+0000 7f4fead9d700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fd40a1310 0x7f4fd40a37d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009d20 tx=0x7f4fdc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.828+0000 7f4ffd3dcb80 1 Processor -- start 2024-05-09T00:06:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.828+0000 7f4ffd3dcb80 1 -- start start 2024-05-09T00:06:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.828+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd1e00 0x558926cd21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.828+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 0x558926ccc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.828+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 0x558926ccccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926c23150 con 0x558926ccc890 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926c20b10 con 0x558926cd2720 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926c1c400 con 0x558926cd1e00 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fea59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 0x558926ccc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fead9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd1e00 0x558926cd21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fea59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 0x558926ccc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58322/0 (socket says 172.21.15.12:58322) 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fea59c700 1 -- 172.21.15.12:0/1299349935 learned_addr learned my addr 172.21.15.12:0/1299349935 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fead9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd1e00 0x558926cd21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46078/0 (socket says 172.21.15.12:46078) 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4feb59e700 1 --2- 172.21.15.12:0/1299349935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 0x558926ccccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fead9d700 1 -- 172.21.15.12:0/1299349935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 msgr2=0x558926ccc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fead9d700 1 --2- 172.21.15.12:0/1299349935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 0x558926ccc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fead9d700 1 -- 172.21.15.12:0/1299349935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 msgr2=0x558926ccccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fead9d700 1 --2- 172.21.15.12:0/1299349935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 0x558926ccccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fead9d700 1 -- 172.21.15.12:0/1299349935 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558926a9d3b0 con 0x558926cd1e00 2024-05-09T00:06:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fead9d700 1 --2- 172.21.15.12:0/1299349935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd1e00 0x558926cd21e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc011ab0 tx=0x7f4fdc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e12b1fc02841cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.829+0000 7f4fd17fa700 1 -- 172.21.15.12:0/1299349935 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc00edf0 con 0x558926cd1e00 2024-05-09T00:06:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4fd17fa700 1 -- 172.21.15.12:0/1299349935 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fdc021070 con 0x558926cd1e00 2024-05-09T00:06:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4fd17fa700 1 -- 172.21.15.12:0/1299349935 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc01d830 con 0x558926cd1e00 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1299349935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd1e00 msgr2=0x558926cd21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/1299349935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd1e00 0x558926cd21e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc011ab0 tx=0x7f4fdc014000 comp rx=0 tx=0).stop 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1299349935 shutdown_connections 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/1299349935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 0x558926ccccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/1299349935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd1e00 0x558926cd21e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/1299349935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 0x558926ccc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1299349935 >> 172.21.15.12:0/1299349935 conn(0x558926c2a210 msgr2=0x558926cc6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1299349935 shutdown_connections 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.830+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1299349935 wait complete. 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4ffd3dcb80 1 -- start start 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4ffd3dcb80 1 Processor -- start 2024-05-09T00:06:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 0x558926cd87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 0x558926cd5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd2720 0x558926cd6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926bb8560 con 0x558926cd2720 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926bb8200 con 0x558926cd1e00 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926a9f6a0 con 0x558926ccc890 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4ffd3dcb80 10 client.0 _ll_register_callbacks cb 0x558926bc1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4feb59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd2720 0x558926cd6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.831+0000 7f4fea59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 0x558926cd5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4feb59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd2720 0x558926cd6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33170/0 (socket says 172.21.15.12:33170) 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4feb59e700 1 -- 172.21.15.12:0/1650469691 learned_addr learned my addr 172.21.15.12:0/1650469691 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4fea59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 0x558926cd5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58330/0 (socket says 172.21.15.12:58330) 2024-05-09T00:06:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4fead9d700 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 0x558926cd87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4feb59e700 1 -- 172.21.15.12:0/1650469691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 msgr2=0x558926cd87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4feb59e700 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 0x558926cd87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4feb59e700 1 -- 172.21.15.12:0/1650469691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 msgr2=0x558926cd5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4feb59e700 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 0x558926cd5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4feb59e700 1 -- 172.21.15.12:0/1650469691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fdc017070 con 0x558926cd2720 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4feb59e700 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd2720 0x558926cd6190 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400be20 tx=0x7f4fe400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd01b467a1d27f31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4fceff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4fceff5700 1 -- 172.21.15.12:0/1650469691 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe400e040 con 0x558926cd2720 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4fceff5700 1 -- 172.21.15.12:0/1650469691 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4fe4023070 con 0x558926cd2720 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1650469691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558926b75270 con 0x558926cd2720 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.832+0000 7f4fceff5700 1 -- 172.21.15.12:0/1650469691 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fe40074e0 con 0x558926cd2720 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4ffd3dcb80 10 client.20949 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1650469691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x558926a49b60 con 0x558926cd2720 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4fceff5700 1 -- 172.21.15.12:0/1650469691 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558926b75270 con 0x558926cd2720 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4fceff5700 1 -- 172.21.15.12:0/1650469691 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4fe402e070 con 0x558926cd2720 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4ffd3dcb80 10 client.20949 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4ffd3dcb80 10 client.20949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4ffd3dcb80 20 client.20949 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558926ce6170 0x558926ce8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4ffd3dcb80 4 client.20949 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1650469691 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558926c9b810 con 0x558926ce6170 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.833+0000 7f4fead9d700 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558926ce6170 0x558926ce8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.834+0000 7f4fead9d700 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558926ce6170 0x558926ce8590 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c021c284d04f5d5e server_cookie=f545fc74366e64e7 in_seq=0 out_seq=0 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.834+0000 7f4fceff5700 10 client.20949 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.835+0000 7f4fceff5700 1 -- 172.21.15.12:0/1650469691 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558926c9b810 con 0x558926ce6170 2024-05-09T00:06:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.835+0000 7f4fceff5700 10 client.20949 handle_command_reply: tid=0 2024-05-09T00:06:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.835+0000 7f4ffd3dcb80 1 client.20949 shutdown 2024-05-09T00:06:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1650469691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd2720 msgr2=0x558926cd6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd2720 0x558926cd6190 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400be20 tx=0x7f4fe400be50 comp rx=0 tx=0).stop 2024-05-09T00:06:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1650469691 shutdown_connections 2024-05-09T00:06:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd2720 0x558926cd6190 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 0x558926cd87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558926ce6170 0x558926ce8590 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/1650469691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 0x558926cd5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4fceff5700 0 client.20949 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:19.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1650469691 >> 172.21.15.12:0/1650469691 conn(0x558926c2a210 msgr2=0x558926cc65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1650469691 shutdown_connections 2024-05-09T00:06:19.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/1650469691 wait complete. 2024-05-09T00:06:19.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.836+0000 7f4ffd3dcb80 20 client.20949 trim_cache size 0 max 16384 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.850+0000 7f4ffd3dcb80 1 Processor -- start 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4ffd3dcb80 1 -- start start 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 0x558926ccc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 0x558926cd60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd2720 0x558926cc6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926cc8ec0 con 0x558926ccc890 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926ced7b0 con 0x558926cd1e00 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926cc7290 con 0x558926cd2720 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4fead9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 0x558926ccc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4fead9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 0x558926ccc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33174/0 (socket says 172.21.15.12:33174) 2024-05-09T00:06:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4fead9d700 1 -- 172.21.15.12:0/182833993 learned_addr learned my addr 172.21.15.12:0/182833993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4feb59e700 1 --2- 172.21.15.12:0/182833993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd2720 0x558926cc6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4fea59c700 1 --2- 172.21.15.12:0/182833993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 0x558926cd60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4fead9d700 1 -- 172.21.15.12:0/182833993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd2720 msgr2=0x558926cc6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4fead9d700 1 --2- 172.21.15.12:0/182833993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd2720 0x558926cc6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4fead9d700 1 -- 172.21.15.12:0/182833993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 msgr2=0x558926cd60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4fead9d700 1 --2- 172.21.15.12:0/182833993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 0x558926cd60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.851+0000 7f4fead9d700 1 -- 172.21.15.12:0/182833993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558926cccf40 con 0x558926ccc890 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.852+0000 7f4fead9d700 1 --2- 172.21.15.12:0/182833993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 0x558926ccc350 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc01cf70 tx=0x7f4fdc0091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d53bef054ecaff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.852+0000 7f4fd1ffb700 1 -- 172.21.15.12:0/182833993 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc005040 con 0x558926ccc890 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.852+0000 7f4fd1ffb700 1 -- 172.21.15.12:0/182833993 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fdc005660 con 0x558926ccc890 2024-05-09T00:06:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.852+0000 7f4fd1ffb700 1 -- 172.21.15.12:0/182833993 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc004710 con 0x558926ccc890 2024-05-09T00:06:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/182833993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 msgr2=0x558926ccc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/182833993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 0x558926ccc350 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc01cf70 tx=0x7f4fdc0091e0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/182833993 shutdown_connections 2024-05-09T00:06:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/182833993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926cd2720 0x558926cc6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/182833993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926ccc890 0x558926ccc350 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/182833993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd1e00 0x558926cd60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/182833993 >> 172.21.15.12:0/182833993 conn(0x558926c2a210 msgr2=0x558926ccfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/182833993 shutdown_connections 2024-05-09T00:06:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/182833993 wait complete. 2024-05-09T00:06:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 -- start start 2024-05-09T00:06:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 Processor -- start 2024-05-09T00:06:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.853+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 0x558926cdcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd1e00 0x558926cdd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4ffd3dcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 0x558926cdb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926cdba20 con 0x558926cd1e00 2024-05-09T00:06:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926cdbb90 con 0x558926cd2720 2024-05-09T00:06:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4ffd3dcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558926cdbd50 con 0x558926ccc890 2024-05-09T00:06:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4ffd3dcb80 10 client.0 _ll_register_callbacks cb 0x7f4fe4008ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fea59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd1e00 0x558926cdd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fea59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd1e00 0x558926cdd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33188/0 (socket says 172.21.15.12:33188) 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fea59c700 1 -- 172.21.15.12:0/882493032 learned_addr learned my addr 172.21.15.12:0/882493032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4feb59e700 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 0x558926cdb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fead9d700 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 0x558926cdcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fea59c700 1 -- 172.21.15.12:0/882493032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 msgr2=0x558926cdcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fea59c700 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 0x558926cdcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fea59c700 1 -- 172.21.15.12:0/882493032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 msgr2=0x558926cdb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fea59c700 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 0x558926cdb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fea59c700 1 -- 172.21.15.12:0/882493032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fdc02a040 con 0x558926cd1e00 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fead9d700 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 0x558926cdcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fea59c700 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd1e00 0x558926cdd4b0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00968a0 tx=0x7f4fe00961f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8adef58e3c58bf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fceff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fceff5700 1 -- 172.21.15.12:0/882493032 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0099520 con 0x558926cd1e00 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/882493032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fdc021070 con 0x558926cd1e00 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.854+0000 7f4fceff5700 1 -- 172.21.15.12:0/882493032 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4fe009ae40 con 0x558926cd1e00 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4fceff5700 1 -- 172.21.15.12:0/882493032 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fe009dc60 con 0x558926cd1e00 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4ffd3dcb80 10 client.20955 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/882493032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x558926be2560 con 0x558926cd1e00 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4fceff5700 1 -- 172.21.15.12:0/882493032 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fe007c020 con 0x558926cd1e00 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4fceff5700 1 -- 172.21.15.12:0/882493032 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4fe00cabc0 con 0x558926cd1e00 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4ffd3dcb80 10 client.20955 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4ffd3dcb80 10 client.20955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4ffd3dcb80 20 client.20955 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558926d3d8f0 0x558926d3fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4ffd3dcb80 4 client.20955 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/882493032 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558926c14ec0 con 0x558926d3d8f0 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.855+0000 7f4fead9d700 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558926d3d8f0 0x558926d3fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.856+0000 7f4fceff5700 10 client.20955 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.856+0000 7f4fead9d700 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558926d3d8f0 0x558926d3fd10 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c633b534c0af9a78 server_cookie=627f6e08ae25d53c in_seq=0 out_seq=0 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4fceff5700 1 -- 172.21.15.12:0/882493032 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558926c14ec0 con 0x558926d3d8f0 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4fceff5700 10 client.20955 handle_command_reply: tid=0 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 client.20955 shutdown 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/882493032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd1e00 msgr2=0x558926cdd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd1e00 0x558926cdd4b0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00968a0 tx=0x7f4fe00961f0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/882493032 shutdown_connections 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558926d3d8f0 0x558926d3fd10 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558926cd2720 0x558926cdb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4fceff5700 0 client.20955 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558926ccc890 0x558926cdcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 --2- 172.21.15.12:0/882493032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558926cd1e00 0x558926cdd4b0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/882493032 >> 172.21.15.12:0/882493032 conn(0x558926c2a210 msgr2=0x558926cd4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/882493032 shutdown_connections 2024-05-09T00:06:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 1 -- 172.21.15.12:0/882493032 wait complete. 2024-05-09T00:06:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ffd3dcb80 20 client.20955 trim_cache size 0 max 16384 2024-05-09T00:06:19.858 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.858+0000 7f4ff1222700 1 -- 172.21.15.12:0/321918096 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fd40a1310 msgr2=0x7f4fd40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fd40a1310 0x7f4fd40a37d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009d20 tx=0x7f4fdc009260 comp rx=0 tx=0).stop 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 -- 172.21.15.12:0/321918096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fec068970 msgr2=0x7f4fec199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fec068970 0x7f4fec199cc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9f0 tx=0x7f4fe000ed00 comp rx=0 tx=0).stop 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 -- 172.21.15.12:0/321918096 shutdown_connections 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4fd40a1310 0x7f4fd40a37d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fec10d940 0x7f4fec19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fec068050 0x7f4fec199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 --2- 172.21.15.12:0/321918096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fec068970 0x7f4fec199cc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 -- 172.21.15.12:0/321918096 >> 172.21.15.12:0/321918096 conn(0x7f4fec076f40 msgr2=0x7f4fec10ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 -- 172.21.15.12:0/321918096 shutdown_connections 2024-05-09T00:06:19.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:19.859+0000 7f4ff1222700 1 -- 172.21.15.12:0/321918096 wait complete. 2024-05-09T00:06:20.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 243547 33.07 MB/sec execute 255 sec latency 126.875 ms 2024-05-09T00:06:20.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:20 smithi204 ceph-mon[47373]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-05-09T00:06:20.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:20 smithi160 ceph-mon[46152]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-05-09T00:06:20.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:20 smithi012 ceph-mon[42503]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-05-09T00:06:20.868 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:06:20.868 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.037+0000 7f868eaa0700 1 -- 172.21.15.12:0/814230894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f868810d940 msgr2=0x7f868810fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.038+0000 7f868eaa0700 1 --2- 172.21.15.12:0/814230894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f868810d940 0x7f868810fd30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f868000b0a0 tx=0x7f868000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.038+0000 7f868eaa0700 1 -- 172.21.15.12:0/814230894 shutdown_connections 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.038+0000 7f868eaa0700 1 --2- 172.21.15.12:0/814230894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f868810d940 0x7f868810fd30 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.038+0000 7f868eaa0700 1 --2- 172.21.15.12:0/814230894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8688069250 0x7f8688069630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.038+0000 7f868eaa0700 1 --2- 172.21.15.12:0/814230894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688069b70 0x7f868810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.038+0000 7f868eaa0700 1 -- 172.21.15.12:0/814230894 >> 172.21.15.12:0/814230894 conn(0x7f8688076f30 msgr2=0x7f8688077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.038+0000 7f868eaa0700 1 -- 172.21.15.12:0/814230894 shutdown_connections 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.038+0000 7f868eaa0700 1 -- 172.21.15.12:0/814230894 wait complete. 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.038+0000 7f868eaa0700 1 Processor -- start 2024-05-09T00:06:21.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.038+0000 7f868eaa0700 1 -- start start 2024-05-09T00:06:21.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868eaa0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688069250 0x7f86881996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868eaa0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688069b70 0x7f8688199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868eaa0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 0x7f868819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868eaa0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868819e600 con 0x7f8688069b70 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868eaa0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868819e770 con 0x7f8688069250 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868eaa0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868819e930 con 0x7f868810d940 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f867ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688069b70 0x7f8688199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f867ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688069b70 0x7f8688199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33214/0 (socket says 172.21.15.12:33214) 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f867ffff700 1 -- 172.21.15.12:0/432872677 learned_addr learned my addr 172.21.15.12:0/432872677 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868c83c700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688069250 0x7f86881996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868d03d700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 0x7f868819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868c83c700 1 -- 172.21.15.12:0/432872677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 msgr2=0x7f868819dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868c83c700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 0x7f868819dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868c83c700 1 -- 172.21.15.12:0/432872677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688069b70 msgr2=0x7f8688199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868c83c700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688069b70 0x7f8688199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868c83c700 1 -- 172.21.15.12:0/432872677 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680009d00 con 0x7f8688069250 2024-05-09T00:06:21.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f867ffff700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688069b70 0x7f8688199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:21.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868d03d700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 0x7f868819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:21.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.039+0000 7f868c83c700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688069250 0x7f86881996e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f868400e9f0 tx=0x7f868400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.040+0000 7f867dffb700 1 -- 172.21.15.12:0/432872677 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868400c9c0 con 0x7f8688069250 2024-05-09T00:06:21.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.040+0000 7f868eaa0700 1 -- 172.21.15.12:0/432872677 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868819ec10 con 0x7f8688069250 2024-05-09T00:06:21.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.040+0000 7f867dffb700 1 -- 172.21.15.12:0/432872677 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8684010430 con 0x7f8688069250 2024-05-09T00:06:21.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.040+0000 7f867dffb700 1 -- 172.21.15.12:0/432872677 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684018680 con 0x7f8688069250 2024-05-09T00:06:21.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.040+0000 7f868eaa0700 1 -- 172.21.15.12:0/432872677 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86881a6450 con 0x7f8688069250 2024-05-09T00:06:21.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.043+0000 7f867dffb700 1 -- 172.21.15.12:0/432872677 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8684010860 con 0x7f8688069250 2024-05-09T00:06:21.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.043+0000 7f867dffb700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86700a1290 0x7f86700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.043+0000 7f867ffff700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86700a1290 0x7f86700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.043+0000 7f867dffb700 1 -- 172.21.15.12:0/432872677 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8684014070 con 0x7f8688069250 2024-05-09T00:06:21.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.044+0000 7f867ffff700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86700a1290 0x7f86700a3750 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f868819ac30 tx=0x7f8678009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.044+0000 7f869ac5ab80 1 Processor -- start 2024-05-09T00:06:21.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.044+0000 7f869ac5ab80 1 -- start start 2024-05-09T00:06:21.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.044+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382ba260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.044+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.044+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.044+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113820b1d0 con 0x5611382ba7a0 2024-05-09T00:06:21.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.044+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561138208b90 con 0x5611382b9e80 2024-05-09T00:06:21.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.044+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561138204480 con 0x5611382b4910 2024-05-09T00:06:21.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f867ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f868d03d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f867ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33220/0 (socket says 172.21.15.12:33220) 2024-05-09T00:06:21.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f867ffff700 1 -- 172.21.15.12:0/3396786662 learned_addr learned my addr 172.21.15.12:0/3396786662 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:21.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f867ffff700 1 -- 172.21.15.12:0/3396786662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 msgr2=0x5611382b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f867ffff700 1 --2- 172.21.15.12:0/3396786662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f867ffff700 1 -- 172.21.15.12:0/3396786662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 msgr2=0x5611382ba260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f867ffff700 1 --2- 172.21.15.12:0/3396786662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382ba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f867ffff700 1 -- 172.21.15.12:0/3396786662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561138085430 con 0x5611382ba7a0 2024-05-09T00:06:21.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f868d03d700 1 --2- 172.21.15.12:0/3396786662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:21.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f867ffff700 1 --2- 172.21.15.12:0/3396786662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b43d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f867800e730 tx=0x7f8678014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e15213b621facb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f866effd700 1 -- 172.21.15.12:0/3396786662 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678004830 con 0x5611382ba7a0 2024-05-09T00:06:21.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f866effd700 1 -- 172.21.15.12:0/3396786662 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8678004e50 con 0x5611382ba7a0 2024-05-09T00:06:21.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.045+0000 7f866effd700 1 -- 172.21.15.12:0/3396786662 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867801e660 con 0x5611382ba7a0 2024-05-09T00:06:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.046+0000 7f869ac5ab80 1 -- 172.21.15.12:0/3396786662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 msgr2=0x5611382b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.046+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/3396786662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b43d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f867800e730 tx=0x7f8678014000 comp rx=0 tx=0).stop 2024-05-09T00:06:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.046+0000 7f869ac5ab80 1 -- 172.21.15.12:0/3396786662 shutdown_connections 2024-05-09T00:06:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.046+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/3396786662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.046+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/3396786662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382ba260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.046+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/3396786662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b43d0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.046+0000 7f869ac5ab80 1 -- 172.21.15.12:0/3396786662 >> 172.21.15.12:0/3396786662 conn(0x561138212290 msgr2=0x5611382aee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.046+0000 7f869ac5ab80 1 -- 172.21.15.12:0/3396786662 shutdown_connections 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.046+0000 7f869ac5ab80 1 -- 172.21.15.12:0/3396786662 wait complete. 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.047+0000 7f869ac5ab80 1 -- start start 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.047+0000 7f869ac5ab80 1 Processor -- start 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.047+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.047+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382b9e80 0x5611382be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.047+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382ba7a0 0x5611382be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.047+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611381a05e0 con 0x5611382b9e80 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.047+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611381a0280 con 0x5611382ba7a0 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.047+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561138087720 con 0x5611382b4910 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f869ac5ab80 10 client.0 _ll_register_callbacks cb 0x5611381a9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f868c83c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f867ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382b9e80 0x5611382be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f868c83c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382bdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46140/0 (socket says 172.21.15.12:46140) 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f868c83c700 1 -- 172.21.15.12:0/313275228 learned_addr learned my addr 172.21.15.12:0/313275228 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f868d03d700 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382ba7a0 0x5611382be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f868c83c700 1 -- 172.21.15.12:0/313275228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382ba7a0 msgr2=0x5611382be6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f868c83c700 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382ba7a0 0x5611382be6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f868c83c700 1 -- 172.21.15.12:0/313275228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382b9e80 msgr2=0x5611382be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f868c83c700 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382b9e80 0x5611382be1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f868c83c700 1 -- 172.21.15.12:0/313275228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8688110860 con 0x5611382b4910 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f867ffff700 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382b9e80 0x5611382be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f868c83c700 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382bdc70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8684099c10 tx=0x7f8684099960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d42f7957a1f440 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f8667fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f8667fff700 1 -- 172.21.15.12:0/313275228 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684099600 con 0x5611382b4910 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f869ac5ab80 1 -- 172.21.15.12:0/313275228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56113815d2f0 con 0x5611382b4910 2024-05-09T00:06:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.048+0000 7f8667fff700 1 -- 172.21.15.12:0/313275228 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8684099f10 con 0x5611382b4910 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f8667fff700 1 -- 172.21.15.12:0/313275228 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f868409dd40 con 0x5611382b4910 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f869ac5ab80 10 client.30667 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f869ac5ab80 1 -- 172.21.15.12:0/313275228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7f8678017070 con 0x5611382b4910 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f8667fff700 1 -- 172.21.15.12:0/313275228 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56113815d2f0 con 0x5611382b4910 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f8667fff700 1 -- 172.21.15.12:0/313275228 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f868400cb20 con 0x5611382b4910 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f869ac5ab80 10 client.30667 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f869ac5ab80 10 client.30667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f869ac5ab80 20 client.30667 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5611382ce220 0x5611382d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f869ac5ab80 4 client.30667 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.049+0000 7f869ac5ab80 1 -- 172.21.15.12:0/313275228 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561138283890 con 0x5611382ce220 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.050+0000 7f867ffff700 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5611382ce220 0x5611382d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.050+0000 7f867ffff700 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5611382ce220 0x5611382d0640 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d76b47922ab6c08 server_cookie=703d9509f512ee44 in_seq=0 out_seq=0 2024-05-09T00:06:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.050+0000 7f8667fff700 10 client.30667 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f8667fff700 1 -- 172.21.15.12:0/313275228 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561138283890 con 0x5611382ce220 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f8667fff700 10 client.30667 handle_command_reply: tid=0 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 client.30667 shutdown 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 -- 172.21.15.12:0/313275228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 msgr2=0x5611382bdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382bdc70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8684099c10 tx=0x7f8684099960 comp rx=0 tx=0).stop 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 -- 172.21.15.12:0/313275228 shutdown_connections 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5611382ce220 0x5611382d0640 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382ba7a0 0x5611382be6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382bdc70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f8667fff700 0 client.30667 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/313275228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382b9e80 0x5611382be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 -- 172.21.15.12:0/313275228 >> 172.21.15.12:0/313275228 conn(0x561138212290 msgr2=0x5611382ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 -- 172.21.15.12:0/313275228 shutdown_connections 2024-05-09T00:06:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 1 -- 172.21.15.12:0/313275228 wait complete. 2024-05-09T00:06:21.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.051+0000 7f869ac5ab80 20 client.30667 trim_cache size 0 max 16384 2024-05-09T00:06:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.071+0000 7f869ac5ab80 1 Processor -- start 2024-05-09T00:06:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.071+0000 7f869ac5ab80 1 -- start start 2024-05-09T00:06:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.071+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611382bf220 con 0x5611382ba7a0 2024-05-09T00:06:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611382abe40 con 0x5611382b9e80 2024-05-09T00:06:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611382b9990 con 0x5611382b4910 2024-05-09T00:06:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f868d03d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f868d03d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33240/0 (socket says 172.21.15.12:33240) 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f868d03d700 1 -- 172.21.15.12:0/3792012261 learned_addr learned my addr 172.21.15.12:0/3792012261 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f868c83c700 1 --2- 172.21.15.12:0/3792012261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f867ffff700 1 --2- 172.21.15.12:0/3792012261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f868d03d700 1 -- 172.21.15.12:0/3792012261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 msgr2=0x5611382b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f868d03d700 1 --2- 172.21.15.12:0/3792012261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f868d03d700 1 -- 172.21.15.12:0/3792012261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 msgr2=0x5611382c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f868d03d700 1 --2- 172.21.15.12:0/3792012261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f868d03d700 1 -- 172.21.15.12:0/3792012261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611382ab560 con 0x5611382ba7a0 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f868d03d700 1 --2- 172.21.15.12:0/3792012261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b9450 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f8680015000 tx=0x7f8680009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0a68321a56c21f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.072+0000 7f866f7fe700 1 -- 172.21.15.12:0/3792012261 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000e040 con 0x5611382ba7a0 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f866f7fe700 1 -- 172.21.15.12:0/3792012261 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f868000bd30 con 0x5611382ba7a0 2024-05-09T00:06:21.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f866f7fe700 1 -- 172.21.15.12:0/3792012261 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680005260 con 0x5611382ba7a0 2024-05-09T00:06:21.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f869ac5ab80 1 -- 172.21.15.12:0/3792012261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 msgr2=0x5611382b9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/3792012261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b9450 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f8680015000 tx=0x7f8680009520 comp rx=0 tx=0).stop 2024-05-09T00:06:21.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f869ac5ab80 1 -- 172.21.15.12:0/3792012261 shutdown_connections 2024-05-09T00:06:21.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/3792012261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382b9450 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/3792012261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/3792012261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f869ac5ab80 1 -- 172.21.15.12:0/3792012261 >> 172.21.15.12:0/3792012261 conn(0x561138212290 msgr2=0x5611382b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f869ac5ab80 1 -- 172.21.15.12:0/3792012261 shutdown_connections 2024-05-09T00:06:21.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.073+0000 7f869ac5ab80 1 -- 172.21.15.12:0/3792012261 wait complete. 2024-05-09T00:06:21.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.074+0000 7f869ac5ab80 1 -- start start 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.074+0000 7f869ac5ab80 1 Processor -- start 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.074+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.074+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f869ac5ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611382c4690 con 0x5611382ba7a0 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611382c4800 con 0x5611382b9e80 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f869ac5ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561138324730 con 0x5611382b4910 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f869ac5ab80 10 client.0 _ll_register_callbacks cb 0x5611381a9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f868d03d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f867ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f868d03d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33256/0 (socket says 172.21.15.12:33256) 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f868c83c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f868d03d700 1 -- 172.21.15.12:0/371853039 learned_addr learned my addr 172.21.15.12:0/371853039 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f868d03d700 1 -- 172.21.15.12:0/371853039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 msgr2=0x5611382c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f868d03d700 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f868d03d700 1 -- 172.21.15.12:0/371853039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 msgr2=0x5611382c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f868d03d700 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382c3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f868d03d700 1 -- 172.21.15.12:0/371853039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86800092e0 con 0x5611382ba7a0 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f868d03d700 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382c3fb0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8680009fd0 tx=0x7f8680005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a09b3b93dc7a58c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f8667fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.075+0000 7f8667fff700 1 -- 172.21.15.12:0/371853039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000e050 con 0x5611382ba7a0 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f8667fff700 1 -- 172.21.15.12:0/371853039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8680003e30 con 0x5611382ba7a0 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f869ac5ab80 1 -- 172.21.15.12:0/371853039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8678004e50 con 0x5611382ba7a0 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f8667fff700 1 -- 172.21.15.12:0/371853039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8680007700 con 0x5611382ba7a0 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f869ac5ab80 10 client.20973 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f869ac5ab80 1 -- 172.21.15.12:0/371853039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x561138031be0 con 0x5611382ba7a0 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f8667fff700 1 -- 172.21.15.12:0/371853039 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8678004e50 con 0x5611382ba7a0 2024-05-09T00:06:21.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f8667fff700 1 -- 172.21.15.12:0/371853039 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f868002e030 con 0x5611382ba7a0 2024-05-09T00:06:21.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f869ac5ab80 10 client.20973 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:21.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f869ac5ab80 10 client.20973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:21.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f869ac5ab80 20 client.20973 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:21.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561138325b90 0x561138327fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f869ac5ab80 4 client.20973 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:06:21.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f869ac5ab80 1 -- 172.21.15.12:0/371853039 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5611381fcf40 con 0x561138325b90 2024-05-09T00:06:21.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.076+0000 7f868c83c700 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561138325b90 0x561138327fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.077+0000 7f868c83c700 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561138325b90 0x561138327fb0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4b214de9c2ea3f server_cookie=55167cfc1a7672b8 in_seq=0 out_seq=0 2024-05-09T00:06:21.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.077+0000 7f8667fff700 10 client.20973 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:21.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f8667fff700 1 -- 172.21.15.12:0/371853039 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5611381fcf40 con 0x561138325b90 2024-05-09T00:06:21.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f8667fff700 10 client.20973 handle_command_reply: tid=0 2024-05-09T00:06:21.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 client.20973 shutdown 2024-05-09T00:06:21.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 -- 172.21.15.12:0/371853039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 msgr2=0x5611382c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382c3fb0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8680009fd0 tx=0x7f8680005280 comp rx=0 tx=0).stop 2024-05-09T00:06:21.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 -- 172.21.15.12:0/371853039 shutdown_connections 2024-05-09T00:06:21.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611382ba7a0 0x5611382c3fb0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611382b4910 0x5611382c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561138325b90 0x561138327fb0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 --2- 172.21.15.12:0/371853039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611382b9e80 0x5611382c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f8667fff700 0 client.20973 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:21.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 -- 172.21.15.12:0/371853039 >> 172.21.15.12:0/371853039 conn(0x561138212290 msgr2=0x5611382bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 -- 172.21.15.12:0/371853039 shutdown_connections 2024-05-09T00:06:21.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 1 -- 172.21.15.12:0/371853039 wait complete. 2024-05-09T00:06:21.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.082+0000 7f869ac5ab80 20 client.20973 trim_cache size 0 max 16384 2024-05-09T00:06:21.079 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:06:21.079 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:06:21.079 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "c5e784fc-8813-4819-a2de-3d1c448de260", 2024-05-09T00:06:21.079 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:06:21.079 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 -- 172.21.15.12:0/432872677 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86700a1290 msgr2=0x7f86700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86700a1290 0x7f86700a3750 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f868819ac30 tx=0x7f8678009260 comp rx=0 tx=0).stop 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 -- 172.21.15.12:0/432872677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688069250 msgr2=0x7f86881996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688069250 0x7f86881996e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f868400e9f0 tx=0x7f868400ed00 comp rx=0 tx=0).stop 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 -- 172.21.15.12:0/432872677 shutdown_connections 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f86700a1290 0x7f86700a3750 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868810d940 0x7f868819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8688069250 0x7f86881996e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 --2- 172.21.15.12:0/432872677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8688069b70 0x7f8688199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 -- 172.21.15.12:0/432872677 >> 172.21.15.12:0/432872677 conn(0x7f8688076f30 msgr2=0x7f868810ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 -- 172.21.15.12:0/432872677 shutdown_connections 2024-05-09T00:06:21.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.083+0000 7f868eaa0700 1 -- 172.21.15.12:0/432872677 wait complete. 2024-05-09T00:06:21.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:06:21.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.257+0000 7feb01e1a700 1 -- 172.21.15.12:0/1185262814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feafc100060 msgr2=0x7feafc100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.257+0000 7feb01e1a700 1 --2- 172.21.15.12:0/1185262814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feafc100060 0x7feafc100440 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7feaec009990 tx=0x7feaec009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.257+0000 7feb01e1a700 1 -- 172.21.15.12:0/1185262814 shutdown_connections 2024-05-09T00:06:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.257+0000 7feb01e1a700 1 --2- 172.21.15.12:0/1185262814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc105160 0x7feafc10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.257+0000 7feb01e1a700 1 --2- 172.21.15.12:0/1185262814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feafc100060 0x7feafc100440 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.257+0000 7feb01e1a700 1 --2- 172.21.15.12:0/1185262814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feafc100a10 0x7feafc104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.257+0000 7feb01e1a700 1 -- 172.21.15.12:0/1185262814 >> 172.21.15.12:0/1185262814 conn(0x7feafc076f40 msgr2=0x7feafc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.257+0000 7feb01e1a700 1 -- 172.21.15.12:0/1185262814 shutdown_connections 2024-05-09T00:06:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.257+0000 7feb01e1a700 1 -- 172.21.15.12:0/1185262814 wait complete. 2024-05-09T00:06:21.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.258+0000 7feb01e1a700 1 Processor -- start 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feb01e1a700 1 -- start start 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feb01e1a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc100060 0x7feafc1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feb01e1a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feafc105160 0x7feafc196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feb01e1a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feafc19abb0 0x7feafc19b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feb01e1a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc19b630 con 0x7feafc105160 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feb01e1a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc19b7a0 con 0x7feafc100060 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feb01e1a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc19b960 con 0x7feafc19abb0 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feafaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feafc105160 0x7feafc196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feafb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc100060 0x7feafc1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feafaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feafc105160 0x7feafc196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33282/0 (socket says 172.21.15.12:33282) 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feafaffd700 1 -- 172.21.15.12:0/1508943172 learned_addr learned my addr 172.21.15.12:0/1508943172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feafbfff700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feafc19abb0 0x7feafc19b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feafaffd700 1 -- 172.21.15.12:0/1508943172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feafc19abb0 msgr2=0x7feafc19b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feafaffd700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feafc19abb0 0x7feafc19b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feafaffd700 1 -- 172.21.15.12:0/1508943172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc100060 msgr2=0x7feafc1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feafaffd700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc100060 0x7feafc1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.259+0000 7feafaffd700 1 -- 172.21.15.12:0/1508943172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec0095f0 con 0x7feafc105160 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.260+0000 7feafbfff700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feafc19abb0 0x7feafc19b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.260+0000 7feafb7fe700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc100060 0x7feafc1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.260+0000 7feafaffd700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feafc105160 0x7feafc196810 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7feaf000e970 tx=0x7feaf000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.260+0000 7feaf8ff9700 1 -- 172.21.15.12:0/1508943172 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0009790 con 0x7feafc105160 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.260+0000 7feaf8ff9700 1 -- 172.21.15.12:0/1508943172 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaf000c7e0 con 0x7feafc105160 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.260+0000 7feb01e1a700 1 -- 172.21.15.12:0/1508943172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feafc1a2ec0 con 0x7feafc105160 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.260+0000 7feaf8ff9700 1 -- 172.21.15.12:0/1508943172 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00054e0 con 0x7feafc105160 2024-05-09T00:06:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.260+0000 7feb01e1a700 1 -- 172.21.15.12:0/1508943172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feafc1a3390 con 0x7feafc105160 2024-05-09T00:06:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.264+0000 7feaf8ff9700 1 -- 172.21.15.12:0/1508943172 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7feaf000c950 con 0x7feafc105160 2024-05-09T00:06:21.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feaf8ff9700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feae40a1310 0x7feae40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feb0dfd4b80 1 Processor -- start 2024-05-09T00:06:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feaf8ff9700 1 -- 172.21.15.12:0/1508943172 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7feaf0014070 con 0x7feafc105160 2024-05-09T00:06:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feb0dfd4b80 1 -- start start 2024-05-09T00:06:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feafb7fe700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feae40a1310 0x7feae40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4ae00 0x557aacf4b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4b720 0x557aacf45350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aace9c150 con 0x557aacf4b720 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aace99b10 con 0x557aacf4ae00 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.265+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aace95400 con 0x557aacf45890 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.266+0000 7feafb7fe700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feae40a1310 0x7feae40a37d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7feaec00b2e0 tx=0x7feaec005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.266+0000 7feafbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.266+0000 7feafbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46186/0 (socket says 172.21.15.12:46186) 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.266+0000 7feafbfff700 1 -- 172.21.15.12:0/1982230358 learned_addr learned my addr 172.21.15.12:0/1982230358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.266+0000 7feafbfff700 1 -- 172.21.15.12:0/1982230358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4ae00 msgr2=0x557aacf4b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.266+0000 7feafbfff700 1 --2- 172.21.15.12:0/1982230358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4ae00 0x557aacf4b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.266+0000 7feafbfff700 1 -- 172.21.15.12:0/1982230358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4b720 msgr2=0x557aacf45350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:06:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.266+0000 7feafbfff700 1 --2- 172.21.15.12:0/1982230358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4b720 0x557aacf45350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.266+0000 7feafbfff700 1 -- 172.21.15.12:0/1982230358 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aacd163b0 con 0x557aacf45890 2024-05-09T00:06:21.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.266+0000 7feafbfff700 1 --2- 172.21.15.12:0/1982230358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45cd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009550 tx=0x7feaf40079f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7364bb79e9d0d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.267+0000 7feae1ffb700 1 -- 172.21.15.12:0/1982230358 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4015850 con 0x557aacf45890 2024-05-09T00:06:21.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.267+0000 7feae1ffb700 1 -- 172.21.15.12:0/1982230358 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaf400f040 con 0x557aacf45890 2024-05-09T00:06:21.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.267+0000 7feae1ffb700 1 -- 172.21.15.12:0/1982230358 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4014d50 con 0x557aacf45890 2024-05-09T00:06:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.268+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/1982230358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 msgr2=0x557aacf45cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.268+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/1982230358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45cd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009550 tx=0x7feaf40079f0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.268+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/1982230358 shutdown_connections 2024-05-09T00:06:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.268+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/1982230358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45cd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.268+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/1982230358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4ae00 0x557aacf4b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.268+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/1982230358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4b720 0x557aacf45350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.268+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/1982230358 >> 172.21.15.12:0/1982230358 conn(0x557aacea3210 msgr2=0x557aacf3fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.268+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/1982230358 shutdown_connections 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.268+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/1982230358 wait complete. 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.268+0000 7feb0dfd4b80 1 -- start start 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feb0dfd4b80 1 Processor -- start 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf47900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf47e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4b720 0x557aacf48380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aace31560 con 0x557aacf4ae00 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aace31200 con 0x557aacf4b720 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aacd186a0 con 0x557aacf45890 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feb0dfd4b80 10 client.0 _ll_register_callbacks cb 0x557aace3ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feafaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf47e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feafaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf47e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33284/0 (socket says 172.21.15.12:33284) 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.269+0000 7feafaffd700 1 -- 172.21.15.12:0/617302473 learned_addr learned my addr 172.21.15.12:0/617302473 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feafbfff700 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4b720 0x557aacf48380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feafb7fe700 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf47900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feafaffd700 1 -- 172.21.15.12:0/617302473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 msgr2=0x557aacf47900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feafaffd700 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf47900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feafaffd700 1 -- 172.21.15.12:0/617302473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4b720 msgr2=0x557aacf48380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feafaffd700 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4b720 0x557aacf48380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feafaffd700 1 -- 172.21.15.12:0/617302473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf4009e30 con 0x557aacf4ae00 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feafaffd700 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf47e40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7feaf00ca660 tx=0x7feaf000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb49943fca256e65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feadaffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feadaffd700 1 -- 172.21.15.12:0/617302473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf009ddf0 con 0x557aacf4ae00 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feadaffd700 1 -- 172.21.15.12:0/617302473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7feaf00a2440 con 0x557aacf4ae00 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.270+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/617302473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557aacdee270 con 0x557aacf4ae00 2024-05-09T00:06:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feadaffd700 1 -- 172.21.15.12:0/617302473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaf00b82f0 con 0x557aacf4ae00 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feb0dfd4b80 10 client.20982 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/617302473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x557aaccc2b60 con 0x557aacf4ae00 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feadaffd700 1 -- 172.21.15.12:0/617302473 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557aacdee270 con 0x557aacf4ae00 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feadaffd700 1 -- 172.21.15.12:0/617302473 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feaf0099590 con 0x557aacf4ae00 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feb0dfd4b80 10 client.20982 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feb0dfd4b80 10 client.20982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feb0dfd4b80 20 client.20982 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557aacf78bd0 0x557aacf7aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feb0dfd4b80 4 client.20982 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.271+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/617302473 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557aacf14810 con 0x557aacf78bd0 2024-05-09T00:06:21.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.272+0000 7feafb7fe700 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557aacf78bd0 0x557aacf7aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.272+0000 7feafb7fe700 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557aacf78bd0 0x557aacf7aff0 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b21886bc08279c4 server_cookie=95e7921554b3a006 in_seq=0 out_seq=0 2024-05-09T00:06:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.272+0000 7feadaffd700 10 client.20982 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:21.270 INFO:tasks.workunit.client.0.smithi012.stdout: 10 244132 33.06 MB/sec execute 256 sec latency 109.024 ms 2024-05-09T00:06:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.277+0000 7feadaffd700 1 -- 172.21.15.12:0/617302473 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557aacf14810 con 0x557aacf78bd0 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.277+0000 7feadaffd700 10 client.20982 handle_command_reply: tid=0 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.277+0000 7feb0dfd4b80 1 client.20982 shutdown 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/617302473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 msgr2=0x557aacf47e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf47e40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7feaf00ca660 tx=0x7feaf000ccd0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/617302473 shutdown_connections 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557aacf78bd0 0x557aacf7aff0 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4b720 0x557aacf48380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf47900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/617302473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf47e40 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feadaffd700 0 client.20982 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/617302473 >> 172.21.15.12:0/617302473 conn(0x557aacea3210 msgr2=0x557aacf3f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/617302473 shutdown_connections 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/617302473 wait complete. 2024-05-09T00:06:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.278+0000 7feb0dfd4b80 20 client.20982 trim_cache size 0 max 16384 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feb0dfd4b80 1 Processor -- start 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feb0dfd4b80 1 -- start start 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf3fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4b720 0x557aacf51650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aacf55de0 con 0x557aacf4ae00 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aacf4db10 con 0x557aacf4b720 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aacf4ea30 con 0x557aacf45890 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf3fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafb7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf3fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33294/0 (socket says 172.21.15.12:33294) 2024-05-09T00:06:21.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafaffd700 1 -- 172.21.15.12:0/531181115 learned_addr learned my addr 172.21.15.12:0/531181115 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafbfff700 1 --2- 172.21.15.12:0/531181115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4b720 0x557aacf51650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafb7fe700 1 -- 172.21.15.12:0/531181115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4b720 msgr2=0x557aacf51650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafb7fe700 1 --2- 172.21.15.12:0/531181115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4b720 0x557aacf51650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafb7fe700 1 -- 172.21.15.12:0/531181115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 msgr2=0x557aacf3fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafb7fe700 1 --2- 172.21.15.12:0/531181115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf3fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafb7fe700 1 -- 172.21.15.12:0/531181115 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aace5b560 con 0x557aacf45890 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.299+0000 7feafaffd700 1 --2- 172.21.15.12:0/531181115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf3fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.300+0000 7feafb7fe700 1 --2- 172.21.15.12:0/531181115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45c70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7feaec0059d0 tx=0x7feaec020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16809b67d0b37ccc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.300+0000 7feae27fc700 1 -- 172.21.15.12:0/531181115 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec031430 con 0x557aacf45890 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.300+0000 7feae27fc700 1 -- 172.21.15.12:0/531181115 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaec02a040 con 0x557aacf45890 2024-05-09T00:06:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.300+0000 7feae27fc700 1 -- 172.21.15.12:0/531181115 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec024670 con 0x557aacf45890 2024-05-09T00:06:21.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.301+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/531181115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 msgr2=0x557aacf45c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.301+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/531181115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45c70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7feaec0059d0 tx=0x7feaec020000 comp rx=0 tx=0).stop 2024-05-09T00:06:21.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.301+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/531181115 shutdown_connections 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.301+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/531181115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4b720 0x557aacf51650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.301+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/531181115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf45890 0x557aacf45c70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.301+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/531181115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf4ae00 0x557aacf3fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.301+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/531181115 >> 172.21.15.12:0/531181115 conn(0x557aacea3210 msgr2=0x557aacf417e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.301+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/531181115 shutdown_connections 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.301+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/531181115 wait complete. 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.301+0000 7feb0dfd4b80 1 -- start start 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feb0dfd4b80 1 Processor -- start 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf45890 0x557aacf64570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4ae00 0x557aacf4fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feb0dfd4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf4b720 0x557aacf505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aacf50cf0 con 0x557aacf45890 2024-05-09T00:06:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aacfce080 con 0x557aacf4ae00 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feb0dfd4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aacfce1f0 con 0x557aacf4b720 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feb0dfd4b80 10 client.0 _ll_register_callbacks cb 0x557aace3ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feafb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf45890 0x557aacf64570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feafaffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4ae00 0x557aacf4fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feafbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf4b720 0x557aacf505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feafaffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4ae00 0x557aacf4fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58454/0 (socket says 172.21.15.12:58454) 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feafb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf45890 0x557aacf64570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33310/0 (socket says 172.21.15.12:33310) 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feafbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf4b720 0x557aacf505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46228/0 (socket says 172.21.15.12:46228) 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.302+0000 7feafaffd700 1 -- 172.21.15.12:0/2120508636 learned_addr learned my addr 172.21.15.12:0/2120508636 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feafbfff700 1 -- 172.21.15.12:0/2120508636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4ae00 msgr2=0x557aacf4fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feafbfff700 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4ae00 0x557aacf4fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feafbfff700 1 -- 172.21.15.12:0/2120508636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf45890 msgr2=0x557aacf64570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feafbfff700 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf45890 0x557aacf64570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feafbfff700 1 -- 172.21.15.12:0/2120508636 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557aaccc2b60 con 0x557aacf4b720 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feafbfff700 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf4b720 0x557aacf505b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000f80 tx=0x7feaf4004f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe4c2784fa40526d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feadaffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feadaffd700 1 -- 172.21.15.12:0/2120508636 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4024070 con 0x557aacf4b720 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feadaffd700 1 -- 172.21.15.12:0/2120508636 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7feaf4014420 con 0x557aacf4b720 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/2120508636 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaf400f040 con 0x557aacf4b720 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feadaffd700 1 -- 172.21.15.12:0/2120508636 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaf4014b70 con 0x557aacf4b720 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feb0dfd4b80 10 client.30691 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.303+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/2120508636 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7feafc10d550 con 0x557aacf4b720 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.304+0000 7feadaffd700 1 -- 172.21.15.12:0/2120508636 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaf4035080 con 0x557aacf4b720 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.304+0000 7feadaffd700 1 -- 172.21.15.12:0/2120508636 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feaf40050d0 con 0x557aacf4b720 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.304+0000 7feb0dfd4b80 10 client.30691 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.304+0000 7feb0dfd4b80 10 client.30691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:21.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.304+0000 7feb0dfd4b80 20 client.30691 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:21.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.304+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557aacfcf8e0 0x557aacfd1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:21.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.304+0000 7feb0dfd4b80 4 client.30691 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:06:21.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.304+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/2120508636 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557aace8dec0 con 0x557aacfcf8e0 2024-05-09T00:06:21.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.304+0000 7feafb7fe700 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557aacfcf8e0 0x557aacfd1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:21.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.304+0000 7feafb7fe700 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557aacfcf8e0 0x557aacfd1d00 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f030bd18dede3cbd server_cookie=a8976c3fb82c82fa in_seq=0 out_seq=0 2024-05-09T00:06:21.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.305+0000 7feadaffd700 10 client.30691 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:21.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feadaffd700 1 -- 172.21.15.12:0/2120508636 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x557aace8dec0 con 0x557aacfcf8e0 2024-05-09T00:06:21.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feadaffd700 10 client.30691 handle_command_reply: tid=0 2024-05-09T00:06:21.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 client.30691 shutdown 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/2120508636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf4b720 msgr2=0x557aacf505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf4b720 0x557aacf505b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000f80 tx=0x7feaf4004f50 comp rx=0 tx=0).stop 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/2120508636 shutdown_connections 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557aacf4b720 0x557aacf505b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557aacf45890 0x557aacf64570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557aacfcf8e0 0x557aacfd1d00 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 --2- 172.21.15.12:0/2120508636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557aacf4ae00 0x557aacf4fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/2120508636 >> 172.21.15.12:0/2120508636 conn(0x557aacea3210 msgr2=0x557aacf515f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feadaffd700 0 client.30691 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/2120508636 shutdown_connections 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 1 -- 172.21.15.12:0/2120508636 wait complete. 2024-05-09T00:06:21.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.307+0000 7feb0dfd4b80 20 client.30691 trim_cache size 0 max 16384 2024-05-09T00:06:21.306 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:06:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (161 inodes in the stack)", 2024-05-09T00:06:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:06:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "c5e784fc-8813-4819-a2de-3d1c448de260": { 2024-05-09T00:06:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:06:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "c5e784fc-8813-4819-a2de-3d1c448de260", 2024-05-09T00:06:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:06:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:06:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:06:21.306 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:06:21.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 -- 172.21.15.12:0/1508943172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feae40a1310 msgr2=0x7feae40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feae40a1310 0x7feae40a37d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7feaec00b2e0 tx=0x7feaec005960 comp rx=0 tx=0).stop 2024-05-09T00:06:21.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 -- 172.21.15.12:0/1508943172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feafc105160 msgr2=0x7feafc196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:21.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feafc105160 0x7feafc196810 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7feaf000e970 tx=0x7feaf000ec80 comp rx=0 tx=0).stop 2024-05-09T00:06:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 -- 172.21.15.12:0/1508943172 shutdown_connections 2024-05-09T00:06:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feae40a1310 0x7feae40a37d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feafc19abb0 0x7feafc19b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc100060 0x7feafc1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 --2- 172.21.15.12:0/1508943172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feafc105160 0x7feafc196810 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 -- 172.21.15.12:0/1508943172 >> 172.21.15.12:0/1508943172 conn(0x7feafc076f40 msgr2=0x7feafc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 -- 172.21.15.12:0/1508943172 shutdown_connections 2024-05-09T00:06:21.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:21.308+0000 7feb01e1a700 1 -- 172.21.15.12:0/1508943172 wait complete. 2024-05-09T00:06:21.320 INFO:tasks.cephfs.filesystem:scrub status for tag:c5e784fc-8813-4819-a2de-3d1c448de260 - {'path': '/', 'tag': 'c5e784fc-8813-4819-a2de-3d1c448de260', 'options': 'recursive,force'} 2024-05-09T00:06:21.512 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:21 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:06:21.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:21 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:06:21.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:21 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:06:22.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 244691 33.06 MB/sec execute 257 sec latency 285.559 ms 2024-05-09T00:06:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:22 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:06:22.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:22 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:06:22.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:22 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:06:22.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:22 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:06:22.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:22 smithi204 ceph-mon[47373]: pgmap v642: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 710 op/s 2024-05-09T00:06:22.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:22 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:06:22.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:22 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:06:22.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:22 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:06:22.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:22 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:06:22.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:22 smithi160 ceph-mon[46152]: pgmap v642: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 710 op/s 2024-05-09T00:06:22.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:22 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:06:22.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:22 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:06:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:22 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:06:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:22 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:06:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:22.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:22 smithi012 ceph-mon[42503]: pgmap v642: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 710 op/s 2024-05-09T00:06:23.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 245297 33.10 MB/sec execute 258 sec latency 117.927 ms 2024-05-09T00:06:24.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:23 smithi204 ceph-mon[47373]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 747 op/s 2024-05-09T00:06:24.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:23 smithi160 ceph-mon[46152]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 747 op/s 2024-05-09T00:06:24.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:23 smithi012 ceph-mon[42503]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 747 op/s 2024-05-09T00:06:24.267 INFO:tasks.workunit.client.0.smithi012.stdout: 10 245880 33.11 MB/sec execute 259 sec latency 100.121 ms 2024-05-09T00:06:25.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 246376 33.06 MB/sec execute 260 sec latency 126.472 ms 2024-05-09T00:06:26.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:25 smithi204 ceph-mon[47373]: pgmap v644: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 743 op/s 2024-05-09T00:06:26.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:25 smithi160 ceph-mon[46152]: pgmap v644: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 743 op/s 2024-05-09T00:06:26.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:25 smithi012 ceph-mon[42503]: pgmap v644: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 743 op/s 2024-05-09T00:06:26.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 246791 33.00 MB/sec execute 261 sec latency 268.551 ms 2024-05-09T00:06:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.188+0000 7fbf83081700 1 -- 172.21.15.12:0/102773306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf7c101110 msgr2=0x7fbf7c1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.188+0000 7fbf83081700 1 --2- 172.21.15.12:0/102773306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf7c101110 0x7fbf7c1014f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009990 tx=0x7fbf70009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.188+0000 7fbf83081700 1 -- 172.21.15.12:0/102773306 shutdown_connections 2024-05-09T00:06:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.188+0000 7fbf83081700 1 --2- 172.21.15.12:0/102773306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf7c106230 0x7fbf7c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.188+0000 7fbf83081700 1 --2- 172.21.15.12:0/102773306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf7c101110 0x7fbf7c1014f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.188+0000 7fbf83081700 1 --2- 172.21.15.12:0/102773306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf7c101ac0 0x7fbf7c105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.188+0000 7fbf83081700 1 -- 172.21.15.12:0/102773306 >> 172.21.15.12:0/102773306 conn(0x7fbf7c0fc9b0 msgr2=0x7fbf7c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.188+0000 7fbf83081700 1 -- 172.21.15.12:0/102773306 shutdown_connections 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.188+0000 7fbf83081700 1 -- 172.21.15.12:0/102773306 wait complete. 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.189+0000 7fbf83081700 1 Processor -- start 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.189+0000 7fbf83081700 1 -- start start 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.189+0000 7fbf83081700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf7c101110 0x7fbf7c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.189+0000 7fbf83081700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf7c106230 0x7fbf7c197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.189+0000 7fbf83081700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf7c19bca0 0x7fbf7c19c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.189+0000 7fbf83081700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c19c720 con 0x7fbf7c106230 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.189+0000 7fbf83081700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c19c890 con 0x7fbf7c19bca0 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.189+0000 7fbf83081700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c19ca50 con 0x7fbf7c101110 2024-05-09T00:06:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.189+0000 7fbf80e1d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf7c101110 0x7fbf7c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.189+0000 7fbf7bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf7c106230 0x7fbf7c197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf80e1d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf7c101110 0x7fbf7c1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46244/0 (socket says 172.21.15.12:46244) 2024-05-09T00:06:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf80e1d700 1 -- 172.21.15.12:0/365736982 learned_addr learned my addr 172.21.15.12:0/365736982 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf8161e700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf7c19bca0 0x7fbf7c19c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf80e1d700 1 -- 172.21.15.12:0/365736982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf7c19bca0 msgr2=0x7fbf7c19c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf7c19bca0 0x7fbf7c19c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf80e1d700 1 -- 172.21.15.12:0/365736982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf7c106230 msgr2=0x7fbf7c197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf7c106230 0x7fbf7c197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf80e1d700 1 -- 172.21.15.12:0/365736982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf700095f0 con 0x7fbf7c101110 2024-05-09T00:06:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf7c106230 0x7fbf7c197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf7c101110 0x7fbf7c1973c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbf70005910 tx=0x7fbf70004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf79ffb700 1 -- 172.21.15.12:0/365736982 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7001d070 con 0x7fbf7c101110 2024-05-09T00:06:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf79ffb700 1 -- 172.21.15.12:0/365736982 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf70022470 con 0x7fbf7c101110 2024-05-09T00:06:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf83081700 1 -- 172.21.15.12:0/365736982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c1a3f50 con 0x7fbf7c101110 2024-05-09T00:06:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.190+0000 7fbf79ffb700 1 -- 172.21.15.12:0/365736982 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf70017650 con 0x7fbf7c101110 2024-05-09T00:06:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.191+0000 7fbf83081700 1 -- 172.21.15.12:0/365736982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c1a4370 con 0x7fbf7c101110 2024-05-09T00:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.193+0000 7fbf79ffb700 1 -- 172.21.15.12:0/365736982 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbf700225e0 con 0x7fbf7c101110 2024-05-09T00:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.194+0000 7fbf79ffb700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf6c0a1310 0x7fbf6c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.194+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf6c0a1310 0x7fbf6c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.194+0000 7fbf79ffb700 1 -- 172.21.15.12:0/365736982 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbf700cdeb0 con 0x7fbf7c101110 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.194+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf6c0a1310 0x7fbf6c0a37d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c198910 tx=0x7fbf68009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf8f23bb80 1 Processor -- start 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf8f23bb80 1 -- start start 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7ee950 0x561f9e7eed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 0x561f9e7e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e7400f0 con 0x561f9e7e9480 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e73dab0 con 0x561f9e7ef270 2024-05-09T00:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e7393a0 con 0x561f9e7ee950 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf80e1d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7ee950 0x561f9e7eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf80e1d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7ee950 0x561f9e7eed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46254/0 (socket says 172.21.15.12:46254) 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf80e1d700 1 -- 172.21.15.12:0/708422129 learned_addr learned my addr 172.21.15.12:0/708422129 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/708422129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 0x561f9e7e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf8161e700 1 --2- 172.21.15.12:0/708422129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf80e1d700 1 -- 172.21.15.12:0/708422129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 msgr2=0x561f9e7e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/708422129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 0x561f9e7e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf80e1d700 1 -- 172.21.15.12:0/708422129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 msgr2=0x561f9e7e98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.195+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/708422129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7e98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.196+0000 7fbf80e1d700 1 -- 172.21.15.12:0/708422129 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f9e5ba350 con 0x561f9e7ee950 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.196+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/708422129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7ee950 0x561f9e7eed30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbf70097560 tx=0x7fbf70009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8da69cf734b3368 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.196+0000 7fbf66ffd700 1 -- 172.21.15.12:0/708422129 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf700aeea0 con 0x561f9e7ee950 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.196+0000 7fbf66ffd700 1 -- 172.21.15.12:0/708422129 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf7009b760 con 0x561f9e7ee950 2024-05-09T00:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.196+0000 7fbf66ffd700 1 -- 172.21.15.12:0/708422129 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf70099490 con 0x561f9e7ee950 2024-05-09T00:06:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/708422129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7ee950 msgr2=0x561f9e7eed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/708422129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7ee950 0x561f9e7eed30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbf70097560 tx=0x7fbf70009310 comp rx=0 tx=0).stop 2024-05-09T00:06:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/708422129 shutdown_connections 2024-05-09T00:06:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/708422129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/708422129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7ee950 0x561f9e7eed30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/708422129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 0x561f9e7e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/708422129 >> 172.21.15.12:0/708422129 conn(0x561f9e7471b0 msgr2=0x561f9e7e3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/708422129 shutdown_connections 2024-05-09T00:06:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/708422129 wait complete. 2024-05-09T00:06:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 -- start start 2024-05-09T00:06:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.197+0000 7fbf8f23bb80 1 Processor -- start 2024-05-09T00:06:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7e9480 0x561f9e7f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 0x561f9e7f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7f3130 0x561f9e7fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e6d5500 con 0x561f9e7f3130 2024-05-09T00:06:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e6d51a0 con 0x561f9e7ef270 2024-05-09T00:06:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e5bc640 con 0x561f9e7e9480 2024-05-09T00:06:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf8f23bb80 10 client.0 _ll_register_callbacks cb 0x561f9e6dead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf8161e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7f3130 0x561f9e7fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf7bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 0x561f9e7f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf8161e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7f3130 0x561f9e7fe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33354/0 (socket says 172.21.15.12:33354) 2024-05-09T00:06:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf8161e700 1 -- 172.21.15.12:0/451545383 learned_addr learned my addr 172.21.15.12:0/451545383 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.198+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7e9480 0x561f9e7f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf7bfff700 1 -- 172.21.15.12:0/451545383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7e9480 msgr2=0x561f9e7f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7e9480 0x561f9e7f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf7bfff700 1 -- 172.21.15.12:0/451545383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7f3130 msgr2=0x561f9e7fe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7f3130 0x561f9e7fe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf7bfff700 1 -- 172.21.15.12:0/451545383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c1a4370 con 0x561f9e7ef270 2024-05-09T00:06:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 0x561f9e7f2bf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x561f9e7e4790 tx=0x7fbf680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ced1c519233899 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:06:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf5bfff700 1 -- 172.21.15.12:0/451545383 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6801e800 con 0x561f9e7ef270 2024-05-09T00:06:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/451545383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f9e692210 con 0x561f9e7ef270 2024-05-09T00:06:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf5bfff700 1 -- 172.21.15.12:0/451545383 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbf6801e960 con 0x561f9e7ef270 2024-05-09T00:06:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf5bfff700 1 -- 172.21.15.12:0/451545383 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf6801eb20 con 0x561f9e7ef270 2024-05-09T00:06:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf8f23bb80 10 client.30524 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.199+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/451545383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x561f9e566b00 con 0x561f9e7ef270 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.200+0000 7fbf5bfff700 1 -- 172.21.15.12:0/451545383 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f9e692210 con 0x561f9e7ef270 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.200+0000 7fbf5bfff700 1 -- 172.21.15.12:0/451545383 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbf6802f070 con 0x561f9e7ef270 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.200+0000 7fbf8f23bb80 10 client.30524 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.200+0000 7fbf8f23bb80 10 client.30524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.200+0000 7fbf8f23bb80 20 client.30524 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.200+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561f9e802ee0 0x561f9e805300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.200+0000 7fbf8f23bb80 4 client.30524 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.200+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/451545383 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f9e7b87b0 con 0x561f9e802ee0 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.200+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561f9e802ee0 0x561f9e805300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.201+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561f9e802ee0 0x561f9e805300 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7c5681c835a6d9d server_cookie=58bc1c8e68fe8e4f in_seq=0 out_seq=0 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.201+0000 7fbf5bfff700 10 client.30524 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf5bfff700 1 -- 172.21.15.12:0/451545383 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f9e7b87b0 con 0x561f9e802ee0 2024-05-09T00:06:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf5bfff700 10 client.30524 handle_command_reply: tid=0 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf8f23bb80 1 client.30524 shutdown 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/451545383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 msgr2=0x561f9e7f2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 0x561f9e7f2bf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x561f9e7e4790 tx=0x7fbf680047c0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/451545383 shutdown_connections 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7f3130 0x561f9e7fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7e9480 0x561f9e7f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561f9e802ee0 0x561f9e805300 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/451545383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7ef270 0x561f9e7f2bf0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf5bfff700 0 client.30524 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/451545383 >> 172.21.15.12:0/451545383 conn(0x561f9e7471b0 msgr2=0x561f9e7e7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.202+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/451545383 shutdown_connections 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.203+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/451545383 wait complete. 2024-05-09T00:06:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.203+0000 7fbf8f23bb80 20 client.30524 trim_cache size 0 max 16384 2024-05-09T00:06:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf8f23bb80 1 Processor -- start 2024-05-09T00:06:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf8f23bb80 1 -- start start 2024-05-09T00:06:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7e3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 0x561f9e7f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e7ff950 con 0x561f9e7e9480 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e7e4530 con 0x561f9e7f2c00 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e800560 con 0x561f9e7f3130 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf80e1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf8161e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 0x561f9e7f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf80e1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7e3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33364/0 (socket says 172.21.15.12:33364) 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf80e1d700 1 -- 172.21.15.12:0/2354037305 learned_addr learned my addr 172.21.15.12:0/2354037305 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/2354037305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.224+0000 7fbf80e1d700 1 -- 172.21.15.12:0/2354037305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 msgr2=0x561f9e7f9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.225+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/2354037305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.225+0000 7fbf80e1d700 1 -- 172.21.15.12:0/2354037305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 msgr2=0x561f9e7f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.225+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/2354037305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 0x561f9e7f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.225+0000 7fbf80e1d700 1 -- 172.21.15.12:0/2354037305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f9e6ff500 con 0x561f9e7e9480 2024-05-09T00:06:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.225+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/2354037305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.225+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/2354037305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7e3910 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000f740 tx=0x7fbf70022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d647def137df7299 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.225+0000 7fbf677fe700 1 -- 172.21.15.12:0/2354037305 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf700aeea0 con 0x561f9e7e9480 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.225+0000 7fbf677fe700 1 -- 172.21.15.12:0/2354037305 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf7009ee90 con 0x561f9e7e9480 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.225+0000 7fbf677fe700 1 -- 172.21.15.12:0/2354037305 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7009b2c0 con 0x561f9e7e9480 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.226+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/2354037305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 msgr2=0x561f9e7e3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.226+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/2354037305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7e3910 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000f740 tx=0x7fbf70022f90 comp rx=0 tx=0).stop 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.226+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/2354037305 shutdown_connections 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.226+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/2354037305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 0x561f9e7f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.226+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/2354037305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7e3910 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.226+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/2354037305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.226+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/2354037305 >> 172.21.15.12:0/2354037305 conn(0x561f9e7471b0 msgr2=0x561f9e7e88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.226+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/2354037305 shutdown_connections 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.226+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/2354037305 wait complete. 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.226+0000 7fbf8f23bb80 1 -- start start 2024-05-09T00:06:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8f23bb80 1 Processor -- start 2024-05-09T00:06:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7f7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 0x561f9e7f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8f23bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e85c910 con 0x561f9e7e9480 2024-05-09T00:06:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e85ca80 con 0x561f9e7f2c00 2024-05-09T00:06:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8f23bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9e85cc40 con 0x561f9e7f3130 2024-05-09T00:06:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8f23bb80 10 client.0 _ll_register_callbacks cb 0x7fbf680045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8161e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf7bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 0x561f9e7f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf80e1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8161e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:46264/0 (socket says 172.21.15.12:46264) 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf7bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 0x561f9e7f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58524/0 (socket says 172.21.15.12:58524) 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.227+0000 7fbf8161e700 1 -- 172.21.15.12:0/459491823 learned_addr learned my addr 172.21.15.12:0/459491823 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf80e1d700 1 -- 172.21.15.12:0/459491823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 msgr2=0x561f9e7f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf80e1d700 1 -- 172.21.15.12:0/459491823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 msgr2=0x561f9e7f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 0x561f9e7f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf80e1d700 1 -- 172.21.15.12:0/459491823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f9e566b00 con 0x561f9e7e9480 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf8161e700 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf80e1d700 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7f7930 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fbf700cbc10 tx=0x7fbf70009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab64bf84192dedb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf5bfff700 1 -- 172.21.15.12:0/459491823 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf700aeea0 con 0x561f9e7e9480 2024-05-09T00:06:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/459491823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf7009b760 con 0x561f9e7e9480 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf5bfff700 1 -- 172.21.15.12:0/459491823 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbf700a3450 con 0x561f9e7e9480 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf5bfff700 1 -- 172.21.15.12:0/459491823 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf7009a750 con 0x561f9e7e9480 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf8f23bb80 10 client.21006 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.228+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/459491823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x561f9e5ba350 con 0x561f9e7e9480 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf5bfff700 1 -- 172.21.15.12:0/459491823 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf7009b760 con 0x561f9e7e9480 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf5bfff700 1 -- 172.21.15.12:0/459491823 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbf7007fd20 con 0x561f9e7e9480 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf8f23bb80 10 client.21006 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf8f23bb80 10 client.21006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf8f23bb80 20 client.21006 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561f9e85dee0 0x561f9e8602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf8f23bb80 4 client.21006 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/459491823 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f9e731e60 con 0x561f9e85dee0 2024-05-09T00:06:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561f9e85dee0 0x561f9e8602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf7bfff700 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561f9e85dee0 0x561f9e8602d0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4246f07f774b56 server_cookie=b8fd36a80c2c8f81 in_seq=0 out_seq=0 2024-05-09T00:06:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.229+0000 7fbf5bfff700 10 client.21006 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:27.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 247192 32.94 MB/sec execute 262 sec latency 300.873 ms 2024-05-09T00:06:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.667+0000 7fbf5bfff700 1 -- 172.21.15.12:0/459491823 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f9e731e60 con 0x561f9e85dee0 2024-05-09T00:06:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.667+0000 7fbf5bfff700 10 client.21006 handle_command_reply: tid=0 2024-05-09T00:06:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.667+0000 7fbf8f23bb80 1 client.21006 shutdown 2024-05-09T00:06:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.668+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/459491823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 msgr2=0x561f9e7f7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.668+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7f7930 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fbf700cbc10 tx=0x7fbf70009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.668+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/459491823 shutdown_connections 2024-05-09T00:06:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.668+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f9e7f3130 0x561f9e7f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.668+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f9e7e9480 0x561f9e7f7930 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.668+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561f9e85dee0 0x561f9e8602d0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.668+0000 7fbf8f23bb80 1 --2- 172.21.15.12:0/459491823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561f9e7f2c00 0x561f9e7f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.668+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/459491823 >> 172.21.15.12:0/459491823 conn(0x561f9e7471b0 msgr2=0x561f9e7f0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.668+0000 7fbf5bfff700 0 client.21006 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.671+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/459491823 shutdown_connections 2024-05-09T00:06:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.671+0000 7fbf8f23bb80 1 -- 172.21.15.12:0/459491823 wait complete. 2024-05-09T00:06:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.675+0000 7fbf8f23bb80 20 client.21006 trim_cache size 0 max 16384 2024-05-09T00:06:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:06:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:06:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:06:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:06:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 -- 172.21.15.12:0/365736982 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf6c0a1310 msgr2=0x7fbf6c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf6c0a1310 0x7fbf6c0a37d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c198910 tx=0x7fbf68009260 comp rx=0 tx=0).stop 2024-05-09T00:06:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 -- 172.21.15.12:0/365736982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf7c101110 msgr2=0x7fbf7c1973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf7c101110 0x7fbf7c1973c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbf70005910 tx=0x7fbf70004730 comp rx=0 tx=0).stop 2024-05-09T00:06:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 -- 172.21.15.12:0/365736982 shutdown_connections 2024-05-09T00:06:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fbf6c0a1310 0x7fbf6c0a37d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf7c19bca0 0x7fbf7c19c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf7c101110 0x7fbf7c1973c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 --2- 172.21.15.12:0/365736982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf7c106230 0x7fbf7c197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 -- 172.21.15.12:0/365736982 >> 172.21.15.12:0/365736982 conn(0x7fbf7c0fc9b0 msgr2=0x7fbf7c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.676+0000 7fbf83081700 1 -- 172.21.15.12:0/365736982 shutdown_connections 2024-05-09T00:06:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:27.677+0000 7fbf83081700 1 -- 172.21.15.12:0/365736982 wait complete. 2024-05-09T00:06:28.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:27 smithi204 ceph-mon[47373]: pgmap v645: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 833 op/s 2024-05-09T00:06:28.014 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:27 smithi160 ceph-mon[46152]: pgmap v645: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 833 op/s 2024-05-09T00:06:28.133 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:27 smithi012 ceph-mon[42503]: pgmap v645: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 833 op/s 2024-05-09T00:06:28.134 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:06:28 smithi012 sudo[134381]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:28.134 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:06:28 smithi012 sudo[134381]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:28.135 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:06:28 smithi012 sudo[134381]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:28.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 247575 32.89 MB/sec execute 263 sec latency 378.809 ms 2024-05-09T00:06:28.414 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:06:28 smithi012 sudo[134381]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:29.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:28 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:06:29.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:28 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:06:29.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:28 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:06:29.164 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:06:28 smithi012 sudo[134384]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:29.165 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:06:28 smithi012 sudo[134384]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:29.165 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:06:28 smithi012 sudo[134384]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:29.165 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:06:28 smithi012 sudo[134384]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:29.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 248389 32.96 MB/sec execute 264 sec latency 114.349 ms 2024-05-09T00:06:29.515 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:06:29 smithi160 sudo[89778]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:29.515 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:06:29 smithi160 sudo[89778]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:29.516 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:06:29 smithi160 sudo[89778]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:29.516 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:06:29 smithi160 sudo[89778]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:30.015 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:06:29 smithi160 sudo[89783]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:30.015 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:06:29 smithi160 sudo[89783]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:30.015 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:06:29 smithi160 sudo[89783]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:30.015 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:06:29 smithi160 sudo[89783]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:30.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:06:30.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:06:30.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:30.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:29 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:30.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:29 smithi160 ceph-mon[46152]: pgmap v646: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 617 op/s 2024-05-09T00:06:30.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:06:30.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:06:30.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:30.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:29 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:30.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:29 smithi012 ceph-mon[42503]: pgmap v646: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 617 op/s 2024-05-09T00:06:30.262 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:06:29 smithi204 sudo[92954]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:30.262 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:06:29 smithi204 sudo[92954]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:30.263 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:06:29 smithi204 sudo[92954]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:30.263 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:06:29 smithi204 sudo[92954]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:30.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:06:30.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:06:30.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:30.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:29 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:30.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:29 smithi204 ceph-mon[47373]: pgmap v646: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 617 op/s 2024-05-09T00:06:30.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 249053 32.98 MB/sec execute 265 sec latency 90.063 ms 2024-05-09T00:06:30.762 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:06:30 smithi204 sudo[92957]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:30.762 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:06:30 smithi204 sudo[92957]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:30.762 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:06:30 smithi204 sudo[92957]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:30.762 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:06:30 smithi204 sudo[92957]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:31.169 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:06:30 smithi012 sudo[134387]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:31.169 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:06:30 smithi012 sudo[134387]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:31.169 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:06:30 smithi012 sudo[134387]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:31.170 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:06:30 smithi012 sudo[134387]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:31.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 249567 32.96 MB/sec execute 266 sec latency 116.229 ms 2024-05-09T00:06:31.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:31 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:31.664 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:06:31 smithi012 sudo[134390]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:31.664 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:06:31 smithi012 sudo[134390]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:31.665 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:06:31 smithi012 sudo[134390]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:31.665 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:06:31 smithi012 sudo[134390]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:31.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:31 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:31.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:31 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:32.265 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:06:31 smithi160 sudo[89790]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:32.265 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:06:31 smithi160 sudo[89790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:32.265 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:06:31 smithi160 sudo[89790]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:32.265 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:06:31 smithi160 sudo[89790]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:32.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 250023 32.93 MB/sec execute 267 sec latency 219.769 ms 2024-05-09T00:06:32.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:32 smithi012 ceph-mon[42503]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 615 op/s 2024-05-09T00:06:32.762 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:32 smithi204 ceph-mon[47373]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 615 op/s 2024-05-09T00:06:32.765 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:06:32 smithi160 sudo[89793]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:32.765 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:06:32 smithi160 sudo[89793]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:32.765 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:06:32 smithi160 sudo[89793]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:32.765 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:06:32 smithi160 sudo[89793]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:32.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:32 smithi160 ceph-mon[46152]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 615 op/s 2024-05-09T00:06:33.270 INFO:tasks.workunit.client.0.smithi012.stdout: 10 250407 32.87 MB/sec execute 268 sec latency 140.873 ms 2024-05-09T00:06:33.444 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:06:33 smithi204 sudo[92963]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:33.445 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:06:33 smithi204 sudo[92963]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:33.445 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:06:33 smithi204 sudo[92963]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:33.445 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:06:33 smithi204 sudo[92963]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:33.445 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:06:33 smithi204 sudo[92966]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:33.445 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:06:33 smithi204 sudo[92966]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:33.445 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:06:33 smithi204 sudo[92966]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:33.720 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:06:33 smithi204 sudo[92966]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:33.721 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:33 smithi204 ceph-mon[47373]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 865 op/s 2024-05-09T00:06:33.977 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:33 smithi012 ceph-mon[42503]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 865 op/s 2024-05-09T00:06:33.977 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:06:33 smithi012 sudo[134393]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:34.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:33 smithi160 ceph-mon[46152]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 865 op/s 2024-05-09T00:06:34.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 251082 32.87 MB/sec execute 269 sec latency 106.910 ms 2024-05-09T00:06:34.414 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:06:33 smithi012 sudo[134393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:34.414 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:06:33 smithi012 sudo[134393]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:34.414 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:06:33 smithi012 sudo[134393]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:34.914 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:06:34 smithi012 sudo[134396]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:34.914 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:06:34 smithi012 sudo[134396]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:34.914 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:06:34 smithi012 sudo[134396]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:34.914 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:06:34 smithi012 sudo[134396]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:35.266 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:06:34 smithi160 sudo[89802]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:35.266 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:06:34 smithi160 sudo[89802]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:35.266 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:06:34 smithi160 sudo[89802]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:35.266 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:06:34 smithi160 sudo[89802]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:35.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 251686 32.88 MB/sec execute 270 sec latency 93.564 ms 2024-05-09T00:06:35.722 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:35 smithi160 ceph-mon[46152]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 620 op/s 2024-05-09T00:06:35.723 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:06:35 smithi160 sudo[89805]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:35.723 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:06:35 smithi160 sudo[89805]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:35.723 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:06:35 smithi160 sudo[89805]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:35.723 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:06:35 smithi160 sudo[89805]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:36.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:35 smithi204 ceph-mon[47373]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 620 op/s 2024-05-09T00:06:36.013 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:06:35 smithi204 sudo[92972]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:36.013 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:06:35 smithi204 sudo[92972]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:36.013 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:06:35 smithi204 sudo[92972]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:36.013 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:06:35 smithi204 sudo[92972]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:35 smithi012 ceph-mon[42503]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 620 op/s 2024-05-09T00:06:36.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 252163 32.84 MB/sec execute 271 sec latency 155.964 ms 2024-05-09T00:06:36.512 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:06:36 smithi204 sudo[92975]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:36.512 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:06:36 smithi204 sudo[92975]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:36.512 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:06:36 smithi204 sudo[92975]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:36.512 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:06:36 smithi204 sudo[92975]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:37.164 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:06:36 smithi012 sudo[134399]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:37.165 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:06:36 smithi012 sudo[134399]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:37.165 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:06:36 smithi012 sudo[134399]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:37.165 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:06:36 smithi012 sudo[134399]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:37.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 252783 32.87 MB/sec execute 272 sec latency 105.984 ms 2024-05-09T00:06:38.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:37 smithi204 ceph-mon[47373]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 746 op/s 2024-05-09T00:06:38.014 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:06:37 smithi160 sudo[89812]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:38.015 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:06:37 smithi160 sudo[89812]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:38.015 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:06:37 smithi160 sudo[89812]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:38.015 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:06:37 smithi160 sudo[89812]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:38.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:37 smithi160 ceph-mon[46152]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 746 op/s 2024-05-09T00:06:38.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:37 smithi012 ceph-mon[42503]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 746 op/s 2024-05-09T00:06:38.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 253315 32.86 MB/sec execute 273 sec latency 186.657 ms 2024-05-09T00:06:38.515 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:06:38 smithi160 sudo[89817]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:38.515 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:06:38 smithi160 sudo[89817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:38.515 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:06:38 smithi160 sudo[89817]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:38.515 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:06:38 smithi160 sudo[89817]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:39.012 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:06:38 smithi204 sudo[92980]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:39.013 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:06:38 smithi204 sudo[92980]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:39.013 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:06:38 smithi204 sudo[92980]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:39.013 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:06:38 smithi204 sudo[92980]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:39.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 253832 32.82 MB/sec execute 274 sec latency 155.719 ms 2024-05-09T00:06:39.512 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:06:39 smithi204 sudo[92984]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:39.512 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:06:39 smithi204 sudo[92984]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:39.512 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:06:39 smithi204 sudo[92984]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:39.513 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:06:39 smithi204 sudo[92984]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:39 smithi012 sudo[134405]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:39 smithi012 sudo[134405]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:39.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:39 smithi012 sudo[134405]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:39.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:39 smithi012 sudo[134405]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:40.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 254161 32.74 MB/sec execute 275 sec latency 962.471 ms 2024-05-09T00:06:40.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 sudo[134408]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:40.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 sudo[134408]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:40.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 sudo[134408]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:40.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 sudo[134408]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:40.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 sudo[92988]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:40.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 sudo[92988]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:40.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 sudo[92988]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:40.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 sudo[92988]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.825+0000 7fae62931700 1 -- 172.21.15.12:0/283880231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae5c101b50 msgr2=0x7fae5c105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.825+0000 7fae62931700 1 --2- 172.21.15.12:0/283880231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae5c101b50 0x7fae5c105b90 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fae50009990 tx=0x7fae50009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.825+0000 7fae62931700 1 -- 172.21.15.12:0/283880231 shutdown_connections 2024-05-09T00:06:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.825+0000 7fae62931700 1 --2- 172.21.15.12:0/283880231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae5c1062c0 0x7fae5c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.825+0000 7fae62931700 1 --2- 172.21.15.12:0/283880231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae5c1011a0 0x7fae5c101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.825+0000 7fae62931700 1 --2- 172.21.15.12:0/283880231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae5c101b50 0x7fae5c105b90 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.825+0000 7fae62931700 1 -- 172.21.15.12:0/283880231 >> 172.21.15.12:0/283880231 conn(0x7fae5c0fca40 msgr2=0x7fae5c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.825+0000 7fae62931700 1 -- 172.21.15.12:0/283880231 shutdown_connections 2024-05-09T00:06:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.825+0000 7fae62931700 1 -- 172.21.15.12:0/283880231 wait complete. 2024-05-09T00:06:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.826+0000 7fae62931700 1 Processor -- start 2024-05-09T00:06:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.826+0000 7fae62931700 1 -- start start 2024-05-09T00:06:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae62931700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae5c1011a0 0x7fae5c197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae62931700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae5c1062c0 0x7fae5c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae62931700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae5c19bda0 0x7fae5c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae62931700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c19c820 con 0x7fae5c19bda0 2024-05-09T00:06:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae62931700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c19c990 con 0x7fae5c1062c0 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae62931700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c19cb50 con 0x7fae5c1011a0 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae60ece700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae5c19bda0 0x7fae5c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae60ece700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae5c19bda0 0x7fae5c19c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57964/0 (socket says 172.21.15.12:57964) 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae60ece700 1 -- 172.21.15.12:0/1929180815 learned_addr learned my addr 172.21.15.12:0/1929180815 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae5bfff700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae5c1011a0 0x7fae5c197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae60ece700 1 -- 172.21.15.12:0/1929180815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae5c1011a0 msgr2=0x7fae5c197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae60ece700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae5c1011a0 0x7fae5c197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae60ece700 1 -- 172.21.15.12:0/1929180815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae5c1062c0 msgr2=0x7fae5c197a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae60ece700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae5c1062c0 0x7fae5c197a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.827+0000 7fae60ece700 1 -- 172.21.15.12:0/1929180815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500095f0 con 0x7fae5c19bda0 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.828+0000 7fae60ece700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae5c19bda0 0x7fae5c19c220 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fae54007ee0 tx=0x7fae5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.828+0000 7fae597fa700 1 -- 172.21.15.12:0/1929180815 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54017070 con 0x7fae5c19bda0 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.828+0000 7fae597fa700 1 -- 172.21.15.12:0/1929180815 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5400f040 con 0x7fae5c19bda0 2024-05-09T00:06:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.828+0000 7fae62931700 1 -- 172.21.15.12:0/1929180815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c1a40b0 con 0x7fae5c19bda0 2024-05-09T00:06:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.828+0000 7fae597fa700 1 -- 172.21.15.12:0/1929180815 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54005740 con 0x7fae5c19bda0 2024-05-09T00:06:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.828+0000 7fae62931700 1 -- 172.21.15.12:0/1929180815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c1a4590 con 0x7fae5c19bda0 2024-05-09T00:06:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.831+0000 7fae597fa700 1 -- 172.21.15.12:0/1929180815 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fae540058a0 con 0x7fae5c19bda0 2024-05-09T00:06:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.832+0000 7fae597fa700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae440a1390 0x7fae440a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.832+0000 7fae5bfff700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae440a1390 0x7fae440a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.832+0000 7fae597fa700 1 -- 172.21.15.12:0/1929180815 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fae540cd300 con 0x7fae5c19bda0 2024-05-09T00:06:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.833+0000 7fae5bfff700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae440a1390 0x7fae440a3850 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009c60 tx=0x7fae4c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.833+0000 7fae6eaebb80 1 Processor -- start 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.833+0000 7fae6eaebb80 1 -- start start 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.834+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5dcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.834+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.834+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5edbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.834+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c5300f0 con 0x55ad8c5dc970 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.834+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c52dab0 con 0x55ad8c5d9900 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.834+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c5293a0 con 0x55ad8c5d8f40 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.834+0000 7fae5b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.834+0000 7fae5b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54726/0 (socket says 172.21.15.12:54726) 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.834+0000 7fae5b7fe700 1 -- 172.21.15.12:0/2082378720 learned_addr learned my addr 172.21.15.12:0/2082378720 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.834+0000 7fae5bfff700 1 --2- 172.21.15.12:0/2082378720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5dcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae60ece700 1 --2- 172.21.15.12:0/2082378720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5edbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae5b7fe700 1 -- 172.21.15.12:0/2082378720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 msgr2=0x55ad8c5edbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae5b7fe700 1 --2- 172.21.15.12:0/2082378720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5edbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae5b7fe700 1 -- 172.21.15.12:0/2082378720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 msgr2=0x55ad8c5dcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae5b7fe700 1 --2- 172.21.15.12:0/2082378720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5dcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae5b7fe700 1 -- 172.21.15.12:0/2082378720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad8c3aa350 con 0x55ad8c5d8f40 2024-05-09T00:06:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae60ece700 1 --2- 172.21.15.12:0/2082378720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5edbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:06:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae5b7fe700 1 --2- 172.21.15.12:0/2082378720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d93c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fae500059d0 tx=0x7fae50005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e081909b3286fe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae427fc700 1 -- 172.21.15.12:0/2082378720 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5001d070 con 0x55ad8c5d8f40 2024-05-09T00:06:40.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae427fc700 1 -- 172.21.15.12:0/2082378720 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5000bbb0 con 0x55ad8c5d8f40 2024-05-09T00:06:40.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.835+0000 7fae427fc700 1 -- 172.21.15.12:0/2082378720 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5000fbd0 con 0x55ad8c5d8f40 2024-05-09T00:06:40.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.836+0000 7fae6eaebb80 1 -- 172.21.15.12:0/2082378720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 msgr2=0x55ad8c5d93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:40.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.836+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/2082378720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d93c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fae500059d0 tx=0x7fae50005a90 comp rx=0 tx=0).stop 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.836+0000 7fae6eaebb80 1 -- 172.21.15.12:0/2082378720 shutdown_connections 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.836+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/2082378720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5edbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.836+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/2082378720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5dcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.836+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/2082378720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d93c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.836+0000 7fae6eaebb80 1 -- 172.21.15.12:0/2082378720 >> 172.21.15.12:0/2082378720 conn(0x55ad8c5371b0 msgr2=0x55ad8c5d3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.836+0000 7fae6eaebb80 1 -- 172.21.15.12:0/2082378720 shutdown_connections 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.836+0000 7fae6eaebb80 1 -- 172.21.15.12:0/2082378720 wait complete. 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.837+0000 7fae6eaebb80 1 -- start start 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.837+0000 7fae6eaebb80 1 Processor -- start 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.837+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.837+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5e3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.837+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5ed530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.837+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c4c5500 con 0x55ad8c5dc970 2024-05-09T00:06:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.837+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c4c51a0 con 0x55ad8c5d9900 2024-05-09T00:06:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.837+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c3ac640 con 0x55ad8c5d8f40 2024-05-09T00:06:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.837+0000 7fae6eaebb80 10 client.0 _ll_register_callbacks cb 0x55ad8c4cead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae5bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae5bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5e2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54740/0 (socket says 172.21.15.12:54740) 2024-05-09T00:06:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae5bfff700 1 -- 172.21.15.12:0/1889043172 learned_addr learned my addr 172.21.15.12:0/1889043172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae5b7fe700 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae60ece700 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5ed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae5bfff700 1 -- 172.21.15.12:0/1889043172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 msgr2=0x55ad8c5e3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae5bfff700 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5e3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae5bfff700 1 -- 172.21.15.12:0/1889043172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 msgr2=0x55ad8c5ed530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae5bfff700 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5ed530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae5bfff700 1 -- 172.21.15.12:0/1889043172 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c10e6b0 con 0x55ad8c5d8f40 2024-05-09T00:06:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae60ece700 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5ed530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:06:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae5bfff700 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5e2c30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fae4c005910 tx=0x7fae4c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a98c79bde50afa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae3b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.838+0000 7fae3b7fe700 1 -- 172.21.15.12:0/1889043172 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c022070 con 0x55ad8c5d8f40 2024-05-09T00:06:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.839+0000 7fae3b7fe700 1 -- 172.21.15.12:0/1889043172 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fae4c01d720 con 0x55ad8c5d8f40 2024-05-09T00:06:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:40.839+0000 7fae6eaebb80 1 -- 172.21.15.12:0/1889043172 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad8c482210 con 0x55ad8c5d8f40 2024-05-09T00:06:41.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 sudo[92991]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:41.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 sudo[92991]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:41.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 sudo[92991]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 sudo[92991]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:06:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 ceph-mon[47373]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.9 MiB/s wr, 567 op/s 2024-05-09T00:06:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:06:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:06:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:06:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:06:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:06:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:06:41.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:40 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:06:41.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:06:41.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 ceph-mon[42503]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.9 MiB/s wr, 567 op/s 2024-05-09T00:06:41.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:06:41.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:06:41.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:06:41.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:06:41.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:06:41.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:06:41.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:40 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:06:41.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:40 smithi160 sudo[89826]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:06:41.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:40 smithi160 sudo[89826]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:41.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:40 smithi160 sudo[89826]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:41.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:40 smithi160 sudo[89826]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:41.268 INFO:tasks.workunit.client.0.smithi012.stdout: 10 255003 32.87 MB/sec execute 276 sec latency 128.827 ms 2024-05-09T00:06:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.286+0000 7fae3b7fe700 1 -- 172.21.15.12:0/1889043172 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae4c01d8e0 con 0x55ad8c5d8f40 2024-05-09T00:06:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.286+0000 7fae6eaebb80 10 client.30724 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.286+0000 7fae6eaebb80 1 -- 172.21.15.12:0/1889043172 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55ad8c356b00 con 0x55ad8c5d8f40 2024-05-09T00:06:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.287+0000 7fae3b7fe700 1 -- 172.21.15.12:0/1889043172 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae4c033080 con 0x55ad8c5d8f40 2024-05-09T00:06:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.287+0000 7fae3b7fe700 1 -- 172.21.15.12:0/1889043172 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fae4c027b90 con 0x55ad8c5d8f40 2024-05-09T00:06:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.287+0000 7fae6eaebb80 10 client.30724 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.287+0000 7fae6eaebb80 10 client.30724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.287+0000 7fae6eaebb80 20 client.30724 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.287+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad8c603ce0 0x55ad8c606100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.287+0000 7fae6eaebb80 4 client.30724 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.287+0000 7fae6eaebb80 1 -- 172.21.15.12:0/1889043172 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad8c5a87b0 con 0x55ad8c603ce0 2024-05-09T00:06:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.288+0000 7fae5b7fe700 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad8c603ce0 0x55ad8c606100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.288+0000 7fae3b7fe700 10 client.30724 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.288+0000 7fae5b7fe700 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad8c603ce0 0x55ad8c606100 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae2cd24bc8946dd2 server_cookie=501a449d98b5c6ca in_seq=0 out_seq=0 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae3b7fe700 1 -- 172.21.15.12:0/1889043172 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad8c5a87b0 con 0x55ad8c603ce0 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae3b7fe700 10 client.30724 handle_command_reply: tid=0 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 client.30724 shutdown 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 -- 172.21.15.12:0/1889043172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 msgr2=0x55ad8c5e2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5e2c30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fae4c005910 tx=0x7fae4c0047c0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 -- 172.21.15.12:0/1889043172 shutdown_connections 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5ed530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5e2c30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad8c603ce0 0x55ad8c606100 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/1889043172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5e3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 -- 172.21.15.12:0/1889043172 >> 172.21.15.12:0/1889043172 conn(0x55ad8c5371b0 msgr2=0x55ad8c5d5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae3b7fe700 0 client.30724 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 -- 172.21.15.12:0/1889043172 shutdown_connections 2024-05-09T00:06:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.291+0000 7fae6eaebb80 1 -- 172.21.15.12:0/1889043172 wait complete. 2024-05-09T00:06:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.292+0000 7fae6eaebb80 20 client.30724 trim_cache size 0 max 16384 2024-05-09T00:06:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae6eaebb80 1 Processor -- start 2024-05-09T00:06:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae6eaebb80 1 -- start start 2024-05-09T00:06:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5d9900 0x55ad8c5e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5dc970 0x55ad8c5ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c5d4a30 con 0x55ad8c5d9900 2024-05-09T00:06:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c5e34b0 con 0x55ad8c5d8f40 2024-05-09T00:06:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c5de860 con 0x55ad8c5dc970 2024-05-09T00:06:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae5bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae60ece700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5dc970 0x55ad8c5ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae5bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:36796/0 (socket says 172.21.15.12:36796) 2024-05-09T00:06:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.313+0000 7fae5bfff700 1 -- 172.21.15.12:0/2283493134 learned_addr learned my addr 172.21.15.12:0/2283493134 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.314+0000 7fae60ece700 1 -- 172.21.15.12:0/2283493134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d8f40 msgr2=0x55ad8c5d9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.314+0000 7fae60ece700 1 --2- 172.21.15.12:0/2283493134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.314+0000 7fae60ece700 1 -- 172.21.15.12:0/2283493134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5d9900 msgr2=0x55ad8c5e2f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:06:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.314+0000 7fae60ece700 1 --2- 172.21.15.12:0/2283493134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5d9900 0x55ad8c5e2f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.314+0000 7fae60ece700 1 -- 172.21.15.12:0/2283493134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad8c4ef500 con 0x55ad8c5dc970 2024-05-09T00:06:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.314+0000 7fae60ece700 1 --2- 172.21.15.12:0/2283493134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5dc970 0x55ad8c5ddcf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fae540caa00 tx=0x7fae5409a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5dfe999674c64b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.314+0000 7fae42ffd700 1 -- 172.21.15.12:0/2283493134 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54085070 con 0x55ad8c5dc970 2024-05-09T00:06:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.314+0000 7fae42ffd700 1 -- 172.21.15.12:0/2283493134 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5409aac0 con 0x55ad8c5dc970 2024-05-09T00:06:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.314+0000 7fae42ffd700 1 -- 172.21.15.12:0/2283493134 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400dbd0 con 0x55ad8c5dc970 2024-05-09T00:06:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.315+0000 7fae6eaebb80 1 -- 172.21.15.12:0/2283493134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5dc970 msgr2=0x55ad8c5ddcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.315+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/2283493134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5dc970 0x55ad8c5ddcf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fae540caa00 tx=0x7fae5409a290 comp rx=0 tx=0).stop 2024-05-09T00:06:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.315+0000 7fae6eaebb80 1 -- 172.21.15.12:0/2283493134 shutdown_connections 2024-05-09T00:06:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.315+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/2283493134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5dc970 0x55ad8c5ddcf0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.315+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/2283493134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5d9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.315+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/2283493134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5d9900 0x55ad8c5e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.316+0000 7fae6eaebb80 1 -- 172.21.15.12:0/2283493134 >> 172.21.15.12:0/2283493134 conn(0x55ad8c5371b0 msgr2=0x55ad8c5cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.316+0000 7fae6eaebb80 1 -- 172.21.15.12:0/2283493134 shutdown_connections 2024-05-09T00:06:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.316+0000 7fae6eaebb80 1 -- 172.21.15.12:0/2283493134 wait complete. 2024-05-09T00:06:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.316+0000 7fae6eaebb80 1 -- start start 2024-05-09T00:06:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.316+0000 7fae6eaebb80 1 Processor -- start 2024-05-09T00:06:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5ea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae6eaebb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c5ed9c0 con 0x55ad8c5dc970 2024-05-09T00:06:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c66a1f0 con 0x55ad8c5d9900 2024-05-09T00:06:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae6eaebb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8c66a330 con 0x55ad8c5d8f40 2024-05-09T00:06:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae6eaebb80 10 client.0 _ll_register_callbacks cb 0x55ad8c4cead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae60ece700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae5bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae5bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5ea960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54750/0 (socket says 172.21.15.12:54750) 2024-05-09T00:06:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae5bfff700 1 -- 172.21.15.12:0/693989341 learned_addr learned my addr 172.21.15.12:0/693989341 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae5bfff700 1 -- 172.21.15.12:0/693989341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 msgr2=0x55ad8c5eaea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:06:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae5bfff700 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5eaea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae5bfff700 1 -- 172.21.15.12:0/693989341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 msgr2=0x55ad8c5eb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae5bfff700 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5eb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.317+0000 7fae5bfff700 1 -- 172.21.15.12:0/693989341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c1a4590 con 0x55ad8c5d8f40 2024-05-09T00:06:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.318+0000 7fae5bfff700 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5ea960 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fae4c011ab0 tx=0x7fae4c02fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25193ccca368e1f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.318+0000 7fae3b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.318+0000 7fae3b7fe700 1 -- 172.21.15.12:0/693989341 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c037070 con 0x55ad8c5d8f40 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.318+0000 7fae3b7fe700 1 -- 172.21.15.12:0/693989341 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fae4c01d930 con 0x55ad8c5d8f40 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.318+0000 7fae6eaebb80 1 -- 172.21.15.12:0/693989341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae5000bbb0 con 0x55ad8c5d8f40 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.318+0000 7fae3b7fe700 1 -- 172.21.15.12:0/693989341 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae4c022040 con 0x55ad8c5d8f40 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.318+0000 7fae6eaebb80 10 client.30730 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.318+0000 7fae6eaebb80 1 -- 172.21.15.12:0/693989341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55ad8c3aec70 con 0x55ad8c5d8f40 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.319+0000 7fae3b7fe700 1 -- 172.21.15.12:0/693989341 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5000bbb0 con 0x55ad8c5d8f40 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.319+0000 7fae3b7fe700 1 -- 172.21.15.12:0/693989341 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fae4c045470 con 0x55ad8c5d8f40 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.319+0000 7fae6eaebb80 10 client.30730 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.319+0000 7fae6eaebb80 10 client.30730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.319+0000 7fae6eaebb80 20 client.30730 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.319+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad8c5ee540 0x55ad8c5f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.319+0000 7fae6eaebb80 4 client.30730 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:06:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.319+0000 7fae6eaebb80 1 -- 172.21.15.12:0/693989341 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ad8c521e60 con 0x55ad8c5ee540 2024-05-09T00:06:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.320+0000 7fae5b7fe700 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad8c5ee540 0x55ad8c5f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.321+0000 7fae5b7fe700 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad8c5ee540 0x55ad8c5f0a00 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c1792d42ef38d65 server_cookie=df996ed226133632 in_seq=0 out_seq=0 2024-05-09T00:06:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.321+0000 7fae3b7fe700 10 client.30730 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:41.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 sudo[89829]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:06:41.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 sudo[89829]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:06:41.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 sudo[89829]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:06:41.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 sudo[89829]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:06:41.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:06:41.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.9 MiB/s wr, 567 op/s 2024-05-09T00:06:41.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:06:41.697 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:06:41.698 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:41 smithi160 ceph-mon[46152]: pgmap v652: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.0 MiB/s wr, 540 op/s 2024-05-09T00:06:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.782+0000 7fae3b7fe700 1 -- 172.21.15.12:0/693989341 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ad8c521e60 con 0x55ad8c5ee540 2024-05-09T00:06:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.782+0000 7fae3b7fe700 10 client.30730 handle_command_reply: tid=0 2024-05-09T00:06:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.782+0000 7fae6eaebb80 1 client.30730 shutdown 2024-05-09T00:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.785+0000 7fae6eaebb80 1 -- 172.21.15.12:0/693989341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 msgr2=0x55ad8c5ea960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.785+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5ea960 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fae4c011ab0 tx=0x7fae4c02fb20 comp rx=0 tx=0).stop 2024-05-09T00:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.785+0000 7fae6eaebb80 1 -- 172.21.15.12:0/693989341 shutdown_connections 2024-05-09T00:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.785+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ad8c5dc970 0x55ad8c5eb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.785+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad8c5d8f40 0x55ad8c5ea960 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.785+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ad8c5ee540 0x55ad8c5f0a00 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.785+0000 7fae6eaebb80 1 --2- 172.21.15.12:0/693989341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad8c5d9900 0x55ad8c5eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.785+0000 7fae6eaebb80 1 -- 172.21.15.12:0/693989341 >> 172.21.15.12:0/693989341 conn(0x55ad8c5371b0 msgr2=0x55ad8c5ddcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.785+0000 7fae3b7fe700 0 client.30730 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.785+0000 7fae6eaebb80 1 -- 172.21.15.12:0/693989341 shutdown_connections 2024-05-09T00:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.786+0000 7fae6eaebb80 1 -- 172.21.15.12:0/693989341 wait complete. 2024-05-09T00:06:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae6eaebb80 20 client.30730 trim_cache size 0 max 16384 2024-05-09T00:06:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:06:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:06:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:06:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:06:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae62931700 1 -- 172.21.15.12:0/1929180815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae440a1390 msgr2=0x7fae440a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae62931700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae440a1390 0x7fae440a3850 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009c60 tx=0x7fae4c009260 comp rx=0 tx=0).stop 2024-05-09T00:06:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae62931700 1 -- 172.21.15.12:0/1929180815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae5c19bda0 msgr2=0x7fae5c19c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae62931700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae5c19bda0 0x7fae5c19c220 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fae54007ee0 tx=0x7fae5400d3b0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae62931700 1 -- 172.21.15.12:0/1929180815 shutdown_connections 2024-05-09T00:06:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae62931700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae440a1390 0x7fae440a3850 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae62931700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae5c19bda0 0x7fae5c19c220 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae62931700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae5c1011a0 0x7fae5c197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae62931700 1 --2- 172.21.15.12:0/1929180815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae5c1062c0 0x7fae5c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.788+0000 7fae62931700 1 -- 172.21.15.12:0/1929180815 >> 172.21.15.12:0/1929180815 conn(0x7fae5c0fca40 msgr2=0x7fae5c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.789+0000 7fae62931700 1 -- 172.21.15.12:0/1929180815 shutdown_connections 2024-05-09T00:06:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:41.789+0000 7fae62931700 1 -- 172.21.15.12:0/1929180815 wait complete. 2024-05-09T00:06:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:41 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:06:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:41 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:06:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:41 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:06:42.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:06:42.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:06:42.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:41 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:06:42.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:41 smithi204 ceph-mon[47373]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:06:42.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:41 smithi204 ceph-mon[47373]: pgmap v652: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.0 MiB/s wr, 540 op/s 2024-05-09T00:06:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:41 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:06:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:41 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:06:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:41 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:06:42.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:06:42.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:06:42.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:41 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:06:42.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:41 smithi012 ceph-mon[42503]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:06:42.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:41 smithi012 ceph-mon[42503]: pgmap v652: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.0 MiB/s wr, 540 op/s 2024-05-09T00:06:42.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 255213 32.77 MB/sec execute 277 sec latency 372.466 ms 2024-05-09T00:06:43.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 255561 32.67 MB/sec execute 278 sec latency 280.941 ms 2024-05-09T00:06:44.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:43 smithi204 ceph-mon[47373]: pgmap v653: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-05-09T00:06:44.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:43 smithi160 ceph-mon[46152]: pgmap v653: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-05-09T00:06:44.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:43 smithi012 ceph-mon[42503]: pgmap v653: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 12 MiB/s wr, 778 op/s 2024-05-09T00:06:44.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 256282 32.71 MB/sec execute 279 sec latency 101.558 ms 2024-05-09T00:06:45.270 INFO:tasks.workunit.client.0.smithi012.stdout: 10 256899 32.72 MB/sec execute 280 sec latency 83.919 ms 2024-05-09T00:06:46.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:45 smithi204 ceph-mon[47373]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.1 MiB/s wr, 522 op/s 2024-05-09T00:06:46.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:45 smithi160 ceph-mon[46152]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.1 MiB/s wr, 522 op/s 2024-05-09T00:06:46.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:45 smithi012 ceph-mon[42503]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.1 MiB/s wr, 522 op/s 2024-05-09T00:06:46.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 257640 32.73 MB/sec execute 281 sec latency 59.989 ms 2024-05-09T00:06:47.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 258374 32.80 MB/sec execute 282 sec latency 110.273 ms 2024-05-09T00:06:48.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:47 smithi204 ceph-mon[47373]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 707 op/s 2024-05-09T00:06:48.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:47 smithi160 ceph-mon[46152]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 707 op/s 2024-05-09T00:06:48.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:47 smithi012 ceph-mon[42503]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 707 op/s 2024-05-09T00:06:48.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 258693 32.73 MB/sec execute 283 sec latency 355.493 ms 2024-05-09T00:06:49.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 259129 32.67 MB/sec execute 284 sec latency 195.591 ms 2024-05-09T00:06:50.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:49 smithi204 ceph-mon[47373]: pgmap v656: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.6 MiB/s wr, 606 op/s 2024-05-09T00:06:50.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:49 smithi160 ceph-mon[46152]: pgmap v656: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.6 MiB/s wr, 606 op/s 2024-05-09T00:06:50.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:49 smithi012 ceph-mon[42503]: pgmap v656: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.6 MiB/s wr, 606 op/s 2024-05-09T00:06:50.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 259746 32.66 MB/sec execute 285 sec latency 112.933 ms 2024-05-09T00:06:51.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 260524 32.69 MB/sec execute 286 sec latency 95.186 ms 2024-05-09T00:06:51.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:06:51.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.491+0000 7f8c0d27f700 1 -- 172.21.15.12:0/1623523156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c08069dd0 msgr2=0x7f8c08105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.491+0000 7f8c0d27f700 1 --2- 172.21.15.12:0/1623523156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c08069dd0 0x7f8c08105b70 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009990 tx=0x7f8bfc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.491+0000 7f8c0d27f700 1 -- 172.21.15.12:0/1623523156 shutdown_connections 2024-05-09T00:06:51.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.491+0000 7f8c0d27f700 1 --2- 172.21.15.12:0/1623523156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c081062a0 0x7f8c0810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.491+0000 7f8c0d27f700 1 --2- 172.21.15.12:0/1623523156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c08069420 0x7f8c08069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.491+0000 7f8c0d27f700 1 --2- 172.21.15.12:0/1623523156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c08069dd0 0x7f8c08105b70 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.491+0000 7f8c0d27f700 1 -- 172.21.15.12:0/1623523156 >> 172.21.15.12:0/1623523156 conn(0x7f8c08076ba0 msgr2=0x7f8c08076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:51.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.491+0000 7f8c0d27f700 1 -- 172.21.15.12:0/1623523156 shutdown_connections 2024-05-09T00:06:51.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.491+0000 7f8c0d27f700 1 -- 172.21.15.12:0/1623523156 wait complete. 2024-05-09T00:06:51.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.492+0000 7f8c0d27f700 1 Processor -- start 2024-05-09T00:06:51.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.492+0000 7f8c0d27f700 1 -- start start 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c0d27f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c08069420 0x7f8c081001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c0d27f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081062a0 0x7f8c08100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c0d27f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c08101e80 0x7f8c08100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c0d27f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081aa090 con 0x7f8c081062a0 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c0d27f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081aa200 con 0x7f8c08101e80 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c0d27f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081aa3c0 con 0x7f8c08069420 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c067fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081062a0 0x7f8c08100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c06ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c08069420 0x7f8c081001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c067fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081062a0 0x7f8c08100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59368/0 (socket says 172.21.15.12:59368) 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c067fc700 1 -- 172.21.15.12:0/3966070001 learned_addr learned my addr 172.21.15.12:0/3966070001 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c077fe700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c08101e80 0x7f8c08100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c067fc700 1 -- 172.21.15.12:0/3966070001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c08069420 msgr2=0x7f8c081001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c067fc700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c08069420 0x7f8c081001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c067fc700 1 -- 172.21.15.12:0/3966070001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c08101e80 msgr2=0x7f8c08100c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c067fc700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c08101e80 0x7f8c08100c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c067fc700 1 -- 172.21.15.12:0/3966070001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc0095f0 con 0x7f8c081062a0 2024-05-09T00:06:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.493+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c08069420 0x7f8c081001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.494+0000 7f8c067fc700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081062a0 0x7f8c08100700 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc000c00 tx=0x7f8bfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.494+0000 7f8beffff700 1 -- 172.21.15.12:0/3966070001 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc01d070 con 0x7f8c081062a0 2024-05-09T00:06:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.494+0000 7f8c0d27f700 1 -- 172.21.15.12:0/3966070001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c081aa640 con 0x7f8c081062a0 2024-05-09T00:06:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.494+0000 7f8c0d27f700 1 -- 172.21.15.12:0/3966070001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c081aaac0 con 0x7f8c081062a0 2024-05-09T00:06:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.494+0000 7f8beffff700 1 -- 172.21.15.12:0/3966070001 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bfc021410 con 0x7f8c081062a0 2024-05-09T00:06:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.494+0000 7f8beffff700 1 -- 172.21.15.12:0/3966070001 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc017650 con 0x7f8c081062a0 2024-05-09T00:06:51.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.497+0000 7f8beffff700 1 -- 172.21.15.12:0/3966070001 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8bfc00fb00 con 0x7f8c081062a0 2024-05-09T00:06:51.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.497+0000 7f8beffff700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf00a1310 0x7f8bf00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.497+0000 7f8beffff700 1 -- 172.21.15.12:0/3966070001 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8bfc0cde50 con 0x7f8c081062a0 2024-05-09T00:06:51.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.498+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf00a1310 0x7f8bf00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.498+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf00a1310 0x7f8bf00a37d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8011490 tx=0x7f8bf8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:51.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.498+0000 7f8c19439b80 1 Processor -- start 2024-05-09T00:06:51.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c19439b80 1 -- start start 2024-05-09T00:06:51.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b9648e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 0x561b96488350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96488cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c19439b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b963df150 con 0x561b96488890 2024-05-09T00:06:51.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c19439b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b963dcb10 con 0x561b9648e720 2024-05-09T00:06:51.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c19439b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b963d8400 con 0x561b9648de00 2024-05-09T00:06:51.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96488cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96488cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59370/0 (socket says 172.21.15.12:59370) 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c077fe700 1 -- 172.21.15.12:0/2686147466 learned_addr learned my addr 172.21.15.12:0/2686147466 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/2686147466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b9648e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c077fe700 1 -- 172.21.15.12:0/2686147466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 msgr2=0x561b9648e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c077fe700 1 --2- 172.21.15.12:0/2686147466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b9648e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c077fe700 1 -- 172.21.15.12:0/2686147466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 msgr2=0x561b96488350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c077fe700 1 --2- 172.21.15.12:0/2686147466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 0x561b96488350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c077fe700 1 -- 172.21.15.12:0/2686147466 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b962593b0 con 0x561b96488890 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.499+0000 7f8c077fe700 1 --2- 172.21.15.12:0/2686147466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96488cd0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8c00003f40 tx=0x7f8c00009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b3756a04f5d9a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.500+0000 7f8bed7fa700 1 -- 172.21.15.12:0/2686147466 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00019070 con 0x561b96488890 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.500+0000 7f8bed7fa700 1 -- 172.21.15.12:0/2686147466 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0000f040 con 0x561b96488890 2024-05-09T00:06:51.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.500+0000 7f8bed7fa700 1 -- 172.21.15.12:0/2686147466 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c000079e0 con 0x561b96488890 2024-05-09T00:06:51.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.500+0000 7f8c19439b80 1 -- 172.21.15.12:0/2686147466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 msgr2=0x561b96488cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.500+0000 7f8c19439b80 1 --2- 172.21.15.12:0/2686147466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96488cd0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8c00003f40 tx=0x7f8c00009ba0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 -- 172.21.15.12:0/2686147466 shutdown_connections 2024-05-09T00:06:51.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 --2- 172.21.15.12:0/2686147466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96488cd0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 --2- 172.21.15.12:0/2686147466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b9648e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 --2- 172.21.15.12:0/2686147466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 0x561b96488350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 -- 172.21.15.12:0/2686147466 >> 172.21.15.12:0/2686147466 conn(0x561b963e6210 msgr2=0x561b96482df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:51.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 -- 172.21.15.12:0/2686147466 shutdown_connections 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 -- 172.21.15.12:0/2686147466 wait complete. 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 -- start start 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 Processor -- start 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b964947c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b96491ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 0x561b964921f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b96374560 con 0x561b96488890 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b96374200 con 0x561b9648e720 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.501+0000 7f8c19439b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9625b6a0 con 0x561b9648de00 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c19439b80 10 client.0 _ll_register_callbacks cb 0x561b9637db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c06ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b964947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c06ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b964947c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59374/0 (socket says 172.21.15.12:59374) 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c06ffd700 1 -- 172.21.15.12:0/378434360 learned_addr learned my addr 172.21.15.12:0/378434360 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:51.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c067fc700 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b96491ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c06ffd700 1 -- 172.21.15.12:0/378434360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 msgr2=0x561b96491ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b96491ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c06ffd700 1 -- 172.21.15.12:0/378434360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 msgr2=0x561b964921f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 0x561b964921f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c06ffd700 1 -- 172.21.15.12:0/378434360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c00003b70 con 0x561b96488890 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b964947c0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8003ad0 tx=0x7f8bf80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bd9ed005150430 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8beaff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8beaff5700 1 -- 172.21.15.12:0/378434360 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf801fa10 con 0x561b96488890 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8beaff5700 1 -- 172.21.15.12:0/378434360 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8bf801fb70 con 0x561b96488890 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.502+0000 7f8c19439b80 1 -- 172.21.15.12:0/378434360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b96331270 con 0x561b96488890 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8beaff5700 1 -- 172.21.15.12:0/378434360 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf801fd30 con 0x561b96488890 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8c19439b80 10 client.21033 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:51.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8c19439b80 1 -- 172.21.15.12:0/378434360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x561b96205b60 con 0x561b96488890 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8beaff5700 1 -- 172.21.15.12:0/378434360 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b96331270 con 0x561b96488890 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8beaff5700 1 -- 172.21.15.12:0/378434360 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8bf802f070 con 0x561b96488890 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8c19439b80 10 client.21033 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8c19439b80 10 client.21033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8c19439b80 20 client.21033 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8c19439b80 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b964a21f0 0x561b964a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8c19439b80 4 client.21033 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8c19439b80 1 -- 172.21.15.12:0/378434360 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b96457810 con 0x561b964a21f0 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.503+0000 7f8c067fc700 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b964a21f0 0x561b964a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.504+0000 7f8c067fc700 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b964a21f0 0x561b964a4610 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1c14d6d2c82e236 server_cookie=14882d1b9e727329 in_seq=0 out_seq=0 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.504+0000 7f8beaff5700 10 client.21033 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8beaff5700 1 -- 172.21.15.12:0/378434360 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b96457810 con 0x561b964a21f0 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8beaff5700 10 client.21033 handle_command_reply: tid=0 2024-05-09T00:06:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 client.21033 shutdown 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 -- 172.21.15.12:0/378434360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 msgr2=0x561b964947c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b964947c0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8003ad0 tx=0x7f8bf80047c0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 -- 172.21.15.12:0/378434360 shutdown_connections 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b964a21f0 0x561b964a4610 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 0x561b964921f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b964947c0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8beaff5700 0 client.21033 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 --2- 172.21.15.12:0/378434360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b96491ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 -- 172.21.15.12:0/378434360 >> 172.21.15.12:0/378434360 conn(0x561b963e6210 msgr2=0x561b964825a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 -- 172.21.15.12:0/378434360 shutdown_connections 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 1 -- 172.21.15.12:0/378434360 wait complete. 2024-05-09T00:06:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.506+0000 7f8c19439b80 20 client.21033 trim_cache size 0 max 16384 2024-05-09T00:06:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.527+0000 7f8c19439b80 1 Processor -- start 2024-05-09T00:06:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.527+0000 7f8c19439b80 1 -- start start 2024-05-09T00:06:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.528+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b96488890 0x561b96488350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.528+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b9648de00 0x561b96482d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.528+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648e720 0x561b964991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.528+0000 7f8c19439b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b96483290 con 0x561b9648de00 2024-05-09T00:06:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.528+0000 7f8c19439b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b96488c70 con 0x561b96488890 2024-05-09T00:06:51.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.528+0000 7f8c19439b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b96483990 con 0x561b9648e720 2024-05-09T00:06:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.528+0000 7f8c077fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648e720 0x561b964991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.528+0000 7f8c077fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648e720 0x561b964991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38964/0 (socket says 172.21.15.12:38964) 2024-05-09T00:06:51.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.528+0000 7f8c077fe700 1 -- 172.21.15.12:0/3843290617 learned_addr learned my addr 172.21.15.12:0/3843290617 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.529+0000 7f8c077fe700 1 -- 172.21.15.12:0/3843290617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b96488890 msgr2=0x561b96488350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.529+0000 7f8c077fe700 1 --2- 172.21.15.12:0/3843290617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b96488890 0x561b96488350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.529+0000 7f8c077fe700 1 -- 172.21.15.12:0/3843290617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b9648de00 msgr2=0x561b96482d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:06:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.529+0000 7f8c077fe700 1 --2- 172.21.15.12:0/3843290617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b9648de00 0x561b96482d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.529+0000 7f8c077fe700 1 -- 172.21.15.12:0/3843290617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b96488f40 con 0x561b9648e720 2024-05-09T00:06:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.529+0000 7f8c077fe700 1 --2- 172.21.15.12:0/3843290617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648e720 0x561b964991d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8c00003e60 tx=0x7f8c00007500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c162300129b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.529+0000 7f8bedffb700 1 -- 172.21.15.12:0/3843290617 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00019040 con 0x561b9648e720 2024-05-09T00:06:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.529+0000 7f8bedffb700 1 -- 172.21.15.12:0/3843290617 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c00007e10 con 0x561b9648e720 2024-05-09T00:06:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.529+0000 7f8bedffb700 1 -- 172.21.15.12:0/3843290617 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00015e50 con 0x561b9648e720 2024-05-09T00:06:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.531+0000 7f8c19439b80 1 -- 172.21.15.12:0/3843290617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648e720 msgr2=0x561b964991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.531+0000 7f8c19439b80 1 --2- 172.21.15.12:0/3843290617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648e720 0x561b964991d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8c00003e60 tx=0x7f8c00007500 comp rx=0 tx=0).stop 2024-05-09T00:06:51.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.533+0000 7f8c19439b80 1 -- 172.21.15.12:0/3843290617 shutdown_connections 2024-05-09T00:06:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.533+0000 7f8c19439b80 1 --2- 172.21.15.12:0/3843290617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648e720 0x561b964991d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.533+0000 7f8c19439b80 1 --2- 172.21.15.12:0/3843290617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b96488890 0x561b96488350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.533+0000 7f8c19439b80 1 --2- 172.21.15.12:0/3843290617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b9648de00 0x561b96482d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.533+0000 7f8c19439b80 1 -- 172.21.15.12:0/3843290617 >> 172.21.15.12:0/3843290617 conn(0x561b963e6210 msgr2=0x561b9648bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.533+0000 7f8c19439b80 1 -- 172.21.15.12:0/3843290617 shutdown_connections 2024-05-09T00:06:51.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.533+0000 7f8c19439b80 1 -- 172.21.15.12:0/3843290617 wait complete. 2024-05-09T00:06:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.534+0000 7f8c19439b80 1 -- start start 2024-05-09T00:06:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.534+0000 7f8c19439b80 1 Processor -- start 2024-05-09T00:06:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.534+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96499630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.534+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b96497a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.534+0000 7f8c19439b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 0x561b964980a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.534+0000 7f8c19439b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b96498780 con 0x561b96488890 2024-05-09T00:06:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.534+0000 7f8c19439b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b964988c0 con 0x561b9648e720 2024-05-09T00:06:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.534+0000 7f8c19439b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b96509710 con 0x561b9648de00 2024-05-09T00:06:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.534+0000 7f8c19439b80 10 client.0 _ll_register_callbacks cb 0x561b9637db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.534+0000 7f8c067fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b96497a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.535+0000 7f8c067fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b96497a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38970/0 (socket says 172.21.15.12:38970) 2024-05-09T00:06:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.535+0000 7f8c067fc700 1 -- 172.21.15.12:0/4085781922 learned_addr learned my addr 172.21.15.12:0/4085781922 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.535+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96499630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.535+0000 7f8c06ffd700 1 -- 172.21.15.12:0/4085781922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 msgr2=0x561b96497a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.535+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b96497a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.535+0000 7f8c06ffd700 1 -- 172.21.15.12:0/4085781922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 msgr2=0x561b964980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.535+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 0x561b964980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.535+0000 7f8c06ffd700 1 -- 172.21.15.12:0/4085781922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b962593b0 con 0x561b96488890 2024-05-09T00:06:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.536+0000 7f8c06ffd700 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96499630 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8003aa0 tx=0x7f8bf8011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348d3ecb77f70287 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.536+0000 7f8beaff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.537+0000 7f8beaff5700 1 -- 172.21.15.12:0/4085781922 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf801faf0 con 0x561b96488890 2024-05-09T00:06:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.537+0000 7f8beaff5700 1 -- 172.21.15.12:0/4085781922 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8bf800e750 con 0x561b96488890 2024-05-09T00:06:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.537+0000 7f8c19439b80 1 -- 172.21.15.12:0/4085781922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c0000f040 con 0x561b96488890 2024-05-09T00:06:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.537+0000 7f8beaff5700 1 -- 172.21.15.12:0/4085781922 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf801fc50 con 0x561b96488890 2024-05-09T00:06:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.537+0000 7f8c19439b80 10 client.21036 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.537+0000 7f8c19439b80 1 -- 172.21.15.12:0/4085781922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x561b9639e560 con 0x561b96488890 2024-05-09T00:06:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.537+0000 7f8beaff5700 1 -- 172.21.15.12:0/4085781922 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0000f040 con 0x561b96488890 2024-05-09T00:06:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.538+0000 7f8beaff5700 1 -- 172.21.15.12:0/4085781922 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8bf802f050 con 0x561b96488890 2024-05-09T00:06:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.538+0000 7f8c19439b80 10 client.21036 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.538+0000 7f8c19439b80 10 client.21036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.538+0000 7f8c19439b80 20 client.21036 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.538+0000 7f8c19439b80 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b9650aeb0 0x561b9650d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.538+0000 7f8c19439b80 4 client.21036 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:06:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.538+0000 7f8c19439b80 1 -- 172.21.15.12:0/4085781922 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561b963d0ec0 con 0x561b9650aeb0 2024-05-09T00:06:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.539+0000 7f8c067fc700 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b9650aeb0 0x561b9650d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.539+0000 7f8c067fc700 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b9650aeb0 0x561b9650d2d0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745201599666efee server_cookie=f444a4f3bc44573a in_seq=0 out_seq=0 2024-05-09T00:06:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.539+0000 7f8beaff5700 10 client.21036 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8beaff5700 1 -- 172.21.15.12:0/4085781922 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561b963d0ec0 con 0x561b9650aeb0 2024-05-09T00:06:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8beaff5700 10 client.21036 handle_command_reply: tid=0 2024-05-09T00:06:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 client.21036 shutdown 2024-05-09T00:06:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 -- 172.21.15.12:0/4085781922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 msgr2=0x561b96499630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96499630 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8003aa0 tx=0x7f8bf8011c90 comp rx=0 tx=0).stop 2024-05-09T00:06:51.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 -- 172.21.15.12:0/4085781922 shutdown_connections 2024-05-09T00:06:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b9650aeb0 0x561b9650d2d0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b9648e720 0x561b964980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b96488890 0x561b96499630 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 --2- 172.21.15.12:0/4085781922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9648de00 0x561b96497a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 -- 172.21.15.12:0/4085781922 >> 172.21.15.12:0/4085781922 conn(0x561b963e6210 msgr2=0x561b96490360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8beaff5700 0 client.21036 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:51.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 -- 172.21.15.12:0/4085781922 shutdown_connections 2024-05-09T00:06:51.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.540+0000 7f8c19439b80 1 -- 172.21.15.12:0/4085781922 wait complete. 2024-05-09T00:06:51.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c19439b80 20 client.21036 trim_cache size 0 max 16384 2024-05-09T00:06:51.537 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:06:51.537 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:06:51.537 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:06:51.537 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:06:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c0d27f700 1 -- 172.21.15.12:0/3966070001 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf00a1310 msgr2=0x7f8bf00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c0d27f700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf00a1310 0x7f8bf00a37d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8011490 tx=0x7f8bf8009260 comp rx=0 tx=0).stop 2024-05-09T00:06:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c0d27f700 1 -- 172.21.15.12:0/3966070001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081062a0 msgr2=0x7f8c08100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c0d27f700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081062a0 0x7f8c08100700 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc000c00 tx=0x7f8bfc004730 comp rx=0 tx=0).stop 2024-05-09T00:06:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c0d27f700 1 -- 172.21.15.12:0/3966070001 shutdown_connections 2024-05-09T00:06:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c0d27f700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8bf00a1310 0x7f8bf00a37d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c0d27f700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c08101e80 0x7f8c08100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c0d27f700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c08069420 0x7f8c081001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c0d27f700 1 --2- 172.21.15.12:0/3966070001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c081062a0 0x7f8c08100700 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.541+0000 7f8c0d27f700 1 -- 172.21.15.12:0/3966070001 >> 172.21.15.12:0/3966070001 conn(0x7f8c08076ba0 msgr2=0x7f8c08107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.542+0000 7f8c0d27f700 1 -- 172.21.15.12:0/3966070001 shutdown_connections 2024-05-09T00:06:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.542+0000 7f8c0d27f700 1 -- 172.21.15.12:0/3966070001 wait complete. 2024-05-09T00:06:51.555 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:06:51.555 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:06:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.727+0000 7fc054574700 1 -- 172.21.15.12:0/1961233135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c102020 msgr2=0x7fc04c102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.727+0000 7fc054574700 1 --2- 172.21.15.12:0/1961233135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c102020 0x7fc04c102400 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009990 tx=0x7fc03c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.728+0000 7fc054574700 1 -- 172.21.15.12:0/1961233135 shutdown_connections 2024-05-09T00:06:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.728+0000 7fc054574700 1 --2- 172.21.15.12:0/1961233135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c10b3f0 0x7fc04c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.728+0000 7fc054574700 1 --2- 172.21.15.12:0/1961233135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c102020 0x7fc04c102400 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.728+0000 7fc054574700 1 --2- 172.21.15.12:0/1961233135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc04c102940 0x7fc04c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.728+0000 7fc054574700 1 -- 172.21.15.12:0/1961233135 >> 172.21.15.12:0/1961233135 conn(0x7fc04c0fb7e0 msgr2=0x7fc04c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.728+0000 7fc054574700 1 -- 172.21.15.12:0/1961233135 shutdown_connections 2024-05-09T00:06:51.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.728+0000 7fc054574700 1 -- 172.21.15.12:0/1961233135 wait complete. 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc054574700 1 Processor -- start 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc054574700 1 -- start start 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc054574700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c102020 0x7fc04c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc054574700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc04c102940 0x7fc04c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc054574700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c10b3f0 0x7fc04c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc054574700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c19c1c0 con 0x7fc04c102020 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc054574700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c19c330 con 0x7fc04c102940 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc054574700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c19c4f0 con 0x7fc04c10b3f0 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc052b11700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c10b3f0 0x7fc04c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc051b0f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc04c102940 0x7fc04c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc051b0f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc04c102940 0x7fc04c1977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48614/0 (socket says 172.21.15.12:48614) 2024-05-09T00:06:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc052b11700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c10b3f0 0x7fc04c19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38976/0 (socket says 172.21.15.12:38976) 2024-05-09T00:06:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.729+0000 7fc051b0f700 1 -- 172.21.15.12:0/628730798 learned_addr learned my addr 172.21.15.12:0/628730798 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.730+0000 7fc051b0f700 1 -- 172.21.15.12:0/628730798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c10b3f0 msgr2=0x7fc04c19bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.730+0000 7fc051b0f700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c10b3f0 0x7fc04c19bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.730+0000 7fc051b0f700 1 -- 172.21.15.12:0/628730798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c102020 msgr2=0x7fc04c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.730+0000 7fc051b0f700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c102020 0x7fc04c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.730+0000 7fc051b0f700 1 -- 172.21.15.12:0/628730798 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c0095f0 con 0x7fc04c102940 2024-05-09T00:06:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.730+0000 7fc051b0f700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc04c102940 0x7fc04c1977e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e9a0 tx=0x7fc04800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.732+0000 7fc0437fe700 1 -- 172.21.15.12:0/628730798 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048009790 con 0x7fc04c102940 2024-05-09T00:06:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.732+0000 7fc0437fe700 1 -- 172.21.15.12:0/628730798 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc04800cbd0 con 0x7fc04c102940 2024-05-09T00:06:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.732+0000 7fc054574700 1 -- 172.21.15.12:0/628730798 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c19c7d0 con 0x7fc04c102940 2024-05-09T00:06:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.732+0000 7fc054574700 1 -- 172.21.15.12:0/628730798 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04c1a4010 con 0x7fc04c102940 2024-05-09T00:06:51.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.732+0000 7fc0437fe700 1 -- 172.21.15.12:0/628730798 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048018680 con 0x7fc04c102940 2024-05-09T00:06:51.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.734+0000 7fc054574700 1 -- 172.21.15.12:0/628730798 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0300052f0 con 0x7fc04c102940 2024-05-09T00:06:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.735+0000 7fc0437fe700 1 -- 172.21.15.12:0/628730798 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc048010430 con 0x7fc04c102940 2024-05-09T00:06:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.735+0000 7fc0437fe700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0380a1290 0x7fc0380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.736+0000 7fc0437fe700 1 -- 172.21.15.12:0/628730798 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc048014070 con 0x7fc04c102940 2024-05-09T00:06:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.736+0000 7fc052310700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0380a1290 0x7fc0380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.736+0000 7fc052310700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0380a1290 0x7fc0380a3750 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0098e0 tx=0x7fc03c005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:51.745+0000 7fc0437fe700 1 -- 172.21.15.12:0/628730798 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc048052d60 con 0x7fc04c102940 2024-05-09T00:06:52.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:51 smithi204 ceph-mon[47373]: pgmap v657: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.5 MiB/s wr, 593 op/s 2024-05-09T00:06:52.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:51 smithi160 ceph-mon[46152]: pgmap v657: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.5 MiB/s wr, 593 op/s 2024-05-09T00:06:52.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.046+0000 7fc054574700 1 -- 172.21.15.12:0/628730798 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc030005160 con 0x7fc04c102940 2024-05-09T00:06:52.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.047+0000 7fc0437fe700 1 -- 172.21.15.12:0/628730798 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+10875 (secure 0 0 0) 0x7fc048099cd0 con 0x7fc04c102940 2024-05-09T00:06:52.042 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:06:52.043 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:06:02.725626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_30395":{"gid":30395,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6833/3969500142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3969500142},{"type":"v1","addr":"172.21.15.160:6833","nonce":3969500142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:06:52.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 -- 172.21.15.12:0/628730798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0380a1290 msgr2=0x7fc0380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0380a1290 0x7fc0380a3750 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0098e0 tx=0x7fc03c005fb0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 -- 172.21.15.12:0/628730798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc04c102940 msgr2=0x7fc04c1977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc04c102940 0x7fc04c1977e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e9a0 tx=0x7fc04800ecb0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 -- 172.21.15.12:0/628730798 shutdown_connections 2024-05-09T00:06:52.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0380a1290 0x7fc0380a3750 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c10b3f0 0x7fc04c19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c102020 0x7fc04c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 --2- 172.21.15.12:0/628730798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc04c102940 0x7fc04c1977e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 -- 172.21.15.12:0/628730798 >> 172.21.15.12:0/628730798 conn(0x7fc04c0fb7e0 msgr2=0x7fc04c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 -- 172.21.15.12:0/628730798 shutdown_connections 2024-05-09T00:06:52.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.050+0000 7fc054574700 1 -- 172.21.15.12:0/628730798 wait complete. 2024-05-09T00:06:52.046 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 76 2024-05-09T00:06:52.058 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:51 smithi012 ceph-mon[42503]: pgmap v657: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.5 MiB/s wr, 593 op/s 2024-05-09T00:06:52.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:06:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.239+0000 7f4548e1c700 1 -- 172.21.15.12:0/754347939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544106280 msgr2=0x7f454410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.239+0000 7f4548e1c700 1 --2- 172.21.15.12:0/754347939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544106280 0x7f454410de00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f453c00b0a0 tx=0x7f453c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.239+0000 7f4548e1c700 1 -- 172.21.15.12:0/754347939 shutdown_connections 2024-05-09T00:06:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.239+0000 7f4548e1c700 1 --2- 172.21.15.12:0/754347939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544106280 0x7f454410de00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.239+0000 7f4548e1c700 1 --2- 172.21.15.12:0/754347939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4544101160 0x7f4544101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.239+0000 7f4548e1c700 1 --2- 172.21.15.12:0/754347939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544101b10 0x7f4544105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.239+0000 7f4548e1c700 1 -- 172.21.15.12:0/754347939 >> 172.21.15.12:0/754347939 conn(0x7f45440fc9e0 msgr2=0x7f45440fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.239+0000 7f4548e1c700 1 -- 172.21.15.12:0/754347939 shutdown_connections 2024-05-09T00:06:52.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.239+0000 7f4548e1c700 1 -- 172.21.15.12:0/754347939 wait complete. 2024-05-09T00:06:52.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4548e1c700 1 Processor -- start 2024-05-09T00:06:52.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4548e1c700 1 -- start start 2024-05-09T00:06:52.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4548e1c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544101160 0x7f4544197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4548e1c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544106280 0x7f4544197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4548e1c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f454419be10 0x7f454419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4548e1c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419c890 con 0x7f454419be10 2024-05-09T00:06:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4548e1c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419ca00 con 0x7f4544101160 2024-05-09T00:06:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4548e1c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419cbc0 con 0x7f4544106280 2024-05-09T00:06:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4542d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f454419be10 0x7f454419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544106280 0x7f4544197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f454259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544101160 0x7f4544197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4542d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f454419be10 0x7f454419c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59420/0 (socket says 172.21.15.12:59420) 2024-05-09T00:06:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544106280 0x7f4544197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38984/0 (socket says 172.21.15.12:38984) 2024-05-09T00:06:52.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.240+0000 7f4542d9d700 1 -- 172.21.15.12:0/3591419631 learned_addr learned my addr 172.21.15.12:0/3591419631 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f4541d9b700 1 -- 172.21.15.12:0/3591419631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544101160 msgr2=0x7f4544197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f4541d9b700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544101160 0x7f4544197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f4541d9b700 1 -- 172.21.15.12:0/3591419631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f454419be10 msgr2=0x7f454419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f4541d9b700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f454419be10 0x7f454419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f4541d9b700 1 -- 172.21.15.12:0/3591419631 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c009d00 con 0x7f4544106280 2024-05-09T00:06:52.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f4541d9b700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544106280 0x7f4544197a70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f453800e9c0 tx=0x7f453800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f45337fe700 1 -- 172.21.15.12:0/3591419631 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453800c9c0 con 0x7f4544106280 2024-05-09T00:06:52.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f4548e1c700 1 -- 172.21.15.12:0/3591419631 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45441a4120 con 0x7f4544106280 2024-05-09T00:06:52.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f45337fe700 1 -- 172.21.15.12:0/3591419631 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4538010430 con 0x7f4544106280 2024-05-09T00:06:52.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f45337fe700 1 -- 172.21.15.12:0/3591419631 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538018680 con 0x7f4544106280 2024-05-09T00:06:52.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.241+0000 7f4548e1c700 1 -- 172.21.15.12:0/3591419631 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45441a4600 con 0x7f4544106280 2024-05-09T00:06:52.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.244+0000 7f45337fe700 1 -- 172.21.15.12:0/3591419631 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f453800cb20 con 0x7f4544106280 2024-05-09T00:06:52.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.244+0000 7f45337fe700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f452c0a1290 0x7f452c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.244+0000 7f45337fe700 1 -- 172.21.15.12:0/3591419631 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4538014070 con 0x7f4544106280 2024-05-09T00:06:52.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.244+0000 7f454259c700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f452c0a1290 0x7f452c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.245+0000 7f454259c700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f452c0a1290 0x7f452c0a3750 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4534009c60 tx=0x7f4534009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.245+0000 7f4554fd6b80 1 Processor -- start 2024-05-09T00:06:52.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4554fd6b80 1 -- start start 2024-05-09T00:06:52.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 0x5634c04515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c0451b10 0x5634c044b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c044bca0 0x5634c044c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c039e150 con 0x5634c044bca0 2024-05-09T00:06:52.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c039bb10 con 0x5634c0451b10 2024-05-09T00:06:52.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c0397400 con 0x5634c04511f0 2024-05-09T00:06:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f454259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 0x5634c04515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4542d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c044bca0 0x5634c044c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c0451b10 0x5634c044b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c0451b10 0x5634c044b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48642/0 (socket says 172.21.15.12:48642) 2024-05-09T00:06:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f454259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 0x5634c04515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:38988/0 (socket says 172.21.15.12:38988) 2024-05-09T00:06:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4541d9b700 1 -- 172.21.15.12:0/3160924044 learned_addr learned my addr 172.21.15.12:0/3160924044 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4542d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c044bca0 0x5634c044c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59424/0 (socket says 172.21.15.12:59424) 2024-05-09T00:06:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4541d9b700 1 -- 172.21.15.12:0/3160924044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 msgr2=0x5634c04515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4541d9b700 1 --2- 172.21.15.12:0/3160924044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 0x5634c04515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4541d9b700 1 -- 172.21.15.12:0/3160924044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c044bca0 msgr2=0x5634c044c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4541d9b700 1 --2- 172.21.15.12:0/3160924044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c044bca0 0x5634c044c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.246+0000 7f4541d9b700 1 -- 172.21.15.12:0/3160924044 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634c02183b0 con 0x5634c0451b10 2024-05-09T00:06:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.247+0000 7f4541d9b700 1 --2- 172.21.15.12:0/3160924044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c0451b10 0x5634c044b760 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4538009230 tx=0x7f4538099a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce78c3ab7b0cb65a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.247+0000 7f4530ff9700 1 -- 172.21.15.12:0/3160924044 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538099ea0 con 0x5634c0451b10 2024-05-09T00:06:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.247+0000 7f4530ff9700 1 -- 172.21.15.12:0/3160924044 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f453809dcd0 con 0x5634c0451b10 2024-05-09T00:06:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.247+0000 7f4530ff9700 1 -- 172.21.15.12:0/3160924044 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453809a100 con 0x5634c0451b10 2024-05-09T00:06:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 -- 172.21.15.12:0/3160924044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c0451b10 msgr2=0x5634c044b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/3160924044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c0451b10 0x5634c044b760 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4538009230 tx=0x7f4538099a60 comp rx=0 tx=0).stop 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 -- 172.21.15.12:0/3160924044 shutdown_connections 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/3160924044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c044bca0 0x5634c044c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/3160924044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 0x5634c04515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/3160924044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c0451b10 0x5634c044b760 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 -- 172.21.15.12:0/3160924044 >> 172.21.15.12:0/3160924044 conn(0x5634c03a5210 msgr2=0x5634c0441df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 -- 172.21.15.12:0/3160924044 shutdown_connections 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 -- 172.21.15.12:0/3160924044 wait complete. 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 -- start start 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.248+0000 7f4554fd6b80 1 Processor -- start 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c044bca0 0x5634c0454e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c04511f0 0x5634c04553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0455900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c0333560 con 0x5634c0451b10 2024-05-09T00:06:52.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c0333200 con 0x5634c04511f0 2024-05-09T00:06:52.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c021a6a0 con 0x5634c044bca0 2024-05-09T00:06:52.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4554fd6b80 10 client.0 _ll_register_callbacks cb 0x5634c033cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:52.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4542d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0455900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c04511f0 0x5634c04553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4542d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0455900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59436/0 (socket says 172.21.15.12:59436) 2024-05-09T00:06:52.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4542d9d700 1 -- 172.21.15.12:0/2982830833 learned_addr learned my addr 172.21.15.12:0/2982830833 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f454259c700 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c044bca0 0x5634c0454e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4541d9b700 1 -- 172.21.15.12:0/2982830833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c044bca0 msgr2=0x5634c0454e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4541d9b700 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c044bca0 0x5634c0454e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4541d9b700 1 -- 172.21.15.12:0/2982830833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 msgr2=0x5634c0455900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4541d9b700 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0455900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4541d9b700 1 -- 172.21.15.12:0/2982830833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45441a4600 con 0x5634c04511f0 2024-05-09T00:06:52.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4542d9d700 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0455900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:52.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4541d9b700 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c04511f0 0x5634c04553c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f45380a24e0 tx=0x7f4538099740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec0753f1fa53a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4529ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:06:52.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.249+0000 7f4529ffb700 1 -- 172.21.15.12:0/2982830833 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538099ea0 con 0x5634c04511f0 2024-05-09T00:06:52.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4554fd6b80 1 -- 172.21.15.12:0/2982830833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634c02f0270 con 0x5634c04511f0 2024-05-09T00:06:52.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4529ffb700 1 -- 172.21.15.12:0/2982830833 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f45380105a0 con 0x5634c04511f0 2024-05-09T00:06:52.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4529ffb700 1 -- 172.21.15.12:0/2982830833 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f453809a100 con 0x5634c04511f0 2024-05-09T00:06:52.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4554fd6b80 10 client.30566 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:52.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4554fd6b80 1 -- 172.21.15.12:0/2982830833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5634c01c4b60 con 0x5634c04511f0 2024-05-09T00:06:52.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4529ffb700 1 -- 172.21.15.12:0/2982830833 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5634c02f0270 con 0x5634c04511f0 2024-05-09T00:06:52.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4529ffb700 1 -- 172.21.15.12:0/2982830833 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4538098e00 con 0x5634c04511f0 2024-05-09T00:06:52.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4554fd6b80 10 client.30566 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4554fd6b80 10 client.30566 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:06:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4554fd6b80 20 client.30566 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5634c0465200 0x5634c04675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4554fd6b80 4 client.30566 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f4554fd6b80 1 -- 172.21.15.12:0/2982830833 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634c0416810 con 0x5634c0465200 2024-05-09T00:06:52.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.250+0000 7f454259c700 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5634c0465200 0x5634c04675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.251+0000 7f454259c700 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5634c0465200 0x5634c04675f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4879abbf360fb326 server_cookie=f4c6b4c3eff459df in_seq=0 out_seq=0 2024-05-09T00:06:52.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.251+0000 7f4529ffb700 10 client.30566 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:06:52.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4529ffb700 1 -- 172.21.15.12:0/2982830833 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634c0416810 con 0x5634c0465200 2024-05-09T00:06:52.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4529ffb700 10 client.30566 handle_command_reply: tid=0 2024-05-09T00:06:52.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4554fd6b80 1 client.30566 shutdown 2024-05-09T00:06:52.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4554fd6b80 1 -- 172.21.15.12:0/2982830833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c04511f0 msgr2=0x5634c04553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c04511f0 0x5634c04553c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f45380a24e0 tx=0x7f4538099740 comp rx=0 tx=0).stop 2024-05-09T00:06:52.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4554fd6b80 1 -- 172.21.15.12:0/2982830833 shutdown_connections 2024-05-09T00:06:52.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0455900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5634c0465200 0x5634c04675f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c044bca0 0x5634c0454e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/2982830833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c04511f0 0x5634c04553c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4529ffb700 0 client.30566 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:06:52.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.252+0000 7f4554fd6b80 1 -- 172.21.15.12:0/2982830833 >> 172.21.15.12:0/2982830833 conn(0x5634c03a5210 msgr2=0x5634c04415a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.253+0000 7f4554fd6b80 1 -- 172.21.15.12:0/2982830833 shutdown_connections 2024-05-09T00:06:52.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.253+0000 7f4554fd6b80 1 -- 172.21.15.12:0/2982830833 wait complete. 2024-05-09T00:06:52.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.253+0000 7f4554fd6b80 20 client.30566 trim_cache size 0 max 16384 2024-05-09T00:06:52.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.269+0000 7f4554fd6b80 1 Processor -- start 2024-05-09T00:06:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.269+0000 7f4554fd6b80 1 -- start start 2024-05-09T00:06:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.269+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 0x5634c0460fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.269+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 0x5634c04439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0454760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c0454ca0 con 0x5634c0451b10 2024-05-09T00:06:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c0458070 con 0x5634c044bca0 2024-05-09T00:06:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c043edc0 con 0x5634c04511f0 2024-05-09T00:06:52.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4542d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0454760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 0x5634c04439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f454259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 0x5634c0460fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4542d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0454760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59444/0 (socket says 172.21.15.12:59444) 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4542d9d700 1 -- 172.21.15.12:0/666977943 learned_addr learned my addr 172.21.15.12:0/666977943 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4541d9b700 1 -- 172.21.15.12:0/666977943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 msgr2=0x5634c0460fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4541d9b700 1 --2- 172.21.15.12:0/666977943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 0x5634c0460fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4541d9b700 1 -- 172.21.15.12:0/666977943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 msgr2=0x5634c0454760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4541d9b700 1 --2- 172.21.15.12:0/666977943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0454760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4541d9b700 1 -- 172.21.15.12:0/666977943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634c035d560 con 0x5634c04511f0 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4542d9d700 1 --2- 172.21.15.12:0/666977943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0454760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.270+0000 7f4541d9b700 1 --2- 172.21.15.12:0/666977943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 0x5634c04439d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f453800f010 tx=0x7f45380996a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8bdee7037ed9ac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.271+0000 7f45317fa700 1 -- 172.21.15.12:0/666977943 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538099ea0 con 0x5634c04511f0 2024-05-09T00:06:52.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.271+0000 7f45317fa700 1 -- 172.21.15.12:0/666977943 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4538010860 con 0x5634c04511f0 2024-05-09T00:06:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.271+0000 7f45317fa700 1 -- 172.21.15.12:0/666977943 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380a1e90 con 0x5634c04511f0 2024-05-09T00:06:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.271+0000 7f4554fd6b80 1 -- 172.21.15.12:0/666977943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 msgr2=0x5634c04439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.271+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/666977943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 0x5634c04439d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f453800f010 tx=0x7f45380996a0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.271+0000 7f4554fd6b80 1 -- 172.21.15.12:0/666977943 shutdown_connections 2024-05-09T00:06:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.271+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/666977943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c0451b10 0x5634c0454760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.271+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/666977943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 0x5634c0460fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.271+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/666977943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c04511f0 0x5634c04439d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.271+0000 7f4554fd6b80 1 -- 172.21.15.12:0/666977943 >> 172.21.15.12:0/666977943 conn(0x5634c03a5210 msgr2=0x5634c044c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.272+0000 7f4554fd6b80 1 -- 172.21.15.12:0/666977943 shutdown_connections 2024-05-09T00:06:52.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.272+0000 7f4554fd6b80 1 -- 172.21.15.12:0/666977943 wait complete. 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.272+0000 7f4554fd6b80 1 -- start start 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.272+0000 7f4554fd6b80 1 Processor -- start 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.272+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 0x5634c045c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.272+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c04511f0 0x5634c044cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.272+0000 7f4554fd6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c0451b10 0x5634c044d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.272+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c044b3c0 con 0x5634c04511f0 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.272+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c044b530 con 0x5634c044bca0 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.272+0000 7f4554fd6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c044d7a0 con 0x5634c0451b10 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f4554fd6b80 10 client.0 _ll_register_callbacks cb 0x5634c033cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f454259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 0x5634c045c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f454259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 0x5634c045c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48666/0 (socket says 172.21.15.12:48666) 2024-05-09T00:06:52.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f454259c700 1 -- 172.21.15.12:0/1587997372 learned_addr learned my addr 172.21.15.12:0/1587997372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f4541d9b700 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c04511f0 0x5634c044cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f454259c700 1 -- 172.21.15.12:0/1587997372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c0451b10 msgr2=0x5634c044d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f454259c700 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c0451b10 0x5634c044d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f454259c700 1 -- 172.21.15.12:0/1587997372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c04511f0 msgr2=0x5634c044cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f454259c700 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c04511f0 0x5634c044cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f454259c700 1 -- 172.21.15.12:0/1587997372 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454410e670 con 0x5634c044bca0 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f454259c700 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 0x5634c045c730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x5634c0467f00 tx=0x7f45340047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf3f4a7aaba8524 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.273+0000 7f4529ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.274+0000 7f4529ffb700 1 -- 172.21.15.12:0/1587997372 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534022070 con 0x5634c044bca0 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.274+0000 7f4529ffb700 1 -- 172.21.15.12:0/1587997372 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4534003d60 con 0x5634c044bca0 2024-05-09T00:06:52.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.274+0000 7f4554fd6b80 1 -- 172.21.15.12:0/1587997372 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f453809dcd0 con 0x5634c044bca0 2024-05-09T00:06:52.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 261230 32.75 MB/sec execute 287 sec latency 115.757 ms 2024-05-09T00:06:52.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.274+0000 7f4529ffb700 1 -- 172.21.15.12:0/1587997372 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f453402b0d0 con 0x5634c044bca0 2024-05-09T00:06:52.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.274+0000 7f4554fd6b80 10 client.30572 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:52.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.274+0000 7f4554fd6b80 1 -- 172.21.15.12:0/1587997372 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5634c01c4b60 con 0x5634c044bca0 2024-05-09T00:06:52.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.275+0000 7f4529ffb700 1 -- 172.21.15.12:0/1587997372 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f453809dcd0 con 0x5634c044bca0 2024-05-09T00:06:52.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.275+0000 7f4529ffb700 1 -- 172.21.15.12:0/1587997372 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f453403b070 con 0x5634c044bca0 2024-05-09T00:06:52.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.276+0000 7f4554fd6b80 10 client.30572 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:52.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.276+0000 7f4554fd6b80 10 client.30572 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.276+0000 7f4554fd6b80 20 client.30572 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.276+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5634c04c0140 0x5634c04c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.276+0000 7f4554fd6b80 4 client.30572 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.276+0000 7f4554fd6b80 1 -- 172.21.15.12:0/1587997372 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5634c038fec0 con 0x5634c04c0140 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.276+0000 7f4541d9b700 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5634c04c0140 0x5634c04c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.276+0000 7f4541d9b700 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5634c04c0140 0x5634c04c2560 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6024399353f0885a server_cookie=2e567063a081e5b in_seq=0 out_seq=0 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.276+0000 7f4529ffb700 10 client.30572 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4529ffb700 1 -- 172.21.15.12:0/1587997372 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5634c038fec0 con 0x5634c04c0140 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4529ffb700 10 client.30572 handle_command_reply: tid=0 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 client.30572 shutdown 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 -- 172.21.15.12:0/1587997372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 msgr2=0x5634c045c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 0x5634c045c730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x5634c0467f00 tx=0x7f45340047c0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 -- 172.21.15.12:0/1587997372 shutdown_connections 2024-05-09T00:06:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x5634c04c0140 0x5634c04c2560 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634c0451b10 0x5634c044d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5634c044bca0 0x5634c045c730 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 --2- 172.21.15.12:0/1587997372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634c04511f0 0x5634c044cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 -- 172.21.15.12:0/1587997372 >> 172.21.15.12:0/1587997372 conn(0x5634c03a5210 msgr2=0x5634c0456720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4529ffb700 0 client.30572 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:06:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 -- 172.21.15.12:0/1587997372 shutdown_connections 2024-05-09T00:06:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 1 -- 172.21.15.12:0/1587997372 wait complete. 2024-05-09T00:06:52.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.277+0000 7f4554fd6b80 20 client.30572 trim_cache size 0 max 16384 2024-05-09T00:06:52.288 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:06:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 -- 172.21.15.12:0/3591419631 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f452c0a1290 msgr2=0x7f452c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f452c0a1290 0x7f452c0a3750 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4534009c60 tx=0x7f4534009260 comp rx=0 tx=0).stop 2024-05-09T00:06:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 -- 172.21.15.12:0/3591419631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544106280 msgr2=0x7f4544197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544106280 0x7f4544197a70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f453800e9c0 tx=0x7f453800ecd0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 -- 172.21.15.12:0/3591419631 shutdown_connections 2024-05-09T00:06:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f452c0a1290 0x7f452c0a3750 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f454419be10 0x7f454419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544101160 0x7f4544197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 --2- 172.21.15.12:0/3591419631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544106280 0x7f4544197a70 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 -- 172.21.15.12:0/3591419631 >> 172.21.15.12:0/3591419631 conn(0x7f45440fc9e0 msgr2=0x7f45440fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 -- 172.21.15.12:0/3591419631 shutdown_connections 2024-05-09T00:06:52.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.278+0000 7f4548e1c700 1 -- 172.21.15.12:0/3591419631 wait complete. 2024-05-09T00:06:52.291 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:06:52.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.446+0000 7fe2fa96a700 1 -- 172.21.15.12:0/2102952999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f4105ef0 msgr2=0x7fe2f41062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.446+0000 7fe2fa96a700 1 --2- 172.21.15.12:0/2102952999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f4105ef0 0x7fe2f41062d0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400b0a0 tx=0x7fe2e400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.447+0000 7fe2fa96a700 1 -- 172.21.15.12:0/2102952999 shutdown_connections 2024-05-09T00:06:52.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.447+0000 7fe2fa96a700 1 --2- 172.21.15.12:0/2102952999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f40ffdd0 0x7fe2f410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.447+0000 7fe2fa96a700 1 --2- 172.21.15.12:0/2102952999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f4105ef0 0x7fe2f41062d0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.447+0000 7fe2fa96a700 1 --2- 172.21.15.12:0/2102952999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2f40ff410 0x7fe2f40ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.447+0000 7fe2fa96a700 1 -- 172.21.15.12:0/2102952999 >> 172.21.15.12:0/2102952999 conn(0x7fe2f40763a0 msgr2=0x7fe2f40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.447+0000 7fe2fa96a700 1 -- 172.21.15.12:0/2102952999 shutdown_connections 2024-05-09T00:06:52.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.447+0000 7fe2fa96a700 1 -- 172.21.15.12:0/2102952999 wait complete. 2024-05-09T00:06:52.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.448+0000 7fe2fa96a700 1 Processor -- start 2024-05-09T00:06:52.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.450+0000 7fe2fa96a700 1 -- start start 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.450+0000 7fe2fa96a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f40ff410 0x7fe2f41a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.450+0000 7fe2fa96a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f40ffdd0 0x7fe2f41a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.450+0000 7fe2fa96a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2f4105ef0 0x7fe2f410cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.450+0000 7fe2fa96a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f410d600 con 0x7fe2f40ff410 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.450+0000 7fe2fa96a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f410d770 con 0x7fe2f40ffdd0 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.450+0000 7fe2fa96a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f410d930 con 0x7fe2f4105ef0 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.451+0000 7fe2f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f40ff410 0x7fe2f41a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.451+0000 7fe2f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f40ff410 0x7fe2f41a38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59466/0 (socket says 172.21.15.12:59466) 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.451+0000 7fe2f3fff700 1 -- 172.21.15.12:0/1845108820 learned_addr learned my addr 172.21.15.12:0/1845108820 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.451+0000 7fe2f8f07700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2f4105ef0 0x7fe2f410cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.451+0000 7fe2f3fff700 1 -- 172.21.15.12:0/1845108820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2f4105ef0 msgr2=0x7fe2f410cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.451+0000 7fe2f3fff700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2f4105ef0 0x7fe2f410cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.451+0000 7fe2f3fff700 1 -- 172.21.15.12:0/1845108820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f40ffdd0 msgr2=0x7fe2f41a3df0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:06:52.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.451+0000 7fe2f3fff700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f40ffdd0 0x7fe2f41a3df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.451+0000 7fe2f3fff700 1 -- 172.21.15.12:0/1845108820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4009d00 con 0x7fe2f40ff410 2024-05-09T00:06:52.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.451+0000 7fe2f3fff700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f40ff410 0x7fe2f41a38b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400b780 tx=0x7fe2e4007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.452+0000 7fe2f1ffb700 1 -- 172.21.15.12:0/1845108820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e400e040 con 0x7fe2f40ff410 2024-05-09T00:06:52.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.452+0000 7fe2fa96a700 1 -- 172.21.15.12:0/1845108820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f410dbb0 con 0x7fe2f40ff410 2024-05-09T00:06:52.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.452+0000 7fe2fa96a700 1 -- 172.21.15.12:0/1845108820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f406aba0 con 0x7fe2f40ff410 2024-05-09T00:06:52.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.452+0000 7fe2f1ffb700 1 -- 172.21.15.12:0/1845108820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2e401d470 con 0x7fe2f40ff410 2024-05-09T00:06:52.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.452+0000 7fe2f1ffb700 1 -- 172.21.15.12:0/1845108820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e40266b0 con 0x7fe2f40ff410 2024-05-09T00:06:52.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.457+0000 7fe306b24b80 1 Processor -- start 2024-05-09T00:06:52.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.457+0000 7fe2f1ffb700 1 -- 172.21.15.12:0/1845108820 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe2e4005040 con 0x7fe2f40ff410 2024-05-09T00:06:52.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.458+0000 7fe306b24b80 1 -- start start 2024-05-09T00:06:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.458+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a3103301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.458+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a310330720 0x55a31032a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.458+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a31032a890 0x55a31032acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.458+0000 7fe306b24b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a310281150 con 0x55a310330720 2024-05-09T00:06:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.458+0000 7fe306b24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31027eb10 con 0x55a31032a890 2024-05-09T00:06:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.458+0000 7fe306b24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31027a400 con 0x55a31032fe00 2024-05-09T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.459+0000 7fe2f1ffb700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2e00a1310 0x7fe2e00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.459+0000 7fe2f1ffb700 1 -- 172.21.15.12:0/1845108820 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe2e40cd880 con 0x7fe2f40ff410 2024-05-09T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.459+0000 7fe2f8f07700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a3103301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.459+0000 7fe2f8f07700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a3103301e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39044/0 (socket says 172.21.15.12:39044) 2024-05-09T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.459+0000 7fe2f8f07700 1 -- 172.21.15.12:0/2048674617 learned_addr learned my addr 172.21.15.12:0/2048674617 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2f8f07700 1 -- 172.21.15.12:0/2048674617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a31032a890 msgr2=0x55a31032acd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2f8f07700 1 --2- 172.21.15.12:0/2048674617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a31032a890 0x55a31032acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2f8f07700 1 -- 172.21.15.12:0/2048674617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a310330720 msgr2=0x55a31032a350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2f8f07700 1 --2- 172.21.15.12:0/2048674617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a310330720 0x55a31032a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2f8f07700 1 -- 172.21.15.12:0/2048674617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3100fb3b0 con 0x55a31032fe00 2024-05-09T00:06:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2f8f07700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2e00a1310 0x7fe2e00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2f8f07700 1 --2- 172.21.15.12:0/2048674617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a3103301e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009e50 tx=0x7fe2ec013740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea2a7fd5efb73831 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2ea7fc700 1 -- 172.21.15.12:0/2048674617 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec010430 con 0x55a31032fe00 2024-05-09T00:06:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2ea7fc700 1 -- 172.21.15.12:0/2048674617 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2ec013150 con 0x55a31032fe00 2024-05-09T00:06:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2ea7fc700 1 -- 172.21.15.12:0/2048674617 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00e610 con 0x55a31032fe00 2024-05-09T00:06:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.460+0000 7fe2f8f07700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2e00a1310 0x7fe2e00a37d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec022000 tx=0x7fe2ec01f000 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.461+0000 7fe306b24b80 1 -- 172.21.15.12:0/2048674617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 msgr2=0x55a3103301e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.461+0000 7fe306b24b80 1 --2- 172.21.15.12:0/2048674617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a3103301e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009e50 tx=0x7fe2ec013740 comp rx=0 tx=0).stop 2024-05-09T00:06:52.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.461+0000 7fe306b24b80 1 -- 172.21.15.12:0/2048674617 shutdown_connections 2024-05-09T00:06:52.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.461+0000 7fe306b24b80 1 --2- 172.21.15.12:0/2048674617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a31032a890 0x55a31032acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.461+0000 7fe306b24b80 1 --2- 172.21.15.12:0/2048674617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a3103301e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.461+0000 7fe306b24b80 1 --2- 172.21.15.12:0/2048674617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a310330720 0x55a31032a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.461+0000 7fe306b24b80 1 -- 172.21.15.12:0/2048674617 >> 172.21.15.12:0/2048674617 conn(0x55a310288210 msgr2=0x55a310324df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.462+0000 7fe306b24b80 1 -- 172.21.15.12:0/2048674617 shutdown_connections 2024-05-09T00:06:52.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.462+0000 7fe306b24b80 1 -- 172.21.15.12:0/2048674617 wait complete. 2024-05-09T00:06:52.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.462+0000 7fe306b24b80 1 -- start start 2024-05-09T00:06:52.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.462+0000 7fe306b24b80 1 Processor -- start 2024-05-09T00:06:52.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032a890 0x55a310333b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032fe00 0x55a3103340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a310334600 0x55a310340220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe306b24b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a310216560 con 0x55a31032fe00 2024-05-09T00:06:52.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe306b24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a310216200 con 0x55a310334600 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe306b24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3100fd6a0 con 0x55a31032a890 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe306b24b80 10 client.0 _ll_register_callbacks cb 0x55a31021fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe2f3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032a890 0x55a310333b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe2f3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032a890 0x55a310333b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39050/0 (socket says 172.21.15.12:39050) 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe2f3fff700 1 -- 172.21.15.12:0/1159341936 learned_addr learned my addr 172.21.15.12:0/1159341936 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe2ebfff700 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032fe00 0x55a3103340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe2f3fff700 1 -- 172.21.15.12:0/1159341936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a310334600 msgr2=0x55a310340220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe2f3fff700 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a310334600 0x55a310340220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe2f3fff700 1 -- 172.21.15.12:0/1159341936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032fe00 msgr2=0x55a3103340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe2f3fff700 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032fe00 0x55a3103340c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.463+0000 7fe2f3fff700 1 -- 172.21.15.12:0/1159341936 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec009b00 con 0x55a31032a890 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.464+0000 7fe2f3fff700 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032a890 0x55a310333b80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4000f80 tx=0x7fe2e409a5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a06841abbb6415 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.464+0000 7fe2d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:52.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.464+0000 7fe2d37fe700 1 -- 172.21.15.12:0/1159341936 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e401daa0 con 0x55a31032a890 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.464+0000 7fe2d37fe700 1 -- 172.21.15.12:0/1159341936 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe2e409aa60 con 0x55a31032a890 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.464+0000 7fe306b24b80 1 -- 172.21.15.12:0/1159341936 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3101d3270 con 0x55a31032a890 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.464+0000 7fe2d37fe700 1 -- 172.21.15.12:0/1159341936 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2e409e890 con 0x55a31032a890 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.464+0000 7fe306b24b80 10 client.30781 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.464+0000 7fe306b24b80 1 -- 172.21.15.12:0/1159341936 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55a3100a7b60 con 0x55a31032a890 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.464+0000 7fe2d37fe700 1 -- 172.21.15.12:0/1159341936 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3101d3270 con 0x55a31032a890 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.465+0000 7fe2d37fe700 1 -- 172.21.15.12:0/1159341936 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe2e40b3140 con 0x55a31032a890 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.465+0000 7fe306b24b80 10 client.30781 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.465+0000 7fe306b24b80 10 client.30781 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.465+0000 7fe306b24b80 20 client.30781 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.465+0000 7fe306b24b80 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55a3103442f0 0x55a3103466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.465+0000 7fe306b24b80 4 client.30781 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.465+0000 7fe306b24b80 1 -- 172.21.15.12:0/1159341936 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3102f9810 con 0x55a3103442f0 2024-05-09T00:06:52.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.465+0000 7fe2ebfff700 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55a3103442f0 0x55a3103466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.465+0000 7fe2ebfff700 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55a3103442f0 0x55a3103466e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0506ffba091d9d6 server_cookie=c0595d5054983eea in_seq=0 out_seq=0 2024-05-09T00:06:52.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.465+0000 7fe2d37fe700 10 client.30781 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.467+0000 7fe2d37fe700 1 -- 172.21.15.12:0/1159341936 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a3102f9810 con 0x55a3103442f0 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.467+0000 7fe2d37fe700 10 client.30781 handle_command_reply: tid=0 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.467+0000 7fe306b24b80 1 client.30781 shutdown 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 1 -- 172.21.15.12:0/1159341936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032a890 msgr2=0x55a310333b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032a890 0x55a310333b80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4000f80 tx=0x7fe2e409a5a0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 1 -- 172.21.15.12:0/1159341936 shutdown_connections 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55a3103442f0 0x55a3103466e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a310334600 0x55a310340220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032a890 0x55a310333b80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 1 --2- 172.21.15.12:0/1159341936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032fe00 0x55a3103340c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 1 -- 172.21.15.12:0/1159341936 >> 172.21.15.12:0/1159341936 conn(0x55a310288210 msgr2=0x55a3103245a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe2d37fe700 0 client.30781 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:06:52.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 1 -- 172.21.15.12:0/1159341936 shutdown_connections 2024-05-09T00:06:52.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 1 -- 172.21.15.12:0/1159341936 wait complete. 2024-05-09T00:06:52.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.468+0000 7fe306b24b80 20 client.30781 trim_cache size 0 max 16384 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.487+0000 7fe306b24b80 1 Processor -- start 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.487+0000 7fe306b24b80 1 -- start start 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a31032a890 0x55a31032a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032fe00 0x55a310324d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a310334600 0x55a31033b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe306b24b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31031f840 con 0x55a31032fe00 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe306b24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a310334420 con 0x55a31032a890 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe306b24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31031fa90 con 0x55a310334600 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe2f8f07700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a310334600 0x55a31033b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe2f8f07700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a310334600 0x55a31033b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39058/0 (socket says 172.21.15.12:39058) 2024-05-09T00:06:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe2f8f07700 1 -- 172.21.15.12:0/1613648769 learned_addr learned my addr 172.21.15.12:0/1613648769 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe2f8f07700 1 -- 172.21.15.12:0/1613648769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a31032a890 msgr2=0x55a31032a350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:06:52.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe2f8f07700 1 --2- 172.21.15.12:0/1613648769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a31032a890 0x55a31032a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe2f8f07700 1 -- 172.21.15.12:0/1613648769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032fe00 msgr2=0x55a310324d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:06:52.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe2f8f07700 1 --2- 172.21.15.12:0/1613648769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032fe00 0x55a310324d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.488+0000 7fe2f8f07700 1 -- 172.21.15.12:0/1613648769 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3103214e0 con 0x55a310334600 2024-05-09T00:06:52.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.489+0000 7fe2f8f07700 1 --2- 172.21.15.12:0/1613648769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a310334600 0x55a31033b160 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec013b40 tx=0x7fe2ec0092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=567fa4add22c039f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.489+0000 7fe2eaffd700 1 -- 172.21.15.12:0/1613648769 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0132c0 con 0x55a310334600 2024-05-09T00:06:52.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.489+0000 7fe2eaffd700 1 -- 172.21.15.12:0/1613648769 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2ec005c20 con 0x55a310334600 2024-05-09T00:06:52.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.489+0000 7fe2eaffd700 1 -- 172.21.15.12:0/1613648769 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec027680 con 0x55a310334600 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.490+0000 7fe306b24b80 1 -- 172.21.15.12:0/1613648769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a310334600 msgr2=0x55a31033b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.490+0000 7fe306b24b80 1 --2- 172.21.15.12:0/1613648769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a310334600 0x55a31033b160 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec013b40 tx=0x7fe2ec0092e0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.490+0000 7fe306b24b80 1 -- 172.21.15.12:0/1613648769 shutdown_connections 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.490+0000 7fe306b24b80 1 --2- 172.21.15.12:0/1613648769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a310334600 0x55a31033b160 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.490+0000 7fe306b24b80 1 --2- 172.21.15.12:0/1613648769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a31032a890 0x55a31032a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.490+0000 7fe306b24b80 1 --2- 172.21.15.12:0/1613648769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032fe00 0x55a310324d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.490+0000 7fe306b24b80 1 -- 172.21.15.12:0/1613648769 >> 172.21.15.12:0/1613648769 conn(0x55a310288210 msgr2=0x55a31032dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.490+0000 7fe306b24b80 1 -- 172.21.15.12:0/1613648769 shutdown_connections 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.490+0000 7fe306b24b80 1 -- 172.21.15.12:0/1613648769 wait complete. 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.491+0000 7fe306b24b80 1 -- start start 2024-05-09T00:06:52.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.491+0000 7fe306b24b80 1 Processor -- start 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.491+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032a890 0x55a31033b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.491+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a31033b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.491+0000 7fe306b24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a310334600 0x55a31032b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.491+0000 7fe306b24b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31032bed0 con 0x55a31032a890 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.491+0000 7fe306b24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31032c040 con 0x55a310334600 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.491+0000 7fe306b24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31032c200 con 0x55a31032fe00 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe306b24b80 10 client.0 _ll_register_callbacks cb 0x55a31021fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032a890 0x55a31033b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032a890 0x55a31033b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59480/0 (socket says 172.21.15.12:59480) 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2f3fff700 1 -- 172.21.15.12:0/3646867905 learned_addr learned my addr 172.21.15.12:0/3646867905 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2ebfff700 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a31033b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2f8f07700 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a310334600 0x55a31032b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2f3fff700 1 -- 172.21.15.12:0/3646867905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 msgr2=0x55a31033b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2f3fff700 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a31033b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2f3fff700 1 -- 172.21.15.12:0/3646867905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a310334600 msgr2=0x55a31032b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2f3fff700 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a310334600 0x55a31032b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2f3fff700 1 -- 172.21.15.12:0/3646867905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3100fb3b0 con 0x55a31032a890 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2ebfff700 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a31033b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2f3fff700 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032a890 0x55a31033b170 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fe2e409a760 tx=0x7fe2e40097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd74290474a9620 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.492+0000 7fe2d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe2d37fe700 1 -- 172.21.15.12:0/3646867905 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4009b80 con 0x55a31032a890 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe2d37fe700 1 -- 172.21.15.12:0/3646867905 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe2e409aa60 con 0x55a31032a890 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe306b24b80 1 -- 172.21.15.12:0/3646867905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2ec013150 con 0x55a31032a890 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe2d37fe700 1 -- 172.21.15.12:0/3646867905 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2e401daa0 con 0x55a31032a890 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe306b24b80 10 client.21069 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:52.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe306b24b80 1 -- 172.21.15.12:0/3646867905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55a310240560 con 0x55a31032a890 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe2d37fe700 1 -- 172.21.15.12:0/3646867905 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2ec013150 con 0x55a31032a890 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe2d37fe700 1 -- 172.21.15.12:0/3646867905 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe2e40a29c0 con 0x55a31032a890 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe306b24b80 10 client.21069 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe306b24b80 10 client.21069 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe306b24b80 20 client.21069 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe306b24b80 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55a31039b200 0x55a31039d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe306b24b80 4 client.21069 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.493+0000 7fe306b24b80 1 -- 172.21.15.12:0/3646867905 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a310272ec0 con 0x55a31039b200 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.494+0000 7fe2ebfff700 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55a31039b200 0x55a31039d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.494+0000 7fe2ebfff700 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55a31039b200 0x55a31039d5f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adc837e5b07d5849 server_cookie=3f8f428dae78ab8b in_seq=0 out_seq=0 2024-05-09T00:06:52.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.494+0000 7fe2d37fe700 10 client.21069 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:06:52.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.495+0000 7fe2d37fe700 1 -- 172.21.15.12:0/3646867905 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a310272ec0 con 0x55a31039b200 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.495+0000 7fe2d37fe700 10 client.21069 handle_command_reply: tid=0 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.495+0000 7fe306b24b80 1 client.21069 shutdown 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 1 -- 172.21.15.12:0/3646867905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032a890 msgr2=0x55a31033b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032a890 0x55a31033b170 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fe2e409a760 tx=0x7fe2e40097f0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 1 -- 172.21.15.12:0/3646867905 shutdown_connections 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55a31039b200 0x55a31039d5f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a310334600 0x55a31032b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a31032a890 0x55a31033b170 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 1 --2- 172.21.15.12:0/3646867905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31032fe00 0x55a31033b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe2d37fe700 0 client.21069 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 1 -- 172.21.15.12:0/3646867905 >> 172.21.15.12:0/3646867905 conn(0x55a310288210 msgr2=0x55a31032ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 1 -- 172.21.15.12:0/3646867905 shutdown_connections 2024-05-09T00:06:52.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 1 -- 172.21.15.12:0/3646867905 wait complete. 2024-05-09T00:06:52.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe306b24b80 20 client.21069 trim_cache size 0 max 16384 2024-05-09T00:06:52.495 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:06:52.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe2fa96a700 1 -- 172.21.15.12:0/1845108820 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2e00a1310 msgr2=0x7fe2e00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe2fa96a700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2e00a1310 0x7fe2e00a37d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec022000 tx=0x7fe2ec01f000 comp rx=0 tx=0).stop 2024-05-09T00:06:52.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe2fa96a700 1 -- 172.21.15.12:0/1845108820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f40ff410 msgr2=0x7fe2f41a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.498+0000 7fe2fa96a700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f40ff410 0x7fe2f41a38b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400b780 tx=0x7fe2e4007940 comp rx=0 tx=0).stop 2024-05-09T00:06:52.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.499+0000 7fe2fa96a700 1 -- 172.21.15.12:0/1845108820 shutdown_connections 2024-05-09T00:06:52.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.499+0000 7fe2fa96a700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2e00a1310 0x7fe2e00a37d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.499+0000 7fe2fa96a700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2f4105ef0 0x7fe2f410cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.499+0000 7fe2fa96a700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f40ff410 0x7fe2f41a38b0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.499+0000 7fe2fa96a700 1 --2- 172.21.15.12:0/1845108820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f40ffdd0 0x7fe2f41a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.499+0000 7fe2fa96a700 1 -- 172.21.15.12:0/1845108820 >> 172.21.15.12:0/1845108820 conn(0x7fe2f40763a0 msgr2=0x7fe2f4101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.499+0000 7fe2fa96a700 1 -- 172.21.15.12:0/1845108820 shutdown_connections 2024-05-09T00:06:52.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.499+0000 7fe2fa96a700 1 -- 172.21.15.12:0/1845108820 wait complete. 2024-05-09T00:06:52.509 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:06:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.812+0000 7fedbff82700 1 -- 172.21.15.12:0/3334009508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb8068c90 msgr2=0x7fedb8105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.812+0000 7fedbff82700 1 --2- 172.21.15.12:0/3334009508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb8068c90 0x7fedb8105b60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feda8009920 tx=0x7feda8009c30 comp rx=0 tx=0).stop 2024-05-09T00:06:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.812+0000 7fedbff82700 1 -- 172.21.15.12:0/3334009508 shutdown_connections 2024-05-09T00:06:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.812+0000 7fedbff82700 1 --2- 172.21.15.12:0/3334009508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb8106290 0x7fedb810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.812+0000 7fedbff82700 1 --2- 172.21.15.12:0/3334009508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedb80682e0 0x7fedb80686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.812+0000 7fedbff82700 1 --2- 172.21.15.12:0/3334009508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb8068c90 0x7fedb8105b60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.812+0000 7fedbff82700 1 -- 172.21.15.12:0/3334009508 >> 172.21.15.12:0/3334009508 conn(0x7fedb8076f40 msgr2=0x7fedb8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.812+0000 7fedbff82700 1 -- 172.21.15.12:0/3334009508 shutdown_connections 2024-05-09T00:06:52.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.812+0000 7fedbff82700 1 -- 172.21.15.12:0/3334009508 wait complete. 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.813+0000 7fedbff82700 1 Processor -- start 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.813+0000 7fedbff82700 1 -- start start 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.813+0000 7fedbff82700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedb80682e0 0x7fedb8197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.813+0000 7fedbff82700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb8106290 0x7fedb8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.813+0000 7fedbff82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb819be00 0x7fedb819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.813+0000 7fedbff82700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb819c880 con 0x7fedb819be00 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.813+0000 7fedbff82700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb819c9f0 con 0x7fedb80682e0 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.813+0000 7fedbff82700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb819cbb0 con 0x7fedb8106290 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.814+0000 7fedbe51f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb819be00 0x7fedb819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.814+0000 7fedbe51f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb819be00 0x7fedb819c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59502/0 (socket says 172.21.15.12:59502) 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.814+0000 7fedbe51f700 1 -- 172.21.15.12:0/390811247 learned_addr learned my addr 172.21.15.12:0/390811247 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.814+0000 7fedbd51d700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb8106290 0x7fedb8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.814+0000 7fedbe51f700 1 -- 172.21.15.12:0/390811247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb8106290 msgr2=0x7fedb8197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.814+0000 7fedbe51f700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb8106290 0x7fedb8197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.814+0000 7fedbe51f700 1 -- 172.21.15.12:0/390811247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedb80682e0 msgr2=0x7fedb8197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.814+0000 7fedbe51f700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedb80682e0 0x7fedb8197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.814+0000 7fedbe51f700 1 -- 172.21.15.12:0/390811247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb0009d00 con 0x7fedb819be00 2024-05-09T00:06:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.814+0000 7fedbe51f700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb819be00 0x7fedb819c280 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fedb000e3c0 tx=0x7fedb000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.815+0000 7fedaeffd700 1 -- 172.21.15.12:0/390811247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00092d0 con 0x7fedb819be00 2024-05-09T00:06:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.815+0000 7fedaeffd700 1 -- 172.21.15.12:0/390811247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedb000f040 con 0x7fedb819be00 2024-05-09T00:06:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.815+0000 7fedaeffd700 1 -- 172.21.15.12:0/390811247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0007cc0 con 0x7fedb819be00 2024-05-09T00:06:52.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.816+0000 7fedbff82700 1 -- 172.21.15.12:0/390811247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda8009520 con 0x7fedb819be00 2024-05-09T00:06:52.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.816+0000 7fedbff82700 1 -- 172.21.15.12:0/390811247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb81a4410 con 0x7fedb819be00 2024-05-09T00:06:52.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.821+0000 7fedcc13cb80 1 Processor -- start 2024-05-09T00:06:52.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.822+0000 7fedaeffd700 1 -- 172.21.15.12:0/390811247 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fedb0003a00 con 0x7fedb819be00 2024-05-09T00:06:52.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedcc13cb80 1 -- start start 2024-05-09T00:06:52.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc1fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc11350 0x5612efc117d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 0x5612efc264e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efb68150 con 0x5612efc11350 2024-05-09T00:06:52.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efb65b10 con 0x5612efc11d10 2024-05-09T00:06:52.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efb61400 con 0x5612efc1f7e0 2024-05-09T00:06:52.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedbd51d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc1fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedbd51d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc1fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39080/0 (socket says 172.21.15.12:39080) 2024-05-09T00:06:52.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedbd51d700 1 -- 172.21.15.12:0/3193707507 learned_addr learned my addr 172.21.15.12:0/3193707507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedaeffd700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda40a1310 0x7feda40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedbd51d700 1 --2- 172.21.15.12:0/3193707507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 0x5612efc264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedbdd1e700 1 --2- 172.21.15.12:0/3193707507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc11350 0x5612efc117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedaeffd700 1 -- 172.21.15.12:0/390811247 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fedb00cc880 con 0x7fedb819be00 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedbd51d700 1 -- 172.21.15.12:0/3193707507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 msgr2=0x5612efc264e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedbd51d700 1 --2- 172.21.15.12:0/3193707507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 0x5612efc264e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedbd51d700 1 -- 172.21.15.12:0/3193707507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc11350 msgr2=0x5612efc117d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.823+0000 7fedbd51d700 1 --2- 172.21.15.12:0/3193707507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc11350 0x5612efc117d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.824+0000 7fedbd51d700 1 -- 172.21.15.12:0/3193707507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612ef9e23b0 con 0x5612efc1f7e0 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.824+0000 7fedbd51d700 1 --2- 172.21.15.12:0/3193707507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc1fbc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feda800c010 tx=0x7feda8019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8c555e2e40cd4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.824+0000 7fedbe51f700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda40a1310 0x7feda40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.824+0000 7feda3fff700 1 -- 172.21.15.12:0/3193707507 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800fb40 con 0x5612efc1f7e0 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.824+0000 7feda3fff700 1 -- 172.21.15.12:0/3193707507 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda800fca0 con 0x5612efc1f7e0 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.824+0000 7feda3fff700 1 -- 172.21.15.12:0/3193707507 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8005700 con 0x5612efc1f7e0 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3193707507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 msgr2=0x5612efc1fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedbe51f700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda40a1310 0x7feda40a37d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009fd0 tx=0x7fedb0007760 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/3193707507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc1fbc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feda800c010 tx=0x7feda8019040 comp rx=0 tx=0).stop 2024-05-09T00:06:52.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3193707507 shutdown_connections 2024-05-09T00:06:52.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/3193707507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 0x5612efc264e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/3193707507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc1fbc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/3193707507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc11350 0x5612efc117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3193707507 >> 172.21.15.12:0/3193707507 conn(0x5612efb6f210 msgr2=0x5612efc0bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3193707507 shutdown_connections 2024-05-09T00:06:52.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3193707507 wait complete. 2024-05-09T00:06:52.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.825+0000 7fedcc13cb80 1 -- start start 2024-05-09T00:06:52.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedcc13cb80 1 Processor -- start 2024-05-09T00:06:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 0x5612efc29390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc11d10 0x5612efc298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc1f7e0 0x5612efc23700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efafd560 con 0x5612efc1f7e0 2024-05-09T00:06:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efafd200 con 0x5612efc11350 2024-05-09T00:06:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ef9e46a0 con 0x5612efc11d10 2024-05-09T00:06:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedcc13cb80 10 client.0 _ll_register_callbacks cb 0x5612efb06b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedbdd1e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc11d10 0x5612efc298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedbdd1e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc11d10 0x5612efc298d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39092/0 (socket says 172.21.15.12:39092) 2024-05-09T00:06:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedbd51d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 0x5612efc29390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedbdd1e700 1 -- 172.21.15.12:0/3198358000 learned_addr learned my addr 172.21.15.12:0/3198358000 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.826+0000 7fedbd51d700 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc1f7e0 0x5612efc23700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7fedbdd1e700 1 -- 172.21.15.12:0/3198358000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 msgr2=0x5612efc29390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7fedbdd1e700 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 0x5612efc29390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7fedbdd1e700 1 -- 172.21.15.12:0/3198358000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc1f7e0 msgr2=0x5612efc23700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7fedbdd1e700 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc1f7e0 0x5612efc23700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7fedbdd1e700 1 -- 172.21.15.12:0/3198358000 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda801f040 con 0x5612efc11d10 2024-05-09T00:06:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7fedbdd1e700 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc11d10 0x5612efc298d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fedb400d260 tx=0x7fedb400d570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd6a6800b414b8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7feda17fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7feda17fa700 1 -- 172.21.15.12:0/3198358000 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb400de00 con 0x5612efc11d10 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3198358000 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612efaba270 con 0x5612efc11d10 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7feda17fa700 1 -- 172.21.15.12:0/3198358000 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fedb4014430 con 0x5612efc11d10 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7feda17fa700 1 -- 172.21.15.12:0/3198358000 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedb400b3e0 con 0x5612efc11d10 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7fedcc13cb80 10 client.30799 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.827+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3198358000 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5612ef98eb60 con 0x5612efc11d10 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.828+0000 7feda17fa700 1 -- 172.21.15.12:0/3198358000 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612efaba270 con 0x5612efc11d10 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.828+0000 7feda17fa700 1 -- 172.21.15.12:0/3198358000 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fedb40127c0 con 0x5612efc11d10 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.828+0000 7fedcc13cb80 10 client.30799 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.828+0000 7fedcc13cb80 10 client.30799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.828+0000 7fedcc13cb80 20 client.30799 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.828+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612efc339b0 0x5612efc35dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.828+0000 7fedcc13cb80 4 client.30799 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.828+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3198358000 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612efbe0810 con 0x5612efc339b0 2024-05-09T00:06:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.828+0000 7fedbd51d700 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612efc339b0 0x5612efc35dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.829+0000 7fedbd51d700 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612efc339b0 0x5612efc35dd0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8417553f1f60343f server_cookie=a3f3f9333d0dd71b in_seq=0 out_seq=0 2024-05-09T00:06:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.829+0000 7feda17fa700 10 client.30799 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7feda17fa700 1 -- 172.21.15.12:0/3198358000 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612efbe0810 con 0x5612efc339b0 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7feda17fa700 10 client.30799 handle_command_reply: tid=0 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 client.30799 shutdown 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3198358000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc11d10 msgr2=0x5612efc298d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc11d10 0x5612efc298d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fedb400d260 tx=0x7fedb400d570 comp rx=0 tx=0).stop 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3198358000 shutdown_connections 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc1f7e0 0x5612efc23700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612efc339b0 0x5612efc35dd0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 0x5612efc29390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/3198358000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc11d10 0x5612efc298d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7feda17fa700 0 client.30799 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:52.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3198358000 >> 172.21.15.12:0/3198358000 conn(0x5612efb6f210 msgr2=0x5612efc0bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3198358000 shutdown_connections 2024-05-09T00:06:52.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 1 -- 172.21.15.12:0/3198358000 wait complete. 2024-05-09T00:06:52.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.833+0000 7fedcc13cb80 20 client.30799 trim_cache size 0 max 16384 2024-05-09T00:06:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedcc13cb80 1 Processor -- start 2024-05-09T00:06:52.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedcc13cb80 1 -- start start 2024-05-09T00:06:52.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 0x5612efc11730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc11d10 0x5612efc0bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc1e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efc31040 con 0x5612efc11d10 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efc24cb0 con 0x5612efc11350 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efc301c0 con 0x5612efc1f7e0 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedbd51d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 0x5612efc11730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedbd51d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 0x5612efc11730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48752/0 (socket says 172.21.15.12:48752) 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedbd51d700 1 -- 172.21.15.12:0/1757170156 learned_addr learned my addr 172.21.15.12:0/1757170156 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.850+0000 7fedbd51d700 1 --2- 172.21.15.12:0/1757170156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc1e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.851+0000 7fedbdd1e700 1 --2- 172.21.15.12:0/1757170156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc11d10 0x5612efc0bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.851+0000 7fedbd51d700 1 -- 172.21.15.12:0/1757170156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 msgr2=0x5612efc1e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.851+0000 7fedbd51d700 1 --2- 172.21.15.12:0/1757170156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc1e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.851+0000 7fedbd51d700 1 -- 172.21.15.12:0/1757170156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc11d10 msgr2=0x5612efc0bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.851+0000 7fedbd51d700 1 --2- 172.21.15.12:0/1757170156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc11d10 0x5612efc0bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.851+0000 7fedbd51d700 1 -- 172.21.15.12:0/1757170156 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612efb27560 con 0x5612efc11350 2024-05-09T00:06:52.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.851+0000 7fedbd51d700 1 --2- 172.21.15.12:0/1757170156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 0x5612efc11730 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7feda8009460 tx=0x7feda8010bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=799ca83ed162a433 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.851+0000 7fedacff9700 1 -- 172.21.15.12:0/1757170156 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800f610 con 0x5612efc11350 2024-05-09T00:06:52.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.851+0000 7fedacff9700 1 -- 172.21.15.12:0/1757170156 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda800f770 con 0x5612efc11350 2024-05-09T00:06:52.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.851+0000 7fedacff9700 1 -- 172.21.15.12:0/1757170156 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8039740 con 0x5612efc11350 2024-05-09T00:06:52.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.852+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1757170156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 msgr2=0x5612efc11730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.852+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/1757170156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 0x5612efc11730 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7feda8009460 tx=0x7feda8010bb0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.852+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1757170156 shutdown_connections 2024-05-09T00:06:52.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.852+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/1757170156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc1e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.852+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/1757170156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11350 0x5612efc11730 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.852+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/1757170156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc11d10 0x5612efc0bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.852+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1757170156 >> 172.21.15.12:0/1757170156 conn(0x5612efb6f210 msgr2=0x5612efc21ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.852+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1757170156 shutdown_connections 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.852+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1757170156 wait complete. 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedcc13cb80 1 -- start start 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedcc13cb80 1 Processor -- start 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 0x5612efc2f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedcc13cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc82840 0x5612efc82ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efc83340 con 0x5612efc82840 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efc834b0 con 0x5612efc11d10 2024-05-09T00:06:52.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedcc13cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612efc83670 con 0x5612efc1f7e0 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedcc13cb80 10 client.0 _ll_register_callbacks cb 0x5612efb06b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedbd51d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 0x5612efc2f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedbd51d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 0x5612efc2f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48768/0 (socket says 172.21.15.12:48768) 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedbd51d700 1 -- 172.21.15.12:0/1307948181 learned_addr learned my addr 172.21.15.12:0/1307948181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.853+0000 7fedbd51d700 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc82840 0x5612efc82ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.854+0000 7fedbdd1e700 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.854+0000 7fedbd51d700 1 -- 172.21.15.12:0/1307948181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 msgr2=0x5612efc2fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.854+0000 7fedbd51d700 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc2fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.854+0000 7fedbd51d700 1 -- 172.21.15.12:0/1307948181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc82840 msgr2=0x5612efc82ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.854+0000 7fedbd51d700 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc82840 0x5612efc82ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.854+0000 7fedbd51d700 1 -- 172.21.15.12:0/1307948181 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda8032040 con 0x5612efc11d10 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.854+0000 7fedbd51d700 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 0x5612efc2f640 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7feda80059d0 tx=0x7feda8010eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20acdcfee70ec1e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.854+0000 7feda17fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:06:52.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.855+0000 7feda17fa700 1 -- 172.21.15.12:0/1307948181 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8040040 con 0x5612efc11d10 2024-05-09T00:06:52.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.855+0000 7feda17fa700 1 -- 172.21.15.12:0/1307948181 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7feda8039360 con 0x5612efc11d10 2024-05-09T00:06:52.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.855+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1307948181 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feda800fca0 con 0x5612efc11d10 2024-05-09T00:06:52.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.855+0000 7feda17fa700 1 -- 172.21.15.12:0/1307948181 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7feda800f530 con 0x5612efc11d10 2024-05-09T00:06:52.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.855+0000 7fedcc13cb80 10 client.30590 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:52.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.855+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1307948181 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5612ef9e6cd0 con 0x5612efc11d10 2024-05-09T00:06:52.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.856+0000 7feda17fa700 1 -- 172.21.15.12:0/1307948181 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda800fca0 con 0x5612efc11d10 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.856+0000 7feda17fa700 1 -- 172.21.15.12:0/1307948181 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feda8015020 con 0x5612efc11d10 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.856+0000 7fedcc13cb80 10 client.30590 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.856+0000 7fedcc13cb80 10 client.30590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.856+0000 7fedcc13cb80 20 client.30590 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.856+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612efc9b050 0x5612efc9d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.856+0000 7fedcc13cb80 4 client.30590 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.856+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1307948181 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612efb59ec0 con 0x5612efc9b050 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.856+0000 7fedbdd1e700 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612efc9b050 0x5612efc9d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.857+0000 7fedbdd1e700 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612efc9b050 0x5612efc9d470 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b884d3c54edff89 server_cookie=feb29c5f05ab737 in_seq=0 out_seq=0 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.857+0000 7feda17fa700 10 client.30590 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7feda17fa700 1 -- 172.21.15.12:0/1307948181 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612efb59ec0 con 0x5612efc9b050 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7feda17fa700 10 client.30590 handle_command_reply: tid=0 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7fedcc13cb80 1 client.30590 shutdown 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1307948181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 msgr2=0x5612efc2f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 0x5612efc2f640 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7feda80059d0 tx=0x7feda8010eb0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1307948181 shutdown_connections 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612efc82840 0x5612efc82ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5612efc9b050 0x5612efc9d470 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5612efc11d10 0x5612efc2f640 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7fedcc13cb80 1 --2- 172.21.15.12:0/1307948181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612efc1f7e0 0x5612efc2fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1307948181 >> 172.21.15.12:0/1307948181 conn(0x5612efb6f210 msgr2=0x5612efc1e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7feda17fa700 0 client.30590 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.858+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1307948181 shutdown_connections 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.859+0000 7fedcc13cb80 1 -- 172.21.15.12:0/1307948181 wait complete. 2024-05-09T00:06:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.859+0000 7fedcc13cb80 20 client.30590 trim_cache size 0 max 16384 2024-05-09T00:06:52.857 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:06:52.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.859+0000 7fedbff82700 1 -- 172.21.15.12:0/390811247 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda40a1310 msgr2=0x7feda40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.859+0000 7fedbff82700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda40a1310 0x7feda40a37d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009fd0 tx=0x7fedb0007760 comp rx=0 tx=0).stop 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.859+0000 7fedbff82700 1 -- 172.21.15.12:0/390811247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb819be00 msgr2=0x7fedb819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.859+0000 7fedbff82700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb819be00 0x7fedb819c280 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fedb000e3c0 tx=0x7fedb000e6d0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.860+0000 7fedbff82700 1 -- 172.21.15.12:0/390811247 shutdown_connections 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.860+0000 7fedbff82700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7feda40a1310 0x7feda40a37d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.860+0000 7fedbff82700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb819be00 0x7fedb819c280 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.860+0000 7fedbff82700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedb80682e0 0x7fedb8197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.860+0000 7fedbff82700 1 --2- 172.21.15.12:0/390811247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb8106290 0x7fedb8197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.860+0000 7fedbff82700 1 -- 172.21.15.12:0/390811247 >> 172.21.15.12:0/390811247 conn(0x7fedb8076f40 msgr2=0x7fedb80fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.860+0000 7fedbff82700 1 -- 172.21.15.12:0/390811247 shutdown_connections 2024-05-09T00:06:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:52.860+0000 7fedbff82700 1 -- 172.21.15.12:0/390811247 wait complete. 2024-05-09T00:06:52.871 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:52 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/628730798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:06:53.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:52 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/628730798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:06:53.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:52 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/628730798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:06:53.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 261626 32.71 MB/sec execute 288 sec latency 311.736 ms 2024-05-09T00:06:53.872 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:06:53.873 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:06:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.034+0000 7fe11ffe5700 1 -- 172.21.15.12:0/845071579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1180689f0 msgr2=0x7fe11810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.034+0000 7fe11ffe5700 1 --2- 172.21.15.12:0/845071579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1180689f0 0x7fe11810dd40 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe1100098f0 tx=0x7fe110009c00 comp rx=0 tx=0).stop 2024-05-09T00:06:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.034+0000 7fe11ffe5700 1 -- 172.21.15.12:0/845071579 shutdown_connections 2024-05-09T00:06:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.034+0000 7fe11ffe5700 1 --2- 172.21.15.12:0/845071579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1180689f0 0x7fe11810dd40 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.034+0000 7fe11ffe5700 1 --2- 172.21.15.12:0/845071579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe118105ed0 0x7fe1181062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.034+0000 7fe11ffe5700 1 --2- 172.21.15.12:0/845071579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe118068030 0x7fe1180684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.034+0000 7fe11ffe5700 1 -- 172.21.15.12:0/845071579 >> 172.21.15.12:0/845071579 conn(0x7fe118076f40 msgr2=0x7fe118077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.034+0000 7fe11ffe5700 1 -- 172.21.15.12:0/845071579 shutdown_connections 2024-05-09T00:06:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.034+0000 7fe11ffe5700 1 -- 172.21.15.12:0/845071579 wait complete. 2024-05-09T00:06:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.035+0000 7fe11ffe5700 1 Processor -- start 2024-05-09T00:06:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.035+0000 7fe11ffe5700 1 -- start start 2024-05-09T00:06:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.035+0000 7fe11ffe5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe118068030 0x7fe1181973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.035+0000 7fe11ffe5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1180689f0 0x7fe118197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11ffe5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe118105ed0 0x7fe11819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11ffe5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11819c300 con 0x7fe118068030 2024-05-09T00:06:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11ffe5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11819c470 con 0x7fe118105ed0 2024-05-09T00:06:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11ffe5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11819c630 con 0x7fe1180689f0 2024-05-09T00:06:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11dd81700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe118068030 0x7fe1181973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11d580700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1180689f0 0x7fe118197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11dd81700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe118068030 0x7fe1181973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59584/0 (socket says 172.21.15.12:59584) 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11dd81700 1 -- 172.21.15.12:0/1219472561 learned_addr learned my addr 172.21.15.12:0/1219472561 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11e582700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe118105ed0 0x7fe11819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11d580700 1 -- 172.21.15.12:0/1219472561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe118105ed0 msgr2=0x7fe11819bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11d580700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe118105ed0 0x7fe11819bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11d580700 1 -- 172.21.15.12:0/1219472561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe118068030 msgr2=0x7fe1181973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11d580700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe118068030 0x7fe1181973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11d580700 1 -- 172.21.15.12:0/1219472561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080095f0 con 0x7fe1180689f0 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.036+0000 7fe11e582700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe118105ed0 0x7fe11819bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.037+0000 7fe11d580700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1180689f0 0x7fe118197920 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe10800eb90 tx=0x7fe10800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.037+0000 7fe10effd700 1 -- 172.21.15.12:0/1219472561 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108009970 con 0x7fe1180689f0 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.037+0000 7fe10effd700 1 -- 172.21.15.12:0/1219472561 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1080044e0 con 0x7fe1180689f0 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.037+0000 7fe11ffe5700 1 -- 172.21.15.12:0/1219472561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe110009520 con 0x7fe1180689f0 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.037+0000 7fe10effd700 1 -- 172.21.15.12:0/1219472561 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108003e50 con 0x7fe1180689f0 2024-05-09T00:06:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.037+0000 7fe11ffe5700 1 -- 172.21.15.12:0/1219472561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1181a3f70 con 0x7fe1180689f0 2024-05-09T00:06:54.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.040+0000 7fe10effd700 1 -- 172.21.15.12:0/1219472561 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe108004000 con 0x7fe1180689f0 2024-05-09T00:06:54.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.041+0000 7fe10effd700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1040a1290 0x7fe1040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.041+0000 7fe12c19fb80 1 Processor -- start 2024-05-09T00:06:54.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.041+0000 7fe10effd700 1 -- 172.21.15.12:0/1219472561 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe108014070 con 0x7fe1180689f0 2024-05-09T00:06:54.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.041+0000 7fe12c19fb80 1 -- start start 2024-05-09T00:06:54.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.041+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 0x557ef444d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.041+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44473d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.041+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef4447910 0x557ef4447d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef439e1d0 con 0x557ef444d7a0 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef439bb90 con 0x557ef444ce80 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef4397480 con 0x557ef4447910 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11d580700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11dd81700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 0x557ef444d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11d580700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44473d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59592/0 (socket says 172.21.15.12:59592) 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11d580700 1 -- 172.21.15.12:0/3767235424 learned_addr learned my addr 172.21.15.12:0/3767235424 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11e582700 1 --2- 172.21.15.12:0/3767235424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef4447910 0x557ef4447d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11dd81700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1040a1290 0x7fe1040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11d580700 1 -- 172.21.15.12:0/3767235424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef4447910 msgr2=0x557ef4447d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11d580700 1 --2- 172.21.15.12:0/3767235424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef4447910 0x557ef4447d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11d580700 1 -- 172.21.15.12:0/3767235424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 msgr2=0x557ef444d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11d580700 1 --2- 172.21.15.12:0/3767235424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 0x557ef444d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11d580700 1 -- 172.21.15.12:0/3767235424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef4218430 con 0x557ef444d7a0 2024-05-09T00:06:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11dd81700 1 --2- 172.21.15.12:0/3767235424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 0x557ef444d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11d580700 1 --2- 172.21.15.12:0/3767235424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44473d0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fe108003fd0 tx=0x7fe10809a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6a1a11ae037e6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe11dd81700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1040a1290 0x7fe1040a3750 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe114009c10 tx=0x7fe11400b040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.042+0000 7fe103fff700 1 -- 172.21.15.12:0/3767235424 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800ce20 con 0x557ef444d7a0 2024-05-09T00:06:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.043+0000 7fe103fff700 1 -- 172.21.15.12:0/3767235424 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10809a9d0 con 0x557ef444d7a0 2024-05-09T00:06:54.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.043+0000 7fe103fff700 1 -- 172.21.15.12:0/3767235424 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108098700 con 0x557ef444d7a0 2024-05-09T00:06:54.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3767235424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 msgr2=0x557ef44473d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/3767235424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44473d0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fe108003fd0 tx=0x7fe10809a0a0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3767235424 shutdown_connections 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/3767235424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef4447910 0x557ef4447d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/3767235424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 0x557ef444d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/3767235424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44473d0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3767235424 >> 172.21.15.12:0/3767235424 conn(0x557ef43a5290 msgr2=0x557ef4441e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3767235424 shutdown_connections 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3767235424 wait complete. 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 -- start start 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.044+0000 7fe12c19fb80 1 Processor -- start 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 0x557ef4450c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef4451140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef4451680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef43335e0 con 0x557ef444d7a0 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef4333280 con 0x557ef4447910 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef421a720 con 0x557ef444ce80 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe12c19fb80 10 client.0 _ll_register_callbacks cb 0x557ef433cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11e582700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef4451680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11e582700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef4451680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59600/0 (socket says 172.21.15.12:59600) 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11dd81700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 0x557ef4450c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11e582700 1 -- 172.21.15.12:0/4257988 learned_addr learned my addr 172.21.15.12:0/4257988 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11d580700 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef4451140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11e582700 1 -- 172.21.15.12:0/4257988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 msgr2=0x557ef4451140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11e582700 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef4451140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11e582700 1 -- 172.21.15.12:0/4257988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 msgr2=0x557ef4450c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11e582700 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 0x557ef4450c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11e582700 1 -- 172.21.15.12:0/4257988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe114008ba0 con 0x557ef444d7a0 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.045+0000 7fe11d580700 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef4451140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:54.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe11e582700 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef4451680 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fe1100098f0 tx=0x7fe11000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3287f6112e388b0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe1017fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe1017fa700 1 -- 172.21.15.12:0/4257988 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11001c040 con 0x557ef444d7a0 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe1017fa700 1 -- 172.21.15.12:0/4257988 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe11000bbc0 con 0x557ef444d7a0 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe12c19fb80 1 -- 172.21.15.12:0/4257988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ef42f02f0 con 0x557ef444d7a0 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe1017fa700 1 -- 172.21.15.12:0/4257988 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe110010f00 con 0x557ef444d7a0 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe12c19fb80 10 client.21099 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe12c19fb80 1 -- 172.21.15.12:0/4257988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7fe1181a3f70 con 0x557ef444d7a0 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe1017fa700 1 -- 172.21.15.12:0/4257988 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ef42f02f0 con 0x557ef444d7a0 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe1017fa700 1 -- 172.21.15.12:0/4257988 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe110015020 con 0x557ef444d7a0 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe12c19fb80 10 client.21099 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe12c19fb80 10 client.21099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe12c19fb80 20 client.21099 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:54.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557ef4461320 0x557ef4463740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe12c19fb80 4 client.21099 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.046+0000 7fe12c19fb80 1 -- 172.21.15.12:0/4257988 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ef4416890 con 0x557ef4461320 2024-05-09T00:06:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.047+0000 7fe11dd81700 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557ef4461320 0x557ef4463740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.047+0000 7fe1017fa700 10 client.21099 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:54.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.047+0000 7fe11dd81700 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557ef4461320 0x557ef4463740 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7034263d1edc9f2 server_cookie=97547e477b0d7df0 in_seq=0 out_seq=0 2024-05-09T00:06:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.051+0000 7fe1017fa700 1 -- 172.21.15.12:0/4257988 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ef4416890 con 0x557ef4461320 2024-05-09T00:06:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.051+0000 7fe1017fa700 10 client.21099 handle_command_reply: tid=0 2024-05-09T00:06:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.051+0000 7fe12c19fb80 1 client.21099 shutdown 2024-05-09T00:06:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.051+0000 7fe12c19fb80 1 -- 172.21.15.12:0/4257988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 msgr2=0x557ef4451680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.051+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef4451680 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fe1100098f0 tx=0x7fe11000b790 comp rx=0 tx=0).stop 2024-05-09T00:06:54.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.052+0000 7fe12c19fb80 1 -- 172.21.15.12:0/4257988 shutdown_connections 2024-05-09T00:06:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.052+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef4451680 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.052+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557ef4461320 0x557ef4463740 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.052+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 0x557ef4450c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.052+0000 7fe1017fa700 0 client.21099 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.052+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/4257988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef4451140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.052+0000 7fe12c19fb80 1 -- 172.21.15.12:0/4257988 >> 172.21.15.12:0/4257988 conn(0x557ef43a5290 msgr2=0x557ef4441620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.052+0000 7fe12c19fb80 1 -- 172.21.15.12:0/4257988 shutdown_connections 2024-05-09T00:06:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.052+0000 7fe12c19fb80 1 -- 172.21.15.12:0/4257988 wait complete. 2024-05-09T00:06:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.052+0000 7fe12c19fb80 20 client.21099 trim_cache size 0 max 16384 2024-05-09T00:06:54.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.072+0000 7fe12c19fb80 1 Processor -- start 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.072+0000 7fe12c19fb80 1 -- start start 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef4447910 0x557ef44473d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 0x557ef444c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444d7a0 0x557ef4441da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef445d140 con 0x557ef4447910 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef4453770 con 0x557ef444ce80 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef443ee70 con 0x557ef444d7a0 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11dd81700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef4447910 0x557ef44473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11e582700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444d7a0 0x557ef4441da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11d580700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 0x557ef444c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11dd81700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef4447910 0x557ef44473d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59602/0 (socket says 172.21.15.12:59602) 2024-05-09T00:06:54.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11dd81700 1 -- 172.21.15.12:0/1849006308 learned_addr learned my addr 172.21.15.12:0/1849006308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11e582700 1 -- 172.21.15.12:0/1849006308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 msgr2=0x557ef444c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11e582700 1 --2- 172.21.15.12:0/1849006308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 0x557ef444c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11e582700 1 -- 172.21.15.12:0/1849006308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef4447910 msgr2=0x557ef44473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11e582700 1 --2- 172.21.15.12:0/1849006308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef4447910 0x557ef44473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11e582700 1 -- 172.21.15.12:0/1849006308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef443e560 con 0x557ef444d7a0 2024-05-09T00:06:54.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11dd81700 1 --2- 172.21.15.12:0/1849006308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef4447910 0x557ef44473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:54.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.073+0000 7fe11e582700 1 --2- 172.21.15.12:0/1849006308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444d7a0 0x557ef4441da0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe110023440 tx=0x7fe110009870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36f48c5f0685ec21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.074+0000 7fe10cff9700 1 -- 172.21.15.12:0/1849006308 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11001c040 con 0x557ef444d7a0 2024-05-09T00:06:54.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.074+0000 7fe10cff9700 1 -- 172.21.15.12:0/1849006308 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe11001d040 con 0x557ef444d7a0 2024-05-09T00:06:54.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.074+0000 7fe10cff9700 1 -- 172.21.15.12:0/1849006308 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1100055d0 con 0x557ef444d7a0 2024-05-09T00:06:54.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 -- 172.21.15.12:0/1849006308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444d7a0 msgr2=0x557ef4441da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/1849006308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444d7a0 0x557ef4441da0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe110023440 tx=0x7fe110009870 comp rx=0 tx=0).stop 2024-05-09T00:06:54.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 -- 172.21.15.12:0/1849006308 shutdown_connections 2024-05-09T00:06:54.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/1849006308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444d7a0 0x557ef4441da0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/1849006308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef4447910 0x557ef44473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/1849006308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef444ce80 0x557ef444c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 -- 172.21.15.12:0/1849006308 >> 172.21.15.12:0/1849006308 conn(0x557ef43a5290 msgr2=0x557ef444bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 -- 172.21.15.12:0/1849006308 shutdown_connections 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 -- 172.21.15.12:0/1849006308 wait complete. 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 -- start start 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.075+0000 7fe12c19fb80 1 Processor -- start 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 0x557ef44ab980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef44abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe12c19fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef44b0d20 con 0x557ef444d7a0 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef44b0e90 con 0x557ef4447910 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe12c19fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef44b1050 con 0x557ef444ce80 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe12c19fb80 10 client.0 _ll_register_callbacks cb 0x557ef433cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11dd81700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 0x557ef44ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11d580700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef44abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11d580700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef44abec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39174/0 (socket says 172.21.15.12:39174) 2024-05-09T00:06:54.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11dd81700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 0x557ef44ab980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48834/0 (socket says 172.21.15.12:48834) 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11d580700 1 -- 172.21.15.12:0/3278013362 learned_addr learned my addr 172.21.15.12:0/3278013362 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11e582700 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11e582700 1 -- 172.21.15.12:0/3278013362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 msgr2=0x557ef44abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11e582700 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef44abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11e582700 1 -- 172.21.15.12:0/3278013362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 msgr2=0x557ef44ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11e582700 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 0x557ef44ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11e582700 1 -- 172.21.15.12:0/3278013362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11401ac50 con 0x557ef444d7a0 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11d580700 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef44abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.076+0000 7fe11dd81700 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 0x557ef44ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.077+0000 7fe11e582700 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44b07e0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fe11000b7d0 tx=0x7fe110003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c14b5d7d70fa85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.077+0000 7fe1017fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.077+0000 7fe1017fa700 1 -- 172.21.15.12:0/3278013362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11001c030 con 0x557ef444d7a0 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.077+0000 7fe1017fa700 1 -- 172.21.15.12:0/3278013362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe110000e40 con 0x557ef444d7a0 2024-05-09T00:06:54.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.077+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3278013362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe10809a9d0 con 0x557ef444d7a0 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.077+0000 7fe1017fa700 1 -- 172.21.15.12:0/3278013362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe110010f00 con 0x557ef444d7a0 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.077+0000 7fe12c19fb80 10 client.21105 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.077+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3278013362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7fe1181a3f70 con 0x557ef444d7a0 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.077+0000 7fe1017fa700 1 -- 172.21.15.12:0/3278013362 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10809a9d0 con 0x557ef444d7a0 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.078+0000 7fe1017fa700 1 -- 172.21.15.12:0/3278013362 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe110045030 con 0x557ef444d7a0 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.078+0000 7fe12c19fb80 10 client.21105 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.078+0000 7fe12c19fb80 10 client.21105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.078+0000 7fe12c19fb80 20 client.21105 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.078+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557ef44b8430 0x557ef44ba820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.078+0000 7fe12c19fb80 4 client.21105 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.078+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3278013362 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557ef438ff40 con 0x557ef44b8430 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.078+0000 7fe11dd81700 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557ef44b8430 0x557ef44ba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.078+0000 7fe11dd81700 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557ef44b8430 0x557ef44ba820 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c558a8356ceec9 server_cookie=e56ef76b9ab3450d in_seq=0 out_seq=0 2024-05-09T00:06:54.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.078+0000 7fe1017fa700 10 client.21105 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe1017fa700 1 -- 172.21.15.12:0/3278013362 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557ef438ff40 con 0x557ef44b8430 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe1017fa700 10 client.21105 handle_command_reply: tid=0 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe12c19fb80 1 client.21105 shutdown 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3278013362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 msgr2=0x557ef44b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44b07e0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fe11000b7d0 tx=0x7fe110003cf0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3278013362 shutdown_connections 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557ef444d7a0 0x557ef44b07e0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557ef44b8430 0x557ef44ba820 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557ef4447910 0x557ef44ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe1017fa700 0 client.21105 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:54.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe12c19fb80 1 --2- 172.21.15.12:0/3278013362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ef444ce80 0x557ef44abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.081+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3278013362 >> 172.21.15.12:0/3278013362 conn(0x557ef43a5290 msgr2=0x557ef444dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3278013362 shutdown_connections 2024-05-09T00:06:54.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe12c19fb80 1 -- 172.21.15.12:0/3278013362 wait complete. 2024-05-09T00:06:54.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe12c19fb80 20 client.21105 trim_cache size 0 max 16384 2024-05-09T00:06:54.080 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:06:54.080 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:06:54.080 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "d27f7dc2-3fdd-453d-aa13-3f5455d2fe5d", 2024-05-09T00:06:54.080 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:06:54.080 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:06:54.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe11ffe5700 1 -- 172.21.15.12:0/1219472561 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1040a1290 msgr2=0x7fe1040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe11ffe5700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1040a1290 0x7fe1040a3750 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe114009c10 tx=0x7fe11400b040 comp rx=0 tx=0).stop 2024-05-09T00:06:54.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe11ffe5700 1 -- 172.21.15.12:0/1219472561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1180689f0 msgr2=0x7fe118197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe11ffe5700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1180689f0 0x7fe118197920 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe10800eb90 tx=0x7fe10800eea0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe11ffe5700 1 -- 172.21.15.12:0/1219472561 shutdown_connections 2024-05-09T00:06:54.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe11ffe5700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1040a1290 0x7fe1040a3750 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe11ffe5700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe118105ed0 0x7fe11819bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe11ffe5700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe118068030 0x7fe1181973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe11ffe5700 1 --2- 172.21.15.12:0/1219472561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1180689f0 0x7fe118197920 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.082+0000 7fe11ffe5700 1 -- 172.21.15.12:0/1219472561 >> 172.21.15.12:0/1219472561 conn(0x7fe118076f40 msgr2=0x7fe1180fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.083+0000 7fe11ffe5700 1 -- 172.21.15.12:0/1219472561 shutdown_connections 2024-05-09T00:06:54.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.083+0000 7fe11ffe5700 1 -- 172.21.15.12:0/1219472561 wait complete. 2024-05-09T00:06:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:53 smithi012 ceph-mon[42503]: pgmap v658: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 795 op/s 2024-05-09T00:06:54.092 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:06:54.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.258+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2128550952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2f81062a0 msgr2=0x7fa2f810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.258+0000 7fa2fdac9700 1 --2- 172.21.15.12:0/2128550952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2f81062a0 0x7fa2f810de20 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000b0a0 tx=0x7fa2f000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.258+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2128550952 shutdown_connections 2024-05-09T00:06:54.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.258+0000 7fa2fdac9700 1 --2- 172.21.15.12:0/2128550952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2f81062a0 0x7fa2f810de20 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.258+0000 7fa2fdac9700 1 --2- 172.21.15.12:0/2128550952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f8101180 0x7fa2f8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.258+0000 7fa2fdac9700 1 --2- 172.21.15.12:0/2128550952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2f8101b30 0x7fa2f8105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.258+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2128550952 >> 172.21.15.12:0/2128550952 conn(0x7fa2f80fc9e0 msgr2=0x7fa2f80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.258+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2128550952 shutdown_connections 2024-05-09T00:06:54.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.258+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2128550952 wait complete. 2024-05-09T00:06:54.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.259+0000 7fa2fdac9700 1 Processor -- start 2024-05-09T00:06:54.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.259+0000 7fa2fdac9700 1 -- start start 2024-05-09T00:06:54.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.259+0000 7fa2fdac9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f8101180 0x7fa2f8195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2fdac9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2f81062a0 0x7fa2f81958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2fdac9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2f8199c20 0x7fa2f819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2fdac9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f819a6a0 con 0x7fa2f8199c20 2024-05-09T00:06:54.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2fdac9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f819a810 con 0x7fa2f81062a0 2024-05-09T00:06:54.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2fdac9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f819a9d0 con 0x7fa2f8101180 2024-05-09T00:06:54.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f8101180 0x7fa2f8195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2f8199c20 0x7fa2f819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f8101180 0x7fa2f8195390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39180/0 (socket says 172.21.15.12:39180) 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2f81062a0 0x7fa2f81958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f77fe700 1 -- 172.21.15.12:0/2922452027 learned_addr learned my addr 172.21.15.12:0/2922452027 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f77fe700 1 -- 172.21.15.12:0/2922452027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2f81062a0 msgr2=0x7fa2f81958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2f81062a0 0x7fa2f81958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f77fe700 1 -- 172.21.15.12:0/2922452027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2f8199c20 msgr2=0x7fa2f819a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2f8199c20 0x7fa2f819a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f77fe700 1 -- 172.21.15.12:0/2922452027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec009520 con 0x7fa2f8101180 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f7fff700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2f8199c20 0x7fa2f819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.260+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f8101180 0x7fa2f8195390 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009c70 tx=0x7fa2e8009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.261+0000 7fa2f4ff9700 1 -- 172.21.15.12:0/2922452027 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8007b20 con 0x7fa2f8101180 2024-05-09T00:06:54.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.261+0000 7fa2f4ff9700 1 -- 172.21.15.12:0/2922452027 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e8015040 con 0x7fa2f8101180 2024-05-09T00:06:54.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.261+0000 7fa2f4ff9700 1 -- 172.21.15.12:0/2922452027 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e80185b0 con 0x7fa2f8101180 2024-05-09T00:06:54.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.261+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2922452027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f0009d00 con 0x7fa2f8101180 2024-05-09T00:06:54.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.261+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2922452027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f81a2220 con 0x7fa2f8101180 2024-05-09T00:06:54.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.263+0000 7fa2f4ff9700 1 -- 172.21.15.12:0/2922452027 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa2e80193f0 con 0x7fa2f8101180 2024-05-09T00:06:54.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.264+0000 7fa2f4ff9700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2e00a1310 0x7fa2e00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.264+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2e00a1310 0x7fa2e00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.264+0000 7fa2f4ff9700 1 -- 172.21.15.12:0/2922452027 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa2e80d0b10 con 0x7fa2f8101180 2024-05-09T00:06:54.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2e00a1310 0x7fa2e00a37d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8196890 tx=0x7fa2ec009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa309c83b80 1 Processor -- start 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa309c83b80 1 -- start start 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f1e00 0x5615a90f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 0x5615a90ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90ec890 0x5615a90eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa309c83b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a9043150 con 0x5615a90f1e00 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa309c83b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a9040b10 con 0x5615a90ec890 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa309c83b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a903c400 con 0x5615a90f2720 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa2f77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f1e00 0x5615a90f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa2f77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f1e00 0x5615a90f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59632/0 (socket says 172.21.15.12:59632) 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa2f6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 0x5615a90ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa2f77fe700 1 -- 172.21.15.12:0/1025459539 learned_addr learned my addr 172.21.15.12:0/1025459539 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.265+0000 7fa2f7fff700 1 --2- 172.21.15.12:0/1025459539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90ec890 0x5615a90eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.266+0000 7fa2f77fe700 1 -- 172.21.15.12:0/1025459539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 msgr2=0x5615a90ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.266+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/1025459539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 0x5615a90ec350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.266+0000 7fa2f77fe700 1 -- 172.21.15.12:0/1025459539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90ec890 msgr2=0x5615a90eccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.266+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/1025459539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90ec890 0x5615a90eccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.266+0000 7fa2f77fe700 1 -- 172.21.15.12:0/1025459539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615a8ebd3b0 con 0x5615a90f1e00 2024-05-09T00:06:54.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.266+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/1025459539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 0x5615a90ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:54.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.266+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/1025459539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f1e00 0x5615a90f21e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80a5b20 tx=0x7fa2e8099f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66786243b916a35d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.266+0000 7fa2ddffb700 1 -- 172.21.15.12:0/1025459539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e809dbd0 con 0x5615a90f1e00 2024-05-09T00:06:54.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.266+0000 7fa2ddffb700 1 -- 172.21.15.12:0/1025459539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e80a1a00 con 0x5615a90f1e00 2024-05-09T00:06:54.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.266+0000 7fa2ddffb700 1 -- 172.21.15.12:0/1025459539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e809ebe0 con 0x5615a90f1e00 2024-05-09T00:06:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:53 smithi204 ceph-mon[47373]: pgmap v658: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 795 op/s 2024-05-09T00:06:54.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.267+0000 7fa309c83b80 1 -- 172.21.15.12:0/1025459539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f1e00 msgr2=0x5615a90f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.267+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1025459539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f1e00 0x5615a90f21e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80a5b20 tx=0x7fa2e8099f90 comp rx=0 tx=0).stop 2024-05-09T00:06:54.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.267+0000 7fa309c83b80 1 -- 172.21.15.12:0/1025459539 shutdown_connections 2024-05-09T00:06:54.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.267+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1025459539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90ec890 0x5615a90eccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.267+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1025459539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f1e00 0x5615a90f21e0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.267+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1025459539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 0x5615a90ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.267+0000 7fa309c83b80 1 -- 172.21.15.12:0/1025459539 >> 172.21.15.12:0/1025459539 conn(0x5615a904a210 msgr2=0x5615a90e6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.267+0000 7fa309c83b80 1 -- 172.21.15.12:0/1025459539 shutdown_connections 2024-05-09T00:06:54.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.267+0000 7fa309c83b80 1 -- 172.21.15.12:0/1025459539 wait complete. 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa309c83b80 1 -- start start 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa309c83b80 1 Processor -- start 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 0x5615a90f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f1e00 0x5615a90f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 0x5615a90f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa309c83b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a8fd8560 con 0x5615a90ec890 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa309c83b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a8fd8200 con 0x5615a90f1e00 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa309c83b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a8ebf6a0 con 0x5615a90f2720 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa309c83b80 10 client.0 _ll_register_callbacks cb 0x5615a8fe1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa2f77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 0x5615a90f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa2f77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 0x5615a90f5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59642/0 (socket says 172.21.15.12:59642) 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa2f77fe700 1 -- 172.21.15.12:0/1306657739 learned_addr learned my addr 172.21.15.12:0/1306657739 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.268+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f1e00 0x5615a90f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2f7fff700 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 0x5615a90f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2f77fe700 1 -- 172.21.15.12:0/1306657739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 msgr2=0x5615a90f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 0x5615a90f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2f77fe700 1 -- 172.21.15.12:0/1306657739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f1e00 msgr2=0x5615a90f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f1e00 0x5615a90f5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2f77fe700 1 -- 172.21.15.12:0/1306657739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec017070 con 0x5615a90ec890 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2f7fff700 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 0x5615a90f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 0x5615a90f5a90 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fa2e809a640 tx=0x7fa2e809c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591434d896e3720d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/1306657739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e809ebe0 con 0x5615a90ec890 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/1306657739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa2e8019e40 con 0x5615a90ec890 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.269+0000 7fa309c83b80 1 -- 172.21.15.12:0/1306657739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615a8f95270 con 0x5615a90ec890 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/1306657739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e809ed40 con 0x5615a90ec890 2024-05-09T00:06:54.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa309c83b80 10 client.21117 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa309c83b80 1 -- 172.21.15.12:0/1306657739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7fa2f81a2220 con 0x5615a90ec890 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/1306657739 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615a8f95270 con 0x5615a90ec890 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/1306657739 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2e80a5bc0 con 0x5615a90ec890 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa309c83b80 10 client.21117 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa309c83b80 10 client.21117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa309c83b80 20 client.21117 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5615a9105f40 0x5615a9108360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa309c83b80 4 client.21117 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.270+0000 7fa309c83b80 1 -- 172.21.15.12:0/1306657739 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615a90bb810 con 0x5615a9105f40 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.271+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5615a9105f40 0x5615a9108360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.271+0000 7fa2d6ffd700 10 client.21117 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:54.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.271+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5615a9105f40 0x5615a9108360 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85dbb01f9ca35c74 server_cookie=f8731ecf3a0ee777 in_seq=0 out_seq=0 2024-05-09T00:06:54.271 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:53 smithi160 ceph-mon[46152]: pgmap v658: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 795 op/s 2024-05-09T00:06:54.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/1306657739 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615a90bb810 con 0x5615a9105f40 2024-05-09T00:06:54.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa2d6ffd700 10 client.21117 handle_command_reply: tid=0 2024-05-09T00:06:54.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 client.21117 shutdown 2024-05-09T00:06:54.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 -- 172.21.15.12:0/1306657739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 msgr2=0x5615a90f5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 0x5615a90f5a90 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fa2e809a640 tx=0x7fa2e809c180 comp rx=0 tx=0).stop 2024-05-09T00:06:54.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 -- 172.21.15.12:0/1306657739 shutdown_connections 2024-05-09T00:06:54.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f2720 0x5615a90f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 0x5615a90f5a90 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5615a9105f40 0x5615a9108360 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1306657739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f1e00 0x5615a90f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa2d6ffd700 0 client.21117 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:54.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 -- 172.21.15.12:0/1306657739 >> 172.21.15.12:0/1306657739 conn(0x5615a904a210 msgr2=0x5615a90e65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.273 INFO:tasks.workunit.client.0.smithi012.stdout: 10 262003 32.66 MB/sec execute 289 sec latency 165.602 ms 2024-05-09T00:06:54.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 -- 172.21.15.12:0/1306657739 shutdown_connections 2024-05-09T00:06:54.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 1 -- 172.21.15.12:0/1306657739 wait complete. 2024-05-09T00:06:54.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.274+0000 7fa309c83b80 20 client.21117 trim_cache size 0 max 16384 2024-05-09T00:06:54.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.296+0000 7fa309c83b80 1 Processor -- start 2024-05-09T00:06:54.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.296+0000 7fa309c83b80 1 -- start start 2024-05-09T00:06:54.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.296+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 0x5615a90ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.296+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.296+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f2720 0x5615a90e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.296+0000 7fa309c83b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a90f6f20 con 0x5615a90ec890 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.296+0000 7fa309c83b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a90f7100 con 0x5615a90f2720 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.296+0000 7fa309c83b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a90f1930 con 0x5615a90f1e00 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.296+0000 7fa2f7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f2720 0x5615a90e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.296+0000 7fa2f7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f2720 0x5615a90e6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48870/0 (socket says 172.21.15.12:48870) 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.297+0000 7fa2f7fff700 1 -- 172.21.15.12:0/1984891569 learned_addr learned my addr 172.21.15.12:0/1984891569 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.297+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/1984891569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.297+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/1984891569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 0x5615a90ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.297+0000 7fa2f6ffd700 1 -- 172.21.15.12:0/1984891569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f2720 msgr2=0x5615a90e6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.297+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/1984891569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f2720 0x5615a90e6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.297+0000 7fa2f6ffd700 1 -- 172.21.15.12:0/1984891569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 msgr2=0x5615a90ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.297+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/1984891569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 0x5615a90ec350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.297+0000 7fa2f6ffd700 1 -- 172.21.15.12:0/1984891569 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615a90ecf40 con 0x5615a90f1e00 2024-05-09T00:06:54.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.297+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/1984891569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f13f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009b50 tx=0x7fa2ec014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79c0a2f3c513bbaa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.298+0000 7fa2de7fc700 1 -- 172.21.15.12:0/1984891569 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec024040 con 0x5615a90f1e00 2024-05-09T00:06:54.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.298+0000 7fa2de7fc700 1 -- 172.21.15.12:0/1984891569 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2ec0298e0 con 0x5615a90f1e00 2024-05-09T00:06:54.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.298+0000 7fa2de7fc700 1 -- 172.21.15.12:0/1984891569 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec01fa70 con 0x5615a90f1e00 2024-05-09T00:06:54.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 -- 172.21.15.12:0/1984891569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 msgr2=0x5615a90f13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1984891569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f13f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009b50 tx=0x7fa2ec014000 comp rx=0 tx=0).stop 2024-05-09T00:06:54.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 -- 172.21.15.12:0/1984891569 shutdown_connections 2024-05-09T00:06:54.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1984891569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90f2720 0x5615a90e6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1984891569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90ec890 0x5615a90ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 --2- 172.21.15.12:0/1984891569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f13f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 -- 172.21.15.12:0/1984891569 >> 172.21.15.12:0/1984891569 conn(0x5615a904a210 msgr2=0x5615a90f2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 -- 172.21.15.12:0/1984891569 shutdown_connections 2024-05-09T00:06:54.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 -- 172.21.15.12:0/1984891569 wait complete. 2024-05-09T00:06:54.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 -- start start 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.301+0000 7fa309c83b80 1 Processor -- start 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90ec890 0x5615a90fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa309c83b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f2720 0x5615a90f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa309c83b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a90fa4e0 con 0x5615a90f2720 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa309c83b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a90fa650 con 0x5615a90ec890 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa309c83b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a915c400 con 0x5615a90f1e00 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa309c83b80 10 client.0 _ll_register_callbacks cb 0x5615a8fe1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa2f6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa2f77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90ec890 0x5615a90fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa2f6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39216/0 (socket says 172.21.15.12:39216) 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa2f6ffd700 1 -- 172.21.15.12:0/439786373 learned_addr learned my addr 172.21.15.12:0/439786373 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa2f7fff700 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f2720 0x5615a90f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa2f6ffd700 1 -- 172.21.15.12:0/439786373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90ec890 msgr2=0x5615a90fd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90ec890 0x5615a90fd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa2f6ffd700 1 -- 172.21.15.12:0/439786373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f2720 msgr2=0x5615a90f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f2720 0x5615a90f9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.302+0000 7fa2f6ffd700 1 -- 172.21.15.12:0/439786373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec00e870 con 0x5615a90f1e00 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.303+0000 7fa2f6ffd700 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f9870 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x5615a90f70d0 tx=0x7fa2ec00e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b0424b22dfbb987 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.303+0000 7fa2d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.303+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/439786373 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec024070 con 0x5615a90f1e00 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.303+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/439786373 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa2ec029620 con 0x5615a90f1e00 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.303+0000 7fa309c83b80 1 -- 172.21.15.12:0/439786373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2e80a1a00 con 0x5615a90f1e00 2024-05-09T00:06:54.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.303+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/439786373 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2ec009100 con 0x5615a90f1e00 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.303+0000 7fa309c83b80 10 client.30838 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.303+0000 7fa309c83b80 1 -- 172.21.15.12:0/439786373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5615a8e69b60 con 0x5615a90f1e00 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.303+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/439786373 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e80a1a00 con 0x5615a90f1e00 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.304+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/439786373 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2ec005050 con 0x5615a90f1e00 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.304+0000 7fa309c83b80 10 client.30838 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.304+0000 7fa309c83b80 10 client.30838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.305+0000 7fa309c83b80 20 client.30838 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.305+0000 7fa309c83b80 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5615a915da70 0x5615a915fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.305+0000 7fa309c83b80 4 client.30838 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.305+0000 7fa309c83b80 1 -- 172.21.15.12:0/439786373 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5615a9034ec0 con 0x5615a915da70 2024-05-09T00:06:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.305+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5615a915da70 0x5615a915fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.305+0000 7fa2d6ffd700 10 client.30838 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.305+0000 7fa2f77fe700 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5615a915da70 0x5615a915fe90 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e8437265a76db2 server_cookie=d84a19bd709ad7a0 in_seq=0 out_seq=0 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa2d6ffd700 1 -- 172.21.15.12:0/439786373 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5615a9034ec0 con 0x5615a915da70 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa2d6ffd700 10 client.30838 handle_command_reply: tid=0 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa309c83b80 1 client.30838 shutdown 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa309c83b80 1 -- 172.21.15.12:0/439786373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 msgr2=0x5615a90f9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa309c83b80 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f9870 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x5615a90f70d0 tx=0x7fa2ec00e6b0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa309c83b80 1 -- 172.21.15.12:0/439786373 shutdown_connections 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa309c83b80 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615a90f2720 0x5615a90f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa309c83b80 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5615a915da70 0x5615a915fe90 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa309c83b80 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615a90ec890 0x5615a90fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa309c83b80 1 --2- 172.21.15.12:0/439786373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615a90f1e00 0x5615a90f9870 secure :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0x5615a90f70d0 tx=0x7fa2ec00e6b0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa2d6ffd700 0 client.30838 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.307+0000 7fa309c83b80 1 -- 172.21.15.12:0/439786373 >> 172.21.15.12:0/439786373 conn(0x5615a904a210 msgr2=0x5615a90f3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa309c83b80 1 -- 172.21.15.12:0/439786373 shutdown_connections 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa309c83b80 1 -- 172.21.15.12:0/439786373 wait complete. 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa309c83b80 20 client.30838 trim_cache size 0 max 16384 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (140 inodes in the stack)", 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "d27f7dc2-3fdd-453d-aa13-3f5455d2fe5d": { 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:06:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "d27f7dc2-3fdd-453d-aa13-3f5455d2fe5d", 2024-05-09T00:06:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:06:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:06:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:06:54.306 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2922452027 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2e00a1310 msgr2=0x7fa2e00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa2fdac9700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2e00a1310 0x7fa2e00a37d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8196890 tx=0x7fa2ec009260 comp rx=0 tx=0).stop 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2922452027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f8101180 msgr2=0x7fa2f8195390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa2fdac9700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f8101180 0x7fa2f8195390 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009c70 tx=0x7fa2e8009f80 comp rx=0 tx=0).stop 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2922452027 shutdown_connections 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa2fdac9700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2e00a1310 0x7fa2e00a37d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa2fdac9700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2f8199c20 0x7fa2f819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa2fdac9700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f8101180 0x7fa2f8195390 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa2fdac9700 1 --2- 172.21.15.12:0/2922452027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2f81062a0 0x7fa2f81958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.309+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2922452027 >> 172.21.15.12:0/2922452027 conn(0x7fa2f80fc9e0 msgr2=0x7fa2f80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.310+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2922452027 shutdown_connections 2024-05-09T00:06:54.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:06:54.310+0000 7fa2fdac9700 1 -- 172.21.15.12:0/2922452027 wait complete. 2024-05-09T00:06:54.322 INFO:tasks.cephfs.filesystem:scrub status for tag:d27f7dc2-3fdd-453d-aa13-3f5455d2fe5d - {'path': '/', 'tag': 'd27f7dc2-3fdd-453d-aa13-3f5455d2fe5d', 'options': 'recursive,force'} 2024-05-09T00:06:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.930+0000 7f2ad1b49700 1 -- 172.21.15.12:0/3237593517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2acc069360 msgr2=0x7f2acc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.930+0000 7f2ad1b49700 1 --2- 172.21.15.12:0/3237593517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2acc069360 0x7f2acc069740 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009990 tx=0x7f2abc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.930+0000 7f2ad1b49700 1 -- 172.21.15.12:0/3237593517 shutdown_connections 2024-05-09T00:06:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.930+0000 7f2ad1b49700 1 --2- 172.21.15.12:0/3237593517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2acc106260 0x7f2acc10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.930+0000 7f2ad1b49700 1 --2- 172.21.15.12:0/3237593517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2acc069360 0x7f2acc069740 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.930+0000 7f2ad1b49700 1 --2- 172.21.15.12:0/3237593517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2acc069d10 0x7f2acc105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.930+0000 7f2ad1b49700 1 -- 172.21.15.12:0/3237593517 >> 172.21.15.12:0/3237593517 conn(0x7f2acc076bc0 msgr2=0x7f2acc076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.930+0000 7f2ad1b49700 1 -- 172.21.15.12:0/3237593517 shutdown_connections 2024-05-09T00:06:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.930+0000 7f2ad1b49700 1 -- 172.21.15.12:0/3237593517 wait complete. 2024-05-09T00:06:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.931+0000 7f2ad1b49700 1 Processor -- start 2024-05-09T00:06:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.931+0000 7f2ad1b49700 1 -- start start 2024-05-09T00:06:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2ad1b49700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2acc069360 0x7f2acc197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2ad1b49700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2acc106260 0x7f2acc1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2ad1b49700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2acc19bc70 0x7f2acc19c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2ad1b49700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc19c6f0 con 0x7f2acc069360 2024-05-09T00:06:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2ad1b49700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc19c860 con 0x7f2acc106260 2024-05-09T00:06:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2ad1b49700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc19ca20 con 0x7f2acc19bc70 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2acb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2acc069360 0x7f2acc197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2acb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2acc069360 0x7f2acc197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59680/0 (socket says 172.21.15.12:59680) 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2acb7fe700 1 -- 172.21.15.12:0/462258645 learned_addr learned my addr 172.21.15.12:0/462258645 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2acbfff700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2acc19bc70 0x7f2acc19c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2acb7fe700 1 -- 172.21.15.12:0/462258645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2acc19bc70 msgr2=0x7f2acc19c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2acb7fe700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2acc19bc70 0x7f2acc19c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2acb7fe700 1 -- 172.21.15.12:0/462258645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2acc106260 msgr2=0x7f2acc1978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2acb7fe700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2acc106260 0x7f2acc1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2acb7fe700 1 -- 172.21.15.12:0/462258645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc0095f0 con 0x7f2acc069360 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.932+0000 7f2acb7fe700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2acc069360 0x7f2acc197390 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2abc005910 tx=0x7f2abc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.933+0000 7f2ac8ff9700 1 -- 172.21.15.12:0/462258645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc01d070 con 0x7f2acc069360 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.933+0000 7f2ad1b49700 1 -- 172.21.15.12:0/462258645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2acc1a3f20 con 0x7f2acc069360 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.933+0000 7f2ac8ff9700 1 -- 172.21.15.12:0/462258645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2abc022470 con 0x7f2acc069360 2024-05-09T00:06:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.933+0000 7f2ac8ff9700 1 -- 172.21.15.12:0/462258645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc017650 con 0x7f2acc069360 2024-05-09T00:06:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.933+0000 7f2ad1b49700 1 -- 172.21.15.12:0/462258645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc1a4340 con 0x7f2acc069360 2024-05-09T00:06:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.935+0000 7f2ac8ff9700 1 -- 172.21.15.12:0/462258645 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2abc0225e0 con 0x7f2acc069360 2024-05-09T00:06:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.936+0000 7f2ac8ff9700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ab40a1310 0x7f2ab40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.936+0000 7f2acaffd700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ab40a1310 0x7f2ab40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.936+0000 7f2ac8ff9700 1 -- 172.21.15.12:0/462258645 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2abc0ce060 con 0x7f2acc069360 2024-05-09T00:06:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2acaffd700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ab40a1310 0x7f2ab40a37d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0005950 tx=0x7f2ac00058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2addd03b80 1 Processor -- start 2024-05-09T00:06:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2addd03b80 1 -- start start 2024-05-09T00:06:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c010540 0x55a67c010920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 0x55a67c014900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c014e40 0x55a67c0152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2addd03b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67bf630f0 con 0x55a67c014e40 2024-05-09T00:06:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2addd03b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67bf60ab0 con 0x55a67c010540 2024-05-09T00:06:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2addd03b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67bf5c3a0 con 0x55a67c010e60 2024-05-09T00:06:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2acbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c014e40 0x55a67c0152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2acaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 0x55a67c014900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2acbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c014e40 0x55a67c0152a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59696/0 (socket says 172.21.15.12:59696) 2024-05-09T00:06:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.937+0000 7f2acbfff700 1 -- 172.21.15.12:0/1987537401 learned_addr learned my addr 172.21.15.12:0/1987537401 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.938+0000 7f2acb7fe700 1 --2- 172.21.15.12:0/1987537401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c010540 0x55a67c010920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.938+0000 7f2acaffd700 1 -- 172.21.15.12:0/1987537401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c010540 msgr2=0x55a67c010920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.938+0000 7f2acaffd700 1 --2- 172.21.15.12:0/1987537401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c010540 0x55a67c010920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.938+0000 7f2acaffd700 1 -- 172.21.15.12:0/1987537401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c014e40 msgr2=0x55a67c0152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.938+0000 7f2acaffd700 1 --2- 172.21.15.12:0/1987537401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c014e40 0x55a67c0152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.938+0000 7f2acaffd700 1 -- 172.21.15.12:0/1987537401 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a67bddd350 con 0x55a67c010e60 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.938+0000 7f2acaffd700 1 --2- 172.21.15.12:0/1987537401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 0x55a67c014900 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2acc1988e0 tx=0x7f2ac0011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96f516960d69afbf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.938+0000 7f2ab1ffb700 1 -- 172.21.15.12:0/1987537401 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0010890 con 0x55a67c010e60 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.938+0000 7f2ab1ffb700 1 -- 172.21.15.12:0/1987537401 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac0025460 con 0x55a67c010e60 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.938+0000 7f2ab1ffb700 1 -- 172.21.15.12:0/1987537401 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac001c680 con 0x55a67c010e60 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.939+0000 7f2addd03b80 1 -- 172.21.15.12:0/1987537401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 msgr2=0x55a67c014900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.939+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1987537401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 0x55a67c014900 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2acc1988e0 tx=0x7f2ac0011000 comp rx=0 tx=0).stop 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.939+0000 7f2addd03b80 1 -- 172.21.15.12:0/1987537401 shutdown_connections 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.939+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1987537401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c014e40 0x55a67c0152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.939+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1987537401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c010540 0x55a67c010920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.939+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1987537401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 0x55a67c014900 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.939+0000 7f2addd03b80 1 -- 172.21.15.12:0/1987537401 >> 172.21.15.12:0/1987537401 conn(0x55a67bf6a1b0 msgr2=0x55a67c006840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.941+0000 7f2addd03b80 1 -- 172.21.15.12:0/1987537401 shutdown_connections 2024-05-09T00:06:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.941+0000 7f2addd03b80 1 -- 172.21.15.12:0/1987537401 wait complete. 2024-05-09T00:06:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.942+0000 7f2addd03b80 1 -- start start 2024-05-09T00:06:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.942+0000 7f2addd03b80 1 Processor -- start 2024-05-09T00:06:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.942+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c010e60 0x55a67c019b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.942+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 0x55a67c01a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.942+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c01a5b0 0x55a67c0260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.942+0000 7f2addd03b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67bef8500 con 0x55a67c010e60 2024-05-09T00:06:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.942+0000 7f2addd03b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67bef81a0 con 0x55a67c014e40 2024-05-09T00:06:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.942+0000 7f2addd03b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67bddf640 con 0x55a67c01a5b0 2024-05-09T00:06:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.942+0000 7f2addd03b80 10 client.0 _ll_register_callbacks cb 0x55a67bf01ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2acaffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 0x55a67c01a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2acaffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 0x55a67c01a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48900/0 (socket says 172.21.15.12:48900) 2024-05-09T00:06:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2acaffd700 1 -- 172.21.15.12:0/1982565762 learned_addr learned my addr 172.21.15.12:0/1982565762 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2acbfff700 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c01a5b0 0x55a67c0260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2acbfff700 1 -- 172.21.15.12:0/1982565762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 msgr2=0x55a67c01a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2acbfff700 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 0x55a67c01a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2acbfff700 1 -- 172.21.15.12:0/1982565762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c010e60 msgr2=0x55a67c019b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2acbfff700 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c010e60 0x55a67c019b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2acbfff700 1 -- 172.21.15.12:0/1982565762 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac000e7f0 con 0x55a67c01a5b0 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2acbfff700 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c01a5b0 0x55a67c0260c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4004710 tx=0x7f2ac4004a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dc6c4b4521ea077 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.943+0000 7f2aaaffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2aaaffd700 1 -- 172.21.15.12:0/1982565762 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400e040 con 0x55a67c01a5b0 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2aaaffd700 1 -- 172.21.15.12:0/1982565762 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2ac401a070 con 0x55a67c01a5b0 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2addd03b80 1 -- 172.21.15.12:0/1982565762 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a67beb5210 con 0x55a67c01a5b0 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2aaaffd700 1 -- 172.21.15.12:0/1982565762 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac4014900 con 0x55a67c01a5b0 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2addd03b80 10 client.30850 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2addd03b80 1 -- 172.21.15.12:0/1982565762 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55a67bd89b00 con 0x55a67c01a5b0 2024-05-09T00:06:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2aaaffd700 1 -- 172.21.15.12:0/1982565762 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a67beb5210 con 0x55a67c01a5b0 2024-05-09T00:06:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2aaaffd700 1 -- 172.21.15.12:0/1982565762 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2ac4025070 con 0x55a67c01a5b0 2024-05-09T00:06:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2addd03b80 10 client.30850 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2addd03b80 10 client.30850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2addd03b80 20 client.30850 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a67c00fee0 0x55a67c030e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2addd03b80 4 client.30850 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:06:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.944+0000 7f2addd03b80 1 -- 172.21.15.12:0/1982565762 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a67bfdb7b0 con 0x55a67c00fee0 2024-05-09T00:06:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.945+0000 7f2acb7fe700 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a67c00fee0 0x55a67c030e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.945+0000 7f2acb7fe700 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a67c00fee0 0x55a67c030e20 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51d78ba6f4fe16 server_cookie=9980fe63e1791de8 in_seq=0 out_seq=0 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.945+0000 7f2aaaffd700 10 client.30850 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2aaaffd700 1 -- 172.21.15.12:0/1982565762 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a67bfdb7b0 con 0x55a67c00fee0 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2aaaffd700 10 client.30850 handle_command_reply: tid=0 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 client.30850 shutdown 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 -- 172.21.15.12:0/1982565762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c01a5b0 msgr2=0x55a67c0260c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c01a5b0 0x55a67c0260c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4004710 tx=0x7f2ac4004a20 comp rx=0 tx=0).stop 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 -- 172.21.15.12:0/1982565762 shutdown_connections 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c01a5b0 0x55a67c0260c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c010e60 0x55a67c019b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a67c00fee0 0x55a67c030e20 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1982565762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 0x55a67c01a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2aaaffd700 0 client.30850 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 -- 172.21.15.12:0/1982565762 >> 172.21.15.12:0/1982565762 conn(0x55a67bf6a1b0 msgr2=0x55a67c00af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 -- 172.21.15.12:0/1982565762 shutdown_connections 2024-05-09T00:06:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 1 -- 172.21.15.12:0/1982565762 wait complete. 2024-05-09T00:06:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.947+0000 7f2addd03b80 20 client.30850 trim_cache size 0 max 16384 2024-05-09T00:06:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2addd03b80 1 Processor -- start 2024-05-09T00:06:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2addd03b80 1 -- start start 2024-05-09T00:06:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c010e60 0x55a67c012800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c014e40 0x55a67c01a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c006910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2addd03b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c035f90 con 0x55a67c01a5b0 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2addd03b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c018060 con 0x55a67c010e60 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2addd03b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c013bd0 con 0x55a67c014e40 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2acbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c006910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2acbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c006910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59712/0 (socket says 172.21.15.12:59712) 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2acbfff700 1 -- 172.21.15.12:0/1793620511 learned_addr learned my addr 172.21.15.12:0/1793620511 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2acaffd700 1 --2- 172.21.15.12:0/1793620511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c014e40 0x55a67c01a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.966+0000 7f2acb7fe700 1 --2- 172.21.15.12:0/1793620511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c010e60 0x55a67c012800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.967+0000 7f2acbfff700 1 -- 172.21.15.12:0/1793620511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c014e40 msgr2=0x55a67c01a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.967+0000 7f2acbfff700 1 --2- 172.21.15.12:0/1793620511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c014e40 0x55a67c01a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.967+0000 7f2acbfff700 1 -- 172.21.15.12:0/1793620511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c010e60 msgr2=0x55a67c012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.967+0000 7f2acbfff700 1 --2- 172.21.15.12:0/1793620511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c010e60 0x55a67c012800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.967+0000 7f2acbfff700 1 -- 172.21.15.12:0/1793620511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a67bf22500 con 0x55a67c01a5b0 2024-05-09T00:06:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.967+0000 7f2acaffd700 1 --2- 172.21.15.12:0/1793620511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c014e40 0x55a67c01a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:06:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.967+0000 7f2acbfff700 1 --2- 172.21.15.12:0/1793620511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c006910 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40046e0 tx=0x7f2ac4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b70ab69f6a422 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.967+0000 7f2ab27fc700 1 -- 172.21.15.12:0/1793620511 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400e070 con 0x55a67c01a5b0 2024-05-09T00:06:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.967+0000 7f2ab27fc700 1 -- 172.21.15.12:0/1793620511 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac400f040 con 0x55a67c01a5b0 2024-05-09T00:06:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.967+0000 7f2ab27fc700 1 -- 172.21.15.12:0/1793620511 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4007760 con 0x55a67c01a5b0 2024-05-09T00:06:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.968+0000 7f2addd03b80 1 -- 172.21.15.12:0/1793620511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 msgr2=0x55a67c006910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.968+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1793620511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c006910 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40046e0 tx=0x7f2ac4007ef0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.968+0000 7f2addd03b80 1 -- 172.21.15.12:0/1793620511 shutdown_connections 2024-05-09T00:06:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.968+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1793620511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c006910 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.968+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1793620511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c010e60 0x55a67c012800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.968+0000 7f2addd03b80 1 --2- 172.21.15.12:0/1793620511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c014e40 0x55a67c01a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.968+0000 7f2addd03b80 1 -- 172.21.15.12:0/1793620511 >> 172.21.15.12:0/1793620511 conn(0x55a67bf6a1b0 msgr2=0x55a67c025ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.968+0000 7f2addd03b80 1 -- 172.21.15.12:0/1793620511 shutdown_connections 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.968+0000 7f2addd03b80 1 -- 172.21.15.12:0/1793620511 wait complete. 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.969+0000 7f2addd03b80 1 -- start start 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.969+0000 7f2addd03b80 1 Processor -- start 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 0x55a67c021290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 0x55a67c011b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2addd03b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c0120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2addd03b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c0126f0 con 0x55a67c01a5b0 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2addd03b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c012860 con 0x55a67c014e40 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2addd03b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c012a20 con 0x55a67c010e60 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2addd03b80 10 client.0 _ll_register_callbacks cb 0x55a67bf01ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2acb7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 0x55a67c021290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2acb7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 0x55a67c021290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39254/0 (socket says 172.21.15.12:39254) 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2acb7fe700 1 -- 172.21.15.12:0/3813276638 learned_addr learned my addr 172.21.15.12:0/3813276638 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2acbfff700 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c0120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.970+0000 7f2acaffd700 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 0x55a67c011b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2acbfff700 1 -- 172.21.15.12:0/3813276638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 msgr2=0x55a67c021290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2acbfff700 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 0x55a67c021290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2acbfff700 1 -- 172.21.15.12:0/3813276638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 msgr2=0x55a67c011b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2acbfff700 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 0x55a67c011b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2acbfff700 1 -- 172.21.15.12:0/3813276638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a67bddd350 con 0x55a67c01a5b0 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2acbfff700 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c0120d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4004010 tx=0x7f2ac4009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfdbf8d24603ed3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2aaaffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2aaaffd700 1 -- 172.21.15.12:0/3813276638 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400e040 con 0x55a67c01a5b0 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2aaaffd700 1 -- 172.21.15.12:0/3813276638 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2ac401a030 con 0x55a67c01a5b0 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2addd03b80 1 -- 172.21.15.12:0/3813276638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ac0025460 con 0x55a67c01a5b0 2024-05-09T00:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2aaaffd700 1 -- 172.21.15.12:0/3813276638 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac4004e30 con 0x55a67c01a5b0 2024-05-09T00:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2addd03b80 10 client.21138 fetch_fsmap learned FSMap version 76 2024-05-09T00:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.971+0000 7f2addd03b80 1 -- 172.21.15.12:0/3813276638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55a67bd89b00 con 0x55a67c01a5b0 2024-05-09T00:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.972+0000 7f2aaaffd700 1 -- 172.21.15.12:0/3813276638 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac403e080 con 0x55a67c01a5b0 2024-05-09T00:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.972+0000 7f2aaaffd700 1 -- 172.21.15.12:0/3813276638 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2ac4025040 con 0x55a67c01a5b0 2024-05-09T00:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.972+0000 7f2addd03b80 10 client.21138 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.972+0000 7f2addd03b80 10 client.21138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.972+0000 7f2addd03b80 20 client.21138 populate_metadata read hostname 'smithi012' 2024-05-09T00:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.972+0000 7f2addd03b80 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a67c085170 0x55a67c0875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.972+0000 7f2addd03b80 4 client.21138 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:06:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.972+0000 7f2addd03b80 1 -- 172.21.15.12:0/3813276638 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a67bf54e60 con 0x55a67c085170 2024-05-09T00:06:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.973+0000 7f2acb7fe700 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a67c085170 0x55a67c0875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.974+0000 7f2acb7fe700 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a67c085170 0x55a67c0875b0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb307005e591bf6 server_cookie=e93eb438025c8193 in_seq=0 out_seq=0 2024-05-09T00:06:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:54.974+0000 7f2aaaffd700 10 client.21138 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:55.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:54 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:06:55.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:55.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:06:55.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:06:55.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:06:55.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:55.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:55.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:54 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:06:55.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:55.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:06:55.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:06:55.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:06:55.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:55.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:55.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:54 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:06:55.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:55.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:06:55.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:06:55.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:06:55.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:55.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:55.270 INFO:tasks.workunit.client.0.smithi012.stdout: 10 262401 32.60 MB/sec execute 290 sec latency 261.772 ms 2024-05-09T00:06:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2aaaffd700 1 -- 172.21.15.12:0/3813276638 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a67bf54e60 con 0x55a67c085170 2024-05-09T00:06:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2aaaffd700 10 client.21138 handle_command_reply: tid=0 2024-05-09T00:06:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 client.21138 shutdown 2024-05-09T00:06:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 -- 172.21.15.12:0/3813276638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 msgr2=0x55a67c0120d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c0120d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4004010 tx=0x7f2ac4009510 comp rx=0 tx=0).stop 2024-05-09T00:06:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 -- 172.21.15.12:0/3813276638 shutdown_connections 2024-05-09T00:06:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a67c01a5b0 0x55a67c0120d0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a67c010e60 0x55a67c021290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a67c085170 0x55a67c0875b0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 --2- 172.21.15.12:0/3813276638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a67c014e40 0x55a67c011b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2aaaffd700 0 client.21138 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:06:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 -- 172.21.15.12:0/3813276638 >> 172.21.15.12:0/3813276638 conn(0x55a67bf6a1b0 msgr2=0x55a67c013040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 -- 172.21.15.12:0/3813276638 shutdown_connections 2024-05-09T00:06:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.444+0000 7f2addd03b80 1 -- 172.21.15.12:0/3813276638 wait complete. 2024-05-09T00:06:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2addd03b80 20 client.21138 trim_cache size 0 max 16384 2024-05-09T00:06:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:06:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:06:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:06:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:06:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2ad1b49700 1 -- 172.21.15.12:0/462258645 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ab40a1310 msgr2=0x7f2ab40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2ad1b49700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ab40a1310 0x7f2ab40a37d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0005950 tx=0x7f2ac00058e0 comp rx=0 tx=0).stop 2024-05-09T00:06:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2ad1b49700 1 -- 172.21.15.12:0/462258645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2acc069360 msgr2=0x7f2acc197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2ad1b49700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2acc069360 0x7f2acc197390 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2abc005910 tx=0x7f2abc004730 comp rx=0 tx=0).stop 2024-05-09T00:06:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2ad1b49700 1 -- 172.21.15.12:0/462258645 shutdown_connections 2024-05-09T00:06:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2ad1b49700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2ab40a1310 0x7f2ab40a37d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2ad1b49700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2acc19bc70 0x7f2acc19c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2ad1b49700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2acc069360 0x7f2acc197390 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2ad1b49700 1 --2- 172.21.15.12:0/462258645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2acc106260 0x7f2acc1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.445+0000 7f2ad1b49700 1 -- 172.21.15.12:0/462258645 >> 172.21.15.12:0/462258645 conn(0x7f2acc076bc0 msgr2=0x7f2acc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.446+0000 7f2ad1b49700 1 -- 172.21.15.12:0/462258645 shutdown_connections 2024-05-09T00:06:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:06:55.446+0000 7f2ad1b49700 1 -- 172.21.15.12:0/462258645 wait complete. 2024-05-09T00:06:56.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:55 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:56.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:55 smithi012 ceph-mon[42503]: pgmap v659: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 10 MiB/s wr, 539 op/s 2024-05-09T00:06:56.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:55 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:56.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:55 smithi204 ceph-mon[47373]: pgmap v659: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 10 MiB/s wr, 539 op/s 2024-05-09T00:06:56.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:55 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:06:56.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:55 smithi160 ceph-mon[46152]: pgmap v659: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 10 MiB/s wr, 539 op/s 2024-05-09T00:06:56.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 262924 32.60 MB/sec execute 291 sec latency 304.987 ms 2024-05-09T00:06:57.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:56 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:57.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:56 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:57.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:56 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:57.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 263432 32.55 MB/sec execute 292 sec latency 114.741 ms 2024-05-09T00:06:58.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:57 smithi012 ceph-mon[42503]: pgmap v660: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 820 op/s 2024-05-09T00:06:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:57 smithi204 ceph-mon[47373]: pgmap v660: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 820 op/s 2024-05-09T00:06:58.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:57 smithi160 ceph-mon[46152]: pgmap v660: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 820 op/s 2024-05-09T00:06:58.269 INFO:tasks.workunit.client.0.smithi012.stdout: 10 263870 32.51 MB/sec execute 293 sec latency 368.202 ms 2024-05-09T00:06:59.270 INFO:tasks.workunit.client.0.smithi012.stdout: 10 264506 32.55 MB/sec execute 294 sec latency 133.641 ms 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:06:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:06:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:06:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:59.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:06:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.270 INFO:tasks.workunit.client.0.smithi012.stdout: 10 265121 32.60 MB/sec execute 295 sec latency 209.283 ms 2024-05-09T00:07:00.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:00 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi204 to 182.4M 2024-05-09T00:07:00.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:00 smithi160 ceph-mon[46152]: Unable to set osd_memory_target on smithi204 to 191345459: error parsing value: Value '191345459' is below minimum 939524096 2024-05-09T00:07:00.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:00 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi012 to 183.5M 2024-05-09T00:07:00.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:00 smithi160 ceph-mon[46152]: Unable to set osd_memory_target on smithi012 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-09T00:07:00.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:00 smithi160 ceph-mon[46152]: pgmap v661: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 689 op/s 2024-05-09T00:07:00.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:00 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi204 to 182.4M 2024-05-09T00:07:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:00 smithi012 ceph-mon[42503]: Unable to set osd_memory_target on smithi204 to 191345459: error parsing value: Value '191345459' is below minimum 939524096 2024-05-09T00:07:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:00 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi012 to 183.5M 2024-05-09T00:07:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:00 smithi012 ceph-mon[42503]: Unable to set osd_memory_target on smithi012 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-09T00:07:00.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:00 smithi012 ceph-mon[42503]: pgmap v661: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 689 op/s 2024-05-09T00:07:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:00 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi204 to 182.4M 2024-05-09T00:07:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:00 smithi204 ceph-mon[47373]: Unable to set osd_memory_target on smithi204 to 191345459: error parsing value: Value '191345459' is below minimum 939524096 2024-05-09T00:07:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:00 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi012 to 183.5M 2024-05-09T00:07:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:00 smithi204 ceph-mon[47373]: Unable to set osd_memory_target on smithi012 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-09T00:07:00.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:00 smithi204 ceph-mon[47373]: pgmap v661: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 689 op/s 2024-05-09T00:07:01.270 INFO:tasks.workunit.client.0.smithi012.stdout: 10 265147 32.49 MB/sec execute 296 sec latency 1135.721 ms 2024-05-09T00:07:02.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:07:02.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:07:02.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-09T00:07:02.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: Unable to set osd_memory_target on smithi160 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-09T00:07:02.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:02.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:02.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:07:02.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:01 smithi204 ceph-mon[47373]: pgmap v662: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 684 op/s 2024-05-09T00:07:02.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:07:02.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:07:02.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-09T00:07:02.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: Unable to set osd_memory_target on smithi160 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-09T00:07:02.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:02.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:02.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:07:02.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:01 smithi160 ceph-mon[46152]: pgmap v662: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 684 op/s 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: Adjusting osd_memory_target on smithi160 to 182.5M 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: Unable to set osd_memory_target on smithi160 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:07:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:01 smithi012 ceph-mon[42503]: pgmap v662: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 684 op/s 2024-05-09T00:07:02.271 INFO:tasks.workunit.client.0.smithi012.stdout: 10 265481 32.40 MB/sec execute 297 sec latency 130.706 ms 2024-05-09T00:07:03.271 INFO:tasks.workunit.client.0.smithi012.stdout: 10 265681 32.32 MB/sec execute 298 sec latency 372.110 ms 2024-05-09T00:07:04.271 INFO:tasks.workunit.client.0.smithi012.stdout: 10 265824 32.24 MB/sec execute 299 sec latency 583.438 ms 2024-05-09T00:07:04.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:07:04.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:07:04.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:07:04.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:04 smithi012 ceph-mon[42503]: osdmap e126: 12 total, 12 up, 12 in 2024-05-09T00:07:04.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:04 smithi012 ceph-mon[42503]: pgmap v664: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 778 op/s 2024-05-09T00:07:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:07:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:07:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:07:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:04 smithi204 ceph-mon[47373]: osdmap e126: 12 total, 12 up, 12 in 2024-05-09T00:07:05.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:04 smithi204 ceph-mon[47373]: pgmap v664: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 778 op/s 2024-05-09T00:07:05.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:07:05.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:07:05.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:07:05.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:04 smithi160 ceph-mon[46152]: osdmap e126: 12 total, 12 up, 12 in 2024-05-09T00:07:05.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:04 smithi160 ceph-mon[46152]: pgmap v664: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 778 op/s 2024-05-09T00:07:05.271 INFO:tasks.workunit.client.0.smithi012.stdout: 10 265995 32.17 MB/sec execute 300 sec latency 556.210 ms 2024-05-09T00:07:06.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:05 smithi012 ceph-mon[42503]: pgmap v665: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 777 op/s 2024-05-09T00:07:06.262 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:05 smithi204 ceph-mon[47373]: pgmap v665: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 777 op/s 2024-05-09T00:07:06.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:05 smithi160 ceph-mon[46152]: pgmap v665: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 777 op/s 2024-05-09T00:07:06.271 INFO:tasks.workunit.client.0.smithi012.stdout: 10 266177 32.08 MB/sec execute 301 sec latency 386.800 ms 2024-05-09T00:07:07.273 INFO:tasks.workunit.client.0.smithi012.stdout: 10 266423 32.00 MB/sec execute 302 sec latency 521.459 ms 2024-05-09T00:07:08.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:07 smithi204 ceph-mon[47373]: pgmap v666: 97 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 63 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.4 MiB/s wr, 608 op/s 2024-05-09T00:07:08.018 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:07 smithi160 ceph-mon[46152]: pgmap v666: 97 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 63 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.4 MiB/s wr, 608 op/s 2024-05-09T00:07:08.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:07 smithi012 ceph-mon[42503]: pgmap v666: 97 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 63 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.4 MiB/s wr, 608 op/s 2024-05-09T00:07:08.273 INFO:tasks.workunit.client.0.smithi012.stdout: 10 266650 31.93 MB/sec execute 303 sec latency 467.035 ms 2024-05-09T00:07:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.628+0000 7f73ad63b700 1 -- 172.21.15.12:0/3134531855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a8107fa0 msgr2=0x7f73a8108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.628+0000 7f73ad63b700 1 --2- 172.21.15.12:0/3134531855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a8107fa0 0x7f73a8108450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b0a0 tx=0x7f73a000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.636+0000 7f73ad63b700 1 -- 172.21.15.12:0/3134531855 shutdown_connections 2024-05-09T00:07:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.636+0000 7f73ad63b700 1 --2- 172.21.15.12:0/3134531855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a8107fa0 0x7f73a8108450 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.636+0000 7f73ad63b700 1 --2- 172.21.15.12:0/3134531855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73a810f630 0x7f73a810fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.636+0000 7f73ad63b700 1 --2- 172.21.15.12:0/3134531855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73a810d440 0x7f73a810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.636+0000 7f73ad63b700 1 -- 172.21.15.12:0/3134531855 >> 172.21.15.12:0/3134531855 conn(0x7f73a806bf80 msgr2=0x7f73a806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.645+0000 7f73ad63b700 1 -- 172.21.15.12:0/3134531855 shutdown_connections 2024-05-09T00:07:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.646+0000 7f73ad63b700 1 -- 172.21.15.12:0/3134531855 wait complete. 2024-05-09T00:07:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.646+0000 7f73ad63b700 1 Processor -- start 2024-05-09T00:07:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.647+0000 7f73ad63b700 1 -- start start 2024-05-09T00:07:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.647+0000 7f73ad63b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73a8107fa0 0x7f73a81a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.647+0000 7f73ad63b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73a810d440 0x7f73a81a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.647+0000 7f73ad63b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a810f630 0x7f73a81aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.647+0000 7f73ad63b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a81ab2a0 con 0x7f73a810d440 2024-05-09T00:07:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.647+0000 7f73ad63b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a81ab410 con 0x7f73a8107fa0 2024-05-09T00:07:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.647+0000 7f73ad63b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a81ab5d0 con 0x7f73a810f630 2024-05-09T00:07:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.650+0000 7f73a6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73a8107fa0 0x7f73a81a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.650+0000 7f73a6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73a8107fa0 0x7f73a81a6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:37272/0 (socket says 172.21.15.12:37272) 2024-05-09T00:07:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.650+0000 7f73a6ffd700 1 -- 172.21.15.12:0/1442715928 learned_addr learned my addr 172.21.15.12:0/1442715928 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.652+0000 7f73a77fe700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a810f630 0x7f73a81aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.654+0000 7f73a6ffd700 1 -- 172.21.15.12:0/1442715928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a810f630 msgr2=0x7f73a81aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.654+0000 7f73a6ffd700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a810f630 0x7f73a81aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.654+0000 7f73a6ffd700 1 -- 172.21.15.12:0/1442715928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73a810d440 msgr2=0x7f73a81a68c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:07:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.654+0000 7f73a6ffd700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73a810d440 0x7f73a81a68c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.654+0000 7f73a6ffd700 1 -- 172.21.15.12:0/1442715928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a0009d00 con 0x7f73a8107fa0 2024-05-09T00:07:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.655+0000 7f73a6ffd700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73a8107fa0 0x7f73a81a6380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f739800b500 tx=0x7f739800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.655+0000 7f738ffff700 1 -- 172.21.15.12:0/1442715928 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398011830 con 0x7f73a8107fa0 2024-05-09T00:07:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.655+0000 7f738ffff700 1 -- 172.21.15.12:0/1442715928 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7398011e50 con 0x7f73a8107fa0 2024-05-09T00:07:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.655+0000 7f738ffff700 1 -- 172.21.15.12:0/1442715928 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739800f2e0 con 0x7f73a8107fa0 2024-05-09T00:07:08.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.660+0000 7f73ad63b700 1 -- 172.21.15.12:0/1442715928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a81ab8b0 con 0x7f73a8107fa0 2024-05-09T00:07:08.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.660+0000 7f73ad63b700 1 -- 172.21.15.12:0/1442715928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a8110140 con 0x7f73a8107fa0 2024-05-09T00:07:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.665+0000 7f73b97f5b80 1 Processor -- start 2024-05-09T00:07:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.665+0000 7f73b97f5b80 1 -- start start 2024-05-09T00:07:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.665+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186440d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.665+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559186441270 0x55918643af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.665+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55918643b480 0x55918643b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.665+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591863920f0 con 0x559186440950 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.665+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918638fab0 con 0x55918643b480 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.665+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918638b3a0 con 0x559186441270 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.666+0000 7f73a77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186440d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.666+0000 7f73a77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186440d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53382/0 (socket says 172.21.15.12:53382) 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.666+0000 7f73a77fe700 1 -- 172.21.15.12:0/4243964259 learned_addr learned my addr 172.21.15.12:0/4243964259 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.666+0000 7f73a77fe700 1 -- 172.21.15.12:0/4243964259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559186441270 msgr2=0x55918643af40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.666+0000 7f73a77fe700 1 --2- 172.21.15.12:0/4243964259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559186441270 0x55918643af40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.666+0000 7f73a77fe700 1 -- 172.21.15.12:0/4243964259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55918643b480 msgr2=0x55918643b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.666+0000 7f73a77fe700 1 --2- 172.21.15.12:0/4243964259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55918643b480 0x55918643b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.666+0000 7f73a77fe700 1 -- 172.21.15.12:0/4243964259 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55918620c350 con 0x559186440950 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.667+0000 7f73a77fe700 1 --2- 172.21.15.12:0/4243964259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186440d30 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b830 tx=0x7f73a0008d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2075f843c063433 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.667+0000 7f738d7fa700 1 -- 172.21.15.12:0/4243964259 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000e070 con 0x559186440950 2024-05-09T00:07:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.667+0000 7f738d7fa700 1 -- 172.21.15.12:0/4243964259 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a00093b0 con 0x559186440950 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.667+0000 7f738d7fa700 1 -- 172.21.15.12:0/4243964259 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a00052a0 con 0x559186440950 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.668+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4243964259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 msgr2=0x559186440d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.668+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/4243964259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186440d30 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b830 tx=0x7f73a0008d30 comp rx=0 tx=0).stop 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.668+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4243964259 shutdown_connections 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.668+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/4243964259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55918643b480 0x55918643b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.668+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/4243964259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186440d30 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.668+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/4243964259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559186441270 0x55918643af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.668+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4243964259 >> 172.21.15.12:0/4243964259 conn(0x5591863991b0 msgr2=0x559186435840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.670+0000 7f738ffff700 1 -- 172.21.15.12:0/1442715928 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f73980105f0 con 0x7f73a8107fa0 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.670+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4243964259 shutdown_connections 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.670+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4243964259 wait complete. 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.670+0000 7f73b97f5b80 1 -- start start 2024-05-09T00:07:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.671+0000 7f73b97f5b80 1 Processor -- start 2024-05-09T00:07:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.671+0000 7f738ffff700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f73900a13b0 0x7f73900a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.671+0000 7f738ffff700 1 -- 172.21.15.12:0/1442715928 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f73980cc570 con 0x7f73a8107fa0 2024-05-09T00:07:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55918643b480 0x5591864472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186444600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 0x559186444cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559186327500 con 0x559186440950 2024-05-09T00:07:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591863271a0 con 0x559186441270 2024-05-09T00:07:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918620e640 con 0x55918643b480 2024-05-09T00:07:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73b97f5b80 10 client.0 _ll_register_callbacks cb 0x559186330ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73a6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55918643b480 0x5591864472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73a6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55918643b480 0x5591864472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48888/0 (socket says 172.21.15.12:48888) 2024-05-09T00:07:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73a6ffd700 1 -- 172.21.15.12:0/4282601875 learned_addr learned my addr 172.21.15.12:0/4282601875 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.672+0000 7f73a67fc700 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186444600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.673+0000 7f73a67fc700 1 -- 172.21.15.12:0/4282601875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55918643b480 msgr2=0x5591864472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.673+0000 7f73a67fc700 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55918643b480 0x5591864472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.673+0000 7f73a67fc700 1 -- 172.21.15.12:0/4282601875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 msgr2=0x559186444cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.673+0000 7f73a67fc700 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 0x559186444cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.673+0000 7f73a67fc700 1 -- 172.21.15.12:0/4282601875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a810b0e0 con 0x559186440950 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.673+0000 7f73a67fc700 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186444600 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f73a81a78d0 tx=0x7f739c00b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28553f81485961e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.673+0000 7f738aff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.673+0000 7f738aff5700 1 -- 172.21.15.12:0/4282601875 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00be60 con 0x559186440950 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.673+0000 7f738aff5700 1 -- 172.21.15.12:0/4282601875 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f739c011420 con 0x559186440950 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.673+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4282601875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591862e4210 con 0x559186440950 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.674+0000 7f738aff5700 1 -- 172.21.15.12:0/4282601875 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f739c0115e0 con 0x559186440950 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.674+0000 7f73b97f5b80 10 client.21144 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.674+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4282601875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5591861b8b00 con 0x559186440950 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.674+0000 7f738aff5700 1 -- 172.21.15.12:0/4282601875 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5591862e4210 con 0x559186440950 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.674+0000 7f738aff5700 1 -- 172.21.15.12:0/4282601875 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f739c019860 con 0x559186440950 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.675+0000 7f73b97f5b80 10 client.21144 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.675+0000 7f73b97f5b80 10 client.21144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.675+0000 7f73b97f5b80 20 client.21144 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.675+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559186454c90 0x5591864570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.675+0000 7f73b97f5b80 4 client.21144 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.675+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4282601875 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55918640a7b0 con 0x559186454c90 2024-05-09T00:07:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.675+0000 7f73a6ffd700 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559186454c90 0x5591864570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.676+0000 7f73a6ffd700 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559186454c90 0x5591864570b0 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6139137f4f7b4bae server_cookie=ef7453e73e933d87 in_seq=0 out_seq=0 2024-05-09T00:07:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.676+0000 7f738aff5700 10 client.21144 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.677+0000 7f73a77fe700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f73900a13b0 0x7f73900a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.678+0000 7f738aff5700 1 -- 172.21.15.12:0/4282601875 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55918640a7b0 con 0x559186454c90 2024-05-09T00:07:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.678+0000 7f738aff5700 10 client.21144 handle_command_reply: tid=0 2024-05-09T00:07:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.678+0000 7f73b97f5b80 1 client.21144 shutdown 2024-05-09T00:07:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.680+0000 7f73a77fe700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f73900a13b0 0x7f73900a3870 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f73a0007a30 tx=0x7f73a00099e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.681+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4282601875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 msgr2=0x559186444600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.681+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186444600 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f73a81a78d0 tx=0x7f739c00b3c0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.687+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4282601875 shutdown_connections 2024-05-09T00:07:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.687+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559186454c90 0x5591864570b0 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.687+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 0x559186444cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.687+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55918643b480 0x5591864472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.687+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/4282601875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186440950 0x559186444600 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.687+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4282601875 >> 172.21.15.12:0/4282601875 conn(0x5591863991b0 msgr2=0x55918643f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.690+0000 7f738aff5700 0 client.21144 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.694+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4282601875 shutdown_connections 2024-05-09T00:07:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.694+0000 7f73b97f5b80 1 -- 172.21.15.12:0/4282601875 wait complete. 2024-05-09T00:07:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.694+0000 7f73b97f5b80 20 client.21144 trim_cache size 0 max 16384 2024-05-09T00:07:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.722+0000 7f73b97f5b80 1 Processor -- start 2024-05-09T00:07:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.722+0000 7f73b97f5b80 1 -- start start 2024-05-09T00:07:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.722+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55918643b480 0x559186435910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.722+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559186440950 0x559186446df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.722+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 0x559186443eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918642da40 con 0x55918643b480 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559186445bb0 con 0x559186441270 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559186436540 con 0x559186440950 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73a77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 0x559186443eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73a6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55918643b480 0x559186435910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73a77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 0x559186443eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:37296/0 (socket says 172.21.15.12:37296) 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73a77fe700 1 -- 172.21.15.12:0/3323415292 learned_addr learned my addr 172.21.15.12:0/3323415292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73a77fe700 1 -- 172.21.15.12:0/3323415292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559186440950 msgr2=0x559186446df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73a77fe700 1 --2- 172.21.15.12:0/3323415292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559186440950 0x559186446df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73a77fe700 1 -- 172.21.15.12:0/3323415292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55918643b480 msgr2=0x559186435910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73a77fe700 1 --2- 172.21.15.12:0/3323415292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55918643b480 0x559186435910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.723+0000 7f73a77fe700 1 -- 172.21.15.12:0/3323415292 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a8067eb0 con 0x559186441270 2024-05-09T00:07:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.724+0000 7f73a77fe700 1 --2- 172.21.15.12:0/3323415292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 0x559186443eb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b830 tx=0x7f73a0025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a758bab8c7351bad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.724+0000 7f738dffb700 1 -- 172.21.15.12:0/3323415292 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0004c50 con 0x559186441270 2024-05-09T00:07:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.724+0000 7f738dffb700 1 -- 172.21.15.12:0/3323415292 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a0009200 con 0x559186441270 2024-05-09T00:07:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.724+0000 7f738dffb700 1 -- 172.21.15.12:0/3323415292 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0024450 con 0x559186441270 2024-05-09T00:07:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.725+0000 7f73b97f5b80 1 -- 172.21.15.12:0/3323415292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 msgr2=0x559186443eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.725+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/3323415292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 0x559186443eb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b830 tx=0x7f73a0025000 comp rx=0 tx=0).stop 2024-05-09T00:07:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.725+0000 7f73b97f5b80 1 -- 172.21.15.12:0/3323415292 shutdown_connections 2024-05-09T00:07:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.725+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/3323415292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559186441270 0x559186443eb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.725+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/3323415292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55918643b480 0x559186435910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.725+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/3323415292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559186440950 0x559186446df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.725+0000 7f73b97f5b80 1 -- 172.21.15.12:0/3323415292 >> 172.21.15.12:0/3323415292 conn(0x5591863991b0 msgr2=0x5591864506e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.737+0000 7f73b97f5b80 1 -- 172.21.15.12:0/3323415292 shutdown_connections 2024-05-09T00:07:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.737+0000 7f73b97f5b80 1 -- 172.21.15.12:0/3323415292 wait complete. 2024-05-09T00:07:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.737+0000 7f73b97f5b80 1 -- start start 2024-05-09T00:07:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.737+0000 7f73b97f5b80 1 Processor -- start 2024-05-09T00:07:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.738+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55918643b480 0x559186447df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.738+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186441270 0x559186448330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.738+0000 7f73b97f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591864a8170 0x5591864aa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.738+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591864aaaa0 con 0x559186441270 2024-05-09T00:07:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.738+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591864aabe0 con 0x55918643b480 2024-05-09T00:07:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.738+0000 7f73b97f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591864aad20 con 0x5591864a8170 2024-05-09T00:07:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.738+0000 7f73b97f5b80 10 client.0 _ll_register_callbacks cb 0x559186330ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.738+0000 7f73a67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186441270 0x559186448330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.738+0000 7f73a67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186441270 0x559186448330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53412/0 (socket says 172.21.15.12:53412) 2024-05-09T00:07:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.738+0000 7f73a67fc700 1 -- 172.21.15.12:0/2502850537 learned_addr learned my addr 172.21.15.12:0/2502850537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.744+0000 7f73a77fe700 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591864a8170 0x5591864aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.744+0000 7f73a67fc700 1 -- 172.21.15.12:0/2502850537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591864a8170 msgr2=0x5591864aa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.744+0000 7f73a67fc700 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591864a8170 0x5591864aa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.744+0000 7f73a67fc700 1 -- 172.21.15.12:0/2502850537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55918643b480 msgr2=0x559186447df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:07:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.744+0000 7f73a67fc700 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55918643b480 0x559186447df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.744+0000 7f73a67fc700 1 -- 172.21.15.12:0/2502850537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55918620c350 con 0x559186441270 2024-05-09T00:07:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.748+0000 7f73a67fc700 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186441270 0x559186448330 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f739c0048a0 tx=0x7f739c006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b5801513052eb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.748+0000 7f738aff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.749+0000 7f738aff5700 1 -- 172.21.15.12:0/2502850537 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c010580 con 0x559186441270 2024-05-09T00:07:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.749+0000 7f738aff5700 1 -- 172.21.15.12:0/2502850537 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f739c0193a0 con 0x559186441270 2024-05-09T00:07:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.749+0000 7f73b97f5b80 1 -- 172.21.15.12:0/2502850537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73a00093b0 con 0x559186441270 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.749+0000 7f738aff5700 1 -- 172.21.15.12:0/2502850537 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f739c005a50 con 0x559186441270 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.749+0000 7f73b97f5b80 10 client.21150 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.749+0000 7f73b97f5b80 1 -- 172.21.15.12:0/2502850537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x559186351500 con 0x559186441270 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.749+0000 7f738aff5700 1 -- 172.21.15.12:0/2502850537 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a00093b0 con 0x559186441270 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.749+0000 7f738aff5700 1 -- 172.21.15.12:0/2502850537 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f739c010cb0 con 0x559186441270 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.750+0000 7f73b97f5b80 10 client.21150 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.750+0000 7f73b97f5b80 10 client.21150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.750+0000 7f73b97f5b80 20 client.21150 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.750+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591864ac3f0 0x5591864ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.750+0000 7f73b97f5b80 4 client.21150 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:07:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.750+0000 7f73b97f5b80 1 -- 172.21.15.12:0/2502850537 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559186383e60 con 0x5591864ac3f0 2024-05-09T00:07:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.750+0000 7f73a6ffd700 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591864ac3f0 0x5591864ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.750+0000 7f73a6ffd700 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591864ac3f0 0x5591864ae810 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386354a32108aa8 server_cookie=44fc6413182bf3da in_seq=0 out_seq=0 2024-05-09T00:07:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:08.750+0000 7f738aff5700 10 client.21150 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.171+0000 7f738aff5700 1 -- 172.21.15.12:0/2502850537 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559186383e60 con 0x5591864ac3f0 2024-05-09T00:07:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.171+0000 7f738aff5700 10 client.21150 handle_command_reply: tid=0 2024-05-09T00:07:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.171+0000 7f73b97f5b80 1 client.21150 shutdown 2024-05-09T00:07:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.171+0000 7f73b97f5b80 1 -- 172.21.15.12:0/2502850537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186441270 msgr2=0x559186448330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.171+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186441270 0x559186448330 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f739c0048a0 tx=0x7f739c006ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.171+0000 7f73b97f5b80 1 -- 172.21.15.12:0/2502850537 shutdown_connections 2024-05-09T00:07:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.172+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591864a8170 0x5591864aa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.172+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5591864ac3f0 0x5591864ae810 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.172+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55918643b480 0x559186447df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.172+0000 7f73b97f5b80 1 --2- 172.21.15.12:0/2502850537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559186441270 0x559186448330 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.172+0000 7f73b97f5b80 1 -- 172.21.15.12:0/2502850537 >> 172.21.15.12:0/2502850537 conn(0x5591863991b0 msgr2=0x5591864506e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.172+0000 7f738aff5700 0 client.21150 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.172+0000 7f73b97f5b80 1 -- 172.21.15.12:0/2502850537 shutdown_connections 2024-05-09T00:07:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.172+0000 7f73b97f5b80 1 -- 172.21.15.12:0/2502850537 wait complete. 2024-05-09T00:07:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.172+0000 7f73b97f5b80 20 client.21150 trim_cache size 0 max 16384 2024-05-09T00:07:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:07:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:07:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:07:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:07:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.173+0000 7f73ad63b700 1 -- 172.21.15.12:0/1442715928 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f73900a13b0 msgr2=0x7f73900a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.173+0000 7f73ad63b700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f73900a13b0 0x7f73900a3870 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f73a0007a30 tx=0x7f73a00099e0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.173+0000 7f73ad63b700 1 -- 172.21.15.12:0/1442715928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73a8107fa0 msgr2=0x7f73a81a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.173+0000 7f73ad63b700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73a8107fa0 0x7f73a81a6380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f739800b500 tx=0x7f739800b8c0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.177+0000 7f73ad63b700 1 -- 172.21.15.12:0/1442715928 shutdown_connections 2024-05-09T00:07:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.177+0000 7f73ad63b700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f73900a13b0 0x7f73900a3870 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.177+0000 7f73ad63b700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73a810f630 0x7f73a81aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.177+0000 7f73ad63b700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73a8107fa0 0x7f73a81a6380 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.177+0000 7f73ad63b700 1 --2- 172.21.15.12:0/1442715928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73a810d440 0x7f73a81a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.177+0000 7f73ad63b700 1 -- 172.21.15.12:0/1442715928 >> 172.21.15.12:0/1442715928 conn(0x7f73a806bf80 msgr2=0x7f73a80719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.177+0000 7f73ad63b700 1 -- 172.21.15.12:0/1442715928 shutdown_connections 2024-05-09T00:07:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:09.177+0000 7f73ad63b700 1 -- 172.21.15.12:0/1442715928 wait complete. 2024-05-09T00:07:09.274 INFO:tasks.workunit.client.0.smithi012.stdout: 10 266949 31.90 MB/sec execute 304 sec latency 476.683 ms 2024-05-09T00:07:10.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:09 smithi204 ceph-mon[47373]: pgmap v667: 97 pgs: 29 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 54 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 581 op/s 2024-05-09T00:07:10.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:09 smithi160 ceph-mon[46152]: pgmap v667: 97 pgs: 29 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 54 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 581 op/s 2024-05-09T00:07:10.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:09 smithi012 ceph-mon[42503]: pgmap v667: 97 pgs: 29 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 54 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 581 op/s 2024-05-09T00:07:10.274 INFO:tasks.workunit.client.0.smithi012.stdout: 10 267610 31.94 MB/sec execute 305 sec latency 108.565 ms 2024-05-09T00:07:11.274 INFO:tasks.workunit.client.0.smithi012.stdout: 10 268206 31.91 MB/sec execute 306 sec latency 78.196 ms 2024-05-09T00:07:12.012 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:11 smithi204 ceph-mon[47373]: pgmap v668: 97 pgs: 18 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 69 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 586 op/s 2024-05-09T00:07:12.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:11 smithi160 ceph-mon[46152]: pgmap v668: 97 pgs: 18 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 69 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 586 op/s 2024-05-09T00:07:12.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:11 smithi012 ceph-mon[42503]: pgmap v668: 97 pgs: 18 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 69 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 586 op/s 2024-05-09T00:07:12.274 INFO:tasks.workunit.client.0.smithi012.stdout: 10 268923 31.96 MB/sec execute 307 sec latency 66.847 ms 2024-05-09T00:07:13.274 INFO:tasks.workunit.client.0.smithi012.stdout: 10 269259 31.93 MB/sec execute 308 sec latency 284.017 ms 2024-05-09T00:07:14.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:13 smithi204 ceph-mon[47373]: pgmap v669: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 639 op/s 2024-05-09T00:07:14.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:13 smithi160 ceph-mon[46152]: pgmap v669: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 639 op/s 2024-05-09T00:07:14.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:13 smithi012 ceph-mon[42503]: pgmap v669: 97 pgs: 97 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 639 op/s 2024-05-09T00:07:14.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 269610 31.84 MB/sec execute 309 sec latency 296.399 ms 2024-05-09T00:07:15.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 270348 31.88 MB/sec execute 310 sec latency 115.957 ms 2024-05-09T00:07:16.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:15 smithi204 ceph-mon[47373]: pgmap v670: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 555 op/s 2024-05-09T00:07:16.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:15 smithi160 ceph-mon[46152]: pgmap v670: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 555 op/s 2024-05-09T00:07:16.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:15 smithi012 ceph-mon[42503]: pgmap v670: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 555 op/s 2024-05-09T00:07:16.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 270891 31.89 MB/sec execute 311 sec latency 139.203 ms 2024-05-09T00:07:17.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:16 smithi204 ceph-mon[47373]: osdmap e127: 12 total, 12 up, 12 in 2024-05-09T00:07:17.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:16 smithi160 ceph-mon[46152]: osdmap e127: 12 total, 12 up, 12 in 2024-05-09T00:07:17.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 271401 31.87 MB/sec execute 312 sec latency 149.963 ms 2024-05-09T00:07:17.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:16 smithi012 ceph-mon[42503]: osdmap e127: 12 total, 12 up, 12 in 2024-05-09T00:07:18.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:17 smithi204 ceph-mon[47373]: pgmap v672: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 670 op/s 2024-05-09T00:07:18.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:17 smithi160 ceph-mon[46152]: pgmap v672: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 670 op/s 2024-05-09T00:07:18.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 271956 31.85 MB/sec execute 313 sec latency 154.755 ms 2024-05-09T00:07:18.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:17 smithi012 ceph-mon[42503]: pgmap v672: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 670 op/s 2024-05-09T00:07:19.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 272217 31.80 MB/sec execute 314 sec latency 559.779 ms 2024-05-09T00:07:20.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:19 smithi204 ceph-mon[47373]: pgmap v673: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 650 op/s 2024-05-09T00:07:20.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:19 smithi160 ceph-mon[46152]: pgmap v673: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 650 op/s 2024-05-09T00:07:20.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:19 smithi012 ceph-mon[42503]: pgmap v673: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 650 op/s 2024-05-09T00:07:20.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 272665 31.77 MB/sec execute 315 sec latency 160.337 ms 2024-05-09T00:07:21.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 273294 31.78 MB/sec execute 316 sec latency 106.310 ms 2024-05-09T00:07:22.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:21 smithi204 ceph-mon[47373]: pgmap v674: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 641 op/s 2024-05-09T00:07:22.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:21 smithi160 ceph-mon[46152]: pgmap v674: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 641 op/s 2024-05-09T00:07:22.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:21 smithi012 ceph-mon[42503]: pgmap v674: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 641 op/s 2024-05-09T00:07:22.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 273967 31.80 MB/sec execute 317 sec latency 92.427 ms 2024-05-09T00:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.337+0000 7f1aa5907700 1 -- 172.21.15.12:0/2139883741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa0101ab0 msgr2=0x7f1aa0105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.337+0000 7f1aa5907700 1 --2- 172.21.15.12:0/2139883741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa0101ab0 0x7f1aa0105af0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009990 tx=0x7f1a94009ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.337+0000 7f1aa5907700 1 -- 172.21.15.12:0/2139883741 shutdown_connections 2024-05-09T00:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.337+0000 7f1aa5907700 1 --2- 172.21.15.12:0/2139883741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1aa0106220 0x7f1aa010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.337+0000 7f1aa5907700 1 --2- 172.21.15.12:0/2139883741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa0101100 0x7f1aa01014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.337+0000 7f1aa5907700 1 --2- 172.21.15.12:0/2139883741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa0101ab0 0x7f1aa0105af0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.337+0000 7f1aa5907700 1 -- 172.21.15.12:0/2139883741 >> 172.21.15.12:0/2139883741 conn(0x7f1aa00fc9a0 msgr2=0x7f1aa00fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.337+0000 7f1aa5907700 1 -- 172.21.15.12:0/2139883741 shutdown_connections 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.337+0000 7f1aa5907700 1 -- 172.21.15.12:0/2139883741 wait complete. 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1aa5907700 1 Processor -- start 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1aa5907700 1 -- start start 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1aa5907700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa0101100 0x7f1aa0197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1aa5907700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa0106220 0x7f1aa0197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1aa5907700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1aa019bca0 0x7f1aa019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1aa5907700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa019c720 con 0x7f1aa0101100 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1aa5907700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa019c890 con 0x7f1aa019bca0 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1aa5907700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa019ca50 con 0x7f1aa0106220 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1a9effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa0101100 0x7f1aa0197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1a9effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa0101100 0x7f1aa0197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35460/0 (socket says 172.21.15.12:35460) 2024-05-09T00:07:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1a9effd700 1 -- 172.21.15.12:0/2146932877 learned_addr learned my addr 172.21.15.12:0/2146932877 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa0106220 0x7f1aa0197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.338+0000 7f1a9f7fe700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1aa019bca0 0x7f1aa019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1a9effd700 1 -- 172.21.15.12:0/2146932877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa0106220 msgr2=0x7f1aa0197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1a9effd700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa0106220 0x7f1aa0197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1a9effd700 1 -- 172.21.15.12:0/2146932877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1aa019bca0 msgr2=0x7f1aa019c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1a9effd700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1aa019bca0 0x7f1aa019c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1a9effd700 1 -- 172.21.15.12:0/2146932877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940095f0 con 0x7f1aa0101100 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa0106220 0x7f1aa0197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1a9effd700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa0101100 0x7f1aa0197410 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000e9a0 tx=0x7f1a9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1aa4905700 1 -- 172.21.15.12:0/2146932877 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90009790 con 0x7f1aa0101100 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1aa4905700 1 -- 172.21.15.12:0/2146932877 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a90010430 con 0x7f1aa0101100 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1aa5907700 1 -- 172.21.15.12:0/2146932877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa01a3fb0 con 0x7f1aa0101100 2024-05-09T00:07:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1aa4905700 1 -- 172.21.15.12:0/2146932877 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90018680 con 0x7f1aa0101100 2024-05-09T00:07:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.339+0000 7f1aa5907700 1 -- 172.21.15.12:0/2146932877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa01a43d0 con 0x7f1aa0101100 2024-05-09T00:07:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.342+0000 7f1aa4905700 1 -- 172.21.15.12:0/2146932877 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1a900105a0 con 0x7f1aa0101100 2024-05-09T00:07:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.343+0000 7f1aa4905700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a880a1310 0x7f1a880a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.343+0000 7f1ab1ac1b80 1 Processor -- start 2024-05-09T00:07:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.343+0000 7f1aa4905700 1 -- 172.21.15.12:0/2146932877 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1a90014070 con 0x7f1aa0101100 2024-05-09T00:07:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.343+0000 7f1ab1ac1b80 1 -- start start 2024-05-09T00:07:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a880a1310 0x7f1a880a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab12ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab124f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab1258a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab07c0f0 con 0x55cbab12b270 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab079ab0 con 0x55cbab125480 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab0753a0 con 0x55cbab12a950 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1a9e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab124f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1a9e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab124f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35464/0 (socket says 172.21.15.12:35464) 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1a9e7fc700 1 -- 172.21.15.12:0/2067058595 learned_addr learned my addr 172.21.15.12:0/2067058595 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1a9f7fe700 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab1258a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.344+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a880a1310 0x7f1a880a37d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400b2e0 tx=0x7f1a940045f0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a9effd700 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab12ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a9effd700 1 -- 172.21.15.12:0/2067058595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 msgr2=0x55cbab1258a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a9effd700 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab1258a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a9effd700 1 -- 172.21.15.12:0/2067058595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 msgr2=0x55cbab124f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a9effd700 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab124f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a9effd700 1 -- 172.21.15.12:0/2067058595 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbaaef6350 con 0x55cbab12a950 2024-05-09T00:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a9f7fe700 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab1258a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab124f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a9effd700 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab12ad30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1a900ca580 tx=0x7f1a90099990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5de1b0dfa0f67e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a85ffb700 1 -- 172.21.15.12:0/2067058595 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9009dd70 con 0x55cbab12a950 2024-05-09T00:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.345+0000 7f1a85ffb700 1 -- 172.21.15.12:0/2067058595 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a900a23c0 con 0x55cbab12a950 2024-05-09T00:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.346+0000 7f1a85ffb700 1 -- 172.21.15.12:0/2067058595 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90099690 con 0x55cbab12a950 2024-05-09T00:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.347+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/2067058595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 msgr2=0x55cbab12ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.347+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab12ad30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1a900ca580 tx=0x7f1a90099990 comp rx=0 tx=0).stop 2024-05-09T00:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.350+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/2067058595 shutdown_connections 2024-05-09T00:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.350+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab1258a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.350+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab12ad30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.350+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/2067058595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab124f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.350+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/2067058595 >> 172.21.15.12:0/2067058595 conn(0x55cbab0831b0 msgr2=0x55cbab11f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.350+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/2067058595 shutdown_connections 2024-05-09T00:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.350+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/2067058595 wait complete. 2024-05-09T00:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.351+0000 7f1ab1ac1b80 1 -- start start 2024-05-09T00:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.351+0000 7f1ab1ac1b80 1 Processor -- start 2024-05-09T00:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab125480 0x55cbab12ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab12a950 0x55cbab12efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab12f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab011500 con 0x55cbab12b270 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab0111a0 con 0x55cbab12a950 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaaef8640 con 0x55cbab125480 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1ab1ac1b80 10 client.0 _ll_register_callbacks cb 0x55cbab01aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab125480 0x55cbab12ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9effd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab125480 0x55cbab12ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59738/0 (socket says 172.21.15.12:59738) 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9effd700 1 -- 172.21.15.12:0/347238383 learned_addr learned my addr 172.21.15.12:0/347238383 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab12a950 0x55cbab12efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9f7fe700 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab12f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9effd700 1 -- 172.21.15.12:0/347238383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab12a950 msgr2=0x55cbab12efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9effd700 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab12a950 0x55cbab12efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9effd700 1 -- 172.21.15.12:0/347238383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 msgr2=0x55cbab12f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9effd700 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab12f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9effd700 1 -- 172.21.15.12:0/347238383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa010e610 con 0x55cbab125480 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.352+0000 7f1a9f7fe700 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab12f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.353+0000 7f1a9effd700 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab125480 0x55cbab12ea70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000c2d0 tx=0x7f1a90010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd95eb95c4eae64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.353+0000 7f1a7effd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.353+0000 7f1a7effd700 1 -- 172.21.15.12:0/347238383 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9009ddf0 con 0x55cbab125480 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.353+0000 7f1a7effd700 1 -- 172.21.15.12:0/347238383 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1a9009a3b0 con 0x55cbab125480 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.353+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/347238383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbaafce210 con 0x55cbab125480 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.353+0000 7f1a7effd700 1 -- 172.21.15.12:0/347238383 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a900b8250 con 0x55cbab125480 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.353+0000 7f1ab1ac1b80 10 client.30877 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.353+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/347238383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7f1aa01a43d0 con 0x55cbab125480 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.353+0000 7f1a7effd700 1 -- 172.21.15.12:0/347238383 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cbaafce210 con 0x55cbab125480 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.353+0000 7f1a7effd700 1 -- 172.21.15.12:0/347238383 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a90098ea0 con 0x55cbab125480 2024-05-09T00:07:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.354+0000 7f1ab1ac1b80 10 client.30877 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.354+0000 7f1ab1ac1b80 10 client.30877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.354+0000 7f1ab1ac1b80 20 client.30877 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.354+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbab147180 0x55cbab149570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.354+0000 7f1ab1ac1b80 4 client.30877 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.354+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/347238383 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbab0f47b0 con 0x55cbab147180 2024-05-09T00:07:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.354+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbab147180 0x55cbab149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.354+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbab147180 0x55cbab149570 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b2a1e3427a35f42 server_cookie=ac523ad6e3e3c519 in_seq=0 out_seq=0 2024-05-09T00:07:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.354+0000 7f1a7effd700 10 client.30877 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1a7effd700 1 -- 172.21.15.12:0/347238383 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbab0f47b0 con 0x55cbab147180 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1a7effd700 10 client.30877 handle_command_reply: tid=0 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 client.30877 shutdown 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/347238383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab125480 msgr2=0x55cbab12ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab125480 0x55cbab12ea70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000c2d0 tx=0x7f1a90010ab0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/347238383 shutdown_connections 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab12f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab125480 0x55cbab12ea70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbab147180 0x55cbab149570 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/347238383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab12a950 0x55cbab12efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1a7effd700 0 client.30877 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/347238383 >> 172.21.15.12:0/347238383 conn(0x55cbab0831b0 msgr2=0x55cbab121f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/347238383 shutdown_connections 2024-05-09T00:07:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/347238383 wait complete. 2024-05-09T00:07:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.360+0000 7f1ab1ac1b80 20 client.30877 trim_cache size 0 max 16384 2024-05-09T00:07:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1ab1ac1b80 1 Processor -- start 2024-05-09T00:07:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1ab1ac1b80 1 -- start start 2024-05-09T00:07:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab124f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab11f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab1334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab11a330 con 0x55cbab12b270 2024-05-09T00:07:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab129840 con 0x55cbab125480 2024-05-09T00:07:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab12d660 con 0x55cbab12a950 2024-05-09T00:07:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1a9f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab1334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1a9e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab11f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1a9e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab11f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59754/0 (socket says 172.21.15.12:59754) 2024-05-09T00:07:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1a9f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab1334b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35484/0 (socket says 172.21.15.12:35484) 2024-05-09T00:07:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1a9e7fc700 1 -- 172.21.15.12:0/1046191194 learned_addr learned my addr 172.21.15.12:0/1046191194 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.382+0000 7f1a9effd700 1 --2- 172.21.15.12:0/1046191194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab124f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.383+0000 7f1a9e7fc700 1 -- 172.21.15.12:0/1046191194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 msgr2=0x55cbab124f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.383+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/1046191194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab124f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.383+0000 7f1a9e7fc700 1 -- 172.21.15.12:0/1046191194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 msgr2=0x55cbab1334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.383+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/1046191194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab1334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.383+0000 7f1a9e7fc700 1 -- 172.21.15.12:0/1046191194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbaaea2b00 con 0x55cbab12a950 2024-05-09T00:07:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.383+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/1046191194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab11f910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400b690 tx=0x7f1a94022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ae2eac1c3eec5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.383+0000 7f1a867fc700 1 -- 172.21.15.12:0/1046191194 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94037070 con 0x55cbab12a950 2024-05-09T00:07:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.383+0000 7f1a867fc700 1 -- 172.21.15.12:0/1046191194 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a9401e040 con 0x55cbab12a950 2024-05-09T00:07:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.383+0000 7f1a867fc700 1 -- 172.21.15.12:0/1046191194 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940266d0 con 0x55cbab12a950 2024-05-09T00:07:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.384+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1046191194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 msgr2=0x55cbab11f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.384+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/1046191194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab11f910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400b690 tx=0x7f1a94022000 comp rx=0 tx=0).stop 2024-05-09T00:07:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.384+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1046191194 shutdown_connections 2024-05-09T00:07:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.384+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/1046191194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12b270 0x55cbab1334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.384+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/1046191194 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab124f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.384+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/1046191194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12a950 0x55cbab11f910 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.384+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1046191194 >> 172.21.15.12:0/1046191194 conn(0x55cbab0831b0 msgr2=0x55cbab11be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.384+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1046191194 shutdown_connections 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.384+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1046191194 wait complete. 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.384+0000 7f1ab1ac1b80 1 -- start start 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1ab1ac1b80 1 Processor -- start 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab143df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12a950 0x55cbab13f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1ab1ac1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12b270 0x55cbab13f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab13ff40 con 0x55cbab12a950 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab1400b0 con 0x55cbab125480 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1ab1ac1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbab19c5d0 con 0x55cbab12b270 2024-05-09T00:07:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1ab1ac1b80 10 client.0 _ll_register_callbacks cb 0x55cbab01aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1a9e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12a950 0x55cbab13f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1a9e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12a950 0x55cbab13f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35494/0 (socket says 172.21.15.12:35494) 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1a9effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1a9e7fc700 1 -- 172.21.15.12:0/1522063115 learned_addr learned my addr 172.21.15.12:0/1522063115 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.385+0000 7f1a9f7fe700 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12b270 0x55cbab13f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a9e7fc700 1 -- 172.21.15.12:0/1522063115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12b270 msgr2=0x55cbab13f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12b270 0x55cbab13f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a9e7fc700 1 -- 172.21.15.12:0/1522063115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 msgr2=0x55cbab143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a9e7fc700 1 -- 172.21.15.12:0/1522063115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9400f7e0 con 0x55cbab12a950 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a9effd700 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a9e7fc700 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12a950 0x55cbab13f260 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400b660 tx=0x7f1a9400f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8eb2275a6d2ab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a7effd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a7effd700 1 -- 172.21.15.12:0/1522063115 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94037050 con 0x55cbab12a950 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a7effd700 1 -- 172.21.15.12:0/1522063115 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1a9402c040 con 0x55cbab12a950 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1522063115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a900a23c0 con 0x55cbab12a950 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1a7effd700 1 -- 172.21.15.12:0/1522063115 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a94032ea0 con 0x55cbab12a950 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1ab1ac1b80 10 client.21168 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.386+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1522063115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55cbab03b500 con 0x55cbab12a950 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1a7effd700 1 -- 172.21.15.12:0/1522063115 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a900a23c0 con 0x55cbab12a950 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1a7effd700 1 -- 172.21.15.12:0/1522063115 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a94026490 con 0x55cbab12a950 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1ab1ac1b80 10 client.21168 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1ab1ac1b80 10 client.21168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1ab1ac1b80 20 client.21168 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbab19d950 0x55cbab19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1ab1ac1b80 4 client.21168 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:07:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1522063115 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cbab06de60 con 0x55cbab19d950 2024-05-09T00:07:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1a9effd700 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbab19d950 0x55cbab19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1a7effd700 10 client.21168 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.387+0000 7f1a9effd700 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbab19d950 0x55cbab19fd70 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=731cb1fef562af08 server_cookie=4600fa08bc3bee5c in_seq=0 out_seq=0 2024-05-09T00:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.831+0000 7f1a7effd700 1 -- 172.21.15.12:0/1522063115 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cbab06de60 con 0x55cbab19d950 2024-05-09T00:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.831+0000 7f1a7effd700 10 client.21168 handle_command_reply: tid=0 2024-05-09T00:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.831+0000 7f1ab1ac1b80 1 client.21168 shutdown 2024-05-09T00:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.831+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1522063115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12a950 msgr2=0x55cbab13f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.831+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12a950 0x55cbab13f260 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400b660 tx=0x7f1a9400f5d0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.832+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1522063115 shutdown_connections 2024-05-09T00:07:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.832+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbab12b270 0x55cbab13f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.832+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55cbab19d950 0x55cbab19fd70 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.832+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cbab125480 0x55cbab143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.832+0000 7f1ab1ac1b80 1 --2- 172.21.15.12:0/1522063115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cbab12a950 0x55cbab13f260 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.832+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1522063115 >> 172.21.15.12:0/1522063115 conn(0x55cbab0831b0 msgr2=0x55cbab11be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.832+0000 7f1a7effd700 0 client.21168 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.835+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1522063115 shutdown_connections 2024-05-09T00:07:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.835+0000 7f1ab1ac1b80 1 -- 172.21.15.12:0/1522063115 wait complete. 2024-05-09T00:07:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.835+0000 7f1ab1ac1b80 20 client.21168 trim_cache size 0 max 16384 2024-05-09T00:07:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:07:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:07:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:07:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:07:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.840+0000 7f1aa5907700 1 -- 172.21.15.12:0/2146932877 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a880a1310 msgr2=0x7f1a880a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.840+0000 7f1aa5907700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a880a1310 0x7f1a880a37d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400b2e0 tx=0x7f1a940045f0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.840+0000 7f1aa5907700 1 -- 172.21.15.12:0/2146932877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa0101100 msgr2=0x7f1aa0197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.840+0000 7f1aa5907700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa0101100 0x7f1aa0197410 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000e9a0 tx=0x7f1a9000ecb0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.840+0000 7f1aa5907700 1 -- 172.21.15.12:0/2146932877 shutdown_connections 2024-05-09T00:07:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.840+0000 7f1aa5907700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1a880a1310 0x7f1a880a37d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.840+0000 7f1aa5907700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1aa019bca0 0x7f1aa019c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.840+0000 7f1aa5907700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1aa0101100 0x7f1aa0197410 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.840+0000 7f1aa5907700 1 --2- 172.21.15.12:0/2146932877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa0106220 0x7f1aa0197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.840+0000 7f1aa5907700 1 -- 172.21.15.12:0/2146932877 >> 172.21.15.12:0/2146932877 conn(0x7f1aa00fc9a0 msgr2=0x7f1aa00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.842+0000 7f1aa5907700 1 -- 172.21.15.12:0/2146932877 shutdown_connections 2024-05-09T00:07:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:22.842+0000 7f1aa5907700 1 -- 172.21.15.12:0/2146932877 wait complete. 2024-05-09T00:07:23.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 274256 31.73 MB/sec execute 318 sec latency 332.155 ms 2024-05-09T00:07:24.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:23 smithi204 ceph-mon[47373]: pgmap v675: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 610 op/s 2024-05-09T00:07:24.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:23 smithi160 ceph-mon[46152]: pgmap v675: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 610 op/s 2024-05-09T00:07:24.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:23 smithi012 ceph-mon[42503]: pgmap v675: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 610 op/s 2024-05-09T00:07:24.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 274683 31.70 MB/sec execute 319 sec latency 296.941 ms 2024-05-09T00:07:24.323 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:07:24.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.563+0000 7f64eff9d700 1 -- 172.21.15.12:0/2054330838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e8101b20 msgr2=0x7f64e8105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.563+0000 7f64eff9d700 1 --2- 172.21.15.12:0/2054330838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e8101b20 0x7f64e8105b60 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f64d80098f0 tx=0x7f64d8009c00 comp rx=0 tx=0).stop 2024-05-09T00:07:24.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.564+0000 7f64eff9d700 1 -- 172.21.15.12:0/2054330838 shutdown_connections 2024-05-09T00:07:24.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.564+0000 7f64eff9d700 1 --2- 172.21.15.12:0/2054330838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e8106290 0x7f64e810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.564+0000 7f64eff9d700 1 --2- 172.21.15.12:0/2054330838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64e8101170 0x7f64e8101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.564+0000 7f64eff9d700 1 --2- 172.21.15.12:0/2054330838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e8101b20 0x7f64e8105b60 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.564+0000 7f64eff9d700 1 -- 172.21.15.12:0/2054330838 >> 172.21.15.12:0/2054330838 conn(0x7f64e80fc9d0 msgr2=0x7f64e80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:24.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.564+0000 7f64eff9d700 1 -- 172.21.15.12:0/2054330838 shutdown_connections 2024-05-09T00:07:24.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.564+0000 7f64eff9d700 1 -- 172.21.15.12:0/2054330838 wait complete. 2024-05-09T00:07:24.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.565+0000 7f64eff9d700 1 Processor -- start 2024-05-09T00:07:24.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.565+0000 7f64eff9d700 1 -- start start 2024-05-09T00:07:24.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64eff9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e8101170 0x7f64e81a2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64eff9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e8106290 0x7f64e81a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64eff9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64e810cfc0 0x7f64e810d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64eff9d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e810da50 con 0x7f64e8106290 2024-05-09T00:07:24.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64eff9d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e810dbc0 con 0x7f64e810cfc0 2024-05-09T00:07:24.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64eff9d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e810dd30 con 0x7f64e8101170 2024-05-09T00:07:24.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64ed538700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e8106290 0x7f64e81a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64ed538700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e8106290 0x7f64e81a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35514/0 (socket says 172.21.15.12:35514) 2024-05-09T00:07:24.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64ed538700 1 -- 172.21.15.12:0/924543395 learned_addr learned my addr 172.21.15.12:0/924543395 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:24.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64ee53a700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64e810cfc0 0x7f64e810d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.566+0000 7f64edd39700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e8101170 0x7f64e81a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.567+0000 7f64ee53a700 1 -- 172.21.15.12:0/924543395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e8101170 msgr2=0x7f64e81a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.567+0000 7f64ee53a700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e8101170 0x7f64e81a2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.567+0000 7f64ee53a700 1 -- 172.21.15.12:0/924543395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e8106290 msgr2=0x7f64e81a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.567+0000 7f64ee53a700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e8106290 0x7f64e81a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.567+0000 7f64ee53a700 1 -- 172.21.15.12:0/924543395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e0009c40 con 0x7f64e810cfc0 2024-05-09T00:07:24.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.567+0000 7f64ee53a700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64e810cfc0 0x7f64e810d450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f64e000efb0 tx=0x7f64e000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:24.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.569+0000 7f64deffd700 1 -- 172.21.15.12:0/924543395 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0003a00 con 0x7f64e810cfc0 2024-05-09T00:07:24.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.569+0000 7f64deffd700 1 -- 172.21.15.12:0/924543395 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e000f040 con 0x7f64e810cfc0 2024-05-09T00:07:24.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.569+0000 7f64deffd700 1 -- 172.21.15.12:0/924543395 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0004520 con 0x7f64e810cfc0 2024-05-09T00:07:24.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.569+0000 7f64eff9d700 1 -- 172.21.15.12:0/924543395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d8009520 con 0x7f64e810cfc0 2024-05-09T00:07:24.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.569+0000 7f64eff9d700 1 -- 172.21.15.12:0/924543395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e81aa980 con 0x7f64e810cfc0 2024-05-09T00:07:24.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.572+0000 7f64deffd700 1 -- 172.21.15.12:0/924543395 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f64e0004680 con 0x7f64e810cfc0 2024-05-09T00:07:24.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.573+0000 7f64deffd700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f64d40a1460 0x7f64d40a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.573+0000 7f64deffd700 1 -- 172.21.15.12:0/924543395 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f64e00cc690 con 0x7f64e810cfc0 2024-05-09T00:07:24.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.574+0000 7f64edd39700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f64d40a1460 0x7f64d40a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.574+0000 7f64fc157b80 1 Processor -- start 2024-05-09T00:07:24.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.575+0000 7f64edd39700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f64d40a1460 0x7f64d40a3920 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005fd0 tx=0x7f64e4009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:24.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.575+0000 7f64fc157b80 1 -- start start 2024-05-09T00:07:24.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.575+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 0x55ecd07971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.575+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd0791350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.575+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0791890 0x55ecd0791cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.575+0000 7f64fc157b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd06e8150 con 0x55ecd0791890 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.575+0000 7f64fc157b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd06e5b10 con 0x55ecd0796e00 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.575+0000 7f64fc157b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd06e1400 con 0x55ecd0797720 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.575+0000 7f64edd39700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 0x55ecd07971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.575+0000 7f64ed538700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd0791350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.576+0000 7f64ed538700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd0791350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59784/0 (socket says 172.21.15.12:59784) 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.576+0000 7f64ed538700 1 -- 172.21.15.12:0/889157239 learned_addr learned my addr 172.21.15.12:0/889157239 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.576+0000 7f64ee53a700 1 --2- 172.21.15.12:0/889157239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0791890 0x55ecd0791cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.576+0000 7f64ed538700 1 -- 172.21.15.12:0/889157239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 msgr2=0x55ecd07971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.576+0000 7f64ed538700 1 --2- 172.21.15.12:0/889157239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 0x55ecd07971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.576+0000 7f64ed538700 1 -- 172.21.15.12:0/889157239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0791890 msgr2=0x55ecd0791cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.576+0000 7f64ed538700 1 --2- 172.21.15.12:0/889157239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0791890 0x55ecd0791cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.576+0000 7f64ed538700 1 -- 172.21.15.12:0/889157239 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd05623b0 con 0x55ecd0797720 2024-05-09T00:07:24.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.576+0000 7f64ed538700 1 --2- 172.21.15.12:0/889157239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd0791350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f64d8005e90 tx=0x7f64d8005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eed253c4a9b1a5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:24.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.577+0000 7f64d3fff700 1 -- 172.21.15.12:0/889157239 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d800fcb0 con 0x55ecd0797720 2024-05-09T00:07:24.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.577+0000 7f64d3fff700 1 -- 172.21.15.12:0/889157239 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64d801d040 con 0x55ecd0797720 2024-05-09T00:07:24.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.577+0000 7f64d3fff700 1 -- 172.21.15.12:0/889157239 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8005040 con 0x55ecd0797720 2024-05-09T00:07:24.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.578+0000 7f64fc157b80 1 -- 172.21.15.12:0/889157239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 msgr2=0x55ecd0791350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.578+0000 7f64fc157b80 1 --2- 172.21.15.12:0/889157239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd0791350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f64d8005e90 tx=0x7f64d8005dc0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.578+0000 7f64fc157b80 1 -- 172.21.15.12:0/889157239 shutdown_connections 2024-05-09T00:07:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.578+0000 7f64fc157b80 1 --2- 172.21.15.12:0/889157239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0791890 0x55ecd0791cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.578+0000 7f64fc157b80 1 --2- 172.21.15.12:0/889157239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 0x55ecd07971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.578+0000 7f64fc157b80 1 --2- 172.21.15.12:0/889157239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd0791350 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.578+0000 7f64fc157b80 1 -- 172.21.15.12:0/889157239 >> 172.21.15.12:0/889157239 conn(0x55ecd06ef210 msgr2=0x55ecd078bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:24.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.585+0000 7f64fc157b80 1 -- 172.21.15.12:0/889157239 shutdown_connections 2024-05-09T00:07:24.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.588+0000 7f64fc157b80 1 -- 172.21.15.12:0/889157239 wait complete. 2024-05-09T00:07:24.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.589+0000 7f64fc157b80 1 -- start start 2024-05-09T00:07:24.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.589+0000 7f64fc157b80 1 Processor -- start 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.589+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd079ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.589+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 0x55ecd079b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0797720 0x55ecd079b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64fc157b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd067d560 con 0x55ecd0796e00 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64fc157b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd067d200 con 0x55ecd0797720 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64fc157b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd05646a0 con 0x55ecd0791890 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64fc157b80 10 client.0 _ll_register_callbacks cb 0x55ecd0686b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64edd39700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd079ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64edd39700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd079ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59800/0 (socket says 172.21.15.12:59800) 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64edd39700 1 -- 172.21.15.12:0/3699172300 learned_addr learned my addr 172.21.15.12:0/3699172300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64ed538700 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 0x55ecd079b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64ee53a700 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0797720 0x55ecd079b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64ed538700 1 -- 172.21.15.12:0/3699172300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 msgr2=0x55ecd079ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:24.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64ed538700 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd079ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64ed538700 1 -- 172.21.15.12:0/3699172300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0797720 msgr2=0x55ecd079b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:24.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.590+0000 7f64ed538700 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0797720 0x55ecd079b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.591+0000 7f64ed538700 1 -- 172.21.15.12:0/3699172300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e810e680 con 0x55ecd0796e00 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.591+0000 7f64ed538700 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 0x55ecd079b090 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009810 tx=0x7f64d800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2e91c1e40f26e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.591+0000 7f64d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.591+0000 7f64d17fa700 1 -- 172.21.15.12:0/3699172300 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8030070 con 0x55ecd0796e00 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.591+0000 7f64fc157b80 1 -- 172.21.15.12:0/3699172300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecd063a270 con 0x55ecd0796e00 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.591+0000 7f64d17fa700 1 -- 172.21.15.12:0/3699172300 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f64d80058e0 con 0x55ecd0796e00 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.591+0000 7f64d17fa700 1 -- 172.21.15.12:0/3699172300 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64d800f670 con 0x55ecd0796e00 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.591+0000 7f64fc157b80 10 client.21180 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.591+0000 7f64fc157b80 1 -- 172.21.15.12:0/3699172300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55ecd050eb60 con 0x55ecd0796e00 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.592+0000 7f64d17fa700 1 -- 172.21.15.12:0/3699172300 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ecd063a270 con 0x55ecd0796e00 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.592+0000 7f64d17fa700 1 -- 172.21.15.12:0/3699172300 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f64d8033020 con 0x55ecd0796e00 2024-05-09T00:07:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.592+0000 7f64fc157b80 10 client.21180 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:24.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.592+0000 7f64fc157b80 10 client.21180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:24.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.592+0000 7f64fc157b80 20 client.21180 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:24.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.592+0000 7f64fc157b80 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ecd07aad90 0x55ecd07ad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.592+0000 7f64fc157b80 4 client.21180 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:24.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.592+0000 7f64fc157b80 1 -- 172.21.15.12:0/3699172300 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecd0760810 con 0x55ecd07aad90 2024-05-09T00:07:24.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.592+0000 7f64edd39700 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ecd07aad90 0x55ecd07ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.593+0000 7f64edd39700 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ecd07aad90 0x55ecd07ad180 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec8e7c6f4ae9839 server_cookie=6139b0591fec30b3 in_seq=0 out_seq=0 2024-05-09T00:07:24.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.596+0000 7f64d17fa700 10 client.21180 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.608+0000 7f64d17fa700 1 -- 172.21.15.12:0/3699172300 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ecd0760810 con 0x55ecd07aad90 2024-05-09T00:07:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.609+0000 7f64d17fa700 10 client.21180 handle_command_reply: tid=0 2024-05-09T00:07:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.609+0000 7f64fc157b80 1 client.21180 shutdown 2024-05-09T00:07:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 1 -- 172.21.15.12:0/3699172300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 msgr2=0x55ecd079b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 0x55ecd079b090 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009810 tx=0x7f64d800fd90 comp rx=0 tx=0).stop 2024-05-09T00:07:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 1 -- 172.21.15.12:0/3699172300 shutdown_connections 2024-05-09T00:07:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ecd07aad90 0x55ecd07ad180 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0797720 0x55ecd079b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd079ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64d17fa700 0 client.21180 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:24.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 1 --2- 172.21.15.12:0/3699172300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 0x55ecd079b090 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 1 -- 172.21.15.12:0/3699172300 >> 172.21.15.12:0/3699172300 conn(0x55ecd06ef210 msgr2=0x55ecd078b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:24.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 1 -- 172.21.15.12:0/3699172300 shutdown_connections 2024-05-09T00:07:24.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 1 -- 172.21.15.12:0/3699172300 wait complete. 2024-05-09T00:07:24.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.611+0000 7f64fc157b80 20 client.21180 trim_cache size 0 max 16384 2024-05-09T00:07:24.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.644+0000 7f64fc157b80 1 Processor -- start 2024-05-09T00:07:24.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64fc157b80 1 -- start start 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd0796510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 0x55ecd079a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0797720 0x55ecd078bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64fc157b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd07922a0 con 0x55ecd0797720 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64fc157b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd078d870 con 0x55ecd0796e00 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64fc157b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd078d640 con 0x55ecd0791890 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64ee53a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0797720 0x55ecd078bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64ed538700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 0x55ecd079a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64edd39700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd0796510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64ed538700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 0x55ecd079a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56250/0 (socket says 172.21.15.12:56250) 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64ed538700 1 -- 172.21.15.12:0/611568067 learned_addr learned my addr 172.21.15.12:0/611568067 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64ee53a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0797720 0x55ecd078bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35534/0 (socket says 172.21.15.12:35534) 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.645+0000 7f64edd39700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd0796510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59814/0 (socket says 172.21.15.12:59814) 2024-05-09T00:07:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.646+0000 7f64edd39700 1 -- 172.21.15.12:0/611568067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 msgr2=0x55ecd079a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:24.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.646+0000 7f64edd39700 1 --2- 172.21.15.12:0/611568067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 0x55ecd079a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.646+0000 7f64edd39700 1 -- 172.21.15.12:0/611568067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0797720 msgr2=0x55ecd078bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:24.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.646+0000 7f64edd39700 1 --2- 172.21.15.12:0/611568067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0797720 0x55ecd078bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.646+0000 7f64edd39700 1 -- 172.21.15.12:0/611568067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd06a7560 con 0x55ecd0791890 2024-05-09T00:07:24.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.646+0000 7f64ee53a700 1 --2- 172.21.15.12:0/611568067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0797720 0x55ecd078bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:24.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.646+0000 7f64edd39700 1 --2- 172.21.15.12:0/611568067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd0796510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009fd0 tx=0x7f64e4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec4b3a94cce7c5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:24.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.647+0000 7f64dcff9700 1 -- 172.21.15.12:0/611568067 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4024040 con 0x55ecd0791890 2024-05-09T00:07:24.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.647+0000 7f64dcff9700 1 -- 172.21.15.12:0/611568067 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e4003ea0 con 0x55ecd0791890 2024-05-09T00:07:24.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.647+0000 7f64dcff9700 1 -- 172.21.15.12:0/611568067 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e401eac0 con 0x55ecd0791890 2024-05-09T00:07:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.648+0000 7f64fc157b80 1 -- 172.21.15.12:0/611568067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 msgr2=0x55ecd0796510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.648+0000 7f64fc157b80 1 --2- 172.21.15.12:0/611568067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd0796510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009fd0 tx=0x7f64e4014000 comp rx=0 tx=0).stop 2024-05-09T00:07:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.648+0000 7f64fc157b80 1 -- 172.21.15.12:0/611568067 shutdown_connections 2024-05-09T00:07:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.648+0000 7f64fc157b80 1 --2- 172.21.15.12:0/611568067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0797720 0x55ecd078bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.648+0000 7f64fc157b80 1 --2- 172.21.15.12:0/611568067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0791890 0x55ecd0796510 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.648+0000 7f64fc157b80 1 --2- 172.21.15.12:0/611568067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0796e00 0x55ecd079a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.648+0000 7f64fc157b80 1 -- 172.21.15.12:0/611568067 >> 172.21.15.12:0/611568067 conn(0x55ecd06ef210 msgr2=0x55ecd079ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.652+0000 7f64fc157b80 1 -- 172.21.15.12:0/611568067 shutdown_connections 2024-05-09T00:07:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.652+0000 7f64fc157b80 1 -- 172.21.15.12:0/611568067 wait complete. 2024-05-09T00:07:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.652+0000 7f64fc157b80 1 -- start start 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.652+0000 7f64fc157b80 1 Processor -- start 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0791890 0x55ecd07a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 0x55ecd07a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64fc157b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd07a0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64fc157b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd07a0a80 con 0x55ecd0796e00 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64fc157b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd07a0bf0 con 0x55ecd0791890 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64fc157b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd07a0db0 con 0x55ecd0797720 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64fc157b80 10 client.0 _ll_register_callbacks cb 0x55ecd0686b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64ee53a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd07a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64ee53a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd07a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59816/0 (socket says 172.21.15.12:59816) 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64ee53a700 1 -- 172.21.15.12:0/1045522777 learned_addr learned my addr 172.21.15.12:0/1045522777 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64ed538700 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 0x55ecd07a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.653+0000 7f64edd39700 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0791890 0x55ecd07a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.654+0000 7f64ee53a700 1 -- 172.21.15.12:0/1045522777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0791890 msgr2=0x55ecd07a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.654+0000 7f64ee53a700 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0791890 0x55ecd07a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.654+0000 7f64ee53a700 1 -- 172.21.15.12:0/1045522777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 msgr2=0x55ecd07a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.654+0000 7f64ee53a700 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 0x55ecd07a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.654+0000 7f64ee53a700 1 -- 172.21.15.12:0/1045522777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e81aa980 con 0x55ecd0797720 2024-05-09T00:07:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.654+0000 7f64edd39700 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0791890 0x55ecd07a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:07:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.654+0000 7f64ee53a700 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd07a0480 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c7a0 tx=0x7f64e0099220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f36d3793434e83 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.654+0000 7f64d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:07:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.654+0000 7f64d17fa700 1 -- 172.21.15.12:0/1045522777 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e00ca210 con 0x55ecd0797720 2024-05-09T00:07:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.654+0000 7f64fc157b80 1 -- 172.21.15.12:0/1045522777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64d801d040 con 0x55ecd0797720 2024-05-09T00:07:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.664+0000 7f64d17fa700 1 -- 172.21.15.12:0/1045522777 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f64e00a1bd0 con 0x55ecd0797720 2024-05-09T00:07:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.664+0000 7f64d17fa700 1 -- 172.21.15.12:0/1045522777 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64e001b310 con 0x55ecd0797720 2024-05-09T00:07:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.664+0000 7f64fc157b80 10 client.30901 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.664+0000 7f64fc157b80 1 -- 172.21.15.12:0/1045522777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7f64e4017070 con 0x55ecd0797720 2024-05-09T00:07:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.666+0000 7f64d17fa700 1 -- 172.21.15.12:0/1045522777 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64d801d040 con 0x55ecd0797720 2024-05-09T00:07:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.666+0000 7f64d17fa700 1 -- 172.21.15.12:0/1045522777 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f64e0098dd0 con 0x55ecd0797720 2024-05-09T00:07:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.666+0000 7f64fc157b80 10 client.30901 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.666+0000 7f64fc157b80 10 client.30901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.666+0000 7f64fc157b80 20 client.30901 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.666+0000 7f64fc157b80 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ecd08024d0 0x55ecd08048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.666+0000 7f64fc157b80 4 client.30901 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:07:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.666+0000 7f64fc157b80 1 -- 172.21.15.12:0/1045522777 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ecd06d9ec0 con 0x55ecd08024d0 2024-05-09T00:07:24.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.671+0000 7f64edd39700 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ecd08024d0 0x55ecd08048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.672+0000 7f64edd39700 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ecd08024d0 0x55ecd08048f0 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea0a5a82f358db0 server_cookie=17433f9822fbc519 in_seq=0 out_seq=0 2024-05-09T00:07:24.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.672+0000 7f64d17fa700 10 client.30901 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.681+0000 7f64d17fa700 1 -- 172.21.15.12:0/1045522777 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ecd06d9ec0 con 0x55ecd08024d0 2024-05-09T00:07:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.681+0000 7f64d17fa700 10 client.30901 handle_command_reply: tid=0 2024-05-09T00:07:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.681+0000 7f64fc157b80 1 client.30901 shutdown 2024-05-09T00:07:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.681+0000 7f64fc157b80 1 -- 172.21.15.12:0/1045522777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 msgr2=0x55ecd07a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.681+0000 7f64fc157b80 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd07a0480 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c7a0 tx=0x7f64e0099220 comp rx=0 tx=0).stop 2024-05-09T00:07:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.682+0000 7f64fc157b80 1 -- 172.21.15.12:0/1045522777 shutdown_connections 2024-05-09T00:07:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.682+0000 7f64fc157b80 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecd0797720 0x55ecd07a0480 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.682+0000 7f64fc157b80 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ecd08024d0 0x55ecd08048f0 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.682+0000 7f64fc157b80 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ecd0791890 0x55ecd07a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.682+0000 7f64fc157b80 1 --2- 172.21.15.12:0/1045522777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecd0796e00 0x55ecd07a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.682+0000 7f64fc157b80 1 -- 172.21.15.12:0/1045522777 >> 172.21.15.12:0/1045522777 conn(0x55ecd06ef210 msgr2=0x55ecd079b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.684+0000 7f64d17fa700 0 client.30901 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.684+0000 7f64fc157b80 1 -- 172.21.15.12:0/1045522777 shutdown_connections 2024-05-09T00:07:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.684+0000 7f64fc157b80 1 -- 172.21.15.12:0/1045522777 wait complete. 2024-05-09T00:07:24.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.684+0000 7f64fc157b80 20 client.30901 trim_cache size 0 max 16384 2024-05-09T00:07:24.680 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:07:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:07:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:07:24.680 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:07:24.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.686+0000 7f64eff9d700 1 -- 172.21.15.12:0/924543395 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f64d40a1460 msgr2=0x7f64d40a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.686+0000 7f64eff9d700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f64d40a1460 0x7f64d40a3920 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005fd0 tx=0x7f64e4009310 comp rx=0 tx=0).stop 2024-05-09T00:07:24.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.686+0000 7f64eff9d700 1 -- 172.21.15.12:0/924543395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64e810cfc0 msgr2=0x7f64e810d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.686+0000 7f64eff9d700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64e810cfc0 0x7f64e810d450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f64e000efb0 tx=0x7f64e000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.688+0000 7f64eff9d700 1 -- 172.21.15.12:0/924543395 shutdown_connections 2024-05-09T00:07:24.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.688+0000 7f64eff9d700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f64d40a1460 0x7f64d40a3920 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.688+0000 7f64eff9d700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64e810cfc0 0x7f64e810d450 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.688+0000 7f64eff9d700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e8101170 0x7f64e81a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.688+0000 7f64eff9d700 1 --2- 172.21.15.12:0/924543395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e8106290 0x7f64e81a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.688+0000 7f64eff9d700 1 -- 172.21.15.12:0/924543395 >> 172.21.15.12:0/924543395 conn(0x7f64e80fc9d0 msgr2=0x7f64e80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:24.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.688+0000 7f64eff9d700 1 -- 172.21.15.12:0/924543395 shutdown_connections 2024-05-09T00:07:24.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.688+0000 7f64eff9d700 1 -- 172.21.15.12:0/924543395 wait complete. 2024-05-09T00:07:24.705 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:07:24.705 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:07:24.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.942+0000 7f2e643e2700 1 -- 172.21.15.12:0/139180361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e5c1028e0 msgr2=0x7f2e5c10adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.942+0000 7f2e643e2700 1 --2- 172.21.15.12:0/139180361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e5c1028e0 0x7f2e5c10adc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0098f0 tx=0x7f2e4c009c00 comp rx=0 tx=0).stop 2024-05-09T00:07:24.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.943+0000 7f2e643e2700 1 -- 172.21.15.12:0/139180361 shutdown_connections 2024-05-09T00:07:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.943+0000 7f2e643e2700 1 --2- 172.21.15.12:0/139180361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e5c10b3e0 0x7f2e5c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.943+0000 7f2e643e2700 1 --2- 172.21.15.12:0/139180361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e5c101fc0 0x7f2e5c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.943+0000 7f2e643e2700 1 --2- 172.21.15.12:0/139180361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e5c1028e0 0x7f2e5c10adc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.943+0000 7f2e643e2700 1 -- 172.21.15.12:0/139180361 >> 172.21.15.12:0/139180361 conn(0x7f2e5c0fb800 msgr2=0x7f2e5c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.943+0000 7f2e643e2700 1 -- 172.21.15.12:0/139180361 shutdown_connections 2024-05-09T00:07:24.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.943+0000 7f2e643e2700 1 -- 172.21.15.12:0/139180361 wait complete. 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.943+0000 7f2e643e2700 1 Processor -- start 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e643e2700 1 -- start start 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e643e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e5c101fc0 0x7f2e5c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e643e2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e5c1028e0 0x7f2e5c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e643e2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e5c10b3e0 0x7f2e5c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e643e2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c19c230 con 0x7f2e5c101fc0 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e643e2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c19c3a0 con 0x7f2e5c1028e0 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e643e2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c19c560 con 0x7f2e5c10b3e0 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e6217e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e5c101fc0 0x7f2e5c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e6217e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e5c101fc0 0x7f2e5c197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35564/0 (socket says 172.21.15.12:35564) 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e6217e700 1 -- 172.21.15.12:0/2159113120 learned_addr learned my addr 172.21.15.12:0/2159113120 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:24.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.944+0000 7f2e6297f700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e5c10b3e0 0x7f2e5c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e6297f700 1 -- 172.21.15.12:0/2159113120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e5c1028e0 msgr2=0x7f2e5c197850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e6297f700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e5c1028e0 0x7f2e5c197850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e6297f700 1 -- 172.21.15.12:0/2159113120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e5c101fc0 msgr2=0x7f2e5c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e6297f700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e5c101fc0 0x7f2e5c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e6297f700 1 -- 172.21.15.12:0/2159113120 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c009520 con 0x7f2e5c10b3e0 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e6297f700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e5c10b3e0 0x7f2e5c19bbf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400efc0 tx=0x7f2e5400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e537fe700 1 -- 172.21.15.12:0/2159113120 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54018070 con 0x7f2e5c10b3e0 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e643e2700 1 -- 172.21.15.12:0/2159113120 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5c19c7e0 con 0x7f2e5c10b3e0 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e537fe700 1 -- 172.21.15.12:0/2159113120 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e5400f040 con 0x7f2e5c10b3e0 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e537fe700 1 -- 172.21.15.12:0/2159113120 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400ce10 con 0x7f2e5c10b3e0 2024-05-09T00:07:24.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.945+0000 7f2e643e2700 1 -- 172.21.15.12:0/2159113120 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c1a4080 con 0x7f2e5c10b3e0 2024-05-09T00:07:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.946+0000 7f2e643e2700 1 -- 172.21.15.12:0/2159113120 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5c1064b0 con 0x7f2e5c10b3e0 2024-05-09T00:07:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.948+0000 7f2e537fe700 1 -- 172.21.15.12:0/2159113120 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2e54003a00 con 0x7f2e5c10b3e0 2024-05-09T00:07:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.948+0000 7f2e537fe700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2e480a1390 0x7f2e480a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.949+0000 7f2e537fe700 1 -- 172.21.15.12:0/2159113120 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2e540cc9b0 con 0x7f2e5c10b3e0 2024-05-09T00:07:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.949+0000 7f2e6217e700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2e480a1390 0x7f2e480a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.950+0000 7f2e6217e700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2e480a1390 0x7f2e480a3850 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2e58005950 tx=0x7f2e58009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:24.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:24.954+0000 7f2e537fe700 1 -- 172.21.15.12:0/2159113120 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2e54095e40 con 0x7f2e5c10b3e0 2024-05-09T00:07:25.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 275292 31.75 MB/sec execute 320 sec latency 182.854 ms 2024-05-09T00:07:25.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.286+0000 7f2e643e2700 1 -- 172.21.15.12:0/2159113120 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e5c069d10 con 0x7f2e5c10b3e0 2024-05-09T00:07:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.287+0000 7f2e537fe700 1 -- 172.21.15.12:0/2159113120 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+10875 (secure 0 0 0) 0x7f2e54099460 con 0x7f2e5c10b3e0 2024-05-09T00:07:25.282 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:07:25.282 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:06:02.725626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_30395":{"gid":30395,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6833/3969500142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3969500142},{"type":"v1","addr":"172.21.15.160:6833","nonce":3969500142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:07:25.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.290+0000 7f2e643e2700 1 -- 172.21.15.12:0/2159113120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2e480a1390 msgr2=0x7f2e480a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.290+0000 7f2e643e2700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2e480a1390 0x7f2e480a3850 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2e58005950 tx=0x7f2e58009310 comp rx=0 tx=0).stop 2024-05-09T00:07:25.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.290+0000 7f2e643e2700 1 -- 172.21.15.12:0/2159113120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e5c10b3e0 msgr2=0x7f2e5c19bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.290+0000 7f2e643e2700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e5c10b3e0 0x7f2e5c19bbf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400efc0 tx=0x7f2e5400c3a0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.291+0000 7f2e643e2700 1 -- 172.21.15.12:0/2159113120 shutdown_connections 2024-05-09T00:07:25.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.291+0000 7f2e643e2700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2e480a1390 0x7f2e480a3850 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.291+0000 7f2e643e2700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e5c10b3e0 0x7f2e5c19bbf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.291+0000 7f2e643e2700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e5c101fc0 0x7f2e5c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.291+0000 7f2e643e2700 1 --2- 172.21.15.12:0/2159113120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e5c1028e0 0x7f2e5c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.291+0000 7f2e643e2700 1 -- 172.21.15.12:0/2159113120 >> 172.21.15.12:0/2159113120 conn(0x7f2e5c0fb800 msgr2=0x7f2e5c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.291+0000 7f2e643e2700 1 -- 172.21.15.12:0/2159113120 shutdown_connections 2024-05-09T00:07:25.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.291+0000 7f2e643e2700 1 -- 172.21.15.12:0/2159113120 wait complete. 2024-05-09T00:07:25.287 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 76 2024-05-09T00:07:25.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:07:25.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.473+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2805664390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a78106290 msgr2=0x7f2a7810de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.473+0000 7f2a7dff6700 1 --2- 172.21.15.12:0/2805664390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a78106290 0x7f2a7810de10 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000b0d0 tx=0x7f2a7000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.474+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2805664390 shutdown_connections 2024-05-09T00:07:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.474+0000 7f2a7dff6700 1 --2- 172.21.15.12:0/2805664390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a78106290 0x7f2a7810de10 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.474+0000 7f2a7dff6700 1 --2- 172.21.15.12:0/2805664390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a780682e0 0x7f2a780686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.474+0000 7f2a7dff6700 1 --2- 172.21.15.12:0/2805664390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a78068c90 0x7f2a78105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.474+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2805664390 >> 172.21.15.12:0/2805664390 conn(0x7f2a78076f40 msgr2=0x7f2a78077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.474+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2805664390 shutdown_connections 2024-05-09T00:07:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.474+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2805664390 wait complete. 2024-05-09T00:07:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.474+0000 7f2a7dff6700 1 Processor -- start 2024-05-09T00:07:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a7dff6700 1 -- start start 2024-05-09T00:07:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a7dff6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a780682e0 0x7f2a780ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a7dff6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a78106290 0x7f2a781003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a7dff6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a781008e0 0x7f2a781a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a7dff6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a5cd0 con 0x7f2a781008e0 2024-05-09T00:07:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a7dff6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a5e40 con 0x7f2a78106290 2024-05-09T00:07:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a7dff6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a6000 con 0x7f2a780682e0 2024-05-09T00:07:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a781008e0 0x7f2a781a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a780682e0 0x7f2a780ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a76ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a78106290 0x7f2a781003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a781008e0 0x7f2a781a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35584/0 (socket says 172.21.15.12:35584) 2024-05-09T00:07:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a780682e0 0x7f2a780ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59862/0 (socket says 172.21.15.12:59862) 2024-05-09T00:07:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a77fff700 1 -- 172.21.15.12:0/2287978081 learned_addr learned my addr 172.21.15.12:0/2287978081 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a777fe700 1 -- 172.21.15.12:0/2287978081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a78106290 msgr2=0x7f2a781003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a777fe700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a78106290 0x7f2a781003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a777fe700 1 -- 172.21.15.12:0/2287978081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a781008e0 msgr2=0x7f2a781a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a777fe700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a781008e0 0x7f2a781a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.475+0000 7f2a777fe700 1 -- 172.21.15.12:0/2287978081 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70009d00 con 0x7f2a780682e0 2024-05-09T00:07:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.476+0000 7f2a777fe700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a780682e0 0x7f2a780ffe60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e9f0 tx=0x7f2a6800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.476+0000 7f2a74ff9700 1 -- 172.21.15.12:0/2287978081 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800c9c0 con 0x7f2a780682e0 2024-05-09T00:07:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.476+0000 7f2a74ff9700 1 -- 172.21.15.12:0/2287978081 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a68010430 con 0x7f2a780682e0 2024-05-09T00:07:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.476+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2287978081 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a781a62e0 con 0x7f2a780682e0 2024-05-09T00:07:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.476+0000 7f2a74ff9700 1 -- 172.21.15.12:0/2287978081 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68018680 con 0x7f2a780682e0 2024-05-09T00:07:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.476+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2287978081 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a781a6840 con 0x7f2a780682e0 2024-05-09T00:07:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.478+0000 7f2a74ff9700 1 -- 172.21.15.12:0/2287978081 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2a68010860 con 0x7f2a780682e0 2024-05-09T00:07:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.480+0000 7f2a8a1b0b80 1 Processor -- start 2024-05-09T00:07:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a74ff9700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a600a1310 0x7f2a600a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a8a1b0b80 1 -- start start 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a74ff9700 1 -- 172.21.15.12:0/2287978081 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2a68014070 con 0x7f2a780682e0 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a77fff700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a600a1310 0x7f2a600a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3ae00 0x55bc9be3b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3b720 0x55bc9be35350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be35890 0x55bc9be35cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bd8c150 con 0x55bc9be35890 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bd89b10 con 0x55bc9be3ae00 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bd85400 con 0x55bc9be3b720 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a76ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3b720 0x55bc9be35350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a76ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3b720 0x55bc9be35350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59864/0 (socket says 172.21.15.12:59864) 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a76ffd700 1 -- 172.21.15.12:0/205170212 learned_addr learned my addr 172.21.15.12:0/205170212 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.481+0000 7f2a77fff700 1 --2- 172.21.15.12:0/205170212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be35890 0x55bc9be35cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a76ffd700 1 -- 172.21.15.12:0/205170212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3ae00 msgr2=0x55bc9be3b1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:07:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a76ffd700 1 --2- 172.21.15.12:0/205170212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3ae00 0x55bc9be3b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a76ffd700 1 -- 172.21.15.12:0/205170212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be35890 msgr2=0x55bc9be35cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a76ffd700 1 --2- 172.21.15.12:0/205170212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be35890 0x55bc9be35cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a76ffd700 1 -- 172.21.15.12:0/205170212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc9bc063b0 con 0x55bc9be3b720 2024-05-09T00:07:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a77fff700 1 --2- 172.21.15.12:0/205170212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be35890 0x55bc9be35cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:07:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a77fff700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a600a1310 0x7f2a600a37d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009fd0 tx=0x7f2a70008bf0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a76ffd700 1 --2- 172.21.15.12:0/205170212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3b720 0x55bc9be35350 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009cb0 tx=0x7f2a6c011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54e8210053fe2956 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a5dffb700 1 -- 172.21.15.12:0/205170212 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c011dd0 con 0x55bc9be3b720 2024-05-09T00:07:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a5dffb700 1 -- 172.21.15.12:0/205170212 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a6c004510 con 0x55bc9be3b720 2024-05-09T00:07:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.482+0000 7f2a5dffb700 1 -- 172.21.15.12:0/205170212 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c005280 con 0x55bc9be3b720 2024-05-09T00:07:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.483+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/205170212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3b720 msgr2=0x55bc9be35350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.483+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/205170212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3b720 0x55bc9be35350 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009cb0 tx=0x7f2a6c011440 comp rx=0 tx=0).stop 2024-05-09T00:07:25.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.483+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/205170212 shutdown_connections 2024-05-09T00:07:25.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.483+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/205170212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be35890 0x55bc9be35cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.483+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/205170212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3ae00 0x55bc9be3b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.484+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/205170212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3b720 0x55bc9be35350 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.484+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/205170212 >> 172.21.15.12:0/205170212 conn(0x55bc9bd93210 msgr2=0x55bc9be2fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.484+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/205170212 shutdown_connections 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.484+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/205170212 wait complete. 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.484+0000 7f2a8a1b0b80 1 -- start start 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a8a1b0b80 1 Processor -- start 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 0x55bc9be3ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3ae00 0x55bc9be3f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bd21560 con 0x55bc9be3ae00 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bd21200 con 0x55bc9be3b720 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bc086a0 con 0x55bc9be35890 2024-05-09T00:07:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a8a1b0b80 10 client.0 _ll_register_callbacks cb 0x55bc9bd2ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:25.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 0x55bc9be3ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56300/0 (socket says 172.21.15.12:56300) 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 0x55bc9be3ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59868/0 (socket says 172.21.15.12:59868) 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.485+0000 7f2a77fff700 1 -- 172.21.15.12:0/2406330611 learned_addr learned my addr 172.21.15.12:0/2406330611 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a76ffd700 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3ae00 0x55bc9be3f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a77fff700 1 -- 172.21.15.12:0/2406330611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 msgr2=0x55bc9be3ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a77fff700 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 0x55bc9be3ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a77fff700 1 -- 172.21.15.12:0/2406330611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3ae00 msgr2=0x55bc9be3f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a77fff700 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3ae00 0x55bc9be3f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a77fff700 1 -- 172.21.15.12:0/2406330611 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6c0098e0 con 0x55bc9be3b720 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a777fe700 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 0x55bc9be3ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a77fff700 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3f670 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d000 tx=0x7f2a70023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a21bceb516e81a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a56ffd700 1 -- 172.21.15.12:0/2406330611 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7000e040 con 0x55bc9be3b720 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/2406330611 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc9bcde270 con 0x55bc9be3b720 2024-05-09T00:07:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.486+0000 7f2a56ffd700 1 -- 172.21.15.12:0/2406330611 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2a70003e10 con 0x55bc9be3b720 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.487+0000 7f2a56ffd700 1 -- 172.21.15.12:0/2406330611 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a70005040 con 0x55bc9be3b720 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.487+0000 7f2a8a1b0b80 10 client.30692 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.487+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/2406330611 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55bc9bbb2b60 con 0x55bc9be3b720 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.487+0000 7f2a56ffd700 1 -- 172.21.15.12:0/2406330611 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc9bcde270 con 0x55bc9be3b720 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.488+0000 7f2a56ffd700 1 -- 172.21.15.12:0/2406330611 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2a7002b850 con 0x55bc9be3b720 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.488+0000 7f2a8a1b0b80 10 client.30692 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.488+0000 7f2a8a1b0b80 10 client.30692 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.488+0000 7f2a8a1b0b80 20 client.30692 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.488+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55bc9be4f390 0x55bc9be517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.488+0000 7f2a8a1b0b80 4 client.30692 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.488+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/2406330611 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc9be04810 con 0x55bc9be4f390 2024-05-09T00:07:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.488+0000 7f2a777fe700 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55bc9be4f390 0x55bc9be517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.489+0000 7f2a777fe700 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55bc9be4f390 0x55bc9be517b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4887e75082a85fc4 server_cookie=28ed757fd7c4e2a2 in_seq=0 out_seq=0 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.489+0000 7f2a56ffd700 10 client.30692 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a56ffd700 1 -- 172.21.15.12:0/2406330611 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc9be04810 con 0x55bc9be4f390 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a56ffd700 10 client.30692 handle_command_reply: tid=0 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 client.30692 shutdown 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/2406330611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 msgr2=0x55bc9be3f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3f670 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d000 tx=0x7f2a70023040 comp rx=0 tx=0).stop 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/2406330611 shutdown_connections 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3f670 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55bc9be4f390 0x55bc9be517b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 0x55bc9be3ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/2406330611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3ae00 0x55bc9be3f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a56ffd700 0 client.30692 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:07:25.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/2406330611 >> 172.21.15.12:0/2406330611 conn(0x55bc9bd93210 msgr2=0x55bc9be2f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/2406330611 shutdown_connections 2024-05-09T00:07:25.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/2406330611 wait complete. 2024-05-09T00:07:25.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.491+0000 7f2a8a1b0b80 20 client.30692 trim_cache size 0 max 16384 2024-05-09T00:07:25.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.510+0000 7f2a8a1b0b80 1 Processor -- start 2024-05-09T00:07:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.510+0000 7f2a8a1b0b80 1 -- start start 2024-05-09T00:07:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.510+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 0x55bc9be35c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.510+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3ae00 0x55bc9be2fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.510+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.510+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9be362d0 con 0x55bc9be3ae00 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.510+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9be4ca30 con 0x55bc9be3b720 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.510+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9be4be20 con 0x55bc9be35890 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.511+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.511+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56316/0 (socket says 172.21.15.12:56316) 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.511+0000 7f2a77fff700 1 -- 172.21.15.12:0/115860313 learned_addr learned my addr 172.21.15.12:0/115860313 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.511+0000 7f2a777fe700 1 --2- 172.21.15.12:0/115860313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 0x55bc9be35c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.511+0000 7f2a77fff700 1 -- 172.21.15.12:0/115860313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 msgr2=0x55bc9be35c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.511+0000 7f2a77fff700 1 --2- 172.21.15.12:0/115860313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 0x55bc9be35c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.511+0000 7f2a77fff700 1 -- 172.21.15.12:0/115860313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3ae00 msgr2=0x55bc9be2fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.511+0000 7f2a77fff700 1 --2- 172.21.15.12:0/115860313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3ae00 0x55bc9be2fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.511+0000 7f2a77fff700 1 -- 172.21.15.12:0/115860313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc9bd4b560 con 0x55bc9be3b720 2024-05-09T00:07:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.511+0000 7f2a77fff700 1 --2- 172.21.15.12:0/115860313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3d4c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000b010 tx=0x7f2a70006220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=608b0d80101c1246 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.512+0000 7f2a5e7fc700 1 -- 172.21.15.12:0/115860313 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7000e070 con 0x55bc9be3b720 2024-05-09T00:07:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.512+0000 7f2a5e7fc700 1 -- 172.21.15.12:0/115860313 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a70009400 con 0x55bc9be3b720 2024-05-09T00:07:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.512+0000 7f2a5e7fc700 1 -- 172.21.15.12:0/115860313 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70005850 con 0x55bc9be3b720 2024-05-09T00:07:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.513+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/115860313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 msgr2=0x55bc9be3d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.513+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/115860313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3d4c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000b010 tx=0x7f2a70006220 comp rx=0 tx=0).stop 2024-05-09T00:07:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.513+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/115860313 shutdown_connections 2024-05-09T00:07:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.513+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/115860313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be3b720 0x55bc9be3d4c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.513+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/115860313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be35890 0x55bc9be35c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.513+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/115860313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3ae00 0x55bc9be2fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.513+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/115860313 >> 172.21.15.12:0/115860313 conn(0x55bc9bd93210 msgr2=0x55bc9be2c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.514+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/115860313 shutdown_connections 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.514+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/115860313 wait complete. 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.514+0000 7f2a8a1b0b80 1 -- start start 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.514+0000 7f2a8a1b0b80 1 Processor -- start 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be35890 0x55bc9be44180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3ae00 0x55bc9be446c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a8a1b0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3b720 0x55bc9be44c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9be46ac0 con 0x55bc9be3b720 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bea4890 con 0x55bc9be35890 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a8a1b0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bea49d0 con 0x55bc9be3ae00 2024-05-09T00:07:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a8a1b0b80 10 client.0 _ll_register_callbacks cb 0x55bc9bd2ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a777fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be35890 0x55bc9be44180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a777fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be35890 0x55bc9be44180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56322/0 (socket says 172.21.15.12:56322) 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a777fe700 1 -- 172.21.15.12:0/3340427755 learned_addr learned my addr 172.21.15.12:0/3340427755 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a76ffd700 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3ae00 0x55bc9be446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a76ffd700 1 -- 172.21.15.12:0/3340427755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be35890 msgr2=0x55bc9be44180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a76ffd700 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be35890 0x55bc9be44180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a76ffd700 1 -- 172.21.15.12:0/3340427755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3b720 msgr2=0x55bc9be44c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a76ffd700 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3b720 0x55bc9be44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a76ffd700 1 -- 172.21.15.12:0/3340427755 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc9bbb2b60 con 0x55bc9be3ae00 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a76ffd700 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3ae00 0x55bc9be446c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009230 tx=0x7f2a6c004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf5a3b0f7c50d273 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.515+0000 7f2a56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a56ffd700 1 -- 172.21.15.12:0/3340427755 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0052d0 con 0x55bc9be3ae00 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/3340427755 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a6c004510 con 0x55bc9be3ae00 2024-05-09T00:07:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a56ffd700 1 -- 172.21.15.12:0/3340427755 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2a6c005430 con 0x55bc9be3ae00 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a56ffd700 1 -- 172.21.15.12:0/3340427755 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a6c005610 con 0x55bc9be3ae00 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a8a1b0b80 10 client.30925 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/3340427755 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55bc9bc0acd0 con 0x55bc9be3ae00 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a56ffd700 1 -- 172.21.15.12:0/3340427755 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a6c004510 con 0x55bc9be3ae00 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a56ffd700 1 -- 172.21.15.12:0/3340427755 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2a6c020070 con 0x55bc9be3ae00 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a8a1b0b80 10 client.30925 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a8a1b0b80 10 client.30925 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a8a1b0b80 20 client.30925 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55bc9bea60f0 0x55bc9bea8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a8a1b0b80 4 client.30925 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.516+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/3340427755 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc9bd7dec0 con 0x55bc9bea60f0 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.517+0000 7f2a777fe700 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55bc9bea60f0 0x55bc9bea8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.517+0000 7f2a56ffd700 10 client.30925 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.517+0000 7f2a777fe700 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55bc9bea60f0 0x55bc9bea8510 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a9e4c4952cd1db5 server_cookie=8a0342cb102188ca in_seq=0 out_seq=0 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a56ffd700 1 -- 172.21.15.12:0/3340427755 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc9bd7dec0 con 0x55bc9bea60f0 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a56ffd700 10 client.30925 handle_command_reply: tid=0 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 client.30925 shutdown 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/3340427755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3ae00 msgr2=0x55bc9be446c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3ae00 0x55bc9be446c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009230 tx=0x7f2a6c004310 comp rx=0 tx=0).stop 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/3340427755 shutdown_connections 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc9be3b720 0x55bc9be44c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bc9be35890 0x55bc9be44180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55bc9bea60f0 0x55bc9bea8510 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 --2- 172.21.15.12:0/3340427755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc9be3ae00 0x55bc9be446c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a56ffd700 0 client.30925 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:07:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/3340427755 >> 172.21.15.12:0/3340427755 conn(0x55bc9bd93210 msgr2=0x55bc9be3d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/3340427755 shutdown_connections 2024-05-09T00:07:25.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 1 -- 172.21.15.12:0/3340427755 wait complete. 2024-05-09T00:07:25.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.518+0000 7f2a8a1b0b80 20 client.30925 trim_cache size 0 max 16384 2024-05-09T00:07:25.520 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:07:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.519+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2287978081 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a600a1310 msgr2=0x7f2a600a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.519+0000 7f2a7dff6700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a600a1310 0x7f2a600a37d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009fd0 tx=0x7f2a70008bf0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.519+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2287978081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a780682e0 msgr2=0x7f2a780ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.519+0000 7f2a7dff6700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a780682e0 0x7f2a780ffe60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e9f0 tx=0x7f2a6800ed00 comp rx=0 tx=0).stop 2024-05-09T00:07:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.521+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2287978081 shutdown_connections 2024-05-09T00:07:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.521+0000 7f2a7dff6700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a600a1310 0x7f2a600a37d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.521+0000 7f2a7dff6700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a781008e0 0x7f2a781a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.521+0000 7f2a7dff6700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a780682e0 0x7f2a780ffe60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.521+0000 7f2a7dff6700 1 --2- 172.21.15.12:0/2287978081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a78106290 0x7f2a781003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.521+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2287978081 >> 172.21.15.12:0/2287978081 conn(0x7f2a78076f40 msgr2=0x7f2a780fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.521+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2287978081 shutdown_connections 2024-05-09T00:07:25.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.521+0000 7f2a7dff6700 1 -- 172.21.15.12:0/2287978081 wait complete. 2024-05-09T00:07:25.536 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:07:25.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:25 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2159113120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.727+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2527455442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc20107fa0 msgr2=0x7fcc20108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.727+0000 7fcc27f3e700 1 --2- 172.21.15.12:0/2527455442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc20107fa0 0x7fcc20108380 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800b0a0 tx=0x7fcc1800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.727+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2527455442 shutdown_connections 2024-05-09T00:07:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.727+0000 7fcc27f3e700 1 --2- 172.21.15.12:0/2527455442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc201140f0 0x7fcc201164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.727+0000 7fcc27f3e700 1 --2- 172.21.15.12:0/2527455442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc20107fa0 0x7fcc20108380 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.727+0000 7fcc27f3e700 1 --2- 172.21.15.12:0/2527455442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc201088c0 0x7fcc20113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.727+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2527455442 >> 172.21.15.12:0/2527455442 conn(0x7fcc2006bff0 msgr2=0x7fcc2006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.727+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2527455442 shutdown_connections 2024-05-09T00:07:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.727+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2527455442 wait complete. 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.728+0000 7fcc27f3e700 1 Processor -- start 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.728+0000 7fcc27f3e700 1 -- start start 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.728+0000 7fcc27f3e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc20107fa0 0x7fcc201a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.728+0000 7fcc27f3e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc201088c0 0x7fcc201a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.728+0000 7fcc27f3e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc201140f0 0x7fcc201a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.728+0000 7fcc27f3e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc201a6ef0 con 0x7fcc201088c0 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.728+0000 7fcc27f3e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc201a7060 con 0x7fcc20107fa0 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.728+0000 7fcc27f3e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc201a7220 con 0x7fcc201140f0 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc25cda700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc20107fa0 0x7fcc201a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc25cda700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc20107fa0 0x7fcc201a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56328/0 (socket says 172.21.15.12:56328) 2024-05-09T00:07:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc25cda700 1 -- 172.21.15.12:0/2834870980 learned_addr learned my addr 172.21.15.12:0/2834870980 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc264db700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc201140f0 0x7fcc201a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc254d9700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc201088c0 0x7fcc201a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc264db700 1 -- 172.21.15.12:0/2834870980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc20107fa0 msgr2=0x7fcc201a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc264db700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc20107fa0 0x7fcc201a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc264db700 1 -- 172.21.15.12:0/2834870980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc201088c0 msgr2=0x7fcc201a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc264db700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc201088c0 0x7fcc201a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc264db700 1 -- 172.21.15.12:0/2834870980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc18009d00 con 0x7fcc201140f0 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc25cda700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc20107fa0 0x7fcc201a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc264db700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc201140f0 0x7fcc201a68b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e940 tx=0x7fcc1c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc12ffd700 1 -- 172.21.15.12:0/2834870980 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c004220 con 0x7fcc201140f0 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc12ffd700 1 -- 172.21.15.12:0/2834870980 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1c004cf0 con 0x7fcc201140f0 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2834870980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc201a7500 con 0x7fcc201140f0 2024-05-09T00:07:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc12ffd700 1 -- 172.21.15.12:0/2834870980 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c005650 con 0x7fcc201140f0 2024-05-09T00:07:25.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.729+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2834870980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc201aed40 con 0x7fcc201140f0 2024-05-09T00:07:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.732+0000 7fcc12ffd700 1 -- 172.21.15.12:0/2834870980 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcc1c004750 con 0x7fcc201140f0 2024-05-09T00:07:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.732+0000 7fcc12ffd700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcc0c0a1310 0x7fcc0c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.733+0000 7fcc25cda700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcc0c0a1310 0x7fcc0c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.733+0000 7fcc12ffd700 1 -- 172.21.15.12:0/2834870980 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcc1c014070 con 0x7fcc201140f0 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.733+0000 7fcc25cda700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcc0c0a1310 0x7fcc0c0a37d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800b780 tx=0x7fcc18009780 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.733+0000 7fcc340f8b80 1 Processor -- start 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc340f8b80 1 -- start start 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b18e00 0x563fe8b191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b13350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b13890 0x563fe8b13cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8a6a150 con 0x563fe8b18e00 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8a67b10 con 0x563fe8b19720 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8a63400 con 0x563fe8b13890 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc254d9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b13350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc254d9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b13350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56332/0 (socket says 172.21.15.12:56332) 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc254d9700 1 -- 172.21.15.12:0/3425810757 learned_addr learned my addr 172.21.15.12:0/3425810757 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc25cda700 1 --2- 172.21.15.12:0/3425810757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b18e00 0x563fe8b191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc264db700 1 --2- 172.21.15.12:0/3425810757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b13890 0x563fe8b13cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc254d9700 1 -- 172.21.15.12:0/3425810757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b13890 msgr2=0x563fe8b13cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc254d9700 1 --2- 172.21.15.12:0/3425810757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b13890 0x563fe8b13cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc254d9700 1 -- 172.21.15.12:0/3425810757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b18e00 msgr2=0x563fe8b191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc254d9700 1 --2- 172.21.15.12:0/3425810757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b18e00 0x563fe8b191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc254d9700 1 -- 172.21.15.12:0/3425810757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fe88e43b0 con 0x563fe8b19720 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.734+0000 7fcc25cda700 1 --2- 172.21.15.12:0/3425810757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b18e00 0x563fe8b191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.735+0000 7fcc254d9700 1 --2- 172.21.15.12:0/3425810757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b13350 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009ac0 tx=0x7fcc14009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=896818d3fcce01c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.735+0000 7fcc0bfff700 1 -- 172.21.15.12:0/3425810757 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14009590 con 0x563fe8b19720 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.735+0000 7fcc0bfff700 1 -- 172.21.15.12:0/3425810757 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1400ed90 con 0x563fe8b19720 2024-05-09T00:07:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.735+0000 7fcc0bfff700 1 -- 172.21.15.12:0/3425810757 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc140123d0 con 0x563fe8b19720 2024-05-09T00:07:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.736+0000 7fcc340f8b80 1 -- 172.21.15.12:0/3425810757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 msgr2=0x563fe8b13350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.736+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/3425810757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b13350 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009ac0 tx=0x7fcc14009dd0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.736+0000 7fcc340f8b80 1 -- 172.21.15.12:0/3425810757 shutdown_connections 2024-05-09T00:07:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.736+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/3425810757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b13890 0x563fe8b13cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.736+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/3425810757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b18e00 0x563fe8b191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.736+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/3425810757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b13350 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.736+0000 7fcc340f8b80 1 -- 172.21.15.12:0/3425810757 >> 172.21.15.12:0/3425810757 conn(0x563fe8a71210 msgr2=0x563fe8b0ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.736+0000 7fcc340f8b80 1 -- 172.21.15.12:0/3425810757 shutdown_connections 2024-05-09T00:07:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.736+0000 7fcc340f8b80 1 -- 172.21.15.12:0/3425810757 wait complete. 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.736+0000 7fcc340f8b80 1 -- start start 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc340f8b80 1 Processor -- start 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 0x563fe8b1cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 0x563fe8b1d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b1d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe89ff560 con 0x563fe8b13890 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe89ff200 con 0x563fe8b19720 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe88e66a0 con 0x563fe8b18e00 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc340f8b80 10 client.0 _ll_register_callbacks cb 0x563fe8a08b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc25cda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 0x563fe8b1cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc264db700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b1d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc25cda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 0x563fe8b1cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35650/0 (socket says 172.21.15.12:35650) 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc264db700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b1d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56338/0 (socket says 172.21.15.12:56338) 2024-05-09T00:07:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.737+0000 7fcc25cda700 1 -- 172.21.15.12:0/619685934 learned_addr learned my addr 172.21.15.12:0/619685934 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc254d9700 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 0x563fe8b1d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc264db700 1 -- 172.21.15.12:0/619685934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 msgr2=0x563fe8b1d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc264db700 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 0x563fe8b1d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc264db700 1 -- 172.21.15.12:0/619685934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 msgr2=0x563fe8b1cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc264db700 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 0x563fe8b1cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc264db700 1 -- 172.21.15.12:0/619685934 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc20116eb0 con 0x563fe8b19720 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc264db700 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b1d670 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0ca190 tx=0x7fcc1c01ff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa1f388727198b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc097fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc097fa700 1 -- 172.21.15.12:0/619685934 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0b1070 con 0x563fe8b19720 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc340f8b80 1 -- 172.21.15.12:0/619685934 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fe89bc270 con 0x563fe8b19720 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.738+0000 7fcc097fa700 1 -- 172.21.15.12:0/619685934 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcc1c09d980 con 0x563fe8b19720 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc097fa700 1 -- 172.21.15.12:0/619685934 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc1c0a1fd0 con 0x563fe8b19720 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc340f8b80 10 client.30710 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc340f8b80 1 -- 172.21.15.12:0/619685934 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7fcc140096f0 con 0x563fe8b19720 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc097fa700 1 -- 172.21.15.12:0/619685934 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563fe89bc270 con 0x563fe8b19720 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc097fa700 1 -- 172.21.15.12:0/619685934 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcc1c0a1b40 con 0x563fe8b19720 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc340f8b80 10 client.30710 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc340f8b80 10 client.30710 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc340f8b80 20 client.30710 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563fe8b2d1b0 0x563fe8b2f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc340f8b80 4 client.30710 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.739+0000 7fcc340f8b80 1 -- 172.21.15.12:0/619685934 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fe8ae2810 con 0x563fe8b2d1b0 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.740+0000 7fcc25cda700 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563fe8b2d1b0 0x563fe8b2f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.740+0000 7fcc097fa700 10 client.30710 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:07:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.740+0000 7fcc25cda700 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563fe8b2d1b0 0x563fe8b2f5a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3228ca715f66224 server_cookie=c56725c1d8264968 in_seq=0 out_seq=0 2024-05-09T00:07:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc097fa700 1 -- 172.21.15.12:0/619685934 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fe8ae2810 con 0x563fe8b2d1b0 2024-05-09T00:07:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc097fa700 10 client.30710 handle_command_reply: tid=0 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 client.30710 shutdown 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 -- 172.21.15.12:0/619685934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 msgr2=0x563fe8b1d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b1d670 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0ca190 tx=0x7fcc1c01ff50 comp rx=0 tx=0).stop 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 -- 172.21.15.12:0/619685934 shutdown_connections 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563fe8b2d1b0 0x563fe8b2f5a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b19720 0x563fe8b1d670 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 0x563fe8b1cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/619685934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 0x563fe8b1d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc097fa700 0 client.30710 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 -- 172.21.15.12:0/619685934 >> 172.21.15.12:0/619685934 conn(0x563fe8a71210 msgr2=0x563fe8b0d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 -- 172.21.15.12:0/619685934 shutdown_connections 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 1 -- 172.21.15.12:0/619685934 wait complete. 2024-05-09T00:07:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.743+0000 7fcc340f8b80 20 client.30710 trim_cache size 0 max 16384 2024-05-09T00:07:25.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.759+0000 7fcc340f8b80 1 Processor -- start 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.759+0000 7fcc340f8b80 1 -- start start 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.759+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b13890 0x563fe8b2f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.759+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 0x563fe8b28c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.759+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b19720 0x563fe8b0dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.759+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8b0fea0 con 0x563fe8b19720 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.759+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8b347b0 con 0x563fe8b13890 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.759+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8b0fa30 con 0x563fe8b18e00 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.759+0000 7fcc264db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b19720 0x563fe8b0dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc264db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b19720 0x563fe8b0dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35654/0 (socket says 172.21.15.12:35654) 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc254d9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 0x563fe8b28c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc264db700 1 -- 172.21.15.12:0/1121111272 learned_addr learned my addr 172.21.15.12:0/1121111272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc25cda700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b13890 0x563fe8b2f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc264db700 1 -- 172.21.15.12:0/1121111272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 msgr2=0x563fe8b28c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc264db700 1 --2- 172.21.15.12:0/1121111272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 0x563fe8b28c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc264db700 1 -- 172.21.15.12:0/1121111272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b13890 msgr2=0x563fe8b2f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc264db700 1 --2- 172.21.15.12:0/1121111272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b13890 0x563fe8b2f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc264db700 1 -- 172.21.15.12:0/1121111272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fe8890b60 con 0x563fe8b19720 2024-05-09T00:07:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc254d9700 1 --2- 172.21.15.12:0/1121111272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 0x563fe8b28c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc264db700 1 --2- 172.21.15.12:0/1121111272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b19720 0x563fe8b0dd20 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009fd0 tx=0x7fcc1c099cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8e5731cdc4285f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc10ff9700 1 -- 172.21.15.12:0/1121111272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0b1070 con 0x563fe8b19720 2024-05-09T00:07:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc10ff9700 1 -- 172.21.15.12:0/1121111272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1c0b9070 con 0x563fe8b19720 2024-05-09T00:07:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.760+0000 7fcc10ff9700 1 -- 172.21.15.12:0/1121111272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c09a740 con 0x563fe8b19720 2024-05-09T00:07:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.761+0000 7fcc340f8b80 1 -- 172.21.15.12:0/1121111272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b19720 msgr2=0x563fe8b0dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.761+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/1121111272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b19720 0x563fe8b0dd20 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009fd0 tx=0x7fcc1c099cd0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.761+0000 7fcc340f8b80 1 -- 172.21.15.12:0/1121111272 shutdown_connections 2024-05-09T00:07:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.761+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/1121111272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b19720 0x563fe8b0dd20 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.761+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/1121111272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b13890 0x563fe8b2f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.761+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/1121111272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b18e00 0x563fe8b28c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.761+0000 7fcc340f8b80 1 -- 172.21.15.12:0/1121111272 >> 172.21.15.12:0/1121111272 conn(0x563fe8a71210 msgr2=0x563fe8b0a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.761+0000 7fcc340f8b80 1 -- 172.21.15.12:0/1121111272 shutdown_connections 2024-05-09T00:07:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.761+0000 7fcc340f8b80 1 -- 172.21.15.12:0/1121111272 wait complete. 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc340f8b80 1 -- start start 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc340f8b80 1 Processor -- start 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 0x563fe8b24360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b18e00 0x563fe8b22150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc340f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b19720 0x563fe8b22690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8b22bd0 con 0x563fe8b13890 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8b22d10 con 0x563fe8b18e00 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc340f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8b22ed0 con 0x563fe8b19720 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc340f8b80 10 client.0 _ll_register_callbacks cb 0x563fe8a08b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc254d9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b18e00 0x563fe8b22150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc254d9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b18e00 0x563fe8b22150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56364/0 (socket says 172.21.15.12:56364) 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc25cda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 0x563fe8b24360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc254d9700 1 -- 172.21.15.12:0/2352773711 learned_addr learned my addr 172.21.15.12:0/2352773711 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.762+0000 7fcc264db700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b19720 0x563fe8b22690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.763+0000 7fcc264db700 1 -- 172.21.15.12:0/2352773711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b18e00 msgr2=0x563fe8b22150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.763+0000 7fcc264db700 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b18e00 0x563fe8b22150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.763+0000 7fcc264db700 1 -- 172.21.15.12:0/2352773711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 msgr2=0x563fe8b24360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.763+0000 7fcc264db700 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 0x563fe8b24360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.763+0000 7fcc264db700 1 -- 172.21.15.12:0/2352773711 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fe88e43b0 con 0x563fe8b19720 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.763+0000 7fcc264db700 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b19720 0x563fe8b22690 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c099850 tx=0x7fcc1c095fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4acbd45deccf2e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.763+0000 7fcc097fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.763+0000 7fcc097fa700 1 -- 172.21.15.12:0/2352773711 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0b1070 con 0x563fe8b19720 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.763+0000 7fcc097fa700 1 -- 172.21.15.12:0/2352773711 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcc1c09a250 con 0x563fe8b19720 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc340f8b80 1 -- 172.21.15.12:0/2352773711 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc1400ed90 con 0x563fe8b19720 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc097fa700 1 -- 172.21.15.12:0/2352773711 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc1c01fe60 con 0x563fe8b19720 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc340f8b80 10 client.30943 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc340f8b80 1 -- 172.21.15.12:0/2352773711 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7fcc140096f0 con 0x563fe8b19720 2024-05-09T00:07:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc097fa700 1 -- 172.21.15.12:0/2352773711 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1400ed90 con 0x563fe8b19720 2024-05-09T00:07:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc097fa700 1 -- 172.21.15.12:0/2352773711 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcc1c09ad20 con 0x563fe8b19720 2024-05-09T00:07:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc340f8b80 10 client.30943 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc340f8b80 10 client.30943 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:07:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc340f8b80 20 client.30943 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563fe8b83690 0x563fe8b85a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc340f8b80 4 client.30943 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:07:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.764+0000 7fcc340f8b80 1 -- 172.21.15.12:0/2352773711 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563fe8a5bec0 con 0x563fe8b83690 2024-05-09T00:07:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.765+0000 7fcc25cda700 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563fe8b83690 0x563fe8b85a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:25 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2159113120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.765+0000 7fcc097fa700 10 client.30943 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:07:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.765+0000 7fcc25cda700 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563fe8b83690 0x563fe8b85a80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b7a76f31c455aba server_cookie=27de83a063adda17 in_seq=0 out_seq=0 2024-05-09T00:07:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.766+0000 7fcc097fa700 1 -- 172.21.15.12:0/2352773711 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563fe8a5bec0 con 0x563fe8b83690 2024-05-09T00:07:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.766+0000 7fcc097fa700 10 client.30943 handle_command_reply: tid=0 2024-05-09T00:07:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.766+0000 7fcc340f8b80 1 client.30943 shutdown 2024-05-09T00:07:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 1 -- 172.21.15.12:0/2352773711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b19720 msgr2=0x563fe8b22690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b19720 0x563fe8b22690 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c099850 tx=0x7fcc1c095fa0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 1 -- 172.21.15.12:0/2352773711 shutdown_connections 2024-05-09T00:07:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x563fe8b83690 0x563fe8b85a80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fe8b19720 0x563fe8b22690 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fe8b13890 0x563fe8b24360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc097fa700 0 client.30943 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:07:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 1 --2- 172.21.15.12:0/2352773711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fe8b18e00 0x563fe8b22150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 1 -- 172.21.15.12:0/2352773711 >> 172.21.15.12:0/2352773711 conn(0x563fe8a71210 msgr2=0x563fe8b1b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 1 -- 172.21.15.12:0/2352773711 shutdown_connections 2024-05-09T00:07:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 1 -- 172.21.15.12:0/2352773711 wait complete. 2024-05-09T00:07:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.767+0000 7fcc340f8b80 20 client.30943 trim_cache size 0 max 16384 2024-05-09T00:07:25.766 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:07:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2834870980 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcc0c0a1310 msgr2=0x7fcc0c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcc0c0a1310 0x7fcc0c0a37d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800b780 tx=0x7fcc18009780 comp rx=0 tx=0).stop 2024-05-09T00:07:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2834870980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc201140f0 msgr2=0x7fcc201a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc201140f0 0x7fcc201a68b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e940 tx=0x7fcc1c00ec50 comp rx=0 tx=0).stop 2024-05-09T00:07:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2834870980 shutdown_connections 2024-05-09T00:07:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcc0c0a1310 0x7fcc0c0a37d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc201140f0 0x7fcc201a68b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc20107fa0 0x7fcc201a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 --2- 172.21.15.12:0/2834870980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc201088c0 0x7fcc201a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2834870980 >> 172.21.15.12:0/2834870980 conn(0x7fcc2006bff0 msgr2=0x7fcc201099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2834870980 shutdown_connections 2024-05-09T00:07:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.768+0000 7fcc27f3e700 1 -- 172.21.15.12:0/2834870980 wait complete. 2024-05-09T00:07:25.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:25 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2159113120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:25.780 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:07:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.988+0000 7f828c883700 1 -- 172.21.15.12:0/2187810259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8288101b10 msgr2=0x7f8288105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.988+0000 7f828c883700 1 --2- 172.21.15.12:0/2187810259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8288101b10 0x7f8288105b50 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f827c009990 tx=0x7f827c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.988+0000 7f828c883700 1 -- 172.21.15.12:0/2187810259 shutdown_connections 2024-05-09T00:07:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.988+0000 7f828c883700 1 --2- 172.21.15.12:0/2187810259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288106280 0x7f828810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.988+0000 7f828c883700 1 --2- 172.21.15.12:0/2187810259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8288101160 0x7f8288101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.988+0000 7f828c883700 1 --2- 172.21.15.12:0/2187810259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8288101b10 0x7f8288105b50 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.988+0000 7f828c883700 1 -- 172.21.15.12:0/2187810259 >> 172.21.15.12:0/2187810259 conn(0x7f82880fc9e0 msgr2=0x7f82880fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.988+0000 7f828c883700 1 -- 172.21.15.12:0/2187810259 shutdown_connections 2024-05-09T00:07:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.988+0000 7f828c883700 1 -- 172.21.15.12:0/2187810259 wait complete. 2024-05-09T00:07:25.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.989+0000 7f828c883700 1 Processor -- start 2024-05-09T00:07:25.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.989+0000 7f828c883700 1 -- start start 2024-05-09T00:07:25.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.989+0000 7f828c883700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8288101160 0x7f82880732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.989+0000 7f828c883700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8288106280 0x7f82880737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.989+0000 7f828c883700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288073d20 0x7f82881a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.989+0000 7f828c883700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881a5b70 con 0x7f8288106280 2024-05-09T00:07:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.989+0000 7f828c883700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881a5ce0 con 0x7f8288101160 2024-05-09T00:07:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.989+0000 7f828c883700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881a5ea0 con 0x7f8288073d20 2024-05-09T00:07:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f828659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8288101160 0x7f82880732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f828659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8288101160 0x7f82880732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56380/0 (socket says 172.21.15.12:56380) 2024-05-09T00:07:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f828659c700 1 -- 172.21.15.12:0/2929289756 learned_addr learned my addr 172.21.15.12:0/2929289756 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f8285d9b700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8288106280 0x7f82880737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f8286d9d700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288073d20 0x7f82881a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f828659c700 1 -- 172.21.15.12:0/2929289756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288073d20 msgr2=0x7f82881a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f828659c700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288073d20 0x7f82881a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f828659c700 1 -- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8288106280 msgr2=0x7f82880737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f828659c700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8288106280 0x7f82880737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f828659c700 1 -- 172.21.15.12:0/2929289756 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0095f0 con 0x7f8288101160 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f8285d9b700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8288106280 0x7f82880737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f8286d9d700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288073d20 0x7f82881a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f828659c700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8288101160 0x7f82880732a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f827800e9a0 tx=0x7f827800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f82777fe700 1 -- 172.21.15.12:0/2929289756 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278009790 con 0x7f8288101160 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.990+0000 7f828c883700 1 -- 172.21.15.12:0/2929289756 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82881a6180 con 0x7f8288101160 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.991+0000 7f82777fe700 1 -- 172.21.15.12:0/2929289756 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8278010430 con 0x7f8288101160 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.991+0000 7f82777fe700 1 -- 172.21.15.12:0/2929289756 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278018680 con 0x7f8288101160 2024-05-09T00:07:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.991+0000 7f828c883700 1 -- 172.21.15.12:0/2929289756 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82881a6660 con 0x7f8288101160 2024-05-09T00:07:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.994+0000 7f82777fe700 1 -- 172.21.15.12:0/2929289756 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f82780105a0 con 0x7f8288101160 2024-05-09T00:07:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.995+0000 7f82777fe700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82700a1290 0x7f82700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.995+0000 7f82777fe700 1 -- 172.21.15.12:0/2929289756 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8278014070 con 0x7f8288101160 2024-05-09T00:07:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.995+0000 7f8285d9b700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82700a1290 0x7f82700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.996+0000 7f8298a3db80 1 Processor -- start 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.996+0000 7f8298a3db80 1 -- start start 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.996+0000 7f8285d9b700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82700a1290 0x7f82700a3750 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f827c00b2e0 tx=0x7f827c005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.996+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.996+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 0x55d0443e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.996+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443e6890 0x55d0443e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.996+0000 7f8298a3db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04433d150 con 0x55d0443ebe00 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.996+0000 7f8298a3db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04433ab10 con 0x55d0443e6890 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.996+0000 7f8298a3db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d044336400 con 0x55d0443ec720 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.996+0000 7f828659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f828659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443ec1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35688/0 (socket says 172.21.15.12:35688) 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8285d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 0x55d0443e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f828659c700 1 -- 172.21.15.12:0/365387359 learned_addr learned my addr 172.21.15.12:0/365387359 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8286d9d700 1 --2- 172.21.15.12:0/365387359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443e6890 0x55d0443e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8285d9b700 1 -- 172.21.15.12:0/365387359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443e6890 msgr2=0x55d0443e6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8285d9b700 1 --2- 172.21.15.12:0/365387359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443e6890 0x55d0443e6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8285d9b700 1 -- 172.21.15.12:0/365387359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 msgr2=0x55d0443ec1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8285d9b700 1 --2- 172.21.15.12:0/365387359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443ec1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8285d9b700 1 -- 172.21.15.12:0/365387359 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0441b73b0 con 0x55d0443ec720 2024-05-09T00:07:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f828659c700 1 --2- 172.21.15.12:0/365387359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8285d9b700 1 --2- 172.21.15.12:0/365387359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 0x55d0443e6350 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f827c006010 tx=0x7f827c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfcf675613d7f472 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8274ff9700 1 -- 172.21.15.12:0/365387359 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c02b070 con 0x55d0443ec720 2024-05-09T00:07:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8274ff9700 1 -- 172.21.15.12:0/365387359 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827c01e040 con 0x55d0443ec720 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.997+0000 7f8274ff9700 1 -- 172.21.15.12:0/365387359 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c004830 con 0x55d0443ec720 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.998+0000 7f8298a3db80 1 -- 172.21.15.12:0/365387359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 msgr2=0x55d0443e6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.998+0000 7f8298a3db80 1 --2- 172.21.15.12:0/365387359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 0x55d0443e6350 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f827c006010 tx=0x7f827c020000 comp rx=0 tx=0).stop 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.998+0000 7f8298a3db80 1 -- 172.21.15.12:0/365387359 shutdown_connections 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.998+0000 7f8298a3db80 1 --2- 172.21.15.12:0/365387359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443e6890 0x55d0443e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.998+0000 7f8298a3db80 1 --2- 172.21.15.12:0/365387359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.998+0000 7f8298a3db80 1 --2- 172.21.15.12:0/365387359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 0x55d0443e6350 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.998+0000 7f8298a3db80 1 -- 172.21.15.12:0/365387359 >> 172.21.15.12:0/365387359 conn(0x55d044344210 msgr2=0x55d0443e0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.998+0000 7f8298a3db80 1 -- 172.21.15.12:0/365387359 shutdown_connections 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.998+0000 7f8298a3db80 1 -- 172.21.15.12:0/365387359 wait complete. 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f8298a3db80 1 -- start start 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f8298a3db80 1 Processor -- start 2024-05-09T00:07:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443e6890 0x55d0443efb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ebe00 0x55d0443f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 0x55d0443f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f8298a3db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0442d2560 con 0x55d0443e6890 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f8298a3db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0442d2200 con 0x55d0443ebe00 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f8298a3db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0441b96a0 con 0x55d0443ec720 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f8298a3db80 10 client.0 _ll_register_callbacks cb 0x55d0442dbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f828659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443e6890 0x55d0443efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f828659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443e6890 0x55d0443efb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35704/0 (socket says 172.21.15.12:35704) 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f828659c700 1 -- 172.21.15.12:0/3461693463 learned_addr learned my addr 172.21.15.12:0/3461693463 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:25.999+0000 7f8285d9b700 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ebe00 0x55d0443f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f8286d9d700 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 0x55d0443f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f828659c700 1 -- 172.21.15.12:0/3461693463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 msgr2=0x55d0443f05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f828659c700 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 0x55d0443f05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f828659c700 1 -- 172.21.15.12:0/3461693463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ebe00 msgr2=0x55d0443f00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f828659c700 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ebe00 0x55d0443f00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f828659c700 1 -- 172.21.15.12:0/3461693463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82881a6660 con 0x55d0443e6890 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f8285d9b700 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ebe00 0x55d0443f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f828659c700 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443e6890 0x55d0443efb60 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f82780cac00 tx=0x7f827809a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d076c4f30920ad90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f826dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f826dffb700 1 -- 172.21.15.12:0/3461693463 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780996b0 con 0x55d0443e6890 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f826dffb700 1 -- 172.21.15.12:0/3461693463 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f827809ddf0 con 0x55d0443e6890 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f8298a3db80 1 -- 172.21.15.12:0/3461693463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d04428f270 con 0x55d0443e6890 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.000+0000 7f826dffb700 1 -- 172.21.15.12:0/3461693463 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82780a2440 con 0x55d0443e6890 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f8298a3db80 10 client.21231 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f8298a3db80 1 -- 172.21.15.12:0/3461693463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7f828810e670 con 0x55d0443e6890 2024-05-09T00:07:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f826dffb700 1 -- 172.21.15.12:0/3461693463 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d04428f270 con 0x55d0443e6890 2024-05-09T00:07:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f826dffb700 1 -- 172.21.15.12:0/3461693463 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f82780a1fb0 con 0x55d0443e6890 2024-05-09T00:07:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f8298a3db80 10 client.21231 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f8298a3db80 10 client.21231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f8298a3db80 20 client.21231 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f8298a3db80 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d044400220 0x55d044402640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f8298a3db80 4 client.21231 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f8298a3db80 1 -- 172.21.15.12:0/3461693463 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0443b5810 con 0x55d044400220 2024-05-09T00:07:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.001+0000 7f8285d9b700 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d044400220 0x55d044402640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.002+0000 7f8285d9b700 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d044400220 0x55d044402640 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7795cbc13892ad58 server_cookie=8eae85ee0cfddeb2 in_seq=0 out_seq=0 2024-05-09T00:07:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.002+0000 7f826dffb700 10 client.21231 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.006+0000 7f826dffb700 1 -- 172.21.15.12:0/3461693463 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0443b5810 con 0x55d044400220 2024-05-09T00:07:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.006+0000 7f826dffb700 10 client.21231 handle_command_reply: tid=0 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 client.21231 shutdown 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 -- 172.21.15.12:0/3461693463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443e6890 msgr2=0x55d0443efb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443e6890 0x55d0443efb60 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f82780cac00 tx=0x7f827809a7b0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 -- 172.21.15.12:0/3461693463 shutdown_connections 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ec720 0x55d0443f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443e6890 0x55d0443efb60 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d044400220 0x55d044402640 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 --2- 172.21.15.12:0/3461693463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ebe00 0x55d0443f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f826dffb700 0 client.21231 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 -- 172.21.15.12:0/3461693463 >> 172.21.15.12:0/3461693463 conn(0x55d044344210 msgr2=0x55d0443e05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 -- 172.21.15.12:0/3461693463 shutdown_connections 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 1 -- 172.21.15.12:0/3461693463 wait complete. 2024-05-09T00:07:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.007+0000 7f8298a3db80 20 client.21231 trim_cache size 0 max 16384 2024-05-09T00:07:26.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8298a3db80 1 Processor -- start 2024-05-09T00:07:26.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8298a3db80 1 -- start start 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443e6890 0x55d0443e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ec720 0x55d0443f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8298a3db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0443f1180 con 0x55d0443ebe00 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8298a3db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0443dddc0 con 0x55d0443ec720 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8298a3db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0443f2b60 con 0x55d0443e6890 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8285d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8285d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443e0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35706/0 (socket says 172.21.15.12:35706) 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8285d9b700 1 -- 172.21.15.12:0/2970595816 learned_addr learned my addr 172.21.15.12:0/2970595816 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f828659c700 1 --2- 172.21.15.12:0/2970595816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443e6890 0x55d0443e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.023+0000 7f8286d9d700 1 --2- 172.21.15.12:0/2970595816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ec720 0x55d0443f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.024+0000 7f8285d9b700 1 -- 172.21.15.12:0/2970595816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443e6890 msgr2=0x55d0443e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.024+0000 7f8285d9b700 1 --2- 172.21.15.12:0/2970595816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443e6890 0x55d0443e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.024+0000 7f8285d9b700 1 -- 172.21.15.12:0/2970595816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ec720 msgr2=0x55d0443f0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.024+0000 7f8285d9b700 1 --2- 172.21.15.12:0/2970595816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ec720 0x55d0443f0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.024+0000 7f8285d9b700 1 -- 172.21.15.12:0/2970595816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d044163b60 con 0x55d0443ebe00 2024-05-09T00:07:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.024+0000 7f8285d9b700 1 --2- 172.21.15.12:0/2970595816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443e0d20 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f827c0059d0 tx=0x7f827c00fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d534519f670a406 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.024+0000 7f82757fa700 1 -- 172.21.15.12:0/2970595816 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c034070 con 0x55d0443ebe00 2024-05-09T00:07:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.024+0000 7f82757fa700 1 -- 172.21.15.12:0/2970595816 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827c035040 con 0x55d0443ebe00 2024-05-09T00:07:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.024+0000 7f82757fa700 1 -- 172.21.15.12:0/2970595816 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c038a90 con 0x55d0443ebe00 2024-05-09T00:07:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 -- 172.21.15.12:0/2970595816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 msgr2=0x55d0443e0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 --2- 172.21.15.12:0/2970595816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443e0d20 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f827c0059d0 tx=0x7f827c00fed0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 -- 172.21.15.12:0/2970595816 shutdown_connections 2024-05-09T00:07:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 --2- 172.21.15.12:0/2970595816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443ec720 0x55d0443f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 --2- 172.21.15.12:0/2970595816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443e6890 0x55d0443e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 --2- 172.21.15.12:0/2970595816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ebe00 0x55d0443e0d20 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 -- 172.21.15.12:0/2970595816 >> 172.21.15.12:0/2970595816 conn(0x55d044344210 msgr2=0x55d0443ee430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 -- 172.21.15.12:0/2970595816 shutdown_connections 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 -- 172.21.15.12:0/2970595816 wait complete. 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 -- start start 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.025+0000 7f8298a3db80 1 Processor -- start 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443e6890 0x55d0443f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ebe00 0x55d0443f3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8298a3db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ec720 0x55d0443f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8298a3db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0443f3b70 con 0x55d0443ec720 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8298a3db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0443f3ce0 con 0x55d0443e6890 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8298a3db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0443f3ea0 con 0x55d0443ebe00 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8298a3db80 10 client.0 _ll_register_callbacks cb 0x55d0442dbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8286d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ec720 0x55d0443f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8285d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ebe00 0x55d0443f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f828659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443e6890 0x55d0443f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8285d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ebe00 0x55d0443f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:59982/0 (socket says 172.21.15.12:59982) 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8285d9b700 1 -- 172.21.15.12:0/747837770 learned_addr learned my addr 172.21.15.12:0/747837770 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8285d9b700 1 -- 172.21.15.12:0/747837770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443e6890 msgr2=0x55d0443f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8285d9b700 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443e6890 0x55d0443f72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8285d9b700 1 -- 172.21.15.12:0/747837770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ec720 msgr2=0x55d0443f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8285d9b700 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ec720 0x55d0443f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.026+0000 7f8285d9b700 1 -- 172.21.15.12:0/747837770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0441b73b0 con 0x55d0443ebe00 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f8285d9b700 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ebe00 0x55d0443f3080 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f827c009ce0 tx=0x7f827c00faa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c98be69a5974d5bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f826dffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:07:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f826dffb700 1 -- 172.21.15.12:0/747837770 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c034040 con 0x55d0443ebe00 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f826dffb700 1 -- 172.21.15.12:0/747837770 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f827c02fbc0 con 0x55d0443ebe00 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f8298a3db80 1 -- 172.21.15.12:0/747837770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f827c01e040 con 0x55d0443ebe00 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f826dffb700 1 -- 172.21.15.12:0/747837770 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f827c02b040 con 0x55d0443ebe00 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f8298a3db80 10 client.30961 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f8298a3db80 1 -- 172.21.15.12:0/747837770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55d0442fc560 con 0x55d0443ebe00 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f826dffb700 1 -- 172.21.15.12:0/747837770 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827c01e040 con 0x55d0443ebe00 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f826dffb700 1 -- 172.21.15.12:0/747837770 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f827c0415c0 con 0x55d0443ebe00 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f8298a3db80 10 client.30961 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f8298a3db80 10 client.30961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f8298a3db80 20 client.30961 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f8298a3db80 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d044456820 0x55d044458c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f8298a3db80 4 client.30961 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.027+0000 7f8298a3db80 1 -- 172.21.15.12:0/747837770 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d04432eec0 con 0x55d044456820 2024-05-09T00:07:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.028+0000 7f828659c700 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d044456820 0x55d044458c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.028+0000 7f826dffb700 10 client.30961 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.028+0000 7f828659c700 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d044456820 0x55d044458c10 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7242efaec8019309 server_cookie=b9d49a8d829136b in_seq=0 out_seq=0 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.029+0000 7f826dffb700 1 -- 172.21.15.12:0/747837770 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d04432eec0 con 0x55d044456820 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.029+0000 7f826dffb700 10 client.30961 handle_command_reply: tid=0 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.029+0000 7f8298a3db80 1 client.30961 shutdown 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.029+0000 7f8298a3db80 1 -- 172.21.15.12:0/747837770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ebe00 msgr2=0x55d0443f3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.029+0000 7f8298a3db80 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ebe00 0x55d0443f3080 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f827c009ce0 tx=0x7f827c00faa0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.030+0000 7f8298a3db80 1 -- 172.21.15.12:0/747837770 shutdown_connections 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.030+0000 7f8298a3db80 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0443ec720 0x55d0443f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.030+0000 7f8298a3db80 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d044456820 0x55d044458c10 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.030+0000 7f8298a3db80 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d0443e6890 0x55d0443f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.030+0000 7f8298a3db80 1 --2- 172.21.15.12:0/747837770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0443ebe00 0x55d0443f3080 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.030+0000 7f826dffb700 0 client.30961 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.030+0000 7f8298a3db80 1 -- 172.21.15.12:0/747837770 >> 172.21.15.12:0/747837770 conn(0x55d044344210 msgr2=0x55d0443eac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.030+0000 7f8298a3db80 1 -- 172.21.15.12:0/747837770 shutdown_connections 2024-05-09T00:07:26.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.030+0000 7f8298a3db80 1 -- 172.21.15.12:0/747837770 wait complete. 2024-05-09T00:07:26.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.030+0000 7f8298a3db80 20 client.30961 trim_cache size 0 max 16384 2024-05-09T00:07:26.030 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 -- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82700a1290 msgr2=0x7f82700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82700a1290 0x7f82700a3750 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f827c00b2e0 tx=0x7f827c005960 comp rx=0 tx=0).stop 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 -- 172.21.15.12:0/2929289756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8288101160 msgr2=0x7f82880732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8288101160 0x7f82880732a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f827800e9a0 tx=0x7f827800ecb0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 -- 172.21.15.12:0/2929289756 shutdown_connections 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82700a1290 0x7f82700a3750 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288073d20 0x7f82881a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8288101160 0x7f82880732a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 --2- 172.21.15.12:0/2929289756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8288106280 0x7f82880737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 -- 172.21.15.12:0/2929289756 >> 172.21.15.12:0/2929289756 conn(0x7f82880fc9e0 msgr2=0x7f82880fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 -- 172.21.15.12:0/2929289756 shutdown_connections 2024-05-09T00:07:26.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:26.032+0000 7f828c883700 1 -- 172.21.15.12:0/2929289756 wait complete. 2024-05-09T00:07:26.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 275700 31.70 MB/sec execute 321 sec latency 108.080 ms 2024-05-09T00:07:26.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:26 smithi012 ceph-mon[42503]: pgmap v676: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 619 op/s 2024-05-09T00:07:26.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:26 smithi204 ceph-mon[47373]: pgmap v676: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 619 op/s 2024-05-09T00:07:26.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:26 smithi160 ceph-mon[46152]: pgmap v676: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 619 op/s 2024-05-09T00:07:27.044 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:07:27.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:07:27.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.245+0000 7fa23be96700 1 -- 172.21.15.12:0/799978396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2341140f0 msgr2=0x7fa2341164e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.245+0000 7fa23be96700 1 --2- 172.21.15.12:0/799978396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2341140f0 0x7fa2341164e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009fc0 tx=0x7fa22c00b330 comp rx=0 tx=0).stop 2024-05-09T00:07:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.246+0000 7fa23be96700 1 -- 172.21.15.12:0/799978396 shutdown_connections 2024-05-09T00:07:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.246+0000 7fa23be96700 1 --2- 172.21.15.12:0/799978396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2341140f0 0x7fa2341164e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.246+0000 7fa23be96700 1 --2- 172.21.15.12:0/799978396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa234107fa0 0x7fa234108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.246+0000 7fa23be96700 1 --2- 172.21.15.12:0/799978396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2341088c0 0x7fa234113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.246+0000 7fa23be96700 1 -- 172.21.15.12:0/799978396 >> 172.21.15.12:0/799978396 conn(0x7fa23406bff0 msgr2=0x7fa23406c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.246+0000 7fa23be96700 1 -- 172.21.15.12:0/799978396 shutdown_connections 2024-05-09T00:07:27.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.246+0000 7fa23be96700 1 -- 172.21.15.12:0/799978396 wait complete. 2024-05-09T00:07:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa23be96700 1 Processor -- start 2024-05-09T00:07:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa23be96700 1 -- start start 2024-05-09T00:07:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa23be96700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa234107fa0 0x7fa234109a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa23be96700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2341088c0 0x7fa234109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa23be96700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2341140f0 0x7fa23410a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa23be96700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341adfa0 con 0x7fa234107fa0 2024-05-09T00:07:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa23be96700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341ae110 con 0x7fa2341088c0 2024-05-09T00:07:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa23be96700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341ae2d0 con 0x7fa2341140f0 2024-05-09T00:07:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa239c32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa234107fa0 0x7fa234109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa239c32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa234107fa0 0x7fa234109a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35726/0 (socket says 172.21.15.12:35726) 2024-05-09T00:07:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa239c32700 1 -- 172.21.15.12:0/2367799706 learned_addr learned my addr 172.21.15.12:0/2367799706 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.247+0000 7fa23a433700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2341140f0 0x7fa23410a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa239c32700 1 -- 172.21.15.12:0/2367799706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2341140f0 msgr2=0x7fa23410a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa239c32700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2341140f0 0x7fa23410a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa239c32700 1 -- 172.21.15.12:0/2367799706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2341088c0 msgr2=0x7fa234109f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:07:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa239c32700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2341088c0 0x7fa234109f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa239c32700 1 -- 172.21.15.12:0/2367799706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c009c40 con 0x7fa234107fa0 2024-05-09T00:07:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa23a433700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2341140f0 0x7fa23410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa239c32700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa234107fa0 0x7fa234109a10 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fa22400d980 tx=0x7fa22400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa22affd700 1 -- 172.21.15.12:0/2367799706 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224009930 con 0x7fa234107fa0 2024-05-09T00:07:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa22affd700 1 -- 172.21.15.12:0/2367799706 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa224010460 con 0x7fa234107fa0 2024-05-09T00:07:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa23be96700 1 -- 172.21.15.12:0/2367799706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2341ae5b0 con 0x7fa234107fa0 2024-05-09T00:07:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa22affd700 1 -- 172.21.15.12:0/2367799706 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400f620 con 0x7fa234107fa0 2024-05-09T00:07:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.248+0000 7fa23be96700 1 -- 172.21.15.12:0/2367799706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2341aeb10 con 0x7fa234107fa0 2024-05-09T00:07:27.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.252+0000 7fa22affd700 1 -- 172.21.15.12:0/2367799706 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa2240105d0 con 0x7fa234107fa0 2024-05-09T00:07:27.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.253+0000 7fa248050b80 1 Processor -- start 2024-05-09T00:07:27.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.253+0000 7fa248050b80 1 -- start start 2024-05-09T00:07:27.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.253+0000 7fa248050b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 0x5564390c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.253+0000 7fa248050b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.253+0000 7fa248050b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 0x5564390c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.253+0000 7fa248050b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564390181d0 con 0x5564390c77a0 2024-05-09T00:07:27.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.253+0000 7fa248050b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556439015b90 con 0x5564390c6e80 2024-05-09T00:07:27.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.253+0000 7fa248050b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556439011480 con 0x5564390c1910 2024-05-09T00:07:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.253+0000 7fa22affd700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2200a1350 0x7fa2200a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.254+0000 7fa22affd700 1 -- 172.21.15.12:0/2367799706 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa2240ccac0 con 0x7fa234107fa0 2024-05-09T00:07:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.254+0000 7fa239c32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.254+0000 7fa239c32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390c13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35730/0 (socket says 172.21.15.12:35730) 2024-05-09T00:07:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.254+0000 7fa239c32700 1 -- 172.21.15.12:0/738700977 learned_addr learned my addr 172.21.15.12:0/738700977 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.254+0000 7fa239c32700 1 -- 172.21.15.12:0/738700977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 msgr2=0x5564390c1d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:07:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.254+0000 7fa239c32700 1 --2- 172.21.15.12:0/738700977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 0x5564390c1d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.254+0000 7fa239c32700 1 -- 172.21.15.12:0/738700977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 msgr2=0x5564390c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.254+0000 7fa239c32700 1 --2- 172.21.15.12:0/738700977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 0x5564390c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.254+0000 7fa239c32700 1 -- 172.21.15.12:0/738700977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556438e92430 con 0x5564390c77a0 2024-05-09T00:07:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.255+0000 7fa239c32700 1 --2- 172.21.15.12:0/738700977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390c13d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fa224099b70 tx=0x7fa224099ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5af55e49da67f49 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.255+0000 7fa23a433700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2200a1350 0x7fa2200a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.255+0000 7fa23a433700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2200a1350 0x7fa2200a3810 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009040 tx=0x7fa22c00baa0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.260+0000 7fa21ffff700 1 -- 172.21.15.12:0/738700977 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22409da10 con 0x5564390c77a0 2024-05-09T00:07:27.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.260+0000 7fa21ffff700 1 -- 172.21.15.12:0/738700977 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2240a2060 con 0x5564390c77a0 2024-05-09T00:07:27.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.260+0000 7fa21ffff700 1 -- 172.21.15.12:0/738700977 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224099e40 con 0x5564390c77a0 2024-05-09T00:07:27.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.262+0000 7fa248050b80 1 -- 172.21.15.12:0/738700977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 msgr2=0x5564390c13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.262+0000 7fa248050b80 1 --2- 172.21.15.12:0/738700977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390c13d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fa224099b70 tx=0x7fa224099ac0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.262+0000 7fa248050b80 1 -- 172.21.15.12:0/738700977 shutdown_connections 2024-05-09T00:07:27.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.262+0000 7fa248050b80 1 --2- 172.21.15.12:0/738700977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 0x5564390c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.262+0000 7fa248050b80 1 --2- 172.21.15.12:0/738700977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 0x5564390c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.262+0000 7fa248050b80 1 --2- 172.21.15.12:0/738700977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390c13d0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.262+0000 7fa248050b80 1 -- 172.21.15.12:0/738700977 >> 172.21.15.12:0/738700977 conn(0x55643901f290 msgr2=0x5564390bbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.262+0000 7fa248050b80 1 -- 172.21.15.12:0/738700977 shutdown_connections 2024-05-09T00:07:27.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.262+0000 7fa248050b80 1 -- 172.21.15.12:0/738700977 wait complete. 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.262+0000 7fa248050b80 1 -- start start 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.263+0000 7fa248050b80 1 Processor -- start 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.263+0000 7fa248050b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.263+0000 7fa248050b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c6e80 0x5564390cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.263+0000 7fa248050b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c77a0 0x5564390cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.263+0000 7fa248050b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556438fad5e0 con 0x5564390c1910 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.263+0000 7fa248050b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556438fad280 con 0x5564390c77a0 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.263+0000 7fa248050b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556438e94720 con 0x5564390c6e80 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.263+0000 7fa248050b80 10 client.0 _ll_register_callbacks cb 0x556438fb6bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa23a433700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c77a0 0x5564390cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa239c32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa23a433700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c77a0 0x5564390cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:56426/0 (socket says 172.21.15.12:56426) 2024-05-09T00:07:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa23a433700 1 -- 172.21.15.12:0/4195848523 learned_addr learned my addr 172.21.15.12:0/4195848523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa239c32700 1 -- 172.21.15.12:0/4195848523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c6e80 msgr2=0x5564390cb120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:07:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa239c32700 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c6e80 0x5564390cb120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa239c32700 1 -- 172.21.15.12:0/4195848523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c77a0 msgr2=0x5564390cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa239c32700 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c77a0 0x5564390cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa239c32700 1 -- 172.21.15.12:0/4195848523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2341aeb10 con 0x5564390c1910 2024-05-09T00:07:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa239c32700 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390cabe0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x5564390c7c00 tx=0x7fa224096050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4deb0a664c44cba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa21d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa21d7fa700 1 -- 172.21.15.12:0/4195848523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240990d0 con 0x5564390c1910 2024-05-09T00:07:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.264+0000 7fa21d7fa700 1 -- 172.21.15.12:0/4195848523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa224099230 con 0x5564390c1910 2024-05-09T00:07:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.265+0000 7fa248050b80 1 -- 172.21.15.12:0/4195848523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556438f6a2f0 con 0x5564390c1910 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.266+0000 7fa21d7fa700 1 -- 172.21.15.12:0/4195848523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa224099e40 con 0x5564390c1910 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.268+0000 7fa248050b80 10 client.21246 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.268+0000 7fa248050b80 1 -- 172.21.15.12:0/4195848523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x556438e3ebe0 con 0x5564390c1910 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.269+0000 7fa21d7fa700 1 -- 172.21.15.12:0/4195848523 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556438f6a2f0 con 0x5564390c1910 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.269+0000 7fa21d7fa700 1 -- 172.21.15.12:0/4195848523 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2240adab0 con 0x5564390c1910 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.269+0000 7fa248050b80 10 client.21246 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.269+0000 7fa248050b80 10 client.21246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.269+0000 7fa248050b80 20 client.21246 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.269+0000 7fa248050b80 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5564390dae70 0x5564390dd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.269+0000 7fa248050b80 4 client.21246 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.269+0000 7fa248050b80 1 -- 172.21.15.12:0/4195848523 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556439090890 con 0x5564390dae70 2024-05-09T00:07:27.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.274+0000 7fa239431700 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5564390dae70 0x5564390dd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 276298 31.66 MB/sec execute 322 sec latency 117.933 ms 2024-05-09T00:07:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.287+0000 7fa239431700 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5564390dae70 0x5564390dd260 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baff2c78a18792fb server_cookie=3b292490279b4341 in_seq=0 out_seq=0 2024-05-09T00:07:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.289+0000 7fa21d7fa700 10 client.21246 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.292+0000 7fa21d7fa700 1 -- 172.21.15.12:0/4195848523 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556439090890 con 0x5564390dae70 2024-05-09T00:07:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.292+0000 7fa21d7fa700 10 client.21246 handle_command_reply: tid=0 2024-05-09T00:07:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.292+0000 7fa248050b80 1 client.21246 shutdown 2024-05-09T00:07:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.293+0000 7fa248050b80 1 -- 172.21.15.12:0/4195848523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 msgr2=0x5564390cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.293+0000 7fa248050b80 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390cabe0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x5564390c7c00 tx=0x7fa224096050 comp rx=0 tx=0).stop 2024-05-09T00:07:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.293+0000 7fa248050b80 1 -- 172.21.15.12:0/4195848523 shutdown_connections 2024-05-09T00:07:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.293+0000 7fa248050b80 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5564390dae70 0x5564390dd260 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.293+0000 7fa248050b80 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c77a0 0x5564390cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.293+0000 7fa248050b80 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390cabe0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.293+0000 7fa248050b80 1 --2- 172.21.15.12:0/4195848523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c6e80 0x5564390cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.293+0000 7fa248050b80 1 -- 172.21.15.12:0/4195848523 >> 172.21.15.12:0/4195848523 conn(0x55643901f290 msgr2=0x5564390bb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.293+0000 7fa21d7fa700 0 client.21246 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:27.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.295+0000 7fa248050b80 1 -- 172.21.15.12:0/4195848523 shutdown_connections 2024-05-09T00:07:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.303+0000 7fa248050b80 1 -- 172.21.15.12:0/4195848523 wait complete. 2024-05-09T00:07:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.304+0000 7fa248050b80 20 client.21246 trim_cache size 0 max 16384 2024-05-09T00:07:27.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa248050b80 1 Processor -- start 2024-05-09T00:07:27.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa248050b80 1 -- start start 2024-05-09T00:07:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa248050b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 0x5564390c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa248050b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 0x5564390c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa248050b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa248050b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564390c75e0 con 0x5564390c77a0 2024-05-09T00:07:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa248050b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564390e23f0 con 0x5564390c6e80 2024-05-09T00:07:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa248050b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564390b6880 con 0x5564390c1910 2024-05-09T00:07:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa239c32700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 0x5564390c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa239c32700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 0x5564390c13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:60014/0 (socket says 172.21.15.12:60014) 2024-05-09T00:07:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.337+0000 7fa239c32700 1 -- 172.21.15.12:0/3764504948 learned_addr learned my addr 172.21.15.12:0/3764504948 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.338+0000 7fa239c32700 1 -- 172.21.15.12:0/3764504948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 msgr2=0x5564390c63d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.338+0000 7fa239c32700 1 --2- 172.21.15.12:0/3764504948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 0x5564390c63d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.338+0000 7fa239c32700 1 -- 172.21.15.12:0/3764504948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 msgr2=0x5564390bbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.338+0000 7fa23a433700 1 --2- 172.21.15.12:0/3764504948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.338+0000 7fa239c32700 1 --2- 172.21.15.12:0/3764504948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390bbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.338+0000 7fa239c32700 1 -- 172.21.15.12:0/3764504948 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564390b8560 con 0x5564390c1910 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.338+0000 7fa23a433700 1 --2- 172.21.15.12:0/3764504948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390bbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.338+0000 7fa239c32700 1 --2- 172.21.15.12:0/3764504948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 0x5564390c13d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa2240998e0 tx=0x7fa224005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f1d7c231515a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.339+0000 7fa228ff9700 1 -- 172.21.15.12:0/3764504948 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224098eb0 con 0x5564390c1910 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.339+0000 7fa228ff9700 1 -- 172.21.15.12:0/3764504948 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2240adab0 con 0x5564390c1910 2024-05-09T00:07:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.339+0000 7fa228ff9700 1 -- 172.21.15.12:0/3764504948 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224098cc0 con 0x5564390c1910 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 -- 172.21.15.12:0/3764504948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 msgr2=0x5564390c13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 --2- 172.21.15.12:0/3764504948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 0x5564390c13d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa2240998e0 tx=0x7fa224005e60 comp rx=0 tx=0).stop 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 -- 172.21.15.12:0/3764504948 shutdown_connections 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 --2- 172.21.15.12:0/3764504948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c77a0 0x5564390bbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 --2- 172.21.15.12:0/3764504948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c1910 0x5564390c13d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 --2- 172.21.15.12:0/3764504948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 0x5564390c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 -- 172.21.15.12:0/3764504948 >> 172.21.15.12:0/3764504948 conn(0x55643901f290 msgr2=0x5564390c5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 -- 172.21.15.12:0/3764504948 shutdown_connections 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 -- 172.21.15.12:0/3764504948 wait complete. 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 -- start start 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.340+0000 7fa248050b80 1 Processor -- start 2024-05-09T00:07:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.341+0000 7fa248050b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.341+0000 7fa248050b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 0x5564390ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.341+0000 7fa248050b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c77a0 0x5564391306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.341+0000 7fa248050b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564390cf070 con 0x5564390c1910 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.341+0000 7fa248050b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556439130c50 con 0x5564390c6e80 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.341+0000 7fa248050b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556439130e10 con 0x5564390c77a0 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.341+0000 7fa248050b80 10 client.0 _ll_register_callbacks cb 0x556438fb6bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.342+0000 7fa239c32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.342+0000 7fa239c32700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35754/0 (socket says 172.21.15.12:35754) 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.342+0000 7fa239c32700 1 -- 172.21.15.12:0/1658072200 learned_addr learned my addr 172.21.15.12:0/1658072200 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.342+0000 7fa23a433700 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c77a0 0x5564391306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.342+0000 7fa239c32700 1 -- 172.21.15.12:0/1658072200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c77a0 msgr2=0x5564391306e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.342+0000 7fa239c32700 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c77a0 0x5564391306e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.342+0000 7fa239c32700 1 -- 172.21.15.12:0/1658072200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 msgr2=0x5564390ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.342+0000 7fa239c32700 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 0x5564390ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.342+0000 7fa239c32700 1 -- 172.21.15.12:0/1658072200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556438e3ebe0 con 0x5564390c1910 2024-05-09T00:07:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.343+0000 7fa239c32700 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390ce3b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa224000c00 tx=0x7fa224005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42e32379910a6bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.343+0000 7fa21d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.343+0000 7fa21d7fa700 1 -- 172.21.15.12:0/1658072200 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240adc20 con 0x5564390c1910 2024-05-09T00:07:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.343+0000 7fa21d7fa700 1 -- 172.21.15.12:0/1658072200 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa2240a1e00 con 0x5564390c1910 2024-05-09T00:07:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.343+0000 7fa248050b80 1 -- 172.21.15.12:0/1658072200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2240a2060 con 0x5564390c1910 2024-05-09T00:07:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.343+0000 7fa21d7fa700 1 -- 172.21.15.12:0/1658072200 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2240992e0 con 0x5564390c1910 2024-05-09T00:07:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.343+0000 7fa248050b80 10 client.21249 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.343+0000 7fa248050b80 1 -- 172.21.15.12:0/1658072200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x556438fd75e0 con 0x5564390c1910 2024-05-09T00:07:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.344+0000 7fa21d7fa700 1 -- 172.21.15.12:0/1658072200 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2240a2060 con 0x5564390c1910 2024-05-09T00:07:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.345+0000 7fa21d7fa700 1 -- 172.21.15.12:0/1658072200 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2240a3070 con 0x5564390c1910 2024-05-09T00:07:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.345+0000 7fa248050b80 10 client.21249 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.345+0000 7fa248050b80 10 client.21249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.345+0000 7fa248050b80 20 client.21249 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.345+0000 7fa248050b80 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556439132030 0x556439134420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.345+0000 7fa248050b80 4 client.21249 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:07:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.345+0000 7fa248050b80 1 -- 172.21.15.12:0/1658072200 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556439009f40 con 0x556439132030 2024-05-09T00:07:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.346+0000 7fa239431700 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556439132030 0x556439134420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.347+0000 7fa239431700 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556439132030 0x556439134420 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ef23e11e7d8269 server_cookie=7966ea5dc1af9df2 in_seq=0 out_seq=0 2024-05-09T00:07:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.347+0000 7fa21d7fa700 10 client.21249 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.352+0000 7fa21d7fa700 1 -- 172.21.15.12:0/1658072200 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556439009f40 con 0x556439132030 2024-05-09T00:07:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.352+0000 7fa21d7fa700 10 client.21249 handle_command_reply: tid=0 2024-05-09T00:07:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.352+0000 7fa248050b80 1 client.21249 shutdown 2024-05-09T00:07:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.353+0000 7fa248050b80 1 -- 172.21.15.12:0/1658072200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 msgr2=0x5564390ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.353+0000 7fa248050b80 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390ce3b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa224000c00 tx=0x7fa224005d40 comp rx=0 tx=0).stop 2024-05-09T00:07:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.353+0000 7fa248050b80 1 -- 172.21.15.12:0/1658072200 shutdown_connections 2024-05-09T00:07:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.353+0000 7fa248050b80 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564390c77a0 0x5564391306e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.353+0000 7fa248050b80 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5564390c1910 0x5564390ce3b0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.353+0000 7fa248050b80 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x556439132030 0x556439134420 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.353+0000 7fa248050b80 1 --2- 172.21.15.12:0/1658072200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5564390c6e80 0x5564390ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.353+0000 7fa21d7fa700 0 client.21249 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.353+0000 7fa248050b80 1 -- 172.21.15.12:0/1658072200 >> 172.21.15.12:0/1658072200 conn(0x55643901f290 msgr2=0x5564390cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.353+0000 7fa248050b80 1 -- 172.21.15.12:0/1658072200 shutdown_connections 2024-05-09T00:07:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.354+0000 7fa248050b80 1 -- 172.21.15.12:0/1658072200 wait complete. 2024-05-09T00:07:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.354+0000 7fa248050b80 20 client.21249 trim_cache size 0 max 16384 2024-05-09T00:07:27.350 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:07:27.350 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:07:27.350 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "f6382178-cdcd-482e-96a6-a31e3da130a8", 2024-05-09T00:07:27.350 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:07:27.350 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:07:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 -- 172.21.15.12:0/2367799706 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2200a1350 msgr2=0x7fa2200a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2200a1350 0x7fa2200a3810 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009040 tx=0x7fa22c00baa0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 -- 172.21.15.12:0/2367799706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa234107fa0 msgr2=0x7fa234109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa234107fa0 0x7fa234109a10 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fa22400d980 tx=0x7fa22400dc90 comp rx=0 tx=0).stop 2024-05-09T00:07:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 -- 172.21.15.12:0/2367799706 shutdown_connections 2024-05-09T00:07:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa2200a1350 0x7fa2200a3810 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2341140f0 0x7fa23410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa234107fa0 0x7fa234109a10 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 --2- 172.21.15.12:0/2367799706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2341088c0 0x7fa234109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 -- 172.21.15.12:0/2367799706 >> 172.21.15.12:0/2367799706 conn(0x7fa23406bff0 msgr2=0x7fa23406fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 -- 172.21.15.12:0/2367799706 shutdown_connections 2024-05-09T00:07:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.355+0000 7fa23be96700 1 -- 172.21.15.12:0/2367799706 wait complete. 2024-05-09T00:07:27.374 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:07:27.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.573+0000 7f50d06ee700 1 -- 172.21.15.12:0/1400010774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50c8101160 msgr2=0x7f50c8101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.573+0000 7f50d06ee700 1 --2- 172.21.15.12:0/1400010774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50c8101160 0x7f50c8101540 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009990 tx=0x7f50c4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.573+0000 7f50d06ee700 1 -- 172.21.15.12:0/1400010774 shutdown_connections 2024-05-09T00:07:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.573+0000 7f50d06ee700 1 --2- 172.21.15.12:0/1400010774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50c8106280 0x7f50c810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.573+0000 7f50d06ee700 1 --2- 172.21.15.12:0/1400010774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50c8101160 0x7f50c8101540 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.573+0000 7f50d06ee700 1 --2- 172.21.15.12:0/1400010774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c8101b10 0x7f50c8105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.573+0000 7f50d06ee700 1 -- 172.21.15.12:0/1400010774 >> 172.21.15.12:0/1400010774 conn(0x7f50c80fc9e0 msgr2=0x7f50c80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.573+0000 7f50d06ee700 1 -- 172.21.15.12:0/1400010774 shutdown_connections 2024-05-09T00:07:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.573+0000 7f50d06ee700 1 -- 172.21.15.12:0/1400010774 wait complete. 2024-05-09T00:07:27.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50d06ee700 1 Processor -- start 2024-05-09T00:07:27.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50d06ee700 1 -- start start 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50d06ee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50c8101160 0x7f50c81973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50d06ee700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50c8106280 0x7f50c8197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50d06ee700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c819bcc0 0x7f50c819c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50d06ee700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c819c740 con 0x7f50c8106280 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50d06ee700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c819c8b0 con 0x7f50c819bcc0 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50d06ee700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c819ca70 con 0x7f50c8101160 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50cdc89700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50c8106280 0x7f50c8197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50cec8b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c819bcc0 0x7f50c819c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50cdc89700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50c8106280 0x7f50c8197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35776/0 (socket says 172.21.15.12:35776) 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.574+0000 7f50cdc89700 1 -- 172.21.15.12:0/1354435153 learned_addr learned my addr 172.21.15.12:0/1354435153 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.575+0000 7f50cec8b700 1 -- 172.21.15.12:0/1354435153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50c8101160 msgr2=0x7f50c81973e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.575+0000 7f50cec8b700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50c8101160 0x7f50c81973e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.575+0000 7f50cec8b700 1 -- 172.21.15.12:0/1354435153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50c8106280 msgr2=0x7f50c8197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.575+0000 7f50cec8b700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50c8106280 0x7f50c8197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.575+0000 7f50cec8b700 1 -- 172.21.15.12:0/1354435153 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c40095f0 con 0x7f50c819bcc0 2024-05-09T00:07:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.575+0000 7f50cec8b700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c819bcc0 0x7f50c819c140 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f50c000efc0 tx=0x7f50c000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.576+0000 7f50bf7fe700 1 -- 172.21.15.12:0/1354435153 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0018070 con 0x7f50c819bcc0 2024-05-09T00:07:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.576+0000 7f50bf7fe700 1 -- 172.21.15.12:0/1354435153 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50c000f040 con 0x7f50c819bcc0 2024-05-09T00:07:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.576+0000 7f50bf7fe700 1 -- 172.21.15.12:0/1354435153 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c000ce10 con 0x7f50c819bcc0 2024-05-09T00:07:27.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.578+0000 7f50d06ee700 1 -- 172.21.15.12:0/1354435153 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c81a3f70 con 0x7f50c819bcc0 2024-05-09T00:07:27.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.578+0000 7f50d06ee700 1 -- 172.21.15.12:0/1354435153 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c81a4400 con 0x7f50c819bcc0 2024-05-09T00:07:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.581+0000 7f50bf7fe700 1 -- 172.21.15.12:0/1354435153 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f50c0003a00 con 0x7f50c819bcc0 2024-05-09T00:07:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.582+0000 7f50bf7fe700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50b40a1310 0x7f50b40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.582+0000 7f50bf7fe700 1 -- 172.21.15.12:0/1354435153 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f50c00cbfd0 con 0x7f50c819bcc0 2024-05-09T00:07:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.582+0000 7f50ce48a700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50b40a1310 0x7f50b40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.582+0000 7f50ce48a700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50b40a1310 0x7f50b40a37d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f50c4005b40 tx=0x7f50c4005ad0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.583+0000 7f50dc8a8b80 1 Processor -- start 2024-05-09T00:07:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.583+0000 7f50dc8a8b80 1 -- start start 2024-05-09T00:07:27.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.583+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf2e00 0x56074bdf31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.583+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf3720 0x56074bded350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.583+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bdedcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.583+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074bd44150 con 0x56074bdf3720 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.583+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074bd41b10 con 0x56074bdf2e00 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.583+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074bd3d400 con 0x56074bded890 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50cec8b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bdedcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50ce48a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf2e00 0x56074bdf31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50cec8b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bdedcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:60048/0 (socket says 172.21.15.12:60048) 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50cec8b700 1 -- 172.21.15.12:0/3444705963 learned_addr learned my addr 172.21.15.12:0/3444705963 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50cdc89700 1 --2- 172.21.15.12:0/3444705963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf3720 0x56074bded350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50cec8b700 1 -- 172.21.15.12:0/3444705963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf2e00 msgr2=0x56074bdf31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50cec8b700 1 --2- 172.21.15.12:0/3444705963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf2e00 0x56074bdf31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50cec8b700 1 -- 172.21.15.12:0/3444705963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf3720 msgr2=0x56074bded350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50cec8b700 1 --2- 172.21.15.12:0/3444705963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf3720 0x56074bded350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50cec8b700 1 -- 172.21.15.12:0/3444705963 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56074bbbe3b0 con 0x56074bded890 2024-05-09T00:07:27.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.584+0000 7f50cec8b700 1 --2- 172.21.15.12:0/3444705963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bdedcd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f50c00090a0 tx=0x7f50c0095550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2068cfa10232a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.585+0000 7f50bcff9700 1 -- 172.21.15.12:0/3444705963 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c001a2d0 con 0x56074bded890 2024-05-09T00:07:27.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.585+0000 7f50bcff9700 1 -- 172.21.15.12:0/3444705963 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50c00997f0 con 0x56074bded890 2024-05-09T00:07:27.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.585+0000 7f50bcff9700 1 -- 172.21.15.12:0/3444705963 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0097520 con 0x56074bded890 2024-05-09T00:07:27.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3444705963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 msgr2=0x56074bdedcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/3444705963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bdedcd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f50c00090a0 tx=0x7f50c0095550 comp rx=0 tx=0).stop 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3444705963 shutdown_connections 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/3444705963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bdedcd0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/3444705963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf2e00 0x56074bdf31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/3444705963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf3720 0x56074bded350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3444705963 >> 172.21.15.12:0/3444705963 conn(0x56074bd4b210 msgr2=0x56074bde7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3444705963 shutdown_connections 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3444705963 wait complete. 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 -- start start 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.586+0000 7f50dc8a8b80 1 Processor -- start 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bded890 0x56074bdf6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bdf2e00 0x56074bdf7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf3720 0x56074bdf7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074bcd9560 con 0x56074bded890 2024-05-09T00:07:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074bcd9200 con 0x56074bdf3720 2024-05-09T00:07:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074bbc06a0 con 0x56074bdf2e00 2024-05-09T00:07:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50ce48a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bded890 0x56074bdf6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50dc8a8b80 10 client.0 _ll_register_callbacks cb 0x56074bce2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50ce48a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bded890 0x56074bdf6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35798/0 (socket says 172.21.15.12:35798) 2024-05-09T00:07:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50ce48a700 1 -- 172.21.15.12:0/3186251526 learned_addr learned my addr 172.21.15.12:0/3186251526 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50cdc89700 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bdf2e00 0x56074bdf7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50cec8b700 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf3720 0x56074bdf7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50cec8b700 1 -- 172.21.15.12:0/3186251526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bdf2e00 msgr2=0x56074bdf7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50cec8b700 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bdf2e00 0x56074bdf7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50cec8b700 1 -- 172.21.15.12:0/3186251526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bded890 msgr2=0x56074bdf6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50cec8b700 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bded890 0x56074bdf6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.587+0000 7f50cec8b700 1 -- 172.21.15.12:0/3186251526 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c81a4400 con 0x56074bdf3720 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.588+0000 7f50cec8b700 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf3720 0x56074bdf7570 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x56074bdf9dc0 tx=0x7f50c0098b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df65a70197dbc238 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.588+0000 7f50b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.588+0000 7f50b1ffb700 1 -- 172.21.15.12:0/3186251526 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c009cf20 con 0x56074bdf3720 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.588+0000 7f50b1ffb700 1 -- 172.21.15.12:0/3186251526 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f50c00a1570 con 0x56074bdf3720 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.588+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3186251526 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56074bc96270 con 0x56074bdf3720 2024-05-09T00:07:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.588+0000 7f50b1ffb700 1 -- 172.21.15.12:0/3186251526 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c0099350 con 0x56074bdf3720 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.588+0000 7f50dc8a8b80 10 client.30752 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.588+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3186251526 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x56074bb6ab60 con 0x56074bdf3720 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.589+0000 7f50b1ffb700 1 -- 172.21.15.12:0/3186251526 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56074bc96270 con 0x56074bdf3720 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.589+0000 7f50b1ffb700 1 -- 172.21.15.12:0/3186251526 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f50c007b030 con 0x56074bdf3720 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.589+0000 7f50dc8a8b80 10 client.30752 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.589+0000 7f50dc8a8b80 10 client.30752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.589+0000 7f50dc8a8b80 20 client.30752 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.589+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56074be0f710 0x56074be11b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.589+0000 7f50dc8a8b80 4 client.30752 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.589+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3186251526 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56074bdbc810 con 0x56074be0f710 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.589+0000 7f50ce48a700 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56074be0f710 0x56074be11b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.590+0000 7f50b1ffb700 10 client.30752 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.590+0000 7f50ce48a700 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56074be0f710 0x56074be11b00 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c76f97e7c47095 server_cookie=a46c66a37fde0b9c in_seq=0 out_seq=0 2024-05-09T00:07:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.591+0000 7f50b1ffb700 1 -- 172.21.15.12:0/3186251526 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56074bdbc810 con 0x56074be0f710 2024-05-09T00:07:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.591+0000 7f50b1ffb700 10 client.30752 handle_command_reply: tid=0 2024-05-09T00:07:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.591+0000 7f50dc8a8b80 1 client.30752 shutdown 2024-05-09T00:07:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3186251526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf3720 msgr2=0x56074bdf7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf3720 0x56074bdf7570 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x56074bdf9dc0 tx=0x7f50c0098b10 comp rx=0 tx=0).stop 2024-05-09T00:07:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3186251526 shutdown_connections 2024-05-09T00:07:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56074be0f710 0x56074be11b00 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf3720 0x56074bdf7570 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bded890 0x56074bdf6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50b1ffb700 0 client.30752 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/3186251526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bdf2e00 0x56074bdf7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3186251526 >> 172.21.15.12:0/3186251526 conn(0x56074bd4b210 msgr2=0x56074bde75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3186251526 shutdown_connections 2024-05-09T00:07:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.594+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/3186251526 wait complete. 2024-05-09T00:07:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.595+0000 7f50dc8a8b80 20 client.30752 trim_cache size 0 max 16384 2024-05-09T00:07:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.623+0000 7f50dc8a8b80 1 Processor -- start 2024-05-09T00:07:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.624+0000 7f50dc8a8b80 1 -- start start 2024-05-09T00:07:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.624+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bded350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.624+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 0x56074bdf2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.624+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf3720 0x56074bde7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.624+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074bde00d0 con 0x56074bdf2e00 2024-05-09T00:07:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.624+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074bdf8160 con 0x56074bdf3720 2024-05-09T00:07:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.624+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074bde9a60 con 0x56074bded890 2024-05-09T00:07:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.624+0000 7f50ce48a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bded350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.624+0000 7f50ce48a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bded350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:60052/0 (socket says 172.21.15.12:60052) 2024-05-09T00:07:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.624+0000 7f50ce48a700 1 -- 172.21.15.12:0/1026406580 learned_addr learned my addr 172.21.15.12:0/1026406580 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.625+0000 7f50ce48a700 1 -- 172.21.15.12:0/1026406580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf3720 msgr2=0x56074bde7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.625+0000 7f50ce48a700 1 --2- 172.21.15.12:0/1026406580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf3720 0x56074bde7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.625+0000 7f50ce48a700 1 -- 172.21.15.12:0/1026406580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 msgr2=0x56074bdf2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.625+0000 7f50ce48a700 1 --2- 172.21.15.12:0/1026406580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 0x56074bdf2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.625+0000 7f50ce48a700 1 -- 172.21.15.12:0/1026406580 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56074bdedf40 con 0x56074bded890 2024-05-09T00:07:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.625+0000 7f50ce48a700 1 --2- 172.21.15.12:0/1026406580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bded350 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f50c400c010 tx=0x7f50c4018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46de5448c166c5bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.627+0000 7f50bd7fa700 1 -- 172.21.15.12:0/1026406580 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4029040 con 0x56074bded890 2024-05-09T00:07:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.627+0000 7f50bd7fa700 1 -- 172.21.15.12:0/1026406580 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50c402a040 con 0x56074bded890 2024-05-09T00:07:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.627+0000 7f50bd7fa700 1 -- 172.21.15.12:0/1026406580 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c401d070 con 0x56074bded890 2024-05-09T00:07:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.628+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/1026406580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 msgr2=0x56074bded350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.628+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/1026406580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bded350 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f50c400c010 tx=0x7f50c4018000 comp rx=0 tx=0).stop 2024-05-09T00:07:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.628+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/1026406580 shutdown_connections 2024-05-09T00:07:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.628+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/1026406580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bdf3720 0x56074bde7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.628+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/1026406580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bded890 0x56074bded350 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.628+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/1026406580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 0x56074bdf2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.628+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/1026406580 >> 172.21.15.12:0/1026406580 conn(0x56074bd4b210 msgr2=0x56074bdf53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.628+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/1026406580 shutdown_connections 2024-05-09T00:07:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.628+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/1026406580 wait complete. 2024-05-09T00:07:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.628+0000 7f50dc8a8b80 1 -- start start 2024-05-09T00:07:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50dc8a8b80 1 Processor -- start 2024-05-09T00:07:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bded890 0x56074be06d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 0x56074be60e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50dc8a8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bdf3720 0x56074be61380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074be5e690 con 0x56074bdf2e00 2024-05-09T00:07:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074be5e800 con 0x56074bded890 2024-05-09T00:07:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50dc8a8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074be5e940 con 0x56074bdf3720 2024-05-09T00:07:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50dc8a8b80 10 client.0 _ll_register_callbacks cb 0x56074bce2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50cdc89700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 0x56074be60e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50cdc89700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 0x56074be60e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35814/0 (socket says 172.21.15.12:35814) 2024-05-09T00:07:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.629+0000 7f50cdc89700 1 -- 172.21.15.12:0/681152143 learned_addr learned my addr 172.21.15.12:0/681152143 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.630+0000 7f50cec8b700 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bdf3720 0x56074be61380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.630+0000 7f50ce48a700 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bded890 0x56074be06d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.630+0000 7f50cdc89700 1 -- 172.21.15.12:0/681152143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bdf3720 msgr2=0x56074be61380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.630+0000 7f50cdc89700 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bdf3720 0x56074be61380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.630+0000 7f50cdc89700 1 -- 172.21.15.12:0/681152143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bded890 msgr2=0x56074be06d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.630+0000 7f50cdc89700 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bded890 0x56074be06d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.630+0000 7f50cdc89700 1 -- 172.21.15.12:0/681152143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c810e670 con 0x56074bdf2e00 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.630+0000 7f50cdc89700 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 0x56074be60e40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f50c8198930 tx=0x7f50b8004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460e6a14a07475e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.630+0000 7f50b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.630+0000 7f50b1ffb700 1 -- 172.21.15.12:0/681152143 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8003bd0 con 0x56074bdf2e00 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/681152143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50c00997f0 con 0x56074bdf2e00 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50b1ffb700 1 -- 172.21.15.12:0/681152143 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f50b8003d30 con 0x56074bdf2e00 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50b1ffb700 1 -- 172.21.15.12:0/681152143 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50b8024450 con 0x56074bdf2e00 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50dc8a8b80 10 client.21264 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/681152143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x56074bd03560 con 0x56074bdf2e00 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50b1ffb700 1 -- 172.21.15.12:0/681152143 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50b8028020 con 0x56074bdf2e00 2024-05-09T00:07:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50b1ffb700 1 -- 172.21.15.12:0/681152143 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f50b80117f0 con 0x56074bdf2e00 2024-05-09T00:07:27.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50dc8a8b80 10 client.21264 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:27.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50dc8a8b80 10 client.21264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:27.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50dc8a8b80 20 client.21264 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:27.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56074be06550 0x56074bdfc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:27.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50dc8a8b80 4 client.21264 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:07:27.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.631+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/681152143 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56074bd35ec0 con 0x56074be06550 2024-05-09T00:07:27.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.634+0000 7f50ce48a700 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56074be06550 0x56074bdfc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:27.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.634+0000 7f50ce48a700 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56074be06550 0x56074bdfc1d0 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f4ef056b1cec263 server_cookie=b816d14c63b56fa2 in_seq=0 out_seq=0 2024-05-09T00:07:27.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.635+0000 7f50b1ffb700 10 client.21264 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50b1ffb700 1 -- 172.21.15.12:0/681152143 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x56074bd35ec0 con 0x56074be06550 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50b1ffb700 10 client.21264 handle_command_reply: tid=0 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 client.21264 shutdown 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/681152143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 msgr2=0x56074be60e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 0x56074be60e40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f50c8198930 tx=0x7f50b8004320 comp rx=0 tx=0).stop 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/681152143 shutdown_connections 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56074bdf3720 0x56074be61380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56074be06550 0x56074bdfc1d0 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56074bded890 0x56074be06d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 --2- 172.21.15.12:0/681152143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56074bdf2e00 0x56074be60e40 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50b1ffb700 0 client.21264 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/681152143 >> 172.21.15.12:0/681152143 conn(0x56074bd4b210 msgr2=0x56074bdf22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/681152143 shutdown_connections 2024-05-09T00:07:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.639+0000 7f50dc8a8b80 1 -- 172.21.15.12:0/681152143 wait complete. 2024-05-09T00:07:27.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50dc8a8b80 20 client.21264 trim_cache size 0 max 16384 2024-05-09T00:07:27.636 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:07:27.636 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (81 inodes in the stack)", 2024-05-09T00:07:27.636 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:07:27.636 INFO:teuthology.orchestra.run.smithi012.stdout: "f6382178-cdcd-482e-96a6-a31e3da130a8": { 2024-05-09T00:07:27.636 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:07:27.636 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "f6382178-cdcd-482e-96a6-a31e3da130a8", 2024-05-09T00:07:27.636 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:07:27.637 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:07:27.637 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:07:27.637 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:07:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50d06ee700 1 -- 172.21.15.12:0/1354435153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50b40a1310 msgr2=0x7f50b40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50d06ee700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50b40a1310 0x7f50b40a37d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f50c4005b40 tx=0x7f50c4005ad0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50d06ee700 1 -- 172.21.15.12:0/1354435153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c819bcc0 msgr2=0x7f50c819c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50d06ee700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c819bcc0 0x7f50c819c140 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f50c000efc0 tx=0x7f50c000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50d06ee700 1 -- 172.21.15.12:0/1354435153 shutdown_connections 2024-05-09T00:07:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50d06ee700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50b40a1310 0x7f50b40a37d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50d06ee700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50c819bcc0 0x7f50c819c140 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50d06ee700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50c8101160 0x7f50c81973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50d06ee700 1 --2- 172.21.15.12:0/1354435153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50c8106280 0x7f50c8197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.640+0000 7f50d06ee700 1 -- 172.21.15.12:0/1354435153 >> 172.21.15.12:0/1354435153 conn(0x7f50c80fc9e0 msgr2=0x7f50c80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.641+0000 7f50d06ee700 1 -- 172.21.15.12:0/1354435153 shutdown_connections 2024-05-09T00:07:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:27.641+0000 7f50d06ee700 1 -- 172.21.15.12:0/1354435153 wait complete. 2024-05-09T00:07:27.650 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:27 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:07:27.651 INFO:tasks.cephfs.filesystem:scrub status for tag:f6382178-cdcd-482e-96a6-a31e3da130a8 - {'path': '/', 'tag': 'f6382178-cdcd-482e-96a6-a31e3da130a8', 'options': 'recursive,force'} 2024-05-09T00:07:27.763 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:27 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:07:27.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:27 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:07:28.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 277033 31.73 MB/sec execute 323 sec latency 126.660 ms 2024-05-09T00:07:28.765 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:28 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:07:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:28 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:28 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:07:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:28 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:07:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:28 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:28 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:28 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:07:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:28 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:28.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:28 smithi160 ceph-mon[46152]: pgmap v677: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 779 op/s 2024-05-09T00:07:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:28 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:07:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:28 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:28 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:07:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:28 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:07:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:28 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:28 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:28 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:07:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:28 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:28.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:28 smithi012 ceph-mon[42503]: pgmap v677: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 779 op/s 2024-05-09T00:07:29.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:28 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:07:29.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:28 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:29.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:28 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:07:29.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:28 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:07:29.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:28 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:29.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:28 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:29.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:28 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:07:29.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:28 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:29.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:28 smithi204 ceph-mon[47373]: pgmap v677: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 779 op/s 2024-05-09T00:07:29.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 277405 31.70 MB/sec execute 324 sec latency 363.026 ms 2024-05-09T00:07:29.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:29 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:30.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:29 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:30.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:29 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:07:30.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 277941 31.66 MB/sec execute 325 sec latency 126.054 ms 2024-05-09T00:07:30.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:30 smithi012 ceph-mon[42503]: pgmap v678: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 12 MiB/s wr, 647 op/s 2024-05-09T00:07:31.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:30 smithi204 ceph-mon[47373]: pgmap v678: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 12 MiB/s wr, 647 op/s 2024-05-09T00:07:31.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:30 smithi160 ceph-mon[46152]: pgmap v678: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 12 MiB/s wr, 647 op/s 2024-05-09T00:07:31.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 278671 31.70 MB/sec execute 326 sec latency 110.742 ms 2024-05-09T00:07:32.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:31 smithi204 ceph-mon[47373]: pgmap v679: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-05-09T00:07:32.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:31 smithi160 ceph-mon[46152]: pgmap v679: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-05-09T00:07:32.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:31 smithi012 ceph-mon[42503]: pgmap v679: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-05-09T00:07:32.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 279129 31.68 MB/sec execute 327 sec latency 148.881 ms 2024-05-09T00:07:33.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 279797 31.68 MB/sec execute 328 sec latency 98.550 ms 2024-05-09T00:07:34.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:33 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:07:34.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:33 smithi204 ceph-mon[47373]: pgmap v680: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 856 op/s 2024-05-09T00:07:34.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:33 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:07:34.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:33 smithi160 ceph-mon[46152]: pgmap v680: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 856 op/s 2024-05-09T00:07:34.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:33 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:07:34.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:33 smithi012 ceph-mon[42503]: pgmap v680: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 856 op/s 2024-05-09T00:07:34.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 280387 31.71 MB/sec execute 329 sec latency 215.136 ms 2024-05-09T00:07:35.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:34 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:07:35.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:34 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:07:35.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:34 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:07:35.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:34 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:07:35.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:34 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:07:35.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:34 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:07:35.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 280913 31.68 MB/sec execute 330 sec latency 119.689 ms 2024-05-09T00:07:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.984+0000 7f04de1d3700 1 -- 172.21.15.12:0/132800130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d8100c60 msgr2=0x7f04d810cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.984+0000 7f04de1d3700 1 --2- 172.21.15.12:0/132800130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d8100c60 0x7f04d810cbe0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f04d000d210 tx=0x7f04d000d520 comp rx=0 tx=0).stop 2024-05-09T00:07:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.984+0000 7f04de1d3700 1 -- 172.21.15.12:0/132800130 shutdown_connections 2024-05-09T00:07:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.984+0000 7f04de1d3700 1 --2- 172.21.15.12:0/132800130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d8100c60 0x7f04d810cbe0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.984+0000 7f04de1d3700 1 --2- 172.21.15.12:0/132800130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04d8105f40 0x7f04d8106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.984+0000 7f04de1d3700 1 --2- 172.21.15.12:0/132800130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04d81002a0 0x7f04d8100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.984+0000 7f04de1d3700 1 -- 172.21.15.12:0/132800130 >> 172.21.15.12:0/132800130 conn(0x7f04d8076c10 msgr2=0x7f04d8077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.984+0000 7f04de1d3700 1 -- 172.21.15.12:0/132800130 shutdown_connections 2024-05-09T00:07:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.984+0000 7f04de1d3700 1 -- 172.21.15.12:0/132800130 wait complete. 2024-05-09T00:07:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.985+0000 7f04de1d3700 1 Processor -- start 2024-05-09T00:07:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.985+0000 7f04de1d3700 1 -- start start 2024-05-09T00:07:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04de1d3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d81002a0 0x7f04d8195070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04de1d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04d8100c60 0x7f04d81955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04de1d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04d8105f40 0x7f04d8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04de1d3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8199f90 con 0x7f04d8105f40 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04de1d3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d819a100 con 0x7f04d8100c60 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04de1d3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d819a2c0 con 0x7f04d81002a0 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04d8100c60 0x7f04d81955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d81002a0 0x7f04d8195070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04d8100c60 0x7f04d81955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:57376/0 (socket says 172.21.15.12:57376) 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d6ffd700 1 -- 172.21.15.12:0/692654663 learned_addr learned my addr 172.21.15.12:0/692654663 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d7fff700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04d8105f40 0x7f04d8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d6ffd700 1 -- 172.21.15.12:0/692654663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d81002a0 msgr2=0x7f04d8195070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d6ffd700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d81002a0 0x7f04d8195070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d6ffd700 1 -- 172.21.15.12:0/692654663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04d8105f40 msgr2=0x7f04d8199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d6ffd700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04d8105f40 0x7f04d8199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d6ffd700 1 -- 172.21.15.12:0/692654663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d000be50 con 0x7f04d8100c60 2024-05-09T00:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.986+0000 7f04d6ffd700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04d8100c60 0x7f04d81955b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00e9c0 tx=0x7f04cc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.987+0000 7f04d4ff9700 1 -- 172.21.15.12:0/692654663 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc004a30 con 0x7f04d8100c60 2024-05-09T00:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.987+0000 7f04de1d3700 1 -- 172.21.15.12:0/692654663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d819a5a0 con 0x7f04d8100c60 2024-05-09T00:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.987+0000 7f04d4ff9700 1 -- 172.21.15.12:0/692654663 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04cc00cc80 con 0x7f04d8100c60 2024-05-09T00:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.987+0000 7f04d4ff9700 1 -- 172.21.15.12:0/692654663 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0054e0 con 0x7f04d8100c60 2024-05-09T00:07:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.987+0000 7f04de1d3700 1 -- 172.21.15.12:0/692654663 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d81a1de0 con 0x7f04d8100c60 2024-05-09T00:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.991+0000 7f04d4ff9700 1 -- 172.21.15.12:0/692654663 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f04cc00c7c0 con 0x7f04d8100c60 2024-05-09T00:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04ea38db80 1 Processor -- start 2024-05-09T00:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04ea38db80 1 -- start start 2024-05-09T00:07:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04d4ff9700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04c00a5720 0x7f04c00a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 0x5581f20e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20e8270 0x5581f20e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04ea38db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f20390f0 con 0x5581f20e2480 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04d7fff700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04c00a5720 0x7f04c00a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04ea38db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f2036ab0 con 0x5581f20e8270 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04ea38db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f20323a0 con 0x5581f20e7950 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.992+0000 7f04d4ff9700 1 -- 172.21.15.12:0/692654663 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04cc014070 con 0x7f04d8100c60 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 0x5581f20e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 0x5581f20e7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:50966/0 (socket says 172.21.15.12:50966) 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d77fe700 1 -- 172.21.15.12:0/406124497 learned_addr learned my addr 172.21.15.12:0/406124497 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d7fff700 1 --2- 172.21.15.12:0/406124497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d77fe700 1 -- 172.21.15.12:0/406124497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20e8270 msgr2=0x5581f20e1f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d77fe700 1 --2- 172.21.15.12:0/406124497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20e8270 0x5581f20e1f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d77fe700 1 -- 172.21.15.12:0/406124497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 msgr2=0x5581f20e28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d77fe700 1 --2- 172.21.15.12:0/406124497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d77fe700 1 -- 172.21.15.12:0/406124497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581f1eb3350 con 0x5581f20e7950 2024-05-09T00:07:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d7fff700 1 --2- 172.21.15.12:0/406124497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:07:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d7fff700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04c00a5720 0x7f04c00a7be0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f04d000b760 tx=0x7f04d000b4a0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04d77fe700 1 --2- 172.21.15.12:0/406124497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 0x5581f20e7d30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f04c800bcf0 tx=0x7f04c80103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892409151512b749 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.993+0000 7f04bdffb700 1 -- 172.21.15.12:0/406124497 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8010d00 con 0x5581f20e7950 2024-05-09T00:07:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.994+0000 7f04bdffb700 1 -- 172.21.15.12:0/406124497 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c8015070 con 0x5581f20e7950 2024-05-09T00:07:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.994+0000 7f04bdffb700 1 -- 172.21.15.12:0/406124497 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8005020 con 0x5581f20e7950 2024-05-09T00:07:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.994+0000 7f04ea38db80 1 -- 172.21.15.12:0/406124497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 msgr2=0x5581f20e7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.994+0000 7f04ea38db80 1 --2- 172.21.15.12:0/406124497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 0x5581f20e7d30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f04c800bcf0 tx=0x7f04c80103d0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.995+0000 7f04ea38db80 1 -- 172.21.15.12:0/406124497 shutdown_connections 2024-05-09T00:07:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.995+0000 7f04ea38db80 1 --2- 172.21.15.12:0/406124497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.995+0000 7f04ea38db80 1 --2- 172.21.15.12:0/406124497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 0x5581f20e7d30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.995+0000 7f04ea38db80 1 --2- 172.21.15.12:0/406124497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20e8270 0x5581f20e1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.995+0000 7f04ea38db80 1 -- 172.21.15.12:0/406124497 >> 172.21.15.12:0/406124497 conn(0x5581f20401b0 msgr2=0x5581f20dc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:35.998+0000 7f04ea38db80 1 -- 172.21.15.12:0/406124497 shutdown_connections 2024-05-09T00:07:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.000+0000 7f04ea38db80 1 -- 172.21.15.12:0/406124497 wait complete. 2024-05-09T00:07:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.001+0000 7f04ea38db80 1 -- start start 2024-05-09T00:07:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.001+0000 7f04ea38db80 1 Processor -- start 2024-05-09T00:07:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.001+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 0x5581f20eb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.001+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20e7950 0x5581f20ebba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.001+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20ec0e0 0x5581f20f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.001+0000 7f04ea38db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f1fce500 con 0x5581f20ec0e0 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.001+0000 7f04ea38db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f1fce1a0 con 0x5581f20e7950 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.001+0000 7f04ea38db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f1eb5640 con 0x5581f20e2480 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.001+0000 7f04ea38db80 10 client.0 _ll_register_callbacks cb 0x5581f1fd7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04d7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20ec0e0 0x5581f20f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04d7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20ec0e0 0x5581f20f7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41280/0 (socket says 172.21.15.12:41280) 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04d7fff700 1 -- 172.21.15.12:0/3498152497 learned_addr learned my addr 172.21.15.12:0/3498152497 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04d77fe700 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 0x5581f20eb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04d7fff700 1 -- 172.21.15.12:0/3498152497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 msgr2=0x5581f20eb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04d7fff700 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 0x5581f20eb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04d7fff700 1 -- 172.21.15.12:0/3498152497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20e7950 msgr2=0x5581f20ebba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04d7fff700 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20e7950 0x5581f20ebba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04d7fff700 1 -- 172.21.15.12:0/3498152497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c800b920 con 0x5581f20ec0e0 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04d7fff700 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20ec0e0 0x5581f20f7cf0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f04d00098b0 tx=0x7f04d000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbbc485b9dae307 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.002+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3498152497 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0010040 con 0x5581f20ec0e0 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3498152497 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04d0005900 con 0x5581f20ec0e0 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04ea38db80 1 -- 172.21.15.12:0/3498152497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581f1f8b210 con 0x5581f20ec0e0 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3498152497 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04d0003e90 con 0x5581f20ec0e0 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04ea38db80 10 client.21276 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04ea38db80 1 -- 172.21.15.12:0/3498152497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5581f1e5fb00 con 0x5581f20ec0e0 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3498152497 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581f1f8b210 con 0x5581f20ec0e0 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3498152497 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f04d00143f0 con 0x5581f20ec0e0 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04ea38db80 10 client.21276 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04ea38db80 10 client.21276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04ea38db80 20 client.21276 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581f20fc130 0x5581f20fe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04ea38db80 4 client.21276 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04ea38db80 1 -- 172.21.15.12:0/3498152497 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581f20b17b0 con 0x5581f20fc130 2024-05-09T00:07:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.003+0000 7f04d77fe700 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581f20fc130 0x5581f20fe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.004+0000 7f04d77fe700 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581f20fc130 0x5581f20fe550 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2abbf6068972c64 server_cookie=7b9e496c3b5585e9 in_seq=0 out_seq=0 2024-05-09T00:07:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.004+0000 7f04b6ffd700 10 client.21276 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3498152497 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581f20b17b0 con 0x5581f20fc130 2024-05-09T00:07:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04b6ffd700 10 client.21276 handle_command_reply: tid=0 2024-05-09T00:07:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04ea38db80 1 client.21276 shutdown 2024-05-09T00:07:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04ea38db80 1 -- 172.21.15.12:0/3498152497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20ec0e0 msgr2=0x5581f20f7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20ec0e0 0x5581f20f7cf0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f04d00098b0 tx=0x7f04d000d520 comp rx=0 tx=0).stop 2024-05-09T00:07:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04ea38db80 1 -- 172.21.15.12:0/3498152497 shutdown_connections 2024-05-09T00:07:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20ec0e0 0x5581f20f7cf0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 0x5581f20eb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581f20fc130 0x5581f20fe550 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3498152497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20e7950 0x5581f20ebba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04b6ffd700 0 client.21276 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04ea38db80 1 -- 172.21.15.12:0/3498152497 >> 172.21.15.12:0/3498152497 conn(0x5581f20401b0 msgr2=0x5581f20e0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.007+0000 7f04ea38db80 1 -- 172.21.15.12:0/3498152497 shutdown_connections 2024-05-09T00:07:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.008+0000 7f04ea38db80 1 -- 172.21.15.12:0/3498152497 wait complete. 2024-05-09T00:07:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.008+0000 7f04ea38db80 20 client.21276 trim_cache size 0 max 16384 2024-05-09T00:07:36.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:35 smithi160 ceph-mon[46152]: pgmap v681: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.4 MiB/s wr, 627 op/s 2024-05-09T00:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.025+0000 7f04ea38db80 1 Processor -- start 2024-05-09T00:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.025+0000 7f04ea38db80 1 -- start start 2024-05-09T00:07:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.025+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.025+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 0x5581f20f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.025+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 0x5581f20f3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.025+0000 7f04ea38db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f20f8c60 con 0x5581f20e2480 2024-05-09T00:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.025+0000 7f04ea38db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f20ee180 con 0x5581f20ec0e0 2024-05-09T00:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.025+0000 7f04ea38db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f20de300 con 0x5581f20e7950 2024-05-09T00:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41292/0 (socket says 172.21.15.12:41292) 2024-05-09T00:07:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d77fe700 1 -- 172.21.15.12:0/2826580338 learned_addr learned my addr 172.21.15.12:0/2826580338 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d6ffd700 1 --2- 172.21.15.12:0/2826580338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 0x5581f20f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d7fff700 1 --2- 172.21.15.12:0/2826580338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 0x5581f20f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d77fe700 1 -- 172.21.15.12:0/2826580338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 msgr2=0x5581f20f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d77fe700 1 --2- 172.21.15.12:0/2826580338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 0x5581f20f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d77fe700 1 -- 172.21.15.12:0/2826580338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 msgr2=0x5581f20f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d77fe700 1 --2- 172.21.15.12:0/2826580338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 0x5581f20f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d77fe700 1 -- 172.21.15.12:0/2826580338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581f1ff8500 con 0x5581f20e2480 2024-05-09T00:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04d77fe700 1 --2- 172.21.15.12:0/2826580338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e5a90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f04c8000c00 tx=0x7f04c8005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2cab9f5ebfb004 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.026+0000 7f04be7fc700 1 -- 172.21.15.12:0/2826580338 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8020040 con 0x5581f20e2480 2024-05-09T00:07:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04be7fc700 1 -- 172.21.15.12:0/2826580338 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c8021040 con 0x5581f20e2480 2024-05-09T00:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04be7fc700 1 -- 172.21.15.12:0/2826580338 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8011440 con 0x5581f20e2480 2024-05-09T00:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04ea38db80 1 -- 172.21.15.12:0/2826580338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 msgr2=0x5581f20e5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04ea38db80 1 --2- 172.21.15.12:0/2826580338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e5a90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f04c8000c00 tx=0x7f04c8005950 comp rx=0 tx=0).stop 2024-05-09T00:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04ea38db80 1 -- 172.21.15.12:0/2826580338 shutdown_connections 2024-05-09T00:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04ea38db80 1 --2- 172.21.15.12:0/2826580338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 0x5581f20f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04ea38db80 1 --2- 172.21.15.12:0/2826580338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e2480 0x5581f20e5a90 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04ea38db80 1 --2- 172.21.15.12:0/2826580338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e7950 0x5581f20f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04ea38db80 1 -- 172.21.15.12:0/2826580338 >> 172.21.15.12:0/2826580338 conn(0x5581f20401b0 msgr2=0x5581f20e5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04ea38db80 1 -- 172.21.15.12:0/2826580338 shutdown_connections 2024-05-09T00:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.027+0000 7f04ea38db80 1 -- 172.21.15.12:0/2826580338 wait complete. 2024-05-09T00:07:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.028+0000 7f04ea38db80 1 -- start start 2024-05-09T00:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.028+0000 7f04ea38db80 1 Processor -- start 2024-05-09T00:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.028+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 0x5581f20f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.028+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e7950 0x5581f20f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.028+0000 7f04ea38db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 0x5581f20f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.028+0000 7f04ea38db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f2151630 con 0x5581f20e7950 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04ea38db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f2151770 con 0x5581f20ec0e0 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04ea38db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f2151930 con 0x5581f20e2480 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04ea38db80 10 client.0 _ll_register_callbacks cb 0x5581f1fd7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e7950 0x5581f20f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e7950 0x5581f20f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41306/0 (socket says 172.21.15.12:41306) 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d6ffd700 1 -- 172.21.15.12:0/3731939942 learned_addr learned my addr 172.21.15.12:0/3731939942 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d77fe700 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 0x5581f20f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d7fff700 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 0x5581f20f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d77fe700 1 -- 172.21.15.12:0/3731939942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 msgr2=0x5581f20f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d77fe700 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 0x5581f20f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d77fe700 1 -- 172.21.15.12:0/3731939942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e7950 msgr2=0x5581f20f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d77fe700 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e7950 0x5581f20f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d77fe700 1 -- 172.21.15.12:0/3731939942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581f1eb3350 con 0x5581f20e2480 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d6ffd700 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e7950 0x5581f20f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d7fff700 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 0x5581f20f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04d77fe700 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 0x5581f20f1e60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f04c8000c00 tx=0x7f04c8028040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c8e69c42151ad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.029+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3731939942 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8010b00 con 0x5581f20e2480 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04ea38db80 1 -- 172.21.15.12:0/3731939942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04c8015070 con 0x5581f20e2480 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3731939942 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04c8036070 con 0x5581f20e2480 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3731939942 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04c8010c60 con 0x5581f20e2480 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04ea38db80 10 client.31006 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04ea38db80 1 -- 172.21.15.12:0/3731939942 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5581f1eb7c70 con 0x5581f20e2480 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3731939942 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c8015070 con 0x5581f20e2480 2024-05-09T00:07:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3731939942 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f04c8042070 con 0x5581f20e2480 2024-05-09T00:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04ea38db80 10 client.31006 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04ea38db80 10 client.31006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04ea38db80 20 client.31006 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581f2152c10 0x5581f2155000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04ea38db80 4 client.31006 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04ea38db80 1 -- 172.21.15.12:0/3731939942 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5581f202ae60 con 0x5581f2152c10 2024-05-09T00:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.030+0000 7f04d6ffd700 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581f2152c10 0x5581f2155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.031+0000 7f04d6ffd700 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581f2152c10 0x5581f2155000 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f403406432e7d6fc server_cookie=9e43ed61821df091 in_seq=0 out_seq=0 2024-05-09T00:07:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.031+0000 7f04b6ffd700 10 client.31006 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:36.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:35 smithi012 ceph-mon[42503]: pgmap v681: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.4 MiB/s wr, 627 op/s 2024-05-09T00:07:36.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:35 smithi204 ceph-mon[47373]: pgmap v681: 97 pgs: 97 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.4 MiB/s wr, 627 op/s 2024-05-09T00:07:36.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 281420 31.66 MB/sec execute 331 sec latency 241.935 ms 2024-05-09T00:07:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.536+0000 7f04b6ffd700 1 -- 172.21.15.12:0/3731939942 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5581f202ae60 con 0x5581f2152c10 2024-05-09T00:07:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.536+0000 7f04b6ffd700 10 client.31006 handle_command_reply: tid=0 2024-05-09T00:07:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.536+0000 7f04ea38db80 1 client.31006 shutdown 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.536+0000 7f04ea38db80 1 -- 172.21.15.12:0/3731939942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 msgr2=0x5581f20f1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.536+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 0x5581f20f1e60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f04c8000c00 tx=0x7f04c8028040 comp rx=0 tx=0).stop 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.537+0000 7f04ea38db80 1 -- 172.21.15.12:0/3731939942 shutdown_connections 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.537+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5581f2152c10 0x5581f2155000 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.537+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5581f20ec0e0 0x5581f20f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.537+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581f20e2480 0x5581f20f1e60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.537+0000 7f04ea38db80 1 --2- 172.21.15.12:0/3731939942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581f20e7950 0x5581f20f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.537+0000 7f04ea38db80 1 -- 172.21.15.12:0/3731939942 >> 172.21.15.12:0/3731939942 conn(0x5581f20401b0 msgr2=0x5581f20e5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.537+0000 7f04b6ffd700 0 client.31006 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.537+0000 7f04ea38db80 1 -- 172.21.15.12:0/3731939942 shutdown_connections 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.537+0000 7f04ea38db80 1 -- 172.21.15.12:0/3731939942 wait complete. 2024-05-09T00:07:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.537+0000 7f04ea38db80 20 client.31006 trim_cache size 0 max 16384 2024-05-09T00:07:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:07:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:07:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:07:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:07:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.538+0000 7f04de1d3700 1 -- 172.21.15.12:0/692654663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04c00a5720 msgr2=0x7f04c00a7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.538+0000 7f04de1d3700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04c00a5720 0x7f04c00a7be0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f04d000b760 tx=0x7f04d000b4a0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.538+0000 7f04de1d3700 1 -- 172.21.15.12:0/692654663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04d8100c60 msgr2=0x7f04d81955b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.538+0000 7f04de1d3700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04d8100c60 0x7f04d81955b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00e9c0 tx=0x7f04cc00ecd0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.539+0000 7f04de1d3700 1 -- 172.21.15.12:0/692654663 shutdown_connections 2024-05-09T00:07:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.539+0000 7f04de1d3700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04c00a5720 0x7f04c00a7be0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.539+0000 7f04de1d3700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04d8105f40 0x7f04d8199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.539+0000 7f04de1d3700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d81002a0 0x7f04d8195070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.539+0000 7f04de1d3700 1 --2- 172.21.15.12:0/692654663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04d8100c60 0x7f04d81955b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.539+0000 7f04de1d3700 1 -- 172.21.15.12:0/692654663 >> 172.21.15.12:0/692654663 conn(0x7f04d8076c10 msgr2=0x7f04d80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.540+0000 7f04de1d3700 1 -- 172.21.15.12:0/692654663 shutdown_connections 2024-05-09T00:07:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:36.540+0000 7f04de1d3700 1 -- 172.21.15.12:0/692654663 wait complete. 2024-05-09T00:07:37.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 281864 31.64 MB/sec execute 332 sec latency 362.909 ms 2024-05-09T00:07:38.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:37 smithi204 ceph-mon[47373]: pgmap v682: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 12 MiB/s wr, 858 op/s 2024-05-09T00:07:38.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:37 smithi160 ceph-mon[46152]: pgmap v682: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 12 MiB/s wr, 858 op/s 2024-05-09T00:07:38.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:37 smithi012 ceph-mon[42503]: pgmap v682: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 12 MiB/s wr, 858 op/s 2024-05-09T00:07:38.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 282371 31.63 MB/sec execute 333 sec latency 128.462 ms 2024-05-09T00:07:39.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 282885 31.62 MB/sec execute 334 sec latency 204.627 ms 2024-05-09T00:07:40.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:39 smithi204 ceph-mon[47373]: pgmap v683: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.3 MiB/s wr, 629 op/s 2024-05-09T00:07:40.015 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:39 smithi160 ceph-mon[46152]: pgmap v683: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.3 MiB/s wr, 629 op/s 2024-05-09T00:07:40.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:39 smithi012 ceph-mon[42503]: pgmap v683: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.3 MiB/s wr, 629 op/s 2024-05-09T00:07:40.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 283360 31.61 MB/sec execute 335 sec latency 121.221 ms 2024-05-09T00:07:41.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 283941 31.59 MB/sec execute 336 sec latency 104.095 ms 2024-05-09T00:07:42.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:41 smithi160 ceph-mon[46152]: pgmap v684: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.9 MiB/s wr, 618 op/s 2024-05-09T00:07:42.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:41 smithi012 ceph-mon[42503]: pgmap v684: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.9 MiB/s wr, 618 op/s 2024-05-09T00:07:42.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:41 smithi204 ceph-mon[47373]: pgmap v684: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.9 MiB/s wr, 618 op/s 2024-05-09T00:07:42.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 284521 31.60 MB/sec execute 337 sec latency 232.790 ms 2024-05-09T00:07:43.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 285065 31.58 MB/sec execute 338 sec latency 106.846 ms 2024-05-09T00:07:44.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:43 smithi012 ceph-mon[42503]: pgmap v685: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 760 op/s 2024-05-09T00:07:44.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:43 smithi204 ceph-mon[47373]: pgmap v685: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 760 op/s 2024-05-09T00:07:44.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:43 smithi160 ceph-mon[46152]: pgmap v685: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 760 op/s 2024-05-09T00:07:44.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 285524 31.56 MB/sec execute 339 sec latency 352.828 ms 2024-05-09T00:07:45.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 286054 31.57 MB/sec execute 340 sec latency 145.818 ms 2024-05-09T00:07:46.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:45 smithi160 ceph-mon[46152]: pgmap v686: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.5 MiB/s wr, 533 op/s 2024-05-09T00:07:46.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:45 smithi012 ceph-mon[42503]: pgmap v686: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.5 MiB/s wr, 533 op/s 2024-05-09T00:07:46.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:45 smithi204 ceph-mon[47373]: pgmap v686: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.5 MiB/s wr, 533 op/s 2024-05-09T00:07:46.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 286612 31.57 MB/sec execute 341 sec latency 105.056 ms 2024-05-09T00:07:47.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 287151 31.55 MB/sec execute 342 sec latency 101.584 ms 2024-05-09T00:07:48.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:47 smithi204 ceph-mon[47373]: pgmap v687: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 711 op/s 2024-05-09T00:07:48.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:47 smithi160 ceph-mon[46152]: pgmap v687: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 711 op/s 2024-05-09T00:07:48.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:47 smithi012 ceph-mon[42503]: pgmap v687: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 711 op/s 2024-05-09T00:07:48.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 287730 31.53 MB/sec execute 343 sec latency 99.510 ms 2024-05-09T00:07:49.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 288220 31.52 MB/sec execute 344 sec latency 247.223 ms 2024-05-09T00:07:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.705+0000 7f8b51546700 1 -- 172.21.15.12:0/3518684567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b4c069730 msgr2=0x7f8b4c069b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.705+0000 7f8b51546700 1 --2- 172.21.15.12:0/3518684567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b4c069730 0x7f8b4c069b90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009990 tx=0x7f8b40009ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.705+0000 7f8b51546700 1 -- 172.21.15.12:0/3518684567 shutdown_connections 2024-05-09T00:07:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.705+0000 7f8b51546700 1 --2- 172.21.15.12:0/3518684567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b4c06a0d0 0x7f8b4c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.705+0000 7f8b51546700 1 --2- 172.21.15.12:0/3518684567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c106f50 0x7f8b4c107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.705+0000 7f8b51546700 1 --2- 172.21.15.12:0/3518684567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b4c069730 0x7f8b4c069b90 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.705+0000 7f8b51546700 1 -- 172.21.15.12:0/3518684567 >> 172.21.15.12:0/3518684567 conn(0x7f8b4c076fd0 msgr2=0x7f8b4c0773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.705+0000 7f8b51546700 1 -- 172.21.15.12:0/3518684567 shutdown_connections 2024-05-09T00:07:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.707+0000 7f8b51546700 1 -- 172.21.15.12:0/3518684567 wait complete. 2024-05-09T00:07:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.707+0000 7f8b51546700 1 Processor -- start 2024-05-09T00:07:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.707+0000 7f8b51546700 1 -- start start 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.708+0000 7f8b51546700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b4c069730 0x7f8b4c199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.708+0000 7f8b51546700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b4c06a0d0 0x7f8b4c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.708+0000 7f8b51546700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c106f50 0x7f8b4c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.708+0000 7f8b51546700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c19e440 con 0x7f8b4c06a0d0 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.708+0000 7f8b51546700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c19e5b0 con 0x7f8b4c069730 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.708+0000 7f8b51546700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c19e770 con 0x7f8b4c106f50 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.708+0000 7f8b4a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b4c06a0d0 0x7f8b4c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.708+0000 7f8b4a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b4c06a0d0 0x7f8b4c199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60562/0 (socket says 172.21.15.12:60562) 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.708+0000 7f8b4a7fc700 1 -- 172.21.15.12:0/1420536222 learned_addr learned my addr 172.21.15.12:0/1420536222 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.708+0000 7f8b4b7fe700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c106f50 0x7f8b4c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b4a7fc700 1 -- 172.21.15.12:0/1420536222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c106f50 msgr2=0x7f8b4c19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c106f50 0x7f8b4c19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b4a7fc700 1 -- 172.21.15.12:0/1420536222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b4c069730 msgr2=0x7f8b4c199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b4affd700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b4c069730 0x7f8b4c199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b4c069730 0x7f8b4c199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b4a7fc700 1 -- 172.21.15.12:0/1420536222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b400095f0 con 0x7f8b4c06a0d0 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b4b7fe700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c106f50 0x7f8b4c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b4affd700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b4c069730 0x7f8b4c199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b4c06a0d0 0x7f8b4c199a60 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005770 tx=0x7f8b400057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b33fff700 1 -- 172.21.15.12:0/1420536222 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4001d070 con 0x7f8b4c06a0d0 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b51546700 1 -- 172.21.15.12:0/1420536222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c19e9f0 con 0x7f8b4c06a0d0 2024-05-09T00:07:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b33fff700 1 -- 172.21.15.12:0/1420536222 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b40021410 con 0x7f8b4c06a0d0 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b33fff700 1 -- 172.21.15.12:0/1420536222 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40017650 con 0x7f8b4c06a0d0 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.709+0000 7f8b51546700 1 -- 172.21.15.12:0/1420536222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c1a6290 con 0x7f8b4c06a0d0 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.714+0000 7f8b33fff700 1 -- 172.21.15.12:0/1420536222 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8b400217f0 con 0x7f8b4c06a0d0 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.714+0000 7f8b5d700b80 1 Processor -- start 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b5d700b80 1 -- start start 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee118f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f3480 0x55ee118f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1184a0f0 con 0x55ee118f8950 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee11847ab0 con 0x55ee118f3480 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee118433a0 con 0x55ee118f9270 2024-05-09T00:07:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b33fff700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b340a1290 0x7f8b340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b4affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b4b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee118f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b4affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44732/0 (socket says 172.21.15.12:44732) 2024-05-09T00:07:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b4affd700 1 -- 172.21.15.12:0/595266310 learned_addr learned my addr 172.21.15.12:0/595266310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b33fff700 1 -- 172.21.15.12:0/1420536222 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8b400cdff0 con 0x7f8b4c06a0d0 2024-05-09T00:07:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.715+0000 7f8b4b7fe700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b340a1290 0x7f8b340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.716+0000 7f8b4affd700 1 -- 172.21.15.12:0/595266310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f3480 msgr2=0x55ee118f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.716+0000 7f8b4affd700 1 --2- 172.21.15.12:0/595266310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f3480 0x55ee118f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.716+0000 7f8b4affd700 1 -- 172.21.15.12:0/595266310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 msgr2=0x55ee118f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.716+0000 7f8b4affd700 1 --2- 172.21.15.12:0/595266310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee118f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.716+0000 7f8b4affd700 1 -- 172.21.15.12:0/595266310 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee116c4350 con 0x55ee118f9270 2024-05-09T00:07:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.716+0000 7f8b4affd700 1 --2- 172.21.15.12:0/595266310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118f2f40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009ca0 tx=0x7f8b3c00f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=809b7d91be361005 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.716+0000 7f8b317fa700 1 -- 172.21.15.12:0/595266310 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00fbe0 con 0x55ee118f9270 2024-05-09T00:07:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.717+0000 7f8b317fa700 1 -- 172.21.15.12:0/595266310 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3c0044f0 con 0x55ee118f9270 2024-05-09T00:07:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.717+0000 7f8b317fa700 1 -- 172.21.15.12:0/595266310 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c011440 con 0x55ee118f9270 2024-05-09T00:07:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.718+0000 7f8b5d700b80 1 -- 172.21.15.12:0/595266310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 msgr2=0x55ee118f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.718+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/595266310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118f2f40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009ca0 tx=0x7f8b3c00f360 comp rx=0 tx=0).stop 2024-05-09T00:07:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.719+0000 7f8b4b7fe700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b340a1290 0x7f8b340a3750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009550 tx=0x7f8b44009050 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.723+0000 7f8b5d700b80 1 -- 172.21.15.12:0/595266310 shutdown_connections 2024-05-09T00:07:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.723+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/595266310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f3480 0x55ee118f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.723+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/595266310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee118f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.723+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/595266310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118f2f40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.723+0000 7f8b5d700b80 1 -- 172.21.15.12:0/595266310 >> 172.21.15.12:0/595266310 conn(0x55ee118511b0 msgr2=0x55ee118ed840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.723+0000 7f8b5d700b80 1 -- 172.21.15.12:0/595266310 shutdown_connections 2024-05-09T00:07:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.723+0000 7f8b5d700b80 1 -- 172.21.15.12:0/595266310 wait complete. 2024-05-09T00:07:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.724+0000 7f8b5d700b80 1 -- start start 2024-05-09T00:07:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.724+0000 7f8b5d700b80 1 Processor -- start 2024-05-09T00:07:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.724+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f3480 0x55ee118fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.725+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee118fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.725+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118fd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.725+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee117df500 con 0x55ee118f8950 2024-05-09T00:07:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.725+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee117df1a0 con 0x55ee118f3480 2024-05-09T00:07:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.725+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee116c6640 con 0x55ee118f9270 2024-05-09T00:07:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.725+0000 7f8b5d700b80 10 client.0 _ll_register_callbacks cb 0x55ee117e8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.728+0000 7f8b4b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.728+0000 7f8b4b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118fd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:44734/0 (socket says 172.21.15.12:44734) 2024-05-09T00:07:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.728+0000 7f8b4b7fe700 1 -- 172.21.15.12:0/3109959807 learned_addr learned my addr 172.21.15.12:0/3109959807 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.728+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee118fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.729+0000 7f8b4a7fc700 1 -- 172.21.15.12:0/3109959807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 msgr2=0x55ee118fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.729+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118fd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.729+0000 7f8b4a7fc700 1 -- 172.21.15.12:0/3109959807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f3480 msgr2=0x55ee118fc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.729+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f3480 0x55ee118fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.729+0000 7f8b4a7fc700 1 -- 172.21.15.12:0/3109959807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c009900 con 0x55ee118f8950 2024-05-09T00:07:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.729+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee118fcb40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f8b400cb750 tx=0x7f8b40096eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e605b2fe6841b726 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.731+0000 7f8b2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.731+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3109959807 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400cbea0 con 0x55ee118f8950 2024-05-09T00:07:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.731+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3109959807 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8b4006c070 con 0x55ee118f8950 2024-05-09T00:07:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3109959807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee1179c210 con 0x55ee118f8950 2024-05-09T00:07:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3109959807 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b4009ab00 con 0x55ee118f8950 2024-05-09T00:07:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b5d700b80 10 client.21294 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3109959807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55ee11670b00 con 0x55ee118f8950 2024-05-09T00:07:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3109959807 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee1179c210 con 0x55ee118f8950 2024-05-09T00:07:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3109959807 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b4009a690 con 0x55ee118f8950 2024-05-09T00:07:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b5d700b80 10 client.21294 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b5d700b80 10 client.21294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b5d700b80 20 client.21294 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ee11915000 0x55ee11917420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b5d700b80 4 client.21294 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.732+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3109959807 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee118c27b0 con 0x55ee11915000 2024-05-09T00:07:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.734+0000 7f8b4affd700 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ee11915000 0x55ee11917420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.736+0000 7f8b4affd700 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ee11915000 0x55ee11917420 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ddb96fc9ff011a server_cookie=c5d84f48e11c5fe2 in_seq=0 out_seq=0 2024-05-09T00:07:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.736+0000 7f8b2eff5700 10 client.21294 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.747+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3109959807 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee118c27b0 con 0x55ee11915000 2024-05-09T00:07:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.747+0000 7f8b2eff5700 10 client.21294 handle_command_reply: tid=0 2024-05-09T00:07:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.747+0000 7f8b5d700b80 1 client.21294 shutdown 2024-05-09T00:07:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.747+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3109959807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 msgr2=0x55ee118fcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.747+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee118fcb40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f8b400cb750 tx=0x7f8b40096eb0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.749+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3109959807 shutdown_connections 2024-05-09T00:07:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.749+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f9270 0x55ee118fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.749+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ee11915000 0x55ee11917420 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.750+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f3480 0x55ee118fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.750+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3109959807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee118fcb40 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.750+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3109959807 >> 172.21.15.12:0/3109959807 conn(0x55ee118511b0 msgr2=0x55ee118f74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.750+0000 7f8b2eff5700 0 client.21294 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.750+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3109959807 shutdown_connections 2024-05-09T00:07:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.750+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3109959807 wait complete. 2024-05-09T00:07:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.750+0000 7f8b5d700b80 20 client.21294 trim_cache size 0 max 16384 2024-05-09T00:07:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.773+0000 7f8b5d700b80 1 Processor -- start 2024-05-09T00:07:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.773+0000 7f8b5d700b80 1 -- start start 2024-05-09T00:07:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.773+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118511b0 0x55ee118ed910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.773+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f3480 0x55ee1190c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.773+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee1190c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.773+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee118e8330 con 0x55ee118f8950 2024-05-09T00:07:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.773+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee118fdc00 con 0x55ee118511b0 2024-05-09T00:07:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.774+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1191c640 con 0x55ee118f3480 2024-05-09T00:07:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.774+0000 7f8b4affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118511b0 0x55ee118ed910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.774+0000 7f8b4affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118511b0 0x55ee118ed910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:38966/0 (socket says 172.21.15.12:38966) 2024-05-09T00:07:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.774+0000 7f8b4affd700 1 -- 172.21.15.12:0/962515956 learned_addr learned my addr 172.21.15.12:0/962515956 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.774+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/962515956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f3480 0x55ee1190c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.775+0000 7f8b4b7fe700 1 --2- 172.21.15.12:0/962515956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee1190c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.775+0000 7f8b4a7fc700 1 -- 172.21.15.12:0/962515956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118511b0 msgr2=0x55ee118ed910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.775+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/962515956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118511b0 0x55ee118ed910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.775+0000 7f8b4a7fc700 1 -- 172.21.15.12:0/962515956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 msgr2=0x55ee1190c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.775+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/962515956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee1190c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.775+0000 7f8b4a7fc700 1 -- 172.21.15.12:0/962515956 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee11809500 con 0x55ee118f3480 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.775+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/962515956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f3480 0x55ee1190c490 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55ee118e5a80 tx=0x7f8b40099750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e35ad3afb297ba93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.777+0000 7f8b31ffb700 1 -- 172.21.15.12:0/962515956 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400cbea0 con 0x55ee118f3480 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.777+0000 7f8b31ffb700 1 -- 172.21.15.12:0/962515956 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b4009ab00 con 0x55ee118f3480 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.777+0000 7f8b31ffb700 1 -- 172.21.15.12:0/962515956 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4009efd0 con 0x55ee118f3480 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.778+0000 7f8b5d700b80 1 -- 172.21.15.12:0/962515956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f3480 msgr2=0x55ee1190c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.778+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/962515956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f3480 0x55ee1190c490 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55ee118e5a80 tx=0x7f8b40099750 comp rx=0 tx=0).stop 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.778+0000 7f8b5d700b80 1 -- 172.21.15.12:0/962515956 shutdown_connections 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.778+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/962515956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118f8950 0x55ee1190c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.778+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/962515956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118511b0 0x55ee118ed910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.778+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/962515956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f3480 0x55ee1190c490 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.778+0000 7f8b5d700b80 1 -- 172.21.15.12:0/962515956 >> 172.21.15.12:0/962515956 conn(0x55ee118f3b50 msgr2=0x55ee118f1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.778+0000 7f8b5d700b80 1 -- 172.21.15.12:0/962515956 shutdown_connections 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.778+0000 7f8b5d700b80 1 -- 172.21.15.12:0/962515956 wait complete. 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.778+0000 7f8b5d700b80 1 -- start start 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b5d700b80 1 Processor -- start 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118511b0 0x55ee11908470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f3480 0x55ee119089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b5d700b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f8950 0x55ee11908ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1196a4f0 con 0x55ee118511b0 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1196a660 con 0x55ee118f8950 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b5d700b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1196a820 con 0x55ee118f3480 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b5d700b80 10 client.0 _ll_register_callbacks cb 0x55ee117e8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b4affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118511b0 0x55ee11908470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b4affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118511b0 0x55ee11908470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60590/0 (socket says 172.21.15.12:60590) 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b4affd700 1 -- 172.21.15.12:0/3307188785 learned_addr learned my addr 172.21.15.12:0/3307188785 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.779+0000 7f8b4b7fe700 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f8950 0x55ee11908ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.780+0000 7f8b4affd700 1 -- 172.21.15.12:0/3307188785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f3480 msgr2=0x55ee119089b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.780+0000 7f8b4affd700 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f3480 0x55ee119089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.780+0000 7f8b4affd700 1 -- 172.21.15.12:0/3307188785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f8950 msgr2=0x55ee11908ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.780+0000 7f8b4affd700 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f8950 0x55ee11908ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.780+0000 7f8b4affd700 1 -- 172.21.15.12:0/3307188785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee11670b00 con 0x55ee118511b0 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.780+0000 7f8b4affd700 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118511b0 0x55ee11908470 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00a010 tx=0x7f8b3c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6089e7367966455 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.780+0000 7f8b2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.780+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3307188785 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c021040 con 0x55ee118511b0 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.780+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3307188785 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8b3c00fdf0 con 0x55ee118511b0 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.780+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3307188785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b3c0044f0 con 0x55ee118511b0 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.781+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3307188785 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b3c026420 con 0x55ee118511b0 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.781+0000 7f8b5d700b80 10 client.21300 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.781+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3307188785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55ee116c8c70 con 0x55ee118511b0 2024-05-09T00:07:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.782+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3307188785 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3c0044f0 con 0x55ee118511b0 2024-05-09T00:07:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.782+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3307188785 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b3c02f070 con 0x55ee118511b0 2024-05-09T00:07:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.782+0000 7f8b5d700b80 10 client.21300 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.782+0000 7f8b5d700b80 10 client.21300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.782+0000 7f8b5d700b80 20 client.21300 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.782+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ee118f9270 0x55ee1196ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.782+0000 7f8b5d700b80 4 client.21300 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:07:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.782+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3307188785 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ee1183be60 con 0x55ee118f9270 2024-05-09T00:07:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.782+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ee118f9270 0x55ee1196ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.784+0000 7f8b4a7fc700 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ee118f9270 0x55ee1196ddf0 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20686e134e842f08 server_cookie=45c5ee6cf43a00ad in_seq=0 out_seq=0 2024-05-09T00:07:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:49.786+0000 7f8b2eff5700 10 client.21300 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:50.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:49 smithi012 ceph-mon[42503]: pgmap v688: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.0 MiB/s wr, 495 op/s 2024-05-09T00:07:50.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:49 smithi204 ceph-mon[47373]: pgmap v688: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.0 MiB/s wr, 495 op/s 2024-05-09T00:07:50.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:49 smithi160 ceph-mon[46152]: pgmap v688: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.0 MiB/s wr, 495 op/s 2024-05-09T00:07:50.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 288546 31.49 MB/sec execute 345 sec latency 363.687 ms 2024-05-09T00:07:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b2eff5700 1 -- 172.21.15.12:0/3307188785 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ee1183be60 con 0x55ee118f9270 2024-05-09T00:07:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b2eff5700 10 client.21300 handle_command_reply: tid=0 2024-05-09T00:07:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b5d700b80 1 client.21300 shutdown 2024-05-09T00:07:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3307188785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118511b0 msgr2=0x55ee11908470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118511b0 0x55ee11908470 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00a010 tx=0x7f8b3c009440 comp rx=0 tx=0).stop 2024-05-09T00:07:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3307188785 shutdown_connections 2024-05-09T00:07:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55ee118f9270 0x55ee1196ddf0 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ee118f8950 0x55ee11908ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee118511b0 0x55ee11908470 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b5d700b80 1 --2- 172.21.15.12:0/3307188785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee118f3480 0x55ee119089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b2eff5700 0 client.21300 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.346+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3307188785 >> 172.21.15.12:0/3307188785 conn(0x55ee118f3b50 msgr2=0x55ee118f2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3307188785 shutdown_connections 2024-05-09T00:07:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b5d700b80 1 -- 172.21.15.12:0/3307188785 wait complete. 2024-05-09T00:07:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b5d700b80 20 client.21300 trim_cache size 0 max 16384 2024-05-09T00:07:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:07:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:07:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:07:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:07:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b51546700 1 -- 172.21.15.12:0/1420536222 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b340a1290 msgr2=0x7f8b340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b51546700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b340a1290 0x7f8b340a3750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009550 tx=0x7f8b44009050 comp rx=0 tx=0).stop 2024-05-09T00:07:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b51546700 1 -- 172.21.15.12:0/1420536222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b4c06a0d0 msgr2=0x7f8b4c199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b51546700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b4c06a0d0 0x7f8b4c199a60 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005770 tx=0x7f8b400057a0 comp rx=0 tx=0).stop 2024-05-09T00:07:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b51546700 1 -- 172.21.15.12:0/1420536222 shutdown_connections 2024-05-09T00:07:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b51546700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8b340a1290 0x7f8b340a3750 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b51546700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c106f50 0x7f8b4c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b51546700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b4c069730 0x7f8b4c199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b51546700 1 --2- 172.21.15.12:0/1420536222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b4c06a0d0 0x7f8b4c199a60 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.347+0000 7f8b51546700 1 -- 172.21.15.12:0/1420536222 >> 172.21.15.12:0/1420536222 conn(0x7f8b4c076fd0 msgr2=0x7f8b4c10c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.348+0000 7f8b51546700 1 -- 172.21.15.12:0/1420536222 shutdown_connections 2024-05-09T00:07:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:07:50.348+0000 7f8b51546700 1 -- 172.21.15.12:0/1420536222 wait complete. 2024-05-09T00:07:51.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 289142 31.52 MB/sec execute 346 sec latency 439.330 ms 2024-05-09T00:07:52.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:51 smithi204 ceph-mon[47373]: pgmap v689: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.5 MiB/s wr, 499 op/s 2024-05-09T00:07:52.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:51 smithi160 ceph-mon[46152]: pgmap v689: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.5 MiB/s wr, 499 op/s 2024-05-09T00:07:52.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:51 smithi012 ceph-mon[42503]: pgmap v689: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.5 MiB/s wr, 499 op/s 2024-05-09T00:07:52.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 289596 31.47 MB/sec execute 347 sec latency 117.509 ms 2024-05-09T00:07:53.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 290219 31.48 MB/sec execute 348 sec latency 108.498 ms 2024-05-09T00:07:54.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:53 smithi160 ceph-mon[46152]: pgmap v690: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-05-09T00:07:54.023 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:53 smithi204 ceph-mon[47373]: pgmap v690: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-05-09T00:07:54.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:53 smithi012 ceph-mon[42503]: pgmap v690: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-05-09T00:07:54.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 290559 31.47 MB/sec execute 349 sec latency 314.642 ms 2024-05-09T00:07:55.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 290879 31.39 MB/sec execute 350 sec latency 193.050 ms 2024-05-09T00:07:56.013 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:55 smithi204 ceph-mon[47373]: pgmap v691: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.7 MiB/s wr, 559 op/s 2024-05-09T00:07:56.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:55 smithi160 ceph-mon[46152]: pgmap v691: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.7 MiB/s wr, 559 op/s 2024-05-09T00:07:56.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:55 smithi012 ceph-mon[42503]: pgmap v691: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.7 MiB/s wr, 559 op/s 2024-05-09T00:07:56.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 291675 31.42 MB/sec execute 351 sec latency 78.376 ms 2024-05-09T00:07:57.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 292354 31.47 MB/sec execute 352 sec latency 121.513 ms 2024-05-09T00:07:57.653 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:07:57.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.829+0000 7f2e13e84700 1 -- 172.21.15.12:0/3844660730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0c100060 msgr2=0x7f2e0c100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.829+0000 7f2e13e84700 1 --2- 172.21.15.12:0/3844660730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0c100060 0x7f2e0c100440 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009990 tx=0x7f2e08009ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.829+0000 7f2e13e84700 1 -- 172.21.15.12:0/3844660730 shutdown_connections 2024-05-09T00:07:57.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.829+0000 7f2e13e84700 1 --2- 172.21.15.12:0/3844660730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e0c105160 0x7f2e0c10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.829+0000 7f2e13e84700 1 --2- 172.21.15.12:0/3844660730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0c100060 0x7f2e0c100440 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.829+0000 7f2e13e84700 1 --2- 172.21.15.12:0/3844660730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0c100a10 0x7f2e0c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.829+0000 7f2e13e84700 1 -- 172.21.15.12:0/3844660730 >> 172.21.15.12:0/3844660730 conn(0x7f2e0c076f40 msgr2=0x7f2e0c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:57.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.829+0000 7f2e13e84700 1 -- 172.21.15.12:0/3844660730 shutdown_connections 2024-05-09T00:07:57.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.829+0000 7f2e13e84700 1 -- 172.21.15.12:0/3844660730 wait complete. 2024-05-09T00:07:57.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.830+0000 7f2e13e84700 1 Processor -- start 2024-05-09T00:07:57.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.830+0000 7f2e13e84700 1 -- start start 2024-05-09T00:07:57.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.830+0000 7f2e13e84700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e0c100060 0x7f2e0c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.830+0000 7f2e13e84700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0c105160 0x7f2e0c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.830+0000 7f2e13e84700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0c198d50 0x7f2e0c1a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.830+0000 7f2e13e84700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c1a39e0 con 0x7f2e0c198d50 2024-05-09T00:07:57.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.830+0000 7f2e13e84700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c1a3b50 con 0x7f2e0c100060 2024-05-09T00:07:57.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.830+0000 7f2e13e84700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c1a3d10 con 0x7f2e0c105160 2024-05-09T00:07:57.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e12421700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0c198d50 0x7f2e0c1a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e11c20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e0c100060 0x7f2e0c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e12421700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0c198d50 0x7f2e0c1a3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60602/0 (socket says 172.21.15.12:60602) 2024-05-09T00:07:57.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e12421700 1 -- 172.21.15.12:0/2910754496 learned_addr learned my addr 172.21.15.12:0/2910754496 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:57.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e1141f700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0c105160 0x7f2e0c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e12421700 1 -- 172.21.15.12:0/2910754496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0c105160 msgr2=0x7f2e0c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e12421700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0c105160 0x7f2e0c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e12421700 1 -- 172.21.15.12:0/2910754496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e0c100060 msgr2=0x7f2e0c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e12421700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e0c100060 0x7f2e0c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e12421700 1 -- 172.21.15.12:0/2910754496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e080095f0 con 0x7f2e0c198d50 2024-05-09T00:07:57.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e11c20700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e0c100060 0x7f2e0c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:57.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e12421700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0c198d50 0x7f2e0c1a3350 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400ef30 tx=0x7f2e0400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:57.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e02ffd700 1 -- 172.21.15.12:0/2910754496 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04018070 con 0x7f2e0c198d50 2024-05-09T00:07:57.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e02ffd700 1 -- 172.21.15.12:0/2910754496 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e0400f040 con 0x7f2e0c198d50 2024-05-09T00:07:57.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.831+0000 7f2e13e84700 1 -- 172.21.15.12:0/2910754496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0c1a3f90 con 0x7f2e0c198d50 2024-05-09T00:07:57.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.832+0000 7f2e02ffd700 1 -- 172.21.15.12:0/2910754496 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0400cd00 con 0x7f2e0c198d50 2024-05-09T00:07:57.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.832+0000 7f2e13e84700 1 -- 172.21.15.12:0/2910754496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c1a4470 con 0x7f2e0c198d50 2024-05-09T00:07:57.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.834+0000 7f2e02ffd700 1 -- 172.21.15.12:0/2910754496 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2e04003a00 con 0x7f2e0c198d50 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.835+0000 7f2e02ffd700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2df80a1310 0x7f2df80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.835+0000 7f2e11c20700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2df80a1310 0x7f2df80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.835+0000 7f2e02ffd700 1 -- 172.21.15.12:0/2910754496 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2e040cc760 con 0x7f2e0c198d50 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.835+0000 7f2e11c20700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2df80a1310 0x7f2df80a37d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800b2e0 tx=0x7f2e08005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e2003eb80 1 Processor -- start 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e2003eb80 1 -- start start 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6d0720 0x5620ef6ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6ca890 0x5620ef6cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef621150 con 0x5620ef6ca890 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef61eb10 con 0x5620ef6cfe00 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef61a400 con 0x5620ef6d0720 2024-05-09T00:07:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e11c20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e12421700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6ca890 0x5620ef6cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e1141f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6d0720 0x5620ef6ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e11c20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:38992/0 (socket says 172.21.15.12:38992) 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.836+0000 7f2e11c20700 1 -- 172.21.15.12:0/293989701 learned_addr learned my addr 172.21.15.12:0/293989701 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.837+0000 7f2e12421700 1 -- 172.21.15.12:0/293989701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6d0720 msgr2=0x5620ef6ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.837+0000 7f2e12421700 1 --2- 172.21.15.12:0/293989701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6d0720 0x5620ef6ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.837+0000 7f2e12421700 1 -- 172.21.15.12:0/293989701 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 msgr2=0x5620ef6d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.837+0000 7f2e12421700 1 --2- 172.21.15.12:0/293989701 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.837+0000 7f2e12421700 1 -- 172.21.15.12:0/293989701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620ef49b3b0 con 0x5620ef6ca890 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.837+0000 7f2e12421700 1 --2- 172.21.15.12:0/293989701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6ca890 0x5620ef6cacd0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2e040c9ec0 tx=0x7f2e04099450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605e6d7bad53834b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.837+0000 7f2df7fff700 1 -- 172.21.15.12:0/293989701 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04004ad0 con 0x5620ef6ca890 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.837+0000 7f2df7fff700 1 -- 172.21.15.12:0/293989701 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e0401b320 con 0x5620ef6ca890 2024-05-09T00:07:57.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.837+0000 7f2df7fff700 1 -- 172.21.15.12:0/293989701 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e040b90c0 con 0x5620ef6ca890 2024-05-09T00:07:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.838+0000 7f2e2003eb80 1 -- 172.21.15.12:0/293989701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6ca890 msgr2=0x5620ef6cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.838+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/293989701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6ca890 0x5620ef6cacd0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2e040c9ec0 tx=0x7f2e04099450 comp rx=0 tx=0).stop 2024-05-09T00:07:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.838+0000 7f2e2003eb80 1 -- 172.21.15.12:0/293989701 shutdown_connections 2024-05-09T00:07:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.838+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/293989701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6ca890 0x5620ef6cacd0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.838+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/293989701 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.838+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/293989701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6d0720 0x5620ef6ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.838+0000 7f2e2003eb80 1 -- 172.21.15.12:0/293989701 >> 172.21.15.12:0/293989701 conn(0x5620ef628210 msgr2=0x5620ef6c4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.838+0000 7f2e2003eb80 1 -- 172.21.15.12:0/293989701 shutdown_connections 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.838+0000 7f2e2003eb80 1 -- 172.21.15.12:0/293989701 wait complete. 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.838+0000 7f2e2003eb80 1 -- start start 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e2003eb80 1 Processor -- start 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6d4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef5b6560 con 0x5620ef6d0720 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef5b6200 con 0x5620ef6cfe00 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef49d6a0 con 0x5620ef6ca890 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e2003eb80 10 client.0 _ll_register_callbacks cb 0x5620ef5bfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e12421700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e12421700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6d4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60620/0 (socket says 172.21.15.12:60620) 2024-05-09T00:07:57.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e12421700 1 -- 172.21.15.12:0/2030829294 learned_addr learned my addr 172.21.15.12:0/2030829294 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:57.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e11c20700 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.839+0000 7f2e1141f700 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6d4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2e12421700 1 -- 172.21.15.12:0/2030829294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 msgr2=0x5620ef6d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:57.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2e12421700 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2e12421700 1 -- 172.21.15.12:0/2030829294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 msgr2=0x5620ef6d4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:57.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2e12421700 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6d4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2e12421700 1 -- 172.21.15.12:0/2030829294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e040b7e70 con 0x5620ef6d0720 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2e12421700 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6d4550 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2e040ca4c0 tx=0x7f2e04099e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6110a785148b2741 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2df57fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2df57fa700 1 -- 172.21.15.12:0/2030829294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e040b2260 con 0x5620ef6d0720 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2df57fa700 1 -- 172.21.15.12:0/2030829294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2e0409a010 con 0x5620ef6d0720 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2e2003eb80 1 -- 172.21.15.12:0/2030829294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620ef573270 con 0x5620ef6d0720 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2df57fa700 1 -- 172.21.15.12:0/2030829294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e04099910 con 0x5620ef6d0720 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2e2003eb80 10 client.21312 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2e2003eb80 1 -- 172.21.15.12:0/2030829294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5620ef447b60 con 0x5620ef6d0720 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.840+0000 7f2df57fa700 1 -- 172.21.15.12:0/2030829294 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620ef573270 con 0x5620ef6d0720 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.841+0000 7f2df57fa700 1 -- 172.21.15.12:0/2030829294 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2e040b9050 con 0x5620ef6d0720 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.841+0000 7f2e2003eb80 10 client.21312 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.841+0000 7f2e2003eb80 10 client.21312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.841+0000 7f2e2003eb80 20 client.21312 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:57.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.841+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620ef6e3d00 0x5620ef6e6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.841+0000 7f2e2003eb80 4 client.21312 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:57.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.841+0000 7f2e2003eb80 1 -- 172.21.15.12:0/2030829294 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620ef699810 con 0x5620ef6e3d00 2024-05-09T00:07:57.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.842+0000 7f2e11c20700 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620ef6e3d00 0x5620ef6e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.842+0000 7f2df57fa700 10 client.21312 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:57.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.842+0000 7f2e11c20700 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620ef6e3d00 0x5620ef6e6120 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de608ebdb0f7f184 server_cookie=670b421a8c165d1a in_seq=0 out_seq=0 2024-05-09T00:07:57.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2df57fa700 1 -- 172.21.15.12:0/2030829294 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620ef699810 con 0x5620ef6e3d00 2024-05-09T00:07:57.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2df57fa700 10 client.21312 handle_command_reply: tid=0 2024-05-09T00:07:57.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 client.21312 shutdown 2024-05-09T00:07:57.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 -- 172.21.15.12:0/2030829294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 msgr2=0x5620ef6d4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6d4550 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2e040ca4c0 tx=0x7f2e04099e50 comp rx=0 tx=0).stop 2024-05-09T00:07:57.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 -- 172.21.15.12:0/2030829294 shutdown_connections 2024-05-09T00:07:57.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6d4550 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620ef6e3d00 0x5620ef6e6120 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/2030829294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6d4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 -- 172.21.15.12:0/2030829294 >> 172.21.15.12:0/2030829294 conn(0x5620ef628210 msgr2=0x5620ef6c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:57.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2df57fa700 0 client.21312 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:57.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 -- 172.21.15.12:0/2030829294 shutdown_connections 2024-05-09T00:07:57.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 1 -- 172.21.15.12:0/2030829294 wait complete. 2024-05-09T00:07:57.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.857+0000 7f2e2003eb80 20 client.21312 trim_cache size 0 max 16384 2024-05-09T00:07:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.878+0000 7f2e2003eb80 1 Processor -- start 2024-05-09T00:07:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.878+0000 7f2e2003eb80 1 -- start start 2024-05-09T00:07:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef573270 con 0x5620ef6d0720 2024-05-09T00:07:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef6d6ac0 con 0x5620ef6cfe00 2024-05-09T00:07:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef6bd110 con 0x5620ef6ca890 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e12421700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e12421700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60632/0 (socket says 172.21.15.12:60632) 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e11c20700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e12421700 1 -- 172.21.15.12:0/46781415 learned_addr learned my addr 172.21.15.12:0/46781415 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e12421700 1 -- 172.21.15.12:0/46781415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 msgr2=0x5620ef6ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e12421700 1 --2- 172.21.15.12:0/46781415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e12421700 1 -- 172.21.15.12:0/46781415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 msgr2=0x5620ef6cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e12421700 1 --2- 172.21.15.12:0/46781415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e12421700 1 -- 172.21.15.12:0/46781415 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620ef5e0560 con 0x5620ef6d0720 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.879+0000 7f2e12421700 1 --2- 172.21.15.12:0/46781415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6c4d20 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f2e040090a0 tx=0x7f2e040c9f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87845bdb67463680 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.880+0000 7f2e00ff9700 1 -- 172.21.15.12:0/46781415 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e040b2260 con 0x5620ef6d0720 2024-05-09T00:07:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.880+0000 7f2e00ff9700 1 -- 172.21.15.12:0/46781415 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e04099450 con 0x5620ef6d0720 2024-05-09T00:07:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.880+0000 7f2e00ff9700 1 -- 172.21.15.12:0/46781415 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e040a1d00 con 0x5620ef6d0720 2024-05-09T00:07:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.880+0000 7f2e2003eb80 1 -- 172.21.15.12:0/46781415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 msgr2=0x5620ef6c4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.880+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/46781415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6c4d20 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f2e040090a0 tx=0x7f2e040c9f70 comp rx=0 tx=0).stop 2024-05-09T00:07:57.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.881+0000 7f2e2003eb80 1 -- 172.21.15.12:0/46781415 shutdown_connections 2024-05-09T00:07:57.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.881+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/46781415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6d0720 0x5620ef6c4d20 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.881+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/46781415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.881+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/46781415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6cfe00 0x5620ef6cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.881+0000 7f2e2003eb80 1 -- 172.21.15.12:0/46781415 >> 172.21.15.12:0/46781415 conn(0x5620ef628210 msgr2=0x5620ef6ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:57.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.881+0000 7f2e2003eb80 1 -- 172.21.15.12:0/46781415 shutdown_connections 2024-05-09T00:07:57.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.881+0000 7f2e2003eb80 1 -- 172.21.15.12:0/46781415 wait complete. 2024-05-09T00:07:57.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.881+0000 7f2e2003eb80 1 -- start start 2024-05-09T00:07:57.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.881+0000 7f2e2003eb80 1 Processor -- start 2024-05-09T00:07:57.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6d9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6cfe00 0x5620ef6d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e2003eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6d0720 0x5620ef7391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef6d9e10 con 0x5620ef6cfe00 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef739700 con 0x5620ef6d0720 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e2003eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ef739840 con 0x5620ef6ca890 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e2003eb80 10 client.0 _ll_register_callbacks cb 0x5620ef5bfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e1141f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6cfe00 0x5620ef6d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e1141f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6cfe00 0x5620ef6d9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60642/0 (socket says 172.21.15.12:60642) 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e12421700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6d0720 0x5620ef7391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e1141f700 1 -- 172.21.15.12:0/3527465052 learned_addr learned my addr 172.21.15.12:0/3527465052 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e11c20700 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6d9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e1141f700 1 -- 172.21.15.12:0/3527465052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 msgr2=0x5620ef6d9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:57.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e1141f700 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6d9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e1141f700 1 -- 172.21.15.12:0/3527465052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6d0720 msgr2=0x5620ef7391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e1141f700 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6d0720 0x5620ef7391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.882+0000 7f2e1141f700 1 -- 172.21.15.12:0/3527465052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620ef447b60 con 0x5620ef6cfe00 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2e1141f700 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6cfe00 0x5620ef6d9570 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc005950 tx=0x7f2dfc010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e353a51383f2bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2df57fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2df57fa700 1 -- 172.21.15.12:0/3527465052 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc004520 con 0x5620ef6cfe00 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2e2003eb80 1 -- 172.21.15.12:0/3527465052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e0401b320 con 0x5620ef6cfe00 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2df57fa700 1 -- 172.21.15.12:0/3527465052 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2dfc003680 con 0x5620ef6cfe00 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2df57fa700 1 -- 172.21.15.12:0/3527465052 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dfc003860 con 0x5620ef6cfe00 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2e2003eb80 10 client.21318 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2e2003eb80 1 -- 172.21.15.12:0/3527465052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5620ef49fcd0 con 0x5620ef6cfe00 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2df57fa700 1 -- 172.21.15.12:0/3527465052 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dfc029080 con 0x5620ef6cfe00 2024-05-09T00:07:57.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2df57fa700 1 -- 172.21.15.12:0/3527465052 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2dfc01da70 con 0x5620ef6cfe00 2024-05-09T00:07:57.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2e2003eb80 10 client.21318 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:57.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2e2003eb80 10 client.21318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:57.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2e2003eb80 20 client.21318 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:57.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620ef73af10 0x5620ef73d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2e2003eb80 4 client.21318 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:07:57.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.883+0000 7f2e2003eb80 1 -- 172.21.15.12:0/3527465052 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5620ef612ec0 con 0x5620ef73af10 2024-05-09T00:07:57.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.884+0000 7f2e11c20700 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620ef73af10 0x5620ef73d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.884+0000 7f2e11c20700 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620ef73af10 0x5620ef73d330 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a3875d778b4ba9 server_cookie=d98dd562f734c743 in_seq=0 out_seq=0 2024-05-09T00:07:57.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.884+0000 7f2df57fa700 10 client.21318 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:57.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2df57fa700 1 -- 172.21.15.12:0/3527465052 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5620ef612ec0 con 0x5620ef73af10 2024-05-09T00:07:57.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2df57fa700 10 client.21318 handle_command_reply: tid=0 2024-05-09T00:07:57.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 client.21318 shutdown 2024-05-09T00:07:57.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 -- 172.21.15.12:0/3527465052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6cfe00 msgr2=0x5620ef6d9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6cfe00 0x5620ef6d9570 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc005950 tx=0x7f2dfc010a00 comp rx=0 tx=0).stop 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 -- 172.21.15.12:0/3527465052 shutdown_connections 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5620ef73af10 0x5620ef73d330 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5620ef6d0720 0x5620ef7391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ef6ca890 0x5620ef6d9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 --2- 172.21.15.12:0/3527465052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620ef6cfe00 0x5620ef6d9570 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2df57fa700 0 client.21318 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 -- 172.21.15.12:0/3527465052 >> 172.21.15.12:0/3527465052 conn(0x5620ef628210 msgr2=0x5620ef6cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 -- 172.21.15.12:0/3527465052 shutdown_connections 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 1 -- 172.21.15.12:0/3527465052 wait complete. 2024-05-09T00:07:57.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.889+0000 7f2e2003eb80 20 client.21318 trim_cache size 0 max 16384 2024-05-09T00:07:57.890 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:07:57.890 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:07:57.890 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:07:57.890 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:07:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 -- 172.21.15.12:0/2910754496 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2df80a1310 msgr2=0x7f2df80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2df80a1310 0x7f2df80a37d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800b2e0 tx=0x7f2e08005fd0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 -- 172.21.15.12:0/2910754496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0c198d50 msgr2=0x7f2e0c1a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0c198d50 0x7f2e0c1a3350 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400ef30 tx=0x7f2e0400c3a0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 -- 172.21.15.12:0/2910754496 shutdown_connections 2024-05-09T00:07:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2df80a1310 0x7f2df80a37d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0c198d50 0x7f2e0c1a3350 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e0c100060 0x7f2e0c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 --2- 172.21.15.12:0/2910754496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e0c105160 0x7f2e0c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 -- 172.21.15.12:0/2910754496 >> 172.21.15.12:0/2910754496 conn(0x7f2e0c076f40 msgr2=0x7f2e0c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:57.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 -- 172.21.15.12:0/2910754496 shutdown_connections 2024-05-09T00:07:57.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:57.890+0000 7f2e13e84700 1 -- 172.21.15.12:0/2910754496 wait complete. 2024-05-09T00:07:57.900 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:07:57.901 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:07:58.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:57 smithi160 ceph-mon[46152]: pgmap v692: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 707 op/s 2024-05-09T00:07:58.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.052+0000 7f2a49a27700 1 -- 172.21.15.12:0/1797717904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a440ff5a0 msgr2=0x7f2a440ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.052+0000 7f2a49a27700 1 --2- 172.21.15.12:0/1797717904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a440ff5a0 0x7f2a440ff980 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009990 tx=0x7f2a34009ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.052+0000 7f2a49a27700 1 -- 172.21.15.12:0/1797717904 shutdown_connections 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.052+0000 7f2a49a27700 1 --2- 172.21.15.12:0/1797717904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a44106160 0x7f2a4410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.052+0000 7f2a49a27700 1 --2- 172.21.15.12:0/1797717904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a440ff5a0 0x7f2a440ff980 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.052+0000 7f2a49a27700 1 --2- 172.21.15.12:0/1797717904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440fff50 0x7f2a44105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.052+0000 7f2a49a27700 1 -- 172.21.15.12:0/1797717904 >> 172.21.15.12:0/1797717904 conn(0x7f2a44076270 msgr2=0x7f2a440fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.052+0000 7f2a49a27700 1 -- 172.21.15.12:0/1797717904 shutdown_connections 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.052+0000 7f2a49a27700 1 -- 172.21.15.12:0/1797717904 wait complete. 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.053+0000 7f2a49a27700 1 Processor -- start 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.053+0000 7f2a49a27700 1 -- start start 2024-05-09T00:07:58.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a49a27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a440ff5a0 0x7f2a44197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a49a27700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a44106160 0x7f2a44197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a49a27700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a4419bbd0 0x7f2a4419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a49a27700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4419c650 con 0x7f2a440ff5a0 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a49a27700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4419c7c0 con 0x7f2a44106160 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a49a27700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4419c980 con 0x7f2a4419bbd0 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a42ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a440ff5a0 0x7f2a44197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a3a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a44106160 0x7f2a44197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a42ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a440ff5a0 0x7f2a44197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60664/0 (socket says 172.21.15.12:60664) 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a42ffd700 1 -- 172.21.15.12:0/1098752696 learned_addr learned my addr 172.21.15.12:0/1098752696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.054+0000 7f2a437fe700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a4419bbd0 0x7f2a4419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a3a7fc700 1 -- 172.21.15.12:0/1098752696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a4419bbd0 msgr2=0x7f2a4419c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a3a7fc700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a4419bbd0 0x7f2a4419c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a3a7fc700 1 -- 172.21.15.12:0/1098752696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a440ff5a0 msgr2=0x7f2a44197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a3a7fc700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a440ff5a0 0x7f2a44197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a3a7fc700 1 -- 172.21.15.12:0/1098752696 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340095f0 con 0x7f2a44106160 2024-05-09T00:07:58.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a3a7fc700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a44106160 0x7f2a44197880 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2a44067900 tx=0x7f2a2c00eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a48a25700 1 -- 172.21.15.12:0/1098752696 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c010900 con 0x7f2a44106160 2024-05-09T00:07:58.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a48a25700 1 -- 172.21.15.12:0/1098752696 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a2c00c430 con 0x7f2a44106160 2024-05-09T00:07:58.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a49a27700 1 -- 172.21.15.12:0/1098752696 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a441a3ee0 con 0x7f2a44106160 2024-05-09T00:07:58.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a48a25700 1 -- 172.21.15.12:0/1098752696 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c018680 con 0x7f2a44106160 2024-05-09T00:07:58.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.055+0000 7f2a49a27700 1 -- 172.21.15.12:0/1098752696 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a441a4360 con 0x7f2a44106160 2024-05-09T00:07:58.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.056+0000 7f2a49a27700 1 -- 172.21.15.12:0/1098752696 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a44109410 con 0x7f2a44106160 2024-05-09T00:07:58.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.059+0000 7f2a48a25700 1 -- 172.21.15.12:0/1098752696 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2a2c00c5a0 con 0x7f2a44106160 2024-05-09T00:07:58.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.059+0000 7f2a48a25700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a300a1290 0x7f2a300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.059+0000 7f2a42ffd700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a300a1290 0x7f2a300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.059+0000 7f2a48a25700 1 -- 172.21.15.12:0/1098752696 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2a2c014070 con 0x7f2a44106160 2024-05-09T00:07:58.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.060+0000 7f2a42ffd700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a300a1290 0x7f2a300a3750 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2a34006010 tx=0x7f2a34005a90 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.063+0000 7f2a48a25700 1 -- 172.21.15.12:0/1098752696 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2a2c0963b0 con 0x7f2a44106160 2024-05-09T00:07:58.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:57 smithi012 ceph-mon[42503]: pgmap v692: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 707 op/s 2024-05-09T00:07:58.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:57 smithi204 ceph-mon[47373]: pgmap v692: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 707 op/s 2024-05-09T00:07:58.279 INFO:tasks.workunit.client.0.smithi012.stdout: 10 292864 31.45 MB/sec execute 353 sec latency 88.694 ms 2024-05-09T00:07:58.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.369+0000 7f2a49a27700 1 -- 172.21.15.12:0/1098752696 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a44102650 con 0x7f2a44106160 2024-05-09T00:07:58.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.370+0000 7f2a48a25700 1 -- 172.21.15.12:0/1098752696 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+10875 (secure 0 0 0) 0x7f2a2c026090 con 0x7f2a44106160 2024-05-09T00:07:58.366 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:07:58.367 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:06:02.725626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_30395":{"gid":30395,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6833/3969500142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3969500142},{"type":"v1","addr":"172.21.15.160:6833","nonce":3969500142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:07:58.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.373+0000 7f2a49a27700 1 -- 172.21.15.12:0/1098752696 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a300a1290 msgr2=0x7f2a300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.373+0000 7f2a49a27700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a300a1290 0x7f2a300a3750 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2a34006010 tx=0x7f2a34005a90 comp rx=0 tx=0).stop 2024-05-09T00:07:58.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.373+0000 7f2a49a27700 1 -- 172.21.15.12:0/1098752696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a44106160 msgr2=0x7f2a44197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.373+0000 7f2a49a27700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a44106160 0x7f2a44197880 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2a44067900 tx=0x7f2a2c00eca0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.376+0000 7f2a49a27700 1 -- 172.21.15.12:0/1098752696 shutdown_connections 2024-05-09T00:07:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.376+0000 7f2a49a27700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f2a300a1290 0x7f2a300a3750 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.376+0000 7f2a49a27700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a4419bbd0 0x7f2a4419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.376+0000 7f2a49a27700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a440ff5a0 0x7f2a44197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.376+0000 7f2a49a27700 1 --2- 172.21.15.12:0/1098752696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a44106160 0x7f2a44197880 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.376+0000 7f2a49a27700 1 -- 172.21.15.12:0/1098752696 >> 172.21.15.12:0/1098752696 conn(0x7f2a44076270 msgr2=0x7f2a441045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.376+0000 7f2a49a27700 1 -- 172.21.15.12:0/1098752696 shutdown_connections 2024-05-09T00:07:58.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.376+0000 7f2a49a27700 1 -- 172.21.15.12:0/1098752696 wait complete. 2024-05-09T00:07:58.373 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 76 2024-05-09T00:07:58.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:07:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.604+0000 7f043404e700 1 -- 172.21.15.12:0/929329809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f042c10d940 msgr2=0x7f042c10fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.604+0000 7f043404e700 1 --2- 172.21.15.12:0/929329809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f042c10d940 0x7f042c10fd30 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f042400b0a0 tx=0x7f042400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.604+0000 7f043404e700 1 -- 172.21.15.12:0/929329809 shutdown_connections 2024-05-09T00:07:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.604+0000 7f043404e700 1 --2- 172.21.15.12:0/929329809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f042c10d940 0x7f042c10fd30 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.604+0000 7f043404e700 1 --2- 172.21.15.12:0/929329809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f042c069250 0x7f042c069630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.604+0000 7f043404e700 1 --2- 172.21.15.12:0/929329809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f042c069b70 0x7f042c10d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.604+0000 7f043404e700 1 -- 172.21.15.12:0/929329809 >> 172.21.15.12:0/929329809 conn(0x7f042c076f30 msgr2=0x7f042c077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.604+0000 7f043404e700 1 -- 172.21.15.12:0/929329809 shutdown_connections 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.604+0000 7f043404e700 1 -- 172.21.15.12:0/929329809 wait complete. 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f043404e700 1 Processor -- start 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f043404e700 1 -- start start 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f043404e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f042c069250 0x7f042c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f043404e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f042c069b70 0x7f042c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f043404e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f042c10d940 0x7f042c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f043404e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c19d600 con 0x7f042c069250 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f043404e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c19d770 con 0x7f042c10d940 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f043404e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c19d930 con 0x7f042c069b70 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f0431dea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f042c069250 0x7f042c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f0431dea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f042c069250 0x7f042c1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37490/0 (socket says 172.21.15.12:37490) 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f0431dea700 1 -- 172.21.15.12:0/2622639653 learned_addr learned my addr 172.21.15.12:0/2622639653 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f04325eb700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f042c10d940 0x7f042c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f04315e9700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f042c069b70 0x7f042c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f0431dea700 1 -- 172.21.15.12:0/2622639653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f042c069b70 msgr2=0x7f042c199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f0431dea700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f042c069b70 0x7f042c199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f0431dea700 1 -- 172.21.15.12:0/2622639653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f042c10d940 msgr2=0x7f042c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f0431dea700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f042c10d940 0x7f042c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.605+0000 7f0431dea700 1 -- 172.21.15.12:0/2622639653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0424009d00 con 0x7f042c069250 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.606+0000 7f0431dea700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f042c069250 0x7f042c1997d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f042800e9a0 tx=0x7f042800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.606+0000 7f0422ffd700 1 -- 172.21.15.12:0/2622639653 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428009790 con 0x7f042c069250 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.606+0000 7f043404e700 1 -- 172.21.15.12:0/2622639653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c19dc10 con 0x7f042c069250 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.606+0000 7f0422ffd700 1 -- 172.21.15.12:0/2622639653 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0428010430 con 0x7f042c069250 2024-05-09T00:07:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.606+0000 7f0422ffd700 1 -- 172.21.15.12:0/2622639653 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428018680 con 0x7f042c069250 2024-05-09T00:07:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.606+0000 7f043404e700 1 -- 172.21.15.12:0/2622639653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c1a6570 con 0x7f042c069250 2024-05-09T00:07:58.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.608+0000 7f0422ffd700 1 -- 172.21.15.12:0/2622639653 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f04280105a0 con 0x7f042c069250 2024-05-09T00:07:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.609+0000 7f0422ffd700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04180a1290 0x7f04180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.610+0000 7f04315e9700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04180a1290 0x7f04180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.610+0000 7f0422ffd700 1 -- 172.21.15.12:0/2622639653 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0428014070 con 0x7f042c069250 2024-05-09T00:07:58.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.610+0000 7f04315e9700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04180a1290 0x7f04180a3750 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f042c19ad20 tx=0x7f041c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.610+0000 7f0440208b80 1 Processor -- start 2024-05-09T00:07:58.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f0440208b80 1 -- start start 2024-05-09T00:07:58.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f0440208b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08fe00 0x558d2a0901e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f0440208b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a08a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f0440208b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08a890 0x558d2a08acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f0440208b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29fe1150 con 0x558d2a08fe00 2024-05-09T00:07:58.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f0440208b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29fdeb10 con 0x558d2a090720 2024-05-09T00:07:58.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f0440208b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29fda400 con 0x558d2a08a890 2024-05-09T00:07:58.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f04325eb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08a890 0x558d2a08acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f04325eb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08a890 0x558d2a08acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57582/0 (socket says 172.21.15.12:57582) 2024-05-09T00:07:58.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f04325eb700 1 -- 172.21.15.12:0/30692358 learned_addr learned my addr 172.21.15.12:0/30692358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f04315e9700 1 --2- 172.21.15.12:0/30692358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a08a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f0431dea700 1 --2- 172.21.15.12:0/30692358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08fe00 0x558d2a0901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f04315e9700 1 -- 172.21.15.12:0/30692358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08a890 msgr2=0x558d2a08acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f04315e9700 1 --2- 172.21.15.12:0/30692358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08a890 0x558d2a08acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f04315e9700 1 -- 172.21.15.12:0/30692358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08fe00 msgr2=0x558d2a0901e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.611+0000 7f04315e9700 1 --2- 172.21.15.12:0/30692358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08fe00 0x558d2a0901e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.612+0000 7f04315e9700 1 -- 172.21.15.12:0/30692358 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d29e5b3b0 con 0x558d2a090720 2024-05-09T00:07:58.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.612+0000 7f04315e9700 1 --2- 172.21.15.12:0/30692358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a08a350 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f041c011c30 tx=0x7f041c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f79805d8dcf38bd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.612+0000 7f0417fff700 1 -- 172.21.15.12:0/30692358 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c004050 con 0x558d2a090720 2024-05-09T00:07:58.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.612+0000 7f0417fff700 1 -- 172.21.15.12:0/30692358 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f041c00e840 con 0x558d2a090720 2024-05-09T00:07:58.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.612+0000 7f0417fff700 1 -- 172.21.15.12:0/30692358 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c003a70 con 0x558d2a090720 2024-05-09T00:07:58.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.613+0000 7f0440208b80 1 -- 172.21.15.12:0/30692358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 msgr2=0x558d2a08a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.613+0000 7f0440208b80 1 --2- 172.21.15.12:0/30692358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a08a350 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f041c011c30 tx=0x7f041c014000 comp rx=0 tx=0).stop 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.613+0000 7f0440208b80 1 -- 172.21.15.12:0/30692358 shutdown_connections 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.613+0000 7f0440208b80 1 --2- 172.21.15.12:0/30692358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08a890 0x558d2a08acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.613+0000 7f0440208b80 1 --2- 172.21.15.12:0/30692358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08fe00 0x558d2a0901e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.613+0000 7f0440208b80 1 --2- 172.21.15.12:0/30692358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a08a350 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.613+0000 7f0440208b80 1 -- 172.21.15.12:0/30692358 >> 172.21.15.12:0/30692358 conn(0x558d29fe8210 msgr2=0x558d2a084df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.613+0000 7f0440208b80 1 -- 172.21.15.12:0/30692358 shutdown_connections 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.613+0000 7f0440208b80 1 -- 172.21.15.12:0/30692358 wait complete. 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.613+0000 7f0440208b80 1 -- start start 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f0440208b80 1 Processor -- start 2024-05-09T00:07:58.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f0440208b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a093b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f0440208b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 0x558d2a094090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f0440208b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a0945d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f0440208b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29f76560 con 0x558d2a08a890 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f0440208b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29f76200 con 0x558d2a090720 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f0440208b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d29e5d6a0 con 0x558d2a08fe00 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f0440208b80 10 client.0 _ll_register_callbacks cb 0x558d29f7fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f04325eb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a0945d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f04325eb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a0945d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33376/0 (socket says 172.21.15.12:33376) 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f04325eb700 1 -- 172.21.15.12:0/1168885364 learned_addr learned my addr 172.21.15.12:0/1168885364 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.614+0000 7f04315e9700 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 0x558d2a094090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f0431dea700 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a093b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04325eb700 1 -- 172.21.15.12:0/1168885364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 msgr2=0x558d2a094090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04325eb700 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 0x558d2a094090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04325eb700 1 -- 172.21.15.12:0/1168885364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 msgr2=0x558d2a093b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04325eb700 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a093b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04325eb700 1 -- 172.21.15.12:0/1168885364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c017070 con 0x558d2a090720 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f0431dea700 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a093b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04315e9700 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 0x558d2a094090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04325eb700 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a0945d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0424009ca0 tx=0x7f042400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1f15423edd3821f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04157fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04157fa700 1 -- 172.21.15.12:0/1168885364 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042400e040 con 0x558d2a090720 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04157fa700 1 -- 172.21.15.12:0/1168885364 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0424003c00 con 0x558d2a090720 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f0440208b80 1 -- 172.21.15.12:0/1168885364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d29f33270 con 0x558d2a090720 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f04157fa700 1 -- 172.21.15.12:0/1168885364 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0424003e00 con 0x558d2a090720 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.615+0000 7f0440208b80 10 client.30809 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.616+0000 7f0440208b80 1 -- 172.21.15.12:0/1168885364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x558d29e07b60 con 0x558d2a090720 2024-05-09T00:07:58.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.616+0000 7f04157fa700 1 -- 172.21.15.12:0/1168885364 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d29f33270 con 0x558d2a090720 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.616+0000 7f04157fa700 1 -- 172.21.15.12:0/1168885364 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f042400b0d0 con 0x558d2a090720 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.616+0000 7f0440208b80 10 client.30809 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.616+0000 7f0440208b80 10 client.30809 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.616+0000 7f0440208b80 20 client.30809 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.616+0000 7f0440208b80 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x558d2a0aca50 0x558d2a0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.616+0000 7f0440208b80 4 client.30809 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.616+0000 7f0440208b80 1 -- 172.21.15.12:0/1168885364 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d2a059810 con 0x558d2a0aca50 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.616+0000 7f0431dea700 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x558d2a0aca50 0x558d2a0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.617+0000 7f04157fa700 10 client.30809 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.617+0000 7f0431dea700 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x558d2a0aca50 0x558d2a0aee70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e8bec148e416e78 server_cookie=577760f4df20c623 in_seq=0 out_seq=0 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f04157fa700 1 -- 172.21.15.12:0/1168885364 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d2a059810 con 0x558d2a0aca50 2024-05-09T00:07:58.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f04157fa700 10 client.30809 handle_command_reply: tid=0 2024-05-09T00:07:58.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f0440208b80 1 client.30809 shutdown 2024-05-09T00:07:58.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f0440208b80 1 -- 172.21.15.12:0/1168885364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 msgr2=0x558d2a0945d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f0440208b80 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a0945d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0424009ca0 tx=0x7f042400bab0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f0440208b80 1 -- 172.21.15.12:0/1168885364 shutdown_connections 2024-05-09T00:07:58.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f0440208b80 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a0945d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f0440208b80 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a093b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f0440208b80 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x558d2a0aca50 0x558d2a0aee70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f0440208b80 1 --2- 172.21.15.12:0/1168885364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 0x558d2a094090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f0440208b80 1 -- 172.21.15.12:0/1168885364 >> 172.21.15.12:0/1168885364 conn(0x558d29fe8210 msgr2=0x558d2a0845a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f04157fa700 0 client.30809 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:07:58.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.619+0000 7f0440208b80 1 -- 172.21.15.12:0/1168885364 shutdown_connections 2024-05-09T00:07:58.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.620+0000 7f0440208b80 1 -- 172.21.15.12:0/1168885364 wait complete. 2024-05-09T00:07:58.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.620+0000 7f0440208b80 20 client.30809 trim_cache size 0 max 16384 2024-05-09T00:07:58.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.635+0000 7f0440208b80 1 Processor -- start 2024-05-09T00:07:58.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.635+0000 7f0440208b80 1 -- start start 2024-05-09T00:07:58.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.635+0000 7f0440208b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a08a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.635+0000 7f0440208b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a08fe00 0x558d2a08f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.635+0000 7f0440208b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a090720 0x558d2a084d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.635+0000 7f0440208b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2a0858e0 con 0x558d2a08a890 2024-05-09T00:07:58.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.635+0000 7f0440208b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2a081dc0 con 0x558d2a08fe00 2024-05-09T00:07:58.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.635+0000 7f0440208b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2a086e50 con 0x558d2a090720 2024-05-09T00:07:58.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0431dea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a08a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0431dea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a08a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37532/0 (socket says 172.21.15.12:37532) 2024-05-09T00:07:58.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0431dea700 1 -- 172.21.15.12:0/636196514 learned_addr learned my addr 172.21.15.12:0/636196514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f04315e9700 1 --2- 172.21.15.12:0/636196514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a08fe00 0x558d2a08f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f04325eb700 1 --2- 172.21.15.12:0/636196514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a090720 0x558d2a084d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0431dea700 1 -- 172.21.15.12:0/636196514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a090720 msgr2=0x558d2a084d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0431dea700 1 --2- 172.21.15.12:0/636196514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a090720 0x558d2a084d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0431dea700 1 -- 172.21.15.12:0/636196514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a08fe00 msgr2=0x558d2a08f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0431dea700 1 --2- 172.21.15.12:0/636196514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a08fe00 0x558d2a08f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0431dea700 1 -- 172.21.15.12:0/636196514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d29fa0560 con 0x558d2a08a890 2024-05-09T00:07:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0431dea700 1 --2- 172.21.15.12:0/636196514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a08a350 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f04280964c0 tx=0x7f042800c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f5cb6918242d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0420ff9700 1 -- 172.21.15.12:0/636196514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04280a2440 con 0x558d2a08a890 2024-05-09T00:07:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0420ff9700 1 -- 172.21.15.12:0/636196514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04280ade90 con 0x558d2a08a890 2024-05-09T00:07:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.636+0000 7f0420ff9700 1 -- 172.21.15.12:0/636196514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04280994b0 con 0x558d2a08a890 2024-05-09T00:07:58.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.637+0000 7f0440208b80 1 -- 172.21.15.12:0/636196514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 msgr2=0x558d2a08a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.637+0000 7f0440208b80 1 --2- 172.21.15.12:0/636196514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a08a350 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f04280964c0 tx=0x7f042800c480 comp rx=0 tx=0).stop 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.637+0000 7f0440208b80 1 -- 172.21.15.12:0/636196514 shutdown_connections 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.637+0000 7f0440208b80 1 --2- 172.21.15.12:0/636196514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a090720 0x558d2a084d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.637+0000 7f0440208b80 1 --2- 172.21.15.12:0/636196514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a08a350 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.637+0000 7f0440208b80 1 --2- 172.21.15.12:0/636196514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a08fe00 0x558d2a08f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.637+0000 7f0440208b80 1 -- 172.21.15.12:0/636196514 >> 172.21.15.12:0/636196514 conn(0x558d29fe8210 msgr2=0x558d2a092420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.637+0000 7f0440208b80 1 -- 172.21.15.12:0/636196514 shutdown_connections 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.637+0000 7f0440208b80 1 -- 172.21.15.12:0/636196514 wait complete. 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0440208b80 1 -- start start 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0440208b80 1 Processor -- start 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0440208b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a099860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0440208b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 0x558d2a099da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0440208b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a09ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0440208b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2a09b200 con 0x558d2a08a890 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0440208b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2a09b370 con 0x558d2a090720 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0440208b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2a09b530 con 0x558d2a08fe00 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0440208b80 10 client.0 _ll_register_callbacks cb 0x558d29f7fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0431dea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f04315e9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 0x558d2a099da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0431dea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a099860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37544/0 (socket says 172.21.15.12:37544) 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.638+0000 7f0431dea700 1 -- 172.21.15.12:0/252238673 learned_addr learned my addr 172.21.15.12:0/252238673 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f04325eb700 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a09ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f0431dea700 1 -- 172.21.15.12:0/252238673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 msgr2=0x558d2a099da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f0431dea700 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 0x558d2a099da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f0431dea700 1 -- 172.21.15.12:0/252238673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 msgr2=0x558d2a09ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f0431dea700 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a09ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f0431dea700 1 -- 172.21.15.12:0/252238673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042805fe70 con 0x558d2a08a890 2024-05-09T00:07:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f04315e9700 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 0x558d2a099da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f04325eb700 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a09ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f0431dea700 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a099860 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f042800c510 tx=0x7f0428099ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2ac4dafd1bc57b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f04157fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:07:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f04157fa700 1 -- 172.21.15.12:0/252238673 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428094bd0 con 0x558d2a08a890 2024-05-09T00:07:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f04157fa700 1 -- 172.21.15.12:0/252238673 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04280b8300 con 0x558d2a08a890 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f0440208b80 1 -- 172.21.15.12:0/252238673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f041c00e840 con 0x558d2a08a890 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f04157fa700 1 -- 172.21.15.12:0/252238673 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0428094d30 con 0x558d2a08a890 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f0440208b80 10 client.21339 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.639+0000 7f0440208b80 1 -- 172.21.15.12:0/252238673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x558d29e5fcd0 con 0x558d2a08a890 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f04157fa700 1 -- 172.21.15.12:0/252238673 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f041c00e840 con 0x558d2a08a890 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f04157fa700 1 -- 172.21.15.12:0/252238673 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f042809a2b0 con 0x558d2a08a890 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f0440208b80 10 client.21339 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f0440208b80 10 client.21339 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f0440208b80 20 client.21339 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f0440208b80 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x558d2a103f40 0x558d2a106330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f0440208b80 4 client.21339 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:07:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f0440208b80 1 -- 172.21.15.12:0/252238673 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558d29fd2ec0 con 0x558d2a103f40 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f04315e9700 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x558d2a103f40 0x558d2a106330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f04157fa700 10 client.21339 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.640+0000 7f04315e9700 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x558d2a103f40 0x558d2a106330 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dda7d7ad75e59481 server_cookie=cbff30db8dc7d3b0 in_seq=0 out_seq=0 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f04157fa700 1 -- 172.21.15.12:0/252238673 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558d29fd2ec0 con 0x558d2a103f40 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f04157fa700 10 client.21339 handle_command_reply: tid=0 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 client.21339 shutdown 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 -- 172.21.15.12:0/252238673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 msgr2=0x558d2a099860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a099860 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f042800c510 tx=0x7f0428099ec0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 -- 172.21.15.12:0/252238673 shutdown_connections 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558d2a090720 0x558d2a09ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558d2a08a890 0x558d2a099860 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x558d2a103f40 0x558d2a106330 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 --2- 172.21.15.12:0/252238673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d2a08fe00 0x558d2a099da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f04157fa700 0 client.21339 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:07:58.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 -- 172.21.15.12:0/252238673 >> 172.21.15.12:0/252238673 conn(0x558d29fe8210 msgr2=0x558d2a090bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 -- 172.21.15.12:0/252238673 shutdown_connections 2024-05-09T00:07:58.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.642+0000 7f0440208b80 1 -- 172.21.15.12:0/252238673 wait complete. 2024-05-09T00:07:58.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f0440208b80 20 client.21339 trim_cache size 0 max 16384 2024-05-09T00:07:58.646 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:07:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 -- 172.21.15.12:0/2622639653 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04180a1290 msgr2=0x7f04180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04180a1290 0x7f04180a3750 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f042c19ad20 tx=0x7f041c009260 comp rx=0 tx=0).stop 2024-05-09T00:07:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 -- 172.21.15.12:0/2622639653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f042c069250 msgr2=0x7f042c1997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f042c069250 0x7f042c1997d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f042800e9a0 tx=0x7f042800ecb0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 -- 172.21.15.12:0/2622639653 shutdown_connections 2024-05-09T00:07:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04180a1290 0x7f04180a3750 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f042c10d940 0x7f042c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f042c069250 0x7f042c1997d0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 --2- 172.21.15.12:0/2622639653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f042c069b70 0x7f042c199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 -- 172.21.15.12:0/2622639653 >> 172.21.15.12:0/2622639653 conn(0x7f042c076f30 msgr2=0x7f042c10ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 -- 172.21.15.12:0/2622639653 shutdown_connections 2024-05-09T00:07:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.643+0000 7f043404e700 1 -- 172.21.15.12:0/2622639653 wait complete. 2024-05-09T00:07:58.656 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:07:58.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.856+0000 7fae473bd700 1 -- 172.21.15.12:0/1340128410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae40105f90 msgr2=0x7fae40106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.856+0000 7fae473bd700 1 --2- 172.21.15.12:0/1340128410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae40105f90 0x7fae40106370 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009920 tx=0x7fae3c009c30 comp rx=0 tx=0).stop 2024-05-09T00:07:58.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.856+0000 7fae473bd700 1 -- 172.21.15.12:0/1340128410 shutdown_connections 2024-05-09T00:07:58.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.856+0000 7fae473bd700 1 --2- 172.21.15.12:0/1340128410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae40100ab0 0x7fae4010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.856+0000 7fae473bd700 1 --2- 172.21.15.12:0/1340128410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae40105f90 0x7fae40106370 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.856+0000 7fae473bd700 1 --2- 172.21.15.12:0/1340128410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae401000f0 0x7fae40100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.856+0000 7fae473bd700 1 -- 172.21.15.12:0/1340128410 >> 172.21.15.12:0/1340128410 conn(0x7fae40076ba0 msgr2=0x7fae40076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.856+0000 7fae473bd700 1 -- 172.21.15.12:0/1340128410 shutdown_connections 2024-05-09T00:07:58.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.856+0000 7fae473bd700 1 -- 172.21.15.12:0/1340128410 wait complete. 2024-05-09T00:07:58.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.857+0000 7fae473bd700 1 Processor -- start 2024-05-09T00:07:58.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.857+0000 7fae473bd700 1 -- start start 2024-05-09T00:07:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.858+0000 7fae473bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae401000f0 0x7fae40072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.858+0000 7fae473bd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae40100ab0 0x7fae4006aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.858+0000 7fae473bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae40105f90 0x7fae4006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.858+0000 7fae473bd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4006b770 con 0x7fae401000f0 2024-05-09T00:07:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.858+0000 7fae473bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4006b8e0 con 0x7fae40105f90 2024-05-09T00:07:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.858+0000 7fae473bd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae401ac270 con 0x7fae40100ab0 2024-05-09T00:07:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.858+0000 7fae4595a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae40105f90 0x7fae4006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.858+0000 7fae4595a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae40105f90 0x7fae4006afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33396/0 (socket says 172.21.15.12:33396) 2024-05-09T00:07:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.858+0000 7fae45159700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae401000f0 0x7fae40072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.858+0000 7fae4595a700 1 -- 172.21.15.12:0/1094265210 learned_addr learned my addr 172.21.15.12:0/1094265210 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.859+0000 7fae4595a700 1 -- 172.21.15.12:0/1094265210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae40100ab0 msgr2=0x7fae4006aa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:07:58.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.859+0000 7fae4595a700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae40100ab0 0x7fae4006aa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.859+0000 7fae4595a700 1 -- 172.21.15.12:0/1094265210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae401000f0 msgr2=0x7fae40072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.859+0000 7fae4595a700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae401000f0 0x7fae40072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.859+0000 7fae4595a700 1 -- 172.21.15.12:0/1094265210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae300095f0 con 0x7fae40105f90 2024-05-09T00:07:58.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.859+0000 7fae45159700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae401000f0 0x7fae40072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:58.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.859+0000 7fae4595a700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae40105f90 0x7fae4006afd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fae3802d9e0 tx=0x7fae3802dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.861+0000 7fae367fc700 1 -- 172.21.15.12:0/1094265210 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3802e950 con 0x7fae40105f90 2024-05-09T00:07:58.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.861+0000 7fae473bd700 1 -- 172.21.15.12:0/1094265210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c009520 con 0x7fae40105f90 2024-05-09T00:07:58.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.861+0000 7fae367fc700 1 -- 172.21.15.12:0/1094265210 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae38036440 con 0x7fae40105f90 2024-05-09T00:07:58.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.861+0000 7fae367fc700 1 -- 172.21.15.12:0/1094265210 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380295e0 con 0x7fae40105f90 2024-05-09T00:07:58.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.861+0000 7fae473bd700 1 -- 172.21.15.12:0/1094265210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae40197670 con 0x7fae40105f90 2024-05-09T00:07:58.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.866+0000 7fae53577b80 1 Processor -- start 2024-05-09T00:07:58.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.867+0000 7fae53577b80 1 -- start start 2024-05-09T00:07:58.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.867+0000 7fae53577b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b6e00 0x5569660b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.867+0000 7fae53577b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.867+0000 7fae53577b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 0x5569660b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.867+0000 7fae53577b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556966008150 con 0x5569660b6e00 2024-05-09T00:07:58.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.867+0000 7fae53577b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556966005b10 con 0x5569660b7720 2024-05-09T00:07:58.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.867+0000 7fae53577b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556966001400 con 0x5569660b1890 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.868+0000 7fae45159700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.868+0000 7fae45159700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33406/0 (socket says 172.21.15.12:33406) 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.868+0000 7fae45159700 1 -- 172.21.15.12:0/1443059818 learned_addr learned my addr 172.21.15.12:0/1443059818 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.868+0000 7fae45159700 1 -- 172.21.15.12:0/1443059818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 msgr2=0x5569660b1cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.868+0000 7fae45159700 1 --2- 172.21.15.12:0/1443059818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 0x5569660b1cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.868+0000 7fae45159700 1 -- 172.21.15.12:0/1443059818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b6e00 msgr2=0x5569660b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.868+0000 7fae45159700 1 --2- 172.21.15.12:0/1443059818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b6e00 0x5569660b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.868+0000 7fae45159700 1 -- 172.21.15.12:0/1443059818 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556965e823b0 con 0x5569660b7720 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.869+0000 7fae367fc700 1 -- 172.21.15.12:0/1094265210 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fae380365b0 con 0x7fae40105f90 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.870+0000 7fae45159700 1 --2- 172.21.15.12:0/1443059818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660b1350 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fae40067bb0 tx=0x7fae3c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbbde70cf2233e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.870+0000 7fae367fc700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae2c0a1390 0x7fae2c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.871+0000 7fae367fc700 1 -- 172.21.15.12:0/1094265210 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fae38006e40 con 0x7fae40105f90 2024-05-09T00:07:58.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.873+0000 7fae44958700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae2c0a1390 0x7fae2c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.874+0000 7fae44958700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae2c0a1390 0x7fae2c0a3850 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fae300095c0 tx=0x7fae30009510 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.876+0000 7fae2b7fe700 1 -- 172.21.15.12:0/1443059818 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c01c050 con 0x5569660b7720 2024-05-09T00:07:58.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.876+0000 7fae2b7fe700 1 -- 172.21.15.12:0/1443059818 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae3c01d040 con 0x5569660b7720 2024-05-09T00:07:58.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.876+0000 7fae2b7fe700 1 -- 172.21.15.12:0/1443059818 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c005420 con 0x5569660b7720 2024-05-09T00:07:58.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 -- 172.21.15.12:0/1443059818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 msgr2=0x5569660b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 --2- 172.21.15.12:0/1443059818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660b1350 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fae40067bb0 tx=0x7fae3c005dc0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 -- 172.21.15.12:0/1443059818 shutdown_connections 2024-05-09T00:07:58.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 --2- 172.21.15.12:0/1443059818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 0x5569660b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 --2- 172.21.15.12:0/1443059818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b6e00 0x5569660b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 --2- 172.21.15.12:0/1443059818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660b1350 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 -- 172.21.15.12:0/1443059818 >> 172.21.15.12:0/1443059818 conn(0x55696600f210 msgr2=0x5569660abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 -- 172.21.15.12:0/1443059818 shutdown_connections 2024-05-09T00:07:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 -- 172.21.15.12:0/1443059818 wait complete. 2024-05-09T00:07:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 -- start start 2024-05-09T00:07:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.877+0000 7fae53577b80 1 Processor -- start 2024-05-09T00:07:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae53577b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 0x5569660bd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae53577b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b6e00 0x5569660baa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae53577b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae53577b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556965f9d560 con 0x5569660b6e00 2024-05-09T00:07:58.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae53577b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556965f9d200 con 0x5569660b7720 2024-05-09T00:07:58.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae53577b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556965e846a0 con 0x5569660b1890 2024-05-09T00:07:58.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae53577b80 10 client.0 _ll_register_callbacks cb 0x556965fa6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:58.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae45159700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 0x5569660bd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae45159700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 0x5569660bd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57642/0 (socket says 172.21.15.12:57642) 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae45159700 1 -- 172.21.15.12:0/3134819934 learned_addr learned my addr 172.21.15.12:0/3134819934 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.878+0000 7fae44958700 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b6e00 0x5569660baa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.879+0000 7fae4595a700 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.879+0000 7fae4595a700 1 -- 172.21.15.12:0/3134819934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 msgr2=0x5569660bd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.879+0000 7fae4595a700 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 0x5569660bd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.879+0000 7fae4595a700 1 -- 172.21.15.12:0/3134819934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b6e00 msgr2=0x5569660baa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.879+0000 7fae4595a700 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b6e00 0x5569660baa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.879+0000 7fae4595a700 1 -- 172.21.15.12:0/3134819934 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4010d550 con 0x5569660b7720 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.879+0000 7fae45159700 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 0x5569660bd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.879+0000 7fae44958700 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b6e00 0x5569660baa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.879+0000 7fae4595a700 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660bb0f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fae380eba80 tx=0x7fae380ebed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65678ae34c39a4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.879+0000 7fae28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:07:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.881+0000 7fae28ff9700 1 -- 172.21.15.12:0/3134819934 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380bab30 con 0x5569660b7720 2024-05-09T00:07:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.881+0000 7fae28ff9700 1 -- 172.21.15.12:0/3134819934 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fae380bf270 con 0x5569660b7720 2024-05-09T00:07:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.881+0000 7fae53577b80 1 -- 172.21.15.12:0/3134819934 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556965f5a270 con 0x5569660b7720 2024-05-09T00:07:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.882+0000 7fae28ff9700 1 -- 172.21.15.12:0/3134819934 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae380c38c0 con 0x5569660b7720 2024-05-09T00:07:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.882+0000 7fae53577b80 10 client.30824 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.882+0000 7fae53577b80 1 -- 172.21.15.12:0/3134819934 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x556965e2eb60 con 0x5569660b7720 2024-05-09T00:07:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.882+0000 7fae28ff9700 1 -- 172.21.15.12:0/3134819934 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556965f5a270 con 0x5569660b7720 2024-05-09T00:07:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.882+0000 7fae28ff9700 1 -- 172.21.15.12:0/3134819934 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fae380ba3a0 con 0x5569660b7720 2024-05-09T00:07:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.882+0000 7fae53577b80 10 client.30824 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.882+0000 7fae53577b80 10 client.30824 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:07:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.882+0000 7fae53577b80 20 client.30824 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.883+0000 7fae53577b80 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5569660c2080 0x5569660c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.883+0000 7fae53577b80 4 client.30824 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.883+0000 7fae53577b80 1 -- 172.21.15.12:0/3134819934 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556966080810 con 0x5569660c2080 2024-05-09T00:07:58.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.886+0000 7fae45159700 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5569660c2080 0x5569660c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.888+0000 7fae45159700 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5569660c2080 0x5569660c44a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db33637ea8c8310c server_cookie=52a0ae931bc90ad8 in_seq=0 out_seq=0 2024-05-09T00:07:58.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.888+0000 7fae28ff9700 10 client.30824 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:07:58.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae28ff9700 1 -- 172.21.15.12:0/3134819934 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556966080810 con 0x5569660c2080 2024-05-09T00:07:58.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae28ff9700 10 client.30824 handle_command_reply: tid=0 2024-05-09T00:07:58.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 client.30824 shutdown 2024-05-09T00:07:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 -- 172.21.15.12:0/3134819934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 msgr2=0x5569660bb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660bb0f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fae380eba80 tx=0x7fae380ebed0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 -- 172.21.15.12:0/3134819934 shutdown_connections 2024-05-09T00:07:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5569660c2080 0x5569660c44a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660bb0f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b1890 0x5569660bd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae28ff9700 0 client.30824 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:07:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 --2- 172.21.15.12:0/3134819934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b6e00 0x5569660baa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 -- 172.21.15.12:0/3134819934 >> 172.21.15.12:0/3134819934 conn(0x55696600f210 msgr2=0x5569660ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 -- 172.21.15.12:0/3134819934 shutdown_connections 2024-05-09T00:07:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.897+0000 7fae53577b80 1 -- 172.21.15.12:0/3134819934 wait complete. 2024-05-09T00:07:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.898+0000 7fae53577b80 20 client.30824 trim_cache size 0 max 16384 2024-05-09T00:07:58.915 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:58 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1098752696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:58.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.920+0000 7fae53577b80 1 Processor -- start 2024-05-09T00:07:58.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.920+0000 7fae53577b80 1 -- start start 2024-05-09T00:07:58.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.920+0000 7fae53577b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b1890 0x5569660c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.920+0000 7fae53577b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 0x5569660be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.920+0000 7fae53577b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b7720 0x5569660bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.920+0000 7fae53577b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569660e52b0 con 0x5569660b7720 2024-05-09T00:07:58.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.920+0000 7fae53577b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569660b8f50 con 0x5569660b1890 2024-05-09T00:07:58.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.920+0000 7fae53577b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569660a4040 con 0x5569660b6e00 2024-05-09T00:07:58.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae45159700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b1890 0x5569660c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae45159700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b1890 0x5569660c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33420/0 (socket says 172.21.15.12:33420) 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae45159700 1 -- 172.21.15.12:0/1497718620 learned_addr learned my addr 172.21.15.12:0/1497718620 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae4595a700 1 --2- 172.21.15.12:0/1497718620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b7720 0x5569660bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae44958700 1 --2- 172.21.15.12:0/1497718620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 0x5569660be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae45159700 1 -- 172.21.15.12:0/1497718620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 msgr2=0x5569660be0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae45159700 1 --2- 172.21.15.12:0/1497718620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 0x5569660be0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae45159700 1 -- 172.21.15.12:0/1497718620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b7720 msgr2=0x5569660bb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae45159700 1 --2- 172.21.15.12:0/1497718620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b7720 0x5569660bb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae45159700 1 -- 172.21.15.12:0/1497718620 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556965fc7560 con 0x5569660b1890 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.921+0000 7fae45159700 1 --2- 172.21.15.12:0/1497718620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b1890 0x5569660c4470 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fae3c000c00 tx=0x7fae3c019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7fb21f230f9b182 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.922+0000 7fae2bfff700 1 -- 172.21.15.12:0/1497718620 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c031070 con 0x5569660b1890 2024-05-09T00:07:58.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.922+0000 7fae2bfff700 1 -- 172.21.15.12:0/1497718620 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae3c00f5a0 con 0x5569660b1890 2024-05-09T00:07:58.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.922+0000 7fae2bfff700 1 -- 172.21.15.12:0/1497718620 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c01c040 con 0x5569660b1890 2024-05-09T00:07:58.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.923+0000 7fae53577b80 1 -- 172.21.15.12:0/1497718620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b1890 msgr2=0x5569660c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.923+0000 7fae53577b80 1 --2- 172.21.15.12:0/1497718620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b1890 0x5569660c4470 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fae3c000c00 tx=0x7fae3c019040 comp rx=0 tx=0).stop 2024-05-09T00:07:58.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.923+0000 7fae53577b80 1 -- 172.21.15.12:0/1497718620 shutdown_connections 2024-05-09T00:07:58.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.923+0000 7fae53577b80 1 --2- 172.21.15.12:0/1497718620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b7720 0x5569660bb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.923+0000 7fae53577b80 1 --2- 172.21.15.12:0/1497718620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b1890 0x5569660c4470 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.923+0000 7fae53577b80 1 --2- 172.21.15.12:0/1497718620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 0x5569660be0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.923+0000 7fae53577b80 1 -- 172.21.15.12:0/1497718620 >> 172.21.15.12:0/1497718620 conn(0x55696600f210 msgr2=0x5569660a8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.924+0000 7fae53577b80 1 -- 172.21.15.12:0/1497718620 shutdown_connections 2024-05-09T00:07:58.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.924+0000 7fae53577b80 1 -- 172.21.15.12:0/1497718620 wait complete. 2024-05-09T00:07:58.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.924+0000 7fae53577b80 1 -- start start 2024-05-09T00:07:58.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.924+0000 7fae53577b80 1 Processor -- start 2024-05-09T00:07:58.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.925+0000 7fae53577b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b1890 0x5569660c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.925+0000 7fae53577b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 0x5569660c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.925+0000 7fae53577b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.925+0000 7fae53577b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569660c5510 con 0x5569660b1890 2024-05-09T00:07:58.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.925+0000 7fae53577b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569660c5680 con 0x5569660b7720 2024-05-09T00:07:58.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.925+0000 7fae53577b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569660c5840 con 0x5569660b6e00 2024-05-09T00:07:58.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.925+0000 7fae53577b80 10 client.0 _ll_register_callbacks cb 0x556965fa6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.925+0000 7fae44958700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 0x5569660c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.926+0000 7fae44958700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 0x5569660c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57654/0 (socket says 172.21.15.12:57654) 2024-05-09T00:07:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.926+0000 7fae44958700 1 -- 172.21.15.12:0/3960230489 learned_addr learned my addr 172.21.15.12:0/3960230489 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.926+0000 7fae4595a700 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.926+0000 7fae4595a700 1 -- 172.21.15.12:0/3960230489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 msgr2=0x5569660c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.926+0000 7fae4595a700 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 0x5569660c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.926+0000 7fae4595a700 1 -- 172.21.15.12:0/3960230489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b1890 msgr2=0x5569660c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.926+0000 7fae4595a700 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b1890 0x5569660c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.926+0000 7fae4595a700 1 -- 172.21.15.12:0/3960230489 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556965e823b0 con 0x5569660b7720 2024-05-09T00:07:58.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.927+0000 7fae4595a700 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660c4e40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fae38000f80 tx=0x7fae38036ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c65be0145adce6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:58.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.927+0000 7fae28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:07:58.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.927+0000 7fae28ff9700 1 -- 172.21.15.12:0/3960230489 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380b7780 con 0x5569660b7720 2024-05-09T00:07:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.927+0000 7fae28ff9700 1 -- 172.21.15.12:0/3960230489 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fae380bf270 con 0x5569660b7720 2024-05-09T00:07:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.928+0000 7fae53577b80 1 -- 172.21.15.12:0/3960230489 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae3c01d040 con 0x5569660b7720 2024-05-09T00:07:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.928+0000 7fae28ff9700 1 -- 172.21.15.12:0/3960230489 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae380c38c0 con 0x5569660b7720 2024-05-09T00:07:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.928+0000 7fae53577b80 10 client.30830 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.928+0000 7fae53577b80 1 -- 172.21.15.12:0/3960230489 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x556965e86cd0 con 0x5569660b7720 2024-05-09T00:07:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.928+0000 7fae28ff9700 1 -- 172.21.15.12:0/3960230489 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae3809e080 con 0x5569660b7720 2024-05-09T00:07:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.928+0000 7fae28ff9700 1 -- 172.21.15.12:0/3960230489 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fae380c0190 con 0x5569660b7720 2024-05-09T00:07:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.929+0000 7fae53577b80 10 client.30830 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.929+0000 7fae53577b80 10 client.30830 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:07:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.929+0000 7fae53577b80 20 client.30830 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.929+0000 7fae53577b80 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556966131fd0 0x5569661343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.929+0000 7fae53577b80 4 client.30830 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:07:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.929+0000 7fae53577b80 1 -- 172.21.15.12:0/3960230489 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556965ff9ec0 con 0x556966131fd0 2024-05-09T00:07:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.929+0000 7fae45159700 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556966131fd0 0x5569661343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:58.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.930+0000 7fae45159700 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556966131fd0 0x5569661343f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f01d49ac655240a server_cookie=fcef9f5a3bd3353f in_seq=0 out_seq=0 2024-05-09T00:07:58.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.930+0000 7fae28ff9700 10 client.30830 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:07:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae28ff9700 1 -- 172.21.15.12:0/3960230489 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556965ff9ec0 con 0x556966131fd0 2024-05-09T00:07:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae28ff9700 10 client.30830 handle_command_reply: tid=0 2024-05-09T00:07:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 client.30830 shutdown 2024-05-09T00:07:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 -- 172.21.15.12:0/3960230489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 msgr2=0x5569660c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660c4e40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fae38000f80 tx=0x7fae38036ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 -- 172.21.15.12:0/3960230489 shutdown_connections 2024-05-09T00:07:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x556966131fd0 0x5569661343f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5569660b7720 0x5569660c4e40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569660b1890 0x5569660c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 --2- 172.21.15.12:0/3960230489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569660b6e00 0x5569660c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae28ff9700 0 client.30830 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:07:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 -- 172.21.15.12:0/3960230489 >> 172.21.15.12:0/3960230489 conn(0x55696600f210 msgr2=0x5569660a8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 -- 172.21.15.12:0/3960230489 shutdown_connections 2024-05-09T00:07:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 1 -- 172.21.15.12:0/3960230489 wait complete. 2024-05-09T00:07:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.935+0000 7fae53577b80 20 client.30830 trim_cache size 0 max 16384 2024-05-09T00:07:58.932 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:07:58.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.936+0000 7fae473bd700 1 -- 172.21.15.12:0/1094265210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae2c0a1390 msgr2=0x7fae2c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.936+0000 7fae473bd700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae2c0a1390 0x7fae2c0a3850 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fae300095c0 tx=0x7fae30009510 comp rx=0 tx=0).stop 2024-05-09T00:07:58.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.936+0000 7fae473bd700 1 -- 172.21.15.12:0/1094265210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae40105f90 msgr2=0x7fae4006afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:58.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.936+0000 7fae473bd700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae40105f90 0x7fae4006afd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fae3802d9e0 tx=0x7fae3802dcf0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.936+0000 7fae473bd700 1 -- 172.21.15.12:0/1094265210 shutdown_connections 2024-05-09T00:07:58.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.936+0000 7fae473bd700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae2c0a1390 0x7fae2c0a3850 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.936+0000 7fae473bd700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae40105f90 0x7fae4006afd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.936+0000 7fae473bd700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae401000f0 0x7fae40072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.936+0000 7fae473bd700 1 --2- 172.21.15.12:0/1094265210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae40100ab0 0x7fae4006aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:58.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.936+0000 7fae473bd700 1 -- 172.21.15.12:0/1094265210 >> 172.21.15.12:0/1094265210 conn(0x7fae40076ba0 msgr2=0x7fae400fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:58.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.939+0000 7fae473bd700 1 -- 172.21.15.12:0/1094265210 shutdown_connections 2024-05-09T00:07:58.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:58.939+0000 7fae473bd700 1 -- 172.21.15.12:0/1094265210 wait complete. 2024-05-09T00:07:58.952 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:07:59.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.237+0000 7f6487c6c700 1 -- 172.21.15.12:0/1231802525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f648011a780 msgr2=0x7f648011cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:59.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.237+0000 7f6487c6c700 1 --2- 172.21.15.12:0/1231802525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f648011a780 0x7f648011cb70 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f6478009f90 tx=0x7f647800b330 comp rx=0 tx=0).stop 2024-05-09T00:07:59.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.247+0000 7f6487c6c700 1 -- 172.21.15.12:0/1231802525 shutdown_connections 2024-05-09T00:07:59.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.247+0000 7f6487c6c700 1 --2- 172.21.15.12:0/1231802525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f648011a780 0x7f648011cb70 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.247+0000 7f6487c6c700 1 --2- 172.21.15.12:0/1231802525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648010f6a0 0x7f6480107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.247+0000 7f6487c6c700 1 --2- 172.21.15.12:0/1231802525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f648010fb10 0x7f64801084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.247+0000 7f6487c6c700 1 -- 172.21.15.12:0/1231802525 >> 172.21.15.12:0/1231802525 conn(0x7f648006bff0 msgr2=0x7f648006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:59.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.247+0000 7f6487c6c700 1 -- 172.21.15.12:0/1231802525 shutdown_connections 2024-05-09T00:07:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.250+0000 7f6487c6c700 1 -- 172.21.15.12:0/1231802525 wait complete. 2024-05-09T00:07:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.250+0000 7f6487c6c700 1 Processor -- start 2024-05-09T00:07:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.252+0000 7f6487c6c700 1 -- start start 2024-05-09T00:07:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.253+0000 7f6487c6c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f648010f6a0 0x7f6480118bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.253+0000 7f6487c6c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648010fb10 0x7f6480119150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.253+0000 7f6487c6c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f648011a780 0x7f6480111dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.253+0000 7f6487c6c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6480112590 con 0x7f648011a780 2024-05-09T00:07:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.253+0000 7f6487c6c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6480112700 con 0x7f648010f6a0 2024-05-09T00:07:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.253+0000 7f6487c6c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64801128c0 con 0x7f648010fb10 2024-05-09T00:07:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.253+0000 7f6486209700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f648011a780 0x7f6480111dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.253+0000 7f6486209700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f648011a780 0x7f6480111dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37638/0 (socket says 172.21.15.12:37638) 2024-05-09T00:07:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.253+0000 7f6486209700 1 -- 172.21.15.12:0/4195994375 learned_addr learned my addr 172.21.15.12:0/4195994375 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.254+0000 7f6486209700 1 -- 172.21.15.12:0/4195994375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648010fb10 msgr2=0x7f6480119150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:07:59.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.254+0000 7f6486209700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648010fb10 0x7f6480119150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.254+0000 7f6486209700 1 -- 172.21.15.12:0/4195994375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f648010f6a0 msgr2=0x7f6480118bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:07:59.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.254+0000 7f6486209700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f648010f6a0 0x7f6480118bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.254+0000 7f6486209700 1 -- 172.21.15.12:0/4195994375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6478009c40 con 0x7f648011a780 2024-05-09T00:07:59.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.254+0000 7f6486209700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f648011a780 0x7f6480111dc0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f6478009b10 tx=0x7f647800baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:59.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.255+0000 7f6476ffd700 1 -- 172.21.15.12:0/4195994375 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647800e040 con 0x7f648011a780 2024-05-09T00:07:59.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.255+0000 7f6476ffd700 1 -- 172.21.15.12:0/4195994375 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6478007700 con 0x7f648011a780 2024-05-09T00:07:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.255+0000 7f6476ffd700 1 -- 172.21.15.12:0/4195994375 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647801b6c0 con 0x7f648011a780 2024-05-09T00:07:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.255+0000 7f6487c6c700 1 -- 172.21.15.12:0/4195994375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6480112b40 con 0x7f648011a780 2024-05-09T00:07:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.255+0000 7f6487c6c700 1 -- 172.21.15.12:0/4195994375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64801169a0 con 0x7f648011a780 2024-05-09T00:07:59.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.261+0000 7f6493e26b80 1 Processor -- start 2024-05-09T00:07:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.262+0000 7f6476ffd700 1 -- 172.21.15.12:0/4195994375 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f647801b820 con 0x7f648011a780 2024-05-09T00:07:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.263+0000 7f6476ffd700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f646c0a1350 0x7f646c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.263+0000 7f6476ffd700 1 -- 172.21.15.12:0/4195994375 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f64780ce250 con 0x7f648011a780 2024-05-09T00:07:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.263+0000 7f6493e26b80 1 -- start start 2024-05-09T00:07:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.264+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.264+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb27720 0x5604adb21350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.264+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb21890 0x5604adb21cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.264+0000 7f6493e26b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ada78150 con 0x5604adb26e00 2024-05-09T00:07:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.264+0000 7f6493e26b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ada75b10 con 0x5604adb27720 2024-05-09T00:07:59.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.264+0000 7f6493e26b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ada71400 con 0x5604adb21890 2024-05-09T00:07:59.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.264+0000 7f6485207700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f646c0a1350 0x7f646c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.264+0000 7f6486209700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb21890 0x5604adb21cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.264+0000 7f6485207700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb27720 0x5604adb21350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.265+0000 7f6485a08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.265+0000 7f6485a08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb271e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37648/0 (socket says 172.21.15.12:37648) 2024-05-09T00:07:59.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.265+0000 7f6485a08700 1 -- 172.21.15.12:0/2592767103 learned_addr learned my addr 172.21.15.12:0/2592767103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:59.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.265+0000 7f6485a08700 1 -- 172.21.15.12:0/2592767103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb21890 msgr2=0x5604adb21cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:59.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.265+0000 7f6485a08700 1 --2- 172.21.15.12:0/2592767103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb21890 0x5604adb21cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.265+0000 7f6485a08700 1 -- 172.21.15.12:0/2592767103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb27720 msgr2=0x5604adb21350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:59.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.265+0000 7f6485a08700 1 --2- 172.21.15.12:0/2592767103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb27720 0x5604adb21350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.265+0000 7f6485a08700 1 -- 172.21.15.12:0/2592767103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604ad8f23b0 con 0x5604adb26e00 2024-05-09T00:07:59.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.266+0000 7f6485a08700 1 --2- 172.21.15.12:0/2592767103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb271e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f6470009c60 tx=0x7f6470009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f493cce4d4963635 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:59.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.266+0000 7f646bfff700 1 -- 172.21.15.12:0/2592767103 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64700097b0 con 0x5604adb26e00 2024-05-09T00:07:59.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.266+0000 7f646bfff700 1 -- 172.21.15.12:0/2592767103 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6470004cf0 con 0x5604adb26e00 2024-05-09T00:07:59.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.266+0000 7f646bfff700 1 -- 172.21.15.12:0/2592767103 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470005720 con 0x5604adb26e00 2024-05-09T00:07:59.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.267+0000 7f6493e26b80 1 -- 172.21.15.12:0/2592767103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 msgr2=0x5604adb271e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:59.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.267+0000 7f6493e26b80 1 --2- 172.21.15.12:0/2592767103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb271e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f6470009c60 tx=0x7f6470009f70 comp rx=0 tx=0).stop 2024-05-09T00:07:59.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.268+0000 7f6485207700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f646c0a1350 0x7f646c0a3810 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x5604adb1dbd0 tx=0x7f647c005e10 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:58 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1098752696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:59.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:58 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1098752696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.278+0000 7f6493e26b80 1 -- 172.21.15.12:0/2592767103 shutdown_connections 2024-05-09T00:07:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.278+0000 7f6493e26b80 1 --2- 172.21.15.12:0/2592767103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb21890 0x5604adb21cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.278+0000 7f6493e26b80 1 --2- 172.21.15.12:0/2592767103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb271e0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.278+0000 7f6493e26b80 1 --2- 172.21.15.12:0/2592767103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb27720 0x5604adb21350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.278+0000 7f6493e26b80 1 -- 172.21.15.12:0/2592767103 >> 172.21.15.12:0/2592767103 conn(0x5604ada7f210 msgr2=0x5604adb1bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.278+0000 7f6493e26b80 1 -- 172.21.15.12:0/2592767103 shutdown_connections 2024-05-09T00:07:59.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.279+0000 7f6493e26b80 1 -- 172.21.15.12:0/2592767103 wait complete. 2024-05-09T00:07:59.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.279+0000 7f6493e26b80 1 -- start start 2024-05-09T00:07:59.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.279+0000 7f6493e26b80 1 Processor -- start 2024-05-09T00:07:59.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.280+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb21890 0x5604adb2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.280+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb2b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.280+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 0x5604adb2b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.280+0000 7f6493e26b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ada0d560 con 0x5604adb26e00 2024-05-09T00:07:59.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.280+0000 7f6493e26b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ada0d200 con 0x5604adb21890 2024-05-09T00:07:59.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.280+0000 7f6493e26b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ad8f46a0 con 0x5604adb27720 2024-05-09T00:07:59.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.280+0000 7f6493e26b80 10 client.0 _ll_register_callbacks cb 0x5604ada16b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:59.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.281+0000 7f6486209700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 0x5604adb2b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.281+0000 7f6486209700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 0x5604adb2b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57690/0 (socket says 172.21.15.12:57690) 2024-05-09T00:07:59.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.281+0000 7f6486209700 1 -- 172.21.15.12:0/3565946834 learned_addr learned my addr 172.21.15.12:0/3565946834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:59.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.281+0000 7f6485207700 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb2b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.281+0000 7f6486209700 1 -- 172.21.15.12:0/3565946834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb21890 msgr2=0x5604adb2ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:59.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.281+0000 7f6486209700 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb21890 0x5604adb2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.281+0000 7f6486209700 1 -- 172.21.15.12:0/3565946834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 msgr2=0x5604adb2b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:59.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.281+0000 7f6486209700 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb2b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.281+0000 7f6486209700 1 -- 172.21.15.12:0/3565946834 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6470009910 con 0x5604adb27720 2024-05-09T00:07:59.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.282+0000 7f6486209700 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 0x5604adb2b580 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6478097900 tx=0x7f647809ae10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe4d8c85319c04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:59.279 INFO:tasks.workunit.client.0.smithi012.stdout: 10 293190 31.40 MB/sec execute 354 sec latency 195.336 ms 2024-05-09T00:07:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.293+0000 7f64697fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:07:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.293+0000 7f64697fa700 1 -- 172.21.15.12:0/3565946834 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647809b860 con 0x5604adb27720 2024-05-09T00:07:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.293+0000 7f64697fa700 1 -- 172.21.15.12:0/3565946834 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f647809bb00 con 0x5604adb27720 2024-05-09T00:07:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.293+0000 7f6493e26b80 1 -- 172.21.15.12:0/3565946834 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604ad9ca270 con 0x5604adb27720 2024-05-09T00:07:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.293+0000 7f64697fa700 1 -- 172.21.15.12:0/3565946834 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f647801bce0 con 0x5604adb27720 2024-05-09T00:07:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.293+0000 7f6493e26b80 10 client.31081 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.293+0000 7f6493e26b80 1 -- 172.21.15.12:0/3565946834 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5604ad89eb60 con 0x5604adb27720 2024-05-09T00:07:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.296+0000 7f64697fa700 1 -- 172.21.15.12:0/3565946834 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64780b7020 con 0x5604adb27720 2024-05-09T00:07:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.297+0000 7f64697fa700 1 -- 172.21.15.12:0/3565946834 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f647809a8f0 con 0x5604adb27720 2024-05-09T00:07:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.297+0000 7f6493e26b80 10 client.31081 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.297+0000 7f6493e26b80 10 client.31081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.297+0000 7f6493e26b80 20 client.31081 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.297+0000 7f6493e26b80 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604adb3ae80 0x5604adb3d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.297+0000 7f6493e26b80 4 client.31081 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:07:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.297+0000 7f6493e26b80 1 -- 172.21.15.12:0/3565946834 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604adaf0810 con 0x5604adb3ae80 2024-05-09T00:07:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.297+0000 7f6485a08700 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604adb3ae80 0x5604adb3d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.312+0000 7f6485a08700 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604adb3ae80 0x5604adb3d2a0 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436590097411cafa server_cookie=e65653bec1d26ef8 in_seq=0 out_seq=0 2024-05-09T00:07:59.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.315+0000 7f64697fa700 10 client.31081 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:59.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.328+0000 7f64697fa700 1 -- 172.21.15.12:0/3565946834 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604adaf0810 con 0x5604adb3ae80 2024-05-09T00:07:59.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.328+0000 7f64697fa700 10 client.31081 handle_command_reply: tid=0 2024-05-09T00:07:59.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.330+0000 7f6493e26b80 1 client.31081 shutdown 2024-05-09T00:07:59.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.331+0000 7f6493e26b80 1 -- 172.21.15.12:0/3565946834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 msgr2=0x5604adb2b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:59.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.331+0000 7f6493e26b80 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 0x5604adb2b580 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6478097900 tx=0x7f647809ae10 comp rx=0 tx=0).stop 2024-05-09T00:07:59.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.331+0000 7f6493e26b80 1 -- 172.21.15.12:0/3565946834 shutdown_connections 2024-05-09T00:07:59.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.331+0000 7f6493e26b80 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 0x5604adb2b580 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.331+0000 7f6493e26b80 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604adb3ae80 0x5604adb3d2a0 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.331+0000 7f6493e26b80 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb21890 0x5604adb2ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.331+0000 7f6493e26b80 1 --2- 172.21.15.12:0/3565946834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb2b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.331+0000 7f6493e26b80 1 -- 172.21.15.12:0/3565946834 >> 172.21.15.12:0/3565946834 conn(0x5604ada7f210 msgr2=0x5604adb1b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:59.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.332+0000 7f64697fa700 0 client.31081 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:59.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.338+0000 7f6493e26b80 1 -- 172.21.15.12:0/3565946834 shutdown_connections 2024-05-09T00:07:59.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.343+0000 7f6493e26b80 1 -- 172.21.15.12:0/3565946834 wait complete. 2024-05-09T00:07:59.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.343+0000 7f6493e26b80 20 client.31081 trim_cache size 0 max 16384 2024-05-09T00:07:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.365+0000 7f6493e26b80 1 Processor -- start 2024-05-09T00:07:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6493e26b80 1 -- start start 2024-05-09T00:07:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb21890 0x5604adb1bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 0x5604adb29910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6493e26b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604adb16800 con 0x5604adb26e00 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6493e26b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604adb42450 con 0x5604adb21890 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6493e26b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604adb14040 con 0x5604adb27720 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6485207700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6485207700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb293d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37660/0 (socket says 172.21.15.12:37660) 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6485207700 1 -- 172.21.15.12:0/2460353578 learned_addr learned my addr 172.21.15.12:0/2460353578 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.366+0000 7f6486209700 1 --2- 172.21.15.12:0/2460353578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 0x5604adb29910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.367+0000 7f6485207700 1 -- 172.21.15.12:0/2460353578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 msgr2=0x5604adb29910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.367+0000 7f6485207700 1 --2- 172.21.15.12:0/2460353578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 0x5604adb29910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.367+0000 7f6485207700 1 -- 172.21.15.12:0/2460353578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb21890 msgr2=0x5604adb1bd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.367+0000 7f6485207700 1 --2- 172.21.15.12:0/2460353578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb21890 0x5604adb1bd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.367+0000 7f6485207700 1 -- 172.21.15.12:0/2460353578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604ada37560 con 0x5604adb26e00 2024-05-09T00:07:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.367+0000 7f6485207700 1 --2- 172.21.15.12:0/2460353578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb293d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f648010ff30 tx=0x7f647c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4054d22c039d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.371+0000 7f6474da4700 1 -- 172.21.15.12:0/2460353578 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c0128e0 con 0x5604adb26e00 2024-05-09T00:07:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.371+0000 7f6474da4700 1 -- 172.21.15.12:0/2460353578 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f647c004830 con 0x5604adb26e00 2024-05-09T00:07:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.371+0000 7f6474da4700 1 -- 172.21.15.12:0/2460353578 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c005260 con 0x5604adb26e00 2024-05-09T00:07:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.372+0000 7f6493e26b80 1 -- 172.21.15.12:0/2460353578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 msgr2=0x5604adb293d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.372+0000 7f6493e26b80 1 --2- 172.21.15.12:0/2460353578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb293d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f648010ff30 tx=0x7f647c012000 comp rx=0 tx=0).stop 2024-05-09T00:07:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.373+0000 7f6493e26b80 1 -- 172.21.15.12:0/2460353578 shutdown_connections 2024-05-09T00:07:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.373+0000 7f6493e26b80 1 --2- 172.21.15.12:0/2460353578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb27720 0x5604adb29910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.373+0000 7f6493e26b80 1 --2- 172.21.15.12:0/2460353578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb21890 0x5604adb1bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.373+0000 7f6493e26b80 1 --2- 172.21.15.12:0/2460353578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb26e00 0x5604adb293d0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.373+0000 7f6493e26b80 1 -- 172.21.15.12:0/2460353578 >> 172.21.15.12:0/2460353578 conn(0x5604ada7f210 msgr2=0x5604adb24c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.373+0000 7f6493e26b80 1 -- 172.21.15.12:0/2460353578 shutdown_connections 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.373+0000 7f6493e26b80 1 -- 172.21.15.12:0/2460353578 wait complete. 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.373+0000 7f6493e26b80 1 -- start start 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.374+0000 7f6493e26b80 1 Processor -- start 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.374+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb21890 0x5604adb301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.374+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb26e00 0x5604adb30720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.374+0000 7f6493e26b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb27720 0x5604adb90340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.374+0000 7f6493e26b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604adb30fe0 con 0x5604adb21890 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.374+0000 7f6493e26b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604adb90880 con 0x5604adb27720 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.374+0000 7f6493e26b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604adb909c0 con 0x5604adb26e00 2024-05-09T00:07:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.374+0000 7f6493e26b80 10 client.0 _ll_register_callbacks cb 0x5604ada16b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6485a08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb21890 0x5604adb301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6485a08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb21890 0x5604adb301e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37670/0 (socket says 172.21.15.12:37670) 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6485207700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb26e00 0x5604adb30720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6485a08700 1 -- 172.21.15.12:0/1565849652 learned_addr learned my addr 172.21.15.12:0/1565849652 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6486209700 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb27720 0x5604adb90340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6485207700 1 -- 172.21.15.12:0/1565849652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb27720 msgr2=0x5604adb90340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6485207700 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb27720 0x5604adb90340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6485207700 1 -- 172.21.15.12:0/1565849652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb21890 msgr2=0x5604adb301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6485207700 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb21890 0x5604adb301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6485207700 1 -- 172.21.15.12:0/1565849652 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c00e840 con 0x5604adb26e00 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.375+0000 7f6485a08700 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb21890 0x5604adb301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:07:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.376+0000 7f6485207700 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb26e00 0x5604adb30720 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f647c014040 tx=0x7f647c012b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=840902b046fd29da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.376+0000 7f64697fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.379+0000 7f64697fa700 1 -- 172.21.15.12:0/1565849652 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c012db0 con 0x5604adb26e00 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.379+0000 7f64697fa700 1 -- 172.21.15.12:0/1565849652 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f647c01a040 con 0x5604adb26e00 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.380+0000 7f6493e26b80 1 -- 172.21.15.12:0/1565849652 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6470004cf0 con 0x5604adb26e00 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.380+0000 7f64697fa700 1 -- 172.21.15.12:0/1565849652 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f647c0051b0 con 0x5604adb26e00 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.380+0000 7f6493e26b80 10 client.31087 fetch_fsmap learned FSMap version 76 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.380+0000 7f6493e26b80 1 -- 172.21.15.12:0/1565849652 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5604ad8f6cd0 con 0x5604adb26e00 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.380+0000 7f64697fa700 1 -- 172.21.15.12:0/1565849652 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6470004cf0 con 0x5604adb26e00 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.380+0000 7f64697fa700 1 -- 172.21.15.12:0/1565849652 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f647c039070 con 0x5604adb26e00 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.381+0000 7f6493e26b80 10 client.31087 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.381+0000 7f6493e26b80 10 client.31087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:07:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.381+0000 7f6493e26b80 20 client.31087 populate_metadata read hostname 'smithi012' 2024-05-09T00:07:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.382+0000 7f6493e26b80 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604adb91fc0 0x5604adb943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.382+0000 7f6493e26b80 4 client.31087 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:07:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.382+0000 7f6493e26b80 1 -- 172.21.15.12:0/1565849652 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5604ada69ec0 con 0x5604adb91fc0 2024-05-09T00:07:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.382+0000 7f6485a08700 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604adb91fc0 0x5604adb943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.382+0000 7f64697fa700 10 client.31087 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.382+0000 7f6485a08700 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604adb91fc0 0x5604adb943e0 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990435fa62d93269 server_cookie=e9bf66e082fd66f in_seq=0 out_seq=0 2024-05-09T00:07:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f64697fa700 1 -- 172.21.15.12:0/1565849652 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5604ada69ec0 con 0x5604adb91fc0 2024-05-09T00:07:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f64697fa700 10 client.31087 handle_command_reply: tid=0 2024-05-09T00:07:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 client.31087 shutdown 2024-05-09T00:07:59.380 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:07:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 -- 172.21.15.12:0/1565849652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb26e00 msgr2=0x5604adb30720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb26e00 0x5604adb30720 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f647c014040 tx=0x7f647c012b10 comp rx=0 tx=0).stop 2024-05-09T00:07:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 -- 172.21.15.12:0/1565849652 shutdown_connections 2024-05-09T00:07:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5604adb91fc0 0x5604adb943e0 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5604adb27720 0x5604adb90340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604adb21890 0x5604adb301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 --2- 172.21.15.12:0/1565849652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604adb26e00 0x5604adb30720 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 -- 172.21.15.12:0/1565849652 >> 172.21.15.12:0/1565849652 conn(0x5604ada7f210 msgr2=0x5604adb25be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f64697fa700 0 client.31087 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:07:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 -- 172.21.15.12:0/1565849652 shutdown_connections 2024-05-09T00:07:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 1 -- 172.21.15.12:0/1565849652 wait complete. 2024-05-09T00:07:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.384+0000 7f6493e26b80 20 client.31087 trim_cache size 0 max 16384 2024-05-09T00:07:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.387+0000 7f6487c6c700 1 -- 172.21.15.12:0/4195994375 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f646c0a1350 msgr2=0x7f646c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.387+0000 7f6487c6c700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f646c0a1350 0x7f646c0a3810 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x5604adb1dbd0 tx=0x7f647c005e10 comp rx=0 tx=0).stop 2024-05-09T00:07:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.387+0000 7f6487c6c700 1 -- 172.21.15.12:0/4195994375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f648011a780 msgr2=0x7f6480111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.387+0000 7f6487c6c700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f648011a780 0x7f6480111dc0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f6478009b10 tx=0x7f647800baa0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.394+0000 7f6487c6c700 1 -- 172.21.15.12:0/4195994375 shutdown_connections 2024-05-09T00:07:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.394+0000 7f6487c6c700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f646c0a1350 0x7f646c0a3810 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.394+0000 7f6487c6c700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f648011a780 0x7f6480111dc0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.394+0000 7f6487c6c700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f648010f6a0 0x7f6480118bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.394+0000 7f6487c6c700 1 --2- 172.21.15.12:0/4195994375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648010fb10 0x7f6480119150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.394+0000 7f6487c6c700 1 -- 172.21.15.12:0/4195994375 >> 172.21.15.12:0/4195994375 conn(0x7f648006bff0 msgr2=0x7f648006f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.394+0000 7f6487c6c700 1 -- 172.21.15.12:0/4195994375 shutdown_connections 2024-05-09T00:07:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:07:59.395+0000 7f6487c6c700 1 -- 172.21.15.12:0/4195994375 wait complete. 2024-05-09T00:08:00.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:07:59 smithi012 ceph-mon[42503]: pgmap v693: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 591 op/s 2024-05-09T00:08:00.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:07:59 smithi204 ceph-mon[47373]: pgmap v693: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 591 op/s 2024-05-09T00:08:00.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:07:59 smithi160 ceph-mon[46152]: pgmap v693: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 591 op/s 2024-05-09T00:08:00.279 INFO:tasks.workunit.client.0.smithi012.stdout: 10 293697 31.40 MB/sec execute 355 sec latency 253.658 ms 2024-05-09T00:08:00.415 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:08:00.416 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:08:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.606+0000 7fd2fe371700 1 -- 172.21.15.12:0/774630291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2f8101b10 msgr2=0x7fd2f8105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.606+0000 7fd2fe371700 1 --2- 172.21.15.12:0/774630291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2f8101b10 0x7fd2f8105b50 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009990 tx=0x7fd2ec009ca0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.606+0000 7fd2fe371700 1 -- 172.21.15.12:0/774630291 shutdown_connections 2024-05-09T00:08:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.606+0000 7fd2fe371700 1 --2- 172.21.15.12:0/774630291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f8106280 0x7fd2f810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.606+0000 7fd2fe371700 1 --2- 172.21.15.12:0/774630291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8101160 0x7fd2f8101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.606+0000 7fd2fe371700 1 --2- 172.21.15.12:0/774630291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2f8101b10 0x7fd2f8105b50 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.606+0000 7fd2fe371700 1 -- 172.21.15.12:0/774630291 >> 172.21.15.12:0/774630291 conn(0x7fd2f80fc9e0 msgr2=0x7fd2f80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.606+0000 7fd2fe371700 1 -- 172.21.15.12:0/774630291 shutdown_connections 2024-05-09T00:08:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.606+0000 7fd2fe371700 1 -- 172.21.15.12:0/774630291 wait complete. 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.607+0000 7fd2fe371700 1 Processor -- start 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.607+0000 7fd2fe371700 1 -- start start 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.607+0000 7fd2fe371700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8101160 0x7fd2f8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.607+0000 7fd2fe371700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f8106280 0x7fd2f8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.607+0000 7fd2fe371700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2f819df00 0x7fd2f819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.607+0000 7fd2fe371700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f819e980 con 0x7fd2f819df00 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.607+0000 7fd2fe371700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f819eaf0 con 0x7fd2f8106280 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.607+0000 7fd2fe371700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f819ecb0 con 0x7fd2f8101160 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fc90e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2f819df00 0x7fd2f819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fc90e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2f819df00 0x7fd2f819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37688/0 (socket says 172.21.15.12:37688) 2024-05-09T00:08:00.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fc90e700 1 -- 172.21.15.12:0/2891095779 learned_addr learned my addr 172.21.15.12:0/2891095779 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8101160 0x7fd2f8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2f77fe700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f8106280 0x7fd2f8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fc90e700 1 -- 172.21.15.12:0/2891095779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8101160 msgr2=0x7fd2f8199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8101160 0x7fd2f8199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fc90e700 1 -- 172.21.15.12:0/2891095779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f8106280 msgr2=0x7fd2f8199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f8106280 0x7fd2f8199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fc90e700 1 -- 172.21.15.12:0/2891095779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec0095f0 con 0x7fd2f819df00 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2f819df00 0x7fd2f819e380 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0007ee0 tx=0x7fd2f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2f57fa700 1 -- 172.21.15.12:0/2891095779 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f000ddd0 con 0x7fd2f819df00 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fe371700 1 -- 172.21.15.12:0/2891095779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f81a6210 con 0x7fd2f819df00 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2f57fa700 1 -- 172.21.15.12:0/2891095779 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2f000f040 con 0x7fd2f819df00 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2f57fa700 1 -- 172.21.15.12:0/2891095779 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00048e0 con 0x7fd2f819df00 2024-05-09T00:08:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.608+0000 7fd2fe371700 1 -- 172.21.15.12:0/2891095779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f81a66f0 con 0x7fd2f819df00 2024-05-09T00:08:00.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.611+0000 7fd2f57fa700 1 -- 172.21.15.12:0/2891095779 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd2f0003a00 con 0x7fd2f819df00 2024-05-09T00:08:00.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.611+0000 7fd2f57fa700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2e00a1290 0x7fd2e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.611+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2e00a1290 0x7fd2e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.611+0000 7fd2f57fa700 1 -- 172.21.15.12:0/2891095779 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd2f00cc560 con 0x7fd2f819df00 2024-05-09T00:08:00.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.612+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2e00a1290 0x7fd2e00a3750 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009c60 tx=0x7fd2e8009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.612+0000 7fd30a52bb80 1 Processor -- start 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.612+0000 7fd30a52bb80 1 -- start start 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x55840924d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924d7a0 0x5584092473d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409247d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840919e1d0 con 0x558409247910 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840919bb90 con 0x55840924ce80 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409197480 con 0x55840924d7a0 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2fc90e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409247d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2fc90e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409247d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37696/0 (socket says 172.21.15.12:37696) 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2fc90e700 1 -- 172.21.15.12:0/496591473 learned_addr learned my addr 172.21.15.12:0/496591473 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2f77fe700 1 --2- 172.21.15.12:0/496591473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924d7a0 0x5584092473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/496591473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x55840924d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2fc90e700 1 -- 172.21.15.12:0/496591473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924d7a0 msgr2=0x5584092473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/496591473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924d7a0 0x5584092473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2fc90e700 1 -- 172.21.15.12:0/496591473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 msgr2=0x55840924d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/496591473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x55840924d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2fc90e700 1 -- 172.21.15.12:0/496591473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558409018430 con 0x558409247910 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/496591473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409247d50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0095b80 tx=0x7fd2f0095ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100112b3ace19474 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2de7fc700 1 -- 172.21.15.12:0/496591473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0099d80 con 0x558409247910 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2de7fc700 1 -- 172.21.15.12:0/496591473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2f0099680 con 0x558409247910 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.613+0000 7fd2de7fc700 1 -- 172.21.15.12:0/496591473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0097b20 con 0x558409247910 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.614+0000 7fd30a52bb80 1 -- 172.21.15.12:0/496591473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 msgr2=0x558409247d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.614+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/496591473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409247d50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0095b80 tx=0x7fd2f0095ac0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.614+0000 7fd30a52bb80 1 -- 172.21.15.12:0/496591473 shutdown_connections 2024-05-09T00:08:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.614+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/496591473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409247d50 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.614+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/496591473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x55840924d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.614+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/496591473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924d7a0 0x5584092473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.614+0000 7fd30a52bb80 1 -- 172.21.15.12:0/496591473 >> 172.21.15.12:0/496591473 conn(0x5584091a5290 msgr2=0x558409241e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.614+0000 7fd30a52bb80 1 -- 172.21.15.12:0/496591473 shutdown_connections 2024-05-09T00:08:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.614+0000 7fd30a52bb80 1 -- 172.21.15.12:0/496591473 wait complete. 2024-05-09T00:08:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.615+0000 7fd30a52bb80 1 -- start start 2024-05-09T00:08:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.615+0000 7fd30a52bb80 1 Processor -- start 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.615+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409250b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.615+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x5584092510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.615+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924d7a0 0x5584092515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.615+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584091335e0 con 0x558409247910 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.615+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409133280 con 0x55840924ce80 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.615+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840901a720 con 0x55840924d7a0 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd30a52bb80 10 client.0 _ll_register_callbacks cb 0x55840913cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409250b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409250b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37702/0 (socket says 172.21.15.12:37702) 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2f7fff700 1 -- 172.21.15.12:0/2432244192 learned_addr learned my addr 172.21.15.12:0/2432244192 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2f77fe700 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x5584092510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924d7a0 0x5584092515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2f7fff700 1 -- 172.21.15.12:0/2432244192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924d7a0 msgr2=0x5584092515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924d7a0 0x5584092515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2f7fff700 1 -- 172.21.15.12:0/2432244192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 msgr2=0x5584092510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x5584092510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2f7fff700 1 -- 172.21.15.12:0/2432244192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f81a66f0 con 0x558409247910 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409250b60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd2e801ec10 tx=0x7fd2e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bab48dd05f53aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2432244192 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e800ebd0 con 0x558409247910 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2432244192 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd2e800ed30 con 0x558409247910 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.616+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2432244192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584090f02f0 con 0x558409247910 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2432244192 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2e802f400 con 0x558409247910 2024-05-09T00:08:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd30a52bb80 10 client.21381 fetch_fsmap learned FSMap version 76 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2432244192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x558408fc4be0 con 0x558409247910 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2432244192 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e8033020 con 0x558409247910 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2432244192 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd2e801c450 con 0x558409247910 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd30a52bb80 10 client.21381 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd30a52bb80 10 client.21381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd30a52bb80 20 client.21381 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584092612a0 0x5584092636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd30a52bb80 4 client.21381 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2432244192 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558409216890 con 0x5584092612a0 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.617+0000 7fd2f77fe700 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584092612a0 0x5584092636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.618+0000 7fd2d77fe700 10 client.21381 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.618+0000 7fd2f77fe700 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584092612a0 0x5584092636c0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f77bcf15e77a93 server_cookie=fcd913142b89eac in_seq=0 out_seq=0 2024-05-09T00:08:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.618+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2432244192 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558409216890 con 0x5584092612a0 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.618+0000 7fd2d77fe700 10 client.21381 handle_command_reply: tid=0 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.618+0000 7fd30a52bb80 1 client.21381 shutdown 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2432244192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 msgr2=0x558409250b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409250b60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd2e801ec10 tx=0x7fd2e80047c0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2432244192 shutdown_connections 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924d7a0 0x5584092515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558409247910 0x558409250b60 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584092612a0 0x5584092636c0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2432244192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x5584092510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd2d77fe700 0 client.21381 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2432244192 >> 172.21.15.12:0/2432244192 conn(0x5584091a5290 msgr2=0x558409241620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2432244192 shutdown_connections 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2432244192 wait complete. 2024-05-09T00:08:00.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.619+0000 7fd30a52bb80 20 client.21381 trim_cache size 0 max 16384 2024-05-09T00:08:00.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.637+0000 7fd30a52bb80 1 Processor -- start 2024-05-09T00:08:00.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.637+0000 7fd30a52bb80 1 -- start start 2024-05-09T00:08:00.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558409247910 0x55840924fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x55840924c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x558409241da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840925e9d0 con 0x55840924d7a0 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409253bd0 con 0x55840924ce80 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584092688e0 con 0x558409247910 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2fc90e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x558409241da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2fc90e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x558409241da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37708/0 (socket says 172.21.15.12:37708) 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2fc90e700 1 -- 172.21.15.12:0/40970612 learned_addr learned my addr 172.21.15.12:0/40970612 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2f77fe700 1 --2- 172.21.15.12:0/40970612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x55840924c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/40970612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558409247910 0x55840924fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2fc90e700 1 -- 172.21.15.12:0/40970612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558409247910 msgr2=0x55840924fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/40970612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558409247910 0x55840924fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2fc90e700 1 -- 172.21.15.12:0/40970612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 msgr2=0x55840924c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/40970612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x55840924c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2fc90e700 1 -- 172.21.15.12:0/40970612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840915d5e0 con 0x55840924d7a0 2024-05-09T00:08:00.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2f77fe700 1 --2- 172.21.15.12:0/40970612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x55840924c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:08:00.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.638+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/40970612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x558409241da0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000d7b0 tx=0x7fd2f0004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74976902ae82a3b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd2deffd700 1 -- 172.21.15.12:0/40970612 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00b90c0 con 0x55840924d7a0 2024-05-09T00:08:00.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd2deffd700 1 -- 172.21.15.12:0/40970612 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2f0099d80 con 0x55840924d7a0 2024-05-09T00:08:00.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd2deffd700 1 -- 172.21.15.12:0/40970612 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0003dd0 con 0x55840924d7a0 2024-05-09T00:08:00.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd30a52bb80 1 -- 172.21.15.12:0/40970612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 msgr2=0x558409241da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/40970612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x558409241da0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000d7b0 tx=0x7fd2f0004380 comp rx=0 tx=0).stop 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd30a52bb80 1 -- 172.21.15.12:0/40970612 shutdown_connections 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/40970612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x558409241da0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/40970612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558409247910 0x55840924fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/40970612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55840924ce80 0x55840924c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd30a52bb80 1 -- 172.21.15.12:0/40970612 >> 172.21.15.12:0/40970612 conn(0x5584091a5290 msgr2=0x55840924db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd30a52bb80 1 -- 172.21.15.12:0/40970612 shutdown_connections 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.639+0000 7fd30a52bb80 1 -- 172.21.15.12:0/40970612 wait complete. 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.640+0000 7fd30a52bb80 1 -- start start 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.640+0000 7fd30a52bb80 1 Processor -- start 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.640+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558409247910 0x5584092b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.640+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924ce80 0x5584092b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.640+0000 7fd30a52bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x5584092579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.640+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409257f80 con 0x55840924d7a0 2024-05-09T00:08:00.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.640+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584092580c0 con 0x558409247910 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.640+0000 7fd30a52bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409258280 con 0x55840924ce80 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd30a52bb80 10 client.0 _ll_register_callbacks cb 0x55840913cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2fc90e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x5584092579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2f77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924ce80 0x5584092b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2fc90e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x5584092579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37724/0 (socket says 172.21.15.12:37724) 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2fc90e700 1 -- 172.21.15.12:0/2034001519 learned_addr learned my addr 172.21.15.12:0/2034001519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558409247910 0x5584092b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2fc90e700 1 -- 172.21.15.12:0/2034001519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924ce80 msgr2=0x5584092b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924ce80 0x5584092b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2fc90e700 1 -- 172.21.15.12:0/2034001519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558409247910 msgr2=0x5584092b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558409247910 0x5584092b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2fc90e700 1 -- 172.21.15.12:0/2034001519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558409018430 con 0x55840924d7a0 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2f77fe700 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924ce80 0x5584092b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:00.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2fc90e700 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x5584092579b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00ca240 tx=0x7fd2f00073e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afd3f7e64b379ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2034001519 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00a1b00 con 0x55840924d7a0 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2034001519 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd2f009d4b0 con 0x55840924d7a0 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2034001519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2f0099680 con 0x55840924d7a0 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.641+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2034001519 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f0095d90 con 0x55840924d7a0 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd30a52bb80 10 client.21387 fetch_fsmap learned FSMap version 76 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2034001519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55840901cd50 con 0x55840924d7a0 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2034001519 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2f0099680 con 0x55840924d7a0 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2034001519 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd2f0098c90 con 0x55840924d7a0 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd30a52bb80 10 client.21387 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd30a52bb80 10 client.21387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd30a52bb80 20 client.21387 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584092b99b0 0x5584092bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd30a52bb80 4 client.21387 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2034001519 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55840918ff40 con 0x5584092b99b0 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.642+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584092b99b0 0x5584092bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.643+0000 7fd2d77fe700 10 client.21387 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.643+0000 7fd2f7fff700 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584092b99b0 0x5584092bbda0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c7dcbe0fbcbca5 server_cookie=1eb490a7c63829b6 in_seq=0 out_seq=0 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd2d77fe700 1 -- 172.21.15.12:0/2034001519 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55840918ff40 con 0x5584092b99b0 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd2d77fe700 10 client.21387 handle_command_reply: tid=0 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 client.21387 shutdown 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2034001519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 msgr2=0x5584092579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x5584092579b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00ca240 tx=0x7fd2f00073e0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2034001519 shutdown_connections 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840924d7a0 0x5584092579b0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5584092b99b0 0x5584092bbda0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558409247910 0x5584092b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 --2- 172.21.15.12:0/2034001519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55840924ce80 0x5584092b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd2d77fe700 0 client.21387 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:00.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2034001519 >> 172.21.15.12:0/2034001519 conn(0x5584091a5290 msgr2=0x55840924db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2034001519 shutdown_connections 2024-05-09T00:08:00.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 1 -- 172.21.15.12:0/2034001519 wait complete. 2024-05-09T00:08:00.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.644+0000 7fd30a52bb80 20 client.21387 trim_cache size 0 max 16384 2024-05-09T00:08:00.645 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:08:00.645 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:08:00.645 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "726f87ca-b256-499f-9644-aeb268b06fea", 2024-05-09T00:08:00.645 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:08:00.645 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:08:00.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 -- 172.21.15.12:0/2891095779 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2e00a1290 msgr2=0x7fd2e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2e00a1290 0x7fd2e00a3750 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009c60 tx=0x7fd2e8009260 comp rx=0 tx=0).stop 2024-05-09T00:08:00.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 -- 172.21.15.12:0/2891095779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2f819df00 msgr2=0x7fd2f819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2f819df00 0x7fd2f819e380 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0007ee0 tx=0x7fd2f000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 -- 172.21.15.12:0/2891095779 shutdown_connections 2024-05-09T00:08:00.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd2e00a1290 0x7fd2e00a3750 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2f819df00 0x7fd2f819e380 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8101160 0x7fd2f8199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 --2- 172.21.15.12:0/2891095779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f8106280 0x7fd2f8199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 -- 172.21.15.12:0/2891095779 >> 172.21.15.12:0/2891095779 conn(0x7fd2f80fc9e0 msgr2=0x7fd2f80fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 -- 172.21.15.12:0/2891095779 shutdown_connections 2024-05-09T00:08:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.645+0000 7fd2fe371700 1 -- 172.21.15.12:0/2891095779 wait complete. 2024-05-09T00:08:00.658 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:08:00.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:00 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:08:00.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:00 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:00.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.812+0000 7f9c75ec9700 1 -- 172.21.15.12:0/2994068310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c70068c90 msgr2=0x7f9c70105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.812+0000 7f9c75ec9700 1 --2- 172.21.15.12:0/2994068310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c70068c90 0x7f9c70105b60 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f9c600099c0 tx=0x7f9c60009cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.812+0000 7f9c75ec9700 1 -- 172.21.15.12:0/2994068310 shutdown_connections 2024-05-09T00:08:00.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.812+0000 7f9c75ec9700 1 --2- 172.21.15.12:0/2994068310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c70106290 0x7f9c7010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.812+0000 7f9c75ec9700 1 --2- 172.21.15.12:0/2994068310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c700682e0 0x7f9c700686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.812+0000 7f9c75ec9700 1 --2- 172.21.15.12:0/2994068310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c70068c90 0x7f9c70105b60 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.812+0000 7f9c75ec9700 1 -- 172.21.15.12:0/2994068310 >> 172.21.15.12:0/2994068310 conn(0x7f9c70076f40 msgr2=0x7f9c70077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.812+0000 7f9c75ec9700 1 -- 172.21.15.12:0/2994068310 shutdown_connections 2024-05-09T00:08:00.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.812+0000 7f9c75ec9700 1 -- 172.21.15.12:0/2994068310 wait complete. 2024-05-09T00:08:00.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.813+0000 7f9c75ec9700 1 Processor -- start 2024-05-09T00:08:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.813+0000 7f9c75ec9700 1 -- start start 2024-05-09T00:08:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.813+0000 7f9c75ec9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c700682e0 0x7f9c70197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.813+0000 7f9c75ec9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c70106290 0x7f9c70197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.813+0000 7f9c75ec9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c7019bc30 0x7f9c7019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.813+0000 7f9c75ec9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7019c6b0 con 0x7f9c700682e0 2024-05-09T00:08:00.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.813+0000 7f9c75ec9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7019c820 con 0x7f9c70106290 2024-05-09T00:08:00.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.813+0000 7f9c75ec9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7019c9e0 con 0x7f9c7019bc30 2024-05-09T00:08:00.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.813+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c700682e0 0x7f9c70197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.813+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c700682e0 0x7f9c70197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37732/0 (socket says 172.21.15.12:37732) 2024-05-09T00:08:00.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/1015790089 learned_addr learned my addr 172.21.15.12:0/1015790089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:00.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6ffff700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c7019bc30 0x7f9c7019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6effd700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c70106290 0x7f9c70197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/1015790089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c7019bc30 msgr2=0x7f9c7019c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c7019bc30 0x7f9c7019c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/1015790089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c70106290 msgr2=0x7f9c70197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c70106290 0x7f9c70197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/1015790089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600095f0 con 0x7f9c700682e0 2024-05-09T00:08:00.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c700682e0 0x7f9c70197350 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800c9b0 tx=0x7f9c5800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6cff9700 1 -- 172.21.15.12:0/1015790089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58007b00 con 0x7f9c700682e0 2024-05-09T00:08:00.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6cff9700 1 -- 172.21.15.12:0/1015790089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c58007c60 con 0x7f9c700682e0 2024-05-09T00:08:00.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c75ec9700 1 -- 172.21.15.12:0/1015790089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c701a3f40 con 0x7f9c700682e0 2024-05-09T00:08:00.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c6cff9700 1 -- 172.21.15.12:0/1015790089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58018660 con 0x7f9c700682e0 2024-05-09T00:08:00.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.814+0000 7f9c75ec9700 1 -- 172.21.15.12:0/1015790089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c701a4360 con 0x7f9c700682e0 2024-05-09T00:08:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.817+0000 7f9c6cff9700 1 -- 172.21.15.12:0/1015790089 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9c5800f450 con 0x7f9c700682e0 2024-05-09T00:08:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.817+0000 7f9c6cff9700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9c5c0a1290 0x7f9c5c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.817+0000 7f9c6cff9700 1 -- 172.21.15.12:0/1015790089 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9c580ccc20 con 0x7f9c700682e0 2024-05-09T00:08:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.817+0000 7f9c6effd700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9c5c0a1290 0x7f9c5c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.818+0000 7f9c6effd700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9c5c0a1290 0x7f9c5c0a3750 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9c60000c00 tx=0x7f9c60005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.818+0000 7f9c82083b80 1 Processor -- start 2024-05-09T00:08:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.818+0000 7f9c82083b80 1 -- start start 2024-05-09T00:08:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.818+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b6071f0 0x55af8b6075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.818+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b607b10 0x55af8b601760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.818+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 0x55af8b602100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.818+0000 7f9c82083b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b554150 con 0x55af8b607b10 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.818+0000 7f9c82083b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b551b10 con 0x55af8b6071f0 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.818+0000 7f9c82083b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b54d400 con 0x55af8b601ca0 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b607b10 0x55af8b601760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b607b10 0x55af8b601760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37744/0 (socket says 172.21.15.12:37744) 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6effd700 1 -- 172.21.15.12:0/612715705 learned_addr learned my addr 172.21.15.12:0/612715705 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6ffff700 1 --2- 172.21.15.12:0/612715705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 0x55af8b602100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/612715705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b6071f0 0x55af8b6075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6effd700 1 -- 172.21.15.12:0/612715705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 msgr2=0x55af8b602100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6effd700 1 --2- 172.21.15.12:0/612715705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 0x55af8b602100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6effd700 1 -- 172.21.15.12:0/612715705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b6071f0 msgr2=0x55af8b6075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6effd700 1 --2- 172.21.15.12:0/612715705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b6071f0 0x55af8b6075d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6effd700 1 -- 172.21.15.12:0/612715705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af8b3ce3b0 con 0x55af8b607b10 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c6effd700 1 --2- 172.21.15.12:0/612715705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b607b10 0x55af8b601760 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f9c600099c0 tx=0x7f9c6001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2f3b80111cc2de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c65ffb700 1 -- 172.21.15.12:0/612715705 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6002b070 con 0x55af8b607b10 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c65ffb700 1 -- 172.21.15.12:0/612715705 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c6001e040 con 0x55af8b607b10 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.819+0000 7f9c65ffb700 1 -- 172.21.15.12:0/612715705 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60004c00 con 0x55af8b607b10 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.820+0000 7f9c82083b80 1 -- 172.21.15.12:0/612715705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b607b10 msgr2=0x55af8b601760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.820+0000 7f9c82083b80 1 --2- 172.21.15.12:0/612715705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b607b10 0x55af8b601760 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f9c600099c0 tx=0x7f9c6001f000 comp rx=0 tx=0).stop 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.820+0000 7f9c82083b80 1 -- 172.21.15.12:0/612715705 shutdown_connections 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.820+0000 7f9c82083b80 1 --2- 172.21.15.12:0/612715705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 0x55af8b602100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.820+0000 7f9c82083b80 1 --2- 172.21.15.12:0/612715705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b6071f0 0x55af8b6075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.820+0000 7f9c82083b80 1 --2- 172.21.15.12:0/612715705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b607b10 0x55af8b601760 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.820+0000 7f9c82083b80 1 -- 172.21.15.12:0/612715705 >> 172.21.15.12:0/612715705 conn(0x55af8b55b210 msgr2=0x55af8b5f7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.820+0000 7f9c82083b80 1 -- 172.21.15.12:0/612715705 shutdown_connections 2024-05-09T00:08:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.820+0000 7f9c82083b80 1 -- 172.21.15.12:0/612715705 wait complete. 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c82083b80 1 -- start start 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c82083b80 1 Processor -- start 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 0x55af8b60db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b60ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b607b10 0x55af8b60b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c82083b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b4e9560 con 0x55af8b6071f0 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c82083b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b4e9200 con 0x55af8b607b10 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c82083b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b3d06a0 con 0x55af8b601ca0 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c82083b80 10 client.0 _ll_register_callbacks cb 0x55af8b4f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c6effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b60ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c6effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b60ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37746/0 (socket says 172.21.15.12:37746) 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c6effd700 1 -- 172.21.15.12:0/4106966541 learned_addr learned my addr 172.21.15.12:0/4106966541 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.821+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 0x55af8b60db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c6ffff700 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b607b10 0x55af8b60b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/4106966541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b607b10 msgr2=0x55af8b60b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b607b10 0x55af8b60b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/4106966541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 msgr2=0x55af8b60ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b60ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/4106966541 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7010e650 con 0x55af8b601ca0 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c6effd700 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b60ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 0x55af8b60db50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9c580ca380 tx=0x7f9c5809a580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ad049aef600135 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c4effd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c4effd700 1 -- 172.21.15.12:0/4106966541 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5800f900 con 0x55af8b601ca0 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c4effd700 1 -- 172.21.15.12:0/4106966541 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9c58099fa0 con 0x55af8b601ca0 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c82083b80 1 -- 172.21.15.12:0/4106966541 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af8b4a6270 con 0x55af8b601ca0 2024-05-09T00:08:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c4effd700 1 -- 172.21.15.12:0/4106966541 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c5809a160 con 0x55af8b601ca0 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c82083b80 10 client.31117 fetch_fsmap learned FSMap version 76 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.822+0000 7f9c82083b80 1 -- 172.21.15.12:0/4106966541 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55af8b37ab60 con 0x55af8b601ca0 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c4effd700 1 -- 172.21.15.12:0/4106966541 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c58063020 con 0x55af8b601ca0 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c4effd700 1 -- 172.21.15.12:0/4106966541 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9c580a1e30 con 0x55af8b601ca0 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c82083b80 10 client.31117 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c82083b80 10 client.31117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c82083b80 20 client.31117 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c82083b80 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55af8b61b390 0x55af8b61d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c82083b80 4 client.31117 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c82083b80 1 -- 172.21.15.12:0/4106966541 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af8b5cc810 con 0x55af8b61b390 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c6effd700 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55af8b61b390 0x55af8b61d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c4effd700 10 client.31117 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.823+0000 7f9c6effd700 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55af8b61b390 0x55af8b61d780 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203bad255adaa00e server_cookie=7e6d0efe5a37f22 in_seq=0 out_seq=0 2024-05-09T00:08:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c4effd700 1 -- 172.21.15.12:0/4106966541 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af8b5cc810 con 0x55af8b61b390 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c4effd700 10 client.31117 handle_command_reply: tid=0 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 client.31117 shutdown 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 -- 172.21.15.12:0/4106966541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 msgr2=0x55af8b60db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 0x55af8b60db50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9c580ca380 tx=0x7f9c5809a580 comp rx=0 tx=0).stop 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 -- 172.21.15.12:0/4106966541 shutdown_connections 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55af8b61b390 0x55af8b61d780 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b607b10 0x55af8b60b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b601ca0 0x55af8b60db50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c4effd700 0 client.31117 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 --2- 172.21.15.12:0/4106966541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b60ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 -- 172.21.15.12:0/4106966541 >> 172.21.15.12:0/4106966541 conn(0x55af8b55b210 msgr2=0x55af8b5f75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 -- 172.21.15.12:0/4106966541 shutdown_connections 2024-05-09T00:08:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.824+0000 7f9c82083b80 1 -- 172.21.15.12:0/4106966541 wait complete. 2024-05-09T00:08:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.825+0000 7f9c82083b80 20 client.31117 trim_cache size 0 max 16384 2024-05-09T00:08:00.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c82083b80 1 Processor -- start 2024-05-09T00:08:00.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c82083b80 1 -- start start 2024-05-09T00:08:00.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b601ca0 0x55af8b60b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b5f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 0x55af8b6124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c82083b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b5f9a00 con 0x55af8b6071f0 2024-05-09T00:08:00.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c82083b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b5f2800 con 0x55af8b601ca0 2024-05-09T00:08:00.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c82083b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b622910 con 0x55af8b607b10 2024-05-09T00:08:00.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b5f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b5f7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37748/0 (socket says 172.21.15.12:37748) 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6effd700 1 -- 172.21.15.12:0/2410435094 learned_addr learned my addr 172.21.15.12:0/2410435094 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6ffff700 1 --2- 172.21.15.12:0/2410435094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 0x55af8b6124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/2410435094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b601ca0 0x55af8b60b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6effd700 1 -- 172.21.15.12:0/2410435094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 msgr2=0x55af8b6124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6effd700 1 --2- 172.21.15.12:0/2410435094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 0x55af8b6124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6effd700 1 -- 172.21.15.12:0/2410435094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b601ca0 msgr2=0x55af8b60b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6effd700 1 --2- 172.21.15.12:0/2410435094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b601ca0 0x55af8b60b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6effd700 1 -- 172.21.15.12:0/2410435094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af8b513560 con 0x55af8b6071f0 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.843+0000 7f9c6ffff700 1 --2- 172.21.15.12:0/2410435094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 0x55af8b6124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:08:00.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.844+0000 7f9c6effd700 1 --2- 172.21.15.12:0/2410435094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b5f7d20 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009350 tx=0x7f9c60004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512be74eee0fa3b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.844+0000 7f9c667fc700 1 -- 172.21.15.12:0/2410435094 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6002b040 con 0x55af8b6071f0 2024-05-09T00:08:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.844+0000 7f9c667fc700 1 -- 172.21.15.12:0/2410435094 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c6000bec0 con 0x55af8b6071f0 2024-05-09T00:08:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.844+0000 7f9c667fc700 1 -- 172.21.15.12:0/2410435094 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000f460 con 0x55af8b6071f0 2024-05-09T00:08:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.844+0000 7f9c82083b80 1 -- 172.21.15.12:0/2410435094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 msgr2=0x55af8b5f7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.844+0000 7f9c82083b80 1 --2- 172.21.15.12:0/2410435094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b5f7d20 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009350 tx=0x7f9c60004740 comp rx=0 tx=0).stop 2024-05-09T00:08:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.844+0000 7f9c82083b80 1 -- 172.21.15.12:0/2410435094 shutdown_connections 2024-05-09T00:08:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.844+0000 7f9c82083b80 1 --2- 172.21.15.12:0/2410435094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 0x55af8b6124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.845+0000 7f9c82083b80 1 --2- 172.21.15.12:0/2410435094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b601ca0 0x55af8b60b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.845+0000 7f9c82083b80 1 --2- 172.21.15.12:0/2410435094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b6071f0 0x55af8b5f7d20 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.845+0000 7f9c82083b80 1 -- 172.21.15.12:0/2410435094 >> 172.21.15.12:0/2410435094 conn(0x55af8b55b210 msgr2=0x55af8b602080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.845+0000 7f9c82083b80 1 -- 172.21.15.12:0/2410435094 shutdown_connections 2024-05-09T00:08:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.845+0000 7f9c82083b80 1 -- 172.21.15.12:0/2410435094 wait complete. 2024-05-09T00:08:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.845+0000 7f9c82083b80 1 -- start start 2024-05-09T00:08:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.845+0000 7f9c82083b80 1 Processor -- start 2024-05-09T00:08:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.845+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b601ca0 0x55af8b6104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.845+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b6071f0 0x55af8b610a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.845+0000 7f9c82083b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 0x55af8b670800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c82083b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b6111c0 con 0x55af8b601ca0 2024-05-09T00:08:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c82083b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b611330 con 0x55af8b6071f0 2024-05-09T00:08:00.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c82083b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af8b670d40 con 0x55af8b607b10 2024-05-09T00:08:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c82083b80 10 client.0 _ll_register_callbacks cb 0x55af8b4f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b601ca0 0x55af8b6104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b601ca0 0x55af8b6104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37752/0 (socket says 172.21.15.12:37752) 2024-05-09T00:08:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/1738740018 learned_addr learned my addr 172.21.15.12:0/1738740018 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6effd700 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b6071f0 0x55af8b610a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6ffff700 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 0x55af8b670800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/1738740018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 msgr2=0x55af8b670800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 0x55af8b670800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/1738740018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b6071f0 msgr2=0x55af8b610a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b6071f0 0x55af8b610a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6f7fe700 1 -- 172.21.15.12:0/1738740018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af8b3ce3b0 con 0x55af8b601ca0 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c6f7fe700 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b601ca0 0x55af8b6104d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f9c58096240 tx=0x7f9c580a2280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336547dd2f0d94e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c4effd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c4effd700 1 -- 172.21.15.12:0/1738740018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58099430 con 0x55af8b601ca0 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c4effd700 1 -- 172.21.15.12:0/1738740018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9c58064080 con 0x55af8b601ca0 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c82083b80 1 -- 172.21.15.12:0/1738740018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af8b545ec0 con 0x55af8b601ca0 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c4effd700 1 -- 172.21.15.12:0/1738740018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c58098df0 con 0x55af8b601ca0 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c82083b80 10 client.21405 fetch_fsmap learned FSMap version 76 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.846+0000 7f9c82083b80 1 -- 172.21.15.12:0/1738740018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55af8b3d2cd0 con 0x55af8b601ca0 2024-05-09T00:08:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.847+0000 7f9c4effd700 1 -- 172.21.15.12:0/1738740018 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c5807e020 con 0x55af8b601ca0 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.847+0000 7f9c4effd700 1 -- 172.21.15.12:0/1738740018 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9c580cab40 con 0x55af8b601ca0 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.847+0000 7f9c82083b80 10 client.21405 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.847+0000 7f9c82083b80 10 client.21405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.847+0000 7f9c82083b80 20 client.21405 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.847+0000 7f9c82083b80 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55af8b672130 0x55af8b674550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.847+0000 7f9c82083b80 4 client.21405 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.847+0000 7f9c82083b80 1 -- 172.21.15.12:0/1738740018 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55af8b39bcf0 con 0x55af8b672130 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.847+0000 7f9c6effd700 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55af8b672130 0x55af8b674550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c4effd700 10 client.21405 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c6effd700 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55af8b672130 0x55af8b674550 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45dde7853f9f49b5 server_cookie=d8eca5344cf50dad in_seq=0 out_seq=0 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c4effd700 1 -- 172.21.15.12:0/1738740018 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55af8b39bcf0 con 0x55af8b672130 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c4effd700 10 client.21405 handle_command_reply: tid=0 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 client.21405 shutdown 2024-05-09T00:08:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 -- 172.21.15.12:0/1738740018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b601ca0 msgr2=0x55af8b6104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b601ca0 0x55af8b6104d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f9c58096240 tx=0x7f9c580a2280 comp rx=0 tx=0).stop 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 -- 172.21.15.12:0/1738740018 shutdown_connections 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55af8b607b10 0x55af8b670800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af8b601ca0 0x55af8b6104d0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55af8b672130 0x55af8b674550 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 --2- 172.21.15.12:0/1738740018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55af8b6071f0 0x55af8b610a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c4effd700 0 client.21405 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 -- 172.21.15.12:0/1738740018 >> 172.21.15.12:0/1738740018 conn(0x55af8b55b210 msgr2=0x55af8b604f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 -- 172.21.15.12:0/1738740018 shutdown_connections 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 1 -- 172.21.15.12:0/1738740018 wait complete. 2024-05-09T00:08:00.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.848+0000 7f9c82083b80 20 client.21405 trim_cache size 0 max 16384 2024-05-09T00:08:00.853 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:08:00.853 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:08:00.853 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:08:00.853 INFO:teuthology.orchestra.run.smithi012.stdout: "726f87ca-b256-499f-9644-aeb268b06fea": { 2024-05-09T00:08:00.853 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:08:00.853 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "726f87ca-b256-499f-9644-aeb268b06fea", 2024-05-09T00:08:00.853 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:08:00.853 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:08:00.853 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:08:00.853 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:08:00.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 -- 172.21.15.12:0/1015790089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9c5c0a1290 msgr2=0x7f9c5c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9c5c0a1290 0x7f9c5c0a3750 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9c60000c00 tx=0x7f9c60005fd0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 -- 172.21.15.12:0/1015790089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c700682e0 msgr2=0x7f9c70197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:00.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c700682e0 0x7f9c70197350 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800c9b0 tx=0x7f9c5800ccc0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 -- 172.21.15.12:0/1015790089 shutdown_connections 2024-05-09T00:08:00.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9c5c0a1290 0x7f9c5c0a3750 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c7019bc30 0x7f9c7019c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c700682e0 0x7f9c70197350 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 --2- 172.21.15.12:0/1015790089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c70106290 0x7f9c70197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 -- 172.21.15.12:0/1015790089 >> 172.21.15.12:0/1015790089 conn(0x7f9c70076f40 msgr2=0x7f9c700fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 -- 172.21.15.12:0/1015790089 shutdown_connections 2024-05-09T00:08:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:00.849+0000 7f9c75ec9700 1 -- 172.21.15.12:0/1015790089 wait complete. 2024-05-09T00:08:00.858 INFO:tasks.cephfs.filesystem:scrub status for tag:726f87ca-b256-499f-9644-aeb268b06fea - {'path': '/', 'tag': '726f87ca-b256-499f-9644-aeb268b06fea', 'options': 'recursive,force'} 2024-05-09T00:08:01.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:00 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:08:01.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:00 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:08:01.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:00 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:01.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:00 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:08:01.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:00 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:08:01.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:00 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:01.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:00 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:08:01.279 INFO:tasks.workunit.client.0.smithi012.stdout: 10 293778 31.32 MB/sec execute 356 sec latency 865.039 ms 2024-05-09T00:08:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:01 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:08:02.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:01 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:08:02.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:01 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:01 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:08:02.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:01 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:01 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:01 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:01 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:08:02.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:01 smithi012 ceph-mon[42503]: pgmap v694: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 594 op/s 2024-05-09T00:08:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:01 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:08:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:01 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:08:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:01 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:01 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:08:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:01 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:01 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:01 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:01 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:08:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:01 smithi204 ceph-mon[47373]: pgmap v694: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 594 op/s 2024-05-09T00:08:02.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:01 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:08:02.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:01 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:08:02.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:01 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:01 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:08:02.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:01 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:01 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:01 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:02.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:01 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:08:02.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:01 smithi160 ceph-mon[46152]: pgmap v694: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 594 op/s 2024-05-09T00:08:02.279 INFO:tasks.workunit.client.0.smithi012.stdout: 10 294155 31.27 MB/sec execute 357 sec latency 144.629 ms 2024-05-09T00:08:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:02 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:02 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:08:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:02 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:08:03.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:02 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:03.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:02 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:08:03.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:02 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:08:03.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:02 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:03.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:02 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:08:03.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:02 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:08:03.280 INFO:tasks.workunit.client.0.smithi012.stdout: 10 294356 31.20 MB/sec execute 358 sec latency 278.856 ms 2024-05-09T00:08:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.521+0000 7f566e7b7700 1 -- 172.21.15.12:0/3989095896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5668105e30 msgr2=0x7f56681038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.521+0000 7f566e7b7700 1 --2- 172.21.15.12:0/3989095896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5668105e30 0x7f56681038b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f565801a9a0 tx=0x7f565801acb0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.521+0000 7f566e7b7700 1 -- 172.21.15.12:0/3989095896 shutdown_connections 2024-05-09T00:08:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.521+0000 7f566e7b7700 1 --2- 172.21.15.12:0/3989095896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56680682e0 0x7f5668068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.521+0000 7f566e7b7700 1 --2- 172.21.15.12:0/3989095896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5668105e30 0x7f56681038b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.521+0000 7f566e7b7700 1 --2- 172.21.15.12:0/3989095896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5668103df0 0x7f5668104270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.521+0000 7f566e7b7700 1 -- 172.21.15.12:0/3989095896 >> 172.21.15.12:0/3989095896 conn(0x7f5668076f10 msgr2=0x7f5668077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.521+0000 7f566e7b7700 1 -- 172.21.15.12:0/3989095896 shutdown_connections 2024-05-09T00:08:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.521+0000 7f566e7b7700 1 -- 172.21.15.12:0/3989095896 wait complete. 2024-05-09T00:08:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.522+0000 7f566e7b7700 1 Processor -- start 2024-05-09T00:08:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.522+0000 7f566e7b7700 1 -- start start 2024-05-09T00:08:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.522+0000 7f566e7b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56680682e0 0x7f5668110370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.522+0000 7f566e7b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5668103df0 0x7f56681108b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.522+0000 7f566e7b7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5668105e30 0x7f56681bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.522+0000 7f566e7b7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56681bc220 con 0x7f5668105e30 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.522+0000 7f566e7b7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56681bc390 con 0x7f5668103df0 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.522+0000 7f566e7b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56681bc550 con 0x7f56680682e0 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.523+0000 7f56677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5668103df0 0x7f56681108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.523+0000 7f56677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5668103df0 0x7f56681108b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33616/0 (socket says 172.21.15.12:33616) 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.523+0000 7f56677fe700 1 -- 172.21.15.12:0/3338845811 learned_addr learned my addr 172.21.15.12:0/3338845811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.523+0000 7f5667fff700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56680682e0 0x7f5668110370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.523+0000 7f566cd54700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5668105e30 0x7f56681bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.523+0000 7f56677fe700 1 -- 172.21.15.12:0/3338845811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56680682e0 msgr2=0x7f5668110370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.523+0000 7f56677fe700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56680682e0 0x7f5668110370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.523+0000 7f56677fe700 1 -- 172.21.15.12:0/3338845811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5668105e30 msgr2=0x7f56681bbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.523+0000 7f56677fe700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5668105e30 0x7f56681bbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.523+0000 7f56677fe700 1 -- 172.21.15.12:0/3338845811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565801a600 con 0x7f5668103df0 2024-05-09T00:08:03.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.524+0000 7f56677fe700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5668103df0 0x7f56681108b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f565c00b530 tx=0x7f565c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.527+0000 7f56657fa700 1 -- 172.21.15.12:0/3338845811 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c011830 con 0x7f5668103df0 2024-05-09T00:08:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.527+0000 7f56657fa700 1 -- 172.21.15.12:0/3338845811 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f565c011e50 con 0x7f5668103df0 2024-05-09T00:08:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.527+0000 7f56657fa700 1 -- 172.21.15.12:0/3338845811 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c00f2e0 con 0x7f5668103df0 2024-05-09T00:08:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.527+0000 7f566e7b7700 1 -- 172.21.15.12:0/3338845811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56681bc830 con 0x7f5668103df0 2024-05-09T00:08:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.527+0000 7f566e7b7700 1 -- 172.21.15.12:0/3338845811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56681c3f50 con 0x7f5668103df0 2024-05-09T00:08:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.531+0000 7f567a971b80 1 Processor -- start 2024-05-09T00:08:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f567a971b80 1 -- start start 2024-05-09T00:08:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f567a971b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 0x5576c2794d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f567a971b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c278ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f567a971b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c278f480 0x5576c278f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f567a971b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c26e60f0 con 0x5576c2794950 2024-05-09T00:08:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f567a971b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c26e3ab0 con 0x5576c2795270 2024-05-09T00:08:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f567a971b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c26df3a0 con 0x5576c278f480 2024-05-09T00:08:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f566cd54700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 0x5576c2794d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f566cd54700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 0x5576c2794d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37788/0 (socket says 172.21.15.12:37788) 2024-05-09T00:08:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f566cd54700 1 -- 172.21.15.12:0/32529275 learned_addr learned my addr 172.21.15.12:0/32529275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f56677fe700 1 --2- 172.21.15.12:0/32529275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c278f480 0x5576c278f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.532+0000 7f56657fa700 1 -- 172.21.15.12:0/3338845811 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f565c00f440 con 0x7f5668103df0 2024-05-09T00:08:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.533+0000 7f566cd54700 1 -- 172.21.15.12:0/32529275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c278f480 msgr2=0x5576c278f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.533+0000 7f566cd54700 1 --2- 172.21.15.12:0/32529275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c278f480 0x5576c278f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.533+0000 7f566cd54700 1 -- 172.21.15.12:0/32529275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 msgr2=0x5576c278ef40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:08:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.533+0000 7f566cd54700 1 --2- 172.21.15.12:0/32529275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c278ef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.533+0000 7f566cd54700 1 -- 172.21.15.12:0/32529275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576c2560350 con 0x5576c2794950 2024-05-09T00:08:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.533+0000 7f566cd54700 1 --2- 172.21.15.12:0/32529275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 0x5576c2794d30 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f566000a430 tx=0x7f5660009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5699c627b90633f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.533+0000 7f56657fa700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f56500a1460 0x7f56500a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.533+0000 7f56657fa700 1 -- 172.21.15.12:0/3338845811 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f565c0cc6a0 con 0x7f5668103df0 2024-05-09T00:08:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.534+0000 7f5667fff700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f56500a1460 0x7f56500a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.534+0000 7f564e7fc700 1 -- 172.21.15.12:0/32529275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660010040 con 0x5576c2794950 2024-05-09T00:08:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.534+0000 7f564e7fc700 1 -- 172.21.15.12:0/32529275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5660013a50 con 0x5576c2794950 2024-05-09T00:08:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.534+0000 7f564e7fc700 1 -- 172.21.15.12:0/32529275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660014af0 con 0x5576c2794950 2024-05-09T00:08:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.535+0000 7f567a971b80 1 -- 172.21.15.12:0/32529275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 msgr2=0x5576c2794d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.535+0000 7f5667fff700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f56500a1460 0x7f56500a3920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5658003800 tx=0x7f5658005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.535+0000 7f567a971b80 1 --2- 172.21.15.12:0/32529275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 0x5576c2794d30 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f566000a430 tx=0x7f5660009650 comp rx=0 tx=0).stop 2024-05-09T00:08:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.535+0000 7f567a971b80 1 -- 172.21.15.12:0/32529275 shutdown_connections 2024-05-09T00:08:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.535+0000 7f567a971b80 1 --2- 172.21.15.12:0/32529275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c278f480 0x5576c278f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.535+0000 7f567a971b80 1 --2- 172.21.15.12:0/32529275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 0x5576c2794d30 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.535+0000 7f567a971b80 1 --2- 172.21.15.12:0/32529275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c278ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.535+0000 7f567a971b80 1 -- 172.21.15.12:0/32529275 >> 172.21.15.12:0/32529275 conn(0x5576c26ed1b0 msgr2=0x5576c2789840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.538+0000 7f567a971b80 1 -- 172.21.15.12:0/32529275 shutdown_connections 2024-05-09T00:08:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.538+0000 7f567a971b80 1 -- 172.21.15.12:0/32529275 wait complete. 2024-05-09T00:08:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.539+0000 7f567a971b80 1 -- start start 2024-05-09T00:08:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.539+0000 7f567a971b80 1 Processor -- start 2024-05-09T00:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f567a971b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c278f480 0x5576c27985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f567a971b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2794950 0x5576c2798b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f567a971b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c2795270 0x5576c2799070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f567a971b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c267b500 con 0x5576c278f480 2024-05-09T00:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f567a971b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c267b1a0 con 0x5576c2794950 2024-05-09T00:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f567a971b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c2562640 con 0x5576c2795270 2024-05-09T00:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f567a971b80 10 client.0 _ll_register_callbacks cb 0x5576c2684ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f56677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2794950 0x5576c2798b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f56677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2794950 0x5576c2798b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33628/0 (socket says 172.21.15.12:33628) 2024-05-09T00:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f56677fe700 1 -- 172.21.15.12:0/3541411787 learned_addr learned my addr 172.21.15.12:0/3541411787 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f56677fe700 1 -- 172.21.15.12:0/3541411787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c2795270 msgr2=0x5576c2799070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f56677fe700 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c2795270 0x5576c2799070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f56677fe700 1 -- 172.21.15.12:0/3541411787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c278f480 msgr2=0x5576c27985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f56677fe700 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c278f480 0x5576c27985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.540+0000 7f56677fe700 1 -- 172.21.15.12:0/3541411787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5660009c60 con 0x5576c2794950 2024-05-09T00:08:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.541+0000 7f56677fe700 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2794950 0x5576c2798b30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f565c0967c0 tx=0x7f565c099990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b678774865547c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.541+0000 7f56477fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:08:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.541+0000 7f56477fe700 1 -- 172.21.15.12:0/3541411787 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c09db80 con 0x5576c2794950 2024-05-09T00:08:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.541+0000 7f56477fe700 1 -- 172.21.15.12:0/3541411787 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f565c0a21d0 con 0x5576c2794950 2024-05-09T00:08:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.541+0000 7f567a971b80 1 -- 172.21.15.12:0/3541411787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576c2638210 con 0x5576c2794950 2024-05-09T00:08:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f56477fe700 1 -- 172.21.15.12:0/3541411787 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f565c0b2260 con 0x5576c2794950 2024-05-09T00:08:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f567a971b80 10 client.30878 fetch_fsmap learned FSMap version 76 2024-05-09T00:08:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f567a971b80 1 -- 172.21.15.12:0/3541411787 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5576c250cb00 con 0x5576c2794950 2024-05-09T00:08:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f56477fe700 1 -- 172.21.15.12:0/3541411787 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5576c2638210 con 0x5576c2794950 2024-05-09T00:08:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f56477fe700 1 -- 172.21.15.12:0/3541411787 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f565c0b91b0 con 0x5576c2794950 2024-05-09T00:08:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f567a971b80 10 client.30878 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:08:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f567a971b80 10 client.30878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f567a971b80 20 client.30878 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f567a971b80 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576c27b9cb0 0x5576c27bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f567a971b80 4 client.30878 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.544+0000 7f567a971b80 1 -- 172.21.15.12:0/3541411787 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576c275e7b0 con 0x5576c27b9cb0 2024-05-09T00:08:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.545+0000 7f5667fff700 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576c27b9cb0 0x5576c27bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.550+0000 7f5667fff700 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576c27b9cb0 0x5576c27bc0d0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a40e69f4d0787 server_cookie=ef9df2e496f6eb79 in_seq=0 out_seq=0 2024-05-09T00:08:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.550+0000 7f56477fe700 10 client.30878 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.551+0000 7f56477fe700 1 -- 172.21.15.12:0/3541411787 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5576c275e7b0 con 0x5576c27b9cb0 2024-05-09T00:08:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.551+0000 7f56477fe700 10 client.30878 handle_command_reply: tid=0 2024-05-09T00:08:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.554+0000 7f567a971b80 1 client.30878 shutdown 2024-05-09T00:08:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.554+0000 7f567a971b80 1 -- 172.21.15.12:0/3541411787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2794950 msgr2=0x5576c2798b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.554+0000 7f567a971b80 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2794950 0x5576c2798b30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f565c0967c0 tx=0x7f565c099990 comp rx=0 tx=0).stop 2024-05-09T00:08:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.556+0000 7f567a971b80 1 -- 172.21.15.12:0/3541411787 shutdown_connections 2024-05-09T00:08:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.556+0000 7f567a971b80 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c2795270 0x5576c2799070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.556+0000 7f567a971b80 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c278f480 0x5576c27985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.556+0000 7f567a971b80 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576c27b9cb0 0x5576c27bc0d0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.556+0000 7f567a971b80 1 --2- 172.21.15.12:0/3541411787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2794950 0x5576c2798b30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.556+0000 7f567a971b80 1 -- 172.21.15.12:0/3541411787 >> 172.21.15.12:0/3541411787 conn(0x5576c26ed1b0 msgr2=0x5576c278df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.556+0000 7f56477fe700 0 client.30878 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.558+0000 7f567a971b80 1 -- 172.21.15.12:0/3541411787 shutdown_connections 2024-05-09T00:08:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.559+0000 7f567a971b80 1 -- 172.21.15.12:0/3541411787 wait complete. 2024-05-09T00:08:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.560+0000 7f567a971b80 20 client.30878 trim_cache size 0 max 16384 2024-05-09T00:08:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.589+0000 7f567a971b80 1 Processor -- start 2024-05-09T00:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.591+0000 7f567a971b80 1 -- start start 2024-05-09T00:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.591+0000 7f567a971b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c278f480 0x5576c2789910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.591+0000 7f567a971b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 0x5576c2792b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.591+0000 7f567a971b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c2793630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.591+0000 7f567a971b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c2794d30 con 0x5576c2794950 2024-05-09T00:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.591+0000 7f567a971b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c27869b0 con 0x5576c2795270 2024-05-09T00:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.591+0000 7f567a971b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c2784330 con 0x5576c278f480 2024-05-09T00:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.591+0000 7f566cd54700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c2793630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.591+0000 7f566cd54700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c2793630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:33638/0 (socket says 172.21.15.12:33638) 2024-05-09T00:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.591+0000 7f566cd54700 1 -- 172.21.15.12:0/2736236741 learned_addr learned my addr 172.21.15.12:0/2736236741 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.592+0000 7f566cd54700 1 -- 172.21.15.12:0/2736236741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c278f480 msgr2=0x5576c2789910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.592+0000 7f566cd54700 1 --2- 172.21.15.12:0/2736236741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c278f480 0x5576c2789910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.592+0000 7f566cd54700 1 -- 172.21.15.12:0/2736236741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 msgr2=0x5576c2792b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.592+0000 7f566cd54700 1 --2- 172.21.15.12:0/2736236741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 0x5576c2792b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.592+0000 7f566cd54700 1 -- 172.21.15.12:0/2736236741 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576c26a5500 con 0x5576c2795270 2024-05-09T00:08:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.592+0000 7f566cd54700 1 --2- 172.21.15.12:0/2736236741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c2793630 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5660009870 tx=0x7f5660004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8720da5fdf405441 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.593+0000 7f564effd700 1 -- 172.21.15.12:0/2736236741 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660010070 con 0x5576c2795270 2024-05-09T00:08:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.593+0000 7f564effd700 1 -- 172.21.15.12:0/2736236741 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5660013dc0 con 0x5576c2795270 2024-05-09T00:08:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.593+0000 7f564effd700 1 -- 172.21.15.12:0/2736236741 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660014ea0 con 0x5576c2795270 2024-05-09T00:08:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.594+0000 7f567a971b80 1 -- 172.21.15.12:0/2736236741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 msgr2=0x5576c2793630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.594+0000 7f567a971b80 1 --2- 172.21.15.12:0/2736236741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c2793630 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5660009870 tx=0x7f5660004be0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.594+0000 7f567a971b80 1 -- 172.21.15.12:0/2736236741 shutdown_connections 2024-05-09T00:08:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.594+0000 7f567a971b80 1 --2- 172.21.15.12:0/2736236741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c2793630 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.594+0000 7f567a971b80 1 --2- 172.21.15.12:0/2736236741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c278f480 0x5576c2789910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.594+0000 7f567a971b80 1 --2- 172.21.15.12:0/2736236741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c2794950 0x5576c2792b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.594+0000 7f567a971b80 1 -- 172.21.15.12:0/2736236741 >> 172.21.15.12:0/2736236741 conn(0x5576c26ed1b0 msgr2=0x5576c27923f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.601+0000 7f567a971b80 1 -- 172.21.15.12:0/2736236741 shutdown_connections 2024-05-09T00:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.601+0000 7f567a971b80 1 -- 172.21.15.12:0/2736236741 wait complete. 2024-05-09T00:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.601+0000 7f567a971b80 1 -- start start 2024-05-09T00:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.602+0000 7f567a971b80 1 Processor -- start 2024-05-09T00:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.605+0000 7f567a971b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c278f480 0x5576c279dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.605+0000 7f567a971b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c2794950 0x5576c279e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.605+0000 7f567a971b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c279e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.605+0000 7f567a971b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c280f380 con 0x5576c278f480 2024-05-09T00:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.605+0000 7f567a971b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c280f4f0 con 0x5576c2795270 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.605+0000 7f567a971b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c280f6b0 con 0x5576c2794950 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.605+0000 7f567a971b80 10 client.0 _ll_register_callbacks cb 0x5576c2684ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f56677fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c2794950 0x5576c279e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f56677fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c2794950 0x5576c279e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57862/0 (socket says 172.21.15.12:57862) 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f56677fe700 1 -- 172.21.15.12:0/3665149382 learned_addr learned my addr 172.21.15.12:0/3665149382 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f566cd54700 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c279e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f566cd54700 1 -- 172.21.15.12:0/3665149382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c2794950 msgr2=0x5576c279e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f566cd54700 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c2794950 0x5576c279e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f566cd54700 1 -- 172.21.15.12:0/3665149382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c278f480 msgr2=0x5576c279dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f566cd54700 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c278f480 0x5576c279dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f566cd54700 1 -- 172.21.15.12:0/3665149382 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576c2560350 con 0x5576c2795270 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f566cd54700 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c279e6b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5660007000 tx=0x7f5660013bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72910744d4a626b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f56477fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:08:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.606+0000 7f56477fe700 1 -- 172.21.15.12:0/3665149382 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660010070 con 0x5576c2795270 2024-05-09T00:08:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.607+0000 7f567a971b80 1 -- 172.21.15.12:0/3665149382 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5660013a50 con 0x5576c2795270 2024-05-09T00:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f56477fe700 1 -- 172.21.15.12:0/3665149382 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5660006e40 con 0x5576c2795270 2024-05-09T00:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f56477fe700 1 -- 172.21.15.12:0/3665149382 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56600138c0 con 0x5576c2795270 2024-05-09T00:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f567a971b80 10 client.30884 fetch_fsmap learned FSMap version 76 2024-05-09T00:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f567a971b80 1 -- 172.21.15.12:0/3665149382 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5576c2564c70 con 0x5576c2795270 2024-05-09T00:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f56477fe700 1 -- 172.21.15.12:0/3665149382 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5660013a50 con 0x5576c2795270 2024-05-09T00:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f56477fe700 1 -- 172.21.15.12:0/3665149382 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f566004e030 con 0x5576c2795270 2024-05-09T00:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f567a971b80 10 client.30884 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f567a971b80 10 client.30884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f567a971b80 20 client.30884 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f567a971b80 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576c27a4cc0 0x5576c27a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f567a971b80 4 client.30884 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.609+0000 7f567a971b80 1 -- 172.21.15.12:0/3665149382 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5576c26d7e60 con 0x5576c27a4cc0 2024-05-09T00:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.610+0000 7f5667fff700 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576c27a4cc0 0x5576c27a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.610+0000 7f56477fe700 10 client.30884 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:03.610+0000 7f5667fff700 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576c27a4cc0 0x5576c27a70b0 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7d6f03aa1126bd2 server_cookie=d66c6de7f798e97f in_seq=0 out_seq=0 2024-05-09T00:08:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.040+0000 7f56477fe700 1 -- 172.21.15.12:0/3665149382 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5576c26d7e60 con 0x5576c27a4cc0 2024-05-09T00:08:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.040+0000 7f56477fe700 10 client.30884 handle_command_reply: tid=0 2024-05-09T00:08:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.040+0000 7f567a971b80 1 client.30884 shutdown 2024-05-09T00:08:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.043+0000 7f567a971b80 1 -- 172.21.15.12:0/3665149382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 msgr2=0x5576c279e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.043+0000 7f567a971b80 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c279e6b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5660007000 tx=0x7f5660013bc0 comp rx=0 tx=0).stop 2024-05-09T00:08:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.047+0000 7f567a971b80 1 -- 172.21.15.12:0/3665149382 shutdown_connections 2024-05-09T00:08:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.047+0000 7f567a971b80 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5576c27a4cc0 0x5576c27a70b0 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.047+0000 7f567a971b80 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5576c2795270 0x5576c279e6b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.047+0000 7f567a971b80 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5576c278f480 0x5576c279dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.047+0000 7f567a971b80 1 --2- 172.21.15.12:0/3665149382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576c2794950 0x5576c279e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.047+0000 7f567a971b80 1 -- 172.21.15.12:0/3665149382 >> 172.21.15.12:0/3665149382 conn(0x5576c26ed1b0 msgr2=0x5576c2793630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.047+0000 7f56477fe700 0 client.30884 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.056+0000 7f567a971b80 1 -- 172.21.15.12:0/3665149382 shutdown_connections 2024-05-09T00:08:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.058+0000 7f567a971b80 1 -- 172.21.15.12:0/3665149382 wait complete. 2024-05-09T00:08:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:08:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:08:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:08:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:08:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.061+0000 7f567a971b80 20 client.30884 trim_cache size 0 max 16384 2024-05-09T00:08:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.067+0000 7f566e7b7700 1 -- 172.21.15.12:0/3338845811 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f56500a1460 msgr2=0x7f56500a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.067+0000 7f566e7b7700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f56500a1460 0x7f56500a3920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5658003800 tx=0x7f5658005fd0 comp rx=0 tx=0).stop 2024-05-09T00:08:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.067+0000 7f566e7b7700 1 -- 172.21.15.12:0/3338845811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5668103df0 msgr2=0x7f56681108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.067+0000 7f566e7b7700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5668103df0 0x7f56681108b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f565c00b530 tx=0x7f565c00b840 comp rx=0 tx=0).stop 2024-05-09T00:08:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.067+0000 7f566e7b7700 1 -- 172.21.15.12:0/3338845811 shutdown_connections 2024-05-09T00:08:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.067+0000 7f566e7b7700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f56500a1460 0x7f56500a3920 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.067+0000 7f566e7b7700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5668105e30 0x7f56681bbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.067+0000 7f566e7b7700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56680682e0 0x7f5668110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.067+0000 7f566e7b7700 1 --2- 172.21.15.12:0/3338845811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5668103df0 0x7f56681108b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.067+0000 7f566e7b7700 1 -- 172.21.15.12:0/3338845811 >> 172.21.15.12:0/3338845811 conn(0x7f5668076f10 msgr2=0x7f56680fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.072+0000 7f566e7b7700 1 -- 172.21.15.12:0/3338845811 shutdown_connections 2024-05-09T00:08:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:04.072+0000 7f566e7b7700 1 -- 172.21.15.12:0/3338845811 wait complete. 2024-05-09T00:08:04.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:03 smithi012 ceph-mon[42503]: pgmap v695: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 812 op/s 2024-05-09T00:08:04.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:03 smithi204 ceph-mon[47373]: pgmap v695: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 812 op/s 2024-05-09T00:08:04.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:03 smithi160 ceph-mon[46152]: pgmap v695: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 812 op/s 2024-05-09T00:08:04.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 294388 31.12 MB/sec execute 359 sec latency 626.783 ms 2024-05-09T00:08:05.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:04 smithi204 ceph-mon[47373]: osdmap e128: 12 total, 12 up, 12 in 2024-05-09T00:08:05.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:04 smithi160 ceph-mon[46152]: osdmap e128: 12 total, 12 up, 12 in 2024-05-09T00:08:05.280 INFO:tasks.workunit.client.0.smithi012.stdout: 10 294430 31.04 MB/sec execute 360 sec latency 1105.928 ms 2024-05-09T00:08:05.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:04 smithi012 ceph-mon[42503]: osdmap e128: 12 total, 12 up, 12 in 2024-05-09T00:08:06.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:05 smithi204 ceph-mon[47373]: pgmap v697: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 704 op/s 2024-05-09T00:08:06.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:05 smithi160 ceph-mon[46152]: pgmap v697: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 704 op/s 2024-05-09T00:08:06.280 INFO:tasks.workunit.client.0.smithi012.stdout: 10 294519 30.97 MB/sec execute 361 sec latency 1182.182 ms 2024-05-09T00:08:06.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:05 smithi012 ceph-mon[42503]: pgmap v697: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 704 op/s 2024-05-09T00:08:07.281 INFO:tasks.workunit.client.0.smithi012.stdout: 10 294672 30.91 MB/sec execute 362 sec latency 1165.065 ms 2024-05-09T00:08:08.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:07 smithi012 ceph-mon[42503]: pgmap v698: 97 pgs: 16 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 54 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 818 op/s 2024-05-09T00:08:08.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:07 smithi204 ceph-mon[47373]: pgmap v698: 97 pgs: 16 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 54 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 818 op/s 2024-05-09T00:08:08.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:07 smithi160 ceph-mon[46152]: pgmap v698: 97 pgs: 16 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 54 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 818 op/s 2024-05-09T00:08:08.281 INFO:tasks.workunit.client.0.smithi012.stdout: 10 294875 30.85 MB/sec execute 363 sec latency 592.549 ms 2024-05-09T00:08:09.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:08 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:08:09.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:08 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:08:09.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:08 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:08:09.281 INFO:tasks.workunit.client.0.smithi012.stdout: 10 295303 30.84 MB/sec execute 364 sec latency 269.157 ms 2024-05-09T00:08:10.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:09 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:08:10.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:09 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:08:10.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:09 smithi012 ceph-mon[42503]: pgmap v699: 97 pgs: 19 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 45 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.8 MiB/s wr, 787 op/s 2024-05-09T00:08:10.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:09 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:08:10.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:09 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:08:10.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:09 smithi204 ceph-mon[47373]: pgmap v699: 97 pgs: 19 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 45 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.8 MiB/s wr, 787 op/s 2024-05-09T00:08:10.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:09 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:08:10.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:09 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:08:10.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:09 smithi160 ceph-mon[46152]: pgmap v699: 97 pgs: 19 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 45 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.8 MiB/s wr, 787 op/s 2024-05-09T00:08:10.281 INFO:tasks.workunit.client.0.smithi012.stdout: 10 295616 30.81 MB/sec execute 365 sec latency 308.808 ms 2024-05-09T00:08:11.281 INFO:tasks.workunit.client.0.smithi012.stdout: 10 296234 30.83 MB/sec execute 366 sec latency 107.292 ms 2024-05-09T00:08:12.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:11 smithi012 ceph-mon[42503]: pgmap v700: 97 pgs: 15 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 60 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 777 op/s 2024-05-09T00:08:12.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:11 smithi204 ceph-mon[47373]: pgmap v700: 97 pgs: 15 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 60 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 777 op/s 2024-05-09T00:08:12.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:11 smithi160 ceph-mon[46152]: pgmap v700: 97 pgs: 15 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 60 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 777 op/s 2024-05-09T00:08:12.281 INFO:tasks.workunit.client.0.smithi012.stdout: 10 296818 30.84 MB/sec execute 367 sec latency 84.735 ms 2024-05-09T00:08:13.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 297326 30.82 MB/sec execute 368 sec latency 96.763 ms 2024-05-09T00:08:14.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:13 smithi012 ceph-mon[42503]: pgmap v701: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 669 op/s 2024-05-09T00:08:14.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:13 smithi204 ceph-mon[47373]: pgmap v701: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 669 op/s 2024-05-09T00:08:14.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:13 smithi160 ceph-mon[46152]: pgmap v701: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 669 op/s 2024-05-09T00:08:14.281 INFO:tasks.workunit.client.0.smithi012.stdout: 10 297819 30.79 MB/sec execute 369 sec latency 180.681 ms 2024-05-09T00:08:15.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 298188 30.77 MB/sec execute 370 sec latency 435.696 ms 2024-05-09T00:08:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:15 smithi012 ceph-mon[42503]: pgmap v702: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.8 MiB/s wr, 578 op/s 2024-05-09T00:08:16.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:15 smithi204 ceph-mon[47373]: pgmap v702: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.8 MiB/s wr, 578 op/s 2024-05-09T00:08:16.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:15 smithi160 ceph-mon[46152]: pgmap v702: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.8 MiB/s wr, 578 op/s 2024-05-09T00:08:16.281 INFO:tasks.workunit.client.0.smithi012.stdout: 10 298702 30.78 MB/sec execute 371 sec latency 153.356 ms 2024-05-09T00:08:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.238+0000 7f590ac3f700 1 -- 172.21.15.12:0/4264071542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5904106260 msgr2=0x7f590410dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.238+0000 7f590ac3f700 1 --2- 172.21.15.12:0/4264071542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5904106260 0x7f590410dde0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f58f80099c0 tx=0x7f58f8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.238+0000 7f590ac3f700 1 -- 172.21.15.12:0/4264071542 shutdown_connections 2024-05-09T00:08:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.238+0000 7f590ac3f700 1 --2- 172.21.15.12:0/4264071542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5904106260 0x7f590410dde0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.238+0000 7f590ac3f700 1 --2- 172.21.15.12:0/4264071542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5904069360 0x7f5904069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.238+0000 7f590ac3f700 1 --2- 172.21.15.12:0/4264071542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904069d10 0x7f5904105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.238+0000 7f590ac3f700 1 -- 172.21.15.12:0/4264071542 >> 172.21.15.12:0/4264071542 conn(0x7f5904076bc0 msgr2=0x7f5904076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.238+0000 7f590ac3f700 1 -- 172.21.15.12:0/4264071542 shutdown_connections 2024-05-09T00:08:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.238+0000 7f590ac3f700 1 -- 172.21.15.12:0/4264071542 wait complete. 2024-05-09T00:08:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f590ac3f700 1 Processor -- start 2024-05-09T00:08:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f590ac3f700 1 -- start start 2024-05-09T00:08:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f590ac3f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904069360 0x7f59041972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f590ac3f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5904106260 0x7f5904197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f590ac3f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590419bbd0 0x7f590419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f590ac3f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590419c650 con 0x7f5904106260 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f590ac3f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590419c7c0 con 0x7f590419bbd0 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f590ac3f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590419c980 con 0x7f5904069360 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f59089db700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904069360 0x7f59041972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f5903fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5904106260 0x7f5904197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f59089db700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904069360 0x7f59041972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57600/0 (socket says 172.21.15.12:57600) 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f5903fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5904106260 0x7f5904197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59632/0 (socket says 172.21.15.12:59632) 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f59089db700 1 -- 172.21.15.12:0/3521801602 learned_addr learned my addr 172.21.15.12:0/3521801602 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.239+0000 7f59091dc700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590419bbd0 0x7f590419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f59089db700 1 -- 172.21.15.12:0/3521801602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590419bbd0 msgr2=0x7f590419c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f59089db700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590419bbd0 0x7f590419c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f59089db700 1 -- 172.21.15.12:0/3521801602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5904106260 msgr2=0x7f5904197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f59089db700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5904106260 0x7f5904197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f59089db700 1 -- 172.21.15.12:0/3521801602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f80095f0 con 0x7f5904069360 2024-05-09T00:08:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f5903fff700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5904106260 0x7f5904197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f59089db700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904069360 0x7f59041972f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00ef10 tx=0x7f58fc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f5901ffb700 1 -- 172.21.15.12:0/3521801602 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00c9c0 con 0x7f5904069360 2024-05-09T00:08:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f5901ffb700 1 -- 172.21.15.12:0/3521801602 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58fc00f040 con 0x7f5904069360 2024-05-09T00:08:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f590ac3f700 1 -- 172.21.15.12:0/3521801602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59041a3e80 con 0x7f5904069360 2024-05-09T00:08:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.240+0000 7f5901ffb700 1 -- 172.21.15.12:0/3521801602 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc014480 con 0x7f5904069360 2024-05-09T00:08:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.241+0000 7f590ac3f700 1 -- 172.21.15.12:0/3521801602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59041a4300 con 0x7f5904069360 2024-05-09T00:08:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.243+0000 7f5901ffb700 1 -- 172.21.15.12:0/3521801602 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f58fc0145e0 con 0x7f5904069360 2024-05-09T00:08:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.244+0000 7f5901ffb700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f58ec0a1290 0x7f58ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.244+0000 7f5901ffb700 1 -- 172.21.15.12:0/3521801602 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f58fc0ccfd0 con 0x7f5904069360 2024-05-09T00:08:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.245+0000 7f5903fff700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f58ec0a1290 0x7f58ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.245+0000 7f5916df9b80 1 Processor -- start 2024-05-09T00:08:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.245+0000 7f5903fff700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f58ec0a1290 0x7f58ec0a3750 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5904198840 tx=0x7f58f4009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.245+0000 7f5916df9b80 1 -- start start 2024-05-09T00:08:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94aca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abcf40 0x559e94abd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abd900 0x559e94ad1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f5916df9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e94a140f0 con 0x559e94aca200 2024-05-09T00:08:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f5916df9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e94a11ab0 con 0x559e94abcf40 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f5916df9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e94a0d3a0 con 0x559e94abd900 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f59089db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94aca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f59089db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94aca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59638/0 (socket says 172.21.15.12:59638) 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f59089db700 1 -- 172.21.15.12:0/3904037999 learned_addr learned my addr 172.21.15.12:0/3904037999 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f59091dc700 1 --2- 172.21.15.12:0/3904037999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abd900 0x559e94ad1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f5903fff700 1 --2- 172.21.15.12:0/3904037999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abcf40 0x559e94abd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f59089db700 1 -- 172.21.15.12:0/3904037999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abd900 msgr2=0x559e94ad1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f59089db700 1 --2- 172.21.15.12:0/3904037999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abd900 0x559e94ad1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f59089db700 1 -- 172.21.15.12:0/3904037999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abcf40 msgr2=0x559e94abd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f59089db700 1 --2- 172.21.15.12:0/3904037999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abcf40 0x559e94abd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.246+0000 7f59089db700 1 -- 172.21.15.12:0/3904037999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e9488e350 con 0x559e94aca200 2024-05-09T00:08:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.247+0000 7f59089db700 1 --2- 172.21.15.12:0/3904037999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94aca5e0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f58fc096b60 tx=0x7f58fc099d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d7e7d4d40e2074 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.247+0000 7f58eaffd700 1 -- 172.21.15.12:0/3904037999 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc09a0f0 con 0x559e94aca200 2024-05-09T00:08:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.247+0000 7f58eaffd700 1 -- 172.21.15.12:0/3904037999 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58fc09df20 con 0x559e94aca200 2024-05-09T00:08:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.247+0000 7f58eaffd700 1 -- 172.21.15.12:0/3904037999 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0995e0 con 0x559e94aca200 2024-05-09T00:08:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.248+0000 7f5916df9b80 1 -- 172.21.15.12:0/3904037999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 msgr2=0x559e94aca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.248+0000 7f5916df9b80 1 --2- 172.21.15.12:0/3904037999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94aca5e0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f58fc096b60 tx=0x7f58fc099d00 comp rx=0 tx=0).stop 2024-05-09T00:08:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.248+0000 7f5916df9b80 1 -- 172.21.15.12:0/3904037999 shutdown_connections 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.248+0000 7f5916df9b80 1 --2- 172.21.15.12:0/3904037999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abd900 0x559e94ad1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.248+0000 7f5916df9b80 1 --2- 172.21.15.12:0/3904037999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94aca5e0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.248+0000 7f5916df9b80 1 --2- 172.21.15.12:0/3904037999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abcf40 0x559e94abd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.248+0000 7f5916df9b80 1 -- 172.21.15.12:0/3904037999 >> 172.21.15.12:0/3904037999 conn(0x559e94a1b1b0 msgr2=0x559e94ab7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.248+0000 7f5916df9b80 1 -- 172.21.15.12:0/3904037999 shutdown_connections 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.248+0000 7f5916df9b80 1 -- 172.21.15.12:0/3904037999 wait complete. 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.248+0000 7f5916df9b80 1 -- start start 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f5916df9b80 1 Processor -- start 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abcf40 0x559e94ad1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 0x559e94acf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94aca200 0x559e94acf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f5916df9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e949a9500 con 0x559e94abcf40 2024-05-09T00:08:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f5916df9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e949a91a0 con 0x559e94abd900 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f5916df9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e94890640 con 0x559e94aca200 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f5916df9b80 10 client.0 _ll_register_callbacks cb 0x559e949b2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f59089db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abcf40 0x559e94ad1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f59089db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abcf40 0x559e94ad1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59648/0 (socket says 172.21.15.12:59648) 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f59089db700 1 -- 172.21.15.12:0/2613971148 learned_addr learned my addr 172.21.15.12:0/2613971148 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f59091dc700 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94aca200 0x559e94acf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.249+0000 7f5903fff700 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 0x559e94acf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f59089db700 1 -- 172.21.15.12:0/2613971148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94aca200 msgr2=0x559e94acf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f59089db700 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94aca200 0x559e94acf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f59089db700 1 -- 172.21.15.12:0/2613971148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 msgr2=0x559e94acf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f59089db700 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 0x559e94acf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f59089db700 1 -- 172.21.15.12:0/2613971148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59041a4300 con 0x559e94abcf40 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f59091dc700 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94aca200 0x559e94acf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f59089db700 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abcf40 0x559e94ad1e70 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f58fc01d000 tx=0x7f58fc099890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16762baaf6bc32a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f58e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f58e3fff700 1 -- 172.21.15.12:0/2613971148 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc09a0f0 con 0x559e94abcf40 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f58e3fff700 1 -- 172.21.15.12:0/2613971148 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f58fc09a560 con 0x559e94abcf40 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.250+0000 7f5916df9b80 1 -- 172.21.15.12:0/2613971148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e94966210 con 0x559e94abcf40 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f58e3fff700 1 -- 172.21.15.12:0/2613971148 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58fc0b82b0 con 0x559e94abcf40 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f5916df9b80 10 client.21423 fetch_fsmap learned FSMap version 76 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f5916df9b80 1 -- 172.21.15.12:0/2613971148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x559e9483ab00 con 0x559e94abcf40 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f58e3fff700 1 -- 172.21.15.12:0/2613971148 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58fc063070 con 0x559e94abcf40 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f58e3fff700 1 -- 172.21.15.12:0/2613971148 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f58fc01bc90 con 0x559e94abcf40 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f5916df9b80 10 client.21423 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f5916df9b80 10 client.21423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f5916df9b80 20 client.21423 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f5916df9b80 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559e94adf0f0 0x559e94ae14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f5916df9b80 4 client.21423 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f5916df9b80 1 -- 172.21.15.12:0/2613971148 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e94a8c7b0 con 0x559e94adf0f0 2024-05-09T00:08:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.251+0000 7f5903fff700 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559e94adf0f0 0x559e94ae14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.252+0000 7f5903fff700 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559e94adf0f0 0x559e94ae14e0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c7ef7c4558fd0b server_cookie=563e20a8b8ad4aea in_seq=0 out_seq=0 2024-05-09T00:08:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.252+0000 7f58e3fff700 10 client.21423 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.258+0000 7f58e3fff700 1 -- 172.21.15.12:0/2613971148 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e94a8c7b0 con 0x559e94adf0f0 2024-05-09T00:08:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.258+0000 7f58e3fff700 10 client.21423 handle_command_reply: tid=0 2024-05-09T00:08:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.259+0000 7f5916df9b80 1 client.21423 shutdown 2024-05-09T00:08:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.261+0000 7f5916df9b80 1 -- 172.21.15.12:0/2613971148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abcf40 msgr2=0x559e94ad1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.261+0000 7f5916df9b80 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abcf40 0x559e94ad1e70 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f58fc01d000 tx=0x7f58fc099890 comp rx=0 tx=0).stop 2024-05-09T00:08:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.261+0000 7f5916df9b80 1 -- 172.21.15.12:0/2613971148 shutdown_connections 2024-05-09T00:08:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.261+0000 7f5916df9b80 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94aca200 0x559e94acf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.261+0000 7f5916df9b80 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abcf40 0x559e94ad1e70 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.261+0000 7f5916df9b80 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559e94adf0f0 0x559e94ae14e0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.261+0000 7f5916df9b80 1 --2- 172.21.15.12:0/2613971148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 0x559e94acf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.261+0000 7f5916df9b80 1 -- 172.21.15.12:0/2613971148 >> 172.21.15.12:0/2613971148 conn(0x559e94a1b1b0 msgr2=0x559e94abbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.261+0000 7f58e3fff700 0 client.21423 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.264+0000 7f5916df9b80 1 -- 172.21.15.12:0/2613971148 shutdown_connections 2024-05-09T00:08:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.264+0000 7f5916df9b80 1 -- 172.21.15.12:0/2613971148 wait complete. 2024-05-09T00:08:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.264+0000 7f5916df9b80 20 client.21423 trim_cache size 0 max 16384 2024-05-09T00:08:17.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:16 smithi204 ceph-mon[47373]: osdmap e129: 12 total, 12 up, 12 in 2024-05-09T00:08:17.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:16 smithi160 ceph-mon[46152]: osdmap e129: 12 total, 12 up, 12 in 2024-05-09T00:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.282+0000 7f5916df9b80 1 Processor -- start 2024-05-09T00:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.282+0000 7f5916df9b80 1 -- start start 2024-05-09T00:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.282+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 0x559e94acf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.282+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 0x559e94ab7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.282+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94ad64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.282+0000 7f5916df9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e94ab8220 con 0x559e94aca200 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.282+0000 7f5916df9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e94adbbf0 con 0x559e94abd900 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.282+0000 7f5916df9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e94aca5e0 con 0x559e94abcf40 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59091dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94ad64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59089db700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 0x559e94acf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f5903fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 0x559e94ab7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59089db700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 0x559e94acf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57636/0 (socket says 172.21.15.12:57636) 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59089db700 1 -- 172.21.15.12:0/3013726437 learned_addr learned my addr 172.21.15.12:0/3013726437 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f5903fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 0x559e94ab7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47382/0 (socket says 172.21.15.12:47382) 2024-05-09T00:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59091dc700 1 -- 172.21.15.12:0/3013726437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 msgr2=0x559e94acf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59091dc700 1 --2- 172.21.15.12:0/3013726437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 0x559e94acf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59091dc700 1 -- 172.21.15.12:0/3013726437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 msgr2=0x559e94ab7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59091dc700 1 --2- 172.21.15.12:0/3013726437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 0x559e94ab7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59091dc700 1 -- 172.21.15.12:0/3013726437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e949d3500 con 0x559e94aca200 2024-05-09T00:08:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59089db700 1 --2- 172.21.15.12:0/3013726437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 0x559e94acf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f59091dc700 1 --2- 172.21.15.12:0/3013726437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94ad64b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f58f80098e0 tx=0x7f58f8005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e2639dec6d6d6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f58eb7fe700 1 -- 172.21.15.12:0/3013726437 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f801d040 con 0x559e94aca200 2024-05-09T00:08:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f58eb7fe700 1 -- 172.21.15.12:0/3013726437 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f800bc50 con 0x559e94aca200 2024-05-09T00:08:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.283+0000 7f58eb7fe700 1 -- 172.21.15.12:0/3013726437 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8003df0 con 0x559e94aca200 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.284+0000 7f5916df9b80 1 -- 172.21.15.12:0/3013726437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 msgr2=0x559e94ad64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.284+0000 7f5916df9b80 1 --2- 172.21.15.12:0/3013726437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94ad64b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f58f80098e0 tx=0x7f58f8005960 comp rx=0 tx=0).stop 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.284+0000 7f5916df9b80 1 -- 172.21.15.12:0/3013726437 shutdown_connections 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.284+0000 7f5916df9b80 1 --2- 172.21.15.12:0/3013726437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94aca200 0x559e94ad64b0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.284+0000 7f5916df9b80 1 --2- 172.21.15.12:0/3013726437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 0x559e94acf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.284+0000 7f5916df9b80 1 --2- 172.21.15.12:0/3013726437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94abd900 0x559e94ab7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.284+0000 7f5916df9b80 1 -- 172.21.15.12:0/3013726437 >> 172.21.15.12:0/3013726437 conn(0x559e94a1b1b0 msgr2=0x559e94ab9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.284+0000 7f5916df9b80 1 -- 172.21.15.12:0/3013726437 shutdown_connections 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.284+0000 7f5916df9b80 1 -- 172.21.15.12:0/3013726437 wait complete. 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5916df9b80 1 -- start start 2024-05-09T00:08:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5916df9b80 1 Processor -- start 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 0x559e94ad4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abd900 0x559e94ad49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5916df9b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94aca200 0x559e94ad4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5916df9b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e94b345c0 con 0x559e94abd900 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5916df9b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e94b34700 con 0x559e94aca200 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5916df9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e94b34840 con 0x559e94abcf40 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5916df9b80 10 client.0 _ll_register_callbacks cb 0x7f58fc099fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5903fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abd900 0x559e94ad49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5903fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abd900 0x559e94ad49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59658/0 (socket says 172.21.15.12:59658) 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f5903fff700 1 -- 172.21.15.12:0/317478085 learned_addr learned my addr 172.21.15.12:0/317478085 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.285+0000 7f59089db700 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 0x559e94ad4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f59091dc700 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94aca200 0x559e94ad4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f5903fff700 1 -- 172.21.15.12:0/317478085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 msgr2=0x559e94ad4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f5903fff700 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 0x559e94ad4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f5903fff700 1 -- 172.21.15.12:0/317478085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94aca200 msgr2=0x559e94ad4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f5903fff700 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94aca200 0x559e94ad4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f5903fff700 1 -- 172.21.15.12:0/317478085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e9483ab00 con 0x559e94abd900 2024-05-09T00:08:17.286 INFO:tasks.workunit.client.0.smithi012.stdout: 10 299255 30.76 MB/sec execute 372 sec latency 108.303 ms 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f5903fff700 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abd900 0x559e94ad49d0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f58f4005d90 tx=0x7f58f40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fa9e958598addc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f58e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f58e3fff700 1 -- 172.21.15.12:0/317478085 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4024040 con 0x559e94abd900 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f58e3fff700 1 -- 172.21.15.12:0/317478085 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f58f402a800 con 0x559e94abd900 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f5916df9b80 1 -- 172.21.15.12:0/317478085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58fc09df20 con 0x559e94abd900 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f58e3fff700 1 -- 172.21.15.12:0/317478085 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58f400e840 con 0x559e94abd900 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f5916df9b80 10 client.21429 fetch_fsmap learned FSMap version 76 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.286+0000 7f5916df9b80 1 -- 172.21.15.12:0/317478085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7f590410e620 con 0x559e94abd900 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.287+0000 7f58e3fff700 1 -- 172.21.15.12:0/317478085 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f403f080 con 0x559e94abd900 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.287+0000 7f58e3fff700 1 -- 172.21.15.12:0/317478085 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 76) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f58f4033440 con 0x559e94abd900 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.287+0000 7f5916df9b80 10 client.21429 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.287+0000 7f5916df9b80 10 client.21429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.287+0000 7f5916df9b80 20 client.21429 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.287+0000 7f5916df9b80 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559e94b36020 0x559e94b38440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.287+0000 7f5916df9b80 4 client.21429 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:08:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.287+0000 7f5916df9b80 1 -- 172.21.15.12:0/317478085 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559e94a05e60 con 0x559e94b36020 2024-05-09T00:08:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.287+0000 7f59089db700 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559e94b36020 0x559e94b38440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.288+0000 7f59089db700 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559e94b36020 0x559e94b38440 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=896c31fd96dd8448 server_cookie=5403eeb94c4d4ae6 in_seq=0 out_seq=0 2024-05-09T00:08:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.288+0000 7f58e3fff700 10 client.21429 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:17.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:16 smithi012 ceph-mon[42503]: osdmap e129: 12 total, 12 up, 12 in 2024-05-09T00:08:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.617+0000 7f58e3fff700 1 -- 172.21.15.12:0/317478085 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559e94a05e60 con 0x559e94b36020 2024-05-09T00:08:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.617+0000 7f58e3fff700 10 client.21429 handle_command_reply: tid=0 2024-05-09T00:08:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.617+0000 7f5916df9b80 1 client.21429 shutdown 2024-05-09T00:08:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 1 -- 172.21.15.12:0/317478085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abd900 msgr2=0x559e94ad49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abd900 0x559e94ad49d0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f58f4005d90 tx=0x7f58f40047c0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 1 -- 172.21.15.12:0/317478085 shutdown_connections 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x559e94b36020 0x559e94b38440 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e94aca200 0x559e94ad4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e94abcf40 0x559e94ad4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 1 --2- 172.21.15.12:0/317478085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e94abd900 0x559e94ad49d0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 1 -- 172.21.15.12:0/317478085 >> 172.21.15.12:0/317478085 conn(0x559e94a1b1b0 msgr2=0x559e94ac9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f58e3fff700 0 client.21429 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 1 -- 172.21.15.12:0/317478085 shutdown_connections 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 1 -- 172.21.15.12:0/317478085 wait complete. 2024-05-09T00:08:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.618+0000 7f5916df9b80 20 client.21429 trim_cache size 0 max 16384 2024-05-09T00:08:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:08:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:08:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:08:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 -- 172.21.15.12:0/3521801602 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f58ec0a1290 msgr2=0x7f58ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f58ec0a1290 0x7f58ec0a3750 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5904198840 tx=0x7f58f4009260 comp rx=0 tx=0).stop 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 -- 172.21.15.12:0/3521801602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904069360 msgr2=0x7f59041972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904069360 0x7f59041972f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00ef10 tx=0x7f58fc00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 -- 172.21.15.12:0/3521801602 shutdown_connections 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f58ec0a1290 0x7f58ec0a3750 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590419bbd0 0x7f590419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904069360 0x7f59041972f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 --2- 172.21.15.12:0/3521801602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5904106260 0x7f5904197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 -- 172.21.15.12:0/3521801602 >> 172.21.15.12:0/3521801602 conn(0x7f5904076bc0 msgr2=0x7f59040fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 -- 172.21.15.12:0/3521801602 shutdown_connections 2024-05-09T00:08:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:17.619+0000 7f590ac3f700 1 -- 172.21.15.12:0/3521801602 wait complete. 2024-05-09T00:08:18.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:17 smithi204 ceph-mon[47373]: pgmap v704: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 544 op/s 2024-05-09T00:08:18.266 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:08:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: ignoring --setuser ceph since I am not root 2024-05-09T00:08:18.266 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:08:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: ignoring --setgroup ceph since I am not root 2024-05-09T00:08:18.266 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:08:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: starting mds.k at 2024-05-09T00:08:18.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:17 smithi160 ceph-mon[46152]: pgmap v704: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 544 op/s 2024-05-09T00:08:18.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 299716 30.74 MB/sec execute 373 sec latency 297.953 ms 2024-05-09T00:08:18.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:17 smithi012 ceph-mon[42503]: pgmap v704: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 544 op/s 2024-05-09T00:08:19.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:18 smithi204 ceph-mon[47373]: daemon mds.k restarted 2024-05-09T00:08:19.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:18 smithi204 ceph-mon[47373]: osdmap e130: 12 total, 12 up, 12 in 2024-05-09T00:08:19.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:18 smithi160 ceph-mon[46152]: daemon mds.k restarted 2024-05-09T00:08:19.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:18 smithi160 ceph-mon[46152]: osdmap e130: 12 total, 12 up, 12 in 2024-05-09T00:08:19.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 300302 30.76 MB/sec execute 374 sec latency 91.270 ms 2024-05-09T00:08:19.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:18 smithi012 ceph-mon[42503]: daemon mds.k restarted 2024-05-09T00:08:19.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:18 smithi012 ceph-mon[42503]: osdmap e130: 12 total, 12 up, 12 in 2024-05-09T00:08:20.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:20 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6832/676326208,v1:172.21.15.160:6833/676326208] up:boot 2024-05-09T00:08:20.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:20 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:08:20.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:20 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:08:20.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:20 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:08:20.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:20 smithi204 ceph-mon[47373]: pgmap v706: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 629 op/s 2024-05-09T00:08:20.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:20 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6832/676326208,v1:172.21.15.160:6833/676326208] up:boot 2024-05-09T00:08:20.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:20 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:08:20.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:20 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:08:20.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:20 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:08:20.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:20 smithi160 ceph-mon[46152]: pgmap v706: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 629 op/s 2024-05-09T00:08:20.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 300661 30.72 MB/sec execute 375 sec latency 399.987 ms 2024-05-09T00:08:20.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:20 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6832/676326208,v1:172.21.15.160:6833/676326208] up:boot 2024-05-09T00:08:20.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:20 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:08:20.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:20 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:08:20.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:20 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:08:20.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:20 smithi012 ceph-mon[42503]: pgmap v706: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 629 op/s 2024-05-09T00:08:21.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 301080 30.69 MB/sec execute 376 sec latency 143.651 ms 2024-05-09T00:08:22.169 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:21 smithi012 ceph-mon[42503]: pgmap v707: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.5 MiB/s wr, 424 op/s 2024-05-09T00:08:22.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:21 smithi204 ceph-mon[47373]: pgmap v707: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.5 MiB/s wr, 424 op/s 2024-05-09T00:08:22.265 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:21 smithi160 ceph-mon[46152]: pgmap v707: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.5 MiB/s wr, 424 op/s 2024-05-09T00:08:22.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 301648 30.70 MB/sec execute 377 sec latency 128.244 ms 2024-05-09T00:08:23.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 302185 30.70 MB/sec execute 378 sec latency 142.958 ms 2024-05-09T00:08:24.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:23 smithi012 ceph-mon[42503]: pgmap v708: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 15 MiB/s wr, 804 op/s 2024-05-09T00:08:24.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:23 smithi204 ceph-mon[47373]: pgmap v708: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 15 MiB/s wr, 804 op/s 2024-05-09T00:08:24.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:23 smithi160 ceph-mon[46152]: pgmap v708: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 15 MiB/s wr, 804 op/s 2024-05-09T00:08:24.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 302782 30.71 MB/sec execute 379 sec latency 112.877 ms 2024-05-09T00:08:25.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 303176 30.68 MB/sec execute 380 sec latency 312.706 ms 2024-05-09T00:08:26.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:25 smithi012 ceph-mon[42503]: pgmap v709: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-09T00:08:26.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:25 smithi204 ceph-mon[47373]: pgmap v709: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-09T00:08:26.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:25 smithi160 ceph-mon[46152]: pgmap v709: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-09T00:08:26.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 303607 30.64 MB/sec execute 381 sec latency 138.767 ms 2024-05-09T00:08:27.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 304363 30.68 MB/sec execute 382 sec latency 91.517 ms 2024-05-09T00:08:28.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:27 smithi012 ceph-mon[42503]: pgmap v710: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 657 op/s 2024-05-09T00:08:28.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:27 smithi204 ceph-mon[47373]: pgmap v710: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 657 op/s 2024-05-09T00:08:28.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:27 smithi160 ceph-mon[46152]: pgmap v710: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 657 op/s 2024-05-09T00:08:28.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 304904 30.68 MB/sec execute 383 sec latency 96.696 ms 2024-05-09T00:08:29.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 305530 30.67 MB/sec execute 384 sec latency 78.643 ms 2024-05-09T00:08:30.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:29 smithi012 ceph-mon[42503]: pgmap v711: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 596 op/s 2024-05-09T00:08:30.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:29 smithi204 ceph-mon[47373]: pgmap v711: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 596 op/s 2024-05-09T00:08:30.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:29 smithi160 ceph-mon[46152]: pgmap v711: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 596 op/s 2024-05-09T00:08:30.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 305995 30.68 MB/sec execute 385 sec latency 370.997 ms 2024-05-09T00:08:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.790+0000 7f9385202700 1 -- 172.21.15.12:0/620293253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9380103180 msgr2=0x7f9380103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.790+0000 7f9385202700 1 --2- 172.21.15.12:0/620293253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9380103180 0x7f9380103560 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9370009990 tx=0x7f9370009ca0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.790+0000 7f9385202700 1 -- 172.21.15.12:0/620293253 shutdown_connections 2024-05-09T00:08:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.790+0000 7f9385202700 1 --2- 172.21.15.12:0/620293253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93801082a0 0x7f938010fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.790+0000 7f9385202700 1 --2- 172.21.15.12:0/620293253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9380103180 0x7f9380103560 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.790+0000 7f9385202700 1 --2- 172.21.15.12:0/620293253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9380103b30 0x7f9380107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.790+0000 7f9385202700 1 -- 172.21.15.12:0/620293253 >> 172.21.15.12:0/620293253 conn(0x7f93800fe9a0 msgr2=0x7f9380100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.790+0000 7f9385202700 1 -- 172.21.15.12:0/620293253 shutdown_connections 2024-05-09T00:08:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.790+0000 7f9385202700 1 -- 172.21.15.12:0/620293253 wait complete. 2024-05-09T00:08:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.791+0000 7f9385202700 1 Processor -- start 2024-05-09T00:08:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f9385202700 1 -- start start 2024-05-09T00:08:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f9385202700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9380103180 0x7f93801994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f9385202700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93801082a0 0x7f9380199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f9385202700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f938019dd80 0x7f938019e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f9385202700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938019e800 con 0x7f9380103180 2024-05-09T00:08:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f9385202700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938019e970 con 0x7f93801082a0 2024-05-09T00:08:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f9385202700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938019eb30 con 0x7f938019dd80 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f937ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9380103180 0x7f93801994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f937ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9380103180 0x7f93801994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48126/0 (socket says 172.21.15.12:48126) 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f937ed9d700 1 -- 172.21.15.12:0/2193985892 learned_addr learned my addr 172.21.15.12:0/2193985892 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.792+0000 7f937f59e700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f938019dd80 0x7f938019e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f937e59c700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93801082a0 0x7f9380199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f937ed9d700 1 -- 172.21.15.12:0/2193985892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f938019dd80 msgr2=0x7f938019e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f937ed9d700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f938019dd80 0x7f938019e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f937ed9d700 1 -- 172.21.15.12:0/2193985892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93801082a0 msgr2=0x7f9380199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f937ed9d700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93801082a0 0x7f9380199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f937ed9d700 1 -- 172.21.15.12:0/2193985892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9378009c40 con 0x7f9380103180 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f937e59c700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93801082a0 0x7f9380199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f937ed9d700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9380103180 0x7f93801994f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f9370005910 tx=0x7f9370009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f9367fff700 1 -- 172.21.15.12:0/2193985892 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937001d070 con 0x7f9380103180 2024-05-09T00:08:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f9385202700 1 -- 172.21.15.12:0/2193985892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93700095f0 con 0x7f9380103180 2024-05-09T00:08:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f9385202700 1 -- 172.21.15.12:0/2193985892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93801a6320 con 0x7f9380103180 2024-05-09T00:08:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f9367fff700 1 -- 172.21.15.12:0/2193985892 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9370022470 con 0x7f9380103180 2024-05-09T00:08:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.793+0000 7f9367fff700 1 -- 172.21.15.12:0/2193985892 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370017650 con 0x7f9380103180 2024-05-09T00:08:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.797+0000 7f9367fff700 1 -- 172.21.15.12:0/2193985892 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f93700177b0 con 0x7f9380103180 2024-05-09T00:08:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.797+0000 7f9367fff700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93680a1390 0x7f93680a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.797+0000 7f9367fff700 1 -- 172.21.15.12:0/2193985892 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f93700cdf70 con 0x7f9380103180 2024-05-09T00:08:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.798+0000 7f93913bcb80 1 Processor -- start 2024-05-09T00:08:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.798+0000 7f937e59c700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93680a1390 0x7f93680a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.798+0000 7f93913bcb80 1 -- start start 2024-05-09T00:08:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.798+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.798+0000 7f937e59c700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93680a1390 0x7f93680a3850 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9374005950 tx=0x7f93740058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.798+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bbef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.798+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 0x558c62bbf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.798+0000 7f93913bcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62b160f0 con 0x558c62bbf480 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.798+0000 7f93913bcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62b13ab0 con 0x558c62bc4ab0 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.798+0000 7f93913bcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62b0f3a0 con 0x558c62bc53d0 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bbef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bbef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42098/0 (socket says 172.21.15.12:42098) 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937e59c700 1 -- 172.21.15.12:0/3608093011 learned_addr learned my addr 172.21.15.12:0/3608093011 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937f59e700 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 0x558c62bbf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937ed9d700 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937ed9d700 1 -- 172.21.15.12:0/3608093011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 msgr2=0x558c62bbef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937ed9d700 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bbef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937ed9d700 1 -- 172.21.15.12:0/3608093011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 msgr2=0x558c62bbf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937ed9d700 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 0x558c62bbf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937ed9d700 1 -- 172.21.15.12:0/3608093011 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c62990350 con 0x558c62bc4ab0 2024-05-09T00:08:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937f59e700 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 0x558c62bbf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937e59c700 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bbef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.799+0000 7f937ed9d700 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc4e90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f93700cb6a0 tx=0x7f937009aec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=874dbd6764670cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.801+0000 7f93657fa700 1 -- 172.21.15.12:0/3608093011 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937009b7f0 con 0x558c62bc4ab0 2024-05-09T00:08:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.801+0000 7f93657fa700 1 -- 172.21.15.12:0/3608093011 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f937009b0f0 con 0x558c62bc4ab0 2024-05-09T00:08:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.801+0000 7f93657fa700 1 -- 172.21.15.12:0/3608093011 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937009b350 con 0x558c62bc4ab0 2024-05-09T00:08:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.802+0000 7f93913bcb80 1 -- 172.21.15.12:0/3608093011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 msgr2=0x558c62bc4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.802+0000 7f93913bcb80 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc4e90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f93700cb6a0 tx=0x7f937009aec0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.802+0000 7f93913bcb80 1 -- 172.21.15.12:0/3608093011 shutdown_connections 2024-05-09T00:08:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.802+0000 7f93913bcb80 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 0x558c62bbf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.802+0000 7f93913bcb80 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc4e90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.802+0000 7f93913bcb80 1 --2- 172.21.15.12:0/3608093011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bbef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.802+0000 7f93913bcb80 1 -- 172.21.15.12:0/3608093011 >> 172.21.15.12:0/3608093011 conn(0x558c62b1d1b0 msgr2=0x558c62bb9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.804+0000 7f93913bcb80 1 -- 172.21.15.12:0/3608093011 shutdown_connections 2024-05-09T00:08:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.804+0000 7f93913bcb80 1 -- 172.21.15.12:0/3608093011 wait complete. 2024-05-09T00:08:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.804+0000 7f93913bcb80 1 -- start start 2024-05-09T00:08:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.804+0000 7f93913bcb80 1 Processor -- start 2024-05-09T00:08:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 0x558c62bc8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f93913bcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62aab500 con 0x558c62bc53d0 2024-05-09T00:08:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f93913bcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62aab1a0 con 0x558c62bc4ab0 2024-05-09T00:08:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f93913bcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62992640 con 0x558c62bbf480 2024-05-09T00:08:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f93913bcb80 10 client.0 _ll_register_callbacks cb 0x558c62ab4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f937ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f937f59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 0x558c62bc8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f937ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42110/0 (socket says 172.21.15.12:42110) 2024-05-09T00:08:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.805+0000 7f937ed9d700 1 -- 172.21.15.12:0/866334943 learned_addr learned my addr 172.21.15.12:0/866334943 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f937ed9d700 1 -- 172.21.15.12:0/866334943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 msgr2=0x558c62bc86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f937ed9d700 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f937ed9d700 1 -- 172.21.15.12:0/866334943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 msgr2=0x558c62bc8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f937ed9d700 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 0x558c62bc8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f937ed9d700 1 -- 172.21.15.12:0/866334943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937400fbb0 con 0x558c62bbf480 2024-05-09T00:08:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f937f59e700 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 0x558c62bc8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f937ed9d700 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcb380 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9370000c00 tx=0x7f9370005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=395c0a11adc2dd57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f9362ff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f9362ff5700 1 -- 172.21.15.12:0/866334943 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937009a7e0 con 0x558c62bbf480 2024-05-09T00:08:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f9362ff5700 1 -- 172.21.15.12:0/866334943 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f937009fe40 con 0x558c62bbf480 2024-05-09T00:08:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f93913bcb80 1 -- 172.21.15.12:0/866334943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c62a68210 con 0x558c62bbf480 2024-05-09T00:08:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.806+0000 7f9362ff5700 1 -- 172.21.15.12:0/866334943 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93700a34e0 con 0x558c62bbf480 2024-05-09T00:08:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.810+0000 7f93913bcb80 10 client.31165 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.810+0000 7f93913bcb80 1 -- 172.21.15.12:0/866334943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7f93801a6320 con 0x558c62bbf480 2024-05-09T00:08:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.810+0000 7f9362ff5700 1 -- 172.21.15.12:0/866334943 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558c62a68210 con 0x558c62bbf480 2024-05-09T00:08:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.810+0000 7f9362ff5700 1 -- 172.21.15.12:0/866334943 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f93700a3050 con 0x558c62bbf480 2024-05-09T00:08:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.810+0000 7f93913bcb80 10 client.31165 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.810+0000 7f93913bcb80 10 client.31165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.810+0000 7f93913bcb80 20 client.31165 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.810+0000 7f93913bcb80 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c62bd86e0 0x558c62bdab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.810+0000 7f93913bcb80 4 client.31165 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.810+0000 7f93913bcb80 1 -- 172.21.15.12:0/866334943 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c62b8e7b0 con 0x558c62bd86e0 2024-05-09T00:08:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.812+0000 7f937e59c700 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c62bd86e0 0x558c62bdab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.812+0000 7f937e59c700 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c62bd86e0 0x558c62bdab00 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=114c9905b2725f7c server_cookie=533f590cfae2c0bf in_seq=0 out_seq=0 2024-05-09T00:08:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.812+0000 7f9362ff5700 10 client.31165 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f9362ff5700 1 -- 172.21.15.12:0/866334943 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c62b8e7b0 con 0x558c62bd86e0 2024-05-09T00:08:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f9362ff5700 10 client.31165 handle_command_reply: tid=0 2024-05-09T00:08:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 client.31165 shutdown 2024-05-09T00:08:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 -- 172.21.15.12:0/866334943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 msgr2=0x558c62bcb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcb380 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9370000c00 tx=0x7f9370005c00 comp rx=0 tx=0).stop 2024-05-09T00:08:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 -- 172.21.15.12:0/866334943 shutdown_connections 2024-05-09T00:08:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 0x558c62bc8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcb380 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c62bd86e0 0x558c62bdab00 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 --2- 172.21.15.12:0/866334943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 -- 172.21.15.12:0/866334943 >> 172.21.15.12:0/866334943 conn(0x558c62b1d1b0 msgr2=0x558c62bb9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f9362ff5700 0 client.31165 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 -- 172.21.15.12:0/866334943 shutdown_connections 2024-05-09T00:08:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.814+0000 7f93913bcb80 1 -- 172.21.15.12:0/866334943 wait complete. 2024-05-09T00:08:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.815+0000 7f93913bcb80 20 client.31165 trim_cache size 0 max 16384 2024-05-09T00:08:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.833+0000 7f93913bcb80 1 Processor -- start 2024-05-09T00:08:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.833+0000 7f93913bcb80 1 -- start start 2024-05-09T00:08:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.833+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 0x558c62bc8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.833+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.833+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bb9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.833+0000 7f93913bcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62bc6ca0 con 0x558c62bbf480 2024-05-09T00:08:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.833+0000 7f93913bcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62bc3340 con 0x558c62bc4ab0 2024-05-09T00:08:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.833+0000 7f93913bcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62bbef40 con 0x558c62bc53d0 2024-05-09T00:08:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937f59e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bb9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54330/0 (socket says 172.21.15.12:54330) 2024-05-09T00:08:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937e59c700 1 -- 172.21.15.12:0/430997423 learned_addr learned my addr 172.21.15.12:0/430997423 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937ed9d700 1 --2- 172.21.15.12:0/430997423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 0x558c62bc8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937f59e700 1 -- 172.21.15.12:0/430997423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 msgr2=0x558c62bc8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937f59e700 1 --2- 172.21.15.12:0/430997423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937f59e700 1 -- 172.21.15.12:0/430997423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 msgr2=0x558c62bc8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937f59e700 1 --2- 172.21.15.12:0/430997423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 0x558c62bc8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937f59e700 1 -- 172.21.15.12:0/430997423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c6293cb00 con 0x558c62bc53d0 2024-05-09T00:08:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.834+0000 7f937ed9d700 1 --2- 172.21.15.12:0/430997423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 0x558c62bc8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.835+0000 7f937f59e700 1 --2- 172.21.15.12:0/430997423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bb9910 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f937800ebb0 tx=0x7f9378009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3096e641439d9be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.835+0000 7f9365ffb700 1 -- 172.21.15.12:0/430997423 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937800e040 con 0x558c62bc53d0 2024-05-09T00:08:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.835+0000 7f9365ffb700 1 -- 172.21.15.12:0/430997423 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f937800f040 con 0x558c62bc53d0 2024-05-09T00:08:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.835+0000 7f9365ffb700 1 -- 172.21.15.12:0/430997423 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93780043b0 con 0x558c62bc53d0 2024-05-09T00:08:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.836+0000 7f93913bcb80 1 -- 172.21.15.12:0/430997423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 msgr2=0x558c62bb9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.836+0000 7f93913bcb80 1 --2- 172.21.15.12:0/430997423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bb9910 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f937800ebb0 tx=0x7f9378009920 comp rx=0 tx=0).stop 2024-05-09T00:08:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.836+0000 7f93913bcb80 1 -- 172.21.15.12:0/430997423 shutdown_connections 2024-05-09T00:08:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.836+0000 7f93913bcb80 1 --2- 172.21.15.12:0/430997423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bc53d0 0x558c62bb9910 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.836+0000 7f93913bcb80 1 --2- 172.21.15.12:0/430997423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bbf480 0x558c62bc8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.836+0000 7f93913bcb80 1 --2- 172.21.15.12:0/430997423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bc8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.836+0000 7f93913bcb80 1 -- 172.21.15.12:0/430997423 >> 172.21.15.12:0/430997423 conn(0x558c62b1d1b0 msgr2=0x558c62bdf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.836+0000 7f93913bcb80 1 -- 172.21.15.12:0/430997423 shutdown_connections 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.836+0000 7f93913bcb80 1 -- 172.21.15.12:0/430997423 wait complete. 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.836+0000 7f93913bcb80 1 -- start start 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f93913bcb80 1 Processor -- start 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bcc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f93913bcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 0x558c62bcc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f93913bcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62c2dc10 con 0x558c62bc53d0 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f93913bcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62c2dd80 con 0x558c62bc4ab0 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f93913bcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c62c3ef50 con 0x558c62bbf480 2024-05-09T00:08:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f93913bcb80 10 client.0 _ll_register_callbacks cb 0x558c62ab4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f937ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f937ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42130/0 (socket says 172.21.15.12:42130) 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f937ed9d700 1 -- 172.21.15.12:0/3692894357 learned_addr learned my addr 172.21.15.12:0/3692894357 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f937f59e700 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 0x558c62bcc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f937ed9d700 1 -- 172.21.15.12:0/3692894357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 msgr2=0x558c62bcc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f937ed9d700 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bcc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f937ed9d700 1 -- 172.21.15.12:0/3692894357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 msgr2=0x558c62bcc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f937ed9d700 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 0x558c62bcc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.837+0000 7f937ed9d700 1 -- 172.21.15.12:0/3692894357 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93801a6320 con 0x558c62bbf480 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.838+0000 7f937ed9d700 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcbad0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9370099440 tx=0x7f937001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb429f4fd8cb5e1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.838+0000 7f9362ff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.838+0000 7f9362ff5700 1 -- 172.21.15.12:0/3692894357 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93700a34e0 con 0x558c62bbf480 2024-05-09T00:08:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.838+0000 7f93913bcb80 1 -- 172.21.15.12:0/3692894357 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f937009b0f0 con 0x558c62bbf480 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.838+0000 7f9362ff5700 1 -- 172.21.15.12:0/3692894357 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f937009fe40 con 0x558c62bbf480 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.838+0000 7f9362ff5700 1 -- 172.21.15.12:0/3692894357 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9370097000 con 0x558c62bbf480 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.838+0000 7f93913bcb80 10 client.31171 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.838+0000 7f93913bcb80 1 -- 172.21.15.12:0/3692894357 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7f937800e890 con 0x558c62bbf480 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.839+0000 7f9362ff5700 1 -- 172.21.15.12:0/3692894357 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f937009b0f0 con 0x558c62bbf480 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.839+0000 7f9362ff5700 1 -- 172.21.15.12:0/3692894357 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f93700843a0 con 0x558c62bbf480 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.839+0000 7f93913bcb80 10 client.31171 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.839+0000 7f93913bcb80 10 client.31171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.839+0000 7f93913bcb80 20 client.31171 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.839+0000 7f93913bcb80 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c62c40280 0x558c62c42670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.839+0000 7f93913bcb80 4 client.31171 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:08:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.839+0000 7f93913bcb80 1 -- 172.21.15.12:0/3692894357 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558c62b07e60 con 0x558c62c40280 2024-05-09T00:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.844+0000 7f937e59c700 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c62c40280 0x558c62c42670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.845+0000 7f937e59c700 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c62c40280 0x558c62c42670 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11a03b2c16d11b8 server_cookie=a9288979241362ae in_seq=0 out_seq=0 2024-05-09T00:08:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:30.845+0000 7f9362ff5700 10 client.31171 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:30.859 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:08:31.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.137+0000 7f45ce108700 1 -- 172.21.15.12:0/4291559452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45c811a780 msgr2=0x7f45c811cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.137+0000 7f45ce108700 1 --2- 172.21.15.12:0/4291559452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45c811a780 0x7f45c811cb70 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009990 tx=0x7f45b8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.146+0000 7f45ce108700 1 -- 172.21.15.12:0/4291559452 shutdown_connections 2024-05-09T00:08:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.146+0000 7f45ce108700 1 --2- 172.21.15.12:0/4291559452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45c811a780 0x7f45c811cb70 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.146+0000 7f45ce108700 1 --2- 172.21.15.12:0/4291559452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45c810f6a0 0x7f45c8107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.146+0000 7f45ce108700 1 --2- 172.21.15.12:0/4291559452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45c810fb10 0x7f45c81084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.146+0000 7f45ce108700 1 -- 172.21.15.12:0/4291559452 >> 172.21.15.12:0/4291559452 conn(0x7f45c806bff0 msgr2=0x7f45c806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:31.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.165+0000 7f45ce108700 1 -- 172.21.15.12:0/4291559452 shutdown_connections 2024-05-09T00:08:31.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.169+0000 7f45ce108700 1 -- 172.21.15.12:0/4291559452 wait complete. 2024-05-09T00:08:31.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.170+0000 7f45ce108700 1 Processor -- start 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.170+0000 7f45ce108700 1 -- start start 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.170+0000 7f45ce108700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45c810f6a0 0x7f45c8118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.170+0000 7f45ce108700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45c810fb10 0x7f45c8119170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.170+0000 7f45ce108700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45c811a780 0x7f45c8111db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.170+0000 7f45ce108700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c8112580 con 0x7f45c810fb10 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.170+0000 7f45ce108700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c81126f0 con 0x7f45c810f6a0 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.170+0000 7f45ce108700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c81128b0 con 0x7f45c811a780 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.172+0000 7f45c7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45c811a780 0x7f45c8111db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.173+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45c810fb10 0x7f45c8119170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.173+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45c810fb10 0x7f45c8119170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48180/0 (socket says 172.21.15.12:48180) 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.173+0000 7f45c6ffd700 1 -- 172.21.15.12:0/907668566 learned_addr learned my addr 172.21.15.12:0/907668566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.174+0000 7f45c6ffd700 1 -- 172.21.15.12:0/907668566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45c811a780 msgr2=0x7f45c8111db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.174+0000 7f45c6ffd700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45c811a780 0x7f45c8111db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.174+0000 7f45c6ffd700 1 -- 172.21.15.12:0/907668566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45c810f6a0 msgr2=0x7f45c8118bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.174+0000 7f45c6ffd700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45c810f6a0 0x7f45c8118bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.174+0000 7f45c6ffd700 1 -- 172.21.15.12:0/907668566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80095f0 con 0x7f45c810fb10 2024-05-09T00:08:31.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.174+0000 7f45c6ffd700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45c810fb10 0x7f45c8119170 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f45c000b9e0 tx=0x7f45c000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:31.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.185+0000 7f45c4ff9700 1 -- 172.21.15.12:0/907668566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c000ca30 con 0x7f45c810fb10 2024-05-09T00:08:31.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.185+0000 7f45c4ff9700 1 -- 172.21.15.12:0/907668566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45c000cb90 con 0x7f45c810fb10 2024-05-09T00:08:31.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.185+0000 7f45c4ff9700 1 -- 172.21.15.12:0/907668566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00125a0 con 0x7f45c810fb10 2024-05-09T00:08:31.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.186+0000 7f45ce108700 1 -- 172.21.15.12:0/907668566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c8116780 con 0x7f45c810fb10 2024-05-09T00:08:31.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.186+0000 7f45ce108700 1 -- 172.21.15.12:0/907668566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c8116c40 con 0x7f45c810fb10 2024-05-09T00:08:31.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.192+0000 7f45c4ff9700 1 -- 172.21.15.12:0/907668566 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f45c0012700 con 0x7f45c810fb10 2024-05-09T00:08:31.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.193+0000 7f45c4ff9700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45b00a14a0 0x7f45b00a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.193+0000 7f45c4ff9700 1 -- 172.21.15.12:0/907668566 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f45c00cd220 con 0x7f45c810fb10 2024-05-09T00:08:31.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.194+0000 7f45c77fe700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45b00a14a0 0x7f45b00a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.202+0000 7f45da2c2b80 1 Processor -- start 2024-05-09T00:08:31.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.203+0000 7f45c77fe700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45b00a14a0 0x7f45b00a3960 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009770 tx=0x7f45bc006cd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:31.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.211+0000 7f45da2c2b80 1 -- start start 2024-05-09T00:08:31.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.211+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccbe00 0x560c28ccc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.211+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccc720 0x560c28cc6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.211+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.211+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28c1d150 con 0x560c28ccc720 2024-05-09T00:08:31.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.211+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28c1ab10 con 0x560c28cc6890 2024-05-09T00:08:31.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.211+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28c16400 con 0x560c28ccbe00 2024-05-09T00:08:31.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.214+0000 7f45c7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.214+0000 7f45c7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54366/0 (socket says 172.21.15.12:54366) 2024-05-09T00:08:31.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.214+0000 7f45c7fff700 1 -- 172.21.15.12:0/2859522997 learned_addr learned my addr 172.21.15.12:0/2859522997 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:31.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.217+0000 7f45c6ffd700 1 --2- 172.21.15.12:0/2859522997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccc720 0x560c28cc6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.223+0000 7f45c7fff700 1 -- 172.21.15.12:0/2859522997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccbe00 msgr2=0x560c28ccc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:31.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.223+0000 7f45c7fff700 1 --2- 172.21.15.12:0/2859522997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccbe00 0x560c28ccc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.223+0000 7f45c7fff700 1 -- 172.21.15.12:0/2859522997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccc720 msgr2=0x560c28cc6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:31.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.223+0000 7f45c7fff700 1 --2- 172.21.15.12:0/2859522997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccc720 0x560c28cc6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.223+0000 7f45c7fff700 1 -- 172.21.15.12:0/2859522997 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c28a973b0 con 0x560c28cc6890 2024-05-09T00:08:31.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.235+0000 7f45c7fff700 1 --2- 172.21.15.12:0/2859522997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6cd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f45b8005b40 tx=0x7f45b8003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fd5b182cd9c1da5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:31.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.249+0000 7f45adffb700 1 -- 172.21.15.12:0/2859522997 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b801d070 con 0x560c28cc6890 2024-05-09T00:08:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.249+0000 7f45adffb700 1 -- 172.21.15.12:0/2859522997 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45b800be80 con 0x560c28cc6890 2024-05-09T00:08:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.249+0000 7f45adffb700 1 -- 172.21.15.12:0/2859522997 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8005480 con 0x560c28cc6890 2024-05-09T00:08:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.250+0000 7f45da2c2b80 1 -- 172.21.15.12:0/2859522997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 msgr2=0x560c28cc6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.250+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/2859522997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6cd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f45b8005b40 tx=0x7f45b8003680 comp rx=0 tx=0).stop 2024-05-09T00:08:31.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.251+0000 7f45da2c2b80 1 -- 172.21.15.12:0/2859522997 shutdown_connections 2024-05-09T00:08:31.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.251+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/2859522997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6cd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.251+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/2859522997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccbe00 0x560c28ccc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.251+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/2859522997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccc720 0x560c28cc6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.251+0000 7f45da2c2b80 1 -- 172.21.15.12:0/2859522997 >> 172.21.15.12:0/2859522997 conn(0x560c28c24210 msgr2=0x560c28cc0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.257+0000 7f45da2c2b80 1 -- 172.21.15.12:0/2859522997 shutdown_connections 2024-05-09T00:08:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.259+0000 7f45da2c2b80 1 -- 172.21.15.12:0/2859522997 wait complete. 2024-05-09T00:08:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.259+0000 7f45da2c2b80 1 -- start start 2024-05-09T00:08:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.259+0000 7f45da2c2b80 1 Processor -- start 2024-05-09T00:08:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.260+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cd2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.260+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28ccfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.260+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccc720 0x560c28cd01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.260+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28bb2560 con 0x560c28ccbe00 2024-05-09T00:08:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.260+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28bb2200 con 0x560c28cc6890 2024-05-09T00:08:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.260+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28a996a0 con 0x560c28ccc720 2024-05-09T00:08:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.260+0000 7f45da2c2b80 10 client.0 _ll_register_callbacks cb 0x560c28bbbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.260+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28ccfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.260+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28ccfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48202/0 (socket says 172.21.15.12:48202) 2024-05-09T00:08:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.260+0000 7f45c6ffd700 1 -- 172.21.15.12:0/3113573726 learned_addr learned my addr 172.21.15.12:0/3113573726 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.261+0000 7f45c6ffd700 1 -- 172.21.15.12:0/3113573726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccc720 msgr2=0x560c28cd01b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:08:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.261+0000 7f45c6ffd700 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccc720 0x560c28cd01b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.261+0000 7f45c6ffd700 1 -- 172.21.15.12:0/3113573726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 msgr2=0x560c28cd2780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:08:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.261+0000 7f45c6ffd700 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cd2780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.261+0000 7f45c6ffd700 1 -- 172.21.15.12:0/3113573726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c811d460 con 0x560c28ccbe00 2024-05-09T00:08:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.262+0000 7f45c6ffd700 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28ccfa70 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f45c0014d10 tx=0x7f45c00caeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8aa5c390eaac72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:31.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.264+0000 7f45a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:08:31.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.273+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3113573726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00999a0 con 0x560c28ccbe00 2024-05-09T00:08:31.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.274+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3113573726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f45c0099b00 con 0x560c28ccbe00 2024-05-09T00:08:31.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.274+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3113573726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c28b6f270 con 0x560c28ccbe00 2024-05-09T00:08:31.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.275+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3113573726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45c00a2410 con 0x560c28ccbe00 2024-05-09T00:08:31.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.276+0000 7f45da2c2b80 10 client.21465 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:31.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.276+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3113573726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x560c28a43b60 con 0x560c28ccbe00 2024-05-09T00:08:31.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.279+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3113573726 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c28b6f270 con 0x560c28ccbe00 2024-05-09T00:08:31.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.279+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3113573726 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f45c00964b0 con 0x560c28ccbe00 2024-05-09T00:08:31.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.279+0000 7f45da2c2b80 10 client.21465 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:31.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.279+0000 7f45da2c2b80 10 client.21465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:31.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.279+0000 7f45da2c2b80 20 client.21465 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:31.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.279+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c28ce0150 0x560c28ce2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.279+0000 7f45da2c2b80 4 client.21465 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:31.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.279+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3113573726 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c28c95810 con 0x560c28ce0150 2024-05-09T00:08:31.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.284+0000 7f45c77fe700 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c28ce0150 0x560c28ce2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 306261 30.64 MB/sec execute 386 sec latency 378.598 ms 2024-05-09T00:08:31.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.289+0000 7f45c77fe700 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c28ce0150 0x560c28ce2570 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ccfb81e3f444a87 server_cookie=33449f6d6f6bd60a in_seq=0 out_seq=0 2024-05-09T00:08:31.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.289+0000 7f45a6ffd700 10 client.21465 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.377+0000 7f9362ff5700 1 -- 172.21.15.12:0/3692894357 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558c62b07e60 con 0x558c62c40280 2024-05-09T00:08:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.377+0000 7f9362ff5700 10 client.31171 handle_command_reply: tid=0 2024-05-09T00:08:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.377+0000 7f93913bcb80 1 client.31171 shutdown 2024-05-09T00:08:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:08:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:08:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:08:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:08:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.377+0000 7f93913bcb80 1 -- 172.21.15.12:0/3692894357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 msgr2=0x558c62bcbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.377+0000 7f93913bcb80 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcbad0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9370099440 tx=0x7f937001d000 comp rx=0 tx=0).stop 2024-05-09T00:08:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.378+0000 7f93913bcb80 1 -- 172.21.15.12:0/3692894357 shutdown_connections 2024-05-09T00:08:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.378+0000 7f93913bcb80 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c62bc53d0 0x558c62bcc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.378+0000 7f93913bcb80 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c62bbf480 0x558c62bcbad0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.378+0000 7f93913bcb80 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558c62c40280 0x558c62c42670 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.378+0000 7f93913bcb80 1 --2- 172.21.15.12:0/3692894357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558c62bc4ab0 0x558c62bcc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.378+0000 7f93913bcb80 1 -- 172.21.15.12:0/3692894357 >> 172.21.15.12:0/3692894357 conn(0x558c62b1d1b0 msgr2=0x558c62bc27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.378+0000 7f9362ff5700 0 client.31171 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.380+0000 7f93913bcb80 1 -- 172.21.15.12:0/3692894357 shutdown_connections 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.380+0000 7f93913bcb80 1 -- 172.21.15.12:0/3692894357 wait complete. 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.380+0000 7f93913bcb80 20 client.31171 trim_cache size 0 max 16384 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 -- 172.21.15.12:0/2193985892 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93680a1390 msgr2=0x7f93680a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93680a1390 0x7f93680a3850 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9374005950 tx=0x7f93740058e0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 -- 172.21.15.12:0/2193985892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9380103180 msgr2=0x7f93801994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9380103180 0x7f93801994f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f9370005910 tx=0x7f9370009570 comp rx=0 tx=0).stop 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 -- 172.21.15.12:0/2193985892 shutdown_connections 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f93680a1390 0x7f93680a3850 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f938019dd80 0x7f938019e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9380103180 0x7f93801994f0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 --2- 172.21.15.12:0/2193985892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93801082a0 0x7f9380199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 -- 172.21.15.12:0/2193985892 >> 172.21.15.12:0/2193985892 conn(0x7f93800fe9a0 msgr2=0x7f93800ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:31.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3113573726 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c28c95810 con 0x560c28ce0150 2024-05-09T00:08:31.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f45a6ffd700 10 client.21465 handle_command_reply: tid=0 2024-05-09T00:08:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 -- 172.21.15.12:0/2193985892 shutdown_connections 2024-05-09T00:08:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f9385202700 1 -- 172.21.15.12:0/2193985892 wait complete. 2024-05-09T00:08:31.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.381+0000 7f45da2c2b80 1 client.21465 shutdown 2024-05-09T00:08:31.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.382+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3113573726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 msgr2=0x560c28ccfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.382+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28ccfa70 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f45c0014d10 tx=0x7f45c00caeb0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.382+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3113573726 shutdown_connections 2024-05-09T00:08:31.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.382+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccc720 0x560c28cd01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.382+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c28ce0150 0x560c28ce2570 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.382+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cd2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.382+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3113573726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28ccfa70 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.382+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3113573726 >> 172.21.15.12:0/3113573726 conn(0x560c28c24210 msgr2=0x560c28cc05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:31.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.382+0000 7f45a6ffd700 0 client.21465 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:31.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.383+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3113573726 shutdown_connections 2024-05-09T00:08:31.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.383+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3113573726 wait complete. 2024-05-09T00:08:31.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.383+0000 7f45da2c2b80 20 client.21465 trim_cache size 0 max 16384 2024-05-09T00:08:31.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.436+0000 7f45da2c2b80 1 Processor -- start 2024-05-09T00:08:31.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.436+0000 7f45da2c2b80 1 -- start start 2024-05-09T00:08:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.436+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.436+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28cc0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.436+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccc720 0x560c28cce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.436+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28cb9040 con 0x560c28ccbe00 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.436+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28ccdfc0 con 0x560c28cc6890 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.436+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28cc7230 con 0x560c28ccc720 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.437+0000 7f45c77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.437+0000 7f45c77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54392/0 (socket says 172.21.15.12:54392) 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.437+0000 7f45c77fe700 1 -- 172.21.15.12:0/1790455030 learned_addr learned my addr 172.21.15.12:0/1790455030 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.437+0000 7f45c77fe700 1 -- 172.21.15.12:0/1790455030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccc720 msgr2=0x560c28cce340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.437+0000 7f45c77fe700 1 --2- 172.21.15.12:0/1790455030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccc720 0x560c28cce340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.437+0000 7f45c77fe700 1 -- 172.21.15.12:0/1790455030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 msgr2=0x560c28cc0d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.437+0000 7f45c77fe700 1 --2- 172.21.15.12:0/1790455030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28cc0d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.437+0000 7f45c77fe700 1 -- 172.21.15.12:0/1790455030 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c28bdc560 con 0x560c28cc6890 2024-05-09T00:08:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.438+0000 7f45c77fe700 1 --2- 172.21.15.12:0/1790455030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6350 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009ec0 tx=0x7f45bc004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ba9ca60e1a587a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.440+0000 7f45ae7fc700 1 -- 172.21.15.12:0/1790455030 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc01a040 con 0x560c28cc6890 2024-05-09T00:08:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.440+0000 7f45ae7fc700 1 -- 172.21.15.12:0/1790455030 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45bc01c040 con 0x560c28cc6890 2024-05-09T00:08:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.440+0000 7f45ae7fc700 1 -- 172.21.15.12:0/1790455030 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc013bd0 con 0x560c28cc6890 2024-05-09T00:08:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.441+0000 7f45da2c2b80 1 -- 172.21.15.12:0/1790455030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 msgr2=0x560c28cc6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.441+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/1790455030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6350 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009ec0 tx=0x7f45bc004870 comp rx=0 tx=0).stop 2024-05-09T00:08:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.443+0000 7f45da2c2b80 1 -- 172.21.15.12:0/1790455030 shutdown_connections 2024-05-09T00:08:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.443+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/1790455030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28ccc720 0x560c28cce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.443+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/1790455030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28cc6890 0x560c28cc6350 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.443+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/1790455030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28cc0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.443+0000 7f45da2c2b80 1 -- 172.21.15.12:0/1790455030 >> 172.21.15.12:0/1790455030 conn(0x560c28c24210 msgr2=0x560c28cdbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.446+0000 7f45da2c2b80 1 -- 172.21.15.12:0/1790455030 shutdown_connections 2024-05-09T00:08:31.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.455+0000 7f45da2c2b80 1 -- 172.21.15.12:0/1790455030 wait complete. 2024-05-09T00:08:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.455+0000 7f45da2c2b80 1 -- start start 2024-05-09T00:08:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.455+0000 7f45da2c2b80 1 Processor -- start 2024-05-09T00:08:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28cc6890 0x560c28cc82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28cc8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45da2c2b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28ccc720 0x560c28cc8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28cd7a70 con 0x560c28ccbe00 2024-05-09T00:08:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28d36670 con 0x560c28ccc720 2024-05-09T00:08:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45da2c2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c28d367b0 con 0x560c28cc6890 2024-05-09T00:08:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45da2c2b80 10 client.0 _ll_register_callbacks cb 0x560c28bbbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28cc8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28cc8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48208/0 (socket says 172.21.15.12:48208) 2024-05-09T00:08:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45c6ffd700 1 -- 172.21.15.12:0/3104232402 learned_addr learned my addr 172.21.15.12:0/3104232402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.456+0000 7f45c7fff700 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28ccc720 0x560c28cc8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.457+0000 7f45c6ffd700 1 -- 172.21.15.12:0/3104232402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28cc6890 msgr2=0x560c28cc82f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:08:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.457+0000 7f45c6ffd700 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28cc6890 0x560c28cc82f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.457+0000 7f45c6ffd700 1 -- 172.21.15.12:0/3104232402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28ccc720 msgr2=0x560c28cc8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.457+0000 7f45c6ffd700 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28ccc720 0x560c28cc8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.457+0000 7f45c6ffd700 1 -- 172.21.15.12:0/3104232402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc0151e0 con 0x560c28ccbe00 2024-05-09T00:08:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.458+0000 7f45c6ffd700 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28cc8830 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f45c0098800 tx=0x7f45c0099e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f2b5518af17639 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:31.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.458+0000 7f45a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:08:31.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.458+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3104232402 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00a2610 con 0x560c28ccbe00 2024-05-09T00:08:31.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.458+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3104232402 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f45c00999a0 con 0x560c28ccbe00 2024-05-09T00:08:31.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.458+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3104232402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45b800be80 con 0x560c28ccbe00 2024-05-09T00:08:31.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.458+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3104232402 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45c0099b60 con 0x560c28ccbe00 2024-05-09T00:08:31.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.458+0000 7f45da2c2b80 10 client.21468 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:31.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.459+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3104232402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x560c28a9bcd0 con 0x560c28ccbe00 2024-05-09T00:08:31.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.459+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3104232402 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45b800be80 con 0x560c28ccbe00 2024-05-09T00:08:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.460+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3104232402 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f45c00b8020 con 0x560c28ccbe00 2024-05-09T00:08:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.460+0000 7f45da2c2b80 10 client.21468 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.460+0000 7f45da2c2b80 10 client.21468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.460+0000 7f45da2c2b80 20 client.21468 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.460+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c28d37b10 0x560c28d39f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.460+0000 7f45da2c2b80 4 client.21468 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:08:31.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.460+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3104232402 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c28c0eec0 con 0x560c28d37b10 2024-05-09T00:08:31.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.464+0000 7f45c77fe700 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c28d37b10 0x560c28d39f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.465+0000 7f45c77fe700 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c28d37b10 0x560c28d39f30 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8eef235fb07149e server_cookie=7e8aa75587b6e95 in_seq=0 out_seq=0 2024-05-09T00:08:31.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.467+0000 7f45a6ffd700 10 client.21468 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:31.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.481+0000 7f45a6ffd700 1 -- 172.21.15.12:0/3104232402 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560c28c0eec0 con 0x560c28d37b10 2024-05-09T00:08:31.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.481+0000 7f45a6ffd700 10 client.21468 handle_command_reply: tid=0 2024-05-09T00:08:31.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.481+0000 7f45da2c2b80 1 client.21468 shutdown 2024-05-09T00:08:31.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3104232402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 msgr2=0x560c28cc8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28cc8830 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f45c0098800 tx=0x7f45c0099e10 comp rx=0 tx=0).stop 2024-05-09T00:08:31.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3104232402 shutdown_connections 2024-05-09T00:08:31.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x560c28d37b10 0x560c28d39f30 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560c28ccc720 0x560c28cc8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560c28cc6890 0x560c28cc82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 1 --2- 172.21.15.12:0/3104232402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c28ccbe00 0x560c28cc8830 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3104232402 >> 172.21.15.12:0/3104232402 conn(0x560c28c24210 msgr2=0x560c28cdbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:31.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45a6ffd700 0 client.21468 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:31.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3104232402 shutdown_connections 2024-05-09T00:08:31.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 1 -- 172.21.15.12:0/3104232402 wait complete. 2024-05-09T00:08:31.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.486+0000 7f45da2c2b80 20 client.21468 trim_cache size 0 max 16384 2024-05-09T00:08:31.485 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:08:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:08:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:08:31.485 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:08:31.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.488+0000 7f45ce108700 1 -- 172.21.15.12:0/907668566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45b00a14a0 msgr2=0x7f45b00a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.488+0000 7f45ce108700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45b00a14a0 0x7f45b00a3960 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009770 tx=0x7f45bc006cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.488+0000 7f45ce108700 1 -- 172.21.15.12:0/907668566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45c810fb10 msgr2=0x7f45c8119170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.488+0000 7f45ce108700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45c810fb10 0x7f45c8119170 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f45c000b9e0 tx=0x7f45c000bda0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.488+0000 7f45ce108700 1 -- 172.21.15.12:0/907668566 shutdown_connections 2024-05-09T00:08:31.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.488+0000 7f45ce108700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f45b00a14a0 0x7f45b00a3960 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.488+0000 7f45ce108700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45c811a780 0x7f45c8111db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.488+0000 7f45ce108700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45c810f6a0 0x7f45c8118bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.488+0000 7f45ce108700 1 --2- 172.21.15.12:0/907668566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45c810fb10 0x7f45c8119170 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.488+0000 7f45ce108700 1 -- 172.21.15.12:0/907668566 >> 172.21.15.12:0/907668566 conn(0x7f45c806bff0 msgr2=0x7f45c806f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:31.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.489+0000 7f45ce108700 1 -- 172.21.15.12:0/907668566 shutdown_connections 2024-05-09T00:08:31.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.489+0000 7f45ce108700 1 -- 172.21.15.12:0/907668566 wait complete. 2024-05-09T00:08:31.514 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:08:31.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:08:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.681+0000 7f89836cb700 1 -- 172.21.15.12:0/3655084504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f897c100530 msgr2=0x7f897c10d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.681+0000 7f89836cb700 1 --2- 172.21.15.12:0/3655084504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f897c100530 0x7f897c10d1f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f896c009990 tx=0x7f896c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.681+0000 7f89836cb700 1 -- 172.21.15.12:0/3655084504 shutdown_connections 2024-05-09T00:08:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.681+0000 7f89836cb700 1 --2- 172.21.15.12:0/3655084504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c10d730 0x7f897c10fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.681+0000 7f89836cb700 1 --2- 172.21.15.12:0/3655084504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897c0ffb80 0x7f897c0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.681+0000 7f89836cb700 1 --2- 172.21.15.12:0/3655084504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f897c100530 0x7f897c10d1f0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.681+0000 7f89836cb700 1 -- 172.21.15.12:0/3655084504 >> 172.21.15.12:0/3655084504 conn(0x7f897c0fb7e0 msgr2=0x7f897c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:31.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.681+0000 7f89836cb700 1 -- 172.21.15.12:0/3655084504 shutdown_connections 2024-05-09T00:08:31.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.681+0000 7f89836cb700 1 -- 172.21.15.12:0/3655084504 wait complete. 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f89836cb700 1 Processor -- start 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f89836cb700 1 -- start start 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f89836cb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c0ffb80 0x7f897c199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f89836cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f897c100530 0x7f897c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f89836cb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897c10d730 0x7f897c19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f89836cb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c19e430 con 0x7f897c100530 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f89836cb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c19e5a0 con 0x7f897c10d730 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f89836cb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c19e760 con 0x7f897c0ffb80 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f8980c66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f897c100530 0x7f897c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f8980c66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f897c100530 0x7f897c199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48228/0 (socket says 172.21.15.12:48228) 2024-05-09T00:08:31.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f8980c66700 1 -- 172.21.15.12:0/4259178774 learned_addr learned my addr 172.21.15.12:0/4259178774 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f8981467700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c0ffb80 0x7f897c199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f8981c68700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897c10d730 0x7f897c19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f8980c66700 1 -- 172.21.15.12:0/4259178774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c0ffb80 msgr2=0x7f897c199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f8980c66700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c0ffb80 0x7f897c199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f8980c66700 1 -- 172.21.15.12:0/4259178774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897c10d730 msgr2=0x7f897c19ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f8980c66700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897c10d730 0x7f897c19ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.682+0000 7f8980c66700 1 -- 172.21.15.12:0/4259178774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c0095f0 con 0x7f897c100530 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.683+0000 7f8980c66700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f897c100530 0x7f897c199a50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f896c000c00 tx=0x7f896c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.683+0000 7f89727fc700 1 -- 172.21.15.12:0/4259178774 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c01d070 con 0x7f897c100530 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.683+0000 7f89836cb700 1 -- 172.21.15.12:0/4259178774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c19e9e0 con 0x7f897c100530 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.683+0000 7f89727fc700 1 -- 172.21.15.12:0/4259178774 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c022470 con 0x7f897c100530 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.683+0000 7f89727fc700 1 -- 172.21.15.12:0/4259178774 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c017650 con 0x7f897c100530 2024-05-09T00:08:31.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.683+0000 7f89836cb700 1 -- 172.21.15.12:0/4259178774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c1a6280 con 0x7f897c100530 2024-05-09T00:08:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.684+0000 7f89836cb700 1 -- 172.21.15.12:0/4259178774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c108950 con 0x7f897c100530 2024-05-09T00:08:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.685+0000 7f89727fc700 1 -- 172.21.15.12:0/4259178774 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f896c0225e0 con 0x7f897c100530 2024-05-09T00:08:31.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.686+0000 7f89727fc700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89680a1290 0x7f89680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.686+0000 7f8981467700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89680a1290 0x7f89680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.686+0000 7f89727fc700 1 -- 172.21.15.12:0/4259178774 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f896c0cde20 con 0x7f897c100530 2024-05-09T00:08:31.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.686+0000 7f8981467700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89680a1290 0x7f89680a3750 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8978009c60 tx=0x7f8978009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:31.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:31.693+0000 7f89727fc700 1 -- 172.21.15.12:0/4259178774 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f896c097340 con 0x7f897c100530 2024-05-09T00:08:32.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.010+0000 7f89836cb700 1 -- 172.21.15.12:0/4259178774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f897c19a810 con 0x7f897c100530 2024-05-09T00:08:32.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.011+0000 7f89727fc700 1 -- 172.21.15.12:0/4259178774 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+10873 (secure 0 0 0) 0x7f896c09a960 con 0x7f897c100530 2024-05-09T00:08:32.008 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:08:32.008 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":21435,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/676326208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":676326208},{"type":"v1","addr":"172.21.15.160:6833","nonce":676326208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":78,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:08:19.008049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 -- 172.21.15.12:0/4259178774 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89680a1290 msgr2=0x7f89680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89680a1290 0x7f89680a3750 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8978009c60 tx=0x7f8978009260 comp rx=0 tx=0).stop 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 -- 172.21.15.12:0/4259178774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f897c100530 msgr2=0x7f897c199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f897c100530 0x7f897c199a50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f896c000c00 tx=0x7f896c004730 comp rx=0 tx=0).stop 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 -- 172.21.15.12:0/4259178774 shutdown_connections 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89680a1290 0x7f89680a3750 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897c10d730 0x7f897c19ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c0ffb80 0x7f897c199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 --2- 172.21.15.12:0/4259178774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f897c100530 0x7f897c199a50 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 -- 172.21.15.12:0/4259178774 >> 172.21.15.12:0/4259178774 conn(0x7f897c0fb7e0 msgr2=0x7f897c0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.013+0000 7f89836cb700 1 -- 172.21.15.12:0/4259178774 shutdown_connections 2024-05-09T00:08:32.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.014+0000 7f89836cb700 1 -- 172.21.15.12:0/4259178774 wait complete. 2024-05-09T00:08:32.012 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 78 2024-05-09T00:08:32.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:08:32.052 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:31 smithi012 ceph-mon[42503]: pgmap v712: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 559 op/s 2024-05-09T00:08:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.178+0000 7fcc08689700 1 -- 172.21.15.12:0/2070650891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00100060 msgr2=0x7fcc00100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.178+0000 7fcc08689700 1 --2- 172.21.15.12:0/2070650891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00100060 0x7fcc00100440 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009990 tx=0x7fcbfc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.178+0000 7fcc08689700 1 -- 172.21.15.12:0/2070650891 shutdown_connections 2024-05-09T00:08:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.178+0000 7fcc08689700 1 --2- 172.21.15.12:0/2070650891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc00105160 0x7fcc0010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.178+0000 7fcc08689700 1 --2- 172.21.15.12:0/2070650891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00100060 0x7fcc00100440 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.178+0000 7fcc08689700 1 --2- 172.21.15.12:0/2070650891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc00100a10 0x7fcc00104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.178+0000 7fcc08689700 1 -- 172.21.15.12:0/2070650891 >> 172.21.15.12:0/2070650891 conn(0x7fcc00076f40 msgr2=0x7fcc00077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.178+0000 7fcc08689700 1 -- 172.21.15.12:0/2070650891 shutdown_connections 2024-05-09T00:08:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.178+0000 7fcc08689700 1 -- 172.21.15.12:0/2070650891 wait complete. 2024-05-09T00:08:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.178+0000 7fcc08689700 1 Processor -- start 2024-05-09T00:08:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc08689700 1 -- start start 2024-05-09T00:08:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc08689700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc00100060 0x7fcc00101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc08689700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105160 0x7fcc00102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc08689700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc00102550 0x7fcc001a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc08689700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001a5be0 con 0x7fcc00105160 2024-05-09T00:08:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc08689700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001a5d50 con 0x7fcc00102550 2024-05-09T00:08:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc08689700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001a5f10 con 0x7fcc00100060 2024-05-09T00:08:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc05c24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105160 0x7fcc00102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc05c24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105160 0x7fcc00102010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48238/0 (socket says 172.21.15.12:48238) 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc06425700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc00100060 0x7fcc00101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc05c24700 1 -- 172.21.15.12:0/23262798 learned_addr learned my addr 172.21.15.12:0/23262798 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc06c26700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc00102550 0x7fcc001a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc06425700 1 -- 172.21.15.12:0/23262798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc00102550 msgr2=0x7fcc001a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc06425700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc00102550 0x7fcc001a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc06425700 1 -- 172.21.15.12:0/23262798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105160 msgr2=0x7fcc00102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc06425700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105160 0x7fcc00102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc06425700 1 -- 172.21.15.12:0/23262798 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc0095f0 con 0x7fcc00100060 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.179+0000 7fcc05c24700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105160 0x7fcc00102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.180+0000 7fcc06425700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc00100060 0x7fcc00101ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc000c00 tx=0x7fcbfc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.180+0000 7fcbf77fe700 1 -- 172.21.15.12:0/23262798 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc01d070 con 0x7fcc00100060 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.180+0000 7fcc08689700 1 -- 172.21.15.12:0/23262798 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc001a6190 con 0x7fcc00100060 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.180+0000 7fcbf77fe700 1 -- 172.21.15.12:0/23262798 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbfc021410 con 0x7fcc00100060 2024-05-09T00:08:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.180+0000 7fcbf77fe700 1 -- 172.21.15.12:0/23262798 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc017650 con 0x7fcc00100060 2024-05-09T00:08:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.180+0000 7fcc08689700 1 -- 172.21.15.12:0/23262798 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc001a6610 con 0x7fcc00100060 2024-05-09T00:08:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.182+0000 7fcbf77fe700 1 -- 172.21.15.12:0/23262798 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcbfc021580 con 0x7fcc00100060 2024-05-09T00:08:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.183+0000 7fcbf77fe700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbec0a1290 0x7fcbec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.183+0000 7fcc05c24700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbec0a1290 0x7fcbec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.183+0000 7fcbf77fe700 1 -- 172.21.15.12:0/23262798 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcbfc0cde20 con 0x7fcc00100060 2024-05-09T00:08:32.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.184+0000 7fcc05c24700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbec0a1290 0x7fcbec0a3750 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcc00103200 tx=0x7fcbf0009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.184+0000 7fcc14843b80 1 Processor -- start 2024-05-09T00:08:32.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.184+0000 7fcc14843b80 1 -- start start 2024-05-09T00:08:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d7421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d73c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d73c890 0x55c05d73ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc14843b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d693150 con 0x55c05d741e00 2024-05-09T00:08:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc14843b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d690b10 con 0x55c05d73c890 2024-05-09T00:08:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc14843b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d68c400 con 0x55c05d742720 2024-05-09T00:08:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc06425700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d7421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc06425700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d7421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48242/0 (socket says 172.21.15.12:48242) 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc06425700 1 -- 172.21.15.12:0/2042833541 learned_addr learned my addr 172.21.15.12:0/2042833541 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc05c24700 1 --2- 172.21.15.12:0/2042833541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d73c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc06425700 1 -- 172.21.15.12:0/2042833541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 msgr2=0x55c05d73c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc06425700 1 --2- 172.21.15.12:0/2042833541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d73c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc06425700 1 -- 172.21.15.12:0/2042833541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d73c890 msgr2=0x55c05d73ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc06425700 1 --2- 172.21.15.12:0/2042833541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d73c890 0x55c05d73ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc06425700 1 -- 172.21.15.12:0/2042833541 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c05d50d3b0 con 0x55c05d741e00 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.185+0000 7fcc06425700 1 --2- 172.21.15.12:0/2042833541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d7421e0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc097a70 tx=0x7fcbfc09add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7a5f29ee146e3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.186+0000 7fcbf4ff9700 1 -- 172.21.15.12:0/2042833541 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc09b700 con 0x55c05d741e00 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.186+0000 7fcbf4ff9700 1 -- 172.21.15.12:0/2042833541 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbfc09b000 con 0x55c05d741e00 2024-05-09T00:08:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.186+0000 7fcbf4ff9700 1 -- 172.21.15.12:0/2042833541 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc09b260 con 0x55c05d741e00 2024-05-09T00:08:32.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.187+0000 7fcc14843b80 1 -- 172.21.15.12:0/2042833541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 msgr2=0x55c05d7421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.187+0000 7fcc14843b80 1 --2- 172.21.15.12:0/2042833541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d7421e0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc097a70 tx=0x7fcbfc09add0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.187+0000 7fcc14843b80 1 -- 172.21.15.12:0/2042833541 shutdown_connections 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.187+0000 7fcc14843b80 1 --2- 172.21.15.12:0/2042833541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d73c890 0x55c05d73ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.187+0000 7fcc14843b80 1 --2- 172.21.15.12:0/2042833541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d7421e0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.187+0000 7fcc14843b80 1 --2- 172.21.15.12:0/2042833541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d73c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.187+0000 7fcc14843b80 1 -- 172.21.15.12:0/2042833541 >> 172.21.15.12:0/2042833541 conn(0x55c05d69a210 msgr2=0x55c05d736df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.187+0000 7fcc14843b80 1 -- 172.21.15.12:0/2042833541 shutdown_connections 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.187+0000 7fcc14843b80 1 -- 172.21.15.12:0/2042833541 wait complete. 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.187+0000 7fcc14843b80 1 -- start start 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc14843b80 1 Processor -- start 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d73c890 0x55c05d748740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d745a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d746170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc14843b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d628560 con 0x55c05d741e00 2024-05-09T00:08:32.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc14843b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d628200 con 0x55c05d73c890 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc14843b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d50f6a0 con 0x55c05d742720 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc14843b80 10 client.0 _ll_register_callbacks cb 0x55c05d631b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc05c24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d745a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc06425700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d73c890 0x55c05d748740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc05c24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d745a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48256/0 (socket says 172.21.15.12:48256) 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc05c24700 1 -- 172.21.15.12:0/3492811460 learned_addr learned my addr 172.21.15.12:0/3492811460 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.188+0000 7fcc06c26700 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d746170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc05c24700 1 -- 172.21.15.12:0/3492811460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 msgr2=0x55c05d746170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc05c24700 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d746170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc05c24700 1 -- 172.21.15.12:0/3492811460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d73c890 msgr2=0x55c05d748740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc05c24700 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d73c890 0x55c05d748740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc05c24700 1 -- 172.21.15.12:0/3492811460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc001a6610 con 0x55c05d741e00 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc06425700 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d73c890 0x55c05d748740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc06c26700 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d746170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc05c24700 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d745a30 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00039d0 tx=0x7fcbf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da5287915764cdf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcbe9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/3492811460 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf001fa90 con 0x55c05d741e00 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/3492811460 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcbf001fbf0 con 0x55c05d741e00 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc14843b80 1 -- 172.21.15.12:0/3492811460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c05d5e5270 con 0x55c05d741e00 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/3492811460 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbf001fdf0 con 0x55c05d741e00 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc14843b80 10 client.21486 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:32.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.189+0000 7fcc14843b80 1 -- 172.21.15.12:0/3492811460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55c05d4b9b60 con 0x55c05d741e00 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/3492811460 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c05d5e5270 con 0x55c05d741e00 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/3492811460 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcbf00273e0 con 0x55c05d741e00 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcc14843b80 10 client.21486 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcc14843b80 10 client.21486 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcc14843b80 20 client.21486 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcc14843b80 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c05d755f90 0x55c05d7583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcc14843b80 4 client.21486 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcc14843b80 1 -- 172.21.15.12:0/3492811460 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c05d70b810 con 0x55c05d755f90 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcc06425700 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c05d755f90 0x55c05d7583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcbe9ffb700 10 client.21486 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:08:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.190+0000 7fcc06425700 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c05d755f90 0x55c05d7583b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c6b5bf7a719e316 server_cookie=9ac6e34203cae162 in_seq=0 out_seq=0 2024-05-09T00:08:32.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/3492811460 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c05d70b810 con 0x55c05d755f90 2024-05-09T00:08:32.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcbe9ffb700 10 client.21486 handle_command_reply: tid=0 2024-05-09T00:08:32.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcc14843b80 1 client.21486 shutdown 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcc14843b80 1 -- 172.21.15.12:0/3492811460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 msgr2=0x55c05d745a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcc14843b80 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d745a30 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00039d0 tx=0x7fcbf00047c0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcc14843b80 1 -- 172.21.15.12:0/3492811460 shutdown_connections 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcc14843b80 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c05d755f90 0x55c05d7583b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcc14843b80 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d746170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcc14843b80 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d73c890 0x55c05d748740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcbe9ffb700 0 client.21486 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcc14843b80 1 --2- 172.21.15.12:0/3492811460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d741e00 0x55c05d745a30 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.192+0000 7fcc14843b80 1 -- 172.21.15.12:0/3492811460 >> 172.21.15.12:0/3492811460 conn(0x55c05d69a210 msgr2=0x55c05d7365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.193+0000 7fcc14843b80 1 -- 172.21.15.12:0/3492811460 shutdown_connections 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.193+0000 7fcc14843b80 1 -- 172.21.15.12:0/3492811460 wait complete. 2024-05-09T00:08:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.193+0000 7fcc14843b80 20 client.21486 trim_cache size 0 max 16384 2024-05-09T00:08:32.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc14843b80 1 Processor -- start 2024-05-09T00:08:32.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc14843b80 1 -- start start 2024-05-09T00:08:32.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d73c890 0x55c05d751990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 0x55c05d7389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d736d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc14843b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d733dc0 con 0x55c05d73c890 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc14843b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d737ae0 con 0x55c05d741e00 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc14843b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d72f040 con 0x55c05d742720 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc06425700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d73c890 0x55c05d751990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc06425700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d73c890 0x55c05d751990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48272/0 (socket says 172.21.15.12:48272) 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc06425700 1 -- 172.21.15.12:0/1143622113 learned_addr learned my addr 172.21.15.12:0/1143622113 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc05c24700 1 --2- 172.21.15.12:0/1143622113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 0x55c05d7389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.209+0000 7fcc06c26700 1 --2- 172.21.15.12:0/1143622113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d736d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.210+0000 7fcc06425700 1 -- 172.21.15.12:0/1143622113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 msgr2=0x55c05d736d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.210+0000 7fcc06425700 1 --2- 172.21.15.12:0/1143622113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d736d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.210+0000 7fcc06425700 1 -- 172.21.15.12:0/1143622113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 msgr2=0x55c05d7389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.210+0000 7fcc06425700 1 --2- 172.21.15.12:0/1143622113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 0x55c05d7389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.210+0000 7fcc06425700 1 -- 172.21.15.12:0/1143622113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c05d652560 con 0x55c05d73c890 2024-05-09T00:08:32.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.210+0000 7fcc05c24700 1 --2- 172.21.15.12:0/1143622113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 0x55c05d7389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:08:32.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.210+0000 7fcc06425700 1 --2- 172.21.15.12:0/1143622113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d73c890 0x55c05d751990 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc09af90 tx=0x7fcbfc0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44b2e69167deb7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.210+0000 7fcbf57fa700 1 -- 172.21.15.12:0/1143622113 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc09eeb0 con 0x55c05d73c890 2024-05-09T00:08:32.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.210+0000 7fcbf57fa700 1 -- 172.21.15.12:0/1143622113 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbfc0a33f0 con 0x55c05d73c890 2024-05-09T00:08:32.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.210+0000 7fcbf57fa700 1 -- 172.21.15.12:0/1143622113 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc09b4c0 con 0x55c05d73c890 2024-05-09T00:08:32.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.211+0000 7fcc14843b80 1 -- 172.21.15.12:0/1143622113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d73c890 msgr2=0x55c05d751990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.211+0000 7fcc14843b80 1 --2- 172.21.15.12:0/1143622113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d73c890 0x55c05d751990 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc09af90 tx=0x7fcbfc0047a0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.211+0000 7fcc14843b80 1 -- 172.21.15.12:0/1143622113 shutdown_connections 2024-05-09T00:08:32.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.211+0000 7fcc14843b80 1 --2- 172.21.15.12:0/1143622113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d742720 0x55c05d736d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.211+0000 7fcc14843b80 1 --2- 172.21.15.12:0/1143622113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d73c890 0x55c05d751990 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.211+0000 7fcc14843b80 1 --2- 172.21.15.12:0/1143622113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 0x55c05d7389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.211+0000 7fcc14843b80 1 -- 172.21.15.12:0/1143622113 >> 172.21.15.12:0/1143622113 conn(0x55c05d69a210 msgr2=0x55c05d73cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.211+0000 7fcc14843b80 1 -- 172.21.15.12:0/1143622113 shutdown_connections 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.211+0000 7fcc14843b80 1 -- 172.21.15.12:0/1143622113 wait complete. 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc14843b80 1 -- start start 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc14843b80 1 Processor -- start 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d73c890 0x55c05d74cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 0x55c05d74d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc14843b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d742720 0x55c05d73d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc14843b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d73e0a0 con 0x55c05d742720 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc14843b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d73e210 con 0x55c05d741e00 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc14843b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05d73e3d0 con 0x55c05d73c890 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc14843b80 10 client.0 _ll_register_callbacks cb 0x55c05d631b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc06425700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d73c890 0x55c05d74cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc06425700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d73c890 0x55c05d74cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42240/0 (socket says 172.21.15.12:42240) 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.212+0000 7fcc06425700 1 -- 172.21.15.12:0/1399126521 learned_addr learned my addr 172.21.15.12:0/1399126521 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcc05c24700 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 0x55c05d74d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcc06425700 1 -- 172.21.15.12:0/1399126521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 msgr2=0x55c05d74d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcc06425700 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 0x55c05d74d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcc06425700 1 -- 172.21.15.12:0/1399126521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d742720 msgr2=0x55c05d73d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcc06425700 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d742720 0x55c05d73d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcc06425700 1 -- 172.21.15.12:0/1399126521 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c05d50d3b0 con 0x55c05d73c890 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcc06425700 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d73c890 0x55c05d74cee0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc005f50 tx=0x7fcbfc00bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=672004e931c65860 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcbe9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/1399126521 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0b9170 con 0x55c05d73c890 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/1399126521 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcbfc0aee40 con 0x55c05d73c890 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.213+0000 7fcc14843b80 1 -- 172.21.15.12:0/1399126521 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbfc09b000 con 0x55c05d73c890 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/1399126521 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbfc0b92d0 con 0x55c05d73c890 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcc14843b80 10 client.31195 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcc14843b80 1 -- 172.21.15.12:0/1399126521 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55c05d511cd0 con 0x55c05d73c890 2024-05-09T00:08:32.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/1399126521 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbfc09b000 con 0x55c05d73c890 2024-05-09T00:08:32.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/1399126521 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcbfc0a2f60 con 0x55c05d73c890 2024-05-09T00:08:32.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcc14843b80 10 client.31195 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:32.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcc14843b80 10 client.31195 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:08:32.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcc14843b80 20 client.31195 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:32.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcc14843b80 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c05d7ac490 0x55c05d7ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcc14843b80 4 client.31195 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:08:32.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.214+0000 7fcc14843b80 1 -- 172.21.15.12:0/1399126521 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c05d684ec0 con 0x55c05d7ac490 2024-05-09T00:08:32.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.215+0000 7fcc05c24700 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c05d7ac490 0x55c05d7ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.215+0000 7fcc05c24700 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c05d7ac490 0x55c05d7ae880 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60ac6351a8a91577 server_cookie=ceeb1bb29f25bdd5 in_seq=0 out_seq=0 2024-05-09T00:08:32.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.215+0000 7fcbe9ffb700 10 client.31195 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:08:32.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.216+0000 7fcbe9ffb700 1 -- 172.21.15.12:0/1399126521 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c05d684ec0 con 0x55c05d7ac490 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.216+0000 7fcbe9ffb700 10 client.31195 handle_command_reply: tid=0 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.216+0000 7fcc14843b80 1 client.31195 shutdown 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 1 -- 172.21.15.12:0/1399126521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d73c890 msgr2=0x55c05d74cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d73c890 0x55c05d74cee0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc005f50 tx=0x7fcbfc00bfc0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 1 -- 172.21.15.12:0/1399126521 shutdown_connections 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c05d742720 0x55c05d73d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c05d7ac490 0x55c05d7ae880 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcbe9ffb700 0 client.31195 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c05d73c890 0x55c05d74cee0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 1 --2- 172.21.15.12:0/1399126521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c05d741e00 0x55c05d74d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 1 -- 172.21.15.12:0/1399126521 >> 172.21.15.12:0/1399126521 conn(0x55c05d69a210 msgr2=0x55c05d73fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 1 -- 172.21.15.12:0/1399126521 shutdown_connections 2024-05-09T00:08:32.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 1 -- 172.21.15.12:0/1399126521 wait complete. 2024-05-09T00:08:32.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.217+0000 7fcc14843b80 20 client.31195 trim_cache size 0 max 16384 2024-05-09T00:08:32.219 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:08:32.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 -- 172.21.15.12:0/23262798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbec0a1290 msgr2=0x7fcbec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbec0a1290 0x7fcbec0a3750 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcc00103200 tx=0x7fcbf0009260 comp rx=0 tx=0).stop 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 -- 172.21.15.12:0/23262798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc00100060 msgr2=0x7fcc00101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc00100060 0x7fcc00101ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc000c00 tx=0x7fcbfc004730 comp rx=0 tx=0).stop 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 -- 172.21.15.12:0/23262798 shutdown_connections 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fcbec0a1290 0x7fcbec0a3750 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc00102550 0x7fcc001a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc00100060 0x7fcc00101ad0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 --2- 172.21.15.12:0/23262798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105160 0x7fcc00102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 -- 172.21.15.12:0/23262798 >> 172.21.15.12:0/23262798 conn(0x7fcc00076f40 msgr2=0x7fcc00068510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 -- 172.21.15.12:0/23262798 shutdown_connections 2024-05-09T00:08:32.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.218+0000 7fcc08689700 1 -- 172.21.15.12:0/23262798 wait complete. 2024-05-09T00:08:32.230 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:08:32.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:31 smithi204 ceph-mon[47373]: pgmap v712: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 559 op/s 2024-05-09T00:08:32.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:31 smithi160 ceph-mon[46152]: pgmap v712: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 559 op/s 2024-05-09T00:08:32.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 306636 30.61 MB/sec execute 387 sec latency 231.832 ms 2024-05-09T00:08:32.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.426+0000 7f997fbcd700 1 -- 172.21.15.12:0/4165166339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9978069b70 msgr2=0x7f997810d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.426+0000 7f997fbcd700 1 --2- 172.21.15.12:0/4165166339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9978069b70 0x7f997810d400 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f9968009990 tx=0x7f9968009ca0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.426+0000 7f997fbcd700 1 -- 172.21.15.12:0/4165166339 shutdown_connections 2024-05-09T00:08:32.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.426+0000 7f997fbcd700 1 --2- 172.21.15.12:0/4165166339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997810d940 0x7f997810fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.426+0000 7f997fbcd700 1 --2- 172.21.15.12:0/4165166339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9978069250 0x7f9978069630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.426+0000 7f997fbcd700 1 --2- 172.21.15.12:0/4165166339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9978069b70 0x7f997810d400 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.426+0000 7f997fbcd700 1 -- 172.21.15.12:0/4165166339 >> 172.21.15.12:0/4165166339 conn(0x7f9978076f30 msgr2=0x7f9978077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.427+0000 7f997fbcd700 1 -- 172.21.15.12:0/4165166339 shutdown_connections 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.427+0000 7f997fbcd700 1 -- 172.21.15.12:0/4165166339 wait complete. 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.427+0000 7f997fbcd700 1 Processor -- start 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.427+0000 7f997fbcd700 1 -- start start 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.427+0000 7f997fbcd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9978069250 0x7f9978199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.427+0000 7f997fbcd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9978069b70 0x7f9978199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.427+0000 7f997fbcd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f997810d940 0x7f997819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.427+0000 7f997fbcd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997819e520 con 0x7f9978069250 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.427+0000 7f997fbcd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997819e690 con 0x7f997810d940 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.427+0000 7f997fbcd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997819e850 con 0x7f9978069b70 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d969700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9978069250 0x7f9978199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d969700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9978069250 0x7f9978199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48294/0 (socket says 172.21.15.12:48294) 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d969700 1 -- 172.21.15.12:0/615822585 learned_addr learned my addr 172.21.15.12:0/615822585 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997e16a700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f997810d940 0x7f997819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d168700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9978069b70 0x7f9978199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d969700 1 -- 172.21.15.12:0/615822585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9978069b70 msgr2=0x7f9978199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d969700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9978069b70 0x7f9978199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d969700 1 -- 172.21.15.12:0/615822585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f997810d940 msgr2=0x7f997819dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d969700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f997810d940 0x7f997819dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d969700 1 -- 172.21.15.12:0/615822585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99680095f0 con 0x7f9978069250 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d168700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9978069b70 0x7f9978199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997d969700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9978069250 0x7f9978199600 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f997400e9a0 tx=0x7f997400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f996effd700 1 -- 172.21.15.12:0/615822585 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974009790 con 0x7f9978069250 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f996effd700 1 -- 172.21.15.12:0/615822585 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f997400cbd0 con 0x7f9978069250 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997fbcd700 1 -- 172.21.15.12:0/615822585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997819eb30 con 0x7f9978069250 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f996effd700 1 -- 172.21.15.12:0/615822585 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974018680 con 0x7f9978069250 2024-05-09T00:08:32.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.428+0000 7f997fbcd700 1 -- 172.21.15.12:0/615822585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99781a6370 con 0x7f9978069250 2024-05-09T00:08:32.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.431+0000 7f996effd700 1 -- 172.21.15.12:0/615822585 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f997400cd40 con 0x7f9978069250 2024-05-09T00:08:32.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.431+0000 7f996effd700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99640a1290 0x7f99640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.432+0000 7f997d168700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99640a1290 0x7f99640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.432+0000 7f996effd700 1 -- 172.21.15.12:0/615822585 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9974014070 con 0x7f9978069250 2024-05-09T00:08:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.432+0000 7f997d168700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99640a1290 0x7f99640a3750 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f996800b2e0 tx=0x7f9968005fd0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.432+0000 7f998bd87b80 1 Processor -- start 2024-05-09T00:08:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f998bd87b80 1 -- start start 2024-05-09T00:08:32.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beae00 0x55dfb3beb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 0x55dfb3be5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3be5890 0x55dfb3be5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f998bd87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3b3c150 con 0x55dfb3be5890 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f998bd87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3b39b10 con 0x55dfb3beb720 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f998bd87b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3b35400 con 0x55dfb3beae00 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f997e16a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3be5890 0x55dfb3be5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f997d969700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beae00 0x55dfb3beb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f997e16a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3be5890 0x55dfb3be5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48296/0 (socket says 172.21.15.12:48296) 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f997d969700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beae00 0x55dfb3beb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42264/0 (socket says 172.21.15.12:42264) 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.433+0000 7f997e16a700 1 -- 172.21.15.12:0/527023954 learned_addr learned my addr 172.21.15.12:0/527023954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.434+0000 7f997d969700 1 -- 172.21.15.12:0/527023954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 msgr2=0x55dfb3be5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.434+0000 7f997d969700 1 --2- 172.21.15.12:0/527023954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 0x55dfb3be5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.434+0000 7f997d969700 1 -- 172.21.15.12:0/527023954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3be5890 msgr2=0x55dfb3be5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.434+0000 7f997d969700 1 --2- 172.21.15.12:0/527023954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3be5890 0x55dfb3be5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.434+0000 7f997d969700 1 -- 172.21.15.12:0/527023954 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfb39b63b0 con 0x55dfb3beae00 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.434+0000 7f997e16a700 1 --2- 172.21.15.12:0/527023954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3be5890 0x55dfb3be5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.434+0000 7f997d969700 1 --2- 172.21.15.12:0/527023954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beae00 0x55dfb3beb1e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f99740ca6b0 tx=0x7f99740999d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feffb03f0d1d488f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.434+0000 7f9963fff700 1 -- 172.21.15.12:0/527023954 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997409dea0 con 0x55dfb3beae00 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.434+0000 7f9963fff700 1 -- 172.21.15.12:0/527023954 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99740a24f0 con 0x55dfb3beae00 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.434+0000 7f9963fff700 1 -- 172.21.15.12:0/527023954 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740997a0 con 0x55dfb3beae00 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.435+0000 7f998bd87b80 1 -- 172.21.15.12:0/527023954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beae00 msgr2=0x55dfb3beb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.435+0000 7f998bd87b80 1 --2- 172.21.15.12:0/527023954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beae00 0x55dfb3beb1e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f99740ca6b0 tx=0x7f99740999d0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.435+0000 7f998bd87b80 1 -- 172.21.15.12:0/527023954 shutdown_connections 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.435+0000 7f998bd87b80 1 --2- 172.21.15.12:0/527023954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3be5890 0x55dfb3be5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.435+0000 7f998bd87b80 1 --2- 172.21.15.12:0/527023954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beae00 0x55dfb3beb1e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.435+0000 7f998bd87b80 1 --2- 172.21.15.12:0/527023954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 0x55dfb3be5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.435+0000 7f998bd87b80 1 -- 172.21.15.12:0/527023954 >> 172.21.15.12:0/527023954 conn(0x55dfb3b43210 msgr2=0x55dfb3bdfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.435+0000 7f998bd87b80 1 -- 172.21.15.12:0/527023954 shutdown_connections 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.435+0000 7f998bd87b80 1 -- 172.21.15.12:0/527023954 wait complete. 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.435+0000 7f998bd87b80 1 -- start start 2024-05-09T00:08:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.436+0000 7f998bd87b80 1 Processor -- start 2024-05-09T00:08:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.436+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3beec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.436+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.436+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 0x55dfb3bef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.436+0000 7f998bd87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3ad1560 con 0x55dfb3beae00 2024-05-09T00:08:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.436+0000 7f998bd87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3ad1200 con 0x55dfb3beb720 2024-05-09T00:08:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.436+0000 7f998bd87b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb39b86a0 con 0x55dfb3be5890 2024-05-09T00:08:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.436+0000 7f998bd87b80 10 client.0 _ll_register_callbacks cb 0x55dfb3adab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997d969700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3beec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997d969700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3beec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42272/0 (socket says 172.21.15.12:42272) 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997d969700 1 -- 172.21.15.12:0/4269618108 learned_addr learned my addr 172.21.15.12:0/4269618108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997d168700 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997e16a700 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 0x55dfb3bef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997d969700 1 -- 172.21.15.12:0/4269618108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 msgr2=0x55dfb3bef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997d969700 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 0x55dfb3bef680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997d969700 1 -- 172.21.15.12:0/4269618108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 msgr2=0x55dfb3bef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997d969700 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997d969700 1 -- 172.21.15.12:0/4269618108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99700112b0 con 0x55dfb3be5890 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f997d969700 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3beec00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f99740ca710 tx=0x7f997409dea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99500e48c875e4e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.437+0000 7f99617fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.438+0000 7f99617fa700 1 -- 172.21.15.12:0/4269618108 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997409a4e0 con 0x55dfb3be5890 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.438+0000 7f99617fa700 1 -- 172.21.15.12:0/4269618108 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f99740b82d0 con 0x55dfb3be5890 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.438+0000 7f998bd87b80 1 -- 172.21.15.12:0/4269618108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfb3a8e270 con 0x55dfb3be5890 2024-05-09T00:08:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.438+0000 7f99617fa700 1 -- 172.21.15.12:0/4269618108 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99740a2060 con 0x55dfb3be5890 2024-05-09T00:08:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.438+0000 7f998bd87b80 10 client.31207 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.438+0000 7f998bd87b80 1 -- 172.21.15.12:0/4269618108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7f99781a6370 con 0x55dfb3be5890 2024-05-09T00:08:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.438+0000 7f99617fa700 1 -- 172.21.15.12:0/4269618108 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dfb3a8e270 con 0x55dfb3be5890 2024-05-09T00:08:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.438+0000 7f99617fa700 1 -- 172.21.15.12:0/4269618108 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f997409edc0 con 0x55dfb3be5890 2024-05-09T00:08:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.438+0000 7f998bd87b80 10 client.31207 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.438+0000 7f998bd87b80 10 client.31207 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:08:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.439+0000 7f998bd87b80 20 client.31207 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.439+0000 7f998bd87b80 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55dfb3bfef20 0x55dfb3c01310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.439+0000 7f998bd87b80 4 client.31207 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.439+0000 7f998bd87b80 1 -- 172.21.15.12:0/4269618108 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfb3bb4810 con 0x55dfb3bfef20 2024-05-09T00:08:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.441+0000 7f997d168700 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55dfb3bfef20 0x55dfb3c01310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.441+0000 7f997d168700 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55dfb3bfef20 0x55dfb3c01310 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9e16987942ba386 server_cookie=12f1440d6f90e97c in_seq=0 out_seq=0 2024-05-09T00:08:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.441+0000 7f99617fa700 10 client.31207 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f99617fa700 1 -- 172.21.15.12:0/4269618108 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dfb3bb4810 con 0x55dfb3bfef20 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f99617fa700 10 client.31207 handle_command_reply: tid=0 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f998bd87b80 1 client.31207 shutdown 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f998bd87b80 1 -- 172.21.15.12:0/4269618108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 msgr2=0x55dfb3beec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f998bd87b80 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3beec00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f99740ca710 tx=0x7f997409dea0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f998bd87b80 1 -- 172.21.15.12:0/4269618108 shutdown_connections 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f998bd87b80 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55dfb3bfef20 0x55dfb3c01310 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f998bd87b80 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 0x55dfb3bef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f998bd87b80 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3beec00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f99617fa700 0 client.31207 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f998bd87b80 1 --2- 172.21.15.12:0/4269618108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f998bd87b80 1 -- 172.21.15.12:0/4269618108 >> 172.21.15.12:0/4269618108 conn(0x55dfb3b43210 msgr2=0x55dfb3bdf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.447+0000 7f998bd87b80 1 -- 172.21.15.12:0/4269618108 shutdown_connections 2024-05-09T00:08:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.448+0000 7f998bd87b80 1 -- 172.21.15.12:0/4269618108 wait complete. 2024-05-09T00:08:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.448+0000 7f998bd87b80 20 client.31207 trim_cache size 0 max 16384 2024-05-09T00:08:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f998bd87b80 1 Processor -- start 2024-05-09T00:08:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f998bd87b80 1 -- start start 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3be5890 0x55dfb3beb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bfac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beb720 0x55dfb3bdfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f998bd87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3bf01d0 con 0x55dfb3beae00 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f998bd87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3c064f0 con 0x55dfb3be5890 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f998bd87b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3bda800 con 0x55dfb3beb720 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997d168700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bfac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997d168700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bfac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48320/0 (socket says 172.21.15.12:48320) 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997d168700 1 -- 172.21.15.12:0/2314610091 learned_addr learned my addr 172.21.15.12:0/2314610091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997e16a700 1 --2- 172.21.15.12:0/2314610091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beb720 0x55dfb3bdfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997d969700 1 --2- 172.21.15.12:0/2314610091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3be5890 0x55dfb3beb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997d168700 1 -- 172.21.15.12:0/2314610091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beb720 msgr2=0x55dfb3bdfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997d168700 1 --2- 172.21.15.12:0/2314610091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beb720 0x55dfb3bdfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997d168700 1 -- 172.21.15.12:0/2314610091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3be5890 msgr2=0x55dfb3beb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997d168700 1 --2- 172.21.15.12:0/2314610091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3be5890 0x55dfb3beb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997d168700 1 -- 172.21.15.12:0/2314610091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfb3962b60 con 0x55dfb3beae00 2024-05-09T00:08:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f997d168700 1 --2- 172.21.15.12:0/2314610091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bfac20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9968000c00 tx=0x7f996801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff38353b89b166e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f996cda4700 1 -- 172.21.15.12:0/2314610091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996802b040 con 0x55dfb3beae00 2024-05-09T00:08:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f996cda4700 1 -- 172.21.15.12:0/2314610091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f996802c040 con 0x55dfb3beae00 2024-05-09T00:08:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.464+0000 7f996cda4700 1 -- 172.21.15.12:0/2314610091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968004880 con 0x55dfb3beae00 2024-05-09T00:08:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.465+0000 7f998bd87b80 1 -- 172.21.15.12:0/2314610091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 msgr2=0x55dfb3bfac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.465+0000 7f998bd87b80 1 --2- 172.21.15.12:0/2314610091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bfac20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9968000c00 tx=0x7f996801f000 comp rx=0 tx=0).stop 2024-05-09T00:08:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.465+0000 7f998bd87b80 1 -- 172.21.15.12:0/2314610091 shutdown_connections 2024-05-09T00:08:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.465+0000 7f998bd87b80 1 --2- 172.21.15.12:0/2314610091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3beb720 0x55dfb3bdfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.465+0000 7f998bd87b80 1 --2- 172.21.15.12:0/2314610091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3be5890 0x55dfb3beb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.465+0000 7f998bd87b80 1 --2- 172.21.15.12:0/2314610091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bfac20 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.465+0000 7f998bd87b80 1 -- 172.21.15.12:0/2314610091 >> 172.21.15.12:0/2314610091 conn(0x55dfb3b43210 msgr2=0x55dfb3bdc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.465+0000 7f998bd87b80 1 -- 172.21.15.12:0/2314610091 shutdown_connections 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.465+0000 7f998bd87b80 1 -- 172.21.15.12:0/2314610091 wait complete. 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.465+0000 7f998bd87b80 1 -- start start 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f998bd87b80 1 Processor -- start 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3bf6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bf4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f998bd87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 0x55dfb3bf4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f998bd87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3bf4bd0 con 0x55dfb3beae00 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f998bd87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3bf4d40 con 0x55dfb3beb720 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f998bd87b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb3bf4f00 con 0x55dfb3be5890 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f998bd87b80 10 client.0 _ll_register_callbacks cb 0x55dfb3adab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f997d168700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bf4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f997d969700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3bf6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f997d969700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3bf6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42302/0 (socket says 172.21.15.12:42302) 2024-05-09T00:08:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f997d168700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bf4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48322/0 (socket says 172.21.15.12:48322) 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.466+0000 7f997d969700 1 -- 172.21.15.12:0/1883109176 learned_addr learned my addr 172.21.15.12:0/1883109176 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f997d969700 1 -- 172.21.15.12:0/1883109176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 msgr2=0x55dfb3bf4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f997d969700 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 0x55dfb3bf4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f997d969700 1 -- 172.21.15.12:0/1883109176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 msgr2=0x55dfb3bf4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f997d969700 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bf4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f997d969700 1 -- 172.21.15.12:0/1883109176 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9978110860 con 0x55dfb3be5890 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f997d969700 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3bf6360 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f997400e970 tx=0x7f997401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9e49f6819ab0785 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f99617fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f99617fa700 1 -- 172.21.15.12:0/1883109176 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740991e0 con 0x55dfb3be5890 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f99617fa700 1 -- 172.21.15.12:0/1883109176 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f997409a4e0 con 0x55dfb3be5890 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f998bd87b80 1 -- 172.21.15.12:0/1883109176 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99740a24f0 con 0x55dfb3be5890 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f99617fa700 1 -- 172.21.15.12:0/1883109176 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99740993a0 con 0x55dfb3be5890 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f998bd87b80 10 client.31213 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.467+0000 7f998bd87b80 1 -- 172.21.15.12:0/1883109176 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55dfb3afb560 con 0x55dfb3be5890 2024-05-09T00:08:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.468+0000 7f99617fa700 1 -- 172.21.15.12:0/1883109176 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99740a24f0 con 0x55dfb3be5890 2024-05-09T00:08:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.468+0000 7f99617fa700 1 -- 172.21.15.12:0/1883109176 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f997409aa00 con 0x55dfb3be5890 2024-05-09T00:08:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.468+0000 7f998bd87b80 10 client.31213 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.468+0000 7f998bd87b80 10 client.31213 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:08:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.468+0000 7f998bd87b80 20 client.31213 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.468+0000 7f998bd87b80 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55dfb3c553d0 0x55dfb3c577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.468+0000 7f998bd87b80 4 client.31213 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:08:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.468+0000 7f998bd87b80 1 -- 172.21.15.12:0/1883109176 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dfb3b2dec0 con 0x55dfb3c553d0 2024-05-09T00:08:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.468+0000 7f997d168700 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55dfb3c553d0 0x55dfb3c577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.470+0000 7f997d168700 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55dfb3c553d0 0x55dfb3c577c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b361b95cdc2fec server_cookie=1c8e396b887efa84 in_seq=0 out_seq=0 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.470+0000 7f99617fa700 10 client.31213 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f99617fa700 1 -- 172.21.15.12:0/1883109176 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dfb3b2dec0 con 0x55dfb3c553d0 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f99617fa700 10 client.31213 handle_command_reply: tid=0 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 client.31213 shutdown 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 -- 172.21.15.12:0/1883109176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 msgr2=0x55dfb3bf6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3bf6360 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f997400e970 tx=0x7f997401f000 comp rx=0 tx=0).stop 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 -- 172.21.15.12:0/1883109176 shutdown_connections 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55dfb3c553d0 0x55dfb3c577c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb3beb720 0x55dfb3bf4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfb3be5890 0x55dfb3bf6360 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 --2- 172.21.15.12:0/1883109176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfb3beae00 0x55dfb3bf4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f99617fa700 0 client.31213 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:08:32.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 -- 172.21.15.12:0/1883109176 >> 172.21.15.12:0/1883109176 conn(0x55dfb3b43210 msgr2=0x55dfb3bed4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 -- 172.21.15.12:0/1883109176 shutdown_connections 2024-05-09T00:08:32.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 1 -- 172.21.15.12:0/1883109176 wait complete. 2024-05-09T00:08:32.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.472+0000 7f998bd87b80 20 client.31213 trim_cache size 0 max 16384 2024-05-09T00:08:32.472 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 -- 172.21.15.12:0/615822585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99640a1290 msgr2=0x7f99640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99640a1290 0x7f99640a3750 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f996800b2e0 tx=0x7f9968005fd0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 -- 172.21.15.12:0/615822585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9978069250 msgr2=0x7f9978199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9978069250 0x7f9978199600 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f997400e9a0 tx=0x7f997400ecb0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 -- 172.21.15.12:0/615822585 shutdown_connections 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f99640a1290 0x7f99640a3750 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f997810d940 0x7f997819dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9978069250 0x7f9978199600 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 --2- 172.21.15.12:0/615822585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9978069b70 0x7f9978199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 -- 172.21.15.12:0/615822585 >> 172.21.15.12:0/615822585 conn(0x7f9978076f30 msgr2=0x7f997810f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 -- 172.21.15.12:0/615822585 shutdown_connections 2024-05-09T00:08:32.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.473+0000 7f997fbcd700 1 -- 172.21.15.12:0/615822585 wait complete. 2024-05-09T00:08:32.485 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:08:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.704+0000 7fe88aa20700 1 -- 172.21.15.12:0/3097092974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe884100a00 msgr2=0x7fe884104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.704+0000 7fe88aa20700 1 --2- 172.21.15.12:0/3097092974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe884100a00 0x7fe884104a40 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fe878009990 tx=0x7fe878009ca0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.705+0000 7fe88aa20700 1 -- 172.21.15.12:0/3097092974 shutdown_connections 2024-05-09T00:08:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.705+0000 7fe88aa20700 1 --2- 172.21.15.12:0/3097092974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe884105170 0x7fe88410ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.705+0000 7fe88aa20700 1 --2- 172.21.15.12:0/3097092974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe884100050 0x7fe884100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.705+0000 7fe88aa20700 1 --2- 172.21.15.12:0/3097092974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe884100a00 0x7fe884104a40 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.705+0000 7fe88aa20700 1 -- 172.21.15.12:0/3097092974 >> 172.21.15.12:0/3097092974 conn(0x7fe884076fa0 msgr2=0x7fe8840773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.705+0000 7fe88aa20700 1 -- 172.21.15.12:0/3097092974 shutdown_connections 2024-05-09T00:08:32.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.705+0000 7fe88aa20700 1 -- 172.21.15.12:0/3097092974 wait complete. 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.705+0000 7fe88aa20700 1 Processor -- start 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe88aa20700 1 -- start start 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe88aa20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe884100050 0x7fe8841982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe88aa20700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe884105170 0x7fe884198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe88aa20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe884198d60 0x7fe8841a3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe88aa20700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8841a3a00 con 0x7fe884198d60 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe88aa20700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8841a3b70 con 0x7fe884100050 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe88aa20700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8841a3d30 con 0x7fe884105170 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe888fbd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe884198d60 0x7fe8841a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe883fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe884100050 0x7fe8841982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe888fbd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe884198d60 0x7fe8841a3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48332/0 (socket says 172.21.15.12:48332) 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe883fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe884100050 0x7fe8841982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54530/0 (socket says 172.21.15.12:54530) 2024-05-09T00:08:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe888fbd700 1 -- 172.21.15.12:0/2572699191 learned_addr learned my addr 172.21.15.12:0/2572699191 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.706+0000 7fe8837fe700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe884105170 0x7fe884198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe888fbd700 1 -- 172.21.15.12:0/2572699191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe884105170 msgr2=0x7fe884198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe888fbd700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe884105170 0x7fe884198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe888fbd700 1 -- 172.21.15.12:0/2572699191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe884100050 msgr2=0x7fe8841982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe888fbd700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe884100050 0x7fe8841982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe888fbd700 1 -- 172.21.15.12:0/2572699191 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8780095f0 con 0x7fe884198d60 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe883fff700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe884100050 0x7fe8841982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe888fbd700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe884198d60 0x7fe8841a3360 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00ef30 tx=0x7fe87c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe8817fa700 1 -- 172.21.15.12:0/2572699191 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c004bb0 con 0x7fe884198d60 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe8817fa700 1 -- 172.21.15.12:0/2572699191 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe87c00f040 con 0x7fe884198d60 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe88aa20700 1 -- 172.21.15.12:0/2572699191 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe884196130 con 0x7fe884198d60 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe8817fa700 1 -- 172.21.15.12:0/2572699191 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c004020 con 0x7fe884198d60 2024-05-09T00:08:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.707+0000 7fe88aa20700 1 -- 172.21.15.12:0/2572699191 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe884196640 con 0x7fe884198d60 2024-05-09T00:08:32.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.709+0000 7fe8817fa700 1 -- 172.21.15.12:0/2572699191 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe87c003a00 con 0x7fe884198d60 2024-05-09T00:08:32.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.710+0000 7fe8817fa700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe86c0a1310 0x7fe86c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.710+0000 7fe8817fa700 1 -- 172.21.15.12:0/2572699191 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe87c0d03d0 con 0x7fe884198d60 2024-05-09T00:08:32.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.710+0000 7fe883fff700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe86c0a1310 0x7fe86c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.711+0000 7fe883fff700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe86c0a1310 0x7fe86c0a37d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe874009de0 tx=0x7fe874009190 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.711+0000 7fe896bdab80 1 Processor -- start 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.711+0000 7fe896bdab80 1 -- start start 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06abe00 0x55c6f06ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 0x55c6f06a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06a6890 0x55c6f06a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe896bdab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f05fd150 con 0x55c6f06abe00 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe896bdab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f05fab10 con 0x55c6f06ac720 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe896bdab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f05f6400 con 0x55c6f06a6890 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe883fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06abe00 0x55c6f06ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe883fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06abe00 0x55c6f06ac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48344/0 (socket says 172.21.15.12:48344) 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe888fbd700 1 --2- 172.21.15.12:0/1642918654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06a6890 0x55c6f06a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe883fff700 1 -- 172.21.15.12:0/1642918654 learned_addr learned my addr 172.21.15.12:0/1642918654 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe8837fe700 1 --2- 172.21.15.12:0/1642918654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 0x55c6f06a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe888fbd700 1 -- 172.21.15.12:0/1642918654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 msgr2=0x55c6f06a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe888fbd700 1 --2- 172.21.15.12:0/1642918654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 0x55c6f06a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe888fbd700 1 -- 172.21.15.12:0/1642918654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06abe00 msgr2=0x55c6f06ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe888fbd700 1 --2- 172.21.15.12:0/1642918654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06abe00 0x55c6f06ac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe888fbd700 1 -- 172.21.15.12:0/1642918654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6f04773b0 con 0x55c6f06a6890 2024-05-09T00:08:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.712+0000 7fe883fff700 1 --2- 172.21.15.12:0/1642918654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06abe00 0x55c6f06ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.713+0000 7fe888fbd700 1 --2- 172.21.15.12:0/1642918654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06a6890 0x55c6f06a6cd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0cdc70 tx=0x7fe87c099aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1eab716c98d1976 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.713+0000 7fe86a7fc700 1 -- 172.21.15.12:0/1642918654 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c09dd30 con 0x55c6f06a6890 2024-05-09T00:08:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.713+0000 7fe86a7fc700 1 -- 172.21.15.12:0/1642918654 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe87c09d630 con 0x55c6f06a6890 2024-05-09T00:08:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.713+0000 7fe86a7fc700 1 -- 172.21.15.12:0/1642918654 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c099d40 con 0x55c6f06a6890 2024-05-09T00:08:32.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.714+0000 7fe896bdab80 1 -- 172.21.15.12:0/1642918654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06a6890 msgr2=0x55c6f06a6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.714+0000 7fe896bdab80 1 --2- 172.21.15.12:0/1642918654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06a6890 0x55c6f06a6cd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0cdc70 tx=0x7fe87c099aa0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.714+0000 7fe896bdab80 1 -- 172.21.15.12:0/1642918654 shutdown_connections 2024-05-09T00:08:32.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.714+0000 7fe896bdab80 1 --2- 172.21.15.12:0/1642918654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06a6890 0x55c6f06a6cd0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.714+0000 7fe896bdab80 1 --2- 172.21.15.12:0/1642918654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06abe00 0x55c6f06ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.714+0000 7fe896bdab80 1 --2- 172.21.15.12:0/1642918654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 0x55c6f06a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.714+0000 7fe896bdab80 1 -- 172.21.15.12:0/1642918654 >> 172.21.15.12:0/1642918654 conn(0x55c6f0604210 msgr2=0x55c6f06a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.714+0000 7fe896bdab80 1 -- 172.21.15.12:0/1642918654 shutdown_connections 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.714+0000 7fe896bdab80 1 -- 172.21.15.12:0/1642918654 wait complete. 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.714+0000 7fe896bdab80 1 -- start start 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe896bdab80 1 Processor -- start 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06a6890 0x55c6f06afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 0x55c6f06b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06ac720 0x55c6f06b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe896bdab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f0592560 con 0x55c6f06ac720 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe896bdab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f0592200 con 0x55c6f06a6890 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe896bdab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f04796a0 con 0x55c6f06abe00 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe896bdab80 10 client.0 _ll_register_callbacks cb 0x55c6f059bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe888fbd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06ac720 0x55c6f06b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe883fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06a6890 0x55c6f06afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe8837fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 0x55c6f06b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe888fbd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06ac720 0x55c6f06b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48354/0 (socket says 172.21.15.12:48354) 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.715+0000 7fe888fbd700 1 -- 172.21.15.12:0/2925449087 learned_addr learned my addr 172.21.15.12:0/2925449087 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe8837fe700 1 -- 172.21.15.12:0/2925449087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06a6890 msgr2=0x55c6f06afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe8837fe700 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06a6890 0x55c6f06afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe8837fe700 1 -- 172.21.15.12:0/2925449087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06ac720 msgr2=0x55c6f06b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe8837fe700 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06ac720 0x55c6f06b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe8837fe700 1 -- 172.21.15.12:0/2925449087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87400eb10 con 0x55c6f06abe00 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe883fff700 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06a6890 0x55c6f06afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe8837fe700 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 0x55c6f06b0030 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe8780059d0 tx=0x7fe878005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52a46c0706d02a64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe8637fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe8637fe700 1 -- 172.21.15.12:0/2925449087 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87801d070 con 0x55c6f06abe00 2024-05-09T00:08:32.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe8637fe700 1 -- 172.21.15.12:0/2925449087 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe878004500 con 0x55c6f06abe00 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe896bdab80 1 -- 172.21.15.12:0/2925449087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6f054f270 con 0x55c6f06abe00 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe8637fe700 1 -- 172.21.15.12:0/2925449087 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8780046c0 con 0x55c6f06abe00 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe896bdab80 10 client.31222 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.716+0000 7fe896bdab80 1 -- 172.21.15.12:0/2925449087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7fe884196640 con 0x55c6f06abe00 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe8637fe700 1 -- 172.21.15.12:0/2925449087 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6f054f270 con 0x55c6f06abe00 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe8637fe700 1 -- 172.21.15.12:0/2925449087 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe87800f860 con 0x55c6f06abe00 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe896bdab80 10 client.31222 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe896bdab80 10 client.31222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe896bdab80 20 client.31222 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe896bdab80 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c6f06c0290 0x55c6f06c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe896bdab80 4 client.31222 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe896bdab80 1 -- 172.21.15.12:0/2925449087 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6f0675810 con 0x55c6f06c0290 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe883fff700 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c6f06c0290 0x55c6f06c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe883fff700 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c6f06c0290 0x55c6f06c26b0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b68e354f8a8901d server_cookie=b0b4bb1c10002c8a in_seq=0 out_seq=0 2024-05-09T00:08:32.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.717+0000 7fe8637fe700 10 client.31222 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.718+0000 7fe8637fe700 1 -- 172.21.15.12:0/2925449087 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6f0675810 con 0x55c6f06c0290 2024-05-09T00:08:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.718+0000 7fe8637fe700 10 client.31222 handle_command_reply: tid=0 2024-05-09T00:08:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.718+0000 7fe896bdab80 1 client.31222 shutdown 2024-05-09T00:08:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 1 -- 172.21.15.12:0/2925449087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 msgr2=0x55c6f06b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 0x55c6f06b0030 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe8780059d0 tx=0x7fe878005800 comp rx=0 tx=0).stop 2024-05-09T00:08:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 1 -- 172.21.15.12:0/2925449087 shutdown_connections 2024-05-09T00:08:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06ac720 0x55c6f06b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c6f06c0290 0x55c6f06c26b0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06a6890 0x55c6f06afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 1 --2- 172.21.15.12:0/2925449087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 0x55c6f06b0030 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe8637fe700 0 client.31222 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 1 -- 172.21.15.12:0/2925449087 >> 172.21.15.12:0/2925449087 conn(0x55c6f0604210 msgr2=0x55c6f06a05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 1 -- 172.21.15.12:0/2925449087 shutdown_connections 2024-05-09T00:08:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 1 -- 172.21.15.12:0/2925449087 wait complete. 2024-05-09T00:08:32.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.719+0000 7fe896bdab80 20 client.31222 trim_cache size 0 max 16384 2024-05-09T00:08:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe896bdab80 1 Processor -- start 2024-05-09T00:08:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe896bdab80 1 -- start start 2024-05-09T00:08:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06abe00 0x55c6f06ab460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06ac720 0x55c6f06a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe896bdab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f06a18e0 con 0x55c6f06a6890 2024-05-09T00:08:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe896bdab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f06bcdc0 con 0x55c6f06abe00 2024-05-09T00:08:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe896bdab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f06b1160 con 0x55c6f06ac720 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe883fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe883fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48370/0 (socket says 172.21.15.12:48370) 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe888fbd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06ac720 0x55c6f06a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe883fff700 1 -- 172.21.15.12:0/793122156 learned_addr learned my addr 172.21.15.12:0/793122156 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.735+0000 7fe8837fe700 1 --2- 172.21.15.12:0/793122156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06abe00 0x55c6f06ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.736+0000 7fe888fbd700 1 -- 172.21.15.12:0/793122156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06abe00 msgr2=0x55c6f06ab460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.736+0000 7fe888fbd700 1 --2- 172.21.15.12:0/793122156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06abe00 0x55c6f06ab460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.736+0000 7fe888fbd700 1 -- 172.21.15.12:0/793122156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 msgr2=0x55c6f06a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.736+0000 7fe888fbd700 1 --2- 172.21.15.12:0/793122156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.736+0000 7fe888fbd700 1 -- 172.21.15.12:0/793122156 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6f0423b60 con 0x55c6f06ac720 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.736+0000 7fe883fff700 1 --2- 172.21.15.12:0/793122156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.736+0000 7fe888fbd700 1 --2- 172.21.15.12:0/793122156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06ac720 0x55c6f06a0d20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00d010 tx=0x7fe87c088040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8fe414b74eb3cdc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.736+0000 7fe86affd700 1 -- 172.21.15.12:0/793122156 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c09dd30 con 0x55c6f06ac720 2024-05-09T00:08:32.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.736+0000 7fe86affd700 1 -- 172.21.15.12:0/793122156 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe87c0a1460 con 0x55c6f06ac720 2024-05-09T00:08:32.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.736+0000 7fe86affd700 1 -- 172.21.15.12:0/793122156 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c099d40 con 0x55c6f06ac720 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.737+0000 7fe896bdab80 1 -- 172.21.15.12:0/793122156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06ac720 msgr2=0x55c6f06a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.737+0000 7fe896bdab80 1 --2- 172.21.15.12:0/793122156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06ac720 0x55c6f06a0d20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00d010 tx=0x7fe87c088040 comp rx=0 tx=0).stop 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.737+0000 7fe896bdab80 1 -- 172.21.15.12:0/793122156 shutdown_connections 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.737+0000 7fe896bdab80 1 --2- 172.21.15.12:0/793122156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06ac720 0x55c6f06a0d20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.737+0000 7fe896bdab80 1 --2- 172.21.15.12:0/793122156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.737+0000 7fe896bdab80 1 --2- 172.21.15.12:0/793122156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06abe00 0x55c6f06ab460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.737+0000 7fe896bdab80 1 -- 172.21.15.12:0/793122156 >> 172.21.15.12:0/793122156 conn(0x55c6f0604210 msgr2=0x55c6f06ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.737+0000 7fe896bdab80 1 -- 172.21.15.12:0/793122156 shutdown_connections 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.737+0000 7fe896bdab80 1 -- 172.21.15.12:0/793122156 wait complete. 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.737+0000 7fe896bdab80 1 -- start start 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe896bdab80 1 Processor -- start 2024-05-09T00:08:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 0x55c6f06b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe896bdab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 0x55c6f06b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe896bdab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f06b5960 con 0x55c6f06a6890 2024-05-09T00:08:32.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe896bdab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f0715790 con 0x55c6f06ac720 2024-05-09T00:08:32.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe896bdab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f07158d0 con 0x55c6f06abe00 2024-05-09T00:08:32.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe896bdab80 10 client.0 _ll_register_callbacks cb 0x55c6f059bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:32.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe888fbd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 0x55c6f06b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe8837fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 0x55c6f06b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe883fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe8837fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 0x55c6f06b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42356/0 (socket says 172.21.15.12:42356) 2024-05-09T00:08:32.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe888fbd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 0x55c6f06b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54556/0 (socket says 172.21.15.12:54556) 2024-05-09T00:08:32.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.738+0000 7fe8837fe700 1 -- 172.21.15.12:0/3582746154 learned_addr learned my addr 172.21.15.12:0/3582746154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe888fbd700 1 -- 172.21.15.12:0/3582746154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 msgr2=0x55c6f06b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe888fbd700 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 0x55c6f06b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe888fbd700 1 -- 172.21.15.12:0/3582746154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 msgr2=0x55c6f06b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe888fbd700 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06b76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe888fbd700 1 -- 172.21.15.12:0/3582746154 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87401a040 con 0x55c6f06ac720 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe883fff700 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe888fbd700 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 0x55c6f06b5280 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0090a0 tx=0x7fe87c022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=291e6428700a5aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe8637fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe8637fe700 1 -- 172.21.15.12:0/3582746154 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0a5ab0 con 0x55c6f06ac720 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe896bdab80 1 -- 172.21.15.12:0/3582746154 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe87c09d630 con 0x55c6f06ac720 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe8637fe700 1 -- 172.21.15.12:0/3582746154 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe87c09dd30 con 0x55c6f06ac720 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe8637fe700 1 -- 172.21.15.12:0/3582746154 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe87c0a1120 con 0x55c6f06ac720 2024-05-09T00:08:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe896bdab80 10 client.30977 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.739+0000 7fe896bdab80 1 -- 172.21.15.12:0/3582746154 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55c6f04773b0 con 0x55c6f06ac720 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.740+0000 7fe8637fe700 1 -- 172.21.15.12:0/3582746154 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe87c09d630 con 0x55c6f06ac720 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.740+0000 7fe8637fe700 1 -- 172.21.15.12:0/3582746154 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe87c0cf070 con 0x55c6f06ac720 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.740+0000 7fe896bdab80 10 client.30977 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.740+0000 7fe896bdab80 10 client.30977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.740+0000 7fe896bdab80 20 client.30977 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.740+0000 7fe896bdab80 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c6f0716b50 0x55c6f0718f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.740+0000 7fe896bdab80 4 client.30977 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.740+0000 7fe896bdab80 1 -- 172.21.15.12:0/3582746154 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6f05eeec0 con 0x55c6f0716b50 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.740+0000 7fe883fff700 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c6f0716b50 0x55c6f0718f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe8637fe700 10 client.30977 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe883fff700 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c6f0716b50 0x55c6f0718f40 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcd54bc7bb720eb3 server_cookie=40513b78a8eb22e7 in_seq=0 out_seq=0 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe8637fe700 1 -- 172.21.15.12:0/3582746154 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6f05eeec0 con 0x55c6f0716b50 2024-05-09T00:08:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe8637fe700 10 client.30977 handle_command_reply: tid=0 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 client.30977 shutdown 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 -- 172.21.15.12:0/3582746154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 msgr2=0x55c6f06b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 0x55c6f06b5280 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0090a0 tx=0x7fe87c022000 comp rx=0 tx=0).stop 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 -- 172.21.15.12:0/3582746154 shutdown_connections 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c6f0716b50 0x55c6f0718f40 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6f06ac720 0x55c6f06b5280 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe8637fe700 0 client.30977 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6f06a6890 0x55c6f06b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 --2- 172.21.15.12:0/3582746154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6f06abe00 0x55c6f06b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 -- 172.21.15.12:0/3582746154 >> 172.21.15.12:0/3582746154 conn(0x55c6f0604210 msgr2=0x55c6f06ab2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 -- 172.21.15.12:0/3582746154 shutdown_connections 2024-05-09T00:08:32.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.741+0000 7fe896bdab80 1 -- 172.21.15.12:0/3582746154 wait complete. 2024-05-09T00:08:32.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.743+0000 7fe896bdab80 20 client.30977 trim_cache size 0 max 16384 2024-05-09T00:08:32.745 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:08:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 -- 172.21.15.12:0/2572699191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe86c0a1310 msgr2=0x7fe86c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe86c0a1310 0x7fe86c0a37d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe874009de0 tx=0x7fe874009190 comp rx=0 tx=0).stop 2024-05-09T00:08:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 -- 172.21.15.12:0/2572699191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe884198d60 msgr2=0x7fe8841a3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe884198d60 0x7fe8841a3360 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00ef30 tx=0x7fe87c00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 -- 172.21.15.12:0/2572699191 shutdown_connections 2024-05-09T00:08:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe86c0a1310 0x7fe86c0a37d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe884198d60 0x7fe8841a3360 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe884100050 0x7fe8841982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 --2- 172.21.15.12:0/2572699191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe884105170 0x7fe884198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 -- 172.21.15.12:0/2572699191 >> 172.21.15.12:0/2572699191 conn(0x7fe884076fa0 msgr2=0x7fe8840fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 -- 172.21.15.12:0/2572699191 shutdown_connections 2024-05-09T00:08:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:32.744+0000 7fe88aa20700 1 -- 172.21.15.12:0/2572699191 wait complete. 2024-05-09T00:08:33.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:32 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/4259178774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:33.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:32 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/4259178774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:33.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:32 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/4259178774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:33.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 307060 30.58 MB/sec execute 388 sec latency 93.091 ms 2024-05-09T00:08:33.758 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:08:33.758 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:08:33.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.947+0000 7f376e2b1700 1 -- 172.21.15.12:0/1225166034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37681062a0 msgr2=0x7f376810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:33.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.947+0000 7f376e2b1700 1 --2- 172.21.15.12:0/1225166034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37681062a0 0x7f376810de20 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f3760009f90 tx=0x7f376000b330 comp rx=0 tx=0).stop 2024-05-09T00:08:33.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.947+0000 7f376e2b1700 1 -- 172.21.15.12:0/1225166034 shutdown_connections 2024-05-09T00:08:33.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.947+0000 7f376e2b1700 1 --2- 172.21.15.12:0/1225166034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37681062a0 0x7f376810de20 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.947+0000 7f376e2b1700 1 --2- 172.21.15.12:0/1225166034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3768101180 0x7f3768101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.947+0000 7f376e2b1700 1 --2- 172.21.15.12:0/1225166034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3768101b30 0x7f3768105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.947+0000 7f376e2b1700 1 -- 172.21.15.12:0/1225166034 >> 172.21.15.12:0/1225166034 conn(0x7f37680fc9e0 msgr2=0x7f37680fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:33.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.948+0000 7f376e2b1700 1 -- 172.21.15.12:0/1225166034 shutdown_connections 2024-05-09T00:08:33.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.948+0000 7f376e2b1700 1 -- 172.21.15.12:0/1225166034 wait complete. 2024-05-09T00:08:33.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.949+0000 7f376e2b1700 1 Processor -- start 2024-05-09T00:08:33.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.949+0000 7f376e2b1700 1 -- start start 2024-05-09T00:08:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f376e2b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37681062a0 0x7f37681106f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f376e2b1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3768110c30 0x7f3768114fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f376e2b1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37681154e0 0x7f3768115960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f376e2b1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37681112a0 con 0x7f37681062a0 2024-05-09T00:08:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f376e2b1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37681113e0 con 0x7f3768110c30 2024-05-09T00:08:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f376e2b1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37681b2fc0 con 0x7f37681154e0 2024-05-09T00:08:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f37677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3768110c30 0x7f3768114fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f37677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3768110c30 0x7f3768114fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54574/0 (socket says 172.21.15.12:54574) 2024-05-09T00:08:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f37677fe700 1 -- 172.21.15.12:0/515115435 learned_addr learned my addr 172.21.15.12:0/515115435 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f3767fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37681062a0 0x7f37681106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.950+0000 7f376c84e700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37681154e0 0x7f3768115960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f37677fe700 1 -- 172.21.15.12:0/515115435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37681154e0 msgr2=0x7f3768115960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f37677fe700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37681154e0 0x7f3768115960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f37677fe700 1 -- 172.21.15.12:0/515115435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37681062a0 msgr2=0x7f37681106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:33.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f37677fe700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37681062a0 0x7f37681106f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f37677fe700 1 -- 172.21.15.12:0/515115435 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3760009c40 con 0x7f3768110c30 2024-05-09T00:08:33.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f376c84e700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37681154e0 0x7f3768115960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:33.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f37677fe700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3768110c30 0x7f3768114fa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f375c00e970 tx=0x7f375c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f37657fa700 1 -- 172.21.15.12:0/515115435 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c009790 con 0x7f3768110c30 2024-05-09T00:08:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f37657fa700 1 -- 172.21.15.12:0/515115435 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f375c00c7c0 con 0x7f3768110c30 2024-05-09T00:08:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f376e2b1700 1 -- 172.21.15.12:0/515115435 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37681b32a0 con 0x7f3768110c30 2024-05-09T00:08:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f37657fa700 1 -- 172.21.15.12:0/515115435 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c0054b0 con 0x7f3768110c30 2024-05-09T00:08:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.951+0000 7f376e2b1700 1 -- 172.21.15.12:0/515115435 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37681b3800 con 0x7f3768110c30 2024-05-09T00:08:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.954+0000 7f37657fa700 1 -- 172.21.15.12:0/515115435 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f375c005630 con 0x7f3768110c30 2024-05-09T00:08:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.955+0000 7f37657fa700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37500a1300 0x7f37500a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.955+0000 7f37657fa700 1 -- 172.21.15.12:0/515115435 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f375c014070 con 0x7f3768110c30 2024-05-09T00:08:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.955+0000 7f3767fff700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37500a1300 0x7f37500a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.956+0000 7f3767fff700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37500a1300 0x7f37500a37c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f37681023a0 tx=0x7f3758009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:33.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f377a46bb80 1 Processor -- start 2024-05-09T00:08:33.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f377a46bb80 1 -- start start 2024-05-09T00:08:33.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca752e80 0x55e4ca753260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca74d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f377a46bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca6a41d0 con 0x55e4ca74d910 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f377a46bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca6a1b90 con 0x55e4ca7537a0 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f377a46bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca69d480 con 0x55e4ca752e80 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f376c84e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f3767fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca752e80 0x55e4ca753260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f376c84e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48390/0 (socket says 172.21.15.12:48390) 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f376c84e700 1 -- 172.21.15.12:0/4149198020 learned_addr learned my addr 172.21.15.12:0/4149198020 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f37677fe700 1 --2- 172.21.15.12:0/4149198020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca74d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f376c84e700 1 -- 172.21.15.12:0/4149198020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca752e80 msgr2=0x55e4ca753260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f376c84e700 1 --2- 172.21.15.12:0/4149198020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca752e80 0x55e4ca753260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f376c84e700 1 -- 172.21.15.12:0/4149198020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 msgr2=0x55e4ca74d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f376c84e700 1 --2- 172.21.15.12:0/4149198020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca74d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.957+0000 7f376c84e700 1 -- 172.21.15.12:0/4149198020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ca51e430 con 0x55e4ca74d910 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.958+0000 7f376c84e700 1 --2- 172.21.15.12:0/4149198020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74dd50 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f376000be20 tx=0x7f3760009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc25c4c836a049a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.958+0000 7f374e7fc700 1 -- 172.21.15.12:0/4149198020 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376000e040 con 0x55e4ca74d910 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.958+0000 7f374e7fc700 1 -- 172.21.15.12:0/4149198020 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37600093e0 con 0x55e4ca74d910 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.958+0000 7f374e7fc700 1 -- 172.21.15.12:0/4149198020 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760005040 con 0x55e4ca74d910 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.959+0000 7f377a46bb80 1 -- 172.21.15.12:0/4149198020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 msgr2=0x55e4ca74dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.959+0000 7f377a46bb80 1 --2- 172.21.15.12:0/4149198020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74dd50 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f376000be20 tx=0x7f3760009660 comp rx=0 tx=0).stop 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.959+0000 7f377a46bb80 1 -- 172.21.15.12:0/4149198020 shutdown_connections 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.959+0000 7f377a46bb80 1 --2- 172.21.15.12:0/4149198020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74dd50 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.959+0000 7f377a46bb80 1 --2- 172.21.15.12:0/4149198020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca752e80 0x55e4ca753260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.959+0000 7f377a46bb80 1 --2- 172.21.15.12:0/4149198020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca74d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.959+0000 7f377a46bb80 1 -- 172.21.15.12:0/4149198020 >> 172.21.15.12:0/4149198020 conn(0x55e4ca6ab290 msgr2=0x55e4ca747e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.959+0000 7f377a46bb80 1 -- 172.21.15.12:0/4149198020 shutdown_connections 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.959+0000 7f377a46bb80 1 -- 172.21.15.12:0/4149198020 wait complete. 2024-05-09T00:08:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.959+0000 7f377a46bb80 1 -- start start 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f377a46bb80 1 Processor -- start 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 0x55e4ca759860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca752e80 0x55e4ca756b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca7537a0 0x55e4ca757290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f377a46bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca6395e0 con 0x55e4ca7537a0 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f377a46bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca639280 con 0x55e4ca752e80 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f377a46bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca520720 con 0x55e4ca74d910 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f377a46bb80 10 client.0 _ll_register_callbacks cb 0x55e4ca642bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f37677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca752e80 0x55e4ca756b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f37677fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca752e80 0x55e4ca756b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54592/0 (socket says 172.21.15.12:54592) 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f37677fe700 1 -- 172.21.15.12:0/2666924028 learned_addr learned my addr 172.21.15.12:0/2666924028 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f376c84e700 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca7537a0 0x55e4ca757290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.960+0000 7f3767fff700 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 0x55e4ca759860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f37677fe700 1 -- 172.21.15.12:0/2666924028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 msgr2=0x55e4ca759860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f37677fe700 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 0x55e4ca759860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f37677fe700 1 -- 172.21.15.12:0/2666924028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca7537a0 msgr2=0x55e4ca757290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f37677fe700 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca7537a0 0x55e4ca757290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f37677fe700 1 -- 172.21.15.12:0/2666924028 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376810e690 con 0x55e4ca752e80 2024-05-09T00:08:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f37677fe700 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca752e80 0x55e4ca756b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f375c00c5b0 tx=0x7f375c09a020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b3b8a2d76ebe94c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f37477fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:08:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f37477fe700 1 -- 172.21.15.12:0/2666924028 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c096640 con 0x55e4ca752e80 2024-05-09T00:08:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f37477fe700 1 -- 172.21.15.12:0/2666924028 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f375c0b1030 con 0x55e4ca752e80 2024-05-09T00:08:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f377a46bb80 1 -- 172.21.15.12:0/2666924028 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4ca5f62f0 con 0x55e4ca752e80 2024-05-09T00:08:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.961+0000 7f37477fe700 1 -- 172.21.15.12:0/2666924028 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f375c07c030 con 0x55e4ca752e80 2024-05-09T00:08:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.962+0000 7f377a46bb80 10 client.30989 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.962+0000 7f377a46bb80 1 -- 172.21.15.12:0/2666924028 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e4ca4cabe0 con 0x55e4ca752e80 2024-05-09T00:08:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.962+0000 7f37477fe700 1 -- 172.21.15.12:0/2666924028 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4ca5f62f0 con 0x55e4ca752e80 2024-05-09T00:08:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.962+0000 7f37477fe700 1 -- 172.21.15.12:0/2666924028 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f375c099420 con 0x55e4ca752e80 2024-05-09T00:08:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.962+0000 7f377a46bb80 10 client.30989 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:33.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.962+0000 7f377a46bb80 10 client.30989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:33.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.962+0000 7f377a46bb80 20 client.30989 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:33.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.962+0000 7f377a46bb80 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e4ca767200 0x55e4ca769620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.962+0000 7f377a46bb80 4 client.30989 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:33.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.962+0000 7f377a46bb80 1 -- 172.21.15.12:0/2666924028 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4ca71c890 con 0x55e4ca767200 2024-05-09T00:08:33.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.964+0000 7f3767fff700 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e4ca767200 0x55e4ca769620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.964+0000 7f3767fff700 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e4ca767200 0x55e4ca769620 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a5114aaf942dc3 server_cookie=bdfc706a1881ce5b in_seq=0 out_seq=0 2024-05-09T00:08:33.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.964+0000 7f37477fe700 10 client.30989 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.968+0000 7f37477fe700 1 -- 172.21.15.12:0/2666924028 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e4ca71c890 con 0x55e4ca767200 2024-05-09T00:08:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.968+0000 7f37477fe700 10 client.30989 handle_command_reply: tid=0 2024-05-09T00:08:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.968+0000 7f377a46bb80 1 client.30989 shutdown 2024-05-09T00:08:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 1 -- 172.21.15.12:0/2666924028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca752e80 msgr2=0x55e4ca756b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca752e80 0x55e4ca756b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f375c00c5b0 tx=0x7f375c09a020 comp rx=0 tx=0).stop 2024-05-09T00:08:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 1 -- 172.21.15.12:0/2666924028 shutdown_connections 2024-05-09T00:08:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca7537a0 0x55e4ca757290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 0x55e4ca759860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e4ca767200 0x55e4ca769620 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 1 --2- 172.21.15.12:0/2666924028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca752e80 0x55e4ca756b50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 1 -- 172.21.15.12:0/2666924028 >> 172.21.15.12:0/2666924028 conn(0x55e4ca6ab290 msgr2=0x55e4ca747620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f37477fe700 0 client.30989 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 1 -- 172.21.15.12:0/2666924028 shutdown_connections 2024-05-09T00:08:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 1 -- 172.21.15.12:0/2666924028 wait complete. 2024-05-09T00:08:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.969+0000 7f377a46bb80 20 client.30989 trim_cache size 0 max 16384 2024-05-09T00:08:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.994+0000 7f377a46bb80 1 Processor -- start 2024-05-09T00:08:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.994+0000 7f377a46bb80 1 -- start start 2024-05-09T00:08:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca752e80 0x55e4ca756af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca747da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f377a46bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca7648b0 con 0x55e4ca74d910 2024-05-09T00:08:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f377a46bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca758150 con 0x55e4ca7537a0 2024-05-09T00:08:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f377a46bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca757ad0 con 0x55e4ca752e80 2024-05-09T00:08:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f3767fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f3767fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48404/0 (socket says 172.21.15.12:48404) 2024-05-09T00:08:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f3767fff700 1 -- 172.21.15.12:0/2008767319 learned_addr learned my addr 172.21.15.12:0/2008767319 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f376c84e700 1 --2- 172.21.15.12:0/2008767319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca747da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f37677fe700 1 --2- 172.21.15.12:0/2008767319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca752e80 0x55e4ca756af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f3767fff700 1 -- 172.21.15.12:0/2008767319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca752e80 msgr2=0x55e4ca756af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f3767fff700 1 --2- 172.21.15.12:0/2008767319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca752e80 0x55e4ca756af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f3767fff700 1 -- 172.21.15.12:0/2008767319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 msgr2=0x55e4ca747da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f3767fff700 1 --2- 172.21.15.12:0/2008767319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca747da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f3767fff700 1 -- 172.21.15.12:0/2008767319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ca744560 con 0x55e4ca74d910 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.995+0000 7f3767fff700 1 --2- 172.21.15.12:0/2008767319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74d3d0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x55e4ca769f50 tx=0x7f3758014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd28498b8b4b182a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.996+0000 7f374effd700 1 -- 172.21.15.12:0/2008767319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758024040 con 0x55e4ca74d910 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.996+0000 7f374effd700 1 -- 172.21.15.12:0/2008767319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3758011b10 con 0x55e4ca74d910 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.996+0000 7f374effd700 1 -- 172.21.15.12:0/2008767319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375801e680 con 0x55e4ca74d910 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.997+0000 7f377a46bb80 1 -- 172.21.15.12:0/2008767319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 msgr2=0x55e4ca74d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.997+0000 7f377a46bb80 1 --2- 172.21.15.12:0/2008767319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74d3d0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x55e4ca769f50 tx=0x7f3758014000 comp rx=0 tx=0).stop 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.997+0000 7f377a46bb80 1 -- 172.21.15.12:0/2008767319 shutdown_connections 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.997+0000 7f377a46bb80 1 --2- 172.21.15.12:0/2008767319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca747da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.997+0000 7f377a46bb80 1 --2- 172.21.15.12:0/2008767319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca74d910 0x55e4ca74d3d0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.997+0000 7f377a46bb80 1 --2- 172.21.15.12:0/2008767319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca752e80 0x55e4ca756af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.997+0000 7f377a46bb80 1 -- 172.21.15.12:0/2008767319 >> 172.21.15.12:0/2008767319 conn(0x55e4ca6ab290 msgr2=0x55e4ca7563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.997+0000 7f377a46bb80 1 -- 172.21.15.12:0/2008767319 shutdown_connections 2024-05-09T00:08:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.997+0000 7f377a46bb80 1 -- 172.21.15.12:0/2008767319 wait complete. 2024-05-09T00:08:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.997+0000 7f377a46bb80 1 -- start start 2024-05-09T00:08:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.998+0000 7f377a46bb80 1 Processor -- start 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.998+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 0x55e4ca7b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.998+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca752e80 0x55e4ca7b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.998+0000 7f377a46bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca75b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.998+0000 7f377a46bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca75bac0 con 0x55e4ca752e80 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.998+0000 7f377a46bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca75bc30 con 0x55e4ca7537a0 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.998+0000 7f377a46bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ca75bdf0 con 0x55e4ca74d910 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.998+0000 7f377a46bb80 10 client.0 _ll_register_callbacks cb 0x55e4ca642bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.998+0000 7f3767fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 0x55e4ca7b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f3767fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 0x55e4ca7b6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42412/0 (socket says 172.21.15.12:42412) 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f3767fff700 1 -- 172.21.15.12:0/3719001173 learned_addr learned my addr 172.21.15.12:0/3719001173 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f37677fe700 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca752e80 0x55e4ca7b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f3767fff700 1 -- 172.21.15.12:0/3719001173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 msgr2=0x55e4ca75b3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f3767fff700 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca75b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f3767fff700 1 -- 172.21.15.12:0/3719001173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca752e80 msgr2=0x55e4ca7b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f3767fff700 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca752e80 0x55e4ca7b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f3767fff700 1 -- 172.21.15.12:0/3719001173 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3758017070 con 0x55e4ca74d910 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f37677fe700 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca752e80 0x55e4ca7b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f3767fff700 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 0x55e4ca7b6a90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f375800a010 tx=0x7f3758005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f45ea25911913b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f37477fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f37477fe700 1 -- 172.21.15.12:0/3719001173 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758024050 con 0x55e4ca74d910 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:33.999+0000 7f37477fe700 1 -- 172.21.15.12:0/3719001173 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f37580295a0 con 0x55e4ca74d910 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f377a46bb80 1 -- 172.21.15.12:0/3719001173 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37600093e0 con 0x55e4ca74d910 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f37477fe700 1 -- 172.21.15.12:0/3719001173 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3758029760 con 0x55e4ca74d910 2024-05-09T00:08:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f377a46bb80 10 client.31246 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f377a46bb80 1 -- 172.21.15.12:0/3719001173 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e4ca6635e0 con 0x55e4ca74d910 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f37477fe700 1 -- 172.21.15.12:0/3719001173 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37600093e0 con 0x55e4ca74d910 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f37477fe700 1 -- 172.21.15.12:0/3719001173 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3758004930 con 0x55e4ca74d910 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f377a46bb80 10 client.31246 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f377a46bb80 10 client.31246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f377a46bb80 20 client.31246 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f377a46bb80 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e4ca7bf950 0x55e4ca7c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f377a46bb80 4 client.31246 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.000+0000 7f377a46bb80 1 -- 172.21.15.12:0/3719001173 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e4ca695f40 con 0x55e4ca7bf950 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.001+0000 7f37677fe700 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e4ca7bf950 0x55e4ca7c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.001+0000 7f37677fe700 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e4ca7bf950 0x55e4ca7c1dd0 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6bde436beaf89a9 server_cookie=510ff7429a27a61 in_seq=0 out_seq=0 2024-05-09T00:08:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.001+0000 7f37477fe700 10 client.31246 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:34.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f37477fe700 1 -- 172.21.15.12:0/3719001173 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e4ca695f40 con 0x55e4ca7bf950 2024-05-09T00:08:34.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f37477fe700 10 client.31246 handle_command_reply: tid=0 2024-05-09T00:08:34.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 client.31246 shutdown 2024-05-09T00:08:34.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 -- 172.21.15.12:0/3719001173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 msgr2=0x55e4ca7b6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 0x55e4ca7b6a90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f375800a010 tx=0x7f3758005910 comp rx=0 tx=0).stop 2024-05-09T00:08:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 -- 172.21.15.12:0/3719001173 shutdown_connections 2024-05-09T00:08:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e4ca7bf950 0x55e4ca7c1dd0 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4ca7537a0 0x55e4ca75b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ca74d910 0x55e4ca7b6a90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 --2- 172.21.15.12:0/3719001173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4ca752e80 0x55e4ca7b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f37477fe700 0 client.31246 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 -- 172.21.15.12:0/3719001173 >> 172.21.15.12:0/3719001173 conn(0x55e4ca6ab290 msgr2=0x55e4ca751c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 -- 172.21.15.12:0/3719001173 shutdown_connections 2024-05-09T00:08:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.004+0000 7f377a46bb80 1 -- 172.21.15.12:0/3719001173 wait complete. 2024-05-09T00:08:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f377a46bb80 20 client.31246 trim_cache size 0 max 16384 2024-05-09T00:08:34.004 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:08:34.004 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:08:34.004 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "fb410f11-9d23-4367-ab3e-04a454b9a015", 2024-05-09T00:08:34.004 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:08:34.004 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:08:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 -- 172.21.15.12:0/515115435 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37500a1300 msgr2=0x7f37500a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37500a1300 0x7f37500a37c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f37681023a0 tx=0x7f3758009310 comp rx=0 tx=0).stop 2024-05-09T00:08:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 -- 172.21.15.12:0/515115435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3768110c30 msgr2=0x7f3768114fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3768110c30 0x7f3768114fa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f375c00e970 tx=0x7f375c00ec80 comp rx=0 tx=0).stop 2024-05-09T00:08:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 -- 172.21.15.12:0/515115435 shutdown_connections 2024-05-09T00:08:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f37500a1300 0x7f37500a37c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37681154e0 0x7f3768115960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37681062a0 0x7f37681106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 --2- 172.21.15.12:0/515115435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3768110c30 0x7f3768114fa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 -- 172.21.15.12:0/515115435 >> 172.21.15.12:0/515115435 conn(0x7f37680fc9e0 msgr2=0x7f37680fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:34.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 -- 172.21.15.12:0/515115435 shutdown_connections 2024-05-09T00:08:34.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.005+0000 7f376e2b1700 1 -- 172.21.15.12:0/515115435 wait complete. 2024-05-09T00:08:34.021 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:08:34.057 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:33 smithi012 ceph-mon[42503]: pgmap v713: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 795 op/s 2024-05-09T00:08:34.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.199+0000 7f1d73cf4700 1 -- 172.21.15.12:0/247153723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d6c10db90 msgr2=0x7f1d6c10ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.199+0000 7f1d73cf4700 1 --2- 172.21.15.12:0/247153723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d6c10db90 0x7f1d6c10ff80 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400b0a0 tx=0x7f1d6400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.199+0000 7f1d73cf4700 1 -- 172.21.15.12:0/247153723 shutdown_connections 2024-05-09T00:08:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.199+0000 7f1d73cf4700 1 --2- 172.21.15.12:0/247153723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d6c10db90 0x7f1d6c10ff80 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.199+0000 7f1d73cf4700 1 --2- 172.21.15.12:0/247153723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6c102010 0x7f1d6c0ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.199+0000 7f1d73cf4700 1 --2- 172.21.15.12:0/247153723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d6c0ff950 0x7f1d6c0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.199+0000 7f1d73cf4700 1 -- 172.21.15.12:0/247153723 >> 172.21.15.12:0/247153723 conn(0x7f1d6c0763a0 msgr2=0x7f1d6c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.199+0000 7f1d73cf4700 1 -- 172.21.15.12:0/247153723 shutdown_connections 2024-05-09T00:08:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.199+0000 7f1d73cf4700 1 -- 172.21.15.12:0/247153723 wait complete. 2024-05-09T00:08:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.200+0000 7f1d73cf4700 1 Processor -- start 2024-05-09T00:08:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.200+0000 7f1d73cf4700 1 -- start start 2024-05-09T00:08:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.200+0000 7f1d73cf4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6c0ff950 0x7f1d6c102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.200+0000 7f1d73cf4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d6c102010 0x7f1d6c102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.200+0000 7f1d73cf4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d6c10db90 0x7f1d6c103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d73cf4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1a9f10 con 0x7f1d6c10db90 2024-05-09T00:08:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d73cf4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1aa050 con 0x7f1d6c102010 2024-05-09T00:08:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d73cf4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1aa1c0 con 0x7f1d6c0ff950 2024-05-09T00:08:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d71a90700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6c0ff950 0x7f1d6c102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d71a90700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6c0ff950 0x7f1d6c102800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42432/0 (socket says 172.21.15.12:42432) 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d71a90700 1 -- 172.21.15.12:0/1977064685 learned_addr learned my addr 172.21.15.12:0/1977064685 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d7128f700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d6c102010 0x7f1d6c102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d72291700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d6c10db90 0x7f1d6c103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d71a90700 1 -- 172.21.15.12:0/1977064685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d6c102010 msgr2=0x7f1d6c102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d71a90700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d6c102010 0x7f1d6c102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d71a90700 1 -- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d6c10db90 msgr2=0x7f1d6c103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d71a90700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d6c10db90 0x7f1d6c103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d71a90700 1 -- 172.21.15.12:0/1977064685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68009520 con 0x7f1d6c0ff950 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d72291700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d6c10db90 0x7f1d6c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d71a90700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6c0ff950 0x7f1d6c102800 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800eb30 tx=0x7f1d6800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d62ffd700 1 -- 172.21.15.12:0/1977064685 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68009880 con 0x7f1d6c0ff950 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d73cf4700 1 -- 172.21.15.12:0/1977064685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d64009d00 con 0x7f1d6c0ff950 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d62ffd700 1 -- 172.21.15.12:0/1977064685 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d680044e0 con 0x7f1d6c0ff950 2024-05-09T00:08:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.201+0000 7f1d62ffd700 1 -- 172.21.15.12:0/1977064685 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68003e90 con 0x7f1d6c0ff950 2024-05-09T00:08:34.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.202+0000 7f1d73cf4700 1 -- 172.21.15.12:0/1977064685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c1aa820 con 0x7f1d6c0ff950 2024-05-09T00:08:34.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.206+0000 7f1d62ffd700 1 -- 172.21.15.12:0/1977064685 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1d68004000 con 0x7f1d6c0ff950 2024-05-09T00:08:34.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.207+0000 7f1d62ffd700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d580a1290 0x7f1d580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.207+0000 7f1d62ffd700 1 -- 172.21.15.12:0/1977064685 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1d68014070 con 0x7f1d6c0ff950 2024-05-09T00:08:34.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.207+0000 7f1d7128f700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d580a1290 0x7f1d580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.208+0000 7f1d7128f700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d580a1290 0x7f1d580a3750 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c103f30 tx=0x7f1d5c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:34.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.209+0000 7f1d7feaeb80 1 Processor -- start 2024-05-09T00:08:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.209+0000 7f1d7feaeb80 1 -- start start 2024-05-09T00:08:34.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.209+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c2e00 0x55e40a5c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.209+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c3720 0x55e40a5bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.209+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.209+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a514150 con 0x55e40a5c2e00 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.209+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a511b10 con 0x55e40a5bd890 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.209+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a50d400 con 0x55e40a5c3720 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.209+0000 7f1d71a90700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c2e00 0x55e40a5c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d71a90700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c2e00 0x55e40a5c31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48426/0 (socket says 172.21.15.12:48426) 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d71a90700 1 -- 172.21.15.12:0/1689200222 learned_addr learned my addr 172.21.15.12:0/1689200222 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d7128f700 1 --2- 172.21.15.12:0/1689200222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c3720 0x55e40a5bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d72291700 1 --2- 172.21.15.12:0/1689200222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d71a90700 1 -- 172.21.15.12:0/1689200222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c3720 msgr2=0x55e40a5bd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:34.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d71a90700 1 --2- 172.21.15.12:0/1689200222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c3720 0x55e40a5bd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d71a90700 1 -- 172.21.15.12:0/1689200222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 msgr2=0x55e40a5bdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:34.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d71a90700 1 --2- 172.21.15.12:0/1689200222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5bdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d71a90700 1 -- 172.21.15.12:0/1689200222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e40a38e3b0 con 0x55e40a5c2e00 2024-05-09T00:08:34.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d71a90700 1 --2- 172.21.15.12:0/1689200222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c2e00 0x55e40a5c31e0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800c5b0 tx=0x7f1d6809a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb49e9ff7641933 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:34.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d57fff700 1 -- 172.21.15.12:0/1689200222 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6809a8e0 con 0x55e40a5c2e00 2024-05-09T00:08:34.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d57fff700 1 -- 172.21.15.12:0/1689200222 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d6800cc90 con 0x55e40a5c2e00 2024-05-09T00:08:34.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.210+0000 7f1d57fff700 1 -- 172.21.15.12:0/1689200222 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d680968f0 con 0x55e40a5c2e00 2024-05-09T00:08:34.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.211+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1689200222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c2e00 msgr2=0x55e40a5c31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.211+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/1689200222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c2e00 0x55e40a5c31e0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800c5b0 tx=0x7f1d6809a080 comp rx=0 tx=0).stop 2024-05-09T00:08:34.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.211+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1689200222 shutdown_connections 2024-05-09T00:08:34.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.211+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/1689200222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5bdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.211+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/1689200222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c2e00 0x55e40a5c31e0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.211+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/1689200222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c3720 0x55e40a5bd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.211+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1689200222 >> 172.21.15.12:0/1689200222 conn(0x55e40a51b210 msgr2=0x55e40a5b7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:34.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.211+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1689200222 shutdown_connections 2024-05-09T00:08:34.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.211+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1689200222 wait complete. 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.212+0000 7f1d7feaeb80 1 -- start start 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.212+0000 7f1d7feaeb80 1 Processor -- start 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 0x55e40a5c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a4a9560 con 0x55e40a5c3720 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a4a9200 con 0x55e40a5bd890 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a3906a0 con 0x55e40a5c2e00 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7feaeb80 10 client.0 _ll_register_callbacks cb 0x55e40a4b2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d72291700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d72291700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48428/0 (socket says 172.21.15.12:48428) 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d72291700 1 -- 172.21.15.12:0/1566408272 learned_addr learned my addr 172.21.15.12:0/1566408272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:34.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7128f700 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 0x55e40a5c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d71a90700 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7128f700 1 -- 172.21.15.12:0/1566408272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 msgr2=0x55e40a5c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7128f700 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7128f700 1 -- 172.21.15.12:0/1566408272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 msgr2=0x55e40a5c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7128f700 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.213+0000 7f1d7128f700 1 -- 172.21.15.12:0/1566408272 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c1aa820 con 0x55e40a5c2e00 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.214+0000 7f1d72291700 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.214+0000 7f1d71a90700 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.214+0000 7f1d7128f700 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 0x55e40a5c6fb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0097c0 tx=0x7f1d5c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c9c0ee960bf940f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.214+0000 7f1d557fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.214+0000 7f1d557fa700 1 -- 172.21.15.12:0/1566408272 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c022070 con 0x55e40a5c2e00 2024-05-09T00:08:34.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.214+0000 7f1d557fa700 1 -- 172.21.15.12:0/1566408272 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1d5c003d40 con 0x55e40a5c2e00 2024-05-09T00:08:34.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.214+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1566408272 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e40a466270 con 0x55e40a5c2e00 2024-05-09T00:08:34.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.214+0000 7f1d557fa700 1 -- 172.21.15.12:0/1566408272 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d5c027420 con 0x55e40a5c2e00 2024-05-09T00:08:34.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.214+0000 7f1d7feaeb80 10 client.31258 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:34.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.214+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1566408272 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e40a33ab60 con 0x55e40a5c2e00 2024-05-09T00:08:34.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.215+0000 7f1d557fa700 1 -- 172.21.15.12:0/1566408272 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e40a466270 con 0x55e40a5c2e00 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.215+0000 7f1d557fa700 1 -- 172.21.15.12:0/1566408272 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1d5c01d5d0 con 0x55e40a5c2e00 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.215+0000 7f1d7feaeb80 10 client.31258 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.215+0000 7f1d7feaeb80 10 client.31258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.215+0000 7f1d7feaeb80 20 client.31258 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.215+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e40a5d72a0 0x55e40a5d96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.215+0000 7f1d7feaeb80 4 client.31258 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.215+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1566408272 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e40a58c810 con 0x55e40a5d72a0 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.216+0000 7f1d71a90700 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e40a5d72a0 0x55e40a5d96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.216+0000 7f1d557fa700 10 client.31258 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.216+0000 7f1d71a90700 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e40a5d72a0 0x55e40a5d96c0 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd7f5f8114b40191 server_cookie=c72654762e5028e0 in_seq=0 out_seq=0 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.218+0000 7f1d557fa700 1 -- 172.21.15.12:0/1566408272 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e40a58c810 con 0x55e40a5d72a0 2024-05-09T00:08:34.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.218+0000 7f1d557fa700 10 client.31258 handle_command_reply: tid=0 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.218+0000 7f1d7feaeb80 1 client.31258 shutdown 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.218+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1566408272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 msgr2=0x55e40a5c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.218+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 0x55e40a5c6fb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0097c0 tx=0x7f1d5c0047c0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.219+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1566408272 shutdown_connections 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.219+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.219+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e40a5d72a0 0x55e40a5d96c0 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.219+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.219+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/1566408272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 0x55e40a5c6fb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.219+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1566408272 >> 172.21.15.12:0/1566408272 conn(0x55e40a51b210 msgr2=0x55e40a5b75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.219+0000 7f1d557fa700 0 client.31258 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.219+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1566408272 shutdown_connections 2024-05-09T00:08:34.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.219+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/1566408272 wait complete. 2024-05-09T00:08:34.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.219+0000 7f1d7feaeb80 20 client.31258 trim_cache size 0 max 16384 2024-05-09T00:08:34.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d7feaeb80 1 Processor -- start 2024-05-09T00:08:34.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d7feaeb80 1 -- start start 2024-05-09T00:08:34.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5bd890 0x55e40a5bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5c2e00 0x55e40a5b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c3720 0x55e40a5c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a5d70d0 con 0x55e40a5bd890 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a5d4950 con 0x55e40a5c2e00 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a5c80e0 con 0x55e40a5c3720 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d71a90700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5bd890 0x55e40a5bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d71a90700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5bd890 0x55e40a5bd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48434/0 (socket says 172.21.15.12:48434) 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d71a90700 1 -- 172.21.15.12:0/3215796692 learned_addr learned my addr 172.21.15.12:0/3215796692 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d7128f700 1 --2- 172.21.15.12:0/3215796692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5c2e00 0x55e40a5b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.245+0000 7f1d72291700 1 --2- 172.21.15.12:0/3215796692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c3720 0x55e40a5c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.246+0000 7f1d71a90700 1 -- 172.21.15.12:0/3215796692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c3720 msgr2=0x55e40a5c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.246+0000 7f1d71a90700 1 --2- 172.21.15.12:0/3215796692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c3720 0x55e40a5c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.246+0000 7f1d71a90700 1 -- 172.21.15.12:0/3215796692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5c2e00 msgr2=0x55e40a5b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.246+0000 7f1d71a90700 1 --2- 172.21.15.12:0/3215796692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5c2e00 0x55e40a5b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.246+0000 7f1d71a90700 1 -- 172.21.15.12:0/3215796692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e40a5bdf40 con 0x55e40a5bd890 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.246+0000 7f1d7128f700 1 --2- 172.21.15.12:0/3215796692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5c2e00 0x55e40a5b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.246+0000 7f1d71a90700 1 --2- 172.21.15.12:0/3215796692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5bd890 0x55e40a5bd350 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f1d6809a170 tx=0x7f1d6800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611bc70bc8e417ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.246+0000 7f1d60ff9700 1 -- 172.21.15.12:0/3215796692 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68062070 con 0x55e40a5bd890 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.246+0000 7f1d60ff9700 1 -- 172.21.15.12:0/3215796692 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d6809e010 con 0x55e40a5bd890 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.246+0000 7f1d60ff9700 1 -- 172.21.15.12:0/3215796692 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68098580 con 0x55e40a5bd890 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.247+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3215796692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5bd890 msgr2=0x55e40a5bd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.247+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/3215796692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5bd890 0x55e40a5bd350 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f1d6809a170 tx=0x7f1d6800ce00 comp rx=0 tx=0).stop 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.247+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3215796692 shutdown_connections 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.247+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/3215796692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c3720 0x55e40a5c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.247+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/3215796692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5bd890 0x55e40a5bd350 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.247+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/3215796692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5c2e00 0x55e40a5b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.247+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3215796692 >> 172.21.15.12:0/3215796692 conn(0x55e40a51b210 msgr2=0x55e40a5c5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:34.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.247+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3215796692 shutdown_connections 2024-05-09T00:08:34.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.247+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3215796692 wait complete. 2024-05-09T00:08:34.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.247+0000 7f1d7feaeb80 1 -- start start 2024-05-09T00:08:34.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d7feaeb80 1 Processor -- start 2024-05-09T00:08:34.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 0x55e40a5bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d7feaeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a5be930 con 0x55e40a5c3720 2024-05-09T00:08:34.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a5beaa0 con 0x55e40a5bd890 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d7feaeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40a5bec60 con 0x55e40a5c2e00 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d7feaeb80 10 client.0 _ll_register_callbacks cb 0x55e40a4b2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d71a90700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d7128f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 0x55e40a5bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d71a90700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54684/0 (socket says 172.21.15.12:54684) 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.248+0000 7f1d71a90700 1 -- 172.21.15.12:0/3490326772 learned_addr learned my addr 172.21.15.12:0/3490326772 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.249+0000 7f1d72291700 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.249+0000 7f1d7128f700 1 -- 172.21.15.12:0/3490326772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 msgr2=0x55e40a5ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.249+0000 7f1d7128f700 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.249+0000 7f1d7128f700 1 -- 172.21.15.12:0/3490326772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 msgr2=0x55e40a5c01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.249+0000 7f1d7128f700 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.249+0000 7f1d7128f700 1 -- 172.21.15.12:0/3490326772 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e40a33ab60 con 0x55e40a5c2e00 2024-05-09T00:08:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.249+0000 7f1d72291700 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.249+0000 7f1d7128f700 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 0x55e40a5bfb70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c01d900 tx=0x7f1d5c003fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8996b931e6587721 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.249+0000 7f1d557fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d557fa700 1 -- 172.21.15.12:0/3490326772 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c022040 con 0x55e40a5c2e00 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3490326772 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d6800cc90 con 0x55e40a5c2e00 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d557fa700 1 -- 172.21.15.12:0/3490326772 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1d5c01d620 con 0x55e40a5c2e00 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d557fa700 1 -- 172.21.15.12:0/3490326772 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d5c030440 con 0x55e40a5c2e00 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d7feaeb80 10 client.31264 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3490326772 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e40a38e3b0 con 0x55e40a5c2e00 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d557fa700 1 -- 172.21.15.12:0/3490326772 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d6800cc90 con 0x55e40a5c2e00 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d557fa700 1 -- 172.21.15.12:0/3490326772 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1d5c0425b0 con 0x55e40a5c2e00 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d7feaeb80 10 client.31264 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d7feaeb80 10 client.31264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d7feaeb80 20 client.31264 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:34.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e40a62e270 0x55e40a630690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d7feaeb80 4 client.31264 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:08:34.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.250+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3490326772 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e40a505ec0 con 0x55e40a62e270 2024-05-09T00:08:34.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.251+0000 7f1d71a90700 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e40a62e270 0x55e40a630690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.251+0000 7f1d71a90700 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e40a62e270 0x55e40a630690 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab5100e7f93dfd server_cookie=6aa7c5572e7c45cb in_seq=0 out_seq=0 2024-05-09T00:08:34.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.251+0000 7f1d557fa700 10 client.31264 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:34.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d557fa700 1 -- 172.21.15.12:0/3490326772 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55e40a505ec0 con 0x55e40a62e270 2024-05-09T00:08:34.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d557fa700 10 client.31264 handle_command_reply: tid=0 2024-05-09T00:08:34.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 client.31264 shutdown 2024-05-09T00:08:34.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3490326772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 msgr2=0x55e40a5bfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 0x55e40a5bfb70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c01d900 tx=0x7f1d5c003fa0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3490326772 shutdown_connections 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e40a5c3720 0x55e40a5c01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e40a62e270 0x55e40a630690 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e40a5bd890 0x55e40a5ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 --2- 172.21.15.12:0/3490326772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e40a5c2e00 0x55e40a5bfb70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d557fa700 0 client.31264 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3490326772 >> 172.21.15.12:0/3490326772 conn(0x55e40a51b210 msgr2=0x55e40a5c23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3490326772 shutdown_connections 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.253+0000 7f1d7feaeb80 1 -- 172.21.15.12:0/3490326772 wait complete. 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d7feaeb80 20 client.31264 trim_cache size 0 max 16384 2024-05-09T00:08:34.253 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:08:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (35 inodes in the stack)", 2024-05-09T00:08:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:08:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "fb410f11-9d23-4367-ab3e-04a454b9a015": { 2024-05-09T00:08:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:08:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "fb410f11-9d23-4367-ab3e-04a454b9a015", 2024-05-09T00:08:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:08:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:08:34.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:08:34.254 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:08:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 -- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d580a1290 msgr2=0x7f1d580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d580a1290 0x7f1d580a3750 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c103f30 tx=0x7f1d5c009260 comp rx=0 tx=0).stop 2024-05-09T00:08:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 -- 172.21.15.12:0/1977064685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6c0ff950 msgr2=0x7f1d6c102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6c0ff950 0x7f1d6c102800 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800eb30 tx=0x7f1d6800ee40 comp rx=0 tx=0).stop 2024-05-09T00:08:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 -- 172.21.15.12:0/1977064685 shutdown_connections 2024-05-09T00:08:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1d580a1290 0x7f1d580a3750 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d6c10db90 0x7f1d6c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6c0ff950 0x7f1d6c102800 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 --2- 172.21.15.12:0/1977064685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d6c102010 0x7f1d6c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 -- 172.21.15.12:0/1977064685 >> 172.21.15.12:0/1977064685 conn(0x7f1d6c0763a0 msgr2=0x7f1d6c10f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 -- 172.21.15.12:0/1977064685 shutdown_connections 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:08:34.254+0000 7f1d73cf4700 1 -- 172.21.15.12:0/1977064685 wait complete. 2024-05-09T00:08:34.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:33 smithi204 ceph-mon[47373]: pgmap v713: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 795 op/s 2024-05-09T00:08:34.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:33 smithi160 ceph-mon[46152]: pgmap v713: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 795 op/s 2024-05-09T00:08:34.268 INFO:tasks.cephfs.filesystem:scrub status for tag:fb410f11-9d23-4367-ab3e-04a454b9a015 - {'path': '/', 'tag': 'fb410f11-9d23-4367-ab3e-04a454b9a015', 'options': 'recursive,force'} 2024-05-09T00:08:34.282 INFO:tasks.workunit.client.0.smithi012.stdout: 10 307599 30.58 MB/sec execute 389 sec latency 123.212 ms 2024-05-09T00:08:35.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:34 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:08:35.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:34 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:34 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:08:35.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:34 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:08:35.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:34 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:08:35.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:34 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:34 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:08:35.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:34 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:34 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:34 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:34 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:08:35.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:34 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:34 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:08:35.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:34 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:08:35.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:34 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:08:35.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:34 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:34 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:08:35.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:34 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:34 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:34 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:34 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:08:35.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:34 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:34 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:08:35.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:34 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:08:35.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:34 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:08:35.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:34 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:34 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:08:35.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:34 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:34 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:34 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:08:35.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 308095 30.55 MB/sec execute 390 sec latency 188.854 ms 2024-05-09T00:08:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:35 smithi012 ceph-mon[42503]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.8 MiB/s wr, 555 op/s 2024-05-09T00:08:36.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:35 smithi204 ceph-mon[47373]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.8 MiB/s wr, 555 op/s 2024-05-09T00:08:36.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:35 smithi160 ceph-mon[46152]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.8 MiB/s wr, 555 op/s 2024-05-09T00:08:36.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 308708 30.55 MB/sec execute 391 sec latency 135.978 ms 2024-05-09T00:08:37.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 309432 30.60 MB/sec execute 392 sec latency 89.004 ms 2024-05-09T00:08:38.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:37 smithi012 ceph-mon[42503]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 758 op/s 2024-05-09T00:08:38.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:37 smithi204 ceph-mon[47373]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 758 op/s 2024-05-09T00:08:38.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:37 smithi160 ceph-mon[46152]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 758 op/s 2024-05-09T00:08:38.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 309854 30.58 MB/sec execute 393 sec latency 164.472 ms 2024-05-09T00:08:39.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 310414 30.59 MB/sec execute 394 sec latency 123.591 ms 2024-05-09T00:08:40.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:39 smithi012 ceph-mon[42503]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-05-09T00:08:40.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:39 smithi204 ceph-mon[47373]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-05-09T00:08:40.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:39 smithi160 ceph-mon[46152]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.6 MiB/s wr, 611 op/s 2024-05-09T00:08:40.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 310729 30.54 MB/sec execute 395 sec latency 199.610 ms 2024-05-09T00:08:41.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 311241 30.53 MB/sec execute 396 sec latency 125.110 ms 2024-05-09T00:08:42.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:41 smithi012 ceph-mon[42503]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 610 op/s 2024-05-09T00:08:42.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:41 smithi204 ceph-mon[47373]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 610 op/s 2024-05-09T00:08:42.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:41 smithi160 ceph-mon[46152]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 610 op/s 2024-05-09T00:08:42.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 311846 30.53 MB/sec execute 397 sec latency 74.877 ms 2024-05-09T00:08:43.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 312348 30.52 MB/sec execute 398 sec latency 180.208 ms 2024-05-09T00:08:44.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:43 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:08:44.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:43 smithi012 ceph-mon[42503]: pgmap v718: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 754 op/s 2024-05-09T00:08:44.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:43 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:08:44.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:43 smithi204 ceph-mon[47373]: pgmap v718: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 754 op/s 2024-05-09T00:08:44.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:43 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:08:44.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:43 smithi160 ceph-mon[46152]: pgmap v718: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 754 op/s 2024-05-09T00:08:44.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 312954 30.53 MB/sec execute 399 sec latency 124.656 ms 2024-05-09T00:08:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.543+0000 7f36cb442700 1 -- 172.21.15.12:0/2672682452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36c4105130 msgr2=0x7f36c410ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.543+0000 7f36cb442700 1 --2- 172.21.15.12:0/2672682452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36c4105130 0x7f36c410ccb0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00b0a0 tx=0x7f36bc00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.543+0000 7f36cb442700 1 -- 172.21.15.12:0/2672682452 shutdown_connections 2024-05-09T00:08:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.543+0000 7f36cb442700 1 --2- 172.21.15.12:0/2672682452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36c4105130 0x7f36c410ccb0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.543+0000 7f36cb442700 1 --2- 172.21.15.12:0/2672682452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36c4100010 0x7f36c41003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.543+0000 7f36cb442700 1 --2- 172.21.15.12:0/2672682452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c41009c0 0x7f36c4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.543+0000 7f36cb442700 1 -- 172.21.15.12:0/2672682452 >> 172.21.15.12:0/2672682452 conn(0x7f36c40fb870 msgr2=0x7f36c40fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.543+0000 7f36cb442700 1 -- 172.21.15.12:0/2672682452 shutdown_connections 2024-05-09T00:08:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.543+0000 7f36cb442700 1 -- 172.21.15.12:0/2672682452 wait complete. 2024-05-09T00:08:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.544+0000 7f36cb442700 1 Processor -- start 2024-05-09T00:08:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.544+0000 7f36cb442700 1 -- start start 2024-05-09T00:08:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.544+0000 7f36cb442700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36c4100010 0x7f36c4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.544+0000 7f36cb442700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36c4105130 0x7f36c4199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.544+0000 7f36cb442700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c419df30 0x7f36c419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.544+0000 7f36cb442700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c419e9b0 con 0x7f36c4100010 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.544+0000 7f36cb442700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c419eb20 con 0x7f36c4105130 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.544+0000 7f36cb442700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c419ece0 con 0x7f36c419df30 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.544+0000 7f36c89dd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36c4105130 0x7f36c4199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c89dd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36c4105130 0x7f36c4199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58442/0 (socket says 172.21.15.12:58442) 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c89dd700 1 -- 172.21.15.12:0/4202223680 learned_addr learned my addr 172.21.15.12:0/4202223680 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c91de700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36c4100010 0x7f36c4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c99df700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c419df30 0x7f36c419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c91de700 1 -- 172.21.15.12:0/4202223680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c419df30 msgr2=0x7f36c419e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c91de700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c419df30 0x7f36c419e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c91de700 1 -- 172.21.15.12:0/4202223680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36c4105130 msgr2=0x7f36c4199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c91de700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36c4105130 0x7f36c4199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c91de700 1 -- 172.21.15.12:0/4202223680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc009d00 con 0x7f36c4100010 2024-05-09T00:08:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c89dd700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36c4105130 0x7f36c4199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c99df700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c419df30 0x7f36c419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.545+0000 7f36c91de700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36c4100010 0x7f36c4199650 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f36c000d750 tx=0x7f36c000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.546+0000 7f36ba7fc700 1 -- 172.21.15.12:0/4202223680 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0004020 con 0x7f36c4100010 2024-05-09T00:08:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.546+0000 7f36ba7fc700 1 -- 172.21.15.12:0/4202223680 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36c0004830 con 0x7f36c4100010 2024-05-09T00:08:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.546+0000 7f36ba7fc700 1 -- 172.21.15.12:0/4202223680 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c00054c0 con 0x7f36c4100010 2024-05-09T00:08:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.546+0000 7f36cb442700 1 -- 172.21.15.12:0/4202223680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c41a6240 con 0x7f36c4100010 2024-05-09T00:08:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.546+0000 7f36cb442700 1 -- 172.21.15.12:0/4202223680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c41a66d0 con 0x7f36c4100010 2024-05-09T00:08:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.550+0000 7f36ba7fc700 1 -- 172.21.15.12:0/4202223680 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f36c00049a0 con 0x7f36c4100010 2024-05-09T00:08:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.551+0000 7f36d75fcb80 1 Processor -- start 2024-05-09T00:08:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.551+0000 7f36ba7fc700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f36b00a1310 0x7f36b00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.551+0000 7f36ba7fc700 1 -- 172.21.15.12:0/4202223680 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f36c00cc560 con 0x7f36c4100010 2024-05-09T00:08:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.551+0000 7f36d75fcb80 1 -- start start 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36c99df700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f36b00a1310 0x7f36b00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 0x55e85d456d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d450f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d4518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d3a80f0 con 0x55e85d456950 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d3a5ab0 con 0x55e85d451480 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d3a13a0 con 0x55e85d457270 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36c99df700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d4518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36c89dd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d450f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36c89dd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d450f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53952/0 (socket says 172.21.15.12:53952) 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36c89dd700 1 -- 172.21.15.12:0/657265481 learned_addr learned my addr 172.21.15.12:0/657265481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36c91de700 1 --2- 172.21.15.12:0/657265481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 0x55e85d456d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36c99df700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f36b00a1310 0x7f36b00a37d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009fd0 tx=0x7f36bc009580 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36c89dd700 1 -- 172.21.15.12:0/657265481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 msgr2=0x55e85d4518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36c89dd700 1 --2- 172.21.15.12:0/657265481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d4518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.552+0000 7f36c89dd700 1 -- 172.21.15.12:0/657265481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 msgr2=0x55e85d456d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.553+0000 7f36c89dd700 1 --2- 172.21.15.12:0/657265481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 0x55e85d456d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.553+0000 7f36c89dd700 1 -- 172.21.15.12:0/657265481 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e85d222350 con 0x55e85d457270 2024-05-09T00:08:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.553+0000 7f36c89dd700 1 --2- 172.21.15.12:0/657265481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d450f40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f36b40115c0 tx=0x7f36b40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=454321785f896254 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.553+0000 7f36af7fe700 1 -- 172.21.15.12:0/657265481 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4019630 con 0x55e85d457270 2024-05-09T00:08:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.553+0000 7f36af7fe700 1 -- 172.21.15.12:0/657265481 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b4019c50 con 0x55e85d457270 2024-05-09T00:08:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.553+0000 7f36af7fe700 1 -- 172.21.15.12:0/657265481 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b400ed40 con 0x55e85d457270 2024-05-09T00:08:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.554+0000 7f36d75fcb80 1 -- 172.21.15.12:0/657265481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 msgr2=0x55e85d450f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.554+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/657265481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d450f40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f36b40115c0 tx=0x7f36b40118d0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.554+0000 7f36d75fcb80 1 -- 172.21.15.12:0/657265481 shutdown_connections 2024-05-09T00:08:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.554+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/657265481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d4518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.554+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/657265481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 0x55e85d456d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.554+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/657265481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d450f40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.554+0000 7f36d75fcb80 1 -- 172.21.15.12:0/657265481 >> 172.21.15.12:0/657265481 conn(0x55e85d3af1b0 msgr2=0x55e85d44b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.554+0000 7f36d75fcb80 1 -- 172.21.15.12:0/657265481 shutdown_connections 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.554+0000 7f36d75fcb80 1 -- 172.21.15.12:0/657265481 wait complete. 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36d75fcb80 1 -- start start 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36d75fcb80 1 Processor -- start 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d45d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 0x55e85d45a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d45ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d33d500 con 0x55e85d456950 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d33d1a0 con 0x55e85d451480 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d224640 con 0x55e85d457270 2024-05-09T00:08:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36d75fcb80 10 client.0 _ll_register_callbacks cb 0x55e85d346ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36c89dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 0x55e85d45a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36c99df700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d45ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36c89dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 0x55e85d45a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55552/0 (socket says 172.21.15.12:55552) 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.555+0000 7f36c89dd700 1 -- 172.21.15.12:0/3684672428 learned_addr learned my addr 172.21.15.12:0/3684672428 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36c91de700 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d45d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36c99df700 1 -- 172.21.15.12:0/3684672428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 msgr2=0x55e85d45d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36c99df700 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d45d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36c99df700 1 -- 172.21.15.12:0/3684672428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 msgr2=0x55e85d45a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36c99df700 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 0x55e85d45a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36c99df700 1 -- 172.21.15.12:0/3684672428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b4009d80 con 0x55e85d457270 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36c89dd700 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 0x55e85d45a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36c99df700 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d45ab70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00b780 tx=0x7f36bc00b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33ceb645c1073aad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36acff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36acff9700 1 -- 172.21.15.12:0/3684672428 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc00e040 con 0x55e85d457270 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36acff9700 1 -- 172.21.15.12:0/3684672428 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f36bc02a660 con 0x55e85d457270 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.556+0000 7f36d75fcb80 1 -- 172.21.15.12:0/3684672428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e85d2fa210 con 0x55e85d457270 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36acff9700 1 -- 172.21.15.12:0/3684672428 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36bc02a820 con 0x55e85d457270 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36d75fcb80 10 client.31276 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36d75fcb80 1 -- 172.21.15.12:0/3684672428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e85d1ceb00 con 0x55e85d457270 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36acff9700 1 -- 172.21.15.12:0/3684672428 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e85d2fa210 con 0x55e85d457270 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36acff9700 1 -- 172.21.15.12:0/3684672428 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f36bc01fc10 con 0x55e85d457270 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36d75fcb80 10 client.31276 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36d75fcb80 10 client.31276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36d75fcb80 20 client.31276 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e85d46ac10 0x55e85d46d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36d75fcb80 4 client.31276 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36d75fcb80 1 -- 172.21.15.12:0/3684672428 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e85d4207b0 con 0x55e85d46ac10 2024-05-09T00:08:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.557+0000 7f36c91de700 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e85d46ac10 0x55e85d46d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.558+0000 7f36c91de700 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e85d46ac10 0x55e85d46d030 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73f7ed956466e78 server_cookie=145fcac2e872bbeb in_seq=0 out_seq=0 2024-05-09T00:08:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.558+0000 7f36acff9700 10 client.31276 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36acff9700 1 -- 172.21.15.12:0/3684672428 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e85d4207b0 con 0x55e85d46ac10 2024-05-09T00:08:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36acff9700 10 client.31276 handle_command_reply: tid=0 2024-05-09T00:08:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 client.31276 shutdown 2024-05-09T00:08:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 -- 172.21.15.12:0/3684672428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 msgr2=0x55e85d45ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d45ab70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00b780 tx=0x7f36bc00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 -- 172.21.15.12:0/3684672428 shutdown_connections 2024-05-09T00:08:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d457270 0x55e85d45ab70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e85d46ac10 0x55e85d46d030 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d45d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/3684672428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d456950 0x55e85d45a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 -- 172.21.15.12:0/3684672428 >> 172.21.15.12:0/3684672428 conn(0x55e85d3af1b0 msgr2=0x55e85d4554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36acff9700 0 client.31276 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 -- 172.21.15.12:0/3684672428 shutdown_connections 2024-05-09T00:08:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 1 -- 172.21.15.12:0/3684672428 wait complete. 2024-05-09T00:08:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.559+0000 7f36d75fcb80 20 client.31276 trim_cache size 0 max 16384 2024-05-09T00:08:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36d75fcb80 1 Processor -- start 2024-05-09T00:08:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36d75fcb80 1 -- start start 2024-05-09T00:08:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d451480 0x55e85d44b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d461c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d457270 0x55e85d462150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d2fa210 con 0x55e85d451480 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d4467c0 con 0x55e85d457270 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d4489b0 con 0x55e85d456950 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36c91de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d451480 0x55e85d44b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36c89dd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d461c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36c91de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d451480 0x55e85d44b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55556/0 (socket says 172.21.15.12:55556) 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36c89dd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d461c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53972/0 (socket says 172.21.15.12:53972) 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36c91de700 1 -- 172.21.15.12:0/160699435 learned_addr learned my addr 172.21.15.12:0/160699435 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36c99df700 1 --2- 172.21.15.12:0/160699435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d457270 0x55e85d462150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36c89dd700 1 -- 172.21.15.12:0/160699435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d457270 msgr2=0x55e85d462150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36c89dd700 1 --2- 172.21.15.12:0/160699435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d457270 0x55e85d462150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36c89dd700 1 -- 172.21.15.12:0/160699435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d451480 msgr2=0x55e85d44b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.580+0000 7f36c89dd700 1 --2- 172.21.15.12:0/160699435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d451480 0x55e85d44b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.581+0000 7f36c89dd700 1 -- 172.21.15.12:0/160699435 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e85d367500 con 0x55e85d456950 2024-05-09T00:08:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.581+0000 7f36c91de700 1 --2- 172.21.15.12:0/160699435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d451480 0x55e85d44b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.581+0000 7f36c89dd700 1 --2- 172.21.15.12:0/160699435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d461c10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55e85d45ba10 tx=0x7f36b4019dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e591da697ef0dc9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.581+0000 7f36affff700 1 -- 172.21.15.12:0/160699435 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b400ee90 con 0x55e85d456950 2024-05-09T00:08:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.581+0000 7f36affff700 1 -- 172.21.15.12:0/160699435 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b4005ce0 con 0x55e85d456950 2024-05-09T00:08:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.581+0000 7f36affff700 1 -- 172.21.15.12:0/160699435 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b40196b0 con 0x55e85d456950 2024-05-09T00:08:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.582+0000 7f36d75fcb80 1 -- 172.21.15.12:0/160699435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 msgr2=0x55e85d461c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.582+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/160699435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d461c10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55e85d45ba10 tx=0x7f36b4019dc0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.582+0000 7f36d75fcb80 1 -- 172.21.15.12:0/160699435 shutdown_connections 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.582+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/160699435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d457270 0x55e85d462150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.582+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/160699435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d451480 0x55e85d44b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.582+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/160699435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d461c10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.582+0000 7f36d75fcb80 1 -- 172.21.15.12:0/160699435 >> 172.21.15.12:0/160699435 conn(0x55e85d3af1b0 msgr2=0x55e85d44f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.582+0000 7f36d75fcb80 1 -- 172.21.15.12:0/160699435 shutdown_connections 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.582+0000 7f36d75fcb80 1 -- 172.21.15.12:0/160699435 wait complete. 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.582+0000 7f36d75fcb80 1 -- start start 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36d75fcb80 1 Processor -- start 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d461fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d45fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36d75fcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d457270 0x55e85d460130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d460840 con 0x55e85d457270 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d4609b0 con 0x55e85d451480 2024-05-09T00:08:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36d75fcb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85d4c0110 con 0x55e85d456950 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36d75fcb80 10 client.0 _ll_register_callbacks cb 0x55e85d346ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36c89dd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d45fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36c89dd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d45fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:53982/0 (socket says 172.21.15.12:53982) 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36c89dd700 1 -- 172.21.15.12:0/199541687 learned_addr learned my addr 172.21.15.12:0/199541687 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36c99df700 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d457270 0x55e85d460130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.583+0000 7f36c91de700 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d461fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36c99df700 1 -- 172.21.15.12:0/199541687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 msgr2=0x55e85d45fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36c99df700 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d45fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36c99df700 1 -- 172.21.15.12:0/199541687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 msgr2=0x55e85d461fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36c99df700 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d461fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36c99df700 1 -- 172.21.15.12:0/199541687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c41a66d0 con 0x55e85d457270 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36c89dd700 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d45fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36c91de700 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d461fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36c99df700 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d457270 0x55e85d460130 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x55e85d44cc80 tx=0x7f36bc02ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13eea938e3956b93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36acff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36acff9700 1 -- 172.21.15.12:0/199541687 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0051e0 con 0x55e85d457270 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36acff9700 1 -- 172.21.15.12:0/199541687 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f36bc01ea50 con 0x55e85d457270 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.584+0000 7f36d75fcb80 1 -- 172.21.15.12:0/199541687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36b4019c50 con 0x55e85d457270 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36acff9700 1 -- 172.21.15.12:0/199541687 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36bc01ec10 con 0x55e85d457270 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36d75fcb80 10 client.21576 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36d75fcb80 1 -- 172.21.15.12:0/199541687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e85d222350 con 0x55e85d457270 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36acff9700 1 -- 172.21.15.12:0/199541687 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36bc04c020 con 0x55e85d457270 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36acff9700 1 -- 172.21.15.12:0/199541687 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f36bc0056c0 con 0x55e85d457270 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36d75fcb80 10 client.21576 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36d75fcb80 10 client.21576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36d75fcb80 20 client.21576 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e85d4c1350 0x55e85d4c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36d75fcb80 4 client.21576 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:08:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36d75fcb80 1 -- 172.21.15.12:0/199541687 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e85d399e60 con 0x55e85d4c1350 2024-05-09T00:08:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.585+0000 7f36c91de700 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e85d4c1350 0x55e85d4c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.586+0000 7f36acff9700 10 client.21576 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:44.586+0000 7f36c91de700 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e85d4c1350 0x55e85d4c3740 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=494c305dacc7254f server_cookie=b293ce3bc14515b6 in_seq=0 out_seq=0 2024-05-09T00:08:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.032+0000 7f36acff9700 1 -- 172.21.15.12:0/199541687 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e85d399e60 con 0x55e85d4c1350 2024-05-09T00:08:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.032+0000 7f36acff9700 10 client.21576 handle_command_reply: tid=0 2024-05-09T00:08:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.033+0000 7f36d75fcb80 1 client.21576 shutdown 2024-05-09T00:08:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.033+0000 7f36d75fcb80 1 -- 172.21.15.12:0/199541687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d457270 msgr2=0x55e85d460130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.033+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d457270 0x55e85d460130 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x55e85d44cc80 tx=0x7f36bc02ae80 comp rx=0 tx=0).stop 2024-05-09T00:08:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.034+0000 7f36d75fcb80 1 -- 172.21.15.12:0/199541687 shutdown_connections 2024-05-09T00:08:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.034+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e85d457270 0x55e85d460130 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.034+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e85d4c1350 0x55e85d4c3740 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.034+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e85d451480 0x55e85d461fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.034+0000 7f36d75fcb80 1 --2- 172.21.15.12:0/199541687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e85d456950 0x55e85d45fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.034+0000 7f36acff9700 0 client.21576 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.034+0000 7f36d75fcb80 1 -- 172.21.15.12:0/199541687 >> 172.21.15.12:0/199541687 conn(0x55e85d3af1b0 msgr2=0x55e85d458d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.034+0000 7f36d75fcb80 1 -- 172.21.15.12:0/199541687 shutdown_connections 2024-05-09T00:08:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.034+0000 7f36d75fcb80 1 -- 172.21.15.12:0/199541687 wait complete. 2024-05-09T00:08:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.034+0000 7f36d75fcb80 20 client.21576 trim_cache size 0 max 16384 2024-05-09T00:08:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:08:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:08:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:08:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:08:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.035+0000 7f36cb442700 1 -- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f36b00a1310 msgr2=0x7f36b00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.035+0000 7f36cb442700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f36b00a1310 0x7f36b00a37d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009fd0 tx=0x7f36bc009580 comp rx=0 tx=0).stop 2024-05-09T00:08:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.035+0000 7f36cb442700 1 -- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36c4100010 msgr2=0x7f36c4199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.035+0000 7f36cb442700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36c4100010 0x7f36c4199650 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f36c000d750 tx=0x7f36c000da60 comp rx=0 tx=0).stop 2024-05-09T00:08:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.037+0000 7f36cb442700 1 -- 172.21.15.12:0/4202223680 shutdown_connections 2024-05-09T00:08:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.037+0000 7f36cb442700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f36b00a1310 0x7f36b00a37d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.037+0000 7f36cb442700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c419df30 0x7f36c419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.037+0000 7f36cb442700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36c4100010 0x7f36c4199650 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.037+0000 7f36cb442700 1 --2- 172.21.15.12:0/4202223680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36c4105130 0x7f36c4199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.037+0000 7f36cb442700 1 -- 172.21.15.12:0/4202223680 >> 172.21.15.12:0/4202223680 conn(0x7f36c40fb870 msgr2=0x7f36c40fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.037+0000 7f36cb442700 1 -- 172.21.15.12:0/4202223680 shutdown_connections 2024-05-09T00:08:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:45.037+0000 7f36cb442700 1 -- 172.21.15.12:0/4202223680 wait complete. 2024-05-09T00:08:45.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:44 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:08:45.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:44 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:08:45.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:44 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:08:45.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:44 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:08:45.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:44 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:08:45.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:44 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:08:45.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 313280 30.49 MB/sec execute 400 sec latency 300.152 ms 2024-05-09T00:08:46.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:45 smithi012 ceph-mon[42503]: pgmap v719: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 501 op/s 2024-05-09T00:08:46.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:45 smithi204 ceph-mon[47373]: pgmap v719: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 501 op/s 2024-05-09T00:08:46.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:45 smithi160 ceph-mon[46152]: pgmap v719: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 501 op/s 2024-05-09T00:08:46.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 313609 30.48 MB/sec execute 401 sec latency 296.946 ms 2024-05-09T00:08:47.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 314145 30.48 MB/sec execute 402 sec latency 93.942 ms 2024-05-09T00:08:48.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:47 smithi012 ceph-mon[42503]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 732 op/s 2024-05-09T00:08:48.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:47 smithi204 ceph-mon[47373]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 732 op/s 2024-05-09T00:08:48.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:47 smithi160 ceph-mon[46152]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 732 op/s 2024-05-09T00:08:48.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 314652 30.46 MB/sec execute 403 sec latency 111.320 ms 2024-05-09T00:08:49.284 INFO:tasks.workunit.client.0.smithi012.stdout: 10 315159 30.47 MB/sec execute 404 sec latency 84.952 ms 2024-05-09T00:08:50.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:49 smithi012 ceph-mon[42503]: pgmap v721: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.4 MiB/s wr, 572 op/s 2024-05-09T00:08:50.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:49 smithi204 ceph-mon[47373]: pgmap v721: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.4 MiB/s wr, 572 op/s 2024-05-09T00:08:50.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:49 smithi160 ceph-mon[46152]: pgmap v721: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.4 MiB/s wr, 572 op/s 2024-05-09T00:08:50.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 315680 30.43 MB/sec execute 405 sec latency 107.325 ms 2024-05-09T00:08:51.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 316055 30.40 MB/sec execute 406 sec latency 321.650 ms 2024-05-09T00:08:52.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:51 smithi012 ceph-mon[42503]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.4 MiB/s wr, 555 op/s 2024-05-09T00:08:52.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:51 smithi204 ceph-mon[47373]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.4 MiB/s wr, 555 op/s 2024-05-09T00:08:52.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:51 smithi160 ceph-mon[46152]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.4 MiB/s wr, 555 op/s 2024-05-09T00:08:52.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 316627 30.42 MB/sec execute 407 sec latency 125.393 ms 2024-05-09T00:08:53.284 INFO:tasks.workunit.client.0.smithi012.stdout: 10 317257 30.42 MB/sec execute 408 sec latency 99.255 ms 2024-05-09T00:08:54.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:53 smithi012 ceph-mon[42503]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 14 MiB/s wr, 739 op/s 2024-05-09T00:08:54.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:53 smithi204 ceph-mon[47373]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 14 MiB/s wr, 739 op/s 2024-05-09T00:08:54.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:53 smithi160 ceph-mon[46152]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 14 MiB/s wr, 739 op/s 2024-05-09T00:08:54.283 INFO:tasks.workunit.client.0.smithi012.stdout: 10 317793 30.43 MB/sec execute 409 sec latency 96.107 ms 2024-05-09T00:08:55.284 INFO:tasks.workunit.client.0.smithi012.stdout: 10 318334 30.44 MB/sec execute 410 sec latency 74.566 ms 2024-05-09T00:08:56.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:55 smithi012 ceph-mon[42503]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 573 op/s 2024-05-09T00:08:56.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:55 smithi160 ceph-mon[46152]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 573 op/s 2024-05-09T00:08:56.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:55 smithi204 ceph-mon[47373]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 573 op/s 2024-05-09T00:08:56.284 INFO:tasks.workunit.client.0.smithi012.stdout: 10 318535 30.39 MB/sec execute 411 sec latency 306.805 ms 2024-05-09T00:08:57.284 INFO:tasks.workunit.client.0.smithi012.stdout: 10 318931 30.35 MB/sec execute 412 sec latency 115.592 ms 2024-05-09T00:08:58.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:57 smithi012 ceph-mon[42503]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 723 op/s 2024-05-09T00:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.182+0000 7f1020863700 1 -- 172.21.15.12:0/3708032080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101c104910 msgr2=0x7f101c10c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.182+0000 7f1020863700 1 --2- 172.21.15.12:0/3708032080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101c104910 0x7f101c10c490 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f101400c1b0 tx=0x7f101400c4c0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.182+0000 7f1020863700 1 -- 172.21.15.12:0/3708032080 shutdown_connections 2024-05-09T00:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.183+0000 7f1020863700 1 --2- 172.21.15.12:0/3708032080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101c104910 0x7f101c10c490 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.183+0000 7f1020863700 1 --2- 172.21.15.12:0/3708032080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c0ffa50 0x7f101c0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.183+0000 7f1020863700 1 --2- 172.21.15.12:0/3708032080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f101c100400 0x7f101c1041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.183+0000 7f1020863700 1 -- 172.21.15.12:0/3708032080 >> 172.21.15.12:0/3708032080 conn(0x7f101c075130 msgr2=0x7f101c075540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.183+0000 7f1020863700 1 -- 172.21.15.12:0/3708032080 shutdown_connections 2024-05-09T00:08:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.183+0000 7f1020863700 1 -- 172.21.15.12:0/3708032080 wait complete. 2024-05-09T00:08:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.183+0000 7f1020863700 1 Processor -- start 2024-05-09T00:08:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.183+0000 7f1020863700 1 -- start start 2024-05-09T00:08:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f1020863700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f101c0ffa50 0x7f101c1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f1020863700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c104910 0x7f101c101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f1020863700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101c101c70 0x7f101c1a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f1020863700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c1a59c0 con 0x7f101c0ffa50 2024-05-09T00:08:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f1020863700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c1a5b30 con 0x7f101c101c70 2024-05-09T00:08:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f1020863700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c1a5cf0 con 0x7f101c104910 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f101a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f101c0ffa50 0x7f101c1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f101a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f101c0ffa50 0x7f101c1011f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47564/0 (socket says 172.21.15.12:47564) 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f101a59c700 1 -- 172.21.15.12:0/3163408010 learned_addr learned my addr 172.21.15.12:0/3163408010 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f1019d9b700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c104910 0x7f101c101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f101a59c700 1 -- 172.21.15.12:0/3163408010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c104910 msgr2=0x7f101c101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f101a59c700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c104910 0x7f101c101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f101a59c700 1 -- 172.21.15.12:0/3163408010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101c101c70 msgr2=0x7f101c1a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f101a59c700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101c101c70 0x7f101c1a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f101a59c700 1 -- 172.21.15.12:0/3163408010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101400ad50 con 0x7f101c0ffa50 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.184+0000 7f101a59c700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f101c0ffa50 0x7f101c1011f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f100c00e9c0 tx=0x7f100c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.185+0000 7f100b7fe700 1 -- 172.21.15.12:0/3163408010 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c00cbd0 con 0x7f101c0ffa50 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.185+0000 7f100b7fe700 1 -- 172.21.15.12:0/3163408010 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100c00eeb0 con 0x7f101c0ffa50 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.185+0000 7f1020863700 1 -- 172.21.15.12:0/3163408010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c1a5fd0 con 0x7f101c0ffa50 2024-05-09T00:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.185+0000 7f100b7fe700 1 -- 172.21.15.12:0/3163408010 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c005580 con 0x7f101c0ffa50 2024-05-09T00:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.185+0000 7f1020863700 1 -- 172.21.15.12:0/3163408010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c1a6450 con 0x7f101c0ffa50 2024-05-09T00:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.188+0000 7f100b7fe700 1 -- 172.21.15.12:0/3163408010 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f100c004750 con 0x7f101c0ffa50 2024-05-09T00:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.188+0000 7f100b7fe700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10040a1290 0x7f10040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.189+0000 7f100b7fe700 1 -- 172.21.15.12:0/3163408010 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f100c014070 con 0x7f101c0ffa50 2024-05-09T00:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.189+0000 7f1019d9b700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10040a1290 0x7f10040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.189+0000 7f1019d9b700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10040a1290 0x7f10040a3750 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f101c102920 tx=0x7f1010009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.190+0000 7f102ca1db80 1 Processor -- start 2024-05-09T00:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.190+0000 7f102ca1db80 1 -- start start 2024-05-09T00:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.190+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 0x55be84fdad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.190+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fdb270 0x55be84fd4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fd5480 0x55be84fd58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f102ca1db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84f2c0f0 con 0x55be84fdb270 2024-05-09T00:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f102ca1db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84f29ab0 con 0x55be84fd5480 2024-05-09T00:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f102ca1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84f253a0 con 0x55be84fda950 2024-05-09T00:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f1019d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fdb270 0x55be84fd4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f101a59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 0x55be84fdad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f1019d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fdb270 0x55be84fd4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47580/0 (socket says 172.21.15.12:47580) 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f1019d9b700 1 -- 172.21.15.12:0/1999692937 learned_addr learned my addr 172.21.15.12:0/1999692937 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f101ad9d700 1 --2- 172.21.15.12:0/1999692937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fd5480 0x55be84fd58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f1019d9b700 1 -- 172.21.15.12:0/1999692937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 msgr2=0x55be84fdad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f1019d9b700 1 --2- 172.21.15.12:0/1999692937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 0x55be84fdad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f1019d9b700 1 -- 172.21.15.12:0/1999692937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fd5480 msgr2=0x55be84fd58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f1019d9b700 1 --2- 172.21.15.12:0/1999692937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fd5480 0x55be84fd58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f1019d9b700 1 -- 172.21.15.12:0/1999692937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be84da6350 con 0x55be84fdb270 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f1019d9b700 1 --2- 172.21.15.12:0/1999692937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fdb270 0x55be84fd4f40 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f1010011d20 tx=0x7f1010014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca15700762918b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.191+0000 7f1008ff9700 1 -- 172.21.15.12:0/1999692937 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101000eb20 con 0x55be84fdb270 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.192+0000 7f1008ff9700 1 -- 172.21.15.12:0/1999692937 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1010021070 con 0x55be84fdb270 2024-05-09T00:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.192+0000 7f1008ff9700 1 -- 172.21.15.12:0/1999692937 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101001d670 con 0x55be84fdb270 2024-05-09T00:08:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.192+0000 7f102ca1db80 1 -- 172.21.15.12:0/1999692937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fdb270 msgr2=0x55be84fd4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.192+0000 7f102ca1db80 1 --2- 172.21.15.12:0/1999692937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fdb270 0x55be84fd4f40 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f1010011d20 tx=0x7f1010014000 comp rx=0 tx=0).stop 2024-05-09T00:08:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.192+0000 7f102ca1db80 1 -- 172.21.15.12:0/1999692937 shutdown_connections 2024-05-09T00:08:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.192+0000 7f102ca1db80 1 --2- 172.21.15.12:0/1999692937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fd5480 0x55be84fd58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.192+0000 7f102ca1db80 1 --2- 172.21.15.12:0/1999692937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 0x55be84fdad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.192+0000 7f102ca1db80 1 --2- 172.21.15.12:0/1999692937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fdb270 0x55be84fd4f40 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.192+0000 7f102ca1db80 1 -- 172.21.15.12:0/1999692937 >> 172.21.15.12:0/1999692937 conn(0x55be84f331b0 msgr2=0x55be84fcf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.193+0000 7f102ca1db80 1 -- 172.21.15.12:0/1999692937 shutdown_connections 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.193+0000 7f102ca1db80 1 -- 172.21.15.12:0/1999692937 wait complete. 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.193+0000 7f102ca1db80 1 -- start start 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.193+0000 7f102ca1db80 1 Processor -- start 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.193+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fd5480 0x55be84fe11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.193+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fda950 0x55be84fde550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.193+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 0x55be84fdeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.193+0000 7f102ca1db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84ec1500 con 0x55be84fda950 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.193+0000 7f102ca1db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84ec11a0 con 0x55be84fd5480 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.193+0000 7f102ca1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84da8640 con 0x55be84fdb270 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f102ca1db80 10 client.0 _ll_register_callbacks cb 0x55be84ecaad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f101ad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 0x55be84fdeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f101ad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 0x55be84fdeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57554/0 (socket says 172.21.15.12:57554) 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f101ad9d700 1 -- 172.21.15.12:0/2281302684 learned_addr learned my addr 172.21.15.12:0/2281302684 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f1019d9b700 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fda950 0x55be84fde550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f101a59c700 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fd5480 0x55be84fe11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f101ad9d700 1 -- 172.21.15.12:0/2281302684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fd5480 msgr2=0x55be84fe11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f101ad9d700 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fd5480 0x55be84fe11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f101ad9d700 1 -- 172.21.15.12:0/2281302684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fda950 msgr2=0x55be84fde550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f101ad9d700 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fda950 0x55be84fde550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f101ad9d700 1 -- 172.21.15.12:0/2281302684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1010017070 con 0x55be84fdb270 2024-05-09T00:08:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f1019d9b700 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fda950 0x55be84fde550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f101ad9d700 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 0x55be84fdeb60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f101400a3c0 tx=0x7f101400a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29d9edc19da96ca9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.194+0000 7f1001ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f1001ffb700 1 -- 172.21.15.12:0/2281302684 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400f040 con 0x55be84fdb270 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f102ca1db80 1 -- 172.21.15.12:0/2281302684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be84e7e210 con 0x55be84fdb270 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f1001ffb700 1 -- 172.21.15.12:0/2281302684 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1014003e10 con 0x55be84fdb270 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f1001ffb700 1 -- 172.21.15.12:0/2281302684 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f101400cea0 con 0x55be84fdb270 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f102ca1db80 10 client.31294 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f102ca1db80 1 -- 172.21.15.12:0/2281302684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55be84d52b00 con 0x55be84fdb270 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f1001ffb700 1 -- 172.21.15.12:0/2281302684 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be84e7e210 con 0x55be84fdb270 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f1001ffb700 1 -- 172.21.15.12:0/2281302684 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f101401a040 con 0x55be84fdb270 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f102ca1db80 10 client.31294 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f102ca1db80 10 client.31294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f102ca1db80 20 client.31294 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.195+0000 7f102ca1db80 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be84feeba0 0x55be84ff0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.196+0000 7f102ca1db80 4 client.31294 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:08:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.196+0000 7f102ca1db80 1 -- 172.21.15.12:0/2281302684 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be84fa47b0 con 0x55be84feeba0 2024-05-09T00:08:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.196+0000 7f101a59c700 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be84feeba0 0x55be84ff0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.196+0000 7f1001ffb700 10 client.31294 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.196+0000 7f101a59c700 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be84feeba0 0x55be84ff0fc0 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af38865b931a69df server_cookie=a48bef622bf50e4d in_seq=0 out_seq=0 2024-05-09T00:08:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.198+0000 7f1001ffb700 1 -- 172.21.15.12:0/2281302684 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be84fa47b0 con 0x55be84feeba0 2024-05-09T00:08:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.198+0000 7f1001ffb700 10 client.31294 handle_command_reply: tid=0 2024-05-09T00:08:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.198+0000 7f102ca1db80 1 client.31294 shutdown 2024-05-09T00:08:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 1 -- 172.21.15.12:0/2281302684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 msgr2=0x55be84fdeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 0x55be84fdeb60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f101400a3c0 tx=0x7f101400a3f0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 1 -- 172.21.15.12:0/2281302684 shutdown_connections 2024-05-09T00:08:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 0x55be84fdeb60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be84feeba0 0x55be84ff0fc0 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fd5480 0x55be84fe11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 1 --2- 172.21.15.12:0/2281302684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fda950 0x55be84fde550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f1001ffb700 0 client.31294 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 1 -- 172.21.15.12:0/2281302684 >> 172.21.15.12:0/2281302684 conn(0x55be84f331b0 msgr2=0x55be84fd94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 1 -- 172.21.15.12:0/2281302684 shutdown_connections 2024-05-09T00:08:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 1 -- 172.21.15.12:0/2281302684 wait complete. 2024-05-09T00:08:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.199+0000 7f102ca1db80 20 client.31294 trim_cache size 0 max 16384 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.217+0000 7f102ca1db80 1 Processor -- start 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f102ca1db80 1 -- start start 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fcf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 0x55be84fd9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fdb270 0x55be84fe0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f102ca1db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84fda580 con 0x55be84fd5480 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f102ca1db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84fdcaa0 con 0x55be84fdb270 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f102ca1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84fd0580 con 0x55be84fda950 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f101a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fcf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f1019d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 0x55be84fd9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f1019d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 0x55be84fd9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:57564/0 (socket says 172.21.15.12:57564) 2024-05-09T00:08:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f101a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fcf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47600/0 (socket says 172.21.15.12:47600) 2024-05-09T00:08:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f1019d9b700 1 -- 172.21.15.12:0/550948807 learned_addr learned my addr 172.21.15.12:0/550948807 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f101ad9d700 1 --2- 172.21.15.12:0/550948807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fdb270 0x55be84fe0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f1019d9b700 1 -- 172.21.15.12:0/550948807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fdb270 msgr2=0x55be84fe0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f1019d9b700 1 --2- 172.21.15.12:0/550948807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fdb270 0x55be84fe0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f1019d9b700 1 -- 172.21.15.12:0/550948807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 msgr2=0x55be84fcf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f1019d9b700 1 --2- 172.21.15.12:0/550948807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fcf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f1019d9b700 1 -- 172.21.15.12:0/550948807 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be84eeb500 con 0x55be84fda950 2024-05-09T00:08:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.218+0000 7f101a59c700 1 --2- 172.21.15.12:0/550948807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fcf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.219+0000 7f1019d9b700 1 --2- 172.21.15.12:0/550948807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 0x55be84fd9390 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1010011b30 tx=0x7f101001cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f0bd190a5dccad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.219+0000 7f10097fa700 1 -- 172.21.15.12:0/550948807 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101000eb20 con 0x55be84fda950 2024-05-09T00:08:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.219+0000 7f10097fa700 1 -- 172.21.15.12:0/550948807 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f101001c8d0 con 0x55be84fda950 2024-05-09T00:08:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.219+0000 7f10097fa700 1 -- 172.21.15.12:0/550948807 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010005550 con 0x55be84fda950 2024-05-09T00:08:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 -- 172.21.15.12:0/550948807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 msgr2=0x55be84fd9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 --2- 172.21.15.12:0/550948807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 0x55be84fd9390 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1010011b30 tx=0x7f101001cc90 comp rx=0 tx=0).stop 2024-05-09T00:08:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 -- 172.21.15.12:0/550948807 shutdown_connections 2024-05-09T00:08:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 --2- 172.21.15.12:0/550948807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fdb270 0x55be84fe0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 --2- 172.21.15.12:0/550948807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fcf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 --2- 172.21.15.12:0/550948807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fda950 0x55be84fd9390 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 -- 172.21.15.12:0/550948807 >> 172.21.15.12:0/550948807 conn(0x55be84f331b0 msgr2=0x55be84fcbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 -- 172.21.15.12:0/550948807 shutdown_connections 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 -- 172.21.15.12:0/550948807 wait complete. 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 -- start start 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 Processor -- start 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.220+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fe5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fda950 0x55be84fd90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f102ca1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 0x55be84fd95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f102ca1db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84fd9b30 con 0x55be84fd5480 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f102ca1db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84fdfce0 con 0x55be84fda950 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f102ca1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be84fdfea0 con 0x55be84fdb270 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f102ca1db80 10 client.0 _ll_register_callbacks cb 0x55be84ecaad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:08:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f101a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fe5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f1019d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fda950 0x55be84fd90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f101a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fe5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47612/0 (socket says 172.21.15.12:47612) 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f101ad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 0x55be84fd95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f101a59c700 1 -- 172.21.15.12:0/423235124 learned_addr learned my addr 172.21.15.12:0/423235124 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f101a59c700 1 -- 172.21.15.12:0/423235124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 msgr2=0x55be84fd95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f101a59c700 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 0x55be84fd95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f101a59c700 1 -- 172.21.15.12:0/423235124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fda950 msgr2=0x55be84fd90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f101a59c700 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fda950 0x55be84fd90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f101a59c700 1 -- 172.21.15.12:0/423235124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c1a6450 con 0x55be84fd5480 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f101a59c700 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fe5e60 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f100c096be0 tx=0x7f100c09a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552413c6928da8cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f1001ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.221+0000 7f1001ffb700 1 -- 172.21.15.12:0/423235124 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c096880 con 0x55be84fd5480 2024-05-09T00:08:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f1001ffb700 1 -- 172.21.15.12:0/423235124 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f100c0b2040 con 0x55be84fd5480 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f102ca1db80 1 -- 172.21.15.12:0/423235124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1010021070 con 0x55be84fd5480 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f1001ffb700 1 -- 172.21.15.12:0/423235124 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f100c0969e0 con 0x55be84fd5480 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f102ca1db80 10 client.21594 fetch_fsmap learned FSMap version 78 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f102ca1db80 1 -- 172.21.15.12:0/423235124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7f101002a040 con 0x55be84fd5480 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f1001ffb700 1 -- 172.21.15.12:0/423235124 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1010021070 con 0x55be84fd5480 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f1001ffb700 1 -- 172.21.15.12:0/423235124 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f100c07b070 con 0x55be84fd5480 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f102ca1db80 10 client.21594 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f102ca1db80 10 client.21594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f102ca1db80 20 client.21594 populate_metadata read hostname 'smithi012' 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f102ca1db80 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be850458b0 0x55be85047cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f102ca1db80 4 client.21594 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f102ca1db80 1 -- 172.21.15.12:0/423235124 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be84f1de60 con 0x55be850458b0 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.222+0000 7f1019d9b700 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be850458b0 0x55be85047cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.223+0000 7f1001ffb700 10 client.21594 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.223+0000 7f1019d9b700 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be850458b0 0x55be85047cd0 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee8265432b865527 server_cookie=ddb5873ffab24c68 in_seq=0 out_seq=0 2024-05-09T00:08:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:57 smithi204 ceph-mon[47373]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 723 op/s 2024-05-09T00:08:58.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:57 smithi160 ceph-mon[46152]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 723 op/s 2024-05-09T00:08:58.287 INFO:tasks.workunit.client.0.smithi012.stdout: 10 319539 30.35 MB/sec execute 413 sec latency 102.389 ms 2024-05-09T00:08:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f1001ffb700 1 -- 172.21.15.12:0/423235124 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be84f1de60 con 0x55be850458b0 2024-05-09T00:08:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f1001ffb700 10 client.21594 handle_command_reply: tid=0 2024-05-09T00:08:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 client.21594 shutdown 2024-05-09T00:08:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 -- 172.21.15.12:0/423235124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 msgr2=0x55be84fe5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fe5e60 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f100c096be0 tx=0x7f100c09a2d0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 -- 172.21.15.12:0/423235124 shutdown_connections 2024-05-09T00:08:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be84fdb270 0x55be84fd95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be84fd5480 0x55be84fe5e60 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be850458b0 0x55be85047cd0 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 --2- 172.21.15.12:0/423235124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be84fda950 0x55be84fd90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 -- 172.21.15.12:0/423235124 >> 172.21.15.12:0/423235124 conn(0x55be84f331b0 msgr2=0x55be84fea720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f1001ffb700 0 client.21594 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:08:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 -- 172.21.15.12:0/423235124 shutdown_connections 2024-05-09T00:08:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 1 -- 172.21.15.12:0/423235124 wait complete. 2024-05-09T00:08:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.637+0000 7f102ca1db80 20 client.21594 trim_cache size 0 max 16384 2024-05-09T00:08:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:08:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:08:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:08:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:08:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 -- 172.21.15.12:0/3163408010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10040a1290 msgr2=0x7f10040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10040a1290 0x7f10040a3750 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f101c102920 tx=0x7f1010009260 comp rx=0 tx=0).stop 2024-05-09T00:08:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 -- 172.21.15.12:0/3163408010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f101c0ffa50 msgr2=0x7f101c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f101c0ffa50 0x7f101c1011f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f100c00e9c0 tx=0x7f100c00ecd0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 -- 172.21.15.12:0/3163408010 shutdown_connections 2024-05-09T00:08:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f10040a1290 0x7f10040a3750 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f101c101c70 0x7f101c1a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f101c0ffa50 0x7f101c1011f0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 --2- 172.21.15.12:0/3163408010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f101c104910 0x7f101c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 -- 172.21.15.12:0/3163408010 >> 172.21.15.12:0/3163408010 conn(0x7f101c075130 msgr2=0x7f101c103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 -- 172.21.15.12:0/3163408010 shutdown_connections 2024-05-09T00:08:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:08:58.638+0000 7f1020863700 1 -- 172.21.15.12:0/3163408010 wait complete. 2024-05-09T00:08:59.288 INFO:tasks.workunit.client.0.smithi012.stdout: 10 319923 30.32 MB/sec execute 414 sec latency 353.920 ms 2024-05-09T00:09:00.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:08:59 smithi012 ceph-mon[42503]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 526 op/s 2024-05-09T00:09:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:08:59 smithi204 ceph-mon[47373]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 526 op/s 2024-05-09T00:09:00.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:08:59 smithi160 ceph-mon[46152]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 526 op/s 2024-05-09T00:09:00.288 INFO:tasks.workunit.client.0.smithi012.stdout: 10 320420 30.31 MB/sec execute 415 sec latency 137.064 ms 2024-05-09T00:09:01.288 INFO:tasks.workunit.client.0.smithi012.stdout: 10 320656 30.28 MB/sec execute 416 sec latency 425.696 ms 2024-05-09T00:09:02.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:01 smithi012 ceph-mon[42503]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 513 op/s 2024-05-09T00:09:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:01 smithi204 ceph-mon[47373]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 513 op/s 2024-05-09T00:09:02.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:01 smithi160 ceph-mon[46152]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 513 op/s 2024-05-09T00:09:02.288 INFO:tasks.workunit.client.0.smithi012.stdout: 10 320803 30.22 MB/sec execute 417 sec latency 263.035 ms 2024-05-09T00:09:03.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:02 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:09:03.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:02 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:09:03.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:02 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:09:03.288 INFO:tasks.workunit.client.0.smithi012.stdout: 10 320893 30.16 MB/sec execute 418 sec latency 193.454 ms 2024-05-09T00:09:04.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:03 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:09:04.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:03 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:09:04.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:03 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:09:04.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:03 smithi012 ceph-mon[42503]: pgmap v728: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 14 MiB/s wr, 803 op/s 2024-05-09T00:09:04.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:03 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:09:04.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:03 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:09:04.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:03 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:09:04.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:03 smithi204 ceph-mon[47373]: pgmap v728: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 14 MiB/s wr, 803 op/s 2024-05-09T00:09:04.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:03 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:09:04.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:03 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:09:04.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:03 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:09:04.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:03 smithi160 ceph-mon[46152]: pgmap v728: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 14 MiB/s wr, 803 op/s 2024-05-09T00:09:04.269 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:09:04.288 INFO:tasks.workunit.client.0.smithi012.stdout: 10 321234 30.15 MB/sec execute 419 sec latency 256.650 ms 2024-05-09T00:09:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.526+0000 7f495d9a7700 1 -- 172.21.15.12:0/832980399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4958100a10 msgr2=0x7f4958104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.526+0000 7f495d9a7700 1 --2- 172.21.15.12:0/832980399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4958100a10 0x7f4958104a30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f494c009990 tx=0x7f494c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.527+0000 7f495d9a7700 1 -- 172.21.15.12:0/832980399 shutdown_connections 2024-05-09T00:09:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.527+0000 7f495d9a7700 1 --2- 172.21.15.12:0/832980399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4958105160 0x7f4958067ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.527+0000 7f495d9a7700 1 --2- 172.21.15.12:0/832980399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4958100060 0x7f4958100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.527+0000 7f495d9a7700 1 --2- 172.21.15.12:0/832980399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4958100a10 0x7f4958104a30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.527+0000 7f495d9a7700 1 -- 172.21.15.12:0/832980399 >> 172.21.15.12:0/832980399 conn(0x7f4958076f40 msgr2=0x7f4958077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:04.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.528+0000 7f495d9a7700 1 -- 172.21.15.12:0/832980399 shutdown_connections 2024-05-09T00:09:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.528+0000 7f495d9a7700 1 -- 172.21.15.12:0/832980399 wait complete. 2024-05-09T00:09:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.529+0000 7f495d9a7700 1 Processor -- start 2024-05-09T00:09:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.533+0000 7f495d9a7700 1 -- start start 2024-05-09T00:09:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.534+0000 7f495d9a7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4958100060 0x7f4958112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.534+0000 7f495d9a7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4958105160 0x7f495810fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.534+0000 7f495d9a7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49581101f0 0x7f49581106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.534+0000 7f495d9a7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495819f890 con 0x7f4958100060 2024-05-09T00:09:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.534+0000 7f495d9a7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495819fa00 con 0x7f49581101f0 2024-05-09T00:09:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.534+0000 7f495d9a7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495819fbc0 con 0x7f4958105160 2024-05-09T00:09:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.534+0000 7f4956ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4958100060 0x7f4958112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.534+0000 7f4956ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4958100060 0x7f4958112be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52586/0 (socket says 172.21.15.12:52586) 2024-05-09T00:09:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.534+0000 7f4956ffd700 1 -- 172.21.15.12:0/1077449179 learned_addr learned my addr 172.21.15.12:0/1077449179 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.536+0000 7f4956ffd700 1 -- 172.21.15.12:0/1077449179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4958105160 msgr2=0x7f495810fcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:09:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.536+0000 7f4956ffd700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4958105160 0x7f495810fcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.536+0000 7f4956ffd700 1 -- 172.21.15.12:0/1077449179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49581101f0 msgr2=0x7f49581106a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:09:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.536+0000 7f4956ffd700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49581101f0 0x7f49581106a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.536+0000 7f4956ffd700 1 -- 172.21.15.12:0/1077449179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c0095f0 con 0x7f4958100060 2024-05-09T00:09:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.537+0000 7f4956ffd700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4958100060 0x7f4958112be0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f494800b550 tx=0x7f494800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.537+0000 7f495c9a5700 1 -- 172.21.15.12:0/1077449179 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948011830 con 0x7f4958100060 2024-05-09T00:09:04.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.537+0000 7f495c9a5700 1 -- 172.21.15.12:0/1077449179 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4948011e50 con 0x7f4958100060 2024-05-09T00:09:04.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.537+0000 7f495c9a5700 1 -- 172.21.15.12:0/1077449179 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948010e90 con 0x7f4958100060 2024-05-09T00:09:04.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.537+0000 7f495d9a7700 1 -- 172.21.15.12:0/1077449179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495819fea0 con 0x7f4958100060 2024-05-09T00:09:04.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.537+0000 7f495d9a7700 1 -- 172.21.15.12:0/1077449179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49581a0360 con 0x7f4958100060 2024-05-09T00:09:04.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.542+0000 7f495c9a5700 1 -- 172.21.15.12:0/1077449179 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f49480104f0 con 0x7f4958100060 2024-05-09T00:09:04.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.543+0000 7f495c9a5700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f49400a1460 0x7f49400a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.543+0000 7f495c9a5700 1 -- 172.21.15.12:0/1077449179 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f49480cc730 con 0x7f4958100060 2024-05-09T00:09:04.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.543+0000 7f4969b61b80 1 Processor -- start 2024-05-09T00:09:04.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.544+0000 7f49567fc700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f49400a1460 0x7f49400a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.544+0000 7f4969b61b80 1 -- start start 2024-05-09T00:09:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.545+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bccc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.545+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccc720 0x56332bcc6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.545+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bcc6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.545+0000 7f4969b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bc1d150 con 0x56332bcc6890 2024-05-09T00:09:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.545+0000 7f4969b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bc1ab10 con 0x56332bccbe00 2024-05-09T00:09:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.545+0000 7f4969b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bc16400 con 0x56332bccc720 2024-05-09T00:09:04.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.545+0000 7f4956ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bccc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.545+0000 7f4956ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bccc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:51416/0 (socket says 172.21.15.12:51416) 2024-05-09T00:09:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.545+0000 7f4956ffd700 1 -- 172.21.15.12:0/2907610529 learned_addr learned my addr 172.21.15.12:0/2907610529 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.546+0000 7f49567fc700 1 --2- 172.21.15.12:0/2907610529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccc720 0x56332bcc6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.546+0000 7f4956ffd700 1 -- 172.21.15.12:0/2907610529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccc720 msgr2=0x56332bcc6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:04.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.546+0000 7f4956ffd700 1 --2- 172.21.15.12:0/2907610529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccc720 0x56332bcc6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.546+0000 7f4956ffd700 1 -- 172.21.15.12:0/2907610529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 msgr2=0x56332bcc6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.546+0000 7f4956ffd700 1 --2- 172.21.15.12:0/2907610529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bcc6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.546+0000 7f4956ffd700 1 -- 172.21.15.12:0/2907610529 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56332ba973b0 con 0x56332bccbe00 2024-05-09T00:09:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.546+0000 7f49567fc700 1 --2- 172.21.15.12:0/2907610529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccc720 0x56332bcc6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:09:04.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.546+0000 7f4956ffd700 1 --2- 172.21.15.12:0/2907610529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bccc1e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f49480a1830 tx=0x7f4948099370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b46b68d89db5de56 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.547+0000 7f493dffb700 1 -- 172.21.15.12:0/2907610529 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494809d710 con 0x56332bccbe00 2024-05-09T00:09:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.547+0000 7f493dffb700 1 -- 172.21.15.12:0/2907610529 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49480a1d60 con 0x56332bccbe00 2024-05-09T00:09:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.547+0000 7f493dffb700 1 -- 172.21.15.12:0/2907610529 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49480a1ad0 con 0x56332bccbe00 2024-05-09T00:09:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.548+0000 7f4969b61b80 1 -- 172.21.15.12:0/2907610529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 msgr2=0x56332bccc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.548+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2907610529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bccc1e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f49480a1830 tx=0x7f4948099370 comp rx=0 tx=0).stop 2024-05-09T00:09:04.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.548+0000 7f49567fc700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f49400a1460 0x7f49400a3920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f494c00b6f0 tx=0x7f494c005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:04.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.550+0000 7f4969b61b80 1 -- 172.21.15.12:0/2907610529 shutdown_connections 2024-05-09T00:09:04.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.550+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2907610529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bcc6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.550+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2907610529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bccc1e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.550+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2907610529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccc720 0x56332bcc6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.550+0000 7f4969b61b80 1 -- 172.21.15.12:0/2907610529 >> 172.21.15.12:0/2907610529 conn(0x56332bc24210 msgr2=0x56332bcc0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:04.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.553+0000 7f4969b61b80 1 -- 172.21.15.12:0/2907610529 shutdown_connections 2024-05-09T00:09:04.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.553+0000 7f4969b61b80 1 -- 172.21.15.12:0/2907610529 wait complete. 2024-05-09T00:09:04.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.553+0000 7f4969b61b80 1 -- start start 2024-05-09T00:09:04.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.554+0000 7f4969b61b80 1 Processor -- start 2024-05-09T00:09:04.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.554+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bccfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.554+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bcd00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.554+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccc720 0x56332bcd0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.554+0000 7f4969b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bbb2560 con 0x56332bcc6890 2024-05-09T00:09:04.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.554+0000 7f4969b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bbb2200 con 0x56332bccbe00 2024-05-09T00:09:04.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.554+0000 7f4969b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332ba996a0 con 0x56332bccc720 2024-05-09T00:09:04.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.554+0000 7f4969b61b80 10 client.0 _ll_register_callbacks cb 0x56332bbbbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:04.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.555+0000 7f49567fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bcd00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:04.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.555+0000 7f49567fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bcd00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:51424/0 (socket says 172.21.15.12:51424) 2024-05-09T00:09:04.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.555+0000 7f49567fc700 1 -- 172.21.15.12:0/2301040246 learned_addr learned my addr 172.21.15.12:0/2301040246 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:04.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.555+0000 7f4956ffd700 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bccfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:04.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.555+0000 7f49567fc700 1 -- 172.21.15.12:0/2301040246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccc720 msgr2=0x56332bcd0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:04.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.555+0000 7f49567fc700 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccc720 0x56332bcd0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.555+0000 7f49567fc700 1 -- 172.21.15.12:0/2301040246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 msgr2=0x56332bccfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:04.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.555+0000 7f49567fc700 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bccfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.555+0000 7f49567fc700 1 -- 172.21.15.12:0/2301040246 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49581a0360 con 0x56332bccbe00 2024-05-09T00:09:04.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.556+0000 7f49567fc700 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bcd00c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f494c015000 tx=0x7f494c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d81703dc3dc9907 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:04.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.556+0000 7f4936ffd700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:09:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.560+0000 7f4936ffd700 1 -- 172.21.15.12:0/2301040246 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c02a070 con 0x56332bccbe00 2024-05-09T00:09:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.560+0000 7f4936ffd700 1 -- 172.21.15.12:0/2301040246 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f494c00fa00 con 0x56332bccbe00 2024-05-09T00:09:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.560+0000 7f4969b61b80 1 -- 172.21.15.12:0/2301040246 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56332bb6f270 con 0x56332bccbe00 2024-05-09T00:09:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.560+0000 7f4936ffd700 1 -- 172.21.15.12:0/2301040246 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f494c01d070 con 0x56332bccbe00 2024-05-09T00:09:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.560+0000 7f4969b61b80 10 client.31049 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.560+0000 7f4969b61b80 1 -- 172.21.15.12:0/2301040246 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x56332ba43b60 con 0x56332bccbe00 2024-05-09T00:09:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.561+0000 7f4936ffd700 1 -- 172.21.15.12:0/2301040246 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56332bb6f270 con 0x56332bccbe00 2024-05-09T00:09:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.561+0000 7f4936ffd700 1 -- 172.21.15.12:0/2301040246 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f494c036070 con 0x56332bccbe00 2024-05-09T00:09:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.561+0000 7f4969b61b80 10 client.31049 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.561+0000 7f4969b61b80 10 client.31049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.561+0000 7f4969b61b80 20 client.31049 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.561+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56332bcf11a0 0x56332bcf35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.561+0000 7f4969b61b80 4 client.31049 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.561+0000 7f4969b61b80 1 -- 172.21.15.12:0/2301040246 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56332bc95810 con 0x56332bcf11a0 2024-05-09T00:09:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.566+0000 7f4956ffd700 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56332bcf11a0 0x56332bcf35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.571+0000 7f4956ffd700 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56332bcf11a0 0x56332bcf35c0 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63fb4bf7fe858c7d server_cookie=5f9b4fda1cc34d54 in_seq=0 out_seq=0 2024-05-09T00:09:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.571+0000 7f4936ffd700 10 client.31049 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.576+0000 7f4936ffd700 1 -- 172.21.15.12:0/2301040246 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56332bc95810 con 0x56332bcf11a0 2024-05-09T00:09:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.576+0000 7f4936ffd700 10 client.31049 handle_command_reply: tid=0 2024-05-09T00:09:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.576+0000 7f4969b61b80 1 client.31049 shutdown 2024-05-09T00:09:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.578+0000 7f4969b61b80 1 -- 172.21.15.12:0/2301040246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 msgr2=0x56332bcd00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.578+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bcd00c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f494c015000 tx=0x7f494c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.578+0000 7f4969b61b80 1 -- 172.21.15.12:0/2301040246 shutdown_connections 2024-05-09T00:09:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.578+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccc720 0x56332bcd0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.578+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bccfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.578+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56332bcf11a0 0x56332bcf35c0 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.578+0000 7f4936ffd700 0 client.31049 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.578+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2301040246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bcd00c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.578+0000 7f4969b61b80 1 -- 172.21.15.12:0/2301040246 >> 172.21.15.12:0/2301040246 conn(0x56332bc24210 msgr2=0x56332bcc05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:04.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.578+0000 7f4969b61b80 1 -- 172.21.15.12:0/2301040246 shutdown_connections 2024-05-09T00:09:04.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.582+0000 7f4969b61b80 1 -- 172.21.15.12:0/2301040246 wait complete. 2024-05-09T00:09:04.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.583+0000 7f4969b61b80 20 client.31049 trim_cache size 0 max 16384 2024-05-09T00:09:04.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.629+0000 7f4969b61b80 1 Processor -- start 2024-05-09T00:09:04.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.629+0000 7f4969b61b80 1 -- start start 2024-05-09T00:09:04.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.629+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bcc6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.629+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccbe00 0x56332bcdbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.629+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccc720 0x56332bcc0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4969b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bcd26a0 con 0x56332bcc6890 2024-05-09T00:09:04.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4969b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bcedd20 con 0x56332bccc720 2024-05-09T00:09:04.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4969b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bcf8800 con 0x56332bccbe00 2024-05-09T00:09:04.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4956ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bcc6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:04.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4956ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bcc6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52614/0 (socket says 172.21.15.12:52614) 2024-05-09T00:09:04.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4956ffd700 1 -- 172.21.15.12:0/2471560358 learned_addr learned my addr 172.21.15.12:0/2471560358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:04.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4956ffd700 1 -- 172.21.15.12:0/2471560358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccbe00 msgr2=0x56332bcdbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:04.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4956ffd700 1 --2- 172.21.15.12:0/2471560358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccbe00 0x56332bcdbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4956ffd700 1 -- 172.21.15.12:0/2471560358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccc720 msgr2=0x56332bcc0d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:09:04.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4956ffd700 1 --2- 172.21.15.12:0/2471560358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccc720 0x56332bcc0d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.630+0000 7f4956ffd700 1 -- 172.21.15.12:0/2471560358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56332bbdc560 con 0x56332bcc6890 2024-05-09T00:09:04.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.631+0000 7f4956ffd700 1 --2- 172.21.15.12:0/2471560358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bcc6350 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f4948099590 tx=0x7f49480cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec9fff7c4635ab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.637+0000 7f493e7fc700 1 -- 172.21.15.12:0/2471560358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948095c20 con 0x56332bcc6890 2024-05-09T00:09:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.637+0000 7f493e7fc700 1 -- 172.21.15.12:0/2471560358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49480b2130 con 0x56332bcc6890 2024-05-09T00:09:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.637+0000 7f493e7fc700 1 -- 172.21.15.12:0/2471560358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49480a18d0 con 0x56332bcc6890 2024-05-09T00:09:04.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.638+0000 7f4969b61b80 1 -- 172.21.15.12:0/2471560358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 msgr2=0x56332bcc6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:04.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.638+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2471560358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bcc6350 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f4948099590 tx=0x7f49480cc040 comp rx=0 tx=0).stop 2024-05-09T00:09:04.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.643+0000 7f4969b61b80 1 -- 172.21.15.12:0/2471560358 shutdown_connections 2024-05-09T00:09:04.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.643+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2471560358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccc720 0x56332bcc0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.643+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2471560358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bcc6890 0x56332bcc6350 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.643+0000 7f4969b61b80 1 --2- 172.21.15.12:0/2471560358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bccbe00 0x56332bcdbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.643+0000 7f4969b61b80 1 -- 172.21.15.12:0/2471560358 >> 172.21.15.12:0/2471560358 conn(0x56332bc24210 msgr2=0x56332bcce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.645+0000 7f4969b61b80 1 -- 172.21.15.12:0/2471560358 shutdown_connections 2024-05-09T00:09:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.646+0000 7f4969b61b80 1 -- 172.21.15.12:0/2471560358 wait complete. 2024-05-09T00:09:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.646+0000 7f4969b61b80 1 -- start start 2024-05-09T00:09:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.646+0000 7f4969b61b80 1 Processor -- start 2024-05-09T00:09:04.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bcc6890 0x56332bcd5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bcd5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f4969b61b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bccc720 0x56332bd466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f4969b61b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bcd5e70 con 0x56332bccc720 2024-05-09T00:09:04.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f4969b61b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bcd5fe0 con 0x56332bccbe00 2024-05-09T00:09:04.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f4969b61b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332bd46c30 con 0x56332bcc6890 2024-05-09T00:09:04.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f4969b61b80 10 client.0 _ll_register_callbacks cb 0x56332bbbbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:04.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f49577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bccc720 0x56332bd466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:04.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f49577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bccc720 0x56332bd466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52624/0 (socket says 172.21.15.12:52624) 2024-05-09T00:09:04.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f49577fe700 1 -- 172.21.15.12:0/1210746575 learned_addr learned my addr 172.21.15.12:0/1210746575 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:04.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f49577fe700 1 -- 172.21.15.12:0/1210746575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bcc6890 msgr2=0x56332bcd5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:04.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f49577fe700 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bcc6890 0x56332bcd5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f49577fe700 1 -- 172.21.15.12:0/1210746575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 msgr2=0x56332bcd5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:04.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f49577fe700 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bcd5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.647+0000 7f49577fe700 1 -- 172.21.15.12:0/1210746575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56332ba973b0 con 0x56332bccc720 2024-05-09T00:09:04.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.648+0000 7f49577fe700 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bccc720 0x56332bd466a0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f495000ec50 tx=0x7f495000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d6ce0e438b12fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:04.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.648+0000 7f4936ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:04.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.648+0000 7f4936ffd700 1 -- 172.21.15.12:0/1210746575 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950009310 con 0x56332bccc720 2024-05-09T00:09:04.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.648+0000 7f4969b61b80 1 -- 172.21.15.12:0/1210746575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49480a1d60 con 0x56332bccc720 2024-05-09T00:09:04.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.648+0000 7f4936ffd700 1 -- 172.21.15.12:0/1210746575 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4950004ad0 con 0x56332bccc720 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.648+0000 7f4936ffd700 1 -- 172.21.15.12:0/1210746575 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4950004cd0 con 0x56332bccc720 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.648+0000 7f4969b61b80 10 client.21609 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.648+0000 7f4969b61b80 1 -- 172.21.15.12:0/1210746575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x56332ba9bcd0 con 0x56332bccc720 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.648+0000 7f4936ffd700 1 -- 172.21.15.12:0/1210746575 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49480a1d60 con 0x56332bccc720 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.649+0000 7f4936ffd700 1 -- 172.21.15.12:0/1210746575 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f495001b070 con 0x56332bccc720 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.649+0000 7f4969b61b80 10 client.21609 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.649+0000 7f4969b61b80 10 client.21609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.649+0000 7f4969b61b80 20 client.21609 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.650+0000 7f4969b61b80 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56332bcf11a0 0x56332bd4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.650+0000 7f4969b61b80 4 client.21609 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.650+0000 7f4969b61b80 1 -- 172.21.15.12:0/1210746575 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56332bc0eec0 con 0x56332bcf11a0 2024-05-09T00:09:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.650+0000 7f4956ffd700 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56332bcf11a0 0x56332bd4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.650+0000 7f4956ffd700 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56332bcf11a0 0x56332bd4a4b0 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76a2b1298be584c server_cookie=332c04f3d1430252 in_seq=0 out_seq=0 2024-05-09T00:09:04.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.650+0000 7f4936ffd700 10 client.21609 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:04.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.666+0000 7f4936ffd700 1 -- 172.21.15.12:0/1210746575 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56332bc0eec0 con 0x56332bcf11a0 2024-05-09T00:09:04.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.666+0000 7f4936ffd700 10 client.21609 handle_command_reply: tid=0 2024-05-09T00:09:04.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.666+0000 7f4969b61b80 1 client.21609 shutdown 2024-05-09T00:09:04.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.667+0000 7f4969b61b80 1 -- 172.21.15.12:0/1210746575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bccc720 msgr2=0x56332bd466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:04.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.667+0000 7f4969b61b80 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bccc720 0x56332bd466a0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f495000ec50 tx=0x7f495000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.667+0000 7f4969b61b80 1 -- 172.21.15.12:0/1210746575 shutdown_connections 2024-05-09T00:09:04.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.667+0000 7f4969b61b80 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56332bccc720 0x56332bd466a0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.667+0000 7f4969b61b80 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56332bcc6890 0x56332bcd5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.667+0000 7f4969b61b80 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56332bcf11a0 0x56332bd4a4b0 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.667+0000 7f4969b61b80 1 --2- 172.21.15.12:0/1210746575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56332bccbe00 0x56332bcd5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.667+0000 7f4969b61b80 1 -- 172.21.15.12:0/1210746575 >> 172.21.15.12:0/1210746575 conn(0x56332bc24210 msgr2=0x56332bccac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:04.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.669+0000 7f4936ffd700 0 client.21609 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:04.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.671+0000 7f4969b61b80 1 -- 172.21.15.12:0/1210746575 shutdown_connections 2024-05-09T00:09:04.670 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:09:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:09:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:09:04.671 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:09:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.673+0000 7f4969b61b80 1 -- 172.21.15.12:0/1210746575 wait complete. 2024-05-09T00:09:04.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.674+0000 7f4969b61b80 20 client.21609 trim_cache size 0 max 16384 2024-05-09T00:09:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.680+0000 7f495d9a7700 1 -- 172.21.15.12:0/1077449179 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f49400a1460 msgr2=0x7f49400a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.680+0000 7f495d9a7700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f49400a1460 0x7f49400a3920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f494c00b6f0 tx=0x7f494c005fb0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.680+0000 7f495d9a7700 1 -- 172.21.15.12:0/1077449179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4958100060 msgr2=0x7f4958112be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.680+0000 7f495d9a7700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4958100060 0x7f4958112be0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f494800b550 tx=0x7f494800b910 comp rx=0 tx=0).stop 2024-05-09T00:09:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.687+0000 7f495d9a7700 1 -- 172.21.15.12:0/1077449179 shutdown_connections 2024-05-09T00:09:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.687+0000 7f495d9a7700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f49400a1460 0x7f49400a3920 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.687+0000 7f495d9a7700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49581101f0 0x7f49581106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.687+0000 7f495d9a7700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4958100060 0x7f4958112be0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.687+0000 7f495d9a7700 1 --2- 172.21.15.12:0/1077449179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4958105160 0x7f495810fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:04.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.687+0000 7f495d9a7700 1 -- 172.21.15.12:0/1077449179 >> 172.21.15.12:0/1077449179 conn(0x7f4958076f40 msgr2=0x7f49580feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:04.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.687+0000 7f495d9a7700 1 -- 172.21.15.12:0/1077449179 shutdown_connections 2024-05-09T00:09:04.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:04.687+0000 7f495d9a7700 1 -- 172.21.15.12:0/1077449179 wait complete. 2024-05-09T00:09:04.711 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:09:04.712 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:09:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.052+0000 7f9bff1d7700 1 -- 172.21.15.12:0/632242367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf8103280 msgr2=0x7f9bf8103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.052+0000 7f9bff1d7700 1 --2- 172.21.15.12:0/632242367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf8103280 0x7f9bf8103660 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009990 tx=0x7f9be8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.053+0000 7f9bff1d7700 1 -- 172.21.15.12:0/632242367 shutdown_connections 2024-05-09T00:09:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.053+0000 7f9bff1d7700 1 --2- 172.21.15.12:0/632242367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bf81083a0 0x7f9bf810ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.053+0000 7f9bff1d7700 1 --2- 172.21.15.12:0/632242367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf8103280 0x7f9bf8103660 secure :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009990 tx=0x7f9be8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.053+0000 7f9bff1d7700 1 --2- 172.21.15.12:0/632242367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bf8103c30 0x7f9bf8107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.053+0000 7f9bff1d7700 1 -- 172.21.15.12:0/632242367 >> 172.21.15.12:0/632242367 conn(0x7f9bf80feb20 msgr2=0x7f9bf8100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.072+0000 7f9bff1d7700 1 -- 172.21.15.12:0/632242367 shutdown_connections 2024-05-09T00:09:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.075+0000 7f9bff1d7700 1 -- 172.21.15.12:0/632242367 wait complete. 2024-05-09T00:09:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.075+0000 7f9bff1d7700 1 Processor -- start 2024-05-09T00:09:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.076+0000 7f9bff1d7700 1 -- start start 2024-05-09T00:09:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.076+0000 7f9bff1d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf81083a0 0x7f9bf819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.076+0000 7f9bff1d7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bf819bad0 0x7f9bf81956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.076+0000 7f9bff1d7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bf8195c30 0x7f9bf81960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.076+0000 7f9bff1d7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf819c130 con 0x7f9bf81083a0 2024-05-09T00:09:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.076+0000 7f9bff1d7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf806a9c0 con 0x7f9bf8195c30 2024-05-09T00:09:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.076+0000 7f9bff1d7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf806ab80 con 0x7f9bf819bad0 2024-05-09T00:09:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.077+0000 7f9bf7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bf819bad0 0x7f9bf81956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.077+0000 7f9bf7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bf819bad0 0x7f9bf81956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42026/0 (socket says 172.21.15.12:42026) 2024-05-09T00:09:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.077+0000 7f9bf7fff700 1 -- 172.21.15.12:0/3828489824 learned_addr learned my addr 172.21.15.12:0/3828489824 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.077+0000 7f9bf7fff700 1 -- 172.21.15.12:0/3828489824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bf8195c30 msgr2=0x7f9bf81960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.077+0000 7f9bf7fff700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bf8195c30 0x7f9bf81960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.077+0000 7f9bf7fff700 1 -- 172.21.15.12:0/3828489824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf81083a0 msgr2=0x7f9bf819b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.077+0000 7f9bf7fff700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf81083a0 0x7f9bf819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.077+0000 7f9bf7fff700 1 -- 172.21.15.12:0/3828489824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80095f0 con 0x7f9bf819bad0 2024-05-09T00:09:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.078+0000 7f9bf7fff700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bf819bad0 0x7f9bf81956f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000b9e0 tx=0x7f9bf000bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:05.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.095+0000 7f9bf5ffb700 1 -- 172.21.15.12:0/3828489824 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf000ca30 con 0x7f9bf819bad0 2024-05-09T00:09:05.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.095+0000 7f9bf5ffb700 1 -- 172.21.15.12:0/3828489824 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bf000cb90 con 0x7f9bf819bad0 2024-05-09T00:09:05.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.095+0000 7f9bf5ffb700 1 -- 172.21.15.12:0/3828489824 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00125a0 con 0x7f9bf819bad0 2024-05-09T00:09:05.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.098+0000 7f9bff1d7700 1 -- 172.21.15.12:0/3828489824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf806ae60 con 0x7f9bf819bad0 2024-05-09T00:09:05.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.098+0000 7f9bff1d7700 1 -- 172.21.15.12:0/3828489824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf806b2f0 con 0x7f9bf819bad0 2024-05-09T00:09:05.098 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:05 smithi012 ceph-mon[42503]: osdmap e131: 12 total, 12 up, 12 in 2024-05-09T00:09:05.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.102+0000 7f9bff1d7700 1 -- 172.21.15.12:0/3828489824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf8104750 con 0x7f9bf819bad0 2024-05-09T00:09:05.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.107+0000 7f9bf5ffb700 1 -- 172.21.15.12:0/3828489824 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9bf000cd00 con 0x7f9bf819bad0 2024-05-09T00:09:05.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.108+0000 7f9bf5ffb700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9be00a9cb0 0x7f9be00ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:05.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.108+0000 7f9bf5ffb700 1 -- 172.21.15.12:0/3828489824 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9bf00cc810 con 0x7f9bf819bad0 2024-05-09T00:09:05.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.109+0000 7f9bfcf73700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9be00a9cb0 0x7f9be00ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:05.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.110+0000 7f9bfcf73700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9be00a9cb0 0x7f9be00ac170 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006010 tx=0x7f9be8005c40 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:05.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.112+0000 7f9bf5ffb700 1 -- 172.21.15.12:0/3828489824 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9bf0095d90 con 0x7f9bf819bad0 2024-05-09T00:09:05.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 321374 30.09 MB/sec execute 420 sec latency 850.607 ms 2024-05-09T00:09:05.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:05 smithi204 ceph-mon[47373]: osdmap e131: 12 total, 12 up, 12 in 2024-05-09T00:09:05.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:05 smithi160 ceph-mon[46152]: osdmap e131: 12 total, 12 up, 12 in 2024-05-09T00:09:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.555+0000 7f9bff1d7700 1 -- 172.21.15.12:0/3828489824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bf806acc0 con 0x7f9bf819bad0 2024-05-09T00:09:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.560+0000 7f9bf5ffb700 1 -- 172.21.15.12:0/3828489824 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+10873 (secure 0 0 0) 0x7f9bf806acc0 con 0x7f9bf819bad0 2024-05-09T00:09:05.561 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:09:05.561 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":21435,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/676326208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":676326208},{"type":"v1","addr":"172.21.15.160:6833","nonce":676326208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":78,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:08:19.008049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:09:05.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.569+0000 7f9bff1d7700 1 -- 172.21.15.12:0/3828489824 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9be00a9cb0 msgr2=0x7f9be00ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:05.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.569+0000 7f9bff1d7700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9be00a9cb0 0x7f9be00ac170 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006010 tx=0x7f9be8005c40 comp rx=0 tx=0).stop 2024-05-09T00:09:05.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.569+0000 7f9bff1d7700 1 -- 172.21.15.12:0/3828489824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bf819bad0 msgr2=0x7f9bf81956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:05.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.569+0000 7f9bff1d7700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bf819bad0 0x7f9bf81956f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000b9e0 tx=0x7f9bf000bda0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.572+0000 7f9bff1d7700 1 -- 172.21.15.12:0/3828489824 shutdown_connections 2024-05-09T00:09:05.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.572+0000 7f9bff1d7700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9be00a9cb0 0x7f9be00ac170 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.572+0000 7f9bff1d7700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bf8195c30 0x7f9bf81960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.572+0000 7f9bff1d7700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bf81083a0 0x7f9bf819b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.572+0000 7f9bff1d7700 1 --2- 172.21.15.12:0/3828489824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bf819bad0 0x7f9bf81956f0 secure :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000b9e0 tx=0x7f9bf000bda0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.572+0000 7f9bff1d7700 1 -- 172.21.15.12:0/3828489824 >> 172.21.15.12:0/3828489824 conn(0x7f9bf80feb20 msgr2=0x7f9bf8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:05.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.579+0000 7f9bff1d7700 1 -- 172.21.15.12:0/3828489824 shutdown_connections 2024-05-09T00:09:05.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.582+0000 7f9bff1d7700 1 -- 172.21.15.12:0/3828489824 wait complete. 2024-05-09T00:09:05.591 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 78 2024-05-09T00:09:05.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:09:05.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.964+0000 7f873287d700 1 -- 172.21.15.12:0/3576862828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c11a780 msgr2=0x7f872c11cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:05.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.964+0000 7f873287d700 1 --2- 172.21.15.12:0/3576862828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c11a780 0x7f872c11cb70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8720009990 tx=0x7f8720009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.970+0000 7f873287d700 1 -- 172.21.15.12:0/3576862828 shutdown_connections 2024-05-09T00:09:05.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.970+0000 7f873287d700 1 --2- 172.21.15.12:0/3576862828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c11a780 0x7f872c11cb70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.970+0000 7f873287d700 1 --2- 172.21.15.12:0/3576862828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c10f6a0 0x7f872c107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.970+0000 7f873287d700 1 --2- 172.21.15.12:0/3576862828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10fb10 0x7f872c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.970+0000 7f873287d700 1 -- 172.21.15.12:0/3576862828 >> 172.21.15.12:0/3576862828 conn(0x7f872c06bff0 msgr2=0x7f872c06c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:05.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.981+0000 7f873287d700 1 -- 172.21.15.12:0/3576862828 shutdown_connections 2024-05-09T00:09:05.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.982+0000 7f873287d700 1 -- 172.21.15.12:0/3576862828 wait complete. 2024-05-09T00:09:05.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.983+0000 7f873287d700 1 Processor -- start 2024-05-09T00:09:05.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.983+0000 7f873287d700 1 -- start start 2024-05-09T00:09:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.984+0000 7f873287d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10f6a0 0x7f872c118c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.984+0000 7f873287d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c10fb10 0x7f872c119190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.984+0000 7f873287d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c11a780 0x7f872c111e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.984+0000 7f873287d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c1125d0 con 0x7f872c10f6a0 2024-05-09T00:09:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.984+0000 7f873287d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c112740 con 0x7f872c10fb10 2024-05-09T00:09:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.984+0000 7f873287d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c112900 con 0x7f872c11a780 2024-05-09T00:09:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.984+0000 7f8730e1a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c11a780 0x7f872c111e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.984+0000 7f8730e1a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c11a780 0x7f872c111e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42030/0 (socket says 172.21.15.12:42030) 2024-05-09T00:09:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.984+0000 7f8730e1a700 1 -- 172.21.15.12:0/712023443 learned_addr learned my addr 172.21.15.12:0/712023443 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:05.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.985+0000 7f8730e1a700 1 -- 172.21.15.12:0/712023443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c10fb10 msgr2=0x7f872c119190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:09:05.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.985+0000 7f8730e1a700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c10fb10 0x7f872c119190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.985+0000 7f8730e1a700 1 -- 172.21.15.12:0/712023443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10f6a0 msgr2=0x7f872c118c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:09:05.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.985+0000 7f8730e1a700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10f6a0 0x7f872c118c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:05.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.985+0000 7f8730e1a700 1 -- 172.21.15.12:0/712023443 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87200095f0 con 0x7f872c11a780 2024-05-09T00:09:05.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.986+0000 7f8730e1a700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c11a780 0x7f872c111e00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f87200098e0 tx=0x7f8720003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:05.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.986+0000 7f87297fa700 1 -- 172.21.15.12:0/712023443 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872001d070 con 0x7f872c11a780 2024-05-09T00:09:05.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.986+0000 7f873287d700 1 -- 172.21.15.12:0/712023443 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c1167a0 con 0x7f872c11a780 2024-05-09T00:09:05.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.986+0000 7f873287d700 1 -- 172.21.15.12:0/712023443 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872c116ca0 con 0x7f872c11a780 2024-05-09T00:09:05.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.986+0000 7f87297fa700 1 -- 172.21.15.12:0/712023443 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8720005580 con 0x7f872c11a780 2024-05-09T00:09:05.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.986+0000 7f87297fa700 1 -- 172.21.15.12:0/712023443 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000fe30 con 0x7f872c11a780 2024-05-09T00:09:05.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.994+0000 7f873ea37b80 1 Processor -- start 2024-05-09T00:09:05.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.995+0000 7f873ea37b80 1 -- start start 2024-05-09T00:09:05.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.995+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d0b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:05.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.995+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0b720 0x55fcc5d05350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:05.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.995+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d05cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:05.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.995+0000 7f873ea37b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5c5c150 con 0x55fcc5d05890 2024-05-09T00:09:05.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.995+0000 7f873ea37b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5c59b10 con 0x55fcc5d0b720 2024-05-09T00:09:05.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:05.995+0000 7f873ea37b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5c55400 con 0x55fcc5d0ae00 2024-05-09T00:09:06.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.002+0000 7f87297fa700 1 -- 172.21.15.12:0/712023443 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8720005050 con 0x7f872c11a780 2024-05-09T00:09:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.003+0000 7f87297fa700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f87140a1390 0x7f87140a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.003+0000 7f8730e1a700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f87140a1390 0x7f87140a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.004+0000 7f87297fa700 1 -- 172.21.15.12:0/712023443 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f87200cde30 con 0x7f872c11a780 2024-05-09T00:09:06.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.007+0000 7f8730e1a700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f87140a1390 0x7f87140a3850 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8720026000 tx=0x7f872000f4e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.010+0000 7f872bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0b720 0x55fcc5d05350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.010+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d0b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.010+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d0b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42042/0 (socket says 172.21.15.12:42042) 2024-05-09T00:09:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.010+0000 7f872b7fe700 1 -- 172.21.15.12:0/1281815958 learned_addr learned my addr 172.21.15.12:0/1281815958 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.010+0000 7f872b7fe700 1 --2- 172.21.15.12:0/1281815958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d05cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.017+0000 7f872bfff700 1 -- 172.21.15.12:0/1281815958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0ae00 msgr2=0x55fcc5d0b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.017+0000 7f872bfff700 1 --2- 172.21.15.12:0/1281815958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d0b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.017+0000 7f872bfff700 1 -- 172.21.15.12:0/1281815958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 msgr2=0x55fcc5d05cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.017+0000 7f872bfff700 1 --2- 172.21.15.12:0/1281815958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d05cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.017+0000 7f872bfff700 1 -- 172.21.15.12:0/1281815958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcc5ad63b0 con 0x55fcc5d0b720 2024-05-09T00:09:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.019+0000 7f872bfff700 1 --2- 172.21.15.12:0/1281815958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0b720 0x55fcc5d05350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f871c009740 tx=0x7f871c009a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9272225ebcc0baf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.024+0000 7f87127fc700 1 -- 172.21.15.12:0/1281815958 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c011830 con 0x55fcc5d0b720 2024-05-09T00:09:06.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.024+0000 7f87127fc700 1 -- 172.21.15.12:0/1281815958 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f871c011e50 con 0x55fcc5d0b720 2024-05-09T00:09:06.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.024+0000 7f87127fc700 1 -- 172.21.15.12:0/1281815958 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c010de0 con 0x55fcc5d0b720 2024-05-09T00:09:06.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.025+0000 7f873ea37b80 1 -- 172.21.15.12:0/1281815958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0b720 msgr2=0x55fcc5d05350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.025+0000 7f873ea37b80 1 --2- 172.21.15.12:0/1281815958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0b720 0x55fcc5d05350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f871c009740 tx=0x7f871c009a50 comp rx=0 tx=0).stop 2024-05-09T00:09:06.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.034+0000 7f873ea37b80 1 -- 172.21.15.12:0/1281815958 shutdown_connections 2024-05-09T00:09:06.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.034+0000 7f873ea37b80 1 --2- 172.21.15.12:0/1281815958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d05cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.034+0000 7f873ea37b80 1 --2- 172.21.15.12:0/1281815958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d0b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.034+0000 7f873ea37b80 1 --2- 172.21.15.12:0/1281815958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0b720 0x55fcc5d05350 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.034+0000 7f873ea37b80 1 -- 172.21.15.12:0/1281815958 >> 172.21.15.12:0/1281815958 conn(0x55fcc5c63210 msgr2=0x55fcc5cffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.039+0000 7f873ea37b80 1 -- 172.21.15.12:0/1281815958 shutdown_connections 2024-05-09T00:09:06.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.043+0000 7f873ea37b80 1 -- 172.21.15.12:0/1281815958 wait complete. 2024-05-09T00:09:06.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.043+0000 7f873ea37b80 1 -- start start 2024-05-09T00:09:06.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.044+0000 7f873ea37b80 1 Processor -- start 2024-05-09T00:09:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.045+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d0eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.046+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d0f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.046+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0b720 0x55fcc5d0f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.046+0000 7f873ea37b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5bf1560 con 0x55fcc5d05890 2024-05-09T00:09:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.046+0000 7f873ea37b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5bf1200 con 0x55fcc5d0ae00 2024-05-09T00:09:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.046+0000 7f873ea37b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5ad86a0 con 0x55fcc5d0b720 2024-05-09T00:09:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.046+0000 7f873ea37b80 10 client.0 _ll_register_callbacks cb 0x55fcc5bfab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.046+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d0eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.046+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d0eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52678/0 (socket says 172.21.15.12:52678) 2024-05-09T00:09:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.046+0000 7f872b7fe700 1 -- 172.21.15.12:0/3755860115 learned_addr learned my addr 172.21.15.12:0/3755860115 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.047+0000 7f872b7fe700 1 -- 172.21.15.12:0/3755860115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0b720 msgr2=0x55fcc5d0f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.047+0000 7f872b7fe700 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0b720 0x55fcc5d0f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.047+0000 7f872b7fe700 1 -- 172.21.15.12:0/3755860115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0ae00 msgr2=0x55fcc5d0f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.047+0000 7f872b7fe700 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d0f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.047+0000 7f872b7fe700 1 -- 172.21.15.12:0/3755860115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c009370 con 0x55fcc5d05890 2024-05-09T00:09:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.047+0000 7f872b7fe700 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d0eb30 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8724013f80 tx=0x7f8724011560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf63ae93f5299800 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.047+0000 7f870b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:06.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.048+0000 7f870b7fe700 1 -- 172.21.15.12:0/3755860115 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87240070b0 con 0x55fcc5d05890 2024-05-09T00:09:06.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.048+0000 7f870b7fe700 1 -- 172.21.15.12:0/3755860115 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8724007210 con 0x55fcc5d05890 2024-05-09T00:09:06.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.048+0000 7f873ea37b80 1 -- 172.21.15.12:0/3755860115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcc5bae270 con 0x55fcc5d05890 2024-05-09T00:09:06.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.052+0000 7f870b7fe700 1 -- 172.21.15.12:0/3755860115 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87240073d0 con 0x55fcc5d05890 2024-05-09T00:09:06.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.052+0000 7f873ea37b80 10 client.21618 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:06.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.052+0000 7f873ea37b80 1 -- 172.21.15.12:0/3755860115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55fcc5a82b60 con 0x55fcc5d05890 2024-05-09T00:09:06.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.053+0000 7f870b7fe700 1 -- 172.21.15.12:0/3755860115 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fcc5bae270 con 0x55fcc5d05890 2024-05-09T00:09:06.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.053+0000 7f870b7fe700 1 -- 172.21.15.12:0/3755860115 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8724019930 con 0x55fcc5d05890 2024-05-09T00:09:06.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.053+0000 7f873ea37b80 10 client.21618 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:06.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.053+0000 7f873ea37b80 10 client.21618 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:09:06.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.053+0000 7f873ea37b80 20 client.21618 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:06.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.053+0000 7f873ea37b80 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fcc5d1f2d0 0x55fcc5d216f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.053+0000 7f873ea37b80 4 client.21618 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:06.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.053+0000 7f873ea37b80 1 -- 172.21.15.12:0/3755860115 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcc5cd4810 con 0x55fcc5d1f2d0 2024-05-09T00:09:06.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.057+0000 7f872bfff700 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fcc5d1f2d0 0x55fcc5d216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.065+0000 7f872bfff700 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fcc5d1f2d0 0x55fcc5d216f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98f64193bac0193a server_cookie=586f88dfa3499210 in_seq=0 out_seq=0 2024-05-09T00:09:06.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.066+0000 7f870b7fe700 10 client.21618 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:09:06.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.066+0000 7f870b7fe700 1 -- 172.21.15.12:0/3755860115 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fcc5cd4810 con 0x55fcc5d1f2d0 2024-05-09T00:09:06.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.066+0000 7f870b7fe700 10 client.21618 handle_command_reply: tid=0 2024-05-09T00:09:06.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.066+0000 7f873ea37b80 1 client.21618 shutdown 2024-05-09T00:09:06.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.067+0000 7f873ea37b80 1 -- 172.21.15.12:0/3755860115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 msgr2=0x55fcc5d0eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.067+0000 7f873ea37b80 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d0eb30 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8724013f80 tx=0x7f8724011560 comp rx=0 tx=0).stop 2024-05-09T00:09:06.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.072+0000 7f873ea37b80 1 -- 172.21.15.12:0/3755860115 shutdown_connections 2024-05-09T00:09:06.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.072+0000 7f873ea37b80 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fcc5d1f2d0 0x55fcc5d216f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.072+0000 7f873ea37b80 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0b720 0x55fcc5d0f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.072+0000 7f873ea37b80 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d0eb30 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.072+0000 7f873ea37b80 1 --2- 172.21.15.12:0/3755860115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d0f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.072+0000 7f873ea37b80 1 -- 172.21.15.12:0/3755860115 >> 172.21.15.12:0/3755860115 conn(0x55fcc5c63210 msgr2=0x55fcc5cff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.072+0000 7f870b7fe700 0 client.21618 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:09:06.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.073+0000 7f873ea37b80 1 -- 172.21.15.12:0/3755860115 shutdown_connections 2024-05-09T00:09:06.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.073+0000 7f873ea37b80 1 -- 172.21.15.12:0/3755860115 wait complete. 2024-05-09T00:09:06.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.073+0000 7f873ea37b80 20 client.21618 trim_cache size 0 max 16384 2024-05-09T00:09:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.095+0000 7f873ea37b80 1 Processor -- start 2024-05-09T00:09:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.098+0000 7f873ea37b80 1 -- start start 2024-05-09T00:09:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.098+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d05350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.098+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0ae00 0x55fcc5cffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.098+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0b720 0x55fcc5d16200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.098+0000 7f873ea37b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5d10130 con 0x55fcc5d05890 2024-05-09T00:09:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.098+0000 7f873ea37b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5d01e50 con 0x55fcc5d0b720 2024-05-09T00:09:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.098+0000 7f873ea37b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5d0b560 con 0x55fcc5d0ae00 2024-05-09T00:09:06.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.100+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d05350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.100+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d05350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52686/0 (socket says 172.21.15.12:52686) 2024-05-09T00:09:06.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.100+0000 7f872b7fe700 1 -- 172.21.15.12:0/4094597757 learned_addr learned my addr 172.21.15.12:0/4094597757 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.101+0000 7f872b7fe700 1 -- 172.21.15.12:0/4094597757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0ae00 msgr2=0x55fcc5cffd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:09:06.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.101+0000 7f872b7fe700 1 --2- 172.21.15.12:0/4094597757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0ae00 0x55fcc5cffd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.101+0000 7f872b7fe700 1 -- 172.21.15.12:0/4094597757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0b720 msgr2=0x55fcc5d16200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.101+0000 7f872b7fe700 1 --2- 172.21.15.12:0/4094597757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0b720 0x55fcc5d16200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.101+0000 7f872b7fe700 1 -- 172.21.15.12:0/4094597757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcc5c1b560 con 0x55fcc5d05890 2024-05-09T00:09:06.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.108+0000 7f872b7fe700 1 --2- 172.21.15.12:0/4094597757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d05350 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f8724001370 tx=0x7f872401afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698d9adc792f4dc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.114+0000 7f8712ffd700 1 -- 172.21.15.12:0/4094597757 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724001c50 con 0x55fcc5d05890 2024-05-09T00:09:06.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.114+0000 7f8712ffd700 1 -- 172.21.15.12:0/4094597757 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8724019480 con 0x55fcc5d05890 2024-05-09T00:09:06.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.114+0000 7f8712ffd700 1 -- 172.21.15.12:0/4094597757 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724004280 con 0x55fcc5d05890 2024-05-09T00:09:06.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.116+0000 7f873ea37b80 1 -- 172.21.15.12:0/4094597757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 msgr2=0x55fcc5d05350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.116+0000 7f873ea37b80 1 --2- 172.21.15.12:0/4094597757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d05350 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f8724001370 tx=0x7f872401afc0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.116+0000 7f873ea37b80 1 -- 172.21.15.12:0/4094597757 shutdown_connections 2024-05-09T00:09:06.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.116+0000 7f873ea37b80 1 --2- 172.21.15.12:0/4094597757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d0b720 0x55fcc5d16200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.116+0000 7f873ea37b80 1 --2- 172.21.15.12:0/4094597757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d05890 0x55fcc5d05350 secure :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f8724001370 tx=0x7f872401afc0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.116+0000 7f873ea37b80 1 --2- 172.21.15.12:0/4094597757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0ae00 0x55fcc5cffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.116+0000 7f873ea37b80 1 -- 172.21.15.12:0/4094597757 >> 172.21.15.12:0/4094597757 conn(0x55fcc5c63210 msgr2=0x55fcc5d08c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.118+0000 7f873ea37b80 1 -- 172.21.15.12:0/4094597757 shutdown_connections 2024-05-09T00:09:06.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.118+0000 7f873ea37b80 1 -- 172.21.15.12:0/4094597757 wait complete. 2024-05-09T00:09:06.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.118+0000 7f873ea37b80 1 -- start start 2024-05-09T00:09:06.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.118+0000 7f873ea37b80 1 Processor -- start 2024-05-09T00:09:06.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.119+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d69930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.119+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0b720 0x55fcc5d69e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.119+0000 7f873ea37b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d727b0 0x55fcc5d74ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.119+0000 7f873ea37b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5d6a5c0 con 0x55fcc5d0ae00 2024-05-09T00:09:06.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.119+0000 7f873ea37b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5d750e0 con 0x55fcc5d727b0 2024-05-09T00:09:06.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.119+0000 7f873ea37b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc5d752a0 con 0x55fcc5d0b720 2024-05-09T00:09:06.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.119+0000 7f873ea37b80 10 client.0 _ll_register_callbacks cb 0x55fcc5bfab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:06.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.120+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d727b0 0x55fcc5d74ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.120+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d727b0 0x55fcc5d74ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:51484/0 (socket says 172.21.15.12:51484) 2024-05-09T00:09:06.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.120+0000 7f872b7fe700 1 -- 172.21.15.12:0/1970848240 learned_addr learned my addr 172.21.15.12:0/1970848240 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.120+0000 7f872b7fe700 1 -- 172.21.15.12:0/1970848240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0b720 msgr2=0x55fcc5d69e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.120+0000 7f872b7fe700 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0b720 0x55fcc5d69e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.120+0000 7f872b7fe700 1 -- 172.21.15.12:0/1970848240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d0ae00 msgr2=0x55fcc5d69930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.120+0000 7f872b7fe700 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d69930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.120+0000 7f872b7fe700 1 -- 172.21.15.12:0/1970848240 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8724011040 con 0x55fcc5d727b0 2024-05-09T00:09:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.125+0000 7f872b7fe700 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d727b0 0x55fcc5d74ba0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8724001370 tx=0x7f8724001dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d9851f1e0318f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.125+0000 7f870b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:09:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.128+0000 7f870b7fe700 1 -- 172.21.15.12:0/1970848240 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724032070 con 0x55fcc5d727b0 2024-05-09T00:09:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.128+0000 7f870b7fe700 1 -- 172.21.15.12:0/1970848240 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f87240195f0 con 0x55fcc5d727b0 2024-05-09T00:09:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.134+0000 7f873ea37b80 1 -- 172.21.15.12:0/1970848240 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f871c011e50 con 0x55fcc5d727b0 2024-05-09T00:09:06.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.140+0000 7f870b7fe700 1 -- 172.21.15.12:0/1970848240 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8724011ea0 con 0x55fcc5d727b0 2024-05-09T00:09:06.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.142+0000 7f873ea37b80 10 client.31055 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:06.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.142+0000 7f873ea37b80 1 -- 172.21.15.12:0/1970848240 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55fcc5adacd0 con 0x55fcc5d727b0 2024-05-09T00:09:06.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.146+0000 7f870b7fe700 1 -- 172.21.15.12:0/1970848240 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f871c011e50 con 0x55fcc5d727b0 2024-05-09T00:09:06.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.147+0000 7f870b7fe700 1 -- 172.21.15.12:0/1970848240 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8724037450 con 0x55fcc5d727b0 2024-05-09T00:09:06.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.147+0000 7f873ea37b80 10 client.31055 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:06.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.147+0000 7f873ea37b80 10 client.31055 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:09:06.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.147+0000 7f873ea37b80 20 client.31055 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:06.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.147+0000 7f873ea37b80 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fcc5d765c0 0x55fcc5d789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.147+0000 7f873ea37b80 4 client.31055 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:09:06.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.147+0000 7f873ea37b80 1 -- 172.21.15.12:0/1970848240 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fcc5c4dec0 con 0x55fcc5d765c0 2024-05-09T00:09:06.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.152+0000 7f872bfff700 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fcc5d765c0 0x55fcc5d789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.152+0000 7f872bfff700 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fcc5d765c0 0x55fcc5d789b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3b01337aabe4dc2 server_cookie=82e36d193693a607 in_seq=0 out_seq=0 2024-05-09T00:09:06.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.155+0000 7f870b7fe700 10 client.31055 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:09:06.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.155+0000 7f870b7fe700 1 -- 172.21.15.12:0/1970848240 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fcc5c4dec0 con 0x55fcc5d765c0 2024-05-09T00:09:06.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.155+0000 7f870b7fe700 10 client.31055 handle_command_reply: tid=0 2024-05-09T00:09:06.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.156+0000 7f873ea37b80 1 client.31055 shutdown 2024-05-09T00:09:06.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.166+0000 7f873ea37b80 1 -- 172.21.15.12:0/1970848240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d727b0 msgr2=0x55fcc5d74ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.166+0000 7f873ea37b80 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d727b0 0x55fcc5d74ba0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8724001370 tx=0x7f8724001dc0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.168+0000 7f873ea37b80 1 -- 172.21.15.12:0/1970848240 shutdown_connections 2024-05-09T00:09:06.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.168+0000 7f873ea37b80 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fcc5d727b0 0x55fcc5d74ba0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.168+0000 7f873ea37b80 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcc5d0ae00 0x55fcc5d69930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.168+0000 7f873ea37b80 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55fcc5d765c0 0x55fcc5d789b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.168+0000 7f873ea37b80 1 --2- 172.21.15.12:0/1970848240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fcc5d0b720 0x55fcc5d69e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.168+0000 7f873ea37b80 1 -- 172.21.15.12:0/1970848240 >> 172.21.15.12:0/1970848240 conn(0x55fcc5c63210 msgr2=0x55fcc5d09c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.171+0000 7f870b7fe700 0 client.31055 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:09:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.171+0000 7f873ea37b80 1 -- 172.21.15.12:0/1970848240 shutdown_connections 2024-05-09T00:09:06.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.171+0000 7f873ea37b80 1 -- 172.21.15.12:0/1970848240 wait complete. 2024-05-09T00:09:06.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.174+0000 7f873ea37b80 20 client.31055 trim_cache size 0 max 16384 2024-05-09T00:09:06.175 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:09:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.185+0000 7f873287d700 1 -- 172.21.15.12:0/712023443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f87140a1390 msgr2=0x7f87140a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.185+0000 7f873287d700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f87140a1390 0x7f87140a3850 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8720026000 tx=0x7f872000f4e0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.185+0000 7f873287d700 1 -- 172.21.15.12:0/712023443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c11a780 msgr2=0x7f872c111e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.185+0000 7f873287d700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c11a780 0x7f872c111e00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f87200098e0 tx=0x7f8720003c10 comp rx=0 tx=0).stop 2024-05-09T00:09:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.193+0000 7f873287d700 1 -- 172.21.15.12:0/712023443 shutdown_connections 2024-05-09T00:09:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.193+0000 7f873287d700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f87140a1390 0x7f87140a3850 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.193+0000 7f873287d700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c11a780 0x7f872c111e00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.193+0000 7f873287d700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10f6a0 0x7f872c118c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.193+0000 7f873287d700 1 --2- 172.21.15.12:0/712023443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c10fb10 0x7f872c119190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.193+0000 7f873287d700 1 -- 172.21.15.12:0/712023443 >> 172.21.15.12:0/712023443 conn(0x7f872c06bff0 msgr2=0x7f872c06f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.193+0000 7f873287d700 1 -- 172.21.15.12:0/712023443 shutdown_connections 2024-05-09T00:09:06.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.193+0000 7f873287d700 1 -- 172.21.15.12:0/712023443 wait complete. 2024-05-09T00:09:06.208 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:09:06.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 321448 30.03 MB/sec execute 421 sec latency 981.685 ms 2024-05-09T00:09:06.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:06 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3828489824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:09:06.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:06 smithi012 ceph-mon[42503]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 706 op/s 2024-05-09T00:09:06.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.439+0000 7f9b944db700 1 -- 172.21.15.12:0/1795153333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c10fb10 msgr2=0x7f9b8c1084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.439+0000 7f9b944db700 1 --2- 172.21.15.12:0/1795153333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c10fb10 0x7f9b8c1084e0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009990 tx=0x7f9b88009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.439+0000 7f9b944db700 1 -- 172.21.15.12:0/1795153333 shutdown_connections 2024-05-09T00:09:06.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.439+0000 7f9b944db700 1 --2- 172.21.15.12:0/1795153333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b8c11a780 0x7f9b8c11cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.439+0000 7f9b944db700 1 --2- 172.21.15.12:0/1795153333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c10f6a0 0x7f9b8c107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.439+0000 7f9b944db700 1 --2- 172.21.15.12:0/1795153333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c10fb10 0x7f9b8c1084e0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.439+0000 7f9b944db700 1 -- 172.21.15.12:0/1795153333 >> 172.21.15.12:0/1795153333 conn(0x7f9b8c06bff0 msgr2=0x7f9b8c06c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.452+0000 7f9b944db700 1 -- 172.21.15.12:0/1795153333 shutdown_connections 2024-05-09T00:09:06.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.456+0000 7f9b944db700 1 -- 172.21.15.12:0/1795153333 wait complete. 2024-05-09T00:09:06.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.457+0000 7f9b944db700 1 Processor -- start 2024-05-09T00:09:06.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.457+0000 7f9b944db700 1 -- start start 2024-05-09T00:09:06.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.457+0000 7f9b944db700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b8c10f6a0 0x7f9b8c118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.457+0000 7f9b944db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c10fb10 0x7f9b8c119170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.457+0000 7f9b944db700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c11a780 0x7f9b8c111db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.457+0000 7f9b944db700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c112580 con 0x7f9b8c10fb10 2024-05-09T00:09:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.457+0000 7f9b944db700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c1126f0 con 0x7f9b8c10f6a0 2024-05-09T00:09:06.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.457+0000 7f9b944db700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c1128b0 con 0x7f9b8c11a780 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.458+0000 7f9b92a78700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c11a780 0x7f9b8c111db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.458+0000 7f9b92a78700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c11a780 0x7f9b8c111db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42090/0 (socket says 172.21.15.12:42090) 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.458+0000 7f9b92a78700 1 -- 172.21.15.12:0/508680768 learned_addr learned my addr 172.21.15.12:0/508680768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.458+0000 7f9b92a78700 1 -- 172.21.15.12:0/508680768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b8c10f6a0 msgr2=0x7f9b8c118bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.458+0000 7f9b92a78700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b8c10f6a0 0x7f9b8c118bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.458+0000 7f9b92a78700 1 -- 172.21.15.12:0/508680768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c10fb10 msgr2=0x7f9b8c119170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.458+0000 7f9b92a78700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c10fb10 0x7f9b8c119170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.458+0000 7f9b92a78700 1 -- 172.21.15.12:0/508680768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b880095f0 con 0x7f9b8c11a780 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.458+0000 7f9b92a78700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c11a780 0x7f9b8c111db0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400b9e0 tx=0x7f9b8400bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.461+0000 7f9b837fe700 1 -- 172.21.15.12:0/508680768 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400c9b0 con 0x7f9b8c11a780 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.461+0000 7f9b837fe700 1 -- 172.21.15.12:0/508680768 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b84014440 con 0x7f9b8c11a780 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.461+0000 7f9b837fe700 1 -- 172.21.15.12:0/508680768 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b840125a0 con 0x7f9b8c11a780 2024-05-09T00:09:06.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.461+0000 7f9b944db700 1 -- 172.21.15.12:0/508680768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c116780 con 0x7f9b8c11a780 2024-05-09T00:09:06.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.461+0000 7f9b944db700 1 -- 172.21.15.12:0/508680768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c116c40 con 0x7f9b8c11a780 2024-05-09T00:09:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.471+0000 7f9ba0695b80 1 Processor -- start 2024-05-09T00:09:06.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.475+0000 7f9ba0695b80 1 -- start start 2024-05-09T00:09:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.475+0000 7f9b837fe700 1 -- 172.21.15.12:0/508680768 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9b84012700 con 0x7f9b8c11a780 2024-05-09T00:09:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.475+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d9101e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d910720 0x56372d90a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d90a890 0x56372d90acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d861150 con 0x56372d910720 2024-05-09T00:09:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d85eb10 con 0x56372d90a890 2024-05-09T00:09:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d85a400 con 0x56372d90fe00 2024-05-09T00:09:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9b92277700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d910720 0x56372d90a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9b92277700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d910720 0x56372d90a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52704/0 (socket says 172.21.15.12:52704) 2024-05-09T00:09:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9b92277700 1 -- 172.21.15.12:0/4180006718 learned_addr learned my addr 172.21.15.12:0/4180006718 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9b92277700 1 -- 172.21.15.12:0/4180006718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 msgr2=0x56372d9101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9b92277700 1 --2- 172.21.15.12:0/4180006718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d9101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9b92277700 1 -- 172.21.15.12:0/4180006718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d90a890 msgr2=0x56372d90acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9b92277700 1 --2- 172.21.15.12:0/4180006718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d90a890 0x56372d90acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9b92277700 1 -- 172.21.15.12:0/4180006718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56372d6db3b0 con 0x56372d910720 2024-05-09T00:09:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.476+0000 7f9b837fe700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b780a14a0 0x7f9b780a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.477+0000 7f9b837fe700 1 -- 172.21.15.12:0/508680768 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9b84006e40 con 0x7f9b8c11a780 2024-05-09T00:09:06.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.477+0000 7f9b92277700 1 --2- 172.21.15.12:0/4180006718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d910720 0x56372d90a350 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009830 tx=0x7f9b7c009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2360740896902d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.479+0000 7f9b80ff9700 1 -- 172.21.15.12:0/4180006718 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c011830 con 0x56372d910720 2024-05-09T00:09:06.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.479+0000 7f9b80ff9700 1 -- 172.21.15.12:0/4180006718 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b7c011e50 con 0x56372d910720 2024-05-09T00:09:06.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.479+0000 7f9b80ff9700 1 -- 172.21.15.12:0/4180006718 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00be60 con 0x56372d910720 2024-05-09T00:09:06.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.480+0000 7f9ba0695b80 1 -- 172.21.15.12:0/4180006718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d910720 msgr2=0x56372d90a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.480+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/4180006718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d910720 0x56372d90a350 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009830 tx=0x7f9b7c009b40 comp rx=0 tx=0).stop 2024-05-09T00:09:06.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.481+0000 7f9b91a76700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b780a14a0 0x7f9b780a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.481+0000 7f9b91a76700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b780a14a0 0x7f9b780a3960 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9b880098e0 tx=0x7f9b88009510 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.482+0000 7f9ba0695b80 1 -- 172.21.15.12:0/4180006718 shutdown_connections 2024-05-09T00:09:06.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.482+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/4180006718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d90a890 0x56372d90acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.482+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/4180006718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d9101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.482+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/4180006718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d910720 0x56372d90a350 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.482+0000 7f9ba0695b80 1 -- 172.21.15.12:0/4180006718 >> 172.21.15.12:0/4180006718 conn(0x56372d868210 msgr2=0x56372d904df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.482+0000 7f9ba0695b80 1 -- 172.21.15.12:0/4180006718 shutdown_connections 2024-05-09T00:09:06.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.482+0000 7f9ba0695b80 1 -- 172.21.15.12:0/4180006718 wait complete. 2024-05-09T00:09:06.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.482+0000 7f9ba0695b80 1 -- start start 2024-05-09T00:09:06.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.483+0000 7f9ba0695b80 1 Processor -- start 2024-05-09T00:09:06.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.483+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90a890 0x56372d913b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.483+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d914070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.483+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d9145b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.483+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d7f6560 con 0x56372d90a890 2024-05-09T00:09:06.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.483+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d7f6200 con 0x56372d910720 2024-05-09T00:09:06.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.483+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d6dd6a0 con 0x56372d90fe00 2024-05-09T00:09:06.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.483+0000 7f9ba0695b80 10 client.0 _ll_register_callbacks cb 0x56372d7ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:06.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b92a78700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d9145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b92a78700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d9145b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:51520/0 (socket says 172.21.15.12:51520) 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b92a78700 1 -- 172.21.15.12:0/1856563038 learned_addr learned my addr 172.21.15.12:0/1856563038 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b91a76700 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d914070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b92277700 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90a890 0x56372d913b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b92a78700 1 -- 172.21.15.12:0/1856563038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 msgr2=0x56372d914070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b92a78700 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d914070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b92a78700 1 -- 172.21.15.12:0/1856563038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90a890 msgr2=0x56372d913b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b92a78700 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90a890 0x56372d913b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b92a78700 1 -- 172.21.15.12:0/1856563038 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7c009460 con 0x56372d910720 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b92a78700 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d9145b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9b840a1270 tx=0x7f9b840c9db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f34e7b097daedd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.486+0000 7f9b75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.487+0000 7f9b75ffb700 1 -- 172.21.15.12:0/1856563038 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b840a17a0 con 0x56372d910720 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.487+0000 7f9b75ffb700 1 -- 172.21.15.12:0/1856563038 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9b840a1310 con 0x56372d910720 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.488+0000 7f9ba0695b80 1 -- 172.21.15.12:0/1856563038 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56372d7b3270 con 0x56372d910720 2024-05-09T00:09:06.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.491+0000 7f9b75ffb700 1 -- 172.21.15.12:0/1856563038 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b840a14d0 con 0x56372d910720 2024-05-09T00:09:06.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.493+0000 7f9ba0695b80 10 client.31058 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:06.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.493+0000 7f9ba0695b80 1 -- 172.21.15.12:0/1856563038 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x56372d687b60 con 0x56372d910720 2024-05-09T00:09:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.501+0000 7f9b75ffb700 1 -- 172.21.15.12:0/1856563038 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56372d7b3270 con 0x56372d910720 2024-05-09T00:09:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.501+0000 7f9b75ffb700 1 -- 172.21.15.12:0/1856563038 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9b84098440 con 0x56372d910720 2024-05-09T00:09:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.501+0000 7f9ba0695b80 10 client.31058 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.501+0000 7f9ba0695b80 10 client.31058 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:09:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.501+0000 7f9ba0695b80 20 client.31058 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.501+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56372d9242a0 0x56372d9266c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.501+0000 7f9ba0695b80 4 client.31058 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.501+0000 7f9ba0695b80 1 -- 172.21.15.12:0/1856563038 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56372d8d9810 con 0x56372d9242a0 2024-05-09T00:09:06.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.506+0000 7f9b92277700 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56372d9242a0 0x56372d9266c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.512+0000 7f9b92277700 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56372d9242a0 0x56372d9266c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=892af70ba2d8a411 server_cookie=a4eae17ddd4b1d21 in_seq=0 out_seq=0 2024-05-09T00:09:06.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.512+0000 7f9b75ffb700 10 client.31058 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:09:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:06 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3828489824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:09:06.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:06 smithi204 ceph-mon[47373]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 706 op/s 2024-05-09T00:09:06.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.519+0000 7f9b75ffb700 1 -- 172.21.15.12:0/1856563038 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56372d8d9810 con 0x56372d9242a0 2024-05-09T00:09:06.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.519+0000 7f9b75ffb700 10 client.31058 handle_command_reply: tid=0 2024-05-09T00:09:06.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.519+0000 7f9ba0695b80 1 client.31058 shutdown 2024-05-09T00:09:06.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.520+0000 7f9ba0695b80 1 -- 172.21.15.12:0/1856563038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 msgr2=0x56372d9145b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.520+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d9145b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9b840a1270 tx=0x7f9b840c9db0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:06 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3828489824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:09:06.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:06 smithi160 ceph-mon[46152]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 706 op/s 2024-05-09T00:09:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.522+0000 7f9ba0695b80 1 -- 172.21.15.12:0/1856563038 shutdown_connections 2024-05-09T00:09:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.522+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56372d9242a0 0x56372d9266c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.522+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d9145b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.522+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90a890 0x56372d913b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.522+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/1856563038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d914070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.522+0000 7f9b75ffb700 0 client.31058 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:09:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.522+0000 7f9ba0695b80 1 -- 172.21.15.12:0/1856563038 >> 172.21.15.12:0/1856563038 conn(0x56372d868210 msgr2=0x56372d9045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.522+0000 7f9ba0695b80 1 -- 172.21.15.12:0/1856563038 shutdown_connections 2024-05-09T00:09:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.522+0000 7f9ba0695b80 1 -- 172.21.15.12:0/1856563038 wait complete. 2024-05-09T00:09:06.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.523+0000 7f9ba0695b80 20 client.31058 trim_cache size 0 max 16384 2024-05-09T00:09:06.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.549+0000 7f9ba0695b80 1 Processor -- start 2024-05-09T00:09:06.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.550+0000 7f9ba0695b80 1 -- start start 2024-05-09T00:09:06.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.550+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90a890 0x56372d90a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.550+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90fe00 0x56372d913410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.550+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d90f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.550+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d915130 con 0x56372d90fe00 2024-05-09T00:09:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.550+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d906e50 con 0x56372d910720 2024-05-09T00:09:06.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.550+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d910590 con 0x56372d90a890 2024-05-09T00:09:06.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.550+0000 7f9b92277700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90a890 0x56372d90a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.550+0000 7f9b92277700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90a890 0x56372d90a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42116/0 (socket says 172.21.15.12:42116) 2024-05-09T00:09:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.550+0000 7f9b92277700 1 -- 172.21.15.12:0/2559201740 learned_addr learned my addr 172.21.15.12:0/2559201740 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.551+0000 7f9b92277700 1 -- 172.21.15.12:0/2559201740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 msgr2=0x56372d90f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.551+0000 7f9b92277700 1 --2- 172.21.15.12:0/2559201740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d90f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.551+0000 7f9b92277700 1 -- 172.21.15.12:0/2559201740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90fe00 msgr2=0x56372d913410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.551+0000 7f9b92277700 1 --2- 172.21.15.12:0/2559201740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90fe00 0x56372d913410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.551+0000 7f9b92277700 1 -- 172.21.15.12:0/2559201740 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56372d820560 con 0x56372d90a890 2024-05-09T00:09:06.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.556+0000 7f9b92277700 1 --2- 172.21.15.12:0/2559201740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90a890 0x56372d90a350 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c004790 tx=0x7f9b7c008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc568833d8b95d1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.558+0000 7f9b817fa700 1 -- 172.21.15.12:0/2559201740 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c020040 con 0x56372d90a890 2024-05-09T00:09:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.558+0000 7f9b817fa700 1 -- 172.21.15.12:0/2559201740 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b7c00b3e0 con 0x56372d90a890 2024-05-09T00:09:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.558+0000 7f9b817fa700 1 -- 172.21.15.12:0/2559201740 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00b860 con 0x56372d90a890 2024-05-09T00:09:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.559+0000 7f9ba0695b80 1 -- 172.21.15.12:0/2559201740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90a890 msgr2=0x56372d90a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.559+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/2559201740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90a890 0x56372d90a350 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c004790 tx=0x7f9b7c008040 comp rx=0 tx=0).stop 2024-05-09T00:09:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.560+0000 7f9ba0695b80 1 -- 172.21.15.12:0/2559201740 shutdown_connections 2024-05-09T00:09:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.560+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/2559201740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d90f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.560+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/2559201740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90a890 0x56372d90a350 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.560+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/2559201740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90fe00 0x56372d913410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.560+0000 7f9ba0695b80 1 -- 172.21.15.12:0/2559201740 >> 172.21.15.12:0/2559201740 conn(0x56372d868210 msgr2=0x56372d90ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.560+0000 7f9ba0695b80 1 -- 172.21.15.12:0/2559201740 shutdown_connections 2024-05-09T00:09:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.560+0000 7f9ba0695b80 1 -- 172.21.15.12:0/2559201740 wait complete. 2024-05-09T00:09:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.560+0000 7f9ba0695b80 1 -- start start 2024-05-09T00:09:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.560+0000 7f9ba0695b80 1 Processor -- start 2024-05-09T00:09:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.561+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90a890 0x56372d9191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.561+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d919720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.561+0000 7f9ba0695b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d9797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.561+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d919f00 con 0x56372d90a890 2024-05-09T00:09:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.561+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d91a070 con 0x56372d910720 2024-05-09T00:09:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.561+0000 7f9ba0695b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372d979d30 con 0x56372d90fe00 2024-05-09T00:09:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.561+0000 7f9ba0695b80 10 client.0 _ll_register_callbacks cb 0x56372d7ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.561+0000 7f9b91a76700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d919720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.561+0000 7f9b91a76700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d919720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42120/0 (socket says 172.21.15.12:42120) 2024-05-09T00:09:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.561+0000 7f9b91a76700 1 -- 172.21.15.12:0/3776857738 learned_addr learned my addr 172.21.15.12:0/3776857738 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.562+0000 7f9b91a76700 1 -- 172.21.15.12:0/3776857738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 msgr2=0x56372d9797a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.562+0000 7f9b91a76700 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d9797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.562+0000 7f9b91a76700 1 -- 172.21.15.12:0/3776857738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90a890 msgr2=0x56372d9191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.562+0000 7f9b91a76700 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90a890 0x56372d9191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.562+0000 7f9b91a76700 1 -- 172.21.15.12:0/3776857738 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56372d6db3b0 con 0x56372d90fe00 2024-05-09T00:09:06.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.562+0000 7f9b91a76700 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d919720 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9b880093f0 tx=0x7f9b8800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf5be0fff4e835a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.563+0000 7f9b75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:09:06.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.565+0000 7f9b75ffb700 1 -- 172.21.15.12:0/3776857738 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88028050 con 0x56372d90fe00 2024-05-09T00:09:06.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.565+0000 7f9b75ffb700 1 -- 172.21.15.12:0/3776857738 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9b88022650 con 0x56372d90fe00 2024-05-09T00:09:06.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.565+0000 7f9ba0695b80 1 -- 172.21.15.12:0/3776857738 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b7c011e50 con 0x56372d90fe00 2024-05-09T00:09:06.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.565+0000 7f9b75ffb700 1 -- 172.21.15.12:0/3776857738 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b8801d070 con 0x56372d90fe00 2024-05-09T00:09:06.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.565+0000 7f9ba0695b80 10 client.31330 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:06.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.565+0000 7f9ba0695b80 1 -- 172.21.15.12:0/3776857738 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x56372d6dfcd0 con 0x56372d90fe00 2024-05-09T00:09:06.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.567+0000 7f9b75ffb700 1 -- 172.21.15.12:0/3776857738 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b7c011e50 con 0x56372d90fe00 2024-05-09T00:09:06.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.568+0000 7f9b75ffb700 1 -- 172.21.15.12:0/3776857738 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9b8802d8f0 con 0x56372d90fe00 2024-05-09T00:09:06.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.571+0000 7f9ba0695b80 10 client.31330 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:06.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.571+0000 7f9ba0695b80 10 client.31330 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:09:06.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.571+0000 7f9ba0695b80 20 client.31330 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:06.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.571+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56372d97b550 0x56372d97d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.571+0000 7f9ba0695b80 4 client.31330 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:09:06.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.571+0000 7f9ba0695b80 1 -- 172.21.15.12:0/3776857738 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56372d852ec0 con 0x56372d97b550 2024-05-09T00:09:06.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.572+0000 7f9b92277700 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56372d97b550 0x56372d97d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.573+0000 7f9b92277700 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56372d97b550 0x56372d97d970 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3404e2210920f932 server_cookie=dacb78625d6ea4e4 in_seq=0 out_seq=0 2024-05-09T00:09:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.574+0000 7f9b75ffb700 10 client.31330 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:09:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.574+0000 7f9b75ffb700 1 -- 172.21.15.12:0/3776857738 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56372d852ec0 con 0x56372d97b550 2024-05-09T00:09:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.574+0000 7f9b75ffb700 10 client.31330 handle_command_reply: tid=0 2024-05-09T00:09:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.574+0000 7f9ba0695b80 1 client.31330 shutdown 2024-05-09T00:09:06.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.578+0000 7f9ba0695b80 1 -- 172.21.15.12:0/3776857738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 msgr2=0x56372d919720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.578+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d919720 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9b880093f0 tx=0x7f9b8800fa70 comp rx=0 tx=0).stop 2024-05-09T00:09:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.578+0000 7f9ba0695b80 1 -- 172.21.15.12:0/3776857738 shutdown_connections 2024-05-09T00:09:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.578+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x56372d97b550 0x56372d97d970 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.578+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56372d910720 0x56372d9797a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.578+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56372d90a890 0x56372d9191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.578+0000 7f9ba0695b80 1 --2- 172.21.15.12:0/3776857738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56372d90fe00 0x56372d919720 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.578+0000 7f9ba0695b80 1 -- 172.21.15.12:0/3776857738 >> 172.21.15.12:0/3776857738 conn(0x56372d868210 msgr2=0x56372d910bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.582+0000 7f9b75ffb700 0 client.31330 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:09:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.583+0000 7f9ba0695b80 1 -- 172.21.15.12:0/3776857738 shutdown_connections 2024-05-09T00:09:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.588+0000 7f9ba0695b80 1 -- 172.21.15.12:0/3776857738 wait complete. 2024-05-09T00:09:06.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.596+0000 7f9ba0695b80 20 client.31330 trim_cache size 0 max 16384 2024-05-09T00:09:06.593 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:09:06.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 -- 172.21.15.12:0/508680768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b780a14a0 msgr2=0x7f9b780a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b780a14a0 0x7f9b780a3960 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9b880098e0 tx=0x7f9b88009510 comp rx=0 tx=0).stop 2024-05-09T00:09:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 -- 172.21.15.12:0/508680768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c11a780 msgr2=0x7f9b8c111db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c11a780 0x7f9b8c111db0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400b9e0 tx=0x7f9b8400bda0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 -- 172.21.15.12:0/508680768 shutdown_connections 2024-05-09T00:09:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9b780a14a0 0x7f9b780a3960 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c11a780 0x7f9b8c111db0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b8c10f6a0 0x7f9b8c118bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 --2- 172.21.15.12:0/508680768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b8c10fb10 0x7f9b8c119170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 -- 172.21.15.12:0/508680768 >> 172.21.15.12:0/508680768 conn(0x7f9b8c06bff0 msgr2=0x7f9b8c06f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 -- 172.21.15.12:0/508680768 shutdown_connections 2024-05-09T00:09:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.598+0000 7f9b944db700 1 -- 172.21.15.12:0/508680768 wait complete. 2024-05-09T00:09:06.621 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:09:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.872+0000 7fb1ad01c700 1 -- 172.21.15.12:0/3644435731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a811a780 msgr2=0x7fb1a811cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.872+0000 7fb1ad01c700 1 --2- 172.21.15.12:0/3644435731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a811a780 0x7fb1a811cb70 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000b0a0 tx=0x7fb1a000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.873+0000 7fb1ad01c700 1 -- 172.21.15.12:0/3644435731 shutdown_connections 2024-05-09T00:09:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.873+0000 7fb1ad01c700 1 --2- 172.21.15.12:0/3644435731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a811a780 0x7fb1a811cb70 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.873+0000 7fb1ad01c700 1 --2- 172.21.15.12:0/3644435731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1a810f6a0 0x7fb1a8107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.873+0000 7fb1ad01c700 1 --2- 172.21.15.12:0/3644435731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a810fb10 0x7fb1a81084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.873+0000 7fb1ad01c700 1 -- 172.21.15.12:0/3644435731 >> 172.21.15.12:0/3644435731 conn(0x7fb1a806bff0 msgr2=0x7fb1a806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.874+0000 7fb1ad01c700 1 -- 172.21.15.12:0/3644435731 shutdown_connections 2024-05-09T00:09:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.874+0000 7fb1ad01c700 1 -- 172.21.15.12:0/3644435731 wait complete. 2024-05-09T00:09:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.875+0000 7fb1ad01c700 1 Processor -- start 2024-05-09T00:09:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.875+0000 7fb1ad01c700 1 -- start start 2024-05-09T00:09:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.876+0000 7fb1ad01c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a810fb10 0x7fb1a81a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.876+0000 7fb1ad01c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a811a780 0x7fb1a81a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.876+0000 7fb1ad01c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1a81aad70 0x7fb1a81ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.876+0000 7fb1ad01c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a81ab7f0 con 0x7fb1a810fb10 2024-05-09T00:09:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.876+0000 7fb1ad01c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a81ab960 con 0x7fb1a81aad70 2024-05-09T00:09:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.876+0000 7fb1ad01c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a81abb20 con 0x7fb1a811a780 2024-05-09T00:09:06.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.879+0000 7fb1a759e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1a81aad70 0x7fb1a81ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.880+0000 7fb1a659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a811a780 0x7fb1a81a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.880+0000 7fb1a659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a811a780 0x7fb1a81a69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42128/0 (socket says 172.21.15.12:42128) 2024-05-09T00:09:06.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.880+0000 7fb1a659c700 1 -- 172.21.15.12:0/1543880020 learned_addr learned my addr 172.21.15.12:0/1543880020 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.880+0000 7fb1a659c700 1 -- 172.21.15.12:0/1543880020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1a81aad70 msgr2=0x7fb1a81ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.880+0000 7fb1a659c700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1a81aad70 0x7fb1a81ab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.880+0000 7fb1a659c700 1 -- 172.21.15.12:0/1543880020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a810fb10 msgr2=0x7fb1a81a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.880+0000 7fb1a659c700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a810fb10 0x7fb1a81a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.880+0000 7fb1a659c700 1 -- 172.21.15.12:0/1543880020 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a0009d00 con 0x7fb1a811a780 2024-05-09T00:09:06.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.881+0000 7fb1a659c700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a811a780 0x7fb1a81a69d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00b500 tx=0x7fb19c00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.885+0000 7fb18ffff700 1 -- 172.21.15.12:0/1543880020 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c011830 con 0x7fb1a811a780 2024-05-09T00:09:06.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.885+0000 7fb18ffff700 1 -- 172.21.15.12:0/1543880020 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb19c011e50 con 0x7fb1a811a780 2024-05-09T00:09:06.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.885+0000 7fb18ffff700 1 -- 172.21.15.12:0/1543880020 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00f2e0 con 0x7fb1a811a780 2024-05-09T00:09:06.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.886+0000 7fb1ad01c700 1 -- 172.21.15.12:0/1543880020 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a81b3080 con 0x7fb1a811a780 2024-05-09T00:09:06.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.886+0000 7fb1ad01c700 1 -- 172.21.15.12:0/1543880020 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a81b3510 con 0x7fb1a811a780 2024-05-09T00:09:06.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.893+0000 7fb1b91d6b80 1 Processor -- start 2024-05-09T00:09:06.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.893+0000 7fb1b91d6b80 1 -- start start 2024-05-09T00:09:06.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.893+0000 7fb18ffff700 1 -- 172.21.15.12:0/1543880020 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb19c00f440 con 0x7fb1a811a780 2024-05-09T00:09:06.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.910+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x5555991351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.910+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x55559912f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.910+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55559912f890 0x55559912fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.910+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555599086150 con 0x555599135720 2024-05-09T00:09:06.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.910+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555599083b10 con 0x55559912f890 2024-05-09T00:09:06.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.910+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559907f400 con 0x555599134e00 2024-05-09T00:09:06.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.911+0000 7fb18ffff700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1900a1460 0x7fb1900a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.911+0000 7fb18ffff700 1 -- 172.21.15.12:0/1543880020 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb19c0cd5b0 con 0x7fb1a811a780 2024-05-09T00:09:06.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.914+0000 7fb1a659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55559912f890 0x55559912fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.914+0000 7fb1a759e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x5555991351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.915+0000 7fb1a759e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x5555991351e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42142/0 (socket says 172.21.15.12:42142) 2024-05-09T00:09:06.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.915+0000 7fb1a759e700 1 -- 172.21.15.12:0/429924949 learned_addr learned my addr 172.21.15.12:0/429924949 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.915+0000 7fb1a759e700 1 -- 172.21.15.12:0/429924949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55559912f890 msgr2=0x55559912fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.915+0000 7fb1a759e700 1 --2- 172.21.15.12:0/429924949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55559912f890 0x55559912fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.915+0000 7fb1a759e700 1 -- 172.21.15.12:0/429924949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 msgr2=0x55559912f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.915+0000 7fb1a759e700 1 --2- 172.21.15.12:0/429924949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x55559912f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.915+0000 7fb1a759e700 1 -- 172.21.15.12:0/429924949 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555598f003b0 con 0x555599134e00 2024-05-09T00:09:06.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.915+0000 7fb1a759e700 1 --2- 172.21.15.12:0/429924949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x5555991351e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0006070 tx=0x7fb1a0008e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=102e55d1634f554f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.916+0000 7fb18d7fa700 1 -- 172.21.15.12:0/429924949 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000e030 con 0x555599134e00 2024-05-09T00:09:06.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.916+0000 7fb18d7fa700 1 -- 172.21.15.12:0/429924949 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1a00093d0 con 0x555599134e00 2024-05-09T00:09:06.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.916+0000 7fb18d7fa700 1 -- 172.21.15.12:0/429924949 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00052a0 con 0x555599134e00 2024-05-09T00:09:06.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.917+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/429924949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 msgr2=0x5555991351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.917+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/429924949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x5555991351e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0006070 tx=0x7fb1a0008e30 comp rx=0 tx=0).stop 2024-05-09T00:09:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.917+0000 7fb1a759e700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1900a1460 0x7fb1900a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.917+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/429924949 shutdown_connections 2024-05-09T00:09:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.917+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/429924949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55559912f890 0x55559912fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.917+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/429924949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x5555991351e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.917+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/429924949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x55559912f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.917+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/429924949 >> 172.21.15.12:0/429924949 conn(0x55559908d210 msgr2=0x555599129df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.919+0000 7fb1a759e700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1900a1460 0x7fb1900a3920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0008e70 tx=0x7fb1a0009540 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.920+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/429924949 shutdown_connections 2024-05-09T00:09:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.927+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/429924949 wait complete. 2024-05-09T00:09:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.927+0000 7fb1b91d6b80 1 -- start start 2024-05-09T00:09:06.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.927+0000 7fb1b91d6b80 1 Processor -- start 2024-05-09T00:09:06.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x555599138b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x5555991390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599139600 0x555599145250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559901b560 con 0x555599135720 2024-05-09T00:09:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559901b200 con 0x555599139600 2024-05-09T00:09:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555598f026a0 con 0x555599134e00 2024-05-09T00:09:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1b91d6b80 10 client.0 _ll_register_callbacks cb 0x555599024b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1a659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x5555991390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1a659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x5555991390c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52772/0 (socket says 172.21.15.12:52772) 2024-05-09T00:09:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1a659c700 1 -- 172.21.15.12:0/1687834192 learned_addr learned my addr 172.21.15.12:0/1687834192 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.928+0000 7fb1a759e700 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599139600 0x555599145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb1a659c700 1 -- 172.21.15.12:0/1687834192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 msgr2=0x555599138b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb1a659c700 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x555599138b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb1a659c700 1 -- 172.21.15.12:0/1687834192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599139600 msgr2=0x555599145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb1a659c700 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599139600 0x555599145250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb1a659c700 1 -- 172.21.15.12:0/1687834192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a811d460 con 0x555599135720 2024-05-09T00:09:06.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb1a759e700 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599139600 0x555599145250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:06.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb1a659c700 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x5555991390c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00bcc0 tx=0x7fb19c09a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59c2665e6cc12f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb18aff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:06.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb18aff5700 1 -- 172.21.15.12:0/1687834192 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c09e370 con 0x555599135720 2024-05-09T00:09:06.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb18aff5700 1 -- 172.21.15.12:0/1687834192 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb19c09e590 con 0x555599135720 2024-05-09T00:09:06.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1687834192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555598fd8270 con 0x555599135720 2024-05-09T00:09:06.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb18aff5700 1 -- 172.21.15.12:0/1687834192 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb19c0a2be0 con 0x555599135720 2024-05-09T00:09:06.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb1b91d6b80 10 client.21636 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:06.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1687834192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x555598eacb60 con 0x555599135720 2024-05-09T00:09:06.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.929+0000 7fb18aff5700 1 -- 172.21.15.12:0/1687834192 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb19c0a6070 con 0x555599135720 2024-05-09T00:09:06.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.930+0000 7fb18aff5700 1 -- 172.21.15.12:0/1687834192 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb19c09a9c0 con 0x555599135720 2024-05-09T00:09:06.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.930+0000 7fb1b91d6b80 10 client.21636 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:06.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.930+0000 7fb1b91d6b80 10 client.21636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:06.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.930+0000 7fb1b91d6b80 20 client.21636 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:06.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.930+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5555991496a0 0x55559914bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.930+0000 7fb1b91d6b80 4 client.21636 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:06.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.930+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1687834192 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555990fe810 con 0x5555991496a0 2024-05-09T00:09:06.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.930+0000 7fb1a6d9d700 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5555991496a0 0x55559914bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.930+0000 7fb1a6d9d700 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5555991496a0 0x55559914bac0 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976ce94d00e03e20 server_cookie=857241698a82396c in_seq=0 out_seq=0 2024-05-09T00:09:06.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.930+0000 7fb18aff5700 10 client.21636 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:06.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.937+0000 7fb18aff5700 1 -- 172.21.15.12:0/1687834192 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5555990fe810 con 0x5555991496a0 2024-05-09T00:09:06.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.937+0000 7fb18aff5700 10 client.21636 handle_command_reply: tid=0 2024-05-09T00:09:06.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.937+0000 7fb1b91d6b80 1 client.21636 shutdown 2024-05-09T00:09:06.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.937+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1687834192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 msgr2=0x5555991390c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.937+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x5555991390c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00bcc0 tx=0x7fb19c09a140 comp rx=0 tx=0).stop 2024-05-09T00:09:06.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.942+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1687834192 shutdown_connections 2024-05-09T00:09:06.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.942+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5555991496a0 0x55559914bac0 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.942+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599139600 0x555599145250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.942+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x555599138b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.942+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1687834192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x5555991390c0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.942+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1687834192 >> 172.21.15.12:0/1687834192 conn(0x55559908d210 msgr2=0x5555991295a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.942+0000 7fb18aff5700 0 client.21636 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:06.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.942+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1687834192 shutdown_connections 2024-05-09T00:09:06.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.946+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1687834192 wait complete. 2024-05-09T00:09:06.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.947+0000 7fb1b91d6b80 20 client.21636 trim_cache size 0 max 16384 2024-05-09T00:09:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.969+0000 7fb1b91d6b80 1 Processor -- start 2024-05-09T00:09:06.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.973+0000 7fb1b91d6b80 1 -- start start 2024-05-09T00:09:06.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.974+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599134e00 0x55559912f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.974+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599135720 0x555599134310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.974+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599139600 0x555599129d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.974+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555599150d10 con 0x555599139600 2024-05-09T00:09:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.974+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555599124800 con 0x555599134e00 2024-05-09T00:09:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.974+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555991370d0 con 0x555599135720 2024-05-09T00:09:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.976+0000 7fb1a759e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599139600 0x555599129d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.976+0000 7fb1a759e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599139600 0x555599129d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52774/0 (socket says 172.21.15.12:52774) 2024-05-09T00:09:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.976+0000 7fb1a759e700 1 -- 172.21.15.12:0/362315536 learned_addr learned my addr 172.21.15.12:0/362315536 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.976+0000 7fb1a759e700 1 -- 172.21.15.12:0/362315536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599135720 msgr2=0x555599134310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:09:06.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.976+0000 7fb1a759e700 1 --2- 172.21.15.12:0/362315536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599135720 0x555599134310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.976+0000 7fb1a759e700 1 -- 172.21.15.12:0/362315536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599134e00 msgr2=0x55559912f350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:09:06.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.976+0000 7fb1a759e700 1 --2- 172.21.15.12:0/362315536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599134e00 0x55559912f350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.976+0000 7fb1a759e700 1 -- 172.21.15.12:0/362315536 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555599045560 con 0x555599139600 2024-05-09T00:09:06.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.977+0000 7fb1a759e700 1 --2- 172.21.15.12:0/362315536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599139600 0x555599129d20 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00079f0 tx=0x7fb1a000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35e38f5106417c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.977+0000 7fb18dffb700 1 -- 172.21.15.12:0/362315536 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000e050 con 0x555599139600 2024-05-09T00:09:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.977+0000 7fb18dffb700 1 -- 172.21.15.12:0/362315536 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1a0029040 con 0x555599139600 2024-05-09T00:09:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.977+0000 7fb18dffb700 1 -- 172.21.15.12:0/362315536 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0037040 con 0x555599139600 2024-05-09T00:09:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.978+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/362315536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599139600 msgr2=0x555599129d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.978+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/362315536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599139600 0x555599129d20 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00079f0 tx=0x7fb1a000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.978+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/362315536 shutdown_connections 2024-05-09T00:09:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.978+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/362315536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599139600 0x555599129d20 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.978+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/362315536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599134e00 0x55559912f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.978+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/362315536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599135720 0x555599134310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.978+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/362315536 >> 172.21.15.12:0/362315536 conn(0x55559908d210 msgr2=0x555599133c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.981+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/362315536 shutdown_connections 2024-05-09T00:09:06.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.981+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/362315536 wait complete. 2024-05-09T00:09:06.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.981+0000 7fb1b91d6b80 1 -- start start 2024-05-09T00:09:06.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.981+0000 7fb1b91d6b80 1 Processor -- start 2024-05-09T00:09:06.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x55559919abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x55559919b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1b91d6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599139600 0x55559913e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559919b9f0 con 0x555599135720 2024-05-09T00:09:06.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559913e6c0 con 0x555599139600 2024-05-09T00:09:06.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1b91d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559913e830 con 0x555599134e00 2024-05-09T00:09:06.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1b91d6b80 10 client.0 _ll_register_callbacks cb 0x555599024b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:06.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1a6d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x55559919abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1a6d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x55559919abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42168/0 (socket says 172.21.15.12:42168) 2024-05-09T00:09:06.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1a6d9d700 1 -- 172.21.15.12:0/1727297369 learned_addr learned my addr 172.21.15.12:0/1727297369 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:06.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.982+0000 7fb1a659c700 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x55559919b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.983+0000 7fb1a6d9d700 1 -- 172.21.15.12:0/1727297369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599139600 msgr2=0x55559913e180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:09:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.983+0000 7fb1a6d9d700 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599139600 0x55559913e180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.983+0000 7fb1a6d9d700 1 -- 172.21.15.12:0/1727297369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 msgr2=0x55559919b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.983+0000 7fb1a6d9d700 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x55559919b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.983+0000 7fb1a6d9d700 1 -- 172.21.15.12:0/1727297369 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555598f003b0 con 0x555599134e00 2024-05-09T00:09:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.983+0000 7fb1a6d9d700 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x55559919abc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb198009010 tx=0x7fb198009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2cfc2d580c456d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.983+0000 7fb18aff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:09:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.983+0000 7fb18aff5700 1 -- 172.21.15.12:0/1727297369 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198011c30 con 0x555599134e00 2024-05-09T00:09:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.983+0000 7fb18aff5700 1 -- 172.21.15.12:0/1727297369 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb198011d90 con 0x555599134e00 2024-05-09T00:09:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.983+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1727297369 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1a00093d0 con 0x555599134e00 2024-05-09T00:09:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.986+0000 7fb18aff5700 1 -- 172.21.15.12:0/1727297369 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb198014070 con 0x555599134e00 2024-05-09T00:09:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.986+0000 7fb1b91d6b80 10 client.31339 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.986+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1727297369 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x555598f04cd0 con 0x555599134e00 2024-05-09T00:09:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.987+0000 7fb18aff5700 1 -- 172.21.15.12:0/1727297369 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1a00093d0 con 0x555599134e00 2024-05-09T00:09:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.987+0000 7fb18aff5700 1 -- 172.21.15.12:0/1727297369 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb198013880 con 0x555599134e00 2024-05-09T00:09:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.987+0000 7fb1b91d6b80 10 client.31339 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.987+0000 7fb1b91d6b80 10 client.31339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.987+0000 7fb1b91d6b80 20 client.31339 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:06.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.987+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5555991b1a30 0x5555991b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.987+0000 7fb1b91d6b80 4 client.31339 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:09:06.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.987+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1727297369 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555599077ec0 con 0x5555991b1a30 2024-05-09T00:09:06.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.992+0000 7fb1a659c700 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5555991b1a30 0x5555991b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.997+0000 7fb18aff5700 10 client.31339 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:06.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:06.997+0000 7fb1a659c700 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5555991b1a30 0x5555991b3e50 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da2b7f2052a3d12c server_cookie=6b8ee001b3ef3b81 in_seq=0 out_seq=0 2024-05-09T00:09:06.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.002+0000 7fb18aff5700 1 -- 172.21.15.12:0/1727297369 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555599077ec0 con 0x5555991b1a30 2024-05-09T00:09:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.002+0000 7fb18aff5700 10 client.31339 handle_command_reply: tid=0 2024-05-09T00:09:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.003+0000 7fb1b91d6b80 1 client.31339 shutdown 2024-05-09T00:09:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.003+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1727297369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 msgr2=0x55559919abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.003+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x55559919abc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb198009010 tx=0x7fb198009ee0 comp rx=0 tx=0).stop 2024-05-09T00:09:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.003+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1727297369 shutdown_connections 2024-05-09T00:09:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.003+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5555991b1a30 0x5555991b3e50 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.003+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555599139600 0x55559913e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.003+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555599134e00 0x55559919abc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.003+0000 7fb1b91d6b80 1 --2- 172.21.15.12:0/1727297369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555599135720 0x55559919b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.003+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1727297369 >> 172.21.15.12:0/1727297369 conn(0x55559908d210 msgr2=0x555599135bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.005+0000 7fb18aff5700 0 client.31339 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.005+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1727297369 shutdown_connections 2024-05-09T00:09:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.005+0000 7fb1b91d6b80 1 -- 172.21.15.12:0/1727297369 wait complete. 2024-05-09T00:09:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.008+0000 7fb1b91d6b80 20 client.31339 trim_cache size 0 max 16384 2024-05-09T00:09:07.004 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:09:07.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.014+0000 7fb1ad01c700 1 -- 172.21.15.12:0/1543880020 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1900a1460 msgr2=0x7fb1900a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:07.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.014+0000 7fb1ad01c700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1900a1460 0x7fb1900a3920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0008e70 tx=0x7fb1a0009540 comp rx=0 tx=0).stop 2024-05-09T00:09:07.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.014+0000 7fb1ad01c700 1 -- 172.21.15.12:0/1543880020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a811a780 msgr2=0x7fb1a81a69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:07.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.014+0000 7fb1ad01c700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a811a780 0x7fb1a81a69d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00b500 tx=0x7fb19c00b8c0 comp rx=0 tx=0).stop 2024-05-09T00:09:07.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.026+0000 7fb1ad01c700 1 -- 172.21.15.12:0/1543880020 shutdown_connections 2024-05-09T00:09:07.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.026+0000 7fb1ad01c700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1900a1460 0x7fb1900a3920 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:07.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.026+0000 7fb1ad01c700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1a81aad70 0x7fb1a81ab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:07.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.026+0000 7fb1ad01c700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a810fb10 0x7fb1a81a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:07.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.026+0000 7fb1ad01c700 1 --2- 172.21.15.12:0/1543880020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a811a780 0x7fb1a81a69d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:07.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.026+0000 7fb1ad01c700 1 -- 172.21.15.12:0/1543880020 >> 172.21.15.12:0/1543880020 conn(0x7fb1a806bff0 msgr2=0x7fb1a8118770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.026+0000 7fb1ad01c700 1 -- 172.21.15.12:0/1543880020 shutdown_connections 2024-05-09T00:09:07.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:07.032+0000 7fb1ad01c700 1 -- 172.21.15.12:0/1543880020 wait complete. 2024-05-09T00:09:07.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 321632 29.98 MB/sec execute 422 sec latency 1134.649 ms 2024-05-09T00:09:08.063 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:09:08.063 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:09:08.101 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:07 smithi012 ceph-mon[42503]: pgmap v731: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.5 MiB/s wr, 746 op/s 2024-05-09T00:09:08.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:07 smithi204 ceph-mon[47373]: pgmap v731: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.5 MiB/s wr, 746 op/s 2024-05-09T00:09:08.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:07 smithi160 ceph-mon[46152]: pgmap v731: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.5 MiB/s wr, 746 op/s 2024-05-09T00:09:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.275+0000 7f61469f4700 1 -- 172.21.15.12:0/1658155447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6140104130 msgr2=0x7f6140104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.275+0000 7f61469f4700 1 --2- 172.21.15.12:0/1658155447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6140104130 0x7f6140104510 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f613001a9d0 tx=0x7f613001ace0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.275+0000 7f61469f4700 1 -- 172.21.15.12:0/1658155447 shutdown_connections 2024-05-09T00:09:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.275+0000 7f61469f4700 1 --2- 172.21.15.12:0/1658155447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61400ffd80 0x7f614010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.275+0000 7f61469f4700 1 --2- 172.21.15.12:0/1658155447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6140104130 0x7f6140104510 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.275+0000 7f61469f4700 1 --2- 172.21.15.12:0/1658155447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61400ff3c0 0x7f61400ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.275+0000 7f61469f4700 1 -- 172.21.15.12:0/1658155447 >> 172.21.15.12:0/1658155447 conn(0x7f6140074d40 msgr2=0x7f6140075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.281+0000 7f61469f4700 1 -- 172.21.15.12:0/1658155447 shutdown_connections 2024-05-09T00:09:08.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.283+0000 7f61469f4700 1 -- 172.21.15.12:0/1658155447 wait complete. 2024-05-09T00:09:08.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.284+0000 7f61469f4700 1 Processor -- start 2024-05-09T00:09:08.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.284+0000 7f61469f4700 1 -- start start 2024-05-09T00:09:08.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.285+0000 7f61469f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61400ff3c0 0x7f6140108880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.285+0000 7f61469f4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61400ffd80 0x7f6140108dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.285+0000 7f61469f4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6140104130 0x7f61401bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.285+0000 7f61469f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61401bbdd0 con 0x7f61400ff3c0 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.285+0000 7f61469f4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61401bbf10 con 0x7f6140104130 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.285+0000 7f61469f4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61401bc0a0 con 0x7f61400ffd80 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.285+0000 7f613ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61400ff3c0 0x7f6140108880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.285+0000 7f613ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61400ff3c0 0x7f6140108880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52808/0 (socket says 172.21.15.12:52808) 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.285+0000 7f613ffff700 1 -- 172.21.15.12:0/2580007973 learned_addr learned my addr 172.21.15.12:0/2580007973 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.286+0000 7f613ffff700 1 -- 172.21.15.12:0/2580007973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61400ffd80 msgr2=0x7f6140108dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.286+0000 7f613ffff700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61400ffd80 0x7f6140108dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.286+0000 7f613ffff700 1 -- 172.21.15.12:0/2580007973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6140104130 msgr2=0x7f61401bb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.286+0000 7f613ffff700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6140104130 0x7f61401bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.286+0000 7f613ffff700 1 -- 172.21.15.12:0/2580007973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613001a600 con 0x7f61400ff3c0 2024-05-09T00:09:08.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.286+0000 7f613ffff700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61400ff3c0 0x7f6140108880 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f613001c590 tx=0x7f613001ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.291+0000 7f613d7fa700 1 -- 172.21.15.12:0/2580007973 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130003c70 con 0x7f61400ff3c0 2024-05-09T00:09:08.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.291+0000 7f613d7fa700 1 -- 172.21.15.12:0/2580007973 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6130003dd0 con 0x7f61400ff3c0 2024-05-09T00:09:08.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.291+0000 7f61469f4700 1 -- 172.21.15.12:0/2580007973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614006bc80 con 0x7f61400ff3c0 2024-05-09T00:09:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.291+0000 7f613d7fa700 1 -- 172.21.15.12:0/2580007973 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613001f6e0 con 0x7f61400ff3c0 2024-05-09T00:09:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.291+0000 7f61469f4700 1 -- 172.21.15.12:0/2580007973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614006c130 con 0x7f61400ff3c0 2024-05-09T00:09:08.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 321871 29.95 MB/sec execute 423 sec latency 599.389 ms 2024-05-09T00:09:08.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.296+0000 7f6152baeb80 1 Processor -- start 2024-05-09T00:09:08.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.296+0000 7f6152baeb80 1 -- start start 2024-05-09T00:09:08.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597fe80 0x5571a5980260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a59807a0 0x5571a597a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597a910 0x5571a597ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f6152baeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a58d11d0 con 0x5571a59807a0 2024-05-09T00:09:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f6152baeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a58ceb90 con 0x5571a597fe80 2024-05-09T00:09:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f6152baeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a58ca480 con 0x5571a597a910 2024-05-09T00:09:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f6144f91700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597fe80 0x5571a5980260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f6144f91700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597fe80 0x5571a5980260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:51602/0 (socket says 172.21.15.12:51602) 2024-05-09T00:09:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f6144f91700 1 -- 172.21.15.12:0/3116352478 learned_addr learned my addr 172.21.15.12:0/3116352478 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f613f7fe700 1 --2- 172.21.15.12:0/3116352478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597a910 0x5571a597ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f613f7fe700 1 -- 172.21.15.12:0/3116352478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597fe80 msgr2=0x5571a5980260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f613f7fe700 1 --2- 172.21.15.12:0/3116352478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597fe80 0x5571a5980260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f613f7fe700 1 -- 172.21.15.12:0/3116352478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a59807a0 msgr2=0x5571a597a3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:09:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f613f7fe700 1 --2- 172.21.15.12:0/3116352478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a59807a0 0x5571a597a3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.297+0000 7f613f7fe700 1 -- 172.21.15.12:0/3116352478 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a574b430 con 0x5571a597a910 2024-05-09T00:09:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.298+0000 7f613f7fe700 1 --2- 172.21.15.12:0/3116352478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597a910 0x5571a597ad50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f61340097c0 tx=0x7f6134009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=945ebb0d5d1c5e23 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.298+0000 7f61267fc700 1 -- 172.21.15.12:0/3116352478 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134011830 con 0x5571a597a910 2024-05-09T00:09:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.298+0000 7f61267fc700 1 -- 172.21.15.12:0/3116352478 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6134011e50 con 0x5571a597a910 2024-05-09T00:09:08.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.298+0000 7f61267fc700 1 -- 172.21.15.12:0/3116352478 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613400f2e0 con 0x5571a597a910 2024-05-09T00:09:08.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f6152baeb80 1 -- 172.21.15.12:0/3116352478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597a910 msgr2=0x5571a597ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f6152baeb80 1 --2- 172.21.15.12:0/3116352478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597a910 0x5571a597ad50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f61340097c0 tx=0x7f6134009ad0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f6152baeb80 1 -- 172.21.15.12:0/3116352478 shutdown_connections 2024-05-09T00:09:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f6152baeb80 1 --2- 172.21.15.12:0/3116352478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597a910 0x5571a597ad50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f6152baeb80 1 --2- 172.21.15.12:0/3116352478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597fe80 0x5571a5980260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f6152baeb80 1 --2- 172.21.15.12:0/3116352478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a59807a0 0x5571a597a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f6152baeb80 1 -- 172.21.15.12:0/3116352478 >> 172.21.15.12:0/3116352478 conn(0x5571a58d8290 msgr2=0x5571a5974e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f6152baeb80 1 -- 172.21.15.12:0/3116352478 shutdown_connections 2024-05-09T00:09:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f6152baeb80 1 -- 172.21.15.12:0/3116352478 wait complete. 2024-05-09T00:09:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f613d7fa700 1 -- 172.21.15.12:0/2580007973 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f613001f840 con 0x7f61400ff3c0 2024-05-09T00:09:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.299+0000 7f6152baeb80 1 -- start start 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.300+0000 7f6152baeb80 1 Processor -- start 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.301+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597a910 0x5571a5983c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.301+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597fe80 0x5571a5984170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.301+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a59846b0 0x5571a5990300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.301+0000 7f6152baeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a58665e0 con 0x5571a597a910 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.301+0000 7f6152baeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5866280 con 0x5571a59846b0 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.301+0000 7f6152baeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a574d720 con 0x5571a597fe80 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.301+0000 7f6152baeb80 10 client.0 _ll_register_callbacks cb 0x5571a586fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f6144f91700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597a910 0x5571a5983c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f6144f91700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597a910 0x5571a5983c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52812/0 (socket says 172.21.15.12:52812) 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f6144f91700 1 -- 172.21.15.12:0/3535162341 learned_addr learned my addr 172.21.15.12:0/3535162341 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f613d7fa700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61280a1370 0x7f61280a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f613d7fa700 1 -- 172.21.15.12:0/2580007973 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f61300ce490 con 0x7f61400ff3c0 2024-05-09T00:09:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f6144f91700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61280a1370 0x7f61280a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f6144f91700 1 -- 172.21.15.12:0/3535162341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597fe80 msgr2=0x5571a5984170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f6144f91700 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597fe80 0x5571a5984170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f6144f91700 1 -- 172.21.15.12:0/3535162341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a59846b0 msgr2=0x5571a5990300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f6144f91700 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a59846b0 0x5571a5990300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.302+0000 7f6144f91700 1 -- 172.21.15.12:0/3535162341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6134009420 con 0x5571a597a910 2024-05-09T00:09:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.303+0000 7f6144f91700 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597a910 0x5571a5983c30 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6138015d20 tx=0x7f6138012540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc268956bba80b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.303+0000 7f6144f91700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61280a1370 0x7f61280a3830 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6138020000 tx=0x7f613801e000 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.306+0000 7f611f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.306+0000 7f611f7fe700 1 -- 172.21.15.12:0/3535162341 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138021d80 con 0x5571a597a910 2024-05-09T00:09:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.306+0000 7f611f7fe700 1 -- 172.21.15.12:0/3535162341 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f61380183f0 con 0x5571a597a910 2024-05-09T00:09:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.306+0000 7f6152baeb80 1 -- 172.21.15.12:0/3535162341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571a58232f0 con 0x5571a597a910 2024-05-09T00:09:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.307+0000 7f611f7fe700 1 -- 172.21.15.12:0/3535162341 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61380185b0 con 0x5571a597a910 2024-05-09T00:09:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.310+0000 7f6152baeb80 10 client.21648 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.310+0000 7f6152baeb80 1 -- 172.21.15.12:0/3535162341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5571a56f7be0 con 0x5571a597a910 2024-05-09T00:09:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.313+0000 7f611f7fe700 1 -- 172.21.15.12:0/3535162341 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571a58232f0 con 0x5571a597a910 2024-05-09T00:09:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.314+0000 7f611f7fe700 1 -- 172.21.15.12:0/3535162341 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6138029520 con 0x5571a597a910 2024-05-09T00:09:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.314+0000 7f6152baeb80 10 client.21648 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.314+0000 7f6152baeb80 10 client.21648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.314+0000 7f6152baeb80 20 client.21648 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.314+0000 7f6152baeb80 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571a599d060 0x5571a599f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.314+0000 7f6152baeb80 4 client.21648 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.314+0000 7f6152baeb80 1 -- 172.21.15.12:0/3535162341 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571a5949890 con 0x5571a599d060 2024-05-09T00:09:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.315+0000 7f613ffff700 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571a599d060 0x5571a599f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.315+0000 7f613ffff700 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571a599d060 0x5571a599f480 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b478c3abf7280c1 server_cookie=dec012e32d33553d in_seq=0 out_seq=0 2024-05-09T00:09:08.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.315+0000 7f611f7fe700 10 client.21648 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:08.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.320+0000 7f611f7fe700 1 -- 172.21.15.12:0/3535162341 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5571a5949890 con 0x5571a599d060 2024-05-09T00:09:08.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.320+0000 7f611f7fe700 10 client.21648 handle_command_reply: tid=0 2024-05-09T00:09:08.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.320+0000 7f6152baeb80 1 client.21648 shutdown 2024-05-09T00:09:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.320+0000 7f6152baeb80 1 -- 172.21.15.12:0/3535162341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597a910 msgr2=0x5571a5983c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.320+0000 7f6152baeb80 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597a910 0x5571a5983c30 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6138015d20 tx=0x7f6138012540 comp rx=0 tx=0).stop 2024-05-09T00:09:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.321+0000 7f6152baeb80 1 -- 172.21.15.12:0/3535162341 shutdown_connections 2024-05-09T00:09:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.321+0000 7f6152baeb80 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571a599d060 0x5571a599f480 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.321+0000 7f6152baeb80 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a59846b0 0x5571a5990300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.321+0000 7f6152baeb80 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597a910 0x5571a5983c30 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.321+0000 7f6152baeb80 1 --2- 172.21.15.12:0/3535162341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597fe80 0x5571a5984170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.321+0000 7f611f7fe700 0 client.21648 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.321+0000 7f6152baeb80 1 -- 172.21.15.12:0/3535162341 >> 172.21.15.12:0/3535162341 conn(0x5571a58d8290 msgr2=0x5571a5974620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.321+0000 7f6152baeb80 1 -- 172.21.15.12:0/3535162341 shutdown_connections 2024-05-09T00:09:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.321+0000 7f6152baeb80 1 -- 172.21.15.12:0/3535162341 wait complete. 2024-05-09T00:09:08.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.321+0000 7f6152baeb80 20 client.21648 trim_cache size 0 max 16384 2024-05-09T00:09:08.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.347+0000 7f6152baeb80 1 Processor -- start 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f6152baeb80 1 -- start start 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a58d8290 0x5571a597f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597a910 0x5571a5974da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597fe80 0x5571a5985be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f6152baeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a59867a0 con 0x5571a58d8290 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f6152baeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5971e40 con 0x5571a597a910 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f6152baeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a59a46d0 con 0x5571a597fe80 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f613ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a58d8290 0x5571a597f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f613ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a58d8290 0x5571a597f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52818/0 (socket says 172.21.15.12:52818) 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f613ffff700 1 -- 172.21.15.12:0/2516512134 learned_addr learned my addr 172.21.15.12:0/2516512134 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.348+0000 7f613f7fe700 1 --2- 172.21.15.12:0/2516512134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597a910 0x5571a5974da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.349+0000 7f613ffff700 1 -- 172.21.15.12:0/2516512134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597fe80 msgr2=0x5571a5985be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.349+0000 7f613ffff700 1 --2- 172.21.15.12:0/2516512134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597fe80 0x5571a5985be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.349+0000 7f613ffff700 1 -- 172.21.15.12:0/2516512134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597a910 msgr2=0x5571a5974da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.349+0000 7f613ffff700 1 --2- 172.21.15.12:0/2516512134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597a910 0x5571a5974da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.349+0000 7f613ffff700 1 -- 172.21.15.12:0/2516512134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a5971560 con 0x5571a58d8290 2024-05-09T00:09:08.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.349+0000 7f613ffff700 1 --2- 172.21.15.12:0/2516512134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a58d8290 0x5571a597f5b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f613001a9a0 tx=0x7f613001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc12fc3e14c8c8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.349+0000 7f6126ffd700 1 -- 172.21.15.12:0/2516512134 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61300b3070 con 0x5571a58d8290 2024-05-09T00:09:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.350+0000 7f6126ffd700 1 -- 172.21.15.12:0/2516512134 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61300bb070 con 0x5571a58d8290 2024-05-09T00:09:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.350+0000 7f6126ffd700 1 -- 172.21.15.12:0/2516512134 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61300a3ad0 con 0x5571a58d8290 2024-05-09T00:09:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.351+0000 7f6152baeb80 1 -- 172.21.15.12:0/2516512134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a58d8290 msgr2=0x5571a597f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.351+0000 7f6152baeb80 1 --2- 172.21.15.12:0/2516512134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a58d8290 0x5571a597f5b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f613001a9a0 tx=0x7f613001ace0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.351+0000 7f6152baeb80 1 -- 172.21.15.12:0/2516512134 shutdown_connections 2024-05-09T00:09:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.351+0000 7f6152baeb80 1 --2- 172.21.15.12:0/2516512134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a597fe80 0x5571a5985be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.351+0000 7f6152baeb80 1 --2- 172.21.15.12:0/2516512134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a58d8290 0x5571a597f5b0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.351+0000 7f6152baeb80 1 --2- 172.21.15.12:0/2516512134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597a910 0x5571a5974da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.351+0000 7f6152baeb80 1 -- 172.21.15.12:0/2516512134 >> 172.21.15.12:0/2516512134 conn(0x5571a5976ca0 msgr2=0x5571a59854d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.351+0000 7f6152baeb80 1 -- 172.21.15.12:0/2516512134 shutdown_connections 2024-05-09T00:09:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.351+0000 7f6152baeb80 1 -- 172.21.15.12:0/2516512134 wait complete. 2024-05-09T00:09:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.351+0000 7f6152baeb80 1 -- start start 2024-05-09T00:09:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f6152baeb80 1 Processor -- start 2024-05-09T00:09:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a58d8290 0x5571a59ed100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597a910 0x5571a59ed640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f6152baeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597fe80 0x5571a59e7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f6152baeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a59e7c30 con 0x5571a597fe80 2024-05-09T00:09:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f6152baeb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a59e7da0 con 0x5571a597a910 2024-05-09T00:09:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f6152baeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a59e7f60 con 0x5571a58d8290 2024-05-09T00:09:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f6152baeb80 10 client.0 _ll_register_callbacks cb 0x5571a586fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f613ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a58d8290 0x5571a59ed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f613ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a58d8290 0x5571a59ed100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:42220/0 (socket says 172.21.15.12:42220) 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.352+0000 7f613ffff700 1 -- 172.21.15.12:0/612352166 learned_addr learned my addr 172.21.15.12:0/612352166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.353+0000 7f6144f91700 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597fe80 0x5571a59e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.353+0000 7f613ffff700 1 -- 172.21.15.12:0/612352166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597a910 msgr2=0x5571a59ed640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.353+0000 7f613ffff700 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597a910 0x5571a59ed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.353+0000 7f613ffff700 1 -- 172.21.15.12:0/612352166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597fe80 msgr2=0x5571a59e7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.353+0000 7f613ffff700 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597fe80 0x5571a59e7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.353+0000 7f613ffff700 1 -- 172.21.15.12:0/612352166 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614006c130 con 0x5571a58d8290 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.353+0000 7f613ffff700 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a58d8290 0x5571a59ed100 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6130005cb0 tx=0x7f613009b070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc72861f13c0b8b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.353+0000 7f611f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.353+0000 7f611f7fe700 1 -- 172.21.15.12:0/612352166 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61300b3050 con 0x5571a58d8290 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.354+0000 7f611f7fe700 1 -- 172.21.15.12:0/612352166 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f613009f480 con 0x5571a58d8290 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.354+0000 7f6152baeb80 1 -- 172.21.15.12:0/612352166 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6134011e50 con 0x5571a58d8290 2024-05-09T00:09:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.354+0000 7f611f7fe700 1 -- 172.21.15.12:0/612352166 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f613000eea0 con 0x5571a58d8290 2024-05-09T00:09:08.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.356+0000 7f6152baeb80 10 client.31348 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:08.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.356+0000 7f6152baeb80 1 -- 172.21.15.12:0/612352166 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5571a58905e0 con 0x5571a58d8290 2024-05-09T00:09:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.356+0000 7f611f7fe700 1 -- 172.21.15.12:0/612352166 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6134011e50 con 0x5571a58d8290 2024-05-09T00:09:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.356+0000 7f611f7fe700 1 -- 172.21.15.12:0/612352166 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f61300b7070 con 0x5571a58d8290 2024-05-09T00:09:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.357+0000 7f6152baeb80 10 client.31348 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.357+0000 7f6152baeb80 10 client.31348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.357+0000 7f6152baeb80 20 client.31348 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:08.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.357+0000 7f6152baeb80 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571a59846b0 0x5571a59f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.357+0000 7f6152baeb80 4 client.31348 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:09:08.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.357+0000 7f6152baeb80 1 -- 172.21.15.12:0/612352166 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5571a58c2f40 con 0x5571a59846b0 2024-05-09T00:09:08.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.361+0000 7f613f7fe700 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571a59846b0 0x5571a59f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.366+0000 7f613f7fe700 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571a59846b0 0x5571a59f6bf0 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4314a364f3d23df server_cookie=fa076ea393c7785d in_seq=0 out_seq=0 2024-05-09T00:09:08.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.367+0000 7f611f7fe700 10 client.31348 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.375+0000 7f611f7fe700 1 -- 172.21.15.12:0/612352166 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5571a58c2f40 con 0x5571a59846b0 2024-05-09T00:09:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.375+0000 7f611f7fe700 10 client.31348 handle_command_reply: tid=0 2024-05-09T00:09:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.375+0000 7f6152baeb80 1 client.31348 shutdown 2024-05-09T00:09:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.375+0000 7f6152baeb80 1 -- 172.21.15.12:0/612352166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a58d8290 msgr2=0x5571a59ed100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.375+0000 7f6152baeb80 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a58d8290 0x5571a59ed100 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6130005cb0 tx=0x7f613009b070 comp rx=0 tx=0).stop 2024-05-09T00:09:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f6152baeb80 1 -- 172.21.15.12:0/612352166 shutdown_connections 2024-05-09T00:09:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f6152baeb80 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571a597fe80 0x5571a59e7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f6152baeb80 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571a58d8290 0x5571a59ed100 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f6152baeb80 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5571a59846b0 0x5571a59f6bf0 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f6152baeb80 1 --2- 172.21.15.12:0/612352166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5571a597a910 0x5571a59ed640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f6152baeb80 1 -- 172.21.15.12:0/612352166 >> 172.21.15.12:0/612352166 conn(0x5571a5976ca0 msgr2=0x5571a5985d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f611f7fe700 0 client.31348 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f6152baeb80 1 -- 172.21.15.12:0/612352166 shutdown_connections 2024-05-09T00:09:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f6152baeb80 1 -- 172.21.15.12:0/612352166 wait complete. 2024-05-09T00:09:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f6152baeb80 20 client.31348 trim_cache size 0 max 16384 2024-05-09T00:09:08.373 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:09:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:09:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "9b1ec53d-22c8-4857-8d0a-fbea50a81826", 2024-05-09T00:09:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:09:08.374 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:09:08.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f61469f4700 1 -- 172.21.15.12:0/2580007973 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61280a1370 msgr2=0x7f61280a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f61469f4700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61280a1370 0x7f61280a3830 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6138020000 tx=0x7f613801e000 comp rx=0 tx=0).stop 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f61469f4700 1 -- 172.21.15.12:0/2580007973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61400ff3c0 msgr2=0x7f6140108880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.376+0000 7f61469f4700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61400ff3c0 0x7f6140108880 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f613001c590 tx=0x7f613001ccf0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.377+0000 7f61469f4700 1 -- 172.21.15.12:0/2580007973 shutdown_connections 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.377+0000 7f61469f4700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f61280a1370 0x7f61280a3830 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.377+0000 7f61469f4700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6140104130 0x7f61401bb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.377+0000 7f61469f4700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61400ff3c0 0x7f6140108880 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.377+0000 7f61469f4700 1 --2- 172.21.15.12:0/2580007973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61400ffd80 0x7f6140108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.377+0000 7f61469f4700 1 -- 172.21.15.12:0/2580007973 >> 172.21.15.12:0/2580007973 conn(0x7f6140074d40 msgr2=0x7f61400fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.377+0000 7f61469f4700 1 -- 172.21.15.12:0/2580007973 shutdown_connections 2024-05-09T00:09:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.377+0000 7f61469f4700 1 -- 172.21.15.12:0/2580007973 wait complete. 2024-05-09T00:09:08.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:09:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.646+0000 7fdcbb78c700 1 -- 172.21.15.12:0/2365114651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcb4101160 msgr2=0x7fdcb4101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.646+0000 7fdcbb78c700 1 --2- 172.21.15.12:0/2365114651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcb4101160 0x7fdcb4101540 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009990 tx=0x7fdcb0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.646+0000 7fdcbb78c700 1 -- 172.21.15.12:0/2365114651 shutdown_connections 2024-05-09T00:09:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.646+0000 7fdcbb78c700 1 --2- 172.21.15.12:0/2365114651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb4106280 0x7fdcb410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.646+0000 7fdcbb78c700 1 --2- 172.21.15.12:0/2365114651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcb4101160 0x7fdcb4101540 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.646+0000 7fdcbb78c700 1 --2- 172.21.15.12:0/2365114651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb4101b10 0x7fdcb4105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.646+0000 7fdcbb78c700 1 -- 172.21.15.12:0/2365114651 >> 172.21.15.12:0/2365114651 conn(0x7fdcb40fc9e0 msgr2=0x7fdcb40fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.646+0000 7fdcbb78c700 1 -- 172.21.15.12:0/2365114651 shutdown_connections 2024-05-09T00:09:08.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.647+0000 7fdcbb78c700 1 -- 172.21.15.12:0/2365114651 wait complete. 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.647+0000 7fdcbb78c700 1 Processor -- start 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.647+0000 7fdcbb78c700 1 -- start start 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.648+0000 7fdcbb78c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb4101160 0x7fdcb40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.648+0000 7fdcbb78c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb4106280 0x7fdcb40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.648+0000 7fdcbb78c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcb4074f10 0x7fdcb4073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.648+0000 7fdcbb78c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb41aa130 con 0x7fdcb4101160 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.648+0000 7fdcbb78c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb41aa2a0 con 0x7fdcb4106280 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.648+0000 7fdcbb78c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb41aa460 con 0x7fdcb4074f10 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.648+0000 7fdcb8d27700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb4106280 0x7fdcb40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.648+0000 7fdcb8d27700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb4106280 0x7fdcb40737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47220/0 (socket says 172.21.15.12:47220) 2024-05-09T00:09:08.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.648+0000 7fdcb8d27700 1 -- 172.21.15.12:0/684132573 learned_addr learned my addr 172.21.15.12:0/684132573 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:08.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.648+0000 7fdcb9d29700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcb4074f10 0x7fdcb4073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.649+0000 7fdcb8d27700 1 -- 172.21.15.12:0/684132573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcb4074f10 msgr2=0x7fdcb4073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.649+0000 7fdcb8d27700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcb4074f10 0x7fdcb4073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.649+0000 7fdcb8d27700 1 -- 172.21.15.12:0/684132573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb4101160 msgr2=0x7fdcb40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.649+0000 7fdcb8d27700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb4101160 0x7fdcb40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.649+0000 7fdcb8d27700 1 -- 172.21.15.12:0/684132573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb00095f0 con 0x7fdcb4106280 2024-05-09T00:09:08.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.649+0000 7fdcb8d27700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb4106280 0x7fdcb40737e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdca400b500 tx=0x7fdca400b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.650+0000 7fdcaa7fc700 1 -- 172.21.15.12:0/684132573 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4011830 con 0x7fdcb4106280 2024-05-09T00:09:08.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.650+0000 7fdcaa7fc700 1 -- 172.21.15.12:0/684132573 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdca4011e50 con 0x7fdcb4106280 2024-05-09T00:09:08.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.650+0000 7fdcaa7fc700 1 -- 172.21.15.12:0/684132573 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca400f2e0 con 0x7fdcb4106280 2024-05-09T00:09:08.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.652+0000 7fdcbb78c700 1 -- 172.21.15.12:0/684132573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb41aa740 con 0x7fdcb4106280 2024-05-09T00:09:08.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.652+0000 7fdcbb78c700 1 -- 172.21.15.12:0/684132573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb41aabd0 con 0x7fdcb4106280 2024-05-09T00:09:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.658+0000 7fdcaa7fc700 1 -- 172.21.15.12:0/684132573 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdca4010560 con 0x7fdcb4106280 2024-05-09T00:09:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.658+0000 7fdcc7946b80 1 Processor -- start 2024-05-09T00:09:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.659+0000 7fdcaa7fc700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdca00a14a0 0x7fdca00a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.659+0000 7fdcaa7fc700 1 -- 172.21.15.12:0/684132573 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdca40cc5a0 con 0x7fdcb4106280 2024-05-09T00:09:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.660+0000 7fdcb9d29700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdca00a14a0 0x7fdca00a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.660+0000 7fdcb9d29700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdca00a14a0 0x7fdca00a3960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009f60 tx=0x7fdcac00d040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.660+0000 7fdcc7946b80 1 -- start start 2024-05-09T00:09:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.661+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c330549e00 0x55c33054a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.661+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c33054a720 0x55c330544350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.661+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330544cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.661+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33049b150 con 0x55c33054a720 2024-05-09T00:09:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.661+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c330498b10 con 0x55c330549e00 2024-05-09T00:09:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.661+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c330494400 con 0x55c330544890 2024-05-09T00:09:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.661+0000 7fdcb9d29700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330544cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.662+0000 7fdcb9d29700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330544cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40928/0 (socket says 172.21.15.12:40928) 2024-05-09T00:09:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.662+0000 7fdcb9d29700 1 -- 172.21.15.12:0/3940310266 learned_addr learned my addr 172.21.15.12:0/3940310266 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.662+0000 7fdcb9d29700 1 -- 172.21.15.12:0/3940310266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c330549e00 msgr2=0x55c33054a1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:09:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.662+0000 7fdcb9d29700 1 --2- 172.21.15.12:0/3940310266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c330549e00 0x55c33054a1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.662+0000 7fdcb9d29700 1 -- 172.21.15.12:0/3940310266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c33054a720 msgr2=0x55c330544350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:09:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.662+0000 7fdcb9d29700 1 --2- 172.21.15.12:0/3940310266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c33054a720 0x55c330544350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.662+0000 7fdcb9d29700 1 -- 172.21.15.12:0/3940310266 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3303153b0 con 0x55c330544890 2024-05-09T00:09:08.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.666+0000 7fdcb9d29700 1 --2- 172.21.15.12:0/3940310266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330544cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0165d0 tx=0x7fdcac011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1be404b5be84de6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.666+0000 7fdc9f7fe700 1 -- 172.21.15.12:0/3940310266 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac010040 con 0x55c330544890 2024-05-09T00:09:08.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.666+0000 7fdc9f7fe700 1 -- 172.21.15.12:0/3940310266 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcac01c040 con 0x55c330544890 2024-05-09T00:09:08.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.666+0000 7fdc9f7fe700 1 -- 172.21.15.12:0/3940310266 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac020920 con 0x55c330544890 2024-05-09T00:09:08.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.667+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3940310266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 msgr2=0x55c330544cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.667+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/3940310266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330544cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0165d0 tx=0x7fdcac011000 comp rx=0 tx=0).stop 2024-05-09T00:09:08.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.669+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3940310266 shutdown_connections 2024-05-09T00:09:08.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.669+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/3940310266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330544cd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.669+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/3940310266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c330549e00 0x55c33054a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.669+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/3940310266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c33054a720 0x55c330544350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.669+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3940310266 >> 172.21.15.12:0/3940310266 conn(0x55c3304a2210 msgr2=0x55c33053edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.675+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3940310266 shutdown_connections 2024-05-09T00:09:08.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.676+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3940310266 wait complete. 2024-05-09T00:09:08.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.676+0000 7fdcc7946b80 1 -- start start 2024-05-09T00:09:08.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.676+0000 7fdcc7946b80 1 Processor -- start 2024-05-09T00:09:08.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.678+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c330544890 0x55c33054da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.678+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330549e00 0x55c33054dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.678+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33054e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.678+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c330430560 con 0x55c330544890 2024-05-09T00:09:08.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.678+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c330430200 con 0x55c33054a720 2024-05-09T00:09:08.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.678+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3303176a0 con 0x55c330549e00 2024-05-09T00:09:08.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.678+0000 7fdcc7946b80 10 client.0 _ll_register_callbacks cb 0x55c330439b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:08.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.678+0000 7fdcb9d29700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33054e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.678+0000 7fdcb9d29700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33054e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47222/0 (socket says 172.21.15.12:47222) 2024-05-09T00:09:08.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.678+0000 7fdcb9d29700 1 -- 172.21.15.12:0/1718179827 learned_addr learned my addr 172.21.15.12:0/1718179827 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:08.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.679+0000 7fdcb9d29700 1 -- 172.21.15.12:0/1718179827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330549e00 msgr2=0x55c33054dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.679+0000 7fdcb9d29700 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330549e00 0x55c33054dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.679+0000 7fdcb9d29700 1 -- 172.21.15.12:0/1718179827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c330544890 msgr2=0x55c33054da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.679+0000 7fdcb9d29700 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c330544890 0x55c33054da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.679+0000 7fdcb9d29700 1 -- 172.21.15.12:0/1718179827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcac00ae70 con 0x55c33054a720 2024-05-09T00:09:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.679+0000 7fdcb9d29700 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33054e4e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdcac004c20 tx=0x7fdcac004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb25ca126f67dc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.679+0000 7fdc9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:09:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.679+0000 7fdc9cff9700 1 -- 172.21.15.12:0/1718179827 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac0209b0 con 0x55c33054a720 2024-05-09T00:09:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.679+0000 7fdc9cff9700 1 -- 172.21.15.12:0/1718179827 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdcac006e40 con 0x55c33054a720 2024-05-09T00:09:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.680+0000 7fdcc7946b80 1 -- 172.21.15.12:0/1718179827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3303ed270 con 0x55c33054a720 2024-05-09T00:09:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.680+0000 7fdc9cff9700 1 -- 172.21.15.12:0/1718179827 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcac020b10 con 0x55c33054a720 2024-05-09T00:09:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.680+0000 7fdcc7946b80 10 client.31073 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.680+0000 7fdcc7946b80 1 -- 172.21.15.12:0/1718179827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55c3302c1b60 con 0x55c33054a720 2024-05-09T00:09:08.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.685+0000 7fdc9cff9700 1 -- 172.21.15.12:0/1718179827 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3303ed270 con 0x55c33054a720 2024-05-09T00:09:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.685+0000 7fdc9cff9700 1 -- 172.21.15.12:0/1718179827 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdcac01f070 con 0x55c33054a720 2024-05-09T00:09:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.685+0000 7fdcc7946b80 10 client.31073 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.685+0000 7fdcc7946b80 10 client.31073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.685+0000 7fdcc7946b80 20 client.31073 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.685+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c33056f0e0 0x55c3305714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.685+0000 7fdcc7946b80 4 client.31073 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.685+0000 7fdcc7946b80 1 -- 172.21.15.12:0/1718179827 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c330513810 con 0x55c33056f0e0 2024-05-09T00:09:08.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.687+0000 7fdcb9528700 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c33056f0e0 0x55c3305714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.694+0000 7fdcb9528700 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c33056f0e0 0x55c3305714d0 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e601e92ff6e4b5f1 server_cookie=2661b482179d5e3f in_seq=0 out_seq=0 2024-05-09T00:09:08.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.695+0000 7fdc9cff9700 10 client.31073 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:08.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdc9cff9700 1 -- 172.21.15.12:0/1718179827 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c330513810 con 0x55c33056f0e0 2024-05-09T00:09:08.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdc9cff9700 10 client.31073 handle_command_reply: tid=0 2024-05-09T00:09:08.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdcc7946b80 1 client.31073 shutdown 2024-05-09T00:09:08.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdcc7946b80 1 -- 172.21.15.12:0/1718179827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 msgr2=0x55c33054e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33054e4e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdcac004c20 tx=0x7fdcac004ab0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdcc7946b80 1 -- 172.21.15.12:0/1718179827 shutdown_connections 2024-05-09T00:09:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c33056f0e0 0x55c3305714d0 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33054e4e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c330544890 0x55c33054da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdc9cff9700 0 client.31073 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/1718179827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330549e00 0x55c33054dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.700+0000 7fdcc7946b80 1 -- 172.21.15.12:0/1718179827 >> 172.21.15.12:0/1718179827 conn(0x55c3304a2210 msgr2=0x55c33053e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.701+0000 7fdcc7946b80 1 -- 172.21.15.12:0/1718179827 shutdown_connections 2024-05-09T00:09:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.701+0000 7fdcc7946b80 1 -- 172.21.15.12:0/1718179827 wait complete. 2024-05-09T00:09:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.701+0000 7fdcc7946b80 20 client.31073 trim_cache size 0 max 16384 2024-05-09T00:09:08.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.730+0000 7fdcc7946b80 1 Processor -- start 2024-05-09T00:09:08.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.730+0000 7fdcc7946b80 1 -- start start 2024-05-09T00:09:08.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.730+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330544350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.730+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c330549e00 0x55c3305493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.730+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33053ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.730+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c330549900 con 0x55c330549e00 2024-05-09T00:09:08.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.730+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c330539800 con 0x55c33054a720 2024-05-09T00:09:08.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.730+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33054a520 con 0x55c330544890 2024-05-09T00:09:08.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.731+0000 7fdcb9d29700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33053ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.731+0000 7fdcb9d29700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33053ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47238/0 (socket says 172.21.15.12:47238) 2024-05-09T00:09:08.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.731+0000 7fdcb9d29700 1 -- 172.21.15.12:0/409883633 learned_addr learned my addr 172.21.15.12:0/409883633 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:08.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.731+0000 7fdcb8d27700 1 --2- 172.21.15.12:0/409883633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c330549e00 0x55c3305493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.731+0000 7fdcb8d27700 1 -- 172.21.15.12:0/409883633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 msgr2=0x55c330544350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:09:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.731+0000 7fdcb8d27700 1 --2- 172.21.15.12:0/409883633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330544350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.731+0000 7fdcb8d27700 1 -- 172.21.15.12:0/409883633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 msgr2=0x55c33053ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.731+0000 7fdcb8d27700 1 --2- 172.21.15.12:0/409883633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33053ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.731+0000 7fdcb8d27700 1 -- 172.21.15.12:0/409883633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c330544f40 con 0x55c330549e00 2024-05-09T00:09:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.732+0000 7fdcb8d27700 1 --2- 172.21.15.12:0/409883633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c330549e00 0x55c3305493c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fdca40c9d90 tx=0x7fdca40995f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67182ec42ced82a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.732+0000 7fdc9ffff700 1 -- 172.21.15.12:0/409883633 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4099e50 con 0x55c330549e00 2024-05-09T00:09:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.732+0000 7fdc9ffff700 1 -- 172.21.15.12:0/409883633 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdca4099750 con 0x55c330549e00 2024-05-09T00:09:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.732+0000 7fdc9ffff700 1 -- 172.21.15.12:0/409883633 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca409d380 con 0x55c330549e00 2024-05-09T00:09:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.733+0000 7fdcc7946b80 1 -- 172.21.15.12:0/409883633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c330549e00 msgr2=0x55c3305493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.733+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/409883633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c330549e00 0x55c3305493c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fdca40c9d90 tx=0x7fdca40995f0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.733+0000 7fdcc7946b80 1 -- 172.21.15.12:0/409883633 shutdown_connections 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.733+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/409883633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c33054a720 0x55c33053ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.733+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/409883633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330544350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.733+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/409883633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c330549e00 0x55c3305493c0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.733+0000 7fdcc7946b80 1 -- 172.21.15.12:0/409883633 >> 172.21.15.12:0/409883633 conn(0x55c3304a2210 msgr2=0x55c33054c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.733+0000 7fdcc7946b80 1 -- 172.21.15.12:0/409883633 shutdown_connections 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.733+0000 7fdcc7946b80 1 -- 172.21.15.12:0/409883633 wait complete. 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.734+0000 7fdcc7946b80 1 -- start start 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.734+0000 7fdcc7946b80 1 Processor -- start 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.734+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330564c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.734+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c330549e00 0x55c330565160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.734+0000 7fdcc7946b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c33054a720 0x55c3305656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.734+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3305c55c0 con 0x55c33054a720 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.734+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3305c5730 con 0x55c330549e00 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.734+0000 7fdcc7946b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3305c58f0 con 0x55c330544890 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdcc7946b80 10 client.0 _ll_register_callbacks cb 0x55c330439b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdcb9528700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330564c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdcb9528700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330564c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:40954/0 (socket says 172.21.15.12:40954) 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdcb9528700 1 -- 172.21.15.12:0/3908056100 learned_addr learned my addr 172.21.15.12:0/3908056100 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdcb9528700 1 -- 172.21.15.12:0/3908056100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c330549e00 msgr2=0x55c330565160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdcb9528700 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c330549e00 0x55c330565160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdcb9528700 1 -- 172.21.15.12:0/3908056100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c33054a720 msgr2=0x55c3305656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdcb9528700 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c33054a720 0x55c3305656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdcb9528700 1 -- 172.21.15.12:0/3908056100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb41aabd0 con 0x55c330544890 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdcb9528700 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330564c20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b5c0 tx=0x7fdcb00050f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ad3344ed4e731e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.735+0000 7fdc9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:09:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.736+0000 7fdc9cff9700 1 -- 172.21.15.12:0/3908056100 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb001d040 con 0x55c330544890 2024-05-09T00:09:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.736+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3908056100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcac01c040 con 0x55c330544890 2024-05-09T00:09:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.736+0000 7fdc9cff9700 1 -- 172.21.15.12:0/3908056100 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdcb00056a0 con 0x55c330544890 2024-05-09T00:09:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.736+0000 7fdc9cff9700 1 -- 172.21.15.12:0/3908056100 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcb0021a90 con 0x55c330544890 2024-05-09T00:09:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.736+0000 7fdcc7946b80 10 client.31360 fetch_fsmap learned FSMap version 78 2024-05-09T00:09:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.736+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3908056100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55c33045a560 con 0x55c330544890 2024-05-09T00:09:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.739+0000 7fdc9cff9700 1 -- 172.21.15.12:0/3908056100 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcac01c040 con 0x55c330544890 2024-05-09T00:09:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.739+0000 7fdc9cff9700 1 -- 172.21.15.12:0/3908056100 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdcb000bd40 con 0x55c330544890 2024-05-09T00:09:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.739+0000 7fdcc7946b80 10 client.31360 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T00:09:08.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.739+0000 7fdcc7946b80 10 client.31360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:08.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.739+0000 7fdcc7946b80 20 client.31360 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:08.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.739+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3305c70e0 0x55c3305c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:08.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.739+0000 7fdcc7946b80 4 client.31360 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:09:08.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.739+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3908056100 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c33048cec0 con 0x55c3305c70e0 2024-05-09T00:09:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.745+0000 7fdcb8d27700 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3305c70e0 0x55c3305c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:08.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.747+0000 7fdcb8d27700 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3305c70e0 0x55c3305c9520 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b8971f4ab22701 server_cookie=b4bf61ec2e711a75 in_seq=0 out_seq=0 2024-05-09T00:09:08.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.747+0000 7fdc9cff9700 10 client.31360 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:08.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.749+0000 7fdc9cff9700 1 -- 172.21.15.12:0/3908056100 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55c33048cec0 con 0x55c3305c70e0 2024-05-09T00:09:08.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.749+0000 7fdc9cff9700 10 client.31360 handle_command_reply: tid=0 2024-05-09T00:09:08.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.749+0000 7fdcc7946b80 1 client.31360 shutdown 2024-05-09T00:09:08.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3908056100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 msgr2=0x55c330564c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330564c20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b5c0 tx=0x7fdcb00050f0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3908056100 shutdown_connections 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c33054a720 0x55c3305656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c330544890 0x55c330564c20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c3305c70e0 0x55c3305c9520 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 1 --2- 172.21.15.12:0/3908056100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c330549e00 0x55c330565160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdc9cff9700 0 client.31360 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3908056100 >> 172.21.15.12:0/3908056100 conn(0x55c3304a2210 msgr2=0x55c33054b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3908056100 shutdown_connections 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 1 -- 172.21.15.12:0/3908056100 wait complete. 2024-05-09T00:09:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.750+0000 7fdcc7946b80 20 client.31360 trim_cache size 0 max 16384 2024-05-09T00:09:08.748 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:09:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (208 inodes in the stack)", 2024-05-09T00:09:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:09:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "9b1ec53d-22c8-4857-8d0a-fbea50a81826": { 2024-05-09T00:09:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:09:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "9b1ec53d-22c8-4857-8d0a-fbea50a81826", 2024-05-09T00:09:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:09:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:09:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:09:08.748 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:09:08.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 -- 172.21.15.12:0/684132573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdca00a14a0 msgr2=0x7fdca00a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdca00a14a0 0x7fdca00a3960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009f60 tx=0x7fdcac00d040 comp rx=0 tx=0).stop 2024-05-09T00:09:08.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 -- 172.21.15.12:0/684132573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb4106280 msgr2=0x7fdcb40737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:08.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb4106280 0x7fdcb40737e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdca400b500 tx=0x7fdca400b810 comp rx=0 tx=0).stop 2024-05-09T00:09:08.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 -- 172.21.15.12:0/684132573 shutdown_connections 2024-05-09T00:09:08.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdca00a14a0 0x7fdca00a3960 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcb4074f10 0x7fdcb4073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcb4101160 0x7fdcb40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 --2- 172.21.15.12:0/684132573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb4106280 0x7fdcb40737e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:08.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 -- 172.21.15.12:0/684132573 >> 172.21.15.12:0/684132573 conn(0x7fdcb40fc9e0 msgr2=0x7fdcb40fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:08.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 -- 172.21.15.12:0/684132573 shutdown_connections 2024-05-09T00:09:08.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:08.751+0000 7fdcbb78c700 1 -- 172.21.15.12:0/684132573 wait complete. 2024-05-09T00:09:08.767 INFO:tasks.cephfs.filesystem:scrub status for tag:9b1ec53d-22c8-4857-8d0a-fbea50a81826 - {'path': '/', 'tag': '9b1ec53d-22c8-4857-8d0a-fbea50a81826', 'options': 'recursive,force'} 2024-05-09T00:09:09.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:08 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:09:09.169 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:08 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:09.169 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:08 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:09.169 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:08 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:09.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:08 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:09:09.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:08 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:09.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:08 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:09.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:08 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:09.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:08 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:09:09.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:08 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:09.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:08 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:09.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:08 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:09.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 322109 29.91 MB/sec execute 424 sec latency 182.712 ms 2024-05-09T00:09:10.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:09 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:10.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:09 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:10.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:09 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:10.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:09 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:10.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:09 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:10.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:09 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:09:10.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:09 smithi012 ceph-mon[42503]: pgmap v732: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.3 MiB/s wr, 732 op/s 2024-05-09T00:09:10.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:09 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:09 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:09 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:09 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:09 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:09 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:09:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:09 smithi204 ceph-mon[47373]: pgmap v732: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.3 MiB/s wr, 732 op/s 2024-05-09T00:09:10.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:09 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:10.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:09 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:10.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:09 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:10.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:09 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:10.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:09 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:10.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:09 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:09:10.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:09 smithi160 ceph-mon[46152]: pgmap v732: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.3 MiB/s wr, 732 op/s 2024-05-09T00:09:10.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 322381 29.87 MB/sec execute 425 sec latency 156.675 ms 2024-05-09T00:09:11.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 322652 29.82 MB/sec execute 426 sec latency 407.624 ms 2024-05-09T00:09:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.812+0000 7f50eb015700 1 -- 172.21.15.12:0/2413264449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e4106230 msgr2=0x7f50e410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.812+0000 7f50eb015700 1 --2- 172.21.15.12:0/2413264449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e4106230 0x7f50e410ddb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00b0a0 tx=0x7f50dc00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.812+0000 7f50eb015700 1 -- 172.21.15.12:0/2413264449 shutdown_connections 2024-05-09T00:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.812+0000 7f50eb015700 1 --2- 172.21.15.12:0/2413264449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e4106230 0x7f50e410ddb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.812+0000 7f50eb015700 1 --2- 172.21.15.12:0/2413264449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50e4101110 0x7f50e41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.812+0000 7f50eb015700 1 --2- 172.21.15.12:0/2413264449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50e4101ac0 0x7f50e4105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.812+0000 7f50eb015700 1 -- 172.21.15.12:0/2413264449 >> 172.21.15.12:0/2413264449 conn(0x7f50e40fc990 msgr2=0x7f50e40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.813+0000 7f50eb015700 1 -- 172.21.15.12:0/2413264449 shutdown_connections 2024-05-09T00:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.813+0000 7f50eb015700 1 -- 172.21.15.12:0/2413264449 wait complete. 2024-05-09T00:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.813+0000 7f50eb015700 1 Processor -- start 2024-05-09T00:09:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.813+0000 7f50eb015700 1 -- start start 2024-05-09T00:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50eb015700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50e4101110 0x7f50e40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50eb015700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50e4106230 0x7f50e40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50eb015700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e4073d20 0x7f50e41a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50eb015700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e41a5c10 con 0x7f50e4106230 2024-05-09T00:09:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50eb015700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e41a5d80 con 0x7f50e4101110 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50eb015700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e41a5f40 con 0x7f50e4073d20 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e8db1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50e4101110 0x7f50e40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50e4106230 0x7f50e40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e8db1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50e4101110 0x7f50e40732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47266/0 (socket says 172.21.15.12:47266) 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e8db1700 1 -- 172.21.15.12:0/2661219895 learned_addr learned my addr 172.21.15.12:0/2661219895 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e3fff700 1 -- 172.21.15.12:0/2661219895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e4073d20 msgr2=0x7f50e41a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e3fff700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e4073d20 0x7f50e41a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e3fff700 1 -- 172.21.15.12:0/2661219895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50e4101110 msgr2=0x7f50e40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e3fff700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50e4101110 0x7f50e40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e3fff700 1 -- 172.21.15.12:0/2661219895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc009d00 con 0x7f50e4106230 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e8db1700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50e4101110 0x7f50e40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.814+0000 7f50e3fff700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50e4106230 0x7f50e40737e0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f50e406a540 tx=0x7f50d000b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.815+0000 7f50e1ffb700 1 -- 172.21.15.12:0/2661219895 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0011830 con 0x7f50e4106230 2024-05-09T00:09:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.815+0000 7f50e1ffb700 1 -- 172.21.15.12:0/2661219895 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d0011e50 con 0x7f50e4106230 2024-05-09T00:09:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.815+0000 7f50e1ffb700 1 -- 172.21.15.12:0/2661219895 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0010de0 con 0x7f50e4106230 2024-05-09T00:09:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.815+0000 7f50eb015700 1 -- 172.21.15.12:0/2661219895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e41a6220 con 0x7f50e4106230 2024-05-09T00:09:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.815+0000 7f50eb015700 1 -- 172.21.15.12:0/2661219895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e41a66e0 con 0x7f50e4106230 2024-05-09T00:09:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.819+0000 7f50f71cfb80 1 Processor -- start 2024-05-09T00:09:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.820+0000 7f50e1ffb700 1 -- 172.21.15.12:0/2661219895 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f50d0010480 con 0x7f50e4106230 2024-05-09T00:09:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.820+0000 7f50e1ffb700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50d40a1390 0x7f50d40a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.820+0000 7f50e1ffb700 1 -- 172.21.15.12:0/2661219895 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f50d00cc5d0 con 0x7f50e4106230 2024-05-09T00:09:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50f71cfb80 1 -- start start 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50e95b2700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50d40a1390 0x7f50d40a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b620950 0x55bf0b620d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b61b480 0x55bf0b61b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b5720f0 con 0x55bf0b61b480 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b56fab0 con 0x55bf0b621270 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b56b3a0 con 0x55bf0b620950 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50e3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50e3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47282/0 (socket says 172.21.15.12:47282) 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50e3fff700 1 -- 172.21.15.12:0/1597534982 learned_addr learned my addr 172.21.15.12:0/1597534982 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.821+0000 7f50e95b2700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50d40a1390 0x7f50d40a3850 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00b830 tx=0x7f50dc009200 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.822+0000 7f50e95b2700 1 --2- 172.21.15.12:0/1597534982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b61b480 0x55bf0b61b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.822+0000 7f50e3fff700 1 -- 172.21.15.12:0/1597534982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b620950 msgr2=0x55bf0b620d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.822+0000 7f50e3fff700 1 --2- 172.21.15.12:0/1597534982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b620950 0x55bf0b620d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.822+0000 7f50e3fff700 1 -- 172.21.15.12:0/1597534982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b61b480 msgr2=0x55bf0b61b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.822+0000 7f50e3fff700 1 --2- 172.21.15.12:0/1597534982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b61b480 0x55bf0b61b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.822+0000 7f50e3fff700 1 -- 172.21.15.12:0/1597534982 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf0b3ec350 con 0x55bf0b621270 2024-05-09T00:09:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.822+0000 7f50e3fff700 1 --2- 172.21.15.12:0/1597534982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61af40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f50d0095c80 tx=0x7f50d00995a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=411f3a2acdcbd92e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.822+0000 7f50ceffd700 1 -- 172.21.15.12:0/1597534982 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0099e80 con 0x55bf0b621270 2024-05-09T00:09:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.822+0000 7f50ceffd700 1 -- 172.21.15.12:0/1597534982 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d0099780 con 0x55bf0b621270 2024-05-09T00:09:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.822+0000 7f50ceffd700 1 -- 172.21.15.12:0/1597534982 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0099c40 con 0x55bf0b621270 2024-05-09T00:09:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.823+0000 7f50f71cfb80 1 -- 172.21.15.12:0/1597534982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 msgr2=0x55bf0b61af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.823+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/1597534982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61af40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f50d0095c80 tx=0x7f50d00995a0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.823+0000 7f50f71cfb80 1 -- 172.21.15.12:0/1597534982 shutdown_connections 2024-05-09T00:09:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.823+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/1597534982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b61b480 0x55bf0b61b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.823+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/1597534982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b620950 0x55bf0b620d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.823+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/1597534982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61af40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.823+0000 7f50f71cfb80 1 -- 172.21.15.12:0/1597534982 >> 172.21.15.12:0/1597534982 conn(0x55bf0b5791b0 msgr2=0x55bf0b615840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.824+0000 7f50f71cfb80 1 -- 172.21.15.12:0/1597534982 shutdown_connections 2024-05-09T00:09:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.824+0000 7f50f71cfb80 1 -- 172.21.15.12:0/1597534982 wait complete. 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.824+0000 7f50f71cfb80 1 -- start start 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.824+0000 7f50f71cfb80 1 Processor -- start 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b624550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b624a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 0x55bf0b624fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b507500 con 0x55bf0b620950 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b5071a0 con 0x55bf0b61b480 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b3ee640 con 0x55bf0b621270 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50f71cfb80 10 client.0 _ll_register_callbacks cb 0x55bf0b510ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e8db1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b624550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e95b2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 0x55bf0b624fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b624a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e95b2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 0x55bf0b624fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41002/0 (socket says 172.21.15.12:41002) 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e8db1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b624550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47284/0 (socket says 172.21.15.12:47284) 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e95b2700 1 -- 172.21.15.12:0/3935985468 learned_addr learned my addr 172.21.15.12:0/3935985468 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e3fff700 1 -- 172.21.15.12:0/3935985468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 msgr2=0x55bf0b624fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e3fff700 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 0x55bf0b624fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e3fff700 1 -- 172.21.15.12:0/3935985468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 msgr2=0x55bf0b624550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e3fff700 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b624550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.825+0000 7f50e3fff700 1 -- 172.21.15.12:0/3935985468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e41a66e0 con 0x55bf0b620950 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.826+0000 7f50e3fff700 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b624a90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f50d00cb000 tx=0x7f50d0098ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842daba6940c24c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.826+0000 7f50c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.826+0000 7f50c3fff700 1 -- 172.21.15.12:0/3935985468 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0099e80 con 0x55bf0b620950 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.826+0000 7f50c3fff700 1 -- 172.21.15.12:0/3935985468 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f50d00a1c00 con 0x55bf0b620950 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.826+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3935985468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf0b4c4210 con 0x55bf0b620950 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.826+0000 7f50c3fff700 1 -- 172.21.15.12:0/3935985468 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50d009d5b0 con 0x55bf0b620950 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.826+0000 7f50f71cfb80 10 client.21666 fetch_fsmap learned FSMap version 79 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.826+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3935985468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x55bf0b398b00 con 0x55bf0b620950 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.826+0000 7f50c3fff700 1 -- 172.21.15.12:0/3935985468 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bf0b4c4210 con 0x55bf0b620950 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.827+0000 7f50c3fff700 1 -- 172.21.15.12:0/3935985468 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 79) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f50d00a1770 con 0x55bf0b620950 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.827+0000 7f50f71cfb80 10 client.21666 fetch_fsmap finished waiting for FSMap version 79 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.827+0000 7f50f71cfb80 10 client.21666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.827+0000 7f50f71cfb80 20 client.21666 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.827+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf0b63d110 0x55bf0b63f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.827+0000 7f50f71cfb80 4 client.21666 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.827+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3935985468 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf0b5ea7b0 con 0x55bf0b63d110 2024-05-09T00:09:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.827+0000 7f50e8db1700 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf0b63d110 0x55bf0b63f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.831+0000 7f50c3fff700 10 client.21666 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.831+0000 7f50e8db1700 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf0b63d110 0x55bf0b63f530 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=210b2b25c44f4e88 server_cookie=78602033fc4673c in_seq=0 out_seq=0 2024-05-09T00:09:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.835+0000 7f50c3fff700 1 -- 172.21.15.12:0/3935985468 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf0b5ea7b0 con 0x55bf0b63d110 2024-05-09T00:09:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.835+0000 7f50c3fff700 10 client.21666 handle_command_reply: tid=0 2024-05-09T00:09:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.835+0000 7f50f71cfb80 1 client.21666 shutdown 2024-05-09T00:09:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.835+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3935985468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 msgr2=0x55bf0b624a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.835+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b624a90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f50d00cb000 tx=0x7f50d0098ea0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.837+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3935985468 shutdown_connections 2024-05-09T00:09:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.837+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 0x55bf0b624fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.837+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf0b63d110 0x55bf0b63f530 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.837+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b624550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.837+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3935985468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b624a90 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.837+0000 7f50c3fff700 0 client.21666 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.837+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3935985468 >> 172.21.15.12:0/3935985468 conn(0x55bf0b5791b0 msgr2=0x55bf0b61f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.837+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3935985468 shutdown_connections 2024-05-09T00:09:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.837+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3935985468 wait complete. 2024-05-09T00:09:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.840+0000 7f50f71cfb80 20 client.21666 trim_cache size 0 max 16384 2024-05-09T00:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.862+0000 7f50f71cfb80 1 Processor -- start 2024-05-09T00:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.862+0000 7f50f71cfb80 1 -- start start 2024-05-09T00:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.862+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b639540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.862+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b615910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.862+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 0x55bf0b617520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.862+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b6129b0 con 0x55bf0b620950 2024-05-09T00:09:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.862+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b617320 con 0x55bf0b61b480 2024-05-09T00:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.862+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b60dab0 con 0x55bf0b621270 2024-05-09T00:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e8db1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b639540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e8db1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b639540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47286/0 (socket says 172.21.15.12:47286) 2024-05-09T00:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e8db1700 1 -- 172.21.15.12:0/2511432118 learned_addr learned my addr 172.21.15.12:0/2511432118 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e3fff700 1 --2- 172.21.15.12:0/2511432118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b615910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e95b2700 1 --2- 172.21.15.12:0/2511432118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 0x55bf0b617520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e3fff700 1 -- 172.21.15.12:0/2511432118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 msgr2=0x55bf0b617520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e3fff700 1 --2- 172.21.15.12:0/2511432118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 0x55bf0b617520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e3fff700 1 -- 172.21.15.12:0/2511432118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 msgr2=0x55bf0b639540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e3fff700 1 --2- 172.21.15.12:0/2511432118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b639540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e3fff700 1 -- 172.21.15.12:0/2511432118 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf0b531500 con 0x55bf0b620950 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.863+0000 7f50e8db1700 1 --2- 172.21.15.12:0/2511432118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b639540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.864+0000 7f50e3fff700 1 --2- 172.21.15.12:0/2511432118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b615910 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f50d0006fd0 tx=0x7f50d00107f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2df1eaa6132e174 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.864+0000 7f50cf7fe700 1 -- 172.21.15.12:0/2511432118 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0099e80 con 0x55bf0b620950 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.864+0000 7f50cf7fe700 1 -- 172.21.15.12:0/2511432118 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d00a1c00 con 0x55bf0b620950 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.864+0000 7f50cf7fe700 1 -- 172.21.15.12:0/2511432118 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d00c9dd0 con 0x55bf0b620950 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 -- 172.21.15.12:0/2511432118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 msgr2=0x55bf0b615910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/2511432118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b615910 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f50d0006fd0 tx=0x7f50d00107f0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 -- 172.21.15.12:0/2511432118 shutdown_connections 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/2511432118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b621270 0x55bf0b617520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/2511432118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b61b480 0x55bf0b639540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/2511432118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b615910 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 -- 172.21.15.12:0/2511432118 >> 172.21.15.12:0/2511432118 conn(0x55bf0b5791b0 msgr2=0x55bf0b623de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 -- 172.21.15.12:0/2511432118 shutdown_connections 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 -- 172.21.15.12:0/2511432118 wait complete. 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 -- start start 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.865+0000 7f50f71cfb80 1 Processor -- start 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b61b480 0x55bf0b6343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b634930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50f71cfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b61ea90 con 0x55bf0b620950 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b61ec00 con 0x55bf0b621270 2024-05-09T00:09:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50f71cfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0b61edc0 con 0x55bf0b61b480 2024-05-09T00:09:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50f71cfb80 10 client.0 _ll_register_callbacks cb 0x55bf0b510ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50e8db1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b61b480 0x55bf0b6343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50e8db1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b61b480 0x55bf0b6343f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:41008/0 (socket says 172.21.15.12:41008) 2024-05-09T00:09:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50e8db1700 1 -- 172.21.15.12:0/3989544317 learned_addr learned my addr 172.21.15.12:0/3989544317 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50e95b2700 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.866+0000 7f50e3fff700 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b634930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.867+0000 7f50e8db1700 1 -- 172.21.15.12:0/3989544317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 msgr2=0x55bf0b61e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.867+0000 7f50e8db1700 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.867+0000 7f50e8db1700 1 -- 172.21.15.12:0/3989544317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 msgr2=0x55bf0b634930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.867+0000 7f50e8db1700 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b634930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.867+0000 7f50e8db1700 1 -- 172.21.15.12:0/3989544317 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d8009bb0 con 0x55bf0b61b480 2024-05-09T00:09:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.867+0000 7f50e95b2700 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.867+0000 7f50e8db1700 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b61b480 0x55bf0b6343f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55bf0b63fdd0 tx=0x7f50d80168f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd624f754d2085c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.867+0000 7f50c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:09:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.871+0000 7f50c3fff700 1 -- 172.21.15.12:0/3989544317 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d801b040 con 0x55bf0b61b480 2024-05-09T00:09:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.871+0000 7f50c3fff700 1 -- 172.21.15.12:0/3989544317 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f50d8014620 con 0x55bf0b61b480 2024-05-09T00:09:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.871+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3989544317 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50d0099780 con 0x55bf0b61b480 2024-05-09T00:09:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.871+0000 7f50c3fff700 1 -- 172.21.15.12:0/3989544317 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50d80147e0 con 0x55bf0b61b480 2024-05-09T00:09:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.877+0000 7f50f71cfb80 10 client.31372 fetch_fsmap learned FSMap version 79 2024-05-09T00:09:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.877+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3989544317 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x55bf0b398b00 con 0x55bf0b61b480 2024-05-09T00:09:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.877+0000 7f50c3fff700 1 -- 172.21.15.12:0/3989544317 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d0099780 con 0x55bf0b61b480 2024-05-09T00:09:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.877+0000 7f50c3fff700 1 -- 172.21.15.12:0/3989544317 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 79) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f50d8023ac0 con 0x55bf0b61b480 2024-05-09T00:09:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.877+0000 7f50f71cfb80 10 client.31372 fetch_fsmap finished waiting for FSMap version 79 2024-05-09T00:09:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.877+0000 7f50f71cfb80 10 client.31372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.877+0000 7f50f71cfb80 20 client.31372 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.877+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf0b693930 0x55bf0b695d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.877+0000 7f50f71cfb80 4 client.31372 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:09:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.877+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3989544317 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bf0b563e60 con 0x55bf0b693930 2024-05-09T00:09:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.881+0000 7f50e3fff700 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf0b693930 0x55bf0b695d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.881+0000 7f50e3fff700 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf0b693930 0x55bf0b695d50 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b1f434016aff23 server_cookie=7021dc36bd7b4684 in_seq=0 out_seq=0 2024-05-09T00:09:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:11.881+0000 7f50c3fff700 10 client.31372 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:12.169 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:11 smithi012 ceph-mon[42503]: pgmap v733: 97 pgs: 18 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 50 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.3 MiB/s wr, 712 op/s 2024-05-09T00:09:12.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:11 smithi204 ceph-mon[47373]: pgmap v733: 97 pgs: 18 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 50 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.3 MiB/s wr, 712 op/s 2024-05-09T00:09:12.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:11 smithi160 ceph-mon[46152]: pgmap v733: 97 pgs: 18 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 50 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.3 MiB/s wr, 712 op/s 2024-05-09T00:09:12.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 323066 29.80 MB/sec execute 427 sec latency 267.507 ms 2024-05-09T00:09:13.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 323587 29.80 MB/sec execute 428 sec latency 191.692 ms 2024-05-09T00:09:14.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:13 smithi012 ceph-mon[42503]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.1 MiB/s wr, 625 op/s 2024-05-09T00:09:14.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:13 smithi204 ceph-mon[47373]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.1 MiB/s wr, 625 op/s 2024-05-09T00:09:14.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:13 smithi160 ceph-mon[46152]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.1 MiB/s wr, 625 op/s 2024-05-09T00:09:14.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 324165 29.80 MB/sec execute 429 sec latency 109.099 ms 2024-05-09T00:09:15.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:15.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:15.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:15.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:15.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:15.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:15.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:14 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:15.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:15.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:15.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:14 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:15.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:15.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:15.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:15.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:15.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:15.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:14 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:15.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 324664 29.81 MB/sec execute 430 sec latency 886.810 ms 2024-05-09T00:09:16.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:15 smithi012 ceph-mon[42503]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.9 MiB/s wr, 537 op/s 2024-05-09T00:09:16.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:15 smithi204 ceph-mon[47373]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.9 MiB/s wr, 537 op/s 2024-05-09T00:09:16.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:15 smithi160 ceph-mon[46152]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.9 MiB/s wr, 537 op/s 2024-05-09T00:09:16.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 324973 29.78 MB/sec execute 431 sec latency 1886.849 ms 2024-05-09T00:09:17.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:17 smithi204 ceph-mon[47373]: osdmap e132: 12 total, 12 up, 12 in 2024-05-09T00:09:17.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:17 smithi160 ceph-mon[46152]: osdmap e132: 12 total, 12 up, 12 in 2024-05-09T00:09:17.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 325340 29.76 MB/sec execute 432 sec latency 2886.891 ms 2024-05-09T00:09:17.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:17 smithi012 ceph-mon[42503]: osdmap e132: 12 total, 12 up, 12 in 2024-05-09T00:09:18.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 325761 29.73 MB/sec execute 433 sec latency 3886.942 ms 2024-05-09T00:09:18.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:18 smithi012 ceph-mon[42503]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 628 op/s 2024-05-09T00:09:18.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:18 smithi204 ceph-mon[47373]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 628 op/s 2024-05-09T00:09:18.521 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:18 smithi160 ceph-mon[46152]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 628 op/s 2024-05-09T00:09:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.637+0000 7f50c3fff700 1 -- 172.21.15.12:0/3989544317 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bf0b563e60 con 0x55bf0b693930 2024-05-09T00:09:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.637+0000 7f50c3fff700 10 client.31372 handle_command_reply: tid=0 2024-05-09T00:09:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.637+0000 7f50f71cfb80 1 client.31372 shutdown 2024-05-09T00:09:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.638+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3989544317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b61b480 msgr2=0x55bf0b6343f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.638+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b61b480 0x55bf0b6343f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55bf0b63fdd0 tx=0x7f50d80168f0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.638+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3989544317 shutdown_connections 2024-05-09T00:09:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.638+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55bf0b693930 0x55bf0b695d50 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.638+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bf0b621270 0x55bf0b61e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.638+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf0b61b480 0x55bf0b6343f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.638+0000 7f50f71cfb80 1 --2- 172.21.15.12:0/3989544317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf0b620950 0x55bf0b634930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.638+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3989544317 >> 172.21.15.12:0/3989544317 conn(0x55bf0b5791b0 msgr2=0x55bf0b622cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.641+0000 7f50c3fff700 0 client.31372 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.641+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3989544317 shutdown_connections 2024-05-09T00:09:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.645+0000 7f50f71cfb80 1 -- 172.21.15.12:0/3989544317 wait complete. 2024-05-09T00:09:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.645+0000 7f50f71cfb80 20 client.31372 trim_cache size 0 max 16384 2024-05-09T00:09:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:09:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:09:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:09:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:09:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.645+0000 7f50eb015700 1 -- 172.21.15.12:0/2661219895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50d40a1390 msgr2=0x7f50d40a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.645+0000 7f50eb015700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50d40a1390 0x7f50d40a3850 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00b830 tx=0x7f50dc009200 comp rx=0 tx=0).stop 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.645+0000 7f50eb015700 1 -- 172.21.15.12:0/2661219895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50e4106230 msgr2=0x7f50e40737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.645+0000 7f50eb015700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50e4106230 0x7f50e40737e0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f50e406a540 tx=0x7f50d000b750 comp rx=0 tx=0).stop 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.646+0000 7f50eb015700 1 -- 172.21.15.12:0/2661219895 shutdown_connections 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.646+0000 7f50eb015700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f50d40a1390 0x7f50d40a3850 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.646+0000 7f50eb015700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e4073d20 0x7f50e41a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.646+0000 7f50eb015700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50e4101110 0x7f50e40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.646+0000 7f50eb015700 1 --2- 172.21.15.12:0/2661219895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50e4106230 0x7f50e40737e0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.646+0000 7f50eb015700 1 -- 172.21.15.12:0/2661219895 >> 172.21.15.12:0/2661219895 conn(0x7f50e40fc990 msgr2=0x7f50e40fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.646+0000 7f50eb015700 1 -- 172.21.15.12:0/2661219895 shutdown_connections 2024-05-09T00:09:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:18.646+0000 7f50eb015700 1 -- 172.21.15.12:0/2661219895 wait complete. 2024-05-09T00:09:19.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 326213 29.71 MB/sec execute 434 sec latency 4216.943 ms 2024-05-09T00:09:19.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:19 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:19.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:19 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:19.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:19 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:20.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 326937 29.74 MB/sec execute 435 sec latency 106.533 ms 2024-05-09T00:09:20.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:20 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:09:20.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:20 smithi012 ceph-mon[42503]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-05-09T00:09:20.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:20 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:09:20.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:20 smithi204 ceph-mon[47373]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-05-09T00:09:20.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:20 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:09:20.516 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:20 smithi160 ceph-mon[46152]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-05-09T00:09:21.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 327362 29.73 MB/sec execute 436 sec latency 298.743 ms 2024-05-09T00:09:22.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:21 smithi012 ceph-mon[42503]: pgmap v739: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 635 op/s 2024-05-09T00:09:22.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:21 smithi204 ceph-mon[47373]: pgmap v739: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 635 op/s 2024-05-09T00:09:22.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:21 smithi160 ceph-mon[46152]: pgmap v739: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 635 op/s 2024-05-09T00:09:22.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 327688 29.70 MB/sec execute 437 sec latency 405.985 ms 2024-05-09T00:09:23.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 328133 29.69 MB/sec execute 438 sec latency 114.671 ms 2024-05-09T00:09:24.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:23 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:09:24.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:23 smithi012 ceph-mon[42503]: pgmap v740: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 578 op/s 2024-05-09T00:09:24.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:23 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:09:24.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:23 smithi204 ceph-mon[47373]: pgmap v740: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 578 op/s 2024-05-09T00:09:24.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:23 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:09:24.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:23 smithi160 ceph-mon[46152]: pgmap v740: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 578 op/s 2024-05-09T00:09:24.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 328559 29.67 MB/sec execute 439 sec latency 125.045 ms 2024-05-09T00:09:25.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:09:25.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:09:25.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:09:25.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:09:25.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:09:25.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:09:25.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 329034 29.65 MB/sec execute 440 sec latency 93.484 ms 2024-05-09T00:09:26.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:25 smithi012 ceph-mon[42503]: pgmap v741: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 578 op/s 2024-05-09T00:09:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:25 smithi204 ceph-mon[47373]: pgmap v741: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 578 op/s 2024-05-09T00:09:26.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:25 smithi160 ceph-mon[46152]: pgmap v741: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 578 op/s 2024-05-09T00:09:26.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 329601 29.65 MB/sec execute 441 sec latency 206.171 ms 2024-05-09T00:09:27.295 INFO:tasks.workunit.client.0.smithi012.stdout: 10 329932 29.64 MB/sec execute 442 sec latency 476.728 ms 2024-05-09T00:09:28.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:27 smithi012 ceph-mon[42503]: pgmap v742: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 15 MiB/s wr, 690 op/s 2024-05-09T00:09:28.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:27 smithi204 ceph-mon[47373]: pgmap v742: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 15 MiB/s wr, 690 op/s 2024-05-09T00:09:28.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:27 smithi160 ceph-mon[46152]: pgmap v742: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 15 MiB/s wr, 690 op/s 2024-05-09T00:09:28.295 INFO:tasks.workunit.client.0.smithi012.stdout: 10 330421 29.64 MB/sec execute 443 sec latency 146.158 ms 2024-05-09T00:09:29.295 INFO:tasks.workunit.client.0.smithi012.stdout: 10 330885 29.62 MB/sec execute 444 sec latency 121.887 ms 2024-05-09T00:09:30.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:29 smithi012 ceph-mon[42503]: pgmap v743: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 491 op/s 2024-05-09T00:09:30.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:29 smithi204 ceph-mon[47373]: pgmap v743: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 491 op/s 2024-05-09T00:09:30.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:29 smithi160 ceph-mon[46152]: pgmap v743: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 491 op/s 2024-05-09T00:09:30.295 INFO:tasks.workunit.client.0.smithi012.stdout: 10 331401 29.62 MB/sec execute 445 sec latency 100.460 ms 2024-05-09T00:09:31.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 331897 29.61 MB/sec execute 446 sec latency 103.370 ms 2024-05-09T00:09:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.779+0000 7fb2270df700 1 -- 172.21.15.12:0/3605166171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2200ffda0 msgr2=0x7fb220100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.779+0000 7fb2270df700 1 --2- 172.21.15.12:0/3605166171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2200ffda0 0x7fb220100220 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009990 tx=0x7fb20c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.779+0000 7fb2270df700 1 -- 172.21.15.12:0/3605166171 shutdown_connections 2024-05-09T00:09:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.779+0000 7fb2270df700 1 --2- 172.21.15.12:0/3605166171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb220100760 0x7fb22010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.779+0000 7fb2270df700 1 --2- 172.21.15.12:0/3605166171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb220105e60 0x7fb220106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.779+0000 7fb2270df700 1 --2- 172.21.15.12:0/3605166171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2200ffda0 0x7fb220100220 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.779+0000 7fb2270df700 1 -- 172.21.15.12:0/3605166171 >> 172.21.15.12:0/3605166171 conn(0x7fb220076f10 msgr2=0x7fb220077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.779+0000 7fb2270df700 1 -- 172.21.15.12:0/3605166171 shutdown_connections 2024-05-09T00:09:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.780+0000 7fb2270df700 1 -- 172.21.15.12:0/3605166171 wait complete. 2024-05-09T00:09:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.780+0000 7fb2270df700 1 Processor -- start 2024-05-09T00:09:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.780+0000 7fb2270df700 1 -- start start 2024-05-09T00:09:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb2270df700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2200ffda0 0x7fb220197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb2270df700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb220100760 0x7fb2201978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb2270df700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb220105e60 0x7fb22019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb2270df700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22019c230 con 0x7fb2200ffda0 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb2270df700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22019c3a0 con 0x7fb220105e60 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb2270df700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22019c560 con 0x7fb220100760 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb224e7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2200ffda0 0x7fb220197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb224e7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2200ffda0 0x7fb220197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41436/0 (socket says 172.21.15.12:41436) 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb224e7b700 1 -- 172.21.15.12:0/2443461425 learned_addr learned my addr 172.21.15.12:0/2443461425 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb21ffff700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb220100760 0x7fb2201978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb22567c700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb220105e60 0x7fb22019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb224e7b700 1 -- 172.21.15.12:0/2443461425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb220100760 msgr2=0x7fb2201978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb224e7b700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb220100760 0x7fb2201978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb224e7b700 1 -- 172.21.15.12:0/2443461425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb220105e60 msgr2=0x7fb22019bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb224e7b700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb220105e60 0x7fb22019bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb224e7b700 1 -- 172.21.15.12:0/2443461425 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218009c40 con 0x7fb2200ffda0 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.781+0000 7fb224e7b700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2200ffda0 0x7fb220197360 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb21400eb30 tx=0x7fb21400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.782+0000 7fb21dffb700 1 -- 172.21.15.12:0/2443461425 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb214009880 con 0x7fb2200ffda0 2024-05-09T00:09:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.782+0000 7fb2270df700 1 -- 172.21.15.12:0/2443461425 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb214009520 con 0x7fb2200ffda0 2024-05-09T00:09:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.782+0000 7fb21dffb700 1 -- 172.21.15.12:0/2443461425 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2140044e0 con 0x7fb2200ffda0 2024-05-09T00:09:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.782+0000 7fb2270df700 1 -- 172.21.15.12:0/2443461425 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c0095f0 con 0x7fb2200ffda0 2024-05-09T00:09:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.782+0000 7fb21dffb700 1 -- 172.21.15.12:0/2443461425 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb214004220 con 0x7fb2200ffda0 2024-05-09T00:09:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.785+0000 7fb21dffb700 1 -- 172.21.15.12:0/2443461425 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb214003b20 con 0x7fb2200ffda0 2024-05-09T00:09:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.786+0000 7fb21dffb700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2100a1290 0x7fb2100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.786+0000 7fb21ffff700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2100a1290 0x7fb2100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.786+0000 7fb21dffb700 1 -- 172.21.15.12:0/2443461425 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb214014070 con 0x7fb2200ffda0 2024-05-09T00:09:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb233299b80 1 Processor -- start 2024-05-09T00:09:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb233299b80 1 -- start start 2024-05-09T00:09:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb21ffff700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2100a1290 0x7fb2100a3750 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0098e0 tx=0x7fb20c005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb233299b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bee7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb233299b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7e60 0x55b71beeb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb233299b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71beebe40 0x55b71beec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb233299b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71be3a0f0 con 0x55b71bee7e60 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb233299b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71be37ab0 con 0x55b71bee7540 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb233299b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71be333a0 con 0x55b71beebe40 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb21ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7e60 0x55b71beeb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.787+0000 7fb224e7b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bee7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb21ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7e60 0x55b71beeb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41450/0 (socket says 172.21.15.12:41450) 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb224e7b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bee7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58308/0 (socket says 172.21.15.12:58308) 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb21ffff700 1 -- 172.21.15.12:0/3220859295 learned_addr learned my addr 172.21.15.12:0/3220859295 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb21ffff700 1 -- 172.21.15.12:0/3220859295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71beebe40 msgr2=0x55b71beec2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb21ffff700 1 --2- 172.21.15.12:0/3220859295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71beebe40 0x55b71beec2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb21ffff700 1 -- 172.21.15.12:0/3220859295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 msgr2=0x55b71bee7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb21ffff700 1 --2- 172.21.15.12:0/3220859295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bee7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb21ffff700 1 -- 172.21.15.12:0/3220859295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b71bcb4350 con 0x55b71bee7e60 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb224e7b700 1 --2- 172.21.15.12:0/3220859295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bee7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb21ffff700 1 --2- 172.21.15.12:0/3220859295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7e60 0x55b71beeb900 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009990 tx=0x7fb20c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aec18fbd909f32d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb20affd700 1 -- 172.21.15.12:0/3220859295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c004700 con 0x55b71bee7e60 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb20affd700 1 -- 172.21.15.12:0/3220859295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb20c01e040 con 0x55b71bee7e60 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.788+0000 7fb20affd700 1 -- 172.21.15.12:0/3220859295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c00f9e0 con 0x55b71bee7e60 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.789+0000 7fb233299b80 1 -- 172.21.15.12:0/3220859295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7e60 msgr2=0x55b71beeb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.789+0000 7fb233299b80 1 --2- 172.21.15.12:0/3220859295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7e60 0x55b71beeb900 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009990 tx=0x7fb20c020000 comp rx=0 tx=0).stop 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.790+0000 7fb233299b80 1 -- 172.21.15.12:0/3220859295 shutdown_connections 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.790+0000 7fb233299b80 1 --2- 172.21.15.12:0/3220859295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71beebe40 0x55b71beec2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.790+0000 7fb233299b80 1 --2- 172.21.15.12:0/3220859295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bee7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.790+0000 7fb233299b80 1 --2- 172.21.15.12:0/3220859295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7e60 0x55b71beeb900 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.790+0000 7fb233299b80 1 -- 172.21.15.12:0/3220859295 >> 172.21.15.12:0/3220859295 conn(0x55b71be411b0 msgr2=0x55b71bedd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.790+0000 7fb233299b80 1 -- 172.21.15.12:0/3220859295 shutdown_connections 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.790+0000 7fb233299b80 1 -- 172.21.15.12:0/3220859295 wait complete. 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.790+0000 7fb233299b80 1 -- start start 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.790+0000 7fb233299b80 1 Processor -- start 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb233299b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71bee7540 0x55b71bef3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb233299b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 0x55b71bef0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb233299b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bef1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb233299b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71bdcf500 con 0x55b71beebe40 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb233299b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71bdcf1a0 con 0x55b71bee7e60 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb233299b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71bcb6640 con 0x55b71bee7540 2024-05-09T00:09:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb233299b80 10 client.0 _ll_register_callbacks cb 0x55b71bdd8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb22567c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bef1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb21ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 0x55b71bef0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb22567c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bef1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41454/0 (socket says 172.21.15.12:41454) 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb22567c700 1 -- 172.21.15.12:0/1782448739 learned_addr learned my addr 172.21.15.12:0/1782448739 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.791+0000 7fb224e7b700 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71bee7540 0x55b71bef3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb22567c700 1 -- 172.21.15.12:0/1782448739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71bee7540 msgr2=0x55b71bef3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb22567c700 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71bee7540 0x55b71bef3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb22567c700 1 -- 172.21.15.12:0/1782448739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 msgr2=0x55b71bef0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb22567c700 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 0x55b71bef0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb22567c700 1 -- 172.21.15.12:0/1782448739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c0095f0 con 0x55b71beebe40 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb22567c700 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bef1150 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb21800ee30 tx=0x7fb218011510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1ea941d17b182b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb1fffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb1fffff700 1 -- 172.21.15.12:0/1782448739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21800e040 con 0x55b71beebe40 2024-05-09T00:09:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb1fffff700 1 -- 172.21.15.12:0/1782448739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb218011aa0 con 0x55b71beebe40 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb233299b80 1 -- 172.21.15.12:0/1782448739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b71bd8c210 con 0x55b71beebe40 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb1fffff700 1 -- 172.21.15.12:0/1782448739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb218011c60 con 0x55b71beebe40 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb233299b80 10 client.21684 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.792+0000 7fb233299b80 1 -- 172.21.15.12:0/1782448739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7fb22010d4f0 con 0x55b71beebe40 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.793+0000 7fb1fffff700 1 -- 172.21.15.12:0/1782448739 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b71bd8c210 con 0x55b71beebe40 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.793+0000 7fb1fffff700 1 -- 172.21.15.12:0/1782448739 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb218015430 con 0x55b71beebe40 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.793+0000 7fb233299b80 10 client.21684 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.793+0000 7fb233299b80 10 client.21684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.793+0000 7fb233299b80 20 client.21684 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.793+0000 7fb233299b80 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b71bee5830 0x55b71bee5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.793+0000 7fb233299b80 4 client.21684 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.793+0000 7fb233299b80 1 -- 172.21.15.12:0/1782448739 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b71beb27b0 con 0x55b71bee5830 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.794+0000 7fb224e7b700 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b71bee5830 0x55b71bee5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.794+0000 7fb1fffff700 10 client.21684 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.794+0000 7fb224e7b700 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b71bee5830 0x55b71bee5c40 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ab5cb6cbfdd975 server_cookie=ee23454962aa6a27 in_seq=0 out_seq=0 2024-05-09T00:09:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb1fffff700 1 -- 172.21.15.12:0/1782448739 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b71beb27b0 con 0x55b71bee5830 2024-05-09T00:09:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb1fffff700 10 client.21684 handle_command_reply: tid=0 2024-05-09T00:09:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 client.21684 shutdown 2024-05-09T00:09:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 -- 172.21.15.12:0/1782448739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 msgr2=0x55b71bef1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bef1150 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb21800ee30 tx=0x7fb218011510 comp rx=0 tx=0).stop 2024-05-09T00:09:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 -- 172.21.15.12:0/1782448739 shutdown_connections 2024-05-09T00:09:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bef1150 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71bee7540 0x55b71bef3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b71bee5830 0x55b71bee5c40 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 --2- 172.21.15.12:0/1782448739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 0x55b71bef0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 -- 172.21.15.12:0/1782448739 >> 172.21.15.12:0/1782448739 conn(0x55b71be411b0 msgr2=0x55b71bee1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb1fffff700 0 client.21684 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 -- 172.21.15.12:0/1782448739 shutdown_connections 2024-05-09T00:09:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 1 -- 172.21.15.12:0/1782448739 wait complete. 2024-05-09T00:09:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.799+0000 7fb233299b80 20 client.21684 trim_cache size 0 max 16384 2024-05-09T00:09:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.820+0000 7fb233299b80 1 Processor -- start 2024-05-09T00:09:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.820+0000 7fb233299b80 1 -- start start 2024-05-09T00:09:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.820+0000 7fb233299b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7540 0x55b71bee5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.820+0000 7fb233299b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 0x55b71bedd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.820+0000 7fb233299b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71beebe40 0x55b71bed9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.820+0000 7fb233299b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71beeab50 con 0x55b71bee7540 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.820+0000 7fb233299b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71bef18e0 con 0x55b71bee7e60 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.820+0000 7fb233299b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71bedf290 con 0x55b71beebe40 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb224e7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7540 0x55b71bee5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb224e7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7540 0x55b71bee5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41456/0 (socket says 172.21.15.12:41456) 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb224e7b700 1 -- 172.21.15.12:0/2490755340 learned_addr learned my addr 172.21.15.12:0/2490755340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb21ffff700 1 --2- 172.21.15.12:0/2490755340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 0x55b71bedd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb22567c700 1 --2- 172.21.15.12:0/2490755340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71beebe40 0x55b71bed9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb21ffff700 1 -- 172.21.15.12:0/2490755340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71beebe40 msgr2=0x55b71bed9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb21ffff700 1 --2- 172.21.15.12:0/2490755340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71beebe40 0x55b71bed9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb21ffff700 1 -- 172.21.15.12:0/2490755340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7540 msgr2=0x55b71bee5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb21ffff700 1 --2- 172.21.15.12:0/2490755340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7540 0x55b71bee5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb21ffff700 1 -- 172.21.15.12:0/2490755340 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b71bc60b00 con 0x55b71bee7e60 2024-05-09T00:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.821+0000 7fb21ffff700 1 --2- 172.21.15.12:0/2490755340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 0x55b71bedd910 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb20c006010 tx=0x7fb20c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d074ee7b17d946de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.822+0000 7fb20b7fe700 1 -- 172.21.15.12:0/2490755340 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c041070 con 0x55b71bee7e60 2024-05-09T00:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.822+0000 7fb20b7fe700 1 -- 172.21.15.12:0/2490755340 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb20c004500 con 0x55b71bee7e60 2024-05-09T00:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.822+0000 7fb20b7fe700 1 -- 172.21.15.12:0/2490755340 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c00f6c0 con 0x55b71bee7e60 2024-05-09T00:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.823+0000 7fb233299b80 1 -- 172.21.15.12:0/2490755340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 msgr2=0x55b71bedd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.823+0000 7fb233299b80 1 --2- 172.21.15.12:0/2490755340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 0x55b71bedd910 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb20c006010 tx=0x7fb20c01a040 comp rx=0 tx=0).stop 2024-05-09T00:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.823+0000 7fb233299b80 1 -- 172.21.15.12:0/2490755340 shutdown_connections 2024-05-09T00:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.823+0000 7fb233299b80 1 --2- 172.21.15.12:0/2490755340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71beebe40 0x55b71bed9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.823+0000 7fb233299b80 1 --2- 172.21.15.12:0/2490755340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71bee7540 0x55b71bee5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.823+0000 7fb233299b80 1 --2- 172.21.15.12:0/2490755340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7e60 0x55b71bedd910 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.823+0000 7fb233299b80 1 -- 172.21.15.12:0/2490755340 >> 172.21.15.12:0/2490755340 conn(0x55b71be411b0 msgr2=0x55b71bee2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.823+0000 7fb233299b80 1 -- 172.21.15.12:0/2490755340 shutdown_connections 2024-05-09T00:09:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.826+0000 7fb233299b80 1 -- 172.21.15.12:0/2490755340 wait complete. 2024-05-09T00:09:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.827+0000 7fb233299b80 1 -- start start 2024-05-09T00:09:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.827+0000 7fb233299b80 1 Processor -- start 2024-05-09T00:09:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb233299b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bf026d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb233299b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71bee7e60 0x55b71bf02c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb233299b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bedc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb233299b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71bedd010 con 0x55b71beebe40 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb233299b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71bedd180 con 0x55b71bee7540 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb233299b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71bedd340 con 0x55b71bee7e60 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb233299b80 10 client.0 _ll_register_callbacks cb 0x55b71bdd8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb224e7b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bf026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb22567c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bedc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb21ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71bee7e60 0x55b71bf02c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb22567c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bedc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41462/0 (socket says 172.21.15.12:41462) 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb22567c700 1 -- 172.21.15.12:0/3556378103 learned_addr learned my addr 172.21.15.12:0/3556378103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb224e7b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bf026d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:58344/0 (socket says 172.21.15.12:58344) 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb22567c700 1 -- 172.21.15.12:0/3556378103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71bee7e60 msgr2=0x55b71bf02c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb22567c700 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71bee7e60 0x55b71bf02c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb22567c700 1 -- 172.21.15.12:0/3556378103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 msgr2=0x55b71bf026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb22567c700 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bf026d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb22567c700 1 -- 172.21.15.12:0/3556378103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b71bcb4350 con 0x55b71beebe40 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.828+0000 7fb224e7b700 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bf026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.829+0000 7fb22567c700 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bedc900 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb218002230 tx=0x7fb218011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabcc536937fb895 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.829+0000 7fb1fffff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.829+0000 7fb1fffff700 1 -- 172.21.15.12:0/3556378103 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21800e040 con 0x55b71beebe40 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.829+0000 7fb233299b80 1 -- 172.21.15.12:0/3556378103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb20c01e040 con 0x55b71beebe40 2024-05-09T00:09:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.829+0000 7fb1fffff700 1 -- 172.21.15.12:0/3556378103 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb218009990 con 0x55b71beebe40 2024-05-09T00:09:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.829+0000 7fb1fffff700 1 -- 172.21.15.12:0/3556378103 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb218015b30 con 0x55b71beebe40 2024-05-09T00:09:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb233299b80 10 client.21690 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb233299b80 1 -- 172.21.15.12:0/3556378103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55b71bdf9500 con 0x55b71beebe40 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb1fffff700 1 -- 172.21.15.12:0/3556378103 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb20c01e040 con 0x55b71beebe40 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb1fffff700 1 -- 172.21.15.12:0/3556378103 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb218005500 con 0x55b71beebe40 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb233299b80 10 client.21690 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb233299b80 10 client.21690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb233299b80 20 client.21690 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb233299b80 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b71bf530f0 0x55b71bf55510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb233299b80 4 client.21690 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb233299b80 1 -- 172.21.15.12:0/3556378103 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b71be2be60 con 0x55b71bf530f0 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.830+0000 7fb224e7b700 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b71bf530f0 0x55b71bf55510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.831+0000 7fb1fffff700 10 client.21690 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:31.831+0000 7fb224e7b700 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b71bf530f0 0x55b71bf55510 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=415f4c86a9b63c08 server_cookie=eddfffec62b9c600 in_seq=0 out_seq=0 2024-05-09T00:09:32.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:31 smithi012 ceph-mon[42503]: pgmap v744: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 481 op/s 2024-05-09T00:09:32.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:31 smithi204 ceph-mon[47373]: pgmap v744: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 481 op/s 2024-05-09T00:09:32.272 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:31 smithi160 ceph-mon[46152]: pgmap v744: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 481 op/s 2024-05-09T00:09:32.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 332150 29.58 MB/sec execute 447 sec latency 364.376 ms 2024-05-09T00:09:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb1fffff700 1 -- 172.21.15.12:0/3556378103 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b71be2be60 con 0x55b71bf530f0 2024-05-09T00:09:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb1fffff700 10 client.21690 handle_command_reply: tid=0 2024-05-09T00:09:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 client.21690 shutdown 2024-05-09T00:09:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 -- 172.21.15.12:0/3556378103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 msgr2=0x55b71bedc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bedc900 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb218002230 tx=0x7fb218011990 comp rx=0 tx=0).stop 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 -- 172.21.15.12:0/3556378103 shutdown_connections 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71beebe40 0x55b71bedc900 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b71bf530f0 0x55b71bf55510 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b71bee7540 0x55b71bf026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 --2- 172.21.15.12:0/3556378103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b71bee7e60 0x55b71bf02c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 -- 172.21.15.12:0/3556378103 >> 172.21.15.12:0/3556378103 conn(0x55b71be411b0 msgr2=0x55b71bee64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb1fffff700 0 client.21690 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 -- 172.21.15.12:0/3556378103 shutdown_connections 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 1 -- 172.21.15.12:0/3556378103 wait complete. 2024-05-09T00:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.522+0000 7fb233299b80 20 client.21690 trim_cache size 0 max 16384 2024-05-09T00:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.523+0000 7fb2270df700 1 -- 172.21.15.12:0/2443461425 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2100a1290 msgr2=0x7fb2100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.523+0000 7fb2270df700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2100a1290 0x7fb2100a3750 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0098e0 tx=0x7fb20c005960 comp rx=0 tx=0).stop 2024-05-09T00:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.523+0000 7fb2270df700 1 -- 172.21.15.12:0/2443461425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2200ffda0 msgr2=0x7fb220197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.523+0000 7fb2270df700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2200ffda0 0x7fb220197360 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb21400eb30 tx=0x7fb21400ee40 comp rx=0 tx=0).stop 2024-05-09T00:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.523+0000 7fb2270df700 1 -- 172.21.15.12:0/2443461425 shutdown_connections 2024-05-09T00:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.523+0000 7fb2270df700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2100a1290 0x7fb2100a3750 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.523+0000 7fb2270df700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb220105e60 0x7fb22019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.523+0000 7fb2270df700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2200ffda0 0x7fb220197360 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.523+0000 7fb2270df700 1 --2- 172.21.15.12:0/2443461425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb220100760 0x7fb2201978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.523+0000 7fb2270df700 1 -- 172.21.15.12:0/2443461425 >> 172.21.15.12:0/2443461425 conn(0x7fb220076f10 msgr2=0x7fb220068500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.527+0000 7fb2270df700 1 -- 172.21.15.12:0/2443461425 shutdown_connections 2024-05-09T00:09:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:32.530+0000 7fb2270df700 1 -- 172.21.15.12:0/2443461425 wait complete. 2024-05-09T00:09:33.299 INFO:tasks.workunit.client.0.smithi012.stdout: 10 332416 29.53 MB/sec execute 448 sec latency 366.695 ms 2024-05-09T00:09:34.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:33 smithi012 ceph-mon[42503]: pgmap v745: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 685 op/s 2024-05-09T00:09:34.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:33 smithi204 ceph-mon[47373]: pgmap v745: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 685 op/s 2024-05-09T00:09:34.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:33 smithi160 ceph-mon[46152]: pgmap v745: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 685 op/s 2024-05-09T00:09:34.299 INFO:tasks.workunit.client.0.smithi012.stdout: 10 332925 29.53 MB/sec execute 449 sec latency 92.303 ms 2024-05-09T00:09:35.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 333493 29.54 MB/sec execute 450 sec latency 90.587 ms 2024-05-09T00:09:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:35 smithi012 ceph-mon[42503]: pgmap v746: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 9.0 MiB/s wr, 492 op/s 2024-05-09T00:09:36.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:35 smithi204 ceph-mon[47373]: pgmap v746: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 9.0 MiB/s wr, 492 op/s 2024-05-09T00:09:36.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:35 smithi160 ceph-mon[46152]: pgmap v746: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 9.0 MiB/s wr, 492 op/s 2024-05-09T00:09:36.299 INFO:tasks.workunit.client.0.smithi012.stdout: 10 333967 29.52 MB/sec execute 451 sec latency 113.786 ms 2024-05-09T00:09:37.299 INFO:tasks.workunit.client.0.smithi012.stdout: 10 334457 29.51 MB/sec execute 452 sec latency 192.110 ms 2024-05-09T00:09:38.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:37 smithi012 ceph-mon[42503]: pgmap v747: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 682 op/s 2024-05-09T00:09:38.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:37 smithi204 ceph-mon[47373]: pgmap v747: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 682 op/s 2024-05-09T00:09:38.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:37 smithi160 ceph-mon[46152]: pgmap v747: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 682 op/s 2024-05-09T00:09:38.299 INFO:tasks.workunit.client.0.smithi012.stdout: 10 335006 29.53 MB/sec execute 453 sec latency 114.879 ms 2024-05-09T00:09:38.768 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:09:38.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.938+0000 7f82cdb60700 1 -- 172.21.15.12:0/2356088699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c80694e0 msgr2=0x7f82c80698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:38.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.939+0000 7f82cdb60700 1 --2- 172.21.15.12:0/2356088699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c80694e0 0x7f82c80698c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f82b80099c0 tx=0x7f82b8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.939+0000 7f82cdb60700 1 -- 172.21.15.12:0/2356088699 shutdown_connections 2024-05-09T00:09:38.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.939+0000 7f82cdb60700 1 --2- 172.21.15.12:0/2356088699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82c8106290 0x7f82c810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.939+0000 7f82cdb60700 1 --2- 172.21.15.12:0/2356088699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c80694e0 0x7f82c80698c0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.939+0000 7f82cdb60700 1 --2- 172.21.15.12:0/2356088699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c8069e90 0x7f82c8105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.939+0000 7f82cdb60700 1 -- 172.21.15.12:0/2356088699 >> 172.21.15.12:0/2356088699 conn(0x7f82c8076f30 msgr2=0x7f82c8077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:38.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.939+0000 7f82cdb60700 1 -- 172.21.15.12:0/2356088699 shutdown_connections 2024-05-09T00:09:38.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.939+0000 7f82cdb60700 1 -- 172.21.15.12:0/2356088699 wait complete. 2024-05-09T00:09:38.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.940+0000 7f82cdb60700 1 Processor -- start 2024-05-09T00:09:38.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.940+0000 7f82cdb60700 1 -- start start 2024-05-09T00:09:38.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82cdb60700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80694e0 0x7f82c81973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82cdb60700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c8106290 0x7f82c8197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82cdb60700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82c819bcb0 0x7f82c819c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82cdb60700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c819c730 con 0x7f82c8106290 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82cdb60700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c819c8a0 con 0x7f82c819bcb0 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82cdb60700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c819ca60 con 0x7f82c80694e0 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80694e0 0x7f82c81973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80694e0 0x7f82c81973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54224/0 (socket says 172.21.15.12:54224) 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c77fe700 1 -- 172.21.15.12:0/2563863191 learned_addr learned my addr 172.21.15.12:0/2563863191 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c8106290 0x7f82c8197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c7fff700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82c819bcb0 0x7f82c819c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c77fe700 1 -- 172.21.15.12:0/2563863191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82c819bcb0 msgr2=0x7f82c819c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c77fe700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82c819bcb0 0x7f82c819c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c77fe700 1 -- 172.21.15.12:0/2563863191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c8106290 msgr2=0x7f82c8197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c77fe700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c8106290 0x7f82c8197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c77fe700 1 -- 172.21.15.12:0/2563863191 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b80095f0 con 0x7f82c80694e0 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.941+0000 7f82c77fe700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80694e0 0x7f82c81973d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b580 tx=0x7f82b800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:38.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.942+0000 7f82c4ff9700 1 -- 172.21.15.12:0/2563863191 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b801c070 con 0x7f82c80694e0 2024-05-09T00:09:38.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.942+0000 7f82c4ff9700 1 -- 172.21.15.12:0/2563863191 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82b800fd40 con 0x7f82c80694e0 2024-05-09T00:09:38.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.942+0000 7f82cdb60700 1 -- 172.21.15.12:0/2563863191 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c81a3f60 con 0x7f82c80694e0 2024-05-09T00:09:38.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.942+0000 7f82c4ff9700 1 -- 172.21.15.12:0/2563863191 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8022b80 con 0x7f82c80694e0 2024-05-09T00:09:38.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.942+0000 7f82cdb60700 1 -- 172.21.15.12:0/2563863191 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c81a4380 con 0x7f82c80694e0 2024-05-09T00:09:38.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.944+0000 7f82c4ff9700 1 -- 172.21.15.12:0/2563863191 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f82b8020070 con 0x7f82c80694e0 2024-05-09T00:09:38.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.945+0000 7f82c4ff9700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82b00a1310 0x7f82b00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.945+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82b00a1310 0x7f82b00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.945+0000 7f82c4ff9700 1 -- 172.21.15.12:0/2563863191 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f82b80cd9a0 con 0x7f82c80694e0 2024-05-09T00:09:38.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.946+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82b00a1310 0x7f82b00a37d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f82c8198920 tx=0x7f82bc009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.946+0000 7f82d9d1ab80 1 Processor -- start 2024-05-09T00:09:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.946+0000 7f82d9d1ab80 1 -- start start 2024-05-09T00:09:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d5e00 0x55c47f1d61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d6720 0x55c47f1d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f127150 con 0x55c47f1d6720 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f124b10 con 0x55c47f1d5e00 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f120400 con 0x55c47f1d0890 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d6720 0x55c47f1d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d6720 0x55c47f1d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58012/0 (socket says 172.21.15.12:58012) 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d5e00 0x55c47f1d61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c6ffd700 1 -- 172.21.15.12:0/1315875267 learned_addr learned my addr 172.21.15.12:0/1315875267 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c7fff700 1 --2- 172.21.15.12:0/1315875267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c77fe700 1 -- 172.21.15.12:0/1315875267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 msgr2=0x55c47f1d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c77fe700 1 --2- 172.21.15.12:0/1315875267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c77fe700 1 -- 172.21.15.12:0/1315875267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d6720 msgr2=0x55c47f1d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c77fe700 1 --2- 172.21.15.12:0/1315875267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d6720 0x55c47f1d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c77fe700 1 -- 172.21.15.12:0/1315875267 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c47efa13b0 con 0x55c47f1d5e00 2024-05-09T00:09:38.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/1315875267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d6720 0x55c47f1d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:38.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c7fff700 1 --2- 172.21.15.12:0/1315875267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:38.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.947+0000 7f82c77fe700 1 --2- 172.21.15.12:0/1315875267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d5e00 0x55c47f1d61e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f82b80cb100 tx=0x7f82b809a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce5f32cb5e863580 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:38.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.948+0000 7f82adffb700 1 -- 172.21.15.12:0/1315875267 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b809ab50 con 0x55c47f1d5e00 2024-05-09T00:09:38.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.948+0000 7f82adffb700 1 -- 172.21.15.12:0/1315875267 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82b809e980 con 0x55c47f1d5e00 2024-05-09T00:09:38.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.948+0000 7f82adffb700 1 -- 172.21.15.12:0/1315875267 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b809a280 con 0x55c47f1d5e00 2024-05-09T00:09:38.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1315875267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d5e00 msgr2=0x55c47f1d61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:38.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/1315875267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d5e00 0x55c47f1d61e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f82b80cb100 tx=0x7f82b809a690 comp rx=0 tx=0).stop 2024-05-09T00:09:38.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1315875267 shutdown_connections 2024-05-09T00:09:38.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/1315875267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/1315875267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d5e00 0x55c47f1d61e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/1315875267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d6720 0x55c47f1d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1315875267 >> 172.21.15.12:0/1315875267 conn(0x55c47f12e210 msgr2=0x55c47f1cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1315875267 shutdown_connections 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1315875267 wait complete. 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 -- start start 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.949+0000 7f82d9d1ab80 1 Processor -- start 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1da040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f0bc560 con 0x55c47f1d5e00 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f0bc200 con 0x55c47f1d6720 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47efa36a0 con 0x55c47f1d0890 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82d9d1ab80 10 client.0 _ll_register_callbacks cb 0x55c47f0c5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:38.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1da040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1da580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39280/0 (socket says 172.21.15.12:39280) 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1da040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58014/0 (socket says 172.21.15.12:58014) 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c7fff700 1 -- 172.21.15.12:0/1242962955 learned_addr learned my addr 172.21.15.12:0/1242962955 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54248/0 (socket says 172.21.15.12:54248) 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c77fe700 1 -- 172.21.15.12:0/1242962955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 msgr2=0x55c47f1da580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c77fe700 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1da580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c77fe700 1 -- 172.21.15.12:0/1242962955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 msgr2=0x55c47f1da040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c77fe700 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1da040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c77fe700 1 -- 172.21.15.12:0/1242962955 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82bc017070 con 0x55c47f1d0890 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.950+0000 7f82c7fff700 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.951+0000 7f82c77fe700 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d9b00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f82b809aae0 tx=0x7f82b809ab50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66d97bee054a4e8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.951+0000 7f82a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.951+0000 7f82a6ffd700 1 -- 172.21.15.12:0/1242962955 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80b3460 con 0x55c47f1d0890 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.951+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1242962955 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c47f079270 con 0x55c47f1d0890 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.951+0000 7f82a6ffd700 1 -- 172.21.15.12:0/1242962955 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f82b809adb0 con 0x55c47f1d0890 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.951+0000 7f82a6ffd700 1 -- 172.21.15.12:0/1242962955 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82b809afb0 con 0x55c47f1d0890 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.951+0000 7f82d9d1ab80 10 client.31399 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.951+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1242962955 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7f82c81a4380 con 0x55c47f1d0890 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.952+0000 7f82a6ffd700 1 -- 172.21.15.12:0/1242962955 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c47f079270 con 0x55c47f1d0890 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.952+0000 7f82a6ffd700 1 -- 172.21.15.12:0/1242962955 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f82b809bb60 con 0x55c47f1d0890 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.952+0000 7f82d9d1ab80 10 client.31399 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.952+0000 7f82d9d1ab80 10 client.31399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.952+0000 7f82d9d1ab80 20 client.31399 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.952+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c47f1e9b70 0x55c47f1ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.952+0000 7f82d9d1ab80 4 client.31399 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.952+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1242962955 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c47f19f810 con 0x55c47f1e9b70 2024-05-09T00:09:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.952+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c47f1e9b70 0x55c47f1ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.952+0000 7f82a6ffd700 10 client.31399 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:38.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.953+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c47f1e9b70 0x55c47f1ebf90 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e1921cb6f9cced server_cookie=8c91e38cc99f07d8 in_seq=0 out_seq=0 2024-05-09T00:09:38.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.954+0000 7f82a6ffd700 1 -- 172.21.15.12:0/1242962955 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c47f19f810 con 0x55c47f1e9b70 2024-05-09T00:09:38.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.954+0000 7f82a6ffd700 10 client.31399 handle_command_reply: tid=0 2024-05-09T00:09:38.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.954+0000 7f82d9d1ab80 1 client.31399 shutdown 2024-05-09T00:09:38.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.954+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1242962955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 msgr2=0x55c47f1d9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:38.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.954+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d9b00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f82b809aae0 tx=0x7f82b809ab50 comp rx=0 tx=0).stop 2024-05-09T00:09:38.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.955+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1242962955 shutdown_connections 2024-05-09T00:09:38.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.955+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c47f1e9b70 0x55c47f1ebf90 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.955+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.955+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d9b00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.955+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/1242962955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1da040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.955+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1242962955 >> 172.21.15.12:0/1242962955 conn(0x55c47f12e210 msgr2=0x55c47f1ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:38.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.955+0000 7f82a6ffd700 0 client.31399 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:38.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.955+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1242962955 shutdown_connections 2024-05-09T00:09:38.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.955+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/1242962955 wait complete. 2024-05-09T00:09:38.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.956+0000 7f82d9d1ab80 20 client.31399 trim_cache size 0 max 16384 2024-05-09T00:09:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.976+0000 7f82d9d1ab80 1 Processor -- start 2024-05-09T00:09:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.976+0000 7f82d9d1ab80 1 -- start start 2024-05-09T00:09:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f19f810 con 0x55c47f1d5e00 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f079270 con 0x55c47f1d6720 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f1cb8e0 con 0x55c47f1d0890 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58018/0 (socket says 172.21.15.12:58018) 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c6ffd700 1 -- 172.21.15.12:0/2864013614 learned_addr learned my addr 172.21.15.12:0/2864013614 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c7fff700 1 --2- 172.21.15.12:0/2864013614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c6ffd700 1 -- 172.21.15.12:0/2864013614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 msgr2=0x55c47f1d5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/2864013614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c6ffd700 1 -- 172.21.15.12:0/2864013614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 msgr2=0x55c47f1da3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/2864013614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1da3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c6ffd700 1 -- 172.21.15.12:0/2864013614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c47f1d0f40 con 0x55c47f1d5e00 2024-05-09T00:09:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.977+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/2864013614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1cad20 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0094f0 tx=0x7f82bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6efe05701261693 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.978+0000 7f82ae7fc700 1 -- 172.21.15.12:0/2864013614 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc022070 con 0x55c47f1d5e00 2024-05-09T00:09:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.978+0000 7f82ae7fc700 1 -- 172.21.15.12:0/2864013614 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82bc023040 con 0x55c47f1d5e00 2024-05-09T00:09:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.978+0000 7f82ae7fc700 1 -- 172.21.15.12:0/2864013614 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc02a7c0 con 0x55c47f1d5e00 2024-05-09T00:09:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2864013614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 msgr2=0x55c47f1cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/2864013614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1cad20 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0094f0 tx=0x7f82bc014000 comp rx=0 tx=0).stop 2024-05-09T00:09:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2864013614 shutdown_connections 2024-05-09T00:09:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/2864013614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1da3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/2864013614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1d5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/2864013614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1cad20 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2864013614 >> 172.21.15.12:0/2864013614 conn(0x55c47f12e210 msgr2=0x55c47f1d4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2864013614 shutdown_connections 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2864013614 wait complete. 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 -- start start 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.979+0000 7f82d9d1ab80 1 Processor -- start 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1e1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1deec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82d9d1ab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1df4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f1dfbb0 con 0x55c47f1d5e00 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f1dfd20 con 0x55c47f1d6720 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82d9d1ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47f23f060 con 0x55c47f1d0890 2024-05-09T00:09:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82d9d1ab80 10 client.0 _ll_register_callbacks cb 0x55c47f0c5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1deec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1deec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58022/0 (socket says 172.21.15.12:58022) 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1e1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54272/0 (socket says 172.21.15.12:54272) 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c6ffd700 1 -- 172.21.15.12:0/2241542563 learned_addr learned my addr 172.21.15.12:0/2241542563 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c7fff700 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1df4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c6ffd700 1 -- 172.21.15.12:0/2241542563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 msgr2=0x55c47f1e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1e1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c6ffd700 1 -- 172.21.15.12:0/2241542563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 msgr2=0x55c47f1df4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1df4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c6ffd700 1 -- 172.21.15.12:0/2241542563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82bc011b60 con 0x55c47f1d5e00 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c77fe700 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82c6ffd700 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1deec0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005d90 tx=0x7f82bc011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdb617e3a5066eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.980+0000 7f82a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82a6ffd700 1 -- 172.21.15.12:0/2241542563 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc022050 con 0x55c47f1d5e00 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82a6ffd700 1 -- 172.21.15.12:0/2241542563 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f82bc02a8d0 con 0x55c47f1d5e00 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2241542563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82b809e980 con 0x55c47f1d5e00 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82a6ffd700 1 -- 172.21.15.12:0/2241542563 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82bc005040 con 0x55c47f1d5e00 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82d9d1ab80 10 client.21708 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2241542563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55c47ef4db60 con 0x55c47f1d5e00 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82a6ffd700 1 -- 172.21.15.12:0/2241542563 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82b809e980 con 0x55c47f1d5e00 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82a6ffd700 1 -- 172.21.15.12:0/2241542563 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f82bc004ce0 con 0x55c47f1d5e00 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82d9d1ab80 10 client.21708 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82d9d1ab80 10 client.21708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82d9d1ab80 20 client.21708 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c47f2408a0 0x55c47f242cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82d9d1ab80 4 client.21708 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:09:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.981+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2241542563 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c47f118ec0 con 0x55c47f2408a0 2024-05-09T00:09:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.982+0000 7f82c77fe700 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c47f2408a0 0x55c47f242cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.982+0000 7f82c77fe700 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c47f2408a0 0x55c47f242cc0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ed39d8e12cd9318 server_cookie=c2b5c61c4ceef04e in_seq=0 out_seq=0 2024-05-09T00:09:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.982+0000 7f82a6ffd700 10 client.21708 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82a6ffd700 1 -- 172.21.15.12:0/2241542563 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c47f118ec0 con 0x55c47f2408a0 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82a6ffd700 10 client.21708 handle_command_reply: tid=0 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 client.21708 shutdown 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2241542563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 msgr2=0x55c47f1deec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1deec0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005d90 tx=0x7f82bc011ec0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2241542563 shutdown_connections 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c47f2408a0 0x55c47f242cc0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c47f1d6720 0x55c47f1df4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c47f1d0890 0x55c47f1e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 --2- 172.21.15.12:0/2241542563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c47f1d5e00 0x55c47f1deec0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82a6ffd700 0 client.21708 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2241542563 >> 172.21.15.12:0/2241542563 conn(0x55c47f12e210 msgr2=0x55c47f1d6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2241542563 shutdown_connections 2024-05-09T00:09:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 1 -- 172.21.15.12:0/2241542563 wait complete. 2024-05-09T00:09:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.984+0000 7f82d9d1ab80 20 client.21708 trim_cache size 0 max 16384 2024-05-09T00:09:38.986 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:09:38.987 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:09:38.987 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:09:38.988 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:09:38.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 -- 172.21.15.12:0/2563863191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82b00a1310 msgr2=0x7f82b00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:38.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82b00a1310 0x7f82b00a37d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f82c8198920 tx=0x7f82bc009260 comp rx=0 tx=0).stop 2024-05-09T00:09:38.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 -- 172.21.15.12:0/2563863191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80694e0 msgr2=0x7f82c81973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:38.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80694e0 0x7f82c81973d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b580 tx=0x7f82b800bfd0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 -- 172.21.15.12:0/2563863191 shutdown_connections 2024-05-09T00:09:38.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f82b00a1310 0x7f82b00a37d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82c819bcb0 0x7f82c819c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80694e0 0x7f82c81973d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 --2- 172.21.15.12:0/2563863191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82c8106290 0x7f82c8197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:38.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 -- 172.21.15.12:0/2563863191 >> 172.21.15.12:0/2563863191 conn(0x7f82c8076f30 msgr2=0x7f82c80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:38.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 -- 172.21.15.12:0/2563863191 shutdown_connections 2024-05-09T00:09:38.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:38.985+0000 7f82cdb60700 1 -- 172.21.15.12:0/2563863191 wait complete. 2024-05-09T00:09:38.996 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:09:38.996 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:09:39.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.155+0000 7f6b25bb0700 1 -- 172.21.15.12:0/645977038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b20101fb0 msgr2=0x7f6b20102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.155+0000 7f6b25bb0700 1 --2- 172.21.15.12:0/645977038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b20101fb0 0x7f6b20102390 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6b100098f0 tx=0x7f6b10009c00 comp rx=0 tx=0).stop 2024-05-09T00:09:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.156+0000 7f6b25bb0700 1 -- 172.21.15.12:0/645977038 shutdown_connections 2024-05-09T00:09:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.156+0000 7f6b25bb0700 1 --2- 172.21.15.12:0/645977038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2010b3d0 0x7f6b2010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.156+0000 7f6b25bb0700 1 --2- 172.21.15.12:0/645977038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b20101fb0 0x7f6b20102390 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.156+0000 7f6b25bb0700 1 --2- 172.21.15.12:0/645977038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b201028d0 0x7f6b2010adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.156+0000 7f6b25bb0700 1 -- 172.21.15.12:0/645977038 >> 172.21.15.12:0/645977038 conn(0x7f6b200fb810 msgr2=0x7f6b200fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.156+0000 7f6b25bb0700 1 -- 172.21.15.12:0/645977038 shutdown_connections 2024-05-09T00:09:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.156+0000 7f6b25bb0700 1 -- 172.21.15.12:0/645977038 wait complete. 2024-05-09T00:09:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.156+0000 7f6b25bb0700 1 Processor -- start 2024-05-09T00:09:39.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b25bb0700 1 -- start start 2024-05-09T00:09:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b25bb0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b20101fb0 0x7f6b201a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b25bb0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b201028d0 0x7f6b201a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b25bb0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2010b3d0 0x7f6b20109e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b25bb0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2010a510 con 0x7f6b20101fb0 2024-05-09T00:09:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b25bb0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2010a680 con 0x7f6b2010b3d0 2024-05-09T00:09:39.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b25bb0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2010a840 con 0x7f6b201028d0 2024-05-09T00:09:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b1f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b20101fb0 0x7f6b201a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b1f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b20101fb0 0x7f6b201a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58034/0 (socket says 172.21.15.12:58034) 2024-05-09T00:09:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b1f7fe700 1 -- 172.21.15.12:0/2947485737 learned_addr learned my addr 172.21.15.12:0/2947485737 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b1ffff700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2010b3d0 0x7f6b20109e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.157+0000 7f6b1effd700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b201028d0 0x7f6b201a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b1effd700 1 -- 172.21.15.12:0/2947485737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2010b3d0 msgr2=0x7f6b20109e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b1effd700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2010b3d0 0x7f6b20109e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b1effd700 1 -- 172.21.15.12:0/2947485737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b20101fb0 msgr2=0x7f6b201a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b1effd700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b20101fb0 0x7f6b201a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b1effd700 1 -- 172.21.15.12:0/2947485737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b10009520 con 0x7f6b201028d0 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b1f7fe700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b20101fb0 0x7f6b201a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b1effd700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b201028d0 0x7f6b201a3b30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e790 tx=0x7f6b1400eaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b1cff9700 1 -- 172.21.15.12:0/2947485737 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14010900 con 0x7f6b201028d0 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b1cff9700 1 -- 172.21.15.12:0/2947485737 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b14018420 con 0x7f6b201028d0 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b25bb0700 1 -- 172.21.15.12:0/2947485737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2010ab20 con 0x7f6b201028d0 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b1cff9700 1 -- 172.21.15.12:0/2947485737 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400f680 con 0x7f6b201028d0 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.158+0000 7f6b25bb0700 1 -- 172.21.15.12:0/2947485737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2006aba0 con 0x7f6b201028d0 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.160+0000 7f6b25bb0700 1 -- 172.21.15.12:0/2947485737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b20069b30 con 0x7f6b201028d0 2024-05-09T00:09:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.161+0000 7f6b1cff9700 1 -- 172.21.15.12:0/2947485737 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6b14018590 con 0x7f6b201028d0 2024-05-09T00:09:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.162+0000 7f6b1cff9700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b080a1290 0x7f6b080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.162+0000 7f6b1cff9700 1 -- 172.21.15.12:0/2947485737 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6b14014070 con 0x7f6b201028d0 2024-05-09T00:09:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.162+0000 7f6b1f7fe700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b080a1290 0x7f6b080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.163+0000 7f6b1f7fe700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b080a1290 0x7f6b080a3750 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6b100094f0 tx=0x7f6b10019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.170+0000 7f6b1cff9700 1 -- 172.21.15.12:0/2947485737 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b14052d60 con 0x7f6b201028d0 2024-05-09T00:09:39.299 INFO:tasks.workunit.client.0.smithi012.stdout: 10 335370 29.51 MB/sec execute 454 sec latency 116.239 ms 2024-05-09T00:09:39.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.490+0000 7f6b25bb0700 1 -- 172.21.15.12:0/2947485737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b20067380 con 0x7f6b201028d0 2024-05-09T00:09:39.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.491+0000 7f6b1cff9700 1 -- 172.21.15.12:0/2947485737 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+10876 (secure 0 0 0) 0x7f6b140960c0 con 0x7f6b201028d0 2024-05-09T00:09:39.488 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:09:39.488 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":21435,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/676326208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":676326208},{"type":"v1","addr":"172.21.15.160:6833","nonce":676326208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:09:19.023721+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 -- 172.21.15.12:0/2947485737 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b080a1290 msgr2=0x7f6b080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b080a1290 0x7f6b080a3750 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6b100094f0 tx=0x7f6b10019040 comp rx=0 tx=0).stop 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 -- 172.21.15.12:0/2947485737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b201028d0 msgr2=0x7f6b201a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b201028d0 0x7f6b201a3b30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e790 tx=0x7f6b1400eaa0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 -- 172.21.15.12:0/2947485737 shutdown_connections 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6b080a1290 0x7f6b080a3750 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2010b3d0 0x7f6b20109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b20101fb0 0x7f6b201a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 --2- 172.21.15.12:0/2947485737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b201028d0 0x7f6b201a3b30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 -- 172.21.15.12:0/2947485737 >> 172.21.15.12:0/2947485737 conn(0x7f6b200fb810 msgr2=0x7f6b20100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 -- 172.21.15.12:0/2947485737 shutdown_connections 2024-05-09T00:09:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.494+0000 7f6b25bb0700 1 -- 172.21.15.12:0/2947485737 wait complete. 2024-05-09T00:09:39.492 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2024-05-09T00:09:39.506 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:09:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.664+0000 7f266c11a700 1 -- 172.21.15.12:0/3007691447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664068c90 msgr2=0x7f2664105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.664+0000 7f266c11a700 1 --2- 172.21.15.12:0/3007691447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664068c90 0x7f2664105b60 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f26540099c0 tx=0x7f2654009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.664+0000 7f266c11a700 1 -- 172.21.15.12:0/3007691447 shutdown_connections 2024-05-09T00:09:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.664+0000 7f266c11a700 1 --2- 172.21.15.12:0/3007691447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664106290 0x7f266410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.664+0000 7f266c11a700 1 --2- 172.21.15.12:0/3007691447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26640682e0 0x7f26640686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.664+0000 7f266c11a700 1 --2- 172.21.15.12:0/3007691447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2664068c90 0x7f2664105b60 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.664+0000 7f266c11a700 1 -- 172.21.15.12:0/3007691447 >> 172.21.15.12:0/3007691447 conn(0x7f2664076f40 msgr2=0x7f2664077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.664+0000 7f266c11a700 1 -- 172.21.15.12:0/3007691447 shutdown_connections 2024-05-09T00:09:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.664+0000 7f266c11a700 1 -- 172.21.15.12:0/3007691447 wait complete. 2024-05-09T00:09:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f266c11a700 1 Processor -- start 2024-05-09T00:09:39.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f266c11a700 1 -- start start 2024-05-09T00:09:39.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f266c11a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26640682e0 0x7f26641973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f266c11a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664106290 0x7f2664197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f266c11a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266419bcc0 0x7f266419c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f266c11a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266419c740 con 0x7f26640682e0 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f266c11a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266419c8b0 con 0x7f2664106290 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f266c11a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266419ca70 con 0x7f266419bcc0 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f2669eb6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26640682e0 0x7f26641973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f2669eb6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26640682e0 0x7f26641973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58052/0 (socket says 172.21.15.12:58052) 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f2669eb6700 1 -- 172.21.15.12:0/1255082757 learned_addr learned my addr 172.21.15.12:0/1255082757 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f266a6b7700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266419bcc0 0x7f266419c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.665+0000 7f26696b5700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664106290 0x7f2664197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f266a6b7700 1 -- 172.21.15.12:0/1255082757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664106290 msgr2=0x7f2664197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f266a6b7700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664106290 0x7f2664197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f266a6b7700 1 -- 172.21.15.12:0/1255082757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26640682e0 msgr2=0x7f26641973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f266a6b7700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26640682e0 0x7f26641973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f266a6b7700 1 -- 172.21.15.12:0/1255082757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26540095f0 con 0x7f266419bcc0 2024-05-09T00:09:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f2669eb6700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26640682e0 0x7f26641973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:39.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f266a6b7700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266419bcc0 0x7f266419c140 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f265c00efc0 tx=0x7f265c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f265affd700 1 -- 172.21.15.12:0/1255082757 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c018070 con 0x7f266419bcc0 2024-05-09T00:09:39.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f265affd700 1 -- 172.21.15.12:0/1255082757 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f265c00f040 con 0x7f266419bcc0 2024-05-09T00:09:39.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f266c11a700 1 -- 172.21.15.12:0/1255082757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26641a3f70 con 0x7f266419bcc0 2024-05-09T00:09:39.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f265affd700 1 -- 172.21.15.12:0/1255082757 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c00cd40 con 0x7f266419bcc0 2024-05-09T00:09:39.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.666+0000 7f266c11a700 1 -- 172.21.15.12:0/1255082757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26641a43f0 con 0x7f266419bcc0 2024-05-09T00:09:39.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.668+0000 7f265affd700 1 -- 172.21.15.12:0/1255082757 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f265c003a00 con 0x7f266419bcc0 2024-05-09T00:09:39.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.669+0000 7f265affd700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26500a1290 0x7f26500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.669+0000 7f265affd700 1 -- 172.21.15.12:0/1255082757 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f265c0cc760 con 0x7f266419bcc0 2024-05-09T00:09:39.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.669+0000 7f2669eb6700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26500a1290 0x7f26500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.670+0000 7f2669eb6700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26500a1290 0x7f26500a3750 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2660011550 tx=0x7f2660009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.671+0000 7f26782d4b80 1 Processor -- start 2024-05-09T00:09:39.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f26782d4b80 1 -- start start 2024-05-09T00:09:39.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 0x561fae2061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 0x561fae200350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae200cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f26782d4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae157150 con 0x561fae200890 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f26782d4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae154b10 con 0x561fae205e00 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f26782d4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae150400 con 0x561fae206720 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f2669eb6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 0x561fae2061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f26696b5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 0x561fae200350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f2669eb6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 0x561fae2061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39344/0 (socket says 172.21.15.12:39344) 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.672+0000 7f2669eb6700 1 -- 172.21.15.12:0/297917255 learned_addr learned my addr 172.21.15.12:0/297917255 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f266a6b7700 1 --2- 172.21.15.12:0/297917255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae200cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f26696b5700 1 -- 172.21.15.12:0/297917255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 msgr2=0x561fae2061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f26696b5700 1 --2- 172.21.15.12:0/297917255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 0x561fae2061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f26696b5700 1 -- 172.21.15.12:0/297917255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 msgr2=0x561fae200cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f26696b5700 1 --2- 172.21.15.12:0/297917255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae200cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f26696b5700 1 -- 172.21.15.12:0/297917255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fadfd13b0 con 0x561fae206720 2024-05-09T00:09:39.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f266a6b7700 1 --2- 172.21.15.12:0/297917255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae200cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:09:39.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f26696b5700 1 --2- 172.21.15.12:0/297917255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 0x561fae200350 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f265400b580 tx=0x7f2654005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76cfb8e123a85627 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f264ffff700 1 -- 172.21.15.12:0/297917255 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265401c050 con 0x561fae206720 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f264ffff700 1 -- 172.21.15.12:0/297917255 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f265401d040 con 0x561fae206720 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.673+0000 7f264ffff700 1 -- 172.21.15.12:0/297917255 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540038e0 con 0x561fae206720 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.674+0000 7f26782d4b80 1 -- 172.21.15.12:0/297917255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 msgr2=0x561fae200350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.674+0000 7f26782d4b80 1 --2- 172.21.15.12:0/297917255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 0x561fae200350 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f265400b580 tx=0x7f2654005fb0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.674+0000 7f26782d4b80 1 -- 172.21.15.12:0/297917255 shutdown_connections 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.674+0000 7f26782d4b80 1 --2- 172.21.15.12:0/297917255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae200cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.674+0000 7f26782d4b80 1 --2- 172.21.15.12:0/297917255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 0x561fae2061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.674+0000 7f26782d4b80 1 --2- 172.21.15.12:0/297917255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 0x561fae200350 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.674+0000 7f26782d4b80 1 -- 172.21.15.12:0/297917255 >> 172.21.15.12:0/297917255 conn(0x561fae15e210 msgr2=0x561fae1fadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.674+0000 7f26782d4b80 1 -- 172.21.15.12:0/297917255 shutdown_connections 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.674+0000 7f26782d4b80 1 -- 172.21.15.12:0/297917255 wait complete. 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.675+0000 7f26782d4b80 1 -- start start 2024-05-09T00:09:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.675+0000 7f26782d4b80 1 Processor -- start 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.675+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 0x561fae209ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.675+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae209ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.675+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae206720 0x561fae20a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.675+0000 7f26782d4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae0ec560 con 0x561fae205e00 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f26782d4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae0ec200 con 0x561fae206720 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f26782d4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fadfd36a0 con 0x561fae200890 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f26782d4b80 10 client.0 _ll_register_callbacks cb 0x561fae0f5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f26696b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae209ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f2669eb6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 0x561fae209ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f266a6b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae206720 0x561fae20a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f26696b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae209ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58058/0 (socket says 172.21.15.12:58058) 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f26696b5700 1 -- 172.21.15.12:0/1350478786 learned_addr learned my addr 172.21.15.12:0/1350478786 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f2669eb6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 0x561fae209ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54328/0 (socket says 172.21.15.12:54328) 2024-05-09T00:09:39.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f2669eb6700 1 -- 172.21.15.12:0/1350478786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae206720 msgr2=0x561fae20a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f2669eb6700 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae206720 0x561fae20a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f2669eb6700 1 -- 172.21.15.12:0/1350478786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 msgr2=0x561fae209ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f2669eb6700 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae209ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f2669eb6700 1 -- 172.21.15.12:0/1350478786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266410e650 con 0x561fae200890 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f26696b5700 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae209ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f2669eb6700 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 0x561fae209ab0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2660009fd0 tx=0x7f26600047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e172d508ee4f387 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f264d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.676+0000 7f264d7fa700 1 -- 172.21.15.12:0/1350478786 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660022070 con 0x561fae200890 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f26782d4b80 1 -- 172.21.15.12:0/1350478786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fae0a9270 con 0x561fae200890 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f264d7fa700 1 -- 172.21.15.12:0/1350478786 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f266001dc60 con 0x561fae200890 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f264d7fa700 1 -- 172.21.15.12:0/1350478786 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f266000ee60 con 0x561fae200890 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f26782d4b80 10 client.31423 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f26782d4b80 1 -- 172.21.15.12:0/1350478786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x561fadf7db60 con 0x561fae200890 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f264d7fa700 1 -- 172.21.15.12:0/1350478786 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2660033020 con 0x561fae200890 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f264d7fa700 1 -- 172.21.15.12:0/1350478786 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f266001c400 con 0x561fae200890 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f26782d4b80 10 client.31423 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f26782d4b80 10 client.31423 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f26782d4b80 20 client.31423 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561fae21e5b0 0x561fae2209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f26782d4b80 4 client.31423 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f26782d4b80 1 -- 172.21.15.12:0/1350478786 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fae1cf810 con 0x561fae21e5b0 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.677+0000 7f26696b5700 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561fae21e5b0 0x561fae2209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.678+0000 7f264d7fa700 10 client.31423 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.678+0000 7f26696b5700 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561fae21e5b0 0x561fae2209d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68a160785304fe41 server_cookie=525c86e1a7c5bcc3 in_seq=0 out_seq=0 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f264d7fa700 1 -- 172.21.15.12:0/1350478786 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561fae1cf810 con 0x561fae21e5b0 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f264d7fa700 10 client.31423 handle_command_reply: tid=0 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 client.31423 shutdown 2024-05-09T00:09:39.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 -- 172.21.15.12:0/1350478786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 msgr2=0x561fae209ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 0x561fae209ab0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2660009fd0 tx=0x7f26600047c0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 -- 172.21.15.12:0/1350478786 shutdown_connections 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae206720 0x561fae20a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561fae21e5b0 0x561fae2209d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 0x561fae209ab0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1350478786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae209ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f264d7fa700 0 client.31423 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 -- 172.21.15.12:0/1350478786 >> 172.21.15.12:0/1350478786 conn(0x561fae15e210 msgr2=0x561fae1fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 -- 172.21.15.12:0/1350478786 shutdown_connections 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 1 -- 172.21.15.12:0/1350478786 wait complete. 2024-05-09T00:09:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.679+0000 7f26782d4b80 20 client.31423 trim_cache size 0 max 16384 2024-05-09T00:09:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26782d4b80 1 Processor -- start 2024-05-09T00:09:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26782d4b80 1 -- start start 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 0x561fae219fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae1fad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae206720 0x561fae204ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26782d4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae1f7dc0 con 0x561fae205e00 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26782d4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae225be0 con 0x561fae206720 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26782d4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae1f5800 con 0x561fae200890 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26696b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae1fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26696b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae1fad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58066/0 (socket says 172.21.15.12:58066) 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f26696b5700 1 -- 172.21.15.12:0/4083287227 learned_addr learned my addr 172.21.15.12:0/4083287227 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.694+0000 7f266a6b7700 1 --2- 172.21.15.12:0/4083287227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae206720 0x561fae204ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.695+0000 7f26696b5700 1 -- 172.21.15.12:0/4083287227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 msgr2=0x561fae219fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.695+0000 7f26696b5700 1 --2- 172.21.15.12:0/4083287227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 0x561fae219fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.695+0000 7f26696b5700 1 -- 172.21.15.12:0/4083287227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae206720 msgr2=0x561fae204ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.695+0000 7f26696b5700 1 --2- 172.21.15.12:0/4083287227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae206720 0x561fae204ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.695+0000 7f26696b5700 1 -- 172.21.15.12:0/4083287227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fae116560 con 0x561fae205e00 2024-05-09T00:09:39.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.695+0000 7f26696b5700 1 --2- 172.21.15.12:0/4083287227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae1fad20 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f2654005b40 tx=0x7f2654005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781e2dbbaa222fcd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.695+0000 7f2658ff9700 1 -- 172.21.15.12:0/4083287227 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654030070 con 0x561fae205e00 2024-05-09T00:09:39.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.695+0000 7f2658ff9700 1 -- 172.21.15.12:0/4083287227 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2654031040 con 0x561fae205e00 2024-05-09T00:09:39.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.695+0000 7f2658ff9700 1 -- 172.21.15.12:0/4083287227 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265401c040 con 0x561fae205e00 2024-05-09T00:09:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 -- 172.21.15.12:0/4083287227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 msgr2=0x561fae1fad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 --2- 172.21.15.12:0/4083287227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae1fad20 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f2654005b40 tx=0x7f2654005410 comp rx=0 tx=0).stop 2024-05-09T00:09:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 -- 172.21.15.12:0/4083287227 shutdown_connections 2024-05-09T00:09:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 --2- 172.21.15.12:0/4083287227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae206720 0x561fae204ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 --2- 172.21.15.12:0/4083287227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae200890 0x561fae219fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 --2- 172.21.15.12:0/4083287227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae205e00 0x561fae1fad20 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 -- 172.21.15.12:0/4083287227 >> 172.21.15.12:0/4083287227 conn(0x561fae15e210 msgr2=0x561fae20a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 -- 172.21.15.12:0/4083287227 shutdown_connections 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 -- 172.21.15.12:0/4083287227 wait complete. 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 -- start start 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.696+0000 7f26782d4b80 1 Processor -- start 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae215940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 0x561fae2114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f26782d4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 0x561fae211a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f26782d4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae212160 con 0x561fae200890 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f26782d4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae2122d0 con 0x561fae205e00 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f26782d4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fae273aa0 con 0x561fae206720 2024-05-09T00:09:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f26782d4b80 10 client.0 _ll_register_callbacks cb 0x561fae0f5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f2669eb6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae215940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f26696b5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 0x561fae2114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f2669eb6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae215940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58078/0 (socket says 172.21.15.12:58078) 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f2669eb6700 1 -- 172.21.15.12:0/1561985564 learned_addr learned my addr 172.21.15.12:0/1561985564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.697+0000 7f266a6b7700 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 0x561fae211a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f2669eb6700 1 -- 172.21.15.12:0/1561985564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 msgr2=0x561fae211a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f2669eb6700 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 0x561fae211a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f2669eb6700 1 -- 172.21.15.12:0/1561985564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 msgr2=0x561fae2114f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f2669eb6700 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 0x561fae2114f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f2669eb6700 1 -- 172.21.15.12:0/1561985564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fadfd13b0 con 0x561fae200890 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f26696b5700 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 0x561fae2114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f2669eb6700 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae215940 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2660016040 tx=0x7f266000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b5d29aaf559366 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f264d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f264d7fa700 1 -- 172.21.15.12:0/1561985564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660022070 con 0x561fae200890 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f264d7fa700 1 -- 172.21.15.12:0/1561985564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2660034080 con 0x561fae200890 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f26782d4b80 1 -- 172.21.15.12:0/1561985564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f265401d040 con 0x561fae200890 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f264d7fa700 1 -- 172.21.15.12:0/1561985564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2660005040 con 0x561fae200890 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f26782d4b80 10 client.21732 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.698+0000 7f26782d4b80 1 -- 172.21.15.12:0/1561985564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x561fadfd5cd0 con 0x561fae200890 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.699+0000 7f264d7fa700 1 -- 172.21.15.12:0/1561985564 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f265401d040 con 0x561fae200890 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.699+0000 7f264d7fa700 1 -- 172.21.15.12:0/1561985564 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2660004cb0 con 0x561fae200890 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.699+0000 7f26782d4b80 10 client.21732 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.699+0000 7f26782d4b80 10 client.21732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.699+0000 7f26782d4b80 20 client.21732 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.699+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561fae2751b0 0x561fae2775a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.699+0000 7f26782d4b80 4 client.21732 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.699+0000 7f26782d4b80 1 -- 172.21.15.12:0/1561985564 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561fae148ec0 con 0x561fae2751b0 2024-05-09T00:09:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.699+0000 7f26696b5700 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561fae2751b0 0x561fae2775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.700+0000 7f26696b5700 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561fae2751b0 0x561fae2775a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd4bfe88bb3ad007 server_cookie=35eb4581fa94582f in_seq=0 out_seq=0 2024-05-09T00:09:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.700+0000 7f264d7fa700 10 client.21732 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:09:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f264d7fa700 1 -- 172.21.15.12:0/1561985564 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561fae148ec0 con 0x561fae2751b0 2024-05-09T00:09:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f264d7fa700 10 client.21732 handle_command_reply: tid=0 2024-05-09T00:09:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 client.21732 shutdown 2024-05-09T00:09:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 -- 172.21.15.12:0/1561985564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 msgr2=0x561fae215940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae215940 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2660016040 tx=0x7f266000ee60 comp rx=0 tx=0).stop 2024-05-09T00:09:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 -- 172.21.15.12:0/1561985564 shutdown_connections 2024-05-09T00:09:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x561fae2751b0 0x561fae2775a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561fae206720 0x561fae211a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561fae200890 0x561fae215940 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 --2- 172.21.15.12:0/1561985564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561fae205e00 0x561fae2114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f264d7fa700 0 client.21732 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:09:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 -- 172.21.15.12:0/1561985564 >> 172.21.15.12:0/1561985564 conn(0x561fae15e210 msgr2=0x561fae20a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 -- 172.21.15.12:0/1561985564 shutdown_connections 2024-05-09T00:09:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.704+0000 7f26782d4b80 1 -- 172.21.15.12:0/1561985564 wait complete. 2024-05-09T00:09:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f26782d4b80 20 client.21732 trim_cache size 0 max 16384 2024-05-09T00:09:39.702 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:09:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 -- 172.21.15.12:0/1255082757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26500a1290 msgr2=0x7f26500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26500a1290 0x7f26500a3750 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2660011550 tx=0x7f2660009260 comp rx=0 tx=0).stop 2024-05-09T00:09:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 -- 172.21.15.12:0/1255082757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266419bcc0 msgr2=0x7f266419c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266419bcc0 0x7f266419c140 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f265c00efc0 tx=0x7f265c00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 -- 172.21.15.12:0/1255082757 shutdown_connections 2024-05-09T00:09:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f26500a1290 0x7f26500a3750 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266419bcc0 0x7f266419c140 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26640682e0 0x7f26641973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 --2- 172.21.15.12:0/1255082757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664106290 0x7f2664197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 -- 172.21.15.12:0/1255082757 >> 172.21.15.12:0/1255082757 conn(0x7f2664076f40 msgr2=0x7f26640fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 -- 172.21.15.12:0/1255082757 shutdown_connections 2024-05-09T00:09:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.705+0000 7f266c11a700 1 -- 172.21.15.12:0/1255082757 wait complete. 2024-05-09T00:09:39.717 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:09:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.876+0000 7f3e589bc700 1 -- 172.21.15.12:0/1586817535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e54101160 msgr2=0x7f3e54101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.876+0000 7f3e589bc700 1 --2- 172.21.15.12:0/1586817535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e54101160 0x7f3e54101540 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f3e440098f0 tx=0x7f3e44009c00 comp rx=0 tx=0).stop 2024-05-09T00:09:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.876+0000 7f3e589bc700 1 -- 172.21.15.12:0/1586817535 shutdown_connections 2024-05-09T00:09:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.876+0000 7f3e589bc700 1 --2- 172.21.15.12:0/1586817535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e54106280 0x7f3e5410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.876+0000 7f3e589bc700 1 --2- 172.21.15.12:0/1586817535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e54101160 0x7f3e54101540 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.876+0000 7f3e589bc700 1 --2- 172.21.15.12:0/1586817535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e54101b10 0x7f3e54105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.876+0000 7f3e589bc700 1 -- 172.21.15.12:0/1586817535 >> 172.21.15.12:0/1586817535 conn(0x7f3e540fc9e0 msgr2=0x7f3e540fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.877+0000 7f3e589bc700 1 -- 172.21.15.12:0/1586817535 shutdown_connections 2024-05-09T00:09:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.877+0000 7f3e589bc700 1 -- 172.21.15.12:0/1586817535 wait complete. 2024-05-09T00:09:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.877+0000 7f3e589bc700 1 Processor -- start 2024-05-09T00:09:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.877+0000 7f3e589bc700 1 -- start start 2024-05-09T00:09:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e589bc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e54101160 0x7f3e54199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e589bc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e54106280 0x7f3e54199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e589bc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e5419de70 0x7f3e5419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e589bc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5419e8f0 con 0x7f3e5419de70 2024-05-09T00:09:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e589bc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5419ea60 con 0x7f3e54106280 2024-05-09T00:09:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e589bc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5419ec20 con 0x7f3e54101160 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e5259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e54101160 0x7f3e54199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e5259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e54101160 0x7f3e54199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54360/0 (socket says 172.21.15.12:54360) 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e51d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e54106280 0x7f3e54199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e5259c700 1 -- 172.21.15.12:0/1093426004 learned_addr learned my addr 172.21.15.12:0/1093426004 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e52d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e5419de70 0x7f3e5419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e51d9b700 1 -- 172.21.15.12:0/1093426004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e54101160 msgr2=0x7f3e54199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e54101160 0x7f3e54199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.878+0000 7f3e51d9b700 1 -- 172.21.15.12:0/1093426004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e5419de70 msgr2=0x7f3e5419e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.879+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e5419de70 0x7f3e5419e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.879+0000 7f3e51d9b700 1 -- 172.21.15.12:0/1093426004 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e44009520 con 0x7f3e54106280 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.879+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e54106280 0x7f3e54199b40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3e48005fd0 tx=0x7f3e4800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.879+0000 7f3e437fe700 1 -- 172.21.15.12:0/1093426004 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48010900 con 0x7f3e54106280 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.879+0000 7f3e437fe700 1 -- 172.21.15.12:0/1093426004 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e48018420 con 0x7f3e54106280 2024-05-09T00:09:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.879+0000 7f3e437fe700 1 -- 172.21.15.12:0/1093426004 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4800f680 con 0x7f3e54106280 2024-05-09T00:09:39.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.879+0000 7f3e589bc700 1 -- 172.21.15.12:0/1093426004 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e541a6180 con 0x7f3e54106280 2024-05-09T00:09:39.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.879+0000 7f3e589bc700 1 -- 172.21.15.12:0/1093426004 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e541a6610 con 0x7f3e54106280 2024-05-09T00:09:39.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.882+0000 7f3e437fe700 1 -- 172.21.15.12:0/1093426004 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3e48018590 con 0x7f3e54106280 2024-05-09T00:09:39.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.883+0000 7f3e437fe700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3e3c0a1310 0x7f3e3c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.884+0000 7f3e437fe700 1 -- 172.21.15.12:0/1093426004 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3e48014070 con 0x7f3e54106280 2024-05-09T00:09:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.884+0000 7f3e5259c700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3e3c0a1310 0x7f3e3c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.884+0000 7f3e64b76b80 1 Processor -- start 2024-05-09T00:09:39.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.884+0000 7f3e5259c700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3e3c0a1310 0x7f3e3c0a37d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400c010 tx=0x7f3e44019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.884+0000 7f3e64b76b80 1 -- start start 2024-05-09T00:09:39.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3578e00 0x55bfd35791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3579720 0x55bfd3573350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3573cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd34ca150 con 0x55bfd3578e00 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd34c7b10 con 0x55bfd3573890 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd34c3400 con 0x55bfd3579720 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e5259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3578e00 0x55bfd35791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e51d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3579720 0x55bfd3573350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e5259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3578e00 0x55bfd35791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58108/0 (socket says 172.21.15.12:58108) 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e5259c700 1 -- 172.21.15.12:0/3365647107 learned_addr learned my addr 172.21.15.12:0/3365647107 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e52d9d700 1 --2- 172.21.15.12:0/3365647107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3573cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e51d9b700 1 -- 172.21.15.12:0/3365647107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 msgr2=0x55bfd3573cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/3365647107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3573cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e51d9b700 1 -- 172.21.15.12:0/3365647107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3578e00 msgr2=0x55bfd35791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/3365647107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3578e00 0x55bfd35791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e51d9b700 1 -- 172.21.15.12:0/3365647107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfd33443b0 con 0x55bfd3579720 2024-05-09T00:09:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e5259c700 1 --2- 172.21.15.12:0/3365647107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3578e00 0x55bfd35791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.885+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/3365647107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3579720 0x55bfd3573350 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3e480ca3d0 tx=0x7f3e480996f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aef841461d2c70e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.886+0000 7f3e40ff9700 1 -- 172.21.15.12:0/3365647107 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48099d90 con 0x55bfd3579720 2024-05-09T00:09:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.886+0000 7f3e40ff9700 1 -- 172.21.15.12:0/3365647107 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e4809dbc0 con 0x55bfd3579720 2024-05-09T00:09:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.886+0000 7f3e40ff9700 1 -- 172.21.15.12:0/3365647107 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48099280 con 0x55bfd3579720 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 -- 172.21.15.12:0/3365647107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3579720 msgr2=0x55bfd3573350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/3365647107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3579720 0x55bfd3573350 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3e480ca3d0 tx=0x7f3e480996f0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 -- 172.21.15.12:0/3365647107 shutdown_connections 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/3365647107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3573cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/3365647107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3578e00 0x55bfd35791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/3365647107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3579720 0x55bfd3573350 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 -- 172.21.15.12:0/3365647107 >> 172.21.15.12:0/3365647107 conn(0x55bfd34d1210 msgr2=0x55bfd356ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 -- 172.21.15.12:0/3365647107 shutdown_connections 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 -- 172.21.15.12:0/3365647107 wait complete. 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 -- start start 2024-05-09T00:09:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.887+0000 7f3e64b76b80 1 Processor -- start 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3573890 0x55bfd357ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd357cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3579720 0x55bfd357d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd345f560 con 0x55bfd3573890 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd345f200 con 0x55bfd3579720 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd33466a0 con 0x55bfd3578e00 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e64b76b80 10 client.0 _ll_register_callbacks cb 0x55bfd3468b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e5259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3573890 0x55bfd357ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e5259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3573890 0x55bfd357ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58112/0 (socket says 172.21.15.12:58112) 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e52d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3579720 0x55bfd357d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e5259c700 1 -- 172.21.15.12:0/741819533 learned_addr learned my addr 172.21.15.12:0/741819533 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd357cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e5259c700 1 -- 172.21.15.12:0/741819533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 msgr2=0x55bfd357cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e5259c700 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd357cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e5259c700 1 -- 172.21.15.12:0/741819533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3579720 msgr2=0x55bfd357d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e5259c700 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3579720 0x55bfd357d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e5259c700 1 -- 172.21.15.12:0/741819533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5410e670 con 0x55bfd3573890 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e52d9d700 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3579720 0x55bfd357d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.888+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd357cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.889+0000 7f3e5259c700 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3573890 0x55bfd357ca90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b780 tx=0x7f3e44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17e48c92dc34c7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.889+0000 7f3e39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.889+0000 7f3e39ffb700 1 -- 172.21.15.12:0/741819533 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4402b040 con 0x55bfd3573890 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.889+0000 7f3e39ffb700 1 -- 172.21.15.12:0/741819533 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3e44005240 con 0x55bfd3573890 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.889+0000 7f3e64b76b80 1 -- 172.21.15.12:0/741819533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfd341c270 con 0x55bfd3573890 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.889+0000 7f3e39ffb700 1 -- 172.21.15.12:0/741819533 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e4400bb70 con 0x55bfd3573890 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.889+0000 7f3e64b76b80 10 client.21744 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.889+0000 7f3e64b76b80 1 -- 172.21.15.12:0/741819533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7f3e541a6610 con 0x55bfd3573890 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.889+0000 7f3e39ffb700 1 -- 172.21.15.12:0/741819533 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bfd341c270 con 0x55bfd3573890 2024-05-09T00:09:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.889+0000 7f3e39ffb700 1 -- 172.21.15.12:0/741819533 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3e44025430 con 0x55bfd3573890 2024-05-09T00:09:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.890+0000 7f3e64b76b80 10 client.21744 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.890+0000 7f3e64b76b80 10 client.21744 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:09:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.890+0000 7f3e64b76b80 20 client.21744 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.890+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bfd35959e0 0x55bfd3597e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.890+0000 7f3e64b76b80 4 client.21744 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.890+0000 7f3e64b76b80 1 -- 172.21.15.12:0/741819533 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfd3542810 con 0x55bfd35959e0 2024-05-09T00:09:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.890+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bfd35959e0 0x55bfd3597e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.890+0000 7f3e39ffb700 10 client.21744 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:09:39.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.890+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bfd35959e0 0x55bfd3597e00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f97b2c8c257f1c1 server_cookie=72adaa618464e28f in_seq=0 out_seq=0 2024-05-09T00:09:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e39ffb700 1 -- 172.21.15.12:0/741819533 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfd3542810 con 0x55bfd35959e0 2024-05-09T00:09:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e39ffb700 10 client.21744 handle_command_reply: tid=0 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 client.21744 shutdown 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 -- 172.21.15.12:0/741819533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3573890 msgr2=0x55bfd357ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3573890 0x55bfd357ca90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b780 tx=0x7f3e44009c00 comp rx=0 tx=0).stop 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 -- 172.21.15.12:0/741819533 shutdown_connections 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bfd35959e0 0x55bfd3597e00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3579720 0x55bfd357d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e39ffb700 0 client.21744 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3573890 0x55bfd357ca90 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/741819533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd357cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 -- 172.21.15.12:0/741819533 >> 172.21.15.12:0/741819533 conn(0x55bfd34d1210 msgr2=0x55bfd356d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 -- 172.21.15.12:0/741819533 shutdown_connections 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 1 -- 172.21.15.12:0/741819533 wait complete. 2024-05-09T00:09:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.893+0000 7f3e64b76b80 20 client.21744 trim_cache size 0 max 16384 2024-05-09T00:09:39.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e64b76b80 1 Processor -- start 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e64b76b80 1 -- start start 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3573350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd3578290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 0x55bfd356dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd357ab10 con 0x55bfd3579720 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd359d010 con 0x55bfd3573890 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd356fe30 con 0x55bfd3578e00 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e52d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 0x55bfd356dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e52d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 0x55bfd356dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58124/0 (socket says 172.21.15.12:58124) 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e52d9d700 1 -- 172.21.15.12:0/1571578878 learned_addr learned my addr 172.21.15.12:0/1571578878 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/1571578878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd3578290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e5259c700 1 --2- 172.21.15.12:0/1571578878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3573350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e52d9d700 1 -- 172.21.15.12:0/1571578878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 msgr2=0x55bfd3578290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e52d9d700 1 --2- 172.21.15.12:0/1571578878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd3578290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e52d9d700 1 -- 172.21.15.12:0/1571578878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 msgr2=0x55bfd3573350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e52d9d700 1 --2- 172.21.15.12:0/1571578878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3573350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e52d9d700 1 -- 172.21.15.12:0/1571578878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfd3573f40 con 0x55bfd3579720 2024-05-09T00:09:39.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.908+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/1571578878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd3578290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:39.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.909+0000 7f3e52d9d700 1 --2- 172.21.15.12:0/1571578878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 0x55bfd356dd20 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c012c50 tx=0x7f3e4c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9beece1a487c48 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.909+0000 7f3e417fa700 1 -- 172.21.15.12:0/1571578878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00e040 con 0x55bfd3579720 2024-05-09T00:09:39.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.909+0000 7f3e417fa700 1 -- 172.21.15.12:0/1571578878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e4c00f040 con 0x55bfd3579720 2024-05-09T00:09:39.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.909+0000 7f3e417fa700 1 -- 172.21.15.12:0/1571578878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0043d0 con 0x55bfd3579720 2024-05-09T00:09:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.910+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1571578878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 msgr2=0x55bfd356dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.910+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/1571578878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 0x55bfd356dd20 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c012c50 tx=0x7f3e4c009920 comp rx=0 tx=0).stop 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.910+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1571578878 shutdown_connections 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.910+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/1571578878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 0x55bfd356dd20 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.910+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/1571578878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3573350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.910+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/1571578878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd3578290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.910+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1571578878 >> 172.21.15.12:0/1571578878 conn(0x55bfd34d1210 msgr2=0x55bfd3577b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.910+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1571578878 shutdown_connections 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.910+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1571578878 wait complete. 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e64b76b80 1 -- start start 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e64b76b80 1 Processor -- start 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3588b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd35890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e64b76b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 0x55bfd35eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd35897e0 con 0x55bfd3579720 2024-05-09T00:09:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd35eb410 con 0x55bfd3573890 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e64b76b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd35eb5a0 con 0x55bfd3578e00 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e64b76b80 10 client.0 _ll_register_callbacks cb 0x55bfd3468b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e5259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3588b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e51d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd35890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e5259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3588b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39430/0 (socket says 172.21.15.12:39430) 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e51d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd35890a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54410/0 (socket says 172.21.15.12:54410) 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e5259c700 1 -- 172.21.15.12:0/1954308034 learned_addr learned my addr 172.21.15.12:0/1954308034 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.911+0000 7f3e52d9d700 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 0x55bfd35eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e5259c700 1 -- 172.21.15.12:0/1954308034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 msgr2=0x55bfd35890a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e5259c700 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd35890a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e5259c700 1 -- 172.21.15.12:0/1954308034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 msgr2=0x55bfd35eaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e5259c700 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 0x55bfd35eaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e5259c700 1 -- 172.21.15.12:0/1954308034 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e48078e70 con 0x55bfd3573890 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e5259c700 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3588b60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b780 tx=0x7f3e44025660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e40dd9df02a578b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e39ffb700 1 -- 172.21.15.12:0/1954308034 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4402b030 con 0x55bfd3573890 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1954308034 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e4809dbc0 con 0x55bfd3573890 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e39ffb700 1 -- 172.21.15.12:0/1954308034 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3e44005090 con 0x55bfd3573890 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e39ffb700 1 -- 172.21.15.12:0/1954308034 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e4400fda0 con 0x55bfd3573890 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e64b76b80 10 client.31169 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.912+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1954308034 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7f3e4c0128d0 con 0x55bfd3573890 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.913+0000 7f3e39ffb700 1 -- 172.21.15.12:0/1954308034 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e4809dbc0 con 0x55bfd3573890 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.913+0000 7f3e39ffb700 1 -- 172.21.15.12:0/1954308034 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3e44048070 con 0x55bfd3573890 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.913+0000 7f3e64b76b80 10 client.31169 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.913+0000 7f3e64b76b80 10 client.31169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.913+0000 7f3e64b76b80 20 client.31169 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.913+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bfd35ec940 0x55bfd35eed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.913+0000 7f3e64b76b80 4 client.31169 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:09:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.913+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1954308034 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bfd34bbec0 con 0x55bfd35ec940 2024-05-09T00:09:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.913+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bfd35ec940 0x55bfd35eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.914+0000 7f3e39ffb700 10 client.31169 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:09:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.914+0000 7f3e51d9b700 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bfd35ec940 0x55bfd35eed60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=482799889f048b91 server_cookie=7d36db474415d5b in_seq=0 out_seq=0 2024-05-09T00:09:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e39ffb700 1 -- 172.21.15.12:0/1954308034 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bfd34bbec0 con 0x55bfd35ec940 2024-05-09T00:09:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e39ffb700 10 client.31169 handle_command_reply: tid=0 2024-05-09T00:09:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e64b76b80 1 client.31169 shutdown 2024-05-09T00:09:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1954308034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 msgr2=0x55bfd3588b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3588b60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b780 tx=0x7f3e44025660 comp rx=0 tx=0).stop 2024-05-09T00:09:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1954308034 shutdown_connections 2024-05-09T00:09:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55bfd35ec940 0x55bfd35eed60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfd3579720 0x55bfd35eaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfd3573890 0x55bfd3588b60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e39ffb700 0 client.31169 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:09:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e64b76b80 1 --2- 172.21.15.12:0/1954308034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfd3578e00 0x55bfd35890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1954308034 >> 172.21.15.12:0/1954308034 conn(0x55bfd34d1210 msgr2=0x55bfd357a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.916+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1954308034 shutdown_connections 2024-05-09T00:09:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e64b76b80 1 -- 172.21.15.12:0/1954308034 wait complete. 2024-05-09T00:09:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e64b76b80 20 client.31169 trim_cache size 0 max 16384 2024-05-09T00:09:39.916 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:09:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 -- 172.21.15.12:0/1093426004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3e3c0a1310 msgr2=0x7f3e3c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3e3c0a1310 0x7f3e3c0a37d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400c010 tx=0x7f3e44019040 comp rx=0 tx=0).stop 2024-05-09T00:09:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 -- 172.21.15.12:0/1093426004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e54106280 msgr2=0x7f3e54199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e54106280 0x7f3e54199b40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3e48005fd0 tx=0x7f3e4800eca0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 -- 172.21.15.12:0/1093426004 shutdown_connections 2024-05-09T00:09:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3e3c0a1310 0x7f3e3c0a37d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e5419de70 0x7f3e5419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e54101160 0x7f3e54199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 --2- 172.21.15.12:0/1093426004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e54106280 0x7f3e54199b40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 -- 172.21.15.12:0/1093426004 >> 172.21.15.12:0/1093426004 conn(0x7f3e540fc9e0 msgr2=0x7f3e540fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 -- 172.21.15.12:0/1093426004 shutdown_connections 2024-05-09T00:09:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:39.917+0000 7f3e589bc700 1 -- 172.21.15.12:0/1093426004 wait complete. 2024-05-09T00:09:39.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:39 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2947485737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:09:39.931 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:09:40.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:39 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2947485737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:09:40.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:39 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2947485737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:09:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.095+0000 7f709cd16700 1 -- 172.21.15.12:0/4221407342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70980694f0 msgr2=0x7f70980698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.095+0000 7f709cd16700 1 --2- 172.21.15.12:0/4221407342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70980694f0 0x7f70980698d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7088009990 tx=0x7f7088009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.095+0000 7f709cd16700 1 -- 172.21.15.12:0/4221407342 shutdown_connections 2024-05-09T00:09:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.095+0000 7f709cd16700 1 --2- 172.21.15.12:0/4221407342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f709810d940 0x7f709810fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.095+0000 7f709cd16700 1 --2- 172.21.15.12:0/4221407342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70980694f0 0x7f70980698d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.095+0000 7f709cd16700 1 --2- 172.21.15.12:0/4221407342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7098069e10 0x7f709810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.095+0000 7f709cd16700 1 -- 172.21.15.12:0/4221407342 >> 172.21.15.12:0/4221407342 conn(0x7f7098076f40 msgr2=0x7f7098077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.095+0000 7f709cd16700 1 -- 172.21.15.12:0/4221407342 shutdown_connections 2024-05-09T00:09:40.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.095+0000 7f709cd16700 1 -- 172.21.15.12:0/4221407342 wait complete. 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.096+0000 7f709cd16700 1 Processor -- start 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.096+0000 7f709cd16700 1 -- start start 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.096+0000 7f709cd16700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70980694f0 0x7f70981996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.096+0000 7f709cd16700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7098069e10 0x7f7098199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.096+0000 7f709cd16700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f709810d940 0x7f709819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.096+0000 7f709cd16700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709819e5e0 con 0x7f709810d940 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.096+0000 7f709cd16700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709819e750 con 0x7f70980694f0 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.096+0000 7f709cd16700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709819e910 con 0x7f7098069e10 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.096+0000 7f7096d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f709810d940 0x7f709819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f709659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70980694f0 0x7f70981996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f7095d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7098069e10 0x7f7098199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f709659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70980694f0 0x7f70981996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39444/0 (socket says 172.21.15.12:39444) 2024-05-09T00:09:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f7096d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f709810d940 0x7f709819dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58164/0 (socket says 172.21.15.12:58164) 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f709659c700 1 -- 172.21.15.12:0/1028970157 learned_addr learned my addr 172.21.15.12:0/1028970157 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f7096d9d700 1 -- 172.21.15.12:0/1028970157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7098069e10 msgr2=0x7f7098199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f7096d9d700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7098069e10 0x7f7098199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f7096d9d700 1 -- 172.21.15.12:0/1028970157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70980694f0 msgr2=0x7f70981996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f7096d9d700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70980694f0 0x7f70981996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f7096d9d700 1 -- 172.21.15.12:0/1028970157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880095f0 con 0x7f709810d940 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f7095d9b700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7098069e10 0x7f7098199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f7096d9d700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f709810d940 0x7f709819dfa0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f709000ef90 tx=0x7f709000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f70877fe700 1 -- 172.21.15.12:0/1028970157 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090018070 con 0x7f709810d940 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f70877fe700 1 -- 172.21.15.12:0/1028970157 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709000f040 con 0x7f709810d940 2024-05-09T00:09:40.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f709cd16700 1 -- 172.21.15.12:0/1028970157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709819eb90 con 0x7f709810d940 2024-05-09T00:09:40.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f70877fe700 1 -- 172.21.15.12:0/1028970157 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000cc30 con 0x7f709810d940 2024-05-09T00:09:40.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.097+0000 7f709cd16700 1 -- 172.21.15.12:0/1028970157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70981a6430 con 0x7f709810d940 2024-05-09T00:09:40.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.100+0000 7f70877fe700 1 -- 172.21.15.12:0/1028970157 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7090003a00 con 0x7f709810d940 2024-05-09T00:09:40.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.101+0000 7f70877fe700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70800a1290 0x7f70800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.101+0000 7f709659c700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70800a1290 0x7f70800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.101+0000 7f70877fe700 1 -- 172.21.15.12:0/1028970157 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f70900cc8e0 con 0x7f709810d940 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.101+0000 7f709659c700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70800a1290 0x7f70800a3750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7088006010 tx=0x7f708800b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f70a8ed0b80 1 Processor -- start 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f70a8ed0b80 1 -- start start 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bba91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba9720 0x55669bba3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba3890 0x55669bba3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669bafa150 con 0x55669bba8e00 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669baf7b10 con 0x55669bba9720 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669baf3400 con 0x55669bba3890 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f709659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bba91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f709659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bba91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58176/0 (socket says 172.21.15.12:58176) 2024-05-09T00:09:40.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f709659c700 1 -- 172.21.15.12:0/3073296178 learned_addr learned my addr 172.21.15.12:0/3073296178 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f7096d9d700 1 --2- 172.21.15.12:0/3073296178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba3890 0x55669bba3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.102+0000 7f7095d9b700 1 --2- 172.21.15.12:0/3073296178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba9720 0x55669bba3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.103+0000 7f709659c700 1 -- 172.21.15.12:0/3073296178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba3890 msgr2=0x55669bba3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.103+0000 7f709659c700 1 --2- 172.21.15.12:0/3073296178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba3890 0x55669bba3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.103+0000 7f709659c700 1 -- 172.21.15.12:0/3073296178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba9720 msgr2=0x55669bba3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.103+0000 7f709659c700 1 --2- 172.21.15.12:0/3073296178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba9720 0x55669bba3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.103+0000 7f709659c700 1 -- 172.21.15.12:0/3073296178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55669b9743b0 con 0x55669bba8e00 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.103+0000 7f709659c700 1 --2- 172.21.15.12:0/3073296178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bba91e0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f7088005810 tx=0x7f7088020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f242b8e62f0d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.103+0000 7f7084ff9700 1 -- 172.21.15.12:0/3073296178 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088004700 con 0x55669bba8e00 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.103+0000 7f7084ff9700 1 -- 172.21.15.12:0/3073296178 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f708801e040 con 0x55669bba8e00 2024-05-09T00:09:40.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.103+0000 7f7084ff9700 1 -- 172.21.15.12:0/3073296178 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708800fb00 con 0x55669bba8e00 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.104+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3073296178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 msgr2=0x55669bba91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.104+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/3073296178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bba91e0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f7088005810 tx=0x7f7088020000 comp rx=0 tx=0).stop 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.104+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3073296178 shutdown_connections 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.104+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/3073296178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba3890 0x55669bba3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.104+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/3073296178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bba91e0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.104+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/3073296178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba9720 0x55669bba3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.104+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3073296178 >> 172.21.15.12:0/3073296178 conn(0x55669bb01210 msgr2=0x55669bb9ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.104+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3073296178 shutdown_connections 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.104+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3073296178 wait complete. 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.104+0000 7f70a8ed0b80 1 -- start start 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f70a8ed0b80 1 Processor -- start 2024-05-09T00:09:40.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bbacb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bbad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba9720 0x55669bbad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669ba8f560 con 0x55669bba8e00 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669ba8f200 con 0x55669bba3890 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669b9766a0 con 0x55669bba9720 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f70a8ed0b80 10 client.0 _ll_register_callbacks cb 0x55669ba98b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f7095d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bbad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f7095d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bbad0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58190/0 (socket says 172.21.15.12:58190) 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f7095d9b700 1 -- 172.21.15.12:0/3149919661 learned_addr learned my addr 172.21.15.12:0/3149919661 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f7096d9d700 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba9720 0x55669bbad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f709659c700 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bbacb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f7095d9b700 1 -- 172.21.15.12:0/3149919661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba9720 msgr2=0x55669bbad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f7095d9b700 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba9720 0x55669bbad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f7095d9b700 1 -- 172.21.15.12:0/3149919661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 msgr2=0x55669bbacb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f7095d9b700 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bbacb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.105+0000 7f7095d9b700 1 -- 172.21.15.12:0/3149919661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7098110860 con 0x55669bba8e00 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f7095d9b700 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bbad0a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f708c011960 tx=0x7f708c011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8c563490b17d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f707dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f707dffb700 1 -- 172.21.15.12:0/3149919661 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00f780 con 0x55669bba8e00 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f707dffb700 1 -- 172.21.15.12:0/3149919661 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f708c00f8e0 con 0x55669bba8e00 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3149919661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55669ba4c270 con 0x55669bba8e00 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f707dffb700 1 -- 172.21.15.12:0/3149919661 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f708c00faa0 con 0x55669bba8e00 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f70a8ed0b80 10 client.21762 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3149919661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55669b920b60 con 0x55669bba8e00 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f707dffb700 1 -- 172.21.15.12:0/3149919661 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55669ba4c270 con 0x55669bba8e00 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f707dffb700 1 -- 172.21.15.12:0/3149919661 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f708c024070 con 0x55669bba8e00 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f70a8ed0b80 10 client.21762 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.106+0000 7f70a8ed0b80 10 client.21762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.107+0000 7f70a8ed0b80 20 client.21762 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:40.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.107+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55669bbbd2f0 0x55669bbbf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.107+0000 7f70a8ed0b80 4 client.21762 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:40.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.107+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3149919661 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55669bb72810 con 0x55669bbbd2f0 2024-05-09T00:09:40.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.107+0000 7f709659c700 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55669bbbd2f0 0x55669bbbf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.107+0000 7f709659c700 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55669bbbd2f0 0x55669bbbf710 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eed8f25e0ed2a8b server_cookie=1e9d96237d63ff1 in_seq=0 out_seq=0 2024-05-09T00:09:40.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.107+0000 7f707dffb700 10 client.21762 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f707dffb700 1 -- 172.21.15.12:0/3149919661 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55669bb72810 con 0x55669bbbd2f0 2024-05-09T00:09:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f707dffb700 10 client.21762 handle_command_reply: tid=0 2024-05-09T00:09:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 client.21762 shutdown 2024-05-09T00:09:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3149919661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 msgr2=0x55669bbad0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bbad0a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f708c011960 tx=0x7f708c011c70 comp rx=0 tx=0).stop 2024-05-09T00:09:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3149919661 shutdown_connections 2024-05-09T00:09:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba9720 0x55669bbad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55669bbbd2f0 0x55669bbbf710 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bbacb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/3149919661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba8e00 0x55669bbad0a0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f707dffb700 0 client.21762 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3149919661 >> 172.21.15.12:0/3149919661 conn(0x55669bb01210 msgr2=0x55669bb9d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3149919661 shutdown_connections 2024-05-09T00:09:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/3149919661 wait complete. 2024-05-09T00:09:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.114+0000 7f70a8ed0b80 20 client.21762 trim_cache size 0 max 16384 2024-05-09T00:09:40.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f70a8ed0b80 1 Processor -- start 2024-05-09T00:09:40.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f70a8ed0b80 1 -- start start 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bb9a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 0x55669bb9dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bbb4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669bb98890 con 0x55669bba9720 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669bb9a750 con 0x55669bba3890 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669bbba8f0 con 0x55669bba8e00 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f7096d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bbb4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f7095d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 0x55669bb9dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f7096d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bbb4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58196/0 (socket says 172.21.15.12:58196) 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f7096d9d700 1 -- 172.21.15.12:0/2301155166 learned_addr learned my addr 172.21.15.12:0/2301155166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f709659c700 1 --2- 172.21.15.12:0/2301155166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bb9a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f7096d9d700 1 -- 172.21.15.12:0/2301155166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 msgr2=0x55669bb9dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f7096d9d700 1 --2- 172.21.15.12:0/2301155166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 0x55669bb9dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f7096d9d700 1 -- 172.21.15.12:0/2301155166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 msgr2=0x55669bb9a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:40.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.130+0000 7f7096d9d700 1 --2- 172.21.15.12:0/2301155166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bb9a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.131+0000 7f7096d9d700 1 -- 172.21.15.12:0/2301155166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55669bab9560 con 0x55669bba9720 2024-05-09T00:09:40.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.131+0000 7f7095d9b700 1 --2- 172.21.15.12:0/2301155166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 0x55669bb9dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:40.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.131+0000 7f7096d9d700 1 --2- 172.21.15.12:0/2301155166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bbb4100 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f70900964a0 tx=0x7f70900993e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b3712c10c9e72e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:40.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.131+0000 7f70857fa700 1 -- 172.21.15.12:0/2301155166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090099a30 con 0x55669bba9720 2024-05-09T00:09:40.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.131+0000 7f70857fa700 1 -- 172.21.15.12:0/2301155166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709009d860 con 0x55669bba9720 2024-05-09T00:09:40.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.131+0000 7f70857fa700 1 -- 172.21.15.12:0/2301155166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090099c90 con 0x55669bba9720 2024-05-09T00:09:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2301155166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 msgr2=0x55669bbb4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/2301155166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bbb4100 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f70900964a0 tx=0x7f70900993e0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2301155166 shutdown_connections 2024-05-09T00:09:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/2301155166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bbb4100 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/2301155166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bb9a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/2301155166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 0x55669bb9dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2301155166 >> 172.21.15.12:0/2301155166 conn(0x55669bb01210 msgr2=0x55669bbac3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2301155166 shutdown_connections 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2301155166 wait complete. 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 -- start start 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 Processor -- start 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.132+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bbb00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 0x55669bbb0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f70a8ed0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bc127a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669bbb0df0 con 0x55669bba9720 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669bbb0f60 con 0x55669bba3890 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f70a8ed0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55669bc12ce0 con 0x55669bba8e00 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f70a8ed0b80 10 client.0 _ll_register_callbacks cb 0x55669ba98b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f709659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bbb00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f709659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bbb00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39464/0 (socket says 172.21.15.12:39464) 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f709659c700 1 -- 172.21.15.12:0/2822945916 learned_addr learned my addr 172.21.15.12:0/2822945916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f7095d9b700 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 0x55669bbb0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f7096d9d700 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bc127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f709659c700 1 -- 172.21.15.12:0/2822945916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 msgr2=0x55669bbb0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f709659c700 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 0x55669bbb0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f709659c700 1 -- 172.21.15.12:0/2822945916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 msgr2=0x55669bc127a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f709659c700 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bc127a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f709659c700 1 -- 172.21.15.12:0/2822945916 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55669b920b60 con 0x55669bba3890 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f7096d9d700 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bc127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f709659c700 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bbb00e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7088005f50 tx=0x7f708800f4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0a5fc4de03db30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.133+0000 7f707dffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f707dffb700 1 -- 172.21.15.12:0/2822945916 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708800b2d0 con 0x55669bba3890 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2822945916 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f708801e040 con 0x55669bba3890 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f707dffb700 1 -- 172.21.15.12:0/2822945916 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f708800f920 con 0x55669bba3890 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f707dffb700 1 -- 172.21.15.12:0/2822945916 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f708800fae0 con 0x55669bba3890 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f70a8ed0b80 10 client.31184 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2822945916 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7f70981a6430 con 0x55669bba3890 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f707dffb700 1 -- 172.21.15.12:0/2822945916 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f708801e040 con 0x55669bba3890 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f707dffb700 1 -- 172.21.15.12:0/2822945916 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7088004500 con 0x55669bba3890 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f70a8ed0b80 10 client.31184 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f70a8ed0b80 10 client.31184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f70a8ed0b80 20 client.31184 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55669bc14050 0x55669bc16470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f70a8ed0b80 4 client.31184 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.134+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2822945916 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55669baebec0 con 0x55669bc14050 2024-05-09T00:09:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.135+0000 7f7095d9b700 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55669bc14050 0x55669bc16470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:40.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.135+0000 7f707dffb700 10 client.31184 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:40.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.135+0000 7f7095d9b700 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55669bc14050 0x55669bc16470 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e164a7d1a90214b server_cookie=d423146bc59a954f in_seq=0 out_seq=0 2024-05-09T00:09:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f707dffb700 1 -- 172.21.15.12:0/2822945916 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55669baebec0 con 0x55669bc14050 2024-05-09T00:09:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f707dffb700 10 client.31184 handle_command_reply: tid=0 2024-05-09T00:09:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 client.31184 shutdown 2024-05-09T00:09:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2822945916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 msgr2=0x55669bbb00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bbb00e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7088005f50 tx=0x7f708800f4d0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2822945916 shutdown_connections 2024-05-09T00:09:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55669bba9720 0x55669bc127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55669bc14050 0x55669bc16470 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55669bba3890 0x55669bbb00e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 --2- 172.21.15.12:0/2822945916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55669bba8e00 0x55669bbb0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f707dffb700 0 client.31184 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2822945916 >> 172.21.15.12:0/2822945916 conn(0x55669bb01210 msgr2=0x55669bba84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2822945916 shutdown_connections 2024-05-09T00:09:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.137+0000 7f70a8ed0b80 1 -- 172.21.15.12:0/2822945916 wait complete. 2024-05-09T00:09:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f70a8ed0b80 20 client.31184 trim_cache size 0 max 16384 2024-05-09T00:09:40.138 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:09:40.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 -- 172.21.15.12:0/1028970157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70800a1290 msgr2=0x7f70800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:40.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70800a1290 0x7f70800a3750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7088006010 tx=0x7f708800b260 comp rx=0 tx=0).stop 2024-05-09T00:09:40.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 -- 172.21.15.12:0/1028970157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f709810d940 msgr2=0x7f709819dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:40.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f709810d940 0x7f709819dfa0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f709000ef90 tx=0x7f709000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 -- 172.21.15.12:0/1028970157 shutdown_connections 2024-05-09T00:09:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f70800a1290 0x7f70800a3750 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f709810d940 0x7f709819dfa0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70980694f0 0x7f70981996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 --2- 172.21.15.12:0/1028970157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7098069e10 0x7f7098199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 -- 172.21.15.12:0/1028970157 >> 172.21.15.12:0/1028970157 conn(0x7f7098076f40 msgr2=0x7f709810f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 -- 172.21.15.12:0/1028970157 shutdown_connections 2024-05-09T00:09:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:40.138+0000 7f709cd16700 1 -- 172.21.15.12:0/1028970157 wait complete. 2024-05-09T00:09:40.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 335858 29.49 MB/sec execute 455 sec latency 86.183 ms 2024-05-09T00:09:40.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:40 smithi012 ceph-mon[42503]: pgmap v748: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.5 MiB/s wr, 521 op/s 2024-05-09T00:09:41.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:40 smithi204 ceph-mon[47373]: pgmap v748: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.5 MiB/s wr, 521 op/s 2024-05-09T00:09:41.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:40 smithi160 ceph-mon[46152]: pgmap v748: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.5 MiB/s wr, 521 op/s 2024-05-09T00:09:41.151 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:09:41.151 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:09:41.299 INFO:tasks.workunit.client.0.smithi012.stdout: 10 336395 29.49 MB/sec execute 456 sec latency 88.200 ms 2024-05-09T00:09:41.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.323+0000 7fc1b2e71700 1 -- 172.21.15.12:0/3209853919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac100ab0 msgr2=0x7fc1ac10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.323+0000 7fc1b2e71700 1 --2- 172.21.15.12:0/3209853919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac100ab0 0x7fc1ac10ccc0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b0d0 tx=0x7fc1a400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.323+0000 7fc1b2e71700 1 -- 172.21.15.12:0/3209853919 shutdown_connections 2024-05-09T00:09:41.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.323+0000 7fc1b2e71700 1 --2- 172.21.15.12:0/3209853919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac100ab0 0x7fc1ac10ccc0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.323+0000 7fc1b2e71700 1 --2- 172.21.15.12:0/3209853919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac105f90 0x7fc1ac106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.323+0000 7fc1b2e71700 1 --2- 172.21.15.12:0/3209853919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ac1000f0 0x7fc1ac100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.323+0000 7fc1b2e71700 1 -- 172.21.15.12:0/3209853919 >> 172.21.15.12:0/3209853919 conn(0x7fc1ac076ba0 msgr2=0x7fc1ac076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.323+0000 7fc1b2e71700 1 -- 172.21.15.12:0/3209853919 shutdown_connections 2024-05-09T00:09:41.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.323+0000 7fc1b2e71700 1 -- 172.21.15.12:0/3209853919 wait complete. 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.324+0000 7fc1b2e71700 1 Processor -- start 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.324+0000 7fc1b2e71700 1 -- start start 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.324+0000 7fc1b2e71700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1000f0 0x7fc1ac101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.324+0000 7fc1b2e71700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac100ab0 0x7fc1ac1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.324+0000 7fc1b2e71700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ac105f90 0x7fc1ac102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.324+0000 7fc1b2e71700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac1a5810 con 0x7fc1ac100ab0 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.324+0000 7fc1b2e71700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac1a5980 con 0x7fc1ac1000f0 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.324+0000 7fc1b2e71700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac1a5b40 con 0x7fc1ac105f90 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1abfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac100ab0 0x7fc1ac1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1b140e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ac105f90 0x7fc1ac102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1abfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac100ab0 0x7fc1ac1023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58222/0 (socket says 172.21.15.12:58222) 2024-05-09T00:09:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1abfff700 1 -- 172.21.15.12:0/4146245963 learned_addr learned my addr 172.21.15.12:0/4146245963 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1000f0 0x7fc1ac101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1abfff700 1 -- 172.21.15.12:0/4146245963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ac105f90 msgr2=0x7fc1ac102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1abfff700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ac105f90 0x7fc1ac102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1abfff700 1 -- 172.21.15.12:0/4146245963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1000f0 msgr2=0x7fc1ac101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1abfff700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1000f0 0x7fc1ac101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1abfff700 1 -- 172.21.15.12:0/4146245963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a4009d00 con 0x7fc1ac100ab0 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1abfff700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac100ab0 0x7fc1ac1023d0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fc19800d950 tx=0x7fc19800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.325+0000 7fc1a9ffb700 1 -- 172.21.15.12:0/4146245963 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198009b40 con 0x7fc1ac100ab0 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.326+0000 7fc1a9ffb700 1 -- 172.21.15.12:0/4146245963 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc198009ca0 con 0x7fc1ac100ab0 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.326+0000 7fc1b2e71700 1 -- 172.21.15.12:0/4146245963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac1a5e20 con 0x7fc1ac100ab0 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.326+0000 7fc1a9ffb700 1 -- 172.21.15.12:0/4146245963 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19800f620 con 0x7fc1ac100ab0 2024-05-09T00:09:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.326+0000 7fc1b2e71700 1 -- 172.21.15.12:0/4146245963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac1a62f0 con 0x7fc1ac100ab0 2024-05-09T00:09:41.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.330+0000 7fc1a9ffb700 1 -- 172.21.15.12:0/4146245963 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc198010460 con 0x7fc1ac100ab0 2024-05-09T00:09:41.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1a9ffb700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc19c0a1310 0x7fc19c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1bf02bb80 1 Processor -- start 2024-05-09T00:09:41.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1bf02bb80 1 -- start start 2024-05-09T00:09:41.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1a9ffb700 1 -- 172.21.15.12:0/4146245963 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc1980ccff0 con 0x7fc1ac100ab0 2024-05-09T00:09:41.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db4e80 0x55c092db5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092db57a0 0x55c092daf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092daf910 0x55c092dafd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092d061d0 con 0x55c092db57a0 2024-05-09T00:09:41.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092d03b90 con 0x55c092db4e80 2024-05-09T00:09:41.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092cff480 con 0x55c092daf910 2024-05-09T00:09:41.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1b140e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092daf910 0x55c092dafd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1b0c0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db4e80 0x55c092db5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1b140e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092daf910 0x55c092dafd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54496/0 (socket says 172.21.15.12:54496) 2024-05-09T00:09:41.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1b140e700 1 -- 172.21.15.12:0/2231937170 learned_addr learned my addr 172.21.15.12:0/2231937170 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:41.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.331+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc19c0a1310 0x7fc19c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.332+0000 7fc1abfff700 1 --2- 172.21.15.12:0/2231937170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092db57a0 0x55c092daf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.332+0000 7fc1abfff700 1 -- 172.21.15.12:0/2231937170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092daf910 msgr2=0x55c092dafd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.332+0000 7fc1abfff700 1 --2- 172.21.15.12:0/2231937170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092daf910 0x55c092dafd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.332+0000 7fc1abfff700 1 -- 172.21.15.12:0/2231937170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db4e80 msgr2=0x55c092db5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.332+0000 7fc1abfff700 1 --2- 172.21.15.12:0/2231937170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db4e80 0x55c092db5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.332+0000 7fc1abfff700 1 -- 172.21.15.12:0/2231937170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c092b80430 con 0x55c092db57a0 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.332+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc19c0a1310 0x7fc19c0a37d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000cfd0 tx=0x7fc1a000c4c0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.332+0000 7fc1abfff700 1 --2- 172.21.15.12:0/2231937170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092db57a0 0x55c092daf3d0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc1980ca750 tx=0x7fc198099d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649e9f63c16bacb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.332+0000 7fc196ffd700 1 -- 172.21.15.12:0/2231937170 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980caea0 con 0x55c092db57a0 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.333+0000 7fc196ffd700 1 -- 172.21.15.12:0/2231937170 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19809df40 con 0x55c092db57a0 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.333+0000 7fc196ffd700 1 -- 172.21.15.12:0/2231937170 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198010710 con 0x55c092db57a0 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.333+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2231937170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092db57a0 msgr2=0x55c092daf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.333+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2231937170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092db57a0 0x55c092daf3d0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc1980ca750 tx=0x7fc198099d20 comp rx=0 tx=0).stop 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.333+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2231937170 shutdown_connections 2024-05-09T00:09:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.333+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2231937170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092daf910 0x55c092dafd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.333+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2231937170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db4e80 0x55c092db5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.333+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2231937170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092db57a0 0x55c092daf3d0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.333+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2231937170 >> 172.21.15.12:0/2231937170 conn(0x55c092d0d290 msgr2=0x55c092da9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.334+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2231937170 shutdown_connections 2024-05-09T00:09:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.334+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2231937170 wait complete. 2024-05-09T00:09:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.334+0000 7fc1bf02bb80 1 -- start start 2024-05-09T00:09:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.334+0000 7fc1bf02bb80 1 Processor -- start 2024-05-09T00:09:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092daf910 0x55c092dbb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092db4e80 0x55c092db8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db57a0 0x55c092db91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092c9b5e0 con 0x55c092db4e80 2024-05-09T00:09:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092c9b280 con 0x55c092daf910 2024-05-09T00:09:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092b82720 con 0x55c092db57a0 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1bf02bb80 10 client.0 _ll_register_callbacks cb 0x55c092ca4bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1b0c0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092daf910 0x55c092dbb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1b0c0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092daf910 0x55c092dbb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39484/0 (socket says 172.21.15.12:39484) 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/1934598078 learned_addr learned my addr 172.21.15.12:0/1934598078 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1b140e700 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db57a0 0x55c092db91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/1934598078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db57a0 msgr2=0x55c092db91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db57a0 0x55c092db91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/1934598078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092db4e80 msgr2=0x55c092db8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092db4e80 0x55c092db8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.335+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/1934598078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac1a62f0 con 0x55c092daf910 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.336+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092daf910 0x55c092dbb7b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0014fd0 tx=0x7fc1a000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1233c8cdc676a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.336+0000 7fc18bfff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.336+0000 7fc18bfff700 1 -- 172.21.15.12:0/1934598078 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0026070 con 0x55c092daf910 2024-05-09T00:09:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.336+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/1934598078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c092c582f0 con 0x55c092daf910 2024-05-09T00:09:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.336+0000 7fc18bfff700 1 -- 172.21.15.12:0/1934598078 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc1a0012a70 con 0x55c092daf910 2024-05-09T00:09:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.336+0000 7fc18bfff700 1 -- 172.21.15.12:0/1934598078 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a000ce70 con 0x55c092daf910 2024-05-09T00:09:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.336+0000 7fc1bf02bb80 10 client.31196 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.336+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/1934598078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55c092b2cbe0 con 0x55c092daf910 2024-05-09T00:09:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.336+0000 7fc18bfff700 1 -- 172.21.15.12:0/1934598078 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c092c582f0 con 0x55c092daf910 2024-05-09T00:09:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.337+0000 7fc18bfff700 1 -- 172.21.15.12:0/1934598078 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1a0029020 con 0x55c092daf910 2024-05-09T00:09:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.337+0000 7fc1bf02bb80 10 client.31196 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.337+0000 7fc1bf02bb80 10 client.31196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.337+0000 7fc1bf02bb80 20 client.31196 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.337+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c092dc9300 0x55c092dcb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.337+0000 7fc1bf02bb80 4 client.31196 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.337+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/1934598078 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c092d7e890 con 0x55c092dc9300 2024-05-09T00:09:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.337+0000 7fc1abfff700 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c092dc9300 0x55c092dcb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.337+0000 7fc1abfff700 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c092dc9300 0x55c092dcb720 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7effa7ef8a4303 server_cookie=c089f7b6c1a7543d in_seq=0 out_seq=0 2024-05-09T00:09:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.337+0000 7fc18bfff700 10 client.31196 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc18bfff700 1 -- 172.21.15.12:0/1934598078 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c092d7e890 con 0x55c092dc9300 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc18bfff700 10 client.31196 handle_command_reply: tid=0 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 client.31196 shutdown 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/1934598078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092daf910 msgr2=0x55c092dbb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092daf910 0x55c092dbb7b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0014fd0 tx=0x7fc1a000c2d0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/1934598078 shutdown_connections 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db57a0 0x55c092db91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c092dc9300 0x55c092dcb720 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092daf910 0x55c092dbb7b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/1934598078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092db4e80 0x55c092db8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc18bfff700 0 client.31196 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/1934598078 >> 172.21.15.12:0/1934598078 conn(0x55c092d0d290 msgr2=0x55c092da9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/1934598078 shutdown_connections 2024-05-09T00:09:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/1934598078 wait complete. 2024-05-09T00:09:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.343+0000 7fc1bf02bb80 20 client.31196 trim_cache size 0 max 16384 2024-05-09T00:09:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.363+0000 7fc1bf02bb80 1 Processor -- start 2024-05-09T00:09:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.363+0000 7fc1bf02bb80 1 -- start start 2024-05-09T00:09:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.363+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092daf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.363+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db4e80 0x55c092da9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.363+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db57a0 0x55c092db2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.363+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092da6e40 con 0x55c092daf910 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.363+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092dc68b0 con 0x55c092db57a0 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.363+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092dd08f0 con 0x55c092db4e80 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1b0c0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092daf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1b0c0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092daf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58250/0 (socket says 172.21.15.12:58250) 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/2546633012 learned_addr learned my addr 172.21.15.12:0/2546633012 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1b140e700 1 --2- 172.21.15.12:0/2546633012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db57a0 0x55c092db2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1abfff700 1 --2- 172.21.15.12:0/2546633012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db4e80 0x55c092da9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/2546633012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db4e80 msgr2=0x55c092da9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/2546633012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db4e80 0x55c092da9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/2546633012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db57a0 msgr2=0x55c092db2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/2546633012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db57a0 0x55c092db2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/2546633012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c092cc55e0 con 0x55c092daf910 2024-05-09T00:09:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1abfff700 1 --2- 172.21.15.12:0/2546633012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db4e80 0x55c092da9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:09:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/2546633012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092daf3d0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000c930 tx=0x7fc1a0005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df238c67b145a06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1977fe700 1 -- 172.21.15.12:0/2546633012 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0026030 con 0x55c092daf910 2024-05-09T00:09:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1977fe700 1 -- 172.21.15.12:0/2546633012 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1a001a040 con 0x55c092daf910 2024-05-09T00:09:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.364+0000 7fc1977fe700 1 -- 172.21.15.12:0/2546633012 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0012a70 con 0x55c092daf910 2024-05-09T00:09:41.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.365+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2546633012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 msgr2=0x55c092daf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.365+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2546633012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092daf3d0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000c930 tx=0x7fc1a0005ce0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.365+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2546633012 shutdown_connections 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.365+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2546633012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db57a0 0x55c092db2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.365+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2546633012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092daf3d0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.365+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2546633012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db4e80 0x55c092da9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.365+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2546633012 >> 172.21.15.12:0/2546633012 conn(0x55c092d0d290 msgr2=0x55c092dafcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.365+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2546633012 shutdown_connections 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.365+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2546633012 wait complete. 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.366+0000 7fc1bf02bb80 1 -- start start 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.366+0000 7fc1bf02bb80 1 Processor -- start 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.366+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092dc0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db4e80 0x55c092db1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1bf02bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db57a0 0x55c092db15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092db1c70 con 0x55c092daf910 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092db1de0 con 0x55c092db4e80 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1bf02bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c092e1f7e0 con 0x55c092db57a0 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1bf02bb80 10 client.0 _ll_register_callbacks cb 0x55c092ca4bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1b0c0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092dc0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1b0c0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092dc0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58260/0 (socket says 172.21.15.12:58260) 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/2648082224 learned_addr learned my addr 172.21.15.12:0/2648082224 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1abfff700 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db4e80 0x55c092db1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1b140e700 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db57a0 0x55c092db15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/2648082224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db57a0 msgr2=0x55c092db15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db57a0 0x55c092db15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/2648082224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db4e80 msgr2=0x55c092db1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db4e80 0x55c092db1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1b0c0d700 1 -- 172.21.15.12:0/2648082224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c092b2cbe0 con 0x55c092daf910 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc1b0c0d700 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092dc0580 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00148f0 tx=0x7fc1a000c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7568f832275b1521 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.367+0000 7fc18bfff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc18bfff700 1 -- 172.21.15.12:0/2648082224 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0026040 con 0x55c092daf910 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc18bfff700 1 -- 172.21.15.12:0/2648082224 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc1a0027070 con 0x55c092daf910 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2648082224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc19809df40 con 0x55c092daf910 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc18bfff700 1 -- 172.21.15.12:0/2648082224 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a0005270 con 0x55c092daf910 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc1bf02bb80 10 client.21783 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2648082224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55c092b84d50 con 0x55c092daf910 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc18bfff700 1 -- 172.21.15.12:0/2648082224 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19809df40 con 0x55c092daf910 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc18bfff700 1 -- 172.21.15.12:0/2648082224 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1a003a070 con 0x55c092daf910 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc1bf02bb80 10 client.21783 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc1bf02bb80 10 client.21783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc1bf02bb80 20 client.21783 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c092e20ad0 0x55c092e22ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc1bf02bb80 4 client.21783 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:09:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.368+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2648082224 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c092cf7f40 con 0x55c092e20ad0 2024-05-09T00:09:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.369+0000 7fc1abfff700 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c092e20ad0 0x55c092e22ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.369+0000 7fc18bfff700 10 client.21783 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.369+0000 7fc1abfff700 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c092e20ad0 0x55c092e22ef0 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af32f141b6324aa server_cookie=dfe93af6129aa4ec in_seq=0 out_seq=0 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc18bfff700 1 -- 172.21.15.12:0/2648082224 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c092cf7f40 con 0x55c092e20ad0 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc18bfff700 10 client.21783 handle_command_reply: tid=0 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 client.21783 shutdown 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2648082224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 msgr2=0x55c092dc0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092dc0580 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00148f0 tx=0x7fc1a000c610 comp rx=0 tx=0).stop 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2648082224 shutdown_connections 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c092db57a0 0x55c092db15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c092daf910 0x55c092dc0580 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c092e20ad0 0x55c092e22ef0 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 --2- 172.21.15.12:0/2648082224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c092db4e80 0x55c092db1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc18bfff700 0 client.21783 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2648082224 >> 172.21.15.12:0/2648082224 conn(0x55c092d0d290 msgr2=0x55c092db2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2648082224 shutdown_connections 2024-05-09T00:09:41.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 1 -- 172.21.15.12:0/2648082224 wait complete. 2024-05-09T00:09:41.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.372+0000 7fc1bf02bb80 20 client.21783 trim_cache size 0 max 16384 2024-05-09T00:09:41.373 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:09:41.373 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:09:41.373 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "a3eae37e-2b3d-4508-a274-ae3540eca36f", 2024-05-09T00:09:41.373 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:09:41.373 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:09:41.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 -- 172.21.15.12:0/4146245963 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc19c0a1310 msgr2=0x7fc19c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc19c0a1310 0x7fc19c0a37d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000cfd0 tx=0x7fc1a000c4c0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 -- 172.21.15.12:0/4146245963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac100ab0 msgr2=0x7fc1ac1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac100ab0 0x7fc1ac1023d0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fc19800d950 tx=0x7fc19800dd10 comp rx=0 tx=0).stop 2024-05-09T00:09:41.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 -- 172.21.15.12:0/4146245963 shutdown_connections 2024-05-09T00:09:41.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc19c0a1310 0x7fc19c0a37d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ac105f90 0x7fc1ac102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1000f0 0x7fc1ac101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 --2- 172.21.15.12:0/4146245963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac100ab0 0x7fc1ac1023d0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 -- 172.21.15.12:0/4146245963 >> 172.21.15.12:0/4146245963 conn(0x7fc1ac076ba0 msgr2=0x7fc1ac104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 -- 172.21.15.12:0/4146245963 shutdown_connections 2024-05-09T00:09:41.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.373+0000 7fc1b2e71700 1 -- 172.21.15.12:0/4146245963 wait complete. 2024-05-09T00:09:41.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:09:41.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:41 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:41.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.610+0000 7f66a4bae700 1 -- 172.21.15.12:0/1644845947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0108bc0 msgr2=0x7f66a010c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.610+0000 7f66a4bae700 1 --2- 172.21.15.12:0/1644845947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0108bc0 0x7f66a010c0f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6690005ce0 tx=0x7f6690009a60 comp rx=0 tx=0).stop 2024-05-09T00:09:41.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.610+0000 7f66a4bae700 1 -- 172.21.15.12:0/1644845947 shutdown_connections 2024-05-09T00:09:41.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.610+0000 7f66a4bae700 1 --2- 172.21.15.12:0/1644845947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66a010c820 0x7f66a0079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.610+0000 7f66a4bae700 1 --2- 172.21.15.12:0/1644845947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a0108210 0x7f66a01085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.610+0000 7f66a4bae700 1 --2- 172.21.15.12:0/1644845947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0108bc0 0x7f66a010c0f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.611+0000 7f66a4bae700 1 -- 172.21.15.12:0/1644845947 >> 172.21.15.12:0/1644845947 conn(0x7f66a006bff0 msgr2=0x7f66a006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.611+0000 7f66a4bae700 1 -- 172.21.15.12:0/1644845947 shutdown_connections 2024-05-09T00:09:41.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.622+0000 7f66a4bae700 1 -- 172.21.15.12:0/1644845947 wait complete. 2024-05-09T00:09:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f66a4bae700 1 Processor -- start 2024-05-09T00:09:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f66a4bae700 1 -- start start 2024-05-09T00:09:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f66a4bae700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0108210 0x7f66a00839e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f66a4bae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66a010c820 0x7f66a007da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f66a4bae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a007dfc0 0x7f66a007e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f66a4bae700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a01c39a0 con 0x7f66a007dfc0 2024-05-09T00:09:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f66a4bae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a01303f0 con 0x7f66a010c820 2024-05-09T00:09:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f66a4bae700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a01305b0 con 0x7f66a0108210 2024-05-09T00:09:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f669ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a007dfc0 0x7f66a007e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f669ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a007dfc0 0x7f66a007e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58274/0 (socket says 172.21.15.12:58274) 2024-05-09T00:09:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f669ed9d700 1 -- 172.21.15.12:0/403890847 learned_addr learned my addr 172.21.15.12:0/403890847 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:41.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.623+0000 7f669dd9b700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66a010c820 0x7f66a007da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.624+0000 7f669dd9b700 1 -- 172.21.15.12:0/403890847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0108210 msgr2=0x7f66a00839e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:09:41.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.624+0000 7f669dd9b700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0108210 0x7f66a00839e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.624+0000 7f669dd9b700 1 -- 172.21.15.12:0/403890847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a007dfc0 msgr2=0x7f66a007e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.624+0000 7f669dd9b700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a007dfc0 0x7f66a007e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.624+0000 7f669dd9b700 1 -- 172.21.15.12:0/403890847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66900095f0 con 0x7f66a010c820 2024-05-09T00:09:41.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.624+0000 7f669dd9b700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66a010c820 0x7f66a007da80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f669000b580 tx=0x7f66900057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.625+0000 7f668f7fe700 1 -- 172.21.15.12:0/403890847 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690003a40 con 0x7f66a010c820 2024-05-09T00:09:41.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.625+0000 7f66a4bae700 1 -- 172.21.15.12:0/403890847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a0130830 con 0x7f66a010c820 2024-05-09T00:09:41.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.625+0000 7f668f7fe700 1 -- 172.21.15.12:0/403890847 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6690004060 con 0x7f66a010c820 2024-05-09T00:09:41.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.625+0000 7f66a4bae700 1 -- 172.21.15.12:0/403890847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a0130d30 con 0x7f66a010c820 2024-05-09T00:09:41.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.625+0000 7f668f7fe700 1 -- 172.21.15.12:0/403890847 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669000ecc0 con 0x7f66a010c820 2024-05-09T00:09:41.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.631+0000 7f66b0d68b80 1 Processor -- start 2024-05-09T00:09:41.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.633+0000 7f668f7fe700 1 -- 172.21.15.12:0/403890847 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f669000e450 con 0x7f66a010c820 2024-05-09T00:09:41.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.633+0000 7f668f7fe700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f66880a1390 0x7f66880a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.633+0000 7f668f7fe700 1 -- 172.21.15.12:0/403890847 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6690012070 con 0x7f66a010c820 2024-05-09T00:09:41.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.634+0000 7f66b0d68b80 1 -- start start 2024-05-09T00:09:41.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.634+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 0x55e155bfd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.634+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bfd720 0x55e155bf7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.634+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 0x55e155bf7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.634+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155b4e150 con 0x55e155bfd720 2024-05-09T00:09:41.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.634+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155b4bb10 con 0x55e155bf7890 2024-05-09T00:09:41.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.634+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155b47400 con 0x55e155bfce00 2024-05-09T00:09:41.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.634+0000 7f669ed9d700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f66880a1390 0x7f66880a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.635+0000 7f669ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 0x55e155bf7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.635+0000 7f669ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 0x55e155bf7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39538/0 (socket says 172.21.15.12:39538) 2024-05-09T00:09:41.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.635+0000 7f669ed9d700 1 -- 172.21.15.12:0/1713359349 learned_addr learned my addr 172.21.15.12:0/1713359349 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:41.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.635+0000 7f669ed9d700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f66880a1390 0x7f66880a3850 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6698012690 tx=0x7f6698012a50 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.635+0000 7f669e59c700 1 --2- 172.21.15.12:0/1713359349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 0x55e155bfd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.635+0000 7f669ed9d700 1 -- 172.21.15.12:0/1713359349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 msgr2=0x55e155bfd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.635+0000 7f669ed9d700 1 --2- 172.21.15.12:0/1713359349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 0x55e155bfd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.635+0000 7f669ed9d700 1 -- 172.21.15.12:0/1713359349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bfd720 msgr2=0x55e155bf7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.635+0000 7f669ed9d700 1 --2- 172.21.15.12:0/1713359349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bfd720 0x55e155bf7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.635+0000 7f669ed9d700 1 -- 172.21.15.12:0/1713359349 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1559c83b0 con 0x55e155bf7890 2024-05-09T00:09:41.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.642+0000 7f669ed9d700 1 --2- 172.21.15.12:0/1713359349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 0x55e155bf7cd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6698018730 tx=0x7f669801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6e11f73702ef0d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.642+0000 7f668cff9700 1 -- 172.21.15.12:0/1713359349 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698023630 con 0x55e155bf7890 2024-05-09T00:09:41.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.642+0000 7f668cff9700 1 -- 172.21.15.12:0/1713359349 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669800f040 con 0x55e155bf7890 2024-05-09T00:09:41.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.642+0000 7f668cff9700 1 -- 172.21.15.12:0/1713359349 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698022a10 con 0x55e155bf7890 2024-05-09T00:09:41.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.643+0000 7f66b0d68b80 1 -- 172.21.15.12:0/1713359349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 msgr2=0x55e155bf7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.643+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/1713359349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 0x55e155bf7cd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6698018730 tx=0x7f669801a000 comp rx=0 tx=0).stop 2024-05-09T00:09:41.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.646+0000 7f66b0d68b80 1 -- 172.21.15.12:0/1713359349 shutdown_connections 2024-05-09T00:09:41.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.646+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/1713359349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 0x55e155bf7cd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.646+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/1713359349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 0x55e155bfd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.646+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/1713359349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bfd720 0x55e155bf7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.646+0000 7f66b0d68b80 1 -- 172.21.15.12:0/1713359349 >> 172.21.15.12:0/1713359349 conn(0x55e155b55210 msgr2=0x55e155bf1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.648+0000 7f66b0d68b80 1 -- 172.21.15.12:0/1713359349 shutdown_connections 2024-05-09T00:09:41.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.648+0000 7f66b0d68b80 1 -- 172.21.15.12:0/1713359349 wait complete. 2024-05-09T00:09:41.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.648+0000 7f66b0d68b80 1 -- start start 2024-05-09T00:09:41.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.648+0000 7f66b0d68b80 1 Processor -- start 2024-05-09T00:09:41.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.649+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 0x55e155c00b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.649+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 0x55e155c010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.649+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bfd720 0x55e155c01600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.649+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155ae3560 con 0x55e155bfd720 2024-05-09T00:09:41.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.649+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155ae3200 con 0x55e155bf7890 2024-05-09T00:09:41.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.649+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1559ca6a0 con 0x55e155bfce00 2024-05-09T00:09:41.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.649+0000 7f66b0d68b80 10 client.0 _ll_register_callbacks cb 0x55e155aecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:41.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.649+0000 7f669dd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 0x55e155c010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.650+0000 7f669e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 0x55e155c00b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.650+0000 7f669dd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 0x55e155c010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54568/0 (socket says 172.21.15.12:54568) 2024-05-09T00:09:41.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.650+0000 7f669dd9b700 1 -- 172.21.15.12:0/341430081 learned_addr learned my addr 172.21.15.12:0/341430081 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:41.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.650+0000 7f669ed9d700 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bfd720 0x55e155c01600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.651+0000 7f669dd9b700 1 -- 172.21.15.12:0/341430081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 msgr2=0x55e155c00b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.651+0000 7f669dd9b700 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 0x55e155c00b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.651+0000 7f669dd9b700 1 -- 172.21.15.12:0/341430081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bfd720 msgr2=0x55e155c01600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.651+0000 7f669dd9b700 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bfd720 0x55e155c01600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.651+0000 7f669dd9b700 1 -- 172.21.15.12:0/341430081 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66980184a0 con 0x55e155bfce00 2024-05-09T00:09:41.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.651+0000 7f669ed9d700 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bfd720 0x55e155c01600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:09:41.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.651+0000 7f669dd9b700 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 0x55e155c010c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f669009a850 tx=0x7f66900cb660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfaf7b88f4f44689 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.651+0000 7f6685ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:09:41.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.652+0000 7f6685ffb700 1 -- 172.21.15.12:0/341430081 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690014e50 con 0x55e155bfce00 2024-05-09T00:09:41.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.652+0000 7f6685ffb700 1 -- 172.21.15.12:0/341430081 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f669009b250 con 0x55e155bfce00 2024-05-09T00:09:41.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.652+0000 7f66b0d68b80 1 -- 172.21.15.12:0/341430081 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e155aa0270 con 0x55e155bfce00 2024-05-09T00:09:41.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.652+0000 7f6685ffb700 1 -- 172.21.15.12:0/341430081 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f669000ee20 con 0x55e155bfce00 2024-05-09T00:09:41.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.653+0000 7f66b0d68b80 10 client.31486 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:41.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.653+0000 7f66b0d68b80 1 -- 172.21.15.12:0/341430081 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55e155974b60 con 0x55e155bfce00 2024-05-09T00:09:41.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.653+0000 7f6685ffb700 1 -- 172.21.15.12:0/341430081 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669007d020 con 0x55e155bfce00 2024-05-09T00:09:41.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.654+0000 7f6685ffb700 1 -- 172.21.15.12:0/341430081 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f66900b3320 con 0x55e155bfce00 2024-05-09T00:09:41.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.654+0000 7f66b0d68b80 10 client.31486 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.654+0000 7f66b0d68b80 10 client.31486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.654+0000 7f66b0d68b80 20 client.31486 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.654+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e155c10ff0 0x55e155c13410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.654+0000 7f66b0d68b80 4 client.31486 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.654+0000 7f66b0d68b80 1 -- 172.21.15.12:0/341430081 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e155bc6810 con 0x55e155c10ff0 2024-05-09T00:09:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.654+0000 7f669e59c700 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e155c10ff0 0x55e155c13410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.655+0000 7f669e59c700 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e155c10ff0 0x55e155c13410 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cdc8c39f383ff16 server_cookie=234cb33b73e3e935 in_seq=0 out_seq=0 2024-05-09T00:09:41.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.655+0000 7f6685ffb700 10 client.31486 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:41.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.659+0000 7f6685ffb700 1 -- 172.21.15.12:0/341430081 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e155bc6810 con 0x55e155c10ff0 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f6685ffb700 10 client.31486 handle_command_reply: tid=0 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 client.31486 shutdown 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 -- 172.21.15.12:0/341430081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 msgr2=0x55e155c010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 0x55e155c010c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f669009a850 tx=0x7f66900cb660 comp rx=0 tx=0).stop 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 -- 172.21.15.12:0/341430081 shutdown_connections 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bfd720 0x55e155c01600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e155c10ff0 0x55e155c13410 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bf7890 0x55e155c00b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/341430081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bfce00 0x55e155c010c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 -- 172.21.15.12:0/341430081 >> 172.21.15.12:0/341430081 conn(0x55e155b55210 msgr2=0x55e155bf15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f6685ffb700 0 client.31486 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 -- 172.21.15.12:0/341430081 shutdown_connections 2024-05-09T00:09:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 1 -- 172.21.15.12:0/341430081 wait complete. 2024-05-09T00:09:41.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.660+0000 7f66b0d68b80 20 client.31486 trim_cache size 0 max 16384 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.682+0000 7f66b0d68b80 1 Processor -- start 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.682+0000 7f66b0d68b80 1 -- start start 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.682+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155bf7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bfd720 0x55e155bfc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 0x55e155c01420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155beedc0 con 0x55e155bf7890 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155bf28e0 con 0x55e155bfd720 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155bfd560 con 0x55e155bf1d20 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155bf7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155bf7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58304/0 (socket says 172.21.15.12:58304) 2024-05-09T00:09:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669e59c700 1 -- 172.21.15.12:0/3972566970 learned_addr learned my addr 172.21.15.12:0/3972566970 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:41.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669dd9b700 1 --2- 172.21.15.12:0/3972566970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bfd720 0x55e155bfc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669ed9d700 1 --2- 172.21.15.12:0/3972566970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 0x55e155c01420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669e59c700 1 -- 172.21.15.12:0/3972566970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 msgr2=0x55e155c01420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669e59c700 1 --2- 172.21.15.12:0/3972566970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 0x55e155c01420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669e59c700 1 -- 172.21.15.12:0/3972566970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bfd720 msgr2=0x55e155bfc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669e59c700 1 --2- 172.21.15.12:0/3972566970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bfd720 0x55e155bfc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669e59c700 1 -- 172.21.15.12:0/3972566970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e155b0d560 con 0x55e155bf7890 2024-05-09T00:09:41.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.683+0000 7f669e59c700 1 --2- 172.21.15.12:0/3972566970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155bf7350 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x55e155c13cc0 tx=0x7f669400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda35f0ceadff078 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.684+0000 7f668d7fa700 1 -- 172.21.15.12:0/3972566970 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694015760 con 0x55e155bf7890 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.684+0000 7f668d7fa700 1 -- 172.21.15.12:0/3972566970 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669401d070 con 0x55e155bf7890 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.684+0000 7f668d7fa700 1 -- 172.21.15.12:0/3972566970 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694024bd0 con 0x55e155bf7890 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 -- 172.21.15.12:0/3972566970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 msgr2=0x55e155bf7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/3972566970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155bf7350 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x55e155c13cc0 tx=0x7f669400bd80 comp rx=0 tx=0).stop 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 -- 172.21.15.12:0/3972566970 shutdown_connections 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/3972566970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 0x55e155c01420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/3972566970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155bf7350 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/3972566970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bfd720 0x55e155bfc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 -- 172.21.15.12:0/3972566970 >> 172.21.15.12:0/3972566970 conn(0x55e155b55210 msgr2=0x55e155bfbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 -- 172.21.15.12:0/3972566970 shutdown_connections 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 -- 172.21.15.12:0/3972566970 wait complete. 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 -- start start 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.685+0000 7f66b0d68b80 1 Processor -- start 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 0x55e155c086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155c04a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f66b0d68b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bfd720 0x55e155c051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155c058e0 con 0x55e155bf7890 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155c674f0 con 0x55e155bfd720 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f66b0d68b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e155c67630 con 0x55e155bf1d20 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f66b0d68b80 10 client.0 _ll_register_callbacks cb 0x55e155aecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155c04a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 0x55e155c086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155c04a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58314/0 (socket says 172.21.15.12:58314) 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 0x55e155c086c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54598/0 (socket says 172.21.15.12:54598) 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669dd9b700 1 -- 172.21.15.12:0/613698457 learned_addr learned my addr 172.21.15.12:0/613698457 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669ed9d700 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bfd720 0x55e155c051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669dd9b700 1 -- 172.21.15.12:0/613698457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 msgr2=0x55e155c086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669dd9b700 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 0x55e155c086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669dd9b700 1 -- 172.21.15.12:0/613698457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bfd720 msgr2=0x55e155c051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669dd9b700 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bfd720 0x55e155c051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669dd9b700 1 -- 172.21.15.12:0/613698457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669400e040 con 0x55e155bf7890 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669e59c700 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 0x55e155c086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f669dd9b700 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155c04a90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f6690098ea0 tx=0x7f669000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65948e4261f2a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f6685ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.686+0000 7f6685ffb700 1 -- 172.21.15.12:0/613698457 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900b3400 con 0x55e155bf7890 2024-05-09T00:09:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f6685ffb700 1 -- 172.21.15.12:0/613698457 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f669007e080 con 0x55e155bf7890 2024-05-09T00:09:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f66b0d68b80 1 -- 172.21.15.12:0/613698457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f669800f040 con 0x55e155bf7890 2024-05-09T00:09:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f6685ffb700 1 -- 172.21.15.12:0/613698457 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f669009a4b0 con 0x55e155bf7890 2024-05-09T00:09:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f66b0d68b80 10 client.21795 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f66b0d68b80 1 -- 172.21.15.12:0/613698457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55e1559cccd0 con 0x55e155bf7890 2024-05-09T00:09:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f6685ffb700 1 -- 172.21.15.12:0/613698457 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669800f040 con 0x55e155bf7890 2024-05-09T00:09:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f6685ffb700 1 -- 172.21.15.12:0/613698457 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f669009bb80 con 0x55e155bf7890 2024-05-09T00:09:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f66b0d68b80 10 client.21795 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f66b0d68b80 10 client.21795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f66b0d68b80 20 client.21795 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e155c68950 0x55e155c6ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f66b0d68b80 4 client.21795 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:09:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f66b0d68b80 1 -- 172.21.15.12:0/613698457 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e155b3fec0 con 0x55e155c68950 2024-05-09T00:09:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.687+0000 7f669e59c700 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e155c68950 0x55e155c6ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.688+0000 7f669e59c700 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e155c68950 0x55e155c6ad40 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c682a3d8bb9e8231 server_cookie=b4763964298d8934 in_seq=0 out_seq=0 2024-05-09T00:09:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.688+0000 7f6685ffb700 10 client.21795 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:41.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f6685ffb700 1 -- 172.21.15.12:0/613698457 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55e155b3fec0 con 0x55e155c68950 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f6685ffb700 10 client.21795 handle_command_reply: tid=0 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 client.21795 shutdown 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 -- 172.21.15.12:0/613698457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 msgr2=0x55e155c04a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155c04a90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f6690098ea0 tx=0x7f669000bf90 comp rx=0 tx=0).stop 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 -- 172.21.15.12:0/613698457 shutdown_connections 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e155c68950 0x55e155c6ad40 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e155bfd720 0x55e155c051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e155bf1d20 0x55e155c086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 --2- 172.21.15.12:0/613698457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e155bf7890 0x55e155c04a90 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 -- 172.21.15.12:0/613698457 >> 172.21.15.12:0/613698457 conn(0x55e155b55210 msgr2=0x55e155bff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f6685ffb700 0 client.21795 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 -- 172.21.15.12:0/613698457 shutdown_connections 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.691+0000 7f66b0d68b80 1 -- 172.21.15.12:0/613698457 wait complete. 2024-05-09T00:09:41.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66b0d68b80 20 client.21795 trim_cache size 0 max 16384 2024-05-09T00:09:41.692 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:09:41.692 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (182 inodes in the stack)", 2024-05-09T00:09:41.692 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:09:41.692 INFO:teuthology.orchestra.run.smithi012.stdout: "a3eae37e-2b3d-4508-a274-ae3540eca36f": { 2024-05-09T00:09:41.692 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:09:41.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "a3eae37e-2b3d-4508-a274-ae3540eca36f", 2024-05-09T00:09:41.692 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:09:41.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:09:41.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:09:41.692 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:09:41.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66a4bae700 1 -- 172.21.15.12:0/403890847 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f66880a1390 msgr2=0x7f66880a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66a4bae700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f66880a1390 0x7f66880a3850 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6698012690 tx=0x7f6698012a50 comp rx=0 tx=0).stop 2024-05-09T00:09:41.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66a4bae700 1 -- 172.21.15.12:0/403890847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66a010c820 msgr2=0x7f66a007da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:41.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66a4bae700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66a010c820 0x7f66a007da80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f669000b580 tx=0x7f66900057c0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66a4bae700 1 -- 172.21.15.12:0/403890847 shutdown_connections 2024-05-09T00:09:41.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66a4bae700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f66880a1390 0x7f66880a3850 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66a4bae700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a007dfc0 0x7f66a007e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66a4bae700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0108210 0x7f66a00839e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66a4bae700 1 --2- 172.21.15.12:0/403890847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66a010c820 0x7f66a007da80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:41.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.692+0000 7f66a4bae700 1 -- 172.21.15.12:0/403890847 >> 172.21.15.12:0/403890847 conn(0x7f66a006bff0 msgr2=0x7f66a0070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:41.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.693+0000 7f66a4bae700 1 -- 172.21.15.12:0/403890847 shutdown_connections 2024-05-09T00:09:41.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:09:41.693+0000 7f66a4bae700 1 -- 172.21.15.12:0/403890847 wait complete. 2024-05-09T00:09:41.705 INFO:tasks.cephfs.filesystem:scrub status for tag:a3eae37e-2b3d-4508-a274-ae3540eca36f - {'path': '/', 'tag': 'a3eae37e-2b3d-4508-a274-ae3540eca36f', 'options': 'recursive,force'} 2024-05-09T00:09:42.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:41 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:42.020 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:41 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:42.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 336659 29.47 MB/sec execute 457 sec latency 274.986 ms 2024-05-09T00:09:42.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:42 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:09:42.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:42 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:42.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:42 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:42.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:42 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:09:42.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:42 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:42.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:42 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:42.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:42 smithi012 ceph-mon[42503]: pgmap v749: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.0 MiB/s wr, 512 op/s 2024-05-09T00:09:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:42 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:09:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:42 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:42 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:42 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:09:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:42 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:42 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:42 smithi204 ceph-mon[47373]: pgmap v749: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.0 MiB/s wr, 512 op/s 2024-05-09T00:09:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:42 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:09:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:42 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:43.016 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:42 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:42 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:09:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:42 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:42 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:43.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:42 smithi160 ceph-mon[46152]: pgmap v749: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.0 MiB/s wr, 512 op/s 2024-05-09T00:09:43.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 337053 29.44 MB/sec execute 458 sec latency 144.256 ms 2024-05-09T00:09:43.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:43 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:44.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:43 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:44.020 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:43 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:09:44.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 337469 29.41 MB/sec execute 459 sec latency 132.695 ms 2024-05-09T00:09:44.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:44 smithi012 ceph-mon[42503]: pgmap v750: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 713 op/s 2024-05-09T00:09:45.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:44 smithi204 ceph-mon[47373]: pgmap v750: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 713 op/s 2024-05-09T00:09:45.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:44 smithi160 ceph-mon[46152]: pgmap v750: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 713 op/s 2024-05-09T00:09:45.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 338094 29.42 MB/sec execute 460 sec latency 89.750 ms 2024-05-09T00:09:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.677+0000 7f9f3c137700 1 -- 172.21.15.12:0/3355372348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f34107010 msgr2=0x7f9f341073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.677+0000 7f9f3c137700 1 --2- 172.21.15.12:0/3355372348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f34107010 0x7f9f341073f0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009990 tx=0x7f9f30009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.677+0000 7f9f3c137700 1 -- 172.21.15.12:0/3355372348 shutdown_connections 2024-05-09T00:09:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.677+0000 7f9f3c137700 1 --2- 172.21.15.12:0/3355372348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f34069b50 0x7f9f3410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.677+0000 7f9f3c137700 1 --2- 172.21.15.12:0/3355372348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f34107010 0x7f9f341073f0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.677+0000 7f9f3c137700 1 --2- 172.21.15.12:0/3355372348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f34069190 0x7f9f34069610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.677+0000 7f9f3c137700 1 -- 172.21.15.12:0/3355372348 >> 172.21.15.12:0/3355372348 conn(0x7f9f34076c10 msgr2=0x7f9f34077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.677+0000 7f9f3c137700 1 -- 172.21.15.12:0/3355372348 shutdown_connections 2024-05-09T00:09:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.677+0000 7f9f3c137700 1 -- 172.21.15.12:0/3355372348 wait complete. 2024-05-09T00:09:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.678+0000 7f9f3c137700 1 Processor -- start 2024-05-09T00:09:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.678+0000 7f9f3c137700 1 -- start start 2024-05-09T00:09:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.678+0000 7f9f3c137700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f34069190 0x7f9f34199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.678+0000 7f9f3c137700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f34069b50 0x7f9f34199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.678+0000 7f9f3c137700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f34107010 0x7f9f3419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.678+0000 7f9f3c137700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3419e490 con 0x7f9f34069190 2024-05-09T00:09:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.678+0000 7f9f3c137700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3419e600 con 0x7f9f34069b50 2024-05-09T00:09:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.678+0000 7f9f3c137700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3419e7c0 con 0x7f9f34107010 2024-05-09T00:09:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.678+0000 7f9f39ed3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f34069190 0x7f9f34199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f3a6d4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f34107010 0x7f9f3419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f39ed3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f34069190 0x7f9f34199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58332/0 (socket says 172.21.15.12:58332) 2024-05-09T00:09:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f396d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f34069b50 0x7f9f34199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f39ed3700 1 -- 172.21.15.12:0/1633731087 learned_addr learned my addr 172.21.15.12:0/1633731087 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f39ed3700 1 -- 172.21.15.12:0/1633731087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f34107010 msgr2=0x7f9f3419de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f34107010 0x7f9f3419de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f39ed3700 1 -- 172.21.15.12:0/1633731087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f34069b50 msgr2=0x7f9f34199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f34069b50 0x7f9f34199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f39ed3700 1 -- 172.21.15.12:0/1633731087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f300095f0 con 0x7f9f34069190 2024-05-09T00:09:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f3a6d4700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f34107010 0x7f9f3419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f34069190 0x7f9f34199570 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f9f30004760 tx=0x7f9f30004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f2affd700 1 -- 172.21.15.12:0/1633731087 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3001d070 con 0x7f9f34069190 2024-05-09T00:09:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f3c137700 1 -- 172.21.15.12:0/1633731087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3419ea40 con 0x7f9f34069190 2024-05-09T00:09:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f2affd700 1 -- 172.21.15.12:0/1633731087 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f30022470 con 0x7f9f34069190 2024-05-09T00:09:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f2affd700 1 -- 172.21.15.12:0/1633731087 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30017650 con 0x7f9f34069190 2024-05-09T00:09:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.679+0000 7f9f3c137700 1 -- 172.21.15.12:0/1633731087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f341a62e0 con 0x7f9f34069190 2024-05-09T00:09:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.682+0000 7f9f2affd700 1 -- 172.21.15.12:0/1633731087 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9f300225e0 con 0x7f9f34069190 2024-05-09T00:09:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.683+0000 7f9f2affd700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f200a1310 0x7f9f200a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.683+0000 7f9f2affd700 1 -- 172.21.15.12:0/1633731087 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9f300cde80 con 0x7f9f34069190 2024-05-09T00:09:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.683+0000 7f9f396d2700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f200a1310 0x7f9f200a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.684+0000 7f9f482f1b80 1 Processor -- start 2024-05-09T00:09:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.684+0000 7f9f396d2700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f200a1310 0x7f9f200a37d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9f3419aac0 tx=0x7f9f24009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.684+0000 7f9f482f1b80 1 -- start start 2024-05-09T00:09:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.684+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 0x561b037204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.684+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03712f40 0x561b037133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.684+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03713900 0x561b03727e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.684+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0366a0f0 con 0x561b03713900 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.684+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b03667ab0 con 0x561b03720100 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.684+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b036633a0 con 0x561b03712f40 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f39ed3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 0x561b037204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f396d2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03712f40 0x561b037133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f39ed3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 0x561b037204e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39592/0 (socket says 172.21.15.12:39592) 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f39ed3700 1 -- 172.21.15.12:0/36094178 learned_addr learned my addr 172.21.15.12:0/36094178 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f3a6d4700 1 --2- 172.21.15.12:0/36094178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03713900 0x561b03727e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f396d2700 1 -- 172.21.15.12:0/36094178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 msgr2=0x561b037204e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f396d2700 1 --2- 172.21.15.12:0/36094178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 0x561b037204e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f396d2700 1 -- 172.21.15.12:0/36094178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03713900 msgr2=0x561b03727e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f396d2700 1 --2- 172.21.15.12:0/36094178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03713900 0x561b03727e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f396d2700 1 -- 172.21.15.12:0/36094178 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b034e4350 con 0x561b03712f40 2024-05-09T00:09:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/36094178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 0x561b037204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f3a6d4700 1 --2- 172.21.15.12:0/36094178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03713900 0x561b03727e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:09:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f396d2700 1 --2- 172.21.15.12:0/36094178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03712f40 0x561b037133c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9f24011c90 tx=0x7f9f24014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53098576600d9387 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.685+0000 7f9f1ffff700 1 -- 172.21.15.12:0/36094178 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24004230 con 0x561b03712f40 2024-05-09T00:09:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.686+0000 7f9f1ffff700 1 -- 172.21.15.12:0/36094178 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f24021070 con 0x561b03712f40 2024-05-09T00:09:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.686+0000 7f9f1ffff700 1 -- 172.21.15.12:0/36094178 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24003be0 con 0x561b03712f40 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.686+0000 7f9f482f1b80 1 -- 172.21.15.12:0/36094178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03712f40 msgr2=0x561b037133c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.686+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/36094178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03712f40 0x561b037133c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9f24011c90 tx=0x7f9f24014000 comp rx=0 tx=0).stop 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.686+0000 7f9f482f1b80 1 -- 172.21.15.12:0/36094178 shutdown_connections 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.686+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/36094178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03713900 0x561b03727e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.686+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/36094178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 0x561b037204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.686+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/36094178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03712f40 0x561b037133c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.686+0000 7f9f482f1b80 1 -- 172.21.15.12:0/36094178 >> 172.21.15.12:0/36094178 conn(0x561b036711b0 msgr2=0x561b0370d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.687+0000 7f9f482f1b80 1 -- 172.21.15.12:0/36094178 shutdown_connections 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.687+0000 7f9f482f1b80 1 -- 172.21.15.12:0/36094178 wait complete. 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.687+0000 7f9f482f1b80 1 -- start start 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.687+0000 7f9f482f1b80 1 Processor -- start 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 0x561b03727dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b03725110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 0x561b03725850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b035ff500 con 0x561b03712f40 2024-05-09T00:09:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b035ff1a0 con 0x561b03720100 2024-05-09T00:09:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b034e6640 con 0x561b03713900 2024-05-09T00:09:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f482f1b80 10 client.0 _ll_register_callbacks cb 0x561b03608ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f396d2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b03725110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f396d2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b03725110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54626/0 (socket says 172.21.15.12:54626) 2024-05-09T00:09:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f39ed3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 0x561b03727dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f396d2700 1 -- 172.21.15.12:0/2521544574 learned_addr learned my addr 172.21.15.12:0/2521544574 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f3a6d4700 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 0x561b03725850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f396d2700 1 -- 172.21.15.12:0/2521544574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 msgr2=0x561b03725850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f396d2700 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 0x561b03725850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f396d2700 1 -- 172.21.15.12:0/2521544574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 msgr2=0x561b03727dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f396d2700 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 0x561b03727dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.688+0000 7f9f396d2700 1 -- 172.21.15.12:0/2521544574 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f341a62e0 con 0x561b03713900 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.689+0000 7f9f396d2700 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b03725110 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400eeb0 tx=0x7f9f2400eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=114d253ffa5cd7e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.689+0000 7f9f1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.689+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/2521544574 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2402a070 con 0x561b03713900 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.689+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/2521544574 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9f24005400 con 0x561b03713900 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.689+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2521544574 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b035bc210 con 0x561b03713900 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.689+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/2521544574 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f240055c0 con 0x561b03713900 2024-05-09T00:09:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.689+0000 7f9f482f1b80 10 client.31504 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.689+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2521544574 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7f9f24017070 con 0x561b03713900 2024-05-09T00:09:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.690+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/2521544574 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b035bc210 con 0x561b03713900 2024-05-09T00:09:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.690+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/2521544574 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f240388e0 con 0x561b03713900 2024-05-09T00:09:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.690+0000 7f9f482f1b80 10 client.31504 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.690+0000 7f9f482f1b80 10 client.31504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.690+0000 7f9f482f1b80 20 client.31504 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.690+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b03735300 0x561b03737720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.690+0000 7f9f482f1b80 4 client.31504 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.690+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2521544574 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b036e27b0 con 0x561b03735300 2024-05-09T00:09:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.690+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b03735300 0x561b03737720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.691+0000 7f9f1d7fa700 10 client.31504 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.691+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b03735300 0x561b03737720 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=145024e9e20ec021 server_cookie=387c28ec5cd5971a in_seq=0 out_seq=0 2024-05-09T00:09:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.692+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/2521544574 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b036e27b0 con 0x561b03735300 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.692+0000 7f9f1d7fa700 10 client.31504 handle_command_reply: tid=0 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.692+0000 7f9f482f1b80 1 client.31504 shutdown 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2521544574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 msgr2=0x561b03725110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b03725110 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400eeb0 tx=0x7f9f2400eee0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2521544574 shutdown_connections 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b03735300 0x561b03737720 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03720100 0x561b03725850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 0x561b03727dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/2521544574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b03725110 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2521544574 >> 172.21.15.12:0/2521544574 conn(0x561b036711b0 msgr2=0x561b03711f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f1d7fa700 0 client.31504 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2521544574 shutdown_connections 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2521544574 wait complete. 2024-05-09T00:09:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.693+0000 7f9f482f1b80 20 client.31504 trim_cache size 0 max 16384 2024-05-09T00:09:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.710+0000 7f9f482f1b80 1 Processor -- start 2024-05-09T00:09:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.710+0000 7f9f482f1b80 1 -- start start 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 0x561b0370f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03713900 0x561b03730f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03720100 0x561b037204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0371fd80 con 0x561b03712f40 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0370f0f0 con 0x561b03713900 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b03720d70 con 0x561b03720100 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f39ed3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 0x561b0370f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f39ed3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 0x561b0370f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58356/0 (socket says 172.21.15.12:58356) 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f39ed3700 1 -- 172.21.15.12:0/2146607138 learned_addr learned my addr 172.21.15.12:0/2146607138 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f396d2700 1 --2- 172.21.15.12:0/2146607138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03713900 0x561b03730f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f39ed3700 1 -- 172.21.15.12:0/2146607138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03720100 msgr2=0x561b037204e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/2146607138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03720100 0x561b037204e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f39ed3700 1 -- 172.21.15.12:0/2146607138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03713900 msgr2=0x561b03730f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/2146607138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03713900 0x561b03730f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.711+0000 7f9f39ed3700 1 -- 172.21.15.12:0/2146607138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b03490b00 con 0x561b03712f40 2024-05-09T00:09:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.712+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/2146607138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 0x561b0370f660 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f9f30097a40 tx=0x7f9f3001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fe26c0094c1dc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.712+0000 7f9f28ff9700 1 -- 172.21.15.12:0/2146607138 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f300a10b0 con 0x561b03712f40 2024-05-09T00:09:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.712+0000 7f9f28ff9700 1 -- 172.21.15.12:0/2146607138 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f3009afd0 con 0x561b03712f40 2024-05-09T00:09:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.712+0000 7f9f28ff9700 1 -- 172.21.15.12:0/2146607138 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f300cb800 con 0x561b03712f40 2024-05-09T00:09:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2146607138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 msgr2=0x561b0370f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/2146607138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 0x561b0370f660 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f9f30097a40 tx=0x7f9f3001a040 comp rx=0 tx=0).stop 2024-05-09T00:09:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2146607138 shutdown_connections 2024-05-09T00:09:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/2146607138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03720100 0x561b037204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/2146607138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03712f40 0x561b0370f660 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/2146607138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03713900 0x561b03730f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2146607138 >> 172.21.15.12:0/2146607138 conn(0x561b036711b0 msgr2=0x561b03711f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2146607138 shutdown_connections 2024-05-09T00:09:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 -- 172.21.15.12:0/2146607138 wait complete. 2024-05-09T00:09:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 -- start start 2024-05-09T00:09:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.713+0000 7f9f482f1b80 1 Processor -- start 2024-05-09T00:09:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03712f40 0x561b0372a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b0372a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f482f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03720100 0x561b0372aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0378a800 con 0x561b03720100 2024-05-09T00:09:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0378a940 con 0x561b03712f40 2024-05-09T00:09:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f482f1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0378aa80 con 0x561b03713900 2024-05-09T00:09:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f482f1b80 10 client.0 _ll_register_callbacks cb 0x561b03608ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f3a6d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03720100 0x561b0372aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f39ed3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03712f40 0x561b0372a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f396d2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b0372a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f3a6d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03720100 0x561b0372aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58366/0 (socket says 172.21.15.12:58366) 2024-05-09T00:09:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f39ed3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03712f40 0x561b0372a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39616/0 (socket says 172.21.15.12:39616) 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f396d2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b0372a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54636/0 (socket says 172.21.15.12:54636) 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f3a6d4700 1 -- 172.21.15.12:0/1194373817 learned_addr learned my addr 172.21.15.12:0/1194373817 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f3a6d4700 1 -- 172.21.15.12:0/1194373817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 msgr2=0x561b0372a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f3a6d4700 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b0372a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f3a6d4700 1 -- 172.21.15.12:0/1194373817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03712f40 msgr2=0x561b0372a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f3a6d4700 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03712f40 0x561b0372a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f3a6d4700 1 -- 172.21.15.12:0/1194373817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3419ea40 con 0x561b03720100 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f396d2700 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b0372a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.714+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03712f40 0x561b0372a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f3a6d4700 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03720100 0x561b0372aee0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c004320 tx=0x7f9f2c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4877f38ad44d9677 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/1194373817 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c004db0 con 0x561b03720100 2024-05-09T00:09:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/1194373817 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9f2c007500 con 0x561b03720100 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f482f1b80 1 -- 172.21.15.12:0/1194373817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f24021070 con 0x561b03720100 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/1194373817 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f2c0076c0 con 0x561b03720100 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f482f1b80 10 client.21813 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f482f1b80 1 -- 172.21.15.12:0/1194373817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x561b03629500 con 0x561b03720100 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/1194373817 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f24021070 con 0x561b03720100 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/1194373817 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f2c026070 con 0x561b03720100 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f482f1b80 10 client.21813 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.715+0000 7f9f482f1b80 10 client.21813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.716+0000 7f9f482f1b80 20 client.21813 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.716+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b0378c2a0 0x561b0378e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.716+0000 7f9f482f1b80 4 client.21813 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.716+0000 7f9f482f1b80 1 -- 172.21.15.12:0/1194373817 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561b0365be60 con 0x561b0378c2a0 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.716+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b0378c2a0 0x561b0378e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.716+0000 7f9f1d7fa700 10 client.21813 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:45.716+0000 7f9f39ed3700 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b0378c2a0 0x561b0378e6c0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf6fa2cb76b1f69 server_cookie=8932efc8b96f41bb in_seq=0 out_seq=0 2024-05-09T00:09:46.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:45 smithi012 ceph-mon[42503]: pgmap v751: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.9 MiB/s wr, 458 op/s 2024-05-09T00:09:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f1d7fa700 1 -- 172.21.15.12:0/1194373817 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561b0365be60 con 0x561b0378c2a0 2024-05-09T00:09:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f1d7fa700 10 client.21813 handle_command_reply: tid=0 2024-05-09T00:09:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 client.21813 shutdown 2024-05-09T00:09:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 -- 172.21.15.12:0/1194373817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03720100 msgr2=0x561b0372aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03720100 0x561b0372aee0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c004320 tx=0x7f9f2c003f90 comp rx=0 tx=0).stop 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 -- 172.21.15.12:0/1194373817 shutdown_connections 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b03720100 0x561b0372aee0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x561b0378c2a0 0x561b0378e6c0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561b03712f40 0x561b0372a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 --2- 172.21.15.12:0/1194373817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b03713900 0x561b0372a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f1d7fa700 0 client.21813 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 -- 172.21.15.12:0/1194373817 >> 172.21.15.12:0/1194373817 conn(0x561b036711b0 msgr2=0x561b037126e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 -- 172.21.15.12:0/1194373817 shutdown_connections 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 1 -- 172.21.15.12:0/1194373817 wait complete. 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.255+0000 7f9f482f1b80 20 client.21813 trim_cache size 0 max 16384 2024-05-09T00:09:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:09:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:09:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:09:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:09:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.260+0000 7f9f3c137700 1 -- 172.21.15.12:0/1633731087 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f200a1310 msgr2=0x7f9f200a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.260+0000 7f9f3c137700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f200a1310 0x7f9f200a37d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9f3419aac0 tx=0x7f9f24009260 comp rx=0 tx=0).stop 2024-05-09T00:09:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.260+0000 7f9f3c137700 1 -- 172.21.15.12:0/1633731087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f34069190 msgr2=0x7f9f34199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.260+0000 7f9f3c137700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f34069190 0x7f9f34199570 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f9f30004760 tx=0x7f9f30004790 comp rx=0 tx=0).stop 2024-05-09T00:09:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.260+0000 7f9f3c137700 1 -- 172.21.15.12:0/1633731087 shutdown_connections 2024-05-09T00:09:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.260+0000 7f9f3c137700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9f200a1310 0x7f9f200a37d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.260+0000 7f9f3c137700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f34107010 0x7f9f3419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.260+0000 7f9f3c137700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f34069190 0x7f9f34199570 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.260+0000 7f9f3c137700 1 --2- 172.21.15.12:0/1633731087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f34069b50 0x7f9f34199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.260+0000 7f9f3c137700 1 -- 172.21.15.12:0/1633731087 >> 172.21.15.12:0/1633731087 conn(0x7f9f34076c10 msgr2=0x7f9f3410c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.261+0000 7f9f3c137700 1 -- 172.21.15.12:0/1633731087 shutdown_connections 2024-05-09T00:09:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:46.261+0000 7f9f3c137700 1 -- 172.21.15.12:0/1633731087 wait complete. 2024-05-09T00:09:46.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:45 smithi204 ceph-mon[47373]: pgmap v751: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.9 MiB/s wr, 458 op/s 2024-05-09T00:09:46.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:45 smithi160 ceph-mon[46152]: pgmap v751: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.9 MiB/s wr, 458 op/s 2024-05-09T00:09:46.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 338474 29.41 MB/sec execute 461 sec latency 395.594 ms 2024-05-09T00:09:47.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 338829 29.39 MB/sec execute 462 sec latency 373.708 ms 2024-05-09T00:09:48.170 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:47 smithi012 ceph-mon[42503]: pgmap v752: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 718 op/s 2024-05-09T00:09:48.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:47 smithi204 ceph-mon[47373]: pgmap v752: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 718 op/s 2024-05-09T00:09:48.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:47 smithi160 ceph-mon[46152]: pgmap v752: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 718 op/s 2024-05-09T00:09:48.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 339253 29.38 MB/sec execute 463 sec latency 114.468 ms 2024-05-09T00:09:49.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:48 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:09:49.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:48 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:09:49.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:48 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:09:49.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 339699 29.38 MB/sec execute 464 sec latency 116.528 ms 2024-05-09T00:09:50.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:49 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:09:50.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:49 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:09:50.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:49 smithi012 ceph-mon[42503]: pgmap v753: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.5 MiB/s wr, 572 op/s 2024-05-09T00:09:50.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:49 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:09:50.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:49 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:09:50.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:49 smithi204 ceph-mon[47373]: pgmap v753: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.5 MiB/s wr, 572 op/s 2024-05-09T00:09:50.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:49 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:09:50.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:49 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:09:50.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:49 smithi160 ceph-mon[46152]: pgmap v753: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.5 MiB/s wr, 572 op/s 2024-05-09T00:09:50.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 340086 29.36 MB/sec execute 465 sec latency 115.552 ms 2024-05-09T00:09:51.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 340413 29.31 MB/sec execute 466 sec latency 128.355 ms 2024-05-09T00:09:52.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:51 smithi012 ceph-mon[42503]: pgmap v754: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.3 MiB/s wr, 564 op/s 2024-05-09T00:09:52.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:51 smithi204 ceph-mon[47373]: pgmap v754: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.3 MiB/s wr, 564 op/s 2024-05-09T00:09:52.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:51 smithi160 ceph-mon[46152]: pgmap v754: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.3 MiB/s wr, 564 op/s 2024-05-09T00:09:52.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 340705 29.29 MB/sec execute 467 sec latency 394.283 ms 2024-05-09T00:09:53.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 341194 29.29 MB/sec execute 468 sec latency 170.853 ms 2024-05-09T00:09:54.169 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:53 smithi012 ceph-mon[42503]: pgmap v755: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 732 op/s 2024-05-09T00:09:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:53 smithi204 ceph-mon[47373]: pgmap v755: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 732 op/s 2024-05-09T00:09:54.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:53 smithi160 ceph-mon[46152]: pgmap v755: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 732 op/s 2024-05-09T00:09:54.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 341708 29.26 MB/sec execute 469 sec latency 123.851 ms 2024-05-09T00:09:55.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 342274 29.28 MB/sec execute 470 sec latency 87.654 ms 2024-05-09T00:09:56.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:55 smithi012 ceph-mon[42503]: pgmap v756: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.7 MiB/s wr, 510 op/s 2024-05-09T00:09:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:55 smithi204 ceph-mon[47373]: pgmap v756: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.7 MiB/s wr, 510 op/s 2024-05-09T00:09:56.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:55 smithi160 ceph-mon[46152]: pgmap v756: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.7 MiB/s wr, 510 op/s 2024-05-09T00:09:56.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 342936 29.31 MB/sec execute 471 sec latency 66.622 ms 2024-05-09T00:09:57.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 343241 29.29 MB/sec execute 472 sec latency 339.939 ms 2024-05-09T00:09:58.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:57 smithi012 ceph-mon[42503]: pgmap v757: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-05-09T00:09:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:57 smithi204 ceph-mon[47373]: pgmap v757: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-05-09T00:09:58.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:57 smithi160 ceph-mon[46152]: pgmap v757: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-05-09T00:09:58.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 343529 29.26 MB/sec execute 473 sec latency 129.698 ms 2024-05-09T00:09:59.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 343868 29.23 MB/sec execute 474 sec latency 150.391 ms 2024-05-09T00:09:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.397+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/628900829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9b4069380 msgr2=0x7fb9b4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.397+0000 7fb9b8d1a700 1 --2- 172.21.15.12:0/628900829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9b4069380 0x7fb9b4069760 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009990 tx=0x7fb9a4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.398+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/628900829 shutdown_connections 2024-05-09T00:09:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.398+0000 7fb9b8d1a700 1 --2- 172.21.15.12:0/628900829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b4106260 0x7fb9b410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.398+0000 7fb9b8d1a700 1 --2- 172.21.15.12:0/628900829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9b4069380 0x7fb9b4069760 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.398+0000 7fb9b8d1a700 1 --2- 172.21.15.12:0/628900829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9b4069d30 0x7fb9b4105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.398+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/628900829 >> 172.21.15.12:0/628900829 conn(0x7fb9b4076bc0 msgr2=0x7fb9b4076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.398+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/628900829 shutdown_connections 2024-05-09T00:09:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.398+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/628900829 wait complete. 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b8d1a700 1 Processor -- start 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b8d1a700 1 -- start start 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b8d1a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9b4069380 0x7fb9b4197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b8d1a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b4106260 0x7fb9b41978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b8d1a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9b419bc50 0x7fb9b419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b8d1a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b419c6d0 con 0x7fb9b4069380 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b8d1a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b419c840 con 0x7fb9b4106260 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b8d1a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b419ca00 con 0x7fb9b419bc50 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9b4069380 0x7fb9b4197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9b4069380 0x7fb9b4197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46656/0 (socket says 172.21.15.12:46656) 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b4106260 0x7fb9b41978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b259c700 1 -- 172.21.15.12:0/2177672668 learned_addr learned my addr 172.21.15.12:0/2177672668 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b2d9d700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9b419bc50 0x7fb9b419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b1d9b700 1 -- 172.21.15.12:0/2177672668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9b419bc50 msgr2=0x7fb9b419c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9b419bc50 0x7fb9b419c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b1d9b700 1 -- 172.21.15.12:0/2177672668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9b4069380 msgr2=0x7fb9b4197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.399+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9b4069380 0x7fb9b4197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.400+0000 7fb9b1d9b700 1 -- 172.21.15.12:0/2177672668 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a40095f0 con 0x7fb9b4106260 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.400+0000 7fb9b259c700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9b4069380 0x7fb9b4197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.400+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b4106260 0x7fb9b41978b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800e9c0 tx=0x7fb9a800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.400+0000 7fb9a37fe700 1 -- 172.21.15.12:0/2177672668 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800c9c0 con 0x7fb9b4106260 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.400+0000 7fb9a37fe700 1 -- 172.21.15.12:0/2177672668 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a8010430 con 0x7fb9b4106260 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.400+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/2177672668 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b41a3f60 con 0x7fb9b4106260 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.400+0000 7fb9a37fe700 1 -- 172.21.15.12:0/2177672668 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8018680 con 0x7fb9b4106260 2024-05-09T00:09:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.400+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/2177672668 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b41a4390 con 0x7fb9b4106260 2024-05-09T00:09:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.402+0000 7fb9a37fe700 1 -- 172.21.15.12:0/2177672668 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb9a800cb20 con 0x7fb9b4106260 2024-05-09T00:09:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.403+0000 7fb9a37fe700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb99c0a1290 0x7fb99c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.403+0000 7fb9b259c700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb99c0a1290 0x7fb99c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.403+0000 7fb9a37fe700 1 -- 172.21.15.12:0/2177672668 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb9a8014070 con 0x7fb9b4106260 2024-05-09T00:09:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.404+0000 7fb9b259c700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb99c0a1290 0x7fb99c0a3750 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4006010 tx=0x7fb9a400b260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.404+0000 7fb9c4ed4b80 1 Processor -- start 2024-05-09T00:09:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.404+0000 7fb9c4ed4b80 1 -- start start 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc8a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8ae60 0x5557bdc8e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdc8f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdbdd0f0 con 0x5557bdc8a540 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdbdaab0 con 0x5557bdc8ee40 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdbd63a0 con 0x5557bdc8ae60 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc8a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc8a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46668/0 (socket says 172.21.15.12:46668) 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b259c700 1 -- 172.21.15.12:0/2342293249 learned_addr learned my addr 172.21.15.12:0/2342293249 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/2342293249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8ae60 0x5557bdc8e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b2d9d700 1 --2- 172.21.15.12:0/2342293249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdc8f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b259c700 1 -- 172.21.15.12:0/2342293249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8ae60 msgr2=0x5557bdc8e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b259c700 1 --2- 172.21.15.12:0/2342293249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8ae60 0x5557bdc8e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b259c700 1 -- 172.21.15.12:0/2342293249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 msgr2=0x5557bdc8f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b259c700 1 --2- 172.21.15.12:0/2342293249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdc8f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b259c700 1 -- 172.21.15.12:0/2342293249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557bda57350 con 0x5557bdc8a540 2024-05-09T00:09:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9b259c700 1 --2- 172.21.15.12:0/2342293249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc8a920 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005680 tx=0x7fb9a4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ace76e4c68c433 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.405+0000 7fb9a0ff9700 1 -- 172.21.15.12:0/2342293249 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a402b070 con 0x5557bdc8a540 2024-05-09T00:09:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.406+0000 7fb9a0ff9700 1 -- 172.21.15.12:0/2342293249 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a401e040 con 0x5557bdc8a540 2024-05-09T00:09:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.406+0000 7fb9a0ff9700 1 -- 172.21.15.12:0/2342293249 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4004980 con 0x5557bdc8a540 2024-05-09T00:09:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.406+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2342293249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 msgr2=0x5557bdc8a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.406+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/2342293249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc8a920 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005680 tx=0x7fb9a4020000 comp rx=0 tx=0).stop 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.406+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2342293249 shutdown_connections 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.406+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/2342293249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdc8f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.406+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/2342293249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc8a920 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.406+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/2342293249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8ae60 0x5557bdc8e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.406+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2342293249 >> 172.21.15.12:0/2342293249 conn(0x5557bdbe41b0 msgr2=0x5557bdc80840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.407+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2342293249 shutdown_connections 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.407+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2342293249 wait complete. 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.407+0000 7fb9c4ed4b80 1 -- start start 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.407+0000 7fb9c4ed4b80 1 Processor -- start 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.407+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 0x5557bdc939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.407+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ae60 0x5557bdc93ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.407+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdc94420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdb72500 con 0x5557bdc8ae60 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdb721a0 con 0x5557bdc8ee40 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bda59640 con 0x5557bdc8a540 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9c4ed4b80 10 client.0 _ll_register_callbacks cb 0x5557bdb7bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ae60 0x5557bdc93ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 0x5557bdc939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 0x5557bdc939a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47728/0 (socket says 172.21.15.12:47728) 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b259c700 1 -- 172.21.15.12:0/1758961395 learned_addr learned my addr 172.21.15.12:0/1758961395 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b2d9d700 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdc94420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b1d9b700 1 -- 172.21.15.12:0/1758961395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 msgr2=0x5557bdc939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 0x5557bdc939a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b1d9b700 1 -- 172.21.15.12:0/1758961395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 msgr2=0x5557bdc94420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdc94420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b1d9b700 1 -- 172.21.15.12:0/1758961395 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b410e650 con 0x5557bdc8ae60 2024-05-09T00:09:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ae60 0x5557bdc93ee0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80ca550 tx=0x7fb9a8099b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d949f33a26daadc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb999ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:09:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb999ffb700 1 -- 172.21.15.12:0/1758961395 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a809dd40 con 0x5557bdc8ae60 2024-05-09T00:09:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb999ffb700 1 -- 172.21.15.12:0/1758961395 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb9a80a2390 con 0x5557bdc8ae60 2024-05-09T00:09:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.408+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/1758961395 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557bdb2f210 con 0x5557bdc8ae60 2024-05-09T00:09:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb999ffb700 1 -- 172.21.15.12:0/1758961395 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a8010860 con 0x5557bdc8ae60 2024-05-09T00:09:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb9c4ed4b80 10 client.21825 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/1758961395 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x5557bda03b00 con 0x5557bdc8ae60 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb999ffb700 1 -- 172.21.15.12:0/1758961395 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a8063080 con 0x5557bdc8ae60 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb999ffb700 1 -- 172.21.15.12:0/1758961395 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb9a80a1f00 con 0x5557bdc8ae60 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb9c4ed4b80 10 client.21825 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb9c4ed4b80 10 client.21825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb9c4ed4b80 20 client.21825 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5557bdca4020 0x5557bdca6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb9c4ed4b80 4 client.21825 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/1758961395 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557bdc557b0 con 0x5557bdca4020 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.409+0000 7fb9b259c700 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5557bdca4020 0x5557bdca6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.410+0000 7fb9b259c700 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5557bdca4020 0x5557bdca6440 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986a0beed2c527f8 server_cookie=f46cf32767ef4cc6 in_seq=0 out_seq=0 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.410+0000 7fb999ffb700 10 client.21825 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb999ffb700 1 -- 172.21.15.12:0/1758961395 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557bdc557b0 con 0x5557bdca4020 2024-05-09T00:09:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb999ffb700 10 client.21825 handle_command_reply: tid=0 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 client.21825 shutdown 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/1758961395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ae60 msgr2=0x5557bdc93ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ae60 0x5557bdc93ee0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80ca550 tx=0x7fb9a8099b50 comp rx=0 tx=0).stop 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/1758961395 shutdown_connections 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5557bdca4020 0x5557bdca6440 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdc94420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 0x5557bdc939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/1758961395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ae60 0x5557bdc93ee0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/1758961395 >> 172.21.15.12:0/1758961395 conn(0x5557bdbe41b0 msgr2=0x5557bdc84f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb999ffb700 0 client.21825 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/1758961395 shutdown_connections 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.411+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/1758961395 wait complete. 2024-05-09T00:09:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.412+0000 7fb9c4ed4b80 20 client.21825 trim_cache size 0 max 16384 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9c4ed4b80 1 Processor -- start 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9c4ed4b80 1 -- start start 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc80910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8ae60 0x5557bdc94240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdcaada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdca6740 con 0x5557bdc8a540 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdca0970 con 0x5557bdc8ee40 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdc8a920 con 0x5557bdc8ae60 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9b259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc80910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9b259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc80910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46694/0 (socket says 172.21.15.12:46694) 2024-05-09T00:09:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9b259c700 1 -- 172.21.15.12:0/576311066 learned_addr learned my addr 172.21.15.12:0/576311066 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9b2d9d700 1 --2- 172.21.15.12:0/576311066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdcaada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.430+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/576311066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8ae60 0x5557bdc94240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.431+0000 7fb9b259c700 1 -- 172.21.15.12:0/576311066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8ae60 msgr2=0x5557bdc94240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.431+0000 7fb9b259c700 1 --2- 172.21.15.12:0/576311066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8ae60 0x5557bdc94240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.431+0000 7fb9b259c700 1 -- 172.21.15.12:0/576311066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 msgr2=0x5557bdcaada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.431+0000 7fb9b259c700 1 --2- 172.21.15.12:0/576311066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdcaada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.431+0000 7fb9b259c700 1 -- 172.21.15.12:0/576311066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557bdb9c500 con 0x5557bdc8a540 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.431+0000 7fb9b2d9d700 1 --2- 172.21.15.12:0/576311066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdcaada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.431+0000 7fb9b259c700 1 --2- 172.21.15.12:0/576311066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc80910 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400ff70 tx=0x7fb9a4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8207dabd9b368433 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.431+0000 7fb9a17fa700 1 -- 172.21.15.12:0/576311066 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a402b040 con 0x5557bdc8a540 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.431+0000 7fb9a17fa700 1 -- 172.21.15.12:0/576311066 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a400fa20 con 0x5557bdc8a540 2024-05-09T00:09:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.431+0000 7fb9a17fa700 1 -- 172.21.15.12:0/576311066 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4004500 con 0x5557bdc8a540 2024-05-09T00:09:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.432+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/576311066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 msgr2=0x5557bdc80910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.432+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/576311066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc80910 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400ff70 tx=0x7fb9a4020000 comp rx=0 tx=0).stop 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.432+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/576311066 shutdown_connections 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.432+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/576311066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ee40 0x5557bdcaada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.432+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/576311066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8a540 0x5557bdc80910 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.432+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/576311066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8ae60 0x5557bdc94240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.432+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/576311066 >> 172.21.15.12:0/576311066 conn(0x5557bdbe41b0 msgr2=0x5557bdca6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.432+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/576311066 shutdown_connections 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.432+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/576311066 wait complete. 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.432+0000 7fb9c4ed4b80 1 -- start start 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9c4ed4b80 1 Processor -- start 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 0x5557bdc9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ae60 0x5557bdc9b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9c4ed4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ee40 0x5557bdc99030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdc996d0 con 0x5557bdc8ee40 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdc99840 con 0x5557bdc8ae60 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9c4ed4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bdc99a00 con 0x5557bdc8a540 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9c4ed4b80 10 client.0 _ll_register_callbacks cb 0x5557bdb7bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9b1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ae60 0x5557bdc9b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9b1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ae60 0x5557bdc9b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:52928/0 (socket says 172.21.15.12:52928) 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9b2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ee40 0x5557bdc99030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9b1d9b700 1 -- 172.21.15.12:0/2000308972 learned_addr learned my addr 172.21.15.12:0/2000308972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9b259c700 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 0x5557bdc9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9b1d9b700 1 -- 172.21.15.12:0/2000308972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 msgr2=0x5557bdc9af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 0x5557bdc9af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9b1d9b700 1 -- 172.21.15.12:0/2000308972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ee40 msgr2=0x5557bdc99030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ee40 0x5557bdc99030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.433+0000 7fb9b1d9b700 1 -- 172.21.15.12:0/2000308972 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a4033040 con 0x5557bdc8ae60 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.434+0000 7fb9b1d9b700 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ae60 0x5557bdc9b4c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80ca600 tx=0x7fb9a8099fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b13c932f3ad65b93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.434+0000 7fb999ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.434+0000 7fb999ffb700 1 -- 172.21.15.12:0/2000308972 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80996b0 con 0x5557bdc8ae60 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.434+0000 7fb999ffb700 1 -- 172.21.15.12:0/2000308972 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb9a80a2390 con 0x5557bdc8ae60 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.434+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2000308972 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9a401e040 con 0x5557bdc8ae60 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.434+0000 7fb999ffb700 1 -- 172.21.15.12:0/2000308972 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a809dd40 con 0x5557bdc8ae60 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.434+0000 7fb9c4ed4b80 10 client.31253 fetch_fsmap learned FSMap version 80 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.434+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2000308972 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x5557bda5bc70 con 0x5557bdc8ae60 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.435+0000 7fb999ffb700 1 -- 172.21.15.12:0/2000308972 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a401e040 con 0x5557bdc8ae60 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.435+0000 7fb999ffb700 1 -- 172.21.15.12:0/2000308972 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb9a8098e70 con 0x5557bdc8ae60 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.435+0000 7fb9c4ed4b80 10 client.31253 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.435+0000 7fb9c4ed4b80 10 client.31253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.435+0000 7fb9c4ed4b80 20 client.31253 populate_metadata read hostname 'smithi012' 2024-05-09T00:09:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.435+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5557bdcfa430 0x5557bdcfc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.435+0000 7fb9c4ed4b80 4 client.31253 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:09:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.435+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2000308972 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5557bdbcee60 con 0x5557bdcfa430 2024-05-09T00:09:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.435+0000 7fb9b259c700 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5557bdcfa430 0x5557bdcfc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.436+0000 7fb999ffb700 10 client.31253 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.436+0000 7fb9b259c700 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5557bdcfa430 0x5557bdcfc850 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f69ef3e4a0d8c09 server_cookie=1197d15bc604ec4b in_seq=0 out_seq=0 2024-05-09T00:09:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.875+0000 7fb999ffb700 1 -- 172.21.15.12:0/2000308972 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5557bdbcee60 con 0x5557bdcfa430 2024-05-09T00:09:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.875+0000 7fb999ffb700 10 client.31253 handle_command_reply: tid=0 2024-05-09T00:09:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.875+0000 7fb9c4ed4b80 1 client.31253 shutdown 2024-05-09T00:09:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.875+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2000308972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ae60 msgr2=0x5557bdc9b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.875+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ae60 0x5557bdc9b4c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80ca600 tx=0x7fb9a8099fd0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.875+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2000308972 shutdown_connections 2024-05-09T00:09:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.876+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bdc8ee40 0x5557bdc99030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.876+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557bdc8a540 0x5557bdc9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.876+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5557bdcfa430 0x5557bdcfc850 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.876+0000 7fb9c4ed4b80 1 --2- 172.21.15.12:0/2000308972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5557bdc8ae60 0x5557bdc9b4c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.876+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2000308972 >> 172.21.15.12:0/2000308972 conn(0x5557bdbe41b0 msgr2=0x5557bdcaada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.876+0000 7fb999ffb700 0 client.31253 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:09:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.876+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2000308972 shutdown_connections 2024-05-09T00:09:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.876+0000 7fb9c4ed4b80 1 -- 172.21.15.12:0/2000308972 wait complete. 2024-05-09T00:09:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.876+0000 7fb9c4ed4b80 20 client.31253 trim_cache size 0 max 16384 2024-05-09T00:09:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:09:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:09:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:09:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:09:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/2177672668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb99c0a1290 msgr2=0x7fb99c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb99c0a1290 0x7fb99c0a3750 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4006010 tx=0x7fb9a400b260 comp rx=0 tx=0).stop 2024-05-09T00:09:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/2177672668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b4106260 msgr2=0x7fb9b41978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b4106260 0x7fb9b41978b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800e9c0 tx=0x7fb9a800ecd0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/2177672668 shutdown_connections 2024-05-09T00:09:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb99c0a1290 0x7fb99c0a3750 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9b419bc50 0x7fb9b419c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9b4069380 0x7fb9b4197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 --2- 172.21.15.12:0/2177672668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b4106260 0x7fb9b41978b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/2177672668 >> 172.21.15.12:0/2177672668 conn(0x7fb9b4076bc0 msgr2=0x7fb9b40fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/2177672668 shutdown_connections 2024-05-09T00:09:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:09:59.877+0000 7fb9b8d1a700 1 -- 172.21.15.12:0/2177672668 wait complete. 2024-05-09T00:10:00.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:09:59 smithi012 ceph-mon[42503]: pgmap v758: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.1 MiB/s wr, 445 op/s 2024-05-09T00:10:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:09:59 smithi204 ceph-mon[47373]: pgmap v758: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.1 MiB/s wr, 445 op/s 2024-05-09T00:10:00.271 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:09:59 smithi160 ceph-mon[46152]: pgmap v758: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.1 MiB/s wr, 445 op/s 2024-05-09T00:10:00.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 344169 29.18 MB/sec execute 475 sec latency 346.897 ms 2024-05-09T00:10:01.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:00 smithi012 ceph-mon[42503]: overall HEALTH_OK 2024-05-09T00:10:01.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:00 smithi204 ceph-mon[47373]: overall HEALTH_OK 2024-05-09T00:10:01.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:00 smithi160 ceph-mon[46152]: overall HEALTH_OK 2024-05-09T00:10:01.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 344716 29.19 MB/sec execute 476 sec latency 152.475 ms 2024-05-09T00:10:02.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:01 smithi012 ceph-mon[42503]: pgmap v759: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.3 MiB/s wr, 446 op/s 2024-05-09T00:10:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:01 smithi204 ceph-mon[47373]: pgmap v759: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.3 MiB/s wr, 446 op/s 2024-05-09T00:10:02.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:01 smithi160 ceph-mon[46152]: pgmap v759: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.3 MiB/s wr, 446 op/s 2024-05-09T00:10:02.300 INFO:tasks.workunit.client.0.smithi012.stdout: 10 344983 29.14 MB/sec execute 477 sec latency 387.545 ms 2024-05-09T00:10:03.301 INFO:tasks.workunit.client.0.smithi012.stdout: 10 345257 29.10 MB/sec execute 478 sec latency 404.574 ms 2024-05-09T00:10:03.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:03 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:10:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:03 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:10:03.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:03 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:10:04.301 INFO:tasks.workunit.client.0.smithi012.stdout: 10 345538 29.08 MB/sec execute 479 sec latency 318.180 ms 2024-05-09T00:10:04.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:04 smithi012 ceph-mon[42503]: pgmap v760: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 718 op/s 2024-05-09T00:10:04.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:04 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:10:04.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:04 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:10:04.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:04 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:10:04.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:04 smithi204 ceph-mon[47373]: pgmap v760: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 718 op/s 2024-05-09T00:10:04.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:04 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:10:04.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:04 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:10:04.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:04 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:10:04.771 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:04 smithi160 ceph-mon[46152]: pgmap v760: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 718 op/s 2024-05-09T00:10:04.771 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:04 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:10:04.771 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:04 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:10:04.771 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:04 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:10:05.301 INFO:tasks.workunit.client.0.smithi012.stdout: 10 345723 29.04 MB/sec execute 480 sec latency 630.579 ms 2024-05-09T00:10:05.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:05 smithi012 ceph-mon[42503]: osdmap e133: 12 total, 12 up, 12 in 2024-05-09T00:10:05.768 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:05 smithi204 ceph-mon[47373]: osdmap e133: 12 total, 12 up, 12 in 2024-05-09T00:10:05.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:05 smithi160 ceph-mon[46152]: osdmap e133: 12 total, 12 up, 12 in 2024-05-09T00:10:06.302 INFO:tasks.workunit.client.0.smithi012.stdout: 10 345909 29.02 MB/sec execute 481 sec latency 688.991 ms 2024-05-09T00:10:06.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:06 smithi204 ceph-mon[47373]: pgmap v762: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 641 op/s 2024-05-09T00:10:06.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:06 smithi160 ceph-mon[46152]: pgmap v762: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 641 op/s 2024-05-09T00:10:06.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:06 smithi012 ceph-mon[42503]: pgmap v762: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 641 op/s 2024-05-09T00:10:07.302 INFO:tasks.workunit.client.0.smithi012.stdout: 10 346055 28.98 MB/sec execute 482 sec latency 830.497 ms 2024-05-09T00:10:08.177 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:07 smithi012 ceph-mon[42503]: pgmap v763: 97 pgs: 29 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 51 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.6 MiB/s wr, 692 op/s 2024-05-09T00:10:08.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:07 smithi160 ceph-mon[46152]: pgmap v763: 97 pgs: 29 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 51 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.6 MiB/s wr, 692 op/s 2024-05-09T00:10:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:07 smithi204 ceph-mon[47373]: pgmap v763: 97 pgs: 29 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 51 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.6 MiB/s wr, 692 op/s 2024-05-09T00:10:08.302 INFO:tasks.workunit.client.0.smithi012.stdout: 10 346195 28.94 MB/sec execute 483 sec latency 918.900 ms 2024-05-09T00:10:09.303 INFO:tasks.workunit.client.0.smithi012.stdout: 10 346381 28.90 MB/sec execute 484 sec latency 622.431 ms 2024-05-09T00:10:10.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:09 smithi012 ceph-mon[42503]: pgmap v764: 97 pgs: 35 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 42 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.4 MiB/s wr, 678 op/s 2024-05-09T00:10:10.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:09 smithi204 ceph-mon[47373]: pgmap v764: 97 pgs: 35 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 42 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.4 MiB/s wr, 678 op/s 2024-05-09T00:10:10.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:09 smithi160 ceph-mon[46152]: pgmap v764: 97 pgs: 35 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 42 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.4 MiB/s wr, 678 op/s 2024-05-09T00:10:10.303 INFO:tasks.workunit.client.0.smithi012.stdout: 10 346758 28.88 MB/sec execute 485 sec latency 117.057 ms 2024-05-09T00:10:11.303 INFO:tasks.workunit.client.0.smithi012.stdout: 10 347184 28.88 MB/sec execute 486 sec latency 110.836 ms 2024-05-09T00:10:11.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:10:11.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.895+0000 7fca54459700 1 -- 172.21.15.12:0/3559849986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c105160 msgr2=0x7fca4c10cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:11.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.895+0000 7fca54459700 1 --2- 172.21.15.12:0/3559849986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c105160 0x7fca4c10cce0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fca4400b0a0 tx=0x7fca4400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.895+0000 7fca54459700 1 -- 172.21.15.12:0/3559849986 shutdown_connections 2024-05-09T00:10:11.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.895+0000 7fca54459700 1 --2- 172.21.15.12:0/3559849986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c105160 0x7fca4c10cce0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.895+0000 7fca54459700 1 --2- 172.21.15.12:0/3559849986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca4c100060 0x7fca4c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.895+0000 7fca54459700 1 --2- 172.21.15.12:0/3559849986 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c100a10 0x7fca4c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.895+0000 7fca54459700 1 -- 172.21.15.12:0/3559849986 >> 172.21.15.12:0/3559849986 conn(0x7fca4c076f40 msgr2=0x7fca4c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.895+0000 7fca54459700 1 -- 172.21.15.12:0/3559849986 shutdown_connections 2024-05-09T00:10:11.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.895+0000 7fca54459700 1 -- 172.21.15.12:0/3559849986 wait complete. 2024-05-09T00:10:11.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca54459700 1 Processor -- start 2024-05-09T00:10:11.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca54459700 1 -- start start 2024-05-09T00:10:11.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca54459700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca4c100060 0x7fca4c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca54459700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c105160 0x7fca4c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca54459700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c198df0 0x7fca4c1a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca54459700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c1a3ad0 con 0x7fca4c105160 2024-05-09T00:10:11.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca54459700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c1a3c40 con 0x7fca4c198df0 2024-05-09T00:10:11.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca54459700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c1a3e00 con 0x7fca4c100060 2024-05-09T00:10:11.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca519f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c105160 0x7fca4c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca521f5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca4c100060 0x7fca4c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca519f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c105160 0x7fca4c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47436/0 (socket says 172.21.15.12:47436) 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca519f4700 1 -- 172.21.15.12:0/3420236966 learned_addr learned my addr 172.21.15.12:0/3420236966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.896+0000 7fca529f6700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c198df0 0x7fca4c1a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca521f5700 1 -- 172.21.15.12:0/3420236966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c198df0 msgr2=0x7fca4c1a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca521f5700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c198df0 0x7fca4c1a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca521f5700 1 -- 172.21.15.12:0/3420236966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c105160 msgr2=0x7fca4c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca521f5700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c105160 0x7fca4c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca521f5700 1 -- 172.21.15.12:0/3420236966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44009d00 con 0x7fca4c100060 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca519f4700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c105160 0x7fca4c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca521f5700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca4c100060 0x7fca4c198370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fca4800e9f0 tx=0x7fca4800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca437fe700 1 -- 172.21.15.12:0/3420236966 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4800c9c0 con 0x7fca4c100060 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca437fe700 1 -- 172.21.15.12:0/3420236966 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca48010430 con 0x7fca4c100060 2024-05-09T00:10:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca54459700 1 -- 172.21.15.12:0/3420236966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c1a40e0 con 0x7fca4c100060 2024-05-09T00:10:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca437fe700 1 -- 172.21.15.12:0/3420236966 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48018680 con 0x7fca4c100060 2024-05-09T00:10:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.897+0000 7fca54459700 1 -- 172.21.15.12:0/3420236966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c1a45f0 con 0x7fca4c100060 2024-05-09T00:10:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.901+0000 7fca437fe700 1 -- 172.21.15.12:0/3420236966 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fca48010860 con 0x7fca4c100060 2024-05-09T00:10:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.901+0000 7fca437fe700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fca380a1290 0x7fca380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.901+0000 7fca437fe700 1 -- 172.21.15.12:0/3420236966 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fca48014070 con 0x7fca4c100060 2024-05-09T00:10:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.901+0000 7fca519f4700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fca380a1290 0x7fca380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.902+0000 7fca60613b80 1 Processor -- start 2024-05-09T00:10:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.902+0000 7fca519f4700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fca380a1290 0x7fca380a3750 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0ff930 tx=0x7fca3c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.902+0000 7fca60613b80 1 -- start start 2024-05-09T00:10:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.902+0000 7fca60613b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4ace00210 0x55c4ace005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.902+0000 7fca60613b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdfa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.902+0000 7fca60613b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4acdfaca0 0x55c4acdfb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.902+0000 7fca60613b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4acd4d150 con 0x55c4ace00b30 2024-05-09T00:10:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.902+0000 7fca60613b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4acd4ab10 con 0x55c4acdfaca0 2024-05-09T00:10:11.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.902+0000 7fca60613b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4acd46400 con 0x55c4ace00210 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca519f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdfa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca519f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdfa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47448/0 (socket says 172.21.15.12:47448) 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca519f4700 1 -- 172.21.15.12:0/3886897666 learned_addr learned my addr 172.21.15.12:0/3886897666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca521f5700 1 --2- 172.21.15.12:0/3886897666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4ace00210 0x55c4ace005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca519f4700 1 -- 172.21.15.12:0/3886897666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4ace00210 msgr2=0x55c4ace005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca519f4700 1 --2- 172.21.15.12:0/3886897666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4ace00210 0x55c4ace005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca519f4700 1 -- 172.21.15.12:0/3886897666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4acdfaca0 msgr2=0x55c4acdfb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca519f4700 1 --2- 172.21.15.12:0/3886897666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4acdfaca0 0x55c4acdfb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca519f4700 1 -- 172.21.15.12:0/3886897666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4acbc73b0 con 0x55c4ace00b30 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca519f4700 1 --2- 172.21.15.12:0/3886897666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdfa760 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fca3c004380 tx=0x7fca3c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d958c542409d7c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca40da4700 1 -- 172.21.15.12:0/3886897666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c003a90 con 0x55c4ace00b30 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca40da4700 1 -- 172.21.15.12:0/3886897666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3c003bf0 con 0x55c4ace00b30 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.903+0000 7fca40da4700 1 -- 172.21.15.12:0/3886897666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00e610 con 0x55c4ace00b30 2024-05-09T00:10:11.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.904+0000 7fca60613b80 1 -- 172.21.15.12:0/3886897666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 msgr2=0x55c4acdfa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.904+0000 7fca60613b80 1 --2- 172.21.15.12:0/3886897666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdfa760 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fca3c004380 tx=0x7fca3c014000 comp rx=0 tx=0).stop 2024-05-09T00:10:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.904+0000 7fca60613b80 1 -- 172.21.15.12:0/3886897666 shutdown_connections 2024-05-09T00:10:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.904+0000 7fca60613b80 1 --2- 172.21.15.12:0/3886897666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4acdfaca0 0x55c4acdfb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.904+0000 7fca60613b80 1 --2- 172.21.15.12:0/3886897666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4ace00210 0x55c4ace005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.904+0000 7fca60613b80 1 --2- 172.21.15.12:0/3886897666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdfa760 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.904+0000 7fca60613b80 1 -- 172.21.15.12:0/3886897666 >> 172.21.15.12:0/3886897666 conn(0x55c4acd54210 msgr2=0x55c4acdf0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.904+0000 7fca60613b80 1 -- 172.21.15.12:0/3886897666 shutdown_connections 2024-05-09T00:10:11.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.904+0000 7fca60613b80 1 -- 172.21.15.12:0/3886897666 wait complete. 2024-05-09T00:10:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.905+0000 7fca60613b80 1 -- start start 2024-05-09T00:10:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.905+0000 7fca60613b80 1 Processor -- start 2024-05-09T00:10:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.905+0000 7fca60613b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4acdfaca0 0x55c4ace06bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.905+0000 7fca60613b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4ace03ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.905+0000 7fca60613b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00b30 0x55c4ace045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.905+0000 7fca60613b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4acce2560 con 0x55c4ace00210 2024-05-09T00:10:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.905+0000 7fca60613b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4acce2200 con 0x55c4ace00b30 2024-05-09T00:10:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.905+0000 7fca60613b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4acbc96a0 con 0x55c4acdfaca0 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.905+0000 7fca60613b80 10 client.0 _ll_register_callbacks cb 0x55c4accebb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.905+0000 7fca519f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4ace03ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca519f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4ace03ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47454/0 (socket says 172.21.15.12:47454) 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca519f4700 1 -- 172.21.15.12:0/2555051966 learned_addr learned my addr 172.21.15.12:0/2555051966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca521f5700 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4acdfaca0 0x55c4ace06bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca529f6700 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00b30 0x55c4ace045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca519f4700 1 -- 172.21.15.12:0/2555051966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4acdfaca0 msgr2=0x55c4ace06bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca519f4700 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4acdfaca0 0x55c4ace06bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca519f4700 1 -- 172.21.15.12:0/2555051966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00b30 msgr2=0x55c4ace045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca519f4700 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00b30 0x55c4ace045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca519f4700 1 -- 172.21.15.12:0/2555051966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c017070 con 0x55c4ace00210 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca521f5700 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4acdfaca0 0x55c4ace06bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca529f6700 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00b30 0x55c4ace045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca519f4700 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4ace03ea0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fca3c003f90 tx=0x7fca3c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6854ebfb9d87009 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca35ffb700 1 -- 172.21.15.12:0/2555051966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c02b070 con 0x55c4ace00210 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca35ffb700 1 -- 172.21.15.12:0/2555051966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fca3c005260 con 0x55c4ace00210 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca60613b80 1 -- 172.21.15.12:0/2555051966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4acc9f270 con 0x55c4ace00210 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca35ffb700 1 -- 172.21.15.12:0/2555051966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca3c005420 con 0x55c4ace00210 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca60613b80 10 client.21843 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.906+0000 7fca60613b80 1 -- 172.21.15.12:0/2555051966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55c4acb73b60 con 0x55c4ace00210 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.907+0000 7fca35ffb700 1 -- 172.21.15.12:0/2555051966 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3c03c080 con 0x55c4ace00210 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.907+0000 7fca35ffb700 1 -- 172.21.15.12:0/2555051966 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca3c024030 con 0x55c4ace00210 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.907+0000 7fca60613b80 10 client.21843 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.907+0000 7fca60613b80 10 client.21843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.907+0000 7fca60613b80 20 client.21843 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.907+0000 7fca60613b80 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c4ace14560 0x55c4ace16980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.907+0000 7fca60613b80 4 client.21843 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:11.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.907+0000 7fca60613b80 1 -- 172.21.15.12:0/2555051966 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4acdc5810 con 0x55c4ace14560 2024-05-09T00:10:11.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.907+0000 7fca521f5700 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c4ace14560 0x55c4ace16980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.908+0000 7fca521f5700 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c4ace14560 0x55c4ace16980 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6791132b2daa5f server_cookie=10c5393c454cd3d8 in_seq=0 out_seq=0 2024-05-09T00:10:11.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.908+0000 7fca35ffb700 10 client.21843 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca35ffb700 1 -- 172.21.15.12:0/2555051966 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c4acdc5810 con 0x55c4ace14560 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca35ffb700 10 client.21843 handle_command_reply: tid=0 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 client.21843 shutdown 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 -- 172.21.15.12:0/2555051966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 msgr2=0x55c4ace03ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4ace03ea0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fca3c003f90 tx=0x7fca3c005d40 comp rx=0 tx=0).stop 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 -- 172.21.15.12:0/2555051966 shutdown_connections 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c4ace14560 0x55c4ace16980 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00b30 0x55c4ace045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4acdfaca0 0x55c4ace06bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 --2- 172.21.15.12:0/2555051966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4ace03ea0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 -- 172.21.15.12:0/2555051966 >> 172.21.15.12:0/2555051966 conn(0x55c4acd54210 msgr2=0x55c4acdf05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca35ffb700 0 client.21843 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 -- 172.21.15.12:0/2555051966 shutdown_connections 2024-05-09T00:10:11.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 1 -- 172.21.15.12:0/2555051966 wait complete. 2024-05-09T00:10:11.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.910+0000 7fca60613b80 20 client.21843 trim_cache size 0 max 16384 2024-05-09T00:10:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca60613b80 1 Processor -- start 2024-05-09T00:10:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca60613b80 1 -- start start 2024-05-09T00:10:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca60613b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4acdfaca0 0x55c4ace045a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca60613b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00210 0x55c4acdfb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca60613b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdf0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca60613b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ace023f0 con 0x55c4ace00b30 2024-05-09T00:10:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca60613b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4acdfb5c0 con 0x55c4ace00210 2024-05-09T00:10:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca60613b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4acdf1260 con 0x55c4acdfaca0 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca529f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdf0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca529f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdf0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47464/0 (socket says 172.21.15.12:47464) 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca529f6700 1 -- 172.21.15.12:0/1418205189 learned_addr learned my addr 172.21.15.12:0/1418205189 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca519f4700 1 --2- 172.21.15.12:0/1418205189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00210 0x55c4acdfb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca529f6700 1 -- 172.21.15.12:0/1418205189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4acdfaca0 msgr2=0x55c4ace045a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca529f6700 1 --2- 172.21.15.12:0/1418205189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4acdfaca0 0x55c4ace045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca529f6700 1 -- 172.21.15.12:0/1418205189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00210 msgr2=0x55c4acdfb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca529f6700 1 --2- 172.21.15.12:0/1418205189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00210 0x55c4acdfb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.932+0000 7fca529f6700 1 -- 172.21.15.12:0/1418205189 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4acd0c560 con 0x55c4ace00b30 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.933+0000 7fca529f6700 1 --2- 172.21.15.12:0/1418205189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdf0d20 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fca4400b0a0 tx=0x7fca440093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c238bce2ad11d8af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.933+0000 7fca415a5700 1 -- 172.21.15.12:0/1418205189 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4400e040 con 0x55c4ace00b30 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.933+0000 7fca415a5700 1 -- 172.21.15.12:0/1418205189 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca440094a0 con 0x55c4ace00b30 2024-05-09T00:10:11.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.933+0000 7fca415a5700 1 -- 172.21.15.12:0/1418205189 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44007930 con 0x55c4ace00b30 2024-05-09T00:10:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.934+0000 7fca60613b80 1 -- 172.21.15.12:0/1418205189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 msgr2=0x55c4acdf0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.934+0000 7fca60613b80 1 --2- 172.21.15.12:0/1418205189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdf0d20 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fca4400b0a0 tx=0x7fca440093b0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.934+0000 7fca60613b80 1 -- 172.21.15.12:0/1418205189 shutdown_connections 2024-05-09T00:10:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.934+0000 7fca60613b80 1 --2- 172.21.15.12:0/1418205189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00b30 0x55c4acdf0d20 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.934+0000 7fca60613b80 1 --2- 172.21.15.12:0/1418205189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4acdfaca0 0x55c4ace045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.934+0000 7fca60613b80 1 --2- 172.21.15.12:0/1418205189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4ace00210 0x55c4acdfb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.934+0000 7fca60613b80 1 -- 172.21.15.12:0/1418205189 >> 172.21.15.12:0/1418205189 conn(0x55c4acd54210 msgr2=0x55c4acded210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 1 -- 172.21.15.12:0/1418205189 shutdown_connections 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 1 -- 172.21.15.12:0/1418205189 wait complete. 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 1 -- start start 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 1 Processor -- start 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4acdfaca0 0x55c4ace0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4acdfc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4ace00b30 0x55c4acdfc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ace0be90 con 0x55c4ace00210 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4acdfcd50 con 0x55c4acdfaca0 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4acdfcf10 con 0x55c4ace00b30 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.935+0000 7fca60613b80 10 client.0 _ll_register_callbacks cb 0x55c4accebb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca519f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4acdfc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca519f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4acdfc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47472/0 (socket says 172.21.15.12:47472) 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca519f4700 1 -- 172.21.15.12:0/2613401210 learned_addr learned my addr 172.21.15.12:0/2613401210 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca519f4700 1 -- 172.21.15.12:0/2613401210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4ace00b30 msgr2=0x55c4acdfc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca519f4700 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4ace00b30 0x55c4acdfc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca519f4700 1 -- 172.21.15.12:0/2613401210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4acdfaca0 msgr2=0x55c4ace0b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca519f4700 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4acdfaca0 0x55c4ace0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca519f4700 1 -- 172.21.15.12:0/2613401210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c10d550 con 0x55c4ace00210 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca519f4700 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4acdfc2d0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x55c4acde91c0 tx=0x7fca3c0307a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21ea3bfeaa9fd85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.936+0000 7fca35ffb700 1 -- 172.21.15.12:0/2613401210 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c02b070 con 0x55c4ace00210 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.937+0000 7fca35ffb700 1 -- 172.21.15.12:0/2613401210 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fca3c030c90 con 0x55c4ace00210 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.937+0000 7fca60613b80 1 -- 172.21.15.12:0/2613401210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca3c003bf0 con 0x55c4ace00210 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.937+0000 7fca35ffb700 1 -- 172.21.15.12:0/2613401210 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca3c030e50 con 0x55c4ace00210 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.937+0000 7fca60613b80 10 client.21849 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.937+0000 7fca60613b80 1 -- 172.21.15.12:0/2613401210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55c4acbcbcd0 con 0x55c4ace00210 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.937+0000 7fca35ffb700 1 -- 172.21.15.12:0/2613401210 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3c003bf0 con 0x55c4ace00210 2024-05-09T00:10:11.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.937+0000 7fca35ffb700 1 -- 172.21.15.12:0/2613401210 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca3c003680 con 0x55c4ace00210 2024-05-09T00:10:11.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.938+0000 7fca60613b80 10 client.21849 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:11.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.938+0000 7fca60613b80 10 client.21849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:11.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.938+0000 7fca60613b80 20 client.21849 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.938+0000 7fca60613b80 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c4ace6b970 0x55c4ace6dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.938+0000 7fca60613b80 4 client.21849 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:10:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.938+0000 7fca60613b80 1 -- 172.21.15.12:0/2613401210 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c4acd3eec0 con 0x55c4ace6b970 2024-05-09T00:10:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.938+0000 7fca521f5700 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c4ace6b970 0x55c4ace6dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.939+0000 7fca35ffb700 10 client.21849 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.939+0000 7fca521f5700 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c4ace6b970 0x55c4ace6dd90 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9dfebc570639727 server_cookie=90e97c3b5e12bbe3 in_seq=0 out_seq=0 2024-05-09T00:10:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca35ffb700 1 -- 172.21.15.12:0/2613401210 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c4acd3eec0 con 0x55c4ace6b970 2024-05-09T00:10:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca35ffb700 10 client.21849 handle_command_reply: tid=0 2024-05-09T00:10:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 client.21849 shutdown 2024-05-09T00:10:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 -- 172.21.15.12:0/2613401210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 msgr2=0x55c4acdfc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4acdfc2d0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x55c4acde91c0 tx=0x7fca3c0307a0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 -- 172.21.15.12:0/2613401210 shutdown_connections 2024-05-09T00:10:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4ace00b30 0x55c4acdfc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c4ace6b970 0x55c4ace6dd90 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4acdfaca0 0x55c4ace0b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 --2- 172.21.15.12:0/2613401210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c4ace00210 0x55c4acdfc2d0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca35ffb700 0 client.21849 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 -- 172.21.15.12:0/2613401210 >> 172.21.15.12:0/2613401210 conn(0x55c4acd54210 msgr2=0x55c4ace02770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 -- 172.21.15.12:0/2613401210 shutdown_connections 2024-05-09T00:10:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 1 -- 172.21.15.12:0/2613401210 wait complete. 2024-05-09T00:10:11.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.944+0000 7fca60613b80 20 client.21849 trim_cache size 0 max 16384 2024-05-09T00:10:11.942 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:10:11.942 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:10:11.942 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:10:11.943 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:10:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 -- 172.21.15.12:0/3420236966 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fca380a1290 msgr2=0x7fca380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fca380a1290 0x7fca380a3750 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0ff930 tx=0x7fca3c009260 comp rx=0 tx=0).stop 2024-05-09T00:10:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 -- 172.21.15.12:0/3420236966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca4c100060 msgr2=0x7fca4c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca4c100060 0x7fca4c198370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fca4800e9f0 tx=0x7fca4800ed00 comp rx=0 tx=0).stop 2024-05-09T00:10:11.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 -- 172.21.15.12:0/3420236966 shutdown_connections 2024-05-09T00:10:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fca380a1290 0x7fca380a3750 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca4c198df0 0x7fca4c1a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca4c100060 0x7fca4c198370 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 --2- 172.21.15.12:0/3420236966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c105160 0x7fca4c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 -- 172.21.15.12:0/3420236966 >> 172.21.15.12:0/3420236966 conn(0x7fca4c076f40 msgr2=0x7fca4c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 -- 172.21.15.12:0/3420236966 shutdown_connections 2024-05-09T00:10:11.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:11.945+0000 7fca54459700 1 -- 172.21.15.12:0/3420236966 wait complete. 2024-05-09T00:10:11.956 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:10:11.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:10:11.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:11 smithi012 ceph-mon[42503]: pgmap v765: 97 pgs: 24 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 57 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.1 MiB/s wr, 648 op/s 2024-05-09T00:10:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.119+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/3239331785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e4100440 msgr2=0x7fe9e41008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.119+0000 7fe9e9f1e700 1 --2- 172.21.15.12:0/3239331785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e4100440 0x7fe9e41008c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80099c0 tx=0x7fe9d8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.119+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/3239331785 shutdown_connections 2024-05-09T00:10:12.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.119+0000 7fe9e9f1e700 1 --2- 172.21.15.12:0/3239331785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4100e00 0x7fe9e410cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.119+0000 7fe9e9f1e700 1 --2- 172.21.15.12:0/3239331785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9e4104d20 0x7fe9e4105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.119+0000 7fe9e9f1e700 1 --2- 172.21.15.12:0/3239331785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e4100440 0x7fe9e41008c0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.119+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/3239331785 >> 172.21.15.12:0/3239331785 conn(0x7fe9e4076270 msgr2=0x7fe9e40fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:12.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.119+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/3239331785 shutdown_connections 2024-05-09T00:10:12.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.119+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/3239331785 wait complete. 2024-05-09T00:10:12.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.120+0000 7fe9e9f1e700 1 Processor -- start 2024-05-09T00:10:12.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.120+0000 7fe9e9f1e700 1 -- start start 2024-05-09T00:10:12.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e9f1e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e4100440 0x7fe9e4195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e9f1e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4100e00 0x7fe9e4196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e9f1e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9e4104d20 0x7fe9e4196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e9f1e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41a55b0 con 0x7fe9e4100440 2024-05-09T00:10:12.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e9f1e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41a5720 con 0x7fe9e4104d20 2024-05-09T00:10:12.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e9f1e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41a58e0 con 0x7fe9e4100e00 2024-05-09T00:10:12.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e4100440 0x7fe9e4195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4100e00 0x7fe9e4196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e4100440 0x7fe9e4195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47480/0 (socket says 172.21.15.12:47480) 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e37fe700 1 -- 172.21.15.12:0/73601227 learned_addr learned my addr 172.21.15.12:0/73601227 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e37fe700 1 -- 172.21.15.12:0/73601227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4100e00 msgr2=0x7fe9e4196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e37fe700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4100e00 0x7fe9e4196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e37fe700 1 -- 172.21.15.12:0/73601227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9e4104d20 msgr2=0x7fe9e4196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e37fe700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9e4104d20 0x7fe9e4196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.121+0000 7fe9e37fe700 1 -- 172.21.15.12:0/73601227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d80095f0 con 0x7fe9e4100440 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.122+0000 7fe9e37fe700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e4100440 0x7fe9e4195e10 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400e9a0 tx=0x7fe9d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.122+0000 7fe9e0ff9700 1 -- 172.21.15.12:0/73601227 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4009790 con 0x7fe9e4100440 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.122+0000 7fe9e0ff9700 1 -- 172.21.15.12:0/73601227 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9d400cb50 con 0x7fe9e4100440 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.122+0000 7fe9e0ff9700 1 -- 172.21.15.12:0/73601227 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4018680 con 0x7fe9e4100440 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.122+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/73601227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e41a5bc0 con 0x7fe9e4100440 2024-05-09T00:10:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.122+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/73601227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e41a6050 con 0x7fe9e4100440 2024-05-09T00:10:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.123+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/73601227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e4108230 con 0x7fe9e4100440 2024-05-09T00:10:12.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.125+0000 7fe9e0ff9700 1 -- 172.21.15.12:0/73601227 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe9d400ccc0 con 0x7fe9e4100440 2024-05-09T00:10:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.125+0000 7fe9e0ff9700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9cc0a1310 0x7fe9cc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.125+0000 7fe9e2ffd700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9cc0a1310 0x7fe9cc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.125+0000 7fe9e0ff9700 1 -- 172.21.15.12:0/73601227 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe9d4014070 con 0x7fe9e4100440 2024-05-09T00:10:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.126+0000 7fe9e2ffd700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9cc0a1310 0x7fe9cc0a37d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009fd0 tx=0x7fe9d8019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:12.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.133+0000 7fe9e0ff9700 1 -- 172.21.15.12:0/73601227 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe9d4052d60 con 0x7fe9e4100440 2024-05-09T00:10:12.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:11 smithi160 ceph-mon[46152]: pgmap v765: 97 pgs: 24 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 57 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.1 MiB/s wr, 648 op/s 2024-05-09T00:10:12.270 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:11 smithi204 ceph-mon[47373]: pgmap v765: 97 pgs: 24 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 57 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.1 MiB/s wr, 648 op/s 2024-05-09T00:10:12.307 INFO:tasks.workunit.client.0.smithi012.stdout: 10 347510 28.84 MB/sec execute 487 sec latency 151.135 ms 2024-05-09T00:10:12.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.581+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/73601227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9e4101450 con 0x7fe9e4100440 2024-05-09T00:10:12.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.582+0000 7fe9e0ff9700 1 -- 172.21.15.12:0/73601227 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+10876 (secure 0 0 0) 0x7fe9d40962c0 con 0x7fe9e4100440 2024-05-09T00:10:12.579 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:10:12.579 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":21435,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/676326208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":676326208},{"type":"v1","addr":"172.21.15.160:6833","nonce":676326208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:09:19.023721+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/4091100213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4091100213},{"type":"v1","addr":"172.21.15.204:6833","nonce":4091100213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:10:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.585+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/73601227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9cc0a1310 msgr2=0x7fe9cc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.585+0000 7fe9e9f1e700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9cc0a1310 0x7fe9cc0a37d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009fd0 tx=0x7fe9d8019040 comp rx=0 tx=0).stop 2024-05-09T00:10:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.585+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/73601227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e4100440 msgr2=0x7fe9e4195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.585+0000 7fe9e9f1e700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e4100440 0x7fe9e4195e10 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400e9a0 tx=0x7fe9d400ecb0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.585+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/73601227 shutdown_connections 2024-05-09T00:10:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.585+0000 7fe9e9f1e700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe9cc0a1310 0x7fe9cc0a37d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.585+0000 7fe9e9f1e700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9e4104d20 0x7fe9e4196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.585+0000 7fe9e9f1e700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e4100440 0x7fe9e4195e10 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.585+0000 7fe9e9f1e700 1 --2- 172.21.15.12:0/73601227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4100e00 0x7fe9e4196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.585+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/73601227 >> 172.21.15.12:0/73601227 conn(0x7fe9e4076270 msgr2=0x7fe9e40fb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.586+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/73601227 shutdown_connections 2024-05-09T00:10:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.586+0000 7fe9e9f1e700 1 -- 172.21.15.12:0/73601227 wait complete. 2024-05-09T00:10:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2024-05-09T00:10:12.598 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:10:12.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.795+0000 7f47365d2700 1 -- 172.21.15.12:0/284531108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4730106290 msgr2=0x7f473010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.795+0000 7f47365d2700 1 --2- 172.21.15.12:0/284531108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4730106290 0x7f473010de10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f472800b0d0 tx=0x7f472800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.795+0000 7f47365d2700 1 -- 172.21.15.12:0/284531108 shutdown_connections 2024-05-09T00:10:12.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.795+0000 7f47365d2700 1 --2- 172.21.15.12:0/284531108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4730106290 0x7f473010de10 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.795+0000 7f47365d2700 1 --2- 172.21.15.12:0/284531108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47300682e0 0x7f47300686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.795+0000 7f47365d2700 1 --2- 172.21.15.12:0/284531108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4730068c90 0x7f4730105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.795+0000 7f47365d2700 1 -- 172.21.15.12:0/284531108 >> 172.21.15.12:0/284531108 conn(0x7f4730076f40 msgr2=0x7f4730077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:12.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.795+0000 7f47365d2700 1 -- 172.21.15.12:0/284531108 shutdown_connections 2024-05-09T00:10:12.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.795+0000 7f47365d2700 1 -- 172.21.15.12:0/284531108 wait complete. 2024-05-09T00:10:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.796+0000 7f47365d2700 1 Processor -- start 2024-05-09T00:10:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.796+0000 7f47365d2700 1 -- start start 2024-05-09T00:10:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f47365d2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47300682e0 0x7f47300ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f47365d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4730106290 0x7f47301003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f47365d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47301008e0 0x7f47301a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f47365d2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47301a5c90 con 0x7f47301008e0 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f47365d2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47301a5e00 con 0x7f4730106290 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f47365d2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47301a5fc0 con 0x7f47300682e0 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f472f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4730106290 0x7f47301003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f472ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47300682e0 0x7f47300ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f472ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47300682e0 0x7f47300ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52438/0 (socket says 172.21.15.12:52438) 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f472ffff700 1 -- 172.21.15.12:0/1087794855 learned_addr learned my addr 172.21.15.12:0/1087794855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f472f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4730106290 0x7f47301003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39276/0 (socket says 172.21.15.12:39276) 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.797+0000 7f4734b6f700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47301008e0 0x7f47301a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f472ffff700 1 -- 172.21.15.12:0/1087794855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4730106290 msgr2=0x7f47301003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f472ffff700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4730106290 0x7f47301003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f472ffff700 1 -- 172.21.15.12:0/1087794855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47301008e0 msgr2=0x7f47301a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f472ffff700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47301008e0 0x7f47301a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f472ffff700 1 -- 172.21.15.12:0/1087794855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728009d00 con 0x7f47300682e0 2024-05-09T00:10:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f472ffff700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47300682e0 0x7f47300ffe60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f472000e9a0 tx=0x7f472000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:12.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f472d7fa700 1 -- 172.21.15.12:0/1087794855 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720009790 con 0x7f47300682e0 2024-05-09T00:10:12.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f472d7fa700 1 -- 172.21.15.12:0/1087794855 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4720010430 con 0x7f47300682e0 2024-05-09T00:10:12.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f47365d2700 1 -- 172.21.15.12:0/1087794855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47301a62a0 con 0x7f47300682e0 2024-05-09T00:10:12.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f472d7fa700 1 -- 172.21.15.12:0/1087794855 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720018680 con 0x7f47300682e0 2024-05-09T00:10:12.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.798+0000 7f47365d2700 1 -- 172.21.15.12:0/1087794855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47301a6800 con 0x7f47300682e0 2024-05-09T00:10:12.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.801+0000 7f472d7fa700 1 -- 172.21.15.12:0/1087794855 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f47200105a0 con 0x7f47300682e0 2024-05-09T00:10:12.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.801+0000 7f472d7fa700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47180a1310 0x7f47180a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.802+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47180a1310 0x7f47180a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.802+0000 7f472d7fa700 1 -- 172.21.15.12:0/1087794855 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4720014070 con 0x7f47300682e0 2024-05-09T00:10:12.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.802+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47180a1310 0x7f47180a37d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4730101590 tx=0x7f4724009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.802+0000 7f474278cb80 1 Processor -- start 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.802+0000 7f474278cb80 1 -- start start 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f474278cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 0x56481c44e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f474278cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 0x56481c448350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f474278cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c448cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f474278cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c39f150 con 0x56481c44e720 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f474278cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c39cb10 con 0x56481c448890 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f474278cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c398400 con 0x56481c44de00 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 0x56481c448350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 0x56481c44e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 0x56481c448350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47500/0 (socket says 172.21.15.12:47500) 2024-05-09T00:10:12.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472f7fe700 1 -- 172.21.15.12:0/1085392804 learned_addr learned my addr 172.21.15.12:0/1085392804 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f4734b6f700 1 --2- 172.21.15.12:0/1085392804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c448cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472f7fe700 1 -- 172.21.15.12:0/1085392804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 msgr2=0x56481c44e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1085392804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 0x56481c44e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472f7fe700 1 -- 172.21.15.12:0/1085392804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 msgr2=0x56481c448cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1085392804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c448cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472f7fe700 1 -- 172.21.15.12:0/1085392804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56481c2193b0 con 0x56481c44e720 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472ffff700 1 --2- 172.21.15.12:0/1085392804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 0x56481c44e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.803+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1085392804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 0x56481c448350 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f4724011d10 tx=0x7f4724014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8875436d739eb7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.804+0000 7f47167fc700 1 -- 172.21.15.12:0/1085392804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724011d40 con 0x56481c44e720 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.804+0000 7f47167fc700 1 -- 172.21.15.12:0/1085392804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4724004020 con 0x56481c44e720 2024-05-09T00:10:12.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.804+0000 7f47167fc700 1 -- 172.21.15.12:0/1085392804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724003bc0 con 0x56481c44e720 2024-05-09T00:10:12.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.804+0000 7f474278cb80 1 -- 172.21.15.12:0/1085392804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 msgr2=0x56481c448350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.804+0000 7f474278cb80 1 --2- 172.21.15.12:0/1085392804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 0x56481c448350 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f4724011d10 tx=0x7f4724014000 comp rx=0 tx=0).stop 2024-05-09T00:10:12.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.805+0000 7f474278cb80 1 -- 172.21.15.12:0/1085392804 shutdown_connections 2024-05-09T00:10:12.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.805+0000 7f474278cb80 1 --2- 172.21.15.12:0/1085392804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c448cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.805+0000 7f474278cb80 1 --2- 172.21.15.12:0/1085392804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 0x56481c44e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.805+0000 7f474278cb80 1 --2- 172.21.15.12:0/1085392804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 0x56481c448350 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.805+0000 7f474278cb80 1 -- 172.21.15.12:0/1085392804 >> 172.21.15.12:0/1085392804 conn(0x56481c3a6210 msgr2=0x56481c442df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:12.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.805+0000 7f474278cb80 1 -- 172.21.15.12:0/1085392804 shutdown_connections 2024-05-09T00:10:12.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.805+0000 7f474278cb80 1 -- 172.21.15.12:0/1085392804 wait complete. 2024-05-09T00:10:12.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.805+0000 7f474278cb80 1 -- start start 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.805+0000 7f474278cb80 1 Processor -- start 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f474278cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c451ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f474278cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 0x56481c452010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f474278cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44e720 0x56481c452550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f474278cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c334560 con 0x56481c44de00 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f474278cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c334200 con 0x56481c448890 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f474278cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c21b6a0 con 0x56481c44e720 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f474278cb80 10 client.0 _ll_register_callbacks cb 0x56481c33db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f472ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c451ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f472ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c451ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39292/0 (socket says 172.21.15.12:39292) 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f472ffff700 1 -- 172.21.15.12:0/1854094242 learned_addr learned my addr 172.21.15.12:0/1854094242 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 0x56481c452010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f472ffff700 1 -- 172.21.15.12:0/1854094242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44e720 msgr2=0x56481c452550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:12.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f472ffff700 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44e720 0x56481c452550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f472ffff700 1 -- 172.21.15.12:0/1854094242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 msgr2=0x56481c452010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f472ffff700 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 0x56481c452010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.806+0000 7f472ffff700 1 -- 172.21.15.12:0/1854094242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47301a6800 con 0x56481c448890 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.807+0000 7f472ffff700 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c451ad0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4720099c40 tx=0x7f4720099f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=106d12808fff6d25 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.807+0000 7f470f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.807+0000 7f470f7fe700 1 -- 172.21.15.12:0/1854094242 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720099630 con 0x56481c448890 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.807+0000 7f470f7fe700 1 -- 172.21.15.12:0/1854094242 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f47200a23c0 con 0x56481c448890 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.807+0000 7f474278cb80 1 -- 172.21.15.12:0/1854094242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56481c2f1270 con 0x56481c448890 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.807+0000 7f470f7fe700 1 -- 172.21.15.12:0/1854094242 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47200a1f30 con 0x56481c448890 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.807+0000 7f474278cb80 10 client.31280 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.807+0000 7f474278cb80 1 -- 172.21.15.12:0/1854094242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7f4724017070 con 0x56481c448890 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.807+0000 7f470f7fe700 1 -- 172.21.15.12:0/1854094242 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56481c2f1270 con 0x56481c448890 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.808+0000 7f470f7fe700 1 -- 172.21.15.12:0/1854094242 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47200b7f90 con 0x56481c448890 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.808+0000 7f474278cb80 10 client.31280 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:12.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.808+0000 7f474278cb80 10 client.31280 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:10:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.808+0000 7f474278cb80 20 client.31280 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.808+0000 7f474278cb80 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56481c462200 0x56481c464620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.808+0000 7f474278cb80 4 client.31280 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.808+0000 7f474278cb80 1 -- 172.21.15.12:0/1854094242 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56481c417810 con 0x56481c462200 2024-05-09T00:10:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.808+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56481c462200 0x56481c464620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.808+0000 7f470f7fe700 10 client.31280 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:10:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.808+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56481c462200 0x56481c464620 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11ceebe90888c045 server_cookie=8084c3561657c83b in_seq=0 out_seq=0 2024-05-09T00:10:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.809+0000 7f470f7fe700 1 -- 172.21.15.12:0/1854094242 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56481c417810 con 0x56481c462200 2024-05-09T00:10:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.809+0000 7f470f7fe700 10 client.31280 handle_command_reply: tid=0 2024-05-09T00:10:12.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.809+0000 7f474278cb80 1 client.31280 shutdown 2024-05-09T00:10:12.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 1 -- 172.21.15.12:0/1854094242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 msgr2=0x56481c451ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c451ad0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4720099c40 tx=0x7f4720099f40 comp rx=0 tx=0).stop 2024-05-09T00:10:12.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 1 -- 172.21.15.12:0/1854094242 shutdown_connections 2024-05-09T00:10:12.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56481c462200 0x56481c464620 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44e720 0x56481c452550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c451ad0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 1 --2- 172.21.15.12:0/1854094242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 0x56481c452010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 1 -- 172.21.15.12:0/1854094242 >> 172.21.15.12:0/1854094242 conn(0x56481c3a6210 msgr2=0x56481c4425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:12.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f470f7fe700 0 client.31280 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:10:12.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 1 -- 172.21.15.12:0/1854094242 shutdown_connections 2024-05-09T00:10:12.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 1 -- 172.21.15.12:0/1854094242 wait complete. 2024-05-09T00:10:12.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.810+0000 7f474278cb80 20 client.31280 trim_cache size 0 max 16384 2024-05-09T00:10:12.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.825+0000 7f474278cb80 1 Processor -- start 2024-05-09T00:10:12.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.825+0000 7f474278cb80 1 -- start start 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f474278cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c448890 0x56481c448350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f474278cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 0x56481c442d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f474278cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c44e720 0x56481c452020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f474278cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c44e560 con 0x56481c44de00 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f474278cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c444a00 con 0x56481c44e720 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f474278cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c450020 con 0x56481c448890 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f472f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 0x56481c442d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f472f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 0x56481c442d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47510/0 (socket says 172.21.15.12:47510) 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f472f7fe700 1 -- 172.21.15.12:0/1658471727 learned_addr learned my addr 172.21.15.12:0/1658471727 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f4734b6f700 1 --2- 172.21.15.12:0/1658471727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c44e720 0x56481c452020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f472f7fe700 1 -- 172.21.15.12:0/1658471727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c448890 msgr2=0x56481c448350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f472ffff700 1 --2- 172.21.15.12:0/1658471727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c448890 0x56481c448350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1658471727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c448890 0x56481c448350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f472f7fe700 1 -- 172.21.15.12:0/1658471727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c44e720 msgr2=0x56481c452020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:12.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1658471727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c44e720 0x56481c452020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f472f7fe700 1 -- 172.21.15.12:0/1658471727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56481c448f40 con 0x56481c44de00 2024-05-09T00:10:12.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.826+0000 7f472ffff700 1 --2- 172.21.15.12:0/1658471727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c448890 0x56481c448350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:10:12.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.827+0000 7f472f7fe700 1 --2- 172.21.15.12:0/1658471727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 0x56481c442d20 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f47240036f0 tx=0x7f4724003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68fab3e4a2b4405 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:12.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.827+0000 7f4716ffd700 1 -- 172.21.15.12:0/1658471727 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724035070 con 0x56481c44de00 2024-05-09T00:10:12.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.827+0000 7f4716ffd700 1 -- 172.21.15.12:0/1658471727 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f472400eaa0 con 0x56481c44de00 2024-05-09T00:10:12.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.827+0000 7f4716ffd700 1 -- 172.21.15.12:0/1658471727 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472402c040 con 0x56481c44de00 2024-05-09T00:10:12.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.828+0000 7f474278cb80 1 -- 172.21.15.12:0/1658471727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 msgr2=0x56481c442d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.828+0000 7f474278cb80 1 --2- 172.21.15.12:0/1658471727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 0x56481c442d20 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f47240036f0 tx=0x7f4724003680 comp rx=0 tx=0).stop 2024-05-09T00:10:12.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.828+0000 7f474278cb80 1 -- 172.21.15.12:0/1658471727 shutdown_connections 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.828+0000 7f474278cb80 1 --2- 172.21.15.12:0/1658471727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c44e720 0x56481c452020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.828+0000 7f474278cb80 1 --2- 172.21.15.12:0/1658471727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c448890 0x56481c448350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.828+0000 7f474278cb80 1 --2- 172.21.15.12:0/1658471727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44de00 0x56481c442d20 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.828+0000 7f474278cb80 1 -- 172.21.15.12:0/1658471727 >> 172.21.15.12:0/1658471727 conn(0x56481c3a6210 msgr2=0x56481c4503a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.828+0000 7f474278cb80 1 -- 172.21.15.12:0/1658471727 shutdown_connections 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.828+0000 7f474278cb80 1 -- 172.21.15.12:0/1658471727 wait complete. 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.828+0000 7f474278cb80 1 -- start start 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f474278cb80 1 Processor -- start 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f474278cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c458e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f474278cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 0x56481c4593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f474278cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 0x56481c4574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f474278cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c457b70 con 0x56481c44e720 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f474278cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c457ce0 con 0x56481c448890 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f474278cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481c457ea0 con 0x56481c44de00 2024-05-09T00:10:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f474278cb80 10 client.0 _ll_register_callbacks cb 0x56481c33db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f472ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c458e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f472ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c458e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39304/0 (socket says 172.21.15.12:39304) 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f4734b6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 0x56481c4574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f472ffff700 1 -- 172.21.15.12:0/2833668337 learned_addr learned my addr 172.21.15.12:0/2833668337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.829+0000 7f472f7fe700 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 0x56481c4593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.830+0000 7f472ffff700 1 -- 172.21.15.12:0/2833668337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 msgr2=0x56481c4593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.830+0000 7f472ffff700 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 0x56481c4593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.830+0000 7f472ffff700 1 -- 172.21.15.12:0/2833668337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 msgr2=0x56481c4574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.830+0000 7f472ffff700 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 0x56481c4574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.830+0000 7f472ffff700 1 -- 172.21.15.12:0/2833668337 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56481c2193b0 con 0x56481c448890 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.830+0000 7f472f7fe700 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 0x56481c4593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.830+0000 7f472ffff700 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c458e60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4720096440 tx=0x7f472000c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=617084d8be856cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.830+0000 7f470f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:10:12.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.833+0000 7f470f7fe700 1 -- 172.21.15.12:0/2833668337 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720096280 con 0x56481c448890 2024-05-09T00:10:12.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.833+0000 7f470f7fe700 1 -- 172.21.15.12:0/2833668337 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f47200a23c0 con 0x56481c448890 2024-05-09T00:10:12.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.833+0000 7f474278cb80 1 -- 172.21.15.12:0/2833668337 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4724004020 con 0x56481c448890 2024-05-09T00:10:12.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.833+0000 7f470f7fe700 1 -- 172.21.15.12:0/2833668337 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4720094b50 con 0x56481c448890 2024-05-09T00:10:12.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.833+0000 7f474278cb80 10 client.31286 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:12.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.833+0000 7f474278cb80 1 -- 172.21.15.12:0/2833668337 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x56481c1c5b60 con 0x56481c448890 2024-05-09T00:10:12.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.833+0000 7f470f7fe700 1 -- 172.21.15.12:0/2833668337 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4724004020 con 0x56481c448890 2024-05-09T00:10:12.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.833+0000 7f470f7fe700 1 -- 172.21.15.12:0/2833668337 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4720094cb0 con 0x56481c448890 2024-05-09T00:10:12.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.834+0000 7f474278cb80 10 client.31286 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.834+0000 7f474278cb80 10 client.31286 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.834+0000 7f474278cb80 20 client.31286 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.834+0000 7f474278cb80 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56481c4c1e40 0x56481c4c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.834+0000 7f474278cb80 4 client.31286 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.834+0000 7f474278cb80 1 -- 172.21.15.12:0/2833668337 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56481c390ec0 con 0x56481c4c1e40 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.834+0000 7f472f7fe700 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56481c4c1e40 0x56481c4c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.834+0000 7f470f7fe700 10 client.31286 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.834+0000 7f472f7fe700 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56481c4c1e40 0x56481c4c4230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4fcf4cdc267fafb server_cookie=39e036bb76d2835d in_seq=0 out_seq=0 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.834+0000 7f470f7fe700 1 -- 172.21.15.12:0/2833668337 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56481c390ec0 con 0x56481c4c1e40 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f470f7fe700 10 client.31286 handle_command_reply: tid=0 2024-05-09T00:10:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 client.31286 shutdown 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 -- 172.21.15.12:0/2833668337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 msgr2=0x56481c458e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c458e60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4720096440 tx=0x7f472000c480 comp rx=0 tx=0).stop 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 -- 172.21.15.12:0/2833668337 shutdown_connections 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56481c44e720 0x56481c4574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56481c448890 0x56481c458e60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x56481c4c1e40 0x56481c4c4230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 --2- 172.21.15.12:0/2833668337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56481c44de00 0x56481c4593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f470f7fe700 0 client.31286 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 -- 172.21.15.12:0/2833668337 >> 172.21.15.12:0/2833668337 conn(0x56481c3a6210 msgr2=0x56481c44cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 -- 172.21.15.12:0/2833668337 shutdown_connections 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 1 -- 172.21.15.12:0/2833668337 wait complete. 2024-05-09T00:10:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.835+0000 7f474278cb80 20 client.31286 trim_cache size 0 max 16384 2024-05-09T00:10:12.835 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 -- 172.21.15.12:0/1087794855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47180a1310 msgr2=0x7f47180a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47180a1310 0x7f47180a37d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4730101590 tx=0x7f4724009260 comp rx=0 tx=0).stop 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 -- 172.21.15.12:0/1087794855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47300682e0 msgr2=0x7f47300ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47300682e0 0x7f47300ffe60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f472000e9a0 tx=0x7f472000ecb0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 -- 172.21.15.12:0/1087794855 shutdown_connections 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47180a1310 0x7f47180a37d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47301008e0 0x7f47301a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47300682e0 0x7f47300ffe60 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 --2- 172.21.15.12:0/1087794855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4730106290 0x7f47301003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 -- 172.21.15.12:0/1087794855 >> 172.21.15.12:0/1087794855 conn(0x7f4730076f40 msgr2=0x7f47300fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 -- 172.21.15.12:0/1087794855 shutdown_connections 2024-05-09T00:10:12.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:12.836+0000 7f47365d2700 1 -- 172.21.15.12:0/1087794855 wait complete. 2024-05-09T00:10:12.847 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:12 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/73601227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:12.849 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:10:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.051+0000 7fe875645700 1 -- 172.21.15.12:0/80203593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe870106210 msgr2=0x7fe87010dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.051+0000 7fe875645700 1 --2- 172.21.15.12:0/80203593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe870106210 0x7fe87010dd90 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fe860009990 tx=0x7fe860009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.052+0000 7fe875645700 1 -- 172.21.15.12:0/80203593 shutdown_connections 2024-05-09T00:10:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.052+0000 7fe875645700 1 --2- 172.21.15.12:0/80203593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe870106210 0x7fe87010dd90 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.052+0000 7fe875645700 1 --2- 172.21.15.12:0/80203593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8701010f0 0x7fe8701014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.052+0000 7fe875645700 1 --2- 172.21.15.12:0/80203593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe870101aa0 0x7fe870105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.052+0000 7fe875645700 1 -- 172.21.15.12:0/80203593 >> 172.21.15.12:0/80203593 conn(0x7fe8700fc9d0 msgr2=0x7fe8700fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.053+0000 7f0557e61700 1 -- 172.21.15.12:0/895366952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f055010fac0 msgr2=0x7f055010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.053+0000 7f0557e61700 1 --2- 172.21.15.12:0/895366952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f055010fac0 0x7f055010d880 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f054c01a9a0 tx=0x7f054c01acb0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.055+0000 7fe875645700 1 -- 172.21.15.12:0/80203593 shutdown_connections 2024-05-09T00:10:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.055+0000 7fe875645700 1 -- 172.21.15.12:0/80203593 wait complete. 2024-05-09T00:10:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.056+0000 7fe875645700 1 Processor -- start 2024-05-09T00:10:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.056+0000 7fe875645700 1 -- start start 2024-05-09T00:10:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.056+0000 7fe875645700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe870106210 0x7fe870116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.056+0000 7fe875645700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe870116840 0x7fe8701103f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.056+0000 7fe875645700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe870110930 0x7fe870110de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.056+0000 7fe875645700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe870116eb0 con 0x7fe870116840 2024-05-09T00:10:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.056+0000 7fe875645700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87006a9c0 con 0x7fe870110930 2024-05-09T00:10:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.056+0000 7fe875645700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87006ab80 con 0x7fe870106210 2024-05-09T00:10:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.057+0000 7fe86e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe870116840 0x7fe8701103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.057+0000 7fe86e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe870116840 0x7fe8701103f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47532/0 (socket says 172.21.15.12:47532) 2024-05-09T00:10:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.057+0000 7fe86e7fc700 1 -- 172.21.15.12:0/2569104225 learned_addr learned my addr 172.21.15.12:0/2569104225 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.057+0000 7fe86e7fc700 1 -- 172.21.15.12:0/2569104225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe870106210 msgr2=0x7fe870116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.057+0000 7fe86e7fc700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe870106210 0x7fe870116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.057+0000 7fe86e7fc700 1 -- 172.21.15.12:0/2569104225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe870110930 msgr2=0x7fe870110de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.057+0000 7fe86e7fc700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe870110930 0x7fe870110de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.057+0000 7fe86e7fc700 1 -- 172.21.15.12:0/2569104225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8600095f0 con 0x7fe870116840 2024-05-09T00:10:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.058+0000 7fe86e7fc700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe870116840 0x7fe8701103f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe85800d980 tx=0x7fe85800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.058+0000 7fe867fff700 1 -- 172.21.15.12:0/2569104225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858009b40 con 0x7fe870116840 2024-05-09T00:10:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.058+0000 7fe867fff700 1 -- 172.21.15.12:0/2569104225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe85800b640 con 0x7fe870116840 2024-05-09T00:10:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.058+0000 7fe875645700 1 -- 172.21.15.12:0/2569104225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87006ae60 con 0x7fe870116840 2024-05-09T00:10:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.058+0000 7fe875645700 1 -- 172.21.15.12:0/2569104225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87006b3c0 con 0x7fe870116840 2024-05-09T00:10:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.058+0000 7fe867fff700 1 -- 172.21.15.12:0/2569104225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85800f3c0 con 0x7fe870116840 2024-05-09T00:10:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.059+0000 7f0557e61700 1 -- 172.21.15.12:0/895366952 shutdown_connections 2024-05-09T00:10:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.059+0000 7f0557e61700 1 --2- 172.21.15.12:0/895366952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0550107fa0 0x7f0550108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.059+0000 7f0557e61700 1 --2- 172.21.15.12:0/895366952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055010f650 0x7f055010d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.059+0000 7f0557e61700 1 --2- 172.21.15.12:0/895366952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f055010fac0 0x7f055010d880 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.059+0000 7f0557e61700 1 -- 172.21.15.12:0/895366952 >> 172.21.15.12:0/895366952 conn(0x7f055006bf10 msgr2=0x7f055006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.060+0000 7f0557e61700 1 -- 172.21.15.12:0/895366952 shutdown_connections 2024-05-09T00:10:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.060+0000 7f0557e61700 1 -- 172.21.15.12:0/895366952 wait complete. 2024-05-09T00:10:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.060+0000 7f0557e61700 1 Processor -- start 2024-05-09T00:10:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.061+0000 7f0557e61700 1 -- start start 2024-05-09T00:10:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.061+0000 7f0557e61700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0550107fa0 0x7f05501a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.061+0000 7f0557e61700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f055010f650 0x7f05501a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.061+0000 7f0557e61700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055010fac0 0x7f055019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.061+0000 7f0557e61700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055019e0f0 con 0x7f055010f650 2024-05-09T00:10:13.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.061+0000 7f0557e61700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055019e260 con 0x7f0550107fa0 2024-05-09T00:10:13.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.061+0000 7f0557e61700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055019e420 con 0x7f055010fac0 2024-05-09T00:10:13.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f05563fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055010fac0 0x7f055019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f05563fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055010fac0 0x7f055019da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52498/0 (socket says 172.21.15.12:52498) 2024-05-09T00:10:13.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f05563fe700 1 -- 172.21.15.12:0/2060526883 learned_addr learned my addr 172.21.15.12:0/2060526883 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f0555bfd700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0550107fa0 0x7f05501a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f05553fc700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f055010f650 0x7f05501a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f05563fe700 1 -- 172.21.15.12:0/2060526883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0550107fa0 msgr2=0x7f05501a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f05563fe700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0550107fa0 0x7f05501a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f05563fe700 1 -- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f055010f650 msgr2=0x7f05501a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f05563fe700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f055010f650 0x7f05501a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f05563fe700 1 -- 172.21.15.12:0/2060526883 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c01a600 con 0x7f055010fac0 2024-05-09T00:10:13.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.062+0000 7f05553fc700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f055010f650 0x7f05501a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.063+0000 7fe8817ffb80 1 Processor -- start 2024-05-09T00:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.064+0000 7fe8817ffb80 1 -- start start 2024-05-09T00:10:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.064+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de4950 0x557258de4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.064+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258de5270 0x557258ddef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.064+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258ddf480 0x557258ddf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.064+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258d360f0 con 0x557258de4950 2024-05-09T00:10:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.064+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258d33ab0 con 0x557258de5270 2024-05-09T00:10:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.064+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258d2f3a0 con 0x557258ddf480 2024-05-09T00:10:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.066+0000 7fe86effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258de5270 0x557258ddef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.066+0000 7fe867fff700 1 -- 172.21.15.12:0/2569104225 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe858009ca0 con 0x7fe870116840 2024-05-09T00:10:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.066+0000 7fe86f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de4950 0x557258de4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.066+0000 7fe86f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de4950 0x557258de4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47552/0 (socket says 172.21.15.12:47552) 2024-05-09T00:10:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.066+0000 7fe86f7fe700 1 -- 172.21.15.12:0/3938853688 learned_addr learned my addr 172.21.15.12:0/3938853688 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.067+0000 7fe86f7fe700 1 -- 172.21.15.12:0/3938853688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258ddf480 msgr2=0x557258ddf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.067+0000 7fe86f7fe700 1 --2- 172.21.15.12:0/3938853688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258ddf480 0x557258ddf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.067+0000 7fe86f7fe700 1 -- 172.21.15.12:0/3938853688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258de5270 msgr2=0x557258ddef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.067+0000 7fe86f7fe700 1 --2- 172.21.15.12:0/3938853688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258de5270 0x557258ddef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.067+0000 7fe86f7fe700 1 -- 172.21.15.12:0/3938853688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557258bb0350 con 0x557258de4950 2024-05-09T00:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.067+0000 7fe867fff700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe85c0a1310 0x7fe85c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.067+0000 7fe867fff700 1 -- 172.21.15.12:0/2569104225 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe8580ce850 con 0x7fe870116840 2024-05-09T00:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.067+0000 7fe86f7fe700 1 --2- 172.21.15.12:0/3938853688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de4950 0x557258de4d30 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe860005f40 tx=0x7fe8600094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd700eaeb9eaed28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.067+0000 7fe86effd700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe85c0a1310 0x7fe85c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.068+0000 7fe8657fa700 1 -- 172.21.15.12:0/3938853688 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860010460 con 0x557258de4950 2024-05-09T00:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.068+0000 7fe8657fa700 1 -- 172.21.15.12:0/3938853688 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8600105c0 con 0x557258de4950 2024-05-09T00:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.068+0000 7fe8657fa700 1 -- 172.21.15.12:0/3938853688 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8600216c0 con 0x557258de4950 2024-05-09T00:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.069+0000 7fe8817ffb80 1 -- 172.21.15.12:0/3938853688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de4950 msgr2=0x557258de4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.069+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/3938853688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de4950 0x557258de4d30 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe860005f40 tx=0x7fe8600094b0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.069+0000 7fe86effd700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe85c0a1310 0x7fe85c0a37d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x557258dda790 tx=0x7fe86800b040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.069+0000 7fe8817ffb80 1 -- 172.21.15.12:0/3938853688 shutdown_connections 2024-05-09T00:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.069+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/3938853688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258ddf480 0x557258ddf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.069+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/3938853688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de4950 0x557258de4d30 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.069+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/3938853688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258de5270 0x557258ddef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.069+0000 7fe8817ffb80 1 -- 172.21.15.12:0/3938853688 >> 172.21.15.12:0/3938853688 conn(0x557258d3d1b0 msgr2=0x557258dd9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.070+0000 7fe8817ffb80 1 -- 172.21.15.12:0/3938853688 shutdown_connections 2024-05-09T00:10:13.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.070+0000 7f05563fe700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055010fac0 0x7f055019da90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f054800b780 tx=0x7f054800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.070+0000 7f0546ffd700 1 -- 172.21.15.12:0/2060526883 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054800f840 con 0x7f055010fac0 2024-05-09T00:10:13.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.070+0000 7f0546ffd700 1 -- 172.21.15.12:0/2060526883 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f054800fe60 con 0x7f055010fac0 2024-05-09T00:10:13.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.070+0000 7f0557e61700 1 -- 172.21.15.12:0/2060526883 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055019e700 con 0x7f055010fac0 2024-05-09T00:10:13.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.070+0000 7f0546ffd700 1 -- 172.21.15.12:0/2060526883 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054800d3d0 con 0x7f055010fac0 2024-05-09T00:10:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.070+0000 7fe8817ffb80 1 -- 172.21.15.12:0/3938853688 wait complete. 2024-05-09T00:10:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.070+0000 7fe8817ffb80 1 -- start start 2024-05-09T00:10:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.070+0000 7fe8817ffb80 1 Processor -- start 2024-05-09T00:10:13.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.070+0000 7f0557e61700 1 -- 172.21.15.12:0/2060526883 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05501aa900 con 0x7f055010fac0 2024-05-09T00:10:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 0x557258deb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258de8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258de8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258ccb500 con 0x557258de5270 2024-05-09T00:10:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258ccb1a0 con 0x557258ddf480 2024-05-09T00:10:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258bb2640 con 0x557258de4950 2024-05-09T00:10:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe8817ffb80 10 client.0 _ll_register_callbacks cb 0x557258cd4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe86effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 0x557258deb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe86f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258de8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe86effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 0x557258deb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39376/0 (socket says 172.21.15.12:39376) 2024-05-09T00:10:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.071+0000 7fe86effd700 1 -- 172.21.15.12:0/1145226105 learned_addr learned my addr 172.21.15.12:0/1145226105 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.072+0000 7fe86f7fe700 1 -- 172.21.15.12:0/1145226105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 msgr2=0x557258de8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.072+0000 7fe86e7fc700 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258de8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.072+0000 7fe86f7fe700 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258de8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.072+0000 7fe86f7fe700 1 -- 172.21.15.12:0/1145226105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 msgr2=0x557258deb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.072+0000 7fe86f7fe700 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 0x557258deb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.072+0000 7fe86f7fe700 1 -- 172.21.15.12:0/1145226105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87010e600 con 0x557258de5270 2024-05-09T00:10:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.072+0000 7fe86e7fc700 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258de8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:10:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.072+0000 7fe86effd700 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 0x557258deb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.073+0000 7fe86f7fe700 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258de8ca0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fe8600098e0 tx=0x7fe860010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5340dad540bbde36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.073+0000 7fe8567fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.073+0000 7fe8567fc700 1 -- 172.21.15.12:0/1145226105 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8600038f0 con 0x557258de5270 2024-05-09T00:10:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.073+0000 7fe8567fc700 1 -- 172.21.15.12:0/1145226105 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe860011e40 con 0x557258de5270 2024-05-09T00:10:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.073+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1145226105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557258dae7b0 con 0x557258de5270 2024-05-09T00:10:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.073+0000 7fe8567fc700 1 -- 172.21.15.12:0/1145226105 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe860003a50 con 0x557258de5270 2024-05-09T00:10:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.073+0000 7f0546ffd700 1 -- 172.21.15.12:0/2060526883 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f054800d530 con 0x7f055010fac0 2024-05-09T00:10:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.073+0000 7fe8817ffb80 10 client.21891 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.073+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1145226105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x557258b5cb00 con 0x557258de5270 2024-05-09T00:10:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.074+0000 7fe8567fc700 1 -- 172.21.15.12:0/1145226105 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557258dae7b0 con 0x557258de5270 2024-05-09T00:10:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.074+0000 7fe8567fc700 1 -- 172.21.15.12:0/1145226105 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe860036070 con 0x557258de5270 2024-05-09T00:10:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.074+0000 7fe8817ffb80 10 client.21891 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.074+0000 7fe8817ffb80 10 client.21891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.074+0000 7fe8817ffb80 20 client.21891 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.074+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557258df8890 0x557258dfacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.074+0000 7fe8817ffb80 4 client.21891 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.074+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1145226105 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557258d27e60 con 0x557258df8890 2024-05-09T00:10:13.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.074+0000 7f0546ffd700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f053c0a13d0 0x7f053c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.074+0000 7f0546ffd700 1 -- 172.21.15.12:0/2060526883 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f05480cdad0 con 0x7f055010fac0 2024-05-09T00:10:13.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.075+0000 7f0555bfd700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f053c0a13d0 0x7f053c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.077+0000 7fe86effd700 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557258df8890 0x557258dfacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.077+0000 7f056401bb80 1 Processor -- start 2024-05-09T00:10:13.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.077+0000 7f0555bfd700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f053c0a13d0 0x7f053c0a3890 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f054000a850 tx=0x7f0540008040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.078+0000 7fe86effd700 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557258df8890 0x557258dfacb0 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e730e8d4db4bca34 server_cookie=d3ecb40f629364da in_seq=0 out_seq=0 2024-05-09T00:10:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.078+0000 7fe8567fc700 10 client.21891 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:13.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f056401bb80 1 -- start start 2024-05-09T00:10:13.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f056401bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f0e00 0x5579f51f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f056401bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51eb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f056401bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51eb890 0x5579f51ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f056401bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f5142150 con 0x5579f51f1720 2024-05-09T00:10:13.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f056401bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f513fb10 con 0x5579f51f0e00 2024-05-09T00:10:13.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f056401bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f513b400 con 0x5579f51eb890 2024-05-09T00:10:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f05553fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f05563fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51eb890 0x5579f51ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f05553fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51eb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47574/0 (socket says 172.21.15.12:47574) 2024-05-09T00:10:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f05563fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51eb890 0x5579f51ebcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52526/0 (socket says 172.21.15.12:52526) 2024-05-09T00:10:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f05553fc700 1 -- 172.21.15.12:0/2000009679 learned_addr learned my addr 172.21.15.12:0/2000009679 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.078+0000 7f0555bfd700 1 --2- 172.21.15.12:0/2000009679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f0e00 0x5579f51f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.079+0000 7f05553fc700 1 -- 172.21.15.12:0/2000009679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51eb890 msgr2=0x5579f51ebcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.079+0000 7f05553fc700 1 --2- 172.21.15.12:0/2000009679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51eb890 0x5579f51ebcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.079+0000 7f05553fc700 1 -- 172.21.15.12:0/2000009679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f0e00 msgr2=0x5579f51f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.079+0000 7f05553fc700 1 --2- 172.21.15.12:0/2000009679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f0e00 0x5579f51f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.079+0000 7f05553fc700 1 -- 172.21.15.12:0/2000009679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f4fbc3b0 con 0x5579f51f1720 2024-05-09T00:10:13.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.079+0000 7f05553fc700 1 --2- 172.21.15.12:0/2000009679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51eb350 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f054c01a9a0 tx=0x7f054c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833bb2057b5c150 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.079+0000 7f053bfff700 1 -- 172.21.15.12:0/2000009679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c003700 con 0x5579f51f1720 2024-05-09T00:10:13.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.079+0000 7f053bfff700 1 -- 172.21.15.12:0/2000009679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f054c01ceb0 con 0x5579f51f1720 2024-05-09T00:10:13.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.079+0000 7f053bfff700 1 -- 172.21.15.12:0/2000009679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c004bc0 con 0x5579f51f1720 2024-05-09T00:10:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.080+0000 7fe8567fc700 1 -- 172.21.15.12:0/1145226105 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557258d27e60 con 0x557258df8890 2024-05-09T00:10:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.080+0000 7fe8567fc700 10 client.21891 handle_command_reply: tid=0 2024-05-09T00:10:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.080+0000 7fe8817ffb80 1 client.21891 shutdown 2024-05-09T00:10:13.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.080+0000 7f056401bb80 1 -- 172.21.15.12:0/2000009679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 msgr2=0x5579f51eb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.080+0000 7f056401bb80 1 --2- 172.21.15.12:0/2000009679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51eb350 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f054c01a9a0 tx=0x7f054c0056d0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.081+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1145226105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 msgr2=0x557258de8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.081+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258de8ca0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fe8600098e0 tx=0x7fe860010c00 comp rx=0 tx=0).stop 2024-05-09T00:10:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.081+0000 7f056401bb80 1 -- 172.21.15.12:0/2000009679 shutdown_connections 2024-05-09T00:10:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.081+0000 7f056401bb80 1 --2- 172.21.15.12:0/2000009679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51eb890 0x5579f51ebcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.081+0000 7f056401bb80 1 --2- 172.21.15.12:0/2000009679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f0e00 0x5579f51f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.081+0000 7f056401bb80 1 --2- 172.21.15.12:0/2000009679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51eb350 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.081+0000 7f056401bb80 1 -- 172.21.15.12:0/2000009679 >> 172.21.15.12:0/2000009679 conn(0x5579f5149210 msgr2=0x5579f51e5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.081+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1145226105 shutdown_connections 2024-05-09T00:10:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.081+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258de8ca0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.081+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557258df8890 0x557258dfacb0 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.081+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 0x557258deb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.081+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1145226105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258de8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.081+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1145226105 >> 172.21.15.12:0/1145226105 conn(0x557258d3d1b0 msgr2=0x557258dddf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.081+0000 7fe8567fc700 0 client.21891 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:13.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.081+0000 7f056401bb80 1 -- 172.21.15.12:0/2000009679 shutdown_connections 2024-05-09T00:10:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.082+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1145226105 shutdown_connections 2024-05-09T00:10:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.082+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1145226105 wait complete. 2024-05-09T00:10:13.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.081+0000 7f056401bb80 1 -- 172.21.15.12:0/2000009679 wait complete. 2024-05-09T00:10:13.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.082+0000 7f056401bb80 1 -- start start 2024-05-09T00:10:13.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.082+0000 7f056401bb80 1 Processor -- start 2024-05-09T00:10:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.082+0000 7fe8817ffb80 20 client.21891 trim_cache size 0 max 16384 2024-05-09T00:10:13.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.082+0000 7f056401bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 0x5579f51f7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.082+0000 7f056401bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f51f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.082+0000 7f056401bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51f5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f056401bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f50d7560 con 0x5579f51eb890 2024-05-09T00:10:13.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f056401bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f50d7200 con 0x5579f51f1720 2024-05-09T00:10:13.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f056401bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f4fbe6a0 con 0x5579f51f0e00 2024-05-09T00:10:13.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f056401bb80 10 client.0 _ll_register_callbacks cb 0x5579f50e0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:13.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f05553fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f51f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f05553fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f51f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52538/0 (socket says 172.21.15.12:52538) 2024-05-09T00:10:13.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f05553fc700 1 -- 172.21.15.12:0/886685009 learned_addr learned my addr 172.21.15.12:0/886685009 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f05563fe700 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f0555bfd700 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 0x5579f51f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f05563fe700 1 -- 172.21.15.12:0/886685009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 msgr2=0x5579f51f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f05563fe700 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f51f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f05563fe700 1 -- 172.21.15.12:0/886685009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 msgr2=0x5579f51f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f05563fe700 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 0x5579f51f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.083+0000 7f05563fe700 1 -- 172.21.15.12:0/886685009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055010b070 con 0x5579f51f1720 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.084+0000 7f05563fe700 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51f5240 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f05480cb1b0 tx=0x7f054809acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd90ac29d3ef185f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.084+0000 7f05397fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.084+0000 7f05397fa700 1 -- 172.21.15.12:0/886685009 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054809a060 con 0x5579f51f1720 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.084+0000 7f05397fa700 1 -- 172.21.15.12:0/886685009 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f054809a1c0 con 0x5579f51f1720 2024-05-09T00:10:13.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.084+0000 7f056401bb80 1 -- 172.21.15.12:0/886685009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579f5094270 con 0x5579f51f1720 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.084+0000 7f05397fa700 1 -- 172.21.15.12:0/886685009 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05480a2b60 con 0x5579f51f1720 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.084+0000 7f056401bb80 10 client.31298 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.084+0000 7f056401bb80 1 -- 172.21.15.12:0/886685009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x5579f4f68b60 con 0x5579f51f1720 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.085+0000 7f05397fa700 1 -- 172.21.15.12:0/886685009 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5579f5094270 con 0x5579f51f1720 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.085+0000 7f05397fa700 1 -- 172.21.15.12:0/886685009 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f054809bb80 con 0x5579f51f1720 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.085+0000 7f056401bb80 10 client.31298 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.085+0000 7f056401bb80 10 client.31298 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.085+0000 7f056401bb80 20 client.31298 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.085+0000 7f056401bb80 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5579f52050f0 0x5579f5207510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.085+0000 7f056401bb80 4 client.31298 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.085+0000 7f056401bb80 1 -- 172.21.15.12:0/886685009 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579f51ba810 con 0x5579f52050f0 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.093+0000 7f0555bfd700 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5579f52050f0 0x5579f5207510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.099+0000 7f05397fa700 10 client.31298 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:10:13.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.099+0000 7f0555bfd700 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5579f52050f0 0x5579f5207510 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1e0e0ffde27822f server_cookie=23a678c87df180d in_seq=0 out_seq=0 2024-05-09T00:10:13.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.101+0000 7f05397fa700 1 -- 172.21.15.12:0/886685009 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579f51ba810 con 0x5579f52050f0 2024-05-09T00:10:13.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.101+0000 7f05397fa700 10 client.31298 handle_command_reply: tid=0 2024-05-09T00:10:13.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.103+0000 7f056401bb80 1 client.31298 shutdown 2024-05-09T00:10:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.108+0000 7fe8817ffb80 1 Processor -- start 2024-05-09T00:10:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.108+0000 7fe8817ffb80 1 -- start start 2024-05-09T00:10:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.109+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 0x557258de2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.109+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258dd9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.109+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258ddb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.109+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258de9b60 con 0x557258de5270 2024-05-09T00:10:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.109+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258dd1ba0 con 0x557258ddf480 2024-05-09T00:10:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.109+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258de9950 con 0x557258de4950 2024-05-09T00:10:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.110+0000 7fe86f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258ddb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.110+0000 7fe86f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258ddb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47594/0 (socket says 172.21.15.12:47594) 2024-05-09T00:10:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.110+0000 7fe86f7fe700 1 -- 172.21.15.12:0/1167108636 learned_addr learned my addr 172.21.15.12:0/1167108636 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.110+0000 7fe86e7fc700 1 --2- 172.21.15.12:0/1167108636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258dd9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.110+0000 7fe86effd700 1 --2- 172.21.15.12:0/1167108636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 0x557258de2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.110+0000 7fe86f7fe700 1 -- 172.21.15.12:0/1167108636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 msgr2=0x557258dd9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.110+0000 7fe86f7fe700 1 --2- 172.21.15.12:0/1167108636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258dd9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.110+0000 7fe86f7fe700 1 -- 172.21.15.12:0/1167108636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 msgr2=0x557258de2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.110+0000 7fe86f7fe700 1 --2- 172.21.15.12:0/1167108636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 0x557258de2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.110+0000 7fe86f7fe700 1 -- 172.21.15.12:0/1167108636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557258cf5500 con 0x557258de5270 2024-05-09T00:10:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.111+0000 7fe86f7fe700 1 --2- 172.21.15.12:0/1167108636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258ddb520 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe8600094f0 tx=0x7fe860005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4e208b75402ad4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.111+0000 7fe865ffb700 1 -- 172.21.15.12:0/1167108636 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8600054e0 con 0x557258de5270 2024-05-09T00:10:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.111+0000 7fe865ffb700 1 -- 172.21.15.12:0/1167108636 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe860005640 con 0x557258de5270 2024-05-09T00:10:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.111+0000 7fe865ffb700 1 -- 172.21.15.12:0/1167108636 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860021460 con 0x557258de5270 2024-05-09T00:10:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.112+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1167108636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 msgr2=0x557258ddb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.112+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1167108636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258ddb520 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe8600094f0 tx=0x7fe860005cc0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.113+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1167108636 shutdown_connections 2024-05-09T00:10:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.113+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1167108636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258de5270 0x557258ddb520 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.113+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1167108636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258ddf480 0x557258de2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.113+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1167108636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258dd9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.113+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1167108636 >> 172.21.15.12:0/1167108636 conn(0x557258d3d1b0 msgr2=0x557258dd5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.113+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1167108636 shutdown_connections 2024-05-09T00:10:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.113+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1167108636 wait complete. 2024-05-09T00:10:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.113+0000 7fe8817ffb80 1 -- start start 2024-05-09T00:10:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.113+0000 7fe8817ffb80 1 Processor -- start 2024-05-09T00:10:13.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.113+0000 7f056401bb80 1 -- 172.21.15.12:0/886685009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 msgr2=0x5579f51f5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.113+0000 7f056401bb80 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51f5240 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f05480cb1b0 tx=0x7f054809acb0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.114+0000 7f056401bb80 1 -- 172.21.15.12:0/886685009 shutdown_connections 2024-05-09T00:10:13.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.114+0000 7f056401bb80 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5579f52050f0 0x5579f5207510 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.114+0000 7f056401bb80 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51f5240 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.114+0000 7f056401bb80 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 0x5579f51f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.114+0000 7f056401bb80 1 --2- 172.21.15.12:0/886685009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f51f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.114+0000 7f05397fa700 0 client.31298 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:10:13.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.114+0000 7f056401bb80 1 -- 172.21.15.12:0/886685009 >> 172.21.15.12:0/886685009 conn(0x5579f5149210 msgr2=0x5579f51e55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.114+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258ddf480 0x557258defc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.114+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258df0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.114+0000 7fe8817ffb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258de5270 0x557258dedbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.114+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258dee1c0 con 0x557258ddf480 2024-05-09T00:10:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.114+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258dee330 con 0x557258de5270 2024-05-09T00:10:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.114+0000 7fe8817ffb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557258dee4f0 con 0x557258de4950 2024-05-09T00:10:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.114+0000 7fe8817ffb80 10 client.0 _ll_register_callbacks cb 0x557258cd4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:13.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.114+0000 7f056401bb80 1 -- 172.21.15.12:0/886685009 shutdown_connections 2024-05-09T00:10:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.114+0000 7f056401bb80 1 -- 172.21.15.12:0/886685009 wait complete. 2024-05-09T00:10:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.114+0000 7f056401bb80 20 client.31298 trim_cache size 0 max 16384 2024-05-09T00:10:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.114+0000 7fe86effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258ddf480 0x557258defc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.114+0000 7fe86effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258ddf480 0x557258defc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47596/0 (socket says 172.21.15.12:47596) 2024-05-09T00:10:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.114+0000 7fe86effd700 1 -- 172.21.15.12:0/1448627601 learned_addr learned my addr 172.21.15.12:0/1448627601 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.115+0000 7fe86effd700 1 -- 172.21.15.12:0/1448627601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 msgr2=0x557258df0150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:10:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.115+0000 7fe86effd700 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258df0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.115+0000 7fe86effd700 1 -- 172.21.15.12:0/1448627601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258de5270 msgr2=0x557258dedbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:10:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.115+0000 7fe86effd700 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258de5270 0x557258dedbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.115+0000 7fe86effd700 1 -- 172.21.15.12:0/1448627601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557258b5cb00 con 0x557258ddf480 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.115+0000 7fe86effd700 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258ddf480 0x557258defc10 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe86800b010 tx=0x7fe868009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b13a91cc371f46 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.115+0000 7fe8567fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.116+0000 7fe8567fc700 1 -- 172.21.15.12:0/1448627601 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868027630 con 0x557258ddf480 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.116+0000 7fe8567fc700 1 -- 172.21.15.12:0/1448627601 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe868027790 con 0x557258ddf480 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.116+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1448627601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557258c88210 con 0x557258ddf480 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.116+0000 7fe8567fc700 1 -- 172.21.15.12:0/1448627601 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe868030420 con 0x557258ddf480 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.116+0000 7fe8817ffb80 10 client.21903 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.116+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1448627601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x557258bb4c70 con 0x557258ddf480 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.117+0000 7fe8567fc700 1 -- 172.21.15.12:0/1448627601 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557258c88210 con 0x557258ddf480 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.118+0000 7fe8567fc700 1 -- 172.21.15.12:0/1448627601 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe868030580 con 0x557258ddf480 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.118+0000 7fe8817ffb80 10 client.21903 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.118+0000 7fe8817ffb80 10 client.21903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.118+0000 7fe8817ffb80 20 client.21903 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.118+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557258e4eeb0 0x557258e512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.118+0000 7fe8817ffb80 4 client.21903 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:10:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.118+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1448627601 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557258b7dc90 con 0x557258e4eeb0 2024-05-09T00:10:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.119+0000 7fe86e7fc700 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557258e4eeb0 0x557258e512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.119+0000 7fe8567fc700 10 client.21903 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.119+0000 7fe86e7fc700 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557258e4eeb0 0x557258e512d0 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=111f288de4c2b643 server_cookie=776b8d048e41d38a in_seq=0 out_seq=0 2024-05-09T00:10:13.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.138+0000 7f056401bb80 1 Processor -- start 2024-05-09T00:10:13.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.139+0000 7f056401bb80 1 -- start start 2024-05-09T00:10:13.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.139+0000 7f056401bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51eb890 0x5579f51f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.139+0000 7f056401bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f5200c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.139+0000 7f056401bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51e5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.139+0000 7f056401bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f5201d20 con 0x5579f51f1720 2024-05-09T00:10:13.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.139+0000 7f056401bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f51eb6f0 con 0x5579f51eb890 2024-05-09T00:10:13.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.139+0000 7f056401bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f51e6eb0 con 0x5579f51f0e00 2024-05-09T00:10:13.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.140+0000 7f05563fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51e5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.140+0000 7f05563fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51e5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47612/0 (socket says 172.21.15.12:47612) 2024-05-09T00:10:13.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.140+0000 7f05563fe700 1 -- 172.21.15.12:0/237303475 learned_addr learned my addr 172.21.15.12:0/237303475 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.140+0000 7f05563fe700 1 -- 172.21.15.12:0/237303475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 msgr2=0x5579f5200c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:10:13.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.140+0000 7f05563fe700 1 --2- 172.21.15.12:0/237303475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f5200c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.140+0000 7f05563fe700 1 -- 172.21.15.12:0/237303475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51eb890 msgr2=0x5579f51f11e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:10:13.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.140+0000 7f05563fe700 1 --2- 172.21.15.12:0/237303475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51eb890 0x5579f51f11e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.140+0000 7f05563fe700 1 -- 172.21.15.12:0/237303475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f5101560 con 0x5579f51f1720 2024-05-09T00:10:13.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.141+0000 7f05563fe700 1 --2- 172.21.15.12:0/237303475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51e5d20 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f05480975e0 tx=0x7f0548096960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4931f3f5b90fb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.141+0000 7f0544ff9700 1 -- 172.21.15.12:0/237303475 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054809bb80 con 0x5579f51f1720 2024-05-09T00:10:13.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.141+0000 7f0544ff9700 1 -- 172.21.15.12:0/237303475 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f054809bce0 con 0x5579f51f1720 2024-05-09T00:10:13.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.141+0000 7f0544ff9700 1 -- 172.21.15.12:0/237303475 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054809a060 con 0x5579f51f1720 2024-05-09T00:10:13.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.142+0000 7f056401bb80 1 -- 172.21.15.12:0/237303475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 msgr2=0x5579f51e5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.142+0000 7f056401bb80 1 --2- 172.21.15.12:0/237303475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51e5d20 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f05480975e0 tx=0x7f0548096960 comp rx=0 tx=0).stop 2024-05-09T00:10:13.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.142+0000 7f056401bb80 1 -- 172.21.15.12:0/237303475 shutdown_connections 2024-05-09T00:10:13.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.142+0000 7f056401bb80 1 --2- 172.21.15.12:0/237303475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51f1720 0x5579f51e5d20 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.142+0000 7f056401bb80 1 --2- 172.21.15.12:0/237303475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51eb890 0x5579f51f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.142+0000 7f056401bb80 1 --2- 172.21.15.12:0/237303475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f5200c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.142+0000 7f056401bb80 1 -- 172.21.15.12:0/237303475 >> 172.21.15.12:0/237303475 conn(0x5579f5149210 msgr2=0x5579f51e2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.145+0000 7f056401bb80 1 -- 172.21.15.12:0/237303475 shutdown_connections 2024-05-09T00:10:13.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.146+0000 7f056401bb80 1 -- 172.21.15.12:0/237303475 wait complete. 2024-05-09T00:10:13.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.146+0000 7f056401bb80 1 -- start start 2024-05-09T00:10:13.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.146+0000 7f056401bb80 1 Processor -- start 2024-05-09T00:10:13.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.147+0000 7f056401bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 0x5579f51fc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.147+0000 7f056401bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f51fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.147+0000 7f056401bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51fa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.147+0000 7f056401bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f51fada0 con 0x5579f51eb890 2024-05-09T00:10:13.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.147+0000 7f056401bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f51faf10 con 0x5579f51f1720 2024-05-09T00:10:13.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.147+0000 7f056401bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f525a5f0 con 0x5579f51f0e00 2024-05-09T00:10:13.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.148+0000 7f056401bb80 10 client.0 _ll_register_callbacks cb 0x5579f50e0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.148+0000 7f0555bfd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 0x5579f51fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.148+0000 7f05563fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51fa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.148+0000 7f05563fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51fa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39442/0 (socket says 172.21.15.12:39442) 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.148+0000 7f05563fe700 1 -- 172.21.15.12:0/2425202227 learned_addr learned my addr 172.21.15.12:0/2425202227 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.148+0000 7f0555bfd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 0x5579f51fc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47624/0 (socket says 172.21.15.12:47624) 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.148+0000 7f0555bfd700 1 -- 172.21.15.12:0/2425202227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 msgr2=0x5579f51fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.149+0000 7f0555bfd700 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f51fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.149+0000 7f0555bfd700 1 -- 172.21.15.12:0/2425202227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 msgr2=0x5579f51fa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.149+0000 7f0555bfd700 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51fa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.149+0000 7f0555bfd700 1 -- 172.21.15.12:0/2425202227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579f4f68b60 con 0x5579f51eb890 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.149+0000 7f05563fe700 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51fa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.149+0000 7f0555bfd700 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 0x5579f51fc550 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f0548098ec0 tx=0x7f0540004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae157bc550a9847 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.149+0000 7f05397fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:13.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.150+0000 7f05397fa700 1 -- 172.21.15.12:0/2425202227 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540027d10 con 0x5579f51eb890 2024-05-09T00:10:13.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.150+0000 7f056401bb80 1 -- 172.21.15.12:0/2425202227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579f5133ec0 con 0x5579f51eb890 2024-05-09T00:10:13.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.150+0000 7f05397fa700 1 -- 172.21.15.12:0/2425202227 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0540021770 con 0x5579f51eb890 2024-05-09T00:10:13.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.150+0000 7f05397fa700 1 -- 172.21.15.12:0/2425202227 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0540027e70 con 0x5579f51eb890 2024-05-09T00:10:13.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.150+0000 7f056401bb80 10 client.21909 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:13.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.150+0000 7f056401bb80 1 -- 172.21.15.12:0/2425202227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x5579f4fc0cd0 con 0x5579f51eb890 2024-05-09T00:10:13.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.154+0000 7f05397fa700 1 -- 172.21.15.12:0/2425202227 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5579f5133ec0 con 0x5579f51eb890 2024-05-09T00:10:13.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.154+0000 7f05397fa700 1 -- 172.21.15.12:0/2425202227 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f05400277a0 con 0x5579f51eb890 2024-05-09T00:10:13.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.155+0000 7f056401bb80 10 client.21909 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:13.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.155+0000 7f056401bb80 10 client.21909 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:10:13.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.155+0000 7f056401bb80 20 client.21909 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:13.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.155+0000 7f056401bb80 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5579f525bd80 0x5579f525e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.155+0000 7f056401bb80 4 client.21909 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:10:13.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.155+0000 7f056401bb80 1 -- 172.21.15.12:0/2425202227 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5579f4f89cf0 con 0x5579f525bd80 2024-05-09T00:10:13.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.158+0000 7f05553fc700 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5579f525bd80 0x5579f525e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.159+0000 7f05553fc700 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5579f525bd80 0x5579f525e170 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8529fbf6fa3e5b4 server_cookie=28db452903c6a60f in_seq=0 out_seq=0 2024-05-09T00:10:13.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.159+0000 7f05397fa700 10 client.21909 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:10:13.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.159+0000 7f05397fa700 1 -- 172.21.15.12:0/2425202227 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5579f4f89cf0 con 0x5579f525bd80 2024-05-09T00:10:13.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.159+0000 7f05397fa700 10 client.21909 handle_command_reply: tid=0 2024-05-09T00:10:13.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.159+0000 7f056401bb80 1 client.21909 shutdown 2024-05-09T00:10:13.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f056401bb80 1 -- 172.21.15.12:0/2425202227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 msgr2=0x5579f51fc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f056401bb80 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 0x5579f51fc550 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f0548098ec0 tx=0x7f0540004770 comp rx=0 tx=0).stop 2024-05-09T00:10:13.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f056401bb80 1 -- 172.21.15.12:0/2425202227 shutdown_connections 2024-05-09T00:10:13.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f056401bb80 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x5579f525bd80 0x5579f525e170 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f056401bb80 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f51f1720 0x5579f51fa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f056401bb80 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f51eb890 0x5579f51fc550 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f056401bb80 1 --2- 172.21.15.12:0/2425202227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579f51f0e00 0x5579f51fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f056401bb80 1 -- 172.21.15.12:0/2425202227 >> 172.21.15.12:0/2425202227 conn(0x5579f5149210 msgr2=0x5579f51eecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f05397fa700 0 client.21909 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:10:13.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f056401bb80 1 -- 172.21.15.12:0/2425202227 shutdown_connections 2024-05-09T00:10:13.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.160+0000 7f056401bb80 1 -- 172.21.15.12:0/2425202227 wait complete. 2024-05-09T00:10:13.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.161+0000 7f056401bb80 20 client.21909 trim_cache size 0 max 16384 2024-05-09T00:10:13.158 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:10:13.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.161+0000 7f0557e61700 1 -- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f053c0a13d0 msgr2=0x7f053c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.161+0000 7f0557e61700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f053c0a13d0 0x7f053c0a3890 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f054000a850 tx=0x7f0540008040 comp rx=0 tx=0).stop 2024-05-09T00:10:13.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.161+0000 7f0557e61700 1 -- 172.21.15.12:0/2060526883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055010fac0 msgr2=0x7f055019da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.161+0000 7f0557e61700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055010fac0 0x7f055019da90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f054800b780 tx=0x7f054800bb40 comp rx=0 tx=0).stop 2024-05-09T00:10:13.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.162+0000 7f0557e61700 1 -- 172.21.15.12:0/2060526883 shutdown_connections 2024-05-09T00:10:13.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.162+0000 7f0557e61700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f053c0a13d0 0x7f053c0a3890 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.162+0000 7f0557e61700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055010fac0 0x7f055019da90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.162+0000 7f0557e61700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0550107fa0 0x7f05501a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.162+0000 7f0557e61700 1 --2- 172.21.15.12:0/2060526883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f055010f650 0x7f05501a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.162+0000 7f0557e61700 1 -- 172.21.15.12:0/2060526883 >> 172.21.15.12:0/2060526883 conn(0x7f055006bf10 msgr2=0x7f05500718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.162+0000 7f0557e61700 1 -- 172.21.15.12:0/2060526883 shutdown_connections 2024-05-09T00:10:13.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.162+0000 7f0557e61700 1 -- 172.21.15.12:0/2060526883 wait complete. 2024-05-09T00:10:13.191 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:10:13.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:12 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/73601227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:13.272 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:12 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/73601227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:13.314 INFO:tasks.workunit.client.0.smithi012.stdout: 10 347794 28.81 MB/sec execute 488 sec latency 322.497 ms 2024-05-09T00:10:13.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.572+0000 7f1c8964a700 1 -- 172.21.15.12:0/3856530936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c84106290 msgr2=0x7f1c8410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.572+0000 7f1c8964a700 1 --2- 172.21.15.12:0/3856530936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c84106290 0x7f1c8410de10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009f90 tx=0x7f1c7c00b330 comp rx=0 tx=0).stop 2024-05-09T00:10:13.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.572+0000 7f1c8964a700 1 -- 172.21.15.12:0/3856530936 shutdown_connections 2024-05-09T00:10:13.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.572+0000 7f1c8964a700 1 --2- 172.21.15.12:0/3856530936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c84106290 0x7f1c8410de10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.572+0000 7f1c8964a700 1 --2- 172.21.15.12:0/3856530936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c84101170 0x7f1c84101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.572+0000 7f1c8964a700 1 --2- 172.21.15.12:0/3856530936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c84101b20 0x7f1c84105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.572+0000 7f1c8964a700 1 -- 172.21.15.12:0/3856530936 >> 172.21.15.12:0/3856530936 conn(0x7f1c840fc9d0 msgr2=0x7f1c840fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.572+0000 7f1c8964a700 1 -- 172.21.15.12:0/3856530936 shutdown_connections 2024-05-09T00:10:13.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.572+0000 7f1c8964a700 1 -- 172.21.15.12:0/3856530936 wait complete. 2024-05-09T00:10:13.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.573+0000 7f1c8964a700 1 Processor -- start 2024-05-09T00:10:13.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.573+0000 7f1c8964a700 1 -- start start 2024-05-09T00:10:13.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c8964a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c84101170 0x7f1c8410d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c8964a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c84106290 0x7f1c8410dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c8964a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c840731d0 0x7f1c84073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c8964a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c84073c80 con 0x7f1c84101170 2024-05-09T00:10:13.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c8964a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c84073dc0 con 0x7f1c84106290 2024-05-09T00:10:13.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c8964a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c84073f30 con 0x7f1c840731d0 2024-05-09T00:10:13.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c837fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c840731d0 0x7f1c84073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c837fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c840731d0 0x7f1c84073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52588/0 (socket says 172.21.15.12:52588) 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c837fe700 1 -- 172.21.15.12:0/4118360679 learned_addr learned my addr 172.21.15.12:0/4118360679 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c82ffd700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c84101170 0x7f1c8410d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c84106290 0x7f1c8410dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c837fe700 1 -- 172.21.15.12:0/4118360679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c84106290 msgr2=0x7f1c8410dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c837fe700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c84106290 0x7f1c8410dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c837fe700 1 -- 172.21.15.12:0/4118360679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c84101170 msgr2=0x7f1c8410d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c837fe700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c84101170 0x7f1c8410d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.574+0000 7f1c837fe700 1 -- 172.21.15.12:0/4118360679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c01a600 con 0x7f1c840731d0 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.575+0000 7f1c837fe700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c840731d0 0x7f1c84073680 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c007ca0 tx=0x7f1c7c007cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.575+0000 7f1c80ff9700 1 -- 172.21.15.12:0/4118360679 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c00e040 con 0x7f1c840731d0 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.575+0000 7f1c80ff9700 1 -- 172.21.15.12:0/4118360679 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c7c007870 con 0x7f1c840731d0 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.575+0000 7f1c80ff9700 1 -- 172.21.15.12:0/4118360679 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c01caa0 con 0x7f1c840731d0 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.575+0000 7f1c8964a700 1 -- 172.21.15.12:0/4118360679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c009c40 con 0x7f1c840731d0 2024-05-09T00:10:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.575+0000 7f1c8964a700 1 -- 172.21.15.12:0/4118360679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c841b5890 con 0x7f1c840731d0 2024-05-09T00:10:13.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.578+0000 7f1c80ff9700 1 -- 172.21.15.12:0/4118360679 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1c7c01bcb0 con 0x7f1c840731d0 2024-05-09T00:10:13.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.578+0000 7f1c80ff9700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c700a1310 0x7f1c700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.579+0000 7f1c80ff9700 1 -- 172.21.15.12:0/4118360679 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1c7c0ce2c0 con 0x7f1c840731d0 2024-05-09T00:10:13.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.580+0000 7f1c82ffd700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c700a1310 0x7f1c700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.580+0000 7f1c82ffd700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c700a1310 0x7f1c700a37d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c005fd0 tx=0x7f1c6c01a320 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.581+0000 7f1c95804b80 1 Processor -- start 2024-05-09T00:10:13.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.581+0000 7f1c95804b80 1 -- start start 2024-05-09T00:10:13.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.581+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae370e00 0x562eae3711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.581+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 0x562eae36b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.581+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae36b890 0x562eae36bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.581+0000 7f1c95804b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae2c2150 con 0x562eae370e00 2024-05-09T00:10:13.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.581+0000 7f1c95804b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae2bfb10 con 0x562eae371720 2024-05-09T00:10:13.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.581+0000 7f1c95804b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae2bb400 con 0x562eae36b890 2024-05-09T00:10:13.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.582+0000 7f1c7bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 0x562eae36b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.582+0000 7f1c7bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 0x562eae36b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39456/0 (socket says 172.21.15.12:39456) 2024-05-09T00:10:13.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.582+0000 7f1c7bfff700 1 -- 172.21.15.12:0/2112844204 learned_addr learned my addr 172.21.15.12:0/2112844204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.582+0000 7f1c7bfff700 1 -- 172.21.15.12:0/2112844204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae36b890 msgr2=0x562eae36bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:10:13.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.582+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/2112844204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae36b890 0x562eae36bcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.582+0000 7f1c7bfff700 1 -- 172.21.15.12:0/2112844204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae370e00 msgr2=0x562eae3711e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:10:13.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.582+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/2112844204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae370e00 0x562eae3711e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.582+0000 7f1c7bfff700 1 -- 172.21.15.12:0/2112844204 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eae13c3b0 con 0x562eae371720 2024-05-09T00:10:13.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.582+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/2112844204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 0x562eae36b350 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400b7e0 tx=0x7f1c7400baf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2dafd9d831850cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.584+0000 7f1c797fa700 1 -- 172.21.15.12:0/2112844204 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740125f0 con 0x562eae371720 2024-05-09T00:10:13.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.584+0000 7f1c797fa700 1 -- 172.21.15.12:0/2112844204 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c74012c10 con 0x562eae371720 2024-05-09T00:10:13.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.584+0000 7f1c797fa700 1 -- 172.21.15.12:0/2112844204 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c74013df0 con 0x562eae371720 2024-05-09T00:10:13.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.585+0000 7f1c95804b80 1 -- 172.21.15.12:0/2112844204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 msgr2=0x562eae36b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.585+0000 7f1c95804b80 1 --2- 172.21.15.12:0/2112844204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 0x562eae36b350 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400b7e0 tx=0x7f1c7400baf0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8567fc700 1 -- 172.21.15.12:0/1448627601 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557258b7dc90 con 0x557258e4eeb0 2024-05-09T00:10:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8567fc700 10 client.21903 handle_command_reply: tid=0 2024-05-09T00:10:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:10:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:10:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:10:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:10:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 client.21903 shutdown 2024-05-09T00:10:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1448627601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258ddf480 msgr2=0x557258defc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258ddf480 0x557258defc10 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe86800b010 tx=0x7fe868009920 comp rx=0 tx=0).stop 2024-05-09T00:10:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1448627601 shutdown_connections 2024-05-09T00:10:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557258e4eeb0 0x557258e512d0 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557258de5270 0x557258dedbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557258ddf480 0x557258defc10 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 --2- 172.21.15.12:0/1448627601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557258de4950 0x557258df0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1448627601 >> 172.21.15.12:0/1448627601 conn(0x557258d3d1b0 msgr2=0x557258de2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8567fc700 0 client.21903 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1448627601 shutdown_connections 2024-05-09T00:10:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.589+0000 7fe8817ffb80 1 -- 172.21.15.12:0/1448627601 wait complete. 2024-05-09T00:10:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe8817ffb80 20 client.21903 trim_cache size 0 max 16384 2024-05-09T00:10:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.586+0000 7f1c95804b80 1 -- 172.21.15.12:0/2112844204 shutdown_connections 2024-05-09T00:10:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.586+0000 7f1c95804b80 1 --2- 172.21.15.12:0/2112844204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae36b890 0x562eae36bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.586+0000 7f1c95804b80 1 --2- 172.21.15.12:0/2112844204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae370e00 0x562eae3711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.586+0000 7f1c95804b80 1 --2- 172.21.15.12:0/2112844204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 0x562eae36b350 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.586+0000 7f1c95804b80 1 -- 172.21.15.12:0/2112844204 >> 172.21.15.12:0/2112844204 conn(0x562eae2c9210 msgr2=0x562eae365df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.586+0000 7f1c95804b80 1 -- 172.21.15.12:0/2112844204 shutdown_connections 2024-05-09T00:10:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.586+0000 7f1c95804b80 1 -- 172.21.15.12:0/2112844204 wait complete. 2024-05-09T00:10:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.587+0000 7f1c95804b80 1 -- start start 2024-05-09T00:10:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.587+0000 7f1c95804b80 1 Processor -- start 2024-05-09T00:10:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.587+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae36b890 0x562eae374c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.587+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 0x562eae375180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.587+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 0x562eae3756c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.587+0000 7f1c95804b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae257560 con 0x562eae371720 2024-05-09T00:10:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.587+0000 7f1c95804b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae257200 con 0x562eae36b890 2024-05-09T00:10:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.587+0000 7f1c95804b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae13e6a0 con 0x562eae370e00 2024-05-09T00:10:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.587+0000 7f1c95804b80 10 client.0 _ll_register_callbacks cb 0x562eae260b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.588+0000 7f1c82ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae36b890 0x562eae374c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.588+0000 7f1c82ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae36b890 0x562eae374c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39458/0 (socket says 172.21.15.12:39458) 2024-05-09T00:10:13.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.588+0000 7f1c82ffd700 1 -- 172.21.15.12:0/2147736775 learned_addr learned my addr 172.21.15.12:0/2147736775 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.588+0000 7f1c837fe700 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 0x562eae3756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.588+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 0x562eae375180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.588+0000 7f1c82ffd700 1 -- 172.21.15.12:0/2147736775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 msgr2=0x562eae375180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.588+0000 7f1c82ffd700 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 0x562eae375180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.588+0000 7f1c82ffd700 1 -- 172.21.15.12:0/2147736775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 msgr2=0x562eae3756c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.588+0000 7f1c82ffd700 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 0x562eae3756c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.588+0000 7f1c82ffd700 1 -- 172.21.15.12:0/2147736775 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7400e040 con 0x562eae36b890 2024-05-09T00:10:13.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.589+0000 7f1c82ffd700 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae36b890 0x562eae374c40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0108e0 tx=0x7f1c6c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6149b02bd474db1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.589+0000 7f1c6a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:10:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe875645700 1 -- 172.21.15.12:0/2569104225 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe85c0a1310 msgr2=0x7fe85c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe875645700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe85c0a1310 0x7fe85c0a37d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x557258dda790 tx=0x7fe86800b040 comp rx=0 tx=0).stop 2024-05-09T00:10:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe875645700 1 -- 172.21.15.12:0/2569104225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe870116840 msgr2=0x7fe8701103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe875645700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe870116840 0x7fe8701103f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe85800d980 tx=0x7fe85800dd40 comp rx=0 tx=0).stop 2024-05-09T00:10:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe875645700 1 -- 172.21.15.12:0/2569104225 shutdown_connections 2024-05-09T00:10:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe875645700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe85c0a1310 0x7fe85c0a37d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe875645700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe870110930 0x7fe870110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe875645700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe870106210 0x7fe870116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe875645700 1 --2- 172.21.15.12:0/2569104225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe870116840 0x7fe8701103f0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.590+0000 7fe875645700 1 -- 172.21.15.12:0/2569104225 >> 172.21.15.12:0/2569104225 conn(0x7fe8700fc9d0 msgr2=0x7fe8700fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.591+0000 7fe875645700 1 -- 172.21.15.12:0/2569104225 shutdown_connections 2024-05-09T00:10:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:13.591+0000 7fe875645700 1 -- 172.21.15.12:0/2569104225 wait complete. 2024-05-09T00:10:13.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.592+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/2147736775 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00f750 con 0x562eae36b890 2024-05-09T00:10:13.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.592+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/2147736775 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1c6c00f8b0 con 0x562eae36b890 2024-05-09T00:10:13.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.592+0000 7f1c95804b80 1 -- 172.21.15.12:0/2147736775 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562eae214270 con 0x562eae36b890 2024-05-09T00:10:13.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.593+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/2147736775 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c6c00fa70 con 0x562eae36b890 2024-05-09T00:10:13.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.593+0000 7f1c95804b80 10 client.31313 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:13.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.593+0000 7f1c95804b80 1 -- 172.21.15.12:0/2147736775 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x562eae0e8b60 con 0x562eae36b890 2024-05-09T00:10:13.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.594+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/2147736775 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562eae214270 con 0x562eae36b890 2024-05-09T00:10:13.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.594+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/2147736775 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c6c02a440 con 0x562eae36b890 2024-05-09T00:10:13.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.594+0000 7f1c95804b80 10 client.31313 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:13.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.594+0000 7f1c95804b80 10 client.31313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:13.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.594+0000 7f1c95804b80 20 client.31313 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:13.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.594+0000 7f1c95804b80 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562eae385470 0x562eae387890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.594+0000 7f1c95804b80 4 client.31313 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:13.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.594+0000 7f1c95804b80 1 -- 172.21.15.12:0/2147736775 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562eae33a810 con 0x562eae385470 2024-05-09T00:10:13.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.609+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562eae385470 0x562eae387890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.627+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562eae385470 0x562eae387890 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc867cc450bc7377 server_cookie=c72845b52d532218 in_seq=0 out_seq=0 2024-05-09T00:10:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.630+0000 7f1c6a7fc700 10 client.31313 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:13.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.635+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/2147736775 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562eae33a810 con 0x562eae385470 2024-05-09T00:10:13.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.635+0000 7f1c6a7fc700 10 client.31313 handle_command_reply: tid=0 2024-05-09T00:10:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.643+0000 7f1c95804b80 1 client.31313 shutdown 2024-05-09T00:10:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.643+0000 7f1c95804b80 1 -- 172.21.15.12:0/2147736775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae36b890 msgr2=0x562eae374c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.643+0000 7f1c95804b80 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae36b890 0x562eae374c40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0108e0 tx=0x7f1c6c004750 comp rx=0 tx=0).stop 2024-05-09T00:10:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.643+0000 7f1c95804b80 1 -- 172.21.15.12:0/2147736775 shutdown_connections 2024-05-09T00:10:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.643+0000 7f1c95804b80 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 0x562eae3756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.643+0000 7f1c95804b80 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562eae385470 0x562eae387890 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.643+0000 7f1c95804b80 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae36b890 0x562eae374c40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.643+0000 7f1c95804b80 1 --2- 172.21.15.12:0/2147736775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 0x562eae375180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.643+0000 7f1c95804b80 1 -- 172.21.15.12:0/2147736775 >> 172.21.15.12:0/2147736775 conn(0x562eae2c9210 msgr2=0x562eae3655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.643+0000 7f1c6a7fc700 0 client.31313 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.644+0000 7f1c95804b80 1 -- 172.21.15.12:0/2147736775 shutdown_connections 2024-05-09T00:10:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.644+0000 7f1c95804b80 1 -- 172.21.15.12:0/2147736775 wait complete. 2024-05-09T00:10:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.648+0000 7f1c95804b80 20 client.31313 trim_cache size 0 max 16384 2024-05-09T00:10:13.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.672+0000 7f1c95804b80 1 Processor -- start 2024-05-09T00:10:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.673+0000 7f1c95804b80 1 -- start start 2024-05-09T00:10:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.673+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae36b890 0x562eae36b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.673+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae370e00 0x562eae365d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.673+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 0x562eae37c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.673+0000 7f1c95804b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae36e940 con 0x562eae371720 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.673+0000 7f1c95804b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae375ba0 con 0x562eae370e00 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.673+0000 7f1c95804b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae38ca40 con 0x562eae36b890 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.674+0000 7f1c82ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae36b890 0x562eae36b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.674+0000 7f1c837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 0x562eae37c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.674+0000 7f1c837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 0x562eae37c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47652/0 (socket says 172.21.15.12:47652) 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.674+0000 7f1c837fe700 1 -- 172.21.15.12:0/661303340 learned_addr learned my addr 172.21.15.12:0/661303340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.674+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/661303340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae370e00 0x562eae365d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.674+0000 7f1c7bfff700 1 -- 172.21.15.12:0/661303340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae36b890 msgr2=0x562eae36b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.674+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/661303340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae36b890 0x562eae36b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.674+0000 7f1c7bfff700 1 -- 172.21.15.12:0/661303340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 msgr2=0x562eae37c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.674+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/661303340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 0x562eae37c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.674+0000 7f1c7bfff700 1 -- 172.21.15.12:0/661303340 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eae281560 con 0x562eae370e00 2024-05-09T00:10:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.675+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/661303340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae370e00 0x562eae365d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1c74013040 tx=0x7f1c7400b2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb5c730397976a99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.675+0000 7f1c79ffb700 1 -- 172.21.15.12:0/661303340 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c74012ea0 con 0x562eae370e00 2024-05-09T00:10:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.675+0000 7f1c79ffb700 1 -- 172.21.15.12:0/661303340 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c7402a020 con 0x562eae370e00 2024-05-09T00:10:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.675+0000 7f1c79ffb700 1 -- 172.21.15.12:0/661303340 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740267b0 con 0x562eae370e00 2024-05-09T00:10:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.676+0000 7f1c95804b80 1 -- 172.21.15.12:0/661303340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae370e00 msgr2=0x562eae365d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.676+0000 7f1c95804b80 1 --2- 172.21.15.12:0/661303340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae370e00 0x562eae365d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1c74013040 tx=0x7f1c7400b2e0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.677+0000 7f1c95804b80 1 -- 172.21.15.12:0/661303340 shutdown_connections 2024-05-09T00:10:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.677+0000 7f1c95804b80 1 --2- 172.21.15.12:0/661303340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae371720 0x562eae37c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.677+0000 7f1c95804b80 1 --2- 172.21.15.12:0/661303340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae36b890 0x562eae36b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.677+0000 7f1c95804b80 1 --2- 172.21.15.12:0/661303340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae370e00 0x562eae365d20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.677+0000 7f1c95804b80 1 -- 172.21.15.12:0/661303340 >> 172.21.15.12:0/661303340 conn(0x562eae2c9210 msgr2=0x562eae36ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.677+0000 7f1c95804b80 1 -- 172.21.15.12:0/661303340 shutdown_connections 2024-05-09T00:10:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.677+0000 7f1c95804b80 1 -- 172.21.15.12:0/661303340 wait complete. 2024-05-09T00:10:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.677+0000 7f1c95804b80 1 -- start start 2024-05-09T00:10:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.677+0000 7f1c95804b80 1 Processor -- start 2024-05-09T00:10:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.678+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae36b890 0x562eae3cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.678+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 0x562eae3cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.678+0000 7f1c95804b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 0x562eae3d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.678+0000 7f1c95804b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae3daaf0 con 0x562eae36b890 2024-05-09T00:10:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.678+0000 7f1c95804b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae3dac60 con 0x562eae371720 2024-05-09T00:10:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.678+0000 7f1c95804b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eae3dae20 con 0x562eae370e00 2024-05-09T00:10:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.678+0000 7f1c95804b80 10 client.0 _ll_register_callbacks cb 0x562eae260b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.678+0000 7f1c7bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 0x562eae3cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.678+0000 7f1c7bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 0x562eae3cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52610/0 (socket says 172.21.15.12:52610) 2024-05-09T00:10:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.678+0000 7f1c7bfff700 1 -- 172.21.15.12:0/1098062063 learned_addr learned my addr 172.21.15.12:0/1098062063 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.679+0000 7f1c7bfff700 1 -- 172.21.15.12:0/1098062063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 msgr2=0x562eae3d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.679+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 0x562eae3d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.679+0000 7f1c7bfff700 1 -- 172.21.15.12:0/1098062063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae36b890 msgr2=0x562eae3cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.679+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae36b890 0x562eae3cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.679+0000 7f1c7bfff700 1 -- 172.21.15.12:0/1098062063 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562eae13c3b0 con 0x562eae370e00 2024-05-09T00:10:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.679+0000 7f1c7bfff700 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 0x562eae3cfe00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1c74008010 tx=0x7f1c74012ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e938b79771fd59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.679+0000 7f1c6a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:10:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.679+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/1098062063 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c74012800 con 0x562eae370e00 2024-05-09T00:10:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.680+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/1098062063 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1c7402b030 con 0x562eae370e00 2024-05-09T00:10:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.680+0000 7f1c95804b80 1 -- 172.21.15.12:0/1098062063 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c74012c10 con 0x562eae370e00 2024-05-09T00:10:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.681+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/1098062063 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c74012960 con 0x562eae370e00 2024-05-09T00:10:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.683+0000 7f1c95804b80 10 client.31582 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.683+0000 7f1c95804b80 1 -- 172.21.15.12:0/1098062063 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x562eae140cd0 con 0x562eae370e00 2024-05-09T00:10:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.687+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/1098062063 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c74012c10 con 0x562eae370e00 2024-05-09T00:10:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.688+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/1098062063 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c7403a050 con 0x562eae370e00 2024-05-09T00:10:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.688+0000 7f1c95804b80 10 client.31582 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.688+0000 7f1c95804b80 10 client.31582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.688+0000 7f1c95804b80 20 client.31582 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.688+0000 7f1c95804b80 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562eae3dc110 0x562eae3de500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.688+0000 7f1c95804b80 4 client.31582 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:10:13.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.688+0000 7f1c95804b80 1 -- 172.21.15.12:0/1098062063 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562eae2b3ec0 con 0x562eae3dc110 2024-05-09T00:10:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.688+0000 7f1c82ffd700 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562eae3dc110 0x562eae3de500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.689+0000 7f1c6a7fc700 10 client.31582 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.689+0000 7f1c82ffd700 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562eae3dc110 0x562eae3de500 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ff48a11bf725e5 server_cookie=8e6e0f757dbbf61b in_seq=0 out_seq=0 2024-05-09T00:10:13.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.697+0000 7f1c6a7fc700 1 -- 172.21.15.12:0/1098062063 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562eae2b3ec0 con 0x562eae3dc110 2024-05-09T00:10:13.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.697+0000 7f1c6a7fc700 10 client.31582 handle_command_reply: tid=0 2024-05-09T00:10:13.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.697+0000 7f1c95804b80 1 client.31582 shutdown 2024-05-09T00:10:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 1 -- 172.21.15.12:0/1098062063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 msgr2=0x562eae3cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 0x562eae3cfe00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1c74008010 tx=0x7f1c74012ea0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 1 -- 172.21.15.12:0/1098062063 shutdown_connections 2024-05-09T00:10:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562eae3dc110 0x562eae3de500 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c6a7fc700 0 client.31582 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562eae371720 0x562eae3d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562eae36b890 0x562eae3cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 1 --2- 172.21.15.12:0/1098062063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562eae370e00 0x562eae3cfe00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 1 -- 172.21.15.12:0/1098062063 >> 172.21.15.12:0/1098062063 conn(0x562eae2c9210 msgr2=0x562eae36fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 1 -- 172.21.15.12:0/1098062063 shutdown_connections 2024-05-09T00:10:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 1 -- 172.21.15.12:0/1098062063 wait complete. 2024-05-09T00:10:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.698+0000 7f1c95804b80 20 client.31582 trim_cache size 0 max 16384 2024-05-09T00:10:13.696 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:10:13.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 -- 172.21.15.12:0/4118360679 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c700a1310 msgr2=0x7f1c700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c700a1310 0x7f1c700a37d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c005fd0 tx=0x7f1c6c01a320 comp rx=0 tx=0).stop 2024-05-09T00:10:13.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 -- 172.21.15.12:0/4118360679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c840731d0 msgr2=0x7f1c84073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:13.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c840731d0 0x7f1c84073680 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c007ca0 tx=0x7f1c7c007cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 -- 172.21.15.12:0/4118360679 shutdown_connections 2024-05-09T00:10:13.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f1c700a1310 0x7f1c700a37d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c840731d0 0x7f1c84073680 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c84101170 0x7f1c8410d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 --2- 172.21.15.12:0/4118360679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c84106290 0x7f1c8410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:13.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 -- 172.21.15.12:0/4118360679 >> 172.21.15.12:0/4118360679 conn(0x7f1c840fc9d0 msgr2=0x7f1c840fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:13.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 -- 172.21.15.12:0/4118360679 shutdown_connections 2024-05-09T00:10:13.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:13.699+0000 7f1c8964a700 1 -- 172.21.15.12:0/4118360679 wait complete. 2024-05-09T00:10:14.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:13 smithi012 ceph-mon[42503]: pgmap v766: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.4 MiB/s wr, 501 op/s 2024-05-09T00:10:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:13 smithi204 ceph-mon[47373]: pgmap v766: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.4 MiB/s wr, 501 op/s 2024-05-09T00:10:14.266 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:13 smithi160 ceph-mon[46152]: pgmap v766: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.4 MiB/s wr, 501 op/s 2024-05-09T00:10:14.308 INFO:tasks.workunit.client.0.smithi012.stdout: 10 348140 28.78 MB/sec execute 489 sec latency 378.306 ms 2024-05-09T00:10:14.715 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:10:14.716 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:10:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.895+0000 7f6a70b7c700 1 -- 172.21.15.12:0/2034975133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0693a0 msgr2=0x7f6a6c069780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.895+0000 7f6a70b7c700 1 --2- 172.21.15.12:0/2034975133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0693a0 0x7f6a6c069780 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009990 tx=0x7f6a5c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.895+0000 7f6a70b7c700 1 -- 172.21.15.12:0/2034975133 shutdown_connections 2024-05-09T00:10:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.895+0000 7f6a70b7c700 1 --2- 172.21.15.12:0/2034975133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c1062a0 0x7f6a6c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.895+0000 7f6a70b7c700 1 --2- 172.21.15.12:0/2034975133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0693a0 0x7f6a6c069780 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.895+0000 7f6a70b7c700 1 --2- 172.21.15.12:0/2034975133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c069d50 0x7f6a6c105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.895+0000 7f6a70b7c700 1 -- 172.21.15.12:0/2034975133 >> 172.21.15.12:0/2034975133 conn(0x7f6a6c076ba0 msgr2=0x7f6a6c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.895+0000 7f6a70b7c700 1 -- 172.21.15.12:0/2034975133 shutdown_connections 2024-05-09T00:10:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.895+0000 7f6a70b7c700 1 -- 172.21.15.12:0/2034975133 wait complete. 2024-05-09T00:10:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.896+0000 7f6a70b7c700 1 Processor -- start 2024-05-09T00:10:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.896+0000 7f6a70b7c700 1 -- start start 2024-05-09T00:10:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a70b7c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0693a0 0x7f6a6c1001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a70b7c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c1062a0 0x7f6a6c100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a70b7c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c100c40 0x7f6a6c1a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a70b7c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c1a5b90 con 0x7f6a6c0693a0 2024-05-09T00:10:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a70b7c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c1a5d00 con 0x7f6a6c1062a0 2024-05-09T00:10:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a70b7c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c1a5ec0 con 0x7f6a6c100c40 2024-05-09T00:10:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a6ad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c100c40 0x7f6a6c1a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a6ad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c100c40 0x7f6a6c1a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52626/0 (socket says 172.21.15.12:52626) 2024-05-09T00:10:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a6ad9d700 1 -- 172.21.15.12:0/1541221204 learned_addr learned my addr 172.21.15.12:0/1541221204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0693a0 0x7f6a6c1001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a69d9b700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c1062a0 0x7f6a6c100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a6ad9d700 1 -- 172.21.15.12:0/1541221204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c1062a0 msgr2=0x7f6a6c100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.897+0000 7f6a6ad9d700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c1062a0 0x7f6a6c100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.898+0000 7f6a6ad9d700 1 -- 172.21.15.12:0/1541221204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0693a0 msgr2=0x7f6a6c1001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.898+0000 7f6a6ad9d700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0693a0 0x7f6a6c1001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.898+0000 7f6a6ad9d700 1 -- 172.21.15.12:0/1541221204 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0095f0 con 0x7f6a6c100c40 2024-05-09T00:10:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.898+0000 7f6a6ad9d700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c100c40 0x7f6a6c1a5650 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400ef30 tx=0x7f6a6400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.898+0000 7f6a5b7fe700 1 -- 172.21.15.12:0/1541221204 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64018070 con 0x7f6a6c100c40 2024-05-09T00:10:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.898+0000 7f6a5b7fe700 1 -- 172.21.15.12:0/1541221204 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a6400f040 con 0x7f6a6c100c40 2024-05-09T00:10:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.898+0000 7f6a5b7fe700 1 -- 172.21.15.12:0/1541221204 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400cd00 con 0x7f6a6c100c40 2024-05-09T00:10:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.898+0000 7f6a70b7c700 1 -- 172.21.15.12:0/1541221204 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c1a6140 con 0x7f6a6c100c40 2024-05-09T00:10:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.898+0000 7f6a70b7c700 1 -- 172.21.15.12:0/1541221204 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c1a6600 con 0x7f6a6c100c40 2024-05-09T00:10:14.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.901+0000 7f6a5b7fe700 1 -- 172.21.15.12:0/1541221204 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6a64003a00 con 0x7f6a6c100c40 2024-05-09T00:10:14.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.902+0000 7f6a5b7fe700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6a540a1340 0x7f6a540a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.902+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6a540a1340 0x7f6a540a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.902+0000 7f6a5b7fe700 1 -- 172.21.15.12:0/1541221204 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6a640cc7c0 con 0x7f6a6c100c40 2024-05-09T00:10:14.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.902+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6a540a1340 0x7f6a540a3800 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00b2e0 tx=0x7f6a5c005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:14.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a7cd36b80 1 Processor -- start 2024-05-09T00:10:14.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a7cd36b80 1 -- start start 2024-05-09T00:10:14.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3fe80 0x55a76aa40260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa3a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3a910 0x55a76aa3ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76a9911d0 con 0x55a76aa407a0 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76a98eb90 con 0x55a76aa3fe80 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76a98a480 con 0x55a76aa3a910 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a69d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa3a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a6a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3fe80 0x55a76aa40260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a69d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa3a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47692/0 (socket says 172.21.15.12:47692) 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.903+0000 7f6a69d9b700 1 -- 172.21.15.12:0/1810053750 learned_addr learned my addr 172.21.15.12:0/1810053750 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.904+0000 7f6a6ad9d700 1 --2- 172.21.15.12:0/1810053750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3a910 0x55a76aa3ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.904+0000 7f6a69d9b700 1 -- 172.21.15.12:0/1810053750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3a910 msgr2=0x55a76aa3ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.904+0000 7f6a69d9b700 1 --2- 172.21.15.12:0/1810053750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3a910 0x55a76aa3ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.904+0000 7f6a69d9b700 1 -- 172.21.15.12:0/1810053750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3fe80 msgr2=0x55a76aa40260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.904+0000 7f6a69d9b700 1 --2- 172.21.15.12:0/1810053750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3fe80 0x55a76aa40260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.904+0000 7f6a69d9b700 1 -- 172.21.15.12:0/1810053750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a76a80b430 con 0x55a76aa407a0 2024-05-09T00:10:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.904+0000 7f6a69d9b700 1 --2- 172.21.15.12:0/1810053750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa3a3d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000bcf0 tx=0x7f6a600103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcee33f28173f12d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.904+0000 7f6a58ff9700 1 -- 172.21.15.12:0/1810053750 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60010d10 con 0x55a76aa407a0 2024-05-09T00:10:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.904+0000 7f6a58ff9700 1 -- 172.21.15.12:0/1810053750 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a60015070 con 0x55a76aa407a0 2024-05-09T00:10:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.904+0000 7f6a58ff9700 1 -- 172.21.15.12:0/1810053750 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60005020 con 0x55a76aa407a0 2024-05-09T00:10:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.905+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1810053750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 msgr2=0x55a76aa3a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.905+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/1810053750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa3a3d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000bcf0 tx=0x7f6a600103d0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.905+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1810053750 shutdown_connections 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.905+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/1810053750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3a910 0x55a76aa3ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.905+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/1810053750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3fe80 0x55a76aa40260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.905+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/1810053750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa3a3d0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.905+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1810053750 >> 172.21.15.12:0/1810053750 conn(0x55a76a998290 msgr2=0x55a76aa34e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.905+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1810053750 shutdown_connections 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.905+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1810053750 wait complete. 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a7cd36b80 1 -- start start 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a7cd36b80 1 Processor -- start 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3a910 0x55a76aa43ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 0x55a76aa440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa44620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76a9265e0 con 0x55a76aa407a0 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76a926280 con 0x55a76aa3a910 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76a80d720 con 0x55a76aa3fe80 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a7cd36b80 10 client.0 _ll_register_callbacks cb 0x55a76a92fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a6ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa44620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a69d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 0x55a76aa440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a69d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 0x55a76aa440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52642/0 (socket says 172.21.15.12:52642) 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a6ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa44620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47700/0 (socket says 172.21.15.12:47700) 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.906+0000 7f6a69d9b700 1 -- 172.21.15.12:0/913638741 learned_addr learned my addr 172.21.15.12:0/913638741 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3a910 0x55a76aa43ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a6ad9d700 1 -- 172.21.15.12:0/913638741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 msgr2=0x55a76aa440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:14.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a6ad9d700 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 0x55a76aa440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a6ad9d700 1 -- 172.21.15.12:0/913638741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3a910 msgr2=0x55a76aa43ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:14.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a6ad9d700 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3a910 0x55a76aa43ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a6ad9d700 1 -- 172.21.15.12:0/913638741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6000b920 con 0x55a76aa407a0 2024-05-09T00:10:14.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a69d9b700 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 0x55a76aa440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:14.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a6ad9d700 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa44620 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f6a640c9ea0 tx=0x7f6a64099250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568ff2abaecb891c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:14.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:14.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a51ffb700 1 -- 172.21.15.12:0/913638741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64099d50 con 0x55a76aa407a0 2024-05-09T00:10:14.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a51ffb700 1 -- 172.21.15.12:0/913638741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6a64099ff0 con 0x55a76aa407a0 2024-05-09T00:10:14.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/913638741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a76a8e32f0 con 0x55a76aa407a0 2024-05-09T00:10:14.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.907+0000 7f6a51ffb700 1 -- 172.21.15.12:0/913638741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a6400ce60 con 0x55a76aa407a0 2024-05-09T00:10:14.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a7cd36b80 10 client.21933 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:14.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/913638741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55a76a7b7be0 con 0x55a76aa407a0 2024-05-09T00:10:14.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a51ffb700 1 -- 172.21.15.12:0/913638741 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a640b6080 con 0x55a76aa407a0 2024-05-09T00:10:14.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a51ffb700 1 -- 172.21.15.12:0/913638741 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6a64098de0 con 0x55a76aa407a0 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a7cd36b80 10 client.21933 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a7cd36b80 10 client.21933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a7cd36b80 20 client.21933 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a76aa54360 0x55a76aa56780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a7cd36b80 4 client.21933 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/913638741 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a76aa09890 con 0x55a76aa54360 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.908+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a76aa54360 0x55a76aa56780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.909+0000 7f6a51ffb700 10 client.21933 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.909+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a76aa54360 0x55a76aa56780 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbd926d70ba8a0bc server_cookie=48ce56d9a217b8a6 in_seq=0 out_seq=0 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a51ffb700 1 -- 172.21.15.12:0/913638741 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a76aa09890 con 0x55a76aa54360 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a51ffb700 10 client.21933 handle_command_reply: tid=0 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 client.21933 shutdown 2024-05-09T00:10:14.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/913638741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 msgr2=0x55a76aa44620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa44620 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f6a640c9ea0 tx=0x7f6a64099250 comp rx=0 tx=0).stop 2024-05-09T00:10:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/913638741 shutdown_connections 2024-05-09T00:10:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa407a0 0x55a76aa44620 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a76aa54360 0x55a76aa56780 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3a910 0x55a76aa43ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/913638741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 0x55a76aa440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a51ffb700 0 client.21933 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/913638741 >> 172.21.15.12:0/913638741 conn(0x55a76a998290 msgr2=0x55a76aa34620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/913638741 shutdown_connections 2024-05-09T00:10:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/913638741 wait complete. 2024-05-09T00:10:14.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.915+0000 7f6a7cd36b80 20 client.21933 trim_cache size 0 max 16384 2024-05-09T00:10:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a7cd36b80 1 Processor -- start 2024-05-09T00:10:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a7cd36b80 1 -- start start 2024-05-09T00:10:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa3f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3fe80 0x55a76aa34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa407a0 0x55a76aa42b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76aa51970 con 0x55a76aa3a910 2024-05-09T00:10:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76aa31e40 con 0x55a76aa3fe80 2024-05-09T00:10:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76aa3a750 con 0x55a76aa407a0 2024-05-09T00:10:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a6a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa3f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a6a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa3f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47710/0 (socket says 172.21.15.12:47710) 2024-05-09T00:10:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a6a59c700 1 -- 172.21.15.12:0/4287136601 learned_addr learned my addr 172.21.15.12:0/4287136601 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a6ad9d700 1 --2- 172.21.15.12:0/4287136601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa407a0 0x55a76aa42b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.941+0000 7f6a69d9b700 1 --2- 172.21.15.12:0/4287136601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3fe80 0x55a76aa34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.942+0000 7f6a6a59c700 1 -- 172.21.15.12:0/4287136601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa407a0 msgr2=0x55a76aa42b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.942+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/4287136601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa407a0 0x55a76aa42b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.942+0000 7f6a6a59c700 1 -- 172.21.15.12:0/4287136601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3fe80 msgr2=0x55a76aa34da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.942+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/4287136601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3fe80 0x55a76aa34da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.942+0000 7f6a6a59c700 1 -- 172.21.15.12:0/4287136601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a76aa31560 con 0x55a76aa3a910 2024-05-09T00:10:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.942+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/4287136601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa3f520 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009990 tx=0x7f6a5c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12881efa033336ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.942+0000 7f6a597fa700 1 -- 172.21.15.12:0/4287136601 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0351b0 con 0x55a76aa3a910 2024-05-09T00:10:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.942+0000 7f6a597fa700 1 -- 172.21.15.12:0/4287136601 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a5c01e040 con 0x55a76aa3a910 2024-05-09T00:10:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.942+0000 7f6a597fa700 1 -- 172.21.15.12:0/4287136601 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c004c90 con 0x55a76aa3a910 2024-05-09T00:10:14.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.943+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/4287136601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 msgr2=0x55a76aa3f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.943+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/4287136601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa3f520 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009990 tx=0x7f6a5c020000 comp rx=0 tx=0).stop 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.943+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/4287136601 shutdown_connections 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.943+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/4287136601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa407a0 0x55a76aa42b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.943+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/4287136601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa3f520 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.943+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/4287136601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa3fe80 0x55a76aa34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.943+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/4287136601 >> 172.21.15.12:0/4287136601 conn(0x55a76a998290 msgr2=0x55a76aa42470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.943+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/4287136601 shutdown_connections 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.943+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/4287136601 wait complete. 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a7cd36b80 1 -- start start 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a7cd36b80 1 Processor -- start 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa49e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 0x55a76aa4a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a7cd36b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa407a0 0x55a76aa4a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76aaaa870 con 0x55a76aa3a910 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76aaaa9b0 con 0x55a76aa407a0 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a7cd36b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76aaaab70 con 0x55a76aa3fe80 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a7cd36b80 10 client.0 _ll_register_callbacks cb 0x55a76a92fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a6a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa49e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a6a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa49e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47720/0 (socket says 172.21.15.12:47720) 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a6a59c700 1 -- 172.21.15.12:0/1439465423 learned_addr learned my addr 172.21.15.12:0/1439465423 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a69d9b700 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 0x55a76aa4a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.944+0000 7f6a6ad9d700 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa407a0 0x55a76aa4a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a6a59c700 1 -- 172.21.15.12:0/1439465423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 msgr2=0x55a76aa4a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 0x55a76aa4a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a6a59c700 1 -- 172.21.15.12:0/1439465423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa407a0 msgr2=0x55a76aa4a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa407a0 0x55a76aa4a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a6a59c700 1 -- 172.21.15.12:0/1439465423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c10e690 con 0x55a76aa3a910 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a6a59c700 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa49e20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00b880 tx=0x7f6a5c009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cea8d08e21e2fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a51ffb700 1 -- 172.21.15.12:0/1439465423 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0048d0 con 0x55a76aa3a910 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a51ffb700 1 -- 172.21.15.12:0/1439465423 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6a5c004a30 con 0x55a76aa3a910 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1439465423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a60015070 con 0x55a76aa3a910 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a51ffb700 1 -- 172.21.15.12:0/1439465423 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a5c004bf0 con 0x55a76aa3a910 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a7cd36b80 10 client.21939 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:14.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.945+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1439465423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55a76a9505e0 con 0x55a76aa3a910 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a51ffb700 1 -- 172.21.15.12:0/1439465423 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a5c04c020 con 0x55a76aa3a910 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a51ffb700 1 -- 172.21.15.12:0/1439465423 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6a5c00f600 con 0x55a76aa3a910 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a7cd36b80 10 client.21939 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a7cd36b80 10 client.21939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a7cd36b80 20 client.21939 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a76aaabf50 0x55a76aaae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a7cd36b80 4 client.21939 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1439465423 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a76a982f40 con 0x55a76aaabf50 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a69d9b700 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a76aaabf50 0x55a76aaae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a51ffb700 10 client.21939 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:14.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.946+0000 7f6a69d9b700 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a76aaabf50 0x55a76aaae370 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eec5b217af4b4859 server_cookie=f72ed47dea49b61b in_seq=0 out_seq=0 2024-05-09T00:10:14.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.947+0000 7f6a51ffb700 1 -- 172.21.15.12:0/1439465423 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a76a982f40 con 0x55a76aaabf50 2024-05-09T00:10:14.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.947+0000 7f6a51ffb700 10 client.21939 handle_command_reply: tid=0 2024-05-09T00:10:14.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 client.21939 shutdown 2024-05-09T00:10:14.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1439465423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 msgr2=0x55a76aa49e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:14.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa49e20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00b880 tx=0x7f6a5c009380 comp rx=0 tx=0).stop 2024-05-09T00:10:14.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1439465423 shutdown_connections 2024-05-09T00:10:14.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a76aaabf50 0x55a76aaae370 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a76aa407a0 0x55a76aa4a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76aa3a910 0x55a76aa49e20 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a51ffb700 0 client.21939 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:14.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 --2- 172.21.15.12:0/1439465423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a76aa3fe80 0x55a76aa4a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1439465423 >> 172.21.15.12:0/1439465423 conn(0x55a76a998290 msgr2=0x55a76aa3ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:14.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1439465423 shutdown_connections 2024-05-09T00:10:14.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 1 -- 172.21.15.12:0/1439465423 wait complete. 2024-05-09T00:10:14.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.948+0000 7f6a7cd36b80 20 client.21939 trim_cache size 0 max 16384 2024-05-09T00:10:14.952 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:10:14.952 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:10:14.952 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "78810daa-60d9-4ff3-86e1-1e4ee43d8af0", 2024-05-09T00:10:14.952 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:10:14.952 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:10:14.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 -- 172.21.15.12:0/1541221204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6a540a1340 msgr2=0x7f6a540a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:14.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6a540a1340 0x7f6a540a3800 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00b2e0 tx=0x7f6a5c005960 comp rx=0 tx=0).stop 2024-05-09T00:10:14.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 -- 172.21.15.12:0/1541221204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c100c40 msgr2=0x7f6a6c1a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:14.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c100c40 0x7f6a6c1a5650 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400ef30 tx=0x7f6a6400c3a0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 -- 172.21.15.12:0/1541221204 shutdown_connections 2024-05-09T00:10:14.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6a540a1340 0x7f6a540a3800 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a6c100c40 0x7f6a6c1a5650 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a6c0693a0 0x7f6a6c1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 --2- 172.21.15.12:0/1541221204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a6c1062a0 0x7f6a6c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:14.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 -- 172.21.15.12:0/1541221204 >> 172.21.15.12:0/1541221204 conn(0x7f6a6c076ba0 msgr2=0x7f6a6c0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:14.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 -- 172.21.15.12:0/1541221204 shutdown_connections 2024-05-09T00:10:14.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:14.949+0000 7f6a70b7c700 1 -- 172.21.15.12:0/1541221204 wait complete. 2024-05-09T00:10:14.961 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:10:15.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.157+0000 7fad9eb97700 1 -- 172.21.15.12:0/1696687959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad98105e60 msgr2=0x7fad98069190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:15.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.157+0000 7fad9eb97700 1 --2- 172.21.15.12:0/1696687959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad98105e60 0x7fad98069190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fad840098f0 tx=0x7fad84009c00 comp rx=0 tx=0).stop 2024-05-09T00:10:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.157+0000 7fad9eb97700 1 -- 172.21.15.12:0/1696687959 shutdown_connections 2024-05-09T00:10:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.157+0000 7fad9eb97700 1 --2- 172.21.15.12:0/1696687959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad98103960 0x7fad98103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.157+0000 7fad9eb97700 1 --2- 172.21.15.12:0/1696687959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad98105e60 0x7fad98069190 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.157+0000 7fad9eb97700 1 --2- 172.21.15.12:0/1696687959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad981062d0 0x7fad980696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.157+0000 7fad9eb97700 1 -- 172.21.15.12:0/1696687959 >> 172.21.15.12:0/1696687959 conn(0x7fad98076ba0 msgr2=0x7fad98076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.158+0000 7fad9eb97700 1 -- 172.21.15.12:0/1696687959 shutdown_connections 2024-05-09T00:10:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.158+0000 7fad9eb97700 1 -- 172.21.15.12:0/1696687959 wait complete. 2024-05-09T00:10:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.158+0000 7fad9eb97700 1 Processor -- start 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.158+0000 7fad9eb97700 1 -- start start 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad9eb97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad98103960 0x7fad98072600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad9eb97700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98105e60 0x7fad98072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad9eb97700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad981062d0 0x7fad9806b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad9eb97700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9806bfd0 con 0x7fad981062d0 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad9eb97700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9806c140 con 0x7fad98103960 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad9eb97700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9806c300 con 0x7fad98105e60 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad97fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98105e60 0x7fad98072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad9c933700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad98103960 0x7fad98072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad97fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98105e60 0x7fad98072b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52692/0 (socket says 172.21.15.12:52692) 2024-05-09T00:10:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad97fff700 1 -- 172.21.15.12:0/2998850559 learned_addr learned my addr 172.21.15.12:0/2998850559 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad9d134700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad981062d0 0x7fad9806b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad97fff700 1 -- 172.21.15.12:0/2998850559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad98103960 msgr2=0x7fad98072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad97fff700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad98103960 0x7fad98072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad97fff700 1 -- 172.21.15.12:0/2998850559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad981062d0 msgr2=0x7fad9806b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad97fff700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad981062d0 0x7fad9806b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad97fff700 1 -- 172.21.15.12:0/2998850559 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad84009520 con 0x7fad98105e60 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.159+0000 7fad97fff700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98105e60 0x7fad98072b80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00e9c0 tx=0x7fad8c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.160+0000 7fad95ffb700 1 -- 172.21.15.12:0/2998850559 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c004a30 con 0x7fad98105e60 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.160+0000 7fad95ffb700 1 -- 172.21.15.12:0/2998850559 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad8c00cc80 con 0x7fad98105e60 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.160+0000 7fad95ffb700 1 -- 172.21.15.12:0/2998850559 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0054e0 con 0x7fad98105e60 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.160+0000 7fad9eb97700 1 -- 172.21.15.12:0/2998850559 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad98070190 con 0x7fad98105e60 2024-05-09T00:10:15.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.160+0000 7fad9eb97700 1 -- 172.21.15.12:0/2998850559 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad98070650 con 0x7fad98105e60 2024-05-09T00:10:15.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.163+0000 7fad95ffb700 1 -- 172.21.15.12:0/2998850559 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fad8c004b90 con 0x7fad98105e60 2024-05-09T00:10:15.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.164+0000 7fad95ffb700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad880a1310 0x7fad880a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.164+0000 7fad95ffb700 1 -- 172.21.15.12:0/2998850559 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fad8c014070 con 0x7fad98105e60 2024-05-09T00:10:15.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.164+0000 7fad9c933700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad880a1310 0x7fad880a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.164+0000 7fadaad51b80 1 Processor -- start 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.164+0000 7fadaad51b80 1 -- start start 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.164+0000 7fad9c933700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad880a1310 0x7fad880a37d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad840098c0 tx=0x7fad84019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b6191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b613350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 0x56314b613cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fadaad51b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b56a150 con 0x56314b618e00 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fadaad51b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b567b10 con 0x56314b613890 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fadaad51b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b563400 con 0x56314b619720 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fad97fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b613350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fad97fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b613350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52696/0 (socket says 172.21.15.12:52696) 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fad9c933700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b6191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fad9d134700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 0x56314b613cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fad97fff700 1 -- 172.21.15.12:0/222044763 learned_addr learned my addr 172.21.15.12:0/222044763 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fad97fff700 1 -- 172.21.15.12:0/222044763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 msgr2=0x56314b613cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fad97fff700 1 --2- 172.21.15.12:0/222044763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 0x56314b613cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fad97fff700 1 -- 172.21.15.12:0/222044763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 msgr2=0x56314b6191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fad97fff700 1 --2- 172.21.15.12:0/222044763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b6191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.165+0000 7fad97fff700 1 -- 172.21.15.12:0/222044763 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56314b3e43b0 con 0x56314b619720 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.166+0000 7fad97fff700 1 --2- 172.21.15.12:0/222044763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b613350 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fad8c096890 tx=0x7fad8c099870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8533007d9a2c493 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.166+0000 7fad82ffd700 1 -- 172.21.15.12:0/222044763 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0b1070 con 0x56314b619720 2024-05-09T00:10:15.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.166+0000 7fad82ffd700 1 -- 172.21.15.12:0/222044763 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad8c09dc50 con 0x56314b619720 2024-05-09T00:10:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.166+0000 7fad82ffd700 1 -- 172.21.15.12:0/222044763 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c09a080 con 0x56314b619720 2024-05-09T00:10:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.167+0000 7fadaad51b80 1 -- 172.21.15.12:0/222044763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 msgr2=0x56314b613350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.167+0000 7fadaad51b80 1 --2- 172.21.15.12:0/222044763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b613350 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fad8c096890 tx=0x7fad8c099870 comp rx=0 tx=0).stop 2024-05-09T00:10:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.167+0000 7fadaad51b80 1 -- 172.21.15.12:0/222044763 shutdown_connections 2024-05-09T00:10:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.167+0000 7fadaad51b80 1 --2- 172.21.15.12:0/222044763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 0x56314b613cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.167+0000 7fadaad51b80 1 --2- 172.21.15.12:0/222044763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b6191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.167+0000 7fadaad51b80 1 --2- 172.21.15.12:0/222044763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b613350 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.167+0000 7fadaad51b80 1 -- 172.21.15.12:0/222044763 >> 172.21.15.12:0/222044763 conn(0x56314b571210 msgr2=0x56314b60ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:15.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.167+0000 7fadaad51b80 1 -- 172.21.15.12:0/222044763 shutdown_connections 2024-05-09T00:10:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.167+0000 7fadaad51b80 1 -- 172.21.15.12:0/222044763 wait complete. 2024-05-09T00:10:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.167+0000 7fadaad51b80 1 -- start start 2024-05-09T00:10:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.168+0000 7fadaad51b80 1 Processor -- start 2024-05-09T00:10:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.168+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 0x56314b61cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.168+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b61d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.168+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b61d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.168+0000 7fadaad51b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b4ff560 con 0x56314b618e00 2024-05-09T00:10:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.168+0000 7fadaad51b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b4ff200 con 0x56314b613890 2024-05-09T00:10:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.168+0000 7fadaad51b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b3e66a0 con 0x56314b619720 2024-05-09T00:10:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.168+0000 7fadaad51b80 10 client.0 _ll_register_callbacks cb 0x56314b508b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.168+0000 7fad9c933700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 0x56314b61cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.168+0000 7fad9c933700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 0x56314b61cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:39570/0 (socket says 172.21.15.12:39570) 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad9c933700 1 -- 172.21.15.12:0/2479705515 learned_addr learned my addr 172.21.15.12:0/2479705515 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad9d134700 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b61d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad97fff700 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b61d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad9d134700 1 -- 172.21.15.12:0/2479705515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 msgr2=0x56314b61cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad9d134700 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 0x56314b61cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad9d134700 1 -- 172.21.15.12:0/2479705515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 msgr2=0x56314b61d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad9d134700 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b61d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad9d134700 1 -- 172.21.15.12:0/2479705515 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad98070650 con 0x56314b619720 2024-05-09T00:10:15.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad9c933700 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 0x56314b61cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad9d134700 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b61d5f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fad9000a970 tx=0x7fad9000ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8858d1f7dd20af71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad77fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad77fff700 1 -- 172.21.15.12:0/2479705515 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90010040 con 0x56314b619720 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fad77fff700 1 -- 172.21.15.12:0/2479705515 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fad90006e40 con 0x56314b619720 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.169+0000 7fadaad51b80 1 -- 172.21.15.12:0/2479705515 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56314b4bc270 con 0x56314b619720 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fad77fff700 1 -- 172.21.15.12:0/2479705515 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad90016860 con 0x56314b619720 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fadaad51b80 10 client.31612 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fadaad51b80 1 -- 172.21.15.12:0/2479705515 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x56314b390b60 con 0x56314b619720 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fad77fff700 1 -- 172.21.15.12:0/2479705515 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56314b4bc270 con 0x56314b619720 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fad77fff700 1 -- 172.21.15.12:0/2479705515 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fad90011bd0 con 0x56314b619720 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fadaad51b80 10 client.31612 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fadaad51b80 10 client.31612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:15.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fadaad51b80 20 client.31612 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:15.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fadaad51b80 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56314b63e400 0x56314b640820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fadaad51b80 4 client.31612 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:15.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fadaad51b80 1 -- 172.21.15.12:0/2479705515 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56314b5e2810 con 0x56314b63e400 2024-05-09T00:10:15.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.170+0000 7fad9c933700 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56314b63e400 0x56314b640820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.171+0000 7fad77fff700 10 client.31612 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:15.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.171+0000 7fad9c933700 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56314b63e400 0x56314b640820 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47183cebf45cd1e server_cookie=7240cf9630f780cc in_seq=0 out_seq=0 2024-05-09T00:10:15.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.173+0000 7fad77fff700 1 -- 172.21.15.12:0/2479705515 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56314b5e2810 con 0x56314b63e400 2024-05-09T00:10:15.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.173+0000 7fad77fff700 10 client.31612 handle_command_reply: tid=0 2024-05-09T00:10:15.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.173+0000 7fadaad51b80 1 client.31612 shutdown 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.173+0000 7fadaad51b80 1 -- 172.21.15.12:0/2479705515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 msgr2=0x56314b61d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.173+0000 7fadaad51b80 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b61d5f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fad9000a970 tx=0x7fad9000ac80 comp rx=0 tx=0).stop 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.174+0000 7fadaad51b80 1 -- 172.21.15.12:0/2479705515 shutdown_connections 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.174+0000 7fadaad51b80 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b619720 0x56314b61d5f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.174+0000 7fadaad51b80 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56314b63e400 0x56314b640820 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.174+0000 7fadaad51b80 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b613890 0x56314b61cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.174+0000 7fadaad51b80 1 --2- 172.21.15.12:0/2479705515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b61d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.174+0000 7fad77fff700 0 client.31612 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.174+0000 7fadaad51b80 1 -- 172.21.15.12:0/2479705515 >> 172.21.15.12:0/2479705515 conn(0x56314b571210 msgr2=0x56314b60d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.174+0000 7fadaad51b80 1 -- 172.21.15.12:0/2479705515 shutdown_connections 2024-05-09T00:10:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.174+0000 7fadaad51b80 1 -- 172.21.15.12:0/2479705515 wait complete. 2024-05-09T00:10:15.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.174+0000 7fadaad51b80 20 client.31612 trim_cache size 0 max 16384 2024-05-09T00:10:15.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fadaad51b80 1 Processor -- start 2024-05-09T00:10:15.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fadaad51b80 1 -- start start 2024-05-09T00:10:15.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b613890 0x56314b613350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b618370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 0x56314b60dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fadaad51b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b60fe60 con 0x56314b618e00 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fadaad51b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b606040 con 0x56314b619720 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fadaad51b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b60f070 con 0x56314b613890 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fad97fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b618370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fad9c933700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b613890 0x56314b613350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fad97fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b618370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47782/0 (socket says 172.21.15.12:47782) 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fad9d134700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 0x56314b60dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.196+0000 7fad97fff700 1 -- 172.21.15.12:0/2346984211 learned_addr learned my addr 172.21.15.12:0/2346984211 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:15.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.197+0000 7fad9d134700 1 -- 172.21.15.12:0/2346984211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b613890 msgr2=0x56314b613350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:15.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.197+0000 7fad9d134700 1 --2- 172.21.15.12:0/2346984211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b613890 0x56314b613350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.197+0000 7fad9d134700 1 -- 172.21.15.12:0/2346984211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 msgr2=0x56314b618370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:15.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.197+0000 7fad9d134700 1 --2- 172.21.15.12:0/2346984211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b618370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.197+0000 7fad9d134700 1 -- 172.21.15.12:0/2346984211 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56314b613f40 con 0x56314b619720 2024-05-09T00:10:15.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.197+0000 7fad97fff700 1 --2- 172.21.15.12:0/2346984211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b618370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:15.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.197+0000 7fad9d134700 1 --2- 172.21.15.12:0/2346984211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 0x56314b60dd20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fad90002230 tx=0x7fad90011230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5f085380bb4e197 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:15.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.197+0000 7fad837fe700 1 -- 172.21.15.12:0/2346984211 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90010040 con 0x56314b619720 2024-05-09T00:10:15.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.197+0000 7fad837fe700 1 -- 172.21.15.12:0/2346984211 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad9001ed20 con 0x56314b619720 2024-05-09T00:10:15.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.197+0000 7fad837fe700 1 -- 172.21.15.12:0/2346984211 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90011450 con 0x56314b619720 2024-05-09T00:10:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.198+0000 7fadaad51b80 1 -- 172.21.15.12:0/2346984211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 msgr2=0x56314b60dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.198+0000 7fadaad51b80 1 --2- 172.21.15.12:0/2346984211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 0x56314b60dd20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fad90002230 tx=0x7fad90011230 comp rx=0 tx=0).stop 2024-05-09T00:10:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.198+0000 7fadaad51b80 1 -- 172.21.15.12:0/2346984211 shutdown_connections 2024-05-09T00:10:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.198+0000 7fadaad51b80 1 --2- 172.21.15.12:0/2346984211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 0x56314b60dd20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.198+0000 7fadaad51b80 1 --2- 172.21.15.12:0/2346984211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b613890 0x56314b613350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.198+0000 7fadaad51b80 1 --2- 172.21.15.12:0/2346984211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b618e00 0x56314b618370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.198+0000 7fadaad51b80 1 -- 172.21.15.12:0/2346984211 >> 172.21.15.12:0/2346984211 conn(0x56314b571210 msgr2=0x56314b617c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.198+0000 7fadaad51b80 1 -- 172.21.15.12:0/2346984211 shutdown_connections 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.198+0000 7fadaad51b80 1 -- 172.21.15.12:0/2346984211 wait complete. 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fadaad51b80 1 -- start start 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fadaad51b80 1 Processor -- start 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b613890 0x56314b6223e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b618e00 0x56314b622920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fadaad51b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 0x56314b68d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fadaad51b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b68ddd0 con 0x56314b613890 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fadaad51b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b68df40 con 0x56314b619720 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fadaad51b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314b68e100 con 0x56314b618e00 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fadaad51b80 10 client.0 _ll_register_callbacks cb 0x56314b508b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fad97fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b618e00 0x56314b622920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fad9c933700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b613890 0x56314b6223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fad97fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b618e00 0x56314b622920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:52708/0 (socket says 172.21.15.12:52708) 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fad97fff700 1 -- 172.21.15.12:0/3447207364 learned_addr learned my addr 172.21.15.12:0/3447207364 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.199+0000 7fad9d134700 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 0x56314b68d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fad97fff700 1 -- 172.21.15.12:0/3447207364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 msgr2=0x56314b68d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fad97fff700 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 0x56314b68d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fad97fff700 1 -- 172.21.15.12:0/3447207364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b613890 msgr2=0x56314b6223e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fad97fff700 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b613890 0x56314b6223e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fad97fff700 1 -- 172.21.15.12:0/3447207364 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56314b3e43b0 con 0x56314b618e00 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fad97fff700 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b618e00 0x56314b622920 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fad8c020000 tx=0x7fad8c009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=306f9039970dd21a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fad77fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fad77fff700 1 -- 172.21.15.12:0/3447207364 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0b1070 con 0x56314b618e00 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fadaad51b80 1 -- 172.21.15.12:0/3447207364 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad8c09dc50 con 0x56314b618e00 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fad77fff700 1 -- 172.21.15.12:0/3447207364 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fad8c0a22a0 con 0x56314b618e00 2024-05-09T00:10:15.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fad77fff700 1 -- 172.21.15.12:0/3447207364 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad8c096530 con 0x56314b618e00 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fadaad51b80 10 client.31618 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.200+0000 7fadaad51b80 1 -- 172.21.15.12:0/3447207364 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x56314b390b60 con 0x56314b618e00 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.201+0000 7fad77fff700 1 -- 172.21.15.12:0/3447207364 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad8c09dc50 con 0x56314b618e00 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.201+0000 7fad77fff700 1 -- 172.21.15.12:0/3447207364 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fad8c09a1a0 con 0x56314b618e00 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.201+0000 7fadaad51b80 10 client.31618 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.201+0000 7fadaad51b80 10 client.31618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.201+0000 7fadaad51b80 20 client.31618 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.201+0000 7fadaad51b80 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56314b695670 0x56314b697a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.201+0000 7fadaad51b80 4 client.31618 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.201+0000 7fadaad51b80 1 -- 172.21.15.12:0/3447207364 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56314b55bec0 con 0x56314b695670 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.201+0000 7fad9c933700 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56314b695670 0x56314b697a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.202+0000 7fad9c933700 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56314b695670 0x56314b697a90 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec553e9befc1564 server_cookie=90c27ce45e236795 in_seq=0 out_seq=0 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.202+0000 7fad77fff700 10 client.31618 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:15.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fad77fff700 1 -- 172.21.15.12:0/3447207364 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x56314b55bec0 con 0x56314b695670 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fad77fff700 10 client.31618 handle_command_reply: tid=0 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 client.31618 shutdown 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 -- 172.21.15.12:0/3447207364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b618e00 msgr2=0x56314b622920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b618e00 0x56314b622920 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fad8c020000 tx=0x7fad8c009550 comp rx=0 tx=0).stop 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 -- 172.21.15.12:0/3447207364 shutdown_connections 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56314b695670 0x56314b697a90 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56314b619720 0x56314b68d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314b613890 0x56314b6223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 --2- 172.21.15.12:0/3447207364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56314b618e00 0x56314b622920 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fad77fff700 0 client.31618 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 -- 172.21.15.12:0/3447207364 >> 172.21.15.12:0/3447207364 conn(0x56314b571210 msgr2=0x56314b619b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 -- 172.21.15.12:0/3447207364 shutdown_connections 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 1 -- 172.21.15.12:0/3447207364 wait complete. 2024-05-09T00:10:15.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.203+0000 7fadaad51b80 20 client.31618 trim_cache size 0 max 16384 2024-05-09T00:10:15.206 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:10:15.206 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (85 inodes in the stack)", 2024-05-09T00:10:15.206 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:10:15.206 INFO:teuthology.orchestra.run.smithi012.stdout: "78810daa-60d9-4ff3-86e1-1e4ee43d8af0": { 2024-05-09T00:10:15.206 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:10:15.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "78810daa-60d9-4ff3-86e1-1e4ee43d8af0", 2024-05-09T00:10:15.206 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:10:15.206 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:10:15.207 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:10:15.207 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:10:15.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 -- 172.21.15.12:0/2998850559 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad880a1310 msgr2=0x7fad880a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad880a1310 0x7fad880a37d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad840098c0 tx=0x7fad84019040 comp rx=0 tx=0).stop 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 -- 172.21.15.12:0/2998850559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98105e60 msgr2=0x7fad98072b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98105e60 0x7fad98072b80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00e9c0 tx=0x7fad8c00ecd0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 -- 172.21.15.12:0/2998850559 shutdown_connections 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad880a1310 0x7fad880a37d0 secure :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad840098c0 tx=0x7fad84019040 comp rx=0 tx=0).stop 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad981062d0 0x7fad9806b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad98103960 0x7fad98072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 --2- 172.21.15.12:0/2998850559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98105e60 0x7fad98072b80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 -- 172.21.15.12:0/2998850559 >> 172.21.15.12:0/2998850559 conn(0x7fad98076ba0 msgr2=0x7fad98100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 -- 172.21.15.12:0/2998850559 shutdown_connections 2024-05-09T00:10:15.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:15.204+0000 7fad9eb97700 1 -- 172.21.15.12:0/2998850559 wait complete. 2024-05-09T00:10:15.218 INFO:tasks.cephfs.filesystem:scrub status for tag:78810daa-60d9-4ff3-86e1-1e4ee43d8af0 - {'path': '/', 'tag': '78810daa-60d9-4ff3-86e1-1e4ee43d8af0', 'options': 'recursive,force'} 2024-05-09T00:10:15.308 INFO:tasks.workunit.client.0.smithi012.stdout: 10 348832 28.80 MB/sec execute 490 sec latency 136.276 ms 2024-05-09T00:10:15.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:15 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:10:15.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:15 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:15.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:15 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:10:15.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:15 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:15.766 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:15 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:10:15.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:15 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:16.308 INFO:tasks.workunit.client.0.smithi012.stdout: 10 349392 28.82 MB/sec execute 491 sec latency 128.148 ms 2024-05-09T00:10:16.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:16 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:10:16.666 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:16.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:16 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:10:16.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:16 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:10:16.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:16.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:16 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:16.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:16 smithi012 ceph-mon[42503]: pgmap v767: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.4 MiB/s wr, 441 op/s 2024-05-09T00:10:16.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:16 smithi012 ceph-mon[42503]: osdmap e134: 12 total, 12 up, 12 in 2024-05-09T00:10:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:16 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:10:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:16.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:16 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:10:16.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:16 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:10:16.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:16.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:16 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:16.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:16 smithi204 ceph-mon[47373]: pgmap v767: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.4 MiB/s wr, 441 op/s 2024-05-09T00:10:16.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:16 smithi204 ceph-mon[47373]: osdmap e134: 12 total, 12 up, 12 in 2024-05-09T00:10:16.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:16 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:10:16.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:16.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:16 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:10:16.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:16 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:10:16.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:16.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:16 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:16.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:16 smithi160 ceph-mon[46152]: pgmap v767: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.4 MiB/s wr, 441 op/s 2024-05-09T00:10:16.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:16 smithi160 ceph-mon[46152]: osdmap e134: 12 total, 12 up, 12 in 2024-05-09T00:10:17.308 INFO:tasks.workunit.client.0.smithi012.stdout: 10 349851 28.81 MB/sec execute 492 sec latency 128.870 ms 2024-05-09T00:10:18.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:17 smithi012 ceph-mon[42503]: pgmap v769: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 607 op/s 2024-05-09T00:10:18.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:17 smithi204 ceph-mon[47373]: pgmap v769: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 607 op/s 2024-05-09T00:10:18.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:17 smithi160 ceph-mon[46152]: pgmap v769: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 607 op/s 2024-05-09T00:10:18.308 INFO:tasks.workunit.client.0.smithi012.stdout: 10 350129 28.79 MB/sec execute 493 sec latency 295.876 ms 2024-05-09T00:10:19.308 INFO:tasks.workunit.client.0.smithi012.stdout: 10 350554 28.78 MB/sec execute 494 sec latency 113.106 ms 2024-05-09T00:10:20.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:19 smithi012 ceph-mon[42503]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 610 op/s 2024-05-09T00:10:20.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:19 smithi204 ceph-mon[47373]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 610 op/s 2024-05-09T00:10:20.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:19 smithi160 ceph-mon[46152]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 610 op/s 2024-05-09T00:10:20.308 INFO:tasks.workunit.client.0.smithi012.stdout: 10 350956 28.76 MB/sec execute 495 sec latency 96.749 ms 2024-05-09T00:10:21.308 INFO:tasks.workunit.client.0.smithi012.stdout: 10 351400 28.73 MB/sec execute 496 sec latency 123.474 ms 2024-05-09T00:10:22.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:21 smithi012 ceph-mon[42503]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 614 op/s 2024-05-09T00:10:22.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:21 smithi204 ceph-mon[47373]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 614 op/s 2024-05-09T00:10:22.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:21 smithi160 ceph-mon[46152]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 614 op/s 2024-05-09T00:10:22.308 INFO:tasks.workunit.client.0.smithi012.stdout: 10 352045 28.74 MB/sec execute 497 sec latency 112.419 ms 2024-05-09T00:10:23.309 INFO:tasks.workunit.client.0.smithi012.stdout: 10 352396 28.73 MB/sec execute 498 sec latency 388.655 ms 2024-05-09T00:10:24.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:23 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:10:24.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:23 smithi012 ceph-mon[42503]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 642 op/s 2024-05-09T00:10:24.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:23 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:10:24.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:23 smithi204 ceph-mon[47373]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 642 op/s 2024-05-09T00:10:24.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:23 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:10:24.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:23 smithi160 ceph-mon[46152]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 642 op/s 2024-05-09T00:10:24.309 INFO:tasks.workunit.client.0.smithi012.stdout: 10 352848 28.73 MB/sec execute 499 sec latency 157.575 ms 2024-05-09T00:10:25.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:10:25.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:24 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:10:25.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:10:25.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:24 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:10:25.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:10:25.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:24 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:10:25.309 INFO:tasks.workunit.client.0.smithi012.stdout: 10 353327 28.73 MB/sec execute 500 sec latency 84.137 ms 2024-05-09T00:10:26.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:25 smithi012 ceph-mon[42503]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 642 op/s 2024-05-09T00:10:26.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:25 smithi204 ceph-mon[47373]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 642 op/s 2024-05-09T00:10:26.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:25 smithi160 ceph-mon[46152]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 642 op/s 2024-05-09T00:10:26.309 INFO:tasks.workunit.client.0.smithi012.stdout: 10 353792 28.72 MB/sec execute 501 sec latency 90.670 ms 2024-05-09T00:10:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.748+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1469781254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc094100280 msgr2=0x7fc094100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.748+0000 7fc09ae1b700 1 --2- 172.21.15.12:0/1469781254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc094100280 0x7fc094100700 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc080009990 tx=0x7fc080009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.748+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1469781254 shutdown_connections 2024-05-09T00:10:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.748+0000 7fc09ae1b700 1 --2- 172.21.15.12:0/1469781254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc094100c40 0x7fc09410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.748+0000 7fc09ae1b700 1 --2- 172.21.15.12:0/1469781254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc094105f20 0x7fc094106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.748+0000 7fc09ae1b700 1 --2- 172.21.15.12:0/1469781254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc094100280 0x7fc094100700 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.748+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1469781254 >> 172.21.15.12:0/1469781254 conn(0x7fc094076bc0 msgr2=0x7fc094076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.748+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1469781254 shutdown_connections 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.748+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1469781254 wait complete. 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc09ae1b700 1 Processor -- start 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc09ae1b700 1 -- start start 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc09ae1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc094100280 0x7fc0941972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc09ae1b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc094100c40 0x7fc094197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc09ae1b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc094105f20 0x7fc09419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc09ae1b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09419c210 con 0x7fc094100280 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc09ae1b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09419c380 con 0x7fc094100c40 2024-05-09T00:10:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc09ae1b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09419c540 con 0x7fc094105f20 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc098bb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc094100280 0x7fc0941972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc093fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc094100c40 0x7fc094197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc0993b8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc094105f20 0x7fc09419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc093fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc094100c40 0x7fc094197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:60048/0 (socket says 172.21.15.12:60048) 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc098bb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc094100280 0x7fc0941972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46418/0 (socket says 172.21.15.12:46418) 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.749+0000 7fc093fff700 1 -- 172.21.15.12:0/1542052180 learned_addr learned my addr 172.21.15.12:0/1542052180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.750+0000 7fc093fff700 1 -- 172.21.15.12:0/1542052180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc094105f20 msgr2=0x7fc09419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.750+0000 7fc093fff700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc094105f20 0x7fc09419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.750+0000 7fc093fff700 1 -- 172.21.15.12:0/1542052180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc094100280 msgr2=0x7fc0941972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.750+0000 7fc093fff700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc094100280 0x7fc0941972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.750+0000 7fc093fff700 1 -- 172.21.15.12:0/1542052180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0800095f0 con 0x7fc094100c40 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.750+0000 7fc0993b8700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc094105f20 0x7fc09419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.750+0000 7fc093fff700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc094100c40 0x7fc094197830 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc080000c00 tx=0x7fc080005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.752+0000 7fc091ffb700 1 -- 172.21.15.12:0/1542052180 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08001d070 con 0x7fc094100c40 2024-05-09T00:10:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.752+0000 7fc091ffb700 1 -- 172.21.15.12:0/1542052180 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc080022470 con 0x7fc094100c40 2024-05-09T00:10:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.752+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1542052180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09419c7c0 con 0x7fc094100c40 2024-05-09T00:10:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.752+0000 7fc091ffb700 1 -- 172.21.15.12:0/1542052180 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080017650 con 0x7fc094100c40 2024-05-09T00:10:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.752+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1542052180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0941a4060 con 0x7fc094100c40 2024-05-09T00:10:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.756+0000 7fc0a6fd5b80 1 Processor -- start 2024-05-09T00:10:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc0a6fd5b80 1 -- start start 2024-05-09T00:10:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e744950 0x55d61e744d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 0x55d61e73ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 0x55d61e73f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e6960f0 con 0x55d61e744950 2024-05-09T00:10:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e693ab0 con 0x55d61e73f480 2024-05-09T00:10:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e68f3a0 con 0x55d61e745270 2024-05-09T00:10:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc0993b8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e744950 0x55d61e744d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc098bb7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 0x55d61e73ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc0993b8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e744950 0x55d61e744d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46426/0 (socket says 172.21.15.12:46426) 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.757+0000 7fc0993b8700 1 -- 172.21.15.12:0/4078826906 learned_addr learned my addr 172.21.15.12:0/4078826906 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc091ffb700 1 -- 172.21.15.12:0/1542052180 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc0800225e0 con 0x7fc094100c40 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc0993b8700 1 -- 172.21.15.12:0/4078826906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 msgr2=0x55d61e73ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc0993b8700 1 --2- 172.21.15.12:0/4078826906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 0x55d61e73ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc0993b8700 1 -- 172.21.15.12:0/4078826906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 msgr2=0x55d61e73f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc0993b8700 1 --2- 172.21.15.12:0/4078826906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 0x55d61e73f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc0993b8700 1 -- 172.21.15.12:0/4078826906 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d61e510350 con 0x55d61e744950 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc098bb7700 1 --2- 172.21.15.12:0/4078826906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 0x55d61e73ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc0993b8700 1 --2- 172.21.15.12:0/4078826906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e744950 0x55d61e744d30 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fc08c003eb0 tx=0x7fc08c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50ea13d52bfdb30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc07effd700 1 -- 172.21.15.12:0/4078826906 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0092d0 con 0x55d61e744950 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc07effd700 1 -- 172.21.15.12:0/4078826906 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08c00f040 con 0x55d61e744950 2024-05-09T00:10:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc07effd700 1 -- 172.21.15.12:0/4078826906 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00e750 con 0x55d61e744950 2024-05-09T00:10:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.758+0000 7fc091ffb700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0840a1310 0x7fc0840a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.759+0000 7fc098bb7700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0840a1310 0x7fc0840a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.759+0000 7fc091ffb700 1 -- 172.21.15.12:0/1542052180 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc0800cde90 con 0x7fc094100c40 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.759+0000 7fc098bb7700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0840a1310 0x7fc0840a37d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55d61e73a790 tx=0x7fc088009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.759+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/4078826906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e744950 msgr2=0x55d61e744d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.759+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/4078826906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e744950 0x55d61e744d30 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fc08c003eb0 tx=0x7fc08c009920 comp rx=0 tx=0).stop 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.759+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/4078826906 shutdown_connections 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.759+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/4078826906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 0x55d61e73f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.760+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/4078826906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e744950 0x55d61e744d30 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.760+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/4078826906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 0x55d61e73ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.760+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/4078826906 >> 172.21.15.12:0/4078826906 conn(0x55d61e69d1b0 msgr2=0x55d61e739840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.760+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/4078826906 shutdown_connections 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.760+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/4078826906 wait complete. 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.760+0000 7fc0a6fd5b80 1 -- start start 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.760+0000 7fc0a6fd5b80 1 Processor -- start 2024-05-09T00:10:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 0x55d61e74b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 0x55d61e748570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e745270 0x55d61e748c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e62b500 con 0x55d61e73f480 2024-05-09T00:10:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e62b1a0 con 0x55d61e745270 2024-05-09T00:10:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e512640 con 0x55d61e744950 2024-05-09T00:10:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc0a6fd5b80 10 client.0 _ll_register_callbacks cb 0x55d61e634ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc093fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 0x55d61e748570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc093fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 0x55d61e748570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:60368/0 (socket says 172.21.15.12:60368) 2024-05-09T00:10:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc093fff700 1 -- 172.21.15.12:0/2574486685 learned_addr learned my addr 172.21.15.12:0/2574486685 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc098bb7700 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 0x55d61e74b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.761+0000 7fc0993b8700 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e745270 0x55d61e748c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.762+0000 7fc093fff700 1 -- 172.21.15.12:0/2574486685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e745270 msgr2=0x55d61e748c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.762+0000 7fc093fff700 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e745270 0x55d61e748c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.762+0000 7fc093fff700 1 -- 172.21.15.12:0/2574486685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 msgr2=0x55d61e74b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.762+0000 7fc093fff700 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 0x55d61e74b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.762+0000 7fc093fff700 1 -- 172.21.15.12:0/2574486685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0880098e0 con 0x55d61e744950 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.762+0000 7fc093fff700 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 0x55d61e748570 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc08009ae40 tx=0x7fc0800cbb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf883c78b469421 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.762+0000 7fc073fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.762+0000 7fc073fff700 1 -- 172.21.15.12:0/2574486685 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08009b5a0 con 0x55d61e744950 2024-05-09T00:10:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.762+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/2574486685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d61e5e8210 con 0x55d61e744950 2024-05-09T00:10:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.762+0000 7fc073fff700 1 -- 172.21.15.12:0/2574486685 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc08009b840 con 0x55d61e744950 2024-05-09T00:10:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.763+0000 7fc073fff700 1 -- 172.21.15.12:0/2574486685 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc08009aaa0 con 0x55d61e744950 2024-05-09T00:10:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.763+0000 7fc0a6fd5b80 10 client.31630 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.763+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/2574486685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x7fc08c003b90 con 0x55d61e744950 2024-05-09T00:10:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.766+0000 7fc073fff700 1 -- 172.21.15.12:0/2574486685 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d61e5e8210 con 0x55d61e744950 2024-05-09T00:10:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.767+0000 7fc073fff700 1 -- 172.21.15.12:0/2574486685 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0800cbd20 con 0x55d61e744950 2024-05-09T00:10:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.767+0000 7fc0a6fd5b80 10 client.31630 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.767+0000 7fc0a6fd5b80 10 client.31630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.768+0000 7fc0a6fd5b80 20 client.31630 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.768+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d61e75ccb0 0x55d61e75f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.768+0000 7fc0a6fd5b80 4 client.31630 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.768+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/2574486685 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d61e70e7b0 con 0x55d61e75ccb0 2024-05-09T00:10:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.768+0000 7fc098bb7700 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d61e75ccb0 0x55d61e75f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.768+0000 7fc098bb7700 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d61e75ccb0 0x55d61e75f0d0 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4003bf786ecfae server_cookie=580157dd205b1f66 in_seq=0 out_seq=0 2024-05-09T00:10:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.768+0000 7fc073fff700 10 client.31630 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.770+0000 7fc073fff700 1 -- 172.21.15.12:0/2574486685 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d61e70e7b0 con 0x55d61e75ccb0 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc073fff700 10 client.31630 handle_command_reply: tid=0 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 client.31630 shutdown 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/2574486685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 msgr2=0x55d61e748570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 0x55d61e748570 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc08009ae40 tx=0x7fc0800cbb30 comp rx=0 tx=0).stop 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/2574486685 shutdown_connections 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d61e75ccb0 0x55d61e75f0d0 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e745270 0x55d61e748c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 0x55d61e74b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/2574486685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 0x55d61e748570 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/2574486685 >> 172.21.15.12:0/2574486685 conn(0x55d61e69d1b0 msgr2=0x55d61e7434c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc073fff700 0 client.31630 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/2574486685 shutdown_connections 2024-05-09T00:10:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/2574486685 wait complete. 2024-05-09T00:10:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.771+0000 7fc0a6fd5b80 20 client.31630 trim_cache size 0 max 16384 2024-05-09T00:10:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc0a6fd5b80 1 Processor -- start 2024-05-09T00:10:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc0a6fd5b80 1 -- start start 2024-05-09T00:10:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 0x55d61e748c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 0x55d61e754720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e745270 0x55d61e739910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e734330 con 0x55d61e745270 2024-05-09T00:10:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e73a330 con 0x55d61e73f480 2024-05-09T00:10:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e746ae0 con 0x55d61e744950 2024-05-09T00:10:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc098bb7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 0x55d61e748c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc093fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 0x55d61e754720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc098bb7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 0x55d61e748c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:60084/0 (socket says 172.21.15.12:60084) 2024-05-09T00:10:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.791+0000 7fc098bb7700 1 -- 172.21.15.12:0/1181534337 learned_addr learned my addr 172.21.15.12:0/1181534337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.792+0000 7fc0993b8700 1 --2- 172.21.15.12:0/1181534337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e745270 0x55d61e739910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.792+0000 7fc098bb7700 1 -- 172.21.15.12:0/1181534337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 msgr2=0x55d61e754720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.792+0000 7fc098bb7700 1 --2- 172.21.15.12:0/1181534337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 0x55d61e754720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.792+0000 7fc098bb7700 1 -- 172.21.15.12:0/1181534337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e745270 msgr2=0x55d61e739910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.792+0000 7fc098bb7700 1 --2- 172.21.15.12:0/1181534337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e745270 0x55d61e739910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.792+0000 7fc098bb7700 1 -- 172.21.15.12:0/1181534337 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d61e4bcb00 con 0x55d61e73f480 2024-05-09T00:10:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.792+0000 7fc098bb7700 1 --2- 172.21.15.12:0/1181534337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 0x55d61e748c60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc088023b80 tx=0x7fc08802c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1ac1cedceaa7eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.792+0000 7fc07f7fe700 1 -- 172.21.15.12:0/1181534337 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088036040 con 0x55d61e73f480 2024-05-09T00:10:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.793+0000 7fc07f7fe700 1 -- 172.21.15.12:0/1181534337 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08803b860 con 0x55d61e73f480 2024-05-09T00:10:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.793+0000 7fc07f7fe700 1 -- 172.21.15.12:0/1181534337 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088028070 con 0x55d61e73f480 2024-05-09T00:10:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.793+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1181534337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 msgr2=0x55d61e748c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.793+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/1181534337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 0x55d61e748c60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc088023b80 tx=0x7fc08802c000 comp rx=0 tx=0).stop 2024-05-09T00:10:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.793+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1181534337 shutdown_connections 2024-05-09T00:10:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.793+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/1181534337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e745270 0x55d61e739910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.793+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/1181534337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e73f480 0x55d61e748c60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.793+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/1181534337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e744950 0x55d61e754720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.793+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1181534337 >> 172.21.15.12:0/1181534337 conn(0x55d61e69d1b0 msgr2=0x55d61e735e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.794+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1181534337 shutdown_connections 2024-05-09T00:10:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.794+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1181534337 wait complete. 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.794+0000 7fc0a6fd5b80 1 -- start start 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.794+0000 7fc0a6fd5b80 1 Processor -- start 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.794+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 0x55d61e74fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.794+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e744950 0x55d61e7501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.794+0000 7fc0a6fd5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 0x55d61e742380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e742a90 con 0x55d61e73f480 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e742c00 con 0x55d61e744950 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0a6fd5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e742dc0 con 0x55d61e745270 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0a6fd5b80 10 client.0 _ll_register_callbacks cb 0x55d61e634ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc098bb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 0x55d61e74fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0993b8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 0x55d61e742380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc098bb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 0x55d61e74fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46456/0 (socket says 172.21.15.12:46456) 2024-05-09T00:10:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc098bb7700 1 -- 172.21.15.12:0/1225162584 learned_addr learned my addr 172.21.15.12:0/1225162584 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc093fff700 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e744950 0x55d61e7501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0993b8700 1 -- 172.21.15.12:0/1225162584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e744950 msgr2=0x55d61e7501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0993b8700 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e744950 0x55d61e7501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0993b8700 1 -- 172.21.15.12:0/1225162584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 msgr2=0x55d61e74fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0993b8700 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 0x55d61e74fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0993b8700 1 -- 172.21.15.12:0/1225162584 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc088004040 con 0x55d61e745270 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc098bb7700 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 0x55d61e74fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc0993b8700 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 0x55d61e742380 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0049e0 tx=0x7fc08c004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec1d4b8b3e3a0af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.795+0000 7fc073fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.796+0000 7fc073fff700 1 -- 172.21.15.12:0/1225162584 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c023070 con 0x55d61e745270 2024-05-09T00:10:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.796+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1225162584 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc08c00f040 con 0x55d61e745270 2024-05-09T00:10:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.796+0000 7fc073fff700 1 -- 172.21.15.12:0/1225162584 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc08c007710 con 0x55d61e745270 2024-05-09T00:10:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.796+0000 7fc073fff700 1 -- 172.21.15.12:0/1225162584 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc08c0078d0 con 0x55d61e745270 2024-05-09T00:10:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.796+0000 7fc0a6fd5b80 10 client.31636 fetch_fsmap learned FSMap version 80 2024-05-09T00:10:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.796+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1225162584 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55d61e655500 con 0x55d61e745270 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.796+0000 7fc073fff700 1 -- 172.21.15.12:0/1225162584 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08c00f040 con 0x55d61e745270 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.796+0000 7fc073fff700 1 -- 172.21.15.12:0/1225162584 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 80) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc08c015450 con 0x55d61e745270 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.797+0000 7fc0a6fd5b80 10 client.31636 fetch_fsmap finished waiting for FSMap version 80 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.797+0000 7fc0a6fd5b80 10 client.31636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.797+0000 7fc0a6fd5b80 20 client.31636 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.797+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d61e7b3470 0x55d61e7b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.797+0000 7fc0a6fd5b80 4 client.31636 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.797+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1225162584 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d61e687e60 con 0x55d61e7b3470 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.797+0000 7fc098bb7700 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d61e7b3470 0x55d61e7b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.800+0000 7fc073fff700 10 client.31636 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:26.800+0000 7fc098bb7700 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d61e7b3470 0x55d61e7b5890 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40cc01bb1fdc7a1e server_cookie=a37171194bad3111 in_seq=0 out_seq=0 2024-05-09T00:10:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.296+0000 7fc073fff700 1 -- 172.21.15.12:0/1225162584 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d61e687e60 con 0x55d61e7b3470 2024-05-09T00:10:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.296+0000 7fc073fff700 10 client.31636 handle_command_reply: tid=0 2024-05-09T00:10:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.296+0000 7fc0a6fd5b80 1 client.31636 shutdown 2024-05-09T00:10:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.299+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1225162584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 msgr2=0x55d61e742380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.299+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 0x55d61e742380 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0049e0 tx=0x7fc08c004f40 comp rx=0 tx=0).stop 2024-05-09T00:10:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.300+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1225162584 shutdown_connections 2024-05-09T00:10:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.300+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d61e745270 0x55d61e742380 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.300+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d61e73f480 0x55d61e74fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.300+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55d61e7b3470 0x55d61e7b5890 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.300+0000 7fc0a6fd5b80 1 --2- 172.21.15.12:0/1225162584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d61e744950 0x55d61e7501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.300+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1225162584 >> 172.21.15.12:0/1225162584 conn(0x55d61e69d1b0 msgr2=0x55d61e747ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.300+0000 7fc073fff700 0 client.31636 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.300+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1225162584 shutdown_connections 2024-05-09T00:10:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.300+0000 7fc0a6fd5b80 1 -- 172.21.15.12:0/1225162584 wait complete. 2024-05-09T00:10:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.302+0000 7fc0a6fd5b80 20 client.31636 trim_cache size 0 max 16384 2024-05-09T00:10:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:10:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:10:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:10:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:10:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1542052180 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0840a1310 msgr2=0x7fc0840a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0840a1310 0x7fc0840a37d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55d61e73a790 tx=0x7fc088009260 comp rx=0 tx=0).stop 2024-05-09T00:10:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1542052180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc094100c40 msgr2=0x7fc094197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc094100c40 0x7fc094197830 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc080000c00 tx=0x7fc080005740 comp rx=0 tx=0).stop 2024-05-09T00:10:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1542052180 shutdown_connections 2024-05-09T00:10:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fc0840a1310 0x7fc0840a37d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc094105f20 0x7fc09419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc094100280 0x7fc0941972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 --2- 172.21.15.12:0/1542052180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc094100c40 0x7fc094197830 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1542052180 >> 172.21.15.12:0/1542052180 conn(0x7fc094076bc0 msgr2=0x7fc0940692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1542052180 shutdown_connections 2024-05-09T00:10:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:27.304+0000 7fc09ae1b700 1 -- 172.21.15.12:0/1542052180 wait complete. 2024-05-09T00:10:27.308 INFO:tasks.workunit.client.0.smithi012.stdout: 10 354111 28.70 MB/sec execute 502 sec latency 361.621 ms 2024-05-09T00:10:28.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:27 smithi012 ceph-mon[42503]: pgmap v774: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 702 op/s 2024-05-09T00:10:28.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:27 smithi204 ceph-mon[47373]: pgmap v774: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 702 op/s 2024-05-09T00:10:28.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:27 smithi160 ceph-mon[46152]: pgmap v774: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 702 op/s 2024-05-09T00:10:28.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 354400 28.68 MB/sec execute 503 sec latency 442.626 ms 2024-05-09T00:10:29.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 354658 28.63 MB/sec execute 504 sec latency 179.735 ms 2024-05-09T00:10:30.016 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:10:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: ignoring --setuser ceph since I am not root 2024-05-09T00:10:30.017 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:10:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: ignoring --setgroup ceph since I am not root 2024-05-09T00:10:30.017 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:10:29 smithi204 ceph-mds[83951]: -- 172.21.15.204:0/3799174260 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56536114e1a0 con 0x565361125000 2024-05-09T00:10:30.017 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:10:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: starting mds.l at 2024-05-09T00:10:30.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:29 smithi204 ceph-mon[47373]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.9 MiB/s wr, 533 op/s 2024-05-09T00:10:30.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:29 smithi204 ceph-mon[47373]: daemon mds.l restarted 2024-05-09T00:10:30.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:29 smithi204 ceph-mon[47373]: osdmap e135: 12 total, 12 up, 12 in 2024-05-09T00:10:30.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:29 smithi012 ceph-mon[42503]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.9 MiB/s wr, 533 op/s 2024-05-09T00:10:30.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:29 smithi012 ceph-mon[42503]: daemon mds.l restarted 2024-05-09T00:10:30.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:29 smithi012 ceph-mon[42503]: osdmap e135: 12 total, 12 up, 12 in 2024-05-09T00:10:30.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:29 smithi160 ceph-mon[46152]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.9 MiB/s wr, 533 op/s 2024-05-09T00:10:30.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:29 smithi160 ceph-mon[46152]: daemon mds.l restarted 2024-05-09T00:10:30.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:29 smithi160 ceph-mon[46152]: osdmap e135: 12 total, 12 up, 12 in 2024-05-09T00:10:30.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 355234 28.62 MB/sec execute 505 sec latency 110.793 ms 2024-05-09T00:10:31.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:30 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6832/1830773792,v1:172.21.15.204:6833/1830773792] up:boot 2024-05-09T00:10:31.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:30 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:10:31.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:30 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:10:31.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:30 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:10:31.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:30 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6832/1830773792,v1:172.21.15.204:6833/1830773792] up:boot 2024-05-09T00:10:31.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:30 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:10:31.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:30 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:10:31.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:30 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:10:31.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:30 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6832/1830773792,v1:172.21.15.204:6833/1830773792] up:boot 2024-05-09T00:10:31.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:30 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:10:31.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:30 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:10:31.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:30 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:10:31.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 355964 28.65 MB/sec execute 506 sec latency 96.543 ms 2024-05-09T00:10:32.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:31 smithi012 ceph-mon[42503]: pgmap v777: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 12 MiB/s wr, 643 op/s 2024-05-09T00:10:32.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:31 smithi204 ceph-mon[47373]: pgmap v777: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 12 MiB/s wr, 643 op/s 2024-05-09T00:10:32.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:31 smithi160 ceph-mon[46152]: pgmap v777: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 12 MiB/s wr, 643 op/s 2024-05-09T00:10:32.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 356487 28.66 MB/sec execute 507 sec latency 111.042 ms 2024-05-09T00:10:33.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 356795 28.64 MB/sec execute 508 sec latency 294.472 ms 2024-05-09T00:10:34.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:33 smithi012 ceph-mon[42503]: pgmap v778: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-05-09T00:10:34.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:33 smithi204 ceph-mon[47373]: pgmap v778: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-05-09T00:10:34.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:33 smithi160 ceph-mon[46152]: pgmap v778: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-05-09T00:10:34.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 357292 28.64 MB/sec execute 509 sec latency 165.406 ms 2024-05-09T00:10:35.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 357733 28.64 MB/sec execute 510 sec latency 80.681 ms 2024-05-09T00:10:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:35 smithi012 ceph-mon[42503]: pgmap v779: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-05-09T00:10:36.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:35 smithi204 ceph-mon[47373]: pgmap v779: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-05-09T00:10:36.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:35 smithi160 ceph-mon[46152]: pgmap v779: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-05-09T00:10:36.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 358118 28.61 MB/sec execute 511 sec latency 102.882 ms 2024-05-09T00:10:37.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 358506 28.58 MB/sec execute 512 sec latency 92.510 ms 2024-05-09T00:10:38.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:37 smithi012 ceph-mon[42503]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 548 op/s 2024-05-09T00:10:38.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:37 smithi204 ceph-mon[47373]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 548 op/s 2024-05-09T00:10:38.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:37 smithi160 ceph-mon[46152]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 548 op/s 2024-05-09T00:10:38.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 359018 28.58 MB/sec execute 513 sec latency 208.816 ms 2024-05-09T00:10:39.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 359562 28.60 MB/sec execute 514 sec latency 114.015 ms 2024-05-09T00:10:40.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:39 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:10:40.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:39 smithi012 ceph-mon[42503]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 511 op/s 2024-05-09T00:10:40.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:39 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:10:40.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:39 smithi204 ceph-mon[47373]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 511 op/s 2024-05-09T00:10:40.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:39 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:10:40.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:39 smithi160 ceph-mon[46152]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 511 op/s 2024-05-09T00:10:40.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 359995 28.59 MB/sec execute 515 sec latency 92.896 ms 2024-05-09T00:10:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.444+0000 7f75576b7700 1 -- 172.21.15.12:0/2303913229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75500682e0 msgr2=0x7f7550068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.444+0000 7f75576b7700 1 --2- 172.21.15.12:0/2303913229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75500682e0 0x7f7550068760 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f7540009990 tx=0x7f7540009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.444+0000 7f75576b7700 1 -- 172.21.15.12:0/2303913229 shutdown_connections 2024-05-09T00:10:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.444+0000 7f75576b7700 1 --2- 172.21.15.12:0/2303913229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7550068ca0 0x7f755010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.444+0000 7f75576b7700 1 --2- 172.21.15.12:0/2303913229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7550107000 0x7f75501073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.444+0000 7f75576b7700 1 --2- 172.21.15.12:0/2303913229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75500682e0 0x7f7550068760 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.444+0000 7f75576b7700 1 -- 172.21.15.12:0/2303913229 >> 172.21.15.12:0/2303913229 conn(0x7f7550076f10 msgr2=0x7f7550077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.444+0000 7f75576b7700 1 -- 172.21.15.12:0/2303913229 shutdown_connections 2024-05-09T00:10:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.444+0000 7f75576b7700 1 -- 172.21.15.12:0/2303913229 wait complete. 2024-05-09T00:10:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.445+0000 7f75576b7700 1 Processor -- start 2024-05-09T00:10:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.445+0000 7f75576b7700 1 -- start start 2024-05-09T00:10:40.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.445+0000 7f75576b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75500682e0 0x7f75501972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.445+0000 7f75576b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7550068ca0 0x7f7550197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.445+0000 7f75576b7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7550107000 0x7f755019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.445+0000 7f75576b7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755019c1e0 con 0x7f7550107000 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.445+0000 7f75576b7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755019c350 con 0x7f75500682e0 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.445+0000 7f75576b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755019c510 con 0x7f7550068ca0 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.445+0000 7f7555c54700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7550107000 0x7f755019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.445+0000 7f7555c54700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7550107000 0x7f755019bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56974/0 (socket says 172.21.15.12:56974) 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f7554c52700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7550068ca0 0x7f7550197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f7555c54700 1 -- 172.21.15.12:0/851017332 learned_addr learned my addr 172.21.15.12:0/851017332 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f7555c54700 1 -- 172.21.15.12:0/851017332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7550068ca0 msgr2=0x7f7550197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f7555c54700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7550068ca0 0x7f7550197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f7555c54700 1 -- 172.21.15.12:0/851017332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75500682e0 msgr2=0x7f75501972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f7555c54700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75500682e0 0x7f75501972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f7555c54700 1 -- 172.21.15.12:0/851017332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400095f0 con 0x7f7550107000 2024-05-09T00:10:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f7555c54700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7550107000 0x7f755019bba0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f754800ef30 tx=0x7f754800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f75467fc700 1 -- 172.21.15.12:0/851017332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548018070 con 0x7f7550107000 2024-05-09T00:10:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f75467fc700 1 -- 172.21.15.12:0/851017332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754800f040 con 0x7f7550107000 2024-05-09T00:10:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f75576b7700 1 -- 172.21.15.12:0/851017332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755019c790 con 0x7f7550107000 2024-05-09T00:10:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f75467fc700 1 -- 172.21.15.12:0/851017332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800cdb0 con 0x7f7550107000 2024-05-09T00:10:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.446+0000 7f75576b7700 1 -- 172.21.15.12:0/851017332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75501a4030 con 0x7f7550107000 2024-05-09T00:10:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.450+0000 7f75467fc700 1 -- 172.21.15.12:0/851017332 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7548003a00 con 0x7f7550107000 2024-05-09T00:10:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.450+0000 7f75467fc700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f753c0a1290 0x7f753c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f7555453700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f753c0a1290 0x7f753c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f75467fc700 1 -- 172.21.15.12:0/851017332 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f75480cc930 con 0x7f7550107000 2024-05-09T00:10:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f7563871b80 1 Processor -- start 2024-05-09T00:10:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f7563871b80 1 -- start start 2024-05-09T00:10:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f7563871b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 0x55a7f277ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f7563871b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f2774f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f7563871b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f2775480 0x55a7f27758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f7563871b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f26cc0f0 con 0x55a7f2775480 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f7563871b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f26c9ab0 con 0x55a7f277a950 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f7563871b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f26c53a0 con 0x55a7f277b270 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.451+0000 7f7555453700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f753c0a1290 0x7f753c0a3750 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f754c005950 tx=0x7f754c0058e0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7554c52700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f2774f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7554c52700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f2774f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47348/0 (socket says 172.21.15.12:47348) 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7554c52700 1 -- 172.21.15.12:0/3173360296 learned_addr learned my addr 172.21.15.12:0/3173360296 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7555453700 1 --2- 172.21.15.12:0/3173360296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 0x55a7f277ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7555c54700 1 --2- 172.21.15.12:0/3173360296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f2775480 0x55a7f27758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7554c52700 1 -- 172.21.15.12:0/3173360296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 msgr2=0x55a7f277ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7554c52700 1 --2- 172.21.15.12:0/3173360296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 0x55a7f277ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7554c52700 1 -- 172.21.15.12:0/3173360296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f2775480 msgr2=0x55a7f27758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7554c52700 1 --2- 172.21.15.12:0/3173360296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f2775480 0x55a7f27758a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7554c52700 1 -- 172.21.15.12:0/3173360296 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7f2546350 con 0x55a7f277b270 2024-05-09T00:10:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.452+0000 7f7554c52700 1 --2- 172.21.15.12:0/3173360296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f2774f40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7540009990 tx=0x7f754000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2d85ffecbaeb5e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.453+0000 7f753b7fe700 1 -- 172.21.15.12:0/3173360296 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754001d070 con 0x55a7f277b270 2024-05-09T00:10:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.453+0000 7f753b7fe700 1 -- 172.21.15.12:0/3173360296 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754001e040 con 0x55a7f277b270 2024-05-09T00:10:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.453+0000 7f753b7fe700 1 -- 172.21.15.12:0/3173360296 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540004760 con 0x55a7f277b270 2024-05-09T00:10:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.453+0000 7f7563871b80 1 -- 172.21.15.12:0/3173360296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 msgr2=0x55a7f2774f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.453+0000 7f7563871b80 1 --2- 172.21.15.12:0/3173360296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f2774f40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7540009990 tx=0x7f754000b610 comp rx=0 tx=0).stop 2024-05-09T00:10:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.453+0000 7f7563871b80 1 -- 172.21.15.12:0/3173360296 shutdown_connections 2024-05-09T00:10:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.453+0000 7f7563871b80 1 --2- 172.21.15.12:0/3173360296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f2775480 0x55a7f27758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.453+0000 7f7563871b80 1 --2- 172.21.15.12:0/3173360296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 0x55a7f277ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.453+0000 7f7563871b80 1 --2- 172.21.15.12:0/3173360296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f2774f40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.453+0000 7f7563871b80 1 -- 172.21.15.12:0/3173360296 >> 172.21.15.12:0/3173360296 conn(0x55a7f26d31b0 msgr2=0x55a7f276f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.454+0000 7f7563871b80 1 -- 172.21.15.12:0/3173360296 shutdown_connections 2024-05-09T00:10:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.454+0000 7f7563871b80 1 -- 172.21.15.12:0/3173360296 wait complete. 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.454+0000 7f7563871b80 1 -- start start 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.454+0000 7f7563871b80 1 Processor -- start 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.454+0000 7f7563871b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f2775480 0x55a7f2781180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.454+0000 7f7563871b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 0x55a7f277e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.454+0000 7f7563871b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f277eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7563871b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f2661500 con 0x55a7f2775480 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7563871b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f26611a0 con 0x55a7f277a950 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7563871b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f2548640 con 0x55a7f277b270 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7563871b80 10 client.0 _ll_register_callbacks cb 0x55a7f266aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7554c52700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 0x55a7f277e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7554c52700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 0x55a7f277e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47128/0 (socket says 172.21.15.12:47128) 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7554c52700 1 -- 172.21.15.12:0/834931702 learned_addr learned my addr 172.21.15.12:0/834931702 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7555c54700 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f277eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7554c52700 1 -- 172.21.15.12:0/834931702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 msgr2=0x55a7f277eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7554c52700 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f277eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7554c52700 1 -- 172.21.15.12:0/834931702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f2775480 msgr2=0x55a7f2781180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7554c52700 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f2775480 0x55a7f2781180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7554c52700 1 -- 172.21.15.12:0/834931702 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755010e630 con 0x55a7f277a950 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7555c54700 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f277eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7554c52700 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 0x55a7f277e470 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f754000b6c0 tx=0x7f754000ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a2b146cdc1ab0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.455+0000 7f7538ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7538ff9700 1 -- 172.21.15.12:0/834931702 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754001d030 con 0x55a7f277a950 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7563871b80 1 -- 172.21.15.12:0/834931702 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7f261e210 con 0x55a7f277a950 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7538ff9700 1 -- 172.21.15.12:0/834931702 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7540004910 con 0x55a7f277a950 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7538ff9700 1 -- 172.21.15.12:0/834931702 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f754000bea0 con 0x55a7f277a950 2024-05-09T00:10:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7563871b80 10 client.31379 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7563871b80 1 -- 172.21.15.12:0/834931702 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55a7f24f2b00 con 0x55a7f277a950 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7538ff9700 1 -- 172.21.15.12:0/834931702 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7f261e210 con 0x55a7f277a950 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7538ff9700 1 -- 172.21.15.12:0/834931702 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f754000fb00 con 0x55a7f277a950 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7563871b80 10 client.31379 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7563871b80 10 client.31379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7563871b80 20 client.31379 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7563871b80 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a7f278e800 0x55a7f2790c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7563871b80 4 client.31379 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.456+0000 7f7563871b80 1 -- 172.21.15.12:0/834931702 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7f27447b0 con 0x55a7f278e800 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.457+0000 7f7555453700 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a7f278e800 0x55a7f2790c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.457+0000 7f7555453700 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a7f278e800 0x55a7f2790c20 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfdace3baccbc774 server_cookie=3a62fb4f873712ab in_seq=0 out_seq=0 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.457+0000 7f7538ff9700 10 client.31379 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7538ff9700 1 -- 172.21.15.12:0/834931702 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a7f27447b0 con 0x55a7f278e800 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7538ff9700 10 client.31379 handle_command_reply: tid=0 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 client.31379 shutdown 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 -- 172.21.15.12:0/834931702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 msgr2=0x55a7f277e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 0x55a7f277e470 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f754000b6c0 tx=0x7f754000ba00 comp rx=0 tx=0).stop 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 -- 172.21.15.12:0/834931702 shutdown_connections 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f277eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f2775480 0x55a7f2781180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a7f278e800 0x55a7f2790c20 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 --2- 172.21.15.12:0/834931702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277a950 0x55a7f277e470 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 -- 172.21.15.12:0/834931702 >> 172.21.15.12:0/834931702 conn(0x55a7f26d31b0 msgr2=0x55a7f27794a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7538ff9700 0 client.31379 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 -- 172.21.15.12:0/834931702 shutdown_connections 2024-05-09T00:10:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 1 -- 172.21.15.12:0/834931702 wait complete. 2024-05-09T00:10:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.458+0000 7f7563871b80 20 client.31379 trim_cache size 0 max 16384 2024-05-09T00:10:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7563871b80 1 Processor -- start 2024-05-09T00:10:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7563871b80 1 -- start start 2024-05-09T00:10:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7563871b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f2775480 0x55a7f277e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7563871b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f276f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7563871b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f2795540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7563871b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f277a5e0 con 0x55a7f277a950 2024-05-09T00:10:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7563871b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f278b660 con 0x55a7f2775480 2024-05-09T00:10:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7563871b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f277e240 con 0x55a7f277b270 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7554c52700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f276f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7554c52700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f276f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56988/0 (socket says 172.21.15.12:56988) 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7555c54700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f2795540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7554c52700 1 -- 172.21.15.12:0/1932970586 learned_addr learned my addr 172.21.15.12:0/1932970586 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7555453700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f2775480 0x55a7f277e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7554c52700 1 -- 172.21.15.12:0/1932970586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 msgr2=0x55a7f2795540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7554c52700 1 --2- 172.21.15.12:0/1932970586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f2795540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7554c52700 1 -- 172.21.15.12:0/1932970586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f2775480 msgr2=0x55a7f277e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7554c52700 1 --2- 172.21.15.12:0/1932970586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f2775480 0x55a7f277e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.475+0000 7f7554c52700 1 -- 172.21.15.12:0/1932970586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7f268b500 con 0x55a7f277a950 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.476+0000 7f7554c52700 1 --2- 172.21.15.12:0/1932970586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f276f910 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f75400098e0 tx=0x7f754002c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36afdec64dd6b446 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.476+0000 7f753bfff700 1 -- 172.21.15.12:0/1932970586 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754001d070 con 0x55a7f277a950 2024-05-09T00:10:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.476+0000 7f753bfff700 1 -- 172.21.15.12:0/1932970586 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754002c7e0 con 0x55a7f277a950 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.476+0000 7f753bfff700 1 -- 172.21.15.12:0/1932970586 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540004970 con 0x55a7f277a950 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 -- 172.21.15.12:0/1932970586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 msgr2=0x55a7f276f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 --2- 172.21.15.12:0/1932970586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f276f910 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f75400098e0 tx=0x7f754002c450 comp rx=0 tx=0).stop 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 -- 172.21.15.12:0/1932970586 shutdown_connections 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 --2- 172.21.15.12:0/1932970586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f277b270 0x55a7f2795540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 --2- 172.21.15.12:0/1932970586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f2775480 0x55a7f277e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 --2- 172.21.15.12:0/1932970586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f276f910 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 -- 172.21.15.12:0/1932970586 >> 172.21.15.12:0/1932970586 conn(0x55a7f26d31b0 msgr2=0x55a7f276be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 -- 172.21.15.12:0/1932970586 shutdown_connections 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 -- 172.21.15.12:0/1932970586 wait complete. 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 -- start start 2024-05-09T00:10:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.477+0000 7f7563871b80 1 Processor -- start 2024-05-09T00:10:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7563871b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f2775480 0x55a7f2785c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7563871b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f2786150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7563871b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277b270 0x55a7f2783ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7563871b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f27843f0 con 0x55a7f277a950 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7563871b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f2784560 con 0x55a7f277b270 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7563871b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f2784720 con 0x55a7f2775480 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7563871b80 10 client.0 _ll_register_callbacks cb 0x55a7f266aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7554c52700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f2786150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7554c52700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f2786150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57002/0 (socket says 172.21.15.12:57002) 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7555453700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f2775480 0x55a7f2785c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7554c52700 1 -- 172.21.15.12:0/2445586441 learned_addr learned my addr 172.21.15.12:0/2445586441 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7555c54700 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277b270 0x55a7f2783ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7554c52700 1 -- 172.21.15.12:0/2445586441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f2775480 msgr2=0x55a7f2785c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7554c52700 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f2775480 0x55a7f2785c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7554c52700 1 -- 172.21.15.12:0/2445586441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277b270 msgr2=0x55a7f2783ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7554c52700 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277b270 0x55a7f2783ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7554c52700 1 -- 172.21.15.12:0/2445586441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7f24f2b00 con 0x55a7f277a950 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7555453700 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f2775480 0x55a7f2785c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7554c52700 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f2786150 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f754003a040 tx=0x7f754002c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a57d40a6ef9391 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.478+0000 7f7538ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7538ff9700 1 -- 172.21.15.12:0/2445586441 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754001d070 con 0x55a7f277a950 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7538ff9700 1 -- 172.21.15.12:0/2445586441 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7540000e40 con 0x55a7f277a950 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7563871b80 1 -- 172.21.15.12:0/2445586441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f754001e040 con 0x55a7f277a950 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7538ff9700 1 -- 172.21.15.12:0/2445586441 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f754002cb50 con 0x55a7f277a950 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7563871b80 10 client.21993 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7563871b80 1 -- 172.21.15.12:0/2445586441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55a7f254ac70 con 0x55a7f277a950 2024-05-09T00:10:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7538ff9700 1 -- 172.21.15.12:0/2445586441 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754001e040 con 0x55a7f277a950 2024-05-09T00:10:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7538ff9700 1 -- 172.21.15.12:0/2445586441 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f754000f460 con 0x55a7f277a950 2024-05-09T00:10:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7563871b80 10 client.21993 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7563871b80 10 client.21993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7563871b80 20 client.21993 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7563871b80 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a7f27e4c50 0x55a7f27e7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7563871b80 4 client.21993 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:10:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7563871b80 1 -- 172.21.15.12:0/2445586441 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a7f26bde60 con 0x55a7f27e4c50 2024-05-09T00:10:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.479+0000 7f7555453700 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a7f27e4c50 0x55a7f27e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.480+0000 7f7538ff9700 10 client.21993 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.480+0000 7f7555453700 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a7f27e4c50 0x55a7f27e7070 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d05600203e46c8 server_cookie=16aa21479c358e2 in_seq=0 out_seq=0 2024-05-09T00:10:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.930+0000 7f7538ff9700 1 -- 172.21.15.12:0/2445586441 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a7f26bde60 con 0x55a7f27e4c50 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.930+0000 7f7538ff9700 10 client.21993 handle_command_reply: tid=0 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.930+0000 7f7563871b80 1 client.21993 shutdown 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.930+0000 7f7563871b80 1 -- 172.21.15.12:0/2445586441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 msgr2=0x55a7f2786150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.930+0000 7f7563871b80 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f2786150 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f754003a040 tx=0x7f754002c720 comp rx=0 tx=0).stop 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.930+0000 7f7563871b80 1 -- 172.21.15.12:0/2445586441 shutdown_connections 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.930+0000 7f7563871b80 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55a7f27e4c50 0x55a7f27e7070 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f7563871b80 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a7f277b270 0x55a7f2783ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f7563871b80 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7f2775480 0x55a7f2785c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f7538ff9700 0 client.21993 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f7563871b80 1 --2- 172.21.15.12:0/2445586441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f277a950 0x55a7f2786150 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f7563871b80 1 -- 172.21.15.12:0/2445586441 >> 172.21.15.12:0/2445586441 conn(0x55a7f26d31b0 msgr2=0x55a7f277e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f7563871b80 1 -- 172.21.15.12:0/2445586441 shutdown_connections 2024-05-09T00:10:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f7563871b80 1 -- 172.21.15.12:0/2445586441 wait complete. 2024-05-09T00:10:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f7563871b80 20 client.21993 trim_cache size 0 max 16384 2024-05-09T00:10:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:10:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:10:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:10:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:10:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f75576b7700 1 -- 172.21.15.12:0/851017332 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f753c0a1290 msgr2=0x7f753c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f75576b7700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f753c0a1290 0x7f753c0a3750 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f754c005950 tx=0x7f754c0058e0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f75576b7700 1 -- 172.21.15.12:0/851017332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7550107000 msgr2=0x7f755019bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f75576b7700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7550107000 0x7f755019bba0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f754800ef30 tx=0x7f754800c3a0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f75576b7700 1 -- 172.21.15.12:0/851017332 shutdown_connections 2024-05-09T00:10:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f75576b7700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f753c0a1290 0x7f753c0a3750 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f75576b7700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7550107000 0x7f755019bba0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f75576b7700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75500682e0 0x7f75501972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f75576b7700 1 --2- 172.21.15.12:0/851017332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7550068ca0 0x7f7550197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.931+0000 7f75576b7700 1 -- 172.21.15.12:0/851017332 >> 172.21.15.12:0/851017332 conn(0x7f7550076f10 msgr2=0x7f75500feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.932+0000 7f75576b7700 1 -- 172.21.15.12:0/851017332 shutdown_connections 2024-05-09T00:10:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:40.932+0000 7f75576b7700 1 -- 172.21.15.12:0/851017332 wait complete. 2024-05-09T00:10:41.313 INFO:tasks.workunit.client.0.smithi012.stdout: 10 360248 28.56 MB/sec execute 516 sec latency 379.698 ms 2024-05-09T00:10:42.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:41 smithi012 ceph-mon[42503]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.0 MiB/s wr, 459 op/s 2024-05-09T00:10:42.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:41 smithi204 ceph-mon[47373]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.0 MiB/s wr, 459 op/s 2024-05-09T00:10:42.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:41 smithi160 ceph-mon[46152]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.0 MiB/s wr, 459 op/s 2024-05-09T00:10:42.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 360735 28.55 MB/sec execute 517 sec latency 121.824 ms 2024-05-09T00:10:43.313 INFO:tasks.workunit.client.0.smithi012.stdout: 10 361090 28.53 MB/sec execute 518 sec latency 204.257 ms 2024-05-09T00:10:44.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:43 smithi012 ceph-mon[42503]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 633 op/s 2024-05-09T00:10:44.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:43 smithi204 ceph-mon[47373]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 633 op/s 2024-05-09T00:10:44.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:43 smithi160 ceph-mon[46152]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 633 op/s 2024-05-09T00:10:44.313 INFO:tasks.workunit.client.0.smithi012.stdout: 10 361427 28.52 MB/sec execute 519 sec latency 199.815 ms 2024-05-09T00:10:45.219 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:10:45.313 INFO:tasks.workunit.client.0.smithi012.stdout: 10 361781 28.49 MB/sec execute 520 sec latency 155.052 ms 2024-05-09T00:10:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.395+0000 7efc27a7d700 1 -- 172.21.15.12:0/3279617012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc20069420 msgr2=0x7efc20069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.395+0000 7efc27a7d700 1 --2- 172.21.15.12:0/3279617012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc20069420 0x7efc20069800 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009990 tx=0x7efc1c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.395+0000 7efc27a7d700 1 -- 172.21.15.12:0/3279617012 shutdown_connections 2024-05-09T00:10:45.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.395+0000 7efc27a7d700 1 --2- 172.21.15.12:0/3279617012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc201062a0 0x7efc2010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.395+0000 7efc27a7d700 1 --2- 172.21.15.12:0/3279617012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc20069420 0x7efc20069800 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.395+0000 7efc27a7d700 1 --2- 172.21.15.12:0/3279617012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc20069dd0 0x7efc20105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.395+0000 7efc27a7d700 1 -- 172.21.15.12:0/3279617012 >> 172.21.15.12:0/3279617012 conn(0x7efc20076ba0 msgr2=0x7efc20076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:45.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.396+0000 7efc27a7d700 1 -- 172.21.15.12:0/3279617012 shutdown_connections 2024-05-09T00:10:45.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.396+0000 7efc27a7d700 1 -- 172.21.15.12:0/3279617012 wait complete. 2024-05-09T00:10:45.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.396+0000 7efc27a7d700 1 Processor -- start 2024-05-09T00:10:45.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.396+0000 7efc27a7d700 1 -- start start 2024-05-09T00:10:45.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc27a7d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc20069420 0x7efc201001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc27a7d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc201062a0 0x7efc20100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc27a7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc20100c40 0x7efc201a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc27a7d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc201a5b80 con 0x7efc20100c40 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc27a7d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc201a5cf0 con 0x7efc20069420 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc27a7d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc201a5eb0 con 0x7efc201062a0 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc25018700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc201062a0 0x7efc20100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc25819700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc20069420 0x7efc201001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc2601a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc20100c40 0x7efc201a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc25018700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc201062a0 0x7efc20100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47400/0 (socket says 172.21.15.12:47400) 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc25018700 1 -- 172.21.15.12:0/3162210337 learned_addr learned my addr 172.21.15.12:0/3162210337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc25819700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc20069420 0x7efc201001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47166/0 (socket says 172.21.15.12:47166) 2024-05-09T00:10:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc2601a700 1 -- 172.21.15.12:0/3162210337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc201062a0 msgr2=0x7efc20100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc2601a700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc201062a0 0x7efc20100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc2601a700 1 -- 172.21.15.12:0/3162210337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc20069420 msgr2=0x7efc201001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc2601a700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc20069420 0x7efc201001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc2601a700 1 -- 172.21.15.12:0/3162210337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0095f0 con 0x7efc20100c40 2024-05-09T00:10:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc25819700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc20069420 0x7efc201001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.397+0000 7efc2601a700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc20100c40 0x7efc201a55b0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7efc1800efc0 tx=0x7efc1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.398+0000 7efc16ffd700 1 -- 172.21.15.12:0/3162210337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18018070 con 0x7efc20100c40 2024-05-09T00:10:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.398+0000 7efc16ffd700 1 -- 172.21.15.12:0/3162210337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc1800f040 con 0x7efc20100c40 2024-05-09T00:10:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.398+0000 7efc27a7d700 1 -- 172.21.15.12:0/3162210337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc201a6130 con 0x7efc20100c40 2024-05-09T00:10:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.398+0000 7efc16ffd700 1 -- 172.21.15.12:0/3162210337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800cd40 con 0x7efc20100c40 2024-05-09T00:10:45.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.398+0000 7efc27a7d700 1 -- 172.21.15.12:0/3162210337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc201a65f0 con 0x7efc20100c40 2024-05-09T00:10:45.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.400+0000 7efc16ffd700 1 -- 172.21.15.12:0/3162210337 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efc18003a00 con 0x7efc20100c40 2024-05-09T00:10:45.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.401+0000 7efc16ffd700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efc0c0a1290 0x7efc0c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.401+0000 7efc16ffd700 1 -- 172.21.15.12:0/3162210337 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7efc180cc880 con 0x7efc20100c40 2024-05-09T00:10:45.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.401+0000 7efc25819700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efc0c0a1290 0x7efc0c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.402+0000 7efc33c37b80 1 Processor -- start 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.402+0000 7efc25819700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efc0c0a1290 0x7efc0c0a3750 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00b2e0 tx=0x7efc1c005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.402+0000 7efc33c37b80 1 -- start start 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.402+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da1e00 0x5577b7da21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.402+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7da2720 0x5577b7d9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.402+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7d9c890 0x5577b7d9ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc33c37b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7cf3150 con 0x5577b7da2720 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc33c37b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7cf0b10 con 0x5577b7da1e00 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc33c37b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7cec400 con 0x5577b7d9c890 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25018700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7da2720 0x5577b7d9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25819700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da1e00 0x5577b7da21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25018700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7da2720 0x5577b7d9c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57032/0 (socket says 172.21.15.12:57032) 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25819700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da1e00 0x5577b7da21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47170/0 (socket says 172.21.15.12:47170) 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25018700 1 -- 172.21.15.12:0/1335857349 learned_addr learned my addr 172.21.15.12:0/1335857349 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc2601a700 1 --2- 172.21.15.12:0/1335857349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7d9c890 0x5577b7d9ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25018700 1 -- 172.21.15.12:0/1335857349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7d9c890 msgr2=0x5577b7d9ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25018700 1 --2- 172.21.15.12:0/1335857349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7d9c890 0x5577b7d9ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25018700 1 -- 172.21.15.12:0/1335857349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da1e00 msgr2=0x5577b7da21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25018700 1 --2- 172.21.15.12:0/1335857349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da1e00 0x5577b7da21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25018700 1 -- 172.21.15.12:0/1335857349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577b7b6d3b0 con 0x5577b7da2720 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25819700 1 --2- 172.21.15.12:0/1335857349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da1e00 0x5577b7da21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc25018700 1 --2- 172.21.15.12:0/1335857349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7da2720 0x5577b7d9c350 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7efc10009d90 tx=0x7efc10011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e35058e7d430c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.403+0000 7efc0bfff700 1 -- 172.21.15.12:0/1335857349 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10004a30 con 0x5577b7da2720 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.404+0000 7efc0bfff700 1 -- 172.21.15.12:0/1335857349 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc100054e0 con 0x5577b7da2720 2024-05-09T00:10:45.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.404+0000 7efc0bfff700 1 -- 172.21.15.12:0/1335857349 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100197f0 con 0x5577b7da2720 2024-05-09T00:10:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.404+0000 7efc33c37b80 1 -- 172.21.15.12:0/1335857349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7da2720 msgr2=0x5577b7d9c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.404+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1335857349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7da2720 0x5577b7d9c350 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7efc10009d90 tx=0x7efc10011720 comp rx=0 tx=0).stop 2024-05-09T00:10:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.404+0000 7efc33c37b80 1 -- 172.21.15.12:0/1335857349 shutdown_connections 2024-05-09T00:10:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.404+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1335857349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7d9c890 0x5577b7d9ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.404+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1335857349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da1e00 0x5577b7da21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.404+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1335857349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7da2720 0x5577b7d9c350 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.404+0000 7efc33c37b80 1 -- 172.21.15.12:0/1335857349 >> 172.21.15.12:0/1335857349 conn(0x5577b7cfa210 msgr2=0x5577b7d96df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.405+0000 7efc33c37b80 1 -- 172.21.15.12:0/1335857349 shutdown_connections 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.405+0000 7efc33c37b80 1 -- 172.21.15.12:0/1335857349 wait complete. 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.405+0000 7efc33c37b80 1 -- start start 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.405+0000 7efc33c37b80 1 Processor -- start 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7da5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7da60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7da6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc33c37b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7c88560 con 0x5577b7d9c890 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc33c37b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7c88200 con 0x5577b7da2720 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc33c37b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7b6f6a0 con 0x5577b7da1e00 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc33c37b80 10 client.0 _ll_register_callbacks cb 0x5577b7c91b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc25819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7da5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc2601a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7da6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc25819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7da5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57042/0 (socket says 172.21.15.12:57042) 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc25819700 1 -- 172.21.15.12:0/3032172691 learned_addr learned my addr 172.21.15.12:0/3032172691 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc25018700 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7da60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc2601a700 1 -- 172.21.15.12:0/3032172691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 msgr2=0x5577b7da60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc2601a700 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7da60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc2601a700 1 -- 172.21.15.12:0/3032172691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 msgr2=0x5577b7da5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc2601a700 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7da5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc2601a700 1 -- 172.21.15.12:0/3032172691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2010e690 con 0x5577b7da2720 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.406+0000 7efc25819700 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7da5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc2601a700 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7da6630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efc180c9fb0 tx=0x7efc180993d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3dff2c197f7f621 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:45.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc097fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc097fa700 1 -- 172.21.15.12:0/3032172691 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18004ad0 con 0x5577b7da2720 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc097fa700 1 -- 172.21.15.12:0/3032172691 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7efc1809a100 con 0x5577b7da2720 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc33c37b80 1 -- 172.21.15.12:0/3032172691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577b7c45270 con 0x5577b7da2720 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc097fa700 1 -- 172.21.15.12:0/3032172691 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc18099a00 con 0x5577b7da2720 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc33c37b80 10 client.31397 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc33c37b80 1 -- 172.21.15.12:0/3032172691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7efc100099f0 con 0x5577b7da2720 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc097fa700 1 -- 172.21.15.12:0/3032172691 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577b7c45270 con 0x5577b7da2720 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc097fa700 1 -- 172.21.15.12:0/3032172691 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7efc18098fd0 con 0x5577b7da2720 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc33c37b80 10 client.31397 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.407+0000 7efc33c37b80 10 client.31397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.408+0000 7efc33c37b80 20 client.31397 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.408+0000 7efc33c37b80 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5577b7db61e0 0x5577b7db8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.408+0000 7efc33c37b80 4 client.31397 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.408+0000 7efc33c37b80 1 -- 172.21.15.12:0/3032172691 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577b7d6b810 con 0x5577b7db61e0 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.408+0000 7efc25819700 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5577b7db61e0 0x5577b7db8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.408+0000 7efc097fa700 10 client.31397 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.408+0000 7efc25819700 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5577b7db61e0 0x5577b7db8600 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e10ce98aa98af6 server_cookie=2602ab4c5a6e1241 in_seq=0 out_seq=0 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc097fa700 1 -- 172.21.15.12:0/3032172691 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5577b7d6b810 con 0x5577b7db61e0 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc097fa700 10 client.31397 handle_command_reply: tid=0 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 client.31397 shutdown 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 -- 172.21.15.12:0/3032172691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 msgr2=0x5577b7da6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7da6630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efc180c9fb0 tx=0x7efc180993d0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 -- 172.21.15.12:0/3032172691 shutdown_connections 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5577b7db61e0 0x5577b7db8600 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7da6630 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7da5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc097fa700 0 client.31397 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 --2- 172.21.15.12:0/3032172691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7da60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 -- 172.21.15.12:0/3032172691 >> 172.21.15.12:0/3032172691 conn(0x5577b7cfa210 msgr2=0x5577b7d965a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 -- 172.21.15.12:0/3032172691 shutdown_connections 2024-05-09T00:10:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.409+0000 7efc33c37b80 1 -- 172.21.15.12:0/3032172691 wait complete. 2024-05-09T00:10:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.410+0000 7efc33c37b80 20 client.31397 trim_cache size 0 max 16384 2024-05-09T00:10:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc33c37b80 1 Processor -- start 2024-05-09T00:10:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc33c37b80 1 -- start start 2024-05-09T00:10:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7db2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7d96d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7da6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc33c37b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7d93dc0 con 0x5577b7d9c890 2024-05-09T00:10:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc33c37b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7d97ca0 con 0x5577b7da2720 2024-05-09T00:10:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc33c37b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7d98e50 con 0x5577b7da1e00 2024-05-09T00:10:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc25819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7db2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc25819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7db2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57052/0 (socket says 172.21.15.12:57052) 2024-05-09T00:10:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc25819700 1 -- 172.21.15.12:0/1766041988 learned_addr learned my addr 172.21.15.12:0/1766041988 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.432+0000 7efc25018700 1 --2- 172.21.15.12:0/1766041988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7d96d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.433+0000 7efc25819700 1 -- 172.21.15.12:0/1766041988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 msgr2=0x5577b7d96d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.433+0000 7efc25819700 1 --2- 172.21.15.12:0/1766041988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7d96d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.433+0000 7efc25819700 1 -- 172.21.15.12:0/1766041988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 msgr2=0x5577b7da6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.433+0000 7efc25819700 1 --2- 172.21.15.12:0/1766041988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7da6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.433+0000 7efc25819700 1 -- 172.21.15.12:0/1766041988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577b7b19b60 con 0x5577b7d9c890 2024-05-09T00:10:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.433+0000 7efc25819700 1 --2- 172.21.15.12:0/1766041988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7db2370 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0059d0 tx=0x7efc1c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdff7d03a8d5ff3d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:45.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.433+0000 7efc14da4700 1 -- 172.21.15.12:0/1766041988 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c02b040 con 0x5577b7d9c890 2024-05-09T00:10:45.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.433+0000 7efc14da4700 1 -- 172.21.15.12:0/1766041988 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc1c02c040 con 0x5577b7d9c890 2024-05-09T00:10:45.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.433+0000 7efc14da4700 1 -- 172.21.15.12:0/1766041988 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c036830 con 0x5577b7d9c890 2024-05-09T00:10:45.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.434+0000 7efc33c37b80 1 -- 172.21.15.12:0/1766041988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 msgr2=0x5577b7db2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.434+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1766041988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7db2370 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0059d0 tx=0x7efc1c020000 comp rx=0 tx=0).stop 2024-05-09T00:10:45.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.434+0000 7efc33c37b80 1 -- 172.21.15.12:0/1766041988 shutdown_connections 2024-05-09T00:10:45.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.434+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1766041988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7da6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.434+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1766041988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7db2370 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.434+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1766041988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7d96d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.434+0000 7efc33c37b80 1 -- 172.21.15.12:0/1766041988 >> 172.21.15.12:0/1766041988 conn(0x5577b7cfa210 msgr2=0x5577b7da4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:45.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.438+0000 7efc33c37b80 1 -- 172.21.15.12:0/1766041988 shutdown_connections 2024-05-09T00:10:45.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.442+0000 7efc33c37b80 1 -- 172.21.15.12:0/1766041988 wait complete. 2024-05-09T00:10:45.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.443+0000 7efc33c37b80 1 -- start start 2024-05-09T00:10:45.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.443+0000 7efc33c37b80 1 Processor -- start 2024-05-09T00:10:45.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.443+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7dab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.443+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7dab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.443+0000 7efc33c37b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7dabbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.443+0000 7efc33c37b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7dada60 con 0x5577b7d9c890 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.443+0000 7efc33c37b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7e0b6a0 con 0x5577b7da2720 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.443+0000 7efc33c37b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7e13ff0 con 0x5577b7da1e00 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.443+0000 7efc33c37b80 10 client.0 _ll_register_callbacks cb 0x5577b7c91b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7dab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25018700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7dab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc2601a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7dabbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25018700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7dab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47432/0 (socket says 172.21.15.12:47432) 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc2601a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7dabbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47196/0 (socket says 172.21.15.12:47196) 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25819700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7dab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57060/0 (socket says 172.21.15.12:57060) 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25018700 1 -- 172.21.15.12:0/1758928855 learned_addr learned my addr 172.21.15.12:0/1758928855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25819700 1 -- 172.21.15.12:0/1758928855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 msgr2=0x5577b7dab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25819700 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7dab690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25819700 1 -- 172.21.15.12:0/1758928855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 msgr2=0x5577b7dabbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25819700 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7dabbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25819700 1 -- 172.21.15.12:0/1758928855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1c0056d0 con 0x5577b7d9c890 2024-05-09T00:10:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25018700 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7dab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc25819700 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7dab150 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7efc1c000c00 tx=0x7efc1c036200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dc8e50829617f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.444+0000 7efc097fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc097fa700 1 -- 172.21.15.12:0/1758928855 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c02b070 con 0x5577b7d9c890 2024-05-09T00:10:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc097fa700 1 -- 172.21.15.12:0/1758928855 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7efc1c036e40 con 0x5577b7d9c890 2024-05-09T00:10:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc33c37b80 1 -- 172.21.15.12:0/1758928855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc100054e0 con 0x5577b7d9c890 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc097fa700 1 -- 172.21.15.12:0/1758928855 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc1c0364b0 con 0x5577b7d9c890 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc33c37b80 10 client.22011 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc33c37b80 1 -- 172.21.15.12:0/1758928855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x5577b7cb2560 con 0x5577b7d9c890 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc097fa700 1 -- 172.21.15.12:0/1758928855 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc100054e0 con 0x5577b7d9c890 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc097fa700 1 -- 172.21.15.12:0/1758928855 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7efc1c004500 con 0x5577b7d9c890 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc33c37b80 10 client.22011 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc33c37b80 10 client.22011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc33c37b80 20 client.22011 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.445+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5577b7e15370 0x5577b7e17790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.446+0000 7efc33c37b80 4 client.22011 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.446+0000 7efc33c37b80 1 -- 172.21.15.12:0/1758928855 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577b7ce4ec0 con 0x5577b7e15370 2024-05-09T00:10:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.446+0000 7efc25018700 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5577b7e15370 0x5577b7e17790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.453 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:10:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:10:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:10:45.453 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:10:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.452+0000 7efc097fa700 10 client.22011 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.452+0000 7efc25018700 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5577b7e15370 0x5577b7e17790 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d652d4be5755f5 server_cookie=f41e7bdcaa356fc7 in_seq=0 out_seq=0 2024-05-09T00:10:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc097fa700 1 -- 172.21.15.12:0/1758928855 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5577b7ce4ec0 con 0x5577b7e15370 2024-05-09T00:10:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc097fa700 10 client.22011 handle_command_reply: tid=0 2024-05-09T00:10:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc33c37b80 1 client.22011 shutdown 2024-05-09T00:10:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc33c37b80 1 -- 172.21.15.12:0/1758928855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 msgr2=0x5577b7dab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7dab150 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7efc1c000c00 tx=0x7efc1c036200 comp rx=0 tx=0).stop 2024-05-09T00:10:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc33c37b80 1 -- 172.21.15.12:0/1758928855 shutdown_connections 2024-05-09T00:10:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5577b7e15370 0x5577b7e17790 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5577b7da2720 0x5577b7dabbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc097fa700 0 client.22011 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5577b7d9c890 0x5577b7dab150 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc33c37b80 1 --2- 172.21.15.12:0/1758928855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b7da1e00 0x5577b7dab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.453+0000 7efc33c37b80 1 -- 172.21.15.12:0/1758928855 >> 172.21.15.12:0/1758928855 conn(0x5577b7cfa210 msgr2=0x5577b7da0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.454+0000 7efc33c37b80 1 -- 172.21.15.12:0/1758928855 shutdown_connections 2024-05-09T00:10:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.454+0000 7efc33c37b80 1 -- 172.21.15.12:0/1758928855 wait complete. 2024-05-09T00:10:45.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.454+0000 7efc33c37b80 20 client.22011 trim_cache size 0 max 16384 2024-05-09T00:10:45.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.462+0000 7efc27a7d700 1 -- 172.21.15.12:0/3162210337 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efc0c0a1290 msgr2=0x7efc0c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.462+0000 7efc27a7d700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efc0c0a1290 0x7efc0c0a3750 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00b2e0 tx=0x7efc1c005960 comp rx=0 tx=0).stop 2024-05-09T00:10:45.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.462+0000 7efc27a7d700 1 -- 172.21.15.12:0/3162210337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc20100c40 msgr2=0x7efc201a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.462+0000 7efc27a7d700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc20100c40 0x7efc201a55b0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7efc1800efc0 tx=0x7efc1800c3a0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.463+0000 7efc27a7d700 1 -- 172.21.15.12:0/3162210337 shutdown_connections 2024-05-09T00:10:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.463+0000 7efc27a7d700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7efc0c0a1290 0x7efc0c0a3750 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.463+0000 7efc27a7d700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc20100c40 0x7efc201a55b0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.463+0000 7efc27a7d700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc20069420 0x7efc201001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.463+0000 7efc27a7d700 1 --2- 172.21.15.12:0/3162210337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc201062a0 0x7efc20100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.463+0000 7efc27a7d700 1 -- 172.21.15.12:0/3162210337 >> 172.21.15.12:0/3162210337 conn(0x7efc20076ba0 msgr2=0x7efc20107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.463+0000 7efc27a7d700 1 -- 172.21.15.12:0/3162210337 shutdown_connections 2024-05-09T00:10:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.463+0000 7efc27a7d700 1 -- 172.21.15.12:0/3162210337 wait complete. 2024-05-09T00:10:45.479 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:10:45.479 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:10:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.638+0000 7f23385f6700 1 -- 172.21.15.12:0/3671852706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f233010b400 msgr2=0x7f233010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.638+0000 7f23385f6700 1 --2- 172.21.15.12:0/3671852706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f233010b400 0x7f233010d870 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f2328009f90 tx=0x7f232800b330 comp rx=0 tx=0).stop 2024-05-09T00:10:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.638+0000 7f23385f6700 1 -- 172.21.15.12:0/3671852706 shutdown_connections 2024-05-09T00:10:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.638+0000 7f23385f6700 1 --2- 172.21.15.12:0/3671852706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f233010b400 0x7f233010d870 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.638+0000 7f23385f6700 1 --2- 172.21.15.12:0/3671852706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2330101fe0 0x7f23301023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.638+0000 7f23385f6700 1 --2- 172.21.15.12:0/3671852706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2330102900 0x7f233010ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.638+0000 7f23385f6700 1 -- 172.21.15.12:0/3671852706 >> 172.21.15.12:0/3671852706 conn(0x7f23300fb7e0 msgr2=0x7f23300fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.638+0000 7f23385f6700 1 -- 172.21.15.12:0/3671852706 shutdown_connections 2024-05-09T00:10:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.638+0000 7f23385f6700 1 -- 172.21.15.12:0/3671852706 wait complete. 2024-05-09T00:10:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f23385f6700 1 Processor -- start 2024-05-09T00:10:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f23385f6700 1 -- start start 2024-05-09T00:10:45.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f23385f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2330101fe0 0x7f23300ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f23385f6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2330102900 0x7f2330100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f23385f6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f233010b400 0x7f23301006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f23385f6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23301a5890 con 0x7f2330101fe0 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f23385f6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23301a5a00 con 0x7f233010b400 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f23385f6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23301a5bc0 con 0x7f2330102900 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f2336392700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2330101fe0 0x7f23300ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f2336392700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2330101fe0 0x7f23300ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57080/0 (socket says 172.21.15.12:57080) 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.639+0000 7f2336392700 1 -- 172.21.15.12:0/2034554639 learned_addr learned my addr 172.21.15.12:0/2034554639 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f2335b91700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2330102900 0x7f2330100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f2336392700 1 -- 172.21.15.12:0/2034554639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2330102900 msgr2=0x7f2330100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f2336392700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2330102900 0x7f2330100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f2336392700 1 -- 172.21.15.12:0/2034554639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f233010b400 msgr2=0x7f23301006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f2336392700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f233010b400 0x7f23301006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f2336392700 1 -- 172.21.15.12:0/2034554639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2328009c40 con 0x7f2330101fe0 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f2335b91700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2330102900 0x7f2330100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f2336392700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2330101fe0 0x7f23300ffc30 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f232c00e9c0 tx=0x7f232c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f23277fe700 1 -- 172.21.15.12:0/2034554639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00cbd0 con 0x7f2330101fe0 2024-05-09T00:10:45.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f23385f6700 1 -- 172.21.15.12:0/2034554639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23301a5ea0 con 0x7f2330101fe0 2024-05-09T00:10:45.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f23277fe700 1 -- 172.21.15.12:0/2034554639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f232c00cd30 con 0x7f2330101fe0 2024-05-09T00:10:45.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f23277fe700 1 -- 172.21.15.12:0/2034554639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c005590 con 0x7f2330101fe0 2024-05-09T00:10:45.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.640+0000 7f23385f6700 1 -- 172.21.15.12:0/2034554639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23301a6380 con 0x7f2330101fe0 2024-05-09T00:10:45.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.641+0000 7f23385f6700 1 -- 172.21.15.12:0/2034554639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2330106530 con 0x7f2330101fe0 2024-05-09T00:10:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.643+0000 7f23277fe700 1 -- 172.21.15.12:0/2034554639 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f232c004750 con 0x7f2330101fe0 2024-05-09T00:10:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.644+0000 7f23277fe700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f231c0a1290 0x7f231c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.644+0000 7f2335b91700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f231c0a1290 0x7f231c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.644+0000 7f23277fe700 1 -- 172.21.15.12:0/2034554639 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f232c014070 con 0x7f2330101fe0 2024-05-09T00:10:45.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.644+0000 7f2335b91700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f231c0a1290 0x7f231c0a3750 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2330101360 tx=0x7f2320008040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:45.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.650+0000 7f23277fe700 1 -- 172.21.15.12:0/2034554639 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f232c096450 con 0x7f2330101fe0 2024-05-09T00:10:45.828 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:45 smithi012 ceph-mon[42503]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.1 MiB/s wr, 450 op/s 2024-05-09T00:10:45.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.981+0000 7f23385f6700 1 -- 172.21.15.12:0/2034554639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2330068890 con 0x7f2330101fe0 2024-05-09T00:10:45.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.982+0000 7f23277fe700 1 -- 172.21.15.12:0/2034554639 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f232c099a70 con 0x7f2330101fe0 2024-05-09T00:10:45.980 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:10:45.980 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":21435,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/676326208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":676326208},{"type":"v1","addr":"172.21.15.160:6833","nonce":676326208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:10:38.863191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_31639":{"gid":31639,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1830773792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1830773792},{"type":"v1","addr":"172.21.15.204:6833","nonce":1830773792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:10:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.984+0000 7f23385f6700 1 -- 172.21.15.12:0/2034554639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f231c0a1290 msgr2=0x7f231c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f231c0a1290 0x7f231c0a3750 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2330101360 tx=0x7f2320008040 comp rx=0 tx=0).stop 2024-05-09T00:10:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 -- 172.21.15.12:0/2034554639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2330101fe0 msgr2=0x7f23300ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2330101fe0 0x7f23300ffc30 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f232c00e9c0 tx=0x7f232c00ed80 comp rx=0 tx=0).stop 2024-05-09T00:10:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 -- 172.21.15.12:0/2034554639 shutdown_connections 2024-05-09T00:10:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f231c0a1290 0x7f231c0a3750 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f233010b400 0x7f23301006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2330101fe0 0x7f23300ffc30 secure :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f232c00e9c0 tx=0x7f232c00ed80 comp rx=0 tx=0).stop 2024-05-09T00:10:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 --2- 172.21.15.12:0/2034554639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2330102900 0x7f2330100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 -- 172.21.15.12:0/2034554639 >> 172.21.15.12:0/2034554639 conn(0x7f23300fb7e0 msgr2=0x7f23300fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 -- 172.21.15.12:0/2034554639 shutdown_connections 2024-05-09T00:10:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:45.985+0000 7f23385f6700 1 -- 172.21.15.12:0/2034554639 wait complete. 2024-05-09T00:10:45.984 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 83 2024-05-09T00:10:45.999 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:10:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.170+0000 7fe256857700 1 -- 172.21.15.12:0/2781496062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250101b10 msgr2=0x7fe250105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.170+0000 7fe256857700 1 --2- 172.21.15.12:0/2781496062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250101b10 0x7fe250105b50 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fe244009990 tx=0x7fe244009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.171+0000 7fe256857700 1 -- 172.21.15.12:0/2781496062 shutdown_connections 2024-05-09T00:10:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.171+0000 7fe256857700 1 --2- 172.21.15.12:0/2781496062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe250106280 0x7fe25010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.171+0000 7fe256857700 1 --2- 172.21.15.12:0/2781496062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe250101160 0x7fe250101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.171+0000 7fe256857700 1 --2- 172.21.15.12:0/2781496062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250101b10 0x7fe250105b50 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.171+0000 7fe256857700 1 -- 172.21.15.12:0/2781496062 >> 172.21.15.12:0/2781496062 conn(0x7fe2500fc9c0 msgr2=0x7fe2500fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.171+0000 7fe256857700 1 -- 172.21.15.12:0/2781496062 shutdown_connections 2024-05-09T00:10:46.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.171+0000 7fe256857700 1 -- 172.21.15.12:0/2781496062 wait complete. 2024-05-09T00:10:46.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.171+0000 7fe256857700 1 Processor -- start 2024-05-09T00:10:46.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.171+0000 7fe256857700 1 -- start start 2024-05-09T00:10:46.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.172+0000 7fe256857700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe250101160 0x7fe2500732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.172+0000 7fe256857700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe250106280 0x7fe2500737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.172+0000 7fe256857700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250074f60 0x7fe250073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.172+0000 7fe256857700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2501a9fe0 con 0x7fe250074f60 2024-05-09T00:10:46.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.172+0000 7fe256857700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25019b540 con 0x7fe250106280 2024-05-09T00:10:46.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.172+0000 7fe256857700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25019b700 con 0x7fe250101160 2024-05-09T00:10:46.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.172+0000 7fe254df4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250074f60 0x7fe250073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.172+0000 7fe254df4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250074f60 0x7fe250073d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57092/0 (socket says 172.21.15.12:57092) 2024-05-09T00:10:46.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.172+0000 7fe254df4700 1 -- 172.21.15.12:0/3865615683 learned_addr learned my addr 172.21.15.12:0/3865615683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.173+0000 7fe254df4700 1 -- 172.21.15.12:0/3865615683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe250101160 msgr2=0x7fe2500732a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:10:46.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.173+0000 7fe254df4700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe250101160 0x7fe2500732a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.173+0000 7fe254df4700 1 -- 172.21.15.12:0/3865615683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe250106280 msgr2=0x7fe2500737e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:10:46.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.173+0000 7fe254df4700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe250106280 0x7fe2500737e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.173+0000 7fe254df4700 1 -- 172.21.15.12:0/3865615683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2440095f0 con 0x7fe250074f60 2024-05-09T00:10:46.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.173+0000 7fe254df4700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250074f60 0x7fe250073d60 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fe248007ee0 tx=0x7fe24800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.174+0000 7fe24d7fa700 1 -- 172.21.15.12:0/3865615683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248017070 con 0x7fe250074f60 2024-05-09T00:10:46.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.175+0000 7fe24d7fa700 1 -- 172.21.15.12:0/3865615683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24800f040 con 0x7fe250074f60 2024-05-09T00:10:46.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.175+0000 7fe256857700 1 -- 172.21.15.12:0/3865615683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25019b9e0 con 0x7fe250074f60 2024-05-09T00:10:46.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.175+0000 7fe24d7fa700 1 -- 172.21.15.12:0/3865615683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480057d0 con 0x7fe250074f60 2024-05-09T00:10:46.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.175+0000 7fe256857700 1 -- 172.21.15.12:0/3865615683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25019bea0 con 0x7fe250074f60 2024-05-09T00:10:46.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.178+0000 7fe24d7fa700 1 -- 172.21.15.12:0/3865615683 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe248005930 con 0x7fe250074f60 2024-05-09T00:10:46.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.178+0000 7fe24d7fa700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2380a1290 0x7fe2380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.178+0000 7fe24ffff700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2380a1290 0x7fe2380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.178+0000 7fe24d7fa700 1 -- 172.21.15.12:0/3865615683 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe2480cc490 con 0x7fe250074f60 2024-05-09T00:10:46.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.179+0000 7fe24ffff700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2380a1290 0x7fe2380a3750 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe24000ba10 tx=0x7fe24000b3f0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.179+0000 7fe262a11b80 1 Processor -- start 2024-05-09T00:10:46.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.179+0000 7fe262a11b80 1 -- start start 2024-05-09T00:10:46.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.179+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.179+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 0x55ab2aa5f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe262a11b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2a9b6150 con 0x55ab2aa65720 2024-05-09T00:10:46.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe262a11b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2a9b3b10 con 0x55ab2aa64e00 2024-05-09T00:10:46.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe262a11b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2a9af400 con 0x55ab2aa5f890 2024-05-09T00:10:46.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe254df4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe254df4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47460/0 (socket says 172.21.15.12:47460) 2024-05-09T00:10:46.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe254df4700 1 -- 172.21.15.12:0/2890248150 learned_addr learned my addr 172.21.15.12:0/2890248150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe24ffff700 1 --2- 172.21.15.12:0/2890248150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe24f7fe700 1 --2- 172.21.15.12:0/2890248150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 0x55ab2aa5f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe254df4700 1 -- 172.21.15.12:0/2890248150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 msgr2=0x55ab2aa651e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe254df4700 1 --2- 172.21.15.12:0/2890248150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa651e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe254df4700 1 -- 172.21.15.12:0/2890248150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 msgr2=0x55ab2aa5f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe254df4700 1 --2- 172.21.15.12:0/2890248150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 0x55ab2aa5f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe254df4700 1 -- 172.21.15.12:0/2890248150 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab2a8303b0 con 0x55ab2aa5f890 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe24f7fe700 1 --2- 172.21.15.12:0/2890248150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 0x55ab2aa5f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.180+0000 7fe254df4700 1 --2- 172.21.15.12:0/2890248150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5fcd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe2480a1530 tx=0x7fe248004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f327274554ed46 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.181+0000 7fe2367fc700 1 -- 172.21.15.12:0/2890248150 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248099ce0 con 0x55ab2aa5f890 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.181+0000 7fe2367fc700 1 -- 172.21.15.12:0/2890248150 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2480995e0 con 0x55ab2aa5f890 2024-05-09T00:10:46.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.181+0000 7fe2367fc700 1 -- 172.21.15.12:0/2890248150 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24800dd60 con 0x55ab2aa5f890 2024-05-09T00:10:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.181+0000 7fe262a11b80 1 -- 172.21.15.12:0/2890248150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 msgr2=0x55ab2aa5fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.181+0000 7fe262a11b80 1 --2- 172.21.15.12:0/2890248150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5fcd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe2480a1530 tx=0x7fe248004b50 comp rx=0 tx=0).stop 2024-05-09T00:10:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.182+0000 7fe262a11b80 1 -- 172.21.15.12:0/2890248150 shutdown_connections 2024-05-09T00:10:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.182+0000 7fe262a11b80 1 --2- 172.21.15.12:0/2890248150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5fcd0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.182+0000 7fe262a11b80 1 --2- 172.21.15.12:0/2890248150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa651e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.182+0000 7fe262a11b80 1 --2- 172.21.15.12:0/2890248150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 0x55ab2aa5f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.182+0000 7fe262a11b80 1 -- 172.21.15.12:0/2890248150 >> 172.21.15.12:0/2890248150 conn(0x55ab2a9bd210 msgr2=0x55ab2aa59df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.182+0000 7fe262a11b80 1 -- 172.21.15.12:0/2890248150 shutdown_connections 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.182+0000 7fe262a11b80 1 -- 172.21.15.12:0/2890248150 wait complete. 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.182+0000 7fe262a11b80 1 -- start start 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.182+0000 7fe262a11b80 1 Processor -- start 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa68ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa68ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 0x55ab2aa69530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe262a11b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2a94b560 con 0x55ab2aa65720 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe262a11b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2a94b200 con 0x55ab2aa64e00 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe262a11b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2a8326a0 con 0x55ab2aa5f890 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe262a11b80 10 client.0 _ll_register_callbacks cb 0x55ab2a954b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe24ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa68ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe24ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa68ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47462/0 (socket says 172.21.15.12:47462) 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe24ffff700 1 -- 172.21.15.12:0/580398030 learned_addr learned my addr 172.21.15.12:0/580398030 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe254df4700 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 0x55ab2aa69530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe24f7fe700 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa68ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe24ffff700 1 -- 172.21.15.12:0/580398030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 msgr2=0x55ab2aa68ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe24ffff700 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa68ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe24ffff700 1 -- 172.21.15.12:0/580398030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 msgr2=0x55ab2aa69530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe24ffff700 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 0x55ab2aa69530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.183+0000 7fe24ffff700 1 -- 172.21.15.12:0/580398030 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25019bea0 con 0x55ab2aa5f890 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.184+0000 7fe24ffff700 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa68ab0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe24000dee0 tx=0x7fe2400047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8387e7ee90fa68e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.184+0000 7fe22f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.184+0000 7fe22f7fe700 1 -- 172.21.15.12:0/580398030 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24000d8a0 con 0x55ab2aa5f890 2024-05-09T00:10:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.184+0000 7fe22f7fe700 1 -- 172.21.15.12:0/580398030 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe240010dd0 con 0x55ab2aa5f890 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.184+0000 7fe262a11b80 1 -- 172.21.15.12:0/580398030 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab2a908270 con 0x55ab2aa5f890 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.184+0000 7fe22f7fe700 1 -- 172.21.15.12:0/580398030 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe240026400 con 0x55ab2aa5f890 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.184+0000 7fe262a11b80 10 client.31687 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.184+0000 7fe262a11b80 1 -- 172.21.15.12:0/580398030 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55ab2a7dcb60 con 0x55ab2aa5f890 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.184+0000 7fe22f7fe700 1 -- 172.21.15.12:0/580398030 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab2a908270 con 0x55ab2aa5f890 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.185+0000 7fe22f7fe700 1 -- 172.21.15.12:0/580398030 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe240024020 con 0x55ab2aa5f890 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.185+0000 7fe262a11b80 10 client.31687 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.185+0000 7fe262a11b80 10 client.31687 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.185+0000 7fe262a11b80 20 client.31687 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.185+0000 7fe262a11b80 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ab2aa79200 0x55ab2aa7b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.185+0000 7fe262a11b80 4 client.31687 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.185+0000 7fe262a11b80 1 -- 172.21.15.12:0/580398030 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab2aa2e810 con 0x55ab2aa79200 2024-05-09T00:10:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.185+0000 7fe24f7fe700 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ab2aa79200 0x55ab2aa7b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.185+0000 7fe22f7fe700 10 client.31687 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.185+0000 7fe24f7fe700 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ab2aa79200 0x55ab2aa7b620 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa19e1ff0a24a363 server_cookie=b04fc1864718866c in_seq=0 out_seq=0 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe22f7fe700 1 -- 172.21.15.12:0/580398030 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab2aa2e810 con 0x55ab2aa79200 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe22f7fe700 10 client.31687 handle_command_reply: tid=0 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 client.31687 shutdown 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 -- 172.21.15.12:0/580398030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 msgr2=0x55ab2aa68ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa68ab0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe24000dee0 tx=0x7fe2400047c0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 -- 172.21.15.12:0/580398030 shutdown_connections 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa65720 0x55ab2aa69530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ab2aa79200 0x55ab2aa7b620 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa5f890 0x55ab2aa68ab0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 --2- 172.21.15.12:0/580398030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa68ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe22f7fe700 0 client.31687 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:10:46.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 -- 172.21.15.12:0/580398030 >> 172.21.15.12:0/580398030 conn(0x55ab2a9bd210 msgr2=0x55ab2aa595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 -- 172.21.15.12:0/580398030 shutdown_connections 2024-05-09T00:10:46.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.186+0000 7fe262a11b80 1 -- 172.21.15.12:0/580398030 wait complete. 2024-05-09T00:10:46.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.187+0000 7fe262a11b80 20 client.31687 trim_cache size 0 max 16384 2024-05-09T00:10:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe262a11b80 1 Processor -- start 2024-05-09T00:10:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe262a11b80 1 -- start start 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa64e00 0x55ab2aa74bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 0x55ab2aa59d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe262a11b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2aa56dc0 con 0x55ab2aa64e00 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe262a11b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2aa807f0 con 0x55ab2aa5f890 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe262a11b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2aa76900 con 0x55ab2aa65720 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe24f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa64e00 0x55ab2aa74bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe24f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa64e00 0x55ab2aa74bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57116/0 (socket says 172.21.15.12:57116) 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe24f7fe700 1 -- 172.21.15.12:0/1988313522 learned_addr learned my addr 172.21.15.12:0/1988313522 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe254df4700 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 0x55ab2aa59d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.203+0000 7fe24ffff700 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe254df4700 1 -- 172.21.15.12:0/1988313522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa5f890 msgr2=0x55ab2aa5f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe254df4700 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe254df4700 1 -- 172.21.15.12:0/1988313522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa64e00 msgr2=0x55ab2aa74bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe254df4700 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa64e00 0x55ab2aa74bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe254df4700 1 -- 172.21.15.12:0/1988313522 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab2a975560 con 0x55ab2aa65720 2024-05-09T00:10:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe24f7fe700 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa64e00 0x55ab2aa74bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe24ffff700 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe254df4700 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 0x55ab2aa59d20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe248099570 tx=0x7fe248004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d45712dff0a31c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe236ffd700 1 -- 172.21.15.12:0/1988313522 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248099ce0 con 0x55ab2aa65720 2024-05-09T00:10:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe236ffd700 1 -- 172.21.15.12:0/1988313522 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2480b1070 con 0x55ab2aa65720 2024-05-09T00:10:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.204+0000 7fe236ffd700 1 -- 172.21.15.12:0/1988313522 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248095cf0 con 0x55ab2aa65720 2024-05-09T00:10:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.205+0000 7fe262a11b80 1 -- 172.21.15.12:0/1988313522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 msgr2=0x55ab2aa59d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.205+0000 7fe262a11b80 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 0x55ab2aa59d20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe248099570 tx=0x7fe248004d50 comp rx=0 tx=0).stop 2024-05-09T00:10:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.205+0000 7fe262a11b80 1 -- 172.21.15.12:0/1988313522 shutdown_connections 2024-05-09T00:10:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.205+0000 7fe262a11b80 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 0x55ab2aa59d20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.205+0000 7fe262a11b80 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa5f890 0x55ab2aa5f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.205+0000 7fe262a11b80 1 --2- 172.21.15.12:0/1988313522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa64e00 0x55ab2aa74bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.205+0000 7fe262a11b80 1 -- 172.21.15.12:0/1988313522 >> 172.21.15.12:0/1988313522 conn(0x55ab2a9bd210 msgr2=0x55ab2aa673a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.205+0000 7fe262a11b80 1 -- 172.21.15.12:0/1988313522 shutdown_connections 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.205+0000 7fe262a11b80 1 -- 172.21.15.12:0/1988313522 wait complete. 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.206+0000 7fe262a11b80 1 -- start start 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.206+0000 7fe262a11b80 1 Processor -- start 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.206+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa5f890 0x55ab2aa6c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.206+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa6c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.206+0000 7fe262a11b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 0x55ab2aace6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.206+0000 7fe262a11b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2aa6cf30 con 0x55ab2aa5f890 2024-05-09T00:10:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.206+0000 7fe262a11b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2aacebf0 con 0x55ab2aa64e00 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.206+0000 7fe262a11b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2aaced30 con 0x55ab2aa65720 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.206+0000 7fe262a11b80 10 client.0 _ll_register_callbacks cb 0x55ab2a954b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe24f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa6c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe24ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa5f890 0x55ab2aa6c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe24ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa5f890 0x55ab2aa6c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57120/0 (socket says 172.21.15.12:57120) 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe24ffff700 1 -- 172.21.15.12:0/2322442268 learned_addr learned my addr 172.21.15.12:0/2322442268 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe254df4700 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 0x55ab2aace6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe254df4700 1 -- 172.21.15.12:0/2322442268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 msgr2=0x55ab2aa6c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe254df4700 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa6c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe254df4700 1 -- 172.21.15.12:0/2322442268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa5f890 msgr2=0x55ab2aa6c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe254df4700 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa5f890 0x55ab2aa6c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe254df4700 1 -- 172.21.15.12:0/2322442268 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab2a7dcb60 con 0x55ab2aa65720 2024-05-09T00:10:46.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe24ffff700 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa5f890 0x55ab2aa6c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe254df4700 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 0x55ab2aace6b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe248099570 tx=0x7fe248009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c9b77e673a1b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe22f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.207+0000 7fe22f7fe700 1 -- 172.21.15.12:0/2322442268 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24809d410 con 0x55ab2aa65720 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe22f7fe700 1 -- 172.21.15.12:0/2322442268 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe2480b3070 con 0x55ab2aa65720 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe262a11b80 1 -- 172.21.15.12:0/2322442268 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2480995e0 con 0x55ab2aa65720 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe22f7fe700 1 -- 172.21.15.12:0/2322442268 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2480a1a60 con 0x55ab2aa65720 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe262a11b80 10 client.31693 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe262a11b80 1 -- 172.21.15.12:0/2322442268 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7fe25010e670 con 0x55ab2aa65720 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe22f7fe700 1 -- 172.21.15.12:0/2322442268 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2480a5080 con 0x55ab2aa65720 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe22f7fe700 1 -- 172.21.15.12:0/2322442268 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe248099840 con 0x55ab2aa65720 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe262a11b80 10 client.31693 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:46.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe262a11b80 10 client.31693 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:10:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe262a11b80 20 client.31693 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe262a11b80 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ab2aad0040 0x55ab2aad2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe262a11b80 4 client.31693 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:10:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe262a11b80 1 -- 172.21.15.12:0/2322442268 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab2a9a7ec0 con 0x55ab2aad0040 2024-05-09T00:10:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.208+0000 7fe24ffff700 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ab2aad0040 0x55ab2aad2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.209+0000 7fe22f7fe700 10 client.31693 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:10:46.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.209+0000 7fe24ffff700 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ab2aad0040 0x55ab2aad2430 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f03656b93e198502 server_cookie=58de6da9826ad140 in_seq=0 out_seq=0 2024-05-09T00:10:46.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe22f7fe700 1 -- 172.21.15.12:0/2322442268 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab2a9a7ec0 con 0x55ab2aad0040 2024-05-09T00:10:46.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe22f7fe700 10 client.31693 handle_command_reply: tid=0 2024-05-09T00:10:46.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe262a11b80 1 client.31693 shutdown 2024-05-09T00:10:46.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe262a11b80 1 -- 172.21.15.12:0/2322442268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 msgr2=0x55ab2aace6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe262a11b80 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 0x55ab2aace6b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe248099570 tx=0x7fe248009390 comp rx=0 tx=0).stop 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe262a11b80 1 -- 172.21.15.12:0/2322442268 shutdown_connections 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe262a11b80 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55ab2aad0040 0x55ab2aad2430 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe262a11b80 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab2aa65720 0x55ab2aace6b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe262a11b80 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab2aa5f890 0x55ab2aa6c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe262a11b80 1 --2- 172.21.15.12:0/2322442268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ab2aa64e00 0x55ab2aa6c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe22f7fe700 0 client.31693 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.211+0000 7fe262a11b80 1 -- 172.21.15.12:0/2322442268 >> 172.21.15.12:0/2322442268 conn(0x55ab2a9bd210 msgr2=0x55ab2aa63ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.212+0000 7fe262a11b80 1 -- 172.21.15.12:0/2322442268 shutdown_connections 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.212+0000 7fe262a11b80 1 -- 172.21.15.12:0/2322442268 wait complete. 2024-05-09T00:10:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.212+0000 7fe262a11b80 20 client.31693 trim_cache size 0 max 16384 2024-05-09T00:10:46.214 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:10:46.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 -- 172.21.15.12:0/3865615683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2380a1290 msgr2=0x7fe2380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2380a1290 0x7fe2380a3750 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe24000ba10 tx=0x7fe24000b3f0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 -- 172.21.15.12:0/3865615683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250074f60 msgr2=0x7fe250073d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250074f60 0x7fe250073d60 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fe248007ee0 tx=0x7fe24800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 -- 172.21.15.12:0/3865615683 shutdown_connections 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe2380a1290 0x7fe2380a3750 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe250074f60 0x7fe250073d60 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe250101160 0x7fe2500732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 --2- 172.21.15.12:0/3865615683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe250106280 0x7fe2500737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 -- 172.21.15.12:0/3865615683 >> 172.21.15.12:0/3865615683 conn(0x7fe2500fc9c0 msgr2=0x7fe2500fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 -- 172.21.15.12:0/3865615683 shutdown_connections 2024-05-09T00:10:46.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.213+0000 7fe256857700 1 -- 172.21.15.12:0/3865615683 wait complete. 2024-05-09T00:10:46.228 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:10:46.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:45 smithi204 ceph-mon[47373]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.1 MiB/s wr, 450 op/s 2024-05-09T00:10:46.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:45 smithi160 ceph-mon[46152]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.1 MiB/s wr, 450 op/s 2024-05-09T00:10:46.313 INFO:tasks.workunit.client.0.smithi012.stdout: 10 362270 28.48 MB/sec execute 521 sec latency 92.455 ms 2024-05-09T00:10:46.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.406+0000 7f6e572f7700 1 -- 172.21.15.12:0/4103067485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50105160 msgr2=0x7f6e5010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.406+0000 7f6e572f7700 1 --2- 172.21.15.12:0/4103067485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50105160 0x7f6e5010cce0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800b0a0 tx=0x7f6e4800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.406+0000 7f6e572f7700 1 -- 172.21.15.12:0/4103067485 shutdown_connections 2024-05-09T00:10:46.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.406+0000 7f6e572f7700 1 --2- 172.21.15.12:0/4103067485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50105160 0x7f6e5010cce0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.406+0000 7f6e572f7700 1 --2- 172.21.15.12:0/4103067485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e50100060 0x7f6e50100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.406+0000 7f6e572f7700 1 --2- 172.21.15.12:0/4103067485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50100a10 0x7f6e50104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.406+0000 7f6e572f7700 1 -- 172.21.15.12:0/4103067485 >> 172.21.15.12:0/4103067485 conn(0x7f6e50076f40 msgr2=0x7f6e50077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.406+0000 7f6e572f7700 1 -- 172.21.15.12:0/4103067485 shutdown_connections 2024-05-09T00:10:46.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.407+0000 7f6e572f7700 1 -- 172.21.15.12:0/4103067485 wait complete. 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.407+0000 7f6e572f7700 1 Processor -- start 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.407+0000 7f6e572f7700 1 -- start start 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e572f7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e50100060 0x7f6e50199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e572f7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50105160 0x7f6e50199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e572f7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e5019cfa0 0x7f6e5019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e572f7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5019d990 con 0x7f6e50105160 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e572f7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5019db00 con 0x7f6e5019cfa0 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e572f7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5019dcc0 con 0x7f6e50100060 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e54892700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50105160 0x7f6e50199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e55093700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e50100060 0x7f6e50199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e54892700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50105160 0x7f6e50199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57144/0 (socket says 172.21.15.12:57144) 2024-05-09T00:10:46.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e54892700 1 -- 172.21.15.12:0/4083879336 learned_addr learned my addr 172.21.15.12:0/4083879336 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e55894700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e5019cfa0 0x7f6e5019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e55093700 1 -- 172.21.15.12:0/4083879336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e5019cfa0 msgr2=0x7f6e5019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e55093700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e5019cfa0 0x7f6e5019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e55093700 1 -- 172.21.15.12:0/4083879336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50105160 msgr2=0x7f6e50199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e55093700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50105160 0x7f6e50199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e55093700 1 -- 172.21.15.12:0/4083879336 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48009d00 con 0x7f6e50100060 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.408+0000 7f6e55093700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e50100060 0x7f6e50199770 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0097d0 tx=0x7f6e4c009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.409+0000 7f6e467fc700 1 -- 172.21.15.12:0/4083879336 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c010040 con 0x7f6e50100060 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.409+0000 7f6e467fc700 1 -- 172.21.15.12:0/4083879336 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4c013c50 con 0x7f6e50100060 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.409+0000 7f6e467fc700 1 -- 172.21.15.12:0/4083879336 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c014e20 con 0x7f6e50100060 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.409+0000 7f6e572f7700 1 -- 172.21.15.12:0/4083879336 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e501a6330 con 0x7f6e50100060 2024-05-09T00:10:46.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.409+0000 7f6e572f7700 1 -- 172.21.15.12:0/4083879336 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e501a6840 con 0x7f6e50100060 2024-05-09T00:10:46.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.411+0000 7f6e467fc700 1 -- 172.21.15.12:0/4083879336 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6e4c013790 con 0x7f6e50100060 2024-05-09T00:10:46.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.412+0000 7f6e467fc700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e3c0a1340 0x7f6e3c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.412+0000 7f6e467fc700 1 -- 172.21.15.12:0/4083879336 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6e4c0cc740 con 0x7f6e50100060 2024-05-09T00:10:46.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.412+0000 7f6e54892700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e3c0a1340 0x7f6e3c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.413+0000 7f6e54892700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e3c0a1340 0x7f6e3c0a3800 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6e5019acc0 tx=0x7f6e40009240 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.413+0000 7f6e634b1b80 1 Processor -- start 2024-05-09T00:10:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.413+0000 7f6e634b1b80 1 -- start start 2024-05-09T00:10:46.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e7405721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e74056c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 0x55e74056ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7404c3150 con 0x55e740571e00 2024-05-09T00:10:46.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7404c0b10 con 0x55e74056c890 2024-05-09T00:10:46.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7404bc400 con 0x55e740572720 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e55093700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e7405721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e55093700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e7405721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57154/0 (socket says 172.21.15.12:57154) 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e55093700 1 -- 172.21.15.12:0/449373826 learned_addr learned my addr 172.21.15.12:0/449373826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e55894700 1 --2- 172.21.15.12:0/449373826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 0x55e74056ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e54892700 1 --2- 172.21.15.12:0/449373826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e74056c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e55093700 1 -- 172.21.15.12:0/449373826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 msgr2=0x55e74056c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e55093700 1 --2- 172.21.15.12:0/449373826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e74056c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e55093700 1 -- 172.21.15.12:0/449373826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 msgr2=0x55e74056ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e55093700 1 --2- 172.21.15.12:0/449373826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 0x55e74056ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e55093700 1 -- 172.21.15.12:0/449373826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e74033d3b0 con 0x55e740571e00 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.414+0000 7f6e55093700 1 --2- 172.21.15.12:0/449373826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e7405721e0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0a17e0 tx=0x7f6e4c099450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d8833687bb2743 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.415+0000 7f6e3b7fe700 1 -- 172.21.15.12:0/449373826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c099f90 con 0x55e740571e00 2024-05-09T00:10:46.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.415+0000 7f6e3b7fe700 1 -- 172.21.15.12:0/449373826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4c099890 con 0x55e740571e00 2024-05-09T00:10:46.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.415+0000 7f6e3b7fe700 1 -- 172.21.15.12:0/449373826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c099af0 con 0x55e740571e00 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.415+0000 7f6e634b1b80 1 -- 172.21.15.12:0/449373826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 msgr2=0x55e7405721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.415+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/449373826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e7405721e0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0a17e0 tx=0x7f6e4c099450 comp rx=0 tx=0).stop 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.416+0000 7f6e634b1b80 1 -- 172.21.15.12:0/449373826 shutdown_connections 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.416+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/449373826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 0x55e74056ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.416+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/449373826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e7405721e0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.416+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/449373826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e74056c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.416+0000 7f6e634b1b80 1 -- 172.21.15.12:0/449373826 >> 172.21.15.12:0/449373826 conn(0x55e7404ca210 msgr2=0x55e740566df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.416+0000 7f6e634b1b80 1 -- 172.21.15.12:0/449373826 shutdown_connections 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.416+0000 7f6e634b1b80 1 -- 172.21.15.12:0/449373826 wait complete. 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.416+0000 7f6e634b1b80 1 -- start start 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.416+0000 7f6e634b1b80 1 Processor -- start 2024-05-09T00:10:46.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e74056c890 0x55e740575ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740576010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e740576550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e740458560 con 0x55e74056c890 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e740458200 con 0x55e740571e00 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74033f6a0 con 0x55e740572720 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e634b1b80 10 client.0 _ll_register_callbacks cb 0x55e740461b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e54892700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740576010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e55894700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e740576550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e54892700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740576010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47300/0 (socket says 172.21.15.12:47300) 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e54892700 1 -- 172.21.15.12:0/1675674846 learned_addr learned my addr 172.21.15.12:0/1675674846 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e55093700 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e74056c890 0x55e740575ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e54892700 1 -- 172.21.15.12:0/1675674846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 msgr2=0x55e740576550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e54892700 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e740576550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e54892700 1 -- 172.21.15.12:0/1675674846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e74056c890 msgr2=0x55e740575ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e54892700 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e74056c890 0x55e740575ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.417+0000 7f6e54892700 1 -- 172.21.15.12:0/1675674846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e501a6840 con 0x55e740571e00 2024-05-09T00:10:46.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.418+0000 7f6e55093700 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e74056c890 0x55e740575ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:46.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.418+0000 7f6e54892700 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740576010 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6e400154d0 tx=0x7f6e40004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a13f7ed01a4b061 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.418+0000 7f6e38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:10:46.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.418+0000 7f6e38ff9700 1 -- 172.21.15.12:0/1675674846 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40015c50 con 0x55e740571e00 2024-05-09T00:10:46.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.418+0000 7f6e634b1b80 1 -- 172.21.15.12:0/1675674846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e740415270 con 0x55e740571e00 2024-05-09T00:10:46.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.418+0000 7f6e38ff9700 1 -- 172.21.15.12:0/1675674846 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6e40015db0 con 0x55e740571e00 2024-05-09T00:10:46.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.423+0000 7f6e38ff9700 1 -- 172.21.15.12:0/1675674846 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e40005a50 con 0x55e740571e00 2024-05-09T00:10:46.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.423+0000 7f6e634b1b80 10 client.31424 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:46.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.423+0000 7f6e634b1b80 1 -- 172.21.15.12:0/1675674846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55e7402e9b60 con 0x55e740571e00 2024-05-09T00:10:46.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.424+0000 7f6e38ff9700 1 -- 172.21.15.12:0/1675674846 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e740415270 con 0x55e740571e00 2024-05-09T00:10:46.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.424+0000 7f6e38ff9700 1 -- 172.21.15.12:0/1675674846 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f6e40021440 con 0x55e740571e00 2024-05-09T00:10:46.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.424+0000 7f6e634b1b80 10 client.31424 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:46.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.424+0000 7f6e634b1b80 10 client.31424 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:10:46.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.424+0000 7f6e634b1b80 20 client.31424 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:46.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.424+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e7405862d0 0x55e7405886f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.424+0000 7f6e634b1b80 4 client.31424 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:46.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.424+0000 7f6e634b1b80 1 -- 172.21.15.12:0/1675674846 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e74053b810 con 0x55e7405862d0 2024-05-09T00:10:46.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.424+0000 7f6e55093700 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e7405862d0 0x55e7405886f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.425+0000 7f6e38ff9700 10 client.31424 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:10:46.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.425+0000 7f6e55093700 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e7405862d0 0x55e7405886f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc85c5b872db5f66 server_cookie=843aed1e606137d9 in_seq=0 out_seq=0 2024-05-09T00:10:46.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e38ff9700 1 -- 172.21.15.12:0/1675674846 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e74053b810 con 0x55e7405862d0 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e38ff9700 10 client.31424 handle_command_reply: tid=0 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 client.31424 shutdown 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 -- 172.21.15.12:0/1675674846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 msgr2=0x55e740576010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740576010 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6e400154d0 tx=0x7f6e40004860 comp rx=0 tx=0).stop 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 -- 172.21.15.12:0/1675674846 shutdown_connections 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e7405862d0 0x55e7405886f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e740576550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e38ff9700 0 client.31424 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e74056c890 0x55e740575ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/1675674846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740576010 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 -- 172.21.15.12:0/1675674846 >> 172.21.15.12:0/1675674846 conn(0x55e7404ca210 msgr2=0x55e7405665a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 -- 172.21.15.12:0/1675674846 shutdown_connections 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.426+0000 7f6e634b1b80 1 -- 172.21.15.12:0/1675674846 wait complete. 2024-05-09T00:10:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.427+0000 7f6e634b1b80 20 client.31424 trim_cache size 0 max 16384 2024-05-09T00:10:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.442+0000 7f6e634b1b80 1 Processor -- start 2024-05-09T00:10:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.442+0000 7f6e634b1b80 1 -- start start 2024-05-09T00:10:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 0x55e74056c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e740581c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e740566d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74055f0b0 con 0x55e740571e00 2024-05-09T00:10:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e740568070 con 0x55e74056c890 2024-05-09T00:10:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e740563dc0 con 0x55e740572720 2024-05-09T00:10:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e54892700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e740581c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e55093700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 0x55e74056c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e54892700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e740581c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57174/0 (socket says 172.21.15.12:57174) 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e55093700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 0x55e74056c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47302/0 (socket says 172.21.15.12:47302) 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e54892700 1 -- 172.21.15.12:0/798248698 learned_addr learned my addr 172.21.15.12:0/798248698 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e55894700 1 --2- 172.21.15.12:0/798248698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e740566d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e55093700 1 -- 172.21.15.12:0/798248698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 msgr2=0x55e740566d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e55093700 1 --2- 172.21.15.12:0/798248698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e740566d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e55093700 1 -- 172.21.15.12:0/798248698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 msgr2=0x55e740581c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e55093700 1 --2- 172.21.15.12:0/798248698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e740581c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e55093700 1 -- 172.21.15.12:0/798248698 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e740482560 con 0x55e74056c890 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e54892700 1 --2- 172.21.15.12:0/798248698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e740581c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.443+0000 7f6e55093700 1 --2- 172.21.15.12:0/798248698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 0x55e74056c350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0a1ed0 tx=0x7f6e4c014880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7547886a817daed4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.444+0000 7f6e3bfff700 1 -- 172.21.15.12:0/798248698 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0b2280 con 0x55e74056c890 2024-05-09T00:10:46.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.444+0000 7f6e3bfff700 1 -- 172.21.15.12:0/798248698 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4c0a1d10 con 0x55e74056c890 2024-05-09T00:10:46.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.444+0000 7f6e3bfff700 1 -- 172.21.15.12:0/798248698 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c098fe0 con 0x55e74056c890 2024-05-09T00:10:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.444+0000 7f6e634b1b80 1 -- 172.21.15.12:0/798248698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 msgr2=0x55e74056c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.444+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/798248698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 0x55e74056c350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0a1ed0 tx=0x7f6e4c014880 comp rx=0 tx=0).stop 2024-05-09T00:10:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.444+0000 7f6e634b1b80 1 -- 172.21.15.12:0/798248698 shutdown_connections 2024-05-09T00:10:46.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.445+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/798248698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e740572720 0x55e740566d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.445+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/798248698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e74056c890 0x55e74056c350 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.445+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/798248698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740571e00 0x55e740581c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.445+0000 7f6e634b1b80 1 -- 172.21.15.12:0/798248698 >> 172.21.15.12:0/798248698 conn(0x55e7404ca210 msgr2=0x55e7405743a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.445+0000 7f6e634b1b80 1 -- 172.21.15.12:0/798248698 shutdown_connections 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.445+0000 7f6e634b1b80 1 -- 172.21.15.12:0/798248698 wait complete. 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.445+0000 7f6e634b1b80 1 -- start start 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.445+0000 7f6e634b1b80 1 Processor -- start 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e74056c890 0x55e740579140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740579680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e634b1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740572720 0x55e7405db7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e740579f60 con 0x55e740572720 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7405dbd00 con 0x55e740571e00 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e634b1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7405dbe40 con 0x55e74056c890 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e634b1b80 10 client.0 _ll_register_callbacks cb 0x55e740461b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:46.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e55894700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740572720 0x55e7405db7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e54892700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740579680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e55894700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740572720 0x55e7405db7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57186/0 (socket says 172.21.15.12:57186) 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e55894700 1 -- 172.21.15.12:0/2957599972 learned_addr learned my addr 172.21.15.12:0/2957599972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e54892700 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740579680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47310/0 (socket says 172.21.15.12:47310) 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e55894700 1 -- 172.21.15.12:0/2957599972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e74056c890 msgr2=0x55e740579140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e55894700 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e74056c890 0x55e740579140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e55894700 1 -- 172.21.15.12:0/2957599972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 msgr2=0x55e740579680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e55894700 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740579680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e55894700 1 -- 172.21.15.12:0/2957599972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7402e9b60 con 0x55e740572720 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.446+0000 7f6e54892700 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740579680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e55894700 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740572720 0x55e7405db7c0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800d010 tx=0x7f6e4800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ebaf59fede1e36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e38ff9700 1 -- 172.21.15.12:0/2957599972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48022070 con 0x55e740572720 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e38ff9700 1 -- 172.21.15.12:0/2957599972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6e480077b0 con 0x55e740572720 2024-05-09T00:10:46.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e634b1b80 1 -- 172.21.15.12:0/2957599972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e4c099890 con 0x55e740572720 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e38ff9700 1 -- 172.21.15.12:0/2957599972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e4800e040 con 0x55e740572720 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e634b1b80 10 client.22053 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e634b1b80 1 -- 172.21.15.12:0/2957599972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55e74033d3b0 con 0x55e740572720 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e38ff9700 1 -- 172.21.15.12:0/2957599972 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4c099890 con 0x55e740572720 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e38ff9700 1 -- 172.21.15.12:0/2957599972 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f6e4800b0d0 con 0x55e740572720 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e634b1b80 10 client.22053 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e634b1b80 10 client.22053 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.447+0000 7f6e634b1b80 20 client.22053 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.448+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e7405dd5c0 0x55e7405df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.448+0000 7f6e634b1b80 4 client.22053 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.448+0000 7f6e634b1b80 1 -- 172.21.15.12:0/2957599972 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e7404b4ec0 con 0x55e7405dd5c0 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.448+0000 7f6e55093700 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e7405dd5c0 0x55e7405df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.448+0000 7f6e38ff9700 10 client.22053 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:10:46.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.448+0000 7f6e55093700 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e7405dd5c0 0x55e7405df9e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37d4d15a0aa64e95 server_cookie=d54b8797ce3b6537 in_seq=0 out_seq=0 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e38ff9700 1 -- 172.21.15.12:0/2957599972 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e7404b4ec0 con 0x55e7405dd5c0 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e38ff9700 10 client.22053 handle_command_reply: tid=0 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 client.22053 shutdown 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 -- 172.21.15.12:0/2957599972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740572720 msgr2=0x55e7405db7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740572720 0x55e7405db7c0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800d010 tx=0x7f6e4800be50 comp rx=0 tx=0).stop 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 -- 172.21.15.12:0/2957599972 shutdown_connections 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55e7405dd5c0 0x55e7405df9e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e740572720 0x55e7405db7c0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e74056c890 0x55e740579140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e38ff9700 0 client.22053 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 --2- 172.21.15.12:0/2957599972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e740571e00 0x55e740579680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 -- 172.21.15.12:0/2957599972 >> 172.21.15.12:0/2957599972 conn(0x55e7404ca210 msgr2=0x55e740575a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 -- 172.21.15.12:0/2957599972 shutdown_connections 2024-05-09T00:10:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 1 -- 172.21.15.12:0/2957599972 wait complete. 2024-05-09T00:10:46.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.452+0000 7f6e634b1b80 20 client.22053 trim_cache size 0 max 16384 2024-05-09T00:10:46.453 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:10:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 -- 172.21.15.12:0/4083879336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e3c0a1340 msgr2=0x7f6e3c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e3c0a1340 0x7f6e3c0a3800 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6e5019acc0 tx=0x7f6e40009240 comp rx=0 tx=0).stop 2024-05-09T00:10:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 -- 172.21.15.12:0/4083879336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e50100060 msgr2=0x7f6e50199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e50100060 0x7f6e50199770 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0097d0 tx=0x7f6e4c009ae0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 -- 172.21.15.12:0/4083879336 shutdown_connections 2024-05-09T00:10:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f6e3c0a1340 0x7f6e3c0a3800 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e5019cfa0 0x7f6e5019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e50100060 0x7f6e50199770 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 --2- 172.21.15.12:0/4083879336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6e50105160 0x7f6e50199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 -- 172.21.15.12:0/4083879336 >> 172.21.15.12:0/4083879336 conn(0x7f6e50076f40 msgr2=0x7f6e500fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 -- 172.21.15.12:0/4083879336 shutdown_connections 2024-05-09T00:10:46.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.453+0000 7f6e572f7700 1 -- 172.21.15.12:0/4083879336 wait complete. 2024-05-09T00:10:46.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:10:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.635+0000 7f897bc7a700 1 -- 172.21.15.12:0/2874830005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8974103d20 msgr2=0x7f8974107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.635+0000 7f897bc7a700 1 --2- 172.21.15.12:0/2874830005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8974103d20 0x7f8974107d60 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f8964009990 tx=0x7f8964009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.636+0000 7f897bc7a700 1 -- 172.21.15.12:0/2874830005 shutdown_connections 2024-05-09T00:10:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.636+0000 7f897bc7a700 1 --2- 172.21.15.12:0/2874830005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8974108490 0x7f8974110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.636+0000 7f897bc7a700 1 --2- 172.21.15.12:0/2874830005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8974103370 0x7f8974103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.636+0000 7f897bc7a700 1 --2- 172.21.15.12:0/2874830005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8974103d20 0x7f8974107d60 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.636+0000 7f897bc7a700 1 -- 172.21.15.12:0/2874830005 >> 172.21.15.12:0/2874830005 conn(0x7f89740febb0 msgr2=0x7f8974100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.636+0000 7f897bc7a700 1 -- 172.21.15.12:0/2874830005 shutdown_connections 2024-05-09T00:10:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.636+0000 7f897bc7a700 1 -- 172.21.15.12:0/2874830005 wait complete. 2024-05-09T00:10:46.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.637+0000 7f897bc7a700 1 Processor -- start 2024-05-09T00:10:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.637+0000 7f897bc7a700 1 -- start start 2024-05-09T00:10:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.637+0000 7f897bc7a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8974103370 0x7f897410e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.637+0000 7f897bc7a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8974108490 0x7f897410e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.637+0000 7f897bc7a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897410ecf0 0x7f89741a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.637+0000 7f897bc7a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89741aa0b0 con 0x7f8974103370 2024-05-09T00:10:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.637+0000 7f897bc7a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897406a9c0 con 0x7f897410ecf0 2024-05-09T00:10:46.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.637+0000 7f897bc7a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897406ab80 con 0x7f8974108490 2024-05-09T00:10:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.638+0000 7f8979215700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8974108490 0x7f897410e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.638+0000 7f8979215700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8974108490 0x7f897410e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47552/0 (socket says 172.21.15.12:47552) 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.638+0000 7f8979215700 1 -- 172.21.15.12:0/1396598933 learned_addr learned my addr 172.21.15.12:0/1396598933 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.638+0000 7f8979215700 1 -- 172.21.15.12:0/1396598933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897410ecf0 msgr2=0x7f89741a9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.638+0000 7f8979215700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897410ecf0 0x7f89741a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.638+0000 7f8979215700 1 -- 172.21.15.12:0/1396598933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8974103370 msgr2=0x7f897410e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.638+0000 7f8979215700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8974103370 0x7f897410e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.638+0000 7f8979215700 1 -- 172.21.15.12:0/1396598933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8970009520 con 0x7f8974108490 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.638+0000 7f8979215700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8974108490 0x7f897410e7b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8964000c00 tx=0x7f8964009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.639+0000 7f896affd700 1 -- 172.21.15.12:0/1396598933 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896401d070 con 0x7f8974108490 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.639+0000 7f896affd700 1 -- 172.21.15.12:0/1396598933 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8964021410 con 0x7f8974108490 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.639+0000 7f896affd700 1 -- 172.21.15.12:0/1396598933 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964017650 con 0x7f8974108490 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.642+0000 7f897bc7a700 1 -- 172.21.15.12:0/1396598933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89640095f0 con 0x7f8974108490 2024-05-09T00:10:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.642+0000 7f897bc7a700 1 -- 172.21.15.12:0/1396598933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897406afe0 con 0x7f8974108490 2024-05-09T00:10:46.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.645+0000 7f896affd700 1 -- 172.21.15.12:0/1396598933 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f896400fb00 con 0x7f8974108490 2024-05-09T00:10:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.646+0000 7f896affd700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89600a1310 0x7f89600a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.646+0000 7f896affd700 1 -- 172.21.15.12:0/1396598933 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f89640cda90 con 0x7f8974108490 2024-05-09T00:10:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.647+0000 7f8979a16700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89600a1310 0x7f89600a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.648+0000 7f8979a16700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89600a1310 0x7f89600a37d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8970005d90 tx=0x7f8970005ce0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.649+0000 7f8987e34b80 1 Processor -- start 2024-05-09T00:10:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8987e34b80 1 -- start start 2024-05-09T00:10:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 0x562dae2651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae25f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae25fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8987e34b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae1b6150 con 0x562dae25f890 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8987e34b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae1b3b10 con 0x562dae265720 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8987e34b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae1af400 con 0x562dae264e00 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8979a16700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 0x562dae2651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f897a217700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae25fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8979a16700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 0x562dae2651e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47564/0 (socket says 172.21.15.12:47564) 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8979a16700 1 -- 172.21.15.12:0/1204894684 learned_addr learned my addr 172.21.15.12:0/1204894684 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.650+0000 7f8979215700 1 --2- 172.21.15.12:0/1204894684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae25f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.651+0000 7f8979a16700 1 -- 172.21.15.12:0/1204894684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 msgr2=0x562dae25f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.651+0000 7f8979a16700 1 --2- 172.21.15.12:0/1204894684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae25f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.651+0000 7f8979a16700 1 -- 172.21.15.12:0/1204894684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 msgr2=0x562dae25fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.651+0000 7f8979a16700 1 --2- 172.21.15.12:0/1204894684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae25fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.651+0000 7f8979a16700 1 -- 172.21.15.12:0/1204894684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dae0303b0 con 0x562dae264e00 2024-05-09T00:10:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.651+0000 7f897a217700 1 --2- 172.21.15.12:0/1204894684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae25fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.651+0000 7f8979a16700 1 --2- 172.21.15.12:0/1204894684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 0x562dae2651e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f897000eea0 tx=0x7f8970011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e5d7b049cf97f3f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.651+0000 7f895ffff700 1 -- 172.21.15.12:0/1204894684 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897000eed0 con 0x562dae264e00 2024-05-09T00:10:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.651+0000 7f895ffff700 1 -- 172.21.15.12:0/1204894684 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8970004cf0 con 0x562dae264e00 2024-05-09T00:10:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.651+0000 7f895ffff700 1 -- 172.21.15.12:0/1204894684 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89700056a0 con 0x562dae264e00 2024-05-09T00:10:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.652+0000 7f8987e34b80 1 -- 172.21.15.12:0/1204894684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 msgr2=0x562dae2651e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.652+0000 7f8987e34b80 1 --2- 172.21.15.12:0/1204894684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 0x562dae2651e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f897000eea0 tx=0x7f8970011000 comp rx=0 tx=0).stop 2024-05-09T00:10:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.652+0000 7f8987e34b80 1 -- 172.21.15.12:0/1204894684 shutdown_connections 2024-05-09T00:10:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.652+0000 7f8987e34b80 1 --2- 172.21.15.12:0/1204894684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae25fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.652+0000 7f8987e34b80 1 --2- 172.21.15.12:0/1204894684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 0x562dae2651e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.652+0000 7f8987e34b80 1 --2- 172.21.15.12:0/1204894684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae25f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.652+0000 7f8987e34b80 1 -- 172.21.15.12:0/1204894684 >> 172.21.15.12:0/1204894684 conn(0x562dae1bd210 msgr2=0x562dae259df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.652+0000 7f8987e34b80 1 -- 172.21.15.12:0/1204894684 shutdown_connections 2024-05-09T00:10:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.652+0000 7f8987e34b80 1 -- 172.21.15.12:0/1204894684 wait complete. 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.653+0000 7f8987e34b80 1 -- start start 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.653+0000 7f8987e34b80 1 Processor -- start 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.653+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae268b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.653+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae264e00 0x562dae269070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.653+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae265720 0x562dae2695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.653+0000 7f8987e34b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae14b560 con 0x562dae265720 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.653+0000 7f8987e34b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae14b200 con 0x562dae264e00 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.653+0000 7f8987e34b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae0326a0 con 0x562dae25f890 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.653+0000 7f8987e34b80 10 client.0 _ll_register_callbacks cb 0x562dae154b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.653+0000 7f8979a16700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae268b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f8979215700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae264e00 0x562dae269070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f8979a16700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae268b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47568/0 (socket says 172.21.15.12:47568) 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f8979215700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae264e00 0x562dae269070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47338/0 (socket says 172.21.15.12:47338) 2024-05-09T00:10:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f8979a16700 1 -- 172.21.15.12:0/2393273566 learned_addr learned my addr 172.21.15.12:0/2393273566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f897a217700 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae265720 0x562dae2695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f8979a16700 1 -- 172.21.15.12:0/2393273566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae264e00 msgr2=0x562dae269070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f8979a16700 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae264e00 0x562dae269070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f8979a16700 1 -- 172.21.15.12:0/2393273566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae265720 msgr2=0x562dae2695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f8979a16700 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae265720 0x562dae2695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f8979a16700 1 -- 172.21.15.12:0/2393273566 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c00e0c0 con 0x562dae25f890 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f897a217700 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae265720 0x562dae2695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f8979a16700 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae268b30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8970009e50 tx=0x7f8970010f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=239be538915cd3a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f895d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.654+0000 7f895d7fa700 1 -- 172.21.15.12:0/2393273566 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897000eed0 con 0x562dae25f890 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.655+0000 7f895d7fa700 1 -- 172.21.15.12:0/2393273566 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f89700052e0 con 0x562dae25f890 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.655+0000 7f8987e34b80 1 -- 172.21.15.12:0/2393273566 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562dae108270 con 0x562dae25f890 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.655+0000 7f895d7fa700 1 -- 172.21.15.12:0/2393273566 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89700054a0 con 0x562dae25f890 2024-05-09T00:10:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.655+0000 7f8987e34b80 10 client.31720 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.655+0000 7f8987e34b80 1 -- 172.21.15.12:0/2393273566 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7f897000eb50 con 0x562dae25f890 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.655+0000 7f895d7fa700 1 -- 172.21.15.12:0/2393273566 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562dae108270 con 0x562dae25f890 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.655+0000 7f895d7fa700 1 -- 172.21.15.12:0/2393273566 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f8970018070 con 0x562dae25f890 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.656+0000 7f8987e34b80 10 client.31720 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.656+0000 7f8987e34b80 10 client.31720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.656+0000 7f8987e34b80 20 client.31720 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.656+0000 7f8987e34b80 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dae2791c0 0x562dae27b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.656+0000 7f8987e34b80 4 client.31720 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.656+0000 7f8987e34b80 1 -- 172.21.15.12:0/2393273566 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562dae22e810 con 0x562dae2791c0 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.656+0000 7f8979215700 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dae2791c0 0x562dae27b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.656+0000 7f895d7fa700 10 client.31720 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.656+0000 7f8979215700 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dae2791c0 0x562dae27b5e0 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c831d3d35dcf91d4 server_cookie=301604b7aac7f82a in_seq=0 out_seq=0 2024-05-09T00:10:46.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f895d7fa700 1 -- 172.21.15.12:0/2393273566 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562dae22e810 con 0x562dae2791c0 2024-05-09T00:10:46.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f895d7fa700 10 client.31720 handle_command_reply: tid=0 2024-05-09T00:10:46.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 client.31720 shutdown 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 -- 172.21.15.12:0/2393273566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 msgr2=0x562dae268b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae268b30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8970009e50 tx=0x7f8970010f60 comp rx=0 tx=0).stop 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 -- 172.21.15.12:0/2393273566 shutdown_connections 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae265720 0x562dae2695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae268b30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dae2791c0 0x562dae27b5e0 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 --2- 172.21.15.12:0/2393273566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae264e00 0x562dae269070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 -- 172.21.15.12:0/2393273566 >> 172.21.15.12:0/2393273566 conn(0x562dae1bd210 msgr2=0x562dae2595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f895d7fa700 0 client.31720 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 -- 172.21.15.12:0/2393273566 shutdown_connections 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.657+0000 7f8987e34b80 1 -- 172.21.15.12:0/2393273566 wait complete. 2024-05-09T00:10:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.658+0000 7f8987e34b80 20 client.31720 trim_cache size 0 max 16384 2024-05-09T00:10:46.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.674+0000 7f8987e34b80 1 Processor -- start 2024-05-09T00:10:46.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.674+0000 7f8987e34b80 1 -- start start 2024-05-09T00:10:46.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae274c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae264e00 0x562dae259d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae269410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8987e34b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae256df0 con 0x562dae264e00 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8987e34b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae266b10 con 0x562dae265720 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8987e34b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae25f700 con 0x562dae25f890 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8979215700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae264e00 0x562dae259d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8979a16700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae274c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8979215700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae264e00 0x562dae259d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57218/0 (socket says 172.21.15.12:57218) 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8979a16700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae274c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47578/0 (socket says 172.21.15.12:47578) 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8979215700 1 -- 172.21.15.12:0/1393821831 learned_addr learned my addr 172.21.15.12:0/1393821831 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f897a217700 1 --2- 172.21.15.12:0/1393821831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae269410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8979215700 1 -- 172.21.15.12:0/1393821831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 msgr2=0x562dae274c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8979215700 1 --2- 172.21.15.12:0/1393821831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae274c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8979215700 1 -- 172.21.15.12:0/1393821831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 msgr2=0x562dae269410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8979215700 1 --2- 172.21.15.12:0/1393821831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae269410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.675+0000 7f8979215700 1 -- 172.21.15.12:0/1393821831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dadfdcb60 con 0x562dae264e00 2024-05-09T00:10:46.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.676+0000 7f8979215700 1 --2- 172.21.15.12:0/1393821831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae264e00 0x562dae259d20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f89640cb2d0 tx=0x7f896401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98390f96337aecb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.676+0000 7f8968da4700 1 -- 172.21.15.12:0/1393821831 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896409ac70 con 0x562dae264e00 2024-05-09T00:10:46.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.676+0000 7f8968da4700 1 -- 172.21.15.12:0/1393821831 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896409eaa0 con 0x562dae264e00 2024-05-09T00:10:46.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.676+0000 7f8968da4700 1 -- 172.21.15.12:0/1393821831 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89640215f0 con 0x562dae264e00 2024-05-09T00:10:46.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.677+0000 7f8987e34b80 1 -- 172.21.15.12:0/1393821831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae264e00 msgr2=0x562dae259d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.677+0000 7f8987e34b80 1 --2- 172.21.15.12:0/1393821831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae264e00 0x562dae259d20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f89640cb2d0 tx=0x7f896401a040 comp rx=0 tx=0).stop 2024-05-09T00:10:46.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.677+0000 7f8987e34b80 1 -- 172.21.15.12:0/1393821831 shutdown_connections 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.677+0000 7f8987e34b80 1 --2- 172.21.15.12:0/1393821831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae269410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.677+0000 7f8987e34b80 1 --2- 172.21.15.12:0/1393821831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae25f890 0x562dae274c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.677+0000 7f8987e34b80 1 --2- 172.21.15.12:0/1393821831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae264e00 0x562dae259d20 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.677+0000 7f8987e34b80 1 -- 172.21.15.12:0/1393821831 >> 172.21.15.12:0/1393821831 conn(0x562dae1bd210 msgr2=0x562dae25fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.677+0000 7f8987e34b80 1 -- 172.21.15.12:0/1393821831 shutdown_connections 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.677+0000 7f8987e34b80 1 -- 172.21.15.12:0/1393821831 wait complete. 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.677+0000 7f8987e34b80 1 -- start start 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.678+0000 7f8987e34b80 1 Processor -- start 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.678+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae2c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.678+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 0x562dae2c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.678+0000 7f8987e34b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae26fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.678+0000 7f8987e34b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae2700d0 con 0x562dae25f890 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.678+0000 7f8987e34b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae270210 con 0x562dae265720 2024-05-09T00:10:46.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.678+0000 7f8987e34b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dae2703d0 con 0x562dae264e00 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.678+0000 7f8987e34b80 10 client.0 _ll_register_callbacks cb 0x562dae154b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.678+0000 7f8979a16700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae2c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.678+0000 7f8979a16700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae2c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57220/0 (socket says 172.21.15.12:57220) 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.679+0000 7f8979a16700 1 -- 172.21.15.12:0/298871597 learned_addr learned my addr 172.21.15.12:0/298871597 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.679+0000 7f897a217700 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae26fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.679+0000 7f8979215700 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 0x562dae2c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.679+0000 7f8979a16700 1 -- 172.21.15.12:0/298871597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 msgr2=0x562dae2c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.679+0000 7f8979a16700 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 0x562dae2c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.679+0000 7f8979a16700 1 -- 172.21.15.12:0/298871597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 msgr2=0x562dae26fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.679+0000 7f8979a16700 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae26fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.679+0000 7f8979a16700 1 -- 172.21.15.12:0/298871597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897406afe0 con 0x562dae25f890 2024-05-09T00:10:46.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.679+0000 7f8979a16700 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae2c6ef0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f897000e9c0 tx=0x7f8970005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c3351d389e3bb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.681+0000 7f895d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.681+0000 7f895d7fa700 1 -- 172.21.15.12:0/298871597 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897000f3c0 con 0x562dae25f890 2024-05-09T00:10:46.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.681+0000 7f895d7fa700 1 -- 172.21.15.12:0/298871597 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f897001c520 con 0x562dae25f890 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.681+0000 7f8987e34b80 1 -- 172.21.15.12:0/298871597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8970004cf0 con 0x562dae25f890 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.681+0000 7f895d7fa700 1 -- 172.21.15.12:0/298871597 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8970004e60 con 0x562dae25f890 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.681+0000 7f8987e34b80 10 client.22065 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.681+0000 7f8987e34b80 1 -- 172.21.15.12:0/298871597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x562dae175560 con 0x562dae25f890 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.681+0000 7f895d7fa700 1 -- 172.21.15.12:0/298871597 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8970004cf0 con 0x562dae25f890 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.681+0000 7f895d7fa700 1 -- 172.21.15.12:0/298871597 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f8970010710 con 0x562dae25f890 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.682+0000 7f8987e34b80 10 client.22065 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.682+0000 7f8987e34b80 10 client.22065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.682+0000 7f8987e34b80 20 client.22065 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.682+0000 7f8987e34b80 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dae2cfbb0 0x562dae2d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.682+0000 7f8987e34b80 4 client.22065 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.682+0000 7f8987e34b80 1 -- 172.21.15.12:0/298871597 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562dae1a7ec0 con 0x562dae2cfbb0 2024-05-09T00:10:46.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.682+0000 7f8979215700 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dae2cfbb0 0x562dae2d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.682+0000 7f8979215700 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dae2cfbb0 0x562dae2d1fd0 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22429d3efdb957a7 server_cookie=20efa58c26fead8 in_seq=0 out_seq=0 2024-05-09T00:10:46.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.682+0000 7f895d7fa700 10 client.22065 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.684+0000 7f895d7fa700 1 -- 172.21.15.12:0/298871597 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562dae1a7ec0 con 0x562dae2cfbb0 2024-05-09T00:10:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.684+0000 7f895d7fa700 10 client.22065 handle_command_reply: tid=0 2024-05-09T00:10:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.684+0000 7f8987e34b80 1 client.22065 shutdown 2024-05-09T00:10:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 1 -- 172.21.15.12:0/298871597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 msgr2=0x562dae2c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae2c6ef0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f897000e9c0 tx=0x7f8970005760 comp rx=0 tx=0).stop 2024-05-09T00:10:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 1 -- 172.21.15.12:0/298871597 shutdown_connections 2024-05-09T00:10:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x562dae2cfbb0 0x562dae2d1fd0 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562dae265720 0x562dae26fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562dae25f890 0x562dae2c6ef0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f895d7fa700 0 client.22065 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:46.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 1 --2- 172.21.15.12:0/298871597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562dae264e00 0x562dae2c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 1 -- 172.21.15.12:0/298871597 >> 172.21.15.12:0/298871597 conn(0x562dae1bd210 msgr2=0x562dae262cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 1 -- 172.21.15.12:0/298871597 shutdown_connections 2024-05-09T00:10:46.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 1 -- 172.21.15.12:0/298871597 wait complete. 2024-05-09T00:10:46.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.685+0000 7f8987e34b80 20 client.22065 trim_cache size 0 max 16384 2024-05-09T00:10:46.685 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:10:46.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 -- 172.21.15.12:0/1396598933 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89600a1310 msgr2=0x7f89600a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89600a1310 0x7f89600a37d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8970005d90 tx=0x7f8970005ce0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 -- 172.21.15.12:0/1396598933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8974108490 msgr2=0x7f897410e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8974108490 0x7f897410e7b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8964000c00 tx=0x7f8964009510 comp rx=0 tx=0).stop 2024-05-09T00:10:46.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 -- 172.21.15.12:0/1396598933 shutdown_connections 2024-05-09T00:10:46.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f89600a1310 0x7f89600a37d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897410ecf0 0x7f89741a9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8974103370 0x7f897410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 --2- 172.21.15.12:0/1396598933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8974108490 0x7f897410e7b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 -- 172.21.15.12:0/1396598933 >> 172.21.15.12:0/1396598933 conn(0x7f89740febb0 msgr2=0x7f89740fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 -- 172.21.15.12:0/1396598933 shutdown_connections 2024-05-09T00:10:46.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:46.686+0000 7f897bc7a700 1 -- 172.21.15.12:0/1396598933 wait complete. 2024-05-09T00:10:46.830 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:46 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2034554639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:47.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:46 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2034554639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:47.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:46 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2034554639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:47.313 INFO:tasks.workunit.client.0.smithi012.stdout: 10 362888 28.49 MB/sec execute 522 sec latency 119.984 ms 2024-05-09T00:10:47.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:10:47.702 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:10:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.872+0000 7f7eaa83a700 1 -- 172.21.15.12:0/496596022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ea4100a10 msgr2=0x7f7ea4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.872+0000 7f7eaa83a700 1 --2- 172.21.15.12:0/496596022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ea4100a10 0x7f7ea4104a30 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009990 tx=0x7f7e98009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.872+0000 7f7eaa83a700 1 -- 172.21.15.12:0/496596022 shutdown_connections 2024-05-09T00:10:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.872+0000 7f7eaa83a700 1 --2- 172.21.15.12:0/496596022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ea4105160 0x7f7ea410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.872+0000 7f7eaa83a700 1 --2- 172.21.15.12:0/496596022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea4100060 0x7f7ea4100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.872+0000 7f7eaa83a700 1 --2- 172.21.15.12:0/496596022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ea4100a10 0x7f7ea4104a30 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.872+0000 7f7eaa83a700 1 -- 172.21.15.12:0/496596022 >> 172.21.15.12:0/496596022 conn(0x7f7ea4076f40 msgr2=0x7f7ea4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.872+0000 7f7eaa83a700 1 -- 172.21.15.12:0/496596022 shutdown_connections 2024-05-09T00:10:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.872+0000 7f7eaa83a700 1 -- 172.21.15.12:0/496596022 wait complete. 2024-05-09T00:10:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.873+0000 7f7eaa83a700 1 Processor -- start 2024-05-09T00:10:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.873+0000 7f7eaa83a700 1 -- start start 2024-05-09T00:10:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.873+0000 7f7eaa83a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea4100060 0x7f7ea4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7eaa83a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ea4105160 0x7f7ea4199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7eaa83a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ea419df30 0x7f7ea419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7eaa83a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea419e9b0 con 0x7f7ea4105160 2024-05-09T00:10:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7eaa83a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea419eb20 con 0x7f7ea419df30 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7eaa83a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea419ece0 con 0x7f7ea4100060 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea4100060 0x7f7ea4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea4100060 0x7f7ea4199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47612/0 (socket says 172.21.15.12:47612) 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea3fff700 1 -- 172.21.15.12:0/1391630807 learned_addr learned my addr 172.21.15.12:0/1391630807 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea37fe700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ea4105160 0x7f7ea4199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ea419df30 0x7f7ea419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea3fff700 1 -- 172.21.15.12:0/1391630807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ea419df30 msgr2=0x7f7ea419e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea3fff700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ea419df30 0x7f7ea419e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea3fff700 1 -- 172.21.15.12:0/1391630807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ea4105160 msgr2=0x7f7ea4199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea3fff700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ea4105160 0x7f7ea4199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea3fff700 1 -- 172.21.15.12:0/1391630807 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980095f0 con 0x7f7ea4100060 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.874+0000 7f7ea3fff700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea4100060 0x7f7ea4199650 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e9a0 tx=0x7f7e9400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.875+0000 7f7ea17fa700 1 -- 172.21.15.12:0/1391630807 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94009790 con 0x7f7ea4100060 2024-05-09T00:10:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.875+0000 7f7ea17fa700 1 -- 172.21.15.12:0/1391630807 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e94010430 con 0x7f7ea4100060 2024-05-09T00:10:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.875+0000 7f7eaa83a700 1 -- 172.21.15.12:0/1391630807 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea4194ff0 con 0x7f7ea4100060 2024-05-09T00:10:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.875+0000 7f7eaa83a700 1 -- 172.21.15.12:0/1391630807 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea4195520 con 0x7f7ea4100060 2024-05-09T00:10:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.875+0000 7f7ea17fa700 1 -- 172.21.15.12:0/1391630807 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94018680 con 0x7f7ea4100060 2024-05-09T00:10:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.878+0000 7f7ea17fa700 1 -- 172.21.15.12:0/1391630807 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7e9400ca10 con 0x7f7ea4100060 2024-05-09T00:10:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.878+0000 7f7ea17fa700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7e8c0a1300 0x7f7e8c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.879+0000 7f7ea37fe700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7e8c0a1300 0x7f7e8c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.879+0000 7f7ea17fa700 1 -- 172.21.15.12:0/1391630807 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7e94014070 con 0x7f7ea4100060 2024-05-09T00:10:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.879+0000 7f7ea37fe700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7e8c0a1300 0x7f7e8c0a37c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800b2e0 tx=0x7f7e98005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:47.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7eb69f4b80 1 Processor -- start 2024-05-09T00:10:47.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7eb69f4b80 1 -- start start 2024-05-09T00:10:47.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a81987260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a819877a0 0x557a819813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a81981910 0x557a81981d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a818d81d0 con 0x557a81981910 2024-05-09T00:10:47.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a818d5b90 con 0x557a81986e80 2024-05-09T00:10:47.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a818d1480 con 0x557a819877a0 2024-05-09T00:10:47.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7ea8dd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a81981910 0x557a81981d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7ea8dd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a81981910 0x557a81981d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57240/0 (socket says 172.21.15.12:57240) 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3604411352 learned_addr learned my addr 172.21.15.12:0/3604411352 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7ea3fff700 1 --2- 172.21.15.12:0/3604411352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a81987260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.880+0000 7f7ea37fe700 1 --2- 172.21.15.12:0/3604411352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a819877a0 0x557a819813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.881+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3604411352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a819877a0 msgr2=0x557a819813d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.881+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/3604411352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a819877a0 0x557a819813d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.881+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3604411352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 msgr2=0x557a81987260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.881+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/3604411352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a81987260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.881+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3604411352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a81752430 con 0x557a81981910 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.881+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/3604411352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a81981910 0x557a81981d50 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c011150 tx=0x7f7e9c011460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2746eab2a580e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.881+0000 7f7e8a7fc700 1 -- 172.21.15.12:0/3604411352 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c01b830 con 0x557a81981910 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.881+0000 7f7e8a7fc700 1 -- 172.21.15.12:0/3604411352 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e9c01be50 con 0x557a81981910 2024-05-09T00:10:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.881+0000 7f7e8a7fc700 1 -- 172.21.15.12:0/3604411352 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c01c3d0 con 0x557a81981910 2024-05-09T00:10:47.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.882+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3604411352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a81981910 msgr2=0x557a81981d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:47.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.882+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3604411352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a81981910 0x557a81981d50 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c011150 tx=0x7f7e9c011460 comp rx=0 tx=0).stop 2024-05-09T00:10:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.882+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3604411352 shutdown_connections 2024-05-09T00:10:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.882+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3604411352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a81981910 0x557a81981d50 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.882+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3604411352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a81987260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.882+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3604411352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a819877a0 0x557a819813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.882+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3604411352 >> 172.21.15.12:0/3604411352 conn(0x557a818df290 msgr2=0x557a8197be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.882+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3604411352 shutdown_connections 2024-05-09T00:10:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.882+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3604411352 wait complete. 2024-05-09T00:10:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.882+0000 7f7eb69f4b80 1 -- start start 2024-05-09T00:10:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.883+0000 7f7eb69f4b80 1 Processor -- start 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.888+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81981910 0x557a8198d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.888+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a8198aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.888+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.888+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8186d5e0 con 0x557a819877a0 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.888+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8186d280 con 0x557a81986e80 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.888+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a81754720 con 0x557a81981910 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.888+0000 7f7eb69f4b80 10 client.0 _ll_register_callbacks cb 0x557a81876bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7ea37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a8198aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7ea37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a8198aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47404/0 (socket says 172.21.15.12:47404) 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7ea37fe700 1 -- 172.21.15.12:0/3117522121 learned_addr learned my addr 172.21.15.12:0/3117522121 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3117522121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81981910 msgr2=0x557a8198d800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:10:47.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81981910 0x557a8198d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3117522121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 msgr2=0x557a8198aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a8198aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3117522121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c00fd30 con 0x557a819877a0 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198b230 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c011120 tx=0x7f7e9c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9022d5017dc4a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7e837fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.889+0000 7f7e837fe700 1 -- 172.21.15.12:0/3117522121 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c01d960 con 0x557a819877a0 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.890+0000 7f7e837fe700 1 -- 172.21.15.12:0/3117522121 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7e9c016050 con 0x557a819877a0 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.890+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3117522121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a8182a2f0 con 0x557a819877a0 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.890+0000 7f7e837fe700 1 -- 172.21.15.12:0/3117522121 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e9c01b840 con 0x557a819877a0 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.890+0000 7f7eb69f4b80 10 client.22077 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.890+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3117522121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x557a816febe0 con 0x557a819877a0 2024-05-09T00:10:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.890+0000 7f7e837fe700 1 -- 172.21.15.12:0/3117522121 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a8182a2f0 con 0x557a819877a0 2024-05-09T00:10:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.890+0000 7f7e837fe700 1 -- 172.21.15.12:0/3117522121 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7e9c03ad20 con 0x557a819877a0 2024-05-09T00:10:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.891+0000 7f7eb69f4b80 10 client.22077 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.891+0000 7f7eb69f4b80 10 client.22077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.891+0000 7f7eb69f4b80 20 client.22077 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.891+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a819ac290 0x557a819ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.891+0000 7f7eb69f4b80 4 client.22077 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.891+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3117522121 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a81950890 con 0x557a819ac290 2024-05-09T00:10:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.892+0000 7f7ea3fff700 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a819ac290 0x557a819ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.892+0000 7f7e837fe700 10 client.22077 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.892+0000 7f7ea3fff700 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a819ac290 0x557a819ae6b0 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8723c4d25e38f3b server_cookie=e613315e53d31d4e in_seq=0 out_seq=0 2024-05-09T00:10:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.898+0000 7f7e837fe700 1 -- 172.21.15.12:0/3117522121 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a81950890 con 0x557a819ac290 2024-05-09T00:10:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.898+0000 7f7e837fe700 10 client.22077 handle_command_reply: tid=0 2024-05-09T00:10:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.898+0000 7f7eb69f4b80 1 client.22077 shutdown 2024-05-09T00:10:47.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.905+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3117522121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 msgr2=0x557a8198b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:47.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.905+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198b230 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c011120 tx=0x7f7e9c00fc90 comp rx=0 tx=0).stop 2024-05-09T00:10:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.906+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3117522121 shutdown_connections 2024-05-09T00:10:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.906+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198b230 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.906+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81981910 0x557a8198d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.906+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a819ac290 0x557a819ae6b0 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.906+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3117522121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a8198aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.906+0000 7f7e837fe700 0 client.22077 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.906+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3117522121 >> 172.21.15.12:0/3117522121 conn(0x557a818df290 msgr2=0x557a8197b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.907+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3117522121 shutdown_connections 2024-05-09T00:10:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.907+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3117522121 wait complete. 2024-05-09T00:10:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.907+0000 7f7eb69f4b80 20 client.22077 trim_cache size 0 max 16384 2024-05-09T00:10:47.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.933+0000 7f7eb69f4b80 1 Processor -- start 2024-05-09T00:10:47.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.933+0000 7f7eb69f4b80 1 -- start start 2024-05-09T00:10:47.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81981910 0x557a819813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81986e80 0x557a81986330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8197bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8197d0d0 con 0x557a819877a0 2024-05-09T00:10:47.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8198bae0 con 0x557a81981910 2024-05-09T00:10:47.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a819822b0 con 0x557a81986e80 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7ea8dd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8197bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7ea8dd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8197bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57258/0 (socket says 172.21.15.12:57258) 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7ea37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81986e80 0x557a81986330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/1674807292 learned_addr learned my addr 172.21.15.12:0/1674807292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/1674807292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81986e80 msgr2=0x557a81986330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/1674807292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81986e80 0x557a81986330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/1674807292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81981910 msgr2=0x557a819813d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/1674807292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81981910 0x557a819813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/1674807292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a818975e0 con 0x557a819877a0 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.934+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/1674807292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8197bda0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c004ad0 tx=0x7f7e9c012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff4094fbd4f13f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.935+0000 7f7e8affd700 1 -- 172.21.15.12:0/1674807292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c011ba0 con 0x557a819877a0 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.935+0000 7f7e8affd700 1 -- 172.21.15.12:0/1674807292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e9c0276c0 con 0x557a819877a0 2024-05-09T00:10:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.935+0000 7f7e8affd700 1 -- 172.21.15.12:0/1674807292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c027b20 con 0x557a819877a0 2024-05-09T00:10:47.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.935+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/1674807292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 msgr2=0x557a8197bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:47.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.935+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/1674807292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8197bda0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c004ad0 tx=0x7f7e9c012f60 comp rx=0 tx=0).stop 2024-05-09T00:10:47.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.936+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/1674807292 shutdown_connections 2024-05-09T00:10:47.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.936+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/1674807292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8197bda0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.936+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/1674807292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81981910 0x557a819813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.936+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/1674807292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81986e80 0x557a81986330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.936+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/1674807292 >> 172.21.15.12:0/1674807292 conn(0x557a818df290 msgr2=0x557a81985c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:47.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.936+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/1674807292 shutdown_connections 2024-05-09T00:10:47.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.936+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/1674807292 wait complete. 2024-05-09T00:10:47.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.936+0000 7f7eb69f4b80 1 -- start start 2024-05-09T00:10:47.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.936+0000 7f7eb69f4b80 1 Processor -- start 2024-05-09T00:10:47.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81981910 0x557a8198ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a8198f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7eb69f4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a81a028c0 con 0x557a819877a0 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a81a02a30 con 0x557a81986e80 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7eb69f4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a81a02bf0 con 0x557a81981910 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7eb69f4b80 10 client.0 _ll_register_callbacks cb 0x557a81876bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7ea8dd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7ea8dd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57266/0 (socket says 172.21.15.12:57266) 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3927584055 learned_addr learned my addr 172.21.15.12:0/3927584055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.938+0000 7f7ea37fe700 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a8198f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.939+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3927584055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81981910 msgr2=0x557a8198ef90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.939+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81981910 0x557a8198ef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.939+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3927584055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 msgr2=0x557a8198f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.939+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a8198f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.939+0000 7f7ea8dd7700 1 -- 172.21.15.12:0/3927584055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a816febe0 con 0x557a819877a0 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.939+0000 7f7ea8dd7700 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198fa10 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0114a0 tx=0x7f7e9c01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f8c751f1e9f383 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.939+0000 7f7e837fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.939+0000 7f7e837fe700 1 -- 172.21.15.12:0/3927584055 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c011ba0 con 0x557a819877a0 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.939+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3927584055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e9c01be50 con 0x557a819877a0 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.939+0000 7f7e837fe700 1 -- 172.21.15.12:0/3927584055 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7e9c016040 con 0x557a819877a0 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7e837fe700 1 -- 172.21.15.12:0/3927584055 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e9c01c3d0 con 0x557a819877a0 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7eb69f4b80 10 client.22083 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3927584055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x557a81756d50 con 0x557a819877a0 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7e837fe700 1 -- 172.21.15.12:0/3927584055 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e9c01be50 con 0x557a819877a0 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7e837fe700 1 -- 172.21.15.12:0/3927584055 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7e9c01cd60 con 0x557a819877a0 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7eb69f4b80 10 client.22083 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7eb69f4b80 10 client.22083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7eb69f4b80 20 client.22083 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a81a03fe0 0x557a81a06400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7eb69f4b80 4 client.22083 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:10:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.940+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3927584055 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557a818c9f40 con 0x557a81a03fe0 2024-05-09T00:10:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.941+0000 7f7ea3fff700 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a81a03fe0 0x557a81a06400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.942+0000 7f7e837fe700 10 client.22083 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.942+0000 7f7ea3fff700 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a81a03fe0 0x557a81a06400 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689f0af7a09a997 server_cookie=776cc2802cf9bd64 in_seq=0 out_seq=0 2024-05-09T00:10:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7e837fe700 1 -- 172.21.15.12:0/3927584055 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557a818c9f40 con 0x557a81a03fe0 2024-05-09T00:10:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7e837fe700 10 client.22083 handle_command_reply: tid=0 2024-05-09T00:10:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7eb69f4b80 1 client.22083 shutdown 2024-05-09T00:10:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3927584055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 msgr2=0x557a8198fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198fa10 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0114a0 tx=0x7f7e9c01baf0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3927584055 shutdown_connections 2024-05-09T00:10:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a819877a0 0x557a8198fa10 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a81981910 0x557a8198ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557a81a03fe0 0x557a81a06400 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7e837fe700 0 client.22083 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7eb69f4b80 1 --2- 172.21.15.12:0/3927584055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a81986e80 0x557a8198f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.944+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3927584055 >> 172.21.15.12:0/3927584055 conn(0x557a818df290 msgr2=0x557a8198c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.946+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3927584055 shutdown_connections 2024-05-09T00:10:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.946+0000 7f7eb69f4b80 1 -- 172.21.15.12:0/3927584055 wait complete. 2024-05-09T00:10:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.946+0000 7f7eb69f4b80 20 client.22083 trim_cache size 0 max 16384 2024-05-09T00:10:47.944 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:10:47.944 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:10:47.945 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "9cfed306-aa48-4d75-b2f6-37367d27dfd1", 2024-05-09T00:10:47.945 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:10:47.945 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 -- 172.21.15.12:0/1391630807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7e8c0a1300 msgr2=0x7f7e8c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7e8c0a1300 0x7f7e8c0a37c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800b2e0 tx=0x7f7e98005960 comp rx=0 tx=0).stop 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 -- 172.21.15.12:0/1391630807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea4100060 msgr2=0x7f7ea4199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea4100060 0x7f7ea4199650 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e9a0 tx=0x7f7e9400ecb0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 -- 172.21.15.12:0/1391630807 shutdown_connections 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7e8c0a1300 0x7f7e8c0a37c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ea419df30 0x7f7ea419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea4100060 0x7f7ea4199650 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 --2- 172.21.15.12:0/1391630807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ea4105160 0x7f7ea4199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 -- 172.21.15.12:0/1391630807 >> 172.21.15.12:0/1391630807 conn(0x7f7ea4076f40 msgr2=0x7f7ea40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 -- 172.21.15.12:0/1391630807 shutdown_connections 2024-05-09T00:10:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:47.947+0000 7f7eaa83a700 1 -- 172.21.15.12:0/1391630807 wait complete. 2024-05-09T00:10:47.963 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:10:48.009 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:47 smithi012 ceph-mon[42503]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.2 MiB/s wr, 514 op/s 2024-05-09T00:10:48.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.222+0000 7ffac8e13700 1 -- 172.21.15.12:0/4127458714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac4108210 msgr2=0x7ffac41085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.222+0000 7ffac8e13700 1 --2- 172.21.15.12:0/4127458714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac4108210 0x7ffac41085f0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ffabc008800 tx=0x7ffabc008b10 comp rx=0 tx=0).stop 2024-05-09T00:10:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.222+0000 7ffac8e13700 1 -- 172.21.15.12:0/4127458714 shutdown_connections 2024-05-09T00:10:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.222+0000 7ffac8e13700 1 --2- 172.21.15.12:0/4127458714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac410c820 0x7ffac4079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.222+0000 7ffac8e13700 1 --2- 172.21.15.12:0/4127458714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac4108210 0x7ffac41085f0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.222+0000 7ffac8e13700 1 --2- 172.21.15.12:0/4127458714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffac4108bc0 0x7ffac410c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.222+0000 7ffac8e13700 1 -- 172.21.15.12:0/4127458714 >> 172.21.15.12:0/4127458714 conn(0x7ffac406bff0 msgr2=0x7ffac406c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.223+0000 7ffac8e13700 1 -- 172.21.15.12:0/4127458714 shutdown_connections 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.223+0000 7ffac8e13700 1 -- 172.21.15.12:0/4127458714 wait complete. 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac8e13700 1 Processor -- start 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac8e13700 1 -- start start 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac8e13700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffac4108210 0x7ffac40838d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac8e13700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac410c820 0x7ffac407d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac8e13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac407deb0 0x7ffac407e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac8e13700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac41bb100 con 0x7ffac407deb0 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac8e13700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac41bb240 con 0x7ffac4108210 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac8e13700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac41bb3b0 con 0x7ffac410c820 2024-05-09T00:10:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac410c820 0x7ffac407d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac410c820 0x7ffac407d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47672/0 (socket says 172.21.15.12:47672) 2024-05-09T00:10:48.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.224+0000 7ffac1d9b700 1 -- 172.21.15.12:0/3041365587 learned_addr learned my addr 172.21.15.12:0/3041365587 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:48.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.225+0000 7ffac259c700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffac4108210 0x7ffac40838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.225+0000 7ffac2d9d700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac407deb0 0x7ffac407e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.226+0000 7ffac2d9d700 1 -- 172.21.15.12:0/3041365587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac410c820 msgr2=0x7ffac407d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.226+0000 7ffac2d9d700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac410c820 0x7ffac407d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.226+0000 7ffac2d9d700 1 -- 172.21.15.12:0/3041365587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffac4108210 msgr2=0x7ffac40838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.226+0000 7ffac2d9d700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffac4108210 0x7ffac40838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.226+0000 7ffac2d9d700 1 -- 172.21.15.12:0/3041365587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc008430 con 0x7ffac407deb0 2024-05-09T00:10:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.226+0000 7ffac2d9d700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac407deb0 0x7ffac407e360 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ffab8004670 tx=0x7ffab80046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.227+0000 7ffab37fe700 1 -- 172.21.15.12:0/3041365587 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab800f040 con 0x7ffac407deb0 2024-05-09T00:10:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.227+0000 7ffab37fe700 1 -- 172.21.15.12:0/3041365587 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab8013ac0 con 0x7ffac407deb0 2024-05-09T00:10:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.227+0000 7ffab37fe700 1 -- 172.21.15.12:0/3041365587 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8012ae0 con 0x7ffac407deb0 2024-05-09T00:10:48.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.227+0000 7ffac8e13700 1 -- 172.21.15.12:0/3041365587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac41bb690 con 0x7ffac407deb0 2024-05-09T00:10:48.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.227+0000 7ffac8e13700 1 -- 172.21.15.12:0/3041365587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac41bbb50 con 0x7ffac407deb0 2024-05-09T00:10:48.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.234+0000 7ffab37fe700 1 -- 172.21.15.12:0/3041365587 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ffab8012c80 con 0x7ffac407deb0 2024-05-09T00:10:48.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.235+0000 7ffad4fcdb80 1 Processor -- start 2024-05-09T00:10:48.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.236+0000 7ffad4fcdb80 1 -- start start 2024-05-09T00:10:48.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.236+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a6b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.236+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6b720 0x5570f5a65350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.236+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a65cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.236+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f59bc150 con 0x5570f5a6ae00 2024-05-09T00:10:48.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.236+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f59b9b10 con 0x5570f5a6b720 2024-05-09T00:10:48.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.236+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f59b5400 con 0x5570f5a65890 2024-05-09T00:10:48.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.237+0000 7ffab37fe700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffaac0a13d0 0x7ffaac0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.237+0000 7ffab37fe700 1 -- 172.21.15.12:0/3041365587 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ffab80cd850 con 0x7ffac407deb0 2024-05-09T00:10:48.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.238+0000 7ffac259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6b720 0x5570f5a65350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.238+0000 7ffac259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6b720 0x5570f5a65350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47462/0 (socket says 172.21.15.12:47462) 2024-05-09T00:10:48.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.238+0000 7ffac259c700 1 -- 172.21.15.12:0/335418444 learned_addr learned my addr 172.21.15.12:0/335418444 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:48.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.238+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/335418444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a65cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.240+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/335418444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a6b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.240+0000 7ffac1d9b700 1 -- 172.21.15.12:0/335418444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6b720 msgr2=0x5570f5a65350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:48.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.240+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/335418444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6b720 0x5570f5a65350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.240+0000 7ffac1d9b700 1 -- 172.21.15.12:0/335418444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 msgr2=0x5570f5a6b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:48.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.240+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/335418444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a6b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.240+0000 7ffac1d9b700 1 -- 172.21.15.12:0/335418444 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570f58363b0 con 0x5570f5a65890 2024-05-09T00:10:48.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.243+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/335418444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a65cd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ffac4072610 tx=0x7ffab400d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6686cd769f864e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:48.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.244+0000 7ffac2d9d700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffaac0a13d0 0x7ffaac0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.253+0000 7ffac2d9d700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffaac0a13d0 0x7ffaac0a3890 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffab8003dc0 tx=0x7ffab8003f70 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:48.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.254+0000 7ffab0ff9700 1 -- 172.21.15.12:0/335418444 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4004030 con 0x5570f5a65890 2024-05-09T00:10:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.254+0000 7ffab0ff9700 1 -- 172.21.15.12:0/335418444 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab4011050 con 0x5570f5a65890 2024-05-09T00:10:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.254+0000 7ffab0ff9700 1 -- 172.21.15.12:0/335418444 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab40193c0 con 0x5570f5a65890 2024-05-09T00:10:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.255+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/335418444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 msgr2=0x5570f5a65cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.255+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/335418444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a65cd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ffac4072610 tx=0x7ffab400d040 comp rx=0 tx=0).stop 2024-05-09T00:10:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.255+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/335418444 shutdown_connections 2024-05-09T00:10:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.255+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/335418444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a65cd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.255+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/335418444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a6b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.255+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/335418444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6b720 0x5570f5a65350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.255+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/335418444 >> 172.21.15.12:0/335418444 conn(0x5570f59c3210 msgr2=0x5570f5a5fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.255+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/335418444 shutdown_connections 2024-05-09T00:10:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.255+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/335418444 wait complete. 2024-05-09T00:10:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.256+0000 7ffad4fcdb80 1 -- start start 2024-05-09T00:10:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.256+0000 7ffad4fcdb80 1 Processor -- start 2024-05-09T00:10:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.257+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a6ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.257+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a6f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.257+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6b720 0x5570f5a6f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.257+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f5951560 con 0x5570f5a6ae00 2024-05-09T00:10:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.257+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f5951200 con 0x5570f5a6b720 2024-05-09T00:10:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.257+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f58386a0 con 0x5570f5a65890 2024-05-09T00:10:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.257+0000 7ffad4fcdb80 10 client.0 _ll_register_callbacks cb 0x5570f595ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.257+0000 7ffac1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a6ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.258+0000 7ffac1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a6ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47676/0 (socket says 172.21.15.12:47676) 2024-05-09T00:10:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.258+0000 7ffac1d9b700 1 -- 172.21.15.12:0/390757721 learned_addr learned my addr 172.21.15.12:0/390757721 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.258+0000 7ffac259c700 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a6f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.259+0000 7ffac259c700 1 -- 172.21.15.12:0/390757721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 msgr2=0x5570f5a6ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.259+0000 7ffac259c700 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a6ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.259+0000 7ffac259c700 1 -- 172.21.15.12:0/390757721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6b720 msgr2=0x5570f5a6f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.259+0000 7ffac259c700 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6b720 0x5570f5a6f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.259+0000 7ffac259c700 1 -- 172.21.15.12:0/390757721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab400bdb0 con 0x5570f5a6ae00 2024-05-09T00:10:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.260+0000 7ffac259c700 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a6f180 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b0b0 tx=0x7ffabc00bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935c7d846cf97880 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.260+0000 7ffaa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.261+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/390757721 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc012040 con 0x5570f5a6ae00 2024-05-09T00:10:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.261+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/390757721 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ffabc00d110 con 0x5570f5a6ae00 2024-05-09T00:10:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.261+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/390757721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570f590e270 con 0x5570f5a6ae00 2024-05-09T00:10:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.261+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/390757721 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffabc004e60 con 0x5570f5a6ae00 2024-05-09T00:10:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.261+0000 7ffad4fcdb80 10 client.22095 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:48.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.261+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/390757721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x5570f57e2b60 con 0x5570f5a6ae00 2024-05-09T00:10:48.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.262+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/390757721 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5570f590e270 con 0x5570f5a6ae00 2024-05-09T00:10:48.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.262+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/390757721 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7ffabc009c00 con 0x5570f5a6ae00 2024-05-09T00:10:48.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.262+0000 7ffad4fcdb80 10 client.22095 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:48.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.262+0000 7ffad4fcdb80 10 client.22095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:48.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.262+0000 7ffad4fcdb80 20 client.22095 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:48.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.262+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5570f5a90470 0x5570f5a92890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.262+0000 7ffad4fcdb80 4 client.22095 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:48.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.262+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/390757721 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570f5a34810 con 0x5570f5a90470 2024-05-09T00:10:48.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.264+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5570f5a90470 0x5570f5a92890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:47 smithi160 ceph-mon[46152]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.2 MiB/s wr, 514 op/s 2024-05-09T00:10:48.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:47 smithi204 ceph-mon[47373]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.2 MiB/s wr, 514 op/s 2024-05-09T00:10:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.277+0000 7ffaa9ffb700 10 client.22095 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:48.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.277+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5570f5a90470 0x5570f5a92890 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76367309089c9d1a server_cookie=d3c2d5a9e9ab2ad8 in_seq=0 out_seq=0 2024-05-09T00:10:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.291+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/390757721 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5570f5a34810 con 0x5570f5a90470 2024-05-09T00:10:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.291+0000 7ffaa9ffb700 10 client.22095 handle_command_reply: tid=0 2024-05-09T00:10:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.291+0000 7ffad4fcdb80 1 client.22095 shutdown 2024-05-09T00:10:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.291+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/390757721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 msgr2=0x5570f5a6f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.291+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a6f180 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b0b0 tx=0x7ffabc00bb80 comp rx=0 tx=0).stop 2024-05-09T00:10:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.295+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/390757721 shutdown_connections 2024-05-09T00:10:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.295+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5570f5a90470 0x5570f5a92890 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.295+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6b720 0x5570f5a6f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.295+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a6ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.295+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/390757721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a6f180 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.295+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/390757721 >> 172.21.15.12:0/390757721 conn(0x5570f59c3210 msgr2=0x5570f5a5f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.295+0000 7ffaa9ffb700 0 client.22095 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.297+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/390757721 shutdown_connections 2024-05-09T00:10:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.298+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/390757721 wait complete. 2024-05-09T00:10:48.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.302+0000 7ffad4fcdb80 20 client.22095 trim_cache size 0 max 16384 2024-05-09T00:10:48.313 INFO:tasks.workunit.client.0.smithi012.stdout: 10 363304 28.48 MB/sec execute 523 sec latency 283.918 ms 2024-05-09T00:10:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.335+0000 7ffad4fcdb80 1 Processor -- start 2024-05-09T00:10:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.335+0000 7ffad4fcdb80 1 -- start start 2024-05-09T00:10:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a65890 0x5570f5a65350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a5fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a6b720 0x5570f5a6d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f5a70280 con 0x5570f5a6ae00 2024-05-09T00:10:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f5a8da00 con 0x5570f5a65890 2024-05-09T00:10:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f5a5ce10 con 0x5570f5a6b720 2024-05-09T00:10:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffac1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a6b720 0x5570f5a6d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffac1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a6b720 0x5570f5a6d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47692/0 (socket says 172.21.15.12:47692) 2024-05-09T00:10:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffac1d9b700 1 -- 172.21.15.12:0/4124879251 learned_addr learned my addr 172.21.15.12:0/4124879251 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffac1d9b700 1 -- 172.21.15.12:0/4124879251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a65890 msgr2=0x5570f5a65350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/4124879251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a65890 0x5570f5a65350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffac1d9b700 1 -- 172.21.15.12:0/4124879251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 msgr2=0x5570f5a5fd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:10:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.336+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/4124879251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a5fd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.337+0000 7ffac1d9b700 1 -- 172.21.15.12:0/4124879251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570f5a65f40 con 0x5570f5a6b720 2024-05-09T00:10:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.337+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/4124879251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a6b720 0x5570f5a6d510 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b520 tx=0x7ffab400b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a8e9c15c719d45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.341+0000 7ffab17fa700 1 -- 172.21.15.12:0/4124879251 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4005240 con 0x5570f5a6b720 2024-05-09T00:10:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.341+0000 7ffab17fa700 1 -- 172.21.15.12:0/4124879251 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab402be90 con 0x5570f5a6b720 2024-05-09T00:10:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.341+0000 7ffab17fa700 1 -- 172.21.15.12:0/4124879251 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab40298f0 con 0x5570f5a6b720 2024-05-09T00:10:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.342+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/4124879251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a6b720 msgr2=0x5570f5a6d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.342+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/4124879251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a6b720 0x5570f5a6d510 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b520 tx=0x7ffab400b6b0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.343+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/4124879251 shutdown_connections 2024-05-09T00:10:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.343+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/4124879251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a6b720 0x5570f5a6d510 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.343+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/4124879251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a65890 0x5570f5a65350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.343+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/4124879251 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6ae00 0x5570f5a5fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.343+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/4124879251 >> 172.21.15.12:0/4124879251 conn(0x5570f59c3210 msgr2=0x5570f5a68d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.343+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/4124879251 shutdown_connections 2024-05-09T00:10:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.343+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/4124879251 wait complete. 2024-05-09T00:10:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.343+0000 7ffad4fcdb80 1 -- start start 2024-05-09T00:10:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.343+0000 7ffad4fcdb80 1 Processor -- start 2024-05-09T00:10:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.344+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a85340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.344+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6ae00 0x5570f5a85880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.344+0000 7ffad4fcdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6b720 0x5570f5ae5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.344+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f5a85f60 con 0x5570f5a6b720 2024-05-09T00:10:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.344+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f5ae5eb0 con 0x5570f5a6ae00 2024-05-09T00:10:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.344+0000 7ffad4fcdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f5ae5ff0 con 0x5570f5a65890 2024-05-09T00:10:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.344+0000 7ffad4fcdb80 10 client.0 _ll_register_callbacks cb 0x5570f595ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.344+0000 7ffac1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a85340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.344+0000 7ffac1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a85340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:47708/0 (socket says 172.21.15.12:47708) 2024-05-09T00:10:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.344+0000 7ffac1d9b700 1 -- 172.21.15.12:0/907569030 learned_addr learned my addr 172.21.15.12:0/907569030 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.345+0000 7ffac259c700 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6ae00 0x5570f5a85880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.345+0000 7ffac259c700 1 -- 172.21.15.12:0/907569030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 msgr2=0x5570f5a85340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:48.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.345+0000 7ffac259c700 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a85340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.345+0000 7ffac259c700 1 -- 172.21.15.12:0/907569030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6b720 msgr2=0x5570f5ae5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:48.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.345+0000 7ffac259c700 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6b720 0x5570f5ae5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.345+0000 7ffac259c700 1 -- 172.21.15.12:0/907569030 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab401e040 con 0x5570f5a6ae00 2024-05-09T00:10:48.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.345+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a85340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:48.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.345+0000 7ffac259c700 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6ae00 0x5570f5a85880 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x5570f590e3b0 tx=0x7ffabc0051b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d49c01e4ca1425c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.345+0000 7ffaa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:10:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.346+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/907569030 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc012040 con 0x5570f5a6ae00 2024-05-09T00:10:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.346+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/907569030 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ffabc005300 con 0x5570f5a6ae00 2024-05-09T00:10:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.346+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/907569030 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffab4011050 con 0x5570f5a6ae00 2024-05-09T00:10:48.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.352+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/907569030 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffabc004e60 con 0x5570f5a6ae00 2024-05-09T00:10:48.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.352+0000 7ffad4fcdb80 10 client.31469 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:48.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.352+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/907569030 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x5570f597b560 con 0x5570f5a6ae00 2024-05-09T00:10:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.354+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/907569030 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab4011050 con 0x5570f5a6ae00 2024-05-09T00:10:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.354+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/907569030 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7ffabc00d110 con 0x5570f5a6ae00 2024-05-09T00:10:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.355+0000 7ffad4fcdb80 10 client.31469 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.355+0000 7ffad4fcdb80 10 client.31469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:48.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.355+0000 7ffad4fcdb80 20 client.31469 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:48.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.355+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5570f5ae72d0 0x5570f5ae96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:48.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.355+0000 7ffad4fcdb80 4 client.31469 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:10:48.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.355+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/907569030 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5570f59adec0 con 0x5570f5ae72d0 2024-05-09T00:10:48.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.356+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5570f5ae72d0 0x5570f5ae96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:48.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.358+0000 7ffaa9ffb700 10 client.31469 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:48.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.358+0000 7ffac1d9b700 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5570f5ae72d0 0x5570f5ae96c0 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3103b49a107db817 server_cookie=e7d3b416d85d51ef in_seq=0 out_seq=0 2024-05-09T00:10:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.363+0000 7ffaa9ffb700 1 -- 172.21.15.12:0/907569030 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5570f59adec0 con 0x5570f5ae72d0 2024-05-09T00:10:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.363+0000 7ffaa9ffb700 10 client.31469 handle_command_reply: tid=0 2024-05-09T00:10:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.366+0000 7ffad4fcdb80 1 client.31469 shutdown 2024-05-09T00:10:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.367+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/907569030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6ae00 msgr2=0x5570f5a85880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.367+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6ae00 0x5570f5a85880 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x5570f590e3b0 tx=0x7ffabc0051b0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.368+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/907569030 shutdown_connections 2024-05-09T00:10:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.368+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570f5a6b720 0x5570f5ae5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.368+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f5a65890 0x5570f5a85340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.368+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5570f5ae72d0 0x5570f5ae96c0 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.368+0000 7ffad4fcdb80 1 --2- 172.21.15.12:0/907569030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570f5a6ae00 0x5570f5a85880 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.368+0000 7ffaa9ffb700 0 client.31469 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.368+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/907569030 >> 172.21.15.12:0/907569030 conn(0x5570f59c3210 msgr2=0x5570f5a6d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.368+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/907569030 shutdown_connections 2024-05-09T00:10:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.368+0000 7ffad4fcdb80 1 -- 172.21.15.12:0/907569030 wait complete. 2024-05-09T00:10:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffad4fcdb80 20 client.31469 trim_cache size 0 max 16384 2024-05-09T00:10:48.368 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:10:48.368 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (198 inodes in the stack)", 2024-05-09T00:10:48.368 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:10:48.368 INFO:teuthology.orchestra.run.smithi012.stdout: "9cfed306-aa48-4d75-b2f6-37367d27dfd1": { 2024-05-09T00:10:48.368 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:10:48.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "9cfed306-aa48-4d75-b2f6-37367d27dfd1", 2024-05-09T00:10:48.368 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:10:48.368 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:10:48.368 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:10:48.369 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:10:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffac8e13700 1 -- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffaac0a13d0 msgr2=0x7ffaac0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffac8e13700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffaac0a13d0 0x7ffaac0a3890 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffab8003dc0 tx=0x7ffab8003f70 comp rx=0 tx=0).stop 2024-05-09T00:10:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffac8e13700 1 -- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac407deb0 msgr2=0x7ffac407e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffac8e13700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac407deb0 0x7ffac407e360 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ffab8004670 tx=0x7ffab80046a0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffac8e13700 1 -- 172.21.15.12:0/3041365587 shutdown_connections 2024-05-09T00:10:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffac8e13700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ffaac0a13d0 0x7ffaac0a3890 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffac8e13700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac407deb0 0x7ffac407e360 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffac8e13700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffac4108210 0x7ffac40838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffac8e13700 1 --2- 172.21.15.12:0/3041365587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffac410c820 0x7ffac407d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.370+0000 7ffac8e13700 1 -- 172.21.15.12:0/3041365587 >> 172.21.15.12:0/3041365587 conn(0x7ffac406bff0 msgr2=0x7ffac4070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.371+0000 7ffac8e13700 1 -- 172.21.15.12:0/3041365587 shutdown_connections 2024-05-09T00:10:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:10:48.371+0000 7ffac8e13700 1 -- 172.21.15.12:0/3041365587 wait complete. 2024-05-09T00:10:48.389 INFO:tasks.cephfs.filesystem:scrub status for tag:9cfed306-aa48-4d75-b2f6-37367d27dfd1 - {'path': '/', 'tag': '9cfed306-aa48-4d75-b2f6-37367d27dfd1', 'options': 'recursive,force'} 2024-05-09T00:10:49.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:48 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:10:49.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:48 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:48 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:10:49.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:48 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:10:49.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:48 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:10:49.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:48 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:48 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:10:49.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:48 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:48 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:48 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:10:49.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:48 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:48 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:10:49.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:48 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:10:49.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:48 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:10:49.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:48 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:48 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:10:49.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:48 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:48 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:48 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:10:49.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:48 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:48 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:10:49.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:48 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:10:49.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:48 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:10:49.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:48 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:48 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:10:49.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:48 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:48 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:49.313 INFO:tasks.workunit.client.0.smithi012.stdout: 10 363742 28.48 MB/sec execute 524 sec latency 349.525 ms 2024-05-09T00:10:50.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:49 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:50.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:49 smithi012 ceph-mon[42503]: pgmap v786: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.6 MiB/s wr, 571 op/s 2024-05-09T00:10:50.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:49 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:50.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:49 smithi160 ceph-mon[46152]: pgmap v786: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.6 MiB/s wr, 571 op/s 2024-05-09T00:10:50.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:49 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:10:50.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:49 smithi204 ceph-mon[47373]: pgmap v786: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.6 MiB/s wr, 571 op/s 2024-05-09T00:10:50.313 INFO:tasks.workunit.client.0.smithi012.stdout: 10 364086 28.47 MB/sec execute 525 sec latency 171.423 ms 2024-05-09T00:10:51.313 INFO:tasks.workunit.client.0.smithi012.stdout: 10 364507 28.45 MB/sec execute 526 sec latency 104.117 ms 2024-05-09T00:10:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:51 smithi012 ceph-mon[42503]: pgmap v787: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.4 MiB/s wr, 505 op/s 2024-05-09T00:10:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:51 smithi204 ceph-mon[47373]: pgmap v787: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.4 MiB/s wr, 505 op/s 2024-05-09T00:10:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:51 smithi160 ceph-mon[46152]: pgmap v787: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.4 MiB/s wr, 505 op/s 2024-05-09T00:10:52.317 INFO:tasks.workunit.client.0.smithi012.stdout: 10 365016 28.44 MB/sec execute 527 sec latency 100.759 ms 2024-05-09T00:10:53.318 INFO:tasks.workunit.client.0.smithi012.stdout: 10 365548 28.44 MB/sec execute 528 sec latency 203.615 ms 2024-05-09T00:10:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.068+0000 7fad7fc9c700 1 -- 172.21.15.12:0/3943662955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad78100c40 msgr2=0x7fad7810cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.068+0000 7fad7fc9c700 1 --2- 172.21.15.12:0/3943662955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad78100c40 0x7fad7810cc30 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fad7000b0a0 tx=0x7fad7000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.068+0000 7fad7fc9c700 1 -- 172.21.15.12:0/3943662955 shutdown_connections 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.068+0000 7fad7fc9c700 1 --2- 172.21.15.12:0/3943662955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad78100c40 0x7fad7810cc30 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.068+0000 7fad7fc9c700 1 --2- 172.21.15.12:0/3943662955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad78105f20 0x7fad78106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.068+0000 7fad7fc9c700 1 --2- 172.21.15.12:0/3943662955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad78100280 0x7fad78100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.068+0000 7fad7fc9c700 1 -- 172.21.15.12:0/3943662955 >> 172.21.15.12:0/3943662955 conn(0x7fad78076bc0 msgr2=0x7fad78076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.069+0000 7fad7fc9c700 1 -- 172.21.15.12:0/3943662955 shutdown_connections 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.069+0000 7fad7fc9c700 1 -- 172.21.15.12:0/3943662955 wait complete. 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.069+0000 7fad7fc9c700 1 Processor -- start 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.069+0000 7fad7fc9c700 1 -- start start 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.069+0000 7fad7fc9c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad78100280 0x7fad78197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7fc9c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad78100c40 0x7fad781978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7fc9c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad78105f20 0x7fad7819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7fc9c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7819c2b0 con 0x7fad78100280 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7fc9c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7819c420 con 0x7fad78100c40 2024-05-09T00:10:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7fc9c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7819c5e0 con 0x7fad78105f20 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7da38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad78100280 0x7fad78197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7da38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad78100280 0x7fad78197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49198/0 (socket says 172.21.15.12:49198) 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7da38700 1 -- 172.21.15.12:0/970994547 learned_addr learned my addr 172.21.15.12:0/970994547 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7d237700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad78100c40 0x7fad781978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7e239700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad78105f20 0x7fad7819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7da38700 1 -- 172.21.15.12:0/970994547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad78105f20 msgr2=0x7fad7819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7da38700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad78105f20 0x7fad7819bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7da38700 1 -- 172.21.15.12:0/970994547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad78100c40 msgr2=0x7fad781978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7da38700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad78100c40 0x7fad781978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7da38700 1 -- 172.21.15.12:0/970994547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70009d00 con 0x7fad78100280 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7da38700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad78100280 0x7fad78197390 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e9a0 tx=0x7fad7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad6effd700 1 -- 172.21.15.12:0/970994547 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74009790 con 0x7fad78100280 2024-05-09T00:10:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad6effd700 1 -- 172.21.15.12:0/970994547 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad74010430 con 0x7fad78100280 2024-05-09T00:10:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad7fc9c700 1 -- 172.21.15.12:0/970994547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7819c8c0 con 0x7fad78100280 2024-05-09T00:10:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.070+0000 7fad6effd700 1 -- 172.21.15.12:0/970994547 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74018680 con 0x7fad78100280 2024-05-09T00:10:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.071+0000 7fad7fc9c700 1 -- 172.21.15.12:0/970994547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad781a4100 con 0x7fad78100280 2024-05-09T00:10:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.073+0000 7fad6effd700 1 -- 172.21.15.12:0/970994547 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fad740105a0 con 0x7fad78100280 2024-05-09T00:10:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.073+0000 7fad6effd700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad640a1290 0x7fad640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.074+0000 7fad7d237700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad640a1290 0x7fad640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.074+0000 7fad6effd700 1 -- 172.21.15.12:0/970994547 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fad74014070 con 0x7fad78100280 2024-05-09T00:10:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.074+0000 7fad7d237700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad640a1290 0x7fad640a3750 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fad781988e0 tx=0x7fad6800b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad8be56b80 1 Processor -- start 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad8be56b80 1 -- start start 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aef4950 0x55952aef4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 0x55952aeeef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aeef480 0x55952aeef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad8be56b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952ae460f0 con 0x55952aeef480 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad8be56b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952ae43ab0 con 0x55952aef4950 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad8be56b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952ae3f3a0 con 0x55952aef5270 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad7d237700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 0x55952aeeef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad7d237700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 0x55952aeeef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:60410/0 (socket says 172.21.15.12:60410) 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad7e239700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aeef480 0x55952aeef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.075+0000 7fad7d237700 1 -- 172.21.15.12:0/664654635 learned_addr learned my addr 172.21.15.12:0/664654635 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.076+0000 7fad7e239700 1 -- 172.21.15.12:0/664654635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 msgr2=0x55952aeeef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.076+0000 7fad7e239700 1 --2- 172.21.15.12:0/664654635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 0x55952aeeef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.076+0000 7fad7e239700 1 -- 172.21.15.12:0/664654635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aef4950 msgr2=0x55952aef4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.076+0000 7fad7e239700 1 --2- 172.21.15.12:0/664654635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aef4950 0x55952aef4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.076+0000 7fad7e239700 1 -- 172.21.15.12:0/664654635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55952acc0350 con 0x55952aeef480 2024-05-09T00:10:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.076+0000 7fad7d237700 1 --2- 172.21.15.12:0/664654635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 0x55952aeeef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.076+0000 7fad7e239700 1 --2- 172.21.15.12:0/664654635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aeef480 0x55952aeef8a0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fad70009fd0 tx=0x7fad70009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c086ca36a073ad3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.076+0000 7fad63fff700 1 -- 172.21.15.12:0/664654635 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000e040 con 0x55952aeef480 2024-05-09T00:10:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.076+0000 7fad63fff700 1 -- 172.21.15.12:0/664654635 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad70007d30 con 0x55952aeef480 2024-05-09T00:10:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.076+0000 7fad63fff700 1 -- 172.21.15.12:0/664654635 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70007720 con 0x55952aeef480 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.077+0000 7fad8be56b80 1 -- 172.21.15.12:0/664654635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aeef480 msgr2=0x55952aeef8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.077+0000 7fad8be56b80 1 --2- 172.21.15.12:0/664654635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aeef480 0x55952aeef8a0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fad70009fd0 tx=0x7fad70009360 comp rx=0 tx=0).stop 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.077+0000 7fad8be56b80 1 -- 172.21.15.12:0/664654635 shutdown_connections 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.077+0000 7fad8be56b80 1 --2- 172.21.15.12:0/664654635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aeef480 0x55952aeef8a0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.077+0000 7fad8be56b80 1 --2- 172.21.15.12:0/664654635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aef4950 0x55952aef4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.077+0000 7fad8be56b80 1 --2- 172.21.15.12:0/664654635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 0x55952aeeef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.077+0000 7fad8be56b80 1 -- 172.21.15.12:0/664654635 >> 172.21.15.12:0/664654635 conn(0x55952ae4d1b0 msgr2=0x55952aee9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.077+0000 7fad8be56b80 1 -- 172.21.15.12:0/664654635 shutdown_connections 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.077+0000 7fad8be56b80 1 -- 172.21.15.12:0/664654635 wait complete. 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.078+0000 7fad8be56b80 1 -- start start 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.078+0000 7fad8be56b80 1 Processor -- start 2024-05-09T00:10:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.078+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aeef480 0x55952aef8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.078+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef4950 0x55952aef8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.078+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 0x55952aef9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.078+0000 7fad8be56b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952addb500 con 0x55952aef4950 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad8be56b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952addb1a0 con 0x55952aeef480 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad8be56b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952acc2640 con 0x55952aef5270 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad8be56b80 10 client.0 _ll_register_callbacks cb 0x55952ade4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7d237700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef4950 0x55952aef8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7d237700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef4950 0x55952aef8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49214/0 (socket says 172.21.15.12:49214) 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7da38700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aeef480 0x55952aef8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7d237700 1 -- 172.21.15.12:0/2397421900 learned_addr learned my addr 172.21.15.12:0/2397421900 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7e239700 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 0x55952aef9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7d237700 1 -- 172.21.15.12:0/2397421900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 msgr2=0x55952aef9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7d237700 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 0x55952aef9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7d237700 1 -- 172.21.15.12:0/2397421900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aeef480 msgr2=0x55952aef8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7d237700 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aeef480 0x55952aef8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7d237700 1 -- 172.21.15.12:0/2397421900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad68010c20 con 0x55952aef4950 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7da38700 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aeef480 0x55952aef8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad7d237700 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef4950 0x55952aef8ad0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fad68011790 tx=0x7fad680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8fcc20e109bac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.079+0000 7fad617fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad617fa700 1 -- 172.21.15.12:0/2397421900 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68028670 con 0x55952aef4950 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad617fa700 1 -- 172.21.15.12:0/2397421900 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fad68017070 con 0x55952aef4950 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad8be56b80 1 -- 172.21.15.12:0/2397421900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55952ad98210 con 0x55952aef4950 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad617fa700 1 -- 172.21.15.12:0/2397421900 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad68011e60 con 0x55952aef4950 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad8be56b80 10 client.22107 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad8be56b80 1 -- 172.21.15.12:0/2397421900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7fad7810d530 con 0x55952aef4950 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad617fa700 1 -- 172.21.15.12:0/2397421900 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55952ad98210 con 0x55952aef4950 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad617fa700 1 -- 172.21.15.12:0/2397421900 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fad68031900 con 0x55952aef4950 2024-05-09T00:10:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad8be56b80 10 client.22107 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad8be56b80 10 client.22107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad8be56b80 20 client.22107 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad8be56b80 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55952af0d0e0 0x55952af0f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad8be56b80 4 client.22107 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:10:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.080+0000 7fad8be56b80 1 -- 172.21.15.12:0/2397421900 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55952aebe7b0 con 0x55952af0d0e0 2024-05-09T00:10:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.081+0000 7fad7da38700 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55952af0d0e0 0x55952af0f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.081+0000 7fad617fa700 10 client.22107 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.081+0000 7fad7da38700 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55952af0d0e0 0x55952af0f500 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5cd704aaddca4f3 server_cookie=aea5ec85e401b112 in_seq=0 out_seq=0 2024-05-09T00:10:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad617fa700 1 -- 172.21.15.12:0/2397421900 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55952aebe7b0 con 0x55952af0d0e0 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad617fa700 10 client.22107 handle_command_reply: tid=0 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 client.22107 shutdown 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 -- 172.21.15.12:0/2397421900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef4950 msgr2=0x55952aef8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef4950 0x55952aef8ad0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fad68011790 tx=0x7fad680047c0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 -- 172.21.15.12:0/2397421900 shutdown_connections 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef5270 0x55952aef9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55952af0d0e0 0x55952af0f500 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aeef480 0x55952aef8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 --2- 172.21.15.12:0/2397421900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef4950 0x55952aef8ad0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad617fa700 0 client.22107 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 -- 172.21.15.12:0/2397421900 >> 172.21.15.12:0/2397421900 conn(0x55952ae4d1b0 msgr2=0x55952aef34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 -- 172.21.15.12:0/2397421900 shutdown_connections 2024-05-09T00:10:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 1 -- 172.21.15.12:0/2397421900 wait complete. 2024-05-09T00:10:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.085+0000 7fad8be56b80 20 client.22107 trim_cache size 0 max 16384 2024-05-09T00:10:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.104+0000 7fad8be56b80 1 Processor -- start 2024-05-09T00:10:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad8be56b80 1 -- start start 2024-05-09T00:10:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aeef480 0x55952af08aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef4950 0x55952aee9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aeeb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad8be56b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952aeeba10 con 0x55952aef5270 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad8be56b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952aeedec0 con 0x55952aeef480 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad8be56b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952aef4d30 con 0x55952aef4950 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad7e239700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aeeb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad7d237700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef4950 0x55952aee9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad7e239700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aeeb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49230/0 (socket says 172.21.15.12:49230) 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad7d237700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef4950 0x55952aee9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:60432/0 (socket says 172.21.15.12:60432) 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad7e239700 1 -- 172.21.15.12:0/3502278491 learned_addr learned my addr 172.21.15.12:0/3502278491 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad7e239700 1 -- 172.21.15.12:0/3502278491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef4950 msgr2=0x55952aee9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad7e239700 1 --2- 172.21.15.12:0/3502278491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef4950 0x55952aee9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad7e239700 1 -- 172.21.15.12:0/3502278491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aeef480 msgr2=0x55952af08aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad7e239700 1 --2- 172.21.15.12:0/3502278491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aeef480 0x55952af08aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.105+0000 7fad7e239700 1 -- 172.21.15.12:0/3502278491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55952ac6cb00 con 0x55952aef5270 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.106+0000 7fad7e239700 1 --2- 172.21.15.12:0/3502278491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aeeb4d0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fad70009fd0 tx=0x7fad70003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8aab0e6f9e0c682 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.106+0000 7fad6cda4700 1 -- 172.21.15.12:0/3502278491 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70007ea0 con 0x55952aef5270 2024-05-09T00:10:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.106+0000 7fad6cda4700 1 -- 172.21.15.12:0/3502278491 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad70003a00 con 0x55952aef5270 2024-05-09T00:10:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.106+0000 7fad6cda4700 1 -- 172.21.15.12:0/3502278491 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000e040 con 0x55952aef5270 2024-05-09T00:10:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 -- 172.21.15.12:0/3502278491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 msgr2=0x55952aeeb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 --2- 172.21.15.12:0/3502278491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aeeb4d0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fad70009fd0 tx=0x7fad70003f40 comp rx=0 tx=0).stop 2024-05-09T00:10:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 -- 172.21.15.12:0/3502278491 shutdown_connections 2024-05-09T00:10:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 --2- 172.21.15.12:0/3502278491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aeeb4d0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 --2- 172.21.15.12:0/3502278491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aeef480 0x55952af08aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 --2- 172.21.15.12:0/3502278491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aef4950 0x55952aee9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 -- 172.21.15.12:0/3502278491 >> 172.21.15.12:0/3502278491 conn(0x55952ae4d1b0 msgr2=0x55952aef6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 -- 172.21.15.12:0/3502278491 shutdown_connections 2024-05-09T00:10:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 -- 172.21.15.12:0/3502278491 wait complete. 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 -- start start 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.107+0000 7fad8be56b80 1 Processor -- start 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aeef480 0x55952af04250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aef4950 0x55952aef23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad8be56b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aef2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad8be56b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952aef2f00 con 0x55952aef5270 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad8be56b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952aef3070 con 0x55952aef4950 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad8be56b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952aef3230 con 0x55952aeef480 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad8be56b80 10 client.0 _ll_register_callbacks cb 0x55952ade4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7e239700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aef2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7da38700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aeef480 0x55952af04250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7e239700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aef2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49238/0 (socket says 172.21.15.12:49238) 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7d237700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aef4950 0x55952aef23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7e239700 1 -- 172.21.15.12:0/686233832 learned_addr learned my addr 172.21.15.12:0/686233832 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7da38700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aeef480 0x55952af04250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:60446/0 (socket says 172.21.15.12:60446) 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7e239700 1 -- 172.21.15.12:0/686233832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aeef480 msgr2=0x55952af04250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7e239700 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aeef480 0x55952af04250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7e239700 1 -- 172.21.15.12:0/686233832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aef4950 msgr2=0x55952aef23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7e239700 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aef4950 0x55952aef23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7e239700 1 -- 172.21.15.12:0/686233832 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55952acc0350 con 0x55952aef5270 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad7e239700 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aef2930 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fad700093a0 tx=0x7fad7000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc2168f5faaa1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.108+0000 7fad617fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad617fa700 1 -- 172.21.15.12:0/686233832 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000e070 con 0x55952aef5270 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad617fa700 1 -- 172.21.15.12:0/686233832 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fad700074a0 con 0x55952aef5270 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad8be56b80 1 -- 172.21.15.12:0/686233832 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad70007d30 con 0x55952aef5270 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad617fa700 1 -- 172.21.15.12:0/686233832 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad70007ea0 con 0x55952aef5270 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad8be56b80 10 client.22113 fetch_fsmap learned FSMap version 83 2024-05-09T00:10:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad8be56b80 1 -- 172.21.15.12:0/686233832 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55952ae05500 con 0x55952aef5270 2024-05-09T00:10:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad617fa700 1 -- 172.21.15.12:0/686233832 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad70007d30 con 0x55952aef5270 2024-05-09T00:10:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad617fa700 1 -- 172.21.15.12:0/686233832 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fad70019030 con 0x55952aef5270 2024-05-09T00:10:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad8be56b80 10 client.22113 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:10:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad8be56b80 10 client.22113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:10:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad8be56b80 20 client.22113 populate_metadata read hostname 'smithi012' 2024-05-09T00:10:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad8be56b80 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55952af63830 0x55952af65c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad8be56b80 4 client.22113 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:10:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.109+0000 7fad8be56b80 1 -- 172.21.15.12:0/686233832 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55952ae37e60 con 0x55952af63830 2024-05-09T00:10:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.113+0000 7fad7da38700 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55952af63830 0x55952af65c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.113+0000 7fad7da38700 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55952af63830 0x55952af65c50 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95e00180f3b09db server_cookie=81d7ce7ef711d0a in_seq=0 out_seq=0 2024-05-09T00:10:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.113+0000 7fad617fa700 10 client.22113 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:54.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:53 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:10:54.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:53 smithi012 ceph-mon[42503]: pgmap v788: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 691 op/s 2024-05-09T00:10:54.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:53 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:10:54.268 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:53 smithi204 ceph-mon[47373]: pgmap v788: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 691 op/s 2024-05-09T00:10:54.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:53 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:10:54.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:53 smithi160 ceph-mon[46152]: pgmap v788: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 691 op/s 2024-05-09T00:10:54.328 INFO:tasks.workunit.client.0.smithi012.stdout: 10 365939 28.43 MB/sec execute 529 sec latency 474.951 ms 2024-05-09T00:10:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.799+0000 7fad617fa700 1 -- 172.21.15.12:0/686233832 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55952ae37e60 con 0x55952af63830 2024-05-09T00:10:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.799+0000 7fad617fa700 10 client.22113 handle_command_reply: tid=0 2024-05-09T00:10:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.799+0000 7fad8be56b80 1 client.22113 shutdown 2024-05-09T00:10:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.800+0000 7fad8be56b80 1 -- 172.21.15.12:0/686233832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 msgr2=0x55952aef2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.800+0000 7fad8be56b80 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aef2930 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fad700093a0 tx=0x7fad7000bce0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.801+0000 7fad8be56b80 1 -- 172.21.15.12:0/686233832 shutdown_connections 2024-05-09T00:10:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.801+0000 7fad8be56b80 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55952aef5270 0x55952aef2930 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.801+0000 7fad8be56b80 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55952aeef480 0x55952af04250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.801+0000 7fad8be56b80 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55952af63830 0x55952af65c50 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.801+0000 7fad8be56b80 1 --2- 172.21.15.12:0/686233832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55952aef4950 0x55952aef23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.801+0000 7fad617fa700 0 client.22113 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:10:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.801+0000 7fad8be56b80 1 -- 172.21.15.12:0/686233832 >> 172.21.15.12:0/686233832 conn(0x55952ae4d1b0 msgr2=0x55952aef6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.801+0000 7fad8be56b80 1 -- 172.21.15.12:0/686233832 shutdown_connections 2024-05-09T00:10:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.801+0000 7fad8be56b80 1 -- 172.21.15.12:0/686233832 wait complete. 2024-05-09T00:10:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.801+0000 7fad8be56b80 20 client.22113 trim_cache size 0 max 16384 2024-05-09T00:10:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:10:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:10:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:10:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:10:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.802+0000 7fad7fc9c700 1 -- 172.21.15.12:0/970994547 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad640a1290 msgr2=0x7fad640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.802+0000 7fad7fc9c700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad640a1290 0x7fad640a3750 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fad781988e0 tx=0x7fad6800b410 comp rx=0 tx=0).stop 2024-05-09T00:10:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.802+0000 7fad7fc9c700 1 -- 172.21.15.12:0/970994547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad78100280 msgr2=0x7fad78197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.802+0000 7fad7fc9c700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad78100280 0x7fad78197390 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e9a0 tx=0x7fad7400ecb0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.803+0000 7fad7fc9c700 1 -- 172.21.15.12:0/970994547 shutdown_connections 2024-05-09T00:10:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.803+0000 7fad7fc9c700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fad640a1290 0x7fad640a3750 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.803+0000 7fad7fc9c700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad78105f20 0x7fad7819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.803+0000 7fad7fc9c700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad78100280 0x7fad78197390 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.803+0000 7fad7fc9c700 1 --2- 172.21.15.12:0/970994547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad78100c40 0x7fad781978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.803+0000 7fad7fc9c700 1 -- 172.21.15.12:0/970994547 >> 172.21.15.12:0/970994547 conn(0x7fad78076bc0 msgr2=0x7fad780ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.803+0000 7fad7fc9c700 1 -- 172.21.15.12:0/970994547 shutdown_connections 2024-05-09T00:10:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:10:54.803+0000 7fad7fc9c700 1 -- 172.21.15.12:0/970994547 wait complete. 2024-05-09T00:10:55.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:54 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:10:55.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:54 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:10:55.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:54 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:10:55.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:54 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:10:55.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:54 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:10:55.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:54 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:10:55.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 366247 28.41 MB/sec execute 530 sec latency 402.653 ms 2024-05-09T00:10:56.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:55 smithi012 ceph-mon[42503]: pgmap v789: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.2 MiB/s wr, 459 op/s 2024-05-09T00:10:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:55 smithi204 ceph-mon[47373]: pgmap v789: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.2 MiB/s wr, 459 op/s 2024-05-09T00:10:56.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:55 smithi160 ceph-mon[46152]: pgmap v789: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.2 MiB/s wr, 459 op/s 2024-05-09T00:10:56.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 366802 28.42 MB/sec execute 531 sec latency 95.389 ms 2024-05-09T00:10:57.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 367290 28.41 MB/sec execute 532 sec latency 98.408 ms 2024-05-09T00:10:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:57 smithi012 ceph-mon[42503]: pgmap v790: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 591 op/s 2024-05-09T00:10:58.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:57 smithi204 ceph-mon[47373]: pgmap v790: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 591 op/s 2024-05-09T00:10:58.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:57 smithi160 ceph-mon[46152]: pgmap v790: 97 pgs: 97 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 591 op/s 2024-05-09T00:10:58.322 INFO:tasks.workunit.client.0.smithi012.stdout: 10 367662 28.40 MB/sec execute 533 sec latency 124.577 ms 2024-05-09T00:10:59.322 INFO:tasks.workunit.client.0.smithi012.stdout: 10 367944 28.37 MB/sec execute 534 sec latency 316.893 ms 2024-05-09T00:11:00.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:10:59 smithi012 ceph-mon[42503]: pgmap v791: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 682 op/s 2024-05-09T00:11:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:10:59 smithi204 ceph-mon[47373]: pgmap v791: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 682 op/s 2024-05-09T00:11:00.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:10:59 smithi160 ceph-mon[46152]: pgmap v791: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 682 op/s 2024-05-09T00:11:00.322 INFO:tasks.workunit.client.0.smithi012.stdout: 10 368301 28.35 MB/sec execute 535 sec latency 166.656 ms 2024-05-09T00:11:01.322 INFO:tasks.workunit.client.0.smithi012.stdout: 10 368466 28.30 MB/sec execute 536 sec latency 770.342 ms 2024-05-09T00:11:02.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:01 smithi012 ceph-mon[42503]: pgmap v792: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 10 MiB/s wr, 580 op/s 2024-05-09T00:11:02.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:01 smithi204 ceph-mon[47373]: pgmap v792: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 10 MiB/s wr, 580 op/s 2024-05-09T00:11:02.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:01 smithi160 ceph-mon[46152]: pgmap v792: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 10 MiB/s wr, 580 op/s 2024-05-09T00:11:02.322 INFO:tasks.workunit.client.0.smithi012.stdout: 10 368819 28.27 MB/sec execute 537 sec latency 194.457 ms 2024-05-09T00:11:03.322 INFO:tasks.workunit.client.0.smithi012.stdout: 10 368910 28.23 MB/sec execute 538 sec latency 221.034 ms 2024-05-09T00:11:04.322 INFO:tasks.workunit.client.0.smithi012.stdout: 10 368937 28.18 MB/sec execute 539 sec latency 919.991 ms 2024-05-09T00:11:05.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:04 smithi160 ceph-mon[46152]: osdmap e136: 12 total, 12 up, 12 in 2024-05-09T00:11:05.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:04 smithi160 ceph-mon[46152]: pgmap v794: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.7 MiB/s wr, 620 op/s 2024-05-09T00:11:05.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:04 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:11:05.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:04 smithi204 ceph-mon[47373]: osdmap e136: 12 total, 12 up, 12 in 2024-05-09T00:11:05.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:04 smithi204 ceph-mon[47373]: pgmap v794: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.7 MiB/s wr, 620 op/s 2024-05-09T00:11:05.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:04 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:11:05.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:04 smithi012 ceph-mon[42503]: osdmap e136: 12 total, 12 up, 12 in 2024-05-09T00:11:05.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:04 smithi012 ceph-mon[42503]: pgmap v794: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.7 MiB/s wr, 620 op/s 2024-05-09T00:11:05.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:04 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:11:05.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 368954 28.13 MB/sec execute 540 sec latency 1025.576 ms 2024-05-09T00:11:06.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:05 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:05 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:05 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:05 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:05 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:05 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:05 smithi204 ceph-mon[47373]: pgmap v795: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.7 MiB/s wr, 620 op/s 2024-05-09T00:11:06.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:05 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:05 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:05 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:05 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:05 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:05 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:05 smithi160 ceph-mon[46152]: pgmap v795: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.7 MiB/s wr, 620 op/s 2024-05-09T00:11:06.322 INFO:tasks.workunit.client.0.smithi012.stdout: 10 369011 28.08 MB/sec execute 541 sec latency 1067.668 ms 2024-05-09T00:11:06.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:05 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:05 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:05 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:05 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:05 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:05 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:06.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:05 smithi012 ceph-mon[42503]: pgmap v795: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.7 MiB/s wr, 620 op/s 2024-05-09T00:11:07.322 INFO:tasks.workunit.client.0.smithi012.stdout: 10 369127 28.05 MB/sec execute 542 sec latency 1229.723 ms 2024-05-09T00:11:07.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:07 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:11:07.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:07 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:11:07.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:07 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:07 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:11:07.560 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:07 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:11:07.560 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:07 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:07.561 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:07 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:11:07.561 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:07 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:11:07.561 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:07 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:11:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.987+0000 7f140bf9c700 1 -- 172.21.15.12:0/2623576858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1404105130 msgr2=0x7f140410ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.987+0000 7f140bf9c700 1 --2- 172.21.15.12:0/2623576858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1404105130 0x7f140410ccb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00b0a0 tx=0x7f13fc00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:11:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.987+0000 7f140bf9c700 1 -- 172.21.15.12:0/2623576858 shutdown_connections 2024-05-09T00:11:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.987+0000 7f140bf9c700 1 --2- 172.21.15.12:0/2623576858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1404105130 0x7f140410ccb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.987+0000 7f140bf9c700 1 --2- 172.21.15.12:0/2623576858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1404100010 0x7f14041003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.987+0000 7f140bf9c700 1 --2- 172.21.15.12:0/2623576858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14041009c0 0x7f1404104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.987+0000 7f140bf9c700 1 -- 172.21.15.12:0/2623576858 >> 172.21.15.12:0/2623576858 conn(0x7f1404076f40 msgr2=0x7f1404077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.990+0000 7f140bf9c700 1 -- 172.21.15.12:0/2623576858 shutdown_connections 2024-05-09T00:11:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.991+0000 7f140bf9c700 1 -- 172.21.15.12:0/2623576858 wait complete. 2024-05-09T00:11:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.991+0000 7f140bf9c700 1 Processor -- start 2024-05-09T00:11:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.991+0000 7f140bf9c700 1 -- start start 2024-05-09T00:11:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.992+0000 7f140bf9c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1404100010 0x7f140410cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.992+0000 7f140bf9c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1404105130 0x7f1404108cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.992+0000 7f140bf9c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1404109360 0x7f1404072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.992+0000 7f140bf9c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1404109810 con 0x7f1404105130 2024-05-09T00:11:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.992+0000 7f140bf9c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1404109950 con 0x7f1404100010 2024-05-09T00:11:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.992+0000 7f140bf9c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1404109b10 con 0x7f1404109360 2024-05-09T00:11:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.992+0000 7f1409537700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1404105130 0x7f1404108cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.992+0000 7f1409537700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1404105130 0x7f1404108cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59838/0 (socket says 172.21.15.12:59838) 2024-05-09T00:11:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.992+0000 7f1409537700 1 -- 172.21.15.12:0/102392799 learned_addr learned my addr 172.21.15.12:0/102392799 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.992+0000 7f1409d38700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1404100010 0x7f140410cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.993+0000 7f1409537700 1 -- 172.21.15.12:0/102392799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1404109360 msgr2=0x7f1404072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.993+0000 7f1409537700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1404109360 0x7f1404072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.993+0000 7f1409537700 1 -- 172.21.15.12:0/102392799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1404100010 msgr2=0x7f140410cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.993+0000 7f1409537700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1404100010 0x7f140410cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.993+0000 7f1409537700 1 -- 172.21.15.12:0/102392799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc009d00 con 0x7f1404105130 2024-05-09T00:11:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.993+0000 7f1409537700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1404105130 0x7f1404108cc0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f13f400b500 tx=0x7f13f400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.996+0000 7f13faffd700 1 -- 172.21.15.12:0/102392799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4011830 con 0x7f1404105130 2024-05-09T00:11:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.996+0000 7f13faffd700 1 -- 172.21.15.12:0/102392799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f4011e50 con 0x7f1404105130 2024-05-09T00:11:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.996+0000 7f13faffd700 1 -- 172.21.15.12:0/102392799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f400f2e0 con 0x7f1404105130 2024-05-09T00:11:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.996+0000 7f140bf9c700 1 -- 172.21.15.12:0/102392799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14041147e0 con 0x7f1404105130 2024-05-09T00:11:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:07.996+0000 7f140bf9c700 1 -- 172.21.15.12:0/102392799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1404114ca0 con 0x7f1404105130 2024-05-09T00:11:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.000+0000 7f13faffd700 1 -- 172.21.15.12:0/102392799 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f13f400f440 con 0x7f1404105130 2024-05-09T00:11:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.001+0000 7f13faffd700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f13f00a1390 0x7f13f00a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.001+0000 7f13faffd700 1 -- 172.21.15.12:0/102392799 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f13f40cd700 con 0x7f1404105130 2024-05-09T00:11:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.002+0000 7f1418156b80 1 Processor -- start 2024-05-09T00:11:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.002+0000 7f1418156b80 1 -- start start 2024-05-09T00:11:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.002+0000 7f1418156b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8b950 0x56156ea8bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.002+0000 7f1418156b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea85f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.002+0000 7f1418156b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 0x56156ea868a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.002+0000 7f1418156b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156e9dd0f0 con 0x56156ea8c270 2024-05-09T00:11:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.002+0000 7f1418156b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156e9daab0 con 0x56156ea8b950 2024-05-09T00:11:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.002+0000 7f1418156b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156e9d63a0 con 0x56156ea86480 2024-05-09T00:11:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.003+0000 7f140a539700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 0x56156ea868a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.003+0000 7f140a539700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 0x56156ea868a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39644/0 (socket says 172.21.15.12:39644) 2024-05-09T00:11:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.003+0000 7f140a539700 1 -- 172.21.15.12:0/2823996133 learned_addr learned my addr 172.21.15.12:0/2823996133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.003+0000 7f140a539700 1 -- 172.21.15.12:0/2823996133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8b950 msgr2=0x56156ea8bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.003+0000 7f140a539700 1 --2- 172.21.15.12:0/2823996133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8b950 0x56156ea8bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.003+0000 7f140a539700 1 -- 172.21.15.12:0/2823996133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 msgr2=0x56156ea85f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.003+0000 7f140a539700 1 --2- 172.21.15.12:0/2823996133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea85f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.004+0000 7f140a539700 1 -- 172.21.15.12:0/2823996133 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56156e857350 con 0x56156ea86480 2024-05-09T00:11:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.004+0000 7f140a539700 1 --2- 172.21.15.12:0/2823996133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 0x56156ea868a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005b00 tx=0x7f13fc005b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1fb7031bd47fac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.006+0000 7f1409d38700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f13f00a1390 0x7f13f00a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.006+0000 7f13effff700 1 -- 172.21.15.12:0/2823996133 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc00e040 con 0x56156ea86480 2024-05-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.006+0000 7f13effff700 1 -- 172.21.15.12:0/2823996133 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13fc009370 con 0x56156ea86480 2024-05-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.006+0000 7f13effff700 1 -- 172.21.15.12:0/2823996133 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc005250 con 0x56156ea86480 2024-05-09T00:11:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.009+0000 7f1418156b80 1 -- 172.21.15.12:0/2823996133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 msgr2=0x56156ea868a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.009+0000 7f1418156b80 1 --2- 172.21.15.12:0/2823996133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 0x56156ea868a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005b00 tx=0x7f13fc005b30 comp rx=0 tx=0).stop 2024-05-09T00:11:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.014+0000 7f1418156b80 1 -- 172.21.15.12:0/2823996133 shutdown_connections 2024-05-09T00:11:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.014+0000 7f1418156b80 1 --2- 172.21.15.12:0/2823996133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 0x56156ea868a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.014+0000 7f1418156b80 1 --2- 172.21.15.12:0/2823996133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8b950 0x56156ea8bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.014+0000 7f1418156b80 1 --2- 172.21.15.12:0/2823996133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea85f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.014+0000 7f1418156b80 1 -- 172.21.15.12:0/2823996133 >> 172.21.15.12:0/2823996133 conn(0x56156e9e41b0 msgr2=0x56156ea80840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.014+0000 7f1418156b80 1 -- 172.21.15.12:0/2823996133 shutdown_connections 2024-05-09T00:11:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.014+0000 7f1418156b80 1 -- 172.21.15.12:0/2823996133 wait complete. 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.014+0000 7f1418156b80 1 -- start start 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.015+0000 7f1418156b80 1 Processor -- start 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.015+0000 7f1418156b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea86480 0x56156ea8f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.015+0000 7f1418156b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 0x56156ea8fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.015+0000 7f1418156b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8c270 0x56156ea90050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.015+0000 7f1418156b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156e972500 con 0x56156ea86480 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.015+0000 7f1418156b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156e9721a0 con 0x56156ea8c270 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.015+0000 7f1418156b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156e859640 con 0x56156ea8b950 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.015+0000 7f1418156b80 10 client.0 _ll_register_callbacks cb 0x56156e97bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.016+0000 7f1409537700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 0x56156ea8fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.016+0000 7f1409537700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 0x56156ea8fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:39654/0 (socket says 172.21.15.12:39654) 2024-05-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.016+0000 7f1409537700 1 -- 172.21.15.12:0/3048054483 learned_addr learned my addr 172.21.15.12:0/3048054483 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.017+0000 7f1409537700 1 -- 172.21.15.12:0/3048054483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8c270 msgr2=0x56156ea90050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.017+0000 7f1409537700 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8c270 0x56156ea90050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.017+0000 7f1409537700 1 -- 172.21.15.12:0/3048054483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea86480 msgr2=0x56156ea8f5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.017+0000 7f1409537700 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea86480 0x56156ea8f5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.017+0000 7f1409537700 1 -- 172.21.15.12:0/3048054483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140410d520 con 0x56156ea8b950 2024-05-09T00:11:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.017+0000 7f1409537700 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 0x56156ea8fb10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f13f40caf50 tx=0x7f13f409afb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4669256228b1518 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.018+0000 7f13ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.018+0000 7f1409d38700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f13f00a1390 0x7f13f00a3850 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1400006ea0 tx=0x7f1400008040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.018+0000 7f13ed7fa700 1 -- 172.21.15.12:0/3048054483 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f40b8ea0 con 0x56156ea8b950 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.018+0000 7f13ed7fa700 1 -- 172.21.15.12:0/3048054483 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f13f4018520 con 0x56156ea8b950 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.018+0000 7f1418156b80 1 -- 172.21.15.12:0/3048054483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56156e92f210 con 0x56156ea8b950 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f13ed7fa700 1 -- 172.21.15.12:0/3048054483 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13f40b3270 con 0x56156ea8b950 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f1418156b80 10 client.31780 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f1418156b80 1 -- 172.21.15.12:0/3048054483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x56156e803b00 con 0x56156ea8b950 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f13ed7fa700 1 -- 172.21.15.12:0/3048054483 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56156e92f210 con 0x56156ea8b950 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f13ed7fa700 1 -- 172.21.15.12:0/3048054483 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f13f40a28a0 con 0x56156ea8b950 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f1418156b80 10 client.31780 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f1418156b80 10 client.31780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f1418156b80 20 client.31780 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f1418156b80 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156ea9fcf0 0x56156eaa2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f1418156b80 4 client.31780 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.019+0000 7f1418156b80 1 -- 172.21.15.12:0/3048054483 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56156ea557b0 con 0x56156ea9fcf0 2024-05-09T00:11:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.021+0000 7f1409d38700 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156ea9fcf0 0x56156eaa2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.021+0000 7f1409d38700 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156ea9fcf0 0x56156eaa2110 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e7735a42ef1345 server_cookie=314059dfaa980eec in_seq=0 out_seq=0 2024-05-09T00:11:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.021+0000 7f13ed7fa700 10 client.31780 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.026+0000 7f13ed7fa700 1 -- 172.21.15.12:0/3048054483 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56156ea557b0 con 0x56156ea9fcf0 2024-05-09T00:11:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.026+0000 7f13ed7fa700 10 client.31780 handle_command_reply: tid=0 2024-05-09T00:11:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.026+0000 7f1418156b80 1 client.31780 shutdown 2024-05-09T00:11:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.037+0000 7f1418156b80 1 -- 172.21.15.12:0/3048054483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 msgr2=0x56156ea8fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.037+0000 7f1418156b80 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 0x56156ea8fb10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f13f40caf50 tx=0x7f13f409afb0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.042+0000 7f1418156b80 1 -- 172.21.15.12:0/3048054483 shutdown_connections 2024-05-09T00:11:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.042+0000 7f1418156b80 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156ea9fcf0 0x56156eaa2110 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.042+0000 7f1418156b80 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8c270 0x56156ea90050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.042+0000 7f1418156b80 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea86480 0x56156ea8f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.042+0000 7f1418156b80 1 --2- 172.21.15.12:0/3048054483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 0x56156ea8fb10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.042+0000 7f13ed7fa700 0 client.31780 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.042+0000 7f1418156b80 1 -- 172.21.15.12:0/3048054483 >> 172.21.15.12:0/3048054483 conn(0x56156e9e41b0 msgr2=0x56156ea8a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.042+0000 7f1418156b80 1 -- 172.21.15.12:0/3048054483 shutdown_connections 2024-05-09T00:11:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.042+0000 7f1418156b80 1 -- 172.21.15.12:0/3048054483 wait complete. 2024-05-09T00:11:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.042+0000 7f1418156b80 20 client.31780 trim_cache size 0 max 16384 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.067+0000 7f1418156b80 1 Processor -- start 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.067+0000 7f1418156b80 1 -- start start 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f1418156b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea86480 0x56156ea9bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f1418156b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 0x56156ea80910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f1418156b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea86860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f1418156b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156ea82290 con 0x56156ea8c270 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f1418156b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156eaa7320 con 0x56156ea86480 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f1418156b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156ea92810 con 0x56156ea8b950 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f140a539700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea86860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f140a539700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea86860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59850/0 (socket says 172.21.15.12:59850) 2024-05-09T00:11:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f140a539700 1 -- 172.21.15.12:0/347030127 learned_addr learned my addr 172.21.15.12:0/347030127 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f140a539700 1 -- 172.21.15.12:0/347030127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 msgr2=0x56156ea80910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f140a539700 1 --2- 172.21.15.12:0/347030127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 0x56156ea80910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f140a539700 1 -- 172.21.15.12:0/347030127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea86480 msgr2=0x56156ea9bd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f140a539700 1 --2- 172.21.15.12:0/347030127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea86480 0x56156ea9bd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.068+0000 7f140a539700 1 -- 172.21.15.12:0/347030127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56156e99c500 con 0x56156ea8c270 2024-05-09T00:11:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.069+0000 7f140a539700 1 --2- 172.21.15.12:0/347030127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea86860 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009180 tx=0x7f13fc009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800163ea3689ee7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.073+0000 7f13f8ff9700 1 -- 172.21.15.12:0/347030127 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc00e040 con 0x56156ea8c270 2024-05-09T00:11:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.073+0000 7f13f8ff9700 1 -- 172.21.15.12:0/347030127 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13fc0074a0 con 0x56156ea8c270 2024-05-09T00:11:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.073+0000 7f13f8ff9700 1 -- 172.21.15.12:0/347030127 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0053b0 con 0x56156ea8c270 2024-05-09T00:11:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.075+0000 7f1418156b80 1 -- 172.21.15.12:0/347030127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 msgr2=0x56156ea86860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.075+0000 7f1418156b80 1 --2- 172.21.15.12:0/347030127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea86860 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009180 tx=0x7f13fc009a00 comp rx=0 tx=0).stop 2024-05-09T00:11:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.075+0000 7f1418156b80 1 -- 172.21.15.12:0/347030127 shutdown_connections 2024-05-09T00:11:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.075+0000 7f1418156b80 1 --2- 172.21.15.12:0/347030127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea86860 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.075+0000 7f1418156b80 1 --2- 172.21.15.12:0/347030127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea86480 0x56156ea9bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.075+0000 7f1418156b80 1 --2- 172.21.15.12:0/347030127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea8b950 0x56156ea80910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.075+0000 7f1418156b80 1 -- 172.21.15.12:0/347030127 >> 172.21.15.12:0/347030127 conn(0x56156e9e41b0 msgr2=0x56156ea8dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.077+0000 7f1418156b80 1 -- 172.21.15.12:0/347030127 shutdown_connections 2024-05-09T00:11:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.077+0000 7f1418156b80 1 -- 172.21.15.12:0/347030127 wait complete. 2024-05-09T00:11:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.077+0000 7f1418156b80 1 -- start start 2024-05-09T00:11:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.077+0000 7f1418156b80 1 Processor -- start 2024-05-09T00:11:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.078+0000 7f1418156b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 0x56156ea970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.078+0000 7f1418156b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8b950 0x56156ea8a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.078+0000 7f1418156b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea8ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.078+0000 7f1418156b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156ea89380 con 0x56156ea8c270 2024-05-09T00:11:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.078+0000 7f1418156b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156ea894f0 con 0x56156ea8b950 2024-05-09T00:11:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.078+0000 7f1418156b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156ea896b0 con 0x56156ea86480 2024-05-09T00:11:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.078+0000 7f1418156b80 10 client.0 _ll_register_callbacks cb 0x56156e97bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.078+0000 7f140a539700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea8ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.078+0000 7f140a539700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea8ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59862/0 (socket says 172.21.15.12:59862) 2024-05-09T00:11:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.078+0000 7f140a539700 1 -- 172.21.15.12:0/359946129 learned_addr learned my addr 172.21.15.12:0/359946129 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.079+0000 7f140a539700 1 -- 172.21.15.12:0/359946129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 msgr2=0x56156ea970f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.079+0000 7f140a539700 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 0x56156ea970f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.079+0000 7f140a539700 1 -- 172.21.15.12:0/359946129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8b950 msgr2=0x56156ea8a400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.079+0000 7f140a539700 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8b950 0x56156ea8a400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.079+0000 7f140a539700 1 -- 172.21.15.12:0/359946129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56156e857350 con 0x56156ea8c270 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.079+0000 7f140a539700 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea8ab40 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00b830 tx=0x7f13fc0078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513dfac1b33cda5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.079+0000 7f13ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.080+0000 7f13ed7fa700 1 -- 172.21.15.12:0/359946129 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc00e040 con 0x56156ea8c270 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.080+0000 7f13ed7fa700 1 -- 172.21.15.12:0/359946129 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f13fc005440 con 0x56156ea8c270 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.080+0000 7f1418156b80 1 -- 172.21.15.12:0/359946129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13fc009370 con 0x56156ea8c270 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.080+0000 7f13ed7fa700 1 -- 172.21.15.12:0/359946129 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13fc005600 con 0x56156ea8c270 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.080+0000 7f1418156b80 10 client.22122 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.080+0000 7f1418156b80 1 -- 172.21.15.12:0/359946129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x56156e85bc70 con 0x56156ea8c270 2024-05-09T00:11:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.081+0000 7f13ed7fa700 1 -- 172.21.15.12:0/359946129 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13fc009370 con 0x56156ea8c270 2024-05-09T00:11:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.081+0000 7f13ed7fa700 1 -- 172.21.15.12:0/359946129 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f13fc019070 con 0x56156ea8c270 2024-05-09T00:11:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.081+0000 7f1418156b80 10 client.22122 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.081+0000 7f1418156b80 10 client.22122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.081+0000 7f1418156b80 20 client.22122 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.081+0000 7f1418156b80 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156eaf6700 0x56156eaf8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.081+0000 7f1418156b80 4 client.22122 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:11:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.081+0000 7f1418156b80 1 -- 172.21.15.12:0/359946129 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56156e9cee60 con 0x56156eaf6700 2024-05-09T00:11:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.084+0000 7f1409d38700 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156eaf6700 0x56156eaf8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.085+0000 7f1409d38700 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156eaf6700 0x56156eaf8b90 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a734c79459059b server_cookie=90a17532eda759a4 in_seq=0 out_seq=0 2024-05-09T00:11:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.085+0000 7f13ed7fa700 10 client.22122 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:08.328 INFO:tasks.workunit.client.0.smithi012.stdout: 10 369293 28.02 MB/sec execute 543 sec latency 944.971 ms 2024-05-09T00:11:08.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:08 smithi012 ceph-mon[42503]: pgmap v796: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 9.1 MiB/s wr, 590 op/s 2024-05-09T00:11:08.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:08 smithi204 ceph-mon[47373]: pgmap v796: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 9.1 MiB/s wr, 590 op/s 2024-05-09T00:11:08.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:08 smithi160 ceph-mon[46152]: pgmap v796: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 9.1 MiB/s wr, 590 op/s 2024-05-09T00:11:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.758+0000 7f13ed7fa700 1 -- 172.21.15.12:0/359946129 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56156e9cee60 con 0x56156eaf6700 2024-05-09T00:11:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.758+0000 7f13ed7fa700 10 client.22122 handle_command_reply: tid=0 2024-05-09T00:11:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.758+0000 7f1418156b80 1 client.22122 shutdown 2024-05-09T00:11:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.759+0000 7f1418156b80 1 -- 172.21.15.12:0/359946129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 msgr2=0x56156ea8ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.759+0000 7f1418156b80 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea8ab40 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00b830 tx=0x7f13fc0078f0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.759+0000 7f1418156b80 1 -- 172.21.15.12:0/359946129 shutdown_connections 2024-05-09T00:11:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.759+0000 7f1418156b80 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156ea8c270 0x56156ea8ab40 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.759+0000 7f1418156b80 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56156ea86480 0x56156ea970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.759+0000 7f1418156b80 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56156eaf6700 0x56156eaf8b90 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.759+0000 7f1418156b80 1 --2- 172.21.15.12:0/359946129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56156ea8b950 0x56156ea8a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.759+0000 7f13ed7fa700 0 client.22122 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.759+0000 7f1418156b80 1 -- 172.21.15.12:0/359946129 >> 172.21.15.12:0/359946129 conn(0x56156e9e41b0 msgr2=0x56156ea855d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.764+0000 7f1418156b80 1 -- 172.21.15.12:0/359946129 shutdown_connections 2024-05-09T00:11:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.764+0000 7f1418156b80 1 -- 172.21.15.12:0/359946129 wait complete. 2024-05-09T00:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.769+0000 7f1418156b80 20 client.22122 trim_cache size 0 max 16384 2024-05-09T00:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:11:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.781+0000 7f140bf9c700 1 -- 172.21.15.12:0/102392799 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f13f00a1390 msgr2=0x7f13f00a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.781+0000 7f140bf9c700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f13f00a1390 0x7f13f00a3850 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1400006ea0 tx=0x7f1400008040 comp rx=0 tx=0).stop 2024-05-09T00:11:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.781+0000 7f140bf9c700 1 -- 172.21.15.12:0/102392799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1404105130 msgr2=0x7f1404108cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.781+0000 7f140bf9c700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1404105130 0x7f1404108cc0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f13f400b500 tx=0x7f13f400b8c0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.782+0000 7f140bf9c700 1 -- 172.21.15.12:0/102392799 shutdown_connections 2024-05-09T00:11:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.782+0000 7f140bf9c700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f13f00a1390 0x7f13f00a3850 secure :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1400006ea0 tx=0x7f1400008040 comp rx=0 tx=0).stop 2024-05-09T00:11:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.782+0000 7f140bf9c700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1404109360 0x7f1404072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.782+0000 7f140bf9c700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1404100010 0x7f140410cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.782+0000 7f140bf9c700 1 --2- 172.21.15.12:0/102392799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1404105130 0x7f1404108cc0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.782+0000 7f140bf9c700 1 -- 172.21.15.12:0/102392799 >> 172.21.15.12:0/102392799 conn(0x7f1404076f40 msgr2=0x7f14040fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.786+0000 7f140bf9c700 1 -- 172.21.15.12:0/102392799 shutdown_connections 2024-05-09T00:11:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:08.787+0000 7f140bf9c700 1 -- 172.21.15.12:0/102392799 wait complete. 2024-05-09T00:11:09.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 369410 27.98 MB/sec execute 544 sec latency 994.226 ms 2024-05-09T00:11:10.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:09 smithi012 ceph-mon[42503]: pgmap v797: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 533 op/s 2024-05-09T00:11:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:09 smithi204 ceph-mon[47373]: pgmap v797: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 533 op/s 2024-05-09T00:11:10.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:09 smithi160 ceph-mon[46152]: pgmap v797: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 533 op/s 2024-05-09T00:11:10.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 369617 27.96 MB/sec execute 545 sec latency 575.056 ms 2024-05-09T00:11:11.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 370006 27.94 MB/sec execute 546 sec latency 254.944 ms 2024-05-09T00:11:12.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:11 smithi012 ceph-mon[42503]: pgmap v798: 97 pgs: 17 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 52 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.4 MiB/s wr, 531 op/s 2024-05-09T00:11:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:11 smithi204 ceph-mon[47373]: pgmap v798: 97 pgs: 17 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 52 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.4 MiB/s wr, 531 op/s 2024-05-09T00:11:12.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:11 smithi160 ceph-mon[46152]: pgmap v798: 97 pgs: 17 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 52 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.4 MiB/s wr, 531 op/s 2024-05-09T00:11:12.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 370534 27.96 MB/sec execute 547 sec latency 90.603 ms 2024-05-09T00:11:13.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 370980 27.95 MB/sec execute 548 sec latency 139.218 ms 2024-05-09T00:11:14.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:13 smithi012 ceph-mon[42503]: pgmap v799: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.2 MiB/s wr, 561 op/s 2024-05-09T00:11:14.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:13 smithi204 ceph-mon[47373]: pgmap v799: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.2 MiB/s wr, 561 op/s 2024-05-09T00:11:14.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:13 smithi160 ceph-mon[46152]: pgmap v799: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.2 MiB/s wr, 561 op/s 2024-05-09T00:11:14.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 371191 27.91 MB/sec execute 549 sec latency 265.947 ms 2024-05-09T00:11:15.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 371556 27.90 MB/sec execute 550 sec latency 198.625 ms 2024-05-09T00:11:16.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:15 smithi012 ceph-mon[42503]: pgmap v800: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 462 op/s 2024-05-09T00:11:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:15 smithi204 ceph-mon[47373]: pgmap v800: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 462 op/s 2024-05-09T00:11:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:15 smithi160 ceph-mon[46152]: pgmap v800: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 462 op/s 2024-05-09T00:11:16.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 372055 27.89 MB/sec execute 551 sec latency 188.650 ms 2024-05-09T00:11:17.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 372525 27.89 MB/sec execute 552 sec latency 211.587 ms 2024-05-09T00:11:17.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:17 smithi012 ceph-mon[42503]: osdmap e137: 12 total, 12 up, 12 in 2024-05-09T00:11:17.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:17 smithi204 ceph-mon[47373]: osdmap e137: 12 total, 12 up, 12 in 2024-05-09T00:11:17.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:17 smithi160 ceph-mon[46152]: osdmap e137: 12 total, 12 up, 12 in 2024-05-09T00:11:18.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 373041 27.88 MB/sec execute 553 sec latency 103.161 ms 2024-05-09T00:11:18.390 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:11:18.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:18 smithi012 ceph-mon[42503]: pgmap v802: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.9 MiB/s wr, 514 op/s 2024-05-09T00:11:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:18 smithi204 ceph-mon[47373]: pgmap v802: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.9 MiB/s wr, 514 op/s 2024-05-09T00:11:18.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:18 smithi160 ceph-mon[46152]: pgmap v802: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.9 MiB/s wr, 514 op/s 2024-05-09T00:11:18.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.585+0000 7f7a8576c700 1 -- 172.21.15.12:0/2631895784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a80105640 msgr2=0x7f7a80105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.585+0000 7f7a8576c700 1 --2- 172.21.15.12:0/2631895784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a80105640 0x7f7a80105a20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009990 tx=0x7f7a70009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.585+0000 7f7a8576c700 1 -- 172.21.15.12:0/2631895784 shutdown_connections 2024-05-09T00:11:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.585+0000 7f7a8576c700 1 --2- 172.21.15.12:0/2631895784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a800fff40 0x7f7a8010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.585+0000 7f7a8576c700 1 --2- 172.21.15.12:0/2631895784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a80105640 0x7f7a80105a20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.585+0000 7f7a8576c700 1 --2- 172.21.15.12:0/2631895784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a800ff580 0x7f7a800ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.585+0000 7f7a8576c700 1 -- 172.21.15.12:0/2631895784 >> 172.21.15.12:0/2631895784 conn(0x7f7a800763a0 msgr2=0x7f7a800fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.585+0000 7f7a8576c700 1 -- 172.21.15.12:0/2631895784 shutdown_connections 2024-05-09T00:11:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.585+0000 7f7a8576c700 1 -- 172.21.15.12:0/2631895784 wait complete. 2024-05-09T00:11:18.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.586+0000 7f7a8576c700 1 Processor -- start 2024-05-09T00:11:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.586+0000 7f7a8576c700 1 -- start start 2024-05-09T00:11:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a8576c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a800ff580 0x7f7a8010b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a8576c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a800fff40 0x7f7a8010bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a8576c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a80105640 0x7f7a801a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a8576c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801a2840 con 0x7f7a800ff580 2024-05-09T00:11:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a8576c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801a29b0 con 0x7f7a80105640 2024-05-09T00:11:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a8576c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801a2b70 con 0x7f7a800fff40 2024-05-09T00:11:18.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a7f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a80105640 0x7f7a801a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a7f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a80105640 0x7f7a801a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42614/0 (socket says 172.21.15.12:42614) 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a7f7fe700 1 -- 172.21.15.12:0/4150829855 learned_addr learned my addr 172.21.15.12:0/4150829855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a800fff40 0x7f7a8010bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.587+0000 7f7a7effd700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a800ff580 0x7f7a8010b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a7f7fe700 1 -- 172.21.15.12:0/4150829855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a800fff40 msgr2=0x7f7a8010bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a7f7fe700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a800fff40 0x7f7a8010bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a7f7fe700 1 -- 172.21.15.12:0/4150829855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a800ff580 msgr2=0x7f7a8010b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a7f7fe700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a800ff580 0x7f7a8010b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a7f7fe700 1 -- 172.21.15.12:0/4150829855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700095f0 con 0x7f7a80105640 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a800fff40 0x7f7a8010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:18.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a7f7fe700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a80105640 0x7f7a801a2220 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7a78007ee0 tx=0x7f7a7800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:18.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a77fff700 1 -- 172.21.15.12:0/4150829855 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78017070 con 0x7f7a80105640 2024-05-09T00:11:18.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a77fff700 1 -- 172.21.15.12:0/4150829855 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a7800f040 con 0x7f7a80105640 2024-05-09T00:11:18.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a77fff700 1 -- 172.21.15.12:0/4150829855 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78005740 con 0x7f7a80105640 2024-05-09T00:11:18.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a8576c700 1 -- 172.21.15.12:0/4150829855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a801a2e50 con 0x7f7a80105640 2024-05-09T00:11:18.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.588+0000 7f7a8576c700 1 -- 172.21.15.12:0/4150829855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8006aba0 con 0x7f7a80105640 2024-05-09T00:11:18.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.593+0000 7f7a91926b80 1 Processor -- start 2024-05-09T00:11:18.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.593+0000 7f7a77fff700 1 -- 172.21.15.12:0/4150829855 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7a780058a0 con 0x7f7a80105640 2024-05-09T00:11:18.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.594+0000 7f7a91926b80 1 -- start start 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.594+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b989e00 0x56036b98a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.594+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b98a720 0x56036b984350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.594+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 0x56036b984cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a91926b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b8db150 con 0x56036b98a720 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a91926b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b8d8b10 con 0x56036b984890 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a91926b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b8d4400 con 0x56036b989e00 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a7e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b989e00 0x56036b98a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a7e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b989e00 0x56036b98a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48412/0 (socket says 172.21.15.12:48412) 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/1787994673 learned_addr learned my addr 172.21.15.12:0/1787994673 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/1787994673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 0x56036b984cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a7effd700 1 --2- 172.21.15.12:0/1787994673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b98a720 0x56036b984350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a77fff700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a6c0a1310 0x7f7a6c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a7effd700 1 -- 172.21.15.12:0/1787994673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b989e00 msgr2=0x56036b98a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:18.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a7effd700 1 --2- 172.21.15.12:0/1787994673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b989e00 0x56036b98a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a7effd700 1 -- 172.21.15.12:0/1787994673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 msgr2=0x56036b984cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a7effd700 1 --2- 172.21.15.12:0/1787994673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 0x56036b984cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.595+0000 7f7a77fff700 1 -- 172.21.15.12:0/4150829855 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7a780cd270 con 0x7f7a80105640 2024-05-09T00:11:18.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.596+0000 7f7a7f7fe700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a6c0a1310 0x7f7a6c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.596+0000 7f7a7effd700 1 -- 172.21.15.12:0/1787994673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56036b7553b0 con 0x56036b98a720 2024-05-09T00:11:18.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.596+0000 7f7a7effd700 1 --2- 172.21.15.12:0/1787994673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b98a720 0x56036b984350 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7a700098e0 tx=0x7f7a70005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218633fd5d6623d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.596+0000 7f7a7f7fe700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a6c0a1310 0x7f7a6c0a37d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7a780caa80 tx=0x7f7a7800b040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.596+0000 7f7a757fa700 1 -- 172.21.15.12:0/1787994673 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70005620 con 0x56036b98a720 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a757fa700 1 -- 172.21.15.12:0/1787994673 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a7001e040 con 0x56036b98a720 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a757fa700 1 -- 172.21.15.12:0/1787994673 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70004920 con 0x56036b98a720 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a91926b80 1 -- 172.21.15.12:0/1787994673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b98a720 msgr2=0x56036b984350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a91926b80 1 --2- 172.21.15.12:0/1787994673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b98a720 0x56036b984350 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7a700098e0 tx=0x7f7a70005f00 comp rx=0 tx=0).stop 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a91926b80 1 -- 172.21.15.12:0/1787994673 shutdown_connections 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a91926b80 1 --2- 172.21.15.12:0/1787994673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 0x56036b984cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a91926b80 1 --2- 172.21.15.12:0/1787994673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b989e00 0x56036b98a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a91926b80 1 --2- 172.21.15.12:0/1787994673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b98a720 0x56036b984350 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a91926b80 1 -- 172.21.15.12:0/1787994673 >> 172.21.15.12:0/1787994673 conn(0x56036b8e2210 msgr2=0x56036b97edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a91926b80 1 -- 172.21.15.12:0/1787994673 shutdown_connections 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.597+0000 7f7a91926b80 1 -- 172.21.15.12:0/1787994673 wait complete. 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a91926b80 1 -- start start 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a91926b80 1 Processor -- start 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b9907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b98db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b98e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a91926b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b870560 con 0x56036b984890 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a91926b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b870200 con 0x56036b989e00 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a91926b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b7576a0 con 0x56036b98a720 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a91926b80 10 client.0 _ll_register_callbacks cb 0x56036b879b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a7e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b9907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.598+0000 7f7a7e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b9907e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50232/0 (socket says 172.21.15.12:50232) 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/948191650 learned_addr learned my addr 172.21.15.12:0/948191650 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b98e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a7effd700 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b98db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/948191650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 msgr2=0x56036b98e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b98e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/948191650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 msgr2=0x56036b98db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b98db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/948191650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a80067ae0 con 0x56036b984890 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a7effd700 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b98db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b9907e0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7a68011d20 tx=0x7f7a68009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be54f98e00469d78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a667fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a667fc700 1 -- 172.21.15.12:0/948191650 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800fea0 con 0x56036b984890 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a91926b80 1 -- 172.21.15.12:0/948191650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56036b82d270 con 0x56036b984890 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.599+0000 7f7a667fc700 1 -- 172.21.15.12:0/948191650 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7a6800e6c0 con 0x56036b984890 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a667fc700 1 -- 172.21.15.12:0/948191650 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a6800e880 con 0x56036b984890 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a91926b80 10 client.22134 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a91926b80 1 -- 172.21.15.12:0/948191650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x56036b701b60 con 0x56036b984890 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a667fc700 1 -- 172.21.15.12:0/948191650 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a6802e020 con 0x56036b984890 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a667fc700 1 -- 172.21.15.12:0/948191650 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7a68029890 con 0x56036b984890 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a91926b80 10 client.22134 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a91926b80 10 client.22134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a91926b80 20 client.22134 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a91926b80 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56036b9af290 0x56036b9b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a91926b80 4 client.22134 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a91926b80 1 -- 172.21.15.12:0/948191650 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56036b953810 con 0x56036b9af290 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.600+0000 7f7a7effd700 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56036b9af290 0x56036b9b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.601+0000 7f7a7effd700 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56036b9af290 0x56036b9b16b0 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59977c6d6a7762a6 server_cookie=fadb1b3822479b49 in_seq=0 out_seq=0 2024-05-09T00:11:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.601+0000 7f7a667fc700 10 client.22134 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.605+0000 7f7a667fc700 1 -- 172.21.15.12:0/948191650 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56036b953810 con 0x56036b9af290 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.605+0000 7f7a667fc700 10 client.22134 handle_command_reply: tid=0 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.605+0000 7f7a91926b80 1 client.22134 shutdown 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.605+0000 7f7a91926b80 1 -- 172.21.15.12:0/948191650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 msgr2=0x56036b9907e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.605+0000 7f7a91926b80 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b9907e0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7a68011d20 tx=0x7f7a68009260 comp rx=0 tx=0).stop 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.606+0000 7f7a91926b80 1 -- 172.21.15.12:0/948191650 shutdown_connections 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.606+0000 7f7a91926b80 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b98e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.606+0000 7f7a91926b80 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b9907e0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.606+0000 7f7a91926b80 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56036b9af290 0x56036b9b16b0 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.606+0000 7f7a91926b80 1 --2- 172.21.15.12:0/948191650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b98db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.606+0000 7f7a91926b80 1 -- 172.21.15.12:0/948191650 >> 172.21.15.12:0/948191650 conn(0x56036b8e2210 msgr2=0x56036b97e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.606+0000 7f7a667fc700 0 client.22134 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:18.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.609+0000 7f7a91926b80 1 -- 172.21.15.12:0/948191650 shutdown_connections 2024-05-09T00:11:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.611+0000 7f7a91926b80 1 -- 172.21.15.12:0/948191650 wait complete. 2024-05-09T00:11:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.611+0000 7f7a91926b80 20 client.22134 trim_cache size 0 max 16384 2024-05-09T00:11:18.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.638+0000 7f7a91926b80 1 Processor -- start 2024-05-09T00:11:18.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.638+0000 7f7a91926b80 1 -- start start 2024-05-09T00:11:18.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 0x56036b97ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b989e00 0x56036b98c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b98c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a91926b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b9846d0 con 0x56036b989e00 2024-05-09T00:11:18.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a91926b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b9b6860 con 0x56036b984890 2024-05-09T00:11:18.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a91926b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b98c0b0 con 0x56036b98a720 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a7e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 0x56036b97ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a7e7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 0x56036b97ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42648/0 (socket says 172.21.15.12:42648) 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/3103725595 learned_addr learned my addr 172.21.15.12:0/3103725595 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/3103725595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b98c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/3103725595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 msgr2=0x56036b98c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/3103725595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b98c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/3103725595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b989e00 msgr2=0x56036b98c430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/3103725595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b989e00 0x56036b98c430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.639+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/3103725595 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56036b89a560 con 0x56036b984890 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.640+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/3103725595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 0x56036b97ed20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7a68029c70 tx=0x7f7a68001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cefd27896092e80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.640+0000 7f7a75ffb700 1 -- 172.21.15.12:0/3103725595 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68009ea0 con 0x56036b984890 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.640+0000 7f7a75ffb700 1 -- 172.21.15.12:0/3103725595 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a6802e030 con 0x56036b984890 2024-05-09T00:11:18.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.640+0000 7f7a75ffb700 1 -- 172.21.15.12:0/3103725595 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68005480 con 0x56036b984890 2024-05-09T00:11:18.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.641+0000 7f7a91926b80 1 -- 172.21.15.12:0/3103725595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 msgr2=0x56036b97ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.641+0000 7f7a91926b80 1 --2- 172.21.15.12:0/3103725595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 0x56036b97ed20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7a68029c70 tx=0x7f7a68001e70 comp rx=0 tx=0).stop 2024-05-09T00:11:18.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.641+0000 7f7a91926b80 1 -- 172.21.15.12:0/3103725595 shutdown_connections 2024-05-09T00:11:18.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.641+0000 7f7a91926b80 1 --2- 172.21.15.12:0/3103725595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b98c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.641+0000 7f7a91926b80 1 --2- 172.21.15.12:0/3103725595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b984890 0x56036b97ed20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.641+0000 7f7a91926b80 1 --2- 172.21.15.12:0/3103725595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b989e00 0x56036b98c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.641+0000 7f7a91926b80 1 -- 172.21.15.12:0/3103725595 >> 172.21.15.12:0/3103725595 conn(0x56036b8e2210 msgr2=0x56036b987ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 1 -- 172.21.15.12:0/3103725595 shutdown_connections 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 1 -- 172.21.15.12:0/3103725595 wait complete. 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 1 -- start start 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 1 Processor -- start 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b995460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b991840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b991d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b992460 con 0x56036b984890 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b9925d0 con 0x56036b989e00 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036ba05750 con 0x56036b98a720 2024-05-09T00:11:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.644+0000 7f7a91926b80 10 client.0 _ll_register_callbacks cb 0x56036b879b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b991840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b995460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b991840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42654/0 (socket says 172.21.15.12:42654) 2024-05-09T00:11:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7effd700 1 -- 172.21.15.12:0/2867512299 learned_addr learned my addr 172.21.15.12:0/2867512299 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b995460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50238/0 (socket says 172.21.15.12:50238) 2024-05-09T00:11:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b991d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/2867512299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 msgr2=0x56036b991d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b991d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/2867512299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 msgr2=0x56036b991840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b991840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7e7fc700 1 -- 172.21.15.12:0/2867512299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56036b701b60 con 0x56036b984890 2024-05-09T00:11:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7effd700 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b991840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a7e7fc700 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b995460 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7a68029bf0 tx=0x7f7a68005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e253c8f04d1daf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.645+0000 7f7a667fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:18.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.651+0000 7f7a667fc700 1 -- 172.21.15.12:0/2867512299 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680052d0 con 0x56036b984890 2024-05-09T00:11:18.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.651+0000 7f7a667fc700 1 -- 172.21.15.12:0/2867512299 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7a6802f080 con 0x56036b984890 2024-05-09T00:11:18.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.651+0000 7f7a91926b80 1 -- 172.21.15.12:0/2867512299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a7001e040 con 0x56036b984890 2024-05-09T00:11:18.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.651+0000 7f7a667fc700 1 -- 172.21.15.12:0/2867512299 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a68005430 con 0x56036b984890 2024-05-09T00:11:18.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.651+0000 7f7a91926b80 10 client.22137 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:18.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.651+0000 7f7a91926b80 1 -- 172.21.15.12:0/2867512299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x56036b759cd0 con 0x56036b984890 2024-05-09T00:11:18.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.651+0000 7f7a667fc700 1 -- 172.21.15.12:0/2867512299 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a7001e040 con 0x56036b984890 2024-05-09T00:11:18.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.652+0000 7f7a667fc700 1 -- 172.21.15.12:0/2867512299 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7a6800fa50 con 0x56036b984890 2024-05-09T00:11:18.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.653+0000 7f7a91926b80 10 client.22137 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:18.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.653+0000 7f7a91926b80 10 client.22137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:18.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.653+0000 7f7a91926b80 20 client.22137 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:18.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.653+0000 7f7a91926b80 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56036ba06a30 0x56036ba08e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.653+0000 7f7a91926b80 4 client.22137 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:11:18.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.653+0000 7f7a91926b80 1 -- 172.21.15.12:0/2867512299 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56036b8ccec0 con 0x56036ba06a30 2024-05-09T00:11:18.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.653+0000 7f7a7effd700 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56036ba06a30 0x56036ba08e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.653+0000 7f7a7effd700 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56036ba06a30 0x56036ba08e50 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee8b1989224a436c server_cookie=50e60bfc1aa2c46c in_seq=0 out_seq=0 2024-05-09T00:11:18.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.653+0000 7f7a667fc700 10 client.22137 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:18.652 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:11:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:11:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:11:18.653 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:11:18.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a667fc700 1 -- 172.21.15.12:0/2867512299 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56036b8ccec0 con 0x56036ba06a30 2024-05-09T00:11:18.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a667fc700 10 client.22137 handle_command_reply: tid=0 2024-05-09T00:11:18.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a91926b80 1 client.22137 shutdown 2024-05-09T00:11:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a91926b80 1 -- 172.21.15.12:0/2867512299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 msgr2=0x56036b995460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a91926b80 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b995460 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7a68029bf0 tx=0x7f7a68005020 comp rx=0 tx=0).stop 2024-05-09T00:11:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a91926b80 1 -- 172.21.15.12:0/2867512299 shutdown_connections 2024-05-09T00:11:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a91926b80 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b98a720 0x56036b991d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a91926b80 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56036b984890 0x56036b995460 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a91926b80 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x56036ba06a30 0x56036ba08e50 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a91926b80 1 --2- 172.21.15.12:0/2867512299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56036b989e00 0x56036b991840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a91926b80 1 -- 172.21.15.12:0/2867512299 >> 172.21.15.12:0/2867512299 conn(0x56036b8e2210 msgr2=0x56036b98c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.654+0000 7f7a667fc700 0 client.22137 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.655+0000 7f7a91926b80 1 -- 172.21.15.12:0/2867512299 shutdown_connections 2024-05-09T00:11:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.655+0000 7f7a91926b80 1 -- 172.21.15.12:0/2867512299 wait complete. 2024-05-09T00:11:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.655+0000 7f7a91926b80 20 client.22137 trim_cache size 0 max 16384 2024-05-09T00:11:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 -- 172.21.15.12:0/4150829855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a6c0a1310 msgr2=0x7f7a6c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a6c0a1310 0x7f7a6c0a37d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7a780caa80 tx=0x7f7a7800b040 comp rx=0 tx=0).stop 2024-05-09T00:11:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 -- 172.21.15.12:0/4150829855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a80105640 msgr2=0x7f7a801a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a80105640 0x7f7a801a2220 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7a78007ee0 tx=0x7f7a7800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 -- 172.21.15.12:0/4150829855 shutdown_connections 2024-05-09T00:11:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7a6c0a1310 0x7f7a6c0a37d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a80105640 0x7f7a801a2220 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a800ff580 0x7f7a8010b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 --2- 172.21.15.12:0/4150829855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a800fff40 0x7f7a8010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 -- 172.21.15.12:0/4150829855 >> 172.21.15.12:0/4150829855 conn(0x7f7a800763a0 msgr2=0x7f7a800fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:18.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 -- 172.21.15.12:0/4150829855 shutdown_connections 2024-05-09T00:11:18.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.659+0000 7f7a8576c700 1 -- 172.21.15.12:0/4150829855 wait complete. 2024-05-09T00:11:18.678 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:11:18.678 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:11:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.959+0000 7f24ef230700 1 -- 172.21.15.12:0/3340941400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e810f5c0 msgr2=0x7f24e810f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.959+0000 7f24ef230700 1 --2- 172.21.15.12:0/3340941400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e810f5c0 0x7f24e810f9a0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f24d801a9a0 tx=0x7f24d801acb0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.962+0000 7f24ef230700 1 -- 172.21.15.12:0/3340941400 shutdown_connections 2024-05-09T00:11:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.962+0000 7f24ef230700 1 --2- 172.21.15.12:0/3340941400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e8107fa0 0x7f24e8108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.962+0000 7f24ef230700 1 --2- 172.21.15.12:0/3340941400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e810f5c0 0x7f24e810f9a0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.962+0000 7f24ef230700 1 --2- 172.21.15.12:0/3340941400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24e810d2b0 0x7f24e810d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.962+0000 7f24ef230700 1 -- 172.21.15.12:0/3340941400 >> 172.21.15.12:0/3340941400 conn(0x7f24e806bf20 msgr2=0x7f24e806c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:18.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.976+0000 7f24ef230700 1 -- 172.21.15.12:0/3340941400 shutdown_connections 2024-05-09T00:11:18.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.979+0000 7f24ef230700 1 -- 172.21.15.12:0/3340941400 wait complete. 2024-05-09T00:11:18.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.980+0000 7f24ef230700 1 Processor -- start 2024-05-09T00:11:18.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.980+0000 7f24ef230700 1 -- start start 2024-05-09T00:11:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.980+0000 7f24ef230700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e8107fa0 0x7f24e81a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.980+0000 7f24ef230700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24e810d2b0 0x7f24e81a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.980+0000 7f24ef230700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e810f5c0 0x7f24e819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.980+0000 7f24ef230700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e819dfe0 con 0x7f24e8107fa0 2024-05-09T00:11:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.980+0000 7f24ef230700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e819e150 con 0x7f24e810d2b0 2024-05-09T00:11:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.980+0000 7f24ef230700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e819e310 con 0x7f24e810f5c0 2024-05-09T00:11:18.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.983+0000 7f24e7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24e810d2b0 0x7f24e81a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.983+0000 7f24e7fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24e810d2b0 0x7f24e81a3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42670/0 (socket says 172.21.15.12:42670) 2024-05-09T00:11:18.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.983+0000 7f24e7fff700 1 -- 172.21.15.12:0/4111274814 learned_addr learned my addr 172.21.15.12:0/4111274814 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:18.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.984+0000 7f24ecfcc700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e8107fa0 0x7f24e81a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.985+0000 7f24ed7cd700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e810f5c0 0x7f24e819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.991+0000 7f24e7fff700 1 -- 172.21.15.12:0/4111274814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e810f5c0 msgr2=0x7f24e819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.991+0000 7f24e7fff700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e810f5c0 0x7f24e819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.991+0000 7f24e7fff700 1 -- 172.21.15.12:0/4111274814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e8107fa0 msgr2=0x7f24e81a3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.991+0000 7f24e7fff700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e8107fa0 0x7f24e81a3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:18.992+0000 7f24e7fff700 1 -- 172.21.15.12:0/4111274814 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d801a600 con 0x7f24e810d2b0 2024-05-09T00:11:19.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.004+0000 7f24e7fff700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24e810d2b0 0x7f24e81a3d90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f24e000ba50 tx=0x7f24e000bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:19.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.007+0000 7f24e5ffb700 1 -- 172.21.15.12:0/4111274814 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000c9b0 con 0x7f24e810d2b0 2024-05-09T00:11:19.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.007+0000 7f24e5ffb700 1 -- 172.21.15.12:0/4111274814 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24e0014440 con 0x7f24e810d2b0 2024-05-09T00:11:19.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.007+0000 7f24e5ffb700 1 -- 172.21.15.12:0/4111274814 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0007590 con 0x7f24e810d2b0 2024-05-09T00:11:19.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.007+0000 7f24ef230700 1 -- 172.21.15.12:0/4111274814 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e819e5f0 con 0x7f24e810d2b0 2024-05-09T00:11:19.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.008+0000 7f24ef230700 1 -- 172.21.15.12:0/4111274814 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e81aa840 con 0x7f24e810d2b0 2024-05-09T00:11:19.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.014+0000 7f24ef230700 1 -- 172.21.15.12:0/4111274814 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d40052f0 con 0x7f24e810d2b0 2024-05-09T00:11:19.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.020+0000 7f24e5ffb700 1 -- 172.21.15.12:0/4111274814 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f24e000cb10 con 0x7f24e810d2b0 2024-05-09T00:11:19.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.021+0000 7f24e5ffb700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f24d00a14a0 0x7f24d00a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.021+0000 7f24e5ffb700 1 -- 172.21.15.12:0/4111274814 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f24e00cc1a0 con 0x7f24e810d2b0 2024-05-09T00:11:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.025+0000 7f24ecfcc700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f24d00a14a0 0x7f24d00a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.025+0000 7f24ecfcc700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f24d00a14a0 0x7f24d00a3960 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f24d8006010 tx=0x7f24d801aef0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.031+0000 7f24e5ffb700 1 -- 172.21.15.12:0/4111274814 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f24e0095730 con 0x7f24e810d2b0 2024-05-09T00:11:19.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 373412 27.86 MB/sec execute 554 sec latency 284.649 ms 2024-05-09T00:11:19.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.392+0000 7f24ef230700 1 -- 172.21.15.12:0/4111274814 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24d4005160 con 0x7f24e810d2b0 2024-05-09T00:11:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.395+0000 7f24e5ffb700 1 -- 172.21.15.12:0/4111274814 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f24e0019090 con 0x7f24e810d2b0 2024-05-09T00:11:19.397 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:11:19.398 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":21435,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/676326208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":676326208},{"type":"v1","addr":"172.21.15.160:6833","nonce":676326208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:10:38.863191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_31639":{"gid":31639,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/1830773792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1830773792},{"type":"v1","addr":"172.21.15.204:6833","nonce":1830773792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:11:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 -- 172.21.15.12:0/4111274814 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f24d00a14a0 msgr2=0x7f24d00a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f24d00a14a0 0x7f24d00a3960 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f24d8006010 tx=0x7f24d801aef0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 -- 172.21.15.12:0/4111274814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24e810d2b0 msgr2=0x7f24e81a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24e810d2b0 0x7f24e81a3d90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f24e000ba50 tx=0x7f24e000bd60 comp rx=0 tx=0).stop 2024-05-09T00:11:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 -- 172.21.15.12:0/4111274814 shutdown_connections 2024-05-09T00:11:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f24d00a14a0 0x7f24d00a3960 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e810f5c0 0x7f24e819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e8107fa0 0x7f24e81a3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 --2- 172.21.15.12:0/4111274814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24e810d2b0 0x7f24e81a3d90 secure :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f24e000ba50 tx=0x7f24e000bd60 comp rx=0 tx=0).stop 2024-05-09T00:11:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 -- 172.21.15.12:0/4111274814 >> 172.21.15.12:0/4111274814 conn(0x7f24e806bf20 msgr2=0x7f24e810aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 -- 172.21.15.12:0/4111274814 shutdown_connections 2024-05-09T00:11:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.404+0000 7f24ef230700 1 -- 172.21.15.12:0/4111274814 wait complete. 2024-05-09T00:11:19.408 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 83 2024-05-09T00:11:19.424 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:11:19.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.668+0000 7fdb76147700 1 -- 172.21.15.12:0/2162478442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb70068050 msgr2=0x7fdb70068430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:19.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.668+0000 7fdb76147700 1 --2- 172.21.15.12:0/2162478442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb70068050 0x7fdb70068430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009990 tx=0x7fdb60009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.670+0000 7fdb76147700 1 -- 172.21.15.12:0/2162478442 shutdown_connections 2024-05-09T00:11:19.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.670+0000 7fdb76147700 1 --2- 172.21.15.12:0/2162478442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb70101fa0 0x7fdb70104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.670+0000 7fdb76147700 1 --2- 172.21.15.12:0/2162478442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb70068050 0x7fdb70068430 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.670+0000 7fdb76147700 1 --2- 172.21.15.12:0/2162478442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70068970 0x7fdb7010d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.670+0000 7fdb76147700 1 -- 172.21.15.12:0/2162478442 >> 172.21.15.12:0/2162478442 conn(0x7fdb70076f40 msgr2=0x7fdb70077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:19.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.671+0000 7fdb76147700 1 -- 172.21.15.12:0/2162478442 shutdown_connections 2024-05-09T00:11:19.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.672+0000 7fdb76147700 1 -- 172.21.15.12:0/2162478442 wait complete. 2024-05-09T00:11:19.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.673+0000 7fdb76147700 1 Processor -- start 2024-05-09T00:11:19.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.673+0000 7fdb76147700 1 -- start start 2024-05-09T00:11:19.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.673+0000 7fdb76147700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70068050 0x7fdb70072470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.673+0000 7fdb76147700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb70068970 0x7fdb700729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.673+0000 7fdb76147700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb70101fa0 0x7fdb7006b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.673+0000 7fdb76147700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7006bd60 con 0x7fdb70068050 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.673+0000 7fdb76147700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7006bea0 con 0x7fdb70068970 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.673+0000 7fdb76147700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7006c060 con 0x7fdb70101fa0 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.674+0000 7fdb6ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb70101fa0 0x7fdb7006b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.674+0000 7fdb6ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb70101fa0 0x7fdb7006b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48472/0 (socket says 172.21.15.12:48472) 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.674+0000 7fdb6ffff700 1 -- 172.21.15.12:0/3378300250 learned_addr learned my addr 172.21.15.12:0/3378300250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.674+0000 7fdb6ffff700 1 -- 172.21.15.12:0/3378300250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb70068970 msgr2=0x7fdb700729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.674+0000 7fdb6ffff700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb70068970 0x7fdb700729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.674+0000 7fdb6ffff700 1 -- 172.21.15.12:0/3378300250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70068050 msgr2=0x7fdb70072470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:11:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.674+0000 7fdb6ffff700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70068050 0x7fdb70072470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.674+0000 7fdb6ffff700 1 -- 172.21.15.12:0/3378300250 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600095f0 con 0x7fdb70101fa0 2024-05-09T00:11:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.674+0000 7fdb6ffff700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb70101fa0 0x7fdb7006b590 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800b780 tx=0x7fdb6800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.675+0000 7fdb6cff9700 1 -- 172.21.15.12:0/3378300250 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6800f840 con 0x7fdb70101fa0 2024-05-09T00:11:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.675+0000 7fdb76147700 1 -- 172.21.15.12:0/3378300250 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7006c340 con 0x7fdb70101fa0 2024-05-09T00:11:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.675+0000 7fdb76147700 1 -- 172.21.15.12:0/3378300250 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb700702f0 con 0x7fdb70101fa0 2024-05-09T00:11:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.675+0000 7fdb6cff9700 1 -- 172.21.15.12:0/3378300250 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb6800fe60 con 0x7fdb70101fa0 2024-05-09T00:11:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.675+0000 7fdb6cff9700 1 -- 172.21.15.12:0/3378300250 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6800d3d0 con 0x7fdb70101fa0 2024-05-09T00:11:19.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.678+0000 7fdb6cff9700 1 -- 172.21.15.12:0/3378300250 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdb6800d650 con 0x7fdb70101fa0 2024-05-09T00:11:19.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.679+0000 7fdb6cff9700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb580a1350 0x7fdb580a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.679+0000 7fdb6cff9700 1 -- 172.21.15.12:0/3378300250 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdb680cca80 con 0x7fdb70101fa0 2024-05-09T00:11:19.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.680+0000 7fdb82301b80 1 Processor -- start 2024-05-09T00:11:19.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.682+0000 7fdb82301b80 1 -- start start 2024-05-09T00:11:19.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.682+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdb701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.682+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb6a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.682+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb6acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.682+0000 7fdb82301b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbdac1150 con 0x563fbdb70720 2024-05-09T00:11:19.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.682+0000 7fdb82301b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbdabeb10 con 0x563fbdb6a890 2024-05-09T00:11:19.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.682+0000 7fdb82301b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbdaba400 con 0x563fbdb6fe00 2024-05-09T00:11:19.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.682+0000 7fdb6effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb6a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.682+0000 7fdb6effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb6a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50286/0 (socket says 172.21.15.12:50286) 2024-05-09T00:11:19.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.682+0000 7fdb6effd700 1 -- 172.21.15.12:0/798223478 learned_addr learned my addr 172.21.15.12:0/798223478 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:19.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.683+0000 7fdb6effd700 1 -- 172.21.15.12:0/798223478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 msgr2=0x563fbdb701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:19.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.683+0000 7fdb6effd700 1 --2- 172.21.15.12:0/798223478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdb701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.683+0000 7fdb6effd700 1 -- 172.21.15.12:0/798223478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 msgr2=0x563fbdb6acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:19.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.683+0000 7fdb6effd700 1 --2- 172.21.15.12:0/798223478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb6acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.683+0000 7fdb6effd700 1 -- 172.21.15.12:0/798223478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fbd93b3b0 con 0x563fbdb70720 2024-05-09T00:11:19.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.683+0000 7fdb6effd700 1 --2- 172.21.15.12:0/798223478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb6a350 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009870 tx=0x7fdb64009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8877eabecfcbae17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:19.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.685+0000 7fdb6f7fe700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb580a1350 0x7fdb580a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.685+0000 7fdb55ffb700 1 -- 172.21.15.12:0/798223478 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64011830 con 0x563fbdb70720 2024-05-09T00:11:19.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.685+0000 7fdb55ffb700 1 -- 172.21.15.12:0/798223478 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb64011e50 con 0x563fbdb70720 2024-05-09T00:11:19.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.686+0000 7fdb55ffb700 1 -- 172.21.15.12:0/798223478 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64010380 con 0x563fbdb70720 2024-05-09T00:11:19.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.686+0000 7fdb82301b80 1 -- 172.21.15.12:0/798223478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 msgr2=0x563fbdb6a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:19.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.686+0000 7fdb82301b80 1 --2- 172.21.15.12:0/798223478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb6a350 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009870 tx=0x7fdb64009b80 comp rx=0 tx=0).stop 2024-05-09T00:11:19.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.687+0000 7fdb82301b80 1 -- 172.21.15.12:0/798223478 shutdown_connections 2024-05-09T00:11:19.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.687+0000 7fdb82301b80 1 --2- 172.21.15.12:0/798223478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb6acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.687+0000 7fdb82301b80 1 --2- 172.21.15.12:0/798223478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdb701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.687+0000 7fdb82301b80 1 --2- 172.21.15.12:0/798223478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb6a350 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.687+0000 7fdb82301b80 1 -- 172.21.15.12:0/798223478 >> 172.21.15.12:0/798223478 conn(0x563fbdac8210 msgr2=0x563fbdb64df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:19.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.687+0000 7fdb6f7fe700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb580a1350 0x7fdb580a3810 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdb600098e0 tx=0x7fdb6000b580 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:19.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.687+0000 7fdb82301b80 1 -- 172.21.15.12:0/798223478 shutdown_connections 2024-05-09T00:11:19.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.687+0000 7fdb82301b80 1 -- 172.21.15.12:0/798223478 wait complete. 2024-05-09T00:11:19.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.688+0000 7fdb82301b80 1 -- start start 2024-05-09T00:11:19.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.688+0000 7fdb82301b80 1 Processor -- start 2024-05-09T00:11:19.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.688+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb73af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.688+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6fe00 0x563fbdb74030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.688+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb70720 0x563fbdb74570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.688+0000 7fdb82301b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbda56560 con 0x563fbdb6fe00 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.688+0000 7fdb82301b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbda56200 con 0x563fbdb6a890 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.688+0000 7fdb82301b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbd93d6a0 con 0x563fbdb70720 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.688+0000 7fdb82301b80 10 client.0 _ll_register_callbacks cb 0x563fbda5fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.689+0000 7fdb6ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb70720 0x563fbdb74570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.689+0000 7fdb6ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb70720 0x563fbdb74570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48500/0 (socket says 172.21.15.12:48500) 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.689+0000 7fdb6ffff700 1 -- 172.21.15.12:0/1598153217 learned_addr learned my addr 172.21.15.12:0/1598153217 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.689+0000 7fdb6effd700 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6fe00 0x563fbdb74030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.689+0000 7fdb6f7fe700 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb73af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.689+0000 7fdb6effd700 1 -- 172.21.15.12:0/1598153217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb70720 msgr2=0x563fbdb74570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.689+0000 7fdb6effd700 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb70720 0x563fbdb74570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.689+0000 7fdb6effd700 1 -- 172.21.15.12:0/1598153217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 msgr2=0x563fbdb73af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:19.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.689+0000 7fdb6effd700 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb73af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.689+0000 7fdb6effd700 1 -- 172.21.15.12:0/1598153217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb640094a0 con 0x563fbdb6fe00 2024-05-09T00:11:19.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.690+0000 7fdb6effd700 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6fe00 0x563fbdb74030 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009840 tx=0x7fdb6400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120625ca5b1de02d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:19.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.690+0000 7fdb4effd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:19.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.690+0000 7fdb4effd700 1 -- 172.21.15.12:0/1598153217 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64006cd0 con 0x563fbdb6fe00 2024-05-09T00:11:19.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.690+0000 7fdb4effd700 1 -- 172.21.15.12:0/1598153217 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdb64011630 con 0x563fbdb6fe00 2024-05-09T00:11:19.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.690+0000 7fdb82301b80 1 -- 172.21.15.12:0/1598153217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fbda13270 con 0x563fbdb6fe00 2024-05-09T00:11:19.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.690+0000 7fdb4effd700 1 -- 172.21.15.12:0/1598153217 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb64011810 con 0x563fbdb6fe00 2024-05-09T00:11:19.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.690+0000 7fdb82301b80 10 client.22149 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:19.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.690+0000 7fdb82301b80 1 -- 172.21.15.12:0/1598153217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x563fbd8e7b60 con 0x563fbdb6fe00 2024-05-09T00:11:19.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.691+0000 7fdb4effd700 1 -- 172.21.15.12:0/1598153217 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563fbda13270 con 0x563fbdb6fe00 2024-05-09T00:11:19.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.691+0000 7fdb4effd700 1 -- 172.21.15.12:0/1598153217 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fdb6402b070 con 0x563fbdb6fe00 2024-05-09T00:11:19.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.691+0000 7fdb82301b80 10 client.22149 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:19.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.691+0000 7fdb82301b80 10 client.22149 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:11:19.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.691+0000 7fdb82301b80 20 client.22149 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:19.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.691+0000 7fdb82301b80 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563fbdb84180 0x563fbdb865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.691+0000 7fdb82301b80 4 client.22149 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:19.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.691+0000 7fdb82301b80 1 -- 172.21.15.12:0/1598153217 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fbdb39810 con 0x563fbdb84180 2024-05-09T00:11:19.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.692+0000 7fdb6f7fe700 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563fbdb84180 0x563fbdb865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.693+0000 7fdb6f7fe700 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563fbdb84180 0x563fbdb865a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=874c4570a396fa58 server_cookie=5c8578660b3264fc in_seq=0 out_seq=0 2024-05-09T00:11:19.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.697+0000 7fdb4effd700 10 client.22149 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:11:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb4effd700 1 -- 172.21.15.12:0/1598153217 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fbdb39810 con 0x563fbdb84180 2024-05-09T00:11:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb4effd700 10 client.22149 handle_command_reply: tid=0 2024-05-09T00:11:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb82301b80 1 client.22149 shutdown 2024-05-09T00:11:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb82301b80 1 -- 172.21.15.12:0/1598153217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6fe00 msgr2=0x563fbdb74030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb82301b80 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6fe00 0x563fbdb74030 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009840 tx=0x7fdb6400b650 comp rx=0 tx=0).stop 2024-05-09T00:11:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb82301b80 1 -- 172.21.15.12:0/1598153217 shutdown_connections 2024-05-09T00:11:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb82301b80 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563fbdb84180 0x563fbdb865a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb82301b80 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb70720 0x563fbdb74570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb82301b80 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb73af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb4effd700 0 client.22149 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:11:19.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb82301b80 1 --2- 172.21.15.12:0/1598153217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6fe00 0x563fbdb74030 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb82301b80 1 -- 172.21.15.12:0/1598153217 >> 172.21.15.12:0/1598153217 conn(0x563fbdac8210 msgr2=0x563fbdb645a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:19.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.700+0000 7fdb82301b80 1 -- 172.21.15.12:0/1598153217 shutdown_connections 2024-05-09T00:11:19.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.703+0000 7fdb82301b80 1 -- 172.21.15.12:0/1598153217 wait complete. 2024-05-09T00:11:19.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.703+0000 7fdb82301b80 20 client.22149 trim_cache size 0 max 16384 2024-05-09T00:11:19.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.743+0000 7fdb82301b80 1 Processor -- start 2024-05-09T00:11:19.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.746+0000 7fdb82301b80 1 -- start start 2024-05-09T00:11:19.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.746+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb80350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.746+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdb7fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.746+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb64d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.746+0000 7fdb82301b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbdb6a700 con 0x563fbdb70720 2024-05-09T00:11:19.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.746+0000 7fdb82301b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbdb8b770 con 0x563fbdb6a890 2024-05-09T00:11:19.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.746+0000 7fdb82301b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbdb5d040 con 0x563fbdb6fe00 2024-05-09T00:11:19.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb64d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb64d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50300/0 (socket says 172.21.15.12:50300) 2024-05-09T00:11:19.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6ffff700 1 -- 172.21.15.12:0/2012413819 learned_addr learned my addr 172.21.15.12:0/2012413819 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:19.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6effd700 1 --2- 172.21.15.12:0/2012413819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdb7fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6f7fe700 1 --2- 172.21.15.12:0/2012413819 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb80350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6effd700 1 -- 172.21.15.12:0/2012413819 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 msgr2=0x563fbdb80350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:19.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6effd700 1 --2- 172.21.15.12:0/2012413819 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb80350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6effd700 1 -- 172.21.15.12:0/2012413819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 msgr2=0x563fbdb64d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:19.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6effd700 1 --2- 172.21.15.12:0/2012413819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb64d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6effd700 1 -- 172.21.15.12:0/2012413819 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fbda80560 con 0x563fbdb6fe00 2024-05-09T00:11:19.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6ffff700 1 --2- 172.21.15.12:0/2012413819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb64d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb6effd700 1 --2- 172.21.15.12:0/2012413819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdb7fcb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdb64023e60 tx=0x7fdb64009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a641043153f300e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.747+0000 7fdb567fc700 1 -- 172.21.15.12:0/2012413819 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64023ee0 con 0x563fbdb6fe00 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.748+0000 7fdb567fc700 1 -- 172.21.15.12:0/2012413819 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb6400ba60 con 0x563fbdb6fe00 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.748+0000 7fdb567fc700 1 -- 172.21.15.12:0/2012413819 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64011890 con 0x563fbdb6fe00 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.749+0000 7fdb82301b80 1 -- 172.21.15.12:0/2012413819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 msgr2=0x563fbdb7fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.749+0000 7fdb82301b80 1 --2- 172.21.15.12:0/2012413819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdb7fcb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdb64023e60 tx=0x7fdb64009190 comp rx=0 tx=0).stop 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.750+0000 7fdb82301b80 1 -- 172.21.15.12:0/2012413819 shutdown_connections 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.750+0000 7fdb82301b80 1 --2- 172.21.15.12:0/2012413819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb70720 0x563fbdb64d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.750+0000 7fdb82301b80 1 --2- 172.21.15.12:0/2012413819 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb6a890 0x563fbdb80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.750+0000 7fdb82301b80 1 --2- 172.21.15.12:0/2012413819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdb7fcb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.750+0000 7fdb82301b80 1 -- 172.21.15.12:0/2012413819 >> 172.21.15.12:0/2012413819 conn(0x563fbdac8210 msgr2=0x563fbdb743d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.750+0000 7fdb82301b80 1 -- 172.21.15.12:0/2012413819 shutdown_connections 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.750+0000 7fdb82301b80 1 -- 172.21.15.12:0/2012413819 wait complete. 2024-05-09T00:11:19.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.750+0000 7fdb82301b80 1 -- start start 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb82301b80 1 Processor -- start 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6a890 0x563fbdbf05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdbf0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb82301b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb70720 0x563fbdbf1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb82301b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbdbfb660 con 0x563fbdb6a890 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb82301b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbdbfb7a0 con 0x563fbdb70720 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb82301b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbdbfb8e0 con 0x563fbdb6fe00 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb82301b80 10 client.0 _ll_register_callbacks cb 0x563fbda5fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb6f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6a890 0x563fbdbf05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb6f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6a890 0x563fbdbf05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50316/0 (socket says 172.21.15.12:50316) 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb6f7fe700 1 -- 172.21.15.12:0/3893750058 learned_addr learned my addr 172.21.15.12:0/3893750058 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.751+0000 7fdb6ffff700 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb70720 0x563fbdbf1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.752+0000 7fdb6f7fe700 1 -- 172.21.15.12:0/3893750058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 msgr2=0x563fbdbf0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:19.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.752+0000 7fdb6f7fe700 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdbf0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.752+0000 7fdb6f7fe700 1 -- 172.21.15.12:0/3893750058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb70720 msgr2=0x563fbdbf1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:19.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.752+0000 7fdb6f7fe700 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb70720 0x563fbdbf1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.752+0000 7fdb6f7fe700 1 -- 172.21.15.12:0/3893750058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb700702f0 con 0x563fbdb6a890 2024-05-09T00:11:19.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.752+0000 7fdb6f7fe700 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6a890 0x563fbdbf05e0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000b900 tx=0x7fdb60000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabac4c1ecaee265 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:19.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.752+0000 7fdb4effd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:19.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.753+0000 7fdb4effd700 1 -- 172.21.15.12:0/3893750058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60029040 con 0x563fbdb6a890 2024-05-09T00:11:19.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.753+0000 7fdb4effd700 1 -- 172.21.15.12:0/3893750058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdb6002e870 con 0x563fbdb6a890 2024-05-09T00:11:19.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.753+0000 7fdb82301b80 1 -- 172.21.15.12:0/3893750058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb64011e50 con 0x563fbdb6a890 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.754+0000 7fdb4effd700 1 -- 172.21.15.12:0/3893750058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb60004e10 con 0x563fbdb6a890 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.754+0000 7fdb82301b80 10 client.22155 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.754+0000 7fdb82301b80 1 -- 172.21.15.12:0/3893750058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x563fbd8e7b60 con 0x563fbdb6a890 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.754+0000 7fdb4effd700 1 -- 172.21.15.12:0/3893750058 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb60087080 con 0x563fbdb6a890 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.754+0000 7fdb4effd700 1 -- 172.21.15.12:0/3893750058 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fdb60083830 con 0x563fbdb6a890 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.755+0000 7fdb82301b80 10 client.22155 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.755+0000 7fdb82301b80 10 client.22155 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.755+0000 7fdb82301b80 20 client.22155 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.755+0000 7fdb82301b80 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563fbdbcea80 0x563fbdbd0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.755+0000 7fdb82301b80 4 client.22155 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.755+0000 7fdb82301b80 1 -- 172.21.15.12:0/3893750058 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563fbdab2ec0 con 0x563fbdbcea80 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.755+0000 7fdb6effd700 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563fbdbcea80 0x563fbdbd0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:19.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.756+0000 7fdb4effd700 10 client.22155 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:11:19.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.756+0000 7fdb6effd700 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563fbdbcea80 0x563fbdbd0ea0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb72818347638e92 server_cookie=3249ce374ddcb96d in_seq=0 out_seq=0 2024-05-09T00:11:19.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.761+0000 7fdb4effd700 1 -- 172.21.15.12:0/3893750058 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563fbdab2ec0 con 0x563fbdbcea80 2024-05-09T00:11:19.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.761+0000 7fdb4effd700 10 client.22155 handle_command_reply: tid=0 2024-05-09T00:11:19.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.763+0000 7fdb82301b80 1 client.22155 shutdown 2024-05-09T00:11:19.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.765+0000 7fdb82301b80 1 -- 172.21.15.12:0/3893750058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6a890 msgr2=0x563fbdbf05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:19.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.765+0000 7fdb82301b80 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6a890 0x563fbdbf05e0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000b900 tx=0x7fdb60000bc0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.765+0000 7fdb82301b80 1 -- 172.21.15.12:0/3893750058 shutdown_connections 2024-05-09T00:11:19.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.765+0000 7fdb82301b80 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fbdb70720 0x563fbdbf1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.765+0000 7fdb82301b80 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fbdb6a890 0x563fbdbf05e0 secure :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000b900 tx=0x7fdb60000bc0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.765+0000 7fdb82301b80 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x563fbdbcea80 0x563fbdbd0ea0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.765+0000 7fdb82301b80 1 --2- 172.21.15.12:0/3893750058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563fbdb6fe00 0x563fbdbf0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.765+0000 7fdb82301b80 1 -- 172.21.15.12:0/3893750058 >> 172.21.15.12:0/3893750058 conn(0x563fbdac8210 msgr2=0x563fbdb80350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:19 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/4111274814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:19.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.768+0000 7fdb4effd700 0 client.22155 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:11:19.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:19 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/4111274814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:19.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.771+0000 7fdb82301b80 1 -- 172.21.15.12:0/3893750058 shutdown_connections 2024-05-09T00:11:19.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.775+0000 7fdb82301b80 1 -- 172.21.15.12:0/3893750058 wait complete. 2024-05-09T00:11:19.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.775+0000 7fdb82301b80 20 client.22155 trim_cache size 0 max 16384 2024-05-09T00:11:19.772 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:11:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.775+0000 7fdb76147700 1 -- 172.21.15.12:0/3378300250 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb580a1350 msgr2=0x7fdb580a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.775+0000 7fdb76147700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb580a1350 0x7fdb580a3810 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdb600098e0 tx=0x7fdb6000b580 comp rx=0 tx=0).stop 2024-05-09T00:11:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.776+0000 7fdb76147700 1 -- 172.21.15.12:0/3378300250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb70101fa0 msgr2=0x7fdb7006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.776+0000 7fdb76147700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb70101fa0 0x7fdb7006b590 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800b780 tx=0x7fdb6800bb40 comp rx=0 tx=0).stop 2024-05-09T00:11:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.776+0000 7fdb76147700 1 -- 172.21.15.12:0/3378300250 shutdown_connections 2024-05-09T00:11:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.776+0000 7fdb76147700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fdb580a1350 0x7fdb580a3810 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.776+0000 7fdb76147700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb70101fa0 0x7fdb7006b590 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.776+0000 7fdb76147700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70068050 0x7fdb70072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.776+0000 7fdb76147700 1 --2- 172.21.15.12:0/3378300250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb70068970 0x7fdb700729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:19.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.776+0000 7fdb76147700 1 -- 172.21.15.12:0/3378300250 >> 172.21.15.12:0/3378300250 conn(0x7fdb70076f40 msgr2=0x7fdb701032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:19.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.778+0000 7fdb76147700 1 -- 172.21.15.12:0/3378300250 shutdown_connections 2024-05-09T00:11:19.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:19.780+0000 7fdb76147700 1 -- 172.21.15.12:0/3378300250 wait complete. 2024-05-09T00:11:19.803 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:11:19.804 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:19 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/4111274814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.050+0000 7f5b44c58700 1 -- 172.21.15.12:0/2071997056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40103930 msgr2=0x7f5b40103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.050+0000 7f5b44c58700 1 --2- 172.21.15.12:0/2071997056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40103930 0x7f5b40103db0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009990 tx=0x7f5b34009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.050+0000 7f5b44c58700 1 -- 172.21.15.12:0/2071997056 shutdown_connections 2024-05-09T00:11:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.050+0000 7f5b44c58700 1 --2- 172.21.15.12:0/2071997056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40103930 0x7f5b40103db0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.050+0000 7f5b44c58700 1 --2- 172.21.15.12:0/2071997056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b40105e30 0x7f5b40068050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.050+0000 7f5b44c58700 1 --2- 172.21.15.12:0/2071997056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b40068590 0x7f5b40068a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.050+0000 7f5b44c58700 1 -- 172.21.15.12:0/2071997056 >> 172.21.15.12:0/2071997056 conn(0x7f5b40076f40 msgr2=0x7f5b40077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.050+0000 7f5b44c58700 1 -- 172.21.15.12:0/2071997056 shutdown_connections 2024-05-09T00:11:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.050+0000 7f5b44c58700 1 -- 172.21.15.12:0/2071997056 wait complete. 2024-05-09T00:11:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.051+0000 7f5b44c58700 1 Processor -- start 2024-05-09T00:11:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.051+0000 7f5b44c58700 1 -- start start 2024-05-09T00:11:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.051+0000 7f5b44c58700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40068590 0x7f5b401a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.051+0000 7f5b44c58700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b40103930 0x7f5b401a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.051+0000 7f5b44c58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b40105e30 0x7f5b4019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.051+0000 7f5b44c58700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4019de50 con 0x7f5b40105e30 2024-05-09T00:11:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.051+0000 7f5b44c58700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4019dfc0 con 0x7f5b40103930 2024-05-09T00:11:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.051+0000 7f5b44c58700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4019e180 con 0x7f5b40068590 2024-05-09T00:11:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40068590 0x7f5b401a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40068590 0x7f5b401a36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48530/0 (socket says 172.21.15.12:48530) 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3e59c700 1 -- 172.21.15.12:0/265418908 learned_addr learned my addr 172.21.15.12:0/265418908 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b40103930 0x7f5b401a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3ed9d700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b40105e30 0x7f5b4019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/265418908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40068590 msgr2=0x7f5b401a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40068590 0x7f5b401a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/265418908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b40105e30 msgr2=0x7f5b4019d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b40105e30 0x7f5b4019d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/265418908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b340095f0 con 0x7f5b40103930 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40068590 0x7f5b401a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.052+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b40103930 0x7f5b401a3bf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000e9f0 tx=0x7f5b3000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.053+0000 7f5b2f7fe700 1 -- 172.21.15.12:0/265418908 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000c9c0 con 0x7f5b40103930 2024-05-09T00:11:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.053+0000 7f5b2f7fe700 1 -- 172.21.15.12:0/265418908 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b30010430 con 0x7f5b40103930 2024-05-09T00:11:20.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.053+0000 7f5b2f7fe700 1 -- 172.21.15.12:0/265418908 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30018680 con 0x7f5b40103930 2024-05-09T00:11:20.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.053+0000 7f5b44c58700 1 -- 172.21.15.12:0/265418908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4019e460 con 0x7f5b40103930 2024-05-09T00:11:20.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.053+0000 7f5b44c58700 1 -- 172.21.15.12:0/265418908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b401aa6a0 con 0x7f5b40103930 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.058+0000 7f5b50e12b80 1 Processor -- start 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.058+0000 7f5b50e12b80 1 -- start start 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.058+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.058+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0da730 0x55c6ac0d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.058+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.058+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ac01a150 con 0x55c6ac0d48a0 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.058+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ac017b10 con 0x55c6ac0da730 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.058+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ac013400 con 0x55c6ac0d9e10 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0da730 0x55c6ac0d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0da730 0x55c6ac0d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42762/0 (socket says 172.21.15.12:42762) 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3ed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0da1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48534/0 (socket says 172.21.15.12:48534) 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3e59c700 1 -- 172.21.15.12:0/3337997632 learned_addr learned my addr 172.21.15.12:0/3337997632 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3e59c700 1 -- 172.21.15.12:0/3337997632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 msgr2=0x55c6ac0da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/3337997632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0da1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3e59c700 1 -- 172.21.15.12:0/3337997632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 msgr2=0x55c6ac0d4ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/3337997632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0d4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3e59c700 1 -- 172.21.15.12:0/3337997632 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6abe943b0 con 0x55c6ac0da730 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3ed9d700 1 --2- 172.21.15.12:0/3337997632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.059+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/3337997632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0da730 0x55c6ac0d4360 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5b380094b0 tx=0x7f5b38009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf37888dd0f18817 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.060+0000 7f5b2cff9700 1 -- 172.21.15.12:0/3337997632 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38003c00 con 0x55c6ac0da730 2024-05-09T00:11:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.060+0000 7f5b2cff9700 1 -- 172.21.15.12:0/3337997632 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b3800f040 con 0x55c6ac0da730 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.060+0000 7f5b2cff9700 1 -- 172.21.15.12:0/3337997632 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38004390 con 0x55c6ac0da730 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.060+0000 7f5b2f7fe700 1 -- 172.21.15.12:0/265418908 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5b30010860 con 0x7f5b40103930 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.060+0000 7f5b50e12b80 1 -- 172.21.15.12:0/3337997632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0da730 msgr2=0x55c6ac0d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.060+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/3337997632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0da730 0x55c6ac0d4360 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5b380094b0 tx=0x7f5b38009920 comp rx=0 tx=0).stop 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.061+0000 7f5b50e12b80 1 -- 172.21.15.12:0/3337997632 shutdown_connections 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.061+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/3337997632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.061+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/3337997632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.061+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/3337997632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0da730 0x55c6ac0d4360 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.061+0000 7f5b50e12b80 1 -- 172.21.15.12:0/3337997632 >> 172.21.15.12:0/3337997632 conn(0x55c6ac021210 msgr2=0x55c6ac0bddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.061+0000 7f5b2f7fe700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5b280a1350 0x7f5b280a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.061+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5b280a1350 0x7f5b280a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.061+0000 7f5b2f7fe700 1 -- 172.21.15.12:0/265418908 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5b30014070 con 0x7f5b40103930 2024-05-09T00:11:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.062+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5b280a1350 0x7f5b280a3810 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5b38007fd0 tx=0x7f5b38009050 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.067+0000 7f5b50e12b80 1 -- 172.21.15.12:0/3337997632 shutdown_connections 2024-05-09T00:11:20.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.067+0000 7f5b50e12b80 1 -- 172.21.15.12:0/3337997632 wait complete. 2024-05-09T00:11:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.068+0000 7f5b50e12b80 1 -- start start 2024-05-09T00:11:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.068+0000 7f5b50e12b80 1 Processor -- start 2024-05-09T00:11:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.068+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0de220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.068+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.068+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 0x55c6ac0cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.068+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6abfaf560 con 0x55c6ac0da730 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.068+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6abfaf200 con 0x55c6ac0d48a0 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.068+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6abe966a0 con 0x55c6ac0d9e10 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.068+0000 7f5b50e12b80 10 client.0 _ll_register_callbacks cb 0x55c6abfb8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.068+0000 7f5b3dd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.069+0000 7f5b3dd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0de760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48538/0 (socket says 172.21.15.12:48538) 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.069+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/4036277391 learned_addr learned my addr 172.21.15.12:0/4036277391 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.069+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0de220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.069+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/4036277391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 msgr2=0x55c6ac0de220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.069+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0de220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.069+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/4036277391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 msgr2=0x55c6ac0cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.069+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 0x55c6ac0cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.069+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/4036277391 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b40114d30 con 0x55c6ac0d9e10 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.069+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0de760 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5b300caa50 tx=0x7f5b30099820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7deaae0b7ce901e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.069+0000 7f5b25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b25ffb700 1 -- 172.21.15.12:0/4036277391 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30099500 con 0x55c6ac0d9e10 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b25ffb700 1 -- 172.21.15.12:0/4036277391 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5b3009dc40 con 0x55c6ac0d9e10 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b50e12b80 1 -- 172.21.15.12:0/4036277391 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6abf6c270 con 0x55c6ac0d9e10 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b25ffb700 1 -- 172.21.15.12:0/4036277391 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b300a2290 con 0x55c6ac0d9e10 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b50e12b80 10 client.31825 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b50e12b80 1 -- 172.21.15.12:0/4036277391 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7f5b38009190 con 0x55c6ac0d9e10 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b25ffb700 1 -- 172.21.15.12:0/4036277391 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6abf6c270 con 0x55c6ac0d9e10 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b25ffb700 1 -- 172.21.15.12:0/4036277391 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5b300caba0 con 0x55c6ac0d9e10 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b50e12b80 10 client.31825 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b50e12b80 10 client.31825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:11:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b50e12b80 20 client.31825 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55c6ac0d0bd0 0x55c6ac0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b50e12b80 4 client.31825 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.070+0000 7f5b50e12b80 1 -- 172.21.15.12:0/4036277391 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6ac092810 con 0x55c6ac0d0bd0 2024-05-09T00:11:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.071+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55c6ac0d0bd0 0x55c6ac0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.071+0000 7f5b25ffb700 10 client.31825 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:11:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.071+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55c6ac0d0bd0 0x55c6ac0d2fc0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=937d883cdf02fb77 server_cookie=fe8927c743046d65 in_seq=0 out_seq=0 2024-05-09T00:11:20.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b25ffb700 1 -- 172.21.15.12:0/4036277391 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6ac092810 con 0x55c6ac0d0bd0 2024-05-09T00:11:20.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b25ffb700 10 client.31825 handle_command_reply: tid=0 2024-05-09T00:11:20.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 client.31825 shutdown 2024-05-09T00:11:20.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 -- 172.21.15.12:0/4036277391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 msgr2=0x55c6ac0de760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0de760 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5b300caa50 tx=0x7f5b30099820 comp rx=0 tx=0).stop 2024-05-09T00:11:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 -- 172.21.15.12:0/4036277391 shutdown_connections 2024-05-09T00:11:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55c6ac0d0bd0 0x55c6ac0d2fc0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 0x55c6ac0cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0de220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/4036277391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0de760 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 -- 172.21.15.12:0/4036277391 >> 172.21.15.12:0/4036277391 conn(0x55c6ac021210 msgr2=0x55c6ac0bd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b25ffb700 0 client.31825 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:11:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 -- 172.21.15.12:0/4036277391 shutdown_connections 2024-05-09T00:11:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.074+0000 7f5b50e12b80 1 -- 172.21.15.12:0/4036277391 wait complete. 2024-05-09T00:11:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.075+0000 7f5b50e12b80 20 client.31825 trim_cache size 0 max 16384 2024-05-09T00:11:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b50e12b80 1 Processor -- start 2024-05-09T00:11:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b50e12b80 1 -- start start 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0dbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0da730 0x55c6ac0d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6abf6c270 con 0x55c6ac0d48a0 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ac0df6a0 con 0x55c6ac0d9e10 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ac0ded90 con 0x55c6ac0da730 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0dbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0dbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50356/0 (socket says 172.21.15.12:50356) 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b3e59c700 1 -- 172.21.15.12:0/2541384911 learned_addr learned my addr 172.21.15.12:0/2541384911 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/2541384911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0bdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.094+0000 7f5b3ed9d700 1 --2- 172.21.15.12:0/2541384911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0da730 0x55c6ac0d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.095+0000 7f5b3e59c700 1 -- 172.21.15.12:0/2541384911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0da730 msgr2=0x55c6ac0d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.095+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/2541384911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0da730 0x55c6ac0d8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.095+0000 7f5b3e59c700 1 -- 172.21.15.12:0/2541384911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d9e10 msgr2=0x55c6ac0bdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.095+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/2541384911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0bdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.095+0000 7f5b3e59c700 1 -- 172.21.15.12:0/2541384911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6ac0d4f50 con 0x55c6ac0d48a0 2024-05-09T00:11:20.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.095+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/2541384911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0dbb20 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800eeb0 tx=0x7f5b3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe7bb250dc421f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.095+0000 7f5b2d7fa700 1 -- 172.21.15.12:0/2541384911 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38055070 con 0x55c6ac0d48a0 2024-05-09T00:11:20.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.095+0000 7f5b2d7fa700 1 -- 172.21.15.12:0/2541384911 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b3800ebc0 con 0x55c6ac0d48a0 2024-05-09T00:11:20.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.095+0000 7f5b2d7fa700 1 -- 172.21.15.12:0/2541384911 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38045070 con 0x55c6ac0d48a0 2024-05-09T00:11:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.096+0000 7f5b50e12b80 1 -- 172.21.15.12:0/2541384911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 msgr2=0x55c6ac0dbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.096+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/2541384911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0dbb20 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800eeb0 tx=0x7f5b3800b040 comp rx=0 tx=0).stop 2024-05-09T00:11:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.096+0000 7f5b50e12b80 1 -- 172.21.15.12:0/2541384911 shutdown_connections 2024-05-09T00:11:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.096+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/2541384911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0da730 0x55c6ac0d8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.096+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/2541384911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0dbb20 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.096+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/2541384911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0bdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.096+0000 7f5b50e12b80 1 -- 172.21.15.12:0/2541384911 >> 172.21.15.12:0/2541384911 conn(0x55c6ac021210 msgr2=0x55c6ac0e4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.096+0000 7f5b50e12b80 1 -- 172.21.15.12:0/2541384911 shutdown_connections 2024-05-09T00:11:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.096+0000 7f5b50e12b80 1 -- 172.21.15.12:0/2541384911 wait complete. 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b50e12b80 1 -- start start 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b50e12b80 1 Processor -- start 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b50e12b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 0x55c6ac0c5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ac0c5b90 con 0x55c6ac0da730 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ac0c5d00 con 0x55c6ac0d48a0 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b50e12b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ac0c5ec0 con 0x55c6ac0d9e10 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b50e12b80 10 client.0 _ll_register_callbacks cb 0x55c6abfb8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b3ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 0x55c6ac0c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b3dd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b3dd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48564/0 (socket says 172.21.15.12:48564) 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b3e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b3ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 0x55c6ac0c5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50372/0 (socket says 172.21.15.12:50372) 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.097+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/1974130900 learned_addr learned my addr 172.21.15.12:0/1974130900 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/1974130900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 msgr2=0x55c6ac0c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/1974130900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 msgr2=0x55c6ac0c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 0x55c6ac0c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b3dd9b700 1 -- 172.21.15.12:0/1974130900 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6abe943b0 con 0x55c6ac0d9e10 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b3ed9d700 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 0x55c6ac0c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b3dd9b700 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0c7bb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5b300caf00 tx=0x7f5b300a2090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f91964ee001aaef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b25ffb700 1 -- 172.21.15.12:0/1974130900 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000cd20 con 0x55c6ac0d9e10 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b50e12b80 1 -- 172.21.15.12:0/1974130900 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b3800f040 con 0x55c6ac0d9e10 2024-05-09T00:11:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b25ffb700 1 -- 172.21.15.12:0/1974130900 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5b30099510 con 0x55c6ac0d9e10 2024-05-09T00:11:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b25ffb700 1 -- 172.21.15.12:0/1974130900 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3000ce80 con 0x55c6ac0d9e10 2024-05-09T00:11:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b50e12b80 10 client.31831 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.098+0000 7f5b50e12b80 1 -- 172.21.15.12:0/1974130900 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55c6abe40b60 con 0x55c6ac0d9e10 2024-05-09T00:11:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.099+0000 7f5b25ffb700 1 -- 172.21.15.12:0/1974130900 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b3800f040 con 0x55c6ac0d9e10 2024-05-09T00:11:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.099+0000 7f5b25ffb700 1 -- 172.21.15.12:0/1974130900 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5b30098d70 con 0x55c6ac0d9e10 2024-05-09T00:11:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.099+0000 7f5b50e12b80 10 client.31831 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.099+0000 7f5b50e12b80 10 client.31831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:11:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.099+0000 7f5b50e12b80 20 client.31831 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.099+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55c6ac1342e0 0x55c6ac136700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.099+0000 7f5b50e12b80 4 client.31831 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:11:20.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.099+0000 7f5b50e12b80 1 -- 172.21.15.12:0/1974130900 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6ac00bec0 con 0x55c6ac1342e0 2024-05-09T00:11:20.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.099+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55c6ac1342e0 0x55c6ac136700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.100+0000 7f5b3e59c700 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55c6ac1342e0 0x55c6ac136700 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91773cc890948c36 server_cookie=fcdf9667829c45f0 in_seq=0 out_seq=0 2024-05-09T00:11:20.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.100+0000 7f5b25ffb700 10 client.31831 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:11:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.101+0000 7f5b25ffb700 1 -- 172.21.15.12:0/1974130900 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6ac00bec0 con 0x55c6ac1342e0 2024-05-09T00:11:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.101+0000 7f5b25ffb700 10 client.31831 handle_command_reply: tid=0 2024-05-09T00:11:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.101+0000 7f5b50e12b80 1 client.31831 shutdown 2024-05-09T00:11:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 1 -- 172.21.15.12:0/1974130900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 msgr2=0x55c6ac0c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0c7bb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5b300caf00 tx=0x7f5b300a2090 comp rx=0 tx=0).stop 2024-05-09T00:11:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 1 -- 172.21.15.12:0/1974130900 shutdown_connections 2024-05-09T00:11:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55c6ac1342e0 0x55c6ac136700 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ac0da730 0x55c6ac0c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6ac0d48a0 0x55c6ac0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b25ffb700 0 client.31831 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:11:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 1 --2- 172.21.15.12:0/1974130900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6ac0d9e10 0x55c6ac0c7bb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 1 -- 172.21.15.12:0/1974130900 >> 172.21.15.12:0/1974130900 conn(0x55c6ac021210 msgr2=0x55c6ac0dc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 1 -- 172.21.15.12:0/1974130900 shutdown_connections 2024-05-09T00:11:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 1 -- 172.21.15.12:0/1974130900 wait complete. 2024-05-09T00:11:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.102+0000 7f5b50e12b80 20 client.31831 trim_cache size 0 max 16384 2024-05-09T00:11:20.104 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:11:20.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.103+0000 7f5b44c58700 1 -- 172.21.15.12:0/265418908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5b280a1350 msgr2=0x7f5b280a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.103+0000 7f5b44c58700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5b280a1350 0x7f5b280a3810 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5b38007fd0 tx=0x7f5b38009050 comp rx=0 tx=0).stop 2024-05-09T00:11:20.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.103+0000 7f5b44c58700 1 -- 172.21.15.12:0/265418908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b40103930 msgr2=0x7f5b401a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.103+0000 7f5b44c58700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b40103930 0x7f5b401a3bf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000e9f0 tx=0x7f5b3000ed00 comp rx=0 tx=0).stop 2024-05-09T00:11:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.104+0000 7f5b44c58700 1 -- 172.21.15.12:0/265418908 shutdown_connections 2024-05-09T00:11:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.104+0000 7f5b44c58700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5b280a1350 0x7f5b280a3810 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.104+0000 7f5b44c58700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b40105e30 0x7f5b4019d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.104+0000 7f5b44c58700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40068590 0x7f5b401a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.104+0000 7f5b44c58700 1 --2- 172.21.15.12:0/265418908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b40103930 0x7f5b401a3bf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.104+0000 7f5b44c58700 1 -- 172.21.15.12:0/265418908 >> 172.21.15.12:0/265418908 conn(0x7f5b40076f40 msgr2=0x7f5b400fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.104+0000 7f5b44c58700 1 -- 172.21.15.12:0/265418908 shutdown_connections 2024-05-09T00:11:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.104+0000 7f5b44c58700 1 -- 172.21.15.12:0/265418908 wait complete. 2024-05-09T00:11:20.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:11:20.325 INFO:tasks.workunit.client.0.smithi012.stdout: 10 373835 27.86 MB/sec execute 555 sec latency 162.286 ms 2024-05-09T00:11:20.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.341+0000 7fad012f1700 1 -- 172.21.15.12:0/1511870184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc10c2d0 msgr2=0x7facfc1187d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.341+0000 7fad012f1700 1 --2- 172.21.15.12:0/1511870184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc10c2d0 0x7facfc1187d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7facf4009f90 tx=0x7facf400b330 comp rx=0 tx=0).stop 2024-05-09T00:11:20.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.341+0000 7fad012f1700 1 -- 172.21.15.12:0/1511870184 shutdown_connections 2024-05-09T00:11:20.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.341+0000 7fad012f1700 1 --2- 172.21.15.12:0/1511870184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc10c2d0 0x7facfc1187d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.341+0000 7fad012f1700 1 --2- 172.21.15.12:0/1511870184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facfc111910 0x7facfc111cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.341+0000 7fad012f1700 1 --2- 172.21.15.12:0/1511870184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facfc10b910 0x7facfc10bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.341+0000 7fad012f1700 1 -- 172.21.15.12:0/1511870184 >> 172.21.15.12:0/1511870184 conn(0x7facfc06bf00 msgr2=0x7facfc109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.344+0000 7fad012f1700 1 -- 172.21.15.12:0/1511870184 shutdown_connections 2024-05-09T00:11:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.348+0000 7fad012f1700 1 -- 172.21.15.12:0/1511870184 wait complete. 2024-05-09T00:11:20.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7fad012f1700 1 Processor -- start 2024-05-09T00:11:20.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7fad012f1700 1 -- start start 2024-05-09T00:11:20.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7fad012f1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facfc10b910 0x7facfc118480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7fad012f1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facfc10c2d0 0x7facfc115570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7fad012f1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc111910 0x7facfc115ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7fad012f1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc1160c0 con 0x7facfc10b910 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7fad012f1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc116230 con 0x7facfc10c2d0 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7fad012f1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc1163f0 con 0x7facfc111910 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7facfaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facfc10b910 0x7facfc118480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7facfa7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facfc10c2d0 0x7facfc115570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7facfb7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc111910 0x7facfc115ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7facfa7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facfc10c2d0 0x7facfc115570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42796/0 (socket says 172.21.15.12:42796) 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.349+0000 7facfa7fc700 1 -- 172.21.15.12:0/1711043340 learned_addr learned my addr 172.21.15.12:0/1711043340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facfb7fe700 1 -- 172.21.15.12:0/1711043340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facfc10c2d0 msgr2=0x7facfc115570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facfb7fe700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facfc10c2d0 0x7facfc115570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facfb7fe700 1 -- 172.21.15.12:0/1711043340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facfc10b910 msgr2=0x7facfc118480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facfb7fe700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facfc10b910 0x7facfc118480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facfb7fe700 1 -- 172.21.15.12:0/1711043340 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face40095f0 con 0x7facfc111910 2024-05-09T00:11:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facfa7fc700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facfc10c2d0 0x7facfc115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facfaffd700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facfc10b910 0x7facfc118480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facfb7fe700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc111910 0x7facfc115ab0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7facf400b6c0 tx=0x7facf4008fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facf3fff700 1 -- 172.21.15.12:0/1711043340 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400e040 con 0x7facfc111910 2024-05-09T00:11:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7fad012f1700 1 -- 172.21.15.12:0/1711043340 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf4009c40 con 0x7facfc111910 2024-05-09T00:11:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facf3fff700 1 -- 172.21.15.12:0/1711043340 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf401d470 con 0x7facfc111910 2024-05-09T00:11:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7facf3fff700 1 -- 172.21.15.12:0/1711043340 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf40052a0 con 0x7facfc111910 2024-05-09T00:11:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.350+0000 7fad012f1700 1 -- 172.21.15.12:0/1711043340 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facfc1a8270 con 0x7facfc111910 2024-05-09T00:11:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.354+0000 7facf3fff700 1 -- 172.21.15.12:0/1711043340 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7facf401d5e0 con 0x7facfc111910 2024-05-09T00:11:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.355+0000 7fad0d4abb80 1 Processor -- start 2024-05-09T00:11:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.355+0000 7fad0d4abb80 1 -- start start 2024-05-09T00:11:20.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d540e00 0x555f2d5411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 0x555f2d53b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d53b890 0x555f2d53bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d492150 con 0x555f2d541720 2024-05-09T00:11:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d48fb10 con 0x555f2d53b890 2024-05-09T00:11:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facf3fff700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7face80a1290 0x7face80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d48b400 con 0x555f2d540e00 2024-05-09T00:11:20.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfa7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d540e00 0x555f2d5411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfa7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d540e00 0x555f2d5411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48586/0 (socket says 172.21.15.12:48586) 2024-05-09T00:11:20.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfa7fc700 1 -- 172.21.15.12:0/443960901 learned_addr learned my addr 172.21.15.12:0/443960901 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:20.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfa7fc700 1 --2- 172.21.15.12:0/443960901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d53b890 0x555f2d53bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facf3fff700 1 -- 172.21.15.12:0/1711043340 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7facf40ce0b0 con 0x7facfc111910 2024-05-09T00:11:20.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfaffd700 1 --2- 172.21.15.12:0/443960901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 0x555f2d53b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfa7fc700 1 -- 172.21.15.12:0/443960901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d53b890 msgr2=0x555f2d53bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfa7fc700 1 --2- 172.21.15.12:0/443960901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d53b890 0x555f2d53bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfb7fe700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7face80a1290 0x7face80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfa7fc700 1 -- 172.21.15.12:0/443960901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 msgr2=0x555f2d53b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfa7fc700 1 --2- 172.21.15.12:0/443960901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 0x555f2d53b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfa7fc700 1 -- 172.21.15.12:0/443960901 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f2d30c3b0 con 0x555f2d540e00 2024-05-09T00:11:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfaffd700 1 --2- 172.21.15.12:0/443960901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 0x555f2d53b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.356+0000 7facfa7fc700 1 --2- 172.21.15.12:0/443960901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d540e00 0x555f2d5411e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7facec011d00 tx=0x7facec0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=868b09de0a3fc9b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.357+0000 7facf17fa700 1 -- 172.21.15.12:0/443960901 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00f9b0 con 0x555f2d540e00 2024-05-09T00:11:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.357+0000 7facf17fa700 1 -- 172.21.15.12:0/443960901 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facec004050 con 0x555f2d540e00 2024-05-09T00:11:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.357+0000 7facf17fa700 1 -- 172.21.15.12:0/443960901 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec003a00 con 0x555f2d540e00 2024-05-09T00:11:20.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 -- 172.21.15.12:0/443960901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d540e00 msgr2=0x555f2d5411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7facfb7fe700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7face80a1290 0x7face80a3750 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7facf40a3120 tx=0x7facf40a1040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/443960901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d540e00 0x555f2d5411e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7facec011d00 tx=0x7facec0047c0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 -- 172.21.15.12:0/443960901 shutdown_connections 2024-05-09T00:11:20.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/443960901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d53b890 0x555f2d53bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/443960901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d540e00 0x555f2d5411e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/443960901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 0x555f2d53b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 -- 172.21.15.12:0/443960901 >> 172.21.15.12:0/443960901 conn(0x555f2d499210 msgr2=0x555f2d535df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 -- 172.21.15.12:0/443960901 shutdown_connections 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 -- 172.21.15.12:0/443960901 wait complete. 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 -- start start 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.358+0000 7fad0d4abb80 1 Processor -- start 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d53b890 0x555f2d544bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 0x555f2d545130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 0x555f2d545670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d427560 con 0x555f2d541720 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d427200 con 0x555f2d540e00 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d30e6a0 con 0x555f2d53b890 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7fad0d4abb80 10 client.0 _ll_register_callbacks cb 0x555f2d430b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:20.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7facfaffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 0x555f2d545130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7facfaffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 0x555f2d545130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42814/0 (socket says 172.21.15.12:42814) 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7facfaffd700 1 -- 172.21.15.12:0/1807438954 learned_addr learned my addr 172.21.15.12:0/1807438954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7facfa7fc700 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d53b890 0x555f2d544bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7facfa7fc700 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 0x555f2d545670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7facfaffd700 1 -- 172.21.15.12:0/1807438954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d53b890 msgr2=0x555f2d544bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7facfaffd700 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d53b890 0x555f2d544bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7facfaffd700 1 -- 172.21.15.12:0/1807438954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 msgr2=0x555f2d545670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7facfaffd700 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 0x555f2d545670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.359+0000 7facfaffd700 1 -- 172.21.15.12:0/1807438954 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec019040 con 0x555f2d540e00 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.360+0000 7facfaffd700 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 0x555f2d545130 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7face4013d40 tx=0x7face40116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=970ee228480a7fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.360+0000 7face27fc700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.360+0000 7face27fc700 1 -- 172.21.15.12:0/1807438954 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4004a30 con 0x555f2d540e00 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.360+0000 7fad0d4abb80 1 -- 172.21.15.12:0/1807438954 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f2d3e4270 con 0x555f2d540e00 2024-05-09T00:11:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.360+0000 7face27fc700 1 -- 172.21.15.12:0/1807438954 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7face4004b90 con 0x555f2d540e00 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.360+0000 7face27fc700 1 -- 172.21.15.12:0/1807438954 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7face4004d50 con 0x555f2d540e00 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.360+0000 7fad0d4abb80 10 client.31544 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.360+0000 7fad0d4abb80 1 -- 172.21.15.12:0/1807438954 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x555f2d2b8b60 con 0x555f2d540e00 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.360+0000 7face27fc700 1 -- 172.21.15.12:0/1807438954 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f2d3e4270 con 0x555f2d540e00 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.361+0000 7face27fc700 1 -- 172.21.15.12:0/1807438954 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7face401b730 con 0x555f2d540e00 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.361+0000 7fad0d4abb80 10 client.31544 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.361+0000 7fad0d4abb80 10 client.31544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.361+0000 7fad0d4abb80 20 client.31544 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.361+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555f2d5553b0 0x555f2d5577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.361+0000 7fad0d4abb80 4 client.31544 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.361+0000 7fad0d4abb80 1 -- 172.21.15.12:0/1807438954 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f2d50a810 con 0x555f2d5553b0 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.361+0000 7facfa7fc700 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555f2d5553b0 0x555f2d5577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.362+0000 7facfa7fc700 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555f2d5553b0 0x555f2d5577d0 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c85dd0b3ed175a server_cookie=9577081e1d921787 in_seq=0 out_seq=0 2024-05-09T00:11:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.362+0000 7face27fc700 10 client.31544 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.363+0000 7face27fc700 1 -- 172.21.15.12:0/1807438954 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f2d50a810 con 0x555f2d5553b0 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.363+0000 7face27fc700 10 client.31544 handle_command_reply: tid=0 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.364+0000 7fad0d4abb80 1 client.31544 shutdown 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.364+0000 7fad0d4abb80 1 -- 172.21.15.12:0/1807438954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 msgr2=0x555f2d545130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.364+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 0x555f2d545130 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7face4013d40 tx=0x7face40116c0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.364+0000 7fad0d4abb80 1 -- 172.21.15.12:0/1807438954 shutdown_connections 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.364+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d541720 0x555f2d545670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.364+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d53b890 0x555f2d544bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.364+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555f2d5553b0 0x555f2d5577d0 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.364+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/1807438954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 0x555f2d545130 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.364+0000 7fad0d4abb80 1 -- 172.21.15.12:0/1807438954 >> 172.21.15.12:0/1807438954 conn(0x555f2d499210 msgr2=0x555f2d5355a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.366+0000 7face27fc700 0 client.31544 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:20.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.366+0000 7fad0d4abb80 1 -- 172.21.15.12:0/1807438954 shutdown_connections 2024-05-09T00:11:20.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.366+0000 7fad0d4abb80 1 -- 172.21.15.12:0/1807438954 wait complete. 2024-05-09T00:11:20.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.368+0000 7fad0d4abb80 20 client.31544 trim_cache size 0 max 16384 2024-05-09T00:11:20.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.386+0000 7fad0d4abb80 1 Processor -- start 2024-05-09T00:11:20.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.386+0000 7fad0d4abb80 1 -- start start 2024-05-09T00:11:20.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d53b890 0x555f2d5444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d540e00 0x555f2d535d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d541720 0x555f2d54c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d537e60 con 0x555f2d540e00 2024-05-09T00:11:20.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d5368e0 con 0x555f2d53b890 2024-05-09T00:11:20.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d530800 con 0x555f2d541720 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7facfaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d540e00 0x555f2d535d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7facfaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d540e00 0x555f2d535d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50414/0 (socket says 172.21.15.12:50414) 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7facfaffd700 1 -- 172.21.15.12:0/3925499209 learned_addr learned my addr 172.21.15.12:0/3925499209 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7facfa7fc700 1 --2- 172.21.15.12:0/3925499209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d53b890 0x555f2d5444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7facfa7fc700 1 --2- 172.21.15.12:0/3925499209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d541720 0x555f2d54c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7facfaffd700 1 -- 172.21.15.12:0/3925499209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d541720 msgr2=0x555f2d54c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7facfaffd700 1 --2- 172.21.15.12:0/3925499209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d541720 0x555f2d54c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7facfaffd700 1 -- 172.21.15.12:0/3925499209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d53b890 msgr2=0x555f2d5444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7facfaffd700 1 --2- 172.21.15.12:0/3925499209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d53b890 0x555f2d5444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.387+0000 7facfaffd700 1 -- 172.21.15.12:0/3925499209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f2d451560 con 0x555f2d540e00 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.388+0000 7facfaffd700 1 --2- 172.21.15.12:0/3925499209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d540e00 0x555f2d535d20 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7face401bb90 tx=0x7face4009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a04002ab1786ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.388+0000 7facf1ffb700 1 -- 172.21.15.12:0/3925499209 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4011ac0 con 0x555f2d540e00 2024-05-09T00:11:20.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.388+0000 7facf1ffb700 1 -- 172.21.15.12:0/3925499209 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7face4013e90 con 0x555f2d540e00 2024-05-09T00:11:20.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.388+0000 7facf1ffb700 1 -- 172.21.15.12:0/3925499209 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4003680 con 0x555f2d540e00 2024-05-09T00:11:20.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.389+0000 7fad0d4abb80 1 -- 172.21.15.12:0/3925499209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d540e00 msgr2=0x555f2d535d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.389+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/3925499209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d540e00 0x555f2d535d20 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7face401bb90 tx=0x7face4009420 comp rx=0 tx=0).stop 2024-05-09T00:11:20.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.389+0000 7fad0d4abb80 1 -- 172.21.15.12:0/3925499209 shutdown_connections 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.389+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/3925499209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d541720 0x555f2d54c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.389+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/3925499209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d53b890 0x555f2d5444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.389+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/3925499209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d540e00 0x555f2d535d20 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.389+0000 7fad0d4abb80 1 -- 172.21.15.12:0/3925499209 >> 172.21.15.12:0/3925499209 conn(0x555f2d499210 msgr2=0x555f2d53ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.389+0000 7fad0d4abb80 1 -- 172.21.15.12:0/3925499209 shutdown_connections 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.389+0000 7fad0d4abb80 1 -- 172.21.15.12:0/3925499209 wait complete. 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.390+0000 7fad0d4abb80 1 -- start start 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.390+0000 7fad0d4abb80 1 Processor -- start 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.390+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d53b890 0x555f2d54a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.390+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 0x555f2d54a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.390+0000 7fad0d4abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d541720 0x555f2d54abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.390+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d5aaa70 con 0x555f2d53b890 2024-05-09T00:11:20.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.390+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d5aabe0 con 0x555f2d540e00 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.390+0000 7fad0d4abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2d5aada0 con 0x555f2d541720 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.390+0000 7fad0d4abb80 10 client.0 _ll_register_callbacks cb 0x555f2d430b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.390+0000 7facfa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d53b890 0x555f2d54a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7facfa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d53b890 0x555f2d54a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50426/0 (socket says 172.21.15.12:50426) 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7facfa7fc700 1 -- 172.21.15.12:0/2538638933 learned_addr learned my addr 172.21.15.12:0/2538638933 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7facfa7fc700 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d541720 0x555f2d54abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7facfa7fc700 1 -- 172.21.15.12:0/2538638933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d541720 msgr2=0x555f2d54abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7facfa7fc700 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d541720 0x555f2d54abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7facfa7fc700 1 -- 172.21.15.12:0/2538638933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 msgr2=0x555f2d54a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7facfa7fc700 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 0x555f2d54a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7facfa7fc700 1 -- 172.21.15.12:0/2538638933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f2d2b8b60 con 0x555f2d53b890 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7facfa7fc700 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d53b890 0x555f2d54a150 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x555f2d54b130 tx=0x7facec021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88287f9194feabc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7face27fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:20.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7face27fc700 1 -- 172.21.15.12:0/2538638933 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec0308e0 con 0x555f2d53b890 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.391+0000 7face27fc700 1 -- 172.21.15.12:0/2538638933 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7facec00f980 con 0x555f2d53b890 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7fad0d4abb80 1 -- 172.21.15.12:0/2538638933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facec004050 con 0x555f2d53b890 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7face27fc700 1 -- 172.21.15.12:0/2538638933 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7facec00fb40 con 0x555f2d53b890 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7fad0d4abb80 10 client.22182 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7fad0d4abb80 1 -- 172.21.15.12:0/2538638933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x555f2d310cd0 con 0x555f2d53b890 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7face27fc700 1 -- 172.21.15.12:0/2538638933 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facec004050 con 0x555f2d53b890 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7face27fc700 1 -- 172.21.15.12:0/2538638933 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7facec03f410 con 0x555f2d53b890 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7fad0d4abb80 10 client.22182 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7fad0d4abb80 10 client.22182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7fad0d4abb80 20 client.22182 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555f2d5ac390 0x555f2d5ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7fad0d4abb80 4 client.22182 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:11:20.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.392+0000 7fad0d4abb80 1 -- 172.21.15.12:0/2538638933 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555f2d483ec0 con 0x555f2d5ac390 2024-05-09T00:11:20.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.393+0000 7facfaffd700 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555f2d5ac390 0x555f2d5ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:20.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.393+0000 7face27fc700 10 client.22182 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:20.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.393+0000 7facfaffd700 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555f2d5ac390 0x555f2d5ae7b0 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd6c7563e184855 server_cookie=bc3ea67cf950e56a in_seq=0 out_seq=0 2024-05-09T00:11:20.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7face27fc700 1 -- 172.21.15.12:0/2538638933 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555f2d483ec0 con 0x555f2d5ac390 2024-05-09T00:11:20.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7face27fc700 10 client.22182 handle_command_reply: tid=0 2024-05-09T00:11:20.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 client.22182 shutdown 2024-05-09T00:11:20.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 -- 172.21.15.12:0/2538638933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d53b890 msgr2=0x555f2d54a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d53b890 0x555f2d54a150 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x555f2d54b130 tx=0x7facec021040 comp rx=0 tx=0).stop 2024-05-09T00:11:20.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 -- 172.21.15.12:0/2538638933 shutdown_connections 2024-05-09T00:11:20.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2d541720 0x555f2d54abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f2d53b890 0x555f2d54a150 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x555f2d5ac390 0x555f2d5ae7b0 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 --2- 172.21.15.12:0/2538638933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555f2d540e00 0x555f2d54a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 -- 172.21.15.12:0/2538638933 >> 172.21.15.12:0/2538638933 conn(0x555f2d499210 msgr2=0x555f2d53fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7face27fc700 0 client.22182 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:20.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 -- 172.21.15.12:0/2538638933 shutdown_connections 2024-05-09T00:11:20.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.396+0000 7fad0d4abb80 1 -- 172.21.15.12:0/2538638933 wait complete. 2024-05-09T00:11:20.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad0d4abb80 20 client.22182 trim_cache size 0 max 16384 2024-05-09T00:11:20.396 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:11:20.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad012f1700 1 -- 172.21.15.12:0/1711043340 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7face80a1290 msgr2=0x7face80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad012f1700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7face80a1290 0x7face80a3750 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7facf40a3120 tx=0x7facf40a1040 comp rx=0 tx=0).stop 2024-05-09T00:11:20.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad012f1700 1 -- 172.21.15.12:0/1711043340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc111910 msgr2=0x7facfc115ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:20.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad012f1700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc111910 0x7facfc115ab0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7facf400b6c0 tx=0x7facf4008fb0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad012f1700 1 -- 172.21.15.12:0/1711043340 shutdown_connections 2024-05-09T00:11:20.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad012f1700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7face80a1290 0x7face80a3750 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad012f1700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc111910 0x7facfc115ab0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad012f1700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facfc10b910 0x7facfc118480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad012f1700 1 --2- 172.21.15.12:0/1711043340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7facfc10c2d0 0x7facfc115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:20.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.397+0000 7fad012f1700 1 -- 172.21.15.12:0/1711043340 >> 172.21.15.12:0/1711043340 conn(0x7facfc06bf00 msgr2=0x7facfc10e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:20.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.401+0000 7fad012f1700 1 -- 172.21.15.12:0/1711043340 shutdown_connections 2024-05-09T00:11:20.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:20.401+0000 7fad012f1700 1 -- 172.21.15.12:0/1711043340 wait complete. 2024-05-09T00:11:20.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:20 smithi204 ceph-mon[47373]: pgmap v803: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 544 op/s 2024-05-09T00:11:20.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:20 smithi160 ceph-mon[46152]: pgmap v803: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 544 op/s 2024-05-09T00:11:20.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:20 smithi012 ceph-mon[42503]: pgmap v803: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 544 op/s 2024-05-09T00:11:21.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 374285 27.87 MB/sec execute 556 sec latency 81.744 ms 2024-05-09T00:11:21.422 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:11:21.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:11:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.603+0000 7f0e767fb700 1 -- 172.21.15.12:0/2500734829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e7010db90 msgr2=0x7f0e7010ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.603+0000 7f0e767fb700 1 --2- 172.21.15.12:0/2500734829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e7010db90 0x7f0e7010ff80 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009990 tx=0x7f0e68009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.608+0000 7f0e767fb700 1 -- 172.21.15.12:0/2500734829 shutdown_connections 2024-05-09T00:11:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.608+0000 7f0e767fb700 1 --2- 172.21.15.12:0/2500734829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e7010db90 0x7f0e7010ff80 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.608+0000 7f0e767fb700 1 --2- 172.21.15.12:0/2500734829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e70102010 0x7f0e700ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.608+0000 7f0e767fb700 1 --2- 172.21.15.12:0/2500734829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e700ff950 0x7f0e700ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.608+0000 7f0e767fb700 1 -- 172.21.15.12:0/2500734829 >> 172.21.15.12:0/2500734829 conn(0x7f0e700763a0 msgr2=0x7f0e700fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.608+0000 7f0e767fb700 1 -- 172.21.15.12:0/2500734829 shutdown_connections 2024-05-09T00:11:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.609+0000 7f0e767fb700 1 -- 172.21.15.12:0/2500734829 wait complete. 2024-05-09T00:11:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.609+0000 7f0e767fb700 1 Processor -- start 2024-05-09T00:11:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.609+0000 7f0e767fb700 1 -- start start 2024-05-09T00:11:21.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.610+0000 7f0e767fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e700ff950 0x7f0e7019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.610+0000 7f0e767fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e70102010 0x7f0e7019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.610+0000 7f0e767fb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e7010db90 0x7f0e70199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.610+0000 7f0e767fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70199cd0 con 0x7f0e70102010 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.610+0000 7f0e767fb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70199e40 con 0x7f0e700ff950 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.610+0000 7f0e767fb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7019a000 con 0x7f0e7010db90 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.610+0000 7f0e6f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e70102010 0x7f0e7019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.610+0000 7f0e6ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e700ff950 0x7f0e7019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.610+0000 7f0e6f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e70102010 0x7f0e7019fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50442/0 (socket says 172.21.15.12:50442) 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.610+0000 7f0e6f7fe700 1 -- 172.21.15.12:0/3473238568 learned_addr learned my addr 172.21.15.12:0/3473238568 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e6f7fe700 1 -- 172.21.15.12:0/3473238568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e7010db90 msgr2=0x7f0e70199670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:11:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e7010db90 0x7f0e70199670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e6f7fe700 1 -- 172.21.15.12:0/3473238568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e700ff950 msgr2=0x7f0e7019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e700ff950 0x7f0e7019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e6f7fe700 1 -- 172.21.15.12:0/3473238568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e680095f0 con 0x7f0e70102010 2024-05-09T00:11:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e700ff950 0x7f0e7019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e70102010 0x7f0e7019fa80 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b530 tx=0x7f0e5800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e6d7fa700 1 -- 172.21.15.12:0/3473238568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58011830 con 0x7f0e70102010 2024-05-09T00:11:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e6d7fa700 1 -- 172.21.15.12:0/3473238568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e58011e50 con 0x7f0e70102010 2024-05-09T00:11:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e6d7fa700 1 -- 172.21.15.12:0/3473238568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800f2e0 con 0x7f0e70102010 2024-05-09T00:11:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e767fb700 1 -- 172.21.15.12:0/3473238568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7019a2e0 con 0x7f0e70102010 2024-05-09T00:11:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.611+0000 7f0e767fb700 1 -- 172.21.15.12:0/3473238568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e701a6530 con 0x7f0e70102010 2024-05-09T00:11:21.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.615+0000 7f0e6d7fa700 1 -- 172.21.15.12:0/3473238568 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0e5800f440 con 0x7f0e70102010 2024-05-09T00:11:21.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.615+0000 7f0e6d7fa700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e5c0a1310 0x7f0e5c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.616+0000 7f0e6d7fa700 1 -- 172.21.15.12:0/3473238568 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0e580cc7d0 con 0x7f0e70102010 2024-05-09T00:11:21.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.616+0000 7f0e829b5b80 1 Processor -- start 2024-05-09T00:11:21.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.616+0000 7f0e829b5b80 1 -- start start 2024-05-09T00:11:21.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.616+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7844ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c7845000 0x55c5c783ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c783f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c778d1d0 con 0x55c5c7845000 2024-05-09T00:11:21.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c778ab90 con 0x55c5c783f170 2024-05-09T00:11:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7786480 con 0x55c5c78446e0 2024-05-09T00:11:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e6ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7844ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e74d98700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c783f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e6ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7844ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48644/0 (socket says 172.21.15.12:48644) 2024-05-09T00:11:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e6ffff700 1 -- 172.21.15.12:0/786403093 learned_addr learned my addr 172.21.15.12:0/786403093 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e6ffff700 1 -- 172.21.15.12:0/786403093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 msgr2=0x55c5c783f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/786403093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c783f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e6ffff700 1 -- 172.21.15.12:0/786403093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c7845000 msgr2=0x55c5c783ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/786403093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c7845000 0x55c5c783ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.617+0000 7f0e6ffff700 1 -- 172.21.15.12:0/786403093 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5c7607430 con 0x55c5c78446e0 2024-05-09T00:11:21.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.622+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/786403093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7844ac0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000b970 tx=0x7f0e6000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ba64b1a9682f202 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.622+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e5c0a1310 0x7f0e5c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.622+0000 7f0e667fc700 1 -- 172.21.15.12:0/786403093 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60016410 con 0x55c5c78446e0 2024-05-09T00:11:21.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.623+0000 7f0e667fc700 1 -- 172.21.15.12:0/786403093 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e60016a30 con 0x55c5c78446e0 2024-05-09T00:11:21.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.623+0000 7f0e667fc700 1 -- 172.21.15.12:0/786403093 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60015c10 con 0x55c5c78446e0 2024-05-09T00:11:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e5c0a1310 0x7f0e5c0a37d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0e6001d000 tx=0x7f0e60015f60 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e829b5b80 1 -- 172.21.15.12:0/786403093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 msgr2=0x55c5c7844ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/786403093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7844ac0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000b970 tx=0x7f0e6000bc80 comp rx=0 tx=0).stop 2024-05-09T00:11:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e829b5b80 1 -- 172.21.15.12:0/786403093 shutdown_connections 2024-05-09T00:11:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/786403093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c783f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/786403093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7844ac0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/786403093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c7845000 0x55c5c783ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e829b5b80 1 -- 172.21.15.12:0/786403093 >> 172.21.15.12:0/786403093 conn(0x55c5c7794290 msgr2=0x55c5c7830e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e829b5b80 1 -- 172.21.15.12:0/786403093 shutdown_connections 2024-05-09T00:11:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e829b5b80 1 -- 172.21.15.12:0/786403093 wait complete. 2024-05-09T00:11:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.624+0000 7f0e829b5b80 1 -- start start 2024-05-09T00:11:21.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.625+0000 7f0e829b5b80 1 Processor -- start 2024-05-09T00:11:21.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.625+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c7848b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.625+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7849040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.625+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c7845000 0x55c5c783d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.625+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c77225e0 con 0x55c5c7845000 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.625+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7722280 con 0x55c5c783f170 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.625+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7609720 con 0x55c5c78446e0 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.625+0000 7f0e829b5b80 10 client.0 _ll_register_callbacks cb 0x55c5c772bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c7848b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c7848b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42860/0 (socket says 172.21.15.12:42860) 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6ffff700 1 -- 172.21.15.12:0/3663242799 learned_addr learned my addr 172.21.15.12:0/3663242799 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7849040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6ffff700 1 -- 172.21.15.12:0/3663242799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 msgr2=0x55c5c7849040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7849040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6ffff700 1 -- 172.21.15.12:0/3663242799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c7845000 msgr2=0x55c5c783d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c7845000 0x55c5c783d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6ffff700 1 -- 172.21.15.12:0/3663242799 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6000b630 con 0x55c5c783f170 2024-05-09T00:11:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7849040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:21.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c7848b00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0e60000f80 tx=0x7f0e60015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=229558e46ba904ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.626+0000 7f0e4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:11:21.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.627+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/3663242799 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600154b0 con 0x55c5c783f170 2024-05-09T00:11:21.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.627+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/3663242799 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0e60016410 con 0x55c5c783f170 2024-05-09T00:11:21.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.627+0000 7f0e829b5b80 1 -- 172.21.15.12:0/3663242799 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5c76df2f0 con 0x55c5c783f170 2024-05-09T00:11:21.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.627+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/3663242799 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e600165d0 con 0x55c5c783f170 2024-05-09T00:11:21.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.627+0000 7f0e829b5b80 10 client.31556 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:21.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.627+0000 7f0e829b5b80 1 -- 172.21.15.12:0/3663242799 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55c5c75b3be0 con 0x55c5c783f170 2024-05-09T00:11:21.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.628+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/3663242799 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5c76df2f0 con 0x55c5c783f170 2024-05-09T00:11:21.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.629+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/3663242799 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f0e6000b9a0 con 0x55c5c783f170 2024-05-09T00:11:21.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.629+0000 7f0e829b5b80 10 client.31556 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:21.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.629+0000 7f0e829b5b80 10 client.31556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:21.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.629+0000 7f0e829b5b80 20 client.31556 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:21.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.629+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c5c78610b0 0x55c5c78634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.629+0000 7f0e829b5b80 4 client.31556 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:21.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.629+0000 7f0e829b5b80 1 -- 172.21.15.12:0/3663242799 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5c7805890 con 0x55c5c78610b0 2024-05-09T00:11:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.636+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c5c78610b0 0x55c5c78634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.637+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c5c78610b0 0x55c5c78634d0 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82890a995c0bacaa server_cookie=1d6debff418a5632 in_seq=0 out_seq=0 2024-05-09T00:11:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.637+0000 7f0e4f7fe700 10 client.31556 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.637+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/3663242799 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5c7805890 con 0x55c5c78610b0 2024-05-09T00:11:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.637+0000 7f0e4f7fe700 10 client.31556 handle_command_reply: tid=0 2024-05-09T00:11:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.638+0000 7f0e829b5b80 1 client.31556 shutdown 2024-05-09T00:11:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.638+0000 7f0e829b5b80 1 -- 172.21.15.12:0/3663242799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 msgr2=0x55c5c7848b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.638+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c7848b00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0e60000f80 tx=0x7f0e60015000 comp rx=0 tx=0).stop 2024-05-09T00:11:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.639+0000 7f0e829b5b80 1 -- 172.21.15.12:0/3663242799 shutdown_connections 2024-05-09T00:11:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.639+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c7845000 0x55c5c783d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.639+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c5c78610b0 0x55c5c78634d0 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.639+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c7848b00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.639+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/3663242799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7849040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.639+0000 7f0e4f7fe700 0 client.31556 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.639+0000 7f0e829b5b80 1 -- 172.21.15.12:0/3663242799 >> 172.21.15.12:0/3663242799 conn(0x55c5c7794290 msgr2=0x55c5c7830620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.639+0000 7f0e829b5b80 1 -- 172.21.15.12:0/3663242799 shutdown_connections 2024-05-09T00:11:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.639+0000 7f0e829b5b80 1 -- 172.21.15.12:0/3663242799 wait complete. 2024-05-09T00:11:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.639+0000 7f0e829b5b80 20 client.31556 trim_cache size 0 max 16384 2024-05-09T00:11:21.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e829b5b80 1 Processor -- start 2024-05-09T00:11:21.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e829b5b80 1 -- start start 2024-05-09T00:11:21.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c78463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c78446e0 0x55c5c7848ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c7845000 0x55c5c7830da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7846930 con 0x55c5c78446e0 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7868720 con 0x55c5c783f170 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7843fb0 con 0x55c5c7845000 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e6f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c78446e0 0x55c5c7848ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e6f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c78446e0 0x55c5c7848ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50454/0 (socket says 172.21.15.12:50454) 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e6f7fe700 1 -- 172.21.15.12:0/322582918 learned_addr learned my addr 172.21.15.12:0/322582918 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e74d98700 1 --2- 172.21.15.12:0/322582918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c7845000 0x55c5c7830da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/322582918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c78463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e6f7fe700 1 -- 172.21.15.12:0/322582918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c7845000 msgr2=0x55c5c7830da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/322582918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c7845000 0x55c5c7830da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e6f7fe700 1 -- 172.21.15.12:0/322582918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 msgr2=0x55c5c78463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/322582918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c78463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.670+0000 7f0e6f7fe700 1 -- 172.21.15.12:0/322582918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5c782d560 con 0x55c5c78446e0 2024-05-09T00:11:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.671+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/322582918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c78446e0 0x55c5c7848ea0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f0e58099930 tx=0x7f0e58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a859871afa0698ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.671+0000 7f0e66ffd700 1 -- 172.21.15.12:0/322582918 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5809d7d0 con 0x55c5c78446e0 2024-05-09T00:11:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.671+0000 7f0e66ffd700 1 -- 172.21.15.12:0/322582918 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e580a1e20 con 0x55c5c78446e0 2024-05-09T00:11:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.671+0000 7f0e66ffd700 1 -- 172.21.15.12:0/322582918 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58018570 con 0x55c5c78446e0 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.672+0000 7f0e829b5b80 1 -- 172.21.15.12:0/322582918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c78446e0 msgr2=0x55c5c7848ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.672+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/322582918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c78446e0 0x55c5c7848ea0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f0e58099930 tx=0x7f0e58004750 comp rx=0 tx=0).stop 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.672+0000 7f0e829b5b80 1 -- 172.21.15.12:0/322582918 shutdown_connections 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.672+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/322582918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c7845000 0x55c5c7830da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.672+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/322582918 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c783f170 0x55c5c78463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.672+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/322582918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c78446e0 0x55c5c7848ea0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.672+0000 7f0e829b5b80 1 -- 172.21.15.12:0/322582918 >> 172.21.15.12:0/322582918 conn(0x55c5c7794290 msgr2=0x55c5c783d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.672+0000 7f0e829b5b80 1 -- 172.21.15.12:0/322582918 shutdown_connections 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.672+0000 7f0e829b5b80 1 -- 172.21.15.12:0/322582918 wait complete. 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.673+0000 7f0e829b5b80 1 -- start start 2024-05-09T00:11:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.673+0000 7f0e829b5b80 1 Processor -- start 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.673+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c783f170 0x55c5c78399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.673+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7839f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.673+0000 7f0e829b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c7845000 0x55c5c78b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.673+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c78b7b40 con 0x55c5c783f170 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.673+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c78b7c80 con 0x55c5c7845000 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.673+0000 7f0e829b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c78b7df0 con 0x55c5c78446e0 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.673+0000 7f0e829b5b80 10 client.0 _ll_register_callbacks cb 0x55c5c772bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e6ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c783f170 0x55c5c78399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e6ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c783f170 0x55c5c78399e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50456/0 (socket says 172.21.15.12:50456) 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e6ffff700 1 -- 172.21.15.12:0/4154089500 learned_addr learned my addr 172.21.15.12:0/4154089500 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e74d98700 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c7845000 0x55c5c78b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e6ffff700 1 -- 172.21.15.12:0/4154089500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 msgr2=0x55c5c7839f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7839f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7839f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e6ffff700 1 -- 172.21.15.12:0/4154089500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c7845000 msgr2=0x55c5c78b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c7845000 0x55c5c78b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e6ffff700 1 -- 172.21.15.12:0/4154089500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e701a6530 con 0x55c5c783f170 2024-05-09T00:11:21.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e74d98700 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c7845000 0x55c5c78b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:21.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.674+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7839f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:21.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.675+0000 7f0e6ffff700 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c783f170 0x55c5c78399e0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f0e60016fd0 tx=0x7f0e60015650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6606e3e1056a22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.675+0000 7f0e4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.675+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/4154089500 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000bb00 con 0x55c5c783f170 2024-05-09T00:11:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.675+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/4154089500 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0e60016410 con 0x55c5c783f170 2024-05-09T00:11:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.675+0000 7f0e829b5b80 1 -- 172.21.15.12:0/4154089500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e60016a30 con 0x55c5c783f170 2024-05-09T00:11:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.675+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/4154089500 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e600165d0 con 0x55c5c783f170 2024-05-09T00:11:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.675+0000 7f0e829b5b80 10 client.22194 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.675+0000 7f0e829b5b80 1 -- 172.21.15.12:0/4154089500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55c5c774c5e0 con 0x55c5c783f170 2024-05-09T00:11:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.676+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/4154089500 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e60016a30 con 0x55c5c783f170 2024-05-09T00:11:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.676+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/4154089500 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f0e60014c60 con 0x55c5c783f170 2024-05-09T00:11:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.676+0000 7f0e829b5b80 10 client.22194 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.676+0000 7f0e829b5b80 10 client.22194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.676+0000 7f0e829b5b80 20 client.22194 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.676+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c5c78b92c0 0x55c5c78bb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.676+0000 7f0e829b5b80 4 client.22194 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:11:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.676+0000 7f0e829b5b80 1 -- 172.21.15.12:0/4154089500 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c5c777ef40 con 0x55c5c78b92c0 2024-05-09T00:11:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.677+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c5c78b92c0 0x55c5c78bb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.677+0000 7f0e6f7fe700 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c5c78b92c0 0x55c5c78bb6e0 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efadba20ae7a36e1 server_cookie=d6f9b7de6d496ed1 in_seq=0 out_seq=0 2024-05-09T00:11:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.677+0000 7f0e4f7fe700 10 client.22194 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.682+0000 7f0e4f7fe700 1 -- 172.21.15.12:0/4154089500 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c5c777ef40 con 0x55c5c78b92c0 2024-05-09T00:11:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.682+0000 7f0e4f7fe700 10 client.22194 handle_command_reply: tid=0 2024-05-09T00:11:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.682+0000 7f0e829b5b80 1 client.22194 shutdown 2024-05-09T00:11:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 1 -- 172.21.15.12:0/4154089500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c783f170 msgr2=0x55c5c78399e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c783f170 0x55c5c78399e0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f0e60016fd0 tx=0x7f0e60015650 comp rx=0 tx=0).stop 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 1 -- 172.21.15.12:0/4154089500 shutdown_connections 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55c5c78b92c0 0x55c5c78bb6e0 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5c7845000 0x55c5c78b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5c783f170 0x55c5c78399e0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e4f7fe700 0 client.22194 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 1 --2- 172.21.15.12:0/4154089500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5c78446e0 0x55c5c7839f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 1 -- 172.21.15.12:0/4154089500 >> 172.21.15.12:0/4154089500 conn(0x55c5c7794290 msgr2=0x55c5c7847150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 1 -- 172.21.15.12:0/4154089500 shutdown_connections 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 1 -- 172.21.15.12:0/4154089500 wait complete. 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.683+0000 7f0e829b5b80 20 client.22194 trim_cache size 0 max 16384 2024-05-09T00:11:21.681 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:11:21.682 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:11:21.682 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "1cae3284-665b-43b5-ab8e-897ff80a78c6", 2024-05-09T00:11:21.682 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:11:21.682 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 -- 172.21.15.12:0/3473238568 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e5c0a1310 msgr2=0x7f0e5c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e5c0a1310 0x7f0e5c0a37d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0e6001d000 tx=0x7f0e60015f60 comp rx=0 tx=0).stop 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 -- 172.21.15.12:0/3473238568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e70102010 msgr2=0x7f0e7019fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e70102010 0x7f0e7019fa80 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b530 tx=0x7f0e5800b840 comp rx=0 tx=0).stop 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 -- 172.21.15.12:0/3473238568 shutdown_connections 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0e5c0a1310 0x7f0e5c0a37d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e7010db90 0x7f0e70199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e700ff950 0x7f0e7019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 --2- 172.21.15.12:0/3473238568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e70102010 0x7f0e7019fa80 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 -- 172.21.15.12:0/3473238568 >> 172.21.15.12:0/3473238568 conn(0x7f0e700763a0 msgr2=0x7f0e7010f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 -- 172.21.15.12:0/3473238568 shutdown_connections 2024-05-09T00:11:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.684+0000 7f0e767fb700 1 -- 172.21.15.12:0/3473238568 wait complete. 2024-05-09T00:11:21.696 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:11:21.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.878+0000 7f46a0320700 1 -- 172.21.15.12:0/359444036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4698069e90 msgr2=0x7f4698105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.878+0000 7f46a0320700 1 --2- 172.21.15.12:0/359444036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4698069e90 0x7f4698105b60 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f46880099c0 tx=0x7f4688009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.878+0000 7f46a0320700 1 -- 172.21.15.12:0/359444036 shutdown_connections 2024-05-09T00:11:21.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.878+0000 7f46a0320700 1 --2- 172.21.15.12:0/359444036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4698106290 0x7f469810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.878+0000 7f46a0320700 1 --2- 172.21.15.12:0/359444036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46980694e0 0x7f46980698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.878+0000 7f46a0320700 1 --2- 172.21.15.12:0/359444036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4698069e90 0x7f4698105b60 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.878+0000 7f46a0320700 1 -- 172.21.15.12:0/359444036 >> 172.21.15.12:0/359444036 conn(0x7f4698076f30 msgr2=0x7f4698077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.878+0000 7f46a0320700 1 -- 172.21.15.12:0/359444036 shutdown_connections 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.878+0000 7f46a0320700 1 -- 172.21.15.12:0/359444036 wait complete. 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.879+0000 7f46a0320700 1 Processor -- start 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.879+0000 7f46a0320700 1 -- start start 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f46a0320700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46980694e0 0x7f4698197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f46a0320700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4698106290 0x7f46981978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f46a0320700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f469819bc70 0x7f469819c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f46a0320700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469819c6f0 con 0x7f46980694e0 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f46a0320700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469819c860 con 0x7f469819bc70 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f46a0320700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469819ca20 con 0x7f4698106290 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f469d8bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4698106290 0x7f46981978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f469d8bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4698106290 0x7f46981978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48708/0 (socket says 172.21.15.12:48708) 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f469d8bb700 1 -- 172.21.15.12:0/1883896785 learned_addr learned my addr 172.21.15.12:0/1883896785 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f469e0bc700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46980694e0 0x7f4698197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f469e8bd700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f469819bc70 0x7f469819c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f469d8bb700 1 -- 172.21.15.12:0/1883896785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f469819bc70 msgr2=0x7f469819c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f469d8bb700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f469819bc70 0x7f469819c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f469d8bb700 1 -- 172.21.15.12:0/1883896785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46980694e0 msgr2=0x7f4698197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f469d8bb700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46980694e0 0x7f4698197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.880+0000 7f469d8bb700 1 -- 172.21.15.12:0/1883896785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880095f0 con 0x7f4698106290 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.881+0000 7f469d8bb700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4698106290 0x7f46981978d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f468800b580 tx=0x7f468800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.881+0000 7f468f7fe700 1 -- 172.21.15.12:0/1883896785 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468801c070 con 0x7f4698106290 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.881+0000 7f468f7fe700 1 -- 172.21.15.12:0/1883896785 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f468800fcc0 con 0x7f4698106290 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.881+0000 7f46a0320700 1 -- 172.21.15.12:0/1883896785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46981a3f20 con 0x7f4698106290 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.881+0000 7f468f7fe700 1 -- 172.21.15.12:0/1883896785 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688022bf0 con 0x7f4698106290 2024-05-09T00:11:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.881+0000 7f46a0320700 1 -- 172.21.15.12:0/1883896785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46981a42e0 con 0x7f4698106290 2024-05-09T00:11:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.884+0000 7f468f7fe700 1 -- 172.21.15.12:0/1883896785 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4688020070 con 0x7f4698106290 2024-05-09T00:11:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.885+0000 7f468f7fe700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46840a1290 0x7f46840a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.885+0000 7f468f7fe700 1 -- 172.21.15.12:0/1883896785 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f46880cdbe0 con 0x7f4698106290 2024-05-09T00:11:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.886+0000 7f46ac4dab80 1 Processor -- start 2024-05-09T00:11:21.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f46ac4dab80 1 -- start start 2024-05-09T00:11:21.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f920150 con 0x55829f9cee00 2024-05-09T00:11:21.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f91db10 con 0x55829f9c9890 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f919400 con 0x55829f9cf720 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f469e8bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f469d8bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f469e8bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42912/0 (socket says 172.21.15.12:42912) 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.887+0000 7f469e8bd700 1 -- 172.21.15.12:0/1755304358 learned_addr learned my addr 172.21.15.12:0/1755304358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.888+0000 7f469d8bb700 1 -- 172.21.15.12:0/1755304358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 msgr2=0x55829f9c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.888+0000 7f469d8bb700 1 --2- 172.21.15.12:0/1755304358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.888+0000 7f469d8bb700 1 -- 172.21.15.12:0/1755304358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 msgr2=0x55829f9cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.888+0000 7f469d8bb700 1 --2- 172.21.15.12:0/1755304358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.888+0000 7f469d8bb700 1 -- 172.21.15.12:0/1755304358 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55829f79a3b0 con 0x55829f9cf720 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.888+0000 7f469d8bb700 1 --2- 172.21.15.12:0/1755304358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9c9350 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f46880cb340 tx=0x7f468809a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb0054b9d9770c7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.888+0000 7f468cda4700 1 -- 172.21.15.12:0/1755304358 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468809ad90 con 0x55829f9cf720 2024-05-09T00:11:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.888+0000 7f468cda4700 1 -- 172.21.15.12:0/1755304358 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f468809ebc0 con 0x55829f9cf720 2024-05-09T00:11:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.888+0000 7f468cda4700 1 -- 172.21.15.12:0/1755304358 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468809a280 con 0x55829f9cf720 2024-05-09T00:11:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.889+0000 7f469e0bc700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46840a1290 0x7f46840a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.889+0000 7f469e0bc700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46840a1290 0x7f46840a3750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4694009df0 tx=0x7f4694009970 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.889+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1755304358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 msgr2=0x55829f9c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.889+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/1755304358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9c9350 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f46880cb340 tx=0x7f468809a8d0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.890+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1755304358 shutdown_connections 2024-05-09T00:11:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.890+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/1755304358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.890+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/1755304358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.890+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/1755304358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9c9350 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.890+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1755304358 >> 172.21.15.12:0/1755304358 conn(0x55829f927210 msgr2=0x55829f9c3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.890+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1755304358 shutdown_connections 2024-05-09T00:11:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.892+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1755304358 wait complete. 2024-05-09T00:11:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.892+0000 7f46ac4dab80 1 -- start start 2024-05-09T00:11:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.892+0000 7f46ac4dab80 1 Processor -- start 2024-05-09T00:11:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.893+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.893+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.893+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.893+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f8b5560 con 0x55829f9cee00 2024-05-09T00:11:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.893+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f8b5200 con 0x55829f9c9890 2024-05-09T00:11:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.893+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f79c6a0 con 0x55829f9cf720 2024-05-09T00:11:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.893+0000 7f46ac4dab80 10 client.0 _ll_register_callbacks cb 0x55829f8beb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.893+0000 7f469e8bd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.893+0000 7f469e8bd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9d31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48714/0 (socket says 172.21.15.12:48714) 2024-05-09T00:11:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.893+0000 7f469e8bd700 1 -- 172.21.15.12:0/3051322425 learned_addr learned my addr 172.21.15.12:0/3051322425 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.894+0000 7f469e8bd700 1 -- 172.21.15.12:0/3051322425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 msgr2=0x55829f9d5750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.894+0000 7f469e8bd700 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9d5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.894+0000 7f469e8bd700 1 -- 172.21.15.12:0/3051322425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 msgr2=0x55829f9d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.894+0000 7f469e8bd700 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.894+0000 7f469e8bd700 1 -- 172.21.15.12:0/3051322425 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46981a42e0 con 0x55829f9cf720 2024-05-09T00:11:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.894+0000 7f469e8bd700 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9d31d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f46900047a0 tx=0x7f4690014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=792fb95473a3947a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.894+0000 7f4681ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.894+0000 7f4681ffb700 1 -- 172.21.15.12:0/3051322425 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469000e040 con 0x55829f9cf720 2024-05-09T00:11:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.894+0000 7f4681ffb700 1 -- 172.21.15.12:0/3051322425 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f469001a070 con 0x55829f9cf720 2024-05-09T00:11:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.894+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3051322425 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55829f872270 con 0x55829f9cf720 2024-05-09T00:11:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f4681ffb700 1 -- 172.21.15.12:0/3051322425 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4690014b90 con 0x55829f9cf720 2024-05-09T00:11:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f46ac4dab80 10 client.31870 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3051322425 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55829f746b60 con 0x55829f9cf720 2024-05-09T00:11:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f4681ffb700 1 -- 172.21.15.12:0/3051322425 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55829f872270 con 0x55829f9cf720 2024-05-09T00:11:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f4681ffb700 1 -- 172.21.15.12:0/3051322425 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4690028420 con 0x55829f9cf720 2024-05-09T00:11:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f46ac4dab80 10 client.31870 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f46ac4dab80 10 client.31870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f46ac4dab80 20 client.31870 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55829f9edf70 0x55829f9f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f46ac4dab80 4 client.31870 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.895+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3051322425 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55829f998810 con 0x55829f9edf70 2024-05-09T00:11:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.897+0000 7f469e0bc700 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55829f9edf70 0x55829f9f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.898+0000 7f469e0bc700 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55829f9edf70 0x55829f9f03f0 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dec100e12748073 server_cookie=2749ff1338eb2017 in_seq=0 out_seq=0 2024-05-09T00:11:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.898+0000 7f4681ffb700 10 client.31870 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.904+0000 7f4681ffb700 1 -- 172.21.15.12:0/3051322425 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55829f998810 con 0x55829f9edf70 2024-05-09T00:11:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.904+0000 7f4681ffb700 10 client.31870 handle_command_reply: tid=0 2024-05-09T00:11:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.904+0000 7f46ac4dab80 1 client.31870 shutdown 2024-05-09T00:11:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3051322425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 msgr2=0x55829f9d31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9d31d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f46900047a0 tx=0x7f4690014600 comp rx=0 tx=0).stop 2024-05-09T00:11:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3051322425 shutdown_connections 2024-05-09T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9d31d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55829f9edf70 0x55829f9f03f0 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/3051322425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9d2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3051322425 >> 172.21.15.12:0/3051322425 conn(0x55829f927210 msgr2=0x55829f9c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f4681ffb700 0 client.31870 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3051322425 shutdown_connections 2024-05-09T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3051322425 wait complete. 2024-05-09T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.906+0000 7f46ac4dab80 20 client.31870 trim_cache size 0 max 16384 2024-05-09T00:11:21.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f46ac4dab80 1 Processor -- start 2024-05-09T00:11:21.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f46ac4dab80 1 -- start start 2024-05-09T00:11:21.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9c9890 0x55829f9c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9cee00 0x55829f9c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cf720 0x55829f9d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f9eb590 con 0x55829f9cf720 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829fa0f3d0 con 0x55829f9cee00 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f9d0fe0 con 0x55829f9c9890 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f469e8bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cf720 0x55829f9d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f469e8bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cf720 0x55829f9d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50480/0 (socket says 172.21.15.12:50480) 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f469d8bb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9cee00 0x55829f9c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.928+0000 7f469e8bd700 1 -- 172.21.15.12:0/3857394201 learned_addr learned my addr 172.21.15.12:0/3857394201 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.929+0000 7f469e8bd700 1 -- 172.21.15.12:0/3857394201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9c9890 msgr2=0x55829f9c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.929+0000 7f469e8bd700 1 --2- 172.21.15.12:0/3857394201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9c9890 0x55829f9c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.929+0000 7f469e8bd700 1 -- 172.21.15.12:0/3857394201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9cee00 msgr2=0x55829f9c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.929+0000 7f469e8bd700 1 --2- 172.21.15.12:0/3857394201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9cee00 0x55829f9c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.929+0000 7f469e8bd700 1 -- 172.21.15.12:0/3857394201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55829f9c9f40 con 0x55829f9cf720 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.929+0000 7f469e8bd700 1 --2- 172.21.15.12:0/3857394201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cf720 0x55829f9d1360 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f469000b010 tx=0x7f4690003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f5dcaa61001e95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.929+0000 7f468d5a5700 1 -- 172.21.15.12:0/3857394201 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469000e030 con 0x55829f9cf720 2024-05-09T00:11:21.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.929+0000 7f468d5a5700 1 -- 172.21.15.12:0/3857394201 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469000f040 con 0x55829f9cf720 2024-05-09T00:11:21.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.929+0000 7f468d5a5700 1 -- 172.21.15.12:0/3857394201 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690014b90 con 0x55829f9cf720 2024-05-09T00:11:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.930+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3857394201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cf720 msgr2=0x55829f9d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.930+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/3857394201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cf720 0x55829f9d1360 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f469000b010 tx=0x7f4690003c10 comp rx=0 tx=0).stop 2024-05-09T00:11:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.930+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3857394201 shutdown_connections 2024-05-09T00:11:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.930+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/3857394201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cf720 0x55829f9d1360 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.930+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/3857394201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9c9890 0x55829f9c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.930+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/3857394201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9cee00 0x55829f9c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.930+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3857394201 >> 172.21.15.12:0/3857394201 conn(0x55829f927210 msgr2=0x55829f9ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.930+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3857394201 shutdown_connections 2024-05-09T00:11:21.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.930+0000 7f46ac4dab80 1 -- 172.21.15.12:0/3857394201 wait complete. 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f46ac4dab80 1 -- start start 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f46ac4dab80 1 Processor -- start 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f46ac4dab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f9f32c0 con 0x55829f9cee00 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f9f3430 con 0x55829f9c9890 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f46ac4dab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829f9f35f0 con 0x55829f9cf720 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f46ac4dab80 10 client.0 _ll_register_callbacks cb 0x55829f8beb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f469e8bd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f469e8bd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9f2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48728/0 (socket says 172.21.15.12:48728) 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.931+0000 7f469d8bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f469e8bd700 1 -- 172.21.15.12:0/1601545913 learned_addr learned my addr 172.21.15.12:0/1601545913 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f469e0bc700 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f469e8bd700 1 -- 172.21.15.12:0/1601545913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 msgr2=0x55829f9f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f469e8bd700 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f469e8bd700 1 -- 172.21.15.12:0/1601545913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 msgr2=0x55829f9f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f469e8bd700 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f469e8bd700 1 -- 172.21.15.12:0/1601545913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55829f746b60 con 0x55829f9cf720 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f469d8bb700 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f469e0bc700 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f469e8bd700 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9f2d80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4690002230 tx=0x7f4690014e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0939a05f27e07fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.932+0000 7f4681ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f4681ffb700 1 -- 172.21.15.12:0/1601545913 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469000e040 con 0x55829f9cf720 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f4681ffb700 1 -- 172.21.15.12:0/1601545913 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f469001a040 con 0x55829f9cf720 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1601545913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f468809ebc0 con 0x55829f9cf720 2024-05-09T00:11:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f4681ffb700 1 -- 172.21.15.12:0/1601545913 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46900049a0 con 0x55829f9cf720 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f46ac4dab80 10 client.31873 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1601545913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55829f8df560 con 0x55829f9cf720 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f4681ffb700 1 -- 172.21.15.12:0/1601545913 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469003f020 con 0x55829f9cf720 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f4681ffb700 1 -- 172.21.15.12:0/1601545913 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4690007bb0 con 0x55829f9cf720 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f46ac4dab80 10 client.31873 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f46ac4dab80 10 client.31873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f46ac4dab80 20 client.31873 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55829fa5bd90 0x55829fa5e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f46ac4dab80 4 client.31873 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1601545913 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55829f911ec0 con 0x55829fa5bd90 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.933+0000 7f469e0bc700 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55829fa5bd90 0x55829fa5e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.934+0000 7f4681ffb700 10 client.31873 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.934+0000 7f469e0bc700 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55829fa5bd90 0x55829fa5e1b0 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454fa242490dfac5 server_cookie=3a16881a5229a60b in_seq=0 out_seq=0 2024-05-09T00:11:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.939+0000 7f4681ffb700 1 -- 172.21.15.12:0/1601545913 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55829f911ec0 con 0x55829fa5bd90 2024-05-09T00:11:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.939+0000 7f4681ffb700 10 client.31873 handle_command_reply: tid=0 2024-05-09T00:11:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.939+0000 7f46ac4dab80 1 client.31873 shutdown 2024-05-09T00:11:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1601545913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 msgr2=0x55829f9f2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9f2d80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4690002230 tx=0x7f4690014e40 comp rx=0 tx=0).stop 2024-05-09T00:11:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1601545913 shutdown_connections 2024-05-09T00:11:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55829f9cf720 0x55829f9f2d80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55829fa5bd90 0x55829fa5e1b0 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55829f9c9890 0x55829f9f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 1 --2- 172.21.15.12:0/1601545913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55829f9cee00 0x55829f9f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1601545913 >> 172.21.15.12:0/1601545913 conn(0x55829f927210 msgr2=0x55829f9d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f4681ffb700 0 client.31873 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:21.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1601545913 shutdown_connections 2024-05-09T00:11:21.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 1 -- 172.21.15.12:0/1601545913 wait complete. 2024-05-09T00:11:21.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.940+0000 7f46ac4dab80 20 client.31873 trim_cache size 0 max 16384 2024-05-09T00:11:21.939 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:11:21.939 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:11:21.939 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:11:21.939 INFO:teuthology.orchestra.run.smithi012.stdout: "1cae3284-665b-43b5-ab8e-897ff80a78c6": { 2024-05-09T00:11:21.939 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:11:21.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "1cae3284-665b-43b5-ab8e-897ff80a78c6", 2024-05-09T00:11:21.940 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:11:21.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:11:21.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:11:21.940 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:11:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 -- 172.21.15.12:0/1883896785 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46840a1290 msgr2=0x7f46840a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46840a1290 0x7f46840a3750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4694009df0 tx=0x7f4694009970 comp rx=0 tx=0).stop 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 -- 172.21.15.12:0/1883896785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4698106290 msgr2=0x7f46981978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4698106290 0x7f46981978d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f468800b580 tx=0x7f468800bfd0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 -- 172.21.15.12:0/1883896785 shutdown_connections 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f46840a1290 0x7f46840a3750 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f469819bc70 0x7f469819c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46980694e0 0x7f4698197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 --2- 172.21.15.12:0/1883896785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4698106290 0x7f46981978d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 -- 172.21.15.12:0/1883896785 >> 172.21.15.12:0/1883896785 conn(0x7f4698076f30 msgr2=0x7f46980fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 -- 172.21.15.12:0/1883896785 shutdown_connections 2024-05-09T00:11:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:21.941+0000 7f46a0320700 1 -- 172.21.15.12:0/1883896785 wait complete. 2024-05-09T00:11:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.947+0000 7fd47a566700 1 -- 172.21.15.12:0/2395906405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd474105170 msgr2=0x7fd4740ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.947+0000 7fd47a566700 1 --2- 172.21.15.12:0/2395906405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd474105170 0x7fd4740ff7e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009f90 tx=0x7fd46c00b330 comp rx=0 tx=0).stop 2024-05-09T00:11:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.947+0000 7fd47a566700 1 -- 172.21.15.12:0/2395906405 shutdown_connections 2024-05-09T00:11:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.947+0000 7fd47a566700 1 --2- 172.21.15.12:0/2395906405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd474105170 0x7fd4740ff7e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.947+0000 7fd47a566700 1 --2- 172.21.15.12:0/2395906405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd474100b60 0x7fd474100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.947+0000 7fd47a566700 1 --2- 172.21.15.12:0/2395906405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd474101510 0x7fd474104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.947+0000 7fd47a566700 1 -- 172.21.15.12:0/2395906405 >> 172.21.15.12:0/2395906405 conn(0x7fd474075130 msgr2=0x7fd474075540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.948+0000 7fd47a566700 1 -- 172.21.15.12:0/2395906405 shutdown_connections 2024-05-09T00:11:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.948+0000 7fd47a566700 1 -- 172.21.15.12:0/2395906405 wait complete. 2024-05-09T00:11:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.948+0000 7fd47a566700 1 Processor -- start 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.948+0000 7fd47a566700 1 -- start start 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd47a566700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd474100b60 0x7fd474199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd47a566700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd474105170 0x7fd474199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd47a566700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd47419dfe0 0x7fd47419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd47a566700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47419ea60 con 0x7fd47419dfe0 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd47a566700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47419ebd0 con 0x7fd474105170 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd47a566700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47419ed90 con 0x7fd474100b60 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd473fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd474100b60 0x7fd474199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd478b03700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd47419dfe0 0x7fd47419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd473fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd474100b60 0x7fd474199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48748/0 (socket says 172.21.15.12:48748) 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd473fff700 1 -- 172.21.15.12:0/3960935742 learned_addr learned my addr 172.21.15.12:0/3960935742 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.949+0000 7fd4737fe700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd474105170 0x7fd474199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd473fff700 1 -- 172.21.15.12:0/3960935742 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd474105170 msgr2=0x7fd474199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd473fff700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd474105170 0x7fd474199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd473fff700 1 -- 172.21.15.12:0/3960935742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd47419dfe0 msgr2=0x7fd47419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd473fff700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd47419dfe0 0x7fd47419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd473fff700 1 -- 172.21.15.12:0/3960935742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c009c40 con 0x7fd474100b60 2024-05-09T00:11:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd473fff700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd474100b60 0x7fd474199700 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd46400e9c0 tx=0x7fd46400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd4717fa700 1 -- 172.21.15.12:0/3960935742 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464004a30 con 0x7fd474100b60 2024-05-09T00:11:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd4717fa700 1 -- 172.21.15.12:0/3960935742 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd46400cc80 con 0x7fd474100b60 2024-05-09T00:11:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd47a566700 1 -- 172.21.15.12:0/3960935742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4741a62f0 con 0x7fd474100b60 2024-05-09T00:11:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd4717fa700 1 -- 172.21.15.12:0/3960935742 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4640054e0 con 0x7fd474100b60 2024-05-09T00:11:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.950+0000 7fd47a566700 1 -- 172.21.15.12:0/3960935742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4741a67d0 con 0x7fd474100b60 2024-05-09T00:11:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.953+0000 7fd4717fa700 1 -- 172.21.15.12:0/3960935742 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd464021440 con 0x7fd474100b60 2024-05-09T00:11:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.954+0000 7fd4717fa700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd45c0a1290 0x7fd45c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.955+0000 7fd4717fa700 1 -- 172.21.15.12:0/3960935742 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd464014070 con 0x7fd474100b60 2024-05-09T00:11:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.955+0000 7fd4737fe700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd45c0a1290 0x7fd45c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.955+0000 7fd486720b80 1 Processor -- start 2024-05-09T00:11:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.955+0000 7fd4737fe700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd45c0a1290 0x7fd45c0a3750 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd47419ac50 tx=0x7fd468009190 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd486720b80 1 -- start start 2024-05-09T00:11:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd486720b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 0x55e23d8dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd486720b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 0x55e23d8e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd486720b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e4240 0x55e23d8e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd486720b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d82e0f0 con 0x55e23d8e4240 2024-05-09T00:11:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd486720b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d82bab0 con 0x55e23d8e0260 2024-05-09T00:11:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd486720b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d8273a0 con 0x55e23d8df940 2024-05-09T00:11:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd473fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 0x55e23d8dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd473fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 0x55e23d8dfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:48750/0 (socket says 172.21.15.12:48750) 2024-05-09T00:11:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd473fff700 1 -- 172.21.15.12:0/2871731189 learned_addr learned my addr 172.21.15.12:0/2871731189 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd4737fe700 1 --2- 172.21.15.12:0/2871731189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 0x55e23d8e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.958+0000 7fd478b03700 1 --2- 172.21.15.12:0/2871731189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e4240 0x55e23d8e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.959+0000 7fd473fff700 1 -- 172.21.15.12:0/2871731189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 msgr2=0x55e23d8e3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.959+0000 7fd473fff700 1 --2- 172.21.15.12:0/2871731189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 0x55e23d8e3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.959+0000 7fd473fff700 1 -- 172.21.15.12:0/2871731189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e4240 msgr2=0x55e23d8e46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.959+0000 7fd473fff700 1 --2- 172.21.15.12:0/2871731189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e4240 0x55e23d8e46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.959+0000 7fd473fff700 1 -- 172.21.15.12:0/2871731189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e23d6a8350 con 0x55e23d8df940 2024-05-09T00:11:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.959+0000 7fd478b03700 1 --2- 172.21.15.12:0/2871731189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e4240 0x55e23d8e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.959+0000 7fd473fff700 1 --2- 172.21.15.12:0/2871731189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 0x55e23d8dfd20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd464096960 tx=0x7fd464096360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=180a37cbad81df48 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.959+0000 7fd45a7fc700 1 -- 172.21.15.12:0/2871731189 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46409dd20 con 0x55e23d8df940 2024-05-09T00:11:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.959+0000 7fd45a7fc700 1 -- 172.21.15.12:0/2871731189 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4640a2370 con 0x55e23d8df940 2024-05-09T00:11:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.959+0000 7fd45a7fc700 1 -- 172.21.15.12:0/2871731189 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464016330 con 0x55e23d8df940 2024-05-09T00:11:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.960+0000 7fd486720b80 1 -- 172.21.15.12:0/2871731189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 msgr2=0x55e23d8dfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.960+0000 7fd486720b80 1 --2- 172.21.15.12:0/2871731189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 0x55e23d8dfd20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd464096960 tx=0x7fd464096360 comp rx=0 tx=0).stop 2024-05-09T00:11:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.960+0000 7fd486720b80 1 -- 172.21.15.12:0/2871731189 shutdown_connections 2024-05-09T00:11:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.960+0000 7fd486720b80 1 --2- 172.21.15.12:0/2871731189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e4240 0x55e23d8e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.960+0000 7fd486720b80 1 --2- 172.21.15.12:0/2871731189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 0x55e23d8dfd20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.960+0000 7fd486720b80 1 --2- 172.21.15.12:0/2871731189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 0x55e23d8e3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.960+0000 7fd486720b80 1 -- 172.21.15.12:0/2871731189 >> 172.21.15.12:0/2871731189 conn(0x55e23d8351b0 msgr2=0x55e23d8d1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.960 INFO:tasks.cephfs.filesystem:scrub status for tag:1cae3284-665b-43b5-ab8e-897ff80a78c6 - {'path': '/', 'tag': '1cae3284-665b-43b5-ab8e-897ff80a78c6', 'options': 'recursive,force'} 2024-05-09T00:11:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.960+0000 7fd486720b80 1 -- 172.21.15.12:0/2871731189 shutdown_connections 2024-05-09T00:11:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.960+0000 7fd486720b80 1 -- 172.21.15.12:0/2871731189 wait complete. 2024-05-09T00:11:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.961+0000 7fd486720b80 1 -- start start 2024-05-09T00:11:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.961+0000 7fd486720b80 1 Processor -- start 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd486720b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8df940 0x55e23d8e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd486720b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 0x55e23d8e9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd486720b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8e4240 0x55e23d8e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd486720b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d7c3500 con 0x55e23d8e0260 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd486720b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d7c31a0 con 0x55e23d8df940 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd486720b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d6aa640 con 0x55e23d8e4240 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd486720b80 10 client.0 _ll_register_callbacks cb 0x55e23d7ccad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd473fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8df940 0x55e23d8e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd473fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8df940 0x55e23d8e8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:42986/0 (socket says 172.21.15.12:42986) 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd473fff700 1 -- 172.21.15.12:0/3888363214 learned_addr learned my addr 172.21.15.12:0/3888363214 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd4737fe700 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 0x55e23d8e9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd473fff700 1 -- 172.21.15.12:0/3888363214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8e4240 msgr2=0x55e23d8e98a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd473fff700 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8e4240 0x55e23d8e98a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd473fff700 1 -- 172.21.15.12:0/3888363214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 msgr2=0x55e23d8e9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd473fff700 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 0x55e23d8e9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd473fff700 1 -- 172.21.15.12:0/3888363214 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4741a67d0 con 0x55e23d8df940 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd473fff700 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8df940 0x55e23d8e8e20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd464099bf0 tx=0x7fd464005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d8e4286f72c08a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.962+0000 7fd4537fe700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd4537fe700 1 -- 172.21.15.12:0/3888363214 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464004da0 con 0x55e23d8df940 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd4537fe700 1 -- 172.21.15.12:0/3888363214 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd4640b81b0 con 0x55e23d8df940 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd486720b80 1 -- 172.21.15.12:0/3888363214 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e23d780210 con 0x55e23d8df940 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd4537fe700 1 -- 172.21.15.12:0/3888363214 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd464062ea0 con 0x55e23d8df940 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd486720b80 10 client.31589 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd486720b80 1 -- 172.21.15.12:0/3888363214 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55e23d654b00 con 0x55e23d8df940 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd4537fe700 1 -- 172.21.15.12:0/3888363214 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd464066020 con 0x55e23d8df940 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd4537fe700 1 -- 172.21.15.12:0/3888363214 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fd4640993e0 con 0x55e23d8df940 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd486720b80 10 client.31589 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd486720b80 10 client.31589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd486720b80 20 client.31589 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.963+0000 7fd486720b80 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e23d901950 0x55e23d903d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.964+0000 7fd486720b80 4 client.31589 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.964+0000 7fd486720b80 1 -- 172.21.15.12:0/3888363214 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e23d8a67b0 con 0x55e23d901950 2024-05-09T00:11:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.964+0000 7fd4737fe700 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e23d901950 0x55e23d903d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.964+0000 7fd4737fe700 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e23d901950 0x55e23d903d70 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8fe529fddcf3152 server_cookie=68189a08adf092fa in_seq=0 out_seq=0 2024-05-09T00:11:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.964+0000 7fd4537fe700 10 client.31589 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.970+0000 7fd4537fe700 1 -- 172.21.15.12:0/3888363214 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e23d8a67b0 con 0x55e23d901950 2024-05-09T00:11:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.970+0000 7fd4537fe700 10 client.31589 handle_command_reply: tid=0 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.970+0000 7fd486720b80 1 client.31589 shutdown 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 1 -- 172.21.15.12:0/3888363214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8df940 msgr2=0x55e23d8e8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8df940 0x55e23d8e8e20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd464099bf0 tx=0x7fd464005870 comp rx=0 tx=0).stop 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 1 -- 172.21.15.12:0/3888363214 shutdown_connections 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8e4240 0x55e23d8e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e23d901950 0x55e23d903d70 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8df940 0x55e23d8e8e20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 1 --2- 172.21.15.12:0/3888363214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 0x55e23d8e9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd4537fe700 0 client.31589 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 1 -- 172.21.15.12:0/3888363214 >> 172.21.15.12:0/3888363214 conn(0x55e23d8351b0 msgr2=0x55e23d8d5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 1 -- 172.21.15.12:0/3888363214 shutdown_connections 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 1 -- 172.21.15.12:0/3888363214 wait complete. 2024-05-09T00:11:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.971+0000 7fd486720b80 20 client.31589 trim_cache size 0 max 16384 2024-05-09T00:11:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.988+0000 7fd486720b80 1 Processor -- start 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd486720b80 1 -- start start 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd486720b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8df940 0x55e23d8de4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd486720b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 0x55e23d8d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd486720b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8e4240 0x55e23d8dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd486720b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d8e2fd0 con 0x55e23d8df940 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd486720b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d8eb740 con 0x55e23d8e0260 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd486720b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d8eb900 con 0x55e23d8e4240 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd473fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8df940 0x55e23d8de4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd473fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8df940 0x55e23d8de4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50538/0 (socket says 172.21.15.12:50538) 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd473fff700 1 -- 172.21.15.12:0/2544073436 learned_addr learned my addr 172.21.15.12:0/2544073436 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd478b03700 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8e4240 0x55e23d8dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd4737fe700 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 0x55e23d8d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd473fff700 1 -- 172.21.15.12:0/2544073436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8e4240 msgr2=0x55e23d8dddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd473fff700 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8e4240 0x55e23d8dddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd473fff700 1 -- 172.21.15.12:0/2544073436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 msgr2=0x55e23d8d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd473fff700 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 0x55e23d8d1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.989+0000 7fd473fff700 1 -- 172.21.15.12:0/2544073436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e23d7ed500 con 0x55e23d8df940 2024-05-09T00:11:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.990+0000 7fd478b03700 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8e4240 0x55e23d8dddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.990+0000 7fd4737fe700 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 0x55e23d8d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.990+0000 7fd473fff700 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8df940 0x55e23d8de4d0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd464005950 tx=0x7fd4640ca610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d030ee393827053 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.990+0000 7fd45affd700 1 -- 172.21.15.12:0/2544073436 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464004da0 con 0x55e23d8df940 2024-05-09T00:11:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.990+0000 7fd45affd700 1 -- 172.21.15.12:0/2544073436 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd464066050 con 0x55e23d8df940 2024-05-09T00:11:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.990+0000 7fd45affd700 1 -- 172.21.15.12:0/2544073436 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4640b2220 con 0x55e23d8df940 2024-05-09T00:11:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.991+0000 7fd486720b80 1 -- 172.21.15.12:0/2544073436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8df940 msgr2=0x55e23d8de4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.991+0000 7fd486720b80 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8df940 0x55e23d8de4d0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd464005950 tx=0x7fd4640ca610 comp rx=0 tx=0).stop 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.991+0000 7fd486720b80 1 -- 172.21.15.12:0/2544073436 shutdown_connections 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.991+0000 7fd486720b80 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8e4240 0x55e23d8dddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.991+0000 7fd486720b80 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8df940 0x55e23d8de4d0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.991+0000 7fd486720b80 1 --2- 172.21.15.12:0/2544073436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e0260 0x55e23d8d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.991+0000 7fd486720b80 1 -- 172.21.15.12:0/2544073436 >> 172.21.15.12:0/2544073436 conn(0x55e23d8351b0 msgr2=0x55e23d8d6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.991+0000 7fd486720b80 1 -- 172.21.15.12:0/2544073436 shutdown_connections 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.991+0000 7fd486720b80 1 -- 172.21.15.12:0/2544073436 wait complete. 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.991+0000 7fd486720b80 1 -- start start 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd486720b80 1 Processor -- start 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd486720b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 0x55e23d8db500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd486720b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 0x55e23d8e8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd486720b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e4240 0x55e23d8e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd486720b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d8e8de0 con 0x55e23d8e0260 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd486720b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d8e8f50 con 0x55e23d8e4240 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd486720b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e23d8e9110 con 0x55e23d8df940 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd486720b80 10 client.0 _ll_register_callbacks cb 0x55e23d7ccad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd4737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 0x55e23d8e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd4737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 0x55e23d8e8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50540/0 (socket says 172.21.15.12:50540) 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd4737fe700 1 -- 172.21.15.12:0/444357510 learned_addr learned my addr 172.21.15.12:0/444357510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd478b03700 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e4240 0x55e23d8e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd473fff700 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 0x55e23d8db500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd4737fe700 1 -- 172.21.15.12:0/444357510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 msgr2=0x55e23d8db500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd4737fe700 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 0x55e23d8db500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd4737fe700 1 -- 172.21.15.12:0/444357510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e4240 msgr2=0x55e23d8e88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd4737fe700 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e4240 0x55e23d8e88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd4737fe700 1 -- 172.21.15.12:0/444357510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46801a040 con 0x55e23d8e0260 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.992+0000 7fd478b03700 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e4240 0x55e23d8e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.993+0000 7fd4737fe700 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 0x55e23d8e8360 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x55e23d8ea360 tx=0x7fd468011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc20e9f9b97a0ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.993+0000 7fd4537fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.993+0000 7fd4537fe700 1 -- 172.21.15.12:0/444357510 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468027050 con 0x55e23d8e0260 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.993+0000 7fd4537fe700 1 -- 172.21.15.12:0/444357510 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd468015a70 con 0x55e23d8e0260 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.993+0000 7fd486720b80 1 -- 172.21.15.12:0/444357510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4640a2370 con 0x55e23d8e0260 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.993+0000 7fd4537fe700 1 -- 172.21.15.12:0/444357510 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd46800eeb0 con 0x55e23d8e0260 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.993+0000 7fd486720b80 10 client.22218 fetch_fsmap learned FSMap version 83 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.993+0000 7fd486720b80 1 -- 172.21.15.12:0/444357510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7fd4741000a0 con 0x55e23d8e0260 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.993+0000 7fd4537fe700 1 -- 172.21.15.12:0/444357510 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4640a2370 con 0x55e23d8e0260 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.994+0000 7fd4537fe700 1 -- 172.21.15.12:0/444357510 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fd46802dd10 con 0x55e23d8e0260 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.994+0000 7fd486720b80 10 client.22218 fetch_fsmap finished waiting for FSMap version 83 2024-05-09T00:11:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.994+0000 7fd486720b80 10 client.22218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.994+0000 7fd486720b80 20 client.22218 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.994+0000 7fd486720b80 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e23d958740 0x55e23d95ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.994+0000 7fd486720b80 4 client.22218 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:11:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.994+0000 7fd486720b80 1 -- 172.21.15.12:0/444357510 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e23d81fe60 con 0x55e23d958740 2024-05-09T00:11:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.994+0000 7fd473fff700 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e23d958740 0x55e23d95ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.994+0000 7fd473fff700 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e23d958740 0x55e23d95ab60 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64dfc2cfa98a3c9 server_cookie=50deca96e2132e33 in_seq=0 out_seq=0 2024-05-09T00:11:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:21.995+0000 7fd4537fe700 10 client.22218 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:22.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:21 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:11:22.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:21 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:22.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:21 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:11:22.173 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:21 smithi012 ceph-mon[42503]: pgmap v804: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 559 op/s 2024-05-09T00:11:22.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:21 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:11:22.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:21 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:22.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:21 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:11:22.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:21 smithi204 ceph-mon[47373]: pgmap v804: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 559 op/s 2024-05-09T00:11:22.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:21 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:11:22.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:21 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:22.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:21 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:11:22.274 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:21 smithi160 ceph-mon[46152]: pgmap v804: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 559 op/s 2024-05-09T00:11:22.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 374492 27.85 MB/sec execute 557 sec latency 317.932 ms 2024-05-09T00:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.388+0000 7fd4537fe700 1 -- 172.21.15.12:0/444357510 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e23d81fe60 con 0x55e23d958740 2024-05-09T00:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.388+0000 7fd4537fe700 10 client.22218 handle_command_reply: tid=0 2024-05-09T00:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.388+0000 7fd486720b80 1 client.22218 shutdown 2024-05-09T00:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.389+0000 7fd486720b80 1 -- 172.21.15.12:0/444357510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 msgr2=0x55e23d8e8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.389+0000 7fd486720b80 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 0x55e23d8e8360 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x55e23d8ea360 tx=0x7fd468011000 comp rx=0 tx=0).stop 2024-05-09T00:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.390+0000 7fd486720b80 1 -- 172.21.15.12:0/444357510 shutdown_connections 2024-05-09T00:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.390+0000 7fd486720b80 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55e23d958740 0x55e23d95ab60 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.390+0000 7fd486720b80 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e23d8e4240 0x55e23d8e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.390+0000 7fd486720b80 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e23d8df940 0x55e23d8db500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.390+0000 7fd486720b80 1 --2- 172.21.15.12:0/444357510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e23d8e0260 0x55e23d8e8360 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.390+0000 7fd486720b80 1 -- 172.21.15.12:0/444357510 >> 172.21.15.12:0/444357510 conn(0x55e23d8351b0 msgr2=0x55e23d8e7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.390+0000 7fd4537fe700 0 client.22218 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.390+0000 7fd486720b80 1 -- 172.21.15.12:0/444357510 shutdown_connections 2024-05-09T00:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.390+0000 7fd486720b80 1 -- 172.21.15.12:0/444357510 wait complete. 2024-05-09T00:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.392+0000 7fd486720b80 20 client.22218 trim_cache size 0 max 16384 2024-05-09T00:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:11:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.396+0000 7fd47a566700 1 -- 172.21.15.12:0/3960935742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd45c0a1290 msgr2=0x7fd45c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.396+0000 7fd47a566700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd45c0a1290 0x7fd45c0a3750 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd47419ac50 tx=0x7fd468009190 comp rx=0 tx=0).stop 2024-05-09T00:11:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.396+0000 7fd47a566700 1 -- 172.21.15.12:0/3960935742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd474100b60 msgr2=0x7fd474199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.396+0000 7fd47a566700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd474100b60 0x7fd474199700 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd46400e9c0 tx=0x7fd46400ecd0 comp rx=0 tx=0).stop 2024-05-09T00:11:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.397+0000 7fd47a566700 1 -- 172.21.15.12:0/3960935742 shutdown_connections 2024-05-09T00:11:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.397+0000 7fd47a566700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd45c0a1290 0x7fd45c0a3750 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.397+0000 7fd47a566700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd47419dfe0 0x7fd47419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.397+0000 7fd47a566700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd474100b60 0x7fd474199700 secure :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd46400e9c0 tx=0x7fd46400ecd0 comp rx=0 tx=0).stop 2024-05-09T00:11:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.397+0000 7fd47a566700 1 --2- 172.21.15.12:0/3960935742 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd474105170 0x7fd474199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.397+0000 7fd47a566700 1 -- 172.21.15.12:0/3960935742 >> 172.21.15.12:0/3960935742 conn(0x7fd474075130 msgr2=0x7fd4741028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.402+0000 7fd47a566700 1 -- 172.21.15.12:0/3960935742 shutdown_connections 2024-05-09T00:11:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:22.404+0000 7fd47a566700 1 -- 172.21.15.12:0/3960935742 wait complete. 2024-05-09T00:11:22.765 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:11:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: ignoring --setuser ceph since I am not root 2024-05-09T00:11:22.765 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:11:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: ignoring --setgroup ceph since I am not root 2024-05-09T00:11:22.765 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:11:22 smithi204 ceph-mds[83951]: -- 172.21.15.204:0/743304545 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5583b4c861a0 con 0x5583b4c5d000 2024-05-09T00:11:22.765 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:11:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: starting mds.l at 2024-05-09T00:11:23.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:22 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:11:23.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:22 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:11:23.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:23.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:23.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:22 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:23.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:22 smithi012 ceph-mon[42503]: daemon mds.l restarted 2024-05-09T00:11:23.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:22 smithi012 ceph-mon[42503]: osdmap e138: 12 total, 12 up, 12 in 2024-05-09T00:11:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:22 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:11:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:22 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:11:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:22 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:22 smithi204 ceph-mon[47373]: daemon mds.l restarted 2024-05-09T00:11:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:22 smithi204 ceph-mon[47373]: osdmap e138: 12 total, 12 up, 12 in 2024-05-09T00:11:23.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:22 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:11:23.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:22 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:11:23.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:23.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:23.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:22 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:23.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:22 smithi160 ceph-mon[46152]: daemon mds.l restarted 2024-05-09T00:11:23.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:22 smithi160 ceph-mon[46152]: osdmap e138: 12 total, 12 up, 12 in 2024-05-09T00:11:23.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 374850 27.81 MB/sec execute 558 sec latency 323.801 ms 2024-05-09T00:11:24.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:23 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6832/3656191694,v1:172.21.15.204:6833/3656191694] up:boot 2024-05-09T00:11:24.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:23 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:11:24.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:23 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:11:24.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:23 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:11:24.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:23 smithi012 ceph-mon[42503]: pgmap v806: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 685 op/s 2024-05-09T00:11:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:23 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6832/3656191694,v1:172.21.15.204:6833/3656191694] up:boot 2024-05-09T00:11:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:23 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:11:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:23 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:11:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:23 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:11:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:23 smithi204 ceph-mon[47373]: pgmap v806: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 685 op/s 2024-05-09T00:11:24.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:23 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6832/3656191694,v1:172.21.15.204:6833/3656191694] up:boot 2024-05-09T00:11:24.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:23 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-09T00:11:24.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:23 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:11:24.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:23 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:11:24.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:23 smithi160 ceph-mon[46152]: pgmap v806: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 685 op/s 2024-05-09T00:11:24.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 375193 27.80 MB/sec execute 559 sec latency 294.657 ms 2024-05-09T00:11:25.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 375608 27.79 MB/sec execute 560 sec latency 149.899 ms 2024-05-09T00:11:26.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:25 smithi012 ceph-mon[42503]: pgmap v807: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 640 op/s 2024-05-09T00:11:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:25 smithi204 ceph-mon[47373]: pgmap v807: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 640 op/s 2024-05-09T00:11:26.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:25 smithi160 ceph-mon[46152]: pgmap v807: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 640 op/s 2024-05-09T00:11:26.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 376132 27.79 MB/sec execute 561 sec latency 86.429 ms 2024-05-09T00:11:27.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 376671 27.77 MB/sec execute 562 sec latency 119.261 ms 2024-05-09T00:11:28.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:27 smithi012 ceph-mon[42503]: pgmap v808: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 703 op/s 2024-05-09T00:11:28.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:27 smithi204 ceph-mon[47373]: pgmap v808: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 703 op/s 2024-05-09T00:11:28.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:27 smithi160 ceph-mon[46152]: pgmap v808: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 703 op/s 2024-05-09T00:11:28.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 377347 27.80 MB/sec execute 563 sec latency 84.213 ms 2024-05-09T00:11:29.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:28 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:11:29.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:28 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:11:29.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:28 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:11:29.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:28 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:11:29.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:28 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:11:29.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:28 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:11:29.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 377714 27.80 MB/sec execute 564 sec latency 262.769 ms 2024-05-09T00:11:30.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:29 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:11:30.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:29 smithi012 ceph-mon[42503]: pgmap v809: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 660 op/s 2024-05-09T00:11:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:29 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:11:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:29 smithi204 ceph-mon[47373]: pgmap v809: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 660 op/s 2024-05-09T00:11:30.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:29 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:11:30.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:29 smithi160 ceph-mon[46152]: pgmap v809: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 660 op/s 2024-05-09T00:11:30.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 377992 27.78 MB/sec execute 565 sec latency 175.645 ms 2024-05-09T00:11:31.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 378414 27.76 MB/sec execute 566 sec latency 145.435 ms 2024-05-09T00:11:32.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 378948 27.76 MB/sec execute 567 sec latency 105.187 ms 2024-05-09T00:11:32.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:32 smithi012 ceph-mon[42503]: pgmap v810: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 620 op/s 2024-05-09T00:11:32.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:32 smithi204 ceph-mon[47373]: pgmap v810: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 620 op/s 2024-05-09T00:11:32.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:32 smithi160 ceph-mon[46152]: pgmap v810: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 620 op/s 2024-05-09T00:11:33.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 379456 27.76 MB/sec execute 568 sec latency 109.508 ms 2024-05-09T00:11:34.327 INFO:tasks.workunit.client.0.smithi012.stdout: 10 379889 27.75 MB/sec execute 569 sec latency 219.596 ms 2024-05-09T00:11:34.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:34 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:11:34.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:34 smithi012 ceph-mon[42503]: pgmap v811: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 665 op/s 2024-05-09T00:11:35.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:34 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:11:35.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:34 smithi204 ceph-mon[47373]: pgmap v811: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 665 op/s 2024-05-09T00:11:35.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:34 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:11:35.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:34 smithi160 ceph-mon[46152]: pgmap v811: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 665 op/s 2024-05-09T00:11:35.330 INFO:tasks.workunit.client.0.smithi012.stdout: 10 380319 27.74 MB/sec execute 570 sec latency 120.419 ms 2024-05-09T00:11:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.551+0000 7f5e7112c700 1 -- 172.21.15.12:0/4160546167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e6c101aa0 msgr2=0x7f5e6c105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.551+0000 7f5e7112c700 1 --2- 172.21.15.12:0/4160546167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e6c101aa0 0x7f5e6c105ae0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009990 tx=0x7f5e60009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.551+0000 7f5e7112c700 1 -- 172.21.15.12:0/4160546167 shutdown_connections 2024-05-09T00:11:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.551+0000 7f5e7112c700 1 --2- 172.21.15.12:0/4160546167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e6c106210 0x7f5e6c10dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.551+0000 7f5e7112c700 1 --2- 172.21.15.12:0/4160546167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6c1010f0 0x7f5e6c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.551+0000 7f5e7112c700 1 --2- 172.21.15.12:0/4160546167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e6c101aa0 0x7f5e6c105ae0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.551+0000 7f5e7112c700 1 -- 172.21.15.12:0/4160546167 >> 172.21.15.12:0/4160546167 conn(0x7f5e6c0fc9d0 msgr2=0x7f5e6c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.551+0000 7f5e7112c700 1 -- 172.21.15.12:0/4160546167 shutdown_connections 2024-05-09T00:11:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.551+0000 7f5e7112c700 1 -- 172.21.15.12:0/4160546167 wait complete. 2024-05-09T00:11:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.552+0000 7f5e7112c700 1 Processor -- start 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e7112c700 1 -- start start 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e7112c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e6c1010f0 0x7f5e6c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e7112c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6c106210 0x7f5e6c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e7112c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e6c073d20 0x7f5e6c1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e7112c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c1a5b70 con 0x7f5e6c106210 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e7112c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c1a5ce0 con 0x7f5e6c073d20 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e7112c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c1a5ea0 con 0x7f5e6c1010f0 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e6ad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e6c1010f0 0x7f5e6c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e6a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6c106210 0x7f5e6c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e6ad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e6c1010f0 0x7f5e6c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:35282/0 (socket says 172.21.15.12:35282) 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/3772118503 learned_addr learned my addr 172.21.15.12:0/3772118503 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e6b59e700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e6c073d20 0x7f5e6c1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/3772118503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e6c073d20 msgr2=0x7f5e6c1a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e6c073d20 0x7f5e6c1a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/3772118503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6c106210 msgr2=0x7f5e6c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6c106210 0x7f5e6c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.553+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/3772118503 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e600095f0 con 0x7f5e6c1010f0 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.554+0000 7f5e6a59c700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6c106210 0x7f5e6c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.554+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e6c1010f0 0x7f5e6c0732a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00e9a0 tx=0x7f5e5c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.554+0000 7f5e53fff700 1 -- 172.21.15.12:0/3772118503 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c009790 con 0x7f5e6c1010f0 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.554+0000 7f5e7112c700 1 -- 172.21.15.12:0/3772118503 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6c1a6180 con 0x7f5e6c1010f0 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.554+0000 7f5e7112c700 1 -- 172.21.15.12:0/3772118503 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6c1a66b0 con 0x7f5e6c1010f0 2024-05-09T00:11:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.554+0000 7f5e53fff700 1 -- 172.21.15.12:0/3772118503 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e5c010430 con 0x7f5e6c1010f0 2024-05-09T00:11:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.554+0000 7f5e53fff700 1 -- 172.21.15.12:0/3772118503 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c018680 con 0x7f5e6c1010f0 2024-05-09T00:11:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.557+0000 7f5e53fff700 1 -- 172.21.15.12:0/3772118503 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5e5c00ca10 con 0x7f5e6c1010f0 2024-05-09T00:11:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.558+0000 7f5e53fff700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5e540a1310 0x7f5e540a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.558+0000 7f5e53fff700 1 -- 172.21.15.12:0/3772118503 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5e5c014070 con 0x7f5e6c1010f0 2024-05-09T00:11:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.558+0000 7f5e6a59c700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5e540a1310 0x7f5e540a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.559+0000 7f5e6a59c700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5e540a1310 0x7f5e540a37d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000b2e0 tx=0x7f5e60005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.559+0000 7f5e7d2e6b80 1 Processor -- start 2024-05-09T00:11:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.559+0000 7f5e7d2e6b80 1 -- start start 2024-05-09T00:11:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be2664ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664f270 0x55be26648f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be26649480 0x55be266498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be265a00f0 con 0x55be2664e950 2024-05-09T00:11:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be2659dab0 con 0x55be2664f270 2024-05-09T00:11:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be265993a0 con 0x55be26649480 2024-05-09T00:11:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be2664ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be2664ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57738/0 (socket says 172.21.15.12:57738) 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/3285989884 learned_addr learned my addr 172.21.15.12:0/3285989884 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6a59c700 1 --2- 172.21.15.12:0/3285989884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664f270 0x55be26648f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6b59e700 1 --2- 172.21.15.12:0/3285989884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be26649480 0x55be266498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/3285989884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be26649480 msgr2=0x55be266498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3285989884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be26649480 0x55be266498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/3285989884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664f270 msgr2=0x55be26648f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3285989884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664f270 0x55be26648f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/3285989884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be2641a350 con 0x55be2664e950 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6a59c700 1 --2- 172.21.15.12:0/3285989884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664f270 0x55be26648f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.560+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3285989884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be2664ed30 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c096960 tx=0x7f5e5c099ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b9c6e54c7d03f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.561+0000 7f5e517fa700 1 -- 172.21.15.12:0/3285989884 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0a2370 con 0x55be2664e950 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.561+0000 7f5e517fa700 1 -- 172.21.15.12:0/3285989884 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e5c00ccc0 con 0x55be2664e950 2024-05-09T00:11:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.561+0000 7f5e517fa700 1 -- 172.21.15.12:0/3285989884 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0a1ee0 con 0x55be2664e950 2024-05-09T00:11:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.561+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3285989884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 msgr2=0x55be2664ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.561+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3285989884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be2664ed30 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c096960 tx=0x7f5e5c099ef0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3285989884 shutdown_connections 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3285989884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be26649480 0x55be266498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3285989884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be2664ed30 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3285989884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664f270 0x55be26648f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3285989884 >> 172.21.15.12:0/3285989884 conn(0x55be265a71b0 msgr2=0x55be26643840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3285989884 shutdown_connections 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3285989884 wait complete. 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 -- start start 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 Processor -- start 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be26649480 0x55be26655190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be26652520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26652ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be26535500 con 0x55be2664e950 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be265351a0 con 0x55be26649480 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be2641c640 con 0x55be2664f270 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e7d2e6b80 10 client.0 _ll_register_callbacks cb 0x55be2653ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.562+0000 7f5e6a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be26652520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e6b59e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26652ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e6a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be26652520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57750/0 (socket says 172.21.15.12:57750) 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e6a59c700 1 -- 172.21.15.12:0/3212223072 learned_addr learned my addr 172.21.15.12:0/3212223072 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be26649480 0x55be26655190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e6b59e700 1 -- 172.21.15.12:0/3212223072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be26649480 msgr2=0x55be26655190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e6b59e700 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be26649480 0x55be26655190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e6b59e700 1 -- 172.21.15.12:0/3212223072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 msgr2=0x55be26652520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e6b59e700 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be26652520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e6b59e700 1 -- 172.21.15.12:0/3212223072 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6c10e600 con 0x55be2664f270 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e6b59e700 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26652ba0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5e64011e90 tx=0x7f5e64014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b681e28b38a1dce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3212223072 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6400e040 con 0x55be2664f270 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3212223072 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be264f2210 con 0x55be2664f270 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3212223072 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5e640093a0 con 0x55be2664f270 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.563+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3212223072 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e64004db0 con 0x55be2664f270 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e7d2e6b80 10 client.31906 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3212223072 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7f5e6c1a66b0 con 0x55be2664f270 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3212223072 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be264f2210 con 0x55be2664f270 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3212223072 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e6401c030 con 0x55be2664f270 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e7d2e6b80 10 client.31906 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e7d2e6b80 10 client.31906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e7d2e6b80 20 client.31906 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be26659d00 0x55be2665c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e7d2e6b80 4 client.31906 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3212223072 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be266187b0 con 0x55be26659d00 2024-05-09T00:11:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.564+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be26659d00 0x55be2665c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.565+0000 7f5e4eff5700 10 client.31906 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.565+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be26659d00 0x55be2665c120 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16571ef00a0419f1 server_cookie=6d72ab5ad0d6d1d1 in_seq=0 out_seq=0 2024-05-09T00:11:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3212223072 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be266187b0 con 0x55be26659d00 2024-05-09T00:11:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e4eff5700 10 client.31906 handle_command_reply: tid=0 2024-05-09T00:11:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 client.31906 shutdown 2024-05-09T00:11:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3212223072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 msgr2=0x55be26652ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26652ba0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5e64011e90 tx=0x7f5e64014600 comp rx=0 tx=0).stop 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3212223072 shutdown_connections 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26652ba0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be26659d00 0x55be2665c120 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be26649480 0x55be26655190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3212223072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be2664e950 0x55be26652520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e4eff5700 0 client.31906 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3212223072 >> 172.21.15.12:0/3212223072 conn(0x55be265a71b0 msgr2=0x55be26647f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3212223072 shutdown_connections 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3212223072 wait complete. 2024-05-09T00:11:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.568+0000 7f5e7d2e6b80 20 client.31906 trim_cache size 0 max 16384 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e7d2e6b80 1 Processor -- start 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e7d2e6b80 1 -- start start 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26655be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664e950 0x55be26643910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26647f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be2667cdb0 con 0x55be26649480 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be26644660 con 0x55be2664e950 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be26657270 con 0x55be2664f270 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e6ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26655be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e6a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664e950 0x55be26643910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e6ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26655be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57766/0 (socket says 172.21.15.12:57766) 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.587+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/2081385983 learned_addr learned my addr 172.21.15.12:0/2081385983 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.588+0000 7f5e6b59e700 1 --2- 172.21.15.12:0/2081385983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26647f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.588+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/2081385983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 msgr2=0x55be26647f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.588+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/2081385983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26647f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.588+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/2081385983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664e950 msgr2=0x55be26643910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.588+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/2081385983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664e950 0x55be26643910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.588+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/2081385983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be263c6b00 con 0x55be26649480 2024-05-09T00:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.588+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/2081385983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26655be0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0ca5e0 tx=0x7f5e5c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d4297e1892cfbb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.588+0000 7f5e51ffb700 1 -- 172.21.15.12:0/2081385983 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c098350 con 0x55be26649480 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.589+0000 7f5e51ffb700 1 -- 172.21.15.12:0/2081385983 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e5c09a150 con 0x55be26649480 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.589+0000 7f5e51ffb700 1 -- 172.21.15.12:0/2081385983 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0a2120 con 0x55be26649480 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.589+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/2081385983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 msgr2=0x55be26655be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.589+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/2081385983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26655be0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0ca5e0 tx=0x7f5e5c0092d0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.589+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/2081385983 shutdown_connections 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.589+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/2081385983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26647f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.589+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/2081385983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26655be0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.589+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/2081385983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664e950 0x55be26643910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.589+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/2081385983 >> 172.21.15.12:0/2081385983 conn(0x55be265a71b0 msgr2=0x55be2663fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/2081385983 shutdown_connections 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/2081385983 wait complete. 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 1 -- start start 2024-05-09T00:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 1 Processor -- start 2024-05-09T00:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26666930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664e950 0x55be26666e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26665540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be266611f0 con 0x55be26649480 2024-05-09T00:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be26661360 con 0x55be2664e950 2024-05-09T00:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be26665a80 con 0x55be2664f270 2024-05-09T00:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.590+0000 7f5e7d2e6b80 10 client.0 _ll_register_callbacks cb 0x55be2653ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26666930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664e950 0x55be26666e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6b59e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26665540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26666930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57768/0 (socket says 172.21.15.12:57768) 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6ad9d700 1 -- 172.21.15.12:0/3738527941 learned_addr learned my addr 172.21.15.12:0/3738527941 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6b59e700 1 -- 172.21.15.12:0/3738527941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664e950 msgr2=0x55be26666e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6b59e700 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664e950 0x55be26666e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6b59e700 1 -- 172.21.15.12:0/3738527941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 msgr2=0x55be26666930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6b59e700 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26666930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6b59e700 1 -- 172.21.15.12:0/3738527941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be2641a350 con 0x55be2664f270 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e6b59e700 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26665540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5e64004470 tx=0x7f5e64014c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f35d5a771e4ad72a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3738527941 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6400e040 con 0x55be2664f270 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3738527941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e5c00ccc0 con 0x55be2664f270 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.591+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3738527941 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5e640048a0 con 0x55be2664f270 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3738527941 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e64009410 con 0x55be2664f270 2024-05-09T00:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e7d2e6b80 10 client.31912 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3738527941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55be2655f500 con 0x55be2664f270 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3738527941 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e5c00ccc0 con 0x55be2664f270 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3738527941 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e64014320 con 0x55be2664f270 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e7d2e6b80 10 client.31912 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e7d2e6b80 10 client.31912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e7d2e6b80 20 client.31912 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be266ca890 0x55be266cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e7d2e6b80 4 client.31912 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3738527941 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be26591e60 con 0x55be266ca890 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.592+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be266ca890 0x55be266cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.593+0000 7f5e6ad9d700 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be266ca890 0x55be266cccb0 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adbb2b8d7f66becc server_cookie=f3dfa27d6ecefd11 in_seq=0 out_seq=0 2024-05-09T00:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:35.593+0000 7f5e4eff5700 10 client.31912 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.103+0000 7f5e4eff5700 1 -- 172.21.15.12:0/3738527941 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be26591e60 con 0x55be266ca890 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.103+0000 7f5e4eff5700 10 client.31912 handle_command_reply: tid=0 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.103+0000 7f5e7d2e6b80 1 client.31912 shutdown 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.103+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3738527941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 msgr2=0x55be26665540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.103+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26665540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5e64004470 tx=0x7f5e64014c30 comp rx=0 tx=0).stop 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3738527941 shutdown_connections 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be2664f270 0x55be26665540 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55be26649480 0x55be26666930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55be266ca890 0x55be266cccb0 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7d2e6b80 1 --2- 172.21.15.12:0/3738527941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be2664e950 0x55be26666e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3738527941 >> 172.21.15.12:0/3738527941 conn(0x55be265a71b0 msgr2=0x55be2664c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e4eff5700 0 client.31912 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3738527941 shutdown_connections 2024-05-09T00:11:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7d2e6b80 1 -- 172.21.15.12:0/3738527941 wait complete. 2024-05-09T00:11:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7d2e6b80 20 client.31912 trim_cache size 0 max 16384 2024-05-09T00:11:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:11:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:11:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:11:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7112c700 1 -- 172.21.15.12:0/3772118503 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5e540a1310 msgr2=0x7f5e540a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7112c700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5e540a1310 0x7f5e540a37d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000b2e0 tx=0x7f5e60005960 comp rx=0 tx=0).stop 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7112c700 1 -- 172.21.15.12:0/3772118503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e6c1010f0 msgr2=0x7f5e6c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7112c700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e6c1010f0 0x7f5e6c0732a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00e9a0 tx=0x7f5e5c00ecb0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7112c700 1 -- 172.21.15.12:0/3772118503 shutdown_connections 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7112c700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5e540a1310 0x7f5e540a37d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7112c700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e6c073d20 0x7f5e6c1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7112c700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e6c1010f0 0x7f5e6c0732a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7112c700 1 --2- 172.21.15.12:0/3772118503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6c106210 0x7f5e6c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.104+0000 7f5e7112c700 1 -- 172.21.15.12:0/3772118503 >> 172.21.15.12:0/3772118503 conn(0x7f5e6c0fc9d0 msgr2=0x7f5e6c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.105+0000 7f5e7112c700 1 -- 172.21.15.12:0/3772118503 shutdown_connections 2024-05-09T00:11:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:36.105+0000 7f5e7112c700 1 -- 172.21.15.12:0/3772118503 wait complete. 2024-05-09T00:11:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:35 smithi012 ceph-mon[42503]: pgmap v812: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 10 MiB/s wr, 485 op/s 2024-05-09T00:11:36.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:35 smithi204 ceph-mon[47373]: pgmap v812: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 10 MiB/s wr, 485 op/s 2024-05-09T00:11:36.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:35 smithi160 ceph-mon[46152]: pgmap v812: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 10 MiB/s wr, 485 op/s 2024-05-09T00:11:36.330 INFO:tasks.workunit.client.0.smithi012.stdout: 10 380709 27.73 MB/sec execute 571 sec latency 402.023 ms 2024-05-09T00:11:37.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 381244 27.74 MB/sec execute 572 sec latency 113.425 ms 2024-05-09T00:11:38.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:37 smithi012 ceph-mon[42503]: pgmap v813: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 556 op/s 2024-05-09T00:11:38.293 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:37 smithi160 ceph-mon[46152]: pgmap v813: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 556 op/s 2024-05-09T00:11:38.294 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:37 smithi204 ceph-mon[47373]: pgmap v813: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 556 op/s 2024-05-09T00:11:38.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 381608 27.73 MB/sec execute 573 sec latency 106.269 ms 2024-05-09T00:11:39.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 382017 27.71 MB/sec execute 574 sec latency 129.459 ms 2024-05-09T00:11:40.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:39 smithi012 ceph-mon[42503]: pgmap v814: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 570 op/s 2024-05-09T00:11:40.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:39 smithi204 ceph-mon[47373]: pgmap v814: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 570 op/s 2024-05-09T00:11:40.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:39 smithi160 ceph-mon[46152]: pgmap v814: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 570 op/s 2024-05-09T00:11:40.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 382416 27.70 MB/sec execute 575 sec latency 173.180 ms 2024-05-09T00:11:41.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 382855 27.69 MB/sec execute 576 sec latency 132.115 ms 2024-05-09T00:11:42.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:41 smithi012 ceph-mon[42503]: pgmap v815: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.8 MiB/s wr, 487 op/s 2024-05-09T00:11:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:41 smithi204 ceph-mon[47373]: pgmap v815: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.8 MiB/s wr, 487 op/s 2024-05-09T00:11:42.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:41 smithi160 ceph-mon[46152]: pgmap v815: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.8 MiB/s wr, 487 op/s 2024-05-09T00:11:42.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 383266 27.69 MB/sec execute 577 sec latency 183.689 ms 2024-05-09T00:11:43.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 383649 27.67 MB/sec execute 578 sec latency 116.686 ms 2024-05-09T00:11:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:43 smithi012 ceph-mon[42503]: pgmap v816: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.5 MiB/s wr, 572 op/s 2024-05-09T00:11:44.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:43 smithi204 ceph-mon[47373]: pgmap v816: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.5 MiB/s wr, 572 op/s 2024-05-09T00:11:44.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:43 smithi160 ceph-mon[46152]: pgmap v816: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.5 MiB/s wr, 572 op/s 2024-05-09T00:11:44.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 384154 27.68 MB/sec execute 579 sec latency 120.741 ms 2024-05-09T00:11:45.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 384461 27.68 MB/sec execute 580 sec latency 291.147 ms 2024-05-09T00:11:46.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:45 smithi012 ceph-mon[42503]: pgmap v817: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 472 op/s 2024-05-09T00:11:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:45 smithi204 ceph-mon[47373]: pgmap v817: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 472 op/s 2024-05-09T00:11:46.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:45 smithi160 ceph-mon[46152]: pgmap v817: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 472 op/s 2024-05-09T00:11:46.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 384821 27.67 MB/sec execute 581 sec latency 171.386 ms 2024-05-09T00:11:47.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 385248 27.66 MB/sec execute 582 sec latency 102.808 ms 2024-05-09T00:11:48.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:47 smithi012 ceph-mon[42503]: pgmap v818: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.0 MiB/s wr, 547 op/s 2024-05-09T00:11:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:47 smithi204 ceph-mon[47373]: pgmap v818: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.0 MiB/s wr, 547 op/s 2024-05-09T00:11:48.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:47 smithi160 ceph-mon[46152]: pgmap v818: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.0 MiB/s wr, 547 op/s 2024-05-09T00:11:48.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 385722 27.65 MB/sec execute 583 sec latency 125.364 ms 2024-05-09T00:11:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.250+0000 7f426f464700 1 -- 172.21.15.12:0/2181987714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4268103cc0 msgr2=0x7f4268107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.250+0000 7f426f464700 1 --2- 172.21.15.12:0/2181987714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4268103cc0 0x7f4268107d00 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f4258009990 tx=0x7f4258009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.251+0000 7f426f464700 1 -- 172.21.15.12:0/2181987714 shutdown_connections 2024-05-09T00:11:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.251+0000 7f426f464700 1 --2- 172.21.15.12:0/2181987714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4268108430 0x7f426810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.251+0000 7f426f464700 1 --2- 172.21.15.12:0/2181987714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4268103310 0x7f42681036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.251+0000 7f426f464700 1 --2- 172.21.15.12:0/2181987714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4268103cc0 0x7f4268107d00 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.251+0000 7f426f464700 1 -- 172.21.15.12:0/2181987714 >> 172.21.15.12:0/2181987714 conn(0x7f42680feb50 msgr2=0x7f4268100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.251+0000 7f426f464700 1 -- 172.21.15.12:0/2181987714 shutdown_connections 2024-05-09T00:11:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.251+0000 7f426f464700 1 -- 172.21.15.12:0/2181987714 wait complete. 2024-05-09T00:11:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426f464700 1 Processor -- start 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426f464700 1 -- start start 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426f464700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4268103310 0x7f4268199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426f464700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4268108430 0x7f4268199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426f464700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426819df00 0x7f426819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426f464700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426819e980 con 0x7f426819df00 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426f464700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426819eaf0 con 0x7f4268108430 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426f464700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426819ecb0 con 0x7f4268103310 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426da01700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426819df00 0x7f426819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426da01700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426819df00 0x7f426819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37662/0 (socket says 172.21.15.12:37662) 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426d200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4268103310 0x7f4268199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426da01700 1 -- 172.21.15.12:0/276901014 learned_addr learned my addr 172.21.15.12:0/276901014 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.252+0000 7f426c9ff700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4268108430 0x7f4268199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f426da01700 1 -- 172.21.15.12:0/276901014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4268103310 msgr2=0x7f4268199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f426da01700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4268103310 0x7f4268199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f426da01700 1 -- 172.21.15.12:0/276901014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4268108430 msgr2=0x7f4268199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f426da01700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4268108430 0x7f4268199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f426da01700 1 -- 172.21.15.12:0/276901014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4264009520 con 0x7f426819df00 2024-05-09T00:11:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f426da01700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426819df00 0x7f426819e380 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f4260007ee0 tx=0x7f426000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f425e7fc700 1 -- 172.21.15.12:0/276901014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000dd20 con 0x7f426819df00 2024-05-09T00:11:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f425e7fc700 1 -- 172.21.15.12:0/276901014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f426000f040 con 0x7f426819df00 2024-05-09T00:11:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f426f464700 1 -- 172.21.15.12:0/276901014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42580095f0 con 0x7f426819df00 2024-05-09T00:11:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f425e7fc700 1 -- 172.21.15.12:0/276901014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600048a0 con 0x7f426819df00 2024-05-09T00:11:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.253+0000 7f426f464700 1 -- 172.21.15.12:0/276901014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42681a6510 con 0x7f426819df00 2024-05-09T00:11:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.256+0000 7f425e7fc700 1 -- 172.21.15.12:0/276901014 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4260003a00 con 0x7f426819df00 2024-05-09T00:11:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.256+0000 7f425e7fc700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42540a1290 0x7f42540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.257+0000 7f425e7fc700 1 -- 172.21.15.12:0/276901014 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f42600cc490 con 0x7f426819df00 2024-05-09T00:11:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.257+0000 7f426d200700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42540a1290 0x7f42540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.257+0000 7f426d200700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42540a1290 0x7f42540a3750 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4264009df0 tx=0x7f4264009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.258+0000 7f427b61eb80 1 Processor -- start 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.258+0000 7f427b61eb80 1 -- start start 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.258+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.258+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d2270 0x5567ea1cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.258+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 0x5567ea1cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.258+0000 7f427b61eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea1230f0 con 0x5567ea1d2270 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.258+0000 7f427b61eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea120ab0 con 0x5567ea1cc480 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.258+0000 7f427b61eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea11c3a0 con 0x5567ea1d1950 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.258+0000 7f426c9ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d2270 0x5567ea1cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426d200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426c9ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d2270 0x5567ea1cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37678/0 (socket says 172.21.15.12:37678) 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426c9ff700 1 -- 172.21.15.12:0/1280909876 learned_addr learned my addr 172.21.15.12:0/1280909876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426da01700 1 --2- 172.21.15.12:0/1280909876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 0x5567ea1cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426d200700 1 -- 172.21.15.12:0/1280909876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 msgr2=0x5567ea1cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426d200700 1 --2- 172.21.15.12:0/1280909876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 0x5567ea1cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426d200700 1 -- 172.21.15.12:0/1280909876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d2270 msgr2=0x5567ea1cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426d200700 1 --2- 172.21.15.12:0/1280909876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d2270 0x5567ea1cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426d200700 1 -- 172.21.15.12:0/1280909876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e9f9d350 con 0x5567ea1d1950 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426c9ff700 1 --2- 172.21.15.12:0/1280909876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d2270 0x5567ea1cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f426d200700 1 --2- 172.21.15.12:0/1280909876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d1d30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4264011ab0 tx=0x7f4264014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc18576953c70f20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f42537fe700 1 -- 172.21.15.12:0/1280909876 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264026430 con 0x5567ea1d1950 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f42537fe700 1 -- 172.21.15.12:0/1280909876 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4264021070 con 0x5567ea1d1950 2024-05-09T00:11:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.259+0000 7f42537fe700 1 -- 172.21.15.12:0/1280909876 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426401d7a0 con 0x5567ea1d1950 2024-05-09T00:11:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.260+0000 7f427b61eb80 1 -- 172.21.15.12:0/1280909876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 msgr2=0x5567ea1d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.260+0000 7f427b61eb80 1 --2- 172.21.15.12:0/1280909876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d1d30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4264011ab0 tx=0x7f4264014000 comp rx=0 tx=0).stop 2024-05-09T00:11:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.260+0000 7f427b61eb80 1 -- 172.21.15.12:0/1280909876 shutdown_connections 2024-05-09T00:11:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.260+0000 7f427b61eb80 1 --2- 172.21.15.12:0/1280909876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 0x5567ea1cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.260+0000 7f427b61eb80 1 --2- 172.21.15.12:0/1280909876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d1d30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.260+0000 7f427b61eb80 1 --2- 172.21.15.12:0/1280909876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d2270 0x5567ea1cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.260+0000 7f427b61eb80 1 -- 172.21.15.12:0/1280909876 >> 172.21.15.12:0/1280909876 conn(0x5567ea12a1b0 msgr2=0x5567ea1c6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.260+0000 7f427b61eb80 1 -- 172.21.15.12:0/1280909876 shutdown_connections 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.260+0000 7f427b61eb80 1 -- 172.21.15.12:0/1280909876 wait complete. 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.261+0000 7f427b61eb80 1 -- start start 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.261+0000 7f427b61eb80 1 Processor -- start 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.261+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1cc480 0x5567ea1d5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.261+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.261+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 0x5567ea1d5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.261+0000 7f427b61eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea0b8500 con 0x5567ea1cc480 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.261+0000 7f427b61eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea0b81a0 con 0x5567ea1d2270 2024-05-09T00:11:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.261+0000 7f427b61eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e9f9f640 con 0x5567ea1d1950 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.261+0000 7f427b61eb80 10 client.0 _ll_register_callbacks cb 0x5567ea0c1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f426c9ff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f426da01700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 0x5567ea1d5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f426c9ff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54766/0 (socket says 172.21.15.12:54766) 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f426c9ff700 1 -- 172.21.15.12:0/3708359 learned_addr learned my addr 172.21.15.12:0/3708359 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f426c9ff700 1 -- 172.21.15.12:0/3708359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 msgr2=0x5567ea1d5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f426c9ff700 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 0x5567ea1d5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f426c9ff700 1 -- 172.21.15.12:0/3708359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1cc480 msgr2=0x5567ea1d5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f426c9ff700 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1cc480 0x5567ea1d5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f426c9ff700 1 -- 172.21.15.12:0/3708359 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4268110820 con 0x5567ea1d1950 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f426c9ff700 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d5a50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4258005920 tx=0x7f4258005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30c4b957c3b650c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f4250ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f4250ff9700 1 -- 172.21.15.12:0/3708359 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425801d070 con 0x5567ea1d1950 2024-05-09T00:11:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f4250ff9700 1 -- 172.21.15.12:0/3708359 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4258004500 con 0x5567ea1d1950 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f427b61eb80 1 -- 172.21.15.12:0/3708359 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567ea075210 con 0x5567ea1d1950 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f4250ff9700 1 -- 172.21.15.12:0/3708359 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42580046c0 con 0x5567ea1d1950 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f427b61eb80 10 client.31924 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.262+0000 7f427b61eb80 1 -- 172.21.15.12:0/3708359 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7f4264017070 con 0x5567ea1d1950 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.263+0000 7f4250ff9700 1 -- 172.21.15.12:0/3708359 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567ea075210 con 0x5567ea1d1950 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.264+0000 7f4250ff9700 1 -- 172.21.15.12:0/3708359 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4258004820 con 0x5567ea1d1950 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.264+0000 7f427b61eb80 10 client.31924 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.264+0000 7f427b61eb80 10 client.31924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.264+0000 7f427b61eb80 20 client.31924 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.264+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567ea1e5cd0 0x5567ea1e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.264+0000 7f427b61eb80 4 client.31924 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.264+0000 7f427b61eb80 1 -- 172.21.15.12:0/3708359 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567ea19b7b0 con 0x5567ea1e5cd0 2024-05-09T00:11:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.264+0000 7f426d200700 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567ea1e5cd0 0x5567ea1e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.264+0000 7f426d200700 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567ea1e5cd0 0x5567ea1e80f0 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c6582c4876bb07 server_cookie=fdab71f30ee2613a in_seq=0 out_seq=0 2024-05-09T00:11:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.264+0000 7f4250ff9700 10 client.31924 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.267+0000 7f4250ff9700 1 -- 172.21.15.12:0/3708359 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567ea19b7b0 con 0x5567ea1e5cd0 2024-05-09T00:11:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.267+0000 7f4250ff9700 10 client.31924 handle_command_reply: tid=0 2024-05-09T00:11:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.267+0000 7f427b61eb80 1 client.31924 shutdown 2024-05-09T00:11:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.267+0000 7f427b61eb80 1 -- 172.21.15.12:0/3708359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 msgr2=0x5567ea1d5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.267+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d5a50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4258005920 tx=0x7f4258005fb0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.268+0000 7f427b61eb80 1 -- 172.21.15.12:0/3708359 shutdown_connections 2024-05-09T00:11:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.268+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567ea1e5cd0 0x5567ea1e80f0 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.268+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 0x5567ea1d5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.268+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1cc480 0x5567ea1d5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.268+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3708359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1d5a50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.268+0000 7f427b61eb80 1 -- 172.21.15.12:0/3708359 >> 172.21.15.12:0/3708359 conn(0x5567ea12a1b0 msgr2=0x5567ea1caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.268+0000 7f4250ff9700 0 client.31924 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.268+0000 7f427b61eb80 1 -- 172.21.15.12:0/3708359 shutdown_connections 2024-05-09T00:11:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.268+0000 7f427b61eb80 1 -- 172.21.15.12:0/3708359 wait complete. 2024-05-09T00:11:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.268+0000 7f427b61eb80 20 client.31924 trim_cache size 0 max 16384 2024-05-09T00:11:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f427b61eb80 1 Processor -- start 2024-05-09T00:11:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f427b61eb80 1 -- start start 2024-05-09T00:11:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 0x5567ea1c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d1950 0x5567ea1cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d2270 0x5567ea1d5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f427b61eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea1c1350 con 0x5567ea1d1950 2024-05-09T00:11:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f427b61eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea1d6bb0 con 0x5567ea1cc480 2024-05-09T00:11:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f427b61eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea1d85d0 con 0x5567ea1d2270 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f426c9ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d1950 0x5567ea1cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f426d200700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 0x5567ea1c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f426c9ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d1950 0x5567ea1cfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37706/0 (socket says 172.21.15.12:37706) 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f426d200700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 0x5567ea1c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49660/0 (socket says 172.21.15.12:49660) 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.287+0000 7f426c9ff700 1 -- 172.21.15.12:0/40766203 learned_addr learned my addr 172.21.15.12:0/40766203 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f426da01700 1 --2- 172.21.15.12:0/40766203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d2270 0x5567ea1d5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f426c9ff700 1 -- 172.21.15.12:0/40766203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d2270 msgr2=0x5567ea1d5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f426c9ff700 1 --2- 172.21.15.12:0/40766203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d2270 0x5567ea1d5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f426c9ff700 1 -- 172.21.15.12:0/40766203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 msgr2=0x5567ea1c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f426c9ff700 1 --2- 172.21.15.12:0/40766203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 0x5567ea1c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f426c9ff700 1 -- 172.21.15.12:0/40766203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e9f49b00 con 0x5567ea1d1950 2024-05-09T00:11:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f426d200700 1 --2- 172.21.15.12:0/40766203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 0x5567ea1c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f426c9ff700 1 --2- 172.21.15.12:0/40766203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d1950 0x5567ea1cfa90 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f4258009cc0 tx=0x7f4258009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4a3ca1fe7f3d35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f4253fff700 1 -- 172.21.15.12:0/40766203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425801d070 con 0x5567ea1d1950 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f4253fff700 1 -- 172.21.15.12:0/40766203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f425801e040 con 0x5567ea1d1950 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.288+0000 7f4253fff700 1 -- 172.21.15.12:0/40766203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425800f930 con 0x5567ea1d1950 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.289+0000 7f427b61eb80 1 -- 172.21.15.12:0/40766203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d1950 msgr2=0x5567ea1cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.289+0000 7f427b61eb80 1 --2- 172.21.15.12:0/40766203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d1950 0x5567ea1cfa90 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f4258009cc0 tx=0x7f4258009310 comp rx=0 tx=0).stop 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.289+0000 7f427b61eb80 1 -- 172.21.15.12:0/40766203 shutdown_connections 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.289+0000 7f427b61eb80 1 --2- 172.21.15.12:0/40766203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d2270 0x5567ea1d5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.289+0000 7f427b61eb80 1 --2- 172.21.15.12:0/40766203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1cc480 0x5567ea1c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.289+0000 7f427b61eb80 1 --2- 172.21.15.12:0/40766203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1d1950 0x5567ea1cfa90 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.289+0000 7f427b61eb80 1 -- 172.21.15.12:0/40766203 >> 172.21.15.12:0/40766203 conn(0x5567ea12a1b0 msgr2=0x5567ea1cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.289+0000 7f427b61eb80 1 -- 172.21.15.12:0/40766203 shutdown_connections 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.289+0000 7f427b61eb80 1 -- 172.21.15.12:0/40766203 wait complete. 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.290+0000 7f427b61eb80 1 -- start start 2024-05-09T00:11:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.290+0000 7f427b61eb80 1 Processor -- start 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1cc480 0x5567ea1dcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1dd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f427b61eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 0x5567ea1dac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f427b61eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea1db370 con 0x5567ea1cc480 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f427b61eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea1db4e0 con 0x5567ea1d2270 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f427b61eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ea1db6a0 con 0x5567ea1d1950 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f427b61eb80 10 client.0 _ll_register_callbacks cb 0x5567ea0c1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426c9ff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426da01700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 0x5567ea1dac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426c9ff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1dd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54788/0 (socket says 172.21.15.12:54788) 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426da01700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 0x5567ea1dac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49666/0 (socket says 172.21.15.12:49666) 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426c9ff700 1 -- 172.21.15.12:0/3625505384 learned_addr learned my addr 172.21.15.12:0/3625505384 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426d200700 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1cc480 0x5567ea1dcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426da01700 1 -- 172.21.15.12:0/3625505384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 msgr2=0x5567ea1dd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426da01700 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1dd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426da01700 1 -- 172.21.15.12:0/3625505384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1cc480 msgr2=0x5567ea1dcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426da01700 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1cc480 0x5567ea1dcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.291+0000 7f426da01700 1 -- 172.21.15.12:0/3625505384 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426402a040 con 0x5567ea1d2270 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.292+0000 7f426da01700 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 0x5567ea1dac60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f42600c9c20 tx=0x7f42600b9190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=813a4c35afdbb810 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.292+0000 7f4250ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.292+0000 7f4250ff9700 1 -- 172.21.15.12:0/3625505384 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600b1070 con 0x5567ea1d2270 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.292+0000 7f4250ff9700 1 -- 172.21.15.12:0/3625505384 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f42600b3060 con 0x5567ea1d2270 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.292+0000 7f427b61eb80 1 -- 172.21.15.12:0/3625505384 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4264021070 con 0x5567ea1d2270 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.292+0000 7f4250ff9700 1 -- 172.21.15.12:0/3625505384 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f426009d410 con 0x5567ea1d2270 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.292+0000 7f427b61eb80 10 client.31637 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.292+0000 7f427b61eb80 1 -- 172.21.15.12:0/3625505384 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7f4264017070 con 0x5567ea1d2270 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.292+0000 7f4250ff9700 1 -- 172.21.15.12:0/3625505384 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4260063020 con 0x5567ea1d2270 2024-05-09T00:11:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.293+0000 7f4250ff9700 1 -- 172.21.15.12:0/3625505384 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4260084140 con 0x5567ea1d2270 2024-05-09T00:11:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.293+0000 7f427b61eb80 10 client.31637 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.293+0000 7f427b61eb80 10 client.31637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.293+0000 7f427b61eb80 20 client.31637 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.293+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567ea232810 0x5567ea234c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.293+0000 7f427b61eb80 4 client.31637 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:11:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.293+0000 7f427b61eb80 1 -- 172.21.15.12:0/3625505384 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5567ea114e60 con 0x5567ea232810 2024-05-09T00:11:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.293+0000 7f426d200700 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567ea232810 0x5567ea234c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.293+0000 7f4250ff9700 10 client.31637 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:49.293+0000 7f426d200700 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567ea232810 0x5567ea234c30 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3efa958c6ef6e403 server_cookie=8f927f31f771b944 in_seq=0 out_seq=0 2024-05-09T00:11:49.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 386279 27.65 MB/sec execute 584 sec latency 103.673 ms 2024-05-09T00:11:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.088+0000 7f4250ff9700 1 -- 172.21.15.12:0/3625505384 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5567ea114e60 con 0x5567ea232810 2024-05-09T00:11:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.088+0000 7f4250ff9700 10 client.31637 handle_command_reply: tid=0 2024-05-09T00:11:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 client.31637 shutdown 2024-05-09T00:11:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 -- 172.21.15.12:0/3625505384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 msgr2=0x5567ea1dac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 0x5567ea1dac60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f42600c9c20 tx=0x7f42600b9190 comp rx=0 tx=0).stop 2024-05-09T00:11:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 -- 172.21.15.12:0/3625505384 shutdown_connections 2024-05-09T00:11:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5567ea232810 0x5567ea234c30 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5567ea1d2270 0x5567ea1dac60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5567ea1cc480 0x5567ea1dcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f4250ff9700 0 client.31637 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 --2- 172.21.15.12:0/3625505384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567ea1d1950 0x5567ea1dd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 -- 172.21.15.12:0/3625505384 >> 172.21.15.12:0/3625505384 conn(0x5567ea12a1b0 msgr2=0x5567ea1d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 -- 172.21.15.12:0/3625505384 shutdown_connections 2024-05-09T00:11:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 1 -- 172.21.15.12:0/3625505384 wait complete. 2024-05-09T00:11:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.089+0000 7f427b61eb80 20 client.31637 trim_cache size 0 max 16384 2024-05-09T00:11:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:11:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:11:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:11:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:11:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 -- 172.21.15.12:0/276901014 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42540a1290 msgr2=0x7f42540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42540a1290 0x7f42540a3750 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4264009df0 tx=0x7f4264009260 comp rx=0 tx=0).stop 2024-05-09T00:11:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 -- 172.21.15.12:0/276901014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426819df00 msgr2=0x7f426819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426819df00 0x7f426819e380 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f4260007ee0 tx=0x7f426000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 -- 172.21.15.12:0/276901014 shutdown_connections 2024-05-09T00:11:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f42540a1290 0x7f42540a3750 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426819df00 0x7f426819e380 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4268103310 0x7f4268199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 --2- 172.21.15.12:0/276901014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4268108430 0x7f4268199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 -- 172.21.15.12:0/276901014 >> 172.21.15.12:0/276901014 conn(0x7f42680feb50 msgr2=0x7f42680ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 -- 172.21.15.12:0/276901014 shutdown_connections 2024-05-09T00:11:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:11:50.090+0000 7f426f464700 1 -- 172.21.15.12:0/276901014 wait complete. 2024-05-09T00:11:50.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:49 smithi012 ceph-mon[42503]: pgmap v819: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.7 MiB/s wr, 506 op/s 2024-05-09T00:11:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:49 smithi204 ceph-mon[47373]: pgmap v819: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.7 MiB/s wr, 506 op/s 2024-05-09T00:11:50.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:49 smithi160 ceph-mon[46152]: pgmap v819: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.7 MiB/s wr, 506 op/s 2024-05-09T00:11:50.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 386513 27.63 MB/sec execute 585 sec latency 631.318 ms 2024-05-09T00:11:51.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 386851 27.63 MB/sec execute 586 sec latency 209.694 ms 2024-05-09T00:11:51.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.123+0000 7ff88ebf2700 1 -- 172.21.15.12:0/6651344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff888105690 msgr2=0x7ff888105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.123+0000 7ff88ebf2700 1 --2- 172.21.15.12:0/6651344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff888105690 0x7ff888105a70 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009990 tx=0x7ff87c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.123+0000 7ff88ebf2700 1 -- 172.21.15.12:0/6651344 shutdown_connections 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.123+0000 7ff88ebf2700 1 --2- 172.21.15.12:0/6651344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff888100220 0x7ff88810c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.123+0000 7ff88ebf2700 1 --2- 172.21.15.12:0/6651344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff888105690 0x7ff888105a70 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.123+0000 7ff88ebf2700 1 --2- 172.21.15.12:0/6651344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8880ff860 0x7ff8880ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.123+0000 7ff88ebf2700 1 -- 172.21.15.12:0/6651344 >> 172.21.15.12:0/6651344 conn(0x7ff8880751c0 msgr2=0x7ff8880755d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.123+0000 7ff88ebf2700 1 -- 172.21.15.12:0/6651344 shutdown_connections 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.123+0000 7ff88ebf2700 1 -- 172.21.15.12:0/6651344 wait complete. 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88ebf2700 1 Processor -- start 2024-05-09T00:11:52.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88ebf2700 1 -- start start 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88ebf2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8880ff860 0x7ff888101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88ebf2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff888100220 0x7ff888101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88ebf2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff888105690 0x7ff888101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88ebf2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8881a5800 con 0x7ff888105690 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88ebf2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8881a5970 con 0x7ff8880ff860 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88ebf2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8881a5b30 con 0x7ff888100220 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88d18f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff888105690 0x7ff888101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff887fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff888100220 0x7ff888101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88d18f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff888105690 0x7ff888101cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37734/0 (socket says 172.21.15.12:37734) 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff887fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff888100220 0x7ff888101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54810/0 (socket says 172.21.15.12:54810) 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.124+0000 7ff88d18f700 1 -- 172.21.15.12:0/2369277505 learned_addr learned my addr 172.21.15.12:0/2369277505 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff88d18f700 1 -- 172.21.15.12:0/2369277505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff888100220 msgr2=0x7ff888101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff88d18f700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff888100220 0x7ff888101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff88d18f700 1 -- 172.21.15.12:0/2369277505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8880ff860 msgr2=0x7ff888101240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff88d18f700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8880ff860 0x7ff888101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff88d18f700 1 -- 172.21.15.12:0/2369277505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c0095f0 con 0x7ff888105690 2024-05-09T00:11:52.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff887fff700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff888100220 0x7ff888101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:52.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff88d18f700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff888105690 0x7ff888101cc0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff88000ef30 tx=0x7ff88000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff885ffb700 1 -- 172.21.15.12:0/2369277505 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880018070 con 0x7ff888105690 2024-05-09T00:11:52.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff885ffb700 1 -- 172.21.15.12:0/2369277505 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff88000f040 con 0x7ff888105690 2024-05-09T00:11:52.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff88ebf2700 1 -- 172.21.15.12:0/2369277505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8881a5db0 con 0x7ff888105690 2024-05-09T00:11:52.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff885ffb700 1 -- 172.21.15.12:0/2369277505 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88000cdb0 con 0x7ff888105690 2024-05-09T00:11:52.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.125+0000 7ff88ebf2700 1 -- 172.21.15.12:0/2369277505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8881a6270 con 0x7ff888105690 2024-05-09T00:11:52.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.127+0000 7ff885ffb700 1 -- 172.21.15.12:0/2369277505 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff880003a00 con 0x7ff888105690 2024-05-09T00:11:52.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.128+0000 7ff885ffb700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8780a1290 0x7ff8780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.128+0000 7ff885ffb700 1 -- 172.21.15.12:0/2369277505 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff8800cc850 con 0x7ff888105690 2024-05-09T00:11:52.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.128+0000 7ff88c98e700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8780a1290 0x7ff8780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.129+0000 7ff88c98e700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8780a1290 0x7ff8780a3750 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0098e0 tx=0x7ff87c005fb0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.129+0000 7ff89adacb80 1 Processor -- start 2024-05-09T00:11:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.129+0000 7ff89adacb80 1 -- start start 2024-05-09T00:11:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.129+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce5fe11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fdb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fdbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff89adacb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5f32150 con 0x558ce5fdb890 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff89adacb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5f2fb10 con 0x558ce5fe1720 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff89adacb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5f2b400 con 0x558ce5fe0e00 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88c98e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce5fe11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88c98e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce5fe11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54812/0 (socket says 172.21.15.12:54812) 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88c98e700 1 -- 172.21.15.12:0/309909743 learned_addr learned my addr 172.21.15.12:0/309909743 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88d18f700 1 --2- 172.21.15.12:0/309909743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fdbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff887fff700 1 --2- 172.21.15.12:0/309909743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fdb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88c98e700 1 -- 172.21.15.12:0/309909743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 msgr2=0x558ce5fdb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88c98e700 1 --2- 172.21.15.12:0/309909743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fdb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88c98e700 1 -- 172.21.15.12:0/309909743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 msgr2=0x558ce5fdbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88c98e700 1 --2- 172.21.15.12:0/309909743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fdbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88c98e700 1 -- 172.21.15.12:0/309909743 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ce5dac3b0 con 0x558ce5fe0e00 2024-05-09T00:11:52.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88d18f700 1 --2- 172.21.15.12:0/309909743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fdbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff88c98e700 1 --2- 172.21.15.12:0/309909743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce5fe11e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00bb70 tx=0x7ff87c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f3047e498d13be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff872ffd700 1 -- 172.21.15.12:0/309909743 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c029070 con 0x558ce5fe0e00 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff872ffd700 1 -- 172.21.15.12:0/309909743 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff87c00be80 con 0x558ce5fe0e00 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.130+0000 7ff872ffd700 1 -- 172.21.15.12:0/309909743 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c01d070 con 0x558ce5fe0e00 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.131+0000 7ff89adacb80 1 -- 172.21.15.12:0/309909743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 msgr2=0x558ce5fe11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.131+0000 7ff89adacb80 1 --2- 172.21.15.12:0/309909743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce5fe11e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00bb70 tx=0x7ff87c01d000 comp rx=0 tx=0).stop 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.131+0000 7ff89adacb80 1 -- 172.21.15.12:0/309909743 shutdown_connections 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.131+0000 7ff89adacb80 1 --2- 172.21.15.12:0/309909743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fdbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.131+0000 7ff89adacb80 1 --2- 172.21.15.12:0/309909743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce5fe11e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.131+0000 7ff89adacb80 1 --2- 172.21.15.12:0/309909743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fdb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.131+0000 7ff89adacb80 1 -- 172.21.15.12:0/309909743 >> 172.21.15.12:0/309909743 conn(0x558ce5f39210 msgr2=0x558ce5fd5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.131+0000 7ff89adacb80 1 -- 172.21.15.12:0/309909743 shutdown_connections 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.131+0000 7ff89adacb80 1 -- 172.21.15.12:0/309909743 wait complete. 2024-05-09T00:11:52.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.132+0000 7ff89adacb80 1 -- start start 2024-05-09T00:11:52.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.132+0000 7ff89adacb80 1 Processor -- start 2024-05-09T00:11:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.132+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fe4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.132+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce5fe5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.132+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fe5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.132+0000 7ff89adacb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5ec7560 con 0x558ce5fdb890 2024-05-09T00:11:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.132+0000 7ff89adacb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5ec7200 con 0x558ce5fe1720 2024-05-09T00:11:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.132+0000 7ff89adacb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5dae6a0 con 0x558ce5fe0e00 2024-05-09T00:11:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff89adacb80 10 client.0 _ll_register_callbacks cb 0x558ce5ed0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:52.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff88c98e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fe4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff88c98e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fe4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37748/0 (socket says 172.21.15.12:37748) 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff88c98e700 1 -- 172.21.15.12:0/3505596187 learned_addr learned my addr 172.21.15.12:0/3505596187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff88d18f700 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fe5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff88c98e700 1 -- 172.21.15.12:0/3505596187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 msgr2=0x558ce5fe5040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff88c98e700 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce5fe5040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff88c98e700 1 -- 172.21.15.12:0/3505596187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 msgr2=0x558ce5fe5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff88c98e700 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fe5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff88c98e700 1 -- 172.21.15.12:0/3505596187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8881a6270 con 0x558ce5fdb890 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff88c98e700 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fe4b00 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00bb40 tx=0x7ff87c004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69959e2b15316dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff867fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff867fff700 1 -- 172.21.15.12:0/3505596187 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c029040 con 0x558ce5fdb890 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff867fff700 1 -- 172.21.15.12:0/3505596187 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff87c02e660 con 0x558ce5fdb890 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.133+0000 7ff89adacb80 1 -- 172.21.15.12:0/3505596187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ce5e84270 con 0x558ce5fdb890 2024-05-09T00:11:52.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff867fff700 1 -- 172.21.15.12:0/3505596187 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff87c02e820 con 0x558ce5fdb890 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff89adacb80 10 client.22269 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff89adacb80 1 -- 172.21.15.12:0/3505596187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7ff888068c00 con 0x558ce5fdb890 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff867fff700 1 -- 172.21.15.12:0/3505596187 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff87c040020 con 0x558ce5fdb890 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff867fff700 1 -- 172.21.15.12:0/3505596187 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff87c03ca40 con 0x558ce5fdb890 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff89adacb80 10 client.22269 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff89adacb80 10 client.22269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff89adacb80 20 client.22269 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff89adacb80 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ce5ff4e80 0x558ce5ff72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff89adacb80 4 client.22269 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.134+0000 7ff89adacb80 1 -- 172.21.15.12:0/3505596187 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ce5faa810 con 0x558ce5ff4e80 2024-05-09T00:11:52.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.136+0000 7ff887fff700 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ce5ff4e80 0x558ce5ff72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.137+0000 7ff887fff700 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ce5ff4e80 0x558ce5ff72a0 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=959492ca44127303 server_cookie=8c602b1049d7fe28 in_seq=0 out_seq=0 2024-05-09T00:11:52.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.137+0000 7ff867fff700 10 client.22269 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff867fff700 1 -- 172.21.15.12:0/3505596187 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ce5faa810 con 0x558ce5ff4e80 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff867fff700 10 client.22269 handle_command_reply: tid=0 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff89adacb80 1 client.22269 shutdown 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff89adacb80 1 -- 172.21.15.12:0/3505596187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 msgr2=0x558ce5fe4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff89adacb80 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fe4b00 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00bb40 tx=0x7ff87c004f60 comp rx=0 tx=0).stop 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff89adacb80 1 -- 172.21.15.12:0/3505596187 shutdown_connections 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff89adacb80 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ce5ff4e80 0x558ce5ff72a0 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff89adacb80 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fe5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff89adacb80 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce5fe4b00 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff867fff700 0 client.22269 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff89adacb80 1 --2- 172.21.15.12:0/3505596187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce5fe5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.139+0000 7ff89adacb80 1 -- 172.21.15.12:0/3505596187 >> 172.21.15.12:0/3505596187 conn(0x558ce5f39210 msgr2=0x558ce5fd55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.140+0000 7ff89adacb80 1 -- 172.21.15.12:0/3505596187 shutdown_connections 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.140+0000 7ff89adacb80 1 -- 172.21.15.12:0/3505596187 wait complete. 2024-05-09T00:11:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.140+0000 7ff89adacb80 20 client.22269 trim_cache size 0 max 16384 2024-05-09T00:11:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.160+0000 7ff89adacb80 1 Processor -- start 2024-05-09T00:11:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.160+0000 7ff89adacb80 1 -- start start 2024-05-09T00:11:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.160+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fdb890 0x558ce5ff0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.160+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fe0e00 0x558ce5fd5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.160+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fe53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.160+0000 7ff89adacb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5fe6170 con 0x558ce5fe0e00 2024-05-09T00:11:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.160+0000 7ff89adacb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5fe5f90 con 0x558ce5fe1720 2024-05-09T00:11:52.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.160+0000 7ff89adacb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5fd7a20 con 0x558ce5fdb890 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff887fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fe0e00 0x558ce5fd5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff887fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fe0e00 0x558ce5fd5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37750/0 (socket says 172.21.15.12:37750) 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff887fff700 1 -- 172.21.15.12:0/2355787396 learned_addr learned my addr 172.21.15.12:0/2355787396 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff88c98e700 1 --2- 172.21.15.12:0/2355787396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fdb890 0x558ce5ff0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff88d18f700 1 --2- 172.21.15.12:0/2355787396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fe53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff887fff700 1 -- 172.21.15.12:0/2355787396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fdb890 msgr2=0x558ce5ff0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff887fff700 1 --2- 172.21.15.12:0/2355787396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fdb890 0x558ce5ff0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff887fff700 1 -- 172.21.15.12:0/2355787396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 msgr2=0x558ce5fe53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff887fff700 1 --2- 172.21.15.12:0/2355787396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fe53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff887fff700 1 -- 172.21.15.12:0/2355787396 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ce5d58b60 con 0x558ce5fe0e00 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff887fff700 1 --2- 172.21.15.12:0/2355787396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fe0e00 0x558ce5fd5d20 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x558ce5fe1b80 tx=0x7ff874013840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aaa9fad4508b021 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.161+0000 7ff8737fe700 1 -- 172.21.15.12:0/2355787396 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874017040 con 0x558ce5fe0e00 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff8737fe700 1 -- 172.21.15.12:0/2355787396 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff874022070 con 0x558ce5fe0e00 2024-05-09T00:11:52.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff8737fe700 1 -- 172.21.15.12:0/2355787396 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874010b20 con 0x558ce5fe0e00 2024-05-09T00:11:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff89adacb80 1 -- 172.21.15.12:0/2355787396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fe0e00 msgr2=0x558ce5fd5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff89adacb80 1 --2- 172.21.15.12:0/2355787396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fe0e00 0x558ce5fd5d20 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x558ce5fe1b80 tx=0x7ff874013840 comp rx=0 tx=0).stop 2024-05-09T00:11:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff89adacb80 1 -- 172.21.15.12:0/2355787396 shutdown_connections 2024-05-09T00:11:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff89adacb80 1 --2- 172.21.15.12:0/2355787396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fe53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff89adacb80 1 --2- 172.21.15.12:0/2355787396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fdb890 0x558ce5ff0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff89adacb80 1 --2- 172.21.15.12:0/2355787396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fe0e00 0x558ce5fd5d20 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff89adacb80 1 -- 172.21.15.12:0/2355787396 >> 172.21.15.12:0/2355787396 conn(0x558ce5f39210 msgr2=0x558ce5fe33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff89adacb80 1 -- 172.21.15.12:0/2355787396 shutdown_connections 2024-05-09T00:11:52.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.162+0000 7ff89adacb80 1 -- 172.21.15.12:0/2355787396 wait complete. 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff89adacb80 1 -- start start 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff89adacb80 1 Processor -- start 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce6044660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce6044ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff89adacb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fea520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff89adacb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5feaaf0 con 0x558ce5fdb890 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff89adacb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5feac60 con 0x558ce5fe1720 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff89adacb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce5feae20 con 0x558ce5fe0e00 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff89adacb80 10 client.0 _ll_register_callbacks cb 0x558ce5ed0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff88c98e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce6044660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff887fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce6044ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.163+0000 7ff88c98e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce6044660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37762/0 (socket says 172.21.15.12:37762) 2024-05-09T00:11:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff88c98e700 1 -- 172.21.15.12:0/4092715882 learned_addr learned my addr 172.21.15.12:0/4092715882 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff88d18f700 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff887fff700 1 -- 172.21.15.12:0/4092715882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 msgr2=0x558ce5fea520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff887fff700 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fea520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff887fff700 1 -- 172.21.15.12:0/4092715882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 msgr2=0x558ce6044660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff887fff700 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce6044660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff887fff700 1 -- 172.21.15.12:0/4092715882 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff874012050 con 0x558ce5fe0e00 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff887fff700 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce6044ba0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x558ce5fd79d0 tx=0x7ff874010540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=236d087b08de197 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff867fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff867fff700 1 -- 172.21.15.12:0/4092715882 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874017070 con 0x558ce5fe0e00 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff867fff700 1 -- 172.21.15.12:0/4092715882 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7ff87401e800 con 0x558ce5fe0e00 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.164+0000 7ff89adacb80 1 -- 172.21.15.12:0/4092715882 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff87c00be80 con 0x558ce5fe0e00 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff867fff700 1 -- 172.21.15.12:0/4092715882 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff874004030 con 0x558ce5fe0e00 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff89adacb80 10 client.31945 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff89adacb80 1 -- 172.21.15.12:0/4092715882 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x558ce5ef1560 con 0x558ce5fe0e00 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff867fff700 1 -- 172.21.15.12:0/4092715882 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff87c00be80 con 0x558ce5fe0e00 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff867fff700 1 -- 172.21.15.12:0/4092715882 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff8740107a0 con 0x558ce5fe0e00 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff89adacb80 10 client.31945 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff89adacb80 10 client.31945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff89adacb80 20 client.31945 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff89adacb80 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ce604c390 0x558ce604e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff89adacb80 4 client.31945 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff89adacb80 1 -- 172.21.15.12:0/4092715882 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558ce5f23ec0 con 0x558ce604c390 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.165+0000 7ff88c98e700 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ce604c390 0x558ce604e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff88c98e700 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ce604c390 0x558ce604e7b0 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=501c5fb3ad2ddd28 server_cookie=ff25ed52a96bead9 in_seq=0 out_seq=0 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff867fff700 10 client.31945 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff867fff700 1 -- 172.21.15.12:0/4092715882 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558ce5f23ec0 con 0x558ce604c390 2024-05-09T00:11:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff867fff700 10 client.31945 handle_command_reply: tid=0 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 client.31945 shutdown 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 -- 172.21.15.12:0/4092715882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 msgr2=0x558ce6044ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce6044ba0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x558ce5fd79d0 tx=0x7ff874010540 comp rx=0 tx=0).stop 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 -- 172.21.15.12:0/4092715882 shutdown_connections 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x558ce604c390 0x558ce604e7b0 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558ce5fe1720 0x558ce5fea520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff867fff700 0 client.31945 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ce5fdb890 0x558ce6044660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 --2- 172.21.15.12:0/4092715882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ce5fe0e00 0x558ce6044ba0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 -- 172.21.15.12:0/4092715882 >> 172.21.15.12:0/4092715882 conn(0x558ce5f39210 msgr2=0x558ce5fdfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 -- 172.21.15.12:0/4092715882 shutdown_connections 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.166+0000 7ff89adacb80 1 -- 172.21.15.12:0/4092715882 wait complete. 2024-05-09T00:11:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff89adacb80 20 client.31945 trim_cache size 0 max 16384 2024-05-09T00:11:52.170 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:11:52.170 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:11:52.170 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:11:52.170 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 -- 172.21.15.12:0/2369277505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8780a1290 msgr2=0x7ff8780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8780a1290 0x7ff8780a3750 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0098e0 tx=0x7ff87c005fb0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 -- 172.21.15.12:0/2369277505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff888105690 msgr2=0x7ff888101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff888105690 0x7ff888101cc0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff88000ef30 tx=0x7ff88000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 -- 172.21.15.12:0/2369277505 shutdown_connections 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7ff8780a1290 0x7ff8780a3750 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff888105690 0x7ff888101cc0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8880ff860 0x7ff888101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 --2- 172.21.15.12:0/2369277505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff888100220 0x7ff888101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 -- 172.21.15.12:0/2369277505 >> 172.21.15.12:0/2369277505 conn(0x7ff8880751c0 msgr2=0x7ff88810cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 -- 172.21.15.12:0/2369277505 shutdown_connections 2024-05-09T00:11:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.167+0000 7ff88ebf2700 1 -- 172.21.15.12:0/2369277505 wait complete. 2024-05-09T00:11:52.172 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:51 smithi012 ceph-mon[42503]: pgmap v820: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.3 MiB/s wr, 394 op/s 2024-05-09T00:11:52.179 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:11:52.179 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:11:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:51 smithi204 ceph-mon[47373]: pgmap v820: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.3 MiB/s wr, 394 op/s 2024-05-09T00:11:52.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:51 smithi160 ceph-mon[46152]: pgmap v820: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.3 MiB/s wr, 394 op/s 2024-05-09T00:11:52.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.331+0000 7f692b0bd700 1 -- 172.21.15.12:0/292521934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6924106180 msgr2=0x7f692410dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.331+0000 7f692b0bd700 1 --2- 172.21.15.12:0/292521934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6924106180 0x7f692410dd00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f691c00b0a0 tx=0x7f691c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.331+0000 7f692b0bd700 1 -- 172.21.15.12:0/292521934 shutdown_connections 2024-05-09T00:11:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.331+0000 7f692b0bd700 1 --2- 172.21.15.12:0/292521934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6924106180 0x7f692410dd00 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.331+0000 7f692b0bd700 1 --2- 172.21.15.12:0/292521934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6924101060 0x7f6924101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.331+0000 7f692b0bd700 1 --2- 172.21.15.12:0/292521934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6924101a10 0x7f6924105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.331+0000 7f692b0bd700 1 -- 172.21.15.12:0/292521934 >> 172.21.15.12:0/292521934 conn(0x7f6924078360 msgr2=0x7f6924078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.332+0000 7f692b0bd700 1 -- 172.21.15.12:0/292521934 shutdown_connections 2024-05-09T00:11:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.332+0000 7f692b0bd700 1 -- 172.21.15.12:0/292521934 wait complete. 2024-05-09T00:11:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.332+0000 7f692b0bd700 1 Processor -- start 2024-05-09T00:11:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f692b0bd700 1 -- start start 2024-05-09T00:11:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f692b0bd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6924101060 0x7f6924197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f692b0bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6924106180 0x7f6924197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f692b0bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f692419bd30 0x7f692419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f692b0bd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692419c7b0 con 0x7f6924106180 2024-05-09T00:11:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f692b0bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692419c920 con 0x7f692419bd30 2024-05-09T00:11:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f692b0bd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692419cae0 con 0x7f6924101060 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f6923fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6924106180 0x7f6924197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f6928e59700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6924101060 0x7f6924197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f6928e59700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6924101060 0x7f6924197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54858/0 (socket says 172.21.15.12:54858) 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f6923fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6924106180 0x7f6924197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37784/0 (socket says 172.21.15.12:37784) 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f6928e59700 1 -- 172.21.15.12:0/931103660 learned_addr learned my addr 172.21.15.12:0/931103660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f692965a700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f692419bd30 0x7f692419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f6923fff700 1 -- 172.21.15.12:0/931103660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6924101060 msgr2=0x7f6924197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f6923fff700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6924101060 0x7f6924197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f6923fff700 1 -- 172.21.15.12:0/931103660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f692419bd30 msgr2=0x7f692419c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f6923fff700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f692419bd30 0x7f692419c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.333+0000 7f6923fff700 1 -- 172.21.15.12:0/931103660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c009d00 con 0x7f6924106180 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.334+0000 7f6923fff700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6924106180 0x7f6924197990 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f691000e9c0 tx=0x7f691000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.334+0000 7f6921ffb700 1 -- 172.21.15.12:0/931103660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000cbd0 con 0x7f6924106180 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.334+0000 7f692b0bd700 1 -- 172.21.15.12:0/931103660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6924073230 con 0x7f6924106180 2024-05-09T00:11:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.334+0000 7f6921ffb700 1 -- 172.21.15.12:0/931103660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f691000eeb0 con 0x7f6924106180 2024-05-09T00:11:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.334+0000 7f6921ffb700 1 -- 172.21.15.12:0/931103660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910018680 con 0x7f6924106180 2024-05-09T00:11:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.334+0000 7f692b0bd700 1 -- 172.21.15.12:0/931103660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6924073790 con 0x7f6924106180 2024-05-09T00:11:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.336+0000 7f692b0bd700 1 -- 172.21.15.12:0/931103660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6924109450 con 0x7f6924106180 2024-05-09T00:11:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.337+0000 7f6921ffb700 1 -- 172.21.15.12:0/931103660 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6910010430 con 0x7f6924106180 2024-05-09T00:11:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.338+0000 7f6921ffb700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69140a1290 0x7f69140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.338+0000 7f6921ffb700 1 -- 172.21.15.12:0/931103660 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6910014070 con 0x7f6924106180 2024-05-09T00:11:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.338+0000 7f6928e59700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69140a1290 0x7f69140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.339+0000 7f6928e59700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69140a1290 0x7f69140a3750 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6918009d20 tx=0x7f6918009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.336 INFO:tasks.workunit.client.0.smithi012.stdout: 10 387279 27.62 MB/sec execute 587 sec latency 94.066 ms 2024-05-09T00:11:52.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.344+0000 7f6921ffb700 1 -- 172.21.15.12:0/931103660 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6910052d60 con 0x7f6924106180 2024-05-09T00:11:52.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.659+0000 7f692b0bd700 1 -- 172.21.15.12:0/931103660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6924198750 con 0x7f6924106180 2024-05-09T00:11:52.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.660+0000 7f6921ffb700 1 -- 172.21.15.12:0/931103660 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+10875 (secure 0 0 0) 0x7f69100962e0 con 0x7f6924106180 2024-05-09T00:11:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:11:52.657 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":21435,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/676326208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":676326208},{"type":"v1","addr":"172.21.15.160:6833","nonce":676326208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":86,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:11:33.601743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_31894":{"gid":31894,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3656191694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3656191694},{"type":"v1","addr":"172.21.15.204:6833","nonce":3656191694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:11:52.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 -- 172.21.15.12:0/931103660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69140a1290 msgr2=0x7f69140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69140a1290 0x7f69140a3750 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6918009d20 tx=0x7f6918009260 comp rx=0 tx=0).stop 2024-05-09T00:11:52.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 -- 172.21.15.12:0/931103660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6924106180 msgr2=0x7f6924197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6924106180 0x7f6924197990 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f691000e9c0 tx=0x7f691000ecd0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 -- 172.21.15.12:0/931103660 shutdown_connections 2024-05-09T00:11:52.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f69140a1290 0x7f69140a3750 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f692419bd30 0x7f692419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6924101060 0x7f6924197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 --2- 172.21.15.12:0/931103660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6924106180 0x7f6924197990 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 -- 172.21.15.12:0/931103660 >> 172.21.15.12:0/931103660 conn(0x7f6924078360 msgr2=0x7f69240fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 -- 172.21.15.12:0/931103660 shutdown_connections 2024-05-09T00:11:52.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.662+0000 7f692b0bd700 1 -- 172.21.15.12:0/931103660 wait complete. 2024-05-09T00:11:52.661 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 86 2024-05-09T00:11:52.675 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:11:52.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.826+0000 7f41f5b7f700 1 -- 172.21.15.12:0/2515733645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41f0100060 msgr2=0x7f41f0100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.826+0000 7f41f5b7f700 1 --2- 172.21.15.12:0/2515733645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41f0100060 0x7f41f0100440 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009990 tx=0x7f41e0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.826+0000 7f41f5b7f700 1 -- 172.21.15.12:0/2515733645 shutdown_connections 2024-05-09T00:11:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.826+0000 7f41f5b7f700 1 --2- 172.21.15.12:0/2515733645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f0105160 0x7f41f010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.826+0000 7f41f5b7f700 1 --2- 172.21.15.12:0/2515733645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41f0100060 0x7f41f0100440 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.826+0000 7f41f5b7f700 1 --2- 172.21.15.12:0/2515733645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f0100a10 0x7f41f0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.826+0000 7f41f5b7f700 1 -- 172.21.15.12:0/2515733645 >> 172.21.15.12:0/2515733645 conn(0x7f41f0076f40 msgr2=0x7f41f0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.826+0000 7f41f5b7f700 1 -- 172.21.15.12:0/2515733645 shutdown_connections 2024-05-09T00:11:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.826+0000 7f41f5b7f700 1 -- 172.21.15.12:0/2515733645 wait complete. 2024-05-09T00:11:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.827+0000 7f41f5b7f700 1 Processor -- start 2024-05-09T00:11:52.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.827+0000 7f41f5b7f700 1 -- start start 2024-05-09T00:11:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.827+0000 7f41f5b7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f0100060 0x7f41f0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.827+0000 7f41f5b7f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f0105160 0x7f41f0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41f5b7f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41f0198c90 0x7f41f01a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41f5b7f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f01a3930 con 0x7f41f0198c90 2024-05-09T00:11:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41f5b7f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f01a3aa0 con 0x7f41f0100060 2024-05-09T00:11:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41f5b7f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f01a3c60 con 0x7f41f0105160 2024-05-09T00:11:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41f0198c90 0x7f41f01a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41ef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f0100060 0x7f41f0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41f0198c90 0x7f41f01a3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37796/0 (socket says 172.21.15.12:37796) 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41ef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f0100060 0x7f41f0198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49756/0 (socket says 172.21.15.12:49756) 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41effff700 1 -- 172.21.15.12:0/210180861 learned_addr learned my addr 172.21.15.12:0/210180861 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41eeffd700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f0105160 0x7f41f0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41ef7fe700 1 -- 172.21.15.12:0/210180861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f0105160 msgr2=0x7f41f0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f0105160 0x7f41f0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41ef7fe700 1 -- 172.21.15.12:0/210180861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41f0198c90 msgr2=0x7f41f01a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41f0198c90 0x7f41f01a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41ef7fe700 1 -- 172.21.15.12:0/210180861 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00095f0 con 0x7f41f0100060 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41effff700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41f0198c90 0x7f41f01a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f0100060 0x7f41f0198210 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f41e0000c00 tx=0x7f41e0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41ecff9700 1 -- 172.21.15.12:0/210180861 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e001d070 con 0x7f41f0100060 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.828+0000 7f41f5b7f700 1 -- 172.21.15.12:0/210180861 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f01a3ee0 con 0x7f41f0100060 2024-05-09T00:11:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.829+0000 7f41ecff9700 1 -- 172.21.15.12:0/210180861 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41e0021410 con 0x7f41f0100060 2024-05-09T00:11:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.829+0000 7f41ecff9700 1 -- 172.21.15.12:0/210180861 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0017650 con 0x7f41f0100060 2024-05-09T00:11:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.829+0000 7f41f5b7f700 1 -- 172.21.15.12:0/210180861 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f01a4300 con 0x7f41f0100060 2024-05-09T00:11:52.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.831+0000 7f41ecff9700 1 -- 172.21.15.12:0/210180861 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f41e0021580 con 0x7f41f0100060 2024-05-09T00:11:52.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.832+0000 7f41ecff9700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41d80a1290 0x7f41d80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.832+0000 7f41ecff9700 1 -- 172.21.15.12:0/210180861 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f41e00cdc50 con 0x7f41f0100060 2024-05-09T00:11:52.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.832+0000 7f41eeffd700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41d80a1290 0x7f41d80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.832+0000 7f4201d39b80 1 Processor -- start 2024-05-09T00:11:52.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.832+0000 7f4201d39b80 1 -- start start 2024-05-09T00:11:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 0x555dc6c9c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6c96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c96ca0 0x555dc6c97100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f4201d39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6be9150 con 0x555dc6c9cb10 2024-05-09T00:11:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f4201d39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6be6b10 con 0x555dc6c9c1f0 2024-05-09T00:11:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f4201d39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6be2400 con 0x555dc6c96ca0 2024-05-09T00:11:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41eeffd700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41d80a1290 0x7f41d80a3750 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f41f00687e0 tx=0x7f41e4009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6c96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41ef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 0x555dc6c9c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41ef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 0x555dc6c9c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49764/0 (socket says 172.21.15.12:49764) 2024-05-09T00:11:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6c96760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37806/0 (socket says 172.21.15.12:37806) 2024-05-09T00:11:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41ef7fe700 1 -- 172.21.15.12:0/1846993779 learned_addr learned my addr 172.21.15.12:0/1846993779 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41effff700 1 --2- 172.21.15.12:0/1846993779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c96ca0 0x555dc6c97100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41eeffd700 1 -- 172.21.15.12:0/1846993779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c96ca0 msgr2=0x555dc6c97100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41eeffd700 1 --2- 172.21.15.12:0/1846993779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c96ca0 0x555dc6c97100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41eeffd700 1 -- 172.21.15.12:0/1846993779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 msgr2=0x555dc6c9c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41eeffd700 1 --2- 172.21.15.12:0/1846993779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 0x555dc6c9c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41eeffd700 1 -- 172.21.15.12:0/1846993779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dc6a633b0 con 0x555dc6c9cb10 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/1846993779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 0x555dc6c9c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41eeffd700 1 --2- 172.21.15.12:0/1846993779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6c96760 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f41e4011c40 tx=0x7f41e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69003d2f799a17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.833+0000 7f41d5ffb700 1 -- 172.21.15.12:0/1846993779 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400eb00 con 0x555dc6c9cb10 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f41d5ffb700 1 -- 172.21.15.12:0/1846993779 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41e4021070 con 0x555dc6c9cb10 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f41d5ffb700 1 -- 172.21.15.12:0/1846993779 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4005440 con 0x555dc6c9cb10 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f4201d39b80 1 -- 172.21.15.12:0/1846993779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 msgr2=0x555dc6c96760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f4201d39b80 1 --2- 172.21.15.12:0/1846993779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6c96760 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f41e4011c40 tx=0x7f41e4014000 comp rx=0 tx=0).stop 2024-05-09T00:11:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f4201d39b80 1 -- 172.21.15.12:0/1846993779 shutdown_connections 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f4201d39b80 1 --2- 172.21.15.12:0/1846993779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c96ca0 0x555dc6c97100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f4201d39b80 1 --2- 172.21.15.12:0/1846993779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 0x555dc6c9c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f4201d39b80 1 --2- 172.21.15.12:0/1846993779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6c96760 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f4201d39b80 1 -- 172.21.15.12:0/1846993779 >> 172.21.15.12:0/1846993779 conn(0x555dc6bf0210 msgr2=0x555dc6c8cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f4201d39b80 1 -- 172.21.15.12:0/1846993779 shutdown_connections 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.834+0000 7f4201d39b80 1 -- 172.21.15.12:0/1846993779 wait complete. 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.835+0000 7f4201d39b80 1 -- start start 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.835+0000 7f4201d39b80 1 Processor -- start 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.835+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 0x555dc6c9ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.835+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.835+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 0x555dc6ca09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f4201d39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6b7e560 con 0x555dc6c9c1f0 2024-05-09T00:11:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f4201d39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6b7e200 con 0x555dc6c96ca0 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f4201d39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6a656a0 con 0x555dc6c9cb10 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f4201d39b80 10 client.0 _ll_register_callbacks cb 0x555dc6b87b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41ef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 0x555dc6c9ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37808/0 (socket says 172.21.15.12:37808) 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41eeffd700 1 -- 172.21.15.12:0/1922089112 learned_addr learned my addr 172.21.15.12:0/1922089112 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41effff700 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 0x555dc6ca09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41eeffd700 1 -- 172.21.15.12:0/1922089112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 msgr2=0x555dc6ca09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41eeffd700 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 0x555dc6ca09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41eeffd700 1 -- 172.21.15.12:0/1922089112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 msgr2=0x555dc6c9ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41eeffd700 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 0x555dc6c9ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41eeffd700 1 -- 172.21.15.12:0/1922089112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f01a4300 con 0x555dc6c9c1f0 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 0x555dc6c9ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:52.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41eeffd700 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca0470 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f41e40099c0 tx=0x7f41e400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330d5a5b171966f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41ceffd700 1 -- 172.21.15.12:0/1922089112 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4005260 con 0x555dc6c9c1f0 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f41ceffd700 1 -- 172.21.15.12:0/1922089112 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f41e40053c0 con 0x555dc6c9c1f0 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.836+0000 7f4201d39b80 1 -- 172.21.15.12:0/1922089112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555dc6b3b270 con 0x555dc6c9c1f0 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f41ceffd700 1 -- 172.21.15.12:0/1922089112 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41e4005580 con 0x555dc6c9c1f0 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f4201d39b80 10 client.22293 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f4201d39b80 1 -- 172.21.15.12:0/1922089112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7f41e4017070 con 0x555dc6c9c1f0 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f41ceffd700 1 -- 172.21.15.12:0/1922089112 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555dc6b3b270 con 0x555dc6c9c1f0 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f41ceffd700 1 -- 172.21.15.12:0/1922089112 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f41e40268c0 con 0x555dc6c9c1f0 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f4201d39b80 10 client.22293 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f4201d39b80 10 client.22293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f4201d39b80 20 client.22293 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f4201d39b80 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x555dc6cb0540 0x555dc6cb2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f4201d39b80 4 client.22293 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:52.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f4201d39b80 1 -- 172.21.15.12:0/1922089112 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dc6c61810 con 0x555dc6cb0540 2024-05-09T00:11:52.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.837+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x555dc6cb0540 0x555dc6cb2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.838+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x555dc6cb0540 0x555dc6cb2960 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54c45ba67baa0f2d server_cookie=3780c7faf18ffbc9 in_seq=0 out_seq=0 2024-05-09T00:11:52.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.838+0000 7f41ceffd700 10 client.22293 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:11:52.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.839+0000 7f41ceffd700 1 -- 172.21.15.12:0/1922089112 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555dc6c61810 con 0x555dc6cb0540 2024-05-09T00:11:52.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.839+0000 7f41ceffd700 10 client.22293 handle_command_reply: tid=0 2024-05-09T00:11:52.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.839+0000 7f4201d39b80 1 client.22293 shutdown 2024-05-09T00:11:52.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 1 -- 172.21.15.12:0/1922089112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9c1f0 msgr2=0x555dc6ca0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca0470 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f41e40099c0 tx=0x7f41e400eb70 comp rx=0 tx=0).stop 2024-05-09T00:11:52.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 1 -- 172.21.15.12:0/1922089112 shutdown_connections 2024-05-09T00:11:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x555dc6cb0540 0x555dc6cb2960 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 0x555dc6ca09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 0x555dc6c9ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f41ceffd700 0 client.22293 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:11:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 1 --2- 172.21.15.12:0/1922089112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca0470 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 1 -- 172.21.15.12:0/1922089112 >> 172.21.15.12:0/1922089112 conn(0x555dc6bf0210 msgr2=0x555dc6c8c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 1 -- 172.21.15.12:0/1922089112 shutdown_connections 2024-05-09T00:11:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 1 -- 172.21.15.12:0/1922089112 wait complete. 2024-05-09T00:11:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.840+0000 7f4201d39b80 20 client.22293 trim_cache size 0 max 16384 2024-05-09T00:11:52.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.855+0000 7f4201d39b80 1 Processor -- start 2024-05-09T00:11:52.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.855+0000 7f4201d39b80 1 -- start start 2024-05-09T00:11:52.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.855+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c96ca0 0x555dc6c96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.855+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 0x555dc6cac070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.855+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 0x555dc6c9f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.855+0000 7f4201d39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6cad1c0 con 0x555dc6c96ca0 2024-05-09T00:11:52.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.855+0000 7f4201d39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6ca2aa0 con 0x555dc6c9c1f0 2024-05-09T00:11:52.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.855+0000 7f4201d39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6c8da70 con 0x555dc6c9cb10 2024-05-09T00:11:52.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.855+0000 7f41ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c96ca0 0x555dc6c96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c96ca0 0x555dc6c96760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37812/0 (socket says 172.21.15.12:37812) 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41ef7fe700 1 -- 172.21.15.12:0/717851018 learned_addr learned my addr 172.21.15.12:0/717851018 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41eeffd700 1 --2- 172.21.15.12:0/717851018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 0x555dc6cac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41effff700 1 --2- 172.21.15.12:0/717851018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 0x555dc6c9f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41ef7fe700 1 -- 172.21.15.12:0/717851018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 msgr2=0x555dc6c9f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/717851018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 0x555dc6c9f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41ef7fe700 1 -- 172.21.15.12:0/717851018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 msgr2=0x555dc6cac070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/717851018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 0x555dc6cac070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41ef7fe700 1 -- 172.21.15.12:0/717851018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dc6a0fb60 con 0x555dc6c96ca0 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41effff700 1 --2- 172.21.15.12:0/717851018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 0x555dc6c9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/717851018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c96ca0 0x555dc6c96760 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f41e00cb490 tx=0x7f41e002d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2b5f512c51b696 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41d67fc700 1 -- 172.21.15.12:0/717851018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e00b3040 con 0x555dc6c96ca0 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41d67fc700 1 -- 172.21.15.12:0/717851018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41e009af50 con 0x555dc6c96ca0 2024-05-09T00:11:52.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.856+0000 7f41d67fc700 1 -- 172.21.15.12:0/717851018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e009b410 con 0x555dc6c96ca0 2024-05-09T00:11:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.857+0000 7f4201d39b80 1 -- 172.21.15.12:0/717851018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c96ca0 msgr2=0x555dc6c96760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.857+0000 7f4201d39b80 1 --2- 172.21.15.12:0/717851018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c96ca0 0x555dc6c96760 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f41e00cb490 tx=0x7f41e002d040 comp rx=0 tx=0).stop 2024-05-09T00:11:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.857+0000 7f4201d39b80 1 -- 172.21.15.12:0/717851018 shutdown_connections 2024-05-09T00:11:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.857+0000 7f4201d39b80 1 --2- 172.21.15.12:0/717851018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9cb10 0x555dc6c9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.857+0000 7f4201d39b80 1 --2- 172.21.15.12:0/717851018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c96ca0 0x555dc6c96760 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.857+0000 7f4201d39b80 1 --2- 172.21.15.12:0/717851018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c9c1f0 0x555dc6cac070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.857+0000 7f4201d39b80 1 -- 172.21.15.12:0/717851018 >> 172.21.15.12:0/717851018 conn(0x555dc6bf0210 msgr2=0x555dc6c97080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.857+0000 7f4201d39b80 1 -- 172.21.15.12:0/717851018 shutdown_connections 2024-05-09T00:11:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.857+0000 7f4201d39b80 1 -- 172.21.15.12:0/717851018 wait complete. 2024-05-09T00:11:52.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.857+0000 7f4201d39b80 1 -- start start 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f4201d39b80 1 Processor -- start 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 0x555dc6ca75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f4201d39b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6ca55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f4201d39b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6ca5cd0 con 0x555dc6c9cb10 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f4201d39b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6ca5e40 con 0x555dc6c96ca0 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f4201d39b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dc6ca6000 con 0x555dc6c9c1f0 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f4201d39b80 10 client.0 _ll_register_callbacks cb 0x555dc6b87b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f41eeffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f41eeffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54918/0 (socket says 172.21.15.12:54918) 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f41ef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 0x555dc6ca75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f41eeffd700 1 -- 172.21.15.12:0/2122395187 learned_addr learned my addr 172.21.15.12:0/2122395187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.858+0000 7f41effff700 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6ca55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f41eeffd700 1 -- 172.21.15.12:0/2122395187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 msgr2=0x555dc6ca75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f41eeffd700 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 0x555dc6ca75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f41eeffd700 1 -- 172.21.15.12:0/2122395187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 msgr2=0x555dc6ca55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:52.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f41eeffd700 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6ca55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f41eeffd700 1 -- 172.21.15.12:0/2122395187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f01a3ee0 con 0x555dc6c9c1f0 2024-05-09T00:11:52.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f41effff700 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6ca55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:52.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f41eeffd700 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca7b00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x555dc6c8d9c0 tx=0x7f41e400ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ab697836851dca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f41ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:52.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f41ceffd700 1 -- 172.21.15.12:0/2122395187 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400e690 con 0x555dc6c9c1f0 2024-05-09T00:11:52.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f41ceffd700 1 -- 172.21.15.12:0/2122395187 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f41e401bde0 con 0x555dc6c9c1f0 2024-05-09T00:11:52.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.859+0000 7f4201d39b80 1 -- 172.21.15.12:0/2122395187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41e4021070 con 0x555dc6c9c1f0 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f41ceffd700 1 -- 172.21.15.12:0/2122395187 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41e4005260 con 0x555dc6c9c1f0 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f4201d39b80 10 client.31969 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f4201d39b80 1 -- 172.21.15.12:0/2122395187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x555dc6ba8560 con 0x555dc6c9c1f0 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f41ceffd700 1 -- 172.21.15.12:0/2122395187 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41e4048020 con 0x555dc6c9c1f0 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f41ceffd700 1 -- 172.21.15.12:0/2122395187 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f41e4026480 con 0x555dc6c9c1f0 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f4201d39b80 10 client.31969 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f4201d39b80 10 client.31969 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f4201d39b80 20 client.31969 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f4201d39b80 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x555dc6d069a0 0x555dc6d08dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f4201d39b80 4 client.31969 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:11:52.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f4201d39b80 1 -- 172.21.15.12:0/2122395187 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555dc6bdaec0 con 0x555dc6d069a0 2024-05-09T00:11:52.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.864+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x555dc6d069a0 0x555dc6d08dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.865+0000 7f41ceffd700 10 client.31969 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:11:52.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.865+0000 7f41ef7fe700 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x555dc6d069a0 0x555dc6d08dc0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e7bce492a5aa543 server_cookie=b19b0fa52fd6b810 in_seq=0 out_seq=0 2024-05-09T00:11:52.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.867+0000 7f41ceffd700 1 -- 172.21.15.12:0/2122395187 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555dc6bdaec0 con 0x555dc6d069a0 2024-05-09T00:11:52.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.867+0000 7f41ceffd700 10 client.31969 handle_command_reply: tid=0 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.867+0000 7f4201d39b80 1 client.31969 shutdown 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 1 -- 172.21.15.12:0/2122395187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9c1f0 msgr2=0x555dc6ca7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca7b00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x555dc6c8d9c0 tx=0x7f41e400ee90 comp rx=0 tx=0).stop 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 1 -- 172.21.15.12:0/2122395187 shutdown_connections 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc6c9cb10 0x555dc6ca55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555dc6c96ca0 0x555dc6ca75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x555dc6d069a0 0x555dc6d08dc0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 1 --2- 172.21.15.12:0/2122395187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555dc6c9c1f0 0x555dc6ca7b00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f41ceffd700 0 client.31969 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 1 -- 172.21.15.12:0/2122395187 >> 172.21.15.12:0/2122395187 conn(0x555dc6bf0210 msgr2=0x555dc6c9b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 1 -- 172.21.15.12:0/2122395187 shutdown_connections 2024-05-09T00:11:52.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 1 -- 172.21.15.12:0/2122395187 wait complete. 2024-05-09T00:11:52.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f4201d39b80 20 client.31969 trim_cache size 0 max 16384 2024-05-09T00:11:52.867 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f41f5b7f700 1 -- 172.21.15.12:0/210180861 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41d80a1290 msgr2=0x7f41d80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f41f5b7f700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41d80a1290 0x7f41d80a3750 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f41f00687e0 tx=0x7f41e4009260 comp rx=0 tx=0).stop 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f41f5b7f700 1 -- 172.21.15.12:0/210180861 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f0100060 msgr2=0x7f41f0198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.868+0000 7f41f5b7f700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f0100060 0x7f41f0198210 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f41e0000c00 tx=0x7f41e0004730 comp rx=0 tx=0).stop 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.869+0000 7f41f5b7f700 1 -- 172.21.15.12:0/210180861 shutdown_connections 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.869+0000 7f41f5b7f700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f41d80a1290 0x7f41d80a3750 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.869+0000 7f41f5b7f700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41f0198c90 0x7f41f01a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.869+0000 7f41f5b7f700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f0100060 0x7f41f0198210 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.869+0000 7f41f5b7f700 1 --2- 172.21.15.12:0/210180861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f0105160 0x7f41f0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.869+0000 7f41f5b7f700 1 -- 172.21.15.12:0/210180861 >> 172.21.15.12:0/210180861 conn(0x7f41f0076f40 msgr2=0x7f41f00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.869+0000 7f41f5b7f700 1 -- 172.21.15.12:0/210180861 shutdown_connections 2024-05-09T00:11:52.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:52.869+0000 7f41f5b7f700 1 -- 172.21.15.12:0/210180861 wait complete. 2024-05-09T00:11:52.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:11:53.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.038+0000 7fb2d81f5700 1 -- 172.21.15.12:0/1806579816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d0100050 msgr2=0x7fb2d0100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.038+0000 7fb2d81f5700 1 --2- 172.21.15.12:0/1806579816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d0100050 0x7fb2d0100430 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009990 tx=0x7fb2cc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.039+0000 7fb2d81f5700 1 -- 172.21.15.12:0/1806579816 shutdown_connections 2024-05-09T00:11:53.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.039+0000 7fb2d81f5700 1 --2- 172.21.15.12:0/1806579816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d0105170 0x7fb2d010ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.039+0000 7fb2d81f5700 1 --2- 172.21.15.12:0/1806579816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d0100050 0x7fb2d0100430 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.039+0000 7fb2d81f5700 1 --2- 172.21.15.12:0/1806579816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d0100a00 0x7fb2d0104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.039+0000 7fb2d81f5700 1 -- 172.21.15.12:0/1806579816 >> 172.21.15.12:0/1806579816 conn(0x7fb2d0076fa0 msgr2=0x7fb2d00773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.039+0000 7fb2d81f5700 1 -- 172.21.15.12:0/1806579816 shutdown_connections 2024-05-09T00:11:53.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.039+0000 7fb2d81f5700 1 -- 172.21.15.12:0/1806579816 wait complete. 2024-05-09T00:11:53.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d81f5700 1 Processor -- start 2024-05-09T00:11:53.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d81f5700 1 -- start start 2024-05-09T00:11:53.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d81f5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d0100050 0x7fb2d0197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d81f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d0105170 0x7fb2d0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d81f5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d019bd00 0x7fb2d019c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d81f5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d019c780 con 0x7fb2d0105170 2024-05-09T00:11:53.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d81f5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d019c8f0 con 0x7fb2d0100050 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d81f5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d019cab0 con 0x7fb2d019bd00 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d5790700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d0105170 0x7fb2d0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d5790700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d0105170 0x7fb2d0197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37836/0 (socket says 172.21.15.12:37836) 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d5790700 1 -- 172.21.15.12:0/872075739 learned_addr learned my addr 172.21.15.12:0/872075739 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d0100050 0x7fb2d0197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d6792700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d019bd00 0x7fb2d019c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d5790700 1 -- 172.21.15.12:0/872075739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d019bd00 msgr2=0x7fb2d019c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d5790700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d019bd00 0x7fb2d019c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d5790700 1 -- 172.21.15.12:0/872075739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d0100050 msgr2=0x7fb2d0197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d5790700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d0100050 0x7fb2d0197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d5790700 1 -- 172.21.15.12:0/872075739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8009c40 con 0x7fb2d0105170 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.040+0000 7fb2d6792700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d019bd00 0x7fb2d019c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.041+0000 7fb2d5790700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d0105170 0x7fb2d0197960 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000e970 tx=0x7fb2c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.041+0000 7fb2c6ffd700 1 -- 172.21.15.12:0/872075739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0009790 con 0x7fb2d0105170 2024-05-09T00:11:53.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.041+0000 7fb2d81f5700 1 -- 172.21.15.12:0/872075739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc0095f0 con 0x7fb2d0105170 2024-05-09T00:11:53.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.041+0000 7fb2c6ffd700 1 -- 172.21.15.12:0/872075739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2c000c7c0 con 0x7fb2d0105170 2024-05-09T00:11:53.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.041+0000 7fb2c6ffd700 1 -- 172.21.15.12:0/872075739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0005460 con 0x7fb2d0105170 2024-05-09T00:11:53.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.041+0000 7fb2d81f5700 1 -- 172.21.15.12:0/872075739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d01a4350 con 0x7fb2d0105170 2024-05-09T00:11:53.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.044+0000 7fb2c6ffd700 1 -- 172.21.15.12:0/872075739 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb2c000c930 con 0x7fb2d0105170 2024-05-09T00:11:53.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.045+0000 7fb2c6ffd700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2bc0a1290 0x7fb2bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.045+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2bc0a1290 0x7fb2bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.045+0000 7fb2c6ffd700 1 -- 172.21.15.12:0/872075739 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb2c0014070 con 0x7fb2d0105170 2024-05-09T00:11:53.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.045+0000 7fb2e43afb80 1 Processor -- start 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2e43afb80 1 -- start start 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2bc0a1290 0x7fb2bc0a3750 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00b2e0 tx=0x7fb2cc005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4d210 0x55cb3ba4d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4db30 0x55cb3ba47760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b99a150 con 0x55cb3ba47ca0 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b997b10 con 0x55cb3ba4d210 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b993400 con 0x55cb3ba4db30 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2d6792700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2d6792700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37850/0 (socket says 172.21.15.12:37850) 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2d5790700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4db30 0x55cb3ba47760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2d6792700 1 -- 172.21.15.12:0/1786480680 learned_addr learned my addr 172.21.15.12:0/1786480680 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.046+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/1786480680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4d210 0x55cb3ba4d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.047+0000 7fb2d5790700 1 -- 172.21.15.12:0/1786480680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4d210 msgr2=0x55cb3ba4d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.047+0000 7fb2d5790700 1 --2- 172.21.15.12:0/1786480680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4d210 0x55cb3ba4d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.047+0000 7fb2d5790700 1 -- 172.21.15.12:0/1786480680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 msgr2=0x55cb3ba480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.047+0000 7fb2d5790700 1 --2- 172.21.15.12:0/1786480680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.047+0000 7fb2d5790700 1 -- 172.21.15.12:0/1786480680 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb3b8143b0 con 0x55cb3ba4db30 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.047+0000 7fb2d6792700 1 --2- 172.21.15.12:0/1786480680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.047+0000 7fb2d5790700 1 --2- 172.21.15.12:0/1786480680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4db30 0x55cb3ba47760 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00ca590 tx=0x7fb2c009a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45f5f22d94069803 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.047+0000 7fb2bbfff700 1 -- 172.21.15.12:0/1786480680 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00a23d0 con 0x55cb3ba4db30 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.047+0000 7fb2bbfff700 1 -- 172.21.15.12:0/1786480680 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2c000cc30 con 0x55cb3ba4db30 2024-05-09T00:11:53.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.047+0000 7fb2bbfff700 1 -- 172.21.15.12:0/1786480680 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c009a3b0 con 0x55cb3ba4db30 2024-05-09T00:11:53.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.048+0000 7fb2e43afb80 1 -- 172.21.15.12:0/1786480680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4db30 msgr2=0x55cb3ba47760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.048+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/1786480680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4db30 0x55cb3ba47760 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00ca590 tx=0x7fb2c009a6d0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.048+0000 7fb2e43afb80 1 -- 172.21.15.12:0/1786480680 shutdown_connections 2024-05-09T00:11:53.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.048+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/1786480680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.048+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/1786480680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4d210 0x55cb3ba4d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.048+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/1786480680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4db30 0x55cb3ba47760 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.048+0000 7fb2e43afb80 1 -- 172.21.15.12:0/1786480680 >> 172.21.15.12:0/1786480680 conn(0x55cb3b9a1210 msgr2=0x55cb3ba3ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.048+0000 7fb2e43afb80 1 -- 172.21.15.12:0/1786480680 shutdown_connections 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.048+0000 7fb2e43afb80 1 -- 172.21.15.12:0/1786480680 wait complete. 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.049+0000 7fb2e43afb80 1 -- start start 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.049+0000 7fb2e43afb80 1 Processor -- start 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.049+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba53b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.049+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba4d210 0x55cb3ba50ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.049+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.049+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b92f560 con 0x55cb3ba4d210 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.049+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b92f200 con 0x55cb3ba4db30 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.049+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3b8166a0 con 0x55cb3ba47ca0 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2e43afb80 10 client.0 _ll_register_callbacks cb 0x55cb3b938b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5f91700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba53b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5790700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba4d210 0x55cb3ba50ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5f91700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba53b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54966/0 (socket says 172.21.15.12:54966) 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5f91700 1 -- 172.21.15.12:0/2637383815 learned_addr learned my addr 172.21.15.12:0/2637383815 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d6792700 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5790700 1 -- 172.21.15.12:0/2637383815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba47ca0 msgr2=0x55cb3ba53b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5790700 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba53b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5790700 1 -- 172.21.15.12:0/2637383815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 msgr2=0x55cb3ba515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5790700 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5790700 1 -- 172.21.15.12:0/2637383815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c8009260 con 0x55cb3ba4d210 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba53b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2d5790700 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba4d210 0x55cb3ba50ed0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0099c50 tx=0x7fb2c00963c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5885e4d01acbf586 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2b97fa700 1 -- 172.21.15.12:0/2637383815 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00a23d0 con 0x55cb3ba4d210 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2b97fa700 1 -- 172.21.15.12:0/2637383815 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb2c00982f0 con 0x55cb3ba4d210 2024-05-09T00:11:53.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.050+0000 7fb2e43afb80 1 -- 172.21.15.12:0/2637383815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb3b8ec270 con 0x55cb3ba4d210 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2b97fa700 1 -- 172.21.15.12:0/2637383815 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2c00b82c0 con 0x55cb3ba4d210 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2e43afb80 10 client.22311 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2e43afb80 1 -- 172.21.15.12:0/2637383815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7fb2d01a4350 con 0x55cb3ba4d210 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2b97fa700 1 -- 172.21.15.12:0/2637383815 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb3b8ec270 con 0x55cb3ba4d210 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2b97fa700 1 -- 172.21.15.12:0/2637383815 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb2c00a1fd0 con 0x55cb3ba4d210 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2e43afb80 10 client.22311 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2e43afb80 10 client.22311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2e43afb80 20 client.22311 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb3ba61140 0x55cb3ba63530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2e43afb80 4 client.22311 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.051+0000 7fb2e43afb80 1 -- 172.21.15.12:0/2637383815 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb3ba12810 con 0x55cb3ba61140 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.053+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb3ba61140 0x55cb3ba63530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.053+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb3ba61140 0x55cb3ba63530 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=358dfaaba250f24e server_cookie=f9954c530b3a5b23 in_seq=0 out_seq=0 2024-05-09T00:11:53.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.053+0000 7fb2b97fa700 10 client.22311 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.054+0000 7fb2b97fa700 1 -- 172.21.15.12:0/2637383815 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb3ba12810 con 0x55cb3ba61140 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.054+0000 7fb2b97fa700 10 client.22311 handle_command_reply: tid=0 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 client.22311 shutdown 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 -- 172.21.15.12:0/2637383815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba4d210 msgr2=0x55cb3ba50ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba4d210 0x55cb3ba50ed0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0099c50 tx=0x7fb2c00963c0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 -- 172.21.15.12:0/2637383815 shutdown_connections 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb3ba61140 0x55cb3ba63530 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2b97fa700 0 client.22311 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba53b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/2637383815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba4d210 0x55cb3ba50ed0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 -- 172.21.15.12:0/2637383815 >> 172.21.15.12:0/2637383815 conn(0x55cb3b9a1210 msgr2=0x55cb3ba3d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 -- 172.21.15.12:0/2637383815 shutdown_connections 2024-05-09T00:11:53.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 1 -- 172.21.15.12:0/2637383815 wait complete. 2024-05-09T00:11:53.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.055+0000 7fb2e43afb80 20 client.22311 trim_cache size 0 max 16384 2024-05-09T00:11:53.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.069+0000 7fb2e43afb80 1 Processor -- start 2024-05-09T00:11:53.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.069+0000 7fb2e43afb80 1 -- start start 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba50e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba3dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba67e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3ba3ae00 con 0x55cb3ba47ca0 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3ba50cc0 con 0x55cb3ba4db30 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3ba3fe30 con 0x55cb3ba4d210 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5f91700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba50e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5f91700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba50e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37878/0 (socket says 172.21.15.12:37878) 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5f91700 1 -- 172.21.15.12:0/527934015 learned_addr learned my addr 172.21.15.12:0/527934015 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5790700 1 --2- 172.21.15.12:0/527934015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba3dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d6792700 1 --2- 172.21.15.12:0/527934015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba67e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5f91700 1 -- 172.21.15.12:0/527934015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 msgr2=0x55cb3ba3dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/527934015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba3dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5f91700 1 -- 172.21.15.12:0/527934015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 msgr2=0x55cb3ba67e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/527934015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba67e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5f91700 1 -- 172.21.15.12:0/527934015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb3b7c0b60 con 0x55cb3ba47ca0 2024-05-09T00:11:53.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5790700 1 --2- 172.21.15.12:0/527934015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba3dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:53.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.070+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/527934015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba50e90 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0059d0 tx=0x7fb2cc023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8827a8c6a6d6b415 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2c4ff9700 1 -- 172.21.15.12:0/527934015 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00ba00 con 0x55cb3ba47ca0 2024-05-09T00:11:53.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2c4ff9700 1 -- 172.21.15.12:0/527934015 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2cc01e040 con 0x55cb3ba47ca0 2024-05-09T00:11:53.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2c4ff9700 1 -- 172.21.15.12:0/527934015 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00f460 con 0x55cb3ba47ca0 2024-05-09T00:11:53.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2e43afb80 1 -- 172.21.15.12:0/527934015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 msgr2=0x55cb3ba50e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/527934015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba50e90 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0059d0 tx=0x7fb2cc023000 comp rx=0 tx=0).stop 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2e43afb80 1 -- 172.21.15.12:0/527934015 shutdown_connections 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/527934015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba67e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/527934015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba50e90 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/527934015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba3dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2e43afb80 1 -- 172.21.15.12:0/527934015 >> 172.21.15.12:0/527934015 conn(0x55cb3b9a1210 msgr2=0x55cb3ba48080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2e43afb80 1 -- 172.21.15.12:0/527934015 shutdown_connections 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.071+0000 7fb2e43afb80 1 -- 172.21.15.12:0/527934015 wait complete. 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.072+0000 7fb2e43afb80 1 -- start start 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.072+0000 7fb2e43afb80 1 Processor -- start 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.072+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba54540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.072+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba54a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.072+0000 7fb2e43afb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba54fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.072+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3bab66a0 con 0x55cb3ba47ca0 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.072+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3bab6810 con 0x55cb3ba4db30 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.072+0000 7fb2e43afb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3bab69d0 con 0x55cb3ba4d210 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2e43afb80 10 client.0 _ll_register_callbacks cb 0x7fb2c8009050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d5790700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba54a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d5790700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba54a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:54976/0 (socket says 172.21.15.12:54976) 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d5790700 1 -- 172.21.15.12:0/4178443302 learned_addr learned my addr 172.21.15.12:0/4178443302 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d6792700 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba54fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba54540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d5790700 1 -- 172.21.15.12:0/4178443302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 msgr2=0x55cb3ba54fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d5790700 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba54fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d5790700 1 -- 172.21.15.12:0/4178443302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 msgr2=0x55cb3ba54540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d5790700 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba54540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d5790700 1 -- 172.21.15.12:0/4178443302 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d01a4350 con 0x55cb3ba4d210 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2d5790700 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba54a80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0099ee0 tx=0x7fb2c000e8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b8595921027b28e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2b97fa700 1 -- 172.21.15.12:0/4178443302 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c009a6d0 con 0x55cb3ba4d210 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2b97fa700 1 -- 172.21.15.12:0/4178443302 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb2c00a23d0 con 0x55cb3ba4d210 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2e43afb80 1 -- 172.21.15.12:0/4178443302 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2c000cc30 con 0x55cb3ba4d210 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.073+0000 7fb2b97fa700 1 -- 172.21.15.12:0/4178443302 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2c0099f50 con 0x55cb3ba4d210 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2e43afb80 10 client.31987 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2e43afb80 1 -- 172.21.15.12:0/4178443302 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7fb2d010d560 con 0x55cb3ba4d210 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2b97fa700 1 -- 172.21.15.12:0/4178443302 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2c000cc30 con 0x55cb3ba4d210 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2b97fa700 1 -- 172.21.15.12:0/4178443302 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb2c00b2cf0 con 0x55cb3ba4d210 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2e43afb80 10 client.31987 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2e43afb80 10 client.31987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2e43afb80 20 client.31987 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb3bab7c00 0x55cb3bab9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2e43afb80 4 client.31987 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2e43afb80 1 -- 172.21.15.12:0/4178443302 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb3b98bec0 con 0x55cb3bab7c00 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.074+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb3bab7c00 0x55cb3bab9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.075+0000 7fb2b97fa700 10 client.31987 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.075+0000 7fb2d5f91700 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb3bab7c00 0x55cb3bab9ff0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f168d961ef8fc617 server_cookie=1eb2cac25d7084dd in_seq=0 out_seq=0 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2b97fa700 1 -- 172.21.15.12:0/4178443302 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb3b98bec0 con 0x55cb3bab7c00 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2b97fa700 10 client.31987 handle_command_reply: tid=0 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 client.31987 shutdown 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 -- 172.21.15.12:0/4178443302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 msgr2=0x55cb3ba54a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba54a80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0099ee0 tx=0x7fb2c000e8f0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 -- 172.21.15.12:0/4178443302 shutdown_connections 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55cb3bab7c00 0x55cb3bab9ff0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55cb3ba4db30 0x55cb3ba54fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb3ba47ca0 0x55cb3ba54540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2b97fa700 0 client.31987 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:11:53.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 --2- 172.21.15.12:0/4178443302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb3ba4d210 0x55cb3ba54a80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 -- 172.21.15.12:0/4178443302 >> 172.21.15.12:0/4178443302 conn(0x55cb3b9a1210 msgr2=0x55cb3ba4b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 -- 172.21.15.12:0/4178443302 shutdown_connections 2024-05-09T00:11:53.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 1 -- 172.21.15.12:0/4178443302 wait complete. 2024-05-09T00:11:53.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.076+0000 7fb2e43afb80 20 client.31987 trim_cache size 0 max 16384 2024-05-09T00:11:53.079 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:11:53.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 -- 172.21.15.12:0/872075739 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2bc0a1290 msgr2=0x7fb2bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2bc0a1290 0x7fb2bc0a3750 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00b2e0 tx=0x7fb2cc005960 comp rx=0 tx=0).stop 2024-05-09T00:11:53.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 -- 172.21.15.12:0/872075739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d0105170 msgr2=0x7fb2d0197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d0105170 0x7fb2d0197960 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000e970 tx=0x7fb2c000ec80 comp rx=0 tx=0).stop 2024-05-09T00:11:53.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 -- 172.21.15.12:0/872075739 shutdown_connections 2024-05-09T00:11:53.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb2bc0a1290 0x7fb2bc0a3750 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d019bd00 0x7fb2d019c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2d0100050 0x7fb2d0197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 --2- 172.21.15.12:0/872075739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2d0105170 0x7fb2d0197960 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 -- 172.21.15.12:0/872075739 >> 172.21.15.12:0/872075739 conn(0x7fb2d0076fa0 msgr2=0x7fb2d0069870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 -- 172.21.15.12:0/872075739 shutdown_connections 2024-05-09T00:11:53.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.077+0000 7fb2d81f5700 1 -- 172.21.15.12:0/872075739 wait complete. 2024-05-09T00:11:53.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:52 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/931103660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:53.089 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:11:53.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.242+0000 7f561043b700 1 -- 172.21.15.12:0/4037350860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5608105f90 msgr2=0x7f5608106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.242+0000 7f561043b700 1 --2- 172.21.15.12:0/4037350860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5608105f90 0x7f5608106370 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f56040099c0 tx=0x7f5604009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.242+0000 7f561043b700 1 -- 172.21.15.12:0/4037350860 shutdown_connections 2024-05-09T00:11:53.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.242+0000 7f561043b700 1 --2- 172.21.15.12:0/4037350860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5608100ab0 0x7f560810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.242+0000 7f561043b700 1 --2- 172.21.15.12:0/4037350860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5608105f90 0x7f5608106370 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.242+0000 7f561043b700 1 --2- 172.21.15.12:0/4037350860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56081000f0 0x7f5608100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.242+0000 7f561043b700 1 -- 172.21.15.12:0/4037350860 >> 172.21.15.12:0/4037350860 conn(0x7f5608076ba0 msgr2=0x7f5608076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.242+0000 7f561043b700 1 -- 172.21.15.12:0/4037350860 shutdown_connections 2024-05-09T00:11:53.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.242+0000 7f561043b700 1 -- 172.21.15.12:0/4037350860 wait complete. 2024-05-09T00:11:53.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f561043b700 1 Processor -- start 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f561043b700 1 -- start start 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f561043b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56081000f0 0x7f5608197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f561043b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5608100ab0 0x7f56081978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f561043b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5608105f90 0x7f560819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f561043b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560819c250 con 0x7f5608100ab0 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f561043b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560819c3c0 con 0x7f5608105f90 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f561043b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560819c580 con 0x7f56081000f0 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f560d9d6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5608100ab0 0x7f56081978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f560d9d6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5608100ab0 0x7f56081978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37906/0 (socket says 172.21.15.12:37906) 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f560d9d6700 1 -- 172.21.15.12:0/2925459433 learned_addr learned my addr 172.21.15.12:0/2925459433 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f560e1d7700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56081000f0 0x7f5608197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.243+0000 7f560e9d8700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5608105f90 0x7f560819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f560d9d6700 1 -- 172.21.15.12:0/2925459433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56081000f0 msgr2=0x7f5608197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f560d9d6700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56081000f0 0x7f5608197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f560d9d6700 1 -- 172.21.15.12:0/2925459433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5608105f90 msgr2=0x7f560819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f560d9d6700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5608105f90 0x7f560819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f560d9d6700 1 -- 172.21.15.12:0/2925459433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56040095f0 con 0x7f5608100ab0 2024-05-09T00:11:53.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f560d9d6700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5608100ab0 0x7f56081978c0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f55f800e9a0 tx=0x7f55f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f55ff7fe700 1 -- 172.21.15.12:0/2925459433 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8009790 con 0x7f5608100ab0 2024-05-09T00:11:53.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f561043b700 1 -- 172.21.15.12:0/2925459433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560819c860 con 0x7f5608100ab0 2024-05-09T00:11:53.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f55ff7fe700 1 -- 172.21.15.12:0/2925459433 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55f8010430 con 0x7f5608100ab0 2024-05-09T00:11:53.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f55ff7fe700 1 -- 172.21.15.12:0/2925459433 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8018680 con 0x7f5608100ab0 2024-05-09T00:11:53.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.244+0000 7f561043b700 1 -- 172.21.15.12:0/2925459433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56081a40a0 con 0x7f5608100ab0 2024-05-09T00:11:53.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.247+0000 7f55ff7fe700 1 -- 172.21.15.12:0/2925459433 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f55f80105a0 con 0x7f5608100ab0 2024-05-09T00:11:53.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.248+0000 7f55ff7fe700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55f40a1290 0x7f55f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.248+0000 7f560e1d7700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55f40a1290 0x7f55f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.248+0000 7f55ff7fe700 1 -- 172.21.15.12:0/2925459433 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f55f8014070 con 0x7f5608100ab0 2024-05-09T00:11:53.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.248+0000 7f560e1d7700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55f40a1290 0x7f55f40a3750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5604009990 tx=0x7f5604019040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.248+0000 7f561c5f5b80 1 Processor -- start 2024-05-09T00:11:53.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f561c5f5b80 1 -- start start 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748210 0x564cea7485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748b30 0x564cea742760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 0x564cea7430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea695150 con 0x564cea748210 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea692b10 con 0x564cea748b30 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea68e400 con 0x564cea742ca0 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f560e1d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748210 0x564cea7485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f560e1d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748210 0x564cea7485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37914/0 (socket says 172.21.15.12:37914) 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f560e1d7700 1 -- 172.21.15.12:0/250560510 learned_addr learned my addr 172.21.15.12:0/250560510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f560e9d8700 1 --2- 172.21.15.12:0/250560510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 0x564cea7430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f560d9d6700 1 --2- 172.21.15.12:0/250560510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748b30 0x564cea742760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f560e1d7700 1 -- 172.21.15.12:0/250560510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 msgr2=0x564cea7430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f560e1d7700 1 --2- 172.21.15.12:0/250560510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 0x564cea7430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.249+0000 7f560e1d7700 1 -- 172.21.15.12:0/250560510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748b30 msgr2=0x564cea742760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.250+0000 7f560e1d7700 1 --2- 172.21.15.12:0/250560510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748b30 0x564cea742760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.250+0000 7f560e1d7700 1 -- 172.21.15.12:0/250560510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cea50f3b0 con 0x564cea748210 2024-05-09T00:11:53.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.250+0000 7f560e1d7700 1 --2- 172.21.15.12:0/250560510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748210 0x564cea7485f0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f5604000c00 tx=0x7f560401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88362fc99a204402 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.250+0000 7f55fcda4700 1 -- 172.21.15.12:0/250560510 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400b9c0 con 0x564cea748210 2024-05-09T00:11:53.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.250+0000 7f55fcda4700 1 -- 172.21.15.12:0/250560510 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f560401d040 con 0x564cea748210 2024-05-09T00:11:53.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.250+0000 7f55fcda4700 1 -- 172.21.15.12:0/250560510 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400f840 con 0x564cea748210 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 -- 172.21.15.12:0/250560510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748210 msgr2=0x564cea7485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/250560510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748210 0x564cea7485f0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f5604000c00 tx=0x7f560401f000 comp rx=0 tx=0).stop 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 -- 172.21.15.12:0/250560510 shutdown_connections 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/250560510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 0x564cea7430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/250560510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748210 0x564cea7485f0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/250560510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748b30 0x564cea742760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 -- 172.21.15.12:0/250560510 >> 172.21.15.12:0/250560510 conn(0x564cea69c210 msgr2=0x564cea738df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 -- 172.21.15.12:0/250560510 shutdown_connections 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 -- 172.21.15.12:0/250560510 wait complete. 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 -- start start 2024-05-09T00:11:53.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.251+0000 7f561c5f5b80 1 Processor -- start 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 0x564cea74bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748210 0x564cea74c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748b30 0x564cea74c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea62a560 con 0x564cea748b30 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea62a200 con 0x564cea748210 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea5116a0 con 0x564cea742ca0 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f561c5f5b80 10 client.0 _ll_register_callbacks cb 0x564cea633b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f560e1d7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 0x564cea74bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f560e9d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748b30 0x564cea74c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f560e1d7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 0x564cea74bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:55012/0 (socket says 172.21.15.12:55012) 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f560e1d7700 1 -- 172.21.15.12:0/2956421847 learned_addr learned my addr 172.21.15.12:0/2956421847 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f560e9d8700 1 -- 172.21.15.12:0/2956421847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 msgr2=0x564cea74bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f560e9d8700 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 0x564cea74bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f560e9d8700 1 -- 172.21.15.12:0/2956421847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748210 msgr2=0x564cea74c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f560e9d8700 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748210 0x564cea74c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.252+0000 7f560e9d8700 1 -- 172.21.15.12:0/2956421847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560810d550 con 0x564cea748b30 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f560e9d8700 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748b30 0x564cea74c980 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f5600011e90 tx=0x7f5600014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84615cb416c8b99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f55f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f55f1ffb700 1 -- 172.21.15.12:0/2956421847 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560000e040 con 0x564cea748b30 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f561c5f5b80 1 -- 172.21.15.12:0/2956421847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564cea5e7270 con 0x564cea748b30 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f55f1ffb700 1 -- 172.21.15.12:0/2956421847 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f56000093a0 con 0x564cea748b30 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f55f1ffb700 1 -- 172.21.15.12:0/2956421847 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5600004db0 con 0x564cea748b30 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f561c5f5b80 10 client.22329 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f561c5f5b80 1 -- 172.21.15.12:0/2956421847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x564cea4bbb60 con 0x564cea748b30 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f55f1ffb700 1 -- 172.21.15.12:0/2956421847 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564cea5e7270 con 0x564cea748b30 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f55f1ffb700 1 -- 172.21.15.12:0/2956421847 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f560001c030 con 0x564cea748b30 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f561c5f5b80 10 client.22329 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f561c5f5b80 10 client.22329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f561c5f5b80 20 client.22329 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.253+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564cea75c650 0x564cea75ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.254+0000 7f561c5f5b80 4 client.22329 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.254+0000 7f561c5f5b80 1 -- 172.21.15.12:0/2956421847 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cea70d810 con 0x564cea75c650 2024-05-09T00:11:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.254+0000 7f560e1d7700 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564cea75c650 0x564cea75ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.254+0000 7f560e1d7700 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564cea75c650 0x564cea75ea70 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37ab0ff7209ebecc server_cookie=9dabb8c8733470bb in_seq=0 out_seq=0 2024-05-09T00:11:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.254+0000 7f55f1ffb700 10 client.22329 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:53.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f55f1ffb700 1 -- 172.21.15.12:0/2956421847 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cea70d810 con 0x564cea75c650 2024-05-09T00:11:53.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f55f1ffb700 10 client.22329 handle_command_reply: tid=0 2024-05-09T00:11:53.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 client.22329 shutdown 2024-05-09T00:11:53.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 -- 172.21.15.12:0/2956421847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748b30 msgr2=0x564cea74c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748b30 0x564cea74c980 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f5600011e90 tx=0x7f5600014600 comp rx=0 tx=0).stop 2024-05-09T00:11:53.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 -- 172.21.15.12:0/2956421847 shutdown_connections 2024-05-09T00:11:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea748b30 0x564cea74c980 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea742ca0 0x564cea74bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564cea75c650 0x564cea75ea70 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/2956421847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748210 0x564cea74c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 -- 172.21.15.12:0/2956421847 >> 172.21.15.12:0/2956421847 conn(0x564cea69c210 msgr2=0x564cea7385a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f55f1ffb700 0 client.22329 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 -- 172.21.15.12:0/2956421847 shutdown_connections 2024-05-09T00:11:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 1 -- 172.21.15.12:0/2956421847 wait complete. 2024-05-09T00:11:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.256+0000 7f561c5f5b80 20 client.22329 trim_cache size 0 max 16384 2024-05-09T00:11:53.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:52 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/931103660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:53.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.270+0000 7f561c5f5b80 1 Processor -- start 2024-05-09T00:11:53.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.270+0000 7f561c5f5b80 1 -- start start 2024-05-09T00:11:53.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea738d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748210 0x564cea74b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748b30 0x564cea753530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea739b60 con 0x564cea742ca0 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea749f50 con 0x564cea748b30 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea748970 con 0x564cea748210 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f560e1d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea738d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f560e1d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea738d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37938/0 (socket says 172.21.15.12:37938) 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f560e1d7700 1 -- 172.21.15.12:0/691064889 learned_addr learned my addr 172.21.15.12:0/691064889 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f560d9d6700 1 --2- 172.21.15.12:0/691064889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748210 0x564cea74b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f560e1d7700 1 -- 172.21.15.12:0/691064889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748210 msgr2=0x564cea74b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f560e1d7700 1 --2- 172.21.15.12:0/691064889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748210 0x564cea74b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f560e1d7700 1 -- 172.21.15.12:0/691064889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748b30 msgr2=0x564cea753530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f560e1d7700 1 --2- 172.21.15.12:0/691064889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748b30 0x564cea753530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f560e1d7700 1 -- 172.21.15.12:0/691064889 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cea654560 con 0x564cea742ca0 2024-05-09T00:11:53.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.271+0000 7f560e1d7700 1 --2- 172.21.15.12:0/691064889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea738d20 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f56040099c0 tx=0x7f5604004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753e10662656affd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.272+0000 7f55fd5a5700 1 -- 172.21.15.12:0/691064889 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604028070 con 0x564cea742ca0 2024-05-09T00:11:53.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.272+0000 7f55fd5a5700 1 -- 172.21.15.12:0/691064889 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5604005650 con 0x564cea742ca0 2024-05-09T00:11:53.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.272+0000 7f55fd5a5700 1 -- 172.21.15.12:0/691064889 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604004c20 con 0x564cea742ca0 2024-05-09T00:11:53.271 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:52 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/931103660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:53.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.272+0000 7f561c5f5b80 1 -- 172.21.15.12:0/691064889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 msgr2=0x564cea738d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.272+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/691064889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea738d20 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f56040099c0 tx=0x7f5604004f00 comp rx=0 tx=0).stop 2024-05-09T00:11:53.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.272+0000 7f561c5f5b80 1 -- 172.21.15.12:0/691064889 shutdown_connections 2024-05-09T00:11:53.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.272+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/691064889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748b30 0x564cea753530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.272+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/691064889 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea738d20 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.272+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/691064889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748210 0x564cea74b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.272+0000 7f561c5f5b80 1 -- 172.21.15.12:0/691064889 >> 172.21.15.12:0/691064889 conn(0x564cea69c210 msgr2=0x564cea73a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 1 -- 172.21.15.12:0/691064889 shutdown_connections 2024-05-09T00:11:53.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 1 -- 172.21.15.12:0/691064889 wait complete. 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 1 -- start start 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 1 Processor -- start 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea7ab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748210 0x564cea7ab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748b30 0x564cea750900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea750ed0 con 0x564cea742ca0 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea751040 con 0x564cea748210 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cea751200 con 0x564cea748b30 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f561c5f5b80 10 client.0 _ll_register_callbacks cb 0x564cea633b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f560e1d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea7ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f560e1d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea7ab000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37946/0 (socket says 172.21.15.12:37946) 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f560e1d7700 1 -- 172.21.15.12:0/762173200 learned_addr learned my addr 172.21.15.12:0/762173200 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.273+0000 7f560d9d6700 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748210 0x564cea7ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f560e9d8700 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748b30 0x564cea750900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f560e1d7700 1 -- 172.21.15.12:0/762173200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748b30 msgr2=0x564cea750900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f560e1d7700 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748b30 0x564cea750900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f560e1d7700 1 -- 172.21.15.12:0/762173200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748210 msgr2=0x564cea7ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f560e1d7700 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748210 0x564cea7ab540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f560e1d7700 1 -- 172.21.15.12:0/762173200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cea50f3b0 con 0x564cea742ca0 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f560e9d8700 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748b30 0x564cea750900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f560d9d6700 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748210 0x564cea7ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f560e1d7700 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea7ab000 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f560400b7b0 tx=0x7f560400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a39f1613b47146d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f55f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f55f1ffb700 1 -- 172.21.15.12:0/762173200 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604028050 con 0x564cea742ca0 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f55f1ffb700 1 -- 172.21.15.12:0/762173200 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f560403de40 con 0x564cea742ca0 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.274+0000 7f561c5f5b80 1 -- 172.21.15.12:0/762173200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f560401d040 con 0x564cea742ca0 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f55f1ffb700 1 -- 172.21.15.12:0/762173200 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5604004d70 con 0x564cea742ca0 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f561c5f5b80 10 client.22335 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:53.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f561c5f5b80 1 -- 172.21.15.12:0/762173200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x564cea513cd0 con 0x564cea742ca0 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f55f1ffb700 1 -- 172.21.15.12:0/762173200 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5604051020 con 0x564cea742ca0 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f55f1ffb700 1 -- 172.21.15.12:0/762173200 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5604043070 con 0x564cea742ca0 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f561c5f5b80 10 client.22335 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f561c5f5b80 10 client.22335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f561c5f5b80 20 client.22335 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564cea7b2930 0x564cea7b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f561c5f5b80 4 client.22335 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f561c5f5b80 1 -- 172.21.15.12:0/762173200 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564cea686ec0 con 0x564cea7b2930 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.275+0000 7f560d9d6700 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564cea7b2930 0x564cea7b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.276+0000 7f560d9d6700 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564cea7b2930 0x564cea7b4d50 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d76ec3e3f6c549 server_cookie=97afbfed48066a4a in_seq=0 out_seq=0 2024-05-09T00:11:53.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.276+0000 7f55f1ffb700 10 client.22335 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:53.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f55f1ffb700 1 -- 172.21.15.12:0/762173200 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564cea686ec0 con 0x564cea7b2930 2024-05-09T00:11:53.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f55f1ffb700 10 client.22335 handle_command_reply: tid=0 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 client.22335 shutdown 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 -- 172.21.15.12:0/762173200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 msgr2=0x564cea7ab000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea7ab000 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f560400b7b0 tx=0x7f560400bf00 comp rx=0 tx=0).stop 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 -- 172.21.15.12:0/762173200 shutdown_connections 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cea748b30 0x564cea750900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564cea742ca0 0x564cea7ab000 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x564cea7b2930 0x564cea7b4d50 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 --2- 172.21.15.12:0/762173200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564cea748210 0x564cea7ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 -- 172.21.15.12:0/762173200 >> 172.21.15.12:0/762173200 conn(0x564cea69c210 msgr2=0x564cea74c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f55f1ffb700 0 client.22335 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 -- 172.21.15.12:0/762173200 shutdown_connections 2024-05-09T00:11:53.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 1 -- 172.21.15.12:0/762173200 wait complete. 2024-05-09T00:11:53.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.280+0000 7f561c5f5b80 20 client.22335 trim_cache size 0 max 16384 2024-05-09T00:11:53.281 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 -- 172.21.15.12:0/2925459433 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55f40a1290 msgr2=0x7f55f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55f40a1290 0x7f55f40a3750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5604009990 tx=0x7f5604019040 comp rx=0 tx=0).stop 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 -- 172.21.15.12:0/2925459433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5608100ab0 msgr2=0x7f56081978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5608100ab0 0x7f56081978c0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f55f800e9a0 tx=0x7f55f800ecb0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 -- 172.21.15.12:0/2925459433 shutdown_connections 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f55f40a1290 0x7f55f40a3750 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5608105f90 0x7f560819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56081000f0 0x7f5608197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 --2- 172.21.15.12:0/2925459433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5608100ab0 0x7f56081978c0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 -- 172.21.15.12:0/2925459433 >> 172.21.15.12:0/2925459433 conn(0x7f5608076ba0 msgr2=0x7f56080fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 -- 172.21.15.12:0/2925459433 shutdown_connections 2024-05-09T00:11:53.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:53.281+0000 7f561043b700 1 -- 172.21.15.12:0/2925459433 wait complete. 2024-05-09T00:11:53.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 387695 27.60 MB/sec execute 588 sec latency 100.944 ms 2024-05-09T00:11:54.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:53 smithi012 ceph-mon[42503]: pgmap v821: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 582 op/s 2024-05-09T00:11:54.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:53 smithi204 ceph-mon[47373]: pgmap v821: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 582 op/s 2024-05-09T00:11:54.270 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:53 smithi160 ceph-mon[46152]: pgmap v821: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 582 op/s 2024-05-09T00:11:54.294 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:11:54.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:11:54.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 388198 27.60 MB/sec execute 589 sec latency 119.171 ms 2024-05-09T00:11:54.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.468+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2995855847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba38103350 msgr2=0x7fba38103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.468+0000 7fba3cf7a700 1 --2- 172.21.15.12:0/2995855847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba38103350 0x7fba38103730 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fba28009990 tx=0x7fba28009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.468+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2995855847 shutdown_connections 2024-05-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.468+0000 7fba3cf7a700 1 --2- 172.21.15.12:0/2995855847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38108470 0x7fba3810fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.468+0000 7fba3cf7a700 1 --2- 172.21.15.12:0/2995855847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba38103350 0x7fba38103730 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.468+0000 7fba3cf7a700 1 --2- 172.21.15.12:0/2995855847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba38103d00 0x7fba38107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.468+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2995855847 >> 172.21.15.12:0/2995855847 conn(0x7fba380febb0 msgr2=0x7fba38100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.468+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2995855847 shutdown_connections 2024-05-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.468+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2995855847 wait complete. 2024-05-09T00:11:54.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.469+0000 7fba3cf7a700 1 Processor -- start 2024-05-09T00:11:54.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.469+0000 7fba3cf7a700 1 -- start start 2024-05-09T00:11:54.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba3cf7a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba38103350 0x7fba38199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba3cf7a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38108470 0x7fba38199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba3cf7a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba3819df70 0x7fba3819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba3cf7a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3819e9f0 con 0x7fba3819df70 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba3cf7a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3819eb60 con 0x7fba38103350 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba3cf7a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3819ed20 con 0x7fba38108470 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba36d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba3819df70 0x7fba3819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba35d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38108470 0x7fba38199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba35d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38108470 0x7fba38199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:55048/0 (socket says 172.21.15.12:55048) 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba36d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba3819df70 0x7fba3819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37966/0 (socket says 172.21.15.12:37966) 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba35d9b700 1 -- 172.21.15.12:0/2075018846 learned_addr learned my addr 172.21.15.12:0/2075018846 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba3659c700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba38103350 0x7fba38199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba35d9b700 1 -- 172.21.15.12:0/2075018846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba38103350 msgr2=0x7fba38199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba35d9b700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba38103350 0x7fba38199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba35d9b700 1 -- 172.21.15.12:0/2075018846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba3819df70 msgr2=0x7fba3819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba35d9b700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba3819df70 0x7fba3819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba35d9b700 1 -- 172.21.15.12:0/2075018846 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba280095f0 con 0x7fba38108470 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.470+0000 7fba36d9d700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba3819df70 0x7fba3819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.471+0000 7fba35d9b700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38108470 0x7fba38199bd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e970 tx=0x7fba2c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.471+0000 7fba277fe700 1 -- 172.21.15.12:0/2075018846 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c009790 con 0x7fba38108470 2024-05-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.471+0000 7fba277fe700 1 -- 172.21.15.12:0/2075018846 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba2c00c7e0 con 0x7fba38108470 2024-05-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.471+0000 7fba277fe700 1 -- 172.21.15.12:0/2075018846 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0054e0 con 0x7fba38108470 2024-05-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.471+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2075018846 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba381a6280 con 0x7fba38108470 2024-05-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.471+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2075018846 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba381a6710 con 0x7fba38108470 2024-05-09T00:11:54.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.473+0000 7fba277fe700 1 -- 172.21.15.12:0/2075018846 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fba2c00c950 con 0x7fba38108470 2024-05-09T00:11:54.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.474+0000 7fba277fe700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fba200a1290 0x7fba200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.474+0000 7fba277fe700 1 -- 172.21.15.12:0/2075018846 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fba2c014070 con 0x7fba38108470 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.475+0000 7fba49134b80 1 Processor -- start 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.475+0000 7fba49134b80 1 -- start start 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba49134b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 0x55de7fe35260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba49134b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe357a0 0x55de7fe2f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba49134b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 0x55de7fe2fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba49134b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fd861d0 con 0x55de7fe357a0 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba49134b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fd83b90 con 0x55de7fe34e80 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba49134b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fd7f480 con 0x55de7fe2f910 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba35d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe357a0 0x55de7fe2f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba36d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 0x55de7fe2fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba35d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe357a0 0x55de7fe2f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37968/0 (socket says 172.21.15.12:37968) 2024-05-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba35d9b700 1 -- 172.21.15.12:0/1308336327 learned_addr learned my addr 172.21.15.12:0/1308336327 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba3659c700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fba200a1290 0x7fba200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba36d9d700 1 -- 172.21.15.12:0/1308336327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 msgr2=0x55de7fe35260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba36d9d700 1 --2- 172.21.15.12:0/1308336327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 0x55de7fe35260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba36d9d700 1 -- 172.21.15.12:0/1308336327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe357a0 msgr2=0x55de7fe2f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba36d9d700 1 --2- 172.21.15.12:0/1308336327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe357a0 0x55de7fe2f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.476+0000 7fba36d9d700 1 -- 172.21.15.12:0/1308336327 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7fc00430 con 0x55de7fe2f910 2024-05-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.477+0000 7fba36d9d700 1 --2- 172.21.15.12:0/1308336327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 0x55de7fe2fd50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fba30003db0 tx=0x7fba30009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5367eed6cb446b4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.477+0000 7fba24ff9700 1 -- 172.21.15.12:0/1308336327 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30004bb0 con 0x55de7fe2f910 2024-05-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.477+0000 7fba24ff9700 1 -- 172.21.15.12:0/1308336327 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba3000f040 con 0x55de7fe2f910 2024-05-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.477+0000 7fba24ff9700 1 -- 172.21.15.12:0/1308336327 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30007d40 con 0x55de7fe2f910 2024-05-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.478+0000 7fba49134b80 1 -- 172.21.15.12:0/1308336327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 msgr2=0x55de7fe2fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.478+0000 7fba3659c700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fba200a1290 0x7fba200a3750 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fba2800b2e0 tx=0x7fba28005180 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.478+0000 7fba49134b80 1 --2- 172.21.15.12:0/1308336327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 0x55de7fe2fd50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fba30003db0 tx=0x7fba30009920 comp rx=0 tx=0).stop 2024-05-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.479+0000 7fba49134b80 1 -- 172.21.15.12:0/1308336327 shutdown_connections 2024-05-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.479+0000 7fba49134b80 1 --2- 172.21.15.12:0/1308336327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 0x55de7fe2fd50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.479+0000 7fba49134b80 1 --2- 172.21.15.12:0/1308336327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 0x55de7fe35260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.479+0000 7fba49134b80 1 --2- 172.21.15.12:0/1308336327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe357a0 0x55de7fe2f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.479+0000 7fba49134b80 1 -- 172.21.15.12:0/1308336327 >> 172.21.15.12:0/1308336327 conn(0x55de7fd8d290 msgr2=0x55de7fe29e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.482+0000 7fba49134b80 1 -- 172.21.15.12:0/1308336327 shutdown_connections 2024-05-09T00:11:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.483+0000 7fba49134b80 1 -- 172.21.15.12:0/1308336327 wait complete. 2024-05-09T00:11:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.483+0000 7fba49134b80 1 -- start start 2024-05-09T00:11:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.483+0000 7fba49134b80 1 Processor -- start 2024-05-09T00:11:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba49134b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 0x55de7fe38c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba49134b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe34e80 0x55de7fe39140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba49134b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe357a0 0x55de7fe39680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba49134b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fd1b5e0 con 0x55de7fe34e80 2024-05-09T00:11:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba49134b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fd1b280 con 0x55de7fe357a0 2024-05-09T00:11:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba49134b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fc02720 con 0x55de7fe2f910 2024-05-09T00:11:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba49134b80 10 client.0 _ll_register_callbacks cb 0x55de7fd24bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba35d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe34e80 0x55de7fe39140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba35d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe34e80 0x55de7fe39140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37976/0 (socket says 172.21.15.12:37976) 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba35d9b700 1 -- 172.21.15.12:0/2939195202 learned_addr learned my addr 172.21.15.12:0/2939195202 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba3659c700 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 0x55de7fe38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba36d9d700 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe357a0 0x55de7fe39680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba35d9b700 1 -- 172.21.15.12:0/2939195202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 msgr2=0x55de7fe38c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba35d9b700 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 0x55de7fe38c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba35d9b700 1 -- 172.21.15.12:0/2939195202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe357a0 msgr2=0x55de7fe39680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba35d9b700 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe357a0 0x55de7fe39680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba35d9b700 1 -- 172.21.15.12:0/2939195202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba30003a60 con 0x55de7fe34e80 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.484+0000 7fba36d9d700 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe357a0 0x55de7fe39680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.485+0000 7fba35d9b700 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe34e80 0x55de7fe39140 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fba2c099d30 tx=0x7fba2c09a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bc42b6369234b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.485+0000 7fba1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.485+0000 7fba1dffb700 1 -- 172.21.15.12:0/2939195202 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0a24b0 con 0x55de7fe34e80 2024-05-09T00:11:54.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.485+0000 7fba49134b80 1 -- 172.21.15.12:0/2939195202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de7fcd82f0 con 0x55de7fe34e80 2024-05-09T00:11:54.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.485+0000 7fba1dffb700 1 -- 172.21.15.12:0/2939195202 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fba2c0b8250 con 0x55de7fe34e80 2024-05-09T00:11:54.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.485+0000 7fba1dffb700 1 -- 172.21.15.12:0/2939195202 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba2c0b8410 con 0x55de7fe34e80 2024-05-09T00:11:54.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.485+0000 7fba49134b80 10 client.22347 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:54.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.485+0000 7fba49134b80 1 -- 172.21.15.12:0/2939195202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55de7fbacbe0 con 0x55de7fe34e80 2024-05-09T00:11:54.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.485+0000 7fba1dffb700 1 -- 172.21.15.12:0/2939195202 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de7fcd82f0 con 0x55de7fe34e80 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.486+0000 7fba1dffb700 1 -- 172.21.15.12:0/2939195202 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fba2c0a2020 con 0x55de7fe34e80 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.486+0000 7fba49134b80 10 client.22347 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.486+0000 7fba49134b80 10 client.22347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.486+0000 7fba49134b80 20 client.22347 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.486+0000 7fba49134b80 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55de7fe473a0 0x55de7fe497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.486+0000 7fba49134b80 4 client.22347 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.486+0000 7fba49134b80 1 -- 172.21.15.12:0/2939195202 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de7fdfe890 con 0x55de7fe473a0 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.486+0000 7fba3659c700 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55de7fe473a0 0x55de7fe497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.486+0000 7fba1dffb700 10 client.22347 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.486+0000 7fba3659c700 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55de7fe473a0 0x55de7fe497c0 crc :-1 s=READY pgs=628 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f68fef07ecefff5c server_cookie=e5997474e4f704d2 in_seq=0 out_seq=0 2024-05-09T00:11:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.488+0000 7fba1dffb700 1 -- 172.21.15.12:0/2939195202 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de7fdfe890 con 0x55de7fe473a0 2024-05-09T00:11:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.488+0000 7fba1dffb700 10 client.22347 handle_command_reply: tid=0 2024-05-09T00:11:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.488+0000 7fba49134b80 1 client.22347 shutdown 2024-05-09T00:11:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.488+0000 7fba49134b80 1 -- 172.21.15.12:0/2939195202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe34e80 msgr2=0x55de7fe39140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.488+0000 7fba49134b80 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe34e80 0x55de7fe39140 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fba2c099d30 tx=0x7fba2c09a7e0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.491+0000 7fba49134b80 1 -- 172.21.15.12:0/2939195202 shutdown_connections 2024-05-09T00:11:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.491+0000 7fba49134b80 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55de7fe473a0 0x55de7fe497c0 crc :-1 s=READY pgs=628 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.491+0000 7fba49134b80 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe357a0 0x55de7fe39680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.491+0000 7fba49134b80 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe2f910 0x55de7fe38c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.491+0000 7fba49134b80 1 --2- 172.21.15.12:0/2939195202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe34e80 0x55de7fe39140 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.491+0000 7fba49134b80 1 -- 172.21.15.12:0/2939195202 >> 172.21.15.12:0/2939195202 conn(0x55de7fd8d290 msgr2=0x55de7fe29620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.494+0000 7fba1dffb700 0 client.22347 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:54.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.494+0000 7fba49134b80 1 -- 172.21.15.12:0/2939195202 shutdown_connections 2024-05-09T00:11:54.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.494+0000 7fba49134b80 1 -- 172.21.15.12:0/2939195202 wait complete. 2024-05-09T00:11:54.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.494+0000 7fba49134b80 20 client.22347 trim_cache size 0 max 16384 2024-05-09T00:11:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.526+0000 7fba49134b80 1 Processor -- start 2024-05-09T00:11:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.526+0000 7fba49134b80 1 -- start start 2024-05-09T00:11:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.526+0000 7fba49134b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe2f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.526+0000 7fba49134b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 0x55de7fe343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.526+0000 7fba49134b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe357a0 0x55de7fe29da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.526+0000 7fba49134b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fe39bd0 con 0x55de7fe2f910 2024-05-09T00:11:54.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.526+0000 7fba49134b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fe34910 con 0x55de7fe34e80 2024-05-09T00:11:54.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.526+0000 7fba49134b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fe3a200 con 0x55de7fe357a0 2024-05-09T00:11:54.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.527+0000 7fba3659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe2f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.527+0000 7fba3659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe2f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37984/0 (socket says 172.21.15.12:37984) 2024-05-09T00:11:54.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.527+0000 7fba3659c700 1 -- 172.21.15.12:0/2769496337 learned_addr learned my addr 172.21.15.12:0/2769496337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:54.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.527+0000 7fba36d9d700 1 --2- 172.21.15.12:0/2769496337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe357a0 0x55de7fe29da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.527+0000 7fba36d9d700 1 -- 172.21.15.12:0/2769496337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 msgr2=0x55de7fe343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.527+0000 7fba36d9d700 1 --2- 172.21.15.12:0/2769496337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 0x55de7fe343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.527+0000 7fba36d9d700 1 -- 172.21.15.12:0/2769496337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 msgr2=0x55de7fe2f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.527+0000 7fba36d9d700 1 --2- 172.21.15.12:0/2769496337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe2f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.527+0000 7fba36d9d700 1 -- 172.21.15.12:0/2769496337 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7fe26560 con 0x55de7fe357a0 2024-05-09T00:11:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.527+0000 7fba36d9d700 1 --2- 172.21.15.12:0/2769496337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe357a0 0x55de7fe29da0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fba30003d50 tx=0x7fba300095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f0972f4cec6a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.528+0000 7fba257fa700 1 -- 172.21.15.12:0/2769496337 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30009190 con 0x55de7fe357a0 2024-05-09T00:11:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.528+0000 7fba257fa700 1 -- 172.21.15.12:0/2769496337 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba300092f0 con 0x55de7fe357a0 2024-05-09T00:11:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.528+0000 7fba257fa700 1 -- 172.21.15.12:0/2769496337 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30007da0 con 0x55de7fe357a0 2024-05-09T00:11:54.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.529+0000 7fba49134b80 1 -- 172.21.15.12:0/2769496337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe357a0 msgr2=0x55de7fe29da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.529+0000 7fba49134b80 1 --2- 172.21.15.12:0/2769496337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe357a0 0x55de7fe29da0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fba30003d50 tx=0x7fba300095e0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.529+0000 7fba49134b80 1 -- 172.21.15.12:0/2769496337 shutdown_connections 2024-05-09T00:11:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.529+0000 7fba49134b80 1 --2- 172.21.15.12:0/2769496337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe357a0 0x55de7fe29da0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.529+0000 7fba49134b80 1 --2- 172.21.15.12:0/2769496337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe2f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.529+0000 7fba49134b80 1 --2- 172.21.15.12:0/2769496337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 0x55de7fe343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.529+0000 7fba49134b80 1 -- 172.21.15.12:0/2769496337 >> 172.21.15.12:0/2769496337 conn(0x55de7fd8d290 msgr2=0x55de7fe33cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.530+0000 7fba49134b80 1 -- 172.21.15.12:0/2769496337 shutdown_connections 2024-05-09T00:11:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.530+0000 7fba49134b80 1 -- 172.21.15.12:0/2769496337 wait complete. 2024-05-09T00:11:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.530+0000 7fba49134b80 1 -- start start 2024-05-09T00:11:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.530+0000 7fba49134b80 1 Processor -- start 2024-05-09T00:11:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba49134b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe3fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba49134b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 0x55de7fe40400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba49134b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe357a0 0x55de7fe3e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba49134b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fe3eb40 con 0x55de7fe2f910 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba49134b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fe3ecb0 con 0x55de7fe34e80 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba49134b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7fe3ee70 con 0x55de7fe357a0 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba49134b80 10 client.0 _ll_register_callbacks cb 0x55de7fd24bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba3659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe3fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba3659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe3fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37998/0 (socket says 172.21.15.12:37998) 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba3659c700 1 -- 172.21.15.12:0/3889402069 learned_addr learned my addr 172.21.15.12:0/3889402069 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba3659c700 1 -- 172.21.15.12:0/3889402069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe357a0 msgr2=0x55de7fe3e5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba3659c700 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe357a0 0x55de7fe3e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba3659c700 1 -- 172.21.15.12:0/3889402069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 msgr2=0x55de7fe40400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba3659c700 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 0x55de7fe40400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.531+0000 7fba3659c700 1 -- 172.21.15.12:0/3889402069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de7fc00430 con 0x55de7fe2f910 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.532+0000 7fba3659c700 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe3fec0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fba28006010 tx=0x7fba28009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14fa8a30454eb2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.532+0000 7fba1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.533+0000 7fba1dffb700 1 -- 172.21.15.12:0/3889402069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2800be90 con 0x55de7fe2f910 2024-05-09T00:11:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.533+0000 7fba1dffb700 1 -- 172.21.15.12:0/3889402069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fba28033070 con 0x55de7fe2f910 2024-05-09T00:11:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.534+0000 7fba49134b80 1 -- 172.21.15.12:0/3889402069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba3000f040 con 0x55de7fe2f910 2024-05-09T00:11:54.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.537+0000 7fba1dffb700 1 -- 172.21.15.12:0/3889402069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba28033230 con 0x55de7fe2f910 2024-05-09T00:11:54.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.537+0000 7fba49134b80 10 client.22353 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:54.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.537+0000 7fba49134b80 1 -- 172.21.15.12:0/3889402069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55de7fd455e0 con 0x55de7fe2f910 2024-05-09T00:11:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.538+0000 7fba1dffb700 1 -- 172.21.15.12:0/3889402069 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba3000f040 con 0x55de7fe2f910 2024-05-09T00:11:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.538+0000 7fba1dffb700 1 -- 172.21.15.12:0/3889402069 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fba2800fae0 con 0x55de7fe2f910 2024-05-09T00:11:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.538+0000 7fba49134b80 10 client.22353 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.538+0000 7fba49134b80 10 client.22353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.538+0000 7fba49134b80 20 client.22353 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.538+0000 7fba49134b80 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55de7fec30d0 0x55de7fec54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.538+0000 7fba49134b80 4 client.22353 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:11:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.538+0000 7fba49134b80 1 -- 172.21.15.12:0/3889402069 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55de7fd77f40 con 0x55de7fec30d0 2024-05-09T00:11:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.538+0000 7fba35d9b700 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55de7fec30d0 0x55de7fec54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.539+0000 7fba1dffb700 10 client.22353 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:54.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.540+0000 7fba35d9b700 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55de7fec30d0 0x55de7fec54f0 crc :-1 s=READY pgs=629 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a01b1e8ea87c721c server_cookie=2a08423560453261 in_seq=0 out_seq=0 2024-05-09T00:11:54.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba1dffb700 1 -- 172.21.15.12:0/3889402069 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55de7fd77f40 con 0x55de7fec30d0 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba1dffb700 10 client.22353 handle_command_reply: tid=0 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 client.22353 shutdown 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 -- 172.21.15.12:0/3889402069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 msgr2=0x55de7fe3fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe3fec0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fba28006010 tx=0x7fba28009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 -- 172.21.15.12:0/3889402069 shutdown_connections 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7fe357a0 0x55de7fe3e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de7fe2f910 0x55de7fe3fec0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55de7fec30d0 0x55de7fec54f0 crc :-1 s=READY pgs=629 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 --2- 172.21.15.12:0/3889402069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55de7fe34e80 0x55de7fe40400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba1dffb700 0 client.22353 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 -- 172.21.15.12:0/3889402069 >> 172.21.15.12:0/3889402069 conn(0x55de7fd8d290 msgr2=0x55de7fe374d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 -- 172.21.15.12:0/3889402069 shutdown_connections 2024-05-09T00:11:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 1 -- 172.21.15.12:0/3889402069 wait complete. 2024-05-09T00:11:54.542 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:11:54.542 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "6a6fc884-720e-478b-adb0-095720207c64", 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.541+0000 7fba49134b80 20 client.22353 trim_cache size 0 max 16384 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2075018846 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fba200a1290 msgr2=0x7fba200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fba200a1290 0x7fba200a3750 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fba2800b2e0 tx=0x7fba28005180 comp rx=0 tx=0).stop 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2075018846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38108470 msgr2=0x7fba38199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38108470 0x7fba38199bd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e970 tx=0x7fba2c00ec80 comp rx=0 tx=0).stop 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2075018846 shutdown_connections 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fba200a1290 0x7fba200a3750 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba3819df70 0x7fba3819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba38103350 0x7fba38199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 --2- 172.21.15.12:0/2075018846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38108470 0x7fba38199bd0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2075018846 >> 172.21.15.12:0/2075018846 conn(0x7fba380febb0 msgr2=0x7fba380fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2075018846 shutdown_connections 2024-05-09T00:11:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.542+0000 7fba3cf7a700 1 -- 172.21.15.12:0/2075018846 wait complete. 2024-05-09T00:11:54.562 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:11:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.929+0000 7f5bc08bb700 1 -- 172.21.15.12:0/3957877521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bbc0ff410 msgr2=0x7f5bbc0ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.929+0000 7f5bc08bb700 1 --2- 172.21.15.12:0/3957877521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bbc0ff410 0x7f5bbc0ff890 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009990 tx=0x7f5bb0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.930+0000 7f5bc08bb700 1 -- 172.21.15.12:0/3957877521 shutdown_connections 2024-05-09T00:11:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.930+0000 7f5bc08bb700 1 --2- 172.21.15.12:0/3957877521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bbc0ffdd0 0x7f5bbc10dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.930+0000 7f5bc08bb700 1 --2- 172.21.15.12:0/3957877521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bbc105ef0 0x7f5bbc1062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.930+0000 7f5bc08bb700 1 --2- 172.21.15.12:0/3957877521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bbc0ff410 0x7f5bbc0ff890 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.930+0000 7f5bc08bb700 1 -- 172.21.15.12:0/3957877521 >> 172.21.15.12:0/3957877521 conn(0x7f5bbc0763a0 msgr2=0x7f5bbc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.930+0000 7f5bc08bb700 1 -- 172.21.15.12:0/3957877521 shutdown_connections 2024-05-09T00:11:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.930+0000 7f5bc08bb700 1 -- 172.21.15.12:0/3957877521 wait complete. 2024-05-09T00:11:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bc08bb700 1 Processor -- start 2024-05-09T00:11:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bc08bb700 1 -- start start 2024-05-09T00:11:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bc08bb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bbc0ff410 0x7f5bbc1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bc08bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bbc0ffdd0 0x7f5bbc1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bc08bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bbc105ef0 0x7f5bbc19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bc08bb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc19c3d0 con 0x7f5bbc105ef0 2024-05-09T00:11:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bc08bb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc19c540 con 0x7f5bbc0ff410 2024-05-09T00:11:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bc08bb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc19c700 con 0x7f5bbc0ffdd0 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bbad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bbc105ef0 0x7f5bbc19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bb9d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bbc0ffdd0 0x7f5bbc1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bbad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bbc105ef0 0x7f5bbc19bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38006/0 (socket says 172.21.15.12:38006) 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bb9d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bbc0ffdd0 0x7f5bbc1979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:55104/0 (socket says 172.21.15.12:55104) 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.931+0000 7f5bbad9d700 1 -- 172.21.15.12:0/2570255377 learned_addr learned my addr 172.21.15.12:0/2570255377 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.932+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/2570255377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bbc0ff410 msgr2=0x7f5bbc1974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.932+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bbc0ff410 0x7f5bbc1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.932+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/2570255377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bbc105ef0 msgr2=0x7f5bbc19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.932+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bbc105ef0 0x7f5bbc19bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.932+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/2570255377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb00095f0 con 0x7f5bbc0ffdd0 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.932+0000 7f5bbad9d700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bbc105ef0 0x7f5bbc19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.932+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bbc0ffdd0 0x7f5bbc1979f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00059d0 tx=0x7f5bb0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.932+0000 7f5bab7fe700 1 -- 172.21.15.12:0/2570255377 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb001d070 con 0x7f5bbc0ffdd0 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.932+0000 7f5bc08bb700 1 -- 172.21.15.12:0/2570255377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc19c980 con 0x7f5bbc0ffdd0 2024-05-09T00:11:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.932+0000 7f5bc08bb700 1 -- 172.21.15.12:0/2570255377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc1a4220 con 0x7f5bbc0ffdd0 2024-05-09T00:11:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.933+0000 7f5bab7fe700 1 -- 172.21.15.12:0/2570255377 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb0022470 con 0x7f5bbc0ffdd0 2024-05-09T00:11:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.933+0000 7f5bab7fe700 1 -- 172.21.15.12:0/2570255377 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0017650 con 0x7f5bbc0ffdd0 2024-05-09T00:11:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bcca75b80 1 Processor -- start 2024-05-09T00:11:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bcca75b80 1 -- start start 2024-05-09T00:11:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dc540 0x5556af0dc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 0x5556af0f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0f74a0 0x5556af0f9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556af033150 con 0x5556af0dce60 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556af030b10 con 0x5556af0dc540 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556af02c400 con 0x5556af0f74a0 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bbad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dc540 0x5556af0dc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bbad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dc540 0x5556af0dc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49954/0 (socket says 172.21.15.12:49954) 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.937+0000 7f5bbad9d700 1 -- 172.21.15.12:0/1672141827 learned_addr learned my addr 172.21.15.12:0/1672141827 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.938+0000 7f5bba59c700 1 --2- 172.21.15.12:0/1672141827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 0x5556af0f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.938+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/1672141827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0f74a0 0x5556af0f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.938+0000 7f5bab7fe700 1 -- 172.21.15.12:0/2570255377 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5bb00177b0 con 0x7f5bbc0ffdd0 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.938+0000 7f5bbad9d700 1 -- 172.21.15.12:0/1672141827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0f74a0 msgr2=0x5556af0f9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.938+0000 7f5bbad9d700 1 --2- 172.21.15.12:0/1672141827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0f74a0 0x5556af0f9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.938+0000 7f5bbad9d700 1 -- 172.21.15.12:0/1672141827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 msgr2=0x5556af0f6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.938+0000 7f5bbad9d700 1 --2- 172.21.15.12:0/1672141827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 0x5556af0f6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.938+0000 7f5bbad9d700 1 -- 172.21.15.12:0/1672141827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556aeead3b0 con 0x5556af0dc540 2024-05-09T00:11:54.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.938+0000 7f5bbad9d700 1 --2- 172.21.15.12:0/1672141827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dc540 0x5556af0dc920 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009530 tx=0x7f5bb4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb14f9794fbd1d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.938+0000 7f5bab7fe700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ba40a1310 0x7f5ba40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.939+0000 7f5bba59c700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ba40a1310 0x7f5ba40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.939+0000 7f5bab7fe700 1 -- 172.21.15.12:0/2570255377 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5bb00ce2a0 con 0x7f5bbc0ffdd0 2024-05-09T00:11:54.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.939+0000 7f5ba8ff9700 1 -- 172.21.15.12:0/1672141827 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400e900 con 0x5556af0dc540 2024-05-09T00:11:54.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.939+0000 7f5ba8ff9700 1 -- 172.21.15.12:0/1672141827 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb400f040 con 0x5556af0dc540 2024-05-09T00:11:54.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.939+0000 7f5ba8ff9700 1 -- 172.21.15.12:0/1672141827 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4037e50 con 0x5556af0dc540 2024-05-09T00:11:54.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.939+0000 7f5bba59c700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ba40a1310 0x7f5ba40a37d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x5556af0d8640 tx=0x7f5bac009190 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.942+0000 7f5bcca75b80 1 -- 172.21.15.12:0/1672141827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dc540 msgr2=0x5556af0dc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.942+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/1672141827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dc540 0x5556af0dc920 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009530 tx=0x7f5bb4009920 comp rx=0 tx=0).stop 2024-05-09T00:11:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.943+0000 7f5bcca75b80 1 -- 172.21.15.12:0/1672141827 shutdown_connections 2024-05-09T00:11:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.943+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/1672141827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0f74a0 0x5556af0f9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.943+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/1672141827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dc540 0x5556af0dc920 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.943+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/1672141827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 0x5556af0f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.943+0000 7f5bcca75b80 1 -- 172.21.15.12:0/1672141827 >> 172.21.15.12:0/1672141827 conn(0x5556af03a210 msgr2=0x5556af0d6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.943+0000 7f5bcca75b80 1 -- 172.21.15.12:0/1672141827 shutdown_connections 2024-05-09T00:11:54.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.946+0000 7f5bcca75b80 1 -- 172.21.15.12:0/1672141827 wait complete. 2024-05-09T00:11:54.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.946+0000 7f5bcca75b80 1 -- start start 2024-05-09T00:11:54.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.946+0000 7f5bcca75b80 1 Processor -- start 2024-05-09T00:11:54.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.946+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dc540 0x5556af0e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 0x5556af0e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0f74a0 0x5556af0ef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556aefc8560 con 0x5556af0dce60 2024-05-09T00:11:54.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556aefc8200 con 0x5556af0f74a0 2024-05-09T00:11:54.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556aeeaf6a0 con 0x5556af0dc540 2024-05-09T00:11:54.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bcca75b80 10 client.0 _ll_register_callbacks cb 0x5556aefd1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:54.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bb9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 0x5556af0e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bb9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 0x5556af0e06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38026/0 (socket says 172.21.15.12:38026) 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/2508475444 learned_addr learned my addr 172.21.15.12:0/2508475444 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bbad9d700 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0f74a0 0x5556af0ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/2508475444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dc540 msgr2=0x5556af0e0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dc540 0x5556af0e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/2508475444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0f74a0 msgr2=0x5556af0ef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0f74a0 0x5556af0ef330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/2508475444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb40091e0 con 0x5556af0dce60 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 0x5556af0e06a0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005f50 tx=0x7f5bb0009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c90b3ebcde8325c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5ba1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.947+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/2508475444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb009ad80 con 0x5556af0dce60 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/2508475444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5bb009bb20 con 0x5556af0dce60 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5bcca75b80 1 -- 172.21.15.12:0/2508475444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556aef85270 con 0x5556af0dce60 2024-05-09T00:11:54.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/2508475444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb009b420 con 0x5556af0dce60 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5bcca75b80 10 client.22365 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5bcca75b80 1 -- 172.21.15.12:0/2508475444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x5556aee59b60 con 0x5556af0dce60 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/2508475444 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5556aef85270 con 0x5556af0dce60 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/2508475444 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bb00a3380 con 0x5556af0dce60 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5bcca75b80 10 client.22365 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5bcca75b80 10 client.22365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5bcca75b80 20 client.22365 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5556af0f33e0 0x5556af0f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5bcca75b80 4 client.22365 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.948+0000 7f5bcca75b80 1 -- 172.21.15.12:0/2508475444 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556af0ab810 con 0x5556af0f33e0 2024-05-09T00:11:54.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.949+0000 7f5bba59c700 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5556af0f33e0 0x5556af0f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.949+0000 7f5ba1ffb700 10 client.22365 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:54.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.949+0000 7f5bba59c700 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5556af0f33e0 0x5556af0f5800 crc :-1 s=READY pgs=630 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ae8f0e45a9be2fd server_cookie=3c59cbc4653b3581 in_seq=0 out_seq=0 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/2508475444 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556af0ab810 con 0x5556af0f33e0 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5ba1ffb700 10 client.22365 handle_command_reply: tid=0 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 client.22365 shutdown 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 -- 172.21.15.12:0/2508475444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 msgr2=0x5556af0e06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 0x5556af0e06a0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005f50 tx=0x7f5bb0009390 comp rx=0 tx=0).stop 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 -- 172.21.15.12:0/2508475444 shutdown_connections 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5556af0f33e0 0x5556af0f5800 crc :-1 s=READY pgs=630 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0f74a0 0x5556af0ef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dc540 0x5556af0e0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/2508475444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dce60 0x5556af0e06a0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 -- 172.21.15.12:0/2508475444 >> 172.21.15.12:0/2508475444 conn(0x5556af03a210 msgr2=0x5556af0d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5ba1ffb700 0 client.22365 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:54.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 -- 172.21.15.12:0/2508475444 shutdown_connections 2024-05-09T00:11:54.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.952+0000 7f5bcca75b80 1 -- 172.21.15.12:0/2508475444 wait complete. 2024-05-09T00:11:54.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.953+0000 7f5bcca75b80 20 client.22365 trim_cache size 0 max 16384 2024-05-09T00:11:54.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.975+0000 7f5bcca75b80 1 Processor -- start 2024-05-09T00:11:54.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.975+0000 7f5bcca75b80 1 -- start start 2024-05-09T00:11:54.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.975+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af03a210 0x5556af0e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.975+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dc540 0x5556af0d6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.975+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dce60 0x5556af0ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.975+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556af0dc920 con 0x5556af0dc540 2024-05-09T00:11:54.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.975+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556af0e3130 con 0x5556af03a210 2024-05-09T00:11:54.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.975+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556af0fe790 con 0x5556af0dce60 2024-05-09T00:11:54.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bb9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dc540 0x5556af0d6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bb9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dc540 0x5556af0d6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38030/0 (socket says 172.21.15.12:38030) 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/3340851023 learned_addr learned my addr 172.21.15.12:0/3340851023 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bba59c700 1 --2- 172.21.15.12:0/3340851023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af03a210 0x5556af0e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bbad9d700 1 --2- 172.21.15.12:0/3340851023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dce60 0x5556af0ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/3340851023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dce60 msgr2=0x5556af0ea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/3340851023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dce60 0x5556af0ea270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/3340851023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af03a210 msgr2=0x5556af0e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/3340851023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af03a210 0x5556af0e04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/3340851023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556af0ddcf0 con 0x5556af0dc540 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/3340851023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dc540 0x5556af0d6d20 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb0005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562b8fd93335405c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.976+0000 7f5ba97fa700 1 -- 172.21.15.12:0/3340851023 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb009ad80 con 0x5556af0dc540 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.977+0000 7f5ba97fa700 1 -- 172.21.15.12:0/3340851023 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb009bb20 con 0x5556af0dc540 2024-05-09T00:11:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.977+0000 7f5ba97fa700 1 -- 172.21.15.12:0/3340851023 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00a3380 con 0x5556af0dc540 2024-05-09T00:11:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.977+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3340851023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dc540 msgr2=0x5556af0d6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.977+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/3340851023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dc540 0x5556af0d6d20 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb0005ad0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.977+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3340851023 shutdown_connections 2024-05-09T00:11:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.977+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/3340851023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dce60 0x5556af0ea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.977+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/3340851023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af03a210 0x5556af0e04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.977+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/3340851023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af0dc540 0x5556af0d6d20 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.977+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3340851023 >> 172.21.15.12:0/3340851023 conn(0x5556af0d8b40 msgr2=0x5556af0e3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3340851023 shutdown_connections 2024-05-09T00:11:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3340851023 wait complete. 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 1 -- start start 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 1 Processor -- start 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af03a210 0x5556af0ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dc540 0x5556af0e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dce60 0x5556af0e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556af0e55d0 con 0x5556af03a210 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556af0e5740 con 0x5556af0dce60 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556af14b670 con 0x5556af0dc540 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bcca75b80 10 client.0 _ll_register_callbacks cb 0x5556aefd1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af03a210 0x5556af0ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bb9d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dc540 0x5556af0e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.978+0000 7f5bba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af03a210 0x5556af0ea5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38034/0 (socket says 172.21.15.12:38034) 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5bba59c700 1 -- 172.21.15.12:0/3424546615 learned_addr learned my addr 172.21.15.12:0/3424546615 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:11:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5bbad9d700 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dce60 0x5556af0e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/3424546615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dce60 msgr2=0x5556af0e4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dce60 0x5556af0e4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/3424546615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af03a210 msgr2=0x5556af0ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af03a210 0x5556af0ea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5bb9d9b700 1 -- 172.21.15.12:0/3424546615 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556aee59b60 con 0x5556af0dc540 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5bb9d9b700 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dc540 0x5556af0e49d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb00cbab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de8c7340b397088f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5ba1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/3424546615 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00d1380 con 0x5556af0dc540 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/3424546615 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5bb009b420 con 0x5556af0dc540 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.979+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3424546615 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bb400f040 con 0x5556af0dc540 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/3424546615 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb009ad80 con 0x5556af0dc540 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5bcca75b80 10 client.32035 fetch_fsmap learned FSMap version 86 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3424546615 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x5556aeff2560 con 0x5556af0dc540 2024-05-09T00:11:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/3424546615 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb400f040 con 0x5556af0dc540 2024-05-09T00:11:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/3424546615 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bb009a380 con 0x5556af0dc540 2024-05-09T00:11:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5bcca75b80 10 client.32035 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:11:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5bcca75b80 10 client.32035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:11:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5bcca75b80 20 client.32035 populate_metadata read hostname 'smithi012' 2024-05-09T00:11:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5556af0f74a0 0x5556af14e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5bcca75b80 4 client.32035 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:11:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3424546615 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5556af024ec0 con 0x5556af0f74a0 2024-05-09T00:11:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.980+0000 7f5bba59c700 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5556af0f74a0 0x5556af14e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.981+0000 7f5bba59c700 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5556af0f74a0 0x5556af14e7e0 crc :-1 s=READY pgs=631 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abdfb18ba6867250 server_cookie=a4cedf18fff8fe78 in_seq=0 out_seq=0 2024-05-09T00:11:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.981+0000 7f5ba1ffb700 10 client.32035 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5ba1ffb700 1 -- 172.21.15.12:0/3424546615 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5556af024ec0 con 0x5556af0f74a0 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5ba1ffb700 10 client.32035 handle_command_reply: tid=0 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 client.32035 shutdown 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3424546615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dc540 msgr2=0x5556af0e49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dc540 0x5556af0e49d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb00cbab0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3424546615 shutdown_connections 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5556af0f74a0 0x5556af14e7e0 crc :-1 s=READY pgs=631 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5556af0dce60 0x5556af0e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556af03a210 0x5556af0ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5ba1ffb700 0 client.32035 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 --2- 172.21.15.12:0/3424546615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556af0dc540 0x5556af0e49d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3424546615 >> 172.21.15.12:0/3424546615 conn(0x5556af0d8b40 msgr2=0x5556af0de750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3424546615 shutdown_connections 2024-05-09T00:11:54.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 1 -- 172.21.15.12:0/3424546615 wait complete. 2024-05-09T00:11:54.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.983+0000 7f5bcca75b80 20 client.32035 trim_cache size 0 max 16384 2024-05-09T00:11:54.985 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:11:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "scrub active (210 inodes in the stack)", 2024-05-09T00:11:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-09T00:11:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "6a6fc884-720e-478b-adb0-095720207c64": { 2024-05-09T00:11:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-09T00:11:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "6a6fc884-720e-478b-adb0-095720207c64", 2024-05-09T00:11:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-09T00:11:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:11:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:11:54.986 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:11:54.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 -- 172.21.15.12:0/2570255377 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ba40a1310 msgr2=0x7f5ba40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ba40a1310 0x7f5ba40a37d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x5556af0d8640 tx=0x7f5bac009190 comp rx=0 tx=0).stop 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 -- 172.21.15.12:0/2570255377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bbc0ffdd0 msgr2=0x7f5bbc1979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bbc0ffdd0 0x7f5bbc1979f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00059d0 tx=0x7f5bb0004730 comp rx=0 tx=0).stop 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 -- 172.21.15.12:0/2570255377 shutdown_connections 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f5ba40a1310 0x7f5ba40a37d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bbc105ef0 0x7f5bbc19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bbc0ff410 0x7f5bbc1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 --2- 172.21.15.12:0/2570255377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bbc0ffdd0 0x7f5bbc1979f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 -- 172.21.15.12:0/2570255377 >> 172.21.15.12:0/2570255377 conn(0x7f5bbc0763a0 msgr2=0x7f5bbc104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 -- 172.21.15.12:0/2570255377 shutdown_connections 2024-05-09T00:11:54.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:11:54.984+0000 7f5bc08bb700 1 -- 172.21.15.12:0/2570255377 wait complete. 2024-05-09T00:11:54.998 INFO:tasks.cephfs.filesystem:scrub status for tag:6a6fc884-720e-478b-adb0-095720207c64 - {'path': '/', 'tag': '6a6fc884-720e-478b-adb0-095720207c64', 'options': 'recursive,force'} 2024-05-09T00:11:55.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:54 smithi012 ceph-mon[42503]: scrub queued for path: / 2024-05-09T00:11:55.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:54 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:55.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:11:55.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:54 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:11:55.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:54 smithi204 ceph-mon[47373]: scrub queued for path: / 2024-05-09T00:11:55.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:54 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:55.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:11:55.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:54 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:11:55.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:54 smithi160 ceph-mon[46152]: scrub queued for path: / 2024-05-09T00:11:55.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:54 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:55.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:11:55.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:54 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:11:55.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 388477 27.59 MB/sec execute 590 sec latency 334.880 ms 2024-05-09T00:11:56.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:55 smithi012 ceph-mon[42503]: scrub summary: active paths [/] 2024-05-09T00:11:56.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:55 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:56.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:55 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:56.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:55 smithi012 ceph-mon[42503]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:56.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:55 smithi012 ceph-mon[42503]: pgmap v822: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 546 op/s 2024-05-09T00:11:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:55 smithi204 ceph-mon[47373]: scrub summary: active paths [/] 2024-05-09T00:11:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:55 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:55 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:55 smithi204 ceph-mon[47373]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:55 smithi204 ceph-mon[47373]: pgmap v822: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 546 op/s 2024-05-09T00:11:56.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:55 smithi160 ceph-mon[46152]: scrub summary: active paths [/] 2024-05-09T00:11:56.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:55 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:56.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:55 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:56.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:55 smithi160 ceph-mon[46152]: scrub summary: idle+waiting paths [/] 2024-05-09T00:11:56.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:55 smithi160 ceph-mon[46152]: pgmap v822: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 546 op/s 2024-05-09T00:11:56.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 388843 27.57 MB/sec execute 591 sec latency 182.103 ms 2024-05-09T00:11:57.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 389328 27.56 MB/sec execute 592 sec latency 84.998 ms 2024-05-09T00:11:58.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:57 smithi012 ceph-mon[42503]: pgmap v823: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 593 op/s 2024-05-09T00:11:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:57 smithi204 ceph-mon[47373]: pgmap v823: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 593 op/s 2024-05-09T00:11:58.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:57 smithi160 ceph-mon[46152]: pgmap v823: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 593 op/s 2024-05-09T00:11:58.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 389919 27.56 MB/sec execute 593 sec latency 101.826 ms 2024-05-09T00:11:59.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 390477 27.58 MB/sec execute 594 sec latency 138.119 ms 2024-05-09T00:12:00.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:11:59 smithi204 ceph-mon[47373]: pgmap v824: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 632 op/s 2024-05-09T00:12:00.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:11:59 smithi160 ceph-mon[46152]: pgmap v824: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 632 op/s 2024-05-09T00:12:00.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 390677 27.55 MB/sec execute 595 sec latency 380.544 ms 2024-05-09T00:12:00.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:11:59 smithi012 ceph-mon[42503]: pgmap v824: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 632 op/s 2024-05-09T00:12:01.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 390745 27.51 MB/sec execute 596 sec latency 885.307 ms 2024-05-09T00:12:02.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:01 smithi012 ceph-mon[42503]: pgmap v825: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.2 MiB/s wr, 571 op/s 2024-05-09T00:12:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:01 smithi204 ceph-mon[47373]: pgmap v825: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.2 MiB/s wr, 571 op/s 2024-05-09T00:12:02.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:01 smithi160 ceph-mon[46152]: pgmap v825: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.2 MiB/s wr, 571 op/s 2024-05-09T00:12:02.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 390746 27.46 MB/sec execute 597 sec latency 1885.417 ms 2024-05-09T00:12:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.253+0000 7fe12b8b9700 1 -- 172.21.15.12:0/3277005561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe124105f20 msgr2=0x7fe124106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.254+0000 7fe12b8b9700 1 --2- 172.21.15.12:0/3277005561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe124105f20 0x7fe124106300 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fe120009990 tx=0x7fe120009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.256+0000 7fe12b8b9700 1 -- 172.21.15.12:0/3277005561 shutdown_connections 2024-05-09T00:12:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.256+0000 7fe12b8b9700 1 --2- 172.21.15.12:0/3277005561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe124100c40 0x7fe12410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.256+0000 7fe12b8b9700 1 --2- 172.21.15.12:0/3277005561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe124105f20 0x7fe124106300 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.256+0000 7fe12b8b9700 1 --2- 172.21.15.12:0/3277005561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe124100280 0x7fe124100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.256+0000 7fe12b8b9700 1 -- 172.21.15.12:0/3277005561 >> 172.21.15.12:0/3277005561 conn(0x7fe124076bc0 msgr2=0x7fe124076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.257+0000 7fe12b8b9700 1 -- 172.21.15.12:0/3277005561 shutdown_connections 2024-05-09T00:12:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.257+0000 7fe12b8b9700 1 -- 172.21.15.12:0/3277005561 wait complete. 2024-05-09T00:12:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.258+0000 7fe12b8b9700 1 Processor -- start 2024-05-09T00:12:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe12b8b9700 1 -- start start 2024-05-09T00:12:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe12b8b9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe124100280 0x7fe124101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe12b8b9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe124100c40 0x7fe124102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe12b8b9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe124105f20 0x7fe124102590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe12b8b9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1241a56c0 con 0x7fe124105f20 2024-05-09T00:12:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe12b8b9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1241a5830 con 0x7fe124100280 2024-05-09T00:12:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe12b8b9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1241a59f0 con 0x7fe124100c40 2024-05-09T00:12:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe129e56700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe124105f20 0x7fe124102590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe129e56700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe124105f20 0x7fe124102590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46044/0 (socket says 172.21.15.12:46044) 2024-05-09T00:12:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe129e56700 1 -- 172.21.15.12:0/4271162718 learned_addr learned my addr 172.21.15.12:0/4271162718 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.261+0000 7fe128e54700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe124100c40 0x7fe124102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe128e54700 1 -- 172.21.15.12:0/4271162718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe124100280 msgr2=0x7fe124101af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:12:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe128e54700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe124100280 0x7fe124101af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe128e54700 1 -- 172.21.15.12:0/4271162718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe124105f20 msgr2=0x7fe124102590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe128e54700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe124105f20 0x7fe124102590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe128e54700 1 -- 172.21.15.12:0/4271162718 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1200095f0 con 0x7fe124100c40 2024-05-09T00:12:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe129e56700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe124105f20 0x7fe124102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe128e54700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe124100c40 0x7fe124102030 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e9f0 tx=0x7fe11400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe11a7fc700 1 -- 172.21.15.12:0/4271162718 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11400eee0 con 0x7fe124100c40 2024-05-09T00:12:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe11a7fc700 1 -- 172.21.15.12:0/4271162718 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe11400cbd0 con 0x7fe124100c40 2024-05-09T00:12:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe11a7fc700 1 -- 172.21.15.12:0/4271162718 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114018680 con 0x7fe124100c40 2024-05-09T00:12:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.262+0000 7fe12b8b9700 1 -- 172.21.15.12:0/4271162718 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe124197170 con 0x7fe124100c40 2024-05-09T00:12:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.263+0000 7fe12b8b9700 1 -- 172.21.15.12:0/4271162718 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe124197630 con 0x7fe124100c40 2024-05-09T00:12:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.266+0000 7fe11a7fc700 1 -- 172.21.15.12:0/4271162718 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe1140187e0 con 0x7fe124100c40 2024-05-09T00:12:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.267+0000 7fe11a7fc700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1100a1310 0x7fe1100a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.267+0000 7fe11a7fc700 1 -- 172.21.15.12:0/4271162718 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe114014070 con 0x7fe124100c40 2024-05-09T00:12:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.267+0000 7fe129655700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1100a1310 0x7fe1100a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.268+0000 7fe129655700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1100a1310 0x7fe1100a37d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe1200098e0 tx=0x7fe120000bc0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.268+0000 7fe137a73b80 1 Processor -- start 2024-05-09T00:12:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.268+0000 7fe137a73b80 1 -- start start 2024-05-09T00:12:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.268+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a08a950 0x55b11a08ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.268+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08b270 0x55b11a084f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.268+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a085480 0x55b11a0858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe137a73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b119fdc0f0 con 0x55b11a085480 2024-05-09T00:12:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe137a73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b119fd9ab0 con 0x55b11a08a950 2024-05-09T00:12:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe137a73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b119fd53a0 con 0x55b11a08b270 2024-05-09T00:12:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe129655700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a08a950 0x55b11a08ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe128e54700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08b270 0x55b11a084f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe129655700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a08a950 0x55b11a08ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:40486/0 (socket says 172.21.15.12:40486) 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe129655700 1 -- 172.21.15.12:0/2460432379 learned_addr learned my addr 172.21.15.12:0/2460432379 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe129e56700 1 --2- 172.21.15.12:0/2460432379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a085480 0x55b11a0858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe129655700 1 -- 172.21.15.12:0/2460432379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08b270 msgr2=0x55b11a084f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe129655700 1 --2- 172.21.15.12:0/2460432379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08b270 0x55b11a084f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe129655700 1 -- 172.21.15.12:0/2460432379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a085480 msgr2=0x55b11a0858a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe129655700 1 --2- 172.21.15.12:0/2460432379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a085480 0x55b11a0858a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe129655700 1 -- 172.21.15.12:0/2460432379 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b119e56350 con 0x55b11a08a950 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe128e54700 1 --2- 172.21.15.12:0/2460432379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08b270 0x55b11a084f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.269+0000 7fe129655700 1 --2- 172.21.15.12:0/2460432379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a08a950 0x55b11a08ad30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe12000bab0 tx=0x7fe12001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3493dc44dcb33fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.270+0000 7fe10f7fe700 1 -- 172.21.15.12:0/2460432379 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120029070 con 0x55b11a08a950 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.270+0000 7fe10f7fe700 1 -- 172.21.15.12:0/2460432379 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe12000bdc0 con 0x55b11a08a950 2024-05-09T00:12:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.270+0000 7fe10f7fe700 1 -- 172.21.15.12:0/2460432379 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12001d070 con 0x55b11a08a950 2024-05-09T00:12:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.271+0000 7fe137a73b80 1 -- 172.21.15.12:0/2460432379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a08a950 msgr2=0x55b11a08ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.271+0000 7fe137a73b80 1 --2- 172.21.15.12:0/2460432379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a08a950 0x55b11a08ad30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe12000bab0 tx=0x7fe12001d000 comp rx=0 tx=0).stop 2024-05-09T00:12:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.271+0000 7fe137a73b80 1 -- 172.21.15.12:0/2460432379 shutdown_connections 2024-05-09T00:12:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.271+0000 7fe137a73b80 1 --2- 172.21.15.12:0/2460432379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a085480 0x55b11a0858a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.271+0000 7fe137a73b80 1 --2- 172.21.15.12:0/2460432379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a08a950 0x55b11a08ad30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.271+0000 7fe137a73b80 1 --2- 172.21.15.12:0/2460432379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08b270 0x55b11a084f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.271+0000 7fe137a73b80 1 -- 172.21.15.12:0/2460432379 >> 172.21.15.12:0/2460432379 conn(0x55b119fe31b0 msgr2=0x55b11a07f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.271+0000 7fe137a73b80 1 -- 172.21.15.12:0/2460432379 shutdown_connections 2024-05-09T00:12:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.271+0000 7fe137a73b80 1 -- 172.21.15.12:0/2460432379 wait complete. 2024-05-09T00:12:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.272+0000 7fe137a73b80 1 -- start start 2024-05-09T00:12:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.272+0000 7fe137a73b80 1 Processor -- start 2024-05-09T00:12:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.272+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 0x55b11a08e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.272+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08b270 0x55b11a08ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.272+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 0x55b11a09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.272+0000 7fe137a73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b119f71500 con 0x55b11a08b270 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.272+0000 7fe137a73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b119f711a0 con 0x55b11a085480 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.272+0000 7fe137a73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b119e58640 con 0x55b11a08f190 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe137a73b80 10 client.0 _ll_register_callbacks cb 0x55b119f7aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe128e54700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08b270 0x55b11a08ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe128e54700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08b270 0x55b11a08ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46068/0 (socket says 172.21.15.12:46068) 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe128e54700 1 -- 172.21.15.12:0/4069293824 learned_addr learned my addr 172.21.15.12:0/4069293824 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe129e56700 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 0x55b11a09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe129e56700 1 -- 172.21.15.12:0/4069293824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 msgr2=0x55b11a08e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe129e56700 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 0x55b11a08e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe129e56700 1 -- 172.21.15.12:0/4069293824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08b270 msgr2=0x55b11a08ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe129e56700 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08b270 0x55b11a08ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe129e56700 1 -- 172.21.15.12:0/4069293824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12410d530 con 0x55b11a08f190 2024-05-09T00:12:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe129e56700 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 0x55b11a09ac90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009c10 tx=0x7fe11c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5096934958e68902 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.273+0000 7fe10cff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:12:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.274+0000 7fe10cff9700 1 -- 172.21.15.12:0/4069293824 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c003a00 con 0x55b11a08f190 2024-05-09T00:12:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.274+0000 7fe10cff9700 1 -- 172.21.15.12:0/4069293824 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe11c003b60 con 0x55b11a08f190 2024-05-09T00:12:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.274+0000 7fe137a73b80 1 -- 172.21.15.12:0/4069293824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b119f2e210 con 0x55b11a08f190 2024-05-09T00:12:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.274+0000 7fe10cff9700 1 -- 172.21.15.12:0/4069293824 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe11c003d20 con 0x55b11a08f190 2024-05-09T00:12:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.274+0000 7fe137a73b80 10 client.32047 fetch_fsmap learned FSMap version 86 2024-05-09T00:12:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.274+0000 7fe137a73b80 1 -- 172.21.15.12:0/4069293824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55b119e02b00 con 0x55b11a08f190 2024-05-09T00:12:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.274+0000 7fe10cff9700 1 -- 172.21.15.12:0/4069293824 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b119f2e210 con 0x55b11a08f190 2024-05-09T00:12:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.274+0000 7fe10cff9700 1 -- 172.21.15.12:0/4069293824 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe11c01e4e0 con 0x55b11a08f190 2024-05-09T00:12:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.274+0000 7fe137a73b80 10 client.32047 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:12:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.274+0000 7fe137a73b80 10 client.32047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:12:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.275+0000 7fe137a73b80 20 client.32047 populate_metadata read hostname 'smithi012' 2024-05-09T00:12:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.275+0000 7fe137a73b80 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b11a09f220 0x55b11a0a1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.275+0000 7fe137a73b80 4 client.32047 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.275+0000 7fe137a73b80 1 -- 172.21.15.12:0/4069293824 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b11a0547b0 con 0x55b11a09f220 2024-05-09T00:12:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.275+0000 7fe129655700 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b11a09f220 0x55b11a0a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.276+0000 7fe129655700 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b11a09f220 0x55b11a0a1640 crc :-1 s=READY pgs=632 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d321cfb155486a server_cookie=cb85413584e69b37 in_seq=0 out_seq=0 2024-05-09T00:12:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.276+0000 7fe10cff9700 10 client.32047 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.280+0000 7fe10cff9700 1 -- 172.21.15.12:0/4069293824 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b11a0547b0 con 0x55b11a09f220 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.280+0000 7fe10cff9700 10 client.32047 handle_command_reply: tid=0 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.280+0000 7fe137a73b80 1 client.32047 shutdown 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 1 -- 172.21.15.12:0/4069293824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 msgr2=0x55b11a09ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 0x55b11a09ac90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009c10 tx=0x7fe11c009510 comp rx=0 tx=0).stop 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 1 -- 172.21.15.12:0/4069293824 shutdown_connections 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 0x55b11a09ac90 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b11a09f220 0x55b11a0a1640 crc :-1 s=READY pgs=632 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 0x55b11a08e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 1 --2- 172.21.15.12:0/4069293824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08b270 0x55b11a08ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe10cff9700 0 client.32047 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 1 -- 172.21.15.12:0/4069293824 >> 172.21.15.12:0/4069293824 conn(0x55b119fe31b0 msgr2=0x55b11a083f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 1 -- 172.21.15.12:0/4069293824 shutdown_connections 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 1 -- 172.21.15.12:0/4069293824 wait complete. 2024-05-09T00:12:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.281+0000 7fe137a73b80 20 client.32047 trim_cache size 0 max 16384 2024-05-09T00:12:03.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.304+0000 7fe137a73b80 1 Processor -- start 2024-05-09T00:12:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.304+0000 7fe137a73b80 1 -- start start 2024-05-09T00:12:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.304+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 0x55b11a07f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.304+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08b270 0x55b11a095be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.304+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 0x55b11a096120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.304+0000 7fe137a73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11a08fa10 con 0x55b11a08b270 2024-05-09T00:12:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.304+0000 7fe137a73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11a09c810 con 0x55b11a085480 2024-05-09T00:12:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.305+0000 7fe137a73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11a077a40 con 0x55b11a08f190 2024-05-09T00:12:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.305+0000 7fe129655700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 0x55b11a07f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.305+0000 7fe129655700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 0x55b11a07f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:40506/0 (socket says 172.21.15.12:40506) 2024-05-09T00:12:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.305+0000 7fe129655700 1 -- 172.21.15.12:0/1408102890 learned_addr learned my addr 172.21.15.12:0/1408102890 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.305+0000 7fe129e56700 1 --2- 172.21.15.12:0/1408102890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 0x55b11a096120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.305+0000 7fe129655700 1 -- 172.21.15.12:0/1408102890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 msgr2=0x55b11a096120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.305+0000 7fe129655700 1 --2- 172.21.15.12:0/1408102890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 0x55b11a096120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.305+0000 7fe129655700 1 -- 172.21.15.12:0/1408102890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08b270 msgr2=0x55b11a095be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:12:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.305+0000 7fe129655700 1 --2- 172.21.15.12:0/1408102890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08b270 0x55b11a095be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.305+0000 7fe129655700 1 -- 172.21.15.12:0/1408102890 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b119f9b500 con 0x55b11a085480 2024-05-09T00:12:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.306+0000 7fe129655700 1 --2- 172.21.15.12:0/1408102890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 0x55b11a07f910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe120005330 tx=0x7fe120004c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d59c3048af1251 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.306+0000 7fe10ffff700 1 -- 172.21.15.12:0/1408102890 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120032070 con 0x55b11a085480 2024-05-09T00:12:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.306+0000 7fe10ffff700 1 -- 172.21.15.12:0/1408102890 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe120033040 con 0x55b11a085480 2024-05-09T00:12:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.306+0000 7fe10ffff700 1 -- 172.21.15.12:0/1408102890 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120036a90 con 0x55b11a085480 2024-05-09T00:12:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.307+0000 7fe137a73b80 1 -- 172.21.15.12:0/1408102890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 msgr2=0x55b11a07f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.307+0000 7fe137a73b80 1 --2- 172.21.15.12:0/1408102890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 0x55b11a07f910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe120005330 tx=0x7fe120004c40 comp rx=0 tx=0).stop 2024-05-09T00:12:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.309+0000 7fe137a73b80 1 -- 172.21.15.12:0/1408102890 shutdown_connections 2024-05-09T00:12:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.309+0000 7fe137a73b80 1 --2- 172.21.15.12:0/1408102890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a08f190 0x55b11a096120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.309+0000 7fe137a73b80 1 --2- 172.21.15.12:0/1408102890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a085480 0x55b11a07f910 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.309+0000 7fe137a73b80 1 --2- 172.21.15.12:0/1408102890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08b270 0x55b11a095be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.309+0000 7fe137a73b80 1 -- 172.21.15.12:0/1408102890 >> 172.21.15.12:0/1408102890 conn(0x55b119fe31b0 msgr2=0x55b11a0a5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:03.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.311+0000 7fe137a73b80 1 -- 172.21.15.12:0/1408102890 shutdown_connections 2024-05-09T00:12:03.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.311+0000 7fe137a73b80 1 -- 172.21.15.12:0/1408102890 wait complete. 2024-05-09T00:12:03.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.312+0000 7fe137a73b80 1 -- start start 2024-05-09T00:12:03.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.312+0000 7fe137a73b80 1 Processor -- start 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.313+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a085480 0x55b11a0960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.313+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08f190 0x55b11a0f0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.313+0000 7fe137a73b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a0f0da0 0x55b11a093bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.313+0000 7fe137a73b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11a0f13c0 con 0x55b11a08f190 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.313+0000 7fe137a73b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11a0f1530 con 0x55b11a0f0da0 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.313+0000 7fe137a73b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11a094100 con 0x55b11a085480 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.313+0000 7fe137a73b80 10 client.0 _ll_register_callbacks cb 0x55b119f7aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129655700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a085480 0x55b11a0960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129655700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a085480 0x55b11a0960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:37238/0 (socket says 172.21.15.12:37238) 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129655700 1 -- 172.21.15.12:0/3222918072 learned_addr learned my addr 172.21.15.12:0/3222918072 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe128e54700 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08f190 0x55b11a0f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129e56700 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a0f0da0 0x55b11a093bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129655700 1 -- 172.21.15.12:0/3222918072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a0f0da0 msgr2=0x55b11a093bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129655700 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a0f0da0 0x55b11a093bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129655700 1 -- 172.21.15.12:0/3222918072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08f190 msgr2=0x55b11a0f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129655700 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08f190 0x55b11a0f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129655700 1 -- 172.21.15.12:0/3222918072 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b119e56350 con 0x55b11a085480 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129e56700 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a0f0da0 0x55b11a093bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe129655700 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a085480 0x55b11a0960a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe120009ce0 tx=0x7fe120005410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eae8b6eafcf82ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe10cff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.314+0000 7fe10cff9700 1 -- 172.21.15.12:0/3222918072 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12001d070 con 0x55b11a085480 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe10cff9700 1 -- 172.21.15.12:0/3222918072 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe1200056b0 con 0x55b11a085480 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe137a73b80 1 -- 172.21.15.12:0/3222918072 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe12000bdc0 con 0x55b11a085480 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe10cff9700 1 -- 172.21.15.12:0/3222918072 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe120032040 con 0x55b11a085480 2024-05-09T00:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe137a73b80 10 client.32053 fetch_fsmap learned FSMap version 86 2024-05-09T00:12:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe137a73b80 1 -- 172.21.15.12:0/3222918072 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55b119e5ac70 con 0x55b11a085480 2024-05-09T00:12:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe10cff9700 1 -- 172.21.15.12:0/3222918072 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe12000bdc0 con 0x55b11a085480 2024-05-09T00:12:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe10cff9700 1 -- 172.21.15.12:0/3222918072 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe12002e540 con 0x55b11a085480 2024-05-09T00:12:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe137a73b80 10 client.32053 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T00:12:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe137a73b80 10 client.32053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T00:12:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe137a73b80 20 client.32053 populate_metadata read hostname 'smithi012' 2024-05-09T00:12:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe137a73b80 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b11a106f10 0x55b11a109300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe137a73b80 4 client.32053 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:12:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe137a73b80 1 -- 172.21.15.12:0/3222918072 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b119fcde60 con 0x55b11a106f10 2024-05-09T00:12:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.315+0000 7fe128e54700 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b11a106f10 0x55b11a109300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.316+0000 7fe128e54700 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b11a106f10 0x55b11a109300 crc :-1 s=READY pgs=633 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39585be3ca90fa07 server_cookie=766114d765bc0e77 in_seq=0 out_seq=0 2024-05-09T00:12:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.316+0000 7fe10cff9700 10 client.32053 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:03.337 INFO:tasks.workunit.client.0.smithi012.stdout: 10 390865 27.43 MB/sec execute 598 sec latency 2507.189 ms 2024-05-09T00:12:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.787+0000 7fe10cff9700 1 -- 172.21.15.12:0/3222918072 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b119fcde60 con 0x55b11a106f10 2024-05-09T00:12:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.787+0000 7fe10cff9700 10 client.32053 handle_command_reply: tid=0 2024-05-09T00:12:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.787+0000 7fe137a73b80 1 client.32053 shutdown 2024-05-09T00:12:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-05-09T00:12:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-05-09T00:12:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-05-09T00:12:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-05-09T00:12:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.789+0000 7fe137a73b80 1 -- 172.21.15.12:0/3222918072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a085480 msgr2=0x55b11a0960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe137a73b80 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a085480 0x55b11a0960a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe120009ce0 tx=0x7fe120005410 comp rx=0 tx=0).stop 2024-05-09T00:12:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe137a73b80 1 -- 172.21.15.12:0/3222918072 shutdown_connections 2024-05-09T00:12:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe137a73b80 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x55b11a106f10 0x55b11a109300 crc :-1 s=READY pgs=633 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe137a73b80 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b11a0f0da0 0x55b11a093bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe137a73b80 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b11a085480 0x55b11a0960a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe137a73b80 1 --2- 172.21.15.12:0/3222918072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b11a08f190 0x55b11a0f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe137a73b80 1 -- 172.21.15.12:0/3222918072 >> 172.21.15.12:0/3222918072 conn(0x55b119fe31b0 msgr2=0x55b11a088380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe10cff9700 0 client.32053 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe137a73b80 1 -- 172.21.15.12:0/3222918072 shutdown_connections 2024-05-09T00:12:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe137a73b80 1 -- 172.21.15.12:0/3222918072 wait complete. 2024-05-09T00:12:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.790+0000 7fe137a73b80 20 client.32053 trim_cache size 0 max 16384 2024-05-09T00:12:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.796+0000 7fe12b8b9700 1 -- 172.21.15.12:0/4271162718 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1100a1310 msgr2=0x7fe1100a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.796+0000 7fe12b8b9700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1100a1310 0x7fe1100a37d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe1200098e0 tx=0x7fe120000bc0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.797+0000 7fe12b8b9700 1 -- 172.21.15.12:0/4271162718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe124100c40 msgr2=0x7fe124102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.797+0000 7fe12b8b9700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe124100c40 0x7fe124102030 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e9f0 tx=0x7fe11400ed00 comp rx=0 tx=0).stop 2024-05-09T00:12:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.799+0000 7fe12b8b9700 1 -- 172.21.15.12:0/4271162718 shutdown_connections 2024-05-09T00:12:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.799+0000 7fe12b8b9700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fe1100a1310 0x7fe1100a37d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.799+0000 7fe12b8b9700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe124105f20 0x7fe124102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.799+0000 7fe12b8b9700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe124100280 0x7fe124101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.799+0000 7fe12b8b9700 1 --2- 172.21.15.12:0/4271162718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe124100c40 0x7fe124102030 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.799+0000 7fe12b8b9700 1 -- 172.21.15.12:0/4271162718 >> 172.21.15.12:0/4271162718 conn(0x7fe124076bc0 msgr2=0x7fe1240692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.806+0000 7fe12b8b9700 1 -- 172.21.15.12:0/4271162718 shutdown_connections 2024-05-09T00:12:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-05-09T00:12:03.806+0000 7fe12b8b9700 1 -- 172.21.15.12:0/4271162718 wait complete. 2024-05-09T00:12:03.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:03 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:12:03.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:03 smithi012 ceph-mon[42503]: osdmap e139: 12 total, 12 up, 12 in 2024-05-09T00:12:04.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:03 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:12:04.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:03 smithi204 ceph-mon[47373]: osdmap e139: 12 total, 12 up, 12 in 2024-05-09T00:12:04.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:03 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:12:04.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:03 smithi160 ceph-mon[46152]: osdmap e139: 12 total, 12 up, 12 in 2024-05-09T00:12:04.341 INFO:tasks.workunit.client.0.smithi012.stdout: 10 390935 27.39 MB/sec execute 599 sec latency 996.570 ms 2024-05-09T00:12:05.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:12:05.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:04 smithi204 ceph-mon[47373]: scrub summary: idle 2024-05-09T00:12:05.020 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:04 smithi204 ceph-mon[47373]: pgmap v827: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.7 MiB/s wr, 542 op/s 2024-05-09T00:12:05.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:12:05.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:04 smithi160 ceph-mon[46152]: scrub summary: idle 2024-05-09T00:12:05.022 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:04 smithi160 ceph-mon[46152]: pgmap v827: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.7 MiB/s wr, 542 op/s 2024-05-09T00:12:05.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:12:05.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:04 smithi012 ceph-mon[42503]: scrub summary: idle 2024-05-09T00:12:05.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:04 smithi012 ceph-mon[42503]: pgmap v827: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.7 MiB/s wr, 542 op/s 2024-05-09T00:12:05.341 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 600 sec 2024-05-09T00:12:06.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:05 smithi012 ceph-mon[42503]: pgmap v828: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.6 MiB/s wr, 516 op/s 2024-05-09T00:12:06.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:05 smithi160 ceph-mon[46152]: pgmap v828: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.6 MiB/s wr, 516 op/s 2024-05-09T00:12:06.271 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:05 smithi204 ceph-mon[47373]: pgmap v828: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.6 MiB/s wr, 516 op/s 2024-05-09T00:12:06.341 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 601 sec 2024-05-09T00:12:07.341 INFO:tasks.workunit.client.0.smithi012.stdout: 2 cleanup 602 sec 2024-05-09T00:12:07.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:07 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:12:07.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:07 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:12:07.526 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:07 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:12:07.911 INFO:tasks.workunit.client.0.smithi012.stdout: 0 cleanup 603 sec 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: Operation Count AvgLat MaxLat 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: ---------------------------------------- 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: NTCreateX 525650 4.272 4216.939 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: Close 385962 0.003 0.116 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: Rename 22281 18.227 2455.127 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: Unlink 106238 4.687 2480.763 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: Deltree 2 180.800 244.920 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: Mkdir 1 0.008 0.008 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: Qpathinfo 476727 4.056 2570.570 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: Qfileinfo 83183 0.001 0.038 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: Qfsinfo 87427 0.404 962.460 2024-05-09T00:12:07.912 INFO:tasks.workunit.client.0.smithi012.stdout: Sfileinfo 42777 0.150 83.304 2024-05-09T00:12:07.913 INFO:tasks.workunit.client.0.smithi012.stdout: Find 184266 0.732 9906.499 2024-05-09T00:12:07.913 INFO:tasks.workunit.client.0.smithi012.stdout: WriteX 260075 1.044 1229.709 2024-05-09T00:12:07.913 INFO:tasks.workunit.client.0.smithi012.stdout: ReadX 825160 0.031 840.929 2024-05-09T00:12:07.913 INFO:tasks.workunit.client.0.smithi012.stdout: LockX 1714 3.271 66.241 2024-05-09T00:12:07.913 INFO:tasks.workunit.client.0.smithi012.stdout: UnlockX 1714 2.879 49.018 2024-05-09T00:12:07.913 INFO:tasks.workunit.client.0.smithi012.stdout: Flush 36840 11.563 895.955 2024-05-09T00:12:07.913 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-05-09T00:12:07.913 INFO:tasks.workunit.client.0.smithi012.stdout:Throughput 27.3853 MB/sec 10 clients 10 procs max_latency=9906.504 ms 2024-05-09T00:12:07.913 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T00:12:07.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T00:12:08.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:08 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:12:08.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:08 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:12:08.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:08 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:12:08.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:08 smithi012 ceph-mon[42503]: pgmap v829: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 9.5 MiB/s wr, 529 op/s 2024-05-09T00:12:08.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:08 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:12:08.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:08 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:12:08.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:08 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:12:08.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:08 smithi204 ceph-mon[47373]: pgmap v829: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 9.5 MiB/s wr, 529 op/s 2024-05-09T00:12:08.521 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:08 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:12:08.521 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:08 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:12:08.521 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:08 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' 2024-05-09T00:12:08.521 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:08 smithi160 ceph-mon[46152]: pgmap v829: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 9.5 MiB/s wr, 529 op/s 2024-05-09T00:12:10.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:09 smithi012 ceph-mon[42503]: pgmap v830: 97 pgs: 16 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 52 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 477 op/s 2024-05-09T00:12:10.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:09 smithi204 ceph-mon[47373]: pgmap v830: 97 pgs: 16 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 52 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 477 op/s 2024-05-09T00:12:10.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:09 smithi160 ceph-mon[46152]: pgmap v830: 97 pgs: 16 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 52 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 477 op/s 2024-05-09T00:12:10.789 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-09T00:12:10.789 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T00:12:11.147 DEBUG:teuthology.parallel:result is None 2024-05-09T00:12:11.147 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T00:12:11.191 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T00:12:11.191 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-09T00:12:11.207 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T00:12:11.243 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-09T00:12:11.253 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-09T00:12:11.253 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T00:12:11.253 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T00:12:11.253 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-09T00:12:11.318 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-09T00:12:11.397 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-09T00:12:11.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.586+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/2800349419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e540ffda0 msgr2=0x7f9e54100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:11.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.586+0000 7f9e5b4ce700 1 --2- 172.21.15.12:0/2800349419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e540ffda0 0x7f9e54100220 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009990 tx=0x7f9e44009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.586+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/2800349419 shutdown_connections 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.586+0000 7f9e5b4ce700 1 --2- 172.21.15.12:0/2800349419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e54100760 0x7f9e5410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.586+0000 7f9e5b4ce700 1 --2- 172.21.15.12:0/2800349419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e54105e60 0x7f9e54106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.586+0000 7f9e5b4ce700 1 --2- 172.21.15.12:0/2800349419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e540ffda0 0x7f9e54100220 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.586+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/2800349419 >> 172.21.15.12:0/2800349419 conn(0x7f9e54076f10 msgr2=0x7f9e54077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.587+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/2800349419 shutdown_connections 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.587+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/2800349419 wait complete. 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.587+0000 7f9e5b4ce700 1 Processor -- start 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.587+0000 7f9e5b4ce700 1 -- start start 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.587+0000 7f9e5b4ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e540ffda0 0x7f9e541994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:11.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.587+0000 7f9e5b4ce700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e54100760 0x7f9e54199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.587+0000 7f9e5b4ce700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e54105e60 0x7f9e5419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.587+0000 7f9e5b4ce700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5419e410 con 0x7f9e540ffda0 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.587+0000 7f9e5b4ce700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5419e580 con 0x7f9e54105e60 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.587+0000 7f9e5b4ce700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5419e740 con 0x7f9e54100760 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5926a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e540ffda0 0x7f9e541994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5926a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e540ffda0 0x7f9e541994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59610/0 (socket says 172.21.15.12:59610) 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5926a700 1 -- 172.21.15.12:0/909877213 learned_addr learned my addr 172.21.15.12:0/909877213 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e58a69700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e54100760 0x7f9e54199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e59a6b700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e54105e60 0x7f9e5419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5926a700 1 -- 172.21.15.12:0/909877213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e54100760 msgr2=0x7f9e54199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5926a700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e54100760 0x7f9e54199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5926a700 1 -- 172.21.15.12:0/909877213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e54105e60 msgr2=0x7f9e5419ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5926a700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e54105e60 0x7f9e5419ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5926a700 1 -- 172.21.15.12:0/909877213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440095f0 con 0x7f9e540ffda0 2024-05-09T00:12:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e58a69700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e54100760 0x7f9e54199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5926a700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e540ffda0 0x7f9e541994f0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000e9a0 tx=0x7f9e5000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e4a7fc700 1 -- 172.21.15.12:0/909877213 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50009790 con 0x7f9e540ffda0 2024-05-09T00:12:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e4a7fc700 1 -- 172.21.15.12:0/909877213 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e50010430 con 0x7f9e540ffda0 2024-05-09T00:12:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/909877213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5419ea20 con 0x7f9e540ffda0 2024-05-09T00:12:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e4a7fc700 1 -- 172.21.15.12:0/909877213 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50018680 con 0x7f9e540ffda0 2024-05-09T00:12:11.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.588+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/909877213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e541a6260 con 0x7f9e540ffda0 2024-05-09T00:12:11.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.589+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/909877213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e54101490 con 0x7f9e540ffda0 2024-05-09T00:12:11.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.591+0000 7f9e4a7fc700 1 -- 172.21.15.12:0/909877213 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9e500105a0 con 0x7f9e540ffda0 2024-05-09T00:12:11.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.591+0000 7f9e4a7fc700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9e400a1290 0x7f9e400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:11.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.592+0000 7f9e58a69700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9e400a1290 0x7f9e400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:11.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.592+0000 7f9e4a7fc700 1 -- 172.21.15.12:0/909877213 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9e50014070 con 0x7f9e540ffda0 2024-05-09T00:12:11.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.592+0000 7f9e58a69700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9e400a1290 0x7f9e400a3750 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9e440098e0 tx=0x7f9e44005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:11.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.596+0000 7f9e4a7fc700 1 -- 172.21.15.12:0/909877213 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9e500964d0 con 0x7f9e540ffda0 2024-05-09T00:12:11.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.842+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/909877213 --> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e5410fcd0 con 0x7f9e400a1290 2024-05-09T00:12:11.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.856+0000 7f9e4a7fc700 1 -- 172.21.15.12:0/909877213 <== mgr.15252 v2:172.21.15.12:6800/2525408937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f9e5410fcd0 con 0x7f9e400a1290 2024-05-09T00:12:11.854 INFO:teuthology.orchestra.run.smithi012.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T23:46:34", "first": "2024-05-08T23:47:00", "last": "2024-05-09T00:12:00", "last_pruned": "2024-05-09T00:12:00", "created_count": 26, "pruned_count": 19, "active": true} 2024-05-09T00:12:11.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/909877213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9e400a1290 msgr2=0x7f9e400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:11.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9e400a1290 0x7f9e400a3750 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9e440098e0 tx=0x7f9e44005960 comp rx=0 tx=0).stop 2024-05-09T00:12:11.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/909877213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e540ffda0 msgr2=0x7f9e541994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:11.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e540ffda0 0x7f9e541994f0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000e9a0 tx=0x7f9e5000ecb0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/909877213 shutdown_connections 2024-05-09T00:12:11.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9e400a1290 0x7f9e400a3750 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e54105e60 0x7f9e5419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e540ffda0 0x7f9e541994f0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 --2- 172.21.15.12:0/909877213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e54100760 0x7f9e54199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:11.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/909877213 >> 172.21.15.12:0/909877213 conn(0x7f9e54076f10 msgr2=0x7f9e540682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:11.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/909877213 shutdown_connections 2024-05-09T00:12:11.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:11.859+0000 7f9e5b4ce700 1 -- 172.21.15.12:0/909877213 wait complete. 2024-05-09T00:12:11.872 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-09T00:12:12.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.013+0000 7f0bd2148700 1 -- 172.21.15.12:0/3906513014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bcc0ffda0 msgr2=0x7f0bcc100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:12.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.013+0000 7f0bd2148700 1 --2- 172.21.15.12:0/3906513014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bcc0ffda0 0x7f0bcc100220 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009990 tx=0x7f0bb4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.013+0000 7f0bd2148700 1 -- 172.21.15.12:0/3906513014 shutdown_connections 2024-05-09T00:12:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.013+0000 7f0bd2148700 1 --2- 172.21.15.12:0/3906513014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc100760 0x7f0bcc10cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.013+0000 7f0bd2148700 1 --2- 172.21.15.12:0/3906513014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bcc105e60 0x7f0bcc106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.013+0000 7f0bd2148700 1 --2- 172.21.15.12:0/3906513014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bcc0ffda0 0x7f0bcc100220 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.013+0000 7f0bd2148700 1 -- 172.21.15.12:0/3906513014 >> 172.21.15.12:0/3906513014 conn(0x7f0bcc076f10 msgr2=0x7f0bcc077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.013+0000 7f0bd2148700 1 -- 172.21.15.12:0/3906513014 shutdown_connections 2024-05-09T00:12:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.013+0000 7f0bd2148700 1 -- 172.21.15.12:0/3906513014 wait complete. 2024-05-09T00:12:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bd2148700 1 Processor -- start 2024-05-09T00:12:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bd2148700 1 -- start start 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bd2148700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0ffda0 0x7f0bcc198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bd2148700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bcc100760 0x7f0bcc198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bd2148700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bcc105e60 0x7f0bcc1a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bd2148700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc198f90 con 0x7f0bcc105e60 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bd2148700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc1a37c0 con 0x7f0bcc0ffda0 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bd2148700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc1a3900 con 0x7f0bcc100760 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bcbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bcc105e60 0x7f0bcc1a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bcbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bcc105e60 0x7f0bcc1a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59618/0 (socket says 172.21.15.12:59618) 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.014+0000 7f0bcbfff700 1 -- 172.21.15.12:0/2772190528 learned_addr learned my addr 172.21.15.12:0/2772190528 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bcaffd700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bcc100760 0x7f0bcc198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bcb7fe700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0ffda0 0x7f0bcc198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bcbfff700 1 -- 172.21.15.12:0/2772190528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bcc100760 msgr2=0x7f0bcc198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bcbfff700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bcc100760 0x7f0bcc198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bcbfff700 1 -- 172.21.15.12:0/2772190528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0ffda0 msgr2=0x7f0bcc198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bcbfff700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0ffda0 0x7f0bcc198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bcbfff700 1 -- 172.21.15.12:0/2772190528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb40095f0 con 0x7f0bcc105e60 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bcbfff700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bcc105e60 0x7f0bcc1a3280 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400b9a0 tx=0x7f0bc400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bc8ff9700 1 -- 172.21.15.12:0/2772190528 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400c9b0 con 0x7f0bcc105e60 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bc8ff9700 1 -- 172.21.15.12:0/2772190528 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc4014440 con 0x7f0bcc105e60 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bc8ff9700 1 -- 172.21.15.12:0/2772190528 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40075d0 con 0x7f0bcc105e60 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bd2148700 1 -- 172.21.15.12:0/2772190528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc1a3b60 con 0x7f0bcc105e60 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.015+0000 7f0bd2148700 1 -- 172.21.15.12:0/2772190528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc1a4020 con 0x7f0bcc105e60 2024-05-09T00:12:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.016+0000 7f0bd2148700 1 -- 172.21.15.12:0/2772190528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc1082c0 con 0x7f0bcc105e60 2024-05-09T00:12:12.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.018+0000 7f0bc8ff9700 1 -- 172.21.15.12:0/2772190528 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0bc40145b0 con 0x7f0bcc105e60 2024-05-09T00:12:12.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.018+0000 7f0bc8ff9700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bb80a1290 0x7f0bb80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:12.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.018+0000 7f0bc8ff9700 1 -- 172.21.15.12:0/2772190528 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0bc4006e40 con 0x7f0bcc105e60 2024-05-09T00:12:12.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.018+0000 7f0bcb7fe700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bb80a1290 0x7f0bb80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:12.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.019+0000 7f0bcb7fe700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bb80a1290 0x7f0bb80a3750 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc005ea0 tx=0x7f0bbc005e30 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:12.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.022+0000 7f0bc8ff9700 1 -- 172.21.15.12:0/2772190528 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0bc4095db0 con 0x7f0bcc105e60 2024-05-09T00:12:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.267+0000 7f0bd2148700 1 -- 172.21.15.12:0/2772190528 --> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0bcc068890 con 0x7f0bb80a1290 2024-05-09T00:12:12.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.282+0000 7f0bc8ff9700 1 -- 172.21.15.12:0/2772190528 <== mgr.15252 v2:172.21.15.12:6800/2525408937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f0bcc068890 con 0x7f0bb80a1290 2024-05-09T00:12:12.280 INFO:teuthology.orchestra.run.smithi012.stdout:/ 1m 6m3h 2024-05-09T00:12:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.284+0000 7f0bd2148700 1 -- 172.21.15.12:0/2772190528 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bb80a1290 msgr2=0x7f0bb80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.284+0000 7f0bd2148700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bb80a1290 0x7f0bb80a3750 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc005ea0 tx=0x7f0bbc005e30 comp rx=0 tx=0).stop 2024-05-09T00:12:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.284+0000 7f0bd2148700 1 -- 172.21.15.12:0/2772190528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bcc105e60 msgr2=0x7f0bcc1a3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.284+0000 7f0bd2148700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bcc105e60 0x7f0bcc1a3280 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400b9a0 tx=0x7f0bc400bd60 comp rx=0 tx=0).stop 2024-05-09T00:12:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.284+0000 7f0bd2148700 1 -- 172.21.15.12:0/2772190528 shutdown_connections 2024-05-09T00:12:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.284+0000 7f0bd2148700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f0bb80a1290 0x7f0bb80a3750 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.284+0000 7f0bd2148700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0bcc105e60 0x7f0bcc1a3280 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.284+0000 7f0bd2148700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0ffda0 0x7f0bcc198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:12.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.284+0000 7f0bd2148700 1 --2- 172.21.15.12:0/2772190528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bcc100760 0x7f0bcc198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:12.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.284+0000 7f0bd2148700 1 -- 172.21.15.12:0/2772190528 >> 172.21.15.12:0/2772190528 conn(0x7f0bcc076f10 msgr2=0x7f0bcc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:12.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.285+0000 7f0bd2148700 1 -- 172.21.15.12:0/2772190528 shutdown_connections 2024-05-09T00:12:12.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:12.285+0000 7f0bd2148700 1 -- 172.21.15.12:0/2772190528 wait complete. 2024-05-09T00:12:12.298 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-09T00:12:12.309 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-09T00:12:12.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:12 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:12:12.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:12 smithi012 ceph-mon[42503]: pgmap v831: 97 pgs: 12 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 67 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.0 MiB/s wr, 466 op/s 2024-05-09T00:12:12.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:12 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:12:12.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:12 smithi204 ceph-mon[47373]: pgmap v831: 97 pgs: 12 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 67 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.0 MiB/s wr, 466 op/s 2024-05-09T00:12:12.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:12 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:12:12.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:12 smithi160 ceph-mon[46152]: pgmap v831: 97 pgs: 12 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 67 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.0 MiB/s wr, 466 op/s 2024-05-09T00:12:13.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:13 smithi012 ceph-mon[42503]: from='client.22392 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:12:13.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:13 smithi012 ceph-mon[42503]: from='client.22398 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:12:13.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:13 smithi204 ceph-mon[47373]: from='client.22392 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:12:13.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:13 smithi204 ceph-mon[47373]: from='client.22398 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:12:13.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:13 smithi160 ceph-mon[46152]: from='client.22392 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:12:13.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:13 smithi160 ceph-mon[46152]: from='client.22398 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:12:14.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:14 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:12:14.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:14 smithi012 ceph-mon[42503]: pgmap v832: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.1 MiB/s wr, 396 op/s 2024-05-09T00:12:14.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:14 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:12:14.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:14 smithi204 ceph-mon[47373]: pgmap v832: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.1 MiB/s wr, 396 op/s 2024-05-09T00:12:14.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:14 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:12:14.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:14 smithi160 ceph-mon[46152]: pgmap v832: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.1 MiB/s wr, 396 op/s 2024-05-09T00:12:16.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:15 smithi012 ceph-mon[42503]: pgmap v833: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 345 op/s 2024-05-09T00:12:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:15 smithi204 ceph-mon[47373]: pgmap v833: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 345 op/s 2024-05-09T00:12:16.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:15 smithi160 ceph-mon[46152]: pgmap v833: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 345 op/s 2024-05-09T00:12:17.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:17 smithi012 ceph-mon[42503]: osdmap e140: 12 total, 12 up, 12 in 2024-05-09T00:12:17.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:17 smithi204 ceph-mon[47373]: osdmap e140: 12 total, 12 up, 12 in 2024-05-09T00:12:17.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:17 smithi160 ceph-mon[46152]: osdmap e140: 12 total, 12 up, 12 in 2024-05-09T00:12:18.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:18 smithi012 ceph-mon[42503]: pgmap v835: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 280 op/s 2024-05-09T00:12:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:18 smithi204 ceph-mon[47373]: pgmap v835: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 280 op/s 2024-05-09T00:12:18.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:18 smithi160 ceph-mon[46152]: pgmap v835: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 280 op/s 2024-05-09T00:12:20.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:20 smithi012 ceph-mon[42503]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:12:20.423 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:20 smithi012 ceph-mon[42503]: pgmap v836: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.1 MiB/s wr, 216 op/s 2024-05-09T00:12:20.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:20 smithi204 ceph-mon[47373]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:12:20.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:20 smithi204 ceph-mon[47373]: pgmap v836: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.1 MiB/s wr, 216 op/s 2024-05-09T00:12:20.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:20 smithi160 ceph-mon[46152]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:12:20.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:20 smithi160 ceph-mon[46152]: pgmap v836: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.1 MiB/s wr, 216 op/s 2024-05-09T00:12:22.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:21 smithi012 ceph-mon[42503]: pgmap v837: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 202 op/s 2024-05-09T00:12:22.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:21 smithi204 ceph-mon[47373]: pgmap v837: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 202 op/s 2024-05-09T00:12:22.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:21 smithi160 ceph-mon[46152]: pgmap v837: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 202 op/s 2024-05-09T00:12:24.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:23 smithi012 ceph-mon[42503]: pgmap v838: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 7.7 MiB/s wr, 165 op/s 2024-05-09T00:12:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:23 smithi204 ceph-mon[47373]: pgmap v838: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 7.7 MiB/s wr, 165 op/s 2024-05-09T00:12:24.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:23 smithi160 ceph-mon[46152]: pgmap v838: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 7.7 MiB/s wr, 165 op/s 2024-05-09T00:12:24.999 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T00:12:25.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 -- 172.21.15.12:0/3664040775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d48106300 msgr2=0x7f7d4810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 --2- 172.21.15.12:0/3664040775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d48106300 0x7f7d4810de80 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009990 tx=0x7f7d38009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 -- 172.21.15.12:0/3664040775 shutdown_connections 2024-05-09T00:12:25.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 --2- 172.21.15.12:0/3664040775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d48106300 0x7f7d4810de80 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 --2- 172.21.15.12:0/3664040775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d481011e0 0x7f7d481015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 --2- 172.21.15.12:0/3664040775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d48101b90 0x7f7d48105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 -- 172.21.15.12:0/3664040775 >> 172.21.15.12:0/3664040775 conn(0x7f7d480fca80 msgr2=0x7f7d480feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 -- 172.21.15.12:0/3664040775 shutdown_connections 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 -- 172.21.15.12:0/3664040775 wait complete. 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 Processor -- start 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.151+0000 7f7d4ef07700 1 -- start start 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4ef07700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d481011e0 0x7f7d480732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4ef07700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48106300 0x7f7d480737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4ef07700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d48073d20 0x7f7d481a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4ef07700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481a5c70 con 0x7f7d481011e0 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4ef07700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481a5de0 con 0x7f7d48106300 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4ef07700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481a5fa0 con 0x7f7d48073d20 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4cca3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d481011e0 0x7f7d480732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4cca3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d481011e0 0x7f7d480732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47654/0 (socket says 172.21.15.12:47654) 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4cca3700 1 -- 172.21.15.12:0/659691860 learned_addr learned my addr 172.21.15.12:0/659691860 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4d4a4700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d48073d20 0x7f7d481a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d47fff700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48106300 0x7f7d480737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4cca3700 1 -- 172.21.15.12:0/659691860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d48073d20 msgr2=0x7f7d481a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d48073d20 0x7f7d481a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4cca3700 1 -- 172.21.15.12:0/659691860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48106300 msgr2=0x7f7d480737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48106300 0x7f7d480737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4cca3700 1 -- 172.21.15.12:0/659691860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d380095f0 con 0x7f7d481011e0 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d47fff700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48106300 0x7f7d480737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d481011e0 0x7f7d480732a0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000ef10 tx=0x7f7d4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.152+0000 7f7d45ffb700 1 -- 172.21.15.12:0/659691860 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000ca70 con 0x7f7d481011e0 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.153+0000 7f7d45ffb700 1 -- 172.21.15.12:0/659691860 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d4000f040 con 0x7f7d481011e0 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.153+0000 7f7d45ffb700 1 -- 172.21.15.12:0/659691860 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40014480 con 0x7f7d481011e0 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.153+0000 7f7d4ef07700 1 -- 172.21.15.12:0/659691860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d481a6220 con 0x7f7d481011e0 2024-05-09T00:12:25.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.153+0000 7f7d4ef07700 1 -- 172.21.15.12:0/659691860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d481a66e0 con 0x7f7d481011e0 2024-05-09T00:12:25.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.156+0000 7f7d45ffb700 1 -- 172.21.15.12:0/659691860 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7d40007500 con 0x7f7d481011e0 2024-05-09T00:12:25.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.156+0000 7f7d45ffb700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d340a1290 0x7f7d340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.156+0000 7f7d45ffb700 1 -- 172.21.15.12:0/659691860 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7d400cd060 con 0x7f7d481011e0 2024-05-09T00:12:25.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.156+0000 7f7d47fff700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d340a1290 0x7f7d340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.157+0000 7f7d47fff700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d340a1290 0x7f7d340a3750 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d480749d0 tx=0x7f7d30009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.157+0000 7f7d5b0c1b80 1 Processor -- start 2024-05-09T00:12:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.157+0000 7f7d5b0c1b80 1 -- start start 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a7608150 con 0x5609a76b7720 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a7605b10 con 0x5609a76b6e00 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a7601400 con 0x5609a76b1890 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d47fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d47fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47656/0 (socket says 172.21.15.12:47656) 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d47fff700 1 -- 172.21.15.12:0/2097323119 learned_addr learned my addr 172.21.15.12:0/2097323119 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d4d4a4700 1 --2- 172.21.15.12:0/2097323119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/2097323119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d47fff700 1 -- 172.21.15.12:0/2097323119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 msgr2=0x5609a76b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d47fff700 1 --2- 172.21.15.12:0/2097323119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d47fff700 1 -- 172.21.15.12:0/2097323119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 msgr2=0x5609a76b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d47fff700 1 --2- 172.21.15.12:0/2097323119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76b71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d47fff700 1 -- 172.21.15.12:0/2097323119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a74823b0 con 0x5609a76b7720 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/2097323119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.158+0000 7f7d47fff700 1 --2- 172.21.15.12:0/2097323119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b1350 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f7d30010e90 tx=0x7f7d30013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26766ab4b6d5e06d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d3effd700 1 -- 172.21.15.12:0/2097323119 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30011db0 con 0x5609a76b7720 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d3effd700 1 -- 172.21.15.12:0/2097323119 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d30015070 con 0x5609a76b7720 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d3effd700 1 -- 172.21.15.12:0/2097323119 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3001c610 con 0x5609a76b7720 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2097323119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 msgr2=0x5609a76b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2097323119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b1350 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f7d30010e90 tx=0x7f7d30013000 comp rx=0 tx=0).stop 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2097323119 shutdown_connections 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2097323119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2097323119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2097323119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b1350 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2097323119 >> 172.21.15.12:0/2097323119 conn(0x5609a760f210 msgr2=0x5609a76abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:25.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2097323119 shutdown_connections 2024-05-09T00:12:25.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.159+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2097323119 wait complete. 2024-05-09T00:12:25.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.160+0000 7f7d5b0c1b80 1 -- start start 2024-05-09T00:12:25.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.160+0000 7f7d5b0c1b80 1 Processor -- start 2024-05-09T00:12:25.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76bd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b6e00 0x5609a76baa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 0x5609a76bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a759d560 con 0x5609a76b6e00 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a759d200 con 0x5609a76b7720 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a74846a0 con 0x5609a76b1890 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d5b0c1b80 10 client.0 _ll_register_callbacks cb 0x5609a75a6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d4cca3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76bd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d47fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b6e00 0x5609a76baa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d4cca3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76bd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:49152/0 (socket says 172.21.15.12:49152) 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d47fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b6e00 0x5609a76baa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47672/0 (socket says 172.21.15.12:47672) 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d4cca3700 1 -- 172.21.15.12:0/3891822441 learned_addr learned my addr 172.21.15.12:0/3891822441 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d4d4a4700 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 0x5609a76bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d4cca3700 1 -- 172.21.15.12:0/3891822441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 msgr2=0x5609a76bb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:25.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 0x5609a76bb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d4cca3700 1 -- 172.21.15.12:0/3891822441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b6e00 msgr2=0x5609a76baa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b6e00 0x5609a76baa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d4cca3700 1 -- 172.21.15.12:0/3891822441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d481a66e0 con 0x5609a76b1890 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76bd750 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7d4009a110 tx=0x7f7d4009a880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce43cf4b52b0f264 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.161+0000 7f7d27fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d27fff700 1 -- 172.21.15.12:0/3891822441 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4009a5e0 con 0x5609a76b1890 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/3891822441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609a755a270 con 0x5609a76b1890 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d27fff700 1 -- 172.21.15.12:0/3891822441 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7d4009c030 con 0x5609a76b1890 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d27fff700 1 -- 172.21.15.12:0/3891822441 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d40063020 con 0x5609a76b1890 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d5b0c1b80 10 client.32074 fetch_fsmap learned FSMap version 89 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/3891822441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x7f7d30010c20 con 0x5609a76b1890 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d27fff700 1 -- 172.21.15.12:0/3891822441 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609a755a270 con 0x5609a76b1890 2024-05-09T00:12:25.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d27fff700 1 -- 172.21.15.12:0/3891822441 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 89) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7d4007b080 con 0x5609a76b1890 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d5b0c1b80 10 client.32074 fetch_fsmap finished waiting for FSMap version 89 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d5b0c1b80 10 client.32074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d5b0c1b80 20 client.32074 populate_metadata read hostname 'smithi012' 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a76cb1c0 0x5609a76cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d5b0c1b80 4 client.32074 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/3891822441 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609a7680810 con 0x5609a76cb1c0 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.162+0000 7f7d47fff700 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a76cb1c0 0x5609a76cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.163+0000 7f7d47fff700 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a76cb1c0 0x5609a76cd5b0 crc :-1 s=READY pgs=634 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860ee36a5eaf4fb1 server_cookie=5395d3125452d72f in_seq=0 out_seq=0 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.163+0000 7f7d27fff700 10 client.32074 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.163+0000 7f7d27fff700 1 -- 172.21.15.12:0/3891822441 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609a7680810 con 0x5609a76cb1c0 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.163+0000 7f7d27fff700 10 client.32074 handle_command_reply: tid=0 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 client.32074 shutdown 2024-05-09T00:12:25.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/3891822441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 msgr2=0x5609a76bd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76bd750 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7d4009a110 tx=0x7f7d4009a880 comp rx=0 tx=0).stop 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/3891822441 shutdown_connections 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a76cb1c0 0x5609a76cd5b0 crc :-1 s=READY pgs=634 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 0x5609a76bb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76bd750 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/3891822441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b6e00 0x5609a76baa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d27fff700 0 client.32074 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/3891822441 >> 172.21.15.12:0/3891822441 conn(0x5609a760f210 msgr2=0x5609a76ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/3891822441 shutdown_connections 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/3891822441 wait complete. 2024-05-09T00:12:25.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.164+0000 7f7d5b0c1b80 20 client.32074 trim_cache size 0 max 16384 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d5b0c1b80 1 Processor -- start 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d5b0c1b80 1 -- start start 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b1890 0x5609a76b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b6e00 0x5609a76abd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 0x5609a76b9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a76a40d0 con 0x5609a76b1890 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a76cafa0 con 0x5609a76b7720 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a76ada00 con 0x5609a76b6e00 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d4cca3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b1890 0x5609a76b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d4cca3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b1890 0x5609a76b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47676/0 (socket says 172.21.15.12:47676) 2024-05-09T00:12:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.183+0000 7f7d4cca3700 1 -- 172.21.15.12:0/2032496753 learned_addr learned my addr 172.21.15.12:0/2032496753 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d47fff700 1 --2- 172.21.15.12:0/2032496753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b6e00 0x5609a76abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d4d4a4700 1 --2- 172.21.15.12:0/2032496753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 0x5609a76b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d4cca3700 1 -- 172.21.15.12:0/2032496753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b6e00 msgr2=0x5609a76abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/2032496753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b6e00 0x5609a76abd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d4cca3700 1 -- 172.21.15.12:0/2032496753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 msgr2=0x5609a76b9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/2032496753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 0x5609a76b9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d4cca3700 1 -- 172.21.15.12:0/2032496753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a742eb60 con 0x5609a76b1890 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d4d4a4700 1 --2- 172.21.15.12:0/2032496753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 0x5609a76b9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/2032496753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b1890 0x5609a76b1350 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f7d400984d0 tx=0x7f7d400965f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383bdd88e3abe195 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d3f7fe700 1 -- 172.21.15.12:0/2032496753 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40099930 con 0x5609a76b1890 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d3f7fe700 1 -- 172.21.15.12:0/2032496753 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d400a2370 con 0x5609a76b1890 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.184+0000 7f7d3f7fe700 1 -- 172.21.15.12:0/2032496753 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40012bf0 con 0x5609a76b1890 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2032496753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b1890 msgr2=0x5609a76b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2032496753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b1890 0x5609a76b1350 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f7d400984d0 tx=0x7f7d400965f0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2032496753 shutdown_connections 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2032496753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b7720 0x5609a76b9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2032496753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b1890 0x5609a76b1350 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2032496753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b6e00 0x5609a76abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2032496753 >> 172.21.15.12:0/2032496753 conn(0x5609a760f210 msgr2=0x5609a76d1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2032496753 shutdown_connections 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2032496753 wait complete. 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 -- start start 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.185+0000 7f7d5b0c1b80 1 Processor -- start 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76c2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d5b0c1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a76b3490 con 0x5609a76b7720 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a76b3600 con 0x5609a76b6e00 2024-05-09T00:12:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d5b0c1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a76b37c0 con 0x5609a76b1890 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d5b0c1b80 10 client.0 _ll_register_callbacks cb 0x5609a75a6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d4d4a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d47fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d47fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76c26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:54694/0 (socket says 172.21.15.12:54694) 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d4d4a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47692/0 (socket says 172.21.15.12:47692) 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d47fff700 1 -- 172.21.15.12:0/2477134213 learned_addr learned my addr 172.21.15.12:0/2477134213 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d4d4a4700 1 -- 172.21.15.12:0/2477134213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 msgr2=0x5609a76c2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d4d4a4700 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76c2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d4d4a4700 1 -- 172.21.15.12:0/2477134213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 msgr2=0x5609a76c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d4d4a4700 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d4d4a4700 1 -- 172.21.15.12:0/2477134213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d30010c20 con 0x5609a76b7720 2024-05-09T00:12:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d47fff700 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d4d4a4700 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b2dd0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x5609a76a6bf0 tx=0x7f7d38005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fc95645976ac5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.186+0000 7f7d27fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d27fff700 1 -- 172.21.15.12:0/2477134213 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3801d030 con 0x5609a76b7720 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d27fff700 1 -- 172.21.15.12:0/2477134213 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7d3800bbc0 con 0x5609a76b7720 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2477134213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d30015070 con 0x5609a76b7720 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d27fff700 1 -- 172.21.15.12:0/2477134213 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d3800bd80 con 0x5609a76b7720 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d5b0c1b80 10 client.22416 fetch_fsmap learned FSMap version 89 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2477134213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x5609a75c7560 con 0x5609a76b7720 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d27fff700 1 -- 172.21.15.12:0/2477134213 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d30015070 con 0x5609a76b7720 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d27fff700 1 -- 172.21.15.12:0/2477134213 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 89) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7d3800f510 con 0x5609a76b7720 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d5b0c1b80 10 client.22416 fetch_fsmap finished waiting for FSMap version 89 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d5b0c1b80 10 client.22416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d5b0c1b80 20 client.22416 populate_metadata read hostname 'smithi012' 2024-05-09T00:12:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a7722630 0x5609a7724a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d5b0c1b80 4 client.22416 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.187+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2477134213 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5609a75f9ec0 con 0x5609a7722630 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a7722630 0x5609a7724a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d4cca3700 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a7722630 0x5609a7724a50 crc :-1 s=READY pgs=635 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ef8b330f796a30e server_cookie=d6a181bc301662a3 in_seq=0 out_seq=0 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d27fff700 10 client.22416 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d27fff700 1 -- 172.21.15.12:0/2477134213 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5609a75f9ec0 con 0x5609a7722630 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d27fff700 10 client.22416 handle_command_reply: tid=0 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d5b0c1b80 1 client.22416 shutdown 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2477134213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 msgr2=0x5609a76b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b2dd0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x5609a76a6bf0 tx=0x7f7d38005790 comp rx=0 tx=0).stop 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2477134213 shutdown_connections 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a76b7720 0x5609a76b2dd0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609a76b1890 0x5609a76c2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5609a7722630 0x5609a7724a50 crc :-1 s=READY pgs=635 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d5b0c1b80 1 --2- 172.21.15.12:0/2477134213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5609a76b6e00 0x5609a76c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d27fff700 0 client.22416 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:25.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.188+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2477134213 >> 172.21.15.12:0/2477134213 conn(0x5609a760f210 msgr2=0x5609a76b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:25.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2477134213 shutdown_connections 2024-05-09T00:12:25.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d5b0c1b80 1 -- 172.21.15.12:0/2477134213 wait complete. 2024-05-09T00:12:25.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d5b0c1b80 20 client.22416 trim_cache size 0 max 16384 2024-05-09T00:12:25.193 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:12:25.193 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-09T00:12:25.193 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-09T00:12:25.193 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:12:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 -- 172.21.15.12:0/659691860 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d340a1290 msgr2=0x7f7d340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d340a1290 0x7f7d340a3750 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d480749d0 tx=0x7f7d30009260 comp rx=0 tx=0).stop 2024-05-09T00:12:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 -- 172.21.15.12:0/659691860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d481011e0 msgr2=0x7f7d480732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d481011e0 0x7f7d480732a0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000ef10 tx=0x7f7d4000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 -- 172.21.15.12:0/659691860 shutdown_connections 2024-05-09T00:12:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f7d340a1290 0x7f7d340a3750 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d48073d20 0x7f7d481a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d481011e0 0x7f7d480732a0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 --2- 172.21.15.12:0/659691860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d48106300 0x7f7d480737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 -- 172.21.15.12:0/659691860 >> 172.21.15.12:0/659691860 conn(0x7f7d480fca80 msgr2=0x7f7d480fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 -- 172.21.15.12:0/659691860 shutdown_connections 2024-05-09T00:12:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.189+0000 7f7d4ef07700 1 -- 172.21.15.12:0/659691860 wait complete. 2024-05-09T00:12:25.201 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:12:25.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:12:25.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.446+0000 7f14c5e5b700 1 -- 172.21.15.12:0/3160234281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c010f5c0 msgr2=0x7f14c010f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.446+0000 7f14c5e5b700 1 --2- 172.21.15.12:0/3160234281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c010f5c0 0x7f14c010f9a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009990 tx=0x7f14b4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.451+0000 7f14c5e5b700 1 -- 172.21.15.12:0/3160234281 shutdown_connections 2024-05-09T00:12:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.451+0000 7f14c5e5b700 1 --2- 172.21.15.12:0/3160234281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14c0107fa0 0x7f14c0108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.451+0000 7f14c5e5b700 1 --2- 172.21.15.12:0/3160234281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c010f5c0 0x7f14c010f9a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.451+0000 7f14c5e5b700 1 --2- 172.21.15.12:0/3160234281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14c010d2b0 0x7f14c010d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.451+0000 7f14c5e5b700 1 -- 172.21.15.12:0/3160234281 >> 172.21.15.12:0/3160234281 conn(0x7f14c006bf20 msgr2=0x7f14c006c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.454+0000 7f14c5e5b700 1 -- 172.21.15.12:0/3160234281 shutdown_connections 2024-05-09T00:12:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.457+0000 7f14c5e5b700 1 -- 172.21.15.12:0/3160234281 wait complete. 2024-05-09T00:12:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.461+0000 7f14c5e5b700 1 Processor -- start 2024-05-09T00:12:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.461+0000 7f14c5e5b700 1 -- start start 2024-05-09T00:12:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.461+0000 7f14c5e5b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14c0107fa0 0x7f14c01a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.461+0000 7f14c5e5b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14c010d2b0 0x7f14c01a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.462+0000 7f14c5e5b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c010f5c0 0x7f14c01aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.462+0000 7f14c5e5b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c01ab120 con 0x7f14c0107fa0 2024-05-09T00:12:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.462+0000 7f14c5e5b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c01ab290 con 0x7f14c010d2b0 2024-05-09T00:12:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.462+0000 7f14c5e5b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c01ab450 con 0x7f14c010f5c0 2024-05-09T00:12:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.464+0000 7f14bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14c0107fa0 0x7f14c01a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.464+0000 7f14bf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14c0107fa0 0x7f14c01a6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47706/0 (socket says 172.21.15.12:47706) 2024-05-09T00:12:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.464+0000 7f14bf7fe700 1 -- 172.21.15.12:0/2580126444 learned_addr learned my addr 172.21.15.12:0/2580126444 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.466+0000 7f14beffd700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14c010d2b0 0x7f14c01a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.472+0000 7f14bffff700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c010f5c0 0x7f14c01aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.473+0000 7f14bf7fe700 1 -- 172.21.15.12:0/2580126444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c010f5c0 msgr2=0x7f14c01aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.473+0000 7f14bf7fe700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c010f5c0 0x7f14c01aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.473+0000 7f14bf7fe700 1 -- 172.21.15.12:0/2580126444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14c010d2b0 msgr2=0x7f14c01a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.473+0000 7f14bf7fe700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14c010d2b0 0x7f14c01a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.473+0000 7f14bf7fe700 1 -- 172.21.15.12:0/2580126444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b40095f0 con 0x7f14c0107fa0 2024-05-09T00:12:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.477+0000 7f14bf7fe700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14c0107fa0 0x7f14c01a6200 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f14b40098e0 tx=0x7f14b400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.482+0000 7f14bcff9700 1 -- 172.21.15.12:0/2580126444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b401d070 con 0x7f14c0107fa0 2024-05-09T00:12:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.482+0000 7f14bcff9700 1 -- 172.21.15.12:0/2580126444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14b4005030 con 0x7f14c0107fa0 2024-05-09T00:12:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.482+0000 7f14bcff9700 1 -- 172.21.15.12:0/2580126444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4004b30 con 0x7f14c0107fa0 2024-05-09T00:12:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.489+0000 7f14c5e5b700 1 -- 172.21.15.12:0/2580126444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c01ab6d0 con 0x7f14c0107fa0 2024-05-09T00:12:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.489+0000 7f14c5e5b700 1 -- 172.21.15.12:0/2580126444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c01ab920 con 0x7f14c0107fa0 2024-05-09T00:12:25.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.495+0000 7f14c5e5b700 1 -- 172.21.15.12:0/2580126444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c006d280 con 0x7f14c0107fa0 2024-05-09T00:12:25.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.503+0000 7f14bcff9700 1 -- 172.21.15.12:0/2580126444 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f14b40051a0 con 0x7f14c0107fa0 2024-05-09T00:12:25.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.503+0000 7f14bcff9700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14a80a13d0 0x7f14a80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:25.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.504+0000 7f14bcff9700 1 -- 172.21.15.12:0/2580126444 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f14b40cdeb0 con 0x7f14c0107fa0 2024-05-09T00:12:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.510+0000 7f14bcff9700 1 -- 172.21.15.12:0/2580126444 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f14b4097340 con 0x7f14c0107fa0 2024-05-09T00:12:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.512+0000 7f14beffd700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14a80a13d0 0x7f14a80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.513+0000 7f14beffd700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14a80a13d0 0x7f14a80a3890 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f14c01a7750 tx=0x7f14b8009040 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:25.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.864+0000 7f14c5e5b700 1 -- 172.21.15.12:0/2580126444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14c01a7530 con 0x7f14c0107fa0 2024-05-09T00:12:25.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.865+0000 7f14bcff9700 1 -- 172.21.15.12:0/2580126444 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f14b409a960 con 0x7f14c0107fa0 2024-05-09T00:12:25.862 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:12:25.863 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":21435,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/676326208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":676326208},{"type":"v1","addr":"172.21.15.160:6833","nonce":676326208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":89,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:12:19.040142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_31894":{"gid":31894,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3656191694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3656191694},{"type":"v1","addr":"172.21.15.204:6833","nonce":3656191694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:12:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 -- 172.21.15.12:0/2580126444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14a80a13d0 msgr2=0x7f14a80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14a80a13d0 0x7f14a80a3890 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f14c01a7750 tx=0x7f14b8009040 comp rx=0 tx=0).stop 2024-05-09T00:12:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 -- 172.21.15.12:0/2580126444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14c0107fa0 msgr2=0x7f14c01a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14c0107fa0 0x7f14c01a6200 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f14b40098e0 tx=0x7f14b400f690 comp rx=0 tx=0).stop 2024-05-09T00:12:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 -- 172.21.15.12:0/2580126444 shutdown_connections 2024-05-09T00:12:25.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f14a80a13d0 0x7f14a80a3890 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c010f5c0 0x7f14c01aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14c0107fa0 0x7f14c01a6200 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 --2- 172.21.15.12:0/2580126444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14c010d2b0 0x7f14c01a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 -- 172.21.15.12:0/2580126444 >> 172.21.15.12:0/2580126444 conn(0x7f14c006bf20 msgr2=0x7f14c010aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 -- 172.21.15.12:0/2580126444 shutdown_connections 2024-05-09T00:12:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:25.867+0000 7f14c5e5b700 1 -- 172.21.15.12:0/2580126444 wait complete. 2024-05-09T00:12:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 89 2024-05-09T00:12:25.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.022+0000 7f75f5747700 1 -- 172.21.15.12:0/4103082676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75f01062a0 msgr2=0x7f75f010de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.022+0000 7f75f5747700 1 --2- 172.21.15.12:0/4103082676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75f01062a0 0x7f75f010de20 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f75e800b0d0 tx=0x7f75e800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.022+0000 7f75f5747700 1 -- 172.21.15.12:0/4103082676 shutdown_connections 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.022+0000 7f75f5747700 1 --2- 172.21.15.12:0/4103082676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75f01062a0 0x7f75f010de20 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.022+0000 7f75f5747700 1 --2- 172.21.15.12:0/4103082676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75f00697a0 0x7f75f0069b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.022+0000 7f75f5747700 1 --2- 172.21.15.12:0/4103082676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75f006a150 0x7f75f0105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.022+0000 7f75f5747700 1 -- 172.21.15.12:0/4103082676 >> 172.21.15.12:0/4103082676 conn(0x7f75f0076f40 msgr2=0x7f75f0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.022+0000 7f75f5747700 1 -- 172.21.15.12:0/4103082676 shutdown_connections 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.022+0000 7f75f5747700 1 -- 172.21.15.12:0/4103082676 wait complete. 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.023+0000 7f75f5747700 1 Processor -- start 2024-05-09T00:12:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.023+0000 7f75f5747700 1 -- start start 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.023+0000 7f75f5747700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75f00697a0 0x7f75f0197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.023+0000 7f75f5747700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75f01062a0 0x7f75f0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.023+0000 7f75f5747700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75f019be00 0x7f75f019c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.023+0000 7f75f5747700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f019c880 con 0x7f75f01062a0 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.023+0000 7f75f5747700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f019c9f0 con 0x7f75f00697a0 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.023+0000 7f75f5747700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f019cbb0 con 0x7f75f019be00 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.023+0000 7f75ee7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75f01062a0 0x7f75f0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.023+0000 7f75ee7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75f01062a0 0x7f75f0197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47724/0 (socket says 172.21.15.12:47724) 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75ee7fc700 1 -- 172.21.15.12:0/2804207751 learned_addr learned my addr 172.21.15.12:0/2804207751 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75eeffd700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75f00697a0 0x7f75f0197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75f019be00 0x7f75f019c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75ee7fc700 1 -- 172.21.15.12:0/2804207751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75f019be00 msgr2=0x7f75f019c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75ee7fc700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75f019be00 0x7f75f019c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75ee7fc700 1 -- 172.21.15.12:0/2804207751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75f00697a0 msgr2=0x7f75f0197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75ee7fc700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75f00697a0 0x7f75f0197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75ee7fc700 1 -- 172.21.15.12:0/2804207751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8009d00 con 0x7f75f01062a0 2024-05-09T00:12:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75f019be00 0x7f75f019c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75eeffd700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75f00697a0 0x7f75f0197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75ee7fc700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75f01062a0 0x7f75f0197a60 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f75e400d980 tx=0x7f75e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75d7fff700 1 -- 172.21.15.12:0/2804207751 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4009930 con 0x7f75f01062a0 2024-05-09T00:12:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75f5747700 1 -- 172.21.15.12:0/2804207751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f01a4110 con 0x7f75f01062a0 2024-05-09T00:12:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75d7fff700 1 -- 172.21.15.12:0/2804207751 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e4010460 con 0x7f75f01062a0 2024-05-09T00:12:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75d7fff700 1 -- 172.21.15.12:0/2804207751 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e400f620 con 0x7f75f01062a0 2024-05-09T00:12:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.024+0000 7f75f5747700 1 -- 172.21.15.12:0/2804207751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f01a45f0 con 0x7f75f01062a0 2024-05-09T00:12:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.027+0000 7f75d7fff700 1 -- 172.21.15.12:0/2804207751 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f75e40108d0 con 0x7f75f01062a0 2024-05-09T00:12:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.027+0000 7f75d7fff700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f75d80a1290 0x7f75d80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.027+0000 7f75eeffd700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f75d80a1290 0x7f75d80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.027+0000 7f75d7fff700 1 -- 172.21.15.12:0/2804207751 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f75e40cc720 con 0x7f75f01062a0 2024-05-09T00:12:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.028+0000 7f75eeffd700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f75d80a1290 0x7f75d80a3750 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f75e0005950 tx=0x7f75e000b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.028+0000 7f7601901b80 1 Processor -- start 2024-05-09T00:12:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.028+0000 7f7601901b80 1 -- start start 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.028+0000 7f7601901b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.028+0000 7f7601901b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 0x55c3935d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.028+0000 7f7601901b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.028+0000 7f7601901b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39352e150 con 0x55c3935d7890 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.028+0000 7f7601901b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39352bb10 con 0x55c3935dce00 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.028+0000 7f7601901b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c393527400 con 0x55c3935dd720 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47728/0 (socket says 172.21.15.12:47728) 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75ef7fe700 1 -- 172.21.15.12:0/1544380676 learned_addr learned my addr 172.21.15.12:0/1544380676 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75ee7fc700 1 --2- 172.21.15.12:0/1544380676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 0x55c3935d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75eeffd700 1 --2- 172.21.15.12:0/1544380676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75ef7fe700 1 -- 172.21.15.12:0/1544380676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 msgr2=0x55c3935d7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/1544380676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 0x55c3935d7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75ef7fe700 1 -- 172.21.15.12:0/1544380676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 msgr2=0x55c3935dd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/1544380676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935dd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75ef7fe700 1 -- 172.21.15.12:0/1544380676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3933a83b0 con 0x55c3935d7890 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/1544380676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7cd0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f75e800cda0 tx=0x7f75e800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159870183f6d4863 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75d57fa700 1 -- 172.21.15.12:0/1544380676 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800e050 con 0x55c3935d7890 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75d57fa700 1 -- 172.21.15.12:0/1544380676 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e8007d70 con 0x55c3935d7890 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.029+0000 7f75d57fa700 1 -- 172.21.15.12:0/1544380676 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e80078a0 con 0x55c3935d7890 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.030+0000 7f7601901b80 1 -- 172.21.15.12:0/1544380676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 msgr2=0x55c3935d7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.030+0000 7f7601901b80 1 --2- 172.21.15.12:0/1544380676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7cd0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f75e800cda0 tx=0x7f75e800cdd0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.030+0000 7f7601901b80 1 -- 172.21.15.12:0/1544380676 shutdown_connections 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.030+0000 7f7601901b80 1 --2- 172.21.15.12:0/1544380676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7cd0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.030+0000 7f7601901b80 1 --2- 172.21.15.12:0/1544380676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935dd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.030+0000 7f7601901b80 1 --2- 172.21.15.12:0/1544380676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 0x55c3935d7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.030+0000 7f7601901b80 1 -- 172.21.15.12:0/1544380676 >> 172.21.15.12:0/1544380676 conn(0x55c393535210 msgr2=0x55c3935d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.030+0000 7f7601901b80 1 -- 172.21.15.12:0/1544380676 shutdown_connections 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.030+0000 7f7601901b80 1 -- 172.21.15.12:0/1544380676 wait complete. 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f7601901b80 1 -- start start 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f7601901b80 1 Processor -- start 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f7601901b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935d7890 0x55c3935e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f7601901b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935e1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f7601901b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935dd720 0x55c3935e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f7601901b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3934c3560 con 0x55c3935dd720 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f7601901b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3934c3200 con 0x55c3935dce00 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f7601901b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3933aa6a0 con 0x55c3935d7890 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f7601901b80 10 client.0 _ll_register_callbacks cb 0x55c3934ccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f75ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935dd720 0x55c3935e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f75ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935dd720 0x55c3935e15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47730/0 (socket says 172.21.15.12:47730) 2024-05-09T00:12:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f75ef7fe700 1 -- 172.21.15.12:0/3800085232 learned_addr learned my addr 172.21.15.12:0/3800085232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f75ee7fc700 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f75eeffd700 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935d7890 0x55c3935e0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f75ef7fe700 1 -- 172.21.15.12:0/3800085232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935d7890 msgr2=0x55c3935e0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935d7890 0x55c3935e0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f75ef7fe700 1 -- 172.21.15.12:0/3800085232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 msgr2=0x55c3935e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.031+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935e1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f75ef7fe700 1 -- 172.21.15.12:0/3800085232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f010e660 con 0x55c3935dd720 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f75eeffd700 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935d7890 0x55c3935e0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935dd720 0x55c3935e15a0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f75e800cd70 tx=0x7f75e8004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdd5e195d259864 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f75d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f75d2ff5700 1 -- 172.21.15.12:0/3800085232 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8007790 con 0x55c3935dd720 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f75d2ff5700 1 -- 172.21.15.12:0/3800085232 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f75e80078f0 con 0x55c3935dd720 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f7601901b80 1 -- 172.21.15.12:0/3800085232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c393480270 con 0x55c3935dd720 2024-05-09T00:12:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f75d2ff5700 1 -- 172.21.15.12:0/3800085232 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e802f460 con 0x55c3935dd720 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f7601901b80 10 client.22434 fetch_fsmap learned FSMap version 89 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f7601901b80 1 -- 172.21.15.12:0/3800085232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55c393354b60 con 0x55c3935dd720 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f75d2ff5700 1 -- 172.21.15.12:0/3800085232 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c393480270 con 0x55c3935dd720 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f75d2ff5700 1 -- 172.21.15.12:0/3800085232 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 89) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f75e800b100 con 0x55c3935dd720 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.032+0000 7f7601901b80 10 client.22434 fetch_fsmap finished waiting for FSMap version 89 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.033+0000 7f7601901b80 10 client.22434 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.033+0000 7f7601901b80 20 client.22434 populate_metadata read hostname 'smithi012' 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.033+0000 7f7601901b80 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c3935f0d60 0x55c3935f3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.033+0000 7f7601901b80 4 client.22434 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.033+0000 7f7601901b80 1 -- 172.21.15.12:0/3800085232 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3935a6810 con 0x55c3935f0d60 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.033+0000 7f75eeffd700 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c3935f0d60 0x55c3935f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.033+0000 7f75d2ff5700 10 client.22434 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.033+0000 7f75eeffd700 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c3935f0d60 0x55c3935f3150 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b15596f80f4c31 server_cookie=52f7f983e05e07dc in_seq=0 out_seq=0 2024-05-09T00:12:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f75d2ff5700 1 -- 172.21.15.12:0/3800085232 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3935a6810 con 0x55c3935f0d60 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f75d2ff5700 10 client.22434 handle_command_reply: tid=0 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 client.22434 shutdown 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 -- 172.21.15.12:0/3800085232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935dd720 msgr2=0x55c3935e15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935dd720 0x55c3935e15a0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f75e800cd70 tx=0x7f75e8004ea0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 -- 172.21.15.12:0/3800085232 shutdown_connections 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935dd720 0x55c3935e15a0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c3935f0d60 0x55c3935f3150 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935d7890 0x55c3935e0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f75d2ff5700 0 client.22434 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 --2- 172.21.15.12:0/3800085232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 -- 172.21.15.12:0/3800085232 >> 172.21.15.12:0/3800085232 conn(0x55c393535210 msgr2=0x55c3935d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 -- 172.21.15.12:0/3800085232 shutdown_connections 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 1 -- 172.21.15.12:0/3800085232 wait complete. 2024-05-09T00:12:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.034+0000 7f7601901b80 20 client.22434 trim_cache size 0 max 16384 2024-05-09T00:12:26.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.048+0000 7f7601901b80 1 Processor -- start 2024-05-09T00:12:26.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.048+0000 7f7601901b80 1 -- start start 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.048+0000 7f7601901b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.048+0000 7f7601901b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.048+0000 7f7601901b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 0x55c3935d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.048+0000 7f7601901b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c393480270 con 0x55c3935d7890 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.048+0000 7f7601901b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3935e3690 con 0x55c3935dce00 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.048+0000 7f7601901b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3935f8320 con 0x55c3935dd720 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47734/0 (socket says 172.21.15.12:47734) 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75eeffd700 1 -- 172.21.15.12:0/2130118218 learned_addr learned my addr 172.21.15.12:0/2130118218 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75ee7fc700 1 --2- 172.21.15.12:0/2130118218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935ecbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/2130118218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 0x55c3935d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75eeffd700 1 -- 172.21.15.12:0/2130118218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 msgr2=0x55c3935d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75eeffd700 1 --2- 172.21.15.12:0/2130118218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 0x55c3935d1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75eeffd700 1 -- 172.21.15.12:0/2130118218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 msgr2=0x55c3935ecbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75eeffd700 1 --2- 172.21.15.12:0/2130118218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935ecbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75eeffd700 1 -- 172.21.15.12:0/2130118218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3934ed560 con 0x55c3935d7890 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/2130118218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 0x55c3935d1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75eeffd700 1 --2- 172.21.15.12:0/2130118218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7350 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f75e0010c00 tx=0x7f75e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aaa15883e30768e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75d5ffb700 1 -- 172.21.15.12:0/2130118218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0026070 con 0x55c3935d7890 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75d5ffb700 1 -- 172.21.15.12:0/2130118218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e0015070 con 0x55c3935d7890 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.049+0000 7f75d5ffb700 1 -- 172.21.15.12:0/2130118218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0004b20 con 0x55c3935d7890 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 -- 172.21.15.12:0/2130118218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 msgr2=0x55c3935d7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 --2- 172.21.15.12:0/2130118218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7350 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f75e0010c00 tx=0x7f75e0013000 comp rx=0 tx=0).stop 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 -- 172.21.15.12:0/2130118218 shutdown_connections 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 --2- 172.21.15.12:0/2130118218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dd720 0x55c3935d1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 --2- 172.21.15.12:0/2130118218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c3935d7350 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 --2- 172.21.15.12:0/2130118218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dce00 0x55c3935ecbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 -- 172.21.15.12:0/2130118218 >> 172.21.15.12:0/2130118218 conn(0x55c393535210 msgr2=0x55c3935ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 -- 172.21.15.12:0/2130118218 shutdown_connections 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 -- 172.21.15.12:0/2130118218 wait complete. 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 -- start start 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.050+0000 7f7601901b80 1 Processor -- start 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f7601901b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c39363f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f7601901b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dce00 0x55c39363fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f7601901b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dd720 0x55c393640010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f7601901b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39364a610 con 0x55c3935d7890 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f7601901b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39364a750 con 0x55c3935dd720 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f7601901b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39364a910 con 0x55c3935dce00 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f7601901b80 10 client.0 _ll_register_callbacks cb 0x55c3934ccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c39363f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75eeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c39363f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47736/0 (socket says 172.21.15.12:47736) 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75eeffd700 1 -- 172.21.15.12:0/1963206763 learned_addr learned my addr 172.21.15.12:0/1963206763 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75ee7fc700 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dce00 0x55c39363fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75ef7fe700 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dd720 0x55c393640010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75eeffd700 1 -- 172.21.15.12:0/1963206763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dce00 msgr2=0x55c39363fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75eeffd700 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dce00 0x55c39363fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75eeffd700 1 -- 172.21.15.12:0/1963206763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dd720 msgr2=0x55c393640010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75eeffd700 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dd720 0x55c393640010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75eeffd700 1 -- 172.21.15.12:0/1963206763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e0011920 con 0x55c3935d7890 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75eeffd700 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c39363f590 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f75e000bd20 tx=0x7f75e0010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f43fc1733b3027 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75d2ff5700 1 -- 172.21.15.12:0/1963206763 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0026050 con 0x55c3935d7890 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f75d2ff5700 1 -- 172.21.15.12:0/1963206763 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f75e0017070 con 0x55c3935d7890 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.051+0000 7f7601901b80 1 -- 172.21.15.12:0/1963206763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75e8007d70 con 0x55c3935d7890 2024-05-09T00:12:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f75d2ff5700 1 -- 172.21.15.12:0/1963206763 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e001f730 con 0x55c3935d7890 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f7601901b80 10 client.22440 fetch_fsmap learned FSMap version 89 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f7601901b80 1 -- 172.21.15.12:0/1963206763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55c3933accd0 con 0x55c3935d7890 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f75d2ff5700 1 -- 172.21.15.12:0/1963206763 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e8007d70 con 0x55c3935d7890 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f75d2ff5700 1 -- 172.21.15.12:0/1963206763 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 89) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f75e002faf0 con 0x55c3935d7890 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f7601901b80 10 client.22440 fetch_fsmap finished waiting for FSMap version 89 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f7601901b80 10 client.22440 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f7601901b80 20 client.22440 populate_metadata read hostname 'smithi012' 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f7601901b80 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c39364c040 0x55c39364e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f7601901b80 4 client.22440 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f7601901b80 1 -- 172.21.15.12:0/1963206763 --> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c39351fec0 con 0x55c39364c040 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.052+0000 7f75ee7fc700 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c39364c040 0x55c39364e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f75d2ff5700 10 client.22440 ms_handle_connect on v2:172.21.15.204:6838/2873951026 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f75ee7fc700 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c39364c040 0x55c39364e460 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11a029d50db46a4d server_cookie=d53944b1f0b8e00 in_seq=0 out_seq=0 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f75d2ff5700 1 -- 172.21.15.12:0/1963206763 <== mds.2 v2:172.21.15.204:6838/2873951026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c39351fec0 con 0x55c39364c040 2024-05-09T00:12:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f75d2ff5700 10 client.22440 handle_command_reply: tid=0 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 client.22440 shutdown 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 -- 172.21.15.12:0/1963206763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 msgr2=0x55c39363f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c39363f590 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f75e000bd20 tx=0x7f75e0010e80 comp rx=0 tx=0).stop 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 -- 172.21.15.12:0/1963206763 shutdown_connections 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3935dd720 0x55c393640010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3935d7890 0x55c39363f590 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.204:6838/2873951026,v1:172.21.15.204:6839/2873951026] conn(0x55c39364c040 0x55c39364e460 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 --2- 172.21.15.12:0/1963206763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3935dce00 0x55c39363fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f75d2ff5700 0 client.22440 ms_handle_reset on v2:172.21.15.204:6838/2873951026 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 -- 172.21.15.12:0/1963206763 >> 172.21.15.12:0/1963206763 conn(0x55c393535210 msgr2=0x55c3935dacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 -- 172.21.15.12:0/1963206763 shutdown_connections 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 1 -- 172.21.15.12:0/1963206763 wait complete. 2024-05-09T00:12:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.053+0000 7f7601901b80 20 client.22440 trim_cache size 0 max 16384 2024-05-09T00:12:26.058 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:12:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 -- 172.21.15.12:0/2804207751 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f75d80a1290 msgr2=0x7f75d80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f75d80a1290 0x7f75d80a3750 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f75e0005950 tx=0x7f75e000b410 comp rx=0 tx=0).stop 2024-05-09T00:12:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 -- 172.21.15.12:0/2804207751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75f01062a0 msgr2=0x7f75f0197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75f01062a0 0x7f75f0197a60 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f75e400d980 tx=0x7f75e400dc90 comp rx=0 tx=0).stop 2024-05-09T00:12:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 -- 172.21.15.12:0/2804207751 shutdown_connections 2024-05-09T00:12:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f75d80a1290 0x7f75d80a3750 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75f019be00 0x7f75f019c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75f00697a0 0x7f75f0197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 --2- 172.21.15.12:0/2804207751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f75f01062a0 0x7f75f0197a60 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 -- 172.21.15.12:0/2804207751 >> 172.21.15.12:0/2804207751 conn(0x7f75f0076f40 msgr2=0x7f75f00ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 -- 172.21.15.12:0/2804207751 shutdown_connections 2024-05-09T00:12:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.054+0000 7f75f5747700 1 -- 172.21.15.12:0/2804207751 wait complete. 2024-05-09T00:12:26.065 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T00:12:26.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:25 smithi012 ceph-mon[42503]: pgmap v839: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.0 MiB/s wr, 165 op/s 2024-05-09T00:12:26.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:25 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2580126444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:26.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.208+0000 7f4fa16ac700 1 -- 172.21.15.12:0/206944699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f9c0ffe00 msgr2=0x7f4f9c100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.208+0000 7f4fa16ac700 1 --2- 172.21.15.12:0/206944699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f9c0ffe00 0x7f4f9c100280 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009990 tx=0x7f4f90009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.208+0000 7f4fa16ac700 1 -- 172.21.15.12:0/206944699 shutdown_connections 2024-05-09T00:12:26.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.208+0000 7f4fa16ac700 1 --2- 172.21.15.12:0/206944699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f9c1007c0 0x7f4f9c10cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.208+0000 7f4fa16ac700 1 --2- 172.21.15.12:0/206944699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f9c105ec0 0x7f4f9c1062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.208+0000 7f4fa16ac700 1 --2- 172.21.15.12:0/206944699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f9c0ffe00 0x7f4f9c100280 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.208+0000 7f4fa16ac700 1 -- 172.21.15.12:0/206944699 >> 172.21.15.12:0/206944699 conn(0x7f4f9c076f10 msgr2=0x7f4f9c077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.208+0000 7f4fa16ac700 1 -- 172.21.15.12:0/206944699 shutdown_connections 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.208+0000 7f4fa16ac700 1 -- 172.21.15.12:0/206944699 wait complete. 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.208+0000 7f4fa16ac700 1 Processor -- start 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4fa16ac700 1 -- start start 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4fa16ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f9c0ffe00 0x7f4f9c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4fa16ac700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f9c1007c0 0x7f4f9c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4fa16ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f9c105ec0 0x7f4f9c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4fa16ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c19c220 con 0x7f4f9c105ec0 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4fa16ac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c19c390 con 0x7f4f9c0ffe00 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4fa16ac700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c19c550 con 0x7f4f9c1007c0 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4f9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f9c105ec0 0x7f4f9c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4f9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f9c105ec0 0x7f4f9c19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47746/0 (socket says 172.21.15.12:47746) 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4f9b7fe700 1 -- 172.21.15.12:0/3259425826 learned_addr learned my addr 172.21.15.12:0/3259425826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f9c1007c0 0x7f4f9c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4f9affd700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f9c0ffe00 0x7f4f9c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4f9b7fe700 1 -- 172.21.15.12:0/3259425826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f9c1007c0 msgr2=0x7f4f9c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4f9b7fe700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f9c1007c0 0x7f4f9c197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4f9b7fe700 1 -- 172.21.15.12:0/3259425826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f9c0ffe00 msgr2=0x7f4f9c197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4f9b7fe700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f9c0ffe00 0x7f4f9c197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.209+0000 7f4f9b7fe700 1 -- 172.21.15.12:0/3259425826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f900095f0 con 0x7f4f9c105ec0 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.210+0000 7f4f9b7fe700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f9c105ec0 0x7f4f9c19bbe0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4f94007ee0 tx=0x7f4f9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.210+0000 7f4f83fff700 1 -- 172.21.15.12:0/3259425826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94017070 con 0x7f4f9c105ec0 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.210+0000 7f4fa16ac700 1 -- 172.21.15.12:0/3259425826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c19c830 con 0x7f4f9c105ec0 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.210+0000 7f4f83fff700 1 -- 172.21.15.12:0/3259425826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f9400f040 con 0x7f4f9c105ec0 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.210+0000 7f4f83fff700 1 -- 172.21.15.12:0/3259425826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94005580 con 0x7f4f9c105ec0 2024-05-09T00:12:26.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.210+0000 7f4fa16ac700 1 -- 172.21.15.12:0/3259425826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c1a4070 con 0x7f4f9c105ec0 2024-05-09T00:12:26.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.212+0000 7f4f83fff700 1 -- 172.21.15.12:0/3259425826 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4f9400db20 con 0x7f4f9c105ec0 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.213+0000 7f4f83fff700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4f840a1290 0x7f4f840a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.213+0000 7f4f9affd700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4f840a1290 0x7f4f840a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.213+0000 7f4f83fff700 1 -- 172.21.15.12:0/3259425826 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4f940cc4d0 con 0x7f4f9c105ec0 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.213+0000 7f4f9affd700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4f840a1290 0x7f4f840a3750 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009c60 tx=0x7f4f8c009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4fad866b80 1 Processor -- start 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4fad866b80 1 -- start start 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 0x55eb787761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb78770350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb78770cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4fad866b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb786c7150 con 0x55eb78776720 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4fad866b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb786c4b10 con 0x55eb78775e00 2024-05-09T00:12:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4fad866b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb786c0400 con 0x55eb78770890 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4f9a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb78770350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4f9a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb78770350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47756/0 (socket says 172.21.15.12:47756) 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/2058913100 learned_addr learned my addr 172.21.15.12:0/2058913100 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4f9affd700 1 --2- 172.21.15.12:0/2058913100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 0x55eb787761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4f9b7fe700 1 --2- 172.21.15.12:0/2058913100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb78770cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/2058913100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 msgr2=0x55eb78770cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/2058913100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb78770cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/2058913100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 msgr2=0x55eb787761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/2058913100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 0x55eb787761e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.214+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/2058913100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb785413b0 con 0x55eb78776720 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.215+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/2058913100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb78770350 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009990 tx=0x7f4f90005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625a0205fd0f54dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.215+0000 7f4f817fa700 1 -- 172.21.15.12:0/2058913100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9001d070 con 0x55eb78776720 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.215+0000 7f4f817fa700 1 -- 172.21.15.12:0/2058913100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f9001e040 con 0x55eb78776720 2024-05-09T00:12:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.215+0000 7f4f817fa700 1 -- 172.21.15.12:0/2058913100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f900048c0 con 0x55eb78776720 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 -- 172.21.15.12:0/2058913100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 msgr2=0x55eb78770350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 --2- 172.21.15.12:0/2058913100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb78770350 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009990 tx=0x7f4f90005bc0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 -- 172.21.15.12:0/2058913100 shutdown_connections 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 --2- 172.21.15.12:0/2058913100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb78770cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 --2- 172.21.15.12:0/2058913100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 0x55eb787761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 --2- 172.21.15.12:0/2058913100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb78770350 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 -- 172.21.15.12:0/2058913100 >> 172.21.15.12:0/2058913100 conn(0x55eb786ce210 msgr2=0x55eb7876adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 -- 172.21.15.12:0/2058913100 shutdown_connections 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 -- 172.21.15.12:0/2058913100 wait complete. 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 -- start start 2024-05-09T00:12:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 Processor -- start 2024-05-09T00:12:26.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78770890 0x55eb78779a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78775e00 0x55eb78779f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.216+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4fad866b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7865c560 con 0x55eb78776720 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4fad866b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7865c200 con 0x55eb78770890 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4fad866b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb785436a0 con 0x55eb78775e00 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4fad866b80 10 client.0 _ll_register_callbacks cb 0x55eb78665b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9a7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78775e00 0x55eb78779f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9a7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78775e00 0x55eb78779f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:49278/0 (socket says 172.21.15.12:49278) 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47764/0 (socket says 172.21.15.12:47764) 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/4263648658 learned_addr learned my addr 172.21.15.12:0/4263648658 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9affd700 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78770890 0x55eb78779a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/4263648658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78770890 msgr2=0x55eb78779a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78770890 0x55eb78779a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/4263648658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 msgr2=0x55eb7877a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/4263648658 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c10d550 con 0x55eb78775e00 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9b7fe700 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78775e00 0x55eb78779f60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009350 tx=0x7f4f9000ba00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc8c864a697b57a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f7eff5700 1 -- 172.21.15.12:0/4263648658 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9001d030 con 0x55eb78775e00 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.217+0000 7f4f7eff5700 1 -- 172.21.15.12:0/4263648658 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4f90004910 con 0x55eb78775e00 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4fad866b80 1 -- 172.21.15.12:0/4263648658 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb78619270 con 0x55eb78775e00 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4f7eff5700 1 -- 172.21.15.12:0/4263648658 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f90004ad0 con 0x55eb78775e00 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4fad866b80 10 client.32113 fetch_fsmap learned FSMap version 89 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4fad866b80 1 -- 172.21.15.12:0/4263648658 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55eb784edb60 con 0x55eb78775e00 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4f7eff5700 1 -- 172.21.15.12:0/4263648658 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eb78619270 con 0x55eb78775e00 2024-05-09T00:12:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4f7eff5700 1 -- 172.21.15.12:0/4263648658 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 89) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4f9000fc20 con 0x55eb78775e00 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4fad866b80 10 client.32113 fetch_fsmap finished waiting for FSMap version 89 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4fad866b80 10 client.32113 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4fad866b80 20 client.32113 populate_metadata read hostname 'smithi012' 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4fad866b80 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55eb78789e20 0x55eb7878c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4fad866b80 4 client.32113 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.218+0000 7f4fad866b80 1 -- 172.21.15.12:0/4263648658 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb7873f810 con 0x55eb78789e20 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.219+0000 7f4f9affd700 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55eb78789e20 0x55eb7878c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.220+0000 7f4f9affd700 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55eb78789e20 0x55eb7878c240 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a62803da818ca449 server_cookie=8e60086acc138d84 in_seq=0 out_seq=0 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.220+0000 7f4f7eff5700 10 client.32113 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4f7eff5700 1 -- 172.21.15.12:0/4263648658 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb7873f810 con 0x55eb78789e20 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4f7eff5700 10 client.32113 handle_command_reply: tid=0 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4fad866b80 1 client.32113 shutdown 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4fad866b80 1 -- 172.21.15.12:0/4263648658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78775e00 msgr2=0x55eb78779f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4fad866b80 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78775e00 0x55eb78779f60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009350 tx=0x7f4f9000ba00 comp rx=0 tx=0).stop 2024-05-09T00:12:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4fad866b80 1 -- 172.21.15.12:0/4263648658 shutdown_connections 2024-05-09T00:12:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4fad866b80 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55eb78789e20 0x55eb7878c240 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4fad866b80 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4fad866b80 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78770890 0x55eb78779a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4fad866b80 1 --2- 172.21.15.12:0/4263648658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78775e00 0x55eb78779f60 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4f7eff5700 0 client.32113 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:12:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.221+0000 7f4fad866b80 1 -- 172.21.15.12:0/4263648658 >> 172.21.15.12:0/4263648658 conn(0x55eb786ce210 msgr2=0x55eb7876a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.222+0000 7f4fad866b80 1 -- 172.21.15.12:0/4263648658 shutdown_connections 2024-05-09T00:12:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.222+0000 7f4fad866b80 1 -- 172.21.15.12:0/4263648658 wait complete. 2024-05-09T00:12:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.222+0000 7f4fad866b80 20 client.32113 trim_cache size 0 max 16384 2024-05-09T00:12:26.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4fad866b80 1 Processor -- start 2024-05-09T00:12:26.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4fad866b80 1 -- start start 2024-05-09T00:12:26.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb7876ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78775e00 0x55eb78775260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78776720 0x55eb78781100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4fad866b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb78767dc0 con 0x55eb78775e00 2024-05-09T00:12:26.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4fad866b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb78791410 con 0x55eb78776720 2024-05-09T00:12:26.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4fad866b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7876c770 con 0x55eb78770890 2024-05-09T00:12:26.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4f9a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78775e00 0x55eb78775260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4f9affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb7876ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4f9b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78776720 0x55eb78781100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4f9a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78775e00 0x55eb78775260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47770/0 (socket says 172.21.15.12:47770) 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.239+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/473436354 learned_addr learned my addr 172.21.15.12:0/473436354 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.240+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/473436354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 msgr2=0x55eb7876ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.240+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/473436354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb7876ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.240+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/473436354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78776720 msgr2=0x55eb78781100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.240+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/473436354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78776720 0x55eb78781100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.240+0000 7f4f9a7fc700 1 -- 172.21.15.12:0/473436354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb78686560 con 0x55eb78775e00 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.240+0000 7f4f9b7fe700 1 --2- 172.21.15.12:0/473436354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78776720 0x55eb78781100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.240+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/473436354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78775e00 0x55eb78775260 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000ff80 tx=0x7f4f90005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2090896be01a6046 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.240+0000 7f4f81ffb700 1 -- 172.21.15.12:0/473436354 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9001d040 con 0x55eb78775e00 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.240+0000 7f4f81ffb700 1 -- 172.21.15.12:0/473436354 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f9002c610 con 0x55eb78775e00 2024-05-09T00:12:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.240+0000 7f4f81ffb700 1 -- 172.21.15.12:0/473436354 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90004c50 con 0x55eb78775e00 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 -- 172.21.15.12:0/473436354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78775e00 msgr2=0x55eb78775260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 --2- 172.21.15.12:0/473436354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78775e00 0x55eb78775260 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000ff80 tx=0x7f4f90005bc0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 -- 172.21.15.12:0/473436354 shutdown_connections 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 --2- 172.21.15.12:0/473436354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78776720 0x55eb78781100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 --2- 172.21.15.12:0/473436354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb7876ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 --2- 172.21.15.12:0/473436354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78775e00 0x55eb78775260 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 -- 172.21.15.12:0/473436354 >> 172.21.15.12:0/473436354 conn(0x55eb786ce210 msgr2=0x55eb78774b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 -- 172.21.15.12:0/473436354 shutdown_connections 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 -- 172.21.15.12:0/473436354 wait complete. 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 -- start start 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.241+0000 7f4fad866b80 1 Processor -- start 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb7877d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 0x55eb7877d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4fad866b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4fad866b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb787df400 con 0x55eb78776720 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4fad866b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb787df570 con 0x55eb78775e00 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4fad866b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb787df730 con 0x55eb78770890 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4fad866b80 10 client.0 _ll_register_callbacks cb 0x55eb78665b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb7877d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 0x55eb7877d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb7877d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:49302/0 (socket says 172.21.15.12:49302) 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47776/0 (socket says 172.21.15.12:47776) 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9affd700 1 -- 172.21.15.12:0/3470687177 learned_addr learned my addr 172.21.15.12:0/3470687177 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9affd700 1 -- 172.21.15.12:0/3470687177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 msgr2=0x55eb7877d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9affd700 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 0x55eb7877d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9affd700 1 -- 172.21.15.12:0/3470687177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 msgr2=0x55eb7877db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9affd700 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9affd700 1 -- 172.21.15.12:0/3470687177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c1a4070 con 0x55eb78770890 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.242+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 0x55eb7877d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4f9affd700 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb7877d0e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55eb7878cb10 tx=0x7f4f8c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810e37051445c65e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4f7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4f7eff5700 1 -- 172.21.15.12:0/3470687177 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c022040 con 0x55eb78770890 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4f7eff5700 1 -- 172.21.15.12:0/3470687177 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4f8c003e70 con 0x55eb78770890 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4fad866b80 1 -- 172.21.15.12:0/3470687177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f9001e040 con 0x55eb78770890 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4f7eff5700 1 -- 172.21.15.12:0/3470687177 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f8c02b0d0 con 0x55eb78770890 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4fad866b80 10 client.32119 fetch_fsmap learned FSMap version 89 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4fad866b80 1 -- 172.21.15.12:0/3470687177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55eb784edb60 con 0x55eb78770890 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4f7eff5700 1 -- 172.21.15.12:0/3470687177 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8c040080 con 0x55eb78770890 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4f7eff5700 1 -- 172.21.15.12:0/3470687177 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 89) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4f8c0348d0 con 0x55eb78770890 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4fad866b80 10 client.32119 fetch_fsmap finished waiting for FSMap version 89 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4fad866b80 10 client.32119 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T00:12:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4fad866b80 20 client.32119 populate_metadata read hostname 'smithi012' 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4fad866b80 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55eb787e0f50 0x55eb787e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4fad866b80 4 client.32119 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.243+0000 7f4fad866b80 1 -- 172.21.15.12:0/3470687177 --> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eb786b8ec0 con 0x55eb787e0f50 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55eb787e0f50 0x55eb787e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4f9a7fc700 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55eb787e0f50 0x55eb787e3370 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=239e1e4235312ce9 server_cookie=57a33789da9b0c62 in_seq=0 out_seq=0 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4f7eff5700 10 client.32119 ms_handle_connect on v2:172.21.15.160:6836/1904912186 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4f7eff5700 1 -- 172.21.15.12:0/3470687177 <== mds.1 v2:172.21.15.160:6836/1904912186 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eb786b8ec0 con 0x55eb787e0f50 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4f7eff5700 10 client.32119 handle_command_reply: tid=0 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4fad866b80 1 client.32119 shutdown 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4fad866b80 1 -- 172.21.15.12:0/3470687177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 msgr2=0x55eb7877d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4fad866b80 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb7877d0e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55eb7878cb10 tx=0x7f4f8c0047c0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4fad866b80 1 -- 172.21.15.12:0/3470687177 shutdown_connections 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4fad866b80 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.160:6836/1904912186,v1:172.21.15.160:6837/1904912186] conn(0x55eb787e0f50 0x55eb787e3370 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4fad866b80 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb78776720 0x55eb7877db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4f7eff5700 0 client.32119 ms_handle_reset on v2:172.21.15.160:6836/1904912186 2024-05-09T00:12:26.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4fad866b80 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb78770890 0x55eb7877d0e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4fad866b80 1 --2- 172.21.15.12:0/3470687177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb78775e00 0x55eb7877d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.244+0000 7f4fad866b80 1 -- 172.21.15.12:0/3470687177 >> 172.21.15.12:0/3470687177 conn(0x55eb786ce210 msgr2=0x55eb787753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fad866b80 1 -- 172.21.15.12:0/3470687177 shutdown_connections 2024-05-09T00:12:26.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fad866b80 1 -- 172.21.15.12:0/3470687177 wait complete. 2024-05-09T00:12:26.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fad866b80 20 client.32119 trim_cache size 0 max 16384 2024-05-09T00:12:26.250 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:12:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 -- 172.21.15.12:0/3259425826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4f840a1290 msgr2=0x7f4f840a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4f840a1290 0x7f4f840a3750 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009c60 tx=0x7f4f8c009260 comp rx=0 tx=0).stop 2024-05-09T00:12:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 -- 172.21.15.12:0/3259425826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f9c105ec0 msgr2=0x7f4f9c19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f9c105ec0 0x7f4f9c19bbe0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4f94007ee0 tx=0x7f4f9400d3b0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 -- 172.21.15.12:0/3259425826 shutdown_connections 2024-05-09T00:12:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f4f840a1290 0x7f4f840a3750 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f9c105ec0 0x7f4f9c19bbe0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f9c0ffe00 0x7f4f9c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 --2- 172.21.15.12:0/3259425826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f9c1007c0 0x7f4f9c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 -- 172.21.15.12:0/3259425826 >> 172.21.15.12:0/3259425826 conn(0x7f4f9c076f10 msgr2=0x7f4f9c069710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 -- 172.21.15.12:0/3259425826 shutdown_connections 2024-05-09T00:12:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.245+0000 7f4fa16ac700 1 -- 172.21.15.12:0/3259425826 wait complete. 2024-05-09T00:12:26.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T00:12:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:25 smithi204 ceph-mon[47373]: pgmap v839: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.0 MiB/s wr, 165 op/s 2024-05-09T00:12:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:25 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2580126444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:26.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:25 smithi160 ceph-mon[46152]: pgmap v839: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.0 MiB/s wr, 165 op/s 2024-05-09T00:12:26.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:25 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2580126444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:26.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.399+0000 7f62c0186700 1 -- 172.21.15.12:0/1344010883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b8069dd0 msgr2=0x7f62b8105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.399+0000 7f62c0186700 1 --2- 172.21.15.12:0/1344010883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b8069dd0 0x7f62b8105b70 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009990 tx=0x7f62a8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.399+0000 7f62c0186700 1 -- 172.21.15.12:0/1344010883 shutdown_connections 2024-05-09T00:12:26.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.399+0000 7f62c0186700 1 --2- 172.21.15.12:0/1344010883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b81062a0 0x7f62b810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.399+0000 7f62c0186700 1 --2- 172.21.15.12:0/1344010883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b8069420 0x7f62b8069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.399+0000 7f62c0186700 1 --2- 172.21.15.12:0/1344010883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b8069dd0 0x7f62b8105b70 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.399+0000 7f62c0186700 1 -- 172.21.15.12:0/1344010883 >> 172.21.15.12:0/1344010883 conn(0x7f62b8076ba0 msgr2=0x7f62b8076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.399+0000 7f62c0186700 1 -- 172.21.15.12:0/1344010883 shutdown_connections 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.399+0000 7f62c0186700 1 -- 172.21.15.12:0/1344010883 wait complete. 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62c0186700 1 Processor -- start 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62c0186700 1 -- start start 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62c0186700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b8069420 0x7f62b8197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62c0186700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b81062a0 0x7f62b81979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62c0186700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b819bcf0 0x7f62b819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62c0186700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b819c770 con 0x7f62b81062a0 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62c0186700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b819c8e0 con 0x7f62b819bcf0 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62c0186700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b819caa0 con 0x7f62b8069420 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62bd721700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b81062a0 0x7f62b81979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62bd721700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b81062a0 0x7f62b81979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47792/0 (socket says 172.21.15.12:47792) 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62bd721700 1 -- 172.21.15.12:0/2181908124 learned_addr learned my addr 172.21.15.12:0/2181908124 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62bdf22700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b8069420 0x7f62b8197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62be723700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b819bcf0 0x7f62b819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62bd721700 1 -- 172.21.15.12:0/2181908124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b8069420 msgr2=0x7f62b8197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62bd721700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b8069420 0x7f62b8197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62bd721700 1 -- 172.21.15.12:0/2181908124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b819bcf0 msgr2=0x7f62b819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.400+0000 7f62bd721700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b819bcf0 0x7f62b819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.401+0000 7f62bd721700 1 -- 172.21.15.12:0/2181908124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80095f0 con 0x7f62b81062a0 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.401+0000 7f62bd721700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b81062a0 0x7f62b81979a0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f62a8000c00 tx=0x7f62a8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.401+0000 7f62aeffd700 1 -- 172.21.15.12:0/2181908124 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a801d070 con 0x7f62b81062a0 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.401+0000 7f62c0186700 1 -- 172.21.15.12:0/2181908124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b81a3fa0 con 0x7f62b81062a0 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.401+0000 7f62aeffd700 1 -- 172.21.15.12:0/2181908124 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62a800fb30 con 0x7f62b81062a0 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.401+0000 7f62aeffd700 1 -- 172.21.15.12:0/2181908124 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8022bb0 con 0x7f62b81062a0 2024-05-09T00:12:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.401+0000 7f62c0186700 1 -- 172.21.15.12:0/2181908124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b81a43c0 con 0x7f62b81062a0 2024-05-09T00:12:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.404+0000 7f62aeffd700 1 -- 172.21.15.12:0/2181908124 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f62a800fca0 con 0x7f62b81062a0 2024-05-09T00:12:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.404+0000 7f62aeffd700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62a40a1290 0x7f62a40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.404+0000 7f62aeffd700 1 -- 172.21.15.12:0/2181908124 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f62a80ce140 con 0x7f62b81062a0 2024-05-09T00:12:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.404+0000 7f62bdf22700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62a40a1290 0x7f62a40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.405+0000 7f62bdf22700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62a40a1290 0x7f62a40a3750 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f62b4005950 tx=0x7f62b400b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.405+0000 7f62cc340b80 1 Processor -- start 2024-05-09T00:12:26.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.405+0000 7f62cc340b80 1 -- start start 2024-05-09T00:12:26.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.405+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749d1f0 0x55735749d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.405+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x557357497760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.405+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557357497ca0 0x557357498100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.405+0000 7f62cc340b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573573ea150 con 0x55735749db10 2024-05-09T00:12:26.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.405+0000 7f62cc340b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573573e7b10 con 0x557357497ca0 2024-05-09T00:12:26.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.405+0000 7f62cc340b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573573e3400 con 0x55735749d1f0 2024-05-09T00:12:26.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bd721700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x557357497760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bd721700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x557357497760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47794/0 (socket says 172.21.15.12:47794) 2024-05-09T00:12:26.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bd721700 1 -- 172.21.15.12:0/2640535669 learned_addr learned my addr 172.21.15.12:0/2640535669 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62be723700 1 --2- 172.21.15.12:0/2640535669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557357497ca0 0x557357498100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bdf22700 1 --2- 172.21.15.12:0/2640535669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749d1f0 0x55735749d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bd721700 1 -- 172.21.15.12:0/2640535669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749d1f0 msgr2=0x55735749d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bd721700 1 --2- 172.21.15.12:0/2640535669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749d1f0 0x55735749d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bd721700 1 -- 172.21.15.12:0/2640535669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557357497ca0 msgr2=0x557357498100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bd721700 1 --2- 172.21.15.12:0/2640535669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557357497ca0 0x557357498100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bd721700 1 -- 172.21.15.12:0/2640535669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573572643b0 con 0x55735749db10 2024-05-09T00:12:26.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bdf22700 1 --2- 172.21.15.12:0/2640535669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749d1f0 0x55735749d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:26.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62bd721700 1 --2- 172.21.15.12:0/2640535669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x557357497760 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f62a8097d00 tx=0x7f62a8022650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aff426b4cc84e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62a3fff700 1 -- 172.21.15.12:0/2640535669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8095ea0 con 0x55735749db10 2024-05-09T00:12:26.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62a3fff700 1 -- 172.21.15.12:0/2640535669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62a809b990 con 0x55735749db10 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.406+0000 7f62a3fff700 1 -- 172.21.15.12:0/2640535669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a809a780 con 0x55735749db10 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.407+0000 7f62cc340b80 1 -- 172.21.15.12:0/2640535669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 msgr2=0x557357497760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.407+0000 7f62cc340b80 1 --2- 172.21.15.12:0/2640535669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x557357497760 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f62a8097d00 tx=0x7f62a8022650 comp rx=0 tx=0).stop 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.407+0000 7f62cc340b80 1 -- 172.21.15.12:0/2640535669 shutdown_connections 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.407+0000 7f62cc340b80 1 --2- 172.21.15.12:0/2640535669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557357497ca0 0x557357498100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.407+0000 7f62cc340b80 1 --2- 172.21.15.12:0/2640535669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749d1f0 0x55735749d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.407+0000 7f62cc340b80 1 --2- 172.21.15.12:0/2640535669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x557357497760 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.407+0000 7f62cc340b80 1 -- 172.21.15.12:0/2640535669 >> 172.21.15.12:0/2640535669 conn(0x5573573f1210 msgr2=0x55735748ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.407+0000 7f62cc340b80 1 -- 172.21.15.12:0/2640535669 shutdown_connections 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.407+0000 7f62cc340b80 1 -- 172.21.15.12:0/2640535669 wait complete. 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.408+0000 7f62cc340b80 1 -- start start 2024-05-09T00:12:26.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.408+0000 7f62cc340b80 1 Processor -- start 2024-05-09T00:12:26.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.408+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557357497ca0 0x5573574a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 0x5573574a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749db10 0x5573574a1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62cc340b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735737f560 con 0x557357497ca0 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62cc340b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735737f200 con 0x55735749d1f0 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62cc340b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573572666a0 con 0x55735749db10 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62cc340b80 10 client.0 _ll_register_callbacks cb 0x557357388b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bdf22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557357497ca0 0x5573574a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bdf22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557357497ca0 0x5573574a0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47810/0 (socket says 172.21.15.12:47810) 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bdf22700 1 -- 172.21.15.12:0/1354126024 learned_addr learned my addr 172.21.15.12:0/1354126024 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bd721700 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 0x5573574a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62be723700 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749db10 0x5573574a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bdf22700 1 -- 172.21.15.12:0/1354126024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749db10 msgr2=0x5573574a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bdf22700 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749db10 0x5573574a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bdf22700 1 -- 172.21.15.12:0/1354126024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 msgr2=0x5573574a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bdf22700 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 0x5573574a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bdf22700 1 -- 172.21.15.12:0/1354126024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b81a43c0 con 0x557357497ca0 2024-05-09T00:12:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62be723700 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749db10 0x5573574a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bd721700 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 0x5573574a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62bdf22700 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557357497ca0 0x5573574a0ec0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f62b4011770 tx=0x7f62b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4efff405d01350e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-09T00:12:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62a17fa700 1 -- 172.21.15.12:0/1354126024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4004a30 con 0x557357497ca0 2024-05-09T00:12:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62a17fa700 1 -- 172.21.15.12:0/1354126024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f62b4017070 con 0x557357497ca0 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.409+0000 7f62cc340b80 1 -- 172.21.15.12:0/1354126024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55735733c270 con 0x557357497ca0 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62a17fa700 1 -- 172.21.15.12:0/1354126024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62b4004b90 con 0x557357497ca0 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62cc340b80 10 client.22470 fetch_fsmap learned FSMap version 89 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62cc340b80 1 -- 172.21.15.12:0/1354126024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x557357210b60 con 0x557357497ca0 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62a17fa700 1 -- 172.21.15.12:0/1354126024 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55735733c270 con 0x557357497ca0 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62a17fa700 1 -- 172.21.15.12:0/1354126024 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 89) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f62b401f770 con 0x557357497ca0 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62cc340b80 10 client.22470 fetch_fsmap finished waiting for FSMap version 89 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62cc340b80 10 client.22470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62cc340b80 20 client.22470 populate_metadata read hostname 'smithi012' 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62cc340b80 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573574b12a0 0x5573574b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62cc340b80 4 client.22470 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62cc340b80 1 -- 172.21.15.12:0/1354126024 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557357462810 con 0x5573574b12a0 2024-05-09T00:12:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.410+0000 7f62bd721700 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573574b12a0 0x5573574b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.411+0000 7f62a17fa700 10 client.22470 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.411+0000 7f62bd721700 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573574b12a0 0x5573574b36c0 crc :-1 s=READY pgs=636 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e905f41a67a5e052 server_cookie=65949d116956ca5a in_seq=0 out_seq=0 2024-05-09T00:12:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.411+0000 7f62a17fa700 1 -- 172.21.15.12:0/1354126024 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f62a8066a90 con 0x5573574b12a0 2024-05-09T00:12:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.411+0000 7f62a17fa700 10 client.22470 handle_command_reply: tid=0 2024-05-09T00:12:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.411+0000 7f62cc340b80 1 client.22470 shutdown 2024-05-09T00:12:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 1 -- 172.21.15.12:0/1354126024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557357497ca0 msgr2=0x5573574a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557357497ca0 0x5573574a0ec0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f62b4011770 tx=0x7f62b40047c0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 1 -- 172.21.15.12:0/1354126024 shutdown_connections 2024-05-09T00:12:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55735749db10 0x5573574a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557357497ca0 0x5573574a0ec0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x5573574b12a0 0x5573574b36c0 crc :-1 s=READY pgs=636 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 1 --2- 172.21.15.12:0/1354126024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 0x5573574a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62a17fa700 0 client.22470 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 1 -- 172.21.15.12:0/1354126024 >> 172.21.15.12:0/1354126024 conn(0x5573573f1210 msgr2=0x55735748d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 1 -- 172.21.15.12:0/1354126024 shutdown_connections 2024-05-09T00:12:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 1 -- 172.21.15.12:0/1354126024 wait complete. 2024-05-09T00:12:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.412+0000 7f62cc340b80 20 client.22470 trim_cache size 0 max 16384 2024-05-09T00:12:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.426+0000 7f62cc340b80 1 Processor -- start 2024-05-09T00:12:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.426+0000 7f62cc340b80 1 -- start start 2024-05-09T00:12:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.426+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x55735748fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.426+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 0x55735748dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.426+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x5573574a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.426+0000 7f62cc340b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573574b8910 con 0x55735749db10 2024-05-09T00:12:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.426+0000 7f62cc340b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557357488800 con 0x55735749d1f0 2024-05-09T00:12:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.426+0000 7f62cc340b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573574a3e60 con 0x557357497ca0 2024-05-09T00:12:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.426+0000 7f62be723700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x5573574a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.426+0000 7f62be723700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x5573574a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47820/0 (socket says 172.21.15.12:47820) 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62be723700 1 -- 172.21.15.12:0/3970418705 learned_addr learned my addr 172.21.15.12:0/3970418705 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62bd721700 1 --2- 172.21.15.12:0/3970418705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 0x55735748dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62bdf22700 1 --2- 172.21.15.12:0/3970418705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x55735748fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62be723700 1 -- 172.21.15.12:0/3970418705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 msgr2=0x55735748fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62be723700 1 --2- 172.21.15.12:0/3970418705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x55735748fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62be723700 1 -- 172.21.15.12:0/3970418705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 msgr2=0x55735748dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62be723700 1 --2- 172.21.15.12:0/3970418705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 0x55735748dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62be723700 1 -- 172.21.15.12:0/3970418705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573573a9560 con 0x55735749db10 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62bdf22700 1 --2- 172.21.15.12:0/3970418705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x55735748fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62be723700 1 --2- 172.21.15.12:0/3970418705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x5573574a1760 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f62b00098c0 tx=0x7f62b00098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb6b972c1b889ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62acff9700 1 -- 172.21.15.12:0/3970418705 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0004410 con 0x55735749db10 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62acff9700 1 -- 172.21.15.12:0/3970418705 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62b000f040 con 0x55735749db10 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.427+0000 7f62acff9700 1 -- 172.21.15.12:0/3970418705 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0016430 con 0x55735749db10 2024-05-09T00:12:26.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 -- 172.21.15.12:0/3970418705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 msgr2=0x5573574a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 --2- 172.21.15.12:0/3970418705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x5573574a1760 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f62b00098c0 tx=0x7f62b00098f0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 -- 172.21.15.12:0/3970418705 shutdown_connections 2024-05-09T00:12:26.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 --2- 172.21.15.12:0/3970418705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749db10 0x5573574a1760 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 --2- 172.21.15.12:0/3970418705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x55735748fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 --2- 172.21.15.12:0/3970418705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749d1f0 0x55735748dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 -- 172.21.15.12:0/3970418705 >> 172.21.15.12:0/3970418705 conn(0x5573573f1210 msgr2=0x557357498080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 -- 172.21.15.12:0/3970418705 shutdown_connections 2024-05-09T00:12:26.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 -- 172.21.15.12:0/3970418705 wait complete. 2024-05-09T00:12:26.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 -- start start 2024-05-09T00:12:26.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.428+0000 7f62cc340b80 1 Processor -- start 2024-05-09T00:12:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x5573574a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749d1f0 0x5573574a8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62cc340b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749db10 0x5573574a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62cc340b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573574a6a80 con 0x55735749d1f0 2024-05-09T00:12:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62cc340b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573574a6bc0 con 0x55735749db10 2024-05-09T00:12:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62cc340b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573574a6d80 con 0x557357497ca0 2024-05-09T00:12:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62cc340b80 10 client.0 _ll_register_callbacks cb 0x557357388b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bd721700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749d1f0 0x5573574a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bdf22700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x5573574a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bdf22700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x5573574a7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:49354/0 (socket says 172.21.15.12:49354) 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bd721700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749d1f0 0x5573574a8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47832/0 (socket says 172.21.15.12:47832) 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bdf22700 1 -- 172.21.15.12:0/4234865376 learned_addr learned my addr 172.21.15.12:0/4234865376 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62be723700 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749db10 0x5573574a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bdf22700 1 -- 172.21.15.12:0/4234865376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749db10 msgr2=0x5573574a63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bdf22700 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749db10 0x5573574a63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bdf22700 1 -- 172.21.15.12:0/4234865376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749d1f0 msgr2=0x5573574a8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bdf22700 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749d1f0 0x5573574a8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bdf22700 1 -- 172.21.15.12:0/4234865376 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b0003ce0 con 0x557357497ca0 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bd721700 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749d1f0 0x5573574a8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62bdf22700 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x5573574a7fd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f62b4011740 tx=0x7f62b4030770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8d27def234dc719 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.429+0000 7f62a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62a17fa700 1 -- 172.21.15.12:0/4234865376 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4010e80 con 0x557357497ca0 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62a17fa700 1 -- 172.21.15.12:0/4234865376 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f62b4017050 con 0x557357497ca0 2024-05-09T00:12:26.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62cc340b80 1 -- 172.21.15.12:0/4234865376 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62a809b990 con 0x557357497ca0 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62a17fa700 1 -- 172.21.15.12:0/4234865376 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62b400b8f0 con 0x557357497ca0 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62cc340b80 10 client.32134 fetch_fsmap learned FSMap version 89 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62cc340b80 1 -- 172.21.15.12:0/4234865376 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x557357268cd0 con 0x557357497ca0 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62a17fa700 1 -- 172.21.15.12:0/4234865376 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62b4046020 con 0x557357497ca0 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62a17fa700 1 -- 172.21.15.12:0/4234865376 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 89) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f62b40278e0 con 0x557357497ca0 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62cc340b80 10 client.32134 fetch_fsmap finished waiting for FSMap version 89 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62cc340b80 10 client.32134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62cc340b80 20 client.32134 populate_metadata read hostname 'smithi012' 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62cc340b80 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557357508910 0x55735750ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62cc340b80 4 client.32134 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62cc340b80 1 -- 172.21.15.12:0/4234865376 --> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5573573dbec0 con 0x557357508910 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.430+0000 7f62bd721700 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557357508910 0x55735750ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62bd721700 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557357508910 0x55735750ad30 crc :-1 s=READY pgs=637 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0b7bb6e5a42d2f server_cookie=9a33f8f62b6e8479 in_seq=0 out_seq=0 2024-05-09T00:12:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62a17fa700 10 client.32134 ms_handle_connect on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62a17fa700 1 -- 172.21.15.12:0/4234865376 <== mds.0 v2:172.21.15.160:6838/924146489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5573573dbec0 con 0x557357508910 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62a17fa700 10 client.32134 handle_command_reply: tid=0 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 client.32134 shutdown 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 -- 172.21.15.12:0/4234865376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 msgr2=0x5573574a7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x5573574a7fd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f62b4011740 tx=0x7f62b4030770 comp rx=0 tx=0).stop 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 -- 172.21.15.12:0/4234865376 shutdown_connections 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.160:6838/924146489,v1:172.21.15.160:6839/924146489] conn(0x557357508910 0x55735750ad30 crc :-1 s=READY pgs=637 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55735749db10 0x5573574a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557357497ca0 0x5573574a7fd0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 --2- 172.21.15.12:0/4234865376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55735749d1f0 0x5573574a8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62a17fa700 0 client.32134 ms_handle_reset on v2:172.21.15.160:6838/924146489 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 -- 172.21.15.12:0/4234865376 >> 172.21.15.12:0/4234865376 conn(0x5573573f1210 msgr2=0x5573574a17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 -- 172.21.15.12:0/4234865376 shutdown_connections 2024-05-09T00:12:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.431+0000 7f62cc340b80 1 -- 172.21.15.12:0/4234865376 wait complete. 2024-05-09T00:12:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62cc340b80 20 client.32134 trim_cache size 0 max 16384 2024-05-09T00:12:26.436 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-09T00:12:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 -- 172.21.15.12:0/2181908124 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62a40a1290 msgr2=0x7f62a40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62a40a1290 0x7f62a40a3750 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f62b4005950 tx=0x7f62b400b410 comp rx=0 tx=0).stop 2024-05-09T00:12:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 -- 172.21.15.12:0/2181908124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b81062a0 msgr2=0x7f62b81979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b81062a0 0x7f62b81979a0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f62a8000c00 tx=0x7f62a8009580 comp rx=0 tx=0).stop 2024-05-09T00:12:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 -- 172.21.15.12:0/2181908124 shutdown_connections 2024-05-09T00:12:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f62a40a1290 0x7f62a40a3750 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62b819bcf0 0x7f62b819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62b8069420 0x7f62b8197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 --2- 172.21.15.12:0/2181908124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62b81062a0 0x7f62b81979a0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 -- 172.21.15.12:0/2181908124 >> 172.21.15.12:0/2181908124 conn(0x7f62b8076ba0 msgr2=0x7f62b80ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 -- 172.21.15.12:0/2181908124 shutdown_connections 2024-05-09T00:12:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:26.432+0000 7f62c0186700 1 -- 172.21.15.12:0/2181908124 wait complete. 2024-05-09T00:12:27.445 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-09T00:12:27.445 INFO:tasks.fwd_scrub:done joining 2024-05-09T00:12:27.446 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T00:12:27.471 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T00:12:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.617+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2005051258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b20106160 msgr2=0x7f3b2010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.617+0000 7f3b25dfa700 1 --2- 172.21.15.12:0/2005051258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b20106160 0x7f3b2010dd30 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800b0a0 tx=0x7f3b1800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:12:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.617+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2005051258 shutdown_connections 2024-05-09T00:12:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.617+0000 7f3b25dfa700 1 --2- 172.21.15.12:0/2005051258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b20106160 0x7f3b2010dd30 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.617+0000 7f3b25dfa700 1 --2- 172.21.15.12:0/2005051258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b200ff5a0 0x7f3b200ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.617+0000 7f3b25dfa700 1 --2- 172.21.15.12:0/2005051258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b200fff50 0x7f3b20105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.617+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2005051258 >> 172.21.15.12:0/2005051258 conn(0x7f3b20076270 msgr2=0x7f3b200fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.617+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2005051258 shutdown_connections 2024-05-09T00:12:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.617+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2005051258 wait complete. 2024-05-09T00:12:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.618+0000 7f3b25dfa700 1 Processor -- start 2024-05-09T00:12:27.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.618+0000 7f3b25dfa700 1 -- start start 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.618+0000 7f3b25dfa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b200ff5a0 0x7f3b201973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.618+0000 7f3b25dfa700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20106160 0x7f3b20197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.618+0000 7f3b25dfa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b2019bcb0 0x7f3b2019c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.618+0000 7f3b25dfa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2019c730 con 0x7f3b200ff5a0 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.618+0000 7f3b25dfa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2019c8a0 con 0x7f3b2019bcb0 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.618+0000 7f3b25dfa700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2019ca60 con 0x7f3b20106160 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.618+0000 7f3b1f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b200ff5a0 0x7f3b201973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b200ff5a0 0x7f3b201973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47836/0 (socket says 172.21.15.12:47836) 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1f7fe700 1 -- 172.21.15.12:0/2759548198 learned_addr learned my addr 172.21.15.12:0/2759548198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1effd700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20106160 0x7f3b20197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1ffff700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b2019bcb0 0x7f3b2019c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1f7fe700 1 -- 172.21.15.12:0/2759548198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20106160 msgr2=0x7f3b20197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1f7fe700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20106160 0x7f3b20197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1f7fe700 1 -- 172.21.15.12:0/2759548198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b2019bcb0 msgr2=0x7f3b2019c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1f7fe700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b2019bcb0 0x7f3b2019c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1f7fe700 1 -- 172.21.15.12:0/2759548198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b18009d00 con 0x7f3b200ff5a0 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1effd700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20106160 0x7f3b20197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1ffff700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b2019bcb0 0x7f3b2019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1f7fe700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b200ff5a0 0x7f3b201973d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000d950 tx=0x7f3b1000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1cff9700 1 -- 172.21.15.12:0/2759548198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10009b40 con 0x7f3b200ff5a0 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1cff9700 1 -- 172.21.15.12:0/2759548198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b10009ca0 con 0x7f3b200ff5a0 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2759548198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b201a3fc0 con 0x7f3b200ff5a0 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b1cff9700 1 -- 172.21.15.12:0/2759548198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000f620 con 0x7f3b200ff5a0 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.619+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2759548198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b201a4440 con 0x7f3b200ff5a0 2024-05-09T00:12:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.620+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2759548198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b201093f0 con 0x7f3b200ff5a0 2024-05-09T00:12:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.622+0000 7f3b1cff9700 1 -- 172.21.15.12:0/2759548198 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3b10010460 con 0x7f3b200ff5a0 2024-05-09T00:12:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.623+0000 7f3b1cff9700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b080a1290 0x7f3b080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.623+0000 7f3b1effd700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b080a1290 0x7f3b080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.623+0000 7f3b1cff9700 1 -- 172.21.15.12:0/2759548198 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3b100cd0c0 con 0x7f3b200ff5a0 2024-05-09T00:12:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.623+0000 7f3b1effd700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b080a1290 0x7f3b080a3750 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3b20198920 tx=0x7f3b14009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:27.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.627+0000 7f3b1cff9700 1 -- 172.21.15.12:0/2759548198 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3b100964c0 con 0x7f3b200ff5a0 2024-05-09T00:12:27.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.849+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2759548198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b20198760 con 0x7f3b200ff5a0 2024-05-09T00:12:27.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.851+0000 7f3b1cff9700 1 -- 172.21.15.12:0/2759548198 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+22978 (secure 0 0 0) 0x7f3b10020090 con 0x7f3b200ff5a0 2024-05-09T00:12:27.849 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:12:27.849 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":140,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-09T00:12:16.031662+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":139,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3622028746":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3062429456":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2381834553":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/716473668":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2695290437":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/1305168755":"2024-05-09T23:46:25.291064+0000","172.21.15.12:6801/1305168755":"2024-05-09T23:46:25.291064+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":33,"length":1},{"begin":46,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:12:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2759548198 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b080a1290 msgr2=0x7f3b080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b080a1290 0x7f3b080a3750 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3b20198920 tx=0x7f3b14009260 comp rx=0 tx=0).stop 2024-05-09T00:12:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2759548198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b200ff5a0 msgr2=0x7f3b201973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b200ff5a0 0x7f3b201973d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000d950 tx=0x7f3b1000dd10 comp rx=0 tx=0).stop 2024-05-09T00:12:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2759548198 shutdown_connections 2024-05-09T00:12:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f3b080a1290 0x7f3b080a3750 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b2019bcb0 0x7f3b2019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b200ff5a0 0x7f3b201973d0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 --2- 172.21.15.12:0/2759548198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20106160 0x7f3b20197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2759548198 >> 172.21.15.12:0/2759548198 conn(0x7f3b20076270 msgr2=0x7f3b20104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2759548198 shutdown_connections 2024-05-09T00:12:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:27.854+0000 7f3b25dfa700 1 -- 172.21.15.12:0/2759548198 wait complete. 2024-05-09T00:12:27.867 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:45:02.165326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:45:03.407498+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 139, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T00:12:27.867 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T00:12:28.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.014+0000 7f33aff82700 1 -- 172.21.15.12:0/481592452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a8100060 msgr2=0x7f33a8100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.014+0000 7f33aff82700 1 --2- 172.21.15.12:0/481592452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a8100060 0x7f33a8100440 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009990 tx=0x7f33a4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.014+0000 7f33aff82700 1 -- 172.21.15.12:0/481592452 shutdown_connections 2024-05-09T00:12:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.014+0000 7f33aff82700 1 --2- 172.21.15.12:0/481592452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a8105160 0x7f33a810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.014+0000 7f33aff82700 1 --2- 172.21.15.12:0/481592452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a8100060 0x7f33a8100440 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.014+0000 7f33aff82700 1 --2- 172.21.15.12:0/481592452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33a8100a10 0x7f33a8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.014+0000 7f33aff82700 1 -- 172.21.15.12:0/481592452 >> 172.21.15.12:0/481592452 conn(0x7f33a8076f40 msgr2=0x7f33a8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.014+0000 7f33aff82700 1 -- 172.21.15.12:0/481592452 shutdown_connections 2024-05-09T00:12:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.014+0000 7f33aff82700 1 -- 172.21.15.12:0/481592452 wait complete. 2024-05-09T00:12:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33aff82700 1 Processor -- start 2024-05-09T00:12:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33aff82700 1 -- start start 2024-05-09T00:12:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33aff82700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a8100060 0x7f33a81962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33aff82700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33a8105160 0x7f33a8196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33aff82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a819aba0 0x7f33a819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33aff82700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a819b620 con 0x7f33a819aba0 2024-05-09T00:12:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33aff82700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a819b790 con 0x7f33a8105160 2024-05-09T00:12:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33aff82700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a819b950 con 0x7f33a8100060 2024-05-09T00:12:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33ae51f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a819aba0 0x7f33a819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33ae51f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a819aba0 0x7f33a819b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47856/0 (socket says 172.21.15.12:47856) 2024-05-09T00:12:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33ae51f700 1 -- 172.21.15.12:0/3421671836 learned_addr learned my addr 172.21.15.12:0/3421671836 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33add1e700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a8100060 0x7f33a81962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.015+0000 7f33ad51d700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33a8105160 0x7f33a8196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f33ae51f700 1 -- 172.21.15.12:0/3421671836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a8100060 msgr2=0x7f33a81962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f33ae51f700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a8100060 0x7f33a81962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f33ae51f700 1 -- 172.21.15.12:0/3421671836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33a8105160 msgr2=0x7f33a8196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f33ae51f700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33a8105160 0x7f33a8196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f33ae51f700 1 -- 172.21.15.12:0/3421671836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a40095f0 con 0x7f33a819aba0 2024-05-09T00:12:28.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f33ad51d700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33a8105160 0x7f33a8196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f33ae51f700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a819aba0 0x7f33a819b020 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f33a0007ee0 tx=0x7f33a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f339effd700 1 -- 172.21.15.12:0/3421671836 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0017070 con 0x7f33a819aba0 2024-05-09T00:12:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f33aff82700 1 -- 172.21.15.12:0/3421671836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a81a2eb0 con 0x7f33a819aba0 2024-05-09T00:12:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f339effd700 1 -- 172.21.15.12:0/3421671836 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33a000f040 con 0x7f33a819aba0 2024-05-09T00:12:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f339effd700 1 -- 172.21.15.12:0/3421671836 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0005740 con 0x7f33a819aba0 2024-05-09T00:12:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.016+0000 7f33aff82700 1 -- 172.21.15.12:0/3421671836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a81a3340 con 0x7f33a819aba0 2024-05-09T00:12:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.017+0000 7f33aff82700 1 -- 172.21.15.12:0/3421671836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a81083d0 con 0x7f33a819aba0 2024-05-09T00:12:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.018+0000 7f339effd700 1 -- 172.21.15.12:0/3421671836 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f33a000dc60 con 0x7f33a819aba0 2024-05-09T00:12:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.019+0000 7f339effd700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33940a1290 0x7f33940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.019+0000 7f33add1e700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33940a1290 0x7f33940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.019+0000 7f339effd700 1 -- 172.21.15.12:0/3421671836 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f33a00cc450 con 0x7f33a819aba0 2024-05-09T00:12:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.019+0000 7f33add1e700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33940a1290 0x7f33940a3750 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f33a400b5c0 tx=0x7f33a4005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.024+0000 7f339effd700 1 -- 172.21.15.12:0/3421671836 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f33a00958b0 con 0x7f33a819aba0 2024-05-09T00:12:28.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:27 smithi012 ceph-mon[42503]: pgmap v840: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.6 MiB/s wr, 175 op/s 2024-05-09T00:12:28.169 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:27 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2759548198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:28.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.240+0000 7f33aff82700 1 -- 172.21.15.12:0/3421671836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33a81975c0 con 0x7f33a819aba0 2024-05-09T00:12:28.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.240+0000 7f339effd700 1 -- 172.21.15.12:0/3421671836 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33a0098ed0 con 0x7f33a819aba0 2024-05-09T00:12:28.238 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T00:12:28.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 -- 172.21.15.12:0/3421671836 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33940a1290 msgr2=0x7f33940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33940a1290 0x7f33940a3750 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f33a400b5c0 tx=0x7f33a4005960 comp rx=0 tx=0).stop 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 -- 172.21.15.12:0/3421671836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a819aba0 msgr2=0x7f33a819b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a819aba0 0x7f33a819b020 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f33a0007ee0 tx=0x7f33a000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 -- 172.21.15.12:0/3421671836 shutdown_connections 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f33940a1290 0x7f33940a3750 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a819aba0 0x7f33a819b020 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a8100060 0x7f33a81962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 --2- 172.21.15.12:0/3421671836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33a8105160 0x7f33a8196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 -- 172.21.15.12:0/3421671836 >> 172.21.15.12:0/3421671836 conn(0x7f33a8076f40 msgr2=0x7f33a80fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 -- 172.21.15.12:0/3421671836 shutdown_connections 2024-05-09T00:12:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.242+0000 7f33aff82700 1 -- 172.21.15.12:0/3421671836 wait complete. 2024-05-09T00:12:28.254 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T00:12:28.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:27 smithi204 ceph-mon[47373]: pgmap v840: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.6 MiB/s wr, 175 op/s 2024-05-09T00:12:28.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:27 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2759548198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:28.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:27 smithi160 ceph-mon[46152]: pgmap v840: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.6 MiB/s wr, 175 op/s 2024-05-09T00:12:28.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:27 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2759548198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.397+0000 7fa3fcc78700 1 -- 172.21.15.12:0/447268654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3f8105160 msgr2=0x7fa3f810cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.397+0000 7fa3fcc78700 1 --2- 172.21.15.12:0/447268654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3f8105160 0x7fa3f810cce0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000b0a0 tx=0x7fa3f000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.397+0000 7fa3fcc78700 1 -- 172.21.15.12:0/447268654 shutdown_connections 2024-05-09T00:12:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.397+0000 7fa3fcc78700 1 --2- 172.21.15.12:0/447268654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3f8105160 0x7fa3f810cce0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.397+0000 7fa3fcc78700 1 --2- 172.21.15.12:0/447268654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f8100060 0x7fa3f8100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.397+0000 7fa3fcc78700 1 --2- 172.21.15.12:0/447268654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3f8100a10 0x7fa3f8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.397+0000 7fa3fcc78700 1 -- 172.21.15.12:0/447268654 >> 172.21.15.12:0/447268654 conn(0x7fa3f8076f40 msgr2=0x7fa3f8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.397+0000 7fa3fcc78700 1 -- 172.21.15.12:0/447268654 shutdown_connections 2024-05-09T00:12:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.397+0000 7fa3fcc78700 1 -- 172.21.15.12:0/447268654 wait complete. 2024-05-09T00:12:28.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.398+0000 7fa3fcc78700 1 Processor -- start 2024-05-09T00:12:28.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.398+0000 7fa3fcc78700 1 -- start start 2024-05-09T00:12:28.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.398+0000 7fa3fcc78700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3f8100060 0x7fa3f8101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.398+0000 7fa3fcc78700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3f8105160 0x7fa3f8102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.398+0000 7fa3fcc78700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f8102550 0x7fa3f81a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.398+0000 7fa3fcc78700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81a5d10 con 0x7fa3f8105160 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.398+0000 7fa3fcc78700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81a5e80 con 0x7fa3f8102550 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.398+0000 7fa3fcc78700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81a6040 con 0x7fa3f8100060 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.398+0000 7fa3f5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3f8105160 0x7fa3f8102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3f5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3f8105160 0x7fa3f8102010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52256/0 (socket says 172.21.15.12:52256) 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3f5d9b700 1 -- 172.21.15.12:0/1793007818 learned_addr learned my addr 172.21.15.12:0/1793007818 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3f659c700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3f8100060 0x7fa3f8101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3f6d9d700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f8102550 0x7fa3f81a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3f5d9b700 1 -- 172.21.15.12:0/1793007818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3f8100060 msgr2=0x7fa3f8101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3f5d9b700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3f8100060 0x7fa3f8101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3f5d9b700 1 -- 172.21.15.12:0/1793007818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f8102550 msgr2=0x7fa3f81a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3f5d9b700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f8102550 0x7fa3f81a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3f5d9b700 1 -- 172.21.15.12:0/1793007818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0009d00 con 0x7fa3f8105160 2024-05-09T00:12:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3f5d9b700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3f8105160 0x7fa3f8102010 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800d950 tx=0x7fa3e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3ef7fe700 1 -- 172.21.15.12:0/1793007818 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8004220 con 0x7fa3f8105160 2024-05-09T00:12:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3ef7fe700 1 -- 172.21.15.12:0/1793007818 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e8007620 con 0x7fa3f8105160 2024-05-09T00:12:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3fcc78700 1 -- 172.21.15.12:0/1793007818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f81a6320 con 0x7fa3f8105160 2024-05-09T00:12:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3ef7fe700 1 -- 172.21.15.12:0/1793007818 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8005690 con 0x7fa3f8105160 2024-05-09T00:12:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.399+0000 7fa3fcc78700 1 -- 172.21.15.12:0/1793007818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f81a6880 con 0x7fa3f8105160 2024-05-09T00:12:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.400+0000 7fa3fcc78700 1 -- 172.21.15.12:0/1793007818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f8109440 con 0x7fa3f8105160 2024-05-09T00:12:28.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.402+0000 7fa3ef7fe700 1 -- 172.21.15.12:0/1793007818 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa3e8004750 con 0x7fa3f8105160 2024-05-09T00:12:28.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.402+0000 7fa3ef7fe700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3e40a1310 0x7fa3e40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.402+0000 7fa3ef7fe700 1 -- 172.21.15.12:0/1793007818 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa3e80cce90 con 0x7fa3f8105160 2024-05-09T00:12:28.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.402+0000 7fa3f659c700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3e40a1310 0x7fa3e40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.403+0000 7fa3f659c700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3e40a1310 0x7fa3e40a37d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0005950 tx=0x7fa3e000b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.406+0000 7fa3ef7fe700 1 -- 172.21.15.12:0/1793007818 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa3e80962f0 con 0x7fa3f8105160 2024-05-09T00:12:28.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.633+0000 7fa3fcc78700 1 -- 172.21.15.12:0/1793007818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3f8102fb0 con 0x7fa3f8105160 2024-05-09T00:12:28.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.633+0000 7fa3ef7fe700 1 -- 172.21.15.12:0/1793007818 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3e80160a0 con 0x7fa3f8105160 2024-05-09T00:12:28.631 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T00:12:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 -- 172.21.15.12:0/1793007818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3e40a1310 msgr2=0x7fa3e40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3e40a1310 0x7fa3e40a37d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0005950 tx=0x7fa3e000b410 comp rx=0 tx=0).stop 2024-05-09T00:12:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 -- 172.21.15.12:0/1793007818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3f8105160 msgr2=0x7fa3f8102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3f8105160 0x7fa3f8102010 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800d950 tx=0x7fa3e800dc60 comp rx=0 tx=0).stop 2024-05-09T00:12:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 -- 172.21.15.12:0/1793007818 shutdown_connections 2024-05-09T00:12:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa3e40a1310 0x7fa3e40a37d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f8102550 0x7fa3f81a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3f8100060 0x7fa3f8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 --2- 172.21.15.12:0/1793007818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3f8105160 0x7fa3f8102010 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 -- 172.21.15.12:0/1793007818 >> 172.21.15.12:0/1793007818 conn(0x7fa3f8076f40 msgr2=0x7fa3f8068370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 -- 172.21.15.12:0/1793007818 shutdown_connections 2024-05-09T00:12:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.636+0000 7fa3fcc78700 1 -- 172.21.15.12:0/1793007818 wait complete. 2024-05-09T00:12:28.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T00:12:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.844+0000 7fb6ba386700 1 -- 172.21.15.12:0/3739032340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b4069d90 msgr2=0x7fb6b4105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.844+0000 7fb6ba386700 1 --2- 172.21.15.12:0/3739032340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b4069d90 0x7fb6b4105b70 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009990 tx=0x7fb69c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.844+0000 7fb6ba386700 1 -- 172.21.15.12:0/3739032340 shutdown_connections 2024-05-09T00:12:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.844+0000 7fb6ba386700 1 --2- 172.21.15.12:0/3739032340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b41062a0 0x7fb6b410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.844+0000 7fb6ba386700 1 --2- 172.21.15.12:0/3739032340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6b40693e0 0x7fb6b40697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.844+0000 7fb6ba386700 1 --2- 172.21.15.12:0/3739032340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b4069d90 0x7fb6b4105b70 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.844+0000 7fb6ba386700 1 -- 172.21.15.12:0/3739032340 >> 172.21.15.12:0/3739032340 conn(0x7fb6b4076ba0 msgr2=0x7fb6b4076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.844+0000 7fb6ba386700 1 -- 172.21.15.12:0/3739032340 shutdown_connections 2024-05-09T00:12:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.844+0000 7fb6ba386700 1 -- 172.21.15.12:0/3739032340 wait complete. 2024-05-09T00:12:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6ba386700 1 Processor -- start 2024-05-09T00:12:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6ba386700 1 -- start start 2024-05-09T00:12:28.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6ba386700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6b40693e0 0x7fb6b41973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6ba386700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b41062a0 0x7fb6b4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6ba386700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b419bcd0 0x7fb6b419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6ba386700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b419c750 con 0x7fb6b41062a0 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6ba386700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b419c8c0 con 0x7fb6b419bcd0 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6ba386700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b419ca80 con 0x7fb6b40693e0 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b41062a0 0x7fb6b4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6b3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6b40693e0 0x7fb6b41973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6b3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6b40693e0 0x7fb6b41973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.12:56184/0 (socket says 172.21.15.12:56184) 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6b3fff700 1 -- 172.21.15.12:0/1495793355 learned_addr learned my addr 172.21.15.12:0/1495793355 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.845+0000 7fb6b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b41062a0 0x7fb6b4197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52274/0 (socket says 172.21.15.12:52274) 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b8923700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b419bcd0 0x7fb6b419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b3fff700 1 -- 172.21.15.12:0/1495793355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b419bcd0 msgr2=0x7fb6b419c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b3fff700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b419bcd0 0x7fb6b419c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b3fff700 1 -- 172.21.15.12:0/1495793355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b41062a0 msgr2=0x7fb6b4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b3fff700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b41062a0 0x7fb6b4197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b3fff700 1 -- 172.21.15.12:0/1495793355 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c0095f0 con 0x7fb6b40693e0 2024-05-09T00:12:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b37fe700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b41062a0 0x7fb6b4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b3fff700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6b40693e0 0x7fb6b41973f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400e9a0 tx=0x7fb6a400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b17fa700 1 -- 172.21.15.12:0/1495793355 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4009790 con 0x7fb6b40693e0 2024-05-09T00:12:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b17fa700 1 -- 172.21.15.12:0/1495793355 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6a400cbd0 con 0x7fb6b40693e0 2024-05-09T00:12:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6ba386700 1 -- 172.21.15.12:0/1495793355 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b41a3fe0 con 0x7fb6b40693e0 2024-05-09T00:12:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6b17fa700 1 -- 172.21.15.12:0/1495793355 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4018680 con 0x7fb6b40693e0 2024-05-09T00:12:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.846+0000 7fb6ba386700 1 -- 172.21.15.12:0/1495793355 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b41a4410 con 0x7fb6b40693e0 2024-05-09T00:12:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.847+0000 7fb6ba386700 1 -- 172.21.15.12:0/1495793355 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b4109570 con 0x7fb6b40693e0 2024-05-09T00:12:28.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.848+0000 7fb6b17fa700 1 -- 172.21.15.12:0/1495793355 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb6a400cd40 con 0x7fb6b40693e0 2024-05-09T00:12:28.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.849+0000 7fb6b17fa700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6a00a1310 0x7fb6a00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:28.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.849+0000 7fb6b37fe700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6a00a1310 0x7fb6a00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:28.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.849+0000 7fb6b17fa700 1 -- 172.21.15.12:0/1495793355 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb6a4014070 con 0x7fb6b40693e0 2024-05-09T00:12:28.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.850+0000 7fb6b37fe700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6a00a1310 0x7fb6a00a37d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00b2e0 tx=0x7fb69c005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:28.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:28.852+0000 7fb6b17fa700 1 -- 172.21.15.12:0/1495793355 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb6a4096360 con 0x7fb6b40693e0 2024-05-09T00:12:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:28 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3421671836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:12:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:28 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1793007818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:12:29.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.070+0000 7fb6ba386700 1 -- 172.21.15.12:0/1495793355 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6b41986f0 con 0x7fb6b40693e0 2024-05-09T00:12:29.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.070+0000 7fb6b17fa700 1 -- 172.21.15.12:0/1495793355 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb6a4099980 con 0x7fb6b40693e0 2024-05-09T00:12:29.068 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T00:12:29.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 -- 172.21.15.12:0/1495793355 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6a00a1310 msgr2=0x7fb6a00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6a00a1310 0x7fb6a00a37d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00b2e0 tx=0x7fb69c005960 comp rx=0 tx=0).stop 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 -- 172.21.15.12:0/1495793355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6b40693e0 msgr2=0x7fb6b41973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6b40693e0 0x7fb6b41973f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400e9a0 tx=0x7fb6a400ecb0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 -- 172.21.15.12:0/1495793355 shutdown_connections 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb6a00a1310 0x7fb6a00a37d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b419bcd0 0x7fb6b419c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6b40693e0 0x7fb6b41973f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 --2- 172.21.15.12:0/1495793355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b41062a0 0x7fb6b4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 -- 172.21.15.12:0/1495793355 >> 172.21.15.12:0/1495793355 conn(0x7fb6b4076ba0 msgr2=0x7fb6b4107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 -- 172.21.15.12:0/1495793355 shutdown_connections 2024-05-09T00:12:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.073+0000 7fb6ba386700 1 -- 172.21.15.12:0/1495793355 wait complete. 2024-05-09T00:12:29.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T00:12:29.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.225+0000 7fb1407bb700 1 -- 172.21.15.12:0/439199322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1380fff50 msgr2=0x7fb138100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:29.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.225+0000 7fb1407bb700 1 --2- 172.21.15.12:0/439199322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1380fff50 0x7fb138100330 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fb134009990 tx=0x7fb134009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.225+0000 7fb1407bb700 1 -- 172.21.15.12:0/439199322 shutdown_connections 2024-05-09T00:12:29.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.225+0000 7fb1407bb700 1 --2- 172.21.15.12:0/439199322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb138105070 0x7fb13810cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.225+0000 7fb1407bb700 1 --2- 172.21.15.12:0/439199322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1380fff50 0x7fb138100330 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.225+0000 7fb1407bb700 1 --2- 172.21.15.12:0/439199322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb138100900 0x7fb138104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.225+0000 7fb1407bb700 1 -- 172.21.15.12:0/439199322 >> 172.21.15.12:0/439199322 conn(0x7fb1380fb810 msgr2=0x7fb1380fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.225+0000 7fb1407bb700 1 -- 172.21.15.12:0/439199322 shutdown_connections 2024-05-09T00:12:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.225+0000 7fb1407bb700 1 -- 172.21.15.12:0/439199322 wait complete. 2024-05-09T00:12:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb1407bb700 1 Processor -- start 2024-05-09T00:12:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb1407bb700 1 -- start start 2024-05-09T00:12:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb1407bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1380fff50 0x7fb1381019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb1407bb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb138105070 0x7fb138101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb1407bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb138102460 0x7fb1381a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb1407bb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1381a5ba0 con 0x7fb1380fff50 2024-05-09T00:12:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb1407bb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1381a5d10 con 0x7fb138105070 2024-05-09T00:12:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb1407bb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1381a5ed0 con 0x7fb138102460 2024-05-09T00:12:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb13e557700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1380fff50 0x7fb1381019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb13e557700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1380fff50 0x7fb1381019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52296/0 (socket says 172.21.15.12:52296) 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb13e557700 1 -- 172.21.15.12:0/3643076089 learned_addr learned my addr 172.21.15.12:0/3643076089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb13dd56700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb138105070 0x7fb138101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb13ed58700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb138102460 0x7fb1381a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb13e557700 1 -- 172.21.15.12:0/3643076089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb138102460 msgr2=0x7fb1381a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb13e557700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb138102460 0x7fb1381a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb13e557700 1 -- 172.21.15.12:0/3643076089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb138105070 msgr2=0x7fb138101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb13e557700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb138105070 0x7fb138101f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.226+0000 7fb13e557700 1 -- 172.21.15.12:0/3643076089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340095f0 con 0x7fb1380fff50 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.227+0000 7fb13e557700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1380fff50 0x7fb1381019e0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fb134005910 tx=0x7fb134004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.227+0000 7fb12f7fe700 1 -- 172.21.15.12:0/3643076089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13401d070 con 0x7fb1380fff50 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.227+0000 7fb12f7fe700 1 -- 172.21.15.12:0/3643076089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb134022470 con 0x7fb1380fff50 2024-05-09T00:12:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.227+0000 7fb1407bb700 1 -- 172.21.15.12:0/3643076089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1381a6150 con 0x7fb1380fff50 2024-05-09T00:12:29.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.227+0000 7fb12f7fe700 1 -- 172.21.15.12:0/3643076089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134017650 con 0x7fb1380fff50 2024-05-09T00:12:29.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.227+0000 7fb1407bb700 1 -- 172.21.15.12:0/3643076089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1381a6570 con 0x7fb1380fff50 2024-05-09T00:12:29.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.228+0000 7fb1407bb700 1 -- 172.21.15.12:0/3643076089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1381092f0 con 0x7fb1380fff50 2024-05-09T00:12:29.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.229+0000 7fb12f7fe700 1 -- 172.21.15.12:0/3643076089 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb1340225e0 con 0x7fb1380fff50 2024-05-09T00:12:29.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.230+0000 7fb12f7fe700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1240a1290 0x7fb1240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:29.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.230+0000 7fb13dd56700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1240a1290 0x7fb1240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:29.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.230+0000 7fb12f7fe700 1 -- 172.21.15.12:0/3643076089 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb1340ce4e0 con 0x7fb1380fff50 2024-05-09T00:12:29.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.230+0000 7fb13dd56700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1240a1290 0x7fb1240a3750 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb138103110 tx=0x7fb1280073d0 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:29.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.234+0000 7fb12f7fe700 1 -- 172.21.15.12:0/3643076089 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb134097970 con 0x7fb1380fff50 2024-05-09T00:12:29.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:28 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3421671836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:12:29.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:28 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1793007818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:12:29.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:28 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3421671836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:12:29.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:28 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1793007818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:12:29.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.516+0000 7fb1407bb700 1 -- 172.21.15.12:0/3643076089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fb138101420 con 0x7fb1380fff50 2024-05-09T00:12:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.527+0000 7fb12f7fe700 1 -- 172.21.15.12:0/3643076089 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+733940 (secure 0 0 0) 0x7fb13409af90 con 0x7fb1380fff50 2024-05-09T00:12:29.525 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:12:29.525 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:12:29.525 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 25, 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: "active_gid": 15252, 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: "active_name": "x", 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addrs": { 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2525408937 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2525408937 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addr": "172.21.15.12:6801/2525408937", 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: "active_change": "2024-05-08T23:46:25.291356+0000", 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: "active_mgr_features": 4540138322906710015, 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: "available": true, 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: "standbys": [ 2024-05-09T00:12:29.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 25118, 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "y", 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540138322906710015, 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.529 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.530 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.531 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.532 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.534 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-05-09T00:12:29.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-05-09T00:12:29.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-05-09T00:12:29.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-05-09T00:12:29.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.541 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.541 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.541 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.541 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.541 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.541 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.541 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-05-09T00:12:29.541 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.543 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-05-09T00:12:29.544 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.545 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-05-09T00:12:29.546 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:29.547 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-05-09T00:12:29.548 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T00:12:29.550 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.551 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.552 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T00:12:29.553 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.554 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.555 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.556 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.556 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.556 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.556 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.566 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.566 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.567 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-05-09T00:12:29.570 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-05-09T00:12:29.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:29.570 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:29.570 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.570 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.570 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.570 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T00:12:29.570 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.571 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap" 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.572 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.573 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.573 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T00:12:29.573 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.573 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.574 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:29.575 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:29.576 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.578 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-05-09T00:12:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.581 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.581 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-05-09T00:12:29.581 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.581 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.581 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T00:12:29.581 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.582 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.582 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-05-09T00:12:29.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-05-09T00:12:29.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.582 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-05-09T00:12:29.583 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.584 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.585 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-05-09T00:12:29.585 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T00:12:29.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:29.585 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.585 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.585 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-05-09T00:12:29.585 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.585 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.585 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.587 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.588 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.589 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.590 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.591 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.594 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.595 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T00:12:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-05-09T00:12:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-05-09T00:12:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-05-09T00:12:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-09T00:12:29.602 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.603 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-05-09T00:12:29.605 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-05-09T00:12:29.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-05-09T00:12:29.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.608 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.610 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.612 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.613 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.614 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.615 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.617 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.618 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.619 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.621 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.623 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.623 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.623 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.623 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.623 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.623 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.623 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.624 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-05-09T00:12:29.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.626 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.628 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T00:12:29.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T00:12:29.631 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.632 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.633 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.634 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.635 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.636 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.637 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-05-09T00:12:29.638 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.639 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.641 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-05-09T00:12:29.641 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-05-09T00:12:29.641 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.641 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.641 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.641 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.641 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.641 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.641 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.641 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.642 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.643 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.644 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.645 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-05-09T00:12:29.646 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.647 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.648 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.649 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.650 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.651 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-05-09T00:12:29.652 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.653 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.654 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.655 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.656 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.657 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.658 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.659 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.661 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.662 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.663 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.664 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.665 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T00:12:29.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.667 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.667 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.667 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.667 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.667 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.667 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.667 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.667 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.667 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.669 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T00:12:29.669 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T00:12:29.669 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.669 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.669 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.669 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.669 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.670 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.671 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.672 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.673 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T00:12:29.674 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.675 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.677 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.678 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.679 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.680 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.681 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.683 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.684 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.685 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T00:12:29.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T00:12:29.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T00:12:29.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.690 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.691 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.692 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T00:12:29.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T00:12:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T00:12:29.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T00:12:29.696 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T00:12:29.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T00:12:29.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T00:12:29.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T00:12:29.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-05-09T00:12:29.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-05-09T00:12:29.702 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-05-09T00:12:29.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-05-09T00:12:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T00:12:29.705 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.706 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.707 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.708 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.709 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.710 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-05-09T00:12:29.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.713 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.716 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-05-09T00:12:29.717 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.718 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.720 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.722 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.724 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.727 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.728 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.729 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.730 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.732 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.734 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.735 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.737 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.738 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.740 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.741 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.742 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.744 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-05-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.747 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.750 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.754 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.755 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.756 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.757 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.758 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.759 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.763 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.764 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.765 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.766 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.767 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.768 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.769 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.770 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-05-09T00:12:29.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-05-09T00:12:29.772 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.773 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.774 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.775 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.777 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.778 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.779 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.780 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.781 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.782 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.783 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.784 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-05-09T00:12:29.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-05-09T00:12:29.786 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-05-09T00:12:29.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-05-09T00:12:29.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-05-09T00:12:29.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.790 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.791 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.792 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.793 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.794 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.795 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.797 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.798 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.799 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.800 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.802 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.803 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.806 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T00:12:29.807 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.808 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.809 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.811 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.812 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.813 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.814 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-05-09T00:12:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.816 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.818 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.819 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.820 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.821 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.822 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.823 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.824 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.825 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.827 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.828 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.829 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.830 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.832 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-05-09T00:12:29.833 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.834 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.835 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.836 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.837 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.839 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.840 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.841 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.842 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.843 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.844 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T00:12:29.845 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.846 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.847 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.848 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.849 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.850 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.851 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.852 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.853 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.854 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.855 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.856 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.857 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.858 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-05-09T00:12:29.858 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-05-09T00:12:29.858 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.858 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.858 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.858 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.858 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.858 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.859 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.861 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.862 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.863 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.864 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-05-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.872 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.873 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.873 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.873 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.873 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.873 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.873 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.873 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.874 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.875 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.877 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.877 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.877 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.877 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T00:12:29.877 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.878 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.879 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.880 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.881 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.882 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.883 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.884 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.885 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.886 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.887 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.888 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.889 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.890 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.890 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.890 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.890 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.890 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.890 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.890 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.891 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.892 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.893 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.894 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.895 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.896 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.897 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.898 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.899 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.900 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.901 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.902 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.903 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.904 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.905 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.908 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.909 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.910 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.911 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.913 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.914 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.915 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.917 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.918 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.922 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.925 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.926 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.927 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.928 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.930 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.931 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.932 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.933 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-05-09T00:12:29.934 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.935 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.937 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.938 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.941 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T00:12:29.942 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.943 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-05-09T00:12:29.944 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-05-09T00:12:29.945 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-05-09T00:12:29.946 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.948 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.950 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.951 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.952 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T00:12:29.954 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.955 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.956 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.957 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.962 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.964 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.965 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T00:12:29.966 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T00:12:29.967 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.969 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.970 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.971 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.973 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.974 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.976 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.977 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.978 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-05-09T00:12:29.979 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 25120, 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "z", 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540138322906710015, 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:29.981 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:29.982 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.983 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.984 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:29.985 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:29.986 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-05-09T00:12:29.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.988 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.989 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.990 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.991 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.992 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.993 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:29.994 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:29.995 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-05-09T00:12:29.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-05-09T00:12:29.997 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.998 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:29.999 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-05-09T00:12:30.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-05-09T00:12:30.001 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-05-09T00:12:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.003 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.004 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.005 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.006 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.007 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.008 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-05-09T00:12:30.009 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap" 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.010 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.011 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.012 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.013 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.014 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.016 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.017 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.018 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.019 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T00:12:30.020 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.022 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T00:12:30.023 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-05-09T00:12:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-05-09T00:12:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.029 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.032 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:30.034 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-05-09T00:12:30.035 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.037 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.038 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.039 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.040 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.042 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.043 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.045 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.049 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.050 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.051 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.052 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.059 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.060 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.061 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.062 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.063 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.068 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-05-09T00:12:30.069 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.071 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T00:12:30.074 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.076 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.077 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.078 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.080 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.081 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.082 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.083 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.084 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.086 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.087 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.088 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.091 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T00:12:30.092 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T00:12:30.093 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.094 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T00:12:30.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T00:12:30.096 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T00:12:30.097 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-05-09T00:12:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T00:12:30.099 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.100 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.101 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.102 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.107 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.108 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.109 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.111 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.112 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.113 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.114 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.115 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-05-09T00:12:30.116 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-05-09T00:12:30.117 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.118 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.120 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.121 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-05-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.131 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.132 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.133 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-05-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-05-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-05-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-05-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-05-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-05-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-05-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-05-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.290 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-05-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-05-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-05-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-05-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-05-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-05-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: "modules": [ 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard", 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: "iostat", 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: "nfs", 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: "restful", 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_schedule" 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap" 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-05-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-05-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-05-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-05-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-05-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-05-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-05-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-05-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-05-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-05-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-05-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-05-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-05-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-05-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-05-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-05-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-05-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-05-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-05-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-05-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-05-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-05-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-05-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-05-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-05-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.691 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.691 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-05-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-05-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-05-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-05-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-05-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-05-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-05-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-05-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-05-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-05-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "services": { 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard": "https://172.21.15.12:8443/" 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "always_on_modules": { 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "octopus": [ 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "pacific": [ 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "quincy": [ 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi012.stdout: "reef": [ 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "last_failure_osd_epoch": 84, 2024-05-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi012.stdout: "active_clients": [ 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1196050754 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "libcephsqlite", 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3874684928 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4142442952 2024-05-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1391007454 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 477369518 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 747845935 2024-05-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:12:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 -- 172.21.15.12:0/3643076089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1240a1290 msgr2=0x7fb1240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1240a1290 0x7fb1240a3750 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb138103110 tx=0x7fb1280073d0 comp rx=0 tx=0).stop 2024-05-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 -- 172.21.15.12:0/3643076089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1380fff50 msgr2=0x7fb1381019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1380fff50 0x7fb1381019e0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fb134005910 tx=0x7fb134004730 comp rx=0 tx=0).stop 2024-05-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 -- 172.21.15.12:0/3643076089 shutdown_connections 2024-05-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fb1240a1290 0x7fb1240a3750 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb138102460 0x7fb1381a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1380fff50 0x7fb1381019e0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 --2- 172.21.15.12:0/3643076089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb138105070 0x7fb138101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 -- 172.21.15.12:0/3643076089 >> 172.21.15.12:0/3643076089 conn(0x7fb1380fb810 msgr2=0x7fb138069630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 -- 172.21.15.12:0/3643076089 shutdown_connections 2024-05-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:29.533+0000 7fb1407bb700 1 -- 172.21.15.12:0/3643076089 wait complete. 2024-05-09T00:12:30.894 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T00:12:30.895 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T00:12:30.896 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:29 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1495793355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:12:30.896 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:29 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3643076089' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T00:12:30.896 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:29 smithi012 ceph-mon[42503]: pgmap v841: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 7.1 MiB/s wr, 181 op/s 2024-05-09T00:12:30.898 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:29 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1495793355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:12:30.898 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:29 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3643076089' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T00:12:30.898 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:29 smithi160 ceph-mon[46152]: pgmap v841: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 7.1 MiB/s wr, 181 op/s 2024-05-09T00:12:30.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:29 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1495793355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:12:30.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:29 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3643076089' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T00:12:30.898 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:29 smithi204 ceph-mon[47373]: pgmap v841: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 7.1 MiB/s wr, 181 op/s 2024-05-09T00:12:31.153 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:32.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:31 smithi012 ceph-mon[42503]: pgmap v842: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 4.7 MiB/s wr, 144 op/s 2024-05-09T00:12:32.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:31 smithi204 ceph-mon[47373]: pgmap v842: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 4.7 MiB/s wr, 144 op/s 2024-05-09T00:12:32.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:31 smithi160 ceph-mon[46152]: pgmap v842: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 4.7 MiB/s wr, 144 op/s 2024-05-09T00:12:32.411 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.002402442, 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000640246, 2024-05-09T00:12:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 453, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 441, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1491728, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 252176, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.074401295, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.027130079, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.040338349, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.010688449, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 441, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.029795108, 2024-05-09T00:12:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000067562 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1491728, 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 252176 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 3, 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 337, 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 480, 2024-05-09T00:12:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 288080, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 6, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 3, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.024030222, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.017095008, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.002935261, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 334, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.069205656, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000207202 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T00:12:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 288080 2024-05-09T00:12:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-09T00:12:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-05-09T00:12:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-09T00:12:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-09T00:12:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-09T00:12:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148731, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 31, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 452, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 1425320, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 452, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 452, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1425320, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:32.424 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:12:32.779 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-09T00:12:32.779 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-09T00:12:32.779 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-09T00:12:32.779 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-09T00:12:32.779 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-09T00:12:32.779 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T00:12:32.780 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T00:12:33.044 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:33.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:33 smithi012 ceph-mon[42503]: pgmap v843: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 4.2 MiB/s wr, 155 op/s 2024-05-09T00:12:34.240 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:12:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 155, 2024-05-09T00:12:34.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 154, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 182688, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 87280, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.023395101, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.008451468, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.008798716, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.003850117, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 153, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.019779989, 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000129280 2024-05-09T00:12:34.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 182688, 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 87280 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 5, 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 118, 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 3104, 2024-05-09T00:12:34.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 96944, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 3, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.008708827, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.006228032, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001458355, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 117, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.020175219, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000172437 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:34.243 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 96944 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.000564555, 2024-05-09T00:12:34.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000452952, 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:34.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T00:12:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T00:12:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-09T00:12:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-05-09T00:12:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-09T00:12:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-09T00:12:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:34.246 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:34.247 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:34.248 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 149998, 2024-05-09T00:12:34.249 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 30, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:34.250 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 156, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 160798, 2024-05-09T00:12:34.251 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 156, 2024-05-09T00:12:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T00:12:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T00:12:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 156, 2024-05-09T00:12:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 160798, 2024-05-09T00:12:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T00:12:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:34.252 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:34.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:34.253 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:12:34.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:33 smithi204 ceph-mon[47373]: pgmap v843: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 4.2 MiB/s wr, 155 op/s 2024-05-09T00:12:34.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:33 smithi160 ceph-mon[46152]: pgmap v843: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 4.2 MiB/s wr, 155 op/s 2024-05-09T00:12:34.713 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-09T00:12:34.713 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-09T00:12:34.713 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-09T00:12:34.713 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-09T00:12:34.713 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-09T00:12:34.713 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T00:12:34.714 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T00:12:35.017 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:35.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:35 smithi012 ceph-mon[42503]: pgmap v844: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 3.0 MiB/s wr, 123 op/s 2024-05-09T00:12:36.129 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001920830, 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000427819, 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:36.130 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.131 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 3, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 337, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 480, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 288080, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 6, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 3, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.024612202, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.017924946, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.002567904, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 334, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.034056787, 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000101966 2024-05-09T00:12:36.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 288080 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 455, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 443, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1492272, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 253360, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-05-09T00:12:36.133 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.074227227, 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.026500010, 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.041082189, 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.010691224, 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 443, 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.037187121, 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000083943 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:36.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1492272, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 253360 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:36.135 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148731, 2024-05-09T00:12:36.137 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 31, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:36.138 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 454, 2024-05-09T00:12:36.139 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 1425580, 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 454, 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 454, 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1425580, 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:36.140 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:12:36.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:35 smithi204 ceph-mon[47373]: pgmap v844: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 3.0 MiB/s wr, 123 op/s 2024-05-09T00:12:36.267 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:35 smithi160 ceph-mon[46152]: pgmap v844: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 3.0 MiB/s wr, 123 op/s 2024-05-09T00:12:36.528 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-09T00:12:36.529 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-09T00:12:36.529 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-09T00:12:36.529 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-09T00:12:36.529 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-09T00:12:36.529 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T00:12:36.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T00:12:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:37.814 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-09T00:12:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4, 2024-05-09T00:12:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 4, 2024-05-09T00:12:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 2944, 2024-05-09T00:12:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 448, 2024-05-09T00:12:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-09T00:12:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.002429296, 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000063785, 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000867640, 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3, 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000942622, 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000314207 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 454, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 444, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1490144, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 254688, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.076075887, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.027777402, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.041723592, 2024-05-09T00:12:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.010823964, 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 443, 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.040043988, 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000090392 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1490144, 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 254688 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 3, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 341, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 480, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 288784, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 6, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 3, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.026165753, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.019414172, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.002686911, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 338, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.036426334, 2024-05-09T00:12:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000107770 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 288784 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148754, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 31, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-09T00:12:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-05-09T00:12:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 457, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 1425970, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 457, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-09T00:12:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-09T00:12:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 457, 2024-05-09T00:12:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1425970, 2024-05-09T00:12:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-09T00:12:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-09T00:12:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-09T00:12:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-09T00:12:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-09T00:12:37.825 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-09T00:12:37.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:37 smithi204 ceph-mon[47373]: pgmap v845: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-05-09T00:12:38.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:37 smithi012 ceph-mon[42503]: pgmap v845: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-05-09T00:12:38.186 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T00:12:38.186 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T00:12:38.186 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-09T00:12:38.186 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-09T00:12:38.186 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-09T00:12:38.186 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T00:12:38.186 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T00:12:38.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:37 smithi160 ceph-mon[46152]: pgmap v845: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-05-09T00:12:38.589 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:12:39.987 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T00:12:39.987 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:39.987 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 670393, 2024-05-09T00:12:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1017037, 2024-05-09T00:12:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 202261280, 2024-05-09T00:12:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1055712192, 2024-05-09T00:12:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 602, 2024-05-09T00:12:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 6, 2024-05-09T00:12:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 38.234663146, 2024-05-09T00:12:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 16.599711531, 2024-05-09T00:12:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 29.773698325, 2024-05-09T00:12:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 1.867265612, 2024-05-09T00:12:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:39.989 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1017032, 2024-05-09T00:12:39.989 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 244.273653610, 2024-05-09T00:12:39.989 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000240182 2024-05-09T00:12:39.989 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.989 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:39.989 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1316543, 2024-05-09T00:12:39.990 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.260436078, 2024-05-09T00:12:39.990 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000197 2024-05-09T00:12:39.990 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.990 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1915904, 2024-05-09T00:12:39.991 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 369568 2024-05-09T00:12:39.991 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.991 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:39.991 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 450781, 2024-05-09T00:12:39.991 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 444480, 2024-05-09T00:12:39.991 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 164691921, 2024-05-09T00:12:39.991 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 662384877, 2024-05-09T00:12:39.992 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 11, 2024-05-09T00:12:39.992 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 6, 2024-05-09T00:12:39.992 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 24.014011943, 2024-05-09T00:12:39.992 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 9.036646139, 2024-05-09T00:12:39.992 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 14.442614700, 2024-05-09T00:12:39.992 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 3.158884130, 2024-05-09T00:12:39.992 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:39.992 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 444481, 2024-05-09T00:12:39.992 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 170.960150609, 2024-05-09T00:12:39.993 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000384628 2024-05-09T00:12:39.993 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.993 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:39.993 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 681620, 2024-05-09T00:12:39.993 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.120400401, 2024-05-09T00:12:39.993 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000176 2024-05-09T00:12:39.993 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.993 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-09T00:12:39.993 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 758512 2024-05-09T00:12:39.994 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.994 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:39.994 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 357315, 2024-05-09T00:12:39.994 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 366329, 2024-05-09T00:12:39.994 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 138119085, 2024-05-09T00:12:39.994 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 638126647, 2024-05-09T00:12:39.994 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 23, 2024-05-09T00:12:39.995 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 6, 2024-05-09T00:12:39.995 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 22.703821620, 2024-05-09T00:12:39.995 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 10.006029523, 2024-05-09T00:12:39.995 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 12.067456819, 2024-05-09T00:12:39.995 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 2.739191018, 2024-05-09T00:12:39.995 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:39.996 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 366327, 2024-05-09T00:12:39.996 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 107.902758583, 2024-05-09T00:12:39.996 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000294553 2024-05-09T00:12:39.996 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.996 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:39.996 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 557310, 2024-05-09T00:12:39.996 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.093893384, 2024-05-09T00:12:39.996 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000168 2024-05-09T00:12:39.996 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.997 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 867872, 2024-05-09T00:12:39.997 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 279616 2024-05-09T00:12:39.997 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.997 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-09T00:12:39.997 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-09T00:12:39.997 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-09T00:12:39.997 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.997 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-09T00:12:39.997 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-09T00:12:39.998 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-09T00:12:39.998 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 196430, 2024-05-09T00:12:39.998 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 263.630027655, 2024-05-09T00:12:39.998 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001342106 2024-05-09T00:12:39.998 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:39.998 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:39.998 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-09T00:12:39.998 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-09T00:12:39.998 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-09T00:12:39.999 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 95439, 2024-05-09T00:12:39.999 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 25.657119644, 2024-05-09T00:12:39.999 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000268832 2024-05-09T00:12:39.999 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:40.000 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.000 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-09T00:12:40.000 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 466179, 2024-05-09T00:12:40.000 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 430153, 2024-05-09T00:12:40.000 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-09T00:12:40.000 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 430153, 2024-05-09T00:12:40.000 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2563.314389759, 2024-05-09T00:12:40.000 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.005959075 2024-05-09T00:12:40.000 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.001 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-09T00:12:40.002 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 36026, 2024-05-09T00:12:40.002 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 26, 2024-05-09T00:12:40.002 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 4, 2024-05-09T00:12:40.003 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 6961, 2024-05-09T00:12:40.003 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 37, 2024-05-09T00:12:40.003 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 41, 2024-05-09T00:12:40.004 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 5098, 2024-05-09T00:12:40.004 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 2435, 2024-05-09T00:12:40.005 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 2586, 2024-05-09T00:12:40.005 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 77, 2024-05-09T00:12:40.006 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 1587, 2024-05-09T00:12:40.006 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 2432, 2024-05-09T00:12:40.007 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 2, 2024-05-09T00:12:40.008 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 2, 2024-05-09T00:12:40.008 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 4, 2024-05-09T00:12:40.008 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 659888, 2024-05-09T00:12:40.008 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 416767, 2024-05-09T00:12:40.009 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 36010, 2024-05-09T00:12:40.009 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 83, 2024-05-09T00:12:40.009 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 4, 2024-05-09T00:12:40.009 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-09T00:12:40.010 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 4449, 2024-05-09T00:12:40.010 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 44, 2024-05-09T00:12:40.011 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-09T00:12:40.011 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 18, 2024-05-09T00:12:40.012 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 1484, 2024-05-09T00:12:40.012 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 32, 2024-05-09T00:12:40.012 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 985, 2024-05-09T00:12:40.014 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-09T00:12:40.014 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 8921, 2024-05-09T00:12:40.015 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-09T00:12:40.016 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 4, 2024-05-09T00:12:40.016 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 234, 2024-05-09T00:12:40.016 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 7, 2024-05-09T00:12:40.016 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 8921, 2024-05-09T00:12:40.016 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-09T00:12:40.018 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:12:40.018 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:12:40.018 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 1865, 2024-05-09T00:12:40.018 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 47, 2024-05-09T00:12:40.019 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 1818, 2024-05-09T00:12:40.019 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 7056, 2024-05-09T00:12:40.019 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 49, 2024-05-09T00:12:40.019 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 38991, 2024-05-09T00:12:40.019 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 249304, 2024-05-09T00:12:40.020 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:12:40.020 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 6055, 2024-05-09T00:12:40.020 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:12:40.020 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 115987, 2024-05-09T00:12:40.020 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 57698, 2024-05-09T00:12:40.021 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 11838, 2024-05-09T00:12:40.021 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 285691 2024-05-09T00:12:40.022 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.022 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-09T00:12:40.022 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 12, 2024-05-09T00:12:40.023 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 30, 2024-05-09T00:12:40.023 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 20, 2024-05-09T00:12:40.023 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 93, 2024-05-09T00:12:40.023 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 226, 2024-05-09T00:12:40.023 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 53, 2024-05-09T00:12:40.024 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-09T00:12:40.024 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:12:40.025 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 61590, 2024-05-09T00:12:40.025 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 61537, 2024-05-09T00:12:40.025 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-09T00:12:40.025 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-09T00:12:40.025 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-09T00:12:40.026 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:12:40.026 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:12:40.026 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-09T00:12:40.026 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-09T00:12:40.026 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 47, 2024-05-09T00:12:40.026 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 20, 2024-05-09T00:12:40.027 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-09T00:12:40.027 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 79, 2024-05-09T00:12:40.027 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-09T00:12:40.027 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-09T00:12:40.028 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.029 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-09T00:12:40.029 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 300330, 2024-05-09T00:12:40.030 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 297569, 2024-05-09T00:12:40.030 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 297569, 2024-05-09T00:12:40.030 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 2761, 2024-05-09T00:12:40.030 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-09T00:12:40.030 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 2681, 2024-05-09T00:12:40.031 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 493, 2024-05-09T00:12:40.031 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 489, 2024-05-09T00:12:40.031 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 489, 2024-05-09T00:12:40.032 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 4, 2024-05-09T00:12:40.032 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-09T00:12:40.032 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 4, 2024-05-09T00:12:40.034 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 1254520931, 2024-05-09T00:12:40.034 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 1263039027, 2024-05-09T00:12:40.034 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2024-05-09T00:12:40.034 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-09T00:12:40.035 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 108702, 2024-05-09T00:12:40.035 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1046.727214284, 2024-05-09T00:12:40.035 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.009629328 2024-05-09T00:12:40.035 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.035 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2024-05-09T00:12:40.035 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.036 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-09T00:12:40.036 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 2933, 2024-05-09T00:12:40.036 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 65833, 2024-05-09T00:12:40.036 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 62900, 2024-05-09T00:12:40.037 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 179, 2024-05-09T00:12:40.037 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 637, 2024-05-09T00:12:40.037 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 458, 2024-05-09T00:12:40.038 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 5098, 2024-05-09T00:12:40.038 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 137510, 2024-05-09T00:12:40.038 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 132412, 2024-05-09T00:12:40.038 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 2, 2024-05-09T00:12:40.041 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 66321, 2024-05-09T00:12:40.041 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 66319, 2024-05-09T00:12:40.041 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 92824, 2024-05-09T00:12:40.041 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 207156 2024-05-09T00:12:40.041 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.041 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-09T00:12:40.041 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 652510, 2024-05-09T00:12:40.041 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 21805, 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 466179, 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 9239, 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 43029, 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 62067, 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 301.406417178, 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004856146 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 35361, 2024-05-09T00:12:40.042 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 331.435401855, 2024-05-09T00:12:40.043 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.009372908 2024-05-09T00:12:40.043 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.043 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-09T00:12:40.043 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.043 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.043 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.043 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.043 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-09T00:12:40.043 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.043 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 111085, 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 416.440370997, 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.003748844 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.044 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.045 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-09T00:12:40.046 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.046 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.046 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.046 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.046 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-09T00:12:40.046 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.046 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.046 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.046 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.047 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-09T00:12:40.047 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 26, 2024-05-09T00:12:40.047 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.014614017, 2024-05-09T00:12:40.047 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000562077 2024-05-09T00:12:40.047 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.047 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-09T00:12:40.047 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 119, 2024-05-09T00:12:40.047 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.669372388, 2024-05-09T00:12:40.048 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.005624978 2024-05-09T00:12:40.048 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.048 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-09T00:12:40.048 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.048 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.048 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.048 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.048 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-09T00:12:40.048 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 26, 2024-05-09T00:12:40.049 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 10.782991479, 2024-05-09T00:12:40.049 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.414730441 2024-05-09T00:12:40.049 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.049 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-09T00:12:40.049 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 15, 2024-05-09T00:12:40.049 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.014732151, 2024-05-09T00:12:40.049 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000982143 2024-05-09T00:12:40.049 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.049 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-09T00:12:40.049 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 144524, 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1142.554349788, 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.007905637 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 13123, 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 63.583332163, 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004845182 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.050 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.051 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.051 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.051 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-09T00:12:40.051 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.051 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.051 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.051 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.051 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-09T00:12:40.051 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 64, 2024-05-09T00:12:40.051 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1.577005867, 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.024640716 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 19, 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 11.765193587, 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.619220715 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.052 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 156, 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 3.018248209, 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.019347744 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.053 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.054 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-09T00:12:40.054 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2024, 2024-05-09T00:12:40.054 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 5.479844548, 2024-05-09T00:12:40.054 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002707433 2024-05-09T00:12:40.054 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.054 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-09T00:12:40.054 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.054 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.054 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 7, 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.018718136, 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002674019 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 61537, 2024-05-09T00:12:40.055 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 274.553797396, 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004461605 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 2, 2024-05-09T00:12:40.056 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 5, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 3, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 2, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 31961, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 15980, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 1555, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:40.057 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:40.058 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-09T00:12:40.059 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 2343151, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 2192, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 264, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 3, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:40.060 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 8064, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 184, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 11442520, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 120489, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-09T00:12:40.061 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 222726, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 1323339975, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 37, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 222726, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 222726, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 7195.757931029, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.032307669 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 222726, 2024-05-09T00:12:40.062 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 300324 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 222726, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 10639, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 212087, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 6492, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 30071, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 1641, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 110296, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 1309, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-09T00:12:40.063 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 37, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 62873, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 8990, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 60142, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-09T00:12:40.064 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-09T00:12:40.065 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 140, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 60, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 12, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2024-05-09T00:12:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 13775, 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 60, 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 4638 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 1 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.067 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 10, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 61537, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 1478485, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 371868243, 2024-05-09T00:12:40.068 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 1478485, 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1478485, 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 371868243, 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-09T00:12:40.069 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 222726, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 1329570972, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 222726, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1329570972, 2024-05-09T00:12:40.070 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:40.071 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 222726, 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 222726, 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 222726, 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 222726, 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-09T00:12:40.072 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 61537, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 6231429, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 61537, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1594, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 6231429, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:40.073 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x55c944fe20a0": { 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 189, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 300330, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1258844723, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 300330, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 108702, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1258844534, 2024-05-09T00:12:40.074 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:40.075 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:40.075 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:40.075 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:40.075 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:40.075 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:40.075 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T00:12:40.138 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:39 smithi160 ceph-mon[46152]: pgmap v846: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-05-09T00:12:40.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:39 smithi012 ceph-mon[42503]: pgmap v846: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-05-09T00:12:40.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:39 smithi204 ceph-mon[47373]: pgmap v846: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-05-09T00:12:40.343 INFO:tasks.check_counter:Daemon mds.b mds.exported=18 2024-05-09T00:12:40.343 INFO:tasks.check_counter:Daemon mds.b mds.imported=32 2024-05-09T00:12:40.343 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-05-09T00:12:40.343 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=26 2024-05-09T00:12:40.343 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=19 2024-05-09T00:12:40.343 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T00:12:40.343 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T00:12:40.572 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:12:41.831 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T00:12:41.831 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:41.831 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 3810, 2024-05-09T00:12:41.831 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 3812, 2024-05-09T00:12:41.831 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 546380592, 2024-05-09T00:12:41.831 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1184917, 2024-05-09T00:12:41.831 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 8, 2024-05-09T00:12:41.831 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.617088293, 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.086911911, 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.403242928, 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.099894811, 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3810, 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.347722492, 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000091265 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.832 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 2087, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 2011, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 429675310, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 733696, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 6, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 5, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.428271471, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.053288118, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.313582432, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.059711020, 2024-05-09T00:12:41.833 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2010, 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.210125134, 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000104539 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1785296, 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 266512 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.834 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1475, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1811, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 461779893, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1204811, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 9, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 6, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.388968973, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.037739986, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.300919965, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.037410353, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1809, 2024-05-09T00:12:41.835 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.199281934, 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000110161 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 747952 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-09T00:12:41.836 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 11043, 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.333818223, 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000030228 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.837 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 0, 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 0, 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-09T00:12:41.838 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 0, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 0, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 48496, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 24485, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 24011, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 0, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 13374, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 0, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 0, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 0, 2024-05-09T00:12:41.839 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 1, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 0, 2024-05-09T00:12:41.840 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T00:12:41.841 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 0 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 0, 2024-05-09T00:12:41.842 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:12:41.843 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2024-05-09T00:12:41.844 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 234342, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 0, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 129, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 901791296, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 1438331003, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 1438313779, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.845 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 234342 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 2122, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 48429, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 46307, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 100, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 276, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 176, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 48496, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 48964, 2024-05-09T00:12:41.846 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 468, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 0, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 0, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 0, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 103288, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 207156 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 0, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 0, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 0, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 0, 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-09T00:12:41.847 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.848 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.849 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.850 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.851 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.852 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.853 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.854 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.855 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.856 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.857 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-09T00:12:41.858 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 2, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 2, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 0, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 2, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 0, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 0, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 3057, 2024-05-09T00:12:41.859 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:41.860 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:41.861 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 320949, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 2434, 2024-05-09T00:12:41.862 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 8064, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 184, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-09T00:12:41.863 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 39482167, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 252277, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 6781, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 21, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 3, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 6781, 2024-05-09T00:12:41.864 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 6781, 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 20.161688532, 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002973261 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 6781, 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 6782 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 6781, 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 6781, 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 0, 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-09T00:12:41.865 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 3692, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 3087, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-09T00:12:41.866 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 1, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-09T00:12:41.867 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-09T00:12:41.868 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 140, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 59, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 11, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 1, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 0, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 2, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 0 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 0, 2024-05-09T00:12:41.869 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 0, 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 0, 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-09T00:12:41.870 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 7368, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1437140338, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 7368, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 7368, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1437140338, 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:41.871 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:41.872 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 6781, 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 1434187687, 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 6781, 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1434187687, 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:41.873 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 6781, 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 6781, 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 6781, 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 6781, 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.874 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-05-09T00:12:41.875 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x562f0aedc5a0": { 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:41.876 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:41.877 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T00:12:42.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:41 smithi012 ceph-mon[42503]: pgmap v847: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.3 MiB/s wr, 100 op/s 2024-05-09T00:12:42.190 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:41 smithi160 ceph-mon[46152]: pgmap v847: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.3 MiB/s wr, 100 op/s 2024-05-09T00:12:42.192 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-09T00:12:42.192 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-09T00:12:42.192 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-05-09T00:12:42.192 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-09T00:12:42.192 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T00:12:42.192 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T00:12:42.193 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T00:12:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:41 smithi204 ceph-mon[47373]: pgmap v847: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.3 MiB/s wr, 100 op/s 2024-05-09T00:12:42.420 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:12:43.402 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 512792, 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 779432, 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 154627936, 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1237128281, 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 20, 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 6, 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 30.742249443, 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 13.273822400, 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 23.719358668, 2024-05-09T00:12:43.403 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 2.117600208, 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 779427, 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 174.257415915, 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000223571 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1016399, 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.206983695, 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000203 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.404 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 640160 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 334663, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 324179, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 121867296, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 579095428, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 98, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 9, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 20.124857503, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 9.069012346, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 11.984516511, 2024-05-09T00:12:43.405 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 1.590219564, 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 324177, 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 52.199185898, 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000161020 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 553094, 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.085848027, 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000155 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3328, 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 126224 2024-05-09T00:12:43.406 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 271467, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 267918, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 117706529, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 493716837, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 9, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 5, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 14.261157602, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 5.586047788, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 9.204317839, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 1.382924563, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 267920, 2024-05-09T00:12:43.407 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 67.470475776, 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000251830 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 441935, 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.072035191, 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000162 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1816272, 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 313040 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-09T00:12:43.408 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 119483, 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 183.409008895, 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001535021 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-09T00:12:43.409 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 70952, 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 18.115953701, 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000255326 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 363745, 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 348171, 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 348171, 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1629.910620697, 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004681350 2024-05-09T00:12:43.410 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 15574, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 13, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 299, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 23, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 23, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 5077, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 2293, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 2122, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 662, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 1801, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 714, 2024-05-09T00:12:43.411 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 0, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 0, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 4, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 507135, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 312993, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 15567, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 50, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 3650, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 13, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 416, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 6, 2024-05-09T00:12:43.412 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 682, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 5606, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 4, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 234, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 7, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 5606, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 630, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 630, 2024-05-09T00:12:43.413 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 4976, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 8, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 17350, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 200860, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 5774, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 84350, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 47056, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 9921, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 251182 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 68, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 172, 2024-05-09T00:12:43.414 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 96, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 211, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 150, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 30, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 45923, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 45893, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-09T00:12:43.415 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 32, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 46, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 234360, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 180990, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 180990, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 53370, 2024-05-09T00:12:43.416 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 556, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 398, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 270, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 270, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 129, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 1, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 901791296, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 1438369990, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 66830, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 579.364968178, 2024-05-09T00:12:43.417 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.008669234 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 2124, 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 48541, 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 46417, 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 137, 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 556, 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 419, 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 5062, 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 97813, 2024-05-09T00:12:43.418 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 92751, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 0, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 51800, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 51800, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 89356, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 198964 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 513989, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 858, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 363745, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 6683, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 6523, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 45706, 2024-05-09T00:12:43.419 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 184.937380867, 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004046238 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1499, 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 15.385635783, 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.010263933 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-09T00:12:43.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 93090, 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 240.569363023, 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002584266 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.421 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.422 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.424 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 151250, 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 985.009240951, 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.006512457 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 9282, 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 42.595092586, 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004588999 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-09T00:12:43.425 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 30, 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.142223272, 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004740775 2024-05-09T00:12:43.426 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 73, 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1.145624510, 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.015693486 2024-05-09T00:12:43.427 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1348, 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2.899968405, 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002151311 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.428 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 45893, 2024-05-09T00:12:43.429 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 157.226091300, 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.003425927 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 2, 2024-05-09T00:12:43.430 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 2, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 0, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 2, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 31179, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 15589, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 1506, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:43.431 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:43.432 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:43.433 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 763768, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 900, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 528, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 6, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-09T00:12:43.434 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 8064, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 184, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 8961599, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 78639, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-09T00:12:43.435 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 123581, 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 1445332392, 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 33, 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 123581, 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 123581, 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1186.255685071, 2024-05-09T00:12:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.009599013 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 123581, 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 124754 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 123581, 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 7105, 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 116476, 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 81, 2024-05-09T00:12:43.437 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 79, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 1486, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 68316, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 885, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 23, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 46580, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 5607, 2024-05-09T00:12:43.438 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 158, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-09T00:12:43.439 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-09T00:12:43.440 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 140, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 59, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 12, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 1151, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 26, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 362 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2024-05-09T00:12:43.441 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 4 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 10, 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 45893, 2024-05-09T00:12:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 1118918, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 293460556, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 1118918, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1118918, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 293460556, 2024-05-09T00:12:43.443 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:43.444 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 123581, 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 1449981601, 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 123581, 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1449981601, 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-09T00:12:43.445 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 123581, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 123581, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 123581, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 123581, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.446 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 45893, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 4649417, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 45893, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1486, 2024-05-09T00:12:43.447 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 4649417, 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x55afe7e7e0a0": { 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 599, 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 234360, 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1434175686, 2024-05-09T00:12:43.448 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 234360, 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 66830, 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1434175087, 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:43.449 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T00:12:43.767 INFO:tasks.check_counter:Daemon mds.h mds.exported=13 2024-05-09T00:12:43.767 INFO:tasks.check_counter:Daemon mds.h mds.imported=6 2024-05-09T00:12:43.767 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-05-09T00:12:43.768 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-09T00:12:43.768 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T00:12:43.768 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T00:12:43.768 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T00:12:43.999 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:12:44.132 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:43 smithi160 ceph-mon[46152]: pgmap v848: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-05-09T00:12:44.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:43 smithi012 ceph-mon[42503]: pgmap v848: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-05-09T00:12:44.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:43 smithi204 ceph-mon[47373]: pgmap v848: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-05-09T00:12:44.937 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T00:12:44.937 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:44.937 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 76, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 75, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 159792, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 41744, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 0, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.011367936, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.002696731, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.006094488, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.001622249, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 74, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.021194939, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000286418 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 159792, 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 41744 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1, 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 57, 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 160, 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 86112, 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 3, 2024-05-09T00:12:44.939 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 2, 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.003384607, 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.001737503, 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.001161608, 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 55, 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.012834214, 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000233349 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:44.940 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 86112 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 4, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 2944, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 448, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 2, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.000987115, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.000058397, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000901051, 2024-05-09T00:12:44.941 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3, 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001339560, 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000446520 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.942 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 0, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:44.943 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:44.944 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 148801, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 27, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-09T00:12:44.945 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 928, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:44.946 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 77, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 149087, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 77, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T00:12:44.947 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T00:12:44.948 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 77, 2024-05-09T00:12:44.948 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 149087, 2024-05-09T00:12:44.948 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T00:12:44.948 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T00:12:44.948 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T00:12:44.948 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T00:12:44.948 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:44.948 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T00:12:44.948 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T00:12:45.308 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-09T00:12:45.309 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-09T00:12:45.309 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-09T00:12:45.309 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-09T00:12:45.309 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-09T00:12:45.309 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T00:12:45.310 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T00:12:45.691 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:12:46.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:45 smithi012 ceph-mon[42503]: pgmap v849: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 747 KiB/s wr, 75 op/s 2024-05-09T00:12:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:45 smithi204 ceph-mon[47373]: pgmap v849: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 747 KiB/s wr, 75 op/s 2024-05-09T00:12:46.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:45 smithi160 ceph-mon[46152]: pgmap v849: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 747 KiB/s wr, 75 op/s 2024-05-09T00:12:47.496 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-09T00:12:47.496 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:47.496 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 4, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 4, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2944, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 448, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 0, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.001744935, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.000039473, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000444169, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.002041135, 2024-05-09T00:12:47.497 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000680378 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 169, 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 168, 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 186560, 2024-05-09T00:12:47.498 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 95136, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 2, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.018307456, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.005823690, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.007864783, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.003644808, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 167, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.022721395, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000136056 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:47.499 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 186560, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 95136 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 127, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 160, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 98432, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 3, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.005614644, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.003816319, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000895202, 2024-05-09T00:12:47.500 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 125, 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.028840060, 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000230720 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 98432 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.501 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 0, 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:47.502 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:47.503 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 149966, 2024-05-09T00:12:47.504 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 29, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:47.505 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 928, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 170, 2024-05-09T00:12:47.506 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 162682, 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 170, 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 170, 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 162682, 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:47.507 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-09T00:12:47.860 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.c 2024-05-09T00:12:47.860 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.c 2024-05-09T00:12:47.860 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-05-09T00:12:47.860 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-05-09T00:12:47.860 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-05-09T00:12:47.861 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T00:12:47.861 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T00:12:48.091 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:12:48.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:47 smithi012 ceph-mon[42503]: pgmap v850: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 698 KiB/s wr, 72 op/s 2024-05-09T00:12:48.204 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:47 smithi204 ceph-mon[47373]: pgmap v850: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 698 KiB/s wr, 72 op/s 2024-05-09T00:12:48.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:47 smithi160 ceph-mon[46152]: pgmap v850: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 698 KiB/s wr, 72 op/s 2024-05-09T00:12:49.082 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-09T00:12:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1901, 2024-05-09T00:12:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1825, 2024-05-09T00:12:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 396414215, 2024-05-09T00:12:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 676075, 2024-05-09T00:12:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 7, 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.427924946, 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.049724099, 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.281781848, 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.093684084, 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1824, 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.182798538, 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000100218 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1785568, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 267104 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2173, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2511, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 692687630, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1422690, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 9, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 7, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.465171047, 2024-05-09T00:12:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.056420602, 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.333212059, 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.058553980, 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2508, 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.216040849, 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000086140 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 747504 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 3494, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 3494, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 576956634, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1085896, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.586969729, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.075212780, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.386364607, 2024-05-09T00:12:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.097461362, 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3493, 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.337159394, 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000096524 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:12:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 11304, 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.366109013, 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000032387 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-05-09T00:12:49.089 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 59352, 2024-05-09T00:12:49.090 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 32312, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 27040, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 13098, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:12:49.091 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-05-09T00:12:49.092 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T00:12:49.093 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-05-09T00:12:49.094 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-05-09T00:12:49.095 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 275153, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-05-09T00:12:49.096 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 129, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 1132478606, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 1666434939, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 1666402840, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 275153 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.097 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 2175, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 59340, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 57165, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 97, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 342, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 245, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 59352, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 60023, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 671, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 109348, 2024-05-09T00:12:49.098 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 207156 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.099 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-05-09T00:12:49.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.100 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.100 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.100 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-05-09T00:12:49.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.102 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.103 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.104 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.105 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.106 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.108 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.109 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.109 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-05-09T00:12:49.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.109 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.109 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.109 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-05-09T00:12:49.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.109 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.110 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.110 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-05-09T00:12:49.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.110 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.110 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.110 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-05-09T00:12:49.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.110 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 1, 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 3, 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 3791, 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.112 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:49.113 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:49.114 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 322257, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 2493, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-05-09T00:12:49.115 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8064, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 184, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 47259610, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 295468, 2024-05-09T00:12:49.116 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:49.117 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-05-09T00:12:49.117 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:49.117 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-05-09T00:12:49.117 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.117 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-05-09T00:12:49.117 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-05-09T00:12:49.117 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 6976, 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 6976, 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 6976, 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 21.203706941, 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003039522 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 6976, 2024-05-09T00:12:49.118 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 6977 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 6976, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 6976, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 3780, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 3194, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-05-09T00:12:49.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:12:49.120 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-05-09T00:12:49.121 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 140, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 59, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 11, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 11, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-05-09T00:12:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T00:12:49.123 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 7564, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1665345330, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 7564, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-09T00:12:49.124 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 7564, 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1665345330, 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-05-09T00:12:49.125 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 6976, 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 1662360617, 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 6976, 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1662360617, 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.126 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 6976, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 6976, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 6976, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 6976, 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:49.127 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-09T00:12:49.128 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x555597c465a0": { 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-05-09T00:12:49.129 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:49.130 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-09T00:12:49.452 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-09T00:12:49.452 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-09T00:12:49.452 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-05-09T00:12:49.452 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-09T00:12:49.452 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T00:12:49.452 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T00:12:49.453 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T00:12:49.681 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:12:50.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:49 smithi012 ceph-mon[42503]: pgmap v851: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 476 KiB/s wr, 41 op/s 2024-05-09T00:12:50.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:49 smithi204 ceph-mon[47373]: pgmap v851: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 476 KiB/s wr, 41 op/s 2024-05-09T00:12:50.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:49 smithi160 ceph-mon[46152]: pgmap v851: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 476 KiB/s wr, 41 op/s 2024-05-09T00:12:50.796 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-09T00:12:50.796 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:50.796 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 392097, 2024-05-09T00:12:50.796 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 398820, 2024-05-09T00:12:50.796 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 137950379, 2024-05-09T00:12:50.796 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 675285997, 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 11, 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 19.905095316, 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 7.943644363, 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 12.525962769, 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 1.748445732, 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 398818, 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 40.431548865, 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000101378 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.797 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 674755, 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.098828341, 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000146 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 5088, 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 30256 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 47475, 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 47720, 2024-05-09T00:12:50.798 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 14084571, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 505249231, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 99, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 10, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 4.300809517, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 1.493524047, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.795999486, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 1.788574264, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 47720, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 5.085729010, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000106574 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:50.799 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 92, 2024-05-09T00:12:50.800 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000007382, 2024-05-09T00:12:50.800 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000080 2024-05-09T00:12:50.800 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.800 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1811280, 2024-05-09T00:12:50.800 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 956544 2024-05-09T00:12:50.800 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.800 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:50.800 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 826392, 2024-05-09T00:12:50.800 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1152052, 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 283588091, 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1442245275, 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 18, 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 42.261017372, 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 18.183757772, 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 30.948841101, 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 2.206753887, 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1152049, 2024-05-09T00:12:50.801 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 146.236938878, 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000126936 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1635108, 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.259256402, 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000158 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 111472 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-05-09T00:12:50.802 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 138663, 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 180.251555477, 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001299925 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 94949, 2024-05-09T00:12:50.803 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 16.094692248, 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000169508 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 417275, 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 398527, 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 398527, 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1826.367429726, 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.004582794 2024-05-09T00:12:50.804 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.805 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-05-09T00:12:50.805 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 18748, 2024-05-09T00:12:50.805 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 2, 2024-05-09T00:12:50.805 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-05-09T00:12:50.805 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 385, 2024-05-09T00:12:50.805 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 30, 2024-05-09T00:12:50.805 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 31, 2024-05-09T00:12:50.805 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 5470, 2024-05-09T00:12:50.805 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 2690, 2024-05-09T00:12:50.805 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 2395, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 385, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 1626, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 1048, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 4, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 5, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 4, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 630120, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 368813, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 18733, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 116, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 5442, 2024-05-09T00:12:50.806 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 1000, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 1, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 23, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 631, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 16, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 864, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 5682, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 4, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 7, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 5682, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-05-09T00:12:50.807 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 630, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 630, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 5052, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 6, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 26815, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 237447, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 5912, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 96395, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 48139, 2024-05-09T00:12:50.808 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 11101, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 295413 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 74, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 154, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 86, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 246, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 174, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 30, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 56711, 2024-05-09T00:12:50.809 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 56681, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 28, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 61, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-05-09T00:12:50.810 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 275154, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 222147, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 222147, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 53007, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 419, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 464, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 336, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 336, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 129, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 1, 2024-05-09T00:12:50.811 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 1132478606, 2024-05-09T00:12:50.812 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 1666435538, 2024-05-09T00:12:50.812 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-05-09T00:12:50.812 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-05-09T00:12:50.812 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 76319, 2024-05-09T00:12:50.812 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 611.671517179, 2024-05-09T00:12:50.812 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.008014668 2024-05-09T00:12:50.812 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.812 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-05-09T00:12:50.812 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.812 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 2196, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 59480, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 57284, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 150, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 655, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 505, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 5470, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 113575, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 108105, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 4, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 62453, 2024-05-09T00:12:50.813 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 62449, 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 94552, 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 198964 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 617073, 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 257, 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 417275, 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 6812, 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 16223, 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 56342, 2024-05-09T00:12:50.814 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 209.013054905, 2024-05-09T00:12:50.815 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003709720 2024-05-09T00:12:50.815 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.815 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-05-09T00:12:50.815 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 257, 2024-05-09T00:12:50.815 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 4.900188450, 2024-05-09T00:12:50.815 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.019066881 2024-05-09T00:12:50.815 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.815 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-05-09T00:12:50.815 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.815 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 112488, 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 232.216494196, 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002064366 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-05-09T00:12:50.816 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000249353, 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000124676 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-05-09T00:12:50.817 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.818 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.818 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.818 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.818 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-05-09T00:12:50.818 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.818 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.818 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.819 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 158870, 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 992.893916804, 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.006249725 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 11907, 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 217.944610050, 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.018303906 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-05-09T00:12:50.821 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 30, 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.314208417, 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.010473613 2024-05-09T00:12:50.822 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 50, 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.805621416, 2024-05-09T00:12:50.823 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.016112428 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1900, 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.799497047, 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001473419 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-05-09T00:12:50.824 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.825 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.826 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.826 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-05-09T00:12:50.826 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 56681, 2024-05-09T00:12:50.826 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 165.479589088, 2024-05-09T00:12:50.826 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002919489 2024-05-09T00:12:50.826 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.827 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:12:50.827 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:12:50.827 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-05-09T00:12:50.827 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.827 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.827 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.827 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:50.827 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 1, 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 2, 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 34317, 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 17158, 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 1566, 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-05-09T00:12:50.828 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:50.829 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:50.830 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:50.830 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:50.830 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:50.830 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:50.830 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:50.830 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:50.830 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:50.830 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:50.831 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:50.832 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:50.832 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:50.832 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:50.833 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:50.833 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:50.834 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:50.834 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:50.835 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:50.835 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:50.836 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:50.837 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:50.837 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-05-09T00:12:50.837 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:50.839 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:50.839 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:50.839 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:50.839 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 654106, 2024-05-09T00:12:50.839 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 951, 2024-05-09T00:12:50.839 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 352, 2024-05-09T00:12:50.840 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 4, 2024-05-09T00:12:50.840 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-05-09T00:12:50.840 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8064, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 184, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-05-09T00:12:50.841 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 9413083, 2024-05-09T00:12:50.842 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 84081, 2024-05-09T00:12:50.842 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:50.842 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-05-09T00:12:50.842 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:50.842 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-05-09T00:12:50.842 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.842 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-05-09T00:12:50.842 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-05-09T00:12:50.842 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-05-09T00:12:50.842 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 144755, 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 1675196274, 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 27, 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 144755, 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 144755, 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1308.250961035, 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.009037691 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 144755, 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 146096 2024-05-09T00:12:50.843 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.844 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 144755, 2024-05-09T00:12:50.844 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 7384, 2024-05-09T00:12:50.844 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 137371, 2024-05-09T00:12:50.844 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-05-09T00:12:50.844 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-05-09T00:12:50.844 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 103, 2024-05-09T00:12:50.844 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 85, 2024-05-09T00:12:50.845 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 1700, 2024-05-09T00:12:50.845 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 78019, 2024-05-09T00:12:50.845 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 1014, 2024-05-09T00:12:50.845 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-05-09T00:12:50.845 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-05-09T00:12:50.845 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-05-09T00:12:50.845 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 30, 2024-05-09T00:12:50.845 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 57538, 2024-05-09T00:12:50.845 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-05-09T00:12:50.845 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 5684, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 170, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:12:50.846 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-05-09T00:12:50.847 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 140, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 59, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 12, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 12, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 1337, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 4, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 412 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.848 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-05-09T00:12:50.849 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-05-09T00:12:50.849 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:12:50.849 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-05-09T00:12:50.849 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-05-09T00:12:50.849 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.849 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-05-09T00:12:50.849 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-05-09T00:12:50.849 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.849 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-05-09T00:12:50.849 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 10, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 56681, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 1265960, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 321648128, 2024-05-09T00:12:50.850 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 1265960, 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 1265960, 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 321648128, 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:50.851 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 144755, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 1680936799, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 144755, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1680936799, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.852 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 144755, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 144755, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 144755, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 144755, 2024-05-09T00:12:50.853 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 56681, 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 5740557, 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:50.854 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 56681, 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 1700, 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 5740557, 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x5632963860a0": { 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 599, 2024-05-09T00:12:50.855 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 275154, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1662241234, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 275154, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 76319, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1662240635, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:50.857 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:50.857 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-09T00:12:51.133 INFO:tasks.check_counter:Daemon mds.i mds.exported=23 2024-05-09T00:12:51.134 INFO:tasks.check_counter:Daemon mds.i mds.imported=16 2024-05-09T00:12:51.134 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-05-09T00:12:51.134 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-09T00:12:51.134 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T00:12:51.134 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T00:12:51.134 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T00:12:51.359 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:12:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:51 smithi204 ceph-mon[47373]: pgmap v852: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 58 KiB/s wr, 11 op/s 2024-05-09T00:12:52.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:51 smithi012 ceph-mon[42503]: pgmap v852: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 58 KiB/s wr, 11 op/s 2024-05-09T00:12:52.269 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:51 smithi160 ceph-mon[46152]: pgmap v852: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 58 KiB/s wr, 11 op/s 2024-05-09T00:12:52.329 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 169, 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 169, 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 16833673, 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 51773, 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 7, 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.025915868, 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.003839810, 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.014426532, 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.005149898, 2024-05-09T00:12:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 168, 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.021441580, 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000127628 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.331 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 172, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 192, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 29959193, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 170191, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.032161685, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.003961771, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.021766835, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.005042150, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 190, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.027258635, 2024-05-09T00:12:52.332 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000143466 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 116944 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 78, 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 73, 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 8563679, 2024-05-09T00:12:52.333 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 27757, 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.011950683, 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.002256396, 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.009158004, 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.001353788, 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 72, 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.014441660, 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000200578 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:12:52.334 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 176672, 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 16560 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-05-09T00:12:52.335 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 600, 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.043589704, 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000072649 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.336 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-05-09T00:12:52.337 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 4298, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 1627, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 2671, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 1974, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 2, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-05-09T00:12:52.338 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 4, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-05-09T00:12:52.339 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 7, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T00:12:52.340 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-05-09T00:12:52.341 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-05-09T00:12:52.342 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 11810, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 4, 2024-05-09T00:12:52.343 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 1254520931, 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 1263039216, 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 1263039027, 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 11810 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 714, 2024-05-09T00:12:52.344 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 3546, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 2832, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 152, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 164, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 12, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 4298, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 4298, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 55512, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 207132 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.345 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.346 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.347 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.348 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-05-09T00:12:52.349 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.350 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.351 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-05-09T00:12:52.352 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.353 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.354 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.355 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.355 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-05-09T00:12:52.355 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.355 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.355 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.355 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.355 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-05-09T00:12:52.357 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.357 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-05-09T00:12:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 2, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 0, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 0, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 3054, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-05-09T00:12:52.360 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-05-09T00:12:52.361 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-05-09T00:12:52.362 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 258363, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 1087, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 88, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 1, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-05-09T00:12:52.363 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8064, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 184, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 5172009, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 44387, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-05-09T00:12:52.364 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 372, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 26, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 4, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 372, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 372, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.323982085, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003559091 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-05-09T00:12:52.365 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 372, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 373 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 372, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 372, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 203, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 166, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-05-09T00:12:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 2, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-05-09T00:12:52.367 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-05-09T00:12:52.368 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 140, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 2, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 8, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 8, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-05-09T00:12:52.369 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-05-09T00:12:52.370 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 415, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 55314125, 2024-05-09T00:12:52.371 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 415, 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 415, 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 55314125, 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-05-09T00:12:52.372 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 372, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 55068161, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 372, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 55068161, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.373 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:52.374 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 372, 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 372, 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 372, 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 372, 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-05-09T00:12:52.375 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:52.376 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x5583b5bb6640": { 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-05-09T00:12:52.377 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-05-09T00:12:52.378 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-05-09T00:12:52.378 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-05-09T00:12:52.378 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-05-09T00:12:52.378 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:52.378 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-05-09T00:12:52.378 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-09T00:12:52.684 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-09T00:12:52.684 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-09T00:12:52.684 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=7 2024-05-09T00:12:52.684 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-09T00:12:52.684 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T00:12:52.685 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T00:12:52.696 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T00:12:52.696 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T00:12:52.696 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T00:12:52.721 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-09T00:12:52.721 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T00:12:52.785 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T00:12:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.980+0000 7f9d82c4f700 1 -- 172.21.15.12:0/1156437666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c106180 msgr2=0x7f9d7c10dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.980+0000 7f9d82c4f700 1 --2- 172.21.15.12:0/1156437666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c106180 0x7f9d7c10dd00 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400b0a0 tx=0x7f9d7400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.981+0000 7f9d82c4f700 1 -- 172.21.15.12:0/1156437666 shutdown_connections 2024-05-09T00:12:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.981+0000 7f9d82c4f700 1 --2- 172.21.15.12:0/1156437666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c106180 0x7f9d7c10dd00 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.981+0000 7f9d82c4f700 1 --2- 172.21.15.12:0/1156437666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d7c101060 0x7f9d7c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.981+0000 7f9d82c4f700 1 --2- 172.21.15.12:0/1156437666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d7c101a10 0x7f9d7c105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.981+0000 7f9d82c4f700 1 -- 172.21.15.12:0/1156437666 >> 172.21.15.12:0/1156437666 conn(0x7f9d7c078360 msgr2=0x7f9d7c078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.981+0000 7f9d82c4f700 1 -- 172.21.15.12:0/1156437666 shutdown_connections 2024-05-09T00:12:52.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.981+0000 7f9d82c4f700 1 -- 172.21.15.12:0/1156437666 wait complete. 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.981+0000 7f9d82c4f700 1 Processor -- start 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d82c4f700 1 -- start start 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d82c4f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d7c101060 0x7f9d7c197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d82c4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c106180 0x7f9d7c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d82c4f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d7c19bca0 0x7f9d7c19c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d82c4f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c19c720 con 0x7f9d7c106180 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d82c4f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c19c890 con 0x7f9d7c101060 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d82c4f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c19ca50 con 0x7f9d7c19bca0 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d7bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c106180 0x7f9d7c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d7bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c106180 0x7f9d7c197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33152/0 (socket says 172.21.15.12:33152) 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d7bfff700 1 -- 172.21.15.12:0/3661871036 learned_addr learned my addr 172.21.15.12:0/3661871036 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:52.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d811ec700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d7c19bca0 0x7f9d7c19c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d809eb700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d7c101060 0x7f9d7c197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d7bfff700 1 -- 172.21.15.12:0/3661871036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d7c19bca0 msgr2=0x7f9d7c19c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d7bfff700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d7c19bca0 0x7f9d7c19c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d7bfff700 1 -- 172.21.15.12:0/3661871036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d7c101060 msgr2=0x7f9d7c197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d7bfff700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d7c101060 0x7f9d7c197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.982+0000 7f9d7bfff700 1 -- 172.21.15.12:0/3661871036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d74009d00 con 0x7f9d7c106180 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.983+0000 7f9d7bfff700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c106180 0x7f9d7c197950 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800e970 tx=0x7f9d6800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.983+0000 7f9d79ffb700 1 -- 172.21.15.12:0/3661871036 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68009790 con 0x7f9d7c106180 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.983+0000 7f9d79ffb700 1 -- 172.21.15.12:0/3661871036 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d6800eec0 con 0x7f9d7c106180 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.983+0000 7f9d82c4f700 1 -- 172.21.15.12:0/3661871036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c1a3fb0 con 0x7f9d7c106180 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.983+0000 7f9d79ffb700 1 -- 172.21.15.12:0/3661871036 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68005590 con 0x7f9d7c106180 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.983+0000 7f9d82c4f700 1 -- 172.21.15.12:0/3661871036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c1a4430 con 0x7f9d7c106180 2024-05-09T00:12:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.984+0000 7f9d82c4f700 1 -- 172.21.15.12:0/3661871036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c109450 con 0x7f9d7c106180 2024-05-09T00:12:52.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.985+0000 7f9d79ffb700 1 -- 172.21.15.12:0/3661871036 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9d6800cbd0 con 0x7f9d7c106180 2024-05-09T00:12:52.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.986+0000 7f9d79ffb700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d6c0a1290 0x7f9d6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.986+0000 7f9d79ffb700 1 -- 172.21.15.12:0/3661871036 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9d68014070 con 0x7f9d7c106180 2024-05-09T00:12:52.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.986+0000 7f9d809eb700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d6c0a1290 0x7f9d6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.986+0000 7f9d809eb700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d6c0a1290 0x7f9d6c0a3750 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009bf0 tx=0x7f9d70009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:52.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:52.991+0000 7f9d79ffb700 1 -- 172.21.15.12:0/3661871036 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d68096330 con 0x7f9d7c106180 2024-05-09T00:12:53.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.208+0000 7f9d82c4f700 1 -- 172.21.15.12:0/3661871036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d7c102530 con 0x7f9d7c106180 2024-05-09T00:12:53.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.210+0000 7f9d79ffb700 1 -- 172.21.15.12:0/3661871036 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+22978 (secure 0 0 0) 0x7f9d68004ef0 con 0x7f9d7c106180 2024-05-09T00:12:53.208 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:12:53.208 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":140,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-09T00:12:16.031662+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":139,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3622028746":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3062429456":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2381834553":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/716473668":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2695290437":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/1305168755":"2024-05-09T23:46:25.291064+0000","172.21.15.12:6801/1305168755":"2024-05-09T23:46:25.291064+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":33,"length":1},{"begin":46,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:12:53.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 -- 172.21.15.12:0/3661871036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d6c0a1290 msgr2=0x7f9d6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d6c0a1290 0x7f9d6c0a3750 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009bf0 tx=0x7f9d70009260 comp rx=0 tx=0).stop 2024-05-09T00:12:53.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 -- 172.21.15.12:0/3661871036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c106180 msgr2=0x7f9d7c197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c106180 0x7f9d7c197950 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800e970 tx=0x7f9d6800ec80 comp rx=0 tx=0).stop 2024-05-09T00:12:53.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 -- 172.21.15.12:0/3661871036 shutdown_connections 2024-05-09T00:12:53.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f9d6c0a1290 0x7f9d6c0a3750 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d7c19bca0 0x7f9d7c19c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d7c101060 0x7f9d7c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 --2- 172.21.15.12:0/3661871036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c106180 0x7f9d7c197950 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 -- 172.21.15.12:0/3661871036 >> 172.21.15.12:0/3661871036 conn(0x7f9d7c078360 msgr2=0x7f9d7c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:53.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 -- 172.21.15.12:0/3661871036 shutdown_connections 2024-05-09T00:12:53.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.212+0000 7f9d82c4f700 1 -- 172.21.15.12:0/3661871036 wait complete. 2024-05-09T00:12:53.225 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:41:47.730840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:45:02.165326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:45:03.407498+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 139, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T00:12:53.225 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T00:12:53.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.371+0000 7f4846717700 1 -- 172.21.15.12:0/2869808360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48401000f0 msgr2=0x7f4840100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.371+0000 7f4846717700 1 --2- 172.21.15.12:0/2869808360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48401000f0 0x7f4840100570 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f48340099c0 tx=0x7f4834009cd0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.371+0000 7f4846717700 1 -- 172.21.15.12:0/2869808360 shutdown_connections 2024-05-09T00:12:53.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.371+0000 7f4846717700 1 --2- 172.21.15.12:0/2869808360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4840100ab0 0x7f484010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.371+0000 7f4846717700 1 --2- 172.21.15.12:0/2869808360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4840105f90 0x7f4840106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.371+0000 7f4846717700 1 --2- 172.21.15.12:0/2869808360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f48401000f0 0x7f4840100570 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.371+0000 7f4846717700 1 -- 172.21.15.12:0/2869808360 >> 172.21.15.12:0/2869808360 conn(0x7f4840076ba0 msgr2=0x7f4840076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:53.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.371+0000 7f4846717700 1 -- 172.21.15.12:0/2869808360 shutdown_connections 2024-05-09T00:12:53.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.371+0000 7f4846717700 1 -- 172.21.15.12:0/2869808360 wait complete. 2024-05-09T00:12:53.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4846717700 1 Processor -- start 2024-05-09T00:12:53.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4846717700 1 -- start start 2024-05-09T00:12:53.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4846717700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48401000f0 0x7f4840198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:53.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4846717700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4840100ab0 0x7f48401986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4846717700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4840105f90 0x7f48401a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4846717700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4840198f00 con 0x7f4840105f90 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4846717700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48401a3730 con 0x7f48401000f0 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4846717700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48401a3870 con 0x7f4840100ab0 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4844cb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4840105f90 0x7f48401a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4844cb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4840105f90 0x7f48401a31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33166/0 (socket says 172.21.15.12:33166) 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f4844cb4700 1 -- 172.21.15.12:0/968423502 learned_addr learned my addr 172.21.15.12:0/968423502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f483f7fe700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4840100ab0 0x7f48401986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.372+0000 7f483ffff700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48401000f0 0x7f4840198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f4844cb4700 1 -- 172.21.15.12:0/968423502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4840100ab0 msgr2=0x7f48401986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f4844cb4700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4840100ab0 0x7f48401986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f4844cb4700 1 -- 172.21.15.12:0/968423502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48401000f0 msgr2=0x7f4840198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f4844cb4700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48401000f0 0x7f4840198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f4844cb4700 1 -- 172.21.15.12:0/968423502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340095f0 con 0x7f4840105f90 2024-05-09T00:12:53.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f483ffff700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48401000f0 0x7f4840198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:53.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f4844cb4700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4840105f90 0x7f48401a31f0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f483800ef60 tx=0x7f483800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:53.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f483d7fa700 1 -- 172.21.15.12:0/968423502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838018070 con 0x7f4840105f90 2024-05-09T00:12:53.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f483d7fa700 1 -- 172.21.15.12:0/968423502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f483800f040 con 0x7f4840105f90 2024-05-09T00:12:53.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f4846717700 1 -- 172.21.15.12:0/968423502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48401a3a70 con 0x7f4840105f90 2024-05-09T00:12:53.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f483d7fa700 1 -- 172.21.15.12:0/968423502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483800cd70 con 0x7f4840105f90 2024-05-09T00:12:53.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.373+0000 7f4846717700 1 -- 172.21.15.12:0/968423502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48401a3f00 con 0x7f4840105f90 2024-05-09T00:12:53.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.374+0000 7f4846717700 1 -- 172.21.15.12:0/968423502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48401083a0 con 0x7f4840105f90 2024-05-09T00:12:53.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.375+0000 7f483d7fa700 1 -- 172.21.15.12:0/968423502 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4838003a00 con 0x7f4840105f90 2024-05-09T00:12:53.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.376+0000 7f483d7fa700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48280a1290 0x7f48280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:53.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.376+0000 7f483ffff700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48280a1290 0x7f48280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:53.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.376+0000 7f483d7fa700 1 -- 172.21.15.12:0/968423502 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f48380cd240 con 0x7f4840105f90 2024-05-09T00:12:53.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.377+0000 7f483ffff700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48280a1290 0x7f48280a3750 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4840067bb0 tx=0x7f4830009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:53.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.380+0000 7f483d7fa700 1 -- 172.21.15.12:0/968423502 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48380966d0 con 0x7f4840105f90 2024-05-09T00:12:53.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:53 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3661871036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:53.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:53 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3661871036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:53.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.596+0000 7f4846717700 1 -- 172.21.15.12:0/968423502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4840101560 con 0x7f4840105f90 2024-05-09T00:12:53.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.596+0000 7f483d7fa700 1 -- 172.21.15.12:0/968423502 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4838099cf0 con 0x7f4840105f90 2024-05-09T00:12:53.595 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-09T00:12:53.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 -- 172.21.15.12:0/968423502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48280a1290 msgr2=0x7f48280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48280a1290 0x7f48280a3750 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4840067bb0 tx=0x7f4830009260 comp rx=0 tx=0).stop 2024-05-09T00:12:53.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 -- 172.21.15.12:0/968423502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4840105f90 msgr2=0x7f48401a31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4840105f90 0x7f48401a31f0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f483800ef60 tx=0x7f483800ef90 comp rx=0 tx=0).stop 2024-05-09T00:12:53.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 -- 172.21.15.12:0/968423502 shutdown_connections 2024-05-09T00:12:53.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f48280a1290 0x7f48280a3750 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4840105f90 0x7f48401a31f0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48401000f0 0x7f4840198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 --2- 172.21.15.12:0/968423502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4840100ab0 0x7f48401986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 -- 172.21.15.12:0/968423502 >> 172.21.15.12:0/968423502 conn(0x7f4840076ba0 msgr2=0x7f48400fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:53.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 -- 172.21.15.12:0/968423502 shutdown_connections 2024-05-09T00:12:53.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.599+0000 7f4846717700 1 -- 172.21.15.12:0/968423502 wait complete. 2024-05-09T00:12:53.610 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:53 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3661871036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:53.611 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T00:12:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.754+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1814430993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a58101b10 msgr2=0x7f8a58105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.754+0000 7f8a5c8e5700 1 --2- 172.21.15.12:0/1814430993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a58101b10 0x7f8a58105b50 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009990 tx=0x7f8a40009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.755+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1814430993 shutdown_connections 2024-05-09T00:12:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.755+0000 7f8a5c8e5700 1 --2- 172.21.15.12:0/1814430993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a58106280 0x7f8a5810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.755+0000 7f8a5c8e5700 1 --2- 172.21.15.12:0/1814430993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a58101160 0x7f8a58101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.755+0000 7f8a5c8e5700 1 --2- 172.21.15.12:0/1814430993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a58101b10 0x7f8a58105b50 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.755+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1814430993 >> 172.21.15.12:0/1814430993 conn(0x7f8a580fc9c0 msgr2=0x7f8a580fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:53.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.755+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1814430993 shutdown_connections 2024-05-09T00:12:53.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.755+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1814430993 wait complete. 2024-05-09T00:12:53.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5c8e5700 1 Processor -- start 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5c8e5700 1 -- start start 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5c8e5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a58101160 0x7f8a581973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5c8e5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a58106280 0x7f8a58197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5c8e5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5819bcb0 0x7f8a5819c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5c8e5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5819c730 con 0x7f8a58101160 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5c8e5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5819c8a0 con 0x7f8a58106280 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5c8e5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5819ca60 con 0x7f8a5819bcb0 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a58101160 0x7f8a581973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a58101160 0x7f8a581973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33194/0 (socket says 172.21.15.12:33194) 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5659c700 1 -- 172.21.15.12:0/1228833351 learned_addr learned my addr 172.21.15.12:0/1228833351 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a55d9b700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a58106280 0x7f8a58197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:53.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a56d9d700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5819bcb0 0x7f8a5819c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5659c700 1 -- 172.21.15.12:0/1228833351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5819bcb0 msgr2=0x7f8a5819c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5659c700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5819bcb0 0x7f8a5819c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5659c700 1 -- 172.21.15.12:0/1228833351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a58106280 msgr2=0x7f8a58197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5659c700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a58106280 0x7f8a58197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a5659c700 1 -- 172.21.15.12:0/1228833351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a400095f0 con 0x7f8a58101160 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.756+0000 7f8a55d9b700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a58106280 0x7f8a58197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.757+0000 7f8a5659c700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a58101160 0x7f8a581973d0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800e9a0 tx=0x7f8a4800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.757+0000 7f8a4f7fe700 1 -- 172.21.15.12:0/1228833351 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48009790 con 0x7f8a58101160 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.757+0000 7f8a4f7fe700 1 -- 172.21.15.12:0/1228833351 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a4800cbd0 con 0x7f8a58101160 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.757+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1228833351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a581a3fc0 con 0x7f8a58101160 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.757+0000 7f8a4f7fe700 1 -- 172.21.15.12:0/1228833351 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48018680 con 0x7f8a58101160 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.757+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1228833351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a581a4490 con 0x7f8a58101160 2024-05-09T00:12:53.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.758+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1228833351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a58109550 con 0x7f8a58101160 2024-05-09T00:12:53.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.759+0000 7f8a4f7fe700 1 -- 172.21.15.12:0/1228833351 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8a4800cd40 con 0x7f8a58101160 2024-05-09T00:12:53.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.760+0000 7f8a4f7fe700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a440a1310 0x7f8a440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:53.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.760+0000 7f8a4f7fe700 1 -- 172.21.15.12:0/1228833351 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8a48014070 con 0x7f8a58101160 2024-05-09T00:12:53.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.760+0000 7f8a55d9b700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a440a1310 0x7f8a440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:53.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.761+0000 7f8a55d9b700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a440a1310 0x7f8a440a37d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006010 tx=0x7f8a40005940 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:53.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.764+0000 7f8a4f7fe700 1 -- 172.21.15.12:0/1228833351 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8a480968b0 con 0x7f8a58101160 2024-05-09T00:12:53.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.980+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1228833351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a581986d0 con 0x7f8a58101160 2024-05-09T00:12:53.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.981+0000 7f8a4f7fe700 1 -- 172.21.15.12:0/1228833351 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a48026020 con 0x7f8a58101160 2024-05-09T00:12:53.979 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-09T00:12:53.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1228833351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a440a1310 msgr2=0x7f8a440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a440a1310 0x7f8a440a37d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006010 tx=0x7f8a40005940 comp rx=0 tx=0).stop 2024-05-09T00:12:53.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1228833351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a58101160 msgr2=0x7f8a581973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:53.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a58101160 0x7f8a581973d0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800e9a0 tx=0x7f8a4800ed60 comp rx=0 tx=0).stop 2024-05-09T00:12:53.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1228833351 shutdown_connections 2024-05-09T00:12:53.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f8a440a1310 0x7f8a440a37d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5819bcb0 0x7f8a5819c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a58101160 0x7f8a581973d0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 --2- 172.21.15.12:0/1228833351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a58106280 0x7f8a58197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:53.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1228833351 >> 172.21.15.12:0/1228833351 conn(0x7f8a580fc9c0 msgr2=0x7f8a580fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:53.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1228833351 shutdown_connections 2024-05-09T00:12:53.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:53.984+0000 7f8a5c8e5700 1 -- 172.21.15.12:0/1228833351 wait complete. 2024-05-09T00:12:53.996 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T00:12:54.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 -- 172.21.15.12:0/1987031948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab58106280 msgr2=0x7fab5810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:54.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 --2- 172.21.15.12:0/1987031948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab58106280 0x7fab5810de00 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b0a0 tx=0x7fab5000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 -- 172.21.15.12:0/1987031948 shutdown_connections 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 --2- 172.21.15.12:0/1987031948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab58106280 0x7fab5810de00 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 --2- 172.21.15.12:0/1987031948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab58101160 0x7fab58101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 --2- 172.21.15.12:0/1987031948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab58101b10 0x7fab58105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 -- 172.21.15.12:0/1987031948 >> 172.21.15.12:0/1987031948 conn(0x7fab580fc9e0 msgr2=0x7fab580fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 -- 172.21.15.12:0/1987031948 shutdown_connections 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 -- 172.21.15.12:0/1987031948 wait complete. 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 Processor -- start 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.139+0000 7fab6006e700 1 -- start start 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab6006e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab58101160 0x7fab58197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab6006e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab58106280 0x7fab58197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab6006e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5819bdf0 0x7fab5819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab6006e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5819c870 con 0x7fab58101160 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab6006e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5819c9e0 con 0x7fab58106280 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab6006e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5819cba0 con 0x7fab5819bdf0 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5de0a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab58101160 0x7fab58197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5de0a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab58101160 0x7fab58197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33200/0 (socket says 172.21.15.12:33200) 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5de0a700 1 -- 172.21.15.12:0/29095287 learned_addr learned my addr 172.21.15.12:0/29095287 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5e60b700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5819bdf0 0x7fab5819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5d609700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab58106280 0x7fab58197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5de0a700 1 -- 172.21.15.12:0/29095287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5819bdf0 msgr2=0x7fab5819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:54.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5de0a700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5819bdf0 0x7fab5819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5de0a700 1 -- 172.21.15.12:0/29095287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab58106280 msgr2=0x7fab58197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:54.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5de0a700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab58106280 0x7fab58197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5de0a700 1 -- 172.21.15.12:0/29095287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50009d00 con 0x7fab58101160 2024-05-09T00:12:54.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5e60b700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5819bdf0 0x7fab5819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:54.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5d609700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab58106280 0x7fab58197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:54.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab5de0a700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab58101160 0x7fab58197510 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00e9a0 tx=0x7fab4c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:54.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.140+0000 7fab4affd700 1 -- 172.21.15.12:0/29095287 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c009790 con 0x7fab58101160 2024-05-09T00:12:54.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.141+0000 7fab4affd700 1 -- 172.21.15.12:0/29095287 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab4c00cbd0 con 0x7fab58101160 2024-05-09T00:12:54.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.141+0000 7fab6006e700 1 -- 172.21.15.12:0/29095287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab581a4100 con 0x7fab58101160 2024-05-09T00:12:54.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.141+0000 7fab4affd700 1 -- 172.21.15.12:0/29095287 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c018680 con 0x7fab58101160 2024-05-09T00:12:54.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.141+0000 7fab6006e700 1 -- 172.21.15.12:0/29095287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab581a4630 con 0x7fab58101160 2024-05-09T00:12:54.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.142+0000 7fab6006e700 1 -- 172.21.15.12:0/29095287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab581094f0 con 0x7fab58101160 2024-05-09T00:12:54.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.143+0000 7fab4affd700 1 -- 172.21.15.12:0/29095287 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fab4c00cd40 con 0x7fab58101160 2024-05-09T00:12:54.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.143+0000 7fab4affd700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab440a1310 0x7fab440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:54.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.144+0000 7fab4affd700 1 -- 172.21.15.12:0/29095287 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fab4c014070 con 0x7fab58101160 2024-05-09T00:12:54.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.144+0000 7fab5d609700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab440a1310 0x7fab440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:54.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.144+0000 7fab5d609700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab440a1310 0x7fab440a37d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fab58198a60 tx=0x7fab54009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:54.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.148+0000 7fab4affd700 1 -- 172.21.15.12:0/29095287 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fab4c052d60 con 0x7fab58101160 2024-05-09T00:12:54.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.364+0000 7fab6006e700 1 -- 172.21.15.12:0/29095287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab58198810 con 0x7fab58101160 2024-05-09T00:12:54.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.364+0000 7fab4affd700 1 -- 172.21.15.12:0/29095287 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab4c099ec0 con 0x7fab58101160 2024-05-09T00:12:54.362 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-09T00:12:54.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.366+0000 7fab6006e700 1 -- 172.21.15.12:0/29095287 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab440a1310 msgr2=0x7fab440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.366+0000 7fab6006e700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab440a1310 0x7fab440a37d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fab58198a60 tx=0x7fab54009260 comp rx=0 tx=0).stop 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.366+0000 7fab6006e700 1 -- 172.21.15.12:0/29095287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab58101160 msgr2=0x7fab58197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.366+0000 7fab6006e700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab58101160 0x7fab58197510 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00e9a0 tx=0x7fab4c00ed60 comp rx=0 tx=0).stop 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.366+0000 7fab6006e700 1 -- 172.21.15.12:0/29095287 shutdown_connections 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.366+0000 7fab6006e700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fab440a1310 0x7fab440a37d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.366+0000 7fab6006e700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab5819bdf0 0x7fab5819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.366+0000 7fab6006e700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab58101160 0x7fab58197510 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.366+0000 7fab6006e700 1 --2- 172.21.15.12:0/29095287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab58106280 0x7fab58197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.366+0000 7fab6006e700 1 -- 172.21.15.12:0/29095287 >> 172.21.15.12:0/29095287 conn(0x7fab580fc9e0 msgr2=0x7fab580fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.367+0000 7fab6006e700 1 -- 172.21.15.12:0/29095287 shutdown_connections 2024-05-09T00:12:54.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.367+0000 7fab6006e700 1 -- 172.21.15.12:0/29095287 wait complete. 2024-05-09T00:12:54.379 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T00:12:54.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.519+0000 7f88dedcb700 1 -- 172.21.15.12:0/803036489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d8106190 msgr2=0x7f88d810dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:54.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.519+0000 7f88dedcb700 1 --2- 172.21.15.12:0/803036489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d8106190 0x7f88d810dd10 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f88d000b0a0 tx=0x7f88d000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.519+0000 7f88dedcb700 1 -- 172.21.15.12:0/803036489 shutdown_connections 2024-05-09T00:12:54.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.519+0000 7f88dedcb700 1 --2- 172.21.15.12:0/803036489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d8106190 0x7f88d810dd10 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.519+0000 7f88dedcb700 1 --2- 172.21.15.12:0/803036489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d8101070 0x7f88d8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.519+0000 7f88dedcb700 1 --2- 172.21.15.12:0/803036489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88d8101a20 0x7f88d8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.519+0000 7f88dedcb700 1 -- 172.21.15.12:0/803036489 >> 172.21.15.12:0/803036489 conn(0x7f88d80fc910 msgr2=0x7f88d80fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.519+0000 7f88dedcb700 1 -- 172.21.15.12:0/803036489 shutdown_connections 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.519+0000 7f88dedcb700 1 -- 172.21.15.12:0/803036489 wait complete. 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88dedcb700 1 Processor -- start 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88dedcb700 1 -- start start 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88dedcb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88d8101070 0x7f88d80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88dedcb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d8106190 0x7f88d80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88dedcb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d8073d20 0x7f88d81a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88dedcb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81a5c30 con 0x7f88d8106190 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88dedcb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81a5da0 con 0x7f88d8073d20 2024-05-09T00:12:54.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88dedcb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81a5f60 con 0x7f88d8101070 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88d7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d8106190 0x7f88d80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88d7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d8106190 0x7f88d80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33212/0 (socket says 172.21.15.12:33212) 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88d7fff700 1 -- 172.21.15.12:0/1012713185 learned_addr learned my addr 172.21.15.12:0/1012713185 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88dd368700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d8073d20 0x7f88d81a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88dcb67700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88d8101070 0x7f88d80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88d7fff700 1 -- 172.21.15.12:0/1012713185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88d8101070 msgr2=0x7f88d80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88d7fff700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88d8101070 0x7f88d80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88d7fff700 1 -- 172.21.15.12:0/1012713185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d8073d20 msgr2=0x7f88d81a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88d7fff700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d8073d20 0x7f88d81a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.520+0000 7f88d7fff700 1 -- 172.21.15.12:0/1012713185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0009d00 con 0x7f88d8106190 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.521+0000 7f88d7fff700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d8106190 0x7f88d80737e0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f88c400e970 tx=0x7f88c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.521+0000 7f88d5ffb700 1 -- 172.21.15.12:0/1012713185 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4009790 con 0x7f88d8106190 2024-05-09T00:12:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.521+0000 7f88dedcb700 1 -- 172.21.15.12:0/1012713185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d81a6240 con 0x7f88d8106190 2024-05-09T00:12:54.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.521+0000 7f88d5ffb700 1 -- 172.21.15.12:0/1012713185 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88c400c7e0 con 0x7f88d8106190 2024-05-09T00:12:54.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.521+0000 7f88d5ffb700 1 -- 172.21.15.12:0/1012713185 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40054e0 con 0x7f88d8106190 2024-05-09T00:12:54.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.521+0000 7f88dedcb700 1 -- 172.21.15.12:0/1012713185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d81a67a0 con 0x7f88d8106190 2024-05-09T00:12:54.522 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:54 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/968423502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:12:54.522 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:54 smithi160 ceph-mon[46152]: pgmap v853: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-05-09T00:12:54.522 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:54 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1228833351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:12:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.522+0000 7f88dedcb700 1 -- 172.21.15.12:0/1012713185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d8109460 con 0x7f88d8106190 2024-05-09T00:12:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.523+0000 7f88d5ffb700 1 -- 172.21.15.12:0/1012713185 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f88c400c950 con 0x7f88d8106190 2024-05-09T00:12:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.524+0000 7f88d5ffb700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f88c80a1280 0x7f88c80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.524+0000 7f88d5ffb700 1 -- 172.21.15.12:0/1012713185 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f88c4014070 con 0x7f88d8106190 2024-05-09T00:12:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.524+0000 7f88dcb67700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f88c80a1280 0x7f88c80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.524+0000 7f88dcb67700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f88c80a1280 0x7f88c80a3740 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00ac50 tx=0x7f88cc00a380 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.528+0000 7f88d5ffb700 1 -- 172.21.15.12:0/1012713185 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f88c4096560 con 0x7f88d8106190 2024-05-09T00:12:54.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:54 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/968423502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:12:54.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:54 smithi012 ceph-mon[42503]: pgmap v853: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-05-09T00:12:54.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:54 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1228833351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:12:54.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.767+0000 7f88dedcb700 1 -- 172.21.15.12:0/1012713185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f88d8074730 con 0x7f88d8106190 2024-05-09T00:12:54.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.767+0000 7f88d5ffb700 1 -- 172.21.15.12:0/1012713185 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v140) v1 ==== 81+0+999 (secure 0 0 0) 0x7f88c4099b80 con 0x7f88d8106190 2024-05-09T00:12:54.765 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3622028746 2024-05-09T23:46:25.291064+0000 2024-05-09T00:12:54.765 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1163805495 2024-05-09T23:39:55.189061+0000 2024-05-09T00:12:54.765 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/31089560 2024-05-09T23:39:55.189061+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/31089560 2024-05-09T23:39:55.189061+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/4231005605 2024-05-09T23:39:32.336894+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/4231005605 2024-05-09T23:39:32.336894+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2069126425 2024-05-09T23:39:32.336894+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4176796914 2024-05-09T23:39:55.189061+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3062429456 2024-05-09T23:46:25.291064+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2114420865 2024-05-09T23:39:32.336894+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2607913710 2024-05-09T23:39:55.189061+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2381834553 2024-05-09T23:46:25.291064+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/716473668 2024-05-09T23:46:25.291064+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2695290437 2024-05-09T23:46:25.291064+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3737654591 2024-05-09T23:39:32.336894+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1305168755 2024-05-09T23:46:25.291064+0000 2024-05-09T00:12:54.766 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1305168755 2024-05-09T23:46:25.291064+0000 2024-05-09T00:12:54.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:54 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/968423502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:12:54.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:54 smithi204 ceph-mon[47373]: pgmap v853: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-05-09T00:12:54.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:54 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1228833351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:12:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 -- 172.21.15.12:0/1012713185 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f88c80a1280 msgr2=0x7f88c80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f88c80a1280 0x7f88c80a3740 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00ac50 tx=0x7f88cc00a380 comp rx=0 tx=0).stop 2024-05-09T00:12:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 -- 172.21.15.12:0/1012713185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d8106190 msgr2=0x7f88d80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d8106190 0x7f88d80737e0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f88c400e970 tx=0x7f88c400ec80 comp rx=0 tx=0).stop 2024-05-09T00:12:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 -- 172.21.15.12:0/1012713185 shutdown_connections 2024-05-09T00:12:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f88c80a1280 0x7f88c80a3740 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d8073d20 0x7f88d81a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88d8101070 0x7f88d80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 --2- 172.21.15.12:0/1012713185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88d8106190 0x7f88d80737e0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:54.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 -- 172.21.15.12:0/1012713185 >> 172.21.15.12:0/1012713185 conn(0x7f88d80fc910 msgr2=0x7f88d80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:54.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 -- 172.21.15.12:0/1012713185 shutdown_connections 2024-05-09T00:12:54.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:54.770+0000 7f88dedcb700 1 -- 172.21.15.12:0/1012713185 wait complete. 2024-05-09T00:12:54.769 INFO:teuthology.orchestra.run.smithi012.stderr:listed 17 entries 2024-05-09T00:12:54.782 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T00:12:54.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:12:54.782 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T00:12:54.876 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T00:12:54.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:12:54.877 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T00:12:55.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:12:55.087 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-09T00:12:55.171 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T00:12:55.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:12:55.172 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T00:12:55.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:12:55.419 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-05-09T00:12:55.611 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:55 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/29095287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:12:55.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:55 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/1012713185' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T00:12:55.674 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T00:12:55.694 INFO:tasks.cephadm:Teardown begin 2024-05-09T00:12:55.695 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T00:12:55.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:12:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:55 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/29095287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:12:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:55 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/1012713185' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T00:12:55.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:55 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/29095287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:12:55.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:55 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/1012713185' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T00:12:55.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.867+0000 7f04c1d5c700 1 -- 172.21.15.12:0/4212693096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101060 msgr2=0x7f04bc101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:55.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.867+0000 7f04c1d5c700 1 --2- 172.21.15.12:0/4212693096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101060 0x7f04bc101440 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b0a0 tx=0x7f04ac00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:12:55.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.867+0000 7f04c1d5c700 1 -- 172.21.15.12:0/4212693096 shutdown_connections 2024-05-09T00:12:55.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.867+0000 7f04c1d5c700 1 --2- 172.21.15.12:0/4212693096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04bc106180 0x7f04bc10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:55.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.868+0000 7f04c1d5c700 1 --2- 172.21.15.12:0/4212693096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101060 0x7f04bc101440 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:55.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.868+0000 7f04c1d5c700 1 --2- 172.21.15.12:0/4212693096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04bc101a10 0x7f04bc105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:55.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.868+0000 7f04c1d5c700 1 -- 172.21.15.12:0/4212693096 >> 172.21.15.12:0/4212693096 conn(0x7f04bc0fc920 msgr2=0x7f04bc0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:55.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.868+0000 7f04c1d5c700 1 -- 172.21.15.12:0/4212693096 shutdown_connections 2024-05-09T00:12:55.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.868+0000 7f04c1d5c700 1 -- 172.21.15.12:0/4212693096 wait complete. 2024-05-09T00:12:55.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.868+0000 7f04c1d5c700 1 Processor -- start 2024-05-09T00:12:55.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.868+0000 7f04c1d5c700 1 -- start start 2024-05-09T00:12:55.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04c1d5c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101060 0x7f04bc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:55.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04c1d5c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04bc106180 0x7f04bc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:55.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04c1d5c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04bc073d20 0x7f04bc1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:55.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04c1d5c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1a5c20 con 0x7f04bc101060 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04c1d5c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1a5d90 con 0x7f04bc106180 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04c1d5c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1a5f50 con 0x7f04bc073d20 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04bb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101060 0x7f04bc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04bb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101060 0x7f04bc0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33228/0 (socket says 172.21.15.12:33228) 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04bb7fe700 1 -- 172.21.15.12:0/2626236077 learned_addr learned my addr 172.21.15.12:0/2626236077 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04baffd700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04bc106180 0x7f04bc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04bb7fe700 1 -- 172.21.15.12:0/2626236077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04bc073d20 msgr2=0x7f04bc1a5620 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04bb7fe700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04bc073d20 0x7f04bc1a5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04bb7fe700 1 -- 172.21.15.12:0/2626236077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04bc106180 msgr2=0x7f04bc0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04bb7fe700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04bc106180 0x7f04bc0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:55.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04bb7fe700 1 -- 172.21.15.12:0/2626236077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac009d00 con 0x7f04bc101060 2024-05-09T00:12:55.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04bb7fe700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101060 0x7f04bc0732a0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b070 tx=0x7f04ac0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:55.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.869+0000 7f04b8ff9700 1 -- 172.21.15.12:0/2626236077 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00e040 con 0x7f04bc101060 2024-05-09T00:12:55.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.870+0000 7f04c1d5c700 1 -- 172.21.15.12:0/2626236077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04bc1a61d0 con 0x7f04bc101060 2024-05-09T00:12:55.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.870+0000 7f04b8ff9700 1 -- 172.21.15.12:0/2626236077 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04ac007950 con 0x7f04bc101060 2024-05-09T00:12:55.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.870+0000 7f04b8ff9700 1 -- 172.21.15.12:0/2626236077 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac01b450 con 0x7f04bc101060 2024-05-09T00:12:55.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.870+0000 7f04c1d5c700 1 -- 172.21.15.12:0/2626236077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc1a6650 con 0x7f04bc101060 2024-05-09T00:12:55.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.871+0000 7f04c1d5c700 1 -- 172.21.15.12:0/2626236077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04bc1093a0 con 0x7f04bc101060 2024-05-09T00:12:55.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.872+0000 7f04b8ff9700 1 -- 172.21.15.12:0/2626236077 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f04ac0040b0 con 0x7f04bc101060 2024-05-09T00:12:55.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.872+0000 7f04b8ff9700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04a80a1290 0x7f04a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:55.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.873+0000 7f04baffd700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04a80a1290 0x7f04a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:55.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.873+0000 7f04b8ff9700 1 -- 172.21.15.12:0/2626236077 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04ac0ce2a0 con 0x7f04bc101060 2024-05-09T00:12:55.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.873+0000 7f04baffd700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04a80a1290 0x7f04a80a3750 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f04bc074980 tx=0x7f04a400b410 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:55.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:55.877+0000 7f04b8ff9700 1 -- 172.21.15.12:0/2626236077 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f04ac097730 con 0x7f04bc101060 2024-05-09T00:12:56.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.149+0000 7f04c1d5c700 1 -- 172.21.15.12:0/2626236077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04bc074730 con 0x7f04bc101060 2024-05-09T00:12:56.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.150+0000 7f04b8ff9700 1 -- 172.21.15.12:0/2626236077 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f04ac09ad50 con 0x7f04bc101060 2024-05-09T00:12:56.148 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:12:56.148 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":21435,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/676326208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":676326208},{"type":"v1","addr":"172.21.15.160:6833","nonce":676326208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":89,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:12:19.040142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_31894":{"gid":31894,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3656191694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3656191694},{"type":"v1","addr":"172.21.15.204:6833","nonce":3656191694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:12:56.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 -- 172.21.15.12:0/2626236077 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04a80a1290 msgr2=0x7f04a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04a80a1290 0x7f04a80a3750 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f04bc074980 tx=0x7f04a400b410 comp rx=0 tx=0).stop 2024-05-09T00:12:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 -- 172.21.15.12:0/2626236077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101060 msgr2=0x7f04bc0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101060 0x7f04bc0732a0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b070 tx=0x7f04ac0096c0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 -- 172.21.15.12:0/2626236077 shutdown_connections 2024-05-09T00:12:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f04a80a1290 0x7f04a80a3750 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04bc073d20 0x7f04bc1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101060 0x7f04bc0732a0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 --2- 172.21.15.12:0/2626236077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04bc106180 0x7f04bc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 -- 172.21.15.12:0/2626236077 >> 172.21.15.12:0/2626236077 conn(0x7f04bc0fc920 msgr2=0x7f04bc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 -- 172.21.15.12:0/2626236077 shutdown_connections 2024-05-09T00:12:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.152+0000 7f04c1d5c700 1 -- 172.21.15.12:0/2626236077 wait complete. 2024-05-09T00:12:56.152 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 89 2024-05-09T00:12:56.164 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T00:12:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.307+0000 7fae97b89700 1 -- 172.21.15.12:0/882389111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae90100910 msgr2=0x7fae90104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.307+0000 7fae97b89700 1 --2- 172.21.15.12:0/882389111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae90100910 0x7fae90104950 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0098f0 tx=0x7fae8c009c00 comp rx=0 tx=0).stop 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.307+0000 7fae97b89700 1 -- 172.21.15.12:0/882389111 shutdown_connections 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.307+0000 7fae97b89700 1 --2- 172.21.15.12:0/882389111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae90105080 0x7fae9010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.307+0000 7fae97b89700 1 --2- 172.21.15.12:0/882389111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae900fff60 0x7fae90100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.307+0000 7fae97b89700 1 --2- 172.21.15.12:0/882389111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae90100910 0x7fae90104950 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.307+0000 7fae97b89700 1 -- 172.21.15.12:0/882389111 >> 172.21.15.12:0/882389111 conn(0x7fae900fb800 msgr2=0x7fae900fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.307+0000 7fae97b89700 1 -- 172.21.15.12:0/882389111 shutdown_connections 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.307+0000 7fae97b89700 1 -- 172.21.15.12:0/882389111 wait complete. 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae97b89700 1 Processor -- start 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae97b89700 1 -- start start 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae97b89700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae900fff60 0x7fae90197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae97b89700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae90105080 0x7fae90197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae97b89700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9019bc20 0x7fae9019c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae97b89700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9019c6a0 con 0x7fae90105080 2024-05-09T00:12:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae97b89700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9019c810 con 0x7fae900fff60 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae97b89700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9019c9d0 con 0x7fae9019bc20 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae95124700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae90105080 0x7fae90197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae95124700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae90105080 0x7fae90197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33242/0 (socket says 172.21.15.12:33242) 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae95124700 1 -- 172.21.15.12:0/3689583153 learned_addr learned my addr 172.21.15.12:0/3689583153 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae95925700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae900fff60 0x7fae90197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.308+0000 7fae96126700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9019bc20 0x7fae9019c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae95124700 1 -- 172.21.15.12:0/3689583153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9019bc20 msgr2=0x7fae9019c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae95124700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9019bc20 0x7fae9019c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae95124700 1 -- 172.21.15.12:0/3689583153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae900fff60 msgr2=0x7fae90197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae95124700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae900fff60 0x7fae90197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae95124700 1 -- 172.21.15.12:0/3689583153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c009520 con 0x7fae90105080 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae95925700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae900fff60 0x7fae90197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae95124700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae90105080 0x7fae90197880 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009230 tx=0x7fae8c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae86ffd700 1 -- 172.21.15.12:0/3689583153 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c01c070 con 0x7fae90105080 2024-05-09T00:12:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae97b89700 1 -- 172.21.15.12:0/3689583153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae901a3ed0 con 0x7fae90105080 2024-05-09T00:12:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae86ffd700 1 -- 172.21.15.12:0/3689583153 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae8c005040 con 0x7fae90105080 2024-05-09T00:12:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae86ffd700 1 -- 172.21.15.12:0/3689583153 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c003990 con 0x7fae90105080 2024-05-09T00:12:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.309+0000 7fae97b89700 1 -- 172.21.15.12:0/3689583153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae901a4300 con 0x7fae90105080 2024-05-09T00:12:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.310+0000 7fae97b89700 1 -- 172.21.15.12:0/3689583153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae90109300 con 0x7fae90105080 2024-05-09T00:12:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.311+0000 7fae86ffd700 1 -- 172.21.15.12:0/3689583153 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fae8c020030 con 0x7fae90105080 2024-05-09T00:12:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.312+0000 7fae86ffd700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae7c0a1290 0x7fae7c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.312+0000 7fae95925700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae7c0a1290 0x7fae7c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.312+0000 7fae86ffd700 1 -- 172.21.15.12:0/3689583153 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fae8c0ce050 con 0x7fae90105080 2024-05-09T00:12:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.313+0000 7fae95925700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae7c0a1290 0x7fae7c0a3750 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fae80005fd0 tx=0x7fae80009310 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.317+0000 7fae86ffd700 1 -- 172.21.15.12:0/3689583153 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fae8c0974e0 con 0x7fae90105080 2024-05-09T00:12:56.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:56 smithi012 ceph-mon[42503]: pgmap v854: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-09T00:12:56.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:56 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2626236077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:56.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.588+0000 7fae97b89700 1 -- 172.21.15.12:0/3689583153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae90101430 con 0x7fae90105080 2024-05-09T00:12:56.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.589+0000 7fae86ffd700 1 -- 172.21.15.12:0/3689583153 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+10877 (secure 0 0 0) 0x7fae8c0051b0 con 0x7fae90105080 2024-05-09T00:12:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:12:56.587 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2488990436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2488990436},{"type":"v1","addr":"172.21.15.12:6835","nonce":2488990436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3503743277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3503743277},{"type":"v1","addr":"172.21.15.12:6839","nonce":3503743277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/2743302125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":2743302125},{"type":"v1","addr":"172.21.15.12:6841","nonce":2743302125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":20007,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/761168719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":761168719},{"type":"v1","addr":"172.21.15.204:6837","nonce":761168719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":20148,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1370059141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":1370059141},{"type":"v1","addr":"172.21.15.12:6837","nonce":1370059141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":21435,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/676326208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":676326208},{"type":"v1","addr":"172.21.15.160:6833","nonce":676326208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":89,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:45:04.027837+0000","modified":"2024-05-09T00:12:19.040142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24515,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1481890752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1481890752},{"type":"v1","addr":"172.21.15.204:6835","nonce":1481890752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3313911488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3313911488},{"type":"v1","addr":"172.21.15.160:6835","nonce":3313911488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/2873951026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":2873951026},{"type":"v1","addr":"172.21.15.204:6839","nonce":2873951026}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.160:6837/1904912186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1904912186},{"type":"v1","addr":"172.21.15.160:6837","nonce":1904912186}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/924146489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":924146489},{"type":"v1","addr":"172.21.15.160:6839","nonce":924146489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_31894":{"gid":31894,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3656191694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3656191694},{"type":"v1","addr":"172.21.15.204:6833","nonce":3656191694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T00:12:56.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 -- 172.21.15.12:0/3689583153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae7c0a1290 msgr2=0x7fae7c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae7c0a1290 0x7fae7c0a3750 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fae80005fd0 tx=0x7fae80009310 comp rx=0 tx=0).stop 2024-05-09T00:12:56.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 -- 172.21.15.12:0/3689583153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae90105080 msgr2=0x7fae90197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae90105080 0x7fae90197880 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009230 tx=0x7fae8c00bf60 comp rx=0 tx=0).stop 2024-05-09T00:12:56.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 -- 172.21.15.12:0/3689583153 shutdown_connections 2024-05-09T00:12:56.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fae7c0a1290 0x7fae7c0a3750 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9019bc20 0x7fae9019c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae900fff60 0x7fae90197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 --2- 172.21.15.12:0/3689583153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae90105080 0x7fae90197880 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 -- 172.21.15.12:0/3689583153 >> 172.21.15.12:0/3689583153 conn(0x7fae900fb800 msgr2=0x7fae900698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 -- 172.21.15.12:0/3689583153 shutdown_connections 2024-05-09T00:12:56.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.591+0000 7fae97b89700 1 -- 172.21.15.12:0/3689583153 wait complete. 2024-05-09T00:12:56.590 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 89 2024-05-09T00:12:56.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T00:12:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.743+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/4261838675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c00fff70 msgr2=0x7fd6c0100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.743+0000 7fd6c4c3c700 1 --2- 172.21.15.12:0/4261838675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c00fff70 0x7fd6c0100350 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009990 tx=0x7fd6b4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.743+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/4261838675 shutdown_connections 2024-05-09T00:12:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.743+0000 7fd6c4c3c700 1 --2- 172.21.15.12:0/4261838675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6c0105090 0x7fd6c010cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.743+0000 7fd6c4c3c700 1 --2- 172.21.15.12:0/4261838675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c00fff70 0x7fd6c0100350 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.743+0000 7fd6c4c3c700 1 --2- 172.21.15.12:0/4261838675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6c0100920 0x7fd6c0104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.743+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/4261838675 >> 172.21.15.12:0/4261838675 conn(0x7fd6c00fb7d0 msgr2=0x7fd6c00fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.743+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/4261838675 shutdown_connections 2024-05-09T00:12:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.744+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/4261838675 wait complete. 2024-05-09T00:12:56.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.744+0000 7fd6c4c3c700 1 Processor -- start 2024-05-09T00:12:56.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.744+0000 7fd6c4c3c700 1 -- start start 2024-05-09T00:12:56.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6c4c3c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6c00fff70 0x7fd6c0101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6c4c3c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6c0105090 0x7fd6c0101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6c4c3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c0102480 0x7fd6c01a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6c4c3c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01a5c00 con 0x7fd6c0102480 2024-05-09T00:12:56.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6c4c3c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01a5d70 con 0x7fd6c00fff70 2024-05-09T00:12:56.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6c4c3c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01a5f30 con 0x7fd6c0105090 2024-05-09T00:12:56.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c0102480 0x7fd6c01a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c0102480 0x7fd6c01a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33250/0 (socket says 172.21.15.12:33250) 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bed9d700 1 -- 172.21.15.12:0/3983517663 learned_addr learned my addr 172.21.15.12:0/3983517663 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6be59c700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6c00fff70 0x7fd6c0101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bdd9b700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6c0105090 0x7fd6c0101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bed9d700 1 -- 172.21.15.12:0/3983517663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6c0105090 msgr2=0x7fd6c0101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bed9d700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6c0105090 0x7fd6c0101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bed9d700 1 -- 172.21.15.12:0/3983517663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6c00fff70 msgr2=0x7fd6c0101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bed9d700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6c00fff70 0x7fd6c0101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bed9d700 1 -- 172.21.15.12:0/3983517663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40095f0 con 0x7fd6c0102480 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bdd9b700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6c0105090 0x7fd6c0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6bed9d700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c0102480 0x7fd6c01a5630 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800b780 tx=0x7fd6b800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.745+0000 7fd6af7fe700 1 -- 172.21.15.12:0/3983517663 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800f840 con 0x7fd6c0102480 2024-05-09T00:12:56.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.746+0000 7fd6af7fe700 1 -- 172.21.15.12:0/3983517663 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6b800fe60 con 0x7fd6c0102480 2024-05-09T00:12:56.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.746+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/3983517663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c01a6210 con 0x7fd6c0102480 2024-05-09T00:12:56.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.746+0000 7fd6af7fe700 1 -- 172.21.15.12:0/3983517663 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800d3d0 con 0x7fd6c0102480 2024-05-09T00:12:56.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.746+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/3983517663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c01a6720 con 0x7fd6c0102480 2024-05-09T00:12:56.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.747+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/3983517663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c0109310 con 0x7fd6c0102480 2024-05-09T00:12:56.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.748+0000 7fd6af7fe700 1 -- 172.21.15.12:0/3983517663 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd6b800f9a0 con 0x7fd6c0102480 2024-05-09T00:12:56.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.748+0000 7fd6af7fe700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd6a80a1310 0x7fd6a80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.749+0000 7fd6af7fe700 1 -- 172.21.15.12:0/3983517663 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd6b80ccb40 con 0x7fd6c0102480 2024-05-09T00:12:56.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.749+0000 7fd6be59c700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd6a80a1310 0x7fd6a80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.749+0000 7fd6be59c700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd6a80a1310 0x7fd6a80a37d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4006010 tx=0x7fd6b400b560 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:56.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.753+0000 7fd6af7fe700 1 -- 172.21.15.12:0/3983517663 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd6b8095fa0 con 0x7fd6c0102480 2024-05-09T00:12:56.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:56 smithi204 ceph-mon[47373]: pgmap v854: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-09T00:12:56.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:56 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2626236077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:56.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:56 smithi160 ceph-mon[46152]: pgmap v854: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-05-09T00:12:56.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:56 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2626236077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:56.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.974+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/3983517663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6c0101440 con 0x7fd6c0102480 2024-05-09T00:12:56.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.976+0000 7fd6af7fe700 1 -- 172.21.15.12:0/3983517663 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+22978 (secure 0 0 0) 0x7fd6b80995c0 con 0x7fd6c0102480 2024-05-09T00:12:56.974 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-09T00:12:56.974 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":140,"fsid":"00f8fb28-0d94-11ef-bc97-c7b262605968","created":"2024-05-08T23:39:01.616862+0000","modified":"2024-05-09T00:12:16.031662+0000","last_up_change":"2024-05-08T23:44:30.122909+0000","last_in_change":"2024-05-08T23:44:13.296753+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:41:47.730840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:45:02.165326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:45:03.407498+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":139,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6051d38-71b4-4946-b055-5ccf985ceed4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6803","nonce":2326888222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6805","nonce":2326888222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6809","nonce":2326888222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2326888222},{"type":"v1","addr":"172.21.15.12:6807","nonce":2326888222}]},"public_addr":"172.21.15.12:6803/2326888222","cluster_addr":"172.21.15.12:6805/2326888222","heartbeat_back_addr":"172.21.15.12:6809/2326888222","heartbeat_front_addr":"172.21.15.12:6807/2326888222","state":["exists","up"]},{"osd":1,"uuid":"83b3995f-c70f-49b5-a4eb-0969ce93f22b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6801","nonce":72539183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6803","nonce":72539183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6807","nonce":72539183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":72539183},{"type":"v1","addr":"172.21.15.160:6805","nonce":72539183}]},"public_addr":"172.21.15.160:6801/72539183","cluster_addr":"172.21.15.160:6803/72539183","heartbeat_back_addr":"172.21.15.160:6807/72539183","heartbeat_front_addr":"172.21.15.160:6805/72539183","state":["exists","up"]},{"osd":2,"uuid":"71a297a7-f16c-4e12-ab9f-f197e06ff07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6801","nonce":474619962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6803","nonce":474619962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6807","nonce":474619962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":474619962},{"type":"v1","addr":"172.21.15.204:6805","nonce":474619962}]},"public_addr":"172.21.15.204:6801/474619962","cluster_addr":"172.21.15.204:6803/474619962","heartbeat_back_addr":"172.21.15.204:6807/474619962","heartbeat_front_addr":"172.21.15.204:6805/474619962","state":["exists","up"]},{"osd":3,"uuid":"1ebe819d-3d5e-4128-8c6a-23921df4320b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6811","nonce":2160100336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6813","nonce":2160100336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6817","nonce":2160100336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2160100336},{"type":"v1","addr":"172.21.15.12:6815","nonce":2160100336}]},"public_addr":"172.21.15.12:6811/2160100336","cluster_addr":"172.21.15.12:6813/2160100336","heartbeat_back_addr":"172.21.15.12:6817/2160100336","heartbeat_front_addr":"172.21.15.12:6815/2160100336","state":["exists","up"]},{"osd":4,"uuid":"0e77c894-013b-40d3-9249-85a0c63076ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6809","nonce":2838790327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6811","nonce":2838790327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6815","nonce":2838790327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2838790327},{"type":"v1","addr":"172.21.15.160:6813","nonce":2838790327}]},"public_addr":"172.21.15.160:6809/2838790327","cluster_addr":"172.21.15.160:6811/2838790327","heartbeat_back_addr":"172.21.15.160:6815/2838790327","heartbeat_front_addr":"172.21.15.160:6813/2838790327","state":["exists","up"]},{"osd":5,"uuid":"f45217cd-6ced-413f-8be8-66c2740e7fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6809","nonce":3032887482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6811","nonce":3032887482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6815","nonce":3032887482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3032887482},{"type":"v1","addr":"172.21.15.204:6813","nonce":3032887482}]},"public_addr":"172.21.15.204:6809/3032887482","cluster_addr":"172.21.15.204:6811/3032887482","heartbeat_back_addr":"172.21.15.204:6815/3032887482","heartbeat_front_addr":"172.21.15.204:6813/3032887482","state":["exists","up"]},{"osd":6,"uuid":"7c76b479-95f8-4516-bf24-3677fa408097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6819","nonce":3301040337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6821","nonce":3301040337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6825","nonce":3301040337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3301040337},{"type":"v1","addr":"172.21.15.12:6823","nonce":3301040337}]},"public_addr":"172.21.15.12:6819/3301040337","cluster_addr":"172.21.15.12:6821/3301040337","heartbeat_back_addr":"172.21.15.12:6825/3301040337","heartbeat_front_addr":"172.21.15.12:6823/3301040337","state":["exists","up"]},{"osd":7,"uuid":"fc3df920-f628-41de-a0eb-fd1d6b1bf06a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6817","nonce":1443429618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6819","nonce":1443429618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6823","nonce":1443429618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":1443429618},{"type":"v1","addr":"172.21.15.160:6821","nonce":1443429618}]},"public_addr":"172.21.15.160:6817/1443429618","cluster_addr":"172.21.15.160:6819/1443429618","heartbeat_back_addr":"172.21.15.160:6823/1443429618","heartbeat_front_addr":"172.21.15.160:6821/1443429618","state":["exists","up"]},{"osd":8,"uuid":"91051f6f-5519-48e0-93a6-e897d0cb3929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6817","nonce":4190277185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6819","nonce":4190277185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6823","nonce":4190277185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":4190277185},{"type":"v1","addr":"172.21.15.204:6821","nonce":4190277185}]},"public_addr":"172.21.15.204:6817/4190277185","cluster_addr":"172.21.15.204:6819/4190277185","heartbeat_back_addr":"172.21.15.204:6823/4190277185","heartbeat_front_addr":"172.21.15.204:6821/4190277185","state":["exists","up"]},{"osd":9,"uuid":"3d7852c8-2f31-4385-84a1-6a4c07279741","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6827","nonce":609239172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6829","nonce":609239172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6833","nonce":609239172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":609239172},{"type":"v1","addr":"172.21.15.12:6831","nonce":609239172}]},"public_addr":"172.21.15.12:6827/609239172","cluster_addr":"172.21.15.12:6829/609239172","heartbeat_back_addr":"172.21.15.12:6833/609239172","heartbeat_front_addr":"172.21.15.12:6831/609239172","state":["exists","up"]},{"osd":10,"uuid":"f8a47be8-7650-4178-86b0-0b529d02ae21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6825","nonce":2916593770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6827","nonce":2916593770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6831","nonce":2916593770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2916593770},{"type":"v1","addr":"172.21.15.160:6829","nonce":2916593770}]},"public_addr":"172.21.15.160:6825/2916593770","cluster_addr":"172.21.15.160:6827/2916593770","heartbeat_back_addr":"172.21.15.160:6831/2916593770","heartbeat_front_addr":"172.21.15.160:6829/2916593770","state":["exists","up"]},{"osd":11,"uuid":"e9361b5a-5dd2-4920-b89f-d040b487b6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6825","nonce":3148638065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6827","nonce":3148638065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6831","nonce":3148638065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3148638065},{"type":"v1","addr":"172.21.15.204:6829","nonce":3148638065}]},"public_addr":"172.21.15.204:6825/3148638065","cluster_addr":"172.21.15.204:6827/3148638065","heartbeat_back_addr":"172.21.15.204:6831/3148638065","heartbeat_front_addr":"172.21.15.204:6829/3148638065","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:08.535513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:26.301634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:41:43.858995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:02.586509+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:20.913312+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:38.384919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:42:57.413634+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:15.488435+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:33.233785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:43:51.955168+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:10.407083+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:44:28.245483+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3622028746":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/1163805495":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6800/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/31089560":"2024-05-09T23:39:55.189061+0000","172.21.15.12:6801/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/4231005605":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2069126425":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/4176796914":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/3062429456":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2114420865":"2024-05-09T23:39:32.336894+0000","172.21.15.12:0/2607913710":"2024-05-09T23:39:55.189061+0000","172.21.15.12:0/2381834553":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/716473668":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/2695290437":"2024-05-09T23:46:25.291064+0000","172.21.15.12:0/3737654591":"2024-05-09T23:39:32.336894+0000","172.21.15.12:6800/1305168755":"2024-05-09T23:46:25.291064+0000","172.21.15.12:6801/1305168755":"2024-05-09T23:46:25.291064+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":33,"length":1},{"begin":46,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:12:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/3983517663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd6a80a1310 msgr2=0x7fd6a80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd6a80a1310 0x7fd6a80a37d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4006010 tx=0x7fd6b400b560 comp rx=0 tx=0).stop 2024-05-09T00:12:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/3983517663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c0102480 msgr2=0x7fd6c01a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c0102480 0x7fd6c01a5630 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800b780 tx=0x7fd6b800bb40 comp rx=0 tx=0).stop 2024-05-09T00:12:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/3983517663 shutdown_connections 2024-05-09T00:12:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fd6a80a1310 0x7fd6a80a37d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6c0102480 0x7fd6c01a5630 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6c00fff70 0x7fd6c0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 --2- 172.21.15.12:0/3983517663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6c0105090 0x7fd6c0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/3983517663 >> 172.21.15.12:0/3983517663 conn(0x7fd6c00fb7d0 msgr2=0x7fd6c0069860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/3983517663 shutdown_connections 2024-05-09T00:12:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:56.978+0000 7fd6c4c3c700 1 -- 172.21.15.12:0/3983517663 wait complete. 2024-05-09T00:12:56.990 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T00:12:57.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.131+0000 7f980d1bd700 1 -- 172.21.15.12:0/2394318956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9808069420 msgr2=0x7f9808069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:57.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.131+0000 7f980d1bd700 1 --2- 172.21.15.12:0/2394318956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9808069420 0x7f9808069800 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009990 tx=0x7f97f8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.131+0000 7f980d1bd700 1 -- 172.21.15.12:0/2394318956 shutdown_connections 2024-05-09T00:12:57.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.131+0000 7f980d1bd700 1 --2- 172.21.15.12:0/2394318956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98081062a0 0x7f980810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.131+0000 7f980d1bd700 1 --2- 172.21.15.12:0/2394318956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9808069420 0x7f9808069800 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.131+0000 7f980d1bd700 1 --2- 172.21.15.12:0/2394318956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9808069dd0 0x7f9808105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.131+0000 7f980d1bd700 1 -- 172.21.15.12:0/2394318956 >> 172.21.15.12:0/2394318956 conn(0x7f9808076ba0 msgr2=0x7f9808076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:57.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.131+0000 7f980d1bd700 1 -- 172.21.15.12:0/2394318956 shutdown_connections 2024-05-09T00:12:57.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.131+0000 7f980d1bd700 1 -- 172.21.15.12:0/2394318956 wait complete. 2024-05-09T00:12:57.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980d1bd700 1 Processor -- start 2024-05-09T00:12:57.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980d1bd700 1 -- start start 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980d1bd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9808069420 0x7f9808197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980d1bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98081062a0 0x7f9808197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980d1bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f980819bc20 0x7f980819c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980d1bd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819c6a0 con 0x7f98081062a0 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980d1bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819c810 con 0x7f980819bc20 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980d1bd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819c9d0 con 0x7f9808069420 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98081062a0 0x7f9808197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98081062a0 0x7f9808197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33272/0 (socket says 172.21.15.12:33272) 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f980659c700 1 -- 172.21.15.12:0/3977087649 learned_addr learned my addr 172.21.15.12:0/3977087649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.132+0000 7f9806d9d700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9808069420 0x7f9808197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f980759e700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f980819bc20 0x7f980819c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:57.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f980659c700 1 -- 172.21.15.12:0/3977087649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9808069420 msgr2=0x7f9808197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f980659c700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9808069420 0x7f9808197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f980659c700 1 -- 172.21.15.12:0/3977087649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f980819bc20 msgr2=0x7f980819c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f980659c700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f980819bc20 0x7f980819c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f980659c700 1 -- 172.21.15.12:0/3977087649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80095f0 con 0x7f98081062a0 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f980659c700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98081062a0 0x7f9808197880 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00d980 tx=0x7f97fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f97effff700 1 -- 172.21.15.12:0/3977087649 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc009930 con 0x7f98081062a0 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f97effff700 1 -- 172.21.15.12:0/3977087649 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97fc010460 con 0x7f98081062a0 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f980d1bd700 1 -- 172.21.15.12:0/3977087649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98081a3f30 con 0x7f98081062a0 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f97effff700 1 -- 172.21.15.12:0/3977087649 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc00f620 con 0x7f98081062a0 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.133+0000 7f980d1bd700 1 -- 172.21.15.12:0/3977087649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98081a43b0 con 0x7f98081062a0 2024-05-09T00:12:57.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.134+0000 7f980d1bd700 1 -- 172.21.15.12:0/3977087649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9808109510 con 0x7f98081062a0 2024-05-09T00:12:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.136+0000 7f97effff700 1 -- 172.21.15.12:0/3977087649 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f97fc0108d0 con 0x7f98081062a0 2024-05-09T00:12:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.136+0000 7f97effff700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97f00a1290 0x7f97f00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.136+0000 7f97effff700 1 -- 172.21.15.12:0/3977087649 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f97fc0cc740 con 0x7f98081062a0 2024-05-09T00:12:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.136+0000 7f9806d9d700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97f00a1290 0x7f97f00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:57.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.137+0000 7f9806d9d700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97f00a1290 0x7f97f00a3750 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f97f8006010 tx=0x7f97f8005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:57.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.140+0000 7f97effff700 1 -- 172.21.15.12:0/3977087649 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97fc095ba0 con 0x7f98081062a0 2024-05-09T00:12:57.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.413+0000 7f980d1bd700 1 -- 172.21.15.12:0/3977087649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9808198640 con 0x7f98081062a0 2024-05-09T00:12:57.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.425+0000 7f97effff700 1 -- 172.21.15.12:0/3977087649 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) v1 ==== 161+0+0 (secure 0 0 0) 0x7f97fc017090 con 0x7f98081062a0 2024-05-09T00:12:57.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 -- 172.21.15.12:0/3977087649 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97f00a1290 msgr2=0x7f97f00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:57.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97f00a1290 0x7f97f00a3750 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f97f8006010 tx=0x7f97f8005960 comp rx=0 tx=0).stop 2024-05-09T00:12:57.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 -- 172.21.15.12:0/3977087649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98081062a0 msgr2=0x7f9808197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:57.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98081062a0 0x7f9808197880 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00d980 tx=0x7f97fc00dc90 comp rx=0 tx=0).stop 2024-05-09T00:12:57.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 -- 172.21.15.12:0/3977087649 shutdown_connections 2024-05-09T00:12:57.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f97f00a1290 0x7f97f00a3750 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f980819bc20 0x7f980819c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9808069420 0x7f9808197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 --2- 172.21.15.12:0/3977087649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98081062a0 0x7f9808197880 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 -- 172.21.15.12:0/3977087649 >> 172.21.15.12:0/3977087649 conn(0x7f9808076ba0 msgr2=0x7f98080ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:57.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 -- 172.21.15.12:0/3977087649 shutdown_connections 2024-05-09T00:12:57.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.427+0000 7f980d1bd700 1 -- 172.21.15.12:0/3977087649 wait complete. 2024-05-09T00:12:57.427 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T00:12:57.440 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T00:12:57.513 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:57 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3689583153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:57.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:57 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3983517663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:57.514 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 00:12:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-b[82741]: ignoring --setuser ceph since I am not root 2024-05-09T00:12:57.514 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 00:12:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-b[82741]: ignoring --setgroup ceph since I am not root 2024-05-09T00:12:57.514 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 00:12:57 smithi160 ceph-mds[82764]: -- 172.21.15.160:0/2120916352 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5612798101a0 con 0x5612797e6c00 2024-05-09T00:12:57.514 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 00:12:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-b[82741]: starting mds.b at 2024-05-09T00:12:57.515 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 00:12:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-h[82391]: ignoring --setuser ceph since I am not root 2024-05-09T00:12:57.515 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 00:12:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-h[82391]: ignoring --setgroup ceph since I am not root 2024-05-09T00:12:57.515 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 00:12:57 smithi160 ceph-mds[82414]: -- 172.21.15.160:0/3830001279 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e2458721a0 con 0x55e245848c00 2024-05-09T00:12:57.515 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 00:12:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-e[82051]: ignoring --setuser ceph since I am not root 2024-05-09T00:12:57.515 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 00:12:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-e[82051]: ignoring --setgroup ceph since I am not root 2024-05-09T00:12:57.515 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 00:12:57 smithi160 ceph-mds[82074]: -- 172.21.15.160:0/1450652994 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5594bc7fa1a0 con 0x5594bc7d0800 2024-05-09T00:12:57.515 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 00:12:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-e[82051]: starting mds.e at 2024-05-09T00:12:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.583+0000 7f59fb51d700 1 -- 172.21.15.12:0/3264350941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f4100060 msgr2=0x7f59f4100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.583+0000 7f59fb51d700 1 --2- 172.21.15.12:0/3264350941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f4100060 0x7f59f4100440 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009990 tx=0x7f59e8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.583+0000 7f59fb51d700 1 -- 172.21.15.12:0/3264350941 shutdown_connections 2024-05-09T00:12:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.583+0000 7f59fb51d700 1 --2- 172.21.15.12:0/3264350941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59f4105160 0x7f59f410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.583+0000 7f59fb51d700 1 --2- 172.21.15.12:0/3264350941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f4100060 0x7f59f4100440 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.583+0000 7f59fb51d700 1 --2- 172.21.15.12:0/3264350941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59f4100a10 0x7f59f4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.583+0000 7f59fb51d700 1 -- 172.21.15.12:0/3264350941 >> 172.21.15.12:0/3264350941 conn(0x7f59f4076f40 msgr2=0x7f59f4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.584+0000 7f59fb51d700 1 -- 172.21.15.12:0/3264350941 shutdown_connections 2024-05-09T00:12:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.584+0000 7f59fb51d700 1 -- 172.21.15.12:0/3264350941 wait complete. 2024-05-09T00:12:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.584+0000 7f59fb51d700 1 Processor -- start 2024-05-09T00:12:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.584+0000 7f59fb51d700 1 -- start start 2024-05-09T00:12:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.584+0000 7f59fb51d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59f4100060 0x7f59f41973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.584+0000 7f59fb51d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59f4105160 0x7f59f4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59fb51d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f419bc80 0x7f59f419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59fb51d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f419c700 con 0x7f59f419bc80 2024-05-09T00:12:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59fb51d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f419c870 con 0x7f59f4100060 2024-05-09T00:12:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59fb51d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f419ca30 con 0x7f59f4105160 2024-05-09T00:12:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f9aba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f419bc80 0x7f59f419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f8ab8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59f4105160 0x7f59f4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f9aba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f419bc80 0x7f59f419c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33290/0 (socket says 172.21.15.12:33290) 2024-05-09T00:12:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f9aba700 1 -- 172.21.15.12:0/2292670290 learned_addr learned my addr 172.21.15.12:0/2292670290 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f92b9700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59f4100060 0x7f59f41973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f9aba700 1 -- 172.21.15.12:0/2292670290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59f4105160 msgr2=0x7f59f4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f9aba700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59f4105160 0x7f59f4197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f9aba700 1 -- 172.21.15.12:0/2292670290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59f4100060 msgr2=0x7f59f41973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f9aba700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59f4100060 0x7f59f41973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f9aba700 1 -- 172.21.15.12:0/2292670290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e80095f0 con 0x7f59f419bc80 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f8ab8700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59f4105160 0x7f59f4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59f9aba700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f419bc80 0x7f59f419c100 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00ef30 tx=0x7f59ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59e67fc700 1 -- 172.21.15.12:0/2292670290 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec018070 con 0x7f59f419bc80 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59e67fc700 1 -- 172.21.15.12:0/2292670290 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59ec00f040 con 0x7f59f419bc80 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59fb51d700 1 -- 172.21.15.12:0/2292670290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f41a3f30 con 0x7f59f419bc80 2024-05-09T00:12:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.585+0000 7f59e67fc700 1 -- 172.21.15.12:0/2292670290 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec00cd00 con 0x7f59f419bc80 2024-05-09T00:12:57.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.586+0000 7f59fb51d700 1 -- 172.21.15.12:0/2292670290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f41a43c0 con 0x7f59f419bc80 2024-05-09T00:12:57.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.586+0000 7f59fb51d700 1 -- 172.21.15.12:0/2292670290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f4101510 con 0x7f59f419bc80 2024-05-09T00:12:57.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.588+0000 7f59e67fc700 1 -- 172.21.15.12:0/2292670290 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f59ec003a00 con 0x7f59f419bc80 2024-05-09T00:12:57.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.588+0000 7f59e67fc700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59e00a1290 0x7f59e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:57.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.588+0000 7f59e67fc700 1 -- 172.21.15.12:0/2292670290 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 8712+0+0 (secure 0 0 0) 0x7f59ec0cd1c0 con 0x7f59f419bc80 2024-05-09T00:12:57.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.588+0000 7f59f92b9700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59e00a1290 0x7f59e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:57.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.589+0000 7f59f92b9700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59e00a1290 0x7f59e00a3750 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b2e0 tx=0x7f59e8005960 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.592+0000 7f59e67fc700 1 -- 172.21.15.12:0/2292670290 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f59ec0965e0 con 0x7f59f419bc80 2024-05-09T00:12:57.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:57 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3689583153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:57.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:57 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3983517663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:57.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:57 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-a[42479]: 2024-05-09T00:12:57.414+0000 7ff27fc6b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T00:12:57.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:57 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3689583153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:57.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:57 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3983517663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:57.766 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 09 00:12:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-f[84273]: ignoring --setuser ceph since I am not root 2024-05-09T00:12:57.767 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 09 00:12:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-f[84273]: ignoring --setgroup ceph since I am not root 2024-05-09T00:12:57.767 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 09 00:12:57 smithi204 ceph-mds[84296]: -- 172.21.15.204:0/4015120253 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558a7e7741a0 con 0x558a7e74ac00 2024-05-09T00:12:57.767 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 09 00:12:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-f[84273]: starting mds.f at 2024-05-09T00:12:57.767 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:12:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: ignoring --setuser ceph since I am not root 2024-05-09T00:12:57.767 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:12:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: ignoring --setgroup ceph since I am not root 2024-05-09T00:12:57.767 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:12:57 smithi204 ceph-mds[83951]: -- 172.21.15.204:0/1558943001 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5568345a01a0 con 0x556834577000 2024-05-09T00:12:57.767 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:12:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: starting mds.l at 2024-05-09T00:12:57.767 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 09 00:12:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-i[84958]: ignoring --setuser ceph since I am not root 2024-05-09T00:12:57.768 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 09 00:12:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-i[84958]: ignoring --setgroup ceph since I am not root 2024-05-09T00:12:57.768 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 09 00:12:57 smithi204 ceph-mds[84981]: -- 172.21.15.204:0/4246058853 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5595368301a0 con 0x559536806c00 2024-05-09T00:12:57.768 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 09 00:12:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-i[84958]: starting mds.i at 2024-05-09T00:12:57.768 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 00:12:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-h[82391]: starting mds.h at 2024-05-09T00:12:57.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:57.864+0000 7f59fb51d700 1 -- 172.21.15.12:0/2292670290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f59f4100a10 con 0x7f59f419bc80 2024-05-09T00:12:58.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.432+0000 7f59e67fc700 1 -- 172.21.15.12:0/2292670290 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v91) v1 ==== 104+0+0 (secure 0 0 0) 0x7f59ec099c00 con 0x7f59f419bc80 2024-05-09T00:12:58.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 -- 172.21.15.12:0/2292670290 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59e00a1290 msgr2=0x7f59e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:58.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59e00a1290 0x7f59e00a3750 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b2e0 tx=0x7f59e8005960 comp rx=0 tx=0).stop 2024-05-09T00:12:58.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 -- 172.21.15.12:0/2292670290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f419bc80 msgr2=0x7f59f419c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:58.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f419bc80 0x7f59f419c100 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00ef30 tx=0x7f59ec00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 -- 172.21.15.12:0/2292670290 shutdown_connections 2024-05-09T00:12:58.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f59e00a1290 0x7f59e00a3750 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59f419bc80 0x7f59f419c100 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59f4100060 0x7f59f41973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 --2- 172.21.15.12:0/2292670290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59f4105160 0x7f59f4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 -- 172.21.15.12:0/2292670290 >> 172.21.15.12:0/2292670290 conn(0x7f59f4076f40 msgr2=0x7f59f4103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 -- 172.21.15.12:0/2292670290 shutdown_connections 2024-05-09T00:12:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.436+0000 7f59fb51d700 1 -- 172.21.15.12:0/2292670290 wait complete. 2024-05-09T00:12:58.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T00:12:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.599+0000 7fa1656f5700 1 -- 172.21.15.12:0/920146606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160101b90 msgr2=0x7fa160105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.599+0000 7fa1656f5700 1 --2- 172.21.15.12:0/920146606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160101b90 0x7fa160105bd0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fa154009990 tx=0x7fa154009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.600+0000 7fa1656f5700 1 -- 172.21.15.12:0/920146606 shutdown_connections 2024-05-09T00:12:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.600+0000 7fa1656f5700 1 --2- 172.21.15.12:0/920146606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa160106300 0x7fa16010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.600+0000 7fa1656f5700 1 --2- 172.21.15.12:0/920146606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1601011e0 0x7fa1601015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.600+0000 7fa1656f5700 1 --2- 172.21.15.12:0/920146606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160101b90 0x7fa160105bd0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.600+0000 7fa1656f5700 1 -- 172.21.15.12:0/920146606 >> 172.21.15.12:0/920146606 conn(0x7fa1600fca80 msgr2=0x7fa1600feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:58.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.600+0000 7fa1656f5700 1 -- 172.21.15.12:0/920146606 shutdown_connections 2024-05-09T00:12:58.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.600+0000 7fa1656f5700 1 -- 172.21.15.12:0/920146606 wait complete. 2024-05-09T00:12:58.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa1656f5700 1 Processor -- start 2024-05-09T00:12:58.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa1656f5700 1 -- start start 2024-05-09T00:12:58.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa1656f5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1601011e0 0x7fa1600732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:58.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa1656f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160106300 0x7fa1600737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa1656f5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa160073d20 0x7fa1601a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa1656f5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1601a5b70 con 0x7fa160106300 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa1656f5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1601a5ce0 con 0x7fa1601011e0 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa1656f5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1601a5ea0 con 0x7fa160073d20 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa15e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160106300 0x7fa1600737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa15e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160106300 0x7fa1600737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40360/0 (socket says 172.21.15.12:40360) 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.601+0000 7fa15e7fc700 1 -- 172.21.15.12:0/559780423 learned_addr learned my addr 172.21.15.12:0/559780423 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa15f7fe700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa160073d20 0x7fa1601a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa15effd700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1601011e0 0x7fa1600732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa15e7fc700 1 -- 172.21.15.12:0/559780423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa160073d20 msgr2=0x7fa1601a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa15e7fc700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa160073d20 0x7fa1601a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa15e7fc700 1 -- 172.21.15.12:0/559780423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1601011e0 msgr2=0x7fa1600732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa15e7fc700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1601011e0 0x7fa1600732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa15e7fc700 1 -- 172.21.15.12:0/559780423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540095f0 con 0x7fa160106300 2024-05-09T00:12:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa15e7fc700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160106300 0x7fa1600737e0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fa1540059d0 tx=0x7fa154009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa147fff700 1 -- 172.21.15.12:0/559780423 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15401d070 con 0x7fa160106300 2024-05-09T00:12:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa147fff700 1 -- 172.21.15.12:0/559780423 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa15400fb30 con 0x7fa160106300 2024-05-09T00:12:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa147fff700 1 -- 172.21.15.12:0/559780423 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154022bf0 con 0x7fa160106300 2024-05-09T00:12:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa1656f5700 1 -- 172.21.15.12:0/559780423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1601a6120 con 0x7fa160106300 2024-05-09T00:12:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.602+0000 7fa1656f5700 1 -- 172.21.15.12:0/559780423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1601a64f0 con 0x7fa160106300 2024-05-09T00:12:58.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.603+0000 7fa1656f5700 1 -- 172.21.15.12:0/559780423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1601095d0 con 0x7fa160106300 2024-05-09T00:12:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.605+0000 7fa147fff700 1 -- 172.21.15.12:0/559780423 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa15400fca0 con 0x7fa160106300 2024-05-09T00:12:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.605+0000 7fa147fff700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa1480a1290 0x7fa1480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.605+0000 7fa147fff700 1 -- 172.21.15.12:0/559780423 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 8712+0+0 (secure 0 0 0) 0x7fa1540ce750 con 0x7fa160106300 2024-05-09T00:12:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.605+0000 7fa15effd700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa1480a1290 0x7fa1480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:58.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.606+0000 7fa15effd700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa1480a1290 0x7fa1480a3750 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa150005ea0 tx=0x7fa150005e30 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:58.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.610+0000 7fa147fff700 1 -- 172.21.15.12:0/559780423 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa154097b70 con 0x7fa160106300 2024-05-09T00:12:58.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:58 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3977087649' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T00:12:58.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:58 smithi012 ceph-mon[42503]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T00:12:58.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:58 smithi012 ceph-mon[42503]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T00:12:58.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:58 smithi012 ceph-mon[42503]: osdmap e141: 12 total, 12 up, 12 in 2024-05-09T00:12:58.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:58 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/3977087649' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T00:12:58.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:58 smithi012 ceph-mon[42503]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T00:12:58.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:58 smithi012 ceph-mon[42503]: pgmap v856: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-09T00:12:58.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:58 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2292670290' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:12:58.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:58 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3977087649' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T00:12:58.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:58 smithi204 ceph-mon[47373]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T00:12:58.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:58 smithi204 ceph-mon[47373]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T00:12:58.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:58 smithi204 ceph-mon[47373]: osdmap e141: 12 total, 12 up, 12 in 2024-05-09T00:12:58.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:58 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/3977087649' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T00:12:58.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:58 smithi204 ceph-mon[47373]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T00:12:58.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:58 smithi204 ceph-mon[47373]: pgmap v856: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-09T00:12:58.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:58 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2292670290' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:12:58.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:58 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3977087649' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T00:12:58.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:58 smithi160 ceph-mon[46152]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T00:12:58.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:58 smithi160 ceph-mon[46152]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T00:12:58.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:58 smithi160 ceph-mon[46152]: osdmap e141: 12 total, 12 up, 12 in 2024-05-09T00:12:58.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:58 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/3977087649' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T00:12:58.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:58 smithi160 ceph-mon[46152]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T00:12:58.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:58 smithi160 ceph-mon[46152]: pgmap v856: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-05-09T00:12:58.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:58 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2292670290' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:12:58.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:58.826+0000 7fa1656f5700 1 -- 172.21.15.12:0/559780423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa160074780 con 0x7fa160106300 2024-05-09T00:12:59.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.436+0000 7fa147fff700 1 -- 172.21.15.12:0/559780423 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa15409b190 con 0x7fa160106300 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.444+0000 7fa1656f5700 1 -- 172.21.15.12:0/559780423 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa1480a1290 msgr2=0x7fa1480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.444+0000 7fa1656f5700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa1480a1290 0x7fa1480a3750 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa150005ea0 tx=0x7fa150005e30 comp rx=0 tx=0).stop 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.444+0000 7fa1656f5700 1 -- 172.21.15.12:0/559780423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160106300 msgr2=0x7fa1600737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.444+0000 7fa1656f5700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160106300 0x7fa1600737e0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fa1540059d0 tx=0x7fa154009580 comp rx=0 tx=0).stop 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.445+0000 7fa1656f5700 1 -- 172.21.15.12:0/559780423 shutdown_connections 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.445+0000 7fa1656f5700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7fa1480a1290 0x7fa1480a3750 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.445+0000 7fa1656f5700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa160073d20 0x7fa1601a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.445+0000 7fa1656f5700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1601011e0 0x7fa1600732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.445+0000 7fa1656f5700 1 --2- 172.21.15.12:0/559780423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160106300 0x7fa1600737e0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.445+0000 7fa1656f5700 1 -- 172.21.15.12:0/559780423 >> 172.21.15.12:0/559780423 conn(0x7fa1600fca80 msgr2=0x7fa1600fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.445+0000 7fa1656f5700 1 -- 172.21.15.12:0/559780423 shutdown_connections 2024-05-09T00:12:59.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.445+0000 7fa1656f5700 1 -- 172.21.15.12:0/559780423 wait complete. 2024-05-09T00:12:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2024-05-09T00:12:59.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T00:12:59.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.662+0000 7f473ec4e700 1 -- 172.21.15.12:0/3526627177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4738106290 msgr2=0x7f473810de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.662+0000 7f473ec4e700 1 --2- 172.21.15.12:0/3526627177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4738106290 0x7f473810de10 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f473000b0d0 tx=0x7f473000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.662+0000 7f473ec4e700 1 -- 172.21.15.12:0/3526627177 shutdown_connections 2024-05-09T00:12:59.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.662+0000 7f473ec4e700 1 --2- 172.21.15.12:0/3526627177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4738106290 0x7f473810de10 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.662+0000 7f473ec4e700 1 --2- 172.21.15.12:0/3526627177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380682e0 0x7f47380686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.662+0000 7f473ec4e700 1 --2- 172.21.15.12:0/3526627177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4738068c90 0x7f4738105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.662+0000 7f473ec4e700 1 -- 172.21.15.12:0/3526627177 >> 172.21.15.12:0/3526627177 conn(0x7f4738076f40 msgr2=0x7f4738077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:59.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.662+0000 7f473ec4e700 1 -- 172.21.15.12:0/3526627177 shutdown_connections 2024-05-09T00:12:59.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.662+0000 7f473ec4e700 1 -- 172.21.15.12:0/3526627177 wait complete. 2024-05-09T00:12:59.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.663+0000 7f473ec4e700 1 Processor -- start 2024-05-09T00:12:59.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.663+0000 7f473ec4e700 1 -- start start 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.663+0000 7f473ec4e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380682e0 0x7f47381974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.663+0000 7f473ec4e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4738106290 0x7f4738197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.663+0000 7f473ec4e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f473819bdb0 0x7f473819c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.663+0000 7f473ec4e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473819c830 con 0x7f473819bdb0 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.663+0000 7f473ec4e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473819c9a0 con 0x7f47380682e0 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.663+0000 7f473ec4e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473819cb60 con 0x7f4738106290 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.663+0000 7f473d1eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f473819bdb0 0x7f473819c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f473d1eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f473819bdb0 0x7f473819c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40384/0 (socket says 172.21.15.12:40384) 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f4737fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4738106290 0x7f4738197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f473d1eb700 1 -- 172.21.15.12:0/2730700709 learned_addr learned my addr 172.21.15.12:0/2730700709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f473c9ea700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380682e0 0x7f47381974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f473d1eb700 1 -- 172.21.15.12:0/2730700709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4738106290 msgr2=0x7f4738197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f473d1eb700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4738106290 0x7f4738197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f473d1eb700 1 -- 172.21.15.12:0/2730700709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380682e0 msgr2=0x7f47381974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f473d1eb700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380682e0 0x7f47381974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f473d1eb700 1 -- 172.21.15.12:0/2730700709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728009520 con 0x7f473819bdb0 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f4737fff700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4738106290 0x7f4738197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f473d1eb700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f473819bdb0 0x7f473819c230 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f473000bcd0 tx=0x7f473000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f4735ffb700 1 -- 172.21.15.12:0/2730700709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473000e040 con 0x7f473819bdb0 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f4735ffb700 1 -- 172.21.15.12:0/2730700709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4730005070 con 0x7f473819bdb0 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f473ec4e700 1 -- 172.21.15.12:0/2730700709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4730009d00 con 0x7f473819bdb0 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.664+0000 7f4735ffb700 1 -- 172.21.15.12:0/2730700709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473001b6a0 con 0x7f473819bdb0 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.665+0000 7f473ec4e700 1 -- 172.21.15.12:0/2730700709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47381a42a0 con 0x7f473819bdb0 2024-05-09T00:12:59.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.666+0000 7f473ec4e700 1 -- 172.21.15.12:0/2730700709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47381094f0 con 0x7f473819bdb0 2024-05-09T00:12:59.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.667+0000 7f4735ffb700 1 -- 172.21.15.12:0/2730700709 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f47300051e0 con 0x7f473819bdb0 2024-05-09T00:12:59.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.668+0000 7f4735ffb700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47240a1290 0x7f47240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.668+0000 7f473c9ea700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47240a1290 0x7f47240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.668+0000 7f4735ffb700 1 -- 172.21.15.12:0/2730700709 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 8216+0+0 (secure 0 0 0) 0x7f47300cd8f0 con 0x7f473819bdb0 2024-05-09T00:12:59.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.668+0000 7f473c9ea700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47240a1290 0x7f47240a3750 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4728009df0 tx=0x7f4728009260 comp rx=0 tx=0).ready entity=mgr.15252 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:59.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T00:12:59.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: Cluster is now healthy 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6835/3938901310,v1:172.21.15.204:6839/3938901310] up:boot 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6832/1565951811,v1:172.21.15.204:6833/1565951811] up:boot 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.204:6834/2013031842,v1:172.21.15.204:6838/2013031842] up:boot 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6838/867074573,v1:172.21.15.160:6839/867074573] up:boot 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6834/2555539822,v1:172.21.15.160:6835/2555539822] up:boot 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: mds.? [v2:172.21.15.160:6836/2365862152,v1:172.21.15.160:6837/2365862152] up:boot 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2292670290' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: fsmap 12 up:standby 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:12:59.670 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:12:59.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:12:59.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:12:59.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:12:59.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:12:59 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/559780423' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:12:59.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.673+0000 7f4735ffb700 1 -- 172.21.15.12:0/2730700709 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4730096ea0 con 0x7f473819bdb0 2024-05-09T00:12:59.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T00:12:59.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T00:12:59.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: Cluster is now healthy 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6835/3938901310,v1:172.21.15.204:6839/3938901310] up:boot 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6832/1565951811,v1:172.21.15.204:6833/1565951811] up:boot 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.204:6834/2013031842,v1:172.21.15.204:6838/2013031842] up:boot 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6838/867074573,v1:172.21.15.160:6839/867074573] up:boot 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6834/2555539822,v1:172.21.15.160:6835/2555539822] up:boot 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: mds.? [v2:172.21.15.160:6836/2365862152,v1:172.21.15.160:6837/2365862152] up:boot 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2292670290' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: fsmap 12 up:standby 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:12:59.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:12:59.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:12:59 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/559780423' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:12:59.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T00:12:59.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T00:12:59.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: Cluster is now healthy 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6835/3938901310,v1:172.21.15.204:6839/3938901310] up:boot 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6832/1565951811,v1:172.21.15.204:6833/1565951811] up:boot 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.204:6834/2013031842,v1:172.21.15.204:6838/2013031842] up:boot 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6838/867074573,v1:172.21.15.160:6839/867074573] up:boot 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6834/2555539822,v1:172.21.15.160:6835/2555539822] up:boot 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: mds.? [v2:172.21.15.160:6836/2365862152,v1:172.21.15.160:6837/2365862152] up:boot 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2292670290' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: fsmap 12 up:standby 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: from='mgr.15252 172.21.15.12:0/161467325' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:12:59.769 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:12:59 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/559780423' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:12:59.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:12:59.975+0000 7f473ec4e700 1 -- 172.21.15.12:0/2730700709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4738102640 con 0x7f473819bdb0 2024-05-09T00:13:00.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.442+0000 7f4735ffb700 1 -- 172.21.15.12:0/2730700709 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) v1 ==== 169+0+0 (secure 0 0 0) 0x7f473009a4c0 con 0x7f473819bdb0 2024-05-09T00:13:00.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.450+0000 7f473ec4e700 1 -- 172.21.15.12:0/2730700709 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47240a1290 msgr2=0x7f47240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:00.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.450+0000 7f473ec4e700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47240a1290 0x7f47240a3750 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4728009df0 tx=0x7f4728009260 comp rx=0 tx=0).stop 2024-05-09T00:13:00.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.450+0000 7f473ec4e700 1 -- 172.21.15.12:0/2730700709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f473819bdb0 msgr2=0x7f473819c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:00.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.450+0000 7f473ec4e700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f473819bdb0 0x7f473819c230 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f473000bcd0 tx=0x7f473000bd00 comp rx=0 tx=0).stop 2024-05-09T00:13:00.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.451+0000 7f473ec4e700 1 -- 172.21.15.12:0/2730700709 shutdown_connections 2024-05-09T00:13:00.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.451+0000 7f473ec4e700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.12:6800/2525408937,v1:172.21.15.12:6801/2525408937] conn(0x7f47240a1290 0x7f47240a3750 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:00.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.451+0000 7f473ec4e700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f473819bdb0 0x7f473819c230 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:00.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.451+0000 7f473ec4e700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47380682e0 0x7f47381974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:00.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.451+0000 7f473ec4e700 1 --2- 172.21.15.12:0/2730700709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4738106290 0x7f4738197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:00.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.451+0000 7f473ec4e700 1 -- 172.21.15.12:0/2730700709 >> 172.21.15.12:0/2730700709 conn(0x7f4738076f40 msgr2=0x7f47380fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:00.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.455+0000 7f473ec4e700 1 -- 172.21.15.12:0/2730700709 shutdown_connections 2024-05-09T00:13:00.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-09T00:13:00.455+0000 7f473ec4e700 1 -- 172.21.15.12:0/2730700709 wait complete. 2024-05-09T00:13:00.453 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2024-05-09T00:13:00.474 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:13:00.506 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:13:00.565 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:13:00.609 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T00:13:00.609 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T00:13:00.629 DEBUG:teuthology.orchestra.run.smithi160:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T00:13:00.646 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T00:13:00.694 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T00:13:00.695 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T00:13:00.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.a 2024-05-09T00:13:00.736 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 00:13:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-09T00:13:00.460+0000 7f7c5bbb8700 -1 client.15276: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-05-09T00:13:00.736 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 00:13:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-09T00:13:00.462+0000 7f7c5bbb8700 -1 client.15276: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-05-09T00:13:00.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:13:00 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/559780423' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T00:13:00.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:13:00 smithi012 ceph-mon[42503]: osdmap e142: 12 total, 12 up, 12 in 2024-05-09T00:13:00.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:13:00 smithi012 ceph-mon[42503]: pgmap v858: 65 pgs: 65 active+clean; 357 MiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:13:00.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:13:00 smithi012 ceph-mon[42503]: from='client.? 172.21.15.12:0/2730700709' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:13:00.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:13:00 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/559780423' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T00:13:00.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:13:00 smithi204 ceph-mon[47373]: osdmap e142: 12 total, 12 up, 12 in 2024-05-09T00:13:00.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:13:00 smithi204 ceph-mon[47373]: pgmap v858: 65 pgs: 65 active+clean; 357 MiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:13:00.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:13:00 smithi204 ceph-mon[47373]: from='client.? 172.21.15.12:0/2730700709' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:13:00.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:13:00 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/559780423' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T00:13:00.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:13:00 smithi160 ceph-mon[46152]: osdmap e142: 12 total, 12 up, 12 in 2024-05-09T00:13:00.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:13:00 smithi160 ceph-mon[46152]: pgmap v858: 65 pgs: 65 active+clean; 357 MiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:13:00.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:13:00 smithi160 ceph-mon[46152]: from='client.? 172.21.15.12:0/2730700709' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:13:01.101 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:13:00 smithi012 systemd[1]: Stopping Ceph mon.a for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:01.401 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:13:01 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-a[42479]: 2024-05-09T00:13:01.100+0000 7ff285476700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:01.401 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:13:01 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-a[42479]: 2024-05-09T00:13:01.100+0000 7ff285476700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T00:13:01.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 09 00:13:01 smithi012 bash[143663]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-a 2024-05-09T00:13:01.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.a.service' 2024-05-09T00:13:01.960 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:01.960 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T00:13:01.960 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T00:13:01.960 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.b 2024-05-09T00:13:02.268 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:13:01 smithi160 systemd[1]: Stopping Ceph mon.b for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:02.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:13:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-b[46129]: 2024-05-09T00:13:02.275+0000 7f9491ab8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:02.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:13:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-b[46129]: 2024-05-09T00:13:02.275+0000 7f9491ab8700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T00:13:02.768 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 00:13:02 smithi160 bash[92421]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-b 2024-05-09T00:13:02.854 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.b.service' 2024-05-09T00:13:02.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:02.898 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T00:13:02.898 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T00:13:02.898 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.c 2024-05-09T00:13:03.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:13:02 smithi204 systemd[1]: Stopping Ceph mon.c for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:03.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:13:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-c[47349]: 2024-05-09T00:13:03.133+0000 7fd991065700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:03.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:13:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-c[47349]: 2024-05-09T00:13:03.133+0000 7fd991065700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T00:13:03.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 09 00:13:03 smithi204 bash[95781]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mon-c 2024-05-09T00:13:03.701 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mon.c.service' 2024-05-09T00:13:03.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:03.750 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T00:13:03.750 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T00:13:03.750 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.x 2024-05-09T00:13:04.041 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 00:13:03 smithi012 systemd[1]: Stopping Ceph mgr.x for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:04.041 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 00:13:04 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x[42752]: 2024-05-09T00:13:04.012+0000 7f7cf53c3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:04.417 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 09 00:13:04 smithi012 bash[143797]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-x 2024-05-09T00:13:04.466 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-09T00:13:04.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:04.513 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T00:13:04.514 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T00:13:04.514 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.y 2024-05-09T00:13:04.843 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 00:13:04 smithi160 systemd[1]: Stopping Ceph mgr.y for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:05.192 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 00:13:04 smithi160 bash[92559]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-y 2024-05-09T00:13:05.192 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 00:13:04 smithi160 systemd[1]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-09T00:13:05.198 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-09T00:13:05.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:05.235 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T00:13:05.235 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T00:13:05.236 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.z 2024-05-09T00:13:05.601 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 09 00:13:05 smithi204 systemd[1]: Stopping Ceph mgr.z for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:05.923 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 09 00:13:05 smithi204 bash[95922]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mgr-z 2024-05-09T00:13:05.923 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 09 00:13:05 smithi204 systemd[1]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-09T00:13:05.929 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-09T00:13:05.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:05.967 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T00:13:05.967 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T00:13:05.967 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.0 2024-05-09T00:13:06.418 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:13:06 smithi012 systemd[1]: Stopping Ceph osd.0 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:06.418 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:13:06 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-0[60992]: 2024-05-09T00:13:06.212+0000 7fbab6f8d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:06.418 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:13:06 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-0[60992]: 2024-05-09T00:13:06.212+0000 7fbab6f8d700 -1 osd.0 143 *** Got signal Terminated *** 2024-05-09T00:13:06.419 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:13:06 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-0[60992]: 2024-05-09T00:13:06.212+0000 7fbab6f8d700 -1 osd.0 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:13:12.168 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 09 00:13:11 smithi012 bash[143939]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-0 2024-05-09T00:13:13.209 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.0.service' 2024-05-09T00:13:13.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:13.262 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T00:13:13.262 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T00:13:13.262 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.1 2024-05-09T00:13:13.711 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:13:13 smithi160 systemd[1]: Stopping Ceph osd.1 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:14.018 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:13:13 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-1[51153]: 2024-05-09T00:13:13.713+0000 7fac164da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:14.018 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:13:13 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-1[51153]: 2024-05-09T00:13:13.713+0000 7fac164da700 -1 osd.1 143 *** Got signal Terminated *** 2024-05-09T00:13:14.018 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:13:13 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-1[51153]: 2024-05-09T00:13:13.713+0000 7fac164da700 -1 osd.1 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:13:19.518 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 00:13:19 smithi160 bash[92702]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-1 2024-05-09T00:13:20.657 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.1.service' 2024-05-09T00:13:20.698 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:20.699 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T00:13:20.699 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T00:13:20.699 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.2 2024-05-09T00:13:21.016 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:13:20 smithi204 systemd[1]: Stopping Ceph osd.2 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:21.016 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:13:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-2[53134]: 2024-05-09T00:13:20.939+0000 7f019048c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:21.016 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:13:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-2[53134]: 2024-05-09T00:13:20.939+0000 7f019048c700 -1 osd.2 143 *** Got signal Terminated *** 2024-05-09T00:13:21.016 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:13:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-2[53134]: 2024-05-09T00:13:20.939+0000 7f019048c700 -1 osd.2 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:13:27.016 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 09 00:13:26 smithi204 bash[96067]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-2 2024-05-09T00:13:28.108 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.2.service' 2024-05-09T00:13:28.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:28.149 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T00:13:28.149 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T00:13:28.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.3 2024-05-09T00:13:28.667 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:13:28 smithi012 systemd[1]: Stopping Ceph osd.3 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:28.668 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:13:28 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-3[68974]: 2024-05-09T00:13:28.408+0000 7feca997b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:28.668 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:13:28 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-3[68974]: 2024-05-09T00:13:28.408+0000 7feca997b700 -1 osd.3 143 *** Got signal Terminated *** 2024-05-09T00:13:28.668 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:13:28 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-3[68974]: 2024-05-09T00:13:28.408+0000 7feca997b700 -1 osd.3 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:13:34.418 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 09 00:13:34 smithi012 bash[144207]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-3 2024-05-09T00:13:34.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:34 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:34.065+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:34.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:34 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:34.294+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:34.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:34 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:34.500+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:35.377 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:35 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:35.026+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:35.387 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.3.service' 2024-05-09T00:13:35.439 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:35.440 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T00:13:35.440 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T00:13:35.440 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.4 2024-05-09T00:13:35.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:35 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:35.132+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:35.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:35 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:35.072+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:35.518 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:13:35 smithi160 systemd[1]: Stopping Ceph osd.4 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:35.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:35 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:35.257+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:35.668 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:35 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:35.548+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:36.018 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:13:35 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-4[59112]: 2024-05-09T00:13:35.710+0000 7f3f0b5e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:36.018 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:13:35 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-4[59112]: 2024-05-09T00:13:35.710+0000 7f3f0b5e5700 -1 osd.4 143 *** Got signal Terminated *** 2024-05-09T00:13:36.018 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:13:35 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-4[59112]: 2024-05-09T00:13:35.710+0000 7f3f0b5e5700 -1 osd.4 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:13:36.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:36.062+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:36.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:36 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:36.132+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:36.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:36 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:36.122+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:36.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:36 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:36.212+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:36.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:36 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:36.584+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:37.417 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:37 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:37.058+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:37.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:37 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:37.083+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:37.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:37 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:37.155+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:37.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:37 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:37.233+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:37.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:37 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:37.625+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:38.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:38 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:38.020+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:38.516 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:38 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:38.072+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:38.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:38 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:38.148+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:38.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:38 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:38.220+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:38.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:38 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:38.664+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:39.417 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:39 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:39.019+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:39.516 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:39 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:39.053+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:39.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:39 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:39.099+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:39.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:39 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:39.229+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:39.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:39 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:39.647+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:40.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:40 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:40.059+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:40.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:40 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:40.059+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:40.516 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:40 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:40.083+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:40.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:40 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:40.071+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:40.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:40 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:40.220+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:40.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:40 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:40.220+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:40.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:40 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:40.645+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:41.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:41 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:41.095+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:41.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:41 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:41.095+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:41.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:41 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:41.060+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:41.516 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:41 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:41.133+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:41.516 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:41 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:41.133+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:14.978423+0000 front 2024-05-09T00:13:14.978357+0000 (oldest deadline 2024-05-09T00:13:40.878583+0000) 2024-05-09T00:13:41.518 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 00:13:41 smithi160 bash[92963]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-4 2024-05-09T00:13:41.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:41 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:41.216+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:41.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:41 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:41.216+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:41.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:41 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:41.639+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:42.264 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:41 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:41.988+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:42.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:42 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:42.053+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:42.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:42 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:42.053+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:42.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:42 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:42.055+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:42.516 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:42 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:42.133+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:42.516 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:42 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:42.133+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:14.978423+0000 front 2024-05-09T00:13:14.978357+0000 (oldest deadline 2024-05-09T00:13:40.878583+0000) 2024-05-09T00:13:42.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:42 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:42.265+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:42.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:42 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:42.265+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:42.611 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.4.service' 2024-05-09T00:13:42.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:42.653 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T00:13:42.654 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T00:13:42.654 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.5 2024-05-09T00:13:42.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:42 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:42.590+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:42.956 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:42 smithi204 systemd[1]: Stopping Ceph osd.5 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:42.956 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:42 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:42.912+0000 7fe5afbc7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:42.956 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:42 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:42.912+0000 7fe5afbc7700 -1 osd.5 143 *** Got signal Terminated *** 2024-05-09T00:13:42.956 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:42 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:42.912+0000 7fe5afbc7700 -1 osd.5 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:13:43.265 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:43 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:43.016+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:43.266 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:43 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:43.093+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:43.266 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:43 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:43.093+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:14.978423+0000 front 2024-05-09T00:13:14.978357+0000 (oldest deadline 2024-05-09T00:13:40.878583+0000) 2024-05-09T00:13:43.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:42 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:42.958+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:43.286 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:43 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:43.034+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:43.417 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:43 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:43.026+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:43.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:43 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:43.026+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:43.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:43 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:43.289+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:43.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:43 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:43.289+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:43.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:43 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:43.596+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:43.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:43 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:43.596+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:44.265 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:44 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:44.012+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:44.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:43 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:43.965+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:44.266 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:44 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:44.127+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:44.266 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:44 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:44.127+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:14.978423+0000 front 2024-05-09T00:13:14.978357+0000 (oldest deadline 2024-05-09T00:13:40.878583+0000) 2024-05-09T00:13:44.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:44 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:44.051+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:44.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:44 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:44.051+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:44.417 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:43 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:43.982+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:44.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:43 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:43.982+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:44.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:44 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:44.267+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:44.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:44 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:44.267+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:44.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:44 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:44.561+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:44.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:44 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:44.561+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:45.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:44 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:44.935+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:45.266 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:45 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:45.149+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:45.266 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:45 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:45.149+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:14.978423+0000 front 2024-05-09T00:13:14.978357+0000 (oldest deadline 2024-05-09T00:13:40.878583+0000) 2024-05-09T00:13:45.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:45 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:45.091+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:45.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:45 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:45.091+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:45.267 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:44 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:44.993+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:45.417 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:44 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:44.995+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:45.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:44 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:44.995+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:45.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:45 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:45.269+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:45.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:45 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:45.269+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:45.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:45 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:45.564+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:45.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:45 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:45.564+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:46.265 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:45 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:45.955+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:46.266 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:46 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:46.119+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:46.266 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:46 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:46.119+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:14.978423+0000 front 2024-05-09T00:13:14.978357+0000 (oldest deadline 2024-05-09T00:13:40.878583+0000) 2024-05-09T00:13:46.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:46 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:46.045+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:46.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:46 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:46.045+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:46.268 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:46 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:45.999+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:46.417 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:46 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:46.027+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:46.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:46 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:46.027+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:46.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:46 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:46.302+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:46.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:46 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:46.302+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:46.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:46 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:46.525+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:46.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:46 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:46.525+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:47.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:46 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:46.945+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:47.266 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:47.115+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.077863+0000 front 2024-05-09T00:13:09.077943+0000 (oldest deadline 2024-05-09T00:13:34.977893+0000) 2024-05-09T00:13:47.267 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5[61741]: 2024-05-09T00:13:47.115+0000 7fe5a89b6700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:14.978423+0000 front 2024-05-09T00:13:14.978357+0000 (oldest deadline 2024-05-09T00:13:40.878583+0000) 2024-05-09T00:13:47.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:47.073+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:47.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:47.073+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:47.268 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:46 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:46.991+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:47.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:47 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:47.061+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:47.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:47 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:47.061+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:47.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:47 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:47.282+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:47.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:47 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:47.282+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:47.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:47 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:47.533+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:47.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:47 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:47.533+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:48.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:47 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:47.920+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:48.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:48 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:48.044+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:48.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:48 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:48.044+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:48.303 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:48 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:48.028+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:48.304 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:48 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:48.028+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:48.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:48 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:48.073+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:48.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:48 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:48.073+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:48.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:48 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:48.073+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.860350+0000 front 2024-05-09T00:13:23.860405+0000 (oldest deadline 2024-05-09T00:13:47.960546+0000) 2024-05-09T00:13:48.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:48 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:48.306+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:48.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:48 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:48.306+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:48.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:48 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:48.554+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:48.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:48 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:48.554+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:49.016 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 09 00:13:48 smithi204 bash[96323]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-5 2024-05-09T00:13:49.016 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:48 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:48.874+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:49.308 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:49 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:49.058+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:49.308 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:49 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:49.058+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:49.308 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:49 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:49.058+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:49.343 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:49 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:49.074+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:49.344 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:49 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:49.074+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:49.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:49.048+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:49.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:49.048+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:49.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:49.048+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.860350+0000 front 2024-05-09T00:13:23.860405+0000 (oldest deadline 2024-05-09T00:13:47.960546+0000) 2024-05-09T00:13:49.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:49 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:49.346+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:49.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:49 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:49.346+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:49.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:49.508+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:49.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:49 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:49.508+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:49.990 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:49 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:49.902+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:49.990 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:49 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:49.902+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:13:49.997 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.5.service' 2024-05-09T00:13:50.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:50.039 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T00:13:50.039 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T00:13:50.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.6 2024-05-09T00:13:50.265 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:50 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:50.087+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:50.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:50 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:50.087+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:50.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:50 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:50.087+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:50.302 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:50.043+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:50.302 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:50.043+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:50.302 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:50.043+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.860350+0000 front 2024-05-09T00:13:23.860405+0000 (oldest deadline 2024-05-09T00:13:47.960546+0000) 2024-05-09T00:13:50.302 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:50 smithi012 systemd[1]: Stopping Ceph osd.6 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:50.302 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:50.303+0000 7fd3d0438700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:50.302 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:50.303+0000 7fd3d0438700 -1 osd.6 143 *** Got signal Terminated *** 2024-05-09T00:13:50.319 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:50.036+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:50.319 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:50.036+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:50.668 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:50.303+0000 7fd3d0438700 -1 osd.6 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:13:50.668 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:50.458+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:50.668 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:50 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:50.458+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:50.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:50.321+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:50.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:50.321+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:50.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:50.321+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:13:51.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:50 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:50.890+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:51.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:50 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:50.890+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:13:51.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:51 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:51.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:51.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:51 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:51.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:51.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:51 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:51.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:51.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:51 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:51.050+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:51.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:51 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:51.050+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:51.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:51 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:51.050+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.860350+0000 front 2024-05-09T00:13:23.860405+0000 (oldest deadline 2024-05-09T00:13:47.960546+0000) 2024-05-09T00:13:51.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:51 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:51.052+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:51.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:51 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:51.052+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:51.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:51 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:51.297+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:51.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:51 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:51.297+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:51.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:51 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:51.297+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:13:51.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:51 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:51.468+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:51.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:51 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:51.468+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:52.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:51 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:51.928+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:52.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:51 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:51.928+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:13:52.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:52 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:52.063+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:52.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:52 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:52.063+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:52.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:52 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:52.063+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:52.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:52.059+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:52.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:52.059+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:52.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:52.059+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.860350+0000 front 2024-05-09T00:13:23.860405+0000 (oldest deadline 2024-05-09T00:13:47.960546+0000) 2024-05-09T00:13:52.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:52 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:52.068+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:52.519 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:52 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:52.068+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:52.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:52 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:52.299+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:52.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:52 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:52.299+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:52.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:52 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:52.299+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:13:52.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:52.492+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:52.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:52.492+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:52.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:52 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:52.492+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:13:53.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:52 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:52.900+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:53.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:52 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:52.900+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:13:53.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:53 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:53.067+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:53.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:53 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:53.067+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:53.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:53 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:53.067+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:53.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:53 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:53.025+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:53.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:53 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:53.025+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:53.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:53 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:53.025+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.860350+0000 front 2024-05-09T00:13:23.860405+0000 (oldest deadline 2024-05-09T00:13:47.960546+0000) 2024-05-09T00:13:53.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:53 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:53.297+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:53.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:53 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:53.297+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:53.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:53 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:53.297+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:13:53.519 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:53 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:53.117+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:53.519 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:53 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:53.117+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:53.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:53 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:53.502+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:53.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:53 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:53.502+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:53.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:53 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:53.502+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:13:54.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:53 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:53.896+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:54.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:53 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:53.896+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:13:54.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:54 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:54.061+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:54.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:54 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:54.061+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:54.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:54 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:54.061+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:54.417 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:54 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:54.025+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:54.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:54 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:54.025+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:54.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:54 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:54.025+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.860350+0000 front 2024-05-09T00:13:23.860405+0000 (oldest deadline 2024-05-09T00:13:47.960546+0000) 2024-05-09T00:13:54.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:54 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:54.097+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:54.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:54 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:54.097+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:54.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:54 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:54.269+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:54.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:54 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:54.269+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:54.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:54 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:54.269+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:13:54.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:54 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:54.453+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:54.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:54 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:54.453+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:54.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:54 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:54.453+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:13:55.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:54 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:54.945+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:55.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:54 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:54.945+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:13:55.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:55 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:55.020+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:55.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:55 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:55.020+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:55.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:55 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:55.020+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:55.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:55 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:55.020+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:13:55.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:55 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:55.028+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.857856+0000 front 2024-05-09T00:13:09.857882+0000 (oldest deadline 2024-05-09T00:13:33.958113+0000) 2024-05-09T00:13:55.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:55 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:55.028+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:13.958683+0000 front 2024-05-09T00:13:13.958534+0000 (oldest deadline 2024-05-09T00:13:39.258707+0000) 2024-05-09T00:13:55.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:55 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6[77728]: 2024-05-09T00:13:55.028+0000 7fd3cd22f700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.860350+0000 front 2024-05-09T00:13:23.860405+0000 (oldest deadline 2024-05-09T00:13:47.960546+0000) 2024-05-09T00:13:55.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:55 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:55.100+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:55.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:55 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:55.100+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:55.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:55 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:55.294+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:55.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:55 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:55.294+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:55.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:55 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:55.294+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:13:55.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:55 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:55.294+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:13:55.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:55 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:55.500+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:55.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:55 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:55.500+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:55.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:55 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:55.500+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:13:55.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:55 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:55.500+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:13:56.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:55 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:55.938+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:56.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:55 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:55.938+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:13:56.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:56 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:56.013+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:56.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:56 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:56.013+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:56.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:56 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:56.013+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:56.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:56 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:56.013+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:13:56.418 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 09 00:13:56 smithi012 bash[144460]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-6 2024-05-09T00:13:56.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:56 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:56.129+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:56.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:56 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:56.129+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:56.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:56 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:56.256+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:56.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:56 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:56.256+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:56.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:56 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:56.256+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:13:56.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:56 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:56.256+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:13:56.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:56 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:56.541+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:56.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:56 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:56.541+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:56.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:56 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:56.541+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:13:56.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:56 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:56.541+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:13:57.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:56 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:56.893+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:57.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:56 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:56.893+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:13:57.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:57.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:57.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:57.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:57.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:57.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:57.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:57.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:13:57.282 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.6.service' 2024-05-09T00:13:57.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:13:57.335 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T00:13:57.335 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T00:13:57.335 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.7 2024-05-09T00:13:57.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:57.169+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:57.519 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:57.169+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:57.519 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:57.169+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:32.808505+0000 front 2024-05-09T00:13:32.808654+0000 (oldest deadline 2024-05-09T00:13:56.308694+0000) 2024-05-09T00:13:57.519 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:57 smithi160 systemd[1]: Stopping Ceph osd.7 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:13:57.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:57.299+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:57.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:57.299+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:57.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:57.299+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:13:57.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:57.299+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:13:57.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:57 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:57.579+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:57.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:57 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:57.579+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:57.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:57 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:57.579+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:13:57.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:57 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:57.579+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:13:58.018 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:57.572+0000 7f709dc26700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:13:58.018 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:57.572+0000 7f709dc26700 -1 osd.7 143 *** Got signal Terminated *** 2024-05-09T00:13:58.018 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:57 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:57.572+0000 7f709dc26700 -1 osd.7 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:13:58.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:57.861+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:58.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:57.861+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:13:58.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:57 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:57.861+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:13:58.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:58 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:58.043+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:58.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:58 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:58.043+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:58.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:58 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:58.043+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:58.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:58 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:58.043+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:13:58.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:58 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:58.170+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:58.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:58 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:58.170+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:58.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:58 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:58.170+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:32.808505+0000 front 2024-05-09T00:13:32.808654+0000 (oldest deadline 2024-05-09T00:13:56.308694+0000) 2024-05-09T00:13:58.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:58 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:58.324+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:58.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:58 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:58.324+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:58.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:58 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:58.324+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:13:58.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:58 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:58.324+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:13:58.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:58 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:58.543+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:58.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:58 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:58.543+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:58.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:58 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:58.543+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:13:58.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:58 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:58.543+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:13:59.085 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:58 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:58.823+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:13:59.085 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:58 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:58.823+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:13:59.085 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:58 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:58.823+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:13:59.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:59 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:59.086+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:13:59.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:59 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:59.086+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:13:59.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:59 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:59.086+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:13:59.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:13:59 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:13:59.086+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:13:59.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:59 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:59.193+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:13:59.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:59 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:59.193+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:13:59.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:13:59 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:13:59.193+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:32.808505+0000 front 2024-05-09T00:13:32.808654+0000 (oldest deadline 2024-05-09T00:13:56.308694+0000) 2024-05-09T00:13:59.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:59 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:59.307+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:13:59.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:59 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:59.307+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:13:59.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:59 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:59.307+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:13:59.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:13:59 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:13:59.307+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:13:59.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:59 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:59.562+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:13:59.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:59 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:59.562+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:13:59.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:59 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:59.562+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:13:59.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:13:59 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:13:59.562+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:00.092 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:59 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:59.822+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:14:00.092 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:59 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:59.822+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:14:00.092 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:13:59 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:13:59.822+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:14:00.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:00 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:00.094+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:00.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:00 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:00.094+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:00.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:00 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:00.094+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:00.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:00 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:00.094+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:00.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:14:00 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:14:00.230+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:14:00.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:14:00 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:14:00.230+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:14:00.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:14:00 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:14:00.230+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:32.808505+0000 front 2024-05-09T00:13:32.808654+0000 (oldest deadline 2024-05-09T00:13:56.308694+0000) 2024-05-09T00:14:00.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:00 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:00.337+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:00.518 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:00 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:00.337+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:00.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:00 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:00.337+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:00.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:00 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:00.337+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:00.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:00.601+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:00.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:00.601+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:00.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:00.601+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:00.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:00 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:00.601+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:01.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:00 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:00.820+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:14:01.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:00 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:00.820+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:14:01.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:00 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:00.820+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:14:01.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:01 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:01.049+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:01.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:01 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:01.049+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:01.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:01 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:01.049+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:01.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:01 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:01.049+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:01.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:14:01 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:14:01.216+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:14:01.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:14:01 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:14:01.216+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:14:01.519 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:14:01 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:14:01.216+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:32.808505+0000 front 2024-05-09T00:13:32.808654+0000 (oldest deadline 2024-05-09T00:13:56.308694+0000) 2024-05-09T00:14:01.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:01 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:01.332+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:01.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:01 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:01.332+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:01.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:01 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:01.332+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:01.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:01 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:01.332+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:01.917 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:01 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:01.615+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:01.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:01 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:01.615+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:01.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:01 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:01.615+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:01.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:01 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:01.615+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:02.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:01 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:01.820+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:14:02.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:01 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:01.820+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:14:02.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:01 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:01.820+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:14:02.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:02.026+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:02.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:02.026+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:02.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:02.026+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:02.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:02.026+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:02.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:14:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:14:02.179+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.005485+0000 front 2024-05-09T00:13:17.005460+0000 (oldest deadline 2024-05-09T00:13:41.705687+0000) 2024-05-09T00:14:02.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:14:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:14:02.179+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:21.706472+0000 front 2024-05-09T00:13:21.706488+0000 (oldest deadline 2024-05-09T00:13:47.006608+0000) 2024-05-09T00:14:02.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:14:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7[67715]: 2024-05-09T00:14:02.179+0000 7f7096a15700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:32.808505+0000 front 2024-05-09T00:13:32.808654+0000 (oldest deadline 2024-05-09T00:13:56.308694+0000) 2024-05-09T00:14:02.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:02.370+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:02.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:02.370+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:02.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:02.370+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:02.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:02 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:02.370+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:02.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:02 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:02.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:02.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:02 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:02.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:02.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:02 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:02.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:02.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:02 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:02.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:03.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:02.839+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:14:03.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:02.839+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:14:03.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:02.839+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:14:03.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:02.839+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.659350+0000 front 2024-05-09T00:13:40.659419+0000 (oldest deadline 2024-05-09T00:14:02.359354+0000) 2024-05-09T00:14:03.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:02.985+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:03.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:02.985+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:03.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:02.985+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:03.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:02.985+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:03.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:02 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:02.985+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:03.518 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 00:14:03 smithi160 bash[93225]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-7 2024-05-09T00:14:03.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:03 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:03.384+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:03.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:03 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:03.384+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:03.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:03 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:03.384+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:03.519 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:03 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:03.384+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:03.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:03 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:03.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:03.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:03 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:03.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:03.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:03 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:03.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:03.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:03 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:03.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:03.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:03 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:03.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:04.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:03.796+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:14:04.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:03.796+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:14:04.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:03.796+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:14:04.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:03.796+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.659350+0000 front 2024-05-09T00:13:40.659419+0000 (oldest deadline 2024-05-09T00:14:02.359354+0000) 2024-05-09T00:14:04.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:03.991+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:04.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:03.991+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:04.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:03.991+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:04.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:03.991+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:04.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:03.991+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:04.275 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.7.service' 2024-05-09T00:14:04.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:04.315 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T00:14:04.315 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T00:14:04.315 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.8 2024-05-09T00:14:04.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:04 smithi204 systemd[1]: Stopping Ceph osd.8 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:04.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:04.567+0000 7f2eefe50700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:04.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:04.567+0000 7f2eefe50700 -1 osd.8 143 *** Got signal Terminated *** 2024-05-09T00:14:04.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:04.567+0000 7f2eefe50700 -1 osd.8 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:14:04.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:04 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:04.411+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:04.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:04 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:04.411+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:04.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:04 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:04.411+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:04.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:04 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:04.411+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:04.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:04 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:04.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:04.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:04 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:04.652+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:04.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:04 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:04.653+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:04.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:04 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:04.653+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:04.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:04 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:04.653+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:05.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:04.813+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:14:05.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:04.813+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:14:05.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:04.813+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:14:05.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:04 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:04.813+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.659350+0000 front 2024-05-09T00:13:40.659419+0000 (oldest deadline 2024-05-09T00:14:02.359354+0000) 2024-05-09T00:14:05.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:05.024+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:05.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:05.024+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:05.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:05.024+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:05.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:05.024+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:05.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:05.024+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:05.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:05 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:05.428+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:05.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:05 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:05.428+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:05.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:05 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:05.428+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:05.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:05 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:05.428+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:05.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:05 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:05.428+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:05.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:05 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:05.630+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:05.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:05 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:05.630+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:05.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:05 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:05.630+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:05.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:05 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:05.630+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:05.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:05 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:05.630+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:06.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:05.792+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:14:06.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:05.792+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:14:06.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:05.792+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:14:06.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:05.792+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.659350+0000 front 2024-05-09T00:13:40.659419+0000 (oldest deadline 2024-05-09T00:14:02.359354+0000) 2024-05-09T00:14:06.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:05.986+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:06.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:05.986+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:06.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:05.986+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:06.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:05.986+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:06.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:05.986+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:06.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:06 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:06.451+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:06.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:06 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:06.451+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:06.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:06 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:06.451+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:06.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:06 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:06.451+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:06.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:06 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:06.451+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:06.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:06 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:06.648+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:06.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:06 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:06.648+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:06.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:06 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:06.648+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:06.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:06 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:06.648+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:06.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:06 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:06.648+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:07.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:06 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:06.775+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:14:07.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:06 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:06.775+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:14:07.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:06 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:06.775+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:14:07.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:06 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:06.775+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.659350+0000 front 2024-05-09T00:13:40.659419+0000 (oldest deadline 2024-05-09T00:14:02.359354+0000) 2024-05-09T00:14:07.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:06 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:06.973+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:07.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:06 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:06.973+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:07.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:06 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:06.973+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:07.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:06 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:06.973+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:07.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:06 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:06.973+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:07.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:07 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:07.419+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:07.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:07 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:07.419+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:07.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:07 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:07.419+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:07.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:07 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:07.419+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:07.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:07 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:07.419+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:07.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:07 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:07.630+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:07.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:07 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:07.630+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:07.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:07 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:07.630+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:07.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:07 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:07.630+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:07.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:07 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:07.630+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:08.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:07 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:07.779+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:14:08.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:07 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:07.779+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:14:08.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:07 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:07.779+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:14:08.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:07 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:07.779+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.659350+0000 front 2024-05-09T00:13:40.659419+0000 (oldest deadline 2024-05-09T00:14:02.359354+0000) 2024-05-09T00:14:08.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:07 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:07.997+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:08.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:07 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:07.997+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:08.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:07 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:07.997+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:08.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:07 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:07.997+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:08.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:07 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:07.997+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:08.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:08 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:08.447+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:08.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:08 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:08.447+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:08.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:08 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:08.447+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:08.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:08 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:08.447+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:08.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:08 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:08.447+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:08.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:08.581+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:08.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:08.581+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:08.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:08.581+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:08.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:08.581+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:08.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:08 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:08.581+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:09.009 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:08 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:08.747+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.054639+0000 front 2024-05-09T00:13:18.054667+0000 (oldest deadline 2024-05-09T00:13:42.754636+0000) 2024-05-09T00:14:09.009 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:08 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:08.747+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.556482+0000 front 2024-05-09T00:13:25.556507+0000 (oldest deadline 2024-05-09T00:13:49.656654+0000) 2024-05-09T00:14:09.009 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:08 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:08.747+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.157702+0000 front 2024-05-09T00:13:33.157645+0000 (oldest deadline 2024-05-09T00:13:57.857707+0000) 2024-05-09T00:14:09.010 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:08 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8[69733]: 2024-05-09T00:14:08.747+0000 7f2ee8c3f700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.659350+0000 front 2024-05-09T00:13:40.659419+0000 (oldest deadline 2024-05-09T00:14:02.359354+0000) 2024-05-09T00:14:09.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.011+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:09.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.011+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:09.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.011+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:09.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.011+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:09.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.011+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:09.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:09 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:09.465+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:09.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:09 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:09.465+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:09.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:09 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:09.465+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:09.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:09 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:09.465+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:09.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:09 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:09.465+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:09.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:09.601+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:09.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:09.601+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:09.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:09.601+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:09.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:09.601+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:09.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:09 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:09.601+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:10.266 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 09 00:14:10 smithi204 bash[96585]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-8 2024-05-09T00:14:10.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:10.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:10.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:10.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:10.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:10.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:09 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:09.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:10.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:10 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:10.453+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:10.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:10 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:10.453+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:10.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:10 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:10.453+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:10.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:10 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:10.453+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:10.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:10 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:10.453+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:10.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:10 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:10.581+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:10.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:10 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:10.581+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:10.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:10 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:10.581+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:10.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:10 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:10.581+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:10.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:10 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:10.581+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:11.437 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:11 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.032+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:11.437 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:11 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.032+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:11.437 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:11 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.032+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:11.438 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:11 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.032+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:11.438 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:11 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.032+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:11.438 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:11 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.032+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:11.444 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.8.service' 2024-05-09T00:14:11.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:11.484 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T00:14:11.484 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T00:14:11.484 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.9 2024-05-09T00:14:11.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:11 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:11.480+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:11.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:11 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:11.480+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:11.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:11 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:11.480+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:11.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:11 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:11.480+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:11.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:11 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:11.480+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:11.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:11 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:11.480+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:11.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:11 smithi012 systemd[1]: Stopping Ceph osd.9 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:11.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:11.623+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:11.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:11.623+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:11.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:11.623+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:11.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:11.623+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:11.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:11.623+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:11.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:11.623+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.715543+0000 front 2024-05-09T00:13:45.715486+0000 (oldest deadline 2024-05-09T00:14:11.015570+0000) 2024-05-09T00:14:11.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:11.740+0000 7fe1d669e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:11.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:11.740+0000 7fe1d669e700 -1 osd.9 143 *** Got signal Terminated *** 2024-05-09T00:14:11.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:11 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:11.740+0000 7fe1d669e700 -1 osd.9 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:14:12.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.999+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:12.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.999+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:12.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.999+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:12.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.999+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:12.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.999+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:12.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:12 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:11.999+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:12.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:12 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:12.517+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:12.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:12 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:12.517+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:12.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:12 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:12.517+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:12.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:12 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:12.517+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:12.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:12 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:12.517+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:12.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:12 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:12.517+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:12.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:12 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:12.626+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:12.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:12 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:12.626+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:12.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:12 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:12.626+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:12.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:12 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:12.626+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:12.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:12 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:12.626+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:12.918 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:12 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:12.626+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.715543+0000 front 2024-05-09T00:13:45.715486+0000 (oldest deadline 2024-05-09T00:14:11.015570+0000) 2024-05-09T00:14:13.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.007+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:13.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.007+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:13.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.007+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:13.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.007+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:13.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.007+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:13.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.007+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:14.018 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:13 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:13.550+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:14.018 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:13 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:13.550+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:14.019 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:13 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:13.550+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:14.019 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:13 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:13.550+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:14.019 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:13 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:13.550+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:14.019 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:13 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:13.550+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:14.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:13 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:13.675+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:14.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:13 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:13.675+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:14.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:13 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:13.675+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:14.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:13 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:13.675+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:14.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:13 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:13.675+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:14.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:13 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:13.675+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.715543+0000 front 2024-05-09T00:13:45.715486+0000 (oldest deadline 2024-05-09T00:14:11.015570+0000) 2024-05-09T00:14:14.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:14.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:14.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:14.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:14.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:14.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:13 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:13.996+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:14.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:14 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:14.505+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:14.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:14 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:14.505+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:14.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:14 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:14.505+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:14.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:14 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:14.505+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:14.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:14 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:14.505+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:14.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:14 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:14.505+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:15.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:14 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:14.688+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:15.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:14 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:14.688+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:15.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:14 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:14.688+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:15.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:14 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:14.688+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:15.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:14 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:14.688+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:15.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:14 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:14.688+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.715543+0000 front 2024-05-09T00:13:45.715486+0000 (oldest deadline 2024-05-09T00:14:11.015570+0000) 2024-05-09T00:14:15.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:14.954+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:15.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:14.954+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:15.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:14.954+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:15.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:14.954+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:15.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:14.954+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:15.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:14.954+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:15.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:15 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:15.470+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:15.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:15 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:15.470+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:15.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:15 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:15.470+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:15.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:15 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:15.470+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:15.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:15 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:15.470+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:15.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:15 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:15.470+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:16.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:15 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:15.671+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:16.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:15 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:15.671+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:16.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:15 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:15.671+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:16.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:15 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:15.671+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:16.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:15 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:15.671+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:16.168 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:15 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:15.671+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.715543+0000 front 2024-05-09T00:13:45.715486+0000 (oldest deadline 2024-05-09T00:14:11.015570+0000) 2024-05-09T00:14:16.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:15 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:15.962+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:16.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:15 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:15.962+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:16.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:15 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:15.962+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:16.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:15 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:15.962+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:16.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:15 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:15.962+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:16.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:15 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:15.962+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:16.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:16 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:16.442+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:16.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:16 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:16.442+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:16.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:16 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:16.442+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:16.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:16 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:16.442+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:16.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:16 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:16.442+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:16.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:16 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:16.442+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:16.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:16 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:16.624+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.008433+0000 front 2024-05-09T00:13:09.008371+0000 (oldest deadline 2024-05-09T00:13:33.708712+0000) 2024-05-09T00:14:16.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:16 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:16.624+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:17.110648+0000 front 2024-05-09T00:13:17.110538+0000 (oldest deadline 2024-05-09T00:13:43.010709+0000) 2024-05-09T00:14:16.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:16 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:16.624+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.912135+0000 front 2024-05-09T00:13:25.912163+0000 (oldest deadline 2024-05-09T00:13:51.812091+0000) 2024-05-09T00:14:16.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:16 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:16.624+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.812543+0000 front 2024-05-09T00:13:31.812572+0000 (oldest deadline 2024-05-09T00:13:54.712682+0000) 2024-05-09T00:14:16.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:16 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:16.624+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.314647+0000 front 2024-05-09T00:13:39.314615+0000 (oldest deadline 2024-05-09T00:14:02.814522+0000) 2024-05-09T00:14:16.919 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:16 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9[86482]: 2024-05-09T00:14:16.624+0000 7fe1d3495700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.715543+0000 front 2024-05-09T00:13:45.715486+0000 (oldest deadline 2024-05-09T00:14:11.015570+0000) 2024-05-09T00:14:17.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:17.005+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:17.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:17.005+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:17.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:17.005+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:17.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:17.005+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:17.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:17.005+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:17.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:17 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:17.005+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:17.668 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 09 00:14:17 smithi012 bash[144716]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-9 2024-05-09T00:14:17.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:17.456+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:17.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:17.456+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:17.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:17.456+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:17.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:17.456+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:17.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:17.456+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:17.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:17 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:17.456+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:18.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:18 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:18.036+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:18.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:18 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:18.036+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:18.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:18 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:18.036+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:18.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:18 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:18.036+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:18.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:18 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:18.036+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:18.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:18 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:18.036+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:18.627 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.9.service' 2024-05-09T00:14:18.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:18.680 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T00:14:18.680 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T00:14:18.680 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.10 2024-05-09T00:14:18.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:18.499+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:18.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:18.499+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:18.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:18.499+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:18.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:18.499+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:18.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:18.499+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:18.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:18.499+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:18.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:18.499+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T00:13:53.527344+0000 front 2024-05-09T00:13:53.527408+0000 (oldest deadline 2024-05-09T00:14:17.627170+0000) 2024-05-09T00:14:18.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 systemd[1]: Stopping Ceph osd.10 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:19.268 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:18.912+0000 7fcb2d42b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:19.268 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:18.912+0000 7fcb2d42b700 -1 osd.10 143 *** Got signal Terminated *** 2024-05-09T00:14:19.268 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:18 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:18.912+0000 7fcb2d42b700 -1 osd.10 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:14:19.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:19.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:19.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:19.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:19.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:19.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:19.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:19.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:19.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:19.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:19.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:19 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:19.047+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:19.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:19.479+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:19.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:19.479+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:19.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:19.479+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:19.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:19.479+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:19.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:19.479+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:19.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:19.479+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:19.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:19 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:19.479+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T00:13:53.527344+0000 front 2024-05-09T00:13:53.527408+0000 (oldest deadline 2024-05-09T00:14:17.627170+0000) 2024-05-09T00:14:20.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:20.004+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:20.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:20.004+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:20.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:20.004+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:20.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:20.004+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:20.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:20.004+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:20.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:20 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:20.004+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:20.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:20 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:20.477+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:20.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:20 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:20.477+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:20.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:20 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:20.477+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:20.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:20 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:20.477+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:20.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:20 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:20.477+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:20.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:20 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:20.477+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:20.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:20 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:20.477+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T00:13:53.527344+0000 front 2024-05-09T00:13:53.527408+0000 (oldest deadline 2024-05-09T00:14:17.627170+0000) 2024-05-09T00:14:21.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:21 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:21.012+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:21.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:21 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:21.012+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:21.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:21 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:21.012+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:21.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:21 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:21.012+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:21.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:21 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:21.012+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:21.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:21 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:21.012+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:21.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:21.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:21.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:21.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:21.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:21.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:21.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:21.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:21.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:21.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:21.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:21.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:21.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:21 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:21.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T00:13:53.527344+0000 front 2024-05-09T00:13:53.527408+0000 (oldest deadline 2024-05-09T00:14:17.627170+0000) 2024-05-09T00:14:22.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:22.053+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:22.529 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:22.053+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:22.529 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:22.053+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:22.529 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:22.053+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:22.529 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:22.053+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:22.529 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:22 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:22.053+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:22.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:22 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:22.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:22.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:22 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:22.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:22.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:22 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:22.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:22.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:22 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:22.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:22.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:22 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:22.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:22.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:22 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:22.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:22.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:22 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:22.516+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T00:13:53.527344+0000 front 2024-05-09T00:13:53.527408+0000 (oldest deadline 2024-05-09T00:14:17.627170+0000) 2024-05-09T00:14:23.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:23 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:23.063+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:23.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:23 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:23.063+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:23.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:23 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:23.063+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:23.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:23 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:23.063+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:23.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:23 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:23.063+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:23.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:23 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:23.063+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:23.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:23 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:23.494+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.220252+0000 front 2024-05-09T00:13:09.220159+0000 (oldest deadline 2024-05-09T00:13:33.920152+0000) 2024-05-09T00:14:23.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:23 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:23.494+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.021077+0000 front 2024-05-09T00:13:18.021116+0000 (oldest deadline 2024-05-09T00:13:39.721305+0000) 2024-05-09T00:14:23.768 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:23 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:23.494+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:25.622267+0000 front 2024-05-09T00:13:25.622323+0000 (oldest deadline 2024-05-09T00:13:49.722278+0000) 2024-05-09T00:14:23.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:23 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:23.494+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:31.423562+0000 front 2024-05-09T00:13:31.423468+0000 (oldest deadline 2024-05-09T00:13:54.923477+0000) 2024-05-09T00:14:23.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:23 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:23.494+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:39.525096+0000 front 2024-05-09T00:13:39.525113+0000 (oldest deadline 2024-05-09T00:14:05.425271+0000) 2024-05-09T00:14:23.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:23 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:23.494+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:45.426010+0000 front 2024-05-09T00:13:45.425905+0000 (oldest deadline 2024-05-09T00:14:10.725925+0000) 2024-05-09T00:14:23.769 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:23 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10[75860]: 2024-05-09T00:14:23.494+0000 7fcb2a222700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-09T00:13:53.527344+0000 front 2024-05-09T00:13:53.527408+0000 (oldest deadline 2024-05-09T00:14:17.627170+0000) 2024-05-09T00:14:24.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:24 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:24.042+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:24.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:24 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:24.042+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:24.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:24 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:24.042+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:24.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:24 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:24.042+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:24.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:24 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:24.042+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:24.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:24 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:24.042+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:24.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:24 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:24.042+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T00:14:02.046648+0000 front 2024-05-09T00:14:02.046631+0000 (oldest deadline 2024-05-09T00:14:23.746539+0000) 2024-05-09T00:14:25.018 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 00:14:24 smithi160 bash[93480]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-10 2024-05-09T00:14:25.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:25 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:25.044+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:25.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:25 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:25.044+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:25.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:25 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:25.044+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:25.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:25 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:25.044+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:25.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:25 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:25.044+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:25.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:25 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:25.044+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:25.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:25 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:25.044+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T00:14:02.046648+0000 front 2024-05-09T00:14:02.046631+0000 (oldest deadline 2024-05-09T00:14:23.746539+0000) 2024-05-09T00:14:25.830 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.10.service' 2024-05-09T00:14:25.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:25.872 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T00:14:25.873 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T00:14:25.873 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.11 2024-05-09T00:14:26.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:25 smithi204 systemd[1]: Stopping Ceph osd.11 for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:26.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:26 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:26.072+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:26.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:26 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:26.072+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:26.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:26 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:26.072+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:26.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:26 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:26.072+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:26.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:26 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:26.072+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:26.266 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:26 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:26.072+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:26.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:26 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:26.072+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T00:14:02.046648+0000 front 2024-05-09T00:14:02.046631+0000 (oldest deadline 2024-05-09T00:14:23.746539+0000) 2024-05-09T00:14:26.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:26 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:26.122+0000 7f2309a72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:26.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:26 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:26.122+0000 7f2309a72700 -1 osd.11 143 *** Got signal Terminated *** 2024-05-09T00:14:26.267 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:26 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:26.122+0000 7f2309a72700 -1 osd.11 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:14:27.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:27.122+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:27.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:27.122+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:27.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:27.122+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:27.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:27.122+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:27.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:27.122+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:27.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:27.122+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:27.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:27 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:27.122+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T00:14:02.046648+0000 front 2024-05-09T00:14:02.046631+0000 (oldest deadline 2024-05-09T00:14:23.746539+0000) 2024-05-09T00:14:28.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:28 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:28.096+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:28.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:28 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:28.096+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:28.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:28 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:28.096+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:28.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:28 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:28.096+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:28.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:28 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:28.096+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:28.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:28 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:28.096+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:28.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:28 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:28.096+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T00:14:02.046648+0000 front 2024-05-09T00:14:02.046631+0000 (oldest deadline 2024-05-09T00:14:23.746539+0000) 2024-05-09T00:14:29.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:29.126+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:29.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:29.126+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:29.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:29.126+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:29.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:29.126+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:29.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:29.126+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:29.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:29.126+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:29.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:29 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:29.126+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T00:14:02.046648+0000 front 2024-05-09T00:14:02.046631+0000 (oldest deadline 2024-05-09T00:14:23.746539+0000) 2024-05-09T00:14:30.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:30 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:30.132+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:30.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:30 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:30.132+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:30.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:30 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:30.132+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:30.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:30 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:30.132+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:30.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:30 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:30.132+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:30.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:30 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:30.132+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:30.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:30 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:30.132+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T00:14:02.046648+0000 front 2024-05-09T00:14:02.046631+0000 (oldest deadline 2024-05-09T00:14:23.746539+0000) 2024-05-09T00:14:31.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:31.118+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-09T00:13:09.335730+0000 front 2024-05-09T00:13:09.335717+0000 (oldest deadline 2024-05-09T00:13:34.035723+0000) 2024-05-09T00:14:31.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:31.118+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T00:13:18.037709+0000 front 2024-05-09T00:13:18.037599+0000 (oldest deadline 2024-05-09T00:13:43.337746+0000) 2024-05-09T00:14:31.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:31.118+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-09T00:13:23.338178+0000 front 2024-05-09T00:13:23.338202+0000 (oldest deadline 2024-05-09T00:13:48.638351+0000) 2024-05-09T00:14:31.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:31.118+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-09T00:13:33.339416+0000 front 2024-05-09T00:13:33.339483+0000 (oldest deadline 2024-05-09T00:13:54.940382+0000) 2024-05-09T00:14:31.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:31.118+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T00:13:40.642586+0000 front 2024-05-09T00:13:40.642677+0000 (oldest deadline 2024-05-09T00:14:02.942722+0000) 2024-05-09T00:14:31.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:31.118+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-09T00:13:42.943096+0000 front 2024-05-09T00:13:42.943109+0000 (oldest deadline 2024-05-09T00:14:09.343751+0000) 2024-05-09T00:14:31.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:31.118+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T00:14:02.046648+0000 front 2024-05-09T00:14:02.046631+0000 (oldest deadline 2024-05-09T00:14:23.746539+0000) 2024-05-09T00:14:31.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:31 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11[78527]: 2024-05-09T00:14:31.118+0000 7f2302861700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-05-09T00:14:07.847339+0000 front 2024-05-09T00:14:07.847368+0000 (oldest deadline 2024-05-09T00:14:30.747390+0000) 2024-05-09T00:14:32.016 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 09 00:14:31 smithi204 bash[96839]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-osd-11 2024-05-09T00:14:32.926 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@osd.11.service' 2024-05-09T00:14:32.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:32.967 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T00:14:32.967 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T00:14:32.967 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.a 2024-05-09T00:14:33.418 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 00:14:33 smithi012 systemd[1]: Stopping Ceph mds.a for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:33.418 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 00:14:33 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-a[94406]: 2024-05-09T00:14:33.228+0000 7fa79e535700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:33.419 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 00:14:33 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-a[94406]: 2024-05-09T00:14:33.228+0000 7fa79e535700 -1 mds.a *** got signal Terminated *** 2024-05-09T00:14:37.294 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 09 00:14:37 smithi012 bash[144985]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-a 2024-05-09T00:14:37.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.a.service' 2024-05-09T00:14:37.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:37.341 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T00:14:37.341 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T00:14:37.342 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.d 2024-05-09T00:14:37.633 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 00:14:37 smithi012 systemd[1]: Stopping Ceph mds.d for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:37.918 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 00:14:37 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-d[93495]: 2024-05-09T00:14:37.634+0000 7fc08d463700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:37.918 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 00:14:37 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-d[93495]: 2024-05-09T00:14:37.634+0000 7fc08d463700 -1 mds.d *** got signal Terminated *** 2024-05-09T00:14:41.494 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 09 00:14:41 smithi012 bash[145124]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-d 2024-05-09T00:14:41.500 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.d.service' 2024-05-09T00:14:41.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:41.539 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T00:14:41.539 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T00:14:41.540 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.g 2024-05-09T00:14:41.857 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 00:14:41 smithi012 systemd[1]: Stopping Ceph mds.g for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:42.168 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 00:14:41 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-g[94018]: 2024-05-09T00:14:41.858+0000 7f729013f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:42.168 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 00:14:41 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-g[94018]: 2024-05-09T00:14:41.858+0000 7f729013f700 -1 mds.g *** got signal Terminated *** 2024-05-09T00:14:43.324 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 09 00:14:42 smithi012 bash[145261]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-g 2024-05-09T00:14:43.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.g.service' 2024-05-09T00:14:43.371 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:43.371 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T00:14:43.371 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T00:14:43.371 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.j 2024-05-09T00:14:43.648 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 00:14:43 smithi012 systemd[1]: Stopping Ceph mds.j for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:43.918 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 00:14:43 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-j[93107]: 2024-05-09T00:14:43.649+0000 7f90ba752700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:43.918 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 00:14:43 smithi012 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-j[93107]: 2024-05-09T00:14:43.649+0000 7f90ba752700 -1 mds.j *** got signal Terminated *** 2024-05-09T00:14:46.382 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 09 00:14:46 smithi012 bash[145401]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-j 2024-05-09T00:14:46.388 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.j.service' 2024-05-09T00:14:46.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:46.427 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T00:14:46.427 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T00:14:46.427 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.b 2024-05-09T00:14:46.768 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 00:14:46 smithi160 systemd[1]: Stopping Ceph mds.b for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:47.268 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 00:14:46 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-b[82741]: 2024-05-09T00:14:46.794+0000 7ff0c0d1f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:47.269 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 00:14:46 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-b[82741]: 2024-05-09T00:14:46.794+0000 7ff0c0d1f700 -1 mds.b *** got signal Terminated *** 2024-05-09T00:14:49.970 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 00:14:49 smithi160 bash[93736]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-b 2024-05-09T00:14:49.976 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.b.service' 2024-05-09T00:14:50.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:50.012 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T00:14:50.012 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T00:14:50.012 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.e 2024-05-09T00:14:50.268 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 00:14:50 smithi160 systemd[1]: Stopping Ceph mds.e for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:50.768 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 00:14:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-e[82051]: 2024-05-09T00:14:50.272+0000 7f06680cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:50.768 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 00:14:50 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-e[82051]: 2024-05-09T00:14:50.272+0000 7f06680cc700 -1 mds.e *** got signal Terminated *** 2024-05-09T00:14:54.019 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 00:14:53 smithi160 bash[93874]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-e 2024-05-09T00:14:54.043 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.e.service' 2024-05-09T00:14:54.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:54.079 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T00:14:54.079 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T00:14:54.079 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.h 2024-05-09T00:14:54.346 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 00:14:54 smithi160 systemd[1]: Stopping Ceph mds.h for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:54.768 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 00:14:54 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-h[82391]: 2024-05-09T00:14:54.347+0000 7f858f8ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:54.768 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 00:14:54 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-h[82391]: 2024-05-09T00:14:54.347+0000 7f858f8ec700 -1 mds.h *** got signal Terminated *** 2024-05-09T00:14:57.932 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 00:14:57 smithi160 bash[94020]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-h 2024-05-09T00:14:57.938 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.h.service' 2024-05-09T00:14:57.972 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:14:57.972 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T00:14:57.972 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T00:14:57.972 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.k 2024-05-09T00:14:58.245 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:14:58 smithi160 systemd[1]: Stopping Ceph mds.k for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:14:58.518 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:14:58 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: 2024-05-09T00:14:58.246+0000 7fa96c2ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:14:58.519 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:14:58 smithi160 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k[81709]: 2024-05-09T00:14:58.246+0000 7fa96c2ca700 -1 mds.k *** got signal Terminated *** 2024-05-09T00:15:02.749 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 00:15:02 smithi160 bash[94159]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-k 2024-05-09T00:15:02.755 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.k.service' 2024-05-09T00:15:02.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:15:02.791 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T00:15:02.791 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T00:15:02.791 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.c 2024-05-09T00:15:03.266 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:15:02 smithi204 systemd[1]: Stopping Ceph mds.c for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:15:03.267 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:15:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: 2024-05-09T00:15:03.043+0000 7f3163533700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:15:03.267 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:15:03 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c[87999]: 2024-05-09T00:15:03.043+0000 7f3163533700 -1 mds.c *** got signal Terminated *** 2024-05-09T00:15:04.834 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 09 00:15:04 smithi204 bash[97106]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-c 2024-05-09T00:15:04.840 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.c.service' 2024-05-09T00:15:04.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:15:04.878 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T00:15:04.878 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T00:15:04.879 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.f 2024-05-09T00:15:05.150 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 09 00:15:04 smithi204 systemd[1]: Stopping Ceph mds.f for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:15:05.516 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 09 00:15:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-f[84273]: 2024-05-09T00:15:05.151+0000 7ff015299700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:15:05.516 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 09 00:15:05 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-f[84273]: 2024-05-09T00:15:05.151+0000 7ff015299700 -1 mds.f *** got signal Terminated *** 2024-05-09T00:15:09.689 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 09 00:15:09 smithi204 bash[97244]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-f 2024-05-09T00:15:09.694 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.f.service' 2024-05-09T00:15:09.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:15:09.729 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T00:15:09.729 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T00:15:09.729 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.i 2024-05-09T00:15:09.998 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 09 00:15:09 smithi204 systemd[1]: Stopping Ceph mds.i for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:15:10.266 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 09 00:15:10 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-i[84958]: 2024-05-09T00:15:10.000+0000 7fe60b2b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:15:10.266 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 09 00:15:10 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-i[84958]: 2024-05-09T00:15:10.000+0000 7fe60b2b8700 -1 mds.i *** got signal Terminated *** 2024-05-09T00:15:13.926 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 09 00:15:13 smithi204 bash[97384]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-i 2024-05-09T00:15:13.932 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.i.service' 2024-05-09T00:15:13.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:15:13.965 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T00:15:13.966 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T00:15:13.966 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.l 2024-05-09T00:15:14.231 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:15:14 smithi204 systemd[1]: Stopping Ceph mds.l for 00f8fb28-0d94-11ef-bc97-c7b262605968... 2024-05-09T00:15:14.516 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:15:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: 2024-05-09T00:15:14.232+0000 7f21ae6fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:15:14.516 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:15:14 smithi204 ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l[83928]: 2024-05-09T00:15:14.232+0000 7f21ae6fb700 -1 mds.l *** got signal Terminated *** 2024-05-09T00:15:17.967 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 09 00:15:17 smithi204 bash[97522]: ceph-00f8fb28-0d94-11ef-bc97-c7b262605968-mds-l 2024-05-09T00:15:17.973 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-00f8fb28-0d94-11ef-bc97-c7b262605968@mds.l.service' 2024-05-09T00:15:18.006 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:15:18.007 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T00:15:18.007 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T00:15:18.229 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-09T00:15:19.796 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T00:15:20.003 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-09T00:15:21.489 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T00:15:21.675 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-09T00:15:23.186 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:15:23.227 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:15:23.264 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:15:23.294 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T00:15:23.297 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181/remote/smithi012/crash 2024-05-09T00:15:23.298 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/crash -- . 2024-05-09T00:15:23.353 INFO:teuthology.orchestra.run.smithi012.stderr:tar: /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T00:15:23.353 INFO:teuthology.orchestra.run.smithi012.stderr:tar: Error is not recoverable: exiting now 2024-05-09T00:15:23.355 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181/remote/smithi160/crash 2024-05-09T00:15:23.356 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/crash -- . 2024-05-09T00:15:23.410 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T00:15:23.410 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now 2024-05-09T00:15:23.412 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181/remote/smithi204/crash 2024-05-09T00:15:23.413 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/crash -- . 2024-05-09T00:15:23.457 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T00:15:23.457 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-05-09T00:15:23.458 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T00:15:23.459 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T00:15:23.501 INFO:teuthology.orchestra.run.smithi012.stderr:grep: /var/log/ceph/00f8fb28-0d94-11ef-bc97-c7b262605968/ceph.log: No such file or directory 2024-05-09T00:15:23.503 INFO:tasks.cephadm:Compressing logs... 2024-05-09T00:15:23.504 DEBUG:teuthology.orchestra.run.smithi012:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T00:15:23.546 DEBUG:teuthology.orchestra.run.smithi160:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T00:15:23.549 DEBUG:teuthology.orchestra.run.smithi204:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T00:15:23.578 INFO:teuthology.orchestra.run.smithi012.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T00:15:23.598 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T00:15:23.620 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T00:29:36.335 INFO:tasks.cephadm:Archiving logs... 2024-05-09T00:29:36.356 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181/remote/smithi012/log 2024-05-09T00:29:36.357 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T00:29:49.079 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181/remote/smithi160/log 2024-05-09T00:29:49.080 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T00:31:35.857 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181/remote/smithi204/log 2024-05-09T00:31:36.080 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T00:32:37.012 INFO:tasks.cephadm:Removing cluster... 2024-05-09T00:32:37.012 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 --force 2024-05-09T00:32:37.215 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-09T00:32:37.588 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 --force 2024-05-09T00:32:38.194 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-09T00:32:39.006 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 00f8fb28-0d94-11ef-bc97-c7b262605968 --force 2024-05-09T00:32:39.497 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 00f8fb28-0d94-11ef-bc97-c7b262605968 2024-05-09T00:32:40.164 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T00:32:40.164 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T00:32:40.188 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T00:32:40.506 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T00:32:40.544 INFO:tasks.cephadm:Teardown complete 2024-05-09T00:32:40.545 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T00:32:41.336 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T00:32:41.337 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T00:32:41.341 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T00:32:41.345 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T00:32:41.516 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T00:32:41.605 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T00:32:41.605 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T00:32:41.610 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T00:32:41.614 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T00:32:41.629 INFO:teuthology.orchestra.run.smithi012.stderr:bash: ntpq: command not found 2024-05-09T00:32:41.632 INFO:teuthology.orchestra.run.smithi160.stderr:bash: ntpq: command not found 2024-05-09T00:32:41.637 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T00:32:41.637 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-05-09T00:32:41.637 INFO:teuthology.orchestra.run.smithi012.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +867us[ +859us] +/- 52ms 2024-05-09T00:32:41.637 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -850us[ -850us] +/- 59ms 2024-05-09T00:32:41.637 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -99us[ -99us] +/- 75ms 2024-05-09T00:32:41.637 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:32:41.638 INFO:teuthology.orchestra.run.smithi204.stderr:bash: ntpq: command not found 2024-05-09T00:32:41.678 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T00:32:41.678 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-09T00:32:41.678 INFO:teuthology.orchestra.run.smithi160.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 +795us[ +788us] +/- 52ms 2024-05-09T00:32:41.678 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 29 -924us[ -931us] +/- 59ms 2024-05-09T00:32:41.678 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 25 -127us[ -134us] +/- 75ms 2024-05-09T00:32:41.678 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:32:41.691 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T00:32:41.691 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-05-09T00:32:41.692 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +873us[ +873us] +/- 52ms 2024-05-09T00:32:41.692 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -869us[ -870us] +/- 59ms 2024-05-09T00:32:41.692 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -111us[ -111us] +/- 75ms 2024-05-09T00:32:41.692 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:32:41.692 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T00:32:41.711 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T00:32:41.712 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T00:32:41.725 DEBUG:teuthology.orchestra.run.smithi012:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T00:32:41.823 DEBUG:teuthology.orchestra.run.smithi160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T00:32:41.964 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T00:32:42.081 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-09T00:32:42.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:32:42.123 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-09T00:32:42.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:32:42.158 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-09T00:32:42.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:32:42.192 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T00:32:42.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T00:32:42.216 INFO:teuthology.task.internal:Duration was 4176.923339 seconds 2024-05-09T00:32:42.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T00:32:42.227 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T00:32:42.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T00:32:42.230 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T00:32:42.233 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T00:32:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T00:32:42.336 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T00:32:42.348 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T00:32:42.836 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T00:32:42.836 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-05-09T00:32:42.836 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T00:32:42.865 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T00:32:42.865 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T00:32:42.909 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-05-09T00:32:42.909 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T00:32:42.950 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T00:32:42.950 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T00:32:42.953 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T00:32:42.955 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T00:32:43.068 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T00:32:43.068 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T00:32:43.071 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T00:32:43.074 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T00:32:43.210 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T00:32:43.224 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T00:32:43.225 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T00:32:43.256 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T00:32:43.312 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T00:32:43.381 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T00:32:43.394 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:32:43.397 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:32:43.399 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:32:43.430 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-05-09T00:32:43.441 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2024-05-09T00:32:43.443 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-05-09T00:32:43.486 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:32:43.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:32:43.509 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:32:43.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:32:43.526 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:32:43.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:32:43.542 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T00:32:43.554 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T00:32:43.555 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181/remote/smithi012 2024-05-09T00:32:43.555 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T00:32:43.609 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181/remote/smithi160 2024-05-09T00:32:43.609 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T00:32:43.651 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698181/remote/smithi204 2024-05-09T00:32:43.652 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T00:32:43.693 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T00:32:43.694 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T00:32:43.696 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T00:32:43.699 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T00:32:43.745 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T00:32:43.757 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T00:32:43.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T00:32:43.769 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T00:32:43.769 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T00:32:43.772 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T00:32:43.774 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T00:32:43.788 INFO:teuthology.orchestra.run.smithi160.stdout: 262164 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 00:32 /home/ubuntu/cephtest 2024-05-09T00:32:43.791 INFO:teuthology.orchestra.run.smithi012.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 00:32 /home/ubuntu/cephtest 2024-05-09T00:32:43.800 INFO:teuthology.orchestra.run.smithi204.stdout: 262164 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 00:32 /home/ubuntu/cephtest 2024-05-09T00:32:43.802 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T00:32:43.814 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T00:32:43.844 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 4176.923338651657 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-09T00:32:43.844 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T00:32:43.928 INFO:teuthology.run:pass